#touch interfaces

Metaio reveals thermal imaging R&D as possible I/O interface for headsets at Augmented World Expo 2014

The Augmented World Expo is one of the major gathering places for companies and researchers working in augmented reality. Held in late May in Santa Clara, the conference is expanding to encompass VR headsets, wearables, the Internet of Things and the focus is to bring IT technology out into the world where it can be put to work or used for fun. Metaio, based in Germany, is one of the leaders in this evolving business. At this year’s show the company teamed with VR headset makers such as Epson, Vuzix, Oculus, and Meta,  to create practical apps for industry. And, as a sort of grand finale, the company’s head of marketing,  Trak Lord  demonstrated some experimental technology being developed at Metaio. The company is using thermal imaging in conjunction with augmented reality to create an invisible I/O. Metaio CEO Thomas Alt says the technology turns the world into a touch screen.

Metaio’s Thermal Touch interface combines an infrared camera with a standard camera on a device. Right now it’s a scary looking Rube Goldberg contraption attached to a tablet, but Alt and Lord say that as mobile devices gain new sensors the addition of an infrared cameras as a standard feature becomes a likely possibility. The Metaio approach proposes glasses equipped with the two cameras, which then solves one of the drawbacks of current glasses systems — there is no easy I/O.

Metaio proposes a Thermal Touch input system. (Source Metaio YouTube video) Metaio’s idea is to have the two cameras work in tandem on a tablet PC. The system can recognize and register the heat signature left by a person’s finger as they touch a surface or object. This allows the person to interact with an object. For instance the Metaio video shows people playing chess or a person interacting with a plastic model. The technology could be a much more convenient replacement for QR codes in magazines and on signs. Metaio sees an immediate application for Thermal Touch as an enabling technology for VR and AR glasses.

Immersion Corporation (Nasdaq:IMMR), the leading developer and licensor of touch feedback technology, Jan 6th announced the launch of a new website offering technical and integration recommendations, user experience guidelines, and research that evaluates the impact of haptics in automotive environments. The site offers auto OEMs and system integrators a comprehensive resource to learn about advancements in the haptic ecosystem and user research.

"Touch screens and touch surfaces are becoming standard interfaces for autos, increasingly managing dynamic content and replacing mechanical buttons," explains Dennis Sheehan, senior vice president of sales and marketing at Immersion. "As demand grows for touch interfaces, Immersion's expertise in haptics design, engineering and UX is essential. By offering guidelines and resources, we are working to help create multi-modal experiences that are capable of being intuitive, enjoyable and safe."

Haptics are being used in mobile, automotive and consumer applications to provide critical touch confirmation, creating confident consumer experiences. In autos specifically, haptics can provide numerous unique safety and design benefits including:

Promoting safety by reducing secondary glance rates and overall task times

Reducing visual distraction, frustration and confusion

Instilling confidence in drivers by delivering a positive user experience

Providing automakers with interior design flexibility

Offering opportunity to create distinctive experiences that extend the automotive brand into the touch interface

With the broad adoption of haptic technology in several consumer markets, the haptic ecosystem has matured, resulting in lower cost, high-quality components. Advancements in design and integration may be referenced for creating more efficient and streamlined systems. Immersion has developed guidelines and technical recommendations for designers that incorporate best practices for haptic implementations based on research conducted by the company and independent parties.

"Our substantial IP portfolio, combined with expertise in user experience, haptic effect authoring, specialized firmware and deep domain knowledge on integrating the broader ecosystem makes Immersion a key partner for auto OEMS and integrators," explains Daniel Brongiel, vice president and head of automotive market at Immersion. "As the industry embraces our technology, we're working with ecosystem leaders to improve safety and create compelling consumer experiences through the use of haptics."

The goal is to transform all of your surroundings into touchscreens, equipping walls, tables, and couches with interactive, intuitive controls.

When it comes to technological innovation, there are two basic approaches. You can start big, flashy, and expensive, and hope that eventually your tech invention comes down enough in price for an average user to afford – think of GPS devices, for instance, which were the realm of high-budget military agencies long before ordinary civilians could dream of buying one; or, you can set out from the beginning to design something life-changing that everyone can have access to, rather than just an elite few.

The research team behind WorldKit, a new, experimental technology system, is trying to straddle the gulf between these two extremes. The goal is to transform all of your surroundings into touchscreens, equipping walls, tables, and couches with interactive, intuitive controls. But the team wants to do so without installing oversized iPads into every surface in your home, which could easily run up a six-figure price tag.

So how does the magic happen? With a simple projector – a projector paired with a depth sensor, to be precise. “It’s this interesting space of having projected interfaces on the environment, using your whole world as a sort of gigantic tablet,” said Chris Harrison, a soon-to-be professor in human-computer interaction at Carnegie Mellon University. Robert Xiao, a PhD candidate at Carnegie Mellon and lead researcher on the project, explained that WorldKit uses a depth camera to sense where flat surfaces are in your environment. “We allow a user to basically select a surface on which they can ‘paint’ an interactive object, like a button or sensor,” Xiao said.

We recently chatted with both Harrison and Xiao about their work on the WorldKit project, and learned just how far their imaginations run when it comes to the future of touch technology and ubiquitous computing. Below, we talk about merging the digital and the physical worlds, as well as creative applications for WorldKit that involve really thinking outside the box (or outside the monitor, in this case).

Understanding WorldKit’s workings

We know; the concept of a touchscreen on any surface is a little far out there, so let’s break it down. WorldKit works by pairing a depth-sensing camera lens, such as the one that the Kinect uses, with a projector lens. Then, programmers write short scripts on a MacBook Pro using Java, similar to those they might write for an Arduino, to tell the depth camera how to react when someone makes certain gestures in front of it. The depth camera interprets the gestures and then tells the projector to react by projecting certain interfaces. For instance, if someone makes a circular gesture, the system can interpret that by projecting a dial where the gesture was made. Then, when someone “adjusts” the dial by gesturing in front of it, the system can adjust a volume control elsewhere.

The brilliance – and the potential frustration – of this system lies in its nearly endless possibilities. Currently, whatever you want WorldKit to do, you must program it to do yourself. Xiao and Harrison expressed hope that one day, once WorldKit reaches the consumer realm, there might be an online forum where people can upload and download programming scripts (much like apps) in order to make their WorldKit system perform certain tasks. However, at the moment, WorldKit remains in an R&D phase in the academic realm, allowing its creators to dream big about what they would like to make it do eventually.

In any case, the easiest way to understand how WorldKit works is to watch a demo video of it in action. In the video, researchers touch various surfaces to “paint” them with light from the projector. Afterward, the WorldKit system uses the selected area to display a chosen interface, such as a menu bar or a sliding lighting-control dial, which can then be manipulated through touch gestures.

Currently, WorldKit’s depth sensor is nothing other than a Kinect – the same one that shipped with the Xbox 360 – that connects to a projector that’s mounted to a ceiling or tripod. While this combo is already sensitive enough to track individual fingers and multi-directional gestures down to the centimeter, it does have one major drawback: size. “Certainly the system as it is right now is kind of big, and we all admit that,” Xiao said.

Lights, user, action: Putting WorldKit to use

But the team has high hopes for the technology on the near horizon. “We’re already seeing cell phones on the market that have projectors built in,” Xiao said. “Maybe the back camera, one day, is a depth sensor  … You could have WorldKit on your phone.” Harrison added that WorldKit could allow users to take full advantage of their phones for the first time. “A lot of smartphones you have nowadays are easily powerful enough to be a laptop, they just don’t have screens big enough to do it,” Harrison said. “So with WorldKit, you could have one of these phones be your laptop, and it would just project your desktop onto your actual desk.”

With projection, you can do some very clever things that basically alter the world in terms of aesthetics.

If Harrison and Xiao can imagine the mobile version of WorldKit on a smartphone in five years’ time, they have an even crazier vision for 10 or 15 years down the line. “We could actually put the entire WorldKit setup into something about the size of a lightbulb,” Xiao said. For these researchers, a lightbulb packed full of WorldKit potential has truly revolutionary implications. “We’re looking at that as almost as big as the lighting revolution of the early 1800s,” Xiao added.

The possibilities for WorldKit, as you might imagine, are limitless. So far, Harrison and Xiao’s ideas have included an away-from-office status button – the virtual version of a post-it note – and a set of digital TV controls. “You won’t ever have to find your remote again,” Xiao said.

The team’s already envisioning much more ambitious applications, such as experimental interior design. According to Harrison, you could make your own wallpaper, or change the look of your couch. “With projection, you can do some very clever things that basically alter the world in terms of aesthetics,” Harrison said. “Instead of mood lighting, you could have mood interaction.”

The miniature version of WorldKit, shown here, uses a tiny depth camera called the CamBoard Nano by PMD.

The CamBoard Nano depth camera pairs with a PicoP projector by Microvision.

Xiao, meanwhile, fantasized about the system’s gaming potential. “You could augment the floor so that you didn’t want to step on it, and then play a lava game,” he said, describing a game where you have to cross from one end of the floor to the other, using only the tables and chairs. “You can imagine this being a very exciting gaming platform if you want to do something physical, instead of just using a controller.”

Blurring the boundaries between digital and physical

Xiao has good reason to be enthusiastic. He believes WorldKit gets at the heart at one of the biggest goals of computing research. “Eventually we’d like to see computers sort of fade into the background, and just become the way you do things,” he said. “Right now, it’s very explicit whenever you’re operating a computer that you are interacting with a computer.”

Indeed, part of what makes WorldKit so exciting is that it incorporates real, physical materials into its virtual play. But Harrison is  more hesitant to claim that this is always a good thing, especially when it comes to broad, philosophical questions about aesthetics. “In art, there’s a lot that’s nice about having it be rich, and physical, and also enduring,” Harrison argued, talking about digitally “painting” a surface using WorldKit. “So when you go over to the digital domain, are we using some of the things that make art a fundamental part of the human experience? Or are we losing something?”

Gesture control, devices that recognize different people, and tricks to make a screen feel as if it has physical buttons could be coming to your gadgets.

Finger moves: A Microsoft research project, called Digits, makes gestural commands mobile.  In a few short years, the technologies found in today's mobile devices—touch screens, gyroscopes, and voice-control software, to name a few—have radically transformed how we access computers. To glimpse what new ideas might have a similar impact in the next few years, you need only to have walked into the Marriott Hotel in Cambridge, Massachusetts, this week. There, researchers from around the world demonstrated new ideas for computer interaction at the ACM Symposium on User Interface Software and Technology . Many were focused on taking mobile devices in directions that today feel strange and new but could before long be as normal as swiping the screen of an iPhone or Android device.

"We see new hardware, like devices activated by tongue movement or muscle-flexing, or prototypes that build on technology we already have in our hands, like Kinect, Wii, or the sensors built into existing phones," said Rob Miller , a professor at MIT's Computer Science and Artificial Intelligence Lab (CSAIL) and the chair of the conference.

One of the most eye-catching, and potentially promising, ideas that was on show makes it possible to perform complex tasks with a flick of the wrist or a snap of the fingers.

The interface, called Digits, created by David Kim , a U.K. researcher at both Microsoft Research and Newcastle University, is worn around the wrist and consists of a motion sensor and an infrared light source and camera. Like a portable version of Microsoft's motion-sensing device for the Xbox Kinect, Digits can follow arm and finger movements with enough accuracy to replicate them on screen or allow control of a complex computer game. "We envision a smaller device that could be worn like a watch that allows users to communicate with their surroundings and personal computing devices with simple hand gestures," said Kim (watch a video of Digits in action ).

Projects like Kim's could be a glimpse into the future of mobile computing. After all, prior to the iPhone's launch, multi-touch interfaces were found only at this kind of event. Researchers believe that mobile computers are still being held back by the limitations of existing control methods, without which they could become even more powerful.

"We have an increasing desire and need to access and work with our computing devices anywhere and everywhere we are," Kim said. "Productive input and interaction on mobile devices is, however, still challenging due to the trade-offs we have to make regarding a device's form factor and input capacity."

The advance of mobile technology has also given researchers easy ways to experiment. Several groups at the conference showed off modifications of existing mobile interfaces designed to give them new capabilities.

Hong Tan , a professor at Purdue University currently working at Microsoft Research Asia, demonstrated a way to add the feel of buttons and other physical controls to a touch screen: vibrating piezoelectric actuators installed on the side of a normal screen generate friction at the point of contact with a finger. The design, dubbed SlickFeel, can make an ordinary sheet of glass feel as if it has physical buttons or even a physical slider with varying levels of resistance. Such haptic feedback could help users find the right control on compact devices like smartphones, or enable the use of a touch screen without looking at it, for example while driving.

Who's that? A touch screen that recognizes different people's fingers, developed by Chris Harrison and colleagues at Disney Research. Credit: Chris Harrison

In another effort to make more of the touch screen, Chris Harrison  of Disney Research presented a way for devices to recognize the swipes and presses of particular people. His interface, a capacitive touch screen with a resistance sensor attached, identifies the unique "impedance profile" of a person's body through his or her fingers. Users need to hold a finger to the device for few seconds the first time they use it, after which subsequent presses are attributed to them. That could allow apps to do things like track modifications to a document made by different people as a tablet is handed around a table (see a video of the screen ). "It's similar to the technology that is already in smartphones," said Harrison. "There are lots of implications for gaming—no more split screens—and for collaborative applications."

The motion and touch sensors in current phones were another target for experimentation.Mayank Goel , a PhD student the University of Washington, and colleagues, modified the software on an Android phone to automatically determine in which hand a person is holding it. The software figures this out by monitoring the angle at which the device is tilted, as revealed by its motion sensor, and the precise shape of pressure on its touch screen. Goel says this can allow a keyboard to automatically adjust to whether a person is using the left or right hand, an adjustment that cut typos by 30 percent in his experiments.

Touchy feely: A malleable interface made by Sean Follmer and colleagues at MIT's Media Lab.

Credit: Sean Follmer

Other prototypes on display were less obviously connected with the gadgets in your pocket today. One was a malleable interface that can be shaped somewhat the way clay can, developed by a team at MIT's Media Lab. Sean Follmer , a PhD student in the lab of ProfessorHiroshi Ishii , demonstrated several versions, including a translucent bendable touch screen laid flat on a table. This was made from a plastic material containing glass beads and oil, with a projector and a 3-D sensor positioned below. Pinches and twists made to the pliable screen changed the colors displayed on it, which were also shown on a 3-D model of the material on a computer screen nearby.

Principles to guide multi touch and gesture user interaction design: Coping with anarchy 

Touch-enabled devices are proliferating at an astonishing rate. Touch, multi touch and gesture user interfaces are here to stay, but the wide variation between platforms – and even between applications on one platform – in what each touch action means, will lead to increasing user frustration. Applying some simple principles can help designers avoid the worst mistakes in touch interaction design.

The growth of touch interfaces

Touch interfaces are rapidly becoming ubiquitous but the user experience is suffering from severe growing pains. Browsing the 3000+ exhibitor booths at the recent Consumer Electronics Show in Las Vegas one could not escape being excited by the proliferation of new multi touch and gesture interactions available on phones, tablets, GPS, car entertainment systems, kiosks, TVs, radios, etc. The troubling part is the glaring lack of standards and the numerous violations of long-standing usability principles. What does this mean to you? Well, if you are involved in any type of web or web application design touch is going to impact your world – real soon, if not already!

With over 1 million new touch-enabled devices every day, you cannot ignore touch and gesture

Touch-enabled devices are proliferating at a mind-boggling rate. At the end of 2011, over one million smartphones were being activated every day – three times the number of babies born in the world every day. Add to this the 66.9 million tablets sold last year and you can’t escape the fact that a large proportion of your customers will soon be accessing your content and applications using touch. Even laptops and desktops increasingly support gestures on touch pads and touch screens.

The problems of lack of standardization for touch interaction design

Designing for touch is very different than designing for mouse interaction. You can’t position your cursor and then decide on the action. Touching the screen not only indicates the point of interest on the screen but also invokes an action. Most often the finger obscures the object with which you are trying to interact. Feedback becomes absolutely critical in touch interfaces to know what and when something has been activated. Often this involves multiple modalities – visual, haptic (vibration), and auditory feedback. With the mouse, all interactions are abstracted and indirect. For example, when we want to scroll, we move a mouse on our desk, which moves a cursor on the screen, which adjusts the position of a scroll bar, which in turn causes the content to scroll. All these actions have to be learned whereas just grabbing something and pushing it where you want it feels much more natural. One advantage of touch is that it can leverage our existing understanding of the physical world. We see what looks like a button and tap or press it to make something happen. To scroll, we touch the content and push it in the direction we want it to move. Where we run into trouble is when we move from direct manipulation to the use of touch, multi touch, and gesture to perform commands. Here there is increasing inconsistency in how user actions are mapped to system commands. In many ways, using gestures to issue commands is a step backwards. It’s like reverting to the days of command line interfaces. Graphical user interfaces (GUIs) provide us with menus that we can read to find (recognize) the command we want. With gesture we have to remember (recall) what each gesture means, in terms of a command. Recall is much more difficult than recognition, as evidenced by the frequency of Google searches like these:

Major software players like Apple, Microsoft, and Google can’t agree on gesture meanings. Hardware vendors like HTC, Samsung and LG often add to the confusion by insisting on putting a proprietary overlay on the user experience. This is not an area for product differentiation. This is an area that requires standardization, predictability, and consistency with user expectations. As the number of gestures supported increases, consistency becomes more and more critical to the user experience. For example, here are just some of the current mappings for the seemingly simple Tap and Hold gesture.

PlatformTAP and HOLD Gesture Meaning(s)Apple iPhoneDisplay an information bubble, magnify content under the finger, or perform specific actions in applicationsApple OSX TrackpadNot supported as a single finger Tap and HoldAndroid SmartphonesOpen context menu, display information bubble, move application icon, select text for editing, enter selection modeWindows Phone 7Show options (context menu) for an itemPalm webOSEnter “reorder” mode, then drag items to move them

Although there is fairly good consensus for tap, double-tap, drag, flick, pinch and spread, other gestures vary widely.

Some best practices to guide touch and gesture interaction design

You may not be able to do much about the inconsistencies across platforms but you can reduce the impact of these inconsistencies by adhering to well-established design principles.

1) Size buttons and controls appropriately for touch and for the task context

Use the recommended size of 9mm square for most touch targets. Double that size for frequently used functions or where there are major consequences of making a mistake. Where vertical space is constrained, maintain a minimum vertical target size of 7mm but make the target 15mm wide or more. Often smaller visual targets are used to promote better positioning accuracy by the user (as long as they are still large enough to look touchable) while the touch target extends beyond the graphic (see examples below from the Microsoft Touch Guidelines).

2) Locate controls at the point of need

Avoid requiring people to make large distance transitions across the screen to get to other targets, especially for common or related tasks. Keep commands or controls where they are most likely to be used and near the bottom of the screen so they can easily be reached when using a smartphone one-handed. Avoid placing small touch targets near the edge of the display where bezels or cases can interfere with the finger touching the desired target.

3) Provide visible gestural cues (affordances) to aid discovery

Many gestures are not intuitive, like having to scroll past the top of content to display a hidden search field or pressing and holding an application icon to reposition it. People often don’t discover these things on their own. Features that you’ve worked hard to provide go undiscovered and unused. If people can’t find a feature, it doesn’t exist. How many of you would discover this Four-Finger Flick supported on touchpads in Windows? “To open the Aero 3D Window management, place four fingers slightly separated near the lower edge of the TouchPad. Flick your fingers in a smooth, straight motion towards the upper edge. Lift your fingers off the TouchPad. To minimize or restore Windows on your desktop, place four fingers slightly separated near the upper edge of the TouchPad. Flick your fingers in a smooth, straight motion towards the lower edge. Lift your fingers off the TouchPad. Repeat the gesture to restore the Windows on your desktop.” Reference: http://www.synaptics.com/solutions/technology/touchpad Totally intuitive right? What do you think this gesture means?

Reference: http://www.lukew.com/touch/TouchGestureGuide.pdf

Sometimes the function is so critical that people know it must exist but may still have difficulty figuring out the puzzle. We’ve observed several instances where people cannot answer an incoming call on an Android phone due to inadequate cues as to what to do. They see what they interpret as a large green phone button and repeatedly try to tap on it, not noticing or understanding the meaning of a little triangle beside the phone icon. For those of you unfamiliar with the Android phone, you have to touch and drag the green icon towards the red (decline call) icon. Makes sense, right? Compare this to the Apple iPhone approach. If the screen is unlocked, you simply touch the Answer button. If the screen is locked, you have to touch and drag first to unlock the screen but the affordance is much stronger. There is a clear directional arrow and textual instructions.

4) Provide adequate feedback for every user and system action

People quickly lose their sense of control when good feedback is missing. They can’t determine the connection between their actions and results. For example, we’ve observed people swiping left to right to try and return to the previous screen of an email application, not realizing that each seemingly unsuccessful swipe was actually deleting unread emails from their list. A common issue on the Android platform is for people to touch the back button one too many times while trying to get back to an application’s Home screen. This kicks them right out of the application. The two operations: backing up through application screens and closing the application are basically treated the same and there is insufficient feedback to indicate that the next Back is going to close the application. This should be prevented. Because the finger often obscures the very action button or control someone is trying to activate, it is important that people be able to see what their finger down action is about to invoke. This gives them a chance to correct mistakes, or at least a better understanding of the result they’ll get upon finger up. In the examples shown, feedback takes the form of an enlarged character shown above the finger or a set of ripples emanating from the targeted button.

Once activated, many systems provide haptic feedback (vibration or pulse) to confirm the action has been triggered. Whenever direct manipulation is supported, the feedback should exhibit real-world properties, such as momentum and friction. If someone gives a long list a fast flick to scroll down in the list, the list should continue scrolling after the finger has left the screen with a velocity and decay dependent on the speed and magnitude of the flick. If someone tries to move something that cannot be moved in the current context, the interface should allow the movement to start but then to settle back into its original position, ideally providing some embedded help as to why this is not a valid action at this time or in this context. This lets the user know the action is valid at certain times, that the system has recognized their gesture, and why the action can’t currently be performed.

5) Foster no-risk exploration and minimize error conditions

Touch interfaces feel more natural and engaging because of their directness. Unfortunately, this directness can also lead to accidental triggering or manipulation. To support peoples’ exploration it is important to provide adequate feedback so people know what happened as a result of their touch or gesture. It is also important to provide a readily accessible Undo capability so people can easily reverse any unwanted actions. You can further reduce risk by clearly separating frequently used actions from destructive actions or preventing undesired action from happening in the first place. The latter can be done by using constrained controls (e.g. option menu versus open text field), requiring confirmations for risky actions, and minimizing error-prone textual entry by providing appropriate defaults, pick lists of recent entries or auto-completion. One often overlooked category of errors occurs when real-time applications interrupt and override the current task context. A common situation is when someone receives an incoming call just as they are about to interact with the currently active application. In the example below, imagine someone who is listening to music wants to skip ahead or back a track. Just as they go to touch the desired button a call comes in and changes the button underlying the touch area they are targeting. They unintentionally either answer or decline the incoming call. The call controls shown in the middle below replace the music player controls. The overlap of touch areas is shown in the merged image on the right.

These types of situations must be taken into account. Buttons either have to be repositioned to avoid the overlaps or delays need to be added to filter out the unintentional touch actions (for example not accepting any touch response time of less than 400 milliseconds after the incoming call action buttons have been displayed. People need to have sufficient feedback that the context has changed.

6) Keep gestures consistent with industry standards and user expectations

Avoid allowing gestures, like swipe, to mean different things depending on the area of the screen that is swiped. You don’t want someone swiping to get to the next page and accidently deleting something because they swiped a bit too high on the screen. One area of confusion and frustration on Android phones is the Menu hard key. Sometimes it generates a menu, sometimes not. There is no way to tell if a menu is available or not. Hopefully this problem will disappear as Android moves from hard keys to soft buttons in the latest OS, Ice Cream Sandwich. The button can be removed when there is no menu. Unfortunately, no one has come up with a good way to indicate or provide cues as to what gestures are available and what they mean. Some touch pad implementations provide a context menu that can display gesture shortcuts but we have not seen anything similar on phones or tablets that appear at the point of need, not buried in tutorials or help sections. Pinch and spread are two common gestures that people are expecting to use whenever they want to zoom in/out or enlarge/shrink maps, text, images, etc. Do not mix metaphors and require use of Plus and Minus buttons in other areas to achieve what users think of as the same effect. Once people learn a standard gesture, they expect it to work the same across all applications. Avoid confusion by never assigning a non-standard meaning to a standard gesture. Develop a unique gesture, if required, but give careful consideration to how people will discover, remember and use this gesture.

7) Ensure the response to all gesture interactions is immediate, accurate and reliable

Responsiveness is critical to creating a satisfying and engaging touch experience. Gesture interactions must take effect immediately. If a person is dragging an object on the screen, the contact point must stay under the person’s finger throughout the drag operation. If a person rotates their fingers through 90 degrees the object also must rotate by the same amount. Any lag or deviation from the gesture’s motion path acts to destroy the impression and illusion of direct manipulation. People start thinking about the interaction rather than about the task they are trying to accomplish. Lack of responsiveness erodes trust and the perception of quality.

Summary – Observe and test with users

A University of Illinois grad student has combined three of my favorite things: touchscreens, Star Wars, and massive space battles.