Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from P-T » The Future of M-Interaction - INTRODUCTION, EVOLVING INTERACTION DESIGN FOR MOBILITY, Speech-Based Audio, Non-Speech Audio, CONCLUSION

Audio-Enhanced Gestural Interaction

gestures users menu items

Gestures are naturally very expressive; we use body gestures without thinking in everyday communication. Gestures can be multidimensional: for example, we can have 2D hand-drawn gestures (Brewster et al., 2003; Pirhonen et al., 2002), 3D hand-generated gestures (Cohen & Ludwig, 1991), or even 3D head-generated gestures (Brewster et al., 2003). Harrison, Fishkin, Gujar, Mochon, and Want (1998) showed that simple, natural gestures can be used for input in a range of different situations on mobile devices. Head-based gestures are already used successfully in software applications for disabled users; as yet, however, their potential has not been fully realised nor fully exploited in other applications. There has, until recently, been little use of audio-enhanced physical hand and body gestures for input on the move; such gestures are advantageous because users do not need to look at a display to interact with it (as they must do, for example, when clicking a button on a screen in a visual display). The combined use of audio and gestural techniques present the most significant potential for viable future m-interaction. Importantly, gestural and audio-based interaction can be eyes-free and, assuming non hand-based gestures, can be used to support hands-free interaction where necessary.

A seminal piece of research that combines audio output and gestural input is Cohen and Ludwig’s Audio Windows (Cohen & Ludwig, 1991). In this system, users wear a headphone-based 3D audio display in which application items are mapped to different areas in the space around them; wearing a data glove, users point at the audio represented items to select them. This technique is powerful in that it allows a rich, complex environment to be created without the need for a visual display—important when considering m-interaction design. Savidis, Stephanidis, Korte, Crispien, and Fellbaum also developed a non-visual 3D audio environment to allow blind users to interact with standard GUIs (Savidis et al., 1996); menu items are mapped to specific places around the user’s head and, while seated, the user can point to any of the audio menu items to make a selection. Although neither of these examples was designed to be used when mobile, they have many potential advantages for m-interaction.

Schmandt and colleagues at MIT have done work on 3D audio in a range of different applications. One, Nomadic Radio , uses 3D audio on a mobile device (Sawhney & Schmandt, 2000). Using non-speech and speech audio to deliver information and messages to users on the move, Nomadic Radio is a wearable audio personal messaging system; users wear a microphone and shoulder-mounted loudspeakers that provide a planar 3D audio environment. The 3D audio presentation has the advantage that it allows users to listen to multiple sound streams simultaneously while still being able to distinguish and separate each one (the “Cocktail Party” effect). The spatial positioning of the sounds around the head also conveys information about the time of occurrence of each message.

Pirhonen et al. (2002) examined the effect of combining non-speech audio feedback and gestures in an interface to an MP3 player on a Compaq iPAQ. They designed a small set of metaphorical gestures, corresponding to the control functions of the player, which users can perform, while walking, simply by dragging their finger across the touch screen of the iPAQ; users receive end-of-gesture audio feedback to confirm their actions. Pirhonen et al. (2002) showed that the audio-gestural interface to the MP3 player is significantly better than the standard, graphically-based media player on the iPAQ.

Brewster et al. (2003) extended the work of Pirhonen et al. (2002) to look at the effect of providing non-speech audio feedback during the course of gesture generation as opposed to simply providing end-of-gesture feedback. They performed a series of experiments during which participants entered, while walking, alphanumeric and geometrical gestures using a gesture recogniser both with and without dynamic audio feedback. They demonstrated that by providing non-speech audio feedback during gesture generation, it is possible to improve the accuracy—and awareness of accuracy—of gestural input on mobile devices when used while walking. Furthermore, during their experiments they tested two different soundscape designs for the audio feedback and found that the simpler the audio feedback design the better to reduce cognitive demands placed upon users.

Fiedlander, Schlueter and Mantei (1998) developed non-visual “Bullseye” menus where menu items ring the user’s cursor in a set of concentric circles divided into quadrants. Non-speech audio cues—a simple beep played without spatialisation—indicate when the user moves across a menu item. A static evaluation of Bullseye menus showed them to be an effective non-visual interaction technique; users are able to select items using just the sounds. Taking this a stage further, Brewster et al. (2003) developed a 3D auditory radial pie menu from which users select menu items using head nods. Menu items are displayed in 3D space around the user’s head at the level of the user’s ears and the user selects an item by nodding in the direction of the item. Brewster et al. (2003) tested three different soundscapes for the presentation of the menu items, each differing in terms of the spatial positioning of the menu items relative to the user’s head. They confirmed that head gestures are a viable means of menu selection and that the soundscape that was most effective placed the user in the middle of the menu, with items presented at the four cardinal points around the user’s head.

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or

Vote down Vote up

over 2 years ago

http://www.guccihandbags.com.co/

http://www.gucci-outlet.in.net/

http://www.gucci--outlet.com.co/

http://www.guccishoes.net.co/

http://www.guccishoes.us.org/

http://www.hermesbags.com.co/

http://www.hermesbirkin.com.co/

http://www.hermesoutlet.net.co/

http://www.hollister.us.org/

http://www.hollisterclothing-store.in.net/

http://www.insanityworkout.net.co/

http://www.iphone-cases.us/

http://www.ralphlaurenpolo.in.net/

http://www.ray-ban-outlet.us.com/

http://www.raybans.us.org/

http://www.rayban-sunglasses.org.uk/

http://www.rayban-sunglasses.us.org/

http://www.raybansunglassesoutlet.net.co/

http://www.raybanwayfarer.in.net/

http://www.replicahandbags.com.co/

http://www.replicawatches.us.com/

http://www.retro-jordans.com/

http://www.rolex-watches.me.uk/

http://www.rosherun.org.uk/

http://www.rosheruns.us/

http://www.salvatoreferragamo.in.net/

http://www.soccer-shoes.org/

http://www.softball-bats.us/

http://www.suprashoe.net/

http://www.swarovskicrystal.com.co/

http://www.swarovskijewelry.com.co/

http://www.swarovski-uk.org.uk/

http://www.the-northface.com.co/

http://www.the-northface.in.net/

http://www.thenorth-face.org.uk/

http://www.thenorthface.us.org/

http://www.thenorthfacejackets.in.net/

http://www.thomassabo-uk.org.uk/

http://www.tiffanyandco.net.co/

http://www.tiffanyjewelry.us.org/

http://www.tory-burch-outlet.in.net/

http://www.tory-burchoutlet.us.com/

http://www.louboutin.jp.net/

http://www.louis-vuittoncanada.ca/

http://www.louisvuitton.jp.net/

http://www.louis--vuitton.org.uk/

http://www.louisvuitton.so/

http://www.louisvuittonas.com/

http://www.edhardy.in.net/

http://www.levisjeans.com.co/

http://www.bcbgdresses.net/

http://www.bebeclothing.net/

http://www.harrods-london.co.uk/

http://www.guccishoes.com.co/

http://www.ralphlaurenoutletonline.us.org/

http://www.true-religion.com.co/

http://www.truereligionjeans.net.co/

http://www.truereligion-outlet.com.co/

http://www.uggaustralia.net.co/

http://www.uggboots.net.co/

http://www.uggbootsclearance.com.co/

http://www.uggsonsale.com.co/

http://www.uggsoutlet.com.co/

http://www.uptocoachoutlet.com/

http://www.vansshoes.us/

http://www.weddingdressesuk.org.uk/

http://www.yogapants.com.co/

http://www.ugg-boots.us.org/

http://www.poloralphlaurenoutlet.net.co/

http://www.burberryoutletonline.ar.com/

http://www.toms-outlet.net.co/

http://www.michaelkors.in.net/

http://www.christianlouboutinoutlet.net.co/

http://www.toryburchsale.com.co/

http://www.pradaoutlet.com.co/

http://www.longchamp-handbags.in.net/

http://www.longchampoutlet.com.co/

http://www.chanel-bags.com.co/

http://www.truereligion-outlet.us.org/

http://www.abercrombie-and-fitch.us.com/

http://www.timberlandboots-outlet.net/

http://www.timberland-shoes.com/

http://www.tommyhilfiger.net.co/

http://www.tommy-hilfigeroutlet.com/

http://www.tomshoesoutlet.com/

http://www.toms-outlet.in.net/

http://www.toms-shoes.com.co/

http://www.hollisterclothing.in.net/

http://www.newbalance-shoes.org/

http://www.converse--shoes.net/

http://www.lululemonoutlet.com.co/

http://www.nfl-jerseys.in.net/

http://www.cheapjerseys.us.org/

http://www.rolex-watches.us.com/

http://www.rolexwatchesforsale.us.com/

http://www.p90xworkout.in.net/

http://www.giuseppezanotti.com.co/

http://www.maccosmetics.net.co/

http://www.instyler.in.net/

http://www.mizunorunning.net/

http://www.handbagsoutlet.com.co/

http://www.hilfigeroutlet.in.net/

http://www.kate-spade.com.co/

http://www.katespade-outlet.com.co/

http://www.kate-spades.com/

http://www.longchamp.us.org/

http://www.longchamp.com.co/