Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from K-O

Multimodal Interfaces

systems based oviatt users’

Definition: Multimodal interfaces process two or more combined user input modes, such asspeech, pen, touch, manual gestures, and gaze, in a coordinated manner with multimedia system output.

They are a new class of emerging systems that aim to recognize naturally occurring forms of human language and behavior, with the incorporation of one or more recognition-based technologies (e.g., speech, pen, vision). Multimodal interfaces represent a paradigm shift away from conventional graphical user interfaces. They are being developed largely because they offer a relatively expressive, transparent, efficient, robust, and highly mobile form of human-computer interaction. They represent users’ preferred interaction style, and they support users’ ability to flexibly combine modalities or to switch from one input mode to another that may be better suited to a particular task or setting.

Multimodal systems have evolved rapidly during the past decade, with steady progress toward building more general and robust systems. Major developments have occurred in the hardware and software needed to support key component technologies incorporated in multimodal systems, in techniques for fusing parallel input streams, in natural language processing (e.g., unification-based integration), and in time-sensitive and hybrid architectures. To date, most current multimodal systems are bimodal, with the two most mature types involving speech and pen or touch input (Oviatt et al., 2000), and audio-visual input (e.g., speech and lip movements; Potamianos et al., 2004). However, these systems have been diversified to include new modality combinations such as speech and manual gesturing, and gaze tracking and manual input. Multimodal applications also range from map-based and virtual reality systems for simulation and training, to multi-biometric person identification/verification systems for security purposes, to medical, educational, and web-based transaction systems (Oviatt et al., 2000).

Given the complex nature of users’ multimodal interaction, cognitive science has played an essential role in guiding the design of robust multimodal systems. The development of well integrated multimodal systems that blend input modes synergistically depends heavily on accurate knowledge of the properties of different input modes and the information content they carry, how multimodal input is integrated and synchronized, how users’ multimodal language is organized, what individual differences exist in users’ multimodal communication patterns, and similar issues (Oviatt, 2003). Prototyping of new multimodal systems also has depended heavily on the use of high-fidelity simulation methods.

Commercial activity on multimodal systems has included PDAs, cell phones, in-vehicle systems, and desktop applications for CAD and other application areas. Commercial systems typically do not include parallel processing of two semantically rich input modes delivered together by the user, and those that are not fusion-based systems but instead process just one mode alternative at a time. In contrast, for over a decade research-level multimodal systems have included fusion-based processing of two parallel input streams that each conveys rich semantic information (Oviatt et al., 2000). Such systems can support mutual disambiguation of input signals. Mutual disambiguation involves recovery from unimodal recognition errors within a multimodal architecture, because semantic information from each input mode supplies partial disambiguation of the other mode (Oviatt, 2003).

One particularly advantageous feature of such multimodal systems is their superior error handling. The error suppression achievable with a bimodal system, compared with a unimodal one, can be in excess of 40%. Trimodal multi-biometric systems have also been demonstrated to perform more robustly than bimodal ones. In addition to improving the average system reliability, a multimodal interface can perform in a more stable manner for challenging user groups (e.g., accented speakers) and in real-world settings (e.g., mobile contexts).

Research has begun to design more tangible, adaptive, and collaborative multimodal systems for a variety of application areas (e.g., meetings, education). Recent progress in cognitive science, neuroscience, and biology is also contributing to the theoretical foundations needed to design future multimodal interfaces with greater flexibility, generality and power.

Multiple Source Alignment for Video Analysis - Abstract, Introduction, Screenplays in movie production practice, System Architecture, a. Screenplay parsing [next] [back] Multimedia Web Information Systems - World Wide Web History, Multimedia on the World Wide Web

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or

Vote down Vote up

about 1 year ago

http://4th.co.in/
packers and movers Hyderabad @ http://4th.co.in/packers-and-movers-hyderabad/

Vote down Vote up

over 1 year ago

http://4th.co.in/packers-and-movers-chennai/

Vote down Vote up

over 1 year ago

Movers Packers Mumbai @ http://4th.co.in/packers-and-movers-mumbai/
Movers Packers Gurgaon @ http://4th.co.in/packers-and-movers-gurgaon/
Movers Packers Delhi @ http://4th.co.in/packers-and-movers-delhi/
http://4th.co.in/

Vote down Vote up

over 1 year ago

Movers and Packers in Delhi @ http://3th.co.in/packers-and-movers-delhi/
Movers and Packers in Noida @ http://3th.co.in/packers-and-movers-noida/
Movers and Packers in Faridabad @ http://3th.co.in/packers-and-movers-faridabad/
Movers and Packers in Ghaziabad @ http://3th.co.in/packers-and-movers-ghaziabad/
Movers and Packers in Chandigarh @ http://3th.co.in/packers-and-movers-chandigarh/

Vote down Vote up

over 1 year ago

Movers and Packers in Delhi @ http://3th.co.in/packers-and-movers-delhi/
Movers and Packers in Noida @ http://3th.co.in/packers-and-movers-noida/
Movers and Packers in Faridabad @ http://3th.co.in/packers-and-movers-faridabad/
Movers and Packers in Ghaziabad @ http://3th.co.in/packers-and-movers-ghaziabad/
Movers and Packers in Chandigarh @ http://3th.co.in/packers-and-movers-chandigarh/

Vote down Vote up

almost 2 years ago

Packers and Movers in Delhi @ http://www.shiftingservices.in/packers-and-movers-delhi.html
Packers and Movers in Gurgaon @ http://www.shiftingservices.in/packers-and-movers-gurgaon.html
Packers and Movers in Bangalore @ http://www.shiftingservices.in/packers-and-movers-bangalore.html
Packers and Movers in Hyderabad @ http://www.shiftingservices.in/packers-and-movers-hyderabad.html

Vote down Vote up

almost 2 years ago

Packers and Movers in Pune @ http://www.shiftingservices.in/packers-and-movers-pune.html
Packers and Movers in Mumbai @ http://www.shiftingservices.in/packers-and-movers-mumbai.html
Packers and Movers in Kolkata @ http://www.shiftingservices.in/packers-and-movers-kolkata.html

Vote down Vote up

about 2 years ago

http://www.official-cheapjerseys.com/ Cheap NFL Jerseys

http://www.oakleysunglassesoutlet.name/ Oakley Sunglasses Outlet

http://www.yeezyboost350.us.com/ Yeezy 350 Boost

http://oakleysunglassescheap.net.co/ Cheap Oakley Sunglasses

http://www.ray-bansunglasses.cc/ Ray-Ban Sunglasses

http://ray-bansunglasses.top/ Ray Ban Sunglasses

http://www.raybanoutletstoresonline.net/ ray ban factory outlet

http://www.ralphlaurenoutletstoreonline.com/ Ralph Lauren Outlet

http://www.raybanoutletsunglasses.org/ ray ban outlet sunglasses

http://poloralphlaurenoutletstore.net/ Polo Ralph Lauren Outlet

http://www.yeezyboost.org/ Yeezy Boost

http://www.nfljerseyswhosale.us.com/ NFL Jerseys

http://www.uggsclearancesaleoutlet.org/ Uggs Clearance Sale Outlet

Vote down Vote up

over 2 years ago

http://www.guccihandbags.com.co/

http://www.gucci-outlet.in.net/

http://www.gucci--outlet.com.co/

http://www.guccishoes.net.co/

http://www.guccishoes.us.org/

http://www.hermesbags.com.co/

http://www.hermesbirkin.com.co/

http://www.hermesoutlet.net.co/

http://www.hollister.us.org/

http://www.hollisterclothing-store.in.net/

http://www.insanityworkout.net.co/

http://www.iphone-cases.us/

http://www.ralphlaurenpolo.in.net/

http://www.ray-ban-outlet.us.com/

http://www.raybans.us.org/

http://www.rayban-sunglasses.org.uk/

http://www.rayban-sunglasses.us.org/

http://www.raybansunglassesoutlet.net.co/

http://www.raybanwayfarer.in.net/

http://www.replicahandbags.com.co/

http://www.replicawatches.us.com/

http://www.retro-jordans.com/

http://www.rolex-watches.me.uk/

http://www.rosherun.org.uk/

http://www.rosheruns.us/

http://www.salvatoreferragamo.in.net/

http://www.soccer-shoes.org/

http://www.softball-bats.us/

http://www.suprashoe.net/

http://www.swarovskicrystal.com.co/

http://www.swarovskijewelry.com.co/

http://www.swarovski-uk.org.uk/

http://www.the-northface.com.co/

http://www.the-northface.in.net/

http://www.thenorth-face.org.uk/

http://www.thenorthface.us.org/

http://www.thenorthfacejackets.in.net/

http://www.thomassabo-uk.org.uk/

http://www.tiffanyandco.net.co/

http://www.tiffanyjewelry.us.org/

http://www.tory-burch-outlet.in.net/

http://www.tory-burchoutlet.us.com/

http://www.louboutin.jp.net/

http://www.louis-vuittoncanada.ca/

http://www.louisvuitton.jp.net/

http://www.louis--vuitton.org.uk/

http://www.louisvuitton.so/

http://www.louisvuittonas.com/

http://www.edhardy.in.net/

http://www.levisjeans.com.co/

http://www.bcbgdresses.net/

http://www.bebeclothing.net/

http://www.harrods-london.co.uk/

http://www.guccishoes.com.co/

http://www.ralphlaurenoutletonline.us.org/

http://www.true-religion.com.co/

http://www.truereligionjeans.net.co/

http://www.truereligion-outlet.com.co/

http://www.uggaustralia.net.co/

http://www.uggboots.net.co/

http://www.uggbootsclearance.com.co/

http://www.uggsonsale.com.co/

http://www.uggsoutlet.com.co/

http://www.uptocoachoutlet.com/

http://www.vansshoes.us/

http://www.weddingdressesuk.org.uk/

http://www.yogapants.com.co/

http://www.ugg-boots.us.org/

http://www.poloralphlaurenoutlet.net.co/

http://www.burberryoutletonline.ar.com/

http://www.toms-outlet.net.co/

http://www.michaelkors.in.net/

http://www.christianlouboutinoutlet.net.co/

http://www.toryburchsale.com.co/

http://www.pradaoutlet.com.co/

http://www.longchamp-handbags.in.net/

http://www.longchampoutlet.com.co/

http://www.chanel-bags.com.co/

http://www.truereligion-outlet.us.org/

http://www.abercrombie-and-fitch.us.com/

http://www.timberlandboots-outlet.net/

http://www.timberland-shoes.com/

http://www.tommyhilfiger.net.co/

http://www.tommy-hilfigeroutlet.com/

http://www.tomshoesoutlet.com/

http://www.toms-outlet.in.net/

http://www.toms-shoes.com.co/

http://www.hollisterclothing.in.net/

http://www.newbalance-shoes.org/

http://www.converse--shoes.net/

http://www.lululemonoutlet.com.co/

http://www.nfl-jerseys.in.net/

http://www.cheapjerseys.us.org/

http://www.rolex-watches.us.com/

http://www.rolexwatchesforsale.us.com/

http://www.p90xworkout.in.net/

http://www.giuseppezanotti.com.co/

http://www.maccosmetics.net.co/

http://www.instyler.in.net/

http://www.mizunorunning.net/

http://www.handbagsoutlet.com.co/

http://www.hilfigeroutlet.in.net/

http://www.kate-spade.com.co/

http://www.katespade-outlet.com.co/

http://www.kate-spades.com/

http://www.longchamp.us.org/

http://www.longchamp.com.co/