Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from P-T

Role of Semantics in Multimedia Applications - Multimedia Applications using Semantic Computation

qos rich video image

Definition: Semantic Web Tools and Standards greatly assist in both description and reasoning stages of computation.

Many types of multimedia applications can – and do – leverage rich information model semantics. Much like textual applications (such as multi-schemata databases) in which ontologies and semantics come into play, multimedia applications constitute a rich proving-ground due to their naturally diverse and disparate properties (i.e. encoding types, temporal attributes, spatial attributes, and so on.)

Multimedia Applications using Semantic Computation

A few of many multimedia domains in which semantics are pivotal include: digital libraries, broadcast media, editing, entertainment, telemedicine, and surveillance. The following list is only partially representative; each entry describes in what sense semantics are exploited or particularly important (not all use the same sense of the term or supporting technologies):

  • The Motion Pictures Expert Group – The MPEG-7 standard attempts to provide a content description interface and tools for describing audiovisual information (e.g. audio, speech, video, 3D) independently of how it is stored or coded. Using Multimedia Description Scheme (MDS) “visual descriptors” will describe visual features of the media – such features include: color, objects, and events, interactions amongst objects, storage format, lyrics and title. A reference schema with carefully contrived syntax and semantics will support this.
  • Classification and Filtering of Image Information – Image classification, searching, and filtering are hard problems that benefit from semantic domain knowledge. For example, it has been shown that when filtering algorithms rely not only on pixel content but also on image semantics then results are superior. That is, the image is first classified and then filtered, allowing the filter search only for features relevant to the type (e.g. nature, nudity, etc.). Alternatively, in the automatic segmentation of video into ‘shots’ separated by camera cuts, it is common to exploit the semantic of video content – e.g. at a high level, news video clips often admit to an almost ‘templated’ sequence of shots (e.g. intro shot, head shot, etc.)
  • Quality of Service – QoS is invariably important in the delivery of multimedia information across a network and when defining service agreements for application performance. QoS can have many dimensions (e.g. jitter, delay, frames per second, etc.). Rich semantic models of QoS requirements and capabilities assist middleware in brokering QoS between clients and providers. Some current applications (particularly Web Service-oriented ones) are beginning to use W3C’s RDF and OWL as bases for both describing QoS requirements and performing brokering. This trend will continue and absorb multimedia services as well.
  • Pervasive Computing (PC) – The PC paradigm is the state of affairs when multimedia devices, agents, and services all seamlessly support human objectives. For example, a user with a laptop enters a new office and is seamlessly offered services in the new context thanks to interacting software agents and services. Underlying the agents is a rich representation of context, user and device requirements, and service capabilities.
  • Multimedia Databases – Multimedia DBMS’s benefit from sophisticated semantic models accommodating rich semantics of multimedia data (e.g. spatio-temporal aspects, etc.)
Roll, Freddy, Roll! [next] [back] Roizen, Michael F. - Physician and author, Career, Sidelights, Selected writings

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or