Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from A-E

Emotion and Multimedia Content

emotional emotions video various

Definition: Multimedia content can today include the representation, measurements and prediction of emotions.

Representing, measuring, and predicting the emotion related to multimedia content is now accepted as being able to add significant value to multimedia systems; for example, to add an additional querying dimension to multimedia content-based retrieval systems, to enable personalized multimedia content filtering based on emotional intensity, and so on. Emotions are governed by our appraisal of environmental stimuli and our motivation to approach or avoid them. Using subjects’ various facial expressions and physiological and behavioral responses to various stimuli, it is possible to infer many emotions, such as happiness, surprise, fear, and so forth. These emotions have been shown to correlate with a two-dimensional space, often termed an affect space , made up of two independent dimensions: valence or pleasantness (negative to positive) and arousal (low to high). For example, fear would invoke negative valence and high arousal. The affect space provides a straightforward but effective means of representing both primary (primitive) and secondary (compound) emotions, which in turn presents a tangible baseline with which to categorize, measure, assess, and predict emotional response.

Approaches to the utilization of emotion in multimedia content research have thus far varied. For example, Colombo and Del Bimbo enable the retrieval of video commercials by emotion based on conformance to semantic indices and similarity to videos held within the database. They abstract emotional content by using a rule-based mapping of perceptual features, such as motion and color, on to the emotional categories of action, excitement, suspense, quietness, relaxation, and happiness. Hanjalic has focused specifically on measuring excitement in video from low-level features, including motion activity, energy in the audio track, and the density of cuts. Results are presented in the form of excitement time curves, modeling temporal frame indices and corresponding excitement levels. The approach has proved successful with various types of content, including that from movies and sports footage. Conversely, Kang has proposed a method to identify fear, sadness and joy from video by analyzing color, motion and shot cut rate, coupled with user relevance feedback to train and improve system performance.

MPEG-7 serves as a first attempt to standardize how multimedia systems may represent emotional metadata. Within its Multimedia Description Schemes, the Affective Description Scheme enables description of a user’s emotional response to various types of multimedia content, such as video segments, semantic entities (e.g. events and objects), and so on. Numeric scores on a scale of -1.0 to 1.0 are used to represent the relative intensity of the affective response with respect to a specified affect type. While any set of affect types can be applied, a set of typical types is provided by the AffectType Classification Scheme: interested, excited, bored, surprised, sad, hateful, angry, expectant, happy, scared, storyComplication and storyShape .

Encryption in RTP [next] [back] Emiliani, Cesare

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or