Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from P-T

Semantic Image Representation and Indexing -  

query semantics visual images

Joo-Hwee Lim
Institute for Info cotmm Research, Singapore

Definition: Besides low-level visual features, such as color, texture, and shapes, high-level semantic information is useful and effective in image retrieval and indexing.

Low-level visual features such as color, texture, and shapes can be easily extracted from images to represent and index image content. However, they are not completely descriptive for meaningful retrieval. High-level semantic information is useful and effective in retrieval. But it depends heavily on semantic regions, which are difficult to obtain themselves. Between low-level features and high-level semantic information, there is an unsolved “semantic gap”.

The semantic gap is due to two inherent problems. One problem is that the extraction of complete semantics from image data is extremely hard as it demands general object recognition and scene understanding. Despite encouraging recent progress in object detection and recognition, unconstrained broad image domain still remains a challenge for computer vision. For instance, consumer photographs exhibit highly varied contents and imperfect image quality due to spontaneous and casual nature of image capturing. The objects in consumer images are usually ill-posed, occluded, and cluttered with poor lighting, focus, and exposure. There is usually large number of object classes in this type of polysemic images. Robust object segmentation for such noisy images is still an open problem

The other problem causing the semantic gap is the complexity, ambiguity and subjectivity in user interpretation. Relevance feedback is regarded as a promising technique to solicit user’s interpretation at post-query interaction . However the correctness of user’s feedback may not be statistically reflected due to the small sampling problem.

Pre-query annotation enables textual search but the tedious manual process is usually incomplete, inconsistent, and context sensitive. Moreover there are situations when image semantics cannot be captured by labeling alone.

Query By Example (QBE) requires a relevant image to be visible or available as an example during query to start with. The semantics of the query is implicit in the content of the query image. Query By Canvas (QBC) let user compose a visual query using geometrical shapes, colors and textures and leave the system to interpret the semantics of the composed query. It is desirable to have the user communicate his or her query expectation to the system using some unambiguous vocabulary. A new query formulation method that allows user to specify visual semantics explicitly is described in the article on Semantic Visual Query and Retrieval.

In order to build semantic image retrieval systems for various application domains such as consumer photographs, medical images etc, it is important to have a structured framework to represent and index images with respect to domain-specific visual semantics. To reduce the human effort in annotating images with visual semantics, a systematic and modular approach to construct visual semantics detectors from statistical learning is essential. These are the subject of this article.


Semantic Medical Image Indexing [next] [back] Semantic Consumer Image Indexing

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or