Semantic Class-Based Image Indexing

Definition: In a unique Class Relative Indexing (CRI) scheme, image classification is a means to compute inter-class semantic image indexes for similarity-based matching and retrieval .

A natural and useful insight is to formulate image retrieval as a classification problem. In very general terms, the goal of image retrieval is to return images of a class C that the user has in mind based on a set of features computed for each image x in the database. In probabilistic sense, the system should return images ranked in the descending return status value of P ( C | x ), whatever C may be defined as desirable. For example, a Bayesian formulation to minimize the probability of retrieval error (i.e. the probability of wrong classification) had been proposed to drive the selection of color and texture features and to unify similarity measures with the maximum likelihood criteria.

Image classification or class-based retrieval approaches are adequate for query by predefined image class. However, the set of relevant images R may not correspond to any predefined class C in general. In a unique Class Relative Indexing (CRI) scheme, image classification is not the end but a means to compute inter-class semantic image indexes for similarity-based matching and retrieval.

When we are dealing with Query By Example (QBE), the set of relevant images R is obscure and a query example q only provides a glimpse into it. In fact, the set of relevant images R does not exist until a query has been specified. However, to anchor the query context, we can define Semantic Support Classes (SSCs) C k , k = 1, 2, M as prototypical instances of the relevance class R and compute the relative memberships to these classes of query q . Similarly we can compute the inter-class index for any database image x . These inter-class memberships allow us to compute a form of categorical similarity between q and x .

Using the softmax function, the image classification output R k given an image x is computed as

The similarity ? ( q,x ) between a query q and an image x is then computed as

This semantic class-based image indexing scheme has been tested on 2400 consumer images using taxonomy of seven predefined classes (i.e. M = 7), aggregated local semantic regions as image features and support vector machines as image classifiers with good precision and recall performance.