Image and Video Analysis

Visual Context Modeling and Analysis

Using Visual Context for High-Level Concept Detection

In this work we investigate detection of high-level concepts in multimedia content through an integrated approach of low-level visual analysis and exploitation of high-level visual context. Visual context deals with two aspects, namely high-level concepts and low-level features of an image, employing a model of a priori specified semantic relations among the concepts and automatically extracted topological relations among visual features, thus combining both conceptual and topological context. Indicative fragments of proposed "fuzzified" ontologies are shown below:


context-relations1.jpg

context-relations2.jpg

Fragments of corresponding visual context ontologies.

In our current approach all relations between concepts are represented in such an ontology, each one of them forming a part of the overall contextual knowledge. We focus on a representative example, i.e. the partOf relation P. The partOf relation, P, is a fuzzy partial ordering on the set of concepts. P(x,y)>0 means that y is part of x. For example x could be tennis and y could be a tennis ball. Its representation may take the form presented in the following figure; two concepts are provided and supposing an RDF namespace &dom following sample part of RDF representation is produced:


context-RDF2.jpg

RDF representation of the partOf relation.

A set of algorithms is also developed, which modify either the confidence values of detected concepts, or the model vectors of low-level features based on which visual detection is performed. Some indicative results from the application of visual context on a set of regions of one sample image containing six possible high-level concepts are presented in the following figure.


context-results.jpg

Indicative contextualization results.

Publications

Journals

Th. Athanasiadis, Ph. Mylonas, Y. Avrithis, S. Kollias. Semantic Image Segmentation and Object Labeling. In IEEE Transactions on Circuits and Systems for Video Technology, vol 17, no. 3, pp. 298 - 312, March 2007.
[ Abstract ]
[ Bibtex ] [ PDF ]
Ph. Mylonas, E. Spyrou, Y. Avrithis, S. Kollias. Using Visual Context and Region Semantics for High-Level Concept Detection. In IEEE Transactions on Multimedia, vol 11, no. 11, pp. 229-243, February 2009.
[ Abstract ]
[ Bibtex ] [ PDF ]

Conferences

Ph. Mylonas, Y. Avrithis. Using Multiple Domain Visual Context in Image Analysis. In Proceedings of 8th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2007), Santorini, Greece, June 2007.
[ Abstract ]
[ Bibtex ] [ PDF ]
E. Spyrou, Ph. Mylonas, Y. Avrithis. Using Region Semantics And Visual Context For Scene Classification. In Proceedings of 1st Workshop on Multimedia Information Retrieval: New Trends and Challenges (ICIP 2008), in conjunction with ICIP, San Diego, California, USA, October 2008.
[ Abstract ]
[ Bibtex ] [ PDF ]
E. Spyrou, Ph. Mylonas, Y. Avrithis. A Visual Context Ontology for Multimedia High-Level Concept Detection. In Proceedings of 5th International Workshop in Modeling and Reasoning in Context (MRC 2008), Delft, The Netherlands, June 2008.
[ Abstract ]
[ Bibtex ] [ PDF ]
E. Spyrou, Ph. Mylonas, Y. Avrithis. Semantic Multimedia Analysis based on Region Types and Visual Context. In Proceedings of 4th IFIP Conference on Artificial Intelligence Applications & Innovations (AIAI 2007), Athens, Greece, September 2007.
[ Abstract ]
[ Bibtex ] [ PDF ]
P. Tzouveli, K. Raftopoulos, K. Ntalianis, S. Kollias. Semantic Object Watermark Re-Synchronization Based on Skeleton Vertex Corresponds. In Proceedings of ACM Multimedia 2010 Workshop - Multimedia in Forensics, Security and Intelligence (MiFor) (MiFor - ACM Multimedia 2010), October 2010.
[ Abstract ]
[ Bibtex ]