Image and Video Analysis

VIRaL: Visual Image Retrieval and Localization

VIRaL is a content-based image search engine. The query is an image, either uploaded, fetched from a given a URL, or chosen from the VIRaL database. Given this single image, it retrieves visually similar images from the database and estimates its location. VIRaL also suggests tags that may be attached to the query image, identifies known landmarks or points of interest, and provides links to relevant Wikipedia articles.

The visual content of an image is represented by local features and descriptors. Visually similar images are obtained using a bag-of-words (BoW) model and inverted files for fast indexing. A spatial matching phase follows to enforce geometric consistency on the top inverted index results. This eliminates false positives due to BoW ignoring geometry. Relaxed Spatial Matching is a new, accelerated method; it is currently investigated in VIRaL-Beta, an experimental version where add-ons and updates are tried first. "Similar of similar" images appear below similar ones. These may be directly or indirectly related to the query image, and they typically depict the area surrounding it.

Location recognition is based on the (known) geo-tags of similar images found. This is made as reliable as possible by identifying the largest group of similar images that appear to agree on their locations. Available textual meta-data (title, description, user tags) of similar images are also analyzed, and the most frequent are suggested as tags. For landmark recognition, the same meta-data are compared to known names of landmarks near the estimated location. Databases of landmarks and points-of-interest (POIs) from Wikipedia and GeoNames are used for this purpose.

Currently our database contains 2.221.176 Flickr® images from Amsterdam, Athens, Barcelona, Beijing, Berlin, Budapest, Buenos Aires, Cairo, Cape Town, Caracas, Chicago, Copenhagen, Delhi, Dortmund, Dubai, Dublin, Edinburgh, Florence, Havana, Helsinki, Hong Kong, Istanbul, Krakow, Lima, Lisbon, London, Los Angeles, Madrid, New York, Paris, Prague, Rome, Sheffield, St. Petersburgh, Sydney, Toronto, Vancouver, Venice and Vienna.

NEW!

VIRaL Explore enables browsing of the entire VIRaL image collection on the world map. Starting in a given city or at any zoom level on the map, it places icons corresponding to grouped photos, along with landmark or POI names and Wikipedia links, if applicable. The VIRaL collection is processed off-line to identify groups of photos depicting the same object, building, or scene, using our Scene Maps method. Most popular groups are shown on the map, according to the zoom level. The panel on the right displays shortcuts to cities or photo groups; clicking an icon triggers a visual query in VIRaL.

NEW!

VIRaL Routes offers a unique browsing experience of personal photo collections. Personal photo sets from trips are processed off-line to identify where they were taken and group them by scene; a route is then constructed on the map, showing icons of visited places, with an interface that is similar to VIRaL Explore. Currently a static collection from 16 cities is included: it is the personal collection of just seven people, ourselves!

Features

The ViRaL image search engine can:

  • receive a visual query: uploaded, fetched from a URL or from its database,
  • return a set of visually similar images from the image database,
  • return a set of "similar of similar" images from the surrounding area,
  • pinpoint similar images on Google Maps,
  • estimate the location of the query image and identify it on Google
  • suggest tags for the query,
  • recognize depicted landmarks or points of interest,
  • provide direct links to relevant Wikipedia articles,
  • show the similarity measure between the query and each retrieved image,
  • visualize feature points, correspondences and inlier features between query and each similar image.

The core search engine is implemented in C++ and the GUI in PHP and Javascript.

Publications

Conferences

Y. Avrithis, Y. Kalantidis, G. Tolias, E. Spyrou. Retrieving Landmark and Non-Landmark Images from Community Photo Collections. In Proceedings of ACM Multimedia (MM 2010), Firenze, Italy, October 2010.
[ Abstract ]
[ Bibtex ] [ PDF ]
G. Tolias, Y. Avrithis. Speeded-up, Relaxed Spatial Matching. In Proceedings of International Conference on Computer Vision (ICCV 2011), Barcelona, Spain, November 2011.
[ Abstract ]
[ Bibtex ] [ PDF ]

Journals

Y. Kalantidis, G. Tolias, Y. Avrithis, M. Phinikettos, E. Spyrou, P. Mylonas, S. Kollias. VIRaL: Visual Image Retrieval and Localization. In Multimedia Tools and Applications (to appear), 2011.
[ Abstract ]
[ Bibtex ] [ PDF ]
Project

The research leading to these results has received funding from the European Community's Seventh Framework Programme FP7/2007-2013 under grant agreement n215453 - WeKnowIt.

Contact

,