|
| |
Information
|
|
|
Comparing the performance of CBIR (Content-Based Image Retrieval)
algorithms is difficult. Private data sets are used, so it is controversial
to compare CBIR algorithms developed by different researchers. Also,
the performance of CBIR algorithms is usually measured on an isolated,
well-tuned PC or workstation. In a real-world environment, however,
the CBIR algorithms would only constitute a minor component among
the many interacting components needed to facilitate a useful CBIR
application e.g., Web-based applications on the Internet. It is
therefore the aim of the Benchathlon to set up a favourable collaborative
environement where standard CBIR evaluation protocols and frameworks
will be developped.
|
|
Comparing the performance of CBIR (Content-Based Image Retrieval)
algorithms is difficult. Private data sets are used, so it is controversial
to compare CBIR algorithms developed by different researchers. Also,
the performance of CBIR algorithms is usually measured on an isolated,
well-tuned PC or workstation. In a real-world environment, however,
the CBIR algorithms would only constitute a minor component among
the many interacting components needed to facilitate a useful CBIR
application e.g., Web-based applications on the Internet. The Internet,
being a shared medium, dramatically changes many of the usual assumptions
about measuring CBIR performance. Any CBIR benchmark should be designed
from a networked systems standpoint. Networked system benchmarks
have been developed for other applications e.g., text retrieval,
and relational database management. These benchmarks typically introduce
communication overhead because the real systems they model are distributed
applications e.g., an airline reservation system. The most common
type of distributed computing architecture uses a client/server
model.
|
|
At the Internet Imaging Conference 2000 the suggestion was put
forward to hold a public contest to assess the merits of various
image retrieval algorithms. The contest would be held at the Internet
Imaging Conference in January 2001. Since the contest would require
a uniform treatment of image retrieval systems, the concept of a
benchmark quickly enters into this scenario. The contest would exercise
one or more such Content Based Image Retrieval (CBIR) benchmarks.
The contest itself became known as the Benchathlon.
The original suggestion for holding an image retrieval contest
was due to Theo Gevers (University of Amsterdam, Netherlands) who,
together with Alberto Del Bimbo (University of Florence, Italy),
provided an initial outline for
how such a contest might be conducted. Yining Deng, Ullas Gargi,
and Lucy Cherkasova (all of Hewlett-Packard Laboratories, USA) together
with Xu Yin (Tsinghua University, China) participated in early discussions
based on the original contest outline. Yining Deng and Ullas Gargi
implemented a test image collection together with a browser interface.
This eased the process for Neil Gunther (Performance Dynamics Consulting)
to write a first prototype BIRDS-I benchmark in Perl. Ullas Gargi
staged the prototype Benchathlon web server at HP Labs. Corinne
Jorgensen (SUNY Buffalo, USA) and Alberto Del Bimbo provided some
of the images for the test collection. B.S. Manjunath (U.C. Santa
Barbara, USA) provided us with a draft of his forthcoming paper
and an explanation of the MPEG-7 normalized rank retrieval
rate.
|
|
Follow up with the Events page.
|
|
|
