• K. Barnard, P. Duygulu, D. A. Forsyth. A method for comparing content based image retrieval methods. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• K. Barnard. Recognition as translating images into text. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003. (Invited Paper),

  ---

• C. Jelmini, S. Marchand-Maillet. DEVA: An extensible object-based annotation model for visual document collections. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• H. Konik, J. Da Rugna. Similiarity distances evaluation for query by example retrieval systems. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• M. L. Kherfi, D. Ziou. Relevance feedback in image retrieval: a new approach using positive and negative examples. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• B. Luo, X. Wang, X. Tang. A World Wide Web based image search engine using text and image content features. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• W. Müller, G. Robbert, A. Henrich. Comparing the performance of two CBIRS indexing schemes. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• Y. M. Ro, S. J. Yang, J. H. Yoon. Automatic image categorization using MPEG-7 description. In Internet Imaging IV, Santa Clara, CA, USA, January, 2003.

  ---

• Image collections: The issue of an image collection was discussed and supported
• Image annotation: Demos of these solutions that are under construction were given:
  • PHP Image annotation tool at University of Geneva
  • Image annotation tool at University Buffalo
• Publications: In order to summarise the Benchathlon session, we give below the article presented during this session:
  • Jörgensen, Corinne, Jörgensen, Peter. Testing a vocabulary for image indexing and ground truthing. In Internet Imaging III, San Jose, California, USA, ~21--22, 2002. ((SPIE Photonics West Conference)),

    ---

  • Pfund, Thomas, Marchand--Maillet, Stephane. Dynamic multimedia annotation tool. In Internet Imaging III, San Jose, California, USA, ~21--22, 2002. ((SPIE Photonics West Conference)),

    ---

  • Chang, Edward Y., Cheng, Kwang--Ting. Supporting Subjective Image Queries without Seeding requirements -- proposing test queries for Benchathlon. In Internet Imaging III, San Jose, California, USA, ~21--22, 2002. ((SPIE Photonics West Conference)),

    ---

  • I. Gagliardi, B. Zonta. Quicklink: Visual Dictionaries on the Web. In Internet Imaging III, San Jose, California, USA, ~21--22, 2002. ((SPIE Photonics West Conference)),

    ---

  • G. Ciocca, R. Schettini. Multimedia Search Engine with Relevance Feedback. In Internet Imaging III, San Jose, California, USA, ~21--22, 2002. ((SPIE Photonics West Conference)),

    ---

  This session has shown that the design of a standard CBIR benchmark is achievable. However, researchers generally have limited time and resources to spend on these issues. Inthis respect, we have re-focused the Benchathlon in first collecting information, data and software in a central point, materialised by this site. It is therefore important that people contribute since information on resources normally stays in the vicinity of the rsource. We are hoping that you will participate to the next session.

• Neil Gunther contributed a first benchmark, called BIRDS-I
  Neil J. Gunther, Giordano Beretta. A Benchmark for Image Retrieval using Distributed Systems over the Internet: BIRDS--I. HP Labs, Palo Alto, Technical Report HPL--2000--162, San Jose, 2001.

  ---

• we achieved a good understanding of the issues and wrote a paper for Internet Imaging II at EI 2001
• the paper allowed us to attract the best people working in the field
• we have this server and a live connection in the conference auditorium

However, we also had a number of fiascos:

• we lacked the leaderhip to drive the effort
• the Perl and cgi scripts were not working and we were not able to figure out why
• we never found all the images we thought we had
• we did not have the ground truth for the few thousand images we did find

In conclusion, we are very well positioned for a new attempt in 2002. Past experience teaches us that we have to move forward without hesitation and motivate interested collaborators by providing leadership.