|
Benchathlon harness requirement analysis
|
Wolfgang Müller
|
Nov 06, 2001 10:30 PST
|
Dear Professor,
Dear Colleague,
I am contacting you on behalf of the Benchathlon
(http://www.benchathlon.net). The Benchathlon works towards a common,
standardized evaluation procedure for CBIRS (content based image retrieval
systems). This message contains a question to yourself or someone in
your group who realy *touches the code* of the content based image
retrieval tool developed in your group.
Part of the work in the Benchathlon is making it easy to benchmark a
given system using the Benchathlon benchmark. As part of this work I
am working on providing a simple example, on how to write a server
communicating in MRML (the communication protocol adopted by the
benchmarking initiative). In addition to that we want to provide a
*bridge* between such a server and a command line tool. Let me detail
a bit further:
We assume that among other things, you have a command line tool where
you can enter a list of image numbers or image urls, then start the
program. We assume the result values returned by your program to be a
list of urls together with some additional information.
I would like to know:
a) Are you using such a command-line tool?
b) What exactly are your input/output formats?
If your answer to point a) is "no", please detail, what
languages/technologies you use (e.g. CORBA/CGI/EJB...), and how much
work it would be for you to modify an exsisting small Perl program, or
alternatively write a small wrapper for your program, so that a)
applies for you.
Please do take the time to answer,
Please do circulate this mail to peers which might be interested.
We believe that the success of a common benchmark depends largely on
the question if people are able to use it without too much learning
curve or too much overhead. We are working hard on making life easy
for you. Please give us a chance to succeed.
Looking forward to hearing from you,
Wolfgang Müller
--
Dr. Wolfgang Müller, assistant == teaching assistant
Personal page: http://cui.unige.ch/~vision/members/WolfgangMueller.html
Maintainer, GNU Image Finding Tool (http://www.gnu.org/software/gift)
|
|
 |
|
