Method Evaluation

The performance of the shape retrieval will be measured using 1st and 2nd tier precision and recall, finally presented as the F-measure. This measure gives a nice overview of the complete retrieval performance and the method especially, since the repository is similar for all participants. The graphs showing the final results will also contain the maximum possible score as a reference.

Precision is defined as the ratio:

relevant retrieved objects / retrieved objects

Objects retrieved from the pair class will count as half objects. For this contest track it means the following:

1st Tier: The first 10 objects retrieved.

Max(10) / 10

2nd Tier The first 20 objects retrieved.

Max(15) / 20

Recall is defined as the ratio:

relevant retrieved objects / total relevant objects

Objects retrieved from the pair class will count as half objects. For this contest track it means the following:

1st Tier: The first 10 objects retrieved.

Max(10) / 15

2nd Tier The first 20 objects retrieved.

Max(15) / 15

F-measure is separately calculated for both 1st and 2nd tier. The F-Measure will only be high if both precision and recall are high:

(2 * Recall * Precision) / (Recall + Precision)