SHREC'13 track:
Retrieval on Textured 3D Models

The aim of this track of SHREC'13 is to evaluate the performance of retrieval algorithms when models vary either by geometric shape or texture, or both. The final report of the track can be found here. Please cite this report as:

Cerri, A., Biasotti, S., Abdelrahman, M., Angulo, J., Berger, K., Chevallier, L., El-Melegy, M. T., Farag, A. A., Lefebvre, F., Giachetti, A., Guermoud, H., Liu, Y.-J., Velasco-Forero, S., Vigouroux, J.-R., Xu, C.-X., and Zhang, J.-B., SHREC'13 Track: Retrieval on Textured 3D Models, In: Proc. Eurographics Workshop on 3D Object Retrieval, Girona, Spain, S. Biasotti et al. Eds. (2013), 73 - 80.

The dataset
The dataset consists of ten classes. Each class is created as follows: First, two 3D meshes are considered and endowed with three different textures, for a total amount of six null models. Then, for each null shape 4 transformations are applied, including one non-rigid deformation, two non-metric-preserving deformations and one addititive noise deformation. What we get at the end is a database containing 24 models in each class, for a total amount of 240 textured shapes. To have a copy of the dataset and the groundtruth, please contact
Silvia Biasotti (email: silvia.biasotti@ge.imati.cnr.it) or
Andrea Cerri (email: andrea.cerri@ge.imati.cnr.it)

Ground truth and evaluation
The collection to search in is made of hundreds of watertight mesh models. Meshes will be provided in one set of watertight meshes without self-intersections. Each model will be used in turn as a query against the remaining part of the database. For a given query, the goal of the track is to retrieve the most similar objects. The retrieval performance will be evaluated according to a ternary relevance scale: If a retrieved object shares both shape and texture with the query, then it is highly relevant; if it shares only shape, it is considered marginally relevant; otherwise, it is not relevant. The evaluation process will be thus based on a 2-level ground-thruth, by using the following evaluation measures: Precision-recall curves, nearest neighbor, first tier, second tier, normalized discounted cumulated gain and average dynamic recall.

Each model will be used in turn as a query against the remaining part of the database. For a given query, the goal of the track is twofold: to retrieve the most similar objects and to classify the query itself. The relevance, marginal relevance or non-relevance of the models for a given query, i.e. the ground truth, will be established a priori. The performance of the retrieval algorithms will be evaluated using the precision-recall curves, nearest neighbor, first tier, second tier, normalized discounted cumulated gain and average dynamic recall. As for the classification task, we will adopt the percentage of success for the first and the second retrieved item, and the confusion matrix among the classes.

Registration and other procedures
Each participant is requested to register to the track by sending an email to
Silvia Biasotti (email: silvia.biasotti@ge.imati.cnr.it) and
Andrea Cerri (email: andrea.cerri@ge.imati.cnr.it)
with the subject "Registration for SHREC13: Retrieval on textured 3D models". Then, you will receive an answer with the instructions to:

• download one or both model collections (PLY/OFF files), which will be available from February 11th to 15th, 2013;
• provide by February 15th, 2013 a plain ASCII file with the dissimilarity matrix, where each entry (i,j) is the distance between models i and j (notice that the lower the value of the entry (i,j), the more you trust the similarity between i and j). Up to 3 dissimilarity matrices may be submitted, resulting from different runs. Each run may come from a different algorithm, or from a different parameter setting;
• provide the program (binary code) that generated the results submitted. If the participant uses different executables for shape description and comparison s/he should send us also the appropriate user’s instructions;
• submit a one page summary of the proposed method by February 22nd, 2013.

Further Information

• the evaluations will be done automatically;
• the track results will be combined into a joint paper, to be published in the proceedings of the Eurographics Workshop on 3D Object Retrieval;
• the description of the track and its results will be presented at the Eurographics Workshop on 3D Object Retrieval (May 11, 2013).