SHREC'14 track: Retrieval and Classification on Textured 3D Models

The aim of this SHREC'14 track is to evaluate the performance of retrieval and classification algorithms when models vary either by geometric shape or texture. The track updates the SHREC'13 track: Retrieval on textured 3D models. The SHREC14 track: Retrieval and classification on textured 3D models final report can be found here. Please cite this report as:

Biasotti, S., Cerri, A., Abdelrahman, M., Aono, M., Ben Hamza, A., El-Melegy, M., Farag, A., Garro, V., Giachetti, A., Giorgi, D., Godil, A., Li, C., Liu, Y.-J., Martono, H.Y., Sanada, C., Tatsuma, A., Velasco-Forero, S., Xu, C.-X., SHREC'14 Track: Retrieval and Classification on Textured 3D Models, In: Proc. Eurographics Workshop on 3D Object Retrieval, Strasbourg, France, B. Bustos, H. Tabia, J.-P. Vandeborre and R. Veltkamp Eds. (2014), 111 - 120.

The dataset
The dataset consists of approximately 500 watertight textured shapes, grouped in equal-sized classes. Each class is created by applying a number of transformations to a starting set of null shapes. Transformations have been designed to alter the geometry and the topology of null shapes under the assumption that models are in turn embedded in the Euclidean space or in a suitable color space. Also, the strength level at which each transformation is applied is randomly selected shape by shape. To have a copy of the dataset and the groundtruth, please contact
Silvia Biasotti (email: silvia.biasotti@ge.imati.cnr.it) or
Andrea Cerri (email: andrea.cerri@ge.imati.cnr.it)

Training set, ground truth and evaluation
The dataset will be divided into training set and test set, in a way that the former is approximately the 20% in size of the latter. In the training set a sample of each class will be provided, possibly by considering different instances of the considered transformations class by class.

Each model will be used in turn as a query against the remaining part of the database. For a given query, the goal of the track is twofold: to retrieve the most similar objects and to classify the query itself. The relevance, marginal relevance or non-relevance of the models for a given query, i.e. the ground truth, will be established a priori. The performance of the retrieval algorithms will be evaluated using the precision-recall curves, nearest neighbor, first tier, second tier, normalized discounted cumulated gain and average dynamic recall. As for the classification task, we will adopt the percentage of success for the first and the second retrieved item, and the confusion matrix among the classes.

Registration and other procedures
Each participant is requested to register to the track by sending an email to
Silvia Biasotti (email: silvia.biasotti@ge.imati.cnr.it) and
Andrea Cerri (email: andrea.cerri@ge.imati.cnr.it)
with the subject "SHREC14 track: Retrieval and classification on textured 3D models". Then, an answer will be sent to each participant with the instructions to:

• download the training set (PLY/OFF files), which will be available from February 6th to 14th, 2014;
• download the whole model collection (PLY/OFF files), which will be available from February 10th to 14th, 2014;
• provide by February 14th, 2014 the results of his/her runs, the program that generated the results (binary code, including appropriate user’s instructions if needed) and a summary of the proposed method. For each task, the results of up to 3 runs may be submitted: Each run may come from a different algorithm, or from a different parameter setting.

Results will then be processed and distributed among the participants by February 21st, 2014 so that the track report can be submitted to the SHREC'14 organizers by February 25th, 2014.

Further Information

• the evaluations will be done automatically;
• the track results will be combined into a joint paper, to be published in the proceedings of the Eurographics Workshop on 3D Object Retrieval;
• the description of the track and its results will be presented at the Eurographics Workshop on 3D Object Retrieval (April 6, 2014).