SHREC'18 track: Recognition of geometric patterns over 3D models

The aim of this SHREC'18 track is to evaluate the performance of automatic algorithms for the recognition of geometric patterns over surfaces. The goal is to identify if and where a given geometric pattern (represented as a query surface fully characterized by a single pattern) is located over the surface of a 3D model. We admit that more than one geometric pattern might be present on the surface of 3D models. The track is organized by IMATI CNR in collaboration with The Cyprus Institute. The SHREC'18 track: Recognition of geometric patterns over 3D models final report will appear, after revision, in the proceedings of the EUROGRAPHICS Workshop on 3D Object Retrieval (3DOR).

Dataset, ground truth and evaluation

The dataset of the 3D models comes from archeological artefacts adopted as use cases in the EU H2020 project GRAVITATE. The challenge is to recognize 5 different types of geometric patterns over a dataset of approximately 30 models.

Each query (a simple pattern) will be used to locate the geometric pattern in the remaining part of the database. For a given query, the goal of the track is to say, for each 3D model, if the pattern is present on that model and where. A 3D model can present one, two or more patterns. The relevance or non-relevance of the models for a given query, and where the pattern is located, will be established and annotated a priori with the help of expert archaeologists.

Apart from a single measure representing the probability that a pattern belongs or not to a 3D model, we also ask the participants to locate the region where the pattern is present. The performance of the algorithms will be evaluated using a recognition rate and an error measure based on the area correctly recognized. Participants can ask to limit the evaluation of their method to the recognition rate.

Registration and instructions for competitors
Each participant is requested to register to the track by sending an email to Silvia Biasotti (email: silvia.biasotti@ge.imati.cnr.it) or Elia Moscoso Thompson (email: elia.moscoso@ge.imati.cnr.it) with the subject "SHREC'18 track: Recognition of geometric patterns over 3D models". Then, an answer will be sent to each participant with the instructions to download the datasets. Then, evey participant is asked to:

• Download a the datasets (PLY/OFF files). The models are organized into two folders: the Query folder contains 8 triangle meshes and the Dataset folder has 30 triangle meshes;
• Run their methods. All the models in the Query folder must be used in turn as a query against all the models in the Dataset folder. All 3D models are represented via PLY files. The models in the Dataset folder are at the higest resolution we have; upong request we can provide simplified models (1M or 100K vertices) or models in the OFF format.
• Provide by the 14th of February 2018:
  • The output of their run(s). The output format is the following (we expect to receive both the outputs, if something is not available we can only partially evaluate the run(s) submitted)
  • A plain ASCII file containing the membership matrix for each run. Up to 3 membership matrices may be submitted, resulting from different algorithms or parameter settings. Entry (i,j) of the matrix is the probability that the pattern contained in the query i in the Query folder is presented over the model j in the Database folder. Probability values should range in the interval [0,1]. We expect the name of this matrix is something like: NameParticipant_runX.txt.

  • For each query model, a plain ASCII file containing the number of the model with a 'M' (e.g. M1) and the indices of triangles that form the region(s) with that geometric pattern in the meshes in the Dataset folder. Notice that the index of a triangle is supposed to correspond to the line that triangle holds in the list of faces of the model .ply file, starting to count from 1, (e.g. for the line starting with M1, the indices of the triangles correspond to position these triangles hold in the file 1.ply).
    For each run, we expect, in total 8 files. We suggest to name such a file NameParticipant_QY_runX.txt, where Y ranges between 1 and 8 and X can be 1, 2 or 3. For instance, if the pattern in the query model 1.ply is recognized by the run1 to be in the followinf dataset models: 1.ply (region identified by the triangles 2, 5, 6, 100, 150); 3.ply (region of triangles 23, 44, 1020, 15000); and 6.ply (region of triangles 83, 130, 456, 3030, 6000); we expect the output file will be named NameParticipant_Q1_run1.txt and the content will look like:

    {M1} {2, 5, 6, 100, 15}

    {M2}{}

    {M3}{23, 44, 1020, 15000}

    {M4}{}

    {M5}{}

    {M6}{83, 130, 456, 3030, 6000}

    ...

    For each file, we expect up to 30 lines. Lines corresponding to models without that specific pattern can be omitted.

  • The executables that generated the results submitted. We accept Windows, Linux or Mac executables (scripts are also acceptable), that should be provided along with all the possible dependencies (.dll, .so, other executables...). Alternatively, the source code may be provided, together with full compilation instructions and their dependencies: we accept C, C++ and Matlab code. If a participant won't submit any executable, we will evaluate his/her run(s) but we will mention this fact in the final report.

Further Information

• The evaluations will be done automatically;
• the track results will be combined into a joint paper, to be published in the proceedings of the Eurographics Workshop on 3D Object Retrieval;
• the description of the track and its results will be presented at the Eurographics Workshop on 3D Object Retrieval (April 15-16, 2018).