Evaluating the performance of Structure from Motion Pipelines

Structure from Motion (SfM) is a pipeline that allows three-dimensional reconstruction starting from a collection of images. How to choose the SfM pipeline best suited for a given task? We report a comparison of different state-of-the-art SfM pipelines in terms of their ability to reconstruct different scenes. We also propose an evaluation procedure that stresses the SfM pipelines using real dataset acquired with high-end devices as well as realistic synthetic dataset. To this end, we created a plug-in module for the Blender software to support the creation of synthetic datasets and the evaluation of the SfM pipeline. The use of synthetic data allows us to easily have arbitrarily large and diverse datasets with, in theory, infinitely precise ground truth. Our evaluation procedure considers both the reconstruction errors as well as the estimation errors of the camera poses used in the reconstruction.

DATASET

The dataset used in our paper is shown in the figure below. Models a-e are syntetic models. Model f is a real statue.

Click here to download the dataset : 

  IVL-SYNTHSFM.zip (298.6 MiB, 6 hits)

If you use this database, please cite the following paper:

@article{bianco-sfm2018,
 author = {Bianco, Simone and Ciocca, Gianluigi and Marelli, Davide},
 year = {2018},
 title = {Evaluating the Performance of Structure from Motion Pipelines},
 volume = {4},
 number = {8},
 journal = {Journal of Imaging},
 doi = {10.3390/jimaging4080098}
}

Publications

1.

Evaluating the Performance of Structure from Motion Pipelines
(Simone Bianco, Gianluigi Ciocca, Davide Marelli) In Journal of Imaging, volume 4, number 8, 2018.

@article{bianco-sfm2018,
 author = {Bianco, Simone and Ciocca, Gianluigi and Marelli, Davide},
 year = {2018},
 title = {Evaluating the Performance of Structure from Motion Pipelines},
 volume = {4},
 number = {8},
 journal = {Journal of Imaging},
 pdf = {/download/bianco-sfm2018.pdf},
 doi = {10.3390/jimaging4080098},
 projectref = {http://www.ivl.disco.unimib.it/activities/evaluating-the-performance-of-structure-from-motion-pipelines/}}
2.

A Blender plug-in for comparing Structure from Motion pipelines
(Simone Bianco, Gianluigi Ciocca, Davide Marelli) In International Conference on Consumer Electronics - Berlin (ICCE-Berlin), pp. -, 2018.

@inproceedings{bianco2018blender-plugin,
 author = {Bianco, Simone and Ciocca, Gianluigi and Marelli, Davide},
 year = {2018},
 pages = {-},
 title = {A Blender plug-in for comparing Structure from Motion pipelines},
 organization = {IEEE},
 booktitle = {International Conference on Consumer Electronics - Berlin (ICCE-Berlin)}}