Strength
What it is useful for
- Covers occlusion, symmetry, 3D accuracy, and image projection together.
- Aggregation over multiple thresholds is more stable than a single success threshold.
BOP / 6D localization aggregation · aggregate
BOP Average Recall
Computes VSD, MSSD, and MSPD recall at multiple thresholds, then aggregates them into BOP's localization score.
AR is not a single distance. When reading BOP tables, distinguish AR_VSD, AR_MSSD, AR_MSPD, AR_D, and cross-dataset AR_C; they represent different aggregation levels.
Principle
The formulas define the computation. The diagrams show whether the error concerns a 3D model, visible surfaces, the image plane, or aggregation across thresholds.
For each error function, first determine whether each predicted pose is correct at a given threshold, then compute recall over all annotated instances.
BOP addresses several application needs together, scoring visible surfaces, worst-case 3D deviation, and perceptible 2D deviation.
How to compute
Compute VSD, MSSD, and MSPD for each test instance.
Compute recall for each error at the BOP-defined thresholds, then obtain AR_e.
Average the three AR_e values to obtain dataset-level AR_D; when required, average over the core datasets to obtain AR_C.
Strength
Caveat
Paper atlas
Links are derived by normalizing the metric names used in each paper. Directly reported metrics are distinguished from metrics associated through a broader protocol.
CVPR · Anna Šárová Mikeštíková, Médéric Fourmy, Martin Cifka et al.
CVPR · Yuchen Che, Jingtu Wu, Hao Zheng et al.
CVPR · Liming Kuang, Yordanka Velikova, Mahdi Saleh et al.
AAAI · Dexin Zuo, Ang Li, Wei Wang et al.
3DV · Sebastian Jung, Leonard Klüpfel, Rudolph Triebel et al.
arXiv · Jongbin Lim, Taeyun Ha, Mingi Choi et al.
ICRA · Panagiotis Sapoutzoglou, Orestis Vaggelis, Athina Zacharia et al.
ICRA / IEEE RA-L · Francesco Di Felice, Alberto Remus, Stefano Gasperini et al.
IJCV · Jian Liu, Wei Sun, Hui Yang et al.
ECCV · Hongli Xu, Jiaqi Hu, Junwen Huang et al.
CVPR · Jianqi Chen, Biao Zhang, Xiangjun Tang et al.
CVPR · Wubin Shi, Shaoyan Gai, Feipeng Da
WACV · Sajjad Pakdamansavoji, Yintao Ma, Amir Rasouli et al.
CVPR · Taeyeop Lee, Bowen Wen, Minjun Kang et al.
CVPR · Sungphill Moon, Hyeontae Son, Dongcheol Hur et al.
IROS · Amir Hamza, Andrea Caraffa, Davide Boscaini et al.
ICRA · Momen Khalil, Vincent Dietrich, Slobodan Ilic
ICRA · Zijiang Zhang, Huimin Lu, Jintong Cai et al.
CVPR · Tuo Cao, Fei Luo, Jiongming Qin et al.
IROS · Thomas Pöllabauer, Michael Gasser, Tristan Wirth et al.
ICCV · Xinhang Liu, Jiawei Shi, Zheng Dang et al.
CVPR · Mengya Liu, Siyuan Li, Ajad Chhatkuli et al.
CVPR · Weijian Deng, Dylan Campbell, Chunyi Sun et al.
IROS · Kuanning Wang, Yuqian Fu, Tianyu Wang et al.
WACV · Peter Hönig, Stefan Thalhammer, Jean-Baptiste Weibel et al.
ICCV · Junwen Huang, Shishir Reddy Vutukur, Peter KT Yu et al.
WACV · Varun Burde, Assia Benbihi, Pavel Burget et al.
CVPR · Jaeguk Kim, Jaewoo Park, Keuntek Lee et al.
IROS · Teng Guo, Jingjin Yu
CVPR · Qingyuan Wang, Rui Song, Jiaojiao Li et al.
CVPR · Ming-Feng Li, Xin Yang, Fu-En Wang et al.
CVPR · Xingyu Liu, Gu Wang, Ruida Zhang et al.
IEEE TCSVT · Jianqiu Chen, Zikun Zhou, Mingshan Sun et al.
CVPR · Bowen Wen, Wei Yang, Jan Kautz et al.
ECCV · Evin Pınar Örnek, Yann Labbé, Bugra Tekin et al.
ECCV · Andrea Caraffa, Davide Boscaini, Amir Hamza et al.
CVPR · Sungphill Moon, Hyeontae Son, Dongcheol Hur et al.
CVPR · Van Nguyen Nguyen, Thibault Groueix, Mathieu Salzmann et al.
CVPR · Yongliang Lin, Yongzhi Su, Praveen Nathan et al.
WACV · Yangzheng Wu, Michael Greenspan
CVPR · Junwen Huang, Hao Yu, Kuan-Ting Yu et al.
CVPR · Yuelong Li, Yafei Mao, Raja Bala et al.
3DV · Shishir Reddy Vutukur, Heike Brock, Benjamin Busam et al.
IROS · Francesco Milano, Jen Jen Chung, Hermann Blum et al.
CVPR · Jaime Corsetti, Davide Boscaini, Changjae Oh et al.
ECCV · Yang You, Kai Xiong, Zhening Yang et al.
ECCV · Yangzheng Wu, Michael Greenspan
CVPR · Jiehong Lin, Lihua Liu, Dekun Lu et al.
ICRA · Philipp Ausserlechner, David Haberger, Stefan Thalhammer et al.
CoRL · Yann Labbé, Lucas Manuelli, Arsalan Mousavian et al.
CVPR · Gu Wang, Fabian Manhardt, Federico Tombari et al.
ECCV · Yann Labbé, Justin Carpentier, Mathieu Aubry et al.
CVPR · Tomas Hodan, Daniel Barath, Jiri Matas
Next
Compare metrics at the same level to understand why one score improves while another stays unchanged.
Visible Surface Discrepancy
Maximum Symmetry-Aware Surface Distance
Maximum Symmetry-Aware Projection Distance
BOP Average Precision for 6D Detection