ECCV 2024 · Paper Collection
ECCV 2024 object pose research expands in two directions: foundation features, training-free matching, and self-supervised domain adaptation reduce dependence on individual objects and real annotations, while pose tasks broaden to large vocabularies, articulated and flexible objects, video tracking, and joint pose–shape–reconstruction states.
16
Core papers included
2
Official Oral
4
Unseen objects and cross-domain generalization
4
Data and benchmark papers
We screened the titles of all 2,387 unique paper records in the official ECCV 2024 virtual directory, then checked the abstracts, task outputs, and experimental subjects of related candidates, including 16 papers.
View the official ECCV 2024 paper directory →Collection in One Sentence
FoundPose, FreeZe, and GS-Pose incorporate foundation features into 6D pose by improving template retrieval, local correspondence, and 3D semantic representations, then applying interpretable geometric solvers.
Omni6D, OmniNOCS, Omni6DPose, and PACE expand categories, instances, and dynamic scenes. U-COPE, OP-Align, flexible surgical instruments, CODERS, and SUP-NeRF extend outputs to parts, shapes, and reconstruction states.
Research Trends
Trends summarize technical changes and subsequent influence. Papers may have multiple attributes, but counts assign each to one group by primary contribution.
01 · Foundation Correspondence
FoundPose uses DINOv2 for template retrieval and 2D–3D correspondence; FreeZe fuses visual and local geometric descriptors; GS-Pose lifts 2D semantics onto object 3D surfaces. All retain explicit geometric estimation.
02 · Vocabulary Scaling
Omni6D spans 166 categories, OmniNOCS expands indoor and outdoor NOCS, and Omni6DPose combines large-scale real and synthetic data. Scale also exposes long-tail categories, symmetry, and cross-category transfer challenges.
03 · Beyond Rigid Pose
U-COPE and OP-Align recover object and part states, flexible-instrument work predicts continuous bending, and CODERS and SUP-NeRF jointly estimate pose, shape, and reconstruction, bringing representations closer to real robotic manipulation targets.
04 · Dynamic & Domain Shift
PACE and Omni6DPose include tracking in benchmarks; RKHSPose and FAFA adapt with unlabeled real data; CODERS uses stereo for transparent and highly reflective materials; FastCAD advances retrieval and alignment to approximately 10 FPS.
Data Statistics
The 16 papers are grouped exclusively by primary contribution; tags, trends, and summaries describe cross-area techniques.
Addresses within-category deformation, scale, and cross-category generalization through uncertain shapes, geometric–semantic correspondences, and large-scale NOCS supervision.
Reduces real pose annotations and object-specific training through foundation-model features, geometric descriptors, self-supervised adaptation, and training-free matching.
Extends rigid pose to part states, flexible instruments, stereo reconstruction, and object-centric NeRF, recovering pose, shape, and structure jointly.
Advances scalable evaluation and deployment through long-tail category datasets, real cluttered videos, general-purpose pose tracking, and real-time CAD alignment.
9 / 16
Unseen-object generalization and articulation / reconstruction together account for 56.3%, highlighting parallel efforts to reduce object priors and expand output states.
4 papers
Primarily contribute data and benchmarks, advancing large-vocabulary NOCS / 6D pose, real cluttered video, and universal tracking.
2 papers
OP-Align and OmniNOCS appear in the official ECCV 2024 Oral schedule; the other included papers are verified as Posters.
Recommended Reading
Official Oral designations come from the ECCV 2024 schedule. Other recommendations are editorial selections based on methodological representativeness, dataset value, and reproducible analysis.
Lifts foundation-model 2D semantics onto object 3D surfaces and combines them with local geometry for correspondence matching, providing a clear example of semantic foundation models paired with geometric estimation.
Official OralExpands NOCS to large-vocabulary indoor and outdoor objects, with a unified model and new evaluation. One of two official Orals in this ECCV 2024 collection.
Priority readingConnects template retrieval, local matching, and 2D–3D geometric estimation through intermediate DINOv2 features, clearly representing the foundation-feature approach to unseen-object pose in 2024.
Representative generalization approachCombines local geometric descriptors and visual foundation-model features into 3D point correspondences, then applies classical robust registration, offering an interpretable baseline with little dependence on task-specific training.
Official OralOrganizes learning without pose labels through object-level and part-level alignment, and introduces real articulated-object evaluation. One of two official Orals in this ECCV 2024 collection.
Representative datasetWith 166 categories, 4,688 instances, and over 800K observations, substantially expands category-level 6D pose vocabulary and incorporates symmetry into unified evaluation.
Representative datasetBrings dynamic clutter, occlusion, and articulation states into a sustained evaluation resource with 300 real videos, 238 objects, and 258K pose annotations.
Representative systemROPE, SOPE, and GenPose++ combine broad object coverage, dynamic tracking, and generative pose inference in one data–model framework.
Paper List
Each card gives the task setting, input modality, presentation type, and an abstract-checked summary. Links point to official ECVA pages until site detail pages are complete, then switch to the site's reading notes.
3 papers
Addresses within-category deformation, scale, and cross-category generalization through uncertain shapes, geometric–semantic correspondences, and large-scale NOCS supervision.
Ruida Zhang · Ziqin Huang · Gu Wang · Chenyangguang Zhang · Yan Di · Xingxing Zuo · Jiwen Tang · Xiangyang Ji
Setting: Joint category-level rotation, translation, and size from one RGB image, explicitly modeling uncertainty in instance shape
Input: RGB-D + CAD model
Summary: Represents shape distributions with a Laplacian mixture instead of one deterministic category template. The network estimates shape and correspondences from image evidence, then uses scale-independent geometric estimation to recover pose and dimensions, addressing within-category variation and scale–depth coupling.
Pengyuan Wang · Takuya Ikeda · Robert Lee · Koichi Nishiwaki
Setting: Trains with synthetic geometric supervision alone and estimates category-level pose on unseen real instances
Input: RGB-D / point cloud
Summary: Projects pretrained visual-model 2D patch features into semantic 3D point clouds and jointly matches them with geometric descriptors to synthetic partial observations. Training requires neither real pose labels nor fine-tuning; NOCS, Wild6D, and SUN RGB-D test transfer from synthetic objects to unseen real instances.
Akshay Krishnan · Abhijit Kundu · Kevis Maninis · James Hays · Matthew Brown
Setting: Uses 2D detection boxes as prompts to predict NOCS, instance masks, 3D pose, and shape across broad indoor and outdoor categories
Input: RGB + 2D boxes
Summary: Builds a large-scale unified dataset with monocular NOCS, object masks, and 3D bounding boxes, offering approximately 20 times as many categories and 200 times as many instances as existing NOCS data. NOCSformer predicts cross-category NOCS, masks, and poses from 2D box prompts.
4 papers
Reduces real pose annotations and object-specific training through foundation-model features, geometric descriptors, self-supervised adaptation, and training-free matching.
Yangzheng Wu · Michael Alan Greenspan
Setting: Transfers from labeled synthetic data to unlabeled real data using pseudo-keypoint self-supervision for instance-level 6DoF pose
Input: RGB
Summary: Forms pseudo-keypoints from source-model predictions on real images and constrains cross-domain structure in a reproducing kernel Hilbert space to reduce damage from noisy pseudo-labels. Evaluates self-supervised sim-to-real transfer on LINEMOD, LM-O, YCB-V, and multiple BOP datasets.
Evin Pınar Örnek · Yann Labbé · Bugra Tekin · Lingni Ma · Cem Keskin · Christian Forster · Tomas Hodan
Setting: Given query RGB, an object mask, and CAD, estimates 6D pose without target-specific training
Input: RGB + CAD model + object mask
Summary: Renders CAD templates offline and extracts DINOv2 patch descriptors, retrieves nearby viewpoints through a visual bag of words, then establishes query-image-to-model-surface 2D–3D correspondences to solve pose. Optional refinement improves accuracy in the BOP unseen-object setting.
Andrea Caraffa · Davide Boscaini · Amir Hamza · Fabio Poiesi
Setting: Given RGB-D, an object mask, and a 3D model, estimates unseen-object pose entirely without training
Input: RGB-D + 3D model + object mask
Summary: Extracts local geometry and visual foundation features separately, lifting image features onto observed point clouds. Fused model–scene correspondences recover SE(3) through robust registration such as RANSAC, with zero-shot generalization evaluated on seven BOP core datasets.
Jingyi Tang · Gu Wang · Zeyu Chen · Shengquan Li · Xiu Li · Xiangyang Ji
Setting: After synthetic training, adapts self-supervised to real underwater video for instance-level 6D pose of UUV manipulation targets
Input: RGB video + 3D model
Summary: Uses frequency-aware augmentation to reduce appearance differences between synthetic and real underwater images, and derives temporal self-supervision from adjacent-frame optical flow. Aims to improve UUV object pose in turbid, color-shifted, scattering environments without real pose labels.
5 papers
Extends rigid pose to part states, flexible instruments, stereo reconstruction, and object-centric NeRF, recovering pose, shape, and structure jointly.
Li Zhang · Weiqing Meng · Yan Zhong · Bin Kong · Mingliang Xu · Jianming Du · Xue Wang · Rujing Wang · Liu Liu
Setting: One model handles rigid and articulated categories, recovering object or part rotation, translation, and dimensions
Input: RGB-D / point cloud
Summary: Unifies rigid and articulated 9D category-level pose through per-part point-pair features, part voting, and joint constraints, allowing one representation to cover both object-level and part-level states.
Yiyao Ma · Kai Chen · Hon-Sing Tong · Ruofeng Wei · Yui-Lun Ng · Ka-Wai Kwok · Qi Dou
Setting: Estimates external orientation and bending configuration of flexible robotic instruments from a single endoscopic image
Input: endoscopic RGB
Summary: Uses instrument-shape priors to guide image features and explicitly learns the coupling between orientation and bending. Deformable shape refinement extends rigid pose to continuously bending tool states for robotic perception in confined surgical views.
Yuchen Che · Ryo Furukawa · Asako Kanezaki
Setting: Joint object-level and part-level poses of category-level articulated objects from a single partial point cloud
Input: point cloud
Summary: Mutually constrains overall and part alignment in canonical space to learn articulated-object states without real pose labels. Outputs object-level and part-level poses and uses a new real benchmark to test generalization from synthetic training to real observations.
Yuliang Guo · Abhinav Kumar · Cheng Zhao · Ruoyu Wang · Xinyu Huang · Liu Ren
Setting: Jointly optimizes category-object dimensions, 6D pose, and object-centric NeRF reconstruction from driving videos
Input: monocular RGB video
Summary: SUP-NeRF decouples dimension estimation and pose refinement to reduce scale–depth ambiguity, using projected-box states for three feed-forward pose updates before object-centric NeRF. Results are 7.01° / 0.68m on nuScenes and 6.79° / 1.06m in KITTI zero-shot evaluation; 50 NeRF post-optimization steps do not improve pose.
Chuanrui Zhang · Yonggen Ling · Minglei Lu · Minghan Qin · Haoqian Wang
Setting: Joint detection, category-level pose, and shape reconstruction for everyday objects with diffuse, specular, transparent, and mixed materials
Input: stereo RGB
Summary: CODERS fuses stereo features and 3D positions through implicit stereo matching, then uses a unified decoder for end-to-end detection, pose, and reconstruction. It avoids monocular scale ambiguity and RGB-D depth failures on transparent or highly reflective surfaces, evaluating sim-to-real transfer on TOD and robotic manipulation experiments.
4 papers
Advances scalable evaluation and deployment through long-tail category datasets, real cluttered videos, general-purpose pose tracking, and real-time CAD alignment.
Mengchen Zhang · Tong Wu · Tai Wang · Tengfei Wang · Ziwei Liu · Dahua Lin
Setting: Category-level 6D pose evaluation across long-tail categories and large instance sets
Input: RGB-D
Summary: Releases a large-vocabulary dataset with 166 categories, 4,688 instances, and over 800K RGB-D observations, alongside a symmetry-aware pose metric. Reveals long-tail generalization and cross-category failures that small-category benchmarks struggle to expose.
Yang You · Kai Xiong · Zhening Yang · Zhengxiang Huang · Junwei Zhou · Ruoxi Shi · Zhou Fang · Adam Harley · Leonidas Guibas · Cewu Lu
Setting: Evaluates rigid and articulated object 6D pose estimation and tracking in real cluttered video
Input: RGB-D video
Summary: PACE contains 300 videos, 55K frames, and 258K pose annotations spanning 238 objects and 43 categories. Includes rigid and articulated objects for per-frame estimation and cross-frame tracking under clutter rather than staged static scenes.
Jiyao Zhang · Weiyao Huang · Bo Peng · Mingdong Wu · Fei Hu · Zijian Chen · Bo Zhao · Hao Dong
Setting: Unified 6D pose estimation and tracking evaluation across many categories and instances, using large-scale real and synthetic data
Input: RGB-D / video
Summary: Builds ROPE (332K real images, over 1.5M annotations, 581 instances) and SOPE (475K synthetic images, over 5M annotations, 4,162 instances), covering 149 categories, and unifies category-level 6D pose estimation and tracking through diffusion-based GenPose++.
Florian Langer · Jihong Ju · Georgi Dikov · Gerhard Reitmayr · Mohsen Ghafoorian
Setting: Jointly retrieves object CAD and recovers scale, rotation, and translation alignment from indoor RGB-D scans or video
Input: RGB-D scan / video + CAD database
Summary: Uses shared features for simultaneous CAD retrieval and upright 9DoF alignment, avoiding separate retrieval and offline optimization. Scan input achieves 61.7% alignment in 50 ms; the video pipeline achieves 48.2% at 10 FPS. Reconstruction accuracy of 29.6% uses the same restricted retrieval pool as Vid2CAD; the main open-library configuration achieves 24.7%.
Scope, Sources & Deduplication
Inclusion: official ECCV 2024 papers primarily outputting instance-level / category-level object 6D or 9D pose, object pose tracking, representations and correspondences directly supporting pose, pose benchmarks, or part states, shapes, and object-centric reconstructions tightly coupled with object pose.
Excluded: human, hand, and head pose; grasp pose alone; 3D detection without object orientation recovery; and systems using existing pose only as a downstream input. Harnessing Text-to-Image, Graph-Based CAPE, and SCAPE study 2D category-agnostic keypoints on MP-100; 6DGS estimates camera pose relative to a 3DGS scene; SpaRP estimates relative camera poses between sparse object images. These are excluded from core object-pose counts.
Boundary rules: SUP-NeRF jointly reconstructs NeRF but explicitly estimates and evaluates vehicle object pose, so it is included. FastCAD jointly performs CAD retrieval and 9DoF object alignment and belongs to the systems group. Generic point-cloud / NeRF registration without an object pose task and object-level evaluation is excluded.
Deduplication: a paper's arXiv, project page, and official ECCV version count once, assigned to the official ECCV publication. A future substantially extended journal version will be linked as a conference–journal extension without increasing the ECCV 2024 count.
Primary sources: Complete official ECCV 2024 directory, Official Oral schedule, ECVA Open Access archive and public author project materials. Summaries, task classifications, and trend judgments are editorial work by this site.
