Object Pose Estimation · 2025 · ICCV · CCF A

6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

Incremental 2D Gaussian Splatting jointly optimizes object pose and representation. Dynamic keyframe selection, graph optimization, and pruning based on opacity statistics support online model-free RGB-D 6D tracking and reconstruction.

Paper

Research area

Object Pose Estimation

Task level

Instance-level

Object setting

Unseen objects

Input modalities

RGB-Dvideo

Requirements

object mask segmentationcamera intrinsicsRGB-D streamonline keyframe tracking

Full paper metadata Authors, affiliations, research area, datasets, metrics, method tags, and reading status Expand

Authors: Yufeng Jin, Vignesh Prasad, Snehal Jauhri, Mathias Franzius, Georgia Chalvatzaki
Affiliations: Technische Universität Darmstadt / Honda Research Institute Europe / Hessian.AI
Conference / journal: Conference · ICCV · CCF A
Research area: Object Pose Estimation
Code availability: Unavailable / not yet recorded
Reading status: Overview:Available Deep Dive:Available
Task setting: Instance-level · Unseen objects
Datasets: HO3D YCBInEOAT
Evaluation metrics: ADD ADD-S Rotation Error Translation Error
Method tags: Novel-object Pose Estimation Synthetic Data & Foundation Models Pose Refinement Multi-object Pose Estimation Absolute Pose Estimation
Requirements: object mask segmentationcamera intrinsicsRGB-D streamonline keyframe tracking

Reading Notes

Overview / Deep Dive

Available src/paper-notes/6dope-gs-iccv2025/coarse.md

Paper overview notes

Quickly assess the paper's scope, problem setting, core method, experimental findings, and whether to read it in depth.

Overview Notes: 6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

1. Basic information

Year / venue: 2025 / ICCV; reading mode: coarse.
Task: Model-free online 6D object tracking and reconstruction.
Input / output: One RGB-D video stream → per-frame object poses, a keyframe pose graph, and a renderable Gaussian Object Field.
Original paper: ICCV Open Access.

2. Positioning in one sentence

6DOPE-GS replaces slow neural fields with incremental 2D Gaussian Splatting and jointly optimizes object representations and keyframe poses during tracking, aiming to run CAD-free RGB-D object tracking and reconstruction online.

2.1 Guide to the paper’s figures and tables

Figure / table	What to examine
Figure 2	How coarse matching, the keyframe graph, Gaussian reconstruction, and pose re-optimization form a closed loop.

6DOPE-GS Figure 2: online tracking and reconstruction pipeline — Original paper, Figure 2: The online 6DOPE-GS pipeline. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; shown here solely for paper analysis.

3. Method overview

A SAM2 mask defines the object in the first frame; LoFTR correspondences and RANSAC produce coarse keyframes.
Dynamic keyframe selection considers spatial coverage and reconstruction confidence to filter erroneous pose updates.
A 2D Gaussian Object Field differentiably renders RGB, depth, and pseudo-normals; optimization alternates between Gaussians and keyframe poses.
Online pose-graph optimization combines constraints from new keyframes. CAD is unnecessary, but reliable segmentation, RGB-D, and sufficient viewpoint variation remain essential.

4. Experimental overview

Datasets: HO3D and YCBInEOAT, plus demonstrations on real dynamic objects.
Metrics: ADD / ADD-S, rotation / translation error, and tracking frequency.
Authors’ claim: Comparable accuracy to model-free baselines with approximately 5× speedup, running at around 3.5 Hz on real video.
My analysis: The central value is a shared online state for reconstruction and pose-graph optimization. However, 3.5 Hz is insufficient for a high-frequency robot control loop, and initial segmentation, hand occlusion, and rapid motion remain practical bottlenecks.

5. Strengths, Risks, and Insights

Strengths: No CAD, an explicit Gaussian representation, and erroneous-keyframe filtering built into the system.
Risks: Requires RGB-D, a single-object mask, and continuous observations; it does not directly replace single-frame novel-object pose estimation.
Insight: 3DGS can serve as intermediate geometry for pose refinement, making when to update the model as important as pose itself.

Available src/paper-notes/6dope-gs-iccv2025/deep.md

In-depth paper notes

Support reproduction, comparison, related-work writing, and identifying opportunities for further research.

Deep Reading Notes: 6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

1. Basic information

Authors / affiliations: Yufeng Jin et al.; Technische Universität Darmstadt, Honda Research Institute Europe, Hessian.AI.
Year / venue: 2025 / ICCV; reading mode: deep.
Paper link: ICCV Open Access.

2. Positioning in one sentence

For RGB-D video without prebuilt CAD, 6DOPE-GS jointly maintains object appearance and current pose through online 2D Gaussian Splatting and an object-level pose graph, targeting slow rendering and updates in model-free reconstruction/tracking.

2.1 Guide to the paper’s figures and tables

6DOPE-GS Figure 2: pipeline — Original paper, Figure 2: An online loop connecting coarse initialization, dynamic keyframes, the Gaussian Object Field, and a pose graph. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; shown here solely for paper analysis.

3. Problem setting and task boundaries

Input: A single RGB-D video, camera intrinsics, and a target-object mask; initialize the object in the first frame and observe it continuously thereafter.
Output: An object $SE(3)$ pose for each keyframe, an online-updated 2D Gaussian Object Field, and a pose graph.
Not required: CAD, offline object scans, or target-specific training.
Required: Sufficient visible surfaces / viewpoint variation, reliable masks, and RGB-D quality. Object-level tracking here does not establish open-world multi-object detection.

4. Method details

Module	Input	Output	Role
Coarse initialization	RGB-D, masks, LoFTR matches	Initial keyframe poses	RANSAC establishes the first trackable views
Dynamic keyframe selection	Candidate frames, coverage, confidence	Trusted keyframes	Rejects erroneous updates that would contaminate the online representation
Gaussian Object Field	Keyframes, RGB/depth/normal supervision	2D Gaussians	Provides an object representation for fast differentiable rendering
Alternating optimization	Gaussians, keyframe poses	Refined field / poses	Optimizes one component with the other fixed to reduce coupling
Pose graph optimization	Relations between new and existing keyframes	Current-frame pose	Updates final tracking results using global consistency

The contribution is the interaction of incremental 2DGS and a pose graph: the field aligns rendering residuals, while the graph corrects accumulated local drift.
Opacity-percentile pruning removes low-contribution Gaussians; the authors group it with erroneous-keyframe filtering as online efficiency / stability design.

5. Experimental analysis

Datasets: YCBInEOAT and HO3D, covering real RGB-D conditions for object tracking/reconstruction and handheld objects, respectively.
Metrics: ADD, ADD-S, rotation / translation error, and tracking frequency.
Authors’ claim: Comparable accuracy to model-free simultaneous tracking/reconstruction baselines with around 5× speedup; real demonstrations run at approximately 3.5 Hz.
My analysis: The experiments support a faster representation, but deployment reliability needs separate lost-track rates for first-frame mask failures, rapid motion, and extensive hand occlusion.

6. Critical Analysis and Reproduction

Strengths: Model-free estimation with explicit, renderable object states; keyframe coverage / confidence filtering is practically useful.
Hidden assumptions: SAM2 masks, LoFTR correspondences, and depth adequately support initialization. Errors in any stage propagate into subsequent graph optimization.
Engineering challenges: Joint tuning of the 2DGS differentiable renderer, RGB/depth/normal loss weights, keyframe thresholds, and GPU memory.
Possible improvements: Integrate uncertainty-aware pose regions into keyframe selection, explicitly model hand-object occlusion, and reduce the 3.5 Hz front-end bottleneck with faster segmentation or object-centric tracking.

7. Research Insights

For dynamic novel objects, object-model construction can become continuous state estimation rather than an offline scan preceding pose estimation.
Suitable as a model-free RGB-D tracking baseline, but not as a generalizable 6D pose baseline receiving only one RGB image.

8. Additional Experimental Figures from the Paper

6DOPE-GS main results and speed–accuracy analysis on YCBInEOAT and HO3D — Excerpt from the paper's experimental pages: Main tracking / reconstruction results on YCBInEOAT and HO3D, and the relationship between online optimization frequency, accuracy, and speed. Source: ICCV 2025 paper PDF. Image copyright belongs to the authors and publisher; shown here solely for paper analysis.

Read these results along two axes: ADD / ADD-S tables indicate that the Gaussian Object Field gains speed without a clear loss in pose accuracy, while curves show diminishing returns from more keyframe updates and optimization iterations. The supported engineering claim is a better speed–accuracy trade-off within an online-modeling budget. It does not establish stable 3.5 Hz operation under arbitrary rapid motion; occlusion-related relocalization and front-end mask costs still require separate evaluation.

Related papers

2026 · CVPR

3PT

3PT unifies detection, segmentation, and 6-DoF pose with two Transformers for 2D / 3D object-centric understanding. Continuous refinement of object representations and rendered-feature alignment support zero-shot cross-object perception on BOP-H3, BOP-Industrial, and BOP-Classic.

2026 · AAAI

6DAttack

Studies backdoor attacks on PnP-based and end-to-end 6-DoF pose pipelines. Structurally 3D triggers and modified pose labels are injected into training data so that models behave normally on clean inputs but produce controlled incorrect poses when triggered.

2026 · ICRA

Active Tactile Exploration

A violation-implicit loss incorporates physical-constraint violations into joint estimation of freely moving unknown convex rigid-body trajectories and cuboid / 20-vertex polytope shapes. Observed / Fisher information at a local MLE estimates EIG to select the next touch. In simulation, the EIG strategy yields stable errors below 4 mm after 5 cuboid actions; real Trifinger results at step 5 substantially outperform random selection. Convex geometry, local optima, multiple initializations, and tactile / proprioceptive errors limit the method.