Object Pose Estimation · 2025 · ICCV · CCF A

SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Image

The method retrieves and fits a deformable SDF from a single image, jointly optimizing shape and pose in a render-and-compare loop. Foundation features initialize 2D–3D alignment, supporting generalization without retraining to occlusion, unusual poses, and in-the-wild images.

Paper Project

Research area

Object Pose Estimation

Task level

Category-level

Object setting

Unseen objects

Input modalities

RGB

Requirements

category-specific morphable SDF priorcamera intrinsicsfoundation featurestest-time optimization

Full paper metadata Authors, affiliations, research area, datasets, metrics, method tags, and reading status Expand

Authors: Dimitrije Antić, Georgios Paschalidis, Shashank Tripathi, Theo Gevers, Sai Kumar Dwivedi, Dimitrios Tzionas
Affiliations: University of Amsterdam / Max Planck Institute for Intelligent Systems
Conference / journal: Conference · ICCV · CCF A
Research area: Object Pose Estimation
Code availability: Code available
Reading status: Overview:Available Deep Dive:Available
Task setting: Category-level · Unseen objects
Datasets: Pix3D Pascal3D+ COMIC
Evaluation metrics: Chamfer Distance
Method tags: Category-level Pose Estimation 2D–3D Correspondences Pose Refinement Synthetic Data & Foundation Models Absolute Pose Estimation
Requirements: category-specific morphable SDF priorcamera intrinsicsfoundation featurestest-time optimization

Reading Notes

Overview / Deep Dive

Available src/paper-notes/sdfit-iccv2025/coarse.md

Paper overview notes

Quickly assess the paper's scope, problem setting, core method, experimental findings, and whether to read it in depth.

Overview Notes: SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Image

1. Basic information

Year / venue: 2025 / ICCV; reading mode: coarse.
Task: jointly recover category-level 3D object pose and shape from a single RGB image.
Paper: ICCV Open Access; project: SDFit.

2. Positioning in one sentence

SDFit constrains shape search to a valid learned morphable-SDF manifold, initializes 2D–3D correspondences with foundation features, then jointly iterates shape and pose through render-and-compare.

2.1 Guide to the paper’s figures and tables

Figure / table	What to inspect
Figure 2	Single-image input, morphable-SDF latent prior, differentiable fitting, and final mesh / pose.

SDFit Figure 2: morphable SDF fitting pipeline — Original Figure 2: the high-level SDFit pipeline. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

3. Method overview

Cross-modal foundation-model retrieval selects an initial mSDF latent from a shape database; image / mSDF feature correspondences initialize pose.
Render-and-compare losses on image masks, normals, and depth-like features iteratively update the shape latent, pose, and scale.
DIT mSDF provides a category-specific deformable shape prior. It needs no retraining for each in-the-wild image, while retaining a category model rather than supporting prior-free open-world pose.

4. Experimental overview

Datasets: Pix3D, Pascal3D+, COMIC; comparisons emphasize occlusion, common / unusual poses, and image alignment.
Authors’ claim: comparable to feed-forward baselines without substantial occlusion, more robust under occlusion and unusual poses, and applicable to natural images without retraining.
My analysis: this approach explicitly optimizes against visual evidence; quality and speed depend on the shape prior, initialization, and test-time optimization. It is unsuitable as a real-time 6D pose system.

5. Strengths, weaknesses, and takeaways

Strengths: joint shape / pose refinement, some flexibility for topology changes, and an interpretable feedback loop.
Risks: category mSDFs, intrinsics / visual cues, and minute-scale optimization budgets are required; cluttered multi-object scenes are outside the main setting.
Takeaway: explicit shape manifolds remain valuable alongside single-pass regression for occlusion and OOD images.

Available src/paper-notes/sdfit-iccv2025/deep.md

In-depth paper notes

Support reproduction, comparison, related-work writing, and identifying opportunities for further research.

Deep Reading Notes: SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Image

1. Basic information

Authors / affiliations: Dimitrije Antić et al.; University of Amsterdam, Max Planck Institute for Intelligent Systems.
Year / venue: 2025 / ICCV; reading mode: deep.
Links: ICCV Open Access / project.

2. Positioning in one sentence

SDFit treats single-image object pose and shape as analysis by synthesis: retrieval / matching initializes a category-specific morphable SDF, then rendered comparisons jointly fit latent shape, pose, and scale within a valid shape manifold.

2.1 Guide to the paper’s figures and tables

SDFit Figure 2: render-and-compare pipeline — Original Figure 2: mSDF prior, image cues, and the SDFit optimization loop. Source: ICCV Open Access. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

3. Problem setting and boundaries

Input: a single RGB image, from which masks, normals, depth-like cues, and foundation features are extracted.
Output: 3D object shape, camera / object pose, and scale. The focus is category-level shape and pose, rather than BOP instance-CAD alignment.
Key priors: a per-category mSDF / DIT latent-shape model and a searchable 3D shape database; avoiding retraining still requires category priors.

4. Method details

Module	Input	Output	Role
Shape retrieval	Image foundation features, 3D database	Initial mSDF latent	Seed optimization near a suitable shape basin
Pose initialization	Image / mSDF feature correspondences	Initial pose	Reduce local minima caused by random pose initialization
Morphable SDF	Latent code	Renderable shape / normals / depth	Constrain search to a feasible shape manifold
Render-and-compare	Observed versus rendered cues	Pose / shape update	Improve pixel alignment and geometric consistency jointly

mSDF handles within-category shape variation and changing topology; foundation features align textured images with implicit surfaces.
Authors’ claim: joint shape and pose refinement is the key distinction from render-and-compare on a fixed shape.

5. Experimental analysis

Pix3D, Pascal3D+, and COMIC cover natural images, category variation, and synthetic occlusion tests; results include shape reconstruction, image alignment, and pose.
In the image-alignment table, SDFit-feat@Sinit reaches mean 2D IoU 85.4, exceeding listed shape + RnC baselines. The authors report about 3 minutes to converge fully and satisfactory results after roughly 45–60 seconds.
My analysis: evidence supports occlusion robustness and operation without retraining for new images, but runtime favors valuable offline analysis / AR content creation over per-frame real-time pose.

6. Critical analysis and reproduction

Strength: shape priors, feature correspondences, and explicit optimization share one feedback loop, making failures easier to diagnose.
Limitations: incorrect retrieval, biased masks / normals, or out-of-category objects can trap optimization in the wrong shape manifold; intrinsics also affect pixel alignment.
Reproduction priorities: mSDF checkpoints, shape-database indexing, differentiable rendering, cue-loss weights, and optimization schedules.
Extensions: maintain multiple latent / pose hypotheses through uncertainty; combine fast feed-forward seeds with brief SDFit refinement.

7. Research takeaways

For occlusion and unusual poses, learning a plausible shape set before pose optimization remains a useful alternative alongside pure regression.

8. Additional original experimental figures

SDFit shape reconstruction, occlusion sensitivity, and shape-decoration runtime — Excerpts from original Tables 1–3, Figures 7–8, and adjacent text: Pix3D / COMIC shape reconstruction, shape-decoration runtime, and Chamfer Distance across occlusion levels. The current excerpt contains no rotation errors. Source: ICCV 2025 paper PDF. Image copyright belongs to the authors and publisher; reproduced here for paper analysis.

This page directly supports shape-reconstruction and occlusion-robustness analysis, with shape-decoration computational budgets. It contains no rotation metric, so it cannot establish rotation accuracy from pose initialization or refinement. Joint fitting also requires separate pose / image-alignment experiments. High 2D IoU alone may reflect silhouette alignment without accurate depth, scale, or rotation around symmetry axes.

Related papers

2026 · CVPR

3PT

3PT unifies detection, segmentation, and 6-DoF pose with two Transformers for 2D / 3D object-centric understanding. Continuous refinement of object representations and rendered-feature alignment support zero-shot cross-object perception on BOP-H3, BOP-Industrial, and BOP-Classic.

2026 · AAAI

6DAttack

Studies backdoor attacks on PnP-based and end-to-end 6-DoF pose pipelines. Structurally 3D triggers and modified pose labels are injected into training data so that models behave normally on clean inputs but produce controlled incorrect poses when triggered.

2026 · ICRA

Active Tactile Exploration

A violation-implicit loss incorporates physical-constraint violations into joint estimation of freely moving unknown convex rigid-body trajectories and cuboid / 20-vertex polytope shapes. Observed / Fisher information at a local MLE estimates EIG to select the next touch. In simulation, the EIG strategy yields stable errors below 4 mm after 5 cuboid actions; real Trifinger results at step 5 substantially outperform random selection. Convex geometry, local optima, multiple initializations, and tactile / proprioceptive errors limit the method.