Paper Reading Notes (Deep): PartPose

1. Basic information and publication status

• Title: PartPose: Attentive 6D Pose Estimation by Focusing on Graspable Parts of Multi-Part Deformable Objects
• Publication: IEEE Robotics and Automation Letters, Vol. 10, No. 12, pp. 13082–13089, 2025
• DOI: 10.1109/LRA.2025.3627067
• Public author version: ResearchGate, with CC BY 4.0 stated on the page and manuscript
• Conference relationship: ICRA 2026 PaperCept lists a presentation, while DOI, volume, and publisher identify RA-L 2025 as the canonical publication. This site includes it in the ICRA 2026 journal-presentation appendix without adding it to proceedings counts.
• Reading mode: Deep Dive

2. Positioning in one sentence

PartPose reframes multipart deformable-object pose as finding a stable, graspable rigid ROI comparable between real images and CAD, then performing render-and-compare only there. Flexible cables, pouches, and strings no longer distort whole-object rigid matching.

2.1 Guide to the paper’s figures and tables

3. Problem setting and task boundaries

3.1 What pose is actually estimated?

Given test RGB-D $(I_i^t, I_i^d)$, object model $M$, and intrinsics $K$, estimate object-to-camera rigid transform ${}^{c_i}T_o$. Unlike standard rigid objects, some real surfaces deform and do not maintain rigid correspondence with $M$.

PartPose retains the object-model frame, selecting an ROI within it rather than defining a new deformation-dependent frame:

$$m=(m_x,m_y,m_z,m_d),$$

The first three parameters offset the ROI origin from the model origin; $m_d$ is its bounding-sphere diameter. Output remains the full model-frame pose, but local rigid evidence supplies most loss / ranking information.

3.2 Inputs, outputs, and priors

Stage	Input	Output	Assumption
Offline preparation	Scan / CAD, 5 training configurations, 8 calibrated views, reference point	Per-object ROI $m^*$	Known robot-camera poses and a stable rigid part
Single-frame initialization	Test RGB-D, intrinsics, reference image / point, $M$, $m^*$	Refined ${}^{c}T_o$	Transferable keypoint, visible ROI, valid depth
Temporal tracking	Current RGB-D, previous pose, $M$, $m^*$	Updated ${}^{c}T_o$	Single-hypothesis refinement covers interframe motion
Robot grasping	${}^{w}T_c$, ${}^{c}T_o$, predefined ${}^{o}T_g$	${}^{w}T_g$	Rigid part and grasp frame agree

Grasp pose is obtained by

$${}^{w}T_g = {}^{w}T_c,{}^{c}T_o,{}^{o}T_g$$

This evaluates an object frame usable for rigid-part grasping, rather than full reconstruction of deformation.

3.3 Difference from articulated / deformable pose

• No cable, string, or pouch shape / articulation prediction;
• No nonrigid correspondence field or deformable registration;
• Grasping is assumed to depend only on one rigid part, deliberately ignoring other regions.

This is both an advantage and a strict limit: rigid-part pose in multipart deformable objects, rather than generic deformable pose.

4. Research context

FoundationPose, SAM-6D, and FreeZe typically reference rigid meshes / reconstructions. Partial matching handles background / occlusion, but deforming target regions are not automatically distinguished from regions that should satisfy rigidity.

The contribution is a task-prior layer:

1. Semantic correspondences locate a user-specified graspable part;
2. A local mask restricts pose evidence;
3. Multiview consistency selects a suitable 3D ROI;
4. Existing pose refinement / ranking is reused.

It changes what FoundationPose sees rather than replacing it.

5. Method details

5.1 Reference keypoint transfer

One manually annotated reference point $k_r$ lies near the graspable ROI center. DINOv2 patch similarity produces test point $k_i^t$.

This is a spatial SAM prompt, rather than enough multipoint correspondences for PnP: low-cost semantics become local geometric observation.

Cross-instance tests transfer source points to same-category targets. Shared caps / labels help, whereas broadly functional cable-appliance categories may map to wrong parts.

5.2 Local ROI segmentation

SAM returns multiple masks; the first is used rather than jointly optimizing mask weights. Retain the largest contour / connected component, with image centroid $(u,v)$.

There are two ROI levels:

• 2D mask ROI: current graspable part in real RGB-D;
• 3D model ROI: renderable model region defined by $m=(m_x,m_y,m_z,m_d)$.

They must correspond under the correct pose for meaningful render-and-compare.

5.3 Translation initialization

Mean masked depth gives $Z$. Back-project with $(u,v)$ and $K$ for camera-frame position $p=(X,Y,Z)$, after retaining the largest mask component to suppress fragments.

This locates hypotheses near the right region without recovering full rotation. Invalid transparent / reflective depth, background leakage, and occlusion-biased mean depth are risks.

5.4 Offline Bayesian ROI optimization

During training, estimate poses from views $c_1,c_2$ and convert them to world coordinates using known robot-camera transforms:

$${}^{w}T_o^{(1)}={}^wT_{c_1}{}^{c_1}T_o,qquad {}^{w}T_o^{(2)}={}^wT_{c_2}{}^{c_2}T_o.$$

Reliable ROIs should produce consistent world poses. Use Euclidean translation difference and quaternion geodesic rotation distance:

$$d_R=2\arccos(|q_1\cdot q_2|).$$

The objective weights are $w_t=1.0$ and $w_r=0.1$. Optuna TPE runs 200 trials, usually converging within 50. Five configurations × eight views form four random pairs per configuration, 20 consistency observations total. One RTX 3090 Ti takes about 3.3 h.

This is a notable design: cross-view pose consistency weakly supervises ROI selection without pixelwise rigid-part labels.

5.5 Local render-and-compare

After translation initialization, FoundationPose generates hypotheses. For each:

1. Locate the model ROI with $m$;
2. Crop real images around $p$;
3. Render at the candidate pose and crop the same spatial range;
4. Predict pose corrections with refinement;
5. Select the final hypothesis by ranking.

Initialization uses 5 refinements. Tracking uses only the previous pose as one hypothesis and 2 refinements, substantially increasing speed.

6. Experiments and evidence

6.1 Acquisition

Item	Setting
Categories	Pouch drinks, cable-attached objects, string-like objects
Instances	3 per category, 9 total
Sensor	Intel RealSense D405 and ring light
Robot	UR5e and ROBOTIS HAND RH-P12-RN two-finger gripper
3D models	KEYENCE VL-700 scans
Working distance	About 30 cm
Training	5 configurations × 8 views per object
Testing	5 configurations × 27 views = 135 images per object
Grasping	20 trials per object, 180 total

Private data prevents independent failure inspection / reevaluation. This is a small controlled test of deformable attachments disrupting rigid matching, rather than a broad BOP benchmark.

6.2 Why not ADD / VSD?

ADD applies predicted / GT transforms to the same rigid surface. Actual deformable parts differ from scanned geometry, so surface distances penalize valid deformation alongside pose error.

The paper uses:

• translation error;
• quaternion rotational distance;
• Success at translation ≤30 mm / rotation ≤0.3 rad;
• Real robotic picking success.

These fit the problem, but thresholds are fairly loose and translation / rotation are scored separately. Direct BOP AR / ADD(-S) rankings are inappropriate.

6.3 Ground truth and risks

GT averages 5 estimates visually verified by overlays, rather than external motion capture. Five participants repeatedly annotate pouch drink 2, power strip, and beads:

• Mean difference: 2 mm / 0.03 rad;
• Maximum difference: 6 mm / 0.07 rad.

These are well below 30 mm / 0.3 rad, supporting stable success judgments. Still, manually aligned model-rendering GT can share biases with model-based matching; external mocap would be stronger.

6.4 Main results

Method	Translation success	Rotation success	Grasp success
PartPose	98.2%	96.4%	87.2%
PartPose-Train-Mask	97.4%	95.6%	Not reported
SAM-6D	52.5%	39.3%	Not reported
PerSAM-FoundationPose	47.9%	35.9%	22.8%
Training-Free-PartPose	40.9%	31.5%	Not reported
Any6D	21.6%	10.1%	Not reported

PartPose performs well across categories:

• Translation: pouch 96.3, cable 99.3, string 99.0;
• Rotation: pouch 91.1, cable 99.0, string 99.0;
• Grasping: pouch 83.3, cable 95.0, string 83.3.

PerSAM-FoundationPose string-like grasp success is only 3.3%, consistent with whole-object masks including strongly deforming strings in rigid alignment.

6.5 Ablations: what matters?

Removing ROI optimization drops translation / rotation success from 98.2 / 96.4 to 40.9 / 31.5. Local matching alone is insufficient; ROI center / scale must be chosen for consistency.

Extra SAM mask-weight training gives 97.4 / 95.6, slightly below default PartPose. Mean errors are:

• PartPose: $7.4\pm12.2$ mm, $0.2\pm0.4$ rad;
• Train-Mask: $8.4\pm17.8$ mm, $0.2\pm0.5$ rad.

A t-test at 5% significance finds no difference, so mask-weight training cannot be claimed beneficial.

Rotation-weight sensitivity: with $w_r\in\{0.01,0.03,0.1,0.3,1.0\}$, mean rotation error stays about 0.2 rad and translation means at 5–7 mm. No significant difference from $w_r=0.1$ is reported, supporting insensitivity within this range.

6.6 Cross-instance transfer

Semantic transfer retains high pouch / string success: 94.1 / 87.4 and 93.0 / 92.6 for translation / rotation. Cable objects fall to 51.1 / 48.9.

This is not simply cable difficulty: category semantics are inconsistent. In power-strip-to-mouse transfer, 86% of keypoints fall on the USB plug—a related visual / functional part, but not the intended rigid grasp ROI.

Geometric transfer is tested only on pouches, falling to 86.7 / 60.4. Scans retain deformable configurations, preventing direct reuse of source ROI frames; CAD-level consistency is needed across instances.

7. Runtime and deployment

Hardware: Intel i9-14900K + RTX 3090 Ti:

Stage	Time
keypoint transfer	0.20 s
ROI segmentation	0.02 s
hypothesis refinement + ranking	$1.7\pm0.3$ s
Total single-frame initialization	About 1.9 s
Initialized tracking	0.014 s / frame

Real-time 0.014 s applies to tracking with a previous pose, one hypothesis, and two refinements. First localization is not real-time; prolonged complete occlusion or large jumps require second-scale reinitialization.

8. Authors’ claims and site assessment

Claim	Evidence	Site assessment
Rigid ROIs suit mixed rigid / flexible objects better	Table II, overlays, no-ROI-training ablation	Strong support: large gains consistent with the mechanism
Better robot grasping	9 objects × 20 real trials	Broadly supported: controlled scenes and few objects
Transfer to unseen same-category instances	Three transfer categories in Table III	Partial: consistent semantics help; cables fail
Real-time tracking	0.014 s / frame	Conditional: initialized, single-hypothesis refinement
Robust objective weighting	Table IV and t-test	Supported for the three tested objects, without broad extrapolation

9. Assumptions and failures

1. A graspable rigid part exists: fully flexible targets lack a stable 6D frame.
2. ROI visibility: occlusion of the rigid part can remove all evidence after excluding flexible regions.
3. Local texture / semantics: weak texture impairs point transfer and matching.
4. Valid ROI depth: reflections, transparency, and edges corrupt translation initialization.
5. Reliable model ROI: different scanned deformation states hinder geometric transfer.
6. Limited lighting differences: rendering-light adaptation remains future work.
7. Task depends on rigid parts: cable topology / pouch-shape manipulation needs more than PartPose output.

10. Reproduction challenges

• Code and data are not public.
• DINOv2, SAM, FoundationPose, Optuna / TPE, and custom robot acquisition must be integrated.
• Accurate robot–camera extrinsics and training-view poses are essential to unbiased consistency objectives.
• ROI parameters, full SAM / prompt postprocessing details, and random seeds are not supplied.
• GT needs visual judgments and pose averaging, beyond running a network.

Begin with three diagnostics before full grasping:

1. Visualize points on intended rigid parts;
2. Check flexible-region leakage into the largest mask component;
3. Plot world-frame pose residuals over TPE trials across views.

11. Potential improvements

11.1 Multiple-ROI beliefs

One occluded / deformed ROI fails. Treat rigid parts as mixture components, dynamically weighted by visibility, render residuals, and covariance for cluttered warehouses.

11.2 Discover rigid, graspable parts automatically

Combine VLM / part segmentation / temporal rigidity: semantics identify handles, caps, or housings; multiview video establishes rigidity; grasp affordances add relevance. Remove manual points while avoiding purely semantic power-strip→USB-plug mismatches.

11.3 Link ROI choice to uncertainty

Fixed TPE optima minimize average inconsistency. A practical system should output covariance / multimodal posteriors, downweight occluded or invalid-depth ROIs, and trigger active camera views when needed.

11.4 Model-free representations to reduce CAD cost

The authors propose model-free extensions. Single / multiview 3DGS, TSDF, or point maps could replace CAD, but must not freeze flexible configurations into wrong geometry. CAD-free does not mean model-error-free.

11.5 Stronger task evaluation

Expand to:

• Larger object libraries and cross-warehouse lighting;
• Graded rigid-ROI occlusion;
• Transparent / reflective rigid parts;
• Relocalization after long tracking;
• Cable tangling, collisions, and dragging after grasping—failures involving the flexible parts.

12. Overall assessment

The main modeling insight is that a semantically whole object does not imply every pixel should obey one rigid pose loss.

A small structural change turns this into a pipeline supported by pose results, ablations, and real grasps. It remains a small controlled system needing CAD, depth, manual points, and offline acquisition. Automatic discovery, multiple-ROI beliefs, and active sensing are the next steps toward warehouse deployment.