Paper Reading Notes (Deep): IndustryShapes

1. Which gap does the dataset address?

Many 6D benchmarks use household groceries, toys, or regular boxes. Industrial components / tools more often involve:

• Large textureless metal / plastic surfaces;
• Specular / partly reflective surfaces;
• Sheets, fixtures, and elongated structures;
• Repeated geometry and rotational symmetry;
• Trays, holders, multiple instances, and occlusion at workstations;
• New parts requiring quick onboarding videos without retraining large instance networks.

IndustryShapes selects 5 industrial objects for these difficulties and splits into Classic known-instance training / testing and Extended novel / model-free estimation with CAD or static onboarding sequences.

2. Five objects and task boundaries

Each object represents a failure source: texturelessness, complex geometry, reflection, thin structure, or symmetry. All provide scaled CAD and photorealistic texture for rendering / synthetic augmentation.

This is a curated stress test rather than foundation-model training data. Its value is diagnosing which material / geometry defeats a method, beyond an overall mean.

3. Classic set

Classic supports instance-level estimators:

• 21 scenes: 13 train, 8 test;
• Training:
  • 1217 images / 4 lab scenes;
  • 1122 images / 8 industrial scenes;
  • 1361 synthetic images of Object 3;
  • 3700 frames total;
• Testing: 923 real industrial images;
• Total: 4623 images and about 6K annotated poses.

Classic uses Intel RealSense D455 at 640×480. Object–camera distances span 200–1000 mm, mostly 400–800 mm, near common assembly-line arm working distances.

Training often uses single-object / simple scenes; testing adds occlusion, multiple objects / instances, boxes, trays, and holders. This generalization test also has differing train–test gaps by object.

4. Extended set and static onboarding

Extended targets model-based and model-free novel-object estimators:

• 10 static RGB-D onboarding sequences, two per object;
• About 650 frames per sequence, about 6.3K total;
• 3 test scenes, over 2K images, about 10.3K annotated instances;
• Handheld RealSense D405, 640×480, office environments, natural lighting, distractors, and occlusion.

One sequence places the object naturally upright; another changes the resting side or flips it to cover other surfaces. Static onboarding means a stationary object with an orbiting camera, useful for video-based reference representations such as model-free FoundationPose.

Combined sets:

Group	annotations	scenes
Classic Train	3.9K	13
Classic Test	1.9K	8
Onboarding	6.3K	10
Extended Test	10.3K	3
Total	22.4K	34

Extended test has only 3 physical scenes; shared scenes with five objects and dense video produce 10.3K instances, which do not imply independent environments.

5. Pose / distance coverage

Coverage is uneven across objects and orientations, constrained by resting pose, trays, and workstation visibility. Handheld D405 gives closer and broader distances, but many frames can still follow one smooth trajectory.

Study pose-distribution generalization by spherical bins or viewpoint entropy, beyond random-frame averages.

6. 6D pose annotation

Two annotation routes:

6.1 Marker-based

Laboratory turntables and marker-compatible industrial scenes use ArUco camera poses plus object / marker calibration.

6.2 SfM + 3D anchor + PnP

Without markers:

1. Agisoft Metashape SfM recovers camera trajectories;
2. Manually choose known 3D anchors on CAD;
3. Software tracks their 2D image locations;
4. Form 2D–3D correspondences;
5. PnP recovers object-to-camera pose.

This reduces framewise manual effort, but depends on SfM stability in textureless industrial settings and anchor geometry.

7. Depth consistency and reporting conventions

Captured depth $d_c$ is compared with GT-pose-rendered depth $d_r$:

$$\delta=d_c-d_r.$$

Split	mean signed	std	mean absolute	median absolute
Classic D455	-1.85 mm	11.6 mm	9.3 mm	7.15 mm
Extended D405	-1.01 mm	5.2 mm	10.2 mm	8.95 mm

The following prose gives absolute depth error below 12 mm for Classic and about 5 mm for Extended. Extended’s 5 mm conflicts with Table IV mean absolute 10.2 mm and appears to confuse standard deviation 5.2 mm with absolute error. This site uses labeled table values and flags the discrepancy for documentation clarification.

Depth residual also mixes pose, CAD, sensor, mask, and boundary errors rather than pure translation error.

8. Evaluation protocol

The paper uses:

• ADD: mean model-point distance under predicted / GT poses;
• VSD: discrepancy of visible rendered surfaces;
• MSSD: symmetry-aware maximum surface distance;
• MSPD: symmetry-aware maximum projection distance;
• BOP AR:

$$AR=\frac{1}{3}(AR_{\text{VSD}}+AR_{\text{MSSD}}+AR_{\text{MSPD}}).$$

ADD is not explicitly symmetry-aware and serves as an intuitive supplement; BOP AR better handles symmetric / near-symmetric geometry such as Object 5.

9. Baselines

Instance-level

• EPOS;
• DOPE;
• ZebraPose.

Novel-object

• FoundPose;
• Model-based FoundationPose (MB, CAD supplied);
• Model-free FoundationPose (MF, onboarding video).

Detection / segmentation

• CNOS;
• SAM6D.

The benchmark omits some contemporary methods such as Gen6D / OnePose++ due to adaptation costs, platform limitations, and per-object training time. It provides reference baselines rather than a complete leaderboard.

10. Overall pose results

Classic

Method	ADD	MSPD	MSSD	VSD	AR
EPOS	0.64	0.54	0.56	0.41	0.51
DOPE	0.13	0.09	0.09	0.05	0.08
ZebraPose	0.61	0.54	0.55	0.40	0.50
FoundPose	0.55	0.31	0.39	0.22	0.30
FoundationPose MB	0.78	0.73	0.74	0.53	0.67
FoundationPose MF	0.44	0.26	0.31	0.18	0.25

Extended

Method	ADD	MSPD	MSSD	VSD	AR
FoundPose	0.34	0.41	0.25	0.18	0.28
FoundationPose MB	0.81	0.83	0.74	0.50	0.69
FoundationPose MF	0.48	0.40	0.37	0.22	0.33

Two clear conclusions:

1. CAD-based FoundationPose MB greatly exceeds static-onboarding MF; 10 videos do not yet match accurate object models.
2. Extended is not harder on every metric. Office distractors / occlusion coexist with different close handheld views, D405 sensing, and object presentation. Columns are not an in-distribution difficulty progression.

11. Per-object results

FoundationPose MB AR:

Split	Obj.1	Obj.2	Obj.3	Obj.4	Obj.5
Classic	0.78	0.49	0.64	0.47	0.31
Extended	0.79	0.54	0.42	0.54	0.56

Reflective, thin Object 5 is hardest on Classic. FoundPose reaches only 0.04 AR on Extended Object 5 versus FoundationPose MB 0.56, supporting accurate CAD rendering / refinement for this object.

Training conditions also confound objectwise gaps: Object 3 has 1361 extra synthetic images; some objects train with clutter and others mainly with simple backgrounds. Overall AR mixes method capability with training coverage.

12. Detection / segmentation

Paper Table VII:

Method	BBox Classic	BBox Extended	Segm Classic	Segm Extended
CNOS	0.240	0.574	0.203	0.512
SAM6D	0.270	0.527	0.345	0.453

Extended mAP is generally higher, possibly from favorable viewpoints, scales, or mask visibility despite office clutter. Do not attribute all pose failures to detection; separate GT masks, predicted masks, and full pipelines.

13. Main limitations

1. Only 5 objects. Useful for deep failure analysis, not broad category statistics.
2. Only 34 scenes. Extended test especially has only 3 physical scenes and correlated frames.
3. Uneven data. Object 3 gets synthetic augmentation; objectwise train/test scene counts differ.
4. Confounded train/test complexity. Some train in cluttered industry, others mainly laboratory single-object settings.
5. Annotation-quality prose / table mismatch. Extended 5.2 versus 10.2 mm needs clarification.
6. Limited model-free benchmark. FoundationPose MF alone cannot compare diverse sequence-modeling approaches.
7. Material sensing. Reflective / thin depth holes interact with pose error, beyond what one depth residual can separate.

14. Recommended use

• Supplement industrial benchmarks such as T-LESS, ITODD, and IPD;
• Report overall, per-object, and per-scene results;
• Specify Classic training, Extended onboarding, or CAD;
• Separate detection / segmentation from pose refinement;
• Split by scene rather than frame;
• Report Object 5 symmetry-aware metrics, depth validity, and failure visualizations separately;
• Cite explicit Table IV annotation-quality columns and explain the prose discrepancy.

15. Overall assessment

IndustryShapes combines known-instance / static-onboarding protocols, laboratory / workstation scenes, and CAD / model-free methods in one BOP-compatible package. It asks:

How much of a strong household-benchmark pose system’s capability survives reflective, thin industrial parts, single-object training to cluttered testing, or short onboarding videos?

FoundationPose MB AR 0.67 / 0.69 gives a strong reference; MF 0.25 / 0.33 and Object 5 failures show industrial novel-object pose remains unsolved.