Single-Shot 6DoF Pose and 3D Size Estimation for Robotic Strawberry Harvesting
A single-stage RGB detector is extended to jointly regress strawberry 6-DoF pose, 3D size, and confidence. Straw6D is automatically generated in Ignition Gazebo, with 12 equivalent rotations around the principal axis accounting for approximate symmetry. Synthetic tests run at around 60 FPS.
A YOLO-style single-stage RGB detector jointly predicts strawberry 6DoF poses, 3D sizes, and confidence. Ignition Gazebo generates Straw6D automatically; training / evaluation select among 12 equivalent rotations approximating axial symmetry.
Original Figure 2: projected points, 3D size, and confidence are predicted, then camera geometry recovers 6DoF pose. Source: authors' public manuscript. Image copyright belongs to the authors and IEEE; reproduced here for paper analysis.Original Figure 3 and part of Figure 4: Gazebo randomizes fruit, lights, camera angles, and distances, with synthetic prediction examples. This crop omits Tables I–II; threshold AP values in the text come separately from those original tables. Source: authors' public manuscript. Image / table copyright belongs to the authors and IEEE; reproduced here for paper analysis.
At a glance
Straw6D: 2,000 synthetic images, 80% training / 20% testing, with RGB, depth, masks, points, and 3D boxes.
3D IoU AP at 0.5 / 0.6 / 0.7 / 0.8: 84.77 / 78.60 / 45.78 / 19.87.
Pose AP: 74.73 at 2cm,20°, and 35.58 at strict 1cm,10°.
RTX 3060 Ti: mean 16.60 ms, about 60 FPS.
Real StrawDI backbone pretraining followed by Straw6D improves qualitative real transfer over synthetic-only training, without real pose GT for quantitative evaluation.
Quick assessment
Strengths: integrated task, data, and real-time network; size and symmetry explicitly support agricultural harvesting.
Limits: only 2,000 images, strict-threshold degradation, qualitative real boxes only, and no robot harvesting success rates.
Available src/paper-notes/strawberry-pose-iros2024/deep.md
In-depth paper notes
Support reproduction, comparison, related-work writing, and identifying opportunities for further research.
Deep Reading Notes: Single-Shot Strawberry 6DoF
1. Key assessment
The main contribution applies category-level 6D pose / 3D size to strawberry harvesting with automatic synthetic data. The network extends single-stage detection; value chiefly comes from approximate fruit-axis symmetry, Gazebo labels, and real-time constraints rather than a new general pose backbone.
2. Task and network
From one RGB image, grid / anchor predictions include:
Projected fruit centers and 2D projections of 3D box corners;
3D sizes;
object / pose confidence;
Rotation / translation recovered through camera projection geometry.
Weighted losses combine coordinates, dimensions, positive confidence, and negative confidence. Weights are λcoord=1, λdim=λconf=5, λconf−no=0.1.
Original Figure 2: 2D projected points are predicted rather than rotation matrices directly, then combined with camera geometry for 6DoF / size. Source: authors' public manuscript. Image copyright belongs to the authors and IEEE; reproduced here for paper analysis.
3. Symmetry handling
Strawberries are approximately rotationally symmetric along the tip-to-calyx axis. One box orientation creates conflicting equivalent appearances; 12 GT rotations around this axis supply minimum-error training / evaluation targets.
Does not capture real symmetry-breaking calyces, deformation, and colors.
It defines task tolerance without establishing absolute axial orientation recovery.
4. Straw6D data
Original generation / prediction page. Ignition Gazebo randomizes plants, fruit, ripeness, lighting, views, and distances. This crop omits threshold tables; strict-threshold degradation in the text comes from original Tables I–II. Source: authors' public manuscript. Image / table copyright belongs to the authors and IEEE; reproduced here for paper analysis.
Gazebo’s 3D-box camera directly labels object states, including partially occluded fruits. Randomization covers:
Fruit counts, sizes, ripeness, and poses per plant;
Loose thresholds perform reasonably, but IoU 0.7→0.8 and angles 20°→10° decline substantially, leaving precise calyx localization / grasping unresolved. The reported 16.60 ms / 60 FPS measures network inference, excluding robot planning / execution.
Real transfer pretrains YOLO 2D heads / backbones on StrawDI, then complete 3D heads on Straw6D. Real fruits across ripeness stages receive detections / 3D boxes, with qualitative evidence only because real 6DoF / size GT is unavailable.
6. Critical analysis
Strengths: automatic labels reduce pose costs, sizes are explicit, and speed suits harvesting frontends.
Limits: small 2,000-image scale, no quantitative real poses, approximate symmetry obscuring calyx direction, and no robotic harvesting success.
Metrics: box IoU mixes position, rotation, and size; add ADD-S / symmetry-aware angular errors to isolate orientation.
Domain gap: real 2D pretraining supplies real supervision, preventing claims of purely synthetic direct transfer.
7. Reproduction checklist
Fix Gazebo assets, randomization ranges, and split seeds;
Publish intrinsics, box-corner ordering, and 12 equivalent rotations;
Report center, rotation, size, and detection errors separately;
Add limited real marker / scan pose GT;
Break down occlusion, ripeness, size, and calyx visibility;
Report harvesting success and fruit damage.
8. Overall assessment
This practical agricultural baseline covers synthetic data, symmetric poses, size, and real-time inference, with strict pose accuracy and real robot validation still open. Position it within task-specific category-level pose plus synthetic data, rather than ranking directly with known-CAD BOP methods.
