Category
10 categories
Bottle, box, can, cup, cutlery, glass, remote, shoe, teapot, and tube, covering everyday household objects and optically challenging transparent and reflective materials.
Dataset Detail · Category-level 6D Pose
HouseCat6D
A multimodal dataset for category-level 6D object perception in real indoor scenes. It combines substantial variation within categories, wide viewpoint coverage, occlusion, transparent and reflective materials, polarization observations, and dense grasp annotations.
Local dataset root
/data1/dataset/housecat6d
The local directory contains extracted training, test, and validation scenes, object meshes, and original archives, ready for data loading, visualization, and evaluation scripts.
Paper: Jung et al., CVPR 2024 Highlight · arXiv:2212.10428
Object instances
194 objects
Each category contains multiple instances to test generalization across variations in shape, appearance, and size within the category.
Real scenes
41 scenes
34 training scenes, 2 validation scenes, and 5 test scenes, with approximately 23.5k frames and 160k object pose annotations.
Sensors
Multimodal observations
Synchronized RGB, active stereo depth, and polarized RGB. Each polarization capture uses 4 filter angles to study complementary material and geometric cues.
Object atlas
10 categories · 194 object models
Expand to browse thumbnails, category statistics, and scene frequencies.
Browse objects
Object atlas
10 categories · 194 object models
Expand to browse thumbnails, category statistics, and scene frequencies.
Category statistics
Models per category
194 total
Bottle 21
Box 23
Food / beverage can 23
Cup 16
Cutlery 17
Drinking glass 17
Remote control 19
Shoe 21
Teapot 19
Squeezable tube / paste packaging 18
Scene frequency
Number of scenes containing each object
Top 15
41
Scenes
274
Object placements
194
Unique objects
fanta3 times
green actys3 times
yellow handle3 times
fork 13 times
knife 13 times
shoe polish3 times
85 alcool2 times
baby cleansing water2 times
dettol washing machine2 times
eres inox2 times
evian frozen2 times
sanitizer small white2 times
shampoo doux2 times
sloth2 times
today2 times
Frequencies are computed from meta.txt across 41 scenes and count scenes containing an object, rather than video frames.
Browse objects
Official model previews are reused where available; other models are rendered offline from a consistent viewpoint. Select a thumbnail to view a larger image.
Showing 194 / 194 objects
Images approximately 320 × 320
Bottle
85 alcool
bottle-85_alcool
Bottle
alcool hands
bottle-alcool_hands
Bottle
avene skincare
bottle-avene_skincare
Bottle
baby cleansing water
bottle-baby_cleansing_water
Bottle
cleansing lotion small
bottle-cleansing_lotion_small
Bottle
deodorant spray
bottle-deodorant_spray
Bottle
dettol washing machine
bottle-dettol_washing_machine
Bottle
eres inox
bottle-eres_inox
Bottle
eres soap
bottle-eres_soap
Bottle
evian frozen
bottle-evian_frozen
Bottle
evian red
bottle-evian_red
Bottle
hg meubeline
bottle-hg_meubeline
Bottle
inox cleaner
bottle-inox_cleaner
Bottle
mouthwash meridol
bottle-mouthwash_meridol
Bottle
nivea
bottle-nivea
Bottle
sanitizer small white
bottle-sanitizer_small_white
Bottle
shampoo doux
bottle-shampoo_doux
Bottle
sloth
bottle-sloth
Bottle
soupline
bottle-soupline
Bottle
today
bottle-today
Bottle
v8 small
bottle-v8_small
Box
antikalk
box-antikalk
Box
barilla
box-barilla
Box
bbq
box-bbq
Box
bisolvon sirup
box-bisolvon_sirup
Box
colgate
box-colgate
Box
corny
box-corny
Box
crunchy muesli
box-crunchy_muesli
Box
ebly
box-ebly
Box
egar omega 3
box-egar_omega_3
Box
garmin
box-garmin
Box
gift
box-gift
Box
iglo
box-iglo
Box
kleenex
box-kleenex
Box
koala
box-koala
Box
leibniz choco
box-leibniz_choco
Box
lieqi
box-lieqi
Box
mc cafe
box-mc_cafe
Box
principe
box-principe
Box
proschmelz
box-proschmelz
Box
special k
box-special_k
Box
sultana yo fruit render
box-sultana_yo_fruit_render
Box
toffifee
box-toffifee
Box
wasabi
box-wasabi
Food / beverage can
beer delta
can-beer_delta
Food / beverage can
cheakpeas
can-cheakpeas
Food / beverage can
coke small
can-coke_small
Food / beverage can
concentrated tall
can-concentrated_tall
Food / beverage can
concentrated tiny
can-concentrated_tiny
Food / beverage can
corn
can-corn
Food / beverage can
corn small
can-corn_small
Food / beverage can
corned beef
can-corned_beef
Food / beverage can
fanta
can-fanta
Food / beverage can
fricadelles
can-fricadelles
Food / beverage can
hoegaarden
can-hoegaarden
Food / beverage can
kidney beans
can-kidney_beans
Food / beverage can
large tuna
can-large_tuna
Food / beverage can
meatball
can-meatball
Food / beverage can
monster
can-monster
Food / beverage can
monster blue
can-monster_blue
Food / beverage can
olive anchois
can-olive_anchois
Food / beverage can
redbull
can-redbull
Food / beverage can
sheba cat
can-sheba_cat
Food / beverage can
small tuna
can-small_tuna
Food / beverage can
spam
can-spam
Food / beverage can
target beef
can-target_beef
Food / beverage can
tuna salad
can-tuna_salad
Cup
green actys
cup-green_actys
Cup
green handle
cup-green_handle
Cup
grey handle
cup-grey_handle
Cup
mc cafe
cup-mc_cafe
Cup
new york
cup-new_york
Cup
new york big
cup-new_york_big
Cup
plastic green flowers
cup-plastic_green_flowers
Cup
red
cup-red
Cup
red heart
cup-red_heart
Cup
stanford
cup-stanford
Cup
teal pattern ikea
cup-teal_pattern_ikea
Cup
white coffee round handle
cup-white_coffee_round_handle
Cup
white hogermann
cup-white_hogermann
Cup
white whisker
cup-white_whisker
Cup
yellow handle
cup-yellow_handle
Cup
yellow white border
cup-yellow_white_border
Cutlery
fork 1
cutlery-fork_1
Cutlery
fork 1 new
cutlery-fork_1_new
Cutlery
fork 2 new
cutlery-fork_2_new
Cutlery
fork 3 new
cutlery-fork_3_new
Cutlery
knife 1
cutlery-knife_1
Cutlery
knife 1 new
cutlery-knife_1_new
Cutlery
knife 2
cutlery-knife_2
Cutlery
knife 2 new
cutlery-knife_2_new
Cutlery
knife 3 new
cutlery-knife_3_new
Cutlery
spoon 1
cutlery-spoon_1
Cutlery
spoon 1 new
cutlery-spoon_1_new
Cutlery
spoon 2
cutlery-spoon_2
Cutlery
spoon 2 new
cutlery-spoon_2_new
Cutlery
spoon 3 new
cutlery-spoon_3_new
Cutlery
spoon 4 new
cutlery-spoon_4_new
Cutlery
spoon 5 new
cutlery-spoon_5_new
Cutlery
spoon 6 new
cutlery-spoon_6_new
Drinking glass
cocktail
glass-cocktail
Drinking glass
new 1
glass-new_1
Drinking glass
new 10
glass-new_10
Drinking glass
new 11
glass-new_11
Drinking glass
new 12
glass-new_12
Drinking glass
new 13
glass-new_13
Drinking glass
new 2
glass-new_2
Drinking glass
new 3
glass-new_3
Drinking glass
new 4
glass-new_4
Drinking glass
new 5
glass-new_5
Drinking glass
new 6
glass-new_6
Drinking glass
new 7
glass-new_7
Drinking glass
new 8
glass-new_8
Drinking glass
new 9
glass-new_9
Drinking glass
small
glass-small
Drinking glass
small 3
glass-small_3
Drinking glass
small 4
glass-small_4
Remote control
aircon chunghop
remote-aircon_chunghop
Remote control
black
remote-black
Remote control
comfee
remote-comfee
Remote control
factory svc
remote-factory_svc
Remote control
grey
remote-grey
Remote control
heitech
remote-heitech
Remote control
infini fun
remote-infini_fun
Remote control
japanese
remote-japanese
Remote control
jaxster
remote-jaxster
Remote control
jaxster d1170
remote-jaxster_d1170
Remote control
led 1
remote-led_1
Remote control
led 2
remote-led_2
Remote control
nvtc
remote-nvtc
Remote control
seki care
remote-seki_care
Remote control
seki medium
remote-seki_medium
Remote control
silver
remote-silver
Remote control
thin silver
remote-thin_silver
Remote control
toy
remote-toy
Remote control
tv white quelle
remote-tv_white_quelle
Shoe
aqua cyan right
shoe-aqua_cyan_right
Shoe
asifn yellow right
shoe-asifn_yellow_right
Shoe
blue 75 right
shoe-blue_75_right
Shoe
cat grey sandal right
shoe-cat_grey_sandal_right
Shoe
crocs pink sandal right
shoe-crocs_pink_sandal_right
Shoe
crocs white cyan right
shoe-crocs_white_cyan_right
Shoe
crocs yellow sandal right
shoe-crocs_yellow_sandal_right
Shoe
fashy white right
shoe-fashy_white_right
Shoe
frog right
shoe-frog_right
Shoe
green viva sandal right
shoe-green_viva_sandal_right
Shoe
hummel green sandal right
shoe-hummel_green_sandal_right
Shoe
magenta holes right
shoe-magenta_holes_right
Shoe
pink black sandal right
shoe-pink_black_sandal_right
Shoe
pink tiny crocs right
shoe-pink_tiny_crocs_right
Shoe
sky blue 250 right
shoe-sky_blue_250_right
Shoe
sky blue holes right
shoe-sky_blue_holes_right
Shoe
sky blue striped right
shoe-sky_blue_striped_right
Shoe
skyblue leda right
shoe-skyblue_leda_right
Shoe
votte sandale
shoe-votte_sandale
Shoe
white black buckles
shoe-white_black_buckles
Shoe
white viva sandal right
shoe-white_viva_sandal_right
Teapot
ambition brand
teapot-ambition_brand
Teapot
big white floral
teapot-big_white_floral
Teapot
blue floral
teapot-blue_floral
Teapot
brown chinese
teapot-brown_chinese
Teapot
brown small
teapot-brown_small
Teapot
green grass
teapot-green_grass
Teapot
new chinese
teapot-new_chinese
Teapot
pale pink
teapot-pale_pink
Teapot
white black line
teapot-white_black_line
Teapot
white blue cone top
teapot-white_blue_cone_top
Teapot
white floral
teapot-white_floral
Teapot
white malacasa
teapot-white_malacasa
Teapot
white rectangle
teapot-white_rectangle
Teapot
white rectangle sprout
teapot-white_rectangle_sprout
Teapot
white royal norfolk
teapot-white_royal_norfolk
Teapot
white small
teapot-white_small
Teapot
white spherical
teapot-white_spherical
Teapot
white was brand
teapot-white_was_brand
Teapot
wooden color
teapot-wooden_color
Squeezable tube / paste packaging
baby nivea comfort
tube-baby_nivea_comfort
Squeezable tube / paste packaging
baby nivea spf
tube-baby_nivea_spf
Squeezable tube / paste packaging
bb cream
tube-bb_cream
Squeezable tube / paste packaging
cleanser white
tube-cleanser_white
Squeezable tube / paste packaging
creme yellow
tube-creme_yellow
Squeezable tube / paste packaging
hansaplast gel
tube-hansaplast_gel
Squeezable tube / paste packaging
kiwi
tube-kiwi
Squeezable tube / paste packaging
mdf filler
tube-mdf_filler
Squeezable tube / paste packaging
perfax
tube-perfax
Squeezable tube / paste packaging
shampoo yellow
tube-shampoo_yellow
Squeezable tube / paste packaging
shoe polish
tube-shoe_polish
Squeezable tube / paste packaging
signal
tube-signal
Squeezable tube / paste packaging
signal pokemon
tube-signal_pokemon
Squeezable tube / paste packaging
skincare hydralist
tube-skincare_hydralist
Squeezable tube / paste packaging
skincare provencale
tube-skincare_provencale
Squeezable tube / paste packaging
toothpaste signal kids bio
tube-toothpaste_signal_kids_bio
Squeezable tube / paste packaging
wasabi
tube-wasabi
Squeezable tube / paste packaging
wipp
tube-wipp
Why HouseCat6D
Which questions can it help answer?
HouseCat6D brings category-level generalization, real occlusions, and optically challenging materials together in a reproducible experimental setting.
Category-level generalization
Training and testing focus on pose, scale, and shape generalization to unseen objects within a category, beyond a single fixed CAD instance.
Material challenges
Categories such as glass and cutlery include transparent, reflective, and low-texture surfaces, exposing limitations of RGB observations for depth and correspondence estimation.
Wide viewpoints and occlusion
Handheld multicamera trajectories cover a wider range of viewpoints while retaining real indoor occlusions, beyond regular tabletop captures.
From perception to grasping
Alongside object poses, some scenes provide 6D parallel-jaw grasp annotations, connecting pose estimation with manipulation feasibility.
Modalities
How the three observation types work together
The website describes HouseCat6D as an RGBD+P dataset. Its files support scene-aligned loading of RGB, depth, pol, normal, instance, nocs, and occlusion data.
RGB
Color image
metric
Provides texture, edge, and semantic cues as the basic input for RGB-only category-level pose methods.
Depth
Active stereo depth
metric
Active stereo depth is more robust across various surface materials and supports scale, translation, and geometric alignment. The directories contain depth maps and some depth_gt data.
Polarization
Four-angle polarimetric RGB
metric
Each frame contains four polarization subimages, providing additional surface cues for transparent, reflective, and low-texture objects.
Splits & Files
Data splits and scene structure
Training, validation, and test splits are defined by scene. This prevents neighboring frames from the same camera trajectory from appearing in both training and testing, and better reflects cross-scene generalization in deployment.
SplitScenesRGB imagesWebsite size
Trainscene01–3420,529Approx. 210G
Validationval_scene1–21,403Approx. 15G
Testtest_scene1–52,929Approx. 30G
Meshesobj_models—Approx. 1G
Image counts use rgb/*.png in the local directories, totaling 24,861 RGB frames; depth and polarization data use the same frame identifiers.
Typical scene directory
scene01/
├── rgb/ # RGB images
├── depth/ # D435-style depth
├── pol/ # 4 polarization subimages
├── normal/ # Surface normals
├── instance/ # Instance masks
├── nocs/ # NOCS correspondences
├── labels/ # Pose labels in the current camera frame
├── camera_pose/ # 4×4 camera poses along the trajectory
├── occlusion/ # Per-object occlusion ratios/content
├── obj_pose_final/ # Object poses in the tracker frame
├── intrinsics.txt # Camera intrinsics
└── meta.txt # Instance id / class id / nameAnnotations
What to check in annotations and evaluation
HouseCat6D annotations follow a different convention from BOP instance-level evaluation; BOP AR scores and category-level NOCS metrics should not be combined indiscriminately.
NOCS-style poses
Label files contain instance_ids, class_ids, bboxes, translations, rotations, and gt_scales; poses are expressed in the current camera coordinate frame.
Symmetry and scale
Category-level methods often estimate rotation, translation, and dimensions jointly. Check the symmetry handling, scale definition, and success thresholds when interpreting results.
Grasping is a separate subtask
Grasp annotations cover only some scenes and are stored in HDF5. Their frame coverage differs from that of the full object pose collection.
Next step
Preparing to use the local data
Future additions can include sample-frame browsing, category statistics, label-loading scripts, NOCS/ADD-S evaluation guides, and mappings to the BOP format.
References: HouseCat6D Dataset page, Paper arXiv:2212.10428.
Citation: Jung et al., “HouseCat6D—A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios,” CVPR 2024.