Simulated Homes Built for Robot Common Sense

What it does

AI2-THOR is a Unity-based simulation framework that generates near-photo-realistic indoor environments for training embodied AI agents. It provides over 200 interactive scenes and 2,600 annotated household objects with realistic physics, enabling researchers to test navigation, manipulation, and common-sense reasoning without physical robots.

The interesting bit

The framework goes beyond static worlds with context-sensitive interactions—place a dirty bowl under running water and it cleans itself, or put a mug in a coffee machine and it fills automatically. It also outputs a wide array of visual modalities like semantic segmentation and depth frames natively, so you don’t have to hack a renderer to get ground-truth labels.

Key highlights

• Three distinct environments: iTHOR for high-level reasoning, ManipulaTHOR for robotic arm tasks, and RoboTHOR for Sim2Real transfer with real-world scene counterparts.
• Over 200 actions and multiple agent types, including multi-agent setups, a LoCoBot, and a drone.
• First-class support for ego-centric RGB, instance and semantic segmentation, depth, normals, and orthographic projections out of the box.
• Built-in domain randomization for materials and lighting to stress-test model generalization.

Caveats

• It requires a graphics card with at least DX9 / shader model 3.0 support and an X server with GLX on Linux, since it relies on Unity 3D rendering.

Verdict

Researchers building embodied AI or computer vision models that need realistic, interactive domestic environments should look here. If your work doesn’t require spatial reasoning or physical interaction, it’s probably overkill.