What it does

Gibson is a robotics simulator built around a simple premise: train your agent in spaces that actually exist. It virtualizes 572 real-world buildings (1,440 floors) scanned into 3D, then drops in a physics engine (Bullet) and a robot body with actual motion constraints. The result is an RL environment where a Husky rover or a legged ant learns hallway navigation with RGB-D camera input, subject to real spatial limits rather than Unity-style open worlds.

The interesting bit

The “Goggles” function—an explicit baked-in mechanism for sim-to-real transfer. The authors don’t just hope the policy generalizes; they built a transfer channel into the environment itself. Also, the name is a flex: James J. Gibson wrote “The Ecological Approach to Visual Perception” in 1979, and the project takes his “we must move in order to perceive” literally.

Key highlights

• 572 real scanned spaces, plus integrations with Stanford 2D3DS and Matterport3D datasets
• Docker-first distribution (nvidia-docker2) with headless-server support via x11vnc
• Physics-backed embodiment through Bulletphysics integration
• CVPR 2018 Spotlight Oral; framerate benchmarks provided (e.g., 109.1 FPS at 128×128 RGBD pre-network on V100)
• ROS configuration and OpenAI Baselines demos included (PPO1/2)

Caveats

• Minimum VRAM requirement is 6GB; CUDA 8+ and specific driver versions required
• Source build is dependency-heavy (GLEW, GLM, Assimp, Boost, OpenMPI, plus pinned PyTorch 0.3.1 and TensorFlow 1.3)
• Core asset package is deliberately slim (39 spaces); full dataset requires separate download and license agreement

Verdict

Grab this if you’re doing embodied RL or sim-to-real robotics research and need semantically realistic indoor navigation. Skip it if you want lightweight, game-engine aesthetics or modern PyTorch/TensorFlow versions—the stack is frozen circa 2018.