Teaching robot arms to grasp one pixel at a time

What it does

Provides a Gym environment and standalone controller for a simulated UR5 arm in MuJoCo. The GraspEnv environment trains agents on a pick-and-place task using only RGB-D camera observations, while MJ_Controller handles inverse kinematics, joint movements, and image transforms. The action space is the image itself: the agent selects a grasp point from a 200×200 pixel grid, yielding roughly 4 mm accuracy and 40,000 possible actions.

The interesting bit

Most sim environments feed the agent joint angles and object poses; this one forces the agent to reason about grasping from raw pixels alone. Rewards are binary—zero or one—delivered only after the robot lifts and transports the object, and the author threw in offline RL datasets, point-cloud utilities, and joint-trajectory plotting for good measure.

Key highlights

• Action space is a 200×200 pixel grid (~40,000 discrete actions, ~4 mm picking accuracy)
• Observations are RGB-D image dictionaries instead of simulation state vectors
• MJ_Controller works standalone for testing models outside of Gym episodes
• Includes depth-to-world transforms and point-cloud generation from camera data
• Offline RL dataset generation scripts in the Offline RL folder
• Optional joint-trajectory plotting with tolerance bands for PID tuning

Caveats

• Explicitly described as “unformatted spaghetti code” from the author’s earlier Python days and no longer under active development
• Inverse kinematics solver does not implement collision avoidance
• Gripper orientation is locked to vertical top-down grasps

Verdict

Useful if you need a reference implementation for visuomotor grasping or pixel-based RL on robot arms. Avoid if you require polished, actively maintained simulation infrastructure.