Synthetic MetaHumans teach VR eye cameras metric depth

What it does

DeepMetricEye reconstructs metric eye-region geometry from the monocular, inward-facing cameras found in VR headsets. It pairs a lightweight PyTorch depth-estimation model with an Unreal Engine 5.2 environment that generates synthetic periocular RGB and depth training data from MetaHuman avatars. The aim is to support measurements like pupil diameter or periocular deformation that 2D imagery alone cannot provide.

The interesting bit

Most depth-estimation datasets ignore the cramped, highly deformable periocular region. This project targets that niche directly, and it ships a complete Unreal Engine 5 synthetic-data factory so you can generate labeled training pairs without handling restricted facial scans.

Key highlights

• Lightweight PyTorch monocular depth model tuned for eye-region geometry reconstruction.
• Bundled UE 5.2 environment renders synthetic MetaHuman periocular pairs with ground-truth depth.
• Only a minimal sample dataset ships with the repo; full facial scans are gated for privacy.
• ISMAR 2023 research code released under GPL-3.0.

Caveats

• Authors note the repo still needs reproducibility fixes, dataset-loader cleanup, and broader VR headset calibration notes.
• The full experimental dataset is restricted due to sensitive facial information; only a minimal verification sample ships with the code.

Verdict

Worth a look if you are doing XR eye-health or sensing research and need a reproducible starting point for metric periocular geometry. Skip it if you want a polished, general-purpose depth-estimation SDK—it is research code with acknowledged rough edges.