A cookbook for Quest's new eye on the real world

What it does QuestCameraKit is a collection of six Unity sample scenes for the Meta Quest 3 and 3S that exercise the Passthrough Camera API. It demonstrates how to sample real-world colors, run YOLO object detection, track QR codes natively, apply camera-fed shaders like frosted glass, query OpenAI with voice and image prompts, and stream video over WebRTC. Think of it as a reference cookbook for mixed-reality computer vision.

The interesting bit Rather than treating passthrough as a background wallpaper, these samples treat the headset as a camera platform—piping stereo camera feeds into neural networks, WebRTC pipes, and material shaders. The project is also admirably frank about platform friction, noting that Editor preview cannot validate camera behavior and that recent on-device testing was blocked by a system sensor lock.

Key highlights

• Six runnable samples packaged into a single headset APK with an in-app menu
• Environment raycasting to sample real-world surface colors from camera pixels
• Bundled YOLO inference via Unity Inference Engine, projecting detection labels into 3D space
• Native QR tracking through Meta MRUK without extra NuGet decoders
• WebRTC streaming and OpenAI voice+image integration (both require external keys and servers)

Caveats

• Camera and tracking workflows require an actual headset build; the Unity Editor preview does not exercise on-device camera behavior
• The README notes that on-device acceptance testing is currently pending because the available Quest 3S session was blocked by a system sensor lock
• OpenAI and WebRTC samples need private API keys, custom signaling infrastructure, and network access to function

Verdict Grab this if you are a Unity developer trying to ship Quest 3/3S mixed-reality features and need a working baseline for camera access. Skip it if you want a production framework—this is explicitly sample code and template glue.