Turning WiFi Static Into 3D Human Skeletons

What it does

WiFi-3D-Fusion is a research pipeline that ingests WiFi Channel State Information — the fine-grained radio metrics usually ignored by your laptop — and runs them through a PyTorch-based detector to estimate 3D human pose and movement. It supports two hardware paths: an ESP32 broadcasting CSI over UDP, or a Linux USB WiFi adapter running Nexmon firmware in monitor mode. The system renders results live in an Open3D viewer or a web-based Three.js interface, and it can bridge its output into related research modules like Person-in-WiFi-3D (CVPR 2024) and NeRF².

The interesting bit

The clever part is treating multipath WiFi reflections as a free, always-on sensing modality rather than noise to filter out. By fusing these radio fingerprints with computer vision techniques, the project attempts to track multiple people and maintain identity across sessions (ReID) using nothing but the ambient signals already bouncing around the room.

Key highlights

• Ingests CSI from either ESP32 UDP streams or Nexmon tcpdump captures, with a real-time movement detector sitting in between.
• Claims continuous learning: the CNN-based detector supposedly adapts thresholds and retrains in the background based on detection confidence and user feedback.
• Bundles bridges to external research code: Person-in-WiFi-3D for multi-person pose, NeRF² for neural radiance fields, and an RSSI volumetric mapper.
• Ships with both an Open3D native viewer and a browser-based Three.js renderer with HUD overlays.
• Explicitly tagged “Research Only,” which is honest given the specialized hardware and Linux-only stack.

Caveats

• Requires specific, non-generic hardware: either an ESP32 flashed with CSI firmware or a Broadcom/Nexmon-compatible USB adapter in monitor mode, plus a Linux host.
• The README notes its own architecture diagrams contain typos and are being revised, suggesting the documentation is still settling.
• Claims around “continuous” and “automatic” model improvement are stated but lack technical detail on how feedback is collected or validated.

Verdict

Worth a look if you are a wireless-sensing researcher or a curious hacker with a spare ESP32 and a tolerance for radio-frequency tinkering. Skip it if you need a plug-and-play room monitor that runs on your existing Windows laptop.