The Portal AI rebuilt to roast you in real time

What it does

GLaDOS is a fully autonomous voice assistant that watches through a camera, listens through a microphone, and speaks whenever it decides it has something to say—no wake word required. It is built around a Society-of-Mind architecture where specialized subagents handle vision, memory, emotion, news, and weather, writing their outputs to shared slots that the main LLM reads to generate context for each response. The project targets sub-600-millisecond round-trip latency using a custom-trained TTS model, ONNX-optimized vision and speech components, and careful thread prioritization.

The interesting bit

Instead of a single monolithic prompt, the system assembles GLaDOS’s “self” fresh each interaction from dynamically merged context sources—personality prompts, emotional state, user preferences, and constitutional modifiers—so the character emerges from the interplay of independent agents rather than a static system message. The autonomy loop runs continuously in the background, but user speech always jumps the priority queue and can interrupt playback mid-sentence.

Key highlights

• Proactive autonomy loop: subagents watch vision, timers, and tasks without blocking the main conversation thread.
• Persistent personality via HEXACO traits and a reactive PAD emotional model across sessions.
• Long-term memory storing facts, preferences, and conversation summaries with automatic token compaction.
• FastVLM (ONNX) vision pipeline for scene understanding and change detection.
• Runs fully offline on an 8GB Rock5b SBC with RK3588 NPU; MCP tools support home automation via stdio, HTTP, or SSE.

Caveats

• Streaming ASR, the observer agent, and physical animatronics are still on the roadmap and not yet implemented.
• The author notes the system has been refactored multiple times since 2023 as better models appeared, so the architecture is still evolving.

Verdict

Developers experimenting with embodied AI, real-time conversational agents, or multi-agent personality systems will find plenty to study here. Those looking for a stable, finished product should note that the author has already refactored the stack multiple times as new models dropped.