LLM Agents Get a 2D Town and a JavaScript Runtime

What it does AI Town is a virtual town simulator inspired by the Generative Agents research paper. It drops LLM-driven characters into a 2D pixel-art world where they can move around, start conversations, and form memories backed by a vector database. The whole thing is built as a starter kit on top of Convex, giving it shared global state, transactions, and a simulation engine out of the box.

The interesting bit Most implementations of this idea are Python notebooks; AI Town deliberately built itself in TypeScript to fit into a modern web stack. It also tries hard to be deployment-agnostic, running locally via Ollama and Docker Compose or scaling up to cloud LLMs and Fly.io without rewriting the core.

Key highlights

• JS/TS-native stack: Uses Convex for the game engine, database, and vector search, with PixiJS handling the frontend rendering.
• Pluggable LLMs: Defaults to entirely local inference with Ollama (llama3), but can swap in OpenAI, Together.ai, or any OpenAI-compatible API by changing a few config values.
• Persistent memory: Agents rely on embeddings stored in Convex’s vector search, though switching embedding models forces a full database wipe.
• Customizable world: Characters, spritesheets, and tilemaps can be edited via TypeScript files and the Tiled map editor; optional background music generation hooks into Replicate.
• Idle pause: The simulation freezes after five minutes of browser inactivity unless you disable the cron job or keep the page open.

Caveats

• Switching LLM providers or embedding dimensions requires wiping the entire database and starting over, since vector memories are tied to the specific model.
• Any tweak to character data also demands a full table wipe and re-upload to Convex, so iteration can be destructive.
• Self-hosting with Docker Compose involves regenerating admin keys if you tear down the containers, and Replicate webhooks need manual proxying to reach a local instance.

Verdict Worth a look if you want to experiment with multi-agent LLM simulations without leaving the JavaScript ecosystem. Skip it if you need a polished, zero-config game or a Python-first research environment.