Giving coding agents ears, and then letting them make noise

What it does

earshot is a CPU-only toolkit that lets an agent actually listen: describe a file in words, score how well it matches a description, find the best moments in a long recording, tell a real recording from a code-synthesized one, and produce a second-by-second listening transcript of a film. It ships as a CLI, a Python API, and an MCP server, so Claude Code and friends can call it directly. The output is deliberately agent-shaped — plain-language summaries first, --json for the full result.

The interesting bit

The loop closes: an agent can synthesize sounds in code (forge, physical modeling with modal resonators, contact excitations, a computed room — no recorded audio inside), then have earshot’s critic score its own renders and iterate. There’s even fit, which reverse-engineers a recording into a parametric recipe module you can vary and morph. The README is also unusually honest about Goodhart: it flatly admits the search optimizes earshot’s own models and that the realness model is easy to fool.

Key highlights

• describe, match, find, timeline, critique — sound as structured evidence, not vibes
• forge makes sounds from recipes and improves them via an evolutionary search with a model-free pre-screen
• fit translates a recording into readable, re-renderable code (median CLAP cosine 0.945 on 12 test recordings)
• sheet renders an annotated PNG — spectrogram, events, loudness, sync markers — for agents that read images
• Runs fully offline on CPU; the optional Gemini audio-LLM critique requires your own key and explicit consent

Caveats

• Each CLI call reloads the models (~20–40 s); the README itself says use the MCP server or Python API for more than a few calls
• Reference recordings are licensed and not in the repo — you bring your own refs, or the critic runs in a weaker “designed” mode
• Blind tests are tiny (one listener, 15 and 12 trials); the author calls the v1 forge result “a coin flip” and no v2 blind-test score is published yet

Verdict

Anyone building agents that produce audio or video with sound — motion design, game SFX, product demos — should look hard at this. If your agents never touch audio, the DSP rabbit hole here is a spectator sport, albeit a well-documented one.