An AI teacher that lives next to your cursor — now open source

What it does

Clicky is a menu bar app (no dock icon) that acts as an AI buddy hovering beside your cursor. You push-to-talk, it transcribes your speech in real time, grabs a screenshot, and streams both to Claude — then answers out loud via ElevenLabs TTS. It can even point: Claude embeds [POINT:x,y:label:screenN] tags in responses, and a blue overlay flies your cursor to specific UI elements across multiple monitors.

The interesting bit

The architecture is a tidy three-API relay: the Swift app talks to a small Cloudflare Worker, which proxies Anthropic, AssemblyAI, and ElevenLabs so your API keys never ship inside the app binary. Also notable — the author has moved new development into the closed-source HeyClicky app, so this repo is now a snapshot to fork and tinker with, not a moving target.

Key highlights

• Real-time voice loop: push-to-talk → AssemblyAI streaming transcription → Claude SSE streaming → ElevenLabs TTS playback
• Cursor pointing via [POINT] tags, working across multiple monitors
• Menu bar app with two NSPanel windows — a dropdown control panel and a full-screen transparent cursor overlay
• Cloudflare Worker keeps API keys out of the app binary; three routes: /chat, /tts, /transcribe-token
• MIT licensed, with a CLAUDE.md architecture doc so AI coding agents can navigate the codebase

Caveats

• Development has effectively moved to the private HeyClicky app; this repo won’t get the new features
• Requires macOS 14.2+ (ScreenCaptureKit), Xcode, and paid API keys for three separate services
• Proxy URLs are hardcoded in the Swift source and need manual editing to point at your own Worker

Verdict

Great read for anyone curious how a screen-aware, voice-driven desktop agent is actually wired together — or a solid starting point for your own AI companion. Skip it if you want an actively maintained open-source project rather than a fork-and-own-it snapshot.