milind-soni/tiptour-macos
A macOS menu bar app that uses Gemini Live AI to understand screen context and execute voice commands for computer control.

TipTour is a macOS menu bar companion that leverages Gemini Live for real-time voice recognition, screen understanding, and tool calling to control the computer. It listens for voice commands while tracking the active app and user-highlighted areas, then executes actions like clicking, typing, opening apps, and editing text through a CUA Driver Core. The system uses macOS Accessibility APIs to target UI elements and supports freeform highlighting to specify exact areas for AI actions. It runs silently from the menu bar without a dock icon.