Automating Arknights with OpenCV, OCR, and stubbornness

What it does

MAA is an open-source assistant for the mobile game Arknights that automates the daily grind—farming stages, managing the in-game base, handling recruitment, and running roguelike modes. Instead of blindly clicking coordinates, it watches the screen with a proper computer vision stack built on OpenCV, PaddleOCR, and ONNX Runtime, recognizing UI elements, character lists, and map tiles in real time.

The interesting bit

What separates this from a simple macro is the depth of integration: it uploads drop and recruitment data to community trackers like Penguin Stats, optimizes base shifts by calculating operator efficiency, and can execute complex battle strategies from shared JSON “homework” files. It also exposes APIs in C, Python, Java, Rust, Golang, Dart, and TypeScript, making the core CV engine useful as a general mobile-automation backend.

Key highlights

• Built on a real CV pipeline—OpenCV, PaddleOCR, and FastDeploy—for parsing game state rather than hard-coded coordinate mashing
• Supports CLI operation on Windows, Linux, and macOS for headless or server-side deployment
• Integrates with community data sources, auto-uploading materials to Penguin Stats and other planners
• Covers multiple international clients (EN, JP, KR, TW), though feature completeness varies by region
• Extensive language bindings and documented protocols for custom integrations

Caveats

• International server support is largely untested; the README warns that bugs are expected due to limited users and manpower
• Adapting the vision pipeline to new UI layouts or overseas clients still requires manual screenshots and JSON edits

Verdict

A solid reference if you’re building computer-vision automation for mobile games or studying OCR-driven UI interaction. If you don’t play Arknights and aren’t researching CV-based bots, this is a very specific tool to skip.