Your Playwright bot’s local hCaptcha department

What it does

hCaptcha Challenger is a Python library that intercepts and solves hCaptcha challenges inside Playwright-driven browser sessions. It routes each challenge type—binary image classification, point selection, drag-and-drop—to a specific local ONNX model (ResNet, YOLOv8, ViT) or, for newer tasks, to a multimodal LLM agent. The goal is to keep the entire solving stack in-house, avoiding third-party anti-captcha services and browser extensions.

The interesting bit

The project treats captcha solving as an “AI vs AI” arms race run entirely on your hardware. It maintains a nested model zoo of exported vision models and even automates its own dataset collection and retraining through GitHub Actions workflows it calls sentinel and collector. When a local model isn’t enough, it falls back to an agentic workflow powered by multimodal LLMs like Gemini.

Key highlights

• Solves specific challenge types (image_label_binary, image_label_area_select: point, and image_drag_drop) using local ResNet, YOLOv8, and spatial reasoning models.
• Recently added an “Agentic Workflow” that hands advanced or novel challenges off to multimodal LLMs.
• Ships with automated CI pipelines to harvest challenge data, annotate it, and retrain the ONNX model zoo.
• Requires no Tampermonkey scripts or paid anti-captcha subscriptions.
• Integrates with Playwright-based browser automation stacks.

Caveats

• Support is uneven: image_label_area_select: bounding box and image_label_multiple_choice currently show no agent capability in the project’s own feature matrix.
• The multimodal LLM fallback is pinned to very recent model previews (e.g., Gemini 2.5 Pro), so behavior may shift as those APIs evolve.

Verdict

Grab it if you’re running Playwright scrapers and want a self-hosted, extensible hCaptcha solver. Look elsewhere if you need a guaranteed, fully supported bypass for every challenge variant today.