What it does AI-Map is a thin Python wrapper around Oxylabs’ hosted API. You pass a starting URL and a natural-language prompt—“job ad pages,” “all blog posts”—and get back a JSON list of matching URLs. The actual crawling and AI interpretation happen on Oxylabs’ servers; this repo is mostly client code and documentation.

The interesting bit The pitch is replacing crawl logic with intent. Instead of hand-crafting sitemap parsers or CSS selectors, you describe what you want and let the service figure out where it lives. Whether the AI is doing something genuinely clever or just wrapping a conventional crawler with an LLM front-end is unclear from the README—the “how” is opaque.

Key highlights

• Hosted API; requires Oxylabs account and API key (free trial: 1,000 credits)
• Optional JavaScript rendering for SPAs; geo-location proxying via ISO2 codes
• Returns simple JSON URL lists, easy to pipe into downstream scrapers
• Companion JavaScript SDK available; this repo is Python-only
• Pricing starts at $12/month for 3,000 credits post-trial

Caveats

• No access to auth-gated, paywalled, or private-network pages
• The README doesn’t explain mapping depth limits, rate limits, or credit costs per request
• “AI” specifics (model used, reasoning approach) are undisclosed

Verdict Worth a spin if you need quick URL discovery without maintaining crawl infrastructure and don’t mind a SaaS dependency. Skip it if you need on-premise control, transparent crawling logic, or heavy-volume scraping where per-credit pricing matters.