An AI audit intern that actually read the standards

What it does

AutoAudit is a Python FastAPI application that turns regulatory frameworks like COBIT, ISO27001, and SOX into a conversational audit assistant. Users chat with an agent, run risk assessments, and check compliance through a web interface, while a Neo4j knowledge graph and a vector RAG system attempt to keep the LLM grounded in actual standards rather than hallucinated ones. It also advertises model fine-tuning capabilities via SFT, RLHF, and LoRA.

The interesting bit

The project treats “agentic RAG” as a first-class citizen, trying to combine retrieval-augmented generation with reinforcement learning feedback loops inside an audit context—an unusual mix of corporate governance and experimental LLM plumbing. Whether that pipeline is fully fleshed out or aspirational is hard to tell from the single-file module names, but the ambition is unmistakably there.

Key highlights

• Conversational audit agent built on LangChain with multi-turn context.
• Knowledge graph integration for COBIT, ISO27001, and SOX standards using Neo4j.
• Agentic RAG with vector search and semantic retrieval to reduce LLM drift.
• Advertised training pipeline supporting SFT, RLHF, and LoRA fine-tuning.
• Web UI and REST API for chat, audit analysis, knowledge management, and model training.

Caveats

• Heavy external dependency stack: requires MySQL, Neo4j, a Qwen API key, and HuggingFace model downloads to function.
• The project structure suggests thin wrappers around large frameworks (audit_agent.py, agentic_rag.py, training_pipeline.py), so it may be more integration glue than novel engine.
• Performance and accuracy claims (85%+ accuracy, <3 s latency, 99.9% uptime) appear in the README without disclosed methodology or benchmarks.

Verdict

Worth a look if you are building LLM tools for compliance or financial audit and want a reference architecture for wiring together LangChain, Neo4j, and FastAPI. Skip it if you need a production-hardened audit platform with proven traceability and model transparency.