Synthetic agent training data that checks its own homework

What it does

DeepFabric is a Python pipeline that generates synthetic training datasets for language models, specifically targeting agentic tool-calling behavior. It uses graph or tree-based topic generation to carve out a curriculum of subtopics, then produces samples that mix reasoning traces with structured tool calls. The output is a standard HuggingFace dataset, ready for fine-tuning frameworks like TRL, Unsloth, or Axolotl, and it includes a built-in evaluation engine that tests the trained model on held-out tasks.

The interesting bit

Instead of spraying redundant prompts and hoping for the best, DeepFabric builds a topic graph to enforce coverage without overlap, aiming to reduce overfit. It also runs generated tool calls through real execution in isolated WebAssembly environments—via a Spin service—to validate that the synthetic data actually conforms to schemas and executes correctly before it ever reaches your model.

Key highlights

• Graph-guided curriculum: Uses topic graphs/trees (mode: graph) to generate diverse, domain-anchored subtopics and avoid redundant samples.
• WASM-validated tool calls: Executes generated tool calls in an isolated WebAssembly (Spin) environment to enforce schema correctness and execution validity before training.
• MCP-native: Imports tool definitions directly from MCP (Model Context Protocol) server schemas, with built-in mocking for common utilities like file operations.
• End-to-end loop: Generates standard HuggingFace datasets, exports directly to training frameworks, and evaluates the resulting model on unseen tool-selection and parameter-accuracy tasks.
• Agentic reasoning formats: Supports chain-of-thought and ReAct-style reasoning traces, with constrained decoding to keep outputs structurally valid.

Caveats

• Extra moving parts: Validating tool calls requires a separate Spin/WebAssembly service, so the pipeline isn’t purely self-contained Python.

Verdict

Worth a look if you are building domain-specific agents and need structured, schema-bound tool-calling datasets without hand-labeling thousands of traces. Skip it if your use case is simple chat fine-tuning with no tool-calling or synthetic curriculum needs.