Typed extraction recipes for vision models

What it does

VLM Run Hub is a catalog of ready-made Pydantic models for extracting structured data from unstructured visuals—think invoices, driver’s licenses, medical insurance cards, and TV news frames. Instead of hand-rolling JSON prompts for every new document type, you import a schema, hand it to a VLM that supports structured outputs, and get back validated Python objects. It covers domains from healthcare and finance to retail and logistics, with each schema intended to plug directly into visual ETL pipelines.

The interesting bit

The hub treats Pydantic not just as validation glue but as the portable contract between you and any VLM provider. Because the schemas are plain Python classes, they work with OpenAI’s structured outputs, Gemini, or the project’s own VLM Run SDK without rewriting prompts—model-agnostic typing by way of inheritance and mypy.

Key highlights

• Catalog spans document processing, accounting, healthcare, retail, media, and aerospace, organized in a YAML registry (catalog.yaml).
• Each schema ships with built-in Pydantic validation, so extracted fields arrive coerced and checked rather than raw LLM text.
• Designed for model-agnostic use: the same Invoice or USDriversLicense class can move between providers.
• Includes contribution guidelines (SCHEMA-GUIDELINES.md) for adding new industry domains.
• Optimized for visual ETL—purpose-built to bridge raw images, video, or PDFs into downstream structured workflows.

Verdict

Worth bookmarking if you are building document-parsing or multimodal extraction pipelines and are tired of maintaining brittle prompt templates. Skip it if you are only doing free-form image captioning or already have a locked-down internal schema library.