Teaching ComfyUI to read your receipts

What it does

ComfyUI-Florence2 plugs Microsoft’s Florence-2 vision foundation model into ComfyUI’s node graph. You can caption images, detect objects, segment scenes, and query documents by connecting nodes instead of writing inference scripts. A built-in loader node can also download official and community finetuned checkpoints directly from HuggingFace into the expected model path.

The interesting bit

The DocVQA node is the unusual angle here: it lets you point the model at a scanned receipt or form, ask a plain-English question like “What is the total amount?”, and get back an answer derived from the visual layout. That turns a generative-art tool into a surprisingly practical document parser.

Key highlights

• Supports official Florence-2 variants (base, large, fine-tuned) and community finetunes, including captioners optimized for Stable Diffusion 3 and Flux.
• DocVQA node extracts information from text-heavy images using natural-language prompts.
• DownloadAndLoadFlorence2Model node auto-pulls weights from HuggingFace to ComfyUI/models/LLM.
• Relies on transformers 4.38.0 or newer.

Caveats

• The README explicitly warns that DocVQA accuracy hinges on image quality and question complexity.
• The project is largely glue code between HuggingFace’s Florence-2 implementation and ComfyUI; don’t expect novel model architecture here.

Verdict

Handy if you want vision-language tasks inside ComfyUI’s visual workflow. If you’re comfortable with Python scripts or APIs, running Florence-2 directly through HuggingFace is probably cleaner.