Teaching vision models to write SVG, not trace curves

What it does

StarVector is a multimodal vision-language model that generates Scalable Vector Graphics code. It accepts an image or a text prompt and outputs raw SVG markup, effectively treating graphic design as a programming language. The project provides 1B and 8B parameter checkpoints, training datasets, and a benchmark suite for evaluation.

The interesting bit

Traditional vectorizers trace curves and often miss semantics; StarVector leverages its StarCoder backbone to reason about shapes, primitives, and structure directly in code space. This lets it use actual SVG primitives like ellipses and polygons rather than approximating everything with paths, producing more compact and accurate files.

Key highlights

• Outperforms conventional tracers (Potrace, VTracer) and even GPT-4-V on the SVG-Bench benchmark using the DinoScore metric.
• Handles three tasks: Image-to-SVG, Text-to-SVG, and diagram generation.
• Trained on SVG-Stack, a curated dataset of 2.1 million samples covering icons, fonts, emoji, and diagrams.
• Explicitly warns it is not trained for natural photos or illustrations; it targets icons, logotypes, charts, and technical diagrams.

Caveats

• The model is strictly limited to synthetic graphics; the authors explicitly note it fails on natural images and illustrations.
• Integration with HuggingFace requires trust_remote_code=True, indicating the architecture is not yet a native part of the standard transformers library.

Verdict

Worth a look if you need to batch-convert icons, logos, or diagrams into clean, editable SVG markup. Skip it if your pipeline handles photographs or complex illustrations, as the authors explicitly note those are out of scope.