Search pipelines that can actually watch videos and read screenshots

What it does

Qwen3-VL-Embedding and Qwen3-VL-Reranker are a pair of retrieval models built atop the Qwen3-VL foundation. The embedding model ingests text, images, screenshots, or video—singly or mixed together—and compresses them into dense vectors. The reranker then scores query-document pairs using cross-attention, outputting a relevance probability by predicting yes and no tokens. Both come in 2B and 8B sizes and support 32K sequence lengths.

The interesting bit

Instead of treating vision and language as separate pipelines, the suite uses a shared Qwen3-VL backbone so a single request can contain multiple images, video clips, and text strings. The embedding model extracts hidden states from the base model’s last layer for independent encoding, while the reranker fuses query and document modalities with cross-attention for finer-grained alignment.

Key highlights

• Dual-tower embedding architecture for efficient large-scale retrieval, paired with a single-tower reranker for precise relevance scoring.
• Supports mixed-modal inputs—text, image, video, and screenshots—in a single dictionary-style request.
• Matryoshka Representation Learning and quantization support on embedding variants for flexible deployment dimensions.
• Instruction-aware tuning allows task-specific prompts (default: “Represent the user’s input”).
• vLLM inference support for both models, requiring version 0.14.0 or newer.

Caveats

• The provided README excerpt omits the Model Performance section, so benchmark comparisons are not visible in the sources.
• The repository is primarily a model release with Python inference wrappers; training code appears limited to published LoRA configurations.

Verdict

Worth a look if you are building multimodal RAG, video search, or cross-modal recommendation systems. Skip it if you need a general-purpose vision-language chat model—this is strictly retrieval infrastructure.