Tencent’s WeMM embeds images, video, and text into shrinkable vectors

What it does WeMM-Embedding is a family of dedicated multimodal encoders—2B, 4B, and 9B parameters—that map text, images, videos, and visual documents into a single normalized vector space. Unlike generative vision-language models repurposed for retrieval, it is built solely to produce embeddings via a special <embedding> token plucked from the last hidden state. The models support Matryoshka representation learning, meaning you can truncate an embedding to as few as 64 dimensions, renormalize, and still retain nearly all of the full-dimension performance on image and video tasks.

The interesting bit The 2B model reportedly keeps 98.7% of its full image-and-video performance when sliced down to 256 dimensions on MMEB-v2, which is the kind of compression that actually matters for storage and bandwidth. The project also ships with a complete evaluation harness for MMEB-v3 that is essentially a minimal fork of the VLM2Vec pipeline, retrofitted for multi-node multi-GPU inference and 64-frame video sampling—suggesting the authors care about reproducible benchmarks as much as model weights.

Key highlights

• Three model sizes (2B, 4B, 9B) with Matryoshka dimension options up to 4096.
• Tops the reported MMEB-v2 and MMEB-v3 leaderboards against open and closed-source rivals like Qwen3-VL-Embedding and GME, at least in the tables provided.
• Supports inference via transformers, sentence-transformers, vLLM, and SGLang.
• Apache 2.0 license for Tencent-authored code.

Caveats

• Audio input is explicitly unsupported, so it is not truly “omni-modal.”
• The evaluation code is a patched version of the VLM2Vec pipeline; if you are not benchmarking on MMEB, you are mostly on your own.
• The README pins transformers==5.2.0 for reproducibility, which may conflict with newer toolchains.

Verdict Worth a look if you are building cross-modal search, recommendation, or RAG pipelines and want one encoder instead of a bouquet of specialists. Skip it if you need audio embeddings or a generative model that answers questions.