Mapping the Messy Frontier of Multimodal RAG
This repo collects and categorizes the exploding literature on Multimodal Retrieval-Augmented Generation into a living taxonomy tied to an ACL 2025 survey paper.

What it does
The repository acts as a companion bibliography to the survey paper Ask in Any Modality, accepted at ACL 2025 Findings. It sorts papers into a detailed taxonomy covering retrieval strategies, fusion mechanisms, augmentation techniques, generation methods, and application domains like medical and fashion. It also catalogs popular datasets for image-text, video-text, and audio-text retrieval.
The interesting bit
Rather than dumping PDFs, the authors impose order on a chaotic, fast-moving field with a pipeline diagram and hierarchical taxonomies that split research by modality and task. The repo is explicitly maintained as a living resource, with updates rolling in as the literature expands.
Key highlights
- Tied to a peer-reviewed ACL 2025 Findings survey paper with a dedicated project website.
- Organizes papers across retrieval, fusion, augmentation, generation, training, and robustness.
- Catalogs datasets spanning image-text, video-text, audio-text, medical, and fashion domains.
- Includes poster and slide materials from the conference.
- Accepts community contributions via pull requests for new papers.
Caveats
- This is a curated reading list and taxonomy, not a runnable framework or codebase.
- The README is thorough but dense; finding a specific paper means navigating nested emoji-laden headers.
Verdict
Researchers and engineers trying to navigate the flood of multimodal RAG papers will find this a useful compass. If you are looking for a drop-in retrieval library, look elsewhere.
Frequently asked
- What is llm-lab-org/Multimodal-RAG-Survey?
- This repo collects and categorizes the exploding literature on Multimodal Retrieval-Augmented Generation into a living taxonomy tied to an ACL 2025 survey paper.
- Is Multimodal-RAG-Survey open source?
- Yes — llm-lab-org/Multimodal-RAG-Survey is an open-source project tracked on heatdrop.
- How popular is Multimodal-RAG-Survey?
- llm-lab-org/Multimodal-RAG-Survey has 533 stars on GitHub.
- Where can I find Multimodal-RAG-Survey?
- llm-lab-org/Multimodal-RAG-Survey is on GitHub at https://github.com/llm-lab-org/Multimodal-RAG-Survey.