A field guide to where knowledge graphs meet vision and language

What it does

KG-MM-Survey is essentially a curated literature index and companion to an arXiv survey paper. It collects and categorizes research at the intersection of knowledge graphs and multi-modal learning, splitting the field into two directions: KG4MM, where structured knowledge improves tasks like visual question answering and image generation, and MM4KG, where multi-modal data expands traditional graph construction, completion, and reasoning. The maintainers also use it to announce their own related work accepted at AAAI, ACL, and SIGIR.

The interesting bit

Rather than dumping papers into a single list, the repo imposes a full task taxonomy—complete with benchmark tables and pipeline diagrams—that treats the intersection as a two-way street. It is as much an attempt to formalize the field’s vocabulary as it is a bibliography.

Key highlights

• Covers both directions: KGs supporting multi-modal tasks (KG4MM) and multi-modal data supporting KGs (MM4KG)
• Organizes papers into granular task categories: visual question answering, cross-modal retrieval, entity alignment, multi-modal relation extraction, and more
• Includes visual taxonomies and benchmark tables (e.g., VQA benchmarks, image classification comparisons)
• Maintained by an active research group with recent publications at AAAI, COLING, and SIGIR
• Serves as the living companion to the 2024 survey paper Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

Caveats

• The repository is a curated reading list and paper tracker, not a codebase or framework—expect markdown and PDF links, not installable tools
• The “News” section is heavily weighted toward the maintainers’ own lab publications, which can blur the line between neutral survey and group showcase

Verdict

Worth bookmarking if you are doing literature review at the intersection of structured knowledge and computer vision. Skip it if you are looking for reusable code or datasets; the value here is the map, not the machinery.