DAMO-NLP-SG/Video-LLaMA
An audio-visual language model that extends LLaMA-2 to understand video and audio content in response to natural language instructions.
Not currently ranked — collecting fresh signals.
star history
Video-LLaMA is an instruction-tuned multi-modal LLM that enables large language models to process video frames, audio tracks, and text simultaneously for video understanding tasks. It builds on LLaMA-2 as the language decoder and combines visual and audio encoders with a projection mechanism to bridge modalities. The model was trained using cross-modal pretraining techniques inspired by BLIP-2 and MiniGPT-4 to align visual/audio representations with the language space.
Frequently asked
- What is DAMO-NLP-SG/Video-LLaMA?
- An audio-visual language model that extends LLaMA-2 to understand video and audio content in response to natural language instructions.
- Is Video-LLaMA open source?
- Yes — DAMO-NLP-SG/Video-LLaMA is open source, released under the BSD-3-Clause license.
- What language is Video-LLaMA written in?
- DAMO-NLP-SG/Video-LLaMA is primarily written in Python.
- How popular is Video-LLaMA?
- DAMO-NLP-SG/Video-LLaMA has 3.1k stars on GitHub.
- Where can I find Video-LLaMA?
- DAMO-NLP-SG/Video-LLaMA is on GitHub at https://github.com/DAMO-NLP-SG/Video-LLaMA.