microsoft/SoM
SoM is a visual prompting technique that overlays spatial marks on images to improve GPT-4V's visual grounding capabilities.
Not currently ranked — collecting fresh signals.
star history
The Set-of-Mark (SoM) prompting method overlays visual markers on images to unlock spatial and visual understanding in large multimodal models like GPT-4V. The project releases a Python toolbox for generating SoM prompts, a GPT-4V demo integration, a vision benchmark for evaluation, and an open-source SoM-LLaVA implementation for empowering other open-source MLLMs with this technique.
Frequently asked
- What is microsoft/SoM?
- SoM is a visual prompting technique that overlays spatial marks on images to improve GPT-4V's visual grounding capabilities.
- Is SoM open source?
- Yes — microsoft/SoM is open source, released under the MIT license.
- What language is SoM written in?
- microsoft/SoM is primarily written in Python.
- How popular is SoM?
- microsoft/SoM has 1.5k stars on GitHub.
- Where can I find SoM?
- microsoft/SoM is on GitHub at https://github.com/microsoft/SoM.