camel-ai/crab
A Python framework for building and running benchmark environments to evaluate multimodal LLM agents.
Not currently ranked — collecting fresh signals.
star history
CRAB provides a framework for creating standardized benchmarks to assess language model agents in cross-platform environments. It allows defining agent tasks through Python decorators and includes a novel graph-based evaluation methodology. The framework supports deploying agent environments via Docker, virtual machines, or in-memory processes while maintaining a unified interface for evaluation.
Frequently asked
- What is camel-ai/crab?
- A Python framework for building and running benchmark environments to evaluate multimodal LLM agents.
- Is crab open source?
- Yes — camel-ai/crab is an open-source project tracked on heatdrop.
- What language is crab written in?
- camel-ai/crab is primarily written in Python.
- How popular is crab?
- camel-ai/crab has 424 stars on GitHub.
- Where can I find crab?
- camel-ai/crab is on GitHub at https://github.com/camel-ai/crab.