OpenCompass
ActiveDescription
OpenCompass is a comprehensive LLM evaluation platform supporting a wide range of models including Llama, Mistral, GPT-4, Qwen, GLM, and Claude across 100+ benchmark datasets.
Key Features
- Comprehensive benchmarking across 100+ datasets covering knowledge, reasoning, math, code and more
- Compatible with major LLMs — Llama, Mistral, GPT-4, Qwen, GLM, Claude, etc.
- CascadeEvaluator for sequential multi-evaluator pipelines on complex assessment scenarios
- Built-in GenericLLMEvaluator (LLM-as-Judge) and MATHVerifyEvaluator for math reasoning
- CompassHub leaderboard and CompassRank model ranking visualizations
- Recommended by Meta AI, integrated into the Llama Get Started workflow
Use Cases
💡 Run comprehensive multi-dimensional benchmarks on custom or fine-tuned models
💡 Compare different LLMs on specific tasks to support model selection decisions
💡 Track performance changes across model versions to detect capability regressions
💡 Build enterprise internal LLM evaluation pipelines for automated model acceptance testing
💡 Reproduce evaluation results from academic papers to verify claimed model capabilities
Categories
Quick Start
Install with pip install opencompass, configure models and datasets via OpenCompass config files, and run opencompass to start evaluation. Supports on-demand dataset loading from ModelScope, with built-in CompassHub leaderboard for result visualization.