Lighteval
ActiveDescription
HuggingFace's all-in-one toolkit for evaluating LLMs across multiple backends, deeply integrated with the HuggingFace ecosystem and providing flexible evaluation metrics and benchmark configuration.
Related Projects
LM Evaluation Harness
12.2k · Python
A framework for few-shot evaluation of language models by EleutherAI, providing standardized evaluation pipelines supporting hundreds of benchmark tasks and widely adopted as a core LLM evaluation tool in the community.
llm-evaluationbenchmarkevaluation-framework +2
Inspect AI
1.9k · Python
A framework for large language model evaluations developed by the UK AI Safety Institute (AISI), providing comprehensive model capability assessment tools with support for safety and alignment testing.
llm-evaluationai-safetyevaluation-framework +2
Opik
18.9k · Python
Opik is an open-source LLM observability platform providing agent tracing, evaluation testing, and prompt experiment management to help developers monitor and optimize AI agent systems.
observabilityllm-evaluationtracing +2
Harbor
1.5k · Python
Framework for running agent evaluations and creating RL environments to measure and improve agent performance
evaluationbenchmarkrl-environments +2