Arize Phoenix
ActiveDescription
Phoenix is an open-source observability and evaluation tool for LLM and agent applications, supporting online tracing and offline diagnosis.
Key Features
- OpenTelemetry-based tracing for LLM application runtime observability
- LLM-powered evaluation for response and retrieval quality benchmarking
- Versioned datasets for experimentation, evaluation, and fine-tuning
- Prompt management with version control, tagging, and experimentation
- Playground to optimize prompts, compare models, and replay traced calls
- Built-in PXI agent for debugging traces and navigating Phoenix
Use Cases
💡 Trace and debug LLM calls across LangChain, LlamaIndex, and OpenAI SDK
💡 Evaluate RAG pipeline retrieval quality with built-in evals
💡 Compare prompt versions and model variants systematically
💡 Monitor production LLM performance and detect regressions
💡 Manage prompt libraries with version control and A/B testing
Strengths & Limitations
✅ Strengths
- • Actively maintained, recent updates
- • High community interest (11.1k stars)
- • Established track record (3 years in production)
⚠️ Limitations
- • High issue backlog (925 open issues)
- • No clear open-source license
Tags
Categories
Quick Start
pip install arize-phoenix → import phoenix as px → px.launch_app() → open http://localhost:6006 → instrument your LLM code with OpenTelemetry