OpenLLMetry
ActiveDescription
OpenLLMetry is an open-source observability tool for LLM applications based on OpenTelemetry, providing tracing, metrics, and monitoring capabilities.
Key Features
- OpenTelemetry-Based Observability — built on OpenTelemetry extensions for complete LLM application tracing and monitoring
- 20+ LLM Provider Instrumentation — supports OpenAI, Anthropic, Cohere, Google Gemini, Mistral, Ollama, HuggingFace, and more
- Vector DB Tracing — instrument Chroma, Pinecone, Qdrant, Weaviate, Milvus, LanceDB, and Marqo
- Framework Support — auto-instrument LangChain, LlamaIndex, Crew, Agno, and other agentic frameworks
- 25+ Destination Integrations — connect to Datadog, Grafana, New Relic, Sentry, Splunk, Honeycomb, and others
- Traceloop SDK — simple one-line init (Traceloop.init()) for instant tracing with standard OpenTelemetry data output
Use Cases
💡 Add end-to-end tracing to LLM applications with minimal code changes
💡 Monitor model performance, latency, and token usage across providers
💡 Debug agentic workflows by tracing tool calls and chain execution
💡 Export traces to existing observability stacks like Datadog or Grafana
💡 Track vector database queries and retrieval performance in RAG pipelines
Strengths & Limitations
✅ Strengths
- • Actively maintained, recent updates
- • High community interest (7.4k stars)
- • Permissive open-source license (Apache-2.0)
- • Established track record (2 years in production)
⚠️ Limitations
- • High issue backlog (640 open issues)
Categories
Quick Start
Install with pip install traceloop-sdk, then add Traceloop.init() to your code. For local debugging, use Traceloop.init(disable_batch=True). Traces are automatically exported to your configured destination via OpenTelemetry.