OpenInference
ActiveDescription
OpenTelemetry instrumentation for AI observability, providing standardized tracing, metrics collection, and span definitions for LLM inference processes to help developers monitor and debug AI agent systems.
Key Features
- OpenTelemetry-based instrumentation spec for tracing LLM inference and application context
- Instrumentation libraries for 15+ ML frameworks: OpenAI, LlamaIndex, LangChain, DSPy, CrewAI, Agno, and more
- Semantic conventions for standardized span attributes across LLM applications
- Native integration with Arize Phoenix for LLM observability and evaluation
- Compatible with any OpenTelemetry-compatible backend (Jaeger, Zipkin, Grafana Tempo)
- Supports both Python and JavaScript/TypeScript instrumentation packages
Use Cases
💡 Monitoring LLM application performance and latency across production deployments
💡 Debugging multi-agent systems by tracing individual LLM calls and tool invocations
💡 Evaluating RAG pipeline quality by tracking retrieval and generation spans
💡 Building observability dashboards for AI agent workflows using OpenTelemetry backends
💡 Comparing model performance and token usage across different providers and configurations
Categories
Quick Start
Install the appropriate instrumentation package for your framework (e.g., `pip install openinference-instrumentation-openai`), configure an OpenTelemetry exporter to your backend, and wrap your LLM calls with the provided tracer. Results are viewable in Arize Phoenix or any OTel-compatible UI.