LiteLLM
ActiveDescription
LiteLLM provides a unified interface and proxy gateway for LLM calls, simplifying multi-model switching, routing, and cost control.
Key Features
- Unified LLM interface — OpenAI-format calls across 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, etc.)
- AI Gateway proxy server — Self-hosted proxy with virtual keys, spend tracking, load balancing, and admin dashboard
- Python SDK — Switch LLM providers in code via the completion() function
- MCP tool bridging — Connect MCP servers to any LLM with tool calling and protocol conversion
- A2A Agent protocol — Agent-to-Agent communication connecting LangGraph, Vertex AI, and other agent frameworks
- 8ms P95 latency — Ultra-low latency at 1k RPS, production-ready deployment
Use Cases
💡 Providing a unified LLM gateway for teams with centralized API key and cost management
💡 Seamlessly switching between LLM providers to avoid vendor lock-in
💡 Providing standardized model calling interfaces for AI agent frameworks
💡 Monitoring and optimizing LLM call costs and performance metrics
Tags
Categories
Quick Start
# Install LiteLLM
pip install litellm
# Set environment variables
export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
# Use unified interface to call different models
from litellm import completion
# Call OpenAI
response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
# Call Anthropic
response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])
# Start AI Gateway
# litellm --model gpt-4o