AxonHub

Active
GitHub Go NOASSERTION

Description

Open-source AI gateway that lets any SDK call 100+ LLMs. Built-in failover, load balancing, cost controls, and end-to-end tracing for production AI agent workloads.

Key Features

  • Unified LLM gateway - call 100+ models through OpenAI and Anthropic compatible protocols
  • Automatic failover - switch to backup providers when the primary one is down
  • Smart load balancing - distribute requests across model instances to avoid overload
  • Cost governance - budgets, quotas, and per-token price alerts per project or team
  • End-to-end tracing - full request traces with OpenTelemetry integration
  • Pluggable auth - API keys, OIDC, and team SSO supported

Use Cases

💡 Unifying access to multiple LLM providers so apps don't reimplement provider switching
💡 Powering budget, quota, and cost dashboards for multi-team AI agent platforms
💡 Load balancing between local and hosted models to avoid single-vendor lock-in
💡 Adding end-to-end tracing to LLM calls so agent failures are easy to diagnose
💡 Connecting private deployments and public cloud models through one SDK

Tags

agent-tools llm-gateway observability cost-control sdk

Categories

⚡ Agent Tools 📊 Observability

Quick Start

# Start AxonHub with Docker
docker run -d --name axonhub -p 8080:8080 \
  -e AXONHUB_MASTER_KEY=sk-xxx \
  looplj/axonhub:latest

# Call any model through the OpenAI-compatible protocol
curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer sk-xxx" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-3-5-sonnet","messages":[{"role":"user","content":"Hello"}]}'
Visit GitHub

Related Projects