exo
NormalDescription
exo is a framework that pools many consumer-grade devices into a distributed LLM inference cluster, turning Macs, PCs and Linux boxes into one GPU pool.
Key Features
- Consumer-device cluster — every node is a desktop or laptop, no datacenter GPUs needed
- P2P auto-discovery — devices on the same LAN join the cluster automatically
- Dynamic partitioning — models are split by layers / tensors across nodes
- Multi-model support — LLaMA, Mistral, Qwen and more on top of the cluster
- OpenAI-compatible API — call the cluster with a standard API
- Cross-platform — macOS, Windows and Linux all participate
Use Cases
💡 Pooling home/office consumer devices into an LLM inference cluster
💡 Running larger models without datacenter GPUs
💡 Providing a private, low-cost inference service for local agents
💡 Calling a home distributed LLM with an OpenAI-compatible API
Tags
Categories
Quick Start
git clone https://github.com/exo-explore/exo.git
cd exo
pip install .
exo