LLMCompiler
StaleDescription
LLM Compiler for Parallel Function Calling (ICML 2024), significantly improving agent tool calling efficiency and speed through parallel execution.
Key Features
- Automatically decomposes tasks into parallel and interdependent function calls
- Supports both open-source (LLaMA, vLLM) and closed-source (GPT) models
- Streaming mode reduces latency by dispatching tasks before full plan generation
- Configurable tool definitions with optional in-context examples
- Integrated with LangChain LangGraph and LlamaIndex ecosystems
- Azure and Friendli endpoint support for enterprise deployments
Use Cases
💡 Multi-tool AI agents requiring parallel API calls
💡 Research pipelines combining multiple data sources simultaneously
💡 Complex reasoning tasks with interdependent function calls
💡 Cost-sensitive applications needing reduced LLM inference overhead
💡 RAG systems with multiple retrieval steps
Categories
Quick Start
Clone the repo, create a conda environment with Python 3.10, install dependencies via pip install -r requirements.txt, set your OPENAI_API_KEY, then run: python run_llm_compiler.py --benchmark hotpotqa --store results.json --stream