text-generation-webui
NormalDescription
The original local LLM interface supporting text generation, vision, tool-calling, and training with both a web UI and API. Runs 100% offline and private.
Key Features
- Multiple backends — Integrates llama.cpp, ExLlamaV3, Transformers, TensorRT-LLM with hot-swapping between backends
- OpenAI/Anthropic-compatible API — Provides Chat, Completions, and Messages endpoints as a local drop-in replacement
- Vision multimodal — Supports image attachments for visual understanding, plus PDF and DOCX file uploads for content chat
- Tool calling — Models can call custom functions during chat (web search, page fetch, math) with MCP server support
- LoRA training — Fine-tune LoRAs on multi-turn chat or raw text datasets with interrupted run resumption
- 100% offline privacy — Zero telemetry, zero external resources, zero remote update requests; runs entirely locally
Use Cases
💡 Local LLM deployment: Run and switch between different open-source LLMs on personal computers with full data privacy
💡 AI application development: Provide local LLM inference via OpenAI-compatible API for custom applications
💡 Model fine-tuning: Use LoRA to fine-tune models on domain-specific data for improved performance on specialized tasks
💡 Visual understanding apps: Leverage multimodal capabilities to analyze images and documents for document Q&A systems
Categories
Quick Start
# Clone the repository
git clone https://github.com/oobabooga/textgen.git && cd textgen
# Create virtual environment
python -m venv venv && source venv/bin/activate
# Install dependencies
pip install -r requirements/portable/requirements.txt --upgrade
# Launch server with API enabled
python server.py --portable --api --auto-launch