LlamaFactory
ActiveDescription
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs. Supports LoRA, QLoRA, RLHF and more for building custom agent models.
Key Features
- 100+ model support - covers LLaMA, Qwen3, DeepSeek, Gemma, GLM, Mistral, Phi and over 100 large language models
- Full training pipeline - supports pre-training, SFT, RLHF, DPO, KTO, ORPO, PPO and complete training workflows
- Efficient fine-tuning - supports 16-bit full tuning, LoRA, QLoRA (2-8 bit) plus GaLore, BAdam, APOLLO and other advanced algorithms
- Multimodal training - supports image understanding, visual grounding, video recognition, audio understanding task fine-tuning
- LLaMA Board GUI - zero-code Gradio-based web UI for visual training configuration and monitoring
- vLLM/SGLang inference - deploy fine-tuned models directly as OpenAI-compatible API services
Use Cases
💡 Researchers fine-tuning open-source LLMs for domain adaptation in healthcare, legal, finance and other verticals
💡 Enterprises building private LLM services by customizing models on consumer GPUs via LoRA/QLoRA
💡 Training agent-specific models by fine-tuning on tool-use and multi-turn dialogue datasets for agent capabilities
💡 Multimodal agent development by fine-tuning vision-language models for image understanding and visual grounding
💡 Model comparison experiments using the unified framework to quickly evaluate different training strategies and hyperparameters
Strengths & Limitations
✅ Strengths
- • Actively maintained, recent updates
- • High community interest (74.1k stars)
- • Permissive open-source license (Apache-2.0)
- • Established track record (3 years in production)
Categories
Quick Start
# Install LLaMA Factory
pip install llamafactory
# Fine-tune with CLI (LoRA example)
llamafactory-cli train \
--model_name_or_path meta-llama/Llama-3-8B-Instruct \
--dataset alpaca_en_demo \
--finetuning_type lora \
--output_dir output/llama3-lora
# Or launch the Web UI
llamafactory-cli webui