ShowUI
NormalDescription
ShowUI is an open-source, end-to-end Vision-Language-Action model for GUI agents and computer use, capable of understanding screenshots and executing precise interface interactions.
Key Features
- End-to-end Vision-Language-Action model for GUI interaction
- UI-Guided Token Selection reduces visual tokens from 1296 to ~167
- Supports grounding, navigation, and computer use tasks
- Compatible with Qwen2-VL and Qwen2.5-VL base models
- Training pipeline with DeepSpeed, QLoRA, FlashAttention2
- API calling via Gradio Client without local GPU requirement
Use Cases
💡 Automated GUI testing and quality assurance
💡 Desktop computer use agents for task automation
💡 Cross-platform UI navigation and interaction
💡 Building custom GUI agents for specific applications
💡 Research on vision-language models for interface understanding
Categories
Quick Start
Install dependencies, download the ShowUI-2B model from HuggingFace, run inference via inference_vllm.ipynb or call the API with python3 api.py -- screenshot and query are inputs, no local GPU needed for API mode