ScaleCUA

Stale
GitHub Python Apache-2.0

Description

Open-sourced computer use agents that can operate on cross-platform environments including Windows, macOS, Ubuntu, and Android. ICLR 2026 Oral paper project.

Key Features

  • Cross-platform GUI operation: supports Windows, macOS, Ubuntu, Android, and Web environments
  • Large-scale training data: operation trajectories across 6 operating systems and 3 task domains
  • State-of-the-art performance: MMBench-GUI L1-Hard 94.4%, OSWorld-G 60.6%, WebArena-Lite-v2 47.4%
  • Two operation modes: native agentic model (single model) and agentic workflow (dual model)
  • Complete training framework: supports training from scratch based on Qwen2.5-VL and InternVL
  • Interactive playground: pre-configured virtual machine environments for Ubuntu, Android, and Web

Use Cases

💡 Automated testing: automatically execute GUI testing tasks across multiple operating system platforms
💡 Cross-platform RPA: use a single agent to execute repetitive tasks on Windows, macOS, and Android
💡 Academic research: reproduce computer use agent experiments from the ICLR 2026 Oral paper
💡 Browser automation: execute complex web interactions and data extraction in web environments
💡 Mobile automation: automatically perform app operations and UI interactions on Android devices

Tags

browser python agent automation

Categories

🌐 Browser Agent

Quick Start

1. Clone the repo: `git clone https://github.com/OpenGVLab/ScaleCUA.git && cd ScaleCUA`;
2. Install dependencies: `pip install -r requirements.txt`;
3. Deploy ScaleCUA models with vLLM (supports both native agentic and workflow modes);
4. Configure Playground environment: launch pre-configured Ubuntu/Android/Web VMs;
5. Run evaluation benchmarks: choose from `AndroidWorld`, `OSWorld`, `WebArenaLite-v2` and other test sets.
Visit GitHub

Related Projects