Open-AutoGLM
StaleDescription
Open-AutoGLM is an open phone agent model and framework enabling AI to autonomously operate smartphone interfaces, unlocking the AI Phone experience for everyone.
Key Features
- Multimodal screen understanding — perceive smartphone screen content via vision-language models (VLM) to understand current interface state
- ADB/HDC device control — control Android and HarmonyOS devices through Android Debug Bridge and HDC protocols
- Natural language task planning — user describes requirements in natural language, system auto-parses intent and plans operation flows
- Sensitive operation confirmation — pauses and requests human takeover for login, captcha, and similar scenarios
- Multi-model deployment support — works with SGLang, vLLM inference engines, local deployment or Zhipu/ModelScope cloud services
- Cross-platform compatibility — supports Android 7.0+, HarmonyOS NEXT, and iPhone (via WebDriverAgent)
Use Cases
💡 Smartphone task automation: auto-open apps, search products, compare prices, and handle daily tasks
💡 Cross-app information retrieval: switch between multiple apps to collect and compare information
💡 Automated GUI testing: perform automated regression testing on mobile applications
💡 Accessibility assistance: complete complex smartphone operations through voice commands for users with disabilities
💡 Data collection and entry: auto-fill forms and submit data within mobile apps
Categories
Quick Start
```bash
pip install -r requirements.txt
pip install -e .
adb devices
python main.py --base-url https://open.bigmodel.cn/api/paas/v4 --model "autoglm-phone" --apikey "your-key" "open Meituan and search for nearby hotpot restaurants"
```