Vision Agents

Active
GitHub Python Apache-2.0

Description

Open Vision Agents by Stream. Build voice and vision agents quickly with any model or video provider, using Stream's edge network for ultra-low latency realtime interactions.

Key Features

  • Real-time video AI with YOLO, Roboflow integration combined with Gemini/OpenAI for live visual understanding
  • Ultra-low latency audio/video under 30ms via Stream's edge network with 500ms join time
  • Native SDK methods from OpenAI, Gemini, and Claude for always-latest LLM capabilities
  • Pluggable video processor pipeline supporting custom PyTorch/ONNX models alongside LLM calls
  • Multi-platform SDKs for React, Android, iOS, Flutter, React Native, and Unity
  • Production-ready with HTTP server, Prometheus metrics, horizontal scaling, and Kubernetes deployment

Use Cases

💡 Real-time video coaching applications (sports, physical therapy, workout guidance)
💡 Multi-modal AI assistants that see, listen, and respond to video streams
💡 Drone fire detection and surveillance with live visual AI processing
💡 Phone-based voice agents with video understanding via Twilio integration

Tags

voice agent python llm framework web

Categories

🌐 Browser Agent ⚡ Agent Tools

Quick Start

Install: uv add vision-agents. Get a Stream API key. Follow the quickstart guide at visionagents.ai to build your first voice or video agent.
Visit GitHub

Related Projects