Speech-to-Speech

Active
GitHub Python Apache-2.0

Description

Build local voice agents with open-source models. An end-to-end speech-to-speech pipeline from HuggingFace for fully local voice AI agent deployment.

Key Features

  • Fully open modular speech-to-speech pipeline: VAD → STT → LLM → TTS
  • Multiple STT backends: Whisper, Parakeet TDT (sub-100ms), Lightning Whisper MLX
  • Flexible LLM integration: HuggingFace Transformers, mlx-lm, OpenAI API
  • Multiple TTS engines: ChatTTS, Pocket TTS (voice cloning), Kokoro-82M, Qwen3-TTS
  • Four deployment modes: Realtime WebSocket, Server/Client, WebSocket, and Local
  • Optimized Apple Silicon support with MLX acceleration for all pipeline stages

Use Cases

💡 Building local voice AI agents without cloud dependencies
💡 Real-time conversational AI with low-latency speech processing
💡 Custom voice agent development with swappable model components
💡 Privacy-sensitive voice applications with fully offline deployment
💡 Prototyping speech-to-speech systems with rapid model experimentation

Tags

voice python llm agent tools

Categories

💬 Chat UI ⚡ Agent Tools

Quick Start

Install: pip install speech-to-speech. Run: speech-to-speech (defaults to Parakeet TDT + OpenAI LLM + Qwen3-TTS). For local Mac: speech-to-speech --local_mac_optimal_settings. Optional backends: pip install "speech-to-speech[kokoro]". Set OPENAI_API_KEY for cloud LLM.
Visit GitHub

Related Projects