Jina AI Serve

Stale
GitHub Python Apache-2.0

Description

Jina AI Serve is a cloud-native framework for building multimodal AI applications, supporting RAG pipelines, agent systems, and multimodal search.

Key Features

  • Multi-protocol support — Build AI services communicating via gRPC, HTTP, and WebSockets
  • LLM streaming output — Token-by-token streaming responses for real-time interactive scenarios
  • Built-in containerization — Native Docker integration and Executor Hub for one-click push and deploy
  • Elastic scaling — Replicas, sharding, and dynamic batching from local dev to production
  • Kubernetes and cloud deployment — Export to K8s YAML or Docker Compose, or one-click deploy to Jina AI Cloud
  • Multimodal data processing — DocArray-based handling with native support for major ML frameworks

Use Cases

💡 Build and deploy multimodal AI microservices
💡 Set up RAG pipelines for semantic search and document Q&A
💡 Create LLM inference services with streaming output
💡 Deploy AI models to cloud production with a single command

Tags

multimodal rag cloud-native ai-framework python

Categories

⚡ Agent Tools 📚 RAG Tools

Quick Start

pip install jina

from jina import Executor, requests, Deployment
from docarray import DocList, BaseDoc

class Prompt(BaseDoc):
    text: str

class MyExecutor(Executor):
    @requests
    def gen(self, docs: DocList[Prompt]):
        return docs

dep = Deployment(uses=MyExecutor, port=12345)
with dep:
    dep.block()
Visit GitHub Visit Website

Related Projects