vLLM
活跃简介
A high-throughput and memory-efficient inference and serving engine for LLMs, featuring PagedAttention, continuous batching, and optimized KV cache management for production deployments.
相关项目
OpenLLM
12.3k · Python
Run any open-source LLMs such as DeepSeek and Llama as OpenAI-compatible API endpoints in the cloud. Supports fine-tuning, quantization, and distributed inference for production-grade LLM deployment.
llmpythonapi +3
OpenRAG
3.9k · Python
一站式检索增强生成（RAG）平台，集成 Langflow、Docling 和 OpenSearch，提供从文档解析到向量检索再到生成的完整流水线，支持多种模型和向量数据库。
ragllmframework +2
WrenAI
15.0k · TypeScript
Open-source text-to-SQL and text-to-chart GenBI agent with a semantic layer. Ask your database questions in natural language and get accurate SQL, charts, and BI insights. Supports 12+ data sources and any LLM.
llmtypescriptagent +2
LangExtract
35.7k · Python
A Python library by Google for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization, designed for data annotation and knowledge extraction workflows.
data-processingllmpython +2