Ollama

Active
GitHub Go MIT

Description

Local LLM runner: open-source models callable as a single CLI binary.

Key Features

  • One-command run — `ollama run llama3` drops into a chat
  • Model registry — Llama, Mistral, Qwen, Gemma built-in
  • OpenAI compatible — Serves /v1/chat/completions
  • Multimodal — Supports vision models like LLaVA
  • Resource-aware — CPU, Metal, CUDA auto-scheduling

Use Cases

💡 Provide LLM inference backend for local agents.
💡 Run lightweight models in CI for unit tests.
💡 Run open-source models for privacy-sensitive scenarios.

Tags

llm local inference agent-tools

Categories

⚡ Agent Tools

Quick Start

# Install
brew install ollama
# Start the service
ollama serve &
# Pull a model and chat
ollama pull llama3
ollama run llama3 'Describe Rust in one sentence'
Visit GitHub

Related Projects