Firecrawl

Active
GitHub TypeScript AGPL-3.0

Description

Firecrawl is the Web Data API for AI, turning web pages into clean, structured, LLM-friendly data with crawl, scrape, and search capabilities.

Key Features

  • Search endpoint — Search the web and get full page content from results in a single call
  • Scrape endpoint — Convert any URL to clean Markdown, HTML, screenshots, or structured JSON
  • Interact endpoint — Scrape a page then interact via AI prompts or code (click, search, navigate)
  • Agent mode — Describe what you need and automatically gather data through multi-step tasks
  • Crawl/Map/Batch — Crawl entire sites, discover all URLs instantly, batch scrape thousands of pages async
  • MCP integration — Connect Firecrawl as a web data source for AI agents or MCP clients with one command

Use Cases

💡 Feed real-time web data into RAG pipelines by injecting scraped content into vector databases
💡 Build competitive monitoring systems that periodically scrape and analyze competitor pricing and products
💡 Provide web browsing capabilities to AI agents for autonomous online search and information extraction
💡 Bulk extract e-commerce product data, converting product pages into structured JSON for analysis
💡 Build content aggregation platforms that auto-collect and curate latest news from multiple websites

Tags

web-scraping mcp rag data-extraction typescript

Categories

🔌 MCP Tools 📚 RAG Tools

Quick Start

# Install Python SDK
pip install firecrawl-py

# Initialize client and scrape a page
from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")
result = app.scrape("https://example.com")
print(result.markdown)
Visit GitHub Visit Website

Related Projects