HomeProjectsArticlesPK BoardAbout
Explore Projects

Unstructured

Active
GitHub HTML Apache-2.0

Description

Unstructured provides document parsing and cleaning capabilities, commonly used in RAG ingestion and preprocessing pipelines.

Key Features

  • Open-source document parsing for PDFs, HTML, Word docs, and more
  • Modular partitioning functions for text extraction and structure detection
  • Docker support with multi-platform images for x86_64 and Apple Silicon
  • Integration-ready for RAG ingestion and preprocessing pipelines
  • Supports images, tables, and complex document layouts
  • PyPI installable with local development setup

Use Cases

💡 Preprocessing unstructured documents for LLM ingestion
💡 Building RAG pipelines that need reliable document parsing
💡 Extracting text and tables from PDFs for downstream analysis
💡 Automating data preprocessing in AI/ML workflows
💡 Converting mixed document formats into structured outputs

Strengths & Limitations

✅ Strengths

  • • Actively maintained, recent updates
  • • High community interest (15.3k stars)
  • • Permissive open-source license (Apache-2.0)
  • • Established track record (3 years in production)

Tags

rag document-processing ingestion python

Categories

📚 RAG Tools

Quick Start

1. Pull the Docker image: `docker pull downloads.unstructured.io/unstructured-io/unstructured:latest`.
2. Or install from PyPI: `pip install unstructured`.
3. Run partitioning on your documents using the `partition` function.
4. Use the structured output in your RAG or LLM pipeline.
Visit GitHub Visit Website View Docs

Related Projects

Sparrow

5.2k · Python
Active A

Sparrow is a structured data extraction tool that supports instruction calling with ML, LLM, and Vision LLM for extracting structured information from documents, suitable for document parsing in RAG pipelines.

data-extractiondocument-processingllm +3
  • · Extract structured data from invoices, receipts, statements, forms and tables
  • · Pluggable architecture mixing Vision LLM, Text LLM, and Agent pipelines
  • · Multiple backends: MLX (Apple Silicon), vLLM (NVIDIA), Ollama, Hugging Face
View full details → View on GitHub

RAGatouille

4.0k · Python
Stale C

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.

ragpythonembedding +1
  • · ColBERT late-interaction retrieval: wraps advanced ColBERT models into simple APIs, enabling late-interaction retrieval methods in RAG pipelines without deep IR research knowledge
  • · End-to-end training and fine-tuning: built-in RAGTrainer and TrainingDataProcessor with automatic deduplication, positive/negative pairing and hard negative mining, supporting training from scratch or fine-tuning pretrained models
  • · Modular composable architecture: DataProcessor, NegativeMiner and other components can be used independently, with support for custom NegativeMiner integration into training pipelines
View on GitHub

MemAgent

1.1k · Python
Stale B

A MemAgent framework that can extrapolate to 3.5M context tokens, along with a training framework for RL training of any agent workflow.

memoryagentrag +2
  • · Ultra-Long Context Processing: Extrapolate from 8K training context to 3.5M tokens with performance loss under 5%
  • · Reinforcement Learning Driven: Trained with RLVR (Reinforcement Learning from Verifiable Rewards), extends DAPO algorithm for end-to-end multi-turn conversation optimization
  • · Linear Time Complexity: Breaks through computational bottlenecks in long-text processing with linear resource scaling
View on GitHub

Related Articles

QdrantRAGVector Database

Qdrant + RAG Retrieval Optimization Guide: From Recall to Answer Quality

Production-focused best practices for index design, filtering, reranking, and evaluation when building RAG retrieval layers with Qdrant.

Read more →