HomeProjectsArticlesPK BoardAbout
Explore Projects

Airflow

Active
GitHub Python Apache-2.0

Description

Apache's workflow orchestration platform, the industry-standard ETL solution.

Key Features

  • DAG model — Explicit dependencies
  • Rich operators — 200+ built-in
  • Extensible — Custom operators are easy
  • K8s integration — Native KubernetesExecutor
  • Visual — Task state at a glance

Use Cases

💡 Enterprise ETL orchestration for data teams.
💡 Schedule and dependency management for LLM workflows.
💡 Model training and release pipelines for ML.

Strengths & Limitations

✅ Strengths

  • • Actively maintained, recent updates
  • • High community interest (46.5k stars)
  • • Permissive open-source license (Apache-2.0)

Tags

workflow data-pipeline k8s

Categories

🔄 Workflow

Quick Start

# Install
pip install apache-airflow
# Init DB
airflow db init
# Start webserver and scheduler
airflow webserver -p 8080 &
airflow scheduler
Visit GitHub

Related Projects

Cultivation World Simulator

2.0k · Python
Active A

An open-source Cultivation World Simulator using Agentic Workflow to create a dynamic, emerging Xianxia world. Showcases multi-agent collaboration in complex scenarios.

multi-agentworkflowagent +1
  • · Every NPC is an independent LLM-driven agent with unique personality, memory, relationships, and decision-making logic
  • · Emergent storytelling with no preset scripts — sect wars, rivalries, and character arcs arise from world rules and AI interactions
  • · Rich cultivation world system including spirit roots, realms, techniques, sects, alchemy, artifacts, and martial tournaments
View on GitHub

make-it-heavy

1.1k · Python
Stale C

A Python framework that emulates Grok Heavy functionality using intelligent multi-agent orchestration. Deploy 4 (or more) specialized AI agents in parallel to deliver comprehensive, multi-perspective analysis on any query.

multi-agentworkflowpython +1
  • · Grok Heavy emulation: multi-agent system delivering deep, comprehensive multi-perspective analysis
  • · Parallel intelligence: deploy 4 (configurable) specialized AI agents simultaneously for different analytical perspectives
  • · Dynamic question generation: AI automatically creates customized research sub-questions from user queries
View on GitHub

pentestagent

3.0k · Python
Active A

PentestAgent is an AI agent framework for black-box security testing, supporting bug bounty, red-team, and penetration testing workflows.

aiai-agentsai-assistant +5
  • · AI-driven black-box security testing framework for bug bounty and red-team workflows
  • · Multi-mode execution: Assist, Agent, Crew multi-agent orchestration, interactive guidance
  • · Docker isolated execution with pre-installed Nmap, Metasploit, SQLMap pentesting tools
View on GitHub