Knowledge Share
Technical articles, tutorials, and insights
Open Source Project #177: Apache Airflow — Workflow Orchestration in Python, the Standard Tool for Data Engineers
Apache Airflow is a platform for programmatically authoring, scheduling, and monitoring workflows. Core concept: DAGs (Directed Acyclic Graphs) — define task dependencies in Python code, trigger by schedule or data events, monitor every step in the Web UI. Covers four major scenarios: ETL/ELT pipelines, ML training pipelines, scheduled reporting, and infrastructure automation. 46.4k Stars, Apache 2.0, version 3.3.0, 600+ built-in Operators and Providers.
Open Source Project #180: Omnigent — Databricks' Meta-Harness for Unified Control of Claude Code, Codex, and Cursor
AI coding agent meta-orchestration framework open-sourced by Databricks co-founder Matei Zaharia. Sits above Claude Code, Codex, Cursor and other tools to provide policy governance (token budgets, tool access limits), cloud sandboxing (Modal/E2B/Daytona/Kubernetes), real-time collaboration (session sharing/co-driving/forking), MLflow observability, and YAML-defined multi-agent orchestration. The Kubernetes analogy for AI agents. 8.1k Stars, Apache 2.0, Alpha stage.