Dagster / Prefect

Understands Dagster/Prefect basics for orchestrating dbt models and data transformations. Runs existing pipelines, reads logs, and troubleshoots simple task failures in analytics workflows.

Uses Dagster or Prefect to build and schedule basic ETL pipelines. Understands assets, tasks, and flow concepts. Monitors pipeline runs and handles retries for transient failures.

Independently implements data pipelines with Dagster / Prefect. Optimizes performance. Ensures data quality.

Independently implements data pipelines with Dagster/Prefect. Optimizes performance. Ensures data quality.

Architects data systems with Dagster / Prefect. Optimizes for big data. Implements data governance and quality frameworks.

Designs data architecture with Dagster/Prefect. Optimizes for big data. Implements data governance and quality frameworks.

Implements modern orchestration for analytics: Dagster assets as native integration with dbt models, software-defined assets for Python transformations. Defines standards for observable, testable pipelines with built-in data quality checks.

Defines orchestration standards: Dagster vs Prefect vs Airflow selection by project, migration strategy. Evaluates software-defined assets (Dagster) vs task-based (Airflow) approaches.

Architects next-gen orchestration: Dagster's asset-based approach for a unified analytics platform, integration with dbt mesh, declarative scheduling. Defines migration strategy from Airflow to asset-centric orchestration.

Designs next-gen orchestration: Dagster for software-defined data assets, Prefect for event-driven, hybrid with Airflow legacy. Defines migration path and coexistence strategy.