Data Engineer
Building and maintaining data pipelines, data warehouses, ensuring data availability and quality
Level:
Data Engineering
14 skills · 70 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Batch Processing | |||||
| Apache Spark | Awareness | Working | Advanced | Advanced | Expert |
| dbt | Awareness | Working | Advanced | Advanced | Expert |
| Pandas / Polars | Awareness | Working | Advanced | Advanced | Expert |
| SQL-based ETL | Awareness | Working | Advanced | Advanced | Expert |
| Stream Processing | |||||
| Stream Processing | Awareness | Working | Advanced | Advanced | Expert |
| Data Lakehouse | |||||
| Delta Lake / Apache Iceberg | Awareness | Working | Advanced | Advanced | Expert |
| Data Lake Architecture | Awareness | Working | Advanced | Advanced | Expert |
| Data Warehousing | |||||
| Data Warehouse Design | Awareness | Working | Advanced | Advanced | Expert |
| Data Governance | |||||
| Data Catalog | Awareness | Working | Advanced | Advanced | Expert |
| Data Lineage | Awareness | Working | Advanced | Advanced | Expert |
| Data Contracts | Awareness | Working | Advanced | Advanced | Expert |
| Data Quality | |||||
| Data Quality | Awareness | Working | Advanced | Advanced | Expert |
| Data Orchestration | |||||
| Apache Airflow | Awareness | Working | Advanced | Advanced | Expert |
| Dagster / Prefect | Awareness | Working | Advanced | Advanced | Expert |