Data Engineer
Building and maintaining data pipelines, data warehouses, ensuring data availability and quality
Data Engineer es un rol en la familia Data Engineering. Tiene 64 habilidades en 5 niveles (de Junior a Principal). 186 habilidades son obligatorias. Dominios clave: Programming Fundamentals, Backend Development, Database Management.
Stack tecnológico
Enfoque por nivel
Writing ETL scripts (Python/SQL). Working with Airflow DAGs. Loading data into warehouse. Monitoring pipelines. SQL queries for analysts.
Designing data pipelines. Working with Spark/Flink. Optimizing SQL queries on large datasets. Data quality checks. Working with data warehouse.
Data platform architecture. Designing data lake/lakehouse. Storage cost optimization. Designing real-time pipelines. Mentoring.
Data platform strategy. DataOps practices. Governance and lineage. Coordination with ML and Analytics. Data quality standards.
Enterprise data strategy. Multi-cloud data architecture. Data mesh. Cost optimization at scale. Vendor evaluation.
Matriz de habilidades
64 habilidades × 5 niveles. Haga clic en una celda para ver detalles.
AI-Assisted Development
4 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| GitHub Copilot | A | W | A | E | E |
| Cursor IDE | A | W | A | A | — |
| ChatGPT / Claude | A | W | A | E | E |
| Prompt Engineering for Code | A | W | A | E | — |
API & Integration
4 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| REST API Design | A | W | A | E | E |
| GraphQL Design | A | W | A | E | E |
| gRPC & Protocol Buffers | A | W | A | E | E |
| API Documentation | A | W | A | E | E |
Architecture & System Design
1 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| System Design Fundamentals | A | W | A | E | E |
Backend Development
6 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Python Web Frameworks | A | W | A | E | E |
| Apache Kafka | A | W | A | E | E |
| Redis | A | W | A | E | E |
| Task Queues | A | W | A | E | E |
| Elasticsearch / OpenSearch | A | W | A | E | E |
| S3 / Object Storage | A | W | A | E | E |
Cloud & Infrastructure
5 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Docker | A | W | A | E | E |
| Kubernetes Core | A | W | A | E | E |
| Terraform | A | W | A | E | E |
| AWS | A | W | A | E | E |
| Network Fundamentals | A | W | A | — | — |
Data Engineering
14 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Apache Spark | A | W | A | E | E |
| dbt | A | W | A | E | E |
| Pandas / Polars | A | W | A | E | E |
| SQL-based ETL | A | W | A | E | E |
| Stream Processing | A | W | A | E | E |
| Delta Lake / Apache Iceberg | A | W | A | E | E |
| Data Lake Architecture | A | W | A | E | E |
| Data Warehouse Design | A | W | A | E | E |
| Data Catalog | A | W | A | E | E |
| Data Lineage | A | W | A | E | E |
| Data Contracts | A | W | A | E | E |
| Data Quality | A | W | A | E | E |
| Apache Airflow | A | W | A | E | E |
| Dagster / Prefect | A | W | A | E | E |
Database Management
9 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| PostgreSQL | A | W | A | E | E |
| ClickHouse | A | W | A | E | E |
| Apache Cassandra | A | W | A | E | E |
| Database Indexing | A | W | A | E | E |
| Query Optimization | A | W | A | E | E |
| Replication & High Availability | A | W | A | E | E |
| Backup & Disaster Recovery | A | W | A | E | E |
| Data Modeling & Schema Design | A | W | A | E | E |
| Database Migrations | A | W | A | E | E |
DevOps & CI/CD
1 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| GitHub Actions / GitLab CI | A | W | A | E | E |
Observability & Monitoring
4 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Structured Logging | A | W | A | E | E |
| Prometheus & Grafana | A | W | A | — | — |
| OpenTelemetry | A | W | A | E | E |
| SLI / SLO / SLA | A | W | A | E | E |
Programming Fundamentals
8 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Algorithms & Complexity | A | W | A | E | E |
| Data Structures | A | W | A | E | E |
| OOP & SOLID Principles | A | W | A | E | E |
| Design Patterns | A | W | A | E | E |
| Multithreading | A | W | A | E | E |
| Async Programming | A | W | A | E | E |
| Code Quality & Refactoring | A | W | A | E | E |
| Type Safety & Type Systems | A | W | A | E | E |
Security
3 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| OWASP & Application Security | A | W | A | E | E |
| Secure Coding Practices | A | W | A | E | E |
| JWT / OAuth2 / OIDC | A | W | A | E | E |
Testing & QA
3 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Unit Testing | A | W | A | E | E |
| Integration Testing | A | W | A | E | E |
| E2E Testing | A | W | A | E | E |
Version Control & Collaboration
2 habilidades| Habilidades | Jun | Mid | Sen | Lead | Princ |
|---|---|---|---|---|---|
| Git Advanced | A | W | A | E | E |
| Code Review | A | W | A | E | E |
Preguntas frecuentes
¿Qué habilidades se necesitan para el rol de Data Engineer?
El rol de Data Engineer requiere 64 habilidades, de las cuales 186 son obligatorias. Las habilidades se distribuyen en 5 niveles: de Junior a Principal. Ver matriz completa.
¿Cómo avanzar al siguiente nivel en el rol de Data Engineer?
Use la Calculadora de grado para evaluar su nivel actual y obtener recomendaciones personalizadas.
¿Qué stack tecnológico se usa en el rol de Data Engineer?
El stack incluye 5 tecnologías en diferentes niveles. Python 3.11+, SQL, Apache Airflow, PostgreSQL/ClickHouse, pandas, Docker, Git, Python 3.12+, SQL advanced, Airflow, Spark/PySpark, ClickHouse/BigQuery, dbt, Kafka basics, S3/GCS, Python, Spark/Flink, ClickHouse/BigQuery/Snowflake, Kafka/Debezium, Delta Lake/Iceberg, Terraform, Kubernetes, Data lineage (OpenLineage)...
¿Cómo define la comunidad los requisitos para el rol de Data Engineer?
Los requisitos del rol son definidos por la comunidad a través de un sistema de propuestas. Cualquier miembro puede sugerir cambios que pasan por votación y revisión de expertos.