Data Engineer
Building and maintaining data pipelines, data warehouses, ensuring data availability and quality
Level:
Programming Fundamentals
8 skills · 40 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Algorithms & Data Structures | |||||
| Algorithms & Complexity | Awareness | Working | Working | Advanced | Advanced |
| Data Structures | Awareness | Working | Working | Advanced | Advanced |
| OOP & Design Patterns | |||||
| OOP & SOLID Principles | Awareness | Working | Working | Advanced | Advanced |
| Design Patterns | Awareness | Working | Working | Advanced | Advanced |
| Concurrency & Parallelism | |||||
| Multithreading | Awareness | Working | Working | Advanced | Advanced |
| Async Programming | Awareness | Working | Working | Advanced | Advanced |
| Clean Code & Refactoring | |||||
| Code Quality & Refactoring | Awareness | Working | Working | Advanced | Advanced |
| Type Systems | |||||
| Type Safety & Type Systems | Awareness | Working | Working | Advanced | Advanced |
Backend Development
6 skills · 30 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Web Frameworks | |||||
| Python Web Frameworks | Awareness | Working | Working | Advanced | Expert |
| Message Queues & Event Streaming | |||||
| Apache Kafka | Awareness | Working | Working | Advanced | Expert |
| Caching | |||||
| Redis | Awareness | Working | Working | Advanced | Advanced |
| Background Jobs & Task Queues | |||||
| Task Queues | Awareness | Working | Working | Advanced | Expert |
| Search Engines | |||||
| Elasticsearch / OpenSearch | Awareness | Working | Advanced | Advanced | Expert |
| File & Object Storage | |||||
| S3 / Object Storage | Awareness | Working | Advanced | Advanced | Expert |
Database Management
9 skills · 45 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Relational Databases | |||||
| PostgreSQL | Awareness | Working | Advanced | Advanced | Expert |
| NoSQL Databases | |||||
| ClickHouse | Awareness | Working | Advanced | Advanced | Expert |
| Apache Cassandra | Awareness | Working | Advanced | Advanced | Expert |
| Database Optimization | |||||
| Database Indexing | Awareness | Working | Advanced | Advanced | Expert |
| Query Optimization | Awareness | Working | Advanced | Advanced | Expert |
| Database Administration | |||||
| Replication & High Availability | Awareness | Working | Advanced | Advanced | Expert |
| Backup & Disaster Recovery | Awareness | Working | Advanced | Advanced | Expert |
| Data Modeling | |||||
| Data Modeling & Schema Design | Awareness | Working | Advanced | Advanced | Expert |
| Database Migrations | Awareness | Working | Advanced | Advanced | Expert |
API & Integration
4 skills · 20 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| REST API | |||||
| REST API Design | Awareness | Working | Working | Advanced | Advanced |
| GraphQL | |||||
| GraphQL Design | Awareness | Working | Working | Advanced | Advanced |
| gRPC | |||||
| gRPC & Protocol Buffers | Awareness | Working | Working | Advanced | Expert |
| API Management | |||||
| API Documentation | Awareness | Working | Working | Advanced | Advanced |
Cloud & Infrastructure
5 skills · 23 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Containerization | |||||
| Docker | Awareness | Working | Working | Advanced | Advanced |
| Kubernetes & Orchestration | |||||
| Kubernetes Core | Awareness | Working | Working | Advanced | Advanced |
| Infrastructure as Code | |||||
| Terraform | Awareness | Working | Working | Advanced | Expert |
| Cloud Providers | |||||
| AWS | Awareness | Working | Working | Advanced | Advanced |
| Networking | |||||
| Network Fundamentals | Awareness | Working | Advanced | — | — |
DevOps & CI/CD
1 skills · 5 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| CI/CD | |||||
| GitHub Actions / GitLab CI | Awareness | Working | Working | Advanced | Advanced |
Testing & QA
3 skills · 15 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Unit Testing | |||||
| Unit Testing | Awareness | Working | Working | Advanced | Advanced |
| Integration Testing | |||||
| Integration Testing | Awareness | Working | Working | Advanced | Advanced |
| E2E Testing | |||||
| E2E Testing | Awareness | Working | Working | Advanced | Advanced |
Security
3 skills · 15 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Application Security | |||||
| OWASP & Application Security | Awareness | Working | Working | Advanced | Advanced |
| Secure Coding Practices | Awareness | Working | Working | Advanced | Advanced |
| Authentication & Authorization | |||||
| JWT / OAuth2 / OIDC | Awareness | Working | Working | Advanced | Advanced |
Data Engineering
14 skills · 70 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Batch Processing | |||||
| Apache Spark | Awareness | Working | Advanced | Advanced | Expert |
| dbt | Awareness | Working | Advanced | Advanced | Expert |
| Pandas / Polars | Awareness | Working | Advanced | Advanced | Expert |
| SQL-based ETL | Awareness | Working | Advanced | Advanced | Expert |
| Stream Processing | |||||
| Stream Processing | Awareness | Working | Advanced | Advanced | Expert |
| Data Lakehouse | |||||
| Delta Lake / Apache Iceberg | Awareness | Working | Advanced | Advanced | Expert |
| Data Lake Architecture | Awareness | Working | Advanced | Advanced | Expert |
| Data Warehousing | |||||
| Data Warehouse Design | Awareness | Working | Advanced | Advanced | Expert |
| Data Governance | |||||
| Data Catalog | Awareness | Working | Advanced | Advanced | Expert |
| Data Lineage | Awareness | Working | Advanced | Advanced | Expert |
| Data Contracts | Awareness | Working | Advanced | Advanced | Expert |
| Data Quality | |||||
| Data Quality | Awareness | Working | Advanced | Advanced | Expert |
| Data Orchestration | |||||
| Apache Airflow | Awareness | Working | Advanced | Advanced | Expert |
| Dagster / Prefect | Awareness | Working | Advanced | Advanced | Expert |
AI-Assisted Development
4 skills · 18 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| AI Coding Assistants | |||||
| GitHub Copilot | Awareness | Working | Working | Advanced | Advanced |
| Cursor IDE | Awareness | Working | Working | Advanced | — |
| ChatGPT / Claude | Awareness | Working | Working | Advanced | Advanced |
| Prompt Engineering | |||||
| Prompt Engineering for Code | Awareness | Working | Working | Advanced | — |
Architecture & System Design
1 skills · 5 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| System Design | |||||
| System Design Fundamentals | Awareness | Working | Working | Advanced | Advanced |
Observability & Monitoring
4 skills · 18 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Logging | |||||
| Structured Logging | Awareness | Working | Working | Advanced | Advanced |
| Metrics & Monitoring | |||||
| Prometheus & Grafana | Awareness | Working | Advanced | — | — |
| Distributed Tracing | |||||
| OpenTelemetry | Awareness | Working | Working | Advanced | Advanced |
| Alerting & On-Call | |||||
| SLI / SLO / SLA | Awareness | Working | Working | Advanced | Advanced |
Version Control & Collaboration
2 skills · 10 requirements| Skill | Junior | Middle | Senior | Lead | Principal |
|---|---|---|---|---|---|
| Git & Workflows | |||||
| Git Advanced | Awareness | Working | Working | Advanced | Advanced |
| Code Review | |||||
| Code Review | Awareness | Working | Working | Advanced | Advanced |