Defines platform performance strategy. Designs algorithms for monitoring systems: time-series downsampling (LTTB), probabilistic data structures for cardinality estimation. Balances accuracy vs cost.
Roles · Site Reliability Engineer (SRE) · Principal
What a Principal } should know
45 core skills, 61 in total. Expectations per skill, and what changes at the next level.
This page lists what a Principal } is expected to know and do, skill by skill. Core skills are the ones a manager and peers assess in a review cycle; the rest count only in self-assessment. Main areas: Programming Fundamentals, Backend Development, Database Management.
Core skills for a Principal
Grouped by area. The label on the right is the expected depth: Awareness, Working, Advanced or Expert.
Programming Fundamentals · 6
Designs event-driven SRE automation: async event processing for alert routing, webhook-based incident orchestration, streaming metrics processing. Defines real-time anomaly detection architecture.
Defines SRE platform quality strategy: testing pyramid for infrastructure (unit → integration → e2e), chaos engineering, automated compliance checking. Establishes reliability engineering culture.
Designs data models for observability platform: time-series storage schemes, log structured data, trace span trees. Defines metrics storage strategy considering retention and query performance.
Designs high-concurrency SRE systems: parallel fleet management, concurrent configuration rollouts, lock-free metrics aggregation pipelines. Defines concurrency models for platform tooling.
Designs SRE platform architecture: plugin system for monitoring backends, abstractions for incident management. Defines where code is justified vs declarative configs (Terraform, K8s manifests).
Backend Development · 3
Designs messaging platform for observability: Kafka for metrics/logs/traces pipeline, retention policies, multi-DC replication for DR. Defines SLA for messaging infrastructure.
Designs SRE platform API: self-service reliability tools, SLO management interface, incident orchestration. Defines tech stack for SRE platform: Python for automation, Go for performance-critical.
Designs caching infrastructure: Redis Sentinel/Cluster for HA, failure modes analysis, capacity planning. Defines when Redis vs in-memory vs distributed cache for SRE tooling.
Database Management · 1
Designs database reliability strategy: PostgreSQL HA patterns (Patroni/Stolon), cross-region replication, RTO/RPO by tier. Defines DB platform standards for the organization.
API & Integration · 1
Designs SRE Platform API strategy: unified API for infrastructure operations, GitOps integration, self-service portal. Defines API governance for cross-team automation.
Cloud & Infrastructure · 9
Designs organizational cloud strategy: multi-cloud vs all-in-AWS, cloud native vs portable, cost optimization at scale. Defines cloud governance and architectural patterns.
Designs container strategy: OCI standards, runtime selection (containerd vs CRI-O), rootless containers. Defines container security framework and compliance requirements.
Designs package management strategy: Helm vs Kustomize vs Carvel, chart governance, multi-cluster distribution. Defines deployment standards for the entire organization.
Designs Kubernetes platform: multi-cluster management (fleet), federation, cluster-as-a-service. Defines K8s evolution strategy: version upgrades, feature adoption, vendor evaluation.
Designs organizational Kubernetes strategy: managed vs self-hosted, multi-tenancy model, cost allocation. Defines platform abstractions on top of K8s for product teams.
Designs traffic management strategy: global server load balancing, multi-CDN, anycast. Defines traffic engineering patterns for multi-region availability and disaster recovery.
Designs network architecture: service mesh, zero-trust networking, global load balancing. Defines network strategy for multi-region and multi-cloud deployment.
Designs organizational IaC strategy: Terraform vs Pulumi vs Crossplane, self-service infrastructure, multi-cloud abstractions. Defines governance for infrastructure changes.
Designs connectivity strategy: VPN vs Direct Connect vs SD-WAN, zero-trust network access. Defines remote access architecture for the organization.
DevOps & CI/CD · 2
Shapes enterprise deployment reliability strategy with ArgoCD as the foundation of declarative infrastructure delivery. Drives architectural innovation in GitOps-based resilience patterns including multi-region active-active deployments, automated incident rollback, and self-healing infrastructure reconciliation. Influences SRE community practices for GitOps adoption and contributes to Argo project reliability engineering standards.
Designs organizational CI/CD strategy: unified deployment platform, GitOps adoption, progressive delivery framework. Defines deployment governance and compliance requirements.
Testing & QA · 1
Shapes enterprise resilience strategy through chaos: designs organization-wide chaos framework, defines compliance requirements for chaos testing (financial services, healthcare). Influences industry practices through publications and talks about chaos engineering ROI.
Security · 3
Designs incident management platform: automated triage, cross-team coordination, incident learning system. Defines organizational incident culture and continuous improvement process.
Designs platform security strategy: zero-trust architecture, supply chain security, platform security controls. Defines security governance for cloud infrastructure.
Designs secrets management strategy: multi-cluster Vault, cross-cloud secrets, zero-trust credential issuance. Defines organizational secrets governance and compliance framework.
AI-Assisted Development · 1
Defines AI strategy for SRE: AIOps for anomaly detection, AI-assisted incident response, LLM for log analysis. Shapes governance for AI in production operations.
Architecture & System Design · 4
Designs capacity management platform: ML-based demand forecasting, automated provisioning, cost optimization at scale. Defines organizational capacity governance.
Designs organizational DR strategy: multi-region architecture, data sovereignty compliance, full-stack failover automation. Defines business continuity framework.
Designs high-load platform: global traffic distribution, multi-region data consistency, edge computing. Defines scalability architecture patterns for the organization.
Defines organizational architecture strategy: reference architecture for reliable systems, platform abstractions, cross-team design patterns. Shapes reliability engineering culture.
Observability & Monitoring · 11
Designs APM platform: unified APM for all services, custom dashboards, automated alerting. Defines vendor selection criteria, negotiation strategy, multi-year roadmap.
Designs continuous profiling platform: fleet-wide profiling, automated regression detection, cost/performance correlation. Defines profiling strategy for production reliability.
Designs organizational metrics framework: unified instrumentation SDK, metrics taxonomy, automated SLI generation. Defines observability cost management strategy.
Designs log management platform: ELK vs Loki vs Datadog, multi-tenant architecture, compliance logging. Defines organizational logging standards and cost optimization.
Designs log aggregation strategy: Loki for Kubernetes-native logging, multi-tenant setup, long-term storage. Defines when Loki vs ELK vs managed (Datadog/Splunk).
Designs distributed tracing platform: Tempo vs Jaeger vs managed, sampling strategy (head vs tail-based), traces-to-metrics conversion. Defines tracing governance.
Designs organizational on-call model: follow-the-sun, tiered support, shared on-call between SRE and dev teams. Defines on-call governance and toil elimination strategy.
Designs OpenTelemetry platform: vendor-neutral observability, multi-signal correlation, OTel Collector scalability. Defines observability strategy and vendor evaluation.
Designs organizational metrics platform: Prometheus/Mimir vs Datadog vs VictoriaMetrics, multi-tenant architecture, cost management. Defines metrics governance.
Designs SLO platform: organizational SLO framework, automated SLO management, SLO-driven architecture decisions. Defines reliability culture and error budget policy.
Designs observability logging strategy: logs-traces-metrics correlation, cost-effective retention, ML-based anomaly detection on logs. Defines organizational logging framework.
Version Control & Collaboration · 2
Defines organizational review standards: architectural review board for infrastructure changes, cross-team review for shared platforms. Shapes engineering excellence.
Defines organizational version control strategy: GitOps adoption framework, infrastructure versioning, cross-team collaboration model. Coordinates infrastructure code standards.
Performance Engineering · 1
Designs latency optimization strategy: global edge deployment, predictive caching, network optimization. Defines organizational performance culture and latency governance.
Additional skills
Not assessed by the team, but part of the self-assessment and the development plan.
} in the open competency matrix: 61 skills across 5 levels. The matrix is free for individuals and stays free.