Algorithms & Data Structures 1
▼
Understands basic algorithmic concepts for ML pipelines: simple data sampling methods, basic sorting and filtering for data preprocessing, configuration parameter search. Follows team guidance on algorithm selection for pipeline automation tasks.
Independently applies algorithmic thinking in MLOps: evaluates hyperparameter search strategies, understands model selection algorithm trade-offs, selects efficient data sampling methods for training. Analyzes computational cost implications of ML pipeline configuration choices.
Reviews algorithmic decisions in the team's ML pipelines, evaluates computational complexity of feature engineering and data preprocessing stages. Defines acceptable latency thresholds for inference services and optimizes batch training pipelines considering O-complexity of operations on large datasets.
Shapes the algorithm optimization strategy across the entire MLOps platform: selecting efficient structures for feature store, optimizing computation graphs in DAG orchestrators. Makes architectural decisions on trade-offs between model accuracy and computational complexity for production systems with millions of predictions per second.
Background Jobs & Task Queues 1
▼
Uses Task Queues at a basic level in Kubeflow/MLflow. Performs simple tasks using ready-made templates. Understands basic concepts and follows team practices.
Independently implements Task Queue tasks in Kubeflow/MLflow. Understands internals and optimizes performance. Writes tests using model monitoring.
Defines the async processing architecture for the MLOps team: Celery/RQ for lightweight tasks, Kubeflow Pipelines for heavy training workflows. Standardizes retry and backoff patterns for GPU training jobs, configures queue monitoring and alerting for stuck ML tasks via Flower and Prometheus.
Designs the ML task orchestration strategy at the platform level: choosing between Celery, Kubeflow Pipelines, and Argo Workflows for different workload types. Defines GPU resource prioritization architecture between teams, fairness scheduling policies, and automatic worker pool scaling for ML computations.
CI/CD 1
▼
Understands GitLab CI/CD pipeline structure (stages, jobs, artifacts). Applies basic practices for triggering ML training pipelines via .gitlab-ci.yml. Follows team conventions for model artifact storage and pipeline variable usage.
Independently creates multi-stage GitLab pipelines for ML workflows (data validation, training, evaluation, registry push). Configures GPU runners, DVC integration, and model registry artifacts. Automates experiment tracking and model promotion with pipeline triggers and API calls.
Defines the DevOps strategy with GitLab CI/CD Advanced. Establishes CI/CD standards. Implements platform engineering approaches.
Shapes the GitLab CI/CD strategy for the organization-level MLOps platform: unified include templates for ML pipelines, standards for model training and deployment stages. Designs GitLab Runners architecture with GPU support, MLflow integration for automatic experiment tracking from pipelines, and model promotion policies through environments.
Concurrency & Parallelism 1
▼
Understands basic async programming in Python: asyncio for concurrent pipeline operations, async model deployment workflows, basic concurrent feature store access patterns. Follows team conventions for async code in ML infrastructure.
Independently applies async programming in MLOps: concurrent model deployment operations, async pipeline stage orchestration, non-blocking feature store queries. Understands trade-offs between sync and async patterns for ML infrastructure operations.
Establishes async programming standards for the team's MLOps services: asyncio for inference API, concurrent data loading in training pipelines, non-blocking metric writes to Prometheus. Reviews correctness of concurrent request handling in model serving and parallel experiment execution in Kubeflow.
Makes strategic decisions on the MLOps platform concurrency model: asynchronous ML pipeline orchestration via event-driven architecture, parallel model training in distributed clusters. Designs backpressure architecture for streaming data and graceful degradation strategies for inference services under high load.
Container Security 1
▼
Understands the fundamentals of Container Security Scanning. Applies basic practices in daily work. Follows recommendations from the team and documentation.
Integrates security scanning into CI/CD for ML images: configuring Trivy/Grype in the inference container build pipeline, filtering false positives for scientific Python packages. Configures vulnerability admission policies considering ML dependency specifics (numpy, scipy, CUDA) and blocks deployment of images with critical CVEs.
Defines container security policies for the MLOps team's infrastructure: scanning standards, acceptable CVE levels for production inference. Implements regular ML image audits, configures automatic base image patching, and controls compliance for containers with access to training data and GPU resources.
Shapes the container security strategy for MLOps at the organizational level: unified scanning policies, trusted registry for ML images, compliance automation. Defines GPU container security requirements, supply chain security standards for ML dependencies, and scanning integration into the platform CI/CD for hundreds of ML projects.
Distributed Tracing 1
▼
Understands basics of OpenTelemetry for ML pipeline observability. Follows team guidelines for instrumenting training jobs, model serving endpoints, and data processing services with OTel SDK. Reads distributed traces to debug pipeline orchestration latency and inference endpoint performance.
Integrates OpenTelemetry into ML services: tracing inference requests through feature store, model serving, and post-processing. Configures custom spans for ML pipeline stages (feature_computation, model_load, prediction), exports to Jaeger/Tempo, and correlates traces with logs for prediction issue root cause analysis.
Defines distributed tracing standards for the MLOps team: mandatory spans for inference pipeline, standard attributes (model.version, prediction.confidence, feature.freshness). Implements OTel SDK as part of the inference framework, configures trace-based alerting on anomalies, and ensures end-to-end visibility for all ML requests in production.
Shapes the distributed tracing strategy for the organization's MLOps platform: unified OTel standard for all ML services, centralized backend. Designs observability architecture for cross-team ML pipelines, defines semantic conventions for ML-specific spans, and integrates traces with experiment tracking for complete lineage from experiment to production prediction.
File & Object Storage 1
▼
Uses S3 / Object Storage at a basic level in Kubeflow/MLflow. Performs simple tasks using ready-made templates. Understands basic concepts and follows team practices.
Independently implements S3 / Object Storage tasks in Kubeflow/MLflow. Understands internals and optimizes performance. Writes tests using model monitoring.
Defines the ML artifact storage strategy for the team: bucket structure for model registry, feature store snapshots, and experiment artifacts. Standardizes naming conventions, configures cross-region replication for DR, and optimizes storage costs through intelligent tiering for rarely used models.
Designs data storage architecture for the organization's MLOps platform: unified strategy for models, datasets, features, and experiment metadata. Makes decisions on choosing between S3, GCS, and MinIO, defines data governance, encryption, and compliance policies for ML artifacts company-wide.
GitOps 1
▼
Understands basic ArgoCD concepts for deploying ML model serving infrastructure and training pipelines. Follows team guidelines for checking sync status of model deployment Applications and reviewing inference service health. Uses existing ArgoCD manifests to deploy ML workloads including model servers, feature stores, and pipeline orchestrators.
Independently configures ArgoCD Applications for ML platform components including model registries, experiment tracking, and serving infrastructure. Writes ApplicationSets for multi-environment model deployment pipelines with promotion workflows. Implements custom health checks for ML-specific resources like InferenceServices, TrainingJobs, and feature store deployments.
Defines ArgoCD-based GitOps strategy for the organization's ML platform and model delivery infrastructure. Establishes standards for model deployment pipelines, governance of ML resource provisioning, and integration with ML experiment tracking systems. Conducts architecture reviews of ML deployment patterns and drives adoption of reproducible model delivery practices.
Shapes enterprise MLOps delivery architecture with ArgoCD at the center of model lifecycle management. Drives innovation in GitOps patterns for ML workloads including automated model retraining triggers, drift-aware deployment strategies, and multi-cluster model distribution. Influences industry standards for declarative ML infrastructure management and contributes to MLOps community practices.
gRPC 1
▼
Understands basics of gRPC & Protocol Buffers for MLOps platform service communication. Reads proto definitions for pipeline orchestration, model registry, and deployment service schemas. Follows team conventions for defining ML infrastructure service contracts.
Develops gRPC services for ML inference: defines proto schemas for prediction requests with typed tensors, implements streaming RPC for sequence processing. Configures gRPC clients with retry policies and deadlines for model serving calls, optimizes large tensor serialization via Protobuf.
Defines gRPC usage standards in the team's ML infrastructure: unified proto definitions for inference APIs, standardized error handling and metadata. Implements gRPC interceptors for prediction logging, latency metric collection, and request tracing via OpenTelemetry across the ML service chain.
Shapes the inter-service communication strategy for the MLOps platform: gRPC as the standard for internal ML services, REST for external APIs. Designs a unified proto repository for the organization, defines backward compatibility policies for inference APIs, and migration strategy from REST to gRPC for latency-critical serving paths.
Infrastructure as Code 1
▼
Understands basic Terraform concepts. Uses ready-made configurations. Performs simple operations under senior guidance.
Writes Terraform modules for deploying ML infrastructure: S3 buckets for model artifacts, ECR for ML images, SageMaker endpoints for inference. Manages state via remote backend, uses workspaces for separating staging/production ML environments, and applies variables for parameterizing GPU instances.
Defines Infrastructure as Code standards for the MLOps team: Terraform module library for typical ML components, review policies for infrastructure changes. Implements Terraform Cloud/Atlantis for collaborative IaC, configures Sentinel policies for GPU instance cost control and ML infrastructure compliance.
Shapes the IaC strategy for the organization's MLOps platform: unified module library for all ML teams, naming and tagging standards for ML resources. Designs multi-account/multi-region architecture for training and serving, defines cost management policies for GPU infrastructure, and DR strategy for critical ML services.
Kubernetes & Orchestration 2
▼
Understands basic Helm concepts. Uses ready-made configurations. Performs simple operations under senior guidance.
Independently configures and manages Helm. Writes IaC for typical tasks. Understands networking and security basics.
Defines infrastructure strategy with Helm. Establishes IaC standards. Conducts architecture reviews. Optimizes FinOps.
Shapes the Helm chart strategy for the organization's MLOps platform: library charts for standardizing inference service deployment, training jobs, and feature pipelines. Designs Helm repository architecture with versioning for Kubeflow, Seldon Core, MLflow, and internal ML components, defines update and rollback policies for ML infrastructure.
Understands basic Kubernetes Advanced concepts. Uses ready-made configurations. Performs simple operations under senior guidance.
Independently configures and manages Kubernetes Advanced. Writes IaC for typical tasks. Understands networking and security basics.
Defines infrastructure strategy with Kubernetes Advanced. Establishes IaC standards. Conducts architecture reviews. Optimizes FinOps.
Defines the strategy for advanced Kubernetes capabilities in the MLOps platform: custom operators for ML model lifecycle management, CRDs for describing training pipelines. Designs GPU scheduling architecture with device plugins, configures topology-aware scheduling for distributed training, and defines autoscaling policies for inference based on GPU utilization.
Message Queues & Event Streaming 1
▼
Uses Apache Kafka at a basic level in Kubeflow/MLflow. Performs simple tasks using ready-made templates. Understands basic concepts and follows team practices.
Independently implements Apache Kafka tasks in Kubeflow/MLflow. Understands internals and optimizes performance. Writes tests using model monitoring.
Defines the event-driven architecture for the MLOps team's infrastructure: topics for model events (deployed, retrained, degraded), feature update streams for online serving. Standardizes Avro/Protobuf schemas for ML events and configures Kafka Connect for integration with feature store and experiment tracker.
Architects the event-driven architecture for the organization-level MLOps platform: unified event bus for model lifecycle, data pipeline triggers, and drift monitoring. Makes decisions on topic partitioning for scaling streaming feature engineering and exactly-once delivery guarantees in critical ML pipelines.
Metrics & Monitoring 2
▼
Learns to track custom ML metrics such as model drift, prediction latency, and data quality scores. Uses existing monitoring pipelines and Grafana dashboards to observe model performance.
Configures custom metrics for ML pipelines including training throughput, feature store freshness, and inference SLOs. Builds alerting rules for model degradation and data pipeline failures. Analyzes production incidents affecting model serving.
Defines custom metrics standards for the MLOps team: ML-specific metrics (prediction_confidence, feature_freshness, model_staleness), business metrics tied to models. Implements a unified metrics library for inference services, standardizes labels and naming conventions for Prometheus, and configures composite alerts for ML system degradation.
Shapes the metrics strategy for the organization's MLOps platform: unified ML metrics taxonomy, standards for model health scoring and platform reliability. Designs automated model ROI calculation systems by linking ML metrics with business KPIs, defines cost-per-prediction metrics and composite health indicators for all production models.
Understands basic Prometheus & Grafana for MLOps: reading model serving performance dashboards, understanding training job metrics (GPU utilization, loss curves), navigating Grafana panels for ML pipeline health. Follows team conventions for monitoring ML infrastructure.
Configures ML service monitoring via Prometheus: exporting metrics from inference endpoints (prediction_latency_seconds, model_predictions_total), configuring ServiceMonitor in Kubernetes. Creates Grafana dashboards for MLOps — model metrics by version, GPU utilization, feature store latency — and configures alerting via Alertmanager.
Defines monitoring standards for the MLOps team: mandatory metrics for each inference service, standard dashboards, alerting policies. Implements SLI/SLO framework for ML services, configures on-call processes for model incidents, and ensures model metric visibility for data science and product teams.
Shapes the observability strategy for the organization's MLOps platform: unified Prometheus/Grafana stack for all ML teams, metric and dashboard standards. Designs scalable monitoring architecture for thousands of ML services, defines organization-level SLO framework, and integrates ML metrics with business KPIs for tracking model ROI.
MLOps 6
▼
Understands the importance of experiment tracking for ML result reproducibility. Can log hyperparameters, metrics, and training artifacts in MLflow or Weights & Biases, compare results of different runs via UI. Follows team conventions for experiment naming and run tagging.
Configures experiment tracking for ML projects: organizing experiments by tasks, automatic logging via callbacks in PyTorch Lightning/Keras. Implements model comparison across multiple metrics, training visualization via W&B plots, dataset and config tracking for full reproducibility of every experiment in the team.
Defines experiment tracking standards for the MLOps team: mandatory metadata for each experiment, tag taxonomy and naming conventions. Implements experiment review processes before production deployment, configures automated result reports, and ensures linkage between experiments, Git PRs, and model deployments.
Shapes the experiment management strategy at the organizational level: unified platform for all ML teams, reproducibility and audit standards. Designs scaling architecture — multi-tenant experiment store, integration with Model Registry and Feature Store for complete ML lineage. Defines storage policies, compliance requirements, and governance for experimental data.
Deploys and monitors feature store infrastructure components. Understands data ingestion pipelines feeding the feature store. Performs basic troubleshooting of feature freshness and availability issues.
Configures feature store platforms like Feast or Tecton for online and offline serving. Builds automated pipelines for feature computation, validation, and backfilling. Monitors feature drift and data quality metrics in production.
Defines the Feature Store strategy at team/product level. Establishes standards and best practices. Conducts reviews.
Defines the Feature Store strategy at the organizational level. Establishes enterprise approaches. Mentors leads and architects.
Understands ML pipeline concepts: data validation, model training, and deployment stages. Configures pipeline runners (Airflow, Kubeflow) and monitors pipeline health metrics.
Implements automated ML pipelines with feature stores, model versioning, and A/B testing infrastructure. Builds CI/CD for model training with automated retraining triggers on data drift detection.
Defines the ML Pipelines strategy at team/product level. Establishes standards and best practices. Conducts reviews.
Defines the ML Pipelines strategy at the organizational level. Establishes enterprise approaches. Mentors leads and architects.
Uses MLflow Tracking for logging basic training metrics: accuracy, loss, F1-score. Can run mlflow ui to view experiments, log model parameters via mlflow.log_param, and save training artifacts. Understands MLflow structure — experiments, runs, artifacts — and navigates previous run results.
Configures MLflow Tracking Server for the team: remote backend store on PostgreSQL, artifact store in S3. Implements automatic logging via mlflow.autolog for PyTorch/TensorFlow/XGBoost, configures custom metrics and tags for experiment filtering. Integrates MLflow into training pipelines for result reproducibility.
Defines MLflow usage standards for the MLOps team: mandatory metrics and tags for each experiment, naming conventions, project structure. Implements best practices for experiment organization — parent/child runs for hyperparameter tuning, nested runs for ensemble models, standardizes MLflow Projects for training reproducibility.
Shapes the experiment tracking strategy at the organizational level: MLflow as a unified platform for all ML teams, integration with corporate SSO and RBAC. Designs multi-tenant MLflow architecture with data isolation between teams, defines retention policies for experiments and artifacts, plans scaling for thousands of simultaneous experiments.
Understands basic ML model monitoring concepts: why tracking prediction quality in production matters, what data drift and concept drift are. Can view dashboards with model metrics, read quality degradation alerts, and perform simple input data distribution checks using pandas and matplotlib.
Configures ML model monitoring in production: collecting prediction logs, calculating quality metrics on new data, detecting data drift via statistical tests (KS-test, PSI). Integrates Evidently AI or Whylogs for automated distribution monitoring, configures Grafana dashboards with model metrics, and sets up accuracy/F1 degradation alerts.
Defines model monitoring standards for the MLOps team: mandatory metrics for each model type, prediction quality SLAs, drift response procedures. Builds ML observability culture — stakeholder dashboards, automated model health reports, and runbooks for quality degradation incidents.
Shapes the ML model monitoring strategy at the organizational level: unified observability platform for all production models, SLA and SLO standards. Designs centralized monitoring architecture for hundreds of models, defines automatic retraining and rollback policies, implements ML governance with bias, fairness, and regulatory compliance tracking.
Understands basic model serving concepts: difference between batch and real-time inference, main model formats (ONNX, SavedModel, pickle). Can deploy a simple model via Flask/FastAPI endpoint, load a model from file, and return predictions. Knows about specialized serving systems — TFServing, Triton, Seldon.
Deploys models to production via specialized serving platforms: TensorFlow Serving for TF models, Triton Inference Server for multi-framework serving. Configures BentoML for packaging models with dependencies, implements batch inference via Spark/Ray, and configures model versioning for seamless production model updates.
Defines the model serving strategy for the MLOps team: standard stack (KServe/Seldon Core on Kubernetes), deployment patterns (canary, shadow, blue-green). Implements unified model rollout process with mandatory quality checks, configures SLA monitoring for latency, and defines runbooks for inference service incidents.
Shapes the model serving strategy at the organizational level: unified serving platform for all model types (CV, NLP, tabular), SLA standards. Designs architecture for scaling to thousands of models — model mesh, serverless inference, edge deployment. Defines GPU infrastructure cost optimization strategy for inference and platform roadmap.
Performance Testing 1
▼
Understands load testing basics for ML infrastructure: testing model inference endpoints under concurrent request load, measuring GPU utilization during batch predictions, and basic throughput/latency benchmarking for model APIs. Follows team practices for ML service load test configuration.
Conducts load testing of inference services using Locust or k6: measuring prediction latency at different RPS, verifying stability under batch requests. Analyzes test results — p50/p95/p99 latency, throughput, error rate — and identifies performance degradation with increasing input data size.
Defines load testing standards for the MLOps team: inference latency SLAs by model type, mandatory tests before production rollout. Implements regular ML service performance testing in CI/CD, configures automated alerts on degradation, and defines benchmarks for comparing model versions by performance.
Shapes the performance engineering strategy for the organization's MLOps platform: latency SLA standards for all inference types, GPU infrastructure capacity planning methodology. Defines platform-level load testing approaches — shared load testing infrastructure, production traffic replay for new model validation, and chaos engineering for ML services.
Relational Databases 1
▼
Performs basic SQL queries against PostgreSQL to extract data needed by ML pipelines: selecting training datasets, checking prediction results. Understands table schemas with model and experiment metadata, writes simple JOIN queries for preparing features from relational sources.
Develops medium-complexity SQL queries for feature engineering from PostgreSQL: window functions for temporal features, aggregations for statistical features. Configures ML pipeline connections to PostgreSQL via SQLAlchemy, optimizes queries for training set extraction, and implements incremental loading for dataset updates.
Defines PostgreSQL usage standards in the MLOps team's infrastructure: schemas for storing experiment results, model metrics, and lineage data. Designs database architecture for MLflow backend store, configures replication for metadata service fault tolerance, and optimizes performance under high-frequency metric writes.
Shapes the PostgreSQL usage strategy at the MLOps platform level: unified standards for metadata store, feature catalog, and audit logging. Makes scaling decisions — Citus for sharding, TimescaleDB for model metric time series, defines migration policies and data governance for ML metadata.
Web Frameworks 1
▼
Uses Python Web Frameworks at a basic level in Kubeflow/MLflow. Performs simple tasks using ready-made templates. Understands basic concepts and follows team practices.
Independently implements Python Web Framework tasks in Kubeflow/MLflow. Understands internals and optimizes performance. Writes tests using model monitoring.
Defines the serving layer architecture for the team's ML models: choosing between FastAPI, Triton Inference Server, and BentoML. Standardizes inference API building patterns — model versioning via URLs, A/B testing via headers, shadow mode for canary deployment of new versions.
Shapes the API layer strategy for the organization's MLOps platform: unified gateway for inference services, authentication and rate limiting standards for ML APIs. Designs multi-model serving architecture with dynamic request routing and automatic scaling based on GPU utilization metrics.