Uses PySpark for large-scale feature engineering. Optimizes Spark jobs (partitioning, caching, broadcast joins). Uses Spark ML for distributed model training.

Designs sklearn Pipelines for production. Performs feature selection (SelectKBest, RFE). Configures hyperparameter tuning (GridSearchCV, RandomizedSearchCV, Optuna). Handles imbalanced data (SMOTE, class_weight).

Reviews ML code: checks for data leakage, feature correctness, model evaluation. Gives constructive feedback. Verifies experiment reproducibility.

Uses Great Expectations/Soda for data validation. Configures automated data quality checks in ML pipeline. Monitors data drift before retraining.

Creates optimized Docker images for ML: multi-stage for training and serving, CUDA-based images. Configures GPU access in Docker. Uses .dockerignore for ML artifacts.

Configures Feast for the project. Defines feature definitions (entities, feature views). Configures materialization for online store. Integrates feature store with training pipeline.

Uses DVC for version control of data and models. Organizes ML code by branches: experiments, features, releases. Resolves conflicts in ML configurations.

Performs hyperparameter tuning for gradient boosting (learning_rate, max_depth, n_estimators, regularization). Handles categorical features (CatBoost native, target encoding). Configures early stopping and cross-validation. Analyzes SHAP values.

Deploys ML services in Kubernetes. Configures resource limits for CPU/GPU workloads. Uses ConfigMaps for model configuration. Configures HPA for ML serving autoscaling.

Designs ML pipelines with Kubeflow/Airflow. Configures parameterized pipelines for different models. Automates retraining with data quality checks. Implements pipeline testing.

Designs MLflow workflow: experiment naming, run tags, artifact storage. Uses Model Registry for versioning. Configures autologging for sklearn/PyTorch. Writes custom MLflow Plugins.

Uses model serving frameworks: Triton, BentoML, Seldon. Configures batch and real-time inference. Optimizes inference latency (ONNX, model optimization). Configures A/B testing for models.

Optimizes pandas code for ML: vectorized operations, category dtype, chunked reading. Uses Polars for faster processing. Writes efficient feature engineering pipelines.

Adds custom metrics to application (counter, gauge, histogram). Writes PromQL queries for dashboards. Creates Grafana dashboards. Configures basic alerts (high error rate, high latency).

Designs ML API with FastAPI: async endpoints, pydantic validation, batch prediction. Implements health checks and model versioning in API. Integrates API with model registry.

Designs custom models in PyTorch. Configures training loop: optimizer, scheduler, early stopping. Uses transfer learning (fine-tuning pretrained models). Logs experiments in MLflow/W&B.

Designs RESTful ML API: batch prediction, model versioning, health checks. Documents ML API with OpenAPI. Implements pagination for prediction results. Handles model errors.

Designs SQL ETL for feature computation. Uses dbt for ML feature transformation. Writes incremental ETL for updating training data. Automates through Airflow.

Writes comprehensive tests for ML: data validation, model prediction format, edge cases. Uses fixtures for ML test data. Tests pipeline components in isolation.

Evaluates algorithm complexity of data processing in ML pipelines. Understands memory/speed trade-offs in feature engineering. Optimizes batch operations considering computational complexity.

Applies type hints in ML code. Uses mypy for static analysis. Writes unit tests for data processing and model evaluation. Organizes ML code into modules (data, features, models, evaluation).

Configures data drift detection (Evidently, NannyML). Monitors feature distributions. Configures alerting on model degradation. Implements automated retraining trigger.

Applies OOP for structuring ML code: abstract classes for models, strategies for feature engineering. Uses patterns for ML component reuse. Writes custom sklearn transformers.

Writes structured logs in JSON format. Adds correlation IDs for tracing. Uses proper log levels. Configures log aggregation (EFK/Loki). Does not log sensitive data (PII, passwords).

Effectively uses data structures for ML: sparse matrices, ordered structures, heaps. Works with pandas MultiIndex and categorical data. Optimizes dataset memory footprint.

Designs experiment tracking workflow. Organizes experiments by projects/tasks. Configures hyperparameter sweeps (Optuna, W&B Sweeps). Analyzes results for decision making.