Data Drift Monitor

Demo

Click to play with sound.

---
name: Data Drift Monitor
description: Establishes a monitoring strategy for data drift and concept drift in production ML systems, including statistical tests, alert thresholds, and retraining triggers. Apply when deploying a model, building an MLOps pipeline, or investigating unexplained model performance degradation.
---

# Data Drift Monitor

Models trained on historical data degrade when the world changes. Drift monitoring is the early-warning system that distinguishes a model aging gracefully from one silently failing.

## 1. Types of Drift to Monitor

Not all drift requires the same response.

- Data drift (covariate shift): input feature distribution changes; model may still work if the label relationship is stable
- Concept drift: the relationship between features and labels changes; requires retraining regardless of input distribution
- Label drift: ground truth label distribution shifts; relevant for classification thresholds
- Upstream data drift: schema changes or pipeline failures masquerading as real drift

## 2. Statistical Tests by Feature Type

Choose tests matched to the feature type.

- Continuous features: Kolmogorov-Smirnov test or Population Stability Index (PSI)
- Categorical features: chi-squared test or Jensen-Shannon divergence
- PSI thresholds: below 0.1 is stable, 0.1-0.2 is moderate (investigate), above 0.2 is significant (act)… install to load the full skill