For ML teams: ship a trustworthy model end-to-end, from EDA to production drift alerts.
Reach for this pack when you own an ML model and need it to survive contact with production. It walks the full lifecycle as a connected discipline: profile the data before you model, engineer leakage-safe features, track experiments so results reproduce, evaluate honestly against real baselines and error slices (for both classic models and LLM systems), document the model for the people it affects, and watch for drift so you retrain on evidence instead of vibes. The payoff is a model you can defend to stakeholders and trust over the long haul - not a notebook that worked once.
Click to play with sound.
Arranged in the author's recommended order. Walk through them in sequence, or open any one on its own.
Runs a structured exploratory data analysis on a new or suspect dataset – schema audit, target analysis, feature profiling, missingness patterns, and leakage checks – ending in a written decision log.
Designs ML features with leakage-safe pipelines, correct categorical encoding by cardinality, numeric transforms, and validation that a feature earns its place.
Designs a reusable, leakage-safe feature store – entity and naming contracts, point-in-time correct training joins, feature versioning, online/offline consistency with staleness SLOs, and governance.
Sets up disciplined ML experiment tracking – run logging schemas, artifact versioning, naming conventions, and reproducibility standards – and produces the run-record template a team actually follows.
Produces a rigorous, honest evaluation report for an ML model – real baselines, business-matched metrics with confidence intervals, slice-level error analysis, calibration checks, and a go/no-go recommendation.
Produces a complete model card – intended use, training data provenance, evaluation results with slices, limitations, and usage guidance – for any model shared beyond its team or affecting people.
Designs a production drift-monitoring plan for ML systems – statistical tests per feature type, PSI and KS alert thresholds, monitoring cadence, and evidence-based retraining triggers – including a runnable PSI calculator.