Data Science & ML

Designs ML features with leakage-safe pipelines, correct categorical encoding by cardinality, numeric transforms, and validation that a feature earns its place. Use when someone asks "how should I encode this high-cardinality column", "why does my model score great offline and fail in production", "what features should I build from this table", or "should I scale these inputs". Do NOT use for building the serving infrastructure that stores and versions features across models - use feature-store-design instead; for detecting when live feature distributions shift after deployment - use data-drift-monitor instead; for initial dataset exploration and profiling - use eda-playbook instead.

Designs a reusable, leakage-safe feature store - entity and naming contracts, point-in-time correct training joins, feature versioning, online/offline consistency with staleness SLOs, and governance. Use when someone asks "should we build a feature store", "how do I share features across models", "our model trains great but serves garbage", "how do I version a feature definition", or is diagnosing training-serving skew. Do NOT use for designing the feature transformations themselves - use ml-feature-engineering instead; for monitoring feature and prediction distributions after deployment - use data-drift-monitor instead; for general relational schema design - use database-schema instead.

Sets up disciplined ML experiment tracking - run logging schemas, artifact versioning, naming conventions, and reproducibility standards - and produces the run-record template a team actually follows. Use when someone asks "how should we track our ML experiments", "why can't we reproduce this result", "how do I set up MLflow or W&B for the team", or is bootstrapping a new ML project. Do NOT use for analyzing product A/B tests - use ab-test-analyzer instead; for monitoring deployed models use data-drift-monitor; for judging whether a trained model is good enough to ship use model-evaluation-report.

Produces a rigorous, honest evaluation report for an ML model - real baselines, business-matched metrics with confidence intervals, slice-level error analysis, calibration checks, and a go/no-go recommendation. Use when someone asks "is this model good enough to ship", "write up the eval for this model", "did we actually beat the old model", or before any model promotion, stakeholder demo, or retraining decision. Do NOT use for evaluating LLM or prompt outputs - use llm-evaluation instead; for analyzing randomized product experiments use ab-test-analyzer; for documenting a shipped model for consumers use model-card-writer.

Produces a complete model card - intended use, training data provenance, evaluation results with slices, limitations, and usage guidance - for any model shared beyond its team or affecting people. Use when someone asks "write a model card", "document this model before release", "what should our model documentation include", or is preparing a model for deployment, handoff, or external publication. Do NOT use for producing the underlying evaluation numbers - use model-evaluation-report instead; for ongoing production monitoring plans use data-drift-monitor.

Designs a production drift-monitoring plan for ML systems - statistical tests per feature type, PSI and KS alert thresholds, monitoring cadence, and evidence-based retraining triggers - including a runnable PSI calculator. Use when someone asks "how do I know if my model is drifting", "set up drift monitoring for this model", "should we retrain", or is investigating unexplained model performance degradation. Do NOT use for judging whether a model is good enough to ship in the first place - use model-evaluation-report instead; for tracking training runs use experiment-tracking; for documenting the model for consumers use model-card-writer.