Sets up disciplined ML experiment tracking - run logging schemas, artifact versioning, naming conventions, and reproducibility standards - and produces the run-record template a team actually follows. Use when someone asks "how should we track our ML experiments", "why can't we reproduce this result", "how do I set up MLflow or W&B for the team", or is bootstrapping a new ML project. Do NOT use for analyzing product A/B tests - use ab-test-analyzer instead; for monitoring deployed models use data-drift-monitor; for judging whether a trained model is good enough to ship use model-evaluation-report.
Click to play with sound.
---
name: Experiment Tracking
description: Sets up disciplined ML experiment tracking - run logging schemas, artifact versioning, naming conventions, and reproducibility standards - and produces the run-record template a team actually follows. Use when someone asks "how should we track our ML experiments", "why can't we reproduce this result", "how do I set up MLflow or W&B for the team", or is bootstrapping a new ML project. Do NOT use for analyzing product A/B tests - use ab-test-analyzer instead; for monitoring deployed models use data-drift-monitor; for judging whether a trained model is good enough to ship use model-evaluation-report.
---
# Experiment Tracking
An experiment no one can reproduce is a result no one can trust. Disciplined tracking is not overhead - it is the minimum viable scientific practice for ML, and the alternative is a promoted model whose provenance nobody can reconstruct when it misbehaves in production. This skill installs the logging schema, naming discipline, and promotion workflow that make every run auditable.
## Operating procedure
Follow the steps in order: the run schema (Step 2) must exist before naming conventions (Step 3) matter, and reproducibility standards (Step 4) are unenforceable until the schema captures seeds and environments.
### Step 1: Gather inputs
Collect these before touching tooling. Where the answer is unknown, record the default and label it a guess.
1. Team size and collaboration need. Default: solo or small team.
2. Existing tooling, if any (MLflow, Weights and Biases, homegrown spreadsheets).
3. Where artifacts live today (local disk, S3, GCS) - shared object storage is required for teams.
4. Model iteration style: iterative training with epochs (deep learning) or single-fit (trees, linear models). This decides whether per-epoch curves are logged.
5. The promotion decision this tracking must serve: what metric, on what test set, decides whether a model ships.
Tooling default: MLflow for self-hosted setups; Weights and Biases when the team needs collaboration features. Both satisfy every requirement below. Use the tracking server, not local file logging, in any team setting, and point artifact storage at a shared location (S3 or GCS) from day one.
… install to load the full skill