Experiment Tracking Setup Skill for Comparing MLflow Runs
Add consistent experiment logs so two model runs can be compared by parameter, held-out metric, and dataset artifact.
中文Install in Claude Code
Use Python, a local MLflow instance, and one synthetic training fixture with a fixed validation split.
- Get the complete `plugins/ai-ml/experiment-tracking-setup/skills/setting-up-experiment-tracking` directory from the pinned GitHub commit.
- Copy it into the target project's `.claude/skills/`. Check `.claude/skills/setting-up-experiment-tracking/SKILL.md`, start Claude Code at the project root, and call `/setting-up-experiment-tracking`.
Example task
Create depth-3 and depth-5 runs in demo-orders. Log max_depth, the shared seed, validation MAE, and a dataset-summary.json artifact in each run.What to check
MLflow displays two identifiable runs with the expected parameters, comparable metrics, and downloadable dataset summaries.
Original SKILL.md on GitHub · Updated
Frequently asked question
Can validation MAE alone reproduce the comparison?
Log data version, split or seed, model parameters, and the relevant environment versions as well.