Model Evaluation Suite Skill for Minority-Class Recall

Compare models on the same holdout with a confusion matrix and per-class recall as well as aggregate accuracy.

Install in Claude Code

Use Python, scikit-learn, and a synthetic CSV with ground truth and two prediction columns. The full source workflow can integrate its model-evaluation-suite plugin.

Get the complete `plugins/ai-ml/model-evaluation-suite/skills/evaluating-machine-learning-models` directory from the pinned GitHub commit.
Copy it into the target project's `.claude/skills/`. Check `.claude/skills/evaluating-machine-learning-models/SKILL.md`, start Claude Code at the project root, and call `/evaluating-machine-learning-models`.

Example task

Create 90 A and 10 B labels. Model one predicts only A. Model two gets eight B labels right and mislabels five A cases as B. Compare accuracy and B recall.

What to check

Model one gets 90% accuracy and zero B recall. Model two gets 93% accuracy and 80% B recall, traceable to its confusion matrix.

Original SKILL.md on GitHub · Updated 2026-11-13

Frequently asked question

Does accuracy show how every class performs?

Review class support, precision, and recall too. Imbalance can hide minority-class failures in the overall metric.