ML Data Pipeline Architecture Skill for Lazy Polars Reads
Compose a lazy Parquet pipeline, inspect the execution plan, and verify the resulting aggregate from a small fixture.
中文Install in Claude Code
Use Python, Polars, and your own Parquet sample with region, quantity, and active fields.
- Get the complete `plugins/devops-tools/skills/ml-data-pipeline-architecture` directory from the pinned GitHub commit.
- Copy it into the target project's `.claude/skills/`. Check `.claude/skills/ml-data-pipeline-architecture/SKILL.md`, start Claude Code at the project root, and call `/ml-data-pipeline-architecture`.
Example task
Create North active 2, North inactive 7, and South active 3. Filter active rows and sum quantity by region before collecting.What to check
The results are North 2 and South 3. Inspect the execution plan for projection or predicate pushdown.
Original SKILL.md on GitHub · Updated
Frequently asked question
Does every LazyFrame operation read the entire Parquet file?
Operations build a plan before execution. Inspect explain to see which optimizations are available for the source and expressions.