medsci-skills 5.15.0 → 5.16.0

package/README.md

@@ -2,14 +2,14 @@
 
 # MedSci Skills
 
-**53 skills that actually work.** Built by a physician-researcher, tested on real publications.
+**54 skills that actually work.** Built by a physician-researcher, tested on real publications.
 
 *MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer — 44 reporting guidelines and risk-of-bias tools, reference/citation verification, and deterministic integrity gates before peer review — now extended by a model-engineering lane that scaffolds reproducible, leakage-safe training repos and audits model validation. Clinical AI model research engineering is in scope; a general AI-scientist platform is not. It competes on clinical submission reliability, not skill count.*
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 [![Release](https://img.shields.io/github/v/release/Aperivue/medsci-skills?style=flat-square&color=blue)](https://github.com/Aperivue/medsci-skills/releases/latest)
 [![CI](https://img.shields.io/github/actions/workflow/status/Aperivue/medsci-skills/validate.yml?branch=main&style=flat-square&label=CI)](https://github.com/Aperivue/medsci-skills/actions/workflows/validate.yml)
-![Skills](https://img.shields.io/badge/Skills-53-brightgreen?style=flat-square)
+![Skills](https://img.shields.io/badge/Skills-54-brightgreen?style=flat-square)
 [![npm](https://img.shields.io/npm/v/medsci-skills?style=flat-square&label=npm&color=cb3837)](https://www.npmjs.com/package/medsci-skills)
 [![Watch the 2-min intro](https://img.shields.io/badge/▶_Watch-2--min_intro-FF0000?style=flat-square&logo=youtube&logoColor=white)](https://youtu.be/MclQ_RIofpE)
 [![good first issues](https://img.shields.io/github/issues/Aperivue/medsci-skills/good%20first%20issue?style=flat-square&label=good%20first%20issues&color=7057ff)](https://github.com/Aperivue/medsci-skills/contribute)
@@ -285,6 +285,8 @@ The E2E pipeline (`orchestrate --e2e`) produces everything up to `qc/`. The `sub
 
 ## What's New
 
+**v5.16** — model-engineering produce-side depth, clinical fine-tuning focus (Items 3–4 of the [produce-side depth roadmap](docs/roadmap_model_engineering_depth.md)). A new **`radiomics-ml`** skill + `check_radiomics_ml` gate for the most common solo-doable clinical-ML workflow — radiomics / tabular features → any classical learner (LASSO / SVM / random forest / XGBoost / …) → a clinical outcome, no GPU — with a learner-agnostic nested-CV / calibration / stability gate; plus a **`model-scaffold` fine-tuning mode** (`--task finetune` + `--from-pretrained`) that adapts a pretrained backbone on collected clinical data with a frozen→unfrozen schedule, discriminative learning rates, and a pretrained-weight provenance record (`PRETRAINED.md` + a `PRETRAINED_PROVENANCE_MISSING` verdict on the existing `check_training_hygiene`, plus a MedSAM-adaptation + train-only diffusion-augmentation guide). Additive and backward-compatible; **54 skills / 44 guidelines / 49 integrity detectors / 22 domain-probe modules**. (See the [CHANGELOG](CHANGELOG.md) for v5.0–v5.15.)
+
 **v5.15** — model-engineering produce-side depth. Two new skills that *produce* what the review lane previously only audited: **`preprocess-imaging`** (DICOM/NIfTI data prep + a `check_preprocessing_leakage` gate that extends the split-leakage moat upstream to the data stage) and **`explainability`** (Grad-CAM / saliency held to the reviewer bar — Adebayo sanity checks, quantitative localisation vs ground truth, attribution-not-validation framing, via `check_explainability_report`). Plus a by-stage skill index, multi-host README/About (Claude Code · Codex · Cursor · Copilot), copy-paste citation ergonomics, and a real-project precision fix. Additive and backward-compatible; **53 skills / 44 guidelines / 48 integrity detectors / 22 domain-probe modules**. (See the [CHANGELOG](CHANGELOG.md) for v5.0–v5.14.)
 
 **v4.10** — reviewer-coverage expansion reverse-engineered from high-IF, CC-BY papers (learn-only under the `reverse_engineer/` license firewall), plus a clinician-friendly update path. Additive and backward-compatible; 45 skills / **44 guidelines** / 36 detectors / **15 domain-probe modules** (was 12):
@@ -402,6 +404,8 @@ MedSci Skills is **opinionated and narrow on purpose**: a single physician-resea
 
 📖 **Per-skill reference:** [`docs/skills/`](docs/skills/) — one page per skill (what it does, when it activates, its Quality Card — purpose, safety boundaries, known limitations, validation, evidence — and bundled resources), generated from each `SKILL.md` + `skill.yml`. See [`docs/skills/AUDIT.md`](docs/skills/AUDIT.md) for how the skills are validated.
 
+🧠 **ML / DL method coverage:** [`docs/method_coverage_map.md`](docs/method_coverage_map.md) — which machine-learning and deep-learning method families (CNN / transformer / segmentation / detection / foundation / diffusion / SSL for imaging; the full classical family — penalised regression, SVM, k-NN, trees, boosting [XGBoost / LightGBM / CatBoost], MLP, ensembles, clustering — for radiomics/tabular; LLM/MLLM) map to which skills for selection, production, validation, interpretation, and reporting.
+
 ```
                               ┌─────────────────────────────────┐
                               │  orchestrate: single entry point │
@@ -451,14 +455,14 @@ ma-scout -> search-lit -> fulltext-retrieval -> design-study ──> write-proto
 
 ### By research stage
 
-All 53 skills, grouped by where they fit in the clinical-manuscript and medical-AI lifecycle. Full descriptions are in the table below; one page per skill lives in the [per-skill reference](docs/skills/).
+All 54 skills, grouped by where they fit in the clinical-manuscript and medical-AI lifecycle. Full descriptions are in the table below; one page per skill lives in the [per-skill reference](docs/skills/).
 
 | Stage | Skills |
 |-------|--------|
 | 🔭 **Discover & scope** | `ma-scout` · `find-cohort-gap` · `search-lit` · `fulltext-retrieval` · `lit-sync` · `author-strategy` |
 | 📐 **Design & plan** | `design-study` · `calc-sample-size` · `define-variables` · `write-protocol` · `fill-protocol` · `design-ai-benchmarking` |
 | 🧹 **Data & analysis** | `deidentify` · `clean-data` · `generate-codebook` · `version-dataset` · `analyze-stats` · `batch-cohort` · `cross-national` · `replicate-study` |
-| 🤖 **Medical-AI model engineering** | `preprocess-imaging` · `architecture-zoo` · `model-scaffold` · `model-validation` · `model-evaluation` · `explainability` · `model-card` · `mllm-eval` |
+| 🤖 **Medical-AI model engineering** | `preprocess-imaging` · `architecture-zoo` · `model-scaffold` · `model-validation` · `model-evaluation` · `explainability` · `radiomics-ml` · `model-card` · `mllm-eval` |
 | ✍️ **Write & visualize** | `write-paper` · `make-figures` · `review-paper` · `present-paper` · `humanize` · `polish-language` · `academic-aio` |
 | ✅ **Comply & verify** | `check-reporting` · `self-review` · `verify-refs` · `manage-refs` |
 | 📤 **Submit & respond** | `find-journal` · `add-journal` · `sync-submission` · `revise` · `peer-review` · `fill-icmje-coi` |
@@ -481,11 +485,12 @@ All 53 skills, grouped by where they fit in the clinical-manuscript and medical-
 | **design-ai-benchmarking** | Design and validity review for benchmarking AI system(s) against a human-expert panel: evaluation-question and arm definition, decoupled multi-dimensional rubrics with anchors, planted calibration probes (positive-control / known-bad / instability / mechanism-contradiction), reviewer-panel construction with per-reviewer randomization, inter-rater reliability targets with separate control-item reliability, LLM-as-judge vs human-as-judge adjudication, construct-independence guards, and a structured JSON rating-export schema. Locks the rubric before data collection. |
 | **model-validation** | Design or audit the clinical-validation study for an engineer-built medical-imaging model (segmentation / classification / detection): patient-level split disjointness and the data-leakage taxonomy, tuning-on-test, internal vs genuine external validation, comparator design, single-run vs multi-seed variance, task-correct metric selection (Metrics Reloaded), test-set sizing, and CLAIM 2024 / TRIPOD+AI / STARD-AI reporting fit. Ships a deterministic split-leakage gate that proves patient disjointness by set arithmetic on the emitted split table. Integrates with MONAI / nnU-Net — does not replace them. |
 | **preprocess-imaging** | Design or audit the data-preparation stage of a medical-imaging model — DICOM/NIfTI intake, resampling, intensity normalization, and the augmentation plan — so the pipeline is leakage-safe before `model-scaffold` builds the training repo. Emits a declarative preprocessing manifest and a deterministic data-stage leakage gate (`check_preprocessing_leakage`) that catches what the split table cannot see: a dataset-level normaliser fit on non-train data (`NORMALIZATION_LEAKAGE`), a data-fitted transform run before the split (`PREPROCESS_BEFORE_SPLIT`), and a patient's slices crossing splits (`PATIENT_CROSS_SPLIT`). Integrates MONAI / TorchIO transforms; never reimplements them or touches real patient data. |
-| **model-scaffold** | Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation (U-Net), classification, detection, image-to-image synthesis, or self-supervised pretraining — the missing middle link between choosing an architecture and validating a trained model. Emits a patient-level seed-locked split as an auditable artifact, a task-appropriate model, train/evaluate scripts that seed every RNG and infer under eval mode, a config, requirements, a reproducibility record, and a Methods stub with VERIFY placeholders (no fabricated numbers). Reproducibility holds by construction; ships a `check_training_hygiene` AST gate + a network-free build→validate challenge. Emits a minimal runnable default model for a forward-pass smoke test, and integrates with MONAI / nnU-Net / TorchIO / timm / torchvision for production-grade models. |
+| **model-scaffold** | Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation (U-Net), classification, detection, image-to-image synthesis, self-supervised pretraining, or fine-tuning a pretrained backbone (transfer learning) — the missing middle link between choosing an architecture and validating a trained model. Emits a patient-level seed-locked split as an auditable artifact, a task-appropriate model, train/evaluate scripts that seed every RNG and infer under eval mode, a config, requirements, a reproducibility record, and a Methods stub with VERIFY placeholders (no fabricated numbers). Fine-tuning mode (`--task finetune`) adds a frozen→unfrozen schedule, discriminative learning rates, and a pretrained-weight provenance record (`PRETRAINED.md`), with a MedSAM-adaptation + train-only diffusion-augmentation guide. Reproducibility holds by construction; ships a `check_training_hygiene` AST gate (RNG seeding, eval-mode inference, train-split-only loaders, pretrained-provenance) + a network-free build→validate challenge. Integrates with MONAI / nnU-Net / TorchIO / timm / torchvision for production-grade models. |
 | **architecture-zoo** | "Which architecture for which research question" decision tool: maps task (classification / segmentation / detection / transfer), modality, data scale, and class imbalance to a paper-grounded architecture shortlist. Curates the foundational curriculum (ResNet / DenseNet / EfficientNet / ViT / Swin; U-Net / 3-D U-Net / Attention & Residual U-Net / nnU-Net / Mask R-CNN; SAM/MedSAM / TotalSegmentator / BiomedCLIP / DINO / MAE / SimCLR) — each with core idea, when-to-use, medical-imaging use, reference implementation, validation setup, and the matching model-scaffold template. Advisory; teaches archetypes, not a live SOTA leaderboard. |
 | **model-card** | Generate the documentation an engineer-built medical-imaging model must carry — a Model Card (Mitchell et al. 2019), a Datasheet for its dataset (Gebru et al. 2021), and a METRIC-informed data-quality pass — filled from user-supplied facts (never fabricated), then verify every required section is present and non-empty with a deterministic completeness gate (`check_model_card_complete`). Model Card / Datasheet are documentation standards vendored as templates, not counted reporting checklists. |
 | **model-evaluation** | Compute and report task-correct held-out metrics for a trained medical-imaging model — segmentation (Dice + a boundary metric HD95/NSD, per structure), classification (AUROC + AUPRC + sensitivity/specificity with bootstrap CIs at the deployment prevalence), or detection (FROC/mAP with a stated IoU criterion) — plus calibration and subgroup slices. Emits a per-case table for analyze-stats and gates the metric choice against Metrics Reloaded / CLAIM 2024 (`check_metric_reporting`). Numbers come only from executed code. |
 | **explainability** | Produce or audit the interpretability/explainability analysis of a medical-imaging model — Grad-CAM / Grad-CAM++ / attention-rollout / saliency / integrated-gradients — so it clears the rigor bar: mandatory Adebayo sanity checks (model- and data-randomisation), a quantitative localisation metric against ground truth (IoU / pointing game / Dice) instead of eyeballed examples, a cohort-level result, and attribution framing rather than "proof the model is correct". Emits an explainability-report manifest + a deterministic gate (`check_explainability_report`): `SALIENCY_AS_VALIDATION`, `NO_SANITY_CHECK`, `NO_LOCALIZATION_METRIC` (Major); `INSUFFICIENT_SANITY`, `CHERRY_PICKED_EXAMPLES`, `MISSING_METHOD` (Minor). Integrates captum / pytorch-grad-cam; never reimplements them or touches real patient data. |
+| **radiomics-ml** | Produce or audit a radiomics / tabular clinical-ML study — features → random forest / XGBoost / regularised logistic → clinical outcome — the most common solo-doable clinical-ML workflow (no GPU, no engineer). Emits a pipeline manifest + a deterministic rigor gate (`check_radiomics_ml`): `NO_NESTED_CV`, `HIGH_DIM_LOW_EVENTS`, `SELECTION_OUTSIDE_CV` (Major); `NO_FEATURE_STABILITY`, `NO_CALIBRATION`, `NO_EXTERNAL_VALIDATION` (Minor). pyradiomics/IBSI settings, nested CV with in-fold selection, ICC stability, SHAP, calibration + decision curve, CLEAR/TRIPOD+AI/PROBAST-AI reporting. Integrates scikit-learn / xgboost / pyradiomics; never reimplements them. |
 | **mllm-eval** | Model-agnostic evaluation harness (closed API or open weights) for an LLM/MLLM on a clinical task — radiology report generation, VQA, clinical text extraction — covering the adjudicated reference standard, clinical-efficacy metrics (RadGraph-F1 / CheXbert-F1 beyond BLEU/ROUGE), faithfulness/hallucination, pretraining-contamination, prompt sensitivity, and a reader study; gates the plan with `check_mllm_eval_completeness` and routes the reviewer audit to the MLLM probe. |
 | **intake-project** | Classifies new research projects, summarizes current state, identifies missing inputs, and recommends next steps. |
 | **grant-builder** | Structures grant proposals: significance, innovation, approach, milestones, and consortium roles. |

package/metadata/distribution_files.json

@@ -2868,8 +2868,13 @@
     },
     {
       "path": "skills/model-scaffold/SKILL.md",
-      "size": 7686,
-      "sha256": "cede2020d3ceee0e1599cbcfb412230ab9c04b08032c8bbaff620e4129ec6785"
+      "size": 9237,
+      "sha256": "d7e736178cb60d2d13cef9174474297bd54bc6d4d1f3862023d802ff0c034a95"
+    },
+    {
+      "path": "skills/model-scaffold/references/finetuning_guide.md",
+      "size": 6208,
+      "sha256": "b7817d53eeeb506b573beaae1889fa863dd81286d474ab99b46b38ad046b32e7"
     },
     {
       "path": "skills/model-scaffold/references/training_guide.md",
@@ -2878,13 +2883,13 @@
     },
     {
       "path": "skills/model-scaffold/scripts/check_training_hygiene.py",
-      "size": 11752,
-      "sha256": "66970049c85e46a0080b5f19eb3b5ac9dfd656a674b6f1698825e968eedd4814"
+      "size": 14095,
+      "sha256": "57542a54dac607e89e7f13e5025bd2ce6b32e629064a18cbb0f0c15fdfc6181c"
     },
     {
       "path": "skills/model-scaffold/scripts/scaffold.py",
-      "size": 40470,
-      "sha256": "33569806ecd230aebb9d35c77a27b8480d23329b00f633d7fee9531a7bcec974"
+      "size": 51482,
+      "sha256": "150abbc4864a92913b202893962c29cda8ec6ce1918f35558aa5b02514b6a0cb"
     },
     {
       "path": "skills/model-scaffold/scripts/scaffold_challenge/expected/split_assignment.csv",
@@ -2898,18 +2903,18 @@
     },
     {
       "path": "skills/model-scaffold/scripts/scaffold_challenge/problem.md",
-      "size": 2678,
-      "sha256": "369e98da65cbd16c67f5ff3f17d99b707d713bbf3239409f9921190c85f8f08a"
+      "size": 3226,
+      "sha256": "950af6c956f73b491b2bee102d884765aaeb783b5842eb428ac2ad81b49f67e8"
     },
     {
       "path": "skills/model-scaffold/scripts/scaffold_challenge/verify.sh",
-      "size": 5136,
-      "sha256": "040acee712943e8392b70b40b790438bec056a3a4e7a282f76455d2a2b791447"
+      "size": 6396,
+      "sha256": "77d81387647abf00de4b03f91a7c7bf9bef38d9679b148762c906eaccfee9f73"
     },
     {
       "path": "skills/model-scaffold/skill.yml",
-      "size": 3195,
-      "sha256": "06c4ed02872bfc38abd1c753c02a640a2bfaa1141fbfdd9f8b9ccbe18a148d82"
+      "size": 3772,
+      "sha256": "aaae59edb50c50b01be2aaca3c94b63ce76af067ff25750248517c5ce91ccf41"
     },
     {
       "path": "skills/model-validation/SKILL.md",
@@ -3411,6 +3416,56 @@
       "size": 1372,
       "sha256": "81caa10775246f2db3f5e08948292f63076d44ffc039d8eec764f1d46aab275f"
     },
+    {
+      "path": "skills/radiomics-ml/SKILL.md",
+      "size": 8273,
+      "sha256": "cc65bfd50db2b6e9b6681e1c9649b306490cd5dd74192bfdf5316362b71f141b"
+    },
+    {
+      "path": "skills/radiomics-ml/references/radiomics_ml_guide.md",
+      "size": 6707,
+      "sha256": "29bf2b2b847c488ddb33e50cd21ea0e2cd4937a1a62ab2655f31227a0e42215a"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml.py",
+      "size": 10179,
+      "sha256": "b1cfbd9991aa8871d5aabc528c9aa22eecf8e46cb5983872bca2359229223840"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml_challenge/expected/strong.txt",
+      "size": 369,
+      "sha256": "c73c776e6558d9a78dacd3c63ae1cc3dee183d9e1a2f27d99b1e0db70f44997e"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml_challenge/expected/weak.txt",
+      "size": 1408,
+      "sha256": "3bb90f31587e94f3f2b267874a9ac58cce5adf3436254bc206092e0acfb86cd7"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml_challenge/fixture/pipeline_strong.json",
+      "size": 322,
+      "sha256": "83d128fe0b728ef52a1fd98ef5b5bd478c3e48455cdb07d8040f9d130df81cee"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml_challenge/fixture/pipeline_weak.json",
+      "size": 313,
+      "sha256": "2fa1a3f98f952faf2e6e1faf1bdbf4675a44dc88679dcfaf18a4f32eb47ba9e7"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml_challenge/problem.md",
+      "size": 894,
+      "sha256": "9a6abad4e12f9b4e9290eb305fb9ad04127fa1cde03961a38ac4ccb5431a2100"
+    },
+    {
+      "path": "skills/radiomics-ml/scripts/check_radiomics_ml_challenge/verify.sh",
+      "size": 1988,
+      "sha256": "b9e55aaf9b29f3e810b837fc373c7a212dc5f778a49bac11854bdb50ed84a184"
+    },
+    {
+      "path": "skills/radiomics-ml/skill.yml",
+      "size": 3790,
+      "sha256": "9b99b87add656873d3f7277ce12490ffea36d919e9538bc4f4fbe1ebbfedc18e"
+    },
     {
       "path": "skills/render-pdf-doc/SKILL.md",
       "size": 7490,

package/metadata/distribution_manifest.json

@@ -1,6 +1,6 @@
 {
   "schema_version": 1,
-  "version": "5.15.0",
+  "version": "5.16.0",
   "owned_skills": [
     "academic-aio",
     "add-journal",
@@ -43,6 +43,7 @@
     "preprocess-imaging",
     "present-paper",
     "publish-skill",
+    "radiomics-ml",
     "render-pdf-doc",
     "replicate-study",
     "review-paper",

package/metadata/skills_catalog.json

@@ -1,6 +1,6 @@
 {
   "_comment": "AUTO-GENERATED by scripts/gen_skills_catalog_json.py from each skills/<slug>/SKILL.md + skill.yml. Machine-readable skill catalog (single source of truth) consumed by external surfaces such as the aperivue.com storefront to gate skill-list completeness. Do not hand-edit; CI gate: python3 scripts/gen_skills_catalog_json.py --check.",
-  "skill_count": 53,
+  "skill_count": 54,
   "categories": [
     {
       "key": "literature_references",
@@ -38,7 +38,8 @@
         "model-evaluation",
         "model-scaffold",
         "model-validation",
-        "preprocess-imaging"
+        "preprocess-imaging",
+        "radiomics-ml"
       ]
     },
     {
@@ -415,7 +416,7 @@
       "layer": "B",
       "owner_domain": "model_development",
       "maturity": "official",
-      "description": "Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation, classification, detection, image-to-image synthesis, or self-supervised pretraining — the missing mid…"
+      "description": "Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation, classification, detection, image-to-image synthesis, self-supervised pretraining, or fine-tuning a pr…"
     },
     {
       "slug": "model-validation",
@@ -480,6 +481,15 @@
       "maturity": "official",
       "description": "Convert a personal agent skill into a distributable, open-source-ready skill."
     },
+    {
+      "slug": "radiomics-ml",
+      "category": "model_engineering",
+      "category_label": "Model Engineering & Validation",
+      "layer": "D",
+      "owner_domain": "model_validation",
+      "maturity": "official",
+      "description": "Produce or audit a radiomics / tabular clinical-ML study — imaging or clinical features → any classical learner (penalised logistic [LASSO / ridge / elastic-net], SVM, k-NN, naive Bayes, LDA/QDA, deci…"
+    },
     {
       "slug": "render-pdf-doc",
       "category": "presentation_tooling",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "medsci-skills",
-  "version": "5.15.0",
+  "version": "5.16.0",
   "description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
   "license": "SEE LICENSE IN LICENSE",
   "homepage": "https://github.com/Aperivue/medsci-skills#readme",

package/skills/model-scaffold/SKILL.md

@@ -2,14 +2,16 @@
 name: model-scaffold
 description: >
   Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation,
-  classification, detection, image-to-image synthesis, or self-supervised pretraining — the missing
-  middle link between choosing an architecture and validating a trained model. Emits a patient-level
-  seed-locked split as an auditable artifact, a task-appropriate model, train and evaluate scripts that
-  seed every RNG and infer under eval mode, a config, requirements, a reproducibility record, and a
-  Methods stub with VERIFY placeholders (no fabricated numbers). The reproducibility guarantees hold by
-  construction, so the build is leakage-safe before any training runs. Integrates with MONAI, nnU-Net,
-  TorchIO, timm, and torchvision — it does not reimplement them.
-triggers: model scaffold, scaffold a model, training repo, PyTorch repo, build a model, train a model, segmentation, classification, detection, image synthesis, self-supervised, SimCLR, Pix2Pix, Faster R-CNN, U-Net, UNet, nnU-Net, MONAI, timm, torchvision, dataloader, train.py, patient-level split, reproducible training, seed everything, generate training code, medical imaging model
+  classification, detection, image-to-image synthesis, self-supervised pretraining, or fine-tuning a
+  pretrained backbone (transfer learning) — the missing middle link between choosing an architecture and
+  validating a trained model. Emits a patient-level seed-locked split as an auditable artifact, a
+  task-appropriate model, train and evaluate scripts that seed every RNG and infer under eval mode, a
+  config, requirements, a reproducibility record, and a Methods stub with VERIFY placeholders (no
+  fabricated numbers). Fine-tuning mode adds a frozen-then-unfrozen schedule, discriminative learning
+  rates, and a pretrained-weight provenance record. The reproducibility guarantees hold by construction,
+  so the build is leakage-safe before any training runs. Integrates with MONAI, nnU-Net, TorchIO, timm,
+  and torchvision — it does not reimplement them.
+triggers: model scaffold, scaffold a model, training repo, PyTorch repo, build a model, train a model, fine-tune, finetune, transfer learning, pretrained backbone, MedSAM, SAM adaptation, segmentation, classification, detection, image synthesis, self-supervised, SimCLR, Pix2Pix, Faster R-CNN, U-Net, UNet, nnU-Net, MONAI, timm, torchvision, dataloader, train.py, patient-level split, reproducible training, seed everything, generate training code, medical imaging model
 tools: Read, Write, Edit, Bash, Grep, Glob
 model: inherit
 ---
@@ -20,7 +22,8 @@ model: inherit
 
 This skill stamps out a **runnable PyTorch training repo** for a medical-imaging task — `--task`
 **segmentation** (U-Net), **classification** (CNN / `timm` backbone), **detection** (torchvision Faster
-R-CNN / FPN), **synthesis** (Pix2Pix generator + PatchGAN), or **ssl** (SimCLR encoder) —
+R-CNN / FPN), **synthesis** (Pix2Pix generator + PatchGAN), **ssl** (SimCLR encoder), or **finetune**
+(transfer-learning a pretrained backbone with a frozen→unfrozen schedule + a provenance record) —
 with the reproducibility guarantees **baked in by construction** — so the build is leakage-safe and
 reproducible before a single epoch runs. It is the imaging analogue of how `/analyze-stats` generates
 runnable statistical code: the generator produces the repo, you run the training on your GPU / Colab,
@@ -34,6 +37,9 @@ in the generated `requirements.txt`); it does not reimplement them.
 ## When to use
 - You have a data manifest (one row per image, with a patient/subject ID) and want a reproducible,
   leakage-safe starting repo for a segmentation model.
+- You want to **fine-tune a pretrained backbone** (transfer learning — the common clinician workflow:
+  a `timm` / MONAI / MedSAM checkpoint adapted to your collected clinical data) with the freeze schedule,
+  discriminative learning rates, and pretrained-weight provenance recorded (`--task finetune`).
 
 ## When NOT to use
 - Auditing an already-trained model's validation design → `/model-validation`.
@@ -54,8 +60,14 @@ patient level off this column.
 python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
   --manifest <manifest.csv> --task segmentation --out model_repo --seed 42 \
   --in-channels 1 --out-channels 1
-# --task = segmentation | classification | detection | synthesis | ssl
-#   (out-channels = num classes for classification, target channels for synthesis)
+# --task = segmentation | classification | detection | synthesis | ssl | finetune
+#   (out-channels = num classes for classification/finetune, target channels for synthesis)
+# fine-tuning a pretrained backbone (transfer learning) on collected clinical data:
+python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
+  --manifest <manifest.csv> --task finetune --out model_repo --seed 42 \
+  --out-channels <num_classes> --from-pretrained timm:resnet50.a1_in1k
+#   emits PRETRAINED.md (provenance) + a frozen→unfrozen train.py with discriminative LRs;
+#   record the exact pretrained source so the fine-tune is reproducible.
 ```
 This writes `model_repo/` with `config.yaml`, `model.py` (the task's model — U-Net / CNN / Faster R-CNN
 / Pix2Pix / SimCLR encoder), `dataset.py` (reads the frozen split), `losses.py` (task-appropriate),
@@ -79,9 +91,11 @@ patient-disjointness proof, and (optionally, locally with torch installed)
 ### Phase 4 — Plug in your data and train
 Implement `dataset.py`'s `_load_image` / `_load_label` for your modality (DICOM / NIfTI / TIFF via
 nibabel / pydicom / tifffile / TorchIO / MONAI transforms). For production, swap `model.py` for MONAI
-`UNet` / `SegResNet` or an nnU-Net plan (see `${CLAUDE_SKILL_DIR}/references/training_guide.md`). Run
-`python train.py` (best model selected on the **val** split), then `python evaluate.py` (predictions on
-the **test** split, touched once).
+`UNet` / `SegResNet` or an nnU-Net plan (see `${CLAUDE_SKILL_DIR}/references/training_guide.md`). For a
+fine-tuning repo (`--task finetune`), fill `PRETRAINED.md` and set the freeze schedule / discriminative
+learning rates (see `${CLAUDE_SKILL_DIR}/references/finetuning_guide.md`, which also covers MedSAM/SAM
+adaptation and train-only diffusion augmentation). Run `python train.py` (best model selected on the
+**val** split), then `python evaluate.py` (predictions on the **test** split, touched once).
 
 ### Phase 5 — Validate, evaluate, publish
 Hand off to `/model-validation` (validation-tier + comparator + metric-selection audit),
@@ -110,7 +124,9 @@ runnability.
 ## Deterministic gates
 - `scripts/scaffold.py` — the generator (stdlib + numpy; deterministic given manifest + seed).
 - `scripts/check_training_hygiene.py` — AST linter: all RNGs seeded, cuDNN deterministic,
-  `eval()` + `no_grad()` inference, no training on a non-train split.
+  `eval()` + `no_grad()` inference, no training on a non-train split, and (fine-tuning) a
+  recorded pretrained-weight provenance when pretrained weights are loaded
+  (`PRETRAINED_PROVENANCE_MISSING`).
 - `scripts/scaffold_challenge/verify.sh` — the build → validate chain, network-free (torch tier
   self-skips).
 

package/skills/model-scaffold/references/finetuning_guide.md

@@ -0,0 +1,91 @@
+# Fine-tuning guide (model-scaffold — `--task finetune`)
+
+Load-on-demand notes for the target workflow of a clinician-researcher: **fine-tune an
+existing pretrained model on collected clinical data**, rather than train from scratch or
+design a new architecture. `scaffold.py --task finetune` emits a leakage-safe transfer-learning
+repo (frozen→unfrozen schedule, discriminative learning rates, a `PRETRAINED.md` provenance
+record); this guide covers the decisions it deliberately leaves to you.
+
+## When to fine-tune (vs train from scratch)
+Fine-tuning a pretrained backbone is the right default whenever the labelled clinical set is
+small (hundreds–low-thousands of patients) — the regime almost every solo clinical study is
+in. From-scratch training needs far more data to beat a fine-tuned ImageNet/medical backbone.
+Pick the backbone in `/architecture-zoo`; adapt it here.
+
+## The provenance record is load-bearing — `PRETRAINED.md`
+A fine-tune is only reproducible if the **exact** pretrained weights are recorded. `--task
+finetune` emits `PRETRAINED.md` with `[VERIFY]` fields; fill them before publishing:
+- **Source** — the exact model + weights tag (`timm:resnet50.a1_in1k`, `MedSAM`, a URL/DOI),
+  set via `--from-pretrained`.
+- **Pretraining data** — the dataset the backbone was pretrained on. **Confirm it does not
+  overlap this study's test set.** A backbone pretrained on (or including) your evaluation
+  images is *pretraining-set contamination*: leakage that no train/val/test split table can
+  see, because it entered through the weights, not the split. This is the fine-tuning analogue
+  of benchmark contamination in LLM evaluation (`/mllm-eval`).
+- **License**, **checkpoint hash** (sha256), **access date** — for auditability.
+
+`check_training_hygiene.py` fires `PRETRAINED_PROVENANCE_MISSING` (Minor) when a training
+script loads pretrained weights (`pretrained=True` / `from_pretrained`) but the repo carries
+no `PRETRAINED.md` and no `pretrained:` block in `config.yaml`. The scaffold passes by
+construction; a hand-rolled repo (a copied Kaggle notebook that does
+`timm.create_model(..., pretrained=True)` and records nothing) fails.
+
+## The frozen→unfrozen schedule
+The emitted `train.py` implements the standard two-phase transfer schedule:
+1. **Freeze the backbone, warm up the fresh head** for a few epochs (`FREEZE_EPOCHS`). The
+   randomly-initialised head would otherwise back-propagate large gradients that damage the
+   pretrained features.
+2. **Unfreeze with discriminative learning rates** — a small learning rate for the pretrained
+   backbone (`BACKBONE_LR`, e.g. 1e-5) and a larger one for the head (`HEAD_LR`, e.g. 1e-3).
+   The backbone should *adapt*, not be overwritten.
+
+Tune the two rates and `FREEZE_EPOCHS` per task. For very small datasets, keep more of the
+backbone frozen (train only the last block + head) and lean on regularisation (weight decay,
+dropout, strong but physiology-preserving augmentation, early stopping on the **val** split).
+
+## BatchNorm and small clinical batches
+Fine-tuning with small batches destabilises BatchNorm running statistics. Options: keep the
+backbone's BN layers in `eval()` (frozen running stats) while the backbone is frozen; switch
+to GroupNorm; or use a larger effective batch (gradient accumulation). Record the choice in
+Methods — it materially affects reproducibility.
+
+## MedSAM / SAM adaptation
+Promptable foundation segmenters (SAM, **MedSAM**) are adapted, not retrained. The common,
+solo-doable path is **adapter / decoder fine-tuning**: freeze the heavy image encoder and
+fine-tune only the mask decoder (and optionally a lightweight adapter or the prompt encoder)
+on your labelled masks. Integrate the upstream implementation
+([MedSAM](https://github.com/bowang-lab/MedSAM),
+[SAM](https://github.com/facebookresearch/segment-anything)) — do not reimplement it. Keep the
+scaffold's split-reading contract (`splits/split_assignment.csv`) so the patient-level split
+is preserved, and record the SAM/MedSAM checkpoint in `PRETRAINED.md`. Prompt design (points /
+boxes / automatic) is part of the method — report it, and report whether prompts used any
+ground-truth information at test time (a leakage trap unique to promptable models).
+
+## Diffusion augmentation — train split only
+A diffusion model can synthesise extra training images (an off-the-shelf augmentation, **not**
+a novel method here). Two hard rules keep it leakage-safe:
+1. **Any generative model used for augmentation must be trained on the TRAIN split only.** If
+   the diffusion (or GAN) model saw val/test images, its samples leak that distribution into
+   training — the generative analogue of fitting a normaliser on full data
+   (`/preprocess-imaging` `NORMALIZATION_LEAKAGE`).
+2. **Synthetic images augment TRAIN only, never val/test.** Evaluating on — or augmenting —
+   the held-out split with synthetic data invalidates the metric. `check_preprocessing_leakage`
+   flags `AUGMENTATION_ON_EVAL`; declare synthetic augmentation in the preprocessing manifest
+   as `applies_to: ["train"]`.
+Report the synthetic:real ratio and show the result holds without synthetic data (a sensitivity
+analysis) — reviewers discount models propped up by synthetic training data.
+
+## Reporting
+Fill `methods_stub.md` and `PRETRAINED.md`, then hand off to `/write-paper` +
+`/check-reporting`. Fine-tuning specifics reviewers expect: the pretrained source + its
+pretraining data (contamination check), the freeze/unfreeze schedule and learning rates,
+and — for SAM/MedSAM — the prompt protocol. Report held-out metrics as **mean ± SD over ≥ 3
+seeds** (`/model-evaluation` → `/analyze-stats`); fine-tuning is seed-sensitive on small data.
+
+## Hand-offs
+- Backbone / architecture choice (incl. SAM/MedSAM, diffusion) → `/architecture-zoo`.
+- Data-stage leakage (normalisation fit, augmentation-on-eval, slice crossing) →
+  `/preprocess-imaging` (`check_preprocessing_leakage`).
+- Split / validation-design audit → `/model-validation` (`check_split_leakage.py`).
+- Held-out metrics + CIs → `/model-evaluation` → `/analyze-stats`.
+- Provenance/model documentation → `/model-card` (Model Card + Datasheet).

package/skills/model-scaffold/scripts/check_training_hygiene.py

@@ -27,6 +27,14 @@ CHECKS (verdicts):
   5. EVAL_SHUFFLE            (Minor)  an evaluation DataLoader uses shuffle=True
                                       (reorders the test set; harmless for metrics but
                                       a smell, and breaks index-aligned outputs).
+  6. PRETRAINED_PROVENANCE_MISSING (Minor)  the training script loads PRETRAINED weights
+                                      (a `pretrained=True` kwarg or a `from_pretrained`
+                                      call — fine-tuning / transfer learning) but the repo
+                                      records no pretrained-weight provenance (no non-empty
+                                      PRETRAINED.md and no `pretrained:` block in
+                                      config.yaml). A fine-tune whose starting checkpoint is
+                                      unrecorded is not reproducible. Fires only in --repo
+                                      mode (the provenance record is a repo-level artifact).
 
 INPUTS
   --repo   a scaffolded repo directory; train.py and evaluate.py are auto-found.
@@ -84,6 +92,7 @@ def _scan(src: str):
         "cudnn_determ": False,        # cudnn.deterministic = True
         "dataset_split": {},          # var name -> split literal it was built with
         "loaders": [],                # (first_arg_name, shuffle_bool)
+        "loads_pretrained": False,    # a pretrained=True kwarg or a from_pretrained call
     }
     # dataset var <- Ctor(..., split="X")
     for node in ast.walk(tree):
@@ -120,6 +129,9 @@ def _scan(src: str):
                 name = first.id if isinstance(first, ast.Name) else None
                 sh = _kw(node, "shuffle")
                 facts["loaders"].append((name, _is_true(sh)))
+            # pretrained-weight load: `...(pretrained=True)` or a `...from_pretrained(...)` call
+            if _is_true(_kw(node, "pretrained")) or tail == "from_pretrained":
+                facts["loads_pretrained"] = True
         # cudnn.deterministic = True (assignment to an Attribute target)
         if isinstance(node, ast.Assign):
             for tgt in node.targets:
@@ -129,7 +141,21 @@ def _scan(src: str):
     return facts
 
 
-def analyze(train: str | None, eval_: str | None) -> dict:
+def _has_pretrained_provenance(repo: Path) -> bool:
+    """True if the repo records pretrained-weight provenance: a non-empty PRETRAINED.md, or
+    a `pretrained:` block in config.yaml."""
+    pm = repo / "PRETRAINED.md"
+    if pm.is_file() and pm.read_text(encoding="utf-8").strip():
+        return True
+    cfg = repo / "config.yaml"
+    if cfg.is_file():
+        for line in cfg.read_text(encoding="utf-8").splitlines():
+            if line.strip().startswith("pretrained:"):
+                return True
+    return False
+
+
+def analyze(train: str | None, eval_: str | None, repo: str | None = None) -> dict:
     claims = []
     tfacts = _scan(Path(train).read_text(encoding="utf-8")) if train else None
     efacts = _scan(Path(eval_).read_text(encoding="utf-8")) if eval_ else None
@@ -160,6 +186,16 @@ def analyze(train: str | None, eval_: str | None) -> dict:
                               f"'{name}' constructed with split=\"{sp}\" — training on the {sp} split",
                     "where": Path(train).name,
                 })
+        # fine-tuning: pretrained weights loaded but no provenance recorded (repo mode only)
+        if tfacts["loads_pretrained"] and repo and not _has_pretrained_provenance(Path(repo)):
+            claims.append({
+                "verdict": "PRETRAINED_PROVENANCE_MISSING", "severity": "Minor",
+                "detail": "the training script loads pretrained weights (fine-tuning) but the "
+                          "repo records no pretrained-weight provenance (a non-empty PRETRAINED.md "
+                          "or a 'pretrained:' block in config.yaml). Record the exact "
+                          "source / checkpoint / license / hash so the fine-tune is reproducible.",
+                "where": Path(train).name,
+            })
 
     if efacts is not None:
         if not (efacts["has_eval"] and efacts["has_no_grad"]):
@@ -229,7 +265,7 @@ def main() -> int:
     args = ap.parse_args()
 
     train, eval_ = _resolve(args.repo, args.train, args.eval_)
-    result = analyze(train, eval_)
+    result = analyze(train, eval_, repo=args.repo)
 
     if not args.quiet:
         print("=" * 41)