medsci-skills 5.24.0 → 5.25.0

package/README.md

@@ -2,14 +2,16 @@
 
 # MedSci Skills
 
-**58 skills that actually work.** Built by a physician-researcher, tested on real publications.
+[English](README.md) | [简体中文](README.zh-CN.md)
 
-*MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer — 47 reporting guidelines and risk-of-bias tools, reference/citation verification, and deterministic integrity gates before peer review — now extended by a model-engineering lane that scaffolds reproducible, leakage-safe training repos and audits model validation. Clinical AI model research engineering is in scope; a general AI-scientist platform is not. It competes on clinical submission reliability, not skill count.*
+**59 skills that actually work.** Built by a physician-researcher, tested on real publications.
+
+*MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer — 49 reporting guidelines and risk-of-bias tools, reference/citation verification, and deterministic integrity gates before peer review — now extended by a model-engineering lane that scaffolds reproducible, leakage-safe training repos and audits model validation. Clinical AI model research engineering is in scope; a general AI-scientist platform is not. It competes on clinical submission reliability, not skill count.*
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 [![Release](https://img.shields.io/github/v/release/Aperivue/medsci-skills?style=flat-square&color=blue)](https://github.com/Aperivue/medsci-skills/releases/latest)
 [![CI](https://img.shields.io/github/actions/workflow/status/Aperivue/medsci-skills/validate.yml?branch=main&style=flat-square&label=CI)](https://github.com/Aperivue/medsci-skills/actions/workflows/validate.yml)
-![Skills](https://img.shields.io/badge/Skills-58-brightgreen?style=flat-square)
+![Skills](https://img.shields.io/badge/Skills-59-brightgreen?style=flat-square)
 [![npm](https://img.shields.io/npm/v/medsci-skills?style=flat-square&label=npm&color=cb3837)](https://www.npmjs.com/package/medsci-skills)
 [![npm downloads](https://img.shields.io/npm/dw/medsci-skills?style=flat-square&label=npm%20downloads&color=cb3837)](https://www.npmjs.com/package/medsci-skills)
 [![Watch the 2-min intro](https://img.shields.io/badge/▶_Watch-2--min_intro-FF0000?style=flat-square&logo=youtube&logoColor=white)](https://youtu.be/MclQ_RIofpE)
@@ -170,9 +172,9 @@ protocol scaffolding, screening/extraction structure, PRISMA-consistent counts a
 diagram, pooled-estimate figures, and a manuscript draft. *Safety:* screening and
 extraction decisions stay with the human review team.
 
-## Live Demos: Four Study Types, Four Full Pipelines
+## Live Demos: Five Study Types, Five Full Pipelines
 
-Four public datasets. Four study types. Demos 1–3 each produce a complete manuscript, publication-ready figures, and a reporting-compliance audit; Demo 4 runs the medical-AI **model-engineering lane** end to end (scaffold → gates → training → evaluation → interpretability).
+Five public datasets. Five study types. Demos 1–3 each produce a complete manuscript, publication-ready figures, and a reporting-compliance audit; Demo 4 runs the medical-AI **model-engineering lane** end to end (scaffold → gates → training → evaluation → interpretability); Demo 5 takes that lane onto a GPU cluster and out to a **genuinely external cohort**, then reports where it broke.
 
 | Demo | Dataset | Study Type | Compliance |
 |------|---------|------------|------------|
@@ -180,6 +182,7 @@ Four public datasets. Four study types. Demos 1–3 each produce a complete manu
 | [Demo 2: BCG Vaccine](demo/02_metafor_bcg/) | `metafor::dat.bcg` (13 RCTs) | Meta-analysis | PRISMA 2020 |
 | [Demo 3: NHANES Obesity](demo/03_nhanes_obesity/) | CDC NHANES 2017-18 | Epidemiology (survey) | STROBE |
 | [Demo 4: PneumoniaMNIST CNN](demo/04_pneumoniamnist_cnn/) | `medmnist` (CC BY 4.0) | Medical-AI model engineering (CNN) | CLAIM / TRIPOD+AI |
+| [Demo 5: MSD → AMOS spleen](demo/05_msd_amos_spleen/) | MSD Task09 + AMOS22 (CC BY 4.0) | 3-D segmentation, external validation + modality shift | CLAIM / Metrics Reloaded |
 
 ### Demo 1: Diagnostic Accuracy — Wisconsin Breast Cancer
 
@@ -264,6 +267,45 @@ data(dat.bcg)  # 13 RCTs, 357,347 participants (Colditz et al. 1994)
 
 **Pipeline:** `analyze-stats` → `make-figures` → `write-paper` → AI pattern scan → `check-reporting` (STROBE) → `self-review` → DOCX build → `present-paper`
 
+### Demo 5: External Validation — MSD → AMOS spleen segmentation
+
+The only demo that leaves the laptop, and the only one that reports a **failure**. It asks whether a
+clinician can carry a deep-learning study to a defensible result without an engineer, then answers by
+running a three-rung external-validation ladder on a GPU cluster and logging every point where the
+answer was no.
+
+| Rung | Cohort | n scored | Dice median [95% CI] |
+|---|---|---:|---|
+| 1 internal | MSD held-out | 9 | **0.9595** [0.9367–0.9734] |
+| 2 genuine external | AMOS **CT** | 298 / 300 | **0.8932** [0.8633–0.9108] |
+| 3 modality shift | AMOS **MRI** | 59 / 60 | **0.0152** [0.0000–0.0626] |
+| 3b counterfactual | AMOS MRI, **rescaled** | 59 / 60 | **0.3016** [0.1744–0.4048] |
+
+Rung 3 is a **constructed** test, and the demo says so: the evaluation plan named the normalisation
+contract and predicted the collapse *before* inference ran. The trained plan carries `CTNormalization`
+into inference and there is no flag that says "this is MRI", so a Hounsfield-unit clip is applied to
+arbitrary-unit images: **0 of 60 MRI cases contain a negative voxel** (against 300 of 300 on CT), and a
+median **23.2 %** of voxels are flattened at the clip ceiling (against 2.7 %). The run exits 0 and
+returns a file for all 60 cases, 20 of them empty. Only ground truth made it loud.
+
+A fourth arm changes **only the input intensity scale** — same checkpoint, no retraining — and
+recovers median Dice to **0.3016** (+0.2864 [+0.1204, +0.4048]) while staying **−0.5916** below the
+CT arm. So both mechanisms are real and differently sized: roughly **0.29 Dice is the preprocessing
+contract, 0.59 a representation that does not transfer**. The arm and its prediction were written
+down before it ran.
+
+And `/profile-imaging` **had already flagged the mixed intensity scale before training — as a
+Minor**, where it sat in `qc/` for nine days. So the gap this demo found is not detection; it is
+**routing and severity**. A gate that fires correctly into a directory no later step reads is,
+operationally, a gate that did not fire. See [the full demo](demo/05_msd_amos_spleen/) — including
+[`FRICTION.md`](demo/05_msd_amos_spleen/FRICTION.md), which lists every point that needed engineering
+knowledge, because the headline question is not answerable honestly without it.
+
+**Pipeline:** `profile-imaging` → `model-sourcing` → `preprocess-imaging` (leakage gate, plus the
+counterfactual that must fail) → `model-validation` (split gate) → nnU-Net training →
+`model-evaluation` → `make-figures` → write-up. `bash reproduce.sh` re-runs the gates and the whole
+across-cohort analysis on a laptop; training needs ~50 GPU-hours and says so.
+
 ### Project Folder Structure
 
 Each demo (and real project) follows this role-based folder layout:
@@ -418,7 +460,7 @@ Earlier in this series: analysis-integrity guards (confounding completeness, cla
 | **Battle-tested** | Used on real manuscript submissions by a practicing physician-researcher | Unknown provenance and validation |
 | **Depth per skill** | 150-600 lines of documentation + bundled reference files (curated journal profile library, checklists, formula sheets, code templates) | Typically thin SKILL.md templates |
 
-**MedSci-Audit** — the verification edge in the first rows above is a named suite of **84 deterministic detectors** (citation & reference integrity, cohort & pool arithmetic, scope/estimand contracts, reporting compliance, and more) that catch fabricated or drifted content before a manuscript reaches a reviewer. See **[`MEDSCI_AUDIT.md`](MEDSCI_AUDIT.md)** for the suite, its six families, and its evaluation evidence.
+**MedSci-Audit** — the verification edge in the first rows above is a named suite of **90 deterministic detectors** (citation & reference integrity, cohort & pool arithmetic, scope/estimand contracts, reporting compliance, and more) that catch fabricated or drifted content before a manuscript reaches a reviewer. See **[`MEDSCI_AUDIT.md`](MEDSCI_AUDIT.md)** for the suite, its six families, and its evaluation evidence.
 
 ---
 
@@ -485,11 +527,11 @@ ma-scout -> search-lit -> fulltext-retrieval -> design-study ──> write-proto
 
 ### By research stage
 
-All 58 skills, grouped by where they fit in the clinical-manuscript and medical-AI lifecycle. Full descriptions are in the table below; one page per skill lives in the [per-skill reference](docs/skills/).
+All 59 skills, grouped by where they fit in the clinical-manuscript and medical-AI lifecycle. Full descriptions are in the table below; one page per skill lives in the [per-skill reference](docs/skills/).
 
 | Stage | Skills |
 |-------|--------|
-| 🔭 **Discover & scope** | `ma-scout` · `find-cohort-gap` · `search-lit` · `fulltext-retrieval` · `lit-sync` · `author-strategy` |
+| 🔭 **Discover & scope** | `ma-scout` · `find-cohort-gap` · `search-lit` · `fulltext-retrieval` · `lit-sync` · `obsidian-paper-vault` · `author-strategy` |
 | 📐 **Design & plan** | `design-study` · `calc-sample-size` · `define-variables` · `write-protocol` · `fill-protocol` · `design-ai-benchmarking` |
 | 🧹 **Data & analysis** | `deidentify` · `clean-data` · `generate-codebook` · `version-dataset` · `analyze-stats` · `batch-cohort` · `cross-national` · `replicate-study` |
 | 🤖 **Medical-AI model engineering** | `profile-imaging` · `preprocess-imaging` · `architecture-zoo` · `model-scaffold` · `model-validation` · `model-evaluation` · `uncertainty-imaging` · `explainability` · `radiomics-ml` · `model-card` · `mllm-eval` |
@@ -507,7 +549,7 @@ All 58 skills, grouped by where they fit in the clinical-manuscript and medical-
 | **search-lit** | PubMed + Semantic Scholar + bioRxiv search with anti-hallucination citation verification. Token-efficient error handling -- CrossRef failures are silently batched, not repeated. BibTeX output tags each entry with `verified`/`verified_by`/`verified_on` fields so downstream skills can trust the citation provenance. |
 | **verify-refs** | Pre-submission reference audit for `.md`, `.docx`, `.bib`, or `.tsv` inputs. Extracts references, verifies DOI/PMID via CrossRef/PubMed when available, and writes `qc/reference_audit.json` as the sole output — row-level status (OK / MISMATCH / UNVERIFIED / FABRICATED) lives inside the JSON `records[]` block. `/search-lit` produces candidate BibTeX; `/lit-sync` owns `manuscript/_src/refs.bib`. |
 | **fulltext-retrieval** | Batch open-access PDF downloader. Unpaywall → PMC → OpenAlex → CrossRef pipeline. OA-only -- no paywall bypass. Input: DOI list or TSV. Optional PDF→Markdown conversion via [pymupdf4llm](https://pymupdf.readthedocs.io/en/latest/pymupdf4llm/) for token-efficient LLM analysis of academic papers. |
-| **check-reporting** | Manuscript compliance audit against 47 reporting guidelines and risk of bias tools (STROBE, STROBE-MR, RECORD, GATHER, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, CROSS, SRQR, COREQ, PRISMA, PRISMA-DTA, PRISMA-P, PRISMA-ScR, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA). Machine-readable JSON summary with `compliance_pct` and `fixable_by_ai` flags for automated pipeline integration. |
+| **check-reporting** | Manuscript compliance audit against 49 reporting guidelines and risk of bias tools (STROBE, STROBE-MR, RECORD, GATHER, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, CROSS, SRQR, COREQ, PRISMA, PRISMA 2020 for Abstracts, PRISMA-DTA, PRISMA-P, PRISMA-ScR, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-3, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA). Machine-readable JSON summary with `compliance_pct` and `fixable_by_ai` flags for automated pipeline integration. |
 | **analyze-stats** | Statistical analysis code generation (Python/R) for diagnostic accuracy, DTA meta-analysis (bivariate/HSROC), inter-rater agreement, survival analysis, demographics tables, regression (logistic/linear), propensity score (matching/IPTW/overlap weighting), and repeated measures (RM ANOVA/GEE/mixed models). Calibration mandatory for prediction models. |
 | **meta-analysis** | Full systematic review and meta-analysis pipeline (8 phases). DTA (bivariate/HSROC) and intervention meta-analysis. Protocol to submission-ready manuscript with PRISMA-DTA compliance. |
 | **make-figures** | Publication-ready figures and visual abstracts: ROC curves, forest plots, PRISMA/CONSORT/STARD flow diagrams, Kaplan-Meier curves, Bland-Altman plots, confusion matrices, and journal-specific visual/graphical abstracts (python-pptx template-based). Communication-first design principles (Nat Hum Behav 2026 — key message, audience, cognitive load, figure-vs-table decision) and five flow-diagram production lessons (official-template fidelity, VML fallback PDF export, docx XML escape, sequential placeholder mapping, version freeze); critic rubric Section G adds 5 communication-first checks. `--study-type` auto-generates the full required figure set; structured `_figure_manifest.md` output for downstream pipeline consumption; D2 enforced as default for flow diagrams. |
@@ -548,6 +590,7 @@ All 58 skills, grouped by where they fit in the clinical-manuscript and medical-
 | **peer-review** | Structured peer review drafting for medical journals. Systematic manuscript analysis, journal-specific formatting (RYAI, INSI, EURE, AJR, KJR), conciseness targets (500-800 words), and pre-submission QC checklist. Constructive developmental tone. |
 | **ma-scout** | Meta-analysis topic discovery and feasibility assessment. Two modes: (A) Professor-first — profile → pillar analysis → MA gaps, (B) Topic-first — question → landscape scan → co-author matching. Multi-source validation (PubMed, PROSPERO, bioRxiv) with realistic k estimation (15-30% discount). |
 | **lit-sync** | Sync research references from .bib files to Zotero library + Obsidian literature notes. Concept extraction from 10+ literature notes with cross-cutting theme discovery. Works after `/search-lit` or standalone. |
+| **obsidian-paper-vault** | Turn a folder of research PDFs into an Obsidian vault: templated literature notes with frontmatter, PDF embed links, and atomic concept notes synthesized across papers. Batch notes are written from pre-extracted text, never from PDF paths handed to subagents. Shares vault folders with `/lit-sync` and never overwrites an existing note. |
 | **academic-aio** | AI search engine (Perplexity / ChatGPT web / Elicit / Consensus / SciSpace) and RAG visibility checklist for medical AI papers. Integrates TRIPOD+AI, CLAIM, STARD-AI, TRIPOD-LLM, DECIDE-AI reporting anchors with generative-engine-optimization (GEO) principles. Covers title, abstract, structured summary boxes (Key Points / Research in Context / Plain-Language Summary), preprints, GitHub README, `CITATION.cff`, Zenodo, and Hugging Face model/dataset cards. Explicit defense against LLM citation fabrication (Agarwal 2025, Nat Commun). Produces a visible PASS/PARTIAL/FAIL checklist; never applies edits silently. Pairs with `write-paper` Phase 4/6/7, runs after `self-review` + `humanize`. |
 | **polish-language** | Academic English consistency linting and non-native (ESL) clarity polish. A deterministic linter (`lint_consistency.py`) flags abbreviation define-once violations, US/UK spelling drift, hyphen-vs-en-dash numeric ranges, `P`/`p` case and impossible `P = 0.000`, hyphenation variants, small-number style, and value/unit spacing — then a gated, style-only clarity pass fixes wording without ever changing numbers, citations, or scientific meaning. Distinct from `humanize` (AI-tell removal) and `check-reporting` (guideline items); bundles a reproducible challenge card. |
 | **manage-refs** | Reference lifecycle as a single skill: citekey ↔ `.bib` validation, journal-CSL pandoc rendering (`render_pandoc.sh`), manuscript ↔ rendered DOCX cross-reference QC (`check_xref.py --strict` is the submission gate), `[N]` ↔ `[@key]` marker conversion, and native Zotero CWYW field-code injection for co-author Word workflows. Hybrid 3-phase strategy (pandoc draft → CWYW transition → Zotero CWYW for circulation/revision/submission). Sole writer of `manuscript_final.docx` and `qc/xref_audit.json`. Split out of `write-paper` Phase 7.6 so `revise`, `peer-review`, `sync-submission`, and `find-journal` can render directly without depending on a sibling skill. |
@@ -696,8 +739,8 @@ Projects declare their source-of-truth layout in `SSOT.yaml`, and a `qc/migratio
 ### Meta-Analysis Failure Modes
 `/meta-analysis` ships empirical failure-mode references (data integrity, review orchestration, submission package drift, post-submission release ops) with four automation hooks: `scripts/prisma_5way_consistency.py` (DI-6 PRISMA number consistency), `scripts/extraction_consensus_log_init.py` (DI-1 dual-extraction scaffold), `scripts/tag_cleanup_gate.sh` (DI-8 placeholder tag gate), and `scripts/verify_package_integrity.py` (SPD SHA-256 manifest for submission bundles).
 
-### 47 Reporting Guidelines & RoB Tools Built-in
-`check-reporting` includes bundled checklists for 47 guidelines and risk-of-bias tools: STROBE, STROBE-MR, RECORD, GATHER, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, CROSS, SRQR, COREQ, PRISMA 2020, PRISMA-DTA, PRISMA-P, PRISMA-ScR, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA. Includes Results/Discussion section boundary checks and machine-readable JSON summary for pipeline integration.
+### 49 Reporting Guidelines & RoB Tools Built-in
+`check-reporting` includes bundled checklists for 49 guidelines and risk-of-bias tools: STROBE, STROBE-MR, RECORD, GATHER, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, CROSS, SRQR, COREQ, PRISMA 2020, PRISMA 2020 for Abstracts, PRISMA-DTA, PRISMA-P, PRISMA-ScR, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-3, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA. Includes Results/Discussion section boundary checks and machine-readable JSON summary for pipeline integration.
 
 ### Publication-Ready Output
 `analyze-stats` generates reproducible Python/R code for 13 analysis types -- including regression, propensity score, and repeated measures -- with mandatory calibration for prediction models. `make-figures` produces journal-specification figures (300 DPI, colorblind-safe palettes, proper dimensions), visual/graphical abstracts, and a tool selection guide (D2 for flow diagrams, matplotlib for data plots). `--study-type` auto-generates the complete figure set for each study design.