medsci-skills 5.8.0 → 5.9.0

package/README.md

@@ -51,7 +51,9 @@ integrity gates** for submission-grade biomedical research. As of **v5.0** it ad
 **model-engineering lane**: choose a paper-grounded architecture, scaffold a
 reproducible, leakage-safe PyTorch training repo, and validate, document, and
 evaluate a medical-imaging or LLM/MLLM model so the work reaches a paper — it
-**integrates** MONAI / nnU-Net, never reimplements them. Clinical AI model research
+ships a minimal runnable default model for a forward-pass smoke test and
+**integrates** MONAI / nnU-Net / timm / torchvision for production-grade models,
+rather than reimplementing the ecosystem. Clinical AI model research
 engineering is in scope; it is **not** a diagnostic tool, an autonomous author, or a
 general AI-scientist platform, and every output requires human-expert verification.
 New here? See the [3 workflows below](#start-here-3-workflows), the
@@ -459,7 +461,7 @@ ma-scout -> search-lit -> fulltext-retrieval -> design-study ──> write-proto
 | **design-study** | Study design review: identifies analysis unit, cohort logic, data leakage risks, comparator design, validation strategy, and reporting guideline fit. |
 | **design-ai-benchmarking** | Design and validity review for benchmarking AI system(s) against a human-expert panel: evaluation-question and arm definition, decoupled multi-dimensional rubrics with anchors, planted calibration probes (positive-control / known-bad / instability / mechanism-contradiction), reviewer-panel construction with per-reviewer randomization, inter-rater reliability targets with separate control-item reliability, LLM-as-judge vs human-as-judge adjudication, construct-independence guards, and a structured JSON rating-export schema. Locks the rubric before data collection. |
 | **model-validation** | Design or audit the clinical-validation study for an engineer-built medical-imaging model (segmentation / classification / detection): patient-level split disjointness and the data-leakage taxonomy, tuning-on-test, internal vs genuine external validation, comparator design, single-run vs multi-seed variance, task-correct metric selection (Metrics Reloaded), test-set sizing, and CLAIM 2024 / TRIPOD+AI / STARD-AI reporting fit. Ships a deterministic split-leakage gate that proves patient disjointness by set arithmetic on the emitted split table. Integrates with MONAI / nnU-Net — does not replace them. |
-| **model-scaffold** | Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation (U-Net), classification, detection, image-to-image synthesis, or self-supervised pretraining — the missing middle link between choosing an architecture and validating a trained model. Emits a patient-level seed-locked split as an auditable artifact, a task-appropriate model, train/evaluate scripts that seed every RNG and infer under eval mode, a config, requirements, a reproducibility record, and a Methods stub with VERIFY placeholders (no fabricated numbers). Reproducibility holds by construction; ships a `check_training_hygiene` AST gate + a network-free build→validate challenge. Integrates with MONAI / nnU-Net / TorchIO / timm / torchvision — does not reimplement them. |
+| **model-scaffold** | Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation (U-Net), classification, detection, image-to-image synthesis, or self-supervised pretraining — the missing middle link between choosing an architecture and validating a trained model. Emits a patient-level seed-locked split as an auditable artifact, a task-appropriate model, train/evaluate scripts that seed every RNG and infer under eval mode, a config, requirements, a reproducibility record, and a Methods stub with VERIFY placeholders (no fabricated numbers). Reproducibility holds by construction; ships a `check_training_hygiene` AST gate + a network-free build→validate challenge. Emits a minimal runnable default model for a forward-pass smoke test, and integrates with MONAI / nnU-Net / TorchIO / timm / torchvision for production-grade models. |
 | **architecture-zoo** | "Which architecture for which research question" decision tool: maps task (classification / segmentation / detection / transfer), modality, data scale, and class imbalance to a paper-grounded architecture shortlist. Curates the foundational curriculum (ResNet / DenseNet / EfficientNet / ViT / Swin; U-Net / 3-D U-Net / Attention & Residual U-Net / nnU-Net / Mask R-CNN; SAM/MedSAM / TotalSegmentator / BiomedCLIP / DINO / MAE / SimCLR) — each with core idea, when-to-use, medical-imaging use, reference implementation, validation setup, and the matching model-scaffold template. Advisory; teaches archetypes, not a live SOTA leaderboard. |
 | **model-card** | Generate the documentation an engineer-built medical-imaging model must carry — a Model Card (Mitchell et al. 2019), a Datasheet for its dataset (Gebru et al. 2021), and a METRIC-informed data-quality pass — filled from user-supplied facts (never fabricated), then verify every required section is present and non-empty with a deterministic completeness gate (`check_model_card_complete`). Model Card / Datasheet are documentation standards vendored as templates, not counted reporting checklists. |
 | **model-evaluation** | Compute and report task-correct held-out metrics for a trained medical-imaging model — segmentation (Dice + a boundary metric HD95/NSD, per structure), classification (AUROC + AUPRC + sensitivity/specificity with bootstrap CIs at the deployment prevalence), or detection (FROC/mAP with a stated IoU criterion) — plus calibration and subgroup slices. Emits a per-case table for analyze-stats and gates the metric choice against Metrics Reloaded / CLAIM 2024 (`check_metric_reporting`). Numbers come only from executed code. |

package/installers/install.py

@@ -167,10 +167,23 @@ def run_self_test() -> int:
     return 0
 
 
+LOG_DIR = REPO_ROOT / "installers" / ".logs"
+LOG_KEEP = 10  # retain the most recent N install logs; prune older
+
+
 def write_log(log_lines: list[str]) -> Path:
+    """Write the timestamped install log to installers/.logs/ (gitignored) and keep only
+    the most recent LOG_KEEP — logs used to accumulate in the repo root."""
+    LOG_DIR.mkdir(parents=True, exist_ok=True)
     stamp = dt.datetime.now().strftime("%Y%m%d-%H%M%S")
-    log_path = REPO_ROOT / f"{stamp}-{LOG_NAME}"
+    log_path = LOG_DIR / f"{stamp}-{LOG_NAME}"
     log_path.write_text("\n".join(log_lines) + "\n", encoding="utf-8")
+    old = sorted(LOG_DIR.glob(f"*-{LOG_NAME}"))
+    for stale in old[:-LOG_KEEP]:
+        try:
+            stale.unlink()
+        except OSError:
+            pass
     return log_path
 
 

package/metadata/distribution_files.json

@@ -23,8 +23,8 @@
     },
     {
       "path": "installers/install.py",
-      "size": 14230,
-      "sha256": "f1a092bbc5ea676a3bfca3dcb724e9e64a07dc1e1e2414ff696fc21cf4f7f8c0"
+      "size": 14720,
+      "sha256": "b22e5d5be62ba1f71a81ad3214aed2cbfefbf97fd52179b21a8452a11ae95fcf"
     },
     {
       "path": "installers/medsci_txn.py",
@@ -153,8 +153,8 @@
     },
     {
       "path": "skills/analyze-stats/SKILL.md",
-      "size": 54473,
-      "sha256": "6f7a194db60588e84ddf8e62e80d6e143d14265e21bdac35efa6efbbdd5d5ffc"
+      "size": 55302,
+      "sha256": "3809b6a55e8a79b7dc48f617e3be0a7faf2a96a6f0ba03d58977a39f267cce81"
     },
     {
       "path": "skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md",
@@ -513,8 +513,8 @@
     },
     {
       "path": "skills/calc-sample-size/SKILL.md",
-      "size": 20967,
-      "sha256": "da3427c78bf251def5433b5ffad106e9069770be347668e6b14bb3ce8e925d40"
+      "size": 23904,
+      "sha256": "0bed1ee5d43f044a490d77fac181900f9e513dfd379831d0446b118b4768b798"
     },
     {
       "path": "skills/calc-sample-size/references/formulas.md",
@@ -531,6 +531,11 @@
       "size": 2142,
       "sha256": "97062f5aa63ce8f5b5e340e6b65683720b3ce1d94e34d201977d9a2601293038"
     },
+    {
+      "path": "skills/calc-sample-size/references/prediction_model_sample_size.md",
+      "size": 5074,
+      "sha256": "bf3d42ad59d2a3b3abc2f00db9559de671514b7b64bd4cf44ec53717966dc635"
+    },
     {
       "path": "skills/calc-sample-size/skill.yml",
       "size": 2958,
@@ -808,8 +813,8 @@
     },
     {
       "path": "skills/check-reporting/scripts/check_framework_naming.py",
-      "size": 7719,
-      "sha256": "bb71f1628375779e00438e6ec317c9637cd7bb9c82d71db516097c991f520d58"
+      "size": 8455,
+      "sha256": "bd44ffb3f55d6539f972cd4bf50f1d9301e865da41e05c8d279776e94fb6d3d7"
     },
     {
       "path": "skills/check-reporting/scripts/check_prisma_figure.py",
@@ -1003,13 +1008,58 @@
     },
     {
       "path": "skills/design-study/SKILL.md",
-      "size": 18282,
-      "sha256": "dfd73871102af6beab3d21f60f650db41ec2e4025d6ee19eae8de61b17851fd4"
+      "size": 19220,
+      "sha256": "d4a05f6dd2530620bd1ec3e1c64ba0625aae25def12892247e61a77724132cd6"
+    },
+    {
+      "path": "skills/design-study/references/dag_adjustment.md",
+      "size": 4390,
+      "sha256": "8b6c9d2b2d4e79652b355cf30f0d56f4232619cad8589489707107f85bef9de3"
+    },
+    {
+      "path": "skills/design-study/references/target_trial_emulation.md",
+      "size": 6025,
+      "sha256": "3f3b8aa8256689a2109e06b0eaa92f0e793869838711c19f0b2d9e9e01dcb0ff"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_challenge/fixture/confounder.json",
+      "size": 111,
+      "sha256": "3d1c43f40a41e91f674cca1d47fd9bb8566e9d74f9ebb18aa097a41fbf0b11ff"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_challenge/fixture/instrument.json",
+      "size": 199,
+      "sha256": "5a99731963404b122cb3e233cc5c850885fe42cc49c9b1cfbac0206ea936703b"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_challenge/fixture/mbias.json",
+      "size": 182,
+      "sha256": "b05f7d537b9a25841172d4325fba858933aedab20718ac78e58547d11f8b6561"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_challenge/fixture/mediator.json",
+      "size": 106,
+      "sha256": "9e3fe1ed554a115058e9d7916856eaeca4d666c00741282960320cb87e1cfe91"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_challenge/problem.md",
+      "size": 2264,
+      "sha256": "79cba6ed5f3259a9e633c88d7a4d6d08994fff5f4f8461482dd86bcd2c7239ab"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_challenge/verify.sh",
+      "size": 2672,
+      "sha256": "87349a1eb69b1c5e1b6120c6e5b1269bc830ef91803e7f09219da2285451bb76"
+    },
+    {
+      "path": "skills/design-study/scripts/adjustment_set_helper.py",
+      "size": 8231,
+      "sha256": "239948c31d513f95f834fdf2d48e9938f7e585a2ebb12387a5abf3b06305dca0"
     },
     {
       "path": "skills/design-study/skill.yml",
-      "size": 1357,
-      "sha256": "6d90940e8eb13c65c6db08bfd4abdf5cf6f5cb01972ef205e19e81810ae9eb49"
+      "size": 1411,
+      "sha256": "3981d5da7734b90e6905ac329e996b145af82bd6a28ec2a8d42918e9ce077ee5"
     },
     {
       "path": "skills/fill-icmje-coi/SKILL.md",
@@ -1953,8 +2003,8 @@
     },
     {
       "path": "skills/make-figures/references/exemplar_plots/README.md",
-      "size": 4913,
-      "sha256": "a5a320c785154ec73e41863302c2b625c9a24aeb213cfac0f195414aa1570f01"
+      "size": 5872,
+      "sha256": "32f93f0c65b2fe353b17dd1345267c85d5646fcf7de7adb9f784be551fc900af"
     },
     {
       "path": "skills/make-figures/references/exemplar_plots/bland_altman.md",
@@ -2116,6 +2166,26 @@
       "size": 17167,
       "sha256": "88c5e0829bb7c6d1fb6d6e1f45e1ce61bccdd91a7ed9a153623e51d8bc30b651"
     },
+    {
+      "path": "skills/make-figures/scripts/render_core_figures.py",
+      "size": 14436,
+      "sha256": "35f91147d51dd73b75dc116be9e0a80588fe1daf502966e82468d3c62b12f25d"
+    },
+    {
+      "path": "skills/make-figures/scripts/render_core_figures_challenge/fixture/synthetic_inputs.json",
+      "size": 1429,
+      "sha256": "24c0944fcb1384f073988d0f6e28b8f375ba4a7eff62fb4e21bc7066726d5ebc"
+    },
+    {
+      "path": "skills/make-figures/scripts/render_core_figures_challenge/problem.md",
+      "size": 2485,
+      "sha256": "a4e62161806d95b637d9ce984541190e5e9e213e2522d1f8740d847ee10c1a30"
+    },
+    {
+      "path": "skills/make-figures/scripts/render_core_figures_challenge/verify.sh",
+      "size": 1907,
+      "sha256": "7f05fd9cd4e2bfdf6c5b7e8e2a61bee5f8fd44d7d2746e29e2ae77adec84b9c0"
+    },
     {
       "path": "skills/make-figures/scripts/validate_pptx_mac_compat.py",
       "size": 8662,
@@ -2123,8 +2193,8 @@
     },
     {
       "path": "skills/make-figures/skill.yml",
-      "size": 2146,
-      "sha256": "bc6f4a4d47ea7a53b733733ee356337edb8803a76fc0d2deb1d889f5931a7b1d"
+      "size": 2205,
+      "sha256": "b973c376998b15cefe9bffdf580889f905b60ddc2dead00cc5cbc900a6fd50f5"
     },
     {
       "path": "skills/make-figures/templates/official/NOTES.md",
@@ -2773,13 +2843,13 @@
     },
     {
       "path": "skills/model-validation/SKILL.md",
-      "size": 9810,
-      "sha256": "1a75a5a1f21f5a8778d0b77db2e99574bf37edda2a291d8dde6aafea4a207ff0"
+      "size": 10836,
+      "sha256": "5cfd4ac6d75b3a474408b10d8eead3fdf7be8a44e1578dcac332cc2203ae75fe"
     },
     {
       "path": "skills/model-validation/references/validation_design.md",
-      "size": 11427,
-      "sha256": "16d43b688ea63745174c7ca8fafd78a7342b26c34ad1e10e1fdbc117cceafb2e"
+      "size": 13780,
+      "sha256": "c7ad02c2679b241d94af70c05f22740a6ad75c61c0c70d71ba2e48c4f3d64bf5"
     },
     {
       "path": "skills/model-validation/scripts/check_split_leakage.py",
@@ -3418,8 +3488,8 @@
     },
     {
       "path": "skills/self-review/SKILL.md",
-      "size": 105186,
-      "sha256": "202bf201cbe1b5dfc3703cddaaf2dfc025af5456d5446656132b1545243b0098"
+      "size": 102413,
+      "sha256": "97219c57684d4eaced9e35b0968ca09da7731095c186a4e97757b4039f43f2f6"
     },
     {
       "path": "skills/self-review/references/domain-probes/ai_overclaiming.md",
@@ -3573,8 +3643,13 @@
     },
     {
       "path": "skills/self-review/references/panel_review_template.md",
-      "size": 14863,
-      "sha256": "a44567b5ea0c47971bcc75b0e6c46f3846d687240b0607ded07183d6fbaad86f"
+      "size": 15095,
+      "sha256": "f9c084e950602ec88f7465dc953ed2309148cab37a3bef5f1b4c80deca2707a4"
+    },
+    {
+      "path": "skills/self-review/references/phases/confounding_completeness.md",
+      "size": 5344,
+      "sha256": "051f2b066d2d708b161c1e97d666d9e214d128f1413a9c4822f0333e748bda14"
     },
     {
       "path": "skills/self-review/scripts/check_artifact_coverage.py",
@@ -3593,8 +3668,8 @@
     },
     {
       "path": "skills/self-review/scripts/check_claim_artifact.py",
-      "size": 10757,
-      "sha256": "2cc6690c917ecb064c9e6834fce6162787881599ba845f8cb6ecddd4d65520b6"
+      "size": 13065,
+      "sha256": "ac29a7e8145152e6641fce8eaf98766cfc943a034db72252cc38538d7ab59db2"
     },
     {
       "path": "skills/self-review/scripts/check_classical_style.py",

package/metadata/distribution_manifest.json

@@ -1,6 +1,6 @@
 {
   "schema_version": 1,
-  "version": "5.8.0",
+  "version": "5.9.0",
   "owned_skills": [
     "academic-aio",
     "add-journal",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "medsci-skills",
-  "version": "5.8.0",
+  "version": "5.9.0",
   "description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
   "license": "SEE LICENSE IN LICENSE",
   "homepage": "https://github.com/Aperivue/medsci-skills#readme",

package/skills/analyze-stats/SKILL.md

@@ -283,9 +283,20 @@ rho = 0.39 between a fasting marker (IQR 0.6 to 3.5 units, SD 3.05) and an index
 marker is associated with about 0.8 index units higher on average (monotonic-linear approximation;
 crude, unadjusted)."
 
-**Output contract**: add a "Real-world translation" line beneath each primary effect size in the
-results table or its footnote. For OR/HR/RR primary outcomes, add an NNT/NNH line with the baseline
-risk stated.
+**Output contract (clinical-utility is a default, not an optional add-on).** Report every
+primary effect in units a clinician acts on, by default — do not leave these as prose to be
+added later:
+- **OR/HR/RR primary outcomes** → report the relative measure **and** the absolute risk at
+  a stated baseline + absolute risk difference + **NNT** (or NNH = 1/ARI), baseline risk
+  explicit. A relative-only headline is incomplete.
+- **Continuous outcomes** → add the IQR/clinically-anchored "Real-world translation" line
+  beneath the effect size.
+- **Prediction / classification (incl. medical-AI) models** → a **decision-curve /
+  net-benefit** pass at the relevant threshold is standard output, not just AUC +
+  calibration. An incremental claim reports added **net benefit / NRI / IDI over the
+  established clinical model**, not the new model's AUC alone. See
+  `references/table-standards/table-types/incremental_value.md` and the `make-figures`
+  `decision_curve` exemplar (and `render_core_figures.py` for the rendered curve).
 
 ## Error Handling
 

package/skills/calc-sample-size/SKILL.md

@@ -20,6 +20,7 @@ Python (alternative), interpret effect sizes clinically, and produce IRB-ready j
 
 - **Formulas**: `${CLAUDE_SKILL_DIR}/references/formulas.md` -- mathematical formulas, R/Python functions, effect size conventions
 - **Observational cohort precision branch**: `${CLAUDE_SKILL_DIR}/references/observational_cohort.md`
+- **Prediction-model / medical-AI sample size (Riley)**: `${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md` -- the current TRIPOD+AI-aligned standard for a clinical prediction/classification model (development via `pmsampsize`, external validation via `pmvalsampsize`, net-benefit precision). Use this instead of EPV-10 whenever the goal is risk prediction for use rather than a single-predictor hypothesis test (Tests 12-13).
 - **Justification prose exemplars**: `${CLAUDE_SKILL_DIR}/references/justification_examples.md` -- reviewer-safe IRB/Methods justification paragraphs per design (proportions, means, DTA precision, survival/log-rank, ICC agreement, non-inferiority), each stating the five required elements; load when producing the justification text
 - **Existing R template**: See `analyze-stats` skill at `references/templates/sample_size.R` for the 7 original tests
 
@@ -49,7 +50,10 @@ What is your primary outcome?
 |   |   +-- NO  --> How many groups?
 |   |       +-- 2 groups, superiority     --> [4] Two-proportion comparison (chi-square)
 |   |       +-- 2 groups, non-inferiority --> [10] Non-inferiority / equivalence
-|   |       +-- Multivariable model       --> [9] Logistic regression
+|   |       +-- Multivariable model       --> single-predictor hypothesis test? --> [9] Logistic regression
+|   |                                     --> clinical prediction / AI model for use?
+|   |                                         +-- developing the model  --> [12] Prediction-model development (Riley)
+|   |                                         +-- externally validating  --> [13] External-validation (Riley)
 |   |
 +-- Continuous (measurement, score)
 |   |
@@ -242,7 +246,10 @@ What is your primary outcome?
 **Approach A: Peduzzi Rule of Thumb (EPV >= 10)**
 - N_events = 10 * n_predictors
 - N_total = N_events / event_rate
-- Simple, widely cited, conservative. Use as a minimum baseline.
+- Simple, widely cited, conservative. Use as a minimum baseline **for a single-predictor
+  hypothesis test only**. For a **clinical prediction / medical-AI model intended for use**,
+  EPV-10 is outdated and reviewer-vulnerable — use the Riley criteria in Test 12
+  (development) / Test 13 (validation) instead.
 
 **Approach B: Hsieh (1989) Formula**
 - Uses the OR of interest for the primary predictor to calculate a more precise sample size.
@@ -320,6 +327,44 @@ N_adj = N_total / (1 - attrition_rate)
 
 ---
 
+### Test 12: Prediction-Model Development (Riley)
+
+**When to use**: developing a clinical prediction / classification model (including a
+medical-AI model evaluated as one) — the goal is risk prediction *for use*, not a single
+predictor's hypothesis test. EPV-10 (Tests 9/11) is outdated here.
+
+**Approach**: the minimum N is the largest satisfying all four Riley criteria
+simultaneously — global shrinkage ≥ 0.9, apparent–adjusted R² gap ≤ 0.05, precise overall
+risk estimate, and (time-to-event) precise baseline survival. Implemented in R `pmsampsize`.
+
+**Required parameters**: number of **candidate predictor parameters** (count dummy/non-linear
+terms), a conservative expected **C-statistic or Cox-Snell R²** (with its literature source),
+and outcome **prevalence** (binary) or **event rate + mean follow-up** (time-to-event).
+
+Read `${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md` for the criteria, the
+`pmsampsize` code, and the reporting requirements. Report N + required events + the binding
+criterion + the assumed C/R² and its source.
+
+---
+
+### Test 13: External-Validation Sample Size (Riley)
+
+**When to use**: sizing an **external validation** of an existing prediction/AI model.
+
+**Approach**: size to estimate the key validation metrics *precisely enough to be
+conclusive* — target the CI width of the **C-statistic**, the **calibration slope**, the
+**calibration-in-the-large / O:E ratio**, and (if a utility claim) **net benefit**.
+Implemented in R `pmvalsampsize`. A floor of ≥ 100 events and ≥ 100 non-events applies, but
+the precise target is usually larger.
+
+**Required parameters**: expected **prevalence**, anticipated **C-statistic**, and the
+target CI widths.
+
+Read `${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md` for the `pmvalsampsize`
+code. Report the targeted CI widths and the resulting events / non-events.
+
+---
+
 ## Scope Limitations
 
 ### Supported

package/skills/calc-sample-size/references/prediction_model_sample_size.md

@@ -0,0 +1,97 @@
+# Prediction-model sample size (Riley) — development & external validation
+
+For a **clinical prediction model** (or a medical-AI model evaluated as one), the
+events-per-variable rule of thumb (EPV ≥ 10) is **outdated and reviewer-vulnerable**. EPV
+does not target the things that determine whether a developed model is trustworthy
+(overfitting, precise risk estimates) or whether a validation is conclusive (precise
+discrimination and calibration). Use the **Riley criteria** instead — they are the current
+TRIPOD+AI-aligned standard, and they are implemented in the R packages `pmsampsize`
+(development) and `pmvalsampsize` (external validation).
+
+Apply this whenever the goal is **risk prediction / classification for use**, not a single
+predictor's hypothesis test. (For a single-predictor association test, the Hsieh/logistic
+power calculation in Test 9 is appropriate.)
+
+## Development sample size — `pmsampsize` (Riley et al.)
+
+The minimum development sample size is the largest N satisfying four criteria
+simultaneously:
+
+1. **Small overfitting** — global shrinkage factor ≥ 0.9.
+2. **Small optimism in apparent fit** — absolute difference between apparent and adjusted
+   Nagelkerke R² ≤ 0.05.
+3. **Precise estimate of overall risk** — margin of error of the mean predicted outcome
+   (the intercept) ≤ 0.05.
+4. **(time-to-event)** precise estimate of the baseline survival at a key time point.
+
+Inputs you must specify (and justify from prior literature, not guess):
+- number of **candidate predictor parameters** (count dummy variables and non-linear terms,
+  not just variables);
+- the anticipated model performance — a conservative **Cox-Snell R²** (derivable from an
+  expected **C-statistic** + outcome prevalence) or expected R²;
+- the **outcome prevalence** (binary) or **event rate + mean follow-up** (time-to-event),
+  or outcome mean/SD (continuous).
+
+```r
+library(pmsampsize)
+# Binary outcome: prevalence 0.20, 30 candidate parameters, expected C ≈ 0.78
+# (convert C to a conservative Cox-Snell R²; pmsampsize accepts csrsquared or cstatistic)
+pmsampsize(type = "b", cstatistic = 0.78, parameters = 30, prevalence = 0.20)
+
+# Time-to-event: rate per person-year, mean follow-up, timepoint of interest
+pmsampsize(type = "s", csrsquared = 0.05, parameters = 30,
+           rate = 0.08, timepoint = 5, meanfup = 4.2)
+```
+
+Report the **minimum N and the required number of events**, the assumed C-statistic / R²
+and its source, and the binding criterion. A development set below this is a standard
+high-impact-journal rejection.
+
+## External-validation sample size — `pmvalsampsize` (Riley et al.)
+
+Size the validation set to estimate the key performance metrics **precisely enough to be
+conclusive** — target the confidence-interval width of each:
+
+- **discrimination** — C-statistic (target CI width, e.g. ±0.05);
+- **calibration slope** (target SE / CI width);
+- **calibration-in-the-large / O:E ratio**;
+- **(if reporting clinical utility)** the standardized **net benefit** at the decision
+  threshold.
+
+```r
+library(pmvalsampsize)
+# Validate a model with expected prevalence 0.20, anticipated C ≈ 0.78,
+# targeting a C-statistic CI half-width of 0.05 and a precise calibration slope
+pmvalsampsize(type = "b", prevalence = 0.20, cstatistic = 0.78,
+              cslope = 1, oe = 1, lpnormal = c(-0.5, 0.5),
+              cstatci = 0.10)   # target CI *width* for the C-statistic
+```
+
+A common **floor** is ≥ 100 events and ≥ 100 non-events (Vergouwe; Collins), but
+`pmvalsampsize` gives the precise target for your metrics and usually requires more.
+
+## Sample size for net benefit / decision-curve
+
+If the model's value claim rests on **net benefit** (decision-curve analysis), size the
+study to estimate net benefit at the relevant threshold with adequate precision —
+`pmvalsampsize` supports a net-benefit target, and the development criteria should be met
+so the model is not overfit before its utility is judged. Do not claim clinical utility
+from a decision curve estimated on an underpowered validation set.
+
+## Reporting
+
+- State that sizing followed the **Riley criteria** (cite the originals) — not EPV-10.
+- Report the assumed C-statistic / R² / prevalence and **where they came from**.
+- For development, report N + events + the binding criterion; for validation, report the
+  targeted CI widths and the resulting N (events and non-events).
+
+## References (cite the originals)
+
+- Riley RD, Snell KIE, Ensor J, et al. Minimum sample size for developing a multivariable
+  prediction model: PART II — binary and time-to-event outcomes. *Stat Med* 2019.
+- Riley RD, Ensor J, Snell KIE, et al. Calculating the sample size required for developing
+  a clinical prediction model. *BMJ* 2020.
+- Riley RD, Debray TPA, Collins GS, et al. Minimum sample size for external validation of a
+  clinical prediction model with a binary outcome. *Stat Med* 2021.
+- Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external
+  validation of a multivariable prognostic model. *Stat Med* 2016.

package/skills/check-reporting/scripts/check_framework_naming.py

@@ -65,6 +65,11 @@ VAGUE = re.compile(
     r"\b(?:recent|current|emerging|latest|evolving)\b\s+"
     r"(?:best[- ]practice|guidance|practice|recommendations?|standards?|guidelines?)",
     re.I)
+# VAGUE only counts inside a reporting-framework context; otherwise "recent
+# best-practice recommendations" about a method is a false positive (F05).
+REPORTING_CUE = re.compile(
+    r"\b(?:report(?:ing|ed)?|checklist|EQUATOR|reporting\s+standard|"
+    r"reporting\s+framework)\b", re.I)
 
 
 def _sentences(text: str) -> list[str]:
@@ -128,15 +133,20 @@ def check(text: str) -> list[dict]:
             "where": m,
         })
 
-    # VAGUE_GUIDANCE
-    for m in VAGUE.finditer(text):
-        claims.append({
-            "verdict": "VAGUE_GUIDANCE",
-            "severity": "Minor",
-            "detail": (f"vague wording '{m.group(0).strip()}' — name the specific framework "
-                       f"and cite it"),
-            "where": m.group(0).strip()[:80],
-        })
+    # VAGUE_GUIDANCE — only when the sentence is clearly about REPORTING (a reporting
+    # guideline / checklist) yet names no specific framework. Gating on a reporting cue
+    # prevents firing on method-level wording like "external validation following recent
+    # best-practice recommendations", which is not a reporting-framework claim at all.
+    for s in _sentences(text):
+        m = VAGUE.search(s)
+        if m and REPORTING_CUE.search(s):
+            claims.append({
+                "verdict": "VAGUE_GUIDANCE",
+                "severity": "Minor",
+                "detail": (f"vague wording '{m.group(0).strip()}' in a reporting context — name "
+                           f"the specific framework and cite it"),
+                "where": m.group(0).strip()[:80],
+            })
 
     return claims
 

package/skills/check-reporting/tests/test_framework_naming.sh

@@ -41,5 +41,21 @@ check "SELF_COINED_LABEL detected (12-AI)" has_verdict SELF_COINED_LABEL
 python3 "$SCRIPT" --manuscript "$CLEAN" --strict --quiet >/dev/null 2>&1
 check "exit 0 on clean manuscript (base named + cited)" test "$?" -eq 0
 
+# (3) FP guard (less-defensive, F05): method-level "recent best-practice recommendations"
+# with NO reporting context must NOT trigger VAGUE_GUIDANCE.
+NOFP="$(mktemp -t fw_nofp_XXXX).md"
+trap 'rm -f "$OUT" "$NOFP"' EXIT
+cat > "$NOFP" <<'EOF'
+# Methods
+We performed external validation following recent best-practice recommendations.
+The imputation strategy was aligned with current guidance for handling missingness.
+EOF
+python3 "$SCRIPT" --manuscript "$NOFP" --out "$OUT" --quiet >/dev/null 2>&1
+check "no VAGUE_GUIDANCE on method-level recommendations (no reporting cue)" python3 -c "
+import json
+d=json.load(open('$OUT'))
+raise SystemExit(0 if not any(c['verdict']=='VAGUE_GUIDANCE' for c in d['claims']) else 1)
+"
+
 echo "fail=$fail"; [[ "$fail" -eq 0 ]] && echo "ALL PASS" || echo "FAILURES: $fail"
 exit "$fail"

package/skills/design-study/SKILL.md

@@ -275,7 +275,8 @@ Recommend one primary guideline:
 - healthy volunteer bias unacknowledged → inflated external validity claims
 - surveillance bias from unequal follow-up frequency between groups
 - **3 bias classification (Hernan/Robins)**: selection bias (who enters), information bias (how measured), confounding (what else differs) — explicitly map each threat
-- **confounding completeness**: pre-specify the adjustment set from a DAG (not a Table-1 p < 0.05 rule), and plan to report whether any measured covariate that turns out imbalanced by exposure but outside the adjustment set leaves the primary estimate robust (an extended-adjustment sensitivity model). At review time `/self-review` Phase 2.5e + the O1–O12 probes in `observational_confounding.md` check this against Table 1 (including O7 over-adjustment, O10 overlapping-subset-gradient discipline, for complex-survey data O11 design-based weighting and O12 data-driven-threshold mining, O13 — a cross-sectional mediation claim cannot order X→M→Y, and O14 — a synergy/joint-effect claim needs the additive interaction scale (RERI/AP/S), not a multiplicative-only test).
+- **comparative / causal question → emulate a target trial.** For a treatment-vs-treatment, screening-vs-no-screening, or drug-A-vs-drug-B question on routinely-collected data, specify the seven target-trial components (eligibility, strategies, assignment, **time zero**, outcome, causal contrast, analysis plan) before extraction — this is what prevents the immortal-time / prevalent-user / confounding-by-indication trio above and turns an association into a defensible causal contrast. New-user + active-comparator design, grace-period clone-censor-weight, and negative controls are in `references/target_trial_emulation.md`.
+- **confounding completeness**: pre-specify the adjustment set from a DAG (not a Table-1 p < 0.05 rule), and plan to report whether any measured covariate that turns out imbalanced by exposure but outside the adjustment set leaves the primary estimate robust (an extended-adjustment sensitivity model). Build the DAG and pre-screen the proposed covariates with `scripts/adjustment_set_helper.py` (flags mediator / collider / descendant adjustment and omitted confounders, and proposes a candidate backdoor set), then derive the **minimal** sufficient set with dagitty — see `references/dag_adjustment.md`. At review time `/self-review` Phase 2.5e + the O1–O12 probes in `observational_confounding.md` check this against Table 1 (including O7 over-adjustment, O10 overlapping-subset-gradient discipline, for complex-survey data O11 design-based weighting and O12 data-driven-threshold mining, O13 — a cross-sectional mediation claim cannot order X→M→Y, and O14 — a synergy/joint-effect claim needs the additive interaction scale (RERI/AP/S), not a multiplicative-only test).
 
 ### Multimodal LLM / report generation
 - no clear rubric for clinical correctness