medsci-skills 5.11.0 → 5.12.0

package/metadata/distribution_files.json

@@ -153,8 +153,13 @@
     },
     {
       "path": "skills/analyze-stats/SKILL.md",
-      "size": 55302,
-      "sha256": "3809b6a55e8a79b7dc48f617e3be0a7faf2a96a6f0ba03d58977a39f267cce81"
+      "size": 55642,
+      "sha256": "5c4a966490cfb8da6e09ebda897599479bd527d7ee3957ccbfceb5abff06306a"
+    },
+    {
+      "path": "skills/analyze-stats/references/analysis_guides/agreement_reliability.md",
+      "size": 6674,
+      "sha256": "599ad2551f547043b2bec79e3139d9c2c3ae33c5025784359b22022f882504c2"
     },
     {
       "path": "skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md",
@@ -2923,8 +2928,8 @@
     },
     {
       "path": "skills/peer-review/SKILL.md",
-      "size": 68302,
-      "sha256": "bf56fbc16bb3e43fb43f09982d36e68f53f8b0f666564e2828e692472ad0660f"
+      "size": 69384,
+      "sha256": "94033b84b41304d7dbd7919c84c7e9502bdaff1b7e2ef8f5dd7ec9cd7147c532"
     },
     {
       "path": "skills/peer-review/references/aczel_2021_reviewer2_patterns.md",
@@ -2993,8 +2998,8 @@
     },
     {
       "path": "skills/peer-review/references/domain-probes/observational_confounding.md",
-      "size": 34617,
-      "sha256": "ec4a11caed6d604e510437827f2b979fcf43ab1ddea6b298eb95673472b2b459"
+      "size": 34795,
+      "sha256": "ff8cb910c7fb83ec08a04ba88866bdf1e0d2f43d972e2861b825595a85b035ee"
     },
     {
       "path": "skills/peer-review/references/domain-probes/polygenic_risk_score.md",
@@ -3488,8 +3493,8 @@
     },
     {
       "path": "skills/self-review/SKILL.md",
-      "size": 104028,
-      "sha256": "22bc51c9adcb1fc9ff9120c2650faaa2f5f3a0482fe1fd781e231e2191e21081"
+      "size": 104731,
+      "sha256": "9c74f317d10c7b491e710debb619dcf536689092a59fe749050a0ad41f4aacd5"
     },
     {
       "path": "skills/self-review/references/domain-probes/ai_overclaiming.md",
@@ -3553,8 +3558,8 @@
     },
     {
       "path": "skills/self-review/references/domain-probes/observational_confounding.md",
-      "size": 34617,
-      "sha256": "ec4a11caed6d604e510437827f2b979fcf43ab1ddea6b298eb95673472b2b459"
+      "size": 34795,
+      "sha256": "ff8cb910c7fb83ec08a04ba88866bdf1e0d2f43d972e2861b825595a85b035ee"
     },
     {
       "path": "skills/self-review/references/domain-probes/polygenic_risk_score.md",
@@ -3668,8 +3673,8 @@
     },
     {
       "path": "skills/self-review/scripts/check_claim_artifact.py",
-      "size": 15879,
-      "sha256": "1daa64eaabe493f05ae7e2129a2056b673ec588487d5b6c73f41f10104822935"
+      "size": 18475,
+      "sha256": "c99c8090205734b0eae981550ed09e6275d94add608e66df78f4533c35f20924"
     },
     {
       "path": "skills/self-review/scripts/check_classical_style.py",
@@ -3686,6 +3691,11 @@
       "size": 21506,
       "sha256": "7d3e67074d58a28ffee52ce64b486231f103a3ddcaf6b3b6ee83ba5f89c63bc2"
     },
+    {
+      "path": "skills/self-review/scripts/check_cv_leakage.py",
+      "size": 6349,
+      "sha256": "431c5f14ad2c4f59e85b2a5e86150ac708d476e8f2f1da859d81953f522b2117"
+    },
     {
       "path": "skills/self-review/scripts/check_editorial_impression.py",
       "size": 22032,
@@ -3743,8 +3753,8 @@
     },
     {
       "path": "skills/self-review/skill.yml",
-      "size": 2345,
-      "sha256": "91883d75f33ee873a6ebfbd4d5f2cf0c292d3b90fe4705d6a35e82a26ecf6523"
+      "size": 2381,
+      "sha256": "9499a87d985d981eac9418d6cd95ed79c1d9724aba8854b7d6713d89679d9f1d"
     },
     {
       "path": "skills/setup-medsci/SKILL.md",
@@ -3898,8 +3908,8 @@
     },
     {
       "path": "skills/write-paper/SKILL.md",
-      "size": 66426,
-      "sha256": "910808b73103a74c26a3a92acb9e7b99529de1b8d9af8efaf740e7ad737fdd09"
+      "size": 66771,
+      "sha256": "1a50c9ceb040a79feaa60e809c7196f9f04b85cab2f5f3402dcaed3f7aee552c"
     },
     {
       "path": "skills/write-paper/references/exemplar_abstract.md",
@@ -3918,8 +3928,8 @@
     },
     {
       "path": "skills/write-paper/references/exemplar_discussion/README.md",
-      "size": 2524,
-      "sha256": "46d1b9e874f669608cc2c2abc400ab5efa74e37958f59e0c3b8f842dd4934e7d"
+      "size": 2727,
+      "sha256": "3bfcd4eb4df4ca0309446a606550cb11adc4d88daa676f2217249691d2e6b042"
     },
     {
       "path": "skills/write-paper/references/exemplar_discussion/ai_validation_tripod_claim.md",
@@ -3931,6 +3941,11 @@
       "size": 2500,
       "sha256": "7d49d926eb9aa78c42ce0e6579ca299f528dec77dafc17ed9451ada0c2340b3c"
     },
+    {
+      "path": "skills/write-paper/references/exemplar_discussion/meta_analysis_prisma.md",
+      "size": 3357,
+      "sha256": "3cc8542c754372b19acbc25b3c6d6211479b4bbec361c7638c5d2550db3ecc5f"
+    },
     {
       "path": "skills/write-paper/references/exemplar_discussion/observational_cohort_strobe.md",
       "size": 3652,
@@ -3943,8 +3958,8 @@
     },
     {
       "path": "skills/write-paper/references/exemplar_methods/README.md",
-      "size": 2247,
-      "sha256": "158c4894b92c1b6c19bda8becd947e455c5050cbef4d7d3606c2c46c3123d006"
+      "size": 2342,
+      "sha256": "6966e8b723d0c4ccc012d54e2b8a22ef59b39c22619e63a8597f9badb380b4e7"
     },
     {
       "path": "skills/write-paper/references/exemplar_methods/ai_validation_tripod_claim.md",
@@ -3956,6 +3971,11 @@
       "size": 2739,
       "sha256": "f403613a0f3625c5c7dab8fa16a21635007821ede2b58d265440db51b55165a6"
     },
+    {
+      "path": "skills/write-paper/references/exemplar_methods/meta_analysis_prisma.md",
+      "size": 4160,
+      "sha256": "a0e696e2f581fa6ae8b1aeb86ff7b9a5aaa6bdb2184fee4352e74d2ee6559d90"
+    },
     {
       "path": "skills/write-paper/references/exemplar_methods/observational_cohort_strobe.md",
       "size": 2364,
@@ -3963,8 +3983,8 @@
     },
     {
       "path": "skills/write-paper/references/exemplar_results/README.md",
-      "size": 2588,
-      "sha256": "0ececbb937fa44a2d89d56124bc7d5d2fea540a459fd0328b0cf534c463baaa1"
+      "size": 2799,
+      "sha256": "14b588f1f4433be71c39f11627148fff797eb87b3b74be75e61514095d4c38ea"
     },
     {
       "path": "skills/write-paper/references/exemplar_results/ai_validation_tripod_claim.md",
@@ -3976,6 +3996,11 @@
       "size": 2772,
       "sha256": "8e45e0b30b4ab505b4eb8c37007f970bfb5aa81375fd011915b07d85c9419598"
     },
+    {
+      "path": "skills/write-paper/references/exemplar_results/meta_analysis_prisma.md",
+      "size": 3170,
+      "sha256": "8aed3ee5d36c15515843d77b5f69fe76f433c9972827a9e18303b38802fac782"
+    },
     {
       "path": "skills/write-paper/references/exemplar_results/observational_cohort_strobe.md",
       "size": 2452,
@@ -4288,8 +4313,8 @@
     },
     {
       "path": "skills/write-paper/references/paper_types/meta_analysis.md",
-      "size": 10163,
-      "sha256": "5384224227f54deb1829a400ec6994b45a1b7cb34c167c158804e9d361466b26"
+      "size": 10536,
+      "sha256": "2c237409005214baab63627966d8f2b31498e3c28bea34c571a6d5591a068ed8"
     },
     {
       "path": "skills/write-paper/references/paper_types/nhis_cohort.md",

package/metadata/distribution_manifest.json

@@ -1,6 +1,6 @@
 {
   "schema_version": 1,
-  "version": "5.11.0",
+  "version": "5.12.0",
   "owned_skills": [
     "academic-aio",
     "add-journal",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "medsci-skills",
-  "version": "5.11.0",
+  "version": "5.12.0",
   "description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
   "license": "SEE LICENSE IN LICENSE",
   "homepage": "https://github.com/Aperivue/medsci-skills#readme",

package/skills/analyze-stats/SKILL.md

@@ -411,6 +411,7 @@ tbl %>% as_flex_table() %>% flextable::save_as_docx(path = "table.docx")
 
 ### Inter-rater Agreement
 
+- **Methodology guide**: `references/analysis_guides/agreement_reliability.md` (**load before generating code** — the pseudoreplication trap for clustered/repeated measurements + the pseudoreplication-safe per-subject / mixed-effects code, ICC model/type selection, agreement-vs-reliability distinction; pairs with self-review probe O18)
 - Table type guide: `references/table-standards/table-types/agreement.md` (ICC with model/type + CI, weighted κ for ordinal, Bland–Altman bias + LoA, reliability-vs-agreement distinction, common errors)
 - Template: `references/templates/agreement_analysis.py`
 - 2 raters + categorical: Cohen's kappa

package/skills/analyze-stats/references/analysis_guides/agreement_reliability.md

@@ -0,0 +1,130 @@
+# Inter-rater Agreement & Reliability Guide
+
+Quantifying how well two or more raters (or a rater and a reference, or repeated
+measurements) **agree**. The coefficient is easy to compute; the two ways these
+analyses fail review are (1) treating **clustered** measurements as independent
+(pseudoreplication) and (2) confusing **agreement** with **reliability**.
+
+---
+
+## When to Use
+
+- **Cohen's kappa** — 2 raters, categorical (nominal) labels.
+- **Weighted kappa** — 2 raters, **ordinal** labels (linear or quadratic weights; disagreement
+  by one category counts less than by three).
+- **Fleiss' kappa** — ≥3 raters, categorical.
+- **Krippendorff's alpha** — any number of raters, any measurement level, tolerates missing data.
+- **ICC (intraclass correlation)** — **continuous** measurements; report the model + type (below).
+- **Bland–Altman** — two continuous methods/raters: bias (mean difference) + 95% limits of agreement.
+- NOT for: a single 2×2 vs a reference standard (that is diagnostic accuracy — see
+  `table-types/diagnostic_accuracy.md`); not for a multi-reader AI-vs-human comparison with reader +
+  case variance (that is an MRMC reader study — see `table-types/reader_study.md`).
+
+---
+
+## Pseudoreplication comes first (this, not the coefficient, is the issue)
+
+If each **subject contributes more than one measurement** — several lesions, aneurysms, nodules,
+slices, or time-points per patient — the rows are **not independent**. Computing agreement on the
+**pooled** rows (or on all pairwise distances) uses an inflated *n*, narrows the CI, and gives an
+**anti-conservative** p-value. This is the single most common reliability-study error a reviewer
+catches (it is flagged by the self-review probe **O18** in `observational_confounding.md`).
+
+Two correct paths — pick one and state it:
+
+1. **Aggregate to the independent unit first**, then compute agreement per subject. This is the
+   simplest defensible analysis when a per-subject summary is meaningful (e.g. mean measurement,
+   majority label, or one index lesion per subject).
+2. **Model the clustering** — a mixed-effects / variance-components ICC with a **subject random
+   effect** (or a GEE with an exchangeable working correlation), so the within-subject correlation
+   is estimated rather than ignored.
+
+A pooled-pairwise test can *flip* on correction: e.g. Mann–Whitney p = 0.02 on 448 pooled
+pairwise distances became p = 0.59 at the per-aneurysm level (n = 112). **Report the unit of
+analysis explicitly**, and when subjects have multiple measurements report a per-subject
+sensitivity analysis.
+
+### Produce the pseudoreplication-safe version
+
+```python
+import pandas as pd
+import pingouin as pg   # ICC with model/type + CI
+
+# long format: one row per (subject, measurement); rater columns rater1..raterK
+df = pd.read_csv("ratings.csv")
+
+# 1) DETECT clustering: more rows than independent subjects
+n_rows, n_subjects = len(df), df["subject_id"].nunique()
+if n_rows > n_subjects:
+    print(f"CLUSTERED: {n_rows} measurements from {n_subjects} subjects "
+          f"({n_rows / n_subjects:.1f} per subject) — do NOT pool as independent.")
+
+# 2a) PER-SUBJECT AGGREGATION (continuous): mean per subject, then ICC on subject means
+per_subj = df.groupby("subject_id")[["rater1", "rater2"]].mean().reset_index()
+long = per_subj.melt(id_vars="subject_id", var_name="rater", value_name="score")
+icc = pg.intraclass_corr(data=long, targets="subject_id", raters="rater", nan_policy="omit")
+print(icc[["Type", "ICC", "CI95%"]])   # report Type (e.g. ICC2/ICC2k) + CI
+
+# 2b) OR MODEL THE CLUSTERING (keep every measurement, subject random effect)
+import statsmodels.formula.api as smf
+df_long = df.melt(id_vars="subject_id", value_vars=["rater1", "rater2"],
+                  var_name="rater", value_name="score")
+m = smf.mixedlm("score ~ 1", data=df_long, groups=df_long["subject_id"])
+res = m.fit()
+var_between = float(res.cov_re.iloc[0, 0]); var_resid = float(res.scale)
+icc_clustered = var_between / (var_between + var_resid)
+print(f"variance-components ICC (subject random effect) = {icc_clustered:.3f}")
+```
+
+---
+
+## ICC: state the model and the type (they are not interchangeable)
+
+- **Model**: one-way random (raters differ per subject), two-way random (same raters, generalise to
+  a rater population), two-way mixed (same raters, these raters only).
+- **Type**: **agreement** vs **consistency** (agreement penalises systematic rater bias; consistency
+  does not), and **single** vs **average** measurement (average-of-k is higher — only report it if
+  the clinical use averages k raters).
+- Report as e.g. **ICC(2,1) = 0.82 (95% CI 0.74–0.88), two-way random, absolute agreement, single
+  rater**. An ICC with no model/type is not interpretable.
+
+---
+
+## Agreement is not reliability
+
+- **Agreement** = do raters give the *same value* (absolute; Bland–Altman bias, absolute-agreement ICC).
+- **Reliability** = can raters *rank/discriminate subjects* consistently (relative; consistency ICC,
+  Pearson/Spearman). A method can be highly reliable yet have poor agreement (a constant offset).
+  State which one the clinical claim needs, and use the matching coefficient.
+
+---
+
+## Reporting
+
+- The coefficient **with a 95% CI** (bootstrap or analytic), the model/type (for ICC), and the
+  **unit of analysis** (per-subject vs per-lesion, and the clustering handling).
+- The interpretation band used (e.g. Landis–Koch), but do not over-interpret a point estimate whose
+  CI spans two bands.
+- For continuous methods: Bland–Altman **bias + 95% limits of agreement**, not just a correlation.
+
+---
+
+## Common failures (flag at review)
+
+- **Pooled/pairwise agreement on clustered data** (pseudoreplication) — the headline coefficient's
+  CI is too narrow; re-run per-subject or with a subject random effect (probe O18).
+- **ICC reported with no model/type** — uninterpretable; the same data yields different ICCs.
+- **Reliability coefficient used to claim agreement** (or vice versa) — a high consistency ICC does
+  not establish that the two methods are interchangeable.
+- **Correlation (r) reported as agreement** for two methods — r ignores a constant/proportional bias;
+  Bland–Altman is required.
+- **Kappa on ordinal labels unweighted** — treats a one-category disagreement as a full disagreement.
+
+---
+
+## Anti-Hallucination
+
+- Never hand-type a coefficient or CI — compute it from the ratings CSV with a seeded script.
+- Do not quote an ICC without the model/type actually estimated by the code.
+- If subjects have multiple measurements, the per-subject sensitivity analysis is **mandatory** —
+  do not report only the pooled number.

package/skills/peer-review/SKILL.md

@@ -118,6 +118,17 @@ within the current submission (poolability of incommensurable studies, a broken
 evaluation instrument). When both classes are present, the **unfixable** class governs the recommendation —
 do not let a long list of fixable items reframe an unfixable core as "addressable in revision."
 
+**Salvage-reframe that shrinks the contribution is NOT a fixable major revision.** When your proposed fix
+for a construct/validity flaw is to *narrow the claim* (e.g. "reframe from a clinical classifier to a
+re-identifiability signal", "scope down to a proof-of-concept"), check whether that narrower framing survives
+the novelty/importance bar. If novelty/importance is ALREADY weak — a co-reviewer or your own scorecard flags
+the work as "expected / well-known finding / unconvincing motivation / limited use case" (Originality or
+Reader-interest ≤ mid) — then the reframe *reduces* the contribution and makes the importance problem worse,
+not better. A contribution shrunk to survive a validity flaw is a **Reject-leaning** outcome (the contribution
+is the product, not addressable-in-revision), not an encourage-major-revision. Deterministic trigger to
+self-audit: if your confidential note says the claim is "narrower than / more modest than claimed" AND your
+recommendation is Reject-family-adjacent, do not upgrade it to major revision on the strength of the reframe.
+
 **Review/narrative/primer escalation** *(the contribution IS the product)*: for a review article there is no
 data to re-analyze; the distinct contribution — novelty, integrative synthesis, domain-specificity — is the
 deliverable itself. Therefore **weak novelty / no distinct contribution / not domain-specific is

package/skills/peer-review/references/domain-probes/observational_confounding.md

@@ -113,6 +113,7 @@ A 17-probe checklist for observational studies (cohort, case-control, cross-sect
 - When an agreement or reader study computes a test (Mann–Whitney, t-test, correlation) on **pooled pairwise distances / reader-pairs** rather than on **independent units** (subjects, lesions, aneurysms), the effective n is inflated (each subject contributes several pairwise rows) and the p-value is anti-conservative. This is the reader/agreement sibling of the analysis-unit / clustering issue in O8: the number of *observations* is not the number of *independent units*.
 - Lead: when a reported test **n exceeds the number of independent subjects/lesions** (e.g. n=448 or 672 pairwise from 112 aneurysms) and no clustering / mixed-effects / per-subject aggregation is stated → flag. Ask for the analysis re-run at the **per-subject** level (or a mixed model with a subject random effect); a pooled-pairwise p can flip (e.g. p=0.02 pooled → p=0.59 per-aneurysm).
 - Severity: MAJOR when a headline agreement/superiority claim rests on the pooled-pairwise test; MINOR when a per-subject sensitivity reproduces it. Applies to all multi-rater agreement / MRMC reader studies.
+- Produce the fix: `analyze-stats` `references/analysis_guides/agreement_reliability.md` has the pseudoreplication-safe per-subject aggregation + subject-random-effect ICC code.
 - **(e) Effect size and resolution honesty at scale.** With the very large N these scans run on, trivially small effects clear any threshold; report **effect magnitudes** and clinical relevance alongside significance, and respect the **resolution floor** (a permutation procedure with k permutations cannot resolve p below ≈ 1/k; FDR has a minimum detectable q at a given hit count). A "number of significant exposures" headline with no effect sizes overstates the finding.
 - Severity: MAJOR when a headline causal/actionable claim rests on uncorrected, single-cohort-only, or univariate-in-a-correlated-block top hits; MINOR when correction + replication + full results are present and the claim is framed as screening. Cross-link O11 (complex-survey design — an NHANES/KNHANES ExWAS must combine design-based standard errors with the multiplicity correction, not one or the other), O12 (the single-exposure threshold/non-linearity analogue), and O2/O7 (confounding / over-adjustment for whichever hit survives). Report the tested-set size, the correction method, and the replication design explicitly.
 

package/skills/self-review/SKILL.md

@@ -362,6 +362,15 @@ These modules carry the same domain-specific critique probes used by `/peer-revi
 | Scoping review (maps the breadth/nature of evidence, clarifies concepts, identifies gaps; PCC framing, charting, optional appraisal — not a focused effectiveness/accuracy question) | `references/domain-probes/scoping_review.md` (SC1–SC8) |
 | Qualitative study (interviews, focus groups, ethnography, grounded theory, phenomenology, document analysis; reflexivity, trustworthiness, thematic analysis — not quantitative validity) | `references/domain-probes/qualitative_research.md` (QL1–QL8) |
 
+For a **classifier / NLP / tabular ML** manuscript, also run the deterministic feature-selection-leakage gate — a data-driven selection (feature selection, log-odds / univariate filtering, vocabulary construction, a threshold) fit on the FULL dataset before cross-validation inflates the CV metric:
+
+```bash
+python3 "${CLAUDE_SKILL_DIR}/scripts/check_cv_leakage.py" \
+  --manuscript manuscript.md --out qc/cv_leakage.json
+```
+
+`CV_SELECTION_LEAKAGE` (Major) fires when a selection token co-occurs with cross-validation and no fold-nesting is disclosed ("within each fold" / "nested CV" suppresses it). This is distinct from patient-vs-image split leakage (`model-validation/check_split_leakage.py`).
+
 When the manuscript matches a row, read `${CLAUDE_SKILL_DIR}/references/domain-probes/<module>.md` and apply each probe as an additional source of Anticipated Major / Minor Comments. The module severity words (MAJOR / MINOR) map to this skill's framing as follows: a conclusion-threatening or design-level finding becomes a **Fatal** Anticipated Major Comment, a reporting-level finding becomes a **Fixable** Anticipated Minor Comment, and each is tagged with the closest category letter (A–K). These probes **complement** categories A–K above; they do not replace them. (The modules are vendored byte-identical from `/peer-review`; do not edit one copy only — run `python3 scripts/check_domain_probe_sync.py --sync`.)
 
 ### Phase 2.5: Numerical Cross-Verification (Internal)

package/skills/self-review/references/domain-probes/observational_confounding.md

@@ -113,6 +113,7 @@ A 17-probe checklist for observational studies (cohort, case-control, cross-sect
 - When an agreement or reader study computes a test (Mann–Whitney, t-test, correlation) on **pooled pairwise distances / reader-pairs** rather than on **independent units** (subjects, lesions, aneurysms), the effective n is inflated (each subject contributes several pairwise rows) and the p-value is anti-conservative. This is the reader/agreement sibling of the analysis-unit / clustering issue in O8: the number of *observations* is not the number of *independent units*.
 - Lead: when a reported test **n exceeds the number of independent subjects/lesions** (e.g. n=448 or 672 pairwise from 112 aneurysms) and no clustering / mixed-effects / per-subject aggregation is stated → flag. Ask for the analysis re-run at the **per-subject** level (or a mixed model with a subject random effect); a pooled-pairwise p can flip (e.g. p=0.02 pooled → p=0.59 per-aneurysm).
 - Severity: MAJOR when a headline agreement/superiority claim rests on the pooled-pairwise test; MINOR when a per-subject sensitivity reproduces it. Applies to all multi-rater agreement / MRMC reader studies.
+- Produce the fix: `analyze-stats` `references/analysis_guides/agreement_reliability.md` has the pseudoreplication-safe per-subject aggregation + subject-random-effect ICC code.
 - **(e) Effect size and resolution honesty at scale.** With the very large N these scans run on, trivially small effects clear any threshold; report **effect magnitudes** and clinical relevance alongside significance, and respect the **resolution floor** (a permutation procedure with k permutations cannot resolve p below ≈ 1/k; FDR has a minimum detectable q at a given hit count). A "number of significant exposures" headline with no effect sizes overstates the finding.
 - Severity: MAJOR when a headline causal/actionable claim rests on uncorrected, single-cohort-only, or univariate-in-a-correlated-block top hits; MINOR when correction + replication + full results are present and the claim is framed as screening. Cross-link O11 (complex-survey design — an NHANES/KNHANES ExWAS must combine design-based standard errors with the multiplicity correction, not one or the other), O12 (the single-exposure threshold/non-linearity analogue), and O2/O7 (confounding / over-adjustment for whichever hit survives). Report the tested-set size, the correction method, and the replication design explicitly.
 

package/skills/self-review/scripts/check_claim_artifact.py

@@ -83,6 +83,16 @@ EVALUE_RE = re.compile(
 NONPRIMARY_KW = ("secondary", "exploratory", "subgroup", "sensitivity", "supporting",
                  "cause-specific", "cancer-specific", "post-hoc", "post hoc", "non-primary")
 
+# The manuscript asserts exactly ONE primary model/analysis (so a script annotating a
+# model as "co-primary" is a third-SSOT drift).
+SINGLE_PRIMARY = re.compile(
+    r"\bsingle\s+primary\b|\ba\s+single\s+primary\b|\bone\s+primary\s+(?:model|analysis|endpoint|outcome)\b"
+    r"|\bthe\s+primary\s+(?:model|analysis|endpoint|outcome)\b[^.]{0,70}?"
+    r"(?:consistent\s+with\s+the\s+(?:registered|pre-?specified)|registered\s+analysis\s+plan)",
+    re.I)
+# A model annotated "co-primary" in analysis code (a comment, string, or variable).
+CO_PRIMARY_CODE = re.compile(r"\bco[-\s]?primary\b", re.I)
+
 STOP = set("the a an of for in on to and or with by is was were are be been being this that "
            "between association associated estimated using model analysis primary outcome "
            "endpoint study patients group as at from".split())
@@ -271,6 +281,45 @@ def check_evalue(manuscript: str) -> list[dict]:
 # manual confirmation against the registration before either is acted on, and a P0
 # that needs hand-confirmation is not a P0. Only explicit re-designation and a
 # non-recomputing E-value are Major.
+def check_code_labels(manuscript: str, scripts_dir: str | None) -> list[dict]:
+    """Reconcile the manuscript's declared primary against analysis-script labels.
+
+    Fires only the specific conflict: the manuscript asserts a SINGLE primary while an
+    analysis script annotates a model as 'co-primary' — the code label is a third SSOT
+    that drifts across revisions. Advisory (code comments can lag)."""
+    claims: list[dict] = []
+    if not scripts_dir:
+        return claims
+    d = Path(scripts_dir)
+    if not d.exists() or not SINGLE_PRIMARY.search(manuscript):
+        return claims
+    for p in sorted(d.rglob("*")):
+        if p.suffix.lower() not in (".r", ".py"):
+            continue
+        try:
+            txt = p.read_text(encoding="utf-8", errors="replace")
+        except OSError:
+            continue
+        m = CO_PRIMARY_CODE.search(txt)
+        if not m:
+            continue
+        ln = txt[:m.start()].count("\n") + 1
+        snippet = txt.splitlines()[ln - 1].strip()[:80] if ln - 1 < len(txt.splitlines()) else ""
+        claims.append({
+            "claim_id": "EST-code-label",
+            "type": "estimand",
+            "prose_value": "manuscript asserts a single primary model/analysis",
+            "artifact_source": f"{p.name}:{ln} labels a model 'co-primary'",
+            "verdict": "PRIMARY_LABEL_CODE_DRIFT",
+            "detail": (f"the manuscript declares a SINGLE primary while an analysis script "
+                       f"annotates a model as co-primary ({p.name}:{ln}: '{snippet}'); reconcile "
+                       f"the code's primary/co-primary label with the declared estimand — code "
+                       f"labels are a third SSOT that can drift across revisions. ADVISORY."),
+        })
+        break  # one is enough to prompt a reconcile
+    return claims
+
+
 MAJOR = {"PRIMARY_REASSIGNED", "EVALUE_ARITHMETIC"}
 
 
@@ -278,6 +327,7 @@ def main() -> int:
     ap = argparse.ArgumentParser(description="Claim-vs-artifact cross-check (estimand + E-value).")
     ap.add_argument("--manuscript", required=True, help="manuscript markdown/text")
     ap.add_argument("--prereg", help="pre-registration / protocol / project.yaml text")
+    ap.add_argument("--scripts", help="analysis-scripts directory (reconcile code primary/co-primary labels)")
     ap.add_argument("--out", help="write JSON artifact to this path")
     ap.add_argument("--strict", action="store_true", help="exit 1 if any Major verdict")
     args = ap.parse_args()
@@ -302,7 +352,8 @@ def main() -> int:
         else:
             sys.stderr.write(f"WARN: prereg not found: {args.prereg} (estimand provenance limited)\n")
 
-    claims = check_estimand(manuscript, prereg, prereg_raw) + check_evalue(manuscript)
+    claims = (check_estimand(manuscript, prereg, prereg_raw) + check_evalue(manuscript)
+              + check_code_labels(manuscript, args.scripts))
     n_major = sum(1 for c in claims if c["verdict"] in MAJOR)
     n_flag = sum(1 for c in claims if c["verdict"] not in MAJOR and c["verdict"] != "OK")
 

package/skills/self-review/scripts/check_cv_leakage.py

@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+"""Feature-selection-outside-CV leakage gate (self-review Phase 2.5 / data-prep).
+
+For a classifier / NLP / tabular manuscript, if feature selection, vocabulary
+construction, log-odds / univariate filtering, or a threshold is chosen on the WHOLE
+dataset and only THEN cross-validation is run, the CV performance is optimistically
+inflated: the selection has already seen the held-out folds. The fix is to nest the
+selection inside each training fold (nested CV). This is a class a statistical
+reviewer catches deterministically, and it is distinct from patient-vs-image split
+leakage (`model-validation/check_split_leakage.py`).
+
+Verdict:
+  CV_SELECTION_LEAKAGE (Major)  a feature-selection / vocabulary / threshold step
+                                co-occurs with a cross-validation description AND no
+                                fold-nesting disclosure ("within each fold", "nested
+                                CV", "inside the training fold") is present. The
+                                headline CV metric is likely optimistic.
+
+Conservative by construction: fires only when BOTH a selection token AND a CV token
+appear AND no nesting-disclosure token is anywhere in the document. A single
+"within each training fold" / "nested cross-validation" sentence suppresses it.
+
+Exit codes: 0 clean/report-only, 1 with --strict when any Major, 2 usage. Stdlib-only.
+
+Usage:
+    python3 check_cv_leakage.py --manuscript manuscript.md \
+        [--out qc/cv_leakage.json] [--strict] [--quiet]
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from pathlib import Path
+
+# A data-driven selection / construction step that must be nested inside CV.
+SELECTION = re.compile(
+    r"\bfeature\s+selection\b|\bselected\s+(?:the\s+)?(?:top\s+)?\d+\s+(?:features|variables|predictors)"
+    r"|\blog[-\s]?odds\b|\bvocabulary\s+(?:construction|was\s+built|building)\b|\bbuilt\s+(?:a\s+)?vocabulary"
+    r"|\bfeature\s+ranking\b|\bunivariate\s+(?:filter|screening|selection)\b|\btop[-\s]?k\s+features"
+    r"|\bmutual[-\s]information\s+(?:selection|ranking)\b|\bchi[-\s]?square(?:d)?\s+selection"
+    r"|\b(?:selected|chose|retained|kept)\s+(?:the\s+)?(?:most\s+)?(?:informative|discriminative|predictive)\s+features"
+    r"|\bthreshold(?:ed|ing)?\b[^.\n]{0,40}\b(?:on|over|across|using)\s+the\s+(?:entire|full|whole|complete)\s+(?:data|dataset|cohort|corpus)"
+    r"|\b(?:LASSO|elastic[-\s]net|recursive\s+feature\s+elimination|RFE|Boruta)\b",
+    re.IGNORECASE)
+
+# A cross-validation evaluation.
+CV = re.compile(
+    r"\bcross[-\s]?validat(?:ion|ed)\b|\b\d+[-\s]?fold\b|\bk[-\s]?fold\b"
+    r"|\bleave[-\s]one[-\s]out\b|\bLOOCV\b|\bstratified\s+(?:\d+[-\s]?)?fold",
+    re.IGNORECASE)
+
+# Disclosure that the selection is correctly nested inside the CV training folds.
+NESTING = re.compile(
+    r"\bnested\s+(?:cross[-\s]?validat|CV)\b"
+    r"|\bwithin\s+each\s+(?:training\s+)?fold\b|\binside\s+(?:the\s+)?training\s+(?:fold|partition|split)"
+    r"|\bper[-\s]?fold\b|\bfold[-\s]specific\b|\bfor\s+each\s+(?:training\s+)?fold\b"
+    r"|\bon\s+the\s+training\s+(?:fold|partition|split|set)\s+only\b"
+    r"|\brefit(?:ted)?\s+within\b|\brepeated\s+(?:in|within|inside)\s+each\s+fold"
+    r"|\bselection\s+was\s+(?:performed|done|repeated)\s+(?:in|within|inside)\s+each\b",
+    re.IGNORECASE)
+
+
+def check(text: str) -> list[dict]:
+    sel = SELECTION.search(text)
+    cv = CV.search(text)
+    if not (sel and cv):
+        return []
+    if NESTING.search(text):
+        return []  # nesting is disclosed
+    return [{
+        "verdict": "CV_SELECTION_LEAKAGE",
+        "severity": "Major",
+        "detail": (f"a data-driven selection step ('{sel.group(0).strip()}') co-occurs with "
+                   f"cross-validation ('{cv.group(0).strip()}') but no fold-nesting is disclosed "
+                   f"(no 'within each fold' / 'nested CV'); if the selection was fit on the full "
+                   f"dataset the CV metric is optimistically inflated — nest it in each training fold"),
+        "where": text[max(0, sel.start() - 30):sel.end() + 50].replace("\n", " ").strip()[:170],
+    }]
+
+
+def analyze(manuscript: str) -> dict:
+    p = Path(manuscript)
+    if not p.is_file():
+        sys.stderr.write(f"ERROR: manuscript not found: {manuscript}\n")
+        sys.exit(2)
+    claims = check(p.read_text(encoding="utf-8"))
+    n_major = sum(1 for c in claims if c["severity"] == "Major")
+    return {
+        "manuscript": str(p),
+        "claims": claims,
+        "summary": {"n_claims": len(claims), "n_major": n_major, "n_flag": len(claims) - n_major,
+                    "verdict": "MAJOR_CANDIDATE" if n_major else "OK"},
+    }
+
+
+def render(result: dict) -> str:
+    lines = ["| Check | Severity | Detail |", "|---|---|---|"]
+    for c in result["claims"]:
+        lines.append(f"| {c['verdict']} | {c['severity']} | {c['detail']} |")
+    if len(lines) == 2:
+        lines.append("| (none) | — | no feature-selection-outside-CV leakage detected |")
+    return "\n".join(lines)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(
+        description="Feature-selection-outside-CV leakage gate (Phase 2.5 / data-prep).")
+    ap.add_argument("--manuscript", required=True, help="manuscript markdown/text")
+    ap.add_argument("--out", help="write JSON artifact to this path")
+    ap.add_argument("--strict", action="store_true", help="exit 1 if any Major claim exists")
+    ap.add_argument("--quiet", action="store_true", help="suppress stdout table")
+    args = ap.parse_args()
+
+    result = analyze(args.manuscript)
+
+    if not args.quiet:
+        print("=" * 44)
+        print(" CV selection-leakage (§2.5 / data-prep)")
+        print("=" * 44)
+        print(render(result))
+        print()
+        if result["summary"]["n_major"]:
+            print("MAJOR candidate: a selection step co-occurs with CV and no fold-nesting is disclosed.")
+        else:
+            print("OK: no feature-selection-outside-CV leakage detected.")
+
+    if args.out:
+        Path(args.out).parent.mkdir(parents=True, exist_ok=True)
+        Path(args.out).write_text(json.dumps(result, indent=2), encoding="utf-8")
+        if not args.quiet:
+            print(f"\nwrote {args.out}")
+
+    return 1 if (args.strict and result["summary"]["n_major"]) else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/skills/self-review/skill.yml

@@ -49,5 +49,6 @@ validation_commands:
   - "bash tests/test_rounded_delta.sh"
   - "bash tests/test_figure_citation.sh"
   - "bash tests/test_emphasis_density.sh"
+  - "bash tests/test_cv_leakage.sh"
   - "feed R0-numbered output into /revise"
 evidence_surface: demo

package/skills/self-review/tests/fixtures/claim_manuscript_single_primary.md

@@ -0,0 +1,3 @@
+## Methods
+Model 3 is the single primary model, consistent with the registered analysis plan.
+The E-value for the primary association (HR 1.34) was 2.02.

package/skills/self-review/tests/fixtures/claim_scripts_consistent/05_primary_cox.R

@@ -0,0 +1,3 @@
+# 05_primary_cox.R
+model3 <- coxph(Surv(t, e) ~ x + covars)   # the single primary model
+model5 <- coxph(Surv(t, e) ~ x + covars2)  # secondary sensitivity analysis

package/skills/self-review/tests/fixtures/claim_scripts_coprimary/05_primary_cox.R

@@ -0,0 +1,3 @@
+# 05_primary_cox.R
+model3 <- coxph(Surv(t, e) ~ x + covars)   # PRIMARY model
+model4 <- coxph(Surv(t, e) ~ x2 + covars)  # CO-PRIMARY model (added at revision)

package/skills/self-review/tests/fixtures/cv_leakage_bad.md

@@ -0,0 +1,2 @@
+## Methods
+We selected the 70 most informative binary features by log-odds ratio computed over the entire dataset, then evaluated the classifier with stratified 5-fold cross-validation.

package/skills/self-review/tests/fixtures/cv_leakage_clean.md

@@ -0,0 +1,2 @@
+## Methods
+Feature selection was repeated within each training fold (nested cross-validation), so no information from the held-out fold informed the selection.

package/skills/self-review/tests/test_claim_artifact.sh

@@ -86,5 +86,20 @@ raise SystemExit(0 if not any(c['verdict']=='ESTIMAND_DRIFT' for c in d['claims'
 python3 "$SCRIPT" --manuscript "$SMAN" --prereg "$SPRE" --strict >/dev/null 2>&1
 check "exit 0 on structured consistent prereg (no Major)" test "$?" -eq 0
 
+# Code-label reconciliation (--scripts): a manuscript asserting a SINGLE primary vs an
+# analysis script annotating a model 'co-primary' -> PRIMARY_LABEL_CODE_DRIFT (advisory,
+# not Major). A consistent scripts dir and the no-flag backward-compatible default.
+SP="$HERE/fixtures/claim_manuscript_single_primary.md"
+python3 "$SCRIPT" --manuscript "$SP" --scripts "$HERE/fixtures/claim_scripts_coprimary" --out "$OUT" >/dev/null 2>&1
+check "PRIMARY_LABEL_CODE_DRIFT on code co-primary vs single-primary manuscript" has_verdict PRIMARY_LABEL_CODE_DRIFT
+python3 "$SCRIPT" --manuscript "$SP" --scripts "$HERE/fixtures/claim_scripts_coprimary" --strict >/dev/null 2>&1
+check "code-label drift is advisory (exit 0 under --strict)" test "$?" -eq 0
+python3 "$SCRIPT" --manuscript "$SP" --scripts "$HERE/fixtures/claim_scripts_consistent" --out "$OUT" >/dev/null 2>&1
+check "no drift when scripts carry no co-primary label" python3 -c "
+import json
+d=json.load(open('$OUT'))
+raise SystemExit(0 if not any(c['verdict']=='PRIMARY_LABEL_CODE_DRIFT' for c in d['claims']) else 1)
+"
+
 echo "fail=$fail"; [[ "$fail" -eq 0 ]] && echo "ALL PASS" || echo "FAILURES: $fail"
 exit "$fail"

package/skills/self-review/tests/test_cv_leakage.sh

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+# Regression test for the feature-selection-outside-CV leakage gate.
+# (bad) log-odds feature selection on the full dataset + 5-fold CV, no nesting ->
+# CV_SELECTION_LEAKAGE; (clean) selection repeated within each training fold (nested
+# CV) -> no flag.
+set -u
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SCRIPT="$HERE/../scripts/check_cv_leakage.py"
+BAD="$HERE/fixtures/cv_leakage_bad.md"
+CLEAN="$HERE/fixtures/cv_leakage_clean.md"
+OUT="$(mktemp -t cvl_XXXX).json"
+trap 'rm -f "$OUT"' EXIT
+fail=0
+check() { local label="$1"; shift
+  if "$@" >/dev/null 2>&1; then printf '  PASS  %s\n' "$label"
+  else printf '  FAIL  %s\n' "$label"; fail=$((fail+1)); fi; }
+[[ -f "$SCRIPT" ]] || { echo "ENV-ERR: script missing" >&2; exit 2; }
+
+python3 "$SCRIPT" --manuscript "$BAD" --out "$OUT" --strict --quiet >/dev/null 2>&1
+check "exit 1 under --strict (Major present)" test "$?" -eq 1
+check "CV_SELECTION_LEAKAGE on selection+CV without nesting" python3 -c "
+import json
+d=json.load(open('$OUT'))
+assert any(c['verdict']=='CV_SELECTION_LEAKAGE' for c in d['claims']), 'not flagged'
+"
+python3 "$SCRIPT" --manuscript "$CLEAN" --strict --quiet >/dev/null 2>&1
+check "exit 0 when selection is nested within each fold" test "$?" -eq 0
+python3 "$SCRIPT" --manuscript "$CLEAN" --out "$OUT" --quiet >/dev/null 2>&1
+check "no leakage flag on nested CV" python3 -c "
+import json
+d=json.load(open('$OUT'))
+assert not d['claims'], d['claims']
+"
+echo "fail=$fail"; [[ "$fail" -eq 0 ]] && echo "ALL PASS" || echo "FAILURES: $fail"
+exit "$fail"

package/skills/write-paper/SKILL.md

@@ -221,7 +221,7 @@ Design all tables and figures BEFORE writing prose. This ensures the narrative s
 
 Write the Methods section first -- it is the most objective and anchors the rest of the paper.
 
-**Before writing:** Load `${CLAUDE_SKILL_DIR}/references/section_guides/methods.md` for PICO structure, backbone article usage, checklist cross-reference, and terminology conventions. For the matching study type, also skim the structure model in `${CLAUDE_SKILL_DIR}/references/exemplar_methods/` (diagnostic-accuracy/STARD, AI-validation/TRIPOD+AI·CLAIM, observational-cohort/STROBE) — it lists, paragraph by paragraph, what each Methods paragraph must establish plus the element that type most often omits. Model the structure; the exemplars are synthetic, with placeholder specifics, not prose to copy.
+**Before writing:** Load `${CLAUDE_SKILL_DIR}/references/section_guides/methods.md` for PICO structure, backbone article usage, checklist cross-reference, and terminology conventions. For the matching study type, also skim the structure model in `${CLAUDE_SKILL_DIR}/references/exemplar_methods/` (diagnostic-accuracy/STARD, AI-validation/TRIPOD+AI·CLAIM, observational-cohort/STROBE, meta-analysis/PRISMA 2020) — it lists, paragraph by paragraph, what each Methods paragraph must establish plus the element that type most often omits. Model the structure; the exemplars are synthetic, with placeholder specifics, not prose to copy.
 
 **Writing order within Methods:**
 1. Study Design and Setting
@@ -255,7 +255,7 @@ Write the Methods section first -- it is the most objective and anchors the rest
 Write Results aligned to the approved tables and figures. **Results = "What did we find?"
 — nothing more.** Every sentence must be a factual statement backed by a number.
 
-**Before writing:** Load `${CLAUDE_SKILL_DIR}/references/section_guides/results.md` for mirror-symmetry rules, flowchart requirements, missing data handling, and the anti-interpretation self-check. For the matching study type, also skim the structure model in `${CLAUDE_SKILL_DIR}/references/exemplar_results/` (diagnostic-accuracy/STARD, AI-validation/TRIPOD+AI·CLAIM, observational-cohort/STROBE) — each follows its `exemplar_methods/` sibling in Methods order, listing what each Results paragraph must establish (flow → baseline/prevalence → primary estimate with CIs → calibration/agreement → subgroups → sensitivity) plus the element that type most often omits. Model the structure; the exemplars are synthetic, with placeholder specifics, not prose to copy.
+**Before writing:** Load `${CLAUDE_SKILL_DIR}/references/section_guides/results.md` for mirror-symmetry rules, flowchart requirements, missing data handling, and the anti-interpretation self-check. For the matching study type, also skim the structure model in `${CLAUDE_SKILL_DIR}/references/exemplar_results/` (diagnostic-accuracy/STARD, AI-validation/TRIPOD+AI·CLAIM, observational-cohort/STROBE, meta-analysis/PRISMA 2020) — each follows its `exemplar_methods/` sibling in Methods order, listing what each Results paragraph must establish (flow → baseline/prevalence → primary estimate with CIs → calibration/agreement → subgroups → sensitivity; for meta-analysis, PRISMA flow → characteristics+provenance → RoB → pooled estimate with I²/τ²/prediction interval → subgroup interaction → publication bias) plus the element that type most often omits. Model the structure; the exemplars are synthetic, with placeholder specifics, not prose to copy.
 
 **Rules:**
 - Every number in the text must match the corresponding table cell exactly.
@@ -296,7 +296,7 @@ Write Results aligned to the approved tables and figures. **Results = "What did
 
 ### Phase 5: Discussion
 
-**Before writing:** Load `${CLAUDE_SKILL_DIR}/references/section_guides/discussion.md` for the 4-paragraph structure, word limits, limitation writing guidelines, and Table/Figure citation rules. For the matching study type, also skim the structure model in `${CLAUDE_SKILL_DIR}/references/exemplar_discussion/` (diagnostic-accuracy/STARD, AI-validation/TRIPOD+AI·CLAIM, observational-cohort/STROBE) — completing the exemplar trio, each lists what every Discussion paragraph must establish (key finding → interpretation/comparison → limitations → generalizability → conclusion matched to the evidence) plus the element that type most often omits (spectrum/verification bias; evidence-tier separation and optimism caveats; mandatory causal caution). For case reports, use `${CLAUDE_SKILL_DIR}/references/exemplar_case_report.md` instead: it controls literature-boundary wording, n=1 causal caution, and bedside teaching-point framing. Model the structure; the exemplars are synthetic, introduce no new results, and are not prose to copy.
+**Before writing:** Load `${CLAUDE_SKILL_DIR}/references/section_guides/discussion.md` for the 4-paragraph structure, word limits, limitation writing guidelines, and Table/Figure citation rules. For the matching study type, also skim the structure model in `${CLAUDE_SKILL_DIR}/references/exemplar_discussion/` (diagnostic-accuracy/STARD, AI-validation/TRIPOD+AI·CLAIM, observational-cohort/STROBE, meta-analysis/PRISMA 2020) — completing the exemplar trio, each lists what every Discussion paragraph must establish (key finding → interpretation/comparison → limitations → generalizability → conclusion matched to the evidence) plus the element that type most often omits (spectrum/verification bias; evidence-tier separation and optimism caveats; mandatory causal caution; for meta-analysis, GRADE certainty + heterogeneity source + non-independence/overlap caveat). For case reports, use `${CLAUDE_SKILL_DIR}/references/exemplar_case_report.md` instead: it controls literature-boundary wording, n=1 causal caution, and bedside teaching-point framing. Model the structure; the exemplars are synthetic, introduce no new results, and are not prose to copy.
 
 **Before drafting, collect user input (Discussion Planning Gate).**
 

package/skills/write-paper/references/exemplar_discussion/README.md

@@ -27,6 +27,9 @@ closes with a conclusion matched to the evidence — introducing no new results.
   CLAIM).
 - `observational_cohort_strobe.md` — mandatory causal caution, reverse-causation and
   unmeasured-confounding limitations, no care-directive conclusion (STROBE).
+- `meta_analysis_prisma.md` — summary-of-evidence with certainty framing, heterogeneity-source
+  and non-independence/overlap caveats, GRADE limitations, no guideline-grade conclusion
+  (PRISMA 2020).
 
 ## Curator guidelines (for adding more)
 

package/skills/write-paper/references/exemplar_discussion/meta_analysis_prisma.md

@@ -0,0 +1,52 @@
+# Discussion structure — systematic review / meta-analysis (PRISMA 2020)
+
+A structure model for the Discussion of a systematic review with quantitative synthesis,
+completing the trio with the `exemplar_methods/` and `exemplar_results/` siblings. Each heading
+is a paragraph; each bullet is *what it must establish*. Fill the `[brackets]`; do not copy this
+text. Follows `section_guides/discussion.md`. Introduces **no new results** and cites **no
+tables/figures**.
+
+## Paragraph 1 — Summary of evidence
+- Restate the primary pooled estimate (effect + 95% CI, k studies, N patients) in 2–3
+  sentences, matched to the Results — introduce no pooled number not already reported.
+- One sentence on what it means clinically, framed to the **certainty of evidence**, not as
+  proof.
+- **If the pooled estimate is null or the CI is wide, frame it by precision, not absence** —
+  state what the CI and the prediction interval *exclude*; an imprecise pool with few studies is
+  "not yet established," not "no effect."
+
+## Paragraphs 2–3 — Interpretation, comparison, and heterogeneity
+- Compare to the 2–3 most recent related syntheses (agree / differ and why: eligibility,
+  populations, measurement, analysis).
+- **Explain heterogeneity**, distinguishing **clinical** (population, technique, threshold) from
+  **statistical** (I²/τ²), and tie it to the subgroup / meta-regression findings — a high I²
+  narrated only as a number, without a source, is a reviewer target.
+- Note whether small-study effects / publication bias plausibly move the estimate.
+
+## Limitations
+- **Study-level risk of bias** carried into the pooled estimate (name the driving domain); the
+  **certainty of evidence** downgrades (GRADE: risk of bias, inconsistency, indirectness,
+  imprecision, publication bias).
+- **Review-level limits**: overlapping populations / shared datasets or benchmarks
+  (non-independence) and how a leave-one-dataset-out check addressed it; few studies for
+  subgroups; language/publication restrictions; a low-powered or absent publication-bias test.
+- If reviewer count, search currency, or a synthesis-method choice deviated from protocol, state
+  it plainly here — consistent with what Methods declared.
+
+## Generalizability
+- The populations and settings the pooled estimate represents and where it may not transfer;
+  how the prediction interval bounds — not the CI — describe the plausible effect in a new study.
+
+## Conclusion
+- Matched to the evidence and its certainty — "the pooled evidence suggests", "moderate-certainty
+  evidence for", or a call for adequately powered / prospective studies; never a care directive or
+  a guideline-grade recommendation the evidence tier does not license (guideline recommendations
+  are the panel's, not a single review's — and disclose any guideline-co-authorship intellectual
+  COI).
+
+## Common omission
+- An explicit **certainty-of-evidence (GRADE) treatment plus a heterogeneity source and a
+  non-independence/overlap caveat** — the Discussion elements MA drafts most often soften, and the
+  ones a synthesis reviewer checks first. Cross-reference `section_guides/discussion.md`, the
+  `peer-review/references/domain-probes/sr_ma.md` probes (P2 non-independence, P4 k=1), and the
+  PRISMA critical items in `peer-review/references/reviewer_calibration/compliance_floor.md`.

package/skills/write-paper/references/exemplar_methods/README.md

@@ -23,6 +23,7 @@ fit). A missing element is a gap to fill before drafting Results.
 - `diagnostic_accuracy_stard.md` — an index-test vs reference-standard accuracy study (STARD).
 - `ai_validation_tripod_claim.md` — an AI/ML model development + validation study (TRIPOD+AI / CLAIM).
 - `observational_cohort_strobe.md` — an exposure→outcome observational cohort (STROBE).
+- `meta_analysis_prisma.md` — a systematic review with quantitative synthesis (PRISMA 2020).
 
 ## Curator guidelines (for adding more)
 

package/skills/write-paper/references/exemplar_methods/meta_analysis_prisma.md

@@ -0,0 +1,65 @@
+# Methods structure — systematic review / meta-analysis (PRISMA 2020)
+
+A structure model for a systematic review with quantitative synthesis. Each heading is a
+paragraph; each bullet is *what it must establish*. Fill the `[brackets]`; do not copy this
+text. Pairs with `paper_types/meta_analysis.md` (which carries the fuller prose templates) —
+this file is the paragraph-order skeleton and the reviewer-critical checklist.
+
+## Protocol and registration
+- Prospective registration (PROSPERO `CRD[number]`) or an OSF/protocol reference, **lodged
+  before data extraction began** — state the registration date relative to the search.
+- Adherence to PRISMA 2020 (name the base guideline *and* the extension used, e.g. PRISMA
+  base + **PRISMA-DTA** for diagnostic accuracy, PRISMA-NMA for network — never just the
+  extension acronym).
+- Any protocol amendments as dated entries (what changed, when, why), not a silent deviation.
+
+## Eligibility criteria
+- PICO(S) as a numbered inclusion list and a **separate** explicit exclusion list (not "did
+  not meet inclusion") — Population, Intervention/Index test, Comparator, Outcome, Study
+  design, and any language/date limits.
+- The effect measure the review is built around (OR / RR / HR / MD / SMD / sensitivity &
+  specificity / AUC) named here, so eligibility and synthesis agree.
+
+## Information sources and search strategy
+- Every database with its coverage dates and the last-search date; grey literature and
+  register searches (ClinicalTrials.gov, conference abstracts) if used.
+- The **full search string for ≥1 database, verbatim** (in-text or as a supplement) — a
+  PRISMA-critical, first-line reviewer item; a paraphrased strategy is not reproducible.
+
+## Study selection and data extraction
+- Dual independent screening (title/abstract then full text) with the agreement metric
+  (Cohen's κ or % agreement) at each stage and how disagreements were resolved; the operational
+  reviewer count here must match what Limitations later concedes (no "dual" here / "single
+  reviewer" there).
+- Standardized dual extraction; the variables extracted; how the reference-standard / outcome
+  cells (e.g. 2×2 TP/FP/FN/TN) were obtained and reconciled against the source (cell-level
+  audit for hand-entered accuracy data).
+
+## Risk of bias / quality assessment
+- The tool matched to the design (**QUADAS-2** for DTA, **RoB 2** for RCTs, ROBINS-I / NOS /
+  MINORS for non-randomized), applied in duplicate; how domain judgements were reached.
+- Whether risk of bias fed a sensitivity analysis (low-RoB-only pool), not just a figure.
+
+## Statistical synthesis
+- The model and estimand: **random-effects a priori** (state the estimator, e.g.
+  DerSimonian–Laird or REML; bivariate / HSROC for DTA) with the pooled effect and **95% CIs**.
+- Heterogeneity: I² **with τ² and a 95% prediction interval** (I² is a proportion, not a
+  magnitude — the PI shows the range of true effects); Cochran's Q threshold.
+- **Pre-specified** subgroup / meta-regression covariates (post-hoc ones labeled exploratory);
+  sensitivity analyses (leave-one-out, prospective-only, low-RoB-only); overlapping-population /
+  shared-benchmark non-independence check.
+- Publication bias (Egger / Deeks funnel asymmetry) **only when ≥10 studies**; certainty of
+  evidence (**GRADE**) if claimed. Software + version + seed.
+
+## Reporting-guideline fit
+- PRISMA 2020 (+ the relevant extension). Critical items: verbatim search strategy for ≥1
+  database, a flow diagram that reconciles, per-study risk of bias, and the protocol/registration
+  reference (see `peer-review/references/reviewer_calibration/compliance_floor.md`).
+
+## Common omission
+- A **pre-specified subgroup set + an overlapping-population / shared-dataset non-independence
+  check**, and naming **base + extension** guideline correctly — the Methods elements MA drafts
+  most often skip, and among the first a synthesis reviewer checks. Cross-reference
+  `section_guides/methods.md`, `paper_types/meta_analysis.md`, and the
+  `peer-review/references/domain-probes/sr_ma.md` probes (P1 comparator existence, P2
+  non-independence, P4 k=1 subgroup).

package/skills/write-paper/references/exemplar_results/README.md

@@ -27,6 +27,9 @@ element that interprets rather than reports belongs in the Discussion.
   calibration, operating point, clinical utility (TRIPOD+AI / CLAIM).
 - `observational_cohort_strobe.md` — assembly, Table 1 by exposure, crude-then-adjusted
   estimate with CIs, subgroups, sensitivity analyses (STROBE).
+- `meta_analysis_prisma.md` — PRISMA flow, study/patient characteristics + provenance, pooled
+  estimate with I²/τ²/prediction interval, subgroup interaction, sensitivity, publication bias
+  (PRISMA 2020).
 
 ## Curator guidelines (for adding more)
 

package/skills/write-paper/references/exemplar_results/meta_analysis_prisma.md

@@ -0,0 +1,51 @@
+# Results structure — systematic review / meta-analysis (PRISMA 2020)
+
+A structure model for the Results of a systematic review with quantitative synthesis. It
+follows its Methods PRISMA sibling in Methods order. Each heading is a paragraph; each bullet
+is *what it must establish*. Fill the `[brackets]`; do not copy this text. Report findings
+only — no interpretation.
+
+## Study selection (Figure 1 — PRISMA flow)
+- The identification→screening→eligibility→inclusion cascade as the PRISMA 2020 flow diagram:
+  records identified, duplicates removed, screened, full texts assessed, and excluded **with
+  reasons and counts**.
+- The cascade reconciles (identified − duplicates − excluded = included); state N studies in the
+  review vs N in the meta-analysis (they can differ). These numbers match the Abstract, Methods,
+  and Table 1 exactly (one locked count, re-derived — not re-typed per document).
+
+## Study and patient characteristics (Table 1)
+- Publication-year span, design mix (prospective / retrospective; RCT / observational), total
+  and per-study N (median, IQR/range), countries, and the effect-relevant descriptors.
+- A **data-provenance column** when studies may share an institution / public database / imaging
+  benchmark — so overlapping-population non-independence is visible, not hidden.
+
+## Risk of bias across studies (Figure 2)
+- The RoB summary (QUADAS-2 / RoB 2 domain plot): how many studies were low risk overall and
+  the domain that drove concern, with counts (N/N).
+
+## Quantitative synthesis — primary outcome (Figure 3, forest plot)
+- The pooled effect with **95% CI**, the study count **k** and pooled **N**, and heterogeneity
+  reported as **I² with τ² and the 95% prediction interval** (state what the PI's bounds imply
+  for a new setting).
+- The forest plot cited; per-study weights visible. For DTA, the bivariate/HSROC summary point
+  (pooled sensitivity & specificity with CIs) — check no sensitivity/specificity swap versus a
+  source.
+
+## Subgroups, meta-regression, and sensitivity
+- Each **pre-specified** subgroup with its estimate + CI **and the test for subgroup interaction**
+  (report the interaction p, not two bare stratum estimates); flag k=1 strata as descriptive, not
+  pooled.
+- Leave-one-out, prospective-only, and low-RoB-only sensitivity results, reported as results — not
+  deferred to a supplement mention only.
+
+## Publication bias / small-study effects
+- Egger / Deeks asymmetry **only if ≥10 studies** (state the count gate); if trim-and-fill was
+  run, the adjusted estimate. Do not over-read a non-significant test — its power is low; say so.
+
+## Common omission
+- The **prediction interval alongside I²**, a **subgroup interaction test** (rather than two
+  separate stratum estimates), and the **overlapping-population/provenance** disclosure — the
+  Results elements MA drafts most often skip. Cross-reference `section_guides/results.md`, the
+  `peer-review/references/domain-probes/sr_ma.md` probes (P1, P2 non-independence, P3 subset-N
+  transparency, P4 k=1), and the PRISMA critical items in
+  `peer-review/references/reviewer_calibration/compliance_floor.md`.

package/skills/write-paper/references/paper_types/meta_analysis.md

@@ -8,6 +8,11 @@
 - **Typical word count:** 4500–6000 words
 - **Structure:** Abstract → Introduction → Methods → Results → Discussion → Conclusions
 
+**Worked structure models** (paragraph order + what each paragraph must establish, synthetic):
+`exemplar_methods/meta_analysis_prisma.md`, `exemplar_results/meta_analysis_prisma.md`,
+`exemplar_discussion/meta_analysis_prisma.md`. This file carries the fuller prose templates;
+the exemplars are the compact skeletons for confirming every load-bearing paragraph is present.
+
 ---
 
 ## Required at Phase 0 (Init)