medsci-skills 5.12.0 → 5.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -153,14 +153,19 @@
153
153
  },
154
154
  {
155
155
  "path": "skills/analyze-stats/SKILL.md",
156
- "size": 55642,
157
- "sha256": "5c4a966490cfb8da6e09ebda897599479bd527d7ee3957ccbfceb5abff06306a"
156
+ "size": 56422,
157
+ "sha256": "ddb73598fe60601942e8b44a87771e3ba583a585304effac4ae206d53ab4f154"
158
158
  },
159
159
  {
160
160
  "path": "skills/analyze-stats/references/analysis_guides/agreement_reliability.md",
161
161
  "size": 6674,
162
162
  "sha256": "599ad2551f547043b2bec79e3139d9c2c3ae33c5025784359b22022f882504c2"
163
163
  },
164
+ {
165
+ "path": "skills/analyze-stats/references/analysis_guides/diagnostic_accuracy.md",
166
+ "size": 9303,
167
+ "sha256": "e788d612aa4227500e997ec405ab15a300d28c3b6fb2a1e1df33823764cb9537"
168
+ },
164
169
  {
165
170
  "path": "skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md",
166
171
  "size": 5362,
@@ -221,6 +226,11 @@
221
226
  "size": 12803,
222
227
  "sha256": "1639f4810fb5946b24e782b3ce463c000222dcac75709e1e3dac78ad4a2ad72f"
223
228
  },
229
+ {
230
+ "path": "skills/analyze-stats/references/analysis_guides/survival.md",
231
+ "size": 7191,
232
+ "sha256": "6b7f220f2aa165fd16eda58b0b40ac743c122c03757bc00f61efc884b203f337"
233
+ },
224
234
  {
225
235
  "path": "skills/analyze-stats/references/analysis_guides/test_selection.md",
226
236
  "size": 3439,
@@ -2953,8 +2963,8 @@
2953
2963
  },
2954
2964
  {
2955
2965
  "path": "skills/peer-review/references/domain-probes/diagnostic_accuracy.md",
2956
- "size": 11692,
2957
- "sha256": "afc69e56578babcc1cf7c7952d2df301fdc829329fb480a5cf426ff6ab65d9f5"
2966
+ "size": 11967,
2967
+ "sha256": "8c0a8bd30fe16507e2997ddba1bbcfb4ee7eade7df44f1981e1427cd3a28d233"
2958
2968
  },
2959
2969
  {
2960
2970
  "path": "skills/peer-review/references/domain-probes/equity_fairness.md",
@@ -3043,8 +3053,8 @@
3043
3053
  },
3044
3054
  {
3045
3055
  "path": "skills/peer-review/references/domain-probes/survival_prognostic.md",
3046
- "size": 13765,
3047
- "sha256": "af0e0323876424eec8ba07be22fe99b3100356e8e4895b5b79635e9a2d76e98e"
3056
+ "size": 14022,
3057
+ "sha256": "96195cd1b575535bc12a133125b9d85e2ca7e756b96a0649e03d4c3696e5312b"
3048
3058
  },
3049
3059
  {
3050
3060
  "path": "skills/peer-review/references/exemplar_reviews/README.md",
@@ -3513,8 +3523,8 @@
3513
3523
  },
3514
3524
  {
3515
3525
  "path": "skills/self-review/references/domain-probes/diagnostic_accuracy.md",
3516
- "size": 11692,
3517
- "sha256": "afc69e56578babcc1cf7c7952d2df301fdc829329fb480a5cf426ff6ab65d9f5"
3526
+ "size": 11967,
3527
+ "sha256": "8c0a8bd30fe16507e2997ddba1bbcfb4ee7eade7df44f1981e1427cd3a28d233"
3518
3528
  },
3519
3529
  {
3520
3530
  "path": "skills/self-review/references/domain-probes/equity_fairness.md",
@@ -3603,8 +3613,8 @@
3603
3613
  },
3604
3614
  {
3605
3615
  "path": "skills/self-review/references/domain-probes/survival_prognostic.md",
3606
- "size": 13765,
3607
- "sha256": "af0e0323876424eec8ba07be22fe99b3100356e8e4895b5b79635e9a2d76e98e"
3616
+ "size": 14022,
3617
+ "sha256": "96195cd1b575535bc12a133125b9d85e2ca7e756b96a0649e03d4c3696e5312b"
3608
3618
  },
3609
3619
  {
3610
3620
  "path": "skills/self-review/references/exemplar_findings/README.md",
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "schema_version": 1,
3
- "version": "5.12.0",
3
+ "version": "5.13.0",
4
4
  "owned_skills": [
5
5
  "academic-aio",
6
6
  "add-journal",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "medsci-skills",
3
- "version": "5.12.0",
3
+ "version": "5.13.0",
4
4
  "description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
5
5
  "license": "SEE LICENSE IN LICENSE",
6
6
  "homepage": "https://github.com/Aperivue/medsci-skills#readme",
@@ -393,6 +393,7 @@ tbl %>% as_flex_table() %>% flextable::save_as_docx(path = "table.docx")
393
393
 
394
394
  ### Diagnostic Accuracy
395
395
 
396
+ - **Methodology guide**: `references/analysis_guides/diagnostic_accuracy.md` (**load before generating code** — every metric with a CI on a stated analysis unit; the confidence-weighted trap [unweighted-baseline AUC + monotonic-encoding check, produce-side of probe D9]; paired DeLong vs MRMC for reader-generalising claims; per-stratum admissibility [D10]; one-scale-per-comparison [D11])
396
397
  - Template: `references/templates/diagnostic_accuracy.py`
397
398
  - Always report: sensitivity, specificity, PPV, NPV, accuracy, AUC
398
399
  - CIs: Wilson score for proportions, DeLong for AUC
@@ -487,6 +488,7 @@ tbl %>% as_flex_table() %>% flextable::save_as_docx(path = "table.docx")
487
488
 
488
489
  ### Survival Analysis
489
490
 
491
+ - **Methodology guide**: `references/analysis_guides/survival.md` (**load before generating code** — competing risks first [naive 1−KM overestimates → produce the Aalen–Johansen/Fine–Gray CIF; cause-specific vs subdistribution for which question, produce-side of probe S3]; PH check → RMST when violated; reverse-KM follow-up + C-index variant [S6]; estimand provenance [S8])
490
492
  - Table type guide: `references/table-standards/table-types/survival_results.md` (Cox results table: events/person-time, reverse-KM median follow-up, univariable + adjusted HR with CI, PH-assumption footnote, EPV/sparse-stratum and RMST-when-PH-violated rules)
491
493
  - Kaplan-Meier curves with number-at-risk table
492
494
  - Log-rank test for group comparison
@@ -0,0 +1,184 @@
1
+ # Diagnostic Accuracy & Reader-Study Guide
2
+
3
+ Estimating how well an index test (or an AI model / reader) separates disease from
4
+ no-disease against a reference standard. The point estimates are easy to compute; the
5
+ ways these analyses fail review are (1) reporting AUC / sensitivity / specificity **without
6
+ CIs or on the wrong analysis unit**, (2) a **confidence-weighted / rating** score whose
7
+ novelty is never tested against the simpler **unweighted** baseline, and (3) comparing two
8
+ AUCs with a **fixed-reader** test when the claim is meant to **generalise to readers**.
9
+
10
+ ---
11
+
12
+ ## When to Use
13
+
14
+ - **Single index test** vs a reference standard → sensitivity, specificity, PPV, NPV, accuracy
15
+ (each with a 95% CI), and AUC (with a DeLong or bootstrap CI).
16
+ - **Two tests / models on the same cases** → a **paired** AUC comparison (DeLong `roc.test`,
17
+ `paired = TRUE`) — never two independent CIs eyeballed for overlap.
18
+ - **Multi-reader multi-case (MRMC)** reader study (AI-vs-reader, modality comparison) → an
19
+ MRMC method (Obuchowski–Rockette / DBM) that carries **reader + case** variance; see
20
+ `table-standards/table-types/reader_study.md` and `make-figures` `exemplar_plots/mrmc_roc.md`.
21
+ - NOT for: pooling accuracy across studies (that is a DTA meta-analysis — bivariate / HSROC via
22
+ `mada`; see the `/meta-analysis` DTA path); not for rater **agreement** (that is
23
+ `analysis_guides/agreement_reliability.md`).
24
+
25
+ ---
26
+
27
+ ## Every metric with a CI, on a stated analysis unit (the floor)
28
+
29
+ A bare AUC / sensitivity / specificity is not reportable. Compute the CI, and state the unit
30
+ (**per-patient vs per-lesion** — multiple lesions per patient are clustered, exactly as in
31
+ `agreement_reliability.md`; a per-lesion count inflates *n*).
32
+
33
+ ```python
34
+ import numpy as np, pandas as pd
35
+ from sklearn.metrics import roc_auc_score
36
+ from statsmodels.stats.proportion import proportion_confint
37
+
38
+ df = pd.read_csv("reader_calls.csv") # truth (0/1), call (0/1), confidence (1..K), stratum
39
+
40
+ # sensitivity / specificity / PPV / NPV at the operating point, each with a Wilson CI
41
+ tp = int(((df.truth == 1) & (df.call == 1)).sum()); fn = int(((df.truth == 1) & (df.call == 0)).sum())
42
+ tn = int(((df.truth == 0) & (df.call == 0)).sum()); fp = int(((df.truth == 0) & (df.call == 1)).sum())
43
+ for name, num, den in [("sensitivity", tp, tp+fn), ("specificity", tn, tn+fp),
44
+ ("PPV", tp, tp+fp), ("NPV", tn, tn+fn)]:
45
+ lo, hi = proportion_confint(num, den, method="wilson")
46
+ print(f"{name} = {num/den:.3f} (95% CI {lo:.3f}-{hi:.3f}), n={den}")
47
+ ```
48
+
49
+ PPV and NPV are **prevalence-dependent** — do not transport them to a population with a
50
+ different base rate; report sensitivity/specificity (prevalence-invariant) plus the study
51
+ prevalence, and recompute predictive values at the target prevalence with Bayes' rule.
52
+
53
+ ---
54
+
55
+ ## The confidence-weighted trap comes first (this, not the AUC, is the issue)
56
+
57
+ When the novelty is a **confidence-weighted / rating-collapsed** score used as the ROC
58
+ predictor, you must (a) confirm the (call × confidence) encoding is **strictly monotone** — a
59
+ folded encoding silently collides the most-confident-positive with a negative call — and
60
+ (b) report the **unweighted binary-call AUC** beside the weighted one, so the weighting earns
61
+ its place. This is the produce-side of self-review / peer-review probe **D9**.
62
+
63
+ ```python
64
+ K = int(df.confidence.max())
65
+
66
+ # intended strictly-monotone composite: negative calls rank below positive calls, and within
67
+ # a call higher confidence pushes further from the decision boundary.
68
+ def composite(call, conf, K):
69
+ return np.where(call == 1, K + conf, K + 1 - conf) # neg: 1..K, pos: K+1..2K
70
+
71
+ # (1) MONOTONIC-ENCODING CHECK — every distinct (call, confidence) must map to a distinct,
72
+ # correctly-ordered score. A collision is the folded-score bug (e.g. real/1 == ai/1).
73
+ combos = df[["call", "confidence"]].drop_duplicates().copy()
74
+ combos["score"] = composite(combos.call.values, combos.confidence.values, K)
75
+ if combos.score.nunique() != len(combos):
76
+ raise ValueError("ENCODING COLLISION — the confidence weighting is not strictly monotone "
77
+ "(folded-score bug); fix the encoding before computing AUC.")
78
+
79
+ # (2) UNWEIGHTED BASELINE beside the weighted primary
80
+ df["score"] = composite(df.call.values, df.confidence.values, K)
81
+ print(f"AUC (confidence-weighted) = {roc_auc_score(df.truth, df.score):.3f}")
82
+ print(f"AUC (unweighted binary call) = {roc_auc_score(df.truth, df.call):.3f}")
83
+ ```
84
+
85
+ If the weighted AUC materially exceeds the unweighted one, the weighting must be justified;
86
+ if it does not, report the simpler estimator as primary. A weighted score that changed a
87
+ hypothesis's direction versus its unweighted baseline is a **post-lock change to disclose**,
88
+ not a silent primary.
89
+
90
+ ---
91
+
92
+ ## AUC confidence intervals and comparing two AUCs
93
+
94
+ DeLong is the field-standard analytic AUC CI and paired comparison; `pROC` (R) is canonical.
95
+
96
+ ```r
97
+ library(pROC)
98
+ d <- read.csv("reader_calls.csv")
99
+ rw <- roc(d$truth, d$score, quiet = TRUE)
100
+ ci.auc(rw) # DeLong 95% CI for the weighted AUC
101
+ ru <- roc(d$truth, d$call, quiet = TRUE)
102
+ roc.test(rw, ru, method = "delong", paired = TRUE) # paired, SAME cases (fixed-reader)
103
+ ```
104
+
105
+ A portable Python bootstrap CI (use when a DeLong implementation is unavailable):
106
+
107
+ ```python
108
+ def auc_ci_bootstrap(y, s, n_boot=2000, seed=42):
109
+ rng = np.random.default_rng(seed); y = np.asarray(y); s = np.asarray(s); n = len(y)
110
+ boots = [roc_auc_score(y[i], s[i]) for i in (rng.integers(0, n, n) for _ in range(n_boot))
111
+ if len(np.unique(y[i])) == 2]
112
+ return tuple(np.percentile(boots, [2.5, 97.5]))
113
+ ```
114
+
115
+ **Fixed-reader vs MRMC.** A paired DeLong test treats the readers/cases as fixed. If the claim
116
+ is that the result **generalises to readers** (a reader sample), a fixed-reader CI understates
117
+ uncertainty — use an MRMC method (Obuchowski–Rockette / DBM) that carries reader + case
118
+ variance, report per-reader and reader-averaged AUC, and state the unit (per-patient vs
119
+ per-lesion) and the superiority/non-inferiority margin. This is probe D9's MRMC lead.
120
+
121
+ ---
122
+
123
+ ## Per-stratum admissibility: produce the table the claim is checked against (D10)
124
+
125
+ A blanket "no subgroup met AUC ≥ 0.75 with lower bound ≥ 0.70" is falsified by a single tabled
126
+ stratum that meets it. Produce the per-stratum AUC + CI and test each row against the rule
127
+ rather than asserting a global negative.
128
+
129
+ ```python
130
+ rule = lambda auc, lo: (auc >= 0.75) and (lo >= 0.70)
131
+ for name, g in df.groupby("stratum"):
132
+ if g.truth.nunique() < 2: # single-class stratum: not estimable
133
+ print(f"{name}: not estimable (one class)"); continue
134
+ auc = roc_auc_score(g.truth, g.score); lo, hi = auc_ci_bootstrap(g.truth, g.score)
135
+ print(f"{name}: AUC {auc:.3f} (95% CI {lo:.3f}-{hi:.3f}) "
136
+ f"{'QUALIFIES' if rule(auc, lo) else 'below'}")
137
+ ```
138
+
139
+ With *k* strata some crossings are expected — frame qualifying strata as hypothesis-generating
140
+ (note the multiplicity), but do **not** deny a row the paper's own table satisfies.
141
+
142
+ ---
143
+
144
+ ## One scale per comparison (D11)
145
+
146
+ Two values sharing a comparison column (or a row-wise "A vs B") must be computed under the
147
+ **same normalisation / definition** — e.g. both volume errors as a standard relative error, not
148
+ one relative and one range-normalised. If they are not identical, recompute on a common scale
149
+ before any superiority/comparability claim, or footnote "not on the same scale".
150
+
151
+ ---
152
+
153
+ ## Reporting
154
+
155
+ - Sensitivity, specificity, PPV, NPV, accuracy, and AUC each **with a 95% CI**; the operating
156
+ point and how it was chosen (Youden vs a prespecified clinical threshold — a threshold picked
157
+ on the same data is optimistic and needs a held-out or cross-validated estimate).
158
+ - The **analysis unit** (per-patient vs per-lesion) and clustering handling; study prevalence.
159
+ - For a weighted-score primary: the **unweighted-baseline AUC** beside it (D9).
160
+ - AUC alone is insufficient for a clinical claim — pair discrimination with **calibration** and a
161
+ **decision-curve / net-benefit** pass at the relevant threshold (see the incremental-value
162
+ table type and the `make-figures` decision-curve exemplar).
163
+
164
+ ---
165
+
166
+ ## Common failures (flag at review)
167
+
168
+ - **AUC / sensitivity / specificity with no CI**, or a per-lesion count reported as if per-patient.
169
+ - **Confidence-weighted AUC with no unweighted baseline** and no monotonic-encoding check (D9) —
170
+ the weighting may have created the result or hidden a folded-score bug.
171
+ - **Two AUCs compared by CI overlap** instead of a paired DeLong test; a **fixed-reader** CI used
172
+ for a claim that generalises to readers (needs MRMC).
173
+ - **"No stratum met the rule" contradicted by the per-stratum table** (D10).
174
+ - **Mixed-normalisation head-to-head** in one column (D11).
175
+ - **PPV/NPV transported** across a prevalence change.
176
+
177
+ ---
178
+
179
+ ## Anti-Hallucination
180
+
181
+ - Never hand-type an AUC, sensitivity/specificity, or CI — compute it from the calls CSV with a
182
+ seeded script; carry each estimate together with its CI (never a bare AUC).
183
+ - Do not report a weighted-score AUC without the unweighted baseline the code produced.
184
+ - Do not quote a DeLong comparison p-value that a paired test on the same cases did not produce.
@@ -0,0 +1,149 @@
1
+ # Survival / Time-to-Event Guide
2
+
3
+ Estimating time-to-event outcomes and prognostic effects. The estimator is easy to
4
+ call; the ways these analyses fail review are (1) ignoring **competing risks** so a naive
5
+ 1−KM **overestimates** the cumulative incidence, (2) reporting a **single time-averaged
6
+ hazard ratio** when the proportional-hazards assumption is violated, and (3) **estimand
7
+ drift** — quoting a subdistribution hazard for an etiologic claim or a cause-specific hazard
8
+ for an absolute-risk claim. This guide produces the right estimand; the operational caveats
9
+ (EPV gate, cluster-robust CIs, interval-censoring) live in the SKILL.md `### Survival
10
+ Analysis` section.
11
+
12
+ ---
13
+
14
+ ## When to Use
15
+
16
+ - **Single event type, right-censored** → Kaplan–Meier (unadjusted) + Cox proportional-hazards
17
+ (adjusted HR with 95% CI); the log-rank test for a KM group comparison.
18
+ - **≥2 competing event types** (recurrence + competing death, or cause-specific mortality) →
19
+ cumulative incidence functions + **cause-specific Cox** or **Fine–Gray** — see below.
20
+ - **Prognostic model discrimination** → a C-index variant matched to the censoring + a
21
+ time-dependent AUC at a clinical horizon (S6).
22
+ - NOT for: events detected only at scheduled visits → interval-censored methods (SKILL.md);
23
+ recurrent events per subject → a cluster-robust or frailty model (SKILL.md cluster rule);
24
+ rater agreement → `analysis_guides/agreement_reliability.md`.
25
+
26
+ ---
27
+
28
+ ## Competing risks come first (this, not the HR, is the issue)
29
+
30
+ If a subject can experience an event that **precludes** the event of interest (death before
31
+ recurrence), treating the competing event as ordinary censoring is *informative* censoring:
32
+ the naive **1−KM overestimates** the cumulative incidence of the event of interest. Produce the
33
+ **cumulative incidence function (CIF)** with an Aalen–Johansen / Fine–Gray estimator instead.
34
+ This is the produce-side of probe **S3**.
35
+
36
+ ```python
37
+ # lifelines: cumulative incidence for a competing-risks event of interest
38
+ from lifelines import AalenJohansenFitter, KaplanMeierFitter
39
+ import pandas as pd
40
+ df = pd.read_csv("survival.csv") # time, event_type (0=censored, 1=interest, 2=competing)
41
+
42
+ ajf = AalenJohansenFitter()
43
+ ajf.fit(df["time"], df["event_type"], event_of_interest=1)
44
+ print(ajf.cumulative_density_.tail(1)) # correct CIF for cause 1
45
+
46
+ kmf = KaplanMeierFitter() # NAIVE (cause 2 censored) — overestimates
47
+ kmf.fit(df["time"], (df["event_type"] == 1).astype(int))
48
+ print("naive 1-KM:", float(1 - kmf.survival_function_.iloc[-1, 0]))
49
+ # the naive value will exceed the Aalen-Johansen CIF whenever the competing event is common.
50
+ ```
51
+
52
+ **Which model for which question (S8 estimand):**
53
+
54
+ - **Cause-specific hazard** (a Cox model that censors the competing event) answers an
55
+ **etiologic** question — "does X change the rate of recurrence among those still at risk?"
56
+ - **Fine–Gray subdistribution hazard** (`cmprsk::crr` / `survival::finegray` in R) answers a
57
+ **prognostic / absolute-risk** question — "does X change the cumulative *incidence*?" Quote an
58
+ sHR for an incidence claim and a cause-specific HR for an etiologic claim — not the reverse.
59
+
60
+ ```r
61
+ library(survival) # Fine-Gray via a weighted Cox on the finegray split
62
+ fg <- finegray(Surv(time, factor(event_type)) ~ ., data = d, etype = 1)
63
+ crr <- coxph(Surv(fgstart, fgstop, fgstatus) ~ x, weight = fgwt, data = fg) # subdistribution HR
64
+ ```
65
+
66
+ ---
67
+
68
+ ## Proportional hazards, and RMST when it fails
69
+
70
+ A single Cox HR is a time-average; if PH is violated it averages a changing effect. Test PH,
71
+ and when it fails report a **restricted mean survival time (RMST) difference** at a fixed
72
+ horizon (an estimand that stays interpretable under non-PH) rather than one HR.
73
+
74
+ ```python
75
+ from lifelines import CoxPHFitter
76
+ cph = CoxPHFitter().fit(df, "time", "event")
77
+ cph.check_assumptions(df, p_value_threshold=0.05) # Schoenfeld; significant -> PH violated
78
+ ```
79
+
80
+ ```r
81
+ library(survRM2)
82
+ rmst2(time, status, arm, tau = 3) # RMST difference at tau=3 years, valid under non-PH
83
+ ```
84
+
85
+ Do not report a single time-averaged HR alongside a significant Schoenfeld test without also
86
+ giving a piecewise/time-stratified HR or an RMST difference (SKILL.md PH-violation rule).
87
+
88
+ ---
89
+
90
+ ## Follow-up and discrimination (S6)
91
+
92
+ - **Reverse Kaplan–Meier median follow-up** — the honest "how long were people followed"
93
+ (median event time answers a different question). Compute it by **swapping the event
94
+ indicator** (censored observations become the "events"); report it per cohort and per
95
+ outcome, with the censoring date.
96
+
97
+ ```python
98
+ kmf.fit(df["time"], 1 - df["event"]) # swap: censored -> event
99
+ print("reverse-KM median follow-up:", kmf.median_survival_time_)
100
+ ```
101
+
102
+ - **C-index variant** — Harrell's C is biased under heavy or non-random censoring; report
103
+ **Uno's IPCW C** (`survC1::Est.Cval` / `timeROC`) and a **time-dependent AUC at a clinical
104
+ horizon** (2-/3-year) beside it, and state which variant and horizon (S6).
105
+
106
+ ---
107
+
108
+ ## Estimand provenance (S8)
109
+
110
+ State the survival estimand explicitly and hold it consistent across Abstract / Methods /
111
+ Results — event-free survival vs cause-specific cumulative incidence vs all-cause mortality,
112
+ and subject vs population level — with the evaluation horizon fixed in advance. Do not
113
+ re-designate the primary endpoint, model, or horizon after seeing results, and make every
114
+ derived statistic (an E-value, an sHR-vs-cause-specific contrast) trace to the *declared
115
+ primary* estimand. The self-review skill automates the registration ↔ manuscript and E-value
116
+ arithmetic checks (Phase 2.5f); see also `estimand-provenance-lock`.
117
+
118
+ ---
119
+
120
+ ## Reporting
121
+
122
+ - KM curves **with a number-at-risk table**; median survival with 95% CI (or the reason it is
123
+ not reached).
124
+ - Cox HR (95% CI) with the **PH check stated**; for competing risks, the CIF and whether the
125
+ HR is cause-specific or subdistribution.
126
+ - Events and person-time per group; the **reverse-KM median follow-up**; EPV for the model.
127
+ - The estimand and horizon, stated once and consistent everywhere.
128
+
129
+ ---
130
+
131
+ ## Common failures (flag at review)
132
+
133
+ - **Competing risks ignored** — naive 1−KM (or a cause-specific model presented as absolute
134
+ risk) overestimates incidence; report a CIF and name cause-specific vs subdistribution (S3/S8).
135
+ - **A single time-averaged HR under a violated PH assumption** — needs a piecewise HR or RMST.
136
+ - **Harrell's C under heavy censoring** with no Uno/IPCW variant and no horizon (S6).
137
+ - **Median *survival* reported as "follow-up"** instead of the reverse-KM follow-up.
138
+ - **Estimand drift** — a primary endpoint/model/horizon re-designated post-hoc, or a derived
139
+ statistic quoted off a non-primary estimate (S8).
140
+
141
+ ---
142
+
143
+ ## Anti-Hallucination
144
+
145
+ - Never hand-type an HR, median, CIF, or CI — compute it from the time-to-event CSV with a
146
+ seeded script, and carry each estimate together with its CI.
147
+ - Do not report a Cox HR without the proportional-hazards check the code actually ran.
148
+ - Under competing risks, do not quote a 1−KM cumulative incidence — report the Aalen–Johansen /
149
+ Fine–Gray CIF the code produced.
@@ -47,6 +47,7 @@ A checklist for **diagnostic test accuracy (DTA) primary studies** — an index
47
47
  - When a reader study's novelty is a **confidence-weighted** (or rating-collapsed) score used as the ROC/AUC predictor, the **unweighted binary-call AUC** must be reported side-by-side (as a sensitivity analysis). Without it, you cannot tell whether the weighting *created* the result or hid an estimator fragility (e.g. a folded/non-monotonic encoding that collapses `real/5` with `ai/1`).
48
48
  - Lead: if the primary predictor is a confidence/rating→single-score collapse and no unweighted binary-call AUC appears, ask for it; also confirm the (call × confidence) encoding is strictly monotonic (no boundary collision) — the folded-score bug understates one hypothesis and can flip another.
49
49
  - Severity: MAJOR when the weighted score is the headline and no unweighted baseline is shown; the weighting must "earn its place" against the simpler estimator.
50
+ - Produce the fix: `analyze-stats` `references/analysis_guides/diagnostic_accuracy.md` has the monotonic-encoding check + the unweighted-baseline AUC beside the weighted primary (and the per-stratum admissibility table for D10 and the one-scale-per-comparison rule for D11).
50
51
 
51
52
  **D10 — "No stratum met threshold X" vs a per-stratum table that does meet X**:
52
53
  - When the manuscript states a numeric admissibility/deployability rule (e.g. "AUC ≥ 0.75 **and** lower 95% bound ≥ 0.70") and concludes "**no stratum met** the rule" / "all strata were below", cross-check that claim against the **per-stratum AUC + CI table**. A blanket negative-stratum claim contradicted by a tabled stratum that literally satisfies the rule (e.g. ultrasound 0.789, CI 0.742–0.834) is a self-contradiction a reviewer verifies with arithmetic.
@@ -28,6 +28,7 @@ A 9-probe checklist for time-to-event outcomes and prognostic model development.
28
28
  - Cause-specific hazards or Fine-Gray subdistribution hazards used?
29
29
  - Patient developing one event still at risk for the other (informative censoring by death)?
30
30
  - If competing-risks structure is ignored and outcomes are treated as independent right-censored events → MAJOR.
31
+ - Produce the fix: `analyze-stats` `references/analysis_guides/survival.md` has the Aalen–Johansen/Fine–Gray cumulative-incidence code (naive 1−KM overestimates) + the cause-specific-vs-subdistribution estimand choice (S8) and the PH→RMST fallback.
31
32
 
32
33
  **S4 — Cutoff derivation optimism**:
33
34
  - Cutoffs derived via maximally selected log-rank statistics, AUC-based Youden's J, or similar data-driven methods?
@@ -47,6 +47,7 @@ A checklist for **diagnostic test accuracy (DTA) primary studies** — an index
47
47
  - When a reader study's novelty is a **confidence-weighted** (or rating-collapsed) score used as the ROC/AUC predictor, the **unweighted binary-call AUC** must be reported side-by-side (as a sensitivity analysis). Without it, you cannot tell whether the weighting *created* the result or hid an estimator fragility (e.g. a folded/non-monotonic encoding that collapses `real/5` with `ai/1`).
48
48
  - Lead: if the primary predictor is a confidence/rating→single-score collapse and no unweighted binary-call AUC appears, ask for it; also confirm the (call × confidence) encoding is strictly monotonic (no boundary collision) — the folded-score bug understates one hypothesis and can flip another.
49
49
  - Severity: MAJOR when the weighted score is the headline and no unweighted baseline is shown; the weighting must "earn its place" against the simpler estimator.
50
+ - Produce the fix: `analyze-stats` `references/analysis_guides/diagnostic_accuracy.md` has the monotonic-encoding check + the unweighted-baseline AUC beside the weighted primary (and the per-stratum admissibility table for D10 and the one-scale-per-comparison rule for D11).
50
51
 
51
52
  **D10 — "No stratum met threshold X" vs a per-stratum table that does meet X**:
52
53
  - When the manuscript states a numeric admissibility/deployability rule (e.g. "AUC ≥ 0.75 **and** lower 95% bound ≥ 0.70") and concludes "**no stratum met** the rule" / "all strata were below", cross-check that claim against the **per-stratum AUC + CI table**. A blanket negative-stratum claim contradicted by a tabled stratum that literally satisfies the rule (e.g. ultrasound 0.789, CI 0.742–0.834) is a self-contradiction a reviewer verifies with arithmetic.
@@ -28,6 +28,7 @@ A 9-probe checklist for time-to-event outcomes and prognostic model development.
28
28
  - Cause-specific hazards or Fine-Gray subdistribution hazards used?
29
29
  - Patient developing one event still at risk for the other (informative censoring by death)?
30
30
  - If competing-risks structure is ignored and outcomes are treated as independent right-censored events → MAJOR.
31
+ - Produce the fix: `analyze-stats` `references/analysis_guides/survival.md` has the Aalen–Johansen/Fine–Gray cumulative-incidence code (naive 1−KM overestimates) + the cause-specific-vs-subdistribution estimand choice (S8) and the PH→RMST fallback.
31
32
 
32
33
  **S4 — Cutoff derivation optimism**:
33
34
  - Cutoffs derived via maximally selected log-rank statistics, AUC-based Youden's J, or similar data-driven methods?