medsci-skills 5.2.0 → 5.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -13
- package/metadata/distribution_files.json +47 -12
- package/metadata/distribution_manifest.json +1 -1
- package/package.json +1 -1
- package/skills/analyze-stats/SKILL.md +9 -0
- package/skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md +79 -0
- package/skills/check-reporting/SKILL.md +6 -2
- package/skills/check-reporting/references/checklists/CHEERS_2022.md +81 -0
- package/skills/check-reporting/references/checklists/RECORD.md +66 -0
- package/skills/check-reporting/scripts/check_checklist_exists.py +3 -0
- package/skills/make-figures/references/reporting_guideline_figure_map.md +2 -2
- package/skills/orchestrate/SKILL.md +1 -1
- package/skills/peer-review/SKILL.md +12 -0
- package/skills/peer-review/references/domain-probes/health_economic_evaluation.md +52 -0
- package/skills/peer-review/references/domain-probes/record_routinely_collected_data.md +49 -0
- package/skills/self-review/SKILL.md +2 -0
- package/skills/self-review/references/domain-probes/health_economic_evaluation.md +52 -0
- package/skills/self-review/references/domain-probes/record_routinely_collected_data.md +49 -0
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
**51 skills that actually work.** Built by a physician-researcher, tested on real publications.
|
|
6
6
|
|
|
7
|
-
*MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer —
|
|
7
|
+
*MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer — 40 reporting guidelines and risk-of-bias tools, reference/citation verification, and deterministic integrity gates before peer review — now extended by a model-engineering lane that scaffolds reproducible, leakage-safe training repos and audits model validation. Clinical AI model research engineering is in scope; a general AI-scientist platform is not. It competes on clinical submission reliability, not skill count.*
|
|
8
8
|
|
|
9
9
|
[](LICENSE)
|
|
10
10
|
[](https://github.com/Aperivue/medsci-skills/releases/latest)
|
|
@@ -281,54 +281,54 @@ The E2E pipeline (`orchestrate --e2e`) produces everything up to `qc/`. The `sub
|
|
|
281
281
|
|
|
282
282
|
## What's New
|
|
283
283
|
|
|
284
|
-
**v4.10** — reviewer-coverage expansion reverse-engineered from high-IF, CC-BY papers (learn-only under the `reverse_engineer/` license firewall), plus a clinician-friendly update path. Additive and backward-compatible; 45 skills / **
|
|
284
|
+
**v4.10** — reviewer-coverage expansion reverse-engineered from high-IF, CC-BY papers (learn-only under the `reverse_engineer/` license firewall), plus a clinician-friendly update path. Additive and backward-compatible; 45 skills / **40 guidelines** / 36 detectors / **15 domain-probe modules** (was 12):
|
|
285
285
|
|
|
286
286
|
- **Three new reviewer domain-probe modules** (`/peer-review` + `/self-review`, vendored byte-identical): **Mendelian randomization** (MR1–MR8 — IV assumptions, pleiotropy-robust sensitivity suite, Steiger, sample overlap, NLMR, drug-target colocalization), **polygenic risk score** (PG1–PG8 — ancestry portability, base/target leakage, incremental value over the clinical model, screening-vs-discrimination, calibration), and **network meta-analysis** (NM1–NM8 — transitivity, incoherence, SUCRA over-interpretation, CINeMA/GRADE-NMA, component-NMA additivity). Plus observational **O17** (agnostic many-exposure-scan multiplicity: ExWAS/EWAS/MWAS).
|
|
287
287
|
- **Two reporting-guideline checklists** (36 → 38): **STROBE-MR** and **PGS-RS / PRS-RS**, with study-type routing. Four new `/analyze-stats` analysis guides (multiplicity, MR, PRS, NMA) and a `/clean-data` implausible-value + cross-field validity reference.
|
|
288
288
|
- **Clinician-friendly update reminders** — the classroom installers enable the in-app "update available" notice + one-click Desktop updater by default; the `npx`/manual paths print how to turn it on; the install guide recommends `npx medsci-skills install --enable-update-notify`.
|
|
289
289
|
|
|
290
|
-
**v4.9** — analysis-integrity hardening promoted from real review cycles, plus journal-mechanics additions. Additive and backward-compatible; still 45 skills /
|
|
290
|
+
**v4.9** — analysis-integrity hardening promoted from real review cycles, plus journal-mechanics additions. Additive and backward-compatible; still 45 skills / 40 guidelines, analysis-integrity detectors **32 → 36**:
|
|
291
291
|
|
|
292
292
|
- **Four new gates** — a **duplicate-bibliography** check (`check_reference_duplication.py`) for the hybrid `[@key]` + hand-typed `## References` build that renders the list twice; a **cross-script binning / composite-indicator** consistency check (`check_binning_consistency.py`, `BINNING_DRIFT` / `DERIVED_DEF_DRIFT`) for a derived categorical or composite indicator defined inconsistently across analysis scripts; a **float citation-order** check (`check_citation_order.py`) for numbered Tables/Figures not first cited in ascending order per series; and an **audit-dump leak** gate (`/sync-submission`) that blocks a `/check-reporting` output mistakenly attached as a submission file.
|
|
293
293
|
- **KJR technical-check conventions + percentage-decimal style**, reader-allocation-under-burden and generative-image-as-study-object reporting (`/design-ai-benchmarking`, `/check-reporting`), and a **Liver International** CSL with that journal's submission mechanics (`/manage-refs`).
|
|
294
294
|
|
|
295
|
-
**v4.8** is the **review-harvest batch** — deterministic detector hardening promoted from real-manuscript review cycles. Additive and backward-compatible; still 45 skills /
|
|
295
|
+
**v4.8** is the **review-harvest batch** — deterministic detector hardening promoted from real-manuscript review cycles. Additive and backward-compatible; still 45 skills / 40 guidelines, analysis-integrity detectors **30 → 32**:
|
|
296
296
|
|
|
297
297
|
- **Two new gates** — `check_supplement_hygiene.py` lints the rendered supplement / tables / caption files (not just the manuscript) for §-labels, placeholders, build markers, response-letter framing, and unresolved body↔supplement cross-references; `check_null_calibration.py` flags a headline negative/equivalence claim made without a minimum-detectable-effect / power / equivalence statement.
|
|
298
298
|
- **Four detector false-positive fixes** — gates no longer fire on a recommended colorblind-safe palette, author-footnote `§` daggers, a correctly-hedged disclaimer, or a tier-label digit; each with a regression fixture and three newly CI-wired test suites.
|
|
299
299
|
- **Nine reviewer-side domain probes** (SR/MA, observational, diagnostic, AI-overclaiming, survival) plus a `/design-study` design-stage ceiling gate for perceptual/reader-AI studies and a reusable confidence-weighted-rating→AUC monotonicity template.
|
|
300
300
|
|
|
301
|
-
**v4.7** is the **self-update foundation** — physician-researchers stay current without GitHub, git, or a terminal. Additive and backward-compatible; still 45 skills /
|
|
301
|
+
**v4.7** is the **self-update foundation** — physician-researchers stay current without GitHub, git, or a terminal. Additive and backward-compatible; still 45 skills / 40 guidelines / 30 detectors:
|
|
302
302
|
|
|
303
303
|
- **Transactional, crash-recoverable installer.** Each install runs through a durable journal state machine recovered on the next run (roll back / forward-clean / fail-closed), with per-target SHA-256 inventories — your modified or third-party skills are backed up and never clobbered or auto-deleted.
|
|
304
304
|
- **One-click self-updater** (`~/.medsci-skills/updater/`, `install.py --check-update`). Verifies the download against the github.com API digest and **never `extractall()`s** (per-entry rejection of traversal / symlink / duplicate / zip-bomb + an allowlist & per-file hash). The release pipeline injects a verified `provenance.json`, attests build provenance, runs on a protected `release` environment, and verifies each ZIP round-trips through the updater's own safe-extract before publishing.
|
|
305
305
|
- **Opt-in update notice (off by default):** `install.py --enable-update-notify` shows a one-line "update available" message at Claude Code session start — no telemetry, reads nothing about your session, installs nothing. `--disable-update-notify` / `MEDSCI_NO_UPDATE_CHECK=1` turn it off. *(Honest scope: the digest/attestation detect transport tampering, not a compromised publisher account — see `SECURITY.md`.)*
|
|
306
306
|
|
|
307
|
-
**v4.6** is a maintainability, governance, and review-depth release — still 45 skills /
|
|
307
|
+
**v4.6** is a maintainability, governance, and review-depth release — still 45 skills / 40 guidelines; analysis-integrity detectors **28 → 30**, domain probes 11 → 12:
|
|
308
308
|
|
|
309
309
|
- **Fairness / equity / subgroup-performance probe (EQ0–EQ6)** for AI/prediction/diagnostic studies that claim cross-population performance, plus two new detectors: an **AI-disclosure + data/code-availability** check (`/sync-submission`) and a **structured-summary-box conformance** check (`/academic-aio`).
|
|
310
310
|
- **Governance + answer-engine layer:** `ROADMAP.md`, `MAINTAINERS.md`, `SECURITY.md`, a maintainer workflow + release checklist, an AEO/GEO `docs/faq.md`, a "Start here: 3 workflows" + "Validation status" section in this README, and a new `maturity` field (official / experimental / community) on every skill.
|
|
311
311
|
- **Token diet (pilot):** `write-paper` Phase 7 integrity audits moved to a load-on-demand reference (~2,559 tokens saved per invocation). Positioning now leads with the compliance moat rather than skill count.
|
|
312
312
|
|
|
313
|
-
**v4.5** deepens the review + submission surface with no new skill or reporting-guideline count (still 45 skills /
|
|
313
|
+
**v4.5** deepens the review + submission surface with no new skill or reporting-guideline count (still 45 skills / 40 guidelines); analysis-integrity detectors **27 → 28**:
|
|
314
314
|
|
|
315
315
|
- **`/clean-data` + `/analyze-stats` — reverse-coded-item / negative-alpha detector.** A multi-item Likert scale with a negatively-worded item must be recoded `(min+max) − x` before the scale total or Cronbach's alpha is computed; left un-recoded, the item correlates negatively with the rest of the scale and alpha collapses (often negative). A negative alpha is a coding bug, not a "multidimensional construct." New stdlib-only `check_reverse_coding.py` returns `REVERSE_CODING_LIKELY` / `REVERSE_CODING_SUSPECT` / `OK` from per-item item-rest correlations + raw alpha; the Likert summary template gains a `--reverse-items` recode flag.
|
|
316
316
|
- **`/peer-review` + `/self-review` — SR/MA + DTA + prediction-model probe batch.** `sr_ma.md` **P12** risk-of-bias table row-sum ↔ traffic-light figure-matrix reconciliation and **P13** included-study ↔ reference-list completeness; `diagnostic_accuracy.md` **D7** index-test-as-enrollment-criterion circularity; `clinical_prediction_model.md` **CP5** intended-use horizon leakage and **CP6** development/CV vs held-out/external validation-nomenclature conflation. Vendored byte-identical into `/self-review`.
|
|
317
317
|
- **`/sync-submission` — embedded absolute-path leak scan.** A `word/*.xml` attribute (e.g. a pandoc-embedded image's `<pic:cNvPr descr="…">`) carrying an absolute home-dir path (`/Users/…`, `/home/…`) is a username leak invisible to a rendered-text scan; now flagged as `docx_embedded_abs_path` under `check_asset_anonymization.py`.
|
|
318
318
|
|
|
319
|
-
**v4.4** adds reviewer/analysis depth with no new skill or reporting-guideline count (still 45 skills /
|
|
319
|
+
**v4.4** adds reviewer/analysis depth with no new skill or reporting-guideline count (still 45 skills / 40 guidelines / 27 detectors):
|
|
320
320
|
|
|
321
321
|
- **`/author-strategy` — trajectory-archetype classification (optional).** Classifies a queried author's PubMed trajectory into abstract career archetypes (A1 infrastructure builder, A2 methodology rule-maker, A3 clinical→AI hybrid, A4 SR/MA volume engine, A5 large-consortium participation, A6 device/technique depth, + a computed composite) as an **explainable, multi-label, confidence-scored heuristic — not an objective verdict**. The rubric is a single canonical YAML (the narrative doc is generated from it); scores exclude `unavailable` signals (h-index/citation/venue-tier → `[VERIFY]`, never fabricated); a **disambiguation gate** binds an approved `corpus_manifest.json` to the CSV (csv + PMID-set hashes) so a surname alone never classifies, and target-author attribution never borrows a co-author's ORCID/affiliation.
|
|
322
322
|
- **`/peer-review` + `/self-review` — Image-Synthesis / cross-modality probe (IS1–IS4)** for studies that synthesize one imaging modality from another and claim the output carries the target's information, plus a reviewer-side reference-integrity spot-check.
|
|
323
323
|
- **`/verify-refs` — OpenAlex tertiary index** recovers conference-proceedings / non-DOI citations (NeurIPS/ICLR/ACL) that fall through PubMed and CrossRef, the free analogue of a portal's second index.
|
|
324
324
|
|
|
325
|
-
**v4.3** hardens the **cross-sectional / observational cohort** review surface end-to-end, much of it reverse-engineered from real CC-BY cohort papers (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills /
|
|
325
|
+
**v4.3** hardens the **cross-sectional / observational cohort** review surface end-to-end, much of it reverse-engineered from real CC-BY cohort papers (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills / 40 guidelines); analysis-integrity detectors **25 → 27**:
|
|
326
326
|
|
|
327
327
|
- **Observational probes O1 → O14** (`/peer-review` + `/self-review`, vendored) — over-adjustment / analysis-unit clustering / outcome construct-validity (O7–O9), overlapping-subset gradient (O10), **complex-survey design & weighting** for NHANES/KNHANES (O11), **data-driven threshold / "inflection-point" mining** (O12), **cross-sectional mediation** temporal-order & sequential-ignorability (O13), and **interaction scale** — additive RERI/AP/S vs multiplicative (O14). Plus a new **clinical-prediction-model** probe module **CP1–CP4** and survival **S9** (panel-data / multistate variance).
|
|
328
328
|
- **Two new detectors (25 → 27)** — `check_wordcount_cap.py` (the revision-inflation trap: body vs journal cap) and `check_paren_spans.py` (em-dash→paren conversions that wrap a whole sentence). Plus a `check_confounding_completeness.py` upgrade (DB-code↔prose alias map, SMD-from-mean±SD, exposure-defining-covariate exemption), a `check_cohort_arithmetic.py` `ANALYSIS_UNIT_UNDISCLOSED` check, a `check_scope_coherence.py` cross-sectional-yield lexicon, and a verify-refs corporate/collective-author render-abort fix.
|
|
329
329
|
- **Analysis & submission tooling** — `/analyze-stats` gains **mediation** and **interaction & effect-modification** guides; `/sync-submission` gains `assemble_supplement.py` (S{N} index↔file integrity) and a `/revise` body-word-count exit gate; `/render-pdf-doc` gains a `scan_glyph_coverage.py` xelatex silent-glyph-drop scan.
|
|
330
330
|
|
|
331
|
-
**v4.2** builds out the case-report capability end-to-end, grounded in real CC-BY case reports (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills /
|
|
331
|
+
**v4.2** builds out the case-report capability end-to-end, grounded in real CC-BY case reports (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills / 40 guidelines); journal profiles **68 → 73**:
|
|
332
332
|
|
|
333
333
|
- **Case-report + case-series writing** — `/write-paper` gains a CARE narrative + 150-word-abstract case-report exemplar, a **case-series** paper type (methods-light mini-cohort, all-cases summary table, counts-not-rates), and **adverse-event/pharmacovigilance** (Naranjo/WHO-UMC causality) and **diagnostic-pitfall/mimic** subtypes.
|
|
334
334
|
- **Radiology / imaging-led track** — a dedicated `exemplar_case_report_radiology.md` (per-modality technique→findings→impression, structured-reporting lexicons BI-RADS/LI-RADS/PI-RADS/TI-RADS/Lung-RADS/O-RADS, quantitative threshold honesty, an interventional-radiology procedure/complication subtype, DICOM de-identification) plus a `/make-figures` annotated multimodality imaging-panel exemplar.
|
|
@@ -452,7 +452,7 @@ ma-scout -> search-lit -> fulltext-retrieval -> design-study ──> write-proto
|
|
|
452
452
|
| **search-lit** | PubMed + Semantic Scholar + bioRxiv search with anti-hallucination citation verification. Token-efficient error handling -- CrossRef failures are silently batched, not repeated. BibTeX output tags each entry with `verified`/`verified_by`/`verified_on` fields so downstream skills can trust the citation provenance. |
|
|
453
453
|
| **verify-refs** | Pre-submission reference audit for `.md`, `.docx`, `.bib`, or `.tsv` inputs. Extracts references, verifies DOI/PMID via CrossRef/PubMed when available, and writes `qc/reference_audit.json` as the sole output — row-level status (OK / MISMATCH / UNVERIFIED / FABRICATED) lives inside the JSON `records[]` block. `/search-lit` produces candidate BibTeX; `/lit-sync` owns `manuscript/_src/refs.bib`. |
|
|
454
454
|
| **fulltext-retrieval** | Batch open-access PDF downloader. Unpaywall → PMC → OpenAlex → CrossRef pipeline. OA-only -- no paywall bypass. Input: DOI list or TSV. Optional PDF→Markdown conversion via [pymupdf4llm](https://pymupdf.readthedocs.io/en/latest/pymupdf4llm/) for token-efficient LLM analysis of academic papers. |
|
|
455
|
-
| **check-reporting** | Manuscript compliance audit against
|
|
455
|
+
| **check-reporting** | Manuscript compliance audit against 40 reporting guidelines and risk of bias tools (STROBE, STROBE-MR, RECORD, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, PRISMA, PRISMA-DTA, PRISMA-P, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA). Machine-readable JSON summary with `compliance_pct` and `fixable_by_ai` flags for automated pipeline integration. |
|
|
456
456
|
| **analyze-stats** | Statistical analysis code generation (Python/R) for diagnostic accuracy, DTA meta-analysis (bivariate/HSROC), inter-rater agreement, survival analysis, demographics tables, regression (logistic/linear), propensity score (matching/IPTW/overlap weighting), and repeated measures (RM ANOVA/GEE/mixed models). Calibration mandatory for prediction models. |
|
|
457
457
|
| **meta-analysis** | Full systematic review and meta-analysis pipeline (8 phases). DTA (bivariate/HSROC) and intervention meta-analysis. Protocol to submission-ready manuscript with PRISMA-DTA compliance. |
|
|
458
458
|
| **make-figures** | Publication-ready figures and visual abstracts: ROC curves, forest plots, PRISMA/CONSORT/STARD flow diagrams, Kaplan-Meier curves, Bland-Altman plots, confusion matrices, and journal-specific visual/graphical abstracts (python-pptx template-based). Communication-first design principles (Nat Hum Behav 2026 — key message, audience, cognitive load, figure-vs-table decision) and five flow-diagram production lessons (official-template fidelity, VML fallback PDF export, docx XML escape, sequential placeholder mapping, version freeze); critic rubric Section G adds 5 communication-first checks. `--study-type` auto-generates the full required figure set; structured `_figure_manifest.md` output for downstream pipeline consumption; D2 enforced as default for flow diagrams. |
|
|
@@ -621,8 +621,8 @@ Projects declare their source-of-truth layout in `SSOT.yaml`, and a `qc/migratio
|
|
|
621
621
|
### Meta-Analysis Failure Modes
|
|
622
622
|
`/meta-analysis` ships empirical failure-mode references (data integrity, review orchestration, submission package drift, post-submission release ops) with four automation hooks: `scripts/prisma_5way_consistency.py` (DI-6 PRISMA number consistency), `scripts/extraction_consensus_log_init.py` (DI-1 dual-extraction scaffold), `scripts/tag_cleanup_gate.sh` (DI-8 placeholder tag gate), and `scripts/verify_package_integrity.py` (SPD SHA-256 manifest for submission bundles).
|
|
623
623
|
|
|
624
|
-
###
|
|
625
|
-
`check-reporting` includes bundled checklists for
|
|
624
|
+
### 40 Reporting Guidelines & RoB Tools Built-in
|
|
625
|
+
`check-reporting` includes bundled checklists for 40 guidelines and risk-of-bias tools: STROBE, STROBE-MR, RECORD, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, PRISMA 2020, PRISMA-DTA, PRISMA-P, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA. Includes Results/Discussion section boundary checks and machine-readable JSON summary for pipeline integration.
|
|
626
626
|
|
|
627
627
|
### Publication-Ready Output
|
|
628
628
|
`analyze-stats` generates reproducible Python/R code for 13 analysis types -- including regression, propensity score, and repeated measures -- with mandatory calibration for prediction models. `make-figures` produces journal-specification figures (300 DPI, colorblind-safe palettes, proper dimensions), visual/graphical abstracts, and a tool selection guide (D2 for flow diagrams, matplotlib for data plots). `--study-type` auto-generates the complete figure set for each study design.
|
|
@@ -153,8 +153,13 @@
|
|
|
153
153
|
},
|
|
154
154
|
{
|
|
155
155
|
"path": "skills/analyze-stats/SKILL.md",
|
|
156
|
-
"size":
|
|
157
|
-
"sha256": "
|
|
156
|
+
"size": 54473,
|
|
157
|
+
"sha256": "6f7a194db60588e84ddf8e62e80d6e143d14265e21bdac35efa6efbbdd5d5ffc"
|
|
158
|
+
},
|
|
159
|
+
{
|
|
160
|
+
"path": "skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md",
|
|
161
|
+
"size": 5362,
|
|
162
|
+
"sha256": "a6f223cf060b1a3b472df82660a11c006c5ea2e72e231b730ce326b00a11044a"
|
|
158
163
|
},
|
|
159
164
|
{
|
|
160
165
|
"path": "skills/analyze-stats/references/analysis_guides/mediation.md",
|
|
@@ -533,8 +538,8 @@
|
|
|
533
538
|
},
|
|
534
539
|
{
|
|
535
540
|
"path": "skills/check-reporting/SKILL.md",
|
|
536
|
-
"size":
|
|
537
|
-
"sha256": "
|
|
541
|
+
"size": 37305,
|
|
542
|
+
"sha256": "fa06d9d4063773a29d58603e3922d89a9b288773cc5b27022ef6eee16352b971"
|
|
538
543
|
},
|
|
539
544
|
{
|
|
540
545
|
"path": "skills/check-reporting/references/LICENSES.md",
|
|
@@ -566,6 +571,11 @@
|
|
|
566
571
|
"size": 5746,
|
|
567
572
|
"sha256": "4bac869891f4b334cdc211409062a1dd3d6a31a0b62218ad4d9ced1a34000f77"
|
|
568
573
|
},
|
|
574
|
+
{
|
|
575
|
+
"path": "skills/check-reporting/references/checklists/CHEERS_2022.md",
|
|
576
|
+
"size": 8054,
|
|
577
|
+
"sha256": "830dc33cc4d74a3ebd4f024bb05fd064174002b150b363c3f3f03a13e2bd9a4f"
|
|
578
|
+
},
|
|
569
579
|
{
|
|
570
580
|
"path": "skills/check-reporting/references/checklists/CLAIM_2024.md",
|
|
571
581
|
"size": 6678,
|
|
@@ -656,6 +666,11 @@
|
|
|
656
666
|
"size": 7571,
|
|
657
667
|
"sha256": "f3527f32851fbf4d446e7f9961b134d455c2fa19d7615ea65168ae0479fb6685"
|
|
658
668
|
},
|
|
669
|
+
{
|
|
670
|
+
"path": "skills/check-reporting/references/checklists/RECORD.md",
|
|
671
|
+
"size": 6530,
|
|
672
|
+
"sha256": "186000d9cd72e025ce483995735c05400450d9356f89f8f95b855d5815e925fa"
|
|
673
|
+
},
|
|
659
674
|
{
|
|
660
675
|
"path": "skills/check-reporting/references/checklists/ROBINS_E.md",
|
|
661
676
|
"size": 8206,
|
|
@@ -763,8 +778,8 @@
|
|
|
763
778
|
},
|
|
764
779
|
{
|
|
765
780
|
"path": "skills/check-reporting/scripts/check_checklist_exists.py",
|
|
766
|
-
"size":
|
|
767
|
-
"sha256": "
|
|
781
|
+
"size": 6700,
|
|
782
|
+
"sha256": "9a513070dd36b06bbddcf6eb0c4a4508f523e421cd8182ccad449c1250473645"
|
|
768
783
|
},
|
|
769
784
|
{
|
|
770
785
|
"path": "skills/check-reporting/scripts/check_checklist_version.py",
|
|
@@ -2004,7 +2019,7 @@
|
|
|
2004
2019
|
{
|
|
2005
2020
|
"path": "skills/make-figures/references/reporting_guideline_figure_map.md",
|
|
2006
2021
|
"size": 6862,
|
|
2007
|
-
"sha256": "
|
|
2022
|
+
"sha256": "54f2aaee54424ebedbe8f38424bebcf52168835222bf50be91da49e816490784"
|
|
2008
2023
|
},
|
|
2009
2024
|
{
|
|
2010
2025
|
"path": "skills/make-figures/references/visual_abstract_templates/european_radiology.pptx",
|
|
@@ -2794,7 +2809,7 @@
|
|
|
2794
2809
|
{
|
|
2795
2810
|
"path": "skills/orchestrate/SKILL.md",
|
|
2796
2811
|
"size": 35203,
|
|
2797
|
-
"sha256": "
|
|
2812
|
+
"sha256": "022099a1b5321fcafb4b9ac54548b212edd2a1a67894730f1fdce1a128c0f6fe"
|
|
2798
2813
|
},
|
|
2799
2814
|
{
|
|
2800
2815
|
"path": "skills/orchestrate/references/dialogue_nodes.md",
|
|
@@ -2818,8 +2833,8 @@
|
|
|
2818
2833
|
},
|
|
2819
2834
|
{
|
|
2820
2835
|
"path": "skills/peer-review/SKILL.md",
|
|
2821
|
-
"size":
|
|
2822
|
-
"sha256": "
|
|
2836
|
+
"size": 62244,
|
|
2837
|
+
"sha256": "93cff5db0c82d06ba1a5b4a64dccb371fc081930bf7c16878c33cb9ba8a3ba40"
|
|
2823
2838
|
},
|
|
2824
2839
|
{
|
|
2825
2840
|
"path": "skills/peer-review/references/aczel_2021_reviewer2_patterns.md",
|
|
@@ -2851,6 +2866,11 @@
|
|
|
2851
2866
|
"size": 7665,
|
|
2852
2867
|
"sha256": "83f2c5116317f96ba564f459f2b679489cc2ad8d2a3ec8078345ce1390680eb2"
|
|
2853
2868
|
},
|
|
2869
|
+
{
|
|
2870
|
+
"path": "skills/peer-review/references/domain-probes/health_economic_evaluation.md",
|
|
2871
|
+
"size": 8393,
|
|
2872
|
+
"sha256": "b322a9a6ae2439c1dc97f6a750572eecb3ce1481c139516665a55c9d571dfe8e"
|
|
2873
|
+
},
|
|
2854
2874
|
{
|
|
2855
2875
|
"path": "skills/peer-review/references/domain-probes/image_synthesis.md",
|
|
2856
2876
|
"size": 6295,
|
|
@@ -2901,6 +2921,11 @@
|
|
|
2901
2921
|
"size": 9293,
|
|
2902
2922
|
"sha256": "23bdfcd9899f02792f4afcd28f1301822c6a5da8fc9223353b3354ca010913cf"
|
|
2903
2923
|
},
|
|
2924
|
+
{
|
|
2925
|
+
"path": "skills/peer-review/references/domain-probes/record_routinely_collected_data.md",
|
|
2926
|
+
"size": 7574,
|
|
2927
|
+
"sha256": "04174053da808ee99f72f21d41225d76e856e94aff5088dffd8040ec40a4aa4e"
|
|
2928
|
+
},
|
|
2904
2929
|
{
|
|
2905
2930
|
"path": "skills/peer-review/references/domain-probes/sr_ma.md",
|
|
2906
2931
|
"size": 17239,
|
|
@@ -3358,8 +3383,8 @@
|
|
|
3358
3383
|
},
|
|
3359
3384
|
{
|
|
3360
3385
|
"path": "skills/self-review/SKILL.md",
|
|
3361
|
-
"size":
|
|
3362
|
-
"sha256": "
|
|
3386
|
+
"size": 95130,
|
|
3387
|
+
"sha256": "7676619bbda458b1cae1ab148797d8a434a71dcdade12eed41deb4de87d603e6"
|
|
3363
3388
|
},
|
|
3364
3389
|
{
|
|
3365
3390
|
"path": "skills/self-review/references/domain-probes/ai_overclaiming.md",
|
|
@@ -3386,6 +3411,11 @@
|
|
|
3386
3411
|
"size": 7665,
|
|
3387
3412
|
"sha256": "83f2c5116317f96ba564f459f2b679489cc2ad8d2a3ec8078345ce1390680eb2"
|
|
3388
3413
|
},
|
|
3414
|
+
{
|
|
3415
|
+
"path": "skills/self-review/references/domain-probes/health_economic_evaluation.md",
|
|
3416
|
+
"size": 8393,
|
|
3417
|
+
"sha256": "b322a9a6ae2439c1dc97f6a750572eecb3ce1481c139516665a55c9d571dfe8e"
|
|
3418
|
+
},
|
|
3389
3419
|
{
|
|
3390
3420
|
"path": "skills/self-review/references/domain-probes/image_synthesis.md",
|
|
3391
3421
|
"size": 6295,
|
|
@@ -3436,6 +3466,11 @@
|
|
|
3436
3466
|
"size": 9293,
|
|
3437
3467
|
"sha256": "23bdfcd9899f02792f4afcd28f1301822c6a5da8fc9223353b3354ca010913cf"
|
|
3438
3468
|
},
|
|
3469
|
+
{
|
|
3470
|
+
"path": "skills/self-review/references/domain-probes/record_routinely_collected_data.md",
|
|
3471
|
+
"size": 7574,
|
|
3472
|
+
"sha256": "04174053da808ee99f72f21d41225d76e856e94aff5088dffd8040ec40a4aa4e"
|
|
3473
|
+
},
|
|
3439
3474
|
{
|
|
3440
3475
|
"path": "skills/self-review/references/domain-probes/sr_ma.md",
|
|
3441
3476
|
"size": 17239,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "medsci-skills",
|
|
3
|
-
"version": "5.
|
|
3
|
+
"version": "5.4.0",
|
|
4
4
|
"description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
|
|
5
5
|
"license": "SEE LICENSE IN LICENSE",
|
|
6
6
|
"homepage": "https://github.com/Aperivue/medsci-skills#readme",
|
|
@@ -456,6 +456,15 @@ tbl %>% as_flex_table() %>% flextable::save_as_docx(path = "table.docx")
|
|
|
456
456
|
- Present a **network plot** (node ∝ sample size, edge ∝ #trials); report global **τ²**; **ranking (SUCRA/P-score) is not a superiority test** — report it with the league table, intervals, and certainty
|
|
457
457
|
- Certainty **per estimate** via **CINeMA / GRADE-NMA** (downgrade indirect-only); component NMA assumes **additivity** (state/check it). Report against **PRISMA-NMA**; risk of bias via **RoB-NMA**. Review-side probes: NM1–NM8 in `network_meta_analysis.md`
|
|
458
458
|
|
|
459
|
+
### Health Economic Evaluation
|
|
460
|
+
|
|
461
|
+
- **Guide**: Load `analysis_guides/health_economic_evaluation.md` before generating code
|
|
462
|
+
- For cost-effectiveness (CEA), cost-utility (CUA, QALY), cost-benefit (CBA), cost-minimisation, or budget-impact analyses; trial-based or decision-model-based (decision tree, **Markov/state-transition**, discrete-event simulation)
|
|
463
|
+
- Compute **incremental cost ΔC, incremental effect ΔE, and the ICER = ΔC/ΔE**; with ≥3 options remove **dominated / extended-dominated** strategies before sequential ICERs; prefer **net benefit (INMB = λΔE − ΔC)** for regression/probabilistic summaries
|
|
464
|
+
- State and justify the **perspective, time horizon (lifetime for chronic disease), discount rate (both costs and outcomes), currency + price year**; QALYs from a named preference-based instrument + value set
|
|
465
|
+
- **Uncertainty is the analytic core**: one-way / **tornado** for drivers, **probabilistic sensitivity analysis (PSA)** with justified parameter distributions (beta for probabilities/utilities, gamma/log-normal for costs) → **cost-effectiveness plane + CEAC**; scenario analyses for structural choices
|
|
466
|
+
- R `heemod` / `dampack` / `hesim` / `BCEA` (state-transition + PSA + CEAC + EVPI), `flexsurv` for survival extrapolation. Report against **CHEERS 2022**; make the "cost-effective" conclusion conditional on a stated willingness-to-pay threshold. Review-side probes: HE1–HE8 in `health_economic_evaluation.md`
|
|
467
|
+
|
|
459
468
|
### Survey/Likert
|
|
460
469
|
|
|
461
470
|
- Descriptive: median, IQR, frequency distribution per item
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Health Economic Evaluation Analysis Guide
|
|
2
|
+
|
|
3
|
+
Comparing the **costs and consequences** of two or more interventions to inform a coverage,
|
|
4
|
+
adoption, or treatment decision. The headline — an **incremental cost-effectiveness ratio (ICER)**
|
|
5
|
+
— is arithmetically trivial; what fails review is **the structural choices behind it** (perspective,
|
|
6
|
+
time horizon, discounting, the effectiveness source, the cost basis, the model, and the propagation
|
|
7
|
+
of uncertainty). So the analysis is mostly the costing, the model, and the sensitivity suite, not the
|
|
8
|
+
ratio. This is the analysis-side companion to review probes **HE1–HE8** in
|
|
9
|
+
`health_economic_evaluation.md` and the **CHEERS 2022** reporting checklist.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## When to Use
|
|
14
|
+
|
|
15
|
+
- A comparative decision question (adopt / reimburse / treat) where both **cost** and **health
|
|
16
|
+
outcome** differ between options: cost-effectiveness (CEA, natural units — life-years, events
|
|
17
|
+
avoided), **cost-utility** (CUA, QALYs — the default for reimbursement), cost-benefit (CBA,
|
|
18
|
+
monetised outcomes), cost-minimisation (only when outcomes are demonstrably equivalent), or
|
|
19
|
+
**budget-impact** (affordability, distinct from cost-effectiveness).
|
|
20
|
+
- **Trial-based** (within-RCT patient-level costs and outcomes) or **decision-model-based** (decision
|
|
21
|
+
tree for short horizons; **Markov / state-transition** or **discrete-event simulation** when timing
|
|
22
|
+
and recurrence matter; lifetime horizons).
|
|
23
|
+
- NOT for: a costing/burden-of-illness description with no comparator (not an economic *evaluation*);
|
|
24
|
+
asserting "cost-effective" from a point ICER with no uncertainty analysis; cost-minimisation when a
|
|
25
|
+
non-inferiority outcome claim has not actually been established.
|
|
26
|
+
|
|
27
|
+
## Core quantities
|
|
28
|
+
|
|
29
|
+
- **Incremental cost** ΔC and **incremental effect** ΔE between an intervention and its comparator;
|
|
30
|
+
the **ICER = ΔC / ΔE** (e.g. cost per QALY gained). With ≥3 options, rank by cost, remove
|
|
31
|
+
**dominated** (more costly, less effective) and **extended-dominated** strategies, then compute
|
|
32
|
+
ICERs sequentially along the efficient frontier.
|
|
33
|
+
- **Net benefit** at a willingness-to-pay threshold λ: **INMB = λ·ΔE − ΔC** (monetary) or
|
|
34
|
+
**INHB = ΔE − ΔC/λ** (health). Net benefit is linear and avoids the ICER's quadrant ambiguity, so
|
|
35
|
+
it is preferred for regression and for probabilistic summaries.
|
|
36
|
+
- **QALYs** = time × **utility** (preference-based, 0=dead, 1=full health) from a named instrument
|
|
37
|
+
(EQ-5D-3L/5L, SF-6D, HUI) and a stated **value set/tariff** for the relevant country.
|
|
38
|
+
|
|
39
|
+
## The structural choices (state and justify each)
|
|
40
|
+
|
|
41
|
+
- **Perspective** — healthcare-system/payer vs **societal**; determines which costs count (societal
|
|
42
|
+
adds productivity and informal-care costs). Apply it consistently.
|
|
43
|
+
- **Time horizon** — long enough to capture all differential costs and effects; **lifetime** for
|
|
44
|
+
chronic disease or interventions with lasting effects. Extrapolation beyond trial data must be
|
|
45
|
+
modelled explicitly (e.g. parametric survival extrapolation) and tested.
|
|
46
|
+
- **Discounting** — apply the jurisdiction's reference-case rate (commonly ~3% or 3.5%) to **both
|
|
47
|
+
costs and outcomes**; sensitivity at alternative rates.
|
|
48
|
+
- **Costing** — report **resource quantities and unit costs separately**; state **currency, price
|
|
49
|
+
year**, and inflation/currency conversion. Match cost categories to the perspective.
|
|
50
|
+
- **Model** — justify structure against natural history; state cycle length and half-cycle
|
|
51
|
+
correction (Markov); validate (internal/face/external/predictive); test **structural** uncertainty
|
|
52
|
+
via scenario analysis.
|
|
53
|
+
|
|
54
|
+
## Uncertainty (the analytic core)
|
|
55
|
+
|
|
56
|
+
- **Deterministic** — one-way and **tornado** diagrams to find the drivers; multi-way / scenario
|
|
57
|
+
analyses for methodological and structural choices (discount rate, time horizon, alternative
|
|
58
|
+
model structures).
|
|
59
|
+
- **Probabilistic (PSA)** — assign each uncertain parameter a **distribution** (beta for
|
|
60
|
+
probabilities and utilities; gamma or log-normal for costs; Dirichlet for transition-probability
|
|
61
|
+
sets), propagate by **Monte Carlo**, and report the **cost-effectiveness plane** (the cloud of ΔC,
|
|
62
|
+
ΔE draws) and the **cost-effectiveness acceptability curve (CEAC)** — P(cost-effective) across a
|
|
63
|
+
range of λ. Report results as net benefit at the relevant threshold with its uncertainty, not a
|
|
64
|
+
bare point ICER.
|
|
65
|
+
- **Value of information** (EVPI/EVPPI) is an optional extension quantifying the expected cost of
|
|
66
|
+
current decision uncertainty / the priority parameters for further research.
|
|
67
|
+
|
|
68
|
+
## Reporting & tools
|
|
69
|
+
|
|
70
|
+
- Report to **CHEERS 2022** (28 items): perspective, horizon, discount rate, currency/price year,
|
|
71
|
+
model rationale, study parameters with distributions, disaggregated costs/outcomes, the ICER,
|
|
72
|
+
and the uncertainty analysis (plane + CEAC). State the **willingness-to-pay threshold** and make
|
|
73
|
+
the "cost-effective" conclusion conditional on it and on the CEAC probability.
|
|
74
|
+
- Tools: R — `heemod` / `dampack` / `hesim` (state-transition + PSA + CEAC), `BCEA` (Bayesian
|
|
75
|
+
cost-effectiveness, CEAC/EVPI), `survival`/`flexsurv` (survival extrapolation); also TreeAge, or
|
|
76
|
+
spreadsheet models with a documented PSA. Make the model or the health-economic analysis plan
|
|
77
|
+
available for scrutiny.
|
|
78
|
+
- Companion review probes: **HE1–HE8** (`peer-review`/`self-review`
|
|
79
|
+
`references/domain-probes/health_economic_evaluation.md`).
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: check-reporting
|
|
3
|
-
description: Check manuscript compliance with medical research reporting guidelines. Supports
|
|
4
|
-
triggers: checklist, reporting guideline, STROBE, STROBE-MR, Mendelian randomization, CONSORT, CONSORT-AI, STARD, STARD-AI, TRIPOD, TRIPOD-LLM, PGS-RS, PRS-RS, polygenic risk score, polygenic score, PRISMA, PRISMA-DTA, PRISMA-P, ARRIVE, CARE, CLAIM, DECIDE-AI, MI-CLEAR-LLM, SPIRIT, SPIRIT-AI, QUADAS, QUADAS-C, RoB, ROBINS, ROBINS-E, ROBIS, ROB-ME, PROBAST, NOS, COSMIN, AMSTAR, SWiM, risk of bias, compliance check, LLM accuracy, large language model, clinical deployment
|
|
3
|
+
description: Check manuscript compliance with medical research reporting guidelines. Supports 40 guidelines including STROBE, STROBE-MR, RECORD, CONSORT, CONSORT-AI, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, ARRIVE, PRISMA, PRISMA-DTA, PRISMA-P, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, MI-CLEAR-LLM, SQUIRE 2.0, CLEAR, MOOSE, GRRAS, SWiM, AMSTAR 2, CHEERS 2022, and risk of bias tools (QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA). Generates item-by-item assessment with PRESENT/MISSING/PARTIAL status.
|
|
4
|
+
triggers: checklist, reporting guideline, STROBE, STROBE-MR, Mendelian randomization, CONSORT, CONSORT-AI, STARD, STARD-AI, TRIPOD, TRIPOD-LLM, PGS-RS, PRS-RS, polygenic risk score, polygenic score, PRISMA, PRISMA-DTA, PRISMA-P, ARRIVE, CARE, CLAIM, DECIDE-AI, MI-CLEAR-LLM, SPIRIT, SPIRIT-AI, QUADAS, QUADAS-C, RoB, ROBINS, ROBINS-E, ROBIS, ROB-ME, PROBAST, NOS, COSMIN, AMSTAR, SWiM, CHEERS, economic evaluation, cost-effectiveness, cost-utility, QALY, ICER, RECORD, RECORD-PE, routinely-collected data, registry, claims, electronic health records, EHR, real-world data, risk of bias, compliance check, LLM accuracy, large language model, clinical deployment
|
|
5
5
|
tools: Read, Write, Edit, Bash, Grep, Glob
|
|
6
6
|
model: inherit
|
|
7
7
|
---
|
|
@@ -29,6 +29,8 @@ compliance report suitable for journal submission.
|
|
|
29
29
|
- `TRIPOD_AI.md` -- prediction models with AI/ML (CC BY 4.0, Collins et al. BMJ 2024)
|
|
30
30
|
- `TRIPOD_LLM.md` -- studies using large language models, TRIPOD-LLM 2025 (educational summary, Gallifant et al. Nat Med 2025)
|
|
31
31
|
- `PGS_RS.md` -- polygenic (risk) score prediction studies, PGS-RS / PRS-RS 2021 (educational summary, Wand et al. Nature 2021)
|
|
32
|
+
- `CHEERS_2022.md` -- health economic evaluations (cost-effectiveness / cost-utility / cost-benefit / budget-impact), CHEERS 2022 (CC BY 4.0, Husereau et al. BMJ 2022)
|
|
33
|
+
- `RECORD.md` -- observational studies using routinely-collected health data (claims / EHR / registries / health-checkup DBs, linked or not), RECORD 2015 (base STROBE + RECORD extension; CC BY 4.0, Benchimol et al. PLoS Med 2015; RECORD-PE for drug studies)
|
|
32
34
|
- `PRISMA_2020.md` -- systematic reviews (CC BY)
|
|
33
35
|
- `ARRIVE_2.md` -- animal studies (CC0)
|
|
34
36
|
- `PRISMA_DTA.md` -- DTA systematic reviews (CC BY, McInnes et al. JAMA 2018)
|
|
@@ -88,6 +90,8 @@ user specification.
|
|
|
88
90
|
|------------|------------------|--------------|
|
|
89
91
|
| Observational study | STROBE | -- |
|
|
90
92
|
| Mendelian randomization study | STROBE-MR (base STROBE + MR extension) | -- |
|
|
93
|
+
| Health economic evaluation (cost-effectiveness / cost-utility / cost-benefit / budget-impact) | CHEERS 2022 | -- |
|
|
94
|
+
| Observational study using routinely-collected data (claims / EHR / registry / health-checkup DB) | RECORD (base STROBE + RECORD extension; RECORD-PE for drug studies) | -- |
|
|
91
95
|
| Randomized controlled trial | CONSORT 2025 | CONSORT-AI |
|
|
92
96
|
| Diagnostic accuracy study | STARD 2015 | STARD-AI |
|
|
93
97
|
| Prediction model (development/validation) | TRIPOD | TRIPOD+AI |
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# CHEERS 2022 Checklist
|
|
2
|
+
|
|
3
|
+
**Consolidated Health Economic Evaluation Reporting Standards 2022**
|
|
4
|
+
Version: CHEERS 2022 (28 items; replaces CHEERS 2013).
|
|
5
|
+
Source: Husereau D, Drummond M, Augustovski F, et al. *BMJ* 2022;376:e067975 (the CHEERS 2022 statement), co-published simultaneously across BMJ, *Value in Health*, *PharmacoEconomics*, *Int J Technol Assess Health Care* and others. CC BY 4.0. https://www.equator-network.org/reporting-guidelines/cheers/ · ISPOR CHEERS Task Force.
|
|
6
|
+
|
|
7
|
+
Apply when the manuscript is a **health economic evaluation** — a comparative analysis of costs and consequences of two or more courses of action: cost-effectiveness (CEA), cost-utility (CUA), cost-benefit (CBA), or cost-minimisation analysis, whether trial-based or decision-model-based (decision tree, Markov/state-transition, discrete-event simulation), including budget-impact and HTA submissions. For the design/validity review of the same study, pair with the HE1–HE8 domain probes in `peer-review` / `self-review` `references/domain-probes/health_economic_evaluation.md`; for the analysis, with `analyze-stats` `references/analysis_guides/health_economic_evaluation.md`.
|
|
8
|
+
|
|
9
|
+
## Checklist Items (28 items)
|
|
10
|
+
|
|
11
|
+
### Title
|
|
12
|
+
|
|
13
|
+
| # | Item | Description |
|
|
14
|
+
|---|------|-------------|
|
|
15
|
+
| 1 | Title | Identify the study as an economic evaluation and specify the interventions being compared. |
|
|
16
|
+
|
|
17
|
+
### Abstract
|
|
18
|
+
|
|
19
|
+
| # | Item | Description |
|
|
20
|
+
|---|------|-------------|
|
|
21
|
+
| 2 | Abstract | Provide a structured summary that highlights context, key methods, results, and alternative analyses. |
|
|
22
|
+
|
|
23
|
+
### Introduction
|
|
24
|
+
|
|
25
|
+
| # | Item | Description |
|
|
26
|
+
|---|------|-------------|
|
|
27
|
+
| 3 | Background and objectives | Give the context for the study, the study question, and its practical relevance for decision making in policy or practice. |
|
|
28
|
+
|
|
29
|
+
### Methods
|
|
30
|
+
|
|
31
|
+
| # | Item | Description |
|
|
32
|
+
|---|------|-------------|
|
|
33
|
+
| 4 | Health economic analysis plan | Indicate whether a health economic analysis plan was developed and where available. |
|
|
34
|
+
| 5 | Study population | Describe characteristics of the study population (such as age range, demographics, socioeconomic, or clinical characteristics). |
|
|
35
|
+
| 6 | Setting and location | Provide relevant contextual information that may influence findings. |
|
|
36
|
+
| 7 | Comparators | Describe the interventions or strategies being compared and why chosen. |
|
|
37
|
+
| 8 | Perspective | State the perspective(s) adopted by the study and why chosen. |
|
|
38
|
+
| 9 | Time horizon | State the time horizon for the study and why appropriate. |
|
|
39
|
+
| 10 | Discount rate | Report the discount rate(s) and reason chosen. |
|
|
40
|
+
| 11 | Selection of outcomes | Describe what outcomes were used as the measure(s) of benefit(s) and harm(s). |
|
|
41
|
+
| 12 | Measurement of outcomes | Describe how outcomes used to capture benefit(s) and harm(s) were measured. |
|
|
42
|
+
| 13 | Valuation of outcomes | Describe the population and methods used to measure and value outcomes. |
|
|
43
|
+
| 14 | Measurement and valuation of resources and costs | Describe how costs were valued. |
|
|
44
|
+
| 15 | Currency, price date, and conversion | Report the dates of the estimated resource quantities and unit costs, plus the currency and year of conversion. |
|
|
45
|
+
| 16 | Rationale and description of model | If modelling is used, describe in detail and why used. Report whether the model is publicly available and where. |
|
|
46
|
+
| 17 | Analytics and assumptions | Describe any methods for analysing or statistically transforming data, any extrapolation methods, and approaches for validating any model used. |
|
|
47
|
+
| 18 | Characterising heterogeneity | Describe any methods used for estimating how the results of the study vary for subgroups. |
|
|
48
|
+
| 19 | Characterising distributional effects | Describe how impacts are distributed across different individuals or whether adjustments were made to reflect priority populations. |
|
|
49
|
+
| 20 | Characterising uncertainty | Describe methods to characterise any sources of uncertainty in the analysis. |
|
|
50
|
+
| 21 | Approach to engagement with patients and others affected by the study | Describe any approaches to engage patients or service recipients, the general public, communities, or stakeholders (such as clinicians or payers) in the design of the study. |
|
|
51
|
+
|
|
52
|
+
### Results
|
|
53
|
+
|
|
54
|
+
| # | Item | Description |
|
|
55
|
+
|---|------|-------------|
|
|
56
|
+
| 22 | Study parameters | Report all analytic inputs (such as values, ranges, references) including uncertainty or distributional assumptions. |
|
|
57
|
+
| 23 | Summary of main results | Report the mean values for the main categories of costs and outcomes of interest and summarise them in the most appropriate overall measure (e.g. the incremental cost-effectiveness ratio, ICER). |
|
|
58
|
+
| 24 | Effect of uncertainty | Describe how uncertainty about analytic judgments, inputs, or projections affect findings. Report the effect of choice of discount rate and time horizon, if relevant. |
|
|
59
|
+
| 25 | Effect of engagement with patients and others affected by the study | Report on any difference patient/service recipient, general public, community, or stakeholder involvement made to the approach or findings of the study. |
|
|
60
|
+
|
|
61
|
+
### Discussion
|
|
62
|
+
|
|
63
|
+
| # | Item | Description |
|
|
64
|
+
|---|------|-------------|
|
|
65
|
+
| 26 | Study findings, limitations, generalisability, and current knowledge | Report key findings, limitations, ethical or equity considerations, and how these could affect patients, policy, or practice. |
|
|
66
|
+
|
|
67
|
+
### Other relevant information
|
|
68
|
+
|
|
69
|
+
| # | Item | Description |
|
|
70
|
+
|---|------|-------------|
|
|
71
|
+
| 27 | Source of funding | Describe how the study was funded and the role of the funder in the identification, design, conduct, and reporting of the analysis. Describe other non-monetary sources of support. |
|
|
72
|
+
| 28 | Conflicts of interest | Describe any potential for conflict of interest among study contributors in accordance with journal policy. In the absence of a journal policy, we recommend authors comply with International Committee of Medical Journal Editors (ICMJE) recommendations. |
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Notes for Assessors
|
|
77
|
+
|
|
78
|
+
- **Highest-yield items** (where economic evaluations most often fail review): **8** (perspective stated and consistent with the costs counted — productivity/informal-care costs belong only to a societal perspective), **9** (time horizon long enough to capture all relevant differential costs and effects — a lifetime horizon for a chronic condition; a truncated horizon flatters whichever arm has early benefit), **10** (both costs *and* outcomes discounted at a stated, justified rate for any horizon beyond ~1 year), **15** (currency *and* price year stated, with the conversion method for multi-source costs), **16–17** (model type/structure justified and validated; structural assumptions and extrapolation declared), and **20 / 24** (uncertainty characterised by *probabilistic* sensitivity analysis — a cost-effectiveness acceptability curve / plane — not a single deterministic ICER). An evaluation that reports a point-estimate ICER with no probabilistic sensitivity analysis is non-compliant on items 20/24.
|
|
79
|
+
- **CHEERS 2022 replaced CHEERS 2013**; cite the 2022 statement (do not cite the 2013 version as current). CHEERS 2022 added explicit items on the analysis plan (4), distributional/equity effects (19), and patient/stakeholder engagement (21/25); the engagement items are reported as "not done" rather than omitted when no engagement occurred.
|
|
80
|
+
- ICER interpretation is not a CHEERS item per se but follows from items 23–24: incremental costs and effects must be reported (not just the ratio), dominance/extended dominance resolved, and the ICER interpreted against a *stated, justified* cost-effectiveness threshold (willingness-to-pay) rather than an arbitrary one. The HE1–HE8 design probes cover these judgments.
|
|
81
|
+
- This checklist was authored as a faithful summary of the CHEERS 2022 statement (Husereau D, et al. *BMJ* 2022;376:e067975, **CC BY 4.0** — the item list and Explanation & Elaboration are reusable/adaptable with attribution) for item-by-item assessment; verify against the published statement and its Explanation & Elaboration for full item wording. Verified 2026-06-29.
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# RECORD Checklist
|
|
2
|
+
|
|
3
|
+
**REporting of studies Conducted using Observational Routinely-collected health Data**
|
|
4
|
+
Version: RECORD 2015 (13 items; a STROBE extension). The pharmacoepidemiology extension is **RECORD-PE** (Langan et al. *BMJ* 2018).
|
|
5
|
+
Source: Benchimol EI, Smeeth L, Guttmann A, et al. *PLoS Medicine* 2015;12(10):e1001885 (the RECORD statement). CC BY 4.0. https://www.record-statement.org/ · EQUATOR Network.
|
|
6
|
+
|
|
7
|
+
Apply when the manuscript is an **observational study conducted using routinely-collected health data** — administrative claims, electronic health records (EHR), disease/population registries, health-administrative or health-checkup databases, or linked versions of these — i.e. data **not collected for the purpose of the specific study**. RECORD extends the base **STROBE** items with reporting specific to secondary-use data: database identity, the codes/algorithms used to define the population and the variables, data linkage and its quality, and the limitations of analysing data collected for another purpose. For a drug safety/effectiveness study in such data, also apply **RECORD-PE**. For the design/validity review of the same study, pair with the RD1–RD8 domain probes in `peer-review` / `self-review` `references/domain-probes/record_routinely_collected_data.md`, and with the observational-confounding probes (`observational_confounding.md`).
|
|
8
|
+
|
|
9
|
+
## Checklist Items (13 items, extending STROBE)
|
|
10
|
+
|
|
11
|
+
### Title and Abstract (STROBE item 1)
|
|
12
|
+
|
|
13
|
+
| # | Item | Description |
|
|
14
|
+
|---|------|-------------|
|
|
15
|
+
| 1.1 | Data type | The type of data used should be specified in the title or abstract. When possible, the name(s) of the database(s) used should be stated. |
|
|
16
|
+
| 1.2 | Geography and timeframe | The geographic region and timeframe within which the study took place should be reported in the title or abstract. |
|
|
17
|
+
| 1.3 | Linkage | If linkage between databases was conducted for the study, this should be clearly stated in the title or abstract. |
|
|
18
|
+
|
|
19
|
+
### Methods — Setting / Participants (STROBE item 6)
|
|
20
|
+
|
|
21
|
+
| # | Item | Description |
|
|
22
|
+
|---|------|-------------|
|
|
23
|
+
| 6.1 | Population selection | The methods of study population selection (such as codes or algorithms used to identify subjects) should be listed in detail. If this is not possible, an explanation should be provided. |
|
|
24
|
+
| 6.2 | Validation of codes | Any validation studies of the codes or algorithms used to select the population should be referenced. If validation was conducted for this study and not published elsewhere, detailed methods and results should be provided. |
|
|
25
|
+
| 6.3 | Linkage diagram | If the study involved linkage of databases, consider use of a flow diagram or other graphical display to demonstrate the data linkage process, including the number of individuals with linked data at each stage. |
|
|
26
|
+
|
|
27
|
+
### Methods — Variables (STROBE item 7)
|
|
28
|
+
|
|
29
|
+
| # | Item | Description |
|
|
30
|
+
|---|------|-------------|
|
|
31
|
+
| 7.1 | Codes for variables | A complete list of codes and algorithms used to classify exposures, outcomes, confounders, and effect modifiers should be provided. If these cannot be reported, an explanation should be provided. |
|
|
32
|
+
|
|
33
|
+
### Methods — Data access and cleaning (STROBE item 12)
|
|
34
|
+
|
|
35
|
+
| # | Item | Description |
|
|
36
|
+
|---|------|-------------|
|
|
37
|
+
| 12.1 | Data access | Authors should describe the extent to which the investigators had access to the database population used to create the study population. |
|
|
38
|
+
| 12.2 | Data cleaning | Authors should provide information on the data cleaning methods used in the study. |
|
|
39
|
+
| 12.3 | Linkage methods | State whether the study included person-level, institutional-level, or other data linkage across two or more databases. The methods of linkage and methods of linkage quality evaluation should be provided. |
|
|
40
|
+
|
|
41
|
+
### Results — Participants (STROBE item 13)
|
|
42
|
+
|
|
43
|
+
| # | Item | Description |
|
|
44
|
+
|---|------|-------------|
|
|
45
|
+
| 13.1 | Selection of included persons | Describe in detail the selection of the persons included in the study (i.e. study population selection) including filtering based on data quality, data availability and linkage. The selection of included persons can be described in the text and/or by means of the study flow diagram. |
|
|
46
|
+
|
|
47
|
+
### Discussion — Limitations (STROBE item 19)
|
|
48
|
+
|
|
49
|
+
| # | Item | Description |
|
|
50
|
+
|---|------|-------------|
|
|
51
|
+
| 19.1 | Secondary-data limitations | Discuss the implications of using data that were not created or collected to answer the specific research question(s). Include discussion of misclassification bias, unmeasured confounding, missing data, and changing eligibility over time, as they pertain to the study being reported. |
|
|
52
|
+
|
|
53
|
+
### Other Information — Data access / cleaning (STROBE item 22)
|
|
54
|
+
|
|
55
|
+
| # | Item | Description |
|
|
56
|
+
|---|------|-------------|
|
|
57
|
+
| 22.1 | Supplemental access | Authors should provide information on how to access any supplemental information such as the study protocol, raw data, or programming code. |
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Notes for Assessors
|
|
62
|
+
|
|
63
|
+
- RECORD is an **extension of STROBE**; for the non-RECORD-specific items the base `STROBE.md` guidance also applies. Report both the base instrument and the extension when describing methods (do not cite RECORD as if it replaced STROBE).
|
|
64
|
+
- The **highest-yield** items are **6.1 / 7.1** (the actual code lists / phenotype algorithms used to define the population, exposures, outcomes, and confounders — the single most common omission; "we identified diabetes from the database" with no codes is non-compliant), **6.2** (whether those algorithms were *validated*, and where), **12.3 / 6.3** (linkage method and linkage-quality evaluation, with a person-flow at each linkage stage), **13.1** (a participant-selection flow that includes data-quality/availability/linkage filtering — not only clinical eligibility), and **19.1** (the limitations specific to secondary-use data: misclassification from codes, unmeasured confounding, informative missingness, and eligibility drift over time).
|
|
65
|
+
- For a **drug safety/effectiveness** study in routinely-collected data, also apply **RECORD-PE** (Langan et al. *BMJ* 2018;363:k3532), which adds items on exposure definition (drug codes, dose, duration, exposure windows), the comparator and new-user/active-comparator design, and immortal-time/protopathic bias.
|
|
66
|
+
- This checklist was authored as a faithful summary of the RECORD statement (Benchimol EI, et al. *PLoS Med* 2015;12(10):e1001885, **CC BY 4.0**) for item-by-item assessment; verify against the published statement and its explanation-and-elaboration document for full item wording. Verified 2026-06-29.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Reporting Guideline → Figure Requirements Map
|
|
2
2
|
|
|
3
3
|
> **Bridge**: this file connects `/make-figures` to `/check-reporting`
|
|
4
|
-
> (
|
|
4
|
+
> (40 reporting guidelines). Each row tells you which figures the guideline
|
|
5
5
|
> **mandates** and how this skill currently supports them. Use during
|
|
6
6
|
> Step 1 (Specify) once the study type is known.
|
|
7
7
|
|
|
@@ -95,7 +95,7 @@ flags them:
|
|
|
95
95
|
|
|
96
96
|
## Cross-references
|
|
97
97
|
|
|
98
|
-
- `/check-reporting` skill — supports all
|
|
98
|
+
- `/check-reporting` skill — supports all 40 guidelines, item-level audit
|
|
99
99
|
- `flow_diagram_lessons.md` — production lessons that apply across all flows
|
|
100
100
|
- `pipeline_concepts_medical_ai.md` — DICOM / annotation / federated /
|
|
101
101
|
architecture diagram conventions
|
|
@@ -48,7 +48,7 @@ You do NOT do the work yourself. You classify, plan, and delegate.
|
|
|
48
48
|
| **meta-analysis** | Systematic review | Full MA pipeline: protocol, search, screening, extraction, synthesis, PRISMA-DTA |
|
|
49
49
|
| **write-paper** | Writing | IMRAD manuscript drafting (8-phase pipeline), any section writing |
|
|
50
50
|
| **self-review** | Quality | Pre-submission self-check with domain probes (Survival / SR-MA / Radiomics / Narrative); optional `--panel` for a high-stakes final QC pass |
|
|
51
|
-
| **check-reporting** | Compliance | Audit against
|
|
51
|
+
| **check-reporting** | Compliance | Audit against 40 reporting guidelines and risk-of-bias tools |
|
|
52
52
|
| **revise** | Revision | Parse reviewer comments, generate point-by-point response, track changes |
|
|
53
53
|
| **grant-builder** | Funding | Structure grant proposals: significance, innovation, approach, milestones |
|
|
54
54
|
| **present-paper** | Presentation | Prepare academic talks: analyze paper, draft scripts, inject slide notes, Q&A prep |
|
|
@@ -268,6 +268,18 @@ Apply this 8-probe checklist (NM1–NM8) **only when the manuscript is a network
|
|
|
268
268
|
|
|
269
269
|
**Probe detail (NM1–NM8), with output templates and the leads-vs-findings discipline:** `${CLAUDE_SKILL_DIR}/references/domain-probes/network_meta_analysis.md`. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to a Major / Minor comment; an unassessed/violated transitivity assumption behind an indirect conclusion (NM1), unexamined incoherence or an unacknowledged star network presented as validated (NM2), a "best treatment" headline driven by SUCRA/P-score without the paired effect size + certainty (NM4), or a conclusion ignoring low CINeMA/GRADE certainty (NM6) are design-level — surface them in the Confidential Comments to the Editor and place the strongest as the Major #1 candidate. A single-study-edge headline (NM3), unexplored network heterogeneity (NM5), a comparison-naive publication-bias claim (NM7), and an unstated component-NMA additivity assumption / over-stated estimand (NM8) are validity/framing-level. Run the pairwise machinery (heterogeneity model, study-count thresholds) via `sr_ma.md`.
|
|
270
270
|
|
|
271
|
+
### Phase 2P: Health Economic Evaluation Extension
|
|
272
|
+
|
|
273
|
+
Apply this 8-probe checklist (HE1–HE8) **only when the manuscript is a health economic evaluation** — a comparative analysis of costs and consequences (cost-effectiveness, cost-utility/QALY, cost-benefit, cost-minimisation, budget-impact/HTA), whether trial-based or decision-model-based (decision tree, Markov, discrete-event simulation). These probes complement (do not replace) the generic Phase 2 issue checklist and the CHEERS 2022 reporting items; they target the structural choices behind the headline ICER — perspective, time horizon, discounting, the effectiveness source, the cost basis, the model, and the propagation of uncertainty.
|
|
274
|
+
|
|
275
|
+
**Probe detail (HE1–HE8), with output templates and the leads-vs-findings discipline:** `${CLAUDE_SKILL_DIR}/references/domain-probes/health_economic_evaluation.md`. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to a Major / Minor comment; a missing/obsolete comparator or perspective inconsistent with the costs counted (HE1), a time horizon truncated below the point where costs and effects diverge or asymmetric/absent discounting (HE2), an unjustified/unvalidated model structure (HE5), a point-estimate ICER with no probabilistic sensitivity analysis / CEAC (HE6), or a "cost-effective" claim with no stated willingness-to-pay threshold and mishandled dominance (HE7) are design-level — surface them in the Confidential Comments to the Editor and place the strongest as the Major #1 candidate. A weak effectiveness source / unstated utility instrument (HE3), perspective-inconsistent costs or a missing price year (HE4), and an undisclosed industry-funder role on a threshold-hugging result (HE8) are validity/framing-level.
|
|
276
|
+
|
|
277
|
+
### Phase 2Q: Routinely-Collected-Data (RWD) Extension
|
|
278
|
+
|
|
279
|
+
Apply this 8-probe checklist (RD1–RD8) **only when the manuscript is an observational study conducted using routinely-collected health data** — administrative claims, electronic health records (EHR), disease/population registries, or health-administrative / health-checkup databases, linked or not. These probes complement (do not replace) the generic Phase 2 checklist, the STROBE + **RECORD** reporting items (**RECORD-PE** for drug studies), and the observational-confounding probes (`observational_confounding.md`). They target what secondary-use data add: whether the database can observe the question, whether phenotype code-lists and linkage are evidenced rather than asserted, and whether data-collected-for-another-purpose limitations are confronted.
|
|
280
|
+
|
|
281
|
+
**Probe detail (RD1–RD8), with output templates and the leads-vs-findings discipline:** `${CLAUDE_SKILL_DIR}/references/domain-probes/record_routinely_collected_data.md`. Load it and apply each probe when the trigger fires. In this skill, map each probe finding to a Major / Minor comment; missing phenotype code-lists / unvalidated algorithms for the population, exposure or outcome (RD2), undisclosed linkage method or linkage-quality evaluation (RD3), a source→analytic selection with no data-quality/availability/linkage flow (RD4), naive complete-case on informatively-missing fields (RD6), or an RWD drug-effect design exposed to immortal-time / prevalent-user bias with no mitigation (RD7) are design-level — surface them in the Confidential Comments to the Editor and place the strongest as the Major #1 candidate. A database that structurally cannot capture the exposure/outcome (RD1), unquantified coding misclassification (RD5), and unacknowledged coding/eligibility drift or no code/protocol availability (RD8) are validity/framing-level. Run the adjustment/collider/analysis-unit machinery via `observational_confounding.md`.
|
|
282
|
+
|
|
271
283
|
### Phase 3: Draft Review
|
|
272
284
|
|
|
273
285
|
Before writing comments, skim the relevant model in `references/exemplar_reviews/` for the
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review.
|
|
2
|
+
Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal
|
|
3
|
+
recommendation. Each consuming skill maps findings to its own output:
|
|
4
|
+
- peer-review: Major / Minor comments + Confidential Comments to the Editor; a
|
|
5
|
+
perspective/time-horizon/uncertainty/ICER-interpretation flaw is placed as Major #1.
|
|
6
|
+
- self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.
|
|
7
|
+
Do NOT edit one copy only — run `python3 scripts/check_domain_probe_sync.py --sync`. -->
|
|
8
|
+
|
|
9
|
+
# Health economic evaluation probes (HE1–HE8)
|
|
10
|
+
|
|
11
|
+
An 8-probe checklist for **health economic evaluations** — comparative analyses of the costs and consequences of two or more options: cost-effectiveness (CEA), cost-utility (CUA, QALY-based), cost-benefit (CBA), cost-minimisation, and budget-impact/HTA analyses, whether **trial-based** or **decision-model-based** (decision tree, Markov/state-transition, discrete-event simulation). These probes complement (do not replace) the generic Phase 2 checklist and the **CHEERS 2022** reporting items; they target the gap between a clean-looking incremental cost-effectiveness ratio (ICER) and whether the structural choices that produced it — perspective, time horizon, discounting, the effectiveness source, the cost basis, the model, and the handling of uncertainty — are defensible. Because the headline number is a single ratio that a reader cannot reproduce from the manuscript, the burden is on the study to *evidence* each structural choice, not assert it. HE2 (time horizon & discounting), HE5 (model structure), and HE6 (uncertainty / probabilistic sensitivity analysis) are the highest-yield; run them first.
|
|
12
|
+
|
|
13
|
+
**HE1 — Decision problem, comparators, and perspective**:
|
|
14
|
+
- Is the decision/policy question explicit, and are **all relevant comparators** included — the current standard of care and, where appropriate, a do-nothing/usual-care arm — rather than a convenient or obsolete comparator that flatters the new intervention? Omitting the true standard of care is the most common way to manufacture a favourable ICER.
|
|
15
|
+
- Is the **analytic perspective** stated (healthcare-system / payer vs **societal**) and justified, and is it applied consistently to what is counted (HE4)? A "societal" claim that omits productivity and informal-care costs — or a payer analysis that smuggles in societal costs — is an internal inconsistency.
|
|
16
|
+
- A missing/obsolete comparator, an unstated perspective, or a perspective inconsistent with the costs counted → MAJOR.
|
|
17
|
+
|
|
18
|
+
**HE2 — Time horizon and discounting**:
|
|
19
|
+
- Is the **time horizon** long enough to capture all material differential costs and health effects? For a chronic disease or a one-off intervention with lifelong consequences, a **lifetime horizon** is usually required; a horizon truncated to trial follow-up flatters whichever arm has earlier benefit and hides downstream costs.
|
|
20
|
+
- Are **both costs and outcomes discounted** at a stated, justified rate (with the jurisdiction's reference-case rate used and a sensitivity at alternative rates)? Discounting only costs, or not discounting a multi-year horizon at all, biases the ICER.
|
|
21
|
+
- A truncated horizon with no extrapolation justification, undiscounted long-horizon results, or asymmetric discounting → MAJOR.
|
|
22
|
+
|
|
23
|
+
**HE3 — Effectiveness source and outcome valuation**:
|
|
24
|
+
- Where do the **effectiveness/relative-effect estimates** come from — a head-to-head RCT or meta-analysis, or a weaker source (single-arm, naive indirect comparison, observational) — and is that source appropriate and its uncertainty carried into the model (HE6)? A model driven by an optimistic point estimate from a weak source is not rescued by elegant modelling.
|
|
25
|
+
- For a **cost-utility analysis**, are **QALYs** built from a **named preference-based instrument** (e.g. EQ-5D-5L) and a stated **value set / tariff** for the relevant population, rather than ad-hoc or mapped utilities with the mapping undisclosed?
|
|
26
|
+
- A weak/again-mismatched effectiveness source, or QALYs with no stated utility instrument/value set → MAJOR (or a prominent limitation if the source is the best available and is treated as uncertain).
|
|
27
|
+
|
|
28
|
+
**HE4 — Costs, resource use, currency, and price year**:
|
|
29
|
+
- Are **resource quantities and unit costs reported separately** (so a reader can re-value for another setting), and are the cost categories **consistent with the stated perspective** (HE1)?
|
|
30
|
+
- Are the **currency, price (cost) year, and any inflation/currency-conversion method** stated? Costs pooled across years or countries without a stated price year and conversion are not interpretable or transferable.
|
|
31
|
+
- Missing price year, perspective-inconsistent cost categories, or undisclosed resource-vs-unit-cost bundling → MAJOR.
|
|
32
|
+
|
|
33
|
+
**HE5 — Model structure, assumptions, and validation**:
|
|
34
|
+
- If **model-based**, is the model **type and structure** (states/events, cycle length, half-cycle correction for a Markov model) described and **justified** against the disease's natural history, rather than chosen for convenience? Is **structural uncertainty** acknowledged (alternative plausible structures), and was the model **validated** (internal consistency, and external/predictive validation where data exist)?
|
|
35
|
+
- Are key **structural assumptions and extrapolation** beyond the data (e.g. survival curve extrapolation, treatment-effect waning) declared and tested?
|
|
36
|
+
- An unjustified structure, undeclared extrapolation, or no validation/structural-sensitivity → MAJOR (the structure can drive the ICER more than any single parameter).
|
|
37
|
+
|
|
38
|
+
**HE6 — Uncertainty: deterministic AND probabilistic**:
|
|
39
|
+
- Is **parameter uncertainty** characterised by a **probabilistic sensitivity analysis (PSA)** — every uncertain input assigned a distribution and propagated by Monte Carlo — and reported as a **cost-effectiveness plane** and **cost-effectiveness acceptability curve (CEAC)**, not just a deterministic point ICER? Are **one-way / tornado** analyses used to identify the drivers, and **scenario analyses** for non-parametric (structural/methodological) uncertainty?
|
|
40
|
+
- Are the parameter **distributions justified** (beta for probabilities/utilities, gamma/log-normal for costs) rather than arbitrary ±20% ranges presented as if probabilistic?
|
|
41
|
+
- A single deterministic ICER with **no PSA**, or "sensitivity analysis" that is one-way only with no probabilistic component → MAJOR.
|
|
42
|
+
|
|
43
|
+
**HE7 — Results presentation and ICER interpretation**:
|
|
44
|
+
- Are **incremental costs and incremental effects reported separately** (and ideally a disaggregated cost/outcome table), not only the ratio? Are **dominance and extended dominance** correctly identified and dominated strategies excluded before computing ICERs?
|
|
45
|
+
- Is the ICER interpreted against a **stated, justified cost-effectiveness threshold (willingness-to-pay)** appropriate to the jurisdiction — not an arbitrary or post-hoc threshold — and is the conclusion ("cost-effective") conditional on that threshold and the uncertainty (CEAC probability), rather than asserted from the point estimate alone? Net monetary/health benefit (INMB/INHB) is a clearer alternative at a given threshold.
|
|
46
|
+
- An ICER reported without incrementals, mishandled dominance, or a "cost-effective" claim with no threshold context or ignoring the CEAC → MAJOR.
|
|
47
|
+
|
|
48
|
+
**HE8 — Equity, generalisability, funding/COI, and CHEERS reporting**:
|
|
49
|
+
- Are **distributional/equity effects** considered where relevant (who bears the costs and gains the health), and is **generalisability/transferability** to other settings or jurisdictions discussed (cost and epidemiology transfer, not just clinical effect)?
|
|
50
|
+
- Are **funding source and the funder's role** and **conflicts of interest** disclosed? An industry-funded evaluation whose ICER sits just below the threshold and is insensitive to plausible assumptions warrants heightened scrutiny of the structural choices (HE1–HE5).
|
|
51
|
+
- Is reporting mapped to **CHEERS 2022**, and is the model / health-economic analysis plan available for scrutiny?
|
|
52
|
+
- No generalisability discussion for a single-jurisdiction model presented as broadly applicable, or undisclosed funder role on a threshold-hugging result → MAJOR (funding/COI) / MINOR (generalisability), per centrality.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review.
|
|
2
|
+
Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal
|
|
3
|
+
recommendation. Each consuming skill maps findings to its own output:
|
|
4
|
+
- peer-review: Major / Minor comments + Confidential Comments to the Editor; a
|
|
5
|
+
code-list / linkage-quality / selection-flow / RWD-bias design flaw is placed as Major #1.
|
|
6
|
+
- self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.
|
|
7
|
+
Do NOT edit one copy only — run `python3 scripts/check_domain_probe_sync.py --sync`. -->
|
|
8
|
+
|
|
9
|
+
# Routinely-collected-data study probes (RD1–RD8)
|
|
10
|
+
|
|
11
|
+
An 8-probe checklist for observational studies conducted using **routinely-collected health data** — administrative claims, electronic health records (EHR), disease/population registries, health-administrative and health-checkup databases, and linked versions of these (data **not collected for the study's purpose**). These probes complement (do not replace) the generic Phase 2 checklist, the **STROBE** + **RECORD** reporting items (and **RECORD-PE** for drug studies), and the observational-confounding probes (`observational_confounding.md`, which cover adjustment/collider/analysis-unit issues). They target what secondary-use data add: whether the database can even observe the question, whether the phenotype code-lists and linkage are evidenced rather than asserted, and whether the limitations endemic to data collected for another purpose are confronted. RD2 (phenotype code-lists & validation), RD3 (linkage quality), and RD4 (participant-selection flow) are the highest-yield; run them first.
|
|
12
|
+
|
|
13
|
+
**RD1 — Database identity, provenance, and fitness-for-purpose**:
|
|
14
|
+
- Is the **database named** and its **type** (claims / EHR / registry / health-checkup), provenance, coverage population, and capture window described — and can it **structurally observe** the exposure and outcome? A claims database cannot see out-of-network or cash-pay care; an EHR cannot see care at other systems; neither reliably sees OTC drugs, over-the-counter outcomes, or death out of hospital.
|
|
15
|
+
- Is the **timeframe and geographic setting** stated (title/abstract per RECORD 1.1–1.2)?
|
|
16
|
+
- An unnamed/undescribed database, or one whose structure cannot capture the named exposure/outcome (so the measure is systematically incomplete), → MAJOR.
|
|
17
|
+
|
|
18
|
+
**RD2 — Phenotype definitions: code-lists and algorithms, evidenced not asserted**:
|
|
19
|
+
- Are the **codes / algorithms** (ICD-9/10, CPT/HCPCS, ATC/NDC drug codes, Read/SNOMED, lab thresholds) used to define the **population, exposure, outcome, confounders, and effect modifiers** provided in full (text or supplement), per RECORD 6.1 / 7.1? "Diabetes/MI/the cohort was identified from the database" with **no code list** is the single most common RECORD failure.
|
|
20
|
+
- Are those definitions **validated** — a referenced validation study, or a PPV/sensitivity estimate — or at least is the lack of validation acknowledged (RECORD 6.2)? An unvalidated outcome algorithm presented as if it were a gold-standard diagnosis is a misclassification risk (RD5).
|
|
21
|
+
- Missing code-lists, or validated-sounding phenotypes with no validation reference/acknowledgement → MAJOR.
|
|
22
|
+
|
|
23
|
+
**RD3 — Data linkage and linkage-quality evaluation**:
|
|
24
|
+
- If two or more databases were **linked**, is the **linkage method** (deterministic on a unique identifier vs **probabilistic**, and on which fields) and the **linkage-quality evaluation** (match/linkage rate, handling of non-matches and false matches, any bias in who links) reported (RECORD 12.3)? Is a **person-flow at each linkage stage** shown (RECORD 6.3)?
|
|
25
|
+
- Are individuals who **failed to link** characterised (linkage is often differential by age/region/insurance), and is the impact on selection considered?
|
|
26
|
+
- Undisclosed linkage method or quality, no linkage-stage flow, or treating the linked subset as representative without examining non-linkage → MAJOR.
|
|
27
|
+
|
|
28
|
+
**RD4 — Participant-selection flow including data-quality filtering**:
|
|
29
|
+
- Is there a **selection/flow** from the source database to the analytic cohort that includes filtering on **data quality, data availability, and linkage** — not only clinical eligibility — with the **N at each step** (RECORD 13.1)? A jump from "the database contains N million records" straight to an analytic N, with the exclusions opaque, hides selection bias.
|
|
30
|
+
- Is the **analysis unit** (persons vs records/encounters/claims) explicit and consistent (cross-link `observational_confounding.md` O8)?
|
|
31
|
+
- No data-driven selection flow, or an unexplained gap between source and analytic N → MAJOR.
|
|
32
|
+
|
|
33
|
+
**RD5 — Misclassification of exposure and outcome**:
|
|
34
|
+
- Are **exposure and outcome misclassification** (from coding/recording, not clinical adjudication) acknowledged and, where possible, **quantified** (validation PPV/sensitivity, quantitative bias analysis, or a sensitivity analysis under alternative definitions)? Are **proxy/surrogate** measures (a prescription ≠ ingestion; a code ≠ the disease) flagged as such?
|
|
35
|
+
- Coded variables treated as gold-standard with no misclassification discussion, or a single rigid definition with no sensitivity to a broader/narrower one → MAJOR (or MINOR if non-differential and acknowledged).
|
|
36
|
+
|
|
37
|
+
**RD6 — Missing data and informative missingness**:
|
|
38
|
+
- Secondary data are frequently **missing-not-at-random** — a lab not ordered is not a normal lab, an unrecorded covariate is not absence of the condition. Is missingness **characterised** (extent, pattern) and handled appropriately (not a naive complete-case that assumes MCAR when missingness is informative; multiple imputation or a sensitivity analysis where justified)?
|
|
39
|
+
- Naive complete-case analysis on informatively-missing EHR fields, or treating "no record of X" as "X absent" without justification → MAJOR.
|
|
40
|
+
|
|
41
|
+
**RD7 — Unmeasured confounding and RWD-specific design bias**:
|
|
42
|
+
- Is **unmeasured/residual confounding** confronted — secondary data often lack lifestyle, disease severity, frailty, or over-the-counter exposures — with a negative-control, E-value, or sensitivity analysis, rather than asserting "adjusted for available confounders" (cross-link `observational_confounding.md`)?
|
|
43
|
+
- For an exposure/drug study, are the biases endemic to RWD addressed by **design**: **immortal-time bias** (time-fixed exposure misclassified person-time), **prevalent-user bias** (new-user / active-comparator design), **protopathic/reverse-causation bias** (a lag/induction window), and confounding by indication? (These are the core of **RECORD-PE**.)
|
|
44
|
+
- An effect estimate with no engagement with unmeasured confounding, or a drug-effect design exposed to immortal-time / prevalent-user bias with no mitigation → MAJOR.
|
|
45
|
+
|
|
46
|
+
**RD8 — Eligibility drift, data access, and reproducibility**:
|
|
47
|
+
- Over the study window, did **coding systems or eligibility/enrolment rules change** (ICD-9→10 transition, formulary or coverage changes), and is that acknowledged (RECORD 19.1)?
|
|
48
|
+
- Are the **extent of data access**, the **data-cleaning methods**, and the **availability of the protocol, derived-variable definitions / code-lists, and analysis code** stated (RECORD 12.1 / 12.2 / 22.1)? Reproducibility in RWD studies rests on the published phenotype definitions and code.
|
|
49
|
+
- Unacknowledged coding/eligibility drift over a multi-year window, or no availability of protocol/code-lists/code for a non-public database, → MAJOR (drift) / MINOR (availability), per centrality.
|
|
@@ -300,6 +300,8 @@ These modules carry the same domain-specific critique probes used by `/peer-revi
|
|
|
300
300
|
| Mendelian randomization (genetic variants as instrumental variables: two-sample summary-data, one-sample, multivariable MR, drug-target / cis-MR, non-linear MR) | `references/domain-probes/mendelian_randomization.md` (MR1–MR8) |
|
|
301
301
|
| Polygenic risk score / polygenic score (PRS / PGS) developed, validated, or applied as a predictor or risk-stratifier | `references/domain-probes/polygenic_risk_score.md` (PG1–PG8) |
|
|
302
302
|
| Network meta-analysis (≥3 interventions via direct + indirect evidence, treatment ranking, incl. component NMA) | `references/domain-probes/network_meta_analysis.md` (NM1–NM8) |
|
|
303
|
+
| Health economic evaluation (cost-effectiveness / cost-utility / cost-benefit / budget-impact; trial-based or decision-model-based — decision tree, Markov, DES) | `references/domain-probes/health_economic_evaluation.md` (HE1–HE8) |
|
|
304
|
+
| Observational study using routinely-collected health data (administrative claims / EHR / disease or population registry / health-checkup DB, linked or not) | `references/domain-probes/record_routinely_collected_data.md` (RD1–RD8) |
|
|
303
305
|
|
|
304
306
|
When the manuscript matches a row, read `${CLAUDE_SKILL_DIR}/references/domain-probes/<module>.md` and apply each probe as an additional source of Anticipated Major / Minor Comments. The module severity words (MAJOR / MINOR) map to this skill's framing as follows: a conclusion-threatening or design-level finding becomes a **Fatal** Anticipated Major Comment, a reporting-level finding becomes a **Fixable** Anticipated Minor Comment, and each is tagged with the closest category letter (A–K). These probes **complement** categories A–K above; they do not replace them. (The modules are vendored byte-identical from `/peer-review`; do not edit one copy only — run `python3 scripts/check_domain_probe_sync.py --sync`.)
|
|
305
307
|
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review.
|
|
2
|
+
Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal
|
|
3
|
+
recommendation. Each consuming skill maps findings to its own output:
|
|
4
|
+
- peer-review: Major / Minor comments + Confidential Comments to the Editor; a
|
|
5
|
+
perspective/time-horizon/uncertainty/ICER-interpretation flaw is placed as Major #1.
|
|
6
|
+
- self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.
|
|
7
|
+
Do NOT edit one copy only — run `python3 scripts/check_domain_probe_sync.py --sync`. -->
|
|
8
|
+
|
|
9
|
+
# Health economic evaluation probes (HE1–HE8)
|
|
10
|
+
|
|
11
|
+
An 8-probe checklist for **health economic evaluations** — comparative analyses of the costs and consequences of two or more options: cost-effectiveness (CEA), cost-utility (CUA, QALY-based), cost-benefit (CBA), cost-minimisation, and budget-impact/HTA analyses, whether **trial-based** or **decision-model-based** (decision tree, Markov/state-transition, discrete-event simulation). These probes complement (do not replace) the generic Phase 2 checklist and the **CHEERS 2022** reporting items; they target the gap between a clean-looking incremental cost-effectiveness ratio (ICER) and whether the structural choices that produced it — perspective, time horizon, discounting, the effectiveness source, the cost basis, the model, and the handling of uncertainty — are defensible. Because the headline number is a single ratio that a reader cannot reproduce from the manuscript, the burden is on the study to *evidence* each structural choice, not assert it. HE2 (time horizon & discounting), HE5 (model structure), and HE6 (uncertainty / probabilistic sensitivity analysis) are the highest-yield; run them first.
|
|
12
|
+
|
|
13
|
+
**HE1 — Decision problem, comparators, and perspective**:
|
|
14
|
+
- Is the decision/policy question explicit, and are **all relevant comparators** included — the current standard of care and, where appropriate, a do-nothing/usual-care arm — rather than a convenient or obsolete comparator that flatters the new intervention? Omitting the true standard of care is the most common way to manufacture a favourable ICER.
|
|
15
|
+
- Is the **analytic perspective** stated (healthcare-system / payer vs **societal**) and justified, and is it applied consistently to what is counted (HE4)? A "societal" claim that omits productivity and informal-care costs — or a payer analysis that smuggles in societal costs — is an internal inconsistency.
|
|
16
|
+
- A missing/obsolete comparator, an unstated perspective, or a perspective inconsistent with the costs counted → MAJOR.
|
|
17
|
+
|
|
18
|
+
**HE2 — Time horizon and discounting**:
|
|
19
|
+
- Is the **time horizon** long enough to capture all material differential costs and health effects? For a chronic disease or a one-off intervention with lifelong consequences, a **lifetime horizon** is usually required; a horizon truncated to trial follow-up flatters whichever arm has earlier benefit and hides downstream costs.
|
|
20
|
+
- Are **both costs and outcomes discounted** at a stated, justified rate (with the jurisdiction's reference-case rate used and a sensitivity at alternative rates)? Discounting only costs, or not discounting a multi-year horizon at all, biases the ICER.
|
|
21
|
+
- A truncated horizon with no extrapolation justification, undiscounted long-horizon results, or asymmetric discounting → MAJOR.
|
|
22
|
+
|
|
23
|
+
**HE3 — Effectiveness source and outcome valuation**:
|
|
24
|
+
- Where do the **effectiveness/relative-effect estimates** come from — a head-to-head RCT or meta-analysis, or a weaker source (single-arm, naive indirect comparison, observational) — and is that source appropriate and its uncertainty carried into the model (HE6)? A model driven by an optimistic point estimate from a weak source is not rescued by elegant modelling.
|
|
25
|
+
- For a **cost-utility analysis**, are **QALYs** built from a **named preference-based instrument** (e.g. EQ-5D-5L) and a stated **value set / tariff** for the relevant population, rather than ad-hoc or mapped utilities with the mapping undisclosed?
|
|
26
|
+
- A weak/again-mismatched effectiveness source, or QALYs with no stated utility instrument/value set → MAJOR (or a prominent limitation if the source is the best available and is treated as uncertain).
|
|
27
|
+
|
|
28
|
+
**HE4 — Costs, resource use, currency, and price year**:
|
|
29
|
+
- Are **resource quantities and unit costs reported separately** (so a reader can re-value for another setting), and are the cost categories **consistent with the stated perspective** (HE1)?
|
|
30
|
+
- Are the **currency, price (cost) year, and any inflation/currency-conversion method** stated? Costs pooled across years or countries without a stated price year and conversion are not interpretable or transferable.
|
|
31
|
+
- Missing price year, perspective-inconsistent cost categories, or undisclosed resource-vs-unit-cost bundling → MAJOR.
|
|
32
|
+
|
|
33
|
+
**HE5 — Model structure, assumptions, and validation**:
|
|
34
|
+
- If **model-based**, is the model **type and structure** (states/events, cycle length, half-cycle correction for a Markov model) described and **justified** against the disease's natural history, rather than chosen for convenience? Is **structural uncertainty** acknowledged (alternative plausible structures), and was the model **validated** (internal consistency, and external/predictive validation where data exist)?
|
|
35
|
+
- Are key **structural assumptions and extrapolation** beyond the data (e.g. survival curve extrapolation, treatment-effect waning) declared and tested?
|
|
36
|
+
- An unjustified structure, undeclared extrapolation, or no validation/structural-sensitivity → MAJOR (the structure can drive the ICER more than any single parameter).
|
|
37
|
+
|
|
38
|
+
**HE6 — Uncertainty: deterministic AND probabilistic**:
|
|
39
|
+
- Is **parameter uncertainty** characterised by a **probabilistic sensitivity analysis (PSA)** — every uncertain input assigned a distribution and propagated by Monte Carlo — and reported as a **cost-effectiveness plane** and **cost-effectiveness acceptability curve (CEAC)**, not just a deterministic point ICER? Are **one-way / tornado** analyses used to identify the drivers, and **scenario analyses** for non-parametric (structural/methodological) uncertainty?
|
|
40
|
+
- Are the parameter **distributions justified** (beta for probabilities/utilities, gamma/log-normal for costs) rather than arbitrary ±20% ranges presented as if probabilistic?
|
|
41
|
+
- A single deterministic ICER with **no PSA**, or "sensitivity analysis" that is one-way only with no probabilistic component → MAJOR.
|
|
42
|
+
|
|
43
|
+
**HE7 — Results presentation and ICER interpretation**:
|
|
44
|
+
- Are **incremental costs and incremental effects reported separately** (and ideally a disaggregated cost/outcome table), not only the ratio? Are **dominance and extended dominance** correctly identified and dominated strategies excluded before computing ICERs?
|
|
45
|
+
- Is the ICER interpreted against a **stated, justified cost-effectiveness threshold (willingness-to-pay)** appropriate to the jurisdiction — not an arbitrary or post-hoc threshold — and is the conclusion ("cost-effective") conditional on that threshold and the uncertainty (CEAC probability), rather than asserted from the point estimate alone? Net monetary/health benefit (INMB/INHB) is a clearer alternative at a given threshold.
|
|
46
|
+
- An ICER reported without incrementals, mishandled dominance, or a "cost-effective" claim with no threshold context or ignoring the CEAC → MAJOR.
|
|
47
|
+
|
|
48
|
+
**HE8 — Equity, generalisability, funding/COI, and CHEERS reporting**:
|
|
49
|
+
- Are **distributional/equity effects** considered where relevant (who bears the costs and gains the health), and is **generalisability/transferability** to other settings or jurisdictions discussed (cost and epidemiology transfer, not just clinical effect)?
|
|
50
|
+
- Are **funding source and the funder's role** and **conflicts of interest** disclosed? An industry-funded evaluation whose ICER sits just below the threshold and is insensitive to plausible assumptions warrants heightened scrutiny of the structural choices (HE1–HE5).
|
|
51
|
+
- Is reporting mapped to **CHEERS 2022**, and is the model / health-economic analysis plan available for scrutiny?
|
|
52
|
+
- No generalisability discussion for a single-jurisdiction model presented as broadly applicable, or undisclosed funder role on a threshold-hugging result → MAJOR (funding/COI) / MINOR (generalisability), per centrality.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
<!-- Domain probe module — shared, vendored BYTE-IDENTICAL by /peer-review and /self-review.
|
|
2
|
+
Severity words below (MAJOR / MINOR / major / minor) denote finding severity, NOT a journal
|
|
3
|
+
recommendation. Each consuming skill maps findings to its own output:
|
|
4
|
+
- peer-review: Major / Minor comments + Confidential Comments to the Editor; a
|
|
5
|
+
code-list / linkage-quality / selection-flow / RWD-bias design flaw is placed as Major #1.
|
|
6
|
+
- self-review: Anticipated Major / Minor Comments (Fatal / Fixable) mapped to category letters.
|
|
7
|
+
Do NOT edit one copy only — run `python3 scripts/check_domain_probe_sync.py --sync`. -->
|
|
8
|
+
|
|
9
|
+
# Routinely-collected-data study probes (RD1–RD8)
|
|
10
|
+
|
|
11
|
+
An 8-probe checklist for observational studies conducted using **routinely-collected health data** — administrative claims, electronic health records (EHR), disease/population registries, health-administrative and health-checkup databases, and linked versions of these (data **not collected for the study's purpose**). These probes complement (do not replace) the generic Phase 2 checklist, the **STROBE** + **RECORD** reporting items (and **RECORD-PE** for drug studies), and the observational-confounding probes (`observational_confounding.md`, which cover adjustment/collider/analysis-unit issues). They target what secondary-use data add: whether the database can even observe the question, whether the phenotype code-lists and linkage are evidenced rather than asserted, and whether the limitations endemic to data collected for another purpose are confronted. RD2 (phenotype code-lists & validation), RD3 (linkage quality), and RD4 (participant-selection flow) are the highest-yield; run them first.
|
|
12
|
+
|
|
13
|
+
**RD1 — Database identity, provenance, and fitness-for-purpose**:
|
|
14
|
+
- Is the **database named** and its **type** (claims / EHR / registry / health-checkup), provenance, coverage population, and capture window described — and can it **structurally observe** the exposure and outcome? A claims database cannot see out-of-network or cash-pay care; an EHR cannot see care at other systems; neither reliably sees OTC drugs, over-the-counter outcomes, or death out of hospital.
|
|
15
|
+
- Is the **timeframe and geographic setting** stated (title/abstract per RECORD 1.1–1.2)?
|
|
16
|
+
- An unnamed/undescribed database, or one whose structure cannot capture the named exposure/outcome (so the measure is systematically incomplete), → MAJOR.
|
|
17
|
+
|
|
18
|
+
**RD2 — Phenotype definitions: code-lists and algorithms, evidenced not asserted**:
|
|
19
|
+
- Are the **codes / algorithms** (ICD-9/10, CPT/HCPCS, ATC/NDC drug codes, Read/SNOMED, lab thresholds) used to define the **population, exposure, outcome, confounders, and effect modifiers** provided in full (text or supplement), per RECORD 6.1 / 7.1? "Diabetes/MI/the cohort was identified from the database" with **no code list** is the single most common RECORD failure.
|
|
20
|
+
- Are those definitions **validated** — a referenced validation study, or a PPV/sensitivity estimate — or at least is the lack of validation acknowledged (RECORD 6.2)? An unvalidated outcome algorithm presented as if it were a gold-standard diagnosis is a misclassification risk (RD5).
|
|
21
|
+
- Missing code-lists, or validated-sounding phenotypes with no validation reference/acknowledgement → MAJOR.
|
|
22
|
+
|
|
23
|
+
**RD3 — Data linkage and linkage-quality evaluation**:
|
|
24
|
+
- If two or more databases were **linked**, is the **linkage method** (deterministic on a unique identifier vs **probabilistic**, and on which fields) and the **linkage-quality evaluation** (match/linkage rate, handling of non-matches and false matches, any bias in who links) reported (RECORD 12.3)? Is a **person-flow at each linkage stage** shown (RECORD 6.3)?
|
|
25
|
+
- Are individuals who **failed to link** characterised (linkage is often differential by age/region/insurance), and is the impact on selection considered?
|
|
26
|
+
- Undisclosed linkage method or quality, no linkage-stage flow, or treating the linked subset as representative without examining non-linkage → MAJOR.
|
|
27
|
+
|
|
28
|
+
**RD4 — Participant-selection flow including data-quality filtering**:
|
|
29
|
+
- Is there a **selection/flow** from the source database to the analytic cohort that includes filtering on **data quality, data availability, and linkage** — not only clinical eligibility — with the **N at each step** (RECORD 13.1)? A jump from "the database contains N million records" straight to an analytic N, with the exclusions opaque, hides selection bias.
|
|
30
|
+
- Is the **analysis unit** (persons vs records/encounters/claims) explicit and consistent (cross-link `observational_confounding.md` O8)?
|
|
31
|
+
- No data-driven selection flow, or an unexplained gap between source and analytic N → MAJOR.
|
|
32
|
+
|
|
33
|
+
**RD5 — Misclassification of exposure and outcome**:
|
|
34
|
+
- Are **exposure and outcome misclassification** (from coding/recording, not clinical adjudication) acknowledged and, where possible, **quantified** (validation PPV/sensitivity, quantitative bias analysis, or a sensitivity analysis under alternative definitions)? Are **proxy/surrogate** measures (a prescription ≠ ingestion; a code ≠ the disease) flagged as such?
|
|
35
|
+
- Coded variables treated as gold-standard with no misclassification discussion, or a single rigid definition with no sensitivity to a broader/narrower one → MAJOR (or MINOR if non-differential and acknowledged).
|
|
36
|
+
|
|
37
|
+
**RD6 — Missing data and informative missingness**:
|
|
38
|
+
- Secondary data are frequently **missing-not-at-random** — a lab not ordered is not a normal lab, an unrecorded covariate is not absence of the condition. Is missingness **characterised** (extent, pattern) and handled appropriately (not a naive complete-case that assumes MCAR when missingness is informative; multiple imputation or a sensitivity analysis where justified)?
|
|
39
|
+
- Naive complete-case analysis on informatively-missing EHR fields, or treating "no record of X" as "X absent" without justification → MAJOR.
|
|
40
|
+
|
|
41
|
+
**RD7 — Unmeasured confounding and RWD-specific design bias**:
|
|
42
|
+
- Is **unmeasured/residual confounding** confronted — secondary data often lack lifestyle, disease severity, frailty, or over-the-counter exposures — with a negative-control, E-value, or sensitivity analysis, rather than asserting "adjusted for available confounders" (cross-link `observational_confounding.md`)?
|
|
43
|
+
- For an exposure/drug study, are the biases endemic to RWD addressed by **design**: **immortal-time bias** (time-fixed exposure misclassified person-time), **prevalent-user bias** (new-user / active-comparator design), **protopathic/reverse-causation bias** (a lag/induction window), and confounding by indication? (These are the core of **RECORD-PE**.)
|
|
44
|
+
- An effect estimate with no engagement with unmeasured confounding, or a drug-effect design exposed to immortal-time / prevalent-user bias with no mitigation → MAJOR.
|
|
45
|
+
|
|
46
|
+
**RD8 — Eligibility drift, data access, and reproducibility**:
|
|
47
|
+
- Over the study window, did **coding systems or eligibility/enrolment rules change** (ICD-9→10 transition, formulary or coverage changes), and is that acknowledged (RECORD 19.1)?
|
|
48
|
+
- Are the **extent of data access**, the **data-cleaning methods**, and the **availability of the protocol, derived-variable definitions / code-lists, and analysis code** stated (RECORD 12.1 / 12.2 / 22.1)? Reproducibility in RWD studies rests on the published phenotype definitions and code.
|
|
49
|
+
- Unacknowledged coding/eligibility drift over a multi-year window, or no availability of protocol/code-lists/code for a non-public database, → MAJOR (drift) / MINOR (availability), per centrality.
|