medsci-skills 5.1.0 → 5.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -13
- package/metadata/distribution_files.json +67 -22
- package/metadata/distribution_manifest.json +1 -1
- package/package.json +1 -1
- package/skills/analyze-stats/SKILL.md +9 -0
- package/skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md +79 -0
- package/skills/check-reporting/SKILL.md +4 -2
- package/skills/check-reporting/references/checklists/CHEERS_2022.md +81 -0
- package/skills/check-reporting/scripts/check_checklist_exists.py +2 -0
- package/skills/make-figures/references/reporting_guideline_figure_map.md +2 -2
- package/skills/mllm-eval/SKILL.md +8 -0
- package/skills/mllm-eval/references/evaluation_axes.md +161 -0
- package/skills/model-evaluation/SKILL.md +12 -0
- package/skills/model-evaluation/references/metric_selection_grounding.md +139 -0
- package/skills/model-evaluation/scripts/check_metric_reporting.py +1 -1
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/det_good_wrapped.md +5 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/det_no_iou.md +4 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/verify.sh +6 -1
- package/skills/model-validation/SKILL.md +6 -0
- package/skills/model-validation/references/validation_design.md +150 -0
- package/skills/orchestrate/SKILL.md +1 -1
- package/skills/peer-review/SKILL.md +6 -0
- package/skills/peer-review/references/domain-probes/health_economic_evaluation.md +52 -0
- package/skills/self-review/SKILL.md +1 -0
- package/skills/self-review/references/domain-probes/health_economic_evaluation.md +52 -0
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
**51 skills that actually work.** Built by a physician-researcher, tested on real publications.
|
|
6
6
|
|
|
7
|
-
*MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer —
|
|
7
|
+
*MedSci Skills is an end-to-end research tool for physician and medical-engineering researchers — design → scaffold → validate → publish — for the clinical manuscript and the medical-AI model behind it. Its moat is the compliance layer — 39 reporting guidelines and risk-of-bias tools, reference/citation verification, and deterministic integrity gates before peer review — now extended by a model-engineering lane that scaffolds reproducible, leakage-safe training repos and audits model validation. Clinical AI model research engineering is in scope; a general AI-scientist platform is not. It competes on clinical submission reliability, not skill count.*
|
|
8
8
|
|
|
9
9
|
[](LICENSE)
|
|
10
10
|
[](https://github.com/Aperivue/medsci-skills/releases/latest)
|
|
@@ -281,54 +281,54 @@ The E2E pipeline (`orchestrate --e2e`) produces everything up to `qc/`. The `sub
|
|
|
281
281
|
|
|
282
282
|
## What's New
|
|
283
283
|
|
|
284
|
-
**v4.10** — reviewer-coverage expansion reverse-engineered from high-IF, CC-BY papers (learn-only under the `reverse_engineer/` license firewall), plus a clinician-friendly update path. Additive and backward-compatible; 45 skills / **
|
|
284
|
+
**v4.10** — reviewer-coverage expansion reverse-engineered from high-IF, CC-BY papers (learn-only under the `reverse_engineer/` license firewall), plus a clinician-friendly update path. Additive and backward-compatible; 45 skills / **39 guidelines** / 36 detectors / **15 domain-probe modules** (was 12):
|
|
285
285
|
|
|
286
286
|
- **Three new reviewer domain-probe modules** (`/peer-review` + `/self-review`, vendored byte-identical): **Mendelian randomization** (MR1–MR8 — IV assumptions, pleiotropy-robust sensitivity suite, Steiger, sample overlap, NLMR, drug-target colocalization), **polygenic risk score** (PG1–PG8 — ancestry portability, base/target leakage, incremental value over the clinical model, screening-vs-discrimination, calibration), and **network meta-analysis** (NM1–NM8 — transitivity, incoherence, SUCRA over-interpretation, CINeMA/GRADE-NMA, component-NMA additivity). Plus observational **O17** (agnostic many-exposure-scan multiplicity: ExWAS/EWAS/MWAS).
|
|
287
287
|
- **Two reporting-guideline checklists** (36 → 38): **STROBE-MR** and **PGS-RS / PRS-RS**, with study-type routing. Four new `/analyze-stats` analysis guides (multiplicity, MR, PRS, NMA) and a `/clean-data` implausible-value + cross-field validity reference.
|
|
288
288
|
- **Clinician-friendly update reminders** — the classroom installers enable the in-app "update available" notice + one-click Desktop updater by default; the `npx`/manual paths print how to turn it on; the install guide recommends `npx medsci-skills install --enable-update-notify`.
|
|
289
289
|
|
|
290
|
-
**v4.9** — analysis-integrity hardening promoted from real review cycles, plus journal-mechanics additions. Additive and backward-compatible; still 45 skills /
|
|
290
|
+
**v4.9** — analysis-integrity hardening promoted from real review cycles, plus journal-mechanics additions. Additive and backward-compatible; still 45 skills / 39 guidelines, analysis-integrity detectors **32 → 36**:
|
|
291
291
|
|
|
292
292
|
- **Four new gates** — a **duplicate-bibliography** check (`check_reference_duplication.py`) for the hybrid `[@key]` + hand-typed `## References` build that renders the list twice; a **cross-script binning / composite-indicator** consistency check (`check_binning_consistency.py`, `BINNING_DRIFT` / `DERIVED_DEF_DRIFT`) for a derived categorical or composite indicator defined inconsistently across analysis scripts; a **float citation-order** check (`check_citation_order.py`) for numbered Tables/Figures not first cited in ascending order per series; and an **audit-dump leak** gate (`/sync-submission`) that blocks a `/check-reporting` output mistakenly attached as a submission file.
|
|
293
293
|
- **KJR technical-check conventions + percentage-decimal style**, reader-allocation-under-burden and generative-image-as-study-object reporting (`/design-ai-benchmarking`, `/check-reporting`), and a **Liver International** CSL with that journal's submission mechanics (`/manage-refs`).
|
|
294
294
|
|
|
295
|
-
**v4.8** is the **review-harvest batch** — deterministic detector hardening promoted from real-manuscript review cycles. Additive and backward-compatible; still 45 skills /
|
|
295
|
+
**v4.8** is the **review-harvest batch** — deterministic detector hardening promoted from real-manuscript review cycles. Additive and backward-compatible; still 45 skills / 39 guidelines, analysis-integrity detectors **30 → 32**:
|
|
296
296
|
|
|
297
297
|
- **Two new gates** — `check_supplement_hygiene.py` lints the rendered supplement / tables / caption files (not just the manuscript) for §-labels, placeholders, build markers, response-letter framing, and unresolved body↔supplement cross-references; `check_null_calibration.py` flags a headline negative/equivalence claim made without a minimum-detectable-effect / power / equivalence statement.
|
|
298
298
|
- **Four detector false-positive fixes** — gates no longer fire on a recommended colorblind-safe palette, author-footnote `§` daggers, a correctly-hedged disclaimer, or a tier-label digit; each with a regression fixture and three newly CI-wired test suites.
|
|
299
299
|
- **Nine reviewer-side domain probes** (SR/MA, observational, diagnostic, AI-overclaiming, survival) plus a `/design-study` design-stage ceiling gate for perceptual/reader-AI studies and a reusable confidence-weighted-rating→AUC monotonicity template.
|
|
300
300
|
|
|
301
|
-
**v4.7** is the **self-update foundation** — physician-researchers stay current without GitHub, git, or a terminal. Additive and backward-compatible; still 45 skills /
|
|
301
|
+
**v4.7** is the **self-update foundation** — physician-researchers stay current without GitHub, git, or a terminal. Additive and backward-compatible; still 45 skills / 39 guidelines / 30 detectors:
|
|
302
302
|
|
|
303
303
|
- **Transactional, crash-recoverable installer.** Each install runs through a durable journal state machine recovered on the next run (roll back / forward-clean / fail-closed), with per-target SHA-256 inventories — your modified or third-party skills are backed up and never clobbered or auto-deleted.
|
|
304
304
|
- **One-click self-updater** (`~/.medsci-skills/updater/`, `install.py --check-update`). Verifies the download against the github.com API digest and **never `extractall()`s** (per-entry rejection of traversal / symlink / duplicate / zip-bomb + an allowlist & per-file hash). The release pipeline injects a verified `provenance.json`, attests build provenance, runs on a protected `release` environment, and verifies each ZIP round-trips through the updater's own safe-extract before publishing.
|
|
305
305
|
- **Opt-in update notice (off by default):** `install.py --enable-update-notify` shows a one-line "update available" message at Claude Code session start — no telemetry, reads nothing about your session, installs nothing. `--disable-update-notify` / `MEDSCI_NO_UPDATE_CHECK=1` turn it off. *(Honest scope: the digest/attestation detect transport tampering, not a compromised publisher account — see `SECURITY.md`.)*
|
|
306
306
|
|
|
307
|
-
**v4.6** is a maintainability, governance, and review-depth release — still 45 skills /
|
|
307
|
+
**v4.6** is a maintainability, governance, and review-depth release — still 45 skills / 39 guidelines; analysis-integrity detectors **28 → 30**, domain probes 11 → 12:
|
|
308
308
|
|
|
309
309
|
- **Fairness / equity / subgroup-performance probe (EQ0–EQ6)** for AI/prediction/diagnostic studies that claim cross-population performance, plus two new detectors: an **AI-disclosure + data/code-availability** check (`/sync-submission`) and a **structured-summary-box conformance** check (`/academic-aio`).
|
|
310
310
|
- **Governance + answer-engine layer:** `ROADMAP.md`, `MAINTAINERS.md`, `SECURITY.md`, a maintainer workflow + release checklist, an AEO/GEO `docs/faq.md`, a "Start here: 3 workflows" + "Validation status" section in this README, and a new `maturity` field (official / experimental / community) on every skill.
|
|
311
311
|
- **Token diet (pilot):** `write-paper` Phase 7 integrity audits moved to a load-on-demand reference (~2,559 tokens saved per invocation). Positioning now leads with the compliance moat rather than skill count.
|
|
312
312
|
|
|
313
|
-
**v4.5** deepens the review + submission surface with no new skill or reporting-guideline count (still 45 skills /
|
|
313
|
+
**v4.5** deepens the review + submission surface with no new skill or reporting-guideline count (still 45 skills / 39 guidelines); analysis-integrity detectors **27 → 28**:
|
|
314
314
|
|
|
315
315
|
- **`/clean-data` + `/analyze-stats` — reverse-coded-item / negative-alpha detector.** A multi-item Likert scale with a negatively-worded item must be recoded `(min+max) − x` before the scale total or Cronbach's alpha is computed; left un-recoded, the item correlates negatively with the rest of the scale and alpha collapses (often negative). A negative alpha is a coding bug, not a "multidimensional construct." New stdlib-only `check_reverse_coding.py` returns `REVERSE_CODING_LIKELY` / `REVERSE_CODING_SUSPECT` / `OK` from per-item item-rest correlations + raw alpha; the Likert summary template gains a `--reverse-items` recode flag.
|
|
316
316
|
- **`/peer-review` + `/self-review` — SR/MA + DTA + prediction-model probe batch.** `sr_ma.md` **P12** risk-of-bias table row-sum ↔ traffic-light figure-matrix reconciliation and **P13** included-study ↔ reference-list completeness; `diagnostic_accuracy.md` **D7** index-test-as-enrollment-criterion circularity; `clinical_prediction_model.md` **CP5** intended-use horizon leakage and **CP6** development/CV vs held-out/external validation-nomenclature conflation. Vendored byte-identical into `/self-review`.
|
|
317
317
|
- **`/sync-submission` — embedded absolute-path leak scan.** A `word/*.xml` attribute (e.g. a pandoc-embedded image's `<pic:cNvPr descr="…">`) carrying an absolute home-dir path (`/Users/…`, `/home/…`) is a username leak invisible to a rendered-text scan; now flagged as `docx_embedded_abs_path` under `check_asset_anonymization.py`.
|
|
318
318
|
|
|
319
|
-
**v4.4** adds reviewer/analysis depth with no new skill or reporting-guideline count (still 45 skills /
|
|
319
|
+
**v4.4** adds reviewer/analysis depth with no new skill or reporting-guideline count (still 45 skills / 39 guidelines / 27 detectors):
|
|
320
320
|
|
|
321
321
|
- **`/author-strategy` — trajectory-archetype classification (optional).** Classifies a queried author's PubMed trajectory into abstract career archetypes (A1 infrastructure builder, A2 methodology rule-maker, A3 clinical→AI hybrid, A4 SR/MA volume engine, A5 large-consortium participation, A6 device/technique depth, + a computed composite) as an **explainable, multi-label, confidence-scored heuristic — not an objective verdict**. The rubric is a single canonical YAML (the narrative doc is generated from it); scores exclude `unavailable` signals (h-index/citation/venue-tier → `[VERIFY]`, never fabricated); a **disambiguation gate** binds an approved `corpus_manifest.json` to the CSV (csv + PMID-set hashes) so a surname alone never classifies, and target-author attribution never borrows a co-author's ORCID/affiliation.
|
|
322
322
|
- **`/peer-review` + `/self-review` — Image-Synthesis / cross-modality probe (IS1–IS4)** for studies that synthesize one imaging modality from another and claim the output carries the target's information, plus a reviewer-side reference-integrity spot-check.
|
|
323
323
|
- **`/verify-refs` — OpenAlex tertiary index** recovers conference-proceedings / non-DOI citations (NeurIPS/ICLR/ACL) that fall through PubMed and CrossRef, the free analogue of a portal's second index.
|
|
324
324
|
|
|
325
|
-
**v4.3** hardens the **cross-sectional / observational cohort** review surface end-to-end, much of it reverse-engineered from real CC-BY cohort papers (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills /
|
|
325
|
+
**v4.3** hardens the **cross-sectional / observational cohort** review surface end-to-end, much of it reverse-engineered from real CC-BY cohort papers (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills / 39 guidelines); analysis-integrity detectors **25 → 27**:
|
|
326
326
|
|
|
327
327
|
- **Observational probes O1 → O14** (`/peer-review` + `/self-review`, vendored) — over-adjustment / analysis-unit clustering / outcome construct-validity (O7–O9), overlapping-subset gradient (O10), **complex-survey design & weighting** for NHANES/KNHANES (O11), **data-driven threshold / "inflection-point" mining** (O12), **cross-sectional mediation** temporal-order & sequential-ignorability (O13), and **interaction scale** — additive RERI/AP/S vs multiplicative (O14). Plus a new **clinical-prediction-model** probe module **CP1–CP4** and survival **S9** (panel-data / multistate variance).
|
|
328
328
|
- **Two new detectors (25 → 27)** — `check_wordcount_cap.py` (the revision-inflation trap: body vs journal cap) and `check_paren_spans.py` (em-dash→paren conversions that wrap a whole sentence). Plus a `check_confounding_completeness.py` upgrade (DB-code↔prose alias map, SMD-from-mean±SD, exposure-defining-covariate exemption), a `check_cohort_arithmetic.py` `ANALYSIS_UNIT_UNDISCLOSED` check, a `check_scope_coherence.py` cross-sectional-yield lexicon, and a verify-refs corporate/collective-author render-abort fix.
|
|
329
329
|
- **Analysis & submission tooling** — `/analyze-stats` gains **mediation** and **interaction & effect-modification** guides; `/sync-submission` gains `assemble_supplement.py` (S{N} index↔file integrity) and a `/revise` body-word-count exit gate; `/render-pdf-doc` gains a `scan_glyph_coverage.py` xelatex silent-glyph-drop scan.
|
|
330
330
|
|
|
331
|
-
**v4.2** builds out the case-report capability end-to-end, grounded in real CC-BY case reports (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills /
|
|
331
|
+
**v4.2** builds out the case-report capability end-to-end, grounded in real CC-BY case reports (learn-only under the license firewall) — no new skill or reporting-guideline count (still 45 skills / 39 guidelines); journal profiles **68 → 73**:
|
|
332
332
|
|
|
333
333
|
- **Case-report + case-series writing** — `/write-paper` gains a CARE narrative + 150-word-abstract case-report exemplar, a **case-series** paper type (methods-light mini-cohort, all-cases summary table, counts-not-rates), and **adverse-event/pharmacovigilance** (Naranjo/WHO-UMC causality) and **diagnostic-pitfall/mimic** subtypes.
|
|
334
334
|
- **Radiology / imaging-led track** — a dedicated `exemplar_case_report_radiology.md` (per-modality technique→findings→impression, structured-reporting lexicons BI-RADS/LI-RADS/PI-RADS/TI-RADS/Lung-RADS/O-RADS, quantitative threshold honesty, an interventional-radiology procedure/complication subtype, DICOM de-identification) plus a `/make-figures` annotated multimodality imaging-panel exemplar.
|
|
@@ -452,7 +452,7 @@ ma-scout -> search-lit -> fulltext-retrieval -> design-study ──> write-proto
|
|
|
452
452
|
| **search-lit** | PubMed + Semantic Scholar + bioRxiv search with anti-hallucination citation verification. Token-efficient error handling -- CrossRef failures are silently batched, not repeated. BibTeX output tags each entry with `verified`/`verified_by`/`verified_on` fields so downstream skills can trust the citation provenance. |
|
|
453
453
|
| **verify-refs** | Pre-submission reference audit for `.md`, `.docx`, `.bib`, or `.tsv` inputs. Extracts references, verifies DOI/PMID via CrossRef/PubMed when available, and writes `qc/reference_audit.json` as the sole output — row-level status (OK / MISMATCH / UNVERIFIED / FABRICATED) lives inside the JSON `records[]` block. `/search-lit` produces candidate BibTeX; `/lit-sync` owns `manuscript/_src/refs.bib`. |
|
|
454
454
|
| **fulltext-retrieval** | Batch open-access PDF downloader. Unpaywall → PMC → OpenAlex → CrossRef pipeline. OA-only -- no paywall bypass. Input: DOI list or TSV. Optional PDF→Markdown conversion via [pymupdf4llm](https://pymupdf.readthedocs.io/en/latest/pymupdf4llm/) for token-efficient LLM analysis of academic papers. |
|
|
455
|
-
| **check-reporting** | Manuscript compliance audit against
|
|
455
|
+
| **check-reporting** | Manuscript compliance audit against 39 reporting guidelines and risk of bias tools (STROBE, STROBE-MR, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, PRISMA, PRISMA-DTA, PRISMA-P, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA). Machine-readable JSON summary with `compliance_pct` and `fixable_by_ai` flags for automated pipeline integration. |
|
|
456
456
|
| **analyze-stats** | Statistical analysis code generation (Python/R) for diagnostic accuracy, DTA meta-analysis (bivariate/HSROC), inter-rater agreement, survival analysis, demographics tables, regression (logistic/linear), propensity score (matching/IPTW/overlap weighting), and repeated measures (RM ANOVA/GEE/mixed models). Calibration mandatory for prediction models. |
|
|
457
457
|
| **meta-analysis** | Full systematic review and meta-analysis pipeline (8 phases). DTA (bivariate/HSROC) and intervention meta-analysis. Protocol to submission-ready manuscript with PRISMA-DTA compliance. |
|
|
458
458
|
| **make-figures** | Publication-ready figures and visual abstracts: ROC curves, forest plots, PRISMA/CONSORT/STARD flow diagrams, Kaplan-Meier curves, Bland-Altman plots, confusion matrices, and journal-specific visual/graphical abstracts (python-pptx template-based). Communication-first design principles (Nat Hum Behav 2026 — key message, audience, cognitive load, figure-vs-table decision) and five flow-diagram production lessons (official-template fidelity, VML fallback PDF export, docx XML escape, sequential placeholder mapping, version freeze); critic rubric Section G adds 5 communication-first checks. `--study-type` auto-generates the full required figure set; structured `_figure_manifest.md` output for downstream pipeline consumption; D2 enforced as default for flow diagrams. |
|
|
@@ -621,8 +621,8 @@ Projects declare their source-of-truth layout in `SSOT.yaml`, and a `qc/migratio
|
|
|
621
621
|
### Meta-Analysis Failure Modes
|
|
622
622
|
`/meta-analysis` ships empirical failure-mode references (data integrity, review orchestration, submission package drift, post-submission release ops) with four automation hooks: `scripts/prisma_5way_consistency.py` (DI-6 PRISMA number consistency), `scripts/extraction_consensus_log_init.py` (DI-1 dual-extraction scaffold), `scripts/tag_cleanup_gate.sh` (DI-8 placeholder tag gate), and `scripts/verify_package_integrity.py` (SPD SHA-256 manifest for submission bundles).
|
|
623
623
|
|
|
624
|
-
###
|
|
625
|
-
`check-reporting` includes bundled checklists for
|
|
624
|
+
### 39 Reporting Guidelines & RoB Tools Built-in
|
|
625
|
+
`check-reporting` includes bundled checklists for 39 guidelines and risk-of-bias tools: STROBE, STROBE-MR, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, CHEERS 2022, PRISMA 2020, PRISMA-DTA, PRISMA-P, MOOSE, ARRIVE, CONSORT, CONSORT-AI, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, SQUIRE 2.0, CLEAR, GRRAS, MI-CLEAR-LLM, SWiM, AMSTAR 2, QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA. Includes Results/Discussion section boundary checks and machine-readable JSON summary for pipeline integration.
|
|
626
626
|
|
|
627
627
|
### Publication-Ready Output
|
|
628
628
|
`analyze-stats` generates reproducible Python/R code for 13 analysis types -- including regression, propensity score, and repeated measures -- with mandatory calibration for prediction models. `make-figures` produces journal-specification figures (300 DPI, colorblind-safe palettes, proper dimensions), visual/graphical abstracts, and a tool selection guide (D2 for flow diagrams, matplotlib for data plots). `--study-type` auto-generates the complete figure set for each study design.
|
|
@@ -153,8 +153,13 @@
|
|
|
153
153
|
},
|
|
154
154
|
{
|
|
155
155
|
"path": "skills/analyze-stats/SKILL.md",
|
|
156
|
-
"size":
|
|
157
|
-
"sha256": "
|
|
156
|
+
"size": 54473,
|
|
157
|
+
"sha256": "6f7a194db60588e84ddf8e62e80d6e143d14265e21bdac35efa6efbbdd5d5ffc"
|
|
158
|
+
},
|
|
159
|
+
{
|
|
160
|
+
"path": "skills/analyze-stats/references/analysis_guides/health_economic_evaluation.md",
|
|
161
|
+
"size": 5362,
|
|
162
|
+
"sha256": "a6f223cf060b1a3b472df82660a11c006c5ea2e72e231b730ce326b00a11044a"
|
|
158
163
|
},
|
|
159
164
|
{
|
|
160
165
|
"path": "skills/analyze-stats/references/analysis_guides/mediation.md",
|
|
@@ -533,8 +538,8 @@
|
|
|
533
538
|
},
|
|
534
539
|
{
|
|
535
540
|
"path": "skills/check-reporting/SKILL.md",
|
|
536
|
-
"size":
|
|
537
|
-
"sha256": "
|
|
541
|
+
"size": 36749,
|
|
542
|
+
"sha256": "2c72f50feec1f6c727797c112631d632321243c6bdbe467d7bcad2be37f214a6"
|
|
538
543
|
},
|
|
539
544
|
{
|
|
540
545
|
"path": "skills/check-reporting/references/LICENSES.md",
|
|
@@ -566,6 +571,11 @@
|
|
|
566
571
|
"size": 5746,
|
|
567
572
|
"sha256": "4bac869891f4b334cdc211409062a1dd3d6a31a0b62218ad4d9ced1a34000f77"
|
|
568
573
|
},
|
|
574
|
+
{
|
|
575
|
+
"path": "skills/check-reporting/references/checklists/CHEERS_2022.md",
|
|
576
|
+
"size": 8054,
|
|
577
|
+
"sha256": "830dc33cc4d74a3ebd4f024bb05fd064174002b150b363c3f3f03a13e2bd9a4f"
|
|
578
|
+
},
|
|
569
579
|
{
|
|
570
580
|
"path": "skills/check-reporting/references/checklists/CLAIM_2024.md",
|
|
571
581
|
"size": 6678,
|
|
@@ -763,8 +773,8 @@
|
|
|
763
773
|
},
|
|
764
774
|
{
|
|
765
775
|
"path": "skills/check-reporting/scripts/check_checklist_exists.py",
|
|
766
|
-
"size":
|
|
767
|
-
"sha256": "
|
|
776
|
+
"size": 6676,
|
|
777
|
+
"sha256": "1eff386a3fc1b10c49b1d7348f0634323cb18fe54aa402b1bbaa3515ec137f37"
|
|
768
778
|
},
|
|
769
779
|
{
|
|
770
780
|
"path": "skills/check-reporting/scripts/check_checklist_version.py",
|
|
@@ -2004,7 +2014,7 @@
|
|
|
2004
2014
|
{
|
|
2005
2015
|
"path": "skills/make-figures/references/reporting_guideline_figure_map.md",
|
|
2006
2016
|
"size": 6862,
|
|
2007
|
-
"sha256": "
|
|
2017
|
+
"sha256": "c026fc4fa0a797b91343314d854ae41b8505fc643a07c7ad007572faa4e55798"
|
|
2008
2018
|
},
|
|
2009
2019
|
{
|
|
2010
2020
|
"path": "skills/make-figures/references/visual_abstract_templates/european_radiology.pptx",
|
|
@@ -2533,8 +2543,13 @@
|
|
|
2533
2543
|
},
|
|
2534
2544
|
{
|
|
2535
2545
|
"path": "skills/mllm-eval/SKILL.md",
|
|
2536
|
-
"size":
|
|
2537
|
-
"sha256": "
|
|
2546
|
+
"size": 6720,
|
|
2547
|
+
"sha256": "afdc167bfccb6bad8a4019e0af5bd7c955129220d360511a89962ecd321bc44d"
|
|
2548
|
+
},
|
|
2549
|
+
{
|
|
2550
|
+
"path": "skills/mllm-eval/references/evaluation_axes.md",
|
|
2551
|
+
"size": 10857,
|
|
2552
|
+
"sha256": "49d77ab63feae5dba5cdae7e47d9de6ea97ee3b9eea4b9cba39ec8b0e185be72"
|
|
2538
2553
|
},
|
|
2539
2554
|
{
|
|
2540
2555
|
"path": "skills/mllm-eval/scripts/check_mllm_eval_completeness.py",
|
|
@@ -2623,18 +2638,23 @@
|
|
|
2623
2638
|
},
|
|
2624
2639
|
{
|
|
2625
2640
|
"path": "skills/model-evaluation/SKILL.md",
|
|
2626
|
-
"size":
|
|
2627
|
-
"sha256": "
|
|
2641
|
+
"size": 5780,
|
|
2642
|
+
"sha256": "334b4ca87a2f672446563f3fb6d78c0585386fac12aee29e10dc09465e892193"
|
|
2628
2643
|
},
|
|
2629
2644
|
{
|
|
2630
2645
|
"path": "skills/model-evaluation/references/metric_guide.md",
|
|
2631
2646
|
"size": 2454,
|
|
2632
2647
|
"sha256": "8d09ca7ce9fb9f66ee4942689294d9b12ae1d892ac67769cd68fdc38a4e220ee"
|
|
2633
2648
|
},
|
|
2649
|
+
{
|
|
2650
|
+
"path": "skills/model-evaluation/references/metric_selection_grounding.md",
|
|
2651
|
+
"size": 9588,
|
|
2652
|
+
"sha256": "56723e73b2b74299140d353921d1dba471bedf9aea0b8e769fc69995c4733995"
|
|
2653
|
+
},
|
|
2634
2654
|
{
|
|
2635
2655
|
"path": "skills/model-evaluation/scripts/check_metric_reporting.py",
|
|
2636
|
-
"size":
|
|
2637
|
-
"sha256": "
|
|
2656
|
+
"size": 9562,
|
|
2657
|
+
"sha256": "fd6c0205f652651a5a43910cb415ed8442100fc553ce04aa5d5905d97743a0bf"
|
|
2638
2658
|
},
|
|
2639
2659
|
{
|
|
2640
2660
|
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/clf_bad.md",
|
|
@@ -2646,6 +2666,16 @@
|
|
|
2646
2666
|
"size": 186,
|
|
2647
2667
|
"sha256": "ba44a3b38b4128fa713555c8b332f211e9087e6bc016d80af4c3074c3ca6ef8e"
|
|
2648
2668
|
},
|
|
2669
|
+
{
|
|
2670
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/det_good_wrapped.md",
|
|
2671
|
+
"size": 285,
|
|
2672
|
+
"sha256": "0c1f5ed5167969870602106e7a7c9a2ae53e3a3ddea5fa280c3beb6bf470c454"
|
|
2673
|
+
},
|
|
2674
|
+
{
|
|
2675
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/det_no_iou.md",
|
|
2676
|
+
"size": 224,
|
|
2677
|
+
"sha256": "08227fbe46829b8eecdfe15680e1bb32087f1e6c5a353b3df6055c67d8300fc0"
|
|
2678
|
+
},
|
|
2649
2679
|
{
|
|
2650
2680
|
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/seg_bad.md",
|
|
2651
2681
|
"size": 119,
|
|
@@ -2663,8 +2693,8 @@
|
|
|
2663
2693
|
},
|
|
2664
2694
|
{
|
|
2665
2695
|
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/verify.sh",
|
|
2666
|
-
"size":
|
|
2667
|
-
"sha256": "
|
|
2696
|
+
"size": 1663,
|
|
2697
|
+
"sha256": "4aa6db49d3552f01a54ddb70484df5d214806d805b836b24beda81d9aca32bd6"
|
|
2668
2698
|
},
|
|
2669
2699
|
{
|
|
2670
2700
|
"path": "skills/model-evaluation/skill.yml",
|
|
@@ -2718,8 +2748,13 @@
|
|
|
2718
2748
|
},
|
|
2719
2749
|
{
|
|
2720
2750
|
"path": "skills/model-validation/SKILL.md",
|
|
2721
|
-
"size":
|
|
2722
|
-
"sha256": "
|
|
2751
|
+
"size": 9810,
|
|
2752
|
+
"sha256": "1a75a5a1f21f5a8778d0b77db2e99574bf37edda2a291d8dde6aafea4a207ff0"
|
|
2753
|
+
},
|
|
2754
|
+
{
|
|
2755
|
+
"path": "skills/model-validation/references/validation_design.md",
|
|
2756
|
+
"size": 11427,
|
|
2757
|
+
"sha256": "16d43b688ea63745174c7ca8fafd78a7342b26c34ad1e10e1fdbc117cceafb2e"
|
|
2723
2758
|
},
|
|
2724
2759
|
{
|
|
2725
2760
|
"path": "skills/model-validation/scripts/check_split_leakage.py",
|
|
@@ -2769,7 +2804,7 @@
|
|
|
2769
2804
|
{
|
|
2770
2805
|
"path": "skills/orchestrate/SKILL.md",
|
|
2771
2806
|
"size": 35203,
|
|
2772
|
-
"sha256": "
|
|
2807
|
+
"sha256": "59b22bc48503a2b597bc4cb10dacf4e31e196c8f9e0e2e367752bb793425ab02"
|
|
2773
2808
|
},
|
|
2774
2809
|
{
|
|
2775
2810
|
"path": "skills/orchestrate/references/dialogue_nodes.md",
|
|
@@ -2793,8 +2828,8 @@
|
|
|
2793
2828
|
},
|
|
2794
2829
|
{
|
|
2795
2830
|
"path": "skills/peer-review/SKILL.md",
|
|
2796
|
-
"size":
|
|
2797
|
-
"sha256": "
|
|
2831
|
+
"size": 60277,
|
|
2832
|
+
"sha256": "168a9aac2bb0c00707c85c75f4c56bfdc90b9e705ceb30d7e1e331dfbf9f51e6"
|
|
2798
2833
|
},
|
|
2799
2834
|
{
|
|
2800
2835
|
"path": "skills/peer-review/references/aczel_2021_reviewer2_patterns.md",
|
|
@@ -2826,6 +2861,11 @@
|
|
|
2826
2861
|
"size": 7665,
|
|
2827
2862
|
"sha256": "83f2c5116317f96ba564f459f2b679489cc2ad8d2a3ec8078345ce1390680eb2"
|
|
2828
2863
|
},
|
|
2864
|
+
{
|
|
2865
|
+
"path": "skills/peer-review/references/domain-probes/health_economic_evaluation.md",
|
|
2866
|
+
"size": 8393,
|
|
2867
|
+
"sha256": "b322a9a6ae2439c1dc97f6a750572eecb3ce1481c139516665a55c9d571dfe8e"
|
|
2868
|
+
},
|
|
2829
2869
|
{
|
|
2830
2870
|
"path": "skills/peer-review/references/domain-probes/image_synthesis.md",
|
|
2831
2871
|
"size": 6295,
|
|
@@ -3333,8 +3373,8 @@
|
|
|
3333
3373
|
},
|
|
3334
3374
|
{
|
|
3335
3375
|
"path": "skills/self-review/SKILL.md",
|
|
3336
|
-
"size":
|
|
3337
|
-
"sha256": "
|
|
3376
|
+
"size": 94894,
|
|
3377
|
+
"sha256": "4145ec34a171e5783205bd2f11b62709302e1f86d6761b4cd9d545da492cbc75"
|
|
3338
3378
|
},
|
|
3339
3379
|
{
|
|
3340
3380
|
"path": "skills/self-review/references/domain-probes/ai_overclaiming.md",
|
|
@@ -3361,6 +3401,11 @@
|
|
|
3361
3401
|
"size": 7665,
|
|
3362
3402
|
"sha256": "83f2c5116317f96ba564f459f2b679489cc2ad8d2a3ec8078345ce1390680eb2"
|
|
3363
3403
|
},
|
|
3404
|
+
{
|
|
3405
|
+
"path": "skills/self-review/references/domain-probes/health_economic_evaluation.md",
|
|
3406
|
+
"size": 8393,
|
|
3407
|
+
"sha256": "b322a9a6ae2439c1dc97f6a750572eecb3ce1481c139516665a55c9d571dfe8e"
|
|
3408
|
+
},
|
|
3364
3409
|
{
|
|
3365
3410
|
"path": "skills/self-review/references/domain-probes/image_synthesis.md",
|
|
3366
3411
|
"size": 6295,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "medsci-skills",
|
|
3
|
-
"version": "5.
|
|
3
|
+
"version": "5.3.0",
|
|
4
4
|
"description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
|
|
5
5
|
"license": "SEE LICENSE IN LICENSE",
|
|
6
6
|
"homepage": "https://github.com/Aperivue/medsci-skills#readme",
|
|
@@ -456,6 +456,15 @@ tbl %>% as_flex_table() %>% flextable::save_as_docx(path = "table.docx")
|
|
|
456
456
|
- Present a **network plot** (node ∝ sample size, edge ∝ #trials); report global **τ²**; **ranking (SUCRA/P-score) is not a superiority test** — report it with the league table, intervals, and certainty
|
|
457
457
|
- Certainty **per estimate** via **CINeMA / GRADE-NMA** (downgrade indirect-only); component NMA assumes **additivity** (state/check it). Report against **PRISMA-NMA**; risk of bias via **RoB-NMA**. Review-side probes: NM1–NM8 in `network_meta_analysis.md`
|
|
458
458
|
|
|
459
|
+
### Health Economic Evaluation
|
|
460
|
+
|
|
461
|
+
- **Guide**: Load `analysis_guides/health_economic_evaluation.md` before generating code
|
|
462
|
+
- For cost-effectiveness (CEA), cost-utility (CUA, QALY), cost-benefit (CBA), cost-minimisation, or budget-impact analyses; trial-based or decision-model-based (decision tree, **Markov/state-transition**, discrete-event simulation)
|
|
463
|
+
- Compute **incremental cost ΔC, incremental effect ΔE, and the ICER = ΔC/ΔE**; with ≥3 options remove **dominated / extended-dominated** strategies before sequential ICERs; prefer **net benefit (INMB = λΔE − ΔC)** for regression/probabilistic summaries
|
|
464
|
+
- State and justify the **perspective, time horizon (lifetime for chronic disease), discount rate (both costs and outcomes), currency + price year**; QALYs from a named preference-based instrument + value set
|
|
465
|
+
- **Uncertainty is the analytic core**: one-way / **tornado** for drivers, **probabilistic sensitivity analysis (PSA)** with justified parameter distributions (beta for probabilities/utilities, gamma/log-normal for costs) → **cost-effectiveness plane + CEAC**; scenario analyses for structural choices
|
|
466
|
+
- R `heemod` / `dampack` / `hesim` / `BCEA` (state-transition + PSA + CEAC + EVPI), `flexsurv` for survival extrapolation. Report against **CHEERS 2022**; make the "cost-effective" conclusion conditional on a stated willingness-to-pay threshold. Review-side probes: HE1–HE8 in `health_economic_evaluation.md`
|
|
467
|
+
|
|
459
468
|
### Survey/Likert
|
|
460
469
|
|
|
461
470
|
- Descriptive: median, IQR, frequency distribution per item
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Health Economic Evaluation Analysis Guide
|
|
2
|
+
|
|
3
|
+
Comparing the **costs and consequences** of two or more interventions to inform a coverage,
|
|
4
|
+
adoption, or treatment decision. The headline — an **incremental cost-effectiveness ratio (ICER)**
|
|
5
|
+
— is arithmetically trivial; what fails review is **the structural choices behind it** (perspective,
|
|
6
|
+
time horizon, discounting, the effectiveness source, the cost basis, the model, and the propagation
|
|
7
|
+
of uncertainty). So the analysis is mostly the costing, the model, and the sensitivity suite, not the
|
|
8
|
+
ratio. This is the analysis-side companion to review probes **HE1–HE8** in
|
|
9
|
+
`health_economic_evaluation.md` and the **CHEERS 2022** reporting checklist.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## When to Use
|
|
14
|
+
|
|
15
|
+
- A comparative decision question (adopt / reimburse / treat) where both **cost** and **health
|
|
16
|
+
outcome** differ between options: cost-effectiveness (CEA, natural units — life-years, events
|
|
17
|
+
avoided), **cost-utility** (CUA, QALYs — the default for reimbursement), cost-benefit (CBA,
|
|
18
|
+
monetised outcomes), cost-minimisation (only when outcomes are demonstrably equivalent), or
|
|
19
|
+
**budget-impact** (affordability, distinct from cost-effectiveness).
|
|
20
|
+
- **Trial-based** (within-RCT patient-level costs and outcomes) or **decision-model-based** (decision
|
|
21
|
+
tree for short horizons; **Markov / state-transition** or **discrete-event simulation** when timing
|
|
22
|
+
and recurrence matter; lifetime horizons).
|
|
23
|
+
- NOT for: a costing/burden-of-illness description with no comparator (not an economic *evaluation*);
|
|
24
|
+
asserting "cost-effective" from a point ICER with no uncertainty analysis; cost-minimisation when a
|
|
25
|
+
non-inferiority outcome claim has not actually been established.
|
|
26
|
+
|
|
27
|
+
## Core quantities
|
|
28
|
+
|
|
29
|
+
- **Incremental cost** ΔC and **incremental effect** ΔE between an intervention and its comparator;
|
|
30
|
+
the **ICER = ΔC / ΔE** (e.g. cost per QALY gained). With ≥3 options, rank by cost, remove
|
|
31
|
+
**dominated** (more costly, less effective) and **extended-dominated** strategies, then compute
|
|
32
|
+
ICERs sequentially along the efficient frontier.
|
|
33
|
+
- **Net benefit** at a willingness-to-pay threshold λ: **INMB = λ·ΔE − ΔC** (monetary) or
|
|
34
|
+
**INHB = ΔE − ΔC/λ** (health). Net benefit is linear and avoids the ICER's quadrant ambiguity, so
|
|
35
|
+
it is preferred for regression and for probabilistic summaries.
|
|
36
|
+
- **QALYs** = time × **utility** (preference-based, 0=dead, 1=full health) from a named instrument
|
|
37
|
+
(EQ-5D-3L/5L, SF-6D, HUI) and a stated **value set/tariff** for the relevant country.
|
|
38
|
+
|
|
39
|
+
## The structural choices (state and justify each)
|
|
40
|
+
|
|
41
|
+
- **Perspective** — healthcare-system/payer vs **societal**; determines which costs count (societal
|
|
42
|
+
adds productivity and informal-care costs). Apply it consistently.
|
|
43
|
+
- **Time horizon** — long enough to capture all differential costs and effects; **lifetime** for
|
|
44
|
+
chronic disease or interventions with lasting effects. Extrapolation beyond trial data must be
|
|
45
|
+
modelled explicitly (e.g. parametric survival extrapolation) and tested.
|
|
46
|
+
- **Discounting** — apply the jurisdiction's reference-case rate (commonly ~3% or 3.5%) to **both
|
|
47
|
+
costs and outcomes**; sensitivity at alternative rates.
|
|
48
|
+
- **Costing** — report **resource quantities and unit costs separately**; state **currency, price
|
|
49
|
+
year**, and inflation/currency conversion. Match cost categories to the perspective.
|
|
50
|
+
- **Model** — justify structure against natural history; state cycle length and half-cycle
|
|
51
|
+
correction (Markov); validate (internal/face/external/predictive); test **structural** uncertainty
|
|
52
|
+
via scenario analysis.
|
|
53
|
+
|
|
54
|
+
## Uncertainty (the analytic core)
|
|
55
|
+
|
|
56
|
+
- **Deterministic** — one-way and **tornado** diagrams to find the drivers; multi-way / scenario
|
|
57
|
+
analyses for methodological and structural choices (discount rate, time horizon, alternative
|
|
58
|
+
model structures).
|
|
59
|
+
- **Probabilistic (PSA)** — assign each uncertain parameter a **distribution** (beta for
|
|
60
|
+
probabilities and utilities; gamma or log-normal for costs; Dirichlet for transition-probability
|
|
61
|
+
sets), propagate by **Monte Carlo**, and report the **cost-effectiveness plane** (the cloud of ΔC,
|
|
62
|
+
ΔE draws) and the **cost-effectiveness acceptability curve (CEAC)** — P(cost-effective) across a
|
|
63
|
+
range of λ. Report results as net benefit at the relevant threshold with its uncertainty, not a
|
|
64
|
+
bare point ICER.
|
|
65
|
+
- **Value of information** (EVPI/EVPPI) is an optional extension quantifying the expected cost of
|
|
66
|
+
current decision uncertainty / the priority parameters for further research.
|
|
67
|
+
|
|
68
|
+
## Reporting & tools
|
|
69
|
+
|
|
70
|
+
- Report to **CHEERS 2022** (28 items): perspective, horizon, discount rate, currency/price year,
|
|
71
|
+
model rationale, study parameters with distributions, disaggregated costs/outcomes, the ICER,
|
|
72
|
+
and the uncertainty analysis (plane + CEAC). State the **willingness-to-pay threshold** and make
|
|
73
|
+
the "cost-effective" conclusion conditional on it and on the CEAC probability.
|
|
74
|
+
- Tools: R — `heemod` / `dampack` / `hesim` (state-transition + PSA + CEAC), `BCEA` (Bayesian
|
|
75
|
+
cost-effectiveness, CEAC/EVPI), `survival`/`flexsurv` (survival extrapolation); also TreeAge, or
|
|
76
|
+
spreadsheet models with a documented PSA. Make the model or the health-economic analysis plan
|
|
77
|
+
available for scrutiny.
|
|
78
|
+
- Companion review probes: **HE1–HE8** (`peer-review`/`self-review`
|
|
79
|
+
`references/domain-probes/health_economic_evaluation.md`).
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: check-reporting
|
|
3
|
-
description: Check manuscript compliance with medical research reporting guidelines. Supports
|
|
4
|
-
triggers: checklist, reporting guideline, STROBE, STROBE-MR, Mendelian randomization, CONSORT, CONSORT-AI, STARD, STARD-AI, TRIPOD, TRIPOD-LLM, PGS-RS, PRS-RS, polygenic risk score, polygenic score, PRISMA, PRISMA-DTA, PRISMA-P, ARRIVE, CARE, CLAIM, DECIDE-AI, MI-CLEAR-LLM, SPIRIT, SPIRIT-AI, QUADAS, QUADAS-C, RoB, ROBINS, ROBINS-E, ROBIS, ROB-ME, PROBAST, NOS, COSMIN, AMSTAR, SWiM, risk of bias, compliance check, LLM accuracy, large language model, clinical deployment
|
|
3
|
+
description: Check manuscript compliance with medical research reporting guidelines. Supports 39 guidelines including STROBE, STROBE-MR, CONSORT, CONSORT-AI, STARD, STARD-AI, TRIPOD, TRIPOD+AI, TRIPOD-LLM, PGS-RS, ARRIVE, PRISMA, PRISMA-DTA, PRISMA-P, CARE, SPIRIT, SPIRIT-AI, CLAIM, DECIDE-AI, MI-CLEAR-LLM, SQUIRE 2.0, CLEAR, MOOSE, GRRAS, SWiM, AMSTAR 2, CHEERS 2022, and risk of bias tools (QUADAS-2, QUADAS-C, RoB 2, ROBINS-I, ROBINS-E, ROBIS, ROB-ME, PROBAST, PROBAST+AI, NOS, COSMIN, RoB NMA). Generates item-by-item assessment with PRESENT/MISSING/PARTIAL status.
|
|
4
|
+
triggers: checklist, reporting guideline, STROBE, STROBE-MR, Mendelian randomization, CONSORT, CONSORT-AI, STARD, STARD-AI, TRIPOD, TRIPOD-LLM, PGS-RS, PRS-RS, polygenic risk score, polygenic score, PRISMA, PRISMA-DTA, PRISMA-P, ARRIVE, CARE, CLAIM, DECIDE-AI, MI-CLEAR-LLM, SPIRIT, SPIRIT-AI, QUADAS, QUADAS-C, RoB, ROBINS, ROBINS-E, ROBIS, ROB-ME, PROBAST, NOS, COSMIN, AMSTAR, SWiM, CHEERS, economic evaluation, cost-effectiveness, cost-utility, QALY, ICER, risk of bias, compliance check, LLM accuracy, large language model, clinical deployment
|
|
5
5
|
tools: Read, Write, Edit, Bash, Grep, Glob
|
|
6
6
|
model: inherit
|
|
7
7
|
---
|
|
@@ -29,6 +29,7 @@ compliance report suitable for journal submission.
|
|
|
29
29
|
- `TRIPOD_AI.md` -- prediction models with AI/ML (CC BY 4.0, Collins et al. BMJ 2024)
|
|
30
30
|
- `TRIPOD_LLM.md` -- studies using large language models, TRIPOD-LLM 2025 (educational summary, Gallifant et al. Nat Med 2025)
|
|
31
31
|
- `PGS_RS.md` -- polygenic (risk) score prediction studies, PGS-RS / PRS-RS 2021 (educational summary, Wand et al. Nature 2021)
|
|
32
|
+
- `CHEERS_2022.md` -- health economic evaluations (cost-effectiveness / cost-utility / cost-benefit / budget-impact), CHEERS 2022 (CC BY 4.0, Husereau et al. BMJ 2022)
|
|
32
33
|
- `PRISMA_2020.md` -- systematic reviews (CC BY)
|
|
33
34
|
- `ARRIVE_2.md` -- animal studies (CC0)
|
|
34
35
|
- `PRISMA_DTA.md` -- DTA systematic reviews (CC BY, McInnes et al. JAMA 2018)
|
|
@@ -88,6 +89,7 @@ user specification.
|
|
|
88
89
|
|------------|------------------|--------------|
|
|
89
90
|
| Observational study | STROBE | -- |
|
|
90
91
|
| Mendelian randomization study | STROBE-MR (base STROBE + MR extension) | -- |
|
|
92
|
+
| Health economic evaluation (cost-effectiveness / cost-utility / cost-benefit / budget-impact) | CHEERS 2022 | -- |
|
|
91
93
|
| Randomized controlled trial | CONSORT 2025 | CONSORT-AI |
|
|
92
94
|
| Diagnostic accuracy study | STARD 2015 | STARD-AI |
|
|
93
95
|
| Prediction model (development/validation) | TRIPOD | TRIPOD+AI |
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# CHEERS 2022 Checklist
|
|
2
|
+
|
|
3
|
+
**Consolidated Health Economic Evaluation Reporting Standards 2022**
|
|
4
|
+
Version: CHEERS 2022 (28 items; replaces CHEERS 2013).
|
|
5
|
+
Source: Husereau D, Drummond M, Augustovski F, et al. *BMJ* 2022;376:e067975 (the CHEERS 2022 statement), co-published simultaneously across BMJ, *Value in Health*, *PharmacoEconomics*, *Int J Technol Assess Health Care* and others. CC BY 4.0. https://www.equator-network.org/reporting-guidelines/cheers/ · ISPOR CHEERS Task Force.
|
|
6
|
+
|
|
7
|
+
Apply when the manuscript is a **health economic evaluation** — a comparative analysis of costs and consequences of two or more courses of action: cost-effectiveness (CEA), cost-utility (CUA), cost-benefit (CBA), or cost-minimisation analysis, whether trial-based or decision-model-based (decision tree, Markov/state-transition, discrete-event simulation), including budget-impact and HTA submissions. For the design/validity review of the same study, pair with the HE1–HE8 domain probes in `peer-review` / `self-review` `references/domain-probes/health_economic_evaluation.md`; for the analysis, with `analyze-stats` `references/analysis_guides/health_economic_evaluation.md`.
|
|
8
|
+
|
|
9
|
+
## Checklist Items (28 items)
|
|
10
|
+
|
|
11
|
+
### Title
|
|
12
|
+
|
|
13
|
+
| # | Item | Description |
|
|
14
|
+
|---|------|-------------|
|
|
15
|
+
| 1 | Title | Identify the study as an economic evaluation and specify the interventions being compared. |
|
|
16
|
+
|
|
17
|
+
### Abstract
|
|
18
|
+
|
|
19
|
+
| # | Item | Description |
|
|
20
|
+
|---|------|-------------|
|
|
21
|
+
| 2 | Abstract | Provide a structured summary that highlights context, key methods, results, and alternative analyses. |
|
|
22
|
+
|
|
23
|
+
### Introduction
|
|
24
|
+
|
|
25
|
+
| # | Item | Description |
|
|
26
|
+
|---|------|-------------|
|
|
27
|
+
| 3 | Background and objectives | Give the context for the study, the study question, and its practical relevance for decision making in policy or practice. |
|
|
28
|
+
|
|
29
|
+
### Methods
|
|
30
|
+
|
|
31
|
+
| # | Item | Description |
|
|
32
|
+
|---|------|-------------|
|
|
33
|
+
| 4 | Health economic analysis plan | Indicate whether a health economic analysis plan was developed and where available. |
|
|
34
|
+
| 5 | Study population | Describe characteristics of the study population (such as age range, demographics, socioeconomic, or clinical characteristics). |
|
|
35
|
+
| 6 | Setting and location | Provide relevant contextual information that may influence findings. |
|
|
36
|
+
| 7 | Comparators | Describe the interventions or strategies being compared and why chosen. |
|
|
37
|
+
| 8 | Perspective | State the perspective(s) adopted by the study and why chosen. |
|
|
38
|
+
| 9 | Time horizon | State the time horizon for the study and why appropriate. |
|
|
39
|
+
| 10 | Discount rate | Report the discount rate(s) and reason chosen. |
|
|
40
|
+
| 11 | Selection of outcomes | Describe what outcomes were used as the measure(s) of benefit(s) and harm(s). |
|
|
41
|
+
| 12 | Measurement of outcomes | Describe how outcomes used to capture benefit(s) and harm(s) were measured. |
|
|
42
|
+
| 13 | Valuation of outcomes | Describe the population and methods used to measure and value outcomes. |
|
|
43
|
+
| 14 | Measurement and valuation of resources and costs | Describe how costs were valued. |
|
|
44
|
+
| 15 | Currency, price date, and conversion | Report the dates of the estimated resource quantities and unit costs, plus the currency and year of conversion. |
|
|
45
|
+
| 16 | Rationale and description of model | If modelling is used, describe in detail and why used. Report whether the model is publicly available and where. |
|
|
46
|
+
| 17 | Analytics and assumptions | Describe any methods for analysing or statistically transforming data, any extrapolation methods, and approaches for validating any model used. |
|
|
47
|
+
| 18 | Characterising heterogeneity | Describe any methods used for estimating how the results of the study vary for subgroups. |
|
|
48
|
+
| 19 | Characterising distributional effects | Describe how impacts are distributed across different individuals or whether adjustments were made to reflect priority populations. |
|
|
49
|
+
| 20 | Characterising uncertainty | Describe methods to characterise any sources of uncertainty in the analysis. |
|
|
50
|
+
| 21 | Approach to engagement with patients and others affected by the study | Describe any approaches to engage patients or service recipients, the general public, communities, or stakeholders (such as clinicians or payers) in the design of the study. |
|
|
51
|
+
|
|
52
|
+
### Results
|
|
53
|
+
|
|
54
|
+
| # | Item | Description |
|
|
55
|
+
|---|------|-------------|
|
|
56
|
+
| 22 | Study parameters | Report all analytic inputs (such as values, ranges, references) including uncertainty or distributional assumptions. |
|
|
57
|
+
| 23 | Summary of main results | Report the mean values for the main categories of costs and outcomes of interest and summarise them in the most appropriate overall measure (e.g. the incremental cost-effectiveness ratio, ICER). |
|
|
58
|
+
| 24 | Effect of uncertainty | Describe how uncertainty about analytic judgments, inputs, or projections affect findings. Report the effect of choice of discount rate and time horizon, if relevant. |
|
|
59
|
+
| 25 | Effect of engagement with patients and others affected by the study | Report on any difference patient/service recipient, general public, community, or stakeholder involvement made to the approach or findings of the study. |
|
|
60
|
+
|
|
61
|
+
### Discussion
|
|
62
|
+
|
|
63
|
+
| # | Item | Description |
|
|
64
|
+
|---|------|-------------|
|
|
65
|
+
| 26 | Study findings, limitations, generalisability, and current knowledge | Report key findings, limitations, ethical or equity considerations, and how these could affect patients, policy, or practice. |
|
|
66
|
+
|
|
67
|
+
### Other relevant information
|
|
68
|
+
|
|
69
|
+
| # | Item | Description |
|
|
70
|
+
|---|------|-------------|
|
|
71
|
+
| 27 | Source of funding | Describe how the study was funded and the role of the funder in the identification, design, conduct, and reporting of the analysis. Describe other non-monetary sources of support. |
|
|
72
|
+
| 28 | Conflicts of interest | Describe any potential for conflict of interest among study contributors in accordance with journal policy. In the absence of a journal policy, we recommend authors comply with International Committee of Medical Journal Editors (ICMJE) recommendations. |
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Notes for Assessors
|
|
77
|
+
|
|
78
|
+
- **Highest-yield items** (where economic evaluations most often fail review): **8** (perspective stated and consistent with the costs counted — productivity/informal-care costs belong only to a societal perspective), **9** (time horizon long enough to capture all relevant differential costs and effects — a lifetime horizon for a chronic condition; a truncated horizon flatters whichever arm has early benefit), **10** (both costs *and* outcomes discounted at a stated, justified rate for any horizon beyond ~1 year), **15** (currency *and* price year stated, with the conversion method for multi-source costs), **16–17** (model type/structure justified and validated; structural assumptions and extrapolation declared), and **20 / 24** (uncertainty characterised by *probabilistic* sensitivity analysis — a cost-effectiveness acceptability curve / plane — not a single deterministic ICER). An evaluation that reports a point-estimate ICER with no probabilistic sensitivity analysis is non-compliant on items 20/24.
|
|
79
|
+
- **CHEERS 2022 replaced CHEERS 2013**; cite the 2022 statement (do not cite the 2013 version as current). CHEERS 2022 added explicit items on the analysis plan (4), distributional/equity effects (19), and patient/stakeholder engagement (21/25); the engagement items are reported as "not done" rather than omitted when no engagement occurred.
|
|
80
|
+
- ICER interpretation is not a CHEERS item per se but follows from items 23–24: incremental costs and effects must be reported (not just the ratio), dominance/extended dominance resolved, and the ICER interpreted against a *stated, justified* cost-effectiveness threshold (willingness-to-pay) rather than an arbitrary one. The HE1–HE8 design probes cover these judgments.
|
|
81
|
+
- This checklist was authored as a faithful summary of the CHEERS 2022 statement (Husereau D, et al. *BMJ* 2022;376:e067975, **CC BY 4.0** — the item list and Explanation & Elaboration are reusable/adaptable with attribution) for item-by-item assessment; verify against the published statement and its Explanation & Elaboration for full item wording. Verified 2026-06-29.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Reporting Guideline → Figure Requirements Map
|
|
2
2
|
|
|
3
3
|
> **Bridge**: this file connects `/make-figures` to `/check-reporting`
|
|
4
|
-
> (
|
|
4
|
+
> (39 reporting guidelines). Each row tells you which figures the guideline
|
|
5
5
|
> **mandates** and how this skill currently supports them. Use during
|
|
6
6
|
> Step 1 (Specify) once the study type is known.
|
|
7
7
|
|
|
@@ -95,7 +95,7 @@ flags them:
|
|
|
95
95
|
|
|
96
96
|
## Cross-references
|
|
97
97
|
|
|
98
|
-
- `/check-reporting` skill — supports all
|
|
98
|
+
- `/check-reporting` skill — supports all 39 guidelines, item-level audit
|
|
99
99
|
- `flow_diagram_lessons.md` — production lessons that apply across all flows
|
|
100
100
|
- `pipeline_concepts_medical_ai.md` — DICOM / annotation / federated /
|
|
101
101
|
architecture diagram conventions
|
|
@@ -106,3 +106,11 @@ mllm-eval (this skill: harness design + completeness gate, model-agnostic)
|
|
|
106
106
|
├─ write-paper + check-reporting (TRIPOD-LLM / MI-CLEAR-LLM)
|
|
107
107
|
└─ self-review / peer-review (ME0–ME8 reviewer probe)
|
|
108
108
|
```
|
|
109
|
+
|
|
110
|
+
## Reference Files
|
|
111
|
+
|
|
112
|
+
- `${CLAUDE_SKILL_DIR}/references/evaluation_axes.md` — the *why* behind the ME2–ME7 axes:
|
|
113
|
+
clinical-efficacy metrics beyond n-gram overlap (e.g. RadGraph-F1 / CheXbert-F1 vs BLEU/ROUGE),
|
|
114
|
+
faithfulness & hallucination, pretraining/benchmark contamination, prompt-sensitivity &
|
|
115
|
+
determinism, answer-matching, and the reader study — each mapped to its gate verdict. Load on
|
|
116
|
+
demand during Phases 2–4.
|