medsci-skills 5.17.0 → 5.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -12
- package/metadata/distribution_files.json +94 -54
- package/metadata/distribution_manifest.json +1 -1
- package/package.json +1 -1
- package/skills/academic-aio/SKILL.md +3 -0
- package/skills/academic-aio/references/ai_tool_citation_framing.md +53 -0
- package/skills/design-study/SKILL.md +13 -0
- package/skills/make-figures/skill.yml +1 -0
- package/skills/make-figures/tests/test_pptx_mac_compat.py +148 -0
- package/skills/manage-refs/SKILL.md +7 -0
- package/skills/mllm-eval/references/evaluation_axes.md +9 -1
- package/skills/mllm-eval/scripts/check_mllm_eval_completeness.py +1 -1
- package/skills/model-evaluation/SKILL.md +21 -8
- package/skills/model-evaluation/references/metric_guide.md +48 -0
- package/skills/model-evaluation/references/metric_selection_grounding.md +7 -0
- package/skills/model-evaluation/scripts/check_metric_reporting.py +106 -3
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/generative_bad.md +2 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/generative_good.md +5 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/interactive_bad.md +2 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/interactive_good.md +6 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/fixture/multiclass_bad.md +2 -0
- package/skills/model-evaluation/scripts/metric_reporting_challenge/verify.sh +13 -1
- package/skills/model-evaluation/skill.yml +1 -1
- package/skills/model-evaluation/tests/test_metric_reporting.sh +42 -0
- package/skills/model-validation/SKILL.md +8 -1
- package/skills/peer-review/SKILL.md +15 -1
- package/skills/render-pdf-doc/SKILL.md +16 -3
- package/skills/render-pdf-doc/scripts/check_deps.sh +48 -17
- package/skills/render-pdf-doc/scripts/render_pdf.sh +35 -8
- package/skills/render-pdf-doc/skill.yml +1 -1
- package/skills/revise/SKILL.md +20 -0
- package/skills/revise/scripts/check_response_claims.py +233 -0
- package/skills/revise/skill.yml +6 -2
- package/skills/revise/tests/test_response_claims.sh +97 -0
- package/skills/sync-submission/SKILL.md +7 -0
- package/skills/sync-submission/scripts/check_cross_artifact_stale.py +62 -4
- package/skills/sync-submission/tests/test_cross_artifact_stale.sh +56 -0
- package/skills/verify-refs/references/manual_checkpoint_guide.md +4 -0
- package/skills/verify-refs/scripts/verify_refs.py +76 -34
- package/skills/verify-refs/skill.yml +1 -0
- package/skills/verify-refs/tests/test_fabricated_author.sh +73 -0
- package/skills/write-paper/SKILL.md +7 -0
- package/skills/write-paper/scripts/gate_backbone_fulltext.py +185 -0
- package/skills/write-paper/skill.yml +5 -1
- package/skills/write-paper/tests/test_backbone_fulltext.sh +85 -0
package/README.md
CHANGED
|
@@ -11,6 +11,7 @@
|
|
|
11
11
|
[](https://github.com/Aperivue/medsci-skills/actions/workflows/validate.yml)
|
|
12
12
|

|
|
13
13
|
[](https://www.npmjs.com/package/medsci-skills)
|
|
14
|
+
[](https://www.npmjs.com/package/medsci-skills)
|
|
14
15
|
[](https://youtu.be/MclQ_RIofpE)
|
|
15
16
|
[](https://github.com/Aperivue/medsci-skills/contribute)
|
|
16
17
|
|
|
@@ -41,8 +42,9 @@
|
|
|
41
42
|
|
|
42
43
|
## What is MedSci Skills?
|
|
43
44
|
|
|
44
|
-
MedSci Skills is an open-source
|
|
45
|
-
research — the manuscript and the medical-AI model alike
|
|
45
|
+
MedSci Skills is an open-source **Agent Skills** collection for **clinical
|
|
46
|
+
research — the manuscript and the medical-AI model alike** — designed to be driven
|
|
47
|
+
directly by AI coding agents (Claude Code, Codex, Cursor, and GitHub Copilot). It helps
|
|
46
48
|
physician-researchers and biomedical/medical-engineering investigators move from
|
|
47
49
|
literature search, study design, statistics, and figures to reporting-guideline
|
|
48
50
|
compliance, citation/reference auditing, numerical-consistency checks, and
|
|
@@ -155,15 +157,16 @@ protocol scaffolding, screening/extraction structure, PRISMA-consistent counts a
|
|
|
155
157
|
diagram, pooled-estimate figures, and a manuscript draft. *Safety:* screening and
|
|
156
158
|
extraction decisions stay with the human review team.
|
|
157
159
|
|
|
158
|
-
## Live Demos:
|
|
160
|
+
## Live Demos: Four Study Types, Four Full Pipelines
|
|
159
161
|
|
|
160
|
-
|
|
162
|
+
Four public datasets. Four study types. Demos 1–3 each produce a complete manuscript, publication-ready figures, and a reporting-compliance audit; Demo 4 runs the medical-AI **model-engineering lane** end to end (scaffold → gates → training → evaluation → interpretability).
|
|
161
163
|
|
|
162
164
|
| Demo | Dataset | Study Type | Compliance |
|
|
163
165
|
|------|---------|------------|------------|
|
|
164
166
|
| [Demo 1: Wisconsin BC](demo/01_wisconsin_bc/) | `sklearn` built-in | Diagnostic accuracy | STARD 2015 |
|
|
165
167
|
| [Demo 2: BCG Vaccine](demo/02_metafor_bcg/) | `metafor::dat.bcg` (13 RCTs) | Meta-analysis | PRISMA 2020 |
|
|
166
168
|
| [Demo 3: NHANES Obesity](demo/03_nhanes_obesity/) | CDC NHANES 2017-18 | Epidemiology (survey) | STROBE |
|
|
169
|
+
| [Demo 4: PneumoniaMNIST CNN](demo/04_pneumoniamnist_cnn/) | `medmnist` (CC BY 4.0) | Medical-AI model engineering (CNN) | CLAIM / TRIPOD+AI |
|
|
167
170
|
|
|
168
171
|
### Demo 1: Diagnostic Accuracy — Wisconsin Breast Cancer
|
|
169
172
|
|
|
@@ -285,6 +288,8 @@ The E2E pipeline (`orchestrate --e2e`) produces everything up to `qc/`. The `sub
|
|
|
285
288
|
|
|
286
289
|
## What's New
|
|
287
290
|
|
|
291
|
+
**v5.18** — reliability & workflow-integrity batch. A new deterministic **response-claim gate** for `/revise` and `/peer-review` (verifies a response letter's "we added / we now cite X" against the *revised manuscript body* — a claimed-but-absent edit is caught before it reaches a reviewer; **detectors 50 → 51**), a **reframe / headline-change survivor scan** (`--retired-term` / `--old-value`, finds stale framing or superseded values left in the body, supplement, and figure legends after a reframe), a pre-drafting **backbone full-text readiness gate** for `/write-paper` (the backbone article is *used*, not merely detected), a **skill-registry consistency validator** (`capabilities.yml` ⇄ `skill.yml`, CI-enforced), AI-tool **citation-framing** guidance for `/academic-aio`, and **Demo 4** (PneumoniaMNIST CNN, the model-engineering lane end to end). Additive and backward-compatible; **55 skills / 44 guidelines / 51 integrity detectors / 22 domain-probe modules**. (See the [CHANGELOG](CHANGELOG.md) for v5.0–v5.17.)
|
|
292
|
+
|
|
288
293
|
**v5.17** — model-engineering produce-side depth, **completion**. Deployment safety + the final roadmap/candidate items: a new **`uncertainty-imaging`** skill + `check_uncertainty_reporting` gate (calibrated per-case uncertainty / OOD guard on a held-out set / abstention at a pre-specified target / calibration-under-shift — for a deployment-framed model), an **MLOps wiring reference** for `model-scaffold` (experiment tracking, config/data/environment versioning, CI-for-ML — pointing to W&B / MLflow / nnU-Net, reimplementing nothing), and an **`architecture-zoo` graph-neural-net family card** (GCN / GraphSAGE / GAT / GIN / BrainGNN for brain connectomes) that closes the last candidate gap. The six-item produce-side depth roadmap and its candidate list are now complete. Additive and backward-compatible; **55 skills / 44 guidelines / 50 integrity detectors / 22 domain-probe modules**. (See the [CHANGELOG](CHANGELOG.md) for v5.0–v5.16.)
|
|
289
294
|
|
|
290
295
|
**v5.16** — model-engineering produce-side depth, clinical fine-tuning focus (Items 3–4 of the [produce-side depth roadmap](docs/roadmap_model_engineering_depth.md)). A new **`radiomics-ml`** skill + `check_radiomics_ml` gate for the most common solo-doable clinical-ML workflow — radiomics / tabular features → any classical learner (LASSO / SVM / random forest / XGBoost / …) → a clinical outcome, no GPU — with a learner-agnostic nested-CV / calibration / stability gate; plus a **`model-scaffold` fine-tuning mode** (`--task finetune` + `--from-pretrained`) that adapts a pretrained backbone on collected clinical data with a frozen→unfrozen schedule, discriminative learning rates, and a pretrained-weight provenance record (`PRETRAINED.md` + a `PRETRAINED_PROVENANCE_MISSING` verdict on the existing `check_training_hygiene`, plus a MedSAM-adaptation + train-only diffusion-augmentation guide). Additive and backward-compatible; **54 skills / 44 guidelines / 49 integrity detectors / 22 domain-probe modules**. (See the [CHANGELOG](CHANGELOG.md) for v5.0–v5.15.)
|
|
@@ -647,7 +652,7 @@ matrices without CSV-coordinate comments are flagged as structural risks even wh
|
|
|
647
652
|
are currently correct, since the next revision will re-introduce the same failure mode.
|
|
648
653
|
|
|
649
654
|
### Reference Safety (Phase 1)
|
|
650
|
-
Projects declare their source-of-truth layout in `SSOT.yaml`, and a `qc/migration_complete` marker gates strict enforcement. `/verify-refs` is the sole writer of `qc/reference_audit.json`. The `MEDSCI_VERIFY_REFS_MODE` env var (`auto` default, `warn`, `enforce`, `off`) controls behavior — `auto` blocks only when both SSOT.yaml and the migration marker are present, otherwise warns. Legacy projects freeze as warn-only; new projects opt in via `scripts/migrate_project_to_ssot.py`. An optional PostToolUse hook (not shipped in this repo — document only) can invoke `/verify-refs` automatically on manuscript saves for users who install it locally at `~/.claude/hooks/verify-refs-guard.sh`; the regression suite (`tests/test_phase1c_hooks.sh`) runs end-to-end only when that local hook is present and is skipped otherwise.
|
|
655
|
+
Projects declare their source-of-truth layout in `SSOT.yaml`, and a `qc/migration_complete` marker gates strict enforcement. `/verify-refs` is the sole writer of `qc/reference_audit.json`. The `MEDSCI_VERIFY_REFS_MODE` env var (`auto` default, `warn`, `enforce`, `off`) controls behavior — `auto` blocks only when both SSOT.yaml and the migration marker are present, otherwise warns. Legacy projects freeze as warn-only; new projects opt in via `scripts/migrate_project_to_ssot.py`. An optional PostToolUse hook (not shipped in this repo — document only) can invoke `/verify-refs` automatically on manuscript saves for users who install it locally at `~/.claude/hooks/verify-refs-guard.sh`; the regression suite (`tests/test_phase1c_hooks.sh`) runs end-to-end only when that local hook is present and is skipped otherwise. The hook gates saves under `*/submission/*/manuscript/*.{docx,md}` and `*/revision/R*/…circulation….docx` (enforce-eligible), and — added for issue #14 — also **warn-only** on pre-submission and mentor-circulation drafts that previously skipped the audit entirely: `*/outgoing/*.{docx,md}`, `*/8_Review_Comments/*/outgoing/*.{docx,md}`, and any `*/circulation/*.{docx,md}`. Warn-only patterns surface a missing audit without blocking rapid iteration and never enforce, regardless of SSOT/migration state.
|
|
651
656
|
|
|
652
657
|
### Meta-Analysis Failure Modes
|
|
653
658
|
`/meta-analysis` ships empirical failure-mode references (data integrity, review orchestration, submission package drift, post-submission release ops) with four automation hooks: `scripts/prisma_5way_consistency.py` (DI-6 PRISMA number consistency), `scripts/extraction_consensus_log_init.py` (DI-1 dual-extraction scaffold), `scripts/tag_cleanup_gate.sh` (DI-8 placeholder tag gate), and `scripts/verify_package_integrity.py` (SPD SHA-256 manifest for submission bundles).
|
|
@@ -782,12 +787,18 @@ from one of these:
|
|
|
782
787
|
|
|
783
788
|
| Want to add… | How | Issue |
|
|
784
789
|
|---|---|---|
|
|
785
|
-
| **A
|
|
790
|
+
| **A CSL citation style** for a journal that lacks one | drop one `.csl` into `manage-refs/citation_styles/` | [#117](https://github.com/Aperivue/medsci-skills/issues/117) |
|
|
791
|
+
| **A de-identification locale pack** for one more country | add one patterns file under `deidentify/` | [#116](https://github.com/Aperivue/medsci-skills/issues/116) |
|
|
792
|
+
| **A README translation** (e.g., zh-CN) | one translated `README` file | [#119](https://github.com/Aperivue/medsci-skills/issues/119) |
|
|
786
793
|
| **A figure exemplar** (ROC, KM, forest, Bland–Altman, confusion matrix…) | one `make-figures/references/exemplar_plots/*.md` anatomy model | [#118](https://github.com/Aperivue/medsci-skills/issues/118) |
|
|
787
|
-
| **A
|
|
788
|
-
| **A de-identification locale pack** for one more country | add patterns to `deidentify/` | [#116](https://github.com/Aperivue/medsci-skills/issues/116) |
|
|
794
|
+
| **A journal profile** (submission rules for a journal we don't cover) | `/add-journal`, or copy an existing `journal_profiles/*.md` | [#115](https://github.com/Aperivue/medsci-skills/issues/115) |
|
|
789
795
|
| **A reporting checklist or peer-review exemplar** | one reference file in the matching skill | [#120](https://github.com/Aperivue/medsci-skills/issues/120) |
|
|
790
|
-
|
|
796
|
+
|
|
797
|
+
Each of these adds **exactly one new file** — none requires editing a count, a generated
|
|
798
|
+
page, or the build config, and the catalog-consistency check auto-derives profile counts from
|
|
799
|
+
disk so it won't flag you. **No assignment needed: just open a PR — the first one that passes
|
|
800
|
+
CI wins, and a maintainer handles any bookkeeping in review.** See the
|
|
801
|
+
[5-minute first-PR quickstart](CONTRIBUTING.md#quickstart-your-first-pr-5-minutes) to get going.
|
|
791
802
|
|
|
792
803
|
Every contribution is gated the same way the maintainers are: it must be a self-contained
|
|
793
804
|
file, pass the CI (`validate.yml` — PII scan, structure, catalog consistency), and carry no
|
|
@@ -801,6 +812,8 @@ or a [detector request](https://github.com/Aperivue/medsci-skills/issues/new?tem
|
|
|
801
812
|
[`docs/maintainer_workflow.md`](docs/maintainer_workflow.md) (review + release process),
|
|
802
813
|
and [`SECURITY.md`](SECURITY.md) (vulnerability reporting + the medical-scope boundary).
|
|
803
814
|
A change that touches a medical/research claim needs Clinical-Lead review.
|
|
815
|
+
Community members who have reported, verified, or shaped the project are credited in
|
|
816
|
+
[`CONTRIBUTORS.md`](CONTRIBUTORS.md).
|
|
804
817
|
|
|
805
818
|
## In the Wild
|
|
806
819
|
|
|
@@ -836,9 +849,9 @@ software citation is how a tool like this earns academic recognition, and it tak
|
|
|
836
849
|
}
|
|
837
850
|
|
|
838
851
|
@article{nam2026agentic,
|
|
839
|
-
author = {Nam, Yoojin and Kim, Namkug},
|
|
840
|
-
title = {{
|
|
841
|
-
An
|
|
852
|
+
author = {Nam, Yoojin and Jeong, Jinhoon and Kim, Namkug},
|
|
853
|
+
title = {{Deterministic Integrity Gates for LLM-Assisted Clinical Manuscript
|
|
854
|
+
Preparation: An Auditable Biomedical Informatics Architecture}},
|
|
842
855
|
year = {2026},
|
|
843
856
|
journal = {arXiv preprint arXiv:2606.09500},
|
|
844
857
|
url = {https://arxiv.org/abs/2606.09500}
|
|
@@ -58,8 +58,13 @@
|
|
|
58
58
|
},
|
|
59
59
|
{
|
|
60
60
|
"path": "skills/academic-aio/SKILL.md",
|
|
61
|
-
"size":
|
|
62
|
-
"sha256": "
|
|
61
|
+
"size": 32276,
|
|
62
|
+
"sha256": "d893714e70af45b6f9d162d5220875685995e4a2c428331348689566e672989c"
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
"path": "skills/academic-aio/references/ai_tool_citation_framing.md",
|
|
66
|
+
"size": 4064,
|
|
67
|
+
"sha256": "e3c8403cc698a45bb5626b233c5cbf18eb283e6a606d6232471f0f0c657d0fec"
|
|
63
68
|
},
|
|
64
69
|
{
|
|
65
70
|
"path": "skills/academic-aio/references/case_studies/kjr_mllm_2025.md",
|
|
@@ -1033,8 +1038,8 @@
|
|
|
1033
1038
|
},
|
|
1034
1039
|
{
|
|
1035
1040
|
"path": "skills/design-study/SKILL.md",
|
|
1036
|
-
"size":
|
|
1037
|
-
"sha256": "
|
|
1041
|
+
"size": 20296,
|
|
1042
|
+
"sha256": "98644b7750b65ff9bb38af7cb6e0b3300169f498332a3f143394cc35f5ba7ff7"
|
|
1038
1043
|
},
|
|
1039
1044
|
{
|
|
1040
1045
|
"path": "skills/design-study/references/dag_adjustment.md",
|
|
@@ -2268,8 +2273,8 @@
|
|
|
2268
2273
|
},
|
|
2269
2274
|
{
|
|
2270
2275
|
"path": "skills/make-figures/skill.yml",
|
|
2271
|
-
"size":
|
|
2272
|
-
"sha256": "
|
|
2276
|
+
"size": 2249,
|
|
2277
|
+
"sha256": "6a8ebbe077c2073bb546a044bbecb5d483dbe2d2a50b31adabba6cf5e3a87990"
|
|
2273
2278
|
},
|
|
2274
2279
|
{
|
|
2275
2280
|
"path": "skills/make-figures/templates/official/NOTES.md",
|
|
@@ -2373,8 +2378,8 @@
|
|
|
2373
2378
|
},
|
|
2374
2379
|
{
|
|
2375
2380
|
"path": "skills/manage-refs/SKILL.md",
|
|
2376
|
-
"size":
|
|
2377
|
-
"sha256": "
|
|
2381
|
+
"size": 18627,
|
|
2382
|
+
"sha256": "abb0f38acbff93ffdff68be6a29b68757027fd7ffec545abd90df2df4113c08c"
|
|
2378
2383
|
},
|
|
2379
2384
|
{
|
|
2380
2385
|
"path": "skills/manage-refs/citation_styles/README.md",
|
|
@@ -2718,13 +2723,13 @@
|
|
|
2718
2723
|
},
|
|
2719
2724
|
{
|
|
2720
2725
|
"path": "skills/mllm-eval/references/evaluation_axes.md",
|
|
2721
|
-
"size":
|
|
2722
|
-
"sha256": "
|
|
2726
|
+
"size": 11609,
|
|
2727
|
+
"sha256": "0bad98d58685c638132c891c7c4d8b3c4620fe0988994d3307458382167d3d92"
|
|
2723
2728
|
},
|
|
2724
2729
|
{
|
|
2725
2730
|
"path": "skills/mllm-eval/scripts/check_mllm_eval_completeness.py",
|
|
2726
|
-
"size":
|
|
2727
|
-
"sha256": "
|
|
2731
|
+
"size": 9386,
|
|
2732
|
+
"sha256": "bb714a45449a26ccafca9e988a95aaa6af92fb8f3b79cfc03f8b47768f37aaec"
|
|
2728
2733
|
},
|
|
2729
2734
|
{
|
|
2730
2735
|
"path": "skills/mllm-eval/scripts/mllm_eval_completeness_challenge/fixture/plan_bad.md",
|
|
@@ -2808,23 +2813,23 @@
|
|
|
2808
2813
|
},
|
|
2809
2814
|
{
|
|
2810
2815
|
"path": "skills/model-evaluation/SKILL.md",
|
|
2811
|
-
"size":
|
|
2812
|
-
"sha256": "
|
|
2816
|
+
"size": 7380,
|
|
2817
|
+
"sha256": "b39293e54402fa79ac73de827201f4cd98bb2cc70fd1cd4fc8da4e940d429b3b"
|
|
2813
2818
|
},
|
|
2814
2819
|
{
|
|
2815
2820
|
"path": "skills/model-evaluation/references/metric_guide.md",
|
|
2816
|
-
"size":
|
|
2817
|
-
"sha256": "
|
|
2821
|
+
"size": 6194,
|
|
2822
|
+
"sha256": "b8cb061cf40b0736961a4f06eb7f6b60f238a0e19f7b47e07f3cb38a82097f52"
|
|
2818
2823
|
},
|
|
2819
2824
|
{
|
|
2820
2825
|
"path": "skills/model-evaluation/references/metric_selection_grounding.md",
|
|
2821
|
-
"size":
|
|
2822
|
-
"sha256": "
|
|
2826
|
+
"size": 10256,
|
|
2827
|
+
"sha256": "535282ae5cfbf2593c3e40926143e2ebd081ebbd78ac972cb0e6aeae839d1e77"
|
|
2823
2828
|
},
|
|
2824
2829
|
{
|
|
2825
2830
|
"path": "skills/model-evaluation/scripts/check_metric_reporting.py",
|
|
2826
|
-
"size":
|
|
2827
|
-
"sha256": "
|
|
2831
|
+
"size": 17964,
|
|
2832
|
+
"sha256": "f1315745ceba694cb2437a412618d86cd7f0a7f4e652f5ced2cc207b67eaa731"
|
|
2828
2833
|
},
|
|
2829
2834
|
{
|
|
2830
2835
|
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/clf_bad.md",
|
|
@@ -2846,6 +2851,31 @@
|
|
|
2846
2851
|
"size": 224,
|
|
2847
2852
|
"sha256": "08227fbe46829b8eecdfe15680e1bb32087f1e6c5a353b3df6055c67d8300fc0"
|
|
2848
2853
|
},
|
|
2854
|
+
{
|
|
2855
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/generative_bad.md",
|
|
2856
|
+
"size": 134,
|
|
2857
|
+
"sha256": "f4ac7e4d2d6dd12125e832a55363579181546d68411f49984680f334db5c8cb4"
|
|
2858
|
+
},
|
|
2859
|
+
{
|
|
2860
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/generative_good.md",
|
|
2861
|
+
"size": 437,
|
|
2862
|
+
"sha256": "d218437b09d8d80fa13aa52a03c65fd6156d95e99204762cafaa7021c4b4b21f"
|
|
2863
|
+
},
|
|
2864
|
+
{
|
|
2865
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/interactive_bad.md",
|
|
2866
|
+
"size": 128,
|
|
2867
|
+
"sha256": "be714934c18566a89c6c7b06951a420d8e2cbfca814e1daf26df87866252a185"
|
|
2868
|
+
},
|
|
2869
|
+
{
|
|
2870
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/interactive_good.md",
|
|
2871
|
+
"size": 500,
|
|
2872
|
+
"sha256": "a45e14485fb02f0ee790350235c652ea2b871aa16cfd89e12a32e49cd6e1b9ca"
|
|
2873
|
+
},
|
|
2874
|
+
{
|
|
2875
|
+
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/multiclass_bad.md",
|
|
2876
|
+
"size": 128,
|
|
2877
|
+
"sha256": "0ab44d8dcdf4ec8281c7afb7db021e61bb368f2f30bd5ec49ed5bcf962319421"
|
|
2878
|
+
},
|
|
2849
2879
|
{
|
|
2850
2880
|
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/fixture/seg_bad.md",
|
|
2851
2881
|
"size": 119,
|
|
@@ -2863,13 +2893,13 @@
|
|
|
2863
2893
|
},
|
|
2864
2894
|
{
|
|
2865
2895
|
"path": "skills/model-evaluation/scripts/metric_reporting_challenge/verify.sh",
|
|
2866
|
-
"size":
|
|
2867
|
-
"sha256": "
|
|
2896
|
+
"size": 2642,
|
|
2897
|
+
"sha256": "3efa9aff1f4ed223b7114740cac9cb67801456ffbfa3e5f7a7f1df7f46ada565"
|
|
2868
2898
|
},
|
|
2869
2899
|
{
|
|
2870
2900
|
"path": "skills/model-evaluation/skill.yml",
|
|
2871
|
-
"size":
|
|
2872
|
-
"sha256": "
|
|
2901
|
+
"size": 2944,
|
|
2902
|
+
"sha256": "d055072bcfb86d84b90f0b29790a262a496a95e4fe8ffbf38d02a28bf9db258e"
|
|
2873
2903
|
},
|
|
2874
2904
|
{
|
|
2875
2905
|
"path": "skills/model-scaffold/SKILL.md",
|
|
@@ -2928,8 +2958,8 @@
|
|
|
2928
2958
|
},
|
|
2929
2959
|
{
|
|
2930
2960
|
"path": "skills/model-validation/SKILL.md",
|
|
2931
|
-
"size":
|
|
2932
|
-
"sha256": "
|
|
2961
|
+
"size": 11524,
|
|
2962
|
+
"sha256": "ac326c03451282a5adeeb3592f511dac47720bba9338dba761bf6ea4679d7c0b"
|
|
2933
2963
|
},
|
|
2934
2964
|
{
|
|
2935
2965
|
"path": "skills/model-validation/references/validation_design.md",
|
|
@@ -3008,8 +3038,8 @@
|
|
|
3008
3038
|
},
|
|
3009
3039
|
{
|
|
3010
3040
|
"path": "skills/peer-review/SKILL.md",
|
|
3011
|
-
"size":
|
|
3012
|
-
"sha256": "
|
|
3041
|
+
"size": 70338,
|
|
3042
|
+
"sha256": "246beb8a3c3c6eef78180a6ab1d87fa49db4f98d55964af5b3ab0ce5d0e7e3b0"
|
|
3013
3043
|
},
|
|
3014
3044
|
{
|
|
3015
3045
|
"path": "skills/peer-review/references/aczel_2021_reviewer2_patterns.md",
|
|
@@ -3478,8 +3508,8 @@
|
|
|
3478
3508
|
},
|
|
3479
3509
|
{
|
|
3480
3510
|
"path": "skills/render-pdf-doc/SKILL.md",
|
|
3481
|
-
"size":
|
|
3482
|
-
"sha256": "
|
|
3511
|
+
"size": 8300,
|
|
3512
|
+
"sha256": "fad21da8e60907c212b6d216ffe591502941c8defa4ce212b521306e00332836"
|
|
3483
3513
|
},
|
|
3484
3514
|
{
|
|
3485
3515
|
"path": "skills/render-pdf-doc/references/known_pitfalls.md",
|
|
@@ -3493,8 +3523,8 @@
|
|
|
3493
3523
|
},
|
|
3494
3524
|
{
|
|
3495
3525
|
"path": "skills/render-pdf-doc/scripts/check_deps.sh",
|
|
3496
|
-
"size":
|
|
3497
|
-
"sha256": "
|
|
3526
|
+
"size": 2183,
|
|
3527
|
+
"sha256": "782efdda541c0a84a8d978bb076d43a1d3a026a00b878205dd6d922a197186f6"
|
|
3498
3528
|
},
|
|
3499
3529
|
{
|
|
3500
3530
|
"path": "skills/render-pdf-doc/scripts/infer_colwidths.py",
|
|
@@ -3503,8 +3533,8 @@
|
|
|
3503
3533
|
},
|
|
3504
3534
|
{
|
|
3505
3535
|
"path": "skills/render-pdf-doc/scripts/render_pdf.sh",
|
|
3506
|
-
"size":
|
|
3507
|
-
"sha256": "
|
|
3536
|
+
"size": 3958,
|
|
3537
|
+
"sha256": "b5a9ebcbb25b1ec85577c9864f60f4f363e90f88896903cff48f7626f5b6b95a"
|
|
3508
3538
|
},
|
|
3509
3539
|
{
|
|
3510
3540
|
"path": "skills/render-pdf-doc/scripts/scan_glyph_coverage.py",
|
|
@@ -3513,8 +3543,8 @@
|
|
|
3513
3543
|
},
|
|
3514
3544
|
{
|
|
3515
3545
|
"path": "skills/render-pdf-doc/skill.yml",
|
|
3516
|
-
"size":
|
|
3517
|
-
"sha256": "
|
|
3546
|
+
"size": 3133,
|
|
3547
|
+
"sha256": "77fe112e5a013d3169290132f9882fbb05b3ebe9317729960582895509c21568"
|
|
3518
3548
|
},
|
|
3519
3549
|
{
|
|
3520
3550
|
"path": "skills/render-pdf-doc/templates/anchor-doc.md",
|
|
@@ -3598,18 +3628,23 @@
|
|
|
3598
3628
|
},
|
|
3599
3629
|
{
|
|
3600
3630
|
"path": "skills/revise/SKILL.md",
|
|
3601
|
-
"size":
|
|
3602
|
-
"sha256": "
|
|
3631
|
+
"size": 28904,
|
|
3632
|
+
"sha256": "c1851293ea0f59c658a1a5b4b303667de3e55f693ee5fddc2148d7245bab60ea"
|
|
3603
3633
|
},
|
|
3604
3634
|
{
|
|
3605
3635
|
"path": "skills/revise/references/r2r_voice.md",
|
|
3606
3636
|
"size": 17042,
|
|
3607
3637
|
"sha256": "d6e657ab4d864e0c12c6d3c8e7db5060dfcea0c56d0f9afccd876702f8b8adcf"
|
|
3608
3638
|
},
|
|
3639
|
+
{
|
|
3640
|
+
"path": "skills/revise/scripts/check_response_claims.py",
|
|
3641
|
+
"size": 9798,
|
|
3642
|
+
"sha256": "ef5e67cf757e146b43c578753fcafd10632676b08eec00328eb5100e20dc79b6"
|
|
3643
|
+
},
|
|
3609
3644
|
{
|
|
3610
3645
|
"path": "skills/revise/skill.yml",
|
|
3611
|
-
"size":
|
|
3612
|
-
"sha256": "
|
|
3646
|
+
"size": 2261,
|
|
3647
|
+
"sha256": "421670bd78193e01966d08d08b22f044a6c396cba0bed99ae50fbb8f6c6564a5"
|
|
3613
3648
|
},
|
|
3614
3649
|
{
|
|
3615
3650
|
"path": "skills/search-lit/SKILL.md",
|
|
@@ -3953,8 +3988,8 @@
|
|
|
3953
3988
|
},
|
|
3954
3989
|
{
|
|
3955
3990
|
"path": "skills/sync-submission/SKILL.md",
|
|
3956
|
-
"size":
|
|
3957
|
-
"sha256": "
|
|
3991
|
+
"size": 28938,
|
|
3992
|
+
"sha256": "deb2d8b9d9939e84f2a0eaf57584b900b62eadf1d398ea60a49e3720d7ab174b"
|
|
3958
3993
|
},
|
|
3959
3994
|
{
|
|
3960
3995
|
"path": "skills/sync-submission/references/journal_availability_policy.json",
|
|
@@ -3993,8 +4028,8 @@
|
|
|
3993
4028
|
},
|
|
3994
4029
|
{
|
|
3995
4030
|
"path": "skills/sync-submission/scripts/check_cross_artifact_stale.py",
|
|
3996
|
-
"size":
|
|
3997
|
-
"sha256": "
|
|
4031
|
+
"size": 11679,
|
|
4032
|
+
"sha256": "613cdbab508fa2767e7b445624829d645c3a917e8f3d58a72dd66bcac7fd0586"
|
|
3998
4033
|
},
|
|
3999
4034
|
{
|
|
4000
4035
|
"path": "skills/sync-submission/scripts/check_disclosure_availability.py",
|
|
@@ -4098,8 +4133,8 @@
|
|
|
4098
4133
|
},
|
|
4099
4134
|
{
|
|
4100
4135
|
"path": "skills/verify-refs/references/manual_checkpoint_guide.md",
|
|
4101
|
-
"size":
|
|
4102
|
-
"sha256": "
|
|
4136
|
+
"size": 4031,
|
|
4137
|
+
"sha256": "4da6bd649d672bcae7505098d4712d6fb253fea8fa3be41c95870587cc6553a4"
|
|
4103
4138
|
},
|
|
4104
4139
|
{
|
|
4105
4140
|
"path": "skills/verify-refs/scripts/verify_cli.sh",
|
|
@@ -4108,13 +4143,13 @@
|
|
|
4108
4143
|
},
|
|
4109
4144
|
{
|
|
4110
4145
|
"path": "skills/verify-refs/scripts/verify_refs.py",
|
|
4111
|
-
"size":
|
|
4112
|
-
"sha256": "
|
|
4146
|
+
"size": 43072,
|
|
4147
|
+
"sha256": "ce0db6183942afdb2dfd55733c8b1330dee1b10ce091ce99c31497f83d9c4e0b"
|
|
4113
4148
|
},
|
|
4114
4149
|
{
|
|
4115
4150
|
"path": "skills/verify-refs/skill.yml",
|
|
4116
|
-
"size":
|
|
4117
|
-
"sha256": "
|
|
4151
|
+
"size": 2291,
|
|
4152
|
+
"sha256": "608e80234418a3560c9662e83a786b20ad16a340b934bfea59e2421712595fc5"
|
|
4118
4153
|
},
|
|
4119
4154
|
{
|
|
4120
4155
|
"path": "skills/version-dataset/SKILL.md",
|
|
@@ -4138,8 +4173,8 @@
|
|
|
4138
4173
|
},
|
|
4139
4174
|
{
|
|
4140
4175
|
"path": "skills/write-paper/SKILL.md",
|
|
4141
|
-
"size":
|
|
4142
|
-
"sha256": "
|
|
4176
|
+
"size": 68059,
|
|
4177
|
+
"sha256": "d05e1bbde228a87abf459ec19484e560dc08b7e88213e36655b680ebecf7dd8f"
|
|
4143
4178
|
},
|
|
4144
4179
|
{
|
|
4145
4180
|
"path": "skills/write-paper/references/exemplar_abstract.md",
|
|
@@ -4631,10 +4666,15 @@
|
|
|
4631
4666
|
"size": 7437,
|
|
4632
4667
|
"sha256": "6bc4032302265ba2c834212e77fbe78b8ddeddba716c833ecf8c65e85e586320"
|
|
4633
4668
|
},
|
|
4669
|
+
{
|
|
4670
|
+
"path": "skills/write-paper/scripts/gate_backbone_fulltext.py",
|
|
4671
|
+
"size": 8358,
|
|
4672
|
+
"sha256": "fa06f0d38c59b319a6cb892f4fb0beeccbf147ec4377a87d0f4b35a49d607050"
|
|
4673
|
+
},
|
|
4634
4674
|
{
|
|
4635
4675
|
"path": "skills/write-paper/skill.yml",
|
|
4636
|
-
"size":
|
|
4637
|
-
"sha256": "
|
|
4676
|
+
"size": 2466,
|
|
4677
|
+
"sha256": "185eb0948913e0fd0c0f6ddcf4bea30ff7d6b50185b5c6c401b51600039b745c"
|
|
4638
4678
|
},
|
|
4639
4679
|
{
|
|
4640
4680
|
"path": "skills/write-protocol/SKILL.md",
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "medsci-skills",
|
|
3
|
-
"version": "5.
|
|
3
|
+
"version": "5.18.0",
|
|
4
4
|
"description": "MedSci Skills — a medical/scientific research skill suite for AI coding agents (Claude Code, Codex, Cursor, Copilot). The npm package is a terminal-friendly installer shortcut; the canonical distribution remains the GitHub repository and the Claude Code plugin marketplace.",
|
|
5
5
|
"license": "SEE LICENSE IN LICENSE",
|
|
6
6
|
"homepage": "https://github.com/Aperivue/medsci-skills#readme",
|
|
@@ -107,6 +107,9 @@ This pattern is the canonical shape LLM extractors parse first.
|
|
|
107
107
|
### 2.4 Reproducibility block
|
|
108
108
|
Include a labeled block (typically end of Methods or a standalone Data/Code Availability section) listing: data availability and license, code availability with DOI, model weights and checkpoints, prompts and configuration files, random seeds, compute environment. This block is disproportionately scraped by AI agents when they cite a paper as reproducible.
|
|
109
109
|
|
|
110
|
+
### 2.4a Citing an AI-assisted tool by use-class
|
|
111
|
+
If the work used an AI-assisted tool (a verification/QA suite, analysis code, or a generative drafting assistant), frame the mention by *what it did*, not by hiding it — under current journal wariness, a proud in-text citation of a **generative** use invites suspicion, while the same placement for a **verification** use reads as rigor (like citing a reference manager or a linter). Split the mention: **verification/QA and analysis** → a **Software / Code-availability statement** (citable); **generative** drafting/humanizing → the journal's **AI-use disclosure field**, not a citation. A self-citation by the tool's author additionally requires a COI disclosure, and you should cite only the functions the work actually used. Full use-class table and rules: `${CLAUDE_SKILL_DIR}/references/ai_tool_citation_framing.md`.
|
|
112
|
+
|
|
110
113
|
### 2.5 Limitations enumeration
|
|
111
114
|
List limitations explicitly and name each one (generalizability, spectrum bias, dataset shift, single-center training, label noise). Papers with enumerated limitations score higher for trustworthiness in LLM summarization benchmarks.
|
|
112
115
|
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# Citing an AI-assisted research tool safely (framing by use-class)
|
|
2
|
+
|
|
3
|
+
When a manuscript used an AI-assisted tool — a reference-verification / QA suite, a
|
|
4
|
+
statistical-analysis helper, or a generative drafting assistant — *where and how you
|
|
5
|
+
name it* changes how an editor or reviewer reads it. Under current journal wariness
|
|
6
|
+
about AI, a proud in-text citation of a **generative** use can invite suspicion or a
|
|
7
|
+
desk-reject, while the identical placement for a **verification** use reads as rigor
|
|
8
|
+
(the same way citing R, SPSS, or a reference manager does). The safe move is to frame
|
|
9
|
+
the mention by what the tool actually *did*, not to hide it.
|
|
10
|
+
|
|
11
|
+
This applies to any AI-assisted tool; it also applies to **self-citation** by a tool's
|
|
12
|
+
author (e.g. citing MedSci Skills in your own paper), which additionally requires a
|
|
13
|
+
conflict-of-interest disclosure.
|
|
14
|
+
|
|
15
|
+
## The three use-classes
|
|
16
|
+
|
|
17
|
+
| Use-class | What it covers | Where it belongs | Citable like software? |
|
|
18
|
+
|---|---|---|---|
|
|
19
|
+
| **Verification / QA** (rigor-signalling) | Reference/citation verification (DOI/PMID against PubMed/CrossRef), reporting-guideline compliance checks, deterministic integrity gates, numerical-consistency checks | **Software / Code-availability statement** (or Methods, as a named tool with version) | **Yes** — cite it plainly, like a reference manager or a linter. It signals rigor. |
|
|
20
|
+
| **Analysis** (neutral) | Statistical-analysis code, figure generation, data-transformation scripts | Methods (named, with version) and/or Code-availability | **Yes** — neutral and citable, like citing R/Python packages. |
|
|
21
|
+
| **Generative** (disclosure, not citation) | Drafting or rewriting prose, "humanizing", summarizing, idea generation | The journal's **AI-use disclosure field / statement** (per its policy), *not* a proud in-text citation | **No** — declare it in the disclosure field; do not farm it into the running text. |
|
|
22
|
+
|
|
23
|
+
## Rules of thumb
|
|
24
|
+
|
|
25
|
+
- **Prefer the deterministic, verifiable functions for the cleanest cite.** A tool's
|
|
26
|
+
reproducible/verifiable capabilities (reference verification, compliance gates,
|
|
27
|
+
analysis code) are the ones that read as rigor and are safest to name in a Software
|
|
28
|
+
statement. Its generative capabilities belong in the disclosure field.
|
|
29
|
+
- **Match the placement to the use-class**, not to the tool. The *same* tool can be
|
|
30
|
+
citable (its QA gate ran your references) *and* disclosure-only (its drafting helper
|
|
31
|
+
touched your prose) in the *same* paper — split the mention accordingly.
|
|
32
|
+
- **Do not self-cite generatively.** If the paper did not materially use the generative
|
|
33
|
+
parts, do not add a self-citation for them to inflate a citation count; cite only the
|
|
34
|
+
functions the work actually used.
|
|
35
|
+
- **Pair a self-citation with a COI disclosure.** If you authored the tool, state the
|
|
36
|
+
relationship (e.g. "Author X develops the cited toolkit") in the COI/competing-interests
|
|
37
|
+
section — the same standard as any other intellectual/financial interest.
|
|
38
|
+
- **Honor the target journal's AI policy first.** Where a journal specifies *where* AI
|
|
39
|
+
use must be declared (Methods vs Acknowledgements vs a dedicated field), that placement
|
|
40
|
+
wins for the generative use-class; the Software-statement guidance above is for the
|
|
41
|
+
verification/analysis classes, which are tool-use, not AI-authorship.
|
|
42
|
+
|
|
43
|
+
## Why this is guidance, not a deterministic gate (yet)
|
|
44
|
+
|
|
45
|
+
A deterministic check ("flag an AI-tool citation placed in running-text Methods for a
|
|
46
|
+
*generative* use") would need both a maintained tool-name allowlist and a reliable
|
|
47
|
+
classifier of *which use-class* a given sentence describes — the latter is high false-
|
|
48
|
+
positive without context the grep cannot see. The reliable, low-FP part (a tool named in
|
|
49
|
+
a Software/Code-availability statement) is already the recommended state, so there is
|
|
50
|
+
nothing to flag. If a bounded, allowlist-driven placement check proves worthwhile on real
|
|
51
|
+
manuscripts, it can be added later; until then this stays advisory. (See
|
|
52
|
+
`~/.claude/rules/manuscript-style-classical.md` §7/§15 for AI-disclosure placement and the
|
|
53
|
+
self-applicability rule.)
|
|
@@ -191,6 +191,19 @@ scale validity; a low overall ICC is interpretable only if raters at least conve
|
|
|
191
191
|
- Require each item to be judged standalone; discourage cross-item references in free-text, which
|
|
192
192
|
signal non-independent rating.
|
|
193
193
|
|
|
194
|
+
**Human-as-operator arm (interactive / promptable AI).** The reader-study patterns above assume the
|
|
195
|
+
human is a *rater / reference* judging outputs. Interactive / promptable segmentation (SAM2, MedSAM2,
|
|
196
|
+
nnInteractive) inverts this: the human is the *operator* who places the prompts, so the measured object
|
|
197
|
+
is the human-operated system's **accuracy + interaction count + time + learning curve**, not a rating.
|
|
198
|
+
Design for it explicitly:
|
|
199
|
+
- Define the operator population and their onboarding; a **learning curve** (performance vs case index)
|
|
200
|
+
is a first-class outcome, not noise to average away.
|
|
201
|
+
- Fix the prompting protocol (allowed prompt types, stopping rule, target Dice) identically to any
|
|
202
|
+
simulated-prompting arm so the two are comparable — **protocol fidelity**, checked in `/model-validation`.
|
|
203
|
+
- Pre-specify the interaction and timing metrics; their deterministic reporting gate is
|
|
204
|
+
`/model-evaluation --task interactive`. (A design document is free-form prose, so the deterministic
|
|
205
|
+
anchor for these items sits at the reporting stage, not on the protocol text.)
|
|
206
|
+
|
|
194
207
|
For an AI-system-versus-human-expert benchmark specifically, route to `/design-ai-benchmarking`, which
|
|
195
208
|
extends this subsection with arm definition, LLM-as-judge versus human-as-judge adjudication, and a
|
|
196
209
|
structured export schema.
|
|
@@ -50,5 +50,6 @@ known_limitations:
|
|
|
50
50
|
validation_commands:
|
|
51
51
|
- "Rscript scripts/generate_flow_diagram.R"
|
|
52
52
|
- "python3 scripts/validate_pptx_mac_compat.py <file>"
|
|
53
|
+
- "python3 tests/test_pptx_mac_compat.py"
|
|
53
54
|
- "bash scripts/render_core_figures_challenge/verify.sh"
|
|
54
55
|
evidence_surface: demo
|