medsci-skills 4.1.0

package/skills/check-reporting/references/checklists/CONSORT.md

@@ -0,0 +1,86 @@
+# CONSORT 2025 Checklist
+
+**Consolidated Standards of Reporting Trials**
+Version: CONSORT 2025
+Source: https://www.consort-spirit.org
+Reference: Hopewell S, Chan AW, Collins GS, et al. CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ 2025;389:e081123 (published simultaneously in BMJ, JAMA, Lancet, Nature Medicine, PLoS Medicine).
+
+> Note: CONSORT 2025 supersedes CONSORT 2010. It is a 30-item checklist (seven new items, three revised, one deleted) restructured with a new Open Science section.
+
+## Checklist Items (30 items)
+
+### Title and Abstract
+
+| # | Item | Description |
+|---|------|-------------|
+| 1a | Title | Identification as a randomised trial. |
+| 1b | Abstract | Structured summary of the trial design, methods, results, and conclusions. |
+
+### Open Science
+
+| # | Item | Description |
+|---|------|-------------|
+| 2 | Trial registration | Name of trial registry, identifying number (with URL) and date of registration. |
+| 3 | Protocol and SAP | Where the trial protocol and statistical analysis plan can be accessed. |
+| 4 | Data, code, materials | Where and how the individual de-identified participant data (including data dictionary), statistical code and any other materials can be accessed. |
+| 5a | Funding | Sources of funding and other support (e.g., supply of drugs), and role of funders in the design, conduct, analysis and reporting of the trial. |
+| 5b | Conflicts of interest | Financial and other conflicts of interest of the manuscript authors. |
+
+### Introduction
+
+| # | Item | Description |
+|---|------|-------------|
+| 6 | Background | Scientific background and rationale. |
+| 7 | Objectives | Specific objectives related to benefits and harms. |
+
+### Methods
+
+| # | Item | Description |
+|---|------|-------------|
+| 8 | Patient and public involvement | Details of patient or public involvement in the design, conduct and reporting of the trial. |
+| 9 | Trial design | Description of trial design including type of trial (e.g., parallel group, crossover), allocation ratio, and framework. |
+| 10 | Changes to trial | Important changes to the trial after it commenced including any outcomes or analyses that were not prespecified, with reason. |
+| 11 | Settings and locations | Settings (e.g., community, hospital) and locations (e.g., countries, sites) where the trial was conducted. |
+| 12a | Eligibility — participants | Eligibility criteria for participants. |
+| 12b | Eligibility — sites/deliverers | If applicable, eligibility criteria for sites and for individuals delivering the interventions. |
+| 13 | Interventions | Intervention and comparator with sufficient details to allow replication. |
+| 14 | Outcomes | Prespecified primary and secondary outcomes, including the specific measurement variable, analysis metric, method of aggregation, and time point for each outcome. |
+| 15 | Harms | How harms were defined and assessed (e.g., systematically, non-systematically). |
+| 16a | Sample size | How sample size was determined, including all assumptions supporting the sample size calculation. |
+| 16b | Interim analyses | Explanation of any interim analyses and stopping guidelines. |
+| 17a | Randomisation — sequence | Who generated the random allocation sequence and the method used. |
+| 17b | Randomisation — restriction | Type of randomisation and details of any restriction (e.g., stratification, blocking and block size). |
+| 18 | Allocation concealment | Mechanism used to implement the random allocation sequence (e.g., central computer/telephone; sequentially numbered, opaque, sealed containers). |
+| 19 | Implementation | Whether the personnel who enrolled and those who assigned participants to the interventions had access to the random allocation sequence. |
+| 20a | Blinding — who | Who was blinded after assignment to interventions (e.g., participants, care providers, outcome assessors, data analysts). |
+| 20b | Blinding — how | If blinded, how blinding was achieved and description of the similarity of interventions. |
+| 21a | Statistical methods | Statistical methods used to compare groups for primary and secondary outcomes, including harms. |
+| 21b | Analysis populations | Definition of who is included in each analysis (e.g., all randomised participants), and in which group. |
+| 21c | Missing data | How missing data were handled in the analysis. |
+| 21d | Additional analyses | Methods for any additional analyses (e.g., subgroup and sensitivity analyses), distinguishing prespecified from post hoc. |
+
+### Results
+
+| # | Item | Description |
+|---|------|-------------|
+| 22a | Participant flow — numbers | For each group, the numbers of participants who were randomly assigned, received intended intervention, and were analysed for the primary outcome. |
+| 22b | Participant flow — losses | For each group, losses and exclusions after randomisation, together with reasons. |
+| 23a | Recruitment — dates | Dates defining the periods of recruitment and follow-up for outcomes of benefits and harms. |
+| 23b | Recruitment — stopping | If relevant, why the trial ended or was stopped. |
+| 24a | Intervention as administered | Intervention and comparator as they were actually administered (e.g., where appropriate, who delivered the intervention/comparator, how participants adhered, whether they were delivered as intended). |
+| 24b | Concomitant care | Concomitant care received during the trial for each group. |
+| 25 | Baseline data | A table showing baseline demographic and clinical characteristics for each group. |
+| 26 | Outcomes and estimation | For each primary and secondary outcome, by group: the number of participants included in the analysis, the number with available data at the outcome time point, result for each group, and the estimated effect size and its precision. |
+| 27 | Harms | All harms or unintended events in each group. |
+| 28 | Ancillary analyses | Any other analyses performed, including subgroup and sensitivity analyses, distinguishing pre-specified from post hoc. |
+
+### Discussion
+
+| # | Item | Description |
+|---|------|-------------|
+| 29 | Interpretation | Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence. |
+| 30 | Limitations | Trial limitations, addressing sources of potential bias, imprecision, generalisability, and, if relevant, multiplicity of analyses. |
+
+---
+
+*Educational summary of the CONSORT 2025 checklist (CC BY 4.0). Cite the original statement (Hopewell et al., BMJ 2025) and consult https://www.consort-spirit.org for the authoritative, full checklist with explanation and elaboration.*

package/skills/check-reporting/references/checklists/COSMIN_RoB.md

@@ -0,0 +1,136 @@
+# COSMIN Risk of Bias Assessment Guide
+
+COnsensus-based Standards for the selection of health Measurement INstruments — Risk of Bias tool for reliability and measurement error.
+Reference: Mokkink LB et al. BMC Medical Research Methodology 2020;20:293.
+Website: https://www.cosmin.nl
+
+## Purpose
+
+The COSMIN Risk of Bias tool assesses the methodological quality of studies on **reliability** and **measurement error** of outcome measurement instruments (e.g., questionnaires, imaging measurements, lab tests).
+
+## Structure
+
+Two parts:
+- **Part A**: Understanding how the study informs on reliability/measurement error (7 elements of a comprehensive research question)
+- **Part B**: Assessing quality using standards (9 for reliability, 8 for measurement error)
+
+Quality rating per standard: Very Good / Adequate / Doubtful / Inadequate
+
+Overall quality uses the **worst-score-counts** principle (lowest rating across all standards).
+
+## Part A: Elements of a Comprehensive Research Question
+
+Extract these 7 elements from the study:
+
+| # | Element |
+|---|---------|
+| 1 | Name of the outcome measurement instrument |
+| 2 | Version or operationalization of the measurement protocol |
+| 3 | Construct measured by the instrument |
+| 4 | Reliability parameter (ICC, kappa, etc.) or measurement error parameter (SEM, LoA, SDC) |
+| 5 | Components of the instrument that will be repeated |
+| 6 | Source(s) of variation that will be varied (time, rater, machine, etc.) |
+| 7 | Patient population studied |
+
+## Components of Outcome Measurement Instruments
+
+### Without biological sampling
+- Equipment (questionnaire forms, imaging device, software)
+- Preparatory actions (training, calibration, patient instructions)
+- Collection of raw data (patient completing questions, rater observing)
+- Data processing and storage (digitization, image reconstruction)
+- Assignment of score (formula, algorithm, clinical judgment)
+- Optional further use (conversion to another scale/classification)
+
+### With biological sampling
+- Equipment (collection tools, containers, lab equipment)
+- Preparatory actions (patient prep, equipment calibration)
+- Collection of sample (blood draw, biopsy)
+- Processing and storage (centrifugation, freezing)
+- Determination of value (assay, measurement)
+- Optional further use (conversion to classification)
+
+## Part B: Standards for Studies on Reliability
+
+### Standard 1: Study Design
+- Were patients measured independently by raters (inter-rater) or on separate occasions (test-retest)?
+- Was the time interval appropriate (short enough to avoid true change, long enough to avoid memory)?
+
+### Standard 2: Missing Data
+- Were missing data adequately handled?
+- Was the percentage of missing data acceptable?
+
+### Standard 3: Sample Size
+- Was the sample size adequate? (minimum 30 recommended, 50+ preferred)
+
+### Standard 4: Patients
+- Were patients selected to represent a range of the construct being measured?
+- Were patients stable between measurements (for test-retest)?
+
+### Standard 5: Independent Measurements
+- Were measurements performed independently of each other?
+- Were raters blinded to previous measurements or other rater's results?
+
+### Standard 6: Similar Conditions
+- Were measurement conditions similar across repeated measurements?
+- Were there important differences in testing conditions?
+
+### Standard 7: Appropriate Statistical Methods
+- Was the appropriate reliability parameter calculated (ICC for continuous, kappa for categorical)?
+- Was the correct ICC model used (one-way, two-way, agreement vs consistency)?
+
+### Standard 8: ICC Model and Formula
+- Were the ICC model and formula correctly specified?
+- Was the ICC type appropriate for the study design (single measures vs average measures)?
+
+### Standard 9: Reporting
+- Were sufficient statistics reported (ICC/kappa with 95% CI)?
+- Were descriptive statistics reported (means, SDs)?
+
+## Part B: Standards for Studies on Measurement Error
+
+### Standard 1: Study Design
+Same as reliability Standard 1.
+
+### Standard 2: Missing Data
+Same as reliability Standard 2.
+
+### Standard 3: Sample Size
+Same as reliability Standard 3.
+
+### Standard 4: Patients
+Same as reliability Standard 4.
+
+### Standard 5: Independent Measurements
+Same as reliability Standard 5.
+
+### Standard 6: Similar Conditions
+Same as reliability Standard 6.
+
+### Standard 7: Appropriate Parameter
+- Was the appropriate measurement error parameter calculated (SEM, LoA, SDC, or percentage agreement)?
+
+### Standard 8: Reporting
+- Were sufficient statistics reported (SEM, LoA, or SDC with 95% CI)?
+- Were Bland-Altman plots provided where appropriate?
+
+## Overall Quality Rating
+
+Uses the **worst-score-counts** principle:
+- Rate each standard as: Very Good / Adequate / Doubtful / Inadequate
+- Overall rating = lowest rating across all applicable standards
+
+| Rating | Interpretation |
+|--------|---------------|
+| Very Good | Study design and methods are optimal for this measurement property |
+| Adequate | Study design and methods are acceptable |
+| Doubtful | Study design or methods raise some concerns |
+| Inadequate | Study design or methods are clearly flawed |
+
+## When to Use
+
+- Systematic reviews of measurement properties of health measurement instruments
+- Selecting outcome measurement instruments for clinical trials or research
+- Developing core outcome sets (COS)
+- Evaluating reliability/agreement of imaging measurements, scoring systems, clinical tests
+- Typically used alongside other COSMIN boxes (content validity, structural validity, etc.)

package/skills/check-reporting/references/checklists/GRRAS.md

@@ -0,0 +1,61 @@
+# GRRAS Checklist
+
+**Guidelines for Reporting Reliability and Agreement Studies**
+Version: GRRAS 2011
+Source: https://doi.org/10.1016/j.jclinepi.2010.03.002
+Reference: Kottner J, Audige L, Brorson S, et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96-106. doi:10.1016/j.jclinepi.2010.03.002
+
+## Checklist Items (15 items)
+
+### Title and Abstract
+
+| # | Item | Description |
+|---|------|-------------|
+| 1 | Identification | Identify in title or abstract that the study addresses reliability and/or agreement. Specify the statistic(s) used (e.g., kappa, ICC, Bland-Altman). |
+
+### Introduction
+
+| # | Item | Description |
+|---|------|-------------|
+| 2 | Rationale | Provide the scientific background and rationale for the reliability/agreement study. |
+| 3 | Objectives | State the specific objectives, including any pre-specified hypotheses (e.g., expected level of agreement). |
+
+### Methods
+
+| # | Item | Description |
+|---|------|-------------|
+| 4 | Study design | Describe the study design, including: fully crossed vs nested, raters as random or fixed effect, number of measurement occasions. |
+| 5 | Sample | Describe the sample: how subjects were sampled, eligibility criteria, settings and locations, and intended variation in the measurand. |
+| 6 | Measurement methods and devices | Describe the measurement methods and devices used, including any preparation, training, or calibration. |
+| 7 | Raters | Describe the raters: number, relevant qualifications and experience, any rater training specific to the study, and whether raters represent the target population of users. |
+| 8 | Measurement procedure | Describe the measurement procedure: setting, timing, number of measurements per subject, time interval between repeated measurements, and standardization of conditions. |
+| 9 | Order of measurements | State how the order of examination was varied (e.g., randomized order, blinding to previous measurements). |
+| 10 | Statistical methods | State the statistical methods used to estimate reliability and/or agreement, including: software, specific statistical model (e.g., ICC model and type), handling of missing data, methods for estimating CIs, and any additional analyses (e.g., Bland-Altman, SEM, MDC). |
+
+### Results
+
+| # | Item | Description |
+|---|------|-------------|
+| 11 | Sample and raters | Report the number of raters and subjects included in the analysis (and any exclusions). Report relevant demographic and clinical characteristics of the sample. |
+| 12 | Reliability/agreement estimates | Report reliability and/or agreement estimates with confidence intervals or standard errors. For continuous measurements, report LoA or SEM/MDC as applicable. |
+| 13 | Additional analyses | Report results of any additional analyses performed (e.g., effect of rater experience, subgroup analyses, time-trend analysis). |
+
+### Discussion
+
+| # | Item | Description |
+|---|------|-------------|
+| 14 | Clinical relevance | Discuss the practical and clinical relevance of the results. Include comparison with previously reported reliability/agreement values, interpretation relative to measurement purpose, and implications for clinical use. |
+| 15 | Limitations | Discuss the limitations of the study, including: spectrum of measurand values, generalizability to other raters/settings, potential sources of bias (e.g., memory effects, learning effects), and any factors that may have inflated or deflated estimates. |
+
+---
+
+## Notes for Assessors
+
+- GRRAS applies to all reliability and agreement studies, including inter-rater, intra-rater, test-retest, and method-comparison designs across any medical discipline.
+- **Reliability vs Agreement**: These are distinct concepts. Reliability (e.g., ICC, kappa) reflects the ability to distinguish between subjects; agreement (e.g., Bland-Altman limits of agreement, SEM) reflects how close scores are on repeated measurements. A study may report one or both.
+- Item 4 (Study design): The ICC model choice (one-way random, two-way random, two-way mixed) and type (single measures, average measures) must match the study design. Flag mismatches as a Major Comment.
+- Item 7 (Raters): Rater characteristics strongly influence reliability estimates. Studies using expert raters only will overestimate reliability in general clinical practice. Note this if raters are not representative.
+- Item 9 (Order of measurements): Randomization of measurement order is critical for avoiding memory and learning effects. Its absence should be flagged as a limitation.
+- Item 12: For ICC, report the model, type, and definition (consistency vs absolute agreement). For kappa, specify weighted vs unweighted and the weighting scheme. Bare numbers without these specifications are PARTIAL.
+- Common in radiology: inter-reader agreement for measurements (Bland-Altman, ICC), segmentation agreement (Dice, ICC of volumes), and diagnostic classification agreement (kappa, percent agreement).
+- GRRAS is complementary to study-type checklists. For example, a diagnostic accuracy study reporting inter-reader agreement should follow STARD for the main analysis and GRRAS for the agreement component.

package/skills/check-reporting/references/checklists/MI_CLEAR_LLM.md

@@ -0,0 +1,167 @@
+# MI-CLEAR-LLM Checklist — Reporting LLM Accuracy Studies in Healthcare
+
+**Reference:** Park SH, Suh CH, Lee JH, Kahn CE Jr, Moy L. Minimum Reporting Items for Clear Evaluation of Accuracy Reports of Large Language Models in Healthcare (MI-CLEAR-LLM). Korean J Radiol. 2024;25(10):865-868. PMID: 39344542
+**Reference (2025 Update):** Park SH, Suh CH, Lee JH, Tejani AS, You SC, Kahn CE Jr, Moy L. MI-CLEAR-LLM: 2025 Updates. Korean J Radiol. 2025;26(12):1123-1132. PMID: 41199132
+
+**Version:** 2025 (expanded from 2024 original)
+**License:** CC BY-NC 4.0 (Korean Society of Radiology)
+**Source:** https://kjronline.org/DOIx.php?id=10.3348/kjr.2025.1522
+
+**Scope:** Studies evaluating the accuracy of LLMs in healthcare applications (diagnosis, triage, clinical decision support, medical question answering, etc.). This is NOT for disclosing LLM use in manuscript writing — for that, see ICMJE/COPE policies and the write-paper skill's LLM disclosure feature.
+
+**vs. CLAIM 2024:** Use CLAIM for imaging AI model validation. Use MI-CLEAR-LLM for studies testing LLM accuracy in healthcare tasks. Both may apply if the study evaluates an LLM integrated with medical imaging interpretation.
+
+---
+
+## Item 1 — LLM Identification and Specifications ⚠️ commonly missed
+
+**Description:** Fully identify the LLM evaluated in the study.
+**Required content:**
+- Model name and version (e.g., GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro)
+- Manufacturer/developer (e.g., OpenAI, Anthropic, Google)
+- Training data cutoff date (if known or disclosed by the developer)
+- Date(s) when queries were executed (LLM outputs may change over time due to updates)
+- Access mode: web-based chatbot interface, API, or locally deployed open-source model
+- For API access: specific API version or endpoint used
+- For open-source models: model weights version, quantization method (if applicable), hardware specifications
+- Whether retrieval-augmented generation (RAG) or internet access was enabled
+- Any fine-tuning or adaptation applied to the base model
+
+**Rationale:** LLMs are frequently updated without version increments. The same model name may produce different outputs on different dates. Without exact version and date documentation, studies are not reproducible.
+
+**Status:** [ ] PRESENT [ ] PARTIAL [ ] MISSING
+
+---
+
+## Item 2 — Stochasticity Handling ⚠️ commonly missed
+
+**Description:** Document how the inherent randomness of LLM outputs was managed.
+**Required content:**
+- Number of query attempts per question/task (e.g., each prompt was run 3 times)
+- How multiple outputs were synthesized (e.g., majority vote, best-of-N, mean score)
+- Reliability analysis across repeated attempts (e.g., agreement rate, Fleiss' kappa)
+- Technical parameter settings that control randomness:
+  - Temperature setting (e.g., temperature = 0 for deterministic output)
+  - Top-p (nucleus sampling) setting
+  - Any other sampling parameters (top-k, frequency penalty, etc.)
+- If parameters were not controllable (e.g., web chatbot interface): state this explicitly
+
+**Rationale:** LLM outputs are stochastic by design. A single query attempt is insufficient to characterize model performance. Studies that do not control or report stochasticity parameters produce unreliable and unreproducible results.
+
+**Status:** [ ] PRESENT [ ] PARTIAL [ ] MISSING
+
+---
+
+## Item 3 — Full Prompt Text ⚠️ commonly missed
+
+**Description:** Provide the complete text of all prompts used, preserving exact formatting.
+**Required content:**
+- Full prompt text including system prompts, user prompts, and any few-shot examples
+- Preserve precise spellings, symbols, punctuation, spaces, line breaks, and formatting
+- If prompts include clinical data or images: describe the format and structure of input data
+- For multi-turn conversations: provide the complete conversation sequence
+- For chain-of-thought or structured prompting: document the full prompting strategy
+- Recommended location: supplementary materials (if prompts are lengthy)
+
+**Rationale:** Minor prompt variations can substantially alter LLM outputs. Without exact prompt text, no reader can assess the study methodology or attempt replication.
+
+**Status:** [ ] PRESENT [ ] PARTIAL [ ] MISSING
+
+---
+
+## Item 4 — Prompt Execution Details ⚠️ commonly missed
+
+**Description:** Explain how prompts were operationally used during the study.
+**Required content:**
+- Whether each query was executed as an independent chat session (new conversation) or within a continuing session
+- Whether multiple queries were submitted simultaneously (batch) or sequentially
+- If sequentially: whether prior outputs could influence subsequent queries (context carryover)
+- Whether any post-processing was applied to LLM outputs (e.g., extracting structured answers from free text, parsing JSON)
+- Who executed the queries (researchers, automated scripts, clinical staff)
+- For API-based studies: batch size, rate limiting, error handling for failed API calls
+
+**Rationale:** Chat session context and query ordering can influence LLM responses. Studies must specify whether the LLM had access to context from prior queries, as this affects comparability.
+
+**Status:** [ ] PRESENT [ ] PARTIAL [ ] MISSING
+
+---
+
+## Item 5 — Prompt Testing and Optimization ⚠️ commonly missed
+
+**Description:** Describe the prompt development process.
+**Required content:**
+- How prompts were created (expert-designed, iteratively refined, generated by another LLM)
+- Rationale for specific wording choices (why this prompt structure was chosen)
+- Whether prompt optimization/engineering was performed
+- If optimized: method used (manual iteration, automated prompt tuning, grid search over prompt variants)
+- Dataset used for prompt optimization (must be independent of test data — see Item 6)
+- Number of prompt iterations tested
+- Whether different prompt strategies were compared (zero-shot vs. few-shot vs. chain-of-thought)
+
+**Rationale:** Prompt design is a critical methodological choice equivalent to feature engineering in traditional ML. Unreported optimization inflates apparent performance if optimization and test data overlap.
+
+**Status:** [ ] PRESENT [ ] PARTIAL [ ] MISSING
+
+---
+
+## Item 6 — Test Data Independence ⚠️ commonly missed
+
+**Description:** Confirm separation between test data and all other data used in the study.
+**Required content:**
+- Explicit statement that test data were not used during prompt development or optimization
+- Whether test data could have been part of the LLM's training data (data contamination risk)
+- If test data were sourced from the internet: provide exact URLs and confirm whether the content was publicly accessible before the LLM's training cutoff date
+- For published medical examination questions: acknowledge that these may be in the LLM's training corpus and discuss implications
+- Mitigation strategies for data contamination (e.g., using unpublished cases, temporal splits, institution-specific data)
+
+**Rationale:** If test data were seen during prompt optimization or were part of the LLM's pretraining corpus, performance estimates are inflated. Unlike traditional ML where training data are known, LLM training data are often undisclosed, making contamination assessment particularly challenging.
+
+**Status:** [ ] PRESENT [ ] PARTIAL [ ] MISSING
+
+---
+
+## Notes for Assessors
+
+### When to Apply MI-CLEAR-LLM
+
+Apply this checklist when the study's primary aim is to evaluate LLM accuracy, performance, or clinical utility in a healthcare context. Examples:
+- LLM answering medical board examination questions
+- LLM performing clinical reasoning or differential diagnosis
+- LLM triaging patient messages or radiology reports
+- LLM extracting structured data from clinical notes
+- LLM generating radiology report impressions
+
+Do NOT apply this checklist when:
+- The study uses LLMs as a tool in the research pipeline (e.g., NLP preprocessing) but does not evaluate LLM accuracy as an outcome
+- The manuscript merely discloses that LLM writing assistance was used
+
+### Common Assessment Pitfalls
+
+1. **Item 1 — "GPT-4" is insufficient.** Require the exact model version (e.g., gpt-4-0613, gpt-4-turbo-2024-04-09) and the access date. OpenAI and other providers frequently update models behind the same name.
+2. **Item 2 — Single-run studies.** If only one query attempt was made per item, mark as PARTIAL at best. The study cannot characterize reliability without repeated measurements.
+3. **Item 3 — "We used a standardized prompt" is insufficient.** The full text must be available, even if in supplementary materials.
+4. **Item 5 — Absent optimization reporting.** Many studies iteratively refine prompts but do not report this process. If the prompt appears highly optimized but no development process is described, flag as MISSING.
+5. **Item 6 — Published exam questions.** Studies using publicly available medical exam questions (USMLE, board exams) should explicitly discuss training data contamination risk. This is a known and well-documented concern.
+
+### Relationship to Other Guidelines
+
+- **CLAIM 2024**: Use alongside MI-CLEAR-LLM if the LLM processes medical images (e.g., vision-language models interpreting radiology images).
+- **TRIPOD+AI**: Use alongside MI-CLEAR-LLM if the LLM is used as a clinical prediction model.
+- **STARD / STARD-AI**: Use alongside MI-CLEAR-LLM if the study evaluates diagnostic accuracy of the LLM against a reference standard.
+
+---
+
+## Summary
+
+| Category | Items | PRESENT | PARTIAL | MISSING |
+|----------|-------|---------|---------|---------|
+| LLM Specification | 1 | /1 | /1 | /1 |
+| Stochasticity | 1 | /1 | /1 | /1 |
+| Prompt Documentation | 2 | /2 | /2 | /2 |
+| Prompt Development | 1 | /1 | /1 | /1 |
+| Data Independence | 1 | /1 | /1 | /1 |
+| **TOTAL** | **6** | /6 | /6 | /6 |
+
+**Verdict:** [ ] READY FOR SUBMISSION [ ] NEEDS REVISION
+
+**Co-application note:** MI-CLEAR-LLM is typically used alongside another guideline (STARD, CLAIM, TRIPOD+AI) depending on the study design. The 6 items here supplement — not replace — the primary reporting guideline.

package/skills/check-reporting/references/checklists/MOOSE.md

@@ -0,0 +1,85 @@
+# MOOSE Checklist
+
+**Meta-analysis Of Observational Studies in Epidemiology**
+Version: MOOSE 2000
+Source: https://doi.org/10.1001/jama.283.15.2008
+Reference: Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008-2012. doi:10.1001/jama.283.15.2008
+
+## Checklist Items (35 items)
+
+### Reporting of Background
+
+| # | Item | Description |
+|---|------|-------------|
+| 1 | Problem definition | Define the clinical or public health problem being addressed. |
+| 2 | Hypothesis statement | State the hypothesis to be tested. |
+| 3 | Study outcome(s) | Describe the study outcome(s) of interest. |
+| 4 | Type of exposure or intervention | Describe the type of exposure or intervention used. |
+| 5 | Type of study designs | Describe the type of study designs included (cohort, case-control, cross-sectional, etc.). |
+| 6 | Study population | Describe the study population. |
+
+### Reporting of Search Strategy
+
+| # | Item | Description |
+|---|------|-------------|
+| 7 | Qualifications of searchers | Report the qualifications of the searchers (e.g., librarian, investigator). |
+| 8 | Search strategy | Describe the search strategy, including time period included and keywords used. |
+| 9 | Effort to include all available studies | Describe efforts to include all available studies, including contact with authors. |
+| 10 | Databases and registries searched | List all databases and registries searched. |
+| 11 | Search software used | Report the search software used, including name and version. |
+| 12 | Use of hand searching | Describe any use of hand searching (e.g., reference lists, conference proceedings). |
+| 13 | List of citations located and excluded | Provide a list of citations located and those excluded, including justifications for exclusions. |
+| 14 | Method for non-English articles | Describe the method of addressing articles published in languages other than English. |
+| 15 | Method for unpublished studies | Describe the method of handling abstracts and unpublished studies. |
+| 16 | Description of author contact | Describe any contact with authors for additional data or clarification. |
+
+### Reporting of Methods
+
+| # | Item | Description |
+|---|------|-------------|
+| 17 | Study relevance | Describe the relevance or appropriateness of studies assembled for assessing the hypothesis to be tested. |
+| 18 | Rationale for data selection and coding | Provide the rationale for the selection and coding of data (e.g., sound clinical principles, convenience, etc.). |
+| 19 | Data classification | Document how data were classified and coded (e.g., multiple raters, blinding, inter-rater reliability). |
+| 20 | Assessment of confounding | Describe the assessment of confounding (e.g., comparability of cases and controls, adjustment for confounders). |
+| 21 | Assessment of study quality | Describe the assessment of study quality, including blinding of quality assessors; stratification or regression on possible predictors of study results. |
+| 22 | Assessment of heterogeneity | Describe the assessment of heterogeneity. |
+| 23 | Statistical methods | Describe statistical methods (e.g., fixed vs random effects, meta-regression, cumulative meta-analysis) in sufficient detail to be replicated. |
+| 24 | Tables and graphics | Provide appropriate tables and graphics. |
+
+### Reporting of Results
+
+| # | Item | Description |
+|---|------|-------------|
+| 25 | Graphic summary | Provide a graphic summarizing individual study estimates and the overall estimate (e.g., forest plot). |
+| 26 | Descriptive table | Provide a table giving descriptive information for each study included (design, sample size, outcome measures, effect sizes, confounders adjusted for). |
+| 27 | Sensitivity testing | Report results of sensitivity testing (e.g., subgroup analysis, influence analysis, varying inclusion criteria). |
+| 28 | Statistical uncertainty | Indicate statistical uncertainty of findings (e.g., 95% confidence intervals). |
+
+### Reporting of Discussion
+
+| # | Item | Description |
+|---|------|-------------|
+| 29 | Quantitative assessment of bias | Provide a quantitative assessment of bias (e.g., publication bias via funnel plot, Egger test, trim-and-fill). |
+| 30 | Justification for exclusion | Justify any exclusions from the meta-analysis. |
+| 31 | Assessment of quality of included studies | Assess the quality of included studies and its impact on the overall findings. |
+
+### Reporting of Conclusions
+
+| # | Item | Description |
+|---|------|-------------|
+| 32 | Alternative explanations | Consider alternative explanations for observed results. |
+| 33 | Generalization | Discuss the generalization of the conclusions (i.e., external validity). |
+| 34 | Guidelines for future research | Provide guidelines for future research. |
+| 35 | Disclosure of funding | Disclose the funding source for the meta-analysis. |
+
+---
+
+## Notes for Assessors
+
+- MOOSE is designed specifically for meta-analyses of **observational** studies (cohort, case-control, cross-sectional). For meta-analyses of RCTs, use PRISMA 2020.
+- MOOSE and PRISMA are complementary. Many journals require adherence to both for observational MAs. When both apply, audit against both checklists separately.
+- Items 7-16 (Search Strategy) overlap substantially with PRISMA but include observational-specific items such as handling of confounding and study quality assessment.
+- Item 20 (Assessment of confounding) is particularly important for observational MAs, as confounding is the primary threat to validity -- unlike RCTs where randomization addresses this.
+- Item 21 (Assessment of study quality) should describe a formal tool. Common tools include NOS (Newcastle-Ottawa Scale), ROBINS-I, or the Joanna Briggs Institute checklists.
+- Item 29 (Quantitative assessment of bias) is critical. Publication bias assessment is expected in all meta-analyses, but is especially important for observational studies where positive-result bias is well documented.
+- MOOSE was published before the PRISMA era (2009/2020). Some journals now accept PRISMA with MOOSE-specific additions. If in doubt, check journal instructions.

package/skills/check-reporting/references/checklists/NOS.md

@@ -0,0 +1,88 @@
+# Newcastle-Ottawa Scale (NOS) Assessment Guide
+
+Quality assessment tool for non-randomised studies in meta-analyses.
+Reference: Wells GA et al. Ottawa Hospital Research Institute.
+
+## Structure
+
+NOS uses a "star system" (maximum 9 stars) across 3 categories.
+Higher stars = higher quality.
+
+## Cohort Studies (max 9 stars)
+
+### Selection (max 4 stars)
+1. **Representativeness of the exposed cohort** (1 star)
+   - a) Truly representative of the average [describe] in the community *
+   - b) Somewhat representative *
+   - c) Selected group of users
+   - d) No description
+
+2. **Selection of the non-exposed cohort** (1 star)
+   - a) Drawn from the same community as the exposed *
+   - b) Drawn from a different source
+   - c) No description
+
+3. **Ascertainment of exposure** (1 star)
+   - a) Secure record (e.g., surgical records) *
+   - b) Structured interview *
+   - c) Written self-report
+   - d) No description
+
+4. **Demonstration that outcome was not present at start** (1 star)
+   - a) Yes *
+   - b) No
+
+### Comparability (max 2 stars)
+5. **Comparability of cohorts on the basis of design or analysis** (up to 2 stars)
+   - a) Study controls for [most important factor] *
+   - b) Study controls for any additional factor *
+
+### Outcome (max 3 stars)
+6. **Assessment of outcome** (1 star)
+   - a) Independent blind assessment *
+   - b) Record linkage *
+   - c) Self-report
+   - d) No description
+
+7. **Was follow-up long enough for outcomes to occur?** (1 star)
+   - a) Yes (select adequate follow-up period) *
+   - b) No
+
+8. **Adequacy of follow-up of cohorts** (1 star)
+   - a) Complete follow-up (all subjects accounted for) *
+   - b) Subjects lost to follow-up unlikely to introduce bias (<=20% lost, or description provided) *
+   - c) Follow-up rate <80% and no description of those lost
+   - d) No statement
+
+## Case-Control Studies (max 9 stars)
+
+### Selection (max 4 stars)
+1. Is the case definition adequate?
+2. Representativeness of the cases
+3. Selection of controls
+4. Definition of controls
+
+### Comparability (max 2 stars)
+5. Comparability of cases and controls (same 2-star system)
+
+### Exposure (max 3 stars)
+6. Ascertainment of exposure
+7. Same method of ascertainment for cases and controls
+8. Non-response rate
+
+## Interpretation
+
+| Stars | Quality |
+|-------|---------|
+| 7-9 | Good (high quality) |
+| 4-6 | Fair (moderate quality) |
+| 0-3 | Poor (low quality) |
+
+Alternative thresholds may be pre-specified in the protocol.
+
+## When to Use
+
+- Observational cohort studies in intervention or exposure meta-analyses
+- Case-control studies
+- Simpler alternative to ROBINS-I when full domain-level assessment is not needed
+- Note: NOS does not provide domain-level judgments -- only an aggregate score