cxrfescore 0.2.2__tar.gz

cxrfescore-0.2.2/.gitignore

@@ -0,0 +1,23 @@
+# Byte-compiled / cache
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+.eggs/
+dist/
+build/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.venv/
+venv/
+*.egg
+
+# Editor / OS
+.DS_Store
+.idea/
+.vscode/
+
+# Local caches / notebooks checkpoints
+.ipynb_checkpoints/
+*.pkl

cxrfescore-0.2.2/CITATION.cff

@@ -0,0 +1,42 @@
+cff-version: 1.2.0
+message: "If you use CXRFEScore, please cite the following paper."
+title: "CXRFEScore"
+type: software
+authors:
+  - family-names: Messina
+    given-names: Pablo
+  - family-names: Vidal
+    given-names: Rene
+  - family-names: Parra
+    given-names: Denis
+  - family-names: Soto
+    given-names: Alvaro
+  - family-names: Araujo
+    given-names: Vladimir
+preferred-citation:
+  type: conference-paper
+  title: "Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation"
+  authors:
+    - family-names: Messina
+      given-names: Pablo
+    - family-names: Vidal
+      given-names: Rene
+    - family-names: Parra
+      given-names: Denis
+    - family-names: Soto
+      given-names: Alvaro
+    - family-names: Araujo
+      given-names: Vladimir
+  collection-title: "Findings of the Association for Computational Linguistics: ACL 2024"
+  month: 8
+  year: 2024
+  publisher: "Association for Computational Linguistics"
+  address: "Bangkok, Thailand"
+  url: "https://aclanthology.org/2024.findings-acl.236/"
+  doi: "10.18653/v1/2024.findings-acl.236"
+  pages: "3955-3986"
+  identifiers:
+    - type: doi
+      value: "10.18653/v1/2024.findings-acl.236"
+repository-code: "https://github.com/PabloMessina/CXRFEScore"
+license: MIT

cxrfescore-0.2.2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Pablo Messina
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

cxrfescore-0.2.2/PKG-INFO

@@ -0,0 +1,237 @@
+Metadata-Version: 2.4
+Name: cxrfescore
+Version: 0.2.2
+Summary: Fact-level embedding metric for evaluating chest X-ray radiology report generation.
+Project-URL: Homepage, https://github.com/PabloMessina/CXRFEScore
+Project-URL: Bug Tracker, https://github.com/PabloMessina/CXRFEScore/issues
+Project-URL: Paper, https://aclanthology.org/2024.findings-acl.236/
+Author-email: Pablo Messina <pablo.messina.g@gmail.com>
+License: MIT License
+        
+        Copyright (c) 2024 Pablo Messina
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Requires-Python: >=3.10
+Requires-Dist: nltk>=3.8.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: platformdirs>=4.0.0
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Dist: torch>=2.0.0
+Requires-Dist: tqdm>=4.65.0
+Requires-Dist: transformers>=4.40.0
+Provides-Extra: dev
+Requires-Dist: build>=1.0.0; extra == 'dev'
+Requires-Dist: matplotlib>=3.7.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: twine>=5.0.0; extra == 'dev'
+Provides-Extra: viz
+Requires-Dist: matplotlib>=3.7.0; extra == 'viz'
+Description-Content-Type: text/markdown
+
+# CXRFEScore
+
+Fact-level embedding metric for evaluating chest X-ray radiology report generation.
+
+CXRFEScore extracts factual statements from reports with a T5 fact extractor, embeds them with CXRFE (or compatible CXR-BERT models), and scores hypothesis/reference pairs via soft bipartite matching of fact embeddings.
+
+Based on:
+
+> Messina et al., *Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation*, Findings of ACL 2024.
+
+## Install
+
+```bash
+pip install cxrfescore
+```
+
+Optional visualization support:
+
+```bash
+pip install cxrfescore[viz]
+```
+
+**Python:** 3.10+
+
+### Runtime dependencies (installed automatically)
+
+| Package | Role |
+|---|---|
+| `torch` | Model inference |
+| `transformers` | Load Hugging Face models |
+| `numpy` | Arrays / aggregation |
+| `scikit-learn` | Cosine similarity |
+| `nltk` | Sentence splitting |
+| `tqdm` | Progress bars |
+| `platformdirs` | Default cache directory |
+
+Optional: `matplotlib` via `cxrfescore[viz]` for `visualize_fact_similarity`.
+
+### Additional requirements (not pip packages)
+
+On first use the metric downloads:
+
+- **Encoder (default):** [`pamessina/CXRFE`](https://huggingface.co/pamessina/CXRFE)
+- **Fact extractor:** [`pamessina/T5FactExtractor`](https://huggingface.co/pamessina/T5FactExtractor)
+- **NLTK** `punkt` / `punkt_tab` (auto-downloaded)
+
+Supported alternate encoders:
+
+- `microsoft/BiomedVLP-CXR-BERT-specialized`
+- `microsoft/BiomedVLP-BioViL-T`
+- `StanfordAIMI/SRR-BERT-Leaves` (CLS embeddings only; label head unused)
+
+A GPU is recommended but not required. CPU works and is slower.
+
+Known-good stack from development: PyTorch 2.x + `transformers` 4.x / 5.x.
+
+## Quick start
+
+```python
+from cxrfescore import CXRFEScore
+
+metric = CXRFEScore(device="cuda")  # or "cpu"
+
+hyps = [
+    "There is a small right pleural effusion. The heart size is normal.",
+]
+refs = [
+    "Small right pleural effusion. Normal heart size.",
+]
+
+result = metric(hyps, refs)
+print(result["mean_similarity"])
+print(result["per_pair_similarity"])
+```
+
+### Extract facts or embeddings only
+
+`extract_facts` treats each input string as a **full report**: sentence-split → T5 fact extraction → unique-fact aggregation (same pipeline as scoring).
+
+```python
+facts_per_report = metric.extract_facts(hyps)
+embeddings = metric.embed_facts(facts_per_report[0])  # shape (num_facts, dim)
+```
+
+### Caching
+
+With `use_cache=True` (default), results are kept **in memory** during the session:
+
+- Sentence→facts: `{cache_dir}/sent_to_facts.pkl` (shared across encoders)
+- Fact→embedding: `{cache_dir}/embeddings/<encoder_name>/fact_to_embedding.pkl` (per encoder)
+
+Disk writes are **not** automatic. Call `save_cache()` explicitly after a large batch or at the end of a job:
+
+```python
+metric = CXRFEScore(use_cache=True, cache_dir="/path/to/cache")
+# ... score / extract / embed ...
+metric.save_cache()
+```
+
+Disable with `use_cache=False`.
+
+Inspect what is already on disk:
+
+```python
+from cxrfescore import inspect_cache
+
+inspect_cache()                 # default cache location
+# or: metric.inspect_cache()    # uses this instance's cache_dir
+```
+
+### Visualization
+
+```python
+# requires: pip install cxrfescore[viz]
+metric.visualize_fact_similarity(ref_report=refs[0], cand_report=hyps[0])
+```
+
+## Examples
+
+| File | Purpose |
+|---|---|
+| [`examples/colab_smoke.md`](examples/colab_smoke.md) | Full Colab smoke + adversarial pairs |
+| [`examples/minimal_cxrfe.md`](examples/minimal_cxrfe.md) | Default CXRFE encoder |
+| [`examples/minimal_cxr_bert_specialized.md`](examples/minimal_cxr_bert_specialized.md) | CXR-BERT specialized |
+| [`examples/minimal_biovil_t.md`](examples/minimal_biovil_t.md) | BioViL-T |
+| [`examples/minimal_srr_bert_leaves.md`](examples/minimal_srr_bert_leaves.md) | SRR-BERT-Leaves CLS |
+| [`examples/minimal_extract_embed.md`](examples/minimal_extract_embed.md) | `extract_facts` / `embed_facts` |
+
+## Try it on Google Colab
+
+After publishing, install in Colab (GPU runtime recommended):
+
+```python
+!pip install cxrfescore
+```
+
+For a pre-release check against TestPyPI:
+
+```python
+!pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ "cxrfescore==0.2.2"
+```
+
+## Citation
+
+If you use CXRFEScore, please cite:
+
+> Pablo Messina, Rene Vidal, Denis Parra, Alvaro Soto, and Vladimir Araujo. 2024. [Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation](https://aclanthology.org/2024.findings-acl.236/). In *Findings of the Association for Computational Linguistics: ACL 2024*, pages 3955–3986, Bangkok, Thailand. Association for Computational Linguistics.
+
+```bibtex
+@inproceedings{messina-etal-2024-extracting,
+    title = "Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation",
+    author = "Messina, Pablo  and
+      Vidal, Rene  and
+      Parra, Denis  and
+      Soto, Alvaro  and
+      Araujo, Vladimir",
+    editor = "Ku, Lun-Wei  and
+      Martins, Andre  and
+      Srikumar, Vivek",
+    booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
+    month = aug,
+    year = "2024",
+    address = "Bangkok, Thailand",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2024.findings-acl.236/",
+    doi = "10.18653/v1/2024.findings-acl.236",
+    pages = "3955--3986"
+}
+```
+
+Paper: [ACL Anthology](https://aclanthology.org/2024.findings-acl.236/) · [PDF](https://aclanthology.org/2024.findings-acl.236.pdf) · [arXiv](https://arxiv.org/abs/2407.01948)
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest                          # unit + mocked tests (default)
+pytest -m integration           # downloads HF models; needs network / GPU optional
+python -m build
+```
+
+## License
+
+MIT

cxrfescore-0.2.2/README.md

@@ -0,0 +1,184 @@
+# CXRFEScore
+
+Fact-level embedding metric for evaluating chest X-ray radiology report generation.
+
+CXRFEScore extracts factual statements from reports with a T5 fact extractor, embeds them with CXRFE (or compatible CXR-BERT models), and scores hypothesis/reference pairs via soft bipartite matching of fact embeddings.
+
+Based on:
+
+> Messina et al., *Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation*, Findings of ACL 2024.
+
+## Install
+
+```bash
+pip install cxrfescore
+```
+
+Optional visualization support:
+
+```bash
+pip install cxrfescore[viz]
+```
+
+**Python:** 3.10+
+
+### Runtime dependencies (installed automatically)
+
+| Package | Role |
+|---|---|
+| `torch` | Model inference |
+| `transformers` | Load Hugging Face models |
+| `numpy` | Arrays / aggregation |
+| `scikit-learn` | Cosine similarity |
+| `nltk` | Sentence splitting |
+| `tqdm` | Progress bars |
+| `platformdirs` | Default cache directory |
+
+Optional: `matplotlib` via `cxrfescore[viz]` for `visualize_fact_similarity`.
+
+### Additional requirements (not pip packages)
+
+On first use the metric downloads:
+
+- **Encoder (default):** [`pamessina/CXRFE`](https://huggingface.co/pamessina/CXRFE)
+- **Fact extractor:** [`pamessina/T5FactExtractor`](https://huggingface.co/pamessina/T5FactExtractor)
+- **NLTK** `punkt` / `punkt_tab` (auto-downloaded)
+
+Supported alternate encoders:
+
+- `microsoft/BiomedVLP-CXR-BERT-specialized`
+- `microsoft/BiomedVLP-BioViL-T`
+- `StanfordAIMI/SRR-BERT-Leaves` (CLS embeddings only; label head unused)
+
+A GPU is recommended but not required. CPU works and is slower.
+
+Known-good stack from development: PyTorch 2.x + `transformers` 4.x / 5.x.
+
+## Quick start
+
+```python
+from cxrfescore import CXRFEScore
+
+metric = CXRFEScore(device="cuda")  # or "cpu"
+
+hyps = [
+    "There is a small right pleural effusion. The heart size is normal.",
+]
+refs = [
+    "Small right pleural effusion. Normal heart size.",
+]
+
+result = metric(hyps, refs)
+print(result["mean_similarity"])
+print(result["per_pair_similarity"])
+```
+
+### Extract facts or embeddings only
+
+`extract_facts` treats each input string as a **full report**: sentence-split → T5 fact extraction → unique-fact aggregation (same pipeline as scoring).
+
+```python
+facts_per_report = metric.extract_facts(hyps)
+embeddings = metric.embed_facts(facts_per_report[0])  # shape (num_facts, dim)
+```
+
+### Caching
+
+With `use_cache=True` (default), results are kept **in memory** during the session:
+
+- Sentence→facts: `{cache_dir}/sent_to_facts.pkl` (shared across encoders)
+- Fact→embedding: `{cache_dir}/embeddings/<encoder_name>/fact_to_embedding.pkl` (per encoder)
+
+Disk writes are **not** automatic. Call `save_cache()` explicitly after a large batch or at the end of a job:
+
+```python
+metric = CXRFEScore(use_cache=True, cache_dir="/path/to/cache")
+# ... score / extract / embed ...
+metric.save_cache()
+```
+
+Disable with `use_cache=False`.
+
+Inspect what is already on disk:
+
+```python
+from cxrfescore import inspect_cache
+
+inspect_cache()                 # default cache location
+# or: metric.inspect_cache()    # uses this instance's cache_dir
+```
+
+### Visualization
+
+```python
+# requires: pip install cxrfescore[viz]
+metric.visualize_fact_similarity(ref_report=refs[0], cand_report=hyps[0])
+```
+
+## Examples
+
+| File | Purpose |
+|---|---|
+| [`examples/colab_smoke.md`](examples/colab_smoke.md) | Full Colab smoke + adversarial pairs |
+| [`examples/minimal_cxrfe.md`](examples/minimal_cxrfe.md) | Default CXRFE encoder |
+| [`examples/minimal_cxr_bert_specialized.md`](examples/minimal_cxr_bert_specialized.md) | CXR-BERT specialized |
+| [`examples/minimal_biovil_t.md`](examples/minimal_biovil_t.md) | BioViL-T |
+| [`examples/minimal_srr_bert_leaves.md`](examples/minimal_srr_bert_leaves.md) | SRR-BERT-Leaves CLS |
+| [`examples/minimal_extract_embed.md`](examples/minimal_extract_embed.md) | `extract_facts` / `embed_facts` |
+
+## Try it on Google Colab
+
+After publishing, install in Colab (GPU runtime recommended):
+
+```python
+!pip install cxrfescore
+```
+
+For a pre-release check against TestPyPI:
+
+```python
+!pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ "cxrfescore==0.2.2"
+```
+
+## Citation
+
+If you use CXRFEScore, please cite:
+
+> Pablo Messina, Rene Vidal, Denis Parra, Alvaro Soto, and Vladimir Araujo. 2024. [Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation](https://aclanthology.org/2024.findings-acl.236/). In *Findings of the Association for Computational Linguistics: ACL 2024*, pages 3955–3986, Bangkok, Thailand. Association for Computational Linguistics.
+
+```bibtex
+@inproceedings{messina-etal-2024-extracting,
+    title = "Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation",
+    author = "Messina, Pablo  and
+      Vidal, Rene  and
+      Parra, Denis  and
+      Soto, Alvaro  and
+      Araujo, Vladimir",
+    editor = "Ku, Lun-Wei  and
+      Martins, Andre  and
+      Srikumar, Vivek",
+    booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
+    month = aug,
+    year = "2024",
+    address = "Bangkok, Thailand",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2024.findings-acl.236/",
+    doi = "10.18653/v1/2024.findings-acl.236",
+    pages = "3955--3986"
+}
+```
+
+Paper: [ACL Anthology](https://aclanthology.org/2024.findings-acl.236/) · [PDF](https://aclanthology.org/2024.findings-acl.236.pdf) · [arXiv](https://arxiv.org/abs/2407.01948)
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest                          # unit + mocked tests (default)
+pytest -m integration           # downloads HF models; needs network / GPU optional
+python -m build
+```
+
+## License
+
+MIT

cxrfescore-0.2.2/examples/colab_smoke.md

@@ -0,0 +1,169 @@
+# Colab smoke test (human)
+
+Run this **after** `cxrfescore` is uploaded to TestPyPI and **before** production PyPI.
+
+## Setup
+
+1. Open [Google Colab](https://colab.research.google.com/).
+2. Runtime → Change runtime type → **GPU** (T4 is enough).
+3. Run the cells below in order.
+
+## Install from TestPyPI
+
+```python
+!pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ "cxrfescore==0.2.2"
+```
+
+After the production release, use:
+
+```python
+!pip install cxrfescore
+```
+
+## Smoke test
+
+```python
+import torch
+from cxrfescore import CXRFEScore
+
+print("torch:", torch.__version__)
+print("cuda available:", torch.cuda.is_available())
+
+metric = CXRFEScore(
+    device="cuda" if torch.cuda.is_available() else "cpu",
+    batch_size=8,
+    num_workers=0,
+    verbose=True,
+)
+
+hyps = [
+    "There is a small right pleural effusion. The heart size is normal.",
+    "Large left pneumothorax with mediastinal shift.",
+]
+refs = [
+    "Small right pleural effusion. Normal heart size.",
+    "No acute cardiopulmonary process. Lungs are clear.",
+]
+
+result = metric(hyps, refs)
+print("mean_similarity:", result["mean_similarity"])
+print("per_pair_similarity:", result["per_pair_similarity"])
+
+# Sanity checks
+assert result["per_pair_similarity"].shape == (2,)
+assert result["per_pair_similarity"][0] > result["per_pair_similarity"][1]
+print("Colab smoke test PASSED")
+```
+
+## Optional visualization (easy pair)
+
+```python
+!pip install "cxrfescore[viz]"
+metric.visualize_fact_similarity(ref_report=refs[0], cand_report=hyps[0])
+```
+
+## Challenging / adversarial pairs
+
+These cases are designed to stress clinical nuances that embedding metrics often mishandle
+(or handle well). For each pair, inspect the printed score **and** the fact-similarity heatmap.
+A good metric should score clinically wrong pairs lower than near-paraphrases.
+
+```python
+adversarial_cases = [
+    {
+        "name": "Negation flip (should score LOW)",
+        "note": "Nearly identical wording, opposite clinical meaning.",
+        "ref": "There is no evidence of pneumothorax.",
+        "hyp": "There is evidence of pneumothorax.",
+    },
+    {
+        "name": "Laterality swap (should score LOW)",
+        "note": "Same finding, wrong side — clinically important error.",
+        "ref": "Moderate right pleural effusion.",
+        "hyp": "Moderate left pleural effusion.",
+    },
+    {
+        "name": "Severity / size swap (should score MID-LOW)",
+        "note": "Correct finding/side, wrong magnitude.",
+        "ref": "Small right pleural effusion.",
+        "hyp": "Large right pleural effusion.",
+    },
+    {
+        "name": "Synonym paraphrase (should score HIGH)",
+        "note": "Different wording, same meaning — robustness check.",
+        "ref": "The heart is enlarged. There is pulmonary edema.",
+        "hyp": "Cardiomegaly is present. Findings consistent with pulmonary oedema.",
+    },
+    {
+        "name": "Presence vs absence of shared anatomy (should score LOW)",
+        "note": "Shares anatomy tokens but asserts opposite finding.",
+        "ref": "The lungs are clear without focal consolidation.",
+        "hyp": "There is focal consolidation in the lungs.",
+    },
+    {
+        "name": "Device / support device mismatch (should score LOW)",
+        "note": "Support-device facts often confuse overlap-based metrics.",
+        "ref": "Endotracheal tube tip is appropriately positioned above the carina.",
+        "hyp": "Nasogastric tube tip is appropriately positioned in the stomach.",
+    },
+]
+```
+
+```python
+# Score all adversarial pairs in one batch
+adv_hyps = [c["hyp"] for c in adversarial_cases]
+adv_refs = [c["ref"] for c in adversarial_cases]
+adv_result = metric(adv_hyps, adv_refs)
+
+print("=== Adversarial pair scores ===")
+for case, score in zip(adversarial_cases, adv_result["per_pair_similarity"]):
+    print(f"\n[{case['name']}] score={float(score):.4f}")
+    print(f"  note: {case['note']}")
+    print(f"  REF: {case['ref']}")
+    print(f"  HYP: {case['hyp']}")
+```
+
+```python
+# Visualize each challenging pair (one heatmap per case)
+for case in adversarial_cases:
+    print("\n" + "=" * 72)
+    print(case["name"])
+    print(case["note"])
+    print("=" * 72)
+    metric.visualize_fact_similarity(
+        ref_report=case["ref"],
+        cand_report=case["hyp"],
+    )
+```
+
+```python
+# Quick relative sanity check (not a hard unit test — inspect failures by eye)
+scores = {c["name"]: float(s) for c, s in zip(adversarial_cases, adv_result["per_pair_similarity"])}
+synonym = scores["Synonym paraphrase (should score HIGH)"]
+negation = scores["Negation flip (should score LOW)"]
+laterality = scores["Laterality swap (should score LOW)"]
+
+print("synonym:", synonym)
+print("negation:", negation)
+print("laterality:", laterality)
+
+if synonym <= negation:
+    print("WARNING: synonym paraphrase did not outscore negation flip — inspect heatmaps.")
+if synonym <= laterality:
+    print("WARNING: synonym paraphrase did not outscore laterality swap — inspect heatmaps.")
+if synonym > negation and synonym > laterality:
+    print("Relative ordering looks reasonable (synonym > negation/laterality).")
+```
+
+## Checklist
+
+- [ ] Install from TestPyPI succeeds
+- [ ] Models download from Hugging Face
+- [ ] CUDA used when GPU runtime is selected
+- [ ] Similar pair scores higher than unrelated pair
+- [ ] Adversarial pairs inspected (negation, laterality, severity, synonyms, devices)
+- [ ] No import / runtime errors
+
+## Citation
+
+If you use CXRFEScore, please cite [Messina et al., Findings of ACL 2024](https://aclanthology.org/2024.findings-acl.236/). Full BibTeX is in the [README](../README.md#citation).

cxrfescore-0.2.2/examples/minimal_biovil_t.md

@@ -0,0 +1,18 @@
+# Minimal example — BioViL-T
+
+```python
+!pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ "cxrfescore==0.2.2"
+
+from cxrfescore import CXRFEScore
+
+metric = CXRFEScore(
+    encoder_model_name="microsoft/BiomedVLP-BioViL-T",
+    device="cuda",
+)
+result = metric(
+    ["There is a small right pleural effusion."],
+    ["Small right pleural effusion."],
+)
+print(result["mean_similarity"])
+metric.save_cache()
+```