allelix 2.0.2__tar.gz → 2.1.1__tar.gz

{allelix-2.0.2 → allelix-2.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: allelix
-Version: 2.0.2
+Version: 2.1.1
 Summary: Open-source genotype analysis toolkit. Format-agnostic ingestion, database-agnostic annotation, offline-first.
 Author: Allelix
 Maintainer-email: dial481 <dial481@users.noreply.github.com>
@@ -47,19 +47,26 @@ Open-source command-line toolkit for analyzing raw genotype files from consumer
 > HTML/JSON/terminal reports, methylation + pharmacogenomics focused
 > commands, report diffing, persistent config with commercial-mode
 > safety switch. Build auto-detection from position data (ADR-0021).
-> No regex on prose anywhere in production. **Latest: v2.0.0** — VCF +
-> gVCF parser with multi-sample handling, batched annotation pipeline
-> for WGS scale, FTDNA Illumina raw parser, R-4 ClinVar CLNSIG drift CI
-> test, CLI package restructure.
+> No regex on prose anywhere in production. **Latest: v2.1.0** —
+> ADR-0035 'Cluster B' coordinated model change: `Variant.ref` carries
+> the forward-strand reference allele, per-Annotation `alt` threading
+> on GWAS / SNPedia / ClinPGx enables allele-specific gnomAD /
+> AlphaMissense / CADD enrichment on array inputs, structured GWAS
+> fields (`trait` / `p_value` / `phecode`) replace prose re-parsing in
+> the rollup, and ClinVar / ClinPGx carrier matching is strand-aware
+> with multi-allelic safety guard. JSON `schema_version` bumped 4 → 5
+> (additive, v4 readers unaffected).
 > Release notes:
 > [`CHANGELOG.md`](https://github.com/allelix/allelix/blob/main/CHANGELOG.md).
 
 ## Quickstart
 
+**Requires Python 3.11 or newer.** On Python 3.10 or older, `pip install allelix` succeeds but the first command fails with `ImportError: cannot import name 'UTC' from 'datetime'` — that's the symptom; the fix is to install on 3.11+. Check with `python --version`.
+
 ```bash
 pip install allelix
 
-# Download reference databases (~15GB with all sources).
+# Download reference databases (~16 GB default install; ~22 GB if you opt into CADD).
 # Use --no-gnomad / --no-alphamissense to skip the large ones.
 # CADD is opt-in: allelix db update --cadd
 allelix db update
@@ -77,7 +84,7 @@ allelix analyze trio.vcf.gz --sample HG002 --output report.html
 allelix analyze your_genotype_file.txt --filter-file my_panel.txt --output report.html
 ```
 
-Requires Python 3.11+. See [Development](#development) for source installs and running tests.
+See [Development](#development) for source installs and running tests, [Managing your data](#managing-your-data) for cache locations and cleanup, and [Troubleshooting](#troubleshooting) for common failure states.
 
 ## Supported Formats
 
@@ -116,7 +123,20 @@ Adding a new format means adding one file to `allelix/parsers/` and registering
 | GWAS Catalog | ✓ | Public domain (EBI/NHGRI). Trait–SNP associations with p-values and effect sizes. Carrier rule (ADR-0007) requires the user to carry the risk allele. P-value magnitude scoring (ADR-0024) maps continuous p-values to the 0–10 scale; unknown-risk-allele entries fire on rsID match alone but are capped at 3.0. |
 | gnomAD | ✓ | ODbL v1.0. **Enrichment annotator** — adds population allele frequency context to existing annotations. Shows how common each variant is in the general population (~16M exome variants from 730K individuals). A pathogenic variant that 35% of people carry reads very differently from one seen in 0.001%. Pre-built cache downloaded via `db update` (~6GB on disk). Use `--no-gnomad` to skip. |
 | AlphaMissense | ✓ | CC BY 4.0. **Enrichment annotator** — adds DeepMind's protein-structure-based pathogenicity predictions to existing annotations. Scores 71M missense variants on a 0–1 scale: <0.34 = likely benign, >0.564 = likely pathogenic. Complements ClinVar's expert classifications with computational predictions — especially valuable for variants ClinVar hasn't reviewed yet. Pre-built cache downloaded via `db update` (~8GB on disk). Use `--no-alphamissense` to skip. |
-| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. Ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%. **Opt-in** — disabled by default (`sources.cadd = false`). Enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run. Pre-built cache (~5 GB on disk, ~120M variant keys). Full mode available via pysam for GRCh38 data (`options.cadd_full = true`). Cache mode covers the large majority of variants present in gnomAD, AlphaMissense, and ClinVar — nearly every position allelix can annotate from its other databases. For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), cache and full mode produce effectively identical results because chip probes overwhelmingly target known, cataloged variants. Full mode adds coverage for novel or private variants that appear only in whole-genome or whole-exome sequencing data and are not in any pre-computed database. If your input is a genotyping chip file, cache mode is all you need. |
+| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. **Opt-in**, disabled by default; see [CADD modes](#cadd-modes) below for details. |
+
+### CADD modes
+
+CADD ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%.
+
+**Opt-in**: enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run.
+
+Two modes:
+
+- **Cache mode** (default): pre-built ~5.8 GB SQLite cache (~120M variant keys). Covers nearly every position allelix can annotate from its other databases (gnomAD, AlphaMissense, ClinVar). For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), this mode is functionally complete — chip probes overwhelmingly target known, cataloged variants.
+- **Full mode** (`options.cadd_full = true`): tabix queries against the complete CADD v1.7 file via pysam (GRCh38 only). Adds coverage for novel or private variants that appear only in WGS/WES data and are not in any pre-computed database. Requires `pip install allelix[cadd]` for the pysam dependency.
+
+If your input is a genotyping chip file, cache mode is all you need.
 
 ### Build coverage asymmetry (GRCh37 vs GRCh38)
 
@@ -214,6 +234,48 @@ gnomAD and AlphaMissense are the largest but add the most interpretive context.
 
 To skip either during download: `allelix db update --no-gnomad --no-alphamissense`. To disable permanently: `allelix config set sources.gnomad false`.
 
+### Managing your data
+
+Allelix stores reference database caches at `~/.local/share/allelix/` (Linux/macOS) or the XDG-equivalent on other platforms. Default install (no CADD) is ~16 GB on disk; with the CADD opt-in it's ~22 GB.
+
+```
+~/.local/share/allelix/
+├── alphamissense.sqlite       ~7.8 GB
+├── gnomad.sqlite              ~6.1 GB
+├── cadd.sqlite                ~5.8 GB   (only if you opted into --cadd)
+├── clinvar.GRCh37.sqlite      ~470 MB
+├── clinvar.GRCh38.sqlite      ~480 MB
+├── gwas_catalog_associations.tsv  ~700 MB
+├── snpedia.sqlite             ~390 MB
+├── gwas.sqlite                ~190 MB
+└── pharmgkb.sqlite            ~14 MB
+```
+
+**The caches are disposable and re-downloadable.** Everything here was fetched by `allelix db update` and can be re-fetched at any time. There's no per-user state in the cache — your genotype files never enter this directory. Safe to delete to reclaim space (or to force a full refresh if you suspect a cache went stale):
+
+```bash
+# Reclaim all ~16-22 GB
+rm -rf ~/.local/share/allelix/
+
+# Reclaim everything except CADD (avoids re-downloading the slowest one)
+find ~/.local/share/allelix/ -mindepth 1 ! -name 'cadd*.sqlite*' -delete
+
+# Re-populate from scratch next time you analyze a file
+allelix db update
+```
+
+**The one thing worth backing up: `config.toml`.** It lives separately (XDG config dir, typically `~/.config/allelix/config.toml`) and stores your license assertions (commercial-mode toggle, CADD opt-in confirmation) and per-source enable/disable settings. It's the only non-reproducible state in the install. The caches will rebuild on the next `db update`; `config.toml` won't.
+
+To fully uninstall allelix and reclaim everything:
+
+```bash
+pip uninstall allelix
+rm -rf ~/.local/share/allelix/    # reference database caches
+rm -rf ~/.config/allelix/         # license + source-toggle config
+```
+
+A `allelix db clean` subcommand to automate the cache-clear step (with confirmation prompt and size report) is planned for v2.2 alongside `allelix db path` (prints the resolved data directory for scripting / backup integration).
+
 ## Data Sources & Licensing
 
 Allelix source code is licensed under the **GNU Affero General Public License v3.0 or later** (AGPL-3.0-or-later). Allelix ships with **zero third-party data**. All reference databases are downloaded by the user at runtime via `allelix db update`. Each database retains its original license on the user's machine:
@@ -256,6 +318,42 @@ Of the 104,806 genotype pages in the archive:
 
 None of these are scraping errors. They are editorial inconsistencies on the source wiki. The annotator handles all of them correctly: incomplete entries are skipped, variant title formats are matched, and no false annotations are produced.
 
+## Troubleshooting
+
+### `ImportError: cannot import name 'UTC' from 'datetime'`
+
+You're on Python 3.10 or older. Allelix requires 3.11+. `python --version` to check; upgrade and reinstall.
+
+### `allelix db update` failed partway through
+
+Run it again. The update is idempotent — completed downloads are skipped (signal-matched against the upstream freshness probe), so re-running picks up where it left off. If a specific source keeps failing, you can isolate it: `allelix db update --no-gnomad` (or `--no-alphamissense` etc.) to skip the failing source for now, then retry that source standalone later.
+
+If the failure is a network timeout on the same source repeatedly, the issue is likely on the source's end (HuggingFace / NCBI / EBI / etc. transient outage). Wait an hour and retry.
+
+### `db status` shows an annotator as "not ready" but the cache files exist on disk
+
+Almost always a schema or interpreter version bump in a recent allelix release that invalidates the pre-bump cache. The fix is `allelix db update` — it'll re-ingest only the affected annotators (everything else signal-skips). See the relevant `CHANGELOG.md` entry for "Cache invalidation" notes per release.
+
+### Build mismatch warning when analyzing a known-good file
+
+Either: (a) the file's header claims one build and the position data is actually a different build (real-world MHG / Tempus exports do this — header says 37, positions are 38; ADR-0021 documents the auto-detection), or (b) the file is a GRCh36-era export (pre-2012), which allelix detects accurately but doesn't ship a ClinVar cache for (see [`docs/grch36-liftover.md`](https://github.com/allelix/allelix/blob/main/docs/grch36-liftover.md) for the external-liftover path).
+
+If you trust the file's header more than allelix's detection, force the build: `--build grch37` or `--build grch38`. The build banner in the analyze output will reflect what was actually used.
+
+### Is my database stale? How fresh is the data?
+
+`allelix db status` shows the ingestion date of each cache. The freshness model is signal-driven: `allelix db update` queries an upstream freshness signal (md5 / etag) per source and re-ingests only when the signal changed. There's no time-based staleness — a 6-month-old cache against an unchanged source IS current.
+
+If you want to force a refresh: `allelix db update --force` re-downloads every cache regardless of freshness signal.
+
+### Output reports contain real genome annotations
+
+Anything `allelix analyze` writes to disk contains real annotations of your genome. Treat the output files (HTML, JSON, terminal redirects) as sensitive — they're as private as the input genotype file. Allelix does not transmit them; the lifecycle is local.
+
+### Where does allelix store its data?
+
+See [Managing your data](#managing-your-data) above for cache locations, the disposability rule, what to back up, and the uninstall procedure.
+
 ## Architecture & Design Decisions
 
 The "why" behind major design choices lives in [`docs/adr/`](https://github.com/allelix/allelix/blob/main/docs/adr/README.md) as Architecture Decision Records. Read these before proposing changes that touch the parser/annotator interfaces, the regulatory posture, or the data-handling model.

{allelix-2.0.2 → allelix-2.1.1}/README.md

@@ -10,19 +10,26 @@ Open-source command-line toolkit for analyzing raw genotype files from consumer
 > HTML/JSON/terminal reports, methylation + pharmacogenomics focused
 > commands, report diffing, persistent config with commercial-mode
 > safety switch. Build auto-detection from position data (ADR-0021).
-> No regex on prose anywhere in production. **Latest: v2.0.0** — VCF +
-> gVCF parser with multi-sample handling, batched annotation pipeline
-> for WGS scale, FTDNA Illumina raw parser, R-4 ClinVar CLNSIG drift CI
-> test, CLI package restructure.
+> No regex on prose anywhere in production. **Latest: v2.1.0** —
+> ADR-0035 'Cluster B' coordinated model change: `Variant.ref` carries
+> the forward-strand reference allele, per-Annotation `alt` threading
+> on GWAS / SNPedia / ClinPGx enables allele-specific gnomAD /
+> AlphaMissense / CADD enrichment on array inputs, structured GWAS
+> fields (`trait` / `p_value` / `phecode`) replace prose re-parsing in
+> the rollup, and ClinVar / ClinPGx carrier matching is strand-aware
+> with multi-allelic safety guard. JSON `schema_version` bumped 4 → 5
+> (additive, v4 readers unaffected).
 > Release notes:
 > [`CHANGELOG.md`](https://github.com/allelix/allelix/blob/main/CHANGELOG.md).
 
 ## Quickstart
 
+**Requires Python 3.11 or newer.** On Python 3.10 or older, `pip install allelix` succeeds but the first command fails with `ImportError: cannot import name 'UTC' from 'datetime'` — that's the symptom; the fix is to install on 3.11+. Check with `python --version`.
+
 ```bash
 pip install allelix
 
-# Download reference databases (~15GB with all sources).
+# Download reference databases (~16 GB default install; ~22 GB if you opt into CADD).
 # Use --no-gnomad / --no-alphamissense to skip the large ones.
 # CADD is opt-in: allelix db update --cadd
 allelix db update
@@ -40,7 +47,7 @@ allelix analyze trio.vcf.gz --sample HG002 --output report.html
 allelix analyze your_genotype_file.txt --filter-file my_panel.txt --output report.html
 ```
 
-Requires Python 3.11+. See [Development](#development) for source installs and running tests.
+See [Development](#development) for source installs and running tests, [Managing your data](#managing-your-data) for cache locations and cleanup, and [Troubleshooting](#troubleshooting) for common failure states.
 
 ## Supported Formats
 
@@ -79,7 +86,20 @@ Adding a new format means adding one file to `allelix/parsers/` and registering
 | GWAS Catalog | ✓ | Public domain (EBI/NHGRI). Trait–SNP associations with p-values and effect sizes. Carrier rule (ADR-0007) requires the user to carry the risk allele. P-value magnitude scoring (ADR-0024) maps continuous p-values to the 0–10 scale; unknown-risk-allele entries fire on rsID match alone but are capped at 3.0. |
 | gnomAD | ✓ | ODbL v1.0. **Enrichment annotator** — adds population allele frequency context to existing annotations. Shows how common each variant is in the general population (~16M exome variants from 730K individuals). A pathogenic variant that 35% of people carry reads very differently from one seen in 0.001%. Pre-built cache downloaded via `db update` (~6GB on disk). Use `--no-gnomad` to skip. |
 | AlphaMissense | ✓ | CC BY 4.0. **Enrichment annotator** — adds DeepMind's protein-structure-based pathogenicity predictions to existing annotations. Scores 71M missense variants on a 0–1 scale: <0.34 = likely benign, >0.564 = likely pathogenic. Complements ClinVar's expert classifications with computational predictions — especially valuable for variants ClinVar hasn't reviewed yet. Pre-built cache downloaded via `db update` (~8GB on disk). Use `--no-alphamissense` to skip. |
-| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. Ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%. **Opt-in** — disabled by default (`sources.cadd = false`). Enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run. Pre-built cache (~5 GB on disk, ~120M variant keys). Full mode available via pysam for GRCh38 data (`options.cadd_full = true`). Cache mode covers the large majority of variants present in gnomAD, AlphaMissense, and ClinVar — nearly every position allelix can annotate from its other databases. For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), cache and full mode produce effectively identical results because chip probes overwhelmingly target known, cataloged variants. Full mode adds coverage for novel or private variants that appear only in whole-genome or whole-exome sequencing data and are not in any pre-computed database. If your input is a genotyping chip file, cache mode is all you need. |
+| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. **Opt-in**, disabled by default; see [CADD modes](#cadd-modes) below for details. |
+
+### CADD modes
+
+CADD ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%.
+
+**Opt-in**: enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run.
+
+Two modes:
+
+- **Cache mode** (default): pre-built ~5.8 GB SQLite cache (~120M variant keys). Covers nearly every position allelix can annotate from its other databases (gnomAD, AlphaMissense, ClinVar). For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), this mode is functionally complete — chip probes overwhelmingly target known, cataloged variants.
+- **Full mode** (`options.cadd_full = true`): tabix queries against the complete CADD v1.7 file via pysam (GRCh38 only). Adds coverage for novel or private variants that appear only in WGS/WES data and are not in any pre-computed database. Requires `pip install allelix[cadd]` for the pysam dependency.
+
+If your input is a genotyping chip file, cache mode is all you need.
 
 ### Build coverage asymmetry (GRCh37 vs GRCh38)
 
@@ -177,6 +197,48 @@ gnomAD and AlphaMissense are the largest but add the most interpretive context.
 
 To skip either during download: `allelix db update --no-gnomad --no-alphamissense`. To disable permanently: `allelix config set sources.gnomad false`.
 
+### Managing your data
+
+Allelix stores reference database caches at `~/.local/share/allelix/` (Linux/macOS) or the XDG-equivalent on other platforms. Default install (no CADD) is ~16 GB on disk; with the CADD opt-in it's ~22 GB.
+
+```
+~/.local/share/allelix/
+├── alphamissense.sqlite       ~7.8 GB
+├── gnomad.sqlite              ~6.1 GB
+├── cadd.sqlite                ~5.8 GB   (only if you opted into --cadd)
+├── clinvar.GRCh37.sqlite      ~470 MB
+├── clinvar.GRCh38.sqlite      ~480 MB
+├── gwas_catalog_associations.tsv  ~700 MB
+├── snpedia.sqlite             ~390 MB
+├── gwas.sqlite                ~190 MB
+└── pharmgkb.sqlite            ~14 MB
+```
+
+**The caches are disposable and re-downloadable.** Everything here was fetched by `allelix db update` and can be re-fetched at any time. There's no per-user state in the cache — your genotype files never enter this directory. Safe to delete to reclaim space (or to force a full refresh if you suspect a cache went stale):
+
+```bash
+# Reclaim all ~16-22 GB
+rm -rf ~/.local/share/allelix/
+
+# Reclaim everything except CADD (avoids re-downloading the slowest one)
+find ~/.local/share/allelix/ -mindepth 1 ! -name 'cadd*.sqlite*' -delete
+
+# Re-populate from scratch next time you analyze a file
+allelix db update
+```
+
+**The one thing worth backing up: `config.toml`.** It lives separately (XDG config dir, typically `~/.config/allelix/config.toml`) and stores your license assertions (commercial-mode toggle, CADD opt-in confirmation) and per-source enable/disable settings. It's the only non-reproducible state in the install. The caches will rebuild on the next `db update`; `config.toml` won't.
+
+To fully uninstall allelix and reclaim everything:
+
+```bash
+pip uninstall allelix
+rm -rf ~/.local/share/allelix/    # reference database caches
+rm -rf ~/.config/allelix/         # license + source-toggle config
+```
+
+A `allelix db clean` subcommand to automate the cache-clear step (with confirmation prompt and size report) is planned for v2.2 alongside `allelix db path` (prints the resolved data directory for scripting / backup integration).
+
 ## Data Sources & Licensing
 
 Allelix source code is licensed under the **GNU Affero General Public License v3.0 or later** (AGPL-3.0-or-later). Allelix ships with **zero third-party data**. All reference databases are downloaded by the user at runtime via `allelix db update`. Each database retains its original license on the user's machine:
@@ -219,6 +281,42 @@ Of the 104,806 genotype pages in the archive:
 
 None of these are scraping errors. They are editorial inconsistencies on the source wiki. The annotator handles all of them correctly: incomplete entries are skipped, variant title formats are matched, and no false annotations are produced.
 
+## Troubleshooting
+
+### `ImportError: cannot import name 'UTC' from 'datetime'`
+
+You're on Python 3.10 or older. Allelix requires 3.11+. `python --version` to check; upgrade and reinstall.
+
+### `allelix db update` failed partway through
+
+Run it again. The update is idempotent — completed downloads are skipped (signal-matched against the upstream freshness probe), so re-running picks up where it left off. If a specific source keeps failing, you can isolate it: `allelix db update --no-gnomad` (or `--no-alphamissense` etc.) to skip the failing source for now, then retry that source standalone later.
+
+If the failure is a network timeout on the same source repeatedly, the issue is likely on the source's end (HuggingFace / NCBI / EBI / etc. transient outage). Wait an hour and retry.
+
+### `db status` shows an annotator as "not ready" but the cache files exist on disk
+
+Almost always a schema or interpreter version bump in a recent allelix release that invalidates the pre-bump cache. The fix is `allelix db update` — it'll re-ingest only the affected annotators (everything else signal-skips). See the relevant `CHANGELOG.md` entry for "Cache invalidation" notes per release.
+
+### Build mismatch warning when analyzing a known-good file
+
+Either: (a) the file's header claims one build and the position data is actually a different build (real-world MHG / Tempus exports do this — header says 37, positions are 38; ADR-0021 documents the auto-detection), or (b) the file is a GRCh36-era export (pre-2012), which allelix detects accurately but doesn't ship a ClinVar cache for (see [`docs/grch36-liftover.md`](https://github.com/allelix/allelix/blob/main/docs/grch36-liftover.md) for the external-liftover path).
+
+If you trust the file's header more than allelix's detection, force the build: `--build grch37` or `--build grch38`. The build banner in the analyze output will reflect what was actually used.
+
+### Is my database stale? How fresh is the data?
+
+`allelix db status` shows the ingestion date of each cache. The freshness model is signal-driven: `allelix db update` queries an upstream freshness signal (md5 / etag) per source and re-ingests only when the signal changed. There's no time-based staleness — a 6-month-old cache against an unchanged source IS current.
+
+If you want to force a refresh: `allelix db update --force` re-downloads every cache regardless of freshness signal.
+
+### Output reports contain real genome annotations
+
+Anything `allelix analyze` writes to disk contains real annotations of your genome. Treat the output files (HTML, JSON, terminal redirects) as sensitive — they're as private as the input genotype file. Allelix does not transmit them; the lifecycle is local.
+
+### Where does allelix store its data?
+
+See [Managing your data](#managing-your-data) above for cache locations, the disposability rule, what to back up, and the uninstall procedure.
+
 ## Architecture & Design Decisions
 
 The "why" behind major design choices lives in [`docs/adr/`](https://github.com/allelix/allelix/blob/main/docs/adr/README.md) as Architecture Decision Records. Read these before proposing changes that touch the parser/annotator interfaces, the regulatory posture, or the data-handling model.

{allelix-2.0.2 → allelix-2.1.1}/allelix/annotators/base.py

@@ -144,27 +144,36 @@ class Annotator(ABC):
     def __del__(self) -> None:
         """Safety-net resource release on GC. Deliberately retained.
 
-        GH #36 (audit second pass) flagged ``__del__`` as a Python
-        antipattern — GC timing is nondeterministic and raised exceptions
-        are silently swallowed. The correct usage pattern is the
-        ``__enter__`` / ``__exit__`` context manager pair below, wired
-        through ``contextlib.ExitStack`` in ``reports/_pipeline.py``.
-
-        However: removing ``__del__`` exposes residual SQLite connection
-        leaks in code paths that construct an annotator outside a
-        context manager. ``ResourceWarning`` is elevated to error by
-        ``pytest`` config, so leaks fail the suite as
-        ``PytestUnraisableExceptionWarning`` — caught in the v2.0.2
-        ship gate when ``__del__`` was first removed. Until every call
-        site is verified to use ``with`` / ``ExitStack`` / explicit
-        ``close()``, this safety net stays. v2.1 task: audit and
-        remove.
+        Status (v2.1.1 audit, GH #36):
+        - **Production callsites are clean.** ``run_analysis`` wires every
+          primary + enrichment annotator into ``contextlib.ExitStack``
+          (v2.0.2 fix). ``allelix/cli/utility.py`` lines 221 and 476 each
+          construct a ``GnomadAnnotator`` inside ``try / finally
+          gnomad.close()``. ``allelix/annotators/__init__.py`` returns a
+          list that the pipeline immediately wraps in ``ExitStack``.
+        - **Test callsites NOT yet clean.** ~27 sites in
+          ``tests/reports/test_pipeline.py`` instantiate annotators
+          directly without a context manager or explicit close. The
+          pipeline closes them via its own ``ExitStack`` during
+          ``run_analysis``, but the test still holds the closed-instance
+          reference, and the underlying sqlite3 ``Connection``'s GC
+          finalizer surfaces a ``ResourceWarning`` during the next test's
+          startup. pytest's ``error::PytestUnraisableExceptionWarning``
+          filter then converts it to a hard failure on the gate.
+
+        Removing ``__del__`` without first refactoring the test sites
+        therefore fails the gate (verified — 3 failures, all in
+        ``tests/reports/test_pipeline.py``, triggered by leaks from
+        earlier tests in the same file). The test-site refactor is
+        tracked separately; ``__del__`` stays as the safety net until
+        that lands. v2.1.x task — see the follow-up issue filed at
+        ``#36``-closure-time.
 
         ``contextlib.suppress(Exception)`` is deliberate — ``__del__``
-        must never raise. The GC timing and shutdown-ordering edges
-        are explicitly silenced; this is exactly the
-        "if you must keep ``__del__``, make absolutely sure it can
-        never raise" mitigation the audit recommended.
+        must never raise. GC timing and shutdown-ordering edges are
+        explicitly silenced; this is exactly the "if you must keep
+        ``__del__``, make absolutely sure it can never raise"
+        mitigation the audit recommended.
         """
         with contextlib.suppress(Exception):
             self.close()

{allelix-2.0.2 → allelix-2.1.1}/allelix/annotators/clinvar.py

@@ -30,6 +30,7 @@ from allelix.databases.manager import (
     verify_file_hash,
 )
 from allelix.models import Annotation
+from allelix.utils.allele import strand_aware_carrier_match
 
 if TYPE_CHECKING:
     from collections.abc import Iterable, Iterator
@@ -384,7 +385,6 @@ class ClinVarAnnotator(Annotator):
             (variant.rsid,),
         ).fetchall()
         annotations: list[Annotation] = []
-        carrier_alleles = {variant.allele1, variant.allele2}
         user_is_multibase = len(variant.allele1) > 1 or len(variant.allele2) > 1
         # ADR-0023: report the user's actual diploid call consistently
         # across annotators, not the matched ALT base alone.
@@ -404,7 +404,14 @@ class ClinVarAnnotator(Annotator):
             clinvar_is_indel = len(ref) > 1 or len(alt) > 1
             if clinvar_is_indel and not user_is_multibase:
                 continue
-            if alt not in carrier_alleles:
+            # ADR-0035 PR 4: strand-aware carrier match. Direct {ref, alt}
+            # membership still covers forward-stranded inputs; the
+            # complement-fallback path fires only when Variant.ref confirms
+            # reverse orientation AND the site isn't palindromic — preserving
+            # the v2.0.1 #18 multi-allelic safety invariant.
+            if not strand_aware_carrier_match(
+                variant.ref, variant.allele1, variant.allele2, ref, alt
+            ):
                 continue
             sig_label = _normalize_clnsig(clnsig) if clnsig else "unknown"
             if not self._include_benign and sig_label in _BENIGN_CLNSIGS:
@@ -562,7 +569,6 @@ class ClinVarAnnotator(Annotator):
             rows = rows_by_build_rsid.get((variant.build, variant.rsid))
             if not rows:
                 continue
-            carrier_alleles = {variant.allele1, variant.allele2}
             user_is_multibase = len(variant.allele1) > 1 or len(variant.allele2) > 1
             user_diploid = _user_diploid(variant)
             for row in rows:
@@ -580,7 +586,11 @@ class ClinVarAnnotator(Annotator):
                 clinvar_is_indel = len(ref) > 1 or len(alt) > 1
                 if clinvar_is_indel and not user_is_multibase:
                     continue
-                if alt not in carrier_alleles:
+                # ADR-0035 PR 4: strand-aware carrier match. See annotate() for
+                # the safety rationale (#18 multi-allelic invariant preserved).
+                if not strand_aware_carrier_match(
+                    variant.ref, variant.allele1, variant.allele2, ref, alt
+                ):
                     continue
                 sig_label = _normalize_clnsig(clnsig) if clnsig else "unknown"
                 if not self._include_benign and sig_label in _BENIGN_CLNSIGS:

{allelix-2.0.2 → allelix-2.1.1}/allelix/annotators/gwas.py

@@ -6,6 +6,7 @@ from __future__ import annotations
 
 import contextlib
 import logging
+import re
 import sqlite3
 import zipfile
 from typing import TYPE_CHECKING, ClassVar
@@ -56,6 +57,27 @@ logger = logging.getLogger(__name__)
 
 _BATCH_CHUNK = 500  # SQLite default SQLITE_MAX_VARIABLE_NUMBER is 999
 
+# ADR-0035 PR 3: PheCode often arrives embedded in the upstream DISEASE/TRAIT
+# field (e.g. ``"Type 2 diabetes (PheCode 250.2)"``). The annotator extracts it
+# at the boundary so downstream code reads ``Annotation.phecode`` as a
+# structured field instead of regex-parsing prose. The trimmed trait flows into
+# ``Annotation.trait``; ``description`` is rebuilt below from the structured
+# parts so display surfaces still show the PheCode label.
+_PHECODE_SUFFIX_RE = re.compile(r"\s*\(PheCode\s+([\d.]+)\)\s*$")
+
+
+def _split_trait_and_phecode(raw_trait: str) -> tuple[str, str]:
+    """Split an upstream trait into (clean_trait, phecode).
+
+    PheCode (when present) is returned as a bare numeric string like
+    ``"250.2"``. Returns ``""`` for phecode when no PheCode suffix is found.
+    The clean trait is stripped of trailing whitespace.
+    """
+    m = _PHECODE_SUFFIX_RE.search(raw_trait)
+    if m is None:
+        return raw_trait.strip(), ""
+    return raw_trait[: m.start()].rstrip(), m.group(1)
+
 
 def _magnitude(p_value: float | None, or_beta: float | None) -> float:
     """Derive magnitude from p-value and optional effect size.
@@ -288,9 +310,14 @@ class GWASCatalogAnnotator(Annotator):
                 mag = min(_magnitude(p_value, or_beta), _UNKNOWN_RISK_ALLELE_MAG_CAP)
                 risk_note = " (risk allele not specified in study)"
 
+            clean_trait, phecode = _split_trait_and_phecode(trait)
+            phecode_suffix = f" (PheCode {phecode})" if phecode else ""
             p_str = f"p={p_value:.1e}" if p_value is not None else "p=N/A"
             gene_str = gene or "—"
-            description = f"GWAS Catalog: {trait} ({p_str}, gene: {gene_str}){risk_note}"
+            description = (
+                f"GWAS Catalog: {clean_trait}{phecode_suffix} "
+                f"({p_str}, gene: {gene_str}){risk_note}"
+            )
 
             references: list[str] = []
             if pubmed_id:
@@ -309,10 +336,13 @@ class GWASCatalogAnnotator(Annotator):
                     attribution=self.attribution,
                     genotype_match=user_diploid,
                     references=references,
-                    condition=trait,
+                    condition=clean_trait,
                     gene=gene or "",
-                    alt="",
+                    alt=risk_allele or "",
                     is_must_include=variant.rsid in _MUST_INCLUDE_RSIDS,
+                    trait=clean_trait,
+                    p_value=p_value,
+                    phecode=phecode,
                 )
             )
         return annotations
@@ -377,9 +407,14 @@ class GWASCatalogAnnotator(Annotator):
                     mag = min(_magnitude(p_value, or_beta), _UNKNOWN_RISK_ALLELE_MAG_CAP)
                     risk_note = " (risk allele not specified in study)"
 
+                clean_trait, phecode = _split_trait_and_phecode(trait)
+                phecode_suffix = f" (PheCode {phecode})" if phecode else ""
                 p_str = f"p={p_value:.1e}" if p_value is not None else "p=N/A"
                 gene_str = gene or "—"
-                description = f"GWAS Catalog: {trait} ({p_str}, gene: {gene_str}){risk_note}"
+                description = (
+                    f"GWAS Catalog: {clean_trait}{phecode_suffix} "
+                    f"({p_str}, gene: {gene_str}){risk_note}"
+                )
 
                 references: list[str] = []
                 if pubmed_id:
@@ -397,10 +432,13 @@ class GWASCatalogAnnotator(Annotator):
                     attribution=self.attribution,
                     genotype_match=user_diploid,
                     references=references,
-                    condition=trait,
+                    condition=clean_trait,
                     gene=gene or "",
-                    alt="",
+                    alt=risk_allele or "",
                     is_must_include=variant.rsid in _MUST_INCLUDE_RSIDS,
+                    trait=clean_trait,
+                    p_value=p_value,
+                    phecode=phecode,
                 )
 
 

{allelix-2.0.2 → allelix-2.1.1}/allelix/annotators/pharmgkb.py

@@ -30,6 +30,7 @@ from allelix.databases.pharmgkb_loader import (
     schema_is_current,
 )
 from allelix.models import Annotation
+from allelix.utils.allele import derive_alt_from_diploid, strand_aware_genotype_match
 
 if TYPE_CHECKING:
     from collections.abc import Callable, Iterable, Iterator
@@ -260,20 +261,25 @@ class PharmGKBAnnotator(Annotator):
         if is_clinvar_homref(variant, self._clinvar_ref_provider):
             return []
 
+        # ADR-0035 PR 4: fetch all rows for the rsid (typically <5) and apply
+        # strand-aware genotype match in Python. Forward-stranded users hit
+        # the same set of rows as the v0.7-era ``WHERE genotype = ?`` SQL;
+        # reverse-stranded users now also match when Variant.ref confirms
+        # orientation AND the source genotype is non-palindromic.
         sql = (
             "SELECT genotype, gene, drugs, phenotype, phenotype_category, "
             "annotation_text, level_of_evidence, score, pgkb_annotation_id "
             "FROM pharmgkb_annotations "
-            "WHERE rsid = ? AND genotype = ? AND is_nonfinding = 0"
+            "WHERE rsid = ? AND is_nonfinding = 0"
         )
-        params = (variant.rsid, user_geno)
+        params = (variant.rsid,)
 
         rows = self._connection().execute(sql, params).fetchall()
         annotations: list[Annotation] = []
         user_diploid = _user_diploid(variant)
         for row in rows:
             (
-                _geno,
+                geno,
                 gene,
                 drugs,
                 phenotype,
@@ -283,6 +289,10 @@ class PharmGKBAnnotator(Annotator):
                 _score,
                 pgkb_annotation_id,
             ) = row
+            if not strand_aware_genotype_match(
+                variant.allele1, variant.allele2, geno, variant.ref
+            ):
+                continue
             sig_label = level_of_evidence.strip().lower() or "unknown"
             description_parts = [f"ClinPGx: {drugs}"] if drugs else ["ClinPGx"]
             if phenotype:
@@ -306,6 +316,7 @@ class PharmGKBAnnotator(Annotator):
                     references=references,
                     condition=phenotype or "",
                     gene=gene or "",
+                    alt=derive_alt_from_diploid(variant.ref, variant.allele1, variant.allele2),
                 )
             )
         return annotations
@@ -352,7 +363,7 @@ class PharmGKBAnnotator(Annotator):
             for row in cursor:
                 rows_by_rsid.setdefault(row[0], []).append(row[1:])
 
-        for variant, user_geno in candidates:
+        for variant, _user_geno in candidates:
             rows = rows_by_rsid.get(variant.rsid)
             if not rows:
                 continue
@@ -369,7 +380,11 @@ class PharmGKBAnnotator(Annotator):
                     _score,
                     pgkb_annotation_id,
                 ) = row
-                if geno != user_geno:
+                # ADR-0035 PR 4: strand-aware genotype match. See annotate() for
+                # the safety rationale (palindrome guard + variant.ref orientation).
+                if not strand_aware_genotype_match(
+                    variant.allele1, variant.allele2, geno, variant.ref
+                ):
                     continue
                 sig_label = level_of_evidence.strip().lower() or "unknown"
                 description_parts = [f"ClinPGx: {drugs}"] if drugs else ["ClinPGx"]
@@ -393,6 +408,7 @@ class PharmGKBAnnotator(Annotator):
                     references=references,
                     condition=phenotype or "",
                     gene=gene or "",
+                    alt=derive_alt_from_diploid(variant.ref, variant.allele1, variant.allele2),
                 )
 
 

{allelix-2.0.2 → allelix-2.1.1}/allelix/annotators/snpedia.py

@@ -29,6 +29,7 @@ from allelix.databases.snpedia_loader import (
     install_prebuilt_cache,
 )
 from allelix.models import Annotation
+from allelix.utils.allele import derive_alt_from_diploid
 
 if TYPE_CHECKING:
     from collections.abc import Callable, Iterable, Iterator
@@ -272,6 +273,7 @@ class SNPediaAnnotator(Annotator):
                     genotype_match=genotype_match,
                     references=[f"https://www.snpedia.com/index.php/{snp_url_path}"],
                     gene=gene or "",
+                    alt=derive_alt_from_diploid(variant.ref, allele1, allele2),
                 )
             )
 
@@ -356,4 +358,5 @@ class SNPediaAnnotator(Annotator):
                     genotype_match=genotype_match,
                     references=[f"https://www.snpedia.com/index.php/{snp_url_path}"],
                     gene=gene or "",
+                    alt=derive_alt_from_diploid(variant.ref, allele1, allele2),
                 )