allelix 2.1.0__tar.gz → 2.1.2__tar.gz

{allelix-2.1.0 → allelix-2.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: allelix
-Version: 2.1.0
+Version: 2.1.2
 Summary: Open-source genotype analysis toolkit. Format-agnostic ingestion, database-agnostic annotation, offline-first.
 Author: Allelix
 Maintainer-email: dial481 <dial481@users.noreply.github.com>
@@ -29,37 +29,42 @@ Requires-Dist: pysam>=0.22; extra == "cadd"
 Provides-Extra: vcf-index
 Requires-Dist: pysam>=0.22; extra == "vcf-index"
 Provides-Extra: dev
+Requires-Dist: mypy>=1.11; extra == "dev"
 Requires-Dist: pre-commit>=3.7; extra == "dev"
 Requires-Dist: pytest>=8.0; extra == "dev"
 Requires-Dist: pytest-cov<8,>=7; extra == "dev"
 Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: types-PyYAML>=6.0; extra == "dev"
 Dynamic: license-file
 
+<div align="center">
+
+<img src="https://allelix.io/apple-touch-icon.png" width="180" alt="Allelix" />
+
+**Open-source genotype analysis. Format-agnostic. Offline-first.**
+
+[Website](https://allelix.io) · [Quickstart](#quickstart) · [Changelog](https://github.com/allelix/allelix/blob/main/CHANGELOG.md)
+
+[![python](https://img.shields.io/pypi/pyversions/allelix.svg)](https://www.python.org/downloads/)
+[![pypi](https://img.shields.io/pypi/v/allelix.svg)](https://pypi.org/project/allelix/)
+[![license](https://img.shields.io/pypi/l/allelix.svg)](LICENSE)
+[![CI](https://github.com/allelix/allelix/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/allelix/allelix/actions/workflows/ci.yml)
+[![privacy: local-first](https://img.shields.io/badge/privacy-local--first-success)](#privacy)
+
+</div>
+
 # Allelix
 
-Open-source command-line toolkit for analyzing raw genotype files from consumer DNA testing services. Format-agnostic ingestion, database-agnostic annotation, offline-first.
-
-> **Status:** Production — eight parser formats (including VCF + gVCF),
-> four annotators (ClinVar + ClinPGx + GWAS Catalog + SNPedia), three
-> enrichment sources (gnomAD population frequencies + AlphaMissense
-> pathogenicity + CADD deleteriousness), licensable-source gating for
-> commercial users, dual-build ClinVar caches (GRCh37 + GRCh38),
-> HTML/JSON/terminal reports, methylation + pharmacogenomics focused
-> commands, report diffing, persistent config with commercial-mode
-> safety switch. Build auto-detection from position data (ADR-0021).
-> No regex on prose anywhere in production. **Latest: v2.0.0** — VCF +
-> gVCF parser with multi-sample handling, batched annotation pipeline
-> for WGS scale, FTDNA Illumina raw parser, R-4 ClinVar CLNSIG drift CI
-> test, CLI package restructure.
-> Release notes:
-> [`CHANGELOG.md`](https://github.com/allelix/allelix/blob/main/CHANGELOG.md).
+> **Status:** Production. Eight parser formats (including VCF + gVCF), four annotators (ClinVar + ClinPGx + GWAS Catalog + SNPedia), three enrichment sources (gnomAD + AlphaMissense + CADD), licensable-source gating for commercial users, dual-build ClinVar caches (GRCh37 + GRCh38), HTML/JSON/terminal reports, methylation + pharmacogenomics focused commands, report diffing, persistent config with commercial-mode safety switch. Build auto-detection from position data (ADR-0021). No regex on prose anywhere in production. The [Changelog](https://github.com/allelix/allelix/blob/main/CHANGELOG.md) tracks every release.
 
 ## Quickstart
 
+**Requires Python 3.11 or newer.** On Python 3.10 or older, `pip install allelix` succeeds but the first command fails with `ImportError: cannot import name 'UTC' from 'datetime'` — that's the symptom; the fix is to install on 3.11+. Check with `python --version`.
+
 ```bash
 pip install allelix
 
-# Download reference databases (~15GB with all sources).
+# Download reference databases (~16 GB default install; ~22 GB if you opt into CADD).
 # Use --no-gnomad / --no-alphamissense to skip the large ones.
 # CADD is opt-in: allelix db update --cadd
 allelix db update
@@ -77,7 +82,7 @@ allelix analyze trio.vcf.gz --sample HG002 --output report.html
 allelix analyze your_genotype_file.txt --filter-file my_panel.txt --output report.html
 ```
 
-Requires Python 3.11+. See [Development](#development) for source installs and running tests.
+See [Development](#development) for source installs and running tests, [Managing your data](#managing-your-data) for cache locations and cleanup, and [Troubleshooting](#troubleshooting) for common failure states.
 
 ## Supported Formats
 
@@ -94,16 +99,17 @@ Requires Python 3.11+. See [Development](#development) for source installs and r
 
 Adding a new format means adding one file to `allelix/parsers/` and registering an instance in the `PARSERS` list in `allelix/parsers/__init__.py`.
 
-### v2.1+ roadmap
+### Roadmap
 
-| Feature | Notes |
+| Release | Theme |
 |---|---|
-| Per-source scoring | Magnitude breakdown by database. Users see which source drove the composite score. |
-| Annotator-level strand awareness (R-1) | Strand-flip matching wired into every annotator's carrier check. Basic `compare` strand support shipped in v1.1; full annotator integration deferred from v2.0.0. |
-| Good / Bad / Neutral repute | Per-annotation repute field. Reframes the report from "here's what's wrong" to "here's your full picture." Requires Annotation model change + renderer updates. |
-| PLINK import | Read .bed/.bim/.fam as an input format (complement to the v1.7.0 export). |
-| PharmCAT integration | Wrap CPIC's PharmCAT as an optional external engine for star-allele / diplotype calling. Requires VCF input (shipped in v2.0.0). |
-| Genome Watchtower | Real-time variant monitoring via database delta feeds. Privacy-preserving: server publishes universal feed, matching happens locally against your deviation set. Replaces full re-analysis with millisecond set intersection. |
+| **v2.2** | Annotation model expansion (cross-source conflict surfacing, risk-allele display, structured MTAG flag), custom-panel UX improvements, methylation panel retune + citations, ClinVar source switch (VCF → `variant_summary.txt`), CLI cleanups (`db clean` / `db path`). |
+| **v2.3** | Per-source magnitude decomposition, Good / Bad / Neutral annotation repute, JSON v3 (AI-legible output contract), plausibility flagging via zygosity × gnomAD MAF, ClinVar review-status weighting, functional-medicine +/− notation. |
+| **v2.4** | ACMG/AMP automated classification engine, variant consequence annotation (VEP-equivalent). |
+| **v2.5+** | CNVs and repeat-expansion support, dbSNP resolution for rsID-less VCFs, gnomAD genomes (non-coding population frequencies), supplemental genotype file merging. |
+| **v3.0** | Ancestry estimation (PCA + 1KG/HGDP reference panels), Polygenic Risk Score (PRS) integration. |
+
+Issues and milestones are tracked at [github.com/allelix/allelix/issues](https://github.com/allelix/allelix/issues). Annotator-level strand-aware carrier matching (R-1) shipped in v2.1.0's ADR-0035 cluster.
 
 ## Supported Databases
 
@@ -116,7 +122,20 @@ Adding a new format means adding one file to `allelix/parsers/` and registering
 | GWAS Catalog | ✓ | Public domain (EBI/NHGRI). Trait–SNP associations with p-values and effect sizes. Carrier rule (ADR-0007) requires the user to carry the risk allele. P-value magnitude scoring (ADR-0024) maps continuous p-values to the 0–10 scale; unknown-risk-allele entries fire on rsID match alone but are capped at 3.0. |
 | gnomAD | ✓ | ODbL v1.0. **Enrichment annotator** — adds population allele frequency context to existing annotations. Shows how common each variant is in the general population (~16M exome variants from 730K individuals). A pathogenic variant that 35% of people carry reads very differently from one seen in 0.001%. Pre-built cache downloaded via `db update` (~6GB on disk). Use `--no-gnomad` to skip. |
 | AlphaMissense | ✓ | CC BY 4.0. **Enrichment annotator** — adds DeepMind's protein-structure-based pathogenicity predictions to existing annotations. Scores 71M missense variants on a 0–1 scale: <0.34 = likely benign, >0.564 = likely pathogenic. Complements ClinVar's expert classifications with computational predictions — especially valuable for variants ClinVar hasn't reviewed yet. Pre-built cache downloaded via `db update` (~8GB on disk). Use `--no-alphamissense` to skip. |
-| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. Ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%. **Opt-in** — disabled by default (`sources.cadd = false`). Enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run. Pre-built cache (~5 GB on disk, ~120M variant keys). Full mode available via pysam for GRCh38 data (`options.cadd_full = true`). Cache mode covers the large majority of variants present in gnomAD, AlphaMissense, and ClinVar — nearly every position allelix can annotate from its other databases. For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), cache and full mode produce effectively identical results because chip probes overwhelmingly target known, cataloged variants. Full mode adds coverage for novel or private variants that appear only in whole-genome or whole-exome sequencing data and are not in any pre-computed database. If your input is a genotyping chip file, cache mode is all you need. |
+| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. **Opt-in**, disabled by default; see [CADD modes](#cadd-modes) below for details. |
+
+### CADD modes
+
+CADD ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%.
+
+**Opt-in**: enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run.
+
+Two modes:
+
+- **Cache mode** (default): pre-built ~5.8 GB SQLite cache (~120M variant keys). Covers nearly every position allelix can annotate from its other databases (gnomAD, AlphaMissense, ClinVar). For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), this mode is functionally complete — chip probes overwhelmingly target known, cataloged variants.
+- **Full mode** (`options.cadd_full = true`): tabix queries against the complete CADD v1.7 file via pysam (GRCh38 only). Adds coverage for novel or private variants that appear only in WGS/WES data and are not in any pre-computed database. Requires `pip install allelix[cadd]` for the pysam dependency.
+
+If your input is a genotyping chip file, cache mode is all you need.
 
 ### Build coverage asymmetry (GRCh37 vs GRCh38)
 
@@ -214,6 +233,48 @@ gnomAD and AlphaMissense are the largest but add the most interpretive context.
 
 To skip either during download: `allelix db update --no-gnomad --no-alphamissense`. To disable permanently: `allelix config set sources.gnomad false`.
 
+### Managing your data
+
+Allelix stores reference database caches at `~/.local/share/allelix/` (Linux/macOS) or the XDG-equivalent on other platforms. Default install (no CADD) is ~16 GB on disk; with the CADD opt-in it's ~22 GB.
+
+```
+~/.local/share/allelix/
+├── alphamissense.sqlite       ~7.8 GB
+├── gnomad.sqlite              ~6.1 GB
+├── cadd.sqlite                ~5.8 GB   (only if you opted into --cadd)
+├── clinvar.GRCh37.sqlite      ~470 MB
+├── clinvar.GRCh38.sqlite      ~480 MB
+├── gwas_catalog_associations.tsv  ~700 MB
+├── snpedia.sqlite             ~390 MB
+├── gwas.sqlite                ~190 MB
+└── pharmgkb.sqlite            ~14 MB
+```
+
+**The caches are disposable and re-downloadable.** Everything here was fetched by `allelix db update` and can be re-fetched at any time. There's no per-user state in the cache — your genotype files never enter this directory. Safe to delete to reclaim space (or to force a full refresh if you suspect a cache went stale):
+
+```bash
+# Reclaim all ~16-22 GB
+rm -rf ~/.local/share/allelix/
+
+# Reclaim everything except CADD (avoids re-downloading the slowest one)
+find ~/.local/share/allelix/ -mindepth 1 ! -name 'cadd*.sqlite*' -delete
+
+# Re-populate from scratch next time you analyze a file
+allelix db update
+```
+
+**The one thing worth backing up: `config.toml`.** It lives separately (XDG config dir, typically `~/.config/allelix/config.toml`) and stores your license assertions (commercial-mode toggle, CADD opt-in confirmation) and per-source enable/disable settings. It's the only non-reproducible state in the install. The caches will rebuild on the next `db update`; `config.toml` won't.
+
+To fully uninstall allelix and reclaim everything:
+
+```bash
+pip uninstall allelix
+rm -rf ~/.local/share/allelix/    # reference database caches
+rm -rf ~/.config/allelix/         # license + source-toggle config
+```
+
+A `allelix db clean` subcommand to automate the cache-clear step (with confirmation prompt and size report) is planned for v2.2 alongside `allelix db path` (prints the resolved data directory for scripting / backup integration).
+
 ## Data Sources & Licensing
 
 Allelix source code is licensed under the **GNU Affero General Public License v3.0 or later** (AGPL-3.0-or-later). Allelix ships with **zero third-party data**. All reference databases are downloaded by the user at runtime via `allelix db update`. Each database retains its original license on the user's machine:
@@ -256,6 +317,42 @@ Of the 104,806 genotype pages in the archive:
 
 None of these are scraping errors. They are editorial inconsistencies on the source wiki. The annotator handles all of them correctly: incomplete entries are skipped, variant title formats are matched, and no false annotations are produced.
 
+## Troubleshooting
+
+### `ImportError: cannot import name 'UTC' from 'datetime'`
+
+You're on Python 3.10 or older. Allelix requires 3.11+. `python --version` to check; upgrade and reinstall.
+
+### `allelix db update` failed partway through
+
+Run it again. The update is idempotent — completed downloads are skipped (signal-matched against the upstream freshness probe), so re-running picks up where it left off. If a specific source keeps failing, you can isolate it: `allelix db update --no-gnomad` (or `--no-alphamissense` etc.) to skip the failing source for now, then retry that source standalone later.
+
+If the failure is a network timeout on the same source repeatedly, the issue is likely on the source's end (HuggingFace / NCBI / EBI / etc. transient outage). Wait an hour and retry.
+
+### `db status` shows an annotator as "not ready" but the cache files exist on disk
+
+Almost always a schema or interpreter version bump in a recent allelix release that invalidates the pre-bump cache. The fix is `allelix db update` — it'll re-ingest only the affected annotators (everything else signal-skips). See the relevant `CHANGELOG.md` entry for "Cache invalidation" notes per release.
+
+### Build mismatch warning when analyzing a known-good file
+
+Either: (a) the file's header claims one build and the position data is actually a different build (real-world MHG / Tempus exports do this — header says 37, positions are 38; ADR-0021 documents the auto-detection), or (b) the file is a GRCh36-era export (pre-2012), which allelix detects accurately but doesn't ship a ClinVar cache for (see [`docs/grch36-liftover.md`](https://github.com/allelix/allelix/blob/main/docs/grch36-liftover.md) for the external-liftover path).
+
+If you trust the file's header more than allelix's detection, force the build: `--build grch37` or `--build grch38`. The build banner in the analyze output will reflect what was actually used.
+
+### Is my database stale? How fresh is the data?
+
+`allelix db status` shows the ingestion date of each cache. The freshness model is signal-driven: `allelix db update` queries an upstream freshness signal (md5 / etag) per source and re-ingests only when the signal changed. There's no time-based staleness — a 6-month-old cache against an unchanged source IS current.
+
+If you want to force a refresh: `allelix db update --force` re-downloads every cache regardless of freshness signal.
+
+### Output reports contain real genome annotations
+
+Anything `allelix analyze` writes to disk contains real annotations of your genome. Treat the output files (HTML, JSON, terminal redirects) as sensitive — they're as private as the input genotype file. Allelix does not transmit them; the lifecycle is local.
+
+### Where does allelix store its data?
+
+See [Managing your data](#managing-your-data) above for cache locations, the disposability rule, what to back up, and the uninstall procedure.
+
 ## Architecture & Design Decisions
 
 The "why" behind major design choices lives in [`docs/adr/`](https://github.com/allelix/allelix/blob/main/docs/adr/README.md) as Architecture Decision Records. Read these before proposing changes that touch the parser/annotator interfaces, the regulatory posture, or the data-handling model.

{allelix-2.1.0 → allelix-2.1.2}/README.md

@@ -1,28 +1,31 @@
+<div align="center">
+
+<img src="https://allelix.io/apple-touch-icon.png" width="180" alt="Allelix" />
+
+**Open-source genotype analysis. Format-agnostic. Offline-first.**
+
+[Website](https://allelix.io) · [Quickstart](#quickstart) · [Changelog](https://github.com/allelix/allelix/blob/main/CHANGELOG.md)
+
+[![python](https://img.shields.io/pypi/pyversions/allelix.svg)](https://www.python.org/downloads/)
+[![pypi](https://img.shields.io/pypi/v/allelix.svg)](https://pypi.org/project/allelix/)
+[![license](https://img.shields.io/pypi/l/allelix.svg)](LICENSE)
+[![CI](https://github.com/allelix/allelix/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/allelix/allelix/actions/workflows/ci.yml)
+[![privacy: local-first](https://img.shields.io/badge/privacy-local--first-success)](#privacy)
+
+</div>
+
 # Allelix
 
-Open-source command-line toolkit for analyzing raw genotype files from consumer DNA testing services. Format-agnostic ingestion, database-agnostic annotation, offline-first.
-
-> **Status:** Production — eight parser formats (including VCF + gVCF),
-> four annotators (ClinVar + ClinPGx + GWAS Catalog + SNPedia), three
-> enrichment sources (gnomAD population frequencies + AlphaMissense
-> pathogenicity + CADD deleteriousness), licensable-source gating for
-> commercial users, dual-build ClinVar caches (GRCh37 + GRCh38),
-> HTML/JSON/terminal reports, methylation + pharmacogenomics focused
-> commands, report diffing, persistent config with commercial-mode
-> safety switch. Build auto-detection from position data (ADR-0021).
-> No regex on prose anywhere in production. **Latest: v2.0.0** — VCF +
-> gVCF parser with multi-sample handling, batched annotation pipeline
-> for WGS scale, FTDNA Illumina raw parser, R-4 ClinVar CLNSIG drift CI
-> test, CLI package restructure.
-> Release notes:
-> [`CHANGELOG.md`](https://github.com/allelix/allelix/blob/main/CHANGELOG.md).
+> **Status:** Production. Eight parser formats (including VCF + gVCF), four annotators (ClinVar + ClinPGx + GWAS Catalog + SNPedia), three enrichment sources (gnomAD + AlphaMissense + CADD), licensable-source gating for commercial users, dual-build ClinVar caches (GRCh37 + GRCh38), HTML/JSON/terminal reports, methylation + pharmacogenomics focused commands, report diffing, persistent config with commercial-mode safety switch. Build auto-detection from position data (ADR-0021). No regex on prose anywhere in production. The [Changelog](https://github.com/allelix/allelix/blob/main/CHANGELOG.md) tracks every release.
 
 ## Quickstart
 
+**Requires Python 3.11 or newer.** On Python 3.10 or older, `pip install allelix` succeeds but the first command fails with `ImportError: cannot import name 'UTC' from 'datetime'` — that's the symptom; the fix is to install on 3.11+. Check with `python --version`.
+
 ```bash
 pip install allelix
 
-# Download reference databases (~15GB with all sources).
+# Download reference databases (~16 GB default install; ~22 GB if you opt into CADD).
 # Use --no-gnomad / --no-alphamissense to skip the large ones.
 # CADD is opt-in: allelix db update --cadd
 allelix db update
@@ -40,7 +43,7 @@ allelix analyze trio.vcf.gz --sample HG002 --output report.html
 allelix analyze your_genotype_file.txt --filter-file my_panel.txt --output report.html
 ```
 
-Requires Python 3.11+. See [Development](#development) for source installs and running tests.
+See [Development](#development) for source installs and running tests, [Managing your data](#managing-your-data) for cache locations and cleanup, and [Troubleshooting](#troubleshooting) for common failure states.
 
 ## Supported Formats
 
@@ -57,16 +60,17 @@ Requires Python 3.11+. See [Development](#development) for source installs and r
 
 Adding a new format means adding one file to `allelix/parsers/` and registering an instance in the `PARSERS` list in `allelix/parsers/__init__.py`.
 
-### v2.1+ roadmap
+### Roadmap
 
-| Feature | Notes |
+| Release | Theme |
 |---|---|
-| Per-source scoring | Magnitude breakdown by database. Users see which source drove the composite score. |
-| Annotator-level strand awareness (R-1) | Strand-flip matching wired into every annotator's carrier check. Basic `compare` strand support shipped in v1.1; full annotator integration deferred from v2.0.0. |
-| Good / Bad / Neutral repute | Per-annotation repute field. Reframes the report from "here's what's wrong" to "here's your full picture." Requires Annotation model change + renderer updates. |
-| PLINK import | Read .bed/.bim/.fam as an input format (complement to the v1.7.0 export). |
-| PharmCAT integration | Wrap CPIC's PharmCAT as an optional external engine for star-allele / diplotype calling. Requires VCF input (shipped in v2.0.0). |
-| Genome Watchtower | Real-time variant monitoring via database delta feeds. Privacy-preserving: server publishes universal feed, matching happens locally against your deviation set. Replaces full re-analysis with millisecond set intersection. |
+| **v2.2** | Annotation model expansion (cross-source conflict surfacing, risk-allele display, structured MTAG flag), custom-panel UX improvements, methylation panel retune + citations, ClinVar source switch (VCF → `variant_summary.txt`), CLI cleanups (`db clean` / `db path`). |
+| **v2.3** | Per-source magnitude decomposition, Good / Bad / Neutral annotation repute, JSON v3 (AI-legible output contract), plausibility flagging via zygosity × gnomAD MAF, ClinVar review-status weighting, functional-medicine +/− notation. |
+| **v2.4** | ACMG/AMP automated classification engine, variant consequence annotation (VEP-equivalent). |
+| **v2.5+** | CNVs and repeat-expansion support, dbSNP resolution for rsID-less VCFs, gnomAD genomes (non-coding population frequencies), supplemental genotype file merging. |
+| **v3.0** | Ancestry estimation (PCA + 1KG/HGDP reference panels), Polygenic Risk Score (PRS) integration. |
+
+Issues and milestones are tracked at [github.com/allelix/allelix/issues](https://github.com/allelix/allelix/issues). Annotator-level strand-aware carrier matching (R-1) shipped in v2.1.0's ADR-0035 cluster.
 
 ## Supported Databases
 
@@ -79,7 +83,20 @@ Adding a new format means adding one file to `allelix/parsers/` and registering
 | GWAS Catalog | ✓ | Public domain (EBI/NHGRI). Trait–SNP associations with p-values and effect sizes. Carrier rule (ADR-0007) requires the user to carry the risk allele. P-value magnitude scoring (ADR-0024) maps continuous p-values to the 0–10 scale; unknown-risk-allele entries fire on rsID match alone but are capped at 3.0. |
 | gnomAD | ✓ | ODbL v1.0. **Enrichment annotator** — adds population allele frequency context to existing annotations. Shows how common each variant is in the general population (~16M exome variants from 730K individuals). A pathogenic variant that 35% of people carry reads very differently from one seen in 0.001%. Pre-built cache downloaded via `db update` (~6GB on disk). Use `--no-gnomad` to skip. |
 | AlphaMissense | ✓ | CC BY 4.0. **Enrichment annotator** — adds DeepMind's protein-structure-based pathogenicity predictions to existing annotations. Scores 71M missense variants on a 0–1 scale: <0.34 = likely benign, >0.564 = likely pathogenic. Complements ClinVar's expert classifications with computational predictions — especially valuable for variants ClinVar hasn't reviewed yet. Pre-built cache downloaded via `db update` (~8GB on disk). Use `--no-alphamissense` to skip. |
-| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. Ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%. **Opt-in** — disabled by default (`sources.cadd = false`). Enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run. Pre-built cache (~5 GB on disk, ~120M variant keys). Full mode available via pysam for GRCh38 data (`options.cadd_full = true`). Cache mode covers the large majority of variants present in gnomAD, AlphaMissense, and ClinVar — nearly every position allelix can annotate from its other databases. For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), cache and full mode produce effectively identical results because chip probes overwhelmingly target known, cataloged variants. Full mode adds coverage for novel or private variants that appear only in whole-genome or whole-exome sequencing data and are not in any pre-computed database. If your input is a genotyping chip file, cache mode is all you need. |
+| CADD | ✓ | LicenseRef-CADD (non-commercial). **Enrichment annotator** — adds PHRED-scaled deleteriousness scores from CADD v1.7. **Opt-in**, disabled by default; see [CADD modes](#cadd-modes) below for details. |
+
+### CADD modes
+
+CADD ranks how deleterious any single-nucleotide variant is using 100+ annotation tracks (coding, non-coding, regulatory). PHRED 10 = top 10% most deleterious, 20 = top 1%, 30 = top 0.1%.
+
+**Opt-in**: enable via `allelix db update --cadd` or `allelix config set sources.cadd true`. Use `--no-cadd` to skip enrichment for a single run.
+
+Two modes:
+
+- **Cache mode** (default): pre-built ~5.8 GB SQLite cache (~120M variant keys). Covers nearly every position allelix can annotate from its other databases (gnomAD, AlphaMissense, ClinVar). For genotyping chip data (23andMe, AncestryDNA, MyHappyGenes, etc.), this mode is functionally complete — chip probes overwhelmingly target known, cataloged variants.
+- **Full mode** (`options.cadd_full = true`): tabix queries against the complete CADD v1.7 file via pysam (GRCh38 only). Adds coverage for novel or private variants that appear only in WGS/WES data and are not in any pre-computed database. Requires `pip install allelix[cadd]` for the pysam dependency.
+
+If your input is a genotyping chip file, cache mode is all you need.
 
 ### Build coverage asymmetry (GRCh37 vs GRCh38)
 
@@ -177,6 +194,48 @@ gnomAD and AlphaMissense are the largest but add the most interpretive context.
 
 To skip either during download: `allelix db update --no-gnomad --no-alphamissense`. To disable permanently: `allelix config set sources.gnomad false`.
 
+### Managing your data
+
+Allelix stores reference database caches at `~/.local/share/allelix/` (Linux/macOS) or the XDG-equivalent on other platforms. Default install (no CADD) is ~16 GB on disk; with the CADD opt-in it's ~22 GB.
+
+```
+~/.local/share/allelix/
+├── alphamissense.sqlite       ~7.8 GB
+├── gnomad.sqlite              ~6.1 GB
+├── cadd.sqlite                ~5.8 GB   (only if you opted into --cadd)
+├── clinvar.GRCh37.sqlite      ~470 MB
+├── clinvar.GRCh38.sqlite      ~480 MB
+├── gwas_catalog_associations.tsv  ~700 MB
+├── snpedia.sqlite             ~390 MB
+├── gwas.sqlite                ~190 MB
+└── pharmgkb.sqlite            ~14 MB
+```
+
+**The caches are disposable and re-downloadable.** Everything here was fetched by `allelix db update` and can be re-fetched at any time. There's no per-user state in the cache — your genotype files never enter this directory. Safe to delete to reclaim space (or to force a full refresh if you suspect a cache went stale):
+
+```bash
+# Reclaim all ~16-22 GB
+rm -rf ~/.local/share/allelix/
+
+# Reclaim everything except CADD (avoids re-downloading the slowest one)
+find ~/.local/share/allelix/ -mindepth 1 ! -name 'cadd*.sqlite*' -delete
+
+# Re-populate from scratch next time you analyze a file
+allelix db update
+```
+
+**The one thing worth backing up: `config.toml`.** It lives separately (XDG config dir, typically `~/.config/allelix/config.toml`) and stores your license assertions (commercial-mode toggle, CADD opt-in confirmation) and per-source enable/disable settings. It's the only non-reproducible state in the install. The caches will rebuild on the next `db update`; `config.toml` won't.
+
+To fully uninstall allelix and reclaim everything:
+
+```bash
+pip uninstall allelix
+rm -rf ~/.local/share/allelix/    # reference database caches
+rm -rf ~/.config/allelix/         # license + source-toggle config
+```
+
+A `allelix db clean` subcommand to automate the cache-clear step (with confirmation prompt and size report) is planned for v2.2 alongside `allelix db path` (prints the resolved data directory for scripting / backup integration).
+
 ## Data Sources & Licensing
 
 Allelix source code is licensed under the **GNU Affero General Public License v3.0 or later** (AGPL-3.0-or-later). Allelix ships with **zero third-party data**. All reference databases are downloaded by the user at runtime via `allelix db update`. Each database retains its original license on the user's machine:
@@ -219,6 +278,42 @@ Of the 104,806 genotype pages in the archive:
 
 None of these are scraping errors. They are editorial inconsistencies on the source wiki. The annotator handles all of them correctly: incomplete entries are skipped, variant title formats are matched, and no false annotations are produced.
 
+## Troubleshooting
+
+### `ImportError: cannot import name 'UTC' from 'datetime'`
+
+You're on Python 3.10 or older. Allelix requires 3.11+. `python --version` to check; upgrade and reinstall.
+
+### `allelix db update` failed partway through
+
+Run it again. The update is idempotent — completed downloads are skipped (signal-matched against the upstream freshness probe), so re-running picks up where it left off. If a specific source keeps failing, you can isolate it: `allelix db update --no-gnomad` (or `--no-alphamissense` etc.) to skip the failing source for now, then retry that source standalone later.
+
+If the failure is a network timeout on the same source repeatedly, the issue is likely on the source's end (HuggingFace / NCBI / EBI / etc. transient outage). Wait an hour and retry.
+
+### `db status` shows an annotator as "not ready" but the cache files exist on disk
+
+Almost always a schema or interpreter version bump in a recent allelix release that invalidates the pre-bump cache. The fix is `allelix db update` — it'll re-ingest only the affected annotators (everything else signal-skips). See the relevant `CHANGELOG.md` entry for "Cache invalidation" notes per release.
+
+### Build mismatch warning when analyzing a known-good file
+
+Either: (a) the file's header claims one build and the position data is actually a different build (real-world MHG / Tempus exports do this — header says 37, positions are 38; ADR-0021 documents the auto-detection), or (b) the file is a GRCh36-era export (pre-2012), which allelix detects accurately but doesn't ship a ClinVar cache for (see [`docs/grch36-liftover.md`](https://github.com/allelix/allelix/blob/main/docs/grch36-liftover.md) for the external-liftover path).
+
+If you trust the file's header more than allelix's detection, force the build: `--build grch37` or `--build grch38`. The build banner in the analyze output will reflect what was actually used.
+
+### Is my database stale? How fresh is the data?
+
+`allelix db status` shows the ingestion date of each cache. The freshness model is signal-driven: `allelix db update` queries an upstream freshness signal (md5 / etag) per source and re-ingests only when the signal changed. There's no time-based staleness — a 6-month-old cache against an unchanged source IS current.
+
+If you want to force a refresh: `allelix db update --force` re-downloads every cache regardless of freshness signal.
+
+### Output reports contain real genome annotations
+
+Anything `allelix analyze` writes to disk contains real annotations of your genome. Treat the output files (HTML, JSON, terminal redirects) as sensitive — they're as private as the input genotype file. Allelix does not transmit them; the lifecycle is local.
+
+### Where does allelix store its data?
+
+See [Managing your data](#managing-your-data) above for cache locations, the disposability rule, what to back up, and the uninstall procedure.
+
 ## Architecture & Design Decisions
 
 The "why" behind major design choices lives in [`docs/adr/`](https://github.com/allelix/allelix/blob/main/docs/adr/README.md) as Architecture Decision Records. Read these before proposing changes that touch the parser/annotator interfaces, the regulatory posture, or the data-handling model.

{allelix-2.1.0 → allelix-2.1.2}/allelix/annotators/__init__.py

@@ -57,18 +57,16 @@ def get_annotators(
     return [clinvar, pharmgkb, gwas, snpedia, gnomad, alphamissense, cadd]
 
 
-_ANNOTATOR_CLASSES: dict[str, type[Annotator]] = {
-    cls.name: cls
-    for cls in [
-        ClinVarAnnotator,
-        PharmGKBAnnotator,
-        GWASCatalogAnnotator,
-        SNPediaAnnotator,
-        GnomadAnnotator,
-        AlphaMissenseAnnotator,
-        CaddAnnotator,
-    ]
-}
+_ANNOTATOR_CLASS_LIST: list[type[Annotator]] = [
+    ClinVarAnnotator,
+    PharmGKBAnnotator,
+    GWASCatalogAnnotator,
+    SNPediaAnnotator,
+    GnomadAnnotator,
+    AlphaMissenseAnnotator,
+    CaddAnnotator,
+]
+_ANNOTATOR_CLASSES: dict[str, type[Annotator]] = {cls.name: cls for cls in _ANNOTATOR_CLASS_LIST}
 
 
 def get_annotator_class(name: str) -> type[Annotator] | None:

{allelix-2.1.0 → allelix-2.1.2}/allelix/annotators/base.py

@@ -4,7 +4,6 @@
 
 from __future__ import annotations
 
-import contextlib
 from abc import ABC, abstractmethod
 from dataclasses import dataclass
 from enum import Enum, auto
@@ -141,34 +140,6 @@ class Annotator(ABC):
         """Bind the annotator to a data directory (created elsewhere)."""
         self.data_dir = data_dir
 
-    def __del__(self) -> None:
-        """Safety-net resource release on GC. Deliberately retained.
-
-        GH #36 (audit second pass) flagged ``__del__`` as a Python
-        antipattern — GC timing is nondeterministic and raised exceptions
-        are silently swallowed. The correct usage pattern is the
-        ``__enter__`` / ``__exit__`` context manager pair below, wired
-        through ``contextlib.ExitStack`` in ``reports/_pipeline.py``.
-
-        However: removing ``__del__`` exposes residual SQLite connection
-        leaks in code paths that construct an annotator outside a
-        context manager. ``ResourceWarning`` is elevated to error by
-        ``pytest`` config, so leaks fail the suite as
-        ``PytestUnraisableExceptionWarning`` — caught in the v2.0.2
-        ship gate when ``__del__`` was first removed. Until every call
-        site is verified to use ``with`` / ``ExitStack`` / explicit
-        ``close()``, this safety net stays. v2.1 task: audit and
-        remove.
-
-        ``contextlib.suppress(Exception)`` is deliberate — ``__del__``
-        must never raise. The GC timing and shutdown-ordering edges
-        are explicitly silenced; this is exactly the
-        "if you must keep ``__del__``, make absolutely sure it can
-        never raise" mitigation the audit recommended.
-        """
-        with contextlib.suppress(Exception):
-            self.close()
-
     def __enter__(self) -> Annotator:
         """Return self for `with` usage."""
         return self

{allelix-2.1.0 → allelix-2.1.2}/allelix/annotators/cadd.py

@@ -24,7 +24,7 @@ from __future__ import annotations
 
 import logging
 import sqlite3
-from typing import TYPE_CHECKING, ClassVar
+from typing import TYPE_CHECKING, Any, ClassVar
 
 from allelix.annotators.base import Annotator, LicenseDescriptor
 from allelix.databases._versions import CADD_SCHEMA_VERSION
@@ -90,9 +90,11 @@ class CaddAnnotator(Annotator):
         self._conn: sqlite3.Connection | None = None
         self._full_mode = full_mode
         self._tabix_path = data_dir / CADD_FULL_FILENAME
-        self._tabix: object | None = None
+        # pysam ships no type stubs (see [tool.mypy.overrides]); the tabix
+        # handle is held as Any so call sites can use `.fetch()` / `.close()`.
+        self._tabix: Any = None
         self._indel_tabix_path = data_dir / CADD_INDEL_FILENAME
-        self._indel_tabix: object | None = None
+        self._indel_tabix: Any = None
 
     def _connection(self) -> sqlite3.Connection:
         if self._conn is None:
@@ -104,11 +106,11 @@ class CaddAnnotator(Annotator):
             self._conn = sqlite3.connect(self._db_path)
         return self._conn
 
-    def _open_tabix(self) -> object:
+    def _open_tabix(self) -> Any:  # noqa: ANN401 — pysam ships no type stubs
         """Open the tabix file for full-mode queries."""
         if self._tabix is None:
             try:
-                import pysam  # type: ignore[import-untyped]
+                import pysam
             except ImportError:
                 raise ImportError(
                     "Full CADD mode requires pysam. Install with: pip install 'allelix[cadd]'"
@@ -122,7 +124,7 @@ class CaddAnnotator(Annotator):
             self._tabix = pysam.TabixFile(str(self._tabix_path))
         return self._tabix
 
-    def _open_indel_tabix(self) -> object | None:
+    def _open_indel_tabix(self) -> Any:  # noqa: ANN401 — pysam ships no type stubs
         """Open the indel tabix file. Returns None if file doesn't exist."""
         if self._indel_tabix is None:
             if not self._indel_tabix_path.exists():
@@ -131,7 +133,7 @@ class CaddAnnotator(Annotator):
             if not tbi.exists():
                 return None
             try:
-                import pysam  # type: ignore[import-untyped]
+                import pysam
             except ImportError:
                 return None
             self._indel_tabix = pysam.TabixFile(str(self._indel_tabix_path))

{allelix-2.1.0 → allelix-2.1.2}/allelix/annotators/clinvar.py

@@ -500,9 +500,10 @@ class ClinVarAnnotator(Annotator):
             for rows in clinvar_rows.values():
                 rows.sort()
             for v in build_variants:
-                rows = clinvar_rows.get((v.chromosome, v.position))
-                if not rows:
+                pos_rows = clinvar_rows.get((v.chromosome, v.position))
+                if not pos_rows:
                     continue
+                rows = pos_rows
                 user_alleles = {v.allele1, v.allele2}
                 # Stamp only on a UNIQUE consistent match. Multiple matches
                 # mean the Variant model can't distinguish them (the 1/1

{allelix-2.1.0 → allelix-2.1.2}/allelix/cli/_helpers.py

@@ -16,7 +16,7 @@ import logging
 import re
 import sys
 import time
-from typing import TYPE_CHECKING, NamedTuple
+from typing import TYPE_CHECKING, NamedTuple, cast
 
 import click
 from rich.console import Console
@@ -34,7 +34,10 @@ from allelix.reports.terminal import render_terminal, render_terminal_diff
 if TYPE_CHECKING:
     from pathlib import Path
 
+    from allelix.annotators.alphamissense import AlphaMissenseAnnotator
     from allelix.annotators.base import Annotator
+    from allelix.annotators.cadd import CaddAnnotator
+    from allelix.annotators.gnomad import GnomadAnnotator
     from allelix.parsers.base import GenotypeParser
 
 console = Console()
@@ -489,9 +492,9 @@ def _run_analysis_command(
             ready,
             skipped_count_provider=lambda: counter.count,
             build_override=build,
-            gnomad=gnomad_annotator,
-            alphamissense=am_annotator,
-            cadd=cadd_annotator,
+            gnomad=cast("GnomadAnnotator | None", gnomad_annotator),
+            alphamissense=cast("AlphaMissenseAnnotator | None", am_annotator),
+            cadd=cast("CaddAnnotator | None", cadd_annotator),
             high_value_rsids=hv_rsids,
         )
     finally: