merge-cli 3.7__tar.gz → 3.9.0__tar.gz

merge_cli-3.9.0/MANIFEST.in

@@ -0,0 +1,6 @@
+recursive-include merge_cli/data/models *.pkl *.py *.md .gitkeep
+# README is included automatically as the long_description; TUTORIAL is not, and
+# README links to it — a source distribution without it has a dead link.
+include TUTORIAL.md
+include MODEL_CARD.md
+recursive-include tests *.py

merge_cli-3.9.0/PKG-INFO

@@ -0,0 +1,232 @@
+Metadata-Version: 2.4
+Name: merge-cli
+Version: 3.9.0
+Summary: MERGE variant pathogenicity prediction CLI: 4 variant types, 14 model features, local/remote modes, MCP server
+Author: MERGE Team
+License-Expression: LicenseRef-Proprietary
+Project-URL: Homepage, https://merge.fanglab.cn
+Keywords: bioinformatics,variant,pathogenicity,genomics,cli
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Environment :: Console
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: click>=8.1
+Requires-Dist: rich>=13.0
+Requires-Dist: requests>=2.31
+Requires-Dist: numpy>=1.26
+Requires-Dist: pandas>=2.1
+Requires-Dist: scikit-learn>=1.8
+Requires-Dist: joblib>=1.3
+Requires-Dist: matplotlib>=3.8
+Requires-Dist: shap>=0.44
+Requires-Dist: catboost>=1.2
+Provides-Extra: mcp
+Requires-Dist: mcp<2,>=1.2; extra == "mcp"
+Provides-Extra: local
+Requires-Dist: pysam>=0.22; extra == "local"
+Provides-Extra: all
+Requires-Dist: merge-cli[local,mcp]; extra == "all"
+
+# merge-cli 3.8
+
+MERGE variant pathogenicity prediction CLI — the professional-user counterpart to
+[merge.fanglab.cn](https://merge.fanglab.cn), with the same models and the same
+ensemble, runnable entirely on your own machine.
+
+**New here? Read [TUTORIAL.md](TUTORIAL.md)** — install, both modes, per-model
+setup, scripting and troubleshooting.
+
+## Highlights
+
+- **Four variant types**: coding, splice, noncoding and **non-SNV** (indel / MNV / delins).
+- **13 models**: AlphaGenome, HyenaDNA, NT, AlphaMissense, ESM-1b, GPN-MSA,
+  Evo2-7B, Evo2-7B-base, Enformer, GENERATOR, GENERATOR-v2, NT-v2, Carbon-3B.
+  (Evo2-1B-base is also supported locally where the GPU allows it.)
+- **Remote and local modes.** Local mode deploys each model into its own conda
+  environment and serves it over HTTP — no Docker, no gateway required.
+- Local precomputed VCF cache lookup for hg38/hg19 coding and splicing SNVs.
+- Bundled MERGE ensemble models (the same four deployment bundles the website runs).
+
+## Quick Start
+
+```bash
+pip install merge-cli
+merge --help
+merge predict --chrom chr1 --pos 69428 --ref T --alt G --genome hg38
+```
+
+Optional extras: `merge-cli[mcp]` to ask for predictions in chat (see below),
+`merge-cli[local]` for local mode (adds pysam, which has no Windows wheels —
+which is why it is not in the base install), or `merge-cli[all]` for both.
+
+## Ask in chat (MCP)
+
+```bash
+pip install "merge-cli[mcp]"
+merge mcp install       # detects Claude Desktop / Claude Code, writes the config
+merge mcp status
+```
+
+Then restart your client and ask *"Is chr1:1040819 G>GC pathogenic?"*. The MCP
+server runs in remote mode only — no GPU needed. Details in
+[TUTORIAL.md §11](TUTORIAL.md).
+
+## Variant types and ensemble models
+
+The task is chosen automatically: a variant whose REF or ALT is not a single base
+is scored as non-SNV; otherwise ANNOVAR decides coding / splice / noncoding.
+Override with `--ensemble-type coding|splice|noncoding|nonsnv`.
+
+| Task | Bundle | Algorithm | Features |
+|---|---|---|---|
+| coding | `MERGE_coding.pkl` | LR_L1 | 6 |
+| splice | `MERGE_splicing.pkl` | CatBoost | 10 |
+| noncoding | `MERGE_noncoding.pkl` | LR_L2 | 10 |
+| non-SNV | `MERGE_nonsnv.pkl` | CatBoost | 10 |
+
+Each bundle embeds its own preprocessing (median imputation → z-scoring →
+classifier), so raw model scores are fed straight in and any model you did not
+run is median-imputed. See `merge_cli/data/models/MODEL_CARD.md` for the exact
+feature panels.
+
+## Models
+
+All models run by default — each one feeds at least one ensemble panel, and a
+model you skip has its feature median-imputed instead. Skip any with `--no-<model>`:
+
+```bash
+merge predict --chrom chr1 --pos 69428 --ref T --alt G --no-carbon --no-nt-v2
+```
+
+Available switches: `--no-alphagenome --no-hyenadna --no-nt --no-alphamissense
+--no-esm1b --no-gpn-msa --no-evo2 --no-enformer --no-generator
+--no-generator-v2 --no-nt-v2 --no-carbon`.
+
+In local mode every model needs its service running; the CLI reports which
+features were computed and which were imputed on every prediction.
+
+### Evo2 variants
+
+`--evo2-model` picks the weights for the primary Evo2 score (remote and local):
+
+```bash
+merge predict --chrom chr1 --pos 69428 --ref T --alt G --evo2-model evo2_7b_base
+```
+
+All three Evo2 variants are scored by default in local mode —
+`evo2_7b_score`, `evo2_7b_base_score` and `evo2_1b_base_score` are separate
+ensemble features. They share one GPU-resident service and are scored one at a
+time, costing a few seconds each. Narrow the set to trade coverage for speed:
+
+```bash
+merge local predict --chrom chr1 --pos 69428 --ref T --alt G \
+    --evo2-models evo2_7b        # primary variant only, faster
+```
+
+`evo2_1b_base` needs a GPU with FP8 (compute capability ≥ 8.9: RTX 4000/5000 Ada,
+L40S, H100). Elsewhere — including in remote mode — its feature is left missing and
+median-imputed by the ensemble, which is what the website does too.
+
+## Local mode
+
+```bash
+merge doctor                                     # check GPU and services
+merge local setup                                # configure paths
+merge local env setup --model evo2               # deploy one model
+merge local env setup --model carbon             # ... or another
+merge local env setup --model all                # everything (large!)
+merge local env start --model evo2
+merge local env status
+merge local predict --chrom chr17 --pos 43092919 --ref A --alt G
+```
+
+Setting a reference FASTA is strongly recommended — the local services use it to
+cut the exact sequence windows the models were scored with during training:
+
+```bash
+merge local setup      # answer the "Reference genome FASTA path" prompt
+# or
+merge local predict ... --genome-ref /path/to/hg38.fa
+```
+
+Without it the services fall back to fetching sequence context from the MERGE
+server, which requires network access.
+
+### Service ports
+
+| Model | Env | Port |
+|---|---|---|
+| AlphaGenome | `alphagenome` | 5000 |
+| HyenaDNA | `hyenadna` | 5001 |
+| NT | `nt` | 5002 |
+| Evo2 (all variants) | `evo2` | 5003 |
+| Enformer | `enformer` | 5004 |
+| Carbon | `carbon` | 5005 |
+| GENERATOR | `generator` | 5006 |
+| GENERATOR-v2 | `generator_v2` | 5007 |
+| NT-v2 | `nt_v2` | 5008 |
+
+Each service exposes `GET /health` and `POST /predict` (`{chrom, pos, ref, alt,
+genome_version}`); the DNA-LM services also expose `POST /batch_predict`.
+
+Each model gets its own conda env, service and port by default, and there is no
+gateway to stand up — so you can deploy only the models you actually need.
+
+Environments may also be **shared**: any environment with the right dependencies
+(torch + `transformers<5` + pyfaidx) can serve several of these models, which
+saves tens of GB. Start the service with that environment's interpreter, e.g.
+`~/miniconda3/envs/NT/bin/python ~/.merge-local-servers/generator_server.py`.
+`merge local env status` shows which environment is really behind each port.
+
+## Precomputed VCF Cache
+
+```bash
+merge precomputed configure --data-dir /path/to/precomputed
+merge precomputed status
+merge precomputed download --genome all --variant-type all
+```
+
+Expected files: `coding_merged.vcf.gz`, `splicing_merged.vcf.gz`, their hg19
+counterparts, and `.tbi` indices. The cache only covers SNVs and the older
+feature set, so it is bypassed for non-SNVs and whenever an opt-in DNA-LM is
+requested.
+
+## Scoring conventions
+
+The local services reproduce the exact quantities the ensemble was trained on.
+Changing any of them silently invalidates the MERGE score:
+
+| Model | Score | Window | VRAM |
+|---|---|---|---|
+| Evo2-7B / -7B-base | mean_PLL(REF) − mean_PLL(ALT) | 8192 bp | ~20 GB |
+| Evo2-1B-base | mean_PLL(REF) − mean_PLL(ALT) | 8192 bp | ~8 GB, needs FP8 |
+| Carbon-3B | logP_sum(REF) − logP_sum(ALT) | 24576 bp | **~20 GB** |
+| GENERATOR / -v2 | NLL(ALT) − NLL(REF) | 600 bp, max_length 128 | ~6 GB |
+| NT-v2 | ‖emb(ALT) − emb(REF)‖₂ | 600 bp, max_length 128 | ~3 GB |
+| NT | ‖emb(ALT) − emb(REF)‖₂ | 8192 bp | ~3 GB |
+| HyenaDNA | log p(ALT) − log p(REF), next token | 1000 bp upstream | ~2 GB |
+
+These windows are not tunable, on purpose. The MERGE bundles were fitted on
+features computed exactly this way, so a shorter window does not merely cost
+accuracy — it puts the feature on a different scale than the model expects, and
+the resulting MERGE score is wrong without anything looking wrong. If a model
+does not fit on your GPU, leave it off: the ensemble median-imputes what is
+missing, which is the honest fallback.
+
+Carbon in particular needs roughly 20 GB of VRAM at the 24576 bp training window.
+
+The `transformers` version is pinned to 4.x in the generated environments:
+NT-v2's remote code does not load under transformers 5.x. `alphagenome` is
+pinned to 0.5.1 to match the MERGE server, so both modes return the same
+AlphaGenome features.
+
+AlphaGenome is the one model whose features are **not** reproducible over time:
+it runs on Google's servers and is updated there, so today's values differ from
+those in the training tables. Every other model reproduces bit-for-bit.
+
+If local services are unavailable, the errors are reported under
+`prediction.errors` and the CLI continues with the models it could reach.

merge_cli-3.9.0/README.md

@@ -0,0 +1,199 @@
+# merge-cli 3.8
+
+MERGE variant pathogenicity prediction CLI — the professional-user counterpart to
+[merge.fanglab.cn](https://merge.fanglab.cn), with the same models and the same
+ensemble, runnable entirely on your own machine.
+
+**New here? Read [TUTORIAL.md](TUTORIAL.md)** — install, both modes, per-model
+setup, scripting and troubleshooting.
+
+## Highlights
+
+- **Four variant types**: coding, splice, noncoding and **non-SNV** (indel / MNV / delins).
+- **13 models**: AlphaGenome, HyenaDNA, NT, AlphaMissense, ESM-1b, GPN-MSA,
+  Evo2-7B, Evo2-7B-base, Enformer, GENERATOR, GENERATOR-v2, NT-v2, Carbon-3B.
+  (Evo2-1B-base is also supported locally where the GPU allows it.)
+- **Remote and local modes.** Local mode deploys each model into its own conda
+  environment and serves it over HTTP — no Docker, no gateway required.
+- Local precomputed VCF cache lookup for hg38/hg19 coding and splicing SNVs.
+- Bundled MERGE ensemble models (the same four deployment bundles the website runs).
+
+## Quick Start
+
+```bash
+pip install merge-cli
+merge --help
+merge predict --chrom chr1 --pos 69428 --ref T --alt G --genome hg38
+```
+
+Optional extras: `merge-cli[mcp]` to ask for predictions in chat (see below),
+`merge-cli[local]` for local mode (adds pysam, which has no Windows wheels —
+which is why it is not in the base install), or `merge-cli[all]` for both.
+
+## Ask in chat (MCP)
+
+```bash
+pip install "merge-cli[mcp]"
+merge mcp install       # detects Claude Desktop / Claude Code, writes the config
+merge mcp status
+```
+
+Then restart your client and ask *"Is chr1:1040819 G>GC pathogenic?"*. The MCP
+server runs in remote mode only — no GPU needed. Details in
+[TUTORIAL.md §11](TUTORIAL.md).
+
+## Variant types and ensemble models
+
+The task is chosen automatically: a variant whose REF or ALT is not a single base
+is scored as non-SNV; otherwise ANNOVAR decides coding / splice / noncoding.
+Override with `--ensemble-type coding|splice|noncoding|nonsnv`.
+
+| Task | Bundle | Algorithm | Features |
+|---|---|---|---|
+| coding | `MERGE_coding.pkl` | LR_L1 | 6 |
+| splice | `MERGE_splicing.pkl` | CatBoost | 10 |
+| noncoding | `MERGE_noncoding.pkl` | LR_L2 | 10 |
+| non-SNV | `MERGE_nonsnv.pkl` | CatBoost | 10 |
+
+Each bundle embeds its own preprocessing (median imputation → z-scoring →
+classifier), so raw model scores are fed straight in and any model you did not
+run is median-imputed. See `merge_cli/data/models/MODEL_CARD.md` for the exact
+feature panels.
+
+## Models
+
+All models run by default — each one feeds at least one ensemble panel, and a
+model you skip has its feature median-imputed instead. Skip any with `--no-<model>`:
+
+```bash
+merge predict --chrom chr1 --pos 69428 --ref T --alt G --no-carbon --no-nt-v2
+```
+
+Available switches: `--no-alphagenome --no-hyenadna --no-nt --no-alphamissense
+--no-esm1b --no-gpn-msa --no-evo2 --no-enformer --no-generator
+--no-generator-v2 --no-nt-v2 --no-carbon`.
+
+In local mode every model needs its service running; the CLI reports which
+features were computed and which were imputed on every prediction.
+
+### Evo2 variants
+
+`--evo2-model` picks the weights for the primary Evo2 score (remote and local):
+
+```bash
+merge predict --chrom chr1 --pos 69428 --ref T --alt G --evo2-model evo2_7b_base
+```
+
+All three Evo2 variants are scored by default in local mode —
+`evo2_7b_score`, `evo2_7b_base_score` and `evo2_1b_base_score` are separate
+ensemble features. They share one GPU-resident service and are scored one at a
+time, costing a few seconds each. Narrow the set to trade coverage for speed:
+
+```bash
+merge local predict --chrom chr1 --pos 69428 --ref T --alt G \
+    --evo2-models evo2_7b        # primary variant only, faster
+```
+
+`evo2_1b_base` needs a GPU with FP8 (compute capability ≥ 8.9: RTX 4000/5000 Ada,
+L40S, H100). Elsewhere — including in remote mode — its feature is left missing and
+median-imputed by the ensemble, which is what the website does too.
+
+## Local mode
+
+```bash
+merge doctor                                     # check GPU and services
+merge local setup                                # configure paths
+merge local env setup --model evo2               # deploy one model
+merge local env setup --model carbon             # ... or another
+merge local env setup --model all                # everything (large!)
+merge local env start --model evo2
+merge local env status
+merge local predict --chrom chr17 --pos 43092919 --ref A --alt G
+```
+
+Setting a reference FASTA is strongly recommended — the local services use it to
+cut the exact sequence windows the models were scored with during training:
+
+```bash
+merge local setup      # answer the "Reference genome FASTA path" prompt
+# or
+merge local predict ... --genome-ref /path/to/hg38.fa
+```
+
+Without it the services fall back to fetching sequence context from the MERGE
+server, which requires network access.
+
+### Service ports
+
+| Model | Env | Port |
+|---|---|---|
+| AlphaGenome | `alphagenome` | 5000 |
+| HyenaDNA | `hyenadna` | 5001 |
+| NT | `nt` | 5002 |
+| Evo2 (all variants) | `evo2` | 5003 |
+| Enformer | `enformer` | 5004 |
+| Carbon | `carbon` | 5005 |
+| GENERATOR | `generator` | 5006 |
+| GENERATOR-v2 | `generator_v2` | 5007 |
+| NT-v2 | `nt_v2` | 5008 |
+
+Each service exposes `GET /health` and `POST /predict` (`{chrom, pos, ref, alt,
+genome_version}`); the DNA-LM services also expose `POST /batch_predict`.
+
+Each model gets its own conda env, service and port by default, and there is no
+gateway to stand up — so you can deploy only the models you actually need.
+
+Environments may also be **shared**: any environment with the right dependencies
+(torch + `transformers<5` + pyfaidx) can serve several of these models, which
+saves tens of GB. Start the service with that environment's interpreter, e.g.
+`~/miniconda3/envs/NT/bin/python ~/.merge-local-servers/generator_server.py`.
+`merge local env status` shows which environment is really behind each port.
+
+## Precomputed VCF Cache
+
+```bash
+merge precomputed configure --data-dir /path/to/precomputed
+merge precomputed status
+merge precomputed download --genome all --variant-type all
+```
+
+Expected files: `coding_merged.vcf.gz`, `splicing_merged.vcf.gz`, their hg19
+counterparts, and `.tbi` indices. The cache only covers SNVs and the older
+feature set, so it is bypassed for non-SNVs and whenever an opt-in DNA-LM is
+requested.
+
+## Scoring conventions
+
+The local services reproduce the exact quantities the ensemble was trained on.
+Changing any of them silently invalidates the MERGE score:
+
+| Model | Score | Window | VRAM |
+|---|---|---|---|
+| Evo2-7B / -7B-base | mean_PLL(REF) − mean_PLL(ALT) | 8192 bp | ~20 GB |
+| Evo2-1B-base | mean_PLL(REF) − mean_PLL(ALT) | 8192 bp | ~8 GB, needs FP8 |
+| Carbon-3B | logP_sum(REF) − logP_sum(ALT) | 24576 bp | **~20 GB** |
+| GENERATOR / -v2 | NLL(ALT) − NLL(REF) | 600 bp, max_length 128 | ~6 GB |
+| NT-v2 | ‖emb(ALT) − emb(REF)‖₂ | 600 bp, max_length 128 | ~3 GB |
+| NT | ‖emb(ALT) − emb(REF)‖₂ | 8192 bp | ~3 GB |
+| HyenaDNA | log p(ALT) − log p(REF), next token | 1000 bp upstream | ~2 GB |
+
+These windows are not tunable, on purpose. The MERGE bundles were fitted on
+features computed exactly this way, so a shorter window does not merely cost
+accuracy — it puts the feature on a different scale than the model expects, and
+the resulting MERGE score is wrong without anything looking wrong. If a model
+does not fit on your GPU, leave it off: the ensemble median-imputes what is
+missing, which is the honest fallback.
+
+Carbon in particular needs roughly 20 GB of VRAM at the 24576 bp training window.
+
+The `transformers` version is pinned to 4.x in the generated environments:
+NT-v2's remote code does not load under transformers 5.x. `alphagenome` is
+pinned to 0.5.1 to match the MERGE server, so both modes return the same
+AlphaGenome features.
+
+AlphaGenome is the one model whose features are **not** reproducible over time:
+it runs on Google's servers and is updated there, so today's values differ from
+those in the training tables. Every other model reproduces bit-for-bit.
+
+If local services are unavailable, the errors are reported under
+`prediction.errors` and the CLI continues with the models it could reach.