gapit 0.3.0__tar.gz → 0.4.0__tar.gz

{gapit-0.3.0 → gapit-0.4.0}/AGENTS.md

@@ -1,7 +1,7 @@
 # AGENTS.md — gapit
 
 > Python reimplementation of [abricate](https://github.com/tseemann/abricate): mass screening of
-> contigs for antimicrobial resistance and virulence genes. **Agent-first**: every output is
+> contigs and reads for known genes. **Agent-first**: every output is
 > machine-readable (JSON / Markdown) by design, not as an afterthought.
 
 ## 1. Mission
@@ -70,16 +70,26 @@ gapit/
 │   ├── dbcodec.py       # gapit/v1 tagged-header codec (percent-encoded ids)
 │   ├── records.py       # records.jsonl truth store + gapit.manifest/1 provenance
 │   ├── dbbuild.py       # deterministic native-db build pipeline with self-check
+│   ├── clusterbuild.py  # cluster-db build pipeline: GBK/GFF → locus FASTA + features.json + typing copy
+│   ├── cluster.py       # cluster-screening engine core: minimap2 asm20 → per-locus/per-gene calls
+│   ├── cluster_math.py  # cs-walk → union coverage/identity math + verdicts (pure functions)
 │   ├── db_ops.py        # db use-cases: provider fetch + list (shared CLI + MCP; no typer)
 │   ├── db_query_ops.py  # db use-cases: search + outdated over installed DBs (shared CLI + MCP)
-│   ├── db_build_ops.py  # db use-case: custom FASTA+TSV → native db build (shared CLI + MCP)
+│   ├── db_build_ops.py  # db use-case: custom FASTA+TSV → gene db, GBK/GFF → cluster db (shared CLI + MCP)
+│   ├── gbfeatures.py    # GenBank FEATURES/ORIGIN parser → LocusFeatures/GeneFeature + gapit.features/1
+│   ├── gffparse.py      # GFF3 parser (embedded ##FASTA or sidecar) → the same locus/gene models
+│   ├── typing_models.py # gapit.typing/1 scoring schema + the evaluation-result models
+│   ├── typing_engine.py # gapit.typing/1 evaluation: rule scoring → phenotype call + breakdown
 │   ├── blast.py         # blastn invocation + tabular output parsing
 │   ├── hits.py          # Hit model, identity/coverage computation, filtering, dedup
+│   ├── fragments.py     # opt-in --merge-fragments cross-contig gene-fragment merging
 │   ├── minimap.py       # COVERAGE_MAP construction (exact abricate arithmetic)
 │   ├── minimap2_run.py  # minimap2 invocation layer for reads mode (streaming PAF, --cs/NM tags)
 │   ├── paf.py           # PAF row parsing + interval arithmetic (minimap2 output boundary)
 │   ├── report.py        # Report model: the canonical in-memory result
-│   ├── screening.py     # blastn screen use-case + shared engine helpers (OutputFormat, AlignerEnum)
+│   ├── engines.py       # shared OutputFormat / AlignerEnum (leaf module for the three engines)
+│   ├── screening.py     # blastn screen use-case + the shared run_screen dispatcher
+│   ├── screening_cluster.py # cluster screen use-case: kind guards, typing wiring, rendering
 │   ├── screening_reads.py # minimap2 use-cases: --r1/--r2 reads + --aligner minimap2 assemblies
 │   ├── reads.py         # FASTQ mode: minimap2 PAF parsing, coverage breadth/depth, presence
 │   ├── summary.py       # summary core: parse report tables into a gene matrix
@@ -98,11 +108,15 @@ gapit/
 │   │   ├── tsv.py       # abricate-compatible TSV/CSV
 │   │   ├── json.py      # versioned JSON (gapit.report/1 et al.)
 │   │   ├── md.py        # Markdown (human + agent readable, YAML frontmatter)
+│   │   ├── cluster.py   # gapit.cluster/1 models + JSON/TSV renderers (typed variant)
+│   │   ├── cluster_md.py # gapit.cluster/1 Markdown renderer (Phenotype column when typed)
 │   │   ├── schemas.py   # registered output models behind `gapit schema`
 │   │   └── summary.py   # summary matrix renderers (TSV/CSV/JSON/MD)
-│   ├── providers/       # 12 DB providers + common.py helpers + snapshots.py loader
+│   ├── providers/       # 16 DB providers (12 gene + 4 kaptive cluster) + common.py + snapshots.py
 │   ├── data/snapshots/  # bundled card + vfdb snapshot archives (.tar.gz)
 │   └── py.typed
+├── scripts/
+│   └── cluster_calibration.py # developer calibration harness for typed cluster dbs
 └── tests/
     ├── data/            # tiny synthetic db + contigs + reads (fast, offline)
     ├── golden/          # expected outputs incl. abricate reference TSVs
@@ -142,7 +156,9 @@ This is what distinguishes gapit from abricate. Treat it as a public API.
 - **DB acquisition** `[gapit-extension]`: `gapit db fetch|list|search|outdated|build|install` —
   provider fetch (bundled card/vfdb snapshots install offline; `--from-source` forces upstream),
   provider listing, records.jsonl gene search, staleness report, custom FASTA→native-db build,
-  and SHA256-verified local-file install.
+  GBK/GFF→cluster-db build (`--typing` installs a validated `gapit.typing/1` spec), the four
+  download-on-fetch kaptive cluster providers (GPL content, never bundled), and SHA256-verified
+  local-file install.
 - **MCP** `[gapit-extension]`: `gapit mcp` / `gapit-mcp` stdio server exposing nine tools:
   read-only `screen` (incl. `aligner minimap2` assembly survey), `screen_reads` (FASTQ via
   minimap2), `summary`, `schema`, `db_list`, `db_search`, `db_outdated`, plus the datadir-mutating
@@ -175,7 +191,9 @@ This is what distinguishes gapit from abricate. Treat it as a public API.
   query contigs against one db → 15-field
   tabular hits → filter by identity / coverage thresholds → **dedup hits sharing identical
   `(contig, qstart, qend)`** (first/best BLAST row wins) → one TSV row per surviving hit.
-  abricate does **not** merge overlapping intervals — do not "improve" this on the default path.
+  abricate   does **not** merge overlapping intervals — do not "improve" this on the default path (the
+  opt-in `--merge-fragments` cross-contig extension is the one sanctioned exception:
+  `fragments.py`).
 - Key computed fields: `%COVERAGE = 100*(length-gaps)/slen` (filtered unrounded, displayed
   `%.2f`), `%IDENTITY` (BLAST pident, never post-filtered), `COVERAGE_MAP` (15-char minimap),
   `GAPS`. Exact formulas, the dedup rule, and the minimap arithmetic live in `SPEC.md` — when

{gapit-0.3.0 → gapit-0.4.0}/CHANGELOG.md

@@ -5,6 +5,58 @@ All notable changes to gapit are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
 project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.0] - 2026-09-30
+
+### Added
+
+- **Gene-cluster databases and cluster screening** (stages 1–3; the blastn gene path and
+  all existing outputs are untouched — parity stays byte-identical):
+  - `gapit db build NAME input.gbk|input.gff3` detects the input kind by suffix and builds
+    a `kind: cluster` database — locus FASTA `sequences`, `gapit.features/1` feature table
+    (`features.json`), makeblastdb index, manifest `kind: cluster` — with `--kind
+    gene|cluster` to override (usage error on contradiction). New parsers (no Biopython):
+    GenBank FEATURES/ORIGIN (Bakta/modern and kaptive-style source-note labels) and GFF3
+    (embedded `##FASTA` or sidecar); compound `join()` CDS locations are rejected with a
+    typed error.
+  - `gapit screen --db <cluster db>` dispatches to the minimap2 `asm20` cluster engine:
+    per-gene present/partial/absent verdicts (`--min-gene-cov`/`--min-gene-id`, default
+    90/90), cs-based union coverage/identity, cross-contig fragmentation resilience, and a
+    best-locus call above `--min-cluster-cov` (default 96). Output is the new
+    `gapit.cluster/1` document (json), the cluster TSV/CSV (one row per file), or
+    Markdown; kind-mismatched flags are usage errors; reads mode rejects cluster
+    databases. The MCP `screen` tool gains `minGeneCov`/`minGeneId`/`minClusterCov`.
+  - `--typing FILE` installs a validated `gapit.typing/1` declarative scoring spec into a
+    cluster db (`typing.json`). Screening a typed database annotates every best call with
+    a `phenotype` and an additive `phenotype_detail` breakdown (score, confidence,
+    per-rule components, runner-up, ambiguity pair); the TSV gains a PHENOTYPE column
+    (typed runs only — untyped output stays byte-identical). Three rule kinds —
+    `weighted_genes`, `cluster_match`, `learned_linear` (schema only; no training code) —
+    behind a `cutoff` + `ambiguity_margin` + `fallback` decision layer. Unknown gene/locus
+    references fail with `TYPING_UNKNOWN_GENE` at build AND screen time.
+  - Four kaptive cluster providers, `kaptive_k`, `kaptive_o`, `kaptive_ak`, `kaptive_oc`
+    (Klebsiella K/O and A. baumannii K/OC antigen loci; Wyres et al. 2020 — cite Kaptive).
+    The database content is GPL-3.0, so nothing is bundled: fetch downloads the pinned
+    Kaptive v2.0.9 GenBank files and builds cluster databases; manifests record the
+    source URL, `license`, and citation `note`. No typing model ships with them yet
+    (locus calls only, phenotype null).
+  - `scripts/cluster_calibration.py`: developer harness that screens a typed cluster db
+    against a labels TSV and prints the per-expected-phenotype score distribution, the
+    called×expected agreement matrix, and the divergence list (the v1 loop for future
+    learned_linear training).
+- `gapit screen --merge-fragments` (blastn contig mode only; off by default): merge gene
+  fragments split across contig boundaries into one reported hit when their union subject
+  coverage reaches `--mincov`. Merged rows carry additive optional `merged` and `fragments`
+  fields in `gapit.report/1` JSON plus a Markdown detail line; the default path (and all
+  existing output) is untouched. The MCP `screen` tool gains the matching `mergeFragments`
+  boolean parameter.
+
+## [0.3.1] - 2026-09-28
+
+### Added
+
+- `-h` now works as an alias for `--help` on every command (root, subcommands,
+  and nested `db` sub-app).
+
 ## [0.3.0] - 2026-09-28
 
 ### Removed

{gapit-0.3.0 → gapit-0.4.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.5
 Name: gapit
-Version: 0.3.0
-Summary: Mass screening of contigs for antimicrobial resistance and virulence genes — agent-first Python reimplementation of abricate
+Version: 0.4.0
+Summary: Mass screening of contigs and reads for known genes — agent-first Python reimplementation of abricate
 Project-URL: Homepage, https://github.com/indexofire/gapit
 Project-URL: Documentation, https://indexofire.github.io/gapit/
 Project-URL: Changelog, https://github.com/indexofire/gapit/blob/main/CHANGELOG.md
@@ -10,7 +10,7 @@ Project-URL: Issues, https://github.com/indexofire/gapit/issues
 Author-email: indexofire <indexofire@gmail.com>
 License-Expression: MIT
 License-File: LICENSE
-Keywords: AMR,bioinformatics,blast,genomics,virulence
+Keywords: AMR,abricate,bioinformatics,blast,gene-screening,genomics,virulence
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: MIT License
@@ -30,9 +30,10 @@ Description-Content-Type: text/markdown
 
 # gapit
 
-Mass screening of contigs and reads for antimicrobial resistance and virulence genes. An
-agent-first Python reimplementation of [abricate](https://github.com/tseemann/abricate):
-byte-compatible TSV plus first-class JSON and Markdown.
+Mass screening of contigs and reads for known genes — AMR, virulence, serotype, plasmid
+replicons, species targets, or any custom database. An agent-first Python reimplementation
+of [abricate](https://github.com/tseemann/abricate): byte-compatible TSV plus first-class
+JSON and Markdown.
 
 ## Why gapit
 
@@ -126,6 +127,12 @@ gapit also reads datadirs built by abricate itself (legacy `~~~` headers). The r
 does not hold: gapit-native databases use the `gapit/v1` header format (see SPEC.md §11),
 which abricate cannot read. Protein databases such as `bacmet2` screen through blastx.
 
+Gene **cluster** databases are a second kind: `gapit db build NAME loci.gbk|gff3` builds a
+`kind: cluster` db (locus calls via minimap2, optional `--typing FILE` phenotype scoring),
+and the four `kaptive_k`/`kaptive_o`/`kaptive_ak`/`kaptive_oc` providers fetch the Kaptive
+Klebsiella and A. baumannii antigen-locus references at install time (GPL-3.0 content, so
+nothing is bundled — cite Kaptive/Wyres et al. 2020 for results).
+
 ## Output contract
 
 - **stdout purity.** Data on stdout, diagnostics on stderr, always. `--quiet` silences
@@ -172,7 +179,7 @@ Register it with an MCP client:
 | `pixi run lint` | ruff check |
 | `pixi run fmt` | ruff format |
 | `pixi run typecheck` | basedpyright (strict) |
-| `pixi run test` | pytest, 374 offline tests |
+| `pixi run test` | pytest, 778 offline tests |
 | `pixi run -e parity parity` | byte-diff screening vs real abricate |
 | `pixi run -e parity summary-parity` | byte-diff summary vs real abricate |
 

{gapit-0.3.0 → gapit-0.4.0}/PLAN.md

@@ -146,11 +146,37 @@ consumes abricate-built datadirs.
   releases via `.github/workflows/release.yml` (OIDC trusted publishing, `workflow_dispatch`
   or `v*` tag). **Deferred by decision (2026-09-18, still open):** bioconda submission —
   recipe remains in-repo; sha256 placeholder to be replaced at submission time.
+- **Bioconda preparation (2026-09-29, submission HELD pending rightsholder go):** recipe
+  finalized (real PyPI sdist sha256, license_file, noarch python) and parity corpus
+  strengthened 6 → 15 byte-identical cases (minus-strand hits, coverage/identity threshold
+  boundaries, blastx protein-db leg via bacmet2) — quality gate for the abricate-replacement
+  claim before submitting to bioconda/bioconda-recipes.
 - **Contract change (2026-09-27, rightsholder decision):** `gapit list` and its
   `gapit.list/1` document removed; `gapit db list` (`gapit.dblist/1`) is the single
   listing surface and abricate `--list` byte-parity is intentionally dropped for it.
   Schema registry is now six documents (report, reads, reads2, summary, error, version).
   Breaking; recorded under \[Unreleased\] in CHANGELOG.md for the next minor bump.
+- **Gene-cluster feature, stage 2 (2026-09-30, additive):** `gapit screen` dispatches
+  cluster-kind databases to the minimap2 `asm20` cluster engine (`gapit.cluster/1`
+  document: best-locus call above `--min-cluster-cov` 96, per-gene
+  present/partial/absent verdicts at `--min-gene-cov`/`--min-gene-id` 90/90, cs-based
+  union coverage/identity, cross-contig fragmentation resilience; `phenotype` reserved
+  null for stage 3). Gene-kind screening is byte-identical (parity 15/15 + summary 6/6
+  unchanged); kind-mismatched flags are usage errors; reads mode rejects cluster dbs.
+  Schema registry gains `cluster` (stage 1 added `features`/`typing`). No version bump.
+- **Gene-cluster feature, stage 3 (2026-09-30, additive; feature complete):** the
+  `gapit.typing/1` evaluation engine (`typing_engine.py`: weighted_genes / cluster_match /
+  learned_linear scoring, cutoff + ambiguity-margin decision with an explainable
+  components breakdown) fills `best.phenotype` + additive `best.phenotype_detail` in
+  gapit.cluster/1 on typed databases; TSV gains a PHENOTYPE column as a typed-only
+  renderer variant (untyped output byte-identical; goldens for both rule kinds + the
+  ambiguous two-locus case). Unknown gene/locus references fail TYPING_UNKNOWN_GENE at
+  build and screen time. Four kaptive providers (`kaptive_k`/`_o`/`_ak`/`_oc`,
+  klebgenomics/Kaptive v2.0.9 pinned raw URLs — v3 decentralized its dbs; GPL-3.0
+  content, download-on-fetch, never bundled; manifest gains optional `license`/`note`).
+  `scripts/cluster_calibration.py` developer harness (score distribution, agreement
+  matrix, divergences) seeds the future learned_linear training loop (training itself
+  out of scope). Parity 15/15 + summary 6/6 unchanged; no version bump.
 
 ## Milestones
 

{gapit-0.3.0 → gapit-0.4.0}/README.md

@@ -1,8 +1,9 @@
 # gapit
 
-Mass screening of contigs and reads for antimicrobial resistance and virulence genes. An
-agent-first Python reimplementation of [abricate](https://github.com/tseemann/abricate):
-byte-compatible TSV plus first-class JSON and Markdown.
+Mass screening of contigs and reads for known genes — AMR, virulence, serotype, plasmid
+replicons, species targets, or any custom database. An agent-first Python reimplementation
+of [abricate](https://github.com/tseemann/abricate): byte-compatible TSV plus first-class
+JSON and Markdown.
 
 ## Why gapit
 
@@ -96,6 +97,12 @@ gapit also reads datadirs built by abricate itself (legacy `~~~` headers). The r
 does not hold: gapit-native databases use the `gapit/v1` header format (see SPEC.md §11),
 which abricate cannot read. Protein databases such as `bacmet2` screen through blastx.
 
+Gene **cluster** databases are a second kind: `gapit db build NAME loci.gbk|gff3` builds a
+`kind: cluster` db (locus calls via minimap2, optional `--typing FILE` phenotype scoring),
+and the four `kaptive_k`/`kaptive_o`/`kaptive_ak`/`kaptive_oc` providers fetch the Kaptive
+Klebsiella and A. baumannii antigen-locus references at install time (GPL-3.0 content, so
+nothing is bundled — cite Kaptive/Wyres et al. 2020 for results).
+
 ## Output contract
 
 - **stdout purity.** Data on stdout, diagnostics on stderr, always. `--quiet` silences
@@ -142,7 +149,7 @@ Register it with an MCP client:
 | `pixi run lint` | ruff check |
 | `pixi run fmt` | ruff format |
 | `pixi run typecheck` | basedpyright (strict) |
-| `pixi run test` | pytest, 374 offline tests |
+| `pixi run test` | pytest, 778 offline tests |
 | `pixi run -e parity parity` | byte-diff screening vs real abricate |
 | `pixi run -e parity summary-parity` | byte-diff summary vs real abricate |
 

{gapit-0.3.0 → gapit-0.4.0}/docs/agents.md

@@ -13,7 +13,7 @@ Version, as one JSON line:
 
 ```console
 $ gapit --version --json
-{"schema":"gapit.version/1","name":"gapit","version":"0.3.0"}
+{"schema":"gapit.version/1","name":"gapit","version":"0.4.0"}
 ```
 
 Output schemas. Six documents are introspectable: `report`, `reads`,

{gapit-0.3.0 → gapit-0.4.0}/docs/databases.md

@@ -184,8 +184,10 @@ unknown `--db NAME` is a usage error (exit 2) listing what is installed.
 
 ## Providers
 
-Twelve providers ship with gapit. `card` and `vfdb` also ship as bundled snapshots inside
-the package, so they install with zero network access.
+Sixteen providers ship with gapit. `card` and `vfdb` also ship as bundled snapshots inside
+the package, so they install with zero network access. The four kaptive providers are
+**cluster** databases (downloaded and built at fetch time; GPL content is never bundled —
+see [Kaptive providers](#kaptive-providers-gpl-downloaded-on-fetch)).
 
 | Name | Content | dbtype |
 |---|---|---|
@@ -201,8 +203,30 @@ the package, so they install with zero network access.
 | `bacmet2` | BacMet2 experimentally confirmed biocide/resistance genes (protein) | prot |
 | `victors` | Victors virulence factors | nucl |
 | `upec_expec_vf` | UPEC/ExPEC virulence genes (FordeGenomics) | nucl |
+| `kaptive_k` | Kaptive Klebsiella K antigen loci (cluster; GPL-3.0, downloaded on fetch) | nucl |
+| `kaptive_o` | Kaptive Klebsiella O antigen loci (cluster; GPL-3.0, downloaded on fetch) | nucl |
+| `kaptive_ak` | Kaptive A. baumannii K antigen loci (cluster; GPL-3.0, downloaded on fetch) | nucl |
+| `kaptive_oc` | Kaptive A. baumannii OC antigen loci (cluster; GPL-3.0, downloaded on fetch) | nucl |
 
 Protein databases (`bacmet2`) screen through `blastx`; nucleotide ones through `blastn`.
+Cluster databases screen through the minimap2 cluster engine
+([screen.md](./screen.md#cluster-databases-kind-cluster)).
+
+### Kaptive providers (GPL, downloaded on fetch)
+
+The four `kaptive_*` providers wrap the reference databases of
+[Kaptive](https://github.com/klebgenomics/Kaptive) (Wyres et al., J Clin Microbiol 2020 —
+please cite Kaptive when you use results from these databases). Kaptive v3 moved the
+databases out of its git repository, so gapit pins the archived **v2.0.9 tag** raw URLs
+(stable direct downloads, verified 2026-09-30).
+
+The database content is **GPL-3.0**, while gapit is MIT — so nothing kaptive is ever
+bundled: `gapit db fetch kaptive_k` (and `_o`, `_ak`, `_oc`) downloads the GenBank file at
+fetch time and builds a `kind: cluster` database through the cluster pipeline. The manifest
+records the upstream URL, the `GPL-3.0 (database content)` license, and the citation note.
+No typing model ships with them (phenotype calls stay null); you get kaptive-style best
+locus calls, and you can later install your own `typing.json` semantics by rebuilding with
+`gapit db build --typing`.
 
 ## Fetching databases
 
@@ -345,7 +369,7 @@ versions. A real one, from the plasmidfinder database:
   "upstream_version": "",
   "tool": {
     "name": "gapit",
-    "version": "0.3.0"
+    "version": "0.4.0"
   },
   "makeblastdb_version": "blastn: 2.17.0+",
   "minimap2_version": "2.31-r1302"
@@ -363,6 +387,8 @@ versions. A real one, from the plasmidfinder database:
 | `dbtype` | string | `nucl` or `prot`, chosen explicitly at build time |
 | `header_format` | string | Always `gapit/v1` for gapit-built databases |
 | `upstream_version` | string | Upstream release label when the source has one |
+| `license` | string | Database-content license, present only when the provider declares one (kaptive: `GPL-3.0 (database content)`) |
+| `note` | string | Free-text provenance note (kaptive: source file + citation), present only when set |
 | `tool` | object | `{name, version}` of the gapit that built the database |
 | `makeblastdb_version` | string | BLAST+ version that built the index |
 | `minimap2_version` | string | minimap2 version in the build environment (reads mode indexes in memory; no `.mmi` is built) |
@@ -527,3 +553,79 @@ abricate datadirs take the same in-memory path.
 A complete `db build` walkthrough with worked examples for every header format, protein
 databases, metadata TSVs, and a troubleshooting table lives in
 [Custom databases](./custom-db.md).
+
+## Cluster databases (GBK/GFF)
+
+`gapit db build NAME input.gbk` (or `.gbff`, `.gb`, `.gff`, `.gff3`, each plain or
+`.gz`/`.bz2`) builds a different kind of database: one record = one gene locus. The input
+kind is detected by suffix (`--kind gene|cluster` must agree or the build is a usage
+error); the FASTA gene pipeline of the previous section is untouched.
+
+```console
+$ gapit db build mycps cps_loci.gbk --datadir ./db
+gapit: generated ./db/mycps/sequences
+gapit: self-check passed for mycps
+gapit: BLAST index built (nucl)
+{"db":"mycps","records":2,"dbtype":"nucl","destination":"./db/mycps"}
+```
+
+Parsing (no Biopython): GenBank records become loci (`LOCUS` name = locus id) with their
+`CDS` features as genes; the source-feature notes carry the label and type (kaptive
+`K locus:`/`K type:` style, with Bakta-style fallbacks to the locus id). GFF3 inputs take
+the sequences from an embedded `##FASTA` block or a `<stem>.fa/.fna/.fasta` sidecar.
+Compound `join()` CDS locations are rejected with a typed error — split the record or use
+simple CDS annotations.
+
+The artifacts differ from a gene database: `sequences` holds one bare-locus-id record per
+locus (plain headers — abricate cannot read these), `features.json` is the
+`gapit.features/1` feature table (introspect with `gapit schema features`), the manifest
+declares `kind: cluster`, and there is no `records.jsonl`. Screening dispatches to the
+minimap2 cluster engine ([screen.md](./screen.md#cluster-databases-kind-cluster)).
+
+### typing.json — declarative phenotype scoring
+
+`--typing FILE` installs a validated `gapit.typing/1` document into the database as
+`typing.json`; screening then annotates every best call with a phenotype and an explainable
+score breakdown (the PHENOTYPE TSV column and `phenotype_detail` in gapit.cluster/1 —
+[outputs.md](./outputs.md#gapitcluster1-cluster-database-screening)). A minimal
+`weighted_genes` example:
+
+```json
+{
+  "schema": "gapit.typing/1",
+  "rules": [
+    {
+      "model": "weighted_genes",
+      "phenotype": "K1",
+      "weights": {"wzx": 1.0, "wzy": 3.0},
+      "negative": {"rfaD": -1.0},
+      "identity_floor": 95.0,
+      "require_any": ["wzx", "wzy"]
+    }
+  ],
+  "cutoff": 0.9,
+  "ambiguity_margin": 0.05,
+  "fallback": "unknown"
+}
+```
+
+Three rule kinds exist — `weighted_genes` (gene presence with an identity floor, negative
+markers, and an any-of gate), `cluster_match` (weighted locus coverage/identity/key-genes
+components with per-component floors), and `learned_linear` (a trained sigmoid over named
+features `gene:<id>:<cov|ident|present>` / `cluster:<locus>:<coverage|identity>`) — plus
+the document-level `cutoff`, `ambiguity_margin`, and `fallback` the decision layer applies.
+The full schema prints with `gapit schema typing`. Validation is strict at build time: a
+malformed document fails with `TYPING_MALFORMED`, and a rule referencing a gene or locus
+the input does not carry fails with `TYPING_UNKNOWN_GENE` before any artifact is written
+(the same check reruns at screen time, so hand-edited databases cannot smuggle dead
+references).
+
+Tuning a typing document against labeled assemblies is what
+`scripts/cluster_calibration.py` is for (developer tool): it screens every labeled sample,
+prints the per-expected-phenotype score distribution, the called×expected agreement
+matrix, and the divergence list — the loop a future `learned_linear` training flow will
+consume:
+
+```console
+$ python scripts/cluster_calibration.py mycps --datadir ./db labels.tsv
+```

{gapit-0.3.0 → gapit-0.4.0}/docs/index.md

@@ -1,9 +1,9 @@
 # gapit documentation
 
-gapit screens contig files and FASTQ reads for antimicrobial resistance (AMR) and virulence
-genes. It is an agent-first Python reimplementation of
-[abricate](https://github.com/tseemann/abricate): abricate-compatible TSV on stdout, plus
-first-class JSON and Markdown outputs with versioned schemas.
+gapit screens contig files and FASTQ reads for known genes — AMR, virulence, serotype,
+plasmid replicons, species targets, or any custom database. It is an agent-first Python
+reimplementation of [abricate](https://github.com/tseemann/abricate): abricate-compatible
+TSV on stdout, plus first-class JSON and Markdown outputs with versioned schemas.
 
 ## Pages
 
@@ -14,7 +14,7 @@ first-class JSON and Markdown outputs with versioned schemas.
 | [Screening contigs](./screen.md) | `gapit screen` on FASTA/GBK/EMBL inputs: thresholds, filters, formats |
 | [Screening reads](./reads.md) | FASTQ and assembly FASTA through minimap2: `--r1`/`--r2`, presets, breadth-based presence, two-stage survey |
 | [Summary](./summary.md) | `gapit summary`: gene presence/absence matrix across report tables |
-| [Databases](./databases.md) | Providers, `gapit db fetch/list/install`, datadirs, native `gapit/v1` format |
+| [Databases](./databases.md) | Providers, `gapit db fetch/list/install`, datadirs, native `gapit/v1` format, cluster databases (GBK/GFF, typing.json, kaptive) |
 | [Custom databases](./custom-db.md) | `gapit db build` walkthrough: any FASTA to a screenable database, worked examples |
 | [Outputs](./outputs.md) | TSV/JSON/Markdown formats, schemas, error envelopes, exit codes |
 | [MCP server](./mcp.md) | `gapit mcp`: analysis + database tools for agent runtimes over stdio JSON-RPC |
@@ -25,13 +25,13 @@ first-class JSON and Markdown outputs with versioned schemas.
 
 | Command | What it does | Docs |
 |---|---|---|
-| `gapit screen` | Screen contig files or FASTQ reads for AMR and virulence genes | [Screening](./screen.md) |
+| `gapit screen` | Screen contig files or FASTQ reads for known genes | [Screening](./screen.md) |
 | `gapit summary` | Summarize report table(s) into a gene presence/absence matrix | [Summary](./summary.md) |
-| `gapit db fetch` | Fetch and build provider database(s) into the datadir | [Databases](./databases.md) |
+| `gapit db fetch` | Fetch and build provider database(s) into the datadir (kaptive cluster dbs download on fetch) | [Databases](./databases.md) |
 | `gapit db list` | List database providers and their installed state | [Databases](./databases.md) |
 | `gapit db install` | Install a local file after verifying its SHA256 | [Databases](./databases.md) |
 | `gapit setupdb` | Build BLAST indices for all databases under the datadir | here |
-| `gapit schema` | Print the JSON Schema of a gapit output document (`report`, `reads`, `reads2`, `summary`, `error`, `version`) | [Outputs](./outputs.md) |
+| `gapit schema` | Print the JSON Schema of a gapit output document (`report`, `reads`, `reads2`, `cluster`, `summary`, `error`, `version`, `features`, `typing`) | [Outputs](./outputs.md) |
 | `gapit mcp` | Run the MCP stdio server (also installed as the `gapit-mcp` console script) | [MCP server](./mcp.md) |
 
 ## Project documents

{gapit-0.3.0 → gapit-0.4.0}/docs/installation.md

@@ -44,9 +44,9 @@ gapit --version
 
 ```console
 $ gapit --version
-gapit 0.3.0
+gapit 0.4.0
 $ gapit --version --json
-{"schema":"gapit.version/1","name":"gapit","version":"0.3.0"}
+{"schema":"gapit.version/1","name":"gapit","version":"0.4.0"}
 ```
 
 ## External binaries

{gapit-0.3.0 → gapit-0.4.0}/docs/mcp.md

@@ -37,10 +37,10 @@ transcribed from the live `tools/list` `inputSchema` objects.
 
 | Tool | Arguments | Returns |
 |---|---|---|
-| `screen` | `files` (array of strings, required), `db` (string, default `ncbi`), `minid` (number), `mincov` (number), `format` (string: `json` \| `tsv` \| `md`, default `json`), `aligner` (string: `blastn` \| `minimap2`, default `blastn`), `min_breadth` (number 0–100, default `90`; minimap2 only), `min_identity` (number 0–100, default `0`; minimap2 only), `min_mapq` (integer ≥ 0, default `0`; minimap2 only), `datadir` (string) | Report text: `gapit.report/1` JSON by default, TSV or Markdown per `format`; `aligner minimap2` runs the reads engine and emits `gapit.reads/1` |
+| `screen` | `files` (array of strings, required), `db` (string, default `ncbi`), `minid` (number), `mincov` (number), `format` (string: `json` \| `tsv` \| `md`, default `json`), `aligner` (string: `blastn` \| `minimap2`, default `blastn`), `mergeFragments` (boolean, default `false`; blastn only — cross-contig fragment merging), `min_breadth` (number 0–100, default `90`; minimap2 only), `min_identity` (number 0–100, default `0`; minimap2 only), `min_mapq` (integer ≥ 0, default `0`; minimap2 only), `minGeneCov` (number 0–100, default `90`; cluster dbs only), `minGeneId` (number 0–100, default `90`; cluster dbs only), `minClusterCov` (number 0–100, default `96`; cluster dbs only), `datadir` (string) | Report text: `gapit.report/1` JSON by default, TSV or Markdown per `format`; `aligner minimap2` runs the reads engine and emits `gapit.reads/1`. A cluster-kind `db` emits `gapit.cluster/1`: best-locus call, per-gene verdicts, and — when the database carries a `typing.json` — the phenotype call with its `phenotype_detail` score breakdown |
 | `screen_reads` | `r1` (array of strings, required — one path per lane), `r2` (array of strings, same count as `r1`), `read_type` (string: `sr` \| `map-ont` \| `map-hifi`, default `sr`), `min_breadth` (number 0–100, default `90`), `min_identity` (number 0–100, default `0`), `min_mapq` (integer ≥ 0, default `0`), `format` (string: `json` \| `md`, default `json`), `db` (string, default `ncbi`), `datadir` (string) | `gapit.reads/1` JSON by default or Markdown per `format`; `min_identity`/`min_mapq` > 0 switches to `gapit.reads/2` |
 | `summary` | `files` (array of report table paths, required), `identity` (boolean), `nopath` (boolean) | `gapit.summary/1` JSON |
-| `schema` | `name` (string, required, one of `error`, `reads`, `reads2`, `report`, `summary`, `version`) | The JSON Schema of that output document |
+| `schema` | `name` (string, required, one of `cluster`, `error`, `features`, `reads`, `reads2`, `report`, `summary`, `typing`, `version`) | The JSON Schema of that output document |
 | `db_list` | none | `gapit.dblist/1`: provider names, install state, record counts |
 | `db_fetch` | `name` (string), `datadir` (string), `force` (boolean, default `false`) | One JSON receipt line per database (`db`, `records`, `dbtype`, `destination`). Name omitted: the card+vfdb default set from bundled snapshots. Network installs can take minutes |
 | `db_build` | `name` (string, required), `fasta` (string, required — a LOCAL filesystem path), `tsv` (string), `dbtype` (string: `nucl` \| `prot`), `description` (string), `datadir` (string), `force` (boolean, default `false`) | One JSON receipt line (`db`, `records`, `dbtype`, `destination`) |
@@ -94,14 +94,14 @@ EOF
 Response line 1, verbatim:
 
 ```text
-{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-06-18","capabilities":{"tools":{}},"serverInfo":{"name":"gapit","version":"0.3.0"}}}
+{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-06-18","capabilities":{"tools":{}},"serverInfo":{"name":"gapit","version":"0.4.0"}}}
 ```
 
 Response line 2 (real output, elided in the middle; each tool carries its full
 `inputSchema`):
 
 ```text
-{"jsonrpc":"2.0","id":2,"result":{"tools":[{"name":"screen","description":"Screen contig files for AMR/virulence genes (json = gapit.report/1; aligner minimap2 = fast assembly survey emitting gapit.reads/1).","inputSchema":{"type":"object","properties":{"files":{"type":"array","items":{"type":"string"}},"db":{"type":"string","default":"ncbi"}, ... },"required":["files"]}}, {"name":"screen_reads","description":"Screen FASTQ reads for genes via minimap2 (json = gapit.reads/1; min_identity/min_mapq > 0 emits gapit.reads/2). Returns the rendered document.","inputSchema":{"type":"object","properties":{"r1":{"type":"array","items":{"type":"string"}},"r2":{"type":"array","items":{"type":"string"}},"read_type":{"type":"string","enum":["sr","map-ont","map-hifi"],"default":"sr"}, ... },"required":["r1"]}}, {"name":"summary", ...}, {"name":"schema", ...}, {"name":"db_list", ...}, {"name":"db_fetch", ...}, {"name":"db_build", ...}, {"name":"db_search", ...}, {"name":"db_outdated", ...}]}}
+{"jsonrpc":"2.0","id":2,"result":{"tools":[{"name":"screen","description":"Screen contig files for known genes (json = gapit.report/1; aligner minimap2 = fast assembly survey emitting gapit.reads/1).","inputSchema":{"type":"object","properties":{"files":{"type":"array","items":{"type":"string"}},"db":{"type":"string","default":"ncbi"}, ... },"required":["files"]}}, {"name":"screen_reads","description":"Screen FASTQ reads for genes via minimap2 (json = gapit.reads/1; min_identity/min_mapq > 0 emits gapit.reads/2). Returns the rendered document.","inputSchema":{"type":"object","properties":{"r1":{"type":"array","items":{"type":"string"}},"r2":{"type":"array","items":{"type":"string"}},"read_type":{"type":"string","enum":["sr","map-ont","map-hifi"],"default":"sr"}, ... },"required":["r1"]}}, {"name":"summary", ...}, {"name":"schema", ...}, {"name":"db_list", ...}, {"name":"db_fetch", ...}, {"name":"db_build", ...}, {"name":"db_search", ...}, {"name":"db_outdated", ...}]}}
 ```
 
 Response line 3 (real output, text content elided). The screen result rides in
@@ -117,7 +117,7 @@ the same run):
 ```json
 {
   "schema": "gapit.report/1",
-  "tool": {"name": "gapit", "version": "0.3.0"},
+  "tool": {"name": "gapit", "version": "0.4.0"},
   "created_at": "2026-09-19T01:15:44Z",
   "params": {"db": "tinyamr", "minid": 80.0, "mincov": 80.0, "threads": 1},
   "files": [
@@ -173,7 +173,7 @@ the same run):
 ```json
 {
   "schema": "gapit.reads/1",
-  "tool": {"name": "gapit", "version": "0.3.0"},
+  "tool": {"name": "gapit", "version": "0.4.0"},
   "created_at": "2026-09-22T00:07:35Z",
   "params": {
     "db": "tinyreads",
@@ -248,7 +248,7 @@ hit is the gene the agent just built the database from (real output):
 ```json
 {
   "schema": "gapit.report/1",
-  "tool": {"name": "gapit", "version": "0.3.0"},
+  "tool": {"name": "gapit", "version": "0.4.0"},
   "created_at": "2026-09-20T13:38:39Z",
   "params": {"db": "myamr", "minid": 80.0, "mincov": 80.0, "threads": 1},
   "files": [