seqtree 0.6.1__tar.gz → 1.0.0__tar.gz

{seqtree-0.6.1 → seqtree-1.0.0}/CHANGELOG.md

@@ -1,7 +1,261 @@
 # Changelog
 
-All notable changes to `seqtree`. Dates are release dates; the project is pre-1.0, so a **minor**
-bump may carry breaking changes.
+All notable changes to `seqtree`. Dates are release dates. From 1.0.0 the project follows semantic
+versioning: breaking changes need a **major** bump.
+
+## [1.0.0] — 2026-09-06
+
+### Added
+
+- **`TextIndex` — exact k-mismatch search over a concatenated reference text, with one index
+  for every query length.** `Index` builds a trie over reference *strings*, so asking a text
+  question with it means enumerating every length-`L` window of the text as its own string —
+  and again for every distinct query length. The human proteome has **68,389,335** nine-mer
+  windows, so one such index costs several gigabytes; a query set spanning **45 distinct
+  lengths** costs 45 of them. That is what ran **> 2 h 10 m without finishing** and filled
+  **225 GB** of index cache downstream. `TextIndex` keys on a k-mer seed table over the flat
+  text, so **`k` belongs to the index, not to the query**: one build answers every length and
+  every `max_subs`.
+
+  ```python
+  ix  = TextIndex.build(records, alphabet="aa", k=4, group_ids=gene_id_per_record)
+  res = ix.search_batch(peptides, max_subs=2, best_only=True, group_by=True, threads=0)
+  ```
+
+  **One search scheme**, not a special case per query shape. Split the query into
+  `b = min(max_subs + 1, L // k)` disjoint blocks of width `L // b >= k`, and probe block `j`'s
+  leading k-mer at radius `c_j` — every k-mer within `c_j` substitutions of it. That is
+  **lossless exactly when `sum(c_j) >= max_subs - b + 1`**: the lightest error vector that could
+  evade every block is `e_j = c_j + 1`, of weight `sum(c_j) + b`, so nothing within `max_subs`
+  can hide below that threshold. Probing at radius `c` costs `N(c) = sum(C(k,i)(A-1)^i, i <= c)`,
+  whose increments climb steeply and are the same for every block, so the cheapest legal scheme
+  takes as many blocks as fit and spreads `r = max(0, max_subs - b + 1)` budget units evenly.
+  `b = max_subs + 1` recovers plain pigeonhole (one exact seed per error) and `b = 1` a single
+  substitution ball over `q[0:k]`; **the useful schemes are in between**, which is where short
+  queries live: at `L = 8, max_subs = 2, k = 4` two disjoint 4-mers fit, so `c = (0, 1)` probes
+  **94** variants where a one-block ball probes **3,267**.
+
+  Two details are not incidental. Spare budget units go on the **last** blocks, worth a further
+  **1.4x**: a candidate already matches its own block's k-mer, so left-to-right verification
+  meets the unconstrained prefix first and rejects after a residue or two — and the high-budget
+  block contributes nearly all the candidates. And a start that two blocks both reach is
+  returned **once**, emitted by the lowest-indexed block that could have produced it, a test read
+  straight off the mismatch positions verification has already computed — so deduplication needs
+  no sort and no hash set, which also removed the candidate sort from the all-exact path.
+
+  The framing is the search scheme of Kianfar, Pockrandt, Torkamandi, Luo & Reinert,
+  *Optimum Search Schemes for Approximate String Matching Using Bidirectional FM-Index*,
+  [arXiv:1711.02035](https://doi.org/10.48550/arXiv.1711.02035). Only the partition-and-budget
+  half transfers: a direct-addressed seed table has no bidirectional extension, so there is no
+  search *order* to optimise over.
+
+  **None of this is a heuristic**, and that is checked rather than asserted: `tests/cpp/
+  test_text_index.cpp` compares against an independent brute-force scan over query length 6–30
+  × `max_subs` 0–3 × `k` ∈ {3, 4, 5}, requiring **set equality** of `(ref_id, offset, n_subs)`
+  *and* of the mismatch detail — not merely that hits were found — plus a case straddling every
+  block boundary, and a separate case asserting the answer is *identical* across `k`, so a
+  dispatch bug cannot pass by agreeing with itself. `bench/bench_text_index.py` repeats the
+  `k`-invariance check on the real proteomes, where hit counts match cell for cell. A missed hit
+  does not degrade the caller's answer, it changes it, from ambiguous to confidently wrong.
+
+  Measured on the human proteome (UP000005640, 147,506 records / **69,578,135 residues**), one
+  thread, Apple M-series, `bench/bench_text_index.py`. The last column is the two-path dispatch
+  this scheme replaced, on the same benchmark:
+
+  | `L` | `max_subs` | `k = 4` | `k = 5` | two-path dispatch |
+  |--:|--:|--:|--:|--:|
+  | 8 | 2 | **1.468** | 4.694 | 34.505 |
+  | 9 | 2 | **1.334** | 4.526 | 35.453 |
+  | 10 | 2 | 1.298 | **0.129** | 40.940 |
+  | 11 | 2 | 1.138 | **0.108** | 30.196 |
+  | 12 | 2 | **0.097** | 0.126 | 0.148 |
+  | 10 | 3 | 3.595 | **0.347** | 423.762 |
+  | 12 | 3 | 1.567 | **0.394** | 415.578 |
+  | 15 | 3 | 1.514 | **0.138** | 400.484 |
+
+  ms/query, single-threaded, and hit counts identical across `k` in every cell — the benchmark
+  asserts that rather than reporting it. So **24–32x at `L = 8–11, max_subs = 2`, and 87–1,221x
+  at `max_subs = 3`**. Against the design document's acceptance targets of <= 0.5 ms/query at
+  `L >= 12` and <= 1.0 at `L = 8–11`: with `k` chosen by the `L = 2k` crossover, **every target
+  from length 10 up is met with an order of magnitude of margin**, and lengths 8–9 land at
+  1.33–1.47 ms — 1.4x over, down from 35x over. Build is 0.55 s at `k = 4` and 0.59 s at
+  `k = 5`, so a corpus spanning both regimes can hold both indexes.
+
+  Threads, human proteome, `L = 12`, `max_subs = 2`, 50,000 queries: 0.103 ms/query on one,
+  0.013 on eight (**7.8x**), 0.008 on all cores (**12.6x**). The ratio is now bounded by the
+  serial CSR flatten rather than by the search, which is why it is short of linear; the flatten
+  preallocates from the workers' actual output instead of growing nine vectors by doubling.
+
+  Results come back as flat CSR arrays rather than one object per hit — a 445 k-query run
+  returns a handful of arrays. `res[i]` materialises `TextHit` objects for one query on demand,
+  `res.arrays()` gives zero-copy buffer views, and `res.to_numpy()` gives numpy views, with
+  numpy imported **on the call** so seqtree keeps its empty dependency list. Order is
+  `(n_subs, ref_id, offset)`, stable across runs and across thread counts.
+
+  Beyond a position: each mismatch is the **pair** `(pos, query_aa, text_aa)`, so a caller
+  ranking by chemistry can tell `L→I` from `L→D` without re-fetching the window. `matrix=`
+  attaches a substitution score to a hit the Hamming predicate has already accepted and never
+  changes which hits return. `best_only=True` walks the distance upward, stops at the first
+  non-empty shell and returns **all** of it, from one index. `group_ids` labels records with
+  arbitrary integers — gene ids, species, clusters — and `group_by=True` folds hits onto them,
+  which makes **a tie between nearest parents a first-class output** rather than something each
+  caller re-derives. `max_hits` caps a query *after* sorting, so the best hits survive, and
+  always sets `res.truncated[i]`: a cap that is invisible is a recall bug wearing a performance
+  costume.
+
+  Saved files are flat with their arrays at known aligned offsets, so `TextIndex.load(path)`
+  **maps** rather than parses and several processes share one copy of the pages — the human
+  `k = 4` index is **348.2 MB** and maps in **0.1 ms** regardless (0.12 s to read it instead,
+  0.46 s to save it). That is 5.0x the text, and 79 % of it is `post_ids`: one `uint32` per
+  in-record k-mer start, every one of which has to be addressable for the answer to be exact.
+  A `.sti` is treated as untrusted input — the header's record, bucket and posting counts are
+  cross-checked against each other and against the file length before any of them sizes an
+  allocation or indexes into the mapping.
+
+- **The alphabet is handled asymmetrically, on purpose.** A residue outside the codec in the
+  **text** — 36 `U` in the human proteome, 33 in mouse — becomes a hole that no seed spans and
+  no hit crosses, and is counted in `ix.num_unknown`; refusing the build over them would make
+  the class unusable on the very reference data it exists for. The same residue in a **query**
+  raises, naming the query: `queries[42] ('CASSUGQYF'): symbol 'U' is not in the alphabet`.
+  `B`/`Z`/`X`/`*` are real symbols, not wildcards — `X` matches only `X`. The hole marker is
+  also what separates records, so a hit spanning a record boundary is structurally impossible
+  rather than prevented by a check that could be forgotten.
+
+- **Limits that would otherwise be silent are now refused.** A query longer than 65,535 residues
+  raises rather than wrapping its 16-bit mismatch positions. A `.sti` whose header disagrees with
+  itself or with the file length is rejected before any field sizes an allocation or indexes into
+  a mapping — on the `mmap` path there is no short read to catch it, so it has to be explicit.
+  `TextIndex.ref_seq` documents that it renders an out-of-alphabet residue as `'X'`, which is
+  lossy and where `'X'` is itself a real symbol: it reports what the index holds, not what was
+  handed to `build`. And the seed table's two counting-sort passes now assert that they visited
+  the same positions, since a disagreement would silently overrun a bucket into its neighbour.
+
+- **`TextIndex.search_batch` no longer terminates the process on an allocation failure.** Its
+  workers were exempted from the tree's `exception_ptr` plumbing on the grounds that queries are
+  validated up front — true, but they still allocate, and a `std::bad_alloc` escaping a
+  `std::thread` entry function calls `std::terminate`, which is an uncatchable `SIGABRT` from
+  Python. It now captures and rethrows after the join, like every other batch entry point.
+
+- **`_core.pyi`.** The package has shipped `py.typed` since the beginning while providing no
+  stubs at all, so every C++ symbol resolved to `Any`. nanobind generates them at build time.
+
+### Changed
+
+- **The bindings moved from pybind11 to nanobind.** Smaller module, lower per-call overhead,
+  and stub generation. Externally visible behaviour is unchanged, including the two things that
+  needed work to keep: `ScoreMatrix`'s buffer protocol, which nanobind has no equivalent for
+  (`Py_bf_getbuffer` is now wired by hand via `nb::type_slots`, so `numpy.asarray(sm)` and
+  `memoryview(sm)` still both wrap it without copying), and the exception mapping that
+  `test_pairwise.py` pins by message text. nanobind's implicit-conversion pass was checked
+  rather than assumed: `threads=1.0` still raises `TypeError` rather than truncating, so none
+  of the ~60 scalar arguments needed `noconvert`.
+
+- **`Development Status` is now `5 - Production/Stable`**, and from this release the project
+  follows semantic versioning — breaking changes need a major bump. The `docs` extra no longer
+  lists `nbsphinx`, which was in neither `docs/conf.py` nor `docs/requirements.txt`.
+
+### Audit pass before release
+
+A code and documentation audit run against the finished 1.0.0 tree. Nothing here changes an API:
+it removes duplication, documents what was already public, and corrects two doc claims that no
+longer matched the code.
+
+#### Documentation and API surface
+
+- **`engine="auto"` is documented correctly.** `docs/engines.rst` claimed `auto` "chooses per
+  query" and "routes matrix-weighted budgets to `seqtrie`". It has never done either since the
+  dispatch was removed: `auto` always resolves to `seqtm`, because `seqtrie` ignores the
+  per-type caps and would silently widen the ball. A reader following the old text and writing
+  `SearchParams(matrix="blosum62", max_penalty=12)` got **1 hit where naming `seqtrie` gives 3**,
+  with nothing in the result to say why. The page now says so and shows both.
+- **`gap_open` in the first example a reader meets.** `docs/getting-started.rst` and the README
+  Quickstart paired BLOSUM62 with `gap_open=8`, against the library's own rule of
+  `2 * matrix.scale()` = **28** for BLOSUM62. At 8 a gap is cheaper than a substitution, which is
+  the exact failure the rule exists to prevent.
+- **README leads with install and a runnable Quickstart**, then a "Which piece do I need?" routing
+  table, then the feature detail — which previously ran ~90 lines before the first `pip install`.
+  The Quickstart gained `TextIndex`, which the feature list advertised but no example used.
+- **`docs/api.rst` is grouped by task** rather than one flat section per class: searching a set,
+  searching a text, scoring, dense matrices, seed-and-extend, significance. `ScoreMatrix` was
+  public, exported and referenced from `docs/distance.rst` but had **no entry on the API page at
+  all**; it has one now. Page titles that described a comparison rather than a capability
+  ("Pairwise alignment without BioPython") or over-narrowed a payload-agnostic feature
+  ("E-values for TCR hits") were renamed, and the toctree order now runs
+  install → concepts → examples → reference → capabilities.
+
+#### Fixed in the same pass
+
+- **59 members of the C++ binding surface had no docstring** — every field of `SearchParams`,
+  `Hit`, `Alignment`, `TextHit` and `Candidate`, and most `SubstitutionMatrix` factories. `api.rst`
+  autodocs with `:undoc-members:`, so each rendered as a bare name rather than failing the build.
+  All 59 are documented, and `tests/python/test_doc_coverage.py` now gates the `_core` classes the
+  way it already gated the Python modules — plus a test that executes the README Quickstart
+  verbatim.
+- **`ArrayView.format` was a `static thread_local char[2]` scratch buffer** that each `getbuffer`
+  overwrote, so two live views of different dtypes on one thread both reported the last format.
+  It is a per-instance member now.
+
+#### Internal
+
+- **One `parallel_for` replaces seven hand-rolled copies** of the atomic cursor / adaptive chunk /
+  `exception_ptr` / rethrow-after-join scaffold (`include/seqtree/parallel.hpp`). The copies had
+  drifted: `gapblock.cpp` carried no exception plumbing at all, and the chunk cap was 1024 in four
+  places and 256 in a fifth. Per-worker scratch is still allocated once per thread, via a
+  `make_local` factory. No measured change: perf gate within 1.11x on every metric, gapblock
+  `score_matrix` 481–521 M pairs/s across three runs against the 532.7 M on record.
+- **`psutil` is out of the `[bench]` extra.** Two of the six copies of peak-RSS used it, and
+  reported *current* RSS rather than peak — a different quantity from the four stdlib `resource`
+  copies beside them. One `bench/_common.py` now holds `peak_rss_mb`, `peak_rss_gb` and `mutate`.
+- `seqtree.evalue.evalue_result` is public: the six-key E-value record, shared by `evalues` and
+  `pmhc_evalue.homolog_evalue`, which had rebuilt it from the identical formula.
+- `control._NoLock` is `contextlib.nullcontext`; `searcher.cpp`'s single-caller `pick_engine()` is
+  inlined to `p.engine != Engine::SeqTrie`.
+
+## [0.7.0] — 2026-08-16
+
+### Added
+
+- **`seqtree.distance` now enumerates a Hamming ball, not just scores one.**
+  `neighbourhood(seq, r=1, alphabet=None, include_self=True, shell=False)` lists the members of
+  the closed ball (`19·L + 1` at `r = 1` over the 20 standard residues);
+  `neighbourhood_union(seqs, ...)` takes the union over many centres, emitting each distinct
+  sequence **once**; `union_size(seqs, ...)` returns the cardinality without building or sorting
+  the result list. `shell=True` returns the sphere at exactly `r` from the *nearest* centre —
+  a per-shell quantity has to be estimated per shell.
+
+  Substitution only, fixed length: Hamming distance is undefined across lengths (`hamming`
+  already raises), and indels are `gapblock`'s problem. Deduplication happens during a
+  multi-source breadth-first walk, so the `Σ 19·L_i` multiset is never materialised — which is
+  the point, since near-duplicate centres overlap heavily. For 200 length-14 junctions all within
+  distance 1 of a common centre, the per-sequence balls double-count **41.7%** (53,400 → 31,122);
+  at spread 2, 4.5%; at spread ≥ 3, nothing.
+
+  Pure Python, measured before being written in C++: 300 junctions of length 14 at `r = 1` is
+  80,100 distinct sequences in **23 ms** on one M3 core (11 ms for `union_size`). `r = 2` over
+  the same input is 9.9 M sequences, 6.8 s and ~1.8 GB.
+
+  `alphabet=None` defaults to the 20 standard amino acids — `amino_acids()` **minus** the
+  ambiguity codes `B`/`Z`/`X` and the stop `*`, which that function does include.
+
+  Shells **partition** the closed ball exactly: shells `0..r` are pairwise disjoint, their union
+  is the ball, and every member sits at distance exactly `d` from its *nearest* centre. A
+  per-shell quantity therefore sums back to the per-ball one. Pinned by a test that checks the
+  distance with the C++ `hamming`, not with the generator's own bookkeeping.
+
+  `neighbourhood_union` and `union_size` **raise `TypeError` on a single string** rather than
+  iterating it. A `str` is iterable, so `union_size("CASSLGQYF")` would otherwise take the
+  sequence's 8 distinct *characters* as the centres and answer `20` instead of `172` — a
+  plausible integer, no error, no way to notice. Pass `["CASSLGQYF"]`, or call `neighbourhood`.
+
+- **`seqtree.__version__`.** Read from the installed distribution metadata, so `pyproject.toml`'s
+  `project.version` is the single source and a release that bumps it cannot leave a stale literal
+  behind. Asserted against the distribution metadata *and* against `pyproject.toml` by a test.
+
+### Fixed
+
+- **The documentation build stamped every page `0.6.1` while the package was `0.7.0`.**
+  `docs/conf.py` hand-copied the version into `release`/`version` and the 0.7.0 bump touched only
+  `pyproject.toml`. Both now read `seqtree.__version__`, which the docs job already installs.
 
 ## [0.6.1] — 2026-07-30
 

{seqtree-0.6.1 → seqtree-1.0.0}/CMakeLists.txt

@@ -13,7 +13,7 @@ endif()  # MSVC Release already uses /O2; -O3 is not a valid MSVC flag
 
 option(SEQTREE_TESTS "Build C++ tests" OFF)
 option(SEQTREE_BENCH "Build C++ benchmarks" OFF)
-option(SEQTREE_PYTHON "Build the pybind11 module" OFF)
+option(SEQTREE_PYTHON "Build the nanobind module" OFF)
 
 find_package(Threads REQUIRED)
 
@@ -22,6 +22,7 @@ add_library(seqtree_core STATIC
   src/substitution_matrix.cpp
   src/positional_matrix.cpp
   src/kmer_index.cpp
+  src/text_index.cpp
   src/trie.cpp
   src/index.cpp
   src/engine_seqtm.cpp
@@ -35,10 +36,21 @@ target_include_directories(seqtree_core PUBLIC include PRIVATE src)
 target_link_libraries(seqtree_core PUBLIC Threads::Threads)
 
 if(SEQTREE_PYTHON)
-  find_package(pybind11 CONFIG REQUIRED)
-  pybind11_add_module(_core src/_bindings.cpp)
+  find_package(Python 3.10 REQUIRED COMPONENTS Interpreter Development.Module)
+  # nanobind ships its CMake package inside the wheel rather than on CMAKE_PREFIX_PATH.
+  execute_process(COMMAND "${Python_EXECUTABLE}" -m nanobind --cmake_dir
+                  OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE nanobind_ROOT)
+  find_package(nanobind CONFIG REQUIRED)
+  # NB_STATIC: one extension module, so link nanobind's runtime into it rather than
+  # building the shared libnanobind nobody else would use.
+  nanobind_add_module(_core NB_STATIC src/_bindings.cpp)
   target_link_libraries(_core PRIVATE seqtree_core)
   install(TARGETS _core DESTINATION seqtree)
+
+  # py.typed has always promised types the package never shipped; nanobind generates them.
+  nanobind_add_stub(_core_stub MODULE _core OUTPUT _core.pyi
+                    PYTHON_PATH $<TARGET_FILE_DIR:_core> DEPENDS _core)
+  install(FILES ${CMAKE_CURRENT_BINARY_DIR}/_core.pyi DESTINATION seqtree)
 endif()
 
 if(SEQTREE_TESTS)
@@ -52,6 +64,7 @@ if(SEQTREE_TESTS)
     tests/cpp/test_serialize.cpp
     tests/cpp/test_positional.cpp
     tests/cpp/test_kmer_index.cpp
+    tests/cpp/test_text_index.cpp
   )
   target_include_directories(seqtree_tests PRIVATE tests/cpp src)
   target_link_libraries(seqtree_tests PRIVATE seqtree_core)

{seqtree-0.6.1 → seqtree-1.0.0}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.2
 Name: seqtree
-Version: 0.6.1
+Version: 1.0.0
 Summary: Fast fuzzy search over biological sequences (C++ core, Python bindings)
 Keywords: sequence-search,fuzzy-matching,CDR3,immunology,bioinformatics,trie
 Author-Email: ISALGO laboratory <mikhail.shugay@gmail.com>
 License: GPL-3.0-or-later
-Classifier: Development Status :: 3 - Alpha
+Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
 Classifier: Programming Language :: Python :: 3
@@ -21,11 +21,9 @@ Requires-Dist: pytest-cov>=4.1; extra == "test"
 Requires-Dist: biopython>=1.81; extra == "test"
 Provides-Extra: bench
 Requires-Dist: huggingface_hub; extra == "bench"
-Requires-Dist: psutil; extra == "bench"
 Provides-Extra: docs
 Requires-Dist: sphinx; extra == "docs"
 Requires-Dist: pydata-sphinx-theme; extra == "docs"
-Requires-Dist: nbsphinx; extra == "docs"
 Provides-Extra: control
 Requires-Dist: huggingface_hub; extra == "control"
 Provides-Extra: pmhc
@@ -44,6 +42,68 @@ Fast fuzzy search over biological sequences (amino-acid or nucleotide), as a C++
 core with a minimal Python binding. Build an immutable index once, then search
 single queries or massive batches in parallel.
 
+## Install
+
+```fish
+pip install seqtree       # prebuilt wheels for CPython 3.10–3.13
+```
+
+Prebuilt wheels cover **Linux x86-64**, **macOS arm64 (Apple Silicon)**, and **Windows x86-64**.
+There are **no Intel/x86-64 macOS wheels** — Intel Macs build from source (see below), which just
+needs a C++20 compiler and CMake (pulled in automatically by the build).
+
+## Quickstart
+
+```python
+import seqtree
+
+idx = seqtree.Index.build(["CASSLAPGATNEKLFF", "CASSLELGATNEKLFF"], alphabet="aa")
+
+p = seqtree.SearchParams(max_subs=2, engine="seqtm")
+for hit in idx.search("CASSLAPGATNEKLFF", p):
+    print(hit.ref_id, hit.score, hit.n_subs)
+
+# parallel batch (releases the GIL)
+results = idx.search_batch(queries, p, threads=0)   # 0 = all cores
+
+# matrix-weighted budget. Name seqtrie -- engine="auto" always means seqtm.
+# gap_open must follow the matrix: 2 * blosum62.scale() == 28, not the default 1.
+pm = seqtree.SearchParams(matrix="blosum62", max_penalty=12, engine="seqtrie", gap_open=28)
+top = idx.search_top("CASSLAPGATNEKLFF", pm, k=5)
+
+# alignment on demand
+aln = idx.align(0, "CASSLELGATNEKLFF", p)
+print(aln.aligned_query, aln.aligned_ref, aln.ops)
+
+# batch-vs-batch (auto-indexes the larger set)
+pairs = seqtree.pairwise_batch(query_set, db_set, p, alphabet="aa")
+
+# a short query against a long TEXT (a proteome), one index for every query length
+tix = seqtree.TextIndex.build(proteome_records, alphabet="aa", k=4)
+res = tix.search_batch(peptides, max_subs=2, threads=0)
+for hit in res[0]:
+    print(hit.ref_id, hit.offset, hit.n_subs, hit.mismatches)
+```
+
+That is the whole core loop. Significance, gap blocks, alignment and distances are in
+[More examples](#more-examples) below, and the
+[docs](https://antigenomics.github.io/seqtree/) explain the why.
+
+## Which piece do I need?
+
+| You have | You want | Use |
+|---|---|---|
+| a set of sequences | the ones within *k* edits of a query | `Index` + `SearchParams` |
+| a long text (proteome, genome) | where a short query occurs within *k* mismatches | `TextIndex` |
+| two sequences | an alignment, or a similarity score | `seqtree.pairwise` |
+| two whole sets | every pairwise distance, densely | `hamming_matrix` / `dist_matrix` / `gapblock.score_matrix` |
+| hits and a background repertoire | whether a hit is more than chance | `load_control` + `evalues` |
+| a V(D)J junction pair | an alignment with one contiguous indel | `seqtree.gapblock` |
+| one sequence | every substitution within radius *r* | `distance.neighbourhood` |
+
+Results are payload-agnostic — `(ref_id, score, n_subs, n_ins, n_dels)`. Downstream libraries map
+`ref_id` back to their own payloads (V gene, MHC, counts) and filter there.
+
 Two search engines over one trie:
 
 - **`seqtm`** — branch-and-bound enumeration. Exact per-type edit caps
@@ -53,10 +113,8 @@ Two search engines over one trie:
   `max_penalty` score budget only; it **ignores the per-type edit caps**. Use it
   when the budget is the whole specification.
 
-`engine="auto"` always picks `seqtm`, because it is the only engine that enforces
-the caps you asked for. Results are payload-agnostic:
-`(ref_id, score, n_subs, n_ins, n_dels)`. Downstream libraries map `ref_id` back
-to their own payloads (V gene, MHC, counts) and filter.
+`engine="auto"` always picks `seqtm`, because it is the only engine that enforces the caps you
+asked for — `seqtrie` runs only when you name it.
 
 Beyond search, seqtree ships:
 
@@ -68,6 +126,18 @@ Beyond search, seqtree ships:
   ([Košmrlj et al., *PNAS* 2008](https://doi.org/10.1073/pnas.0808081105); MJ contact energies from
   Miyazawa & Jernigan, *J Mol Biol* 1996) — letting dissimilar-but-chemically-equivalent loops align.
   Plus custom matrices via `SubstitutionMatrix.from_similarity` (Gram penalty `s(a,a)+s(b,b)−2·s(a,b)`).
+- **Text search** — `TextIndex` does exact k-mismatch (Hamming) search over a *concatenated text*
+  — a proteome, a genome — where `Index` would need one build per query length. The human proteome
+  has **68,389,335** nine-mer windows, so a query set spanning **45 distinct lengths** costs 45
+  multi-gigabyte builds; here `k` belongs to the index and **one build answers every length and
+  every `max_subs`**. Exact, not heuristic: one search scheme — `b = min(m+1, L/k)` disjoint
+  blocks, block `j` probed at radius `c_j`, lossless exactly when `Σc_j ≥ m − b + 1` — with
+  completeness pinned by brute-force **set equality** over L 6–30 × `max_subs` 0–3 ×
+  k ∈ {3,4,5}, and by the answer being identical across `k`. On the human proteome
+  (69,578,135 residues) one thread answers a 9-mer within 2 substitutions in **1.3 ms** and a
+  15-mer within 3 in **1.5 ms**. Results come back as flat CSR arrays with zero-copy
+  `numpy` views, mismatches as `(pos, query_aa, text_aa)` pairs, an optional fold onto
+  caller-supplied group ids that makes a tie explicit, and a cap that is always reported.
 - **E-values / significance** — calibrate hit counts against a background control repertoire
   (`load_control` + `evalues`), the TCRNET approach on a finite-sample footing. See the
   [E-value guide](https://antigenomics.github.io/seqtree/evalue.html).
@@ -94,7 +164,11 @@ Beyond search, seqtree ships:
   `hamming_matrix` / `levenshtein_matrix` score a whole set against a whole set in one GIL-released
   C++ call (`numpy.asarray` wraps the result with no copy) — no `python-Levenshtein` or `rapidfuzz`
   dependency needed. Hamming requires equal lengths (it raises otherwise); comparison is
-  case-sensitive.
+  case-sensitive. The same module *enumerates* a Hamming ball as well as scoring one:
+  `neighbourhood(seq, r)` lists its `19·L + 1` members, and `neighbourhood_union(seqs, r)` takes
+  the union over many centres with each distinct sequence emitted **once** — deduplicated during
+  the walk, so the `Σ 19·L_i` multiset never exists. For a tight specificity group that is a 41.7%
+  saving, not a rounding correction.
 - **Island profiles** — `IslandProfile.fit` builds a position weight matrix over a set of
   frame-aligned junctions (an *island*) and scores a query column by column against the island
   consensus, as a non-negative penalty that flows through `threshold_for_evalue` unchanged. At a
@@ -102,51 +176,14 @@ Beyond search, seqtree ships:
   min-over-members; at a loose cutoff the two are indistinguishable, so it earns its keep only
   where the cutoff is strict.
 
-## Install
-
-```fish
-pip install seqtree       # prebuilt wheels for CPython 3.10–3.13
-```
-
-Prebuilt wheels cover **Linux x86-64**, **macOS arm64 (Apple Silicon)**, and **Windows x86-64**.
-There are **no Intel/x86-64 macOS wheels** — Intel Macs build from source (see below), which just
-needs a C++17 compiler and CMake (pulled in automatically by the build).
-
-## Build from source
-
-Needs [uv](https://docs.astral.sh/uv/) (`brew install uv`); `setup.sh` uses it for the venv and
-the editable install.
-
-```fish
-bash setup.sh            # uv-managed .venv + editable install
-bash setup.sh --tests    # + pytest
-bash setup.sh --bench     # + benchmark deps (huggingface_hub, psutil)
-```
-
-## Quickstart
+## More examples
 
 ```python
+import numpy as np
 import seqtree
+from seqtree.pairwise import align, score, dist_matrix
 
-idx = seqtree.Index.build(["CASSLAPGATNEKLFF", "CASSLELGATNEKLFF"], alphabet="aa")
-
-p = seqtree.SearchParams(max_subs=2, engine="seqtm")
-for hit in idx.search("CASSLAPGATNEKLFF", p):
-    print(hit.ref_id, hit.score, hit.n_subs)
-
-# parallel batch (releases the GIL)
-results = idx.search_batch(queries, p, threads=0)   # 0 = all cores
-
-# matrix-weighted budget
-pm = seqtree.SearchParams(matrix="BLOSUM62", max_penalty=12, engine="seqtrie")
-top = idx.search_top("CASSLAPGATNEKLFF", pm, k=5)
-
-# alignment on demand
-aln = idx.align(0, "CASSLELGATNEKLFF", p)
-print(aln.aligned_query, aln.aligned_ref, aln.ops)
-
-# batch-vs-batch (auto-indexes the larger set)
-pairs = seqtree.pairwise_batch(query_set, db_set, p, alphabet="aa")
+mat = seqtree.SubstitutionMatrix.blosum62()
 
 # E-values against a background control repertoire (TCRNET-style significance)
 control = seqtree.load_control("human_trb_aa", size=1_000_000)
@@ -163,7 +200,6 @@ thetas = seqtree.threshold_for_evalue(target, control, queries, ceiling, e_targe
 from seqtree.gapblock import GapBlockIndex, central_prior, embed_in_frame
 
 gbi = GapBlockIndex(cdr3s, "aa", d_max=2)
-mat = seqtree.SubstitutionMatrix.blosum62()
 for ref_id, score, block_len, block_pos in gbi.search(
         "CASSLGQAYEQYF", 40, mat, gap_open=2 * mat.scale(),
         gap_prior=central_prior(int(1.5 * mat.scale()))):
@@ -175,15 +211,13 @@ embed_in_frame("CASSGQAYEQYF", width=14, c=4)      # 'CASS--GQAYEQYF'
 # a whole query set vs a whole reference set, in one GIL-released C++ call
 from seqtree.gapblock import score_matrix, IslandProfile
 sm = score_matrix(clonotypes, prototypes, mat, gap_open=2 * mat.scale(), threads=0)
-import numpy as np
-distances = np.asarray(sm)                          # (len(clonotypes), len(prototypes)) int32, zero-copy
+distances = np.asarray(sm)                          # (n_clonotypes, n_prototypes) int32, zero-copy
 
 # a position weight matrix over an island, still a non-negative penalty (feeds threshold_for_evalue)
 profile = IslandProfile.fit(island_members)
 profile.score("CASSLGQAYEQYF")                      # 0 on the consensus, > 0 for deviations
 
 # ordinary pairwise alignment -- Needleman-Wunsch / Smith-Waterman, no BioPython
-from seqtree.pairwise import align, score, dist_matrix
 score("CASSLGQAYEQYF", "CASSPGQAYEQF", mat)                    # global, BLAST defaults (11/1)
 score("WWWAAAWWW", "KKKAAAKKK", mat, mode="local")             # Smith-Waterman
 score("AAA", "AAAAA", mat, gap_open=5, gap_extend=5)           # linear gaps: open == extend
@@ -196,6 +230,24 @@ from seqtree.distance import hamming, levenshtein, hamming_matrix, levenshtein_m
 hamming("CASSLGQYF", "CASSPGQYF")                             # 1  (equal length only)
 levenshtein("kitten", "sitting")                             # 3
 h = np.asarray(hamming_matrix(umis, umis, threads=0))        # (len, len) int32, zero-copy
+
+# enumerate the ball, deduplicated across centres (substitution only, fixed length)
+from seqtree.distance import neighbourhood, neighbourhood_union, union_size
+neighbourhood("CASSLGQYF")                                   # 172 = 19*9 + 1
+union_size(junctions)                                        # size the job before running it
+for variant in neighbourhood_union(junctions, r=1):          # each distinct sequence once
+    ...
+```
+
+## Build from source
+
+Needs [uv](https://docs.astral.sh/uv/) (`brew install uv`); `setup.sh` uses it for the venv and
+the editable install.
+
+```fish
+bash setup.sh            # uv-managed .venv + editable install
+bash setup.sh --tests    # + pytest
+bash setup.sh --bench    # + benchmark deps (huggingface_hub)
 ```
 
 ## Tests