mirpy-lib 3.3.0__tar.gz → 3.6.0__tar.gz

{mirpy_lib-3.3.0 → mirpy_lib-3.6.0}/.gitignore

@@ -10,7 +10,7 @@ tests/assets/vdjdb.slim.txt.gz
 tests/assets/real_repertoires/
 tests/assets/srx_repertoires/
 *.airr.tsv
-notebooks/assets/
+examples/assets/
 
 ## VSCode
 
@@ -184,10 +184,10 @@ cython_debug/
 #  and can be added to the global gitignore or merged into this file.  For a more nuclear
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
-mir/docs
+src/mir/docs
 .idea
 public
-notebooks/assets/derived/
+examples/assets/derived/
 
 # ML run artifacts (checkpoints/metrics) — never committed
 experiments/runs/

mirpy_lib-3.6.0/CHANGELOG.md

@@ -0,0 +1,285 @@
+# Changelog
+
+All notable changes to `mirpy-lib` (import `mir`). This project follows semantic versioning; the v3 line is a
+greenfield ML/embedding rewrite (the classical v1.x/v2 toolkit is frozen on branch `legacy-v2`).
+
+## 3.6.0 — 2026-07-30
+
+Default-on functional filtering, a new default clone-size weight for repertoire embedding, and a
+clearer crash message — a minor bump since the repertoire-embedding default changes the numeric
+output for callers who didn't pass `weight=` explicitly.
+
+### Added
+
+- **`mir embed clonotypes` / `mir embed repertoires` drop non-coding clonotypes by default**
+  (`--filter-functional`, default on; `--no-filter-functional` to disable) via
+  `vdjtools.preprocess.filter_functional` — a stop codon or legacy out-of-frame marker
+  (`[*atgc#~_?]`) in `junction_aa` otherwise either crashes (`_`) or silently produces a
+  numerically meaningless embedding (`*`). `mir embed repertoires` skips (with a warning) any
+  sample/locus left empty after filtering, instead of crashing the whole batch.
+- **`"duplicate_count"` (linear, `g(a)=a`) and `"log2p1"` (`g(a)=log2(1+a)`) clone-size weights**
+  for repertoire embedding, alongside the existing `"distinct"` (`g≡1`) and `"log1p"`/`"anscombe"`.
+  `"log2p1"` is the **new default** — see Changed.
+
+### Fixed
+
+- **`TCREmp.embed` raises a clear error when `junction_aa` contains `'_'`** (the legacy vdjtools
+  out-of-frame marker), naming the value count and pointing at `filter_functional`, instead of
+  letting seqtree's opaque `"symbol '_' is not in the alphabet"` propagate uncaught. `'*'` (stop
+  codon) is unaffected by this check — it doesn't crash, so filter it via `filter_functional` if
+  you don't want it silently embedded.
+
+### Changed
+
+- **Default clone-size weight for repertoire embedding is now `"log2p1"`** (`g=log2(1+a)`),
+  changed from `"log1p"` (natural log). Affects `mir.repertoire.sample_embedding`,
+  `RepertoireSpace.sample_cloud`, `sample_descriptor`, `class_witness`, `mir.explain.channel_drivers`,
+  and `mir embed repertoires --weight`. This changes the numeric values of `Φ(S)`'s mean/second
+  blocks (and any MMD computed from them) for callers who relied on the implicit default — pass
+  `weight="log1p"` to reproduce the old default exactly.
+
+## 3.5.0 — 2026-07-28
+
+A documentation-accuracy pass plus the fixes found by the accompanying code audit. No public API
+removed; two new keyword arguments, one runtime dependency dropped, and three paths that used to
+fail silently now fail loudly — hence a minor bump rather than a patch.
+
+### Added
+
+- **Prototype replicates — `load_prototypes(..., replicate=r)`, `n_replicates()`, and
+  `replicate=` on `TCREmp`/`PairedTCREmp.from_defaults` and both `mir embed` commands.** Each
+  bundled chain ships 10 000 real receptors whose row order is already a uniform shuffle, so a
+  disjoint block of `n` rows is an independent draw from the same pool: `replicate=r` returns block
+  `r`, giving `10000 // n` replicates (**10** at the common `n=1000`) with no new data and no RNG.
+  This answers "is my result an artefact of *which* prototypes I drew?" — a question a nested
+  `n_prototypes` sweep cannot answer, since those draws are prefixes of one another.
+
+  `replicate=0` is unchanged and remains *the* prototype set behind every preset, bundled codec and
+  published number. `prototype_hash` now covers the replicate index, so `CodecBundle`,
+  `RepertoireSpace`, `DonorCohort` and `SetEncoderBundle` refuse to mix draws (artifacts written
+  before this release read as draw 0). README and the user guide state the default, the provenance
+  (real repertoires, `seed=42`), and that cross-replicate embeddings are incomparable.
+- **`generate_background(..., species=, source="arda")`** — the P_gen background can now be drawn
+  in the **arda IMGT allele namespace** (the same frame as the prototypes and baked germline
+  distances, so generated V/J calls resolve exactly instead of taking the allele cascade) and for
+  **mouse**. Both need a `vdjtools` shipping the bundled `arda` model set: those 9 models were
+  already in the wheel but unreachable, fixed upstream in `vdjtools.model.load_bundled`. The human
+  `learned`/`olga` path is unchanged and still works on older vdjtools; asking for arda/mouse
+  without it raises a message naming the requirement.
+- **`neighbor_enrichment(..., k_max=, seed=)`** — forwarded to the `backend="ann"` engine, which
+  already warned "raise `k_max`" for a saturated neighbour ball but gave no way to do it.
+
+### Fixed
+
+- **Germline distances silently maxed out for 42 human and 71 mouse alleles**, including
+  **`IGHV3-23*01` and `IGHV1-69*01` — the two most-used human IGHV genes.** arda emits an
+  *ambiguity group* (`"IGHV1-69*01,IGHV1-69D*01"`) as one row when the alleles share an identical
+  germline region, and the allele index registered only the joined string, so a query naming a
+  member matched nothing and took the max-distance fallback — quietly, since a fallback is also the
+  legitimate answer for a genuinely unknown gene. Members are now indexed to their group's row (a
+  standalone row still wins). Across every bundled locus, 333 of 457 group-member names resolved to
+  the fallback before; now none do.
+- **`mir embed repertoires --blocks diversity` crashed** with `ValueError: zero-dimensional arrays
+  cannot be concatenated`: `SampleEmbedding.vector` assumed the kernel-mean block was always
+  present. A mean-less Φ is now a valid vector, and `mmd_distance` / `mmd_matrix` say *why* MMD
+  needs the mean block instead of raising a `NoneType` `TypeError`.
+
+### Changed
+
+- **`GermlineDistances.matrix` resolves each *distinct* allele once** instead of once per row, then
+  gathers a small `(n_distinct, K)` table and expands it. A repertoire has ~10⁵–10⁶ rows but ~10²
+  distinct alleles, so this is the dominant cost of `TCREmp.embed`: measured **2.32 s → 1.04 s** for
+  200k clonotypes × 1000 prototypes (the germline block itself 0.765 s → 0.072 s), bit-identical
+  output, nulls still take the allele fallback.
+- **`RandomFourierFeatures.transform` computes in place** after the matmul — peak memory for a deep
+  sample drops ~3× (measured 9.2 → 3.2 GiB at n=200k, D=2048), output identical.
+- **`RepertoireSpace.save` / `DonorCohort.save` refuse a model built with a custom `matrix=` or
+  `alignment=`.** Neither knob is recorded in `meta`, so `load` rebuilt the default
+  gapblock/BLOSUM62 space — a *different* coordinate system that still passed the prototype-hash
+  check. Failing at save time keeps the comparability invariant honest.
+- **`TCREmp.embed` rejects null `junction_aa`** with a message naming the column and the row count,
+  instead of an opaque `TypeError: object of type 'NoneType' has no len()` from inside seqtree.
+- **`fit_donor_embeddings` warns when a locus has too few donors to fit the identity PCA** — that
+  block was silently emitted as all-NaN, imputed to a constant, information-free channel.
+- **`alignment="sw"` raises a directed `ImportError`** ("pip install biopython") instead of a bare
+  `ModuleNotFoundError`; `python -m mir.embedding.tcremp` now skips the SW leg when BioPython is
+  absent, so the self-check passes in the default `[dev,bench]` environment.
+- **Determinism**: `bench.theory._mutate` no longer "mutates" a residue to itself (5.3% of `k=1`
+  substitutions were no-ops, and it disagreed with `density._mutate1`); `load_vdjdb` and
+  `generate_prototypes` no longer depend on `.unique()`'s arbitrary row order.
+- **Single-sourced the version**: `pyproject.toml` takes it from `src/mir/__init__.py`
+  (`[tool.hatch.version]`), `docs/conf.py` imports `mir.__version__`, and `publish.yml` validates
+  the release tag against the built wheel — one copy instead of three.
+- **Corrected the documented gap-block ↔ Smith-Waterman relationship.** `mir.distances.junction`
+  claimed the two "agree for equal-length CDR3s and rank-correlate ≥0.99 on gapped pairs". Measured
+  over all 44 850 pairs of 300 bundled human-TRB prototypes: close pairs do agree exactly, but SW is
+  a *local* alignment, so distant pairs diverge (equal-length outliers 181 vs 77) and the overall
+  rank correlation is ρ≈0.78. The docstring now says that, and that the two are different coordinate
+  systems rather than interchangeable scales.
+- **Tests**: the survival-scorer test is no longer marked `integration` (lifelines ships in
+  `[bench]`, which CI installs), lifting `mir.bench.eval` from 23% to 64% coverage in CI;
+  `test_set_encoder.py` skips instead of failing on a torch-free install, so the documented
+  `pytest tests/` is green again; the VDJdb loader test no longer depends on a gitignored fixture it
+  could never find in CI (`mir.bench.vdjdb` 44% → 100%); and `TCREmp`'s public coordinate knobs
+  (`metric="sqrt"`, custom `matrix=`, `alignment="sw"`, null rejection) have real tests instead of
+  assertions buried in a `__main__` block. Fast-tier coverage 64% → 66%.
+- **CI gained an `optional-tiers` job** (`[dev,bench,ml]` + BioPython, `-m "not benchmark"`): all of
+  `mir.ml` — ~20% of the library, including both prototype-hash comparability guards — previously
+  ran in no CI configuration, because the only job installs `[dev,bench]` and every torch test skips.
+- Docs corrections: the density `backend=` default is `"kdtree"` (all-core exact), not `"exact"`;
+  the `cv_cindex` snippets in the user guide and in `mir.explain` now match the real
+  `(durations, events, *, base, block)` signature (the old form scored every channel `NaN`);
+  `CodecBundle.load` is documented as *not* verifying (its `forward_encoder` does); `encoder.code`
+  vs `encoder.encode`; the `mir embed repertoires` flag list is complete; `SOURCES.md` records the
+  prototypes as `arda-real` (experimental) with `src/`-prefixed regenerate commands; the examples'
+  run lines point at `examples/`, not the old `notebooks/`.
+
+### Removed
+
+- **`tqdm`** from the runtime dependencies — nothing in the package imports it.
+- **`nbsphinx` / `nbsphinx-link` / `ipython`** from the `[docs]` extra, and the unreferenced
+  `docs/requirements.txt`: the docs have contained no notebooks since the examples became marimo
+  scripts. The Sphinx build stays zero-warning under `-W`.
+
+## 3.4.0 — 2026-07-18
+
+Minor: a command-line interface, a uv-based dev setup, and a documentation overhaul. No public
+Python API removed; one optional-dependency group split out.
+
+### Added
+
+- **`mir` command-line interface** (`[project.scripts]`, also `python -m mir.cli`) — the two
+  embedding scales without writing Python:
+  - `mir embed clonotypes SAMPLE` → a per-clonotype TCREMP embedding table (`e0…`).
+  - `mir embed repertoires SAMPLE…` → one repertoire vector `Φ(S)` per sample **per chain** on one
+    shared basis (`phi0…`), with optional `--mmd` pairwise-distance output.
+
+  Inputs are any format `vdjtools.io` reads (AIRR/vdjtools/MiXCR/immunoSEQ/parquet); output is TSV
+  or Parquet. See `mir embed <cmd> -h`. Tests in `tests/test_cli.py`.
+- **`mir.repertoire.correct_batch`** — Harmony-like cluster-aware batch correction on a stacked
+  sample×feature Φ matrix. Removes the batch offset *per soft cluster* (batch-diversity-penalised),
+  so a batch confounded with a biological cluster is corrected without erasing that biology; reduces
+  exactly to `mir.cohort.residualize` at `n_clusters=1` / `theta=0` (`prop:batch`).
+- **`[ann]` optional-dependency group** for the approximate-NN density backend (`pynndescent`).
+
+### Changed
+
+- **Development now uses a repo-local `.venv` via [uv](https://docs.astral.sh/uv/)** instead of
+  conda. `setup.sh` is rewritten (bash/zsh portable; `--dev-parents`, `--docs`, `--tests`); the
+  conda `environment.yml` is removed. Runtime is unchanged — still a pure-Python `py3-none-any` wheel.
+- **`pynndescent` moved from `[bench]` to the new `[ann]` extra.** `[bench]` is now all pure-wheel
+  (no numba/llvmlite), so `pip install "mirpy-lib[bench]"` resolves cleanly on any Python. Users of
+  `density.neighbor_enrichment(backend="ann")` should install `"mirpy-lib[ann]"`.
+- **`vdjtools>=3.0.0`** (was `>=2.3.0`).
+- Documentation overhaul: a use-case-driven user guide, the two CLI commands documented, an
+  examples/notebooks page, `mir.cohort` and `mir.bench.eval` added to the API reference, a logo, and
+  the sample-embedding schematic + real depth-robustness figure. Zero-warning Sphinx build.
+- **Repo layout** (no effect on the installed package): adopted the **src-layout** (`mir/` →
+  `src/mir/`); renamed `notebooks/` → `examples/`; and moved the working result/plan markdown out of
+  the repo root — `THEORY.md` to the manuscript repo, `BENCHMARKS.md` / `REPERTOIRE_{EMBEDDING,LESSONS}.md`
+  / `SQRT_D_MIGRATION.md` / `ROADMAP.md` to `2026-mirpy-analysis/benchmarks/`. Root keeps
+  README / CHANGELOG / CLAUDE / SOURCES.
+
+### Fixed
+
+- `DEFAULT_GAP_POSITIONS = (3, 4, -4, -3)` had three independent definitions
+  (`distances/junction`, `embedding/tcremp`, `ml/bundle`); now defined once in `distances.junction`
+  and imported, so the coordinate constant cannot drift.
+- `cohort.cluster_samples` docstring described itself; `AntigenMetric` and `mir.bench.eval` gained
+  the docstrings/module-map entries they were missing.
+
+## 3.3.0 — 2026-07-17
+
+Minor: one new public parameter, nothing removed or changed.
+
+### Added
+
+- **`fit_density_space(chunk_size=)`** — embed and project in batches so the full raw matrix is never
+  resident. Peak memory becomes `max(pca_fit_cap, chunk_size) × n_features` instead of
+  `len(df) × n_features`: measured **10.60 GB → 1.81 GB** at 450k pooled clonotypes, and flat in `N`
+  (vs linear), at no wall-clock cost. This is what makes whole-cohort density arms runnable on a
+  laptop — the 4.2M-clonotype pooled arm is ~51 GB raw and ~102 GB once `scaler.transform` upcasts to
+  float64. Chunking is bit-exact at the embedding level (`_embed` of a slice == the slice of
+  `_embed`); the projected coordinates agree to float noise (~1e-7 relative), since BLAS summation
+  order depends on batch shape.
+
+### Fixed
+
+- `fit_density_space`'s `pca_fit_cap` docstring claimed it "lets whole repertoires be embedded without
+  a full-matrix PCA". It caps the **fit**, not the memory — both raw matrices were already
+  materialized before the PCA was fitted. Documented, and `chunk_size=` is the actual remedy.
+- **`mir.__version__` was stale** — it read `3.1.1` on the published 3.2.0, because the release bump
+  moved `pyproject.toml` but not `mir/__init__.py`, and `publish.yml` only validates *pyproject ==
+  tag*. Both now read 3.3.0. (`__version__` is still hand-maintained; deriving it from
+  `importlib.metadata` would retire this failure mode for good.)
+- `tests/assets/olga_humanTRB_1000.txt.gz` was a slice of the alphabetically sorted VDJdb TRB dump,
+  not OLGA output as its name and `SOURCES.md` claimed — so it was **not** an antigen-naive null (12%
+  of rows had a Hamming-1 neighbour vs 0.2% for real OLGA). Regenerated from `olga-generate_sequences`;
+  provenance and a byte-reproducible regenerate command recorded in `SOURCES.md`. No test was
+  invalidated (they use it only as a generic TRB junction pool), but external calibrations that treated
+  it as a synthetic negative control were comparing VDJdb against itself. Not shipped (tests are
+  excluded from the sdist); listed here because it invalidates results, not code.
+
+## 3.2.0 — 2026-07-17
+
+Minor: one new public module, nothing removed or changed.
+
+### Added
+
+- **`mir.explain`** (T7) — explainable readouts over any repertoire feature matrix.
+  `ChannelSpec` / `ChannelBuilder` / `stack_embeddings` attach the name→column map that `Φ.vector`
+  does not carry (`stack_embeddings` is exact: `X[i] == embs[i].vector`, names only, no transform);
+  `channel_report` ablates each named channel under a caller-supplied scorer (leave-one-in by
+  default; `mode="both"` adds the conditional half that exposes *redundant* channels — high `delta`,
+  `delta_out≈0`; optional row-permutation p-values); `channel_drivers` hops from a winning
+  **kernel-mean** channel to the clonotypes driving it via `class_witness`, and refuses channels with
+  no clonotype pre-image (a Hill number's "drivers" are a category error, not an open question).
+  Scorer-agnostic by design — the library never sees the labels and ships no scorers, so a Cox
+  C-index and a CV AUC both plug in. **No existing module changed.**
+
+## 3.1.1 — 2026-07-14
+
+Maintenance re-release of 3.1.0 with no functional changes — 3.1.0 was withdrawn from PyPI, and
+this version restores the package under a fresh, clean version. The API, coordinates, and behaviour
+are identical to 3.1.0 (see below).
+
+## 3.1.0 — 2026-07-14
+
+The Part-2 feature tier on top of the 3.0 consolidated embedding core: neural codecs, continuous-density
+methods, and the sample-level (repertoire) embedding, all on arda-native coordinates.
+
+### Added
+- **`mir.density`** (T6) — graph-free continuous-density TCRNET/ALICE: balloon adaptive-radius enrichment
+  (Poisson/binomial + BH q, water-level calibration), abundance-aware weighted mass, backends
+  `exact`(BallTree) / `kdtree`(multicore) / `ann`(pynndescent).
+- **`mir.repertoire`** (T7) — sample-level embedding `Φ(S)` = RFF kernel mean ‖ coverage-standardized Hill ‖
+  second-moment Fisher; `mmd_distance`/`mmd_matrix` (now with **`unbiased=True`** diagonal-removed MMD²),
+  `hla_stratified_mmd`, `class_witness` motif finder.
+- **`mir.ml`** (Part 2, `[ml]` extra) — forward/inverse/pgen/unified neural codecs + `CodecBundle`
+  (prototype-hash-verified shipping); learned repertoire track `set_encoder` (Set-Transformer/DeepRC).
+- **Bench** — clustering `method=` (dbscan/hdbscan/optics); `bench.theory` T6 `tcrnet_convergence`,
+  `codec_losslessness`.
+- **Repertoire benchmarks** — `experiments/benchmark_repertoire_{aging,depth,cmvhla,hla,yfv,spikein,
+  agediverge}.py` and the COVID cohort suite `{covidbatch,covidhla,covidstatus,covidpaired}.py`
+  (`airr_covid19`, local-first + HF fallback). Recorded baselines in `BENCHMARKS.md`.
+
+### Changed
+- **Coordinate system re-pinned to arda-native** germline + real-repertoire prototypes (versioned; any model
+  trained on the old coords must be retrained).
+- **`mir.ml` device selection** now **CUDA → MPS → CPU** (was MPS-only) with a `MIR_DEVICE` env override and
+  CUDA seeding — GPU support beyond Apple silicon.
+- Documented all parallelism knobs (README "Performance & parallelism"): `TCREmp(threads=0)` all-core,
+  density `backend="kdtree"` multicore-exact, `cluster(n_jobs=…)`, BLAS env.
+
+### Fixed
+- **Unbiased repertoire MMD** — the biased V-statistic's `1/n_eff` self-term inflated low-diversity samples and
+  faked a divergence signal; `unbiased=True` removes it. Aging-divergence re-evaluated at depth: real but
+  diversity-coupled, not an independent axis.
+
+### Notes
+- New lessons for the theory appendix in `REPERTOIRE_LESSONS.md`; full findings in `THEORY.md` T7.
+- Pure-Python `py3-none-any` wheel; native code comes from `seqtree`/`vdjtools` wheels.
+
+## 3.0.0 — greenfield v3 embedding core (unreleased)
+- Prototype (TCREMP) embedding on `seqtree.gapblock` + baked arda germline distances; vdjtools reuse (no AIRR
+  data-model layer of its own); bench harness (VDJdb Table S1); theory scaffold (`THEORY.md` S1–S3, T1–T5).

{mirpy_lib-3.3.0 → mirpy_lib-3.6.0}/CLAUDE.md

@@ -8,9 +8,12 @@ frozen on branch **`legacy-v2`** (`mirpy-lib` 2.x) — do not develop there.
 
 ## Repo split (2026-07-16) — three homes
 This repo is now **the library + its CI/test data only**. Analysis and manuscript live elsewhere:
-- **`~/vcs/code/mirpy`** (here): `mir/` library, unit + fast/slow CI tests (`tests/`, on slim bundled
-  data), plans for new features, docs, and **slim tabular benchmark results** for README/docs. Benchmark
-  *scripts* and the theory *appendix* have moved out (below).
+- **`~/vcs/code/mirpy`** (here): **src-layout** — the library is `src/mir/` (import `mir`); unit +
+  fast/slow CI tests (`tests/`, on slim bundled data), runnable `examples/` (marimo), docs. As of the
+  3.4.x cleanup the result/plan markdown moved out: **`THEORY.md` → the manuscript repo**;
+  **`BENCHMARKS.md`, `REPERTOIRE_{EMBEDDING,LESSONS}.md`, `SQRT_D_MIGRATION.md`, `ROADMAP.md` →
+  `2026-mirpy-analysis/benchmarks/`**. Only `README`/`CHANGELOG`/`CLAUDE`/`SOURCES` stay at the root.
+  Benchmark *scripts* and the theory *appendix* had already moved out (below).
 - **`~/vcs/projects/2026-mirpy-analysis`**: all **benchmark scripts** (`benchmarks/` — local + aldan3),
   full result docs (BENCHMARKS/THEORY/…), figures, run outputs, dataset catalog. Run the mirpy library
   from here; refresh numbers-of-record here.
@@ -53,7 +56,7 @@ negative S2) — so prototypes use real reads, not model generation.
 mirpy is normally read-only to the sibling repos; the `from_arda` builder + a tandem-D generation
 fix were added to `vdjtools` under the owner's direction (this is that owner's ecosystem).
 
-## Layout (`mir/`)
+## Layout (`src/mir/`)
 - `aliases.py`, `alleles.py` — species/locus + allele normalization.
 - `distances/junction.py` — `junction_distance_matrix` (seqtree.gapblock). Coordinate knobs (all
   default to the published space, threaded through `TCREmp`): `metric="squared"` (=`d`, default) |
@@ -88,6 +91,20 @@ fix were added to `vdjtools` under the owner's direction (this is that owner's e
   Opt-in `fit_repertoire_space(n_eigs=r)` swaps the second-moment block's full `D₂(D₂+1)/2` upper-triangle
   for its top-`r` eigenvalues (rotation-invariant spectrum; default `None` = upper-tri, unchanged — but
   lossy for the *directional* HLA imprint, so the full triangle stays the recommended block; `benchmark_repertoire_spectral.py`).
+  Also `fit_repertoire_spaces` (one basis **per locus** — the multi-chain fit), `centroid_atypicality`
+  (per-sample cosine distance to a group centroid — a Φ-geometry op feeding the digital-donor atypicality channel),
+  and `correct_batch` (Harmony-like cluster-aware batch correction on a stacked Φ matrix; reduces to
+  `cohort.residualize` at `n_clusters=1`/`theta=0` — `prop:batch`; test in `test_repertoire.py`).
+- `cohort.py` — **the digital donor** (T.7): `fit_donor_embeddings`/`DonorCohort` fuse per-chain identity
+  (kernel-mean, cross-sample PCA-reduced) ‖ diversity ‖ coverage across loci through **one `ChannelBuilder`**,
+  with an `extra_channels` hook for the analysis's own tissue/clinical blocks; comparability bites twice
+  (per-locus `prototype_hash` **and** the stored identity PCA) so `save`/`load` verify every hash and
+  `transform` is the only held-out path. Plus `residualize` (batch cookbook), `cluster_samples`
+  (MMD→precomputed-metric cluster), `incidence_biomarkers` (subject-incidence Fisher, delegating to
+  `vdjtools.biomarker.fisher`). Generalizes the analysis-repo `_tcga_embedding.build_embedding` glue.
+- `bench/eval.py` — the scorers `channel_report` consumes (kept out of `explain.py` so it stays scorer-free):
+  `cv_auc` / `held_out_auc` (classification), `cv_cindex` / `km_logrank` (Cox survival, `[bench]`→lifelines),
+  `kmer_matrix` (baseline).
 - `explain.py` — **explainable readouts over any feature matrix** (T7). `ChannelSpec`/`ChannelBuilder`/
   `stack_embeddings` attach the name→column map `Φ.vector` does not carry (per-chain blocks merge by
   name); `channel_report(X, spec, scorer, base=, mode="in"|"out"|"both", n_permutations=)` ablates each
@@ -99,14 +116,22 @@ fix were added to `vdjtools` under the owner's direction (this is that owner's e
   has no clonotype pre-image and it raises. Depends one-way on `repertoire.py`; nothing there changed.
 - `ml/` — Part 2 (torch), neural codecs + `set_encoder.py` (learned repertoire track: Set-Transformer/DeepRC
   attention pooling + `SetEncoderBundle`).
+- `cli.py` — the `mir` console script (argparse, stdlib): `embed clonotypes` (TCREmp table) /
+  `embed repertoires` (per-locus `fit_repertoire_space`→`sample_embedding` Φ(S), optional `--mmd`).
+  Reads via `vdjtools.io.read`; writes TSV/parquet.
 - `resources/` — `prototypes/` (TSVs + manifest), `gene_library/` (region_annotations.txt),
   `germline_dist/` (baked `.npz`, from `build_germline_dist.py`).
 
 ## Build / test / run
-- Conda env **`mirpy`** (Python 3.12; do NOT use `.venv` here). `pip install -e .`
-  (pure-Python hatchling; no C build). Extras: `[bench] [annotate] [build] [ml] [docs] [dev]`.
-- Tests: `python -m pytest tests/ -q` (78 pass; `-m "not integration"` for the ~5s fast tier —
-  the pynndescent ANN parity test carries a one-time JIT cost). All self-contained on bundled resources.
+- **Repo-local `.venv` via uv** (Python 3.12; conda retired 2026-07-18). `bash setup.sh`
+  (bash/zsh; `--dev-parents` editable-installs `../seqtree ../vdjtools ../vdjmatch`, `--docs`,
+  `--tests`), or `uv pip install -e ".[dev,bench]"`. Pure-Python hatchling; no C build for `mir`
+  itself. Extras: `[bench] [ann] [annotate] [build] [ml] [docs] [dev] [examples]` (`[ann]` =
+  pynndescent, split out of `[bench]` so `[bench]` stays numba-free / clean to resolve).
+- CLI: `mir embed clonotypes SAMPLE` (→ clonotype embedding table) / `mir embed repertoires
+  SAMPLE… ` (→ per-sample-per-chain Φ(S)); `mir/cli.py`, `tests/test_cli.py`.
+- Tests: `python -m pytest tests/ -q` (`-m "not integration and not benchmark"` for the fast tier;
+  torch/pynndescent tests skip unless `[ml]`/`[ann]` installed). All self-contained on bundled resources.
 - Experiments: `python benchmarks/reproduce_supplementary.py` (theory S1–S3),
   `python benchmarks/benchmark_vdjdb.py` (Table S1). Analyses: `analyze_prototype_counts.py`
   (geometry saturates by K≈100 — T.1/S4), `analyze_pc_decomposition.py` (V/J η² ≈0.44/0.49,
@@ -126,6 +151,16 @@ fix were added to `vdjtools` under the owner's direction (this is that owner's e
   GPU only in `mir.ml`: `pick_device()` = **CUDA → MPS → CPU** auto, override `device=`/`MIR_DEVICE`.
 
 ## Open loops / next steps
+- **Embedding-tier roadmap** (`2026-mirpy-analysis/benchmarks/ROADMAP.md`, moved out in the 3.4.x
+  cleanup) — the "vdjtools at the embedding level" audit +
+  plan (three verbs: make / measure / generate-decode). **Phase 0** (robustness + optimization quick wins)
+  and **Phase 1** (cohort tier: `bench/eval.py`, `repertoire.{fit_repertoire_spaces,centroid_atypicality}`,
+  `cohort.py` digital donor) are **DONE**. Next: **Phase 2** generative loop (`generate.py` `DescriptorDensity`
+  + `evolve`/`sample`, `CodecBundle.from_unified/from_decoder`), then multimodal encoders + embedding
+  trajectory. **Analysis-repo follow-up:** refactor `_tcga_embedding.build_embedding` onto
+  `cohort.fit_donor_embeddings` (+ `extra_channels` for isotype/composition/atypicality) and re-verify the
+  pan-cancer ΔC numbers. NB Phase 0 flipped the density default to `backend="kdtree"` — re-verify any
+  recorded balloon-mode baselines (±1 boundary counts).
 - **v3.0 remaining**: 10X paired benchmark; docs (Sphinx theory section + notebooks); CI; publish
   `py3-none-any` wheel; regenerate `generate_prototypes.py` via `vdjtools.model.generate`.
 - **Bench tuning**: raw kneedle eps over-merges; `cluster(eps_factor=0.4)` recovers the paper