kmate 0.1.0__tar.gz

kmate-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tatiana Bellagio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

kmate-0.1.0/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.4
+Name: kmate
+Version: 0.1.0
+Summary: k-mer-based founder-mixture frequency estimation for pool-seq
+Author: Tatiana Bellagio
+License: MIT
+Project-URL: Homepage, https://github.com/Tatianabellagio/kMate
+Keywords: pool-seq,k-mer,allele-frequency,pangenome,EM,GrENE-net
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: pysam
+Dynamic: license-file
+
+<p align="left">
+  <img src="assets/kMate_logo.png" alt="kMate" width="420">
+</p>
+
+# kMate
+
+[![license: MIT](https://img.shields.io/github/license/Tatianabellagio/kMate)](LICENSE)
+
+Per-sample, per-record **allele-frequency estimation from pooled sequencing** against a
+multi-founder reference panel. kMate runs a weighted k-mer Poisson EM on the founder
+simplex to estimate founder frequencies (`h`), then projects through a per-record
+presence/absence matrix (`var_pa`, the founder × variant alt-allele matrix $V_\mathrm{pa}$)
+to allele frequencies for **SNPs, indels, and SVs in a single pass**, with no per-variant genotyping.
+
+## Overview
+
+<p align="center">
+  <a href="poster_PEQG/poster_peqg.pdf">
+    <img src="assets/poster_peqg.png" alt="kMate PEQG 2026 poster: tracking structural-variant trajectories across climates with alignment-free allele-frequency estimation" width="900">
+  </a>
+</p>
+
+The picture above (our [PEQG 2026 poster](poster_PEQG/poster_peqg.pdf), click to enlarge) walks through the whole idea: the
+GrENE-Net experiment evolved an equal mixture of **231 *Arabidopsis* founders** at 43 climate
+sites over 3 years, pool-sequencing the surviving populations each generation. kMate takes those
+pooled k-mer counts, solves a Poisson EM for the founder mixture against the panel's
+`kmer_pa`/`var_pa` matrices, and reads out per-record allele frequencies for **SNPs *and* SVs** at
+once. Benchmarked against simulated pools at 10× coverage, estimates track the truth closely, letting
+us follow structural-variant frequency trajectories across climates (e.g. a 181-bp insertion in
+the cold-regulated *COR413-PM2* gene, rising in cold gardens and falling in warm ones).
+
+## Install
+
+Create the `kmate` environment (mamba or conda) with its dependencies, then install the package:
+
+```bash
+git clone https://github.com/Tatianabellagio/kMate.git
+cd kMate
+mamba create -n kmate -c conda-forge -c bioconda python numpy scipy pysam jellyfish samtools
+mamba activate kmate
+pip install -e .          # installs the `kmate` command (no compilation step)
+```
+
+Python deps: `numpy`, `scipy`, `pysam`. kMate also calls `jellyfish` (k-mer counting) and `samtools` (read handling), both installed by the `mamba create` above. This gives you a `kmate` command with subcommands (`kmate --help`).
+
+### Verify the install
+
+```bash
+kmate selftest
+```
+
+This runs the bundled tiny fixture (a real Chr1 panel slice + a simulated 5-founder pool) end-to-end — exercising the full k-mer-count → EM → AF-projection path through `jellyfish`/`samtools` — and checks that the planted founder mixture is recovered. It takes a few seconds, needs no network, and prints `PASS` on a correct install. Run this **before** pointing kMate at your own data.
+
+## Usage
+
+kMate processes **one pooled sample at a time, per chromosome**.
+
+**You need**
+- **Pooled reads**: paired FASTQ (`R1.fq R2.fq`) of one pool/sample.
+- **A reference panel** encoded as per-chromosome matrices: `kmer_pa` (k-mer × founder presence/absence), `var_pa` (founder × variant alt-allele), and record `meta`. Built once from your founders' phased VCF (see [Building a panel](#building-a-panel)). The 231-founder *Arabidopsis thaliana* panel used by GrENE-Net is available on request; the matrix files are large and are not stored in the Git repo.
+
+**Run**
+
+```bash
+kmate run \
+    --kmer-pa-prefix data/kmer_pa_231_arch3_filt2inv/kmer_pa \
+    --var-pa     panel/arch3/chr1/var_pa_231_arch3_chr1.var_pa.npz \
+    --var-called panel/arch3/chr1/var_pa_231_arch3_chr1.var_called.npz \
+    --var-meta   panel/arch3/chr1/var_pa_231_arch3_chr1.meta.npz \
+    --reads R1.fq R2.fq --sample MYSAMPLE --out MYSAMPLE.tsv \
+    --threads 8 --chroms Chr1 --kmer-weight inv_mb --block-mode global
+```
+
+(`kmate run --help` lists every flag. Existing scripts that call `python src/per_sample_per_chrom.py ...` still work via thin shims that forward to the package.)
+
+**Estimator mode** (`--block-mode`)
+- `global`: one founder mixture per chromosome. Use for **selfing / inbred / founder (F0)** pools.
+- `window`: per-window mixture with HMM smoothing, for **recombinant** pools. `--block-mode window` alone reproduces the production "star2" recipe (10 kb windows, 5 smoothing passes).
+
+**Output**: a per-record TSV, one row per panel variant (SNP / indel / SV):
+
+| chrom | pos | ref_len | alt_len | alt_freq | info | n_called | se |
+|---|---|---|---|---|---|---|---|
+
+`alt_freq` is the estimated alternate-allele frequency in the pool; `n_called` and `se` carry support/uncertainty. (`--var-called` adds a per-record called-mask; `--kmer-db` lets you count k-mers once and query per-chrom instead of re-scanning reads; `--hash-size` tunes the Jellyfish hash, e.g. lower it to `100M` on memory-capped jobs.)
+
+## How it works
+
+From the read k-mer spectrum, kMate solves a weighted **Poisson EM on the 231-founder
+simplex** for the founder mixture `h`, then projects `h` through the panel's
+presence/absence matrix `var_pa` to a per-record allele frequency for SNPs, indels and SVs
+together, in one pass. Processing one chromosome at a time keeps peak memory ~5× below a
+genome-wide solve (per-chrom `h` agrees to ~0.1%). Full math + code wiring: [`ALGORITHM.md`](ALGORITHM.md).
+
+## Building a panel
+
+To run kMate on your own founder set you build the panel matrices once from a multi-founder
+**phased VCF**: `var_pa` from the founder genotypes and `kmer_pa` from a k-mer index of the
+founders. The builders live in [`panel/`](panel/) and [`data/`](data/). The bundled
+231-founder *Arabidopsis* panel (used by GrENE-Net) and its exact construction are documented
+in [`docs/PIPELINE_STATE.md`](docs/PIPELINE_STATE.md) §0.
+
+## Repository layout
+
+```
+src/kmate/   the kMate package (em_solver, kmer_count, block_em, per_sample_per_chrom, cli, selftest)
+pyproject.toml, conda/   packaging: pip-installable `kmate` CLI + conda recipe
+panel/       founder-panel construction (var_pa builders, k-mer index)
+data/        prebuilt panel matrices (kmer_pa_*, var_pa_*) + sample lists
+grenenet/    GrENE-Net application: production scale-out over the evolved cohort
+benchmarks/  end-to-end accuracy benchmarks (p80 control, p231 headline)
+sims/        pool-seq simulation framework (AF truth); see sims/README.md
+docs/        methods + analysis writeups
+```
+
+## Documentation
+
+| Doc | What it is |
+|---|---|
+| [`docs/PIPELINE_STATE.md`](docs/PIPELINE_STATE.md) | Production inputs, run recipe, and environment; the project source of truth. |
+| [`ALGORITHM.md`](ALGORITHM.md) | The kMate algorithm, math, and code wiring. |
+| [`BACKGROUND.md`](BACKGROUND.md) | Project framing, known biases, and design decisions. |
+| [`SAVIO_HPC.md`](SAVIO_HPC.md) | Cluster ops (partitions, sbatch recipes). |
+
+## Using kMate
+
+kMate is not yet published as a standalone method. If you are interested in using kMate
+for your project, or in collaborating, please get in touch:
+
+**Tatiana Bellagio** (tatianabellagio@gmail.com)
+
+kMate was developed for, and underlies the allele-frequency analyses of, the GrENE-Net
+outdoor evolution experiment in *Arabidopsis thaliana*.
+
+## License
+
+Released under the [MIT License](LICENSE).

kmate-0.1.0/README.md

@@ -0,0 +1,137 @@
+<p align="left">
+  <img src="assets/kMate_logo.png" alt="kMate" width="420">
+</p>
+
+# kMate
+
+[![license: MIT](https://img.shields.io/github/license/Tatianabellagio/kMate)](LICENSE)
+
+Per-sample, per-record **allele-frequency estimation from pooled sequencing** against a
+multi-founder reference panel. kMate runs a weighted k-mer Poisson EM on the founder
+simplex to estimate founder frequencies (`h`), then projects through a per-record
+presence/absence matrix (`var_pa`, the founder × variant alt-allele matrix $V_\mathrm{pa}$)
+to allele frequencies for **SNPs, indels, and SVs in a single pass**, with no per-variant genotyping.
+
+## Overview
+
+<p align="center">
+  <a href="poster_PEQG/poster_peqg.pdf">
+    <img src="assets/poster_peqg.png" alt="kMate PEQG 2026 poster: tracking structural-variant trajectories across climates with alignment-free allele-frequency estimation" width="900">
+  </a>
+</p>
+
+The picture above (our [PEQG 2026 poster](poster_PEQG/poster_peqg.pdf), click to enlarge) walks through the whole idea: the
+GrENE-Net experiment evolved an equal mixture of **231 *Arabidopsis* founders** at 43 climate
+sites over 3 years, pool-sequencing the surviving populations each generation. kMate takes those
+pooled k-mer counts, solves a Poisson EM for the founder mixture against the panel's
+`kmer_pa`/`var_pa` matrices, and reads out per-record allele frequencies for **SNPs *and* SVs** at
+once. Benchmarked against simulated pools at 10× coverage, estimates track the truth closely, letting
+us follow structural-variant frequency trajectories across climates (e.g. a 181-bp insertion in
+the cold-regulated *COR413-PM2* gene, rising in cold gardens and falling in warm ones).
+
+## Install
+
+Create the `kmate` environment (mamba or conda) with its dependencies, then install the package:
+
+```bash
+git clone https://github.com/Tatianabellagio/kMate.git
+cd kMate
+mamba create -n kmate -c conda-forge -c bioconda python numpy scipy pysam jellyfish samtools
+mamba activate kmate
+pip install -e .          # installs the `kmate` command (no compilation step)
+```
+
+Python deps: `numpy`, `scipy`, `pysam`. kMate also calls `jellyfish` (k-mer counting) and `samtools` (read handling), both installed by the `mamba create` above. This gives you a `kmate` command with subcommands (`kmate --help`).
+
+### Verify the install
+
+```bash
+kmate selftest
+```
+
+This runs the bundled tiny fixture (a real Chr1 panel slice + a simulated 5-founder pool) end-to-end — exercising the full k-mer-count → EM → AF-projection path through `jellyfish`/`samtools` — and checks that the planted founder mixture is recovered. It takes a few seconds, needs no network, and prints `PASS` on a correct install. Run this **before** pointing kMate at your own data.
+
+## Usage
+
+kMate processes **one pooled sample at a time, per chromosome**.
+
+**You need**
+- **Pooled reads**: paired FASTQ (`R1.fq R2.fq`) of one pool/sample.
+- **A reference panel** encoded as per-chromosome matrices: `kmer_pa` (k-mer × founder presence/absence), `var_pa` (founder × variant alt-allele), and record `meta`. Built once from your founders' phased VCF (see [Building a panel](#building-a-panel)). The 231-founder *Arabidopsis thaliana* panel used by GrENE-Net is available on request; the matrix files are large and are not stored in the Git repo.
+
+**Run**
+
+```bash
+kmate run \
+    --kmer-pa-prefix data/kmer_pa_231_arch3_filt2inv/kmer_pa \
+    --var-pa     panel/arch3/chr1/var_pa_231_arch3_chr1.var_pa.npz \
+    --var-called panel/arch3/chr1/var_pa_231_arch3_chr1.var_called.npz \
+    --var-meta   panel/arch3/chr1/var_pa_231_arch3_chr1.meta.npz \
+    --reads R1.fq R2.fq --sample MYSAMPLE --out MYSAMPLE.tsv \
+    --threads 8 --chroms Chr1 --kmer-weight inv_mb --block-mode global
+```
+
+(`kmate run --help` lists every flag. Existing scripts that call `python src/per_sample_per_chrom.py ...` still work via thin shims that forward to the package.)
+
+**Estimator mode** (`--block-mode`)
+- `global`: one founder mixture per chromosome. Use for **selfing / inbred / founder (F0)** pools.
+- `window`: per-window mixture with HMM smoothing, for **recombinant** pools. `--block-mode window` alone reproduces the production "star2" recipe (10 kb windows, 5 smoothing passes).
+
+**Output**: a per-record TSV, one row per panel variant (SNP / indel / SV):
+
+| chrom | pos | ref_len | alt_len | alt_freq | info | n_called | se |
+|---|---|---|---|---|---|---|---|
+
+`alt_freq` is the estimated alternate-allele frequency in the pool; `n_called` and `se` carry support/uncertainty. (`--var-called` adds a per-record called-mask; `--kmer-db` lets you count k-mers once and query per-chrom instead of re-scanning reads; `--hash-size` tunes the Jellyfish hash, e.g. lower it to `100M` on memory-capped jobs.)
+
+## How it works
+
+From the read k-mer spectrum, kMate solves a weighted **Poisson EM on the 231-founder
+simplex** for the founder mixture `h`, then projects `h` through the panel's
+presence/absence matrix `var_pa` to a per-record allele frequency for SNPs, indels and SVs
+together, in one pass. Processing one chromosome at a time keeps peak memory ~5× below a
+genome-wide solve (per-chrom `h` agrees to ~0.1%). Full math + code wiring: [`ALGORITHM.md`](ALGORITHM.md).
+
+## Building a panel
+
+To run kMate on your own founder set you build the panel matrices once from a multi-founder
+**phased VCF**: `var_pa` from the founder genotypes and `kmer_pa` from a k-mer index of the
+founders. The builders live in [`panel/`](panel/) and [`data/`](data/). The bundled
+231-founder *Arabidopsis* panel (used by GrENE-Net) and its exact construction are documented
+in [`docs/PIPELINE_STATE.md`](docs/PIPELINE_STATE.md) §0.
+
+## Repository layout
+
+```
+src/kmate/   the kMate package (em_solver, kmer_count, block_em, per_sample_per_chrom, cli, selftest)
+pyproject.toml, conda/   packaging: pip-installable `kmate` CLI + conda recipe
+panel/       founder-panel construction (var_pa builders, k-mer index)
+data/        prebuilt panel matrices (kmer_pa_*, var_pa_*) + sample lists
+grenenet/    GrENE-Net application: production scale-out over the evolved cohort
+benchmarks/  end-to-end accuracy benchmarks (p80 control, p231 headline)
+sims/        pool-seq simulation framework (AF truth); see sims/README.md
+docs/        methods + analysis writeups
+```
+
+## Documentation
+
+| Doc | What it is |
+|---|---|
+| [`docs/PIPELINE_STATE.md`](docs/PIPELINE_STATE.md) | Production inputs, run recipe, and environment; the project source of truth. |
+| [`ALGORITHM.md`](ALGORITHM.md) | The kMate algorithm, math, and code wiring. |
+| [`BACKGROUND.md`](BACKGROUND.md) | Project framing, known biases, and design decisions. |
+| [`SAVIO_HPC.md`](SAVIO_HPC.md) | Cluster ops (partitions, sbatch recipes). |
+
+## Using kMate
+
+kMate is not yet published as a standalone method. If you are interested in using kMate
+for your project, or in collaborating, please get in touch:
+
+**Tatiana Bellagio** (tatianabellagio@gmail.com)
+
+kMate was developed for, and underlies the allele-frequency analyses of, the GrENE-Net
+outdoor evolution experiment in *Arabidopsis thaliana*.
+
+## License
+
+Released under the [MIT License](LICENSE).

kmate-0.1.0/pyproject.toml

@@ -0,0 +1,30 @@
+[build-system]
+requires = ["setuptools>=61", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "kmate"
+version = "0.1.0"
+description = "k-mer-based founder-mixture frequency estimation for pool-seq"
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+authors = [{ name = "Tatiana Bellagio" }]
+keywords = ["pool-seq", "k-mer", "allele-frequency", "pangenome", "EM", "GrENE-net"]
+# Python deps only. Non-Python runtime tools (jellyfish, samtools) are declared
+# in the conda recipe (conda/meta.yaml), since pip cannot install them.
+dependencies = ["numpy", "scipy", "pysam"]
+
+[project.scripts]
+kmate = "kmate.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/Tatianabellagio/kMate"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["kmate*"]
+
+[tool.setuptools.package-data]
+# Ship the tiny end-to-end self-test fixture (`kmate selftest`).
+kmate = ["data/selftest/*"]

kmate-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

kmate-0.1.0/src/kmate/__init__.py

@@ -0,0 +1,7 @@
+"""kMate — k-mer-based founder-mixture frequency estimation for pool-seq.
+
+Two estimators (`--block-mode global|window`), optional per-bubble k-mer
+de-replication weighting (`--kmer-weight inv_mb`), and a count-once/query path
+for scale-out. See `kmate --help` and the docs/ directory.
+"""
+__version__ = "0.1.0"

kmate-0.1.0/src/kmate/block_em.py

@@ -0,0 +1,295 @@
+"""
+Per-LD-block founder-frequency estimation.
+
+Architecture (HAFpipe / hapFIRE style):
+    For evolved pool samples, chromosomes are mosaic — different genomic
+    regions carry different ancestry mixtures because of recombination +
+    selection. A single genome-wide h discards this signal. Per-block h
+    captures local ancestry, and per-record alt_freq becomes:
+
+        alt_freq(record) = h_local[block(record)] @ var_pa[:, record]
+
+Block definitions:
+    - 'window': fixed bp windows (default 200 kb). Simple; ignores LD.
+    - 'ld' (TODO): LD-cluster based on founder-genotype correlation
+      between adjacent bubbles. Variable-length, biologically correct.
+      hapFIRE uses BigLD.
+
+For a window of W kb in Arabidopsis (selfing ~97%), expected founder
+haplotypes per window is small (10–50 distinct), so EM is well-conditioned
+even with K_window ~ 100 k-mers/founder.
+"""
+from __future__ import annotations
+import time
+from dataclasses import dataclass
+
+import numpy as np
+from .em_solver import solve_em
+
+
+@dataclass
+class BlockSpec:
+    """Genomic block: (chrom, start, end) in 1-based coordinates."""
+    chrom: str
+    start: int
+    end: int
+
+    def __repr__(self):
+        return f"{self.chrom}:{self.start}-{self.end}"
+
+
+def define_windows(bubble_chrom, bubble_start, bubble_end,
+                   window_bp=200_000, window_step: int | None = None):
+    """Define fixed-bp windows covering all bubbles.
+
+    Args:
+        window_bp: width of each window in bp.
+        window_step: step between window starts. If None or == window_bp, windows
+            are disjoint (legacy behaviour). If < window_bp, windows overlap by
+            (window_bp - window_step). E.g. window_bp=10000, window_step=3000 →
+            10 kb windows stepping every 3 kb, each k-mer in ~3-4 covering
+            windows. Used for Route 2 (overlapping-window smoothing).
+
+    Returns list of BlockSpec tuples, ordered by (chrom, start).
+    """
+    if window_step is None:
+        window_step = window_bp
+    if window_step <= 0:
+        raise ValueError(f"window_step must be > 0, got {window_step}")
+    blocks = []
+    for chrom in sorted(set(bubble_chrom)):
+        m = bubble_chrom == chrom
+        if not m.any():
+            continue
+        chrom_max = int(bubble_end[m].max())
+        n_w = (chrom_max + window_step - 1) // window_step
+        for w in range(n_w):
+            start = w * window_step + 1
+            end = start + window_bp - 1
+            blocks.append(BlockSpec(chrom=chrom, start=start, end=end))
+    return blocks
+
+
+def assign_kmers_to_blocks(kmer_bubble_id, bubble_chrom, bubble_start,
+                           bubble_end, blocks):
+    """For each k-mer, return its block index (or -1 if no block contains it).
+
+    A k-mer is assigned to the block whose [start, end] range overlaps its
+    bubble centroid. Bubbles spanning a block boundary are assigned to the
+    block containing their start.
+
+    Returns a 1D K-vec assuming disjoint windows (each k-mer in exactly one
+    block).
+    """
+    K = len(kmer_bubble_id)
+    bubble_centroid = (bubble_start + bubble_end) // 2
+
+    # Precompute per-chrom block index ranges for fast lookup
+    chrom_blocks = {}
+    for i, b in enumerate(blocks):
+        chrom_blocks.setdefault(b.chrom, []).append((b.start, b.end, i))
+    for c in chrom_blocks:
+        chrom_blocks[c].sort()
+
+    # For each bubble, find its block
+    bubble_block = np.full(len(bubble_chrom), -1, dtype=np.int32)
+    for b_idx in range(len(bubble_chrom)):
+        c = str(bubble_chrom[b_idx])
+        p = int(bubble_centroid[b_idx])
+        if c not in chrom_blocks:
+            continue
+        # Linear scan within chromosome — could binary search, but block
+        # count per chrom is small (~150 for 200kb on Chr1)
+        for s, e, idx in chrom_blocks[c]:
+            if s <= p <= e:
+                bubble_block[b_idx] = idx
+                break
+
+    # Map k-mers via their bubble_id
+    kmer_block = bubble_block[kmer_bubble_id]
+    return kmer_block
+
+
+def assign_records_to_blocks(record_chrom, record_pos, blocks):
+    """Per-var_pa-record block index. Same scheme as assign_kmers_to_blocks."""
+    chrom_blocks = {}
+    for i, b in enumerate(blocks):
+        chrom_blocks.setdefault(b.chrom, []).append((b.start, b.end, i))
+    for c in chrom_blocks:
+        chrom_blocks[c].sort()
+
+    rec_block = np.full(len(record_chrom), -1, dtype=np.int32)
+    # Group records by chrom and use searchsorted for vectorization
+    for c, items in chrom_blocks.items():
+        starts = np.array([s for s, e, idx in items])
+        ends = np.array([e for s, e, idx in items])
+        idxs = np.array([idx for s, e, idx in items])
+        mask = record_chrom == c
+        if not mask.any():
+            continue
+        pos_c = record_pos[mask]
+        # find the block whose start <= pos <= end. Since blocks are
+        # contiguous and sorted, binary search by start.
+        i_starts = np.searchsorted(starts, pos_c, side="right") - 1
+        i_starts = np.clip(i_starts, 0, len(starts) - 1)
+        in_range = (pos_c >= starts[i_starts]) & (pos_c <= ends[i_starts])
+        block_for_rec = np.where(in_range, idxs[i_starts], -1)
+        rec_block[mask] = block_for_rec
+    return rec_block
+
+
+def solve_em_per_block(counts, kmer_pa_dense, kmer_block, n_blocks,
+                       coverage, em_max_iter=200, tol=1e-7,
+                       min_kmers_per_block=200, verbose=False,
+                       n_workers=4,
+                       global_anchor_weight: float = 0.0,
+                       omega=None):
+    """Run EM independently per block.
+
+    omega: optional K-vec of per-k-mer weights ω_k (e.g. 1/m_b). Sliced per block
+    and passed to solve_em (omega=None → unweighted MLE; identical to old behavior).
+
+    Args:
+        counts: K-vec of observed counts (already filtered to block coverage)
+        kmer_pa_dense: F × K float32 kmer_pa matrix
+        kmer_block: K-vec of block index per k-mer (-1 = no block)
+        n_blocks: number of blocks
+        coverage: scalar coverage estimate
+        min_kmers_per_block: blocks with fewer NONZERO-count k-mers fall
+            back to a global-h estimate (their h is undefined locally).
+        n_workers: thread workers for per-block EM (numpy releases GIL during
+            BLAS, so threading shares memory. Default 4. Set to 1 for serial.)
+        global_anchor_weight: λ ≥ 0. If > 0, per-block EM is solved with a
+            Dirichlet pseudocount centered on `global_h` (the chrom-wide EM
+            solution that's already computed for fallback). λ = 0 → pure
+            per-window MLE (legacy behaviour). 0.05–0.5 = mild anchor.
+
+    Returns:
+        h_blocks: (n_blocks, F) float32 — per-block ancestry vectors. For
+            blocks with too few k-mers, falls back to global-h (computed
+            once over all k-mers).
+        block_status: (n_blocks,) int — 0=local fit, 1=global fallback,
+            2=excluded (no k-mers).
+        global_h: (F,) — fallback value
+    """
+    from concurrent.futures import ThreadPoolExecutor
+    try:
+        from threadpoolctl import threadpool_limits
+    except ImportError:
+        # threadpoolctl only bounds BLAS oversubscription across the parallel
+        # per-block fits; the EM is still correct without it (just potentially
+        # thread-greedy). Degrade gracefully rather than hard-crash a cohort run
+        # if the env is missing the package.
+        from contextlib import contextmanager
+        @contextmanager
+        def threadpool_limits(limits=None):
+            yield
+
+    F = kmer_pa_dense.shape[0]
+    h_blocks = np.zeros((n_blocks, F), dtype=np.float32)
+    block_status = np.full(n_blocks, 2, dtype=np.int8)
+
+    # Global fallback: EM on ALL k-mers
+    if verbose:
+        print(f"  computing global-h fallback...", flush=True)
+    t = time.time()
+    global_h, info = solve_em(counts, kmer_pa_dense, coverage,
+                              max_iter=em_max_iter, tol=tol,
+                              omega=omega)
+    global_h = global_h.astype(np.float32)
+    if verbose:
+        print(f"    global EM: {info['iterations']} iters, {time.time()-t:.0f}s",
+              flush=True)
+        if global_anchor_weight > 0:
+            print(f"    global_anchor_weight λ={global_anchor_weight} — "
+                  f"per-window EM will be anchored toward h_global",
+                  flush=True)
+
+    nz = counts > 0
+    t0 = time.time()
+
+    # Pre-compute per-block k-mer index lists once.
+    block_kmer_idx = [np.flatnonzero(kmer_block == b) for b in range(n_blocks)]
+
+    def _fit_one(b):
+        idxs = block_kmer_idx[b]
+        if len(idxs) == 0:
+            return b, global_h, 2
+        idxs_nz = idxs[nz[idxs]]
+        if len(idxs_nz) < min_kmers_per_block:
+            return b, global_h, 1
+        cn_b = np.ascontiguousarray(kmer_pa_dense[:, idxs_nz])
+        c_b = counts[idxs_nz]
+        omega_b = None if omega is None else omega[idxs_nz]
+        if global_anchor_weight > 0:
+            h_b, _ = solve_em(c_b, cn_b, coverage,
+                              max_iter=em_max_iter, tol=tol,
+                              prior_h=global_h,
+                              prior_weight=global_anchor_weight,
+                              omega=omega_b)
+        else:
+            h_b, _ = solve_em(c_b, cn_b, coverage,
+                              max_iter=em_max_iter, tol=tol,
+                              omega=omega_b)
+        return b, h_b.astype(np.float32), 0
+
+    # Limit per-thread BLAS to avoid oversubscription. n_workers × inner_threads
+    # should ≈ total cores.
+    inner_threads = max(1, 8 // n_workers)
+    if n_workers == 1:
+        results = [_fit_one(b) for b in range(n_blocks)]
+    else:
+        with threadpool_limits(limits=inner_threads):
+            with ThreadPoolExecutor(max_workers=n_workers) as ex:
+                results = list(ex.map(_fit_one, range(n_blocks)))
+
+    n_local = n_global = n_excluded = 0
+    for b, h_b, st in results:
+        h_blocks[b] = h_b
+        block_status[b] = st
+        if st == 0: n_local += 1
+        elif st == 1: n_global += 1
+        else: n_excluded += 1
+
+    if verbose:
+        print(f"  per-block EM: {n_local} local fits, {n_global} global "
+              f"fallbacks, {n_excluded} excluded; {time.time()-t0:.0f}s "
+              f"(workers={n_workers}, inner_threads={inner_threads})",
+              flush=True)
+    return h_blocks, block_status, global_h
+
+
+def project_blocks_to_records(h_blocks, global_h, var_pa, record_block,
+                              var_called=None):
+    """Project per-block h to per-record alt freqs by hard window assignment.
+
+    Each record uses the h of the window containing it, h_blocks[record_block[r]]
+    (already set to global_h for fallback/excluded windows by solve_em_per_block;
+    records with no window, record_block == -1, use global_h). With HMM smoothing
+    applied upstream, this hard assignment is the production projection.
+
+    With var_called, AF is the missing-aware (h@var_pa)/(h@var_called);
+    without it, ./. is treated as REF (→ SV AF under-call at high missingness).
+
+    Returns (alt_freq, info), both length-N float64. info[r] is the projection
+    denominator — the h-weighted called mass at r (1.0 when no called mask).
+    """
+    N = var_pa.shape[1]
+    out = np.zeros(N, dtype=np.float64)
+    info = np.ones(N, dtype=np.float64)
+    rb = np.asarray(record_block)
+    var_pa_csc = var_pa.tocsc()
+    cvc_csc = var_called.tocsc() if var_called is not None else None
+    for b in np.unique(rb):
+        mask = (rb == b)
+        if not mask.any():
+            continue
+        h = (h_blocks[b] if b >= 0 else global_h).astype(np.float32)
+        num = (h @ var_pa_csc[:, mask].toarray()).astype(np.float64)
+        if cvc_csc is not None:
+            den = (h @ cvc_csc[:, mask].toarray()).astype(np.float64)
+            out[mask] = num / np.maximum(den, 1e-12)
+            info[mask] = den
+        else:
+            out[mask] = num
+    return out, info