unipan 0.4.0__tar.gz

unipan-0.4.0/PKG-INFO

@@ -0,0 +1,12 @@
+Metadata-Version: 2.4
+Name: unipan
+Version: 0.4.0
+Summary: UniPan: independent pangenome analysis engine (Python prototype)
+Author: Nagendra Pratap Singh
+Requires-Python: >=3.9
+Requires-Dist: numpy>=1.22
+Requires-Dist: numba>=0.59
+Provides-Extra: validate
+Requires-Dist: mappy>=2.24; extra == "validate"
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"

unipan-0.4.0/README.md

@@ -0,0 +1,104 @@
+# UniPan
+
+**Anchor-based pangenome structural-variant discovery with breakpoint refinement and read-level validation**
+
+UniPan compares chromosome-scale genome assemblies against a reference and reports
+syntenic blocks, indels, inversions, rearrangements and dispersed copies. It targets the
+cases where current tools are weakest: inversions in divergent sequence and
+repeat-mediated breakpoints.
+
+> **Status: research prototype (v1.1, pre-release).** Interfaces and outputs may change.
+> Results marked *preliminary* are under validation and should not be cited.
+
+## How it works
+
+| Module | What it does |
+|---|---|
+| M1 - anchors | Canonical (k=19, w=10) minimizers, invertible hash, compact NumPy index, Numba kernels |
+| M2 - chaining | Single-copy anchors, bounded DP chaining with backtracking, orientation-agnostic fragment selection, dispersed-copy detection, junction typing, syntenic blocks |
+| Breakpoint CI | Confidence interval in both genomes plus a repeat/unique context label per junction |
+| Inversion events | Consecutive inverted fragments merged into one event (scripts/m2_inversion_events.py) |
+| Breakpoint refinement | Local base-level re-alignment of each breakpoint window; reports repeat sequence at the junction (scripts/refine_breakpoints.py, prototype) |
+
+## Installation
+
+    conda create -n unipan python=3.11 -y && conda activate unipan
+    git clone https://github.com/nps-genomics/UniPan.git && cd UniPan
+    pip install -e ".[validate,test]"
+    pytest -q
+
+## Quick start
+
+    unipan run      --fasta-dir data/genomes --reference Col-0 --out results/
+    unipan validate --fasta-dir data/genomes --reference Col-0 --out results/
+    python scripts/m2_inversion_events.py results/
+    python scripts/refine_breakpoints.py results/m2_inversion_events.tsv data/genomes Col-0 results/events_refined.tsv
+
+One FASTA per genome in a folder; the genome ID is the file name stem.
+
+### Main outputs
+
+| File | Content |
+|---|---|
+| m2_fragments.tsv | Collinear fragments: coordinates in both genomes, orientation, anchors, status |
+| m2_junctions.tsv | Junctions: type, size change, breakpoint CI, repeat context |
+| m2_syntenic_blocks.tsv | Syntenic blocks |
+| m2_inversion_events.tsv | One row per inversion event with outer breakpoint intervals |
+| validation_vs_minimap2.tsv | Per-event recall / precision status |
+
+## Validation framework
+
+| Script | Purpose |
+|---|---|
+| unipan validate | Locus-based comparison with minimap2; chromosome-end and satellite masks; high-confidence subset |
+| scripts/syri_at_events.py, scripts/compare_syri_local.py | Comparison with SyRI |
+| scripts/compare_pggb.py, scripts/pggb_gfa_inversions.py | Comparison with PGGB |
+| scripts/run_longread_validation.sh | Per accession: refinement, strand-switch read test with random control, SyRI labels |
+| scripts/longread_event_test.py | Strand-switch ONT read test of inversion events (reads vs reference) |
+| scripts/longread_inversion_check.py | Assembly integrity at breakpoints (reads vs own assembly) |
+| scripts/record_result.py | Freezes a finished result into paper/ with a manifest (git commit, md5s, command) |
+
+Raw ONT reads are filtered with fastplong (mean Q >= 7, length >= 1 kb) before mapping; untrimmed reads are never mapped.
+
+## Results so far
+
+Data: five *Arabidopsis thaliana* chromosome-level assemblies (Col-0 reference; Bla-1, Cvi-0,
+Ita-0, KZ9) and ONT reads for Bla-1, Ita-0 and KZ9 (paper/data_sources.tsv). Every result is
+tracked in paper/results_registry.tsv.
+
+| Result | Status |
+|---|---|
+| Genome-wide vs minimap2: recall 92.9%, precision 95.3% (96.2% in high-confidence regions); 11 min, 7.5 GB for 5 genomes | final |
+| KZ9, Bla-1, Ita-0: 19/23 testable UniPan-only inversions confirmed by strand-switching ONT reads vs 1/900 random controls; SyRI labels these loci HDR/NOTAL (minimap2 asm5 and asm20). 36/62 UniPan-only events not testable (breakpoint interval > 20 kb) | final |
+| Breakpoint refinement: median breakpoint interval 40 to 17 kb (KZ9), 14 to 4.2 kb (Bla-1), 20 to 4.9 kb (Ita-0) | preliminary |
+| Human pilot (HG002 vs CHM13, chr8): 3.9 Mb 8p23.1 inversion on one haplotype; 10 min, 11.5 GB | preliminary |
+
+## Known limitations
+
+- Breakpoints inside long repeats are placed only approximately; refinement is a prototype.
+- Small isolated inversion calls far from any collinear anchor are often unreliable.
+- Centromeric and satellite regions are not analysed (single-copy anchors only).
+- Tested so far on *A. thaliana* (5 genomes) and one human chromosome.
+
+## Repository layout
+
+| Folder | Content |
+|---|---|
+| unipan/ | Core package (io, m1, m2, _fast, validate, cli) |
+| tests/ | Unit and synthetic tests |
+| scripts/ | Benchmarking, validation and analysis scripts |
+| paper/ | Frozen results: results_registry.tsv, data_sources.tsv, main/, supp/ |
+| grant/ | Preliminary-data material |
+| ROADMAP.md | Design decisions and version log |
+
+Raw data (assemblies, reads, BAMs) are not stored here; accessions and checksums are in
+paper/data_sources.tsv and each result's MANIFEST.json.
+
+## Authors
+
+Nagendra Pratap Singh (developer), Dr. Amit Dhingra (PI), Texas A&M University
+
+## Citation and license
+
+Manuscript in preparation. License to be decided before public release. Please contact the
+authors before using this code or its results.

unipan-0.4.0/pyproject.toml

@@ -0,0 +1,25 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "unipan"
+version = "0.4.0"
+description = "UniPan: independent pangenome analysis engine (Python prototype)"
+authors = [{ name = "Nagendra Pratap Singh" }]
+requires-python = ">=3.9"
+dependencies = ["numpy>=1.22", "numba>=0.59"]
+
+[project.optional-dependencies]
+validate = ["mappy>=2.24"]
+test = ["pytest"]
+
+[project.scripts]
+unipan = "unipan.cli:main"
+
+[tool.setuptools]
+packages = ["unipan"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+norecursedirs = ["paper", "scripts", "data", "results*", "logs", ".git"]

unipan-0.4.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

unipan-0.4.0/tests/test_call.py

@@ -0,0 +1,101 @@
+"""End-to-end `unipan call` on synthetic genomes with a known inversion."""
+import gzip
+import random
+
+from unipan.cli import main
+from unipan.io import read_tsv
+
+COMP = str.maketrans("ACGT", "TGCA")
+
+
+def rc(s):
+    return s.translate(COMP)[::-1]
+
+
+def mutate(s, rate, rng):
+    b = list(s)
+    for i in range(len(b)):
+        if rng.random() < rate:
+            b[i] = rng.choice("ACGT".replace(b[i], ""))
+    return "".join(b)
+
+
+def test_call_finds_inversion_with_refined_breakpoints(tmp_path):
+    rng = random.Random(7)
+    ref = "".join(rng.choice("ACGT") for _ in range(300_000))
+    lo, hi = 120_000, 150_000                              # 30 kb inversion
+    tgt = mutate(ref[:lo] + rc(ref[lo:hi]) + ref[hi:], 0.01, rng)
+    other = mutate(ref, 0.01, rng)                         # collinear genome
+    d = tmp_path / "g"; d.mkdir()
+    for name, seq in (("R", ref), ("T", tgt), ("C", other)):
+        (d / f"{name}.fa").write_text(f">chr1\n{seq}\n")
+    out = tmp_path / "out"
+    main(["call", "--fasta-dir", str(d), "--reference", "R", "--out", str(out)])
+    E = [e for e in read_tsv(out / "calls" / "inversion_events.tsv") if e["target"] == "T"]
+    assert len(E) == 1
+    e = E[0]
+    assert e["tier"] == "HIGH" and e["refine_status"] == "REFINED"
+    assert abs(int(e["refined_left_hi"]) - lo) <= 100 and abs(int(e["refined_right_lo"]) - hi) <= 100
+    assert not [x for x in read_tsv(out / "calls" / "inversion_events.tsv") if x["target"] == "C"]
+    rec = [l for l in gzip.open(out / "calls" / "inversions.multisample.vcf.gz", "rt") if not l.startswith("#")]
+    assert len(rec) == 1
+    assert rec[0].rstrip("\n").split("\t")[9:] == ["0:.", "0:.", "1:HIGH"]     # samples R, C, T
+
+
+def test_threads_give_identical_output(tmp_path):
+    rng = random.Random(11)
+    ref = "".join(rng.choice("ACGT") for _ in range(200_000))
+    d = tmp_path / "g"; d.mkdir()
+    (d / "R.fa").write_text(f">chr1\n{ref}\n")
+    for i, (lo, hi) in enumerate(((50_000, 70_000), (100_000, 115_000), (150_000, 170_000))):
+        tgt = mutate(ref[:lo] + rc(ref[lo:hi]) + ref[hi:], 0.01, rng)
+        (d / f"T{i}.fa").write_text(f">chr1\n{tgt}\n")
+    outs = []
+    for th in (1, 3):
+        out = tmp_path / f"o{th}"
+        main(["call", "--fasta-dir", str(d), "--reference", "R", "--out", str(out),
+              "--threads", str(th)])
+        outs.append(out)
+    for f in ("intermediate/m2_fragments.tsv", "intermediate/m2_junctions.tsv", "intermediate/m2_syntenic_blocks.tsv",
+              "calls/inversion_events.tsv", "calls/inversions.vcf.gz", "calls/inversions.multisample.vcf.gz",
+              "calls/inversions.vcf.gz.tbi", "summary.tsv", "intermediate/breakpoint_hits.tsv.gz"):
+        assert (outs[0] / f).read_bytes() == (outs[1] / f).read_bytes(), f
+
+
+def test_result_folder_layout(tmp_path):
+    rng = random.Random(3)
+    ref = "".join(rng.choice("ACGT") for _ in range(200_000))
+    d = tmp_path / "g"; d.mkdir()
+    (d / "R.fa").write_text(f">chr1\n{ref}\n")
+    tgt = mutate(ref[:60_000] + rc(ref[60_000:90_000]) + ref[90_000:], 0.01, rng)
+    (d / "T.fa").write_text(f">chr1\n{tgt}\n")
+    out = tmp_path / "out"
+    main(["call", "--fasta-dir", str(d), "--reference", "R", "--out", str(out)])
+    for f in ("README.txt", "summary.tsv", "calls/inversions.vcf.gz", "calls/inversions.vcf.gz.tbi",
+              "calls/inversions.multisample.vcf.gz.tbi", "calls/inversions.bed", "calls/inversion_events.high.tsv",
+              "qc/params.json", "qc/timing.json", "intermediate/m2_fragments.tsv"):
+        assert (out / f).exists(), f
+    assert not (out / "calls" / "inversions.vcf").exists()          # plain VCF replaced by .vcf.gz
+    s = read_tsv(out / "summary.tsv")
+    assert [r["genome"] for r in s] == ["T"] and s[0]["events"] == "1" and s[0]["HIGH"] == "1"
+    h = read_tsv(out / "calls" / "inversion_events.high.tsv")
+    assert len(h) == 1 and abs(int(h[0]["start"]) - 60_001) <= 2000
+
+
+def test_figures_and_report(tmp_path):
+    pytest = __import__("pytest")
+    pytest.importorskip("matplotlib")
+    rng = random.Random(4)
+    ref = "".join(rng.choice("ACGT") for _ in range(200_000))
+    d = tmp_path / "g"; d.mkdir()
+    (d / "R.fa").write_text(f">chr1\n{ref}\n")
+    (d / "T.fa").write_text(f">chr1\n{mutate(ref[:50_000] + rc(ref[50_000:80_000]) + ref[80_000:], 0.01, rng)}\n")
+    out = tmp_path / "out"
+    main(["call", "--fasta-dir", str(d), "--reference", "R", "--out", str(out), "--no-plots"])
+    assert not list((out / "figures").iterdir())
+    main(["report", str(out)])
+    for n in ("genome_overview", "size_distribution", "genotype_matrix", "breakpoints"):
+        assert (out / "figures" / f"{n}.png").stat().st_size > 1000 and (out / "figures" / f"{n}.pdf").exists()
+    assert "figures/" in (out / "README.txt").read_text()
+    e = read_tsv(out / "calls" / "inversion_events.tsv")[0]
+    assert e["event_id"] == "UNIPAN_INV_1" and (out / "figures" / "events" / "UNIPAN_INV_1.png").stat().st_size > 1000

unipan-0.4.0/tests/test_chromrun.py

@@ -0,0 +1,74 @@
+"""--per-chromosome must give byte-identical results to whole-genome streaming, including repeats
+shared between chromosomes (genome-wide single-copy rule), a translocated segment, a target chromosome
+with a different name, an unrelated target, several workers, and refinement."""
+import random
+
+from unipan.cli import main
+
+COMP = str.maketrans("ACGT", "TGCA")
+FILES = ("intermediate/m2_fragments.tsv", "intermediate/m2_junctions.tsv",
+         "intermediate/m2_syntenic_blocks.tsv", "intermediate/breakpoint_hits.tsv.gz",
+         "calls/inversion_events.tsv", "calls/inversion_events.high.tsv", "calls/inversions.vcf.gz",
+         "calls/inversions.multisample.vcf.gz", "calls/inversions.bed", "summary.tsv")
+
+
+def rc(s):
+    return s.translate(COMP)[::-1]
+
+
+def mutate(s, rate, rng):
+    b = list(s)
+    for i in range(len(b)):
+        if rng.random() < rate:
+            b[i] = rng.choice("ACGT".replace(b[i], ""))
+    return "".join(b)
+
+
+def _genomes(d):
+    rng = random.Random(31)
+    R = lambda n: "".join(rng.choice("ACGT") for _ in range(n))
+    rep = R(4_000)                                             # on chr1 and chr2: not single-copy
+    c1 = R(80_000) + rep + R(70_000)
+    c2 = R(50_000) + rep + R(60_000)
+    c3 = R(90_000)
+    (d / "R.fa").write_text(f">chr1\n{c1}\n>chr2\n{c2}\n>chr3\n{c3}\n")
+    t1 = c1[:40_000] + rc(c1[40_000:65_000]) + c1[65_000:]      # inversion on chr1
+    (d / "A.fa").write_text(f">a1\n{mutate(t1, 0.01, rng)}\n>a2\n{mutate(c2, 0.01, rng)}\n>a3\n{mutate(c3, 0.01, rng)}\n")
+    t2 = c2[:20_000] + rc(c2[20_000:45_000]) + c2[45_000:]      # inversion on chr2
+    t3 = c3[:30_000] + c1[100_000:130_000] + c3[30_000:]        # chr1 segment inserted into chr3
+    (d / "B.fa").write_text(f">chr2\n{mutate(t2, 0.01, rng)}\n>chr3\n{mutate(t3, 0.01, rng)}\n>chr1\n{mutate(c1, 0.01, rng)}\n")
+    (d / "Z.fa").write_text(">u\n" + R(30_000) + "\n")          # unrelated
+
+
+def test_per_chromosome_identical(tmp_path):
+    d = tmp_path / "g"; d.mkdir(); _genomes(d)
+    runs = {}
+    for tag, extra in (("whole", ["--threads", "1"]),
+                       ("mem1", ["--threads", "1", "--per-chromosome", "--store", "memory"]),
+                       ("mem3", ["--threads", "3", "--per-chromosome", "--store", "memory"]),
+                       ("disk1", ["--threads", "1", "--per-chromosome", "--store", "disk"]),
+                       ("disk3", ["--threads", "3", "--per-chromosome", "--store", "disk"]),
+                       ("auto3", ["--threads", "3", "--per-chromosome"]),
+                       ("disk3w1", ["--threads", "3", "--per-chromosome", "--store", "disk", "--disk-writers", "1"])):
+        out = tmp_path / tag
+        main(["call", "--fasta-dir", str(d), "--reference", "R", "--out", str(out), "--no-plots"] + extra)
+        runs[tag] = out
+    assert (runs["whole"] / "calls" / "inversion_events.tsv").read_text().count("\n") >= 3
+    for tag in ("mem1", "mem3", "disk1", "disk3", "auto3", "disk3w1"):
+        assert not (runs[tag] / "intermediate" / "stores").exists()
+        for f in FILES:
+            assert (runs["whole"] / f).read_bytes() == (runs[tag] / f).read_bytes(), (tag, f)
+
+
+def test_store_mode_plan(tmp_path):
+    from unipan.chromrun import plan_workers_chrom
+    files = {}
+    for g in ("R", "A", "B", "C"):
+        files[g] = tmp_path / f"{g}.fa"
+        files[g].write_text(">c\n" + "A" * 1_000_000 + "\n")
+    n, plan = plan_workers_chrom(4, files, "R", max_memory_gb=100)
+    assert plan["store"] == "memory" and n == 3
+    n, plan = plan_workers_chrom(4, files, "R", max_memory_gb=0.5)          # nothing fits: disk, 1 worker
+    assert plan["store"] == "disk" and n == 1
+    n, plan = plan_workers_chrom(4, files, "R", max_memory_gb=100, store="disk")
+    assert plan["store"] == "disk"

unipan-0.4.0/tests/test_chromstore.py

@@ -0,0 +1,82 @@
+"""Per-chromosome store: minimizers identical to index_genomes, counts identical to the
+genome-wide copy_counts32, for any number of buckets (incl. repeats shared across chromosomes,
+a chromosome shorter than k, and gzip input)."""
+import gzip
+import random
+
+import numpy as np
+import pytest
+
+from unipan.chromstore import build_store, build_store_memory, chrom_arrays, chrom_seq, iter_fasta, open_store
+from unipan.io import read_fasta
+from unipan.m1 import index_genomes
+from unipan.m2 import copy_counts32, target_profile
+
+
+def _fasta(d, gz=False):
+    rng = random.Random(4)
+    rep = "".join(rng.choice("ACGT") for _ in range(2_000))        # copy on chr1, chr2, chr3
+    rep2 = "".join(rng.choice("ACGT") for _ in range(700))         # twice on chr2
+    chrs = {
+        "chr1": "".join(rng.choice("ACGT") for _ in range(60_000)) + rep + "".join(rng.choice("ACGT") for _ in range(30_000)),
+        "chr2": rep2 + "".join(rng.choice("acgt") for _ in range(40_000)) + rep + rep2,
+        "tiny": "ACGTA",                                            # shorter than k
+        "chr3": "".join(rng.choice("ACGT") for _ in range(25_000)) + rep + "NNNNN" + "".join(rng.choice("ACGT") for _ in range(9_000)),
+    }
+    txt = "".join(f">{n} description\n" + "\n".join(s[i:i + 61] for i in range(0, len(s), 61)) + "\n" for n, s in chrs.items())
+    p = d / ("g.fa.gz" if gz else "g.fa")
+    if gz:
+        with gzip.open(p, "wt") as f:
+            f.write(txt)
+    else:
+        p.write_text(txt)
+    return p
+
+
+@pytest.mark.parametrize("buckets", [1, 3, 7])
+def test_store_equals_in_memory(tmp_path, buckets):
+    fa = _fasta(tmp_path)
+    man = build_store(fa, tmp_path / "store", buckets=buckets)
+    genome = read_fasta(fa)
+    idx = index_genomes({"g": genome})["g"]
+    prof = target_profile(idx)
+    assert [c["name"] for c in man["chromosomes"]] == list(genome)
+    assert not (tmp_path / "store" / "buckets").exists()
+    for c in man["chromosomes"]:
+        i, name = c["index"], c["name"]
+        pos, code, strand, cnt = chrom_arrays(tmp_path / "store", i)
+        assert np.array_equal(pos, idx[name][0]) and np.array_equal(code, idx[name][1])
+        assert np.array_equal(strand, idx[name][2])
+        a, b = int(prof["off"][i]), int(prof["off"][i + 1])
+        assert np.array_equal(cnt, prof["cnt"][a:b]), (name, buckets)
+        assert c["length"] == len(genome[name])
+        if c["length"]:
+            assert bytes(chrom_seq(tmp_path / "store", i)) == genome[name].encode()
+    assert (prof["cnt"] > 1).any()                                   # the test really has repeats
+
+
+def test_gzip_and_iter_fasta(tmp_path):
+    fa = _fasta(tmp_path, gz=True)
+    assert dict(iter_fasta(fa)) == read_fasta(fa)
+    man = build_store(fa, tmp_path / "s", max_memory_gb=0.000001)   # forces many buckets
+    from unipan.chromstore import max_buckets
+    assert 1 < man["buckets"] <= max_buckets() and open_store(tmp_path / "s")["buckets"] == man["buckets"]
+    idx = index_genomes({"g": read_fasta(fa)})["g"]
+    full = copy_counts32(np.concatenate([idx[n][1] for n in idx]))
+    got = np.concatenate([chrom_arrays(tmp_path / "s", c["index"])[3] for c in man["chromosomes"]])
+    assert np.array_equal(got, full)
+
+
+@pytest.mark.parametrize("gz", [False, True])
+def test_memory_store_equals_disk_store(tmp_path, gz):
+    fa = _fasta(tmp_path, gz=gz)
+    man = build_store(fa, tmp_path / "store", buckets=3)
+    ms = build_store_memory(fa)
+    assert open_store(ms)["chromosomes"] == man["chromosomes"]
+    assert open_store(ms)["repeated_codes"] == man["repeated_codes"]
+    for c in man["chromosomes"]:
+        i = c["index"]
+        for a, b in zip(chrom_arrays(tmp_path / "store", i), chrom_arrays(ms, i)):
+            assert a.dtype == b.dtype and np.array_equal(a, b)
+        if c["length"]:
+            assert bytes(chrom_seq(tmp_path / "store", i)) == bytes(chrom_seq(ms, i))

unipan-0.4.0/tests/test_events.py

@@ -0,0 +1,96 @@
+from unipan.events import events_from_fragments, tier, write_vcf, write_bed
+
+
+def frag(rs, re, ts, te, o, n, status="primary"):
+    return dict(reference="R", target="T", ref_chr="c1", target_chr="t1", status=status,
+                ref_start=rs, ref_end=re, target_start=ts, target_end=te, orientation=o, anchors=n)
+
+
+def test_inversion_split_in_three_fragments_is_one_event():
+    F = [frag(0, 100_000, 0, 100_000, "+", 20_000),
+         frag(100_500, 140_000, 159_000, 199_500, "-", 6_000),     # inverted, split by repeats
+         frag(141_000, 170_000, 129_000, 158_000, "-", 5_000),
+         frag(171_000, 199_000, 100_500, 128_500, "-", 5_000),
+         frag(200_000, 300_000, 200_000, 300_000, "+", 20_000)]
+    E = events_from_fragments(F)
+    assert len(E) == 1
+    e = E[0]
+    assert (e["ref_start"], e["ref_end"], e["n_fragments"]) == (100_500, 199_000, 3)
+    assert (e["left_ci"], e["right_ci"]) == (500, 1_000)
+    assert e["tier"] == "HIGH"
+
+
+def test_tiers():
+    base = dict(left_ci=100, right_ci=100, anchors_per_kb=90)
+    assert tier(base) == "HIGH"
+    assert tier(dict(base, anchors_per_kb=16)) == "LOW_SUPPORT"
+    assert tier(dict(base, right_ci=60_000)) == "MEDIUM"
+    assert tier(dict(base, left_ci=2_000_000, anchors_per_kb=5)) == "ISOLATED"
+
+
+def test_reverse_background_and_secondary_ignored():
+    F = [frag(0, 100_000, 300_000, 400_000, "-", 20_000),
+         frag(101_000, 150_000, 250_000, 299_000, "-", 9_000),
+         frag(151_000, 160_000, 200_000, 209_000, "+", 1_500),     # inversion against a '-' background
+         frag(161_000, 260_000, 100_000, 199_000, "-", 20_000),
+         frag(10_000, 20_000, 5_000, 15_000, "+", 900, status="secondary")]
+    E = events_from_fragments(F)
+    assert len(E) == 1 and E[0]["background"] == "-" and E[0]["ref_start"] == 151_000
+
+
+def test_vcf_and_bed(tmp_path):
+    F = [frag(0, 100_000, 0, 100_000, "+", 20_000), frag(100_500, 140_000, 159_000, 199_500, "-", 6_000),
+         frag(141_000, 300_000, 200_000, 359_000, "+", 30_000)]
+    E = events_from_fragments(F)
+    write_vcf(tmp_path / "x.vcf", E, {"c1": 300_000})
+    lines = [l for l in open(tmp_path / "x.vcf") if not l.startswith("#")]
+    f = lines[0].split("\t")
+    assert f[0] == "c1" and f[1] == "100501" and f[4] == "<INV>" and f[6] == "PASS"
+    assert "SVTYPE=INV;END=140000;" in f[7] and "CIPOS=-500,0" in f[7] and "CIEND=0,1000" in f[7]
+    write_bed(tmp_path / "x.bed", E)
+    assert open(tmp_path / "x.bed").read().startswith("c1\t100500\t140000\tINV|T|HIGH")
+
+
+def test_multisample_genotypes(tmp_path):
+    from unipan.events import merge_events, write_multisample_vcf
+
+    def fr(t, rs, re, ts, te, o, n):
+        return dict(reference="R", target=t, ref_chr="c1", target_chr="t1", status="primary",
+                    ref_start=rs, ref_end=re, target_start=ts, target_end=te, orientation=o, anchors=n)
+    F = []
+    for t in ("A", "B"):                                   # A and B carry the inversion
+        F += [fr(t, 0, 100_000, 0, 100_000, "+", 20_000), fr(t, 100_500, 140_000, 159_000, 199_500, "-", 6_000),
+              fr(t, 141_000, 300_000, 200_000, 359_000, "+", 30_000)]
+    F += [fr("C", 0, 300_000, 0, 300_000, "+", 60_000)]    # C collinear across the locus
+    F += [fr("D", 0, 90_000, 0, 90_000, "+", 18_000),      # D: locus not covered -> unresolved
+          fr("D", 160_000, 300_000, 160_000, 300_000, "+", 28_000)]
+    E = events_from_fragments(F)
+    loci, samples = merge_events(E, F)
+    assert len(loci) == 1 and samples == ["R", "A", "B", "C", "D"]
+    assert [loci[0]["gt"][s][0] for s in samples] == ["0", "1", "1", "0", "."]
+    write_multisample_vcf(tmp_path / "m.vcf", loci, samples)
+    rec = [l for l in open(tmp_path / "m.vcf") if not l.startswith("#")][0].rstrip("\n").split("\t")
+    assert "AC=2;AN=4;NMISS=1" in rec[7] and rec[8] == "GT:TR"
+    assert rec[9:] == ["0:.", "1:HIGH", "1:HIGH", "0:.", ".:."]
+
+
+def test_collinear_split_fragments_genotype_zero():
+    from unipan.events import merge_events
+
+    def fr(t, rs, re, ts, te, o, n):
+        return dict(reference="R", target=t, ref_chr="c1", target_chr="t1", status="primary",
+                    ref_start=rs, ref_end=re, target_start=ts, target_end=te, orientation=o, anchors=n)
+    F = [fr("A", 0, 100_000, 0, 100_000, "+", 20_000), fr("A", 100_500, 140_000, 159_000, 199_500, "-", 6_000),
+         fr("A", 141_000, 300_000, 200_000, 359_000, "+", 30_000),
+         fr("E", 0, 115_000, 0, 115_000, "+", 23_000),       # collinear, split by an indel inside the locus
+         fr("E", 116_000, 300_000, 116_200, 300_200, "+", 37_000)]
+    loci, samples = merge_events(events_from_fragments(F), F)
+    assert [loci[0]["gt"][s][0] for s in samples] == ["0", "1", "0"]
+
+
+def test_refine_size_check():
+    from unipan.refine import size_check
+    assert size_check(395, 6027) == "CORE"          # small inverted core in a divergent block
+    assert size_check(166_000, 2_600) == "SUSPECT"  # paralog jump
+    assert size_check(-19_000, 11_600) == "OVERLAP" # junctions inside inverted repeat
+    assert size_check(13_800, 13_827) == "REFINED"

unipan-0.4.0/tests/test_fast.py

@@ -0,0 +1,59 @@
+"""The compiled kernels must reproduce the pure-Python reference exactly."""
+import random
+
+import pytest
+
+from unipan import _fast
+from unipan.m1 import minimizers
+from unipan.m2 import _dp_chains_py as dp_chains
+
+
+def _rand_seq(n, seed, n_runs=3):
+    rng = random.Random(seed)
+    s = [rng.choice("ACGT") for _ in range(n)]
+    for _ in range(n_runs):
+        a = rng.randrange(0, n - 60)
+        for i in range(a, a + rng.randrange(1, 60)):
+            s[i] = "N"
+    for _ in range(5):
+        a = rng.randrange(0, n - 300)
+        unit = "".join(rng.choice("ACGT") for _ in range(rng.randrange(1, 8)))
+        s[a:a + 250] = (unit * 300)[:250]
+    return "".join(s)
+
+
+@pytest.mark.parametrize("seed", range(6))
+@pytest.mark.parametrize("k,w", [(19, 10), (15, 5), (21, 11)])
+def test_minimizers_kernel_matches_python(seed, k, w):
+    seq = _rand_seq(20_000, seed)
+    ref = minimizers(seq, k, w)
+    pos, code, strand = _fast.minimizers_arrays(seq, k, w)
+    fast = list(zip(pos.tolist(), code.tolist(),
+                    ["+" if s > 0 else "-" for s in strand.tolist()]))
+    assert fast == ref
+
+
+def _rand_anchors(n, seed, orient):
+    rng = random.Random(seed)
+    pts, x, y = set(), 0, 50_000 if orient == "-" else 0
+    for _ in range(n):
+        x += rng.randrange(1, 40)
+        step = rng.randrange(1, 40)
+        if rng.random() < 0.02:
+            step += rng.randrange(-30, 600)
+        y = y + step if orient == "+" else y - step
+        pts.add((x, y))
+        if rng.random() < 0.05:
+            pts.add((x + rng.randrange(1, 5), rng.randrange(0, 60_000)))
+    return list(pts)
+
+
+@pytest.mark.parametrize("seed", range(5))
+@pytest.mark.parametrize("orient", ["+", "-"])
+def test_dp_kernel_matches_python(seed, orient):
+    anchors = _rand_anchors(3000, seed, orient)
+    random.Random(seed).shuffle(anchors)
+    for max_prev, max_jump in [(256, 4000), (16, 300)]:
+        ref = dp_chains(anchors, orient, max_prev, max_jump)
+        fast = _fast.dp_chains_fast(anchors, orient, max_prev, max_jump)
+        assert fast == ref

unipan-0.4.0/tests/test_localalign.py

@@ -0,0 +1,46 @@
+"""UniPan's own local aligner (breakpoint refinement without minimap2)."""
+import random
+
+from unipan.localalign import local_hits
+
+COMP = str.maketrans("ACGT", "TGCA")
+
+
+def rc(s):
+    return s.translate(COMP)[::-1]
+
+
+def mutate(s, rate, rng):
+    b = list(s)
+    for i in range(len(b)):
+        if rng.random() < rate:
+            b[i] = rng.choice("ACGT".replace(b[i], ""))
+    return "".join(b)
+
+
+def test_repeat_mediated_inversion_breakpoint():
+    """3 kb inverted repeat at the junctions, 3 % divergence: the forward hit must end
+    at the junction and the reverse hit must start at the inverted-repeat copy, so the
+    overlap equals the repeat length (what refinement reports as BPREPEAT)."""
+    g = random.Random(3)
+    ref = "".join(g.choice("ACGT") for _ in range(400_000))
+    irep = "".join(g.choice("ACGT") for _ in range(3000))
+    lo, hi = 150_000, 200_000
+    ref = ref[:lo - 3000] + irep + ref[lo:hi] + rc(irep) + ref[hi + 3000:]
+    tgt = mutate(ref[:lo] + rc(ref[lo:hi]) + ref[hi:], 0.03, g)
+    w0 = lo - 20_000
+    H = sorted(local_hits(ref[w0:lo + 20_000], tgt, 0))
+    fwd = [h for h in H if h.strand == 1]
+    rev = [h for h in H if h.strand == -1]
+    assert len(fwd) == 1 and len(rev) == 1
+    assert abs(w0 + fwd[0].q_en - lo) <= 5                    # junction
+    assert abs(w0 + rev[0].q_st - (lo - 3000)) <= 5           # inverted-repeat copy
+    assert abs(rev[0].r_en - (hi + 3000)) <= 5                # reverse hit lands on the far junction
+
+
+def test_no_spurious_inverted_hit_in_collinear_sequence():
+    g = random.Random(5)
+    ref = "".join(g.choice("ACGT") for _ in range(120_000))
+    tgt = mutate(ref, 0.02, g)
+    H = local_hits(ref[40_000:80_000], tgt, 0)
+    assert H and all(h.strand == 1 for h in H)

unipan-0.4.0/tests/test_m1.py

@@ -0,0 +1,41 @@
+import random
+
+from unipan.m1 import canonical_kmers, minimizers
+
+COMP = str.maketrans("ACGT", "TGCA")
+
+
+def revcomp(s):
+    return s.translate(COMP)[::-1]
+
+
+def test_canonical_codes_are_strand_independent():
+    random.seed(1)
+    s = "".join(random.choice("ACGT") for _ in range(500))
+    fwd = {code for _, code, _ in canonical_kmers(s, 19)}
+    rev = {code for _, code, _ in canonical_kmers(revcomp(s), 19)}
+    assert fwd == rev
+
+
+def test_reverse_complement_flips_strand_and_mirrors_position():
+    random.seed(2)
+    s = "".join(random.choice("ACGT") for _ in range(300))
+    n, k = len(s), 19
+    f = {code: (p, st) for p, code, st in canonical_kmers(s, k)}
+    for p, code, st in canonical_kmers(revcomp(s), k):
+        fp, fst = f[code]
+        assert fp == n - k - p
+        assert fst != st
+
+
+def test_minimizers_skip_n_and_cover_sequence():
+    random.seed(3)
+    s = "".join(random.choice("ACGT") for _ in range(2000))
+    s = s[:1000] + "N" * 50 + s[1050:]
+    mins = minimizers(s, 19, 10)
+    pos = [p for p, _, _ in mins]
+    assert pos == sorted(pos)
+    assert not any(980 < p < 1050 for p in pos)       # no k-mer spans the Ns
+    gaps = [b - a for a, b in zip(pos, pos[1:]) if b < 950 or a > 1050]
+    assert max(gaps) <= 10                             # window guarantee
+    assert 1 / 6.5 < len(mins) / len(s) < 1 / 4.5     # density ~ 2/(w+1)