vflank 0.1.0__tar.gz

vflank-0.1.0/.claude/skills/ddpcr-conventions/SKILL.md

@@ -0,0 +1,144 @@
+---
+name: ddpcr-conventions
+description: ddPCR assay-design domain reference for the vflank repo — genomic coordinate systems (1-based MAF vs 0-based pysam), MAF column semantics, gnomAD allele-frequency masking, flank masking modes A–D, heterozygous/consensus handling, and fusion-junction logic. Use when editing or reasoning about flank extraction, coordinate math, SNP/population-frequency masking, BAM consensus, or fusion breakpoints in this codebase.
+---
+
+# ddPCR conventions & domain reference
+
+Background knowledge for working on vflank correctly. The code-level conventions
+live in `CLAUDE.md`; this is the *why* and the biology that prevents silent,
+expensive mistakes.
+
+## What ddPCR needs from us, and why masking matters
+
+Droplet Digital PCR quantifies a target by partitioning DNA into thousands of
+droplets and amplifying with a primer pair + a sequence-specific probe. The
+assay only works if the **primers and probe anneal cleanly to the template that
+is actually present in the patient.**
+
+The failure mode vflank exists to prevent: if a primer or probe sits over a
+**polymorphic position** (a common germline SNP, or a patient-private variant),
+then in patients carrying the alternate allele the oligo mismatches → reduced
+binding, allele dropout, or assay failure. So we **mask** positions that vary,
+turning them into `N` (or an IUPAC code) so the downstream designer avoids
+placing an oligo 3′ end there. We mask the **flanks** (primer-landing territory);
+the variant of interest itself is always shown literally as `[REF/ALT]` — that's
+the target, not something to hide.
+
+Default flank is ±200 bp, matching ddPCR amplicon scale (~60–200 bp products,
+designed from a wider candidate window).
+
+## Coordinate systems — get this wrong and the sequence is silently wrong
+
+- **MAF**: 1-based, fully-closed `[start, end]`. A SNP has `start == end`.
+- **pysam** (`FastaFile.fetch`, tabix `fetch`): 0-based, half-open `[start, end)`.
+- **VCF** `POS`: 1-based.
+
+Flank extraction (`core/flanks.ReferenceFlankSource`):
+```
+left  = fetch(chrom, max(0, start - flank - 1), start - 1)   # bases before variant
+right = fetch(chrom, end, end + flank)                        # bases after variant
+```
+Masking a 1-based VCF position into a flank string:
+```
+idx = pos - region_start_0based - 1
+```
+**pysam never raises on an over-run** — it returns a truncated string at contig
+ends. A flank shorter than requested is a real condition; report it, never drop
+the record silently.
+
+## MAF column semantics (TCGA/MSK)
+
+Required: `Chromosome`, `Start_Position`, `End_Position`, `Reference_Allele`,
+`Tumor_Seq_Allele2` (the somatic alt — *Allele2*, not Allele1). Metadata used in
+headers: `Hugo_Symbol`, `HGVSp_Short`, `HGVSc`, `Tumor_Sample_Barcode`.
+
+Allele encoding: `-` denotes an empty allele (insertion has `Reference_Allele = -`;
+deletion has `Tumor_Seq_Allele2 = -`). Chromosome may arrive as `7`, `chr7`,
+`Chr7`, numeric `23/24/25` (X/Y/MT), or `M/chrM`; `core/chrom.normalise_chrom`
+canonicalises all of these to the bare form.
+
+## gnomAD allele-frequency masking
+
+We mask a flank position if a gnomAD variant there is a **single-base
+substitution** (REF length 1, every ALT length 1) whose **max AF ≥ threshold**
+(default 0.001 = 0.1%). Indels are deliberately *not* masked (conservative —
+avoids over-masking primer territory).
+
+AF fields differ by build/release:
+- **gnomAD v4.1 (hg38)** exposes both `AF` and `AF_grpmax`; we take the max.
+- **gnomAD v2.1.1 (hg19)** has `AF` but **no `AF_grpmax`** — only `AF` is used.
+- Guard against `.` and `NaN` AF tokens (the parser rejects `NaN` via `f == f`).
+
+Files are per-chromosome bgzipped VCFs + tabix index; resolution is by known
+filename patterns (`core/popfreq.GNOMAD_PATTERNS`, keyed by build then
+genome/exome). A contig legitimately absent from a VCF (e.g. MT) is logged at
+DEBUG, not failed.
+
+### Two backends and `--pop-data` (genome / exome / both)
+
+The mask comes from one of two interchangeable sources (same `get_positions`
+interface, selected by `--pop-source`):
+- **`vcf`** (`GnomadStore`) — local per-chromosome VCFs. Reproducible, bulk, HPC.
+- **`api`** (`GnomadApiSource`) — gnomAD GraphQL API, no download, rate-limited
+  to ~10 req/IP/60s → small cohorts only. Region cache + throttle + backoff.
+
+`--pop-data {genome,exome,both}` (default `genome`): flanks often fall in
+non-coding regions where **only genomes have data**, so genome is the default;
+`exome` adds power in coding regions; `both` masks the **union** (common in
+either cohort). gnomAD v4 (hg38) is the only build with a pooled `joint` set; we
+use max-AF union for `both` for one code path.
+
+API AF rule (no per-population `af` field): `af = max(seq.af, max(ac/an over
+populations))` across the chosen kinds; SNPs only. Build → dataset:
+hg19 → `gnomad_r2_1`/GRCh37, hg38 → `gnomad_r4`/GRCh38; region query is 1-based
+inclusive. Requesting `--pop-data exome`/`both` on the VCF source without the
+exome files **fails fast** (`GnomadStore.preflight`) — never a silent
+genome-only fallback.
+
+## Flank source modes (the `FlankSource` strategy)
+
+| Mode | Inputs | Flank source | Masking |
+|------|--------|--------------|---------|
+| A | MAF + FASTA | reference | none |
+| B | + gnomAD | reference | common SNPs → N |
+| C | + sample BAM | patient consensus, reference fallback at low depth | het / low-confidence → N or IUPAC |
+| D | all | patient consensus | gnomAD ∪ observed-het |
+
+A and B are implemented (`ReferenceFlankSource`). C/D are the differentiator:
+patient consensus catches **private/rare** variants gnomAD never sees — the ones
+that silently break a primer for one specific patient.
+
+### Consensus (modes C/D) plan
+- Build from BAM via `bcftools mpileup → call → consensus --iupac-codes`, or
+  pysam pileup. Validate against `samtools consensus` as an oracle.
+- A position is **heterozygous** (→ mask) if the second-most-common allele
+  exceeds ~25–30% with sufficient depth; below a min-depth threshold, fall back
+  to the reference base and flag it.
+- Indels in the pileup shift coordinates — the hard part. `kindel` is the
+  reference implementation for CIGAR-described indel reconciliation, but it is
+  haploid/clonal (viral) and does **not** flag heterozygosity, so its majority
+  call must be augmented with diploid het detection.
+
+## Fusion / structural-variant path (SV)
+
+A gene fusion is defined by two breakpoints on (possibly) different chromosomes.
+The chimeric **junction sequence** is the only fusion-specific template, so the
+ddPCR probe must **span the junction**. vflank builds it in `core/fusion.py`:
+fetch each partner's flank, orient per the strand bit (`0`=plus, `1`=minus),
+concatenate across the breakpoint with **no separator** (an early `"-"` join
+corrupted the junction base — never insert non-ACGT). Masking is applied in
+genomic space *before* reverse-complement (`revcomp(N)=N`). The corrected
+junction model and the iCallSV `CT`→strand mapping are in
+docs/research/sv-vcf-input.md. Probe design over the junction is delegated to
+Primer3 (not implemented yet).
+
+## Downstream emit formats (where vflank stops)
+
+- **Olivar** (small-variant amplicons) consumes a FASTA + a SNP CSV with columns
+  `START, STOP, FREQ` (1-based). vflank's gnomAD scan and BAM-het detection both
+  produce exactly this — `--emit-olivar` is the integration seam. The `FREQ`
+  column is where patient-specific risk (from a BAM) gets injected.
+- **Primer3** handles the fusion-junction probe.
+
+vflank produces inputs for these tools; it does not design primers itself.

vflank-0.1.0/.dockerignore

@@ -0,0 +1,28 @@
+.git
+.github
+.claude
+docs
+site
+tests
+*.egg-info
+__pycache__
+*.py[cod]
+.pytest_cache
+.mypy_cache
+.ruff_cache
+.venv
+venv
+.coverage
+htmlcov
+# data files
+*.fasta
+*.fa
+*.fai
+*.gzi
+*.bam
+*.bai
+*.vcf
+*.vcf.gz
+*.vcf.bgz
+*.tbi
+*.csi

vflank-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,47 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    name: Lint & type-check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - run: pip install -e ".[dev]"
+      - name: Ruff
+        run: ruff check src tests
+      - name: Mypy
+        run: mypy src/vflank/core src/vflank/io
+
+  test:
+    name: Test (${{ matrix.os }}, py${{ matrix.python-version }})
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12"]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - run: pip install -e ".[dev]"
+      - name: Pytest
+        run: pytest --cov=vflank --cov-report=term-missing

vflank-0.1.0/.github/workflows/docs.yml

@@ -0,0 +1,41 @@
+name: Docs
+
+on:
+  push:
+    branches: [main]
+    tags: ["v*"]
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # mike needs full history + the gh-pages branch
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - run: pip install -e ".[docs]"
+      - name: Configure git
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+          git fetch origin gh-pages --depth=1 || true
+
+      # main -> the rolling "dev" docs
+      - name: Deploy dev docs
+        if: github.ref == 'refs/heads/main'
+        run: mike deploy --push --update-aliases dev
+
+      # vX.Y.Z tag -> that version, aliased "latest" and set as default
+      - name: Deploy release docs
+        if: startsWith(github.ref, 'refs/tags/v')
+        run: |
+          VERSION="${GITHUB_REF_NAME#v}"
+          mike deploy --push --update-aliases "$VERSION" latest
+          mike set-default --push latest

vflank-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,61 @@
+name: Release
+
+# Fires when a GitHub Release is published (its tag, e.g. v0.1.0, drives the
+# version). Publishes the package to PyPI and the image to GHCR.
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+jobs:
+  pypi:
+    name: Publish to PyPI
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read   # required for actions/checkout (permissions: block is exhaustive)
+      id-token: write  # OIDC for PyPI Trusted Publishing (no API token needed)
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Build sdist + wheel
+        run: |
+          pip install build
+          python -m build
+      - name: Publish
+        uses: pypa/gh-action-pypi-publish@release/v1
+
+  docker:
+    name: Publish image to GHCR
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: docker/setup-buildx-action@v3
+      - name: Log in to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Image metadata (tags + labels)
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository }}  # lowercased automatically
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=raw,value=latest
+      - name: Build and push
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max

vflank-0.1.0/.gitignore

@@ -0,0 +1,39 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+
+# Tooling caches
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Bioinformatics artifacts (large; never commit)
+*.fasta
+*.fa
+*.fai
+*.gzi
+*.bam
+*.bai
+*.vcf
+*.vcf.gz
+*.vcf.bgz
+*.tbi
+*.csi
+flanking_sequences.fasta
+
+# OS / editor
+.DS_Store
+*.swp
+.idea/
+.vscode/
+
+# Claude Code local (per-user) settings
+.claude/settings.local.json

vflank-0.1.0/.pre-commit-config.yaml

@@ -0,0 +1,23 @@
+# Run: pre-commit install   (then hooks run on `git commit`)
+# Or:  pre-commit run --all-files
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.8.4
+    hooks:
+      - id: ruff
+        args: [--fix]
+      - id: ruff-format
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.13.0
+    hooks:
+      - id: mypy
+        files: ^src/vflank/(core|io)/
+        additional_dependencies: [types-setuptools]
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      - id: check-yaml
+      - id: check-toml
+      - id: check-added-large-files

vflank-0.1.0/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+All notable changes to this project are documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
+adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-06-12
+
+### Added
+- Small-variant flank extraction from MAF (`vflank small run`), with `inspect`
+  and `list-vcf` helpers.
+- Common-SNP masking from two interchangeable backends (`--pop-source`):
+  local gnomAD VCFs and the gnomAD GraphQL API (no download).
+- `--pop-data {genome,exome,both}` for both backends.
+- Per-variant deduplication keyed on `CHR_POS_REF_ALT` (`--dedup/--no-dedup`).
+- Structural-variant junction extraction (`vflank fusion run`) from the simple
+  iCallSV/iAnnotateSV breakpoint TSV (columns matched by name), with
+  reverse-complement-aware junction construction and optional flank masking.
+- Genome-build guard (hg19/hg38 vs FASTA), flank-truncation detection, and a
+  categorised skip summary + optional TSV run report.
+- Documentation site (MkDocs Material) and GitHub Actions CI.
+
+[Unreleased]: https://github.com/rhshah/vFlank/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/rhshah/vFlank/releases/tag/v0.1.0

vflank-0.1.0/CLAUDE.md

@@ -0,0 +1,102 @@
+# CLAUDE.md — working guide for the vflank repository
+
+## What this project is
+
+`vflank` is the **variant-aware, masked-flank front-end** of a ddPCR
+assay-design pipeline. It extracts the sequence flanking genomic variants and
+masks positions that would compromise a primer/probe, then emits clean target
+sequences.
+
+It is **not** a primer designer. Design is delegated downstream — Olivar
+(small-variant amplicons) and Primer3 (fusion-junction probes) — invoked
+out-of-process. Do not add a primer/probe design algorithm to this package; add
+*emit formats* that feed those tools. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
+for the full plan, scope boundary, and milestone roadmap.
+
+## Repository layout
+
+```
+src/vflank/
+├── core/   chrom · variant · flanks · popfreq · popfreq_api · fusion · skips
+├── io/     maf · reference · fasta · breakpoints · report   ← file access
+├── cli/    app (root) · small (run/inspect/list-vcf) · fusion (run)
+├── logging.py · errors.py
+tests/      unit/ · integration/
+docs/ARCHITECTURE.md                              ← design & roadmap
+docs/research/                                     ← gnomAD-API & SV/VCF design notes
+```
+
+The original reference scripts (`get_flanking_sequence.py`,
+`design_fusion_primers.py`, `config_ES_CTDNA_03.cfg`) have been fully ported /
+re-implemented and removed. The conventions extracted from them live in
+[docs/research/](docs/research/) (corrected fusion-junction model, iCallSV
+strand mapping, etc.).
+
+## Quality gate — run before declaring any change done
+
+```bash
+python -m ruff check src tests
+python -m mypy src/vflank/core src/vflank/io
+python -m pytest
+```
+
+All three must pass. Tests run without installing the package (`pyproject.toml`
+sets `pythonpath = ["src"]`). The dev environment is mambaforge Python 3.10 with
+`typer`, `rich`, `pysam`, `pandas`, `pytest`, `ruff`, `mypy` available.
+
+## Working discipline (the bar for this repo)
+
+- **Review before and after every change.** Before: read the surrounding code
+  and run the gate. After: re-run the gate; re-read the diff for duplication,
+  dead code, and unused symbols.
+- **No silent failures.** Every error path must surface — raise a typed
+  `VflankError`, or log at an appropriate level, or record and report in the run
+  summary. Do not swallow exceptions or return empty results without a log. The
+  existing patterns: flank truncation at contig ends is detected and reported;
+  contig-absent in a VCF is logged at DEBUG; build mismatch warns.
+- **No dead or duplicated code.** If you remove the last caller of something,
+  remove the thing. Don't add an exception class / helper "for later."
+- **Keep the hot kernels pure.** `chrom.normalise_chrom`, `popfreq.parse_common_snp_positions`,
+  and `flanks.mask_sequence` are pure functions over plain values so they are
+  unit-testable without pysam and can later be ported to Rust. Preserve that.
+- **Update comments, logging, and tests with the code** — not as an afterthought.
+
+## Coordinate conventions (the #1 source of bugs — read before editing flanks)
+
+- **MAF coordinates are 1-based, fully-closed `[start, end]`.**
+- **pysam (`FastaFile.fetch`, tabix) is 0-based, half-open `[start, end)`.**
+- Flank math (see `core/flanks.ReferenceFlankSource.fetch`):
+  - left flank  = `fa.fetch(chrom, max(0, start-flank-1), start-1)` — bases *before* the variant
+  - right flank = `fa.fetch(chrom, end, end+flank)` — bases *after* the variant
+  - the variant interval `[start, end]` itself is excluded from both flanks
+- Masking maps a 1-based VCF position back to a 0-based index *within a flank*:
+  `idx = pos - region_start_0based - 1` (see `flanks.mask_sequence`).
+- pysam silently returns a **short string** when a window runs off a contig end —
+  always treat a shorter-than-requested flank as a real condition to report.
+
+## Domain knowledge
+
+For the deeper biology (why masking matters for ddPCR, gnomAD AF semantics across
+builds, masking modes A–D, indel/het caveats), invoke the `ddpcr-conventions`
+skill — it carries the reference detail that doesn't belong in always-on context.
+
+## Conventions
+
+- **Chromosome handling:** the canonical internal form is the *bare* chromosome
+  (`"7"`, `"X"`, `"MT"`). Normalise MAF input with `core/chrom.normalise_chrom`;
+  convert to a file's notation only at fetch time via `ReferenceFasta.contig` /
+  the gnomAD store. Notation (`chr1` vs `1`) is auto-detected per file.
+- **Output:** two FASTA records per variant — raw and `Masked__…` — with the
+  variant shown literally as `[REF/ALT]` between the flanks. Only single-base
+  population SNPs are masked.
+- **CLI:** Typer; status/Rich output goes to **stderr** (the `console` in
+  `logging.py`); stdout/files are for data. Library/diagnostic messages go
+  through the `vflank` logger; the CLI presents the formatted summary.
+- **Errors:** raise `VflankError` subclasses (`errors.py`) for user-facing
+  failures; the CLI catches them and prints a clean message.
+
+## Git
+
+- Work on a branch; the foundation is committed on `main`.
+- End commit messages with: `Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>`
+- Commit/push only when asked.

vflank-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,38 @@
+# Contributing to vflank
+
+Thanks for your interest in improving vflank! This is the short version; the
+full [Developer Guide](docs/DEVELOPER.md) covers setup, layout, and how to
+extend the package.
+
+## Quick start
+
+```bash
+git clone https://github.com/rhshah/vFlank.git
+cd vFlank
+pip install -e ".[dev]"
+python -m pytest          # tests resolve `vflank` from src/ automatically
+```
+
+## The quality gate (run before every PR)
+
+```bash
+python -m ruff check src tests
+python -m mypy src/vflank/core src/vflank/io
+python -m pytest
+```
+
+All three must pass; CI runs them on Linux + macOS across Python 3.10–3.12.
+
+## Conventions
+
+- Work on a feature branch; keep `core/` pure and I/O-free.
+- No silent failures — surface every error path (raise / log / report).
+- Add or update tests in the matching `tests/` subtree.
+- Match the surrounding style; the coordinate conventions in
+  [CLAUDE.md](CLAUDE.md) are the #1 thing to get right when touching flanks.
+- End commit messages and PRs with a clear, present-tense summary.
+
+## Reporting issues
+
+Open an issue at https://github.com/rhshah/vFlank/issues with a minimal
+reproduction (a few-line MAF/TSV + the command) where possible.

vflank-0.1.0/Dockerfile

@@ -0,0 +1,24 @@
+# syntax=docker/dockerfile:1
+
+# --- Build the wheel ---
+FROM python:3.12-slim AS builder
+WORKDIR /src
+COPY pyproject.toml README.md LICENSE ./
+COPY src ./src
+RUN pip install --no-cache-dir build && python -m build --wheel --outdir /dist
+
+# --- Runtime ---
+FROM python:3.12-slim
+LABEL org.opencontainers.image.source="https://github.com/rhshah/vFlank" \
+      org.opencontainers.image.description="Variant-aware flanking-sequence extraction and masking for ddPCR assay design" \
+      org.opencontainers.image.licenses="Apache-2.0"
+
+# pysam ships manylinux wheels with htslib bundled, so no system htslib needed.
+COPY --from=builder /dist/*.whl /tmp/
+RUN pip install --no-cache-dir /tmp/*.whl && rm -rf /tmp/*.whl
+
+# Reference FASTAs / VCFs are mounted at runtime, e.g.:
+#   docker run --rm -v "$PWD:/data" ghcr.io/rhshah/vflank \
+#       small run /data/variants.maf -r /data/GRCh37.fasta -g hg19 -o /data/out.fasta
+ENTRYPOINT ["vflank"]
+CMD ["--help"]