mafutils 0.2.0__tar.gz → 0.3.0__tar.gz

mafutils-0.3.0/.github/workflows/tests.yml

@@ -0,0 +1,28 @@
+name: Tests
+
+on:
+  pull_request:
+    branches: [main]
+  push:
+    branches: [main]
+
+jobs:
+  test:
+    name: pytest
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out source
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install package and pytest
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e . pytest
+
+      - name: Run tests
+        run: pytest tests/ -v

{mafutils-0.2.0 → mafutils-0.3.0}/.gitignore

@@ -9,4 +9,7 @@ data/
 # Regenerable benchmark pipeline output.
 benchmarks/results/
 benchmarks/summary.html
-benchmarks/.snakemake/
+benchmarks/.snakemake/
+
+# Stray runtime log mafutils fetch writes to its output dir by default.
+maf_fetch.log

{mafutils-0.2.0 → mafutils-0.3.0}/DEVELOPMENT.md

@@ -1,5 +1,9 @@
 # mafutils Development Notes
 
+Internal implementation notes, gotchas, and maintainer workflows (testing,
+releasing) for developing `mafutils` itself — see [`README.md`](README.md)
+for user-facing installation/usage docs.
+
 - `tests/example.maf.scaffold.idx` is preserved as an older, headerless
   scaffold-index fixture for comparison — it deliberately sits at
   `example.maf`'s *default* scaffold-index path
@@ -155,6 +159,53 @@
   backward-seek risk entirely rather than relying on any per-region
   processing order.
 
+## Development setup
+
+From a source checkout, an editable install picks up code changes without
+reinstalling:
+
+```bash
+pip install -e .
+```
+
+Or run directly against the checkout without installing at all:
+
+```bash
+python -m mafutils --help
+```
+
+## Testing
+
+From inside this `mafutils/` directory:
+
+```bash
+pytest tests/
+```
+
+`tests/test_compression.py` covers cross-cutting compression behavior
+(bgzip/gzip detection, index headers, index auto-derivation, and the
+per-command parallel/sequential/fallback logic) across all three compression
+types. `tests/test_validate.py` covers index integrity (size/mtime/hash
+header fields, `mafutils validate`'s three-way verdict, and `--verify-hash`
+on `fetch`/`stats`/`gc`). `tests/test_stats.py` checks `mafutils stats`'s
+computed `overall.tsv`/`species.tsv` values against numbers hand-derived
+directly from `tests/example.maf`'s raw content (not just "doesn't crash"),
+including a regression guard that the sequential (`-p 1`, no
+`ProcessPoolExecutor`) and real multi-worker (`-p 2`+) paths produce
+identical output.
+
+`tests/test_real_data.py` runs `stats`/`gc`/`fetch` against
+`tests/real-excerpt.maf` -- 8 real, complete alignment blocks extracted
+read-only from gwct's actual ~42GB production MAF
+(`data/hamsters/uncompressed/...`), not hand-crafted. Unlike
+`example.maf`-based tests (which verify exact hand-computed values),
+these check plausibility/robustness on genuinely real data -- real
+species-naming conventions, real gap patterns, real block-size
+distribution -- since a hand-crafted fixture wouldn't think to include
+whatever real data actually looks like. This is deliberately still a
+tiny, committed fixture, not the real dataset itself: `data/hamsters/`
+stays untouched and out of git (see the `.gitignore` `data/` entry).
+
 ## Releasing to PyPI
 
 - Publishing is triggered by pushing a Git tag that matches `v*` (for example

{mafutils-0.2.0/mafutils.egg-info → mafutils-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mafutils
-Version: 0.2.0
+Version: 0.3.0
 Summary: Utilities for indexing, fetching, and summarizing MAFs.
 Author: Gregg Thomas
 License: MIT License
@@ -50,7 +50,7 @@ It currently provides five commands:
 
 ## Disclaimer
 
-This project was developed with significant assistance from a large language model (GPT-5 / Codex).
+This project was developed with significant assistance from large language models (GPT-5 / Codex, Claude Sonnet).
 
 ## Third-Party Code
 
@@ -61,69 +61,62 @@ for full attribution and license text.
 
 ## Installation
 
-From this `mafutils/` directory:
-
 ```bash
-pip install -e .
+pip install mafutils
 ```
 
-If you are using the current shared workflow environment instead of a separate
-package environment, you can also run it directly with:
-
-```bash
-python -m mafutils --help
-```
+This installs the `mafutils` command.
 
 ## Quick Start
 
 Show top-level help:
 
 ```bash
-python -m mafutils --help
+mafutils --help
 ```
 
 Create block and scaffold indexes for a MAF (defaults to
 `input.maf.block.idx` / `input.maf.scaffold.idx` if output paths are omitted):
 
 ```bash
-python -m mafutils index input.maf
+mafutils index input.maf
 ```
 
 Fetch trimmed MAF regions from a BED file (index defaults to
 `input.maf.block.idx`):
 
 ```bash
-python -m mafutils fetch input.maf regions.bed -o outdir
+mafutils fetch input.maf regions.bed -o outdir
 ```
 
 Fetch FASTA output instead of MAF:
 
 ```bash
-python -m mafutils fetch input.maf regions.bed -o outdir -f -fh species-coords-id
+mafutils fetch input.maf regions.bed -o outdir -f -fh species-coords-id
 ```
 
 Extract full scaffolds using a scaffold index:
 
 ```bash
-python -m mafutils fetch input.maf scaffolds.bed -m scaffold -o outdir
+mafutils fetch input.maf scaffolds.bed -m scaffold -o outdir
 ```
 
 Summarize an indexed MAF:
 
 ```bash
-python -m mafutils stats input.maf -o summary/example
+mafutils stats input.maf -o summary/example
 ```
 
 Calculate per-species GC content:
 
 ```bash
-python -m mafutils gc input.maf -o summary/example
+mafutils gc input.maf -o summary/example
 ```
 
 Check whether an index is still trustworthy (see Index Integrity below):
 
 ```bash
-python -m mafutils validate input.maf
+mafutils validate input.maf
 ```
 
 ## Compression
@@ -139,13 +132,10 @@ python -m mafutils validate input.maf
 If you plan to use `--processes > 1` against compressed input, compress with
 `bgzip` rather than plain `gzip` to get real parallel speedup.
 
-**No extra dependency for bgzip:** BGZF support is provided by
-`mafutils/lib/bgzf.py`, vendored from [Biopython](https://biopython.org/)
-(see the notice at the top of that file and `THIRD_PARTY_LICENSES.md`) rather
-than by depending on the `biopython` package, which would otherwise pull in
-`numpy` for no other reason. This does mean upstream bug fixes to that file
-aren't picked up automatically — see `DEVELOPMENT.md` if you're debugging a
-bgzip-related issue.
+**No extra dependency for bgzip:** BGZF support is vendored in
+`mafutils/lib/bgzf.py` rather than depending on the `biopython` package —
+see `DEVELOPMENT.md` for why, and [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md)
+for attribution.
 
 ## Index Integrity
 
@@ -184,7 +174,7 @@ For a "check once, trust thereafter" workflow instead of passing
 Create block and scaffold indexes for a MAF file.
 
 ```bash
-python -m mafutils index MAF_FILE [BLOCK_INDEX] [SCAFFOLD_INDEX]
+mafutils index MAF_FILE [BLOCK_INDEX] [SCAFFOLD_INDEX]
 ```
 
 Arguments:
@@ -200,7 +190,7 @@ Arguments:
 Fetch regions or scaffolds from a MAF using an existing index.
 
 ```bash
-python -m mafutils fetch [OPTIONS] MAF_FILE BED_FILE
+mafutils fetch [OPTIONS] MAF_FILE BED_FILE
 ```
 
 Arguments:
@@ -224,6 +214,7 @@ Options:
 | `--fasta-dedupe` | FASTA duplicate handling: `none` or `most-seq` |
 | `--processes`, `-p` | Number of worker processes (see Compression above — plain gzip always runs single-process) |
 | `--mode`, `-m` | Fetch mode: `block` or `scaffold` |
+| `--scaffold-subdirs` | Group output files into subfolders named by reference scaffold (`<output>/<scaffold>/<basename>`) instead of one flat directory |
 | `--verbose` | Emit warning lines from each completed batch |
 | `--profile` | Log internal timing breakdowns |
 | `--verify-hash` | Verify the index's stored content hash against the MAF file (see Index Integrity above) |
@@ -239,7 +230,7 @@ file) order, and decodes shared blocks only once even when regions overlap.
 Summarize an indexed MAF at overall, species, and block levels.
 
 ```bash
-python -m mafutils stats [OPTIONS] MAF_FILE [INDEX_FILE]
+mafutils stats [OPTIONS] MAF_FILE [INDEX_FILE]
 ```
 
 Arguments:
@@ -276,7 +267,7 @@ GC is computed as `(G+C) / (A+C+G+T)`, case-insensitive; gaps, `N`, and other
 ambiguity codes are excluded from both the numerator and denominator.
 
 ```bash
-python -m mafutils gc MAF_FILE [INDEX_FILE] [OPTIONS]
+mafutils gc MAF_FILE [INDEX_FILE] [OPTIONS]
 ```
 
 Arguments:
@@ -329,7 +320,7 @@ two indexes weren't built together (e.g. only one was rebuilt) and
 shouldn't be trusted as a matched pair.
 
 ```bash
-python -m mafutils validate MAF_FILE [INDEX_FILE]
+mafutils validate MAF_FILE [INDEX_FILE]
 ```
 
 Arguments:
@@ -349,23 +340,7 @@ verdict. Exit codes:
 | `1` | MISMATCH | A conclusive difference was found (compression, size, or hash against the MAF file; or the block and scaffold index headers disagree) — rebuild the index(es). |
 | `2` | UNVERIFIABLE | Nothing contradicts, but there's no stored hash to be fully sure (the index predates this feature), or the scaffold index is missing/headerless so the pair couldn't be cross-checked. |
 
-## Testing
-
-From inside this `mafutils/` directory:
-
-```bash
-pytest tests/
-```
-
-`tests/test_compression.py` covers cross-cutting compression behavior
-(bgzip/gzip detection, index headers, index auto-derivation, and the
-per-command parallel/sequential/fallback logic described above) across all
-three compression types. `tests/test_validate.py` covers index integrity
-(size/mtime/hash header fields, `mafutils validate`'s three-way verdict, and
-`--verify-hash` on `fetch`/`stats`/`gc`).
-
 ## Notes
 
-- `python -m mafutils ...` is the supported invocation from a source checkout.
-- The installed console entrypoint is `mafutils ...` once the package is
-  installed.
+For development setup (installing from a source checkout), running the test
+suite, and internal implementation notes, see [`DEVELOPMENT.md`](DEVELOPMENT.md).

{mafutils-0.2.0 → mafutils-0.3.0}/README.md

@@ -13,7 +13,7 @@ It currently provides five commands:
 
 ## Disclaimer
 
-This project was developed with significant assistance from a large language model (GPT-5 / Codex).
+This project was developed with significant assistance from large language models (GPT-5 / Codex, Claude Sonnet).
 
 ## Third-Party Code
 
@@ -24,69 +24,62 @@ for full attribution and license text.
 
 ## Installation
 
-From this `mafutils/` directory:
-
 ```bash
-pip install -e .
+pip install mafutils
 ```
 
-If you are using the current shared workflow environment instead of a separate
-package environment, you can also run it directly with:
-
-```bash
-python -m mafutils --help
-```
+This installs the `mafutils` command.
 
 ## Quick Start
 
 Show top-level help:
 
 ```bash
-python -m mafutils --help
+mafutils --help
 ```
 
 Create block and scaffold indexes for a MAF (defaults to
 `input.maf.block.idx` / `input.maf.scaffold.idx` if output paths are omitted):
 
 ```bash
-python -m mafutils index input.maf
+mafutils index input.maf
 ```
 
 Fetch trimmed MAF regions from a BED file (index defaults to
 `input.maf.block.idx`):
 
 ```bash
-python -m mafutils fetch input.maf regions.bed -o outdir
+mafutils fetch input.maf regions.bed -o outdir
 ```
 
 Fetch FASTA output instead of MAF:
 
 ```bash
-python -m mafutils fetch input.maf regions.bed -o outdir -f -fh species-coords-id
+mafutils fetch input.maf regions.bed -o outdir -f -fh species-coords-id
 ```
 
 Extract full scaffolds using a scaffold index:
 
 ```bash
-python -m mafutils fetch input.maf scaffolds.bed -m scaffold -o outdir
+mafutils fetch input.maf scaffolds.bed -m scaffold -o outdir
 ```
 
 Summarize an indexed MAF:
 
 ```bash
-python -m mafutils stats input.maf -o summary/example
+mafutils stats input.maf -o summary/example
 ```
 
 Calculate per-species GC content:
 
 ```bash
-python -m mafutils gc input.maf -o summary/example
+mafutils gc input.maf -o summary/example
 ```
 
 Check whether an index is still trustworthy (see Index Integrity below):
 
 ```bash
-python -m mafutils validate input.maf
+mafutils validate input.maf
 ```
 
 ## Compression
@@ -102,13 +95,10 @@ python -m mafutils validate input.maf
 If you plan to use `--processes > 1` against compressed input, compress with
 `bgzip` rather than plain `gzip` to get real parallel speedup.
 
-**No extra dependency for bgzip:** BGZF support is provided by
-`mafutils/lib/bgzf.py`, vendored from [Biopython](https://biopython.org/)
-(see the notice at the top of that file and `THIRD_PARTY_LICENSES.md`) rather
-than by depending on the `biopython` package, which would otherwise pull in
-`numpy` for no other reason. This does mean upstream bug fixes to that file
-aren't picked up automatically — see `DEVELOPMENT.md` if you're debugging a
-bgzip-related issue.
+**No extra dependency for bgzip:** BGZF support is vendored in
+`mafutils/lib/bgzf.py` rather than depending on the `biopython` package —
+see `DEVELOPMENT.md` for why, and [`THIRD_PARTY_LICENSES.md`](THIRD_PARTY_LICENSES.md)
+for attribution.
 
 ## Index Integrity
 
@@ -147,7 +137,7 @@ For a "check once, trust thereafter" workflow instead of passing
 Create block and scaffold indexes for a MAF file.
 
 ```bash
-python -m mafutils index MAF_FILE [BLOCK_INDEX] [SCAFFOLD_INDEX]
+mafutils index MAF_FILE [BLOCK_INDEX] [SCAFFOLD_INDEX]
 ```
 
 Arguments:
@@ -163,7 +153,7 @@ Arguments:
 Fetch regions or scaffolds from a MAF using an existing index.
 
 ```bash
-python -m mafutils fetch [OPTIONS] MAF_FILE BED_FILE
+mafutils fetch [OPTIONS] MAF_FILE BED_FILE
 ```
 
 Arguments:
@@ -187,6 +177,7 @@ Options:
 | `--fasta-dedupe` | FASTA duplicate handling: `none` or `most-seq` |
 | `--processes`, `-p` | Number of worker processes (see Compression above — plain gzip always runs single-process) |
 | `--mode`, `-m` | Fetch mode: `block` or `scaffold` |
+| `--scaffold-subdirs` | Group output files into subfolders named by reference scaffold (`<output>/<scaffold>/<basename>`) instead of one flat directory |
 | `--verbose` | Emit warning lines from each completed batch |
 | `--profile` | Log internal timing breakdowns |
 | `--verify-hash` | Verify the index's stored content hash against the MAF file (see Index Integrity above) |
@@ -202,7 +193,7 @@ file) order, and decodes shared blocks only once even when regions overlap.
 Summarize an indexed MAF at overall, species, and block levels.
 
 ```bash
-python -m mafutils stats [OPTIONS] MAF_FILE [INDEX_FILE]
+mafutils stats [OPTIONS] MAF_FILE [INDEX_FILE]
 ```
 
 Arguments:
@@ -239,7 +230,7 @@ GC is computed as `(G+C) / (A+C+G+T)`, case-insensitive; gaps, `N`, and other
 ambiguity codes are excluded from both the numerator and denominator.
 
 ```bash
-python -m mafutils gc MAF_FILE [INDEX_FILE] [OPTIONS]
+mafutils gc MAF_FILE [INDEX_FILE] [OPTIONS]
 ```
 
 Arguments:
@@ -292,7 +283,7 @@ two indexes weren't built together (e.g. only one was rebuilt) and
 shouldn't be trusted as a matched pair.
 
 ```bash
-python -m mafutils validate MAF_FILE [INDEX_FILE]
+mafutils validate MAF_FILE [INDEX_FILE]
 ```
 
 Arguments:
@@ -312,23 +303,7 @@ verdict. Exit codes:
 | `1` | MISMATCH | A conclusive difference was found (compression, size, or hash against the MAF file; or the block and scaffold index headers disagree) — rebuild the index(es). |
 | `2` | UNVERIFIABLE | Nothing contradicts, but there's no stored hash to be fully sure (the index predates this feature), or the scaffold index is missing/headerless so the pair couldn't be cross-checked. |
 
-## Testing
-
-From inside this `mafutils/` directory:
-
-```bash
-pytest tests/
-```
-
-`tests/test_compression.py` covers cross-cutting compression behavior
-(bgzip/gzip detection, index headers, index auto-derivation, and the
-per-command parallel/sequential/fallback logic described above) across all
-three compression types. `tests/test_validate.py` covers index integrity
-(size/mtime/hash header fields, `mafutils validate`'s three-way verdict, and
-`--verify-hash` on `fetch`/`stats`/`gc`).
-
 ## Notes
 
-- `python -m mafutils ...` is the supported invocation from a source checkout.
-- The installed console entrypoint is `mafutils ...` once the package is
-  installed.
+For development setup (installing from a source checkout), running the test
+suite, and internal implementation notes, see [`DEVELOPMENT.md`](DEVELOPMENT.md).

{mafutils-0.2.0 → mafutils-0.3.0}/mafutils/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.2.0'
-__version_tuple__ = version_tuple = (0, 2, 0)
+__version__ = version = '0.3.0'
+__version_tuple__ = version_tuple = (0, 3, 0)
 
-__commit_id__ = commit_id = 'g5d2163b7c'
+__commit_id__ = commit_id = 'g694487dcd'