peakatail 0.3.0__tar.gz

peakatail-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 BMGLab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

peakatail-0.3.0/PKG-INFO

@@ -0,0 +1,337 @@
+Metadata-Version: 2.4
+Name: peakatail
+Version: 0.3.0
+Summary: PeakATail — single-cell poly(A) site detection and APA analysis
+Author-email: Amir Amiri Tabat <amiramiritabat01@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/BMGLab/PeakATail
+Project-URL: Repository, https://github.com/BMGLab/PeakATail
+Keywords: bioinformatics,scRNA-seq,polyadenylation,APA
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pysam>=0.22
+Requires-Dist: anndata>=0.10
+Requires-Dist: scanpy>=1.10
+Requires-Dist: pandas>=2.1
+Requires-Dist: scipy>=1.11
+Requires-Dist: numpy>=1.26
+Requires-Dist: igraph>=0.11
+Requires-Dist: leidenalg>=0.10
+Requires-Dist: louvain>=0.8.0
+Requires-Dist: pybedtools>=0.10
+Requires-Dist: psutil>=5.9
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: joblib>=1.3
+Requires-Dist: statsmodels>=0.14
+Requires-Dist: sortedcontainers>=2.4
+Requires-Dist: pyarrow>=15.0
+Requires-Dist: tqdm>=4.66
+Requires-Dist: click>=8.1
+Requires-Dist: rich>=13.7
+Requires-Dist: questionary>=2.0
+Requires-Dist: matplotlib>=3.8
+Requires-Dist: seaborn>=0.13
+Requires-Dist: plotly>=5.18
+Requires-Dist: kaleido>=0.2.1
+Requires-Dist: upsetplot>=0.9
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-timeout>=2.4; extra == "test"
+Requires-Dist: pyfaidx>=0.7; extra == "test"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: pytest-timeout>=2.4; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Requires-Dist: mypy>=1.10; extra == "dev"
+Provides-Extra: all
+Requires-Dist: peakatail[dev]; extra == "all"
+Dynamic: license-file
+
+# PeakATail
+
+<p align="center">
+    <img src="https://raw.githubusercontent.com/BMGLab/PeakATail/develop/docs/assets/logos/main_logo.png" alt="PeakATail — coiled snake with poly(A) tail" width="640">
+</p>
+
+PeakATail is a Python tool for single-cell poly(A) site (PAS) detection and alternative polyadenylation (APA) analysis. It works with any scRNA-seq BAM that carries a `CB:Z` (corrected cell barcode) tag — STARsolo, CellRanger, Alevin-fry, or any aligner that emits the standard 10x-style tag schema. UMI (`UB:Z`) tags are NOT required or used: PeakATail counts raw read 3'ends, not UMI-deduplicated molecules. PeakATail does NOT correct barcodes; your aligner must apply a barcode whitelist (e.g. STARsolo's `--soloCBwhitelist`). From the input BAM it calls polyadenylation sites at the read level, builds a per-cell PAS count matrix, clusters cells by their APA profiles, and provides downstream analyses including differential APA testing between clusters, 3'UTR length quantification, and cross-dataset cluster matching. Developed at BMGLab.
+
+[![Docs](https://img.shields.io/badge/docs-bmglab.github.io%2FPeakATail-blue)](https://bmglab.github.io/PeakATail/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](pyproject.toml)
+
+<!-- TODO[verify]: A CI workflow badge is not included because the repository currently contains only a docs-deploy workflow (.github/workflows/docs.yml). Add a badge once a CI workflow (e.g. .github/workflows/ci.yml) is committed. -->
+
+---
+
+## Quick install
+
+### System prerequisites
+
+PeakATail calls `samtools` and `bedtools` as subprocesses. Both must be on your `PATH` before running the pipeline.
+
+```bash
+# Debian/Ubuntu
+sudo apt-get install samtools bedtools
+
+# macOS (Homebrew)
+brew install samtools bedtools
+```
+
+The code has been tested against samtools >= 1.10 (enforced at runtime in `ema/datasets/manager.py`).
+
+### Python version
+
+Python **3.11 or later** is required (declared in `pyproject.toml` as `requires-python = ">=3.11"`).
+
+### Install from source
+
+```bash
+git clone https://github.com/BMGLab/PeakATail.git
+cd PeakATail
+
+# Create and activate a virtual environment
+python3.11 -m venv .venv
+source .venv/bin/activate
+
+# Install with uv (recommended — matches the development environment)
+pip install uv
+uv pip install -e .
+
+# Or with plain pip
+pip install -e .
+```
+
+All Python dependencies are declared in `pyproject.toml` and installed automatically.
+
+---
+
+## Quick start
+
+```bash
+# Step 1: copy the example config and edit paths
+cp example.yaml my_run.yaml
+# Edit my_run.yaml — set datasets[].bams, gtf, seqlen, cb_len, barcode_tag
+
+# Step 2: run the full pipeline (peak calling → annotation → clustering)
+uv run peakatail run --config my_run.yaml --threads 4
+
+# Step 3: differential APA between every cluster pair
+uv run peakatail switch diff \
+    -i peakatail_runs/<run>/per_dataset/<ds>/clusters.h5ad \
+    --pasbed peakatail_runs/<run>/per_dataset/<ds>/pasbed.bed \
+    --strategy fisher
+
+# Step 4: 3'UTR length quantification across clusters
+uv run peakatail switch length \
+    -i peakatail_runs/<run>/per_dataset/<ds>/clusters.h5ad \
+    --strategy classic
+```
+
+The pipeline writes all output under `peakatail_runs/<name>_<timestamp>/`. A typical run directory looks like:
+
+```
+peakatail_runs/emaout_20240501_143022/
+    run_config.json                  # full resolved parameters
+    peakatail_<ts>.log               # structured run log
+    per_dataset/
+        <dataset_id>/
+            raw/                     # pre-filter BEDs and matrices
+            posbed.bed               # filtered positive-strand PAS BED
+            negbed.bed               # filtered negative-strand PAS BED
+            pasbed.bed               # combined filtered PAS BED (both strands)
+            filtered_cb.tsv          # barcodes that passed min_read filter
+            pas_gene.tsv             # PAS-to-gene mapping table
+            annotatedpas.bed         # pasbed.bed extended with gene_id column
+            annotated_matrix.mtx     # post-annotation count matrix (MatrixMarket)
+            annotated_pas_ids.tsv    # row index for annotated_matrix.mtx
+            annotated_cells.tsv      # column index for annotated_matrix.mtx
+            preprocessed.h5ad        # filtered AnnData before clustering
+            clusters.h5ad            # AnnData with leiden cluster labels
+    switch_diff_<ts>/                # output of `peakatail switch diff` (auto-routed)
+    switch_length_<ts>/              # output of `peakatail switch length` (auto-routed)
+    switch_match_<ts>/               # output of `peakatail switch match` (auto-routed)
+    switch_geneview_<ts>/            # output of `peakatail switch geneview` (auto-routed)
+```
+
+---
+
+## What PeakATail produces
+
+### Peak calling stage
+
+| File | Location | Description |
+|---|---|---|
+| `raw/pos.bed` | `per_dataset/<ds>/raw/` | Unfiltered positive-strand PAS calls (concatenated across BAM replicates) |
+| `raw/neg.bed` | `per_dataset/<ds>/raw/` | Unfiltered negative-strand PAS calls |
+| `raw/pas.bed` | `per_dataset/<ds>/raw/` | Union of raw pos + neg BED (pre-filter) |
+| `raw/pos.mtx` | `per_dataset/<ds>/raw/` | Raw count matrix, positive strand (MatrixMarket) |
+| `raw/neg.mtx` | `per_dataset/<ds>/raw/` | Raw count matrix, negative strand (MatrixMarket) |
+| `raw/cb.tsv` | `per_dataset/<ds>/raw/` | Raw cell-barcode index aligned to raw MTX columns |
+| `posbed.bed` | `per_dataset/<ds>/` | Filtered positive-strand PAS BED |
+| `negbed.bed` | `per_dataset/<ds>/` | Filtered negative-strand PAS BED |
+| `pasbed.bed` | `per_dataset/<ds>/` | Combined filtered PAS BED (both strands); used as input for downstream commands |
+| `filtered_cb.tsv` | `per_dataset/<ds>/` | Barcodes that passed the `min_read` filter, with filter threshold in header |
+
+### Annotation stage
+
+| File | Location | Description |
+|---|---|---|
+| `pas_gene.tsv` | `per_dataset/<ds>/` | Two-column table: `pas_id`, `gene_id` |
+| `annotatedpas.bed` | `per_dataset/<ds>/` | `pasbed.bed` extended with a trailing `gene_id` column |
+| `annotated_matrix.mtx` | `per_dataset/<ds>/` | MatrixMarket sparse count matrix (rows = annotated PAS, cols = cells) |
+| `annotated_pas_ids.tsv` | `per_dataset/<ds>/` | Row index for `annotated_matrix.mtx` |
+| `annotated_cells.tsv` | `per_dataset/<ds>/` | Column index for `annotated_matrix.mtx` |
+
+### Clustering stage
+
+| File | Location | Description |
+|---|---|---|
+| `preprocessed.h5ad` | `per_dataset/<ds>/` | AnnData after cell/PAS filtering, before cluster labels are assigned |
+| `clusters.h5ad` | `per_dataset/<ds>/` | AnnData with `leiden` cluster labels in `.obs`; primary input for all `switch` subcommands |
+
+### Differential APA stage (`peakatail switch diff`)
+
+| File | Location | Description |
+|---|---|---|
+| `diff_<c1>_vs_<c2>.tsv` | `switch_diff_<ts>/` | Per-cluster-pair differential APA results table |
+| `cluster_match.tsv` | `switch_match_<ts>/` | Cross-dataset cluster correspondence scores (`peakatail switch match`) |
+| `pdui_classic.tsv` | `switch_length_<ts>/` | Per-cell PDUI scores when strategy is `classic` |
+| `proportion.tsv` | `switch_length_<ts>/` | Per-cell per-PAS proportion scores when strategy is `proportion` |
+| `entropy_shannon.tsv` | `switch_length_<ts>/` | Per-cell Shannon entropy scores when strategy is `shannon` |
+
+---
+
+## Available CLI commands
+
+All commands are accessed through the `peakatail` entry point installed by `pip install -e .`. The old `ema` command still works as a deprecated alias.
+
+| Command | Description | Docs |
+|---|---|---|
+| `peakatail run` | Run the full pipeline: peak calling, annotation, clustering | [cli/run](https://bmglab.github.io/PeakATail/cli/run/) |
+| `peakatail reannotate` | Branch a finished run into a new trim/filter/clustering variant without re-peak-calling | [cli/reannotate](https://bmglab.github.io/PeakATail/cli/reannotate/) |
+| `peakatail switch diff` | Differential APA test across cluster pairs (Fisher / NB regression) | [cli/switch-diff](https://bmglab.github.io/PeakATail/cli/switch-diff/) |
+| `peakatail switch length` | 3'UTR shortening/lengthening quantification (PDUI variants) | [cli/switch-length](https://bmglab.github.io/PeakATail/cli/switch-length/) |
+| `peakatail switch trend` | Ordered-covariate (e.g. stage-progression) APA-length trend: slope + Spearman + direction | [cli](https://bmglab.github.io/PeakATail/cli/) |
+| `peakatail switch combine` | Stitch stage/celltype-labelled `clusters.h5ad` files into one grouped h5ad for cross-group testing | [cli](https://bmglab.github.io/PeakATail/cli/) |
+| `peakatail switch match` | Cross-dataset cluster matching | [cli/switch-match](https://bmglab.github.io/PeakATail/cli/switch-match/) |
+| `peakatail switch geneview` | Gene-track visualisation: per-cluster PAS coverage and proportions | [cli/switch-geneview](https://bmglab.github.io/PeakATail/cli/switch-geneview/) |
+| `peakatail collapse` | Pool `samtools merge` RG-suffixed run tags back into per-library cells | [cli](https://bmglab.github.io/PeakATail/cli/) |
+| `peakatail merge` | Merge multiple BAM files into one sorted and indexed BAM | [cli/merge](https://bmglab.github.io/PeakATail/cli/merge/) |
+| `peakatail parse-gtf` | Pre-warm the GTF cache so subsequent runs start immediately | [cli/parse-gtf](https://bmglab.github.io/PeakATail/cli/parse-gtf/) |
+| `peakatail wizard` | Interactive setup wizard (also invoked by bare `ema`) | [cli/wizard](https://bmglab.github.io/PeakATail/cli/wizard/) |
+
+Use `--list-strategies` on `peakatail run`, `peakatail switch diff`, `peakatail switch length`, and `peakatail switch match` to see the strategies registered in the current installation.
+
+Every command accepts `--help` for full flag documentation.
+
+---
+
+## Available strategies
+
+### Peak calling (`peakatail run --peak-strategy`)
+
+| Name | Description |
+|---|---|
+| `original` | Wraps the original `pasfind()` logic unchanged; used as a baseline for comparison |
+| `lambda_poisson` | MACS2-style: Poisson p-value against a local lambda estimated from floor positions |
+| `sierra_iterative` | Sierra-style iterative peak subtraction; finds multiple PAS per UTR |
+| `lambda_gradient` | Local-lambda estimation combined with gradient-based peak delineation |
+
+### Clustering (`peakatail run --cluster-method`)
+
+| Name | Description |
+|---|---|
+| `leiden_tfidf` | TF-IDF normalisation followed by LSI dimensionality reduction and Leiden community detection |
+| `leiden_libsize` | Library-size normalisation followed by PCA and Leiden community detection |
+| `external` | Load pre-computed cluster labels from a file instead of running clustering |
+
+### Cross-dataset cluster matching (`peakatail switch match --strategy` / `peakatail run --match-method`)
+
+| Name | Description |
+|---|---|
+| `marker_overlap` | Match clusters by overlap of top marker PAS sets |
+| `jaccard` | Match clusters by Jaccard similarity of cell sets |
+| `mnn` | Mutual nearest neighbours in a shared LSI embedding |
+
+### 3'UTR length quantification (`peakatail switch length --strategy`)
+
+| Name | Output file | Description |
+|---|---|---|
+| `classic` | `pdui_classic.tsv` | Classic 2-PAS PDUI: distal_count / (proximal + distal); proximal/distal selected by genomic or transcript-coordinate rank |
+| `proportion` | `proportion.tsv` | Per-PAS proportion vector: reads per PAS as fraction of gene total per cell |
+| `shannon` | `entropy_shannon.tsv` | Shannon entropy of the per-PAS proportion distribution per (gene, cell) |
+
+### Differential APA testing (`peakatail switch diff --strategy`)
+
+| Name | Description |
+|---|---|
+| `fisher` | Fisher's exact test on per-PAS read counts between two cluster groups |
+| `nb_pairwise` | Negative binomial regression, pairwise cluster comparison |
+| `nb_multi` | Negative binomial regression, multi-condition |
+
+---
+
+## Documentation
+
+Full docs: **https://bmglab.github.io/PeakATail/**
+
+The MkDocs documentation site is built automatically from the `develop` and `main` branches via the `.github/workflows/docs.yml` GitHub Actions workflow and published to GitHub Pages.
+
+---
+
+## Repository layout
+
+```
+PeakATail/
+    ema/                    # core Python package (entry point: ema.cli:main)
+        cli/                # Click subcommands and config schema
+        strategies/         # peak-calling strategy registry
+        clustering/         # clustering strategies and cross-dataset matching
+        quantification/     # PDUI / proportion / entropy strategies
+        switch_test/        # differential APA testing strategies
+        outputs.py          # all file I/O for the pipeline
+        main.py             # pipeline orchestration
+        downstream_runner.py# per-dataset worker (safe for multiprocessing)
+    data/                   # example data files
+    test/                   # test suite (pytest)
+    other_repos/            # reference implementations (Sierra, SCAPE, scTail, etc.)
+    example.yaml            # canonical YAML config template
+    combined_polya_scrna_methods.csv  # comparison of poly(A) scRNA methods
+    pyproject.toml          # package metadata, dependencies, entry points
+    ROADMAP.md              # development and publication roadmap
+    LICENSE                 # MIT License
+    Dockerfile              # container build
+```
+
+---
+
+## Citation
+
+If you use PeakATail, please cite the software entry below for now. The `author`
+field mirrors the package author declared in `pyproject.toml`; the full
+manuscript author list and a machine-readable `CITATION.cff` will be added with
+the paper release (tracked in the release-engineering issue).
+
+<!-- TODO[verify]: No CITATION.cff file exists yet. Replace this software entry
+with the published journal citation — and expand `author` to the full manuscript
+author list — once the paper is out. -->
+
+```bibtex
+@software{peakatail,
+  author  = {Amiri Tabat, Amir},
+  title   = {{PeakATail}: single-cell poly(A) site detection and APA analysis},
+  url     = {https://github.com/BMGLab/PeakATail},
+  version = {0.3.0},
+  note    = {Preprint in preparation; replace with the journal citation when available}
+}
+```
+
+---
+
+## License
+
+This project is licensed under the **MIT License** — see the [LICENSE](LICENSE) file for the full text. `pyproject.toml` declares the matching SPDX identifier (`license = "MIT"`).

peakatail-0.3.0/README.md

@@ -0,0 +1,284 @@
+# PeakATail
+
+<p align="center">
+    <img src="https://raw.githubusercontent.com/BMGLab/PeakATail/develop/docs/assets/logos/main_logo.png" alt="PeakATail — coiled snake with poly(A) tail" width="640">
+</p>
+
+PeakATail is a Python tool for single-cell poly(A) site (PAS) detection and alternative polyadenylation (APA) analysis. It works with any scRNA-seq BAM that carries a `CB:Z` (corrected cell barcode) tag — STARsolo, CellRanger, Alevin-fry, or any aligner that emits the standard 10x-style tag schema. UMI (`UB:Z`) tags are NOT required or used: PeakATail counts raw read 3'ends, not UMI-deduplicated molecules. PeakATail does NOT correct barcodes; your aligner must apply a barcode whitelist (e.g. STARsolo's `--soloCBwhitelist`). From the input BAM it calls polyadenylation sites at the read level, builds a per-cell PAS count matrix, clusters cells by their APA profiles, and provides downstream analyses including differential APA testing between clusters, 3'UTR length quantification, and cross-dataset cluster matching. Developed at BMGLab.
+
+[![Docs](https://img.shields.io/badge/docs-bmglab.github.io%2FPeakATail-blue)](https://bmglab.github.io/PeakATail/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](pyproject.toml)
+
+<!-- TODO[verify]: A CI workflow badge is not included because the repository currently contains only a docs-deploy workflow (.github/workflows/docs.yml). Add a badge once a CI workflow (e.g. .github/workflows/ci.yml) is committed. -->
+
+---
+
+## Quick install
+
+### System prerequisites
+
+PeakATail calls `samtools` and `bedtools` as subprocesses. Both must be on your `PATH` before running the pipeline.
+
+```bash
+# Debian/Ubuntu
+sudo apt-get install samtools bedtools
+
+# macOS (Homebrew)
+brew install samtools bedtools
+```
+
+The code has been tested against samtools >= 1.10 (enforced at runtime in `ema/datasets/manager.py`).
+
+### Python version
+
+Python **3.11 or later** is required (declared in `pyproject.toml` as `requires-python = ">=3.11"`).
+
+### Install from source
+
+```bash
+git clone https://github.com/BMGLab/PeakATail.git
+cd PeakATail
+
+# Create and activate a virtual environment
+python3.11 -m venv .venv
+source .venv/bin/activate
+
+# Install with uv (recommended — matches the development environment)
+pip install uv
+uv pip install -e .
+
+# Or with plain pip
+pip install -e .
+```
+
+All Python dependencies are declared in `pyproject.toml` and installed automatically.
+
+---
+
+## Quick start
+
+```bash
+# Step 1: copy the example config and edit paths
+cp example.yaml my_run.yaml
+# Edit my_run.yaml — set datasets[].bams, gtf, seqlen, cb_len, barcode_tag
+
+# Step 2: run the full pipeline (peak calling → annotation → clustering)
+uv run peakatail run --config my_run.yaml --threads 4
+
+# Step 3: differential APA between every cluster pair
+uv run peakatail switch diff \
+    -i peakatail_runs/<run>/per_dataset/<ds>/clusters.h5ad \
+    --pasbed peakatail_runs/<run>/per_dataset/<ds>/pasbed.bed \
+    --strategy fisher
+
+# Step 4: 3'UTR length quantification across clusters
+uv run peakatail switch length \
+    -i peakatail_runs/<run>/per_dataset/<ds>/clusters.h5ad \
+    --strategy classic
+```
+
+The pipeline writes all output under `peakatail_runs/<name>_<timestamp>/`. A typical run directory looks like:
+
+```
+peakatail_runs/emaout_20240501_143022/
+    run_config.json                  # full resolved parameters
+    peakatail_<ts>.log               # structured run log
+    per_dataset/
+        <dataset_id>/
+            raw/                     # pre-filter BEDs and matrices
+            posbed.bed               # filtered positive-strand PAS BED
+            negbed.bed               # filtered negative-strand PAS BED
+            pasbed.bed               # combined filtered PAS BED (both strands)
+            filtered_cb.tsv          # barcodes that passed min_read filter
+            pas_gene.tsv             # PAS-to-gene mapping table
+            annotatedpas.bed         # pasbed.bed extended with gene_id column
+            annotated_matrix.mtx     # post-annotation count matrix (MatrixMarket)
+            annotated_pas_ids.tsv    # row index for annotated_matrix.mtx
+            annotated_cells.tsv      # column index for annotated_matrix.mtx
+            preprocessed.h5ad        # filtered AnnData before clustering
+            clusters.h5ad            # AnnData with leiden cluster labels
+    switch_diff_<ts>/                # output of `peakatail switch diff` (auto-routed)
+    switch_length_<ts>/              # output of `peakatail switch length` (auto-routed)
+    switch_match_<ts>/               # output of `peakatail switch match` (auto-routed)
+    switch_geneview_<ts>/            # output of `peakatail switch geneview` (auto-routed)
+```
+
+---
+
+## What PeakATail produces
+
+### Peak calling stage
+
+| File | Location | Description |
+|---|---|---|
+| `raw/pos.bed` | `per_dataset/<ds>/raw/` | Unfiltered positive-strand PAS calls (concatenated across BAM replicates) |
+| `raw/neg.bed` | `per_dataset/<ds>/raw/` | Unfiltered negative-strand PAS calls |
+| `raw/pas.bed` | `per_dataset/<ds>/raw/` | Union of raw pos + neg BED (pre-filter) |
+| `raw/pos.mtx` | `per_dataset/<ds>/raw/` | Raw count matrix, positive strand (MatrixMarket) |
+| `raw/neg.mtx` | `per_dataset/<ds>/raw/` | Raw count matrix, negative strand (MatrixMarket) |
+| `raw/cb.tsv` | `per_dataset/<ds>/raw/` | Raw cell-barcode index aligned to raw MTX columns |
+| `posbed.bed` | `per_dataset/<ds>/` | Filtered positive-strand PAS BED |
+| `negbed.bed` | `per_dataset/<ds>/` | Filtered negative-strand PAS BED |
+| `pasbed.bed` | `per_dataset/<ds>/` | Combined filtered PAS BED (both strands); used as input for downstream commands |
+| `filtered_cb.tsv` | `per_dataset/<ds>/` | Barcodes that passed the `min_read` filter, with filter threshold in header |
+
+### Annotation stage
+
+| File | Location | Description |
+|---|---|---|
+| `pas_gene.tsv` | `per_dataset/<ds>/` | Two-column table: `pas_id`, `gene_id` |
+| `annotatedpas.bed` | `per_dataset/<ds>/` | `pasbed.bed` extended with a trailing `gene_id` column |
+| `annotated_matrix.mtx` | `per_dataset/<ds>/` | MatrixMarket sparse count matrix (rows = annotated PAS, cols = cells) |
+| `annotated_pas_ids.tsv` | `per_dataset/<ds>/` | Row index for `annotated_matrix.mtx` |
+| `annotated_cells.tsv` | `per_dataset/<ds>/` | Column index for `annotated_matrix.mtx` |
+
+### Clustering stage
+
+| File | Location | Description |
+|---|---|---|
+| `preprocessed.h5ad` | `per_dataset/<ds>/` | AnnData after cell/PAS filtering, before cluster labels are assigned |
+| `clusters.h5ad` | `per_dataset/<ds>/` | AnnData with `leiden` cluster labels in `.obs`; primary input for all `switch` subcommands |
+
+### Differential APA stage (`peakatail switch diff`)
+
+| File | Location | Description |
+|---|---|---|
+| `diff_<c1>_vs_<c2>.tsv` | `switch_diff_<ts>/` | Per-cluster-pair differential APA results table |
+| `cluster_match.tsv` | `switch_match_<ts>/` | Cross-dataset cluster correspondence scores (`peakatail switch match`) |
+| `pdui_classic.tsv` | `switch_length_<ts>/` | Per-cell PDUI scores when strategy is `classic` |
+| `proportion.tsv` | `switch_length_<ts>/` | Per-cell per-PAS proportion scores when strategy is `proportion` |
+| `entropy_shannon.tsv` | `switch_length_<ts>/` | Per-cell Shannon entropy scores when strategy is `shannon` |
+
+---
+
+## Available CLI commands
+
+All commands are accessed through the `peakatail` entry point installed by `pip install -e .`. The old `ema` command still works as a deprecated alias.
+
+| Command | Description | Docs |
+|---|---|---|
+| `peakatail run` | Run the full pipeline: peak calling, annotation, clustering | [cli/run](https://bmglab.github.io/PeakATail/cli/run/) |
+| `peakatail reannotate` | Branch a finished run into a new trim/filter/clustering variant without re-peak-calling | [cli/reannotate](https://bmglab.github.io/PeakATail/cli/reannotate/) |
+| `peakatail switch diff` | Differential APA test across cluster pairs (Fisher / NB regression) | [cli/switch-diff](https://bmglab.github.io/PeakATail/cli/switch-diff/) |
+| `peakatail switch length` | 3'UTR shortening/lengthening quantification (PDUI variants) | [cli/switch-length](https://bmglab.github.io/PeakATail/cli/switch-length/) |
+| `peakatail switch trend` | Ordered-covariate (e.g. stage-progression) APA-length trend: slope + Spearman + direction | [cli](https://bmglab.github.io/PeakATail/cli/) |
+| `peakatail switch combine` | Stitch stage/celltype-labelled `clusters.h5ad` files into one grouped h5ad for cross-group testing | [cli](https://bmglab.github.io/PeakATail/cli/) |
+| `peakatail switch match` | Cross-dataset cluster matching | [cli/switch-match](https://bmglab.github.io/PeakATail/cli/switch-match/) |
+| `peakatail switch geneview` | Gene-track visualisation: per-cluster PAS coverage and proportions | [cli/switch-geneview](https://bmglab.github.io/PeakATail/cli/switch-geneview/) |
+| `peakatail collapse` | Pool `samtools merge` RG-suffixed run tags back into per-library cells | [cli](https://bmglab.github.io/PeakATail/cli/) |
+| `peakatail merge` | Merge multiple BAM files into one sorted and indexed BAM | [cli/merge](https://bmglab.github.io/PeakATail/cli/merge/) |
+| `peakatail parse-gtf` | Pre-warm the GTF cache so subsequent runs start immediately | [cli/parse-gtf](https://bmglab.github.io/PeakATail/cli/parse-gtf/) |
+| `peakatail wizard` | Interactive setup wizard (also invoked by bare `ema`) | [cli/wizard](https://bmglab.github.io/PeakATail/cli/wizard/) |
+
+Use `--list-strategies` on `peakatail run`, `peakatail switch diff`, `peakatail switch length`, and `peakatail switch match` to see the strategies registered in the current installation.
+
+Every command accepts `--help` for full flag documentation.
+
+---
+
+## Available strategies
+
+### Peak calling (`peakatail run --peak-strategy`)
+
+| Name | Description |
+|---|---|
+| `original` | Wraps the original `pasfind()` logic unchanged; used as a baseline for comparison |
+| `lambda_poisson` | MACS2-style: Poisson p-value against a local lambda estimated from floor positions |
+| `sierra_iterative` | Sierra-style iterative peak subtraction; finds multiple PAS per UTR |
+| `lambda_gradient` | Local-lambda estimation combined with gradient-based peak delineation |
+
+### Clustering (`peakatail run --cluster-method`)
+
+| Name | Description |
+|---|---|
+| `leiden_tfidf` | TF-IDF normalisation followed by LSI dimensionality reduction and Leiden community detection |
+| `leiden_libsize` | Library-size normalisation followed by PCA and Leiden community detection |
+| `external` | Load pre-computed cluster labels from a file instead of running clustering |
+
+### Cross-dataset cluster matching (`peakatail switch match --strategy` / `peakatail run --match-method`)
+
+| Name | Description |
+|---|---|
+| `marker_overlap` | Match clusters by overlap of top marker PAS sets |
+| `jaccard` | Match clusters by Jaccard similarity of cell sets |
+| `mnn` | Mutual nearest neighbours in a shared LSI embedding |
+
+### 3'UTR length quantification (`peakatail switch length --strategy`)
+
+| Name | Output file | Description |
+|---|---|---|
+| `classic` | `pdui_classic.tsv` | Classic 2-PAS PDUI: distal_count / (proximal + distal); proximal/distal selected by genomic or transcript-coordinate rank |
+| `proportion` | `proportion.tsv` | Per-PAS proportion vector: reads per PAS as fraction of gene total per cell |
+| `shannon` | `entropy_shannon.tsv` | Shannon entropy of the per-PAS proportion distribution per (gene, cell) |
+
+### Differential APA testing (`peakatail switch diff --strategy`)
+
+| Name | Description |
+|---|---|
+| `fisher` | Fisher's exact test on per-PAS read counts between two cluster groups |
+| `nb_pairwise` | Negative binomial regression, pairwise cluster comparison |
+| `nb_multi` | Negative binomial regression, multi-condition |
+
+---
+
+## Documentation
+
+Full docs: **https://bmglab.github.io/PeakATail/**
+
+The MkDocs documentation site is built automatically from the `develop` and `main` branches via the `.github/workflows/docs.yml` GitHub Actions workflow and published to GitHub Pages.
+
+---
+
+## Repository layout
+
+```
+PeakATail/
+    ema/                    # core Python package (entry point: ema.cli:main)
+        cli/                # Click subcommands and config schema
+        strategies/         # peak-calling strategy registry
+        clustering/         # clustering strategies and cross-dataset matching
+        quantification/     # PDUI / proportion / entropy strategies
+        switch_test/        # differential APA testing strategies
+        outputs.py          # all file I/O for the pipeline
+        main.py             # pipeline orchestration
+        downstream_runner.py# per-dataset worker (safe for multiprocessing)
+    data/                   # example data files
+    test/                   # test suite (pytest)
+    other_repos/            # reference implementations (Sierra, SCAPE, scTail, etc.)
+    example.yaml            # canonical YAML config template
+    combined_polya_scrna_methods.csv  # comparison of poly(A) scRNA methods
+    pyproject.toml          # package metadata, dependencies, entry points
+    ROADMAP.md              # development and publication roadmap
+    LICENSE                 # MIT License
+    Dockerfile              # container build
+```
+
+---
+
+## Citation
+
+If you use PeakATail, please cite the software entry below for now. The `author`
+field mirrors the package author declared in `pyproject.toml`; the full
+manuscript author list and a machine-readable `CITATION.cff` will be added with
+the paper release (tracked in the release-engineering issue).
+
+<!-- TODO[verify]: No CITATION.cff file exists yet. Replace this software entry
+with the published journal citation — and expand `author` to the full manuscript
+author list — once the paper is out. -->
+
+```bibtex
+@software{peakatail,
+  author  = {Amiri Tabat, Amir},
+  title   = {{PeakATail}: single-cell poly(A) site detection and APA analysis},
+  url     = {https://github.com/BMGLab/PeakATail},
+  version = {0.3.0},
+  note    = {Preprint in preparation; replace with the journal citation when available}
+}
+```
+
+---
+
+## License
+
+This project is licensed under the **MIT License** — see the [LICENSE](LICENSE) file for the full text. `pyproject.toml` declares the matching SPDX identifier (`license = "MIT"`).

peakatail-0.3.0/ema/__init__.py

@@ -0,0 +1,22 @@
+"""PeakATail package init.
+
+IMPORTANT: this module sets BLAS / OpenMP thread caps BEFORE any submodule
+(or numpy/scipy/scanpy) gets imported.  Without this, the multiprocessing
+``spawn`` Pool used by peak calling leaves the parent process's BLAS thread
+pool in a state where downstream ``scanpy``/``leidenalg`` calls deadlock on
+a futex inside OpenBLAS — observed as a hang at the clustering step on
+datasets larger than a few hundred cells.
+
+We default to single-threaded BLAS in the parent because (a) all heavy
+matrix work is already parallelised at the multiprocessing level (one
+worker per dataset / per tile), and (b) the parent process orchestrates
+spawn workers — over-subscribing BLAS threads in the parent only competes
+with worker CPU.  Users who want multi-threaded BLAS for a specific call
+can override by exporting these vars themselves before invoking ``ema``.
+"""
+import os as _os
+
+for _var in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS",
+             "MKL_NUM_THREADS", "BLIS_NUM_THREADS",
+             "VECLIB_MAXIMUM_THREADS", "NUMEXPR_NUM_THREADS"):
+    _os.environ.setdefault(_var, "1")