biotapy 0.2.0__tar.gz → 0.3.0__tar.gz

{biotapy-0.2.0 → biotapy-0.3.0}/CHANGELOG.md

@@ -10,6 +10,44 @@ and this project adheres to [Semantic Versioning][].
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-10-07
+
+### Added
+
+- `bt.pp.clr`: centred log-ratio transform into `layers["clr"]`, equal to
+  `vegan::decostand(x, "clr", pseudocount = 0.5)`.
+- `bt.pp.philr`: PhILR balances along the phylogeny into `obsm["X_philr"]`,
+  equal to `philr::philr` with its default weights.
+- `bt.da.linda`, `bt.da.ancombc2`: differential abundance by LinDA (a port
+  of `MicrobiomeStat::linda`) and ANCOM-BC2 (through scikit-bio), both in
+  Python and compared with R in golden tests.
+- `bt.da.aldex2`, `bt.da.maaslin3`: ALDEx2 and MaAsLin 3's abundance model,
+  run in R through rpy2. They need R, the R package, and the new extra
+  `biotapy[r]` (`rpy2>=3.6.8`, GPL-2.0-or-later, built against your R).
+- One result table for every method: log2 `effect`, `se`, `pvalue`,
+  Benjamini-Hochberg `qvalue` over the features the method tested,
+  `direction`, `method` and `contrast`. Methods take `group`, `covariates`
+  and `reference` instead of a formula, and never filter features.
+- `bt.da.consensus` and `bt.pl.consensus`: count, per feature, the methods
+  that call it and whether they agree on its direction, as a table and as a
+  dot matrix.
+- A differential abundance guide, a page per method, and a tutorial on the
+  GlobalPatterns genera.
+- asv benchmarks for `pp.philr`, `da.linda` and `da.ancombc2`, with their
+  baselines in the docs.
+
+### Changed
+
+- The readers infer `x_kind` `"counts"` only for non-negative whole numbers,
+  and the functions that need counts (`pp.rarefy`, `tl.alpha`'s
+  `observed_features` and `chao1`, weighted `tl.unifrac`) refuse a table with
+  a negative value.
+- `bt.pl.bar` and `bt.pl.heatmap` raise `ValueError` for a table with a
+  negative value, such as `layers["clr"]`, instead of drawing it as
+  abundances.
+- The Coming-from-R page marks the phyloseq calls biotapy does not cover yet
+  "not in 0.3".
+
 ## [0.2.0] - 2026-10-05
 
 ### Added

{biotapy-0.2.0 → biotapy-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: biotapy
-Version: 0.2.0
+Version: 0.3.0
 Summary: mia-style microbiome toolkit for Python on AnnData/TreeData
 Project-URL: Documentation, https://biotapy.readthedocs.io/
 Project-URL: Homepage, https://github.com/pedrocr83/biotapy
@@ -33,6 +33,8 @@ Requires-Dist: session-info2
 Requires-Dist: threadpoolctl>=3.5
 Requires-Dist: treedata<0.4,>=0.3.1
 Requires-Dist: xarray
+Provides-Extra: r
+Requires-Dist: rpy2>=3.6.8; extra == 'r'
 Description-Content-Type: text/markdown
 
 # biotapy
@@ -51,9 +53,9 @@ against R on real data.
 
 ## Status
 
-**biotapy 0.2 is an early release.** The API can still change between minor versions.
+**biotapy 0.3 is an early release.** The API can still change between minor versions.
 
-What 0.2 does (full signatures in the [API reference][api]):
+What 0.3 does (full signatures in the [API reference][api]):
 
 - **Readers**: `bt.io.read_biom` (BIOM 1.0/2.1), `bt.io.read_qiime2`
   (`.qza` artifacts, no QIIME 2 install needed), `bt.io.read_dada2`
@@ -71,17 +73,24 @@ What 0.2 does (full signatures in the [API reference][api]):
   and `bt.datasets.enzyme` (the ENZYME EC hierarchy), downloaded and cached
   on first use.
 - **Preprocessing**: `bt.pp.relative`, `bt.pp.tax_glom`,
-  `bt.pp.filter_features`, `bt.pp.filter_samples` and `bt.pp.rarefy`.
+  `bt.pp.filter_features`, `bt.pp.filter_samples` and `bt.pp.rarefy`, and
+  the compositional transforms `bt.pp.clr` and `bt.pp.philr` (checked against
+  vegan and philr).
 - **Function**: `bt.fn.load_hierarchy`, `bt.fn.func_glom` and `bt.fn.renorm`
   (checked against HUMAnN's own output), `bt.fn.contributions` and
   `bt.fn.functional_redundancy`.
 - **Tools**: `bt.tl.alpha`, `bt.tl.beta`, `bt.tl.unifrac`, `bt.tl.pcoa`,
   `bt.tl.nmds` and `bt.tl.permanova`, each checked against R on real data.
+- **Differential abundance**: `bt.da.linda` and `bt.da.ancombc2` in Python,
+  `bt.da.aldex2` and `bt.da.maaslin3` through R (the `r` extra below), all
+  returning one result table and checked against their R packages, and
+  `bt.da.consensus`, which reports where the methods agree.
 - **Plots**: `bt.pl.bar`, `bt.pl.richness`, `bt.pl.ordination`,
-  `bt.pl.scree`, `bt.pl.heatmap` and `bt.pl.contributions`.
+  `bt.pl.scree`, `bt.pl.heatmap`, `bt.pl.contributions` and
+  `bt.pl.consensus`.
 
-Next, in 0.3: compositional transforms (CLR, PhILR) and differential abundance
-(ANCOM-BC, LinDA, ALDEx2, MaAsLin 3) behind one result format. See the
+Next, in 0.4: multi-omics conventions on MuData, leak-free scikit-learn
+transformers, a PyTorch loader and an interface for embedding models. See the
 [roadmap][roadmap]; no dates are promised.
 
 ## Installation
@@ -108,6 +117,17 @@ pixi add --pypi biotapy
 The development version installs straight from GitHub:
 `pip install git+https://github.com/pedrocr83/biotapy.git`.
 
+ALDEx2 and MaAsLin 3 run in R. They need R with the R packages
+(`BiocManager::install(c("ALDEx2", "maaslin3"))`) and the `r` extra, which
+builds rpy2 against that R (Linux and macOS):
+
+```bash
+pip install 'biotapy[r]'
+```
+
+rpy2 is GPL-2.0-or-later and the R packages carry their own licences; biotapy
+does not ship any of them.
+
 On Python 3.14, the `biom-format` dependency has no wheels yet, so it is built
 from source and needs a C compiler until biom-format publishes 3.14 wheels
 ([biocore/biom-format#1004][biom-format-1004]).

{biotapy-0.2.0 → biotapy-0.3.0}/README.md

@@ -14,9 +14,9 @@ against R on real data.
 
 ## Status
 
-**biotapy 0.2 is an early release.** The API can still change between minor versions.
+**biotapy 0.3 is an early release.** The API can still change between minor versions.
 
-What 0.2 does (full signatures in the [API reference][api]):
+What 0.3 does (full signatures in the [API reference][api]):
 
 - **Readers**: `bt.io.read_biom` (BIOM 1.0/2.1), `bt.io.read_qiime2`
   (`.qza` artifacts, no QIIME 2 install needed), `bt.io.read_dada2`
@@ -34,17 +34,24 @@ What 0.2 does (full signatures in the [API reference][api]):
   and `bt.datasets.enzyme` (the ENZYME EC hierarchy), downloaded and cached
   on first use.
 - **Preprocessing**: `bt.pp.relative`, `bt.pp.tax_glom`,
-  `bt.pp.filter_features`, `bt.pp.filter_samples` and `bt.pp.rarefy`.
+  `bt.pp.filter_features`, `bt.pp.filter_samples` and `bt.pp.rarefy`, and
+  the compositional transforms `bt.pp.clr` and `bt.pp.philr` (checked against
+  vegan and philr).
 - **Function**: `bt.fn.load_hierarchy`, `bt.fn.func_glom` and `bt.fn.renorm`
   (checked against HUMAnN's own output), `bt.fn.contributions` and
   `bt.fn.functional_redundancy`.
 - **Tools**: `bt.tl.alpha`, `bt.tl.beta`, `bt.tl.unifrac`, `bt.tl.pcoa`,
   `bt.tl.nmds` and `bt.tl.permanova`, each checked against R on real data.
+- **Differential abundance**: `bt.da.linda` and `bt.da.ancombc2` in Python,
+  `bt.da.aldex2` and `bt.da.maaslin3` through R (the `r` extra below), all
+  returning one result table and checked against their R packages, and
+  `bt.da.consensus`, which reports where the methods agree.
 - **Plots**: `bt.pl.bar`, `bt.pl.richness`, `bt.pl.ordination`,
-  `bt.pl.scree`, `bt.pl.heatmap` and `bt.pl.contributions`.
+  `bt.pl.scree`, `bt.pl.heatmap`, `bt.pl.contributions` and
+  `bt.pl.consensus`.
 
-Next, in 0.3: compositional transforms (CLR, PhILR) and differential abundance
-(ANCOM-BC, LinDA, ALDEx2, MaAsLin 3) behind one result format. See the
+Next, in 0.4: multi-omics conventions on MuData, leak-free scikit-learn
+transformers, a PyTorch loader and an interface for embedding models. See the
 [roadmap][roadmap]; no dates are promised.
 
 ## Installation
@@ -71,6 +78,17 @@ pixi add --pypi biotapy
 The development version installs straight from GitHub:
 `pip install git+https://github.com/pedrocr83/biotapy.git`.
 
+ALDEx2 and MaAsLin 3 run in R. They need R with the R packages
+(`BiocManager::install(c("ALDEx2", "maaslin3"))`) and the `r` extra, which
+builds rpy2 against that R (Linux and macOS):
+
+```bash
+pip install 'biotapy[r]'
+```
+
+rpy2 is GPL-2.0-or-later and the R packages carry their own licences; biotapy
+does not ship any of them.
+
 On Python 3.14, the `biom-format` dependency has no wheels yet, so it is built
 from source and needs a C compiler until biom-format publishes 3.14 wheels
 ([biocore/biom-format#1004][biom-format-1004]).

{biotapy-0.2.0 → biotapy-0.3.0}/docs/_data/r_idioms.toml

@@ -23,8 +23,8 @@
 "phyloseq::prune_samples" = '`tdata[keep].copy()`'
 "phyloseq::subset_taxa" = '`tdata[:, tdata.var["phylum"] == "Chlamydiae"].copy()`'
 "phyloseq::subset_samples" = '`tdata[tdata.obs["SampleType"] == "Feces"].copy()`'
-"phyloseq::tip_glom" = "not in 0.2"
-"phyloseq::merge_samples" = "not in 0.2"
-"phyloseq::psmelt" = "not in 0.2"
-"phyloseq::plot_tree" = "not in 0.2"
-"phyloseq::plot_net" = "not in 0.2"
+"phyloseq::tip_glom" = "not in 0.3"
+"phyloseq::merge_samples" = "not in 0.3"
+"phyloseq::psmelt" = "not in 0.3"
+"phyloseq::plot_tree" = "not in 0.3"
+"phyloseq::plot_net" = "not in 0.3"

{biotapy-0.2.0 → biotapy-0.3.0}/docs/api.md

@@ -49,8 +49,10 @@ Public functions are listed here as they ship, from Phase 1 onward.
 .. autosummary::
     :toctree: generated
 
+    pp.clr
     pp.filter_features
     pp.filter_samples
+    pp.philr
     pp.rarefy
     pp.relative
     pp.tax_glom
@@ -89,6 +91,22 @@ Public functions are listed here as they ship, from Phase 1 onward.
     tl.unifrac
 ```
 
+## Differential abundance
+
+```{eval-rst}
+.. module:: biotapy.da
+.. currentmodule:: biotapy
+
+.. autosummary::
+    :toctree: generated
+
+    da.aldex2
+    da.ancombc2
+    da.consensus
+    da.linda
+    da.maaslin3
+```
+
 ## Plots
 
 ```{eval-rst}
@@ -99,6 +117,7 @@ Public functions are listed here as they ship, from Phase 1 onward.
     :toctree: generated
 
     pl.bar
+    pl.consensus
     pl.contributions
     pl.heatmap
     pl.ordination

{biotapy-0.2.0 → biotapy-0.3.0}/docs/coming_from_r.md

@@ -2,7 +2,7 @@
 
 Every public biotapy function names its R equivalent in its docstring. This table is generated
 from those lines each time the docs are built, plus a short list of phyloseq accessors that are
-plain AnnData/TreeData code (`docs/_data/r_idioms.toml`). Rows marked "not in 0.2" have no
+plain AnnData/TreeData code (`docs/_data/r_idioms.toml`). Rows marked "not in 0.3" have no
 biotapy equivalent yet.
 
 biotapy keeps samples as rows, so `tdata.X` is phyloseq's `otu_table` with

{biotapy-0.2.0 → biotapy-0.3.0}/docs/contributing.md

@@ -57,9 +57,9 @@ uv run --group test pytest -m "network or golden"
 
 These tests download `bt.datasets.global_patterns()`, `bt.datasets.enterotype()`
 and `bt.datasets.esophagus()` through [pooch](https://www.fatiando.org/pooch/)
-and compare biotapy with R on that data: `pp.relative`, `pp.tax_glom`, filtering, rarefaction
+and compare biotapy with R on that data: `pp.relative`, `pp.clr`, `pp.tax_glom`, filtering, rarefaction
 (its invariants), alpha and beta diversity, UniFrac, PCoA, NMDS and PERMANOVA against phyloseq,
-vegan, ape and picante. Set
+vegan, ape and picante, `pp.philr` against philr, `da.linda` against MicrobiomeStat and `da.ancombc2` against ANCOMBC. Set
 `BIOTAPY_DATA_DIR` to point the pooch cache somewhere other than the default
 per-user cache directory - CI caches it across runs the same way:
 
@@ -67,6 +67,19 @@ per-user cache directory - CI caches it across runs the same way:
 BIOTAPY_DATA_DIR=.pooch uv run --group test pytest -m "network or golden"
 ```
 
+### R bridge tests
+
+Tests that call R through rpy2 (`bt.da.aldex2`, `bt.da.maaslin3`) carry the marker `r` and are excluded from the
+runs above. They need R with the packages `tests/r/Dockerfile` installs and the `r` extra; those
+that read GlobalPatterns also need the pooch cache:
+
+```bash
+BIOTAPY_DATA_DIR=.pooch uv run --group test --extra r pytest -m r
+```
+
+CI runs them in the `r-bridge` job, with R 4.5.3 and the Bioconductor 3.22 packages the golden image
+pins.
+
 ### Regenerating the R golden files
 
 The golden CSVs under `tests/golden/` and the R-written fixtures under

biotapy-0.3.0/docs/guide/differential_abundance.md

@@ -0,0 +1,134 @@
+# Differential abundance
+
+Differential abundance (DA) asks which features are more abundant in one group of samples than in
+another. Every `bt.da` method takes the same arguments and returns the same table, so their answers
+can be put side by side. The {doc}`tutorial </tutorials/differential_abundance>` runs them on
+GlobalPatterns.
+
+## One question, one table
+
+A method takes the `obs` column whose effect you want (`group`), the `covariates` to adjust for,
+and the `reference` level that the other level is compared with:
+
+```python
+import biotapy as bt
+
+tdata = bt.datasets.toy()
+table = bt.da.linda(tdata, "group")  # "B vs A": A, the first category, is the reference
+table = bt.da.linda(tdata, "group", reference="B")  # "A vs B": every effect changes sign
+```
+
+`group` is a categorical, string or bool column with two levels, or a numeric column, whose effect
+is then a slope. Covariates are numeric, or categorical with one indicator per level against
+their first. There are no formula strings: R and patsy take the alphabetically first level as the
+reference and silently drop samples with a missing value. biotapy takes the reference you give, or
+the first category, says which way round the effect is in the `contrast` column, and raises on a
+missing value, on a constant column, on a group with more than two levels and on collinear covariates.
+
+The table has one row per feature, in `var_names` order:
+
+| Column | Meaning |
+|---|---|
+| `effect` | log2 fold change of the other level over the reference, or the slope of a numeric group: per standard deviation in LinDA and MaAsLin 3, per unit in ANCOM-BC2, not offered by ALDEx2. ALDEx2's is the median log2 difference between the groups, MaAsLin 3's the coefficient minus the median coefficient |
+| `se` | its standard error; NaN for every feature in ALDEx2, which reports none |
+| `pvalue` | the method's p-value |
+| `qvalue` | Benjamini-Hochberg adjusted p-value, over the features the method tested |
+| `direction` | sign of `effect`: -1, 0 or 1 |
+| `method` | the method's name |
+| `contrast` | `"<level> vs <reference>"`, or the name of a numeric group |
+
+A feature a method cannot test keeps its row, with NaN `effect`, `pvalue` and `qvalue` and `direction` 0 (a NaN `se` alone means
+nothing for ALDEx2). Methods never
+filter features: filter once, with `bt.pp.filter_features`, before running any method, so every
+method tests the same features. Methods need raw counts in `X` and a read in every sample
+(`bt.pp.filter_samples(tdata, min_depth=1)` drops empty ones).
+
+## Choosing methods
+
+biotapy has four methods. They model compositional counts differently, so they call different
+features, and no one of them is right on every dataset:
+
+| Method | Model | Runs in | Covariates | Numeric `group` | `seed` |
+|---|---|---|---|---|---|
+| {doc}`bt.da.linda </methods/linda>` | least squares of log2 centred log-ratios, minus the mode of all coefficients | Python | yes | per standard deviation | no |
+| {doc}`bt.da.ancombc2 </methods/ancombc2>` | least squares of log counts corrected for each sample's sampling fraction | Python (scikit-bio) | yes | per unit | no |
+| {doc}`bt.da.aldex2 </methods/aldex2>` | Welch t-tests on Monte Carlo draws of centred log-ratios | R | no | no | yes |
+| {doc}`bt.da.maaslin3 </methods/maaslin3>` | least squares of log2 relative abundance where present, tested against the median | R | yes | per standard deviation | yes |
+
+Each method's page gives its model, the R defaults biotapy changes, and how closely biotapy
+matches the R package on real data. Two things to know before you choose:
+
+- ANCOM-BC2 and MaAsLin 3 leave zeros out, so a feature with no read in one group cannot be
+  fitted: its row is NaN. LinDA adds 0.5 to every count when the table has a zero, and ALDEx2
+  draws around every count, so both test it.
+- `reference` changes more than the sign in ANCOM-BC2 (its bias correction is fitted against the
+  reference level) and ALDEx2 (its effect's random resampling follows the label order), and
+  slightly in MaAsLin 3 (its p-values move by up to about 1.2e-3 with the same `seed`, the calls do not).
+  Choose it on the biology, the control or baseline level, before you look at any result.
+
+(da-methods-in-r)=
+## Methods that run in R
+
+ALDEx2 and MaAsLin 3 exist only in R, so biotapy calls them there through
+[rpy2](https://rpy2.github.io/). They need R, the R package, and biotapy's `r` extra, which builds
+rpy2 against that R (Linux and macOS; it fails to install when no R is found):
+
+```bash
+Rscript -e 'install.packages("BiocManager"); BiocManager::install(c("ALDEx2", "maaslin3"))'
+pip install 'biotapy[r]'
+```
+
+rpy2 is GPL-2.0-or-later and the R packages have their own licences; biotapy itself does not ship
+any of them. Without rpy2 or the R package, the call raises an `ImportError` that names what to
+install. Each call converts `X` to a dense table once, because rpy2 has no sparse converter.
+
+Both methods draw random numbers in R. `seed` seeds R for the call, and your R session's own
+random state is restored afterwards, so the same seed gives the same table. Each R warning raised
+during the call, such as ALDEx2's for fewer than 128 `mc_samples`, is re-emitted as a Python
+`UserWarning`; an R error is raised as a `RuntimeError`. Repeated `var_names` or `obs_names`
+raise: call `adata.var_names_make_unique()` first.
+
+```python
+table = bt.da.aldex2(tdata, "group", seed=0)
+table = bt.da.maaslin3(tdata, "group", seed=0)
+```
+
+## Where methods agree
+
+`bt.da.consensus` puts the tables of several methods side by side and counts, for each feature,
+the methods that call it significant (`qvalue < alpha`, strictly) and whether they agree on its
+direction:
+
+```python
+results = [bt.da.ancombc2(tdata, "group"), bt.da.linda(tdata, "group")]
+table = bt.da.consensus(results)  # alpha=0.05, min_methods=2
+table[table["consensus"]]
+```
+
+A feature is a consensus hit when at least `min_methods` methods call it and all of them give it
+the same sign. Methods that call it in opposite directions mark a `conflict`, which is never a
+consensus. A method that could not test a feature does not count against it: `n_tested` says how
+many methods tested each feature. The tables must come from different methods and compare the same
+`contrast`, so run every method with the same `group` and `reference`. The
+{doc}`consensus page </methods/consensus>` defines every column.
+
+Methods disagree a lot on real data, and the literature is split on what to do about it: Nearing
+et al. (2022) recommend a consensus of several methods, Pelto et al. (2025) one simple method. Either
+way, choose the methods before you look at their results; trying methods until one finds what you
+hoped for is selective reporting, and a consensus table does not protect against it. Methods that
+share a model, such as ANCOM-BC and ANCOM-BC2, also agree more often for that reason alone.
+
+## Plotting the consensus
+
+`bt.pl.consensus` draws the consensus table as a dot matrix: one row per feature that at least one
+method calls, one column per method. A filled dot is a call, red for a positive effect and blue for
+a negative one; a hollow dot is a feature the method tested without calling it; no dot, a feature
+the method could not test. Consensus features have bold labels. Rows are sorted by the number of
+methods calling them, then by their mean absolute effect, and `top` keeps the first 30:
+
+```python
+ax = bt.pl.consensus(table, top=20)
+```
+
+Feature ids are often accession numbers; to label rows with a rank, rename the table's index first,
+for example `table.rename(index=tdata.var["genus"])`.

{biotapy-0.2.0 → biotapy-0.3.0}/docs/guide/diversity.md

@@ -45,7 +45,7 @@ bt.tl.unifrac(tdata, weighted=True, inplace=True)  # obsp["weighted_unifrac"]
   Without `binary = TRUE` phyloseq computes vegan's quantitative Jaccard, a different number.
 - Weighted UniFrac is normalized to 0-1 by default, as in phyloseq; pass `normalized=False` for
   the raw value.
-- Weighted UniFrac needs raw counts: `x_kind` `"counts"` and whole numbers. scikit-bio's tree code
+- Weighted UniFrac needs raw counts: `x_kind` `"counts"` and non-negative whole numbers. scikit-bio's tree code
   casts abundances to integers, so proportions would all become 0. Unweighted UniFrac uses
   presence only and runs on relative abundances too.
 - A tree whose root has three or more children is used rooted where it is drawn. phyloseq

{biotapy-0.2.0 → biotapy-0.3.0}/docs/guide/filtering.md

@@ -49,7 +49,7 @@ out = bt.pp.rarefy(tdata, depth=60, seed=0)
 - Samples with fewer than `depth` reads are dropped, with one warning naming them; a sample with
   exactly `depth` reads is kept.
 - Features left all-zero are dropped, as with phyloseq's `trimOTUs = TRUE`.
-- `X` must hold raw counts: `uns["biotapy"]["x_kind"] == "counts"` and whole numbers.
+- `X` must hold raw counts: `uns["biotapy"]["x_kind"] == "counts"` and non-negative whole numbers.
 - phyloseq samples with replacement by default (`replace = TRUE`). biotapy always samples
   without, like `replace = FALSE`.
 - The same `seed` gives the same result. R and NumPy use different random generators, so the

{biotapy-0.2.0 → biotapy-0.3.0}/docs/guide/index.md

@@ -14,5 +14,6 @@ function
 filtering
 diversity
 ordination
+differential_abundance
 plotting
 ```

{biotapy-0.2.0 → biotapy-0.3.0}/docs/guide/reading_data.md

@@ -5,7 +5,7 @@ Readers turn a file format into a `TreeData` that follows the one
 
 BIOM, QIIME 2, DADA2 and phyloseq tables do not record whether they hold
 counts or proportions, so their readers infer `uns["biotapy"]["x_kind"]` from
-the values: whole numbers are `"counts"`; otherwise, if every sample with a
+the values: non-negative whole numbers are `"counts"`; otherwise, if every sample with a
 nonzero total sums to 1 (within `1e-3`), `"relative"`; anything else is
 `"abundance"`. The HUMAnN, MetaPhlAn and PICRUSt2 readers take it from the
 format instead, as their sections say. Functions that need raw counts check

biotapy-0.3.0/docs/guide/transforms.md

@@ -0,0 +1,86 @@
+# Transforms
+
+Transforms add a layer (or, for PhILR, an `obsm` table) to a copy of your data; they never touch `X` or drop
+anything else, so you can always get back to what you started from.
+
+## Relative abundance
+
+`bt.pp.relative` scales every sample (row) so its features sum to one, and stores the result in
+`layers["relative"]`:
+
+```python
+import biotapy as bt
+
+tdata = bt.datasets.toy()
+out = bt.pp.relative(tdata)
+out.layers["relative"][0].sum()  # 1.0
+```
+
+`X` still holds the original counts; `out.uns["biotapy"]["provenance"]` records that
+`pp.relative` ran.
+
+### All-zero samples
+
+A sample with no reads has nothing to divide by. Where phyloseq's
+`transform_sample_counts` divides by zero and returns `NaN`, biotapy leaves an
+all-zero sample as all zeros, so downstream steps do not have to special-case `NaN`.
+
+## Centred log-ratio (CLR)
+
+Microbiome counts are compositional: a sequencing run fixes the total, so only ratios between
+features carry information. `bt.pp.clr` takes the logarithm of each value relative to the
+geometric mean of its sample, and stores it in `layers["clr"]`:
+
+```python
+out = bt.pp.clr(tdata)  # pseudocount=0.5
+out.layers["clr"][0].sum()  # 0.0, up to rounding
+```
+
+The logarithm of zero is undefined, so a pseudocount is added to every value first, zeros or
+not, as `mia::transformAssay(method = "clr", pseudocount = 0.5)` and
+`vegan::decostand(x, "clr", pseudocount = 0.5)` do. The default 0.5 suits counts. On relative
+abundances or CPM it would swamp the rarest features, so biotapy warns when the pseudocount is
+larger than the smallest non-zero value; pass one on the data's scale, for example half that value.
+
+An all-zero sample gives an all-zero CLR row.
+
+`layers["clr"]` is dense: CLR has no zeros, so it takes 8 bytes per sample and feature, and the call peaks at three to five such arrays, the most when `X` is mostly non-zero.
+
+## PhILR
+
+`bt.pp.philr` turns a sample into balances along its phylogeny: one per internal node with two
+children, the log-ratio of the geometric means of the taxa under the node's two children, scaled so
+that the balances are orthonormal coordinates. They go to `obsm["X_philr"]`, a table with one
+column per balance:
+
+```python
+tdata = bt.datasets.toy()[:, :6].copy()
+out = bt.pp.philr(tdata)  # pseudocount=0.5, as in pp.clr
+out.obsm["X_philr"]  # columns root, n1, n4, n2, n5
+```
+
+A balance is positive when the node's first child is more abundant than its second, as in
+`philr::philr` with its default uniform weights. Distances between samples' balances equal
+their Aitchison distances, the Euclidean distances between their CLR vectors.
+
+PhILR needs a rooted binary tree. Filtering features leaves nodes with one child; those define
+no balance and are skipped. A node with three or more children raises: that includes the root
+of an unrooted tree (`bt.datasets.toy()` has one, which is why the example keeps f1-f6). Root
+the tree and resolve its polytomies before reading it, for example with `ape::multi2di` in R, or
+in Python with scikit-bio, which resolves them arbitrarily, as `multi2di` does:
+
+```python
+from skbio import TreeNode
+
+tree = TreeNode.read("tree.nwk")  # root it first (outgroup or midpoint) if it is unrooted: your choice
+tree.bifurcate()  # in place; every node ends with two children
+tree.write("binary.nwk")
+# then read your table again with tree="binary.nwk"
+```
+
+Balance names are the node names in `vart["phylo"]`, which differ from the ones `philr` in R
+makes with `makeNodeLabel`; to compare balances across tools, match them by their numerator and
+denominator taxa, not by name.
+
+Filtering features with `bt.pp.filter_features` drops `layers["clr"]` and `obsm["X_philr"]`;
+plain `adata[:, ...]` slicing keeps the old values, so transform after subsetting.

{biotapy-0.2.0 → biotapy-0.3.0}/docs/index.md

@@ -16,6 +16,7 @@ tutorials/index.md
 :caption: User guide
 
 guide/index.md
+methods/index.md
 coming_from_r.md
 ```
 

biotapy-0.3.0/docs/methods/aldex2.md

@@ -0,0 +1,84 @@
+# ALDEx2
+
+{func}`bt.da.aldex2 <biotapy.da.aldex2>` runs `ALDEx2::aldex` in R through rpy2: it draws
+plausible relative abundances for each sample, takes their log2 centred log-ratios and compares
+the two groups with a Welch t-test on every draw. It needs R, the R package ALDEx2 and biotapy's
+`r` extra ({ref}`Methods that run in R <da-methods-in-r>`).
+
+```python
+import biotapy as bt
+
+table = bt.da.aldex2(bt.datasets.toy(), "group", seed=0)
+```
+
+## Model
+
+For each sample $i$, ALDEx2 draws `mc_samples` proportion vectors from a Dirichlet distribution
+whose parameters are the sample's counts plus 0.5, so a draw reflects how uncertain proportions
+are at that sequencing depth. Each draw is turned into log2 centred log-ratios over all features
+(`denom = "all"`):
+
+$$
+c^{(k)}_{ij} = \log_2 p^{(k)}_{ij} - \frac{1}{m} \sum_{l=1}^{m} \log_2 p^{(k)}_{il},
+\qquad p^{(k)}_{i\cdot} \sim \operatorname{Dirichlet}(y_{i\cdot} + 0.5)
+$$
+
+For each draw $k$ and feature $j$, a Welch t-test compares the two groups' values $c^{(k)}_{ij}$;
+`pvalue` is ALDEx2's `we.ep`: for each draw ALDEx2 doubles the one-sided Welch p-value in each
+direction (capped at 1), averages each direction over the draws, and keeps the smaller average.
+`effect` is ALDEx2's `diff.btw`, the median difference between values resampled at random from each
+group's pooled draws, already log2. ALDEx2
+reports no standard error, so `se` is NaN for every feature, and `qvalue` is the
+Benjamini-Hochberg correction of `pvalue`, as for every method. A feature with no read in any
+sample is dropped by ALDEx2 and is NaN in the table (not tested).
+
+## Units
+
+`effect` is the median log2 difference of the centred log-ratios between the other level and
+`reference`. ALDEx2 compares two groups only: a numeric `group`, covariates and a level in fewer
+than two samples are refused.
+
+## Compared with R
+
+biotapy calls `ALDEx2::aldex` (ALDEx2 1.42.0) with its defaults:
+
+| `aldex` argument or output | R default | biotapy |
+|---|---|---|
+| `conditions` | required | the labels `"0"` (`reference`) and `"1"`: ALDEx2 fails on a factor and sorts character labels in the R session's locale |
+| `mc.samples` | `128` | `mc_samples=128`; R warns below 128, and biotapy re-emits it as a `UserWarning` |
+| `test`, `effect`, `denom` | `"t"`, `TRUE`, `"all"` | the same |
+| `paired.test`, `iterate`, `gamma` | `FALSE`, `FALSE`, `NULL` | the same: no paired test, no scale model |
+| `we.ep` | output | `pvalue` |
+| `we.eBH` | output | not carried: it averages the draws' corrected values, a different quantity; `qvalue` is BH of `we.ep` |
+| `diff.btw` | output | `effect` |
+| `effect`, `overlap`, `wi.ep`, `wi.eBH`, `rab.*`, `diff.win` | outputs | not carried: ALDEx2's `effect` is a standardised size, not a fold change |
+| random state | R's global seed | `seed` sets R's seed for the call, and the R session's random state is put back afterwards |
+
+On the GlobalPatterns genera below, BH of `we.ep` calls 11 genera and ALDEx2's `we.eBH` 19.
+
+## Agreement with R
+
+ALDEx2 is random, but `seed` becomes one integer for R's `set.seed`, so biotapy and R draw the same
+numbers. The golden test runs `set.seed(...); ALDEx2::aldex(...)` with that integer on the 636
+GlobalPatterns genera in at least 20% of samples, human hosts against the rest: `effect` and
+`diff.btw` agree to a relative 4e-15 and `pvalue` and `we.ep` to 8.4e-13 (checked at 1e-7). The
+golden passes `reference="human"`, the level R sorts first, so both sides resample `diff.btw` in the same
+label order. With other seeds, effects correlate
+at 0.993 (Spearman) and 11 to 13 genera are called, depending on the seed (13 at `seed=0`, the
+tutorial's).
+
+## Choosing the reference
+
+Swapping `reference` does more than flip the sign. ALDEx2 draws the same Monte Carlo instances, so
+`pvalue` and the calls are unchanged, but `diff.btw` comes from a random resampling of each group's
+values done in label order, so the same `seed` gives different effects, and the two runs are not
+exact mirror images. They differ from exact antisymmetry by 0.1 to 0.3 log2 on `toy()`, depending
+on the seed (its effects are about 4 log2 wide), and by up to 0.65 log2 on the GlobalPatterns
+genera, 15 of 636 of which then do not change direction. The p-values and the calls are the same.
+
+## Reference
+
+Fernandes AD, Reid JN, Macklaim JM, McMurrough TA, Edgell DR, Gloor GB (2014) Unifying the
+analysis of high-throughput sequencing datasets: characterizing RNA-seq, 16S rRNA gene sequencing
+and selective growth experiments by compositional data analysis. *Microbiome* 2:15.
+[doi:10.1186/2049-2618-2-15](https://doi.org/10.1186/2049-2618-2-15)