scatrans 0.9.8.dev0__tar.gz → 0.9.8.dev2__tar.gz

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/CHANGELOG.md

@@ -5,9 +5,61 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.9.8] - 2026-07-03
+
+### Changed (user-visible default behavior)
+- **`active_score()` built-in `significant` list** now uses the same default thresholds as
+  `filter_active_genes(preset="heuristic")`: `logFC > 0.35`, `unspliced_excess_residual > 1.0`,
+  `active_score >= 55`, `active_score_fdr < 0.25`, plus `p_adj` / `unspliced_excess_fdr` cutoffs.
+  Re-running analyses with defaults may return a **different** significant gene set than 0.9.2
+  (typically more genes when signal is present; aligns with the documented post-hoc workflow).
+- **`active_score(logfc_cutoff=...)` default** lowered from `0.5` to `0.35` to match the heuristic
+  preset (only affects the built-in significant mask and metadata; ranking in `all_results` unchanged).
+- **`gamma_method="empirical_bayes"` + `prior_weight`**: fixed a numeric floor so `prior_weight` in
+  the usual tuning range (0.5–5.0, including the default 5.0) now changes `count_pseudocount` and
+  shrinkage weights instead of being pinned at 1.0. EB gamma values may differ from 0.9.2 at the
+  same `prior_weight`.
+
+### Fixed
+- **`scat.pl.gseaplot`**: RES curve / hit ticks and the bottom ranked-metric bar now share the same
+  gene order by preferring `gsea_result.attrs["ranking"]` over caller-supplied row order.
+- **`scat.pl` robustness**: placeholder figures for empty or incomplete volcano/comet inputs;
+  `bias_diagnostic_plot` validates external `axes`; `active_genes_heatmap` default `show=True` no
+  longer hits a Python `UnboundLocalError` on `plt`.
+- **PyDESeq2 pseudobulk DE**: log warning + `de_df.attrs` diagnostics when genes are skipped by
+  `min_counts_per_gene` or marked NaN by DESeq2 independent filtering (neutral fill is not
+  "tested and non-significant").
+- **Memento DE**: log warning + `n_genes_not_returned_by_memento` when memento drops genes
+  internally; reindexed neutral rows are documented in diagnostics.
+- **`restore_raw_counts`**: reject `layers["counts"]` restore when `raw_gene_list` order differs
+  from current `var_names` (same guard as `adata.raw`).
+- **`active_score` pseudobulk**: `pb_x_layer="X"` now means `adata.X`, matching
+  `differential_expression()`.
+- **`_velocity._estimate_eb_prior_from_reference`**: removed unreachable trim fallback branch;
+  trim-skipped path labeled `empirical_bayes_median_mad`.
+- **`_utils._get_group_mean`**: removed redundant sparse/dense branches.
+
+### Added
+- Shared `HEURISTIC_FILTER_DEFAULTS` in `tl.py` (single source for heuristic preset + built-in
+  significant mask).
+- Tests: gseaplot ranking alignment, `prior_weight` EB sensitivity, significant vs heuristic
+  filter parity, PyDESeq2/Memento diagnostic paths, `restore_raw_counts` order guard,
+  `active_genes_heatmap` default show, `pb_x_layer="X"` sentinel.
+
+### Documentation
+- README built-in `significant` section updated to match `HEURISTIC_FILTER_DEFAULTS`.
+
 ## [0.9.2 bugfix 2026-07-02]
 
 ### Fixed
+- **compare_enrichment / concat_compare_results metadata**: `attrs["clusters"]` now lists only clusters that contributed rows to the combined table (skipped, failed, and empty per-cluster runs remain in `per_cluster` diagnostics only). Fixes ghost cluster names in both the main compare API and the concat wrapper.
+- **DE schema validation**: `_run_de_wrapper` validates that all backends return `logFC`, `p_val`, `p_adj` with at least one finite value per column.
+- **MixedLM robustness**: near-constant genes use variance threshold; non-converged fits and missing condition coefficients are counted as failed fits instead of guessing the second coefficient; `failed_fit_rate` added to result attrs with percentage in warning log.
+- **DE warnings**: selective `_de_warning_context()` suppresses deprecation/future noise only (scanpy/PyDESeq2/MixedLM/BH), no longer blanket `ignore` on all warning categories.
+- **Memento audit column**: native adjusted p-values preserved as `memento_p_adj_native` when returned by memento-de; package `p_adj` still uses BH for cross-backend consistency.
+- **DE edge-case tests**: new `tests/test_de_edge_cases.py` (schema validation, constant-gene MixedLM failure counting, scanpy direction agreement, all-zero gene finiteness).
+- **Diagnostics / docs**: `failed_fit_rate` exposed in `active_score` mixed-model diagnostics and `differential_expression` metadata; README adds a short DE backend decision guide.
+- **copy_input performance**: `active_score` and `differential_expression` now honor `copy_input=False` — combined obs filters (subset + target/reference) perform at most one `AnnData.copy()` when `copy_input=True`, and zero when `copy_input=False` and no obs filtering is required (previously an unconditional `[keep_mask].copy()` always ran).
 - **filter_active_genes permissive mode**: default/permissive thresholds for `pval_cutoff`, `active_score_fdr_cutoff`, and `unspliced_excess_fdr_cutoff` now use `float("inf")` instead of `1.0`, so genes with adjusted p-value or permutation FDR exactly equal to 1.0 are no longer silently dropped (strict `<` vs `1.0` bug).
 - **CI lint**: removed unused `import scanpy as sc` from `tl.py` and `_permutation.py` (leftover from permutation refactor); fixed `test_enrich_api.py` formatting.
 - **CI tests**: `test_pp_bias_cli` GTF generator tests now `pytest.importorskip("gtfparse")` so base installs without `scatrans[gene_features]` skip instead of failing.

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/CITATION.cff

@@ -8,7 +8,7 @@ authors:
 repository-code: "https://github.com/scATrans/scatrans"
 url: "https://github.com/scATrans/scatrans"
 license: Apache-2.0
-version: 0.9.2
+version: 0.9.8
 keywords:
   - single-cell
   - RNA-seq

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scatrans
-Version: 0.9.8.dev0
+Version: 0.9.8.dev2
 Summary: Single-cell Active Transcription Analysis
 Author: scATrans Developers
 License: Apache-2.0
@@ -76,6 +76,18 @@ It also supports conventional differential expression workflows (no velocity dat
 
 The simple wrappers (`active_score_simple`, `run_default_pipeline`) keep permutation off by default so new users explore ranked tables first; enable `use_permutation=True` explicitly when you need FDR on unspliced excess.
 
+### Choosing a DE backend (decision guide)
+
+| Your design | Recommended backend | Caveats |
+|-------------|---------------------|---------|
+| Exploratory / default | scanpy `wilcoxon` or `t-test` on normalized data | Fast; standard pseudoreplication limits |
+| ≥2 biological replicates per group, aggregated counts | `use_pseudobulk=True` + `pydeseq2` | Requires raw counts (`store_raw_counts`); DESeq2 assumptions |
+| Few pseudobulk samples, no DESeq2 | `use_pseudobulk=True` + `pseudobulk_de_backend="scanpy"` | Non-parametric on aggregated profiles |
+| Cell-level data + true sample replicates | `use_mixed_model=True` + `sample_col` | Lightweight LMM (log1p); check `diagnostics["mixed_model"]["failed_fit_rate"]` — not NB-GLMM/voom |
+| Method-of-moments cell-level DE | `use_memento_de=True` | Raw integer counts required; compare `memento_p_adj_native` vs package `p_adj` if auditing |
+
+Always run `recommend_workflow(...)` first; inspect `adata.uns["scatrans"]["diagnostics"]` (bias, gamma, permutation `disabled_reason`) before publication claims.
+
 ## Quick Reference + Reporting Checklist (one page)
 
 | Step | Function | Key outputs |
@@ -793,16 +805,25 @@ Primary result columns use **unspliced / nascent excess** terminology (not RNA v
 
 ### Built-in `significant` gene list
 
-When `use_permutation=True`, the internal mask requires **all** of:
+When `use_permutation=True`, the built-in mask uses the same default thresholds as
+`filter_active_genes(..., preset="heuristic")` (see `HEURISTIC_FILTER_DEFAULTS` in
+`tl.py`). Under default parameters it requires **all** of:
 
-- `logFC > logfc_cutoff` (default 0.5)
+- `logFC > logfc_cutoff` (default **0.35**)
 - `p_adj < pval_cutoff` (default 0.05)
-- `unspliced_excess_residual > 0`
+- `unspliced_excess_residual > 1.0` (default residual magnitude cutoff)
+- `active_score >= 55.0`
+- `active_score_fdr < 0.25` (when permutation computed composite-score FDR)
 - `unspliced_excess_fdr < unspliced_excess_fdr_cutoff` (default 0.05)
 
-Without `use_permutation=True`, the built-in `significant` list is **empty** (FDR on unspliced excess cannot be computed). Use `all_results` + `filter_active_genes` for custom thresholds.
+Without `use_permutation=True`, the built-in `significant` list is **empty** (FDR on
+unspliced excess cannot be computed). Use `all_results` + `filter_active_genes` for
+custom thresholds.
 
-On real data the built-in list often returns zero or few genes. Use the full table in `all_results`, sorted by `active_score` descending.
+On low-signal data the built-in list may still be small. Use the full table in
+`all_results`, sorted by `active_score` descending. If you need different cutoffs,
+pass explicit arguments to `filter_active_genes` rather than assuming the built-in
+list matches a custom `logfc_cutoff` override on `active_score()`.
 
 After each run inspect the diagnostics:
 

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/README.md

@@ -25,6 +25,18 @@ It also supports conventional differential expression workflows (no velocity dat
 
 The simple wrappers (`active_score_simple`, `run_default_pipeline`) keep permutation off by default so new users explore ranked tables first; enable `use_permutation=True` explicitly when you need FDR on unspliced excess.
 
+### Choosing a DE backend (decision guide)
+
+| Your design | Recommended backend | Caveats |
+|-------------|---------------------|---------|
+| Exploratory / default | scanpy `wilcoxon` or `t-test` on normalized data | Fast; standard pseudoreplication limits |
+| ≥2 biological replicates per group, aggregated counts | `use_pseudobulk=True` + `pydeseq2` | Requires raw counts (`store_raw_counts`); DESeq2 assumptions |
+| Few pseudobulk samples, no DESeq2 | `use_pseudobulk=True` + `pseudobulk_de_backend="scanpy"` | Non-parametric on aggregated profiles |
+| Cell-level data + true sample replicates | `use_mixed_model=True` + `sample_col` | Lightweight LMM (log1p); check `diagnostics["mixed_model"]["failed_fit_rate"]` — not NB-GLMM/voom |
+| Method-of-moments cell-level DE | `use_memento_de=True` | Raw integer counts required; compare `memento_p_adj_native` vs package `p_adj` if auditing |
+
+Always run `recommend_workflow(...)` first; inspect `adata.uns["scatrans"]["diagnostics"]` (bias, gamma, permutation `disabled_reason`) before publication claims.
+
 ## Quick Reference + Reporting Checklist (one page)
 
 | Step | Function | Key outputs |
@@ -742,16 +754,25 @@ Primary result columns use **unspliced / nascent excess** terminology (not RNA v
 
 ### Built-in `significant` gene list
 
-When `use_permutation=True`, the internal mask requires **all** of:
+When `use_permutation=True`, the built-in mask uses the same default thresholds as
+`filter_active_genes(..., preset="heuristic")` (see `HEURISTIC_FILTER_DEFAULTS` in
+`tl.py`). Under default parameters it requires **all** of:
 
-- `logFC > logfc_cutoff` (default 0.5)
+- `logFC > logfc_cutoff` (default **0.35**)
 - `p_adj < pval_cutoff` (default 0.05)
-- `unspliced_excess_residual > 0`
+- `unspliced_excess_residual > 1.0` (default residual magnitude cutoff)
+- `active_score >= 55.0`
+- `active_score_fdr < 0.25` (when permutation computed composite-score FDR)
 - `unspliced_excess_fdr < unspliced_excess_fdr_cutoff` (default 0.05)
 
-Without `use_permutation=True`, the built-in `significant` list is **empty** (FDR on unspliced excess cannot be computed). Use `all_results` + `filter_active_genes` for custom thresholds.
+Without `use_permutation=True`, the built-in `significant` list is **empty** (FDR on
+unspliced excess cannot be computed). Use `all_results` + `filter_active_genes` for
+custom thresholds.
 
-On real data the built-in list often returns zero or few genes. Use the full table in `all_results`, sorted by `active_score` descending.
+On low-signal data the built-in list may still be small. Use the full table in
+`all_results`, sorted by `active_score` descending. If you need different cutoffs,
+pass explicit arguments to `filter_active_genes` rather than assuming the built-in
+list matches a custom `logfc_cutoff` override on `active_score()`.
 
 After each run inspect the diagnostics:
 

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/pyproject.toml

@@ -71,7 +71,7 @@ scatrans = [
 
 [tool.setuptools_scm]
 write_to = "src/scatrans/_version.py"
-fallback_version = "0.9.2"
+fallback_version = "0.9.8"
 version_scheme = "guess-next-dev"
 local_scheme = "no-local-version"
 

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/scatrans.egg-info/SOURCES.txt

@@ -41,10 +41,12 @@ tests/__init__.py
 tests/conftest.py
 tests/test_basic.py
 tests/test_de_backends.py
+tests/test_de_edge_cases.py
 tests/test_enrich_api.py
 tests/test_enrich_go.py
 tests/test_pl_coverage.py
 tests/test_pl_extended.py
 tests/test_pp_bias_cli.py
 tests/test_public_api.py
+tests/test_regression_bugs.py
 tests/test_tl_coverage.py

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/src/scatrans/__init__.py

@@ -87,7 +87,7 @@ __all__ = [
 try:
     from ._version import version as __version__
 except ImportError:
-    __version__ = "0.9.2"
+    __version__ = "0.9.8"
 
 # Optional: prevent some internal modules from appearing too prominently
 # in casual inspection while still allowing advanced users to do

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/src/scatrans/_de.py

@@ -9,6 +9,7 @@ from __future__ import annotations
 
 import logging
 import warnings
+from contextlib import contextmanager
 from importlib.metadata import version
 from typing import Any
 
@@ -29,6 +30,44 @@ from ._utils import (
 )
 
 logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())
+
+_DE_REQUIRED_COLS = frozenset({"logFC", "p_val", "p_adj"})
+
+# Suppress noisy deprecation/future warnings from scanpy/anndata/PyDESeq2 during DE,
+# but keep convergence/runtime/user warnings visible to callers.
+_DE_SUPPRESS_WARNING_CATEGORIES = (
+    DeprecationWarning,
+    FutureWarning,
+    PendingDeprecationWarning,
+)
+
+
+@contextmanager
+def _de_warning_context():
+    with warnings.catch_warnings():
+        for cat in _DE_SUPPRESS_WARNING_CATEGORIES:
+            warnings.simplefilter("ignore", category=cat)
+        yield
+
+
+def _validate_de_result(de_df: pd.DataFrame, *, backend: str) -> pd.DataFrame:
+    """Assert all DE backends return the minimal schema expected downstream."""
+    missing = _DE_REQUIRED_COLS - set(de_df.columns)
+    if missing:
+        raise RuntimeError(
+            f"DE backend '{backend}' returned incomplete results; "
+            f"missing columns: {sorted(missing)}"
+        )
+    if len(de_df) == 0:
+        return de_df
+    for col in _DE_REQUIRED_COLS:
+        vals = pd.to_numeric(de_df[col], errors="coerce")
+        if not np.isfinite(vals.to_numpy()).any():
+            raise RuntimeError(
+                f"DE backend '{backend}' returned no finite values in column {col!r}."
+            )
+    return de_df
 
 
 def _pydeseq2_uses_design_factors() -> bool:
@@ -94,15 +133,18 @@ def _run_de_wrapper(
     if use_mixed_model:
         if sample_col is None:
             raise ValueError("sample_col must be provided when use_mixed_model=True")
-        return _run_mixedlm_de(
-            adata,
-            groupby=groupby,
-            target_group=target_group,
-            reference_group=reference_group,
-            sample_col=sample_col,
-            n_jobs=n_jobs,
-            labels=labels,
-            mixed_model_pval=mixed_model_pval,
+        return _validate_de_result(
+            _run_mixedlm_de(
+                adata,
+                groupby=groupby,
+                target_group=target_group,
+                reference_group=reference_group,
+                sample_col=sample_col,
+                n_jobs=n_jobs,
+                labels=labels,
+                mixed_model_pval=mixed_model_pval,
+            ),
+            backend="mixedlm",
         )
 
     if use_memento_de:
@@ -111,16 +153,19 @@ def _run_de_wrapper(
                 "use_memento_de=True is not supported with use_pseudobulk=True "
                 "(Memento is a cell-level method-of-moments estimator; use PyDESeq2 for pseudobulk)."
             )
-        return _run_memento_de(
-            adata,
-            groupby=groupby,
-            target_group=target_group,
-            reference_group=reference_group,
-            labels=labels,
-            capture_rate=memento_capture_rate,
-            num_boot=memento_num_boot,
-            n_cpus=memento_n_cpus,
-            counts=counts,
+        return _validate_de_result(
+            _run_memento_de(
+                adata,
+                groupby=groupby,
+                target_group=target_group,
+                reference_group=reference_group,
+                labels=labels,
+                capture_rate=memento_capture_rate,
+                num_boot=memento_num_boot,
+                n_cpus=memento_n_cpus,
+                counts=counts,
+            ),
+            backend="memento",
         )
 
     ad_temp = adata.copy() if labels is not None else adata
@@ -220,8 +265,7 @@ def _run_de_wrapper(
             index=counts_use.index,
         )
 
-        with warnings.catch_warnings():
-            warnings.simplefilter("ignore")
+        with _de_warning_context():
             if _pydeseq2_uses_design_factors():
                 dds = DeseqDataSet(
                     counts=counts_use,
@@ -257,18 +301,48 @@ def _run_de_wrapper(
                 )
             stat_res.summary()
 
-        res2 = stat_res.results_df.copy().reindex(ad_temp.var_names)
+        res_df = stat_res.results_df.copy()
+        n_genes_filtered_low_count = int((~gene_keep).sum())
+        n_genes_nan_from_deseq2 = (
+            int(res_df["padj"].isna().sum()) if "padj" in res_df.columns else 0
+        )
+        res2 = res_df.reindex(ad_temp.var_names)
         de_df = pd.DataFrame(index=ad_temp.var_names)
-        de_df["logFC"] = res2["log2FoldChange"].fillna(0.0)
-        de_df["p_val"] = res2.get("pvalue", pd.Series(1.0, index=res2.index)).fillna(1.0)
-        de_df["p_adj"] = res2.get("padj", pd.Series(1.0, index=res2.index)).fillna(1.0)
+        de_df["logFC"] = res2["log2FoldChange"].reindex(ad_temp.var_names)
+        de_df["p_val"] = res2.get("pvalue", pd.Series(np.nan, index=res2.index)).reindex(
+            ad_temp.var_names
+        )
+        de_df["p_adj"] = res2.get("padj", pd.Series(np.nan, index=res2.index)).reindex(
+            ad_temp.var_names
+        )
+        _validate_de_result(de_df, backend="pydeseq2")
+        de_df["logFC"] = de_df["logFC"].fillna(0.0)
+        de_df["p_val"] = de_df["p_val"].fillna(1.0)
+        de_df["p_adj"] = de_df["p_adj"].fillna(1.0)
+        n_total = len(de_df)
+        de_df.attrs["n_genes_filtered_low_count"] = n_genes_filtered_low_count
+        de_df.attrs["n_genes_nan_from_deseq2"] = n_genes_nan_from_deseq2
+        de_df.attrs["pydeseq2_neutral_fill"] = True
+        if n_total > 0 and (n_genes_filtered_low_count > 0 or n_genes_nan_from_deseq2 > 0):
+            logger.warning(
+                "PyDESeq2: %d/%d genes (%.1f%%) skipped by min_counts_per_gene filter; "
+                "%d/%d genes (%.1f%%) had NaN padj from DESeq2 independent filtering/outliers. "
+                "These genes appear as neutral values (logFC=0, p_adj=1) and are NOT "
+                "'tested and non-significant'. "
+                "See de_df.attrs['n_genes_filtered_low_count'] and ['n_genes_nan_from_deseq2'].",
+                n_genes_filtered_low_count,
+                n_total,
+                100.0 * n_genes_filtered_low_count / n_total,
+                n_genes_nan_from_deseq2,
+                n_total,
+                100.0 * n_genes_nan_from_deseq2 / n_total,
+            )
         return de_df
 
     else:
         # Standard scanpy path (works for both regular and pseudobulk when not using pydeseq2)
         rank_key = "_scatrans_rank_genes_groups"
-        with warnings.catch_warnings():
-            warnings.simplefilter("ignore")
+        with _de_warning_context():
             sc.tl.rank_genes_groups(
                 ad_temp,
                 groupby=use_groupby,
@@ -281,13 +355,16 @@ def _run_de_wrapper(
             "names"
         )
         de_df = pd.DataFrame(index=ad_temp.var_names)
-        raw_lfc = de_raw["logfoldchanges"].reindex(ad_temp.var_names).fillna(0.0)
         # scanpy rank_genes_groups always returns logfoldchanges on log2 scale:
         # log2( (expm1(mean_t) + eps) / (expm1(mean_r) + eps) ), independent of the
         # statistical method (wilcoxon / t-test / etc.). No secondary conversion needed.
-        de_df["logFC"] = raw_lfc
-        de_df["p_val"] = de_raw["pvals"].reindex(ad_temp.var_names).fillna(1.0)
-        de_df["p_adj"] = de_raw["pvals_adj"].reindex(ad_temp.var_names).fillna(1.0)
+        de_df["logFC"] = de_raw["logfoldchanges"].reindex(ad_temp.var_names)
+        de_df["p_val"] = de_raw["pvals"].reindex(ad_temp.var_names)
+        de_df["p_adj"] = de_raw["pvals_adj"].reindex(ad_temp.var_names)
+        _validate_de_result(de_df, backend=f"scanpy:{de_method}")
+        de_df["logFC"] = de_df["logFC"].fillna(0.0)
+        de_df["p_val"] = de_df["p_val"].fillna(1.0)
+        de_df["p_adj"] = de_df["p_adj"].fillna(1.0)
         return de_df
 
 
@@ -354,13 +431,12 @@ def _run_mixedlm_de(
     # The explicit flag avoids counting truly neutral (logFC~0, p~1, dvar~0) biological genes as "failed".
     def _fit_gene_mixed(idx: int):
         y = expr_mat[:, idx].astype(float)
-        # guard against all-zero / constant (mixedlm will be singular)
-        if np.allclose(y, y[0]):
+        # guard against near-constant expression (mixedlm will be singular)
+        if float(np.nanvar(y)) < 1e-12:
             return idx, 0.0, 1.0, 1.0, 0.0, True
         df = pd.DataFrame({"y": y, "condition": condition, "sample": samples})
         try:
-            with warnings.catch_warnings():
-                warnings.simplefilter("ignore")
+            with _de_warning_context():
                 # Full model
                 md_full = smf.mixedlm("y ~ C(condition)", df, groups=df["sample"])
                 m_full = md_full.fit(reml=False, maxiter=200, disp=False)
@@ -369,23 +445,18 @@ def _run_mixedlm_de(
                 md_null = smf.mixedlm("y ~ 1", df, groups=df["sample"])
                 m_null = md_null.fit(reml=False, maxiter=200, disp=False)
 
+            if not getattr(m_full, "converged", True) or not getattr(m_null, "converged", True):
+                return idx, 0.0, 1.0, 1.0, 0.0, True
+
             # LRT statistic and p (chi2 df=1 for the added fixed effect term(s))
             lrt_stat = -2.0 * (m_null.llf - m_full.llf)
             lrt_p = float(chi2.sf(max(lrt_stat, 0.0), 1))
 
-            # Extract condition coef (target vs ref)
-            # The param name is typically "C(condition)[T.<target>]" or similar
-            coef_name = None
-            for pname in m_full.params.index:
-                if "condition" in str(pname) and target_group in str(pname):
-                    coef_name = pname
-                    break
+            # Extract condition coef (target vs ref) — exact statsmodels name only
+            expected_coef = f"C(condition)[T.{target_group}]"
+            coef_name = expected_coef if expected_coef in m_full.params.index else None
             if coef_name is None:
-                # fallback: take the second coef if intercept + one more
-                if len(m_full.params) >= 2:
-                    coef_name = m_full.params.index[1]
-                else:
-                    coef_name = m_full.params.index[0]
+                return idx, 0.0, 1.0, 1.0, 0.0, True
             logfc = float(m_full.params.get(coef_name, 0.0))
             p_wald = float(m_full.pvalues.get(coef_name, 1.0))
 
@@ -441,8 +512,7 @@ def _run_mixedlm_de(
             logger.warning("mixed_model_pval must be 'wald' or 'lrt'; falling back to 'wald'.")
         main_pvals = p_walds
 
-    with warnings.catch_warnings():
-        warnings.simplefilter("ignore")
+    with _de_warning_context():
         p_adjs = multipletests(main_pvals, method="fdr_bh")[1]
 
     de_df = pd.DataFrame(index=var_names)
@@ -456,14 +526,19 @@ def _run_mixedlm_de(
     # Use explicit failure flag returned from workers (avoids counting true biological
     # neutral genes (logFC~0, p~1, dvar~0) as "failed fits").
     n_failed = int(np.sum(failed_flags))
+    n_total = len(de_df)
+    failed_rate = (n_failed / n_total) if n_total else 0.0
     de_df.attrs["n_genes_failed_fit"] = n_failed
+    de_df.attrs["failed_fit_rate"] = failed_rate
     if n_failed > 0:
         logger.warning(
-            "MixedLM: %d/%d genes had degenerate fits (e.g. small n per group, collinearity) "
+            "MixedLM: %d/%d genes (%.1f%%) had degenerate or non-convergent fits "
+            "(near-constant expression, singular Hessian, missing condition coefficient, etc.) "
             "and received neutral values (logFC=0, p_val=1, delta_variance=0). "
             "See diagnostics['mixed_model']['n_genes_failed_fit'].",
             n_failed,
-            len(de_df),
+            n_total,
+            100.0 * failed_rate,
         )
     return de_df
 
@@ -535,6 +610,15 @@ def _run_memento_de(
             else:
                 raise ValueError(f"counts='{counts}' layer not found in adata.layers")
         elif isinstance(counts, ad.AnnData):
+            if not np.array_equal(counts.obs_names, ad_temp.obs_names):
+                missing = ad_temp.obs_names.difference(counts.obs_names)
+                if len(missing):
+                    raise ValueError(
+                        f"counts AnnData is missing {len(missing)} cell(s) required by the "
+                        f"comparison subset (first missing: {missing[0]!r}). "
+                        "Pass counts aligned to adata.obs_names or use a matrix layer."
+                    )
+                counts = counts[ad_temp.obs_names]
             raw_counts = counts.X
             if counts.var_names.tolist() != ad_temp.var_names.tolist():
                 common = counts.var_names.intersection(ad_temp.var_names)
@@ -637,13 +721,40 @@ def _run_memento_de(
         pval_raw = result["de_pval"]
     else:
         pval_raw = pd.Series(1.0, index=res_index)
-    pvals = pd.to_numeric(pval_raw, errors="coerce").reindex(res_index).fillna(1.0)
-    de_df["p_val"] = pvals
-
-    # BH adjustment (Memento may return raw p; we make p_adj consistent with other backends)
-    with warnings.catch_warnings():
-        warnings.simplefilter("ignore")
-        de_df["p_adj"] = multipletests(pvals.values, method="fdr_bh")[1]
+    pvals_for_correction = pd.to_numeric(pval_raw, errors="coerce").reindex(res_index)
+    valid_pval_mask = pvals_for_correction.notna()
+    de_df["p_val"] = pvals_for_correction.fillna(1.0)
+
+    # Preserve Memento-native adjusted p if present (audit trail); package p_adj uses BH for consistency.
+    for native_col in ("de_padj", "de_padjs", "padj", "p_adj", "de_fdr"):
+        if isinstance(result, pd.DataFrame) and native_col in result.columns:
+            de_df["memento_p_adj_native"] = pd.to_numeric(
+                result[native_col], errors="coerce"
+            ).reindex(res_index)
+            break
+
+    de_df["p_adj"] = 1.0
+    if valid_pval_mask.sum() > 0:
+        with _de_warning_context():
+            de_df.loc[valid_pval_mask, "p_adj"] = multipletests(
+                pvals_for_correction[valid_pval_mask].values, method="fdr_bh"
+            )[1]
+
+    # Memento typically drops low-coverage genes entirely (not NaN rows). Genes absent
+    # from the memento result are reindexed below with neutral fill — same UX issue as
+    # PyDESeq2 filtered genes, but via row omission rather than NaN padj.
+    n_genes_not_returned = int(len(adata.var_names.difference(res_index)))
+    de_df.attrs["n_genes_not_returned_by_memento"] = n_genes_not_returned
+    de_df.attrs["n_genes_missing_pval"] = int((~valid_pval_mask).sum())
+    if n_genes_not_returned > 0:
+        logger.warning(
+            "Memento: %d/%d genes were not returned by binary_test_1d (e.g. internal "
+            "min_perc_group filtering) and will appear as neutral values (logFC=0, p_adj=1) "
+            "after reindexing. These were not tested. "
+            "See de_df.attrs['n_genes_not_returned_by_memento'].",
+            n_genes_not_returned,
+            len(adata.var_names),
+        )
 
     # Expose Memento's native columns for users who want mean + variability signals
     for src, dst in [

{scatrans-0.9.8.dev0 → scatrans-0.9.8.dev2}/src/scatrans/_permutation.py

@@ -26,6 +26,46 @@ from ._utils import (
 from ._velocity import _compute_velocity_delta
 
 logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())
+
+
+def _compute_perm_velocity_delta(
+    *,
+    velocity_source: str,
+    uns_layer: Any,
+    spl_layer: Any,
+    t_mask: np.ndarray,
+    r_mask: np.ndarray,
+    prior_weight: float,
+    gamma_method: str,
+    eb_prior: dict[str, Any] | None,
+) -> tuple[np.ndarray, np.ndarray, np.ndarray, dict[str, Any]]:
+    """Velocity delta for one permutation replicate.
+
+    For ``scvelo_moments*`` sources the layers are pre-smoothed Mu/Ms from the main
+    analysis (label permutation keeps smoothed values fixed). For heuristic sources
+    they are library-size-normalized unspliced/spliced layers.
+    """
+    if velocity_source.startswith("scvelo_moments"):
+        # Same groupwise ratio estimator as _compute_moments_velocity_delta (on Mu/Ms).
+        return _compute_velocity_delta(
+            uns_layer,
+            spl_layer,
+            t_mask,
+            r_mask,
+            prior_weight,
+            gamma_method=gamma_method,
+            eb_prior=eb_prior,
+        )
+    return _compute_velocity_delta(
+        uns_layer,
+        spl_layer,
+        t_mask,
+        r_mask,
+        prior_weight,
+        gamma_method=gamma_method,
+        eb_prior=eb_prior,
+    )
 
 
 def _single_permutation_task(
@@ -60,6 +100,7 @@ def _single_permutation_task(
     memento_capture_rate: float = 0.07,
     memento_num_boot: int = 5000,
     memento_n_cpus: int = -1,
+    velocity_source: str = "heuristic_global_ratio",
 ) -> tuple[np.ndarray, np.ndarray]:
     """One permutation replicate.
 
@@ -108,12 +149,13 @@ def _single_permutation_task(
 
     t_mask = shuffled_labels == target_group
     r_mask = shuffled_labels == reference_group
-    delta_velocity, _, _gamma_ref, _ = _compute_velocity_delta(
-        uns_layer,
-        spl_layer,
-        t_mask,
-        r_mask,
-        prior_weight,
+    delta_velocity, _, _gamma_ref, _ = _compute_perm_velocity_delta(
+        velocity_source=velocity_source,
+        uns_layer=uns_layer,
+        spl_layer=spl_layer,
+        t_mask=t_mask,
+        r_mask=r_mask,
+        prior_weight=prior_weight,
         gamma_method=gamma_method,
         eb_prior=eb_prior,
     )
@@ -177,6 +219,7 @@ def run_permutation_test(
     real_score: np.ndarray,
     real_residual: np.ndarray,
     eb_prior: dict[str, Any] | None = None,
+    velocity_source: str = "heuristic_global_ratio",
     bias_correction: str = "huber_length_intron",
     # Memento forwarding for advanced consistent permutation
     use_memento_de: bool = False,
@@ -224,6 +267,7 @@ def run_permutation_test(
                 de_preprocess,
                 strict_pydeseq2_counts,
                 eb_prior,
+                velocity_source=velocity_source,
                 bias_correction=bias_correction,
                 use_memento_de=use_memento_de,
                 memento_capture_rate=memento_capture_rate,
@@ -287,7 +331,7 @@ def run_permutation_test(
     # - For very small permutation spaces (n_perm < 100, common in pseudobulk with few samples),
     #   we mark use_fdr=False and provide reason so callers can avoid using FDR for significance
     #   (p-values become coarse; BH adjustment less reliable). This eliminates vestigial logic in tl.py.
-    use_fdr = n_perm >= 100
+    use_fdr = n_success >= 100
     disabled_reason = None if use_fdr else "small_permutation_space"
 
     return (