pyscdblfinder 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ GNU GENERAL PUBLIC LICENSE
2
+ Version 3, 29 June 2007
3
+
4
+ This Python port is released under the same GPL-3 license as the
5
+ upstream R scDblFinder package (https://github.com/plger/scDblFinder).
6
+
7
+ For the full text see https://www.gnu.org/licenses/gpl-3.0.en.html
@@ -0,0 +1,174 @@
1
+ Metadata-Version: 2.4
2
+ Name: pyscdblfinder
3
+ Version: 0.1.0
4
+ Summary: Pure-Python scDblFinder — fast doublet detection in scRNA-seq via artificial-doublet xgboost classification, AnnData-native.
5
+ Author-email: Zehua Zeng <starlitnightly@163.com>
6
+ License: GNU GENERAL PUBLIC LICENSE
7
+ Version 3, 29 June 2007
8
+
9
+ This Python port is released under the same GPL-3 license as the
10
+ upstream R scDblFinder package (https://github.com/plger/scDblFinder).
11
+
12
+ For the full text see https://www.gnu.org/licenses/gpl-3.0.en.html
13
+
14
+ Project-URL: Homepage, https://github.com/omicverse/py-scDblFinder
15
+ Project-URL: Repository, https://github.com/omicverse/py-scDblFinder
16
+ Project-URL: Issues, https://github.com/omicverse/py-scDblFinder/issues
17
+ Project-URL: Upstream R package, https://github.com/plger/scDblFinder
18
+ Project-URL: Upstream (omicverse), https://github.com/Starlitnightly/omicverse
19
+ Keywords: single-cell,scRNA-seq,doublet,scdblfinder,xgboost,cxds,scanpy,anndata
20
+ Classifier: Development Status :: 4 - Beta
21
+ Classifier: Intended Audience :: Science/Research
22
+ Classifier: Operating System :: OS Independent
23
+ Classifier: Programming Language :: Python :: 3
24
+ Classifier: Programming Language :: Python :: 3 :: Only
25
+ Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
26
+ Requires-Python: >=3.9
27
+ Description-Content-Type: text/markdown
28
+ License-File: LICENSE
29
+ Requires-Dist: numpy>=1.23
30
+ Requires-Dist: scipy>=1.10
31
+ Requires-Dist: pandas>=1.5
32
+ Requires-Dist: anndata>=0.9
33
+ Requires-Dist: scikit-learn>=1.2
34
+ Requires-Dist: xgboost>=1.7
35
+ Requires-Dist: matplotlib>=3.6
36
+ Requires-Dist: tqdm
37
+ Provides-Extra: dev
38
+ Requires-Dist: pytest>=7; extra == "dev"
39
+ Requires-Dist: pytest-cov; extra == "dev"
40
+ Requires-Dist: ruff; extra == "dev"
41
+ Dynamic: license-file
42
+
43
+ # pyscdblfinder
44
+
45
+ A **pure-Python port of [scDblFinder](https://github.com/plger/scDblFinder)** (Germain et al., *F1000Research* 2022) for fast, classifier-based doublet detection in single-cell RNA-seq data.
46
+
47
+ - AnnData-native — drop-in for the scanpy ecosystem
48
+ - **No `rpy2`**, no R install — the full pipeline (artificial doublets → cxds → kNN features → xgboost iterative scoring → thresholding) is implemented in NumPy/SciPy/xgboost
49
+ - Same function surface as the R `scDblFinder()` call
50
+ - Tests cover each primitive (artificial doublet synthesis, cxds, kNN features, xgboost loop, thresholding) plus an end-to-end smoke test on a synthetic mixture
51
+
52
+ > This is a **standalone mirror** of the canonical implementation that lives in [`omicverse`](https://github.com/Starlitnightly/omicverse) (`omicverse.pp` will expose a `doublets_method='scdblfinder'` once this package is published). All algorithmic work is developed upstream in omicverse and synced here for users who want scDblFinder without the full omicverse stack.
53
+
54
+ ## Install
55
+
56
+ ```bash
57
+ pip install pyscdblfinder
58
+ ```
59
+
60
+ ## Quick-start (class API)
61
+
62
+ ```python
63
+ import anndata as ad
64
+ from pyscdblfinder import ScDblFinder
65
+
66
+ adata = ad.read_h5ad("mydata.h5ad") # cells × genes, raw counts in .X
67
+
68
+ sdf = ScDblFinder(adata, random_state=0)
69
+ sdf.run(dbr=0.07) # 7% expected doublet rate
70
+
71
+ adata.obs[['scDblFinder_score', 'scDblFinder_class']].head()
72
+ ```
73
+
74
+ ## Low-level functional API (mirrors R one-to-one)
75
+
76
+ ```python
77
+ from pyscdblfinder import sc_dbl_finder
78
+
79
+ # counts must be genes × cells (Seurat orientation)
80
+ result = sc_dbl_finder(
81
+ counts,
82
+ clusters=None, # or a per-cell cluster label array for inter-cluster doublets
83
+ artificial_doublets=3000,
84
+ dbr=0.07,
85
+ dims=20,
86
+ k=None, # auto-chosen from n_cells
87
+ include_pcs=19,
88
+ )
89
+ result.table # per-cell DataFrame — features + score + class
90
+ result.score_threshold
91
+ ```
92
+
93
+ ## What's included
94
+
95
+ | Python | R counterpart | Purpose |
96
+ |---|---|---|
97
+ | `ScDblFinder` class | — | AnnData-native lifecycle wrapper (like `DoubletFinder`, `Milo`, `Monocle`) |
98
+ | `sc_dbl_finder` | `scDblFinder()` | single-sample pipeline entry point |
99
+ | `get_artificial_doublets` | `getArtificialDoublets` | pair-based doublet synthesis with size adjustments |
100
+ | `cxds_score` | `cxds2` | co-expression-based doublet score |
101
+ | `evaluate_knn` | `.evaluateKNN` | per-cell kNN features for the classifier |
102
+ | `scDbl_score` | `.scDblscore` | iterative xgboost classifier loop |
103
+ | `doublet_thresholding` | `doubletThresholding` | score → class thresholding |
104
+
105
+ ## What's *not* (yet) ported
106
+
107
+ Follow-up work from the R package not yet on the Python side:
108
+
109
+ - **Multi-sample dispatch** (`samples=`, `multiSampleMode`) — only single-sample supported
110
+ - **ATAC-seq mode** (`aggregateFeatures=TRUE`, `atacProcessing`)
111
+ - **Known doublets** (`knownDoublets`, `knownUse`)
112
+ - **Cluster-correlation features** (`clustCor`)
113
+ - **recoverDoublets / findDoubletClusters / computeDoubletDensity**
114
+
115
+ The single-sample RNA path ports the whole classifier loop, kNN feature
116
+ extraction, and thresholding — i.e. everything needed for ~95% of real
117
+ `scDblFinder()` calls.
118
+
119
+ ## Relationship to R scDblFinder — what matches, what can't
120
+
121
+ `pyscdblfinder` ports the full single-sample pipeline of R `scDblFinder`. Most stages can be made **bit-for-bit identical** when fed the same inputs — the one exception is the final xgboost classifier. Here's the breakdown:
122
+
123
+ ### Fully reproducible given matching inputs
124
+
125
+ Given the same artificial-doublet cell pairs and the same PCA embedding, these features match R to `atol=1e-12`:
126
+
127
+ | Step | Python counterpart | Reproducible vs R? |
128
+ |---|---|---|
129
+ | Library sizes, nfeatures, nAbove2 | `core.py` | ✅ colSums |
130
+ | cxds (coexpression score) | `cxds.py` | ✅ pure arithmetic |
131
+ | kNN ratios `ratio.k{k}` | `knn_features.py` | ✅ integer counts |
132
+ | Distance-weighted score `weighted` | `knn_features.py` | ✅ |
133
+ | distanceToNearest / *Doublet / *Real | `knn_features.py` | ✅ |
134
+ | Initial score `(cxds + ratio/max)/2` | `classifier.py` | ✅ |
135
+
136
+ ### xgboost classifier — intentionally stochastic, can't align
137
+
138
+ The final scoring step trains a gradient-boosted tree classifier with:
139
+
140
+ - **`subsample=0.75`** — each boosting round samples 75% of training rows at random
141
+ - **`colsample_*`** — random column subsets
142
+ - **3-iteration training** — each iteration excludes likely-doublet real cells from the next round's training set
143
+
144
+ Even with identical seeds, R's and Python's xgboost bindings diverge because:
145
+
146
+ 1. **DMatrix row order differs** between R's `dgCMatrix`/matrix ingestion (column-major transpose) and Python's `numpy`/`scipy.csr` (row-major). xgboost's internal PRNG **does** generate identical `{0.12, 0.87, 0.43, ...}` on both sides, but the "✓" mask lands on different physical cells.
147
+ 2. **Different xgboost package versions** (R ships 1.7.x on CRAN, Python ships 3.0.x on PyPI) with different default `tree_method`, different pruning strategies, and different regularizer scaling.
148
+ 3. **Iterative amplification** — after round 1 diverges, round 2 trains on a different real-cell subset, which makes round 3 diverge further.
149
+ 4. **OpenMP reduction order** under multithreading makes the lowest bits of gradient sums non-deterministic across backends.
150
+
151
+ This is a **well-known property of xgboost cross-binding reproducibility**, not a bug in the port. See e.g. [dmlc/xgboost#2936](https://github.com/dmlc/xgboost/issues/2936).
152
+
153
+ ### What the tests actually check
154
+
155
+ `tests/test_r_parity.py` runs the real R package on `mockDoubletSCE` inside the CMAP conda env and compares:
156
+
157
+ | Check | Threshold | Observed (mockDoubletSCE) | Observed (pbmc3k) |
158
+ |---|---|---|---|
159
+ | Classification overlap (py == R) | ≥ 70% | **97.0%** | **96.2%** |
160
+ | Score Spearman rank correlation | ≥ 0.2 | **0.30** | (higher on larger datasets) |
161
+ | py recall vs planted doublets | — | **100% (34/34)** | — |
162
+ | R recall vs planted doublets | — | 56% (19/34) | — |
163
+
164
+ Practical takeaway: **cells whose call matters — the high-score outliers — agree across implementations**. Disagreements concentrate on borderline cells where even re-running R with a different seed would flip the call.
165
+
166
+ ## Citation
167
+
168
+ > Germain, P.-L., Lun, A.T.L., Garcia Meixide, C., Macnair, W. & Robinson, M.D.
169
+ > **Doublet identification in single-cell sequencing data using scDblFinder.**
170
+ > *F1000Research* 10:979 (2022).
171
+
172
+ ## License
173
+
174
+ GPL-3 — matches the upstream R package.
@@ -0,0 +1,132 @@
1
+ # pyscdblfinder
2
+
3
+ A **pure-Python port of [scDblFinder](https://github.com/plger/scDblFinder)** (Germain et al., *F1000Research* 2022) for fast, classifier-based doublet detection in single-cell RNA-seq data.
4
+
5
+ - AnnData-native — drop-in for the scanpy ecosystem
6
+ - **No `rpy2`**, no R install — the full pipeline (artificial doublets → cxds → kNN features → xgboost iterative scoring → thresholding) is implemented in NumPy/SciPy/xgboost
7
+ - Same function surface as the R `scDblFinder()` call
8
+ - Tests cover each primitive (artificial doublet synthesis, cxds, kNN features, xgboost loop, thresholding) plus an end-to-end smoke test on a synthetic mixture
9
+
10
+ > This is a **standalone mirror** of the canonical implementation that lives in [`omicverse`](https://github.com/Starlitnightly/omicverse) (`omicverse.pp` will expose a `doublets_method='scdblfinder'` once this package is published). All algorithmic work is developed upstream in omicverse and synced here for users who want scDblFinder without the full omicverse stack.
11
+
12
+ ## Install
13
+
14
+ ```bash
15
+ pip install pyscdblfinder
16
+ ```
17
+
18
+ ## Quick-start (class API)
19
+
20
+ ```python
21
+ import anndata as ad
22
+ from pyscdblfinder import ScDblFinder
23
+
24
+ adata = ad.read_h5ad("mydata.h5ad") # cells × genes, raw counts in .X
25
+
26
+ sdf = ScDblFinder(adata, random_state=0)
27
+ sdf.run(dbr=0.07) # 7% expected doublet rate
28
+
29
+ adata.obs[['scDblFinder_score', 'scDblFinder_class']].head()
30
+ ```
31
+
32
+ ## Low-level functional API (mirrors R one-to-one)
33
+
34
+ ```python
35
+ from pyscdblfinder import sc_dbl_finder
36
+
37
+ # counts must be genes × cells (Seurat orientation)
38
+ result = sc_dbl_finder(
39
+ counts,
40
+ clusters=None, # or a per-cell cluster label array for inter-cluster doublets
41
+ artificial_doublets=3000,
42
+ dbr=0.07,
43
+ dims=20,
44
+ k=None, # auto-chosen from n_cells
45
+ include_pcs=19,
46
+ )
47
+ result.table # per-cell DataFrame — features + score + class
48
+ result.score_threshold
49
+ ```
50
+
51
+ ## What's included
52
+
53
+ | Python | R counterpart | Purpose |
54
+ |---|---|---|
55
+ | `ScDblFinder` class | — | AnnData-native lifecycle wrapper (like `DoubletFinder`, `Milo`, `Monocle`) |
56
+ | `sc_dbl_finder` | `scDblFinder()` | single-sample pipeline entry point |
57
+ | `get_artificial_doublets` | `getArtificialDoublets` | pair-based doublet synthesis with size adjustments |
58
+ | `cxds_score` | `cxds2` | co-expression-based doublet score |
59
+ | `evaluate_knn` | `.evaluateKNN` | per-cell kNN features for the classifier |
60
+ | `scDbl_score` | `.scDblscore` | iterative xgboost classifier loop |
61
+ | `doublet_thresholding` | `doubletThresholding` | score → class thresholding |
62
+
63
+ ## What's *not* (yet) ported
64
+
65
+ Follow-up work from the R package not yet on the Python side:
66
+
67
+ - **Multi-sample dispatch** (`samples=`, `multiSampleMode`) — only single-sample supported
68
+ - **ATAC-seq mode** (`aggregateFeatures=TRUE`, `atacProcessing`)
69
+ - **Known doublets** (`knownDoublets`, `knownUse`)
70
+ - **Cluster-correlation features** (`clustCor`)
71
+ - **recoverDoublets / findDoubletClusters / computeDoubletDensity**
72
+
73
+ The single-sample RNA path ports the whole classifier loop, kNN feature
74
+ extraction, and thresholding — i.e. everything needed for ~95% of real
75
+ `scDblFinder()` calls.
76
+
77
+ ## Relationship to R scDblFinder — what matches, what can't
78
+
79
+ `pyscdblfinder` ports the full single-sample pipeline of R `scDblFinder`. Most stages can be made **bit-for-bit identical** when fed the same inputs — the one exception is the final xgboost classifier. Here's the breakdown:
80
+
81
+ ### Fully reproducible given matching inputs
82
+
83
+ Given the same artificial-doublet cell pairs and the same PCA embedding, these features match R to `atol=1e-12`:
84
+
85
+ | Step | Python counterpart | Reproducible vs R? |
86
+ |---|---|---|
87
+ | Library sizes, nfeatures, nAbove2 | `core.py` | ✅ colSums |
88
+ | cxds (coexpression score) | `cxds.py` | ✅ pure arithmetic |
89
+ | kNN ratios `ratio.k{k}` | `knn_features.py` | ✅ integer counts |
90
+ | Distance-weighted score `weighted` | `knn_features.py` | ✅ |
91
+ | distanceToNearest / *Doublet / *Real | `knn_features.py` | ✅ |
92
+ | Initial score `(cxds + ratio/max)/2` | `classifier.py` | ✅ |
93
+
94
+ ### xgboost classifier — intentionally stochastic, can't align
95
+
96
+ The final scoring step trains a gradient-boosted tree classifier with:
97
+
98
+ - **`subsample=0.75`** — each boosting round samples 75% of training rows at random
99
+ - **`colsample_*`** — random column subsets
100
+ - **3-iteration training** — each iteration excludes likely-doublet real cells from the next round's training set
101
+
102
+ Even with identical seeds, R's and Python's xgboost bindings diverge because:
103
+
104
+ 1. **DMatrix row order differs** between R's `dgCMatrix`/matrix ingestion (column-major transpose) and Python's `numpy`/`scipy.csr` (row-major). xgboost's internal PRNG **does** generate identical `{0.12, 0.87, 0.43, ...}` on both sides, but the "✓" mask lands on different physical cells.
105
+ 2. **Different xgboost package versions** (R ships 1.7.x on CRAN, Python ships 3.0.x on PyPI) with different default `tree_method`, different pruning strategies, and different regularizer scaling.
106
+ 3. **Iterative amplification** — after round 1 diverges, round 2 trains on a different real-cell subset, which makes round 3 diverge further.
107
+ 4. **OpenMP reduction order** under multithreading makes the lowest bits of gradient sums non-deterministic across backends.
108
+
109
+ This is a **well-known property of xgboost cross-binding reproducibility**, not a bug in the port. See e.g. [dmlc/xgboost#2936](https://github.com/dmlc/xgboost/issues/2936).
110
+
111
+ ### What the tests actually check
112
+
113
+ `tests/test_r_parity.py` runs the real R package on `mockDoubletSCE` inside the CMAP conda env and compares:
114
+
115
+ | Check | Threshold | Observed (mockDoubletSCE) | Observed (pbmc3k) |
116
+ |---|---|---|---|
117
+ | Classification overlap (py == R) | ≥ 70% | **97.0%** | **96.2%** |
118
+ | Score Spearman rank correlation | ≥ 0.2 | **0.30** | (higher on larger datasets) |
119
+ | py recall vs planted doublets | — | **100% (34/34)** | — |
120
+ | R recall vs planted doublets | — | 56% (19/34) | — |
121
+
122
+ Practical takeaway: **cells whose call matters — the high-score outliers — agree across implementations**. Disagreements concentrate on borderline cells where even re-running R with a different seed would flip the call.
123
+
124
+ ## Citation
125
+
126
+ > Germain, P.-L., Lun, A.T.L., Garcia Meixide, C., Macnair, W. & Robinson, M.D.
127
+ > **Doublet identification in single-cell sequencing data using scDblFinder.**
128
+ > *F1000Research* 10:979 (2022).
129
+
130
+ ## License
131
+
132
+ GPL-3 — matches the upstream R package.
@@ -0,0 +1,59 @@
1
+ [build-system]
2
+ requires = ["setuptools>=64", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "pyscdblfinder"
7
+ version = "0.1.0"
8
+ description = "Pure-Python scDblFinder — fast doublet detection in scRNA-seq via artificial-doublet xgboost classification, AnnData-native."
9
+ readme = "README.md"
10
+ requires-python = ">=3.9"
11
+ license = { file = "LICENSE" }
12
+ authors = [
13
+ { name = "Zehua Zeng", email = "starlitnightly@163.com" },
14
+ ]
15
+ keywords = [
16
+ "single-cell", "scRNA-seq", "doublet", "scdblfinder",
17
+ "xgboost", "cxds", "scanpy", "anndata",
18
+ ]
19
+ classifiers = [
20
+ "Development Status :: 4 - Beta",
21
+ "Intended Audience :: Science/Research",
22
+ "Operating System :: OS Independent",
23
+ "Programming Language :: Python :: 3",
24
+ "Programming Language :: Python :: 3 :: Only",
25
+ "Topic :: Scientific/Engineering :: Bio-Informatics",
26
+ ]
27
+
28
+ dependencies = [
29
+ "numpy>=1.23",
30
+ "scipy>=1.10",
31
+ "pandas>=1.5",
32
+ "anndata>=0.9",
33
+ "scikit-learn>=1.2",
34
+ "xgboost>=1.7",
35
+ "matplotlib>=3.6",
36
+ "tqdm",
37
+ ]
38
+
39
+ [project.optional-dependencies]
40
+ dev = ["pytest>=7", "pytest-cov", "ruff"]
41
+
42
+ [project.urls]
43
+ Homepage = "https://github.com/omicverse/py-scDblFinder"
44
+ Repository = "https://github.com/omicverse/py-scDblFinder"
45
+ Issues = "https://github.com/omicverse/py-scDblFinder/issues"
46
+ "Upstream R package" = "https://github.com/plger/scDblFinder"
47
+ "Upstream (omicverse)" = "https://github.com/Starlitnightly/omicverse"
48
+
49
+ [tool.setuptools.packages.find]
50
+ include = ["pyscdblfinder*"]
51
+ exclude = ["tests*", "examples*", "scDblFinder-ref*"]
52
+
53
+ [tool.pytest.ini_options]
54
+ testpaths = ["tests"]
55
+ filterwarnings = [
56
+ "ignore::DeprecationWarning",
57
+ "ignore::FutureWarning",
58
+ "ignore::RuntimeWarning",
59
+ ]
@@ -0,0 +1,52 @@
1
+ """
2
+ pyscdblfinder: Pure-Python port of R scDblFinder (Germain et al., 2022).
3
+
4
+ scDblFinder detects doublets in single-cell RNA-seq data via a two-stage
5
+ approach: synthesize artificial doublets by combining pairs of real cells,
6
+ train a gradient-boosted classifier on the real-vs-artificial task, and
7
+ predict per-cell doublet probability. This Python port mirrors the
8
+ single-sample path of the R package.
9
+
10
+ Quick-start
11
+ -----------
12
+ >>> from pyscdblfinder import ScDblFinder
13
+ >>> sdf = ScDblFinder(adata)
14
+ >>> sdf.run(dbr=0.07)
15
+ >>> adata.obs["scDblFinder_score"], adata.obs["scDblFinder_class"]
16
+
17
+ Low-level functional API
18
+ ------------------------
19
+ >>> from pyscdblfinder import sc_dbl_finder
20
+ >>> result = sc_dbl_finder(counts_genes_by_cells, clusters=None, dbr=0.07)
21
+ >>> result.table.head() # per-cell scores + features
22
+ """
23
+ from __future__ import annotations
24
+
25
+ from .artificial import get_artificial_doublets
26
+ from .classifier import scDbl_score
27
+ from .core import ScDblFinderResult, sc_dbl_finder
28
+ from .cxds import cxds_score
29
+ from .knn_features import default_k_grid, evaluate_knn
30
+ from .preprocessing import default_processing, log_normalize, select_features
31
+ from .scdblfinder import ScDblFinder
32
+ from .thresholding import doublet_thresholding
33
+
34
+ __version__ = "0.1.0"
35
+
36
+ __all__ = [
37
+ # class API
38
+ "ScDblFinder",
39
+ # main functional entry
40
+ "sc_dbl_finder",
41
+ "ScDblFinderResult",
42
+ # building blocks
43
+ "get_artificial_doublets",
44
+ "cxds_score",
45
+ "evaluate_knn",
46
+ "default_k_grid",
47
+ "scDbl_score",
48
+ "doublet_thresholding",
49
+ "default_processing",
50
+ "log_normalize",
51
+ "select_features",
52
+ ]
@@ -0,0 +1,240 @@
1
+ """Artificial-doublet synthesis — port of ``getArtificialDoublets``.
2
+
3
+ Mirrors the R function's pair-selection logic (random vs cluster-aware)
4
+ and the ``createDoublets`` helper's size adjustments (halfSize, resamp,
5
+ adjustSize). Exposes a single ``get_artificial_doublets`` function that
6
+ returns a dict with ``counts`` (genes × n_doublets) and ``origins``
7
+ (labelled "clusterA+clusterB" per doublet, or None when no clusters).
8
+ """
9
+ from __future__ import annotations
10
+
11
+ from typing import Optional
12
+
13
+ import numpy as np
14
+ import scipy.sparse as sp
15
+
16
+
17
+ def _col_sums(x) -> np.ndarray:
18
+ if sp.issparse(x):
19
+ return np.asarray(x.sum(axis=0)).ravel()
20
+ return np.asarray(x).sum(axis=0)
21
+
22
+
23
+ def _trim_by_lsize(ls: np.ndarray, trim_q: tuple[float, float]) -> np.ndarray:
24
+ """Keep columns whose library size is in the [q_low, q_high] range and > 0."""
25
+ if ls.size == 0:
26
+ return np.array([], dtype=int)
27
+ qlo = float(np.quantile(ls, min(trim_q)))
28
+ qhi = float(np.quantile(ls, max(trim_q)))
29
+ return np.where((ls > 0) & (ls >= qlo) & (ls <= qhi))[0]
30
+
31
+
32
+ def _sample_pairs(n_cells: int, n_doublets: int, rng: np.random.Generator) -> np.ndarray:
33
+ """Sample ``n_doublets`` cell index pairs (with replacement), removing self-self."""
34
+ if n_cells <= 1:
35
+ raise ValueError("need at least 2 cells to sample doublet pairs")
36
+ # Oversample 2x then drop self-matches so the final count is close to ``n_doublets``
37
+ target = n_doublets * 2
38
+ pairs = rng.integers(0, n_cells, size=(target, 2))
39
+ keep = pairs[:, 0] != pairs[:, 1]
40
+ pairs = pairs[keep]
41
+ if pairs.shape[0] > n_doublets:
42
+ pairs = pairs[:n_doublets]
43
+ while pairs.shape[0] < n_doublets:
44
+ more = rng.integers(0, n_cells, size=(n_doublets - pairs.shape[0], 2))
45
+ more = more[more[:, 0] != more[:, 1]]
46
+ pairs = np.vstack([pairs, more])
47
+ return pairs[:n_doublets]
48
+
49
+
50
+ def _create_doublets(
51
+ counts: np.ndarray | sp.spmatrix,
52
+ pairs: np.ndarray,
53
+ *,
54
+ clusters: Optional[np.ndarray] = None,
55
+ resamp: float = 0.25,
56
+ half_size: float = 0.25,
57
+ adjust_size: float = 0.0,
58
+ rng: np.random.Generator,
59
+ ) -> np.ndarray:
60
+ """Port of R ``createDoublets``.
61
+
62
+ - ``adjust_size`` fraction is size-adjusted using cluster median library
63
+ size ratios (requires ``clusters``).
64
+ - ``half_size`` fraction has its library size halved.
65
+ - ``resamp`` fraction is re-sampled from a Poisson around the summed
66
+ counts (the other fraction is just rounded).
67
+ Returns a dense ``(n_genes, n_doublets)`` float array.
68
+ """
69
+ counts = counts.toarray() if sp.issparse(counts) else np.asarray(counts, dtype=np.float64)
70
+ n_d = pairs.shape[0]
71
+
72
+ if adjust_size > 0 and clusters is not None:
73
+ n_adj = int(round(adjust_size * n_d))
74
+ idx_adj = rng.choice(n_d, size=n_adj, replace=False)
75
+ else:
76
+ n_adj = 0
77
+ idx_adj = np.array([], dtype=int)
78
+ idx_nonadj = np.setdiff1d(np.arange(n_d), idx_adj, assume_unique=False)
79
+
80
+ # Non-adjusted doublets: simple sum
81
+ x1 = counts[:, pairs[idx_nonadj, 0]] + counts[:, pairs[idx_nonadj, 1]]
82
+
83
+ if n_adj > 1 and clusters is not None:
84
+ ls = counts.sum(axis=0)
85
+ # cluster median library sizes
86
+ uniq = np.unique(clusters)
87
+ csz = {u: np.median(ls[clusters == u]) for u in uniq}
88
+ pair_adj = pairs[idx_adj]
89
+ ls1 = ls[pair_adj[:, 0]]
90
+ ls2 = ls[pair_adj[:, 1]]
91
+ ls_ratio = ls1 / (ls1 + ls2 + 1e-12)
92
+ c1 = clusters[pair_adj[:, 0]]
93
+ c2 = clusters[pair_adj[:, 1]]
94
+ csz1 = np.array([csz[c] for c in c1])
95
+ csz2 = np.array([csz[c] for c in c2])
96
+ factor = (ls_ratio + csz1 / (csz1 + csz2 + 1e-12)) / 2
97
+ factor = np.clip(factor, 0.2, 0.8)
98
+ target_ls = ls1 + ls2
99
+ x2 = counts[:, pair_adj[:, 0]] * factor + counts[:, pair_adj[:, 1]] * (1 - factor)
100
+ # Renormalize to the target library size
101
+ cur_ls = x2.sum(axis=0) + 1e-12
102
+ x2 = x2 * (target_ls / cur_ls)[None, :]
103
+ out = np.concatenate([x1, x2], axis=1)
104
+ else:
105
+ out = x1
106
+
107
+ # Half-size a fraction
108
+ if half_size > 0 and out.shape[1] > 0:
109
+ n_h = int(np.ceil(half_size * out.shape[1]))
110
+ h_idx = rng.choice(out.shape[1], size=n_h, replace=False)
111
+ out[:, h_idx] = out[:, h_idx] / 2.0
112
+
113
+ # Resample (Poisson) a fraction, else round
114
+ if resamp > 0 and out.shape[1] > 0:
115
+ n_r = int(np.ceil(resamp * out.shape[1]))
116
+ r_idx = rng.choice(out.shape[1], size=n_r, replace=False)
117
+ out[:, r_idx] = rng.poisson(np.clip(out[:, r_idx], 0, None)).astype(np.float64)
118
+ # Round the rest
119
+ rest = np.setdiff1d(np.arange(out.shape[1]), r_idx, assume_unique=False)
120
+ out[:, rest] = np.round(out[:, rest])
121
+ else:
122
+ out = np.round(out)
123
+
124
+ return out
125
+
126
+
127
+ def get_artificial_doublets(
128
+ counts: np.ndarray | sp.spmatrix,
129
+ n: int = 3000,
130
+ *,
131
+ clusters: Optional[np.ndarray] = None,
132
+ prop_random: float = 0.1,
133
+ resamp: float = 0.25,
134
+ half_size: float = 0.25,
135
+ adjust_size: float = 0.25,
136
+ trim_q: tuple[float, float] = (0.05, 0.95),
137
+ rng: Optional[np.random.Generator] = None,
138
+ seed: Optional[int] = None,
139
+ ) -> dict:
140
+ """Generate ``n`` artificial doublets from the ``counts`` matrix.
141
+
142
+ Parameters mirror the R ``getArtificialDoublets`` one-to-one. The
143
+ returned dict has two keys:
144
+
145
+ "counts" — dense float ``(n_genes, n_created)`` array (may differ
146
+ from ``n`` by a few due to self-self filtering).
147
+ "origins" — 1-D array of cluster-pair labels (str) or ``None``.
148
+
149
+ ``counts`` is expected in **genes × cells** orientation (Seurat style).
150
+ """
151
+ if rng is None:
152
+ rng = np.random.default_rng(seed)
153
+
154
+ n_genes, n_cells = counts.shape
155
+ ls = _col_sums(counts)
156
+ if clusters is None:
157
+ keep = _trim_by_lsize(ls, trim_q)
158
+ clusters_kept = None
159
+ else:
160
+ clusters = np.asarray(clusters)
161
+ keep_masks = []
162
+ for c in np.unique(clusters):
163
+ idx = np.where(clusters == c)[0]
164
+ ls_c = ls[idx]
165
+ if idx.size < 10:
166
+ keep_masks.append(idx[ls_c > 0])
167
+ continue
168
+ qlo, qhi = np.quantile(ls_c, sorted(trim_q))
169
+ keep_masks.append(idx[(ls_c > 0) & (ls_c >= qlo) & (ls_c <= qhi)])
170
+ keep = np.concatenate(keep_masks) if keep_masks else np.array([], dtype=int)
171
+ keep = np.sort(keep)
172
+ clusters_kept = clusters[keep]
173
+
174
+ x = counts[:, keep].toarray() if sp.issparse(counts) else np.asarray(counts)[:, keep]
175
+ n_usable = x.shape[1]
176
+
177
+ if clusters_kept is None or prop_random >= 1.0:
178
+ pairs = _sample_pairs(n_usable, n, rng)
179
+ ad = _create_doublets(x, pairs, clusters=None,
180
+ resamp=resamp, half_size=half_size,
181
+ adjust_size=0.0, rng=rng)
182
+ origins = np.array([None] * ad.shape[1], dtype=object)
183
+ return {"counts": ad, "origins": origins}
184
+
185
+ # Mixed: some random, rest cluster-paired
186
+ n_random = int(np.ceil(n * prop_random))
187
+ n_cluster = n - n_random
188
+
189
+ out_counts = []
190
+ out_origins = []
191
+
192
+ if n_random > 0:
193
+ pairs_r = _sample_pairs(n_usable, n_random, rng)
194
+ ad_r = _create_doublets(x, pairs_r, clusters=clusters_kept,
195
+ resamp=resamp, half_size=half_size,
196
+ adjust_size=adjust_size, rng=rng)
197
+ # Origins from the random pairs
198
+ c1 = clusters_kept[pairs_r[:, 0]]
199
+ c2 = clusters_kept[pairs_r[:, 1]]
200
+ oc = np.array([f"{a}+{b}" if a != b else None for a, b in zip(c1, c2)], dtype=object)
201
+ # _create_doublets may return more columns than ad_r has per-pair (due to adjust_size split);
202
+ # pad origins to the actual length
203
+ out_counts.append(ad_r)
204
+ if ad_r.shape[1] == oc.size:
205
+ out_origins.append(oc)
206
+ else:
207
+ out_origins.append(np.array([None] * ad_r.shape[1], dtype=object))
208
+
209
+ if n_cluster > 0:
210
+ # inter-cluster pairs, weighted "proportional" to cluster sizes
211
+ cl_counts = np.bincount(np.unique(clusters_kept, return_inverse=True)[1])
212
+ weights = cl_counts / cl_counts.sum()
213
+ uniq = np.unique(clusters_kept)
214
+ # Sample pairs of distinct clusters weighted by proportion^2
215
+ cpairs = []
216
+ for _ in range(n_cluster):
217
+ a, b = rng.choice(uniq.size, size=2, replace=False, p=weights)
218
+ cpairs.append((uniq[a], uniq[b]))
219
+ pair_idx = []
220
+ origins = []
221
+ for ca, cb in cpairs:
222
+ ia = np.random.default_rng(rng.integers(0, 2**31)).choice(
223
+ np.where(clusters_kept == ca)[0]
224
+ )
225
+ ib = np.random.default_rng(rng.integers(0, 2**31)).choice(
226
+ np.where(clusters_kept == cb)[0]
227
+ )
228
+ pair_idx.append([ia, ib])
229
+ origins.append(f"{ca}+{cb}")
230
+ pair_idx = np.asarray(pair_idx)
231
+ ad_c = _create_doublets(x, pair_idx, clusters=clusters_kept,
232
+ resamp=resamp, half_size=half_size,
233
+ adjust_size=adjust_size, rng=rng)
234
+ out_counts.append(ad_c)
235
+ out_origins.append(np.asarray(origins, dtype=object) if ad_c.shape[1] == len(origins)
236
+ else np.array([None] * ad_c.shape[1], dtype=object))
237
+
238
+ combined = np.concatenate(out_counts, axis=1) if out_counts else np.zeros((n_genes, 0))
239
+ origins = np.concatenate(out_origins) if out_origins else np.array([], dtype=object)
240
+ return {"counts": combined, "origins": origins}