iaode 0.2.0__tar.gz → 0.2.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- iaode-0.2.2/PKG-INFO +694 -0
- iaode-0.2.2/README.md +645 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode/BEN.py +6 -3
- {iaode-0.2.0 → iaode-0.2.2}/iaode/DRE.py +16 -13
- {iaode-0.2.0 → iaode-0.2.2}/iaode/__init__.py +7 -1
- iaode-0.2.2/iaode/agent.py +850 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode/annotation.py +7 -3
- iaode-0.2.2/iaode/datasets.py +330 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode/environment.py +126 -96
- {iaode-0.2.0 → iaode-0.2.2}/iaode/mixin.py +8 -2
- iaode-0.2.2/iaode/model.py +486 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode/module.py +240 -137
- {iaode-0.2.0 → iaode-0.2.2}/iaode/utils.py +7 -6
- iaode-0.2.2/iaode.egg-info/PKG-INFO +694 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode.egg-info/SOURCES.txt +1 -0
- {iaode-0.2.0 → iaode-0.2.2}/pyproject.toml +1 -1
- iaode-0.2.0/PKG-INFO +0 -399
- iaode-0.2.0/README.md +0 -350
- iaode-0.2.0/iaode/agent.py +0 -190
- iaode-0.2.0/iaode/model.py +0 -634
- iaode-0.2.0/iaode.egg-info/PKG-INFO +0 -399
- {iaode-0.2.0 → iaode-0.2.2}/LICENSE +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/MANIFEST.in +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode/LSE.py +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode.egg-info/dependency_links.txt +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode.egg-info/requires.txt +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/iaode.egg-info/top_level.txt +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/requirements.txt +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/setup.cfg +0 -0
- {iaode-0.2.0 → iaode-0.2.2}/setup.py +0 -0
iaode-0.2.2/PKG-INFO
ADDED
|
@@ -0,0 +1,694 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: iaode
|
|
3
|
+
Version: 0.2.2
|
|
4
|
+
Summary: Interpretable Autoencoder with Ordinary Differential Equations for single-cell omics analysis
|
|
5
|
+
Author-email: Zeyu Fu <fuzeyu99@126.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/PeterPonyu/iAODE
|
|
8
|
+
Project-URL: Documentation, https://iaode.readthedocs.io
|
|
9
|
+
Project-URL: Repository, https://github.com/PeterPonyu/iAODE
|
|
10
|
+
Project-URL: Bug Tracker, https://github.com/PeterPonyu/iAODE/issues
|
|
11
|
+
Keywords: single-cell,omics,dimensionality-reduction,neural-ode,variational-autoencoder,trajectory-inference,scRNA-seq,scATAC-seq
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Intended Audience :: Science/Research
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
|
|
21
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
+
Requires-Python: >=3.9
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Requires-Dist: numpy>=1.20.0
|
|
26
|
+
Requires-Dist: pandas>=1.3.0
|
|
27
|
+
Requires-Dist: scipy>=1.7.0
|
|
28
|
+
Requires-Dist: scikit-learn>=1.0.0
|
|
29
|
+
Requires-Dist: torch>=1.10.0
|
|
30
|
+
Requires-Dist: torchdiffeq>=0.2.0
|
|
31
|
+
Requires-Dist: anndata>=0.8.0
|
|
32
|
+
Requires-Dist: scanpy>=1.8.0
|
|
33
|
+
Requires-Dist: scvi-tools>=0.16.0
|
|
34
|
+
Requires-Dist: requests>=2.25.0
|
|
35
|
+
Requires-Dist: tqdm>=4.60.0
|
|
36
|
+
Provides-Extra: dev
|
|
37
|
+
Requires-Dist: pytest>=7.0.0; extra == "dev"
|
|
38
|
+
Requires-Dist: pytest-cov>=3.0.0; extra == "dev"
|
|
39
|
+
Requires-Dist: black>=22.0.0; extra == "dev"
|
|
40
|
+
Requires-Dist: flake8>=4.0.0; extra == "dev"
|
|
41
|
+
Requires-Dist: mypy>=0.950; extra == "dev"
|
|
42
|
+
Requires-Dist: build>=0.7.0; extra == "dev"
|
|
43
|
+
Requires-Dist: twine>=4.0.0; extra == "dev"
|
|
44
|
+
Provides-Extra: docs
|
|
45
|
+
Requires-Dist: sphinx>=4.5.0; extra == "docs"
|
|
46
|
+
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
|
|
47
|
+
Requires-Dist: sphinx-autodoc-typehints>=1.18.0; extra == "docs"
|
|
48
|
+
Dynamic: license-file
|
|
49
|
+
|
|
50
|
+
# iAODE: Interpretable Accessibility ODE VAE for scATAC-seq
|
|
51
|
+
|
|
52
|
+
[](https://badge.fury.io/py/iaode) [](https://www.python.org/downloads/) [](https://opensource.org/licenses/MIT)
|
|
53
|
+
|
|
54
|
+
`iAODE` (interpretable Accessibility ODE VAE) is a lightweight deep learning framework purpose-built for single‑cell ATAC‑seq (scATAC‑seq) data. It couples a variational autoencoder (VAE) with a Neural ODE and an interpretable bottleneck to jointly achieve:
|
|
55
|
+
|
|
56
|
+
1. **Robust modeling** of sparse count accessibility profiles (NB / ZINB likelihoods)
|
|
57
|
+
2. **Continuous trajectory inference** (Neural ODE pseudotime + velocity)
|
|
58
|
+
3. **Interpretable latent factors** (biologically aligned bottleneck)
|
|
59
|
+
4. **Scalable preprocessing** (TF‑IDF + highly variable peak selection) aligned with Signac / SnapATAC2 best practices
|
|
60
|
+
|
|
61
|
+
The design targets the unique characteristics of scATAC data: extreme sparsity, library size variability, heterogeneous peak accessibility kinetics, and dynamic regulatory trajectories.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Core Architecture
|
|
66
|
+
|
|
67
|
+
```
|
|
68
|
+
Raw Peak Counts --(TF-IDF)--> Normalized Matrix --(HVP Selection)--> Peak Subset
|
|
69
|
+
|
|
|
70
|
+
v
|
|
71
|
+
Encoder (MLP / Residual / Transformer) --> q(z|x)
|
|
72
|
+
| \
|
|
73
|
+
| +--> Bottleneck (i_dim) -> Interpretable factors
|
|
74
|
+
v
|
|
75
|
+
Neural ODE f(z,t) -> z_ode(t) (pseudotime + dynamics)
|
|
76
|
+
| \
|
|
77
|
+
| +--> Consistency Loss (q_z vs z_ode)
|
|
78
|
+
v
|
|
79
|
+
Decoder (NB / ZINB / MSE) -> Reconstruction x_hat
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
**Key Components**:
|
|
83
|
+
- **TF‑IDF Normalization**: Stabilizes cell‑wise peak depth, emphasizes specific accessibility
|
|
84
|
+
- **Highly Variable Peaks (HVP)**: Variance / VMR / deviance‑based selection
|
|
85
|
+
- **NB/ZINB Likelihoods**: Models over‑dispersion & zero inflation
|
|
86
|
+
- **Neural ODE**: Smooth accessibility progression with pseudotime & velocity
|
|
87
|
+
- **Interpretable Bottleneck**: Linear compression preserving biologically decodable axes
|
|
88
|
+
- **Multi‑Objective Regularization**: β‑VAE KL, β‑TC, DIP, InfoVAE MMD, ODE consistency
|
|
89
|
+
|
|
90
|
+
**Loss Function**:
|
|
91
|
+
```
|
|
92
|
+
Loss = recon + i_recon + ODE_consistency + β·KL + dip·DIP + tc·TC + info·MMD
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Installation
|
|
98
|
+
|
|
99
|
+
### From PyPI (Recommended)
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
pip install iaode
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### From Source
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
git clone https://github.com/PeterPonyu/iAODE.git
|
|
109
|
+
cd iAODE
|
|
110
|
+
pip install -e .
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Requirements
|
|
114
|
+
|
|
115
|
+
- Python ≥ 3.9
|
|
116
|
+
- PyTorch ≥ 1.10.0
|
|
117
|
+
- AnnData ≥ 0.8.0
|
|
118
|
+
- Scanpy ≥ 1.8.0
|
|
119
|
+
- scvi-tools ≥ 0.16.0
|
|
120
|
+
|
|
121
|
+
See `requirements.txt` for complete dependencies.
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
## Quick Start
|
|
126
|
+
|
|
127
|
+
### Basic scATAC-seq Workflow
|
|
128
|
+
|
|
129
|
+
```python
|
|
130
|
+
import scanpy as sc
|
|
131
|
+
import iaode
|
|
132
|
+
from iaode.utils import tfidf_normalization, select_highly_variable_peaks
|
|
133
|
+
from iaode.annotation import load_10x_h5_data
|
|
134
|
+
|
|
135
|
+
# 1. Download and load scATAC-seq data (auto-cached)
|
|
136
|
+
h5_file, gtf_file = iaode.datasets.mouse_brain_5k_atacseq()
|
|
137
|
+
print(f"Data downloaded to: {h5_file.parent}")
|
|
138
|
+
|
|
139
|
+
# Load peak count matrix
|
|
140
|
+
adata = load_10x_h5_data(str(h5_file))
|
|
141
|
+
adata.layers['counts'] = adata.X.copy()
|
|
142
|
+
print(f"Loaded: {adata.n_obs} cells × {adata.n_vars} peaks")
|
|
143
|
+
|
|
144
|
+
# 2. TF-IDF normalization (Signac/SnapATAC2 best practice)
|
|
145
|
+
tfidf_normalization(
|
|
146
|
+
adata,
|
|
147
|
+
scale_factor=1e4,
|
|
148
|
+
log_tf=False,
|
|
149
|
+
log_idf=True,
|
|
150
|
+
inplace=True
|
|
151
|
+
)
|
|
152
|
+
|
|
153
|
+
# 3. Highly variable peak (HVP) selection
|
|
154
|
+
select_highly_variable_peaks(
|
|
155
|
+
adata,
|
|
156
|
+
n_top_peaks=20000,
|
|
157
|
+
method='signac',
|
|
158
|
+
min_accessibility=0.01,
|
|
159
|
+
max_accessibility=0.95,
|
|
160
|
+
inplace=True
|
|
161
|
+
)
|
|
162
|
+
|
|
163
|
+
# Subset to HVPs
|
|
164
|
+
hvp_mask = adata.var['highly_variable']
|
|
165
|
+
adata = adata[:, hvp_mask].copy()
|
|
166
|
+
print(f"Retained {adata.n_vars} highly variable peaks")
|
|
167
|
+
|
|
168
|
+
# 4. Train iAODE model
|
|
169
|
+
model = iaode.agent(
|
|
170
|
+
adata,
|
|
171
|
+
layer='counts',
|
|
172
|
+
latent_dim=32, # Higher for scATAC complexity
|
|
173
|
+
hidden_dim=512, # Deeper for regulatory patterns
|
|
174
|
+
encoder_type='mlp',
|
|
175
|
+
loss_mode='zinb', # Best for sparse scATAC data
|
|
176
|
+
use_ode=False
|
|
177
|
+
)
|
|
178
|
+
|
|
179
|
+
model.fit(epochs=400, patience=25, val_every=10)
|
|
180
|
+
|
|
181
|
+
# 5. Extract latent representation
|
|
182
|
+
latent = model.get_latent()
|
|
183
|
+
adata.obsm['X_iaode'] = latent
|
|
184
|
+
|
|
185
|
+
# 6. Visualize with UMAP
|
|
186
|
+
sc.pp.neighbors(adata, use_rep='X_iaode')
|
|
187
|
+
sc.tl.umap(adata)
|
|
188
|
+
|
|
189
|
+
# Color by QC metrics
|
|
190
|
+
sc.pl.umap(adata, color=['n_genes_by_counts', 'total_counts'])
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
### scATAC-seq with Trajectory Inference
|
|
194
|
+
|
|
195
|
+
```python
|
|
196
|
+
import scanpy as sc
|
|
197
|
+
import iaode
|
|
198
|
+
from iaode.utils import tfidf_normalization, select_highly_variable_peaks
|
|
199
|
+
from iaode.annotation import load_10x_h5_data
|
|
200
|
+
|
|
201
|
+
# 1. Load and preprocess scATAC-seq data
|
|
202
|
+
h5_file, gtf_file = iaode.datasets.mouse_brain_5k_atacseq()
|
|
203
|
+
adata = load_10x_h5_data(str(h5_file))
|
|
204
|
+
adata.layers['counts'] = adata.X.copy()
|
|
205
|
+
|
|
206
|
+
# TF-IDF normalization
|
|
207
|
+
tfidf_normalization(adata, scale_factor=1e4, log_tf=False, log_idf=True, inplace=True)
|
|
208
|
+
|
|
209
|
+
# Select highly variable peaks
|
|
210
|
+
select_highly_variable_peaks(adata, n_top_peaks=20000, method='signac', inplace=True)
|
|
211
|
+
adata = adata[:, adata.var['highly_variable']].copy()
|
|
212
|
+
|
|
213
|
+
# 2. Train iAODE with Neural ODE for trajectory inference
|
|
214
|
+
model = iaode.agent(
|
|
215
|
+
adata,
|
|
216
|
+
layer='counts',
|
|
217
|
+
use_ode=True, # Enable Neural ODE dynamics
|
|
218
|
+
i_dim=16, # Interpretable bottleneck dimension
|
|
219
|
+
latent_dim=32,
|
|
220
|
+
hidden_dim=512,
|
|
221
|
+
encoder_type='mlp',
|
|
222
|
+
loss_mode='zinb'
|
|
223
|
+
)
|
|
224
|
+
|
|
225
|
+
model.fit(epochs=400, patience=25, val_every=10)
|
|
226
|
+
|
|
227
|
+
# 3. Extract trajectory-related representations
|
|
228
|
+
latent = model.get_latent() # Latent space (z)
|
|
229
|
+
iembed = model.get_iembed() # Interpretable regulatory factors
|
|
230
|
+
pseudotime = model.get_pseudotime() # ODE time parameter
|
|
231
|
+
velocity = model.get_velocity() # Latent velocity field
|
|
232
|
+
|
|
233
|
+
# Store in AnnData
|
|
234
|
+
adata.obsm['X_iaode'] = latent
|
|
235
|
+
adata.obsm['X_iembed'] = iembed
|
|
236
|
+
adata.obs['pseudotime'] = pseudotime
|
|
237
|
+
adata.obsm['velocity'] = velocity
|
|
238
|
+
|
|
239
|
+
# 4. Visualize trajectory
|
|
240
|
+
sc.pp.neighbors(adata, use_rep='X_iaode')
|
|
241
|
+
sc.tl.umap(adata)
|
|
242
|
+
|
|
243
|
+
# Color UMAP by pseudotime to reveal developmental trajectory
|
|
244
|
+
sc.pl.umap(adata, color='pseudotime', cmap='viridis')
|
|
245
|
+
|
|
246
|
+
# Visualize velocity field (requires UMAP coordinates)
|
|
247
|
+
E_grid, V_grid = model.get_vfres(
|
|
248
|
+
adata,
|
|
249
|
+
zs_key='X_iaode',
|
|
250
|
+
E_key='X_umap',
|
|
251
|
+
stream=True,
|
|
252
|
+
density=1.5
|
|
253
|
+
)
|
|
254
|
+
|
|
255
|
+
import matplotlib.pyplot as plt
|
|
256
|
+
fig, ax = plt.subplots(figsize=(8, 6))
|
|
257
|
+
sc.pl.umap(adata, color='pseudotime', ax=ax, show=False)
|
|
258
|
+
ax.streamplot(E_grid[0], E_grid[1], V_grid[0], V_grid[1],
|
|
259
|
+
color='gray', density=1.5, linewidth=0.5, arrowsize=1)
|
|
260
|
+
plt.tight_layout()
|
|
261
|
+
plt.show()
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
### scATAC-seq Peak Annotation Pipeline
|
|
265
|
+
|
|
266
|
+
**Automatic data download** - iAODE automatically downloads and caches datasets:
|
|
267
|
+
|
|
268
|
+
```python
|
|
269
|
+
import iaode
|
|
270
|
+
|
|
271
|
+
# Downloads mouse brain 5k scATAC-seq + GENCODE vM25 GTF
|
|
272
|
+
# Files cached in ~/.iaode/data/ and reused on subsequent runs
|
|
273
|
+
h5_file, gtf_file = iaode.datasets.mouse_brain_5k_atacseq()
|
|
274
|
+
|
|
275
|
+
# Run complete annotation pipeline
|
|
276
|
+
adata = iaode.annotation_pipeline(
|
|
277
|
+
h5_file=str(h5_file),
|
|
278
|
+
gtf_file=str(gtf_file),
|
|
279
|
+
promoter_upstream=2000, # TSS upstream region
|
|
280
|
+
promoter_downstream=500, # TSS downstream region
|
|
281
|
+
apply_tfidf=True, # TF-IDF normalization
|
|
282
|
+
select_hvp=True, # Select highly variable peaks
|
|
283
|
+
n_top_peaks=20000 # Number of HVPs to retain
|
|
284
|
+
)
|
|
285
|
+
|
|
286
|
+
# AnnData now contains:
|
|
287
|
+
# - adata.var['peak_type']: promoter/exonic/intronic/intergenic
|
|
288
|
+
# - adata.var['gene_name']: Associated gene names
|
|
289
|
+
# - adata.var['distance_to_tss']: Distance to nearest TSS
|
|
290
|
+
# - adata.var['highly_variable']: HVP selection mask
|
|
291
|
+
# - Preprocessed with TF-IDF and HVP selection
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
**Available datasets:**
|
|
295
|
+
|
|
296
|
+
```python
|
|
297
|
+
# Mouse brain 5k scATAC-seq
|
|
298
|
+
h5, gtf = iaode.datasets.mouse_brain_5k_atacseq()
|
|
299
|
+
|
|
300
|
+
# Human PBMC 5k scATAC-seq
|
|
301
|
+
h5, gtf = iaode.datasets.human_pbmc_5k_atacseq()
|
|
302
|
+
|
|
303
|
+
# Cache management
|
|
304
|
+
iaode.datasets.list_cached_files() # Show cached files
|
|
305
|
+
iaode.datasets.clear_cache() # Clear all cached data
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
|
|
309
|
+
---
|
|
310
|
+
|
|
311
|
+
## Examples
|
|
312
|
+
|
|
313
|
+
Comprehensive examples are available in the `examples/` directory. See **[examples/README.md](examples/README.md)** for detailed documentation.
|
|
314
|
+
|
|
315
|
+
### Example Scripts
|
|
316
|
+
|
|
317
|
+
| Script | Purpose | Modality | Best For |
|
|
318
|
+
|--------|---------|----------|----------|
|
|
319
|
+
| **`basic_usage.py`** | scATAC-seq with 2D/histogram visualizations | scATAC | Getting started with scATAC-seq |
|
|
320
|
+
| **`atacseq_annotation.py`** | Peak-to-gene annotation + QC plots | scATAC | Understanding peak annotation |
|
|
321
|
+
| **`model_evaluation_atac.py`** | Benchmark iAODE vs scVI on scATAC-seq | scATAC | Comparative analysis (chromatin) |
|
|
322
|
+
| **`model_evaluation_rna.py`** | Benchmark iAODE vs scVI on scRNA-seq | scRNA | Comparative analysis (transcriptome) |
|
|
323
|
+
| **`trajectory_inference_atac.py`** | Neural ODE trajectory with scATAC-seq | scATAC | Chromatin accessibility dynamics |
|
|
324
|
+
| **`trajectory_inference_rna.py`** | Neural ODE trajectory with scRNA-seq | scRNA | Transcriptional trajectory |
|
|
325
|
+
|
|
326
|
+
|
|
327
|
+
### Running Examples
|
|
328
|
+
|
|
329
|
+
```bash
|
|
330
|
+
cd examples
|
|
331
|
+
|
|
332
|
+
# Getting started with scATAC-seq
|
|
333
|
+
python basic_usage.py
|
|
334
|
+
|
|
335
|
+
# Peak-to-gene annotation with auto-download
|
|
336
|
+
python atacseq_annotation.py
|
|
337
|
+
|
|
338
|
+
# Benchmark models on scATAC-seq data
|
|
339
|
+
python model_evaluation_atac.py
|
|
340
|
+
|
|
341
|
+
# Trajectory inference with Neural ODE (scRNA-seq)
|
|
342
|
+
python trajectory_inference_rna.py
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
All examples save outputs to `examples/outputs/<example_name>/` by default. See **[examples/README.md](examples/README.md)** for detailed usage instructions and customization options.
|
|
346
|
+
|
|
347
|
+
---
|
|
348
|
+
|
|
349
|
+
## API Reference
|
|
350
|
+
|
|
351
|
+
### `iaode.agent` - Main Model Interface
|
|
352
|
+
|
|
353
|
+
**Initialization:**
|
|
354
|
+
|
|
355
|
+
```python
|
|
356
|
+
model = iaode.agent(
|
|
357
|
+
adata, # AnnData object
|
|
358
|
+
layer='counts', # Data layer to use
|
|
359
|
+
latent_dim=10, # Latent space dimension
|
|
360
|
+
hidden_dim=128, # Hidden layer dimension
|
|
361
|
+
i_dim=None, # Interpretable bottleneck dim (required if use_ode=True)
|
|
362
|
+
use_ode=False, # Enable Neural ODE
|
|
363
|
+
loss_mode='nb', # Loss function: 'mse', 'nb', 'zinb'
|
|
364
|
+
encoder_type='mlp', # Encoder: 'mlp', 'residual_mlp', 'transformer', 'linear'
|
|
365
|
+
lr=1e-4, # Learning rate
|
|
366
|
+
batch_size=128, # Batch size
|
|
367
|
+
beta=1.0, # KL divergence weight
|
|
368
|
+
recon=1.0, # Reconstruction loss weight
|
|
369
|
+
tc=0.0, # Total correlation weight
|
|
370
|
+
dip=0.0, # DIP weight
|
|
371
|
+
info=0.0 # InfoVAE MMD weight
|
|
372
|
+
)
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
**Training:**
|
|
376
|
+
|
|
377
|
+
```python
|
|
378
|
+
model.fit(
|
|
379
|
+
epochs=100, # Maximum epochs
|
|
380
|
+
patience=20, # Early stopping patience
|
|
381
|
+
val_every=5, # Validation frequency
|
|
382
|
+
early_stop=True # Enable early stopping
|
|
383
|
+
)
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
**Representation Extraction:**
|
|
387
|
+
|
|
388
|
+
```python
|
|
389
|
+
# Basic representations
|
|
390
|
+
latent = model.get_latent() # Latent space (n_cells, latent_dim)
|
|
391
|
+
iembed = model.get_iembed() # Interpretable factors (n_cells, i_dim)
|
|
392
|
+
|
|
393
|
+
# Trajectory-specific (requires use_ode=True)
|
|
394
|
+
pseudotime = model.get_pseudotime() # ODE time parameter (n_cells,)
|
|
395
|
+
velocity = model.get_velocity() # Latent velocity (n_cells, latent_dim)
|
|
396
|
+
|
|
397
|
+
# Vector field for visualization (requires UMAP in adata.obsm['X_umap'])
|
|
398
|
+
E_grid, V_grid = model.get_vfres(
|
|
399
|
+
adata,
|
|
400
|
+
zs_key='X_iaode', # Latent representation key
|
|
401
|
+
E_key='X_umap', # Embedding key for visualization
|
|
402
|
+
stream=True, # Return streamplot-compatible format
|
|
403
|
+
density=1.5 # Grid density
|
|
404
|
+
)
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
**Evaluation Metrics:**
|
|
408
|
+
|
|
409
|
+
```python
|
|
410
|
+
# Training metrics
|
|
411
|
+
metrics = model.get_resource_metrics()
|
|
412
|
+
# Returns: {'train_time': float, 'actual_epochs': int, 'peak_memory_gb': float}
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
---
|
|
416
|
+
|
|
417
|
+
### `iaode.annotation_pipeline` - scATAC-seq Preprocessing
|
|
418
|
+
|
|
419
|
+
**Complete annotation and preprocessing pipeline:**
|
|
420
|
+
|
|
421
|
+
```python
|
|
422
|
+
adata = iaode.annotation_pipeline(
|
|
423
|
+
h5_file, # Path to 10X H5 file
|
|
424
|
+
gtf_file, # Path to GTF annotation
|
|
425
|
+
promoter_upstream=2000, # TSS upstream extension (bp)
|
|
426
|
+
promoter_downstream=500, # TSS downstream extension (bp)
|
|
427
|
+
apply_tfidf=True, # Apply TF-IDF normalization
|
|
428
|
+
select_hvp=True, # Select highly variable peaks
|
|
429
|
+
n_top_peaks=20000, # Number of HVPs to retain
|
|
430
|
+
hvp_method='signac', # HVP method: 'signac', 'snapatac2', 'deviance'
|
|
431
|
+
min_accessibility=0.01, # Min peak accessibility fraction
|
|
432
|
+
max_accessibility=0.95 # Max peak accessibility fraction
|
|
433
|
+
)
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
**Returns AnnData with:**
|
|
437
|
+
- `adata.var['peak_type']`: Peak annotation (promoter/exonic/intronic/intergenic)
|
|
438
|
+
- `adata.var['gene_name']`: Associated gene names
|
|
439
|
+
- `adata.var['distance_to_tss']`: Distance to nearest TSS
|
|
440
|
+
- `adata.var['highly_variable']`: HVP selection mask
|
|
441
|
+
- Preprocessed and normalized counts
|
|
442
|
+
|
|
443
|
+
---
|
|
444
|
+
|
|
445
|
+
### Evaluation Functions
|
|
446
|
+
|
|
447
|
+
#### Dimensionality Reduction Quality
|
|
448
|
+
|
|
449
|
+
```python
|
|
450
|
+
from iaode import evaluate_dimensionality_reduction
|
|
451
|
+
|
|
452
|
+
metrics = evaluate_dimensionality_reduction(
|
|
453
|
+
X_high, # High-dimensional data (n_cells, n_features)
|
|
454
|
+
X_low, # Low-dimensional embedding (n_cells, n_latent)
|
|
455
|
+
k=10, # Number of neighbors
|
|
456
|
+
verbose=True
|
|
457
|
+
)
|
|
458
|
+
|
|
459
|
+
# Returns:
|
|
460
|
+
# - distance_correlation: Global structure preservation (Spearman ρ)
|
|
461
|
+
# - Q_local: Local neighborhood quality
|
|
462
|
+
# - Q_global: Global structure quality
|
|
463
|
+
# - K_max: Local-global transition point
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
#### Latent Space Quality
|
|
467
|
+
|
|
468
|
+
```python
|
|
469
|
+
from iaode import evaluate_single_cell_latent_space
|
|
470
|
+
|
|
471
|
+
metrics = evaluate_single_cell_latent_space(
|
|
472
|
+
latent_space, # Latent representation (n_cells, n_latent)
|
|
473
|
+
data_type='trajectory', # 'trajectory' or 'steady_state'
|
|
474
|
+
verbose=True
|
|
475
|
+
)
|
|
476
|
+
|
|
477
|
+
# Returns:
|
|
478
|
+
# - manifold_dimensionality: Dimensional efficiency (0-1)
|
|
479
|
+
# - spectral_decay_rate: Eigenvalue concentration
|
|
480
|
+
# - participation_ratio: Dimensional balance
|
|
481
|
+
# - anisotropy_score: Directionality strength
|
|
482
|
+
# - trajectory_directionality: Dominant axis strength
|
|
483
|
+
# - noise_resilience: Signal-to-noise ratio
|
|
484
|
+
# - overall_quality: Aggregate score
|
|
485
|
+
```
|
|
486
|
+
|
|
487
|
+
#### Model Benchmarking
|
|
488
|
+
|
|
489
|
+
```python
|
|
490
|
+
from iaode import DataSplitter, train_scvi_models, evaluate_scvi_models
|
|
491
|
+
|
|
492
|
+
# Create consistent train/val/test splits
|
|
493
|
+
splitter = DataSplitter(
|
|
494
|
+
n_samples=adata.n_obs,
|
|
495
|
+
test_size=0.15,
|
|
496
|
+
val_size=0.15,
|
|
497
|
+
random_state=42
|
|
498
|
+
)
|
|
499
|
+
|
|
500
|
+
# Train scVI family models
|
|
501
|
+
scvi_results = train_scvi_models(
|
|
502
|
+
adata,
|
|
503
|
+
splitter,
|
|
504
|
+
n_latent=10,
|
|
505
|
+
n_epochs=400,
|
|
506
|
+
batch_size=128
|
|
507
|
+
)
|
|
508
|
+
|
|
509
|
+
# Evaluate all models
|
|
510
|
+
scvi_metrics = evaluate_scvi_models(
|
|
511
|
+
scvi_results,
|
|
512
|
+
adata,
|
|
513
|
+
splitter.test_idx
|
|
514
|
+
)
|
|
515
|
+
```
|
|
516
|
+
|
|
517
|
+
---
|
|
518
|
+
|
|
519
|
+
### Preprocessing Utilities
|
|
520
|
+
|
|
521
|
+
```python
|
|
522
|
+
from iaode.utils import tfidf_normalization, select_highly_variable_peaks
|
|
523
|
+
|
|
524
|
+
# TF-IDF normalization (Signac/SnapATAC2 style)
|
|
525
|
+
tfidf_normalization(
|
|
526
|
+
adata,
|
|
527
|
+
scale_factor=1e4,
|
|
528
|
+
log_tf=False,
|
|
529
|
+
log_idf=True,
|
|
530
|
+
inplace=True
|
|
531
|
+
)
|
|
532
|
+
|
|
533
|
+
# Highly variable peak selection
|
|
534
|
+
select_highly_variable_peaks(
|
|
535
|
+
adata,
|
|
536
|
+
n_top_peaks=20000,
|
|
537
|
+
method='signac', # or 'snapatac2', 'deviance'
|
|
538
|
+
min_accessibility=0.01,
|
|
539
|
+
max_accessibility=0.95,
|
|
540
|
+
inplace=True
|
|
541
|
+
)
|
|
542
|
+
```
|
|
543
|
+
|
|
544
|
+
---
|
|
545
|
+
|
|
546
|
+
## Advanced Usage
|
|
547
|
+
|
|
548
|
+
### Custom Encoder Architecture
|
|
549
|
+
|
|
550
|
+
```python
|
|
551
|
+
# Transformer encoder for large-scale data
|
|
552
|
+
model = iaode.agent(
|
|
553
|
+
adata,
|
|
554
|
+
encoder_type='transformer',
|
|
555
|
+
encoder_num_layers=4,
|
|
556
|
+
encoder_n_heads=8,
|
|
557
|
+
encoder_d_model=256,
|
|
558
|
+
hidden_dim=512,
|
|
559
|
+
latent_dim=32
|
|
560
|
+
)
|
|
561
|
+
```
|
|
562
|
+
|
|
563
|
+
### Multi-Objective Regularization
|
|
564
|
+
|
|
565
|
+
```python
|
|
566
|
+
# Fine-tune regularization weights
|
|
567
|
+
model = iaode.agent(
|
|
568
|
+
adata,
|
|
569
|
+
recon=1.0, # Reconstruction loss
|
|
570
|
+
beta=1.0, # KL divergence
|
|
571
|
+
tc=0.5, # Total correlation (disentanglement)
|
|
572
|
+
dip=0.1, # DIP (dimension-wise independence)
|
|
573
|
+
info=0.05 # InfoVAE MMD (distribution matching)
|
|
574
|
+
)
|
|
575
|
+
```
|
|
576
|
+
|
|
577
|
+
### Custom Training Loop
|
|
578
|
+
|
|
579
|
+
```python
|
|
580
|
+
# Manual training with custom logic
|
|
581
|
+
for epoch in range(100):
|
|
582
|
+
train_loss = model.train_epoch()
|
|
583
|
+
|
|
584
|
+
if epoch % 5 == 0:
|
|
585
|
+
val_loss, val_score = model.validate()
|
|
586
|
+
print(f"Epoch {epoch}: Val Loss={val_loss:.4f}")
|
|
587
|
+
|
|
588
|
+
if should_stop(val_loss):
|
|
589
|
+
model.load_best_model()
|
|
590
|
+
break
|
|
591
|
+
```
|
|
592
|
+
|
|
593
|
+
---
|
|
594
|
+
|
|
595
|
+
## Citation
|
|
596
|
+
|
|
597
|
+
If you use iAODE in your research, please cite:
|
|
598
|
+
|
|
599
|
+
```bibtex
|
|
600
|
+
@software{iaode2025,
|
|
601
|
+
author = {Zeyu Fu},
|
|
602
|
+
title = {iAODE: Interpretable Accessibility ODE VAE for Single-Cell Chromatin Dynamics},
|
|
603
|
+
year = {2025},
|
|
604
|
+
publisher = {GitHub},
|
|
605
|
+
url = {https://github.com/PeterPonyu/iAODE}
|
|
606
|
+
}
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
---
|
|
610
|
+
|
|
611
|
+
## Contributing
|
|
612
|
+
|
|
613
|
+
Contributions are welcome! Please:
|
|
614
|
+
|
|
615
|
+
1. Fork the repository
|
|
616
|
+
2. Create a feature branch (`git checkout -b feature/AmazingFeature`)
|
|
617
|
+
3. Commit changes (`git commit -m 'Add AmazingFeature'`)
|
|
618
|
+
4. Push to branch (`git push origin feature/AmazingFeature`)
|
|
619
|
+
5. Open a Pull Request
|
|
620
|
+
|
|
621
|
+
For major changes, please open an issue first to discuss proposed changes.
|
|
622
|
+
|
|
623
|
+
---
|
|
624
|
+
|
|
625
|
+
## License
|
|
626
|
+
|
|
627
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
628
|
+
|
|
629
|
+
---
|
|
630
|
+
|
|
631
|
+
## Acknowledgments
|
|
632
|
+
|
|
633
|
+
Built upon ideas from:
|
|
634
|
+
- **scVI-tools**: scVI, PEAKVI, POISSONVI architectures
|
|
635
|
+
- **Signac** and **SnapATAC2**: scATAC-seq best practices
|
|
636
|
+
- **Neural ODE** literature: Continuous latent dynamics
|
|
637
|
+
|
|
638
|
+
PyTorch, AnnData, and Scanpy ecosystems provide the foundation.
|
|
639
|
+
|
|
640
|
+
---
|
|
641
|
+
|
|
642
|
+
## Contact
|
|
643
|
+
|
|
644
|
+
- **GitHub Issues**: [https://github.com/PeterPonyu/iAODE/issues](https://github.com/PeterPonyu/iAODE/issues)
|
|
645
|
+
- **Email**: fuzeyu99@126.com
|
|
646
|
+
|
|
647
|
+
---
|
|
648
|
+
|
|
649
|
+
## Changelog
|
|
650
|
+
|
|
651
|
+
### v0.2.0 (2025-11-20)
|
|
652
|
+
|
|
653
|
+
**Documentation & Usability**
|
|
654
|
+
- Comprehensive documentation overhaul (README, examples/README, QUICKSTART)
|
|
655
|
+
- Separated scRNA-seq and scATAC-seq workflows with clear examples
|
|
656
|
+
- Added automatic dataset download and caching system
|
|
657
|
+
- Created `examples/data/download_data.sh` helper script
|
|
658
|
+
- Verified all API usage examples against actual implementation
|
|
659
|
+
|
|
660
|
+
**Code Quality**
|
|
661
|
+
- Cleaned example scripts: removed unused imports, improved error handling
|
|
662
|
+
- Fixed all cross-references between examples and core modules
|
|
663
|
+
- Enhanced inline documentation and error messages
|
|
664
|
+
- Verified method signatures: `get_velocity()`, `get_vfres()`, `get_pseudotime()`, etc.
|
|
665
|
+
|
|
666
|
+
**Package Improvements**
|
|
667
|
+
- Complete scATAC-seq preprocessing pipeline with TF-IDF and HVP selection
|
|
668
|
+
- Improved peak annotation with distance-to-TSS computation
|
|
669
|
+
- Enhanced evaluation metrics (DRE + LSE frameworks)
|
|
670
|
+
- Better handling of sparse data and zero inflation
|
|
671
|
+
|
|
672
|
+
### v0.1.2 (2025-11-19)
|
|
673
|
+
|
|
674
|
+
**Metadata & Contact**
|
|
675
|
+
- Updated author name to "Zeyu Fu"
|
|
676
|
+
- Set primary contact email to fuzeyu99@126.com
|
|
677
|
+
- Fixed BibTeX citation formatting
|
|
678
|
+
|
|
679
|
+
### v0.1.1 (2025-11-19)
|
|
680
|
+
|
|
681
|
+
**Bug Fixes**
|
|
682
|
+
- Added missing `requests` dependency for scvi-tools compatibility
|
|
683
|
+
- Dropped Python 3.8 support (requires Python ≥3.9)
|
|
684
|
+
- Fixed CI test imports and matrix configuration
|
|
685
|
+
|
|
686
|
+
### v0.1.0 (2025-11-19)
|
|
687
|
+
|
|
688
|
+
**Initial Release**
|
|
689
|
+
- VAE with Neural ODE support for trajectory inference
|
|
690
|
+
- Complete scATAC-seq peak annotation pipeline
|
|
691
|
+
- Comprehensive evaluation metrics
|
|
692
|
+
- Benchmark framework vs scVI models
|
|
693
|
+
- Multiple encoder types (MLP, Residual, Transformer, Linear)
|
|
694
|
+
- Multiple loss modes (MSE, NB, ZINB)
|