iaode 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- iaode-0.1.0/LICENSE +21 -0
- iaode-0.1.0/MANIFEST.in +8 -0
- iaode-0.1.0/PKG-INFO +373 -0
- iaode-0.1.0/README.md +325 -0
- iaode-0.1.0/iaode/BEN.py +435 -0
- iaode-0.1.0/iaode/DRE.py +427 -0
- iaode-0.1.0/iaode/LSE.py +635 -0
- iaode-0.1.0/iaode/__init__.py +97 -0
- iaode-0.1.0/iaode/agent.py +190 -0
- iaode-0.1.0/iaode/annotation.py +588 -0
- iaode-0.1.0/iaode/environment.py +268 -0
- iaode-0.1.0/iaode/mixin.py +353 -0
- iaode-0.1.0/iaode/model.py +634 -0
- iaode-0.1.0/iaode/module.py +473 -0
- iaode-0.1.0/iaode/utils.py +326 -0
- iaode-0.1.0/iaode.egg-info/PKG-INFO +373 -0
- iaode-0.1.0/iaode.egg-info/SOURCES.txt +22 -0
- iaode-0.1.0/iaode.egg-info/dependency_links.txt +1 -0
- iaode-0.1.0/iaode.egg-info/requires.txt +24 -0
- iaode-0.1.0/iaode.egg-info/top_level.txt +1 -0
- iaode-0.1.0/pyproject.toml +101 -0
- iaode-0.1.0/requirements.txt +17 -0
- iaode-0.1.0/setup.cfg +4 -0
- iaode-0.1.0/setup.py +18 -0
iaode-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2024 Zeyu Fu
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
iaode-0.1.0/MANIFEST.in
ADDED
iaode-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,373 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: iaode
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Interpretable Autoencoder with Ordinary Differential Equations for single-cell omics analysis
|
|
5
|
+
Author-email: Zeyu Fu <zeyufu@example.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/PeterPonyu/iAODE
|
|
8
|
+
Project-URL: Documentation, https://iaode.readthedocs.io
|
|
9
|
+
Project-URL: Repository, https://github.com/PeterPonyu/iAODE
|
|
10
|
+
Project-URL: Bug Tracker, https://github.com/PeterPonyu/iAODE/issues
|
|
11
|
+
Keywords: single-cell,omics,dimensionality-reduction,neural-ode,variational-autoencoder,trajectory-inference,scRNA-seq,scATAC-seq
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Intended Audience :: Science/Research
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
+
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
|
|
21
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
+
Requires-Python: >=3.8
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Requires-Dist: numpy>=1.20.0
|
|
26
|
+
Requires-Dist: pandas>=1.3.0
|
|
27
|
+
Requires-Dist: scipy>=1.7.0
|
|
28
|
+
Requires-Dist: scikit-learn>=1.0.0
|
|
29
|
+
Requires-Dist: torch>=1.10.0
|
|
30
|
+
Requires-Dist: torchdiffeq>=0.2.0
|
|
31
|
+
Requires-Dist: anndata>=0.8.0
|
|
32
|
+
Requires-Dist: scanpy>=1.8.0
|
|
33
|
+
Requires-Dist: scvi-tools>=0.16.0
|
|
34
|
+
Requires-Dist: tqdm>=4.60.0
|
|
35
|
+
Provides-Extra: dev
|
|
36
|
+
Requires-Dist: pytest>=7.0.0; extra == "dev"
|
|
37
|
+
Requires-Dist: pytest-cov>=3.0.0; extra == "dev"
|
|
38
|
+
Requires-Dist: black>=22.0.0; extra == "dev"
|
|
39
|
+
Requires-Dist: flake8>=4.0.0; extra == "dev"
|
|
40
|
+
Requires-Dist: mypy>=0.950; extra == "dev"
|
|
41
|
+
Requires-Dist: build>=0.7.0; extra == "dev"
|
|
42
|
+
Requires-Dist: twine>=4.0.0; extra == "dev"
|
|
43
|
+
Provides-Extra: docs
|
|
44
|
+
Requires-Dist: sphinx>=4.5.0; extra == "docs"
|
|
45
|
+
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
|
|
46
|
+
Requires-Dist: sphinx-autodoc-typehints>=1.18.0; extra == "docs"
|
|
47
|
+
Dynamic: license-file
|
|
48
|
+
|
|
49
|
+
# iAODE: Interpretable Autoencoder with Ordinary Differential Equations
|
|
50
|
+
|
|
51
|
+
[](https://badge.fury.io/py/iaode)
|
|
52
|
+
[](https://www.python.org/downloads/)
|
|
53
|
+
[](https://opensource.org/licenses/MIT)
|
|
54
|
+
|
|
55
|
+
A deep learning framework for single-cell omics data analysis that combines variational autoencoders (VAE) with neural ordinary differential equations (ODE) for trajectory inference and dimensionality reduction.
|
|
56
|
+
|
|
57
|
+
## Features
|
|
58
|
+
|
|
59
|
+
- **Interpretable Dimensionality Reduction**: VAE-based architecture with multiple loss modes (MSE, NB, ZINB)
|
|
60
|
+
- **Trajectory Inference**: Neural ODE integration for continuous trajectory modeling
|
|
61
|
+
- **scATAC-seq Peak Annotation**: Comprehensive peak-to-gene annotation pipeline following best practices
|
|
62
|
+
- **Comprehensive Evaluation**: Built-in metrics for dimensionality reduction and latent space quality
|
|
63
|
+
- **Benchmark Framework**: Compare against state-of-the-art methods (scVI, PEAKVI, POISSONVI)
|
|
64
|
+
- **Flexible Architecture**: Support for multiple encoder types (MLP, Residual MLP, Linear, Transformer)
|
|
65
|
+
|
|
66
|
+
## Installation
|
|
67
|
+
|
|
68
|
+
### From PyPI (Recommended)
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
pip install iaode
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### From Source
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
git clone https://github.com/PeterPonyu/iAODE.git
|
|
78
|
+
cd iAODE
|
|
79
|
+
pip install -e .
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### Dependencies
|
|
83
|
+
|
|
84
|
+
- Python >= 3.8
|
|
85
|
+
- PyTorch >= 1.10.0
|
|
86
|
+
- AnnData >= 0.8.0
|
|
87
|
+
- Scanpy >= 1.8.0
|
|
88
|
+
- scvi-tools >= 0.16.0
|
|
89
|
+
- See `requirements.txt` for complete list
|
|
90
|
+
|
|
91
|
+
## Quick Start
|
|
92
|
+
|
|
93
|
+
### Basic Usage
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
import anndata as ad
|
|
97
|
+
import iaode
|
|
98
|
+
|
|
99
|
+
# Load your single-cell data
|
|
100
|
+
adata = ad.read_h5ad('your_data.h5ad')
|
|
101
|
+
|
|
102
|
+
# Create and train model
|
|
103
|
+
model = iaode.agent(
|
|
104
|
+
adata,
|
|
105
|
+
layer='counts',
|
|
106
|
+
latent_dim=10,
|
|
107
|
+
hidden_dim=128,
|
|
108
|
+
use_ode=True, # Enable neural ODE for trajectory
|
|
109
|
+
loss_mode='nb', # Negative binomial loss for count data
|
|
110
|
+
encoder_type='mlp' # MLP encoder architecture
|
|
111
|
+
)
|
|
112
|
+
|
|
113
|
+
# Train with early stopping
|
|
114
|
+
model.fit(
|
|
115
|
+
epochs=100,
|
|
116
|
+
patience=20,
|
|
117
|
+
val_every=5,
|
|
118
|
+
early_stop=True
|
|
119
|
+
)
|
|
120
|
+
|
|
121
|
+
# Get latent representation
|
|
122
|
+
latent = model.get_latent()
|
|
123
|
+
|
|
124
|
+
# Get interpretable embedding
|
|
125
|
+
iembed = model.get_iembed()
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
### scATAC-seq Peak Annotation
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
import iaode
|
|
132
|
+
|
|
133
|
+
# Complete annotation pipeline
|
|
134
|
+
adata = iaode.annotation_pipeline(
|
|
135
|
+
h5_file='filtered_peak_bc_matrix.h5',
|
|
136
|
+
gtf_file='gencode.v44.annotation.gtf',
|
|
137
|
+
output_h5ad='annotated_peaks.h5ad',
|
|
138
|
+
|
|
139
|
+
# Annotation parameters
|
|
140
|
+
promoter_upstream=2000,
|
|
141
|
+
promoter_downstream=500,
|
|
142
|
+
gene_body=True,
|
|
143
|
+
gene_type='protein_coding',
|
|
144
|
+
|
|
145
|
+
# TF-IDF normalization
|
|
146
|
+
apply_tfidf=True,
|
|
147
|
+
tfidf_scale_factor=1e4,
|
|
148
|
+
|
|
149
|
+
# Highly variable peaks
|
|
150
|
+
select_hvp=True,
|
|
151
|
+
n_top_peaks=20000,
|
|
152
|
+
hvp_method='signac'
|
|
153
|
+
)
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### Evaluation and Benchmarking
|
|
157
|
+
|
|
158
|
+
```python
|
|
159
|
+
from iaode import (
|
|
160
|
+
evaluate_dimensionality_reduction,
|
|
161
|
+
evaluate_single_cell_latent_space,
|
|
162
|
+
DataSplitter,
|
|
163
|
+
train_scvi_models
|
|
164
|
+
)
|
|
165
|
+
|
|
166
|
+
# Evaluate dimensionality reduction quality
|
|
167
|
+
dr_metrics = evaluate_dimensionality_reduction(
|
|
168
|
+
X_high=adata.X,
|
|
169
|
+
X_low=latent,
|
|
170
|
+
k=10,
|
|
171
|
+
verbose=True
|
|
172
|
+
)
|
|
173
|
+
|
|
174
|
+
# Evaluate latent space for single-cell data
|
|
175
|
+
ls_metrics = evaluate_single_cell_latent_space(
|
|
176
|
+
latent_space=latent,
|
|
177
|
+
data_type='trajectory', # or 'steady_state'
|
|
178
|
+
verbose=True
|
|
179
|
+
)
|
|
180
|
+
|
|
181
|
+
# Benchmark against scVI models
|
|
182
|
+
splitter = DataSplitter(
|
|
183
|
+
n_samples=adata.n_obs,
|
|
184
|
+
test_size=0.15,
|
|
185
|
+
val_size=0.15,
|
|
186
|
+
random_state=42
|
|
187
|
+
)
|
|
188
|
+
|
|
189
|
+
scvi_results = train_scvi_models(
|
|
190
|
+
adata,
|
|
191
|
+
splitter,
|
|
192
|
+
n_latent=10,
|
|
193
|
+
n_epochs=400,
|
|
194
|
+
batch_size=128
|
|
195
|
+
)
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
## Model Architecture
|
|
199
|
+
|
|
200
|
+
### Core Components
|
|
201
|
+
|
|
202
|
+
1. **Encoder**: Maps input data to latent distribution
|
|
203
|
+
- Types: MLP, Residual MLP, Linear, Transformer
|
|
204
|
+
- Outputs: mean, log-variance, sampled latent vector
|
|
205
|
+
|
|
206
|
+
2. **Decoder**: Reconstructs data from latent representation
|
|
207
|
+
- Supports MSE, NB, ZINB loss modes
|
|
208
|
+
- Learns library size and dispersion parameters
|
|
209
|
+
|
|
210
|
+
3. **Neural ODE** (Optional): Models continuous trajectories
|
|
211
|
+
- Latent ODE function for trajectory inference
|
|
212
|
+
- Time encoder predicts pseudo-time
|
|
213
|
+
|
|
214
|
+
4. **Information Bottleneck**: Additional interpretable layer
|
|
215
|
+
- Projects latent space to lower-dimensional embedding
|
|
216
|
+
- Enhances interpretability
|
|
217
|
+
|
|
218
|
+
### Loss Components
|
|
219
|
+
|
|
220
|
+
- **Reconstruction**: MSE / NB / ZINB likelihood
|
|
221
|
+
- **KL Divergence**: Regularizes latent distribution
|
|
222
|
+
- **β-TCVAE**: Total correlation decomposition
|
|
223
|
+
- **DIP**: Disentanglement via learned projections
|
|
224
|
+
- **MMD**: Maximum mean discrepancy
|
|
225
|
+
- **ODE Consistency**: Aligns VAE and ODE latents
|
|
226
|
+
|
|
227
|
+
## API Reference
|
|
228
|
+
|
|
229
|
+
### Main Classes
|
|
230
|
+
|
|
231
|
+
#### `iaode.agent`
|
|
232
|
+
|
|
233
|
+
High-level interface for model training and inference.
|
|
234
|
+
|
|
235
|
+
**Parameters:**
|
|
236
|
+
- `adata` (AnnData): Input single-cell data
|
|
237
|
+
- `layer` (str): Data layer to use (default: 'counts')
|
|
238
|
+
- `latent_dim` (int): Latent space dimension (default: 10)
|
|
239
|
+
- `hidden_dim` (int): Hidden layer dimension (default: 128)
|
|
240
|
+
- `use_ode` (bool): Enable neural ODE (default: False)
|
|
241
|
+
- `loss_mode` (str): Loss function ('mse', 'nb', 'zinb')
|
|
242
|
+
- `encoder_type` (str): Encoder architecture ('mlp', 'mlp_residual', 'linear', 'transformer')
|
|
243
|
+
- `lr` (float): Learning rate (default: 1e-4)
|
|
244
|
+
- `beta` (float): KL divergence weight (default: 1.0)
|
|
245
|
+
- `recon` (float): Reconstruction loss weight (default: 1.0)
|
|
246
|
+
|
|
247
|
+
**Methods:**
|
|
248
|
+
- `fit(epochs, patience, val_every, early_stop)`: Train model
|
|
249
|
+
- `get_latent()`: Get latent representation
|
|
250
|
+
- `get_iembed()`: Get interpretable embedding
|
|
251
|
+
- `get_test_latent()`: Get test set latent representation
|
|
252
|
+
|
|
253
|
+
### Annotation Functions
|
|
254
|
+
|
|
255
|
+
#### `iaode.annotation_pipeline`
|
|
256
|
+
|
|
257
|
+
Complete scATAC-seq peak annotation and preprocessing.
|
|
258
|
+
|
|
259
|
+
**Parameters:**
|
|
260
|
+
- `h5_file` (str): Path to 10X H5 file
|
|
261
|
+
- `gtf_file` (str): Path to GTF annotation
|
|
262
|
+
- `promoter_upstream` (int): TSS upstream extension (default: 2000)
|
|
263
|
+
- `promoter_downstream` (int): TSS downstream extension (default: 500)
|
|
264
|
+
- `apply_tfidf` (bool): Apply TF-IDF normalization (default: True)
|
|
265
|
+
- `select_hvp` (bool): Select highly variable peaks (default: True)
|
|
266
|
+
- `n_top_peaks` (int): Number of HVPs (default: 20000)
|
|
267
|
+
|
|
268
|
+
### Evaluation Functions
|
|
269
|
+
|
|
270
|
+
#### `iaode.evaluate_dimensionality_reduction`
|
|
271
|
+
|
|
272
|
+
Evaluate dimensionality reduction quality.
|
|
273
|
+
|
|
274
|
+
**Returns:** Dict with distance_correlation, Q_local, Q_global, K_max
|
|
275
|
+
|
|
276
|
+
#### `iaode.evaluate_single_cell_latent_space`
|
|
277
|
+
|
|
278
|
+
Evaluate single-cell latent space quality.
|
|
279
|
+
|
|
280
|
+
**Returns:** Dict with manifold_dimensionality, spectral_decay_rate, participation_ratio, etc.
|
|
281
|
+
|
|
282
|
+
## Advanced Usage
|
|
283
|
+
|
|
284
|
+
### Custom Encoder Architecture
|
|
285
|
+
|
|
286
|
+
```python
|
|
287
|
+
model = iaode.agent(
|
|
288
|
+
adata,
|
|
289
|
+
encoder_type='transformer',
|
|
290
|
+
encoder_num_layers=4,
|
|
291
|
+
encoder_n_heads=8,
|
|
292
|
+
encoder_d_model=256,
|
|
293
|
+
hidden_dim=512
|
|
294
|
+
)
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
### Multiple Loss Terms
|
|
298
|
+
|
|
299
|
+
```python
|
|
300
|
+
model = iaode.agent(
|
|
301
|
+
adata,
|
|
302
|
+
recon=1.0, # Reconstruction loss
|
|
303
|
+
beta=1.0, # KL divergence
|
|
304
|
+
tc=0.5, # Total correlation
|
|
305
|
+
dip=0.1, # Disentanglement
|
|
306
|
+
info=0.05 # MMD loss
|
|
307
|
+
)
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
### Custom Training Loop
|
|
311
|
+
|
|
312
|
+
```python
|
|
313
|
+
# Manual training with custom logic
|
|
314
|
+
for epoch in range(100):
|
|
315
|
+
train_loss = model.train_epoch()
|
|
316
|
+
|
|
317
|
+
if epoch % 5 == 0:
|
|
318
|
+
val_loss, val_score = model.validate()
|
|
319
|
+
print(f"Epoch {epoch}: Val Loss={val_loss:.4f}")
|
|
320
|
+
|
|
321
|
+
# Custom early stopping logic
|
|
322
|
+
if should_stop(val_loss):
|
|
323
|
+
model.load_best_model()
|
|
324
|
+
break
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
## Citing iAODE
|
|
328
|
+
|
|
329
|
+
If you use iAODE in your research, please cite:
|
|
330
|
+
|
|
331
|
+
```bibtex
|
|
332
|
+
@software{iaode2024,
|
|
333
|
+
author = {Fu, Zeyu},
|
|
334
|
+
title = {iAODE: Interpretable Autoencoder with Ordinary Differential Equations},
|
|
335
|
+
year = {2024},
|
|
336
|
+
url = {https://github.com/PeterPonyu/iAODE}
|
|
337
|
+
}
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
## Contributing
|
|
341
|
+
|
|
342
|
+
Contributions are welcome! Please feel free to submit a Pull Request.
|
|
343
|
+
|
|
344
|
+
1. Fork the repository
|
|
345
|
+
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
|
|
346
|
+
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
|
|
347
|
+
4. Push to the branch (`git push origin feature/AmazingFeature`)
|
|
348
|
+
5. Open a Pull Request
|
|
349
|
+
|
|
350
|
+
## License
|
|
351
|
+
|
|
352
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
353
|
+
|
|
354
|
+
## Acknowledgments
|
|
355
|
+
|
|
356
|
+
- Built on top of PyTorch, AnnData, and Scanpy ecosystems
|
|
357
|
+
- Inspired by scVI-tools, Signac, and SnapATAC2 best practices
|
|
358
|
+
- Neural ODE implementation using torchdiffeq
|
|
359
|
+
|
|
360
|
+
## Contact
|
|
361
|
+
|
|
362
|
+
For questions and feedback:
|
|
363
|
+
- GitHub Issues: [https://github.com/PeterPonyu/iAODE/issues](https://github.com/PeterPonyu/iAODE/issues)
|
|
364
|
+
- Email: zeyufu@example.com
|
|
365
|
+
|
|
366
|
+
## Changelog
|
|
367
|
+
|
|
368
|
+
### v0.1.0 (2024-11-19)
|
|
369
|
+
- Initial release
|
|
370
|
+
- VAE with neural ODE support
|
|
371
|
+
- scATAC-seq peak annotation pipeline
|
|
372
|
+
- Comprehensive evaluation metrics
|
|
373
|
+
- Benchmark framework for scVI models
|