pyaging 0.0.6__tar.gz → 0.0.7__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of pyaging might be problematic. Click here for more details.
- {pyaging-0.0.6 → pyaging-0.0.7}/PKG-INFO +25 -7
- {pyaging-0.0.6 → pyaging-0.0.7}/README.md +24 -6
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/predict/_pred.py +26 -21
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/predict/_pred_utils.py +112 -23
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/preprocess/_preprocess_utils.py +5 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/utils/_utils.py +79 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyproject.toml +1 -1
- {pyaging-0.0.6 → pyaging-0.0.7}/LICENSE +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/__init__.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/data/__init__.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/data/_data.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/logger/__init__.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/logger/_logger.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/models/__init__.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/models/_models.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/predict/__init__.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/predict/_postprocessing.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/predict/_preprocessing.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/preprocess/__init__.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/preprocess/_preprocess.py +0 -0
- {pyaging-0.0.6 → pyaging-0.0.7}/pyaging/utils/__init__.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: pyaging
|
|
3
|
-
Version: 0.0.
|
|
3
|
+
Version: 0.0.7
|
|
4
4
|
Summary: A Python-based compendium of GPU-optimized aging clocks.
|
|
5
5
|
License: BSD
|
|
6
6
|
Author: Lucas Paulo de Lima Camillo
|
|
@@ -48,6 +48,30 @@ Description-Content-Type: text/markdown
|
|
|
48
48
|
|
|
49
49
|
With a growing number of aging clocks and biomarkers of aging, comparing and analyzing them can be challenging. `pyaging` simplifies this process, allowing researchers to input various molecular layers (DNA methylation, histone ChIP-Seq, ATAC-seq, transcriptomics, etc.) and quickly analyze them using multiple aging clocks, thanks to its GPU-backed infrastructure. This makes it an ideal tool for large datasets and multi-layered analysis.
|
|
50
50
|
|
|
51
|
+
## ❓ Can't find an aging clock?
|
|
52
|
+
|
|
53
|
+
If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks! We are also happy to adapt to any licensing terms for commercial entities.
|
|
54
|
+
|
|
55
|
+
## 💬 Community Discussion
|
|
56
|
+
For coding-related queries, feedback, and discussions, please visit our [GitHub Issues](https://github.com/rsinghlab/pyaging/issues) page.
|
|
57
|
+
|
|
58
|
+
## 📖 Citation
|
|
59
|
+
|
|
60
|
+
To cite `pyaging`, please use the following:
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
@article {de_Lima_Camillo_pyaging,
|
|
64
|
+
author = {Lucas Paulo de Lima Camillo},
|
|
65
|
+
title = {pyaging: a Python-based compendium of GPU-optimized aging clocks},
|
|
66
|
+
elocation-id = {2023.11.28.569069},
|
|
67
|
+
year = {2023},
|
|
68
|
+
doi = {10.1101/2023.11.28.569069},
|
|
69
|
+
publisher = {Cold Spring Harbor Laboratory},
|
|
70
|
+
URL = {https://www.biorxiv.org/content/early/2023/11/29/2023.11.28.569069},
|
|
71
|
+
journal = {bioRxiv}
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
51
75
|
## 📝 To-Do List
|
|
52
76
|
|
|
53
77
|
- [X] Incorporate more murine DNA methylation clocks
|
|
@@ -57,10 +81,4 @@ With a growing number of aging clocks and biomarkers of aging, comparing and ana
|
|
|
57
81
|
- [ ] Integrate scAge clocks (this is proving to be difficult)
|
|
58
82
|
- [ ] Integrate scRNAseq clocks (and datasets)
|
|
59
83
|
|
|
60
|
-
## ❓ Can't find an aging clock?
|
|
61
|
-
|
|
62
|
-
If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks! We are also happy to adapt to any licensing terms for commercial entities.
|
|
63
|
-
|
|
64
|
-
## 💬 Community Discussion
|
|
65
|
-
For coding-related queries, feedback, and discussions, please visit our [GitHub Issues](https://github.com/rsinghlab/pyaging/issues) page.
|
|
66
84
|
|
|
@@ -23,6 +23,30 @@
|
|
|
23
23
|
|
|
24
24
|
With a growing number of aging clocks and biomarkers of aging, comparing and analyzing them can be challenging. `pyaging` simplifies this process, allowing researchers to input various molecular layers (DNA methylation, histone ChIP-Seq, ATAC-seq, transcriptomics, etc.) and quickly analyze them using multiple aging clocks, thanks to its GPU-backed infrastructure. This makes it an ideal tool for large datasets and multi-layered analysis.
|
|
25
25
|
|
|
26
|
+
## ❓ Can't find an aging clock?
|
|
27
|
+
|
|
28
|
+
If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks! We are also happy to adapt to any licensing terms for commercial entities.
|
|
29
|
+
|
|
30
|
+
## 💬 Community Discussion
|
|
31
|
+
For coding-related queries, feedback, and discussions, please visit our [GitHub Issues](https://github.com/rsinghlab/pyaging/issues) page.
|
|
32
|
+
|
|
33
|
+
## 📖 Citation
|
|
34
|
+
|
|
35
|
+
To cite `pyaging`, please use the following:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
@article {de_Lima_Camillo_pyaging,
|
|
39
|
+
author = {Lucas Paulo de Lima Camillo},
|
|
40
|
+
title = {pyaging: a Python-based compendium of GPU-optimized aging clocks},
|
|
41
|
+
elocation-id = {2023.11.28.569069},
|
|
42
|
+
year = {2023},
|
|
43
|
+
doi = {10.1101/2023.11.28.569069},
|
|
44
|
+
publisher = {Cold Spring Harbor Laboratory},
|
|
45
|
+
URL = {https://www.biorxiv.org/content/early/2023/11/29/2023.11.28.569069},
|
|
46
|
+
journal = {bioRxiv}
|
|
47
|
+
}
|
|
48
|
+
```
|
|
49
|
+
|
|
26
50
|
## 📝 To-Do List
|
|
27
51
|
|
|
28
52
|
- [X] Incorporate more murine DNA methylation clocks
|
|
@@ -32,9 +56,3 @@ With a growing number of aging clocks and biomarkers of aging, comparing and ana
|
|
|
32
56
|
- [ ] Integrate scAge clocks (this is proving to be difficult)
|
|
33
57
|
- [ ] Integrate scRNAseq clocks (and datasets)
|
|
34
58
|
|
|
35
|
-
## ❓ Can't find an aging clock?
|
|
36
|
-
|
|
37
|
-
If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks! We are also happy to adapt to any licensing terms for commercial entities.
|
|
38
|
-
|
|
39
|
-
## 💬 Community Discussion
|
|
40
|
-
For coding-related queries, feedback, and discussions, please visit our [GitHub Issues](https://github.com/rsinghlab/pyaging/issues) page.
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
import anndata
|
|
2
2
|
import gc
|
|
3
|
+
import warnings
|
|
4
|
+
from anndata import ImplicitModificationWarning
|
|
3
5
|
|
|
4
6
|
from ._pred_utils import *
|
|
5
7
|
|
|
@@ -78,6 +80,7 @@ def predict_age(
|
|
|
78
80
|
clock_name = clock_name.lower()
|
|
79
81
|
(
|
|
80
82
|
features,
|
|
83
|
+
reference_feature_values,
|
|
81
84
|
weight_dict,
|
|
82
85
|
preprocessing,
|
|
83
86
|
postprocessing,
|
|
@@ -86,21 +89,24 @@ def predict_age(
|
|
|
86
89
|
) = load_clock(clock_name, dir, logger, indent_level=2)
|
|
87
90
|
|
|
88
91
|
# Check and update adata for missing features
|
|
89
|
-
original_var_names = list(adata.var_names)
|
|
90
92
|
adata = check_features_in_adata(
|
|
91
|
-
adata,
|
|
93
|
+
adata,
|
|
94
|
+
clock_name,
|
|
95
|
+
features,
|
|
96
|
+
reference_feature_values,
|
|
97
|
+
logger,
|
|
98
|
+
indent_level=2,
|
|
92
99
|
)
|
|
93
100
|
|
|
94
101
|
# Apply preprocessing
|
|
95
|
-
|
|
96
|
-
adata
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
)
|
|
102
|
+
adata = preprocess_data(
|
|
103
|
+
adata,
|
|
104
|
+
preprocessing,
|
|
105
|
+
preprocessing_helper,
|
|
106
|
+
features,
|
|
107
|
+
logger,
|
|
108
|
+
indent_level=2,
|
|
109
|
+
)
|
|
104
110
|
|
|
105
111
|
# Filter features and then extract data matrix
|
|
106
112
|
x_numpy = filter_features_and_extract_data(
|
|
@@ -127,15 +133,14 @@ def predict_age(
|
|
|
127
133
|
predicted_ages_tensor, logger, indent_level=2
|
|
128
134
|
)
|
|
129
135
|
|
|
130
|
-
# Apply postprocessing
|
|
131
|
-
|
|
132
|
-
predicted_ages
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
)
|
|
136
|
+
# Apply postprocessing
|
|
137
|
+
predicted_ages = postprocess_data(
|
|
138
|
+
predicted_ages,
|
|
139
|
+
postprocessing,
|
|
140
|
+
postprocessing_helper,
|
|
141
|
+
logger,
|
|
142
|
+
indent_level=2,
|
|
143
|
+
)
|
|
139
144
|
|
|
140
145
|
# Add predicted ages to adata
|
|
141
146
|
add_pred_ages_adata(adata, predicted_ages, clock_name, logger, indent_level=2)
|
|
@@ -149,7 +154,7 @@ def predict_age(
|
|
|
149
154
|
)
|
|
150
155
|
|
|
151
156
|
# Reduce feature size to original features
|
|
152
|
-
adata = adata
|
|
157
|
+
adata = filter_missing_features(adata, logger, indent_level=2)
|
|
153
158
|
|
|
154
159
|
# Flush memory
|
|
155
160
|
gc.collect()
|
|
@@ -47,6 +47,7 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
|
|
|
47
47
|
tuple
|
|
48
48
|
A tuple containing the following components of the clock:
|
|
49
49
|
- features: The features used by the clock.
|
|
50
|
+
- reference_feature_values: Reference values for features in case features are missing.
|
|
50
51
|
- weight_dict: A dictionary of weights used in the clock's model.
|
|
51
52
|
- preprocessing: Any preprocessing steps required for the clock's input data.
|
|
52
53
|
- postprocessing: Any postprocessing steps applied to the clock's output.
|
|
@@ -65,7 +66,7 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
|
|
|
65
66
|
|
|
66
67
|
Examples
|
|
67
68
|
--------
|
|
68
|
-
>>> features,
|
|
69
|
+
>>> features, reference_feature_values, _, _, _, _, _ = load_clock("clock1", "pyaging_data", logger)
|
|
69
70
|
>>> print(features)
|
|
70
71
|
['feature1', 'feature2', ...]
|
|
71
72
|
|
|
@@ -81,6 +82,7 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
|
|
|
81
82
|
|
|
82
83
|
# Extract relevant information from the clock dictionary
|
|
83
84
|
features = clock_dict["features"]
|
|
85
|
+
reference_feature_values = clock_dict.get("reference_feature_values", None)
|
|
84
86
|
weight_dict = clock_dict["weight_dict"]
|
|
85
87
|
preprocessing = clock_dict.get("preprocessing", None)
|
|
86
88
|
postprocessing = clock_dict.get("postprocessing", None)
|
|
@@ -89,6 +91,7 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
|
|
|
89
91
|
|
|
90
92
|
return (
|
|
91
93
|
features,
|
|
94
|
+
reference_feature_values,
|
|
92
95
|
weight_dict,
|
|
93
96
|
preprocessing,
|
|
94
97
|
postprocessing,
|
|
@@ -102,6 +105,7 @@ def check_features_in_adata(
|
|
|
102
105
|
adata: anndata.AnnData,
|
|
103
106
|
clock_name: str,
|
|
104
107
|
features: List[str],
|
|
108
|
+
reference_feature_values: List[float],
|
|
105
109
|
logger,
|
|
106
110
|
indent_level: int = 2,
|
|
107
111
|
) -> anndata.AnnData:
|
|
@@ -110,8 +114,9 @@ def check_features_in_adata(
|
|
|
110
114
|
|
|
111
115
|
This function checks an AnnData object (commonly used in single-cell analysis) to ensure
|
|
112
116
|
that it contains all the necessary features specified in the 'features' list. If any features
|
|
113
|
-
are missing, they are added to the AnnData object with a default value of 0
|
|
114
|
-
for downstream analyses where the presence of all specified
|
|
117
|
+
are missing, they are added to the AnnData object with a default value of 0 or with a reference
|
|
118
|
+
value if given. This is crucial for downstream analyses where the presence of all specified
|
|
119
|
+
features is assumed.
|
|
115
120
|
|
|
116
121
|
Parameters
|
|
117
122
|
----------
|
|
@@ -127,6 +132,10 @@ def check_features_in_adata(
|
|
|
127
132
|
A list of features (e.g., gene names or other identifiers) that are expected to be
|
|
128
133
|
present in the 'adata'.
|
|
129
134
|
|
|
135
|
+
reference_feature_values : list
|
|
136
|
+
A list of the reference values for each feature. The order must match the order of features.
|
|
137
|
+
If no
|
|
138
|
+
|
|
130
139
|
logger : Logger
|
|
131
140
|
A logger object used for logging information about the process, such as the number
|
|
132
141
|
of missing features.
|
|
@@ -139,17 +148,17 @@ def check_features_in_adata(
|
|
|
139
148
|
-------
|
|
140
149
|
anndata.AnnData
|
|
141
150
|
The updated AnnData object, which includes any missing features added with a default
|
|
142
|
-
value of 0.
|
|
151
|
+
value of 0 (or reference value if provided).
|
|
143
152
|
|
|
144
153
|
Notes
|
|
145
154
|
-----
|
|
146
155
|
This function is particularly useful in preprocessing steps where the consistency of
|
|
147
156
|
data structure across different datasets is crucial. The function modifies the AnnData
|
|
148
|
-
object
|
|
149
|
-
these modifications.
|
|
157
|
+
object if there are missing features and logs detailed information about these modifications.
|
|
150
158
|
|
|
151
159
|
The added features are initialized with zeros. This approach, while providing completeness,
|
|
152
|
-
may introduce biases if not accounted for in downstream analyses.
|
|
160
|
+
may introduce biases if not accounted for in downstream analyses. If reference values are
|
|
161
|
+
provided, then they are used instead of zeros.
|
|
153
162
|
|
|
154
163
|
Examples
|
|
155
164
|
--------
|
|
@@ -187,32 +196,52 @@ def check_features_in_adata(
|
|
|
187
196
|
if missing_features:
|
|
188
197
|
logger.warning(
|
|
189
198
|
f"{num_missing_features} out of {total_features} features "
|
|
190
|
-
f"({percent_missing:.2f}%) are missing
|
|
191
|
-
f"added with default value 0: {missing_features[:np.min([3, num_missing_features])]}, etc.",
|
|
199
|
+
f"({percent_missing:.2f}%) are missing: {missing_features[:np.min([3, num_missing_features])]}, etc.",
|
|
192
200
|
indent_level=indent_level + 1,
|
|
193
201
|
)
|
|
194
202
|
|
|
195
|
-
#
|
|
196
|
-
|
|
203
|
+
# If there are reference values provided
|
|
204
|
+
if reference_feature_values:
|
|
205
|
+
logger.info(
|
|
206
|
+
f"Using reference feature values for {clock_name}",
|
|
207
|
+
indent_level=indent_level + 1,
|
|
208
|
+
)
|
|
209
|
+
|
|
210
|
+
# Map features to reference values
|
|
211
|
+
feature_value_map = dict(zip(features, reference_feature_values))
|
|
212
|
+
|
|
213
|
+
# Pre-allocate with reference values, if missing, use a default value (e.g., 0)
|
|
214
|
+
missing_data = np.array(
|
|
215
|
+
[feature_value_map.get(f, 0) for f in missing_features] * adata.n_obs
|
|
216
|
+
).reshape(adata.n_obs, num_missing_features)
|
|
217
|
+
else:
|
|
218
|
+
logger.info(
|
|
219
|
+
f"Filling missing features entirely with 0",
|
|
220
|
+
indent_level=indent_level + 1,
|
|
221
|
+
)
|
|
222
|
+
|
|
223
|
+
# Create an empty array
|
|
224
|
+
missing_data = np.zeros((adata.n_obs, num_missing_features))
|
|
225
|
+
|
|
197
226
|
adata_empty = anndata.AnnData(
|
|
198
|
-
X=
|
|
227
|
+
X=missing_data,
|
|
199
228
|
obs=adata.obs,
|
|
200
229
|
var=pd.DataFrame(
|
|
201
230
|
np.ones((num_missing_features, 1)),
|
|
202
231
|
index=missing_features,
|
|
203
232
|
columns=["percent_na"],
|
|
204
233
|
),
|
|
205
|
-
layers=dict(zip(adata.layers.keys(), [
|
|
234
|
+
layers=dict(zip(adata.layers.keys(), [missing_data] * len(adata.layers))),
|
|
206
235
|
uns=adata.uns,
|
|
207
236
|
)
|
|
208
237
|
|
|
209
|
-
# Concatenate original adata with the
|
|
238
|
+
# Concatenate original adata with the missing adata
|
|
210
239
|
adata = anndata.concat(
|
|
211
240
|
[adata, adata_empty], axis=1, merge="same", uns_merge="unique"
|
|
212
241
|
)
|
|
213
242
|
|
|
214
243
|
logger.info(
|
|
215
|
-
f"Expanded adata with {num_missing_features} missing features
|
|
244
|
+
f"Expanded adata with {num_missing_features} missing features",
|
|
216
245
|
indent_level=indent_level + 1,
|
|
217
246
|
)
|
|
218
247
|
else:
|
|
@@ -311,7 +340,8 @@ def initialize_model(
|
|
|
311
340
|
"ocampoatac1",
|
|
312
341
|
"ocampoatac2",
|
|
313
342
|
"bitage",
|
|
314
|
-
"
|
|
343
|
+
"zhangmortality",
|
|
344
|
+
"zhangen",
|
|
315
345
|
"leecpc",
|
|
316
346
|
"leerpc",
|
|
317
347
|
"leerefinedrpc",
|
|
@@ -418,6 +448,14 @@ def preprocess_data(
|
|
|
418
448
|
np.array([...])
|
|
419
449
|
|
|
420
450
|
"""
|
|
451
|
+
# Skip if it there is no preprocessing to be done
|
|
452
|
+
if preprocessing is None:
|
|
453
|
+
logger.info(
|
|
454
|
+
"There is no preprocessing to be done",
|
|
455
|
+
indent_level=3,
|
|
456
|
+
)
|
|
457
|
+
return adata
|
|
458
|
+
|
|
421
459
|
# Skip if it has already found the preprocessing layer
|
|
422
460
|
if f"X_{preprocessing}" in adata.layers:
|
|
423
461
|
logger.info(
|
|
@@ -435,9 +473,7 @@ def preprocess_data(
|
|
|
435
473
|
|
|
436
474
|
logger.info(f"Preprocessing data with function {preprocessing}", indent_level=3)
|
|
437
475
|
# Apply specified preprocessing method
|
|
438
|
-
if preprocessing == "
|
|
439
|
-
adata.X = np.log1p(adata.X)
|
|
440
|
-
elif preprocessing == "tpm_norm_log1p":
|
|
476
|
+
if preprocessing == "tpm_norm_log1p":
|
|
441
477
|
adata.X = tpm_norm_log1p(adata.X, preprocessing_helper)
|
|
442
478
|
elif preprocessing == "binarize":
|
|
443
479
|
adata.X = binarize(adata.X)
|
|
@@ -526,6 +562,14 @@ def postprocess_data(
|
|
|
526
562
|
(num_samples, num_features)
|
|
527
563
|
|
|
528
564
|
"""
|
|
565
|
+
# Skip if it there is no postprocessing to be done
|
|
566
|
+
if postprocessing is None:
|
|
567
|
+
logger.info(
|
|
568
|
+
"There is no postprocessing to be done",
|
|
569
|
+
indent_level=3,
|
|
570
|
+
)
|
|
571
|
+
return data
|
|
572
|
+
|
|
529
573
|
logger.info(f"Postprocessing data with function {postprocessing}", indent_level=3)
|
|
530
574
|
# Apply specified postprocessing method using vectorization
|
|
531
575
|
if postprocessing == "anti_log_linear":
|
|
@@ -604,9 +648,9 @@ def predict_ages_with_model(
|
|
|
604
648
|
|
|
605
649
|
"""
|
|
606
650
|
# Check if the size of data is greater than 1000
|
|
607
|
-
if data.size(0) >
|
|
651
|
+
if data.size(0) > 1024:
|
|
608
652
|
logger.info(
|
|
609
|
-
f"Number of samples ({data.size(0)}) is bigger than
|
|
653
|
+
f"Number of samples ({data.size(0)}) is bigger than 1024. Predicting age with DataLoader",
|
|
610
654
|
indent_level=indent_level + 1,
|
|
611
655
|
)
|
|
612
656
|
|
|
@@ -614,13 +658,15 @@ def predict_ages_with_model(
|
|
|
614
658
|
dataset = TensorDataset(data)
|
|
615
659
|
|
|
616
660
|
# Create a DataLoader
|
|
617
|
-
batch_size =
|
|
661
|
+
batch_size = 1024
|
|
618
662
|
dataloader = DataLoader(dataset, batch_size=batch_size)
|
|
619
663
|
|
|
620
664
|
# Use the DataLoader for batched prediction
|
|
621
665
|
predictions = []
|
|
622
666
|
with torch.no_grad():
|
|
623
|
-
for batch in
|
|
667
|
+
for batch in main_tqdm(
|
|
668
|
+
dataloader, indent_level=indent_level + 1, logger=logger
|
|
669
|
+
):
|
|
624
670
|
batch_data = batch[0]
|
|
625
671
|
batch_pred = model(batch_data)
|
|
626
672
|
predictions.append(batch_pred)
|
|
@@ -874,6 +920,49 @@ def add_pred_ages_adata(
|
|
|
874
920
|
adata.obs[clock_name] = predicted_ages
|
|
875
921
|
|
|
876
922
|
|
|
923
|
+
@progress("Return adata to original size")
|
|
924
|
+
def filter_missing_features(
|
|
925
|
+
adata: anndata.AnnData,
|
|
926
|
+
logger,
|
|
927
|
+
indent_level: int = 2,
|
|
928
|
+
) -> anndata.AnnData:
|
|
929
|
+
"""
|
|
930
|
+
Returns adata with original features.
|
|
931
|
+
|
|
932
|
+
This function checks for variables that have 100% of samples with missing features, and removes them
|
|
933
|
+
from the adata object. It is useful for returning the adata in the original state.
|
|
934
|
+
|
|
935
|
+
Parameters
|
|
936
|
+
----------
|
|
937
|
+
adata : anndata.AnnData
|
|
938
|
+
The AnnData object that will be filtered.
|
|
939
|
+
|
|
940
|
+
logger : Logger
|
|
941
|
+
A logger object for logging the progress or relevant information during the operation.
|
|
942
|
+
|
|
943
|
+
indent_level : int, optional
|
|
944
|
+
The indentation level for logging messages, by default 2.
|
|
945
|
+
|
|
946
|
+
Returns
|
|
947
|
+
-------
|
|
948
|
+
anndata.AnnData
|
|
949
|
+
The filtered adata in which all features appear in at least one sample.
|
|
950
|
+
|
|
951
|
+
Notes
|
|
952
|
+
-----
|
|
953
|
+
During filtering, the adata object is filtered based on a column called "percent_na" which should be less
|
|
954
|
+
than 1. If the column is not present, an error will appear.
|
|
955
|
+
|
|
956
|
+
Examples
|
|
957
|
+
--------
|
|
958
|
+
>>> adata = anndata.AnnData(np.random.rand(5, 10))
|
|
959
|
+
>>> adata = filter_missing_features(adata, logger)
|
|
960
|
+
|
|
961
|
+
"""
|
|
962
|
+
adata = adata[:, adata.var["percent_na"] < 1].copy()
|
|
963
|
+
return adata
|
|
964
|
+
|
|
965
|
+
|
|
877
966
|
@progress("Add clock metadata to adata.uns")
|
|
878
967
|
def add_clock_metadata_adata(
|
|
879
968
|
adata: anndata.AnnData,
|
|
@@ -202,6 +202,11 @@ def create_anndata_object(
|
|
|
202
202
|
obs_names = df.index
|
|
203
203
|
var_names = df.columns
|
|
204
204
|
|
|
205
|
+
# Check for duplicate features
|
|
206
|
+
if len(np.unique(var_names)) != len(var_names):
|
|
207
|
+
logger.error("There are duplicate feature names!")
|
|
208
|
+
raise ValueError
|
|
209
|
+
|
|
205
210
|
obs = pd.DataFrame(index=obs_names)
|
|
206
211
|
var = pd.DataFrame(index=var_names)
|
|
207
212
|
|
|
@@ -313,6 +313,11 @@ def cite_clock(clock_name: str, dir: str = "pyaging_data") -> None:
|
|
|
313
313
|
citation = clock_dict["citation"]
|
|
314
314
|
logger.info(f"Citation for {clock_name}:", indent_level=2)
|
|
315
315
|
logger.info(citation, indent_level=2)
|
|
316
|
+
logger.info("Please also consider citing pyaging :)", indent_level=2)
|
|
317
|
+
logger.info(
|
|
318
|
+
'de Lima Camillo, Lucas Paulo. "pyaging: a Python-based compendium of GPU-optimized aging clocks." bioRxiv (2023): 2023-11.',
|
|
319
|
+
indent_level=2,
|
|
320
|
+
)
|
|
316
321
|
else:
|
|
317
322
|
logger.warning(f"Citation not found in {clock_name}", indent_level=2)
|
|
318
323
|
else:
|
|
@@ -438,3 +443,77 @@ def get_clock_metadata(clock_name: str, dir: str = "pyaging_data") -> None:
|
|
|
438
443
|
logger.finish_progress(f"{message} finished")
|
|
439
444
|
|
|
440
445
|
logger.done()
|
|
446
|
+
|
|
447
|
+
|
|
448
|
+
def get_clock_weights(clock_name: str, dir: str = "pyaging_data") -> dict:
|
|
449
|
+
"""
|
|
450
|
+
Loads the specified aging clock from a remote source and returns a dictionary with its
|
|
451
|
+
components.
|
|
452
|
+
|
|
453
|
+
This function downloads the weights and configuration of a specified aging clock from a
|
|
454
|
+
remote server. It then loads and returns a dictionary with various components of the clock
|
|
455
|
+
such as its features, preprocessing or postprocessing steps, etc.
|
|
456
|
+
|
|
457
|
+
Parameters
|
|
458
|
+
----------
|
|
459
|
+
clock_name : str
|
|
460
|
+
The name of the aging clock whose metadata is to be retrieved. The name is case-insensitive.
|
|
461
|
+
dir : str
|
|
462
|
+
The directory to deposit the downloaded file. Defaults to 'pyaging_data'.
|
|
463
|
+
|
|
464
|
+
Returns
|
|
465
|
+
-------
|
|
466
|
+
dict
|
|
467
|
+
A dictionary containing the following components of the clock:
|
|
468
|
+
- features: The features used by the clock.
|
|
469
|
+
- reference_feature_values: Reference values for features in case features are missing.
|
|
470
|
+
- weight_dict: A dictionary of weights used in the clock's model.
|
|
471
|
+
- preprocessing: Any preprocessing steps required for the clock's input data.
|
|
472
|
+
- postprocessing: Any postprocessing steps applied to the clock's output.
|
|
473
|
+
- preprocessing_helper: Any preprocessing helper file.
|
|
474
|
+
- postprocessing_helper: Any postprocessing helper file.
|
|
475
|
+
|
|
476
|
+
Notes
|
|
477
|
+
-----
|
|
478
|
+
The clock's weights and configuration are assumed to be stored in a .pt (PyTorch) file
|
|
479
|
+
on a remote server. The URL for the clock is constructed based on the clock's name.
|
|
480
|
+
If the clock or its components are not found, the function may fail or return incomplete
|
|
481
|
+
information.
|
|
482
|
+
|
|
483
|
+
The logger is used extensively for progress tracking and information logging, enhancing
|
|
484
|
+
transparency and user experience.
|
|
485
|
+
|
|
486
|
+
Examples
|
|
487
|
+
--------
|
|
488
|
+
>>> clock_dict = get_clock_weights("clock1", "pyaging_data")
|
|
489
|
+
|
|
490
|
+
"""
|
|
491
|
+
|
|
492
|
+
logger = LoggerManager.gen_logger("get_clock_weights")
|
|
493
|
+
logger.first_info("Starting get_clock_weights function")
|
|
494
|
+
|
|
495
|
+
# Load all metadata
|
|
496
|
+
all_clock_metadata = load_clock_metadata(dir, logger, indent_level=1)
|
|
497
|
+
|
|
498
|
+
# Lowercase clock name
|
|
499
|
+
clock_name = clock_name.lower()
|
|
500
|
+
clock_dict = all_clock_metadata[clock_name]
|
|
501
|
+
|
|
502
|
+
# Check if clock name is available
|
|
503
|
+
if clock_name not in all_clock_metadata.keys():
|
|
504
|
+
logger.error(f"Clock {clock_name} is not yet available on pyaging")
|
|
505
|
+
raise ValueError
|
|
506
|
+
|
|
507
|
+
# Download weights
|
|
508
|
+
url = f"https://pyaging.s3.amazonaws.com/clocks/weights/{clock_name}.pt"
|
|
509
|
+
download(url, dir, logger, indent_level=1)
|
|
510
|
+
|
|
511
|
+
# Define the path to the clock weights file
|
|
512
|
+
weights_path = os.path.join(dir, f"{clock_name}.pt")
|
|
513
|
+
|
|
514
|
+
# Load the clock dictionary from the file
|
|
515
|
+
clock_dict = torch.load(weights_path)
|
|
516
|
+
|
|
517
|
+
logger.done()
|
|
518
|
+
|
|
519
|
+
return clock_dict
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|