pyaging 0.0.2__tar.gz → 0.0.4__tar.gz

{pyaging-0.0.2 → pyaging-0.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pyaging
-Version: 0.0.2
+Version: 0.0.4
 Summary: A Python-based compendium of GPU-optimized aging clocks.
 License: BSD
 Author: Lucas Paulo de Lima Camillo
@@ -43,53 +43,18 @@ Description-Content-Type: text/markdown
 
 `pyaging` is a cutting-edge Python package designed for the longevity research community, offering a comprehensive suite of GPU-optimized biological aging clocks.
 
-[Installation](https://pyaging.readthedocs.io/en/latest/installation.html) - [Search, cite, and get metadata](https://pyaging.readthedocs.io/en/latest/tutorial_utils.html) - [Bulk DNA methylation](https://pyaging.readthedocs.io/en/latest/tutorial_dnam.html) - [Bulk histone mark ChIP-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_histonemarkchipseq.html) - [Bulk ATAC-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_atacseq.html) - [Bulk RNA-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_rnaseq.html) - [API Reference](https://pyaging.readthedocs.io/en/latest/pyaging.html)
+[Installation](https://pyaging.readthedocs.io/en/latest/installation.html) - [Search, cite, and get metadata](https://pyaging.readthedocs.io/en/latest/tutorial_utils.html) - [Bulk DNA methylation](https://pyaging.readthedocs.io/en/latest/tutorial_dnam.html) - [Bulk histone mark ChIP-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_histonemarkchipseq.html) - [Bulk ATAC-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_atacseq.html) - [Bulk RNA-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_rnaseq.html) - [Blood chemistry](https://pyaging.readthedocs.io/en/latest/tutorial_bloodchemistry.html) - [API Reference](https://pyaging.readthedocs.io/en/latest/pyaging.html)
 
 With a growing number of aging clocks, comparing and analyzing them can be challenging. `pyaging` simplifies this process, allowing researchers to input various molecular layers (DNA methylation, histone ChIP-Seq, ATAC-seq, transcriptomics, etc.) and quickly analyze them using multiple aging clocks, thanks to its GPU-backed infrastructure. This makes it an ideal tool for large datasets and multi-layered analysis.
 
 ## 📝 To-Do List
 
-- [X] Integrate `black` for PEP8 compliant code formatting
-- [X] Improve and expand `readthedocs` documentation
-- [X] Add option to specify download directory for datasets
-- [X] Rename datasets to reflect source publications
-- [X] Include mammalian array example datasets
-- [X] Add feature to control logging verbosity
-- [X] Publish to PyPi
-- [X] Add percentage of missing features in anndata after age prediction for each clock
-- [X] Add and populate "Notes" tab in metadata
-- [X] Add PhenoAge based on blood chemistry analysis (and datasets)
-- [X] Add table in readthedocs with all clock metadata 
 - [ ] Integrate scAge and scRNAseq clocks (and datasets)
 - [ ] Incorporate murine DNA methylation and proteomic clocks (and datasets)
 
-## ⚙️ Current update pipeline
-
-For version update:
-
-- [ ] Update package version in pyproject.toml
-- [ ] Update package version in docs/source/conf.py
-- [ ] `poetry update`
-- [ ] `poetry build`
-- [ ] `poetry run pytest --nbmake tutorials/`
-- [ ] `make html` in docs folder
-- [ ] `git add .`
-- [ ] `git commit -m "commit message"`
-- [ ] `git push`
-- [ ] `git tag -a vX.X.X -m "release message"`
-- [ ] `git push origin vX.X.X`
-
-For adding a new clock:
-
-- [ ] Add clock notebook
-- [ ] Run `join_metadata.ipynb`
-- [ ] Upload new weights to AWS
-- [ ] Upload new `all_clock_metadata.pt` to AWS
-
-
 ## ❓ Can't find an aging clock?
 
-If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks!
+If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks! We are also happy to adapt to any licensing terms for commercial entities.
 
 ## 💬 Community Discussion
 For coding-related queries, feedback, and discussions, please visit our [GitHub Issues](https://github.com/rsinghlab/pyaging/issues) page.

{pyaging-0.0.2 → pyaging-0.0.4}/README.md

@@ -18,53 +18,18 @@
 
 `pyaging` is a cutting-edge Python package designed for the longevity research community, offering a comprehensive suite of GPU-optimized biological aging clocks.
 
-[Installation](https://pyaging.readthedocs.io/en/latest/installation.html) - [Search, cite, and get metadata](https://pyaging.readthedocs.io/en/latest/tutorial_utils.html) - [Bulk DNA methylation](https://pyaging.readthedocs.io/en/latest/tutorial_dnam.html) - [Bulk histone mark ChIP-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_histonemarkchipseq.html) - [Bulk ATAC-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_atacseq.html) - [Bulk RNA-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_rnaseq.html) - [API Reference](https://pyaging.readthedocs.io/en/latest/pyaging.html)
+[Installation](https://pyaging.readthedocs.io/en/latest/installation.html) - [Search, cite, and get metadata](https://pyaging.readthedocs.io/en/latest/tutorial_utils.html) - [Bulk DNA methylation](https://pyaging.readthedocs.io/en/latest/tutorial_dnam.html) - [Bulk histone mark ChIP-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_histonemarkchipseq.html) - [Bulk ATAC-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_atacseq.html) - [Bulk RNA-Seq](https://pyaging.readthedocs.io/en/latest/tutorial_rnaseq.html) - [Blood chemistry](https://pyaging.readthedocs.io/en/latest/tutorial_bloodchemistry.html) - [API Reference](https://pyaging.readthedocs.io/en/latest/pyaging.html)
 
 With a growing number of aging clocks, comparing and analyzing them can be challenging. `pyaging` simplifies this process, allowing researchers to input various molecular layers (DNA methylation, histone ChIP-Seq, ATAC-seq, transcriptomics, etc.) and quickly analyze them using multiple aging clocks, thanks to its GPU-backed infrastructure. This makes it an ideal tool for large datasets and multi-layered analysis.
 
 ## 📝 To-Do List
 
-- [X] Integrate `black` for PEP8 compliant code formatting
-- [X] Improve and expand `readthedocs` documentation
-- [X] Add option to specify download directory for datasets
-- [X] Rename datasets to reflect source publications
-- [X] Include mammalian array example datasets
-- [X] Add feature to control logging verbosity
-- [X] Publish to PyPi
-- [X] Add percentage of missing features in anndata after age prediction for each clock
-- [X] Add and populate "Notes" tab in metadata
-- [X] Add PhenoAge based on blood chemistry analysis (and datasets)
-- [X] Add table in readthedocs with all clock metadata 
 - [ ] Integrate scAge and scRNAseq clocks (and datasets)
 - [ ] Incorporate murine DNA methylation and proteomic clocks (and datasets)
 
-## ⚙️ Current update pipeline
-
-For version update:
-
-- [ ] Update package version in pyproject.toml
-- [ ] Update package version in docs/source/conf.py
-- [ ] `poetry update`
-- [ ] `poetry build`
-- [ ] `poetry run pytest --nbmake tutorials/`
-- [ ] `make html` in docs folder
-- [ ] `git add .`
-- [ ] `git commit -m "commit message"`
-- [ ] `git push`
-- [ ] `git tag -a vX.X.X -m "release message"`
-- [ ] `git push origin vX.X.X`
-
-For adding a new clock:
-
-- [ ] Add clock notebook
-- [ ] Run `join_metadata.ipynb`
-- [ ] Upload new weights to AWS
-- [ ] Upload new `all_clock_metadata.pt` to AWS
-
-
 ## ❓ Can't find an aging clock?
 
-If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks!
+If you have recently developed an aging clock and would like it to be integrated into `pyaging`, please [email us](lucas_camillo@alumni.brown.edu). We aim to incorporate it within two weeks! We are also happy to adapt to any licensing terms for commercial entities.
 
 ## 💬 Community Discussion
 For coding-related queries, feedback, and discussions, please visit our [GitHub Issues](https://github.com/rsinghlab/pyaging/issues) page.

{pyaging-0.0.2 → pyaging-0.0.4}/pyaging/logger/_logger.py

@@ -273,7 +273,8 @@ class Logger:
         self.logger_stream_handler.terminator = saved_terminator
 
     def start_progress(self, message, indent_level=1, *args, **kwargs):
-        self.log_time()
+        now = time.time()
+        self.previous_timestamp = now
         message = format_logging_message(
             "⚙️ " + message, logging.INFO, indent_level=indent_level
         )
@@ -281,8 +282,8 @@ class Logger:
         self.logger_stream_handler.flush()
 
     def finish_progress(self, progress_name="", time_unit="s", indent_level=1):
-        # self.log_time()
-        # self.report_progress(percent=100, progress_name=progress_name)#
+        now = time.time()
+        self.time_passed = now - self.previous_timestamp
 
         saved_terminator = self.logger_stream_handler.terminator
         self.logger_stream_handler.terminator = ""

{pyaging-0.0.2 → pyaging-0.0.4}/pyaging/predict/_pred.py

@@ -19,15 +19,15 @@ def predict_age(
 
     Parameters
     ----------
-    adata : AnnData
+    adata: AnnData
         An AnnData object. The object should have .X attribute for the
         data matrix and .var_names for feature names.
 
-    clock_names : str or list of str, optional
+    clock_names: str or list of str, optional
         Names of the aging clocks to be applied. It can be a single clock name as a string or a list
         of clock names, by default "horvath2013".
 
-    dir : str
+    dir: str
         The directory to deposit the downloaded file. Defaults to "pyaging_data".
 
     verbose: bool
@@ -45,8 +45,8 @@ def predict_age(
     The predicted ages are appended to the .obs attribute of the AnnData object with the clock name as
     the key. The metadata of each clock used in the prediction is stored in the .uns attribute.
 
-    It is important that the input AnnData object's .X attribute contains data
-    suitable for age prediction.
+    It is important that the input AnnData object's .X attribute contains data suitable for age 
+    prediction.
 
     The function automatically handles the transfer of data and models to the appropriate compute
     device (CPU or GPU) based on system configuration.
@@ -71,30 +71,30 @@ def predict_age(
     device = set_torch_device(logger)
 
     for clock_name in clock_names:
-        logger.info(f"Processing clock: {clock_name}", indent_level=1)
+        logger.info(f"🕒 Processing clock: {clock_name}", indent_level=1)
 
         # Load and prepare the clock
         clock_name = clock_name.lower()
-        features, weight_dict, preprocessing, postprocessing, clock_dict = load_clock(
+        features, weight_dict, preprocessing, postprocessing, preprocessing_helper, postprocessing_helper = load_clock(
             clock_name, dir, logger, indent_level=2
         )
 
         # Check and update adata for missing features
-        adata_expanded = check_features_in_adata(
+        adata = check_features_in_adata(
             adata, clock_name, features, logger, indent_level=2
         )
 
+        # Apply preprocessing 
+        if preprocessing:
+            adata = preprocess_data(
+                adata, preprocessing, preprocessing_helper, features, logger, indent_level=2
+            )
+            
         # Filter features and then extract data matrix
         x_numpy = filter_features_and_extract_data(
-            adata_expanded, features, logger, indent_level=2
+            adata, preprocessing, features, logger, indent_level=2
         )
 
-        # Apply preprocessing if specified
-        if preprocessing:
-            x_numpy = preprocess_data(
-                preprocessing, x_numpy, clock_dict, logger, indent_level=2
-            )
-
         # Convert numpy array to tensor
         x_tensor = convert_numpy_array_to_tensor(
             x_numpy, device, logger, indent_level=2
@@ -118,7 +118,7 @@ def predict_age(
         # Apply postprocessing if specified
         if postprocessing:
             predicted_ages = postprocess_data(
-                postprocessing, predicted_ages, clock_dict, logger, indent_level=2
+                predicted_ages, postprocessing, postprocessing_helper, logger, indent_level=2
             )
 
         # Add predicted ages to adata
@@ -126,11 +126,12 @@ def predict_age(
 
         # Load all clocks metadata
         all_clock_metadata = load_clock_metadata(dir, logger, indent_level=2)
-
+        
         # Add clock metadata to adata object
         add_clock_metadata_adata(
             adata, clock_name, all_clock_metadata, logger, indent_level=2
         )
+        
 
     logger.done()
     return adata

{pyaging-0.0.2 → pyaging-0.0.4}/pyaging/predict/_pred_utils.py

@@ -8,7 +8,7 @@ import ntpath
 from urllib.request import urlretrieve
 
 from ..models import *
-from ..utils import *
+from ..utils import progress, load_clock_metadata, download
 from ..logger import LoggerManager, main_tqdm, silence_logger
 from ._preprocessing import *
 from ._postprocessing import *
@@ -48,7 +48,8 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
         - weight_dict: A dictionary of weights used in the clock's model.
         - preprocessing: Any preprocessing steps required for the clock's input data.
         - postprocessing: Any postprocessing steps applied to the clock's output.
-        - clock_dict: The complete clock dictionary as loaded from the file.
+        - preprocessing_helper: Any preprocessing helper file.
+        - postprocessing_helper: Any postprocessing helper file.
 
     Notes
     -----
@@ -62,13 +63,13 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
 
     Examples
     --------
-    >>> features, weight_dict, preprocessing, postprocessing, clock_dict = load_clock("clock1", "pyaging_data", logger)
+    >>> features, weight_dict, _, _, _, _ = load_clock("clock1", "pyaging_data", logger)
     >>> print(features)
     ['feature1', 'feature2', ...]
 
     """
     url = f"https://pyaging.s3.amazonaws.com/clocks/weights/{clock_name}.pt"
-    download(url, dir, logger, indent_level=2)
+    download(url, dir, logger, indent_level=indent_level)
 
     # Define the path to the clock weights file
     weights_path = os.path.join(dir, f"{clock_name}.pt")
@@ -81,8 +82,10 @@ def load_clock(clock_name: str, dir: str, logger, indent_level: int = 2) -> Tupl
     weight_dict = clock_dict["weight_dict"]
     preprocessing = clock_dict.get("preprocessing", None)
     postprocessing = clock_dict.get("postprocessing", None)
+    preprocessing_helper = clock_dict.get("preprocessing_helper", None)
+    postprocessing_helper = clock_dict.get("postprocessing_helper", None)
 
-    return features, weight_dict, preprocessing, postprocessing, clock_dict
+    return features, weight_dict, preprocessing, postprocessing, preprocessing_helper, postprocessing_helper
 
 
 @progress("Check features in adata")
@@ -176,24 +179,27 @@ def check_features_in_adata(
             f"{num_missing_features} out of {total_features} features "
             f"({percent_missing:.2f}%) are missing and will be "
             f"added with default value 0: {missing_features[:np.min([3, num_missing_features])]}, etc.",
-            indent_level=3,
+            indent_level=indent_level+1,
         )
 
         # Create an empty AnnData object for missing features
+        empty_data = np.zeros((adata.n_obs, num_missing_features))
         adata_empty = anndata.AnnData(
-            np.zeros((adata.n_obs, num_missing_features)),
+            X=empty_data,
             obs=adata.obs,
             var=pd.DataFrame(index=missing_features),
+            layers=dict(zip(adata.layers.keys(), [empty_data] * len(adata.layers)))
         )
 
         # Concatenate original adata with the empty adata
-        adata = anndata.concat([adata, adata_empty], axis=1)
+        adata = anndata.concat([adata, adata_empty], axis=1, merge='same')
+                    
         logger.info(
             f"Expanded adata with {num_missing_features} missing features.",
-            indent_level=3,
+            indent_level=indent_level+1,
         )
     else:
-        logger.info("All features are present in adata.var_names.", indent_level=2)
+        logger.info("All features are present in adata.var_names.", indent_level=indent_level+1)
 
     return adata
 
@@ -285,6 +291,10 @@ def initialize_model(
         "ocampoatac1",
         "ocampoatac2",
         "bitage",
+        "zhang",
+        "leecpc",
+        "leerpc",
+        "leerefinedrpc",
     ]:
         model = LinearModel(len(features))
     elif clock_name in [
@@ -328,34 +338,36 @@ def initialize_model(
 
 @progress("Preprocess data")
 def preprocess_data(
+    adata: anndata.AnnData,
     preprocessing: str,
-    data: torch.Tensor,
-    clock_dict: dict,
+    preprocessing_helper,
+    features: List[str],
     logger,
     indent_level: int = 2,
-) -> torch.Tensor:
+) -> anndata.AnnData:
     """
     Preprocess the input data based on the specified method and the clock dictionary.
 
     This function applies a specified preprocessing method to the input data, which is necessary
     for aligning the data format with the requirements of the aging clock models. The function
     supports multiple preprocessing methods, including scaling, log transformation, binarization, etc.
-    The specific preprocessing method to be used is indicated by the `preprocessing` parameter, and
-    any additional parameters required for preprocessing are obtained from `clock_dict`.
+    The specific preprocessing method to be used is indicated by the `preprocessing` parameter. A new 
+    layers with the format X_{preprocessing} is added to the adata object.
 
     Parameters
     ----------
+    adata : anndata.AnnData
+        The input data to be preprocessed in AnnData format.
+
     preprocessing : str
         The name of the preprocessing method to apply. Supported methods include 'scale', 'log1p',
-        'binarize', etc.
+        'binarize', and 'quantile_normalization_with_gold_standard'.
 
-    data : torch.Tensor or similar
-        The input data to be preprocessed. It should be in a format compatible with the specified
-        preprocessing method.
+    preprocessing_helper:
+        Any preprocessing helper file, from an scikit-learn scaler object to a list of standard values.
 
-    clock_dict : dict
-        A dictionary containing clock-specific information, which may include helper functions or
-        parameters needed for preprocessing.
+    features : List[str]
+        The features used by the clock.
 
     logger : Logger
         A logger object for logging the progress and any information or warnings during preprocessing.
@@ -366,7 +378,7 @@ def preprocess_data(
 
     Returns
     -------
-    processed_data : torch.Tensor or similar
+    adata : anndata.AnnData
         The preprocessed data, ready to be input into the aging clock models.
 
     Notes
@@ -375,31 +387,50 @@ def preprocess_data(
     For instance, 'scale' may expect a numeric tensor, while 'log1p' requires non-negative values.
     Users must ensure that the input data is appropriate for the selected preprocessing method.
 
-    The `clock_dict` may include specific parameters or helper functions like `preprocessing_helper`
-    for the 'scale' method, which should be defined elsewhere in the codebase.
-
     Examples
     --------
-    >>> processed_data = preprocess_data("scale", data, clock_dict, logger)
-    >>> print(processed_data.shape)
-    torch.Size([...])
+    >>> processed_adata = preprocess_data(adata, "scale", preprocessing_helper, features, logger)
+    >>> print(processed_adata.shape)
+    np.array([...])
 
     """
+    # Skip if it has already found the preprocessing layer
+    if f"X_{preprocessing}" in adata.layers:
+        logger.info(f"Layer with {preprocessing} preprocessing is already in adata", indent_level=3)
+        return adata
+
+    # Move to adata.X for preprocessing
+    adata.X = adata.layers["X_imputed"].copy() if "X_imputed" in adata.layers else adata.layers["X_original"].copy()
+
     logger.info(f"Preprocessing data with function {preprocessing}", indent_level=3)
     # Apply specified preprocessing method
     if preprocessing == "scale":
-        data = scale(data, clock_dict["preprocessing_helper"])
+        X = adata[:, features].X
+        X = scale(X, preprocessing_helper)
+        adata[:, features].X = X
     elif preprocessing == "log1p":
-        data = np.log1p(data)
+        X = adata.X
+        X = np.log1p(X)
+        adata.X = X
     elif preprocessing == "binarize":
-        data = binarize(data)
-    return data
+        X = adata.X
+        X = binarize(X)
+        adata.X = X
+    elif preprocessing == "quantile_normalization_with_gold_standard":
+        gold_standard_df = pd.DataFrame(dict(zip(preprocessing_helper['gold_standard_probes'], preprocessing_helper['gold_standard_means'])), index=['means']).T
+        common_features = np.intersect1d(adata.var_names, gold_standard_df.index.tolist())
+        X = adata[:, common_features].X
+        X = quantile_normalize_with_gold_standard(X, gold_standard_df.loc[common_features, 'means'].tolist())
+        adata[:, common_features].X = X
+    adata.layers[f"X_{preprocessing}"] = adata.X
+        
+    return adata
 
 
 @progress("Postprocess data")
 def postprocess_data(
-    postprocessing: str,
     data: np.ndarray,
+    postprocessing: str,
     clock_dict: dict,
     logger,
     indent_level: int = 2,
@@ -414,16 +445,16 @@ def postprocess_data(
 
     Parameters
     ----------
-    postprocessing : str
-        The name of the postprocessing method to be applied. Supported methods include
-        'anti_log_linear', 'anti_logp2', 'anti_log', etc.
 
     data : array-like
         The data to be postprocessed. It should be compatible with the specified postprocessing method.
 
-    clock_dict : dict
-        A dictionary containing any additional information or parameters required for postprocessing.
-        For example, parameters for scaling factors or offsets used in anti-log transformations.
+    postprocessing : str
+        The name of the postprocessing method to be applied. Supported methods include
+        'anti_log_linear', 'anti_logp2', 'anti_log', etc.
+
+    postprocessing_helper:
+        Any postprocessing helper file, from an scikit-learn scaler object to a list of standard values.
 
     logger : Logger
         A logger object for logging the progress, information, or warnings during postprocessing.
@@ -446,7 +477,7 @@ def postprocess_data(
 
     Examples
     --------
-    >>> postprocessed_data = postprocess_data("anti_log_linear", data, clock_dict, logger)
+    >>> postprocessed_data = postprocess_data(data, "anti_log_linear", postprocessing_helper, logger)
     >>> print(postprocessed_data.shape)
     (num_samples, num_features)
 
@@ -620,10 +651,10 @@ def convert_numpy_array_to_tensor(
 
 @progress("Filter features and extract data matrix")
 def filter_features_and_extract_data(
-    adata: anndata.AnnData, features: List[str], logger, indent_level: int = 2
+    adata: anndata.AnnData, preprocessing: str, features: List[str], logger, indent_level: int = 2
 ) -> np.ndarray:
     """
-    Filter features from an AnnData object and returns the .X data matrix as a Numpy array.
+    Filter features from an AnnData object
 
     This function processes an AnnData object, which is commonly used in bioinformatics for storing large
     gene expression datasets. It extracts the data matrix corresponding to a specified set of features (genes or
@@ -635,6 +666,9 @@ def filter_features_and_extract_data(
         The AnnData object containing the dataset. Its `.X` attribute is expected to be a matrix where rows
         correspond to samples and columns correspond to features.
 
+    preprocessing : str
+        The name of the preprocessing method used (if any). Used to search adata.layers for data matrix.
+
     features : list of str
         A list of feature names to be included in the output array. Only these features from the AnnData object will
         be extracted and converted.
@@ -663,12 +697,24 @@ def filter_features_and_extract_data(
     --------
     >>> adata = anndata.AnnData(np.random.rand(5, 10))
     >>> features = ['gene1', 'gene2', 'gene3']
-    >>> numpy_array = filter_features_and_extract_data(adata, features, logger)
+    >>> numpy_array = filter_features_and_extract_data(adata, None, features, logger)
     >>> numpy_array.shape
     array([5, 3])
 
     """
-    return np.array(adata[:, features].X, dtype=np.float32)
+    if preprocessing is not None:
+        layer_name = f"X_{preprocessing}"
+    elif "X_imputed" in adata.layers:
+        layer_name = "X_imputed"
+    else:
+        layer_name = "X_original"
+    #x_numpy = np.array(adata[:, features].layers[layer_name])
+    layer_data = adata[:, features].layers[layer_name]
+
+    if not isinstance(layer_data, np.ndarray):
+        layer_data = layer_data.toarray() if hasattr(layer_data, 'toarray') else np.array(layer_data, dtype=np.float32)
+
+    return layer_data
 
 
 @progress("Add predicted ages to adata")

pyaging-0.0.4/pyaging/predict/_preprocessing.py

@@ -0,0 +1,57 @@
+import numpy as np
+
+
+def scale(x, scaler):
+    """
+    Scales the input data using the provided scaler.
+    """
+    # Apply the scaling transformation to the NumPy array
+    x_scaled = scaler.transform(x)
+
+    return x_scaled
+
+
+def binarize(x):
+    """
+    Binarizes an array based on the median of each row, excluding zeros.
+    """
+
+    # Create a mask for non-zero elements
+    non_zero_mask = x != 0
+
+    # Apply mask, calculate median for each row, and change data
+    medians = np.zeros(x.shape[0])
+    for i, row in enumerate(x):
+        non_zero_elements = row[non_zero_mask[i]]
+        x[i] = x[i] > np.median(non_zero_elements)
+
+    return x
+
+
+def quantile_normalize_with_gold_standard(x, gold_standard_means):
+    """
+    Apply quantile normalization on x using gold standard means.
+    """
+    # Create a copy of x to avoid modifying a view
+    x_normalized = x.copy()
+
+    # Sort the gold standard means
+    sorted_gold_standard = np.sort(gold_standard_means)
+
+    # Iterate through each row in x_normalized
+    for i in range(x_normalized.shape[0]):
+        # Sort the row data and store the original indices
+        sorted_indices = np.argsort(x_normalized[i, :])
+        sorted_data = x_normalized[i, sorted_indices]
+
+        # Map the sorted data to their quantile values in the gold standard
+        quantile_indices = np.round(
+            np.linspace(0, len(sorted_gold_standard) - 1, len(sorted_data))
+        ).astype(int)
+        normalized_data = sorted_gold_standard[quantile_indices]
+
+        # Re-order the normalized data to the original order
+        original_order_indices = np.argsort(sorted_indices)
+        x_normalized[i, :] = normalized_data[original_order_indices]
+
+    return x_normalized

{pyaging-0.0.2 → pyaging-0.0.4}/pyaging/preprocess/_preprocess.py

@@ -127,7 +127,7 @@ def df_to_adata(
         The DataFrame containing biological data. Rows represent samples, and columns represent features.
 
     metadata_cols : List[str]
-        A list with the name of the columns in 'df' which are part of the metadata. They will be added 
+        A list with the name of the columns in 'df' which are part of the metadata. They will be added
         to adata.obs rather than adata.X.
 
     imputer_strategy : str, optional

{pyaging-0.0.2 → pyaging-0.0.4}/pyaging/preprocess/_preprocess_utils.py

@@ -96,6 +96,7 @@ def impute_missing_values(
     logger.info(f"Imputing missing values using {strategy} strategy", indent_level=2)
     X = imputer.fit_transform(X)
     adata.X = X
+    adata.layers["X_imputed"] = adata.X
     return adata
 
 
@@ -208,6 +209,7 @@ def create_anndata_object(
         X=X,
         obs=obs,
         var=var,
+        layers={'X_original': X}
     )
 
     return adata
@@ -314,7 +316,7 @@ def add_unstructured_data(
     -------
     >>> from anndata import AnnData
     >>> adata = AnnData(np.random.rand(5, 3))
-    >>> add_unstructured_data(adata, 'mean', logger)
+    >>> adata = add_unstructured_data(adata, 'mean', logger)
     # This will add the imputer strategy 'mean' and the data type 'dna_methylation' to the AnnData object.
 
     """

{pyaging-0.0.2 → pyaging-0.0.4}/pyaging/utils/_utils.py

@@ -56,7 +56,7 @@ def progress(message: str) -> None:
         @wraps(func)
         def wrapper(*args, **kwargs):
             # Extract indent_level from kwargs, default to 1 if not provided
-            indent_level = kwargs.pop("indent_level", 1)
+            indent_level = kwargs["indent_level"] if "indent_level" in  kwargs.keys() else 1
 
             logger = args[-1]  # Assumes logger is the last positional argument
             logger.start_progress(f"{message} started", indent_level=indent_level)
@@ -70,7 +70,7 @@ def progress(message: str) -> None:
 
 
 @progress("Load all clock metadata")
-def load_clock_metadata(dir: str, logger, indent_level: int = 1) -> dict:
+def load_clock_metadata(dir: str, logger, indent_level: int = 2) -> dict:
     """
     Loads the clock metadata from a specified source.
 
@@ -114,12 +114,11 @@ def load_clock_metadata(dir: str, logger, indent_level: int = 1) -> dict:
 
     """
     url = f"https://pyaging.s3.amazonaws.com/clocks/metadata/all_clock_metadata.pt"
-    download(url, dir, logger, indent_level=2)
+    download(url, dir, logger, indent_level=indent_level)
     all_clock_metadata = torch.load(f"{dir}/all_clock_metadata.pt")
     return all_clock_metadata
 
 
-@progress("Download data")
 def download(url: str, dir: str, logger, indent_level: int = 1):
     """
     Downloads a file from a specified URL to a local directory.
@@ -165,11 +164,11 @@ def download(url: str, dir: str, logger, indent_level: int = 1):
     file_path = os.path.join(dir, file_path)
 
     if os.path.exists(file_path):
-        logger.info(f"Data found in {file_path}", indent_level=indent_level + 1)
+        logger.info(f"Data found in {file_path}", indent_level=indent_level+1)
     else:
         if not os.path.exists(dir):
             os.mkdir(dir)
-        logger.info(f"Downloading data to {file_path}", indent_level=indent_level + 1)
+        logger.info(f"Downloading data to {file_path}", indent_level=indent_level+1)
         logger.indent_level = indent_level + 1
         urlretrieve(url, file_path, reporthook=logger.request_report_hook)
 
@@ -220,7 +219,7 @@ def find_clock_by_doi(search_doi: str, dir: str = "pyaging_data") -> None:
     logger.first_info("Starting find_clock_by_doi function")
 
     # Load all metadata
-    all_clock_metadata = load_clock_metadata(dir, logger)
+    all_clock_metadata = load_clock_metadata(dir, logger, indent_level=1)
 
     # Message to indicate the start of the search process
     message = "Searching for clock based on DOI"
@@ -300,7 +299,7 @@ def cite_clock(clock_name: str, dir: str = "pyaging_data") -> None:
     clock_name = clock_name.lower()
 
     # Load all metadata
-    all_clock_metadata = load_clock_metadata(dir, logger)
+    all_clock_metadata = load_clock_metadata(dir, logger, indent_level=1)
 
     message = f"Searching for citation of clock {clock_name}"
     logger.start_progress(f"{message} started")
@@ -365,7 +364,7 @@ def show_all_clocks(dir: str = "pyaging_data") -> None:
     logger.first_info("Starting show_all_clocks function")
 
     # Load all metadata
-    all_clock_metadata = load_clock_metadata(dir, logger)
+    all_clock_metadata = load_clock_metadata(dir, logger, indent_level=1)
 
     # Message to indicate the start of the search process
     message = "Showing all available clock names"
@@ -423,7 +422,7 @@ def get_clock_metadata(clock_name: str, dir: str = "pyaging_data") -> None:
     logger.first_info("Starting get_clock_metadata function")
 
     # Load all metadata
-    all_clock_metadata = load_clock_metadata(dir, logger)
+    all_clock_metadata = load_clock_metadata(dir, logger, indent_level=1)
 
     # Lowercase clock name
     clock_name = clock_name.lower()

{pyaging-0.0.2 → pyaging-0.0.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "pyaging"
-version = "0.0.2"
+version = "0.0.4"
 description = "A Python-based compendium of GPU-optimized aging clocks."
 authors = ["Lucas Paulo de Lima Camillo <lucas_camillo@alumni.brown.edu>"]
 license = "BSD"

pyaging-0.0.2/pyaging/predict/_preprocessing.py

@@ -1,28 +0,0 @@
-import numpy as np
-
-
-def scale(data, scaler):
-    """
-    Scales the input data using the provided scaler.
-    """
-    # Apply the scaling transformation to the NumPy array
-    data_scaled = scaler.transform(data)
-
-    return data_scaled
-
-
-def binarize(data):
-    """
-    Binarizes an array based on the median of each row, excluding zeros.
-    """
-
-    # Create a mask for non-zero elements
-    non_zero_mask = data != 0
-
-    # Apply mask, calculate median for each row, and change data
-    medians = np.zeros(data.shape[0])
-    for i, row in enumerate(data):
-        non_zero_elements = row[non_zero_mask[i]]
-        data[i] = data[i] > np.median(non_zero_elements)
-
-    return data