celldl 0.1.1__tar.gz

celldl-0.1.1/PKG-INFO

@@ -0,0 +1,15 @@
+Metadata-Version: 2.2
+Name: celldl
+Version: 0.1.1
+Summary: CellDL: Defining Cell Identity by Learning Transcriptome Distributions from Single-Cell Data
+Author: Yin yusong
+Author-email: yyusong526@gmail.com
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: requires-python
+Dynamic: summary

celldl-0.1.1/README.md

@@ -0,0 +1,140 @@
+# CellDL: Defining Cell Identity by Learning Transcriptome Distributions
+
+[![PyPI version](https://badge.fury.io/py/CellDL.svg)](https://badge.fury.io/py/CellDL)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**CellDL** is a deep probabilistic representation learning framework designed to redefine how cell identity is modeled in single-cell RNA-seq (scRNA-seq) data.
+
+## 📖 Introduction & Motivation
+
+Cell identity defines what a cell is, how it functions, and what it can become. Currently, most computational approaches adopt a deterministic paradigm, compressing the cellular transcriptional state into a single, fixed vector. This approach treats a dynamic, stochastic entity as a static point, discarding the variability and uncertainty essential to biological nature.
+
+**CellDL moves from point estimates to probabilistic representations.**
+
+It represents each cell through a set of gene-wise probability distributions. By leveraging a decoupled deep learning architecture, CellDL captures the full distribution of transcriptional states, preserving biological heterogeneity and variability that traditional methods miss.
+
+## 🚀 Key Features
+
+*   **Probabilistic Representation**: Models gene expression using parametric distributions (e.g., IZIP, ZINB) rather than fixed values.
+*   **Decoupled Architecture**: Uses a shared encoder for global cell state and decoupled heads for inferring gene-specific distribution parameters ($\lambda$, $\phi$, etc.).
+*   **Biologically-Informed Denoising**: Reconstructs expression profiles based on the expected values ($\mathbb{E}$) of learned distributions, effectively removing technical noise while keeping biological signals.
+*   **Generative Data Augmentation**: Generates realistic synthetic cells via controlled perturbation of learned parameters, facilitating the analysis of rare cell populations.
+
+## 🛠 Model Architecture
+
+<div align="center">
+  <img src="docs/fig1.png" alt="CellDL Architecture" width="800"/>
+  <p><em>Figure 1: Schematic of the CellDL model architecture. The model maps cells to a latent embedding and decodes them into gene-specific distributional parameters.</em></p>
+</div>
+
+CellDL employs a **decoupled autoencoder architecture**:
+1.  **Encoder**: Maps the raw count matrix to a non-linear latent embedding.
+2.  **Decoupled Decoders**: Independently infer the parameters of the underlying distribution (e.g., Mean expression rate $\lambda$ and Dropout probability $\phi$).
+3.  **Objective**: Minimizes the difference between the expected value of the predicted distribution and the observed data using a self-supervised expectation-based loss.
+
+## 📦 Installation
+
+### Install from PyPI
+```bash
+pip install CellDL
+```
+
+### Install from Source
+```bash
+git clone https://github.com/yys-arch/CellDL.git
+cd CellDL
+pip install .
+```
+
+**Requirements:** Python >= 3.10, TensorFlow, Scanpy, AnnData, etc.
+
+## 💻 Usage Tutorial
+
+### 1. Data Preprocessing
+CellDL provides a robust preprocessing pipeline including HVG selection and normalization.
+
+```python
+import scanpy as sc
+from CellDL import data_preprocessing
+
+# Load data
+adata = sc.read_h5ad("your_data.h5ad")
+
+# Preprocess: Filter, Log-normalize, and Select HVGs
+adata = data_preprocessing(
+    adata, 
+    assay="10x 3' v3",   # Optional filtering
+    gene_mean_min=0.0125,
+    gene_mean_max=3,
+    gene_disp_min=0.5
+)
+```
+
+### 2. Model Training
+Initialize and train the model using one of the supported distribution modes. The paper highlights the **IZIP (Independent Zero-Inflated Poisson)** mode.
+
+```python
+from CellDL import build_model, train_model, save_trained_model
+
+# Build model with IZIP distribution (Recommended)
+model = build_model(adata, mode='IZIP_mode', bottle_dim=512)
+
+# Train
+history = train_model(model, adata, epochs=1000, batch_size=32)
+
+# Save
+save_trained_model(model, 'models/celldl_model.keras')
+```
+
+### 3. Denoising (Signal Reconstruction)
+Reconstruct gene expression using the expected value of the inferred distribution.
+
+```python
+from CellDL import load_trained_model, denoise_data
+
+model = load_trained_model('models/celldl_model.keras')
+adata_denoised = denoise_data(model, adata)
+
+# Result is stored in .obsm
+print(adata_denoised.obsm['rna_denoised'])
+```
+
+### 4. Synthetic Data Generation (Sample Expansion)
+Generate synthetic cells to augment rare populations by perturbing the learned parameters.
+
+```python
+from CellDL import generate_sc_synthetic_data
+
+# Generate 5 synthetic cells for every original cell
+adata_synthetic = generate_sc_synthetic_data(model, adata, num_samples=5, deviation_scale=0.1)
+```
+
+## 📊 Supported Distributions
+
+While the manuscript focuses on IZIP, the package supports multiple distribution families to fit different data characteristics:
+
+*   `IZIP_mode`: Independent Zero-Inflated Poisson (**Default**)
+*   `ZINB_mode`: Zero-Inflated Negative Binomial
+*   `NB_mode`: Negative Binomial
+*   `Mix_P_NB_mode`: Mixture of Poisson and NB
+*   (See documentation for full list of mixture models)
+
+## 📂 Data Availability
+
+The datasets used in our manuscript to benchmark and validate CellDL are publicly available through the [CZ CELLxGENE Discover](https://cellxgene.cziscience.com/) platform.
+
+| Dataset / Tissue | File Name / Description | Source Link |
+| :--- | :--- | :--- |
+| **Heart** | Tabula Sapiens - Heart | [Collection Link](https://cellxgene.cziscience.com/collections/e5f58829-1a66-40b5-a624-9046778e74f5) |
+| **Bladder** | Tabula Sapiens - Bladder | [Collection Link](https://cellxgene.cziscience.com/collections/e5f58829-1a66-40b5-a624-9046778e74f5) |
+| **Breast** | scRNA-seq data - all cells | [Collection Link](https://cellxgene.cziscience.com/collections/4195ab4c-20bd-4cd3-8b3d-65601277e731) |
+| **Bone Marrow** | Fetal Bone Marrow (10x) | [Blood and immune development...](https://cellxgene.cziscience.com/) |
+| **Large Intestine**| Tabula Sapiens - Large_Intestine | [Collection Link](https://cellxgene.cziscience.com/collections/e5f58829-1a66-40b5-a624-9046778e74f5) |
+| **Lung** | Tabula Sapiens - Lung | [Collection Link](https://cellxgene.cziscience.com/collections/e5f58829-1a66-40b5-a624-9046778e74f5) |
+| **Skin** | Skin | [Collection Link](https://cellxgene.cziscience.com/collections/43d4bb39-21af-4d05-b973-4c1fed7b916c) |
+| **Spleen** | Tabula Sapiens - Spleen | [Collection Link](https://cellxgene.cziscience.com/collections/e5f58829-1a66-40b5-a624-9046778e74f5) |
+| **iPSC-Derived EBs**<br>(Wellington et al. 2024) | Developmental Regulation of Endothelium | [Collection Link](https://cellxgene.cziscience.com/collections/4a2c25af-558a-45fc-bc9a-54ec44a1d63f) |
+
+## 📧 Contact
+
+Email: yyusong526@gmail.com

celldl-0.1.1/celldl/__init__.py

@@ -0,0 +1,3 @@
+import scanpy as sc
+from functions import (data_preprocessing, build_model, train_model, save_trained_model,
+                       load_trained_model, denoise_data, generate_sc_synthetic_data)

celldl-0.1.1/celldl/__main__.py

@@ -0,0 +1,33 @@
+import scanpy as sc
+from functions import (data_preprocessing, build_model, train_model, save_trained_model,
+                       load_trained_model, denoise_data, generate_sc_synthetic_data)
+
+
+def main_train():
+    scobj = sc.read_h5ad("your dataset")
+    scobj = data_preprocessing(scobj)
+
+    model = build_model(scobj, mode='IZIP_mode')
+    train_model(model, scobj, epochs=1000)
+
+    save_trained_model(model, 'CellDL_model.keras')
+
+def main_denoise():
+    model = load_trained_model('CellDL_model.keras')
+    scobj = sc.read_h5ad("your dataset")
+    scobj = data_preprocessing(scobj)
+    scobj_denoised = denoise_data(model, scobj)
+    return scobj_denoised
+
+def main_synthetic():
+    model = load_trained_model('CellDL_model.keras')
+    scobj = sc.read_h5ad("your dataset")
+    scobj = data_preprocessing(scobj)
+    scobj_synthetic = generate_sc_synthetic_data(model, scobj)
+    return scobj_synthetic
+
+
+if __name__ == "__main__":
+    main_train()
+    scobj_denoised = main_denoise()
+    scobj_synthetic = main_synthetic()

celldl-0.1.1/celldl/functions.py

@@ -0,0 +1,480 @@
+from tqdm import tqdm
+import numpy as np
+import pandas as pd
+import scipy
+import anndata
+import tensorflow as tf
+from tensorflow_probability import distributions as tfd
+import tf_keras.optimizers as opt
+from sklearn.preprocessing import StandardScaler
+from sklearn.metrics import adjusted_mutual_info_score
+from scipy.stats import spearmanr
+import scanpy as sc
+import warnings
+warnings.filterwarnings('ignore')
+from tf_keras import layers, models, losses, callbacks, initializers
+Model = models.Model
+Input = layers.Input
+Dense = layers.Dense
+Activation = layers.Activation
+BatchNormalization = layers.BatchNormalization
+Lambda = layers.Lambda
+PReLU = layers.PReLU
+EarlyStopping = callbacks.EarlyStopping
+MeanSquaredError = losses.MeanSquaredError
+load_model = models.load_model
+
+
+# ==============================================================================
+# Distribution Mean Functions
+# ==============================================================================
+
+@tf.function
+def rna_Negbinom_pmf(inputs):
+    """Mean of Negative Binomial (r=dispersion, theta=prob)."""
+    r, theta = inputs
+    nb = tfd.NegativeBinomial(total_count=r, probs=theta)
+    return nb.mean()
+
+
+def rna_Inflatednegbinom_pmf(inputs):
+    """Mean of Zero-Inflated Negative Binomial."""
+    r, theta, inflated_loc_prob = inputs
+    zinb = tfd.ZeroInflatedNegativeBinomial(total_count=r, probs=theta, inflated_loc_probs=inflated_loc_prob)
+    return zinb.mean()
+
+
+def rna_Inflatedpoisson(inputs):
+    """Mean of Zero-Inflated Poisson."""
+    lambda_, inflated_loc_prob = inputs
+    poissonb = tfd.Poisson(lambda_)
+    zip_dist = tfd.Inflated(distribution=poissonb, inflated_loc_probs=inflated_loc_prob)
+    return zip_dist.mean()
+
+
+def rna_Indinflatedpoisson(inputs):
+    """Mean of Independent Zero-Inflated Poisson."""
+    lambda_, inflated_loc_prob = inputs
+    poissonb = tfd.Poisson(lambda_)
+    ind_zip = tfd.Independent(
+        distribution=tfd.Inflated(distribution=poissonb, inflated_loc_probs=inflated_loc_prob),
+        reinterpreted_batch_ndims=0
+    )
+    return ind_zip.mean()
+
+
+def rna_Mixpoissonnb(inputs):
+    """Mean of Mixture (Poisson + Negative Binomial)."""
+    lambda_, r, theta, cat = inputs
+    poisson = tfd.Poisson(lambda_)
+    nb = tfd.NegativeBinomial(total_count=r, probs=theta)
+    mixpoissonnb = tfd.Mixture(
+        cat=tfd.Categorical(tf.stack([cat, 1 - cat], axis=-1)),
+        components=[poisson, nb]
+    )
+    return mixpoissonnb.mean()
+
+
+def rna_zindmixpoissonnb(inputs):
+    """Mean of Zero-Inflated Mixture (Poisson + NB)."""
+    lambda_, r, theta, cat, inflated_loc_prob = inputs
+    poisson = tfd.Poisson(lambda_)
+    nb = tfd.NegativeBinomial(total_count=r, probs=theta)
+    mixpoissonnb = tfd.Mixture(
+        cat=tfd.Categorical(tf.stack([cat, 1 - cat], axis=-1)),
+        components=[poisson, nb]
+    )
+    zindmixpoissonnb = tfd.Inflated(distribution=mixpoissonnb, inflated_loc_probs=inflated_loc_prob)
+    return zindmixpoissonnb.mean()
+
+
+def rna_Mixpoissonlognormal(inputs):
+    """Mean of Mixture (Poisson + LogNormal)."""
+    lambda_, loc, scale, cat = inputs
+    poisson = tfd.Poisson(lambda_)
+    lognormal = tfd.LogNormal(loc=loc, scale=scale)
+    mixpoissonlognormal = tfd.Mixture(
+        cat=tfd.Categorical(tf.stack([cat, 1 - cat], axis=-1)),
+        components=[poisson, lognormal]
+    )
+    return mixpoissonlognormal.mean()
+
+
+def rna_zindmixpoissonlognormal(inputs):
+    """Mean of Zero-Inflated Mixture (Poisson + LogNormal)."""
+    lambda_, loc, scale, cat, inflated_loc_prob = inputs
+    poisson = tfd.Poisson(lambda_)
+    lognormal = tfd.LogNormal(loc=loc, scale=scale)
+    mixpoissonlognormal = tfd.Mixture(
+        cat=tfd.Categorical(tf.stack([cat, 1 - cat], axis=-1)),
+        components=[poisson, lognormal]
+    )
+    zindmixpoissonlognormal = tfd.Inflated(distribution=mixpoissonlognormal, inflated_loc_probs=inflated_loc_prob)
+    return zindmixpoissonlognormal.mean()
+
+
+def rna_indzindmixpoissonlognormal(inputs):
+    """Mean of Independent Zero-Inflated Mixture (Poisson + LogNormal)."""
+    lambda_, loc, scale, cat, inflated_loc_prob = inputs
+    poisson = tfd.Poisson(lambda_)
+    lognormal = tfd.LogNormal(loc=loc, scale=scale)
+    mixpoissonlognormal = tfd.Mixture(
+        cat=tfd.Categorical(tf.stack([cat, 1 - cat], axis=-1)),
+        components=[poisson, lognormal]
+    )
+    zindmixpoissonlognormal = tfd.Inflated(distribution=mixpoissonlognormal, inflated_loc_probs=inflated_loc_prob)
+    ind_zind = tfd.Independent(distribution=zindmixpoissonlognormal, reinterpreted_batch_ndims=0)
+    return ind_zind.mean()
+
+
+def rna_indzindmixnblognormal(inputs):
+    """Mean of Independent Zero-Inflated Mixture (NB + LogNormal)."""
+    r, theta, loc, scale, cat, inflated_loc_prob = inputs
+    nb = tfd.NegativeBinomial(total_count=r, probs=theta)
+    lognormal = tfd.LogNormal(loc=loc, scale=scale)
+    mixnblognormal = tfd.Mixture(
+        cat=tfd.Categorical(tf.stack([cat, 1 - cat], axis=-1)),
+        components=[nb, lognormal]
+    )
+    zindmixnblognormal = tfd.Inflated(distribution=mixnblognormal, inflated_loc_probs=inflated_loc_prob)
+    ind_zind = tfd.Independent(distribution=zindmixnblognormal, reinterpreted_batch_ndims=0)
+    return ind_zind.mean()
+
+
+# ==============================================================================
+# Reconstruction Layers
+# ==============================================================================
+
+def NB_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for NB distribution."""
+    NorAct = lambda x: tf.clip_by_value(tf.nn.softplus(x), 3, 1e10)
+    rna_r = Dense(input_dim_rna, kernel_initializer=inikernel, activation=NorAct, name="rna_r")(h_rna_decoder_z)
+    rna_theta = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_theta")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_Negbinom_pmf, output_shape=(input_dim_rna,), name="rna_denoised")([rna_r, rna_theta])
+    return rna_mean
+
+
+def ZINB_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for ZINB distribution."""
+    NorAct = lambda x: tf.clip_by_value(tf.nn.softplus(x), 3, 1e10)
+    rna_r = Dense(input_dim_rna, kernel_initializer=inikernel, activation=NorAct, name="rna_r")(h_rna_decoder_z)
+    rna_theta = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_theta")(
+        h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_Inflatednegbinom_pmf, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_r, rna_theta, rna_zerorate])
+    return rna_mean
+
+
+def ZIP_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for ZIP distribution."""
+    rna_lambda_ = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(
+        h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_Inflatedpoisson, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_lambda_, rna_zerorate])
+    return rna_mean
+
+
+def IZIP_reconstruct(input_dim, h_rna_decoder_z, inikernel):
+    """Output layer for IZIP distribution."""
+    rna_lambda_ = Dense(input_dim, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_Indinflatedpoisson, output_shape=(input_dim,), name="rna_denoised")(
+        [rna_lambda_, rna_zerorate])
+    return rna_mean
+
+
+def Mix_P_NB_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for Mixture (Poisson + NB)."""
+    NorAct = lambda x: tf.clip_by_value(tf.nn.softplus(x), 3, 1e10)
+    rna_lambda_ = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(
+        h_rna_decoder_z)
+    rna_r = Dense(input_dim_rna, kernel_initializer=inikernel, activation=NorAct, name="rna_r")(h_rna_decoder_z)
+    rna_theta = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_theta")(
+        h_rna_decoder_z)
+    rna_cat = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_cat")(h_rna_decoder_z)
+    rna_mean = Lambda(rna_Mixpoissonnb, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_lambda_, rna_r, rna_theta, rna_cat])
+    return rna_mean
+
+
+def ZIMix_P_NB_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for Zero-Inflated Mixture (Poisson + NB)."""
+    NorAct = lambda x: tf.clip_by_value(tf.nn.softplus(x), 3, 1e10)
+    rna_lambda_ = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(
+        h_rna_decoder_z)
+    rna_r = Dense(input_dim_rna, kernel_initializer=inikernel, activation=NorAct, name="rna_r")(h_rna_decoder_z)
+    rna_theta = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_theta")(
+        h_rna_decoder_z)
+    rna_cat = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_cat")(h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_zindmixpoissonnb, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_lambda_, rna_r, rna_theta, rna_cat, rna_zerorate])
+    return rna_mean
+
+
+def Mix_P_logNormal_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for Mixture (Poisson + LogNormal)."""
+    rna_lambda_ = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(
+        h_rna_decoder_z)
+    rna_loc = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_loc")(h_rna_decoder_z)
+    rna_scale = Dense(input_dim_rna, kernel_initializer=inikernel, activation="linear", name="rna_scale")(
+        h_rna_decoder_z)
+    rna_cat = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_cat")(h_rna_decoder_z)
+    rna_mean = Lambda(rna_Mixpoissonlognormal, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_lambda_, rna_loc, rna_scale, rna_cat])
+    return rna_mean
+
+
+def ZIMix_P_logNormal_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for Zero-Inflated Mixture (Poisson + LogNormal)."""
+    rna_lambda_ = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(
+        h_rna_decoder_z)
+    rna_loc = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_loc")(h_rna_decoder_z)
+    rna_scale = Dense(input_dim_rna, kernel_initializer=inikernel, activation="linear", name="rna_scale")(
+        h_rna_decoder_z)
+    rna_cat = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_cat")(h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_zindmixpoissonlognormal, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_lambda_, rna_loc, rna_scale, rna_cat, rna_zerorate])
+    return rna_mean
+
+
+def IZIMix_P_logNormal_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for Independent Zero-Inflated Mixture (Poisson + LogNormal)."""
+    rna_lambda_ = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_lambda_")(
+        h_rna_decoder_z)
+    rna_loc = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_loc")(h_rna_decoder_z)
+    rna_scale = Dense(input_dim_rna, kernel_initializer=inikernel, activation="linear", name="rna_scale")(
+        h_rna_decoder_z)
+    rna_cat = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_cat")(h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_indzindmixpoissonlognormal, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_lambda_, rna_loc, rna_scale, rna_cat, rna_zerorate])
+    return rna_mean
+
+
+def IZIMix_NB_logNormal_reconstruct(input_dim_rna, h_rna_decoder_z, inikernel):
+    """Output layer for Independent Zero-Inflated Mixture (NB + LogNormal)."""
+    NorAct = lambda x: tf.clip_by_value(tf.nn.softplus(x), 3, 1e10)
+    rna_r = Dense(input_dim_rna, kernel_initializer=inikernel, activation=NorAct, name="rna_r")(h_rna_decoder_z)
+    rna_theta = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_theta")(
+        h_rna_decoder_z)
+    rna_loc = Dense(input_dim_rna, kernel_initializer=inikernel, activation="relu", name="rna_loc")(h_rna_decoder_z)
+    rna_scale = Dense(input_dim_rna, kernel_initializer=inikernel, activation="linear", name="rna_scale")(
+        h_rna_decoder_z)
+    rna_cat = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_cat")(h_rna_decoder_z)
+    rna_zerorate = Dense(input_dim_rna, kernel_initializer=inikernel, activation="sigmoid", name="rna_zerorate")(
+        h_rna_decoder_z)
+    rna_mean = Lambda(rna_indzindmixnblognormal, output_shape=(input_dim_rna,), name="rna_denoised")(
+        [rna_r, rna_theta, rna_loc, rna_scale, rna_cat, rna_zerorate])
+    return rna_mean
+
+
+# ==============================================================================
+# Core Functions
+# ==============================================================================
+
+def load_data(filepath, donor_id, assay, gene_mean_min, gene_mean_max, gene_disp_min):
+    """Loads and preprocesses data (Legacy wrapper). See data_preprocessing."""
+    scobj = sc.read_h5ad(filepath)
+    scobj = scobj[scobj.obs['donor_id'] == donor_id, :]
+    scobj = scobj[scobj.obs['assay'] == assay, :]
+    if scobj.raw.X is not None:
+        scobj.X = scobj.raw.X
+    scobj.var_names_make_unique()
+    scobj.var.index = pd.Index(scobj.var['feature_name'].values)
+    sc.pp.log1p(scobj)
+    sc.pp.highly_variable_genes(scobj, min_mean=gene_mean_min, max_mean=gene_mean_max, min_disp=gene_disp_min)
+    scobj = scobj[:, scobj.var["highly_variable"]]
+    scobj.obsm["rna_nor"] = scobj.X.toarray()
+    scobj.obsm["X_input"] = 1 + scobj.obsm["rna_nor"]
+    scaler = StandardScaler()
+    scobj.obsm["X_input"] = scaler.fit_transform(scobj.obsm["X_input"])
+    return scobj
+
+
+def build_model(scobj, seed=100, bottle_dim=512, mode='IZIP_mode'):
+    """
+    Builds the CellDL model.
+    Modes: 'IZIP_mode' (default), 'NB_mode', 'ZINB_mode', 'ZIP_mode', 'Mix_P_NB_mode', etc.
+    """
+    inikernel = initializers.glorot_uniform(seed=seed)
+    if "X_input" not in scobj.obsm:
+        raise ValueError("scobj.obsm['X_input'] missing. Run data_preprocessing first.")
+    input_dim = scobj.obsm["X_input"].shape[1]
+    input_data = Input(shape=(input_dim,), name='X_input')
+
+    # Encoder
+    h = input_data
+    for units in [2048, 1024]:
+        h = Dense(units, kernel_initializer=inikernel)(h)
+        h = BatchNormalization()(h)
+        h = PReLU()(h)
+
+    # Bottleneck
+    h = Dense(bottle_dim, kernel_initializer=inikernel, name="rna_features")(h)
+    h = Activation("relu")(h)
+
+    # Decoder
+    h = Dense(input_dim, kernel_initializer=inikernel, name="rec_dim")(h)
+    h = Activation("relu")(h)
+
+    # Distribution Heads
+    if mode == 'IZIP_mode':
+        rna_mean = IZIP_reconstruct(input_dim, h, inikernel)
+    elif mode == 'NB_mode':
+        rna_mean = NB_reconstruct(input_dim, h, inikernel)
+    elif mode == 'ZINB_mode':
+        rna_mean = ZINB_reconstruct(input_dim, h, inikernel)
+    elif mode == 'ZIP_mode':
+        rna_mean = ZIP_reconstruct(input_dim, h, inikernel)
+    elif mode == 'Mix_P_NB_mode':
+        rna_mean = Mix_P_NB_reconstruct(input_dim, h, inikernel)
+    elif mode == 'ZIMix_P_NB_mode':
+        rna_mean = ZIMix_P_NB_reconstruct(input_dim, h, inikernel)
+    elif mode == 'Mix_P_logNormal_mode':
+        rna_mean = Mix_P_logNormal_reconstruct(input_dim, h, inikernel)
+    elif mode == 'ZIMix_P_logNormal_mode':
+        rna_mean = ZIMix_P_logNormal_reconstruct(input_dim, h, inikernel)
+    elif mode == 'IZIMix_P_logNormal_mode':
+        rna_mean = IZIMix_P_logNormal_reconstruct(input_dim, h, inikernel)
+    elif mode == 'IZIMix_NB_logNormal_mode':
+        rna_mean = IZIMix_NB_logNormal_reconstruct(input_dim, h, inikernel)
+    else:
+        raise ValueError(f"Unknown mode: {mode}")
+
+    model = Model(inputs=input_data, outputs=rna_mean)
+    return model
+
+
+def train_model(model, scobj, lr=0.001, batch_size=32, epochs=3000):
+    """Trains the model using RMSprop and EarlyStopping."""
+    optimizer = opt.RMSprop(learning_rate=lr, clipvalue=5)
+    model.compile(optimizer=optimizer, loss=MeanSquaredError())
+    callbacks_list = [EarlyStopping(monitor="loss", patience=15, verbose=2)]
+
+    history = model.fit(
+        x=scobj.obsm["X_input"], y=scobj.obsm["rna_nor"],
+        epochs=epochs, callbacks=callbacks_list,
+        batch_size=batch_size, shuffle=True, verbose=1
+    )
+    return history
+
+
+def denoise_data(model, scobj):
+    """Denoises data by calculating the expected value of the learned distribution."""
+    temp_denoised_rna = Model(inputs=model.inputs, outputs=model.get_layer("rna_denoised").output).predict(
+        [scobj.obsm["X_input"]])
+    scobj.obsm["rna_denoised"] = temp_denoised_rna
+    scobj_denoised = sc.AnnData(
+        X=temp_denoised_rna, obs=scobj.obs, var=scobj.var,
+        obsm=scobj.obsm, layers=scobj.layers, uns=scobj.uns, varm=scobj.varm
+    )
+    return scobj_denoised
+
+
+def calculate_spearman_correlation(scobj):
+    """Calculates mean Spearman correlation between denoised and raw data."""
+    temp_denoised_rna = scobj.obsm["rna_denoised"]
+    corr_list = [spearmanr(x, y)[0] for x, y in zip(temp_denoised_rna, scobj.obsm["rna_nor"])]
+    return np.mean(corr_list)
+
+
+def save_trained_model(model, filepath):
+    """Saves the trained model to a file."""
+    model.save(filepath)
+
+
+def load_trained_model(filepath):
+    """Loads a trained model with custom distribution layers."""
+    custom_objects = {
+        'rna_Negbinom_pmf': rna_Negbinom_pmf,
+        'rna_Inflatednegbinom_pmf': rna_Inflatednegbinom_pmf,
+        'rna_Inflatedpoisson': rna_Inflatedpoisson,
+        'rna_Indinflatedpoisson': rna_Indinflatedpoisson,
+        'rna_Mixpoissonnb': rna_Mixpoissonnb,
+        'rna_zindmixpoissonnb': rna_zindmixpoissonnb,
+        'rna_Mixpoissonlognormal': rna_Mixpoissonlognormal,
+        'rna_zindmixpoissonlognormal': rna_zindmixpoissonlognormal,
+        'rna_indzindmixpoissonlognormal': rna_indzindmixpoissonlognormal,
+        'rna_indzindmixnblognormal': rna_indzindmixnblognormal,
+    }
+    return load_model(filepath, custom_objects=custom_objects, safe_mode=False)
+
+
+def generate_sc_synthetic_data(model, scobj, num_samples=1, deviation_scale=0.1):
+    """Generates synthetic cells by perturbing learned distribution parameters."""
+    X_input = scobj.obsm["X_input"]
+    num_cells, num_genes = X_input.shape
+
+    # Extract parameters and predict
+    lambda_vals = Model(inputs=model.inputs, outputs=model.get_layer("rna_lambda_").output).predict(X_input)
+    zero_vals = Model(inputs=model.inputs, outputs=model.get_layer("rna_zerorate").output).predict(X_input)
+
+    # Calculate perturbed mean
+    mean_vals = np.repeat((1 - zero_vals) * lambda_vals, num_samples, axis=0)
+    noise = np.random.uniform(-deviation_scale, deviation_scale, size=mean_vals.shape) * mean_vals
+    synthetic_data = np.clip(mean_vals + noise, a_min=0, a_max=None)
+
+    # Construct metadata
+    synthetic_obs = pd.DataFrame(np.repeat(scobj.obs.values, num_samples, axis=0), columns=scobj.obs.columns)
+    synthetic_obs['original_cell_index'] = np.repeat(np.arange(num_cells), num_samples)
+
+
+def data_preprocessing(scobj, assay=None, ID=None, gene_mean_min=0.0125, gene_mean_max=3, gene_disp_min=0.5):
+    """
+    Preprocess single-cell data for CellDL: filter, normalize, and select HVGs.
+
+    Args:
+        scobj: AnnData object.
+        assay: (Optional) Filter by 'assay' column.
+        ID: (Optional) Filter by 'donor_id' column.
+        gene_mean_min/max, gene_disp_min: Thresholds for Highly Variable Genes.
+
+    Returns:
+        AnnData object with prepared input in `.obsm['X_input']`.
+    """
+    # Use raw counts if available
+    if scobj.raw is not None:
+        scobj.X = scobj.raw.X
+    scobj.var_names_make_unique()
+
+    # Filter by assay or ID if specified and columns exist
+    if assay is not None:
+        if 'assay' in scobj.obs.columns:
+            scobj = scobj[scobj.obs['assay'] == assay].copy()
+        else:
+            warnings.warn(f"'assay' column missing; skipping filter assay='{assay}'.")
+
+    if ID is not None:
+        if 'donor_id' in scobj.obs.columns:
+            scobj = scobj[scobj.obs['donor_id'] == ID].copy()
+        else:
+            warnings.warn(f"'donor_id' column missing; skipping filter ID='{ID}'.")
+
+    # Use feature names if available
+    if 'feature_name' in scobj.var.columns:
+        scobj.var.index = pd.Index(scobj.var['feature_name'].values)
+
+    # Standard preprocessing
+    sc.pp.log1p(scobj)
+    sc.pp.highly_variable_genes(scobj, min_mean=gene_mean_min, max_mean=gene_mean_max, min_disp=gene_disp_min)
+    scobj = scobj[:, scobj.var["highly_variable"]].copy()
+
+    # Prepare dense input for model (handle sparse matrices)
+    scobj.obsm["rna_nor"] = scobj.X.toarray() if scipy.sparse.issparse(scobj.X) else scobj.X
+
+    # Scale data (StandardScaler)
+    scaler = StandardScaler()
+    scobj.obsm["X_input"] = scaler.fit_transform(1 + scobj.obsm["rna_nor"])
+
+    return scobj

celldl-0.1.1/celldl.egg-info/PKG-INFO

@@ -0,0 +1,15 @@
+Metadata-Version: 2.2
+Name: celldl
+Version: 0.1.1
+Summary: CellDL: Defining Cell Identity by Learning Transcriptome Distributions from Single-Cell Data
+Author: Yin yusong
+Author-email: yyusong526@gmail.com
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: requires-python
+Dynamic: summary

celldl-0.1.1/celldl.egg-info/SOURCES.txt

@@ -0,0 +1,9 @@
+README.md
+setup.py
+celldl/__init__.py
+celldl/__main__.py
+celldl/functions.py
+celldl.egg-info/PKG-INFO
+celldl.egg-info/SOURCES.txt
+celldl.egg-info/dependency_links.txt
+celldl.egg-info/top_level.txt

celldl-0.1.1/celldl.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

celldl-0.1.1/celldl.egg-info/top_level.txt

@@ -0,0 +1 @@
+celldl

celldl-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

celldl-0.1.1/setup.py

@@ -0,0 +1,16 @@
+from setuptools import setup, find_packages
+
+setup(
+    name='celldl',
+    version='0.1.1',
+    author='Yin yusong',
+    author_email='yyusong526@gmail.com',
+    description='CellDL: Defining Cell Identity by Learning Transcriptome Distributions from Single-Cell Data',
+    packages=find_packages(),
+    classifiers=[
+        'Programming Language :: Python :: 3',
+        'License :: OSI Approved :: MIT License',
+        'Operating System :: OS Independent',
+    ],
+    python_requires='>=3.10',
+)