dspin 1.3.2__tar.gz → 1.3.4__tar.gz

{dspin-1.3.2/dspin.egg-info → dspin-1.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: dspin
-Version: 1.3.2
+Version: 1.3.4
 Summary: Regulatory network models from single-cell perturbation profiling
 Home-page: https://github.com/JialongJiang/DSPIN
 Author: Jialong Jiang

dspin-1.3.4/README.md

@@ -0,0 +1,175 @@
+# DSPIN
+
+Tutorial, code and examples of the D-SPIN framework for the preprint "D-SPIN constructs gene regulatory network models from multiplexed scRNA-seq data revealing organizing principles of cellular perturbation response" ([bioRxiv](https://www.biorxiv.org/content/10.1101/2023.04.19.537364))
+
+<div align="center">
+  <img src="figure/readme/Overview_20250523.png" width="800">
+</div>
+
+## Installation
+
+D-SPIN is implemented in Python and MATLAB. The Python code is sufficient for the analysis of most datasets with less than 100k cells and a few hundreds of conditions. The MATLAB code is more efficient on large datasets as MATLAB can be easily deployed on clusters for parallelization with the build-in "parfor" function. 
+
+The python code can be installed with 
+
+`pip install dspin`
+
+The MATLAB code can be downloaded from the folder "DSPIN_matlab", and directly executable in MATLAB after specifying the path to the data. 
+
+## Demos
+
+Two demos of D-SPIN are available on Google Colab. 
+
+The first demo reconstructs the regulatory network of simulated hematopoietic stem cell (HSC) differentiation network with perturbations using the BEELINE framework (Pratapa, Aditya, et al. Nature methods, 2020). 
+
+[Demo1](https://colab.research.google.com/drive/1Nja8AAcP0L7Ag6DGbyn21SxGafsYX4fA?usp=sharing)
+
+The second demo reconstructs regulatory network and response vector in a public single-cell dataset of immune dictionary (Cui, Ang, et al. Nature, 2024), where mouse were treated with different cytokines and lymph nodes were than collected and profiled. 
+
+[Demo2](https://colab.research.google.com/drive/1E8v4awQ-m4D02emG0jp6kmRDmkZ8hjjN?usp=sharing)
+
+## Dependencies
+
+DSPIN package was tested with the following python packages versions:
+
+- python (3.9.18)
+- anndata (0.10.3)
+- matplotlib (3.8.2)
+- scanpy (1.9.6)
+- tqdm (4.65.0)
+- leidenalg (0.10.1)
+- igraph (0.10.8)
+
+Note: other versions may work as well.
+
+## Input data
+
+D‑SPIN can work with many perturbation types **as long as they share the same core regulatory network**:
+
+* **Genetic screens** – Perturb‑seq, CRISPR, RNAi  
+* **Chemical or signaling cues** – drug treatments, growth‑factor changes  
+* **Physiological differences** – healthy vs disease, different patients, time‑course samples  
+* **Spatial niches** – local micro‑environments from spatial transcriptomics  
+
+Because D‑SPIN models the *entire* distribution of transcriptional states, it expects **single‑cell RNA‑seq** data. Bulk data is possible but less informative.
+
+#### AnnData requirements
+
+* `adata.X` — log‑normalized (`log1p`) count matrix **after** QC: remove low‑quality cells, filter high mitochondrial content, keep highly variable genes.  
+* `adata.obs['sample_id']` — condition label; cells with the same value form one perturbation group (≥ 25 cells per group is ideal).  
+* `adata.obs['batch']` — batch identifier for cells processed together; D‑SPIN corrects batch effects by comparing conditions within each batch.  
+* `adata.obs['if_control']` — `True` for controls, `False` otherwise; perturbation effects are measured relative to these controls (or to the overall average if no controls exist).  
+
+## Building network models with D‑SPIN
+
+#### Gene‑level network models
+
+```python
+from dspin.dspin import DSPIN
+
+model = DSPIN(adata, save_path, num_spin=adata.shape[1])
+model.network_inference(
+    sample_id_key='sample_id',
+    method='pseudo_likelihood',
+    directed=True,
+    run_with_matlab=False,
+    params={'stepsz': 0.05, 'lam_l1_j': 0.01}
+)
+model.response_relative_to_control(
+    sample_id_key='sample_id',
+    if_control_key='if_control',
+    batch_key='batch'
+)
+```
+
+* **Results saved to `model`:**
+  * `model.network` – regulatory weights **J**
+  * `model.responses` – gene‑level responses **h**
+  * `model.relative_responses` – responses relative to control (per batch)
+
+* **Key options**
+  * `num_spin=adata.shape[1]` → build a *gene‑level* network  
+  * `directed=True` → only supported with `pseudo_likelihood`  
+  * `run_with_matlab=True` → skip Python inference and write variables to `save_path` for the MATLAB version (speeds up very large data sets)  
+  * `params` → hyper‑parameters such as step size, training epochs, and L1/L2 regularisation  
+
+---
+
+#### Program‑level network models
+
+```python
+from dspin.dspin import DSPIN
+
+model = DSPIN(adata, save_path, num_spin=15)      # 15 gene programs
+model.gene_program_discovery(
+    num_repeat=10,
+    seed=0,
+    cluster_key='cell_type'                       # balance cell types
+)
+model.network_inference(
+    sample_id_key='sample_id',
+    method='mcmc_maximum_likelihood'
+)
+model.response_relative_to_control()
+```
+
+* **Extra outputs**
+  * Gene‑program compositions in `save_path/onmf`
+  * Consensus program gene lists in `save_path`
+
+* **Tips**
+  * Choose `num_spin` ≈ 5 × (number of cell types); keep ≤ 40  
+  * `cluster_key` lets D‑SPIN down‑sample over‑represented cell types before oNMF  
+
+---
+
+#### Finding gene regulators of programs
+
+```python
+# model_program : program‑level DSPIN object
+# model_gene    : gene‑level   DSPIN object
+model_gene.program_regulator_discovery(
+    model_program.program_representation,
+    sample_id_key='sample_id',
+    params={'stepsz': 0.02, 'lam_l1_interaction': 0.01}
+)
+```
+
+* **Prerequisite** – `adata.obs` metadata must match between the two models so each cell’s gene and program states align.
+
+* **Outputs in `model_gene`:**
+  * `model_gene.program_interactions` – regression coefficients **J** (gene ↔ program)
+  * `model_gene.program_activities` – global activity **h** for each program
+
+
+## Application to the T cell population of the immune dictionary dataset
+
+
+<div align="center">
+  <img src="figure/readme/Figure2_immune_dictionary.png" width="600">
+</div>
+
+<div align="left">
+  <em>Figure&nbsp; Overview and program discovery for the immune dictionary dataset. (A) t-distributed stochastic neighbour (t-SNE) embedding for a subsetted cell population from the immune dictionary dataset \cite{cui2024dictionary}. The subset dataset includes CD4, CD8, and regulatory T cells treated by 12 different cytokines, as well as corresponding control samples treated by phosphate-buffered saline (PBS). 
+(B) Heatmaps of gene expression and discretized gene program levels for control and IFN-$\alpha$1-treated samples. The gene programs are weighted averages of single-gene expressions that characterize and denoise the major expression pattern of the gene matrix.</em>
+</div>
+
+<div align="center">
+  <img src="figure/readme/Figure3_program_network.png" width="600">
+</div>
+
+<div align="left">
+  <em>Figure&nbsp; Program/gene-level regulatory network inferred by D-SPIN.
+(A) Diagram of D-SPIN-inferred network model on gene programs. The network is partitioned into 4 modules, each associated with a T cell type in the population.
+(B) Heatmap of the program response of each cytokine. Clustering of the response partitioned the cytokines into 3 major categories. 
+(C) Diagram of the core subnetwork of the D-SPIN-inferred gene network model. The node sizes scale with the number of identified interactions. The network is partitioned into 4 modules, each primarily composed of genes that have elevated expression in one specific T cell type.
+(D) Interaction diagram for the subnetwork of IFN-$\alpha$1/IFN-$\beta$ and IFN-$\gamma$ acting on the program P18-IFN-$\gamma$ response. D-SPIN model shows that Type I and Type II interferons have different effectors to activate the program.</em>
+</div>
+
+## References
+
+1. Jiang, Jialong, et al. "D-SPIN constructs gene regulatory network models from multiplexed scRNA-seq data revealing organizing principles of cellular perturbation response." bioRxiv (2023).
+
+2. Pratapa, Aditya, et al. "Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data." Nature methods 17.2 (2020): 147-154.
+
+3. Cui, Ang, et al. "Dictionary of immune responses to cytokines at single-cell resolution." Nature 625.7994 (2024): 377-384.

{dspin-1.3.2 → dspin-1.3.4}/dspin/annotate.py

@@ -4,7 +4,6 @@
 @Author  :   Jialong Jiang
 '''
 
-
 import csv
 import pandas as pd
 from typing import List

{dspin-1.3.2 → dspin-1.3.4}/dspin/compute.py

@@ -20,6 +20,7 @@ from tqdm import tqdm
 from typing import Tuple, List, Callable, Any, Dict
 import warnings
 from scipy.sparse import issparse
+from joblib import Parallel, delayed, parallel_backend
 
 
 def category_balance_number(
@@ -107,7 +108,7 @@ def summary_components(all_components: np.array,
         gene_weight = np.sum(all_components ** 2, axis=0) ** 0.5
         gene_sele_filt = gene_weight > np.mean(gene_weight)
         all_components_sub = all_components[:, gene_sele_filt]
-        kmeans_gene = KMeans(n_clusters=2 * num_spin, random_state=0, n_init=10).fit(all_components_sub.T)
+        kmeans_gene = KMeans(n_clusters=min(2 * num_spin, all_components_sub.shape[1]), random_state=0, n_init=10).fit(all_components_sub.T)
 
         # Initialize an array to store the average component for each cluster.
         components_kmeans = np.zeros((num_spin, num_gene))
@@ -526,7 +527,7 @@ def sample_states(samp_full: np.array, onmf_rep_tri: np.array) -> Tuple[np.array
     return state_list, samp_list
 
 
-def para_moments(j_mat: np.array, h_vec: np.array) -> Tuple[np.array, np.array]:
+def para_moments_numpy(j_mat: np.array, h_vec: np.array) -> Tuple[np.array, np.array]:
     """
     Calculate the correlation and mean parameters given a j matrix and an h vector.
 
@@ -565,6 +566,74 @@ def para_moments(j_mat: np.array, h_vec: np.array) -> Tuple[np.array, np.array]:
 
     return corr_para, mean_para
 
+@numba.njit(fastmath=True, cache=True)
+def para_moments(j_mat: np.ndarray,
+                       h_vec: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
+    """
+    Calculate the correlation and mean parameters given an interaction
+    matrix and an external field vector, using a Numba‑accelerated kernel.
+
+    Parameters
+    ----------
+    j_mat : np.ndarray
+        Interaction matrix.
+    h_vec : np.ndarray
+        External field vector.
+
+    Returns
+    -------
+    Tuple[np.ndarray, np.ndarray]
+        The correlation parameter array of shape (n, n).
+        The mean parameter array of shape (n,).
+    """
+    num_spin: int = j_mat.shape[0]
+
+    # Copy so we can safely modify the diagonal in‑place
+    j_dbl = j_mat.copy()
+    for ii in range(num_spin):
+        j_dbl[ii, ii] += j_mat[ii, ii]
+
+    num_sample: int = 3 ** num_spin
+
+    partition: float = 0.0
+    mean_para = np.zeros(num_spin, dtype=np.float64)
+    corr_para = np.zeros((num_spin, num_spin), dtype=np.float64)
+
+    spins = np.empty(num_spin, dtype=np.int8)
+
+    for idx in range(num_sample):
+        # Decode `idx` → base‑3 spin configuration in {‑1, 0, 1}
+        tmp = idx
+        for s in range(num_spin - 1, -1, -1):
+            spins[s] = (tmp % 3) - 1
+            tmp //= 3
+
+        # Energy: E = −h·s − ½ sᵀJs
+        energy: float = 0.0
+        for ii in range(num_spin):
+            energy -= h_vec[ii] * spins[ii]
+
+        js_term: float = 0.0
+        for ii in range(num_spin):
+            dot: float = 0.0
+            for jj in range(num_spin):
+                dot += j_dbl[ii, jj] * spins[jj]
+            js_term += spins[ii] * dot
+        energy -= 0.5 * js_term
+
+        weight: float = np.exp(-energy)
+
+        partition += weight
+        for ii in range(num_spin):
+            mean_para[ii] += spins[ii] * weight
+            for jj in range(num_spin):
+                corr_para[ii, jj] += spins[ii] * spins[jj] * weight
+
+    mean_para /= partition
+    corr_para /= partition
+
+    return corr_para, mean_para
+
 
 @numba.njit(fastmath=True)
 def pseudol_gradient(cur_j: np.ndarray, 
@@ -602,7 +671,7 @@ def pseudol_gradient(cur_j: np.ndarray,
     for ii in range(num_spin):
         j_no_diag[ii, ii] = 0.0
 
-    eff_h = j_no_diag @ cur_state + cur_h.reshape(num_spin, 1)  
+    eff_h = j_no_diag @ cur_state + cur_h 
 
     exp_plus = np.empty_like(eff_h)
     exp_minus = np.empty_like(eff_h)
@@ -727,10 +796,10 @@ def samp_moments(j_mat: np.ndarray,
     return corr_para, mean_para
 
 
-def compute_gradient(cur_j: np.ndarray, 
-                     cur_h: np.ndarray, 
-                     raw_data: List[Tuple[np.ndarray, np.ndarray]], 
-                     method: str, train_dat: Dict[str, Any]) -> Tuple[np.ndarray, np.ndarray]:
+def compute_gradient(cur_j: np.ndarray,
+                            cur_h: np.ndarray,
+                            raw_data: List[Tuple[np.ndarray, np.ndarray]],
+                            method: str, train_dat: Dict[str, Any]) -> Tuple[np.ndarray, np.ndarray]:
     """
     Compute the gradient based on the specified method.
 
@@ -763,7 +832,7 @@ def compute_gradient(cur_j: np.ndarray,
         # Computing gradients using the specified method.
         if method == 'pseudo_likelihood':
             j_grad, h_grad = pseudol_gradient(
-                cur_j, cur_h[:, kk].reshape(- 1, 1), raw_data[kk], directed=train_dat['directed'])
+                cur_j, cur_h[:, kk: kk + 1], raw_data[kk], directed=train_dat['directed'])
             h_grad = h_grad.flatten()
         else:
             # Distinguishing between other methods and computing gradients
@@ -772,7 +841,7 @@ def compute_gradient(cur_j: np.ndarray,
                 corr_para, mean_para = para_moments(cur_j, cur_h[:, kk])
             elif method == 'mcmc_maximum_likelihood':
                 corr_para, mean_para = samp_moments(
-                    cur_j, cur_h, train_dat['mcmc_samplingsz'], train_dat['mcmc_samplingmix'], train_dat['mcmc_samplegap'])
+                    cur_j, cur_h[:, kk], train_dat['mcmc_samplingsz'], train_dat['mcmc_samplingmix'], train_dat['mcmc_samplegap'])
             j_grad = corr_para - raw_data[kk][0]
             h_grad = mean_para - raw_data[kk][1].flatten()
 
@@ -782,6 +851,77 @@ def compute_gradient(cur_j: np.ndarray,
     return rec_jgrad, rec_hgrad
 
 
+def _gradient_one_round(kk: int,
+                        cur_j: np.ndarray,
+                        cur_h: np.ndarray,
+                        raw_data_k,
+                        method: str,
+                        train_dat: Dict[str, Any]):
+    """Helper that compute gradient for a single round."""
+    if method == 'pseudo_likelihood':
+        j_grad, h_grad = pseudol_gradient(cur_j, cur_h[:, kk:kk + 1], raw_data_k, directed=train_dat['directed'])
+        h_grad = h_grad.flatten()
+    else:
+        if method == 'maximum_likelihood':
+            corr_para, mean_para = para_moments(cur_j, cur_h[:, kk])
+        elif method == 'mcmc_maximum_likelihood':
+            corr_para, mean_para = samp_moments(
+                    cur_j, cur_h[:, kk], train_dat['mcmc_samplingsz'], train_dat['mcmc_samplingmix'], train_dat['mcmc_samplegap'])
+        j_grad  = corr_para - raw_data_k[0]
+        h_grad  = mean_para  - raw_data_k[1].flatten()
+
+    return kk, j_grad, h_grad
+
+
+def compute_gradient_para(cur_j: np.ndarray,
+                     cur_h: np.ndarray,
+                     raw_data: List[Tuple[np.ndarray, np.ndarray]],
+                     method: str, train_dat: Dict[str, Any]) -> Tuple[np.ndarray, np.ndarray]:
+    """
+    Compute the gradient based on the specified method with parallel processing.
+
+    Parameters
+    ----------
+    cur_j : np.ndarray
+        Current j matrix.
+    cur_h : np.ndarray
+        Current h matrix.
+    raw_data : List[Tuple[np.ndarray, np.ndarray]]
+        Raw data used to calculate the gradient (each element is a tuple of correlation and mean).
+    method : str
+        The method to calculate the gradient; possible values are 'pseudo_likelihood',
+        'maximum_likelihood', and 'mcmc_maximum_likelihood'.
+    train_dat : Dict[str, Any]
+        Training data and hyperparameters.
+
+    Returns
+    -------
+    Tuple[np.array, np.array]
+        The gradient of j.
+        The gradient of h.
+    """
+    num_spin, num_round = cur_h.shape
+    rec_jgrad = np.zeros((num_spin, num_spin, num_round))
+    rec_hgrad = np.zeros((num_spin, num_round))
+
+    n_jobs = train_dat.get('max_workers', min(num_round, os.cpu_count())) 
+    chunk = train_dat.get('chunk_size', 1)          # rounds per task
+    backend = 'loky'                                # process‑based
+
+    # print(f'Processing with {n_jobs} workers...')
+    with parallel_backend(backend, n_jobs=n_jobs):
+        results = Parallel(batch_size=chunk)(
+            delayed(_gradient_one_round)(kk, cur_j, cur_h, raw_data[kk], method, train_dat)
+            for kk in range(num_round)
+        )
+
+    for kk, j_grad, h_grad in results:
+        rec_jgrad[:, :, kk] = j_grad
+        rec_hgrad[:, kk] = h_grad
+
+    return rec_jgrad, rec_hgrad
+
+
 def apply_regularization(rec_jgrad: np.ndarray, 
                          rec_hgrad: np.ndarray, 
                          cur_j: np.ndarray, 
@@ -964,6 +1104,8 @@ def learn_network_adam(raw_data: Any,
 
     backtrack_counter = 0
     counter = 1
+    
+    tqdm._instances.clear()
     pbar = tqdm(total=num_epoch)
 
     while counter <= num_epoch:

{dspin-1.3.2 → dspin-1.3.4}/dspin/dspin.py

@@ -34,6 +34,7 @@ import matplotlib.patheffects as patheffects
 import warnings
 import itertools
 from typing import List, Dict, Optional, Tuple, Union
+from scipy.stats import skew
 
 
 class AbstractDSPIN(ABC):
@@ -206,6 +207,7 @@ class AbstractDSPIN(ABC):
             self._samp_list = self._samp_list[example_list_ind]
             raw_data = raw_data[example_list_ind]
             self._raw_data = raw_data
+            print("Using a subset of {} samples for network inference.".format(len(self._samp_list)))
 
         num_sample = len(raw_data)
         params = {'num_epoch': 400,
@@ -337,8 +339,8 @@ class AbstractDSPIN(ABC):
             raise ValueError(f"Column '{batch_key}' not found in adata.obs.")
         
         sample_list = self._samp_list
-        sample_to_control = self.adata.obs.groupby(sample_id_key)[if_control_key].first().to_dict()
-        sample_to_batch = self.adata.obs.groupby(sample_id_key)[batch_key].first().to_dict()
+        sample_to_control = self.adata.obs.groupby(sample_id_key, observed=False)[if_control_key].first().to_dict()
+        sample_to_batch = self.adata.obs.groupby(sample_id_key, observed=False)[batch_key].first().to_dict()
 
         if_control = np.array([sample_to_control[sample] for sample in sample_list])
         batch_index = np.array([sample_to_batch[sample] for sample in sample_list])
@@ -382,8 +384,20 @@ class GeneDSPIN(AbstractDSPIN):
         else:
             self._onmf_rep_ori = adata.X
 
+        gene_skew = np.zeros(num_spin)
+
+        for ii in range(num_spin):
+            cur_gene = self._onmf_rep_ori[:, ii]
+            gene_skew[ii] = skew(cur_gene, bias=False)
+
+        if gene_skew.mean() > 2:
+            default_clip = 95
+            print("Input gene expression seem to be heavy-tailed. Using 95 percentile for clipping before discretization.")
+        else:
+            default_clip = 100
+
         dparams = {'use_default_discretize': True,
-                   'clip_percentile': 95,
+                   'clip_percentile': default_clip,
                    'num_init': 10}
         dparams.update(discretize_params)
         
@@ -484,6 +498,42 @@ class ProgramDSPIN(AbstractDSPIN):
     def onmf_decomposition(self, value):
         self._onmf_summary = value
 
+    def compute_onmf_expression(self, 
+                                sample_matrix: np.ndarray,
+                                gene_group_ind: List[List[int]]):
+        """
+        Compute the oNMF expression for a given sample matrix and gene components of each program.
+
+        Parameters
+        ----------
+        sample_matrix : np.ndarray
+            The sample matrix containing gene expressions.
+        gene_group_ind : List[List[int]]
+            List of lists containing indices of genes for each program component.
+
+        Returns
+        -------
+        NMF
+            An NMF object containing the computed components.
+        """
+        num_spin = self.num_spin
+        adata = self.adata
+
+        # Initialize and compute the summary matrix
+        components_summary = np.zeros((num_spin, adata.shape[1]))
+        for ii in range(num_spin):
+            sub_matrix = sample_matrix[:, gene_group_ind[ii]]
+            sub_onmf = NMF(n_components=1, init='random', random_state=0).fit(sub_matrix)
+            components_summary[ii, gene_group_ind[ii]] = sub_onmf.components_
+
+        # Normalize the summary components and finalize the onmf_summary object
+        components_summary = normalize(components_summary, axis=1, norm='l2')
+        onmf_summary = NMF(n_components=num_spin, init='random', random_state=0)
+        onmf_summary.components_ = components_summary
+
+        return onmf_summary
+
+
     def summarize_onmf_result(self,
                               large_subsample_matrix: np.ndarray,
                               num_onmf_components: int,
@@ -504,10 +554,6 @@ class ProgramDSPIN(AbstractDSPIN):
         NMF
             An NMF object containing the summarized components.
         """
-
-        # Retrieve the number of spins and annotated data
-        num_spin = self.num_spin
-        adata = self.adata
         prior_programs_mask = self.prior_programs_mask
         seed_list = self.onmf_parameters['seed_list']
         summary_method = self.onmf_parameters['summary_method']
@@ -520,7 +566,7 @@ class ProgramDSPIN(AbstractDSPIN):
             cur_onmf = np.load(self.save_path + f'onmf/onmf_components_{num_onmf_components}_repeat_{cur_seed}.npy', allow_pickle=True).item()
             rec_components[ii] = cur_onmf.components_
 
-        all_components = rec_components.reshape(-1, np.sum(prior_programs_mask))
+        all_components = rec_components.reshape(- 1, np.sum(prior_programs_mask))
 
         gene_group_ind = summary_components(all_components, num_onmf_components, summary_method=summary_method, figure_folder=self.fig_folder)
 
@@ -529,17 +575,8 @@ class ProgramDSPIN(AbstractDSPIN):
         gene_group_ind = [sub_mask_ind[gene_list] for gene_list in gene_group_ind]
         gene_group_ind += self.prior_programs_ind
 
-        # Initialize and compute the summary matrix
-        components_summary = np.zeros((num_spin, adata.shape[1]))
-        for ii in range(num_spin):
-            sub_matrix = large_subsample_matrix[:, gene_group_ind[ii]]
-            sub_onmf = NMF(n_components=1, init='random', random_state=0).fit(sub_matrix)
-            components_summary[ii, gene_group_ind[ii]] = sub_onmf.components_
-
-        # Normalize the summary components and finalize the onmf_summary object
-        components_summary = normalize(components_summary, axis=1, norm='l2')
-        onmf_summary = NMF(n_components=num_spin, init='random', random_state=0)
-        onmf_summary.components_ = components_summary
+        # Evaluate the expression level of each program and the weights for each gene in the program 
+        onmf_summary = self.compute_onmf_expression(large_subsample_matrix, gene_group_ind)
 
         return onmf_summary
 
@@ -549,7 +586,6 @@ class ProgramDSPIN(AbstractDSPIN):
                                cluster_key: str = 'leiden',
                                mode: str = 'compute_summary',
                                prior_programs: List[List[str]] = None,
-                               clip_percentile: float = 100,
                                params: dict = {},
                                discretize_params: dict = {}) -> None:
         """
@@ -567,10 +603,10 @@ class ProgramDSPIN(AbstractDSPIN):
             Mode of operation. Options: 'compute_summary', 'compute_only', 'summary_only'. Default is 'compute_summary'.
         prior_programs : List[List[str]], optional
             List of predefined gene programs. Default is None.
-        clip_percentile : float, optional
-            Percentile for clipping values before discretization. Default is 100.
         params : dict, optional
             Additional parameters for oNMF. Default is an empty dictionary.
+        discretize_params : dict, optional
+            Parameters for discretization of the oNMF representation. Default is an empty dictionary.
         """
 
         if mode not in ['compute_summary', 'compute_only', 'summary_only']:
@@ -595,6 +631,7 @@ class ProgramDSPIN(AbstractDSPIN):
         
         if onmf_parameters['num_subsample_large'] is None:
             onmf_parameters['num_subsample_large'] = min(onmf_parameters['num_subsample'] * 5, adata.shape[0])
+            print(onmf_parameters['num_subsample_large'])
 
         if onmf_parameters['balance_method'] not in ['equal', 'proportional', 'squareroot', None]:
             raise ValueError('balance_method must be one of equal, proportional, squareroot, or None')
@@ -607,7 +644,13 @@ class ProgramDSPIN(AbstractDSPIN):
             prior_program_ind = [[np.where(adata.var_names == gene)[0][0] for gene in gene_list] for gene_list in prior_programs]
             preprogram_flat = [gene for program in prior_programs for gene in program]
             if len(prior_programs) > num_spin:
-                raise ValueError('Number of prior_programs must be less than the number of spins')
+                raise ValueError('Number of prior programs must be less than the number of programs')
+            elif len(prior_programs) == num_spin:
+                warnings.warn('Number of prior programs is equal to the number of programs. No oNMF is performed.')
+                mode = 'summary_only'
+            elif len(prior_programs) == 0:
+                warnings.warn('Empty prior programs')
+
             # Generate mask to exclude genes in prior programs from the annotated data
             prior_program_mask = ~ np.isin(adata.var_names, preprogram_flat)
             onmf_parameters['num_onmf_components'] -= len(prior_programs)
@@ -639,13 +682,19 @@ class ProgramDSPIN(AbstractDSPIN):
                 compute_onmf_decomposition(sub_gene_matrix_normed, num_onmf_components, seed=cur_seed, params=onmf_parameters, save_path=self.save_path + f'onmf/onmf_components_{num_onmf_components}_repeat_{cur_seed}.npy')
             
         if mode == 'compute_summary' or mode == 'summary_only':
+            
+            cluster_label_raw = adata.obs[cluster_key].values
+            cluster_label = pd.factorize(cluster_label_raw)[0]
 
             # Perform subsampling and standard deviation clipping on the matrix
-            std_clipped, sub_large_gene_matrix_normed = subsample_normalize_gene_matrix(cur_gene_matrix, cluster_label, num_subsample=onmf_parameters['num_subsample_large'], seed=seed + rep, params=onmf_parameters)
+            std_clipped, sub_large_gene_matrix_normed = subsample_normalize_gene_matrix(adata.X, cluster_label, num_subsample=onmf_parameters['num_subsample_large'], seed=seed, params=onmf_parameters)
             self.matrix_std = std_clipped
             
             # Summarize the oNMF decompositions
-            onmf_summary = self.summarize_onmf_result(sub_large_gene_matrix_normed, num_onmf_components, num_repeat)
+            if num_onmf_components > 0:
+                onmf_summary = self.summarize_onmf_result(sub_large_gene_matrix_normed, num_onmf_components, num_repeat)
+            else:
+                onmf_summary = self.compute_onmf_expression(sub_large_gene_matrix_normed, prior_program_ind)
             
             self._onmf_summary = onmf_summary
 

{dspin-1.3.2 → dspin-1.3.4/dspin.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: dspin
-Version: 1.3.2
+Version: 1.3.4
 Summary: Regulatory network models from single-cell perturbation profiling
 Home-page: https://github.com/JialongJiang/DSPIN
 Author: Jialong Jiang

{dspin-1.3.2 → dspin-1.3.4}/setup.py

@@ -21,7 +21,7 @@ install_requires = [
 # ]
 
 # Managing your version (adjust accordingly)
-version = "1.3.2"  # You can manage versioning in a more sophisticated way if needed
+version = "1.3.4"  # You can manage versioning in a more sophisticated way if needed
 
 # Long description (usually a README file)
 readme = open("README_PyPI.md").read()  # Adjust the file name if necessary

dspin-1.3.2/README.md

@@ -1,134 +0,0 @@
-# DSPIN
-
-Tutorial, code and examples of the D-SPIN framework for the preprint "D-SPIN constructs gene regulatory network models from multiplexed scRNA-seq data revealing organizing principles of cellular perturbation response" ([bioRxiv](https://www.biorxiv.org/content/10.1101/2023.04.19.537364))
-
-![alternativetext](figure/readme/Figure1_20230309_Inna.png)
-
-## Installation
-
-D-SPIN is implemented in Python and Matlab. The Python code is sufficient for analysis of small datasets with around 20k cells and tens of conditions. The Matlab code is only used for network inference on large datasets as Matlab can be easily deployed on clusters for parallelization with the build-in "parfor" function. 
-
-The python code can be installed with 
-
-`pip install dspin`
-
-The Matlab code can be downloaded from the folder "DSPIN_matlab", and directly executable in Matlab after specifying the path to the data. 
-
-## Demos
-
-Two demos of D-SPIN are available on Google Colab. 
-
-The first demo reconstructs the regulatory network of simulated hematopoietic stem cell (HSC) differentiation network with perturbations using the BEELINE framework (Pratapa, Aditya, et al. Nature methods, 2020). 
-
-[Demo1](https://colab.research.google.com/drive/1mjJ09_E9kjEYsGCzf10-aZfuOqHUEV5u?usp=sharing)
-
-The second demo reconstructs regulatory network and response vector in a single-cell dataset collected in the ThomsonLab.In the dataset, human peripheral blood mononuclear cells (PBMCs) were treated with various signaling molecules with different dosages. 
-
-[Demo2](https://colab.research.google.com/drive/1U2WbmKvai3Y_-vlLV0SY1dv1LHdFjg2w?usp=sharing)
-
-## Dependencies
-
-DSPIN package was tested with the following python packages versions:
-
-- python (3.9.18)
-- anndata (0.10.3)
-- matplotlib (3.8.2)
-- scanpy (1.9.6)
-- tqdm (4.65.0)
-- leidenalg (0.10.1)
-- igraph (0.10.8)
-
-Note: other versions may work as well.
-
-## D-SPIN Overview
-
-D-SPIN contains three main steps: 
-* Gene program discovery
-* Network inference
-* Network analysis
-
-The input data should be AnnData object after typical single-cell preprocessing steps including cell counts normalization, log transformation, highly-varible gene selection, and clustering (optional). The input data should contain the following information. The attribute name is subject to the user while the following are defaults of the code:
-* Normalized, log-transformed and filtered gene expression matrix in adata.X
-* Sample information for each cell in adata.obs['sample_id']
-* Batch information for each cell in adata.obs['batch']
-* Clustering information for each cell in adata.obs['leiden']
-* If some samples are control conditions, the relative response vectors will be computed for each batch with respect to the average of control conditions. The control conditions can be indicated in adata.obs['if_control'] with 1 for control and 0 for non-control. Note that all cells in the same sample and batch should have the same value in adata.obs['if_control'].
-
-The D-SPIN is initiallized by model = dspin.DSPIN(adata) with the following major arguments:
-
-* adata: AnnData object with the above information
-* save_path: path to save the results
-* num_spin (default 15): number of gene programs in the model
-* filter_threshold (default 0.02): threshold for minimal proportion of cells that each gene program is expressed in
-
-### Gene program discovery
-
-By default, D-SPIN use orgthogonal non-negative matrix factorization (oNMF) to discover gene programs that coexpress in the data. The user can also specify pre-defined gene programs in full or partial. There are three senarios depending on the num_spin and the input parameter prior_programs.
-* No pre-defined gene programs: empty prior_programs (default)
-* Full pre-defined gene programs: the number of gene programs in prior_programs is equal to num_spin
-* Partial pre-defined gene programs: the number of gene programs in prior_programs is less than num_spin
-
-The gene program discovery contains two stages, the first stage subset and normalize the gene matrix, and the second stage run oNMF multiple times on the subset matrix to obtain consensus gene programs. Subsetting the gene matrix is primarily for computational efficiency, and preferrentially sampling cells with smaller population also helps to avoid overfitting on over-represented cell states in the dataset. As the algorithm for oNMF is stochastic, the algorithm is run multiples times and results from multiple run are combined to obtain a set of consensus gene program. Also, the number of oNMF components can be specified by the user to discover gene programs with finer resolution. For example, run the algorithm with 20 components in each individual run but combine the 20 programs into 15 consensus programs. 
-
-Overall, in the gene program discovery function model.gene_program_discovery() , D-SPIN takes the following major arguments: 
-
-* num_onmf_components (int, optional): The number of oNMF components. Default is infer based on num_spin and pre-assigned programs.
-* num_subsample (int, optional): Number of cells to use in each individual oNMF components computation. Default is 10000.
-* num_subsample_large (int, optional): Number of cells to use in computing consensus oNMF components. Default is 5 times of num_subsample.
-* num_repeat (int, optional): Number of times to repeat the oNMF computation. Default is 10.
-* balance_obs (str, optional): Cell type or clustering label in adata.obs for balanced sampling. Default is None.
-* balance_method (str, optional): Method used for balancing. Options are 'squareroot', 'equal', 'proportional'. Default is 'squareroot'. 
-* max_sample_rate (float, optional): Maximum oversampling rate for underrepresented cell type or clustering categories. Default is 2.
-* prior_programs (List[List[str]], optional): List of pre-assigned gene programs. Default is None.
-* summary_method (str, optional): Method used for computing sensus oNMF components. Options are 'kmeans' and 'leiden'. Default is 'kmeans'.
-
-### Network inference
-
-In the network inference step, D-SPIN automatically choose between three inference methods depending on the number of gene programs:
-
-* Exact maximum-likelihood inference (MLE)
-* Markov-Chain Monte-Carlo (MCMC) maximum-likelihood inference 
-* Pseudo-likelihood inference (PL). 
-
-MLE is exactly solution of the problem but only applies to small network. MCMC is slightly more scalable to networks with 20-40 nodes. PL takes more simplified approximation to the problem which scales to hundreds of nodes, but requires larger cell number for the approximation to be effective. in the gene program discovery function model.network_inference(), D-SPIN takes the following major arguments
-
-* sample_col_name (str, optional): Column name in adata.obs for sample information. Default is 'sample_id'
-* control_col_name (str, optional): Column name in adata.obs for whether the cell is in a control sample. If not provided, all cells are treated as non-control and relative response vector are the same as response vector.
-* batch_col_name (str, optional): Column name in adata.obs for batch information. Relative response are computed by the control response vector in the same batch. If not provided, all cells are treated as one batch. 
-* method (str, optional): Method used for network inference. Options are 'auto', 'maximum_likelihood', 'mcmc_maximum_likelihood', 'pseudo_likelihood'.
-* example_list (List[str], optional): List of samples to use for network inference. Default is using all samples available in adata.obs[sample_col_name].
-* record_step (int, optional): Number of steps to record training progress. Default is 10.
-* params (Dict[str, Any], optional): Hypterparameters for the gradient descent of network inference. Default is None.
-
-
-### Network analysis 
-
-After network inference, D-SPIN provides three outputs:
-
-* Inferred network adjacenty matrix in model.network 
-* Inferred pertrubation response in model.response
-* Relative perturbation response to control in model.relative_response
-
-Typical downstream analysis includes network visualization, module identification, and graph clustering of perturbation response with the network. 
-
-## Application to signaling response data of human PBMCs
-
-D-SPIN takes single-cell sequencing data of multiple perturbation conditions. In the second demo, PBMCs are treated with different signaling molecules such as CD3 antibody, LPS, IL1B, and TGFB1
-
-![alternativetext](figure/thomsonlab_signaling/example_conditions.png)
-
-D-SPIN identifies a set of gene programs that coexpress in the data, and represent each cell as a combination of gene program expression states. 
-
-![alternativetext](figure/thomsonlab_signaling/gene_program_example.png)
-
-D-SPIN uses cross-correlation and mean of each perturbation condition to inferred a unified regulatory network and the response vector of each perturbation condition. The inference can be parallelized across perturbation conditions. The inference code is in Matlab using "parfor", while for demo purpose Python code (without parallelization) is provided.
-
-The inferred regulatory network and perturbations can be jointly analyzed to reveal how perturbations act in the context of the regulatory network.
-
-![alternativetext](figure/thomsonlab_signaling/joint_network_perturbation.png)
-
-## References
-
-1. Jiang, Jialong, et al. "D-SPIN constructs gene regulatory network models from multiplexed scRNA-seq data revealing organizing principles of cellular perturbation response." bioRxiv (2023).
-
-2. Pratapa, Aditya, et al. "Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data." Nature methods 17.2 (2020): 147-154.