@saibolla/ada 0.1.2

package/skills/scvi-tools/references/models-atac-seq.md

@@ -0,0 +1,321 @@
+# ATAC-seq and Chromatin Accessibility Models
+
+This document covers models for analyzing single-cell ATAC-seq and chromatin accessibility data in scvi-tools.
+
+## PeakVI
+
+**Purpose**: Analysis and integration of single-cell ATAC-seq data using peak counts.
+
+**Key Features**:
+- Variational autoencoder specifically designed for scATAC-seq peak data
+- Learns low-dimensional representations of chromatin accessibility
+- Performs batch correction across samples
+- Enables differential accessibility testing
+- Integrates multiple ATAC-seq datasets
+
+**When to Use**:
+- Analyzing scATAC-seq peak count matrices
+- Integrating multiple ATAC-seq experiments
+- Batch correction of chromatin accessibility data
+- Dimensionality reduction for ATAC-seq
+- Differential accessibility analysis between cell types or conditions
+
+**Data Requirements**:
+- Peak count matrix (cells × peaks)
+- Binary or count data for peak accessibility
+- Batch/sample annotations (optional, for batch correction)
+
+**Basic Usage**:
+```python
+import scvi
+
+# Prepare data (peaks should be in adata.X)
+# Optional: filter peaks
+sc.pp.filter_genes(adata, min_cells=3)
+
+# Setup data
+scvi.model.PEAKVI.setup_anndata(
+    adata,
+    batch_key="batch"
+)
+
+# Train model
+model = scvi.model.PEAKVI(adata)
+model.train()
+
+# Get latent representation (batch-corrected)
+latent = model.get_latent_representation()
+adata.obsm["X_PeakVI"] = latent
+
+# Differential accessibility
+da_results = model.differential_accessibility(
+    groupby="cell_type",
+    group1="TypeA",
+    group2="TypeB"
+)
+```
+
+**Key Parameters**:
+- `n_latent`: Dimensionality of latent space (default: 10)
+- `n_hidden`: Number of nodes per hidden layer (default: 128)
+- `n_layers`: Number of hidden layers (default: 1)
+- `region_factors`: Whether to learn region-specific factors (default: True)
+- `latent_distribution`: Distribution for latent space ("normal" or "ln")
+
+**Outputs**:
+- `get_latent_representation()`: Low-dimensional embeddings for cells
+- `get_accessibility_estimates()`: Normalized accessibility values
+- `differential_accessibility()`: Statistical testing for differential peaks
+- `get_region_factors()`: Peak-specific scaling factors
+
+**Best Practices**:
+1. Filter out low-quality peaks (present in very few cells)
+2. Include batch information if integrating multiple samples
+3. Use latent representations for clustering and UMAP visualization
+4. Consider using `region_factors=True` for datasets with high technical variation
+5. Store latent embeddings in `adata.obsm` for downstream analysis with scanpy
+
+## PoissonVI
+
+**Purpose**: Quantitative analysis of scATAC-seq fragment counts (more detailed than peak counts).
+
+**Key Features**:
+- Models fragment counts directly (not just peak presence/absence)
+- Poisson distribution for count data
+- Captures quantitative differences in accessibility
+- Enables fine-grained analysis of chromatin state
+
+**When to Use**:
+- Analyzing fragment-level ATAC-seq data
+- Need quantitative accessibility measurements
+- Higher resolution analysis than binary peak calls
+- Investigating gradual changes in chromatin accessibility
+
+**Data Requirements**:
+- Fragment count matrix (cells × genomic regions)
+- Count data (not binary)
+
+**Basic Usage**:
+```python
+scvi.model.POISSONVI.setup_anndata(
+    adata,
+    batch_key="batch"
+)
+
+model = scvi.model.POISSONVI(adata)
+model.train()
+
+# Get results
+latent = model.get_latent_representation()
+accessibility = model.get_accessibility_estimates()
+```
+
+**Key Differences from PeakVI**:
+- **PeakVI**: Best for standard peak count matrices, faster
+- **PoissonVI**: Best for quantitative fragment counts, more detailed
+
+**When to Choose PoissonVI over PeakVI**:
+- Working with fragment counts rather than called peaks
+- Need to capture quantitative differences
+- Have high-quality, high-coverage data
+- Interested in subtle accessibility changes
+
+## scBasset
+
+**Purpose**: Deep learning approach to scATAC-seq analysis with interpretability and motif analysis.
+
+**Key Features**:
+- Convolutional neural network (CNN) architecture for sequence-based analysis
+- Models raw DNA sequences, not just peak counts
+- Enables motif discovery and transcription factor (TF) binding prediction
+- Provides interpretable feature importance
+- Performs batch correction
+
+**When to Use**:
+- Want to incorporate DNA sequence information
+- Interested in TF motif analysis
+- Need interpretable models (which sequences drive accessibility)
+- Analyzing regulatory elements and TF binding sites
+- Predicting accessibility from sequence alone
+
+**Data Requirements**:
+- Peak sequences (extracted from genome)
+- Peak accessibility matrix
+- Genome reference (for sequence extraction)
+
+**Basic Usage**:
+```python
+# scBasset requires sequence information
+# First, extract sequences for peaks
+from scbasset import utils
+sequences = utils.fetch_sequences(adata, genome="hg38")
+
+# Setup and train
+scvi.model.SCBASSET.setup_anndata(
+    adata,
+    batch_key="batch"
+)
+
+model = scvi.model.SCBASSET(adata, sequences=sequences)
+model.train()
+
+# Get latent representation
+latent = model.get_latent_representation()
+
+# Interpret model: which sequences/motifs are important
+importance_scores = model.get_feature_importance()
+```
+
+**Key Parameters**:
+- `n_latent`: Latent space dimensionality
+- `conv_layers`: Number of convolutional layers
+- `n_filters`: Number of filters per conv layer
+- `filter_size`: Size of convolutional filters
+
+**Advanced Features**:
+- **In silico mutagenesis**: Predict how sequence changes affect accessibility
+- **Motif enrichment**: Identify enriched TF motifs in accessible regions
+- **Batch correction**: Similar to other scvi-tools models
+- **Transfer learning**: Fine-tune on new datasets
+
+**Interpretability Tools**:
+```python
+# Get importance scores for sequences
+importance = model.get_sequence_importance(region_indices=[0, 1, 2])
+
+# Predict accessibility for new sequences
+predictions = model.predict_accessibility(new_sequences)
+```
+
+## Model Selection for ATAC-seq
+
+### PeakVI
+**Choose when**:
+- Standard scATAC-seq analysis workflow
+- Have peak count matrices (most common format)
+- Need fast, efficient batch correction
+- Want straightforward differential accessibility
+- Prioritize computational efficiency
+
+**Advantages**:
+- Fast training and inference
+- Proven track record for scATAC-seq
+- Easy integration with scanpy workflow
+- Robust batch correction
+
+### PoissonVI
+**Choose when**:
+- Have fragment-level count data
+- Need quantitative accessibility measures
+- Interested in subtle differences
+- Have high-coverage, high-quality data
+
+**Advantages**:
+- More detailed quantitative information
+- Better for gradient changes
+- Appropriate statistical model for counts
+
+### scBasset
+**Choose when**:
+- Want to incorporate DNA sequence
+- Need biological interpretation (motifs, TFs)
+- Interested in regulatory mechanisms
+- Have computational resources for CNN training
+- Want predictive power for new sequences
+
+**Advantages**:
+- Sequence-based, biologically interpretable
+- Motif and TF analysis built-in
+- Predictive modeling capabilities
+- In silico perturbation experiments
+
+## Workflow Example: Complete ATAC-seq Analysis
+
+```python
+import scvi
+import scanpy as sc
+
+# 1. Load and preprocess ATAC-seq data
+adata = sc.read_h5ad("atac_data.h5ad")
+
+# 2. Filter low-quality peaks
+sc.pp.filter_genes(adata, min_cells=10)
+
+# 3. Setup and train PeakVI
+scvi.model.PEAKVI.setup_anndata(
+    adata,
+    batch_key="sample"
+)
+
+model = scvi.model.PEAKVI(adata, n_latent=20)
+model.train(max_epochs=400)
+
+# 4. Extract latent representation
+latent = model.get_latent_representation()
+adata.obsm["X_PeakVI"] = latent
+
+# 5. Downstream analysis
+sc.pp.neighbors(adata, use_rep="X_PeakVI")
+sc.tl.umap(adata)
+sc.tl.leiden(adata, key_added="clusters")
+
+# 6. Differential accessibility
+da_results = model.differential_accessibility(
+    groupby="clusters",
+    group1="0",
+    group2="1"
+)
+
+# 7. Save model
+model.save("peakvi_model")
+```
+
+## Integration with Gene Expression (RNA+ATAC)
+
+For paired multimodal data (RNA+ATAC from same cells), use **MultiVI** instead:
+
+```python
+# For 10x Multiome or similar paired data
+scvi.model.MULTIVI.setup_anndata(
+    adata,
+    batch_key="sample",
+    modality_key="modality"  # "RNA" or "ATAC"
+)
+
+model = scvi.model.MULTIVI(adata)
+model.train()
+
+# Get joint latent space
+latent = model.get_latent_representation()
+```
+
+See `models-multimodal.md` for more details on multimodal integration.
+
+## Best Practices for ATAC-seq Analysis
+
+1. **Quality Control**:
+   - Filter cells with very low or very high peak counts
+   - Remove peaks present in very few cells
+   - Filter mitochondrial and sex chromosome peaks if needed
+
+2. **Batch Correction**:
+   - Always include `batch_key` if integrating multiple samples
+   - Consider technical covariates (sequencing depth, TSS enrichment)
+
+3. **Feature Selection**:
+   - Unlike RNA-seq, all peaks are often used
+   - Consider filtering very rare peaks for efficiency
+
+4. **Latent Dimensions**:
+   - Start with `n_latent=10-30` depending on dataset complexity
+   - Larger values for more heterogeneous datasets
+
+5. **Downstream Analysis**:
+   - Use latent representations for clustering and visualization
+   - Link peaks to genes for regulatory analysis
+   - Perform motif enrichment on cluster-specific peaks
+
+6. **Computational Considerations**:
+   - ATAC-seq matrices are often very large (many peaks)
+   - Consider downsampling peaks for initial exploration
+   - Use GPU acceleration for large datasets

package/skills/scvi-tools/references/models-multimodal.md

@@ -0,0 +1,367 @@
+# Multimodal and Multi-omics Integration Models
+
+This document covers models for joint analysis of multiple data modalities in scvi-tools.
+
+## totalVI (Total Variational Inference)
+
+**Purpose**: Joint analysis of CITE-seq data (simultaneous RNA and protein measurements from same cells).
+
+**Key Features**:
+- Jointly models gene expression and protein abundance
+- Learns shared low-dimensional representations
+- Enables protein imputation from RNA data
+- Performs differential expression for both modalities
+- Handles batch effects in both RNA and protein layers
+
+**When to Use**:
+- Analyzing CITE-seq or REAP-seq data
+- Joint RNA + surface protein measurements
+- Imputing missing proteins
+- Integrating protein and RNA information
+- Multi-batch CITE-seq integration
+
+**Data Requirements**:
+- AnnData with gene expression in `.X` or a layer
+- Protein measurements in `.obsm["protein_expression"]`
+- Same cells measured for both modalities
+
+**Basic Usage**:
+```python
+import scvi
+
+# Setup data - specify both RNA and protein layers
+scvi.model.TOTALVI.setup_anndata(
+    adata,
+    layer="counts",  # RNA counts
+    protein_expression_obsm_key="protein_expression",  # Protein counts
+    batch_key="batch"
+)
+
+# Train model
+model = scvi.model.TOTALVI(adata)
+model.train()
+
+# Get joint latent representation
+latent = model.get_latent_representation()
+
+# Get normalized values for both modalities
+rna_normalized = model.get_normalized_expression()
+protein_normalized = model.get_normalized_expression(
+    transform_batch="batch1",
+    protein_expression=True
+)
+
+# Differential expression (works for both RNA and protein)
+rna_de = model.differential_expression(groupby="cell_type")
+protein_de = model.differential_expression(
+    groupby="cell_type",
+    protein_expression=True
+)
+```
+
+**Key Parameters**:
+- `n_latent`: Latent space dimensionality (default: 20)
+- `n_layers_encoder`: Number of encoder layers (default: 1)
+- `n_layers_decoder`: Number of decoder layers (default: 1)
+- `protein_dispersion`: Protein dispersion handling ("protein" or "protein-batch")
+- `empirical_protein_background_prior`: Use empirical background for proteins
+
+**Advanced Features**:
+
+**Protein Imputation**:
+```python
+# Impute missing proteins for RNA-only cells
+# (useful for mapping RNA-seq to CITE-seq reference)
+protein_foreground = model.get_protein_foreground_probability()
+imputed_proteins = model.get_normalized_expression(
+    protein_expression=True,
+    n_samples=25
+)
+```
+
+**Denoising**:
+```python
+# Get denoised counts for both modalities
+denoised_rna = model.get_normalized_expression(n_samples=25)
+denoised_protein = model.get_normalized_expression(
+    protein_expression=True,
+    n_samples=25
+)
+```
+
+**Best Practices**:
+1. Use empirical protein background prior for datasets with ambient protein
+2. Consider protein-specific dispersion for heterogeneous protein data
+3. Use joint latent space for clustering (better than RNA alone)
+4. Validate protein imputation with known markers
+5. Check protein QC metrics before training
+
+## MultiVI (Multi-modal Variational Inference)
+
+**Purpose**: Integration of paired and unpaired multi-omic data (e.g., RNA + ATAC, paired and unpaired cells).
+
+**Key Features**:
+- Handles paired data (same cells) and unpaired data (different cells)
+- Integrates multiple modalities: RNA, ATAC, proteins, etc.
+- Missing modality imputation
+- Learns shared representations across modalities
+- Flexible integration strategy
+
+**When to Use**:
+- 10x Multiome data (paired RNA + ATAC)
+- Integrating separate RNA-seq and ATAC-seq experiments
+- Some cells with both modalities, some with only one
+- Cross-modality imputation tasks
+
+**Data Requirements**:
+- AnnData with multiple modalities
+- Modality indicators (which measurements each cell has)
+- Can handle:
+  - All cells with both modalities (fully paired)
+  - Mix of paired and unpaired cells
+  - Completely unpaired datasets
+
+**Basic Usage**:
+```python
+# Prepare data with modality information
+# adata.X should contain all features (genes + peaks)
+# adata.var["modality"] indicates "Gene" or "Peak"
+# adata.obs["modality"] indicates which modality each cell has
+
+scvi.model.MULTIVI.setup_anndata(
+    adata,
+    batch_key="batch",
+    modality_key="modality"  # Column indicating cell modality
+)
+
+model = scvi.model.MULTIVI(adata)
+model.train()
+
+# Get joint latent representation
+latent = model.get_latent_representation()
+
+# Impute missing modalities
+# E.g., predict ATAC for RNA-only cells
+imputed_accessibility = model.get_accessibility_estimates(
+    indices=rna_only_indices
+)
+
+# Get normalized expression/accessibility
+rna_normalized = model.get_normalized_expression()
+atac_normalized = model.get_accessibility_estimates()
+```
+
+**Key Parameters**:
+- `n_genes`: Number of gene features
+- `n_regions`: Number of accessibility regions
+- `n_latent`: Latent dimensionality (default: 20)
+
+**Integration Scenarios**:
+
+**Scenario 1: Fully Paired (10x Multiome)**:
+```python
+# All cells have both RNA and ATAC
+# Single modality key: "paired"
+adata.obs["modality"] = "paired"
+```
+
+**Scenario 2: Partially Paired**:
+```python
+# Some cells have both, some RNA-only, some ATAC-only
+adata.obs["modality"] = ["RNA+ATAC", "RNA", "ATAC", ...]
+```
+
+**Scenario 3: Completely Unpaired**:
+```python
+# Separate RNA and ATAC experiments
+adata.obs["modality"] = ["RNA"] * n_rna + ["ATAC"] * n_atac
+```
+
+**Advanced Use Cases**:
+
+**Cross-Modality Prediction**:
+```python
+# Predict peaks from gene expression
+accessibility_from_rna = model.get_accessibility_estimates(
+    indices=rna_only_cells
+)
+
+# Predict genes from accessibility
+expression_from_atac = model.get_normalized_expression(
+    indices=atac_only_cells
+)
+```
+
+**Modality-Specific Analysis**:
+```python
+# Separate analysis per modality
+rna_subset = adata[adata.obs["modality"].str.contains("RNA")]
+atac_subset = adata[adata.obs["modality"].str.contains("ATAC")]
+```
+
+## MrVI (Multi-resolution Variational Inference)
+
+**Purpose**: Multi-sample analysis accounting for sample-specific and shared variation.
+
+**Key Features**:
+- Simultaneously analyzes multiple samples/conditions
+- Decomposes variation into:
+  - Shared variation (common across samples)
+  - Sample-specific variation
+- Enables sample-level comparisons
+- Identifies sample-specific cell states
+
+**When to Use**:
+- Comparing multiple biological samples or conditions
+- Identifying sample-specific vs. shared cell states
+- Disease vs. healthy sample comparisons
+- Understanding inter-sample heterogeneity
+- Multi-donor studies
+
+**Basic Usage**:
+```python
+scvi.model.MRVI.setup_anndata(
+    adata,
+    layer="counts",
+    batch_key="batch",
+    sample_key="sample"  # Critical: defines biological samples
+)
+
+model = scvi.model.MRVI(adata, n_latent=10, n_latent_sample=5)
+model.train()
+
+# Get representations
+shared_latent = model.get_latent_representation()  # Shared across samples
+sample_specific = model.get_sample_specific_representation()
+
+# Sample distance matrix
+sample_distances = model.get_sample_distances()
+```
+
+**Key Parameters**:
+- `n_latent`: Dimensionality of shared latent space
+- `n_latent_sample`: Dimensionality of sample-specific space
+- `sample_key`: Column defining biological samples
+
+**Analysis Workflow**:
+```python
+# 1. Identify shared cell types across samples
+sc.pp.neighbors(adata, use_rep="X_MrVI_shared")
+sc.tl.umap(adata)
+sc.tl.leiden(adata, key_added="shared_clusters")
+
+# 2. Analyze sample-specific variation
+sample_repr = model.get_sample_specific_representation()
+
+# 3. Compare samples
+distances = model.get_sample_distances()
+
+# 4. Find sample-enriched genes
+de_results = model.differential_expression(
+    groupby="sample",
+    group1="Disease",
+    group2="Healthy"
+)
+```
+
+**Use Cases**:
+- **Multi-donor studies**: Separate donor effects from cell type variation
+- **Disease studies**: Identify disease-specific vs. shared biology
+- **Time series**: Separate temporal from stable variation
+- **Batch + biology**: Disentangle technical and biological variation
+
+## totalVI vs. MultiVI vs. MrVI: When to Use Which?
+
+### totalVI
+**Use for**: CITE-seq (RNA + protein, same cells)
+- Paired measurements
+- Single modality type per feature
+- Focus: protein imputation, joint analysis
+
+### MultiVI
+**Use for**: Multiple modalities (RNA + ATAC, etc.)
+- Paired, unpaired, or mixed
+- Different feature types
+- Focus: cross-modality integration and imputation
+
+### MrVI
+**Use for**: Multi-sample RNA-seq
+- Single modality (RNA)
+- Multiple biological samples
+- Focus: sample-level variation decomposition
+
+## Integration Best Practices
+
+### For CITE-seq (totalVI)
+1. **Quality control proteins**: Remove low-quality antibodies
+2. **Background subtraction**: Use empirical background prior
+3. **Joint clustering**: Use joint latent space, not RNA alone
+4. **Validation**: Check known markers in both modalities
+
+### For Multiome/Multi-modal (MultiVI)
+1. **Feature filtering**: Filter genes and peaks independently
+2. **Balance modalities**: Ensure reasonable representation of each
+3. **Modality weights**: Consider if one modality dominates
+4. **Imputation validation**: Validate imputed values carefully
+
+### For Multi-sample (MrVI)
+1. **Sample definition**: Carefully define biological samples
+2. **Sample size**: Need sufficient cells per sample
+3. **Covariate handling**: Properly account for batch vs. sample
+4. **Interpretation**: Distinguish technical from biological variation
+
+## Complete Example: CITE-seq Analysis with totalVI
+
+```python
+import scvi
+import scanpy as sc
+
+# 1. Load CITE-seq data
+adata = sc.read_h5ad("cite_seq.h5ad")
+
+# 2. QC and filtering
+sc.pp.filter_genes(adata, min_cells=3)
+sc.pp.highly_variable_genes(adata, n_top_genes=4000)
+
+# Protein QC
+protein_counts = adata.obsm["protein_expression"]
+# Remove low-quality proteins
+
+# 3. Setup totalVI
+scvi.model.TOTALVI.setup_anndata(
+    adata,
+    layer="counts",
+    protein_expression_obsm_key="protein_expression",
+    batch_key="batch"
+)
+
+# 4. Train
+model = scvi.model.TOTALVI(adata, n_latent=20)
+model.train(max_epochs=400)
+
+# 5. Extract joint representation
+latent = model.get_latent_representation()
+adata.obsm["X_totalVI"] = latent
+
+# 6. Clustering on joint space
+sc.pp.neighbors(adata, use_rep="X_totalVI")
+sc.tl.umap(adata)
+sc.tl.leiden(adata, resolution=0.5)
+
+# 7. Differential expression for both modalities
+rna_de = model.differential_expression(
+    groupby="leiden",
+    group1="0",
+    group2="1"
+)
+
+protein_de = model.differential_expression(
+    groupby="leiden",
+    group1="0",
+    group2="1",
+    protein_expression=True
+)
+
+# 8. Save model
+model.save("totalvi_model")
+```