@pikaa-ai/pikaa 0.3.23 → 0.3.24

package/skills/scvelo/SKILL.md

@@ -1,328 +0,0 @@
----
-name: scvelo
-description: RNA velocity analysis with scVelo. Estimate cell state transitions from unspliced/spliced mRNA dynamics, infer trajectory directions, compute latent time, and identify driver genes in single-cell RNA-seq data. Complements Scanpy/scVI-tools for trajectory inference.
-license: BSD-3-Clause
-compatibility: Requires Python 3.10+ with scvelo, scanpy, and anndata. Verified against scvelo 0.3.4, whose dynamical model and pl.scatter need pandas<3 and whose stochastic estimator needs numpy<2; the deterministic estimator works on current releases.
-metadata:
-  version: "1.2"
-  skill-author: Kuan-lin Huang
----
-
-# scVelo — RNA Velocity Analysis
-
-## Overview
-
-scVelo is the leading Python package for RNA velocity analysis in single-cell RNA-seq data. It infers cell state transitions by modeling the kinetics of mRNA splicing — using the ratio of unspliced (pre-mRNA) to spliced (mature mRNA) abundances to determine whether a gene is being upregulated or downregulated in each cell. This allows reconstruction of developmental trajectories and identification of cell fate decisions without requiring time-course data.
-
-**Installation:** `uv pip install scvelo`
-
-**Key resources:**
-- Documentation: https://scvelo.readthedocs.io/
-- GitHub: https://github.com/theislab/scvelo
-- Paper: Bergen et al. (2020) Nature Biotechnology. PMID: 32747759
-
-## When to Use This Skill
-
-Use scVelo when:
-
-- **Trajectory inference from snapshot data**: Determine which direction cells are differentiating
-- **Cell fate prediction**: Identify progenitor cells and their downstream fates
-- **Driver gene identification**: Find genes whose dynamics best explain observed trajectories
-- **Developmental biology**: Model hematopoiesis, neurogenesis, epithelial-to-mesenchymal transitions
-- **Latent time estimation**: Order cells along a pseudotime derived from splicing dynamics
-- **Complement to Scanpy**: Add directional information to UMAP embeddings
-
-## Prerequisites
-
-scVelo requires count matrices for both **unspliced** and **spliced** RNA. These are generated by:
-1. **STARsolo** or **kallisto|bustools** with `lamanno` mode
-2. **velocyto** CLI: `velocyto run10x` / `velocyto run`
-3. **alevin-fry** / **simpleaf** with spliced/unspliced output
-
-Data is stored in an `AnnData` object with `layers["spliced"]` and `layers["unspliced"]`.
-
-## Standard RNA Velocity Workflow
-
-### 1. Setup and Data Loading
-
-```python
-import scvelo as scv
-import scanpy as sc
-import numpy as np
-import matplotlib.pyplot as plt
-
-# Configure settings
-scv.settings.verbosity = 3       # Show computation steps
-scv.settings.presenter_view = True
-scv.settings.set_figure_params('scvelo')
-
-# Load data (AnnData with spliced/unspliced layers)
-# Option A: Load from loom (velocyto output)
-adata = scv.read("cellranger_output.loom", cache=True)
-
-# Option B: Merge velocyto loom with Scanpy-processed AnnData
-adata_processed = sc.read_h5ad("processed.h5ad")  # Has UMAP, clusters
-adata_velocity = scv.read("velocyto.loom")
-adata = scv.utils.merge(adata_processed, adata_velocity)
-
-# Verify layers
-print(adata)
-# obs × var: N × G
-# layers: 'spliced', 'unspliced' (required)
-# obsm['X_umap'] (required for visualization)
-```
-
-### 2. Preprocessing
-
-```python
-# Filter and normalize. As of scVelo 0.3, filter_and_normalize() only filters
-# genes and normalizes per cell -- it no longer takes n_top_genes and no longer
-# log-transforms, so the log step and HVG selection come from Scanpy.
-scv.pp.filter_and_normalize(
-    adata,
-    min_shared_counts=20    # Minimum counts in spliced+unspliced
-)
-sc.pp.log1p(adata)
-sc.pp.highly_variable_genes(adata, n_top_genes=2000, subset=True)
-
-# Compute first and second order moments (means and variances)
-# knn_connectivities must be computed first
-sc.pp.neighbors(adata, n_neighbors=30, n_pcs=30)
-scv.pp.moments(
-    adata,
-    n_pcs=30,
-    n_neighbors=30
-)
-```
-
-### 3. Velocity Estimation — Stochastic Model
-
-The stochastic model is fast and suitable for exploratory analysis:
-
-```python
-# Stochastic velocity (faster, less accurate)
-scv.tl.velocity(adata, mode='stochastic')
-scv.tl.velocity_graph(adata)
-
-# Visualize
-scv.pl.velocity_embedding_stream(
-    adata,
-    basis='umap',
-    color='leiden',
-    title="RNA Velocity (Stochastic)"
-)
-```
-
-### 4. Velocity Estimation — Dynamical Model (Recommended)
-
-The dynamical model fits the full splicing kinetics and is more accurate:
-
-```python
-# Recover dynamics (computationally intensive; ~10-30 min for 10K cells)
-scv.tl.recover_dynamics(adata, n_jobs=4)
-
-# Compute velocity from dynamical model
-scv.tl.velocity(adata, mode='dynamical')
-scv.tl.velocity_graph(adata)
-```
-
-### 5. Latent Time
-
-The dynamical model enables computation of a shared latent time (pseudotime):
-
-```python
-# Compute latent time
-scv.tl.latent_time(adata)
-
-# Visualize latent time on UMAP
-scv.pl.scatter(
-    adata,
-    color='latent_time',
-    color_map='gnuplot',
-    size=80,
-    title='Latent time'
-)
-
-# Identify top genes ordered by latent time
-top_genes = adata.var['fit_likelihood'].sort_values(ascending=False).index[:300]
-scv.pl.heatmap(
-    adata,
-    var_names=top_genes,
-    sortby='latent_time',
-    col_color='leiden',
-    n_convolve=100
-)
-```
-
-### 6. Driver Gene Analysis
-
-```python
-# Identify genes with highest velocity fit
-scv.tl.rank_velocity_genes(adata, groupby='leiden', min_corr=0.3)
-df = scv.DataFrame(adata.uns['rank_velocity_genes']['names'])
-print(df.head(10))
-
-# Speed and coherence
-scv.tl.velocity_confidence(adata)
-scv.pl.scatter(
-    adata,
-    c=['velocity_length', 'velocity_confidence'],
-    cmap='coolwarm',
-    perc=[5, 95]
-)
-
-# Phase portraits for specific genes
-scv.pl.velocity(adata, ['Cpe', 'Gnao1', 'Ins2'],
-               ncols=3, figsize=(16, 4))
-```
-
-### 7. Velocity Arrows and Pseudotime
-
-```python
-# Arrow plot on UMAP
-scv.pl.velocity_embedding(
-    adata,
-    arrow_length=3,
-    arrow_size=2,
-    color='leiden',
-    basis='umap'
-)
-
-# Stream plot (cleaner visualization)
-scv.pl.velocity_embedding_stream(
-    adata,
-    basis='umap',
-    color='leiden',
-    smooth=0.8,
-    min_mass=4
-)
-
-# Velocity pseudotime (alternative to latent time)
-scv.tl.velocity_pseudotime(adata)
-scv.pl.scatter(adata, color='velocity_pseudotime', cmap='gnuplot')
-```
-
-### 8. PAGA Trajectory Graph
-
-```python
-# PAGA graph with velocity-informed transitions
-scv.tl.paga(adata, groups='leiden')
-df = scv.get_df(adata, 'paga/transitions_confidence', precision=2).T
-df.style.background_gradient(cmap='Blues').format('{:.2g}')
-
-# Plot PAGA with velocity
-scv.pl.paga(
-    adata,
-    basis='umap',
-    size=50,
-    alpha=0.1,
-    min_edge_width=2,
-    node_size_scale=1.5
-)
-```
-
-## Complete Workflow Script
-
-```python
-import scvelo as scv
-import scanpy as sc
-
-def run_rna_velocity(adata, n_top_genes=2000, mode='dynamical', n_jobs=4):
-    """
-    Complete RNA velocity workflow.
-
-    Args:
-        adata: AnnData with 'spliced' and 'unspliced' layers, UMAP in obsm
-        n_top_genes: Number of top HVGs for velocity
-        mode: 'stochastic' (fast) or 'dynamical' (accurate)
-        n_jobs: Parallel jobs for dynamical model
-
-    Returns:
-        Processed AnnData with velocity information
-    """
-    scv.settings.verbosity = 2
-
-    # 1. Preprocessing (scVelo 0.3 dropped log/HVG from filter_and_normalize)
-    scv.pp.filter_and_normalize(adata, min_shared_counts=20)
-    sc.pp.log1p(adata)
-    sc.pp.highly_variable_genes(adata, n_top_genes=n_top_genes, subset=True)
-
-    if 'neighbors' not in adata.uns:
-        sc.pp.neighbors(adata, n_neighbors=30)
-
-    scv.pp.moments(adata, n_pcs=30, n_neighbors=30)
-
-    # 2. Velocity estimation
-    if mode == 'dynamical':
-        scv.tl.recover_dynamics(adata, n_jobs=n_jobs)
-
-    scv.tl.velocity(adata, mode=mode)
-    scv.tl.velocity_graph(adata)
-
-    # 3. Downstream analyses
-    if mode == 'dynamical':
-        scv.tl.latent_time(adata)
-        scv.tl.rank_velocity_genes(adata, groupby='leiden', min_corr=0.3)
-
-    scv.tl.velocity_confidence(adata)
-    scv.tl.velocity_pseudotime(adata)
-
-    return adata
-```
-
-## Key Output Fields in AnnData
-
-After running the workflow, the following fields are added:
-
-| Location | Key | Description |
-|----------|-----|-------------|
-| `adata.layers` | `velocity` | RNA velocity per gene per cell |
-| `adata.layers` | `fit_t` | Fitted latent time per gene per cell |
-| `adata.obsm` | `velocity_umap` | 2D velocity vectors on UMAP |
-| `adata.obs` | `velocity_pseudotime` | Pseudotime from velocity |
-| `adata.obs` | `latent_time` | Latent time from dynamical model |
-| `adata.obs` | `velocity_length` | Speed of each cell |
-| `adata.obs` | `velocity_confidence` | Confidence score per cell |
-| `adata.var` | `fit_likelihood` | Gene-level model fit quality |
-| `adata.var` | `fit_alpha` | Transcription rate |
-| `adata.var` | `fit_beta` | Splicing rate |
-| `adata.var` | `fit_gamma` | Degradation rate |
-| `adata.uns` | `velocity_graph` | Cell-cell transition probability matrix |
-
-## Velocity Models Comparison
-
-| Model | Speed | Accuracy | When to Use |
-|-------|-------|----------|-------------|
-| `stochastic` | Fast | Moderate | Exploratory; large datasets |
-| `deterministic` | Medium | Moderate | Simple linear kinetics |
-| `dynamical` | Slow | High | Publication-quality; identifies driver genes |
-
-## Best Practices
-
-- **Start with stochastic mode** for exploration; switch to dynamical for final analysis
-- **Need good coverage of unspliced reads**: Short reads (< 100 bp) may miss intron coverage
-- **Minimum 2,000 cells**: RNA velocity is noisy with fewer cells
-- **Velocity should be coherent**: Arrows should follow known biology; randomness indicates issues
-- **k-NN bandwidth matters**: Too few neighbors → noisy velocity; too many → oversmoothed
-- **Sanity check**: Root cells (progenitors) should have high unspliced/spliced ratios for marker genes
-- **Dynamical model requires distinct kinetic states**: Works best for clear differentiation processes
-
-## Troubleshooting
-
-| Problem | Solution |
-|---------|---------|
-| Missing unspliced layer | Re-run velocyto or use STARsolo with `--soloFeatures Gene Velocyto` |
-| Very few velocity genes | Lower `min_shared_counts`; check sequencing depth |
-| Random-looking arrows | Try different `n_neighbors` or velocity model |
-| Memory error with dynamical | Set `n_jobs=1`; reduce `n_top_genes` |
-| Negative velocity everywhere | Check that spliced/unspliced layers are not swapped |
-
-## Additional Resources
-
-- **scVelo documentation**: https://scvelo.readthedocs.io/
-- **Tutorial notebooks**: https://scvelo.readthedocs.io/tutorials/
-- **GitHub**: https://github.com/theislab/scvelo
-- **Paper**: Bergen V et al. (2020) Nature Biotechnology. PMID: 32747759
-- **velocyto** (preprocessing): http://velocyto.org/
-- **CellRank** (fate prediction, extends scVelo): https://cellrank.readthedocs.io/
-- **dynamo** (metabolic labeling alternative): https://dynamo-release.readthedocs.io/

package/skills/scvi-tools/SKILL.md

@@ -1,201 +0,0 @@
----
-name: scvi-tools
-description: Deep generative models for single-cell omics. Use when you need probabilistic batch correction (scVI), transfer learning, differential expression with uncertainty, or multi-modal integration (TOTALVI, MultiVI). Best for advanced modeling, batch effects, multimodal data. For standard analysis pipelines use scanpy.
-license: BSD-3-Clause license
-metadata:
-  version: "1.1"
-  skill-author: K-Dense Inc.
----
-
-# scvi-tools
-
-## Overview
-
-scvi-tools is a comprehensive Python framework for probabilistic models in single-cell genomics. Built on PyTorch and PyTorch Lightning, it provides deep generative models using variational inference for analyzing diverse single-cell data modalities. Current stable release: **scvi-tools 1.4.3** (May 2026).
-
-**Model namespaces matter:** core models (scVI, scANVI, totalVI, MultiVI, PeakVI, AUTOZI, CondSCVI, DestVI, LinearSCVI, AmortizedLDA, JaxSCVI) live under `scvi.model`. Most other models (VeloVI, contrastiveVI, CellAssign, PoissonVI, scBasset, MrVI, MethylVI/MethylANVI, CytoVI, SysVI, Decipher, gimVI, scVIVA, ResolVI, Stereoscope, Solo, totalANVI, DIAGVI) live under `scvi.external`. The reference files specify the correct namespace per model.
-
-## When to Use This Skill
-
-Use this skill when:
-- Analyzing single-cell RNA-seq data (dimensionality reduction, batch correction, integration)
-- Working with single-cell ATAC-seq or chromatin accessibility data
-- Integrating multimodal data (CITE-seq, multiome, paired/unpaired datasets)
-- Analyzing spatial transcriptomics data (deconvolution, spatial mapping)
-- Performing differential expression analysis on single-cell data
-- Conducting cell type annotation or transfer learning tasks
-- Working with specialized single-cell modalities (methylation, cytometry, RNA velocity)
-- Building custom probabilistic models for single-cell analysis
-
-## Core Capabilities
-
-scvi-tools provides models organized by data modality:
-
-### 1. Single-Cell RNA-seq Analysis
-Core models for expression analysis, batch correction, and integration. See `references/models-scrna-seq.md` for:
-- **scVI**: Unsupervised dimensionality reduction and batch correction
-- **scANVI**: Semi-supervised cell type annotation and integration
-- **AUTOZI**: Zero-inflation detection and modeling
-- **VeloVI**: RNA velocity analysis
-- **contrastiveVI**: Perturbation effect isolation
-
-### 2. Chromatin Accessibility (ATAC-seq)
-Models for analyzing single-cell chromatin data. See `references/models-atac-seq.md` for:
-- **PeakVI**: Peak-based ATAC-seq analysis and integration
-- **PoissonVI**: Quantitative fragment count modeling
-- **scBasset**: Deep learning approach with motif analysis
-
-### 3. Multimodal & Multi-omics Integration
-Joint analysis of multiple data types. See `references/models-multimodal.md` for:
-- **totalVI**: CITE-seq protein and RNA joint modeling
-- **totalANVI**: Semi-supervised CITE-seq (totalVI with cell-type labels)
-- **MultiVI**: Paired and unpaired multi-omic integration (MuData-based)
-- **MrVI**: Multi-resolution cross-sample analysis
-- **DIAGVI**: Diagonal integration of unpaired single-cell datasets (added in 1.4.3)
-
-### 4. Spatial Transcriptomics
-Spatially-resolved transcriptomics analysis. See `references/models-spatial.md` for:
-- **DestVI**: Multi-resolution spatial deconvolution
-- **Stereoscope**: Cell type deconvolution
-- **Tangram**: Spatial mapping and integration
-- **scVIVA**: Cell-environment relationship analysis
-
-### 5. Specialized Modalities
-Additional specialized analysis tools. See `references/models-specialized.md` for:
-- **MethylVI/MethylANVI**: Single-cell methylation analysis
-- **CytoVI**: Flow/mass cytometry batch correction
-- **Solo**: Doublet detection
-- **CellAssign**: Marker-based cell type annotation
-
-## Typical Workflow
-
-All scvi-tools models follow a consistent API pattern:
-
-```python
-# 1. Load and preprocess data (AnnData format)
-import scvi
-import scanpy as sc
-
-adata = scvi.data.heart_cell_atlas_subsampled()
-sc.pp.filter_genes(adata, min_counts=3)
-sc.pp.highly_variable_genes(adata, n_top_genes=1200)
-
-# 2. Register data with model (specify layers, covariates)
-scvi.model.SCVI.setup_anndata(
-    adata,
-    layer="counts",  # Use raw counts, not log-normalized
-    batch_key="batch",
-    categorical_covariate_keys=["donor"],
-    continuous_covariate_keys=["percent_mito"]
-)
-
-# 3. Create and train model
-model = scvi.model.SCVI(adata)
-model.train()
-
-# 4. Extract latent representations and normalized values
-latent = model.get_latent_representation()
-normalized = model.get_normalized_expression(library_size=1e4)
-
-# 5. Store in AnnData for downstream analysis
-adata.obsm["X_scVI"] = latent
-adata.layers["scvi_normalized"] = normalized
-
-# 6. Downstream analysis with scanpy
-sc.pp.neighbors(adata, use_rep="X_scVI")
-sc.tl.umap(adata)
-sc.tl.leiden(adata)
-```
-
-**Key Design Principles:**
-- **Raw counts required**: Models expect unnormalized count data for optimal performance
-- **Unified API**: Consistent interface across all models (setup → train → extract)
-- **AnnData-centric**: Seamless integration with the scanpy ecosystem
-- **GPU acceleration**: Automatic utilization of available GPUs
-- **Batch correction**: Handle technical variation through covariate registration
-
-## Common Analysis Tasks
-
-### Differential Expression
-Probabilistic DE analysis using the learned generative models:
-
-```python
-de_results = model.differential_expression(
-    groupby="cell_type",
-    group1="TypeA",
-    group2="TypeB",
-    mode="change",  # Use composite hypothesis testing
-    delta=0.25      # Minimum effect size threshold
-)
-```
-
-See `references/differential-expression.md` for detailed methodology and interpretation.
-
-### Model Persistence
-Save and load trained models:
-
-```python
-# Save model
-model.save("./model_directory", overwrite=True)
-
-# Load model
-model = scvi.model.SCVI.load("./model_directory", adata=adata)
-```
-
-### Batch Correction and Integration
-Integrate datasets across batches or studies:
-
-```python
-# Register batch information
-scvi.model.SCVI.setup_anndata(adata, batch_key="study")
-
-# Model automatically learns batch-corrected representations
-model = scvi.model.SCVI(adata)
-model.train()
-latent = model.get_latent_representation()  # Batch-corrected
-```
-
-## Theoretical Foundations
-
-scvi-tools is built on:
-- **Variational inference**: Approximate posterior distributions for scalable Bayesian inference
-- **Deep generative models**: VAE architectures that learn complex data distributions
-- **Amortized inference**: Shared neural networks for efficient learning across cells
-- **Probabilistic modeling**: Principled uncertainty quantification and statistical testing
-
-See `references/theoretical-foundations.md` for detailed background on the mathematical framework.
-
-## Additional Resources
-
-- **Workflows**: `references/workflows.md` contains common workflows, best practices, hyperparameter tuning, and GPU optimization
-- **Model References**: Detailed documentation for each model category in the `references/` directory
-- **Official Documentation**: https://docs.scvi-tools.org/en/stable/
-- **Tutorials**: https://docs.scvi-tools.org/en/stable/tutorials/index.html
-- **API Reference**: https://docs.scvi-tools.org/en/stable/api/index.html
-
-## Installation
-
-Requires Python **3.12+** (scvi-tools 1.4 dropped older versions).
-
-```bash
-uv pip install scvi-tools
-# For GPU support
-uv pip install "scvi-tools[cuda]"
-```
-
-For reproducible environments, pin a version: `uv pip install scvi-tools==1.4.3`.
-
-**Compute backends:** training defaults to PyTorch (CPU/GPU/TPU). A JAX backend
-(`scvi.model.JaxSCVI`) and an experimental MLX backend for Apple silicon
-(`scvi.model.mlxSCVI`) are available for select models.
-
-## Best Practices
-
-1. **Use raw counts**: Always provide unnormalized count data to models
-2. **Filter genes**: Remove low-count genes before analysis (e.g., `min_counts=3`)
-3. **Register covariates**: Include known technical factors (batch, donor, etc.) in `setup_anndata`
-4. **Feature selection**: Use highly variable genes for improved performance
-5. **Model saving**: Always save trained models to avoid retraining
-6. **GPU usage**: Enable GPU acceleration for large datasets (`accelerator="gpu"`)
-7. **Scanpy integration**: Store outputs in AnnData objects for downstream analysis
-