@bgicli/bgicli 2.1.1 → 2.2.0

package/data/skills/single-cell-rna-qc/SKILL.md

@@ -0,0 +1,175 @@
+---
+name: single-cell-rna-qc
+description: Performs quality control on single-cell RNA-seq data (.h5ad or .h5 files) using scverse best practices with MAD-based filtering and comprehensive visualizations. Use when users request QC analysis, filtering low-quality cells, assessing data quality, or following scverse/scanpy best practices for single-cell analysis.
+---
+
+# Single-Cell RNA-seq Quality Control
+
+Automated QC workflow for single-cell RNA-seq data following scverse best practices.
+
+## When to Use This Skill
+
+Use when users:
+- Request quality control or QC on single-cell RNA-seq data
+- Want to filter low-quality cells or assess data quality
+- Need QC visualizations or metrics
+- Ask to follow scverse/scanpy best practices
+- Request MAD-based filtering or outlier detection
+
+**Supported input formats:**
+- `.h5ad` files (AnnData format from scanpy/Python workflows)
+- `.h5` files (10X Genomics Cell Ranger output)
+
+**Default recommendation**: Use Approach 1 (complete pipeline) unless the user has specific custom requirements or explicitly requests non-standard filtering logic.
+
+## Approach 1: Complete QC Pipeline (Recommended for Standard Workflows)
+
+For standard QC following scverse best practices, use the convenience script `scripts/qc_analysis.py`:
+
+```bash
+python3 scripts/qc_analysis.py input.h5ad
+# or for 10X Genomics .h5 files:
+python3 scripts/qc_analysis.py raw_feature_bc_matrix.h5
+```
+
+The script automatically detects the file format and loads it appropriately.
+
+**When to use this approach:**
+- Standard QC workflow with adjustable thresholds (all cells filtered the same way)
+- Batch processing multiple datasets
+- Quick exploratory analysis
+- User wants the "just works" solution
+
+**Requirements:** anndata, scanpy, scipy, matplotlib, seaborn, numpy
+
+**Parameters:**
+
+Customize filtering thresholds and gene patterns using command-line parameters:
+- `--output-dir` - Output directory
+- `--mad-counts`, `--mad-genes`, `--mad-mt` - MAD thresholds for counts/genes/MT%
+- `--mt-threshold` - Hard mitochondrial % cutoff
+- `--min-cells` - Gene filtering threshold
+- `--mt-pattern`, `--ribo-pattern`, `--hb-pattern` - Gene name patterns for different species
+
+Use `--help` to see current default values.
+
+**Outputs:**
+
+All files are saved to `<input_basename>_qc_results/` directory by default (or to the directory specified by `--output-dir`):
+- `qc_metrics_before_filtering.png` - Pre-filtering visualizations
+- `qc_filtering_thresholds.png` - MAD-based threshold overlays
+- `qc_metrics_after_filtering.png` - Post-filtering quality metrics
+- `<input_basename>_filtered.h5ad` - Clean, filtered dataset ready for downstream analysis
+- `<input_basename>_with_qc.h5ad` - Original data with QC annotations preserved
+
+If copying outputs to `/mnt/user-data/outputs/` for user access, copy individual files (not the entire directory) so users can preview them directly as Claude.ai artifacts.
+
+### Workflow Steps
+
+The script performs the following steps:
+
+1. **Calculate QC metrics** - Count depth, gene detection, mitochondrial/ribosomal/hemoglobin content
+2. **Apply MAD-based filtering** - Permissive outlier detection using MAD thresholds for counts/genes/MT%
+3. **Filter genes** - Remove genes detected in few cells
+4. **Generate visualizations** - Comprehensive before/after plots with threshold overlays
+
+## Approach 2: Modular Building Blocks (For Custom Workflows)
+
+For custom analysis workflows or non-standard requirements, use the modular utility functions from `scripts/qc_core.py` and `scripts/qc_plotting.py`:
+
+```python
+# Run from scripts/ directory, or add scripts/ to sys.path if needed
+import anndata as ad
+from qc_core import calculate_qc_metrics, detect_outliers_mad, filter_cells
+from qc_plotting import plot_qc_distributions  # Only if visualization needed
+
+adata = ad.read_h5ad('input.h5ad')
+calculate_qc_metrics(adata, inplace=True)
+# ... custom analysis logic here
+```
+
+**When to use this approach:**
+- Different workflow needed (skip steps, change order, apply different thresholds to subsets)
+- Conditional logic (e.g., filter neurons differently than other cells)
+- Partial execution (only metrics/visualization, no filtering)
+- Integration with other analysis steps in a larger pipeline
+- Custom filtering criteria beyond what command-line params support
+
+**Available utility functions:**
+
+From `qc_core.py` (core QC operations):
+- `calculate_qc_metrics(adata, mt_pattern, ribo_pattern, hb_pattern, inplace=True)` - Calculate QC metrics and annotate adata
+- `detect_outliers_mad(adata, metric, n_mads, verbose=True)` - MAD-based outlier detection, returns boolean mask
+- `apply_hard_threshold(adata, metric, threshold, operator='>', verbose=True)` - Apply hard cutoffs, returns boolean mask
+- `filter_cells(adata, mask, inplace=False)` - Apply boolean mask to filter cells
+- `filter_genes(adata, min_cells=20, min_counts=None, inplace=True)` - Filter genes by detection
+- `print_qc_summary(adata, label='')` - Print summary statistics
+
+From `qc_plotting.py` (visualization):
+- `plot_qc_distributions(adata, output_path, title)` - Generate comprehensive QC plots
+- `plot_filtering_thresholds(adata, outlier_masks, thresholds, output_path)` - Visualize filtering thresholds
+- `plot_qc_after_filtering(adata, output_path)` - Generate post-filtering plots
+
+**Example custom workflows:**
+
+**Example 1: Only calculate metrics and visualize, don't filter yet**
+```python
+adata = ad.read_h5ad('input.h5ad')
+calculate_qc_metrics(adata, inplace=True)
+plot_qc_distributions(adata, 'qc_before.png', title='Initial QC')
+print_qc_summary(adata, label='Before filtering')
+```
+
+**Example 2: Apply only MT% filtering, keep other metrics permissive**
+```python
+adata = ad.read_h5ad('input.h5ad')
+calculate_qc_metrics(adata, inplace=True)
+
+# Only filter high MT% cells
+high_mt = apply_hard_threshold(adata, 'pct_counts_mt', 10, operator='>')
+adata_filtered = filter_cells(adata, ~high_mt)
+adata_filtered.write('filtered.h5ad')
+```
+
+**Example 3: Different thresholds for different subsets**
+```python
+adata = ad.read_h5ad('input.h5ad')
+calculate_qc_metrics(adata, inplace=True)
+
+# Apply type-specific QC (assumes cell_type metadata exists)
+neurons = adata.obs['cell_type'] == 'neuron'
+other_cells = ~neurons
+
+# Neurons tolerate higher MT%, other cells use stricter threshold
+neuron_qc = apply_hard_threshold(adata[neurons], 'pct_counts_mt', 15, operator='>')
+other_qc = apply_hard_threshold(adata[other_cells], 'pct_counts_mt', 8, operator='>')
+```
+
+## Best Practices
+
+1. **Be permissive with filtering** - Default thresholds intentionally retain most cells to avoid losing rare populations
+2. **Inspect visualizations** - Always review before/after plots to ensure filtering makes biological sense
+3. **Consider dataset-specific factors** - Some tissues naturally have higher mitochondrial content (e.g., neurons, cardiomyocytes)
+4. **Check gene annotations** - Mitochondrial gene prefixes vary by species (mt- for mouse, MT- for human)
+5. **Iterate if needed** - QC parameters may need adjustment based on the specific experiment or tissue type
+
+## Reference Materials
+
+For detailed QC methodology, parameter rationale, and troubleshooting guidance, see `references/scverse_qc_guidelines.md`. This reference provides:
+- Detailed explanations of each QC metric and why it matters
+- Rationale for MAD-based thresholds and why they're better than fixed cutoffs
+- Guidelines for interpreting QC visualizations (histograms, violin plots, scatter plots)
+- Species-specific considerations for gene annotations
+- When and how to adjust filtering parameters
+- Advanced QC considerations (ambient RNA correction, doublet detection)
+
+Load this reference when users need deeper understanding of the methodology or when troubleshooting QC issues.
+
+## Next Steps After QC
+
+Typical downstream analysis steps:
+- Ambient RNA correction (SoupX, CellBender)
+- Doublet detection (scDblFinder)
+- Normalization (log-normalize, scran)
+- Feature selection and dimensionality reduction
+- Clustering and cell type annotation

package/data/skills/single-cellphone-db/SKILL.md

@@ -0,0 +1,68 @@
+---
+name: single-cell-cellphonedb-communication-mapping
+title: Single-cell CellPhoneDB communication mapping
+description: Run omicverse's CellPhoneDB v5 wrapper on annotated single-cell data to infer ligand-receptor networks and produce CellChat-style visualisations.
+---
+
+# Single-cell CellPhoneDB communication mapping
+
+## Overview
+Apply this skill when a user wants to quantify ligand–receptor communication between annotated single-cell populations and display the networks with `CellChatViz`. It distils the workflow from [`t_cellphonedb.ipynb`](../../omicverse_guide/docs/Tutorials-single/t_cellphonedb.ipynb), which analyses EVT trophoblast data.
+
+## Instructions
+1. **Prepare the environment**
+   - Use an environment with `omicverse>=0.2`, `scanpy`, `anndata`, `pandas`, `matplotlib`, and `cellphonedb` resources. The tutorial assumes the pre-built CellPhoneDB v5 SQLite bundle downloaded as `cellphonedb.zip` in the working directory.
+   - Activate omicverse plotting defaults via `ov.plot_set()` so that downstream figures follow the project palette.
+2. **Load and subset the annotated AnnData object**
+   - Read the normalised counts with `adata = ov.read('data/cpdb/normalised_log_counts.h5ad')`.
+   - Filter to the cell populations of interest using `adata.obs['cell_labels']` (e.g., EVT, dNK, VCT). Ensure `adata.obs['cell_labels']` is categorical and free of missing values so CellPhoneDB groups cells correctly.
+   - Confirm values are log-normalised (`adata.X.max()` should be <10 and non-integer); raw counts inflate CellPhoneDB permutations.
+3. **Run CellPhoneDB via omicverse**
+   - Execute `ov.single.run_cellphonedb_v5` with the curated AnnData and metadata column:
+     ```python
+     cpdb_results, adata_cpdb = ov.single.run_cellphonedb_v5(
+         adata,
+         cpdb_file_path='./cellphonedb.zip',
+         celltype_key='cell_labels',
+         min_cell_fraction=0.005,
+         min_genes=200,
+         min_cells=3,
+         iterations=1000,
+         threshold=0.1,
+         pvalue=0.05,
+         threads=10,
+         output_dir='./cpdb_results',
+         cleanup_temp=True,
+     )
+     ```
+   - Persist the outputs for reuse (`ov.utils.save(cpdb_results, ...)`, `adata_cpdb.write(...)`). Saving avoids recomputing permutations.
+4. **Initialise CellChat-style visualisation**
+   - Create a colour dictionary that maps ordered `cell_labels` categories to `adata.uns['cell_labels_colors']` from previous plots.
+   - Instantiate the viewer: `viz = ov.pl.CellChatViz(adata_cpdb, palette=color_dict)`. Inspect `adata_cpdb` to ensure communication slots (`uns`/`obsm`) were populated.
+5. **Summarise global communication**
+   - Derive aggregated counts/weights with `viz.compute_aggregated_network(pvalue_threshold=0.05, use_means=True)`.
+   - Plot overall interaction strength and counts using `viz.netVisual_circle(...)` with matching figure sizes and colormaps.
+   - Generate outgoing/incoming per-celltype circles using `viz.netVisual_individual_circle` and `viz.netVisual_individual_circle_incoming` to highlight senders versus receivers.
+6. **Interrogate specific pathways**
+   - Compute pathway summaries: `pathway_comm = viz.compute_pathway_communication(method='mean', min_lr_pairs=2, min_expression=0.1)`.
+   - Identify significant signalling routes with `viz.get_significant_pathways_v2(...)`, then plot selected pathways using `viz.netVisual_aggregate(..., layout='circle')`, `viz.netVisual_chord_cell(...)`, or `viz.netVisual_heatmap_marsilea(...)`.
+   - For ligand–receptor focus, call `viz.netVisual_chord_LR(...)` or `viz.netAnalysis_contribution(pathway)` to surface dominant pairs.
+7. **System-level visualisations**
+   - Compose bubble summaries for multiple pathways with `viz.netVisual_bubble_marsilea(...)`, optionally restricting `sources_use`/`targets_use`.
+   - Display gene-level chords via `viz.netVisual_chord_gene(...)` to inspect signalling directionality.
+   - Evaluate signalling roles using `viz.netAnalysis_computeCentrality()`, `viz.netAnalysis_signalingRole_network_marsilea(...)`, `viz.netAnalysis_signalingRole_scatter(...)`, and `viz.netAnalysis_signalingRole_heatmap(...)` for incoming/outgoing programmes.
+8. **Troubleshooting tips**
+   - **Metadata alignment**: CellPhoneDB requires a categorical `celltype_key`. If the column contains spaces, mixed casing, or `NaN`, clean it (`adata.obs['cell_labels'] = adata.obs['cell_labels'].astype('category').cat.remove_unused_categories()`).
+   - **Database bundle**: `cpdb_file_path` must point to a full CellPhoneDB v5 SQLite zip. If omicverse raises `FileNotFoundError` or missing receptor tables, re-download the bundle from the official release and ensure the zip is not corrupted.
+   - **Permutation failures**: Low cell counts per group (<`min_cells`) cause early termination. Increase `min_cell_fraction` thresholds or merge sparse clusters before rerunning.
+   - **Palette mismatches**: When colours render incorrectly, rebuild `color_dict` from `adata.uns['cell_labels_colors']` after sorting categories to keep nodes and legends consistent.
+
+## Examples
+- "Run CellPhoneDB on our trophoblast dataset and export both the cpdb results pickle and processed AnnData."
+- "Highlight significant 'Signaling by Fibroblast growth factor' interactions with chord and bubble plots."
+- "Generate outgoing versus incoming communication circles to compare dNK subsets."
+
+## References
+- Tutorial notebook: [`t_cellphonedb.ipynb`](../../omicverse_guide/docs/Tutorials-single/t_cellphonedb.ipynb)
+- Example data: [`omicverse_guide/docs/Tutorials-single/data/cpdb/`](../../omicverse_guide/docs/Tutorials-single/data/cpdb/)
+- Quick copy/paste commands: [`reference.md`](reference.md)

package/data/skills/single-clustering/SKILL.md

@@ -0,0 +1,75 @@
+---
+name: single-cell-clustering-and-batch-correction-with-omicverse
+title: Single-cell clustering and batch correction with omicverse
+description: Guide Claude through omicverse's single-cell clustering workflow, covering preprocessing, QC, multimethod clustering, topic modeling, cNMF, and cross-batch integration as demonstrated in t_cluster.ipynb and t_single_batch.ipynb.
+---
+
+# Single-cell clustering and batch correction with omicverse
+
+## Overview
+This skill distills the single-cell tutorials [`t_cluster.ipynb`](../../omicverse_guide/docs/Tutorials-single/t_cluster.ipynb) and [`t_single_batch.ipynb`](../../omicverse_guide/docs/Tutorials-single/t_single_batch.ipynb). Use it when a user wants to preprocess an `AnnData` object, explore clustering alternatives (Leiden, Louvain, scICE, GMM, topic/cNMF models), and evaluate or harmonise batches with omicverse utilities.
+
+## Instructions
+1. **Import libraries and set plotting defaults**
+   - Load `omicverse as ov`, `scanpy as sc`, and plotting helpers (`scvelo as scv` when using dentate gyrus demo data).
+   - Apply `ov.plot_set()` or `ov.utils.ov_plot_set()` so figures adopt omicverse styling before embedding plots.
+2. **Load data and annotate batches**
+   - For demo clustering, fetch `scv.datasets.dentategyrus()`; for integration, read provided `.h5ad` files via `ov.read()` and set `adata.obs['batch']` identifiers for each cohort.
+   - Confirm inputs are sparse numeric matrices; convert with `adata.X = adata.X.astype(np.int64)` when required for QC steps.
+3. **Run quality control**
+   - Execute `ov.pp.qc(adata, tresh={'mito_perc': 0.2, 'nUMIs': 500, 'detected_genes': 250}, batch_key='batch')` to drop low-quality cells and inspect summary statistics per batch.
+   - Save intermediate filtered objects (`adata.write_h5ad(...)`) so users can resume from clean checkpoints.
+4. **Preprocess and select features**
+   - Call `ov.pp.preprocess(adata, mode='shiftlog|pearson', n_HVGs=3000, batch_key=None)` to normalise, log-transform, and flag highly variable genes; assign `adata.raw = adata` and subset to `adata.var.highly_variable_features` for downstream modelling.
+   - Scale expression (`ov.pp.scale(adata)`) and compute PCA scores with `ov.pp.pca(adata, layer='scaled', n_pcs=50)`. Encourage reviewing variance explained via `ov.utils.plot_pca_variance_ratio(adata)`.
+5. **Construct neighbourhood graph and baseline clustering**
+   - Build neighbour graph using `sc.pp.neighbors(adata, n_neighbors=15, n_pcs=50, use_rep='scaled|original|X_pca')` or `ov.pp.neighbors(...)`.
+   - Generate Leiden or Louvain labels through `ov.utils.cluster(adata, method='leiden'|'louvain', resolution=1)`, `ov.single.leiden(adata, resolution=1.0)`, or `ov.pp.leiden(adata, resolution=1)`; remind users that resolution tunes granularity.
+   - **IMPORTANT - Dependency checks**: Always verify prerequisites before clustering or plotting:
+     ```python
+     # Before clustering: check neighbors graph exists
+     if 'neighbors' not in adata.uns:
+         if 'X_pca' in adata.obsm:
+             ov.pp.neighbors(adata, n_neighbors=15, use_rep='X_pca')
+         else:
+             raise ValueError("PCA must be computed before neighbors graph")
+
+     # Before plotting by cluster: check clustering was performed
+     if 'leiden' not in adata.obs:
+         ov.single.leiden(adata, resolution=1.0)
+     ```
+   - Visualise embeddings with `ov.pl.embedding(adata, basis='X_umap', color=['clusters','leiden'], frameon='small', wspace=0.5)` and confirm cluster separation. Always check that columns in `color=` parameter exist in `adata.obs` before plotting.
+6. **Explore advanced clustering strategies**
+   - **scICE consensus**: instantiate `model = ov.utils.cluster(adata, method='scICE', use_rep='scaled|original|X_pca', resolution_range=(4,20), n_boot=50, n_steps=11)` and inspect stability via `model.plot_ic(figsize=(6,4))` before selecting `model.best_k` groups.
+   - **Gaussian mixtures**: run `ov.utils.cluster(..., method='GMM', n_components=21, covariance_type='full', tol=1e-9, max_iter=1000)` for model-based assignments.
+   - **Topic modelling**: fit `LDA_obj = ov.utils.LDA_topic(...)`, review `LDA_obj.plot_topic_contributions(6)`, derive cluster calls with `LDA_obj.predicted(k)` and optionally refine using `LDA_obj.get_results_rfc(...)`.
+   - **cNMF programs**: initialise `cnmf_obj = ov.single.cNMF(... components=np.arange(5,11), n_iter=20, num_highvar_genes=2000, output_dir=...)`, factorise (`factorize`, `combine`), select K via `k_selection_plot`, and propagate usage scores back with `cnmf_obj.get_results(...)` and `cnmf_obj.get_results_rfc(...)`.
+7. **Evaluate clustering quality**
+   - Compare predicted labels against known references with `adjusted_rand_score(adata.obs['clusters'], adata.obs['leiden'])` and report metrics for each method (Leiden, Louvain, GMM, LDA variants, cNMF models) to justify chosen parameters.
+8. **Embed with multiple layouts**
+   - Use `ov.utils.mde(...)` to create MDE projections from different latent spaces (`adata.obsm["scaled|original|X_pca"]`, harmonised embeddings, topic compositions) and plot via `ov.utils.embedding(..., color=['batch','cell_type'])` or `ov.pl.embedding` for consistent review of cluster/batch mixing.
+9. **Perform batch correction and integration**
+   - Apply `ov.single.batch_correction(adata, batch_key='batch', methods='harmony'|'combat'|'scanorama'|'scVI'|'CellANOVA', n_pcs=50, ...)` sequentially to generate harmonised embeddings stored in `adata.obsm` (`X_harmony`, `X_combat`, `X_scanorama`, `X_scVI`, `X_cellanova`). For `scVI`, mention latent size (`n_latent=30`) and `gene_likelihood="nb"`; for CellANOVA define control pools via `control_dict`.
+   - After each correction, project to 2D with `ov.utils.mde` and visualise `batch` vs `cell_type` to check mixing and conservation.
+10. **Benchmark integration performance**
+    - Persist final object (`adata.write_h5ad('neurips2021_batch_all.h5ad', compression='gzip')`) and reload when necessary.
+    - Use `scib_metrics.benchmark.Benchmarker` with embeddings list (`["X_pca", "X_combat", "X_harmony", "X_cellanova", "X_scanorama", "X_mira_topic", "X_mira_feature", "X_scVI"]`) to compute batch-vs-biology trade-offs via `bm.benchmark()` and summarise with `bm.plot_results_table(min_max_scale=False)`.
+11. **General troubleshooting**
+    - Ensure `adata.raw` captures the unscaled log-normalised matrix before subsetting to HVGs.
+    - Confirm `use_rep='scaled|original|X_pca'` strings exist in `adata.obsm` prior to clustering; rerun preprocessing if missing.
+    - Monitor memory when running cNMF or scVI; adjust `n_iter`, `components`, or latent dimensions for smaller datasets.
+    - **Pipeline dependency errors**: When you encounter errors like "Could not find 'leiden' in adata.obs", always check and add prerequisites:
+      - Before leiden/louvain clustering → ensure `'neighbors' in adata.uns`
+      - Before plotting by clustering → ensure the cluster column exists in `adata.obs`
+      - Before UMAP/embedding → ensure PCA or another dimensionality reduction is complete
+    - **Code generation pattern**: When generating multi-step code, use defensive checks rather than assuming prior steps completed successfully. This prevents cascading failures when users run steps out of order or in separate sessions.
+
+## Examples
+- "Normalise dentate gyrus cells, compare Leiden, scICE, and GMM clusters, and report ARI scores versus provided `clusters`."
+- "Batch-correct three NeurIPS datasets with Harmony and scVI, produce MDE embeddings coloured by `batch` and `cell_type`, and benchmark the embeddings."
+- "Fit topic and cNMF models on a preprocessed AnnData object, retrieve classifier-refined cluster calls, and visualise the resulting programs on UMAP."
+
+## References
+- Clustering walkthrough: [`t_cluster.ipynb`](../../omicverse_guide/docs/Tutorials-single/t_cluster.ipynb)
+- Batch integration walkthrough: [`t_single_batch.ipynb`](../../omicverse_guide/docs/Tutorials-single/t_single_batch.ipynb)
+- Quick copy/paste commands: [`reference.md`](reference.md)

package/data/skills/single-downstream-analysis/SKILL.md

@@ -0,0 +1,150 @@
+---
+name: single-cell-downstream-analysis
+title: Single-cell downstream analysis
+description: Checklist-style reference for OmicVerse downstream tutorials covering AUCell scoring, metacell DEG, and related exports.
+---
+
+# Single-cell downstream analysis quick-reference
+
+This skill sheet distills the OmicVerse single-cell downstream tutorials into an executable checklist. Each module
+highlights **prerequisites**, the **core API entry points**, **interpretation checkpoints**, **resource planning notes**, and
+any **optional validation or export steps** surfaced in the notebooks.
+
+## AUCell pathway scoring (`t_aucell.ipynb`)
+- **Prerequisites**
+  - Download pathway collections (GO, KEGG, or custom) that match the organism under study before running the tutorial.
+  - Ensure an `AnnData` object with clustering/embedding (`adata.obsm['X_umap']`) is prepared.
+- **Core calls**
+  - `ov.single.geneset_aucell` for one pathway; `ov.single.pathway_aucell` for multiple pathways.
+  - `ov.single.pathway_aucell_enrichment` to score all pathways in a library (set `num_workers` for parallelism).
+- **Result checks**
+  - Interpret AUCell scores as expression-like values (0–1). Use `sc.pl.embedding` to confirm pathway activity patterns.
+  - Run `sc.tl.rank_genes_groups` on the AUCell `AnnData` to find cluster-enriched pathways and visualize with
+    `sc.pl.rank_genes_groups_dotplot`.
+- **Resources**
+  - Library-wide scoring can be CPU-intensive; allocate workers (`num_workers=8` in tutorial) and sufficient memory for the
+    dense AUCell matrix.
+- **Optional validation / exports**
+  - Persist scores with `adata_aucs.write_h5ad('...')` for reuse.
+  - Plot enriched pathways via `ov.single.pathway_enrichment` and `ov.single.pathway_enrichment_plot` heatmaps.
+
+## scRNA-seq DEG (bulk-style meta cell) (`t_scdeg.ipynb`)
+- **Prerequisites**
+  - Run quality control and preprocessing (`ov.pp.qc`, `ov.pp.preprocess`, `ov.pp.scale`, `ov.pp.pca`).
+  - Retain raw counts in `adata.raw` before HVG filtering.
+- **Core calls**
+  - Construct differential objects with `ov.bulk.pyDEG(test_adata.to_df(...).T)` for full-cell and metacell views.
+  - Build metacells via `ov.single.MetaCell(..., use_gpu=True)` when GPU is available for acceleration.
+- **Result checks**
+  - Inspect volcano plots (`dds.plot_volcano`) and targeted boxplots (`dds.plot_boxplot`) for top DEGs.
+  - Map DEG markers back to UMAP embeddings using `ov.utils.embedding` to confirm localization.
+- **Resources**
+  - Metacell construction benefits from GPU but can fall back to CPU; ensure enough memory for transposed dense matrices
+    passed to `pyDEG`.
+- **Optional validation / exports**
+  - Save metacell embeddings with matplotlib figures; adjust `legend_*` settings for publication-ready visuals.
+
+## scRNA-seq DEG (cell-type & composition) (`t_deg_single.ipynb`)
+- **Prerequisites**
+  - Annotated `adata` with `condition`, `cell_label`, and optional `batch` metadata.
+  - Initialize mixed CPU/GPU resources when using graph-based DA methods (`ov.settings.cpu_gpu_mixed_init()`).
+- **Core calls**
+  - `ov.single.DEG(..., method='wilcoxon'|'t-test'|'memento-de')` with `deg_obj.run(...)` to target cell types.
+  - `ov.single.DCT(..., method='sccoda'|'milo')` for differential composition testing.
+  - Graph setup for Milo: `ov.pp.preprocess`, `ov.single.batch_correction`, `ov.pp.neighbors`, `ov.pp.umap`.
+- **Result checks**
+  - Review DEG tables from `deg_obj` (Wilcoxon / memento) and adjust capture rate / bootstraps for stability.
+  - For scCODA, tune FDR via `sim_results.set_fdr()`; interpret boxplots with condition-level shifts.
+  - Milo diagnostics: histogram of P-values, logFC vs –log10 FDR scatter, beeswarm of differential abundance.
+- **Resources**
+  - Memento and Milo require multiple CPUs (`num_cpus`, `num_boot`, high `k`); ensure adequate compute time.
+  - Harmony/scVI batch correction needs GPU memory when enabled; plan for VRAM usage.
+- **Optional validation / exports**
+  - Visual diagnostics include UMAP overlays (`ov.pl.embedding`), Milo beeswarm plots, and custom color palettes.
+
+## scDrug response prediction (`t_scdrug.ipynb`)
+- **Prerequisites**
+  - Fetch tumor-focused dataset (e.g., `infercnvpy.datasets.maynard2020_3k`).
+  - Download reference assets **before** running predictions:
+    - Gene annotations via `ov.utils.get_gene_annotation` (requires GTF from GENCODE or T2T-CHM13).
+    - `ov.utils.download_GDSC_data()` and `ov.utils.download_CaDRReS_model()` for drug-response models.
+    - Clone CaDRReS-Sc repo (`git clone https://github.com/CSB5/CaDRReS-Sc`).
+- **Core calls**
+  - Tumor resolution detection: `ov.single.autoResolution(adata, cpus=4)`.
+  - Drug response runner: `ov.single.Drug_Response(adata, scriptpath='CaDRReS-Sc', modelpath='models/', output='result')`.
+- **Result checks**
+  - Inspect clustering and IC50 outputs stored under `output`; cross-reference with inferred CNV states.
+- **Resources**
+  - Requires external CaDRReS-Sc environment (Python/R dependencies) and storage for model downloads.
+  - Running inferCNV preprocessing may need multiple CPUs and substantial RAM.
+- **Optional validation / exports**
+  - Persist intermediate `AnnData` (`adata.write('scanpyobj.h5ad')`) to reuse for downstream analyses or re-runs.
+
+## SCENIC regulon discovery (`t_scenic.ipynb`)
+- **Prerequisites**
+  - Mouse hematopoiesis dataset loaded via `ov.single.mouse_hsc_nestorowa16()` (or provide preprocessed data with raw counts).
+  - Download cisTarget ranking databases (`*.feather`) and motif annotations (`motifs-*.tbl`) for the species; allocate
+    >3 GB disk space and verify paths (`db_glob`, `motif_path`).
+- **Core calls**
+  - Initialize analysis: `ov.single.SCENIC(adata, db_glob=..., motif_path=..., n_jobs=12)`.
+  - Run RegDiffusion-based GRN inference, regulon pruning, and AUCell scoring via the SCENIC object methods.
+- **Result checks**
+  - Examine regulon activity matrices (`scenic_obj.auc_mtx.head()`), RSS scores, and embeddings colored by regulon activity.
+  - Use RSS plots, dendrograms, and AUCell distributions to interpret TF specificity and activity thresholds.
+- **Resources**
+  - Multi-core CPU recommended (`n_jobs` matches available cores); ensure enough RAM for motif enrichment.
+  - Large downloads and intermediate objects (pickle/h5ad) require disk space.
+- **Optional validation / exports**
+  - Save `scenic_obj` (`ov.utils.save`) and regulon AnnData (`regulon_ad.write`).
+  - Optional plots: RSS per cell type, regulon embeddings, AUC histograms with threshold lines, GRN network visualizations.
+
+## cNMF program discovery (`t_cnmf.ipynb`)
+- **Prerequisites**
+  - Preprocess with HVG selection (`ov.pp.preprocess`), scaling (`ov.pp.scale`), PCA, and have UMAP embeddings for inspection.
+  - Select component range (e.g., `np.arange(5, 11)`) and iterations; ensure output directory exists.
+- **Core calls**
+  - Instantiate analysis: `ov.single.cNMF(..., output_dir='...', name='...')`.
+  - Factorization workflow: `cnmf_obj.factorize(...)`, `cnmf_obj.combine(...)`, `cnmf_obj.k_selection_plot()`,
+    `cnmf_obj.consensus(...)`.
+  - Extract results: `cnmf_obj.load_results(...)`, `cnmf_obj.get_results(...)`, optional RF classifier via `get_results_rfc`.
+- **Result checks**
+  - Evaluate stability via K-selection plot and local density histogram; confirm chosen K with consensus heatmaps.
+  - Inspect topic usage embeddings (`ov.pl.embedding`), cluster labels, and dotplots of top genes.
+- **Resources**
+  - Multiple iterations and components are CPU-heavy; consider distributing workers (`total_workers`) and verifying disk
+    space for intermediate factorization files.
+- **Optional validation / exports**
+  - Visualizations include Euclidean distance heatmaps, density histograms, UMAP overlays for topics/clusters, and dotplots.
+
+## NOCD overlapping communities (`t_nocd.ipynb`)
+- **Prerequisites**
+  - Prepare AnnData via `ov.single.scanpy_lazy` (automated preprocessing) before running NOCD.
+  - Note: Tutorial warns NOCD implementation is under active development—expect variability.
+- **Core calls**
+  - Pipeline wrapper: `scbrca = ov.single.scnocd(adata)` followed by chained methods (`matrix_transform`, `matrix_normalize`,
+    `GNN_configure`, `GNN_preprocess`, `GNN_model`, `GNN_result`, `GNN_plot`, `cal_nocd`, `calculate_nocd`).
+- **Result checks**
+  - Compare standard Leiden clusters versus NOCD outputs on UMAP embeddings to identify multi-fate cells.
+- **Resources**
+  - Graph neural network stages can be GPU-accelerated; ensure CUDA availability or be prepared for longer CPU runtimes.
+  - Track memory usage when constructing large adjacency matrices.
+- **Optional validation / exports**
+  - Generate multiple UMAP overlays (`sc.pl.umap`) for `nocd`, `nocd_n`, and Leiden labels using shared color maps.
+
+## Lazy pipeline & reporting (`t_lazy.ipynb`)
+- **Prerequisites**
+  - Install OmicVerse ≥1.7.0 with lazy utilities; supported species currently human/mouse.
+  - Prepare batch metadata (`sample_key`) and optionally initialize hybrid compute (`ov.settings.cpu_gpu_mixed_init()`).
+- **Core calls**
+  - Turnkey preprocessing: `ov.single.lazy(adata, species='mouse', sample_key='batch', ...)` with optional `reforce_steps`
+    and module-specific kwargs.
+  - Reporting: `ov.single.generate_scRNA_report(...)` to build HTML summary; `ov.generate_reference_table(adata)` for
+    citation tracking.
+- **Result checks**
+  - Inspect generated embeddings (`ov.pl.embedding`) for quality and annotation alignment.
+  - Review HTML report for QC metrics, normalization, batch correction, and embeddings.
+- **Resources**
+  - Steps like Harmony or scVI may invoke GPU; confirm hardware availability or adjust `reforce_steps` accordingly.
+  - Report generation writes to disk; ensure output path is writable.
+- **Optional validation / exports**
+  - Customize embeddings by color key; store HTML report and reference table alongside project documentation.

package/data/skills/single-multiomics/SKILL.md

@@ -0,0 +1,44 @@
+---
+name: single-cell-multi-omics-integration
+title: Single-cell multi-omics integration
+description: Quick-reference sheet for OmicVerse tutorials spanning MOFA, GLUE pairing, SIMBA integration, TOSICA transfer, and StaVIA cartography.
+---
+
+# Single-Cell Multi-Omics Tutorials Cheat Sheet
+
+This skill walk-through summarizes the OmicVerse notebooks that cover paired and unpaired multi-omic integration, multi-batch embedding, reference transfer, and trajectory cartography.
+
+## MOFA on paired scRNA + scATAC (`t_mofa.ipynb`)
+- **Data preparation:** Load preprocessed AnnData objects for RNA (`rna_p_n_raw.h5ad`) and ATAC (`atac_p_n_raw.h5ad`) with `ov.utils.read`, and initialise `pyMOFA` with matching `omics` and `omics_name` lists.
+- **Model training:** Call `mofa_preprocess()` to select highly variable features and run the factor model with `mofa_run(outfile=...)`, which exports the learned MOFA+ factors to an HDF5 model file.
+- **Result inspection:** Reload downstream AnnData, append factor scores via `ov.single.factor_exact`, and explore factor–cluster associations using `factor_correlation`, `get_weights`, and the plotting helpers in `pyMOFAART` (`plot_r2`, `plot_cor`, `plot_factor`, `plot_weights`, etc.).
+- **Export workflow:** Persist factors and weights through the MOFA HDF5 artifact and reuse them by instantiating `pyMOFAART(model_path=...)` for later annotation or visualisation sessions.
+- **Dependencies & hardware:** Requires `mofapy2`; plots optionally rely on `pymde`/`scvi-tools` but run on CPU.
+
+## MOFA after GLUE pairing (`t_mofa_glue.ipynb`)
+- **Data preparation:** Start from GLUE-derived embeddings (`rna-emb.h5ad`, `atac.emb.h5ad`), build a `GLUE_pair` object, and run `correlation()` to align unpaired cells before subsetting to highly variable features.
+- **Model training:** Instantiate `pyMOFA` with the aligned AnnData objects, run `mofa_preprocess()`, and save the joint factors through `mofa_run(outfile='models/chen_rna_atac.hdf5')`.
+- **Result inspection:** Use `pyMOFAART` plus AnnData that now contains the GLUE embeddings to compute factors (`get_factors`) and visualise variance explained, factor–cluster correlations, and ranked feature weights.
+- **Export workflow:** Reuse the saved MOFA HDF5 model for downstream inspection; GLUE embeddings can be embedded with `scvi.model.utils.mde` (GPU-accelerated MDE is optional, `sc.tl.umap` works on CPU).
+- **Dependencies & hardware:** Requires both `mofapy2` and the GLUE tooling (`scglue`, `scvi-tools`, `pymde`); GPU acceleration only affects optional MDE visualisation.
+
+## SIMBA batch integration (`t_simba.ipynb`)
+- **Data preparation:** Fetch the concatenated AnnData (`simba_adata_raw.h5ad`) derived from multiple pancreas studies and pass it, alongside a results directory, to `pySIMBA`.
+- **Model training:** Execute `preprocess(...)` to bin features and build a SIMBA-compatible graph, then call `gen_graph()` followed by `train(num_workers=...)` to launch PyTorch-BigGraph optimisation (can scale with CPU workers) and `load(...)` to resume trained checkpoints.
+- **Result inspection:** Apply `batch_correction()` to obtain the harmonised AnnData with SIMBA embeddings (`X_simba`) and visualise using `mde`/`sc.tl.umap` coloured by cell type or batch.
+- **Export workflow:** Training outputs reside in the workdir (e.g., `result_human_pancreas/pbg/graph0`); reuse them with `simba_object.load(...)` for later analyses.
+- **Dependencies & hardware:** Requires installing `simba` and `simba_pbg` (PyTorch BigGraph backend). GPU is optional; make sure adequate CPU threads and memory are available for graph training.
+
+## TOSICA reference transfer (`t_tosica.ipynb`)
+- **Data preparation:** Download demo AnnData references (`demo_train.h5ad`, `demo_test.h5ad`) and required gene-set GMT files via `ov.utils.download_tosica_gmt()`; confirm datasets are log-normalised before training.
+- **Model training:** Create `pyTOSICA` with the reference AnnData, chosen pathway mask, label key, project directory, and batch size; train with `train(epochs=...)`, then persist weights with `save()` and optionally reload via `load()`.
+- **Result inspection:** Generate predictions on query AnnData through `predicted(pre_adata=...)`, embed with OmicVerse preprocessing and GPU-enabled `mde` (UMAP fallback available), and explore pathway attention to interpret transformer heads.
+- **Export workflow:** Saved project folder keeps model checkpoints and attention summaries; reuse the exported assets to annotate future datasets without retraining from scratch.
+- **Dependencies & hardware:** Needs TOSICA (PyTorch transformer) plus downloaded gene-set masks; avoid setting `depth=2` if memory is constrained. GPU acceleration improves embedding (`mde`) but training runs on standard PyTorch (CPU/GPU depending on environment).
+
+## StaVIA trajectory cartography (`t_stavia.ipynb`)
+- **Data preparation:** Load example dentate gyrus velocity data via `scvelo.datasets.dentategyrus()`, preprocess with OmicVerse (`preprocess`, `scale`, `pca`, neighbours, UMAP) to populate the AnnData matrices used by VIA.
+- **Model training:** Configure VIA hyperparameters (components, neighbours, seeds, root selection) and instantiate/run `VIA.core.VIA` on the chosen representation (`adata.obsm['scaled|original|X_pca']`).
+- **Result inspection:** Store outputs such as pseudotime (`single_cell_pt_markov`), cluster graph abstractions, trajectory curves, atlas views, and stream plots through VIA plotting helpers.
+- **Export workflow:** Persist derived visualisations and animations (e.g., `animate_streamplot_ov`, `animate_atlas`) to files (`.gif`) for reporting; recompute edge bundles via `make_edgebundle_milestone` when needed.
+- **Dependencies & hardware:** Relies on `scvelo`, `pyVIA`, and OmicVerse plotting; computations are CPU-bound though producing large stream/animation outputs benefits from ample memory.