@bgicli/bgicli 2.1.1 → 2.2.0

package/data/skills/gene-panel-design-agent/SKILL.md

@@ -0,0 +1,192 @@
+<!--
+# COPYRIGHT NOTICE
+# This file is part of the "Universal Biomedical Skills" project.
+# Copyright (c) 2026 MD BABU MIA, PhD <md.babu.mia@mssm.edu>
+# All Rights Reserved.
+#
+# This code is proprietary and confidential.
+# Unauthorized copying of this file, via any medium is strictly prohibited.
+#
+# Provenance: Authenticated by MD BABU MIA
+
+-->
+
+---
+name: 'gene-panel-design-agent'
+description: 'AI-powered design of targeted gene panels for clinical and research applications including cancer diagnostics, pharmacogenomics, and rare disease testing.'
+measurable_outcome: Execute skill workflow successfully with valid output within 15 minutes.
+allowed-tools:
+  - read_file
+  - run_shell_command
+---
+
+
+# Gene Panel Design Agent
+
+The **Gene Panel Design Agent** provides AI-driven design of targeted sequencing panels for clinical diagnostics, cancer profiling, pharmacogenomics, and research applications.
+
+## When to Use This Skill
+
+* When designing custom gene panels for clinical or research use.
+* To optimize panel content for specific disease areas.
+* For balancing panel size with diagnostic yield.
+* When designing probes for hybrid capture or amplicon approaches.
+* To validate panel performance computationally.
+
+## Core Capabilities
+
+1. **Gene Selection**: Evidence-based gene prioritization for disease areas.
+
+2. **Target Region Definition**: Specify exons, introns, UTRs, promoters to include.
+
+3. **Probe Design**: In silico probe/primer design for capture or amplicon.
+
+4. **Coverage Prediction**: Estimate uniformity and dropout risk.
+
+5. **Validation Planning**: Design positive controls and performance metrics.
+
+6. **Cost Optimization**: Balance panel size with clinical utility.
+
+## Workflow
+
+1. **Input**: Disease focus, required genes, platform choice, size constraints.
+
+2. **Gene Prioritization**: Rank genes by clinical evidence level.
+
+3. **Region Definition**: Define target coordinates.
+
+4. **Probe Design**: Generate capture probes or primers.
+
+5. **Coverage Simulation**: Predict sequencing performance.
+
+6. **Optimization**: Iterate design for uniformity.
+
+7. **Output**: Panel BED file, probe sequences, validation plan.
+
+## Example Usage
+
+**User**: "Design a comprehensive solid tumor panel covering actionable mutations and resistance markers."
+
+**Agent Action**:
+```bash
+python3 Skills/Genomics/Gene_Panel_Design_Agent/panel_designer.py \
+    --disease solid_tumor \
+    --gene_sources nccn,civic,oncokb \
+    --platform hybcap \
+    --target_size 1.5mb \
+    --include_fusions true \
+    --include_cnv_backbone true \
+    --output panel_design/
+```
+
+## Panel Design Considerations
+
+| Factor | Impact | Optimization |
+|--------|--------|--------------|
+| Panel size | Cost, depth | Prioritize high-evidence genes |
+| GC content | Coverage uniformity | Probe design, blockers |
+| Repeat regions | Mapping challenges | Avoid or boost coverage |
+| Homologous regions | Misalignment | Unique design, blockers |
+| Structural variants | Detection | Intronic coverage, breakpoints |
+| CNV detection | Require backbone | Tiled probes across genome |
+
+## Gene Prioritization Sources
+
+| Source | Content | Evidence Level |
+|--------|---------|----------------|
+| OncoKB | Actionable alterations | FDA/guideline levels |
+| CIViC | Clinical variants | Community-curated |
+| ClinVar | Pathogenic variants | Classification criteria |
+| NCCN | Guideline genes | Clinical practice |
+| COSMIC | Cancer genes | Census tier 1/2 |
+
+## Panel Types
+
+**Comprehensive Cancer Panel** (300-700 genes):
+- All known cancer drivers
+- Actionable mutations
+- Resistance markers
+- MSI/TMB estimation
+
+**Focused Tumor Panel** (50-100 genes):
+- Most actionable genes
+- Cost-effective
+- Higher depth possible
+
+**Pharmacogenomics Panel**:
+- CPIC/DPWG genes
+- CYP450, HLA, transporters
+- Star allele compatible design
+
+**Rare Disease Panel**:
+- Disease-specific genes
+- Deep intronic variants
+- CNV detection
+
+## AI/ML Components
+
+**Gene Ranking**:
+- Literature mining for evidence
+- Mutation frequency weighting
+- Actionability scoring
+
+**Probe Optimization**:
+- GC content balancing
+- Tm normalization
+- Off-target minimization
+
+**Coverage Prediction**:
+- ML models from historical data
+- GC-coverage relationships
+- Dropout prediction
+
+## Validation Planning
+
+**Performance Metrics**:
+- Coverage uniformity (CV)
+- On-target rate
+- Sensitivity by variant type
+- Reproducibility
+
+**Reference Materials**:
+- Horizon Discovery cell lines
+- SeraCare controls
+- Well-characterized samples
+- In silico spike-ins
+
+## Technical Specifications
+
+| Platform | Typical Size | Depth | CNV Capable |
+|----------|--------------|-------|-------------|
+| Hybrid capture | 1-3 Mb | 500-1000x | Yes (with backbone) |
+| Amplicon | 10-500 kb | 1000-5000x | Limited |
+| Anchored multiplex | Variable | Variable | Fusions |
+
+## Prerequisites
+
+* Python 3.10+
+* BEDTools for coordinate manipulation
+* Probe design algorithms
+* Reference genome and annotations
+
+## Related Skills
+
+* CRISPR_Design_Agent - For guide design
+* Variant_Interpretation - For variant selection
+* Tumor_Mutational_Burden_Agent - For TMB panel requirements
+
+## Output Files
+
+| File | Content | Purpose |
+|------|---------|---------|
+| panel.bed | Target coordinates | Sequencing design |
+| probes.fa | Probe sequences | Manufacturing |
+| genes.csv | Gene list with rationale | Documentation |
+| validation.pdf | QC plan | Laboratory setup |
+
+## Author
+
+AI Group - Biomedical AI Platform
+
+
+<!-- AUTHOR_SIGNATURE: 9a7f3c2e-MD-BABU-MIA-2026-MSSM-SECURE -->

package/data/skills/geniml/SKILL.md

@@ -0,0 +1,312 @@
+---
+name: geniml
+description: This skill should be used when working with genomic interval data (BED files) for machine learning tasks. Use for training region embeddings (Region2Vec, BEDspace), single-cell ATAC-seq analysis (scEmbed), building consensus peaks (universes), or any ML-based analysis of genomic regions. Applies to BED file collections, scATAC-seq data, chromatin accessibility datasets, and region-based genomic feature learning.
+---
+
+# Geniml: Genomic Interval Machine Learning
+
+## Overview
+
+Geniml is a Python package for building machine learning models on genomic interval data from BED files. It provides unsupervised methods for learning embeddings of genomic regions, single cells, and metadata labels, enabling similarity searches, clustering, and downstream ML tasks.
+
+## Installation
+
+Install geniml using uv:
+
+```bash
+uv uv pip install geniml
+```
+
+For ML dependencies (PyTorch, etc.):
+
+```bash
+uv uv pip install 'geniml[ml]'
+```
+
+Development version from GitHub:
+
+```bash
+uv uv pip install git+https://github.com/databio/geniml.git
+```
+
+## Core Capabilities
+
+Geniml provides five primary capabilities, each detailed in dedicated reference files:
+
+### 1. Region2Vec: Genomic Region Embeddings
+
+Train unsupervised embeddings of genomic regions using word2vec-style learning.
+
+**Use for:** Dimensionality reduction of BED files, region similarity analysis, feature vectors for downstream ML.
+
+**Workflow:**
+1. Tokenize BED files using a universe reference
+2. Train Region2Vec model on tokens
+3. Generate embeddings for regions
+
+**Reference:** See `references/region2vec.md` for detailed workflow, parameters, and examples.
+
+### 2. BEDspace: Joint Region and Metadata Embeddings
+
+Train shared embeddings for region sets and metadata labels using StarSpace.
+
+**Use for:** Metadata-aware searches, cross-modal queries (region→label or label→region), joint analysis of genomic content and experimental conditions.
+
+**Workflow:**
+1. Preprocess regions and metadata
+2. Train BEDspace model
+3. Compute distances
+4. Query across regions and labels
+
+**Reference:** See `references/bedspace.md` for detailed workflow, search types, and examples.
+
+### 3. scEmbed: Single-Cell Chromatin Accessibility Embeddings
+
+Train Region2Vec models on single-cell ATAC-seq data for cell-level embeddings.
+
+**Use for:** scATAC-seq clustering, cell-type annotation, dimensionality reduction of single cells, integration with scanpy workflows.
+
+**Workflow:**
+1. Prepare AnnData with peak coordinates
+2. Pre-tokenize cells
+3. Train scEmbed model
+4. Generate cell embeddings
+5. Cluster and visualize with scanpy
+
+**Reference:** See `references/scembed.md` for detailed workflow, parameters, and examples.
+
+### 4. Consensus Peaks: Universe Building
+
+Build reference peak sets (universes) from BED file collections using multiple statistical methods.
+
+**Use for:** Creating tokenization references, standardizing regions across datasets, defining consensus features with statistical rigor.
+
+**Workflow:**
+1. Combine BED files
+2. Generate coverage tracks
+3. Build universe using CC, CCF, ML, or HMM method
+
+**Methods:**
+- **CC (Coverage Cutoff)**: Simple threshold-based
+- **CCF (Coverage Cutoff Flexible)**: Confidence intervals for boundaries
+- **ML (Maximum Likelihood)**: Probabilistic modeling of positions
+- **HMM (Hidden Markov Model)**: Complex state modeling
+
+**Reference:** See `references/consensus_peaks.md` for method comparison, parameters, and examples.
+
+### 5. Utilities: Supporting Tools
+
+Additional tools for caching, randomization, evaluation, and search.
+
+**Available utilities:**
+- **BBClient**: BED file caching for repeated access
+- **BEDshift**: Randomization preserving genomic context
+- **Evaluation**: Metrics for embedding quality (silhouette, Davies-Bouldin, etc.)
+- **Tokenization**: Region tokenization utilities (hard, soft, universe-based)
+- **Text2BedNN**: Neural search backends for genomic queries
+
+**Reference:** See `references/utilities.md` for detailed usage of each utility.
+
+## Common Workflows
+
+### Basic Region Embedding Pipeline
+
+```python
+from geniml.tokenization import hard_tokenization
+from geniml.region2vec import region2vec
+from geniml.evaluation import evaluate_embeddings
+
+# Step 1: Tokenize BED files
+hard_tokenization(
+    src_folder='bed_files/',
+    dst_folder='tokens/',
+    universe_file='universe.bed',
+    p_value_threshold=1e-9
+)
+
+# Step 2: Train Region2Vec
+region2vec(
+    token_folder='tokens/',
+    save_dir='model/',
+    num_shufflings=1000,
+    embedding_dim=100
+)
+
+# Step 3: Evaluate
+metrics = evaluate_embeddings(
+    embeddings_file='model/embeddings.npy',
+    labels_file='metadata.csv'
+)
+```
+
+### scATAC-seq Analysis Pipeline
+
+```python
+import scanpy as sc
+from geniml.scembed import ScEmbed
+from geniml.io import tokenize_cells
+
+# Step 1: Load data
+adata = sc.read_h5ad('scatac_data.h5ad')
+
+# Step 2: Tokenize cells
+tokenize_cells(
+    adata='scatac_data.h5ad',
+    universe_file='universe.bed',
+    output='tokens.parquet'
+)
+
+# Step 3: Train scEmbed
+model = ScEmbed(embedding_dim=100)
+model.train(dataset='tokens.parquet', epochs=100)
+
+# Step 4: Generate embeddings
+embeddings = model.encode(adata)
+adata.obsm['scembed_X'] = embeddings
+
+# Step 5: Cluster with scanpy
+sc.pp.neighbors(adata, use_rep='scembed_X')
+sc.tl.leiden(adata)
+sc.tl.umap(adata)
+```
+
+### Universe Building and Evaluation
+
+```bash
+# Generate coverage
+cat bed_files/*.bed > combined.bed
+uniwig -m 25 combined.bed chrom.sizes coverage/
+
+# Build universe with coverage cutoff
+geniml universe build cc \
+  --coverage-folder coverage/ \
+  --output-file universe.bed \
+  --cutoff 5 \
+  --merge 100 \
+  --filter-size 50
+
+# Evaluate universe quality
+geniml universe evaluate \
+  --universe universe.bed \
+  --coverage-folder coverage/ \
+  --bed-folder bed_files/
+```
+
+## CLI Reference
+
+Geniml provides command-line interfaces for major operations:
+
+```bash
+# Region2Vec training
+geniml region2vec --token-folder tokens/ --save-dir model/ --num-shuffle 1000
+
+# BEDspace preprocessing
+geniml bedspace preprocess --input regions/ --metadata labels.csv --universe universe.bed
+
+# BEDspace training
+geniml bedspace train --input preprocessed.txt --output model/ --dim 100
+
+# BEDspace search
+geniml bedspace search -t r2l -d distances.pkl -q query.bed -n 10
+
+# Universe building
+geniml universe build cc --coverage-folder coverage/ --output universe.bed --cutoff 5
+
+# BEDshift randomization
+geniml bedshift --input peaks.bed --genome hg38 --preserve-chrom --iterations 100
+```
+
+## When to Use Which Tool
+
+**Use Region2Vec when:**
+- Working with bulk genomic data (ChIP-seq, ATAC-seq, etc.)
+- Need unsupervised embeddings without metadata
+- Comparing region sets across experiments
+- Building features for downstream supervised learning
+
+**Use BEDspace when:**
+- Metadata labels available (cell types, tissues, conditions)
+- Need to query regions by metadata or vice versa
+- Want joint embedding space for regions and labels
+- Building searchable genomic databases
+
+**Use scEmbed when:**
+- Analyzing single-cell ATAC-seq data
+- Clustering cells by chromatin accessibility
+- Annotating cell types from scATAC-seq
+- Integration with scanpy is desired
+
+**Use Universe Building when:**
+- Need reference peak sets for tokenization
+- Combining multiple experiments into consensus
+- Want statistically rigorous region definitions
+- Building standard references for a project
+
+**Use Utilities when:**
+- Need to cache remote BED files (BBClient)
+- Generating null models for statistics (BEDshift)
+- Evaluating embedding quality (Evaluation)
+- Building search interfaces (Text2BedNN)
+
+## Best Practices
+
+### General Guidelines
+
+- **Universe quality is critical**: Invest time in building comprehensive, well-constructed universes
+- **Tokenization validation**: Check coverage (>80% ideal) before training
+- **Parameter tuning**: Experiment with embedding dimensions, learning rates, and training epochs
+- **Evaluation**: Always validate embeddings with multiple metrics and visualizations
+- **Documentation**: Record parameters and random seeds for reproducibility
+
+### Performance Considerations
+
+- **Pre-tokenization**: For scEmbed, always pre-tokenize cells for faster training
+- **Memory management**: Large datasets may require batch processing or downsampling
+- **Computational resources**: ML/HMM universe methods are computationally intensive
+- **Model caching**: Use BBClient to avoid repeated downloads
+
+### Integration Patterns
+
+- **With scanpy**: scEmbed embeddings integrate seamlessly as `adata.obsm` entries
+- **With BEDbase**: Use BBClient for accessing remote BED repositories
+- **With Hugging Face**: Export trained models for sharing and reproducibility
+- **With R**: Use reticulate for R integration (see utilities reference)
+
+## Related Projects
+
+Geniml is part of the BEDbase ecosystem:
+
+- **BEDbase**: Unified platform for genomic regions
+- **BEDboss**: Processing pipeline for BED files
+- **Gtars**: Genomic tools and utilities
+- **BBClient**: Client for BEDbase repositories
+
+## Additional Resources
+
+- **Documentation**: https://docs.bedbase.org/geniml/
+- **GitHub**: https://github.com/databio/geniml
+- **Pre-trained models**: Available on Hugging Face (databio organization)
+- **Publications**: Cited in documentation for methodological details
+
+## Troubleshooting
+
+**"Tokenization coverage too low":**
+- Check universe quality and completeness
+- Adjust p-value threshold (try 1e-6 instead of 1e-9)
+- Ensure universe matches genome assembly
+
+**"Training not converging":**
+- Adjust learning rate (try 0.01-0.05 range)
+- Increase training epochs
+- Check data quality and preprocessing
+
+**"Out of memory errors":**
+- Reduce batch size for scEmbed
+- Process data in chunks
+- Use pre-tokenization for single-cell data
+
+**"StarSpace not found" (BEDspace):**
+- Install StarSpace separately: https://github.com/facebookresearch/StarSpace
+- Set `--path-to-starspace` parameter correctly
+
+For detailed troubleshooting and method-specific issues, consult the appropriate reference file.

package/data/skills/genome-compare/SKILL.md

@@ -0,0 +1,127 @@
+---
+name: genome-compare
+description: Compare your genome to George Church (PGP-1) and estimate ancestry composition via IBS and EM admixture
+version: 0.1.0
+author: Manuel Corpas
+license: MIT
+tags: [genome-comparison, IBS, ancestry, PGP, admixture]
+metadata:
+  openclaw:
+    requires:
+      bins:
+        - python3
+      env: []
+      config: []
+    always: false
+    emoji: "🧬"
+    homepage: https://github.com/ClawBio/ClawBio
+    os: [macos, linux]
+    install: []
+    trigger_keywords:
+      - genome comparison
+      - IBS
+      - identity by state
+      - George Church
+      - Corpasome
+      - pairwise
+---
+
+# 🧬 Genome Comparator
+
+You are the **Genome Comparator**, a specialised ClawBio skill for pairwise genome comparison and ancestry estimation.
+
+## Why This Exists
+
+- **Without it**: Comparing two genomes requires PLINK, custom scripts, and ancestry reference panels — hours of bioinformatics setup
+- **With it**: Upload a 23andMe file and instantly see IBS similarity to George Church, per-chromosome breakdown, and ancestry composition
+- **Why ClawBio**: Uses a bundled PGP-1 reference genome (CC0 public domain) and an EM admixture algorithm calibrated to continental ancestry-informative markers
+
+## Core Capabilities
+
+1. **Identity By State (IBS)**: Compare a user's genome against George Church's public 23andMe data (PGP-1, hu43860C). Report SNP overlap, identity, and relationship context.
+2. **Ancestry Composition**: Estimate continental ancestry proportions (African, European, East Asian, South Asian, Americas) from ancestry-informative markers using an EM admixture algorithm.
+3. **Chromosome Breakdown**: Show per-chromosome IBS scores and overlap counts.
+
+## Input Formats
+
+| Format | Extension | Required Fields | Example |
+|--------|-----------|-----------------|---------|
+| 23andMe raw data | `.txt`, `.txt.gz` | rsid, chromosome, position, genotype | `data/manuel_corpas_23andme.txt.gz` |
+
+## Reference Genome
+
+**George Church** (hu43860C) — the first participant in the [Personal Genome Project](https://pgp.med.harvard.edu/). Professor of Genetics at Harvard Medical School. His 23andMe data (569,226 SNPs, CC0 public domain) is bundled in `data/george_church_23andme.txt.gz`.
+
+## Workflow
+
+1. **Parse**: Read user's 23andMe file and George Church reference (both support `.txt.gz`)
+2. **Overlap**: Find shared SNP positions between the two genomes
+3. **IBS**: Calculate identity-by-state score across all overlapping loci
+4. **Ancestry**: Run EM admixture algorithm on ancestry-informative markers
+5. **Visualise**: Generate per-chromosome IBS bar chart, ancestry pie, IBS context gauge, ancestry comparison
+6. **Report**: Write `report.md` with summary, IBS analysis, ancestry composition, and methods
+
+## CLI Reference
+
+```bash
+# Demo: Manuel Corpas vs George Church
+python skills/genome-compare/genome_compare.py --demo --output results/
+
+# Your own data vs George Church
+python skills/genome-compare/genome_compare.py --input your_23andme.txt --output results/
+
+# Via ClawBio runner
+python clawbio.py run compare --demo
+python clawbio.py run compare --input <file> --output <dir>
+```
+
+## Demo
+
+```bash
+python clawbio.py run compare --demo
+```
+
+Expected output: A report comparing Manuel Corpas (PGP-UK uk6D0CFA) vs George Church (PGP-1 hu43860C). IBS score ~0.74 (consistent with two unrelated Europeans). Ancestry estimates for both individuals. Four figures generated.
+
+## Output Structure
+
+```
+output_directory/
+├── report.md                       # Full comparison report
+├── result.json                     # Machine-readable IBS and ancestry data
+├── figures/
+│   ├── chromosome_ibs.png          # Per-chromosome IBS bar chart
+│   ├── ancestry_pie.png            # Ancestry composition pie chart
+│   ├── ibs_context.png             # IBS score on relationship spectrum gauge
+│   └── ancestry_comparison.png     # Side-by-side ancestry comparison
+└── reproducibility/
+    └── commands.sh                 # Exact command to reproduce
+```
+
+## Dependencies
+
+**Required**:
+- Python 3.10+
+- `numpy` >= 1.24
+- `matplotlib` >= 3.7
+
+## Safety
+
+- All processing is local. Genetic data never leaves the machine.
+- Ancestry estimation is approximate — for clinical-grade results, use ADMIXTURE or professional services.
+- ClawBio is a research and educational tool. It is not a medical device.
+
+## Integration with Bio Orchestrator
+
+**Trigger conditions** — the orchestrator routes here when:
+- User asks to compare genomes, mentions IBS, George Church, or Corpasome
+- User provides a 23andMe file and asks "how similar am I to..."
+
+**Chaining partners**:
+- `claw-ancestry-pca`: More detailed ancestry analysis with SGDP reference panel
+- `profile-report`: Genome comparison results feed into the unified genomic profile
+
+## Citations
+
+- Church GM. The Personal Genome Project. Mol Syst Biol. 2005;1:2005.0030.
+- Corpas M. Crowdsourcing the Corpasome. Source Code Biol Med. 2013;8:13.