@bgicli/bgicli 2.1.1 → 2.2.0

package/data/skills/foldseek/SKILL.md

@@ -0,0 +1,179 @@
+---
+name: foldseek
+description: >
+  Structure similarity search with Foldseek. Use this skill when:
+  (1) Finding similar structures in PDB/AFDB databases,
+  (2) Structural homology search,
+  (3) Database queries by 3D structure,
+  (4) Finding remote homologs not detected by sequence,
+  (5) Clustering structures by similarity.
+
+  For sequence similarity, use uniprot BLAST.
+  For structure prediction, use chai or boltz.
+license: MIT
+category: utilities
+tags: [search, structure, database, similarity]
+---
+
+# Foldseek Structure Search
+
+## Prerequisites
+
+| Requirement | Minimum | Recommended |
+|-------------|---------|-------------|
+| Python | 3.8+ | 3.10 |
+| RAM | 8GB | 16GB |
+| Disk | 10GB | 50GB (for local databases) |
+
+## How to run
+
+**Note**: Foldseek can run locally or via web server. No GPU required.
+
+### Option 1: Web Server (Quick; rate-limited, use sparingly)
+```bash
+# Upload structure to web server
+curl -X POST "https://search.foldseek.com/api/ticket" \
+  -F "q=@query.pdb" \
+  -F "database[]=afdb50" \
+  -F "database[]=pdb100"
+```
+
+### Option 2: Local installation
+```bash
+# Install Foldseek
+conda install -c conda-forge -c bioconda foldseek
+
+# Search PDB
+foldseek easy-search query.pdb /path/to/pdb100 results.m8 tmp/
+
+# Search AlphaFold DB
+foldseek easy-search query.pdb /path/to/afdb50 results.m8 tmp/
+```
+
+### Option 3: Python API
+```python
+import subprocess
+import pandas as pd
+
+def foldseek_search(query_pdb, database, output="results.m8"):
+    """Run Foldseek search."""
+    subprocess.run([
+        "foldseek", "easy-search",
+        query_pdb, database, output, "tmp/",
+        "--format-output", "query,target,pident,alnlen,evalue,bits"
+    ])
+    return pd.read_csv(output, sep="\t",
+                       names=["query", "target", "pident", "alnlen", "evalue", "bits"])
+```
+
+## Key parameters
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `--min-seq-id` | 0.0 | Minimum sequence identity |
+| `-e` | 0.001 | E-value threshold |
+| `--alignment-type` | 2 | 0=3Di, 1=TM, 2=3Di+AA |
+| `--max-seqs` | 300 | Max hits to pass through prefilter; reducing this affects sensitivity |
+
+## Databases
+
+| Database | Description | Size |
+|----------|-------------|------|
+| `pdb100` | PDB clustered at 100% | ~200K structures |
+| `afdb50` | AlphaFold DB at 50% | ~67M structures |
+| `swissprot` | SwissProt structures | ~500K structures |
+| `cath50` | CATH domains | ~50K domains |
+
+## Output format
+
+```
+# results.m8 (tabular)
+query   target          pident  alnlen  evalue  bits
+query   1abc_A          85.2    120     1e-45   180.5
+query   2def_B          72.1    115     1e-32   145.2
+```
+
+## Sample output
+
+### Successful run
+```
+$ foldseek easy-search query.pdb pdb100 results.m8 tmp/
+[INFO] Loading database: pdb100 (194,527 entries)
+[INFO] Searching...
+[INFO] Found 127 hits
+
+Top 5 hits:
+1. 1abc_A - 85.2% identity, E=1e-45
+2. 2def_B - 72.1% identity, E=1e-32
+3. 3ghi_C - 68.5% identity, E=1e-28
+4. 4jkl_A - 55.3% identity, E=1e-18
+5. 5mno_B - 42.1% identity, E=1e-10
+```
+
+## Decision tree
+
+```
+Should I use Foldseek?
+│
+├─ What are you searching?
+│  ├─ By 3D structure → Foldseek ✓
+│  ├─ By sequence → Use BLAST (uniprot skill)
+│  └─ Both → Run both, compare results
+│
+└─ What do you need?
+   ├─ Find structural homologs → Foldseek ✓
+   ├─ Remote homolog detection → Foldseek ✓
+   ├─ Structural clustering → Foldseek ✓
+   └─ Functional annotation → Cross-reference with UniProt
+```
+
+## Common use cases
+
+### Find similar designs
+```bash
+# Compare your design to PDB
+foldseek easy-search design.pdb pdb100 similar_natural.m8 tmp/
+```
+
+### Novelty check
+```bash
+# Ensure design is novel (low similarity to known)
+foldseek easy-search design.pdb afdb50 novelty.m8 tmp/
+
+# Novel if: top hit identity < 30%
+```
+
+### Scaffold search
+```bash
+# Find scaffolds for motif grafting
+foldseek easy-search motif.pdb pdb100 scaffolds.m8 tmp/ \
+  --min-seq-id 0.0 -e 10
+```
+
+---
+
+## Verify
+
+```bash
+wc -l results.m8  # Number of hits
+```
+
+---
+
+## Troubleshooting
+
+**No hits**: Lower e-value threshold, try larger database
+**Too many hits**: Increase min-seq-id threshold
+**Slow search**: Use smaller database
+
+### Error interpretation
+
+| Error | Cause | Fix |
+|-------|-------|-----|
+| `Database not found` | Wrong path | Check database location |
+| `Invalid PDB` | Malformed structure | Validate PDB format |
+| `Out of memory` | Large database | Use more RAM or web server |
+
+---
+
+**Next**: Download hits with `pdb` skill → use for scaffold design.

package/data/skills/galaxy-bridge/SKILL.md

@@ -0,0 +1,215 @@
+---
+name: galaxy-bridge
+description: "Galaxy tool discovery, intelligent recommendation, and execution — 8,000+ bioinformatics tools from usegalaxy.org with multi-signal scoring and workflow suggestions"
+version: 0.2.0
+author: Manuel Corpas
+license: MIT
+tags: [galaxy, bioinformatics, tool-discovery, workflows, NGS, genomics, proteomics, metagenomics]
+metadata:
+  openclaw:
+    requires:
+      bins:
+        - python3
+      env:
+        - GALAXY_URL
+        - GALAXY_API_KEY
+      config: []
+    always: false
+    emoji: "🌌"
+    homepage: https://github.com/ClawBio/ClawBio
+    os: [macos, linux]
+    install:
+      - kind: pip
+        package: bioblend
+        bins: []
+    trigger_keywords:
+      - galaxy
+      - usegalaxy
+      - tool shed
+      - bioblend
+      - run on galaxy
+      - galaxy tool
+      - galaxy workflow
+      - NGS pipeline
+---
+
+# Galaxy Bridge
+
+**ClawBio's gateway to the Galaxy ecosystem — 1,770+ production bioinformatics tools, discoverable and executable through natural language.**
+
+## Why This Exists
+
+Galaxy (usegalaxy.org) hosts the world's largest collection of curated bioinformatics tools — 1,770+ on the main server alone, covering everything from FASTQ QC to metagenomics to protein structure prediction. But discovering the right tool requires knowing exact tool IDs, navigating nested ToolShed categories, and understanding parameter schemas.
+
+Galaxy Bridge makes these tools **agent-accessible**: search by natural language, execute via CLI, and chain Galaxy tools with ClawBio's local skills for cross-platform workflows that neither system can do alone.
+
+## Core Capabilities
+
+1. **Intelligent tool recommendation** — describe a task in plain English; multi-signal scoring across 7 dimensions returns the best Galaxy tool with explanations
+2. **Workflow suggestions** — 8 pre-defined pipeline templates (RNA-seq DE, metagenomics, variant calling, WES germline, ChIP-seq, nanopore, genome assembly, variant annotation)
+3. **Input format awareness** — provide your file extension (.fastq, .bam, .vcf) for format-aware recommendations
+4. **Version deduplication** — 8,182 catalog entries collapse to ~2,300 unique tools; latest version preferred, version count as maturity signal
+5. **EDAM ontology resolution** — 108 EDAM topic/operation IDs resolved to human-readable labels for richer matching
+6. **Natural language search** — keyword-based search across 8,000+ Galaxy tools by name, description, section, and EDAM terms
+7. **Remote execution** — run Galaxy tools on usegalaxy.org via BioBlend API
+8. **Category browsing** — explore 86 ToolShed categories with tool counts
+9. **Tool detail inspection** — view inputs, outputs, and parameter schemas
+10. **Offline demo mode** — FastQC demo with pre-cached results (no API key needed)
+11. **Cross-platform chaining** — Galaxy VEP → ClawBio PharmGx, Galaxy Kraken2 → ClawBio metagenomics
+
+## Input Formats
+
+| Format | Extension | Required Fields | Example |
+|--------|-----------|----------------|---------|
+| FASTQ | `.fq`, `.fastq`, `.fq.gz` | Sequence reads | Illumina paired-end reads |
+| VCF | `.vcf`, `.vcf.gz` | Variant calls | Annotated VCF for VEP |
+| BAM | `.bam` | Aligned reads | BWA-MEM2 output |
+| FASTA | `.fa`, `.fasta` | Sequences | Reference genome |
+| Tabular | `.tsv`, `.csv` | Varies by tool | Gene expression matrix |
+
+## Workflow
+
+1. **Search** — User describes what they need → bridge searches local catalog + Galaxy API
+2. **Select** — Ranked results with descriptions, versions, and categories
+3. **Configure** — Show tool inputs/outputs schema; user provides files and parameters
+4. **Execute** — Upload input to Galaxy, run tool, poll for completion
+5. **Retrieve** — Download outputs to local directory
+6. **Bundle** — Generate reproducibility package (commands.sh, environment.yml, checksums)
+
+## CLI Reference
+
+```bash
+# Intelligent tool recommendation (new in v0.2.0)
+python galaxy_bridge.py --recommend "quality control on my sequencing reads"
+python galaxy_bridge.py --recommend "classify microbial species" --format .fastq
+python galaxy_bridge.py --recommend "call variants" --format .bam
+python galaxy_bridge.py --recommend "annotate variants from WES" --format .vcf
+
+# Workflow / pipeline suggestions (new in v0.2.0)
+python galaxy_bridge.py --workflow "RNA-seq differential expression"
+python galaxy_bridge.py --workflow "metagenomics"
+python galaxy_bridge.py --workflow "whole exome sequencing"
+
+# Search for tools by keyword
+python galaxy_bridge.py --search "metagenomics profiling"
+python galaxy_bridge.py --search "variant annotation"
+python galaxy_bridge.py --search "RNA-seq differential expression"
+
+# Browse Galaxy ToolShed categories
+python galaxy_bridge.py --list-categories
+
+# View tool details (inputs, outputs, parameters)
+python galaxy_bridge.py --tool-details toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.74+galaxy1
+
+# Run a tool on Galaxy (requires GALAXY_API_KEY)
+python galaxy_bridge.py --run fastqc --input reads.fq.gz --output /tmp/qc_results
+
+# Demo mode (works offline, no API key needed)
+python galaxy_bridge.py --demo
+```
+
+## Recommendation Engine
+
+The `--recommend` flag uses **multi-signal scoring** across 7 dimensions to rank tools:
+
+| Signal | Max Points | Description |
+|--------|-----------|-------------|
+| Section match | 30 | Tool's Galaxy category matches the detected task |
+| Preferred tool | 20 | Tool is a known best-in-class for the task |
+| Exact name match | 15 | Tool name appears in the query |
+| Keyword match | 15 | Query words found in tool name/description |
+| EDAM ontology | 10 | EDAM topic/operation IDs match the task |
+| Format compatibility | 10 | Tool accepts the specified input format |
+| Version maturity | 5 | Tools with more versions score higher (log scale) |
+
+**15 task categories** are recognised: Quality Control, Read Mapping, Variant Calling, Variant Annotation, WES/WGS, RNA-seq, Metagenomics, Genome Assembly, Genome Annotation, Phylogenetics, ChIP-seq, Single-cell, Proteomics, Nanopore, BAM Processing.
+
+**8 workflow templates**: WES Germline, WES Annotation, RNA-seq DE, Metagenomics Profiling, Variant Calling, ChIP-seq, Nanopore Assembly, Genome Assembly.
+
+## Demo
+
+Running `--demo` executes a simulated FastQC analysis using pre-cached results:
+
+```
+$ python galaxy_bridge.py --demo
+
+Galaxy Bridge — Demo Mode (offline)
+====================================
+Tool: FastQC v0.74+galaxy1
+Input: demo/demo_reads.fq (bundled synthetic FASTQ, 1000 reads)
+Output: demo/fastqc_demo_output.html
+
+Result: PASS — Per base sequence quality ✓
+        PASS — Per sequence quality scores ✓
+        WARN — Per base sequence content (normal for Illumina)
+        PASS — Sequence length distribution ✓
+
+Reproducibility bundle written to demo/reproducibility/
+```
+
+## Galaxy Tool Categories
+
+The bridge indexes tools across all 56 Galaxy ToolShed categories, including:
+
+- **Sequence Analysis** (~30 tools): FastQC, Trimmomatic, Cutadapt, fastp
+- **Metagenomics** (~25 tools): Kraken2, MetaPhlAn, HUMAnN, QIIME2
+- **Variant Analysis** (~25 tools): VEP, SnpSift, BCFtools, FreeBayes
+- **RNA** (~20 tools): HISAT2, StringTie, featureCounts, DESeq2
+- **Proteomics** (~15 tools): MaxQuant, SearchGUI, PeptideShaker
+- **Phylogenetics** (~15 tools): IQ-TREE, RAxML, MAFFT, MUSCLE
+- **Genome Annotation** (~15 tools): Prokka, Augustus, MAKER
+- **Assembly** (~15 tools): SPAdes, Flye, Unicycler, MEGAHIT
+- **Single Cell** (~10 tools): Scanpy, CellRanger, Seurat
+- **ChIP-seq/Epigenetics** (~10 tools): MACS2, deepTools, DiffBind
+- **GWAS** (~10 tools): PLINK, REGENIE, BOLT-LMM
+- **Nanopore** (~10 tools): NanoPlot, Medaka, minimap2
+
+## Output Structure
+
+```
+output_dir/
+├── report.md              # Analysis summary with methods and results
+├── result.json            # Machine-readable: tool ID, version, parameters, output paths
+├── galaxy_outputs/        # Raw outputs downloaded from Galaxy
+│   ├── fastqc_report.html
+│   └── ...
+└── reproducibility/
+    ├── commands.sh        # Galaxy API calls to reproduce
+    ├── environment.yml    # Tool versions and Galaxy server info
+    └── checksums.sha256   # SHA-256 of all inputs and outputs
+```
+
+## Dependencies
+
+**Required:**
+- Python 3.9+
+- bioblend (Galaxy Python SDK)
+
+**Optional (for execution):**
+- `GALAXY_URL` environment variable (default: `https://usegalaxy.org`)
+- `GALAXY_API_KEY` environment variable (register at usegalaxy.org)
+
+## Safety
+
+- **Local-first search**: Tool discovery uses the bundled `galaxy_catalog.json` — no API calls needed
+- **API key optional**: Demo mode and search work without credentials
+- **No data retention**: Uploaded files are deleted from Galaxy after output retrieval
+- **Reproducibility**: Every execution generates a full provenance bundle
+- **Disclaimer**: ClawBio is a research and educational tool. It is not a medical device and does not provide clinical diagnoses. Consult a healthcare professional before making any medical decisions.
+
+## Integration with Bio Orchestrator
+
+**Triggers when**: User mentions "galaxy", "usegalaxy", "tool shed", "run on galaxy", "NGS pipeline", or references a Galaxy tool ID.
+
+**Chaining partners**:
+- `pharmgx-reporter` — Galaxy VEP annotates variants → PharmGx generates dosage report
+- `claw-metagenomics` — Galaxy Kraken2 → ClawBio metagenomics profiling
+- `equity-scorer` — Galaxy VCF processing → HEIM equity scoring
+- `vcf-annotator` — Galaxy VEP/SnpSift ↔ ClawBio annotation
+
+## Citations
+
+- [Galaxy Project](https://galaxyproject.org/) — Afgan et al. (2018) Nucleic Acids Research
+- [BioBlend](https://bioblend.readthedocs.io/) — Sloggett et al. (2013) Bioinformatics
+- [usegalaxy.org](https://usegalaxy.org/) — Main Galaxy public server
+- [Galaxy ToolShed](https://toolshed.g2.bx.psu.edu/) — Community tool repository

package/data/skills/gene-database/SKILL.md

@@ -0,0 +1,173 @@
+---
+name: gene-database
+description: "Query NCBI Gene via E-utilities/Datasets API. Search by symbol/ID, retrieve gene info (RefSeqs, GO, locations, phenotypes), batch lookups, for gene annotation and functional analysis."
+---
+
+# Gene Database
+
+## Overview
+
+NCBI Gene is a comprehensive database integrating gene information from diverse species. It provides nomenclature, reference sequences (RefSeqs), chromosomal maps, biological pathways, genetic variations, phenotypes, and cross-references to global genomic resources.
+
+## When to Use This Skill
+
+This skill should be used when working with gene data including searching by gene symbol or ID, retrieving gene sequences and metadata, analyzing gene functions and pathways, or performing batch gene lookups.
+
+## Quick Start
+
+NCBI provides two main APIs for gene data access:
+
+1. **E-utilities** (Traditional): Full-featured API for all Entrez databases with flexible querying
+2. **NCBI Datasets API** (Newer): Optimized for gene data retrieval with simplified workflows
+
+Choose E-utilities for complex queries and cross-database searches. Choose Datasets API for straightforward gene data retrieval with metadata and sequences in a single request.
+
+## Common Workflows
+
+### Search Genes by Symbol or Name
+
+To search for genes by symbol or name across organisms:
+
+1. Use the `scripts/query_gene.py` script with E-utilities ESearch
+2. Specify the gene symbol and organism (e.g., "BRCA1 in human")
+3. The script returns matching Gene IDs
+
+Example query patterns:
+- Gene symbol: `insulin[gene name] AND human[organism]`
+- Gene with disease: `dystrophin[gene name] AND muscular dystrophy[disease]`
+- Chromosome location: `human[organism] AND 17q21[chromosome]`
+
+### Retrieve Gene Information by ID
+
+To fetch detailed information for known Gene IDs:
+
+1. Use `scripts/fetch_gene_data.py` with the Datasets API for comprehensive data
+2. Alternatively, use `scripts/query_gene.py` with E-utilities EFetch for specific formats
+3. Specify desired output format (JSON, XML, or text)
+
+The Datasets API returns:
+- Gene nomenclature and aliases
+- Reference sequences (RefSeqs) for transcripts and proteins
+- Chromosomal location and mapping
+- Gene Ontology (GO) annotations
+- Associated publications
+
+### Batch Gene Lookups
+
+For multiple genes simultaneously:
+
+1. Use `scripts/batch_gene_lookup.py` for efficient batch processing
+2. Provide a list of gene symbols or IDs
+3. Specify the organism for symbol-based queries
+4. The script handles rate limiting automatically (10 requests/second with API key)
+
+This workflow is useful for:
+- Validating gene lists
+- Retrieving metadata for gene panels
+- Cross-referencing gene identifiers
+- Building gene annotation tables
+
+### Search by Biological Context
+
+To find genes associated with specific biological functions or phenotypes:
+
+1. Use E-utilities with Gene Ontology (GO) terms or phenotype keywords
+2. Query by pathway names or disease associations
+3. Filter by organism, chromosome, or other attributes
+
+Example searches:
+- By GO term: `GO:0006915[biological process]` (apoptosis)
+- By phenotype: `diabetes[phenotype] AND mouse[organism]`
+- By pathway: `insulin signaling pathway[pathway]`
+
+### API Access Patterns
+
+**Rate Limits:**
+- Without API key: 3 requests/second for E-utilities, 5 requests/second for Datasets API
+- With API key: 10 requests/second for both APIs
+
+**Authentication:**
+Register for a free NCBI API key at https://www.ncbi.nlm.nih.gov/account/ to increase rate limits.
+
+**Error Handling:**
+Both APIs return standard HTTP status codes. Common errors include:
+- 400: Malformed query or invalid parameters
+- 429: Rate limit exceeded
+- 404: Gene ID not found
+
+Retry failed requests with exponential backoff.
+
+## Script Usage
+
+### query_gene.py
+
+Query NCBI Gene using E-utilities (ESearch, ESummary, EFetch).
+
+```bash
+python scripts/query_gene.py --search "BRCA1" --organism "human"
+python scripts/query_gene.py --id 672 --format json
+python scripts/query_gene.py --search "insulin[gene] AND diabetes[disease]"
+```
+
+### fetch_gene_data.py
+
+Fetch comprehensive gene data using NCBI Datasets API.
+
+```bash
+python scripts/fetch_gene_data.py --gene-id 672
+python scripts/fetch_gene_data.py --symbol BRCA1 --taxon human
+python scripts/fetch_gene_data.py --symbol TP53 --taxon "Homo sapiens" --output json
+```
+
+### batch_gene_lookup.py
+
+Process multiple gene queries efficiently.
+
+```bash
+python scripts/batch_gene_lookup.py --file gene_list.txt --organism human
+python scripts/batch_gene_lookup.py --ids 672,7157,5594 --output results.json
+```
+
+## API References
+
+For detailed API documentation including endpoints, parameters, response formats, and examples, refer to:
+
+- `references/api_reference.md` - Comprehensive API documentation for E-utilities and Datasets API
+- `references/common_workflows.md` - Additional examples and use case patterns
+
+Search these references when needing specific API endpoint details, parameter options, or response structure information.
+
+## Data Formats
+
+NCBI Gene data can be retrieved in multiple formats:
+
+- **JSON**: Structured data ideal for programmatic processing
+- **XML**: Detailed hierarchical format with full metadata
+- **GenBank**: Sequence data with annotations
+- **FASTA**: Sequence data only
+- **Text**: Human-readable summaries
+
+Choose JSON for modern applications, XML for legacy systems requiring detailed metadata, and FASTA for sequence analysis workflows.
+
+## Best Practices
+
+1. **Always specify organism** when searching by gene symbol to avoid ambiguity
+2. **Use Gene IDs** for precise lookups when available
+3. **Batch requests** when working with multiple genes to minimize API calls
+4. **Cache results** locally to reduce redundant queries
+5. **Include API key** in scripts for higher rate limits
+6. **Handle errors gracefully** with retry logic for transient failures
+7. **Validate gene symbols** before batch processing to catch typos
+
+## Resources
+
+This skill includes:
+
+### scripts/
+- `query_gene.py` - Query genes using E-utilities (ESearch, ESummary, EFetch)
+- `fetch_gene_data.py` - Fetch gene data using NCBI Datasets API
+- `batch_gene_lookup.py` - Handle multiple gene queries efficiently
+
+### references/
+- `api_reference.md` - Detailed API documentation for both E-utilities and Datasets API
+- `common_workflows.md` - Examples of common gene queries and use cases