@bgicli/bgicli 2.1.1 → 2.2.0

package/data/skills/literature-search/SKILL.md

@@ -0,0 +1,214 @@
+---
+name: literature-search
+description: Comprehensive scientific literature search across PubMed, arXiv, bioRxiv, medRxiv. Natural language queries powered by Valyu semantic search.
+keywords:
+  - literature-search
+  - scientific-literature
+  - multi-source-search
+  - comprehensive-search
+  - research-aggregation
+  - semantic-search
+license: MIT
+---
+
+
+# Literature Search
+
+Search across all major scientific literature databases (PubMed, arXiv, bioRxiv, medRxiv) simultaneously using natural language queries powered by Valyu's semantic search API.
+
+## Why This Skill is Powerful
+
+- **No API Parameter Parsing**: Just pass natural language queries directly - no need to construct complex search parameters
+- **Semantic Search**: Understands the meaning of your query, not just keyword matching
+- **Full-Text Access**: Returns complete article content, not just abstracts
+- **Image Links**: Includes figures and images from papers
+- **Comprehensive Coverage**: Search across PubMed, arXiv, bioRxiv, and medRxiv simultaneously
+- **Unified Results**: Get results from all sources in a single query
+
+## Requirements
+
+1. Node.js 18+ (uses built-in fetch)
+2. Valyu API key from https://platform.valyu.ai ($10 free credits)
+
+## CRITICAL: Script Path Resolution
+
+The `scripts/search` commands in this documentation are relative to this skill's installation directory.
+
+Before running any command, locate the script using:
+
+```bash
+LITERATURE_SCRIPT=$(find ~/.claude/plugins/cache -name "search" -path "*/literature-search/*/scripts/*" -type f 2>/dev/null | head -1)
+```
+
+Then use the full path for all commands:
+```bash
+$LITERATURE_SCRIPT "CRISPR gene editing advances" 15
+```
+
+## API Key Setup Flow
+
+When you run a search and receive `"setup_required": true`, follow this flow:
+
+1. **Ask the user for their API key:**
+   "To search scientific literature, I need your Valyu API key. Get one free ($10 credits) at https://platform.valyu.ai"
+
+2. **Once the user provides the key, run:**
+   ```bash
+   scripts/search setup <api-key>
+   ```
+
+3. **Retry the original search.**
+
+## When to Use This Skill
+
+- Comprehensive literature reviews across all domains
+- Finding all relevant research on a topic
+- Cross-domain scientific discovery
+- Combining biomedical, physics, and preprint literature
+- Emerging research across disciplines
+## Output Format
+
+```json
+{
+  "success": true,
+  "type": "literature_search",
+  "query": "CRISPR gene editing advances",
+  "result_count": 15,
+  "results": [
+    {
+      "title": "Article Title",
+      "url": "https://...",
+      "content": "Full article text with figures...",
+      "source": "pubmed|arxiv|biorxiv|medrxiv",
+      "relevance_score": 0.95,
+      "images": ["https://example.com/figure1.jpg"]
+    }
+  ],
+  "cost": 0.025
+}
+```
+
+## Processing Results
+
+### With jq
+
+```bash
+# Get article titles
+scripts/search "query" 20 | jq -r '.results[].title'
+
+# Get URLs
+scripts/search "query" 20 | jq -r '.results[].url'
+
+# Extract full content
+scripts/search "query" 20 | jq -r '.results[].content'
+
+# Filter by source
+scripts/search "query" 20 | jq -r '.results[] | select(.source == "arxiv") | .title'
+```
+
+## Common Use Cases
+
+### Comprehensive Literature Review
+
+```bash
+# Search across all sources for thorough review
+scripts/search "mechanisms of cellular senescence" 100
+```
+
+### Cross-Disciplinary Research
+
+```bash
+# Find papers spanning multiple fields
+scripts/search "quantum computing applications in drug discovery" 50
+```
+
+### Recent Developments
+
+```bash
+# Get latest preprints and publications
+scripts/search "foundation models for protein folding" 30
+```
+
+### Medical Research
+
+```bash
+# Search biomedical literature comprehensively
+scripts/search "immunotherapy checkpoint inhibitors resistance" 40
+```
+
+
+## Error Handling
+
+All commands return JSON with `success` field:
+
+```json
+{
+  "success": false,
+  "error": "Error message"
+}
+```
+
+Exit codes:
+- `0` - Success
+- `1` - Error (check JSON for details)
+
+## API Endpoint
+
+- Base URL: `https://api.valyu.ai/v1`
+- Endpoint: `/search`
+- Authentication: X-API-Key header
+
+## Architecture
+
+```
+scripts/
+├── search          # Bash wrapper
+└── search.mjs      # Node.js CLI
+```
+
+Direct API calls using Node.js built-in `fetch()`, zero external dependencies.
+
+## Adding to Your Project
+
+If you're building an AI project and want to integrate Literature Search directly into your application, use the Valyu SDK:
+
+### Python Integration
+
+```python
+from valyu import Valyu
+
+client = Valyu(api_key="your-api-key")
+
+response = client.search(
+    query="your search query here",
+    included_sources=["valyu/valyu-pubmed", "valyu/valyu-arxiv", "valyu/valyu-biorxiv", "valyu/valyu-medrxiv"],
+    max_results=20
+)
+
+for result in response["results"]:
+    print(f"Title: {result['title']}")
+    print(f"URL: {result['url']}")
+    print(f"Content: {result['content'][:500]}...")
+```
+
+### TypeScript Integration
+
+```typescript
+import { Valyu } from "valyu-js";
+
+const client = new Valyu("your-api-key");
+
+const response = await client.search({
+  query: "your search query here",
+  includedSources: ["valyu/valyu-pubmed", "valyu/valyu-arxiv", "valyu/valyu-biorxiv", "valyu/valyu-medrxiv"],
+  maxResults: 20
+});
+
+response.results.forEach((result) => {
+  console.log(`Title: ${result.title}`);
+  console.log(`URL: ${result.url}`);
+  console.log(`Content: ${result.content.substring(0, 500)}...`);
+});
+```
+
+See the [Valyu docs](https://docs.valyu.ai) for full integration examples and SDK reference.

package/data/skills/lobster-bioinformatics/SKILL.md

@@ -0,0 +1,305 @@
+---
+name: lobster-bioinformatics
+description: Run bioinformatics analyses using Lobster AI - single-cell RNA-seq, bulk RNA-seq, literature mining, dataset discovery, quality control, and visualization. Use when analyzing genomics data, searching for papers/datasets, or working with H5AD, CSV, GEO/SRA accessions, or biological data. Requires lobster-ai package installed.
+---
+
+# Lobster Bioinformatics Agent
+
+Lobster AI is a bioinformatics platform that combines specialized AI agents with open-source tools to analyze multi-omics data through natural language.
+
+## When to use this Skill
+
+Use Lobster when the user asks to:
+- Analyze single-cell RNA-seq data (QC, clustering, annotation, markers)
+- Perform bulk RNA-seq analysis (differential expression, complex designs)
+- Search scientific literature (PubMed, PMC, full-text retrieval)
+- Discover datasets (GEO, SRA, ENA (free) and PRIDE, MASSive (cloud))
+- Run quality control on biological data
+- Generate bioinformatics visualizations (UMAP, volcano plots, heatmaps)
+- Download and process biological datasets
+- Work with H5AD, CSV, Excel, 10X formats
+- Extract methods or metadata from papers
+
+## Requirements
+
+Lobster must be installed and configured:
+
+```bash
+# Check if Lobster is installed
+which lobster
+
+# If not installed:
+uv pip install lobster-ai
+lobster init --help #to see non-interactive
+```
+
+Lobster requires an LLM provider (Ollama, Anthropic, or AWS Bedrock).
+
+## Pre-flight check (IMPORTANT)
+
+**Before running any analysis, always verify Lobster is ready:**
+```bash
+lobster config-test --json
+```
+
+Returns structured JSON:
+```json
+{
+  "valid": true,
+  "env_file": "/path/to/.env",
+  "checks": {
+    "llm_provider": {"status": "pass", "provider": "bedrock", "message": "Connected"},
+    "ncbi_api": {"status": "pass", "has_key": true, "message": "Connected"},
+    "workspace": {"status": "pass", "path": "/path/to/workspace", "message": "Writable"}
+  }
+}
+```
+
+This command validates:
+- **LLM provider** - Ollama server running + models installed, or Anthropic/Bedrock API keys valid
+- **NCBI API** - PubMed/GEO access (optional but recommended)
+- **Workspace** - Directory writable for output files
+
+**Expected output for a working setup:**
+```
+✅ LLM Provider: bedrock (connected)
+✅ NCBI API: Connected (with API key)
+✅ Workspace: Writable
+✅ Configuration Valid
+```
+
+**If config-test fails:**
+
+| Error | Solution |
+|-------|----------|
+| No LLM provider configured | Run `lobster init` |
+| Ollama server not accessible | Start Ollama: `ollama serve` |
+| Ollama: No models installed | After asking user - Install a model: `ollama pull gpt-oss:20b` |
+| Anthropic/Bedrock API error | Check API key validity in `.env` |
+| NCBI API not configured | Add `NCBI_API_KEY` to `.env` (optional) |
+| Workspace not writable | Check directory permissions |
+
+**Quick status checks:**
+```bash
+# Show configuration values (masked)
+lobster config-show
+
+# Show subscription tier and available agents
+lobster status
+```
+
+## Usage
+
+### Basic syntax
+
+```bash
+# Single query (non-interactive)
+lobster query "<natural language request>"
+
+# With custom workspace
+lobster query --workspace /path/to/workspace "<request>"
+
+# With reasoning mode (for complex tasks)
+lobster query --reasoning "<request>"
+```
+
+### Session continuity (multi-turn conversations)
+
+Lobster supports conversation continuity via `--session-id`, enabling follow-up questions that reference previous context either by setting sessin-id to latest or a string of your choice:
+
+```bash
+# default session
+lobster query "Search PubMed for CRISPR papers"
+# Output: Session: session_20241208_150000 (use --session-id latest for follow-ups)
+# then follow up with 
+lobster query --session-id latest "Download the first dataset from that search"
+
+#or use custom session id
+lobster query --session-id "crispr_search_1" "Search PubMed for CRISPR papers"
+#follow up with 
+lobster query --session-id "crispr_search_1" "show me metadata from the first paper"
+```
+
+**Best practices:**
+- Always use `--session-id latest` for follow-up queries
+- Session files are saved in workspace as `session_*.json`
+- Use same `--workspace` for related queries to maintain context
+- Session contains conversation history, not tool execution state
+
+**Workspace-based sessions:**
+```bash
+# Project 1: Cancer research
+lobster query --workspace ~/cancer-project "Search for breast cancer datasets"
+lobster query --workspace ~/cancer-project --session-id latest "Download the best one"
+
+# Project 2: Immunology (separate session)
+lobster query --workspace ~/immuno-project "Search for T cell datasets"
+lobster query --workspace ~/immuno-project --session-id latest "Analyze that"
+```
+
+### Common patterns
+
+**Single-cell analysis:**
+```bash
+lobster query "Download GSE109564 and perform quality control"
+lobster query "Cluster the dataset and find marker genes"
+lobster query "Create UMAP visualization colored by cell type"
+```
+
+**Literature mining:**
+```bash
+lobster query "Search PubMed for CRISPR screens in cancer"
+lobster query "Find papers about CAR-T therapy and extract their GEO datasets"
+lobster query "Get the full text and methods section for PMID:12345678"
+```
+
+**Dataset discovery:**
+```bash
+lobster query "Search GEO for single-cell pancreatic beta cell datasets"
+lobster query "Validate GSE200997 metadata for required fields: cell_type, tissue"
+lobster query "Download SRA dataset SRP123456"
+```
+
+**Data analysis:**
+```bash
+lobster query "Load counts.csv and run differential expression analysis"
+lobster query "Perform batch correction on the loaded dataset"
+lobster query "Generate volcano plot for DE results"
+```
+
+**Quality control:**
+```bash
+lobster query "Assess quality metrics for the loaded dataset"
+lobster query "Filter cells with <200 genes or >8000 genes"
+lobster query "Identify doublets using scrublet"
+```
+
+## Output handling
+
+Lobster outputs are saved in the workspace directory (default: `.lobster_workspace/`):
+
+**Key files to check:**
+- `*.h5ad` - Processed datasets (AnnData format)
+- `*.html` - Interactive visualizations
+- `*.png` - Static plots for publications
+- `*.csv` - Exported data tables
+- `*.json` - Metadata and provenance
+
+**To read results:**
+```bash
+# List workspace files
+ls -lh .lobster_workspace/
+
+# Read specific outputs
+cat .lobster_workspace/analysis_summary.json
+```
+
+## Integration workflow
+
+**Example 1: Analyze dataset and extract results**
+
+```bash
+# Step 1: Run analysis
+lobster query --session-id "gse109564" "Download GSE109564, run QC, and cluster cells"
+
+# Step 2: Check outputs
+ls .lobster_workspace/*.h5ad
+ls .lobster_workspace/*.html
+
+# Step 3: Extract specific data
+lobster query --session-id "gse109564" "Export cluster markers to CSV"
+
+# Step 4: Use results in your code
+# Results are now in .lobster_workspace/markers.csv
+```
+
+**Example 2: Literature mining workflow**
+
+```bash
+# Step 1: Find papers
+lobster query "Search for papers about immune checkpoint inhibitors in melanoma"
+
+# Step 2: Extract datasets
+lobster query "Extract all GEO dataset IDs from the cached papers"
+
+# Step 3: Validate datasets
+lobster query "Check which datasets have cell_type and treatment metadata"
+
+# Step 4: Download best match
+lobster query "Download the dataset with most samples"
+```
+
+## Advanced features
+
+**Export reproducible notebooks:**
+```bash
+lobster query "Export the analysis pipeline as a Jupyter notebook"
+# Creates a Papermill-compatible notebook in workspace
+```
+
+**Workspace management:**
+```bash
+# Use custom workspace per project
+lobster query --workspace ./project1-data "Analyze counts.csv"
+lobster query --workspace ./project2-data "Analyze other-counts.csv"
+```
+
+**Provider switching (if multiple LLM providers configured):**
+```bash
+# Use specific provider
+lobster query --provider ollama "Run expensive analysis"  # Free local
+lobster query --provider anthropic "Quick task"  # Fast cloud
+```
+
+## Troubleshooting
+
+**Command not found:**
+- Verify installation: `which lobster`
+- Install: `uv pip install lobster-ai`
+- Configure: `lobster init`
+
+**Rate limit errors:**
+- Using Anthropic? Switch to Ollama (free) or AWS Bedrock (enterprise)
+- Wait 60 seconds and retry
+- Configure Ollama: `ollama pull llama3:8b-instruct && export LOBSTER_LLM_PROVIDER=ollama`
+
+**Analysis errors:**
+- Check workspace: `ls .lobster_workspace/`
+- View session log: `cat ~/.lobster/.session.json`
+- Try with reasoning: `lobster query --reasoning "<request>"`
+
+**No output files:**
+- Verify workspace location: `lobster query "show workspace info"`
+- Check for errors in command output
+- Ensure request was analysis (not just information retrieval)
+
+## Tips for effective use
+
+1. **Be specific:** Instead of "analyze data", say "perform single-cell clustering with resolution 0.5"
+2. **Chain operations:** "Download GSE12345, run QC, cluster, and export markers to CSV"
+3. **Check outputs:** Always verify generated files in `.lobster_workspace/`
+4. **Use reasoning mode:** For complex multi-step tasks, add `--reasoning` flag
+5. **Provide context:** Reference specific files, datasets, or previous results
+
+## Limitations
+
+- Lobster requires active LLM provider (Ollama/Anthropic/Bedrock)
+- Large datasets (>100K cells) may be slow depending on system resources
+- Some features require premium subscription (proteomics, metadata assistant)
+- Full-text paper access limited by journal availability
+- Rate limits apply when using cloud LLM providers
+
+## Documentation
+
+- Wiki: https://github.com/the-omics-os/lobster-local/wiki
+- Examples: https://github.com/the-omics-os/lobster-local/wiki/27-examples-cookbook
+- Installation: https://github.com/the-omics-os/lobster-local/wiki/02-installation
+- Configuration: https://github.com/the-omics-os/lobster-local/wiki/03-configuration
+
+## Version
+
+This Skill is compatible with:
+- Lobster AI v0.3.1.4+
+- Claude Code v1.0+
+
+For issues or questions: https://github.com/the-omics-os/lobster-local/issues

package/data/skills/long-read-sequencing-agent/SKILL.md

@@ -0,0 +1,181 @@
+<!--
+# COPYRIGHT NOTICE
+# This file is part of the "Universal Biomedical Skills" project.
+# Copyright (c) 2026 MD BABU MIA, PhD <md.babu.mia@mssm.edu>
+# All Rights Reserved.
+#
+# This code is proprietary and confidential.
+# Unauthorized copying of this file, via any medium is strictly prohibited.
+#
+# Provenance: Authenticated by MD BABU MIA
+
+-->
+
+---
+name: 'long-read-sequencing-agent'
+description: 'AI-powered analysis of long-read sequencing data (PacBio, ONT) for structural variant detection, isoform discovery, epigenetic modifications, and de novo assembly.'
+measurable_outcome: Execute skill workflow successfully with valid output within 15 minutes.
+allowed-tools:
+  - read_file
+  - run_shell_command
+---
+
+
+# Long-Read Sequencing Agent
+
+The **Long-Read Sequencing Agent** provides comprehensive AI-driven analysis of long-read sequencing data from PacBio (HiFi) and Oxford Nanopore (ONT) platforms. It enables structural variant detection, full-length isoform discovery, base modification calling, and de novo genome assembly.
+
+## When to Use This Skill
+
+* When detecting structural variants (SVs) missed by short-read sequencing.
+* To characterize full-length transcript isoforms and alternative splicing.
+* For detecting DNA base modifications (5mC, 6mA) directly from sequencing.
+* When performing de novo genome assembly for complex regions.
+* To phase variants and generate fully-resolved haplotypes.
+
+## Core Capabilities
+
+1. **Structural Variant Detection**: AI-enhanced SV calling for deletions, insertions, inversions, translocations, and complex rearrangements.
+
+2. **Isoform Discovery**: Full-length transcript sequencing for novel isoform and fusion detection.
+
+3. **Base Modification Calling**: Direct detection of DNA methylation (5mC, 5hmC, 6mA) from native sequencing.
+
+4. **Haplotype Phasing**: Phase-resolved assemblies and variant calling.
+
+5. **De Novo Assembly**: Assemble complex genomic regions (centromeres, telomeres, HLA).
+
+6. **Error Correction**: AI-based error correction for long-read data.
+
+## Platform Comparison
+
+| Feature | PacBio HiFi | ONT (R10+) |
+|---------|-------------|------------|
+| Read length | 15-25 kb | >100 kb possible |
+| Accuracy | >99.9% (HiFi) | >99% (Q20+) |
+| Base mods | 5mC, 6mA | 5mC, 5hmC, 6mA, more |
+| Throughput | 20-40 Gb/run | 100+ Gb/run |
+| Cost | Higher | Lower |
+
+## Workflow
+
+1. **Input**: Long-read FASTQ/BAM files from PacBio or ONT sequencing.
+
+2. **QC & Alignment**: Filter reads by quality, align to reference genome.
+
+3. **SV Calling**: Detect structural variants using Sniffles, PBSV, or CuteSV.
+
+4. **Isoform Analysis**: Identify full-length isoforms with IsoSeq or FLAIR.
+
+5. **Modification Calling**: Extract base modifications from signal data.
+
+6. **Phasing**: Generate haplotype-resolved variant calls.
+
+7. **Output**: SV calls, isoform annotations, modification maps, phased assemblies.
+
+## Example Usage
+
+**User**: "Analyze this PacBio HiFi dataset for structural variants and DNA methylation in a cancer sample."
+
+**Agent Action**:
+```bash
+python3 Skills/Genomics/Long_Read_Sequencing_Agent/longread_analyzer.py \
+    --input cancer_hifi.bam \
+    --platform pacbio_hifi \
+    --reference GRCh38.fa \
+    --sv_calling sniffles2 \
+    --methylation true \
+    --phasing true \
+    --output longread_results/
+```
+
+## Structural Variant Detection
+
+| Tool | Platform | SV Types | Strengths |
+|------|----------|----------|-----------|
+| Sniffles2 | Both | All SV types | Speed, accuracy |
+| PBSV | PacBio | All SV types | HiFi optimized |
+| CuteSV | Both | All SV types | Sensitivity |
+| SAVANA | Both | Somatic SVs | Cancer-specific |
+| Jasmine | Both | Population SV | Multi-sample |
+
+**SV Size Spectrum**:
+- Small SVs: 50-500 bp (often missed by short-read)
+- Medium SVs: 500 bp - 10 kb
+- Large SVs: >10 kb
+- Complex SVs: Multi-breakpoint events
+
+## Isoform Analysis
+
+**Full-Length Transcript Sequencing**:
+- Capture full gene structures (5' to 3')
+- Detect novel exons and splice junctions
+- Identify gene fusions
+- Quantify isoform expression
+
+**Tools**:
+- IsoSeq3 (PacBio): Clustering and polishing
+- FLAIR (Both): Isoform discovery and quantification
+- StringTie2 (Both): Guided assembly
+- SQANTI3: Isoform classification and QC
+
+## Base Modification Detection
+
+| Modification | Detection | Biological Role |
+|--------------|-----------|-----------------|
+| 5mC | Both platforms | Gene silencing |
+| 5hmC | ONT primarily | Active demethylation |
+| 6mA | Both platforms | Bacterial/mitochondrial |
+| BrdU | ONT | Replication timing |
+
+**Resolution**: Single-base, single-molecule, strand-specific
+
+## AI/ML Components
+
+**Error Correction**:
+- DeepConsensus (PacBio): Transformer for HiFi calling
+- Medaka (ONT): Neural network polishing
+- PEPPER-Margin-DeepVariant: AI variant calling
+
+**SV Classification**:
+- Deep learning for complex SV characterization
+- ML filters for false positive reduction
+- Multi-sample joint calling
+
+## Clinical Applications
+
+1. **Cancer Genomics**: Detect SVs driving oncogene activation
+2. **Rare Disease**: Resolve variants in complex regions
+3. **Pharmacogenomics**: Phase CYP450 star alleles
+4. **HLA Typing**: Full-resolution typing for transplant
+5. **Repeat Expansions**: Size tandem repeat diseases
+
+## Prerequisites
+
+* Python 3.10+
+* Sniffles2, PBSV, CuteSV for SV calling
+* minimap2/pbmm2 for alignment
+* High-memory system (64GB+ recommended)
+
+## Related Skills
+
+* Long_Read_SV_Caller - For specialized SV analysis
+* Variant_Interpretation - For variant annotation
+* Epigenomics_MethylGPT_Agent - For methylation analysis
+
+## Output Files
+
+| Output | Format | Content |
+|--------|--------|---------|
+| SVs | VCF | Structural variants |
+| Methylation | BED/bigWig | Modification calls |
+| Isoforms | GTF | Transcript annotations |
+| Phased | VCF | Haplotype-resolved variants |
+| Assembly | FASTA | Assembled contigs |
+
+## Author
+
+AI Group - Biomedical AI Platform
+
+
+<!-- AUTHOR_SIGNATURE: 9a7f3c2e-MD-BABU-MIA-2026-MSSM-SECURE -->