pi-skill-search 0.1.0

package/skills/pymoo/SKILL.md

@@ -0,0 +1,90 @@
+---
+name: pymoo
+description: Multi-objective optimization framework. NSGA-II, NSGA-III, MOEA/D, Pareto fronts, constraint handling, benchmarks (ZDT, DTLZ), for engineering design and optimization problems.
+---
+
+# Pymoo - Multi-Objective Optimization in Python
+
+## Overview
+
+Pymoo is a comprehensive Python framework for optimization with emphasis on multi-objective problems. Solve single and multi-objective optimization using state-of-the-art algorithms (NSGA-II/III, MOEA/D), benchmark problems (ZDT, DTLZ), customizable genetic operators, and multi-criteria decision making methods. Excels at finding trade-off solutions (Pareto fronts) for problems with conflicting objectives.
+
+## When to Use This Skill
+
+This skill should be used when:
+- Solving optimization problems with one or multiple objectives
+- Finding Pareto-optimal solutions and analyzing trade-offs
+- Implementing evolutionary algorithms (GA, DE, PSO, NSGA-II/III)
+- Working with constrained optimization problems
+- Benchmarking algorithms on standard test problems (ZDT, DTLZ, WFG)
+- Customizing genetic operators (crossover, mutation, selection)
+- Visualizing high-dimensional optimization results
+- Making decisions from multiple competing solutions
+- Handling binary, discrete, continuous, or mixed-variable problems
+
+## Core Concepts
+
+### The Unified Interface
+
+Pymoo uses a consistent `minimize()` function for all optimization tasks:
+
+```python
+from pymoo.optimize import minimize
+
+result = minimize(
+    problem,        # What to optimize
+    algorithm,      # How to optimize
+    termination,    # When to stop
+    seed=1,
+    verbose=True
+)
+```
+
+### Problem Types
+
+**Single-objective:** One objective to minimize/maximize
+**Multi-objective:** 2-3 conflicting objectives → Pareto front
+**Many-objective:** 4+ objectives → High-dimensional Pareto front
+**Constrained:** Objectives + inequality/equality constraints
+**Dynamic:** Time-varying objectives or constraints
+
+## Quick Start Workflows
+
+### Workflow 1: Single-Objective Optimization
+
+**When:** Optimizing one objective function
+
+**Steps:**
+1. Define or select problem
+2. Choose single-objective algorithm (GA, DE, PSO, CMA-ES)
+3. Configure termination criteria
+4. Run optimization
+5. Extract best solution
+
+**Example:**
+```python
+from pymoo.algorithms.soo.nonconvex.ga import GA
+from pymoo.problems import get_problem
+from pymoo.optimize import minimize
+
+# Built-in problem
+problem = get_problem("rastrigin", n_var=10)
+
+# Configure Genetic Algorithm
+algorithm = GA(
+    pop_size=100,
+    eliminate_duplicates=True
+)
+
+# Optimize
+result = minimize(
+    problem,
+    algorithm,
+    ('n_gen', 200),
+    seed=1,
+    verbose=True
+)
+
+print(f"Best solution: {result.X}")
+print(f"Best objective: {result.F[0]}")
+```

package/skills/pyopenms/SKILL.md

@@ -0,0 +1,119 @@
+---
+name: pyopenms
+description: Complete mass spectrometry analysis platform. Use for proteomics workflows feature detection, peptide identification, protein quantification, and complex LC-MS/MS pipelines. Supports extensive file formats and algorithms. Best for proteomics, comprehensive MS data processing. For simple spectral comparison and metabolite ID use matchms.
+---
+
+# PyOpenMS
+
+## Overview
+
+PyOpenMS provides Python bindings to the OpenMS library for computational mass spectrometry, enabling analysis of proteomics and metabolomics data. Use for handling mass spectrometry file formats, processing spectral data, detecting features, identifying peptides/proteins, and performing quantitative analysis.
+
+## Core Capabilities
+
+PyOpenMS organizes functionality into these domains:
+
+# Read mzML file
+exp = ms.MSExperiment()
+ms.MzMLFile().load("data.mzML", exp)
+
+# Access spectra
+for spectrum in exp:
+    mz, intensity = spectrum.get_peaks()
+    print(f"Spectrum: {len(mz)} peaks")
+```
+
+**For detailed file handling**: See `(see docs)`
+
+### 2. Signal Processing
+
+Process raw spectral data with smoothing, filtering, centroiding, and normalization.
+
+Basic spectrum processing:
+
+```python
+# Smooth spectrum with Gaussian filter
+gaussian = ms.GaussFilter()
+params = gaussian.getParameters()
+params.setValue("gaussian_width", 0.1)
+gaussian.setParameters(params)
+gaussian.filterExperiment(exp)
+```
+
+**For algorithm details**: See `(see docs)`
+
+### 3. Feature Detection
+
+Detect and link features across spectra and samples for quantitative analysis.
+
+```python
+# Detect features
+ff = ms.FeatureFinder()
+ff.run("centroided", exp, features, params, ms.FeatureMap())
+```
+
+**For complete workflows**: See `(see docs)`
+
+### 4. Peptide and Protein Identification
+
+Integrate with search engines and process identification results.
+
+**Supported engines**: Comet, Mascot, MSGFPlus, XTandem, OMSSA, Myrimatch
+
+Basic identification workflow:
+
+```python
+# Load identification data
+protein_ids = []
+peptide_ids = []
+ms.IdXMLFile().load("identifications.idXML", protein_ids, peptide_ids)
+
+# Apply FDR filtering
+fdr = ms.FalseDiscoveryRate()
+fdr.apply(peptide_ids)
+```
+
+**For detailed workflows**: See `(see docs)`
+
+### 5. Metabolomics Analysis
+
+Perform untargeted metabolomics preprocessing and analysis.
+
+Typical workflow:
+1. Load and process raw data
+2. Detect features
+3. Align retention times across samples
+4. Link features to consensus map
+5. Annotate with compound databases
+
+**For complete metabolomics workflows**: See `(see docs)`
+
+## Data Structures
+
+PyOpenMS uses these primary objects:
+
+- **MSExperiment**: Collection of spectra and chromatograms
+- **MSSpectrum**: Single mass spectrum with m/z and intensity pairs
+- **MSChromatogram**: Chromatographic trace
+- **Feature**: Detected chromatographic peak with quality metrics
+- **FeatureMap**: Collection of features
+- **PeptideIdentification**: Search results for peptides
+- **ProteinIdentification**: Search results for proteins
+
+**For detailed documentation**: See `(see docs)`
+
+## Common Workflows
+
+### Quick Start: Load and Explore Data
+
+```python
+import pyopenms as ms
+
+# Load mzML file
+exp = ms.MSExperiment()
+ms.MzMLFile().load("sample.mzML", exp)
+
+# Get basic statistics
+print(f"Number of spectra: {exp.getNrSpectra()}")
+
+

package/skills/pysam/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: pysam
+description: Genomic file toolkit. Read/write SAM/BAM/CRAM alignments, VCF/BCF variants, FASTA/FASTQ sequences, extract regions, calculate coverage, for NGS data processing pipelines.
+---
+
+# Pysam
+
+## Overview
+
+Pysam is a Python module for reading, manipulating, and writing genomic datasets. Read/write SAM/BAM/CRAM alignment files, VCF/BCF variant files, and FASTA/FASTQ sequences with a Pythonic interface to htslib. Query tabix-indexed files, perform pileup analysis for coverage, and execute samtools/bcftools commands.
+
+## When to Use This Skill
+
+This skill should be used when:
+- Working with sequencing alignment files (BAM/CRAM)
+- Analyzing genetic variants (VCF/BCF)
+- Extracting reference sequences or gene regions
+- Processing raw sequencing data (FASTQ)
+- Calculating coverage or read depth
+- Implementing bioinformatics analysis pipelines
+- Quality control of sequencing data
+- Variant calling and annotation workflows
+
+## Quick Start
+
+### Basic Examples
+
+**Read alignment file:**
+```python
+import pysam
+
+# Open BAM file and fetch reads in region
+samfile = pysam.AlignmentFile("example.bam", "rb")
+for read in samfile.fetch("chr1", 1000, 2000):
+    print(f"{read.query_name}: {read.reference_start}")
+samfile.close()
+```
+
+**Read variant file:**
+```python
+# Open VCF file and iterate variants
+vcf = pysam.VariantFile("variants.vcf")
+for variant in vcf:
+    print(f"{variant.chrom}:{variant.pos} {variant.ref}>{variant.alts}")
+vcf.close()
+```
+
+**Query reference sequence:**
+```python
+# Open FASTA and extract sequence
+fasta = pysam.FastaFile("reference.fasta")
+sequence = fasta.fetch("chr1", 1000, 2000)
+print(sequence)
+fasta.close()
+
+## Core Capabilities
+
+### 1. Alignment File Operations (SAM/BAM/CRAM)
+
+Use the `AlignmentFile` class to work with aligned sequencing reads. This is appropriate for analyzing mapping results, calculating coverage, extracting reads, or quality control.
+
+**Common operations:**
+- Open and read BAM/SAM/CRAM files
+- Fetch reads from specific genomic regions
+- Filter reads by mapping quality, flags, or other criteria
+- Write filtered or modified alignments
+- Calculate coverage statistics
+- Perform pileup analysis (base-by-base coverage)
+- Access read sequences, quality scores, and alignment information
+
+**Reference:** See `(see docs)` for detailed documentation on:
+- Opening and reading alignment files
+
+### 2. Variant File Operations (VCF/BCF)
+
+Use the `VariantFile` class to work with genetic variants from variant calling pipelines. This is appropriate for variant analysis, filtering, annotation, or population genetics.
+
+**Common operations:**
+- Read and write VCF/BCF files
+- Query variants in specific regions
+- Access variant information (position, alleles, quality)
+- Extract genotype data for samples
+- Filter variants by quality, allele frequency, or other criteria
+- Annotate variants with additional information
+- Subset samples or regions
+
+**Reference:** See `(see docs)` for detailed documentation on:
+- Opening and reading variant files
+
+### 3. Sequence File Operations (FASTA/FASTQ)
+
+Use `FastaFile` for random access to reference sequences and `FastxFile` for reading raw sequencing data. This is appropriate for extracting gene sequences, validating variants against reference, or processing raw reads.
+
+**Common operations:**
+- Query reference sequences by genomic coordinates
+- Extract sequences for genes or regions of interest
+- Read FASTQ files with quality scores
+- Validate variant reference alleles
+- Calculate sequence statistics
+- Filter reads by quality or length
+- Convert between FASTA and FASTQ formats
+
+**Reference:** See `(see docs)` for detailed documentation on:
+- FASTA file access and indexing
+
+## Key Concepts
+
+### Coordinate Systems
+
+**Critical:** Pysam uses **0-based, half-open** coordinates (Python convention):
+- Start positions are 0-based (first base is position 0)
+- End positions are exclusive (not included in the range)
+- Region 1000-2000 includes bases 1000-1999 (1000 bases total)
+
+**Exception:** Region strings in `fetch()` follow samtools convention (1-based):
+```python
+
+

package/skills/pyspark/SKILL.md

@@ -0,0 +1,30 @@
+---
+name: pyspark
+description: Distributed data processing with Apache Spark. Use when working with large-scale datasets, distributed SQL, DataFrame operations, ETL pipelines, or cluster computing. Trigger on imports of pyspark, SparkSession, or mentions of big data, distributed, cluster, ETL, Spark, DataFrame at scale.
+---
+# pyspark
+
+Use this skill for large-scale distributed data processing.
+
+## Core patterns
+
+- **Session**: `SparkSession.builder.appName('analysis').getOrCreate()`.
+- **Read**: `spark.read.parquet('data/')` or `spark.read.csv('data.csv', header=True, inferSchema=True)`.
+- **Transform**: `df.filter()`, `df.select()`, `df.groupBy().agg()`, `df.join(other, on='key')`.
+- **SQL**: `df.createOrReplaceTempView('table')` → `spark.sql('SELECT * FROM table')`.
+- **Write**: `df.write.parquet('output/', mode='overwrite')`.
+
+## Rules
+
+- Always use `coalesce(1)` or `repartition()` before writing small outputs.
+- Persist intermediate DataFrames used multiple times: `df.persist(StorageLevel.MEMORY_AND_DISK)`.
+- Use broadcast join for small/large table joins: `broadcast(small_df)`.
+- Avoid `collect()` on large DataFrames — use `take(n)` or `toPandas()` with caution.
+
+## Anti-patterns
+
+- Don't call `toPandas()` on large DataFrames — it collects all data to driver.
+- Don't use Python UDFs when built-in Spark SQL functions suffice.
+- Don't create `SparkSession` per operation — reuse across the application.
+
+

package/skills/pytdc/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: pytdc
+description: Therapeutics Data Commons. AI-ready drug discovery datasets (ADME, toxicity, DTI), benchmarks, scaffold splits, molecular oracles, for therapeutic ML and pharmacological prediction.
+---
+
+# PyTDC (Therapeutics Data Commons)
+
+## Overview
+
+PyTDC is an open-science platform providing AI-ready datasets and benchmarks for drug discovery and development. Access curated datasets spanning the entire therapeutics pipeline with standardized evaluation metrics and meaningful data splits, organized into three categories: single-instance prediction (molecular/protein properties), multi-instance prediction (drug-target interactions, DDI), and generation (molecule generation, retrosynthesis).
+
+## When to Use This Skill
+
+This skill should be used when:
+- Working with drug discovery or therapeutic ML datasets
+- Benchmarking machine learning models on standardized pharmaceutical tasks
+- Predicting molecular properties (ADME, toxicity, bioactivity)
+- Predicting drug-target or drug-drug interactions
+- Generating novel molecules with desired properties
+- Accessing curated datasets with proper train/test splits (scaffold, cold-split)
+- Using molecular oracles for property optimization
+
+## Quick Start
+
+The basic pattern for accessing any TDC dataset follows this structure:
+
+```python
+from tdc.<problem> import 
+data = (name='')
+split = data.get_split(method='scaffold', seed=1, frac=[0.7, 0.1, 0.2])
+df = data.get_data(format='df')
+```
+
+Where:
+- `<problem>`: One of `single_pred`, `multi_pred`, or `generation`
+- ``: Specific task category (e.g., ADME, DTI, MolGen)
+- ``: Dataset name within that task
+
+# Returns dict with 'train', 'valid', 'test' DataFrames
+```
+
+## Single-Instance Prediction Tasks
+
+Single-instance prediction involves forecasting properties of individual biomedical entities (molecules, proteins, etc.).
+
+### Available Task Categories
+
+#### 1. ADME (Absorption, Distribution, Metabolism, Excretion)
+
+Predict pharmacokinetic properties of drug molecules.
+
+```python
+from tdc.single_pred import ADME
+data = ADME(name='Caco2_Wang')  # Intestinal permeability
+# Other datasets: HIA_Hou, Bioavailability_Ma, Lipophilicity_AstraZeneca, etc.
+```
+
+**Common ADME datasets:**
+- Caco2 - Intestinal permeability
+- HIA - Human intestinal absorption
+- Bioavailability - Oral bioavailability
+- Lipophilicity - Octanol-water partition coefficient
+- Solubility - Aqueous solubility
+- BBB - Blood-brain barrier penetration
+- CYP - Cytochrome P450 metabolism
+
+#### 2. Toxicity (Tox)
+
+Predict toxicity and adverse effects of compounds.
+
+# Other datasets: AMES, DILI, Carcinogens_Lagunin, etc.
+```
+
+**Common toxicity datasets:**
+- hERG - Cardiac toxicity
+- AMES - Mutagenicity
+- DILI - Drug-induced liver injury
+- Carcinogens - Carcinogenicity
+- ClinTox - Clinical trial toxicity
+
+#### 3. HTS (High-Throughput Screening)
+
+Bioactivity predictions from screening data.
+
+```python
+
+## Multi-Instance Prediction Tasks
+
+Multi-instance prediction involves forecasting properties of interactions between multiple biomedical entities.
+
+### Available Task Categories
+
+#### 1. DTI (Drug-Target Interaction)
+
+Predict binding affinity between drugs and protein targets.
+
+```python
+from tdc.multi_pred import DTI
+data = DTI(name='BindingDB_Kd')
+split = data.get_split()
+
+

package/skills/pytorch/SKILL.md

@@ -0,0 +1,31 @@
+---
+name: pytorch
+description: Deep learning framework for building and training neural networks. Use when creating CNN, RNN, Transformer, or custom architectures, training models with GPU acceleration, implementing custom loss functions, or optimizing with autograd. Trigger on imports of torch, torchvision, torchaudio, nn.Module, or mentions of neural network training, GPU, CUDA, tensor operations.
+---
+# pytorch
+
+Use this skill for deep learning model development.
+
+## Core patterns
+
+- **Model**: Subclass `nn.Module`, define `__init__` and `forward()`.
+- **Training loop**: Forward → loss → `loss.backward()` → `optimizer.step()` → `optimizer.zero_grad()`.
+- **Data**: `Dataset` + `DataLoader(shuffle=True, num_workers=4, pin_memory=True)`.
+- **GPU**: `tensor.to(device)`, `model.to(device)`. Check `torch.cuda.is_available()`.
+- **Saving**: `torch.save(model.state_dict(), path)` / `model.load_state_dict(torch.load(path))`.
+
+## Rules
+
+- Use `torch.no_grad()` context during inference and evaluation.
+- Set `model.eval()` before validation; `model.train()` before training.
+- Use `nn.Sequential` for simple stacks; custom `forward()` for complex architectures.
+- Learning rate scheduling: call `scheduler.step()` after `optimizer.step()`.
+- Mixed precision: `torch.amp.autocast('cuda')` + `GradScaler` for faster training.
+
+## Anti-patterns
+
+- Don't forget `optimizer.zero_grad()` — gradients accumulate by default.
+- Don't use `.item()` inside training loop on large tensors — only for scalar metrics.
+- Don't hardcode device — always use `device = 'cuda' if torch.cuda.is_available() else 'cpu'`.
+
+

package/skills/pytorch-lightning/SKILL.md

@@ -0,0 +1,119 @@
+---
+name: pytorch-lightning
+description: Deep learning framework (PyTorch Lightning). Organize PyTorch code into LightningModules, configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B, TensorBoard), distributed training (DDP, FSDP, DeepSpeed), for scalable neural network training.
+---
+
+# PyTorch Lightning
+
+## Overview
+
+PyTorch Lightning is a deep learning framework that organizes PyTorch code to eliminate boilerplate while maintaining full flexibility. Automate training workflows, multi-device orchestration, and implement best practices for neural network training and scaling across multiple GPUs/TPUs.
+
+## When to Use This Skill
+
+This skill should be used when:
+- Building, training, or deploying neural networks using PyTorch Lightning
+- Organizing PyTorch code into LightningModules
+- Configuring Trainers for multi-GPU/TPU training
+- Implementing data pipelines with LightningDataModules
+- Working with callbacks, logging, and distributed training strategies (DDP, FSDP, DeepSpeed)
+- Structuring deep learning projects professionally
+
+## Core Capabilities
+
+### 1. LightningModule - Model Definition
+
+Organize PyTorch models into six logical sections:
+
+1. **Initialization** - `__init__()` and `setup()`
+2. **Training Loop** - `training_step(batch, batch_idx)`
+3. **Validation Loop** - `validation_step(batch, batch_idx)`
+4. **Test Loop** - `test_step(batch, batch_idx)`
+5. **Prediction** - `predict_step(batch, batch_idx)`
+6. **Optimizer Configuration** - `configure_optimizers()`
+
+**Quick template reference:** See `scripts/template_lightning_module.py` for a complete boilerplate.
+
+**Detailed documentation:** Read `(see docs)` for comprehensive method documentation, hooks, properties, and best practices.
+
+### 2. Trainer - Training Automation
+
+The Trainer automates the training loop, device management, gradient operations, and callbacks. Key features:
+
+- Multi-GPU/TPU support with strategy selection (DDP, FSDP, DeepSpeed)
+- Automatic mixed precision training
+- Gradient accumulation and clipping
+- Checkpointing and early stopping
+- Progress bars and logging
+
+**Quick setup reference:** See `scripts/quick_trainer_setup.py` for common Trainer configurations.
+
+**Detailed documentation:** Read `(see docs)` for all parameters, methods, and configuration options.
+
+### 3. LightningDataModule - Data Pipeline Organization
+
+Encapsulate all data processing steps in a reusable class:
+
+1. `prepare_data()` - Download and process data (single-process)
+2. `setup()` - Create datasets and apply transforms (per-GPU)
+3. `train_dataloader()` - Return training DataLoader
+4. `val_dataloader()` - Return validation DataLoader
+5. `test_dataloader()` - Return test DataLoader
+
+**Quick template reference:** See `scripts/template_datamodule.py` for a complete boilerplate.
+
+**Detailed documentation:** Read `(see docs)` for method details and usage patterns.
+
+### 4. Callbacks - Extensible Training Logic
+
+Add custom functionality at specific training hooks without modifying your LightningModule. Built-in callbacks include:
+
+- **ModelCheckpoint** - Save best/latest models
+- **EarlyStopping** - Stop when metrics plateau
+- **LearningRateMonitor** - Track LR scheduler changes
+- **BatchSizeFinder** - Auto-determine optimal batch size
+
+**Detailed documentation:** Read `(see docs)` for built-in callbacks and custom callback creation.
+
+### 5. Logging - Experiment Tracking
+
+Integrate with multiple logging platforms:
+
+- TensorBoard (default)
+- Weights & Biases (WandbLogger)
+- MLflow (MLFlowLogger)
+- Neptune (NeptuneLogger)
+- Comet (CometLogger)
+- CSV (CSVLogger)
+
+Log metrics using `self.log("metric_name", value)` in any LightningModule method.
+
+**Detailed documentation:** Read `(see docs)` for logger setup and configuration.
+
+### 6. Distributed Training - Scale to Multiple Devices
+
+Choose the right strategy based on model size:
+
+- **DDP** - For models <500M parameters (ResNet, smaller transformers)
+- **FSDP** - For models 500M+ parameters (large transformers, recommended for Lightning users)
+- **DeepSpeed** - For cutting-edge features and fine-grained control
+
+Configure with: `Trainer(strategy="ddp", accelerator="gpu", devices=4)`
+
+**Detailed documentation:** Read `(see docs)` for strategy comparison and configuration.
+
+### 7. Best Practices
+
+- Device agnostic code - Use `self.device` instead of `.cuda()`
+- Hyperparameter saving - Use `self.save_hyperparameters()` in `__init__()`
+- Metric logging - Use `self.log()` for automatic aggregation across devices
+- Reproducibility - Use `seed_everything()` and `Trainer(deterministic=True)`
+- Debugging - Use `Trainer(fast_dev_run=True)` to test with 1 batch
+
+**Detailed documentation:** Read `(see docs)` for common patterns and pitfalls.
+
+## Quick Workflow
+
+1. **Define model:**
+
+