@bgicli/bgicli 2.1.0 → 2.2.0

package/data/skills/bio-sequence-properties/SKILL.md

@@ -0,0 +1,397 @@
+<!--
+# COPYRIGHT NOTICE
+# This file is part of the "Universal Biomedical Skills" project.
+# Copyright (c) 2026 MD BABU MIA, PhD <md.babu.mia@mssm.edu>
+# All Rights Reserved.
+#
+# This code is proprietary and confidential.
+# Unauthorized copying of this file, via any medium is strictly prohibited.
+#
+# Provenance: Authenticated by MD BABU MIA
+
+-->
+
+---
+name: bio-sequence-properties
+description: Calculate sequence properties like GC content, molecular weight, isoelectric point, and GC skew using Biopython. Use when analyzing sequence composition, computing physical properties, or comparing sequences.
+tool_type: python
+primary_tool: Bio.SeqUtils
+measurable_outcome: Execute skill workflow successfully with valid output within 15 minutes.
+allowed-tools:
+  - read_file
+  - run_shell_command
+---
+
+# Sequence Properties
+
+Calculate physical and chemical properties of biological sequences using Biopython.
+
+## Required Imports
+
+```python
+from Bio.Seq import Seq
+from Bio.SeqUtils import gc_fraction, molecular_weight, GC123, GC_skew, nt_search, seq1, seq3
+from Bio.SeqUtils.ProtParam import ProteinAnalysis
+```
+
+## DNA/RNA Properties
+
+### GC Content
+
+```python
+from Bio.SeqUtils import gc_fraction
+
+seq = Seq('ATGCGATCGATCGATCGATCG')
+gc = gc_fraction(seq)  # Returns 0.476... (fraction)
+gc_percent = gc * 100  # Convert to percentage
+```
+
+Handle ambiguous bases:
+```python
+gc = gc_fraction(seq, ambiguous='ignore')  # Ignore N bases in calculation
+gc = gc_fraction(seq, ambiguous='weighted')  # Weight by probability
+```
+
+### GC at Codon Positions (GC123)
+
+Analyze GC content at each codon position (useful for codon bias analysis):
+
+```python
+from Bio.SeqUtils import GC123
+
+seq = Seq('ATGCGATCGATCGATCGATCG')
+gc_total, gc_pos1, gc_pos2, gc_pos3 = GC123(seq)
+# gc_total: overall GC%
+# gc_pos1: GC% at 1st codon position
+# gc_pos2: GC% at 2nd codon position
+# gc_pos3: GC% at 3rd codon position (wobble)
+```
+
+### GC Skew
+
+Calculate (G-C)/(G+C) in sliding windows to identify replication origins:
+
+```python
+from Bio.SeqUtils import GC_skew
+
+seq = Seq('ATGCGATCGATCGATCGATCG' * 10)
+skew_values = GC_skew(seq, window=100)  # Returns list of skew values
+```
+
+### Molecular Weight
+
+```python
+from Bio.SeqUtils import molecular_weight
+
+dna = Seq('ATGCGATCG')
+mw = molecular_weight(dna)  # Single-stranded DNA weight
+
+# Double-stranded DNA
+mw_ds = molecular_weight(dna, double_stranded=True)
+
+# Circular DNA
+mw_circ = molecular_weight(dna, circular=True)
+
+# RNA
+rna = Seq('AUGCGAUCG')
+mw_rna = molecular_weight(rna, seq_type='RNA')
+
+# Protein
+protein = Seq('MRCRS')
+mw_prot = molecular_weight(protein, seq_type='protein')
+```
+
+### Melting Temperature
+
+```python
+from Bio.SeqUtils import MeltingTemp
+
+seq = Seq('ATGCGATCGATCG')
+
+# Basic Tm (Wallace rule - short oligos)
+tm_wallace = MeltingTemp.Tm_Wallace(seq)
+
+# GC method (more accurate for longer sequences)
+tm_gc = MeltingTemp.Tm_GC(seq)
+
+# Nearest neighbor (most accurate)
+tm_nn = MeltingTemp.Tm_NN(seq)
+
+# With salt correction
+tm_corrected = MeltingTemp.Tm_NN(seq, Na=50, Mg=1.5)
+```
+
+### Sequence Search with IUPAC Codes (nt_search)
+
+Built-in search that handles IUPAC ambiguity codes:
+
+```python
+from Bio.SeqUtils import nt_search
+
+seq = 'ATGCGATCGATCGATNGATC'
+# Search for pattern with ambiguous bases
+result = nt_search(seq, 'GATNGATC')  # N matches any base
+# Returns: ['GATNGATC', 8] - pattern and position(s)
+```
+
+### Base Composition
+
+```python
+def base_composition(seq):
+    seq_str = str(seq).upper()
+    total = len(seq_str)
+    return {base: seq_str.count(base) / total * 100 for base in 'ATGC'}
+```
+
+## Amino Acid Code Conversion
+
+Convert between 1-letter and 3-letter amino acid codes:
+
+```python
+from Bio.SeqUtils import seq1, seq3
+
+# 3-letter to 1-letter
+one_letter = seq1('MetAlaGlyTrp')  # Returns 'MAGW'
+
+# 1-letter to 3-letter
+three_letter = seq3('MAGW')  # Returns 'MetAlaGlyTrp'
+
+# With custom separator
+three_letter = seq3('MAGW', join='-')  # Returns 'Met-Ala-Gly-Trp'
+```
+
+## Protein Properties
+
+### Using ProteinAnalysis
+
+```python
+from Bio.SeqUtils.ProtParam import ProteinAnalysis
+
+protein = ProteinAnalysis('MAEGEITTFTALTEKFNLPPGNYKKPKLLYCSNG')
+```
+
+### Basic Properties
+
+```python
+mw = protein.molecular_weight()           # Molecular weight in Daltons
+pi = protein.isoelectric_point()          # Isoelectric point
+aa_comp = protein.amino_acids_percent     # Amino acid composition (property)
+aa_count = protein.count_amino_acids()    # Raw amino acid counts
+```
+
+### Stability and Hydropathy
+
+```python
+instability = protein.instability_index()  # < 40 = stable, > 40 = unstable
+gravy = protein.gravy()                    # Negative = hydrophilic, Positive = hydrophobic
+arom = protein.aromaticity()               # Fraction of Phe+Trp+Tyr
+```
+
+### Charge at Specific pH
+
+```python
+charge = protein.charge_at_pH(7.0)  # Net charge at pH 7.0
+charge_acidic = protein.charge_at_pH(4.0)
+charge_basic = protein.charge_at_pH(10.0)
+```
+
+### Flexibility Profile
+
+```python
+flexibility = protein.flexibility()  # List of flexibility values per residue
+# Based on Vihinen, 1994 scale
+```
+
+### Secondary Structure
+
+```python
+helix, turn, sheet = protein.secondary_structure_fraction()
+```
+
+### Extinction Coefficient
+
+```python
+eps_reduced, eps_cystine = protein.molar_extinction_coefficient()
+# eps_reduced: all Cys are reduced
+# eps_cystine: all Cys form disulfide bonds
+```
+
+### Protein Scale Profiles
+
+Calculate profiles using any amino acid scale:
+
+```python
+# Kyte-Doolittle hydropathy profile
+kd_scale = {'A': 1.8, 'R': -4.5, 'N': -3.5, ...}
+profile = protein.protein_scale(kd_scale, window=7)
+```
+
+## Code Patterns
+
+### Analyze Multiple Sequences
+
+```python
+from Bio import SeqIO
+from Bio.SeqUtils import gc_fraction
+
+def analyze_fasta(filename):
+    results = []
+    for record in SeqIO.parse(filename, 'fasta'):
+        results.append({
+            'id': record.id,
+            'length': len(record.seq),
+            'gc': gc_fraction(record.seq) * 100
+        })
+    return results
+```
+
+### GC Content Distribution (Sliding Windows)
+
+```python
+from Bio.SeqUtils import gc_fraction
+
+def gc_distribution(seq, window_size=100, step=50):
+    gc_values = []
+    for i in range(0, len(seq) - window_size + 1, step):
+        window = seq[i:i + window_size]
+        gc_values.append((i, gc_fraction(window) * 100))
+    return gc_values
+```
+
+### GC Skew Plot Data
+
+```python
+from Bio.SeqUtils import GC_skew
+
+def gc_skew_analysis(seq, window=1000):
+    skew = GC_skew(seq, window=window)
+    positions = list(range(0, len(seq) - window + 1, window))
+    cumulative = []
+    total = 0
+    for s in skew:
+        total += s
+        cumulative.append(total)
+    return positions, skew, cumulative
+```
+
+### Full Protein Analysis Report
+
+```python
+from Bio.SeqUtils.ProtParam import ProteinAnalysis
+
+def protein_report(sequence):
+    protein = ProteinAnalysis(str(sequence).replace('*', ''))
+    helix, turn, sheet = protein.secondary_structure_fraction()
+    return {
+        'length': len(sequence),
+        'molecular_weight': protein.molecular_weight(),
+        'isoelectric_point': protein.isoelectric_point(),
+        'charge_at_pH7': protein.charge_at_pH(7.0),
+        'instability_index': protein.instability_index(),
+        'gravy': protein.gravy(),
+        'aromaticity': protein.aromaticity(),
+        'helix_fraction': helix,
+        'turn_fraction': turn,
+        'sheet_fraction': sheet,
+    }
+```
+
+### Dinucleotide Frequencies
+
+```python
+from collections import Counter
+
+def dinucleotide_freq(seq):
+    seq_str = str(seq)
+    dinucs = [seq_str[i:i+2] for i in range(len(seq_str) - 1)]
+    counts = Counter(dinucs)
+    total = sum(counts.values())
+    return {di: count / total for di, count in counts.items()}
+```
+
+### CpG Observed/Expected Ratio
+
+```python
+def cpg_ratio(seq):
+    seq_str = str(seq).upper()
+    cpg = seq_str.count('CG')
+    c = seq_str.count('C')
+    g = seq_str.count('G')
+    expected = (c * g) / len(seq_str) if len(seq_str) > 0 else 0
+    return cpg / expected if expected > 0 else 0
+```
+
+## Property Reference
+
+### DNA/RNA Properties
+
+| Property | Function | Notes |
+|----------|----------|-------|
+| GC content | `gc_fraction()` | Returns fraction (0-1) |
+| GC by position | `GC123()` | Total, pos1, pos2, pos3 |
+| GC skew | `GC_skew()` | (G-C)/(G+C) in windows |
+| Molecular weight | `molecular_weight()` | In Daltons |
+| Melting temp | `MeltingTemp.Tm_*()` | Multiple methods |
+| IUPAC search | `nt_search()` | Handles ambiguity codes |
+
+### Protein Properties
+
+| Property | Method | Typical Range |
+|----------|--------|---------------|
+| MW | `molecular_weight()` | Daltons |
+| pI | `isoelectric_point()` | 0-14 |
+| Charge | `charge_at_pH(pH)` | Varies |
+| Instability | `instability_index()` | <40 stable |
+| GRAVY | `gravy()` | -2 to +2 |
+| Aromaticity | `aromaticity()` | 0-0.15 |
+| Flexibility | `flexibility()` | Per-residue list |
+
+### Amino Acid Codes
+
+| Function | Input | Output |
+|----------|-------|--------|
+| `seq1()` | 'MetAlaGly' | 'MAG' |
+| `seq3()` | 'MAG' | 'MetAlaGly' |
+
+## Common Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `KeyError` in ProteinAnalysis | Non-standard amino acid | Remove X, *, etc. |
+| Wrong MW | Wrong seq_type | Specify 'RNA' or 'protein' |
+| Invalid Tm | Sequence too short | Use >10 bp for accurate Tm |
+| Empty GC_skew | Sequence shorter than window | Reduce window size |
+
+## Decision Tree
+
+```
+Need sequence properties?
+├── DNA/RNA sequence?
+│   ├── GC content? → gc_fraction()
+│   ├── GC at codon positions? → GC123()
+│   ├── GC skew (replication origin)? → GC_skew()
+│   ├── Molecular weight? → molecular_weight()
+│   ├── Melting temperature? → MeltingTemp.Tm_NN()
+│   └── Search with IUPAC codes? → nt_search()
+├── Protein sequence?
+│   └── Create ProteinAnalysis object
+│       ├── Size → molecular_weight()
+│       ├── Charge → isoelectric_point(), charge_at_pH()
+│       ├── Stability → instability_index()
+│       ├── Hydrophobicity → gravy()
+│       ├── Flexibility → flexibility()
+│       └── Structure → secondary_structure_fraction()
+└── Convert amino acid codes?
+    └── seq1() / seq3()
+```
+
+## Related Skills
+
+- seq-objects - Create Seq objects for property calculation
+- sequence-io/sequence-statistics - File-level statistics (N50, totals)
+- codon-usage - GC123 for codon position analysis
+- transcription-translation - Translate before protein analysis
+- alignment-files/bam-statistics - samtools stats for alignment-level GC and quality stats
+
+
+<!-- AUTHOR_SIGNATURE: 9a7f3c2e-MD-BABU-MIA-2026-MSSM-SECURE -->

package/data/skills/bio-sequence-similarity/SKILL.md

@@ -0,0 +1,335 @@
+<!--
+# COPYRIGHT NOTICE
+# This file is part of the "Universal Biomedical Skills" project.
+# Copyright (c) 2026 MD BABU MIA, PhD <md.babu.mia@mssm.edu>
+# All Rights Reserved.
+#
+# This code is proprietary and confidential.
+# Unauthorized copying of this file, via any medium is strictly prohibited.
+#
+# Provenance: Authenticated by MD BABU MIA
+
+-->
+
+---
+name: bio-sequence-similarity
+description: Find homologous sequences using iterative BLAST (PSI-BLAST), profile HMMs (HMMER), and reciprocal best hit analysis. Use when identifying orthologs, distant homologs, or protein family members where standard BLAST is not sensitive enough.
+tool_type: mixed
+primary_tool: BLAST+
+measurable_outcome: Execute skill workflow successfully with valid output within 15 minutes.
+allowed-tools:
+  - read_file
+  - run_shell_command
+---
+
+# Sequence Similarity Searches
+
+Advanced methods for finding homologous sequences beyond standard BLAST.
+
+## PSI-BLAST (Position-Specific Iterated BLAST)
+
+Builds a position-specific scoring matrix (PSSM) through iterations to find distant homologs.
+
+### Basic PSI-BLAST
+
+```bash
+psiblast -query protein.fasta -db nr -out results.txt -num_iterations 3
+```
+
+### Save PSSM for Reuse
+
+```bash
+psiblast -query protein.fasta -db nr \
+    -out results.txt \
+    -out_pssm pssm.asn \
+    -out_ascii_pssm pssm.txt \
+    -num_iterations 5
+```
+
+### Use Existing PSSM
+
+```bash
+psiblast -in_pssm pssm.asn -db nr -out results.txt
+```
+
+### Output Format
+
+```bash
+psiblast -query protein.fasta -db nr \
+    -out results.txt \
+    -outfmt 6 \
+    -num_iterations 3 \
+    -inclusion_ethresh 0.001
+```
+
+### Key Parameters
+
+```bash
+psiblast -query protein.fasta -db nr \
+    -num_iterations 5 \
+    -inclusion_ethresh 0.001 \
+    -evalue 0.01 \
+    -num_threads 8 \
+    -out results.txt
+```
+
+### PSI-BLAST Parameters
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| -num_iterations | 1 | Number of iterations |
+| -inclusion_ethresh | 0.002 | E-value for PSSM inclusion |
+| -evalue | 10 | E-value threshold for reporting |
+| -num_threads | 1 | CPU threads |
+
+## HMMER for Profile Searches
+
+HMMER uses profile hidden Markov models for sensitive sequence searches.
+
+### Search with Single Sequence
+
+```bash
+jackhmmer -o results.txt -A aligned.sto --cpu 8 query.fasta database.fasta
+```
+
+### Build Profile from Alignment
+
+```bash
+hmmbuild profile.hmm alignment.sto
+```
+
+### Search Database with Profile
+
+```bash
+hmmsearch -o results.txt --tblout hits.tbl profile.hmm database.fasta
+hmmsearch -o results.txt --domtblout domains.tbl profile.hmm database.fasta
+```
+
+### Download Pfam Profiles
+
+```bash
+wget https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz
+gunzip Pfam-A.hmm.gz
+hmmpress Pfam-A.hmm
+```
+
+### Scan Sequence Against Pfam
+
+```bash
+hmmscan --tblout pfam_hits.tbl --domtblout domains.tbl Pfam-A.hmm query.fasta
+```
+
+### Parse HMMER Output
+
+```bash
+grep -v "^#" hits.tbl | head
+awk '$5 < 1e-10' hits.tbl
+```
+
+### HMMER Output Columns (--tblout)
+
+| Column | Description |
+|--------|-------------|
+| 1 | Target name |
+| 2 | Accession |
+| 3 | Query name |
+| 4 | Query accession |
+| 5 | E-value (full sequence) |
+| 6 | Score (full sequence) |
+| 7 | Bias |
+| 8 | E-value (best domain) |
+| 9 | Score (best domain) |
+
+## Reciprocal Best Hit (RBH) Analysis
+
+Find orthologs using bidirectional best hits.
+
+### Create BLAST Databases
+
+```bash
+makeblastdb -in species_A.fasta -dbtype prot -out species_A_db
+makeblastdb -in species_B.fasta -dbtype prot -out species_B_db
+```
+
+### Bidirectional BLAST
+
+```bash
+blastp -query species_A.fasta -db species_B_db -outfmt 6 -evalue 1e-5 -max_target_seqs 1 > A_vs_B.txt
+blastp -query species_B.fasta -db species_A_db -outfmt 6 -evalue 1e-5 -max_target_seqs 1 > B_vs_A.txt
+```
+
+### Find Reciprocal Best Hits
+
+```bash
+awk 'FNR==NR {a[$1]=$2; next} $2 in a && a[$2]==$1 {print $1"\t"$2}' \
+    A_vs_B.txt B_vs_A.txt > reciprocal_best_hits.txt
+```
+
+### Python RBH Script
+
+```python
+def find_rbh(forward_blast, reverse_blast):
+    '''Find reciprocal best hits from BLAST results'''
+    forward = {}
+    with open(forward_blast) as f:
+        for line in f:
+            parts = line.strip().split('\t')
+            query, subject = parts[0], parts[1]
+            if query not in forward:
+                forward[query] = subject
+
+    reverse = {}
+    with open(reverse_blast) as f:
+        for line in f:
+            parts = line.strip().split('\t')
+            query, subject = parts[0], parts[1]
+            if query not in reverse:
+                reverse[query] = subject
+
+    rbh = []
+    for a, b in forward.items():
+        if b in reverse and reverse[b] == a:
+            rbh.append((a, b))
+
+    return rbh
+
+rbh_pairs = find_rbh('A_vs_B.txt', 'B_vs_A.txt')
+for a, b in rbh_pairs:
+    print(f'{a}\t{b}')
+```
+
+## Delta-BLAST
+
+Uses conserved domain database for more sensitive initial search.
+
+```bash
+deltablast -query protein.fasta -db nr -rpsdb cdd_delta -out results.txt
+```
+
+## PHI-BLAST (Pattern-Hit Initiated)
+
+Search with a pattern plus sequence.
+
+```bash
+phi_pattern="G-x(2)-[ST]-x-[RK]"
+phiblast -query protein.fasta -db nr -pattern "$phi_pattern" -out results.txt
+```
+
+## Iterative Search with Biopython
+
+```python
+from Bio.Blast import NCBIWWW, NCBIXML
+
+with open('query.fasta') as f:
+    query = f.read()
+
+result_handle = NCBIWWW.qblast('psiblast', 'nr', query, expect=0.001, word_size=3)
+
+with open('psiblast_result.xml', 'w') as out:
+    out.write(result_handle.read())
+result_handle.close()
+
+with open('psiblast_result.xml') as f:
+    records = NCBIXML.parse(f)
+    for record in records:
+        for alignment in record.alignments:
+            for hsp in alignment.hsps:
+                if hsp.expect < 1e-10:
+                    print(f'{alignment.hit_def[:50]}: E={hsp.expect}')
+```
+
+## HMMER with Biopython
+
+```python
+from Bio import SearchIO
+
+results = SearchIO.parse('hmmsearch_output.txt', 'hmmer3-text')
+for query_result in results:
+    print(f'Query: {query_result.id}')
+    for hit in query_result:
+        print(f'  Hit: {hit.id}, E-value: {hit.evalue}')
+        for hsp in hit:
+            print(f'    Domain: {hsp.bitscore} bits')
+```
+
+## Jackhmmer (Iterative HMMER)
+
+Similar to PSI-BLAST but uses HMM profiles.
+
+```bash
+jackhmmer -N 5 -o results.txt --tblout hits.tbl query.fasta database.fasta
+jackhmmer -N 5 -A iterations.sto --chkhmm checkpoint query.fasta database.fasta
+```
+
+## OrthoFinder for Multi-Species Orthologs
+
+```bash
+orthofinder -f proteomes/ -t 8
+orthofinder -f proteomes/ -t 8 -M msa
+```
+
+### Prepare Input
+
+```bash
+mkdir proteomes
+cp species_*.fasta proteomes/
+```
+
+### Output Files
+
+| File | Content |
+|------|---------|
+| Orthogroups.tsv | All orthogroups |
+| Orthogroups_SingleCopyOrthologues.txt | 1:1 orthologs |
+| Species_Tree/ | Inferred species tree |
+| Gene_Trees/ | Individual gene trees |
+
+## E-value vs Bit Score
+
+| E-value | Interpretation |
+|---------|----------------|
+| < 1e-50 | Highly significant, likely homolog |
+| 1e-50 to 1e-10 | Significant, probable homolog |
+| 1e-10 to 1e-3 | Marginal, possible remote homolog |
+| > 0.01 | Not significant |
+
+## Complete Ortholog Finding Pipeline
+
+```bash
+#!/bin/bash
+SPECIES_A=$1
+SPECIES_B=$2
+EVALUE=1e-10
+THREADS=8
+
+echo "Building databases..."
+makeblastdb -in $SPECIES_A -dbtype prot -out db_A
+makeblastdb -in $SPECIES_B -dbtype prot -out db_B
+
+echo "Running forward BLAST..."
+blastp -query $SPECIES_A -db db_B -outfmt 6 -evalue $EVALUE \
+    -max_target_seqs 1 -num_threads $THREADS > forward.txt
+
+echo "Running reverse BLAST..."
+blastp -query $SPECIES_B -db db_A -outfmt 6 -evalue $EVALUE \
+    -max_target_seqs 1 -num_threads $THREADS > reverse.txt
+
+echo "Finding reciprocal best hits..."
+awk 'FNR==NR {best[$1]=$2; next}
+     $2 in best && best[$2]==$1 {print $1"\t"$2}' \
+     forward.txt reverse.txt > orthologs.txt
+
+echo "Found $(wc -l < orthologs.txt) ortholog pairs"
+
+rm -f db_A.* db_B.*
+```
+
+## Related Skills
+
+- blast-searches - Basic remote BLAST
+- local-blast - Local BLAST databases
+- entrez-fetch - Download sequences
+- alignment - Align identified homologs
+
+
+<!-- AUTHOR_SIGNATURE: 9a7f3c2e-MD-BABU-MIA-2026-MSSM-SECURE -->