@bgicli/bgicli 2.1.0 → 2.2.0

package/data/skills/bio-genome-assembly-metagenome-assembly/SKILL.md

@@ -0,0 +1,227 @@
+<!--
+# COPYRIGHT NOTICE
+# This file is part of the "Universal Biomedical Skills" project.
+# Copyright (c) 2026 MD BABU MIA, PhD <md.babu.mia@mssm.edu>
+# All Rights Reserved.
+#
+# This code is proprietary and confidential.
+# Unauthorized copying of this file, via any medium is strictly prohibited.
+#
+# Provenance: Authenticated by MD BABU MIA
+
+-->
+
+---
+name: bio-genome-assembly-metagenome-assembly
+description: Metagenome assembly from long reads using metaFlye and metaSPAdes with binning strategies. Use when reconstructing genomes from microbial communities, recovering metagenome-assembled genomes (MAGs), or resolving strain-level variation in complex samples.
+tool_type: cli
+primary_tool: metaFlye
+measurable_outcome: Execute skill workflow successfully with valid output within 15 minutes.
+allowed-tools:
+  - read_file
+  - run_shell_command
+---
+
+# Metagenome Assembly
+
+## Overview
+
+Metagenome assembly reconstructs genomes from mixed microbial communities. Long reads enable recovery of complete circular genomes and resolution of strain-level differences.
+
+## metaFlye (Long Reads)
+
+```bash
+# ONT metagenome assembly
+flye --nano-raw reads.fastq.gz \
+    --meta \
+    --out-dir flye_meta \
+    --threads 32
+
+# PacBio HiFi metagenome
+flye --pacbio-hifi reads.hifi.fastq.gz \
+    --meta \
+    --out-dir flye_meta_hifi \
+    --threads 32
+
+# Key output files:
+# assembly.fasta - assembled contigs
+# assembly_graph.gfa - assembly graph
+# assembly_info.txt - contig statistics
+```
+
+## metaSPAdes (Short Reads)
+
+```bash
+# Illumina paired-end metagenome
+metaspades.py -1 R1.fastq.gz -2 R2.fastq.gz \
+    -o spades_meta \
+    -t 32 \
+    -m 500
+
+# With multiple libraries
+metaspades.py \
+    --pe1-1 lib1_R1.fq.gz --pe1-2 lib1_R2.fq.gz \
+    --pe2-1 lib2_R1.fq.gz --pe2-2 lib2_R2.fq.gz \
+    -o spades_meta -t 32
+```
+
+## Hybrid Assembly
+
+```bash
+# Combine short and long reads
+flye --nano-raw ont_reads.fastq.gz \
+    --meta \
+    --out-dir flye_hybrid \
+    --threads 32
+
+# Polish with short reads
+pilon --genome flye_hybrid/assembly.fasta \
+    --frags short_reads.bam \
+    --output polished \
+    --threads 16
+```
+
+## Key Parameters
+
+### metaFlye
+
+| Parameter | Description |
+|-----------|-------------|
+| --meta | Metagenome mode (handles uneven coverage) |
+| --min-overlap | Minimum overlap for assembly (default: auto) |
+| --genome-size | Estimated total size (optional for meta) |
+| --iterations | Polishing iterations (default: 1) |
+| --keep-haplotypes | Preserve strain variants |
+
+### metaSPAdes
+
+| Parameter | Description |
+|-----------|-------------|
+| -m | Memory limit in GB |
+| --only-assembler | Skip error correction |
+| -k | K-mer sizes (auto-selected by default) |
+| --phred-offset | Quality encoding (33 or 64) |
+
+## Binning Workflow
+
+```bash
+# Step 1: Map reads back to assembly
+minimap2 -ax map-ont -t 32 assembly.fasta reads.fastq.gz | \
+    samtools sort -o mapped.bam -
+
+# Step 2: Generate depth file
+jgi_summarize_bam_contig_depths --outputDepth depth.txt mapped.bam
+
+# Step 3: Bin with MetaBAT2
+metabat2 -i assembly.fasta -a depth.txt -o bins/bin -t 32
+
+# Step 4: Assess bin quality with CheckM2
+checkm2 predict --input bins/ --output-directory checkm2_out -x fa --threads 32
+```
+
+## SemiBin2 (Deep Learning Binning)
+
+```bash
+# Single-sample binning
+SemiBin2 single_easy_bin \
+    -i assembly.fasta \
+    -b mapped.bam \
+    -o semibin_out \
+    --environment global
+
+# Multi-sample binning (better for time-series)
+SemiBin2 multi_easy_bin \
+    -i assembly.fasta \
+    -b sample1.bam sample2.bam sample3.bam \
+    -o semibin_multi
+```
+
+## Quality Assessment
+
+```bash
+# Assembly stats
+seqkit stats assembly.fasta
+
+# CheckM2 for bin completeness
+checkm2 predict -i bins/ -o checkm2_out -x fa -t 32
+
+# GTDB-Tk for taxonomic classification
+gtdbtk classify_wf --genome_dir bins/ --out_dir gtdbtk_out --cpus 32
+
+# QUAST for assembly metrics
+metaquast.py -o metaquast_out assembly.fasta -t 32
+```
+
+## Circular Genome Detection
+
+```bash
+# Flye marks circular contigs in assembly_info.txt
+grep "Y" flye_meta/assembly_info.txt | cut -f1 > circular_contigs.txt
+
+# Extract circular contigs
+seqkit grep -f circular_contigs.txt assembly.fasta > circular_genomes.fasta
+```
+
+## Python Pipeline
+
+```python
+import subprocess
+from pathlib import Path
+import pandas as pd
+
+def run_metaflye(reads, output_dir, read_type='nano-raw', threads=32):
+    cmd = ['flye', f'--{read_type}', reads, '--meta', '--out-dir', output_dir, '--threads', str(threads)]
+    subprocess.run(cmd, check=True)
+    return Path(output_dir) / 'assembly.fasta'
+
+def run_binning(assembly, bam, output_dir, threads=32):
+    depth_file = Path(output_dir) / 'depth.txt'
+    subprocess.run(['jgi_summarize_bam_contig_depths', '--outputDepth', str(depth_file), bam], check=True)
+
+    bins_dir = Path(output_dir) / 'bins'
+    bins_dir.mkdir(exist_ok=True)
+    subprocess.run(['metabat2', '-i', assembly, '-a', str(depth_file), '-o', str(bins_dir / 'bin'), '-t', str(threads)], check=True)
+
+    return bins_dir
+
+def assess_bins(bins_dir, output_dir, threads=32):
+    subprocess.run(['checkm2', 'predict', '--input', str(bins_dir), '--output-directory', output_dir, '-x', 'fa', '--threads', str(threads)], check=True)
+
+    results = pd.read_csv(Path(output_dir) / 'quality_report.tsv', sep='\t')
+    high_quality = results[(results['Completeness'] > 90) & (results['Contamination'] < 5)]
+    return high_quality
+
+# Example workflow
+assembly = run_metaflye('ont_reads.fq.gz', 'flye_out')
+bins = run_binning(str(assembly), 'mapped.bam', 'binning_out')
+hq_bins = assess_bins(bins, 'checkm2_out')
+print(f'High-quality MAGs: {len(hq_bins)}')
+```
+
+## Expected Outputs
+
+| Metric | Good Assembly |
+|--------|---------------|
+| N50 | >50 kb |
+| Largest contig | >1 Mb |
+| HQ MAGs (>90% complete, <5% contam) | Varies by sample |
+| Circular genomes | Sample dependent |
+
+## Troubleshooting
+
+| Issue | Solution |
+|-------|----------|
+| Few long contigs | Increase read depth or length |
+| High chimeric rate | Use --keep-haplotypes in Flye |
+| Poor binning | Add more samples for differential coverage |
+| Missing taxa | Check read QC; consider targeted enrichment |
+
+## Related Skills
+
+- genome-assembly/contamination-detection - CheckM2/GUNC
+- metagenomics/taxonomic-profiling - Kraken2/Bracken
+- metagenomics/functional-profiling - HUMAnN
+- long-read-sequencing/read-qc - Input quality control
+
+
+<!-- AUTHOR_SIGNATURE: 9a7f3c2e-MD-BABU-MIA-2026-MSSM-SECURE -->

package/data/skills/bio-genome-assembly-scaffolding/SKILL.md

@@ -0,0 +1,204 @@
+<!--
+# COPYRIGHT NOTICE
+# This file is part of the "Universal Biomedical Skills" project.
+# Copyright (c) 2026 MD BABU MIA, PhD <md.babu.mia@mssm.edu>
+# All Rights Reserved.
+#
+# This code is proprietary and confidential.
+# Unauthorized copying of this file, via any medium is strictly prohibited.
+#
+# Provenance: Authenticated by MD BABU MIA
+
+-->
+
+---
+name: bio-genome-assembly-scaffolding
+description: Scaffold contigs into chromosome-level assemblies using Hi-C data with YaHS, 3D-DNA, SALSA2, and validate with BUSCO and contact maps. Use when scaffolding contigs to chromosome-level assemblies.
+tool_type: cli
+primary_tool: YaHS
+measurable_outcome: Execute skill workflow successfully with valid output within 15 minutes.
+allowed-tools:
+  - read_file
+  - run_shell_command
+---
+
+# Genome Scaffolding
+
+## Hi-C Data Preprocessing
+
+```bash
+# Align Hi-C reads to draft assembly
+bwa index draft_assembly.fa
+bwa mem -5SP -t 16 draft_assembly.fa hic_R1.fq.gz hic_R2.fq.gz | \
+    samtools view -@ 8 -bhS - > aligned.bam
+
+# Filter for Hi-C contacts (pairtools)
+pairtools parse --min-mapq 40 --walks-policy 5unique --max-inter-align-gap 30 \
+    --nproc-in 8 --nproc-out 8 --chroms-path draft_assembly.fa.fai aligned.bam | \
+    pairtools sort --nproc 8 | \
+    pairtools dedup --nproc 8 --mark-dups | \
+    pairtools split --output-pairs contacts.pairs.gz
+```
+
+## YaHS Scaffolding (Recommended)
+
+```bash
+# Index assembly
+samtools faidx draft_assembly.fa
+
+# Convert BAM to BED
+bedtools bamtobed -i aligned.bam | sort -k4 > aligned.bed
+
+# Run YaHS
+yahs draft_assembly.fa aligned.bed -o scaffolds
+
+# Output files:
+# scaffolds_scaffolds_final.fa - final scaffolds
+# scaffolds_scaffolds_final.agp - AGP file
+# scaffolds.bin - contact matrix
+```
+
+## YaHS with Error Correction
+
+```bash
+# Run with error correction
+yahs draft_assembly.fa aligned.bed -o scaffolds --no-contig-ec
+
+# Generate contact map for juicebox
+juicer pre scaffolds.bin scaffolds_scaffolds_final.agp draft_assembly.fa.fai | \
+    sort -k2,2d -k6,6d -T ./ --parallel=8 -S 50G | \
+    awk 'NF' > scaffolds.pre.txt
+
+# Create .hic file
+java -Xmx48G -jar juicer_tools.jar pre scaffolds.pre.txt scaffolds.hic \
+    <(cut -f1,2 scaffolds_scaffolds_final.fa.fai)
+```
+
+## 3D-DNA Pipeline
+
+```bash
+# Prepare input (requires Juicer aligned data)
+# Run Juicer first to get merged_nodups.txt
+
+# Run 3D-DNA
+run-asm-pipeline.sh -r 2 draft_assembly.fa merged_nodups.txt
+
+# Output: draft_assembly.final.fasta
+
+# Generate review assembly for Juicebox
+run-asm-pipeline-post-review.sh -r draft_assembly.final.review.assembly \
+    draft_assembly.final.fasta merged_nodups.txt
+```
+
+## SALSA2 Scaffolding
+
+```bash
+# Run SALSA2
+python run_pipeline.py -a draft_assembly.fa -l draft_assembly.fa.fai \
+    -b aligned.bed -e GATC -o salsa_output -m yes
+
+# With multiple restriction enzymes
+python run_pipeline.py -a draft_assembly.fa -l draft_assembly.fa.fai \
+    -b aligned.bed -e GATC,GANTC -o salsa_output -m yes -p yes
+```
+
+## Generate Contact Map
+
+```bash
+# Using cooler
+cooler cload pairs -c1 2 -p1 3 -c2 4 -p2 5 \
+    draft_assembly.fa.fai:10000 contacts.pairs.gz scaffolds.cool
+
+# Balance matrix
+cooler balance scaffolds.cool
+
+# Multi-resolution (mcool)
+cooler zoomify scaffolds.cool -o scaffolds.mcool
+```
+
+## Visualize with HiGlass
+
+```bash
+# Convert to higlass format
+clodius aggregate bedfile --chromsizes-filename chrom.sizes \
+    --output-file scaffolds.beddb scaffold_boundaries.bed
+
+# Load into higlass server
+docker run --detach --publish 8888:80 \
+    --volume ~/hg-data:/data \
+    higlass/higlass-docker:latest
+```
+
+## Manual Curation (Juicebox)
+
+```bash
+# Load .hic file in Juicebox Assembly Tools (JBAT)
+# Perform manual corrections:
+# - Break misjoins
+# - Order/orient scaffolds
+# - Merge scaffolds
+
+# Export corrected assembly
+# File -> Export Assembly -> FASTA
+```
+
+## Post-Scaffolding Gap Filling
+
+```bash
+# TGS-GapCloser for long-read gap filling
+tgsgapcloser --scaff scaffolds.fa --reads ont_reads.fq.gz \
+    --output filled --thread 16 --ne
+
+# LR_Gapcloser alternative
+LR_Gapcloser.sh -i scaffolds.fa -l ont_reads.fq.gz -t 16 -o gapclosed.fa
+```
+
+## Validate Scaffolding
+
+```bash
+# Check chromosome-scale contiguity
+seqkit stats scaffolds.fa
+
+# BUSCO on scaffolds
+busco -i scaffolds.fa -l eukaryota_odb10 -o busco_scaffolds -m genome -c 16
+
+# N50/L50 statistics
+assembly-stats scaffolds.fa
+
+# Compare pre/post scaffolding
+quast.py draft_assembly.fa scaffolds.fa -o quast_comparison
+```
+
+## Check Telomeres
+
+```bash
+# Find telomeric repeats (vertebrate TTAGGG)
+seqkit locate -i -p 'TTAGGG{10,}' scaffolds.fa > telomeres_forward.bed
+seqkit locate -i -p 'CCCTAA{10,}' scaffolds.fa > telomeres_reverse.bed
+
+# Count chromosomes with telomeres on both ends
+awk '$2 < 1000' telomeres_forward.bed | cut -f1 | sort -u > left_telomeres.txt
+awk -v OFS='\t' 'NR==FNR{len[$1]=$2;next} $3 > len[$1]-1000' \
+    scaffolds.fa.fai telomeres_reverse.bed | cut -f1 | sort -u > right_telomeres.txt
+comm -12 left_telomeres.txt right_telomeres.txt > complete_chromosomes.txt
+```
+
+## Rename to Chromosomes
+
+```bash
+# After manual curation, rename scaffolds to chromosomes
+awk '/^>/{print ">chr" ++i; next}{print}' scaffolds.fa > chromosomes.fa
+
+# Or with mapping file
+seqkit replace -p '(.+)' -r '{kv}' -k name_mapping.tsv scaffolds.fa > chromosomes.fa
+```
+
+## Related Skills
+
+- genome-assembly/long-read-assembly - Generate initial contigs
+- genome-assembly/assembly-polishing - Polish before scaffolding
+- genome-assembly/assembly-qc - Validate final assembly
+- hi-c-analysis/hic-data-io - Hi-C data processing
+
+
+<!-- AUTHOR_SIGNATURE: 9a7f3c2e-MD-BABU-MIA-2026-MSSM-SECURE -->