@bgicli/bgicli 2.1.1 → 2.2.0

package/data/workflows/gwas-to-function-twas/references/installation-guide.md

@@ -0,0 +1,414 @@
+# Installation Guide: gwas-to-function-twas
+
+Comprehensive setup instructions for FUSION, S-PrediXcan, and all required
+dependencies.
+
+## Quick Start (S-PrediXcan - Recommended for Beginners)
+
+```bash
+# Create conda environment
+conda create -n metaxcan python=3.9
+conda activate metaxcan
+
+# Install S-PrediXcan
+pip install pandas numpy scipy plotnine plotnine-prism seaborn statsmodels metaxcan
+
+# Download expression weights (1 tissue for testing)
+mkdir -p weights/GTEx_v8
+cd weights/GTEx_v8
+wget https://predictdb.org/post/2021/07/21/gtex-v8-models-on-eqtl-and-sqtl/Whole_Blood.db
+
+# Ready to run!
+```
+
+**Expected time:** 10-15 minutes **Disk space:** ~2 GB (includes Python + 1
+tissue weight) **See:** [S-PrediXcan Setup](#s-predixcan-setup) below for
+complete instructions
+
+---
+
+## Installation Options
+
+### Option 1: S-PrediXcan (Fast, Python-based)
+
+**Pros:**
+
+- ✅ Fast installation (10-15 min)
+- ✅ Low memory requirements (<16 GB RAM)
+- ✅ 10-100x faster runtime than FUSION
+- ✅ Cross-platform (macOS, Linux, Windows via WSL)
+
+**Cons:**
+
+- ❌ Single prediction model (MASHR only)
+- ❌ Requires external colocalization tool
+
+**Best for:** Exploratory analysis, large-scale screens, local machines
+
+### Option 2: FUSION (Comprehensive, R-based)
+
+**Pros:**
+
+- ✅ Multiple prediction models (BSLMM, lasso, elastic net)
+- ✅ Built-in colocalization
+- ✅ Publication-standard tool
+
+**Cons:**
+
+- ❌ Complex R dependencies
+- ❌ High memory requirements (>32 GB RAM)
+- ❌ Slower runtime (2-6 hours per tissue)
+
+**Best for:** In-depth analysis, method comparisons, HPC environments
+
+---
+
+## S-PrediXcan Setup
+
+### Step 1: Create Python Environment
+
+**Using conda (recommended):**
+
+```bash
+conda create -n metaxcan python=3.9
+conda activate metaxcan
+```
+
+**Using venv:**
+
+```bash
+python3.9 -m venv metaxcan_env
+source metaxcan_env/bin/activate  # macOS/Linux
+# metaxcan_env\Scripts\activate  # Windows
+```
+
+### Step 2: Install Python Packages
+
+```bash
+pip install pandas numpy scipy statsmodels metaxcan
+pip install plotnine plotnine-prism seaborn  # For visualization
+```
+
+**Verify installation:**
+
+```bash
+python -c "import metaxcan; print(f'MetaXcan version: {metaxcan.__version__}')"
+```
+
+### Step 3: Download GTEx v8 Expression Weights
+
+**Option A: Manual download (recommended for transparency)**
+
+Visit https://predictdb.org/post/2021/07/21/gtex-v8-models-on-eqtl-and-sqtl/
+
+Download individual tissues or all 54 tissues:
+
+```bash
+mkdir -p weights/GTEx_v8
+cd weights/GTEx_v8
+
+# Example: Download Whole_Blood
+wget https://predictdb.org/post/2021/07/21/gtex-v8-models-on-eqtl-and-sqtl/mashr/mashr_Whole_Blood.db
+wget https://predictdb.org/post/2021/07/21/gtex-v8-models-on-eqtl-and-sqtl/mashr/mashr_Whole_Blood.txt.gz
+
+# Repeat for other tissues or download all with:
+wget -r -np -nH --cut-dirs=6 -A "*.db,*.txt.gz" https://predictdb.org/post/2021/07/21/gtex-v8-models-on-eqtl-and-sqtl/mashr/
+```
+
+**Option B: Use helper script**
+
+```python
+from assets.eval.eval_helpers import load_gtex_weights
+weights = load_gtex_weights(tissues=["Whole_Blood", "Artery_Coronary"])
+```
+
+**Disk space:** ~100 MB per tissue, ~5 GB for all 54 tissues
+
+### Step 4: Download LD Reference (for covariance calculation)
+
+**Option A: Use pre-computed covariances (recommended)**
+
+PredictDB provides pre-computed LD matrices:
+
+```bash
+cd weights/GTEx_v8
+wget https://predictdb.org/post/2021/07/21/gtex-v8-models-on-eqtl-and-sqtl/gtex_v8_expression_mashr_ld_matrices.tar.gz
+tar xzf gtex_v8_expression_mashr_ld_matrices.tar.gz
+```
+
+**Option B: Download 1000 Genomes genotypes**
+
+```bash
+mkdir -p 1000G_EUR
+cd 1000G_EUR
+wget https://zenodo.org/record/3518299/files/1000G_EUR.tar.gz
+tar xzf 1000G_EUR.tar.gz
+```
+
+**Disk space:** ~3 GB (pre-computed) or ~5 GB (full genotypes)
+
+### Step 5: Test Installation
+
+```bash
+# Test S-PrediXcan
+python -c "from metaxcan import Utilities; print('S-PrediXcan ready!')"
+
+# Quick test run (requires test GWAS data)
+python scripts/run_spredixxcan.py --help
+```
+
+---
+
+## FUSION Setup
+
+### Step 1: Install R and Required Packages
+
+**Install R ≥4.0:**
+
+- macOS: `brew install r` or download from https://cran.r-project.org/
+- Linux: `sudo apt-get install r-base` (Ubuntu/Debian) or equivalent
+- Verify: `R --version`
+
+**Install R packages:**
+
+```R
+# Launch R
+R
+
+# Install FUSION dependencies
+install.packages(c('optparse', 'RColorBrewer', 'data.table', 'glmnet'))
+install.packages(c('reshape', 'ggplot2', 'Rcpp', 'RcppArmadillo'))
+```
+
+### Step 2: Clone FUSION Repository
+
+```bash
+cd ~/software  # Or your preferred location
+git clone https://github.com/gusevlab/fusion_twas.git
+cd fusion_twas
+```
+
+### Step 3: Download FUSION LDREF Panel
+
+```bash
+cd ~/software/fusion_twas
+
+# Download LD reference (~2 GB compressed, ~5 GB uncompressed)
+wget https://data.broadinstitute.org/alkesgroup/FUSION/LDREF.tar.bz2
+tar xjvf LDREF.tar.bz2
+
+# Verify
+ls LDREF/1000G.EUR.*/  # Should show .bim, .bed, .fam files
+```
+
+### Step 4: Download GTEx v8 Weights (FUSION format)
+
+```bash
+mkdir -p weights/GTEx_v8_FUSION
+cd weights/GTEx_v8_FUSION
+
+# Download all GTEx v8 tissues from FUSION website
+wget http://gusevlab.org/projects/fusion/gtex_v8_fusion_weights.tar.bz2
+tar xjvf gtex_v8_fusion_weights.tar.bz2
+
+# Or download individual tissues
+# See: http://gusevlab.org/projects/fusion/#reference-datasets
+```
+
+**Disk space:** ~10 GB for all tissues
+
+### Step 5: Set Environment Variables
+
+Add to `~/.bashrc` or `~/.zshrc`:
+
+```bash
+export FUSION_PATH=~/software/fusion_twas
+export PATH=$FUSION_PATH:$PATH
+```
+
+Reload: `source ~/.bashrc` or `source ~/.zshrc`
+
+### Step 6: Test FUSION Installation
+
+```bash
+# Test FUSION help
+Rscript $FUSION_PATH/FUSION.assoc_test.R --help
+
+# Quick test (requires test GWAS data)
+python scripts/run_fusion.py --help
+```
+
+---
+
+## Common Installation Issues
+
+### Issue 1: Python package conflicts
+
+**Error:** `ERROR: metaxcan X.X requires pandas>=Y.Y, but you have pandas Z.Z`
+
+**Solution:**
+
+```bash
+# Create fresh conda environment
+conda create -n metaxcan_clean python=3.9
+conda activate metaxcan_clean
+pip install metaxcan --no-cache-dir
+```
+
+### Issue 2: R package installation fails
+
+**Error:** `installation of package 'X' had non-zero exit status`
+
+**Solution (macOS):**
+
+```bash
+# Install Xcode command line tools
+xcode-select --install
+
+# Install gfortran (required for some R packages)
+brew install gcc
+```
+
+**Solution (Linux):**
+
+```bash
+sudo apt-get install libcurl4-openssl-dev libssl-dev libxml2-dev
+```
+
+### Issue 3: FUSION LDREF download slow/fails
+
+**Error:** `wget: unable to resolve host address`
+
+**Solution:**
+
+```bash
+# Use curl instead
+curl -O https://data.broadinstitute.org/alkesgroup/FUSION/LDREF.tar.bz2
+
+# Or use aria2c for faster download with resume capability
+brew install aria2  # macOS
+sudo apt-get install aria2  # Linux
+aria2c -x 16 -s 16 https://data.broadinstitute.org/alkesgroup/FUSION/LDREF.tar.bz2
+```
+
+### Issue 4: GTEx weights download incomplete
+
+**Error:** `wget recursive download gets stuck`
+
+**Solution:**
+
+```bash
+# Download individual tissues manually
+# List of all 54 GTEx v8 tissues at: https://predictdb.org/
+# Or use the download script provided in assets/eval/eval_helpers.py
+```
+
+### Issue 5: Memory errors during FUSION
+
+**Error:** `Error: cannot allocate vector of size X GB`
+
+**Solution:**
+
+- Use S-PrediXcan instead (10x less memory)
+- Run FUSION chromosome-by-chromosome
+- Increase swap space (not recommended for performance)
+- Use HPC with >32 GB RAM
+
+---
+
+## Validation Checklist
+
+After installation, verify:
+
+- [ ] **Python environment active**: `which python` shows correct path
+- [ ] **Python packages installed**: `pip list | grep metaxcan`
+- [ ] **R installed (if using FUSION)**: `R --version`
+- [ ] **R packages available**: `R -e "library(optparse)"`
+- [ ] **Expression weights downloaded**: `ls weights/GTEx_v8/*.db` (S-PrediXcan)
+      or `ls weights/GTEx_v8_FUSION/*.pos` (FUSION)
+- [ ] **LD reference available**: `ls LDREF/` or
+      `ls weights/GTEx_v8/gtex_v8_ld/`
+- [ ] **Scripts executable**: `python scripts/run_spredixxcan.py --help`
+- [ ] **Test data available**:
+      `python -c "from assets.eval.eval_helpers import load_test_gwas; load_test_gwas()"`
+
+---
+
+## Disk Space Requirements
+
+**Minimal (chr22 only, 1 tissue):**
+
+- Python/R + packages: ~2 GB
+- 1 tissue weights: ~100 MB
+- LD reference (minimal): ~500 MB
+- **Total: ~3 GB**
+
+**Standard (genome-wide, 5 tissues):**
+
+- Python/R + packages: ~2 GB
+- 5 tissue weights: ~500 MB
+- LD reference (full): ~5 GB
+- **Total: ~8 GB**
+
+**Complete (all 54 tissues, both tools):**
+
+- Python/R + packages: ~3 GB
+- All tissue weights (S-PrediXcan): ~5 GB
+- All tissue weights (FUSION): ~10 GB
+- LD references (both): ~8 GB
+- **Total: ~26 GB**
+
+---
+
+## Computational Requirements
+
+**S-PrediXcan:**
+
+- CPU: 2+ cores (benefits from 4-8 cores)
+- RAM: 8 GB minimum, 16 GB recommended
+- Disk: See above
+- Runtime: 5-30 minutes per tissue (genome-wide)
+
+**FUSION:**
+
+- CPU: 4+ cores recommended
+- RAM: 32 GB minimum, 64 GB recommended
+- Disk: See above
+- Runtime: 2-6 hours per tissue (genome-wide)
+
+---
+
+## Next Steps
+
+After installation:
+
+1. **Test with example data**: Run `python scripts/run_spredixxcan.py --help`
+2. **Download test GWAS**: See
+   [assets/eval/datasets/README.md](../assets/eval/datasets/README.md)
+3. **Run quick test**: Execute `python assets/eval/test_tier1_basic.py` (when
+   implemented)
+4. **Read usage guides**:
+   [spredixxcan_best_practices.md](spredixxcan_best_practices.md) or
+   [fusion_best_practices.md](fusion_best_practices.md)
+
+---
+
+## Additional Resources
+
+**S-PrediXcan:**
+
+- Documentation: https://github.com/hakyimlab/MetaXcan
+- PredictDB: https://predictdb.org/
+- Tutorial: https://github.com/hakyimlab/MetaXcan/wiki
+
+**FUSION:**
+
+- Documentation: http://gusevlab.org/projects/fusion/
+- GitHub: https://github.com/gusevlab/fusion_twas
+- Paper: Gusev A, et al. (2016) _Nat Genet_ 48:245-252
+
+**GTEx:**
+
+- Portal: https://gtexportal.org/
+- V8 release notes: https://gtexportal.org/home/releaseInfoPage
+- eQTL data: https://gtexportal.org/home/datasets

package/data/workflows/gwas-to-function-twas/references/ldsc_qc_guidelines.md

@@ -0,0 +1,287 @@
+# LDSC QC Guidelines for GWAS Summary Statistics
+
+This document provides detailed guidance on interpreting LDSC (LD Score
+Regression) intercept tests to distinguish polygenicity from population
+stratification in GWAS summary statistics.
+
+---
+
+## Overview
+
+LDSC intercept analysis is the gold standard for assessing whether genomic
+inflation (λGC > 1) in GWAS results is due to:
+
+- **Polygenicity**: True polygenic architecture (many small-effect variants)
+- **Confounding**: Population stratification, batch effects, or cryptic
+  relatedness
+
+**Key principle:** LDSC separates these two sources of inflation by leveraging
+LD patterns.
+
+---
+
+## Running LDSC Intercept Test
+
+### Basic Command
+
+```bash
+python ldsc.py \
+  --h2 gwas_sumstats.txt \
+  --ref-ld-chr eur_w_ld_chr/ \
+  --w-ld-chr eur_w_ld_chr/ \
+  --out gwas_qc
+```
+
+### Required Input Format
+
+GWAS summary statistics must have these columns:
+
+- `SNP`: rsID or chr:pos
+- `A1`: Effect allele
+- `A2`: Reference allele
+- `Z`: Z-score (or compute from BETA/SE)
+- `N`: Sample size
+- `P`: P-value (optional)
+
+### LD Score Files
+
+Download from LDSC website:
+
+- **EUR**: European LD scores (1000 Genomes EUR)
+- **EAS**: East Asian LD scores
+- **AFR**: African LD scores
+
+Use LD scores that match your GWAS ancestry.
+
+---
+
+## Interpreting LDSC Output
+
+### Key Metrics
+
+1. **LDSC Intercept**: Estimated intercept from LD score regression
+2. **Intercept SE**: Standard error of intercept estimate
+3. **Ratio**: Proportion of inflation due to confounding
+
+### Output Example
+
+```
+Mean chi^2 = 1.456
+Lambda GC = 1.392
+Intercept: 1.023 (0.008)
+Ratio: 0.086 (0.032)
+```
+
+---
+
+## Interpretation Guidelines
+
+### Intercept Values
+
+| Intercept     | Ratio     | Interpretation                                          | Action                                       |
+| ------------- | --------- | ------------------------------------------------------- | -------------------------------------------- |
+| **≈ 1.0**     | < 0.10    | ✅ **Good quality** - Inflation is polygenic            | Proceed with analysis                        |
+| **1.0-1.05**  | 0.10-0.15 | ⚠️ **Acceptable** - Mostly polygenic, minor confounding | Acceptable for TWAS                          |
+| **1.05-1.10** | 0.15-0.30 | ⚠️ **Moderate issues** - Substantial confounding        | Consider re-QC or apply intercept correction |
+| **> 1.10**    | > 0.30    | ❌ **Poor quality** - High confounding                  | Re-QC GWAS before TWAS                       |
+
+### Ratio Interpretation
+
+**Ratio = (Intercept - 1) / (λGC - 1)**
+
+- **Ratio < 0.10**: Less than 10% of inflation due to confounding (excellent)
+- **Ratio = 0.10-0.30**: 10-30% confounding (acceptable)
+- **Ratio > 0.30**: More than 30% confounding (problematic)
+
+---
+
+## Common Scenarios
+
+### Scenario 1: High λGC, Low Intercept (Good)
+
+```
+Lambda GC = 1.5
+Intercept = 1.02
+Ratio = 0.04
+```
+
+**Interpretation**: High inflation is due to polygenicity (many true
+associations), not confounding.
+
+**Action**: ✅ Proceed with TWAS - this is ideal for well-powered polygenic
+traits.
+
+### Scenario 2: Moderate λGC, High Intercept (Bad)
+
+```
+Lambda GC = 1.2
+Intercept = 1.12
+Ratio = 0.60
+```
+
+**Interpretation**: Most inflation is due to confounding, not true polygenic
+signal.
+
+**Action**: ❌ **Do not proceed** with TWAS until confounding is resolved.
+
+**Possible causes:**
+
+- Population stratification (insufficient PC adjustment)
+- Batch effects
+- Cryptic relatedness
+- Genotyping errors
+
+**Solutions:**
+
+1. Re-run GWAS with more principal components
+2. Check for batch effects
+3. Filter samples with high relatedness
+4. Apply genomic control correction
+
+### Scenario 3: Low λGC, Intercept ≈ 1 (Underpowered)
+
+```
+Lambda GC = 1.02
+Intercept = 1.00
+Ratio = 0.00
+```
+
+**Interpretation**: No inflation - either well-controlled or underpowered GWAS.
+
+**Action**:
+
+- Check sample size (N > 5,000 recommended)
+- If N is large, trait may be weakly heritable
+- TWAS may have limited power to detect associations
+
+---
+
+## When LDSC Detects Confounding
+
+### Step 1: Identify Source of Confounding
+
+Common sources:
+
+1. **Population stratification**
+   - Check PCA plots for outliers
+   - Increase number of PCs in GWAS model
+   - Consider restricting to homogeneous ancestry
+
+2. **Batch effects**
+   - Check for genotyping batch differences
+   - Include batch covariates in GWAS
+
+3. **Cryptic relatedness**
+   - Use KING or PLINK to identify related samples
+   - Remove one sample from each related pair (pi-hat > 0.2)
+
+4. **Low-quality genotypes**
+   - Stricter QC filters (call rate > 99%, HWE p > 1e-6)
+   - Remove low-quality samples
+
+### Step 2: Apply Corrections
+
+**Option A: Re-run GWAS** (preferred)
+
+- Fix underlying QC issues
+- Re-run association analysis with corrected data
+
+**Option B: Apply LDSC Intercept Correction** (if re-running not feasible)
+
+- Divide chi-square statistics by LDSC intercept
+- `chi2_corrected = chi2_observed / intercept`
+- Updates P-values to remove confounding inflation
+
+---
+
+## LDSC for Different Ancestries
+
+### European Ancestry
+
+Use EUR LD scores (most common):
+
+```bash
+--ref-ld-chr eur_w_ld_chr/
+```
+
+### East Asian Ancestry
+
+Use EAS LD scores:
+
+```bash
+--ref-ld-chr eas_ldscores/
+```
+
+### African Ancestry
+
+Use AFR LD scores:
+
+```bash
+--ref-ld-chr afr_ldscores/
+```
+
+### Admixed or Multi-Ancestry
+
+- Use LD scores matching the largest ancestry component
+- Or compute custom LD scores from your study population
+
+---
+
+## Integration with TWAS Workflow
+
+### When to Run LDSC Intercept QC
+
+**Timing**: Before running TWAS (Step 1 of workflow)
+
+**Why critical for TWAS**:
+
+- TWAS inherits GWAS confounding
+- Population stratification can cause false positive gene associations
+- LDSC ensures GWAS is clean before imputing gene expression
+
+### Decision Tree
+
+```
+Run LDSC Intercept Test
+         |
+         ├─> Ratio < 0.15? ──> ✅ Proceed with TWAS
+         |
+         └─> Ratio ≥ 0.15? ──> ⚠️ Investigate confounding
+                                    |
+                                    ├─> Can re-QC GWAS? ──> Re-run GWAS ──> Repeat LDSC
+                                    |
+                                    └─> Cannot re-QC? ──> Apply intercept correction ──> Proceed with caution
+```
+
+---
+
+## Advanced: Partitioned LDSC for Tissue Selection
+
+For rigorous tissue selection (recommended for Tier 2+ analyses), use LDSC
+partitioned heritability:
+
+**See [gwas-heritability-ldsc](../../gwas-heritability-ldsc/) workflow** for:
+
+- Identifying tissues enriched for trait heritability
+- Statistical evidence for tissue relevance
+- Data-driven tissue recommendations for TWAS
+
+This provides unbiased tissue selection compared to biology-based assumptions.
+
+---
+
+## References
+
+- **LDSC Method**: Bulik-Sullivan BK, et al. (2015) LD Score regression
+  distinguishes confounding from polygenicity in genome-wide association
+  studies. _Nat Genet_ 47:291-295.
+  [doi:10.1038/ng.3211](https://doi.org/10.1038/ng.3211)
+
+- **Partitioned Heritability**: Finucane HK, et al. (2015) Partitioning
+  heritability by functional annotation using genome-wide association summary
+  statistics. _Nat Genet_ 47:1228-1235.
+  [doi:10.1038/ng.3404](https://doi.org/10.1038/ng.3404)
+
+---
+
+**Last Updated:** 2026-01-28