biopytools 0.21.1__tar.gz

biopytools-0.21.1/PKG-INFO

@@ -0,0 +1,27 @@
+Metadata-Version: 2.4
+Name: biopytools
+Version: 0.21.1
+Summary: Bioinformatics tools package with Python.
+Author-email: Xiang LI <lixiang117423@gmail.com>
+Project-URL: Homepage, https://github.com/lixiang117423/biopytools
+Project-URL: Documentation, https://lixiang117423.github.io/article/biopytools-readme/
+Project-URL: Repository, https://github.com/lixiang117423/biopytools.git
+Project-URL: Issues, https://github.com/lixiang117423/biopytools/issues
+Requires-Python: >=3.10
+Requires-Dist: pandas
+Requires-Dist: numpy
+Requires-Dist: pyfastx>=0.8.4
+Requires-Dist: scikit-learn>=1.0.0
+Requires-Dist: matplotlib>=3.5.0
+Requires-Dist: seaborn>=0.11.0
+Requires-Dist: click>=8.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=6.0; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: isort; extra == "dev"
+Requires-Dist: flake8; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs; extra == "docs"
+Requires-Dist: mkdocs-material; extra == "docs"
+Requires-Dist: mkdocs-mermaid2-plugin; extra == "docs"

biopytools-0.21.1/README.md

@@ -0,0 +1,114 @@
+# BioPyTools
+
+A Python toolkit for bioinformatics analysis and computational biology.
+
+一个用于生物信息学分析和计算生物学的Python工具包。
+
+## 简介 | Overview
+
+BioPyTools 是一个专为生物信息学研究设计的Python工具包，提供了一系列常用的生物数据分析功能。
+
+BioPyTools is a Python toolkit designed for bioinformatics research, providing a series of commonly used biological data analysis functions.
+
+## 系统要求 | Requirements
+
+- Python >= 3.8
+- NumPy >= 1.19.0
+- Pandas >= 1.2.0
+- Matplotlib >= 3.3.0
+
+## 环境配置 / Environment Setup
+
+相关的Conda 环境配置文件位于 [`conda_env/`](conda_env/) 目录下。
+
+Conda environment files can be found in the [`conda_env/`](conda_env/) directory.
+
+## 安装方法 | Installation
+
+### 从源码安装 | Install from source
+
+```bash
+git clone https://github.com/lixiang117423/biopytools.git
+cd biopytools
+pip install -e .
+
+# or
+pip install .
+```
+
+## 使用方法 | Usage
+
+### 查看帮助 | Getting Help
+
+```bash
+biopytools -h
+Usage: biopytools [OPTIONS] COMMAND [ARGS]...
+
+  BioPyTools - 生物信息学分析工具包
+
+  要查看特定命令的帮助，请运行：biopytools <命令> -h/--help, 如biopytools annovar -h
+
+Options:
+  -v, --version  Show the version and exit.
+  -h, --help     Show this message and exit.
+```
+
+## 模块文档 | README
+
+[admixture](./docs/admixture.md) - [群体结构分析软件Admixture](https://genome.cshlp.org/content/19/9/1655)
+
+[agp2table](./docs/agp2table.md) - agp文件转换为table
+
+[annovar](./docs/annovar.md) - [ANNOVAR变异功能注释工具](https://academic.oup.com/nar/article/38/16/e164/1749458)
+
+[bam-cov](./docs/bam_coverage_stats.md) - BAM文件覆盖度统计
+
+[blast](./docs/blast_v2.md) - [序列比对工具](https://academic.oup.com/nar/article/36/suppl_2/W5/2505810?login=true)
+
+[busco](./docs/busco.md) - [BUSCO](https://academic.oup.com/bioinformatics/article/31/19/3210/211866)
+
+[bwa](docs/bam_coverage_stats.md) - [BWA比对](https://academic.oup.com/bioinformatics/article/25/14/1754/225615)
+
+[deepbsa](./docs/deepbsa.md) - [DeepBSA](https://github.com/lizhao007/DeepBSA/)
+
+[dual-rnaseq](docs/dual_rnaseq.md) - 双向RNA-Seq
+
+[fastp](./docs/fastp.md) - [fastq文件质控](https://onlinelibrary.wiley.com/doi/10.1002/imt2.70078)
+
+[genomescope](docs/genomescope.md) - 使用[GenomeScope 2.0](https://github.com/tbenavi1/genomescope2.0)和[Smudgeplot](https://github.com/KamilSJaron/smudgeplot)评估基因组大小和倍性
+
+[iseq](./docs/iseq.md) - [iSeq下载测序数据](https://github.com/BioOmics/iSeq)
+
+[rna-seq](./docs/rnaseq.md) - [Hisat2](https://www.nature.com/articles/s41587-019-0201-4) + [StringTie2](https://link.springer.com/article/10.1186/s13059-019-1910-1)转录组流程
+
+[sra2fastq](docs/sra2fastq.md) - SRA文件转FSATQ文件-基于[parallel-fastq-dump](https://github.com/rvalieris/parallel-fastq-dump)
+
+[vcf2pca](docs/vcf2pca.md) - 基于[VCF2PCACluster](https://github.com/hewm2008/VCF2PCACluster)和PLINK的VCF文件做PCA
+
+[vcf2phylip](docs/vcf2phylip.md) - VCF文件转PHYLIP格式文件
+
+
+## 许可证 | License
+
+本项目采用 MIT 许可证 - 详见 [LICENSE](LICENSE) 文件
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 作者信息 | Author
+
+**李详 (Xiang Li)**
+- Email: lixiang117423@gmail.com
+- GitHub: [@lixiang117423](https://github.com/lixiang117423)
+
+## 致谢 | Acknowledgments
+
+- 感谢所有为本项目做出贡献的开发者 | Thanks to all developers who contributed to this project
+- 感谢开源社区的支持 | Thanks to the open source community for support
+
+## 问题反馈 | Issues
+
+如果遇到问题或有建议，请在GitHub上提交issue：
+
+If you encounter problems or have suggestions, please submit an issue on GitHub:
+
+https://github.com/lixiang117423/biopytools/issues

biopytools-0.21.1/biopytools/admixture/__init__.py

@@ -0,0 +1,35 @@
+"""
+ADMIXTURE群体结构分析工具包|ADMIXTURE Population Structure Analysis Toolkit
+
+功能: VCF到ADMIXTURE分析的完整流程，支持群体结构分析和协变量生成
+Features: Complete pipeline from VCF to ADMIXTURE analysis, supporting population structure analysis and covariate generation
+
+作者|Author: Xiang LI
+版本|Version: 1.0.0
+日期|Date: 2024-12-30
+
+使用示例|Usage Examples:
+    from biopytools.admixture import AdmixtureAnalyzer, AdmixtureConfig
+
+    # 创建分析器|Create analyzer
+    analyzer = AdmixtureAnalyzer(
+        vcf_file="data.vcf.gz",
+        output_dir="admixture_results",
+        min_k=2,
+        max_k=10
+    )
+
+    # 运行分析|Run analysis
+    analyzer.run_analysis()
+
+命令行使用|Command Line Usage:
+    python -m admixture.main -v input.vcf -o results -k 2 -K 10
+"""
+
+__version__ = "1.0.0"
+__author__ = "Xiang LI"
+
+from .main import AdmixtureAnalyzer
+from .config import AdmixtureConfig
+
+__all__ = ['AdmixtureAnalyzer', 'AdmixtureConfig']

biopytools-0.21.1/biopytools/admixture/analysis.py

@@ -0,0 +1,200 @@
+"""
+ ADMIXTURE分析核心模块|ADMIXTURE Analysis Core Module 
+"""
+
+import os
+import glob
+import pandas as pd
+from pathlib import Path
+from .utils import CommandRunner
+
+class AdmixtureAnalyzer:
+    """ADMIXTURE分析器|ADMIXTURE Analyzer"""
+    
+    def __init__(self, config, logger, cmd_runner: CommandRunner):
+        self.config = config
+        self.logger = logger
+        self.cmd_runner = cmd_runner
+    
+    def run_admixture_analysis(self, plink_prefix: str):
+        """ 运行ADMIXTURE分析|Run ADMIXTURE analysis"""
+        self.logger.info("  开始ADMIXTURE分析|Starting ADMIXTURE analysis")
+        
+        bed_file = f"{plink_prefix}.bed"
+        if not os.path.exists(bed_file):
+            raise FileNotFoundError(f" PLINK文件不存在|PLINK file not found: {bed_file}")
+        
+        #  为每个K值运行ADMIXTURE|Run ADMIXTURE for each K value
+        for k in range(self.config.min_k, self.config.max_k + 1):
+            self._run_single_k(bed_file, k)
+        
+        #  计算交叉验证误差|Calculate cross-validation error
+        best_k = self._find_best_k()
+        
+        self.logger.info(f" ADMIXTURE分析完成|ADMIXTURE analysis completed")
+        self.logger.info(f" 最优K值|Best K value: {best_k}")
+        
+        return best_k
+    
+    def _run_single_k(self, bed_file: str, k: int):
+        """ 为单个K值运行ADMIXTURE|Run ADMIXTURE for single K value"""
+        self.logger.info(f" 运行ADMIXTURE K={k}|Running ADMIXTURE K={k}")
+        
+        #  设置输出文件名|Set output file names
+        log_file = os.path.join(self.config.output_dir, f"log_{k}.out")
+        
+        #  构建命令，使用相对路径避免长路径问题
+        bed_basename = os.path.basename(bed_file)
+        cmd = (
+            f"cd {self.config.output_dir} && "
+            f"admixture --cv={self.config.cv_folds} -j{self.config.threads} "
+            f"{bed_basename} {k} > log_{k}.out 2>&1"
+        )
+        
+        #  运行命令|Run command
+        try:
+            result = self.cmd_runner.run(cmd, f"ADMIXTURE分析 K={k}|ADMIXTURE analysis K={k}")
+            
+            #  检查输出文件是否生成成功
+            expected_q_file = os.path.join(self.config.output_dir, f"{os.path.splitext(bed_basename)[0]}.{k}.Q")
+            if os.path.exists(expected_q_file):
+                self.logger.info(f"  K={k} 分析成功完成|K={k} analysis completed successfully")
+            else:
+                self.logger.warning(f" K={k} 分析可能未完全成功，Q文件不存在|K={k} analysis may not be fully successful, Q file missing")
+                
+        except Exception as e:
+            self.logger.error(f" K={k}分析失败|K={k} analysis failed: {e}")
+            # 尝试读取日志文件了解错误原因
+            if os.path.exists(log_file):
+                with open(log_file, 'r') as f:
+                    log_content = f.read()
+                    self.logger.error(f" ADMIXTURE日志|ADMIXTURE log:\n{log_content}")
+            raise
+    
+    def _find_best_k(self):
+        """ 找到最优K值|Find best K value"""
+        cv_results = []
+        
+        # 从log文件中提取CV误差|Extract CV error from log files
+        for k in range(self.config.min_k, self.config.max_k + 1):
+            log_file = os.path.join(self.config.output_dir, f"log_{k}.out")
+            
+            if os.path.exists(log_file):
+                with open(log_file, 'r') as f:
+                    for line in f:
+                        if 'CV error' in line:
+                            cv_error = float(line.strip().split()[-1])
+                            cv_results.append({'K': k, 'CV_error': cv_error})
+                            break
+        
+        if not cv_results:
+            raise ValueError(" 未找到CV误差信息|No CV error information found")
+        
+        # 找到最小CV误差对应的K值|Find K with minimum CV error
+        cv_df = pd.DataFrame(cv_results)
+        best_k = cv_df.loc[cv_df['CV_error'].idxmin(), 'K']
+        
+        #  保存CV结果|Save CV results
+        cv_file = os.path.join(self.config.output_dir, "cv_results.csv")
+        cv_df.to_csv(cv_file, index=False)
+        
+        return int(best_k)
+
+class ResultsProcessor:
+    """ 结果处理器|Results Processor"""
+    
+    def __init__(self, config, logger):
+        self.config = config
+        self.logger = logger
+    
+    def process_results(self, best_k: int):
+        """ 处理分析结果|Process analysis results"""
+        self.logger.info(" 处理分析结果|Processing analysis results")
+        
+        #  读取Q文件（个体祖先成分）| Read Q file (individual ancestry proportions)
+        q_file = os.path.join(self.config.output_dir, f"{self.config.base_name}.{best_k}.Q")
+        q_data = self._read_q_file(q_file, best_k)
+        
+        #  读取P文件（等位基因频率）| Read P file (allele frequencies)
+        p_file = os.path.join(self.config.output_dir, f"{self.config.base_name}.{best_k}.P")
+        p_data = self._read_p_file(p_file, best_k)
+        
+        #  计算统计信息|Calculate statistics
+        stats = self._calculate_statistics(q_data, best_k)
+        
+        #  保存处理后的结果|Save processed results
+        self._save_processed_results(q_data, p_data, stats, best_k)
+        
+        return q_data, p_data, stats
+    
+    def _read_q_file(self, q_file: str, k: int):
+        """ 读取Q文件|Read Q file"""
+        if not os.path.exists(q_file):
+            raise FileNotFoundError(f" Q文件不存在|Q file not found: {q_file}")
+        
+        q_data = pd.read_csv(q_file, sep=r'\s+', header=None)
+        q_data.columns = [f"Pop{i+1}" for i in range(k)]
+        
+        #  添加个体信息|Add individual information
+        fam_file = os.path.join(self.config.output_dir, f"{self.config.base_name}.fam")
+        if os.path.exists(fam_file):
+            fam_data = pd.read_csv(fam_file, sep=r'\s+', header=None)
+            q_data['FID'] = fam_data.iloc[:, 0]
+            q_data['IID'] = fam_data.iloc[:, 1]
+        
+        return q_data
+    
+    def _read_p_file(self, p_file: str, k: int):
+        """ 读取P文件|Read P file"""
+        if not os.path.exists(p_file):
+            self.logger.warning(f" P文件不存在|P file not found: {p_file}")
+            return None
+        
+        p_data = pd.read_csv(p_file, sep=r'\s+', header=None)
+        p_data.columns = [f"Pop{i+1}" for i in range(k)]
+        
+        return p_data
+    
+    def _calculate_statistics(self, q_data: pd.DataFrame, k: int):
+        """ 计算统计信息|Calculate statistics"""
+        pop_cols = [f"Pop{i+1}" for i in range(k)]
+        
+        #  计算每个个体的最大祖先成分|Calculate max ancestry for each individual
+        max_ancestry = q_data[pop_cols].max(axis=1)
+        
+        # 计算混合程度|Calculate admixture level
+        admixture_level = 1 - max_ancestry
+        
+        #  统计信息|Statistics
+        stats = {
+            'total_individuals': len(q_data),
+            'highly_admixed': sum(admixture_level > 0.3),
+            'pure_individuals': sum(max_ancestry > 0.9),
+            'mean_admixture_level': admixture_level.mean(),
+            'mean_max_ancestry': max_ancestry.mean()
+        }
+        
+        return stats
+    
+    def _save_processed_results(self, q_data: pd.DataFrame, p_data: pd.DataFrame, 
+                               stats: dict, best_k: int):
+        """ 保存处理后的结果|Save processed results"""
+        # 保存详细Q矩阵|Save detailed Q matrix
+        q_output = os.path.join(self.config.output_dir, "admixture_proportions.csv")
+        q_data.to_csv(q_output, index=False)
+        
+        #  保存统计信息|Save statistics
+        stats_output = os.path.join(self.config.output_dir, "admixture_statistics.txt")
+        with open(stats_output, 'w') as f:
+            f.write(" ADMIXTURE分析统计信息|ADMIXTURE Analysis Statistics\n")
+            f.write("=" * 80 + "\n\n")
+            f.write(f" 最优K值|Best K value: {best_k}\n")
+            f.write(f"总个体数|Total individuals: {stats['total_individuals']}\n")
+            f.write(f" 高度混合个体数|Highly admixed individuals: {stats['highly_admixed']}\n")
+            f.write(f"纯合个体数|Pure individuals: {stats['pure_individuals']}\n")
+            f.write(f"平均混合程度|Mean admixture level: {stats['mean_admixture_level']:.3f}\n")
+            f.write(f" 平均最大祖先成分|Mean max ancestry: {stats['mean_max_ancestry']:.3f}\n")
+        
+        self.logger.info(f" 结果已保存|Results saved:")
+        self.logger.info(f"  -  个体祖先成分|Individual ancestry proportions: {q_output}")
+        self.logger.info(f"  -  统计信息|Statistics: {stats_output}")

biopytools-0.21.1/biopytools/admixture/config.py

@@ -0,0 +1,82 @@
+"""
+ADMIXTURE分析配置管理模块|ADMIXTURE Analysis Configuration Management Module
+"""
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Optional, List
+
+
+@dataclass
+class AdmixtureConfig:
+    """ADMIXTURE分析配置类|ADMIXTURE Analysis Configuration Class"""
+
+    # 必需文件|Required files
+    vcf_file: str
+    output_dir: str = "admixture_results"
+
+    # 分析参数|Analysis parameters
+    min_k: int = 2
+    max_k: int = 10
+    cv_folds: int = 5
+    threads: int = 64
+
+    # 质控参数|Quality control parameters
+    maf: float = 0.05
+    missing_rate: float = 0.1
+    hwe_pvalue: float = 1e-6
+
+    # 处理选项|Processing options
+    skip_preprocessing: bool = False
+    keep_intermediate: bool = False
+
+    # 日志配置|Logging configuration
+    log_level: str = "INFO"
+    quiet: bool = False
+    verbose: int = 0
+
+    # 执行控制|Execution control
+    force: bool = False
+    dry_run: bool = False
+    force: bool = False
+
+    # 内部属性|Internal attributes
+    base_name: str = "admixture_ready"
+
+    def __post_init__(self):
+        """初始化后处理|Post-initialization processing"""
+        self.output_path = Path(self.output_dir)
+        self.output_path.mkdir(parents=True, exist_ok=True)
+
+        # 标准化路径|Normalize paths
+        self.vcf_file = os.path.normpath(os.path.abspath(self.vcf_file))
+        self.output_dir = os.path.normpath(os.path.abspath(self.output_dir))
+
+    def validate(self):
+        """验证配置参数|Validate configuration parameters"""
+        errors = []
+
+        # 检查输入文件|Check input file
+        if not os.path.exists(self.vcf_file):
+            errors.append(f"VCF文件不存在|VCF file does not exist: {self.vcf_file}")
+
+        # 检查K值范围|Check K range
+        if self.min_k < 1 or self.max_k < self.min_k:
+            errors.append(f"无效的K值范围|Invalid K range: {self.min_k} to {self.max_k}")
+
+        # 检查线程数|Check thread count
+        if self.threads <= 0:
+            errors.append(f"线程数必须为正整数|Thread count must be positive: {self.threads}")
+
+        # 检查质控参数|Check QC parameters
+        if not 0 <= self.maf <= 0.5:
+            errors.append(f"MAF值应在0-0.5之间|MAF should be between 0-0.5: {self.maf}")
+
+        if not 0 <= self.missing_rate <= 1:
+            errors.append(f"缺失率应在0-1之间|Missing rate should be between 0-1: {self.missing_rate}")
+
+        if errors:
+            raise ValueError("配置错误|Configuration error:\n" + "\n".join(errors))
+
+        return True