RefgenDetector 3.0.0__tar.gz

refgendetector-3.0.0/LICENSE

@@ -0,0 +1,3 @@
+RefgenDetector is released under GNU General Public License v3.0.
+
+It was funded by ELIXIR, the research infrastructure for life-science data (ELIXIR Beacon Implementation Studies 2019-2021 and 2022-2023).  

refgendetector-3.0.0/PKG-INFO

@@ -0,0 +1,300 @@
+Metadata-Version: 2.4
+Name: RefgenDetector
+Version: 3.0.0
+Summary: RefgenDetector
+Author: Mireia Marin i Ginestar
+Author-email: <mireia.marin@crg.eu>
+Keywords: python
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: Unix
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: argparse
+Requires-Dist: pysam
+Requires-Dist: psutil
+Requires-Dist: rich
+Requires-Dist: pandas
+Requires-Dist: dnspython
+Requires-Dist: msgpack
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: keywords
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
+
+# EGA - RefgenDetector
+
+RefgenDetector is a bioinformatics tool that **infers the reference genome assembly** used to create aligment files (BAM/CRAM/header) and VCFs. 
+
+## Aligment Files
+
+It identifies major genome releases and derived assemblies across humans and multiple other species by analyzing contig names and lengths **from the header**. Benchmarking against 94 synthetic datasets achieved a 100% accuracy rate, while large-scale testing on 918,404 real-world files demonstrated 97.13% correctness, failing only when files’ headers are incomplete.
+
+### Description
+
+RefgenDetector is able to infer the following reference genomes:
+
+**Primates**
+
+👤 Homo sapiens
+
+- hg16
+- hg17
+- hg18
+- GRCh37
+- GRCh38
+- T2T
+
+🐒 Pan troglodytes
+
+- pantro3_0
+- Pan_troglodytes-2.1
+
+🐵 Macaca mulatta
+
+- Mmul10
+- rheMac8
+- rheMac3
+
+**Rodents**
+
+🐭 Mus musculus
+
+- mm7
+- mm8
+- mm9
+- mm10
+- mm39
+
+🐀 Rattus norvegicus
+
+- mRatBN7_2
+- Rnor_6_0
+
+**Other Mammals**
+
+🐷 Sus scrofa
+
+- Sscrofa10_2
+- Sscrofa11_1
+
+**Vertebrates (Non-Mammalian)**
+
+🐟 Danio Rerio
+
+- danRer10
+- danRer11
+
+**Invertebrates**
+
+🪰 Drosophila Melanogaster
+
+- dm5
+- dm6
+
+🐛 Caenorhabditis elegans
+
+- WBcel215
+- WBcel235
+
+**Microorganisms & Plants**
+
+🧫 Escherichia coli
+
+- ASM886v2
+- ASM584v2
+
+🌱 Arabidopsis thaliana
+
+- TAIR
+
+🍺 Saccharomyces cerevisiae
+
+- R64
+
+## `ref_manager.py` - Customize the assemblies database. 
+
+`ref_manager.py` provides command-line management of reference genomes used by RefgenDetector. It allows users to add custom assemblies from FASTA index (`.fai`) files, list all available references, and remove previously added custom entries without modifying the source code.
+
+### Usage
+
+```bash
+python ref_manager.py <command> [options]
+```
+
+### Commands
+
+#### Add a reference
+
+```bash
+python ref_manager.py add <genome.fai> <reference_name> <species>
+```
+
+Registers a new reference from a valid `.fai` file. If the contig structure matches an existing reference, the entry is not added.
+
+#### List references
+
+```bash
+python ref_manager.py list
+```
+
+Displays all available references, including both built-in and user-defined assemblies.
+
+#### Remove a reference
+
+```bash
+python ref_manager.py remove <reference_name>
+```
+
+Removes a custom reference from the local database. Built-in references cannot be removed.
+
+### Notes
+
+- Custom references are stored separately from the default reference database.
+- Input files must be valid FASTA index files generated with `samtools faidx`.
+- Duplicate assemblies are detected based on exact contig composition.
+
+## Variant Calling Files (VCFs)
+
+From VCF files only 4 human assemblies can be inferred: 
+
+- Hg18
+- GRCh37
+- GRCh38
+- T2T
+
+Two different sources of information are used to infer the reference genome from variant calling files
+
+* **Header**
+  
+In the VCF specification it is recommended, but **not mandatory** that the VCF header includes tags describing the reference and contigs backing the data contained in the file. When present, the tool will analyze this information and output the reference genome version based on the contig lengths, following the same logic of the aligment files inference.
+
+* **Variants**
+
+To infer the reference genome from a VCF the tool will read the VCF file in chunks of 100.000 variants, avoiding to load the complete file in memory. The `POS` and `REF` columns will be extracted and compared to the msgpack files.
+
+The msgpack files were created comparing the nucleotides in each position for hg18, GRCh37, GRCh38 and T2T. Each file contains a list of the positions where each reference had a different nucleotide (distinguishing positions). 
+
+By getting the number of matches between these distinguishing positions and the `REF` present in the VCF we infer the reference genome version used to call the variants. 
+
+## Requirements
+
+- Python 3.10.6
+
+Depending on how you want to install the package:
+
+- pip
+- Docker
+
+Download the `msgpack` files for the inference with VCFs: 
+
+1. [Download the msgpack reference](https://crgcnag-my.sharepoint.com/:u:/g/personal/mimarin_crg_es/IQDa5CICZDAoRZmbfhBG3ZPEAWdVnNqvefFJB_r5Hc8aM70?e=kID7zn)
+
+2. Move the `msgpack` to the correct path:
+
+```
+mv msgpack.zip /refgenDetector/src/refgenDetector/
+unzip /refgenDetector/src/refgenDetector/msgpack.zip
+```
+
+## Installation
+
+### Cloning this repository
+
+1. Clone this repository
+
+2. ``` $ cd PATH_WHERE_YOU_CLONED_THE_REPOSITORY/src/refgenDetector ```
+
+3. ``$ python3 refgenDetector_main.py -h ``
+
+### From pypi
+
+``$ pip install refgenDetector``
+
+### From Docker 
+``
+
+## Usage
+
+You can get the help menu by running:
+
+```
+$ refgenDetector -h
+```
+
+```
+usage: INFERRING THE REFERENCE GENOME USED TO ALIGN BAM OR CRAM FILE [-h] -f FILE -t {BAM/CRAM,Header,VCF,BIM} [--md5] [-a] [-v MAX_N_VAR] [-m MATCHES] [-r]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -f FILE, --file FILE  Input file path
+  -t {BAM/CRAM,Header,VCF,BIM}, --type {BAM/CRAM,Header,VCF,BIM}
+                        Type of files to analyze.
+  --md5                 Print md5 values if present in header.
+  -a, --assembly        Print assembly if present in header.
+  -v MAX_N_VAR, --max_n_var MAX_N_VAR
+                        Maximum number of variants to read before stopping inference. The file is processed in chunks of 100,000 variants, so this value must be a multiple of 100,000 (e.g. 100000,
+                        200000, 300000, ...).
+  -m MATCHES, --matches MATCHES
+                        Number of matches required before stopping. [DEFAULT:5000]
+  -r, --resources       When set, print execution time, CPU, memory, and disk I/O usage
+```
+
+## Test RefgenDetector
+
+In the folder **examples** you can find headers, BAM and CRAMs to test the working of RefgenDetector.
+
+*All this files belong to the [synthetics data cohort](https://ega-archive.org/synthetic-data) from the European
+Genome-Phenome Archive ([EGA](https://ega-archive.org/)).*
+
+### Test with headers in a TXT
+
+In the folder TEST_HEADERS there are four headers obtained from synthetic BAM an CRAMs stored in the EGA. Each one of
+them belongs to a different synthetic study:
+
+- Test Study for EGA using data from 1000 Genomes Project - Phase
+  3 [EGAS00001005042](https://ega-archive.org/studies/EGAS00001005042).
+- Synthetic data - Genome in a Bottle - [EGAS00001005591](https://ega-archive.org/studies/EGAS00001005591).
+- Human genomic and phenotypic synthetic data for the study of rare
+  diseases - [EGAS00001005702](https://ega-archive.org/studies/EGAS00001005702).
+- CINECA synthetic data.Please note: This study contains synthetic data (with cohort “participants” / ”subjects” marked
+  with FAKE) has no identifiable data and cannot be used to make any inference about cohort data or
+  results - [EGAS00001002472](https://ega-archive.org/studies/EGAS00001002472).
+
+Further information about them can be found in the file *where_to_find_this_files.txt*, saved in the same folder.
+
+To run RefgenDetector with the files:
+
+1. Modify the txt *path_to_headers* so the paths match those in your computer.
+2. Run:
+
+   ``` $ refgenDetector -p /PATH_WHERE_YOU_CLONED_THE_REPOSITORY/refgenDetector/examples/path_to_headers -t Headers```
+
+### Test with BAM and CRAMs
+
+In the folder TEST_BAM_CRAM there are a BAM and a CRAM obtained from synthetic BAM an CRAMs stored in the EGA. They
+belong to the synthetic study - Test Study for EGA using data from 1000 Genomes Project - Phase
+3 [EGAS00001005042](https://ega-archive.org/studies/EGAS00001005042).
+
+Further information about them can be found in the file *where_to_find_this_files.txt*, saved in the same folder.
+
+To run RefgenDetector with the files:
+
+1. Modify the txt *path_to_bam_cram* so the paths match those in your computer.
+
+2. Run:
+
+   ``` $ refgenDetector -p /PATH_WHERE_YOU_CLONED_THE_REPOSITORY/refgenDetector_pip-master/examples/path_to_bam_cram -t BAM/CRAM```
+
+
+
+## Licence and funding
+
+RefgenDetector is released under GNU General Public License v3.0.
+
+It was funded by ELIXIR, the research infrastructure for life-science data (ELIXIR Beacon Implementation Studies
+2019-2021 and 2022-2023).

refgendetector-3.0.0/README.md

@@ -0,0 +1,272 @@
+# EGA - RefgenDetector
+
+RefgenDetector is a bioinformatics tool that **infers the reference genome assembly** used to create aligment files (BAM/CRAM/header) and VCFs. 
+
+## Aligment Files
+
+It identifies major genome releases and derived assemblies across humans and multiple other species by analyzing contig names and lengths **from the header**. Benchmarking against 94 synthetic datasets achieved a 100% accuracy rate, while large-scale testing on 918,404 real-world files demonstrated 97.13% correctness, failing only when files’ headers are incomplete.
+
+### Description
+
+RefgenDetector is able to infer the following reference genomes:
+
+**Primates**
+
+👤 Homo sapiens
+
+- hg16
+- hg17
+- hg18
+- GRCh37
+- GRCh38
+- T2T
+
+🐒 Pan troglodytes
+
+- pantro3_0
+- Pan_troglodytes-2.1
+
+🐵 Macaca mulatta
+
+- Mmul10
+- rheMac8
+- rheMac3
+
+**Rodents**
+
+🐭 Mus musculus
+
+- mm7
+- mm8
+- mm9
+- mm10
+- mm39
+
+🐀 Rattus norvegicus
+
+- mRatBN7_2
+- Rnor_6_0
+
+**Other Mammals**
+
+🐷 Sus scrofa
+
+- Sscrofa10_2
+- Sscrofa11_1
+
+**Vertebrates (Non-Mammalian)**
+
+🐟 Danio Rerio
+
+- danRer10
+- danRer11
+
+**Invertebrates**
+
+🪰 Drosophila Melanogaster
+
+- dm5
+- dm6
+
+🐛 Caenorhabditis elegans
+
+- WBcel215
+- WBcel235
+
+**Microorganisms & Plants**
+
+🧫 Escherichia coli
+
+- ASM886v2
+- ASM584v2
+
+🌱 Arabidopsis thaliana
+
+- TAIR
+
+🍺 Saccharomyces cerevisiae
+
+- R64
+
+## `ref_manager.py` - Customize the assemblies database. 
+
+`ref_manager.py` provides command-line management of reference genomes used by RefgenDetector. It allows users to add custom assemblies from FASTA index (`.fai`) files, list all available references, and remove previously added custom entries without modifying the source code.
+
+### Usage
+
+```bash
+python ref_manager.py <command> [options]
+```
+
+### Commands
+
+#### Add a reference
+
+```bash
+python ref_manager.py add <genome.fai> <reference_name> <species>
+```
+
+Registers a new reference from a valid `.fai` file. If the contig structure matches an existing reference, the entry is not added.
+
+#### List references
+
+```bash
+python ref_manager.py list
+```
+
+Displays all available references, including both built-in and user-defined assemblies.
+
+#### Remove a reference
+
+```bash
+python ref_manager.py remove <reference_name>
+```
+
+Removes a custom reference from the local database. Built-in references cannot be removed.
+
+### Notes
+
+- Custom references are stored separately from the default reference database.
+- Input files must be valid FASTA index files generated with `samtools faidx`.
+- Duplicate assemblies are detected based on exact contig composition.
+
+## Variant Calling Files (VCFs)
+
+From VCF files only 4 human assemblies can be inferred: 
+
+- Hg18
+- GRCh37
+- GRCh38
+- T2T
+
+Two different sources of information are used to infer the reference genome from variant calling files
+
+* **Header**
+  
+In the VCF specification it is recommended, but **not mandatory** that the VCF header includes tags describing the reference and contigs backing the data contained in the file. When present, the tool will analyze this information and output the reference genome version based on the contig lengths, following the same logic of the aligment files inference.
+
+* **Variants**
+
+To infer the reference genome from a VCF the tool will read the VCF file in chunks of 100.000 variants, avoiding to load the complete file in memory. The `POS` and `REF` columns will be extracted and compared to the msgpack files.
+
+The msgpack files were created comparing the nucleotides in each position for hg18, GRCh37, GRCh38 and T2T. Each file contains a list of the positions where each reference had a different nucleotide (distinguishing positions). 
+
+By getting the number of matches between these distinguishing positions and the `REF` present in the VCF we infer the reference genome version used to call the variants. 
+
+## Requirements
+
+- Python 3.10.6
+
+Depending on how you want to install the package:
+
+- pip
+- Docker
+
+Download the `msgpack` files for the inference with VCFs: 
+
+1. [Download the msgpack reference](https://crgcnag-my.sharepoint.com/:u:/g/personal/mimarin_crg_es/IQDa5CICZDAoRZmbfhBG3ZPEAWdVnNqvefFJB_r5Hc8aM70?e=kID7zn)
+
+2. Move the `msgpack` to the correct path:
+
+```
+mv msgpack.zip /refgenDetector/src/refgenDetector/
+unzip /refgenDetector/src/refgenDetector/msgpack.zip
+```
+
+## Installation
+
+### Cloning this repository
+
+1. Clone this repository
+
+2. ``` $ cd PATH_WHERE_YOU_CLONED_THE_REPOSITORY/src/refgenDetector ```
+
+3. ``$ python3 refgenDetector_main.py -h ``
+
+### From pypi
+
+``$ pip install refgenDetector``
+
+### From Docker 
+``
+
+## Usage
+
+You can get the help menu by running:
+
+```
+$ refgenDetector -h
+```
+
+```
+usage: INFERRING THE REFERENCE GENOME USED TO ALIGN BAM OR CRAM FILE [-h] -f FILE -t {BAM/CRAM,Header,VCF,BIM} [--md5] [-a] [-v MAX_N_VAR] [-m MATCHES] [-r]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -f FILE, --file FILE  Input file path
+  -t {BAM/CRAM,Header,VCF,BIM}, --type {BAM/CRAM,Header,VCF,BIM}
+                        Type of files to analyze.
+  --md5                 Print md5 values if present in header.
+  -a, --assembly        Print assembly if present in header.
+  -v MAX_N_VAR, --max_n_var MAX_N_VAR
+                        Maximum number of variants to read before stopping inference. The file is processed in chunks of 100,000 variants, so this value must be a multiple of 100,000 (e.g. 100000,
+                        200000, 300000, ...).
+  -m MATCHES, --matches MATCHES
+                        Number of matches required before stopping. [DEFAULT:5000]
+  -r, --resources       When set, print execution time, CPU, memory, and disk I/O usage
+```
+
+## Test RefgenDetector
+
+In the folder **examples** you can find headers, BAM and CRAMs to test the working of RefgenDetector.
+
+*All this files belong to the [synthetics data cohort](https://ega-archive.org/synthetic-data) from the European
+Genome-Phenome Archive ([EGA](https://ega-archive.org/)).*
+
+### Test with headers in a TXT
+
+In the folder TEST_HEADERS there are four headers obtained from synthetic BAM an CRAMs stored in the EGA. Each one of
+them belongs to a different synthetic study:
+
+- Test Study for EGA using data from 1000 Genomes Project - Phase
+  3 [EGAS00001005042](https://ega-archive.org/studies/EGAS00001005042).
+- Synthetic data - Genome in a Bottle - [EGAS00001005591](https://ega-archive.org/studies/EGAS00001005591).
+- Human genomic and phenotypic synthetic data for the study of rare
+  diseases - [EGAS00001005702](https://ega-archive.org/studies/EGAS00001005702).
+- CINECA synthetic data.Please note: This study contains synthetic data (with cohort “participants” / ”subjects” marked
+  with FAKE) has no identifiable data and cannot be used to make any inference about cohort data or
+  results - [EGAS00001002472](https://ega-archive.org/studies/EGAS00001002472).
+
+Further information about them can be found in the file *where_to_find_this_files.txt*, saved in the same folder.
+
+To run RefgenDetector with the files:
+
+1. Modify the txt *path_to_headers* so the paths match those in your computer.
+2. Run:
+
+   ``` $ refgenDetector -p /PATH_WHERE_YOU_CLONED_THE_REPOSITORY/refgenDetector/examples/path_to_headers -t Headers```
+
+### Test with BAM and CRAMs
+
+In the folder TEST_BAM_CRAM there are a BAM and a CRAM obtained from synthetic BAM an CRAMs stored in the EGA. They
+belong to the synthetic study - Test Study for EGA using data from 1000 Genomes Project - Phase
+3 [EGAS00001005042](https://ega-archive.org/studies/EGAS00001005042).
+
+Further information about them can be found in the file *where_to_find_this_files.txt*, saved in the same folder.
+
+To run RefgenDetector with the files:
+
+1. Modify the txt *path_to_bam_cram* so the paths match those in your computer.
+
+2. Run:
+
+   ``` $ refgenDetector -p /PATH_WHERE_YOU_CLONED_THE_REPOSITORY/refgenDetector_pip-master/examples/path_to_bam_cram -t BAM/CRAM```
+
+
+
+## Licence and funding
+
+RefgenDetector is released under GNU General Public License v3.0.
+
+It was funded by ELIXIR, the research infrastructure for life-science data (ELIXIR Beacon Implementation Studies
+2019-2021 and 2022-2023).

refgendetector-3.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

refgendetector-3.0.0/setup.py

@@ -0,0 +1,32 @@
+from setuptools import setup, find_packages
+# read the contents of your README file
+from pathlib import Path
+this_directory = Path(__file__).parent
+long_description = (this_directory / "README.md").read_text()
+
+VERSION = '3.0.0'
+DESCRIPTION = 'RefgenDetector'
+
+# Setting up
+setup(
+    name="RefgenDetector",
+    version=VERSION,
+    author="Mireia Marin i Ginestar",
+    author_email="<mireia.marin@crg.eu>",
+    description=DESCRIPTION,
+    long_description=long_description,
+    long_description_content_type='text/markdown',
+    install_requires=['argparse', 'pysam', 'psutil', 'rich', 'pandas', 'dnspython', 'msgpack'],
+    keywords=['python'],
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "Operating System :: Unix"],
+    entry_points={
+        'console_scripts': [
+            'refgenDetector=refgenDetector.refgenDetector_main:main',
+        ],
+    },
+    packages=find_packages(where='src'),
+    package_dir={'': 'src'}
+
+)