yeasti 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
yeasti-0.1.0/LICENSE ADDED
@@ -0,0 +1,7 @@
1
+ Copyright (c) 2026 Anthony Aylward
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
4
+
5
+ The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
6
+
7
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,7 @@
1
+ include src/yeasti/data/ENSEMBL_README
2
+ include src/yeasti/data/GFF_README
3
+ include src/yeasti/data/GTF_README
4
+ include src/yeasti/data/Saccharomyces_cerevisiae.R64-1-1.dna_sm.chromosome.I.fa.gz
5
+ include src/yeasti/data/Saccharomyces_cerevisiae.R64-1-1.63.chromosome.I.gff3.gz
6
+ include src/yeasti/data/Saccharomyces_cerevisiae.R64-1-1.63.chromosome.I.gtf.gz
7
+ include src/yeasti/data/metadata.csv
yeasti-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,169 @@
1
+ Metadata-Version: 2.4
2
+ Name: yeasti
3
+ Version: 0.1.0
4
+ Summary: Small dataset of yeast chromosome I
5
+ Author-email: Anthony Aylward <anthony.aylward@protonmail.com>
6
+ Project-URL: Homepage, https://gitlab.com/aaylward/yeasti
7
+ Project-URL: Documentation, https://aaylward.gitlab.io/yeasti
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Operating System :: OS Independent
10
+ Classifier: License :: OSI Approved :: MIT License
11
+ Requires-Python: >=3.11
12
+ Description-Content-Type: text/markdown
13
+ License-File: LICENSE
14
+ Requires-Dist: biopython
15
+ Requires-Dist: pysradb
16
+ Dynamic: license-file
17
+
18
+ # YeastI
19
+
20
+ ## Introduction
21
+
22
+ This package contains a set of minimal eukaryotic genome and transcriptome datasets from chromosme I of *Saccharomyces cerevisiae*. They may, for example, be useful for rapidly testing a RNA-seq bioinformatics pipeline.
23
+
24
+ ## Installation
25
+
26
+ Ensure your computing environment includes Python 3.11 or later and the following bioinformatics tools:
27
+ - sra-tools
28
+ - pigz
29
+ - fastp
30
+ - kraken2
31
+
32
+ Conda is useful for setting up this environment. Then, YeastI can be installed with Pip:
33
+
34
+ ```sh
35
+ conda env create -c cond-forge -c bioconda -n yeasti \
36
+ biopython pysradb sra-tools pigz fastp kraken2
37
+ conda activate yeasti
38
+ pip install yeasti
39
+ ```
40
+
41
+ ## Usage
42
+
43
+ Run `yeasti` without arguments to list the help text of all subcommands
44
+
45
+ ```sh
46
+ yeasti
47
+ ```
48
+ ```
49
+ usage: yeasti [-h] [--version] {init,path,metadata,download,sheet,trim,classify} ...
50
+
51
+ Small dataset of yeast chromosome 1
52
+
53
+ positional arguments:
54
+ {init,path,metadata,download,sheet,trim,classify}
55
+ init fully initialize the dataset by downloading, trimming, and classifying RNA-seq reads
56
+ path show paths to data directory and files
57
+ metadata fetch sra metadata
58
+ download download RNA-seq reads
59
+ sheet sample sheet for yeasti rna-seq dataset
60
+ trim trim downloaded reads
61
+ classify classify downloaded reads
62
+
63
+ options:
64
+ -h, --help show this help message and exit
65
+ --version show program's version number and exit
66
+ ```
67
+
68
+ To download and downsample all RNA-seq data, run:
69
+
70
+ ```sh
71
+ yeasti init
72
+ ```
73
+
74
+ Use `yeasti path` to get the absolute path for a data file
75
+ ```sh
76
+ yeasti path data-dir
77
+ yeasti path genome
78
+ yeasti path cds
79
+ yeasti path pep
80
+ yeasti path gff
81
+ yeasti path gtf
82
+ yeasti path sheet
83
+ ```
84
+
85
+ To see the sample sheet listing the locations of the RNA-seq reads:
86
+ ```sh
87
+ yeasti sheet
88
+ ```
89
+
90
+ If you need the data at another location, you can symbolically link to the YeastI dataset, or copy the contents to another directory
91
+ ```sh
92
+ ln -s `yeasti path data-dir` test_data/
93
+ cp -r `yeasti path data-dir` test_data/
94
+ ```
95
+
96
+ To see the SRA metadata of the RNA-seq reads:
97
+ ```sh
98
+ yeasti metadata
99
+ ```
100
+
101
+ To execute the initialization steps individually:
102
+ ```sh
103
+ yeasti download
104
+ yeasti trim
105
+ yeasti classify
106
+ ```
107
+
108
+ ## Datasets
109
+ ![yeasti-datasets.png](docs/source/_static/yeasti-datasets.png)
110
+
111
+ There are two groups of datasets:
112
+ 1. The directly included package data of **chromosome I** sequence and annotation from assembly R64-1-1 at [Ensembl](https://mart.ensembl.org/Saccharomyces_cerevisiae/Info/Index), including:
113
+ - Genome sequence (FASTA)
114
+ - Coding sequences (FASTA)
115
+ - Peptide sequences (FASTA)
116
+ - Gene annotation (GFF3, GTF)
117
+ 2. The RNA-seq reads from [Prigent et al. (2024)](https://www.nature.com/articles/s41467-024-51309-6), which are downloaded from [SRA/NCBI BioProject PRJNA842028](https://www.ncbi.nlm.nih.gov/bioproject/PRJNA842028/) and downsampled to include only chromosome I reads by running an included script.
118
+ - The study sequenced mRNA from *S. cerevisiae* under sulfur starvation at four timepoints with three replicates each, but only two of the timepoints are included in YeastI (T0 and T120).
119
+ - Other similar dataset references can also be found at the [Saccharomyces Genome Database](https://www.yeastgenome.org/).
120
+
121
+ ## Downsampling RNA-seq
122
+ ![yeasti-init.png](docs/source/_static/yeasti-init.png)
123
+
124
+ During the initialization process, after the RNA-seq reads are downloaded they are downsampled by:
125
+ 1. Adapter and quality trimming with `fastp`
126
+ 2. Classification with `kraken2` to filter out all reads that do not match chromosome I
127
+
128
+ ## Configuration
129
+ You can configure some YEASTI settings using environment variables or a TOML file.
130
+
131
+ ### Environment variables
132
+ ```sh
133
+ export YEASTI_DOWNLOAD_DIR = /absolute/path/to/directory/
134
+ export YEASTI_TAXID = <integer>
135
+ export YEASTI_SAMPLE_SHEET = /absolute/path/to/sample_sheet.csv
136
+ export YEASTI_KRAKEN2_DB /absolute/path/to/kraken2_db/
137
+ ```
138
+
139
+ ### Config file
140
+ By default, YeastI looks for configuration in TOML format at `$HOME/.yeasti.toml`
141
+
142
+ ```toml
143
+ download_dir = /absolute/path/to/directory/
144
+ taxid = <integer>
145
+ sample_sheet = /absolute/path/to/sample_sheet.csv
146
+ kraken2_db = /absolute/path/to/kraken2_db/
147
+ ```
148
+
149
+ The location of the config file can also be set using an environment variable:
150
+ ```sh
151
+ export YEASTI_CONFIG_FILE=/absolute/path/to/alternate/config.toml
152
+ ```
153
+
154
+ ## Python API
155
+
156
+ YeastI resources can also be accessed from within python:
157
+
158
+ ```python
159
+ import yeasti
160
+ yeasti.init()
161
+ yeasti.path('genome')
162
+ metadata = yeasti.metadata()
163
+ sample_sheet = yeasti.create_sample_sheet(metadata)
164
+ yeasti.write_sample_sheet(sample_sheet, 'sample_sheet.csv')
165
+ yeasti.download()
166
+ yeasti.trim('reads_1.fastq.gz', 'reads_2.fastq.gz')
167
+ yeasti.classify('reads_1.fastq.gz', 'reads_2.fastq.gz')
168
+ ```
169
+
yeasti-0.1.0/README.md ADDED
@@ -0,0 +1,152 @@
1
+ # YeastI
2
+
3
+ ## Introduction
4
+
5
+ This package contains a set of minimal eukaryotic genome and transcriptome datasets from chromosme I of *Saccharomyces cerevisiae*. They may, for example, be useful for rapidly testing a RNA-seq bioinformatics pipeline.
6
+
7
+ ## Installation
8
+
9
+ Ensure your computing environment includes Python 3.11 or later and the following bioinformatics tools:
10
+ - sra-tools
11
+ - pigz
12
+ - fastp
13
+ - kraken2
14
+
15
+ Conda is useful for setting up this environment. Then, YeastI can be installed with Pip:
16
+
17
+ ```sh
18
+ conda env create -c cond-forge -c bioconda -n yeasti \
19
+ biopython pysradb sra-tools pigz fastp kraken2
20
+ conda activate yeasti
21
+ pip install yeasti
22
+ ```
23
+
24
+ ## Usage
25
+
26
+ Run `yeasti` without arguments to list the help text of all subcommands
27
+
28
+ ```sh
29
+ yeasti
30
+ ```
31
+ ```
32
+ usage: yeasti [-h] [--version] {init,path,metadata,download,sheet,trim,classify} ...
33
+
34
+ Small dataset of yeast chromosome 1
35
+
36
+ positional arguments:
37
+ {init,path,metadata,download,sheet,trim,classify}
38
+ init fully initialize the dataset by downloading, trimming, and classifying RNA-seq reads
39
+ path show paths to data directory and files
40
+ metadata fetch sra metadata
41
+ download download RNA-seq reads
42
+ sheet sample sheet for yeasti rna-seq dataset
43
+ trim trim downloaded reads
44
+ classify classify downloaded reads
45
+
46
+ options:
47
+ -h, --help show this help message and exit
48
+ --version show program's version number and exit
49
+ ```
50
+
51
+ To download and downsample all RNA-seq data, run:
52
+
53
+ ```sh
54
+ yeasti init
55
+ ```
56
+
57
+ Use `yeasti path` to get the absolute path for a data file
58
+ ```sh
59
+ yeasti path data-dir
60
+ yeasti path genome
61
+ yeasti path cds
62
+ yeasti path pep
63
+ yeasti path gff
64
+ yeasti path gtf
65
+ yeasti path sheet
66
+ ```
67
+
68
+ To see the sample sheet listing the locations of the RNA-seq reads:
69
+ ```sh
70
+ yeasti sheet
71
+ ```
72
+
73
+ If you need the data at another location, you can symbolically link to the YeastI dataset, or copy the contents to another directory
74
+ ```sh
75
+ ln -s `yeasti path data-dir` test_data/
76
+ cp -r `yeasti path data-dir` test_data/
77
+ ```
78
+
79
+ To see the SRA metadata of the RNA-seq reads:
80
+ ```sh
81
+ yeasti metadata
82
+ ```
83
+
84
+ To execute the initialization steps individually:
85
+ ```sh
86
+ yeasti download
87
+ yeasti trim
88
+ yeasti classify
89
+ ```
90
+
91
+ ## Datasets
92
+ ![yeasti-datasets.png](docs/source/_static/yeasti-datasets.png)
93
+
94
+ There are two groups of datasets:
95
+ 1. The directly included package data of **chromosome I** sequence and annotation from assembly R64-1-1 at [Ensembl](https://mart.ensembl.org/Saccharomyces_cerevisiae/Info/Index), including:
96
+ - Genome sequence (FASTA)
97
+ - Coding sequences (FASTA)
98
+ - Peptide sequences (FASTA)
99
+ - Gene annotation (GFF3, GTF)
100
+ 2. The RNA-seq reads from [Prigent et al. (2024)](https://www.nature.com/articles/s41467-024-51309-6), which are downloaded from [SRA/NCBI BioProject PRJNA842028](https://www.ncbi.nlm.nih.gov/bioproject/PRJNA842028/) and downsampled to include only chromosome I reads by running an included script.
101
+ - The study sequenced mRNA from *S. cerevisiae* under sulfur starvation at four timepoints with three replicates each, but only two of the timepoints are included in YeastI (T0 and T120).
102
+ - Other similar dataset references can also be found at the [Saccharomyces Genome Database](https://www.yeastgenome.org/).
103
+
104
+ ## Downsampling RNA-seq
105
+ ![yeasti-init.png](docs/source/_static/yeasti-init.png)
106
+
107
+ During the initialization process, after the RNA-seq reads are downloaded they are downsampled by:
108
+ 1. Adapter and quality trimming with `fastp`
109
+ 2. Classification with `kraken2` to filter out all reads that do not match chromosome I
110
+
111
+ ## Configuration
112
+ You can configure some YEASTI settings using environment variables or a TOML file.
113
+
114
+ ### Environment variables
115
+ ```sh
116
+ export YEASTI_DOWNLOAD_DIR = /absolute/path/to/directory/
117
+ export YEASTI_TAXID = <integer>
118
+ export YEASTI_SAMPLE_SHEET = /absolute/path/to/sample_sheet.csv
119
+ export YEASTI_KRAKEN2_DB /absolute/path/to/kraken2_db/
120
+ ```
121
+
122
+ ### Config file
123
+ By default, YeastI looks for configuration in TOML format at `$HOME/.yeasti.toml`
124
+
125
+ ```toml
126
+ download_dir = /absolute/path/to/directory/
127
+ taxid = <integer>
128
+ sample_sheet = /absolute/path/to/sample_sheet.csv
129
+ kraken2_db = /absolute/path/to/kraken2_db/
130
+ ```
131
+
132
+ The location of the config file can also be set using an environment variable:
133
+ ```sh
134
+ export YEASTI_CONFIG_FILE=/absolute/path/to/alternate/config.toml
135
+ ```
136
+
137
+ ## Python API
138
+
139
+ YeastI resources can also be accessed from within python:
140
+
141
+ ```python
142
+ import yeasti
143
+ yeasti.init()
144
+ yeasti.path('genome')
145
+ metadata = yeasti.metadata()
146
+ sample_sheet = yeasti.create_sample_sheet(metadata)
147
+ yeasti.write_sample_sheet(sample_sheet, 'sample_sheet.csv')
148
+ yeasti.download()
149
+ yeasti.trim('reads_1.fastq.gz', 'reads_2.fastq.gz')
150
+ yeasti.classify('reads_1.fastq.gz', 'reads_2.fastq.gz')
151
+ ```
152
+
@@ -0,0 +1,34 @@
1
+ [build-system]
2
+ requires = ["setuptools"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "yeasti"
7
+ version = "0.1.0"
8
+ authors = [
9
+ { name="Anthony Aylward", email="anthony.aylward@protonmail.com" },
10
+ ]
11
+ description = "Small dataset of yeast chromosome I"
12
+ readme = "README.md"
13
+ requires-python = ">=3.11"
14
+ classifiers = [
15
+ "Programming Language :: Python :: 3",
16
+ "Operating System :: OS Independent",
17
+ "License :: OSI Approved :: MIT License",
18
+ ]
19
+
20
+ dependencies = [
21
+ "biopython",
22
+ "pysradb"
23
+ ]
24
+
25
+ [project.urls]
26
+ "Homepage" = "https://gitlab.com/aaylward/yeasti"
27
+ "Documentation" = "https://aaylward.gitlab.io/yeasti"
28
+
29
+ [project.scripts]
30
+ yeasti = "yeasti.yeasti:main"
31
+
32
+ [tool.setuptools]
33
+ package-dir = {yeasti="src/yeasti"}
34
+ include-package-data = true
yeasti-0.1.0/setup.cfg ADDED
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,13 @@
1
+ """
2
+ yeasti
3
+ """
4
+
5
+ from yeasti.version import __version__
6
+ from yeasti.metadata import fetch_sra_metadata as metadata
7
+ from yeasti.download import download
8
+ from yeasti.sample_sheet import create_sample_sheet, write_sample_sheet
9
+ from yeasti.data_path import data_path as path
10
+ from yeasti.init import init
11
+ from yeasti.env import YEASTI_TAXID as taxid
12
+ from yeasti.trim import trim
13
+ from yeasti.classify import classify
@@ -0,0 +1,102 @@
1
+ #===============================================================================
2
+ # classify.py
3
+ #===============================================================================
4
+
5
+ # Imports ======================================================================
6
+
7
+ import gzip
8
+ import os
9
+ from Bio import SeqIO
10
+ from subprocess import run
11
+ from shutil import which
12
+ from pathlib import Path
13
+ from yeasti.env import CPU_COUNT, YEASTI_KRAKEN2_DB, YEASTI_GENOME, YEASTI_TAXID, YEASTI_DOWNLOAD_DIR
14
+
15
+ # Functions ====================================================================
16
+
17
+ def add_kraken_taxid(seqio_record):
18
+ """Use kraken2 to classify reads
19
+
20
+ Parameters
21
+ ----------
22
+ seqio_record
23
+ Bio.SeqRecord object
24
+
25
+ Returns
26
+ -------
27
+ Bio.SeqRecord
28
+ record with taxid added
29
+ """
30
+
31
+ seqio_record.id += f'|kraken:taxid|{YEASTI_TAXID}'
32
+ return seqio_record
33
+
34
+
35
+ def classify_kraken2(
36
+ fastq_in_1, fastq_in_2,
37
+ fastq_out_format,
38
+ threads: int = CPU_COUNT
39
+ ):
40
+ """Use kraken2 to classify reads
41
+
42
+ Parameters
43
+ ----------
44
+ fastq_in_1, fastq_in_2
45
+ paired-end fastq input files
46
+ fastq_out_format
47
+ format string for fastq output (#.fastq.gz)
48
+ threads : int
49
+ number of threads
50
+ """
51
+
52
+ if not which('kraken2'):
53
+ raise RuntimeError('kraken2 not found')
54
+ if not Path(YEASTI_KRAKEN2_DB).exists():
55
+ os.mkdir(YEASTI_KRAKEN2_DB)
56
+ run(('k2', 'download-taxonomy', '--db', YEASTI_KRAKEN2_DB), check=True)
57
+ run(('k2', 'add-to-library', '--threads', str(threads), '--no-masking', '--db', YEASTI_KRAKEN2_DB, '--file', str(YEASTI_GENOME)[:-3]), check=True)
58
+ run(('k2', 'build', '--threads', str(threads), '--db', YEASTI_KRAKEN2_DB), check=True)
59
+ run(('k2', 'clean', '--db', YEASTI_KRAKEN2_DB), check=True)
60
+ run((
61
+ 'k2', 'classify',
62
+ '--threads', str(threads),
63
+ '--db', YEASTI_KRAKEN2_DB,
64
+ '--quick',
65
+ '--paired',
66
+ '--output', '-',
67
+ '--classified-out', fastq_out_format,
68
+ fastq_in_1,
69
+ fastq_in_2
70
+ ), check=True)
71
+
72
+
73
+ def classify(
74
+ fastq_in_1, fastq_in_2,
75
+ temp_dir=YEASTI_DOWNLOAD_DIR,
76
+ threads: int = CPU_COUNT
77
+ ):
78
+ """Use kraken2 to classify reads
79
+
80
+ Parameters
81
+ ----------
82
+ fastq_1, fastq_2
83
+ paired-end fastq input files
84
+ temp_dir
85
+ directory for temporary files
86
+ threads : int
87
+ number of cores to use for adapter trimming
88
+ """
89
+
90
+ with gzip.open(YEASTI_GENOME, 'rt') as f_in, open(str(YEASTI_GENOME)[:-3], 'wt') as f_out:
91
+ SeqIO.write((add_kraken_taxid(r) for r in SeqIO.parse(f_in, 'fasta')), f_out, 'fasta')
92
+ classify_kraken2(
93
+ fastq_in_1,
94
+ fastq_in_2,
95
+ Path(temp_dir) / f'{Path(fastq_in_1).name[:-11]}#.fastq',
96
+ threads=threads
97
+ )
98
+ for fq in fastq_in_1, fastq_in_2:
99
+ with open(fq, 'wb') as f:
100
+ run(('pigz', '--stdout', str(Path(temp_dir) / Path(fq).name)[:-3]), stdout=f, check=True)
101
+ os.remove(str(Path(temp_dir) / Path(fq).name)[:-3])
102
+ os.remove(str(YEASTI_GENOME)[:-3])
@@ -0,0 +1,35 @@
1
+ #===============================================================================
2
+ # construct_read_processing_func.py
3
+ #===============================================================================
4
+
5
+ # Imports ======================================================================
6
+
7
+ from multiprocessing import Pool
8
+ from functools import partial
9
+ from math import floor
10
+ from tempfile import TemporaryDirectory
11
+ from yeasti.env import CPU_COUNT
12
+
13
+ # Functions ====================================================================
14
+
15
+ def construct_read_processing_func(func, parallel=False):
16
+ if parallel:
17
+ def read_processing_func(sample_sheet, processes: int = CPU_COUNT):
18
+ fastq_files = tuple(zip(sample_sheet['fastq_1'], sample_sheet['fastq_2']))
19
+ if processes > 1:
20
+ with Pool(processes=min(processes, len(fastq_files))) as pool, TemporaryDirectory() as temp_dir_name:
21
+ pool.starmap(
22
+ partial(func, temp_dir=temp_dir_name, threads=max(1, floor(processes / len(fastq_files)))),
23
+ fastq_files
24
+ )
25
+ else:
26
+ with TemporaryDirectory() as temp_dir_name:
27
+ for fq1, fq2 in fastq_files:
28
+ func(fq1, fq2, temp_dir=temp_dir_name, threads=processes)
29
+ else:
30
+ def read_processing_func(sample_sheet, processes: int = CPU_COUNT):
31
+ fastq_files = tuple(zip(sample_sheet['fastq_1'], sample_sheet['fastq_2']))
32
+ with TemporaryDirectory() as temp_dir_name:
33
+ for fq1, fq2 in fastq_files:
34
+ func(fq1, fq2, temp_dir=temp_dir_name, threads=processes)
35
+ return read_processing_func
@@ -0,0 +1,120 @@
1
+ #### README ####
2
+
3
+ IMPORTANT: Please note you can download subsets of data via the
4
+ BioMart data mining tool.
5
+ See https://www.ensembl.org/info/data/biomart/ for more information.
6
+
7
+ The genome assembly represented here corresponds to
8
+ GCA_000146045.2
9
+
10
+ #######################
11
+ Fasta DNA dumps
12
+ #######################
13
+
14
+ -----------
15
+ FILE NAMES
16
+ ------------
17
+ The files are consistently named following this pattern:
18
+ <species>.<assembly>.<sequence type>.<id type>.<id>.fa.gz
19
+
20
+ <species>: The systematic name of the species.
21
+ <assembly>: The assembly build name.
22
+ <sequence type>:
23
+ * 'dna' - unmasked genomic DNA sequences.
24
+ * 'dna_rm' - masked genomic DNA. Interspersed repeats and low
25
+ complexity regions are detected with the RepeatMasker tool and masked
26
+ by replacing repeats with 'N's.
27
+ * 'dna_sm' - soft-masked genomic DNA. All repeats and low complexity regions
28
+ have been replaced with lowercased versions of their nucleic base
29
+ <id type> One of the following:
30
+ * 'chromosome' - The top-level coordinate system in most species in Ensembl
31
+ * 'nonchromosomal' - Contains DNA that has not been assigned a chromosome
32
+ * 'seqlevel' - This is usually sequence scaffolds, chunks or clones.
33
+ -- 'scaffold' - Larger sequence contigs from the assembly of shorter
34
+ sequencing reads (often from whole genome shotgun, WGS) which could
35
+ not yet be assembled into chromosomes. Often more genome sequencing
36
+ is needed to narrow gaps and establish a tiling path.
37
+ -- 'chunk' - While contig sequences can be assembled into large entities,
38
+ they sometimes have to be artificially broken down into smaller entities
39
+ called 'chunks'. This is due to limitations in the annotation
40
+ pipeline and the finite record size imposed by MySQL which stores the
41
+ sequence and annotation information.
42
+ -- 'clone' - In general this is the smallest sequence entity. It is often
43
+ identical to the sequence of one BAC clone, or sequence region
44
+ of one BAC clone which forms the tiling path.
45
+ <id>: The actual sequence identifier. Depending on the <id type> the <id>
46
+ could represent the name of a chromosome, a scaffold, a contig, a clone ..
47
+ Field is empty for seqlevel files
48
+ fa: All files in these directories represent FASTA database files
49
+ gz: All files are compacted with GNU Zip for storage efficiency.
50
+
51
+
52
+ EXAMPLES
53
+ The genomic sequence of human chromosome 1:
54
+ Homo_sapiens.GRCh37.dna.chromosome.1.fa.gz
55
+
56
+ The masked version of the genome sequence on human chromosome 1
57
+ (contains '_rm' or '_sm' in the name):
58
+ Homo_sapiens.GRCh37.dna_rm.chromosome.1.fa.gz
59
+ Homo_sapiens.GRCh37.dna_sm.chromosome.1.fa.gz
60
+
61
+ Non-chromosomal assembly sequences:
62
+ e.g. mitochondrial genome, sequence contigs not yet mapped on chromosomes
63
+ Homo_sapiens.GRCh37.dna.nonchromosomal.fa.gz
64
+ Homo_sapiens.GRCh37.dna_rm.nonchromosomal.fa.gz
65
+ Homo_sapiens.GRCh37.dna_sm.nonchromosomal.fa.gz
66
+
67
+ ---------
68
+ TOPLEVEL
69
+ ---------
70
+ These files contains all sequence regions flagged as toplevel in an Ensembl
71
+ schema. This includes chromsomes, regions not assembled into chromosomes and
72
+ N padded haplotype/patch regions.
73
+
74
+ EXAMPLES
75
+
76
+ Toplevel sequences unmasked:
77
+ Homo_sapiens.GRCh37.dna.toplevel.fa.gz
78
+
79
+ Toplevel soft/hard masked sequences:
80
+ Homo_sapiens.GRCh37.dna_sm.toplevel.fa.gz
81
+ Homo_sapiens.GRCh37.dna_rm.toplevel.fa.gz
82
+
83
+ -----------------
84
+ PRIMARY ASSEMBLY
85
+ -----------------
86
+ Primary assembly contains all toplevel sequence regions excluding haplotypes
87
+ and patches. This file is best used for performing sequence similarity searches
88
+ where patch and haplotype sequences would confuse analysis. If the primary
89
+ assembly file is not present, that indicates that there are no haplotype/patch
90
+ regions, and the 'toplevel' file is equivalent.
91
+
92
+ EXAMPLES
93
+
94
+ Primary assembly sequences unmasked:
95
+ Homo_sapiens.GRCh37.dna.primary_assembly.fa.gz
96
+
97
+ Primary assembly soft/hard masked sequences:
98
+ Homo_sapiens.GRCh37.dna_sm.primary_assembly.fa.gz
99
+ Homo_sapiens.GRCh37.dna_rm.primary_assembly.fa.gz
100
+
101
+ --------------
102
+ SPECIAL CASES
103
+ --------------
104
+ Some chromosomes have alternate haplotypes which are presented in files with
105
+ the haplotype sequence only:
106
+ Homo_sapiens.GRCh37.dna_rm.chromosome.HSCHR6_MHC_QBL.fa.gz
107
+ Homo_sapiens.GRCh37.dna_rm.chromosome.HSCHR17_1.fa.gz
108
+
109
+ All alternative assembly and patch regions have their sequence padded
110
+ with N's to ensure alignment programs can report the correct index
111
+ regions
112
+
113
+ e.g. A patch region with a start position of 1,000,001 will have 1e6 N's added
114
+ its start so an alignment program will report coordinates with respect to the
115
+ whole chromosome.
116
+
117
+ Human has sequenced Y chromosomes and the pseudoautosomal region (PAR)
118
+ on the Y is annotated. By definition the PAR region is identical on the
119
+ X and Y chromosome. The Y chromosome file contains the Y chromsome
120
+ minus these repeated PAR regions i.e. the unique portion of Y.