xengsort 2.2.0__tar.gz → 2.2.2__tar.gz

{xengsort-2.2.0/xengsort.egg-info → xengsort-2.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: xengsort
-Version: 2.2.0
+Version: 2.2.2
 Summary: xenograft sorting of FASTQ reads into five categories: host, graft, both, neither, ambiguous.
 Author-email: Jens Zentgraf <zentgraf@cs.uni-saarland.de>, Sven Rahmann <rahmann@cs.uni-saarland.de>
 License: MIT License
@@ -52,6 +52,7 @@ This tool, xengsort, uses 3-way bucketed Cuckoo hashing to efficiently solve the
 For a description of the method an evaluation on several datasets, please see our
 [article in "Algorithms for Molecular Biology"](https://almob.biomedcentral.com/articles/10.1186/s13015-021-00181-w), or the initial [WABI 2020 publication](https://drops.dagstuhl.de/opus/volltexte/2020/12793/).
 (There was also a [preprint on bioRxiv](https://www.biorxiv.org/content/10.1101/2020.05.14.095604v2).)
+If you use xengsort to sort converted reads (RRBS or WGBS) there is a [bioRxiv preprint](https://www.biorxiv.org/content/10.64898/2026.10.06.756952v1.abstract) available.
 BibTeX entrys for citation can be found at the end of this README file.
 
 In case of problems,
@@ -71,24 +72,23 @@ Thank you!
 
 We here provide an example showing how to run xengsort on a human-captured mouse exome (part of one of the datasets described in the paper).
 This tutorial uses the workflow management system `Snakemake` to run the example.
-This is, however, not necessary to use `xengsort`. 
+This is, however, not necessary to use `xengsort`.
 Please refer to the Usage Guide below to learn how to run `xengsort` on your own data manually.
 
 
 Our software `xengsort` is provided as a Python package.
-For efficiency, it uses just-in-time compilation provided by the [numba](http://numba.pydata.org/) package. 
+For efficiency, it uses just-in-time compilation provided by the [numba](http://numba.pydata.org/) package.
 We here use the package manager [conda](https://docs.conda.io/en/latest/) to manage Python packages and to use a separate environment for `xengsort`.
 
 
-### Install the conda package manager (miniconda)
-Go to https://docs.conda.io/en/latest/miniconda.html and download the Miniconda installer:
-Choose Python 3.10 (or higher), your operating system, and preferably the 64 bit version.
-Follow the instructions of the installer and append the conda executable to your PATH (even if the installer does not recommend it). 
+### Install the conda package manager (conda-forge)
+Go to https://conda-forge.org/download/ and download the conda-forge installer:
+Choose your operating system, follow the instructions of the installer and append the conda executable to your PATH (even if the installer does not recommend it).
 You can let the installer do it, or do it manually by editing your ``.bashrc`` or similar file under Linux or MacOS, or by editing the environment variables under Windows.
 To verify that the installation works, open a new terminal and execute
 ```
-conda --version # ideally 22.9.xx or higher
-python --version # ideally 3.10.xx or higher
+conda --version # ideally 26.9.xx or higher
+python --version # ideally 3.12.xx or higher
 ```
 
 
@@ -122,7 +122,7 @@ within the cloned directory tree.
 
 
 ### Create and activate a conda environment
-To run our software, a [conda](https://docs.conda.io/en/latest/) environment with the required libraries needs to be created.
+To run our software, a [conda](https://conda-forge.org/docs/) environment with the required libraries needs to be created.
 A list of needed libraries is provided in the ``environment.yml`` file in the cloned repository;
 it can be used to create a new environment:
 
@@ -133,7 +133,7 @@ conda env create
 which will create an environment named ``xengsort`` with the required dependencies,
 using the provided ``environment.yml`` file in the same directory.
 
-A more explicit alternative is to run 
+A more explicit alternative is to run
 ```
 conda create --name xengsort -c conda-forge -c bioconda --file requirements.txt
 ```
@@ -217,11 +217,11 @@ To build an index for xengsort, several parameters must be provided, which are d
 First, a file name and a path for the index must be chosen.
 The index is stored in two files. We will use `myindex` to store the index in the current folder.
 
-Second, two reference genomes (host and graft) must be provided (in FASTA files). 
-We assume that they are given as `host.fa.gz` and `graft.fa.gz`. 
+Second, two reference genomes (host and graft) must be provided (in FASTA files).
+We assume that they are given as `host.fa.gz` and `graft.fa.gz`.
 The files can be provided as an uncompressed file or compressed using `gzip`, `bzip2` or `xz`.
-The corresponding options are `-H` or `--host` and `-G` or `--graft`. 
-Each option can take several arguments as files. 
+The corresponding options are `-H` or `--host` and `-G` or `--graft`.
+Each option can take several arguments as files.
 ```
 xengsort index --index myindex -H host.fa.gz -G graft.fa.gz -n 4_500_000_000  [OPTIONS]
 ```
@@ -249,7 +249,7 @@ The final important parameter is about parallelization:
 
 Most of the parameters can also be provided in a config file (`.yaml`):
 - `--cfg` or `--config` defines the path the the config file.
- 
+
 ### How to classify
 
 To classify a FASTQ sample (one or several single-end or paired-end files), make sure you are in an environment where xengsort and its dependencies are installed.
@@ -257,7 +257,7 @@ Then run the `xengsort classify` command with a previously built index, such as
 ```
 xengsort classify --index myindex --fastq single.fq.gz --prefix myresults --mode count
 ```
-for single-end reads, or 
+for single-end reads, or
 ```
 xengsort classify --index myindex --fastq paired.1.fq.gz --pairs paired.2.fq.gz --prefix myresults --mode count
 ```
@@ -275,7 +275,7 @@ The parameter `--prefix` or equivalently `--out` is required and defines the pre
   - `{prefix}.both.1.fq.gz`: reads that could originate from both
   - `{prefix}.neither.1.fq.gz`: reads that originate from neither host nor graft
   - `{prefix}.ambiguous.1.fq.gz`: (few) ambiguous reads that cannot be classified,
-  
+
   and similarly with `.2.fq.gz`. For single-end reads, there is only `.fq.gz` (no numbers).
 
 The compression type can be specified using the `--compression` parameter.
@@ -301,7 +301,7 @@ xengsort index --index myindex -H host.fa.gz -G graft.fa.gz -n 9_000_000_000 --b
 If an index is build using the bisulfite parameter, also the read is converted in the classification step.
 It is not necessary to change the parameters of the classification step.
 Currently, it only supports the count-based classification.
-
+A detailed description on how the benchmarks on bisulfite sequencing data was performed can be found [here](https://github.com/S-Altend/MethSortBench/tree/main/03_read_sorting).
 
 Please report a [GitLab issue](https://gitlab.com/genomeinformatics/xengsort/-/issues) in case of problems.
 
@@ -315,13 +315,13 @@ Happy sorting!
 *  **Snakemake throws errors when attempting to download the example dataset from SRA**.
 
 This may happen for several reasons:
-  1. The dataset is quite large, over 20 GB. Make sure you have sufficient space (also temporary space) and sufficient disk quota (on a shared system) to store all of the files. 
+  1. The dataset is quite large, over 20 GB. Make sure you have sufficient space (also temporary space) and sufficient disk quota (on a shared system) to store all of the files.
   1. Another source of trouble may be that for some reason and older version of `sra-tools` gets called. We use the `fasterq-dump` tool to download the data, which did not exist in older verisons. To use a clean environment with a recent version of `sra-tools`, you can run snakemake as `snakemake -j 16 --use-conda`, which will generate a separate environment only for the download (this takes additional time, but may resolve the problem).
   1. In earlier versions, the dataset was downloaded from a different location which is no longer available. Please check that you are using at least v1.1.0 (`xengsort --version`). If not, please do a fresh clone and follow the tutorial to set up a new environment (delete the old one first) and run `pip install -e .`, as explained above in the tutorial.
 
 *  **Indexing stops after a few seconds with a failure.**
 
-The most likely cause of this is that you have not specified the size of the hash table (`-n` parameter). 
+The most likely cause of this is that you have not specified the size of the hash table (`-n` parameter).
 (From version 1.0.1, the `-n` parameter is required, and the error cannot occur anymore. Please update to the latest version!)
 For the human and mouse genomes this is approximately `-n 4_500_000_000` (4.5 billion).
 It is unfortunately a limitation of the implementation that the hash table size has to be specified in advance.
@@ -377,3 +377,14 @@ In addition, you may also cite the WABI 2020 proceedigs paper:
   annote =	{Keywords: xenograft sorting, alignment-free method, Cuckoo hashing, k-mer}
 }
 ```
+
+If you use xengsort to sort converted reads, please also cite our corresponding bioRxiv preprint:
+```
+@article {Altendorfer2026.10.06.756952,
+  author = {Altendorfer, Stefan and Zentgraf, Jens and Schmitz, Johanna Elena and Plass, Christoph and Gerh{\"a}user, Clarissa and Rahmann, Sven},
+  title = {High-performance xenograft read-sorting for DNA methylation analysis},
+  year = {2026},
+  doi = {10.64898/2026.10.06.756952},
+  journal = {bioRxiv}
+}
+```

{xengsort-2.2.0 → xengsort-2.2.2}/README.md

@@ -5,6 +5,7 @@ This tool, xengsort, uses 3-way bucketed Cuckoo hashing to efficiently solve the
 For a description of the method an evaluation on several datasets, please see our
 [article in "Algorithms for Molecular Biology"](https://almob.biomedcentral.com/articles/10.1186/s13015-021-00181-w), or the initial [WABI 2020 publication](https://drops.dagstuhl.de/opus/volltexte/2020/12793/).
 (There was also a [preprint on bioRxiv](https://www.biorxiv.org/content/10.1101/2020.05.14.095604v2).)
+If you use xengsort to sort converted reads (RRBS or WGBS) there is a [bioRxiv preprint](https://www.biorxiv.org/content/10.64898/2026.10.06.756952v1.abstract) available.
 BibTeX entrys for citation can be found at the end of this README file.
 
 In case of problems,
@@ -24,24 +25,23 @@ Thank you!
 
 We here provide an example showing how to run xengsort on a human-captured mouse exome (part of one of the datasets described in the paper).
 This tutorial uses the workflow management system `Snakemake` to run the example.
-This is, however, not necessary to use `xengsort`. 
+This is, however, not necessary to use `xengsort`.
 Please refer to the Usage Guide below to learn how to run `xengsort` on your own data manually.
 
 
 Our software `xengsort` is provided as a Python package.
-For efficiency, it uses just-in-time compilation provided by the [numba](http://numba.pydata.org/) package. 
+For efficiency, it uses just-in-time compilation provided by the [numba](http://numba.pydata.org/) package.
 We here use the package manager [conda](https://docs.conda.io/en/latest/) to manage Python packages and to use a separate environment for `xengsort`.
 
 
-### Install the conda package manager (miniconda)
-Go to https://docs.conda.io/en/latest/miniconda.html and download the Miniconda installer:
-Choose Python 3.10 (or higher), your operating system, and preferably the 64 bit version.
-Follow the instructions of the installer and append the conda executable to your PATH (even if the installer does not recommend it). 
+### Install the conda package manager (conda-forge)
+Go to https://conda-forge.org/download/ and download the conda-forge installer:
+Choose your operating system, follow the instructions of the installer and append the conda executable to your PATH (even if the installer does not recommend it).
 You can let the installer do it, or do it manually by editing your ``.bashrc`` or similar file under Linux or MacOS, or by editing the environment variables under Windows.
 To verify that the installation works, open a new terminal and execute
 ```
-conda --version # ideally 22.9.xx or higher
-python --version # ideally 3.10.xx or higher
+conda --version # ideally 26.9.xx or higher
+python --version # ideally 3.12.xx or higher
 ```
 
 
@@ -75,7 +75,7 @@ within the cloned directory tree.
 
 
 ### Create and activate a conda environment
-To run our software, a [conda](https://docs.conda.io/en/latest/) environment with the required libraries needs to be created.
+To run our software, a [conda](https://conda-forge.org/docs/) environment with the required libraries needs to be created.
 A list of needed libraries is provided in the ``environment.yml`` file in the cloned repository;
 it can be used to create a new environment:
 
@@ -86,7 +86,7 @@ conda env create
 which will create an environment named ``xengsort`` with the required dependencies,
 using the provided ``environment.yml`` file in the same directory.
 
-A more explicit alternative is to run 
+A more explicit alternative is to run
 ```
 conda create --name xengsort -c conda-forge -c bioconda --file requirements.txt
 ```
@@ -170,11 +170,11 @@ To build an index for xengsort, several parameters must be provided, which are d
 First, a file name and a path for the index must be chosen.
 The index is stored in two files. We will use `myindex` to store the index in the current folder.
 
-Second, two reference genomes (host and graft) must be provided (in FASTA files). 
-We assume that they are given as `host.fa.gz` and `graft.fa.gz`. 
+Second, two reference genomes (host and graft) must be provided (in FASTA files).
+We assume that they are given as `host.fa.gz` and `graft.fa.gz`.
 The files can be provided as an uncompressed file or compressed using `gzip`, `bzip2` or `xz`.
-The corresponding options are `-H` or `--host` and `-G` or `--graft`. 
-Each option can take several arguments as files. 
+The corresponding options are `-H` or `--host` and `-G` or `--graft`.
+Each option can take several arguments as files.
 ```
 xengsort index --index myindex -H host.fa.gz -G graft.fa.gz -n 4_500_000_000  [OPTIONS]
 ```
@@ -202,7 +202,7 @@ The final important parameter is about parallelization:
 
 Most of the parameters can also be provided in a config file (`.yaml`):
 - `--cfg` or `--config` defines the path the the config file.
- 
+
 ### How to classify
 
 To classify a FASTQ sample (one or several single-end or paired-end files), make sure you are in an environment where xengsort and its dependencies are installed.
@@ -210,7 +210,7 @@ Then run the `xengsort classify` command with a previously built index, such as
 ```
 xengsort classify --index myindex --fastq single.fq.gz --prefix myresults --mode count
 ```
-for single-end reads, or 
+for single-end reads, or
 ```
 xengsort classify --index myindex --fastq paired.1.fq.gz --pairs paired.2.fq.gz --prefix myresults --mode count
 ```
@@ -228,7 +228,7 @@ The parameter `--prefix` or equivalently `--out` is required and defines the pre
   - `{prefix}.both.1.fq.gz`: reads that could originate from both
   - `{prefix}.neither.1.fq.gz`: reads that originate from neither host nor graft
   - `{prefix}.ambiguous.1.fq.gz`: (few) ambiguous reads that cannot be classified,
-  
+
   and similarly with `.2.fq.gz`. For single-end reads, there is only `.fq.gz` (no numbers).
 
 The compression type can be specified using the `--compression` parameter.
@@ -254,7 +254,7 @@ xengsort index --index myindex -H host.fa.gz -G graft.fa.gz -n 9_000_000_000 --b
 If an index is build using the bisulfite parameter, also the read is converted in the classification step.
 It is not necessary to change the parameters of the classification step.
 Currently, it only supports the count-based classification.
-
+A detailed description on how the benchmarks on bisulfite sequencing data was performed can be found [here](https://github.com/S-Altend/MethSortBench/tree/main/03_read_sorting).
 
 Please report a [GitLab issue](https://gitlab.com/genomeinformatics/xengsort/-/issues) in case of problems.
 
@@ -268,13 +268,13 @@ Happy sorting!
 *  **Snakemake throws errors when attempting to download the example dataset from SRA**.
 
 This may happen for several reasons:
-  1. The dataset is quite large, over 20 GB. Make sure you have sufficient space (also temporary space) and sufficient disk quota (on a shared system) to store all of the files. 
+  1. The dataset is quite large, over 20 GB. Make sure you have sufficient space (also temporary space) and sufficient disk quota (on a shared system) to store all of the files.
   1. Another source of trouble may be that for some reason and older version of `sra-tools` gets called. We use the `fasterq-dump` tool to download the data, which did not exist in older verisons. To use a clean environment with a recent version of `sra-tools`, you can run snakemake as `snakemake -j 16 --use-conda`, which will generate a separate environment only for the download (this takes additional time, but may resolve the problem).
   1. In earlier versions, the dataset was downloaded from a different location which is no longer available. Please check that you are using at least v1.1.0 (`xengsort --version`). If not, please do a fresh clone and follow the tutorial to set up a new environment (delete the old one first) and run `pip install -e .`, as explained above in the tutorial.
 
 *  **Indexing stops after a few seconds with a failure.**
 
-The most likely cause of this is that you have not specified the size of the hash table (`-n` parameter). 
+The most likely cause of this is that you have not specified the size of the hash table (`-n` parameter).
 (From version 1.0.1, the `-n` parameter is required, and the error cannot occur anymore. Please update to the latest version!)
 For the human and mouse genomes this is approximately `-n 4_500_000_000` (4.5 billion).
 It is unfortunately a limitation of the implementation that the hash table size has to be specified in advance.
@@ -330,3 +330,14 @@ In addition, you may also cite the WABI 2020 proceedigs paper:
   annote =	{Keywords: xenograft sorting, alignment-free method, Cuckoo hashing, k-mer}
 }
 ```
+
+If you use xengsort to sort converted reads, please also cite our corresponding bioRxiv preprint:
+```
+@article {Altendorfer2026.10.06.756952,
+  author = {Altendorfer, Stefan and Zentgraf, Jens and Schmitz, Johanna Elena and Plass, Christoph and Gerh{\"a}user, Clarissa and Rahmann, Sven},
+  title = {High-performance xenograft read-sorting for DNA methylation analysis},
+  year = {2026},
+  doi = {10.64898/2026.10.06.756952},
+  journal = {bioRxiv}
+}
+```

{xengsort-2.2.0 → xengsort-2.2.2}/pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "xengsort"
-version = "2.2.0"
+version = "2.2.2"
 
 license = {file = "LICENSE"}
 

{xengsort-2.2.0 → xengsort-2.2.2}/xengsort/cptasks_kmers.py

@@ -544,15 +544,27 @@ def compile_cptask_insert_filtered_subkeys(
         myfilter,
         constant_value=0,
         maxfailures=0,
-        maxwalk=1000):
+        maxwalk=1000,
+        prefetch_distance=8):
     debugprint0, debugprint1, debugprint2 = debug.debugprint
     dassert = debug.dassert
+
+    @njit(nogil=True)
+    def dummy_prefetch(*_):
+        return
+
+    prefetch_filter = prefetch_table = dummy_prefetch
     if myfilter is not None:
         wwo = "with"
         lookup_and_insert_in_subfilter = myfilter.private.lookup_and_insert_in_subfilter
+        table_prefetch_distance = 0
+        filter_prefetch_distance = prefetch_distance
+        prefetch_filter = myfilter.private.prefetch_block_by_subkey_subfilter
     else:
         wwo = "without"
-
+        table_prefetch_distance = prefetch_distance
+        filter_prefetch_distance = 0
+        prefetch_table = myhashtable.private.prefetch_subkey
         @njit(nogil=True)
         def lookup_and_insert_in_subfilter(ft, st, subkey):
             return True
@@ -599,7 +611,12 @@ def compile_cptask_insert_filtered_subkeys(
             # dassert(buffer_type == 1, __name__, 'buffer_type == 1', buffer_type, message='0 means sequence buffer')
 
             # consume the buffer
-            for subkey in active_buffer[:nsubkeys]:
+            for ii in range(nsubkeys):
+                subkey = active_buffer[ii]
+                if filter_prefetch_distance and ii + filter_prefetch_distance < nsubkeys:
+                    prefetch_filter(ft, active_buffer[ii + prefetch_distance], st)
+                if table_prefetch_distance and ii + table_prefetch_distance < nsubkeys:
+                    prefetch_table(ht, active_buffer[ii + prefetch_distance], st)
                 if lookup_and_insert_in_subfilter(ft, st, subkey):
                     status, result = update(ht, st, subkey, value)  # see docstring below
                     if status & 128 == 0:
@@ -618,9 +635,11 @@ def compile_cptask_insert_filtered_subkeys(
 
 def compile_cptask_update_existing_subkeys(
         myhashtable,
-        constant_value=1):
+        constant_value=1,
+        prefetch_distance=8):
     debugprint0, debugprint1, debugprint2 = debug.debugprint
     update_existing = myhashtable.private.update_existing_ssk
+    prefetch = myhashtable.private.prefetch_subkey
     value = uint64(constant_value)
     debugprint1(f"- compiling cptask_update_existing_subkeys: {constant_value=}")
 
@@ -652,6 +671,8 @@ def compile_cptask_update_existing_subkeys(
 
             # consume the buffer
             for ii in range(nsubkeys):
+                if prefetch_distance and ii + prefetch_distance < nsubkeys:
+                    prefetch(ht, active_buffer[ii + prefetch_distance], st)
                 subkey = active_buffer[ii]
                 _, _ = update_existing(ht, st, subkey, value)  # see docstring below
                 # status, result = update_existing(ht, st, subkey, value)  # see docstring below

{xengsort-2.2.0 → xengsort-2.2.2}/xengsort/fastcash_main.py

@@ -7,6 +7,7 @@ from argparse import ArgumentParser
 from importlib import import_module  # dynamically import subcommand
 from importlib.metadata import metadata
 from os.path import split as splitpath, splitext
+from collections import defaultdict
 
 from .lowlevel.debug import set_debugfunctions
 
@@ -28,7 +29,7 @@ def add_hash_table_arguments(p: ArgumentParser):
         help="bucket size, i.e. number of elements on a bucket")
     p.add_argument("--fill", type=float, default="0.9",
         help="desired fill rate of the hash table")
-    p.add_argument("--subtables", type=int, default=5, 
+    p.add_argument("--subtables", type=int, default=5,
         help="number of subtables used; subtables+1 threads are used")
 
 
@@ -51,20 +52,20 @@ def add_hash_arguments(p: ArgumentParser):
         help="number of objects to be stored")
     p.add_argument("--type", default="default",  # default="3c_fbcbvb",
         help="hash type (e.g. [s]3c_fbcbvb, 2v_fbcbvb), implemented in hash_{TYPE}.py")
-    p.add_argument("--unaligned", action="store_const", 
+    p.add_argument("--unaligned", action="store_const",
         const=False, dest="aligned", default=None,
         help="use unaligned buckets (smaller, slightly slower; default)")
     p.add_argument("--aligned", action="store_const",
         const=True, dest="aligned", default=None,
         help="use power-of-two aligned buckets (faster, but larger)")
-    p.add_argument("--hashfunctions", "--functions", 
+    p.add_argument("--hashfunctions", "--functions",
         help="hash functions: 'default', 'random', or func1:func2[:func3]")
     # less important hash options
     p.add_argument("--nostatistics", "--nostats", action="store_true",
         help="do not compute or show index statistics at the end")
     p.add_argument("--maxwalk", metavar="INT", type=int, default=500,
         help="maximum length of random walk through hash table before failing [500]")
-    p.add_argument("--maxfailures", metavar="INT", type=int, default=0, 
+    p.add_argument("--maxfailures", metavar="INT", type=int, default=0,
         help="continue even after this many failures [default:0; forever:-1]")
     p.add_argument("--walkseed", type=int, default=7,
         help="seed for random walks while inserting elements [7]")

{xengsort-2.2.0 → xengsort-2.2.2}/xengsort/hash_s3c_fbcbvb.py

@@ -134,6 +134,22 @@ def build_hash(universe, n, subtables, bucketsize,
     def prefetch_bucket(table, subtable, bucket):
         startbit = subtable * subtablebits + bucket * bucketsizebits
         hprefetch(table, startbit)
+        hprefetch(table, startbit + uint64(bucketsizebits - 1))
+
+    @njit(nogil=True, locals=dict(
+        bucket=int64, startbit=uint64))
+    def prefetch_subkey(table, subkey, subtable):
+        bucket, _ = get_bs1(subkey)
+        startbit = subtable * subtablebits + bucket * bucketsizebits
+        hprefetch(table, startbit)
+
+        bucket, _ = get_bs2(subkey)
+        startbit = subtable * subtablebits + bucket * bucketsizebits
+        hprefetch(table, startbit)
+
+        bucket, _ = get_bs3(subkey)
+        startbit = subtable * subtablebits + bucket * bucketsizebits
+        hprefetch(table, startbit)
 
     @njit(nogil=True, locals=dict(
         bucket=int64, startbit=uint64))

{xengsort-2.2.0 → xengsort-2.2.2}/xengsort/io/fastaio.py

@@ -103,6 +103,7 @@ def cptask_read_fasta_into_buffers(fd, W, inbuffers, incontrol, ininfos, outbuff
     inside_seq = True
     header_seq = False
     nseq = 0
+    last_seq_length = 2 * W
     while True:
         old = nactive
         old_buffer = active_buffer
@@ -134,7 +135,11 @@ def cptask_read_fasta_into_buffers(fd, W, inbuffers, incontrol, ininfos, outbuff
             if ntodo < 0:
                 errorcode = ntodo
                 break
+            # If we have a long header but a seq smaller W, the offset can be bigger than W
+            if header_seq and last_seq_length < W:
+                break
             # Nothing read and the offset is not a full (gapped) k-mer
+            # I think this should trigger
             if offset < W:
                 break
             # Nothing read but we still have some data in the buffer
@@ -157,7 +162,6 @@ def cptask_read_fasta_into_buffers(fd, W, inbuffers, incontrol, ininfos, outbuff
                 offset = (W - 1)
         else:
             offset = end_buffer - linemarkbuffers[nactive, nseq - 1, 3]
-
         outcontrol[nactive, 7] = nseq
         outcontrol[nactive, 6] = 0  # buffer type sequence data
 
@@ -417,7 +421,7 @@ def fastaextract(args):
     seps = args.separators
     sfx = [args.suffix] if args.suffix else []
     seps = [_SEPARATORS.get(sep.upper(), sep) for sep in seps]
-    if items is None: 
+    if items is None:
         items = list()
         seps = list()
     if len(seps) == 1:
@@ -436,7 +440,7 @@ def fastaextract(args):
 def get_header_info(header, items, assigner, seps):
     fields = header.decode("ascii").split()
     assigners = [i for (i, field) in enumerate(fields) if assigner in field]
-    if 0 in assigners: 
+    if 0 in assigners:
         assigners.remove(0)
     D = dict()
     if items is None: