GenTS 0.1.0__py3-none-any.whl

GenTS-0.1.0.dist-info/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Cameron Cummins
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

GenTS-0.1.0.dist-info/METADATA

@@ -0,0 +1,179 @@
+Metadata-Version: 2.1
+Name: GenTS
+Version: 0.1.0
+Summary: A useful tool for post-processing CESM2 history files into the timeseries format.
+Author-email: Cameron Cummins <cameron.cummins@utexas.edu>
+Project-URL: Homepage, https://github.com/AgentOxygen/GenTS
+Project-URL: Issues, https://github.com/AgentOxygen/GenTS
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# Time Series Generation
+
+>Cameron Cummins<br>
+Computational Engineer<br>
+Contact: cameron.cummins@utexas.edu<br>
+Webpage: [https://www.jsg.utexas.edu/student/cameron_cummins](https://www.jsg.utexas.edu/student/cameron_cummins)<br>
+Affiliation: Persad Aero-Climate Lab, The University of Texas at Austin
+
+>Adam Phillips<br>
+Mentor and Advisor<br>
+Contact: asphilli@ucar.edu<br>
+Webpage: [https://staff.cgd.ucar.edu/asphilli/](https://staff.cgd.ucar.edu/asphilli/)<br>
+Affiliation: Climate and Global Dynamics Lab, National Center for Atmospheric Research
+
+## *This Project is still in Development*
+It works pretty well and can likely handle most cases, but some features aren't finished yet. Stay tuned for more!
+
+## Converting from History Files to Time Series Files
+Original model output produced by CESM2 is stored by timestep, with multiple geophysical variables stored in a single netCDF3 file together. These files are referred to as **history files** as each file effectively represents a snapshot of many variables at separate moments in time. This is intuitive from the model's perspective because variables are computed together, after solving many differential equations, at each step in the time series. However, this format is cumbersome for scientists because analysis is often performed over a large number of timesteps for only a few variables. A more practical format is to store all the timesteps for single variables in separate files. Datasets stored in this manner are known as **time series files**.
+
+## Capabilities and Current Limitations
+- [x] Functional core to convert history file to time series file in parallel with Dask
+- [x] Automatic path parsing
+- [x] Compatible with all model components of CESM
+- [x] Compatible with large ensembles
+- [x] Compatible with non-rectilinear grids
+- [x] Automatic path resolving and group detection (detects different experiments, timesteps, and other sub-directories)
+- [x] Customizable output directory structure that automatically reflects input directory structure
+- [x] Adjustable time chunk size
+- [x] Adjustable time range selection
+- [x] Customizable variable selection
+- [x] Adjustable file compression
+- [x] Resumable process (such as exceeding wall time or encountering an error)
+- [x] Documented API
+- [x] Fully written in Python (no NCO, CDO, or other subprocess commands)
+- [ ] (WIP) Verification tools for ensuring file integrity
+- [ ] (WIP) Command line interface
+- [ ] (WIP) Automatic cluster configuration/recommendation
+
+## Dependencies
+Ensure that you have the following packages installed. If you are using Casper, activate conda environment "npl-2024b".
+- `python >= 3.11.9`
+- `dask >= 2024.7.0`
+- `dask_jobqueue >= 0.8.5`
+- `netCDF4 >= 1.7.1`
+- `numpy >= 1.26.4`
+
+
+## How To Use
+The process of converting history files into time series files is almost entirely I/O dependent and benefits heavily from parallelism that increases data throughput. We leverage Dask to parallelize this process: reading in history files and writing out their respective time series datasets across multiple worker processes. These functions can be run without Dask, but it will execute in serial and likely be significantly slower.
+
+Start by cloning this repository to a directory of your choosing. This can be done by navigating to a directory and running the following command, assuming you have `git` installed:
+````
+git clone https://github.com/AgentOxygen/timeseries_generation.git
+````
+You can then either import `timeseries_generation.py` as a module in an interactive Jupyter Notebook (recommended) or in a custom Python script. ~~You may also execute it as a command from the terminal~~(not implemented yet).
+
+For those that learn faster by doing than reading, there is a template notebook available called `template_notebook.ipynb`.
+
+### Jupyter Notebook
+To use this package in an interactive Jupyter notebook, create a notebook in the repository directory and start a Dask cluster using either a LocalCluster or your HPC's job-queue. Be sure to activate an environment with the correct dependencies (this can be done either at the start of the Jupyter Server or by selecting the appropriate kernel in the bottom left bar). Here is an example of a possible cluster on Casper using the NPL 2024b conda environment and the PBS job scheduler:
+````
+from timeseries_generation import ModelOutputDatabase
+from dask_jobqueue import PBSCluster
+from dask.distributed import Client
+
+
+cluster = PBSCluster(
+    cores=20,
+    memory='40GB',
+    processes=20,
+    queue='casper',
+    resource_spec='select=1:ncpus=20:mem=40GB',
+    account='PROJECT CODE GOES HERE',
+    walltime='02:00:00',
+    local_directory="/local_scratch/"
+)
+
+cluster.scale(200)
+client = Client(cluster)
+````
+Here, we spin up 200 workers, each on one core with 2 GB of memory (so 200 total cores) with maximum wall clocks of 2 hours (note that this is done in PBS batches of 20 cores per machine). The `local_directory` parameter allows each worker to use their respective node's NVMe storage for scratch work if needed. Note that these cores may be spread across multiple machines. This process should not use more than two gigabytes of memory and does not utilize multi-threading (one process per core is optimal). How the other properties of the cluster should be configured (number of cores, number of different machines, and wall time) is difficult to determine, ~~but benchmarks are available below to get a good idea~~ (WIP). If generation process is interrupted, it can be resumed without re-generating existing time series data. Note that the ``Client`` object is important for connecting the Jupyter instance to the Dask cluster. You can monitor the Dask dashboard using the URL indicated by the `client` object (run just `client` in a cell block to get the Jupyter GUI). Once created, you can then initialize a ``ModelOutputDatabase`` class object with the appropriate parameters. Here is an example of common parameters you may use:
+````
+model_database = ModelOutputDatabase(
+    hf_head_dir="PATH TO HEAD DIRECTORY FOR HISTORY FILES (MULTIPLE MODEL RUNS, ONE RUN, OR INDIVIDUAL COMPONENTS)",
+    ts_head_dir="PATH TO HEAD DIRECTORY FOR REFLECTIVE DIRECTORY STRUCTURE AND TIME SERIES FILES",
+    dir_name_swaps={
+        "hist": "proc/tseries"
+    },
+    file_exclusions=[".once.nc", ".pop.hv.nc", ".initial_hist."]
+)
+````
+The ``ModelOutputDatabase`` object centralizes the API and acts as the starting point for all history-to-time-series operations. Creating the database will automatically search the input directories under `hf_head_dir` and group the history files by ensemble member, model component, timestep, and any other differences between subdirectories and naming patterns. The structures and names are arbitrary, so you can input a directory containing an ensemble of model archives or a single model component from a single run (or a random directory with history files in it, it should be able to handle it). It does not trigger any computation. `ts_head_dir` tells the class where to create a directory structure that matches the one under `hf_head_dir`. Only directories with `.nc` files will be reflected and optionally any other file names that contain the keywords specified by `file_exclusions` will be excluded. `dir_name_swaps` takes a Python dictionary with keys equal to directory names found under `hf_head_dir` that should be renamed. In the example above, all directories named `hist/` are renamed to two new directories `proc/tseries` to remain consistent with current time series generation conventions. There are many other parameters that can be specified to filter out which history files/variables are converted and how the time series files are formatted (see the section below titled *ModelOutputDatabase Parameters*). To trigger the time series generation process, call ``.run()``:
+````
+model_database.run()
+client.shutdown()
+````
+It is good practice to shutdown the cluster after the process is complete. If metadata of the history files needs to be inspected before generating time series (to verify which history files are being detected), ``.build()`` can be used to obtain metadata for all history files. This process is parallelized and is lightweight, typically taking less than a minute to run on a large-enough cluster. This function is automatically called by ``.run()`` before generating the time series datasets.
+
+You can monitor the Dask dashboard using the URL provided by the `client` object mentioned previously. This is a good way to track progress and check for errors.
+
+### What to Expect
+
+When you create ``ModelOutputDatabase``, it recursively searches the directory tree at `hf_head_dir`. If this tree is large, it may take some time. It is run in serial, so it won't show any information in the Dask dashboard, but you should see some print output. Calling `.run()` will first check to see if the database has been built, if not, it will automatically call `.build()` (which you can optionally call before `.run()` to inspect metadata). The `.build()` function aggregates the metadata for *all* of the history files under `hf_head_dir`. This process is parallelized using the Dask cluster and doesn't read much data, but opens a lot of files. This can take anywhere from less than a minute to 10 minutes depending on how the filesystem is organized. Once the metadata is aggregated in parallel, it is then processed in serial to for the arguments for time series generation. This step shouldn't take more than a few minutes and all progress will be communicated through print statements. After calculating the arguments, the `generateTimeSeries` is parallelized across the cluster with progress updated in the Dask dashboard. This is the final function call that actually preforms the time series generation by concatenating and merging variable data from the history files and writing them to disk. This process can take minutes to hours depending on the size of the history file output, the number of variables exported, the configuration of the Dask cluster, and traffic on the file system.
+
+Errors may occur due to hiccups in file-reads, but they tend to be rare. These errors are not resolved in real-time, but reported to the user so that a smaller second-pass can be preformed to fix any incomplete or missing time series files. You might also notice that some tasks take longer than others or that the rate of progress sometimes slows down. This is likely due to different variable shapes/sizes and variability in disk I/O.
+
+You may see a warning:
+````
+UserWarning: Sending large graph of size ....
+````
+It's a performance warning from Dask and doesn't mean anything is broken or that you did anything wrong. It might be the cause of some latency which I am hoping to eliminate in a future update, but just ignore it for now.
+
+By default, existing files are not overwritten, but this can be configured as shown in *ModelOutputDatabase Parameters*. Immidiately before the program releases the handle for a time series file, an attribute is appended called `timeseries_process` as explained in *Additional Attributes*. This attribute is used to verify that a file was properly created (being the last thing added, it indicates the process completed successfully). If it does not exist in the file with the correct value, the program will assume it is broken and needs to be replaced. This is to protect against unexpected interruptions such as system crashes, I/O errors, or wall-clock limits.
+
+## ModelOutputDatabase Parameters
+
+The ``ModelOutputDatabase`` constructor has multiple optional parameters that can be customized to fit your use-case:
+````
+class ModelOutputDatabase:
+    def __init__(self,
+                 hf_head_dir: str,
+                 ts_head_dir: str,
+                 dir_name_swaps: dict = {},
+                 file_exclusions: list = [],
+                 dir_exclusions: list = ["rest", "logs"],
+                 include_variables: list = None,
+                 exclude_variables: list = None,
+                 year_start: int = None,
+                 year_end: int = None,
+                 compression_level: int = None,
+                 variable_compression_levels: dict = None) -> None:
+````
+ - `hf_head_dir` : Required string/Path object, path to head directory to structure with subdirectories containing history files.
+ - `ts_head_dir` : Required string/Path object, path to head directory where structure reflecting `hf_head_dir` will be created and time series files will be written to.
+
+Note that for a Python class, the `self` parameter is an internal parameter skipped when creating the class object (`hf_head_dir` is the first parameter you can pass to `ModelOutputDatabase()`). You are only *required* to provide the first two parameters, `hf_head_dir` and `ts_head_dir`, but many assumptions will be made which you can also deduce from the default values in the constructor:
+1. All netCDF files under `hf_head_dir`, including all sub-directories except for "rest" and "logs", will be treated as history files.
+3. An identical directory structure to that of `hf_head_dir` will be created under `ts_head_dir` with the exception of time-step sub-directories (such as "month_1" and "day_1")
+4. The entire time series for all history files will be concatenated.
+5. All variables containing the time dimension will be concatenated and have time series files.
+6. No compression will be used (compression level 0).
+
+These defaults can be overwritten to customize how your time series files are generated:
+ - `dir_name_swaps` : Optional dictionary, dictionary for swapping out keyword directory names in the structure under `hf_head_dir` (e.g. `{"hist" : "proc/tseries"}`)
+ - `file_exclusions` : Optional list/array, file names containing any of the keywords in this list will be excluded from the database.
+ - `dir_exclusions` : Optional list, directory names containing any of the keywords in this list will be excluded from the database.
+ - `include_variables` : Optional list, variables to include in either creating individual time series files for adding as auxiliary variables.
+ - `exclude_variables` : Optional list, variables to exclude from either creating individual time series files for adding as auxiliary variables.
+ - `year_start` : Optional int, starting year for time series generation, must be later than first history file timestamp to have an effect.
+ - `year_end` : Optional int, ending year for time series generation, must be later than last history file timestamp to have an effect.
+ - `compression_level` : Optional int, compression level to pass to netCDF4 engine when generating time series files.
+ - `variable_compression_levels` : Optional dictionary, compression levels to apply to specific variables (variable name is key and the compression level is the value).
+
+## Additional Attributes
+When generating a new time series file, two new attributes are created in the global attributes dictionary:
+1. ``timeseries_software_version`` - Indicates the version of this software used to produce the time series dataset. This is important for identifying if the data is subject to any bugs that may exist in previous iterations of the code.
+2. ``timeseries_process`` - If the process is interrupted, some files may exist but not contain all the data or possibly be corrupted due to early termination of the file handle. This boolean attribute is created at initialization with a ``False`` value that is only set to ``True`` immediately before the file handle closes (it is the very last operation). When re-building the database after an interruption, this flag is checked to determine whether the file should be skipped or removed and re-computed.
+
+## Benchmarks
+WIP
+
+## Why not use Xarray?
+Xarray is an amazing tool that makes *most* big data processes run significantly faster with a lot less hassle (and it works seemlessly with Dask). Time series generation, however, has zero analytical computations involved and is almost entirely I/O bound. Data is simply read, concatenated, merged, and written. I (Cameron) tried many different approaches with Xarray, but they all had performance issues that aren't a problem in most other situations. For example, reading the metadata of *all* of the history files is quite useful for determining which history files go together and what variables are primary versus auxiliary. Reading metadata using xarray is pretty slow at the moment, but that may be due to the fact that it adds so much more functionality on top of standard netCDF operations (which we don't really need for this problem). **Xarray calls the netcdf4-Python engine under the hood, so we just use the engine directly instead**. There are other features that could potentially make Xarray viable in the future, such as taking advantage of chunking. However, current model output is in the netCDF-3 format, which does not support inherently chunking, and I couldn't figure out how to control which workers recieved specific file-sized chunks using the Xarray API (which is important for synchronization in parallel, otherwise resulting in lots of latency and memory usage).
+

GenTS-0.1.0.dist-info/RECORD

@@ -0,0 +1,7 @@
+gents/__init__.py,sha256=moRfDtyoRmf2BTU-HMRQmFJKFCjBoRitdETfkPN4kF0,26
+gents/ts_gen.py,sha256=g0q17W86YetWKtsRs5fQD7zuvoB3j82e4wOVF4MaA-Q,40094
+GenTS-0.1.0.dist-info/LICENSE,sha256=wApUf6Jb9isL6obtiIiUfvBW0Ak5CFn-QdwmH_kO888,1072
+GenTS-0.1.0.dist-info/METADATA,sha256=OEJcK8ZhvWo6CQZ9ZkKTNAh6iBNzjeGWk8wIeiVPJRY,17709
+GenTS-0.1.0.dist-info/WHEEL,sha256=Mdi9PDNwEZptOjTlUcAth7XJDFtKrHYaQMPulZeBCiQ,91
+GenTS-0.1.0.dist-info/top_level.txt,sha256=cknc8W-GnKZvhwglXIf9rXFYeyHbnftsljVD5qjGoYU,6
+GenTS-0.1.0.dist-info/RECORD,,

GenTS-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (73.0.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

GenTS-0.1.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+gents

gents/__init__.py

@@ -0,0 +1 @@
+from gents.ts_gen import *

gents/ts_gen.py

@@ -0,0 +1,967 @@
+#!/usr/bin/env python
+"""
+genTS.py
+
+Automatic system for generating time-series datasets from history files
+efficiently by leveraging the netCDF4 engine and Dask.
+
+Developer: Cameron Cummins
+Contact: cameron.cummins@utexas.edu
+Last Header Update: 8/22/24
+"""
+from time import time
+import numpy as np
+from pathlib import Path
+from dask import delayed
+import netCDF4
+import cftime
+from os.path import isfile, getsize
+from os import remove
+import dask.distributed
+import dask.bag as db
+
+
+def generateReflectiveOutputDirectory(input_head: Path,
+                                      output_head: Path,
+                                      parent_dir: Path,
+                                      swaps: dict = {}) -> str:
+    r"""Generates similar directory structure under new head directory.
+
+    This function allows for generating an output directory structure
+    that reflects the strucutre of the input directory structure,
+    typically containing the history files, specified by `input_head`
+    and `parent_dir`. The directory path return can then be created to
+    generate a similar structure for the timeseries files.
+
+    The parent directory is compared against the input head directory to
+    identify subdirectories. The subdirectories are then appended to the
+    output head directory. Swaps are preformed to replace keyword
+    subdirectories with a new directory name or sequence of subdirectories.
+
+    Parameters
+    ----------
+    input_head : Path
+        Path for head directory of input directory structure, typically
+        the head directory for the history files.
+    output_head : Path
+        Path for head directory of output directory structure, typically
+        the head directory for the timeseries files.
+    parent_dir : Path
+        Path for to subdirectory under `input_head`, typically containing
+        the history files (parent directory).
+    swaps: dict
+        Dictionary containing keyword swaps for the returned directory
+        structure. For example, ``hist`` may replace ``proc/tseries/``
+        (e.g. {``hist``: ``proc/tseries/``}).
+
+    Returns
+    -------
+    output_dir : str
+        New directory structure under `output_head` that reflects `parent_dir`
+        and contains any keywords swaps specified by `swaps`.
+    """
+    raw_sub_dir = str(parent_dir).split(str(input_head))[-1]
+    raw_sub_dir_parsed = raw_sub_dir.split("/")
+    for key in swaps:
+        for index in range(len(raw_sub_dir_parsed)):
+            if raw_sub_dir_parsed[index] == key:
+                raw_sub_dir_parsed[index] = swaps[key]
+                break
+    output_dir = str(output_head) + "/"
+    for dir in raw_sub_dir_parsed:
+        output_dir += dir + "/"
+    return output_dir
+
+
+def solveForGroupsLeftToRight(file_names: list,
+                              delimiter: str = ".") -> list:
+    r"""Compares file name substrings to identify groups.
+
+    Each string in `file_names` is converted to a string and parsed using
+    the delimiter specified by `delimiter`. Each sequence of resutling
+    substrings is compared from left to right, until a difference is
+    identified. Paths are then grouped based on these differences.
+
+    The ``nc`` substring is excluded from the comparison
+    of substrings since all history files will contain this substring.
+
+    Each groups unique substring is returned, but the list of history file
+    names for each group is not returned because some groups may be inclusive
+    of other groups, resulting in overlap between history files. This is solved
+    for in `generateTimeSeriesGroupPaths`.
+
+    Therefore, this function should be called for each unique output directory
+    to group history files that may be different by timestep or some other
+    unique substring in the file names.
+
+    Parameters
+    ----------
+    file_names : list(str)
+        List of string-like objects containing the names of history files.
+    delimiter : str
+        String to parse each file name by to produce substrings for
+        comparison.
+
+    Returns
+    -------
+    groups : list(str)
+        List of unique substrings identified in file names.
+    """
+    varying_parsed_names = [str(name).split(delimiter) for name in file_names]
+
+    lengths = {}
+    for parsed in varying_parsed_names:
+        if len(parsed) in lengths:
+            lengths[len(parsed)].append(parsed)
+        else:
+            lengths[len(parsed)] = [parsed]
+
+    groups = []
+    for length_index in lengths:
+        parsed_names = np.array(lengths[length_index])
+        unique_strs = []
+        for i in range(parsed_names.shape[1]):
+            unique_strs.append(np.unique(parsed_names[:, i]))
+        common_str = ""
+        for index in range(len(unique_strs)):
+            if len(unique_strs[index]) == 1:
+                if unique_strs[index][0] != "nc":
+                    common_str += unique_strs[index][0] + delimiter
+            else:
+                break
+        if len(unique_strs[index]) != len(parsed_names):
+            for group in unique_strs[index]:
+                groups.append(common_str + group + delimiter)
+        else:
+            groups.append(common_str)
+    return groups
+
+
+def generateTimeSeriesGroupPaths(paths: list,
+                                 input_head_dir: Path,
+                                 output_head_dir: Path,
+                                 dir_name_swaps: dict = {}) -> dict:
+    r"""Groups history files together by common substring patterns.
+
+    This function returns a dictionary that stores output templates
+    as keys and their corresponding history file paths as values.
+    Each output template is an incomplete path with a full parent
+    directory and partial file name. The parent directory path indicates
+    which directory the timeseries files produced by the respective history
+    file paths (stored in a list as the value to the output template which
+    serves as the key). The partial file name is the suffix for which
+    timeseries files produced from that particular collection of history
+    files will be named from.
+
+    This primiarly accomplishes two things:
+        1. History files are appropriately grouped by differences in the
+           file names and that these groups are carried over into the
+           file names for the timeseries datasets.
+        2. Multiple groups can exist together in the same directory. This
+           is typically the case for model output that contains more than
+           one average timestep.
+
+    Parameters
+    ----------
+    paths : list(Path)
+        List of Path objects that point to a large, unsorted, unfiltered list
+        of history files.
+    input_head_dir : Path
+        Path for head directory of input directory structure for the history
+        file locations stored in `path`, used to generate reflective directory
+        structure under `output_head_dir`.
+    output_head_dir : Path
+        Path to head direcory that reflective subdirectories will be generated
+        under for outputing timeseries files.
+    dir_name_swaps: dict
+        Dictionary containing keyword swaps for output directory structure.
+        For example, ``hist`` may replace ``proc/tseries/``
+        (e.g. {``hist``: ``proc/tseries/``}).
+
+    Returns
+    -------
+    timeseries_groups : dict
+        Dictionary containing output templates as keys and the paths to their
+        corresponding history files as values.
+    """
+    timeseries_groups = {}
+    parent_directories = {}
+    for path in paths:
+        if path.parent in parent_directories:
+            parent_directories[path.parent].append(path)
+        else:
+            parent_directories[path.parent] = [path]
+
+    for parent in parent_directories:
+        groups = solveForGroupsLeftToRight([path.name for path in parent_directories[parent]])
+        conflicts = {}
+        for group in groups:
+            for comparable_group in groups:
+                if group != comparable_group and group in comparable_group:
+                    if group in conflicts:
+                        conflicts[group].append(comparable_group)
+                    else:
+                        conflicts[group] = [comparable_group]
+        group_to_paths = {group: [] for group in groups}
+        for path in parent_directories[parent]:
+            for group in groups:
+                if group in str(path.name):
+                    if group in conflicts:
+                        conflict = False
+                        for conflict_group in conflicts[group]:
+                            if conflict_group in str(path.name):
+                                conflict = True
+                                break
+                        if not conflict:
+                            group_to_paths[group].append(path)
+                    else:
+                        group_to_paths[group].append(path)
+
+        group_output_dir = generateReflectiveOutputDirectory(input_head_dir, output_head_dir, parent, swaps=dir_name_swaps)
+        for group in group_to_paths:
+            timeseries_groups[Path(group_output_dir + "/" + group)] = group_to_paths[group]
+    return timeseries_groups
+
+
+def isVariableAuxiliary(variable_meta: dict,
+                        auxiliary_dims: list = ["nbnd", "chars", "string_length", "hist_interval"],
+                        primary_dims: list = ["time"]) -> bool:
+    r"""Determines if a variable is auxiliary based on metadata.
+
+    This function determins which variables are deemed ``auxiliary``
+    and thus stored in every resulting timeseries file. They should
+    be small variables, typically of one dimension. Coordinate variables
+    should meet this criteria, but some descriptive variables may have
+    additional dimensions that are small in size.
+
+    For now, these variables are identified by containing any of the dimensions
+    specified in `auxiliary_dims`.
+
+    Some variables are also small enough to fit in every timeseries file
+    because they do not evolve with time (do not contain the time dimension).
+    Variables that do not contain dimensions in `auxiliary_dims` will still
+    be considered auxiliary if they do not any of the contain dimensions
+    specified in `primary_dims`.
+
+    The defautls are configured somewhat arbitrarily and will likely
+    need to be improved for comaptability with other model runs (and
+    definitely for other models). We may also need to implement a new method
+    for identifying auxiliary variables other than just dimensions.
+
+    Parameters
+    ----------
+    variable_meta : dict
+        Dictionary containing metadata produced by `getHistoryFileMetaData`
+
+    Returns
+    -------
+    bool
+        True if the variable is auxiliary, false if it is not.
+    """
+    dims = np.unique(variable_meta["dimensions"])
+
+    for tag in auxiliary_dims:
+        if tag in dims:
+            return True
+
+    if len(dims) > 1:
+        for tag in primary_dims:
+            if tag in dims:
+                return False
+
+    return True
+
+
+def getHistoryFileMetaData(hs_file_path: Path) -> dict:
+    r"""Builds dictionary containing metadata information for history file.
+
+    This function reads coordinate and attribute information in the netCDF
+    file to obtian metadata quickly. No variable data is loaded, so this
+    function call is relatively lightweight and easily parallelized.
+
+    The metadata pulled is specific to the timeseries generation stack and
+    stored loosely in a dictionary. This may be converted into a class of
+    it's own in the future.
+
+    Parameters
+    ----------
+    hs_file_path : Path
+        Path to history file to pull metadata from.
+
+    Returns
+    -------
+    meta : dict
+        Dictionary containing useful metadata for specified history file.
+    """
+    meta = {}
+    ds = netCDF4.Dataset(hs_file_path, mode="r")
+    try:
+        meta["file_size"] = getsize(hs_file_path)
+        meta["variables"] = list(ds.variables)
+        meta["global_attrs"] = {key: getattr(ds, key) for key in ds.ncattrs()}
+        meta["variable_meta"] = {}
+        if "time" in meta["variables"]:
+            meta["time"] = cftime.num2date(ds["time"][:], units=ds["time"].units, calendar=ds["time"].calendar)
+        else:
+            meta["time"] = None
+    
+        for variable in meta["variables"]:
+            meta["variable_meta"][variable] = {}
+            if type(ds[variable]) is netCDF4._netCDF4._Variable:
+                for key in ds[variable].ncattrs():
+                    meta["variable_meta"][variable][key] = ds[variable].__getattr__(key)
+            else:
+                for key in ds[variable].ncattrs():
+                    meta["variable_meta"][variable][key] = ds[variable].getncattr(key)
+    
+            meta["variable_meta"][variable]["dimensions"] = list(ds[variable].dimensions)
+            meta["variable_meta"][variable]["data_type"] = ds[variable].dtype
+            meta["variable_meta"][variable]["shape"] = ds[variable].shape
+    
+        meta["primary_variables"] = []
+        meta["auxiliary_variables"] = []
+        for variable in meta["variable_meta"]:
+            if isVariableAuxiliary(meta["variable_meta"][variable]):
+                meta["auxiliary_variables"].append(variable)
+            else:
+                meta["primary_variables"].append(variable)
+    except Exception as e:
+        meta["exception"] = e
+    ds.close()
+    return meta
+
+
+def getYearSlices(years: list, slice_length: int) -> list:
+    r"""Generates list of index tuples for slicing time array into year chunks.
+
+    Given a list of years, this function will divide the list into chunks, with
+    maximum length specified by `slice_length`. The chunking is configured
+    to align boundary years to multiples of the slice length. In other words,
+    the timeseries may begin and end with a slice chunk shorter than
+    `slice length`. For example, if a timeseries begins in 2015 and ends in
+    2092 is chunked by 10 years, the first chunk will be 2015-2019 and the last
+    chunk will be 2090-2092. Everything between will be 10 years (2020-2029,
+    2030-2039, etc).
+
+    Parameters
+    ----------
+    years : list
+        Path to history file to pull metadata from.
+    slice_length : int
+        Number of years per chunk to slice.
+
+    Returns
+    -------
+    meta : dict
+        Dictionary containing useful metadata for specified history file.
+    """
+    slices = []
+    last_slice_yr = years[0]
+    for index in range(len(years)):
+        if years[index] % slice_length == 0 and years[index] != last_slice_yr:
+            if len(slices) == 0:
+                slices.append((0, index))
+            else:
+                slices.append((slices[-1][1], index))
+            last_slice_yr = years[index]
+    if len(slices) == 0:
+        slices.append((0, index + 1))
+    elif slices[-1][1] != index + 1:
+        slices.append((slices[-1][1], index + 1))
+    return slices
+
+
+def generateTimeSeries(output_template: Path,
+                       hf_paths: list,
+                       auxiliary_variables: list,
+                       primary_variables: list,
+                       time_str_format: str,
+                       compression_level : int = None,
+                       overwrite: bool = False,
+                       debug_timing: bool = True) -> tuple:
+    r"""Generates timeseries files from history files.
+
+    This is the primary function for generating the timeseries files and
+    controls most of the I/O processing. This function can operate on one
+    timeseries file at a time and access history files independently,
+    thus this function can be called in parallel across multiple workers.
+
+    All history files specified in `hf_paths` are opened and aggregated along
+    the time dimension. They are assumed to be of the same group. The
+    ``primary_variables`` key in `metadata` specifies which variables in each
+    of the history files should be aggregated. Each variable is read separately
+    and computed within this function call. Therefore, this list should be
+    short for each function call, therefore more calls can be made on other
+    workers for greater throughput. However, this list can be greater than one
+    variable if this process needs to be consolidated.
+
+    The naming of each timeseries files is determined via the `output_template`
+    and `time_str_format`.
+
+    Parameters
+    ----------
+    output_template : Path
+        Incomplete path with parent subdirectory to output timeseries file to
+        and name to use as suffix in naming it.
+    hf_paths : list
+        List of paths pointing to the history files from which these timeseries
+        files will be generated from.
+    auxiliary_variables : list
+        
+    primary_variables : list
+        
+    time_str_format : str
+        Date format code used to convert the CFTime object to a string. This
+        may vary for different timesteps and should be determined before
+        calling this function.
+    compression_level : int
+        
+    overwrite : bool
+        Whether or not to overwrite timeseries files if they already exist at
+        the generated paths (Default: False).
+    debug_timing : bool
+        Includes the time to generate each resulting timeseries file in the
+        output tuples.
+
+    Returns
+    -------
+    ts_paths : list
+        Path to each timeseries file generated.
+    tuple (if debug_timing)
+        Time to compute timeseries at each path in `ts_paths`.
+    """
+    debug_start_time = time()
+    output_template.parent.mkdir(parents=True, exist_ok=True)
+
+    auxiliary_ds = netCDF4.MFDataset(hf_paths, aggdim="time", exclude=primary_variables)
+    auxiliary_ds.set_auto_mask(False)
+    auxiliary_ds.set_auto_scale(False)
+    auxiliary_ds.set_always_mask(False)
+
+    time_start_str = cftime.num2date(auxiliary_ds["time"][0],
+                                     units=auxiliary_ds["time"].units,
+                                     calendar=auxiliary_ds["time"].calendar).strftime(time_str_format)
+    time_end_str = cftime.num2date(auxiliary_ds["time"][-1],
+                                   units=auxiliary_ds["time"].units,
+                                   calendar=auxiliary_ds["time"].calendar).strftime(time_str_format)
+
+    auxiliary_variable_data = {}
+    for auxiliary_var in auxiliary_variables:
+        attrs = {}
+        for key in auxiliary_ds[auxiliary_var].ncattrs():
+            if type(auxiliary_ds[auxiliary_var]) is netCDF4._netCDF4._Variable:
+                attrs[key] = auxiliary_ds[auxiliary_var].__getattr__(key)
+            else:
+                attrs[key] = auxiliary_ds[auxiliary_var].getncattr(key)
+        auxiliary_variable_data[auxiliary_var] = {
+            "attrs": attrs,
+            "dimensions": auxiliary_ds[auxiliary_var].dimensions,
+            "shape": auxiliary_ds[auxiliary_var].shape,
+            "dtype": auxiliary_ds[auxiliary_var].dtype,
+            "data": auxiliary_ds[auxiliary_var][:],
+        }
+    auxiliary_ds.close()
+
+    primary_ds = netCDF4.MFDataset(hf_paths, aggdim="time", exclude=auxiliary_variables)
+    primary_ds.set_auto_mask(False)
+    primary_ds.set_auto_scale(False)
+    primary_ds.set_always_mask(False)
+
+    complevel = compression_level
+    compression = None
+    if complevel > 0:
+        compression = "bzip2"
+    
+    ts_paths = []
+    for primary_var in primary_variables:
+        ts_path = output_template.parent / f"{output_template.name}{primary_var}.{time_start_str}.{time_end_str}.nc"
+        ts_paths.append(ts_path)
+        if isfile(ts_path) and not overwrite:
+            try:
+                ds = netCDF4.Dataset(ts_path, mode="r")
+                attrs = {key: getattr(ds, key) for key in ds.ncattrs()}
+                ds.close()
+                if "timeseries_process_complete" in attrs and attrs["timeseries_process"] == "complete":
+                    continue
+                else:
+                    remove(ts_path)
+            except OSError:
+                remove(ts_path)
+        elif isfile(ts_path) and overwrite:
+            remove(ts_path)
+        ts_ds = netCDF4.Dataset(ts_path, mode="w")
+        for dim_index, dim in enumerate(primary_ds[primary_var].dimensions):
+            if dim not in ts_ds.dimensions:
+                if dim == "time":
+                    ts_ds.createDimension(dim, None)
+                else:
+                    ts_ds.createDimension(dim, primary_ds[primary_var].shape[dim_index])
+
+        var_data = ts_ds.createVariable(primary_var,
+                                        primary_ds[primary_var].dtype,
+                                        primary_ds[primary_var].dimensions, complevel=complevel, compression=compression)
+        var_data.set_auto_mask(False)
+        var_data.set_auto_scale(False)
+        var_data.set_always_mask(False)
+
+        attrs = {}
+        if type(primary_ds[primary_var]) is netCDF4._netCDF4._Variable:
+            for key in primary_ds[primary_var].ncattrs():
+                attrs[key] = primary_ds[primary_var].__getattr__(key)
+        else:
+            for key in primary_ds[primary_var].ncattrs():
+                attrs[key] = primary_ds[primary_var].getncattr(key)
+
+        ts_ds[primary_var].setncatts(attrs)
+
+        time_chunk_size = 1
+        if len(primary_ds[primary_var].shape) > 0 and "time" in primary_ds[primary_var].dimensions:
+            for i in range(0, primary_ds[primary_var].shape[0], time_chunk_size):
+                if i + time_chunk_size > primary_ds[primary_var].shape[0]:
+                    time_chunk_size = primary_ds[primary_var].shape[0] - i
+                var_data[i:i + time_chunk_size] = primary_ds[primary_var][i:i + time_chunk_size]
+        else:
+            var_data[:] = primary_ds[primary_var][:]
+
+        for auxiliary_var in auxiliary_variable_data:
+            for dim_index, dim in enumerate(auxiliary_variable_data[auxiliary_var]["dimensions"]):
+                if dim not in ts_ds.dimensions:
+                    ts_ds.createDimension(dim, auxiliary_variable_data[auxiliary_var]["shape"][dim_index])
+            aux_var_data = ts_ds.createVariable(auxiliary_var,
+                                                auxiliary_variable_data[auxiliary_var]["dtype"],
+                                                auxiliary_variable_data[auxiliary_var]["dimensions"])
+            aux_var_data.set_auto_mask(False)
+            aux_var_data.set_auto_scale(False)
+            aux_var_data.set_always_mask(False)
+            ts_ds[auxiliary_var].setncatts(auxiliary_variable_data[auxiliary_var]["attrs"])
+
+            aux_var_data[:] = auxiliary_variable_data[auxiliary_var]["data"]
+        ts_ds.setncatts({key: getattr(primary_ds, key) for key in primary_ds.ncattrs()} | {"timeseries_software_version": "0.8", "timeseries_process": "complete"})
+        ts_ds.close()
+
+    if debug_timing:
+        return (time() - debug_start_time, ts_paths)
+    else:
+        return ts_paths
+
+
+class ModelOutputDatabase:
+    r"""Database for centralizing all history-to-timeseries file operations.
+
+    Attributes
+    ----------
+    None
+
+    Methods
+    -------
+    getHistoryFileMetaData()
+        Returns metadata for particulary history file.
+    getTimeSeriesGroups()
+        Returns list of paths to detected history files grouped.
+    getHistoryFilePaths()
+        Returns list of paths to detected history files.
+    getTimeStepHours()
+        Determines timestep for group history files.
+    getTimeStepStr()
+        Determines timestep label for group of history files.
+    getTimeStepStrFormat()
+        Determines date string format code based on timstep label.
+    build()
+        Builds database by reading each history file and storing metadata.
+    run()
+        Starts process for generating timeseries files.
+    """
+
+    def __init__(self,
+                 hf_head_dir: str,
+                 ts_head_dir: str,
+                 dir_name_swaps: dict = {},
+                 file_exclusions: list = [],
+                 dir_exclusions: list = ["rest", "logs"],
+                 timeseries_year_length: int = 10,
+                 overwrite: bool = False,
+                 include_variables: list = None,
+                 exclude_variables: list = None,
+                 year_start: int = None,
+                 year_end: int = None,
+                 compression_level: int = None,
+                 variable_compression_levels: dict = None) -> None:
+        r"""Parameters
+        ----------
+        hf_head_dir : str
+            Path to head directory to structure with subdirectories containing
+            history files.
+        ts_head_dir : str
+            Path to head directory where structure reflecting `hf_head_dir`
+            will be created and timeseries files will be written to.
+        dir_name_swaps : dict
+            Dictionary for swapping out keyword directory names in the
+            structure under `hf_head_dir` (e.g. ``{"hist" : "proc/tseries"}``
+        file_exclusions : list
+            File names containing any of the keywords in this list will be
+            excluded from the database.
+        dir_exclusions : list
+            Directory names containing any of the keywords in this list will be
+            excluded from the database.
+        timeseries_year_length : int
+            Number of years each timeseries file should be chunked to using
+            `getYearSlices` (Default: 10).
+        overwrite : bool
+            Whether or not to overwrite timeseries files if they already exist
+            at the generated paths (Default: False).
+        include_variables : list
+            Variables to include in either creating individual timeseries files
+            for adding as auxiliary variables.
+        exclude_variables : list
+            Variables to exclude from either creating individual timeseries
+            files for adding as auxiliary variables.
+        year_start : int
+            Starting year for timeseries generation, must be later than first
+            history file timestamp to have an effect.
+        year_end : int
+            Ending year for timeseries generation, must be later than last
+            history file timestamp to have an effect.
+        compression_level : int
+            Compression level to pass to netCDF4 engine when generating
+            timeseries files.
+        variable_compression_levels : dict
+            Compression levels to apply to specific variables (variable name is
+            key and the compression level is the value).
+        """
+        self.log("Initializing...")
+        self.__hf_head_dir = Path(hf_head_dir)
+        self.__ts_head_dir = Path(ts_head_dir)
+        self.__overwrite = overwrite
+        self.__timeseries_year_length = timeseries_year_length
+        self.__include_variables = include_variables
+        self.__exclude_variables = exclude_variables
+        if self.__exclude_variables is None:
+            self.__exclude_variables = []
+        self.__year_start = year_start
+        self.__year_end = year_end
+        self.__compression_level = compression_level
+        if self.__compression_level is None:
+            self.__compression_level = 0
+        self.__variable_compression_levels = variable_compression_levels
+        self.__total_size = 0
+        self.__built = False
+
+        start_t = time()
+        self.log(f"Searching tree for netCDF files: '{hf_head_dir}'")
+        self.__history_file_paths = []
+        for path in sorted(self.__hf_head_dir.rglob("*.nc")):
+            exclude = False
+            for exclusion in file_exclusions:
+                if exclusion in path.name:
+                    exclude = True
+
+            directory_names = [directory.name for directory in sorted(path.parents)]
+
+            for exclusion in dir_exclusions:
+                if exclusion in directory_names:
+                    exclude = True
+
+            if not exclude:
+                self.__history_file_paths.append(path)
+        self.log("\tDone.")
+        start_t = time()
+        self.log("Grouping history files... ")
+        self.__timeseries_group_paths = generateTimeSeriesGroupPaths(self.__history_file_paths, hf_head_dir, ts_head_dir, dir_name_swaps=dir_name_swaps)
+        self.log("Done.")
+
+    def log(self, msg, end="\n"):
+        print(msg, end=end)
+
+    def getHistoryFileMetaData(self,
+                               history_file_path: Path) -> dict:
+        r"""Returns metadata for particulary history file.
+
+        Metadata is generated by `getHistoryFileMetaData`.
+        """
+        return self.__history_file_metas[history_file_path]
+
+    def getTimeSeriesGroups(self):
+        r"""Returns list of paths to detected history files grouped.
+
+        Groups are determined by `generateTimeSeriesGroupPaths`,
+        based on which directories the history files are sotred in and
+        substrings in the file names.
+        """
+        return self.__timeseries_group_paths
+
+    def getHistoryFilePaths(self):
+        r"""Returns list of paths to detected history files."""
+        return self.__history_file_paths
+
+    def getTimeStepHours(self,
+                         hf_paths: list):
+        r"""Determines timestep for group history files.
+
+        Given a group of history files in a timeseries, this function
+        determines what the timestep is in hours.
+
+        Note that this function relies on the time coordinate obtained from
+        metadata, not the file name.
+
+        Parameters
+        ----------
+        hf_paths : list
+            List of paths to history files.
+
+        Returns
+        -------
+        float
+            Timestep in hours.
+        """
+        times = []
+        for path in hf_paths:
+            hf_time = self.__history_file_metas[path]["time"]
+            if len(hf_time) == 0:
+                return 0
+            elif len(hf_time) == 1:
+                times.append(hf_time[0])
+            else:
+                times.append(hf_time[0])
+                times.append(hf_time[1])
+                break
+
+        times.sort()
+        if len(times) == 1:
+            return 0.0
+        else:
+            return (times[1] - times[0]).total_seconds() / 60 / 60
+
+    def getTimeStepStr(self,
+                       hf_paths: list) -> str:
+        r"""Determines timestep label for group of history files.
+
+        Given a group of history files in a timeseries, this function
+        determines what group label should be used for the parent directory.
+        This is also used to determine how the time range is shown in the
+        file names.
+
+        The timestep is obtained from `ModelOutputDatabase.getTimeStepHours`
+
+        Parameters
+        ----------
+        hf_paths : list
+            List of paths to history files.
+
+        Returns
+        -------
+        str
+            Label for timestep
+        """
+        if "time_period_freq" in self.__history_file_metas[hf_paths[0]]["global_attrs"]:
+            return self.__history_file_metas[hf_paths[0]]["global_attrs"]["time_period_freq"]
+
+        dt_hrs = self.getTimeStepHours(hf_paths)
+        if dt_hrs >= 24*364:
+            return f"year_{int(np.ceil(dt_hrs / (24*365)))}"
+        elif dt_hrs >= 24*28:
+            return f"month_{int(np.ceil(dt_hrs / (24*31)))}"
+        elif dt_hrs >= 24:
+            return f"day_{int(np.ceil(dt_hrs / (24)))}"
+        else:
+            return f"hour_{int(np.ceil(dt_hrs))}"
+
+    def getTimeStepStrFormat(self,
+                             timestep_str: str) -> str:
+        r"""Determines date string format code based on timstep label.
+
+        This function determines how the date ranges for each timeseries
+        dataset is shown in the file name.
+
+        Parameters
+        ----------
+        timestep_str : str
+            Label for timestep group (e.g. ``day_1``).
+
+        Returns
+        -------
+        str
+            Date string format code for converting CFTime object to string for
+            file naming.
+        """
+        if "hour" in timestep_str:
+            return "%Y-%m-%d-%H"
+        elif "day" in timestep_str:
+            return "%Y-%m-%d"
+        elif "month" in timestep_str:
+            return "%Y-%m"
+        elif "year" in timestep_str:
+            return "%Y"
+        else:
+            return "%Y-%m-%d-%H"
+
+    def build(self,
+              client: dask.distributed.Client = None) -> None:
+        r"""Builds database by reading each history file and storing metadata.
+
+        Should be called before `run()`
+
+        This only reads metadata (no variable data is loaded) and is therefore
+        lightweight. This function recursviely reads the directory structure
+        starting at the head directory specified at class initialization.
+
+        Although reading metadata is lightweight, latency can increase with
+        a large number of files. Therefore, Dask is used to convert metadata
+        calls into lazy delayed functions and execute on a cluster. Unless a
+        client is specified, it will automatically detect the Dask global
+        client if it exists. If neither exists, then it proceeds in serial.
+
+        Parameters
+        ----------
+        client : dask.distributed.Client
+            Client object to use for Dask parallelization.
+
+        Returns
+        -------
+        None
+        """
+        start_bt = time()
+        self.log("Starting Build.")
+        if client is None:
+            client = dask.distributed.client._get_global_client()
+
+        self.__gen_ts_args_templates = []
+        self.__gen_ts_args_hf_paths = []
+        self.__gen_ts_args_auxiliary_vars = []
+        self.__gen_ts_args_primary_vars = []
+        self.__gen_ts_args_time_formats = []
+        self.__gen_ts_args_comp_levels = []
+        self.__gen_ts_args_overwrites = []
+
+        start_t = time()
+        self.__history_file_metas = {}
+        if client is None:
+            self.log("Dask client not detected, proceeding serially.")
+            self.log("\tGathering metadata...")
+            for path in self.__history_file_paths:
+                self.__history_file_metas[path] = getHistoryFileMetaData(path)
+            self.log("\t Done.")
+        else:
+            self.log("Dask client detected.")
+            self.log("\tGathering metadata...")
+            bag = db.from_sequence(self.__history_file_paths, npartitions=20000).map(getHistoryFileMetaData)
+            metas = bag.compute()
+            for index, path in enumerate(self.__history_file_paths):
+                self.__history_file_metas[path] = metas[index]
+            self.log("\tDone.")
+
+        start_t = time()
+        self.log("\tComputing timeseries arguments...")
+        new_timeseries_group_paths = {}
+        for path_template in self.__timeseries_group_paths:
+            hs_file_paths = self.__timeseries_group_paths[path_template]
+            timestep_str = self.getTimeStepStr(hs_file_paths)
+            new_path_template = (path_template.parent / timestep_str) / path_template.name
+            new_timeseries_group_paths[new_path_template] = hs_file_paths
+        self.__timeseries_group_paths = new_timeseries_group_paths
+
+        for meta in metas:
+            self.__total_size += meta["file_size"]
+
+        self.__generateTimeSeries_args = []
+        for output_template in self.getTimeSeriesGroups():
+            hf_paths = self.getTimeSeriesGroups()[output_template]
+            years = [self.getHistoryFileMetaData(hf_path)["time"][0].year for hf_path in hf_paths]
+
+            for start_index, end_index in getYearSlices(years, self.__timeseries_year_length):
+                within_range = True
+                if self.__year_start is not None and years[start_index] < self.__year_start:
+                    within_range = False
+                    for index in range(start_index, end_index):
+                        if years[start_index] >= self.__year_start:
+                            start_index = index
+                            within_range = True
+                            break
+                if self.__year_end is not None and years[end_index-1] > self.__year_end:
+                    within_range = False
+                    for index in range(start_index, end_index):
+                        if years[end_index-1] <= self.__year_end:
+                            end_index = index
+                            within_range = True
+                            break
+
+                if not within_range or end_index <= start_index:
+                    continue
+
+                slice_paths = hf_paths[start_index:end_index]
+                metadata = self.getHistoryFileMetaData(slice_paths[0])
+                time_str_format = self.getTimeStepStrFormat(self.getTimeStepStr(slice_paths))
+
+                for primary_variable in metadata["primary_variables"]:
+                    if self.__include_variables is not None and primary_variable not in self.__include_variables:
+                        continue
+                    if primary_variable not in self.__exclude_variables:
+                        primary_variables = np.array([primary_variable])
+                        auxiliary_variables = np.array(metadata["auxiliary_variables"])
+                        if self.__variable_compression_levels is not None and primary_variable in self.__variable_compression_levels:
+                            compression_level = self.__variable_compression_levels[primary_variable]
+                        else:
+                            compression_level = self.__compression_level
+                        self.__gen_ts_args_templates.append(output_template)
+                        self.__gen_ts_args_hf_paths.append(slice_paths)
+                        self.__gen_ts_args_auxiliary_vars.append(auxiliary_variables)
+                        self.__gen_ts_args_primary_vars.append(primary_variables)
+                        self.__gen_ts_args_time_formats.append(time_str_format)
+                        self.__gen_ts_args_comp_levels.append(compression_level)
+                        self.__gen_ts_args_overwrites.append(self.__overwrite)
+        self.log("\tDone.")
+        self.log("Build complete.")
+        self.__built = True
+
+    def getGenTSArgs(self):
+        return self.__generateTimeSeries_args
+
+    def run(self,
+            client: dask.distributed.Client = None,
+            serial: bool = False):
+        r"""Starts process for generating timeseries files.
+
+        This function generates the appropriate parameters for the database and
+        parallelizes `generateTimeSeries` using the Dask cluster if it exists
+        or runs in serial. The call is blocking and will wait for all
+        timeseries files to be written to disk before returning the paths.
+
+        Parameters
+        ----------
+        client : dask.distributed.Client
+            Client object to use for Dask parallelization.
+        serial : bool
+            Whether or not to force serial execution and not use Dask
+            (Default: False).
+
+        Returns
+        -------
+        ts_paths : list
+            List of paths pointing to timeseries files generated.
+        """
+        if not self.__built:
+            self.log("Building arguments.")
+            self.build(client=client)
+        else:
+            self.log("Build already complete. Skipping build step.")
+
+        if client is None:
+            client = dask.distributed.client._get_global_client()
+
+        ts_paths = []
+        if client is None or serial:
+            self.log("ERROR: No Dask client detected, serial run() not yet implemented.")
+            pass
+        else:
+            self.log("Dask client detected, mapping arguments to generateTimeSeries() in parallel.")
+            futures = client.map(generateTimeSeries,
+                                 self.__gen_ts_args_templates,
+                                 self.__gen_ts_args_hf_paths,
+                                 self.__gen_ts_args_auxiliary_vars,
+                                 self.__gen_ts_args_primary_vars,
+                                 self.__gen_ts_args_time_formats,
+                                 self.__gen_ts_args_comp_levels,
+                                 self.__gen_ts_args_overwrites)
+            self.log("Map complete, awaiting cluster computation...")
+            ts_paths = client.gather(futures, errors="skip")
+            self.log(f"\tDone.")
+        return ts_paths