teehr 0.4.1__tar.gz

teehr-0.4.1/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 RTI International
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

teehr-0.4.1/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.1
+Name: teehr
+Version: 0.4.1
+Summary: Tools for Exploratory Evaluation in Hydrologic Research
+License: GNU v3
+Author: RTI International
+Requires-Python: >=3.10,<4.0
+Classifier: License :: Other/Proprietary License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: arch (>=7.0.0,<8.0.0)
+Requires-Dist: bokeh (>=3.5.0,<4.0.0)
+Requires-Dist: dask[dataframe] (>=2024.11.1,<2025.0.0)
+Requires-Dist: dataretrieval (>=1.0.9,<2.0.0)
+Requires-Dist: duckdb (>=1.0.0,<2.0.0)
+Requires-Dist: fsspec (>=2023.10.0)
+Requires-Dist: gcsfs (>=2023.10.0)
+Requires-Dist: geopandas (>=0.14.0,<0.15.0)
+Requires-Dist: h5py (==3.12.1)
+Requires-Dist: httpx (>=0.25.1,<0.26.0)
+Requires-Dist: hydrotools-metrics (>=1.3.3,<2.0.0)
+Requires-Dist: hydrotools-nwis-client (>=3.3.1,<4.0.0)
+Requires-Dist: kerchunk (>=0.2.2,<0.3.0)
+Requires-Dist: netcdf4 (==1.6.5)
+Requires-Dist: numba (>=0.60.0,<0.61.0)
+Requires-Dist: pandas (>=2.2.0,<3.0.0)
+Requires-Dist: pandera[pyspark] (>=0.20.4,<0.21.0)
+Requires-Dist: pyarrow (>=15.0.0,<16.0.0)
+Requires-Dist: pydantic (>=2.4.2,<3.0.0)
+Requires-Dist: pyspark[pandas-on-spark] (>=3.5.2,<4.0.0)
+Requires-Dist: rasterio (>=1.3.9,<2.0.0)
+Requires-Dist: rioxarray (>=0.15.0,<0.16.0)
+Requires-Dist: s3fs (>=2023.10.0)
+Requires-Dist: ujson (>=5.8.0,<6.0.0)
+Requires-Dist: zarr (>=2.16.1,<3.0.0)
+Description-Content-Type: text/markdown
+
+![alt text](https://github.com/RTIInternational/teehr/blob/main/docs/images/teehr.png)
+
+| | |
+| --- | --- |
+| ![alt text](https://ciroh.ua.edu/wp-content/uploads/2022/08/CIROHLogo_200x200.png) | Funding for this project was provided by the National Oceanic & Atmospheric Administration (NOAA), awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003). |
+
+
+# TEEHR - Tools for Exploratory Evaluation in Hydrologic Research
+TEEHR (pronounced "tier") is a python tool set for loading, storing,
+processing and visualizing hydrologic data, particularly National Water
+Model data, for the purpose of exploring and evaluating the datasets to
+assess their skill and performance.
+
+NOTE: THIS PROJECT IS UNDER DEVELOPMENT - EXPECT TO FIND BROKEN AND INCOMPLETE CODE.
+
+## Documentation
+[TEEHR Documentation](https://rtiinternational.github.io/teehr/)
+
+## How to Install TEEHR
+We do not currently push TEEHR to PyPI, so the easiest way to install it is directly from GitHub.
+If using `pip` to install TEEHR, we recommend installing TEEHR in a virtual environment.
+The code below should create a new virtual environment and install TEEHR in it.
+
+```bash
+# Create directory for your code and create a new virtual environment.
+mkdir teehr_examples
+cd teehr_examples
+python3 -m venv .venv
+source .venv/bin/activate
+
+# Install using pip. The version can be changed to install a different version.
+pip install 'teehr @ git+https://github.com/RTIInternational/teehr@v0.4.1'
+
+# Download the required JAR files for Spark to interact with AWS S3.
+python -m teehr.utils.install_spark_jars
+```
+Use Docker
+```bash
+$ docker build -t teehr:v0.4.1 .
+$ docker run -it --rm --volume $HOME:$HOME -p 8888:8888 teehr:v0.4.1 jupyter lab --ip 0.0.0.0 $HOME
+```
+
+## Examples
+For examples of how to use TEEHR, see the [examples](examples).  We will maintain a basic set of example Jupyter Notebooks demonstrating how to use the TEEHR tools.
+
+
+## Resources
+In May of 2023 we put on a workshop at the CIROH 1st Annual Training and Developers Conference.  The workshop materials and presentation are available in the workshop GitHub repository: [teehr-may-2023-workshop](https://github.com/RTIInternational/teehr-may-2023-workshop).  This workshop was based on version 0.1.0.
+
+## Versioning
+The TEEHR project follows semantic versioning as described here: [https://semver.org/](https://semver.org/).
+Note, per the specification, "Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.".  We are solidly in "major version zero" territory, and trying to move fast, so expect breaking changes often.
+

teehr-0.4.1/README.md

@@ -0,0 +1,52 @@
+![alt text](https://github.com/RTIInternational/teehr/blob/main/docs/images/teehr.png)
+
+| | |
+| --- | --- |
+| ![alt text](https://ciroh.ua.edu/wp-content/uploads/2022/08/CIROHLogo_200x200.png) | Funding for this project was provided by the National Oceanic & Atmospheric Administration (NOAA), awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003). |
+
+
+# TEEHR - Tools for Exploratory Evaluation in Hydrologic Research
+TEEHR (pronounced "tier") is a python tool set for loading, storing,
+processing and visualizing hydrologic data, particularly National Water
+Model data, for the purpose of exploring and evaluating the datasets to
+assess their skill and performance.
+
+NOTE: THIS PROJECT IS UNDER DEVELOPMENT - EXPECT TO FIND BROKEN AND INCOMPLETE CODE.
+
+## Documentation
+[TEEHR Documentation](https://rtiinternational.github.io/teehr/)
+
+## How to Install TEEHR
+We do not currently push TEEHR to PyPI, so the easiest way to install it is directly from GitHub.
+If using `pip` to install TEEHR, we recommend installing TEEHR in a virtual environment.
+The code below should create a new virtual environment and install TEEHR in it.
+
+```bash
+# Create directory for your code and create a new virtual environment.
+mkdir teehr_examples
+cd teehr_examples
+python3 -m venv .venv
+source .venv/bin/activate
+
+# Install using pip. The version can be changed to install a different version.
+pip install 'teehr @ git+https://github.com/RTIInternational/teehr@v0.4.1'
+
+# Download the required JAR files for Spark to interact with AWS S3.
+python -m teehr.utils.install_spark_jars
+```
+Use Docker
+```bash
+$ docker build -t teehr:v0.4.1 .
+$ docker run -it --rm --volume $HOME:$HOME -p 8888:8888 teehr:v0.4.1 jupyter lab --ip 0.0.0.0 $HOME
+```
+
+## Examples
+For examples of how to use TEEHR, see the [examples](examples).  We will maintain a basic set of example Jupyter Notebooks demonstrating how to use the TEEHR tools.
+
+
+## Resources
+In May of 2023 we put on a workshop at the CIROH 1st Annual Training and Developers Conference.  The workshop materials and presentation are available in the workshop GitHub repository: [teehr-may-2023-workshop](https://github.com/RTIInternational/teehr-may-2023-workshop).  This workshop was based on version 0.1.0.
+
+## Versioning
+The TEEHR project follows semantic versioning as described here: [https://semver.org/](https://semver.org/).
+Note, per the specification, "Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.".  We are solidly in "major version zero" territory, and trying to move fast, so expect breaking changes often.

teehr-0.4.1/pyproject.toml

@@ -0,0 +1,76 @@
+[tool.poetry]
+name = "teehr"
+version = "0.4.1"
+description = "Tools for Exploratory Evaluation in Hydrologic Research"
+authors = [
+    "RTI International",
+    "Matthew Denno <mdenno@rti.org>",
+    "Katie van Werkhoven <kvanwerkhoven@rti.org>",
+    "Sam Lamont <slamont@rti.org>",
+]
+license = "GNU v3"
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = "^3.10"
+geopandas = "^0.14.0"
+duckdb = "^1.0.0"
+pydantic = "^2.4.2"
+hydrotools-metrics = "^1.3.3"
+hydrotools-nwis-client = "^3.3.1"
+dask = {extras = ["dataframe"], version = "^2024.11.1"}
+fsspec = ">=2023.10.0"
+ujson = "^5.8.0"
+kerchunk = "^0.2.2"
+gcsfs = ">=2023.10.0"
+s3fs = ">=2023.10.0"
+zarr = "^2.16.1"
+rioxarray = "^0.15.0"
+rasterio = "^1.3.9"
+h5py = "3.12.1"
+pyarrow = "^15.0.0"
+httpx = "^0.25.1"
+pandas = "^2.2.0"
+pyspark = {extras = ["pandas-on-spark"], version = "^3.5.2"}
+dataretrieval = "^1.0.9"
+numba = "^0.60.0"
+arch = "^7.0.0"
+pandera = {extras = ["pyspark"], version = "^0.20.4"}
+netcdf4 = "1.6.5"
+bokeh = "^3.5.0"
+
+[tool.poetry.group.test.dependencies]
+pytest = "^7.4.3"
+flake8 = "^6.1.0"
+
+[tool.poetry.group.dev.dependencies]
+sphinx = "^7.2.6"
+sphinx-design = "^0.5.0"
+numpydoc = "^1.6.0"
+pre-commit = "^3.6.0"
+pre-commit-hooks = "^4.5.0"
+flake8-docstrings = "^1.7.0"
+myst-nb = "^1.0.0"
+nbstripout = "^0.7.1"
+pickleshare = "^0.7.5"
+sphinx-autoapi = "3.0.0"
+pydata-sphinx-theme = "^0.16.0"
+sphinx-autobuild = "^2024.10.3"
+
+
+[tool.numpydoc_validation]
+checks = [
+    "all",   # report on all checks, except the below
+    "EX01",  # No examples section found
+    "RT01",  # No Returns section found
+    "SA01",  # See Also section not found
+    "ES01",  # No extended summary found
+    "GL01",  # Docstring text (summary) should start in the line immediately
+    "SS06",  # Summary should fit in a single line
+    "PR01",  # Parameters {missing_params} not documented
+    "GL02"   # Closing quotes should be placed in the line after the last text in docstring"
+]
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

teehr-0.4.1/src/teehr/__init__.py

@@ -0,0 +1,40 @@
+from teehr.evaluation.evaluation import Evaluation  # noqa
+from teehr.models.metrics.metric_models import Metrics  # noqa
+from teehr.models.metrics.metric_enums import Operators  # noqa
+from teehr.models.tables import (  # noqa
+    Configuration,
+    Attribute,
+    Unit,
+    Variable
+)
+from teehr.models.metrics.bootstrap_models import Bootstrappers  # noqa
+from teehr.models.filters import (  # noqa
+    UnitFilter,
+    ConfigurationFilter,
+    VariableFilter,
+    AttributeFilter,
+    LocationFilter,
+    LocationAttributeFilter,
+    LocationCrosswalkFilter,
+    TimeseriesFilter,
+    JoinedTimeseriesFilter
+)
+
+# For docs
+from teehr.evaluation.tables import (  # noqa
+    BaseTable,
+    UnitTable,
+    VariableTable,
+    AttributeTable,
+    ConfigurationTable,
+    LocationTable,
+    LocationAttributeTable,
+    LocationCrosswalkTable,
+    PrimaryTimeseriesTable,
+    SecondaryTimeseriesTable,
+    JoinedTimeseriesTable,
+)
+from teehr.evaluation.fetch import Fetch  # noqa
+from teehr.visualization.dataframe_accessor import TEEHRDataFrameAccessor  # noqa
+from teehr.models.metrics import metric_models, bootstrap_models  # noqa
+from teehr.metrics import metric_funcs # noqa

teehr-0.4.1/src/teehr/const.py

@@ -0,0 +1,36 @@
+"""This module contains constants
+ used throughout the package."""
+
+
+# Primary evaluation directories
+DATASET_DIR = "dataset"
+CACHE_DIR = "cache"
+SCRIPTS_DIR = "scripts"
+LOCATIONS_DIR = "locations"
+PRIMARY_TIMESERIES_DIR = "primary_timeseries"
+LOCATION_CROSSWALKS_DIR = "location_crosswalks"
+LOCATION_ATTRIBUTES_DIR = "location_attributes"
+SECONDARY_TIMESERIES_DIR = "secondary_timeseries"
+JOINED_TIMESERIES_DIR = "joined_timeseries"
+
+# Domain files and directories
+VARIABLES_DIR = "variables"
+CONFIGURATIONS_DIR = "configurations"
+UNITS_DIR = "units"
+ATTRIBUTES_DIR = "attributes"
+VARIABLES_FILE = "variables.csv"
+CONFIGURATIONS_FILE = "configurations.csv"
+UNITS_FILE = "units.csv"
+ATTRIBUTES_FILE = "attributes.csv"
+
+# Fetching cache directories
+FETCHING_CACHE_DIR = "fetching"
+USGS_CACHE_DIR = "usgs"
+NWM_CACHE_DIR = "nwm"
+KERCHUNK_DIR = "kerchunk"
+WEIGHTS_DIR = "weights"
+
+# Loading cache directories
+LOADING_CACHE_DIR = "loading"
+
+S3_EVALUATIONS_PATH = "s3://ciroh-rti-public-data/teehr-data-warehouse/v0_4_evaluations/evaluations.yaml"

teehr-0.4.1/src/teehr/evaluation/__init__.py

@@ -0,0 +1 @@
+"""Evaluation module for TEEHR."""

teehr-0.4.1/src/teehr/evaluation/evaluation.py

@@ -0,0 +1,324 @@
+"""Evaluation module."""
+from datetime import datetime
+from typing import Union, Literal, List
+from pathlib import Path
+from pyspark.sql import SparkSession
+from pyspark import SparkConf
+import logging
+from teehr.loading.utils import (
+    copy_template_to,
+)
+from teehr.loading.s3.clone_from_s3 import (
+    list_s3_evaluations,
+    clone_from_s3
+)
+import teehr.const as const
+from teehr.evaluation.fetch import Fetch
+from teehr.evaluation.metrics import Metrics
+from teehr.evaluation.tables import (
+    UnitTable,
+    VariableTable,
+    AttributeTable,
+    ConfigurationTable,
+    LocationTable,
+    LocationAttributeTable,
+    LocationCrosswalkTable,
+    PrimaryTimeseriesTable,
+    SecondaryTimeseriesTable,
+    JoinedTimeseriesTable,
+)
+import pandas as pd
+from teehr.visualization.dataframe_accessor import TEEHRDataFrameAccessor # noqa
+
+
+logger = logging.getLogger(__name__)
+
+
+class Evaluation:
+    """The Evaluation class.
+
+    This is the main class for the TEEHR evaluation.
+    """
+
+    def __init__(
+        self,
+        dir_path: Union[str, Path],
+        create_dir: bool = False,
+        spark: SparkSession = None
+    ):
+        """
+        Initialize the Evaluation class.
+
+        Parameters
+        ----------
+        dir_path : Union[str, Path]
+            The path to the evaluation directory.
+        spark : SparkSession, optional
+            The SparkSession object, by default None
+        """
+        self.dir_path = dir_path
+        self.spark = spark
+
+        self.dataset_dir = Path(
+            self.dir_path, const.DATASET_DIR
+        )
+        self.cache_dir = Path(
+            self.dir_path, const.CACHE_DIR
+        )
+        self.scripts_dir = Path(
+            self.dir_path, const.SCRIPTS_DIR
+        )
+        self.units_dir = Path(
+            self.dataset_dir, const.UNITS_DIR
+        )
+        self.variables_dir = Path(
+            self.dataset_dir, const.VARIABLES_DIR
+        )
+        self.configurations_dir = Path(
+            self.dataset_dir, const.CONFIGURATIONS_DIR
+        )
+        self.attributes_dir = Path(
+            self.dataset_dir, const.ATTRIBUTES_DIR
+        )
+        self.locations_dir = Path(
+            self.dataset_dir, const.LOCATIONS_DIR
+        )
+        self.location_crosswalks_dir = Path(
+            self.dataset_dir, const.LOCATION_CROSSWALKS_DIR
+        )
+        self.location_attributes_dir = Path(
+            self.dataset_dir, const.LOCATION_ATTRIBUTES_DIR
+        )
+        self.primary_timeseries_dir = Path(
+            self.dataset_dir, const.PRIMARY_TIMESERIES_DIR
+        )
+        self.secondary_timeseries_dir = Path(
+            self.dataset_dir, const.SECONDARY_TIMESERIES_DIR
+        )
+        self.joined_timeseries_dir = Path(
+            self.dataset_dir, const.JOINED_TIMESERIES_DIR
+        )
+
+        if not Path(self.dir_path).is_dir():
+            if create_dir:
+                logger.info(f"Creating directory {self.dir_path}.")
+                Path(self.dir_path).mkdir(parents=True, exist_ok=True)
+            else:
+                logger.error(f"Directory {self.dir_path} does not exist.")
+                raise NotADirectoryError
+
+        # Create a local Spark Session if one is not provided.
+        if not self.spark:
+            logger.info("Creating a new Spark session.")
+            conf = (
+                SparkConf()
+                .setAppName("TEEHR")
+                .setMaster("local[*]")
+                .set("spark.sql.sources.partitionOverwriteMode", "dynamic")
+                .set("spark.hadoop.fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem")
+                .set("spark.hadoop.fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider")
+                .set("spark.sql.execution.arrow.pyspark.enabled", "true")
+            )
+            self.spark = SparkSession.builder.config(conf=conf).getOrCreate()
+
+    @property
+    def fetch(self) -> Fetch:
+        """The fetch component class for accessing external data."""
+        return Fetch(self)
+
+    @property
+    def metrics(self) -> Metrics:
+        """The metrics component class for calculating performance metrics."""
+        return Metrics(self)
+
+    @property
+    def units(self) -> UnitTable:
+        """Access the units table."""
+        return UnitTable(self)
+
+    @property
+    def variables(self) -> VariableTable:
+        """Access the variables table."""
+        return VariableTable(self)
+
+    @property
+    def attributes(self) -> AttributeTable:
+        """Access the attributes table."""
+        return AttributeTable(self)
+
+    @property
+    def configurations(self) -> ConfigurationTable:
+        """Access the configurations table."""
+        return ConfigurationTable(self)
+
+    @property
+    def locations(self) -> LocationTable:
+        """Access the locations table."""
+        return LocationTable(self)
+
+    @property
+    def location_attributes(self) -> LocationAttributeTable:
+        """Access the location attributes table."""
+        return LocationAttributeTable(self)
+
+    @property
+    def location_crosswalks(self) -> LocationCrosswalkTable:
+        """Access the location crosswalks table."""
+        return LocationCrosswalkTable(self)
+
+    @property
+    def primary_timeseries(self) -> PrimaryTimeseriesTable:
+        """Access the primary timeseries table."""
+        return PrimaryTimeseriesTable(self)
+
+    @property
+    def secondary_timeseries(self) -> SecondaryTimeseriesTable:
+        """Access the secondary timeseries table."""
+        return SecondaryTimeseriesTable(self)
+
+    @property
+    def joined_timeseries(self) -> JoinedTimeseriesTable:
+        """Access the joined timeseries table."""
+        return JoinedTimeseriesTable(self)
+
+    def enable_logging(self):
+        """Enable logging."""
+        logger = logging.getLogger("teehr")
+        # logger.addHandler(logging.StreamHandler())
+        handler = logging.FileHandler(Path(self.dir_path, 'teehr.log'))
+        handler.setFormatter(
+            logging.Formatter(
+                "%(asctime)s %(levelname)s %(message)s"
+            )
+        )
+        logger.addHandler(
+            handler
+        )
+        logger.setLevel(logging.DEBUG)
+
+    def clone_template(self):
+        """Create a study from the standard template.
+
+        This method mainly copies the template directory to the specified
+        evaluation directory.
+        """
+        teehr_root = Path(__file__).parent.parent
+        template_dir = Path(teehr_root, "template")
+        logger.info(f"Copying template from {template_dir} to {self.dir_path}")
+        copy_template_to(template_dir, self.dir_path)
+
+    @staticmethod
+    def list_s3_evaluations(
+        format: Literal["pandas", "list"] = "pandas"
+    ) -> Union[list, pd.DataFrame]:
+        """List the evaluations available on S3.
+
+        Parameters
+        ----------
+        format : str, optional
+            The format of the output. Either "pandas" or "list".
+            The default is "pandas".
+        """
+        return list_s3_evaluations(format=format)
+
+    def clone_from_s3(
+        self,
+        evaluation_name: str,
+        primary_location_ids: List[str] = None,
+        start_date: Union[str, datetime] = None,
+        end_date: Union[str, datetime] = None,
+    ):
+        """Fetch the study data from S3.
+
+        Copies the study from s3 to the local directory, with the option
+        to subset the dataset by primary location ID, start and end dates.
+
+        Parameters
+        ----------
+        evaluation_name : str
+            The name of the evaluation to clone from S3.
+            Use the list_s3_evaluations method to get the available
+            evaluations.
+        primary_location_ids : List[str], optional
+            The list of primary location ids to subset the data.
+            The default is None.
+        start_date : Union[str, datetime], optional
+            The start date to subset the data.
+            The default is None.
+        end_date : Union[str, datetime], optional
+            The end date to subset the data.
+            The default is None.
+
+        Notes
+        -----
+
+        Includes the following tables:
+            - units
+            - variables
+            - attributes
+            - configurations
+            - locations
+            - location_attributes
+            - location_crosswalks
+            - primary_timeseries
+            - secondary_timeseries
+            - joined_timeseries
+
+        Also includes the user_defined_fields.py script.
+
+        """
+        return clone_from_s3(
+            self,
+            evaluation_name,
+            primary_location_ids,
+            start_date,
+            end_date
+        )
+
+    def clean_cache(self):
+        """Clean temporary files.
+
+        Includes removing temporary files.
+        """
+        logger.info(f"Removing temporary files from {self.cache_dir}")
+        self.cache_dir.rmdir()
+        self.cache_dir.mkdir()
+
+    def sql(self, query: str):
+        """Run a SQL query on the Spark session against the TEEHR tables.
+
+        Parameters
+        ----------
+        query : str
+            The SQL query to run.
+
+        Returns
+        -------
+        pyspark.sql.DataFrame
+            The result of the SQL query.
+            This is lazily evaluated so you need to call an action (e.g., sdf.show()) to get the result.
+
+        This method has access to the following tables preloaded as temporary views:
+            - units
+            - variables
+            - attributes
+            - configurations
+            - locations
+            - location_attributes
+            - location_crosswalks
+            - primary_timeseries
+            - secondary_timeseries
+            - joined_timeseries
+        """
+        self.units.to_sdf().createOrReplaceTempView("units")
+        self.variables.to_sdf().createOrReplaceTempView("variables")
+        self.attributes.to_sdf().createOrReplaceTempView("attributes")
+        self.configurations.to_sdf().createOrReplaceTempView("configurations")
+        self.locations.to_sdf().createOrReplaceTempView("locations")
+        self.location_attributes.to_sdf().createOrReplaceTempView("location_attributes")
+        self.location_crosswalks.to_sdf().createOrReplaceTempView("location_crosswalks")
+        self.primary_timeseries.to_sdf().createOrReplaceTempView("primary_timeseries")
+        self.secondary_timeseries.to_sdf().createOrReplaceTempView("secondary_timeseries")
+        self.joined_timeseries.to_sdf().createOrReplaceTempView("joined_timeseries")
+
+        return self.spark.sql(query)