primo-optimizer 0.1__tar.gz

primo_optimizer-0.1/LICENSE.md

@@ -0,0 +1,39 @@
+License
+=======
+
+PRIMO - The P&A Project Optimizer Toolkit Copyright (c) 2023---2026, by the software owners:  KeyLogic Systems, LLC.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the U.S. Dept. of Energy, National Energy Technology Laboratory, KeyLogic Systems, LLC, nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+You are under no obligation whatsoever to provide any bug fixes, patches, or
+upgrades to the features, functionality or performance of the source code
+("Enhancements") to anyone; however, if you choose to make your Enhancements
+available either publicly, or directly to National Energy Technology Laboratory,
+or directly to  KeyLogic Systems, LLC, without imposing a separate written license
+agreement for such Enhancements, then you hereby grant National Energy Technology Laboratory
+and  KeyLogic Systems, LLC the following license: a  non-exclusive, royalty-free perpetual
+license to install, use, modify, prepare derivative works, incorporate into other computer software,
+distribute, and sublicense such enhancements or derivative works thereof, in
+binary and source code form.

primo_optimizer-0.1/PKG-INFO

@@ -0,0 +1,68 @@
+Metadata-Version: 2.1
+Name: primo-optimizer
+Version: 0.1
+Summary: PRIMO - The P&A Project Optimizer
+Home-page: https://edx.netl.doe.gov/nemri/
+Author: PRIMO team
+Maintainer: Miguel Zamarripa
+Maintainer-email: Miguel.Zamarripa-Perez@netl.doe.gov
+License: BSD
+Keywords: PRIMO,MERP,NEMRI,methane emissions,optimization,process modeling,operations research,well plugging
+Platform: windows
+Platform: linux
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Natural Language :: English
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: Unix
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Programming Language :: Python :: 3 :: Only
+Requires-Python: >=3.8, <=4.0
+Description-Content-Type: text/plain
+License-File: LICENSE.md
+Requires-Dist: appengine-python-standard
+Requires-Dist: censusgeocode
+Requires-Dist: folium
+Requires-Dist: geopandas
+Requires-Dist: gurobipy
+Requires-Dist: highspy
+Requires-Dist: haversine
+Requires-Dist: ipyfilechooser
+Requires-Dist: ipyleaflet
+Requires-Dist: ipywidgets
+Requires-Dist: kneed
+Requires-Dist: matplotlib
+Requires-Dist: notebook
+Requires-Dist: numpy<=1.28
+Requires-Dist: openpyxl
+Requires-Dist: pandas
+Requires-Dist: pyarrow
+Requires-Dist: pyomo
+Requires-Dist: pyscipopt
+Requires-Dist: python-dotenv
+Requires-Dist: scikit-learn
+Requires-Dist: rasterio
+Requires-Dist: xlsxwriter
+Provides-Extra: dev
+Requires-Dist: addheader; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: myst-parser; extra == "dev"
+Requires-Dist: nbsphinx; extra == "dev"
+Requires-Dist: pre-commit; extra == "dev"
+Requires-Dist: sphinx; extra == "dev"
+Requires-Dist: sphinx-rtd-theme; extra == "dev"
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+
+PRIMO - The P&A Project Optimizer toolkit was produced under the  Methane Emissions Reduction Program (MERP)  and National Energy Technology Laboratory's (NETL) National Emissions Reduction Initiative (NEMRI). The major deliverable of this project will be an open-source, optimization-based, downloadable and executable decision-support application, PRIMO, that can be run by companies, technology providers, research organizations, and regulators.

primo_optimizer-0.1/README.md

@@ -0,0 +1,58 @@
+# PRIMO - The P&A Project Optimizer Toolkit
+
+PRIMO - The P&A Project Optimizer Toolkit aims to provide multi-scale, simulation-based, open source
+computational tools and models to support the Methane Emissions Reduction Program (MERP) and the National
+Emissions Reduction Initiative (NEMRI).
+
+## Getting Started
+
+Our complete documentation is currently being built, but here is a summarized set of steps to get started using the framework.
+
+While not required, we encourage the installation of [Anaconda](https://www.anaconda.com/products/individual#Downloads) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) and using the `conda` command to create a separate python environment in which to install the PRIMO Toolkit.
+
+Use conda to create a new "primo-env" (could be any name you like) environment then activate that environment:
+```bash
+conda create --name primo-env python
+conda activate primo-env
+```
+
+Install all dependencies with:
+```
+pip install -r requirements.txt
+```
+For developers, install dependencies with:
+```
+pip install -r requirements-dev.txt
+```
+
+For users interested in using the free solver SCIP to solve PRIMO's optimization problems, please run:
+```
+conda install -c conda-forge -y scip=9.1.0
+```
+
+To test the installation of the primo package, execute:
+```
+pytest primo\utils\tests\test_imports.py
+```
+The above test, if executed successfully, confirms that primo package is now installed and available in the "primo-env" package that was just created. 
+
+To use the utilities implemented in the PRIMO package that query the U.S. Census API and Bing Maps API, appropriate API keys must be obtained
+by signing up with the respective services. These keys must be configured in a .env file in the parent directory. For more details, please see:
+[API Keys](TBD)
+
+Users can also employ other commercial solvers, for example Gurobi, to solve the optimization problem. 
+However, users are responsible for configuring and setting up these solvers themselves
+
+General, background and overview information is available at the [NEMRI website](https://edx.netl.doe.gov/nemri/).
+
+## Funding acknowledgements
+
+This work was conducted as part of the [National Emissions Reduction Initiative](https://edx.netl.doe.gov/nemri/)
+with support through the [Environmental Protection Agency - Methane Emissions Reduction Program](https://www.epa.gov/inflation-reduction-act/methane-emissions-reduction-program)
+within the U.S. Department of Energy’s [Office of Fossil Energy and Carbon Management (FECM)](https://www.energy.gov/fecm/office-fossil-energy-and-carbon-management).
+As of 2023, additional support was provided by FECM’s [Solid Oxide Fuel Cell Program](https://www.energy.gov/fecm/science-innovation/clean-coal-research/solid-oxide-fuel-cells),
+and [Transformative Power Generation Program](https://www.energy.gov/fecm/science-innovation/office-clean-coal-and-carbon-management/advanced-energy-systems/transformative).
+
+## Contributing
+
+**By contributing to this repository, you are agreeing to all the terms set out in the LICENSE.md and COPYRIGHT.md files in this directory.**

primo_optimizer-0.1/primo/__init__.py

@@ -0,0 +1,13 @@
+#################################################################################
+# PRIMO - The P&A Project Optimizer was produced under the Methane Emissions Reduction Program (MERP)
+# and National Energy Technology Laboratory's (NETL) National Emissions Reduction Initiative (NEMRI).
+#
+# NOTICE. This Software was developed under funding from the U.S. Government and the U.S.
+# Government consequently retains certain rights. As such, the U.S. Government has been granted for
+# itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in
+# the Software to reproduce, distribute copies to the public, prepare derivative works, and perform
+# publicly and display publicly, and to permit others to do so.
+#################################################################################
+
+RELEASE = "0.1.dev0"
+VERSION = "0.1"

primo_optimizer-0.1/primo/data_parser/__init__.py

@@ -0,0 +1,10 @@
+#################################################################################
+# PRIMO - The P&A Project Optimizer was produced under the Methane Emissions Reduction Program (MERP)
+# and National Energy Technology Laboratory's (NETL) National Emissions Reduction Initiative (NEMRI).
+#
+# NOTICE. This Software was developed under funding from the U.S. Government and the U.S.
+# Government consequently retains certain rights. As such, the U.S. Government has been granted for
+# itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in
+# the Software to reproduce, distribute copies to the public, prepare derivative works, and perform
+# publicly and display publicly, and to permit others to do so.
+#################################################################################

primo_optimizer-0.1/primo/data_parser/data_model.py

@@ -0,0 +1,197 @@
+#################################################################################
+# PRIMO - The P&A Project Optimizer was produced under the Methane Emissions Reduction Program (MERP)
+# and National Energy Technology Laboratory's (NETL) National Emissions Reduction Initiative (NEMRI).
+#
+# NOTICE. This Software was developed under funding from the U.S. Government and the U.S.
+# Government consequently retains certain rights. As such, the U.S. Government has been granted for
+# itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in
+# the Software to reproduce, distribute copies to the public, prepare derivative works, and perform
+# publicly and display publicly, and to permit others to do so.
+#################################################################################
+
+# Standard lib
+from __future__ import annotations
+from dataclasses import dataclass
+import logging
+from typing import Dict, Union
+
+
+# User defined lib
+from primo.utils.geo_utils import get_distance
+from primo.utils.raise_exception import raise_exception
+
+LOGGER = logging.getLogger(__name__)
+
+# TODO: Write tests for entire module
+
+
+@dataclass(frozen=True)
+class Well:
+    """
+    A Well dataclass
+    """
+
+    __slots__ = (
+        "well_id",
+        "lat",
+        "long",
+        "score",
+        "is_disadvantaged",
+    )
+    well_id: str
+    lat: Union[float, None]
+    long: Union[float, None]
+    score: float
+    is_disadvantaged: bool
+
+    def haversine_distance(self, other_well: Well) -> float:
+        """
+        Returns the distance in miles between two wells calculated with Haversine formula.
+
+        Parameters
+        ----------
+        other_well : Well
+            The other well under consideration
+
+        Returns
+        -------
+        distance : float
+            The distance between self and other_well
+        """
+        origin = (self.lat, self.long)
+        dest = (other_well.lat, other_well.long)
+        return get_distance(origin, dest)
+
+
+@dataclass(frozen=True)
+class CampaignCandidates:
+    """
+    A dataclass that combines wells that are potential candidates to be included
+    in a plugging campaign
+    """
+
+    __slots__ = ("campaign_id", "well_dict", "n_wells")
+    campaign_id: str
+    well_dict: Dict[str, Well]
+    n_wells: int
+
+    def __post_init__(self):
+        """
+        Run some validation checks
+        """
+        for well_id, well in self.well_dict.items():
+            if well_id != well.well_id:
+                msg = (
+                    f"Dictionary for {self.campaign_id} is corrupted. Expect key "
+                    "to be the same as well_id"
+                )
+                raise_exception(msg, ValueError)
+
+        if self.n_wells != len(self.well_dict):
+            LOGGER.warning(
+                f"{self.n_wells} incorrectly initialized for {self.campaign_id}"
+            )
+            LOGGER.warning("Automatically correcting!")
+            self.n_wells = len(self.well_dict)
+
+
+@dataclass(frozen=True)
+class OptInputs:
+    """
+    Succinctly captures all inputs relevant to the optimization model
+    """
+
+    __slots__ = (
+        "well_dict",
+        "campaign_candidates",
+        "budget",
+        "mobilization_cost",
+        "owner_well_count",
+        "max_wells_per_owner",
+        "dac_budget_fraction",
+        "distance_threshold",
+    )
+    well_dict: Dict[str, Well]
+    campaign_candidates: Dict[str, CampaignCandidates]
+    budget: float
+    mobilization_cost: Dict[int, float]
+    owner_well_count: Dict[str, Well]
+    max_wells_per_owner: float
+    dac_budget_fraction: float
+    distance_threshold: float
+
+    def __post_init__(self):
+        """
+        Run some validation checks
+        """
+        # Create a set to retrieve unique well_id (identifiers that do not repeat).
+        well_id_set = set(self.well_dict.keys())
+
+        if self.budget <= 0:
+            msg = f"Budget cannot be negative. Received " f"{self.budget} for budget."
+            raise_exception(msg, ValueError)
+
+        for well_id, well in self.well_dict.items():
+            if well_id != well.well_id:
+                msg = (
+                    f"Dictionary for optimization inputs corrupted. "
+                    f"Expect key: {well_id} to be the same as"
+                    f" well_id: {well.well_id}"
+                )
+                raise_exception(msg, ValueError)
+
+        max_wells = 0
+        # Iterate through each campaign using its campaign_id.
+        for campaign_id, campaign in self.campaign_candidates.items():
+            # Lines 167-173 aim to verify the accuracy of the
+            # campaign_candidates construction.
+            if campaign_id != campaign.campaign_id:
+                msg = (
+                    f"Dictionary of candidates corrupted. "
+                    f"Expect key: {campaign_id} to be the same as "
+                    f"campaign_id: {campaign.campaign_id}"
+                )
+                raise_exception(msg, ValueError)
+
+            for well_id in campaign.well_dict:
+                # Lines 178-183 aim to verify whether the wells in CampaignCandidates
+                # are all sourced from the input well data.
+                if well_id not in self.well_dict:
+                    msg = (
+                        f"Campaign candidate: {campaign_id} includes well: {well_id}"
+                        f". This well is not provided in well_dict"
+                    )
+                    raise_exception(msg, ValueError)
+
+                # Lines 187-190 aim to check if a well appears more than once within
+                # the same campaign or across different campaigns.
+                if well_id not in well_id_set:
+                    msg = f"Well: {well_id} is included in multiple clusters (campaign candidates)."
+                    raise_exception(msg, ValueError)
+                well_id_set.remove(well_id)
+                # Remove a well_id from the well_id_set the first time it is checked. If the well_id
+                # appears a second time and has already been removed from the unique well_id_set,
+                # an error message on line 188 will be raised.
+                # Each cluster represents a campaign candidate.
+
+            max_wells = max(max_wells, len(campaign.well_dict))
+
+        # Upon iterating through all campaigns and well_ids within those campaigns, all
+        # well_ids in well_id_set should be dropped. If not, it indicates that not all wells
+        # are assigned to a cluster during the clustering step. An error message will be raised at
+        #  Line 202.
+        if well_id_set:
+            msg = "The clustering is not exhaustive."
+            raise_exception(msg, ValueError)
+        for n_wells in range(1, max_wells + 1):
+            cost = self.mobilization_cost.get(n_wells)
+            if cost is None:
+                msg = f"Mobilization cost for n_wells: {n_wells} is missing"
+                raise_exception(msg, ValueError)
+
+            if cost < 0:
+                msg = (
+                    f"Mobilization cost for campaign with n_wells: {n_wells}"
+                    f" is negative: {cost}"
+                )
+                raise_exception(msg, ValueError)

primo_optimizer-0.1/primo/data_parser/input_parser.py

@@ -0,0 +1,161 @@
+#################################################################################
+# PRIMO - The P&A Project Optimizer was produced under the Methane Emissions Reduction Program (MERP)
+# and National Energy Technology Laboratory's (NETL) National Emissions Reduction Initiative (NEMRI).
+#
+# NOTICE. This Software was developed under funding from the U.S. Government and the U.S.
+# Government consequently retains certain rights. As such, the U.S. Government has been granted for
+# itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in
+# the Software to reproduce, distribute copies to the public, prepare derivative works, and perform
+# publicly and display publicly, and to permit others to do so.
+#################################################################################
+
+# Standard libs
+import argparse
+import logging
+import os
+from typing import Dict
+
+# Installed libs
+import pandas as pd
+
+# User-defined libs
+from primo.data_parser.data_model import CampaignCandidates, OptInputs, Well
+from primo.utils.raise_exception import raise_exception
+
+LOGGER = logging.getLogger(__name__)
+
+
+class InputParser:
+    """
+    InputParser class
+    Implements methods for parsing input data from csv
+    """
+
+    def __init__(self, args: argparse.Namespace):
+        """
+        Initialize the class and read in a CSV file.
+
+        Parameters
+        ----------
+        args : argparse.Namespace
+            The arguments parsed from the command line.
+
+        Raises
+        ------
+        ValueError
+            If the specified file does not exist or is not a CSV file.
+        """
+        file_path = args.input_file
+        # Check 1: Check file exists
+        if not os.path.exists(file_path):
+            raise_exception(f"File: {file_path} does not exist.", ValueError)
+
+        # Check 2: Check file is a csv file
+        if file_path.suffix != ".csv":
+            raise_exception(
+                f"File: {file_path} is expected to be an csv file", ValueError
+            )
+
+        self.well_data = pd.read_csv(file_path)
+        self.args = args
+        LOGGER.info(f"Initialized class and read file: {file_path}")
+
+    def parse_data(
+        self,
+        mobilization_cost: Dict[int, float],
+    ) -> OptInputs:
+        """
+        Parse data to create an OptInputs object that models all input parameters
+        to set up the optimization model.
+
+        Parameters
+        ----------
+        mobilization_cost : Dict[int, float]
+            The cost associated with mobilization for a campaign based on the number
+            of wells in a campaign.
+
+        Returns
+        -------
+        opt_inputs : OptInputs
+            A fully instantiated object for inputs to the optimization model.
+
+        Raises
+        ------
+        ValueError
+            If a well ID is repeated in the CSV file.
+        """
+        budget = self.args.total_budget
+        max_wells_per_owner = self.args.max_wells_per_owner
+        well_dict = {}
+        candidates = {}
+        # dac_budget_fraction is the percentage of budget that needs
+        # to be spent on plugging wells in the disadvantaged community
+        dac_budget_fraction = self.args.dac_weight_percent
+        distance_threshold = self.args.maximum_well_distance
+
+        # new data
+        owner_well_count = (
+            self.well_data.groupby("Operator Name")["Well ID"].apply(list).to_dict()
+        )
+
+        dac_weight = self.args.dac_weight
+
+        if self.args.dac_weight_percent is not None:
+            if dac_weight is not None:
+                threshold = dac_weight / 100 * self.args.dac_weight_percent
+            else:
+                raise_exception(
+                    "Disadvantaged Community Impact must be selected as a "
+                    "priority factor if you wish to implement the DAC constraint.",
+                    ValueError,
+                )
+        else:
+            # When the user does not provide a value for disadvantaged community,
+            # all wells are set as not disadvantaged.
+            # The maximum score possible for disadvantaged_community_score is 100.
+            threshold = 100
+        for _, row in self.well_data.iterrows():
+            well_id = row["Well ID"]
+            lat = row["Latitude"]
+            long = row["Longitude"]
+            score = row["Priority Score [0-100]"]
+            if dac_weight is not None:
+                disadvantaged_community_score = row[f"DAC Score [0-{int(dac_weight)}]"]
+                is_disadvantaged = float(disadvantaged_community_score > threshold)
+            else:
+                # When the user does not select DAC as a priority factor,
+                # all wells are assumed to not be located in a disadvantaged community.
+                is_disadvantaged = float(False)
+
+            if well_id in well_dict:
+                msg = f"Well id must be unique. Well id: {well_id} is repeated in csv file"
+                raise_exception(msg, ValueError)
+
+            well_dict[well_id] = Well(well_id, lat, long, score, is_disadvantaged)
+
+        LOGGER.info("Parsing for campaign candidates")
+
+        for _, row in self.well_data.iterrows():
+            well_id = row["Well ID"]
+            cluster = row["cluster"]
+            if cluster not in candidates:
+                candidates.setdefault(cluster, {})
+            candidates[cluster][well_id] = well_dict[well_id]
+
+        campaign_candidates = {}
+        for campaign_id, campaign_dict in candidates.items():
+            LOGGER.info(f"Preparing cluster: {campaign_id}")
+            campaign_candidates[campaign_id] = CampaignCandidates(
+                campaign_id, campaign_dict, len(campaign_dict)
+            )
+
+        return OptInputs(
+            well_dict,
+            campaign_candidates,
+            budget,
+            mobilization_cost,
+            owner_well_count,
+            max_wells_per_owner,
+            dac_budget_fraction,
+            distance_threshold,
+        )

primo_optimizer-0.1/primo/opt_model/__init__.py

@@ -0,0 +1,13 @@
+#################################################################################
+# PRIMO - The P&A Project Optimizer was produced under the Methane Emissions Reduction Program (MERP)
+# and National Energy Technology Laboratory's (NETL) National Emissions Reduction Initiative (NEMRI).
+#
+# NOTICE. This Software was developed under funding from the U.S. Government and the U.S.
+# Government consequently retains certain rights. As such, the U.S. Government has been granted for
+# itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in
+# the Software to reproduce, distribute copies to the public, prepare derivative works, and perform
+# publicly and display publicly, and to permit others to do so.
+#################################################################################
+
+# Optimization Feasibility tolerance
+FEASIBILITY_TOLERANCE = 1e-6