mdpsolver 0.9.9__tar.gz → 0.10.0__tar.gz

mdpsolver-0.9.9/README.md → mdpsolver-0.10.0/PKG-INFO

@@ -1,12 +1,28 @@
-# MDPSolver
+Metadata-Version: 2.2
+Name: mdpsolver
+Version: 0.10.0
+Summary: A fast solver for Markov Decision Processes
+Author-Email: Anders Reenberg Andersen <andersra@live.dk>, Jesper Fink Andersen <jesperfinkandersen@gmail.com>
+Classifier: Development Status :: 4 - Beta
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Operating System :: MacOS
+Project-URL: Homepage, https://github.com/areenberg/MDPSolver
+Project-URL: Issues, https://github.com/areenberg/MDPSolver/issues
+Requires-Python: >=3.10
+Provides-Extra: test
+Requires-Dist: numpy; extra == "test"
+Description-Content-Type: text/markdown
 
-<img src="https://github.com/areenberg/MDPSolver/blob/anders_development/Images/MDPSolver_logo_3.png" alt="Logo" width="180">
+# MDPSolver
 
 MDPSolver is a Python package for large Markov Decision Processes (MDPs) with infinite-horizons. 
 
 ## Features
 
-* Fast solver: Our C++-based solver is substantially faster than other MDP packages available for Python. See details in the documentation.
+* Fast solver: Our C++-based solver is substantially faster than other CPU-based MDP packages available for Python. See details in the documentation.
 * Two optimality criteria: *Discounted* and *Average* reward.
 * Three optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.
 * Three value-update methods: *Standard*, *Gauss–Seidel*, and *Successive over-relaxation*.
@@ -15,25 +31,7 @@ MDPSolver is a Python package for large Markov Decision Processes (MDPs) with in
 
 # Installation
 
-## Linux
-
-Install directly from PyPI with:
-
-```
-pip install mdpsolver
-```
-
-MDPSolver works *out of the box* on Ubuntu 22 and newer.
-
-### GLIBC not found
-
-Some users will encounter the `version 'GLIBC_2.32' not found` error when attempting to import MDPSolver in Python. In this case, it might help to manually compile and replace the SO-file for the optimization module in the MDPSolver package. See the steps on how to solve the issue in the [documentation](https://github.com/areenberg/MDPSolver/wiki#how-to-install).
-
-## Windows
-
-Requires [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) (17.9) with MSVC C++ compiler and libraries installed.
-
-After installing Visual Studio (incl. MSVC C++ compiler and libraries), install directly from PyPI with:
+Prebuilt wheels are available for **Linux**, **Windows**, and **macOS (14+)** across Python 3.10–3.14. Install directly from PyPI with:
 
 ```
 pip install mdpsolver

mdpsolver-0.9.9/src/mdpsolver.egg-info/PKG-INFO → mdpsolver-0.10.0/README.md

@@ -1,29 +1,10 @@
-Metadata-Version: 2.4
-Name: mdpsolver
-Version: 0.9.9
-Summary: A fast solver for Markov Decision Processes
-Author-email: Anders Reenberg Andersen <andersra@live.dk>, Jesper Fink Andersen <jesperfinkandersen@gmail.com>
-Project-URL: Homepage, https://github.com/areenberg/MDPSolver
-Project-URL: Issues, https://github.com/areenberg/MDPSolver/issues
-Classifier: Development Status :: 3 - Alpha
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: Microsoft :: Windows
-Classifier: Operating System :: POSIX :: Linux
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Dynamic: license-file
-
 # MDPSolver
 
-<img src="https://github.com/areenberg/MDPSolver/blob/anders_development/Images/MDPSolver_logo_3.png" alt="Logo" width="180">
-
 MDPSolver is a Python package for large Markov Decision Processes (MDPs) with infinite-horizons. 
 
 ## Features
 
-* Fast solver: Our C++-based solver is substantially faster than other MDP packages available for Python. See details in the documentation.
+* Fast solver: Our C++-based solver is substantially faster than other CPU-based MDP packages available for Python. See details in the documentation.
 * Two optimality criteria: *Discounted* and *Average* reward.
 * Three optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.
 * Three value-update methods: *Standard*, *Gauss–Seidel*, and *Successive over-relaxation*.
@@ -32,25 +13,7 @@ MDPSolver is a Python package for large Markov Decision Processes (MDPs) with in
 
 # Installation
 
-## Linux
-
-Install directly from PyPI with:
-
-```
-pip install mdpsolver
-```
-
-MDPSolver works *out of the box* on Ubuntu 22 and newer.
-
-### GLIBC not found
-
-Some users will encounter the `version 'GLIBC_2.32' not found` error when attempting to import MDPSolver in Python. In this case, it might help to manually compile and replace the SO-file for the optimization module in the MDPSolver package. See the steps on how to solve the issue in the [documentation](https://github.com/areenberg/MDPSolver/wiki#how-to-install).
-
-## Windows
-
-Requires [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) (17.9) with MSVC C++ compiler and libraries installed.
-
-After installing Visual Studio (incl. MSVC C++ compiler and libraries), install directly from PyPI with:
+Prebuilt wheels are available for **Linux**, **Windows**, and **macOS (14+)** across Python 3.10–3.14. Install directly from PyPI with:
 
 ```
 pip install mdpsolver

mdpsolver-0.10.0/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["scikit-build-core>=0.10", "pybind11>=2.13"]
+build-backend = "scikit_build_core.build"
+
+[tool.scikit-build]
+# The CMake project lives outside Python/, next to the rest of the C++ sources.
+cmake.source-dir = "../CPP_Source_Code"
+build-dir = "build/{wheel_tag}"
+# CMakeLists.txt installs the extension to DESTINATION mdpsolver itself,
+# so wheel.install-dir stays at the default (wheel root) here.
+sdist.include = ["src/mdpsolver/*.py"]
+
+[tool.scikit-build.cmake.define]
+CMAKE_BUILD_TYPE = "Release"
+
+[project]
+name = "mdpsolver"
+version = "0.10.0"
+authors = [
+  { name="Anders Reenberg Andersen", email="andersra@live.dk" },
+  { name="Jesper Fink Andersen", email="jesperfinkandersen@gmail.com" },
+]
+description = "A fast solver for Markov Decision Processes"
+readme = "README.md"
+requires-python = ">=3.10"
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: Microsoft :: Windows",
+    "Operating System :: POSIX :: Linux",
+    "Operating System :: MacOS"
+]
+
+[project.optional-dependencies]
+test = ["numpy"]
+
+[project.urls]
+Homepage = "https://github.com/areenberg/MDPSolver"
+Issues = "https://github.com/areenberg/MDPSolver/issues"

{mdpsolver-0.9.9 → mdpsolver-0.10.0}/src/mdpsolver/model.py

@@ -6,9 +6,9 @@ and solve Markov Decision Processes (MDPs). It provides methods to configure the
 solve it, and extract the resulting policy and value vectors.
 """
 
+import sys
 from mdpsolver import solvermodule
 
-
 class model:
     """
     Creates an MDPSolver object.
@@ -27,19 +27,19 @@ class model:
 
     def solve(
         self,
-        algorithm="mpi",
-        tolerance=1e-3,
-        update="standard",
-        criterion="discounted",
-        parIterLim=100,
-        SORrelaxation=1.0,
-        initPolicy=list(),
-        initValueVector=list(),
-        verbose=False,
-        postProcessing=True,
-        makeFinalCheck=True,
-        parallel=True,
-    ):
+        algorithm: str = "mpi",
+        tolerance: float = 1e-3,
+        update: str = "standard",
+        criterion: str = "discounted",
+        parIterLim: int = 100,
+        SORrelaxation: float = 1.0,
+        initPolicy: list = list(),
+        initValueVector: list = list(),
+        verbose: bool = False,
+        postProcessing: bool = True,
+        makeFinalCheck: bool = True,
+        parallel: bool = True,
+    ) -> None:
         """
         Derive an epsilon-optimal policy for the selected MDP model.
 
@@ -60,6 +60,20 @@ class model:
         Returns:
             None
         """
+        
+        if not (algorithm == "mpi" or algorithm == "pi" or algorithm == "vi"):
+            sys.exit("Error: Algorithm type not recognized. Select either: 'mpi', 'pi', or 'vi'.")
+        if tolerance <= 0.0:
+            sys.exit("Error: The tolerance needs to be a positive number.")
+        if not (update == "standard" or update == "gs" or update == "sor"):
+            sys.exit("Error: Update method not recognized. Select either: 'standard', 'gs', or 'sor'.")
+        if not (criterion == "discounted" or criterion == "average"):
+            sys.exit("Error: Optimality criterion not recognized. Select either: 'discounted' or 'average'.")
+        if parIterLim <= 0 and algorithm == "mpi":
+            sys.exit("Error: The partial evaluation limit needs to be a positive number.")
+        if (SORrelaxation <= 0.0 or SORrelaxation >= 2.0) and update == "sor":
+            sys.exit("Error: The relaxation parameter needs to be between 0.0 and 2.0 (i.e. 0 < SORrelaxation < 2).")
+        
         self.mdl.solve(
             algorithm=algorithm,
             tolerance=tolerance,
@@ -75,7 +89,7 @@ class model:
             parallel=parallel,
         )
 
-    def getRuntime(self):
+    def getRuntime(self) -> float:
         """
         Get the runtime of the last solver execution.
 
@@ -84,15 +98,15 @@ class model:
         """
         return self.mdl.getRuntime()
 
-    def printPolicy(self):
+    def printPolicy(self) -> None:
         """Print the entire policy to the terminal."""
         self.mdl.printPolicy()
 
-    def printValueVector(self):
+    def printValueVector(self) -> None:
         """Print the entire value vector to the terminal."""
         self.mdl.printValueVector()
 
-    def getAction(self, stateIndex=0):
+    def getAction(self, stateIndex: int = 0) -> int:
         """
         Get the action from the optimized policy for a specific state.
 
@@ -104,7 +118,7 @@ class model:
         """
         return self.mdl.getAction(stateIndex=stateIndex)
 
-    def getValue(self, stateIndex=0):
+    def getValue(self, stateIndex: int = 0) -> float:
         """
         Get the value from the optimized value vector for a specific state.
 
@@ -116,7 +130,7 @@ class model:
         """
         return self.mdl.getValue(stateIndex=stateIndex)
 
-    def getPolicy(self):
+    def getPolicy(self) -> list:
         """
         Get the entire optimized policy.
 
@@ -134,7 +148,7 @@ class model:
         """
         return self.mdl.getValueVector()
 
-    def saveToFile(self, fileName="result.csv", type="policy"):
+    def saveToFile(self, fileName: str = "result.csv", type: str = "policy") -> None:
         """
         Save the optimized policy or value vector to a file.
 
@@ -145,20 +159,24 @@ class model:
         Returns:
             None
         """
+        
+        if not (type == "policy" or type == "value"):
+            sys.exit("Error: Type not recognized. Select either: 'policy' or 'value'.")
+
         return self.mdl.saveToFile(fileName=fileName, type=type)
 
     def mdp(
         self,
-        discount=0.99,
-        rewards=list(),
-        rewardsElementwise=list(),
-        rewardsFromFile="rewards.csv",
-        tranMatWithZeros=list(),
-        tranMatElementwise=list(),
-        tranMatProbs=list(),
-        tranMatColumns=list(),
-        tranMatFromFile="transitions.csv",
-    ):
+        discount: float = 0.99,
+        rewards: list = list(),
+        rewardsElementwise: list = list(),
+        rewardsFromFile: str = "rewards.csv",
+        tranMatWithZeros: list = list(),
+        tranMatElementwise: list = list(),
+        tranMatProbs: list = list(),
+        tranMatColumns: list = list(),
+        tranMatFromFile: str = "transitions.csv",
+    ) -> None:
         """
         Define the generic MDP model.
 
@@ -176,6 +194,10 @@ class model:
         Returns:
             None
         """
+        
+        if discount <= 0.0 or discount >= 1.0:
+            sys.exit("Error: The discount needs to be in the interval between 0.0 and 1.0 (i.e. 0 < discount < 1).")
+        
         self.mdl.mdp(
             discount=discount,
             rewards=rewards,

mdpsolver-0.10.0/tests/example_model.py

@@ -0,0 +1,139 @@
+import numpy as np
+import os
+
+# Create transition probability matrix and rewards for the 'mdpsolver' examples.
+
+
+# -----------------------------
+#   FUNCTIONS
+# -----------------------------
+
+
+def exitDist(startK, arrRate, serRate, tol=1e-9):
+    # calculate distribution over states observed by
+    # the next arrival assuming the system currently
+    # contains 'startK' customers.
+
+    g = [i * serRate / (i * serRate + arrRate) for i in range(startK, -1, -1)]
+    d = [arrRate / (i * serRate + arrRate) for i in range(startK, -1, -1)]
+
+    Dmat = np.diag(d)
+    Zmat = np.zeros((len(g), len(d)))
+    Imat = np.eye(len(g))
+    Gmat = np.zeros((len(g), len(g)))
+    for i in range(startK):
+        Gmat[i, i + 1] = g[i]
+    mat = np.block([[Gmat, Dmat], [Zmat, Imat]])
+    p = np.zeros((1, (startK + 1) * 2))
+    p[0, 0] = 1
+
+    while np.sum(p[0, : startK + 1]) >= tol:
+        p = np.matmul(p, mat)
+
+    return p[0, startK + 1 :]
+
+
+def tranMat(arrRate1, arrRate2, serRate, capacity, fileName):
+    # calculate transition probabilities and
+    # and save them in a file
+
+    ns = capacity + 1
+    arrRate = arrRate1 + arrRate2
+    pc1 = arrRate1 / arrRate
+    pc2 = 1 - pc1
+
+    if os.path.exists(fileName):
+        os.remove(fileName)
+    with open(fileName, "a") as file:
+        file.write(f"stateFrom,action,stateTo,probability\n")
+        for sidx in range(ns):  # stateFrom (customer type 1)
+            for aidx in range(2):
+                if aidx == 0:  # reject
+                    if sidx > 0:
+                        dist = exitDist(sidx, arrRate, serRate)
+                    else:
+                        dist = np.array([1])
+                    dist1 = pc1 * dist
+                    dist2 = pc2 * dist
+                    for jidx in range(sidx + 1):
+                        out = [sidx, aidx, jidx, dist1[sidx - jidx]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+                    for jidx in range(ns, ns + sidx + 1):
+                        out = [sidx, aidx, jidx, dist2[sidx - (jidx - ns)]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+                elif aidx == 1 and sidx < capacity:  # accept
+                    startK = sidx + 1
+                    dist = exitDist(startK, arrRate, serRate)
+                    dist1 = pc1 * dist
+                    dist2 = pc2 * dist
+                    for jidx in range(startK + 1):
+                        out = [sidx, aidx, jidx, dist1[startK - jidx]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+                    for jidx in range(ns, ns + startK + 1):
+                        out = [sidx, aidx, jidx, dist2[startK - (jidx - ns)]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+
+        for sidx in range(ns):  # stateFrom (customer type 2)
+            for aidx in range(2):
+                if aidx == 0:  # reject
+                    if sidx > 0:
+                        dist = exitDist(sidx, arrRate, serRate)
+                    else:
+                        dist = np.array([1])
+                    dist1 = pc1 * dist
+                    dist2 = pc2 * dist
+                    for jidx in range(sidx + 1):
+                        out = [sidx + ns, aidx, jidx, dist1[sidx - jidx]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+                    for jidx in range(ns, ns + sidx + 1):
+                        out = [sidx + ns, aidx, jidx, dist2[sidx - (jidx - ns)]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+                elif aidx == 1 and sidx < capacity:  # accept
+                    startK = sidx + 1
+                    dist = exitDist(startK, arrRate, serRate)
+                    dist1 = pc1 * dist
+                    dist2 = pc2 * dist
+                    for jidx in range(startK + 1):
+                        out = [sidx + ns, aidx, jidx, dist1[startK - jidx]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+                    for jidx in range(ns, ns + startK + 1):
+                        out = [sidx + ns, aidx, jidx, dist2[startK - (jidx - ns)]]
+                        str_out = ",".join(str(num) for num in out)
+                        file.write(str_out + "\n")
+
+
+def rewards(capacity, rewardCust1, rewardCust2, fileName):
+
+    ns = capacity + 1
+
+    if os.path.exists(fileName):
+        os.remove(fileName)
+    with open(fileName, "a") as file:
+        file.write(f"stateFrom,action,reward\n")
+        for sidx in range(ns):  # stateFrom (customer type 1)
+            for aidx in range(2):
+                if aidx == 0:
+                    out = [sidx, aidx, 0]
+                    str_out = ",".join(str(num) for num in out)
+                    file.write(str_out + "\n")
+                elif aidx == 1 and sidx < capacity:
+                    out = [sidx, aidx, rewardCust1]
+                    str_out = ",".join(str(num) for num in out)
+                    file.write(str_out + "\n")
+        for sidx in range(ns):  # stateFrom (customer type 2)
+            for aidx in range(2):
+                if aidx == 0:
+                    out = [sidx + ns, aidx, 0]
+                    str_out = ",".join(str(num) for num in out)
+                    file.write(str_out + "\n")
+                elif aidx == 1 and sidx < capacity:
+                    out = [sidx + ns, aidx, rewardCust2]
+                    str_out = ",".join(str(num) for num in out)
+                    file.write(str_out + "\n")