mdpsolver 0.9.2__tar.gz → 0.9.4__tar.gz

{mdpsolver-0.9.2/src/mdpsolver.egg-info → mdpsolver-0.9.4}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.1
 Name: mdpsolver
-Version: 0.9.2
-Summary: An efficient solver for Markov Decision Processes
+Version: 0.9.4
+Summary: A fast solver for Markov Decision Processes
 Author-email: Anders Reenberg Andersen <andersra@live.dk>, Jesper Fink Andersen <jfan@dtu.dk>
 Project-URL: Homepage, https://github.com/areenberg/MDPSolver
 Project-URL: Issues, https://github.com/areenberg/MDPSolver/issues
@@ -13,15 +13,16 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 
-# `mdpsolver`: An efficient solver for Markov Decision Processes
+# `mdpsolver`: A fast solver for Markov Decision Processes
  
-`mdpsolver` is the Python package for deriving the policies of Markov Decision Processes (MDPs) with infinite-horizon and the expected total discounted reward optimality criterion. 
+`mdpsolver` is the Python package for Markov Decision Processes (MDPs) with discounted rewards and infinite-horizon. 
 
 ## Features
 
+* Fast solver-engine: Up-to **30x faster** than other Python-based solvers (see details in the documentation).
 * Available on PyPI.
-* Solver-engine developed in C++.
-* Available optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.  
+* Three optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.
+* Three value-update methods: *Standard*, *Gauss–Seidel*, *Successive over-relaxation*.  
 * Includes support for sparse matrices.
 
 # Quick start guide
@@ -137,9 +138,9 @@ mdl.mdp(discount=0.8,
         tranMatFromFile="transitions.csv")
 ```
 
-# User manual
+# Documentation
 
-Further information can be found in the wiki for `mdpsolver` on Github.
+Documentation can be found in the wiki for `mdpsolver` on Github.
 
 # How to cite
 

{mdpsolver-0.9.2 → mdpsolver-0.9.4}/README.md

@@ -1,131 +1,132 @@
-# `mdpsolver`: An efficient solver for Markov Decision Processes
- 
-`mdpsolver` is the Python package for deriving the policies of Markov Decision Processes (MDPs) with infinite-horizon and the expected total discounted reward optimality criterion. 
-
-## Features
-
-* Available on PyPI.
-* Solver-engine developed in C++.
-* Available optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.  
-* Includes support for sparse matrices.
-
-# Quick start guide
-
-The following shows how to get quickly started with `mdpsolver`.
-
-## Installation
-
-Download and install `mdpsolver` directly from PyPI. 
-
-```
-pip install mdpsolver
-```
-
-## Usage
-
-Start by specifying the reward function and transition probabilities as lists. The following is an example of a simple MDP containing **three states** and **two actions in each state**.
-
-```python
-#Import packages
-import mdpsolver
-
-#Rewards (3 states x 2 actions)
-#e.g. choosing second action in first state gives reward=-1
-rewards = [[5,-1],
-           [1,-2],
-           [50,0]]
-
-#Transition probabilities (3 from_states x 2 actions x 3 to_states)
-#e.g. choosing first action in third state gives a probability of 0.6 of staying in third state
-tranMatWithZeros = [[[0.9,0.1,0.0],[0.1,0.9,0.0]],
-                    [[0.4,0.5,0.1],[0.3,0.5,0.2]],
-                    [[0.2,0.2,0.6],[0.5,0.5,0.0]]]
-```
-
-Now, create the model object and insert the problem parameters.
-
-```python
-#Create model object
-mdl = mdpsolver.model()
-
-#Insert the problem parameters
-mdl.mdp(discount=0.8,
-        rewards=rewards,
-        tranMatWithZeros=tranMatWithZeros)
-```
-
-We can now optimize the policy.
-
-```python
-mdl.solve()
-```
-
-The optimized policy can be returned in a variety of ways. Here, we return the policy as a list and print directly in the terminal. 
-
-```python
-print(mdl.getPolicy())
-#[1, 1, 0]
-```
-
-## Sparse transition matrix?
-
-`mdpsolver` has three alternative formats for large and highly sparse transition probability matrices.
-
-**(1) Elementwise representation (excluding elements containing zeros):**
-
-```python
-#[from_state,action,to_state,probability]
-tranMatElementwise = [[0,0,0,0.9],
-                      [0,0,1,0.1],
-                      [0,1,0,0.1],
-                      [0,1,1,0.9],
-                      [1,0,0,0.4],
-                      [1,0,1,0.5],
-                      [1,0,2,0.1],
-                      [1,1,0,0.3],
-                      [1,1,1,0.5],
-                      [1,1,2,0.2],
-                      [2,0,0,0.2],
-                      [2,0,1,0.2],
-                      [2,0,2,0.6],
-                      [2,1,0,0.5],
-                      [2,1,1,0.5]]
-
-mdl.mdp(discount=0.8,
-        rewards=rewards,
-        tranMatElementwise=tranMatElementwise)
-```
-
-**(2) Probabilities and column (to_state) indices in separate lists:**
-
-```python
-tranMatProbs = [[[0.9,0.1],[0.1,0.9]],
-                [[0.4,0.5,0.1],[0.3,0.5,0.2]],
-                [[0.2,0.2,0.6],[0.5,0.5]]]
-
-tranMatColumns = [[[0,1],[0,1]],
-                [[0,1,2],[0,1,2]],
-                [[0,1,2],[0,1]]]
-
-mdl.mdp(discount=0.8,
-        rewards=rewards,
-        tranMatProbs=tranMatProbs,
-        tranMatColumns=tranMatColumns)
-
-```
-
-**(3) Load the elementwise representation from a file:**
-
-```python
-mdl.mdp(discount=0.8,
-        rewards=rewards,
-        tranMatFromFile="transitions.csv")
-```
-
-# User manual
-
-Further information can be found in the wiki for `mdpsolver` on Github.
-
-# How to cite
-
+# `mdpsolver`: A fast solver for Markov Decision Processes
+ 
+`mdpsolver` is the Python package for Markov Decision Processes (MDPs) with discounted rewards and infinite-horizon. 
+
+## Features
+
+* Fast solver-engine: Up-to **30x faster** than other Python-based solvers (see details in the documentation).
+* Available on PyPI.
+* Three optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.
+* Three value-update methods: *Standard*, *Gauss–Seidel*, *Successive over-relaxation*.  
+* Includes support for sparse matrices.
+
+# Quick start guide
+
+The following shows how to get quickly started with `mdpsolver`.
+
+## Installation
+
+Download and install `mdpsolver` directly from PyPI. 
+
+```
+pip install mdpsolver
+```
+
+## Usage
+
+Start by specifying the reward function and transition probabilities as lists. The following is an example of a simple MDP containing **three states** and **two actions in each state**.
+
+```python
+#Import packages
+import mdpsolver
+
+#Rewards (3 states x 2 actions)
+#e.g. choosing second action in first state gives reward=-1
+rewards = [[5,-1],
+           [1,-2],
+           [50,0]]
+
+#Transition probabilities (3 from_states x 2 actions x 3 to_states)
+#e.g. choosing first action in third state gives a probability of 0.6 of staying in third state
+tranMatWithZeros = [[[0.9,0.1,0.0],[0.1,0.9,0.0]],
+                    [[0.4,0.5,0.1],[0.3,0.5,0.2]],
+                    [[0.2,0.2,0.6],[0.5,0.5,0.0]]]
+```
+
+Now, create the model object and insert the problem parameters.
+
+```python
+#Create model object
+mdl = mdpsolver.model()
+
+#Insert the problem parameters
+mdl.mdp(discount=0.8,
+        rewards=rewards,
+        tranMatWithZeros=tranMatWithZeros)
+```
+
+We can now optimize the policy.
+
+```python
+mdl.solve()
+```
+
+The optimized policy can be returned in a variety of ways. Here, we return the policy as a list and print directly in the terminal. 
+
+```python
+print(mdl.getPolicy())
+#[1, 1, 0]
+```
+
+## Sparse transition matrix?
+
+`mdpsolver` has three alternative formats for large and highly sparse transition probability matrices.
+
+**(1) Elementwise representation (excluding elements containing zeros):**
+
+```python
+#[from_state,action,to_state,probability]
+tranMatElementwise = [[0,0,0,0.9],
+                      [0,0,1,0.1],
+                      [0,1,0,0.1],
+                      [0,1,1,0.9],
+                      [1,0,0,0.4],
+                      [1,0,1,0.5],
+                      [1,0,2,0.1],
+                      [1,1,0,0.3],
+                      [1,1,1,0.5],
+                      [1,1,2,0.2],
+                      [2,0,0,0.2],
+                      [2,0,1,0.2],
+                      [2,0,2,0.6],
+                      [2,1,0,0.5],
+                      [2,1,1,0.5]]
+
+mdl.mdp(discount=0.8,
+        rewards=rewards,
+        tranMatElementwise=tranMatElementwise)
+```
+
+**(2) Probabilities and column (to_state) indices in separate lists:**
+
+```python
+tranMatProbs = [[[0.9,0.1],[0.1,0.9]],
+                [[0.4,0.5,0.1],[0.3,0.5,0.2]],
+                [[0.2,0.2,0.6],[0.5,0.5]]]
+
+tranMatColumns = [[[0,1],[0,1]],
+                [[0,1,2],[0,1,2]],
+                [[0,1,2],[0,1]]]
+
+mdl.mdp(discount=0.8,
+        rewards=rewards,
+        tranMatProbs=tranMatProbs,
+        tranMatColumns=tranMatColumns)
+
+```
+
+**(3) Load the elementwise representation from a file:**
+
+```python
+mdl.mdp(discount=0.8,
+        rewards=rewards,
+        tranMatFromFile="transitions.csv")
+```
+
+# Documentation
+
+Documentation can be found in the wiki for `mdpsolver` on Github.
+
+# How to cite
+
 [![DOI](https://zenodo.org/badge/294063917.svg)](https://zenodo.org/badge/latestdoi/294063917)

{mdpsolver-0.9.2 → mdpsolver-0.9.4}/pyproject.toml

@@ -10,12 +10,12 @@ where = ["src"]
 
 [project]
 name = "mdpsolver"
-version = "0.9.2"
+version = "0.9.4"
 authors = [
   { name="Anders Reenberg Andersen", email="andersra@live.dk" },
   { name="Jesper Fink Andersen", email="jfan@dtu.dk" },
 ]
-description = "An efficient solver for Markov Decision Processes"
+description = "A fast solver for Markov Decision Processes"
 readme = "README.md"
 requires-python = ">=3.10"
 classifiers = [

{mdpsolver-0.9.2 → mdpsolver-0.9.4}/src/mdpsolver/model.py

@@ -21,7 +21,9 @@ class model:
               SORrelaxation=1.0,
               initPolicy=list(),
               initValueVector=list(),
-              verbose=False):
+              verbose=False,
+              postProcessing=True,
+              makeFinalCheck=True):
         #solves the problem employing the specified algorithm
         self.mdl.solve(algorithm=algorithm,
               tolerance=tolerance,
@@ -30,7 +32,9 @@ class model:
               SORrelaxation=SORrelaxation,
               initPolicy=initPolicy,
               initValueVector=initValueVector,
-              verbose=verbose)
+              verbose=verbose,
+              postProcessing=postProcessing,
+              makeFinalCheck=makeFinalCheck)
     
     def getRuntime(self):
         #returns the runtime in milliseconds

{mdpsolver-0.9.2 → mdpsolver-0.9.4/src/mdpsolver.egg-info}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.1
 Name: mdpsolver
-Version: 0.9.2
-Summary: An efficient solver for Markov Decision Processes
+Version: 0.9.4
+Summary: A fast solver for Markov Decision Processes
 Author-email: Anders Reenberg Andersen <andersra@live.dk>, Jesper Fink Andersen <jfan@dtu.dk>
 Project-URL: Homepage, https://github.com/areenberg/MDPSolver
 Project-URL: Issues, https://github.com/areenberg/MDPSolver/issues
@@ -13,15 +13,16 @@ Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
 
-# `mdpsolver`: An efficient solver for Markov Decision Processes
+# `mdpsolver`: A fast solver for Markov Decision Processes
  
-`mdpsolver` is the Python package for deriving the policies of Markov Decision Processes (MDPs) with infinite-horizon and the expected total discounted reward optimality criterion. 
+`mdpsolver` is the Python package for Markov Decision Processes (MDPs) with discounted rewards and infinite-horizon. 
 
 ## Features
 
+* Fast solver-engine: Up-to **30x faster** than other Python-based solvers (see details in the documentation).
 * Available on PyPI.
-* Solver-engine developed in C++.
-* Available optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.  
+* Three optimization algorithms: *Value iteration*, *Policy iteration*, and *Modified policy iteration*.
+* Three value-update methods: *Standard*, *Gauss–Seidel*, *Successive over-relaxation*.  
 * Includes support for sparse matrices.
 
 # Quick start guide
@@ -137,9 +138,9 @@ mdl.mdp(discount=0.8,
         tranMatFromFile="transitions.csv")
 ```
 
-# User manual
+# Documentation
 
-Further information can be found in the wiki for `mdpsolver` on Github.
+Documentation can be found in the wiki for `mdpsolver` on Github.
 
 # How to cite