POMDPPlanners 0.1.0__tar.gz

pomdpplanners-0.1.0/LICENSE.md

@@ -0,0 +1,7 @@
+Copyright 2025 Yaacov Pariente
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

pomdpplanners-0.1.0/PKG-INFO

@@ -0,0 +1,378 @@
+Metadata-Version: 2.4
+Name: POMDPPlanners
+Version: 0.1.0
+Summary: A Python package for POMDP planning algorithms and environments
+Author-email: Yaacov Pariente <yaacovpar@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/yaacovpariente/POMDPPlanners
+Project-URL: Bug Reports, https://github.com/yaacovpariente/POMDPPlanners/issues
+Project-URL: Documentation, https://yaacovpariente.github.io/POMDPPlanners/
+Keywords: pomdp,planning,reinforcement-learning,monte-carlo-tree-search,artificial-intelligence
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: numpy>=1.21.0
+Requires-Dist: ray>=2.0.0
+Requires-Dist: pandas>=1.3.0
+Requires-Dist: matplotlib>=3.4.0
+Requires-Dist: scipy>=1.7.0
+Requires-Dist: gymnasium>=0.29.0
+Requires-Dist: anytree>=2.8.0
+Requires-Dist: mlflow>=2.0.0
+Requires-Dist: redis>=4.0.0
+Requires-Dist: seaborn>=0.11.0
+Requires-Dist: plotly>=5.0.0
+Requires-Dist: diskcache>=5.0.0
+Requires-Dist: joblib>=1.0.0
+Requires-Dist: tqdm>=4.62.0
+Requires-Dist: optuna>=2.0.0
+Requires-Dist: pyyaml>=6.0.0
+Requires-Dist: dask>=2023.3.0
+Requires-Dist: distributed>=2023.3.0
+Requires-Dist: dask_jobqueue>=0.8.0
+Requires-Dist: cachey>=0.2.1
+Requires-Dist: torch>=2.0.0
+Requires-Dist: tensorboard>=2.0.0
+Requires-Dist: line_profiler>=4.0.0
+Requires-Dist: memory_profiler>=0.60.0
+Requires-Dist: psutil>=5.8.0
+Requires-Dist: bokeh>=3.0.0
+Provides-Extra: dev
+Requires-Dist: pylint>=2.17.0; extra == "dev"
+Requires-Dist: black==23.12.1; extra == "dev"
+Requires-Dist: pyright>=1.1.0; extra == "dev"
+Requires-Dist: pre-commit>=3.0.0; extra == "dev"
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: nbval>=0.9.6; extra == "dev"
+Requires-Dist: papermill>=2.6.0; extra == "dev"
+Requires-Dist: setuptools>=65.0.0; extra == "dev"
+Requires-Dist: ipykernel>=6.0.0; extra == "dev"
+Requires-Dist: jupyter-client>=7.0.0; extra == "dev"
+Requires-Dist: nbformat>=5.0.0; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: sphinx>=5.0.0; extra == "docs"
+Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
+Requires-Dist: sphinx-autodoc-typehints>=1.17.0; extra == "docs"
+Requires-Dist: myst-parser>=0.18.0; extra == "docs"
+Dynamic: license-file
+
+# POMDPPlanners
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/release/python-3100/)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+
+A comprehensive Python package for **POMDP (Partially Observable Markov Decision Process)** planning algorithms and environments. POMDPPlanners provides standardized simulation studies for research and reliable implementations of planning algorithms for industrial applications.
+
+## 🎯 Key Features
+
+- **Comprehensive Algorithm Library**: Implementations of state-of-the-art POMDP planning algorithms including POMCP, POMCPOW, POMCP-DPW, PFT-DPW, Sparse PFT, BetaZero, ConstrainedZero, and more
+- **Rich Environment Collection**: Classic and modern POMDP environments (Tiger, Light-Dark, RockSample, LaserTag, PacMan, CartPole, Push, Safety-Ant-Velocity, etc.)
+- **Flexible Belief Representations**: Particle filters, weighted beliefs, Gaussian beliefs, Gaussian mixture beliefs, and vectorized belief updaters
+- **Simulation Framework**: Complete experiment management with hyperparameter tuning, high-level evaluation workflows, and distributed computing support
+- **Visualization Tools**: Built-in plotting and visualization capabilities for analysis and debugging
+- **Production Ready**: Designed for both research experiments and industrial applications
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+# Clone the repository
+git clone https://github.com/yaacovpariente/POMDPPlanners.git
+cd POMDPPlanners
+
+# Create and activate virtual environment
+python -m venv .venv
+source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+
+# Install package (standard)
+pip install -e .
+
+# Install with development dependencies
+pip install -e ".[dev]"
+```
+
+### Basic Usage
+
+```python
+from POMDPPlanners.environments.tiger_pomdp import TigerPOMDP
+from POMDPPlanners.planners.mcts_planners.pomcp import POMCP
+from POMDPPlanners.core.belief import WeightedParticleBelief
+
+# Create environment and planner
+env = TigerPOMDP()
+planner = POMCP(env, num_simulations=1000)
+
+# Initialize belief and run planning
+initial_belief = WeightedParticleBelief.create_uniform_belief(
+    env.get_states(), num_particles=1000
+)
+
+# Get action from planner
+action = planner.get_action(initial_belief)
+print(f"Recommended action: {action}")
+```
+
+## 🏗️ Architecture Overview
+
+### Core Components
+
+- **`POMDPPlanners.core`**: Fundamental abstractions and interfaces
+  - `environment.py`: Base `Environment`, `DiscreteActionsEnvironment`, `SpaceType`, `ObservationModel`, `StateTransitionModel`
+  - `policy.py`: Base `Policy` and `TrainablePolicy` classes with config management and logging
+  - `belief/`: Belief state representations including:
+    - `WeightedParticleBelief`, `UnweightedParticleBelief` — particle filter beliefs
+    - `GaussianBelief`, `GaussianMixtureBelief` — parametric beliefs
+    - `VectorizedWeightedParticleBelief` — vectorized particle belief for performance
+    - Gaussian belief updaters and vectorized particle belief updaters
+  - `cost.py`: Cost function abstractions for constrained planning
+  - `distributions.py`: Probability distribution implementations
+  - `tree.py`: Tree-based data structures for planning algorithms
+  - `config_types.py`: Centralized configuration type definitions
+
+- **`POMDPPlanners.environments`**: POMDP environment implementations
+- **`POMDPPlanners.planners`**: Planning algorithm implementations
+- **`POMDPPlanners.simulations`**: Experiment management and execution framework
+- **`POMDPPlanners.utils`**: Helper functions and visualization tools
+
+### Supported Algorithms
+
+#### MCTS-Based Planners
+- **POMCP**: Partially Observable Monte Carlo Planning
+- **POMCPOW**: POMCP with Online Widening (progressive widening for observations)
+- **POMCP-DPW**: POMCP with Double Progressive Widening
+- **PFT-DPW**: Progressive Widening with Particle Filter Trees
+- **Sparse PFT**: Sparse sampling with Particle Filter Trees
+
+#### Neural MCTS Planners
+- **BetaZero**: Neural-guided MCTS for POMDPs — adapts AlphaZero to belief-space planning with a dual-head neural network (value + policy)
+- **ConstrainedZero**: Safety-constrained extension of BetaZero for chance-constrained POMDPs — adds a failure-probability head and adaptive threshold calibration via conformal inference
+
+#### Other Planners
+- **Sparse Sampling**: Classical sparse sampling algorithm
+- **Open Loop Planners**: Non-feedback planning approaches (discrete action sequences)
+
+### Available Environments
+
+- **Tiger POMDP**: Classic two-door problem
+- **Light-Dark POMDP**: Navigation with position-dependent observation noise (discrete and continuous variants)
+- **Rock Sample POMDP**: Grid-world rover navigation with multiple rock samples; rover must sense and collect good rocks
+- **Laser Tag POMDP**: Tag-based pursuit environment (discrete and continuous geometry variants)
+- **PacMan POMDP**: Partially observable Pac-Man navigation with ghost uncertainty
+- **CartPole POMDP**: Partially observable cart-pole balancing
+- **Mountain Car POMDP**: Partially observable mountain car
+- **Push POMDP**: Object manipulation environment
+- **Safety Ant Velocity**: Safety-constrained locomotion task
+
+### Simulations Framework
+
+- **`POMDPPlanners.simulations.simulator`**: Main simulation runner
+- **`POMDPPlanners.simulations.episodes`**: Episode execution logic
+- **`POMDPPlanners.simulations.workflows`**: High-level evaluation and optimization workflows
+  - `evaluation.py`, `optimization.py`, `integrated.py`
+  - `planner_evaluation_workflow.py`, `hyperparameter_tuning_evaluation_workflows.py`
+- **`POMDPPlanners.simulations.simulation_apis`**: Pluggable backends for experiment execution (see Distributed Computing section)
+- **`POMDPPlanners.simulations.simulations_deployment`**: Task managers and caching for distributed runs
+
+### Utilities
+
+- **`POMDPPlanners.utils.visualization`**: Visualization submodule with
+  - `metrics_plots.py`, `returns_plots.py`, `tree_plots.py`, `policy_simulation_plots.py`
+- **`POMDPPlanners.utils.config_loader`**: Configuration file management
+- **`POMDPPlanners.utils.logger`**: Centralized logging setup
+- **`POMDPPlanners.utils.statistics_utils`**: Statistical analysis functions
+- **`POMDPPlanners.utils.tree_statistics`**: Tree statistics computation
+
+## 📊 Running Experiments
+
+### Simple Experiment
+
+```python
+from POMDPPlanners.simulations.simulator import Simulator
+from POMDPPlanners.utils.config_loader import load_config
+
+# Load experiment configuration
+config = load_config("experiments/configs/tiger_pomcp_experiment.yaml")
+
+# Run simulation
+simulator = Simulator(config)
+results = simulator.run()
+
+print(f"Average reward: {results['average_reward']}")
+```
+
+### Hyperparameter Tuning
+
+```python
+from POMDPPlanners.simulations.hyper_parameter_tuning_simulations import HyperParameterTuningSimulations
+
+# Define hyperparameter space
+hyperparams = {
+    "num_simulations": [500, 1000, 2000],
+    "exploration_constant": [1.0, 2.0, 5.0],
+    "discount_factor": [0.9, 0.95, 0.99]
+}
+
+# Run tuning experiment
+tuner = HyperParameterTuningSimulations(
+    base_config="experiments/configs/base_config.yaml",
+    hyperparameters=hyperparams
+)
+best_params = tuner.optimize()
+```
+
+### Example Notebooks
+
+Interactive Jupyter notebooks with detailed usage examples are available in `docs/examples/`:
+
+- `docs/examples/basic_usage.ipynb` — Environment setup, belief initialization, and basic planning
+- `docs/examples/hyperparameter_tuning.ipynb` — End-to-end hyperparameter search
+- `docs/examples/planners_comparison.ipynb` — Side-by-side comparison of planning algorithms
+
+## 🌐 Distributed Computing
+
+POMDPPlanners supports multiple execution backends for scaling experiments:
+
+| Backend | Description | Use Case |
+|---------|-------------|----------|
+| **Local** | Sequential single-process | Development and debugging |
+| **Dask** | Distributed multi-machine cluster | Large-scale parallelism |
+| **PBS** | HPC cluster via `dask-jobqueue` | Supercomputer / PBS job scheduler |
+
+Select the backend via the simulation API in `POMDPPlanners.simulations.simulation_apis`:
+
+```python
+# Local sequential execution
+from POMDPPlanners.simulations.simulation_apis.local_simulations_api import LocalSimulationsAPI
+
+# Dask distributed execution
+from POMDPPlanners.simulations.simulation_apis.dask_simulations_api import DaskSimulationsAPI
+
+# PBS (HPC) cluster execution
+from POMDPPlanners.simulations.simulation_apis.pbs_simulations_api import PBSSimulationsAPI
+```
+
+## 🧪 Testing
+
+Run the comprehensive test suite:
+
+```bash
+# Activate virtual environment
+source .venv/bin/activate
+
+# Run all tests
+pytest
+
+# Run specific test categories
+pytest POMDPPlanners/tests/test_core/
+pytest POMDPPlanners/tests/test_environments/
+pytest POMDPPlanners/tests/test_planners/
+
+# Run with verbose output
+pytest -v
+
+# Run specific test file
+pytest POMDPPlanners/tests/test_core/test_belief.py
+```
+
+## 🔧 Development
+
+### Code Quality
+
+```bash
+# Format code
+black .
+
+# Type checking
+python -m pyright POMDPPlanners/
+
+# Run linting
+pylint POMDPPlanners/
+flake8 .
+
+# Install pre-commit hooks
+pre-commit install
+```
+
+### Virtual Environment
+
+**Important**: Always activate the virtual environment before development:
+
+```bash
+source .venv/bin/activate  # Linux/Mac
+# .venv\Scripts\activate   # Windows
+```
+
+All commands should be run within this environment for consistent dependency management.
+
+## 📚 Documentation
+
+Comprehensive documentation is generated from docstrings using Sphinx:
+
+```bash
+# Build documentation
+cd docs/
+sphinx-build -b html . _build/html
+
+# Serve locally
+python -m http.server 8000 -d _build/html
+```
+
+Visit the documentation at: [Project Documentation](https://yaacovpariente.github.io/POMDPPlanners/) *(available when repository becomes public)*
+
+## 🤝 Contributing
+
+We welcome contributions of all kinds! Please see our [Contributing Guide](CONTRIBUTING.md) for details on:
+
+- Setting up the development environment
+- Coding guidelines and standards
+- Submitting pull requests
+- Reporting bugs and requesting features
+
+Please also read our [Code of Conduct](CODE_OF_CONDUCT.md).
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details.
+
+## 🎓 Citation
+
+If you use POMDPPlanners in your research, please cite:
+
+```bibtex
+@software{pomdpplanners2025,
+  title={POMDPPlanners: A Python Package for POMDP Planning Algorithms},
+  author={Pariente, Yaacov},
+  year={2025},
+  url={https://github.com/yaacovpariente/POMDPPlanners}
+}
+```
+
+## 🛠️ Requirements
+
+- Python 3.10 or higher
+- Core dependencies managed via `pyproject.toml` (`pip install -e .`)
+- Development dependencies: `pip install -e ".[dev]"`
+
+## 🔗 Links
+
+- **Repository**: https://github.com/yaacovpariente/POMDPPlanners
+- **Documentation**: https://yaacovpariente.github.io/POMDPPlanners/ *(available when repository becomes public)*
+- **Issues**: https://github.com/yaacovpariente/POMDPPlanners/issues
+- **Discussions**: https://github.com/yaacovpariente/POMDPPlanners/discussions
+
+---
+
+**Made with ❤️ for the POMDP research community**

pomdpplanners-0.1.0/POMDPPlanners/__init__.py

@@ -0,0 +1,3 @@
+"""POMDPPlanners - A Python package for POMDP planning algorithms and environments."""
+
+__version__ = "0.1.0"

pomdpplanners-0.1.0/POMDPPlanners/configs/__init__.py

@@ -0,0 +1,4 @@
+from POMDPPlanners.configs.environment_configs import EnvironmentConfigsAPI
+from POMDPPlanners.configs.planners_hyperparam_configs import PlannersHyperparamConfigs
+
+__all__ = ["PlannersHyperparamConfigs", "EnvironmentConfigsAPI"]