saferl-lite 0.1.0__tar.gz → 0.1.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. saferl_lite-0.1.3/PKG-INFO +241 -0
  2. saferl_lite-0.1.3/README.md +189 -0
  3. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/about.md +1 -1
  4. saferl_lite-0.1.3/saferl_lite.egg-info/PKG-INFO +241 -0
  5. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/setup.py +2 -2
  6. saferl_lite-0.1.0/PKG-INFO +0 -139
  7. saferl_lite-0.1.0/README.md +0 -87
  8. saferl_lite-0.1.0/saferl_lite.egg-info/PKG-INFO +0 -139
  9. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/LICENSE +0 -0
  10. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/MANIFEST.in +0 -0
  11. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/agents/__init__.py +0 -0
  12. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/agents/constrained_dqn.py +0 -0
  13. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/agents/constraints.py +0 -0
  14. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/docs/index.md +0 -0
  15. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/evaluation/metrics.md +0 -0
  16. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/examples/cartpole.md +0 -0
  17. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/explainability/saliency.md +0 -0
  18. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/explainability/shap.md +0 -0
  19. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/mkdocs.yml +0 -0
  20. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/reference/agents.md +0 -0
  21. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/reference/constraints.md +0 -0
  22. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/reference/envs.md +0 -0
  23. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/reference.md +0 -0
  24. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/usage/installation.md +0 -0
  25. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/docs/usage/quickstart.md +0 -0
  26. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/envs/__init__.py +0 -0
  27. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/envs/wrappers.py +0 -0
  28. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/explainability/__init__.py +0 -0
  29. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/explainability/saliency.py +0 -0
  30. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/explainability/shap_explainer.py +0 -0
  31. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/requirements.txt +0 -0
  32. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/saferl_lite.egg-info/SOURCES.txt +0 -0
  33. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/saferl_lite.egg-info/dependency_links.txt +0 -0
  34. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/saferl_lite.egg-info/requires.txt +0 -0
  35. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/saferl_lite.egg-info/top_level.txt +0 -0
  36. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/setup.cfg +0 -0
  37. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/tests/test_constraints.py +0 -0
  38. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/tests/test_metrics.py +0 -0
  39. {saferl_lite-0.1.0 → saferl_lite-0.1.3}/tests/test_safe_env.py +0 -0
@@ -0,0 +1,241 @@
1
+ Metadata-Version: 2.4
2
+ Name: saferl-lite
3
+ Version: 0.1.3
4
+ Summary: A lightweight, explainable, and constrained reinforcement learning toolkit.
5
+ Home-page: https://github.com/satyamcser/saferl-lite
6
+ Author: Satyam Mishra
7
+ Author-email: satyam@example.com
8
+ Project-URL: Documentation, https://github.com/satyamcser/saferl-lite/tree/main/docs
9
+ Project-URL: Source, https://github.com/satyamcser/saferl-lite
10
+ Project-URL: Bug Tracker, https://github.com/satyamcser/saferl-lite/issues
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: License :: OSI Approved :: MIT License
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
16
+ Requires-Python: >=3.8
17
+ Description-Content-Type: text/markdown
18
+ License-File: LICENSE
19
+ Requires-Dist: gym
20
+ Requires-Dist: gymnasium
21
+ Requires-Dist: numpy
22
+ Requires-Dist: torch
23
+ Requires-Dist: matplotlib
24
+ Requires-Dist: seaborn
25
+ Requires-Dist: pre-commit
26
+ Requires-Dist: flake8
27
+ Requires-Dist: pyyaml
28
+ Requires-Dist: shap
29
+ Requires-Dist: captum
30
+ Requires-Dist: typer
31
+ Requires-Dist: scikit-learn
32
+ Requires-Dist: pandas
33
+ Requires-Dist: pytest
34
+ Requires-Dist: pytest-cov
35
+ Requires-Dist: coverage
36
+ Requires-Dist: mkdocs
37
+ Requires-Dist: wandb
38
+ Requires-Dist: mkdocs>=1.5
39
+ Requires-Dist: mkdocs-material>=9.5
40
+ Requires-Dist: mkdocstrings[python]
41
+ Dynamic: author
42
+ Dynamic: author-email
43
+ Dynamic: classifier
44
+ Dynamic: description
45
+ Dynamic: description-content-type
46
+ Dynamic: home-page
47
+ Dynamic: license-file
48
+ Dynamic: project-url
49
+ Dynamic: requires-dist
50
+ Dynamic: requires-python
51
+ Dynamic: summary
52
+
53
+ # 🔐 SafeRL-Lite
54
+
55
+ A **lightweight, explainable, and modular** Python library for **Constrained Reinforcement Learning (Safe RL)** with real-time **SHAP & saliency-based explainability**, custom metrics, and Gym-compatible wrappers.
56
+
57
+ **New update with ease to setup, install and import coming soon.
58
+
59
+ By:
60
+ - Satyam Mishra, Vision Mentors Ltd., Hanoi, Vietnam
61
+ - Shivam Mishra, Phung Thao Vi, Vietnam National University, Hanoi, Vietnam
62
+ - Dr. Vishwanath Bijalwan, SR University, Warangal, India
63
+ - Dr. Vijay Bhaskar Semwal, MANIT, Bhopal, India
64
+ - Dr. Abdul Manan Khan, University of West London, London, UK
65
+
66
+ <p align="center">
67
+ <a href="https://github.com/satyamcser/saferl-lite/blob/main/LICENSE">
68
+ <img src="https://img.shields.io/github/license/satyamcser/saferl-lite?style=flat-square" alt="License">
69
+ </a>
70
+ <a href="https://github.com/satyamcser/saferl-lite/stargazers">
71
+ <img src="https://img.shields.io/github/stars/satyamcser/saferl-lite?style=flat-square" alt="Stars">
72
+ </a>
73
+ <a href="https://pypi.org/project/saferl-lite/">
74
+ <img src="https://img.shields.io/pypi/v/saferl-lite?style=flat-square" alt="PyPI version">
75
+ </a>
76
+ <a href="https://github.com/satyamcser/saferl-lite/actions/workflows/ci.yml">
77
+ <img src="https://img.shields.io/github/actions/workflow/status/satyamcser/saferl-lite/ci.yml?branch=main&style=flat-square" alt="Build Status">
78
+ </a>
79
+ </p>
80
+
81
+
82
+ ---
83
+
84
+ ## 🌟 Overview
85
+
86
+ **SafeRL-Lite** empowers reinforcement learning agents to act under **safety constraints**, while remaining **interpretable** and **modular** for fast experimentation. It wraps standard Gym environments and DQN-based agents with:
87
+
88
+ - ✅ Safety constraint logic
89
+ - 🔍 Visual explainability (SHAP, saliency maps)
90
+ - 📊 Violation and reward tracking
91
+ - 🧪 Built-in testing and evaluations
92
+
93
+ ---
94
+
95
+ ## ✅ Problem We Solved
96
+
97
+ Modern Reinforcement Learning (RL) agents are powerful but unsafe and opaque:
98
+
99
+ - 🚫 They frequently violate safety constraints during learning or deployment (e.g., fall off a cliff in navigation tasks).
100
+
101
+ - 😕 Their decision-making is a black box: humans can’t understand why a certain action was chosen.
102
+
103
+ - 🔍 Standard RL libraries lack native support for:
104
+
105
+ - Enforcing hard constraints during training.
106
+
107
+ - Explaining decisions using methods like SHAP or saliency maps.
108
+
109
+ ## ✅ Our Solution
110
+ SafeRL-Lite is a lightweight Python library that:
111
+
112
+ 1. 📏 Adds a SafetyWrapper around any Gym environment to enforce safety constraints (e.g., bounding actions, limiting states).
113
+
114
+ 2. 🧠 Integrates explainability methods:
115
+
116
+ - SHAPExplainer (model-agnostic local explanations).
117
+
118
+ - SaliencyExplainer (gradient-based sensitivity maps).
119
+
120
+ 3. 🔧 Wraps Constrained DQNs with ease, enabling safety-compliant Q-learning.
121
+
122
+ 4. 📊 Offers built-in metrics like violation count and safe episode tracking.
123
+
124
+ ## ✅ Novelty
125
+ While Safe RL and Explainable RL are separately studied, no prior lightweight library:
126
+
127
+ - Combines hard safety constraints with post-hoc interpretability.
128
+
129
+ - Is designed to be minimal, pluggable, and easily installable (pip install saferl-lite) for education, experimentation, or safe deployment.
130
+
131
+ - Enables real-time SHAP or saliency visualization for Gym-based agents out-of-the-box.
132
+ ``` bash
133
+ SafeRL-Lite is the first minimal library to unify constraint satisfaction and explainability in reinforcement learning — without heavy dependencies or overhead.
134
+ ```
135
+
136
+ ## ✅ Our Contribution
137
+ 1. 🔐 Constraint Wrapper API: Drop-in Gym wrapper for defining and enforcing logical constraints on observations, actions, and reward signals.
138
+
139
+ 2. 🧠 Explainability Modules: Plug-and-play SHAP and saliency explainer classes for deep Q-networks.
140
+
141
+ 3. 📦 PyPI-Ready Toolkit: Easily installed, documented, and CI/CD tested; built for research and reproducibility.
142
+
143
+ 4. 📈 Metrics for Constraint Violation: Tracks unsafe episodes, per-step violations, and integrates cleanly with WandB or TensorBoard.
144
+
145
+ ## ✅ Technical Explanation
146
+ - We define a custom SafeEnvWrapper(gym.Env) that:
147
+
148
+ - Intercepts actions.
149
+
150
+ - Applies logical rules or thresholding.
151
+
152
+ - Optionally overrides rewards or terminations if constraints are violated.
153
+
154
+ - A ConstrainedDQNAgent uses:
155
+
156
+ - Safety-wrapped Gym envs.
157
+
158
+ - Standard Q-learning with optional penalty_on_violation flag.
159
+
160
+ - Post-training, the SHAPExplainer and SaliencyExplainer:
161
+
162
+ - Generate local attributions using input perturbations or gradient norms.
163
+
164
+ - Can visualize per-state or per-action explanations.
165
+
166
+
167
+
168
+ ## ✅ Satyam's Explanation
169
+ ```bash
170
+ Imagine you're teaching a robot to walk — but there’s lava on the floor!
171
+ You don’t just want it to learn fast, you want it to stay safe and explain why it stepped left, not right.
172
+ ```
173
+ SafeRL-Lite is like a safety helmet and voicebox for robots:
174
+
175
+ - The helmet makes sure they don’t do dangerous stuff.
176
+
177
+ - The voicebox lets them say why they made that move.
178
+
179
+ ## 🔧 Installation
180
+
181
+ > 📦 PyPI
182
+ ```bash
183
+ pip install saferl-lite
184
+ ```
185
+
186
+ ## 🛠️ From source:
187
+
188
+ ```bash
189
+ git clone https://github.com/satyamcser/saferl-lite.git
190
+ cd saferl-lite
191
+ pip install -e .
192
+ ```
193
+
194
+ ## 🚀 Quickstart
195
+ Train a constrained DQN agent with saliency-based explainability:
196
+
197
+ ```bash
198
+ python train.py --env CartPole-v1 --constraint pole_angle --explain shap
199
+ ```
200
+
201
+ 🔹 This:
202
+
203
+ - Adds a pole-angle constraint wrapper to the Gym env
204
+
205
+ - Logs violations
206
+
207
+ - Displays SHAP or saliency explanations for agent decisions
208
+
209
+ ## 🧠 Features
210
+ #### ✅ Constrained RL
211
+ - Add custom constraints via wrapper or logic class
212
+
213
+ - Violation logging and reward shaping
214
+
215
+ - Safe vs unsafe episode tracking
216
+
217
+ #### 🔍 Explainability
218
+ - SaliencyExplainer — gradient-based visual heatmaps
219
+
220
+ - SHAPExplainer — feature contribution values per decision
221
+
222
+ - Compatible with any PyTorch-based agent
223
+
224
+ #### 📊 Metrics
225
+ - Constraint violation rate
226
+
227
+ - Episode reward
228
+
229
+ - Cumulative safe reward
230
+
231
+ - Action entropy & temporal behavior stats
232
+
233
+ #### 📚 Modularity
234
+ - Swap out agents, constraints, evaluators, or explainers
235
+
236
+ - Supports Gym environments
237
+
238
+ - Configurable training pipeline
239
+
240
+ ## 📜 Citation
241
+ Coming soon after arXiv/preprint release.
@@ -0,0 +1,189 @@
1
+ # 🔐 SafeRL-Lite
2
+
3
+ A **lightweight, explainable, and modular** Python library for **Constrained Reinforcement Learning (Safe RL)** with real-time **SHAP & saliency-based explainability**, custom metrics, and Gym-compatible wrappers.
4
+
5
+ **New update with ease to setup, install and import coming soon.
6
+
7
+ By:
8
+ - Satyam Mishra, Vision Mentors Ltd., Hanoi, Vietnam
9
+ - Shivam Mishra, Phung Thao Vi, Vietnam National University, Hanoi, Vietnam
10
+ - Dr. Vishwanath Bijalwan, SR University, Warangal, India
11
+ - Dr. Vijay Bhaskar Semwal, MANIT, Bhopal, India
12
+ - Dr. Abdul Manan Khan, University of West London, London, UK
13
+
14
+ <p align="center">
15
+ <a href="https://github.com/satyamcser/saferl-lite/blob/main/LICENSE">
16
+ <img src="https://img.shields.io/github/license/satyamcser/saferl-lite?style=flat-square" alt="License">
17
+ </a>
18
+ <a href="https://github.com/satyamcser/saferl-lite/stargazers">
19
+ <img src="https://img.shields.io/github/stars/satyamcser/saferl-lite?style=flat-square" alt="Stars">
20
+ </a>
21
+ <a href="https://pypi.org/project/saferl-lite/">
22
+ <img src="https://img.shields.io/pypi/v/saferl-lite?style=flat-square" alt="PyPI version">
23
+ </a>
24
+ <a href="https://github.com/satyamcser/saferl-lite/actions/workflows/ci.yml">
25
+ <img src="https://img.shields.io/github/actions/workflow/status/satyamcser/saferl-lite/ci.yml?branch=main&style=flat-square" alt="Build Status">
26
+ </a>
27
+ </p>
28
+
29
+
30
+ ---
31
+
32
+ ## 🌟 Overview
33
+
34
+ **SafeRL-Lite** empowers reinforcement learning agents to act under **safety constraints**, while remaining **interpretable** and **modular** for fast experimentation. It wraps standard Gym environments and DQN-based agents with:
35
+
36
+ - ✅ Safety constraint logic
37
+ - 🔍 Visual explainability (SHAP, saliency maps)
38
+ - 📊 Violation and reward tracking
39
+ - 🧪 Built-in testing and evaluations
40
+
41
+ ---
42
+
43
+ ## ✅ Problem We Solved
44
+
45
+ Modern Reinforcement Learning (RL) agents are powerful but unsafe and opaque:
46
+
47
+ - 🚫 They frequently violate safety constraints during learning or deployment (e.g., fall off a cliff in navigation tasks).
48
+
49
+ - 😕 Their decision-making is a black box: humans can’t understand why a certain action was chosen.
50
+
51
+ - 🔍 Standard RL libraries lack native support for:
52
+
53
+ - Enforcing hard constraints during training.
54
+
55
+ - Explaining decisions using methods like SHAP or saliency maps.
56
+
57
+ ## ✅ Our Solution
58
+ SafeRL-Lite is a lightweight Python library that:
59
+
60
+ 1. 📏 Adds a SafetyWrapper around any Gym environment to enforce safety constraints (e.g., bounding actions, limiting states).
61
+
62
+ 2. 🧠 Integrates explainability methods:
63
+
64
+ - SHAPExplainer (model-agnostic local explanations).
65
+
66
+ - SaliencyExplainer (gradient-based sensitivity maps).
67
+
68
+ 3. 🔧 Wraps Constrained DQNs with ease, enabling safety-compliant Q-learning.
69
+
70
+ 4. 📊 Offers built-in metrics like violation count and safe episode tracking.
71
+
72
+ ## ✅ Novelty
73
+ While Safe RL and Explainable RL are separately studied, no prior lightweight library:
74
+
75
+ - Combines hard safety constraints with post-hoc interpretability.
76
+
77
+ - Is designed to be minimal, pluggable, and easily installable (pip install saferl-lite) for education, experimentation, or safe deployment.
78
+
79
+ - Enables real-time SHAP or saliency visualization for Gym-based agents out-of-the-box.
80
+ ``` bash
81
+ SafeRL-Lite is the first minimal library to unify constraint satisfaction and explainability in reinforcement learning — without heavy dependencies or overhead.
82
+ ```
83
+
84
+ ## ✅ Our Contribution
85
+ 1. 🔐 Constraint Wrapper API: Drop-in Gym wrapper for defining and enforcing logical constraints on observations, actions, and reward signals.
86
+
87
+ 2. 🧠 Explainability Modules: Plug-and-play SHAP and saliency explainer classes for deep Q-networks.
88
+
89
+ 3. 📦 PyPI-Ready Toolkit: Easily installed, documented, and CI/CD tested; built for research and reproducibility.
90
+
91
+ 4. 📈 Metrics for Constraint Violation: Tracks unsafe episodes, per-step violations, and integrates cleanly with WandB or TensorBoard.
92
+
93
+ ## ✅ Technical Explanation
94
+ - We define a custom SafeEnvWrapper(gym.Env) that:
95
+
96
+ - Intercepts actions.
97
+
98
+ - Applies logical rules or thresholding.
99
+
100
+ - Optionally overrides rewards or terminations if constraints are violated.
101
+
102
+ - A ConstrainedDQNAgent uses:
103
+
104
+ - Safety-wrapped Gym envs.
105
+
106
+ - Standard Q-learning with optional penalty_on_violation flag.
107
+
108
+ - Post-training, the SHAPExplainer and SaliencyExplainer:
109
+
110
+ - Generate local attributions using input perturbations or gradient norms.
111
+
112
+ - Can visualize per-state or per-action explanations.
113
+
114
+
115
+
116
+ ## ✅ Satyam's Explanation
117
+ ```bash
118
+ Imagine you're teaching a robot to walk — but there’s lava on the floor!
119
+ You don’t just want it to learn fast, you want it to stay safe and explain why it stepped left, not right.
120
+ ```
121
+ SafeRL-Lite is like a safety helmet and voicebox for robots:
122
+
123
+ - The helmet makes sure they don’t do dangerous stuff.
124
+
125
+ - The voicebox lets them say why they made that move.
126
+
127
+ ## 🔧 Installation
128
+
129
+ > 📦 PyPI
130
+ ```bash
131
+ pip install saferl-lite
132
+ ```
133
+
134
+ ## 🛠️ From source:
135
+
136
+ ```bash
137
+ git clone https://github.com/satyamcser/saferl-lite.git
138
+ cd saferl-lite
139
+ pip install -e .
140
+ ```
141
+
142
+ ## 🚀 Quickstart
143
+ Train a constrained DQN agent with saliency-based explainability:
144
+
145
+ ```bash
146
+ python train.py --env CartPole-v1 --constraint pole_angle --explain shap
147
+ ```
148
+
149
+ 🔹 This:
150
+
151
+ - Adds a pole-angle constraint wrapper to the Gym env
152
+
153
+ - Logs violations
154
+
155
+ - Displays SHAP or saliency explanations for agent decisions
156
+
157
+ ## 🧠 Features
158
+ #### ✅ Constrained RL
159
+ - Add custom constraints via wrapper or logic class
160
+
161
+ - Violation logging and reward shaping
162
+
163
+ - Safe vs unsafe episode tracking
164
+
165
+ #### 🔍 Explainability
166
+ - SaliencyExplainer — gradient-based visual heatmaps
167
+
168
+ - SHAPExplainer — feature contribution values per decision
169
+
170
+ - Compatible with any PyTorch-based agent
171
+
172
+ #### 📊 Metrics
173
+ - Constraint violation rate
174
+
175
+ - Episode reward
176
+
177
+ - Cumulative safe reward
178
+
179
+ - Action entropy & temporal behavior stats
180
+
181
+ #### 📚 Modularity
182
+ - Swap out agents, constraints, evaluators, or explainers
183
+
184
+ - Supports Gym environments
185
+
186
+ - Configurable training pipeline
187
+
188
+ ## 📜 Citation
189
+ Coming soon after arXiv/preprint release.
@@ -59,7 +59,7 @@ This project is licensed under the **MIT License**. See the [LICENSE](https://gi
59
59
  ## 🌐 Links
60
60
 
61
61
  - GitHub: [https://github.com/satyamcser/saferl-lite](https://github.com/satyamcser/saferl-lite)
62
- - PyPI: *Coming soon*
62
+ - PyPI: [https://pypi.org/project/saferl-lite/](https://pypi.org/project/saferl-lite/)
63
63
  - Docs: *This site*
64
64
 
65
65
  ---
@@ -0,0 +1,241 @@
1
+ Metadata-Version: 2.4
2
+ Name: saferl-lite
3
+ Version: 0.1.3
4
+ Summary: A lightweight, explainable, and constrained reinforcement learning toolkit.
5
+ Home-page: https://github.com/satyamcser/saferl-lite
6
+ Author: Satyam Mishra
7
+ Author-email: satyam@example.com
8
+ Project-URL: Documentation, https://github.com/satyamcser/saferl-lite/tree/main/docs
9
+ Project-URL: Source, https://github.com/satyamcser/saferl-lite
10
+ Project-URL: Bug Tracker, https://github.com/satyamcser/saferl-lite/issues
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: License :: OSI Approved :: MIT License
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
16
+ Requires-Python: >=3.8
17
+ Description-Content-Type: text/markdown
18
+ License-File: LICENSE
19
+ Requires-Dist: gym
20
+ Requires-Dist: gymnasium
21
+ Requires-Dist: numpy
22
+ Requires-Dist: torch
23
+ Requires-Dist: matplotlib
24
+ Requires-Dist: seaborn
25
+ Requires-Dist: pre-commit
26
+ Requires-Dist: flake8
27
+ Requires-Dist: pyyaml
28
+ Requires-Dist: shap
29
+ Requires-Dist: captum
30
+ Requires-Dist: typer
31
+ Requires-Dist: scikit-learn
32
+ Requires-Dist: pandas
33
+ Requires-Dist: pytest
34
+ Requires-Dist: pytest-cov
35
+ Requires-Dist: coverage
36
+ Requires-Dist: mkdocs
37
+ Requires-Dist: wandb
38
+ Requires-Dist: mkdocs>=1.5
39
+ Requires-Dist: mkdocs-material>=9.5
40
+ Requires-Dist: mkdocstrings[python]
41
+ Dynamic: author
42
+ Dynamic: author-email
43
+ Dynamic: classifier
44
+ Dynamic: description
45
+ Dynamic: description-content-type
46
+ Dynamic: home-page
47
+ Dynamic: license-file
48
+ Dynamic: project-url
49
+ Dynamic: requires-dist
50
+ Dynamic: requires-python
51
+ Dynamic: summary
52
+
53
+ # 🔐 SafeRL-Lite
54
+
55
+ A **lightweight, explainable, and modular** Python library for **Constrained Reinforcement Learning (Safe RL)** with real-time **SHAP & saliency-based explainability**, custom metrics, and Gym-compatible wrappers.
56
+
57
+ **New update with ease to setup, install and import coming soon.
58
+
59
+ By:
60
+ - Satyam Mishra, Vision Mentors Ltd., Hanoi, Vietnam
61
+ - Shivam Mishra, Phung Thao Vi, Vietnam National University, Hanoi, Vietnam
62
+ - Dr. Vishwanath Bijalwan, SR University, Warangal, India
63
+ - Dr. Vijay Bhaskar Semwal, MANIT, Bhopal, India
64
+ - Dr. Abdul Manan Khan, University of West London, London, UK
65
+
66
+ <p align="center">
67
+ <a href="https://github.com/satyamcser/saferl-lite/blob/main/LICENSE">
68
+ <img src="https://img.shields.io/github/license/satyamcser/saferl-lite?style=flat-square" alt="License">
69
+ </a>
70
+ <a href="https://github.com/satyamcser/saferl-lite/stargazers">
71
+ <img src="https://img.shields.io/github/stars/satyamcser/saferl-lite?style=flat-square" alt="Stars">
72
+ </a>
73
+ <a href="https://pypi.org/project/saferl-lite/">
74
+ <img src="https://img.shields.io/pypi/v/saferl-lite?style=flat-square" alt="PyPI version">
75
+ </a>
76
+ <a href="https://github.com/satyamcser/saferl-lite/actions/workflows/ci.yml">
77
+ <img src="https://img.shields.io/github/actions/workflow/status/satyamcser/saferl-lite/ci.yml?branch=main&style=flat-square" alt="Build Status">
78
+ </a>
79
+ </p>
80
+
81
+
82
+ ---
83
+
84
+ ## 🌟 Overview
85
+
86
+ **SafeRL-Lite** empowers reinforcement learning agents to act under **safety constraints**, while remaining **interpretable** and **modular** for fast experimentation. It wraps standard Gym environments and DQN-based agents with:
87
+
88
+ - ✅ Safety constraint logic
89
+ - 🔍 Visual explainability (SHAP, saliency maps)
90
+ - 📊 Violation and reward tracking
91
+ - 🧪 Built-in testing and evaluations
92
+
93
+ ---
94
+
95
+ ## ✅ Problem We Solved
96
+
97
+ Modern Reinforcement Learning (RL) agents are powerful but unsafe and opaque:
98
+
99
+ - 🚫 They frequently violate safety constraints during learning or deployment (e.g., fall off a cliff in navigation tasks).
100
+
101
+ - 😕 Their decision-making is a black box: humans can’t understand why a certain action was chosen.
102
+
103
+ - 🔍 Standard RL libraries lack native support for:
104
+
105
+ - Enforcing hard constraints during training.
106
+
107
+ - Explaining decisions using methods like SHAP or saliency maps.
108
+
109
+ ## ✅ Our Solution
110
+ SafeRL-Lite is a lightweight Python library that:
111
+
112
+ 1. 📏 Adds a SafetyWrapper around any Gym environment to enforce safety constraints (e.g., bounding actions, limiting states).
113
+
114
+ 2. 🧠 Integrates explainability methods:
115
+
116
+ - SHAPExplainer (model-agnostic local explanations).
117
+
118
+ - SaliencyExplainer (gradient-based sensitivity maps).
119
+
120
+ 3. 🔧 Wraps Constrained DQNs with ease, enabling safety-compliant Q-learning.
121
+
122
+ 4. 📊 Offers built-in metrics like violation count and safe episode tracking.
123
+
124
+ ## ✅ Novelty
125
+ While Safe RL and Explainable RL are separately studied, no prior lightweight library:
126
+
127
+ - Combines hard safety constraints with post-hoc interpretability.
128
+
129
+ - Is designed to be minimal, pluggable, and easily installable (pip install saferl-lite) for education, experimentation, or safe deployment.
130
+
131
+ - Enables real-time SHAP or saliency visualization for Gym-based agents out-of-the-box.
132
+ ``` bash
133
+ SafeRL-Lite is the first minimal library to unify constraint satisfaction and explainability in reinforcement learning — without heavy dependencies or overhead.
134
+ ```
135
+
136
+ ## ✅ Our Contribution
137
+ 1. 🔐 Constraint Wrapper API: Drop-in Gym wrapper for defining and enforcing logical constraints on observations, actions, and reward signals.
138
+
139
+ 2. 🧠 Explainability Modules: Plug-and-play SHAP and saliency explainer classes for deep Q-networks.
140
+
141
+ 3. 📦 PyPI-Ready Toolkit: Easily installed, documented, and CI/CD tested; built for research and reproducibility.
142
+
143
+ 4. 📈 Metrics for Constraint Violation: Tracks unsafe episodes, per-step violations, and integrates cleanly with WandB or TensorBoard.
144
+
145
+ ## ✅ Technical Explanation
146
+ - We define a custom SafeEnvWrapper(gym.Env) that:
147
+
148
+ - Intercepts actions.
149
+
150
+ - Applies logical rules or thresholding.
151
+
152
+ - Optionally overrides rewards or terminations if constraints are violated.
153
+
154
+ - A ConstrainedDQNAgent uses:
155
+
156
+ - Safety-wrapped Gym envs.
157
+
158
+ - Standard Q-learning with optional penalty_on_violation flag.
159
+
160
+ - Post-training, the SHAPExplainer and SaliencyExplainer:
161
+
162
+ - Generate local attributions using input perturbations or gradient norms.
163
+
164
+ - Can visualize per-state or per-action explanations.
165
+
166
+
167
+
168
+ ## ✅ Satyam's Explanation
169
+ ```bash
170
+ Imagine you're teaching a robot to walk — but there’s lava on the floor!
171
+ You don’t just want it to learn fast, you want it to stay safe and explain why it stepped left, not right.
172
+ ```
173
+ SafeRL-Lite is like a safety helmet and voicebox for robots:
174
+
175
+ - The helmet makes sure they don’t do dangerous stuff.
176
+
177
+ - The voicebox lets them say why they made that move.
178
+
179
+ ## 🔧 Installation
180
+
181
+ > 📦 PyPI
182
+ ```bash
183
+ pip install saferl-lite
184
+ ```
185
+
186
+ ## 🛠️ From source:
187
+
188
+ ```bash
189
+ git clone https://github.com/satyamcser/saferl-lite.git
190
+ cd saferl-lite
191
+ pip install -e .
192
+ ```
193
+
194
+ ## 🚀 Quickstart
195
+ Train a constrained DQN agent with saliency-based explainability:
196
+
197
+ ```bash
198
+ python train.py --env CartPole-v1 --constraint pole_angle --explain shap
199
+ ```
200
+
201
+ 🔹 This:
202
+
203
+ - Adds a pole-angle constraint wrapper to the Gym env
204
+
205
+ - Logs violations
206
+
207
+ - Displays SHAP or saliency explanations for agent decisions
208
+
209
+ ## 🧠 Features
210
+ #### ✅ Constrained RL
211
+ - Add custom constraints via wrapper or logic class
212
+
213
+ - Violation logging and reward shaping
214
+
215
+ - Safe vs unsafe episode tracking
216
+
217
+ #### 🔍 Explainability
218
+ - SaliencyExplainer — gradient-based visual heatmaps
219
+
220
+ - SHAPExplainer — feature contribution values per decision
221
+
222
+ - Compatible with any PyTorch-based agent
223
+
224
+ #### 📊 Metrics
225
+ - Constraint violation rate
226
+
227
+ - Episode reward
228
+
229
+ - Cumulative safe reward
230
+
231
+ - Action entropy & temporal behavior stats
232
+
233
+ #### 📚 Modularity
234
+ - Swap out agents, constraints, evaluators, or explainers
235
+
236
+ - Supports Gym environments
237
+
238
+ - Configurable training pipeline
239
+
240
+ ## 📜 Citation
241
+ Coming soon after arXiv/preprint release.
@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
2
2
 
3
3
  setup(
4
4
  name="saferl-lite",
5
- version="0.1.0",
5
+ version="0.1.3",
6
6
  author="Satyam Mishra",
7
7
  author_email="satyam@example.com",
8
8
  description="A lightweight, explainable, and constrained reinforcement learning toolkit.",
@@ -10,7 +10,7 @@ setup(
10
10
  long_description_content_type="text/markdown",
11
11
  url="https://github.com/satyamcser/saferl-lite",
12
12
  project_urls={
13
- "Documentation": "https://satyamcser.github.io/saferl-lite/",
13
+ "Documentation": "https://github.com/satyamcser/saferl-lite/tree/main/docs",
14
14
  "Source": "https://github.com/satyamcser/saferl-lite",
15
15
  "Bug Tracker": "https://github.com/satyamcser/saferl-lite/issues",
16
16
  },
@@ -1,139 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: saferl-lite
3
- Version: 0.1.0
4
- Summary: A lightweight, explainable, and constrained reinforcement learning toolkit.
5
- Home-page: https://github.com/satyamcser/saferl-lite
6
- Author: Satyam Mishra
7
- Author-email: satyam@example.com
8
- Project-URL: Documentation, https://satyamcser.github.io/saferl-lite/
9
- Project-URL: Source, https://github.com/satyamcser/saferl-lite
10
- Project-URL: Bug Tracker, https://github.com/satyamcser/saferl-lite/issues
11
- Classifier: Programming Language :: Python :: 3
12
- Classifier: License :: OSI Approved :: MIT License
13
- Classifier: Operating System :: OS Independent
14
- Classifier: Intended Audience :: Science/Research
15
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
16
- Requires-Python: >=3.8
17
- Description-Content-Type: text/markdown
18
- License-File: LICENSE
19
- Requires-Dist: gym
20
- Requires-Dist: gymnasium
21
- Requires-Dist: numpy
22
- Requires-Dist: torch
23
- Requires-Dist: matplotlib
24
- Requires-Dist: seaborn
25
- Requires-Dist: pre-commit
26
- Requires-Dist: flake8
27
- Requires-Dist: pyyaml
28
- Requires-Dist: shap
29
- Requires-Dist: captum
30
- Requires-Dist: typer
31
- Requires-Dist: scikit-learn
32
- Requires-Dist: pandas
33
- Requires-Dist: pytest
34
- Requires-Dist: pytest-cov
35
- Requires-Dist: coverage
36
- Requires-Dist: mkdocs
37
- Requires-Dist: wandb
38
- Requires-Dist: mkdocs>=1.5
39
- Requires-Dist: mkdocs-material>=9.5
40
- Requires-Dist: mkdocstrings[python]
41
- Dynamic: author
42
- Dynamic: author-email
43
- Dynamic: classifier
44
- Dynamic: description
45
- Dynamic: description-content-type
46
- Dynamic: home-page
47
- Dynamic: license-file
48
- Dynamic: project-url
49
- Dynamic: requires-dist
50
- Dynamic: requires-python
51
- Dynamic: summary
52
-
53
- # 🔐 SafeRL-Lite
54
-
55
- A **lightweight, explainable, and modular** Python library for **Constrained Reinforcement Learning (Safe RL)** with real-time **SHAP & saliency-based explainability**, custom metrics, and Gym-compatible wrappers.
56
-
57
- <p align="center">
58
- <img src="https://img.shields.io/github/license/satyamcser/saferl-lite?style=flat-square">
59
- <img src="https://img.shields.io/github/stars/satyamcser/saferl-lite?style=flat-square">
60
- <img src="https://img.shields.io/pypi/v/saferl-lite?style=flat-square">
61
- <img src="https://img.shields.io/github/actions/workflow/status/satyamcser/saferl-lite/ci.yml?branch=main&style=flat-square">
62
- </p>
63
-
64
- ---
65
-
66
- ## 🌟 Overview
67
-
68
- **SafeRL-Lite** empowers reinforcement learning agents to act under **safety constraints**, while remaining **interpretable** and **modular** for fast experimentation. It wraps standard Gym environments and DQN-based agents with:
69
-
70
- - ✅ Safety constraint logic
71
- - 🔍 Visual explainability (SHAP, saliency maps)
72
- - 📊 Violation and reward tracking
73
- - 🧪 Built-in testing and evaluations
74
-
75
- ---
76
-
77
- ## 🔧 Installation
78
-
79
- > 📦 PyPI (coming soon)
80
- ```bash
81
- pip install saferl-lite
82
- ```
83
-
84
- ## 🛠️ From source:
85
-
86
- ```bash
87
- git clone https://github.com/satyamcser/saferl-lite.git
88
- cd saferl-lite
89
- pip install -e .
90
- ```
91
-
92
- ## 🚀 Quickstart
93
- Train a constrained DQN agent with saliency-based explainability:
94
-
95
- ```bash
96
- python train.py --env CartPole-v1 --constraint pole_angle --explain shap
97
- ```
98
-
99
- 🔹 This:
100
-
101
- - Adds a pole-angle constraint wrapper to the Gym env
102
-
103
- - Logs violations
104
-
105
- - Displays SHAP or saliency explanations for agent decisions
106
-
107
- ## 🧠 Features
108
- #### ✅ Constrained RL
109
- - Add custom constraints via wrapper or logic class
110
-
111
- - Violation logging and reward shaping
112
-
113
- - Safe vs unsafe episode tracking
114
-
115
- #### 🔍 Explainability
116
- - SaliencyExplainer — gradient-based visual heatmaps
117
-
118
- - SHAPExplainer — feature contribution values per decision
119
-
120
- - Compatible with any PyTorch-based agent
121
-
122
- #### 📊 Metrics
123
- - Constraint violation rate
124
-
125
- - Episode reward
126
-
127
- - Cumulative safe reward
128
-
129
- - Action entropy & temporal behavior stats
130
-
131
- #### 📚 Modularity
132
- - Swap out agents, constraints, evaluators, or explainers
133
-
134
- - Supports Gym environments
135
-
136
- - Configurable training pipeline
137
-
138
- ## 📜 Citation
139
- Coming soon after arXiv/preprint release.
@@ -1,87 +0,0 @@
1
- # 🔐 SafeRL-Lite
2
-
3
- A **lightweight, explainable, and modular** Python library for **Constrained Reinforcement Learning (Safe RL)** with real-time **SHAP & saliency-based explainability**, custom metrics, and Gym-compatible wrappers.
4
-
5
- <p align="center">
6
- <img src="https://img.shields.io/github/license/satyamcser/saferl-lite?style=flat-square">
7
- <img src="https://img.shields.io/github/stars/satyamcser/saferl-lite?style=flat-square">
8
- <img src="https://img.shields.io/pypi/v/saferl-lite?style=flat-square">
9
- <img src="https://img.shields.io/github/actions/workflow/status/satyamcser/saferl-lite/ci.yml?branch=main&style=flat-square">
10
- </p>
11
-
12
- ---
13
-
14
- ## 🌟 Overview
15
-
16
- **SafeRL-Lite** empowers reinforcement learning agents to act under **safety constraints**, while remaining **interpretable** and **modular** for fast experimentation. It wraps standard Gym environments and DQN-based agents with:
17
-
18
- - ✅ Safety constraint logic
19
- - 🔍 Visual explainability (SHAP, saliency maps)
20
- - 📊 Violation and reward tracking
21
- - 🧪 Built-in testing and evaluations
22
-
23
- ---
24
-
25
- ## 🔧 Installation
26
-
27
- > 📦 PyPI (coming soon)
28
- ```bash
29
- pip install saferl-lite
30
- ```
31
-
32
- ## 🛠️ From source:
33
-
34
- ```bash
35
- git clone https://github.com/satyamcser/saferl-lite.git
36
- cd saferl-lite
37
- pip install -e .
38
- ```
39
-
40
- ## 🚀 Quickstart
41
- Train a constrained DQN agent with saliency-based explainability:
42
-
43
- ```bash
44
- python train.py --env CartPole-v1 --constraint pole_angle --explain shap
45
- ```
46
-
47
- 🔹 This:
48
-
49
- - Adds a pole-angle constraint wrapper to the Gym env
50
-
51
- - Logs violations
52
-
53
- - Displays SHAP or saliency explanations for agent decisions
54
-
55
- ## 🧠 Features
56
- #### ✅ Constrained RL
57
- - Add custom constraints via wrapper or logic class
58
-
59
- - Violation logging and reward shaping
60
-
61
- - Safe vs unsafe episode tracking
62
-
63
- #### 🔍 Explainability
64
- - SaliencyExplainer — gradient-based visual heatmaps
65
-
66
- - SHAPExplainer — feature contribution values per decision
67
-
68
- - Compatible with any PyTorch-based agent
69
-
70
- #### 📊 Metrics
71
- - Constraint violation rate
72
-
73
- - Episode reward
74
-
75
- - Cumulative safe reward
76
-
77
- - Action entropy & temporal behavior stats
78
-
79
- #### 📚 Modularity
80
- - Swap out agents, constraints, evaluators, or explainers
81
-
82
- - Supports Gym environments
83
-
84
- - Configurable training pipeline
85
-
86
- ## 📜 Citation
87
- Coming soon after arXiv/preprint release.
@@ -1,139 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: saferl-lite
3
- Version: 0.1.0
4
- Summary: A lightweight, explainable, and constrained reinforcement learning toolkit.
5
- Home-page: https://github.com/satyamcser/saferl-lite
6
- Author: Satyam Mishra
7
- Author-email: satyam@example.com
8
- Project-URL: Documentation, https://satyamcser.github.io/saferl-lite/
9
- Project-URL: Source, https://github.com/satyamcser/saferl-lite
10
- Project-URL: Bug Tracker, https://github.com/satyamcser/saferl-lite/issues
11
- Classifier: Programming Language :: Python :: 3
12
- Classifier: License :: OSI Approved :: MIT License
13
- Classifier: Operating System :: OS Independent
14
- Classifier: Intended Audience :: Science/Research
15
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
16
- Requires-Python: >=3.8
17
- Description-Content-Type: text/markdown
18
- License-File: LICENSE
19
- Requires-Dist: gym
20
- Requires-Dist: gymnasium
21
- Requires-Dist: numpy
22
- Requires-Dist: torch
23
- Requires-Dist: matplotlib
24
- Requires-Dist: seaborn
25
- Requires-Dist: pre-commit
26
- Requires-Dist: flake8
27
- Requires-Dist: pyyaml
28
- Requires-Dist: shap
29
- Requires-Dist: captum
30
- Requires-Dist: typer
31
- Requires-Dist: scikit-learn
32
- Requires-Dist: pandas
33
- Requires-Dist: pytest
34
- Requires-Dist: pytest-cov
35
- Requires-Dist: coverage
36
- Requires-Dist: mkdocs
37
- Requires-Dist: wandb
38
- Requires-Dist: mkdocs>=1.5
39
- Requires-Dist: mkdocs-material>=9.5
40
- Requires-Dist: mkdocstrings[python]
41
- Dynamic: author
42
- Dynamic: author-email
43
- Dynamic: classifier
44
- Dynamic: description
45
- Dynamic: description-content-type
46
- Dynamic: home-page
47
- Dynamic: license-file
48
- Dynamic: project-url
49
- Dynamic: requires-dist
50
- Dynamic: requires-python
51
- Dynamic: summary
52
-
53
- # 🔐 SafeRL-Lite
54
-
55
- A **lightweight, explainable, and modular** Python library for **Constrained Reinforcement Learning (Safe RL)** with real-time **SHAP & saliency-based explainability**, custom metrics, and Gym-compatible wrappers.
56
-
57
- <p align="center">
58
- <img src="https://img.shields.io/github/license/satyamcser/saferl-lite?style=flat-square">
59
- <img src="https://img.shields.io/github/stars/satyamcser/saferl-lite?style=flat-square">
60
- <img src="https://img.shields.io/pypi/v/saferl-lite?style=flat-square">
61
- <img src="https://img.shields.io/github/actions/workflow/status/satyamcser/saferl-lite/ci.yml?branch=main&style=flat-square">
62
- </p>
63
-
64
- ---
65
-
66
- ## 🌟 Overview
67
-
68
- **SafeRL-Lite** empowers reinforcement learning agents to act under **safety constraints**, while remaining **interpretable** and **modular** for fast experimentation. It wraps standard Gym environments and DQN-based agents with:
69
-
70
- - ✅ Safety constraint logic
71
- - 🔍 Visual explainability (SHAP, saliency maps)
72
- - 📊 Violation and reward tracking
73
- - 🧪 Built-in testing and evaluations
74
-
75
- ---
76
-
77
- ## 🔧 Installation
78
-
79
- > 📦 PyPI (coming soon)
80
- ```bash
81
- pip install saferl-lite
82
- ```
83
-
84
- ## 🛠️ From source:
85
-
86
- ```bash
87
- git clone https://github.com/satyamcser/saferl-lite.git
88
- cd saferl-lite
89
- pip install -e .
90
- ```
91
-
92
- ## 🚀 Quickstart
93
- Train a constrained DQN agent with saliency-based explainability:
94
-
95
- ```bash
96
- python train.py --env CartPole-v1 --constraint pole_angle --explain shap
97
- ```
98
-
99
- 🔹 This:
100
-
101
- - Adds a pole-angle constraint wrapper to the Gym env
102
-
103
- - Logs violations
104
-
105
- - Displays SHAP or saliency explanations for agent decisions
106
-
107
- ## 🧠 Features
108
- #### ✅ Constrained RL
109
- - Add custom constraints via wrapper or logic class
110
-
111
- - Violation logging and reward shaping
112
-
113
- - Safe vs unsafe episode tracking
114
-
115
- #### 🔍 Explainability
116
- - SaliencyExplainer — gradient-based visual heatmaps
117
-
118
- - SHAPExplainer — feature contribution values per decision
119
-
120
- - Compatible with any PyTorch-based agent
121
-
122
- #### 📊 Metrics
123
- - Constraint violation rate
124
-
125
- - Episode reward
126
-
127
- - Cumulative safe reward
128
-
129
- - Action entropy & temporal behavior stats
130
-
131
- #### 📚 Modularity
132
- - Swap out agents, constraints, evaluators, or explainers
133
-
134
- - Supports Gym environments
135
-
136
- - Configurable training pipeline
137
-
138
- ## 📜 Citation
139
- Coming soon after arXiv/preprint release.
File without changes
File without changes
File without changes
File without changes