lambda-guard-boosting 0.2.2__tar.gz

lambda_guard_boosting-0.2.2/LICENSE.md

@@ -0,0 +1,34 @@
+![License](https://img.shields.io/badge/License-MIT-blue.svg)
+© 2026 **Fabrizio Di Sciorio, PhD**
+
+---
+
+Permission is hereby granted, free of charge, to any person obtaining a copy  
+of this software and associated documentation files (the "Software"), to **deal in the Software without restriction**, including without limitation the rights to:
+
+- ✅ Use  
+- ✅ Copy  
+- ✅ Modify  
+- ✅ Merge  
+- ✅ Publish  
+- ✅ Distribute  
+- ✅ Sublicense  
+- ✅ Sell copies of the Software  
+
+and to permit persons to whom the Software is furnished to do so, **subject to the following conditions**:
+
+---
+
+The above copyright notice and this permission notice shall be included in **all copies or substantial portions of the Software**.
+
+---
+
+### ⚠️ Disclaimer
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF:
+
+- MERCHANTABILITY  
+- FITNESS FOR A PARTICULAR PURPOSE  
+- NONINFRINGEMENT  
+
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

lambda_guard_boosting-0.2.2/PKG-INFO

@@ -0,0 +1,229 @@
+Metadata-Version: 2.4
+Name: lambda-guard-boosting
+Version: 0.2.2
+Summary: Overfitting detection for Gradient Boosting models using λ-Guard methodology.
+Author-email: "Fabrizio Di Sciorio, PhD" <fabriziodisciorio91@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/faberBI/lambdaguard
+Project-URL: Documentation, https://github.com/faberBI/lambdaguard
+Project-URL: BugTracker, https://github.com/faberBI/lambdaguard/issues
+Keywords: machine-learning,gradient-boosting,overfitting,boosting,lambda-guard
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: numpy<2.2,>=1.26
+Requires-Dist: pandas<3.0,>=2.2
+Requires-Dist: scikit-learn<2.0,>=1.3
+Requires-Dist: matplotlib<4.0,>=3.8
+Requires-Dist: seaborn<0.14,>=0.12
+Requires-Dist: xgboost<4.0,>=1.7
+Requires-Dist: lightgbm<5.0,>=4.4
+Requires-Dist: catboost<2.0,>=1.1
+Dynamic: license-file
+
+<p align="center">
+  <img src="docs/logo.png" alt="λ-Guard" width="160"/>
+</p>
+
+<p align="center">
+<strong>Overfitting detection for Gradient Boosting</strong> — <em>no validation set required</em><br>
+<i>Detect the moment when your model stops learning signal and starts memorizing structure.</i>
+</p>
+
+<p align="center">
+  <a href="https://github.com/faberBI/lambdaguard/actions/workflows/tests.yml">
+    <img src="https://img.shields.io/github/actions/workflow/status/faberBI/lambdaguard/tests.yml?branch=main&logo=github" alt="Tests Status">
+  </a>
+  <a href="https://coveralls.io/github/faberBI/lambdaguard">
+    <img src="https://img.shields.io/coveralls/github/faberBI/lambdaguard/main.svg" alt="Coverage Status">
+  </a>
+  <a href="https://pypi.org/project/lambdaguard/">
+    <img src="https://img.shields.io/pypi/v/lambdaguard?logo=python" alt="PyPI Version">
+  </a>
+  <a href="https://opensource.org/licenses/MIT">
+    <img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License MIT">
+  </a>
+</p>
+
+---
+
+## ❓ Why λ-Guard?
+
+In Gradient Boosting, overfitting often appears **before the validation error rises**.  
+By that point, the model is already:
+
+- ✂️ Splitting features into extremely fine regions  
+- 🍃 Fitting leaves supported by very few observations  
+- 🌪 Sensitive to tiny perturbations  
+
+It’s **no longer improving predictions**, it’s **memorizing the training dataset**.  
+**λ-Guard detects that moment automatically.**
+
+---
+
+## 🧠 Core Intuition
+
+A boosting model learns two things simultaneously:
+
+| Component | Role |
+|-----------|------|
+| Geometry  | partitions the feature space |
+| Predictor | assigns values to each region |
+
+Overfitting occurs when:
+
+*"Geometry keeps growing, but predictor stops extracting real information."*
+
+λ-Guard measures three key signals:
+
+- 📦 **Capacity** → structural complexity  
+- 🎯 **Alignment** → extracted signal  
+- 🌊 **Stability** → fragility of predictions
+
+---
+
+## 🧩 Representation Matrix
+
+Every tree divides the feature space into **leaves**.  
+We record where each observation falls:
+Z[i,j] = 1 if sample i falls in leaf j
+Z[i,j] = 0 otherwise
+
+- Rows → observations  
+- Columns → leaves across all trees  
+
+Think of **Z** as the **representation learned by the ensemble**.
+
+- Linear regression → hat matrix **H**  
+- Boosting → representation **Z**
+
+---
+
+## 📦 Capacity — Structural Complexity
+
+- 🔹 Low C → few effective regions  
+- 🔹 High C → model fragments space  
+
+Late-stage boosting **increases C quickly**, often without improving predictions.
+
+---
+
+## 🎯 Alignment — Useful Information
+
+- 🔹 High A → trees add real predictive signal  
+- 🔹 Low A → trees mostly refine boundaries  
+
+*"After some trees, alignment saturates."*  
+Boosting continues **growing structure** even if prediction stops improving.
+
+---
+
+## 🌊 Stability — Sensitivity to Perturbations
+
+- 🔹 Low S → smooth, robust model  
+- 🔹 High S → brittle, sensitive model  
+
+**Stability is the first signal to explode during overfitting.**
+
+---
+
+## 🔥 The Overfitting Index λ
+
+| Situation | λ |
+|-----------|---|
+| Compact structure + stable predictions | low |
+| Many regions + weak signal | high |
+| Unstable predictions | very high |
+
+**Interpretation:** measures how much structural complexity is wasted.  
+Normalized λ ∈ [0,1] can be used to **compare models**.
+
+
+## 🧪 Structural Overfitting Test
+
+Detect if a few training points dominate the model using **approximate leverage**:
+H_ii ≈ Σ_trees (learning_rate / leaf_size)
+T1 = mean(H_ii) # global complexity
+T2 = max(H_ii)/mean(H_ii) # local memorization
+
+
+**Bootstrap procedure:**
+
+1. Repeat B times: resample training data, recompute T1 & T2  
+2. Compute p-values:  
+   - p1 = P(T1_boot ≥ T1_obs)  
+   - p2 = P(T2_boot ≥ T2_obs)  
+
+Reject structural stability if:
+
+p1 < α OR p2 < α
+
+
+---
+
+## 📊 What λ-Guard Distinguishes
+
+| Regime | Meaning |
+|--------|---------|
+| ✅ Stable | smooth generalization |
+| 📈 Global overfitting | too many effective parameters |
+| ⚠️ Local memorization | few points dominate |
+| 💥 Extreme | interpolation behavior |
+
+---
+
+## 🧭 When to Use
+
+- Monitor boosting during training  
+- Hyperparameter tuning  
+- Small datasets (no validation split)  
+- Diagnose late-stage performance collapse
+
+---
+
+## ⚙️ Installation
+
+Install via GitHub:
+
+```bash
+pip install git+https://github.com/faberBI/lambdaguard.git
+
+from sklearn.ensemble import GradientBoostingRegressor
+from lambdaguard.ofi import overfitting_index
+from lambdaguard.lambda_guard import lambda_guard_test, interpret
+from lambdaguard.cusum import detect_structural_overfitting_cusum_robust
+import pandas as pd
+
+# Fit a model
+model = GradientBoostingRegressor(n_estimators=50, max_depth=3)
+model.fit(X_train, y_train)
+
+# Compute Overfitting Index
+ofi_res = overfitting_index(model, X_train, y_train)
+
+# Lambda-guard test
+lg_res = lambda_guard_test(model, X_train)
+print(interpret(lg_res))
+
+# CUSUM-based detection
+df = pd.DataFrame([
+    {"model": "GBR", "n_estimators": 50, "max_depth": 3, "A": 0.8, "OFI_norm": 0.2},
+    {"model": "GBR", "n_estimators": 100, "max_depth": 5, "A": 0.85, "OFI_norm": 0.3},
+])
+cusum_res = detect_structural_overfitting_cusum_robust(df, model_name="GBR")
+
+```
+
+## 📜 Citation
+
+If you use **λ-Guard** in your research or projects, please cite the following:
+
+**Fabrizio Di Sciorio, PhD**  
+*Universidad de Almeria — Business and Economics Department*  
+> "λ-Guard: Structural Overfitting Detection for Gradient Boosting Models"  
+

lambda_guard_boosting-0.2.2/README.md

@@ -0,0 +1,201 @@
+<p align="center">
+  <img src="docs/logo.png" alt="λ-Guard" width="160"/>
+</p>
+
+<p align="center">
+<strong>Overfitting detection for Gradient Boosting</strong> — <em>no validation set required</em><br>
+<i>Detect the moment when your model stops learning signal and starts memorizing structure.</i>
+</p>
+
+<p align="center">
+  <a href="https://github.com/faberBI/lambdaguard/actions/workflows/tests.yml">
+    <img src="https://img.shields.io/github/actions/workflow/status/faberBI/lambdaguard/tests.yml?branch=main&logo=github" alt="Tests Status">
+  </a>
+  <a href="https://coveralls.io/github/faberBI/lambdaguard">
+    <img src="https://img.shields.io/coveralls/github/faberBI/lambdaguard/main.svg" alt="Coverage Status">
+  </a>
+  <a href="https://pypi.org/project/lambdaguard/">
+    <img src="https://img.shields.io/pypi/v/lambdaguard?logo=python" alt="PyPI Version">
+  </a>
+  <a href="https://opensource.org/licenses/MIT">
+    <img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License MIT">
+  </a>
+</p>
+
+---
+
+## ❓ Why λ-Guard?
+
+In Gradient Boosting, overfitting often appears **before the validation error rises**.  
+By that point, the model is already:
+
+- ✂️ Splitting features into extremely fine regions  
+- 🍃 Fitting leaves supported by very few observations  
+- 🌪 Sensitive to tiny perturbations  
+
+It’s **no longer improving predictions**, it’s **memorizing the training dataset**.  
+**λ-Guard detects that moment automatically.**
+
+---
+
+## 🧠 Core Intuition
+
+A boosting model learns two things simultaneously:
+
+| Component | Role |
+|-----------|------|
+| Geometry  | partitions the feature space |
+| Predictor | assigns values to each region |
+
+Overfitting occurs when:
+
+*"Geometry keeps growing, but predictor stops extracting real information."*
+
+λ-Guard measures three key signals:
+
+- 📦 **Capacity** → structural complexity  
+- 🎯 **Alignment** → extracted signal  
+- 🌊 **Stability** → fragility of predictions
+
+---
+
+## 🧩 Representation Matrix
+
+Every tree divides the feature space into **leaves**.  
+We record where each observation falls:
+Z[i,j] = 1 if sample i falls in leaf j
+Z[i,j] = 0 otherwise
+
+- Rows → observations  
+- Columns → leaves across all trees  
+
+Think of **Z** as the **representation learned by the ensemble**.
+
+- Linear regression → hat matrix **H**  
+- Boosting → representation **Z**
+
+---
+
+## 📦 Capacity — Structural Complexity
+
+- 🔹 Low C → few effective regions  
+- 🔹 High C → model fragments space  
+
+Late-stage boosting **increases C quickly**, often without improving predictions.
+
+---
+
+## 🎯 Alignment — Useful Information
+
+- 🔹 High A → trees add real predictive signal  
+- 🔹 Low A → trees mostly refine boundaries  
+
+*"After some trees, alignment saturates."*  
+Boosting continues **growing structure** even if prediction stops improving.
+
+---
+
+## 🌊 Stability — Sensitivity to Perturbations
+
+- 🔹 Low S → smooth, robust model  
+- 🔹 High S → brittle, sensitive model  
+
+**Stability is the first signal to explode during overfitting.**
+
+---
+
+## 🔥 The Overfitting Index λ
+
+| Situation | λ |
+|-----------|---|
+| Compact structure + stable predictions | low |
+| Many regions + weak signal | high |
+| Unstable predictions | very high |
+
+**Interpretation:** measures how much structural complexity is wasted.  
+Normalized λ ∈ [0,1] can be used to **compare models**.
+
+
+## 🧪 Structural Overfitting Test
+
+Detect if a few training points dominate the model using **approximate leverage**:
+H_ii ≈ Σ_trees (learning_rate / leaf_size)
+T1 = mean(H_ii) # global complexity
+T2 = max(H_ii)/mean(H_ii) # local memorization
+
+
+**Bootstrap procedure:**
+
+1. Repeat B times: resample training data, recompute T1 & T2  
+2. Compute p-values:  
+   - p1 = P(T1_boot ≥ T1_obs)  
+   - p2 = P(T2_boot ≥ T2_obs)  
+
+Reject structural stability if:
+
+p1 < α OR p2 < α
+
+
+---
+
+## 📊 What λ-Guard Distinguishes
+
+| Regime | Meaning |
+|--------|---------|
+| ✅ Stable | smooth generalization |
+| 📈 Global overfitting | too many effective parameters |
+| ⚠️ Local memorization | few points dominate |
+| 💥 Extreme | interpolation behavior |
+
+---
+
+## 🧭 When to Use
+
+- Monitor boosting during training  
+- Hyperparameter tuning  
+- Small datasets (no validation split)  
+- Diagnose late-stage performance collapse
+
+---
+
+## ⚙️ Installation
+
+Install via GitHub:
+
+```bash
+pip install git+https://github.com/faberBI/lambdaguard.git
+
+from sklearn.ensemble import GradientBoostingRegressor
+from lambdaguard.ofi import overfitting_index
+from lambdaguard.lambda_guard import lambda_guard_test, interpret
+from lambdaguard.cusum import detect_structural_overfitting_cusum_robust
+import pandas as pd
+
+# Fit a model
+model = GradientBoostingRegressor(n_estimators=50, max_depth=3)
+model.fit(X_train, y_train)
+
+# Compute Overfitting Index
+ofi_res = overfitting_index(model, X_train, y_train)
+
+# Lambda-guard test
+lg_res = lambda_guard_test(model, X_train)
+print(interpret(lg_res))
+
+# CUSUM-based detection
+df = pd.DataFrame([
+    {"model": "GBR", "n_estimators": 50, "max_depth": 3, "A": 0.8, "OFI_norm": 0.2},
+    {"model": "GBR", "n_estimators": 100, "max_depth": 5, "A": 0.85, "OFI_norm": 0.3},
+])
+cusum_res = detect_structural_overfitting_cusum_robust(df, model_name="GBR")
+
+```
+
+## 📜 Citation
+
+If you use **λ-Guard** in your research or projects, please cite the following:
+
+**Fabrizio Di Sciorio, PhD**  
+*Universidad de Almeria — Business and Economics Department*  
+> "λ-Guard: Structural Overfitting Detection for Gradient Boosting Models"  
+

lambda_guard_boosting-0.2.2/lambda_guard_boosting.egg-info/PKG-INFO

@@ -0,0 +1,229 @@
+Metadata-Version: 2.4
+Name: lambda-guard-boosting
+Version: 0.2.2
+Summary: Overfitting detection for Gradient Boosting models using λ-Guard methodology.
+Author-email: "Fabrizio Di Sciorio, PhD" <fabriziodisciorio91@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/faberBI/lambdaguard
+Project-URL: Documentation, https://github.com/faberBI/lambdaguard
+Project-URL: BugTracker, https://github.com/faberBI/lambdaguard/issues
+Keywords: machine-learning,gradient-boosting,overfitting,boosting,lambda-guard
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: numpy<2.2,>=1.26
+Requires-Dist: pandas<3.0,>=2.2
+Requires-Dist: scikit-learn<2.0,>=1.3
+Requires-Dist: matplotlib<4.0,>=3.8
+Requires-Dist: seaborn<0.14,>=0.12
+Requires-Dist: xgboost<4.0,>=1.7
+Requires-Dist: lightgbm<5.0,>=4.4
+Requires-Dist: catboost<2.0,>=1.1
+Dynamic: license-file
+
+<p align="center">
+  <img src="docs/logo.png" alt="λ-Guard" width="160"/>
+</p>
+
+<p align="center">
+<strong>Overfitting detection for Gradient Boosting</strong> — <em>no validation set required</em><br>
+<i>Detect the moment when your model stops learning signal and starts memorizing structure.</i>
+</p>
+
+<p align="center">
+  <a href="https://github.com/faberBI/lambdaguard/actions/workflows/tests.yml">
+    <img src="https://img.shields.io/github/actions/workflow/status/faberBI/lambdaguard/tests.yml?branch=main&logo=github" alt="Tests Status">
+  </a>
+  <a href="https://coveralls.io/github/faberBI/lambdaguard">
+    <img src="https://img.shields.io/coveralls/github/faberBI/lambdaguard/main.svg" alt="Coverage Status">
+  </a>
+  <a href="https://pypi.org/project/lambdaguard/">
+    <img src="https://img.shields.io/pypi/v/lambdaguard?logo=python" alt="PyPI Version">
+  </a>
+  <a href="https://opensource.org/licenses/MIT">
+    <img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License MIT">
+  </a>
+</p>
+
+---
+
+## ❓ Why λ-Guard?
+
+In Gradient Boosting, overfitting often appears **before the validation error rises**.  
+By that point, the model is already:
+
+- ✂️ Splitting features into extremely fine regions  
+- 🍃 Fitting leaves supported by very few observations  
+- 🌪 Sensitive to tiny perturbations  
+
+It’s **no longer improving predictions**, it’s **memorizing the training dataset**.  
+**λ-Guard detects that moment automatically.**
+
+---
+
+## 🧠 Core Intuition
+
+A boosting model learns two things simultaneously:
+
+| Component | Role |
+|-----------|------|
+| Geometry  | partitions the feature space |
+| Predictor | assigns values to each region |
+
+Overfitting occurs when:
+
+*"Geometry keeps growing, but predictor stops extracting real information."*
+
+λ-Guard measures three key signals:
+
+- 📦 **Capacity** → structural complexity  
+- 🎯 **Alignment** → extracted signal  
+- 🌊 **Stability** → fragility of predictions
+
+---
+
+## 🧩 Representation Matrix
+
+Every tree divides the feature space into **leaves**.  
+We record where each observation falls:
+Z[i,j] = 1 if sample i falls in leaf j
+Z[i,j] = 0 otherwise
+
+- Rows → observations  
+- Columns → leaves across all trees  
+
+Think of **Z** as the **representation learned by the ensemble**.
+
+- Linear regression → hat matrix **H**  
+- Boosting → representation **Z**
+
+---
+
+## 📦 Capacity — Structural Complexity
+
+- 🔹 Low C → few effective regions  
+- 🔹 High C → model fragments space  
+
+Late-stage boosting **increases C quickly**, often without improving predictions.
+
+---
+
+## 🎯 Alignment — Useful Information
+
+- 🔹 High A → trees add real predictive signal  
+- 🔹 Low A → trees mostly refine boundaries  
+
+*"After some trees, alignment saturates."*  
+Boosting continues **growing structure** even if prediction stops improving.
+
+---
+
+## 🌊 Stability — Sensitivity to Perturbations
+
+- 🔹 Low S → smooth, robust model  
+- 🔹 High S → brittle, sensitive model  
+
+**Stability is the first signal to explode during overfitting.**
+
+---
+
+## 🔥 The Overfitting Index λ
+
+| Situation | λ |
+|-----------|---|
+| Compact structure + stable predictions | low |
+| Many regions + weak signal | high |
+| Unstable predictions | very high |
+
+**Interpretation:** measures how much structural complexity is wasted.  
+Normalized λ ∈ [0,1] can be used to **compare models**.
+
+
+## 🧪 Structural Overfitting Test
+
+Detect if a few training points dominate the model using **approximate leverage**:
+H_ii ≈ Σ_trees (learning_rate / leaf_size)
+T1 = mean(H_ii) # global complexity
+T2 = max(H_ii)/mean(H_ii) # local memorization
+
+
+**Bootstrap procedure:**
+
+1. Repeat B times: resample training data, recompute T1 & T2  
+2. Compute p-values:  
+   - p1 = P(T1_boot ≥ T1_obs)  
+   - p2 = P(T2_boot ≥ T2_obs)  
+
+Reject structural stability if:
+
+p1 < α OR p2 < α
+
+
+---
+
+## 📊 What λ-Guard Distinguishes
+
+| Regime | Meaning |
+|--------|---------|
+| ✅ Stable | smooth generalization |
+| 📈 Global overfitting | too many effective parameters |
+| ⚠️ Local memorization | few points dominate |
+| 💥 Extreme | interpolation behavior |
+
+---
+
+## 🧭 When to Use
+
+- Monitor boosting during training  
+- Hyperparameter tuning  
+- Small datasets (no validation split)  
+- Diagnose late-stage performance collapse
+
+---
+
+## ⚙️ Installation
+
+Install via GitHub:
+
+```bash
+pip install git+https://github.com/faberBI/lambdaguard.git
+
+from sklearn.ensemble import GradientBoostingRegressor
+from lambdaguard.ofi import overfitting_index
+from lambdaguard.lambda_guard import lambda_guard_test, interpret
+from lambdaguard.cusum import detect_structural_overfitting_cusum_robust
+import pandas as pd
+
+# Fit a model
+model = GradientBoostingRegressor(n_estimators=50, max_depth=3)
+model.fit(X_train, y_train)
+
+# Compute Overfitting Index
+ofi_res = overfitting_index(model, X_train, y_train)
+
+# Lambda-guard test
+lg_res = lambda_guard_test(model, X_train)
+print(interpret(lg_res))
+
+# CUSUM-based detection
+df = pd.DataFrame([
+    {"model": "GBR", "n_estimators": 50, "max_depth": 3, "A": 0.8, "OFI_norm": 0.2},
+    {"model": "GBR", "n_estimators": 100, "max_depth": 5, "A": 0.85, "OFI_norm": 0.3},
+])
+cusum_res = detect_structural_overfitting_cusum_robust(df, model_name="GBR")
+
+```
+
+## 📜 Citation
+
+If you use **λ-Guard** in your research or projects, please cite the following:
+
+**Fabrizio Di Sciorio, PhD**  
+*Universidad de Almeria — Business and Economics Department*  
+> "λ-Guard: Structural Overfitting Detection for Gradient Boosting Models"  
+

lambda_guard_boosting-0.2.2/lambda_guard_boosting.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+LICENSE.md
+README.md
+pyproject.toml
+lambda_guard_boosting.egg-info/PKG-INFO
+lambda_guard_boosting.egg-info/SOURCES.txt
+lambda_guard_boosting.egg-info/dependency_links.txt
+lambda_guard_boosting.egg-info/requires.txt
+lambda_guard_boosting.egg-info/top_level.txt
+lambdaguard/__init__.py
+lambdaguard/cusum.py
+lambdaguard/lambdaguard.py
+lambdaguard/ofi.py
+tests/test_cusum.py
+tests/test_ofi.py

lambda_guard_boosting-0.2.2/lambda_guard_boosting.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

lambda_guard_boosting-0.2.2/lambda_guard_boosting.egg-info/requires.txt

@@ -0,0 +1,8 @@
+numpy<2.2,>=1.26
+pandas<3.0,>=2.2
+scikit-learn<2.0,>=1.3
+matplotlib<4.0,>=3.8
+seaborn<0.14,>=0.12
+xgboost<4.0,>=1.7
+lightgbm<5.0,>=4.4
+catboost<2.0,>=1.1

lambda_guard_boosting-0.2.2/lambda_guard_boosting.egg-info/top_level.txt

@@ -0,0 +1,2 @@
+dist
+lambdaguard

lambda_guard_boosting-0.2.2/lambdaguard/__init__.py

@@ -0,0 +1,18 @@
+__version__ = "0.2.2"
+
+from .ofi import generalization_index, instability_index, create_model, run_experiment_multi_model, plot_all_multi_model, regression_test
+from .lambda_guard import lambda_guard_test, boosting_leverage, interpret
+from .cusum import lambda_detect
+
+__all__ = [
+    "generalization_index",
+    "instability_index",
+    "create_model",
+    "run_experiment_multi_model",
+    "plot_all_multi_model",
+    "regression_test",
+    "lambda_guard_test",
+    "boosting_leverage",
+    "interpret",
+    "lambda_detect"
+]

lambda_guard_boosting-0.2.2/lambdaguard/cusum.py

@@ -0,0 +1,92 @@
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+
+def lambda_detect(
+    df,
+    model_name,
+    complexity_metric="combined",
+    lambda_col="OFI_norm",
+    alignment_col="A",
+    smooth_window=3,
+    cusum_threshold_factor=1.5,
+    baseline_points=10
+):
+    df_model = df[df["model"] == model_name].copy()
+    
+    if complexity_metric == "combined":
+        df_model["complexity"] = df_model["n_estimators"] * df_model["max_depth"]
+    else:
+        df_model["complexity"] = df_model[complexity_metric]
+
+    df_model = df_model.sort_values("complexity").reset_index(drop=True)
+
+    lambdas = df_model[lambda_col].values
+    alignment = df_model[alignment_col].values
+    complexity = df_model["complexity"].values
+
+    # --- Normalizzazione z-score su baseline ---
+    lambda_baseline = lambdas[:baseline_points]
+    mu_lambda = np.mean(lambda_baseline)
+    sigma_lambda = np.std(lambda_baseline) + 1e-8
+    lambda_z = (lambdas - mu_lambda) / sigma_lambda
+
+    # --- Derivate ---
+    delta_lambda = np.diff(lambda_z)
+    delta_lambda = pd.Series(delta_lambda).rolling(smooth_window, min_periods=1).mean().values
+
+    delta2_lambda = np.diff(delta_lambda)
+    delta2_lambda = pd.Series(delta2_lambda).rolling(smooth_window, min_periods=1).mean().values
+
+    # --- CUSUM cumulativo ---
+    mean_d2 = np.mean(delta2_lambda)
+    std_d2 = np.std(delta2_lambda) + 1e-8
+    centered_d2 = delta2_lambda - mean_d2
+
+    cusum = np.zeros_like(centered_d2)
+    for i in range(1, len(centered_d2)):
+        cusum[i] = max(0, cusum[i-1] + centered_d2[i])
+
+    cusum_threshold = cusum_threshold_factor * std_d2
+
+    # --- Individuazione del change point ---
+    change_index = None
+    delta_alignment = np.diff(alignment)
+    for i, val in enumerate(cusum):
+        align_flat = delta_alignment[i] < 0.01 if i < len(delta_alignment) else False
+        if val > cusum_threshold and align_flat:
+            change_index = i + 2
+            break
+
+    # --- Miglior modello prima dell'overfitting basato su OFI minimo ---
+    best_model_before_overfitting = None
+    if change_index is not None and change_index > 0:
+        pre_overfit_df = df_model.iloc[:change_index]
+        best_row = pre_overfit_df.loc[pre_overfit_df[lambda_col].idxmin()]
+        best_model_before_overfitting = (
+            f"{best_row['model']} | {best_row['dataset']} | "
+            f"min_samples_leaf={best_row['min_samples_leaf']} | "
+            f"n_estimators={best_row['n_estimators']} | "
+            f"max_depth={best_row['max_depth']}"
+        )
+
+    # --- Plot λ_z ---
+    plt.figure(figsize=(10,5))
+    plt.plot(complexity, lambda_z, '-o', label='λ_z (normalized)', color='tab:blue')
+    if change_index is not None:
+        plt.axvline(complexity[change_index], color='red', linestyle='--', label='Change Point')
+        plt.scatter(complexity[change_index], lambda_z[change_index], color='red', s=100)
+    plt.xlabel("Complexity (n_estimators*max_depth)")
+    plt.ylabel("λ_z (normalized)")
+    plt.title(f"Structural Overfitting Detection - {model_name}")
+    plt.grid(True)
+    plt.legend()
+    plt.show()
+
+    return {
+        "overfitting_detected": change_index is not None,
+        "change_index": change_index,
+        "complexity_at_change": complexity[change_index] if change_index is not None else None,
+        "lambda_z_at_change": lambda_z[change_index] if change_index is not None else None,
+        "best_model_before_overfitting": best_model_before_overfitting
+    }

lambda_guard_boosting-0.2.2/lambdaguard/lambdaguard.py

@@ -0,0 +1,58 @@
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+
+def boosting_leverage(model, X):
+    n = X.shape[0]
+    influence = np.zeros(n)
+    for est in model.estimators_.ravel():
+        leaf_id = est.apply(X)
+        unique, counts = np.unique(leaf_id, return_counts=True)
+        leaf_sizes = dict(zip(unique, counts))
+        lr = model.learning_rate
+        for i in range(n):
+            influence[i] += lr / leaf_sizes[leaf_id[i]]
+    return influence
+
+def lambda_guard_test(model, X, B=300, alpha=0.05, plot=True):
+    n = X.shape[0]
+    H = boosting_leverage(model, X)
+    T1_obs = H.sum() / n
+    T2_obs = H.max() / H.mean()
+    T1_boot = np.zeros(B)
+    T2_boot = np.zeros(B)
+    for b in range(B):
+        idx = np.random.choice(n, n, replace=True)
+        Hb = boosting_leverage(model, X[idx])
+        T1_boot[b] = Hb.sum() / n
+        T2_boot[b] = Hb.max() / Hb.mean()
+    q1, q2 = np.quantile(T1_boot, 1-alpha), np.quantile(T2_boot, 1-alpha)
+    p1, p2 = np.mean(T1_boot >= T1_obs), np.mean(T2_boot >= T2_obs)
+    reject = (p1 < alpha) or (p2 < alpha)
+    if plot:
+        fig, axes = plt.subplots(1,2,figsize=(12,4))
+        axes[0].hist(T1_boot, bins=30, density=True, alpha=0.7)
+        axes[0].axvline(T1_obs, color="black", label="Observed")
+        axes[0].axvline(q1, color="red", linestyle="--", label="Critical")
+        axes[0].set_title("T1: Effective DoF ratio")
+        axes[0].legend()
+        axes[1].hist(T2_boot, bins=30, density=True, alpha=0.7)
+        axes[1].axvline(T2_obs, color="black", label="Observed")
+        axes[1].axvline(q2, color="red", linestyle="--", label="Critical")
+        axes[1].set_title("T2: Peak leverage ratio")
+        axes[1].legend()
+        plt.show()
+    return {
+        "T1_df_ratio": T1_obs, "critical_df_ratio": q1, "p_df_ratio": p1,
+        "T2_peak_ratio": T2_obs, "critical_peak_ratio": q2, "p_peak_ratio": p2,
+        "reject_H0": reject
+    }
+
+def interpret(res):
+    if not res["reject_H0"]:
+        return "✔ REGIME STABILE / GENERALIZZANTE"
+    if res["p_df_ratio"] < 0.05 and res["p_peak_ratio"] < 0.05:
+        return "✖ REGIME INTERPOLANTE (OVERFITTING FORTE)"
+    if res["p_df_ratio"] < 0.05:
+        return "✖ COMPLESSITÀ GLOBALE ECCESSIVA"
+    return "✖ (LEVERAGE SPIKES)"

lambda_guard_boosting-0.2.2/lambdaguard/ofi.py

@@ -0,0 +1,157 @@
+# ============================================================
+# FULL EXPERIMENT: LAMBDA GUARD - MULTI-MODEL (OPTIMIZED)
+# ============================================================
+
+import numpy as np
+import pandas as pd
+import seaborn as sns
+import matplotlib.pyplot as plt
+from itertools import product
+
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import mean_squared_error
+from sklearn.datasets import make_regression
+
+from sklearn.ensemble import GradientBoostingRegressor
+import xgboost as xgb
+import lightgbm as lgb
+from catboost import CatBoostRegressor
+
+# -----------------------------
+# GENERALIZATION COMPONENTS
+# -----------------------------
+def generalization_index(model, X, y):
+    preds = model.predict(X)
+    A = np.corrcoef(preds, y)[0, 1] if np.std(preds) > 0 else 0
+    C = np.var(preds)
+    GI = A / C if C > 0 else 0
+    return GI, A, C
+
+def instability_index(model, X, noise_std=1e-3, seed=42):
+    rng = np.random.default_rng(seed)
+    noise = rng.normal(0, noise_std, X.shape)
+    preds_clean = model.predict(X)
+    preds_noisy = model.predict(X + noise)
+    S = np.mean(np.abs(preds_clean - preds_noisy)) / (np.std(preds_clean) + 1e-8)
+    return S
+
+# -----------------------------
+# MODEL FACTORY
+# -----------------------------
+def create_model(model_name, n_estimators=100, max_depth=3, learning_rate=0.05, min_samples_leaf=1):
+    if model_name == "GBR":
+        return GradientBoostingRegressor(
+            n_estimators=n_estimators,
+            max_depth=max_depth,
+            learning_rate=learning_rate,
+            subsample=0.8,
+            min_samples_leaf=min_samples_leaf,
+            random_state=42
+        )
+    elif model_name == "XGB":
+        return xgb.XGBRegressor(
+            n_estimators=n_estimators,
+            max_depth=max_depth,
+            learning_rate=learning_rate,
+            subsample=0.8,
+            random_state=42,
+            verbosity=0
+        )
+    elif model_name == "LGBM":
+        return lgb.LGBMRegressor(
+            n_estimators=n_estimators,
+            max_depth=max_depth,
+            learning_rate=learning_rate,
+            subsample=0.8,
+            min_child_samples=min_samples_leaf,
+            random_state=42,
+            verbose=-1
+        )
+    elif model_name == "CAT":
+        return CatBoostRegressor(
+            iterations=n_estimators,
+            depth=max_depth,
+            learning_rate=learning_rate,
+            random_seed=42,
+            verbose=0
+        )
+    else:
+        raise ValueError(f"Unknown model {model_name}")
+
+# -----------------------------
+# EXPERIMENT FUNCTION
+# -----------------------------
+def run_experiment_multi_model(X, y, dataset_name, model_names=["GBR"], 
+                               n_estimators_list=[50,100,200], max_depth_list=[3,5,7], min_samples_leaf_list=[2,5]):
+    print(f"\n{'='*70}\nDATASET: {dataset_name}\n{'='*70}")
+    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+    results = []
+
+    for model_name in model_names:
+        for sl, n_est, depth in product(min_samples_leaf_list, n_estimators_list, max_depth_list):
+            model = create_model(model_name, n_estimators=n_est, max_depth=depth, min_samples_leaf=sl)
+            model.fit(X_train, y_train)
+
+            GI, A, C = generalization_index(model, X_train, y_train)
+            S = instability_index(model, X_train)
+            OFI = (C / (A + C)) * S
+            G_norm = A / (A + C)
+
+            y_train_pred = model.predict(X_train)
+            y_test_pred = model.predict(X_test)
+            rmse_train = np.sqrt(mean_squared_error(y_train, y_train_pred))
+            rmse_test = np.sqrt(mean_squared_error(y_test, y_test_pred))
+            gap = rmse_test - rmse_train
+
+            results.append({
+                "model": model_name,
+                "dataset": dataset_name,
+                "min_samples_leaf": sl,
+                "n_estimators": n_est,
+                "max_depth": depth,
+                "A": A,
+                "C": C,
+                "GI": GI,
+                "G_norm": G_norm,
+                "Instability": S,
+                "OFI": OFI,
+                "Train_RMSE": rmse_train,
+                "Test_RMSE": rmse_test,
+                "Gap": gap
+            })
+
+    df = pd.DataFrame(results)
+    df["OFI_norm"] = df.groupby("model")["OFI"].transform(lambda x: (x - x.min()) / (x.max() - x.min()))
+    return df
+
+# -----------------------------
+# PLOT FUNCTION
+# -----------------------------
+def plot_all_multi_model(df, metric="Gap"):
+    for model_name in df["model"].unique():
+        df_model = df[df["model"] == model_name]
+        for x_col in ["G_norm", "OFI_norm"]:
+            plt.figure(figsize=(6,5))
+            sns.regplot(data=df_model, x=x_col, y=metric)
+            plt.title(f"{model_name} - {x_col} vs {metric}")
+            plt.grid(True)
+            plt.show()
+
+
+# -----------------------------
+# REGRESSION TEST
+# -----------------------------
+def regression_test(df):
+  X = df['OFI_norm']
+  y = df['Gap']
+  X_const = sm.add_constant(X)
+  model = sm.OLS(y, X_const).fit()
+  print(model.summary())
+  plt.figure(figsize=(8,6))
+  plt.scatter(df['OFI_norm'], df_model['Gap'], alpha=0.6)
+  plt.plot(df['OFI_norm'], model.predict(X_const), color='red', linewidth=2)
+  plt.xlabel('Lambda')
+  plt.ylabel('RMSE gap (test - train)')
+  plt.title(f'Regression Gap RMSE vs Lambda - {df_model["model"].iloc[0]}')
+  plt.grid(True)
+  plt.show()

lambda_guard_boosting-0.2.2/pyproject.toml

@@ -0,0 +1,41 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "lambda-guard-boosting"
+version = "0.2.2"
+description = "Overfitting detection for Gradient Boosting models using λ-Guard methodology."
+readme = "README.md"
+license = {text = "MIT"}
+authors = [
+    {name="Fabrizio Di Sciorio, PhD", email="fabriziodisciorio91@gmail.com"}
+]
+keywords = ["machine-learning", "gradient-boosting", "overfitting", "boosting", "lambda-guard"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Python Modules"
+]
+requires-python = ">=3.8"
+dependencies = [
+    "numpy>=1.26,<2.2",
+    "pandas>=2.2,<3.0",
+    "scikit-learn>=1.3,<2.0",
+    "matplotlib>=3.8,<4.0",
+    "seaborn>=0.12,<0.14",
+    "xgboost>=1.7,<4.0",
+    "lightgbm>=4.4,<5.0",
+    "catboost>=1.1,<2.0"
+]
+
+[project.urls]
+Homepage = "https://github.com/faberBI/lambdaguard"
+Documentation = "https://github.com/faberBI/lambdaguard"
+BugTracker = "https://github.com/faberBI/lambdaguard/issues"
+
+[tool.setuptools.packages.find]
+where = ["."]
+exclude = ["notebooks*", "tests*", "docs*"]

lambda_guard_boosting-0.2.2/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

lambda_guard_boosting-0.2.2/tests/test_cusum.py

@@ -0,0 +1,21 @@
+import numpy as np
+import pandas as pd
+from lambdaguard.cusum import lambda_detect
+
+def test_lambda_detect_basic():
+    # Fake dataset with 2 models and increasing OFI
+    df = pd.DataFrame({
+        "model": ["GBR"]*5,
+        "dataset": ["test"]*5,
+        "min_samples_leaf": [2]*5,
+        "n_estimators": [10,20,30,40,50],
+        "max_depth": [3]*5,
+        "A": np.linspace(0.8, 0.95, 5),
+        "C": np.linspace(0.1,0.5,5),
+        "OFI": np.linspace(0.1,0.6,5),
+        "OFI_norm": np.linspace(0.1,0.6,5)
+    })
+
+    result = lambda_detect(df, "GBR")
+    assert "overfitting_detected" in result
+    assert "best_model_before_overfitting" in result

lambda_guard_boosting-0.2.2/tests/test_ofi.py

@@ -0,0 +1,26 @@
+import numpy as np
+import pandas as pd
+from sklearn.datasets import make_regression
+from sklearn.ensemble import GradientBoostingRegressor
+
+from lambdaguard.ofi import generalization_index, instability_index, create_model
+
+def test_generalization_index():
+    X, y = make_regression(n_samples=50, n_features=5, noise=0.1, random_state=42)
+    model = GradientBoostingRegressor(n_estimators=10, random_state=42)
+    model.fit(X, y)
+    GI, A, C = generalization_index(model, X, y)
+    assert 0 <= GI or GI <= 1e10, "GI seems off"
+    assert np.isfinite(A), "Alignment not finite"
+    assert np.isfinite(C), "Complexity not finite"
+
+def test_instability_index():
+    X, y = make_regression(n_samples=50, n_features=5, noise=0.1, random_state=42)
+    model = GradientBoostingRegressor(n_estimators=10, random_state=42)
+    model.fit(X, y)
+    S = instability_index(model, X)
+    assert S >= 0, "Instability should be non-negative"
+
+def test_create_model():
+    model = create_model("GBR", n_estimators=5, max_depth=2)
+    assert model.n_estimators == 5