adaptive-greedy-search 2.0.0__tar.gz

adaptive_greedy_search-2.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mohammad Jawad Hasan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

adaptive_greedy_search-2.0.0/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: adaptive-greedy-search
+Version: 2.0.0
+Summary: AGS (Adaptive Greedy Search): surrogate-guided hyperparameter search with plateau escape and pruned cross-validation.
+Author: Mohammad Jawad Hasan
+License: MIT
+Project-URL: Homepage, https://github.com/MJwd-Hasan/adaptive-greedy-search
+Project-URL: Repository, https://github.com/MJwd-Hasan/adaptive-greedy-search
+Keywords: hyperparameter-optimization,machine-learning,scikit-learn,AutoML,TPE
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20
+Requires-Dist: scikit-learn>=1.0
+Dynamic: license-file
+
+# Adaptive Greedy Search (AGS)
+
+Surrogate-guided hyperparameter search over a discrete grid. Greedy
+hill-climbing with a TPE (or Gaussian Process) surrogate, radius-based
+plateau escape instead of a full-grid scan, and sequential fold-by-fold
+cross-validation with bound-based pruning so clearly-uncompetitive
+candidates get cut short instead of running to completion.
+
+```python
+from ags import AdaptiveGreedySearch
+```
+
+## Why
+
+`GridSearchCV` evaluates every combination — correct, but wasteful once
+the grid gets large. `RandomizedSearchCV` is cheap but has no memory:
+it never uses what it already learned from earlier trials to pick the
+next one. Bayesian approaches like Optuna's TPE fix that by modeling
+which regions of the space look promising, but a general-purpose
+sampler still spends full cross-validation budget on every trial, good
+or bad.
+
+AGS combines both ideas for the specific case of a **discrete**
+hyperparameter grid:
+
+- **A surrogate model** (TPE by default) learns from every evaluated
+  point which regions of the grid look promising, and steers the next
+  choice there instead of sampling blindly.
+- **Greedy hill-climbing with plateau escape**: it moves to the best
+  neighboring grid point each step, and when the immediate neighborhood
+  is exhausted, expands outward ring by ring (bounded by
+  `max_neighbor_radius`) rather than falling back to scanning the whole
+  grid.
+- **Pruned cross-validation**: each candidate's CV folds run
+  sequentially, and a candidate is stopped early once it's
+  mathematically or statistically out of contention against the best
+  result seen so far — so compute isn't wasted finishing out folds for
+  a configuration that's already lost.
+
+The result: for a fixed evaluation budget, it tends to land on a better
+configuration than random search on reasonably smooth hyperparameter
+landscapes, and at lower wall-clock cost than grid search or plain
+Bayesian search, because it also cuts the *inside* of each evaluation
+(the CV folds), not just the *number* of evaluations.
+
+**Where it's not the right tool:** landscapes where quality is scattered
+with no local structure (grid distance doesn't correlate with
+performance), very small evaluation budgets (the surrogate needs a
+handful of points before it's useful), or cases where you need the
+completeness guarantee of exhaustive grid search.
+
+## Installation
+
+```bash
+pip install adaptive-greedy-search
+```
+
+Or, from a cloned copy of this repository:
+
+```bash
+pip install -e .
+```
+
+Requires Python 3.8+, `numpy`, and `scikit-learn` (installed automatically).
+
+## Quickstart
+
+```python
+from ags import AdaptiveGreedySearch
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.datasets import make_classification
+
+X, y = make_classification(n_samples=500, n_features=20, random_state=0)
+
+param_grid = {
+    "n_estimators": [50, 100, 150, 200],
+    "max_depth": [3, 5, 7, 9, None],
+    "min_samples_split": [2, 4, 6, 8],
+}
+
+search = AdaptiveGreedySearch(
+    RandomForestClassifier(random_state=0),
+    param_grid,
+    cv=5,                  # pruning gives little benefit below cv=5; use 5-10
+    scoring="accuracy",
+    max_evaluations=30,
+)
+search.fit(X, y)
+
+print(search.best_params)
+print(search.best_score)
+print(f"folds saved by pruning: {search.folds_saved}/{search.total_folds_possible}")
+```
+
+## Key parameters
+
+| Parameter | Default | What it does |
+|---|---|---|
+| `cv` | `5` | Number of CV folds. Pruning is far more effective at `5` or `10`. |
+| `scoring` | `"accuracy"` | Any scikit-learn scorer string (`"accuracy"`, `"neg_mean_squared_error"`, `"roc_auc"`, ...). |
+| `max_evaluations` | `25` | Hard cap on the number of grid points evaluated. |
+| `surrogate_type` | `"tpe"` | `"tpe"` (Tree-structured Parzen Estimator, cheap to refit) or `"gp"` (Gaussian Process, better calibrated uncertainty but costlier as evaluations grow). |
+| `pruning_strategy` | `"percentile"` | `"percentile"` (aggressive — compares a candidate against the historical distribution of other candidates at the same fold count), `"optimistic"` (safe — only prunes when a candidate is *mathematically* unable to beat the current best), or `"none"` (exhaustive CV, no pruning). |
+| `pruning_percentile` | `25` | Lower = stricter pruning (fewer candidates survive) when using `"percentile"`. |
+| `early_stopping_patience` | `5` | Stop the whole search if the best score hasn't improved for this many consecutive evaluations. Set to `None` to always run to `max_evaluations`. |
+| `max_neighbor_radius` | `4` | How many rings outward the plateau-escape step is allowed to search before falling back to scoring the remaining grid. |
+
+## Choosing a pruning strategy
+
+- Want speed and can tolerate a small chance of pruning a candidate
+  that might have recovered? Use the default `"percentile"`.
+- Want a guarantee that pruning never changes the final answer versus
+  running exhaustively? Use `"optimistic"`.
+- Establishing a baseline, or debugging unexpected results? Use
+  `"none"` to fall back to plain exhaustive cross-validation.
+
+## What you get back after `.fit(X, y)`
+
+- `best_params`, `best_score`, `best_state` — the winning configuration.
+- `history` — a list of dicts, one per evaluated candidate, including
+  `n_folds_used`, `n_folds_total`, and whether/why it was pruned.
+- `n_evaluations`, `total_time`, `stopped_early`.
+- `n_pruned`, `total_folds_run`, `total_folds_possible`, `folds_saved`
+  — how much cross-validation work was actually skipped.
+
+## License
+
+MIT © Mohammad Jawad Hasan

adaptive_greedy_search-2.0.0/README.md

@@ -0,0 +1,128 @@
+# Adaptive Greedy Search (AGS)
+
+Surrogate-guided hyperparameter search over a discrete grid. Greedy
+hill-climbing with a TPE (or Gaussian Process) surrogate, radius-based
+plateau escape instead of a full-grid scan, and sequential fold-by-fold
+cross-validation with bound-based pruning so clearly-uncompetitive
+candidates get cut short instead of running to completion.
+
+```python
+from ags import AdaptiveGreedySearch
+```
+
+## Why
+
+`GridSearchCV` evaluates every combination — correct, but wasteful once
+the grid gets large. `RandomizedSearchCV` is cheap but has no memory:
+it never uses what it already learned from earlier trials to pick the
+next one. Bayesian approaches like Optuna's TPE fix that by modeling
+which regions of the space look promising, but a general-purpose
+sampler still spends full cross-validation budget on every trial, good
+or bad.
+
+AGS combines both ideas for the specific case of a **discrete**
+hyperparameter grid:
+
+- **A surrogate model** (TPE by default) learns from every evaluated
+  point which regions of the grid look promising, and steers the next
+  choice there instead of sampling blindly.
+- **Greedy hill-climbing with plateau escape**: it moves to the best
+  neighboring grid point each step, and when the immediate neighborhood
+  is exhausted, expands outward ring by ring (bounded by
+  `max_neighbor_radius`) rather than falling back to scanning the whole
+  grid.
+- **Pruned cross-validation**: each candidate's CV folds run
+  sequentially, and a candidate is stopped early once it's
+  mathematically or statistically out of contention against the best
+  result seen so far — so compute isn't wasted finishing out folds for
+  a configuration that's already lost.
+
+The result: for a fixed evaluation budget, it tends to land on a better
+configuration than random search on reasonably smooth hyperparameter
+landscapes, and at lower wall-clock cost than grid search or plain
+Bayesian search, because it also cuts the *inside* of each evaluation
+(the CV folds), not just the *number* of evaluations.
+
+**Where it's not the right tool:** landscapes where quality is scattered
+with no local structure (grid distance doesn't correlate with
+performance), very small evaluation budgets (the surrogate needs a
+handful of points before it's useful), or cases where you need the
+completeness guarantee of exhaustive grid search.
+
+## Installation
+
+```bash
+pip install adaptive-greedy-search
+```
+
+Or, from a cloned copy of this repository:
+
+```bash
+pip install -e .
+```
+
+Requires Python 3.8+, `numpy`, and `scikit-learn` (installed automatically).
+
+## Quickstart
+
+```python
+from ags import AdaptiveGreedySearch
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.datasets import make_classification
+
+X, y = make_classification(n_samples=500, n_features=20, random_state=0)
+
+param_grid = {
+    "n_estimators": [50, 100, 150, 200],
+    "max_depth": [3, 5, 7, 9, None],
+    "min_samples_split": [2, 4, 6, 8],
+}
+
+search = AdaptiveGreedySearch(
+    RandomForestClassifier(random_state=0),
+    param_grid,
+    cv=5,                  # pruning gives little benefit below cv=5; use 5-10
+    scoring="accuracy",
+    max_evaluations=30,
+)
+search.fit(X, y)
+
+print(search.best_params)
+print(search.best_score)
+print(f"folds saved by pruning: {search.folds_saved}/{search.total_folds_possible}")
+```
+
+## Key parameters
+
+| Parameter | Default | What it does |
+|---|---|---|
+| `cv` | `5` | Number of CV folds. Pruning is far more effective at `5` or `10`. |
+| `scoring` | `"accuracy"` | Any scikit-learn scorer string (`"accuracy"`, `"neg_mean_squared_error"`, `"roc_auc"`, ...). |
+| `max_evaluations` | `25` | Hard cap on the number of grid points evaluated. |
+| `surrogate_type` | `"tpe"` | `"tpe"` (Tree-structured Parzen Estimator, cheap to refit) or `"gp"` (Gaussian Process, better calibrated uncertainty but costlier as evaluations grow). |
+| `pruning_strategy` | `"percentile"` | `"percentile"` (aggressive — compares a candidate against the historical distribution of other candidates at the same fold count), `"optimistic"` (safe — only prunes when a candidate is *mathematically* unable to beat the current best), or `"none"` (exhaustive CV, no pruning). |
+| `pruning_percentile` | `25` | Lower = stricter pruning (fewer candidates survive) when using `"percentile"`. |
+| `early_stopping_patience` | `5` | Stop the whole search if the best score hasn't improved for this many consecutive evaluations. Set to `None` to always run to `max_evaluations`. |
+| `max_neighbor_radius` | `4` | How many rings outward the plateau-escape step is allowed to search before falling back to scoring the remaining grid. |
+
+## Choosing a pruning strategy
+
+- Want speed and can tolerate a small chance of pruning a candidate
+  that might have recovered? Use the default `"percentile"`.
+- Want a guarantee that pruning never changes the final answer versus
+  running exhaustively? Use `"optimistic"`.
+- Establishing a baseline, or debugging unexpected results? Use
+  `"none"` to fall back to plain exhaustive cross-validation.
+
+## What you get back after `.fit(X, y)`
+
+- `best_params`, `best_score`, `best_state` — the winning configuration.
+- `history` — a list of dicts, one per evaluated candidate, including
+  `n_folds_used`, `n_folds_total`, and whether/why it was pruned.
+- `n_evaluations`, `total_time`, `stopped_early`.
+- `n_pruned`, `total_folds_run`, `total_folds_possible`, `folds_saved`
+  — how much cross-validation work was actually skipped.
+
+## License
+
+MIT © Mohammad Jawad Hasan

adaptive_greedy_search-2.0.0/adaptive_greedy_search.egg-info/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: adaptive-greedy-search
+Version: 2.0.0
+Summary: AGS (Adaptive Greedy Search): surrogate-guided hyperparameter search with plateau escape and pruned cross-validation.
+Author: Mohammad Jawad Hasan
+License: MIT
+Project-URL: Homepage, https://github.com/MJwd-Hasan/adaptive-greedy-search
+Project-URL: Repository, https://github.com/MJwd-Hasan/adaptive-greedy-search
+Keywords: hyperparameter-optimization,machine-learning,scikit-learn,AutoML,TPE
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20
+Requires-Dist: scikit-learn>=1.0
+Dynamic: license-file
+
+# Adaptive Greedy Search (AGS)
+
+Surrogate-guided hyperparameter search over a discrete grid. Greedy
+hill-climbing with a TPE (or Gaussian Process) surrogate, radius-based
+plateau escape instead of a full-grid scan, and sequential fold-by-fold
+cross-validation with bound-based pruning so clearly-uncompetitive
+candidates get cut short instead of running to completion.
+
+```python
+from ags import AdaptiveGreedySearch
+```
+
+## Why
+
+`GridSearchCV` evaluates every combination — correct, but wasteful once
+the grid gets large. `RandomizedSearchCV` is cheap but has no memory:
+it never uses what it already learned from earlier trials to pick the
+next one. Bayesian approaches like Optuna's TPE fix that by modeling
+which regions of the space look promising, but a general-purpose
+sampler still spends full cross-validation budget on every trial, good
+or bad.
+
+AGS combines both ideas for the specific case of a **discrete**
+hyperparameter grid:
+
+- **A surrogate model** (TPE by default) learns from every evaluated
+  point which regions of the grid look promising, and steers the next
+  choice there instead of sampling blindly.
+- **Greedy hill-climbing with plateau escape**: it moves to the best
+  neighboring grid point each step, and when the immediate neighborhood
+  is exhausted, expands outward ring by ring (bounded by
+  `max_neighbor_radius`) rather than falling back to scanning the whole
+  grid.
+- **Pruned cross-validation**: each candidate's CV folds run
+  sequentially, and a candidate is stopped early once it's
+  mathematically or statistically out of contention against the best
+  result seen so far — so compute isn't wasted finishing out folds for
+  a configuration that's already lost.
+
+The result: for a fixed evaluation budget, it tends to land on a better
+configuration than random search on reasonably smooth hyperparameter
+landscapes, and at lower wall-clock cost than grid search or plain
+Bayesian search, because it also cuts the *inside* of each evaluation
+(the CV folds), not just the *number* of evaluations.
+
+**Where it's not the right tool:** landscapes where quality is scattered
+with no local structure (grid distance doesn't correlate with
+performance), very small evaluation budgets (the surrogate needs a
+handful of points before it's useful), or cases where you need the
+completeness guarantee of exhaustive grid search.
+
+## Installation
+
+```bash
+pip install adaptive-greedy-search
+```
+
+Or, from a cloned copy of this repository:
+
+```bash
+pip install -e .
+```
+
+Requires Python 3.8+, `numpy`, and `scikit-learn` (installed automatically).
+
+## Quickstart
+
+```python
+from ags import AdaptiveGreedySearch
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.datasets import make_classification
+
+X, y = make_classification(n_samples=500, n_features=20, random_state=0)
+
+param_grid = {
+    "n_estimators": [50, 100, 150, 200],
+    "max_depth": [3, 5, 7, 9, None],
+    "min_samples_split": [2, 4, 6, 8],
+}
+
+search = AdaptiveGreedySearch(
+    RandomForestClassifier(random_state=0),
+    param_grid,
+    cv=5,                  # pruning gives little benefit below cv=5; use 5-10
+    scoring="accuracy",
+    max_evaluations=30,
+)
+search.fit(X, y)
+
+print(search.best_params)
+print(search.best_score)
+print(f"folds saved by pruning: {search.folds_saved}/{search.total_folds_possible}")
+```
+
+## Key parameters
+
+| Parameter | Default | What it does |
+|---|---|---|
+| `cv` | `5` | Number of CV folds. Pruning is far more effective at `5` or `10`. |
+| `scoring` | `"accuracy"` | Any scikit-learn scorer string (`"accuracy"`, `"neg_mean_squared_error"`, `"roc_auc"`, ...). |
+| `max_evaluations` | `25` | Hard cap on the number of grid points evaluated. |
+| `surrogate_type` | `"tpe"` | `"tpe"` (Tree-structured Parzen Estimator, cheap to refit) or `"gp"` (Gaussian Process, better calibrated uncertainty but costlier as evaluations grow). |
+| `pruning_strategy` | `"percentile"` | `"percentile"` (aggressive — compares a candidate against the historical distribution of other candidates at the same fold count), `"optimistic"` (safe — only prunes when a candidate is *mathematically* unable to beat the current best), or `"none"` (exhaustive CV, no pruning). |
+| `pruning_percentile` | `25` | Lower = stricter pruning (fewer candidates survive) when using `"percentile"`. |
+| `early_stopping_patience` | `5` | Stop the whole search if the best score hasn't improved for this many consecutive evaluations. Set to `None` to always run to `max_evaluations`. |
+| `max_neighbor_radius` | `4` | How many rings outward the plateau-escape step is allowed to search before falling back to scoring the remaining grid. |
+
+## Choosing a pruning strategy
+
+- Want speed and can tolerate a small chance of pruning a candidate
+  that might have recovered? Use the default `"percentile"`.
+- Want a guarantee that pruning never changes the final answer versus
+  running exhaustively? Use `"optimistic"`.
+- Establishing a baseline, or debugging unexpected results? Use
+  `"none"` to fall back to plain exhaustive cross-validation.
+
+## What you get back after `.fit(X, y)`
+
+- `best_params`, `best_score`, `best_state` — the winning configuration.
+- `history` — a list of dicts, one per evaluated candidate, including
+  `n_folds_used`, `n_folds_total`, and whether/why it was pruned.
+- `n_evaluations`, `total_time`, `stopped_early`.
+- `n_pruned`, `total_folds_run`, `total_folds_possible`, `folds_saved`
+  — how much cross-validation work was actually skipped.
+
+## License
+
+MIT © Mohammad Jawad Hasan

adaptive_greedy_search-2.0.0/adaptive_greedy_search.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+README.md
+pyproject.toml
+adaptive_greedy_search.egg-info/PKG-INFO
+adaptive_greedy_search.egg-info/SOURCES.txt
+adaptive_greedy_search.egg-info/dependency_links.txt
+adaptive_greedy_search.egg-info/requires.txt
+adaptive_greedy_search.egg-info/top_level.txt
+ags/__init__.py
+ags/core.py
+ags/surrogates.py
+ags/utils.py

adaptive_greedy_search-2.0.0/adaptive_greedy_search.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

adaptive_greedy_search-2.0.0/adaptive_greedy_search.egg-info/requires.txt

@@ -0,0 +1,2 @@
+numpy>=1.20
+scikit-learn>=1.0

adaptive_greedy_search-2.0.0/adaptive_greedy_search.egg-info/top_level.txt

@@ -0,0 +1 @@
+ags

adaptive_greedy_search-2.0.0/ags/__init__.py

@@ -0,0 +1,20 @@
+"""
+AGS -- Adaptive Greedy Search
+
+Surrogate-guided hyperparameter search over a discrete grid: greedy
+radius-1 hill-climbing with a TPE (or GP) surrogate, radius-based
+plateau escape instead of a full-grid scan, and sequential fold-by-fold
+cross-validation with bound-based pruning to skip clearly-uncompetitive
+candidates early.
+
+    from ags import AdaptiveGreedySearch
+
+is all you need -- everything else (numpy, scikit-learn) is imported
+internally by the package.
+"""
+
+from .core import AdaptiveGreedySearch
+from .surrogates import TPESurrogate, build_gp_surrogate
+
+__all__ = ["AdaptiveGreedySearch", "TPESurrogate", "build_gp_surrogate"]
+__version__ = "2.0.0"

adaptive_greedy_search-2.0.0/ags/core.py

@@ -0,0 +1,404 @@
+"""
+Core AdaptiveGreedySearch implementation (AGS v2).
+"""
+
+import itertools
+import time
+import numpy as np
+
+from sklearn.base import clone, is_classifier
+from sklearn.model_selection import KFold, StratifiedKFold
+from sklearn.metrics import get_scorer
+
+from .surrogates import make_surrogate
+from .utils import safe_index
+
+
+class AdaptiveGreedySearch:
+    """
+    Surrogate-guided greedy search over a discrete hyperparameter grid.
+
+    Search strategy:
+        1. Random initial exploration.
+        2. Surrogate model fitting (TPE by default, or GP).
+        3. Greedy radius-1 neighbor selection (batched surrogate UCB).
+        4. Radius-based plateau escape when the immediate neighborhood
+           is exhausted (expands outward, stops at the first ring with
+           unevaluated candidates, scores them in one batched call).
+        5. Batched surrogate fallback over any remaining states, for the
+           rare case where even the expanded neighborhood is exhausted.
+        6. Optional early stopping if the best score hasn't improved for
+           `early_stopping_patience` consecutive evaluations.
+
+    Evaluation:
+        Each candidate is scored via cross-validation, run SEQUENTIALLY
+        fold by fold (not parallel `cross_val_score`), so that a
+        bound-based pruning rule can stop a clearly-uncompetitive
+        candidate early instead of finishing out every fold:
+          - "optimistic": safe/conservative. Prunes only when even the
+            best possible outcome for the remaining folds (assuming
+            they all hit the scorer's theoretical max) can't beat the
+            current best.
+          - "percentile": more aggressive. Prunes when a candidate's
+            partial mean falls below a percentile of how other
+            candidates were doing at the same fold count.
+
+    Acquisition:
+        UCB = predicted_mean + exploration_weight * predicted_std
+    """
+
+    def __init__(
+        self,
+        estimator,
+        param_grid,
+        cv=5,
+        scoring="accuracy",
+        initial_points=8,
+        max_evaluations=25,
+        exploration_weight=2.0,
+        random_state=42,
+        max_neighbor_radius=4,
+        early_stopping_patience=5,
+        surrogate_type="tpe",
+        tpe_gamma=0.25,
+        tpe_bandwidth=1.0,
+        enable_pruning=True,
+        pruning_strategy="percentile",   # "optimistic" | "percentile" | "none"
+        pruning_margin=0.0,
+        pruning_percentile=25,
+        min_folds_before_pruning=2,
+        min_history_for_percentile=10,
+        score_upper_bound=None,
+    ):
+
+        self.estimator = estimator
+        self.param_grid = param_grid
+        self.cv = cv
+        self.scoring = scoring
+
+        self.initial_points = initial_points
+        self.max_evaluations = max_evaluations
+        self.exploration_weight = exploration_weight
+        self.random_state = random_state
+        self.max_neighbor_radius = max_neighbor_radius
+        self.early_stopping_patience = early_stopping_patience
+        self.surrogate_type = surrogate_type
+        self.tpe_gamma = tpe_gamma
+        self.tpe_bandwidth = tpe_bandwidth
+
+        self.enable_pruning = enable_pruning
+        self.pruning_strategy = pruning_strategy
+        self.pruning_margin = pruning_margin
+        self.pruning_percentile = pruning_percentile
+        self.min_folds_before_pruning = min_folds_before_pruning
+        self.min_history_for_percentile = min_history_for_percentile
+
+        self.rng = np.random.default_rng(random_state)
+
+        self.parameter_names = list(param_grid.keys())
+        self.values = [list(param_grid[p]) for p in self.parameter_names]
+        self.all_states = list(
+            itertools.product(*[range(len(v)) for v in self.values])
+        )
+
+        self.evaluated = {}
+        self.history = []
+
+        # --- pruning bookkeeping ---
+        self._partial_mean_history = {}       # {fold_index: [partial_means...]}
+        self._observed_max_fold_score = -np.inf
+        self.score_upper_bound = (
+            score_upper_bound
+            if score_upper_bound is not None
+            else self._infer_score_upper_bound(scoring)
+        )
+
+        if is_classifier(estimator):
+            self._cv_splitter = StratifiedKFold(
+                n_splits=cv, shuffle=True, random_state=random_state
+            )
+        else:
+            self._cv_splitter = KFold(
+                n_splits=cv, shuffle=True, random_state=random_state
+            )
+
+        self._scorer = get_scorer(scoring)
+
+        self.surrogate = make_surrogate(
+            surrogate_type,
+            tpe_gamma=tpe_gamma,
+            tpe_bandwidth=tpe_bandwidth,
+            random_state=random_state,
+        )
+
+    # ========================================================
+    # Upper bound inference for the "optimistic" pruning rule.
+    # ========================================================
+
+    @staticmethod
+    def _infer_score_upper_bound(scoring):
+        if not isinstance(scoring, str):
+            return None
+        if scoring.startswith("neg_"):
+            return 0.0  # e.g. neg_mean_squared_error: perfect = 0
+        return 1.0  # accuracy, f1*, roc_auc*, r2, precision*, recall*, etc.
+
+    # ========================================================
+    # STATE -> REAL HYPERPARAMETERS
+    # ========================================================
+
+    def state_to_params(self, state):
+        return {
+            p: self.values[j][state[j]]
+            for j, p in enumerate(self.parameter_names)
+        }
+
+    def distance(self, a, b):
+        return sum(abs(x - y) for x, y in zip(a, b))
+
+    def get_neighbors(self, state):
+        neighbors = []
+        for i in range(len(state)):
+            for step in (-1, 1):
+                new_state = list(state)
+                new_state[i] += step
+                if 0 <= new_state[i] < len(self.values[i]):
+                    new_state = tuple(new_state)
+                    if new_state not in self.evaluated:
+                        neighbors.append(new_state)
+        return neighbors
+
+    # ========================================================
+    # EVALUATE -- sequential, fold by fold, with pruning
+    # ========================================================
+
+    def evaluate(self, state, X, y):
+        params = self.state_to_params(state)
+
+        base_model = clone(self.estimator)
+        base_model.set_params(**params)
+        if "n_jobs" in base_model.get_params():
+            base_model.set_params(n_jobs=1)
+
+        K = self._cv_splitter.get_n_splits(X, y)
+
+        incumbent = max(self.evaluated.values()) if self.evaluated else None
+
+        fold_scores = []
+        pruned = False
+        prune_reason = None
+
+        start = time.perf_counter()
+
+        for k, (train_idx, test_idx) in enumerate(
+            self._cv_splitter.split(X, y), start=1
+        ):
+            X_train, X_test = safe_index(X, train_idx), safe_index(X, test_idx)
+            y_train, y_test = safe_index(y, train_idx), safe_index(y, test_idx)
+
+            fold_model = clone(base_model)
+            fold_model.fit(X_train, y_train)
+            fold_score = float(self._scorer(fold_model, X_test, y_test))
+            fold_scores.append(fold_score)
+
+            if fold_score > self._observed_max_fold_score:
+                self._observed_max_fold_score = fold_score
+
+            partial_mean = float(np.mean(fold_scores))
+            self._partial_mean_history.setdefault(k, []).append(partial_mean)
+
+            is_last_fold = (k == K)
+            can_prune = (
+                self.enable_pruning
+                and incumbent is not None
+                and not is_last_fold
+                and k >= self.min_folds_before_pruning
+            )
+
+            if can_prune and self.pruning_strategy == "optimistic":
+                upper_bound = (
+                    self.score_upper_bound
+                    if self.score_upper_bound is not None
+                    else self._observed_max_fold_score
+                )
+                best_possible_mean = (
+                    sum(fold_scores) + (K - k) * upper_bound
+                ) / K
+                if best_possible_mean < incumbent - self.pruning_margin:
+                    pruned = True
+                    prune_reason = "optimistic_bound"
+                    break
+
+            elif can_prune and self.pruning_strategy == "percentile":
+                hist = self._partial_mean_history.get(k, [])
+                if len(hist) >= self.min_history_for_percentile:
+                    threshold = np.percentile(hist, self.pruning_percentile)
+                    if partial_mean < threshold:
+                        pruned = True
+                        prune_reason = "percentile_bound"
+                        break
+
+        runtime = time.perf_counter() - start
+        score = float(np.mean(fold_scores))
+
+        self.evaluated[state] = score
+        self.history.append({
+            "state": state,
+            "params": params,
+            "score": score,
+            "runtime": runtime,
+            "n_folds_used": len(fold_scores),
+            "n_folds_total": K,
+            "pruned": pruned,
+            "prune_reason": prune_reason,
+        })
+
+        return score
+
+    # ========================================================
+    # SURROGATE
+    # ========================================================
+
+    def fit_surrogate(self):
+        X_train = np.array(list(self.evaluated.keys()), dtype=float)
+        y_train = np.array(list(self.evaluated.values()), dtype=float)
+        self.surrogate.fit(X_train, y_train)
+
+    def predict_uncertainty(self, states):
+        states_array = np.array(states, dtype=float)
+        mean, std = self.surrogate.predict(states_array, return_std=True)
+        return mean, std
+
+    def calculate_f(self, state, start_state):
+        """Diagnostic blended distance/quality score. Not in the hot path."""
+        mean, std = self.predict_uncertainty([state])
+        mean = mean[0]
+        std = std[0]
+
+        upper_score = mean + (self.exploration_weight * std)
+
+        observed = np.array(list(self.evaluated.values()))
+        lo, hi = observed.min(), observed.max()
+        span = hi - lo
+        norm_score = np.clip((upper_score - lo) / span, 0.0, 1.0) if span > 0 else 0.5
+
+        g_raw = self.distance(start_state, state)
+        g = np.clip(g_raw / max(self.max_neighbor_radius, 1), 0.0, 1.0)
+
+        h = 1 - norm_score
+        return g + h
+
+    def greedy_select(self):
+        best_state = max(self.evaluated, key=self.evaluated.get)
+        neighbors = self.get_neighbors(best_state)
+        if not neighbors:
+            return None
+        mean, std = self.predict_uncertainty(neighbors)
+        upper_score = mean + (self.exploration_weight * std)
+        return neighbors[int(np.argmax(upper_score))]
+
+    def _resolve_plateau(self, start_state):
+        visited = {start_state}
+        frontier = [start_state]
+        candidates = []
+
+        for _ in range(self.max_neighbor_radius):
+            next_frontier = []
+            for state in frontier:
+                for i in range(len(state)):
+                    for step in (-1, 1):
+                        new_state = list(state)
+                        new_state[i] += step
+                        if 0 <= new_state[i] < len(self.values[i]):
+                            new_state = tuple(new_state)
+                            if new_state in visited:
+                                continue
+                            visited.add(new_state)
+                            next_frontier.append(new_state)
+                            if new_state not in self.evaluated:
+                                candidates.append(new_state)
+            frontier = next_frontier
+            if candidates or not frontier:
+                break
+
+        if not candidates:
+            return None
+
+        mean, std = self.predict_uncertainty(candidates)
+        upper = mean + self.exploration_weight * std
+        return candidates[int(np.argmax(upper))]
+
+    # ========================================================
+    # MAIN FIT
+    # ========================================================
+
+    def fit(self, X, y):
+        total_start = time.perf_counter()
+
+        initial_size = min(
+            self.initial_points, len(self.all_states), self.max_evaluations
+        )
+        initial_states = self.rng.choice(
+            len(self.all_states), size=initial_size, replace=False
+        )
+        for index in initial_states:
+            self.evaluate(self.all_states[index], X, y)
+
+        best_so_far = max(self.evaluated.values())
+        evaluations_since_improvement = 0
+        self.stopped_early = False
+
+        while len(self.evaluated) < min(self.max_evaluations, len(self.all_states)):
+
+            if (
+                self.early_stopping_patience is not None
+                and evaluations_since_improvement >= self.early_stopping_patience
+            ):
+                self.stopped_early = True
+                break
+
+            if len(self.evaluated) >= 3:
+                self.fit_surrogate()
+
+                next_state = self.greedy_select()
+
+                if next_state is None:
+                    current_best = max(self.evaluated, key=self.evaluated.get)
+                    next_state = self._resolve_plateau(current_best)
+
+                if next_state is None:
+                    remaining = [s for s in self.all_states if s not in self.evaluated]
+                    if not remaining:
+                        break
+                    mean, std = self.predict_uncertainty(remaining)
+                    upper = mean + self.exploration_weight * std
+                    next_state = remaining[int(np.argmax(upper))]
+
+                new_score = self.evaluate(next_state, X, y)
+
+                if new_score > best_so_far:
+                    best_so_far = new_score
+                    evaluations_since_improvement = 0
+                else:
+                    evaluations_since_improvement += 1
+
+            else:
+                remaining = [s for s in self.all_states if s not in self.evaluated]
+                if not remaining:
+                    break
+                random_state = remaining[self.rng.integers(len(remaining))]
+                self.evaluate(random_state, X, y)
+
+        best_state = max(self.evaluated, key=self.evaluated.get)
+        self.best_state = best_state
+        self.best_score = self.evaluated[best_state]
+        self.best_params = self.state_to_params(best_state)
+        self.n_evaluations = len(self.evaluated)
+        self.total_time = time.perf_counter() - total_start
+
+        self.n_pruned = sum(1 for h in self.history if h["pruned"])
+        self.total_folds_run = sum(h["n_folds_used"] for h in self.history)
+        self.total_folds_possible = sum(h["n_folds_total"] for h in self.history)
+        self.folds_saved = self.total_folds_possible - self.total_folds_run
+
+        return self

adaptive_greedy_search-2.0.0/ags/surrogates.py

@@ -0,0 +1,91 @@
+"""
+Surrogate models used by AdaptiveGreedySearch to score unevaluated
+hyperparameter configurations.
+
+Both surrogates expose the same interface:
+    .fit(X, y)
+    .predict(states, return_std=True) -> (mean, std)
+so AdaptiveGreedySearch never needs to know which one it's using.
+"""
+
+import numpy as np
+
+from sklearn.gaussian_process import GaussianProcessRegressor
+from sklearn.gaussian_process.kernels import ConstantKernel, Matern, WhiteKernel
+from sklearn.neighbors import KernelDensity
+
+
+class TPESurrogate:
+    """
+    Minimal Tree-structured Parzen Estimator surrogate (same idea as
+    Optuna's default sampler). Splits observed points into "good" (top
+    `gamma` fraction by score) and "bad" (the rest), fits a KDE over
+    each group in grid-coordinate space, and scores candidates by the
+    good/bad log-density ratio.
+    """
+
+    def __init__(self, gamma=0.25, bandwidth=1.0):
+        self.gamma = gamma
+        self.bandwidth = bandwidth
+        self.good_kde = None
+        self.bad_kde = None
+        self._y_min = 0.0
+        self._y_max = 1.0
+
+    def fit(self, X, y):
+        X = np.asarray(X, dtype=float)
+        y = np.asarray(y, dtype=float)
+
+        order = np.argsort(y)[::-1]
+        n_good = max(1, int(np.ceil(self.gamma * len(y))))
+        good_idx = order[:n_good]
+        bad_idx = order[n_good:]
+        if len(bad_idx) == 0:
+            bad_idx = good_idx
+
+        self._y_min, self._y_max = float(y.min()), float(y.max())
+
+        self.good_kde = KernelDensity(bandwidth=self.bandwidth).fit(X[good_idx])
+        self.bad_kde = KernelDensity(bandwidth=self.bandwidth).fit(X[bad_idx])
+        return self
+
+    def predict(self, states, return_std=True):
+        states = np.asarray(states, dtype=float)
+
+        log_l = self.good_kde.score_samples(states)
+        log_g = self.bad_kde.score_samples(states)
+        tpe_score = log_l - log_g
+
+        squashed = 1.0 / (1.0 + np.exp(-tpe_score))
+        mean = self._y_min + squashed * (self._y_max - self._y_min)
+
+        if not return_std:
+            return mean
+
+        std = (1.0 / (1.0 + np.abs(tpe_score))) * (self._y_max - self._y_min)
+        return mean, std
+
+
+def build_gp_surrogate(random_state=None):
+    """Construct the Matern-kernel GaussianProcessRegressor surrogate."""
+    kernel = (
+        ConstantKernel(1.0, (1e-3, 1e3))
+        * Matern(length_scale=1.0, nu=2.5)
+        + WhiteKernel(noise_level=1e-5, noise_level_bounds=(1e-8, 1e-1))
+    )
+    return GaussianProcessRegressor(
+        kernel=kernel,
+        alpha=1e-6,
+        normalize_y=True,
+        n_restarts_optimizer=3,
+        random_state=random_state,
+    )
+
+
+def make_surrogate(surrogate_type, tpe_gamma=0.25, tpe_bandwidth=1.0, random_state=None):
+    """Factory: build the requested surrogate ("tpe" or "gp")."""
+    if surrogate_type == "tpe":
+        return TPESurrogate(gamma=tpe_gamma, bandwidth=tpe_bandwidth)
+    elif surrogate_type == "gp":
+        return build_gp_surrogate(random_state=random_state)
+    raise ValueError(f"Unknown surrogate_type '{surrogate_type}', expected 'tpe' or 'gp'")

adaptive_greedy_search-2.0.0/ags/utils.py

@@ -0,0 +1,8 @@
+"""Small shared helpers for the AGS package."""
+
+
+def safe_index(data, idx):
+    """Index a numpy array or a pandas DataFrame/Series the same way."""
+    if hasattr(data, "iloc"):
+        return data.iloc[idx]
+    return data[idx]

adaptive_greedy_search-2.0.0/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "adaptive-greedy-search"
+version = "2.0.0"
+description = "AGS (Adaptive Greedy Search): surrogate-guided hyperparameter search with plateau escape and pruned cross-validation."
+readme = "README.md"
+license = {text = "MIT"}
+authors = [
+    {name = "Mohammad Jawad Hasan"}
+]
+requires-python = ">=3.8"
+keywords = ["hyperparameter-optimization", "machine-learning", "scikit-learn", "AutoML", "TPE"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Science/Research",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "numpy>=1.20",
+    "scikit-learn>=1.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/MJwd-Hasan/adaptive-greedy-search"
+Repository = "https://github.com/MJwd-Hasan/adaptive-greedy-search"
+
+[tool.setuptools]
+packages = ["ags"]

adaptive_greedy_search-2.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+