hone-python 0.10.0__tar.gz → 0.11.0__tar.gz

{hone_python-0.10.0 → hone_python-0.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hone-python
-Version: 0.10.0
+Version: 0.11.0
 Summary: A python-based optimization engine/library.
 License-Expression: BSD-3-Clause
 License-File: LICENSE
@@ -207,6 +207,49 @@ Constraints should be quick to check: hone checks them for every candidate it co
 
 Starting points must satisfy the constraints. If the constraints leave nothing to search, or a region too small to find in 100,000 random draws, the optimizer raises `InfeasibleSearchSpace` when it's created, before any evaluation. Each optimizer keeps to constraints in its own way: random search and Bayesian optimization only consider points that satisfy them, CMA-ES draws a candidate again while it breaks one, and the genetic optimizer moves such a child back toward its fitter parent until it is feasible, which tends to land it near the boundary, where constrained optima usually are. On the benchmark's constrained problems the choice between optimizers follows the same pattern as without constraints; see [benchmarks/README.md](benchmarks/README.md#constrained-problems). [examples/pressure_vessel.py](examples/pressure_vessel.py) designs a pressure vessel of least cost under three constraints and finds the best published design. [examples/maintenance_schedule.py](examples/maintenance_schedule.py) chooses a maintenance schedule for a production line, scored by a noisy simulation of a year, with a constraint on technician hours and a function constraint that depends on the chosen shift; it halves the expected yearly cost of a sensible schedule chosen by hand.
 
+### Conditional parameters
+
+Some settings only exist when another takes certain values: a kernel's settings only for the model that has kernels, a third layer's size only with three layers or more. Write each condition as `test >> parameters`, read "then", and pass them to `SearchSpace`:
+
+```python
+from hone import ChoiceParameter, RangeParameter, SearchSpace
+
+model = ChoiceParameter("model", ["svm", "forest"])
+kernel = ChoiceParameter("kernel", ["linear", "rbf"])
+C = RangeParameter("C", 1e-3, 1e3, log=True)
+gamma = RangeParameter("gamma", 1e-5, 1, log=True)
+trees = RangeParameter("trees", 10, 500, integer=True)
+
+search_space = SearchSpace(
+    [model, kernel, C, gamma, trees],
+    conditions=[
+        model["svm"] >> [kernel, C],
+        kernel["rbf"] >> gamma,  # only with an svm too, since kernel is
+        model["forest"] >> trees,
+    ],
+)
+
+def train(model, kernel=None, C=None, gamma=None, trees=None):
+    ...
+```
+
+The left of `>>` is a test: a choice parameter indexed by the values that switch the parameters on, such as `model["svm"]` or `optimizer["sgd", "nesterov"]`, or a comparison of range parameters in brackets, such as `(layers >= 3) >> units_3`. The brackets are needed, since Python reads `layers >= 3 >> units_3` as `layers >= (3 >> units_3)`; hone raises an error saying so. The right is a parameter or a list of them, and each parameter can have one condition. Conditions nest: a parameter whose test uses a parameter that is switched off is off too.
+
+A parameter that is switched off isn't part of the point. The objective isn't passed it, so give it a default, which hone checks before the run starts; results, progress and `best_params` leave it out; and a starting point must leave it out too. Two points that differ only in a parameter switched off are the same point, so they are averaged together with `noise=True` and the genetic optimizer doesn't count one as new.
+
+A constraint applies only while every parameter it uses is switched on, and `test >> constraint` makes a constraint apply only while the test holds:
+
+```python
+depth = RangeParameter("depth", 2, 20, integer=True)
+search_space = SearchSpace(
+    [model, trees, depth],
+    constraints=[model["forest"] >> (trees * depth <= 2000)],
+    conditions=[model["forest"] >> [trees, depth]],
+)
+```
+
+Every optimizer supports conditions; CMA-ES only conditions on range parameters, since it doesn't search choices. Bayesian optimization gains the most: a parameter switched off looks the same to its model whatever value it holds, so values that don't matter add no noise. On the benchmark's conditional problems its error is ×0.0004 that of searching every parameter all the time, at 10 evaluations per dimension. With conditions the genetic optimizer breeds each branch within its own niche by default (`niche_share` of 0.5, below), so a wide, easy branch doesn't crowd out a narrow, better one; see [benchmarks/README.md](benchmarks/README.md#conditional-problems). [examples/choose_a_model.py](examples/choose_a_model.py) chooses a classifier, among an SVM with three kernels, a random forest and logistic regression, together with its settings.
+
 ### Configuration
 
 `GeneticOptimizerConfig` validates its values when created. It can also be loaded from a JSON or YAML file:
@@ -219,7 +262,7 @@ Set `seed` to make a run repeatable. `max_evaluations` sets a budget: the run st
 
 Parents are chosen by tournament: `tournament_size` (default 4) individuals are drawn and the fittest wins.
 
-When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default.
+When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default, except in a search space with conditions: there each branch is a niche, and 0.5 is the default, since a branch with a better optimum is often harder to find.
 
 Each value of a new individual mutates with probability `mutation_rate`: by default 1 over the number of variables, and at least 0.1, so in two variables half of them mutate. A range value moves by a random step whose typical size is a fraction of the width of its range. A choice switches to another option, and a fixed value never changes.
 

{hone_python-0.10.0 → hone_python-0.11.0}/README.md

@@ -186,6 +186,49 @@ Constraints should be quick to check: hone checks them for every candidate it co
 
 Starting points must satisfy the constraints. If the constraints leave nothing to search, or a region too small to find in 100,000 random draws, the optimizer raises `InfeasibleSearchSpace` when it's created, before any evaluation. Each optimizer keeps to constraints in its own way: random search and Bayesian optimization only consider points that satisfy them, CMA-ES draws a candidate again while it breaks one, and the genetic optimizer moves such a child back toward its fitter parent until it is feasible, which tends to land it near the boundary, where constrained optima usually are. On the benchmark's constrained problems the choice between optimizers follows the same pattern as without constraints; see [benchmarks/README.md](benchmarks/README.md#constrained-problems). [examples/pressure_vessel.py](examples/pressure_vessel.py) designs a pressure vessel of least cost under three constraints and finds the best published design. [examples/maintenance_schedule.py](examples/maintenance_schedule.py) chooses a maintenance schedule for a production line, scored by a noisy simulation of a year, with a constraint on technician hours and a function constraint that depends on the chosen shift; it halves the expected yearly cost of a sensible schedule chosen by hand.
 
+### Conditional parameters
+
+Some settings only exist when another takes certain values: a kernel's settings only for the model that has kernels, a third layer's size only with three layers or more. Write each condition as `test >> parameters`, read "then", and pass them to `SearchSpace`:
+
+```python
+from hone import ChoiceParameter, RangeParameter, SearchSpace
+
+model = ChoiceParameter("model", ["svm", "forest"])
+kernel = ChoiceParameter("kernel", ["linear", "rbf"])
+C = RangeParameter("C", 1e-3, 1e3, log=True)
+gamma = RangeParameter("gamma", 1e-5, 1, log=True)
+trees = RangeParameter("trees", 10, 500, integer=True)
+
+search_space = SearchSpace(
+    [model, kernel, C, gamma, trees],
+    conditions=[
+        model["svm"] >> [kernel, C],
+        kernel["rbf"] >> gamma,  # only with an svm too, since kernel is
+        model["forest"] >> trees,
+    ],
+)
+
+def train(model, kernel=None, C=None, gamma=None, trees=None):
+    ...
+```
+
+The left of `>>` is a test: a choice parameter indexed by the values that switch the parameters on, such as `model["svm"]` or `optimizer["sgd", "nesterov"]`, or a comparison of range parameters in brackets, such as `(layers >= 3) >> units_3`. The brackets are needed, since Python reads `layers >= 3 >> units_3` as `layers >= (3 >> units_3)`; hone raises an error saying so. The right is a parameter or a list of them, and each parameter can have one condition. Conditions nest: a parameter whose test uses a parameter that is switched off is off too.
+
+A parameter that is switched off isn't part of the point. The objective isn't passed it, so give it a default, which hone checks before the run starts; results, progress and `best_params` leave it out; and a starting point must leave it out too. Two points that differ only in a parameter switched off are the same point, so they are averaged together with `noise=True` and the genetic optimizer doesn't count one as new.
+
+A constraint applies only while every parameter it uses is switched on, and `test >> constraint` makes a constraint apply only while the test holds:
+
+```python
+depth = RangeParameter("depth", 2, 20, integer=True)
+search_space = SearchSpace(
+    [model, trees, depth],
+    constraints=[model["forest"] >> (trees * depth <= 2000)],
+    conditions=[model["forest"] >> [trees, depth]],
+)
+```
+
+Every optimizer supports conditions; CMA-ES only conditions on range parameters, since it doesn't search choices. Bayesian optimization gains the most: a parameter switched off looks the same to its model whatever value it holds, so values that don't matter add no noise. On the benchmark's conditional problems its error is ×0.0004 that of searching every parameter all the time, at 10 evaluations per dimension. With conditions the genetic optimizer breeds each branch within its own niche by default (`niche_share` of 0.5, below), so a wide, easy branch doesn't crowd out a narrow, better one; see [benchmarks/README.md](benchmarks/README.md#conditional-problems). [examples/choose_a_model.py](examples/choose_a_model.py) chooses a classifier, among an SVM with three kernels, a random forest and logistic regression, together with its settings.
+
 ### Configuration
 
 `GeneticOptimizerConfig` validates its values when created. It can also be loaded from a JSON or YAML file:
@@ -198,7 +241,7 @@ Set `seed` to make a run repeatable. `max_evaluations` sets a budget: the run st
 
 Parents are chosen by tournament: `tournament_size` (default 4) individuals are drawn and the fittest wins.
 
-When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default.
+When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default, except in a search space with conditions: there each branch is a niche, and 0.5 is the default, since a branch with a better optimum is often harder to find.
 
 Each value of a new individual mutates with probability `mutation_rate`: by default 1 over the number of variables, and at least 0.1, so in two variables half of them mutate. A range value moves by a random step whose typical size is a fraction of the width of its range. A choice switches to another option, and a fixed value never changes.
 

{hone_python-0.10.0 → hone_python-0.11.0}/hone/__init__.py

@@ -10,7 +10,7 @@ from hone.optimizers import (
     UnsupportedSearchSpace,
     choose_optimizer,
 )
-from hone.parameters import ChoiceParameter, Constraint, FixedParameter, InfeasibleSearchSpace, RangeParameter, SearchSpace
+from hone.parameters import ChoiceParameter, Condition, Constraint, FixedParameter, InfeasibleSearchSpace, RangeParameter, SearchSpace
 from hone.result import GenerationStats, Progress, Result, RunInfo, print_progress
 
 __all__ = [
@@ -19,6 +19,7 @@ __all__ = [
     "CMAES",
     "CMAESConfig",
     "ChoiceParameter",
+    "Condition",
     "Constraint",
     "EvaluationFailed",
     "FixedParameter",

{hone_python-0.10.0 → hone_python-0.11.0}/hone/benchmarks/__init__.py

@@ -1,4 +1,5 @@
 """Benchmark problems and a runner for comparing optimizers. Run `python -m hone.benchmarks --help`."""
+from .conditional import conditional_problems
 from .constrained import constrained_problems
 from .ml import ml_problems
 from .problems import Problem, noisy_problems, quick_problems, standard_problems

{hone_python-0.10.0 → hone_python-0.11.0}/hone/benchmarks/__main__.py

@@ -6,6 +6,7 @@ from collections.abc import Sequence
 from importlib.metadata import version
 
 from . import ml
+from .conditional import conditional_problems
 from .constrained import constrained_problems
 from .problems import noisy_problems, quick_problems, standard_problems
 from .runner import BUDGETS, load_summaries, regressions, run_benchmark, save_summaries, summarize, to_markdown
@@ -28,6 +29,9 @@ def main(argv: list[str] | None = None) -> int:
     suite.add_argument("--ml", action="store_true", help="run the machine learning tuning problems instead (needs scikit-learn)")
     suite.add_argument("--noisy", action="store_true", help="run the problems with noise added to every evaluation instead")
     suite.add_argument("--constrained", action="store_true", help="run the problems with constraints instead")
+    suite.add_argument(
+        "--conditional", action="store_true", help="run the problems with conditional parameters, and their flat versions, instead"
+    )
     parser.add_argument("--seeds", type=int, help="number of seeds per problem (default 10, or 5 with --quick)")
     parser.add_argument(
         "--budgets",
@@ -53,6 +57,8 @@ def main(argv: list[str] | None = None) -> int:
         problems, default_budgets = noisy_problems(), NOISY_BUDGETS
     elif args.constrained:
         problems, default_budgets = constrained_problems(), BUDGETS
+    elif args.conditional:
+        problems, default_budgets = conditional_problems(), BUDGETS
     else:
         problems, default_budgets = standard_problems(), BUDGETS
     seeds = args.seeds or (5 if args.quick else 10)

hone_python-0.11.0/hone/benchmarks/conditional.py

@@ -0,0 +1,109 @@
+"""Conditional test problems, each in two versions: with conditions, and with every parameter switched on.
+
+- branches-3x2d and branches-3x5d: a choice of three branches, each a function of its own
+  numbers. The best branch is a steep, narrow bowl reaching 0; the decoys are a wide, shallow
+  bowl whose floor is 1 and a shallow Rastrigin grid whose floor is 0.5. Random points score
+  better in the decoys, so an optimizer has to look past them.
+- nested-9: a choice of model; one model has a choice of kernel, and one kernel has numbers of
+  its own, so the best point is two conditions deep. The other model is a wide, shallow decoy
+  whose floor is 0.25.
+- layers-7: a layer count from 1 to 5 whose larger values switch on more numbers, as a
+  network's units per layer would be. Three layers is best. Its conditions test a range, so
+  CMA-ES can search it.
+
+The "-flat" version of each has the same parameters, function and optimum, with no conditions:
+every parameter is always on and the function ignores those it doesn't use, which was the way
+to search such a space before conditions. Comparing the two shows what conditions are worth.
+"""
+from typing import Any, Callable
+
+import numpy as np
+
+from hone.parameters import ChoiceParameter, Condition, Parameter, RangeParameter, SearchSpace
+
+from .problems import BOUND, Problem, _shift, rastrigin, sphere
+
+
+def _problem(name: str, function: Callable[..., float], parameters: list[Parameter], conditions: list[Condition]) -> list[Problem]:
+    function.__name__ = name.replace("-", "_")
+    return [
+        Problem(
+            name=name,
+            function=function,
+            search_space=SearchSpace(parameters, conditions=conditions),
+            optimum=0.0,
+            tags=frozenset({"conditional"}),
+        ),
+        Problem(
+            name=f"{name}-flat",
+            function=function,
+            search_space=SearchSpace(parameters),
+            optimum=0.0,
+            tags=frozenset({"conditional", "flat"}),
+        ),
+    ]
+
+
+def _numbers(prefix: str, count: int) -> list[RangeParameter]:
+    return [RangeParameter(f"{prefix}{i}", -BOUND, BOUND) for i in range(count)]
+
+
+def _vector(params: dict[str, Any], numbers: list[RangeParameter]) -> np.ndarray:
+    return np.array([params[p.name] for p in numbers], dtype=float)
+
+
+def branches(dimensions: int) -> list[Problem]:
+    name = f"branches-3x{dimensions}d"
+    branch = ChoiceParameter("branch", ["narrow", "wide", "grid"])
+    numbers = {b: _numbers(f"{b}_", dimensions) for b in branch.choices}
+    shifts = {b: _shift(f"{name}/{b}", dimensions) for b in branch.choices}
+    # (function, scale, floor) for each branch
+    bases = {"narrow": (sphere, 4.0, 0.0), "wide": (sphere, 0.05, 1.0), "grid": (rastrigin, 0.1, 0.5)}
+
+    def function(branch: str, **params: Any) -> float:
+        base, scale, floor = bases[branch]
+        return floor + scale * float(base(_vector(params, numbers[branch]) - shifts[branch]))
+
+    parameters: list[Parameter] = [branch, *(p for b in branch.choices for p in numbers[b])]
+    return _problem(name, function, parameters, [branch[b] >> numbers[b] for b in branch.choices])
+
+
+def nested() -> list[Problem]:
+    name = "nested-9"
+    model = ChoiceParameter("model", ["kernel_machine", "forest"])
+    kernel = ChoiceParameter("kernel", ["linear", "rbf"])
+    shared, width, forest = _numbers("s", 2), _numbers("g", 2), _numbers("f", 3)
+    centres = {key: _shift(f"{name}/{key}", len(group)) for key, group in [("s", shared), ("g", width), ("f", forest)]}
+
+    def function(model: str, kernel: str | None = None, **params: Any) -> float:
+        if model == "forest":
+            return 0.25 + 0.05 * float(sphere(_vector(params, forest) - centres["f"]))
+        value = float(sphere(_vector(params, shared) - centres["s"]))
+        if kernel == "linear":
+            return 0.5 + value
+        return value + 4 * float(sphere(_vector(params, width) - centres["g"]))
+
+    parameters: list[Parameter] = [model, kernel, *shared, *width, *forest]
+    conditions = [model["kernel_machine"] >> [kernel, *shared], kernel["rbf"] >> width, model["forest"] >> forest]
+    return _problem(name, function, parameters, conditions)
+
+
+def layers() -> list[Problem]:
+    name = "layers-7"
+    count = RangeParameter("layers", 1, 5, integer=True)
+    rate = RangeParameter("rate", -BOUND, BOUND)
+    units = _numbers("units_", 5)
+    centre = _shift(f"{name}/units", 5)
+    best_rate = float(_shift(f"{name}/rate", 1)[0])
+
+    def function(layers: int, rate: float, **params: Any) -> float:
+        used = units[:layers]
+        return 0.5 * abs(layers - 3) + float(sphere(_vector(params, used) - centre[:layers])) + (rate - best_rate) ** 2
+
+    parameters: list[Parameter] = [count, rate, *units]
+    return _problem(name, function, parameters, [(count >= i + 1) >> units[i] for i in range(1, 5)])
+
+
+def conditional_problems() -> list[Problem]:
+    """Each conditional problem, followed by its flat version."""
+    return [*branches(2), *branches(5), *nested(), *layers()]

{hone_python-0.10.0 → hone_python-0.11.0}/hone/benchmarks/ml.py

@@ -16,7 +16,7 @@ from typing import Callable
 
 from threadpoolctl import threadpool_limits
 
-from hone.parameters import ChoiceParameter, RangeParameter, SearchSpace
+from hone.parameters import ChoiceParameter, Condition, Parameter, RangeParameter, SearchSpace
 
 from .problems import Problem
 
@@ -92,11 +92,56 @@ def random_forest_wine(max_depth: int, min_samples_leaf: int, max_features: floa
     return _cross_validate(model, "wine", classification=True)
 
 
+def model_selection_breast_cancer(
+    model: str,
+    C: float | None = None,
+    gamma: float | None = None,
+    max_depth: int | None = None,
+    min_samples_leaf: int | None = None,
+    max_features: float | None = None,
+    n_neighbors: int | None = None,
+    weights: str | None = None,
+) -> float:
+    """One of three classifiers, and its settings, on the 569 breast tumour samples svm-breast-cancer uses.
+
+    A support vector classifier with an RBF kernel, a 20-tree random forest, or k nearest
+    neighbours. With conditions, only the chosen model's settings are searched.
+    """
+    from sklearn.ensemble import RandomForestClassifier
+    from sklearn.neighbors import KNeighborsClassifier
+    from sklearn.pipeline import make_pipeline
+    from sklearn.preprocessing import StandardScaler
+    from sklearn.svm import SVC
+
+    if model == "svm":
+        estimator = make_pipeline(StandardScaler(), SVC(C=C, gamma=gamma))
+    elif model == "forest":
+        estimator = RandomForestClassifier(
+            n_estimators=20, max_depth=max_depth, min_samples_leaf=min_samples_leaf, max_features=max_features, random_state=0, n_jobs=1
+        )
+    else:
+        estimator = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=n_neighbors, weights=weights))
+    return _cross_validate(estimator, "breast_cancer", classification=True)
+
+
+_MODEL = ChoiceParameter("model", ["svm", "forest", "knn"])
+_SVM = [RangeParameter("C", 1e-3, 1e3, log=True), RangeParameter("gamma", 1e-5, 10, log=True)]
+_FOREST = [
+    RangeParameter("max_depth", 1, 12, integer=True),
+    RangeParameter("min_samples_leaf", 1, 20, integer=True),
+    RangeParameter("max_features", 0.05, 1.0),
+]
+_KNN: list[Parameter] = [RangeParameter("n_neighbors", 1, 50, integer=True), ChoiceParameter("weights", ["uniform", "distance"])]
+MODEL_SELECTION: list[Parameter] = [_MODEL, *_SVM, *_FOREST, *_KNN]
+MODEL_CONDITIONS: list[Condition] = [_MODEL["svm"] >> _SVM, _MODEL["forest"] >> _FOREST, _MODEL["knn"] >> _KNN]
+
 # The best value of each problem found by REFERENCE_SEARCH.
 OPTIMA = {
     "svm-breast-cancer": 0.014066138798323302,  # accuracy 98.6%, found by CMA-ES
     "kernel-ridge-diabetes": 0.4961748264794844,  # R² 0.504, found by Bayesian optimization
     "random-forest-wine": 0.005555555555555536,  # accuracy 99.4%, found by Bayesian optimization
+    # Every reference run chose the support vector classifier, and the best matched svm-breast-cancer's.
+    "model-selection-breast-cancer": 0.014066138798323302,
 }
 
 SPACES: dict[str, tuple[Callable[..., float], list]] = {
@@ -125,9 +170,30 @@ SPACES: dict[str, tuple[Callable[..., float], list]] = {
 
 
 def ml_problems() -> list[Problem]:
-    """The machine learning tuning problems. Needs scikit-learn."""
+    """The machine learning tuning problems. Needs scikit-learn.
+
+    model-selection-breast-cancer comes twice: with conditions, and as model-selection-breast-cancer-flat
+    with every setting always searched, as before conditions, to show what they are worth.
+    """
     _require_sklearn()
-    return [
+    problems = [
         Problem(name=name, function=function, search_space=SearchSpace(parameters), optimum=OPTIMA[name], tags=frozenset({"ml"}))
         for name, (function, parameters) in SPACES.items()
     ]
+    optimum = OPTIMA["model-selection-breast-cancer"]
+    return problems + [
+        Problem(
+            name="model-selection-breast-cancer",
+            function=model_selection_breast_cancer,
+            search_space=SearchSpace(MODEL_SELECTION, conditions=MODEL_CONDITIONS),
+            optimum=optimum,
+            tags=frozenset({"ml", "conditional"}),
+        ),
+        Problem(
+            name="model-selection-breast-cancer-flat",
+            function=model_selection_breast_cancer,
+            search_space=SearchSpace(MODEL_SELECTION),
+            optimum=optimum,
+            tags=frozenset({"ml", "conditional", "flat"}),
+        ),
+    ]

{hone_python-0.10.0 → hone_python-0.11.0}/hone/config/genetic_optimizer_config.py

@@ -48,8 +48,9 @@ class GeneticOptimizerConfig(OptimizerConfig):
     tournament_p: float = Field(default=1.0, ge=0, le=1)  # chance a tournament picks its best rather than its worst
     # Share of children bred within niches of parents with the same choice parameter values, so
     # each option keeps tuning its own numbers. Helps problems where the best choice is found
-    # late, on large budgets; costs on small ones. 0 turns niches off.
-    niche_share: float = Field(default=0.0, ge=0, lt=1)
+    # late, on large budgets; costs on small ones. 0 turns niches off. None, the default, means
+    # 0.5 for a search space with conditions, where each branch is a niche, and 0 otherwise.
+    niche_share: float | None = Field(default=None, ge=0, lt=1)
 
     @model_validator(mode="before")
     @classmethod

{hone_python-0.10.0 → hone_python-0.11.0}/hone/objective_function/objective_function.py

@@ -43,5 +43,20 @@ class ObjectiveFunction(BaseModel):
         )
         if unknown:
             raise ValueError(f"{self.name}() does not accept parameters: {', '.join(unknown)}")
+        # A parameter a condition switches off isn't passed, so the function needs a default for it.
+        no_default = sorted(
+            name
+            for name in names
+            if self.search_space.is_conditional(name)
+            and name in accepted
+            and accepted[name].default is inspect.Parameter.empty
+            and accepted[name].kind is not inspect.Parameter.VAR_KEYWORD
+        )
+        if no_default:
+            raise ValueError(
+                f"{self.name}() isn't passed {', '.join(no_default)} at points where a condition switches "
+                f"{'it' if len(no_default) == 1 else 'them'} off, so give {'it' if len(no_default) == 1 else 'each'} a default, "
+                f"such as {no_default[0]}=None"
+            )
         if missing:
             raise ValueError(f"{self.name}() requires parameters missing from the search space: {', '.join(missing)}")

{hone_python-0.10.0 → hone_python-0.11.0}/hone/optimizers/base.py

@@ -74,10 +74,15 @@ class Optimizer(ABC):
         config: The optimizer's configuration. Defaults are used if omitted.
         initial_points: Points to evaluate first, such as a known good design or the best
             parameters of an earlier run: one dict of parameter values, or a list of them.
-            Fixed parameters may be left out.
+            Fixed parameters may be left out, and so must parameters a condition switches off.
+
+    Every individual an optimizer breeds holds a value for every parameter, including those
+    the conditions switch off at its point, so an optimizer can keep them to use again. The
+    base class leaves them out of what it hands out and reports: trials, results and progress.
     """
     config_class: ClassVar[type[OptimizerConfig]] = OptimizerConfig
     supports_constraints: ClassVar[bool] = False  # whether it keeps to a search space's constraints
+    supports_conditions: ClassVar[bool] = False  # whether it handles parameters that conditions switch on and off
 
     def __init__(
         self,
@@ -94,6 +99,8 @@ class Optimizer(ABC):
             raise TypeError(f"{type(self).__name__} needs a {self.config_class.__name__}, got {type(self.config).__name__}")
         if search_space.constraints and not self.supports_constraints:
             raise UnsupportedSearchSpace(f"{type(self).__name__} doesn't support constraints yet; RandomSearch does")
+        if search_space.conditions and not self.supports_conditions:
+            raise UnsupportedSearchSpace(f"{type(self).__name__} doesn't support conditional parameters yet; RandomSearch does")
         if isinstance(initial_points, dict):
             initial_points = [initial_points]
         self.initial_points = [search_space.check(point) for point in initial_points or []]
@@ -177,13 +184,20 @@ class Optimizer(ABC):
         reissued = [outstanding[i] for i in self._reissue if i in outstanding][:n]
         self._reissue = [i for i in self._reissue if i in outstanding and i not in {ind.id for ind in reissued}]
         if self._done:
-            return [Trial(id=ind.id, params=dict(ind.params), generation=ind.generation) for ind in reissued]
+            return [self._trial(ind) for ind in reissued]
         ready = [ind for ind_id, ind in self._waiting.items() if ind_id not in self._handed_out]
         limits = [None if n is None else n - len(reissued), self._budget_left()]
         ready = ready[: min((limit for limit in limits if limit is not None), default=len(ready))]
         self._handed_out.update(ind.id for ind in ready)
         self._handed_out_total += len(ready)
-        return [Trial(id=ind.id, params=dict(ind.params), generation=ind.generation) for ind in reissued + ready]
+        return [self._trial(ind) for ind in reissued + ready]
+
+    def _trial(self, individual: Individual) -> Trial:
+        return Trial(id=individual.id, params=self.search_space.active(individual.params), generation=individual.generation)
+
+    def _key(self, params: dict[str, Any]) -> tuple:
+        """A hashable form of the point params are at: two that differ only where conditions switch parameters off share it."""
+        return _key(self.search_space.active(params))
 
     def tell(self, trial: Trial | int, value: float | None = None, *, failed: bool = False) -> None:
         """Report the outcome of a trial from ask(), by the trial or its id.
@@ -215,7 +229,7 @@ class Optimizer(ABC):
             individual.fitness = float(value)
             if self.config.noise:
                 # Average every evaluation of the same point, so a lucky value regresses toward its mean.
-                key = _key(individual.params)
+                key = self._key(individual.params)
                 self._observations.setdefault(key, []).append(float(value))
                 self._points.setdefault(key, dict(individual.params))
                 individual.fitness = float(np.mean(self._observations[key]))
@@ -271,8 +285,8 @@ class Optimizer(ABC):
             # Once the finalists have been re-evaluated, the one with the best average; before
             # that, the most promising point by the optimizer's best estimate.
             key = self._ranked_keys(self._finalists)[0] if self._finalists else self._finalist_order()[0]
-            return self._average(key), self._points[key]
-        return _fitness(self._best), self._best.params
+            return self._average(key), self.search_space.active(self._points[key])
+        return _fitness(self._best), self.search_space.active(self._best.params)
 
     def _run_info(self) -> RunInfo:
         return RunInfo(
@@ -538,7 +552,7 @@ class Optimizer(ABC):
         """Set each successful individual's fitness to the current average of every evaluation of its point."""
         for individual in individuals:
             if not individual.failed and individual.fitness is not None:
-                individual.fitness = self._average(_key(individual.params))
+                individual.fitness = self._average(self._key(individual.params))
 
     def _final_population(self, generation: int) -> list[Individual]:
         """The final round with noise: the most promising points, each evaluated again.
@@ -561,7 +575,14 @@ class Optimizer(ABC):
         return list(self._late.values())
 
     def _initial_individuals(self, generation: int) -> list[Individual]:
-        return [Individual(params=dict(point), generation=generation) for point in self.initial_points]
+        """The initial points as individuals, with a random value for each parameter a condition switches off."""
+        return [
+            Individual(
+                params={p.name: point[p.name] if p.name in point else p.sample(self.rng) for p in self.search_space.parameters},
+                generation=generation,
+            )
+            for point in self.initial_points
+        ]
 
     def _random_individual(self, generation: int) -> Individual:
         return Individual(params=self.search_space.sample(self.rng), generation=generation)
@@ -592,7 +613,7 @@ class Optimizer(ABC):
             evaluations=evaluations,
             failures=failures,
             best_fitness=succeeded[0].fitness,
-            best_params=dict(succeeded[0].params),
+            best_params=self.search_space.active(succeeded[0].params),
             worst_fitness=succeeded[-1].fitness,
             mean=float(np.mean(fitnesses)),
             median=float(np.median(fitnesses)),

{hone_python-0.10.0 → hone_python-0.11.0}/hone/optimizers/bayesian.py

@@ -7,6 +7,9 @@ simulations, experiments, training runs. It makes the most of a small budget.
 Range parameters are scaled to [0, 1], on a log scale where they have one; integers are searched
 as numbers and rounded. Each choice
 parameter becomes one column per choice, holding 1 for the chosen one and 0 for the rest.
+A parameter a condition switches off is encoded the same way whatever value it holds: a range at
+INACTIVE, and a choice with every column 0. So points that differ only where parameters are
+switched off are the same point to the model, and the values they happen to hold add no noise.
 """
 import math
 from typing import Any, List
@@ -20,12 +23,14 @@ from hone.config import BayesianConfig
 from hone.optimizers._gp import GaussianProcess
 from hone.optimizers.base import Individual, Optimizer, UnsupportedSearchSpace
 from hone.parameters import ChoiceParameter, Parameter, RangeParameter, SearchSpace
+from hone.parameters.conditions import ChoiceTest, holds
 
 GLOBAL_CANDIDATES = 1000  # random candidates per search of the model
 LOCAL_CANDIDATES = 500  # candidates near the best points found so far
 REFINED = 5  # the best candidates are refined by gradient search
 REFIT_EVERY = 5  # past FIT_EVERY_UNTIL observations, refit the model's hyperparameters only this often
 FIT_EVERY_UNTIL = 50
+INACTIVE = 0.5  # where a range switched off by a condition is encoded
 
 
 class BayesianOptimizer(Optimizer):
@@ -46,6 +51,7 @@ class BayesianOptimizer(Optimizer):
     """
     config_class = BayesianConfig
     supports_constraints = True
+    supports_conditions = True
     config: BayesianConfig
 
     def __init__(
@@ -209,6 +215,8 @@ class _Encoding:
 
     def __init__(self, search_space: SearchSpace) -> None:
         self.names = search_space.names  # not the search space itself, whose function constraints may not pickle
+        # The conditions alone, to tell which parameters a point switches off.
+        self._conditions = SearchSpace(search_space.parameters, conditions=search_space.conditions) if search_space.conditions else None
         self._blocks: list[tuple[RangeParameter | ChoiceParameter, slice]] = []
         self._constant: dict[str, Any] = {}
         column = 0
@@ -234,14 +242,42 @@ class _Encoding:
         self.numeric = np.array(numeric, dtype=int)
 
     def encode(self, params: dict) -> np.ndarray:
+        on = self._conditions.active(params) if self._conditions is not None else params
         x = np.zeros(self.columns)
         for p, block in self._blocks:
-            if isinstance(p, RangeParameter):
+            if p.name not in on:
+                x[block] = INACTIVE if isinstance(p, RangeParameter) else 0.0
+            elif isinstance(p, RangeParameter):
                 x[block] = p.to_unit(params[p.name])
             else:
                 x[block.start + p.choices.index(params[p.name])] = 1.0
         return x
 
+    def _switch_off(self, X: np.ndarray) -> np.ndarray:
+        """X with the columns of parameters each row switches off set as encode() sets them."""
+        if self._conditions is None:
+            return X
+        blocks = {p.name: (p, block) for p, block in self._blocks}
+        on: dict[str, np.ndarray] = {}
+        for parameter in self._conditions._order:
+            test = self._conditions._tests.get(parameter.name)
+            if test is None:
+                on[parameter.name] = np.ones(len(X), dtype=bool)
+                continue
+            mask = np.logical_and.reduce([on[name] for name in test.names()])
+            if isinstance(test, ChoiceTest):
+                if test.name in blocks:
+                    p, block = blocks[test.name]
+                    assert isinstance(p, ChoiceParameter)
+                    mask &= X[:, [block.start + p.choices.index(value) for value in test.values]].max(axis=1) > 0.5
+            else:
+                for row in np.flatnonzero(mask):
+                    mask[row] = holds(test, self.decode(X[row]))
+            on[parameter.name] = mask
+        for p, block in self._blocks:
+            X[~on[p.name], block] = INACTIVE if isinstance(p, RangeParameter) else 0.0
+        return X
+
     def decode(self, x: np.ndarray) -> dict:
         values = dict(self._constant)
         for p, block in self._blocks:
@@ -261,7 +297,7 @@ class _Encoding:
                 chosen = np.argmax(X[:, block], axis=1)
                 X[:, block] = 0.0
                 X[np.arange(len(X)), block.start + chosen] = 1.0
-        return X
+        return self._switch_off(X)
 
     def spread(self, rng: np.random.Generator, count: int) -> np.ndarray:
         """count points spread over the space: a Latin hypercube for numbers, random choices otherwise."""

{hone_python-0.10.0 → hone_python-0.11.0}/hone/optimizers/cmaes.py

@@ -39,7 +39,8 @@ class CMAES(Optimizer):
     interact, which makes it strong on smooth continuous problems.
 
     Range parameters, continuous or integer, are searched; fixed parameters are passed
-    through. Choice parameters are not supported: use GeneticOptimizer for those.
+    through. Choice parameters are not supported: use GeneticOptimizer for those. A range a
+    condition switches off keeps its place in the distribution; its value is ignored.
 
     The search starts at the centre of every range. With initial points, it evaluates them
     first and starts at the best one instead; a smaller initial_step then keeps the search
@@ -52,6 +53,7 @@ class CMAES(Optimizer):
     """
     config_class = CMAESConfig
     supports_constraints = True
+    supports_conditions = True
     config: CMAESConfig
 
     def __init__(