softopt 0.2.1__tar.gz → 0.4.0__tar.gz

{softopt-0.2.1/softopt.egg-info → softopt-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: softopt
-Version: 0.2.1
+Version: 0.4.0
 Summary: A standalone optimizer for problems with a known computation graph, built on Klein–Maimon soft-number calculus
 Author: Ido Angel
 License: MIT
@@ -58,19 +58,31 @@ pip install "softopt[torch]"   # + the PyTorch optimizer (quotes needed on zsh,
 ### NumPy
 
 ```python
-from softopt import SoftOpt
+import numpy as np
+from softopt import SoftOpt, soft_compile
 
-# g_delta(theta, delta) must return the EXACT directional derivative of
-# your true, differentiable objective along `delta`, at `theta` — computed
-# from your own known model (a circuit, a projection, a physical law),
-# not estimated from noisy measurements.
-opt = SoftOpt(n_params, g_delta, lr=0.02)
+# Write your model once, in plain Python. No special arithmetic.
+def my_model(theta):
+    x, y = theta
+    return (x - 2.0)**2 + np.exp(y) * x
+
+# soft_compile turns it into the exact directional-derivative function
+# SoftOpt needs, and verifies the result against your own model.
+g_delta = soft_compile(my_model, n_params=2)
+
+opt = SoftOpt(2, g_delta, lr=0.02)
 
 for step in range(num_steps):
     grad_estimate = my_gradient_estimate(theta)   # e.g. from SPSA on real hardware
     theta = opt.step(theta, grad_estimate)
 ```
 
+Your model needs to be built from arithmetic and the elementary functions the soft algebra defines: `+ - * / **` (including negative and fractional exponents, and `2.0 ** x`), `abs`, `sqrt`, `exp`, `log`, `log10`, `sin`, `cos`, `tanh`, comparisons, and numpy's versions of those. Loops and `if`/`else` branching on parameter values work too — the compiled derivative is then valid on the branch your parameters are actually in.
+
+The one thing to avoid is converting a traced value back to a plain number mid-model — `float(x)`, or `np.array(params)` on the parameter list, both silently discard the derivative. `soft_compile`'s verification catches this and says so.
+
+If you'd rather write the derivative function yourself — because your model lives in a simulator SoftOpt can't trace, or because you want the speed of a hand-tuned implementation — pass any `g_delta(theta, delta)` that returns the exact directional derivative, and skip `soft_compile` entirely.
+
 ### PyTorch
 
 ```python
@@ -84,15 +96,6 @@ for batch in data:
 
 `loss_fn(model, batch)` must be an exactly re-evaluatable, differentiable function of the model's own parameters — a full-batch loss, a physics simulator, a known projection model. SoftOpt uses PyTorch's own forward-mode autodiff (`torch.func.jvp`) to get the same exact directional derivative and curvature the NumPy version computes by hand.
 
-## Two correction modes
-
-SoftOpt ships with two ways of turning the computed derivative/curvature into a parameter update:
-
-- **`newton`** (default) — a bounded Newton step `t* = clip(-D1/D2, bounds)`. Best when the curvature (D2) is consistently one-signed along random directions — true for essentially every gradient-based physics/circuit optimization problem we tested (VQE, GRAPE, camera calibration, portfolio, ...).
-- **`mobius`** — a bounded, sign-safe step derived from the book's own Möbius map (Ch. 5.3), for problems whose curvature is *not* reliably one-signed. Pass `mode="mobius"` to `SoftOpt`/`SoftOptTorch` if `newton` underperforms plain Adam on your problem — that pattern is itself informative about your landscape's curvature.
-
-**How do I know which one to use?** Right now, empirically: run a short comparison against plain Adam with `mode="newton"` first; if it clearly loses, try `mode="mobius"`. There is also an experimental `mode="auto"` that samples curvature sign near your starting point and picks for you — we tested it honestly and it is **not yet reliable** (on GRAPE, a domain we know needs `newton` with high confidence, it only picked correctly 40% of the time across random starting points). It's included so you can inspect `opt.detected_mode` and help us characterize when it works, but don't depend on it yet.
-
 ## Validated results
 
 All results below use IBM's `FakeFez` noise model via Qiskit + Aer, or realistic finite-sample/measurement noise for the non-quantum domains, with `torch.optim.Adam` as the baseline. Improvement is the reduction in gap to the known optimum (or, for Portfolio, the reduction in loss).

{softopt-0.2.1 → softopt-0.4.0}/README.md

@@ -31,19 +31,31 @@ pip install "softopt[torch]"   # + the PyTorch optimizer (quotes needed on zsh,
 ### NumPy
 
 ```python
-from softopt import SoftOpt
+import numpy as np
+from softopt import SoftOpt, soft_compile
 
-# g_delta(theta, delta) must return the EXACT directional derivative of
-# your true, differentiable objective along `delta`, at `theta` — computed
-# from your own known model (a circuit, a projection, a physical law),
-# not estimated from noisy measurements.
-opt = SoftOpt(n_params, g_delta, lr=0.02)
+# Write your model once, in plain Python. No special arithmetic.
+def my_model(theta):
+    x, y = theta
+    return (x - 2.0)**2 + np.exp(y) * x
+
+# soft_compile turns it into the exact directional-derivative function
+# SoftOpt needs, and verifies the result against your own model.
+g_delta = soft_compile(my_model, n_params=2)
+
+opt = SoftOpt(2, g_delta, lr=0.02)
 
 for step in range(num_steps):
     grad_estimate = my_gradient_estimate(theta)   # e.g. from SPSA on real hardware
     theta = opt.step(theta, grad_estimate)
 ```
 
+Your model needs to be built from arithmetic and the elementary functions the soft algebra defines: `+ - * / **` (including negative and fractional exponents, and `2.0 ** x`), `abs`, `sqrt`, `exp`, `log`, `log10`, `sin`, `cos`, `tanh`, comparisons, and numpy's versions of those. Loops and `if`/`else` branching on parameter values work too — the compiled derivative is then valid on the branch your parameters are actually in.
+
+The one thing to avoid is converting a traced value back to a plain number mid-model — `float(x)`, or `np.array(params)` on the parameter list, both silently discard the derivative. `soft_compile`'s verification catches this and says so.
+
+If you'd rather write the derivative function yourself — because your model lives in a simulator SoftOpt can't trace, or because you want the speed of a hand-tuned implementation — pass any `g_delta(theta, delta)` that returns the exact directional derivative, and skip `soft_compile` entirely.
+
 ### PyTorch
 
 ```python
@@ -57,15 +69,6 @@ for batch in data:
 
 `loss_fn(model, batch)` must be an exactly re-evaluatable, differentiable function of the model's own parameters — a full-batch loss, a physics simulator, a known projection model. SoftOpt uses PyTorch's own forward-mode autodiff (`torch.func.jvp`) to get the same exact directional derivative and curvature the NumPy version computes by hand.
 
-## Two correction modes
-
-SoftOpt ships with two ways of turning the computed derivative/curvature into a parameter update:
-
-- **`newton`** (default) — a bounded Newton step `t* = clip(-D1/D2, bounds)`. Best when the curvature (D2) is consistently one-signed along random directions — true for essentially every gradient-based physics/circuit optimization problem we tested (VQE, GRAPE, camera calibration, portfolio, ...).
-- **`mobius`** — a bounded, sign-safe step derived from the book's own Möbius map (Ch. 5.3), for problems whose curvature is *not* reliably one-signed. Pass `mode="mobius"` to `SoftOpt`/`SoftOptTorch` if `newton` underperforms plain Adam on your problem — that pattern is itself informative about your landscape's curvature.
-
-**How do I know which one to use?** Right now, empirically: run a short comparison against plain Adam with `mode="newton"` first; if it clearly loses, try `mode="mobius"`. There is also an experimental `mode="auto"` that samples curvature sign near your starting point and picks for you — we tested it honestly and it is **not yet reliable** (on GRAPE, a domain we know needs `newton` with high confidence, it only picked correctly 40% of the time across random starting points). It's included so you can inspect `opt.detected_mode` and help us characterize when it works, but don't depend on it yet.
-
 ## Validated results
 
 All results below use IBM's `FakeFez` noise model via Qiskit + Aer, or realistic finite-sample/measurement noise for the non-quantum domains, with `torch.optim.Adam` as the baseline. Improvement is the reduction in gap to the known optimum (or, for Portfolio, the reduction in loss).

{softopt-0.2.1 → softopt-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "softopt"
-version = "0.2.1"
+version = "0.4.0"
 description = "A standalone optimizer for problems with a known computation graph, built on Klein–Maimon soft-number calculus"
 readme = "README.md"
 license = { text = "MIT" }

{softopt-0.2.1 → softopt-0.4.0}/softopt/__init__.py

@@ -5,23 +5,25 @@ calculus, for problems with a known, differentiable computation graph
 
     from softopt import SoftOpt          # numpy version
     from softopt import SoftOptTorch     # PyTorch version (torch.optim.Optimizer)
+    from softopt import soft_compile     # turn a plain-Python model into g_delta
     from softopt import SoftNumber       # the algebraic primitive itself
 
 See README.md for the exact requirement your problem must satisfy
 (the "known computation graph" criterion) and worked examples.
 """
 from .numpy_optimizer import SoftOpt
+from .compile import soft_compile
 from .soft_number import (
     SoftNumber, sadd, sneg, ssub, smul, sinv, sdiv,
-    spow, sroot, ssqrt, ssin, scos, sexp,
+    spow, sroot, ssqrt, ssin, scos, sexp, slog,
 )
 
 __all__ = [
-    "SoftOpt", "SoftNumber",
+    "SoftOpt", "SoftNumber", "soft_compile",
     "sadd", "sneg", "ssub", "smul", "sinv", "sdiv",
-    "spow", "sroot", "ssqrt", "ssin", "scos", "sexp",
+    "spow", "sroot", "ssqrt", "ssin", "scos", "sexp", "slog",
 ]
-__version__ = "0.2.1"
+__version__ = "0.4.0"
 
 try:
     from .torch_optimizer import SoftOpt as SoftOptTorch

softopt-0.4.0/softopt/compile.py

@@ -0,0 +1,101 @@
+"""
+soft_compile: turn a plain-Python model into the exact directional-
+derivative function (`g_delta`) that SoftOpt needs -- without writing any
+soft-number arithmetic yourself.
+
+    from softopt import SoftOpt, soft_compile
+
+    def my_model(theta):          # plain Python. numpy works too.
+        x, y = theta
+        return x**2 + np.exp(y) * x
+
+    g_delta = soft_compile(my_model, n_params=2)
+    opt = SoftOpt(2, g_delta, lr=0.02)
+
+Your model is traced once with SoftNumber objects in place of floats, so
+every operation carries its exact derivative alongside its value. This is
+exact -- not a finite-difference approximation.
+
+Your model must be built from arithmetic and the elementary functions the
+soft algebra defines (+ - * / ** sqrt exp sin cos, and numpy's versions of
+those). Branching on a parameter's *value* (`if x > 0:`) traces only the
+branch taken at that point, which is correct locally but means the
+compiled function is valid piecewise -- see `verify=True`.
+"""
+import numpy as np
+
+from .soft_number import SoftNumber
+
+
+def soft_compile(model_fn, n_params=None, verify=True, verify_tol=1e-5):
+    """model_fn: takes a sequence of scalars, returns a scalar.
+    n_params: needed only for verification.
+    verify: check the compiled derivative against finite differences at a
+        random point before returning it. Catches the common mistakes --
+        a model that isn't actually differentiable, uses an unsupported
+        operation, or silently drops the traced parameters."""
+    def g_delta(theta, delta):
+        soft = [SoftNumber(float(d), float(t)) for t, d in zip(theta, delta)]
+        result = model_fn(soft)
+        if not isinstance(result, SoftNumber):
+            raise TypeError(
+                "the compiled model returned a plain number, not a SoftNumber "
+                "-- its output doesn't depend on the traced parameters, or an "
+                "operation in it discarded them (e.g. float(), np.array(...), "
+                "or comparison-based rounding)"
+            )
+        return result.a
+
+    if verify:
+        if n_params is None:
+            raise ValueError("pass n_params to verify the compiled model, "
+                              "or verify=False to skip the check")
+        _verify(model_fn, g_delta, n_params, verify_tol)
+
+    return g_delta
+
+
+def _verify(model_fn, g_delta, n_params, tol):
+    """Compare the exact compiled derivative against a finite difference of
+    the same model evaluated with plain floats."""
+    rng = np.random.default_rng(0)
+    for attempt in range(5):
+        theta = rng.normal(0.5, 1.0, n_params)
+        direction = rng.normal(0, 1, n_params)
+        try:
+            exact = g_delta(theta, direction)
+        except ZeroDivisionError:
+            continue  # landed on a singular point; try another
+        h = 1e-6
+        try:
+            f_plus = float(model_fn(list(theta + h * direction)))
+            f_minus = float(model_fn(list(theta - h * direction)))
+        except Exception as e:
+            raise RuntimeError(
+                f"the model raised {type(e).__name__} when called with plain "
+                f"floats: {e}. soft_compile needs a model that works with both "
+                "floats and SoftNumbers."
+            ) from e
+        approx = (f_plus - f_minus) / (2 * h)
+
+        scale = max(abs(approx), abs(exact), 1.0)
+        if abs(exact - approx) / scale < tol:
+            return  # verified
+        if abs(approx) < 1e-12 and abs(exact) < 1e-12:
+            continue  # flat direction here; uninformative, try another
+
+        raise RuntimeError(
+            f"compiled derivative ({exact:.8g}) disagrees with a finite "
+            f"difference of your model ({approx:.8g}). Your model may use an "
+            "operation the soft algebra doesn't define, or may not be "
+            "differentiable at this point. Pass verify=False to skip this "
+            "check if you're confident it's correct."
+        )
+    # every attempt landed somewhere uninformative -- don't fail, but don't
+    # claim verification either
+    import warnings
+    warnings.warn(
+        "soft_compile could not verify the model: every test point was "
+        "either singular or flat. The compiled function is returned unchecked.",
+        RuntimeWarning,
+    )

softopt-0.4.0/softopt/numpy_optimizer.py

@@ -0,0 +1,98 @@
+"""
+SoftOpt: a COMPLETE, standalone optimizer -- not a wrapper around Adam.
+Exactly one step() call does both the Adam-style base update AND the
+validated Newton-style correction. You don't bring your own optimizer;
+this IS the optimizer, in the same sense that torch.optim.Adam IS an
+optimizer (internally momentum + RMS-scaling), not a wrapper around SGD.
+
+Requirement: you supply g_delta(theta, delta) -> the exact directional
+derivative of your true objective along delta, computed from your own
+known model (circuit, projection, physical law). `soft_compile` generates
+this for you from a model written in ordinary Python.
+
+The correction is a bounded Newton step derived from the exact first and
+second directional derivatives of your model.
+"""
+import numpy as np
+
+
+class SoftOpt:
+    def __init__(self, n_params, g_delta, lr=0.02, betas=(0.9, 0.999), eps=1e-8,
+                 newton_lo=-0.5, newton_hi=0.5, eta_fallback=0.3,
+                 h_fd=1e-4, seed=None, _test_frozen_constant=1.0):
+        self.g_delta = g_delta
+        self.lr = lr
+        self.b1, self.b2 = betas
+        self.eps = eps
+        self.newton_lo = newton_lo
+        self.newton_hi = newton_hi
+        self.eta_fallback = eta_fallback
+        self.h_fd = h_fd
+        self.m = None
+        self.v = None
+        self.t = 0
+        self.rng = np.random.default_rng(seed)
+        self._test_frozen_constant = _test_frozen_constant
+
+    def step(self, theta, grad_estimate, _test_magnitude_source=None, _test_foreign_sampler=None, _test_rng=None):
+        """One complete optimizer step: Adam base update + Newton-style
+        correction from the known model, in a single call.
+        theta: current parameters (np.ndarray)
+        grad_estimate: a gradient estimate of the loss w.r.t. theta (e.g.
+            from SPSA or backprop) -- exactly what you'd normally hand
+            to Adam.
+        Returns: theta for the next step.
+
+        The _test_* arguments are TEST-ONLY hooks used exclusively by the
+        causal-validation ablation harness to deliberately corrupt the
+        correction (feed a frozen constant or a foreign point's curvature
+        instead of the genuine one) so that "does real content matter"
+        can be checked against the EXACT SAME code path real usage takes.
+        Normal usage never sets these; leaving them unset gives the
+        genuine, real behavior.
+        """
+        theta = np.asarray(theta, dtype=float)
+        grad_estimate = np.asarray(grad_estimate, dtype=float)
+
+        # --- Adam base update ---
+        self.t += 1
+        if self.m is None:
+            self.m = np.zeros_like(theta)
+            self.v = np.zeros_like(theta)
+        self.m = self.b1 * self.m + (1 - self.b1) * grad_estimate
+        self.v = self.b2 * self.v + (1 - self.b2) * grad_estimate ** 2
+        mh = self.m / (1 - self.b1 ** self.t)
+        vh = self.v / (1 - self.b2 ** self.t)
+        theta_after_adam = theta - self.lr * mh / (np.sqrt(vh) + self.eps)
+
+        if _test_magnitude_source == 'plain':
+            return theta_after_adam
+
+        # --- correction from the known, exact model ---
+        n = theta.shape[0]
+        gen_rng = _test_rng if _test_rng is not None else self.rng
+        delta = gen_rng.choice([-1.0, 1.0], size=n)
+        D1 = self.g_delta(theta_after_adam, delta)
+        gp = self.g_delta(theta_after_adam + self.h_fd * delta, delta)
+        gm = self.g_delta(theta_after_adam - self.h_fd * delta, delta)
+        D2_real = (gp - gm) / (2 * self.h_fd)
+
+        if _test_magnitude_source in (None, 'real'):
+            D2_used = D2_real
+        elif _test_magnitude_source == 'frozen':
+            D2_used = self._test_frozen_constant
+        elif _test_magnitude_source in ('foreign_point_and_dir', 'foreign_point_same_dir'):
+            theta_f = _test_foreign_sampler(gen_rng)
+            delta_f = gen_rng.choice([-1.0, 1.0], size=n) if _test_magnitude_source == 'foreign_point_and_dir' else delta
+            gp_f = self.g_delta(theta_f + self.h_fd * delta_f, delta_f)
+            gm_f = self.g_delta(theta_f - self.h_fd * delta_f, delta_f)
+            D2_used = abs((gp_f - gm_f) / (2 * self.h_fd))
+        else:
+            raise ValueError(_test_magnitude_source)
+
+        if D2_real > 1e-8 and D2_used > 1e-9:
+            t_star = float(np.clip(-D1 / D2_used, self.newton_lo, self.newton_hi))
+        else:
+            t_star = float(np.clip(-self.eta_fallback * D1, self.newton_lo, self.newton_hi))
+
+        return theta_after_adam + t_star * delta

{softopt-0.2.1 → softopt-0.4.0}/softopt/soft_number.py

@@ -56,13 +56,21 @@ def sdiv(x, y):
 
 # --- Section 3.3.4: Lemma 3.3, integer powers ---
 def spow(x, n):
-    if not isinstance(n, int) or n < 0:
-        raise ValueError("spow implements Lemma 3.3 for non-negative "
-                          "integers only; use sroot(x, n) for Lemma 3.4/3.5")
+    """Integer powers via Lemma 3.3; fractional powers by composing
+    Lemma 3.3 with Lemma 3.5's root (e.g. x**2.5 = (x**5)**(1/2))."""
     a, b = x
-    if n == 0:
-        return (0.0, 1.0)
-    return (n * a * b ** (n - 1), b**n)
+    if isinstance(n, float) and n.is_integer():
+        n = int(n)
+    if isinstance(n, int):
+        if n < 0:
+            return sinv(spow(x, -n))
+        if n == 0:
+            return (0.0, 1.0)
+        return (n * a * b ** (n - 1), b**n)
+    # fractional: x**(p/q). Use the chain rule form directly -- for
+    # f(t)=t^n the extension rule gives (a*n*b^(n-1), b^n), which holds
+    # for real n as well, by Lemma 6.1's general f(a,b)=(a*f'(b), f(b)).
+    return (a * n * b ** (n - 1), b**n)
 
 
 # --- Section 3.3.5: Lemma 3.4 (square root), Lemma 3.5 (n-th root) ---
@@ -106,6 +114,14 @@ def sexp(x):
     return (a * e, e)
 
 
+def slog(x):
+    """Natural logarithm, Lemma 6.1's general rule with f(t)=ln(t)."""
+    a, b = x
+    if b <= 0:
+        raise ValueError("slog requires a positive real component")
+    return (a / b, np.log(b))
+
+
 class SoftNumber:
     """Object wrapper around the tuple functions above -- same math,
     operator syntax. See module docstring."""
@@ -157,8 +173,58 @@ class SoftNumber:
         return self._coerce(other) / self
 
     def __pow__(self, n):
+        if isinstance(n, SoftNumber):
+            # x**y with both soft: x**y = exp(y * ln x)
+            return (self.log() * n).exp()
         return self._from_tuple(spow(self.as_tuple(), n))
 
+    def __rpow__(self, base):
+        """base ** self, where base is an ordinary number.
+        d/dt base^t = base^t * ln(base)."""
+        if base <= 0:
+            raise ValueError(f"{base}**SoftNumber requires a positive base")
+        val = base ** self.b
+        return SoftNumber(self.a * val * np.log(base), val)
+
+    def log(self):
+        """Natural log, via Lemma 6.1's general rule with f=ln."""
+        return self._from_tuple(slog(self.as_tuple()))
+
+    def tanh(self):
+        t = np.tanh(self.b)
+        return SoftNumber(self.a * (1.0 - t * t), t)
+
+    def log10(self):
+        return SoftNumber(self.a / (self.b * np.log(10.0)), np.log10(self.b))
+
+    def __abs__(self):
+        """|x|: the real part determines the branch; the derivative follows
+        its sign (undefined exactly at 0, as for any absolute value)."""
+        if self.b < 0:
+            return -self
+        return SoftNumber(self.a, self.b)
+
+    # --- comparisons: ordered by the real component (book Definition 3.1,
+    # "Order": if a < b then a0 < b0). This lets models branch on parameter
+    # values normally; the compiled derivative is then valid on that branch.
+    def __lt__(self, other):
+        return self.b < self._coerce(other).b
+
+    def __le__(self, other):
+        return self.b <= self._coerce(other).b
+
+    def __gt__(self, other):
+        return self.b > self._coerce(other).b
+
+    def __ge__(self, other):
+        return self.b >= self._coerce(other).b
+
+    def __float__(self):
+        """The real component. Note this DISCARDS the derivative -- if your
+        model calls float() on a traced value, the compiled derivative will
+        be wrong, which soft_compile's verification is designed to catch."""
+        return float(self.b)
+
     def root(self, n):
         return self._from_tuple(sroot(self.as_tuple(), n))
 

{softopt-0.2.1 → softopt-0.4.0/softopt.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: softopt
-Version: 0.2.1
+Version: 0.4.0
 Summary: A standalone optimizer for problems with a known computation graph, built on Klein–Maimon soft-number calculus
 Author: Ido Angel
 License: MIT
@@ -58,19 +58,31 @@ pip install "softopt[torch]"   # + the PyTorch optimizer (quotes needed on zsh,
 ### NumPy
 
 ```python
-from softopt import SoftOpt
+import numpy as np
+from softopt import SoftOpt, soft_compile
 
-# g_delta(theta, delta) must return the EXACT directional derivative of
-# your true, differentiable objective along `delta`, at `theta` — computed
-# from your own known model (a circuit, a projection, a physical law),
-# not estimated from noisy measurements.
-opt = SoftOpt(n_params, g_delta, lr=0.02)
+# Write your model once, in plain Python. No special arithmetic.
+def my_model(theta):
+    x, y = theta
+    return (x - 2.0)**2 + np.exp(y) * x
+
+# soft_compile turns it into the exact directional-derivative function
+# SoftOpt needs, and verifies the result against your own model.
+g_delta = soft_compile(my_model, n_params=2)
+
+opt = SoftOpt(2, g_delta, lr=0.02)
 
 for step in range(num_steps):
     grad_estimate = my_gradient_estimate(theta)   # e.g. from SPSA on real hardware
     theta = opt.step(theta, grad_estimate)
 ```
 
+Your model needs to be built from arithmetic and the elementary functions the soft algebra defines: `+ - * / **` (including negative and fractional exponents, and `2.0 ** x`), `abs`, `sqrt`, `exp`, `log`, `log10`, `sin`, `cos`, `tanh`, comparisons, and numpy's versions of those. Loops and `if`/`else` branching on parameter values work too — the compiled derivative is then valid on the branch your parameters are actually in.
+
+The one thing to avoid is converting a traced value back to a plain number mid-model — `float(x)`, or `np.array(params)` on the parameter list, both silently discard the derivative. `soft_compile`'s verification catches this and says so.
+
+If you'd rather write the derivative function yourself — because your model lives in a simulator SoftOpt can't trace, or because you want the speed of a hand-tuned implementation — pass any `g_delta(theta, delta)` that returns the exact directional derivative, and skip `soft_compile` entirely.
+
 ### PyTorch
 
 ```python
@@ -84,15 +96,6 @@ for batch in data:
 
 `loss_fn(model, batch)` must be an exactly re-evaluatable, differentiable function of the model's own parameters — a full-batch loss, a physics simulator, a known projection model. SoftOpt uses PyTorch's own forward-mode autodiff (`torch.func.jvp`) to get the same exact directional derivative and curvature the NumPy version computes by hand.
 
-## Two correction modes
-
-SoftOpt ships with two ways of turning the computed derivative/curvature into a parameter update:
-
-- **`newton`** (default) — a bounded Newton step `t* = clip(-D1/D2, bounds)`. Best when the curvature (D2) is consistently one-signed along random directions — true for essentially every gradient-based physics/circuit optimization problem we tested (VQE, GRAPE, camera calibration, portfolio, ...).
-- **`mobius`** — a bounded, sign-safe step derived from the book's own Möbius map (Ch. 5.3), for problems whose curvature is *not* reliably one-signed. Pass `mode="mobius"` to `SoftOpt`/`SoftOptTorch` if `newton` underperforms plain Adam on your problem — that pattern is itself informative about your landscape's curvature.
-
-**How do I know which one to use?** Right now, empirically: run a short comparison against plain Adam with `mode="newton"` first; if it clearly loses, try `mode="mobius"`. There is also an experimental `mode="auto"` that samples curvature sign near your starting point and picks for you — we tested it honestly and it is **not yet reliable** (on GRAPE, a domain we know needs `newton` with high confidence, it only picked correctly 40% of the time across random starting points). It's included so you can inspect `opt.detected_mode` and help us characterize when it works, but don't depend on it yet.
-
 ## Validated results
 
 All results below use IBM's `FakeFez` noise model via Qiskit + Aer, or realistic finite-sample/measurement noise for the non-quantum domains, with `torch.optim.Adam` as the baseline. Improvement is the reduction in gap to the known optimum (or, for Portfolio, the reduction in loss).

{softopt-0.2.1 → softopt-0.4.0}/softopt.egg-info/SOURCES.txt

@@ -2,6 +2,7 @@ LICENSE
 README.md
 pyproject.toml
 softopt/__init__.py
+softopt/compile.py
 softopt/numpy_optimizer.py
 softopt/soft_number.py
 softopt/torch_optimizer.py
@@ -11,4 +12,5 @@ softopt.egg-info/dependency_links.txt
 softopt.egg-info/requires.txt
 softopt.egg-info/top_level.txt
 tests/test_basic.py
+tests/test_compile.py
 tests/test_soft_number.py

{softopt-0.2.1 → softopt-0.4.0}/tests/test_basic.py

@@ -14,7 +14,7 @@ def quadratic_g_delta(theta, delta):
     return float(2 * theta @ delta)
 
 
-def test_newton_mode_converges_on_convex_quadratic():
+def test_converges_on_convex_quadratic():
     opt = SoftOpt(5, quadratic_g_delta, lr=0.1, seed=0)
     theta = np.ones(5)
     for _ in range(200):
@@ -23,20 +23,6 @@ def test_newton_mode_converges_on_convex_quadratic():
     assert np.allclose(theta, 0.0, atol=1e-2)
 
 
-def test_mobius_mode_runs_without_error():
-    opt = SoftOpt(5, quadratic_g_delta, lr=0.1, mode="mobius", seed=0)
-    theta = np.ones(5)
-    for _ in range(50):
-        grad = 2 * theta
-        theta = opt.step(theta, grad)
-    assert np.all(np.isfinite(theta))
-
-
-def test_invalid_mode_raises():
-    with pytest.raises(ValueError):
-        SoftOpt(5, quadratic_g_delta, mode="not_a_real_mode")
-
-
 def test_step_is_deterministic_given_seed():
     opt1 = SoftOpt(3, quadratic_g_delta, lr=0.05, seed=42)
     opt2 = SoftOpt(3, quadratic_g_delta, lr=0.05, seed=42)

softopt-0.4.0/tests/test_compile.py

@@ -0,0 +1,147 @@
+"""
+Tests for soft_compile -- generating g_delta from a plain-Python model.
+"""
+import numpy as np
+import pytest
+from softopt import soft_compile, SoftOpt
+
+
+def test_polynomial_matches_finite_difference():
+    def model(p):
+        x, y, z = p
+        return x * x + y * y * y + x * y * z
+
+    g = soft_compile(model, n_params=3)
+    theta = np.array([1.3, -0.7, 2.1])
+    direction = np.array([1.0, -1.0, 1.0])
+    h = 1e-6
+    approx = (model(list(theta + h * direction)) - model(list(theta - h * direction))) / (2 * h)
+    assert np.isclose(g(theta, direction), approx, rtol=1e-6)
+
+
+def test_elementary_functions_work():
+    def model(p):
+        x, y = p
+        return np.exp(x) * np.sin(y) + np.sqrt(x * x + 1.0)
+
+    g = soft_compile(model, n_params=2)
+    theta = np.array([0.4, 1.1])
+    direction = np.array([1.0, -1.0])
+    h = 1e-6
+    approx = (model(list(theta + h * direction)) - model(list(theta - h * direction))) / (2 * h)
+    assert np.isclose(g(theta, direction), approx, rtol=1e-5)
+
+
+def test_fractional_and_negative_powers():
+    def model(p):
+        x, = p
+        return x ** 2.5 + x ** -1
+
+    g = soft_compile(model, n_params=1)
+    theta = np.array([1.7])
+    direction = np.array([1.0])
+    h = 1e-6
+    approx = (model(list(theta + h * direction)) - model(list(theta - h * direction))) / (2 * h)
+    assert np.isclose(g(theta, direction), approx, rtol=1e-5)
+
+
+def test_model_ignoring_params_is_rejected():
+    def model(p):
+        return 5.0
+
+    with pytest.raises(TypeError, match="doesn't depend on the traced parameters"):
+        soft_compile(model, n_params=2)
+
+
+def test_verify_requires_n_params():
+    with pytest.raises(ValueError, match="pass n_params"):
+        soft_compile(lambda p: p[0] * p[0])
+
+
+def test_verify_can_be_skipped():
+    g = soft_compile(lambda p: p[0] * p[0], verify=False)
+    assert np.isclose(g(np.array([3.0]), np.array([1.0])), 6.0)
+
+
+def test_compiled_model_drives_the_optimizer():
+    """End-to-end: compile a model, optimize with it, reach the minimum."""
+    def model(p):
+        x, y = p
+        return (x - 2.0) ** 2 + (y + 1.0) ** 2
+
+    g = soft_compile(model, n_params=2)
+    opt = SoftOpt(2, g, lr=0.1, seed=0)
+    theta = np.array([0.0, 0.0])
+    for _ in range(300):
+        grad = np.array([2 * (theta[0] - 2.0), 2 * (theta[1] + 1.0)])
+        theta = opt.step(theta, grad)
+    assert np.allclose(theta, [2.0, -1.0], atol=1e-2)
+
+
+def _fd_check(model, n, seed=77):
+    from softopt import soft_compile
+    g = soft_compile(model, n_params=n)
+    rng = np.random.default_rng(seed)
+    th = rng.normal(0.4, 0.9, n)
+    dl = rng.normal(0, 1, n)
+    exact = g(th, dl)
+    h = 1e-6
+    fd = (float(model(list(th + h * dl))) - float(model(list(th - h * dl)))) / (2 * h)
+    scale = max(abs(fd), abs(exact), 1.0)
+    assert abs(exact - fd) / scale < 1e-4, f"exact={exact} fd={fd}"
+
+
+def test_loops_over_parameters():
+    def model(p):
+        x, = p
+        total = x * 0.0
+        for i in range(5):
+            total = total + x ** (i + 1) / (i + 1)
+        return total
+    _fd_check(model, 1)
+
+
+def test_branching_on_parameter_value():
+    def model(p):
+        x, = p
+        return x * x if x > 0 else -x * x * x
+    _fd_check(model, 1)
+
+
+def test_abs_and_max():
+    def model(p):
+        x, y = p
+        return abs(x) + max(y, 0.0)
+    _fd_check(model, 2)
+
+
+def test_numpy_log_and_tanh():
+    def model(p):
+        x, = p
+        return np.log(np.exp(x) + 2.0) + np.tanh(x)
+    _fd_check(model, 1)
+
+
+def test_base_to_the_power_of_a_parameter():
+    def model(p):
+        x, = p
+        return 2.0 ** x + np.e ** x
+    _fd_check(model, 1)
+
+
+def test_object_array_quadratic_form():
+    def model(p):
+        v = np.array(p, dtype=object)
+        M = np.array([[1.0, 0.5], [0.5, 2.0]])
+        return sum(v[i] * M[i, j] * v[j] for i in range(2) for j in range(2))
+    _fd_check(model, 2)
+
+
+def test_model_that_casts_to_float_is_caught():
+    """np.array(p) on a list of SoftNumbers silently converts to floats,
+    dropping the derivative -- verification must catch this."""
+    def model(p):
+        v = np.array(p)      # drops the soft components
+        return float(v @ v)
+    with pytest.raises(TypeError, match="doesn't depend on the traced parameters"):
+        soft_compile(model, n_params=2)

softopt-0.2.1/softopt/numpy_optimizer.py

@@ -1,168 +0,0 @@
-"""
-SoftOpt: a COMPLETE, standalone optimizer -- not a wrapper around Adam.
-Exactly one step() call does both the Adam-style base update AND the
-validated Newton-style correction. You don't bring your own optimizer;
-this IS the optimizer, in the same sense that torch.optim.Adam IS an
-optimizer (internally momentum + RMS-scaling), not a wrapper around SGD.
-
-Requirement (the "known computation graph" criterion, validated across
-nine domains today): you must supply g_delta(theta, delta) -> the exact
-directional derivative of your TRUE, differentiable objective along
-delta, computed via Klein-Maimon soft-number propagation through your
-own known model (circuit, projection, force field, ...). If you cannot
-write g_delta without sampling the world, SoftOpt is not the right tool
--- use plain Adam/SGD instead.
-
-Three correction modes (see README for details):
-- mode="newton" (default): bounded Newton step, best for one-signed
-  curvature (most physics/circuit problems).
-- mode="mobius": bounded, sign-safe Mobius-map step (book Ch.5.3), for
-  landscapes whose curvature isn't reliably one-signed (e.g. QAOA).
-- mode="auto" (EXPERIMENTAL, not yet reliable -- see README): attempts
-  to diagnose which of the above fits your problem by sampling D2's
-  sign near the starting point. Tested against GRAPE (a domain that
-  needs "newton" with high confidence) it only picked "newton" 40% of
-  the time -- a single starting region's local curvature sign is not
-  a reliable predictor of the right mode for the whole optimization.
-  Shipped anyway, clearly labeled, so you can inspect `opt.detected_mode`
-  and help characterize when it does/doesn't work, but do not rely on
-  it yet -- the empirically-validated way to choose is still: try both
-  modes on a short run and keep whichever beats plain Adam.
-"""
-import numpy as np
-
-
-def _mobius_B(x, y):
-    """Book Ch.5.3's Möbius map, bounded to [-1,1] -- a sign-safe
-    alternative to raw Newton division for landscapes whose curvature
-    isn't reliably one-signed (e.g. QAOA, empirically)."""
-    denom = abs(x) + abs(y)
-    if denom < 1e-12:
-        return 0.0
-    sgn = 1.0 if x >= 0 else -1.0
-    return y * sgn / denom
-
-
-class SoftOpt:
-    def __init__(self, n_params, g_delta, lr=0.02, betas=(0.9, 0.999), eps=1e-8,
-                 mode="newton", auto_detect_steps=30, auto_detect_threshold=0.8,
-                 newton_lo=-0.5, newton_hi=0.5, eta_fallback=0.3,
-                 eta_mobius=0.3, h_fd=1e-4, seed=None,
-                 _test_frozen_constant=1.0):
-        self.g_delta = g_delta
-        self.lr = lr
-        self.b1, self.b2 = betas
-        self.eps = eps
-        if mode not in ("newton", "mobius", "auto"):
-            raise ValueError("mode must be 'newton', 'mobius', or 'auto'")
-        self.mode = mode
-        self.auto_detect_steps = auto_detect_steps
-        self.auto_detect_threshold = auto_detect_threshold
-        self._auto_pending = (mode == "auto")
-        self.detected_mode = None if mode == "auto" else mode
-        self.newton_lo = newton_lo
-        self.newton_hi = newton_hi
-        self.eta_fallback = eta_fallback
-        self.eta_mobius = eta_mobius
-        self.h_fd = h_fd
-        self.m = None
-        self.v = None
-        self.t = 0
-        self.rng = np.random.default_rng(seed)
-        self._test_frozen_constant = _test_frozen_constant
-
-    def _diagnose_mode(self, theta0):
-        """Run once, before any optimization step: sample D2's sign
-        across several nearby points (not just theta0 itself -- a single
-        point's local curvature sign is not always representative) and
-        several random directions per point, matching the methodology
-        that correctly distinguished GRAPE/VQE's one-signed curvature
-        from QAOA's sign-indefinite curvature."""
-        n = theta0.shape[0]
-        n_points = 5
-        dirs_per_point = max(1, self.auto_detect_steps // n_points)
-        signs = []
-        for p in range(n_points):
-            point = theta0 if p == 0 else theta0 + self.rng.normal(scale=0.1, size=n)
-            for _ in range(dirs_per_point):
-                delta = self.rng.choice([-1.0, 1.0], size=n)
-                gp = self.g_delta(point + self.h_fd * delta, delta)
-                gm = self.g_delta(point - self.h_fd * delta, delta)
-                D2 = (gp - gm) / (2 * self.h_fd)
-                signs.append(1.0 if D2 >= 0 else -1.0)
-        signs = np.array(signs)
-        agreement = max(np.mean(signs > 0), np.mean(signs < 0))
-        self.detected_mode = "newton" if agreement >= self.auto_detect_threshold else "mobius"
-        self.mode = self.detected_mode
-
-    def step(self, theta, grad_estimate, _test_magnitude_source=None, _test_foreign_sampler=None, _test_rng=None):
-        """One complete optimizer step: Adam base update + Newton-style
-        correction from the known model, in a single call.
-        theta: current parameters (np.ndarray)
-        grad_estimate: a gradient estimate of the loss w.r.t. theta (e.g.
-            from SPSA or backprop) -- exactly what you'd normally hand
-            to Adam.
-        Returns: theta for the next step.
-
-        The _test_* arguments are TEST-ONLY hooks used exclusively by the
-        causal-validation ablation harness to deliberately corrupt the
-        correction (feed a frozen constant or a foreign point's curvature
-        instead of the genuine one) so that "does real content matter"
-        can be checked against the EXACT SAME code path real usage takes.
-        Normal usage never sets these; leaving them unset gives the
-        genuine, real behavior.
-        """
-        theta = np.asarray(theta, dtype=float)
-        grad_estimate = np.asarray(grad_estimate, dtype=float)
-
-        if self._auto_pending:
-            self._diagnose_mode(theta)
-            self._auto_pending = False
-
-        # --- Adam base update ---
-        self.t += 1
-        if self.m is None:
-            self.m = np.zeros_like(theta)
-            self.v = np.zeros_like(theta)
-        self.m = self.b1 * self.m + (1 - self.b1) * grad_estimate
-        self.v = self.b2 * self.v + (1 - self.b2) * grad_estimate ** 2
-        mh = self.m / (1 - self.b1 ** self.t)
-        vh = self.v / (1 - self.b2 ** self.t)
-        theta_after_adam = theta - self.lr * mh / (np.sqrt(vh) + self.eps)
-
-        if _test_magnitude_source == 'plain':
-            return theta_after_adam
-
-        # --- correction from the known, exact model ---
-        n = theta.shape[0]
-        gen_rng = _test_rng if _test_rng is not None else self.rng
-        delta = gen_rng.choice([-1.0, 1.0], size=n)
-        D1 = self.g_delta(theta_after_adam, delta)
-        gp = self.g_delta(theta_after_adam + self.h_fd * delta, delta)
-        gm = self.g_delta(theta_after_adam - self.h_fd * delta, delta)
-        D2_real = (gp - gm) / (2 * self.h_fd)
-
-        if _test_magnitude_source in (None, 'real'):
-            D2_used = D2_real
-        elif _test_magnitude_source == 'frozen':
-            D2_used = self._test_frozen_constant
-        elif _test_magnitude_source in ('foreign_point_and_dir', 'foreign_point_same_dir'):
-            theta_f = _test_foreign_sampler(gen_rng)
-            delta_f = gen_rng.choice([-1.0, 1.0], size=n) if _test_magnitude_source == 'foreign_point_and_dir' else delta
-            gp_f = self.g_delta(theta_f + self.h_fd * delta_f, delta_f)
-            gm_f = self.g_delta(theta_f - self.h_fd * delta_f, delta_f)
-            D2_used = abs((gp_f - gm_f) / (2 * self.h_fd))
-        else:
-            raise ValueError(_test_magnitude_source)
-
-        if self.mode == "mobius":
-            B = _mobius_B(D1, D2_used)
-            t_star = self.eta_mobius * B
-        else:
-            activate = D2_real > 1e-8
-            if activate and D2_used > 1e-9:
-                t_star = float(np.clip(-D1 / D2_used, self.newton_lo, self.newton_hi))
-            else:
-                t_star = float(np.clip(-self.eta_fallback * D1, self.newton_lo, self.newton_hi))
-
-        return theta_after_adam + t_star * delta