softopt 0.3.0__tar.gz → 0.5.0__tar.gz

{softopt-0.3.0/softopt.egg-info → softopt-0.5.0}/PKG-INFO

@@ -1,13 +1,13 @@
 Metadata-Version: 2.4
 Name: softopt
-Version: 0.3.0
+Version: 0.5.0
 Summary: A standalone optimizer for problems with a known computation graph, built on Klein–Maimon soft-number calculus
 Author: Ido Angel
 License: MIT
 Project-URL: Homepage, https://github.com/mobiuai/softopt
 Project-URL: Repository, https://github.com/mobiuai/softopt
 Project-URL: Issues, https://github.com/mobiuai/softopt/issues
-Keywords: optimizer,optimization,soft-logic,soft-numbers,quantum-computing,vqe,qaoa,pytorch
+Keywords: optimizer,optimization,soft-logic,soft-numbers,uncertainty,sensitivity,quantum-computing,vqe
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: MIT License
@@ -18,11 +18,8 @@ Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy>=1.20
-Provides-Extra: torch
-Requires-Dist: torch>=2.0; extra == "torch"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0; extra == "dev"
-Requires-Dist: torch>=2.0; extra == "dev"
 Dynamic: license-file
 
 # SoftOpt
@@ -49,14 +46,11 @@ Full reinforcement learning (or anything else with a continuously **moving targe
 ## Installation
 
 ```bash
-pip install softopt            # numpy version only
-pip install "softopt[torch]"   # + the PyTorch optimizer (quotes needed on zsh, macOS's default shell)
+pip install softopt
 ```
 
 ## Quick start
 
-### NumPy
-
 ```python
 import numpy as np
 from softopt import SoftOpt, soft_compile
@@ -83,28 +77,6 @@ The one thing to avoid is converting a traced value back to a plain number mid-m
 
 If you'd rather write the derivative function yourself — because your model lives in a simulator SoftOpt can't trace, or because you want the speed of a hand-tuned implementation — pass any `g_delta(theta, delta)` that returns the exact directional derivative, and skip `soft_compile` entirely.
 
-### PyTorch
-
-```python
-from softopt import SoftOptTorch
-
-opt = SoftOptTorch(model.parameters(), model, loss_fn, lr=1e-3)
-
-for batch in data:
-    loss = opt.step(batch)   # one call: backward + Adam update + soft-number correction
-```
-
-`loss_fn(model, batch)` must be an exactly re-evaluatable, differentiable function of the model's own parameters — a full-batch loss, a physics simulator, a known projection model. SoftOpt uses PyTorch's own forward-mode autodiff (`torch.func.jvp`) to get the same exact directional derivative and curvature the NumPy version computes by hand.
-
-## Two correction modes
-
-SoftOpt ships with two ways of turning the computed derivative/curvature into a parameter update:
-
-- **`newton`** (default) — a bounded Newton step `t* = clip(-D1/D2, bounds)`. Best when the curvature (D2) is consistently one-signed along random directions — true for essentially every gradient-based physics/circuit optimization problem we tested (VQE, GRAPE, camera calibration, portfolio, ...).
-- **`mobius`** — a bounded, sign-safe step derived from the book's own Möbius map (Ch. 5.3), for problems whose curvature is *not* reliably one-signed. Pass `mode="mobius"` to `SoftOpt`/`SoftOptTorch` if `newton` underperforms plain Adam on your problem — that pattern is itself informative about your landscape's curvature.
-
-**How do I know which one to use?** Right now, empirically: run a short comparison against plain Adam with `mode="newton"` first; if it clearly loses, try `mode="mobius"`. There is also an experimental `mode="auto"` that samples curvature sign near your starting point and picks for you — we tested it honestly and it is **not yet reliable** (on GRAPE, a domain we know needs `newton` with high confidence, it only picked correctly 40% of the time across random starting points). It's included so you can inspect `opt.detected_mode` and help us characterize when it works, but don't depend on it yet.
-
 ## Validated results
 
 All results below use IBM's `FakeFez` noise model via Qiskit + Aer, or realistic finite-sample/measurement noise for the non-quantum domains, with `torch.optim.Adam` as the baseline. Improvement is the reduction in gap to the known optimum (or, for Portfolio, the reduction in loss).

{softopt-0.3.0 → softopt-0.5.0}/README.md

@@ -22,14 +22,11 @@ Full reinforcement learning (or anything else with a continuously **moving targe
 ## Installation
 
 ```bash
-pip install softopt            # numpy version only
-pip install "softopt[torch]"   # + the PyTorch optimizer (quotes needed on zsh, macOS's default shell)
+pip install softopt
 ```
 
 ## Quick start
 
-### NumPy
-
 ```python
 import numpy as np
 from softopt import SoftOpt, soft_compile
@@ -56,28 +53,6 @@ The one thing to avoid is converting a traced value back to a plain number mid-m
 
 If you'd rather write the derivative function yourself — because your model lives in a simulator SoftOpt can't trace, or because you want the speed of a hand-tuned implementation — pass any `g_delta(theta, delta)` that returns the exact directional derivative, and skip `soft_compile` entirely.
 
-### PyTorch
-
-```python
-from softopt import SoftOptTorch
-
-opt = SoftOptTorch(model.parameters(), model, loss_fn, lr=1e-3)
-
-for batch in data:
-    loss = opt.step(batch)   # one call: backward + Adam update + soft-number correction
-```
-
-`loss_fn(model, batch)` must be an exactly re-evaluatable, differentiable function of the model's own parameters — a full-batch loss, a physics simulator, a known projection model. SoftOpt uses PyTorch's own forward-mode autodiff (`torch.func.jvp`) to get the same exact directional derivative and curvature the NumPy version computes by hand.
-
-## Two correction modes
-
-SoftOpt ships with two ways of turning the computed derivative/curvature into a parameter update:
-
-- **`newton`** (default) — a bounded Newton step `t* = clip(-D1/D2, bounds)`. Best when the curvature (D2) is consistently one-signed along random directions — true for essentially every gradient-based physics/circuit optimization problem we tested (VQE, GRAPE, camera calibration, portfolio, ...).
-- **`mobius`** — a bounded, sign-safe step derived from the book's own Möbius map (Ch. 5.3), for problems whose curvature is *not* reliably one-signed. Pass `mode="mobius"` to `SoftOpt`/`SoftOptTorch` if `newton` underperforms plain Adam on your problem — that pattern is itself informative about your landscape's curvature.
-
-**How do I know which one to use?** Right now, empirically: run a short comparison against plain Adam with `mode="newton"` first; if it clearly loses, try `mode="mobius"`. There is also an experimental `mode="auto"` that samples curvature sign near your starting point and picks for you — we tested it honestly and it is **not yet reliable** (on GRAPE, a domain we know needs `newton` with high confidence, it only picked correctly 40% of the time across random starting points). It's included so you can inspect `opt.detected_mode` and help us characterize when it works, but don't depend on it yet.
-
 ## Validated results
 
 All results below use IBM's `FakeFez` noise model via Qiskit + Aer, or realistic finite-sample/measurement noise for the non-quantum domains, with `torch.optim.Adam` as the baseline. Improvement is the reduction in gap to the known optimum (or, for Portfolio, the reduction in loss).

{softopt-0.3.0 → softopt-0.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "softopt"
-version = "0.3.0"
+version = "0.5.0"
 description = "A standalone optimizer for problems with a known computation graph, built on Klein–Maimon soft-number calculus"
 readme = "README.md"
 license = { text = "MIT" }
@@ -12,7 +12,7 @@ requires-python = ">=3.9"
 authors = [
     { name = "Ido Angel" }
 ]
-keywords = ["optimizer", "optimization", "soft-logic", "soft-numbers", "quantum-computing", "vqe", "qaoa", "pytorch"]
+keywords = ["optimizer", "optimization", "soft-logic", "soft-numbers", "uncertainty", "sensitivity", "quantum-computing", "vqe"]
 classifiers = [
     "Development Status :: 4 - Beta",
     "Intended Audience :: Science/Research",
@@ -26,8 +26,7 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
-torch = ["torch>=2.0"]
-dev = ["pytest>=7.0", "torch>=2.0"]
+dev = ["pytest>=7.0"]
 
 [project.urls]
 Homepage = "https://github.com/mobiuai/softopt"

{softopt-0.3.0 → softopt-0.5.0}/softopt/__init__.py

@@ -23,11 +23,4 @@ __all__ = [
     "sadd", "sneg", "ssub", "smul", "sinv", "sdiv",
     "spow", "sroot", "ssqrt", "ssin", "scos", "sexp", "slog",
 ]
-__version__ = "0.3.0"
-
-try:
-    from .torch_optimizer import SoftOpt as SoftOptTorch
-    __all__.append("SoftOptTorch")
-except ImportError:
-    # torch is an optional dependency (pip install softopt[torch])
-    pass
+__version__ = "0.5.0"

softopt-0.5.0/softopt/numpy_optimizer.py

@@ -0,0 +1,98 @@
+"""
+SoftOpt: a COMPLETE, standalone optimizer -- not a wrapper around Adam.
+Exactly one step() call does both the Adam-style base update AND the
+validated Newton-style correction. You don't bring your own optimizer;
+this IS the optimizer, in the same sense that torch.optim.Adam IS an
+optimizer (internally momentum + RMS-scaling), not a wrapper around SGD.
+
+Requirement: you supply g_delta(theta, delta) -> the exact directional
+derivative of your true objective along delta, computed from your own
+known model (circuit, projection, physical law). `soft_compile` generates
+this for you from a model written in ordinary Python.
+
+The correction is a bounded Newton step derived from the exact first and
+second directional derivatives of your model.
+"""
+import numpy as np
+
+
+class SoftOpt:
+    def __init__(self, n_params, g_delta, lr=0.02, betas=(0.9, 0.999), eps=1e-8,
+                 newton_lo=-0.5, newton_hi=0.5, eta_fallback=0.3,
+                 h_fd=1e-4, seed=None, _test_frozen_constant=1.0):
+        self.g_delta = g_delta
+        self.lr = lr
+        self.b1, self.b2 = betas
+        self.eps = eps
+        self.newton_lo = newton_lo
+        self.newton_hi = newton_hi
+        self.eta_fallback = eta_fallback
+        self.h_fd = h_fd
+        self.m = None
+        self.v = None
+        self.t = 0
+        self.rng = np.random.default_rng(seed)
+        self._test_frozen_constant = _test_frozen_constant
+
+    def step(self, theta, grad_estimate, _test_magnitude_source=None, _test_foreign_sampler=None, _test_rng=None):
+        """One complete optimizer step: Adam base update + Newton-style
+        correction from the known model, in a single call.
+        theta: current parameters (np.ndarray)
+        grad_estimate: a gradient estimate of the loss w.r.t. theta (e.g.
+            from SPSA or backprop) -- exactly what you'd normally hand
+            to Adam.
+        Returns: theta for the next step.
+
+        The _test_* arguments are TEST-ONLY hooks used exclusively by the
+        causal-validation ablation harness to deliberately corrupt the
+        correction (feed a frozen constant or a foreign point's curvature
+        instead of the genuine one) so that "does real content matter"
+        can be checked against the EXACT SAME code path real usage takes.
+        Normal usage never sets these; leaving them unset gives the
+        genuine, real behavior.
+        """
+        theta = np.asarray(theta, dtype=float)
+        grad_estimate = np.asarray(grad_estimate, dtype=float)
+
+        # --- Adam base update ---
+        self.t += 1
+        if self.m is None:
+            self.m = np.zeros_like(theta)
+            self.v = np.zeros_like(theta)
+        self.m = self.b1 * self.m + (1 - self.b1) * grad_estimate
+        self.v = self.b2 * self.v + (1 - self.b2) * grad_estimate ** 2
+        mh = self.m / (1 - self.b1 ** self.t)
+        vh = self.v / (1 - self.b2 ** self.t)
+        theta_after_adam = theta - self.lr * mh / (np.sqrt(vh) + self.eps)
+
+        if _test_magnitude_source == 'plain':
+            return theta_after_adam
+
+        # --- correction from the known, exact model ---
+        n = theta.shape[0]
+        gen_rng = _test_rng if _test_rng is not None else self.rng
+        delta = gen_rng.choice([-1.0, 1.0], size=n)
+        D1 = self.g_delta(theta_after_adam, delta)
+        gp = self.g_delta(theta_after_adam + self.h_fd * delta, delta)
+        gm = self.g_delta(theta_after_adam - self.h_fd * delta, delta)
+        D2_real = (gp - gm) / (2 * self.h_fd)
+
+        if _test_magnitude_source in (None, 'real'):
+            D2_used = D2_real
+        elif _test_magnitude_source == 'frozen':
+            D2_used = self._test_frozen_constant
+        elif _test_magnitude_source in ('foreign_point_and_dir', 'foreign_point_same_dir'):
+            theta_f = _test_foreign_sampler(gen_rng)
+            delta_f = gen_rng.choice([-1.0, 1.0], size=n) if _test_magnitude_source == 'foreign_point_and_dir' else delta
+            gp_f = self.g_delta(theta_f + self.h_fd * delta_f, delta_f)
+            gm_f = self.g_delta(theta_f - self.h_fd * delta_f, delta_f)
+            D2_used = abs((gp_f - gm_f) / (2 * self.h_fd))
+        else:
+            raise ValueError(_test_magnitude_source)
+
+        if D2_real > 1e-8 and D2_used > 1e-9:
+            t_star = float(np.clip(-D1 / D2_used, self.newton_lo, self.newton_hi))
+        else:
+            t_star = float(np.clip(-self.eta_fallback * D1, self.newton_lo, self.newton_hi))
+
+        return theta_after_adam + t_star * delta

{softopt-0.3.0 → softopt-0.5.0/softopt.egg-info}/PKG-INFO

@@ -1,13 +1,13 @@
 Metadata-Version: 2.4
 Name: softopt
-Version: 0.3.0
+Version: 0.5.0
 Summary: A standalone optimizer for problems with a known computation graph, built on Klein–Maimon soft-number calculus
 Author: Ido Angel
 License: MIT
 Project-URL: Homepage, https://github.com/mobiuai/softopt
 Project-URL: Repository, https://github.com/mobiuai/softopt
 Project-URL: Issues, https://github.com/mobiuai/softopt/issues
-Keywords: optimizer,optimization,soft-logic,soft-numbers,quantum-computing,vqe,qaoa,pytorch
+Keywords: optimizer,optimization,soft-logic,soft-numbers,uncertainty,sensitivity,quantum-computing,vqe
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: MIT License
@@ -18,11 +18,8 @@ Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy>=1.20
-Provides-Extra: torch
-Requires-Dist: torch>=2.0; extra == "torch"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0; extra == "dev"
-Requires-Dist: torch>=2.0; extra == "dev"
 Dynamic: license-file
 
 # SoftOpt
@@ -49,14 +46,11 @@ Full reinforcement learning (or anything else with a continuously **moving targe
 ## Installation
 
 ```bash
-pip install softopt            # numpy version only
-pip install "softopt[torch]"   # + the PyTorch optimizer (quotes needed on zsh, macOS's default shell)
+pip install softopt
 ```
 
 ## Quick start
 
-### NumPy
-
 ```python
 import numpy as np
 from softopt import SoftOpt, soft_compile
@@ -83,28 +77,6 @@ The one thing to avoid is converting a traced value back to a plain number mid-m
 
 If you'd rather write the derivative function yourself — because your model lives in a simulator SoftOpt can't trace, or because you want the speed of a hand-tuned implementation — pass any `g_delta(theta, delta)` that returns the exact directional derivative, and skip `soft_compile` entirely.
 
-### PyTorch
-
-```python
-from softopt import SoftOptTorch
-
-opt = SoftOptTorch(model.parameters(), model, loss_fn, lr=1e-3)
-
-for batch in data:
-    loss = opt.step(batch)   # one call: backward + Adam update + soft-number correction
-```
-
-`loss_fn(model, batch)` must be an exactly re-evaluatable, differentiable function of the model's own parameters — a full-batch loss, a physics simulator, a known projection model. SoftOpt uses PyTorch's own forward-mode autodiff (`torch.func.jvp`) to get the same exact directional derivative and curvature the NumPy version computes by hand.
-
-## Two correction modes
-
-SoftOpt ships with two ways of turning the computed derivative/curvature into a parameter update:
-
-- **`newton`** (default) — a bounded Newton step `t* = clip(-D1/D2, bounds)`. Best when the curvature (D2) is consistently one-signed along random directions — true for essentially every gradient-based physics/circuit optimization problem we tested (VQE, GRAPE, camera calibration, portfolio, ...).
-- **`mobius`** — a bounded, sign-safe step derived from the book's own Möbius map (Ch. 5.3), for problems whose curvature is *not* reliably one-signed. Pass `mode="mobius"` to `SoftOpt`/`SoftOptTorch` if `newton` underperforms plain Adam on your problem — that pattern is itself informative about your landscape's curvature.
-
-**How do I know which one to use?** Right now, empirically: run a short comparison against plain Adam with `mode="newton"` first; if it clearly loses, try `mode="mobius"`. There is also an experimental `mode="auto"` that samples curvature sign near your starting point and picks for you — we tested it honestly and it is **not yet reliable** (on GRAPE, a domain we know needs `newton` with high confidence, it only picked correctly 40% of the time across random starting points). It's included so you can inspect `opt.detected_mode` and help us characterize when it works, but don't depend on it yet.
-
 ## Validated results
 
 All results below use IBM's `FakeFez` noise model via Qiskit + Aer, or realistic finite-sample/measurement noise for the non-quantum domains, with `torch.optim.Adam` as the baseline. Improvement is the reduction in gap to the known optimum (or, for Portfolio, the reduction in loss).

{softopt-0.3.0 → softopt-0.5.0}/softopt.egg-info/SOURCES.txt

@@ -5,7 +5,6 @@ softopt/__init__.py
 softopt/compile.py
 softopt/numpy_optimizer.py
 softopt/soft_number.py
-softopt/torch_optimizer.py
 softopt.egg-info/PKG-INFO
 softopt.egg-info/SOURCES.txt
 softopt.egg-info/dependency_links.txt

{softopt-0.3.0 → softopt-0.5.0}/softopt.egg-info/requires.txt

@@ -2,7 +2,3 @@ numpy>=1.20
 
 [dev]
 pytest>=7.0
-torch>=2.0
-
-[torch]
-torch>=2.0

{softopt-0.3.0 → softopt-0.5.0}/tests/test_basic.py

@@ -14,7 +14,7 @@ def quadratic_g_delta(theta, delta):
     return float(2 * theta @ delta)
 
 
-def test_newton_mode_converges_on_convex_quadratic():
+def test_converges_on_convex_quadratic():
     opt = SoftOpt(5, quadratic_g_delta, lr=0.1, seed=0)
     theta = np.ones(5)
     for _ in range(200):
@@ -23,20 +23,6 @@ def test_newton_mode_converges_on_convex_quadratic():
     assert np.allclose(theta, 0.0, atol=1e-2)
 
 
-def test_mobius_mode_runs_without_error():
-    opt = SoftOpt(5, quadratic_g_delta, lr=0.1, mode="mobius", seed=0)
-    theta = np.ones(5)
-    for _ in range(50):
-        grad = 2 * theta
-        theta = opt.step(theta, grad)
-    assert np.all(np.isfinite(theta))
-
-
-def test_invalid_mode_raises():
-    with pytest.raises(ValueError):
-        SoftOpt(5, quadratic_g_delta, mode="not_a_real_mode")
-
-
 def test_step_is_deterministic_given_seed():
     opt1 = SoftOpt(3, quadratic_g_delta, lr=0.05, seed=42)
     opt2 = SoftOpt(3, quadratic_g_delta, lr=0.05, seed=42)
@@ -54,11 +40,3 @@ def test_plain_arm_skips_correction():
     theta = np.array([1.0, 1.0, 1.0])
     result = opt.step(theta, 2 * theta, _test_magnitude_source="plain")
     assert np.all(np.isfinite(result))
-
-
-def test_torch_optimizer_importable_if_torch_present():
-    try:
-        import torch  # noqa: F401
-    except ImportError:
-        pytest.skip("torch not installed")
-    from softopt import SoftOptTorch  # noqa: F401

softopt-0.3.0/softopt/numpy_optimizer.py

@@ -1,168 +0,0 @@
-"""
-SoftOpt: a COMPLETE, standalone optimizer -- not a wrapper around Adam.
-Exactly one step() call does both the Adam-style base update AND the
-validated Newton-style correction. You don't bring your own optimizer;
-this IS the optimizer, in the same sense that torch.optim.Adam IS an
-optimizer (internally momentum + RMS-scaling), not a wrapper around SGD.
-
-Requirement (the "known computation graph" criterion, validated across
-nine domains today): you must supply g_delta(theta, delta) -> the exact
-directional derivative of your TRUE, differentiable objective along
-delta, computed via Klein-Maimon soft-number propagation through your
-own known model (circuit, projection, force field, ...). If you cannot
-write g_delta without sampling the world, SoftOpt is not the right tool
--- use plain Adam/SGD instead.
-
-Three correction modes (see README for details):
-- mode="newton" (default): bounded Newton step, best for one-signed
-  curvature (most physics/circuit problems).
-- mode="mobius": bounded, sign-safe Mobius-map step (book Ch.5.3), for
-  landscapes whose curvature isn't reliably one-signed (e.g. QAOA).
-- mode="auto" (EXPERIMENTAL, not yet reliable -- see README): attempts
-  to diagnose which of the above fits your problem by sampling D2's
-  sign near the starting point. Tested against GRAPE (a domain that
-  needs "newton" with high confidence) it only picked "newton" 40% of
-  the time -- a single starting region's local curvature sign is not
-  a reliable predictor of the right mode for the whole optimization.
-  Shipped anyway, clearly labeled, so you can inspect `opt.detected_mode`
-  and help characterize when it does/doesn't work, but do not rely on
-  it yet -- the empirically-validated way to choose is still: try both
-  modes on a short run and keep whichever beats plain Adam.
-"""
-import numpy as np
-
-
-def _mobius_B(x, y):
-    """Book Ch.5.3's Möbius map, bounded to [-1,1] -- a sign-safe
-    alternative to raw Newton division for landscapes whose curvature
-    isn't reliably one-signed (e.g. QAOA, empirically)."""
-    denom = abs(x) + abs(y)
-    if denom < 1e-12:
-        return 0.0
-    sgn = 1.0 if x >= 0 else -1.0
-    return y * sgn / denom
-
-
-class SoftOpt:
-    def __init__(self, n_params, g_delta, lr=0.02, betas=(0.9, 0.999), eps=1e-8,
-                 mode="newton", auto_detect_steps=30, auto_detect_threshold=0.8,
-                 newton_lo=-0.5, newton_hi=0.5, eta_fallback=0.3,
-                 eta_mobius=0.3, h_fd=1e-4, seed=None,
-                 _test_frozen_constant=1.0):
-        self.g_delta = g_delta
-        self.lr = lr
-        self.b1, self.b2 = betas
-        self.eps = eps
-        if mode not in ("newton", "mobius", "auto"):
-            raise ValueError("mode must be 'newton', 'mobius', or 'auto'")
-        self.mode = mode
-        self.auto_detect_steps = auto_detect_steps
-        self.auto_detect_threshold = auto_detect_threshold
-        self._auto_pending = (mode == "auto")
-        self.detected_mode = None if mode == "auto" else mode
-        self.newton_lo = newton_lo
-        self.newton_hi = newton_hi
-        self.eta_fallback = eta_fallback
-        self.eta_mobius = eta_mobius
-        self.h_fd = h_fd
-        self.m = None
-        self.v = None
-        self.t = 0
-        self.rng = np.random.default_rng(seed)
-        self._test_frozen_constant = _test_frozen_constant
-
-    def _diagnose_mode(self, theta0):
-        """Run once, before any optimization step: sample D2's sign
-        across several nearby points (not just theta0 itself -- a single
-        point's local curvature sign is not always representative) and
-        several random directions per point, matching the methodology
-        that correctly distinguished GRAPE/VQE's one-signed curvature
-        from QAOA's sign-indefinite curvature."""
-        n = theta0.shape[0]
-        n_points = 5
-        dirs_per_point = max(1, self.auto_detect_steps // n_points)
-        signs = []
-        for p in range(n_points):
-            point = theta0 if p == 0 else theta0 + self.rng.normal(scale=0.1, size=n)
-            for _ in range(dirs_per_point):
-                delta = self.rng.choice([-1.0, 1.0], size=n)
-                gp = self.g_delta(point + self.h_fd * delta, delta)
-                gm = self.g_delta(point - self.h_fd * delta, delta)
-                D2 = (gp - gm) / (2 * self.h_fd)
-                signs.append(1.0 if D2 >= 0 else -1.0)
-        signs = np.array(signs)
-        agreement = max(np.mean(signs > 0), np.mean(signs < 0))
-        self.detected_mode = "newton" if agreement >= self.auto_detect_threshold else "mobius"
-        self.mode = self.detected_mode
-
-    def step(self, theta, grad_estimate, _test_magnitude_source=None, _test_foreign_sampler=None, _test_rng=None):
-        """One complete optimizer step: Adam base update + Newton-style
-        correction from the known model, in a single call.
-        theta: current parameters (np.ndarray)
-        grad_estimate: a gradient estimate of the loss w.r.t. theta (e.g.
-            from SPSA or backprop) -- exactly what you'd normally hand
-            to Adam.
-        Returns: theta for the next step.
-
-        The _test_* arguments are TEST-ONLY hooks used exclusively by the
-        causal-validation ablation harness to deliberately corrupt the
-        correction (feed a frozen constant or a foreign point's curvature
-        instead of the genuine one) so that "does real content matter"
-        can be checked against the EXACT SAME code path real usage takes.
-        Normal usage never sets these; leaving them unset gives the
-        genuine, real behavior.
-        """
-        theta = np.asarray(theta, dtype=float)
-        grad_estimate = np.asarray(grad_estimate, dtype=float)
-
-        if self._auto_pending:
-            self._diagnose_mode(theta)
-            self._auto_pending = False
-
-        # --- Adam base update ---
-        self.t += 1
-        if self.m is None:
-            self.m = np.zeros_like(theta)
-            self.v = np.zeros_like(theta)
-        self.m = self.b1 * self.m + (1 - self.b1) * grad_estimate
-        self.v = self.b2 * self.v + (1 - self.b2) * grad_estimate ** 2
-        mh = self.m / (1 - self.b1 ** self.t)
-        vh = self.v / (1 - self.b2 ** self.t)
-        theta_after_adam = theta - self.lr * mh / (np.sqrt(vh) + self.eps)
-
-        if _test_magnitude_source == 'plain':
-            return theta_after_adam
-
-        # --- correction from the known, exact model ---
-        n = theta.shape[0]
-        gen_rng = _test_rng if _test_rng is not None else self.rng
-        delta = gen_rng.choice([-1.0, 1.0], size=n)
-        D1 = self.g_delta(theta_after_adam, delta)
-        gp = self.g_delta(theta_after_adam + self.h_fd * delta, delta)
-        gm = self.g_delta(theta_after_adam - self.h_fd * delta, delta)
-        D2_real = (gp - gm) / (2 * self.h_fd)
-
-        if _test_magnitude_source in (None, 'real'):
-            D2_used = D2_real
-        elif _test_magnitude_source == 'frozen':
-            D2_used = self._test_frozen_constant
-        elif _test_magnitude_source in ('foreign_point_and_dir', 'foreign_point_same_dir'):
-            theta_f = _test_foreign_sampler(gen_rng)
-            delta_f = gen_rng.choice([-1.0, 1.0], size=n) if _test_magnitude_source == 'foreign_point_and_dir' else delta
-            gp_f = self.g_delta(theta_f + self.h_fd * delta_f, delta_f)
-            gm_f = self.g_delta(theta_f - self.h_fd * delta_f, delta_f)
-            D2_used = abs((gp_f - gm_f) / (2 * self.h_fd))
-        else:
-            raise ValueError(_test_magnitude_source)
-
-        if self.mode == "mobius":
-            B = _mobius_B(D1, D2_used)
-            t_star = self.eta_mobius * B
-        else:
-            activate = D2_real > 1e-8
-            if activate and D2_used > 1e-9:
-                t_star = float(np.clip(-D1 / D2_used, self.newton_lo, self.newton_hi))
-            else:
-                t_star = float(np.clip(-self.eta_fallback * D1, self.newton_lo, self.newton_hi))
-
-        return theta_after_adam + t_star * delta

softopt-0.3.0/softopt/torch_optimizer.py

@@ -1,150 +0,0 @@
-"""
-SoftOpt (PyTorch): a COMPLETE, standalone torch.optim.Optimizer -- not a
-wrapper. Use it exactly like torch.optim.Adam:
-
-    opt = SoftOpt(model.parameters(), model, loss_fn, lr=1e-3)
-    loss = opt.step(batch)
-
-One step() call does the Adam-style base update AND the Newton-style
-correction, using genuine forward-mode automatic differentiation
-(torch.func.jvp) to get an exact directional derivative and curvature of
-your loss along a random probe direction -- mathematically the same
-quantity a hand-written Klein-Maimon soft-number propagation would give
-(dual numbers are isomorphic to single-axis soft numbers per the book's
-own Appendix A.2), computed via PyTorch's own AD engine instead of
-hand-written soft-number code.
-
-Requirement (the "known computation graph" criterion, validated across
-nine domains): loss_fn(model, batch) must be an exactly re-evaluatable,
-differentiable function -- e.g. a full-batch loss over a fixed dataset,
-a physics/circuit simulator, a known projection model. If your loss is
-a small stochastic minibatch sample that changes meaning between calls
-(the classic deep-RL moving-target problem), this mechanism is not
-validated to help -- use plain Adam/SGD instead.
-"""
-import numpy as np
-import torch
-import torch.func as tfunc
-from torch.optim import Optimizer
-from torch.nn.utils.stateless import _reparametrize_module
-
-
-class SoftOpt(Optimizer):
-    def __init__(self, params, model, loss_fn, lr=1e-3, betas=(0.9, 0.999), eps=1e-8,
-                 newton_lo=-0.5, newton_hi=0.5, eta_fallback=0.3, h_fd=1e-4, seed=None,
-                 _test_frozen_constant=1.0):
-        defaults = dict(lr=lr, betas=betas, eps=eps)
-        super().__init__(params, defaults)
-        self.model = model
-        self.loss_fn = loss_fn
-        self.newton_lo = newton_lo
-        self.newton_hi = newton_hi
-        self.eta_fallback = eta_fallback
-        self.h_fd = h_fd
-        self.rng = np.random.default_rng(seed)
-        self._test_frozen_constant = _test_frozen_constant
-
-    def _adam_update(self):
-        for group in self.param_groups:
-            b1, b2 = group['betas']
-            for p in group['params']:
-                if p.grad is None:
-                    continue
-                state = self.state[p]
-                if len(state) == 0:
-                    state['step'] = 0
-                    state['exp_avg'] = torch.zeros_like(p)
-                    state['exp_avg_sq'] = torch.zeros_like(p)
-                state['step'] += 1
-                state['exp_avg'].mul_(b1).add_(p.grad, alpha=1 - b1)
-                state['exp_avg_sq'].mul_(b2).addcmul_(p.grad, p.grad, value=1 - b2)
-                bc1 = 1 - b1 ** state['step']
-                bc2 = 1 - b2 ** state['step']
-                denom = (state['exp_avg_sq'] / bc2).sqrt().add_(group['eps'])
-                p.data.addcdiv_(state['exp_avg'], denom, value=-group['lr'] / bc1)
-
-    def step(self, batch, _test_magnitude_source=None, _test_foreign_sampler=None):
-        """batch: whatever loss_fn(model, batch) needs. Returns the loss
-        (a float) from the pre-correction forward pass.
-        _test_* are TEST-ONLY hooks for the causal-validation harness --
-        normal usage never sets these.
-        """
-        all_params = [p for group in self.param_groups for p in group['params']]
-
-        # --- standard forward/backward for the Adam step ---
-        with torch.enable_grad():
-            self.zero_grad()
-            loss = self.loss_fn(self.model, batch)
-            loss.backward()
-        loss_value = float(loss.detach())
-
-        with torch.no_grad():
-            self._adam_update()
-
-        if _test_magnitude_source == 'plain':
-            return loss_value
-
-        # --- Newton-style correction via genuine forward-mode AD ---
-        with torch.no_grad():
-            named = {id(p): name for name, p in self.model.named_parameters()}
-            all_named = dict(self.model.named_parameters())
-            names = [n for n, p in all_named.items() if any(p is q for q in all_params)]
-            flat0 = torch.cat([all_named[n].detach().reshape(-1) for n in names])
-            n_total = flat0.numel()
-            delta = torch.tensor(self.rng.choice([-1.0, 1.0], size=n_total).astype(np.float32))
-
-            fixed_vals = {n: p.detach() for n, p in all_named.items() if n not in names}
-
-            def make_scalar_loss(the_batch):
-                def scalar_loss(flat):
-                    pd = {}
-                    idx = 0
-                    for n in names:
-                        p = all_named[n]
-                        nn_ = p.numel()
-                        pd[n] = flat[idx:idx + nn_].reshape(p.shape)
-                        idx += nn_
-                    pd.update(fixed_vals)
-                    with _reparametrize_module(self.model, pd):
-                        return self.loss_fn(self.model, the_batch)
-                return scalar_loss
-
-            def D1_D2_at(point_theta, point_delta, point_batch):
-                scalar_loss = make_scalar_loss(point_batch)
-                _, d1 = tfunc.jvp(scalar_loss, (point_theta,), (point_delta,))
-                def d_along(fp):
-                    _, d = tfunc.jvp(scalar_loss, (fp,), (point_delta,))
-                    return d
-                _, d2 = tfunc.jvp(d_along, (point_theta,), (point_delta,))
-                return float(d1), float(d2)
-
-            D1, D2_real = D1_D2_at(flat0, delta, batch)
-            activate = D2_real > 1e-8
-
-            if _test_magnitude_source in (None, 'real'):
-                D2_used = D2_real
-            elif _test_magnitude_source == 'frozen':
-                D2_used = self._test_frozen_constant
-            elif _test_magnitude_source in ('foreign_point_and_dir', 'foreign_point_same_dir'):
-                theta_f, batch_f = _test_foreign_sampler(self.rng)
-                delta_f = (torch.tensor(self.rng.choice([-1.0, 1.0], size=n_total).astype(np.float32))
-                           if _test_magnitude_source == 'foreign_point_and_dir' else delta)
-                _, D2_f = D1_D2_at(theta_f, delta_f, batch_f)
-                D2_used = abs(D2_f)
-            else:
-                raise ValueError(_test_magnitude_source)
-
-            if activate and D2_used > 1e-9:
-                t_star = float(np.clip(-D1 / D2_used, self.newton_lo, self.newton_hi))
-            else:
-                t_star = float(np.clip(-self.eta_fallback * D1, self.newton_lo, self.newton_hi))
-
-            new_flat = flat0 + t_star * delta
-            idx = 0
-            for n in names:
-                p = all_named[n]
-                nn_ = p.numel()
-                p.data.copy_(new_flat[idx:idx + nn_].reshape(p.shape))
-                idx += nn_
-
-        return loss_value