ulpwise 0.3.2__tar.gz → 0.3.3__tar.gz

{ulpwise-0.3.2 → ulpwise-0.3.3}/CHANGELOG.md

@@ -1,5 +1,60 @@
 # Changelog
 
+## 0.3.3 (2026-10-08)
+
+- Two regression corpus cases for peft, both open upstream with a fix PR: peft #3769, `add_weighted_adapter` with
+  `combination_type='svd'` crashed for Conv1d and Conv3d LoRA layers because only Conv2d deltas were flattened before
+  the SVD (the case combines two rank 2 adapters at `svd_rank=4` and expects the output of both adapters active, which
+  the fix matches to 2e-6), and peft #3830, OFT `module_dropout` was never applied, so ten training forwards of the
+  same input were identical where the fix spreads them by 0.56.
+- `ulpwise scan`: new `where-nan-gradient` rule (medium) for `where(d > eps, f(d), other)` with `f` a division by `d`
+  or a `sqrt`, `log`, `acos` or `asin` of it. `where` evaluates both branches and hands the discarded one a zero
+  gradient, and the backward of `f` at the singularity turns that zero into `0 / 0 = nan`, so the guard protects the
+  value and not the gradient. kornia #5579, found by the kornia conventions audit, is the example: the mutual
+  information losses divided by the range of the signal inside `where(diff > eps, ...)` and a constant input or
+  target got a nan gradient. The condition may be a comparison, a name bound to one, or `~`, `&`, `|` and a subscript
+  of those, and the method form `a.where(cond, b)` counts. Quiet for `numpy.where`, which has no autograd, for a
+  comparison against a number above 1 (`strength < 50` in kornia's JPEG scale is a piecewise definition), for a
+  floor named `eps`, `tol`, `floor`, `thresh`, `tiny`, `bound` or `limit` on the other side of the comparison, and
+  once `d` is re-bound to a `where`, `clamp` or `maximum` of itself, which is the recommended fix and the shape of
+  kornia's `fit_line` and `rotation_matrix_to_quaternion`. On kornia main d15741e2 the rule reports four lines, on
+  ultralytics main 8df3534 two, both in metric code, and nothing in torchvision.
+- `ulpwise scan`: new `clamp-at-singularity` rule (medium) for `clamp(x, min=0).sqrt()`, `sqrt(clamp(x, min=0))`,
+  `clamp(c, -1, 1).acos()` and the `clip`, `clamp_min`, `clamp_max`, `asin` and `log` forms: the bound is the point
+  where the next function has an infinite derivative, and clamp's derivative at its own bound is 1 on torch 2.5.1 and
+  2.9.1 and 0 on 2.14, so the gradient there is `inf` or `nan` on the older half of a supported range. kornia #4229,
+  found by the kornia conventions audit, is the example, `_cdist`'s `clamp(min=0.0).sqrt()` with identical float16
+  descriptors, and kornia #5500's `clamp(-1, 1).acos()` the second. A bound strictly inside the domain, `min=1e-8`,
+  is a floor and is not reported, a bound given by a name is not read, and numpy is quiet. On kornia main d15741e2
+  the rule reports the three lines of `_solve_cubic_real`, on ultralytics main 8df3534 one line of `DepthMetrics`.
+- `ulpwise scan`: a finding inside a nested function is reported once, under the nested function, instead of once
+  more under the enclosing one, and the enclosing function's call and division counts no longer include it.
+- `ulpwise scan`: new `eps-floor` rule (medium) for `x * (1 - eps) + eps`, in either order and with `self.eps` too: the
+  floor moves every value, 0 becomes `eps` and 1 stays 1, so the entries of a one-hot sum to `1 + (C - 1) eps` and a
+  perfect prediction scores a loss that grows with the image. kornia #5538, found by the kornia conventions audit, is
+  the example: `one_hot` floored its zeros with 1e-6 and a perfect prediction of a 256 × 384 image with a one-pixel
+  class scored a macro Dice loss of 0.0234. On kornia main 9e188e08 the rule reports that line and nothing else, and
+  nothing in ultralytics or torchvision; `(p + eps) / (q + eps)` is not a floor and stays quiet.
+- `ulpwise scan`: the `hypot-by-hand` advice no longer offers `norm` as the replacement. `torch.linalg.vector_norm` and
+  `numpy.linalg.norm` square the entries first too, so in float32 both return `inf` for a vector with two entries of
+  1.8e19 while `hypot` returns 2.5e19; the advice now says to divide a vector by its largest absolute entry when the
+  point is a direction. The kornia maintainer caught the case in review of kornia #5505, where `v / v.norm()` turned
+  such vectors into zeros and the angle between them read 0.
+- `ulpwise scan`: `acos-for-angle` moves from `info` to `medium` and names the mechanism: next to 1 the cosine has no
+  digits left for a small angle, so `acos` recovers the angle with an absolute error of `sqrt(eps)`, 0.02 degrees in
+  float32, whatever the angle. The kornia scan listed `kornia/metrics/pose.py` under this rule, and both functions
+  there, `angle_error_mat` and `angle_error_vec`, return exactly 0 for every rotation below 0.03 degrees in float32
+  and 180.0 for 179.99 degrees (kornia #5500); the rule cites it.
+- Regression corpus case for kornia #5500: a 0.01 degree rotation scores 0.0 in both metrics on kornia main 050ac77f.
+- The kornia #5500 corpus case records its fix, kornia #5505, merged 2026-10-06: `angle_error_mat` reads the sine from
+  the skew part of the relative rotation and `angle_error_vec` from the cross product of the scaled vectors, and both
+  return `atan2(sin, cos)`, so a 0.01 degree rotation scores 0.0099990 in float32. The case stays an expected failure
+  until a kornia release carries the fix.
+- `ulpwise scan`: `logsumexp-by-hand` no longer fires on the stable form, where every `exp` argument has its
+  maximum subtracted first (`log(exp(x - x.max()).sum())`, or `s = x - m` with `m` bound to a `max`, `amax`,
+  `maximum` or `torch.max(...).values`). The lm-evaluation-harness scan listed the numerically stable `_log_softmax`
+  of `lm_eval/models/_onnx_base.py` next to a real `log(exp(a) + exp(b))`; it now reports only the real one.
+
 ## 0.3.2 (2026-10-06)
 
 - Two regression corpus cases for pytorch #199850: `torch.erf` in bfloat16 and float16 on CPU returns 0 at and

{ulpwise-0.3.2 → ulpwise-0.3.3}/Cargo.lock

@@ -120,7 +120,7 @@ checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
 
 [[package]]
 name = "ulpwise"
-version = "0.3.2"
+version = "0.3.3"
 dependencies = [
  "pyo3",
 ]

{ulpwise-0.3.2 → ulpwise-0.3.3}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "ulpwise"
-version = "0.3.2"
+version = "0.3.3"
 edition = "2021"
 rust-version = "1.86"
 license = "MIT OR Apache-2.0"

{ulpwise-0.3.2 → ulpwise-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ulpwise
-Version: 0.3.2
+Version: 0.3.3
 Classifier: Development Status :: 3 - Alpha
 Classifier: Framework :: Pytest
 Classifier: Intended Audience :: Developers
@@ -228,6 +228,9 @@ PYTHONPATH=/tmp/kornia-0.8.3 ulpwise corpus --repo kornia     # 31 present, 0 fi
 | pytorch #198664 `erfcx` off by `x*x/2` ulps for negative `x` (`exp` at the rounded square) | rounding | open |
 | pytorch #199850 `torch.erf` in bfloat16 and float16 on CPU returns 0 at and below 1.8e-7 and loses relative accuracy below 1e-3 (13404 bfloat16 ulps, 5 float16 ulps) | cancellation | open |
 | pytorch #199867 `torch.special.logit` in float16 and bfloat16 on CPU rounds `1 - x` and `x / (1 - x)` to the input dtype before the log, logit(0.499756) is -0.000488 for -0.000977 (512 float16 ulps, 64 bfloat16 ulps) | rounding | open |
+| kornia #5505 `angle_error_mat` and `angle_error_vec` returned exactly 0 for every rotation below 0.03 degrees in float32 (`acos` next to 1, issue #5500) | cancellation | 2026-10-06 |
+| peft #3769 `add_weighted_adapter` with `combination_type='svd'` crashed for Conv1d and Conv3d LoRA layers | crash | open |
+| peft #3830 OFT `module_dropout` was never applied, every training forward used all rotation blocks | no-op | open |
 
 Cases marked `open` have an issue with the complete patch attached and no merged fix yet. Every case
 is an expected failure until the installed release contains the fix (`fixed_in_release` in
@@ -246,22 +249,26 @@ ulpwise scan path/to/repo --rules one-minus-cos,small-angle-division --top 40
 ```
 
 The scan is static and needs nothing installed: it parses each file with `ast`, walks every
-function and matches twelve patterns, each with a severity, the reason it loses digits or
-overflows, the usual replacement and, where one exists, the upstream bug it comes from.
+function and matches fifteen patterns, each with a severity, the reason it loses digits,
+overflows or loses its gradient, the usual replacement and, where one exists, the upstream bug it
+comes from.
 
 | rule | severity | pattern |
 |---|---|---|
 | `exp-of-square` | high | `exp(x * x)`, `exp(x ** 2)`, `(-x.pow(2)).exp()`: the rounding error of the square is multiplied by `x * x / 2` ulps (pytorch #198664) |
 | `sin-of-pi-times` | high | `sin(pi * x)`, `cos(pi * x)`: the product is rounded before the argument reduction (pytorch #198663) |
 | `softplus-by-hand` | high | `log(1 + exp(x))` |
-| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))` |
-| `hypot-by-hand` | high | `sqrt(a * a + b * b)` |
+| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))`; quiet when every `exp` argument has its maximum subtracted first, `log(exp(x - x.max()).sum())`, the stable form |
+| `hypot-by-hand` | high | `sqrt(a * a + b * b)`; the advice no longer says `norm`, which squares first too and overflows at 1.8e19 in float32 (kornia #5505 review) |
 | `sqrt-of-difference` | medium | `sqrt(a - b)` |
 | `one-minus-cos` | medium | `1 - cos(x)` (kornia #4897) |
 | `log1p-by-hand`, `expm1-by-hand` | medium | `log(1 + x)`, `exp(x) - 1` |
 | `atan-of-quotient` | medium | `atan(y / x)` |
 | `small-angle-division` | medium | `/ theta`, `/ theta ** 2`, `/ sin(theta)` in a function that takes `sin` or `cos` of `theta` and has no `where`, `clamp`, `eps` or series in sight (kornia #4838, #4897) |
-| `acos-for-angle` | info | `acos`, `asin` used to recover an angle |
+| `acos-for-angle` | medium | `acos`, `asin` used to recover an angle: the angle comes back with an absolute error of `sqrt(eps)`, 0.02 degrees in float32 (kornia #5500) |
+| `eps-floor` | medium | `x * (1 - eps) + eps`: every value moves, 0 becomes `eps`, so a one-hot's entries sum to `1 + (C - 1) eps` and a perfect prediction scores a loss that grows with the image (kornia #5538, found by the kornia conventions audit) |
+| `where-nan-gradient` | medium | `where(d > eps, f(d), other)` with `f` a division by `d` or a `sqrt`, `log`, `acos` or `asin` of it: `where` evaluates both branches and hands the discarded one a zero gradient, and the backward of `f` at the singularity turns that zero into `0 / 0 = nan`, so the guard protects the value and not the gradient; quiet for `numpy.where`, for a comparison against a number above 1, for a floor named `eps`, `tol` or `floor` on the other side, and once `d` is re-bound to a `where`, `clamp` or `maximum` of itself (kornia #5579, found by the kornia conventions audit) |
+| `clamp-at-singularity` | medium | `clamp(x, min=0).sqrt()`, `sqrt(clamp(x, min=0))`, `clamp(c, -1, 1).acos()`: the bound is the point where the next function has an infinite derivative, and clamp's derivative at its own bound is not the same across torch versions, 1 on 2.5.1 and 2.9.1 and 0 on 2.14, so the gradient there is `inf` or `nan` on the older half of a supported range; a bound strictly inside the domain, `min=1e-8`, is a floor and is not reported, and `numpy` is quiet (kornia #4229, found by the kornia conventions audit; kornia #5500) |
 
 On kornia `main` at `e05b0ee` the scan takes 4 s for 506 files and reports 35 findings. The
 `one-minus-cos` and `small-angle-division` findings are the four lines of `So3.right_jacobian` and
@@ -270,6 +277,13 @@ and `Se3.exp` (also #4897) is under `one-minus-cos`. The scan puts `ellipse_to_l
 on the list too, for a `sqrt` of a difference; the bug there was a different one, so that entry is
 what the scan is: a reading list, not a verdict.
 
+The two gradient rules are reading lists in the same sense. On kornia `main` at `d15741e2` they
+report seven lines: three `clamp` bounds in `_solve_cubic_real`, and four `where` guards, in
+`PatchDominantGradientOrientation`, `compute_correspond_epilines`, `_crop_scale_translation` and
+`_get_convex_edges`, of which the first two sit on a differentiable path and the last two draw
+boxes and polygons. On ultralytics `main` at `8df3534` they report three lines, all in metrics or
+inference code that is never differentiated, and nothing in torchvision.
+
 `--run` adds the dynamic half. The module level functions with the most elementary math are
 imported and called with the same 91 point grid (both signs of `1e-8` to `1e3`, and zero) for every
 required argument, in float32 and in float64, torch first and numpy second, and the float32 result

{ulpwise-0.3.2 → ulpwise-0.3.3}/README.md

@@ -199,6 +199,9 @@ PYTHONPATH=/tmp/kornia-0.8.3 ulpwise corpus --repo kornia     # 31 present, 0 fi
 | pytorch #198664 `erfcx` off by `x*x/2` ulps for negative `x` (`exp` at the rounded square) | rounding | open |
 | pytorch #199850 `torch.erf` in bfloat16 and float16 on CPU returns 0 at and below 1.8e-7 and loses relative accuracy below 1e-3 (13404 bfloat16 ulps, 5 float16 ulps) | cancellation | open |
 | pytorch #199867 `torch.special.logit` in float16 and bfloat16 on CPU rounds `1 - x` and `x / (1 - x)` to the input dtype before the log, logit(0.499756) is -0.000488 for -0.000977 (512 float16 ulps, 64 bfloat16 ulps) | rounding | open |
+| kornia #5505 `angle_error_mat` and `angle_error_vec` returned exactly 0 for every rotation below 0.03 degrees in float32 (`acos` next to 1, issue #5500) | cancellation | 2026-10-06 |
+| peft #3769 `add_weighted_adapter` with `combination_type='svd'` crashed for Conv1d and Conv3d LoRA layers | crash | open |
+| peft #3830 OFT `module_dropout` was never applied, every training forward used all rotation blocks | no-op | open |
 
 Cases marked `open` have an issue with the complete patch attached and no merged fix yet. Every case
 is an expected failure until the installed release contains the fix (`fixed_in_release` in
@@ -217,22 +220,26 @@ ulpwise scan path/to/repo --rules one-minus-cos,small-angle-division --top 40
 ```
 
 The scan is static and needs nothing installed: it parses each file with `ast`, walks every
-function and matches twelve patterns, each with a severity, the reason it loses digits or
-overflows, the usual replacement and, where one exists, the upstream bug it comes from.
+function and matches fifteen patterns, each with a severity, the reason it loses digits,
+overflows or loses its gradient, the usual replacement and, where one exists, the upstream bug it
+comes from.
 
 | rule | severity | pattern |
 |---|---|---|
 | `exp-of-square` | high | `exp(x * x)`, `exp(x ** 2)`, `(-x.pow(2)).exp()`: the rounding error of the square is multiplied by `x * x / 2` ulps (pytorch #198664) |
 | `sin-of-pi-times` | high | `sin(pi * x)`, `cos(pi * x)`: the product is rounded before the argument reduction (pytorch #198663) |
 | `softplus-by-hand` | high | `log(1 + exp(x))` |
-| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))` |
-| `hypot-by-hand` | high | `sqrt(a * a + b * b)` |
+| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))`; quiet when every `exp` argument has its maximum subtracted first, `log(exp(x - x.max()).sum())`, the stable form |
+| `hypot-by-hand` | high | `sqrt(a * a + b * b)`; the advice no longer says `norm`, which squares first too and overflows at 1.8e19 in float32 (kornia #5505 review) |
 | `sqrt-of-difference` | medium | `sqrt(a - b)` |
 | `one-minus-cos` | medium | `1 - cos(x)` (kornia #4897) |
 | `log1p-by-hand`, `expm1-by-hand` | medium | `log(1 + x)`, `exp(x) - 1` |
 | `atan-of-quotient` | medium | `atan(y / x)` |
 | `small-angle-division` | medium | `/ theta`, `/ theta ** 2`, `/ sin(theta)` in a function that takes `sin` or `cos` of `theta` and has no `where`, `clamp`, `eps` or series in sight (kornia #4838, #4897) |
-| `acos-for-angle` | info | `acos`, `asin` used to recover an angle |
+| `acos-for-angle` | medium | `acos`, `asin` used to recover an angle: the angle comes back with an absolute error of `sqrt(eps)`, 0.02 degrees in float32 (kornia #5500) |
+| `eps-floor` | medium | `x * (1 - eps) + eps`: every value moves, 0 becomes `eps`, so a one-hot's entries sum to `1 + (C - 1) eps` and a perfect prediction scores a loss that grows with the image (kornia #5538, found by the kornia conventions audit) |
+| `where-nan-gradient` | medium | `where(d > eps, f(d), other)` with `f` a division by `d` or a `sqrt`, `log`, `acos` or `asin` of it: `where` evaluates both branches and hands the discarded one a zero gradient, and the backward of `f` at the singularity turns that zero into `0 / 0 = nan`, so the guard protects the value and not the gradient; quiet for `numpy.where`, for a comparison against a number above 1, for a floor named `eps`, `tol` or `floor` on the other side, and once `d` is re-bound to a `where`, `clamp` or `maximum` of itself (kornia #5579, found by the kornia conventions audit) |
+| `clamp-at-singularity` | medium | `clamp(x, min=0).sqrt()`, `sqrt(clamp(x, min=0))`, `clamp(c, -1, 1).acos()`: the bound is the point where the next function has an infinite derivative, and clamp's derivative at its own bound is not the same across torch versions, 1 on 2.5.1 and 2.9.1 and 0 on 2.14, so the gradient there is `inf` or `nan` on the older half of a supported range; a bound strictly inside the domain, `min=1e-8`, is a floor and is not reported, and `numpy` is quiet (kornia #4229, found by the kornia conventions audit; kornia #5500) |
 
 On kornia `main` at `e05b0ee` the scan takes 4 s for 506 files and reports 35 findings. The
 `one-minus-cos` and `small-angle-division` findings are the four lines of `So3.right_jacobian` and
@@ -241,6 +248,13 @@ and `Se3.exp` (also #4897) is under `one-minus-cos`. The scan puts `ellipse_to_l
 on the list too, for a `sqrt` of a difference; the bug there was a different one, so that entry is
 what the scan is: a reading list, not a verdict.
 
+The two gradient rules are reading lists in the same sense. On kornia `main` at `d15741e2` they
+report seven lines: three `clamp` bounds in `_solve_cubic_real`, and four `where` guards, in
+`PatchDominantGradientOrientation`, `compute_correspond_epilines`, `_crop_scale_translation` and
+`_get_convex_edges`, of which the first two sit on a differentiable path and the last two draw
+boxes and polygons. On ultralytics `main` at `8df3534` they report three lines, all in metrics or
+inference code that is never differentiated, and nothing in torchvision.
+
 `--run` adds the dynamic half. The module level functions with the most elementary math are
 imported and called with the same 91 point grid (both signs of `1e-8` to `1e3`, and zero) for every
 required argument, in float32 and in float64, torch first and numpy second, and the float32 result

{ulpwise-0.3.2 → ulpwise-0.3.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "ulpwise"
-version = "0.3.2"
+version = "0.3.3"
 description = "Numerical conformance testing for ML code: float edge values, exact rounding oracles, knife-edge finders and a pytest plugin"
 readme = "README.md"
 license = { text = "MIT OR Apache-2.0" }

{ulpwise-0.3.2 → ulpwise-0.3.3}/python/ulpwise/__init__.py

@@ -30,7 +30,7 @@ from ._core import special as _special_core
 from ._core import ulp_distance as _ulp_distance_core
 from ._core import ulp_distances as _ulp_distances_core
 
-__version__ = "0.3.2"
+__version__ = "0.3.3"
 
 __all__ = [
     "UNARY_OPS",

{ulpwise-0.3.2 → ulpwise-0.3.3}/python/ulpwise/corpus/cases.json

@@ -1103,6 +1103,70 @@
     "max_ulp": 1
    },
    "before_fix": "[-0.00048828125, -0.01129150390625, -0.039337158203125, 0.0797119140625] on torch 2.14.0 (512, 72, 23 and 6.7 ulp)"
+  },
+  {
+   "id": "kornia-5500-angle-error-small-angle-acos",
+   "repo": "kornia/kornia",
+   "pr": 5505,
+   "issue": 5500,
+   "merged_at": "2026-10-06T17:27:13Z",
+   "kind": "cancellation",
+   "title": "angle_error_mat and angle_error_vec return exactly 0 for every rotation below 0.03 degrees in float32",
+   "symptom": "Both metrics recover the angle as acos of a cosine, (tr(R1^T R2) - 1) / 2 and v1 . v2 / (|v1| |v2|). Next to 1 the float32 cosine has no digits left for a small angle, so a 0.01 degree rotation scores 0.0, a 0.05 degree one is 3 to 21 % off depending on the rotations and 179.99 degrees comes back as 180.0; the clamp(-1, 1).acos() also has a -inf / nan backward at exactly 0 and 180 degrees on torch 2.5.1. The patch reads sin(theta) from the skew part of the relative rotation, the cross product of the unit vectors for angle_error_vec, and returns atan2(sin, cos).",
+   "requires": [
+    "torch",
+    "kornia"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nimport kornia\nR1 = kornia.geometry.axis_angle_to_rotation_matrix(torch.tensor([[0.3, -0.5, 0.8]], dtype=torch.float64))\nR2 = R1 @ kornia.geometry.axis_angle_to_rotation_matrix(torch.deg2rad(torch.tensor([[0.0, 0.0, 0.01]], dtype=torch.float64)))\nv1 = torch.tensor([[0.0, 0.6, 0.8]], dtype=torch.float64)\nv2 = (kornia.geometry.axis_angle_to_rotation_matrix(torch.deg2rad(torch.tensor([[0.01, 0.0, 0.0]], dtype=torch.float64))) @ v1[..., None])[..., 0]\nactual = [kornia.metrics.angle_error_mat(R1.float(), R2.float()).item(), kornia.metrics.angle_error_vec(v1.float(), v2.float()).item()]\nexpected = [0.01, 0.01]\n",
+   "check": {
+    "type": "allclose",
+    "atol": 0.0,
+    "rtol": 0.001
+   },
+   "before_fix": "[0.0, 0.0] on kornia main 050ac77f"
+  },
+  {
+   "id": "peft-3769-add-weighted-adapter-svd-conv1d-conv3d",
+   "repo": "huggingface/peft",
+   "pr": null,
+   "issue": 3769,
+   "merged_at": null,
+   "kind": "crash",
+   "title": "add_weighted_adapter with combination_type='svd' crashed for Conv1d and Conv3d LoRA layers",
+   "symptom": "_svd_generalized_task_arithmetic_weighted_adapter checked isinstance(target, Conv2d) and flattened only that, so a Conv1d or Conv3d delta weight reached the SVD with its kernel dims in place and the matmul raised. The fix tests _ConvNd and always flattens from dim 1; PR 3903 is open with it.",
+   "requires": [
+    "torch",
+    "peft"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nfrom peft import LoraConfig, get_peft_model\n\nclass Net(torch.nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv = torch.nn.Conv1d(4, 8, kernel_size=3)\n\n    def forward(self, x):\n        return self.conv(x)\n\ntorch.manual_seed(0)\nconfig = LoraConfig(target_modules=['conv'], r=2, init_lora_weights=False)\nmodel = get_peft_model(Net(), config, adapter_name='a')\nmodel.add_adapter('b', config)\nx = torch.randn(1, 4, 10)\nwith torch.no_grad():\n    model.base_model.set_adapter(['a', 'b'])\n    both = model(x)\n    # svd_rank=4 holds the rank 2 + 2 sum of the two deltas exactly\n    model.add_weighted_adapter(['a', 'b'], [1.0, 1.0], 'ab', combination_type='svd', svd_rank=4)\n    model.set_adapter('ab')\n    combined = model(x)\nactual = combined.flatten().tolist()\nexpected = both.flatten().tolist()\n",
+   "check": {
+    "type": "allclose",
+    "atol": 1e-05,
+    "rtol": 1e-05
+   },
+   "before_fix": "RuntimeError inside _svd_generalized_task_arithmetic_weighted_adapter on peft main 91ca7945: Expected size for first two dimensions of batch2 tensor to be: [8, 4] but got: [8, 3]"
+  },
+  {
+   "id": "peft-3830-oft-module-dropout-never-applied",
+   "repo": "huggingface/peft",
+   "pr": null,
+   "issue": 3830,
+   "merged_at": null,
+   "kind": "no-op",
+   "title": "OFT module_dropout was never applied, every training forward used all rotation blocks",
+   "symptom": "OFTLayer built self.oft_dropout from module_dropout but OFTRotationModule.forward never called it, so module_dropout=0.5 behaved like 0 and ten training forwards of the same input were identical. The fix passes the dropout into the rotation module and applies it to the Cayley blocks; PR 3905 is open with it.",
+   "requires": [
+    "torch",
+    "peft"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nfrom peft import OFTConfig, get_peft_model\n\nclass Net(torch.nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.lin = torch.nn.Linear(16, 12)\n\n    def forward(self, x):\n        return self.lin(x)\n\ntorch.manual_seed(0)\nconfig = OFTConfig(target_modules=['lin'], r=4, oft_block_size=0, module_dropout=0.5, init_weights=False)\nmodel = get_peft_model(Net(), config)\nmodel.train()\nx = torch.randn(8, 16)\nwith torch.no_grad():\n    outputs = [model(x) for _ in range(10)]\n# module_dropout replaces half of the four rotation blocks by the identity in each training forward,\n# so ten forwards cannot all agree\nactual = [max((outputs[0] - out).abs().max().item() for out in outputs[1:])]\nexpected = [1.0]  # any positive spread\n",
+   "check": {
+    "type": "sign"
+   },
+   "before_fix": "[0.0] on peft main 91ca7945; with the fix the ten forwards spread by 0.56"
   }
  ]
 }

{ulpwise-0.3.2 → ulpwise-0.3.3}/python/ulpwise/scan.py

@@ -65,15 +65,16 @@ RULES = {
     ),
     "logsumexp-by-hand": (
         "high",
-        "log of a sum of exps: any large term overflows the sum to inf",
+        "log of a sum of exps with no maximum subtracted first: any large term overflows the sum to inf",
         "logsumexp, or subtract the maximum before exp",
         "",
     ),
     "hypot-by-hand": (
         "high",
         "sqrt(a * a + b * b): the squares overflow or underflow long before the result would",
-        "hypot(a, b) or norm",
-        "",
+        "hypot(a, b); a torch or numpy norm squares the entries first too and overflows at sqrt(max), 1.8e19 in float32, "
+        "so to normalise a vector divide by its largest absolute entry rather than by its norm",
+        "kornia #5505 review, angle_error_vec divided by the norm and returned 0 for float32 vectors longer than 1.8e19",
     ),
     "sqrt-of-difference": (
         "medium",
@@ -112,10 +113,28 @@ RULES = {
         "kornia #4838 and #4897, So3.log and the Jacobians",
     ),
     "acos-for-angle": (
-        "info",
-        "acos or asin to recover an angle: the derivative is infinite at the ends, so the angle near 0 or pi loses half its digits",
+        "medium",
+        "acos or asin to recover an angle: next to 1 the cosine has no digits left for a small angle, so the angle comes back with an absolute error of sqrt(eps), 0.02 degrees in float32, whatever its size, and the derivative is infinite at the ends",
         "atan2 of the cross product norm and the dot product",
-        "",
+        "kornia #5500, angle_error_mat and angle_error_vec return 0 below 0.03 degrees in float32",
+    ),
+    "eps-floor": (
+        "medium",
+        "x * (1 - eps) + eps: every value moves, 0 becomes eps and 1 stays 1, so the entries of a one-hot sum to 1 + (C - 1) eps and a perfect prediction scores a loss that grows with the image",
+        "keep the values exact and add eps only to the log argument or the denominator that needs it, or default eps to 0",
+        "kornia #5538 (found by the kornia conventions audit), one_hot floored its zeros with 1e-6 and a perfect prediction scored a macro Dice loss of 0.0234 on a 256 x 384 image",
+    ),
+    "where-nan-gradient": (
+        "medium",
+        "where(d > eps, f(d), other) with f a division by d or a sqrt, log, acos or asin of it: where evaluates both branches and hands the discarded one a zero gradient, and the backward of a division or a sqrt at zero turns that zero into 0 / 0 = nan, which reaches the inputs at exactly the points the guard was written for; the forward value is right, only the gradient is lost, so this matters in a loss or a layer and not in a metric that is never differentiated; numpy has no autograd and only warns",
+        "substitute inside the branch that gets differentiated: safe = where(d > eps, d, ones_like(d)), then divide by safe or take the sqrt or log of safe, so that the discarded branch is finite too",
+        "kornia #5579 (found by the kornia conventions audit), the mutual information losses divided by the range of the signal inside where(diff > eps, ...) and a constant input or target got a nan gradient",
+    ),
+    "clamp-at-singularity": (
+        "medium",
+        "clamp(x, min=0).sqrt(), sqrt(clamp(x, min=0)), clamp(c, -1, 1).acos(): the clamp bound is the point where the derivative of the next function is infinite, and on torch 2.5.1 and 2.9.1 clamp hands the incoming gradient through at its own bound, so the gradient there is inf or nan (torch 2.14 returns 0 at the bound, which hides it on the newer half of a supported range); a clamp to a value strictly inside the domain, such as min=1e-8, is a floor and is not reported",
+        "take the value from the real argument and the gradient from a safe one, f(where(at_bound, safe, x)) with the value fixed up by where, or atan2 instead of acos for an angle",
+        "kornia #4229 (found by the kornia conventions audit), _cdist's clamp(min=0.0).sqrt() gave nan gradients for identical float16 descriptors on torch 2.5.1; kornia #5500, clamp(-1, 1).acos() in angle_error_mat had a -inf backward at exactly 0 and 180 degrees on torch 2.5.1",
     ),
 }
 SEVERITY_ORDER = {"high": 0, "medium": 1, "info": 2}
@@ -227,7 +246,281 @@ def _angle_names(body: ast.AST) -> set:
     return out
 
 
+MAXIMUM = {"max", "amax", "maximum", "fmax"}
+
+
+def _is_maximum(node: ast.AST) -> Optional[ast.AST]:
+    """The value whose maximum is taken in ``x.max(-1)``, ``np.max(x, axis=-1, keepdims=True)``, ``torch.amax(x)``,
+    ``torch.maximum(a, b)`` and the ``.values`` or ``[0]`` of a ``torch.max`` result, else None."""
+    node = _strip(node)
+    if isinstance(node, ast.Subscript):
+        node = node.value
+    if isinstance(node, ast.Attribute) and node.attr == "values":
+        node = node.value
+    name, arg = _callee(node)
+    return arg if name in MAXIMUM else None
+
+
+def _maxima_and_shifted(fn: ast.AST) -> Tuple[set, set]:
+    """Names bound in ``fn`` to a maximum, and names bound to a value with a maximum subtracted from it."""
+    maxima: set = set()
+    shifted: set = set()
+    assignments = [
+        (node.targets if isinstance(node, ast.Assign) else [node.target], node.value)
+        for node in ast.walk(fn)
+        if isinstance(node, (ast.Assign, ast.AnnAssign)) and node.value is not None
+    ]
+    for targets, value in assignments:
+        for target in targets:
+            if isinstance(target, (ast.Tuple, ast.List)) and target.elts:
+                target = target.elts[0]  # ``m, _ = x.max(-1)``
+            if isinstance(target, ast.Name) and _is_maximum(value) is not None:
+                maxima.add(target.id)
+    for targets, value in assignments:
+        for target in targets:
+            if isinstance(target, ast.Name) and _is_shifted(value, maxima, set()):
+                shifted.add(target.id)
+    return maxima, shifted
+
+
+def _is_shifted(node: ast.AST, maxima: set, shifted: set) -> bool:
+    """``x - x.max(...)``, ``x - m`` with ``m`` bound to a maximum, or a name bound to such a difference: the
+    argument of ``exp`` in the stable form of logsumexp, which cannot overflow."""
+    node = _strip(node)
+    if isinstance(node, ast.Name):
+        return node.id in shifted
+    if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Sub):
+        right = _strip(node.right)
+        return _is_maximum(right) is not None or (isinstance(right, ast.Name) and right.id in maxima)
+    return False
+
+
+def _every_exp_shifted(node: ast.AST, maxima: set, shifted: set) -> bool:
+    """Every ``exp`` inside ``node`` is applied to a value with its maximum subtracted."""
+    args = []
+    for sub in ast.walk(node):
+        name, arg = _callee(sub)
+        if name in EXP and arg is not None:
+            args.append(arg)
+    return bool(args) and all(_is_shifted(arg, maxima, shifted) for arg in args)
+
+
 _GUARD = re.compile(r"\bwhere\b|\bclamp|\bclip\b|\beps\b|\bfinfo\b|\btaylor\b|\bseries\b|small.angle|\bmasked", re.I)
+_EPS_NAME = re.compile(r"^_?(eps|epsilon|smoothing|label_smoothing)$", re.I)
+
+
+def _eps_key(node: ast.AST) -> Optional[str]:
+    """``'eps'`` for a name or attribute that looks like a smoothing constant (``eps``, ``self.eps``), else None."""
+    node = _strip(node)
+    if isinstance(node, ast.Name) and _EPS_NAME.match(node.id):
+        return node.id
+    if isinstance(node, ast.Attribute) and _EPS_NAME.match(node.attr):
+        return ast.unparse(node)
+    return None
+
+
+def _is_eps_floor(node: ast.AST) -> bool:
+    """``x * (1 - eps) + eps`` in either order: the eps smoothing floor that moves every value."""
+    if not (isinstance(node, ast.BinOp) and isinstance(node.op, ast.Add)):
+        return False
+    for eps, other in ((node.right, node.left), (node.left, node.right)):
+        key = _eps_key(eps)
+        other = _strip(other)
+        if key is None or not (isinstance(other, ast.BinOp) and isinstance(other.op, ast.Mult)):
+            continue
+        for factor in (other.left, other.right):
+            factor = _strip(factor)
+            if (
+                isinstance(factor, ast.BinOp)
+                and isinstance(factor.op, ast.Sub)
+                and _is_const(factor.left, 1)
+                and _eps_key(factor.right) == key
+            ):
+                return True
+    return False
+
+
+SINGULAR_AT_ZERO = {"sqrt", "rsqrt", "log", "log2", "log10"}
+SINGULAR_AT_ONE = {"acos", "arccos", "asin", "arcsin", "atanh", "arctanh"}
+CLAMP = {"clamp", "clip", "clamp_min", "clamp_max"}
+NUMPY_MODULES = {"np", "numpy"}
+
+
+def _bare(node: ast.AST) -> Optional[ast.AST]:
+    """The quantity a comparison tests, with unary minus, ``abs``, a square and a subscript such as
+    ``mask[..., None]`` stripped: ``theta.abs() < eps`` and ``theta_sq > 0`` test ``theta`` and ``theta_sq``."""
+    node = _strip(node)
+    while True:
+        name, arg = _callee(node)
+        if name in {"abs", "absolute"} and arg is not None:
+            node = _strip(arg)
+            continue
+        if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Pow):
+            node = _strip(node.left)
+            continue
+        break
+    if isinstance(node, ast.Constant):
+        return None
+    identifier = node.id if isinstance(node, ast.Name) else node.attr if isinstance(node, ast.Attribute) else ""
+    if _THRESHOLD.search(identifier):
+        return None  # var > floor, r > radicand_floor: the floor is the threshold, not the quantity
+    return node
+
+
+_THRESHOLD = re.compile(r"eps|tol|floor|thresh|tiny|bound|limit", re.I)
+SAFE_MAKERS = {"where", "clamp", "clip", "clamp_min", "maximum"}
+
+
+def _made_safe(fn: ast.AST) -> dict:
+    """Names re-bound in ``fn`` to a ``where``, ``clamp`` or ``maximum`` of themselves, with the line of the
+    re-binding: after ``w_sum = where(has_weight, w_sum, ones_like(w_sum))`` a division by ``w_sum`` is safe."""
+    out: dict = {}
+    for node in ast.walk(fn):
+        if isinstance(node, ast.Assign) and len(node.targets) == 1 and isinstance(node.targets[0], ast.Name):
+            target = node.targets[0].id
+            value = _strip(node.value)
+            name, _ = _callee(value)
+            if name in SAFE_MAKERS and any(isinstance(sub, ast.Name) and sub.id == target for sub in ast.walk(value)):
+                out[target] = min(node.lineno, out.get(target, node.lineno))
+    return out
+
+
+def _comparisons(fn: ast.AST) -> dict:
+    """Names bound in ``fn`` to a comparison, with the quantities that comparison tests: ``small = theta.abs() < 1e-4``
+    gives ``{"small": [theta]}``."""
+    out: dict = {}
+    for node in ast.walk(fn):
+        if isinstance(node, (ast.Assign, ast.AnnAssign)) and node.value is not None:
+            targets = node.targets if isinstance(node, ast.Assign) else [node.target]
+            tested = _tested(node.value, {})
+            if tested:
+                for target in targets:
+                    if isinstance(target, ast.Name):
+                        out[target.id] = tested
+    return out
+
+
+def _tested(cond: ast.AST, comparisons: dict) -> List[ast.AST]:
+    """The quantities compared in ``cond``: the two sides of every comparison in it, through ``~``, ``&``, ``|`` and a
+    subscript, and the comparisons behind the names bound in ``comparisons``."""
+    out: List[ast.AST] = []
+    for sub in ast.walk(cond):
+        if isinstance(sub, ast.Compare):
+            sides = [sub.left] + sub.comparators
+            if any(abs(_number(side) or 0) > 1 for side in sides):
+                continue  # strength < 50 is a piecewise definition, not a guard at a singularity
+            for side in sides:
+                quantity = _bare(side)
+                if quantity is not None:
+                    out.append(quantity)
+        elif isinstance(sub, ast.Name) and sub.id in comparisons:
+            out.extend(comparisons[sub.id])
+    return out
+
+
+def _singular_use(branch: ast.AST, tested: Sequence[ast.AST]) -> bool:
+    """``branch`` divides by one of ``tested``, raises it to a negative or fractional power, or takes the sqrt, log,
+    acos or asin of it, so that its derivative is infinite where the comparison fails."""
+    for sub in ast.walk(branch):
+        if isinstance(sub, ast.BinOp) and isinstance(sub.op, ast.Div):
+            den = _strip(sub.right)
+            if isinstance(den, ast.BinOp) and isinstance(den.op, ast.Pow):
+                den = _strip(den.left)
+            if any(_same(den, t) for t in tested):
+                return True
+        if isinstance(sub, ast.BinOp) and isinstance(sub.op, ast.Pow):
+            exponent = sub.right
+            negative = isinstance(exponent, ast.UnaryOp) and isinstance(exponent.op, ast.USub)
+            exponent = _strip(exponent)
+            if isinstance(exponent, ast.Constant) and isinstance(exponent.value, (int, float)):
+                if negative or exponent.value != int(exponent.value):
+                    if any(_same(_strip(sub.left), t) for t in tested):
+                        return True
+        name, arg = _callee(sub)
+        if name in SINGULAR_AT_ZERO | SINGULAR_AT_ONE and arg is not None:
+            if any(_same(_strip(arg), t) for t in tested):
+                return True
+    return False
+
+
+def _where_parts(node: ast.AST) -> Optional[Tuple[ast.AST, List[ast.AST]]]:
+    """(condition, branches) of ``where(cond, a, b)``, ``torch.where(cond, a, b)`` and the method form
+    ``a.where(cond, b)``; None for anything else, including ``numpy.where``, which has no autograd."""
+    name, _ = _callee(node)
+    if name != "where" or not isinstance(node, ast.Call):
+        return None
+    fn = node.func
+    if isinstance(fn, ast.Attribute):
+        module = fn.value
+        if isinstance(module, ast.Name) and module.id in NUMPY_MODULES:
+            return None
+        if len(node.args) == 2:  # a.where(cond, b)
+            return node.args[0], [module, node.args[1]]
+    if len(node.args) >= 3:
+        return node.args[0], [node.args[1], node.args[2]]
+    return None
+
+
+def _number(node: ast.AST) -> Optional[float]:
+    negative = isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.USub)
+    node = _strip(node)
+    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)) and not isinstance(node.value, bool):
+        return -node.value if negative else node.value
+    return None
+
+
+MODULES = {"torch", "np", "numpy", "jnp", "jax", "tf", "F"}
+
+
+def _clamp_bounds(node: ast.AST) -> Optional[Tuple[Optional[float], Optional[float]]]:
+    """(lower, upper) of a ``clamp`` / ``clip`` call whose bounds are literal numbers; a bound given by a name is None,
+    and the result is None when ``node`` is not a clamp."""
+    if not isinstance(node, ast.Call):
+        return None
+    name, _ = _callee(node)
+    if name not in CLAMP:
+        return None
+    positional = list(node.args)
+    fn = node.func
+    module_form = isinstance(fn, ast.Name) or (
+        isinstance(fn, ast.Attribute)
+        and (isinstance(fn.value, ast.Attribute) or (isinstance(fn.value, ast.Name) and fn.value.id in MODULES))
+    )
+    if module_form and positional:
+        positional = positional[1:]  # clamp(x, lo, hi), torch.clamp(x, lo, hi): the first argument is the value
+    lo = hi = None
+    if name == "clamp_max":
+        hi = _number(positional[0]) if positional else None
+    else:
+        lo = _number(positional[0]) if positional else None
+        hi = _number(positional[1]) if len(positional) > 1 else None
+    for kw in node.keywords:
+        if kw.arg in {"min", "a_min"}:
+            lo = _number(kw.value)
+        elif kw.arg in {"max", "a_max"}:
+            hi = _number(kw.value)
+    return lo, hi
+
+
+def _numpy_call(node: ast.AST) -> bool:
+    if not isinstance(node, ast.Call) or not isinstance(node.func, ast.Attribute):
+        return False
+    return isinstance(node.func.value, ast.Name) and node.func.value.id in NUMPY_MODULES
+
+
+def _clamp_at_singularity(name: str, node: ast.AST, arg: ast.AST) -> bool:
+    """``f(clamp(x, ...))`` where a clamp bound is the point at which ``f`` has an infinite derivative; quiet for
+    numpy, which has no autograd."""
+    arg = _strip(arg)
+    bounds = _clamp_bounds(arg)
+    if bounds is None or _numpy_call(node) or _numpy_call(arg):
+        return False
+    lo, hi = bounds
+    if name in SINGULAR_AT_ZERO:
+        return lo == 0
+    if name in SINGULAR_AT_ONE:
+        return lo == -1 or hi == 1
+    return False
 
 
 def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines: Sequence[str]) -> Tuple[List[Finding], HotSpot]:
@@ -236,6 +529,9 @@ def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines:
     divisions = 0
     names: List[str] = []
     angles = _angle_names(fn)
+    maxima, shifted = _maxima_and_shifted(fn)
+    comparisons = _comparisons(fn)
+    made_safe = _made_safe(fn)
     fn_source = "\n".join(source_lines[fn.lineno - 1 : getattr(fn, "end_lineno", fn.lineno)])
     guarded = bool(_GUARD.search(fn_source))
 
@@ -243,7 +539,7 @@ def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines:
         line = getattr(node, "lineno", fn.lineno)
         findings.append(Finding(rule, path, line, qualname, source_lines[line - 1].strip()[:160]))
 
-    for node in ast.walk(fn):
+    for node in _own_nodes(fn):
         name, arg = _callee(node)
         if name in ELEMENTARY:
             calls += 1
@@ -256,6 +552,16 @@ def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines:
             den_name = den.id if isinstance(den, ast.Name) else None
             if not guarded and angles and (den_name in angles or _contains_call(den, SIN)):
                 add("small-angle-division", node)
+        if _is_eps_floor(node):
+            add("eps-floor", node)
+        parts = _where_parts(node)
+        if parts is not None:
+            tested = [
+                t for t in _tested(parts[0], comparisons)
+                if not (isinstance(t, ast.Name) and made_safe.get(t.id, node.lineno) < node.lineno)
+            ]
+            if tested and any(_singular_use(branch, tested) for branch in parts[1]):
+                add("where-nan-gradient", node)
         if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Sub):
             left, right = _strip(node.left), _strip(node.right)
             rn, _ = _callee(right)
@@ -267,6 +573,8 @@ def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines:
         if name is None or arg is None:
             continue
         inner = _strip(arg)
+        if name in SINGULAR_AT_ZERO | SINGULAR_AT_ONE and _clamp_at_singularity(name, node, inner):
+            add("clamp-at-singularity", node)
         if name in EXP and _is_square(inner):
             add("exp-of-square", node)
         elif name in SIN:
@@ -280,13 +588,15 @@ def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines:
                 if (_is_const(a, 1) and bn in EXP) or (_is_const(b, 1) and an in EXP):
                     add("softplus-by-hand", node)
                 elif an in EXP and bn in EXP:
-                    add("logsumexp-by-hand", node)
+                    if not _every_exp_shifted(inner, maxima, shifted):
+                        add("logsumexp-by-hand", node)
                 elif _is_const(a, 1) or _is_const(b, 1):
                     add("log1p-by-hand", node)
             else:
                 sn, sarg = _callee(inner)
                 if sn == "sum" and sarg is not None and _contains_call(sarg, EXP):
-                    add("logsumexp-by-hand", node)
+                    if not _every_exp_shifted(sarg, maxima, shifted):
+                        add("logsumexp-by-hand", node)
         elif name in SQRT:
             if isinstance(inner, ast.BinOp) and isinstance(inner.op, ast.Add) and _is_square(inner.left) and _is_square(inner.right):
                 add("hypot-by-hand", node)
@@ -302,6 +612,17 @@ def _findings_for_function(fn: ast.AST, qualname: str, path: str, source_lines:
     return findings, HotSpot(path, fn.lineno, qualname, calls, divisions, sorted(set(names)))
 
 
+def _own_nodes(fn: ast.AST) -> Iterable[ast.AST]:
+    """The nodes of ``fn`` without those of a nested function or class, which ``_functions`` lists on their own."""
+    stack = [fn]
+    while stack:
+        node = stack.pop()
+        yield node
+        for child in ast.iter_child_nodes(node):
+            if not isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
+                stack.append(child)
+
+
 def _functions(tree: ast.Module) -> Iterable[Tuple[ast.AST, str]]:
     def walk(node: ast.AST, prefix: str) -> Iterable[Tuple[ast.AST, str]]:
         for child in ast.iter_child_nodes(node):

{ulpwise-0.3.2 → ulpwise-0.3.3}/tests/test_scan.py

@@ -12,6 +12,7 @@ from ulpwise import scan
 SNIPPET = textwrap.dedent(
     '''
     import math
+    import numpy as np
     import torch
 
 
@@ -41,6 +42,21 @@ SNIPPET = textwrap.dedent(
         return be.log(be.exp(x).sum((0, 1)))
 
 
+    def lse_shifted(x):
+        m = x.max(-1, keepdim=True).values
+        s = x - m
+        return torch.log(torch.exp(s).sum(-1)) + m.squeeze(-1)
+
+
+    def lse_shifted_inline(a, b):
+        m = torch.maximum(a, b)
+        return torch.log(torch.exp(a - m) + torch.exp(b - m)) + m
+
+
+    def lse_half_shifted(a, b):
+        return torch.log(torch.exp(a - a.max()) + torch.exp(b))
+
+
     def norm2(a, b):
         return torch.sqrt(a ** 2 + b ** 2)
 
@@ -54,11 +70,93 @@ SNIPPET = textwrap.dedent(
 
 
     def so3_jacobian_guarded(theta, w):
+        # quiet for small-angle-division, but the discarded branch still differentiates sin(theta) / theta at 0
         small = theta.abs() < 1e-4
         s = torch.where(small, 1 - theta ** 2 / 6, torch.sin(theta) / theta)
         return w / s
 
 
+    def so3_jacobian_safe(theta, w):
+        small = theta.abs() < 1e-4
+        safe_theta = torch.where(small, torch.ones_like(theta), theta)
+        s = torch.where(small, 1 - theta ** 2 / 6, torch.sin(safe_theta) / safe_theta)
+        return w / s
+
+
+    def normalize_signal(data, eps):
+        min_val, _ = data.min(dim=-1)
+        diff = (data.max(dim=-1)[0] - min_val).unsqueeze(-1)
+        return torch.where(diff > eps, (data - min_val.unsqueeze(-1)) / diff, 0.0)
+
+
+    def sqrt_where(theta_sq):
+        nonzero = theta_sq > 0
+        return torch.where(nonzero[..., None], theta_sq.sqrt(), 0.0)
+
+
+    def where_unrelated(x, y):
+        return torch.where(x > 0, x / y, 0.0)
+
+
+    def where_numpy(x, d):
+        return np.where(d > 0, x / d, 0.0)
+
+
+    def cdist_tail(dm):
+        return dm.clamp(min=0.0).sqrt()
+
+
+    def angle_from_trace(w):
+        return torch.acos(torch.clamp(w, -1.0, 1.0))
+
+
+    def floored_sqrt(x):
+        return torch.sqrt(x.clamp(min=1e-8))
+
+
+    def np_arc(dot):
+        return np.arccos(np.clip(dot, -1, 1))
+
+
+    def piecewise_scale(strength):
+        return torch.where(strength < 50, 5000.0 / strength, 200.0 - 2.0 * strength)
+
+
+    def floored_std(var):
+        floor = torch.full_like(var, 1e-10)
+        above = var > floor
+        safe_var = torch.where(above, var, torch.ones_like(var))
+        return torch.where(above, safe_var.sqrt(), floor.sqrt())
+
+
+    def rebound_mean(w, x):
+        w_sum = w.sum(-1)
+        has_weight = w_sum != 0
+        w_sum = torch.where(has_weight, w_sum, torch.ones_like(w_sum))
+        return torch.where(has_weight, (w * x).sum(-1) / w_sum, x.mean(-1))
+
+
+    def outer(x):
+        def inner(y):
+            return torch.exp(y * y)
+
+        return inner(x)
+
+
+    def one_hot_floor(labels, eps):
+        t = torch.nn.functional.one_hot(labels, 3).float()
+        return t * (1.0 - eps) + eps
+
+
+    class Smoother:
+        def forward(self, t):
+            return self.eps + (1 - self.eps) * t
+
+
+    def eps_guard(p, q, eps):
+        return (p + eps) / (q + eps) * (1 - eps)
+
+
     def misc(x, y):
         a = torch.log(1 + x)
         b = torch.exp(x) - 1
@@ -73,15 +171,26 @@ SNIPPET = textwrap.dedent(
 EXPECTED = {
     ("exp-of-square", "erfcx_neg"),
     ("exp-of-square", "misc"),
+    ("exp-of-square", "outer.inner"),
     ("sin-of-pi-times", "trigamma_reflect"),
     ("softplus-by-hand", "softplus"),
     ("logsumexp-by-hand", "lse"),
     ("logsumexp-by-hand", "lse_dim"),
     ("logsumexp-by-hand", "lse_backend"),
+    ("logsumexp-by-hand", "lse_half_shifted"),
     ("hypot-by-hand", "norm2"),
     ("acos-for-angle", "angle_between"),
+    ("eps-floor", "one_hot_floor"),
+    ("eps-floor", "Smoother.forward"),
     ("one-minus-cos", "so3_jacobian"),
     ("small-angle-division", "so3_jacobian"),
+    ("where-nan-gradient", "so3_jacobian_guarded"),
+    ("where-nan-gradient", "normalize_signal"),
+    ("where-nan-gradient", "sqrt_where"),
+    ("clamp-at-singularity", "cdist_tail"),
+    ("clamp-at-singularity", "angle_from_trace"),
+    ("acos-for-angle", "angle_from_trace"),
+    ("acos-for-angle", "np_arc"),
     ("log1p-by-hand", "misc"),
     ("expm1-by-hand", "misc"),
     ("atan-of-quotient", "misc"),
@@ -93,8 +202,14 @@ def test_every_rule_fires_once_on_its_snippet_and_the_guard_silences_it():
     findings, spots = scan.scan_source(SNIPPET, "demo.py")
     assert {(f.rule, f.function) for f in findings} == EXPECTED
     assert {rule for rule, _ in EXPECTED} == set(scan.RULES)  # every rule has a snippet
-    assert sum(f.rule == "exp-of-square" for f in findings) == 3  # x * x, x.pow(2), square(x)
-    assert not [f for f in findings if f.function == "so3_jacobian_guarded"]
+    assert sum(f.rule == "exp-of-square" for f in findings) == 4  # x * x, x.pow(2), square(x), once in outer.inner
+    quiet = {
+        "so3_jacobian_safe", "where_unrelated", "where_numpy", "floored_sqrt", "piecewise_scale", "floored_std",
+        "rebound_mean", "outer", "lse_shifted", "lse_shifted_inline", "eps_guard",
+    }
+    assert not [f for f in findings if f.function in quiet]
+    assert [f.rule for f in findings if f.function == "so3_jacobian_guarded"] == ["where-nan-gradient"]
+    assert [f.rule for f in findings if f.function == "np_arc"] == ["acos-for-angle"]  # numpy has no autograd
     assert all(f.snippet and f.line > 0 and f.path == "demo.py" for f in findings)
     busiest = max(spots, key=lambda s: s.calls)
     assert busiest.function == "misc" and busiest.calls == 7 and "exp" in busiest.names
@@ -125,7 +240,7 @@ def test_scan_tree_skips_tests_and_reports(tmp_path):
     with pytest.raises(ValueError):
         scan.checkout("not a repository at all")
     text = scan.render(findings, spots, "pkg", None, nfiles)
-    assert text.startswith("ulpwise scan of pkg: 2 files") and "exp-of-square [high] x3" in text
+    assert text.startswith("ulpwise scan of pkg: 2 files") and "exp-of-square [high] x4" in text
     md = scan.render(findings, spots, "pkg", "abc123", nfiles, top=3, markdown=True)
     assert md.startswith("# ulpwise scan of pkg @ abc123") and "| calls |" in md and md.count("\n| ") == 4
 
@@ -139,7 +254,7 @@ def test_scan_cli(tmp_path, capsys):
     assert "findings" in capsys.readouterr().out and report.read_text().startswith("# ulpwise scan of")
     assert main(["scan", str(tmp_path), "--rules", "acos-for-angle", "--fail-on", "high"]) == 0
     out = capsys.readouterr().out
-    assert "acos-for-angle [info] x1" in out and "exp-of-square" not in out
+    assert "acos-for-angle [medium] x3" in out and "exp-of-square" not in out
     assert main(["scan", str(tmp_path), "--rules", "no-such-rule"]) == 2
     assert main(["scan", str(tmp_path / "missing")]) == 2