ulpwise 0.3.1__tar.gz → 0.3.3__tar.gz

{ulpwise-0.3.1 → ulpwise-0.3.3}/CHANGELOG.md

@@ -1,5 +1,83 @@
 # Changelog
 
+## 0.3.3 (2026-10-08)
+
+- Two regression corpus cases for peft, both open upstream with a fix PR: peft #3769, `add_weighted_adapter` with
+  `combination_type='svd'` crashed for Conv1d and Conv3d LoRA layers because only Conv2d deltas were flattened before
+  the SVD (the case combines two rank 2 adapters at `svd_rank=4` and expects the output of both adapters active, which
+  the fix matches to 2e-6), and peft #3830, OFT `module_dropout` was never applied, so ten training forwards of the
+  same input were identical where the fix spreads them by 0.56.
+- `ulpwise scan`: new `where-nan-gradient` rule (medium) for `where(d > eps, f(d), other)` with `f` a division by `d`
+  or a `sqrt`, `log`, `acos` or `asin` of it. `where` evaluates both branches and hands the discarded one a zero
+  gradient, and the backward of `f` at the singularity turns that zero into `0 / 0 = nan`, so the guard protects the
+  value and not the gradient. kornia #5579, found by the kornia conventions audit, is the example: the mutual
+  information losses divided by the range of the signal inside `where(diff > eps, ...)` and a constant input or
+  target got a nan gradient. The condition may be a comparison, a name bound to one, or `~`, `&`, `|` and a subscript
+  of those, and the method form `a.where(cond, b)` counts. Quiet for `numpy.where`, which has no autograd, for a
+  comparison against a number above 1 (`strength < 50` in kornia's JPEG scale is a piecewise definition), for a
+  floor named `eps`, `tol`, `floor`, `thresh`, `tiny`, `bound` or `limit` on the other side of the comparison, and
+  once `d` is re-bound to a `where`, `clamp` or `maximum` of itself, which is the recommended fix and the shape of
+  kornia's `fit_line` and `rotation_matrix_to_quaternion`. On kornia main d15741e2 the rule reports four lines, on
+  ultralytics main 8df3534 two, both in metric code, and nothing in torchvision.
+- `ulpwise scan`: new `clamp-at-singularity` rule (medium) for `clamp(x, min=0).sqrt()`, `sqrt(clamp(x, min=0))`,
+  `clamp(c, -1, 1).acos()` and the `clip`, `clamp_min`, `clamp_max`, `asin` and `log` forms: the bound is the point
+  where the next function has an infinite derivative, and clamp's derivative at its own bound is 1 on torch 2.5.1 and
+  2.9.1 and 0 on 2.14, so the gradient there is `inf` or `nan` on the older half of a supported range. kornia #4229,
+  found by the kornia conventions audit, is the example, `_cdist`'s `clamp(min=0.0).sqrt()` with identical float16
+  descriptors, and kornia #5500's `clamp(-1, 1).acos()` the second. A bound strictly inside the domain, `min=1e-8`,
+  is a floor and is not reported, a bound given by a name is not read, and numpy is quiet. On kornia main d15741e2
+  the rule reports the three lines of `_solve_cubic_real`, on ultralytics main 8df3534 one line of `DepthMetrics`.
+- `ulpwise scan`: a finding inside a nested function is reported once, under the nested function, instead of once
+  more under the enclosing one, and the enclosing function's call and division counts no longer include it.
+- `ulpwise scan`: new `eps-floor` rule (medium) for `x * (1 - eps) + eps`, in either order and with `self.eps` too: the
+  floor moves every value, 0 becomes `eps` and 1 stays 1, so the entries of a one-hot sum to `1 + (C - 1) eps` and a
+  perfect prediction scores a loss that grows with the image. kornia #5538, found by the kornia conventions audit, is
+  the example: `one_hot` floored its zeros with 1e-6 and a perfect prediction of a 256 × 384 image with a one-pixel
+  class scored a macro Dice loss of 0.0234. On kornia main 9e188e08 the rule reports that line and nothing else, and
+  nothing in ultralytics or torchvision; `(p + eps) / (q + eps)` is not a floor and stays quiet.
+- `ulpwise scan`: the `hypot-by-hand` advice no longer offers `norm` as the replacement. `torch.linalg.vector_norm` and
+  `numpy.linalg.norm` square the entries first too, so in float32 both return `inf` for a vector with two entries of
+  1.8e19 while `hypot` returns 2.5e19; the advice now says to divide a vector by its largest absolute entry when the
+  point is a direction. The kornia maintainer caught the case in review of kornia #5505, where `v / v.norm()` turned
+  such vectors into zeros and the angle between them read 0.
+- `ulpwise scan`: `acos-for-angle` moves from `info` to `medium` and names the mechanism: next to 1 the cosine has no
+  digits left for a small angle, so `acos` recovers the angle with an absolute error of `sqrt(eps)`, 0.02 degrees in
+  float32, whatever the angle. The kornia scan listed `kornia/metrics/pose.py` under this rule, and both functions
+  there, `angle_error_mat` and `angle_error_vec`, return exactly 0 for every rotation below 0.03 degrees in float32
+  and 180.0 for 179.99 degrees (kornia #5500); the rule cites it.
+- Regression corpus case for kornia #5500: a 0.01 degree rotation scores 0.0 in both metrics on kornia main 050ac77f.
+- The kornia #5500 corpus case records its fix, kornia #5505, merged 2026-10-06: `angle_error_mat` reads the sine from
+  the skew part of the relative rotation and `angle_error_vec` from the cross product of the scaled vectors, and both
+  return `atan2(sin, cos)`, so a 0.01 degree rotation scores 0.0099990 in float32. The case stays an expected failure
+  until a kornia release carries the fix.
+- `ulpwise scan`: `logsumexp-by-hand` no longer fires on the stable form, where every `exp` argument has its
+  maximum subtracted first (`log(exp(x - x.max()).sum())`, or `s = x - m` with `m` bound to a `max`, `amax`,
+  `maximum` or `torch.max(...).values`). The lm-evaluation-harness scan listed the numerically stable `_log_softmax`
+  of `lm_eval/models/_onnx_base.py` next to a real `log(exp(a) + exp(b))`; it now reports only the real one.
+
+## 0.3.2 (2026-10-06)
+
+- Two regression corpus cases for pytorch #199850: `torch.erf` in bfloat16 and float16 on CPU returns 0 at and
+  below 1.8e-7 and loses relative accuracy below 1e-3 (13404 bfloat16 ulps, 5 float16 ulps at worst), found with
+  the half precision rows of `ulpwise survey --functions erf --backends torch --dtypes f16,bf16`.
+- Two regression corpus cases for pytorch #199867: `torch.special.logit` in float16 and bfloat16 on CPU rounds `1 - x`
+  and `x / (1 - x)` to the input dtype before the log, so logit(0.499756) in float16 is -0.000488 for an exact -0.000977
+  (512 ulp, 64 bfloat16 ulp at worst), found with `ulpwise survey --functions logit --backends torch --dtypes f16,bf16`.
+- `studies/accuracy-survey-2026-10-half`: the second accuracy survey, torch 2.14.0+cpu in float16 and bfloat16 for the
+  45 functions with half precision CPU kernels. 33 float16 and 34 bfloat16 rows of 45 are correctly rounded; `erf`
+  (pytorch #199850) and `logit` (pytorch #199867) lose their digits before the rounding step, `polygamma(2, x)` in
+  float16 is off at every half integer in `(-1024, -256)` because the Hurwitz zeta sum accumulates in float for the
+  reduced types, and the activation tails and the `rsqrt` and `i0e` vector versus scalar disagreements of September
+  show again.
+- `ulpwise survey`: the `polygamma_1` and `polygamma_2` references at a non positive integer are now the signed
+  infinity of the pole, `+inf` for an odd order (the limit from both sides) and `-inf` for an even one (the sign of
+  `(-1) ** (n + 1) * n! * zeta(n + 1, x)`, which is what scipy, torch and jax return), instead of a domain error
+  that expected `nan`. The f16 and bf16 grids reach the integers from 2048 and 256 up, so every negative point
+  past there was a pole and counted as a nonfinite mismatch: on torch 2.14 the two functions had 354 of 3546
+  float16 points and 492 of 4016 bfloat16 points each, `polygamma_2` has 0 now in every dtype and `polygamma_1`
+  keeps 491 bfloat16, 4 float32 and 2 float16 points where torch returns a large finite value instead of `inf`
+  at a negative integer (pytorch #198663).
+
 ## 0.3.1 (2026-10-05)
 
 - `ulpwise survey` takes `f16` and `bf16` in `--dtypes` (the default stays `f32,f64`): torch and jax get half

{ulpwise-0.3.1 → ulpwise-0.3.3}/Cargo.lock

@@ -120,7 +120,7 @@ checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
 
 [[package]]
 name = "ulpwise"
-version = "0.3.1"
+version = "0.3.3"
 dependencies = [
  "pyo3",
 ]

{ulpwise-0.3.1 → ulpwise-0.3.3}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "ulpwise"
-version = "0.3.1"
+version = "0.3.3"
 edition = "2021"
 rust-version = "1.86"
 license = "MIT OR Apache-2.0"

{ulpwise-0.3.1 → ulpwise-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ulpwise
-Version: 0.3.1
+Version: 0.3.3
 Classifier: Development Status :: 3 - Alpha
 Classifier: Framework :: Pytest
 Classifier: Intended Audience :: Developers
@@ -226,6 +226,11 @@ PYTHONPATH=/tmp/kornia-0.8.3 ulpwise corpus --repo kornia     # 31 present, 0 fi
 | torchvision #9676 `clamp_bounding_boxes` collapses slightly tilted rotated boxes to a point | geometry | open |
 | pytorch #198663 `polygamma(1, x)` float64 keeps 9 digits (series stops at `1/42`), float32 loses all for large negative `x` | truncation, rounding | open |
 | pytorch #198664 `erfcx` off by `x*x/2` ulps for negative `x` (`exp` at the rounded square) | rounding | open |
+| pytorch #199850 `torch.erf` in bfloat16 and float16 on CPU returns 0 at and below 1.8e-7 and loses relative accuracy below 1e-3 (13404 bfloat16 ulps, 5 float16 ulps) | cancellation | open |
+| pytorch #199867 `torch.special.logit` in float16 and bfloat16 on CPU rounds `1 - x` and `x / (1 - x)` to the input dtype before the log, logit(0.499756) is -0.000488 for -0.000977 (512 float16 ulps, 64 bfloat16 ulps) | rounding | open |
+| kornia #5505 `angle_error_mat` and `angle_error_vec` returned exactly 0 for every rotation below 0.03 degrees in float32 (`acos` next to 1, issue #5500) | cancellation | 2026-10-06 |
+| peft #3769 `add_weighted_adapter` with `combination_type='svd'` crashed for Conv1d and Conv3d LoRA layers | crash | open |
+| peft #3830 OFT `module_dropout` was never applied, every training forward used all rotation blocks | no-op | open |
 
 Cases marked `open` have an issue with the complete patch attached and no merged fix yet. Every case
 is an expected failure until the installed release contains the fix (`fixed_in_release` in
@@ -244,22 +249,26 @@ ulpwise scan path/to/repo --rules one-minus-cos,small-angle-division --top 40
 ```
 
 The scan is static and needs nothing installed: it parses each file with `ast`, walks every
-function and matches twelve patterns, each with a severity, the reason it loses digits or
-overflows, the usual replacement and, where one exists, the upstream bug it comes from.
+function and matches fifteen patterns, each with a severity, the reason it loses digits,
+overflows or loses its gradient, the usual replacement and, where one exists, the upstream bug it
+comes from.
 
 | rule | severity | pattern |
 |---|---|---|
 | `exp-of-square` | high | `exp(x * x)`, `exp(x ** 2)`, `(-x.pow(2)).exp()`: the rounding error of the square is multiplied by `x * x / 2` ulps (pytorch #198664) |
 | `sin-of-pi-times` | high | `sin(pi * x)`, `cos(pi * x)`: the product is rounded before the argument reduction (pytorch #198663) |
 | `softplus-by-hand` | high | `log(1 + exp(x))` |
-| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))` |
-| `hypot-by-hand` | high | `sqrt(a * a + b * b)` |
+| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))`; quiet when every `exp` argument has its maximum subtracted first, `log(exp(x - x.max()).sum())`, the stable form |
+| `hypot-by-hand` | high | `sqrt(a * a + b * b)`; the advice no longer says `norm`, which squares first too and overflows at 1.8e19 in float32 (kornia #5505 review) |
 | `sqrt-of-difference` | medium | `sqrt(a - b)` |
 | `one-minus-cos` | medium | `1 - cos(x)` (kornia #4897) |
 | `log1p-by-hand`, `expm1-by-hand` | medium | `log(1 + x)`, `exp(x) - 1` |
 | `atan-of-quotient` | medium | `atan(y / x)` |
 | `small-angle-division` | medium | `/ theta`, `/ theta ** 2`, `/ sin(theta)` in a function that takes `sin` or `cos` of `theta` and has no `where`, `clamp`, `eps` or series in sight (kornia #4838, #4897) |
-| `acos-for-angle` | info | `acos`, `asin` used to recover an angle |
+| `acos-for-angle` | medium | `acos`, `asin` used to recover an angle: the angle comes back with an absolute error of `sqrt(eps)`, 0.02 degrees in float32 (kornia #5500) |
+| `eps-floor` | medium | `x * (1 - eps) + eps`: every value moves, 0 becomes `eps`, so a one-hot's entries sum to `1 + (C - 1) eps` and a perfect prediction scores a loss that grows with the image (kornia #5538, found by the kornia conventions audit) |
+| `where-nan-gradient` | medium | `where(d > eps, f(d), other)` with `f` a division by `d` or a `sqrt`, `log`, `acos` or `asin` of it: `where` evaluates both branches and hands the discarded one a zero gradient, and the backward of `f` at the singularity turns that zero into `0 / 0 = nan`, so the guard protects the value and not the gradient; quiet for `numpy.where`, for a comparison against a number above 1, for a floor named `eps`, `tol` or `floor` on the other side, and once `d` is re-bound to a `where`, `clamp` or `maximum` of itself (kornia #5579, found by the kornia conventions audit) |
+| `clamp-at-singularity` | medium | `clamp(x, min=0).sqrt()`, `sqrt(clamp(x, min=0))`, `clamp(c, -1, 1).acos()`: the bound is the point where the next function has an infinite derivative, and clamp's derivative at its own bound is not the same across torch versions, 1 on 2.5.1 and 2.9.1 and 0 on 2.14, so the gradient there is `inf` or `nan` on the older half of a supported range; a bound strictly inside the domain, `min=1e-8`, is a floor and is not reported, and `numpy` is quiet (kornia #4229, found by the kornia conventions audit; kornia #5500) |
 
 On kornia `main` at `e05b0ee` the scan takes 4 s for 506 files and reports 35 findings. The
 `one-minus-cos` and `small-angle-division` findings are the four lines of `So3.right_jacobian` and
@@ -268,6 +277,13 @@ and `Se3.exp` (also #4897) is under `one-minus-cos`. The scan puts `ellipse_to_l
 on the list too, for a `sqrt` of a difference; the bug there was a different one, so that entry is
 what the scan is: a reading list, not a verdict.
 
+The two gradient rules are reading lists in the same sense. On kornia `main` at `d15741e2` they
+report seven lines: three `clamp` bounds in `_solve_cubic_real`, and four `where` guards, in
+`PatchDominantGradientOrientation`, `compute_correspond_epilines`, `_crop_scale_translation` and
+`_get_convex_edges`, of which the first two sit on a differentiable path and the last two draw
+boxes and polygons. On ultralytics `main` at `8df3534` they report three lines, all in metrics or
+inference code that is never differentiated, and nothing in torchvision.
+
 `--run` adds the dynamic half. The module level functions with the most elementary math are
 imported and called with the same 91 point grid (both signs of `1e-8` to `1e3`, and zero) for every
 required argument, in float32 and in float64, torch first and numpy second, and the float32 result
@@ -341,6 +357,21 @@ logged and skipped.
 - scipy's float64 `lgamma` does not handle the zeros at 1 and 2, and its Bessel functions lose the
   phase at large `x`.
 
+[`studies/accuracy-survey-2026-10-half`](https://github.com/Nicholas022400701/ulpwise/blob/main/studies/accuracy-survey-2026-10-half/README.md)
+is the second run, the 45 functions with torch float16 and bfloat16 CPU kernels measured in the ulps of
+those dtypes on the same build. 33 of the 45 float16 rows and 34 of the bfloat16 rows are correctly
+rounded over the whole grid; the exceptions:
+
+- `erf` goes through a formula with 1.5e-7 absolute error, 254 bfloat16 ulps and `-0` below
+  `|x| = 1.8e-7` (pytorch #199850).
+- `logit` rounds `1 - x` and `x / (1 - x)` to the input dtype before the log, 512 float16 ulps at
+  `x = 0.499756` while the CUDA kernel computes in float (pytorch #199867).
+- `polygamma(2, x)` in float16 is off at every half integer in `(-1024, -256)`, 1,540 ulps at
+  `x = -1023.5`, because the Hurwitz zeta sum accumulates in float for the reduced types and in
+  double for float32.
+- the `gelu`, `gelu_tanh`, `silu` and `sigmoid` tails and the `rsqrt` and `i0e` vector versus scalar
+  disagreements are the float32 findings of September seen through coarser ulps.
+
 ## How the exact oracle works
 
 For `sqrt(x)` the candidate `r` and the midpoint `m` between `r` and its neighbour are written as

{ulpwise-0.3.1 → ulpwise-0.3.3}/README.md

@@ -197,6 +197,11 @@ PYTHONPATH=/tmp/kornia-0.8.3 ulpwise corpus --repo kornia     # 31 present, 0 fi
 | torchvision #9676 `clamp_bounding_boxes` collapses slightly tilted rotated boxes to a point | geometry | open |
 | pytorch #198663 `polygamma(1, x)` float64 keeps 9 digits (series stops at `1/42`), float32 loses all for large negative `x` | truncation, rounding | open |
 | pytorch #198664 `erfcx` off by `x*x/2` ulps for negative `x` (`exp` at the rounded square) | rounding | open |
+| pytorch #199850 `torch.erf` in bfloat16 and float16 on CPU returns 0 at and below 1.8e-7 and loses relative accuracy below 1e-3 (13404 bfloat16 ulps, 5 float16 ulps) | cancellation | open |
+| pytorch #199867 `torch.special.logit` in float16 and bfloat16 on CPU rounds `1 - x` and `x / (1 - x)` to the input dtype before the log, logit(0.499756) is -0.000488 for -0.000977 (512 float16 ulps, 64 bfloat16 ulps) | rounding | open |
+| kornia #5505 `angle_error_mat` and `angle_error_vec` returned exactly 0 for every rotation below 0.03 degrees in float32 (`acos` next to 1, issue #5500) | cancellation | 2026-10-06 |
+| peft #3769 `add_weighted_adapter` with `combination_type='svd'` crashed for Conv1d and Conv3d LoRA layers | crash | open |
+| peft #3830 OFT `module_dropout` was never applied, every training forward used all rotation blocks | no-op | open |
 
 Cases marked `open` have an issue with the complete patch attached and no merged fix yet. Every case
 is an expected failure until the installed release contains the fix (`fixed_in_release` in
@@ -215,22 +220,26 @@ ulpwise scan path/to/repo --rules one-minus-cos,small-angle-division --top 40
 ```
 
 The scan is static and needs nothing installed: it parses each file with `ast`, walks every
-function and matches twelve patterns, each with a severity, the reason it loses digits or
-overflows, the usual replacement and, where one exists, the upstream bug it comes from.
+function and matches fifteen patterns, each with a severity, the reason it loses digits,
+overflows or loses its gradient, the usual replacement and, where one exists, the upstream bug it
+comes from.
 
 | rule | severity | pattern |
 |---|---|---|
 | `exp-of-square` | high | `exp(x * x)`, `exp(x ** 2)`, `(-x.pow(2)).exp()`: the rounding error of the square is multiplied by `x * x / 2` ulps (pytorch #198664) |
 | `sin-of-pi-times` | high | `sin(pi * x)`, `cos(pi * x)`: the product is rounded before the argument reduction (pytorch #198663) |
 | `softplus-by-hand` | high | `log(1 + exp(x))` |
-| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))` |
-| `hypot-by-hand` | high | `sqrt(a * a + b * b)` |
+| `logsumexp-by-hand` | high | `log(exp(a) + exp(b))`, `log(sum(exp(x)))`, `log(exp(x).sum(-1))`; quiet when every `exp` argument has its maximum subtracted first, `log(exp(x - x.max()).sum())`, the stable form |
+| `hypot-by-hand` | high | `sqrt(a * a + b * b)`; the advice no longer says `norm`, which squares first too and overflows at 1.8e19 in float32 (kornia #5505 review) |
 | `sqrt-of-difference` | medium | `sqrt(a - b)` |
 | `one-minus-cos` | medium | `1 - cos(x)` (kornia #4897) |
 | `log1p-by-hand`, `expm1-by-hand` | medium | `log(1 + x)`, `exp(x) - 1` |
 | `atan-of-quotient` | medium | `atan(y / x)` |
 | `small-angle-division` | medium | `/ theta`, `/ theta ** 2`, `/ sin(theta)` in a function that takes `sin` or `cos` of `theta` and has no `where`, `clamp`, `eps` or series in sight (kornia #4838, #4897) |
-| `acos-for-angle` | info | `acos`, `asin` used to recover an angle |
+| `acos-for-angle` | medium | `acos`, `asin` used to recover an angle: the angle comes back with an absolute error of `sqrt(eps)`, 0.02 degrees in float32 (kornia #5500) |
+| `eps-floor` | medium | `x * (1 - eps) + eps`: every value moves, 0 becomes `eps`, so a one-hot's entries sum to `1 + (C - 1) eps` and a perfect prediction scores a loss that grows with the image (kornia #5538, found by the kornia conventions audit) |
+| `where-nan-gradient` | medium | `where(d > eps, f(d), other)` with `f` a division by `d` or a `sqrt`, `log`, `acos` or `asin` of it: `where` evaluates both branches and hands the discarded one a zero gradient, and the backward of `f` at the singularity turns that zero into `0 / 0 = nan`, so the guard protects the value and not the gradient; quiet for `numpy.where`, for a comparison against a number above 1, for a floor named `eps`, `tol` or `floor` on the other side, and once `d` is re-bound to a `where`, `clamp` or `maximum` of itself (kornia #5579, found by the kornia conventions audit) |
+| `clamp-at-singularity` | medium | `clamp(x, min=0).sqrt()`, `sqrt(clamp(x, min=0))`, `clamp(c, -1, 1).acos()`: the bound is the point where the next function has an infinite derivative, and clamp's derivative at its own bound is not the same across torch versions, 1 on 2.5.1 and 2.9.1 and 0 on 2.14, so the gradient there is `inf` or `nan` on the older half of a supported range; a bound strictly inside the domain, `min=1e-8`, is a floor and is not reported, and `numpy` is quiet (kornia #4229, found by the kornia conventions audit; kornia #5500) |
 
 On kornia `main` at `e05b0ee` the scan takes 4 s for 506 files and reports 35 findings. The
 `one-minus-cos` and `small-angle-division` findings are the four lines of `So3.right_jacobian` and
@@ -239,6 +248,13 @@ and `Se3.exp` (also #4897) is under `one-minus-cos`. The scan puts `ellipse_to_l
 on the list too, for a `sqrt` of a difference; the bug there was a different one, so that entry is
 what the scan is: a reading list, not a verdict.
 
+The two gradient rules are reading lists in the same sense. On kornia `main` at `d15741e2` they
+report seven lines: three `clamp` bounds in `_solve_cubic_real`, and four `where` guards, in
+`PatchDominantGradientOrientation`, `compute_correspond_epilines`, `_crop_scale_translation` and
+`_get_convex_edges`, of which the first two sit on a differentiable path and the last two draw
+boxes and polygons. On ultralytics `main` at `8df3534` they report three lines, all in metrics or
+inference code that is never differentiated, and nothing in torchvision.
+
 `--run` adds the dynamic half. The module level functions with the most elementary math are
 imported and called with the same 91 point grid (both signs of `1e-8` to `1e3`, and zero) for every
 required argument, in float32 and in float64, torch first and numpy second, and the float32 result
@@ -312,6 +328,21 @@ logged and skipped.
 - scipy's float64 `lgamma` does not handle the zeros at 1 and 2, and its Bessel functions lose the
   phase at large `x`.
 
+[`studies/accuracy-survey-2026-10-half`](https://github.com/Nicholas022400701/ulpwise/blob/main/studies/accuracy-survey-2026-10-half/README.md)
+is the second run, the 45 functions with torch float16 and bfloat16 CPU kernels measured in the ulps of
+those dtypes on the same build. 33 of the 45 float16 rows and 34 of the bfloat16 rows are correctly
+rounded over the whole grid; the exceptions:
+
+- `erf` goes through a formula with 1.5e-7 absolute error, 254 bfloat16 ulps and `-0` below
+  `|x| = 1.8e-7` (pytorch #199850).
+- `logit` rounds `1 - x` and `x / (1 - x)` to the input dtype before the log, 512 float16 ulps at
+  `x = 0.499756` while the CUDA kernel computes in float (pytorch #199867).
+- `polygamma(2, x)` in float16 is off at every half integer in `(-1024, -256)`, 1,540 ulps at
+  `x = -1023.5`, because the Hurwitz zeta sum accumulates in float for the reduced types and in
+  double for float32.
+- the `gelu`, `gelu_tanh`, `silu` and `sigmoid` tails and the `rsqrt` and `i0e` vector versus scalar
+  disagreements are the float32 findings of September seen through coarser ulps.
+
 ## How the exact oracle works
 
 For `sqrt(x)` the candidate `r` and the midpoint `m` between `r` and its neighbour are written as

{ulpwise-0.3.1 → ulpwise-0.3.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "ulpwise"
-version = "0.3.1"
+version = "0.3.3"
 description = "Numerical conformance testing for ML code: float edge values, exact rounding oracles, knife-edge finders and a pytest plugin"
 readme = "README.md"
 license = { text = "MIT OR Apache-2.0" }

{ulpwise-0.3.1 → ulpwise-0.3.3}/python/ulpwise/__init__.py

@@ -30,7 +30,7 @@ from ._core import special as _special_core
 from ._core import ulp_distance as _ulp_distance_core
 from ._core import ulp_distances as _ulp_distances_core
 
-__version__ = "0.3.1"
+__version__ = "0.3.3"
 
 __all__ = [
     "UNARY_OPS",

{ulpwise-0.3.1 → ulpwise-0.3.3}/python/ulpwise/corpus/cases.json

@@ -1019,6 +1019,154 @@
     "rtol": 0.0
    },
    "before_fix": "[1] on ultralytics 8.4.163: nc = 1 and no labels, \"ignoring corrupt image/label: non-normalized or out of bounds coordinates [1.15 1.15 ...]\""
+  },
+  {
+   "id": "pytorch-199850-erf-bfloat16-small-arguments",
+   "repo": "pytorch/pytorch",
+   "pr": null,
+   "issue": 199850,
+   "merged_at": null,
+   "kind": "cancellation",
+   "title": "torch.erf in bfloat16 on CPU returns 0 for |x| <= 1.8e-7 and loses relative accuracy below 1e-3",
+   "symptom": "bfloat16 and float16 erf go through Vectorized<float>::erf(), the Abramowitz and Stegun 7.1.26 formula 1 - r * t * exp(-x * x), whose absolute error is 1.5e-7; below |x| = 1e-3 that error is a visible fraction of erf(x) ~ 1.128 x and below 1.5e-7 the 1 - 1 rounds to exactly 0. Of the 16128 bfloat16 inputs in (0, 0.5], 14102 are more than 1 ulp off, 13379 return 0, the worst is 13404 ulp. The MKL float32 and float64 paths are correct; float32 without MKL and under torch.compile takes the same code (issue #198183). The patch adds a Taylor series branch for |x| < 0.5 (Draft PR #199851).",
+   "requires": [
+    "torch"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nxs = torch.tensor([1e-7, 5e-7, 1e-6, 1e-5], dtype=torch.bfloat16)\nactual = torch.erf(xs).tolist()\n# erf at the bfloat16 inputs in float64, rounded to bfloat16\nexpected = [1.1315569281578064e-07, 5.62518835067749e-07, 1.125037670135498e-06, 1.1324882507324219e-05]\n",
+   "check": {
+    "type": "max_ulp",
+    "dtype": "bf16",
+    "max_ulp": 1
+   },
+   "before_fix": "[0.0, 4.172325134277344e-07, 1.1920928955078125e-06, 1.1563301086425781e-05] on torch 2.14.0 (13299, 55, 9 and 4 ulp)"
+  },
+  {
+   "id": "pytorch-199850-erf-float16-small-arguments",
+   "repo": "pytorch/pytorch",
+   "pr": null,
+   "issue": 199850,
+   "merged_at": null,
+   "kind": "cancellation",
+   "title": "torch.erf in float16 on CPU returns 0 for |x| <= 1.8e-7 and is up to 5 ulp off below 1e-3",
+   "symptom": "Same Vectorized<float>::erf() path as the bfloat16 case; float16 has three more bits, so the 1.5e-7 absolute error shows as 1240 of the 14336 inputs in (0, 0.5] more than 1 ulp off, at most 5 ulp, and the three inputs at or below 1.8e-7 return 0.",
+   "requires": [
+    "torch"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nxs = torch.tensor([1e-7, 5e-7, 1e-6, 1e-5], dtype=torch.float16)\nactual = torch.erf(xs).tolist()\n# erf at the float16 inputs in float64, rounded to float16\nexpected = [1.1920928955078125e-07, 5.364418029785156e-07, 1.1324882507324219e-06, 1.1324882507324219e-05]\n",
+   "check": {
+    "type": "max_ulp",
+    "dtype": "f16",
+    "max_ulp": 1
+   },
+   "before_fix": "[0.0, 4.172325134277344e-07, 1.1920928955078125e-06, 1.1563301086425781e-05] on torch 2.14.0 (2, 2, 1 and 4 ulp)"
+  },
+  {
+   "id": "pytorch-199867-logit-bfloat16-intermediate-rounding",
+   "repo": "pytorch/pytorch",
+   "pr": null,
+   "issue": 199867,
+   "merged_at": null,
+   "kind": "rounding",
+   "title": "torch.special.logit in bfloat16 on CPU rounds 1 - x and x / (1 - x) to bfloat16 before the log, 64 ulp at x = 0.498047",
+   "symptom": "The CPU kernel computes 1 - x and x / (1 - x) in the input dtype and only the log in float, so near x = 0.5, where logit is small, the two roundings take most of the digits: logit(0.499756) in float16 is -0.000488 for an exact -0.000977 (512 ulp), 582 of the 15358 float16 values in (0, 1) are more than 1 ulp off and 59 of the 16254 bfloat16 values, 64 ulp at worst. The CUDA kernel computes in float. The patch computes in float and rounds once, as sigmoid_kernel does for the reduced types.",
+   "requires": [
+    "torch"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nxs = torch.tensor([0.498046875, 0.490234375, 0.470703125, 0.51953125], dtype=torch.bfloat16)\nactual = torch.special.logit(xs).tolist()\n# logit at the bfloat16 inputs in float64, rounded to bfloat16\nexpected = [-0.0078125, -0.0390625, -0.1171875, 0.078125]\n",
+   "check": {
+    "type": "max_ulp",
+    "dtype": "bf16",
+    "max_ulp": 1
+   },
+   "before_fix": "[-0.00390625, -0.035888671875, -0.1201171875, 0.0751953125] on torch 2.14.0 (64, 13, 5.7 and 6.1 ulp)"
+  },
+  {
+   "id": "pytorch-199867-logit-float16-intermediate-rounding",
+   "repo": "pytorch/pytorch",
+   "pr": null,
+   "issue": 199867,
+   "merged_at": null,
+   "kind": "rounding",
+   "title": "torch.special.logit in float16 on CPU rounds 1 - x and x / (1 - x) to float16 before the log, 512 ulp at x = 0.499756",
+   "symptom": "The CPU kernel computes 1 - x and x / (1 - x) in the input dtype and only the log in float, so near x = 0.5, where logit is small, the two roundings take most of the digits: logit(0.499756) in float16 is -0.000488 for an exact -0.000977 (512 ulp), 582 of the 15358 float16 values in (0, 1) are more than 1 ulp off and 59 of the 16254 bfloat16 values, 64 ulp at worst. The CUDA kernel computes in float. The patch computes in float and rounds once, as sigmoid_kernel does for the reduced types.",
+   "requires": [
+    "torch"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nxs = torch.tensor([0.499755859375, 0.497314453125, 0.489990234375, 0.52001953125], dtype=torch.float16)\nactual = torch.special.logit(xs).tolist()\n# logit at the float16 inputs in float64, rounded to float16\nexpected = [-0.0009765625, -0.0107421875, -0.0400390625, 0.08013916015625]\n",
+   "check": {
+    "type": "max_ulp",
+    "dtype": "f16",
+    "max_ulp": 1
+   },
+   "before_fix": "[-0.00048828125, -0.01129150390625, -0.039337158203125, 0.0797119140625] on torch 2.14.0 (512, 72, 23 and 6.7 ulp)"
+  },
+  {
+   "id": "kornia-5500-angle-error-small-angle-acos",
+   "repo": "kornia/kornia",
+   "pr": 5505,
+   "issue": 5500,
+   "merged_at": "2026-10-06T17:27:13Z",
+   "kind": "cancellation",
+   "title": "angle_error_mat and angle_error_vec return exactly 0 for every rotation below 0.03 degrees in float32",
+   "symptom": "Both metrics recover the angle as acos of a cosine, (tr(R1^T R2) - 1) / 2 and v1 . v2 / (|v1| |v2|). Next to 1 the float32 cosine has no digits left for a small angle, so a 0.01 degree rotation scores 0.0, a 0.05 degree one is 3 to 21 % off depending on the rotations and 179.99 degrees comes back as 180.0; the clamp(-1, 1).acos() also has a -inf / nan backward at exactly 0 and 180 degrees on torch 2.5.1. The patch reads sin(theta) from the skew part of the relative rotation, the cross product of the unit vectors for angle_error_vec, and returns atan2(sin, cos).",
+   "requires": [
+    "torch",
+    "kornia"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nimport kornia\nR1 = kornia.geometry.axis_angle_to_rotation_matrix(torch.tensor([[0.3, -0.5, 0.8]], dtype=torch.float64))\nR2 = R1 @ kornia.geometry.axis_angle_to_rotation_matrix(torch.deg2rad(torch.tensor([[0.0, 0.0, 0.01]], dtype=torch.float64)))\nv1 = torch.tensor([[0.0, 0.6, 0.8]], dtype=torch.float64)\nv2 = (kornia.geometry.axis_angle_to_rotation_matrix(torch.deg2rad(torch.tensor([[0.01, 0.0, 0.0]], dtype=torch.float64))) @ v1[..., None])[..., 0]\nactual = [kornia.metrics.angle_error_mat(R1.float(), R2.float()).item(), kornia.metrics.angle_error_vec(v1.float(), v2.float()).item()]\nexpected = [0.01, 0.01]\n",
+   "check": {
+    "type": "allclose",
+    "atol": 0.0,
+    "rtol": 0.001
+   },
+   "before_fix": "[0.0, 0.0] on kornia main 050ac77f"
+  },
+  {
+   "id": "peft-3769-add-weighted-adapter-svd-conv1d-conv3d",
+   "repo": "huggingface/peft",
+   "pr": null,
+   "issue": 3769,
+   "merged_at": null,
+   "kind": "crash",
+   "title": "add_weighted_adapter with combination_type='svd' crashed for Conv1d and Conv3d LoRA layers",
+   "symptom": "_svd_generalized_task_arithmetic_weighted_adapter checked isinstance(target, Conv2d) and flattened only that, so a Conv1d or Conv3d delta weight reached the SVD with its kernel dims in place and the matmul raised. The fix tests _ConvNd and always flattens from dim 1; PR 3903 is open with it.",
+   "requires": [
+    "torch",
+    "peft"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nfrom peft import LoraConfig, get_peft_model\n\nclass Net(torch.nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv = torch.nn.Conv1d(4, 8, kernel_size=3)\n\n    def forward(self, x):\n        return self.conv(x)\n\ntorch.manual_seed(0)\nconfig = LoraConfig(target_modules=['conv'], r=2, init_lora_weights=False)\nmodel = get_peft_model(Net(), config, adapter_name='a')\nmodel.add_adapter('b', config)\nx = torch.randn(1, 4, 10)\nwith torch.no_grad():\n    model.base_model.set_adapter(['a', 'b'])\n    both = model(x)\n    # svd_rank=4 holds the rank 2 + 2 sum of the two deltas exactly\n    model.add_weighted_adapter(['a', 'b'], [1.0, 1.0], 'ab', combination_type='svd', svd_rank=4)\n    model.set_adapter('ab')\n    combined = model(x)\nactual = combined.flatten().tolist()\nexpected = both.flatten().tolist()\n",
+   "check": {
+    "type": "allclose",
+    "atol": 1e-05,
+    "rtol": 1e-05
+   },
+   "before_fix": "RuntimeError inside _svd_generalized_task_arithmetic_weighted_adapter on peft main 91ca7945: Expected size for first two dimensions of batch2 tensor to be: [8, 4] but got: [8, 3]"
+  },
+  {
+   "id": "peft-3830-oft-module-dropout-never-applied",
+   "repo": "huggingface/peft",
+   "pr": null,
+   "issue": 3830,
+   "merged_at": null,
+   "kind": "no-op",
+   "title": "OFT module_dropout was never applied, every training forward used all rotation blocks",
+   "symptom": "OFTLayer built self.oft_dropout from module_dropout but OFTRotationModule.forward never called it, so module_dropout=0.5 behaved like 0 and ten training forwards of the same input were identical. The fix passes the dropout into the rotation module and applies it to the Cayley blocks; PR 3905 is open with it.",
+   "requires": [
+    "torch",
+    "peft"
+   ],
+   "fixed_in_release": null,
+   "repro": "import torch\nfrom peft import OFTConfig, get_peft_model\n\nclass Net(torch.nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.lin = torch.nn.Linear(16, 12)\n\n    def forward(self, x):\n        return self.lin(x)\n\ntorch.manual_seed(0)\nconfig = OFTConfig(target_modules=['lin'], r=4, oft_block_size=0, module_dropout=0.5, init_weights=False)\nmodel = get_peft_model(Net(), config)\nmodel.train()\nx = torch.randn(8, 16)\nwith torch.no_grad():\n    outputs = [model(x) for _ in range(10)]\n# module_dropout replaces half of the four rotation blocks by the identity in each training forward,\n# so ten forwards cannot all agree\nactual = [max((outputs[0] - out).abs().max().item() for out in outputs[1:])]\nexpected = [1.0]  # any positive spread\n",
+   "check": {
+    "type": "sign"
+   },
+   "before_fix": "[0.0] on peft main 91ca7945; with the fix the ten forwards spread by 0.56"
   }
  ]
-}
+}