ulpwise 0.1.2__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. ulpwise-0.2.0/CHANGELOG.md +49 -0
  2. {ulpwise-0.1.2 → ulpwise-0.2.0}/Cargo.lock +1 -1
  3. {ulpwise-0.1.2 → ulpwise-0.2.0}/Cargo.toml +1 -1
  4. {ulpwise-0.1.2 → ulpwise-0.2.0}/PKG-INFO +57 -5
  5. {ulpwise-0.1.2 → ulpwise-0.2.0}/README.md +53 -4
  6. {ulpwise-0.1.2 → ulpwise-0.2.0}/pyproject.toml +2 -1
  7. {ulpwise-0.1.2 → ulpwise-0.2.0}/python/ulpwise/__init__.py +1 -1
  8. {ulpwise-0.1.2 → ulpwise-0.2.0}/python/ulpwise/__main__.py +11 -0
  9. {ulpwise-0.1.2 → ulpwise-0.2.0}/python/ulpwise/corpus/__init__.py +4 -0
  10. {ulpwise-0.1.2 → ulpwise-0.2.0}/python/ulpwise/corpus/cases.json +209 -2
  11. ulpwise-0.2.0/python/ulpwise/survey.py +692 -0
  12. ulpwise-0.2.0/studies/accuracy-survey-2026-09/README.md +143 -0
  13. ulpwise-0.2.0/studies/accuracy-survey-2026-09/results.csv +319 -0
  14. ulpwise-0.2.0/studies/accuracy-survey-2026-09/results.md +139 -0
  15. {ulpwise-0.1.2 → ulpwise-0.2.0}/tests/test_corpus.py +1 -1
  16. ulpwise-0.2.0/tests/test_survey.py +53 -0
  17. ulpwise-0.1.2/CHANGELOG.md +0 -28
  18. {ulpwise-0.1.2 → ulpwise-0.2.0}/.github/workflows/ci.yml +0 -0
  19. {ulpwise-0.1.2 → ulpwise-0.2.0}/.github/workflows/release.yml +0 -0
  20. {ulpwise-0.1.2 → ulpwise-0.2.0}/.gitignore +0 -0
  21. {ulpwise-0.1.2 → ulpwise-0.2.0}/CONTRIBUTING.md +0 -0
  22. {ulpwise-0.1.2 → ulpwise-0.2.0}/LICENSE-APACHE +0 -0
  23. {ulpwise-0.1.2 → ulpwise-0.2.0}/LICENSE-MIT +0 -0
  24. {ulpwise-0.1.2 → ulpwise-0.2.0}/examples/sqrt_knife_values.py +0 -0
  25. {ulpwise-0.1.2 → ulpwise-0.2.0}/examples/torch_sqrt_conformance.py +0 -0
  26. {ulpwise-0.1.2 → ulpwise-0.2.0}/python/ulpwise/plugin.py +0 -0
  27. {ulpwise-0.1.2 → ulpwise-0.2.0}/src/edge.rs +0 -0
  28. {ulpwise-0.1.2 → ulpwise-0.2.0}/src/exact.rs +0 -0
  29. {ulpwise-0.1.2 → ulpwise-0.2.0}/src/knife.rs +0 -0
  30. {ulpwise-0.1.2 → ulpwise-0.2.0}/src/lib.rs +0 -0
  31. {ulpwise-0.1.2 → ulpwise-0.2.0}/src/py.rs +0 -0
  32. {ulpwise-0.1.2 → ulpwise-0.2.0}/src/ulp.rs +0 -0
  33. {ulpwise-0.1.2 → ulpwise-0.2.0}/tests/test_core.py +0 -0
@@ -0,0 +1,49 @@
1
+ # Changelog
2
+
3
+ ## 0.2.0 (2026-09-26)
4
+
5
+ - `ulpwise survey` (`ulpwise.survey`): accuracy survey of 61 elementary and special functions of
6
+ torch, numpy, scipy and jax against a 200 bit mpmath reference, in ulps of the dtype, with the
7
+ worst input per row. For torch it also counts inputs where the vectorized kernel and the scalar
8
+ tail disagree and inputs that would fail the `OpInfo` reference test tolerance, default and per op
9
+ override, read from `op_db`. Writes `results.csv` and `results.md`. Optional extra
10
+ `ulpwise[survey]` pulls in mpmath.
11
+ - `studies/accuracy-survey-2026-09`: the first run and its reading notes.
12
+ - Corpus: `max_ulp` check type, and five open cases with the complete patch attached to the issue:
13
+ pytorch #198448 (`torch.sqrt` float64 rounding), pytorch #198583 (Bessel and Airy `p1evl` leading
14
+ 1), kornia #4838 and #4897 (small angle series), torchvision #9676 (rotated box clamp). Open cases
15
+ carry `issue`, `pr: null` and `fixed_in_release: null` and stay expected failures until a release
16
+ contains the fix.
17
+ - Corpus: three more open cases, pytorch #198663 (`polygamma(1, x)`: float64 series truncation and
18
+ float32 reflection argument) and pytorch #198664 (`erfcx` negative branch, `exp` at the rounded
19
+ square), both with the complete patch attached to the issue.
20
+ - Corpus: kornia #4768, `ellipse_to_laf` described the wrong ellipse whenever `b != 0`.
21
+ - Corpus: ultralytics #26330, OBB datasets with plain box labels are rejected at load time; the repro
22
+ writes a one-image dataset to a temporary directory, so it needs ultralytics but no weights.
23
+
24
+ ## 0.1.2 (2026-09-24)
25
+
26
+ - `ulpwise` console script, so `uvx ulpwise midpoint sqrt 0.85`, `pipx run ulpwise ...` and a plain
27
+ `ulpwise ...` inside a virtualenv work without `python -m`.
28
+
29
+ ## 0.1.1 (2026-09-24)
30
+
31
+ No code changes. The AI disclosure in the README was shortened and the package was republished
32
+ from a repository with a fresh history.
33
+
34
+ ## 0.1.0 (2026-09-24)
35
+
36
+ First release.
37
+
38
+ - Rust core: ordered float views and ulp distances (`ulp`), named edge values computed from the
39
+ format (`edge`), exact rounding oracles for `sqrt`, reciprocal and division with correctly rounded
40
+ results that do not depend on the platform libm, f64 referenced midpoint reports for 15 more
41
+ `f32` functions (`exact`), and knife-edge scans (`knife`).
42
+ - Python package built with maturin: `ulp_distance`, `assert_max_ulp` and `max_ulp` for floats,
43
+ lists, numpy arrays and torch tensors, `special`, `neighbours`, `binade_edges`, `all_floats`,
44
+ `midpoint`, `knife_edges`, `sqrt_cr`, a CLI (`python -m ulpwise`) and a pytest plugin
45
+ (`edge_f32` / `edge_f64` parametrization, `assert_max_ulp` fixture).
46
+ - Regression corpus of 11 upstream bugs with runnable repros (kornia, pytorch, pytorch/rl, timm,
47
+ peft), run by `tests/test_corpus.py` against whatever is installed.
48
+ - `examples/torch_sqrt_conformance.py`: measures how often a platform's `torch.sqrt` and
49
+ `numpy.sqrt` disagree with correct rounding at knife-edge inputs.
@@ -120,7 +120,7 @@ checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
120
120
 
121
121
  [[package]]
122
122
  name = "ulpwise"
123
- version = "0.1.2"
123
+ version = "0.2.0"
124
124
  dependencies = [
125
125
  "pyo3",
126
126
  ]
@@ -1,6 +1,6 @@
1
1
  [package]
2
2
  name = "ulpwise"
3
- version = "0.1.2"
3
+ version = "0.2.0"
4
4
  edition = "2021"
5
5
  rust-version = "1.86"
6
6
  license = "MIT OR Apache-2.0"
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: ulpwise
3
- Version: 0.1.2
3
+ Version: 0.2.0
4
4
  Classifier: Development Status :: 3 - Alpha
5
5
  Classifier: Framework :: Pytest
6
6
  Classifier: Intended Audience :: Developers
@@ -9,8 +9,11 @@ Classifier: Programming Language :: Python :: 3
9
9
  Classifier: Programming Language :: Rust
10
10
  Classifier: Topic :: Software Development :: Testing
11
11
  Classifier: Topic :: Scientific/Engineering :: Mathematics
12
+ Requires-Dist: numpy ; extra == 'survey'
13
+ Requires-Dist: mpmath>=1.3 ; extra == 'survey'
12
14
  Requires-Dist: pytest>=7 ; extra == 'test'
13
15
  Requires-Dist: numpy ; extra == 'test'
16
+ Provides-Extra: survey
14
17
  Provides-Extra: test
15
18
  License-File: LICENSE-APACHE
16
19
  License-File: LICENSE-MIT
@@ -40,6 +43,10 @@ platform and green on another:
40
43
  3. **Where do two implementations disagree?** Knife-edge scans: the inputs whose exact result sits
41
44
  within `tol` ulp of a rounding midpoint, so that two implementations that differ by one ulp
42
45
  return different floats. 8.4 million `f32` inputs are scanned in about 0.3 s.
46
+ 4. **How accurate are the functions I call, and would the library's own tests notice?**
47
+ `ulpwise survey` measures 61 elementary and special functions of torch, numpy, scipy and jax in
48
+ ulps against a 200 bit mpmath reference and, for torch, checks every error against the tolerance
49
+ of torch's own `OpInfo` reference test, default and per op override.
43
50
 
44
51
  ## Why this exists
45
52
 
@@ -149,6 +156,7 @@ known to contain the fix, so the run tells you which bugs are present in your en
149
156
  |---|---|---|
150
157
  | kornia #4683 second derivative sign in `spatial_gradient(order=2)` | sign | 2026-09-22 |
151
158
  | kornia #4767 mixed second order kernel scale, wrong `hessian_response` determinant | scale | 2026-09-23 |
159
+ | kornia #4768 `ellipse_to_laf` under-tilted every ellipse with `b != 0` | geometry | 2026-09-24 |
152
160
  | pytorch #198006 `Multinomial.entropy()` evaluated in the default dtype | dtype | 2026-09-23 |
153
161
  | pytorch/rl #4443 `arange(0, 1, 1/n)` gives `n + 1` positions for 140 values of `n` below 2000 | rounding | 2026-09-20 |
154
162
  | pytorch/rl #4444 `min_value or -inf` drops `min_value=0` | falsy zero | 2026-09-20 |
@@ -158,9 +166,52 @@ known to contain the fix, so the run tells you which bugs are present in your en
158
166
  | timm #2791 AdaMuon conv LR scale computed from the wrong dims | scale | 2026-09-18 |
159
167
  | timm #2792 Kron `__setstate__` shadowed | crash | 2026-09-18 |
160
168
  | peft #3777 pointwise Conv3d took the conv2d 1x1 shortcut | shape | 2026-09-21 |
169
+ | ultralytics #26330 OBB train and val on plain box labels crashed in the validator or the loss instead of at load time | crash | 2026-09-25 |
170
+ | pytorch #198448 `torch.sqrt` float64 not correctly rounded at 27 of 64 knife edges | rounding | open |
171
+ | pytorch #198583 `bessel_j0/j1/y0/y1`, `airy_ai` float64 lose up to 12 digits (`p1evl` leading 1) | digits | open |
172
+ | kornia #4838 `axis_angle_to_rotation_matrix` drops the `theta^2` terms below 1e-3 rad | series | open |
173
+ | kornia #4897 `So3.log`, the `So3` Jacobians and `Se3.exp/log` lose all digits for small angles | series | open |
174
+ | torchvision #9676 `clamp_bounding_boxes` collapses slightly tilted rotated boxes to a point | geometry | open |
175
+ | pytorch #198663 `polygamma(1, x)` float64 keeps 9 digits (series stops at `1/42`), float32 loses all for large negative `x` | truncation, rounding | open |
176
+ | pytorch #198664 `erfcx` off by `x*x/2` ulps for negative `x` (`exp` at the rounded square) | rounding | open |
177
+
178
+ Cases marked `open` have an issue with the complete patch attached and no merged fix yet; they are
179
+ expected failures until a release contains the fix (`fixed_in_release` in `cases.json`), and the
180
+ `max_ulp` check type measures the digits directly. The ultralytics case builds its one-image dataset
181
+ in a temporary directory and needs no weights; two more ultralytics fixes (#26240, #26246) are not in
182
+ the corpus yet because their repros need model weights or the COCO evaluator.
183
+
184
+ ## Accuracy survey
161
185
 
162
- Two ultralytics fixes (#26240, #26246) are not in the corpus yet because their repros need model
163
- weights and a dataset layout; they will come with fixtures.
186
+ ```sh
187
+ pip install 'ulpwise[survey]' torch scipy jax # mpmath is the reference, the rest are backends
188
+ ulpwise survey --out survey # results.csv and results.md, about 3 minutes
189
+ ulpwise survey --functions bessel_j0,polygamma_1 --backends torch,scipy --dtypes f64 --points 2000
190
+ ```
191
+
192
+ For every function in `ulpwise.survey.REGISTRY` (exp, log, trig and hyperbolic functions, erf and
193
+ friends, gamma family, torch.special Bessel and Airy functions, the activation functions), every
194
+ dtype and every installed backend, the survey evaluates a log spaced grid over the function's domain
195
+ plus the named edge values of the dtype, computes the exact value with mpmath at the rounded input,
196
+ and reports max, p99 and median error in ulps, the fraction of inputs beyond 1 and 10 ulps, non
197
+ finite mismatches and the worst input. For torch it also reports how many inputs the vectorized
198
+ kernel and the scalar tail disagree on, and how many inputs would fail torch's reference test under
199
+ the dtype default tolerance and under the op's `OpInfo` override, read from `op_db`.
200
+
201
+ [`studies/accuracy-survey-2026-09`](studies/accuracy-survey-2026-09/README.md) is the first run
202
+ (torch 2.14.0+cpu, numpy 2.2.6, scipy 1.18.1, jax 0.11.2, Linux x86_64 AVX512). The short version:
203
+
204
+ - torch's `bessel_j0/j1/y0/y1` and `airy_ai` in float64 are off by 2.6e9 to 3.9e12 ulps and the
205
+ `precisionOverride({torch.float64: 1e-05})` on their tests hides every failing input.
206
+ - torch's `polygamma(1, x)` in float64 keeps about 9 digits (4.0e6 ulps, 46 percent of inputs
207
+ beyond 10 ulps) and passes the default float64 tolerance, which at `rtol = atol = 1e-7` tolerates
208
+ about 4.5e8 ulps.
209
+ - for 12 of 61 torch functions the AVX512 kernel and the scalar tail return different floats for
210
+ the same input, up to 246 of 619 inputs for `mish`.
211
+ - jax on CPU flushes subnormals to zero, its float64 `erfinv` loses 5 digits near the ends of the
212
+ interval and its float64 `log_ndtr` loses 3 digits between `x = 5.4` and 8.
213
+ - scipy's float64 `lgamma` does not handle the zeros at 1 and 2, and its Bessel functions lose the
214
+ phase at large `x`.
164
215
 
165
216
  ## How the exact oracle works
166
217
 
@@ -178,8 +229,9 @@ cross-checks both against `fractions.Fraction` and `decimal.Decimal` at 80 digit
178
229
  - Mutation scoring for numerical tests: single token mutants of the code under test (`abs`, a
179
230
  dropped `sqrt`, `/ 4` for `/ 16`) run against the test suite, reporting which survive.
180
231
  - `float16` and `bfloat16` ulps and edge values.
181
- - Exact references for transcendental functions (correctly rounded `exp`, `log`, ...) so the `f64`
182
- ones can be scanned too.
232
+ - Exact references for transcendental functions in Rust (correctly rounded `exp`, `log`, ...) so
233
+ the `f64` knife-edge scans do not need mpmath.
234
+ - Survey backends for CUDA and MPS, and `float16` / `bfloat16` rows.
183
235
  - Zero copy paths for numpy arrays and torch tensors.
184
236
  - More corpus entries, with fixtures for the cases that need data.
185
237
 
@@ -15,6 +15,10 @@ platform and green on another:
15
15
  3. **Where do two implementations disagree?** Knife-edge scans: the inputs whose exact result sits
16
16
  within `tol` ulp of a rounding midpoint, so that two implementations that differ by one ulp
17
17
  return different floats. 8.4 million `f32` inputs are scanned in about 0.3 s.
18
+ 4. **How accurate are the functions I call, and would the library's own tests notice?**
19
+ `ulpwise survey` measures 61 elementary and special functions of torch, numpy, scipy and jax in
20
+ ulps against a 200 bit mpmath reference and, for torch, checks every error against the tolerance
21
+ of torch's own `OpInfo` reference test, default and per op override.
18
22
 
19
23
  ## Why this exists
20
24
 
@@ -124,6 +128,7 @@ known to contain the fix, so the run tells you which bugs are present in your en
124
128
  |---|---|---|
125
129
  | kornia #4683 second derivative sign in `spatial_gradient(order=2)` | sign | 2026-09-22 |
126
130
  | kornia #4767 mixed second order kernel scale, wrong `hessian_response` determinant | scale | 2026-09-23 |
131
+ | kornia #4768 `ellipse_to_laf` under-tilted every ellipse with `b != 0` | geometry | 2026-09-24 |
127
132
  | pytorch #198006 `Multinomial.entropy()` evaluated in the default dtype | dtype | 2026-09-23 |
128
133
  | pytorch/rl #4443 `arange(0, 1, 1/n)` gives `n + 1` positions for 140 values of `n` below 2000 | rounding | 2026-09-20 |
129
134
  | pytorch/rl #4444 `min_value or -inf` drops `min_value=0` | falsy zero | 2026-09-20 |
@@ -133,9 +138,52 @@ known to contain the fix, so the run tells you which bugs are present in your en
133
138
  | timm #2791 AdaMuon conv LR scale computed from the wrong dims | scale | 2026-09-18 |
134
139
  | timm #2792 Kron `__setstate__` shadowed | crash | 2026-09-18 |
135
140
  | peft #3777 pointwise Conv3d took the conv2d 1x1 shortcut | shape | 2026-09-21 |
141
+ | ultralytics #26330 OBB train and val on plain box labels crashed in the validator or the loss instead of at load time | crash | 2026-09-25 |
142
+ | pytorch #198448 `torch.sqrt` float64 not correctly rounded at 27 of 64 knife edges | rounding | open |
143
+ | pytorch #198583 `bessel_j0/j1/y0/y1`, `airy_ai` float64 lose up to 12 digits (`p1evl` leading 1) | digits | open |
144
+ | kornia #4838 `axis_angle_to_rotation_matrix` drops the `theta^2` terms below 1e-3 rad | series | open |
145
+ | kornia #4897 `So3.log`, the `So3` Jacobians and `Se3.exp/log` lose all digits for small angles | series | open |
146
+ | torchvision #9676 `clamp_bounding_boxes` collapses slightly tilted rotated boxes to a point | geometry | open |
147
+ | pytorch #198663 `polygamma(1, x)` float64 keeps 9 digits (series stops at `1/42`), float32 loses all for large negative `x` | truncation, rounding | open |
148
+ | pytorch #198664 `erfcx` off by `x*x/2` ulps for negative `x` (`exp` at the rounded square) | rounding | open |
149
+
150
+ Cases marked `open` have an issue with the complete patch attached and no merged fix yet; they are
151
+ expected failures until a release contains the fix (`fixed_in_release` in `cases.json`), and the
152
+ `max_ulp` check type measures the digits directly. The ultralytics case builds its one-image dataset
153
+ in a temporary directory and needs no weights; two more ultralytics fixes (#26240, #26246) are not in
154
+ the corpus yet because their repros need model weights or the COCO evaluator.
155
+
156
+ ## Accuracy survey
136
157
 
137
- Two ultralytics fixes (#26240, #26246) are not in the corpus yet because their repros need model
138
- weights and a dataset layout; they will come with fixtures.
158
+ ```sh
159
+ pip install 'ulpwise[survey]' torch scipy jax # mpmath is the reference, the rest are backends
160
+ ulpwise survey --out survey # results.csv and results.md, about 3 minutes
161
+ ulpwise survey --functions bessel_j0,polygamma_1 --backends torch,scipy --dtypes f64 --points 2000
162
+ ```
163
+
164
+ For every function in `ulpwise.survey.REGISTRY` (exp, log, trig and hyperbolic functions, erf and
165
+ friends, gamma family, torch.special Bessel and Airy functions, the activation functions), every
166
+ dtype and every installed backend, the survey evaluates a log spaced grid over the function's domain
167
+ plus the named edge values of the dtype, computes the exact value with mpmath at the rounded input,
168
+ and reports max, p99 and median error in ulps, the fraction of inputs beyond 1 and 10 ulps, non
169
+ finite mismatches and the worst input. For torch it also reports how many inputs the vectorized
170
+ kernel and the scalar tail disagree on, and how many inputs would fail torch's reference test under
171
+ the dtype default tolerance and under the op's `OpInfo` override, read from `op_db`.
172
+
173
+ [`studies/accuracy-survey-2026-09`](studies/accuracy-survey-2026-09/README.md) is the first run
174
+ (torch 2.14.0+cpu, numpy 2.2.6, scipy 1.18.1, jax 0.11.2, Linux x86_64 AVX512). The short version:
175
+
176
+ - torch's `bessel_j0/j1/y0/y1` and `airy_ai` in float64 are off by 2.6e9 to 3.9e12 ulps and the
177
+ `precisionOverride({torch.float64: 1e-05})` on their tests hides every failing input.
178
+ - torch's `polygamma(1, x)` in float64 keeps about 9 digits (4.0e6 ulps, 46 percent of inputs
179
+ beyond 10 ulps) and passes the default float64 tolerance, which at `rtol = atol = 1e-7` tolerates
180
+ about 4.5e8 ulps.
181
+ - for 12 of 61 torch functions the AVX512 kernel and the scalar tail return different floats for
182
+ the same input, up to 246 of 619 inputs for `mish`.
183
+ - jax on CPU flushes subnormals to zero, its float64 `erfinv` loses 5 digits near the ends of the
184
+ interval and its float64 `log_ndtr` loses 3 digits between `x = 5.4` and 8.
185
+ - scipy's float64 `lgamma` does not handle the zeros at 1 and 2, and its Bessel functions lose the
186
+ phase at large `x`.
139
187
 
140
188
  ## How the exact oracle works
141
189
 
@@ -153,8 +201,9 @@ cross-checks both against `fractions.Fraction` and `decimal.Decimal` at 80 digit
153
201
  - Mutation scoring for numerical tests: single token mutants of the code under test (`abs`, a
154
202
  dropped `sqrt`, `/ 4` for `/ 16`) run against the test suite, reporting which survive.
155
203
  - `float16` and `bfloat16` ulps and edge values.
156
- - Exact references for transcendental functions (correctly rounded `exp`, `log`, ...) so the `f64`
157
- ones can be scanned too.
204
+ - Exact references for transcendental functions in Rust (correctly rounded `exp`, `log`, ...) so
205
+ the `f64` knife-edge scans do not need mpmath.
206
+ - Survey backends for CUDA and MPS, and `float16` / `bfloat16` rows.
158
207
  - Zero copy paths for numpy arrays and torch tensors.
159
208
  - More corpus entries, with fixtures for the cases that need data.
160
209
 
@@ -4,7 +4,7 @@ build-backend = "maturin"
4
4
 
5
5
  [project]
6
6
  name = "ulpwise"
7
- version = "0.1.2"
7
+ version = "0.2.0"
8
8
  description = "Numerical conformance testing for ML code: float edge values, exact rounding oracles, knife-edge finders and a pytest plugin"
9
9
  readme = "README.md"
10
10
  license = { text = "MIT OR Apache-2.0" }
@@ -25,6 +25,7 @@ dependencies = []
25
25
 
26
26
  [project.optional-dependencies]
27
27
  test = ["pytest>=7", "numpy"]
28
+ survey = ["numpy", "mpmath>=1.3"]
28
29
 
29
30
  [project.scripts]
30
31
  ulpwise = "ulpwise.__main__:main"
@@ -25,7 +25,7 @@ from ._core import ( # noqa: F401
25
25
  ulp_distances,
26
26
  )
27
27
 
28
- __version__ = "0.1.2"
28
+ __version__ = "0.2.0"
29
29
 
30
30
  __all__ = [
31
31
  "UNARY_OPS",
@@ -39,6 +39,13 @@ def main(argv=None):
39
39
  u.add_argument("b", type=float)
40
40
  u.add_argument("--dtype", default="f64")
41
41
 
42
+ v = sub.add_parser("survey", help="accuracy survey of elementary and special functions against mpmath")
43
+ v.add_argument("--backends", default="torch,numpy,scipy,jax", help="comma separated: torch numpy scipy jax")
44
+ v.add_argument("--dtypes", default="f32,f64")
45
+ v.add_argument("--points", type=int, default=600, help="grid points per function and dtype")
46
+ v.add_argument("--functions", default=None, help="comma separated subset of function names")
47
+ v.add_argument("--out", default="survey", help="output directory for results.csv and results.md")
48
+
42
49
  args = parser.parse_args(argv)
43
50
  if args.cmd == "knife":
44
51
  hits = ulpwise.knife_edges(args.op, args.lo, args.hi, args.tol, args.dtype, args.limit, args.stride)
@@ -62,6 +69,10 @@ def main(argv=None):
62
69
  print(f"{name:>26} {value!r}")
63
70
  elif args.cmd == "ulp":
64
71
  print(ulpwise.ulp_distance(args.a, args.b, args.dtype))
72
+ elif args.cmd == "survey":
73
+ from ulpwise import survey as _survey
74
+
75
+ return _survey.main(args)
65
76
  return 0
66
77
 
67
78
 
@@ -67,6 +67,10 @@ def run(case: Dict[str, Any]) -> None:
67
67
  ok = math.copysign(1.0, a) == math.copysign(1.0, e) and (a != 0) == (e != 0)
68
68
  elif kind == "allclose":
69
69
  ok = math.isfinite(a) and abs(a - e) <= check.get("atol", 0.0) + check.get("rtol", 0.0) * abs(e)
70
+ elif kind == "max_ulp":
71
+ from ulpwise import ulp_distance # noqa: PLC0415
72
+
73
+ ok = math.isfinite(a) and ulp_distance(a, e, check.get("dtype", "f64")) <= check["max_ulp"]
70
74
  else:
71
75
  raise ValueError(f"unknown check type {kind!r}")
72
76
  if not ok:
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "schema": 1,
3
- "description": "Regression corpus: numerical and API bugs found by this project's contribution pipeline and merged upstream. Each repro sets `actual` and `expected`; `check` says how to compare. `before_fix` is what the buggy release produced, quoted from the merged pull request.",
3
+ "description": "Regression corpus: numerical and API bugs found by this project's contribution pipeline. Each repro sets `actual` and `expected`; `check` says how to compare. `before_fix` is what the buggy release produced. Cases with a `pr` are merged upstream; cases with an `issue` and no `pr` are reported upstream with a patch attached and still open, so they are expected failures until the fix lands and `fixed_in_release` is set.",
4
4
  "cases": [
5
5
  {
6
6
  "id": "kornia-4683-second-order-derivative-sign",
@@ -42,6 +42,27 @@
42
42
  },
43
43
  "before_fix": "[-3.1605, 0.25, -2.1605]"
44
44
  },
45
+ {
46
+ "id": "kornia-4768-ellipse-to-laf-tilt",
47
+ "repo": "kornia/kornia",
48
+ "pr": 4768,
49
+ "merged_at": "2026-09-24T09:11:00Z",
50
+ "kind": "geometry",
51
+ "title": "ellipse_to_laf took the triangular square root of [[a, 0], [b, c]] instead of the symmetric [[a, b], [b, c]], so every ellipse with b != 0 came back under-tilted",
52
+ "symptom": "The region described by the returned LAF, inverse(A @ A.T), was [[1.0625, 0.25], [0.25, 1]] for the ellipse (a, b, c) = (1, 0.5, 1) and [[2.18, -0.3], [-0.3, 0.5]] for (2, -0.9, 0.5); LAFAffineShapeEstimator inherited the tilt error for every patch whose dominant gradient is not axis-aligned.",
53
+ "requires": [
54
+ "torch",
55
+ "kornia"
56
+ ],
57
+ "fixed_in_release": null,
58
+ "repro": "import torch\nimport kornia.feature as KF\n# Oxford ellipses [x, y, a, b, c]; the LAF's A must satisfy A @ A.T == inverse([[a, b], [b, c]])\nells = torch.tensor([[[0.0, 0.0, 1.0, 0.5, 1.0], [3.0, -2.0, 2.0, -0.9, 0.5]]], dtype=torch.float64)\nA = KF.ellipse_to_laf(ells)[..., :2]\nregion = torch.linalg.inv(A @ A.transpose(-1, -2))[0]\nactual = region.reshape(-1).tolist()\nexpected = [1.0, 0.5, 0.5, 1.0, 2.0, -0.9, -0.9, 0.5]\n",
59
+ "check": {
60
+ "type": "allclose",
61
+ "atol": 1e-12,
62
+ "rtol": 0.0
63
+ },
64
+ "before_fix": "[1.0625, 0.25, 0.25, 1.0, 2.18, -0.3, -0.3, 0.5]"
65
+ },
45
66
  {
46
67
  "id": "pytorch-198006-multinomial-entropy-dtype",
47
68
  "repo": "pytorch/pytorch",
@@ -224,6 +245,192 @@
224
245
  "rtol": 1e-05
225
246
  },
226
247
  "before_fix": "RuntimeError from the matmul inside get_delta_weight during merge_adapter"
248
+ },
249
+ {
250
+ "id": "pytorch-198448-sqrt-not-correctly-rounded",
251
+ "repo": "pytorch/pytorch",
252
+ "pr": null,
253
+ "issue": 198448,
254
+ "merged_at": null,
255
+ "kind": "rounding",
256
+ "title": "torch.sqrt on CPU is not correctly rounded when the build routes it through MKL VML",
257
+ "symptom": "At float32 knife edges in [0.5, 1) torch.sqrt returned the float below the correctly rounded result on 7,131 of 16,996 inputs (always down); numpy.sqrt and torch.sqrt in float64 then cast were right on all of them. The result depends on the platform, so a test that pins one of these inputs is green on one CI runner and red on another.",
258
+ "requires": [
259
+ "torch"
260
+ ],
261
+ "fixed_in_release": null,
262
+ "repro": "import torch\nimport ulpwise\nxs = [x for x, _, _, _ in ulpwise.knife_edges('sqrt', 0.5, 1.0, 1e-3, 'f32', 64)]\nactual = torch.sqrt(torch.tensor(xs, dtype=torch.float32)).tolist()\nexpected = [ulpwise.sqrt_cr(x, 'f32') for x in xs]\n",
263
+ "check": {
264
+ "type": "equal"
265
+ },
266
+ "before_fix": "27 of the first 64 knife edges rounded down instead of up on torch 2.14.0+cpu (MKL, AVX512); builds without MKL VML may pass"
267
+ },
268
+ {
269
+ "id": "pytorch-198583-bessel-p1evl-leading-one",
270
+ "repo": "pytorch/pytorch",
271
+ "pr": null,
272
+ "issue": 198583,
273
+ "merged_at": null,
274
+ "kind": "coefficient",
275
+ "title": "bessel_j0, bessel_j1, bessel_y0, bessel_y1 and airy_ai drop the implied leading 1 of the Cephes p1evl denominators",
276
+ "symptom": "Ten Horner loops in Math.h start at 0.0 where Cephes' p1evl starts at 1.0, so the float64 results are off by up to 1e-6 absolute (2.4e-5 relative for j0 at 5.5, 1.0e-5 for y1 at 5.27, 3.3e-5 for airy_ai at -2.30); the OpInfo tests tolerate it through a float64 precisionOverride of 1e-5. Reported by the-Shallow for j0, j1 and y0; y1 and airy_ai added in the comments with the patch.",
277
+ "requires": [
278
+ "torch"
279
+ ],
280
+ "fixed_in_release": null,
281
+ "repro": "import torch\nx = torch.tensor([5.26993], dtype=torch.float64)\nactual = [\n torch.special.bessel_j0(x).item(),\n torch.special.bessel_j1(x).item(),\n torch.special.bessel_y0(x).item(),\n torch.special.bessel_y1(x).item(),\n torch.special.airy_ai(torch.tensor([-2.30154], dtype=torch.float64)).item(),\n]\n# mpmath at 200 bits, rounded to float64\nexpected = [-0.086199871814967743, -0.34549232119438467, -0.33594130771681818, 0.054955745481668872, 0.025628209343452262]\n",
282
+ "check": {
283
+ "type": "max_ulp",
284
+ "dtype": "f64",
285
+ "max_ulp": 8
286
+ },
287
+ "before_fix": "[-0.08620011880207763, -0.34549245198818446, -0.3359412504444201, 0.05495518143365105, 0.025629044485656297] on torch 2.14.0 (1.8e10 to 2.5e14 ulp)"
288
+ },
289
+ {
290
+ "id": "kornia-4838-axis-angle-low-angle-terms",
291
+ "repo": "kornia/kornia",
292
+ "pr": null,
293
+ "issue": 4838,
294
+ "merged_at": null,
295
+ "kind": "truncation",
296
+ "title": "axis_angle_to_rotation_matrix low-angle branch is off by theta**3 / 6",
297
+ "symptom": "For theta**2 <= 1e-6 the branch returns I + [v]x + [v]x**2 / 2, so at theta = 9.99e-4 in float64 the matrix is 1.3e-10 from the rotation of its input and from quaternion_to_rotation_matrix(axis_angle_to_quaternion(v)), next to a general branch exact to 2.2e-16.",
298
+ "requires": [
299
+ "torch",
300
+ "kornia"
301
+ ],
302
+ "fixed_in_release": null,
303
+ "repro": "import torch\nimport kornia.geometry.conversions as C\naxis = torch.tensor([1.0, 2.0, 3.0], dtype=torch.float64) / 14.0**0.5\nv = (9.99e-4 * axis)[None]\nactual = C.axis_angle_to_rotation_matrix(v).flatten().tolist()\nexpected = C.quaternion_to_rotation_matrix(C.axis_angle_to_quaternion(v)).flatten().tolist()\n",
304
+ "check": {
305
+ "type": "allclose",
306
+ "atol": 1e-12,
307
+ "rtol": 0.0
308
+ },
309
+ "before_fix": "max |direct - quaternion route| = 1.33e-10 on kornia main 320f461"
310
+ },
311
+ {
312
+ "id": "kornia-4897-so3-se3-small-angle-digits",
313
+ "repo": "kornia/kornia",
314
+ "pr": null,
315
+ "issue": 4897,
316
+ "merged_at": null,
317
+ "kind": "cancellation",
318
+ "title": "So3.log, the So3 Jacobians and Se3.exp/log lose all their digits for small rotations",
319
+ "symptom": "So3.log uses 2 * acos(real) and Se3.exp/log form (1 - cos(theta)) / theta**2 and (theta - sin(theta)) / theta**3 directly, so in float32 a rotation of 1e-4 rad logs back to 0 and Se3.exp([1, 0, 0, 0, 0, 1e-4]) translates by (1, 0, 0) instead of (1, 5e-5, 0); the Jacobians are nan at the identity.",
320
+ "requires": [
321
+ "torch",
322
+ "kornia"
323
+ ],
324
+ "fixed_in_release": null,
325
+ "repro": "import torch\nfrom kornia.geometry.liegroup import Se3, So3\nrot = So3.exp(torch.tensor([[1e-4, 0.0, 0.0]], dtype=torch.float32))\nse = Se3.exp(torch.tensor([[1.0, 0.0, 0.0, 0.0, 0.0, 1e-4]], dtype=torch.float32))\nactual = [rot.log()[0, 0].item(), se.t[0, 1].item()]\nexpected = [1e-4, 5e-5]\n",
326
+ "check": {
327
+ "type": "allclose",
328
+ "atol": 0.0,
329
+ "rtol": 0.001
330
+ },
331
+ "before_fix": "[0.0, 0.0] on kornia main 7ddf731"
332
+ },
333
+ {
334
+ "id": "torchvision-9676-rotated-box-clamp-collapse",
335
+ "repo": "pytorch/vision",
336
+ "pr": null,
337
+ "issue": 9676,
338
+ "merged_at": null,
339
+ "kind": "ordering",
340
+ "title": "clamp_bounding_boxes collapses slightly tilted rotated boxes to a point",
341
+ "symptom": "_order_bounding_boxes_points picks the first corner with (y / y_max + 100 * x / x_max).min(), so for tilts within about half a degree of 0 or 180 the y term outweighs the x difference and a box that crosses the left edge clamps to [0, 200, 0, 0, 0] in both soft and hard mode.",
342
+ "requires": [
343
+ "torch",
344
+ "torchvision"
345
+ ],
346
+ "fixed_in_release": null,
347
+ "repro": "import torch\nfrom torchvision import tv_tensors\nfrom torchvision.transforms.v2 import functional as F\nbox = tv_tensors.BoundingBoxes(\n torch.tensor([[13.355, 19.723, 75.347, 48.114, -0.2]]), format='CXCYWHR', canvas_size=(200, 300)\n)\nout = F.clamp_bounding_boxes(box)\nactual = [(out[0, 2] * out[0, 3]).item()] # area of the clamped box\nexpected = [1.0] # any positive area\n",
348
+ "check": {
349
+ "type": "sign"
350
+ },
351
+ "before_fix": "[0.0, 200.0, 0.0, 0.0, 0.0] on torchvision 0.29.0, area 0"
352
+ },
353
+ {
354
+ "id": "pytorch-198663-trigamma-float64-series-truncation",
355
+ "repo": "pytorch/pytorch",
356
+ "pr": null,
357
+ "issue": 198663,
358
+ "merged_at": null,
359
+ "kind": "truncation",
360
+ "title": "torch.polygamma(1, x) keeps about 9 digits in float64 because trigamma stops the asymptotic series at the 1/42 term",
361
+ "symptom": "trigamma(double) in Math.h shifts x up by 6 and sums the series through B_6, so the first dropped term, 1/(30 x^9), leaves a relative error near 1e-9 in every float64 result: 4.0e6 ulps at x = 0.9047, 557 of 800 positive inputs more than 4 ulps off, 3.0e7 ulps on negative non integers. The reference test tolerance for float64 is rtol = atol = 1e-7, about 4.5e8 ulps, so it never saw it.",
362
+ "requires": [
363
+ "torch"
364
+ ],
365
+ "fixed_in_release": null,
366
+ "repro": "import torch\nxs = torch.tensor([0.9047357242349293, 7.75, -2.75], dtype=torch.float64)\nactual = torch.polygamma(1, xs).tolist()\n# mpmath polygamma(1, x) at 200 bits, rounded to float64\nexpected = [1.9074856057949192, 0.13771379144765566, 19.433868949488463]\n",
367
+ "check": {
368
+ "type": "max_ulp",
369
+ "dtype": "f64",
370
+ "max_ulp": 8
371
+ },
372
+ "before_fix": "[1.9074856066876746, 0.13771379144953064, 19.43386894944752] on torch 2.14.0 (4.0e6, 6.8e4 and 1.2e4 ulp)"
373
+ },
374
+ {
375
+ "id": "pytorch-198663-trigamma-float32-reflection-argument",
376
+ "repo": "pytorch/pytorch",
377
+ "pr": null,
378
+ "issue": 198663,
379
+ "merged_at": null,
380
+ "kind": "rounding",
381
+ "title": "torch.polygamma(1, x) in float32 loses every digit for large negative x because the reflection evaluates sin(pi * x) at the rounded product",
382
+ "symptom": "pi * x is rounded to float32 before the sine, so at x = -60618.9921875 the sine is taken 0.0078 away from the right point and the result is 31981.7 for 16387.3; the poles at the non positive integers come out finite (1.3e15 at -1) instead of inf. Same code in the CUDA trigamma_string and calc_trigamma.",
383
+ "requires": [
384
+ "torch"
385
+ ],
386
+ "fixed_in_release": null,
387
+ "repro": "import torch\nxs = torch.tensor([-60618.9921875, -12345.03125], dtype=torch.float32) # exact in float32\nactual = torch.polygamma(1, xs).tolist()\n# mpmath polygamma(1, x) at 200 bits, rounded to float32\nexpected = [16387.291015625, 1027.296142578125]\n",
388
+ "check": {
389
+ "type": "max_ulp",
390
+ "dtype": "f32",
391
+ "max_ulp": 8
392
+ },
393
+ "before_fix": "[31981.72265625, 967.1527709960938] on torch 2.14.0"
394
+ },
395
+ {
396
+ "id": "pytorch-198664-erfcx-negative-rounded-square",
397
+ "repo": "pytorch/pytorch",
398
+ "pr": null,
399
+ "issue": 198664,
400
+ "merged_at": null,
401
+ "kind": "rounding",
402
+ "title": "torch.special.erfcx is off by up to x*x/2 ulps for negative x because exp(x*x) is taken at the rounded product",
403
+ "symptom": "The negative branch returns 2 * exp(x * x) - erfcx_y100(400 / (4 - x)); the absolute rounding error of x * x becomes the relative error of exp, so the result is off by up to ulp(x * x) / 2: 44 float32 ulps at x = -8.44, 157 float64 ulps at x = -23.25, 504 float64 ulps at worst over 800 negative inputs. scipy.special.erfcx returns the same wrong float64 value, so only a multiprecision reference sees it.",
404
+ "requires": [
405
+ "torch"
406
+ ],
407
+ "fixed_in_release": null,
408
+ "repro": "import torch\nxs = torch.tensor([-23.25459451307119, -9.1], dtype=torch.float64)\nactual = torch.special.erfcx(xs).tolist()\n# exp(x * x) * erfc(x) from mpmath at 200 bits, rounded to float64\nexpected = [1.435935128289743e+235, 1.8405856937190915e+36]\n",
409
+ "check": {
410
+ "type": "max_ulp",
411
+ "dtype": "f64",
412
+ "max_ulp": 8
413
+ },
414
+ "before_fix": "[1.4359351282897875e+235, 1.8405856937190814e+36] on torch 2.14.0 (157 and 34 ulp)"
415
+ },
416
+ {
417
+ "id": "ultralytics-26330-obb-dataset-box-only-labels",
418
+ "repo": "ultralytics/ultralytics",
419
+ "pr": 26330,
420
+ "merged_at": "2026-09-25T22:12:25Z",
421
+ "kind": "crash",
422
+ "title": "OBB val and train on a dataset with plain box labels crashed deep in the validator or the loss instead of failing at load time",
423
+ "symptom": "With task=obb the loader builds the rotated boxes from the polygon points, so a detect dataset arrives as (0, 5) boxes next to n classes: yolo val crashed with IndexError in OBBValidator._prepare_batch and training with a TypeError in the loss. Segment datasets already raised a ValueError at load time; the fix applies the same box/segment count check when use_obb is set, with the task named in the message.",
424
+ "requires": [
425
+ "torch",
426
+ "ultralytics"
427
+ ],
428
+ "fixed_in_release": null,
429
+ "repro": "import tempfile\nfrom pathlib import Path\nimport cv2\nimport numpy as np\nfrom ultralytics.cfg import get_cfg\nfrom ultralytics.data import build_yolo_dataset\nroot = Path(tempfile.mkdtemp())\n(root / \"images\").mkdir()\n(root / \"labels\").mkdir()\ncv2.imwrite(str(root / \"images\" / \"0.jpg\"), np.zeros((32, 32, 3), np.uint8))\n(root / \"labels\" / \"0.txt\").write_text(\"0 0.5 0.5 0.4 0.4\\n\") # a plain box row, not the 4 corners of an OBB\ndata = {\"names\": {0: \"a\"}, \"nc\": 1, \"channels\": 3}\ntry:\n build_yolo_dataset(get_cfg(overrides={\"task\": \"obb\", \"imgsz\": 32}), str(root / \"images\"), 1, data, mode=\"val\")\n actual = [\"no error at load time\"]\nexcept ValueError as e:\n actual = [\"ValueError\" if \"OBB dataset requires equal numbers of boxes and segments\" in str(e) else str(e)]\nexpected = [\"ValueError\"]\n",
430
+ "check": {
431
+ "type": "equal"
432
+ },
433
+ "before_fix": "['no error at load time'] on ultralytics 8.4.163: the dataset loads and the crash comes later, in val or in the loss"
227
434
  }
228
435
  ]
229
- }
436
+ }