fast-taylor-verifier 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Yasin Sonmez
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,564 @@
1
+ Metadata-Version: 2.4
2
+ Name: fast-taylor-verifier
3
+ Version: 0.1.0
4
+ Summary: Taylor-model bounds on batches of boxes in PyTorch and CUDA, and a branch and bound over boxes
5
+ License-Expression: MIT
6
+ Project-URL: Repository, https://github.com/YasinSonmez/fast-taylor-verifier
7
+ Project-URL: Paper code, https://github.com/YasinSonmez/certified-roa-fvi
8
+ Keywords: taylor models,verification,interval arithmetic,branch and bound,neural networks,cuda,pytorch
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Operating System :: POSIX :: Linux
11
+ Classifier: Environment :: GPU :: NVIDIA CUDA
12
+ Classifier: Topic :: Scientific/Engineering :: Mathematics
13
+ Requires-Python: >=3.10
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: torch
17
+ Requires-Dist: ninja
18
+ Dynamic: license-file
19
+
20
+ # fast-taylor-verifier
21
+
22
+ [![PyPI](https://img.shields.io/pypi/v/fast-taylor-verifier)](https://pypi.org/project/fast-taylor-verifier/)
23
+ [![Python](https://img.shields.io/pypi/pyversions/fast-taylor-verifier)](https://pypi.org/project/fast-taylor-verifier/)
24
+
25
+ **Fast GPU-parallel Taylor-model bounds for smooth functions and neural networks.**
26
+
27
+ `fast-taylor-verifier` computes lower and upper bounds of a function over an entire input region, including functions
28
+ involving supported smooth neural networks. Unlike testing a function at sampled points, these bounds apply throughout
29
+ the region.
30
+
31
+ The package uses first-, second-, or third-order Taylor models to bound functions written with supported PyTorch
32
+ operations. It is designed for problems with a small number of input dimensions where many boxes must be evaluated.
33
+
34
+ Typical applications include:
35
+
36
+ - **Neural-network verification:** Bound the outputs and derivatives of supported smooth networks and check inequalities
37
+ involving them.
38
+ - **Control and robotics:** Verify Lyapunov and barrier-function conditions, including Lie-derivative inequalities used
39
+ in stability and safety analysis.
40
+ - **Global optimization:** Bound function minima over a region using branch-and-bound.
41
+ - **Numerical analysis:** Bound Jacobians and other derivatives of nonlinear functions.
42
+
43
+ To verify an inequality, the package divides the input region into boxes and computes bounds on each box. Boxes where
44
+ the inequality holds are discarded; the remaining boxes are subdivided and checked again.
45
+
46
+ ![A branch-and-bound proof, second- vs first-order Taylor models](https://raw.githubusercontent.com/YasinSonmez/fast-taylor-verifier/main/assets/proof_movie.gif)
47
+
48
+ *A proof that a neural Lyapunov function decreases (dV/dt < 0) on a region, as it runs. Green boxes are proved and red ones are split again; darker boxes are smaller. With second-order Taylor models (left) the proof needs 7.6k boxes, with first-order ones (right) 60k. At the end, the colour shows how often each box was split. From the [paper repository](https://github.com/YasinSonmez/certified-roa-fvi).*
49
+
50
+ The implementation runs with PyTorch on CPU or GPU. For higher performance, it also generates specialized CUDA kernels
51
+ that evaluate large numbers of boxes in parallel. On a two-dimensional neural Lyapunov verification benchmark, the CUDA
52
+ implementation was approximately 600× faster than CROWN on the same GPU.
53
+
54
+ > **Paper.** This package was developed for *Learning Controllers with Certified Regions of Attraction via Fitted Value
55
+ > Iteration and Second-Order Taylor Bounds*. The code to reproduce the paper is in
56
+ > [YasinSonmez/certified-roa-fvi](https://github.com/YasinSonmez/certified-roa-fvi).
57
+
58
+ ## Contents
59
+
60
+ - [Installation](#installation)
61
+ - [Quickstart: prove an inequality](#quickstart-prove-an-inequality)
62
+ - [Computing bounds directly](#computing-bounds-directly)
63
+ - [Bound a neural network](#bound-a-neural-network)
64
+ - [How it works](#how-it-works)
65
+ - [More examples](#more-examples)
66
+ - [Performance](#performance)
67
+ - [Supported operations](#supported-operations)
68
+ - [Proving a property: `branch_and_bound`](#proving-a-property-branch_and_bound)
69
+ - [CUDA](#cuda)
70
+ - [Neural networks and derivatives](#neural-networks-and-derivatives)
71
+ - [Limitations](#limitations)
72
+ - [API reference](#api-reference)
73
+ - [Running the tests](#running-the-tests)
74
+ - [License](#license)
75
+
76
+ ## Installation
77
+
78
+ ```bash
79
+ pip install fast-taylor-verifier
80
+ ```
81
+
82
+ | Requirement | Needed for |
83
+ |---|---|
84
+ | Python ≥ 3.10, PyTorch | everything; the PyTorch code runs on CPU or GPU |
85
+ | an NVIDIA GPU, the CUDA toolkit (`nvcc`) matching your PyTorch build, a C++ compiler, `ninja` | the CUDA kernels in `fast_taylor_verifier.cuda` |
86
+
87
+ The optional CUDA kernels are compiled on first use and cached for later runs (see [CUDA](#cuda)).
88
+
89
+ ## Quickstart: prove an inequality
90
+
91
+ Prove that `x² + y² - 0.5xy > 0.37` for all `(x, y)` in `[0.5, 1]²`. `check` receives a batch of boxes, given by
92
+ their lower corners `L` and upper corners `U`, and returns the boxes where the lower bound is not yet above 0.37.
93
+ `branch_and_bound` splits those and calls `check` again:
94
+
95
+ ```python
96
+ import torch
97
+ from fast_taylor_verifier import TaylorModel, branch_and_bound, evaluate
98
+
99
+ def f(p): # p = (x, y), one point
100
+ x, y = p
101
+ return x ** 2 + y ** 2 - 0.5 * x * y
102
+
103
+ def check(L, U):
104
+ m = evaluate(f, TaylorModel.box((L + U) / 2, (U - L) / 2)) # f on every box of the batch
105
+ lo, hi = m.bounds()
106
+ not_proved = lo[:, 0] <= 0.37
107
+ return not_proved, m.scores()[not_proved] # scores: which side of each box to split
108
+
109
+ r = branch_and_bound(torch.full((1, 2), 0.5), torch.ones(1, 2), check, batch=4096, h_min=1e-4)
110
+ print("proved" if r["failed"] == r["queued"] == 0 else "not proved")
111
+ ```
112
+
113
+ ## Computing bounds directly
114
+
115
+ You write `f` for a single point `x`, with `x[i]` its `i`-th input, using PyTorch functions. The same `f` runs on an
116
+ ordinary tensor such as `torch.tensor([0.5, 1.0])`. To bound it, create a batch of `B` boxes with `TaylorModel.box`,
117
+ from their centres and half-widths (two `(B, n)` tensors), and call `evaluate`, which runs `f` on all boxes at once:
118
+
119
+ ```python
120
+ import torch
121
+ from fast_taylor_verifier import TaylorModel, evaluate
122
+
123
+ def f(x): # x = (x[0], x[1]), one point
124
+ return torch.sin(x[0]) * x[1] + torch.exp(-x[1] ** 2)
125
+
126
+ centres = torch.tensor([[0.0, 0.0], [0.5, 1.0], [1.0, -1.0]])
127
+ boxes = TaylorModel.box(centres, torch.full((3, 2), 0.1)) # 3 boxes, each of half-width 0.1 in both directions
128
+ lo, hi = evaluate(f, boxes).bounds() # (3, 1) each
129
+ ```
130
+
131
+ A function with several outputs returns a tuple, `return (y0, y1)`; the bounds then have one column per output.
132
+
133
+ For each box `b`, `lo[b] ≤ f(y) ≤ hi[b]` for every point `y` in the box:
134
+
135
+ ```
136
+ box 1: 0.9799 ≤ f ≤ 1.0101
137
+ box 2: 0.7246 ≤ f ≤ 0.9726
138
+ box 3: -0.6849 ≤ f ≤ -0.2465
139
+ ```
140
+
141
+ ## Bound a neural network
142
+
143
+ The package also works with supported smooth neural networks. This example computes output bounds for a tanh multilayer
144
+ perceptron on a two-dimensional input box:
145
+
146
+ ```python
147
+ import torch
148
+ from torch import nn
149
+ from types import SimpleNamespace
150
+ from fast_taylor_verifier import TaylorModel, network
151
+
152
+ torch.manual_seed(0)
153
+ net = SimpleNamespace(s=nn.Sequential(
154
+ nn.Linear(2, 16),
155
+ nn.Tanh(),
156
+ nn.Linear(16, 1),
157
+ ))
158
+
159
+ centres = torch.tensor([[0.0, 0.0]])
160
+ half_widths = torch.tensor([[0.1, 0.1]])
161
+ boxes = TaylorModel.box(centres, half_widths)
162
+
163
+ with torch.no_grad():
164
+ output, along = network(net, boxes)
165
+ lo, hi = output.bounds()
166
+ print(lo.item(), hi.item())
167
+ ```
168
+
169
+ `network` expects `net.s` to be an `nn.Sequential` of `nn.Linear` layers with `nn.Tanh` activations and one output. The
170
+ returned `along(v)` also bounds the directional derivative of the network along a vector field `v`, which is useful for
171
+ Lyapunov and other derivative inequalities. See [Neural networks and derivatives](#neural-networks-and-derivatives).
172
+
173
+ ## How it works
174
+
175
+ Take a box with centre `x0` and half-widths `h`, and write a point in it as `x = x0 + d` with `|d| ≤ h`. A **Taylor
176
+ model** of a quantity `u(x)` on the box is a polynomial in `d` plus an error interval `[-r, r]`:
177
+
178
+ ```
179
+ u(x0 + d) ∈ c + Lᵀd + dᵀQd + T[d, d, d] + [-r, r] for every |d| ≤ h
180
+ ```
181
+
182
+ The order of the model is the degree of the polynomial: 1, 2 (with `Q`) or 3 (with `T`).
183
+
184
+ The computation starts from the input itself, `x = x0 + d`, which is exact (`r = 0`), and follows the operations in
185
+ `f`:
186
+
187
+ - **Sums** add the polynomials and the errors.
188
+ - **Products** multiply the polynomials. Terms of degree above the order are bounded on the box and added to `r`.
189
+ - **Functions** such as `sin(u)` are replaced by their Taylor polynomial around the centre value of `u`. The
190
+ truncation error is bounded with the next derivative of `sin` over the range of `u`, and added to `r`.
191
+
192
+ At the end, the bounds are the range of the polynomial on the box plus `[-r, r]`.
193
+
194
+ The reason to keep a polynomial is that it remembers how each quantity depends on `x`. Interval arithmetic, the
195
+ simplest alternative, keeps only a range per quantity: for `x` in `[0, 1]` it computes `x - x` as
196
+ `[0, 1] - [0, 1] = [-1, 1]`, whereas a Taylor model gives exactly `0`. As a result, the bounds of a Taylor model get
197
+ closer to the true range as boxes get smaller, and those of interval arithmetic often do not. For `f` above, on boxes
198
+ centred at `(0.7, 0.3)`, the width of the bounds exceeds the width of the true range by:
199
+
200
+ | Half-width of the box | Interval arithmetic | Order 1 | Order 2 | Order 3 |
201
+ |---|---|---|---|---|
202
+ | 0.4 | 160 % | 150 % | 82 % | 80 % |
203
+ | 0.1 | 330 % | 62 % | 28 % | 27 % |
204
+ | 0.025 | 340 % | 15 % | 6 % | 6 % |
205
+
206
+ Tighter bounds mean that fewer boxes need to be split to prove a property.
207
+
208
+ ## More examples
209
+
210
+ The timings are on one NVIDIA RTX 5090.
211
+
212
+ ### Bound the global minimum of a function
213
+
214
+ Here `check` remembers the smallest value of `f` found so far at a box centre. A box whose lower bound is above that
215
+ value (minus a tolerance of `1e-4`) cannot contain a smaller value, so it is discarded. When every box is discarded,
216
+ the global minimum is known to within the tolerance:
217
+
218
+ ```python
219
+ import torch
220
+ from fast_taylor_verifier import TaylorModel, branch_and_bound, evaluate
221
+
222
+ def camel(x): # the six-hump camel function
223
+ a, b = x
224
+ return (4 - 2.1 * a ** 2 + a ** 4 / 3) * a ** 2 + a * b + (4 * b ** 2 - 4) * b ** 2
225
+
226
+ best = [float("inf")] # smallest value found so far
227
+
228
+ def check(L, U):
229
+ m = evaluate(camel, TaylorModel.box((L + U) / 2, (U - L) / 2))
230
+ best[0] = min(best[0], m.c.min().item()) # m.c: the values at the box centres
231
+ may_be_lower = m.bounds()[0][:, 0] < best[0] - 1e-4
232
+ return may_be_lower, m.scores()[may_be_lower]
233
+
234
+ r = branch_and_bound(torch.tensor([[-2.0, -1.0]]), torch.tensor([[2.0, 1.0]]), check, batch=4096, h_min=1e-6)
235
+ print(f"global minimum in [{best[0] - 1e-4:.5f}, {best[0]:.5f}] after {r['boxes']} boxes")
236
+ # global minimum in [-1.03169, -1.03159] after 203 boxes
237
+ ```
238
+
239
+ ### Verify a neural certificate for a controlled system
240
+
241
+ Consider a system `ẋ = f(x, π(x))` with a neural-network controller `π`, and a neural network `V(x)` meant to prove
242
+ that the system converges (a Lyapunov function) or stays safe (a barrier function). The proof requires that `V`
243
+ decreases along the system's trajectories, `∇V(x) · f(x, π(x)) < 0`, at every point of a region. `network(net, x)`
244
+ returns the model of `V` on each box and a function `along` that bounds the derivative of `V` along any vector field:
245
+
246
+ ```python
247
+ s, along = network(net, x) # s: V on the boxes; along(v): ∇V · v
248
+ lie_lo, lie_hi = along(evaluate(closed_loop, x)).bounds() # closed_loop(p): ẋ at one point p
249
+ not_proved = lie_hi[:, 0] >= 0
250
+ ```
251
+
252
+ Times to prove the region of attraction of a
253
+ neural controller, from our paper:
254
+
255
+ | System | States | Boxes | Time |
256
+ |---|---|---|---|
257
+ | Van der Pol oscillator | 2 | 12 thousand | 15 ms |
258
+ | Inverted pendulum | 2 | 7.5 million | 0.39 s |
259
+ | Cart-pole | 4 | 16 million | 0.33 s |
260
+ | Planar quadrotor | 6 | 1.1 billion | 63 s |
261
+
262
+ On the neural barrier-function benchmarks of Vertovec et al. (L4DC 2026), it proves the same five certificates
263
+ 2.6 to 81 times faster than their method.
264
+
265
+ ### Bound a Jacobian
266
+
267
+ `jacobian(f, x)` bounds every entry of the Jacobian `∂f/∂x` on each box. The largest absolute value over all boxes
268
+ bounds each partial derivative over the whole region, which gives a Lipschitz constant:
269
+
270
+ ```python
271
+ from fast_taylor_verifier import jacobian
272
+
273
+ lo, hi = jacobian(f, TaylorModel.box(centres, half_widths)) # each (B, m, n)
274
+ max_abs = torch.maximum(lo.abs(), hi.abs()).amax(0) # (m, n): bound of |∂f_i/∂x_j| on all boxes
275
+ ```
276
+
277
+ ## Performance
278
+
279
+ The same region-of-attraction proof (Van der Pol oscillator, a neural Lyapunov function), with the bounds computed
280
+ three ways:
281
+
282
+ | Bounds computed by | Boxes | Time |
283
+ |---|---|---|
284
+ | CROWN ([auto_LiRPA](https://github.com/Verified-Intelligence/auto_LiRPA)) | 33 thousand | 8.3 s |
285
+ | `TaylorModel` in PyTorch, order 2 | 16 thousand | 0.47 s |
286
+ | `cuda`, order 2 | 16 thousand | 14 ms |
287
+
288
+ CROWN bounds general neural networks with linear functions. For this problem, a smooth function of two inputs, the
289
+ order-2 Taylor models needed half as many boxes, and the CUDA kernel was about 600 times faster than CROWN on the same
290
+ GPU.
291
+
292
+ ## Supported operations
293
+
294
+ In a function `f` passed to `evaluate`, `jacobian` or `cuda.compile`:
295
+
296
+ | Operation | How to write it |
297
+ |---|---|
298
+ | inputs | `x[i]`, or `a, b = x` |
299
+ | outputs | return one value, or a tuple of values |
300
+ | arithmetic | `a + b`, `a - b`, `a * b`, `a / b`, `-a`, with numbers or other values |
301
+ | integer powers | `a ** k` for `k = 1, 2, 3, ...` |
302
+ | functions | `torch.sin`, `torch.cos`, `torch.tanh`, `torch.exp`, `torch.log` (or `a.sin()`, ...), `a.reciprocal()` |
303
+
304
+ `f` may not branch on values (`if x[0] > 0:`), because on a box `x[0]` is a range, not a number.
305
+
306
+ On boxes where the argument of `log` or `reciprocal` may be zero or negative, the bounds are infinite.
307
+
308
+ <details>
309
+ <summary><b>Batched code</b></summary>
310
+
311
+ A `TaylorModel` holding `B` boxes and `k` outputs also behaves like a `(B, k)` tensor, so code written for batches of
312
+ points runs on it directly: `f(TaylorModel.box(centres, half_widths))`. This is how `network` works, and it supports a
313
+ few more operations: `x[:, i:j]` (slices only, so shapes stay `(B, 1)`), `torch.cat([...], 1)`,
314
+ `a.sum(1, keepdim=True)`, `a.linear(W, b)` (`a Wᵀ + b`) and `a.sech2()`.
315
+
316
+ </details>
317
+
318
+ The model `m` returned by `evaluate` gives:
319
+
320
+ | Attribute | Shape | Contents |
321
+ |---|---|---|
322
+ | `m.bounds()` | `(B, k)`, `(B, k)` | lower and upper bounds on each box |
323
+ | `m.c` | `(B, k)` | the value at each box's centre |
324
+ | `m.width()` | `(B, k)` | how far the value can move from the centre value within the box |
325
+ | `m.scores()` | `(B, n)` | how much each input direction contributes to the variation of the first output on the box; `branch_and_bound` splits the box along the largest |
326
+
327
+ ## Proving a property: `branch_and_bound`
328
+
329
+ ```python
330
+ r = branch_and_bound(L, U, check, batch, h_min)
331
+ ```
332
+
333
+ `L` and `U` are `(B, n)` tensors with the lower and upper corners of the starting boxes. You write `check(L, U)`: it
334
+ receives a batch of boxes and returns
335
+
336
+ 1. a boolean tensor of shape `(B,)` that is `True` for the boxes that are **not yet proved**, and
337
+ 2. split scores for those boxes, of shape `(number not proved, n)`, or `None` to split each box along its widest side.
338
+
339
+ `branch_and_bound` keeps a stack of boxes, passes up to `batch` boxes at a time to `check`, and splits every box that
340
+ is not proved into two halves along the side with the largest score (sides shorter than `2·h_min` are not split). A
341
+ box that is not proved when its largest half-width is at most `h_min` is not split further and is counted as
342
+ **failed**. The result is a dictionary:
343
+
344
+ | Key | Contents |
345
+ |---|---|
346
+ | `boxes` | number of boxes checked |
347
+ | `failed` | number of failed boxes |
348
+ | `queued` | number of boxes still waiting when the search stopped early (deadline, `max_failed` or `stop`) |
349
+ | `batch` | the batch size at the end; it is halved when CUDA runs out of memory |
350
+
351
+ The property is proved when `failed` and `queued` are both 0.
352
+
353
+ `check` is ordinary Python, so it can do more than compare a bound: keep state between calls (as in the
354
+ [global-minimum example](#bound-the-global-minimum-of-a-function)), record boxes where the property is violated, or
355
+ combine several bounds.
356
+
357
+ <details>
358
+ <summary><b>Optional arguments</b></summary>
359
+
360
+ | Argument | Meaning |
361
+ |---|---|
362
+ | `scale` | the unit of length in each direction, a number or an `(n,)` tensor; `h_min` and the comparison of side lengths use it |
363
+ | `aspect` | only split sides that are at least `aspect` times the longest side of the box (default 0: any side) |
364
+ | `tie_break` | add `tie_break` times the side length to the scores, so that equal scores favour the longer side |
365
+ | `deadline` | stop at this `time.time()` |
366
+ | `max_failed` | stop when more boxes than this have failed |
367
+ | `stop` | a function called before each batch; the search stops when it returns `True` |
368
+ | `log`, `log_every` | `log(boxes, queued)` is called at most every `log_every` seconds |
369
+
370
+ </details>
371
+
372
+ `split(L, U, score, ...)` performs one splitting step and can be used on its own.
373
+
374
+ ## CUDA
375
+
376
+ Putting PyTorch tensors on a CUDA device makes `TaylorModel` use ordinary PyTorch GPU operations; it
377
+ does **not** compile the whole function into one kernel. `cuda.compile(f)` instead generates a specialized
378
+ CUDA kernel for supported operations, with one GPU thread per box. It runs `f` once to record its operations,
379
+ then generates and compiles the kernel:
380
+
381
+ ```python
382
+ from fast_taylor_verifier import Settings, cuda
383
+
384
+ bound = cuda.compile(f) # f from "Computing bounds directly" above
385
+ centres = centres.to("cuda")
386
+ half_widths = torch.full_like(centres, 0.1)
387
+ box = cuda.Box(centres, half_widths, Settings(order=2))
388
+ lo, hi = bound(box) # same bounds as evaluate(f, TaylorModel.box(...)).bounds()
389
+ ```
390
+
391
+ `compile` supports the operations in the first table of [Supported operations](#supported-operations); for networks, use
392
+ `cuda.network`. The PyTorch and CUDA versions compute the same bounds up to float32 rounding, which the tests check.
393
+ Used inside `check`, to prove `f > 0.84` on `[0.5, 1]²` (the minimum there is 0.847):
394
+
395
+ ```python
396
+ def check(L, U):
397
+ lo, hi = bound(cuda.Box((L + U) / 2, (U - L) / 2, Settings(order=2)))
398
+ return lo[:, 0] <= 0.84, None
399
+
400
+ r = branch_and_bound(torch.full((1, 2), 0.5, device="cuda"), torch.ones(1, 2, device="cuda"), check,
401
+ batch=4096, h_min=1e-4)
402
+ print("proved" if r["failed"] == r["queued"] == 0 else "not proved")
403
+ ```
404
+
405
+ <details>
406
+ <summary><b>Compilation and caching</b></summary>
407
+
408
+ The CUDA code needs an NVIDIA GPU, the CUDA toolkit (`nvcc`) matching your PyTorch build, a C++ compiler and `ninja`.
409
+ The kernels are compiled the first time they are used, once for each function, number of inputs and settings, and
410
+ stored in `~/.cache/torch_extensions`. Compiling takes about a minute at order 2 and several minutes at order 3 with
411
+ many inputs. Later runs load the stored kernel.
412
+
413
+ </details>
414
+
415
+ <details>
416
+ <summary><b>Writing your own CUDA kernel</b></summary>
417
+
418
+ `cuda.compile` handles functions of the box. For anything else, such as a network inside the check or several bounds
419
+ per box, you can write a CUDA kernel that uses the same Taylor-model operations and compile it with `cuda.build`:
420
+
421
+ ```python
422
+ ext = cuda.build("my_test", "void run(torch::Tensor x0, torch::Tensor h, torch::Tensor out);",
423
+ open("my_test.cu").read(), "run", cuda.flags(settings))
424
+ ```
425
+
426
+ The source includes `taylor.cuh` for the Taylor-model operations and, to evaluate a tanh network, `network.cu`
427
+ (`launch_network`). For a box with `N` inputs, a model is stored as an array `float m[NC]` (`NC = M<N>::NC`) holding,
428
+ in order, `c`, the `N` entries of `L`, the entries `Q_ij` with `i ≤ j`, the entries `T_ijk` with `i ≤ j ≤ k` (order 3
429
+ only), and `r`. This is the packed format. The operations modify their arguments in place; `h` holds the box's
430
+ half-widths:
431
+
432
+ | Function | Effect |
433
+ |---|---|
434
+ | `copy<N>(a, b)`, `add<N>(a, b)`, `axpy<N>(a, s, b)` | `a = b`, `a += b`, `a += s·b` |
435
+ | `mul<N>(a, b, h)`, `scale<N>(a, s)`, `neg<N>(a)` | `b = a·b`, `a = s·a`, `a = -a` |
436
+ | `tanh_tm`, `sech2_tm`, `sin_tm`, `cos_tm`, `exp_tm`, `log_tm`, `recip_tm` `<N>(a, h)` | `a = g(a)` |
437
+ | `bounds<N>(a, h, lo, hi)`, `upper<N>(a, h)` | the bounds of `a` on the box |
438
+
439
+ A kernel for `f(x) = x0² + x1² − 0.5·x0·x1` on 2-D boxes:
440
+
441
+ ```cuda
442
+ #include "taylor.cuh"
443
+ __global__ void quad_kernel(const float* x0, const float* hw, float* out, int B) {
444
+ constexpr int N = 2, NC = M<N>::NC;
445
+ int b = blockIdx.x * blockDim.x + threadIdx.x;
446
+ if (b >= B) return;
447
+ float h[N], x[N][NC];
448
+ for (int i = 0; i < N; ++i) { // the box: x_i = x0_i + d_i
449
+ h[i] = hw[b * N + i];
450
+ for (int p = 0; p < NC; ++p) x[i][p] = 0.f;
451
+ x[i][0] = x0[b * N + i]; x[i][1 + i] = 1.f;
452
+ }
453
+ float f[NC], t[NC];
454
+ copy<N>(f, x[0]); mul<N>(x[0], f, h); // f = x0 x0
455
+ copy<N>(t, x[1]); mul<N>(x[1], t, h); add<N>(f, t); // f += x1 x1
456
+ copy<N>(t, x[1]); mul<N>(x[0], t, h); axpy<N>(f, -0.5f, t); // f -= 0.5 x0 x1
457
+ bounds<N>(f, h, out[2 * b], out[2 * b + 1]);
458
+ }
459
+ void run(torch::Tensor x0, torch::Tensor h, torch::Tensor out) {
460
+ int B = x0.size(0);
461
+ quad_kernel<<<(B + 127) / 128, 128>>>(x0.data_ptr<float>(), h.data_ptr<float>(), out.data_ptr<float>(), B);
462
+ C10_CUDA_KERNEL_LAUNCH_CHECK();
463
+ }
464
+ ```
465
+
466
+ To pass your own compile-time constants, add them to the flags: `cuda.flags(settings, NAME=value, ...)` adds
467
+ `-DNAME=value`.
468
+
469
+ </details>
470
+
471
+ ## Neural networks and derivatives
472
+
473
+ ### Network outputs and directional derivatives
474
+
475
+ `network(net, x)` evaluates a supported tanh network `net.s` over the boxes `x`. The network must be an `nn.Sequential`
476
+ of `nn.Linear` layers separated by `nn.Tanh`, with one output. It returns a model `s` of the network output and a
477
+ function `along(v)` modeling `∇s · v` for a Taylor-model vector field `v`:
478
+
479
+ ```python
480
+ def field(p): # a vector field, written for one point: returns n values
481
+ return p[1], -torch.sin(p[0]) - 0.1 * p[1]
482
+
483
+ s, along = network(net, x)
484
+ s_lo, s_hi = s.bounds()
485
+ lie_lo, lie_hi = along(evaluate(field, x)).bounds() # bounds of ∇s(x) · field(x) on each box
486
+ ```
487
+
488
+ This is particularly useful for checking Lie-derivative inequalities in control, but `along` can be used with other
489
+ vector fields as well. For a standalone network example, see [Bound a neural network](#bound-a-neural-network).
490
+
491
+ ### Jacobians
492
+
493
+ `jacobian(f, x)` returns `(lo, hi)`, each of shape `(B, m, n)`, for a function with `n` inputs and `m` outputs. It uses
494
+ forward-mode differentiation. For batched functions, the lower-level `Dual(v, t)` API carries a Taylor model of a value
495
+ and its derivative in direction `t`; `f(Dual(x, t)).t.bounds()` bounds that directional derivative.
496
+
497
+ ## Limitations
498
+
499
+ - **Floating point.** The bounds are computed in float32, and rounding errors are not accounted for, in PyTorch or in
500
+ CUDA. The bounds are exact for real-number arithmetic but can be off by rounding; a proof should leave a small
501
+ relative margin.
502
+ - **Operations.** Only the operations listed above. `abs`, `max`, ReLU, `sqrt` and non-integer powers are not
503
+ supported.
504
+ - **Box size.** Bounds are tight on small boxes. On large boxes, or where the function changes quickly, they are
505
+ loose; `branch_and_bound` handles this by splitting.
506
+ - **Number of inputs.** A model has about `n²/2` coefficients at order 2 and `n³/6` at order 3, for `n` inputs. The
507
+ CUDA kernels keep each box's models in registers or shared memory. We have used them with up to 6 inputs; more
508
+ inputs work but run slower.
509
+ - **GPU.** The CUDA code needs an NVIDIA GPU and `nvcc`. The PyTorch code runs wherever PyTorch runs.
510
+
511
+ ## API reference
512
+
513
+ <details>
514
+ <summary><b>Settings</b></summary>
515
+
516
+ `TaylorModel.box` and `cuda.Box` take a `Settings`. Every model computed from a box uses the settings of that box, and
517
+ each compiled CUDA kernel is specific to one `Settings`.
518
+
519
+ | Field | Default | Meaning |
520
+ |---|---|---|
521
+ | `order` | `2` | degree of the polynomial: 1, 2 or 3. A higher order gives tighter bounds on larger boxes, at a higher cost per box |
522
+ | `cheb` | `False` | for functions such as `sin`, also compute a polynomial that interpolates the function at Chebyshev points, and keep whichever gives the smaller error |
523
+ | `cheb_min_width` | `1e-3` | use the Chebyshev polynomial only when the argument's range is wider than `cheb_min_width × (1 + \|midpoint\|)` |
524
+
525
+ ```python
526
+ x = TaylorModel.box(centres, half_widths, Settings(order=3, cheb=True))
527
+ ```
528
+
529
+ </details>
530
+
531
+ <details>
532
+ <summary><b>Functions and classes</b></summary>
533
+
534
+ | Name | Description |
535
+ |---|---|
536
+ | `TaylorModel.box(centres, half_widths, settings=Settings())` | `B` boxes as models, from two `(B, n)` tensors |
537
+ | `Settings(order=2, cheb=False, cheb_min_width=1e-3)` | how the models are computed |
538
+ | `evaluate(f, x)` | the model of `f` (written for one point) on the boxes `x`: `.bounds()`, `.c`, `.scores()` |
539
+ | `jacobian(f, x)` | bounds of the Jacobian of `f`: `(lo, hi)`, each `(B, m, n)` |
540
+ | `Dual(v, t)` | a model and its derivative in one direction (forward mode) |
541
+ | `network(net, x)` | a tanh network on the boxes: `(s, along)` |
542
+ | `branch_and_bound(L, U, check, batch, h_min, ...)` | prove a property by splitting boxes |
543
+ | `split(L, U, score, ...)` | split boxes into halves along their highest-scoring side |
544
+ | `cuda.Box(centres, half_widths, settings)` | boxes for the CUDA functions |
545
+ | `cuda.compile(f)` | `f` (written for one point) as a CUDA kernel: `bound(box)` returns `(lo, hi)` |
546
+ | `cuda.network(net, box, tangents)` | a tanh network and its derivatives in the given directions |
547
+ | `cuda.dynamics(sys, box)` | packed models of the terms `a` and `b_j` of a control-affine system |
548
+ | `cuda.build(prefix, signature, source, function, flags)` | compile your own kernel that uses `taylor.cuh` |
549
+ | `cuda.flags(settings, **defines)`, `cuda.num_coeffs(n, order)`, `cuda.mlp_weights(net)` | helpers for your own kernels |
550
+
551
+ </details>
552
+
553
+ ## Running the tests
554
+
555
+ The tests need a CUDA GPU (the first run compiles the kernels, which takes several minutes):
556
+
557
+ ```bash
558
+ pip install -e .
559
+ python tests/test_fast_taylor_verifier.py # or: pytest tests
560
+ ```
561
+
562
+ ## License
563
+
564
+ MIT; see [LICENSE](https://github.com/YasinSonmez/fast-taylor-verifier/blob/main/LICENSE).