tinydiffeq 0.1.0__tar.gz → 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/PKG-INFO +66 -13
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/README.md +63 -11
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/docs/adaptive_ad.md +48 -13
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/docs/api.md +8 -0
- tinydiffeq-0.3.0/docs/dae.md +120 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/docs/index.md +31 -17
- tinydiffeq-0.3.0/docs/llms.txt +12 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/docs/sde.md +6 -6
- tinydiffeq-0.3.0/docs/static_shapes.md +124 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/mkdocs.yml +1 -1
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/pyproject.toml +3 -2
- tinydiffeq-0.3.0/src/tinydiffeq/__init__.py +45 -0
- tinydiffeq-0.3.0/src/tinydiffeq/controllers.py +182 -0
- tinydiffeq-0.3.0/src/tinydiffeq/dae.py +453 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/src/tinydiffeq/interpolation.py +7 -6
- tinydiffeq-0.3.0/src/tinydiffeq/ode.py +230 -0
- tinydiffeq-0.1.0/src/tinydiffeq/saveat.py → tinydiffeq-0.3.0/src/tinydiffeq/save_at.py +12 -11
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/src/tinydiffeq/sde.py +35 -29
- tinydiffeq-0.3.0/src/tinydiffeq/solution.py +49 -0
- tinydiffeq-0.3.0/src/tinydiffeq/solvers.py +141 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/tests/kernels_reference.py +56 -45
- tinydiffeq-0.3.0/tests/test_ad.py +238 -0
- tinydiffeq-0.3.0/tests/test_adaptive.py +371 -0
- tinydiffeq-0.3.0/tests/test_dae.py +406 -0
- tinydiffeq-0.3.0/tests/test_float64_subprocess.py +329 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/tests/test_gpu.py +44 -8
- tinydiffeq-0.3.0/tests/test_recompile.py +118 -0
- tinydiffeq-0.3.0/tests/test_save_at.py +139 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/tests/test_sde.py +25 -25
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/tests/test_solvers_fixed.py +58 -30
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/uv.lock +18 -2
- tinydiffeq-0.1.0/docs/llms.txt +0 -12
- tinydiffeq-0.1.0/docs/migration.md +0 -95
- tinydiffeq-0.1.0/docs/static_shapes.md +0 -96
- tinydiffeq-0.1.0/src/tinydiffeq/__init__.py +0 -37
- tinydiffeq-0.1.0/src/tinydiffeq/controllers.py +0 -72
- tinydiffeq-0.1.0/src/tinydiffeq/ode.py +0 -176
- tinydiffeq-0.1.0/src/tinydiffeq/solution.py +0 -28
- tinydiffeq-0.1.0/src/tinydiffeq/solvers.py +0 -129
- tinydiffeq-0.1.0/tests/test_ad.py +0 -205
- tinydiffeq-0.1.0/tests/test_adaptive.py +0 -233
- tinydiffeq-0.1.0/tests/test_float64_subprocess.py +0 -158
- tinydiffeq-0.1.0/tests/test_recompile.py +0 -73
- tinydiffeq-0.1.0/tests/test_saveat.py +0 -86
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/.githooks/pre-commit +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/.github/workflows/ci.yml +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/.github/workflows/docs.yml +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/.github/workflows/publish.yml +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/.gitignore +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/.python-version +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/LICENSE +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/src/tinydiffeq/quadrature.py +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/tests/conftest.py +0 -0
- {tinydiffeq-0.1.0 → tinydiffeq-0.3.0}/tests/test_quadrature.py +0 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: tinydiffeq
|
|
3
|
-
Version: 0.
|
|
4
|
-
Summary: Tiny differentiable ODE/SDE solvers for JAX
|
|
3
|
+
Version: 0.3.0
|
|
4
|
+
Summary: Tiny differentiable ODE/SDE/DAE solvers for JAX with static shapes and composable AD
|
|
5
5
|
Project-URL: Homepage, https://github.com/HighDimensionalEconLab/tinydiffeq
|
|
6
6
|
Project-URL: Repository, https://github.com/HighDimensionalEconLab/tinydiffeq
|
|
7
7
|
Project-URL: Documentation, https://highdimensionaleconlab.github.io/tinydiffeq/
|
|
@@ -18,6 +18,7 @@ Classifier: Programming Language :: Python :: 3.13
|
|
|
18
18
|
Classifier: Topic :: Scientific/Engineering :: Mathematics
|
|
19
19
|
Requires-Python: >=3.11
|
|
20
20
|
Requires-Dist: jax>=0.7.0
|
|
21
|
+
Requires-Dist: nlls-gram>=1.6.0
|
|
21
22
|
Description-Content-Type: text/markdown
|
|
22
23
|
|
|
23
24
|
# tinydiffeq
|
|
@@ -29,20 +30,22 @@ Description-Content-Type: text/markdown
|
|
|
29
30
|
[](https://github.com/HighDimensionalEconLab/tinydiffeq/blob/main/LICENSE)
|
|
30
31
|
[](https://github.com/astral-sh/ruff)
|
|
31
32
|
|
|
32
|
-
Tiny differentiable ODE/SDE solvers for JAX: fixed-step Euler/RK4,
|
|
33
|
-
Tsit5 with
|
|
33
|
+
Tiny differentiable ODE/SDE/DAE solvers for JAX: fixed-step Euler/RK4,
|
|
34
|
+
adaptive Tsit5 with integral or proportional-integral step-size control,
|
|
35
|
+
Euler–Maruyama for Itô SDEs, and nonstiff semi-explicit index-1 DAEs.
|
|
34
36
|
One bounded `lax.scan` of exactly `max_steps` iterations serves fixed and
|
|
35
37
|
adaptive stepping, so shapes are static, nothing recompiles as tolerances or
|
|
36
38
|
curvature change, and every solve is differentiable in **both** forward and
|
|
37
39
|
reverse mode — including reverse-over-forward, the pattern a
|
|
38
40
|
Levenberg–Marquardt optimizer with geodesic acceleration needs when it
|
|
39
|
-
differentiates through a rollout.
|
|
41
|
+
differentiates through a rollout. After a solve reaches its horizon, a
|
|
42
|
+
`lax.cond` skips solver and controller work during the padded scan tail.
|
|
40
43
|
|
|
41
44
|
This is a deliberately small, jvp/vjp-friendly subset of
|
|
42
45
|
[diffrax](https://docs.kidger.site/diffrax/). Use diffrax if you need pytree
|
|
43
|
-
states, stiff
|
|
44
|
-
|
|
45
|
-
`
|
|
46
|
+
states, stiff or fully implicit solvers, higher-index DAEs, full
|
|
47
|
+
derivative-term PID control, events, dense output, or checkpointed/backsolve
|
|
48
|
+
adjoints. The DAE algebraic solve uses `nlls-gram`.
|
|
46
49
|
|
|
47
50
|
## Install
|
|
48
51
|
|
|
@@ -78,22 +81,72 @@ def f(x, t, args, p):
|
|
|
78
81
|
sol = solve_ode(
|
|
79
82
|
f, Tsit5(), 0.0, 2.0, jnp.asarray(1.0),
|
|
80
83
|
p=jnp.asarray(1.3),
|
|
81
|
-
|
|
84
|
+
dt_0=0.1,
|
|
82
85
|
controller=IController(rtol=1e-8, atol=1e-10),
|
|
83
86
|
max_steps=512,
|
|
84
|
-
|
|
87
|
+
save_at=SaveAt(ts=jnp.linspace(0.0, 2.0, 21)), # fixed output shape,
|
|
85
88
|
) # however many steps adapt
|
|
86
89
|
print(sol.xs) # states on the grid
|
|
87
|
-
print(sol.ok) # reached
|
|
90
|
+
print(sol.ok) # reached t_1 within the max_steps budget?
|
|
88
91
|
```
|
|
89
92
|
|
|
93
|
+
`IController()` and `PIController()` choose tolerances from `x_0.dtype`:
|
|
94
|
+
`rtol=1e-4, atol=1e-6` for float32 and `rtol=1e-7, atol=1e-9` for
|
|
95
|
+
float64. Pass explicit values when tolerances are part of your model's
|
|
96
|
+
scientific specification. The default `dt_min` is
|
|
97
|
+
`10 * finfo(dtype).eps * max(1, abs(t_1))`.
|
|
98
|
+
|
|
99
|
+
`max_steps` is the total internal **attempt budget**: accepted steps plus
|
|
100
|
+
rejections. It is not normally the number of returned times. Endpoint mode
|
|
101
|
+
returns one time/state, `SaveAt(ts=...)` returns the requested grid, and
|
|
102
|
+
`SaveAt(steps=True)` returns the initial state and accepted internal steps as
|
|
103
|
+
a contiguous prefix of `max_steps + 1` rows. The remaining rows repeat the
|
|
104
|
+
last accepted state by default; `sol.accepted` distinguishes data from
|
|
105
|
+
padding. Rejected attempts never appear in the returned trajectory.
|
|
106
|
+
|
|
107
|
+
`SaveAt(ts=...)` also accepts a Python sequence. These are observation times:
|
|
108
|
+
the adaptive controller still chooses its own internal mesh, and cubic
|
|
109
|
+
Hermite interpolation evaluates the solution at every requested point.
|
|
110
|
+
|
|
111
|
+
## Semi-explicit DAEs
|
|
112
|
+
|
|
113
|
+
For a square index-1 system `dy/dt = f(y, z, t, args, p)` and
|
|
114
|
+
`0 = g(y, z, t, args, p)`:
|
|
115
|
+
|
|
116
|
+
```python
|
|
117
|
+
from tinydiffeq import IController, Tsit5, solve_semi_explicit_dae
|
|
118
|
+
|
|
119
|
+
|
|
120
|
+
def dae_f(y, z, t, args, p):
|
|
121
|
+
return p * z
|
|
122
|
+
|
|
123
|
+
|
|
124
|
+
def dae_g(y, z):
|
|
125
|
+
return z - y
|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
dae_sol = solve_semi_explicit_dae(
|
|
129
|
+
dae_f, dae_g, Tsit5(), 0.0, 1.0,
|
|
130
|
+
jnp.asarray(1.0), jnp.asarray(0.5),
|
|
131
|
+
p=jnp.asarray(2.0), dt_0=0.1,
|
|
132
|
+
controller=IController(), max_steps=128,
|
|
133
|
+
)
|
|
134
|
+
print(dae_sol.ys, dae_sol.zs)
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
`z_0` is a guess and is made consistent automatically. Algebraic roots use
|
|
138
|
+
an implicitly differentiated square LM solve, so JVP, VJP, and
|
|
139
|
+
reverse-over-forward propagate with respect to `y`, `t`, and `p`. See the
|
|
140
|
+
[DAE documentation](https://highdimensionaleconlab.github.io/tinydiffeq/dae/)
|
|
141
|
+
for root controls, `SaveAt`, and scope limits.
|
|
142
|
+
|
|
90
143
|
## Gradients through the solve
|
|
91
144
|
|
|
92
145
|
```python
|
|
93
146
|
def endpoint(p):
|
|
94
147
|
return solve_ode(
|
|
95
148
|
f, Tsit5(), 0.0, 2.0, jnp.asarray(1.0), p=p,
|
|
96
|
-
|
|
149
|
+
dt_0=0.1, controller=IController(rtol=1e-10, atol=1e-12),
|
|
97
150
|
max_steps=512,
|
|
98
151
|
).xs
|
|
99
152
|
|
|
@@ -109,7 +162,7 @@ non-differentiable either way, and the error-ratio power blows up at exactly
|
|
|
109
162
|
zero error); the states differentiate fully through the RK stages. See the
|
|
110
163
|
[docs](https://highdimensionaleconlab.github.io/tinydiffeq/) for the design
|
|
111
164
|
contracts: static shapes and `SaveAt`, AD through adaptive stepping, SDE key
|
|
112
|
-
semantics, and
|
|
165
|
+
semantics, and the package API.
|
|
113
166
|
|
|
114
167
|
## License
|
|
115
168
|
|
|
@@ -7,20 +7,22 @@
|
|
|
7
7
|
[](https://github.com/HighDimensionalEconLab/tinydiffeq/blob/main/LICENSE)
|
|
8
8
|
[](https://github.com/astral-sh/ruff)
|
|
9
9
|
|
|
10
|
-
Tiny differentiable ODE/SDE solvers for JAX: fixed-step Euler/RK4,
|
|
11
|
-
Tsit5 with
|
|
10
|
+
Tiny differentiable ODE/SDE/DAE solvers for JAX: fixed-step Euler/RK4,
|
|
11
|
+
adaptive Tsit5 with integral or proportional-integral step-size control,
|
|
12
|
+
Euler–Maruyama for Itô SDEs, and nonstiff semi-explicit index-1 DAEs.
|
|
12
13
|
One bounded `lax.scan` of exactly `max_steps` iterations serves fixed and
|
|
13
14
|
adaptive stepping, so shapes are static, nothing recompiles as tolerances or
|
|
14
15
|
curvature change, and every solve is differentiable in **both** forward and
|
|
15
16
|
reverse mode — including reverse-over-forward, the pattern a
|
|
16
17
|
Levenberg–Marquardt optimizer with geodesic acceleration needs when it
|
|
17
|
-
differentiates through a rollout.
|
|
18
|
+
differentiates through a rollout. After a solve reaches its horizon, a
|
|
19
|
+
`lax.cond` skips solver and controller work during the padded scan tail.
|
|
18
20
|
|
|
19
21
|
This is a deliberately small, jvp/vjp-friendly subset of
|
|
20
22
|
[diffrax](https://docs.kidger.site/diffrax/). Use diffrax if you need pytree
|
|
21
|
-
states, stiff
|
|
22
|
-
|
|
23
|
-
`
|
|
23
|
+
states, stiff or fully implicit solvers, higher-index DAEs, full
|
|
24
|
+
derivative-term PID control, events, dense output, or checkpointed/backsolve
|
|
25
|
+
adjoints. The DAE algebraic solve uses `nlls-gram`.
|
|
24
26
|
|
|
25
27
|
## Install
|
|
26
28
|
|
|
@@ -56,22 +58,72 @@ def f(x, t, args, p):
|
|
|
56
58
|
sol = solve_ode(
|
|
57
59
|
f, Tsit5(), 0.0, 2.0, jnp.asarray(1.0),
|
|
58
60
|
p=jnp.asarray(1.3),
|
|
59
|
-
|
|
61
|
+
dt_0=0.1,
|
|
60
62
|
controller=IController(rtol=1e-8, atol=1e-10),
|
|
61
63
|
max_steps=512,
|
|
62
|
-
|
|
64
|
+
save_at=SaveAt(ts=jnp.linspace(0.0, 2.0, 21)), # fixed output shape,
|
|
63
65
|
) # however many steps adapt
|
|
64
66
|
print(sol.xs) # states on the grid
|
|
65
|
-
print(sol.ok) # reached
|
|
67
|
+
print(sol.ok) # reached t_1 within the max_steps budget?
|
|
66
68
|
```
|
|
67
69
|
|
|
70
|
+
`IController()` and `PIController()` choose tolerances from `x_0.dtype`:
|
|
71
|
+
`rtol=1e-4, atol=1e-6` for float32 and `rtol=1e-7, atol=1e-9` for
|
|
72
|
+
float64. Pass explicit values when tolerances are part of your model's
|
|
73
|
+
scientific specification. The default `dt_min` is
|
|
74
|
+
`10 * finfo(dtype).eps * max(1, abs(t_1))`.
|
|
75
|
+
|
|
76
|
+
`max_steps` is the total internal **attempt budget**: accepted steps plus
|
|
77
|
+
rejections. It is not normally the number of returned times. Endpoint mode
|
|
78
|
+
returns one time/state, `SaveAt(ts=...)` returns the requested grid, and
|
|
79
|
+
`SaveAt(steps=True)` returns the initial state and accepted internal steps as
|
|
80
|
+
a contiguous prefix of `max_steps + 1` rows. The remaining rows repeat the
|
|
81
|
+
last accepted state by default; `sol.accepted` distinguishes data from
|
|
82
|
+
padding. Rejected attempts never appear in the returned trajectory.
|
|
83
|
+
|
|
84
|
+
`SaveAt(ts=...)` also accepts a Python sequence. These are observation times:
|
|
85
|
+
the adaptive controller still chooses its own internal mesh, and cubic
|
|
86
|
+
Hermite interpolation evaluates the solution at every requested point.
|
|
87
|
+
|
|
88
|
+
## Semi-explicit DAEs
|
|
89
|
+
|
|
90
|
+
For a square index-1 system `dy/dt = f(y, z, t, args, p)` and
|
|
91
|
+
`0 = g(y, z, t, args, p)`:
|
|
92
|
+
|
|
93
|
+
```python
|
|
94
|
+
from tinydiffeq import IController, Tsit5, solve_semi_explicit_dae
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
def dae_f(y, z, t, args, p):
|
|
98
|
+
return p * z
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
def dae_g(y, z):
|
|
102
|
+
return z - y
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
dae_sol = solve_semi_explicit_dae(
|
|
106
|
+
dae_f, dae_g, Tsit5(), 0.0, 1.0,
|
|
107
|
+
jnp.asarray(1.0), jnp.asarray(0.5),
|
|
108
|
+
p=jnp.asarray(2.0), dt_0=0.1,
|
|
109
|
+
controller=IController(), max_steps=128,
|
|
110
|
+
)
|
|
111
|
+
print(dae_sol.ys, dae_sol.zs)
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
`z_0` is a guess and is made consistent automatically. Algebraic roots use
|
|
115
|
+
an implicitly differentiated square LM solve, so JVP, VJP, and
|
|
116
|
+
reverse-over-forward propagate with respect to `y`, `t`, and `p`. See the
|
|
117
|
+
[DAE documentation](https://highdimensionaleconlab.github.io/tinydiffeq/dae/)
|
|
118
|
+
for root controls, `SaveAt`, and scope limits.
|
|
119
|
+
|
|
68
120
|
## Gradients through the solve
|
|
69
121
|
|
|
70
122
|
```python
|
|
71
123
|
def endpoint(p):
|
|
72
124
|
return solve_ode(
|
|
73
125
|
f, Tsit5(), 0.0, 2.0, jnp.asarray(1.0), p=p,
|
|
74
|
-
|
|
126
|
+
dt_0=0.1, controller=IController(rtol=1e-10, atol=1e-12),
|
|
75
127
|
max_steps=512,
|
|
76
128
|
).xs
|
|
77
129
|
|
|
@@ -87,7 +139,7 @@ non-differentiable either way, and the error-ratio power blows up at exactly
|
|
|
87
139
|
zero error); the states differentiate fully through the RK stages. See the
|
|
88
140
|
[docs](https://highdimensionaleconlab.github.io/tinydiffeq/) for the design
|
|
89
141
|
contracts: static shapes and `SaveAt`, AD through adaptive stepping, SDE key
|
|
90
|
-
semantics, and
|
|
142
|
+
semantics, and the package API.
|
|
91
143
|
|
|
92
144
|
## License
|
|
93
145
|
|
|
@@ -5,10 +5,31 @@ differentiable in forward mode, reverse mode, and reverse-over-forward. That
|
|
|
5
5
|
holds because of three deliberate choices about **what is not
|
|
6
6
|
differentiated**.
|
|
7
7
|
|
|
8
|
+
## Default tolerances follow the state precision
|
|
9
|
+
|
|
10
|
+
`IController()` and `PIController()` resolve omitted tolerances from the
|
|
11
|
+
state dtype at trace time:
|
|
12
|
+
|
|
13
|
+
| State/time dtype | `rtol` | `atol` | unit-scale `dt_min` |
|
|
14
|
+
|---|---:|---:|---:|
|
|
15
|
+
| float32 | `1e-4` | `1e-6` | `10 * eps` ≈ `1.19e-6` |
|
|
16
|
+
| float64 | `1e-7` | `1e-9` | `10 * eps` ≈ `2.22e-15` |
|
|
17
|
+
|
|
18
|
+
The policy never changes `jax_enable_x64`: a float32 `x_0` stays float32 even
|
|
19
|
+
when x64 is enabled globally. Explicit tolerances override the defaults and
|
|
20
|
+
are cast to the state dtype. Use explicit values whenever tolerances are part
|
|
21
|
+
of a reproducibility or accuracy contract.
|
|
22
|
+
|
|
23
|
+
Automatic `dt_min` is `10 * eps * max(1, abs(t_1))`; set it explicitly when
|
|
24
|
+
the relevant time scale differs materially from the absolute horizon. The
|
|
25
|
+
controller also floors exact-zero error ratios at machine epsilon before
|
|
26
|
+
applying a negative power; the growth-factor clip then selects maximal step
|
|
27
|
+
growth without introducing infinities.
|
|
28
|
+
|
|
8
29
|
## The controller is stop-gradiented
|
|
9
30
|
|
|
10
|
-
The step-size
|
|
11
|
-
next-step
|
|
31
|
+
The adaptive step-size controllers (`IController` and `PIController`) compute
|
|
32
|
+
their scaled error norms and next-step factors inside `stop_gradient`:
|
|
12
33
|
|
|
13
34
|
> accept/reject is a non-differentiable branch either way, the gradient of
|
|
14
35
|
> `E**(-1/order)` blows up at the exact-zero error of a flat-start policy,
|
|
@@ -21,13 +42,27 @@ pattern actually taken*. This is the same convention diffrax uses.
|
|
|
21
42
|
|
|
22
43
|
The `E**(-1/5)` blow-up is not hypothetical: a policy initialized flat gives
|
|
23
44
|
an exactly-zero error estimate on the first step, and without the
|
|
24
|
-
`stop_gradient` (plus the
|
|
45
|
+
`stop_gradient` (plus the machine-epsilon error floor) the backward pass is NaN
|
|
25
46
|
from iteration one. `tests/test_ad.py::test_grad_finite_on_flat_field` pins
|
|
26
47
|
this.
|
|
27
48
|
|
|
49
|
+
`PIController` additionally carries the previous accepted error ratio. Its
|
|
50
|
+
step-size factor is
|
|
51
|
+
|
|
52
|
+
```text
|
|
53
|
+
safety * E_n**(-(p_coeff + i_coeff) / order)
|
|
54
|
+
* E_prev**(p_coeff / order)
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
and `E_prev` changes only after acceptance. The whole recurrence is
|
|
58
|
+
controller-internal and stop-gradiented. Setting `p_coeff=0, i_coeff=1`
|
|
59
|
+
reproduces `IController` bit for bit; the default `p_coeff=0.4, i_coeff=0.3`
|
|
60
|
+
is less sensitive to oscillatory error estimates and typically rejects fewer
|
|
61
|
+
attempts on harder problems.
|
|
62
|
+
|
|
28
63
|
## The horizon clip is the growth guard
|
|
29
64
|
|
|
30
|
-
Every attempt is clipped so it cannot step past `
|
|
65
|
+
Every attempt is clipped so it cannot step past `t_1`, and — deliberately
|
|
31
66
|
unlike diffrax — the controller's next-step proposal is computed from the
|
|
32
67
|
**clipped** step:
|
|
33
68
|
|
|
@@ -36,15 +71,15 @@ unlike diffrax — the controller's next-step proposal is computed from the
|
|
|
36
71
|
> whose Gauss–Newton linearization stalls a trust-region optimizer
|
|
37
72
|
> differentiating through the rollout.
|
|
38
73
|
|
|
39
|
-
With `
|
|
74
|
+
With `factor_max = 5`, a flat field would otherwise reach `dt ≈ t_1/4` within
|
|
40
75
|
a few accepted steps; residuals sampled from three or four giant steps make
|
|
41
76
|
the optimizer's linear model useless. Clipping first means the proposal can
|
|
42
|
-
never exceed `
|
|
77
|
+
never exceed `factor_max × remaining horizon`.
|
|
43
78
|
|
|
44
79
|
One refinement over a bare `min(dt, remaining)`: when the remaining horizon
|
|
45
80
|
is within `max_steps × eps` of the desired step, the step is stretched to
|
|
46
|
-
land on `
|
|
47
|
-
`t` one accumulated ulp short of `
|
|
81
|
+
land on `t_1` exactly. Summing `n` rounded steps of `(t_1 - t_0)/n` can leave
|
|
82
|
+
`t` one accumulated ulp short of `t_1`, and without the stretch that sliver
|
|
48
83
|
would cost an extra iteration a `max_steps = n` budget doesn't have.
|
|
49
84
|
|
|
50
85
|
## Interpolation knots are non-differentiable
|
|
@@ -74,15 +109,15 @@ on the output is not enough — reverse mode differentiates both branches, and
|
|
|
74
109
|
Levenberg–Marquardt geodesic-acceleration pattern) all work through
|
|
75
110
|
adaptive solves and interpolated output, verified against closed forms in
|
|
76
111
|
`tests/test_ad.py`.
|
|
77
|
-
- `jax.vmap` over `
|
|
112
|
+
- `jax.vmap` over `x_0` or `p` gives genuinely per-lane adaptivity: each lane
|
|
78
113
|
accepts/rejects independently through the masked scan, and batched results
|
|
79
114
|
equal the individual solves exactly.
|
|
80
115
|
|
|
81
116
|
## What to watch
|
|
82
117
|
|
|
83
|
-
- Reductions over `SaveAt(steps=True)`
|
|
84
|
-
|
|
85
|
-
reject.
|
|
86
|
-
|
|
118
|
+
- Reductions over the valid prefix of `SaveAt(steps=True)` are
|
|
119
|
+
**discontinuous** in the inputs: its length changes when an accept flips to
|
|
120
|
+
a reject. Use `sol.accepted` when padding must not contribute. Because the
|
|
121
|
+
default tail repeats the endpoint, `xs[-1]` remains the reached final state.
|
|
87
122
|
- Finite-difference checks of adaptive solves are noisy for the same reason;
|
|
88
123
|
compare AD against closed forms or use fixed-step solvers for FD tests.
|
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
|
|
5
5
|
::: tinydiffeq.solve_ode
|
|
6
6
|
|
|
7
|
+
::: tinydiffeq.solve_semi_explicit_dae
|
|
8
|
+
|
|
7
9
|
::: tinydiffeq.solve_sde
|
|
8
10
|
|
|
9
11
|
## Solvers
|
|
@@ -22,10 +24,16 @@
|
|
|
22
24
|
|
|
23
25
|
::: tinydiffeq.IController
|
|
24
26
|
|
|
27
|
+
::: tinydiffeq.PIController
|
|
28
|
+
|
|
25
29
|
## Output selection and results
|
|
26
30
|
|
|
27
31
|
::: tinydiffeq.SaveAt
|
|
28
32
|
|
|
33
|
+
::: tinydiffeq.LMRootSolver
|
|
34
|
+
|
|
35
|
+
::: tinydiffeq.DAESolution
|
|
36
|
+
|
|
29
37
|
::: tinydiffeq.Solution
|
|
30
38
|
|
|
31
39
|
## Utilities
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
# Semi-Explicit Index-1 DAEs
|
|
2
|
+
|
|
3
|
+
`solve_semi_explicit_dae` integrates nonstiff systems of the form
|
|
4
|
+
|
|
5
|
+
$$
|
|
6
|
+
\dot y = f(y, z, t, \mathrm{args}, p), \qquad
|
|
7
|
+
0 = g(y, z, t, \mathrm{args}, p),
|
|
8
|
+
$$
|
|
9
|
+
|
|
10
|
+
where the algebraic equation is square and $g_z$ is nonsingular along the
|
|
11
|
+
solution. `y` and `z` are array states. The implementation supports fixed-step
|
|
12
|
+
RK4 and Tsit5 with fixed or adaptive control.
|
|
13
|
+
|
|
14
|
+
The algebraic solve uses
|
|
15
|
+
[`nlls-gram`](https://highdimensionaleconlab.github.io/nlls_gram/square_systems/)'s
|
|
16
|
+
solve-only `SquareLevenbergMarquardt`: an augmented-QR damped step for the
|
|
17
|
+
primal root and a direct implicit Jacobian solve for derivatives.
|
|
18
|
+
|
|
19
|
+
## Minimal examples
|
|
20
|
+
|
|
21
|
+
Consider
|
|
22
|
+
|
|
23
|
+
$$
|
|
24
|
+
\dot y = pz, \qquad 0=z-y,
|
|
25
|
+
$$
|
|
26
|
+
|
|
27
|
+
whose reduced solution is $y(t)=z(t)=y_0e^{pt}$.
|
|
28
|
+
|
|
29
|
+
```python
|
|
30
|
+
import jax.numpy as jnp
|
|
31
|
+
|
|
32
|
+
from tinydiffeq import (
|
|
33
|
+
IController,
|
|
34
|
+
RK4,
|
|
35
|
+
Tsit5,
|
|
36
|
+
solve_semi_explicit_dae,
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def f(y, z, t, args, p):
|
|
41
|
+
return p * z
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
def g(y, z):
|
|
45
|
+
return z - y
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
fixed = solve_semi_explicit_dae(
|
|
49
|
+
f, g, RK4(), 0.0, 1.0,
|
|
50
|
+
jnp.asarray(1.0), jnp.asarray(0.5),
|
|
51
|
+
p=jnp.asarray(2.0), dt_0=0.01, max_steps=100,
|
|
52
|
+
)
|
|
53
|
+
|
|
54
|
+
adaptive = solve_semi_explicit_dae(
|
|
55
|
+
f, g, Tsit5(), 0.0, 1.0,
|
|
56
|
+
jnp.asarray(1.0), jnp.asarray(0.5),
|
|
57
|
+
p=jnp.asarray(2.0), dt_0=0.1,
|
|
58
|
+
controller=IController(), max_steps=128,
|
|
59
|
+
)
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
`z_0` is a root-finding guess, not an assumed-consistent initial value. Both
|
|
63
|
+
calls first solve `g(y_0, z, t_0, args, p) = 0`, so the `0.5` guess becomes
|
|
64
|
+
the consistent value `1.0`.
|
|
65
|
+
|
|
66
|
+
## Nonlinear-solve and AD contract
|
|
67
|
+
|
|
68
|
+
The default root configuration is:
|
|
69
|
+
|
|
70
|
+
```python
|
|
71
|
+
from tinydiffeq import LMRootSolver
|
|
72
|
+
|
|
73
|
+
root_solver = LMRootSolver(
|
|
74
|
+
max_steps=8,
|
|
75
|
+
atol=None, # 1e-6 float32, 1e-10 float64
|
|
76
|
+
init_damping=1e-3,
|
|
77
|
+
)
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The outer `max_steps` counts attempted time steps, including adaptive
|
|
81
|
+
rejections. `root_solver.max_steps` separately bounds one algebraic root.
|
|
82
|
+
|
|
83
|
+
Every root passes `(y, t, p)` through nlls-gram's differentiated parameter
|
|
84
|
+
pytree. Thus it differentiates the defining constraint,
|
|
85
|
+
|
|
86
|
+
$$
|
|
87
|
+
\dot z = -g_z^{-1}
|
|
88
|
+
(g_y\dot y + g_t\dot t + g_p\dot p),
|
|
89
|
+
$$
|
|
90
|
+
|
|
91
|
+
rather than differentiating the LM iterations. The warm-start guess has zero
|
|
92
|
+
derivative by design. `args` is fixed data; put every differentiated model
|
|
93
|
+
quantity in `p`. JVP, VJP, `vmap`, and reverse-over-forward compose through
|
|
94
|
+
the complete DAE solve.
|
|
95
|
+
|
|
96
|
+
An adaptive stage-root failure rejects the time-step attempt and asks the
|
|
97
|
+
controller for a smaller step. A fixed-step root failure terminates the solve.
|
|
98
|
+
In either case `sol.ok` is false if the endpoint is not reached with valid
|
|
99
|
+
algebraic states.
|
|
100
|
+
|
|
101
|
+
## Saving output
|
|
102
|
+
|
|
103
|
+
All `SaveAt` modes are supported:
|
|
104
|
+
|
|
105
|
+
- `SaveAt(t_1=True)` returns the consistent endpoint.
|
|
106
|
+
- `SaveAt(steps=True)` returns the initial point and accepted internal steps
|
|
107
|
+
as a padded `max_steps + 1` buffer with the usual `accepted` mask.
|
|
108
|
+
- `SaveAt(ts=grid)` Hermite-interpolates `y` onto the requested grid and then
|
|
109
|
+
solves `g=0` at each requested time. It never interpolates `z` independently,
|
|
110
|
+
so every returned algebraic state satisfies the constraint.
|
|
111
|
+
|
|
112
|
+
The result is `DAESolution(ts, ys, zs, ok, num_accepted, accepted)`.
|
|
113
|
+
|
|
114
|
+
## Deliberate limits
|
|
115
|
+
|
|
116
|
+
This is an explicit, nonstiff, semi-explicit index-1 integrator. It does not
|
|
117
|
+
support mass-matrix or fully implicit DAEs, stiff implicit time stepping,
|
|
118
|
+
higher-index constraints, or automatic index reduction. It is an initial-value
|
|
119
|
+
solver: it does not determine unknown initial costates or solve boundary-value
|
|
120
|
+
or saddle-path conditions.
|
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
# tinydiffeq
|
|
2
2
|
|
|
3
|
-
`tinydiffeq` is a deliberately tiny set of differentiable ODE/SDE
|
|
4
|
-
for JAX: fixed-step Euler and RK4, adaptive Tsit5 with
|
|
5
|
-
|
|
3
|
+
`tinydiffeq` is a deliberately tiny set of differentiable ODE/SDE/DAE
|
|
4
|
+
integrators for JAX: fixed-step Euler and RK4, adaptive Tsit5 with integral or
|
|
5
|
+
proportional-integral step-size control, fixed-step Euler–Maruyama for Itô
|
|
6
|
+
SDEs, and nonstiff semi-explicit index-1 DAEs. Time integration runs
|
|
6
7
|
inside one bounded `lax.scan` with static shapes, and every solve is
|
|
7
8
|
differentiable in **both** forward and reverse mode — including
|
|
8
9
|
reverse-over-forward, the pattern a Levenberg–Marquardt optimizer with
|
|
@@ -13,8 +14,8 @@ It is a jvp/vjp-friendly subset of
|
|
|
13
14
|
need any of:**
|
|
14
15
|
|
|
15
16
|
- pytree states (tinydiffeq states are single arrays, scalar or vector)
|
|
16
|
-
- stiff or implicit solvers
|
|
17
|
-
- PID step-size control
|
|
17
|
+
- stiff or fully implicit solvers, or higher-index DAEs
|
|
18
|
+
- full derivative-term PID step-size control
|
|
18
19
|
- events, root-finding, or backward-time integration
|
|
19
20
|
- dense output / continuous interpolation objects
|
|
20
21
|
- checkpointed or backsolve adjoints for long horizons
|
|
@@ -54,12 +55,15 @@ f(x, t, args, p)
|
|
|
54
55
|
nothing stops you differentiating with respect to it, but the library's
|
|
55
56
|
contracts and tests treat it as constants.
|
|
56
57
|
- `p` holds differentiable parameters — any pytree, e.g. neural-network
|
|
57
|
-
weights. jvp/vjp with respect to `p` and `
|
|
58
|
+
weights. jvp/vjp with respect to `p` and `x_0` are first-class and tested.
|
|
58
59
|
|
|
59
60
|
The arity is inspected once and the function is wrapped into the canonical
|
|
60
61
|
four-argument form, so the compiled code is identical for all four. There is
|
|
61
62
|
no special autonomous code path: an unused `t` is dead-code-eliminated.
|
|
62
63
|
`drift` and `diffusion` in [`solve_sde`](sde.md) follow the same convention.
|
|
64
|
+
Semi-explicit DAE fields use `(y, z)`, `(y, z, t)`,
|
|
65
|
+
`(y, z, t, args)`, or `(y, z, t, args, p)`; see
|
|
66
|
+
[Semi-Explicit DAEs](dae.md).
|
|
63
67
|
|
|
64
68
|
## Minimal example
|
|
65
69
|
|
|
@@ -78,22 +82,28 @@ def f(x, t, args, p):
|
|
|
78
82
|
sol = solve_ode(
|
|
79
83
|
f, Tsit5(), 0.0, 2.0, jnp.asarray(1.0),
|
|
80
84
|
p=jnp.asarray(1.3),
|
|
81
|
-
|
|
85
|
+
dt_0=0.1,
|
|
82
86
|
controller=IController(rtol=1e-8, atol=1e-10),
|
|
83
87
|
max_steps=512,
|
|
84
|
-
|
|
88
|
+
save_at=SaveAt(ts=jnp.linspace(0.0, 2.0, 21)),
|
|
85
89
|
)
|
|
86
90
|
sol.xs # (21,) states on the grid, however many internal steps were taken
|
|
87
|
-
sol.ok # False if max_steps ran out before
|
|
91
|
+
sol.ok # False if max_steps ran out before t_1
|
|
88
92
|
```
|
|
89
93
|
|
|
94
|
+
With no arguments, `IController()` and `PIController()` use precision-aware
|
|
95
|
+
tolerances: `rtol=1e-4, atol=1e-6` for float32 states and
|
|
96
|
+
`rtol=1e-7, atol=1e-9` for float64 states. Explicit values override the
|
|
97
|
+
policy and are cast to the state dtype. Automatic `dt_min` is
|
|
98
|
+
`10 * eps * max(1, abs(t_1))` in the time dtype.
|
|
99
|
+
|
|
90
100
|
Gradients go straight through the solve:
|
|
91
101
|
|
|
92
102
|
```python
|
|
93
103
|
def endpoint(p):
|
|
94
104
|
return solve_ode(
|
|
95
105
|
f, Tsit5(), 0.0, 2.0, jnp.asarray(1.0), p=p,
|
|
96
|
-
|
|
106
|
+
dt_0=0.1, controller=IController(rtol=1e-10, atol=1e-12),
|
|
97
107
|
max_steps=512,
|
|
98
108
|
).xs
|
|
99
109
|
|
|
@@ -106,20 +116,24 @@ jax.grad(lambda p: jax.jvp(endpoint, (p,), (jnp.asarray(1.0),))[1])(
|
|
|
106
116
|
|
|
107
117
|
## Design contracts at a glance
|
|
108
118
|
|
|
109
|
-
- **`
|
|
110
|
-
-
|
|
111
|
-
|
|
119
|
+
- **`dt_0` is required.** There is no initial-step heuristic.
|
|
120
|
+
- **`max_steps` counts attempted internal steps**, including rejections. It
|
|
121
|
+
controls the bounded scan and only becomes an output-row count in
|
|
122
|
+
`SaveAt(steps=True)`, which returns `max_steps + 1` padded rows including
|
|
123
|
+
the initial state. Accepted steps form a contiguous prefix; rejected
|
|
124
|
+
attempts are omitted and the tail repeats the last accepted state.
|
|
125
|
+
- **Forward time only**: `t_1 > t_0`.
|
|
126
|
+
- **Never poisons.** `sol.ok` reports whether `t_1` was reached; callers that
|
|
112
127
|
want diverging residuals do `jnp.where(sol.ok, sol.xs, jnp.inf)`.
|
|
113
128
|
- **`project`** (an idempotent clamp, e.g. positivity) is applied at every
|
|
114
129
|
point where the vector field is evaluated and to every accepted state.
|
|
115
130
|
- **Never sets `jax_enable_x64`.** The time dtype follows
|
|
116
|
-
`jnp.result_type(
|
|
131
|
+
`jnp.result_type(x_0, float)`; float32 problems stay float32 even under
|
|
117
132
|
x64.
|
|
118
133
|
- Solvers, controllers, `SaveAt`, and `Solution` are frozen dataclasses
|
|
119
|
-
registered as pytrees: numeric fields (tolerances, grids, `
|
|
134
|
+
registered as pytrees: numeric fields (tolerances, grids, `dt_0`, `x_0`) are
|
|
120
135
|
data leaves, so changing them never recompiles.
|
|
121
136
|
|
|
122
137
|
Read next: [Static Shapes](static_shapes.md) for the bounded-scan design and
|
|
123
138
|
`SaveAt`, [Adaptive Stepping and AD](adaptive_ad.md) for what is and is not
|
|
124
|
-
differentiated, [SDEs](sde.md), and
|
|
125
|
-
[Migration](migration.md) if you are replacing hand-rolled RK4/Tsit5 loops.
|
|
139
|
+
differentiated, [SDEs](sde.md), and the [API Reference](api.md).
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# tinydiffeq
|
|
2
|
+
|
|
3
|
+
> Tiny differentiable ODE/SDE/DAE solvers for JAX. Fixed-step (Euler, RK4) and adaptive (Tsit5 + integral or proportional-integral controller) explicit Runge-Kutta inside one bounded lax.scan of exactly max_steps attempted steps: static shapes, one compilation across explicit controller values/curvature/initial conditions, and solves differentiable in both forward and reverse mode (including reverse-over-forward) with O(max_steps) memory. Nonstiff semi-explicit index-1 DAEs use RK4 or Tsit5 plus nlls-gram's square LM algebraic solve, whose custom JVP implicitly differentiates each root with respect to the differential state, time, and parameters. Adaptive-controller defaults are precision-aware: rtol=1e-4, atol=1e-6 for float32 and rtol=1e-7, atol=1e-9 for float64; dt_min is 10*eps*max(1, abs(t_1)) in the time dtype. Expensive solver work is skipped after the horizon via lax.cond while the scan emits its padded tail. Array states only (scalar or vector). SaveAt selects endpoint, cubic-Hermite interpolation onto a requested grid, or a compact accepted-step prefix padded to max_steps + 1. For DAE observation grids, only the differential state is interpolated and the algebraic constraint is re-solved. solve_sde is fixed-step Euler-Maruyama with presampled diagonal noise keyed on a PRNG key. A deliberately small subset of diffrax: use diffrax for pytree states, stiff or fully implicit solvers, higher-index DAEs, full derivative-term PID control, events, dense output, and adjoint methods.
|
|
4
|
+
|
|
5
|
+
## Docs
|
|
6
|
+
|
|
7
|
+
- [Home — positioning, vector-field signature convention f(x, t, args, p), minimal examples](https://highdimensionaleconlab.github.io/tinydiffeq/)
|
|
8
|
+
- [Static shapes — the bounded-scan design, SaveAt observation grids, compact accepted steps and padding, why nothing recompiles](https://highdimensionaleconlab.github.io/tinydiffeq/static_shapes/)
|
|
9
|
+
- [Adaptive stepping and AD — stop-gradiented controller rationale, horizon-clip growth guard, non-differentiable interpolation knots, double-where NaN safety](https://highdimensionaleconlab.github.io/tinydiffeq/adaptive_ad/)
|
|
10
|
+
- [Semi-explicit DAEs — index-1 contract, algebraic LM roots, implicit AD, failure behavior, and SaveAt](https://highdimensionaleconlab.github.io/tinydiffeq/dae/)
|
|
11
|
+
- [SDEs — Euler-Maruyama orders, fixed-noise key semantics, shared-path strong-convergence testing, why SaveAt(ts) raises](https://highdimensionaleconlab.github.io/tinydiffeq/sde/)
|
|
12
|
+
- [API reference — ODE/SDE/DAE solves, solvers, controllers, root configuration, SaveAt, solution types, interpolation, and quadrature](https://highdimensionaleconlab.github.io/tinydiffeq/api/)
|
|
@@ -14,11 +14,11 @@ increments.
|
|
|
14
14
|
from tinydiffeq import solve_sde, EulerMaruyama, SaveAt
|
|
15
15
|
|
|
16
16
|
sol = solve_sde(
|
|
17
|
-
drift, diffusion, EulerMaruyama(), 0.0, 1.0,
|
|
17
|
+
drift, diffusion, EulerMaruyama(), 0.0, 1.0, x_0,
|
|
18
18
|
key=jax.random.PRNGKey(0),
|
|
19
19
|
n_steps=256,
|
|
20
20
|
p=(mu, sigma),
|
|
21
|
-
|
|
21
|
+
save_at=SaveAt(steps=True),
|
|
22
22
|
)
|
|
23
23
|
```
|
|
24
24
|
|
|
@@ -38,13 +38,13 @@ for now; a roadmap issue sketches what it would take here.
|
|
|
38
38
|
The Brownian increments are presampled once,
|
|
39
39
|
|
|
40
40
|
```python
|
|
41
|
-
|
|
41
|
+
d_w = jnp.sqrt(dt) * jax.random.normal(key, (n_steps,) + x_0.shape)
|
|
42
42
|
```
|
|
43
43
|
|
|
44
44
|
so a fixed `key` pins the entire noise path:
|
|
45
45
|
|
|
46
46
|
- **Reproducible** — the same key gives the same path, every call.
|
|
47
|
-
- **Differentiable with respect to `
|
|
47
|
+
- **Differentiable with respect to `x_0` and `p`** (not `key`): with the path
|
|
48
48
|
held fixed, the solution map is smooth, and jvp/vjp against finite
|
|
49
49
|
differences are tested in `tests/test_sde.py`. This is exactly the "common
|
|
50
50
|
random numbers" setup simulation-based estimators want.
|
|
@@ -61,7 +61,7 @@ X_T = X_0 \exp\left((\mu - \tfrac{\sigma^2}{2})T + \sigma W_T\right),
|
|
|
61
61
|
\qquad W_T = \textstyle\sum_k \Delta W_k .
|
|
62
62
|
$$
|
|
63
63
|
|
|
64
|
-
Because the increments `
|
|
64
|
+
Because the increments `d_w` are exactly reproducible from the key, the test
|
|
65
65
|
regenerates them, computes the exact endpoint on the same path, and checks
|
|
66
66
|
the mean absolute error slope across `dt` levels lands in `[0.35, 0.65]`.
|
|
67
67
|
That shared-path construction is the right way to measure strong error —
|
|
@@ -69,7 +69,7 @@ comparing against an independently sampled exact solution measures nothing.
|
|
|
69
69
|
|
|
70
70
|
## SaveAt for SDEs
|
|
71
71
|
|
|
72
|
-
`SaveAt(
|
|
72
|
+
`SaveAt(t_1=True)` (default) and `SaveAt(steps=True)` (`n_steps + 1` rows,
|
|
73
73
|
all accepted) are supported. `SaveAt(ts=...)` **raises**: cubic Hermite
|
|
74
74
|
interpolation assumes smooth trajectories and is simply wrong between the
|
|
75
75
|
points of a rough path. Land your grid on the step boundaries instead by
|