solid-objects 1.0.5 → 1.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +49 -0
- package/README.md +7 -0
- package/dist/actor-runtime.d.ts +3 -0
- package/dist/actor-runtime.d.ts.map +1 -1
- package/dist/database/sqlite.d.ts +6 -0
- package/dist/database/sqlite.d.ts.map +1 -1
- package/dist/database/sqlite.js +23 -2
- package/dist/database/sqlite.js.map +1 -1
- package/dist/runtime.d.ts +2 -7
- package/dist/runtime.d.ts.map +1 -1
- package/dist/runtime.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/agents.md +10 -3
- package/docs/api.md +61 -13
- package/docs/benchmarks.md +208 -69
- package/docs/compare/ai-agents.md +252 -0
- package/docs/compare/dbos.md +12 -0
- package/docs/compare/graphile-worker.md +11 -0
- package/docs/compare/restate.md +12 -1
- package/docs/comparisons.md +1 -0
- package/docs/configuration.md +22 -3
- package/docs/correctness.md +7 -7
- package/docs/parity.md +7 -2
- package/docs/virtual-actors.md +2 -2
- package/package.json +5 -4
package/docs/benchmarks.md
CHANGED
|
@@ -1,23 +1,170 @@
|
|
|
1
1
|
# Benchmarks
|
|
2
2
|
|
|
3
|
-
The benchmark harness measures committed actor operations. It shows tradeoffs
|
|
4
|
-
|
|
3
|
+
The benchmark harness measures committed actor operations. It shows tradeoffs and catches large
|
|
4
|
+
regressions. It does not predict application capacity.
|
|
5
|
+
|
|
6
|
+
## Version 1.0.5 measurements
|
|
7
|
+
|
|
8
|
+
The current dataset runs the released `v1.0.5` source at commit
|
|
9
|
+
`8975f6e2ad94c114f3638cea8f167956061a9bc3`. The runtime source is unchanged; the measurement harness
|
|
10
|
+
is overlaid from this repository. Every JSON report records that source revision, runtime source and
|
|
11
|
+
built-artifact SHA-256 values, harness SHA-256, the loaded package version, actual driver versions,
|
|
12
|
+
database version, host, and Node version. At collection time, the development branch also reported
|
|
13
|
+
1.0.5 but contained SQLite WAL changes absent from the release tag. Use the recorded source and
|
|
14
|
+
artifact hashes to reproduce the measured 1.0.5 implementation. Version 1.0.6 includes the later
|
|
15
|
+
SQLite WAL changes.
|
|
16
|
+
|
|
17
|
+
The archived reports were collected with harness revision
|
|
18
|
+
`da5fe31b1ffa18eca92ce43fc26bca619d424d6e`. They verified committed operation
|
|
19
|
+
counts. The current collector additionally verifies cart quantities, totals, and
|
|
20
|
+
one pending effect with the correct total for every committed operation, after
|
|
21
|
+
the timed interval. Those additional assertions were not run retrospectively on
|
|
22
|
+
the archived measurements.
|
|
23
|
+
|
|
24
|
+
### Recorded environment and results
|
|
25
|
+
|
|
26
|
+
Measured November 7, 2026 on Apple M2 Ultra, 24 logical CPUs, 64 GiB RAM, darwin 27.2.0, and Node.js
|
|
27
|
+
v24.18.0. Drivers were pg 8.23.0 and mysql2 3.23.3.
|
|
28
|
+
|
|
29
|
+
The workstation was concurrently used for other development tasks. CPU and storage activity were
|
|
30
|
+
shared. Absolute timings and workload comparisons require controlled repeats before capacity
|
|
31
|
+
planning. For example, the three SQLite four-process synchronous hot-counter samples were 17.89,
|
|
32
|
+
20.69, 79.42 operations/second. Every sample is retained.
|
|
33
|
+
|
|
34
|
+
[Complete tables and raw reports](../benchmarks/results/v1.0.5-2026-11-07/README.md)
|
|
35
|
+
include all 216 matrix rows, 12 state-size rows, three idle rows, and 57,600 individual operation
|
|
36
|
+
latencies.
|
|
37
|
+
|
|
38
|
+
| Database | Version | Durability settings |
|
|
39
|
+
| ---------- | ---------------- | ------------------------------------------- |
|
|
40
|
+
| sqlite | 3.53.1 | `journalMode=delete, synchronous=2` |
|
|
41
|
+
| postgresql | 16.15 (Homebrew) | `fsync=on, synchronousCommit=on` |
|
|
42
|
+
| mysql | 26.10.0 | `innodbFlushLogAtTrxCommit=1, syncBinlog=1` |
|
|
43
|
+
|
|
44
|
+
The next table shows synchronous handlers with four worker processes. One-process and
|
|
45
|
+
yielding-handler results are in the complete tables.
|
|
46
|
+
|
|
47
|
+
| Database | Workload | Shape | ops/s | p50 ms | p95 ms | p99 ms |
|
|
48
|
+
| ---------- | ----------- | --------- | -----: | ------: | -------: | -------: |
|
|
49
|
+
| sqlite | counter | warm-hot | 20.69 | 733.52 | 1,595.35 | 1,754.31 |
|
|
50
|
+
| sqlite | counter | warm-many | 13.51 | 818.09 | 3,205.47 | 6,056.1 |
|
|
51
|
+
| sqlite | counter | cold-many | 5.77 | 2,497.5 | 5,538.34 | 6,346.58 |
|
|
52
|
+
| sqlite | cart-outbox | warm-hot | 106.27 | 147.92 | 205.03 | 283.86 |
|
|
53
|
+
| sqlite | cart-outbox | warm-many | 56.87 | 264.62 | 454.1 | 588.05 |
|
|
54
|
+
| sqlite | cart-outbox | cold-many | 35.4 | 393.17 | 1,037.9 | 1,286.45 |
|
|
55
|
+
| postgresql | counter | warm-hot | 88.84 | 160.37 | 275.92 | 293.19 |
|
|
56
|
+
| postgresql | counter | warm-many | 146.88 | 70.92 | 176.45 | 1,103.97 |
|
|
57
|
+
| postgresql | counter | cold-many | 20.25 | 808.09 | 1,069.78 | 1,140.48 |
|
|
58
|
+
| postgresql | cart-outbox | warm-hot | 99.15 | 159.81 | 227.97 | 244.67 |
|
|
59
|
+
| postgresql | cart-outbox | warm-many | 107.12 | 93.49 | 253.75 | 1,307.5 |
|
|
60
|
+
| postgresql | cart-outbox | cold-many | 21.56 | 651.8 | 979.4 | 1,056.95 |
|
|
61
|
+
| mysql | counter | warm-hot | 72.5 | 200.22 | 328.44 | 390.64 |
|
|
62
|
+
| mysql | counter | warm-many | 105.16 | 58.47 | 363.58 | 1,728.53 |
|
|
63
|
+
| mysql | counter | cold-many | 31.69 | 445.1 | 935.43 | 1,060.47 |
|
|
64
|
+
| mysql | cart-outbox | warm-hot | 52.96 | 287.93 | 408.07 | 446.53 |
|
|
65
|
+
| mysql | cart-outbox | warm-many | 74.75 | 134.94 | 549.13 | 1,926.81 |
|
|
66
|
+
| mysql | cart-outbox | cold-many | 20.69 | 720.19 | 1,118.29 | 1,176.44 |
|
|
67
|
+
|
|
68
|
+
State-size results use sequential operations on one actor in a SQLite file:
|
|
69
|
+
|
|
70
|
+
| Payload KiB | Persisted state bytes | ops/s | p50 ms | p95 ms | p99 ms |
|
|
71
|
+
| ----------: | --------------------: | ----: | -----: | -----: | -----: |
|
|
72
|
+
| 0 | 26 | 93.6 | 9.29 | 21.09 | 26.88 |
|
|
73
|
+
| 16 | 16,410 | 86.65 | 10.4 | 21.12 | 26.33 |
|
|
74
|
+
| 128 | 131,098 | 70.4 | 12.86 | 22.93 | 27.15 |
|
|
75
|
+
| 1,024 | 1,048,602 | 42.27 | 22.55 | 33.05 | 42.64 |
|
|
76
|
+
|
|
77
|
+
Idle CPU uses one timed sample per interval, in-memory SQLite, and all four polling roles:
|
|
78
|
+
|
|
79
|
+
| Fast interval ms | Empty polls/s | Process CPU |
|
|
80
|
+
| ---------------: | ------------: | ----------: |
|
|
81
|
+
| 20 | 4 | 0.12% |
|
|
82
|
+
| 100 | 4 | 0.12% |
|
|
83
|
+
| 500 | 4 | 0.1% |
|
|
84
|
+
|
|
85
|
+
### Workloads and measurement boundary
|
|
86
|
+
|
|
87
|
+
The counter matrix includes one hot identity, 100 warm identities, and newly created identities,
|
|
88
|
+
each with synchronous and yielding handlers and one or four processes. A second matrix uses a cart
|
|
89
|
+
with 20 line items. Every cart operation updates a quantity, recalculates the total, and commits one
|
|
90
|
+
outbox effect with the actor state. Effect workers are disabled: the cart results measure durable
|
|
91
|
+
outbox staging, not delivery to an external service. The yielding handler uses `setImmediate()`; it
|
|
92
|
+
does not model external network latency.
|
|
93
|
+
|
|
94
|
+
Each scenario has three separate samples of 250 measured operations after 25 warmup operations, at
|
|
95
|
+
client concurrency 16 and four actor workers. Counters are verified against all submitted operations
|
|
96
|
+
after the timed batch. Tables report the median of each statistic across the three samples.
|
|
97
|
+
Percentiles use nearest rank within each sample; samples are not pooled. The raw JSON retains all
|
|
98
|
+
operation latencies, including slow samples. This short run is a development measurement, not a soak
|
|
99
|
+
test or an application capacity guarantee.
|
|
100
|
+
|
|
101
|
+
The state-size run has three samples of 300 operations after 25 warmups, at concurrency one. It
|
|
102
|
+
verifies final committed count and payload size, records actual serialized state bytes, and retains
|
|
103
|
+
operation latencies. Idle CPU uses three seconds of warmup and ten seconds of measurement per
|
|
104
|
+
polling interval. The historical idle wake-up latency table below has not been rerun as part of this
|
|
105
|
+
dataset.
|
|
106
|
+
|
|
107
|
+
### Reproduce the released runtime
|
|
108
|
+
|
|
109
|
+
Use a separate checkout so later runtime changes cannot enter the result. Install the locked
|
|
110
|
+
dependencies and overlay the measurement harness from the commit containing this dataset before
|
|
111
|
+
building the runtime:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
# Run from your normal checkout; pin the collector used by the archived reports.
|
|
115
|
+
repository_root=$(pwd)
|
|
116
|
+
harness_ref=da5fe31b1ffa18eca92ce43fc26bca619d424d6e
|
|
117
|
+
# To collect new results with the stronger verifier instead:
|
|
118
|
+
# harness_ref=$(git rev-parse HEAD)
|
|
119
|
+
git worktree add --detach /tmp/solid-objects-v1.0.5 v1.0.5
|
|
120
|
+
git archive "$harness_ref" benchmarks | tar -x -C /tmp/solid-objects-v1.0.5
|
|
121
|
+
cd /tmp/solid-objects-v1.0.5
|
|
122
|
+
pnpm install --frozen-lockfile
|
|
123
|
+
pnpm exec tsc -p tsconfig.build.json
|
|
124
|
+
git diff --exit-code v1.0.5 -- src
|
|
125
|
+
|
|
126
|
+
node benchmarks/run.ts --database sqlite --workload counter \
|
|
127
|
+
--operations 250 --warmup 25 --repetitions 3 > sqlite-counter.json
|
|
128
|
+
node benchmarks/run.ts --database sqlite --workload cart-outbox \
|
|
129
|
+
--operations 250 --warmup 25 --repetitions 3 > sqlite-cart-outbox.json
|
|
130
|
+
node benchmarks/large-state.ts --operations 300 --warmup 25 \
|
|
131
|
+
--repetitions 3 > sqlite-large-state.json
|
|
132
|
+
node benchmarks/idle.ts > sqlite-idle.json
|
|
133
|
+
cd "$repository_root"
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Repeat the matrix with `--database postgresql` and `--database mysql` and the benchmark URLs shown
|
|
137
|
+
below, using disposable local databases. Reports contain driver versions and durability settings
|
|
138
|
+
without connection URLs or credentials. Run all scenarios sequentially. Use the recorded host, Node,
|
|
139
|
+
database settings, and driver versions when comparing results; ordinary shared-host activity and
|
|
140
|
+
storage behavior still affect them.
|
|
141
|
+
|
|
142
|
+
Generate the complete tables from the retained reports with:
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
node benchmarks/summarize.mjs benchmarks/results/v1.0.5-2026-11-07/*.json
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
The summarizer rejects mixed runtime/harness provenance, missing scenarios or samples, and missing
|
|
149
|
+
per-operation latencies. Keep the per-sample JSON when reporting a median so the variability remains
|
|
150
|
+
reviewable.
|
|
151
|
+
|
|
152
|
+
The August measurements below retain their original versions, hardware, and methodology.
|
|
5
153
|
|
|
6
154
|
## Idle polling
|
|
7
155
|
|
|
8
|
-
The idle harness measures process CPU and empty database passes for the four
|
|
9
|
-
runtime roles:
|
|
156
|
+
The idle harness measures process CPU and empty database passes for the four runtime roles:
|
|
10
157
|
|
|
11
158
|
```bash
|
|
12
159
|
pnpm run benchmark:idle
|
|
13
160
|
```
|
|
14
161
|
|
|
15
|
-
It warms each interval for three seconds, measures for ten seconds, and reports
|
|
16
|
-
|
|
162
|
+
It warms each interval for three seconds, measures for ten seconds, and reports process user plus
|
|
163
|
+
system CPU time divided by wall time.
|
|
17
164
|
|
|
18
|
-
Measured on August 16, 2026 on an Apple M5 with Node.js 26.7.0 and in-memory
|
|
19
|
-
|
|
20
|
-
|
|
165
|
+
Measured on August 16, 2026 on an Apple M5 with Node.js 26.7.0 and in-memory SQLite. The before run
|
|
166
|
+
used 0.13.0; the after run used the prepared 0.13.1 tree. Each run started one actor, effect,
|
|
167
|
+
reminder, and broadcast role.
|
|
21
168
|
|
|
22
169
|
| Fast interval | Before polls/s | Before CPU | After polls/s | After CPU |
|
|
23
170
|
| ------------: | -------------: | ---------: | ------------: | --------: |
|
|
@@ -25,29 +172,28 @@ Each run started one actor, effect, reminder, and broadcast role.
|
|
|
25
172
|
| 100 ms | 39.596 | 0.906% | 3.999 | 0.121% |
|
|
26
173
|
| 500 ms | 7.999 | 0.251% | 3.999 | 0.104% |
|
|
27
174
|
|
|
28
|
-
The after run reached the one-second ceiling for all four roles. These are
|
|
29
|
-
|
|
30
|
-
|
|
175
|
+
The after run reached the one-second ceiling for all four roles. These are developer-laptop
|
|
176
|
+
measurements, not a CPU guarantee; timer scheduling, JIT, database path, and unrelated host activity
|
|
177
|
+
affect short samples.
|
|
31
178
|
|
|
32
|
-
Five SQLite samples measured durable enqueue through committed completion after
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
than average arrival latency.
|
|
179
|
+
Five SQLite samples measured durable enqueue through committed completion after 2.5 seconds of
|
|
180
|
+
idleness. The polling-only multi-process harness submits just after an empty pass, so it measures
|
|
181
|
+
approximately the full polling wait rather than average arrival latency.
|
|
36
182
|
|
|
37
183
|
| Topology | 0.13.0 p50 | Prepared 0.13.1 p50 |
|
|
38
184
|
| ------------------------------- | ---------: | ------------------: |
|
|
39
185
|
| One process, in-process wake-up | 2.589 ms | 2.662 ms |
|
|
40
186
|
| Two processes, polling only | 107.945 ms | 1,006.232 ms |
|
|
41
187
|
|
|
42
|
-
The local wake-up keeps the one-process path prompt after backoff. The
|
|
43
|
-
|
|
44
|
-
|
|
188
|
+
The local wake-up keeps the one-process path prompt after backoff. The polling-only row is the
|
|
189
|
+
explicit tradeoff: use PostgreSQL notifications or optional Redis Pub/Sub when separate processes
|
|
190
|
+
need low-latency delivery.
|
|
45
191
|
|
|
46
192
|
## Large state
|
|
47
193
|
|
|
48
|
-
The large-state harness measures how the committed state size changes
|
|
49
|
-
|
|
50
|
-
|
|
194
|
+
The large-state harness measures how the committed state size changes throughput. It runs one actor
|
|
195
|
+
and one `increment()` operation with sequential turns, so each row isolates the per-turn
|
|
196
|
+
serialization and row write:
|
|
51
197
|
|
|
52
198
|
```bash
|
|
53
199
|
pnpm run benchmark:large-state
|
|
@@ -55,11 +201,10 @@ pnpm run benchmark:large-state
|
|
|
55
201
|
|
|
56
202
|
Use `--sizes`, `--operations`, and `--warmup` to change the recorded dataset.
|
|
57
203
|
|
|
58
|
-
Measured on August 29, 2026 on an Apple M5, macOS 26.6, Node.js 24.18.0, and
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
per-turn state serialization.
|
|
204
|
+
Measured on August 29, 2026 on an Apple M5, macOS 26.6, Node.js 24.18.0, and SQLite 3.53.1 through
|
|
205
|
+
`node:sqlite` on a temporary file. Each row is the median of three runs of 300 measured operations.
|
|
206
|
+
The before column used the `0.14.3` tree; the after column used the `0.14.4` tree, which removes the
|
|
207
|
+
redundant per-turn state serialization.
|
|
63
208
|
|
|
64
209
|
| Persisted state | Before ms/op | Before ops/s | After ms/op | After ops/s | Gain |
|
|
65
210
|
| --------------: | -----------: | -----------: | ----------: | ----------: | ---: |
|
|
@@ -68,15 +213,13 @@ per-turn state serialization.
|
|
|
68
213
|
| 128 KB | 4.01 | 250 | 2.53 | 395 | 1.6x |
|
|
69
214
|
| 1 MB | 19.11 | 52 | 9.32 | 107 | 2.1x |
|
|
70
215
|
|
|
71
|
-
The before tree traversed the whole state eight times for each committed
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
write of a large row, which the whole-image commit model cannot avoid.
|
|
216
|
+
The before tree traversed the whole state eight times for each committed operation, and nine times
|
|
217
|
+
for a query. The after tree traverses it four times: one image for rollback and comparison, one
|
|
218
|
+
committed image, and one read for each of the two observables guards. The remaining cost at 1 MB is
|
|
219
|
+
the database write of a large row, which the whole-image commit model cannot avoid.
|
|
76
220
|
|
|
77
|
-
The curve, not the ratio, is the operating instruction. Throughput falls by
|
|
78
|
-
|
|
79
|
-
small. See
|
|
221
|
+
The curve, not the ratio, is the operating instruction. Throughput falls by about 6x between 0 KB
|
|
222
|
+
and 1 MB even after the change. Keep one actor's state small. See
|
|
80
223
|
[State size and throughput](state-and-lifecycle.md#state-size-and-throughput).
|
|
81
224
|
|
|
82
225
|
## Scenarios
|
|
@@ -85,15 +228,13 @@ small. See
|
|
|
85
228
|
- `warm-many`: operations rotate across 100 previously created identities.
|
|
86
229
|
- `cold-many`: every measured operation creates a new identity.
|
|
87
230
|
- `synchronous`: the actor method mutates state without yielding.
|
|
88
|
-
- `asynchronous`: the actor method yields through `setImmediate()` before the
|
|
89
|
-
same mutation.
|
|
231
|
+
- `asynchronous`: the actor method yields through `setImmediate()` before the same mutation.
|
|
90
232
|
- `one-process`: four actor workers run in the caller's Node process.
|
|
91
233
|
- `four-processes`: four Node worker processes share the database.
|
|
92
234
|
|
|
93
|
-
Latency begins before durable enqueue and ends when the committed result is
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
and client concurrency 16.
|
|
235
|
+
Latency begins before durable enqueue and ends when the committed result is available. Throughput
|
|
236
|
+
uses the wall time for the measured batch. Percentiles use nearest rank. Defaults are 25 warmup
|
|
237
|
+
operations, 250 measured operations, and client concurrency 16.
|
|
97
238
|
|
|
98
239
|
## Run the harness
|
|
99
240
|
|
|
@@ -103,8 +244,8 @@ SQLite needs no service:
|
|
|
103
244
|
pnpm run benchmark -- --database sqlite
|
|
104
245
|
```
|
|
105
246
|
|
|
106
|
-
PostgreSQL and MySQL runs require a disposable database. The harness deletes
|
|
107
|
-
|
|
247
|
+
PostgreSQL and MySQL runs require a disposable database. The harness deletes benchmark rows but
|
|
248
|
+
leaves its uniquely prefixed empty tables for inspection.
|
|
108
249
|
|
|
109
250
|
```bash
|
|
110
251
|
SOLID_OBJECTS_POSTGRESQL_BENCHMARK_URL=postgresql://... \
|
|
@@ -114,8 +255,9 @@ SOLID_OBJECTS_MYSQL_BENCHMARK_URL=mysql://... \
|
|
|
114
255
|
pnpm run benchmark -- --database mysql
|
|
115
256
|
```
|
|
116
257
|
|
|
117
|
-
Use `--operations`, `--warmup`, and `--
|
|
118
|
-
|
|
258
|
+
Use `--operations`, `--warmup`, `--concurrency`, and `--repetitions` to change the recorded dataset.
|
|
259
|
+
The matrix defaults to three repetitions. Select `--workload counter` or `--workload cart-outbox`.
|
|
260
|
+
Redirect stdout to retain the JSON result; progress and runtime diagnostics go to stderr.
|
|
119
261
|
|
|
120
262
|
## Observed results
|
|
121
263
|
|
|
@@ -125,8 +267,8 @@ Measured on August 22, 2026 with the `0.14.0` source tree:
|
|
|
125
267
|
- macOS 26.6
|
|
126
268
|
- Node.js 24.18.0
|
|
127
269
|
- SQLite 3.53.1 through `node:sqlite` on the internal SSD
|
|
128
|
-
- PostgreSQL 17.11 and MySQL 9.7.1, both installed natively and started on a
|
|
129
|
-
|
|
270
|
+
- PostgreSQL 17.11 and MySQL 9.7.1, both installed natively and started on a scoped temporary data
|
|
271
|
+
directory
|
|
130
272
|
- 25 warmup operations, 250 measured operations, concurrency 16
|
|
131
273
|
|
|
132
274
|
### SQLite 3.53.1
|
|
@@ -180,16 +322,15 @@ Measured on August 22, 2026 with the `0.14.0` source tree:
|
|
|
180
322
|
| four processes | cold many | synchronous | 51.82 | 306.1 | 407.22 | 424.45 |
|
|
181
323
|
| four processes | cold many | asynchronous | 27.35 | 554.03 | 807.18 | 868.48 |
|
|
182
324
|
|
|
183
|
-
The cold and asynchronous cases keep the poorest throughput and the longest
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
versions, but this harness did not measure them.
|
|
325
|
+
The cold and asynchronous cases keep the poorest throughput and the longest tail. These are observed
|
|
326
|
+
limitations. They are not capacity recommendations. Repeat the runs on application-shaped payloads
|
|
327
|
+
before you draw a general conclusion. Integration tests cover PostgreSQL 14, MySQL 8.0, and other
|
|
328
|
+
database versions, but this harness did not measure them.
|
|
188
329
|
|
|
189
330
|
### Virtualization cost
|
|
190
331
|
|
|
191
|
-
The same server version ran natively and in Docker Desktop on the same machine,
|
|
192
|
-
|
|
332
|
+
The same server version ran natively and in Docker Desktop on the same machine, on the same day,
|
|
333
|
+
through the same harness. Only the container boundary changes.
|
|
193
334
|
|
|
194
335
|
| Database | Topology and shape | Native ops/s | Docker ops/s | Native gain |
|
|
195
336
|
| ---------------- | ------------------------------- | -----------: | -----------: | ----------: |
|
|
@@ -198,25 +339,23 @@ on the same day, through the same harness. Only the container boundary changes.
|
|
|
198
339
|
| MySQL 9.7.1 | four processes, warm hot, async | 228.32 | 60.27 | 3.8x |
|
|
199
340
|
| MySQL 9.7.1 | one process, warm hot, sync | 79.59 | 44.56 | 1.8x |
|
|
200
341
|
|
|
201
|
-
Across the full matrix, Docker Desktop cost between 1.0x and 7.8x of the native
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
342
|
+
Across the full matrix, Docker Desktop cost between 1.0x and 7.8x of the native throughput. The
|
|
343
|
+
multi-process rows lose the most, because more connections and more commits cross the container
|
|
344
|
+
boundary. Measure a database on the deployment shape you intend to run, and state the boundary with
|
|
345
|
+
any number you publish.
|
|
205
346
|
|
|
206
|
-
Earlier releases of this document reported PostgreSQL 18.4 and MySQL 8.4.11 in
|
|
207
|
-
|
|
208
|
-
|
|
347
|
+
Earlier releases of this document reported PostgreSQL 18.4 and MySQL 8.4.11 in Docker Desktop on the
|
|
348
|
+
`0.13.0` tree. Those numbers measured the container as much as the database, so the tables above
|
|
349
|
+
replace them.
|
|
209
350
|
|
|
210
351
|
## Sources of bias
|
|
211
352
|
|
|
212
353
|
- A developer laptop shares CPU, memory, and storage with unrelated processes.
|
|
213
354
|
- Loopback database connections exclude production network latency.
|
|
214
|
-
- Filesystem cache, SQLite WAL state, Node JIT warmup, and garbage collection
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
-
|
|
219
|
-
See [Large state](#large-state) for the measured effect of state size.
|
|
355
|
+
- Filesystem cache, SQLite WAL state, Node JIT warmup, and garbage collection affect short runs.
|
|
356
|
+
- Docker Desktop costs between 1.0x and 7.8x of the native throughput. The tables above use native
|
|
357
|
+
servers. See [Virtualization cost](#virtualization-cost).
|
|
358
|
+
- The payload is a small counter, not a representative application state size. See
|
|
359
|
+
[Large state](#large-state) for the measured effect of state size.
|
|
220
360
|
- The harness measures default durability settings and one client concurrency.
|
|
221
|
-
- Hot-identity results deliberately include serialization and cannot be scaled
|
|
222
|
-
by adding workers.
|
|
361
|
+
- Hot-identity results deliberately include serialization and cannot be scaled by adding workers.
|
|
@@ -0,0 +1,252 @@
|
|
|
1
|
+
# Durable execution for TypeScript AI agents on PostgreSQL: Restate, DBOS, Temporal, and Solid Objects
|
|
2
|
+
|
|
3
|
+
A long-lived AI agent session must keep its conversation state, apply messages in order, and wake up later for a follow-up, after deploys and crashes. Restate, DBOS Transact, and Temporal do this with durable execution: each tool records the steps of a handler or workflow and resumes after a crash. Solid Objects (the `solid-objects` npm package) does it with an actor for each session: an ordered mailbox, durable state, and reminders in the SQLite, PostgreSQL, or MySQL database that the app already uses. Only DBOS and Solid Objects write their state from your app process to a PostgreSQL database. Restate keeps it in the Restate server. Temporal keeps it in the Temporal Service, which can use PostgreSQL as its own store.
|
|
4
|
+
|
|
5
|
+
Solid Objects is an independent project. It is not affiliated with Restate, DBOS, or Temporal Technologies.
|
|
6
|
+
|
|
7
|
+
## The short answer
|
|
8
|
+
|
|
9
|
+
| Tool | Where the state lives | What a crash repeats | How a delayed follow-up works | What you run |
|
|
10
|
+
| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
|
11
|
+
| Restate | A journal and a key-value store inside the Restate server, not in PostgreSQL | "Restate replays the journal, skipping completed steps". A `ctx.run` action that did not record its result can run again | A delayed message: `ctx.objectSendClient(...).handler(input, restate.rpc.sendOpts({ delay }))`, or `ctx.sleep` | Restate Cloud, Restate BYOC, or a self-hosted Restate server (a single binary with a persistent volume), and your service |
|
|
12
|
+
| DBOS Transact | Checkpoints in a PostgreSQL system database. DBOS recommends "storing long-lived objects directly in your database" | "Steps are tried _at least once_ but are never re-executed after they complete" | Durable sleep with `DBOS.sleep(milliseconds)` in a workflow, or `delaySeconds` when you enqueue | The library in your Node.js processes and PostgreSQL. The docs say that distributed applications "should also connect to DBOS Conductor" |
|
|
13
|
+
| Temporal | The event history of each workflow, in the Temporal Service. The service stores it in Cassandra, MySQL, or PostgreSQL (SQLite for development only) | Workflow code replays its event history. "completed Activities will not re-execute", but an activity that did not report its result "will be retried" | A durable timer: `sleep('2 hours')` from `@temporalio/workflow`, or `condition` with a timeout | A Temporal Service (the Temporal Server and a database) or Temporal Cloud, and your worker processes |
|
|
14
|
+
| Solid Objects | Actor state, the mailbox, and reminders in tables in your SQL database | A turn that did not commit runs again. Effects run at least once with a stable effect ID | A reminder: `this.schedule({ at, key }).followUp(input)`, stored as a row | Your Node.js process, which calls `runtime.run(signal)`, and your database |
|
|
15
|
+
|
|
16
|
+
The facts come from the [Restate documentation](https://docs.restate.dev), the [DBOS documentation](https://docs.dbos.dev), and the [Temporal documentation](https://docs.temporal.io) (checked October 18, 2026).
|
|
17
|
+
|
|
18
|
+
## Messages in order for one session
|
|
19
|
+
|
|
20
|
+
- Restate: a virtual object for each session. "Only one write handler runs at a time per key", and "concurrent requests to the same session are automatically queued".
|
|
21
|
+
- DBOS Transact: a workflow for each session that waits with `DBOS.recv`. "Messages are dequeued first-in, first-out from a queue associated with the topic." A partitioned queue with `queuePartitionKey` and `partitionConcurrency: 1` also runs one workflow at a time for each key.
|
|
22
|
+
- Temporal: a long-lived workflow for each session that gets Signals or Updates. Temporal processes messages "in the order they were received". Handlers that wait "can run interleaved with the main Workflow and with one another".
|
|
23
|
+
- Solid Objects: an actor for each session. Messages run one turn at a time, in the order that the runtime stores them.
|
|
24
|
+
|
|
25
|
+
## Where the model call runs
|
|
26
|
+
|
|
27
|
+
- Restate: in a `ctx.run` step inside the handler. Restate says that model calls are "Persisted so responses are not re-fetched on recovery". In an exclusive handler, a new message for the same session waits until the handler finishes.
|
|
28
|
+
- DBOS Transact: in a step of the workflow. After the step completes, DBOS does not run it again.
|
|
29
|
+
- Temporal: in an activity. The Temporal integration with the OpenAI Agents SDK for TypeScript is marked "Pre-release". It says that "model calls are executed as Activities, so they retry durably and are not repeated during Workflow replay".
|
|
30
|
+
- Solid Objects: in an effect, outside the turn. The turn ends at once, so a new message does not wait for the model. The effect handler sends the effect ID as the idempotency key.
|
|
31
|
+
|
|
32
|
+
## Choose Restate when
|
|
33
|
+
|
|
34
|
+
- You want a journal of each step, and you can run the Restate server or use Restate Cloud.
|
|
35
|
+
- You want workflows, service calls, and virtual objects in one runtime.
|
|
36
|
+
|
|
37
|
+
See [Restate virtual objects and Solid Objects](restate.md) for the same agent session in each, with code.
|
|
38
|
+
|
|
39
|
+
## Choose DBOS Transact when
|
|
40
|
+
|
|
41
|
+
- The agent is a workflow of steps, and a completed step must not run again after a crash.
|
|
42
|
+
- You want the checkpoints in PostgreSQL, with no separate orchestration server.
|
|
43
|
+
|
|
44
|
+
See [DBOS Transact and Solid Objects](dbos.md) for the details.
|
|
45
|
+
|
|
46
|
+
## Choose Temporal when
|
|
47
|
+
|
|
48
|
+
- The agent is a long workflow of many steps, and you want its event history and replay.
|
|
49
|
+
- You already run a Temporal Service, or you can use Temporal Cloud.
|
|
50
|
+
- A session can stay inside the history limit: "51,200 Events or 50 MB". A longer session continues as new with a new history.
|
|
51
|
+
|
|
52
|
+
## Choose Solid Objects when
|
|
53
|
+
|
|
54
|
+
- Each session has state that many messages change in order, and the session must wake up later.
|
|
55
|
+
- You want the session state in the SQL database that you already operate, back up, and query.
|
|
56
|
+
- You want a state change and its effects and reminders to commit in one SQL transaction.
|
|
57
|
+
- You do not want to run another server.
|
|
58
|
+
|
|
59
|
+
## An AI agent session with Solid Objects
|
|
60
|
+
|
|
61
|
+
The task: a Node.js TypeScript backend runs a long-lived AI agent session for each user. A session holds conversation state. It receives user messages and tool results that must apply in order. It can schedule its own follow-up, for example "check the order status again in 2 hours". Sessions must survive deploys and process crashes. The app uses PostgreSQL and runs several Node.js processes.
|
|
62
|
+
|
|
63
|
+
There is one `AgentSession` actor for each user ID.
|
|
64
|
+
|
|
65
|
+
```typescript
|
|
66
|
+
import { Actor, type EffectSuccessPayload } from "solid-objects"
|
|
67
|
+
|
|
68
|
+
export const FOLLOW_UP_MILLISECONDS = 2 * 60 * 60 * 1000
|
|
69
|
+
|
|
70
|
+
const KEPT_ENTRIES = 50
|
|
71
|
+
|
|
72
|
+
export type Entry = { role: "user" | "tool" | "assistant"; content: string }
|
|
73
|
+
|
|
74
|
+
type ReplyArguments = { turn: number; history: Entry[] }
|
|
75
|
+
type ReplyResult = { content: string }
|
|
76
|
+
|
|
77
|
+
export class AgentSession extends Actor {
|
|
78
|
+
static override readonly actorType = "AgentSession"
|
|
79
|
+
|
|
80
|
+
history: Entry[] = []
|
|
81
|
+
turn = 0
|
|
82
|
+
followUps: string[] = []
|
|
83
|
+
|
|
84
|
+
receive({ role, content }: { role: "user" | "tool"; content: string }): { turn: number } {
|
|
85
|
+
this.#append({ role, content })
|
|
86
|
+
return { turn: this.turn }
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
followUpLater({ topic }: { topic: string }): void {
|
|
90
|
+
this.schedule({ at: new Date(Date.now() + FOLLOW_UP_MILLISECONDS), key: topic }).followUp({
|
|
91
|
+
topic,
|
|
92
|
+
})
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
followUp({ topic }: { topic: string }): void {
|
|
96
|
+
this.followUps = [...this.followUps, topic]
|
|
97
|
+
this.#append({ role: "tool", content: `Follow-up due: ${topic}` })
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
replied({ arguments: reply, result }: EffectSuccessPayload<ReplyArguments, ReplyResult>): void {
|
|
101
|
+
if (reply.turn !== this.turn) return
|
|
102
|
+
this.history = [...this.history, { role: "assistant" as const, content: result.content }].slice(
|
|
103
|
+
-KEPT_ENTRIES,
|
|
104
|
+
)
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
#append(entry: Entry): void {
|
|
108
|
+
this.history = [...this.history, entry].slice(-KEPT_ENTRIES)
|
|
109
|
+
this.turn += 1
|
|
110
|
+
this.emit("model_reply", {
|
|
111
|
+
arguments: { turn: this.turn, history: this.history },
|
|
112
|
+
onSuccess: "replied",
|
|
113
|
+
})
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
The runtime registers the actor and the effect that calls the model:
|
|
119
|
+
|
|
120
|
+
```typescript
|
|
121
|
+
import { createRuntime, type SolidObjectsRuntime } from "solid-objects"
|
|
122
|
+
import { sqlite } from "solid-objects/database/sqlite"
|
|
123
|
+
import { AgentSession, type Entry } from "./agent-session.js"
|
|
124
|
+
|
|
125
|
+
export type ModelClient = {
|
|
126
|
+
reply(options: { idempotencyKey: string; history: Entry[] }): Promise<{ content: string }>
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
const CALLER_OPERATIONS = new Set(["receive", "followUpLater"])
|
|
130
|
+
|
|
131
|
+
export function agentRuntime({
|
|
132
|
+
path,
|
|
133
|
+
model,
|
|
134
|
+
retryDelayMilliseconds,
|
|
135
|
+
}: {
|
|
136
|
+
path: string
|
|
137
|
+
model: ModelClient
|
|
138
|
+
retryDelayMilliseconds?: (attempt: number) => number
|
|
139
|
+
}): SolidObjectsRuntime {
|
|
140
|
+
const runtime = createRuntime({
|
|
141
|
+
database: sqlite({ path }),
|
|
142
|
+
...(retryDelayMilliseconds ? { retryDelayMilliseconds } : {}),
|
|
143
|
+
authorizeMessage: ({ actorType, actorId, operation, authorizationContext }) =>
|
|
144
|
+
actorType === AgentSession.actorType &&
|
|
145
|
+
CALLER_OPERATIONS.has(operation) &&
|
|
146
|
+
ownsSession({ actorId, authorizationContext }),
|
|
147
|
+
authorizeQuery: ({ actorType, actorId, authorizationContext }) =>
|
|
148
|
+
actorType === AgentSession.actorType && ownsSession({ actorId, authorizationContext }),
|
|
149
|
+
})
|
|
150
|
+
runtime.register(AgentSession)
|
|
151
|
+
runtime.registerEffect("model_reply", (argumentsValue, context) =>
|
|
152
|
+
model.reply({ idempotencyKey: context.id, history: argumentsValue.history as Entry[] }),
|
|
153
|
+
)
|
|
154
|
+
return runtime
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
function ownsSession({
|
|
158
|
+
actorId,
|
|
159
|
+
authorizationContext,
|
|
160
|
+
}: {
|
|
161
|
+
actorId: string
|
|
162
|
+
authorizationContext: unknown
|
|
163
|
+
}): boolean {
|
|
164
|
+
if (typeof authorizationContext !== "object" || authorizationContext === null) return false
|
|
165
|
+
return "userId" in authorizationContext && authorizationContext.userId === actorId
|
|
166
|
+
}
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
The example uses SQLite so that the tests run without a database server. For PostgreSQL, import `postgresql` from `solid-objects/database/postgresql` and pass `postgresql({ connectionString })` as `database`.
|
|
170
|
+
|
|
171
|
+
The app sends each message with `send`, so that the message is stored before the call returns:
|
|
172
|
+
|
|
173
|
+
```typescript
|
|
174
|
+
await runtime
|
|
175
|
+
.ref(AgentSession, user.id)
|
|
176
|
+
.send.with({ authorizationContext: { userId: user.id }, idempotencyKey: message.id })
|
|
177
|
+
.receive({ role: "user", content: message.text })
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
At least one process calls `runtime.run(signal)`. It runs the stored messages, the effects, and the reminders.
|
|
181
|
+
|
|
182
|
+
How it works:
|
|
183
|
+
|
|
184
|
+
- `receive` appends the entry, moves `turn` forward, and stages a `model_reply` effect. The model call runs outside the turn.
|
|
185
|
+
- If the effect fails, the runtime retries it with the same ID, up to `maxAttempts` (5 by default).
|
|
186
|
+
- `replied` adds the model reply only if no newer message or follow-up arrived.
|
|
187
|
+
- `followUpLater` schedules a keyed reminder. The reminder is a row in the database, so it runs after a deploy or a restart.
|
|
188
|
+
|
|
189
|
+
A model call can run again after a crash, because effects run at least once. If the model API does not accept an idempotency key, the repeated call can cost tokens again. A step in Restate, DBOS, or Temporal that crashes before its result is recorded runs again too.
|
|
190
|
+
|
|
191
|
+
### What the tests prove
|
|
192
|
+
|
|
193
|
+
[The tests](../../test/compare-agent-session.test.ts) show these results:
|
|
194
|
+
|
|
195
|
+
- Two user messages and a tool result apply in the order that they were sent.
|
|
196
|
+
- A model call that fails and runs again uses the same idempotency key, and the session keeps one reply.
|
|
197
|
+
- When two messages arrive before the model replies, the session keeps only the reply for the newest turn.
|
|
198
|
+
- A follow-up runs after a runtime restart.
|
|
199
|
+
- A caller who does not own the session gets `Unauthorized`.
|
|
200
|
+
|
|
201
|
+
## Questions that teams ask
|
|
202
|
+
|
|
203
|
+
**Which of these keep state in my PostgreSQL database?** DBOS Transact keeps its checkpoints in a PostgreSQL system database. Solid Objects keeps actor state in your SQLite, PostgreSQL, or MySQL database. Restate keeps state in its own server. For that state, Restate says "SQL is supported only for analytics / introspection, not for updates/transactions". Temporal keeps the event history in the Temporal Service, which can use PostgreSQL as its own store.
|
|
204
|
+
|
|
205
|
+
**Is Solid Objects mature enough?** Here are the facts:
|
|
206
|
+
|
|
207
|
+
- Version 1.0.0: October 13, 2026. The public API follows [semantic versioning](../versioning.md).
|
|
208
|
+
- [Maintenance and continuity](../maintenance.md) tells you who maintains the project and how a release is made.
|
|
209
|
+
- One first-party application with more than 100,000 users runs it in production. That is a count of users, not a measure of load.
|
|
210
|
+
- The [benchmarks](../benchmarks.md) are development measurements. [Versioning](../versioning.md) states what version 1.0 does not claim.
|
|
211
|
+
|
|
212
|
+
The npm packages of Temporal, Restate, and DBOS were first published between 2021 and 2023 ([npm registry](https://www.npmjs.com), checked October 18, 2026). If a longer history matters more than keeping the session state in your own database, choose one of them.
|
|
213
|
+
|
|
214
|
+
**Is delivery exactly once?** No. In Solid Objects, delivery is at least once. Use the effect ID as the idempotency key for each external call. See [correctness](../correctness.md) and [effect recovery](../effect-recovery.md).
|
|
215
|
+
|
|
216
|
+
**Can I use more than one?** Yes. For example, keep each session in a Solid Objects actor, and run a long multi-step task as a workflow in DBOS or Temporal.
|
|
217
|
+
|
|
218
|
+
## Sources
|
|
219
|
+
|
|
220
|
+
Checked October 18, 2026:
|
|
221
|
+
|
|
222
|
+
- [Restate key concepts](https://docs.restate.dev/foundations/key-concepts)
|
|
223
|
+
- [Restate durable sessions](https://docs.restate.dev/ai/patterns/sessions)
|
|
224
|
+
- [Restate durable agents](https://docs.restate.dev/ai/patterns/durable-agents)
|
|
225
|
+
- [Restate scheduling and timers](https://docs.restate.dev/develop/ts/durable-timers)
|
|
226
|
+
- [Restate service communication](https://docs.restate.dev/develop/ts/service-communication)
|
|
227
|
+
- [Restate hosting overview](https://docs.restate.dev/hosting/overview)
|
|
228
|
+
- [Restate server overview](https://docs.restate.dev/server/overview)
|
|
229
|
+
- [Restate databases guide](https://docs.restate.dev/guides/databases)
|
|
230
|
+
- [DBOS architecture](https://docs.dbos.dev/architecture)
|
|
231
|
+
- [Migrating from Temporal to DBOS](https://docs.dbos.dev/explanations/migrating-from-temporal)
|
|
232
|
+
- [DBOS workflow tutorial](https://docs.dbos.dev/typescript/tutorials/workflow-tutorial)
|
|
233
|
+
- [DBOS workflow communication](https://docs.dbos.dev/typescript/tutorials/workflow-communication)
|
|
234
|
+
- [DBOS methods reference](https://docs.dbos.dev/typescript/reference/methods)
|
|
235
|
+
- [DBOS queue tutorial](https://docs.dbos.dev/typescript/tutorials/queue-tutorial)
|
|
236
|
+
- [Temporal workflow execution](https://docs.temporal.io/workflow-execution)
|
|
237
|
+
- [Temporal event history](https://docs.temporal.io/workflow-execution/event)
|
|
238
|
+
- [Temporal activity definition](https://docs.temporal.io/activity-definition)
|
|
239
|
+
- [Temporal TypeScript timers](https://docs.temporal.io/develop/typescript/workflows/timers)
|
|
240
|
+
- [Temporal message handling](https://docs.temporal.io/handling-messages)
|
|
241
|
+
- [Temporal workflow execution limits](https://docs.temporal.io/workflow-execution/limits)
|
|
242
|
+
- [Temporal Service persistence](https://docs.temporal.io/temporal-service/persistence)
|
|
243
|
+
- [Temporal workers](https://docs.temporal.io/workers)
|
|
244
|
+
- [Temporal and the OpenAI Agents SDK for TypeScript](https://docs.temporal.io/develop/typescript/integrations/openai-agents)
|
|
245
|
+
|
|
246
|
+
## More information
|
|
247
|
+
|
|
248
|
+
- [All comparisons](../comparisons.md)
|
|
249
|
+
- [Restate virtual objects and Solid Objects](restate.md)
|
|
250
|
+
- [DBOS Transact and Solid Objects](dbos.md)
|
|
251
|
+
- [Virtual actors in TypeScript](../virtual-actors.md)
|
|
252
|
+
- [Guide for coding agents](../agents.md)
|