stablemates-workhorse 0.9.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b030c71db8e2fa0f3b2e14e4ac34442270e1643ea1105efe904bc965ab6e4823
4
- data.tar.gz: 90256bbd761dfa37265e9e0fce4f5f49534e378361a625bbc6bc27cf24b7634b
3
+ metadata.gz: 1686623ff5e225f372245830a169d9b248db6bd195ed3f446679e01098bc9610
4
+ data.tar.gz: bde5f7e6517d2a7a3621b437237db776673156693b22fc1a46f42aa28305b563
5
5
  SHA512:
6
- metadata.gz: 9d580a9360fe1772b51567673be3d787ea287ba7df8693d93d8ea4b153620f0a4c0b4c5ad492593830127ccd2c78a1063b5ac5f9174bbbea6aae3f1ca2880b70
7
- data.tar.gz: d353d8e0ce3d2b03b869212dee916bcead4c41034de9403c83ab1f3e3a74eaaadfa36bf0c0c0c03f22728efdee770ef437c3de369f51d1f36765f5861948679f
6
+ metadata.gz: eef44a6c8715cc1d6c0f7983fb42b3662332ad1302fd6d705e60eca9f2d4ed099d32fd53d1303a2ee1a0cdfa9ee08c3624c57516c967c22cb62f0366180bdf4b
7
+ data.tar.gz: e9137fa4c8b8f6a5f20f800cb4abc482e6c750721758fb5432f407c229816b4a4186700e882eeb8859a8d21f8bcfa2f2be6be1ae734ef574a003227a8db82a94
data/CHANGELOG.md CHANGED
@@ -3,6 +3,585 @@
3
3
  `stablemates-workhorse` gem versions and release notes live here. The gem carries the version the
4
4
  other SDKs carry, because every tag names one release of all of them.
5
5
 
6
+ ## 0.11.0 — 2026-10-28
7
+
8
+ The npm packages, Python distribution, Go module, Rust crate, Ruby gem, JVM artifacts, and .NET
9
+ packages release from one source commit.
10
+
11
+ Requires **schema v76**, Ruby **3.3** or newer, and PostgreSQL **15** or newer.
12
+
13
+ ### Upgrade steps
14
+
15
+ **Migrate the schema before starting updated processes.** Run `workhorse schema migrate` before any
16
+ process of this release starts, producers too. Migrations 0088, 0089, and 0090 move the schema
17
+ from version 86 to version **89**. They replace function bodies and change no stored rows. The SDK
18
+ compatibility floor stays at **76**. A 0.10 process keeps working on version 89, because a
19
+ migration inside a major line only adds.
20
+
21
+ **Give each worker a pool with a known size of at least 3.** `Worker.new` no longer accepts
22
+ `shared_heartbeats:`. Remove the option. If a worker pool reports a size below 3, or reports no
23
+ size, set its size to 3 or more. Each worker process needs 3 connections: the listener, the
24
+ heartbeat connection, and one statement connection. A `polling_only: true` worker opens no
25
+ listener, but it still needs a pool of 3 (SM-1901).
26
+
27
+ ### Breaking changes
28
+
29
+ - **`shared_heartbeats:` is removed.** Every worker now reserves a dedicated heartbeat connection
30
+ on its pool (ADR 0071). Before, `shared_heartbeats: true` let heartbeat rounds borrow ordinary
31
+ pool connections and let a pool below 3 run. Then a heartbeat could wait behind claims and
32
+ settlements, so a lease could expire and a task could run twice. Now `Worker.new` raises
33
+ `ArgumentError` when it gets `shared_heartbeats:`, and the message names the removal. A worker
34
+ on a pool with a size below 3, or with no reported size, raises `ArgumentError` at construction.
35
+ The worker no longer logs `workhorse.worker.polling_only`, because its pool always has a
36
+ connection for the listener (SM-1901).
37
+
38
+ ### Behavior changes
39
+
40
+ These changes can make code that worked on 0.10 act differently:
41
+
42
+ 1. An idempotency TTL of exactly 0 still selects the default. The queue now raises
43
+ `ArgumentError` for a TTL that is not 0 and rounds to 0 ms (SM-1857). See "An idempotency
44
+ TTL that rounds to 0 ms is refused".
45
+ 2. The TypeScript `Queue`, `Admin`, and `Worker.runOnce()` now check the schema before their first
46
+ statement (SM-1864). This SDK already did, so it does not change.
47
+ 3. After a lease lapses while a handler runs, a stale failure or release now settles a due
48
+ deadline or attempt timeout at once. The worker reports `deadline_exceeded` or `timeout`, not
49
+ `lease_lost` (SM-1884). See "A stale failure or release after a lapsed lease settles its
50
+ cause".
51
+ 4. Every SDK removes the heartbeat opt-out (SM-1899, SM-1901, SM-1902). In this SDK,
52
+ `Worker.new` raises `ArgumentError` for `shared_heartbeats:`, and for a pool with a size below
53
+ 3 or with no reported size. See "Breaking changes".
54
+
55
+ ### A step of the wall clock no longer moves the expiry request
56
+
57
+ A worker asks PostgreSQL to end an attempt when its deadline or its attempt timeout passes. Before
58
+ this change, the worker read the wall clock again after each wait on the lease window (SM-1869). If
59
+ the host clock stepped back 10 minutes, the first expiry request came 10 minutes late. If the clock
60
+ stepped forward, the request came early and got `not_due`. Each retry then locked the task row until
61
+ the real due time.
62
+
63
+ Now the worker converts the expiry to a monotonic due time once, when its expiry
64
+ watch starts. After that, it reads only the monotonic clock.
65
+
66
+ ### Row retention lag skips a member of an unfinished task group
67
+
68
+ Terminal cleanup keeps each member of a task group that has not finished, because the group needs
69
+ its member outcomes when it finishes. Before this change, queue health counted such a member as row
70
+ retention lag after it passed its windows (SM-1851). An open or sealed group can stay unfinished for
71
+ any length of time. Thus health reported "Retention cleanup is late for task records, finished
72
+ results", and the lag grew without end. The public demo keeps one open and one sealed group on
73
+ purpose, so its Health page showed the report every day.
74
+
75
+ Migration 0088 (`0088-count-only-prunable-rows-as-row-retention-lag.sql`) moves the schema to
76
+ version 87. Now `queue_health_v1` skips a member of an unfinished group when it measures task
77
+ identity and terminal outcome lag. For a group member, lag counts from the latest of window
78
+ expiry, history release, and group finish. The health document keeps its keys, and the schema floor
79
+ does not change.
80
+ Every process of every SDK version gets the new result when the migration commits.
81
+
82
+ ### A stale failure or release after a lapsed lease settles its cause
83
+
84
+ A lease can lapse while a handler runs, with no change to the fence. Then `fail_v1` and
85
+ `release_owned_v1` answer `stale`, but `expire_owned_v1` can still settle a due deadline or attempt
86
+ timeout under that fence. Before this change, the worker settled nothing, and lease recovery
87
+ settled the task later (SM-1884).
88
+
89
+ Now the worker calls `expire_owned_v1` at once and settles the due boundary, as the Go and Rust
90
+ workers do:
91
+
92
+ - After a lapsed lease with a due deadline, the handler outcome was `lease_lost`. Now it is
93
+ `deadline_exceeded`, and the task fails at once.
94
+ - After a lapsed lease with a due attempt timeout, the handler outcome was `lease_lost`. Now it is
95
+ `timeout`, and PostgreSQL applies the retry policy at once.
96
+ - After a lapsed lease with no due boundary, the outcome stays `lease_lost`.
97
+
98
+ The handler span records `deadline_exceeded` or `timeout_exceeded`. Before this change, it
99
+ recorded `stale`.
100
+
101
+ If `fail_v1` or `release_owned_v1` answers `cancel_requested` and `acknowledge_cancel_v1` refuses,
102
+ the worker settles nothing. After a failed cancellation acknowledgement, the handler outcome was
103
+ `unknown`. Now it is `lease_lost`. The handler span records `stale`. Before this change, it recorded
104
+ `unknown`. A late heartbeat round can also deliver the cancellation request, after a failure, a
105
+ release, or a refused completion. If the acknowledgement then refuses, the handler outcome was
106
+ `canceled`. Now it is `lease_lost`.
107
+
108
+ ### The dashboard bundle uses Mantine 9.7.4
109
+
110
+ The Rack dashboard backend serves the shared dashboard browser bundle. After 0.10.0, a dependency
111
+ update moved the dashboard to Mantine 9.7.4 and rebuilt that bundle (SM-1839). The gem now carries
112
+ the rebuilt `read-surface-1.tar.gz`. Its `bundle.json` records the sha256
113
+ `a7d61ed1d0433e38a8d227cbcc8f98a0bf668f2d3bb1d47391b1882fd69bf795`.
114
+
115
+ The `dashboard/v1` procedures and the read surface version do not change (SM-1858).
116
+
117
+ ### A worker no longer keeps the tier entries of a task that the executor refuses
118
+
119
+ The worker records each claimed task in its dispatch state. A fast-tier claim records the task ID
120
+ in `@fast_task_ids`. A full-tier claim records the task ID and a deferred tier read in
121
+ `@tier_reads`. Only the end of a handler run removed these entries.
122
+
123
+ Before this change, the handler executor could refuse a task, for example after its shutdown. The
124
+ task then never ran, and the worker kept its tier entry for the life of the `Worker` object
125
+ (SM-1853). Each refusal kept the entry of at most one task.
126
+
127
+ Now the worker removes the task from both collections when the executor refuses the task.
128
+
129
+ ### The dashboard page tells each cache not to store it
130
+
131
+ The dashboard page holds `window.workhorseDashboard`, which contains the `auditActor` of the
132
+ operator. Before this change, the Rack dashboard backend sent the page with only `Content-Type`. A
133
+ shared cache in front of the application could store the page by heuristic. A later visitor could
134
+ then get the stored page and see the actor name of an earlier operator. A browser could also show
135
+ the page from its cache after logout (SM-1870).
136
+
137
+ Now the Rack dashboard backend sends the page with `Cache-Control: no-store`, as the TypeScript host
138
+ does. Each asset keeps `Cache-Control: public, max-age=31536000, immutable`.
139
+
140
+ ### An idempotency TTL that rounds to 0 ms is refused
141
+
142
+ The queue rounds `Idempotency#ttl` to the nearest millisecond. Before this change, a positive TTL
143
+ below 0.0005 s, or a small negative TTL such as -0.0004 s, rounded to 0. The queue then kept the key
144
+ for the default 24 hours without a warning (SM-1857).
145
+
146
+ Now the queue raises `ArgumentError` for a TTL that is not 0 and rounds to 0 ms. An exact 0 still
147
+ selects the default. ADR 0115 records the rule for all seven SDKs.
148
+
149
+ ### A stalled registration refresh no longer delays a stop
150
+
151
+ A worker refreshes its `worker_registry` row at each registry interval. Before this change, the
152
+ dispatch loop sent that refresh itself, with no time limit. If the statement stalled, for example on
153
+ a lock, `stop` could not start the stop path until the statement returned. Also, the draining
154
+ registration and the deregistration ran on two threads in no order. If the registration committed
155
+ last, the row of the stopped worker came back until the registry prune removed it (SM-1868).
156
+
157
+ Now each registry write runs on its own thread, after the write before it. The dispatch loop starts
158
+ a periodic refresh only when no registry write still runs, and it does not wait for the refresh.
159
+ Thus `run` returns within `shutdown_grace` plus the unwind window, and the deregistration runs
160
+ after every registration write of the run.
161
+
162
+ ### Recovery no longer locks a timed-out attempt that waits for its cancellation
163
+
164
+ An operator cancels a running task after its execution timeout passed, and its lease is still live.
165
+ Before migration 0089, each maintenance tick locked that attempt in its timeout scan, but settled
166
+ nothing, because the cancellation wins over the timeout. While the tick ran, the heartbeat of the
167
+ owning worker reported the lease busy. The locked rows also used the recovery limit, so a real
168
+ timeout behind them waited for their leases to expire (SM-1866).
169
+
170
+ Now the full-tier timeout scan skips such an attempt while its lease is live and its deadline is
171
+ ahead, as the fast tier does. If the deadline passed after the timeout, recovery closes the task
172
+ `canceled` at once.
173
+
174
+ ### Fast-tier recovery closes a canceled attempt when its deadline passes
175
+
176
+ A fast-tier task has a pending cancellation and a live lease. Its execution timeout passes, and
177
+ later its deadline passes. Before migration 0090, each maintenance tick locked that attempt and
178
+ settled nothing until the lease expired. The cancellation blocks the timeout, and recovery took the
179
+ deadline only when the deadline was not later than the timeout (SM-1881).
180
+
181
+ Now fast-tier recovery closes the task `canceled` when its deadline passes, as the full tier does.
182
+
183
+ ### The dashboard Schedules page reads a bounded number of occurrences
184
+
185
+ Before migration 0089, `dashboard_cron_v1` counted every retained occurrence of each listed
186
+ schedule. A schedule that fires each second keeps about 1.2 million occurrences in the default
187
+ retention of 14 days. Each poll of the Schedules page read all of them (SM-1866).
188
+
189
+ Now the function reads at most 1,001 occurrences for each listed schedule. `occurrenceCount` stops
190
+ at 1,000, and the new optional key `occurrenceCountCapped` is `true` when more occurrences are
191
+ retained. The bundled Schedules page then shows `1,000+`. `lastFiredAt` is the fire time of the
192
+ newest occurrence. The `dashboard/v1` change is additive. A database below schema 88 returns an
193
+ exact count and no `occurrenceCountCapped`. The page change rebuilds the shared dashboard bundle
194
+ again. The sha256 in "The dashboard bundle uses Mantine 9.7.4" is the sha256 of that bundle.
195
+
196
+ ### The worker registry prune finds expired kept pauses through their index
197
+
198
+ Before migration 0089, `prune_worker_registry_v1` deleted expired kept pauses with a condition that
199
+ the planner could not match to `worker_pause_tombstone_expires_idx`. Each maintenance pass read the
200
+ whole table. Now the delete reads only the expired rows through the index (SM-1866).
201
+
202
+ ### A child set and a task group check every request before their first statement
203
+
204
+ Before this change, `HandlerContext#run_children` and `#run_children_all` checked the options of
205
+ each child while they serialized it. Thus the first child could read its payload contract before
206
+ the SDK refused the second child. `Queue#create_task_group` and `Queue#task_group` checked the
207
+ callbacks after the compatibility query and the member contract reads. A member add through
208
+ `HandlerContext#task_group` read the membership of its task first (SM-1883).
209
+
210
+ Now these calls check the options of every request before their first statement, also before the
211
+ deferred tier read of a child set. The errors do not change. Only their time changes.
212
+
213
+ ## 0.10.0 — 2026-10-26
214
+
215
+ The npm packages, Python distribution, Go module, Rust crate, Ruby gem, JVM artifacts, and .NET
216
+ packages release from one source commit.
217
+
218
+ Requires **schema v76**, Ruby **3.3** or newer, and PostgreSQL **15** or newer.
219
+
220
+ ### Upgrade steps
221
+
222
+ **Migrate the schema before starting updated processes.** Run `workhorse schema migrate` before any
223
+ process of this release starts, producers too. Migrations 0075 through 0087 move the schema to
224
+ version **86**. The SDK compatibility floor rises from 73 to **76**, because every SDK statement
225
+ catalogue now names `dashboard_retention_preview_v1` and `deregister_worker_v2`. An updated worker
226
+ refuses to start on a schema below version 76. A 0.9 process keeps working on version 86, because a
227
+ migration inside a major line only adds.
228
+
229
+ ### A failed reporter callback gives a catalogued log record
230
+
231
+ A raise from `on_registration_error` or `on_notification_error` gave a log line with the error
232
+ message and no event name (SM-1726). Now the line is the `workhorse.worker.reporter_failed` event
233
+ with `workhorse.worker.id`, `workhorse.worker.reporter` (`registration_error` or
234
+ `notification_error`), and `error.type`. It no longer carries the error message. The logger level,
235
+ the standard error fallback, and the bound do not change.
236
+
237
+ ### The worker refuses a `poll_interval` above 2,147,483.647 seconds
238
+
239
+ Before this change, `poll_interval:` had no upper limit (SM-1778). Now `Worker.new` raises
240
+ `ArgumentError` for a value above 2,147,483.647 seconds, the limit of the registry column. Every
241
+ SDK accepts an explicit poll interval from 1 through 2,147,483,647 ms (ADR 0112). Zero stays
242
+ refused, and `nil` keeps the default.
243
+
244
+ ### The Rack dashboard refuses a cross-origin read
245
+
246
+ A Rack CORS middleware can add CORS headers to `Stablemates::Workhorse::Dashboard` responses. If
247
+ that policy allowed credentials, a page on another origin could read the dashboard, task payloads
248
+ included, with the operator's cookies (SM-1675). Now the application answers 403 with
249
+ `{"error":"A same-origin request is required"}` for a page, an asset, or a read procedure whose
250
+ `Origin` header names another origin.
251
+
252
+ The same-origin check compared the `Host` text exactly, so `Host: example.test:443` with
253
+ `Origin: https://example.test` failed. Now letter case and the default port do not matter, as in
254
+ the allowed hosts check. An `Origin` with a path, a query, a fragment, or a user fails.
255
+
256
+ A request without `Origin`, such as a top-level navigation, and a same-origin request still
257
+ work. A mutation keeps its own refusal. The change needs no migration.
258
+
259
+ Upgrade step: if a proxy in front of the host ends TLS, the host must see the scheme that the
260
+ browser used. Otherwise the dashboard page's own module scripts carry an `Origin` that does not
261
+ match, and the host refuses them.
262
+
263
+ ### The dashboard Schedules page shows the last skip of a schedule
264
+
265
+ Migration 0087 (`0087-show-the-last-schedule-skip-on-the-dashboard.sql`) moves the schema to version 86. The `dashboard/v1` cron document gains `lastSkippedAt`, `lastSkippedCount`, and
266
+ `lastSkippedThrough` on each application schedule, an additive change. The bundled Schedules page
267
+ shows a `Skipped` badge with the count and the time of the last skip of a `skip` schedule (SM-1708).
268
+ The migration does not raise the SDK compatibility floor.
269
+
270
+ ### The worker counts each failed heartbeat round
271
+
272
+ A heartbeat round can fail before PostgreSQL answers, for example when the connection breaks or the
273
+ round reaches its deadline. Before this change, such a round left no metric (SM-1704).
274
+
275
+ Now the worker counts each failed round once in the new counter
276
+ `workhorse.worker.heartbeat.round_failure`, with the unit `{round}`. The bounded attribute
277
+ `workhorse.heartbeat.failure_reason` gives the reason: `timeout`, `connection`, `database`, or
278
+ `other`. This applies to the reserved heartbeat connection and to `shared_heartbeats: true`. A
279
+ failed round still gives no ownership verdict, so `workhorse.worker.heartbeat.failure` does not
280
+ change. The worker adds no log record. The new instrument is an addition.
281
+
282
+ ### A canceled task leaves its batch while it waits
283
+
284
+ A batch handler waits up to its linger for a partial batch. Before this change, a waiting task
285
+ stayed in the batch after a cancel request or a lease loss cancelled its cancellation token. The
286
+ batch handler then got a task that the worker no longer owned, and the task settled only after the
287
+ linger (SM-1709).
288
+
289
+ Now the worker removes such a task from the batch at once. The task raises the `CancelledError` of
290
+ the token reason, and the batch handler never gets it. The other waiting tasks keep their order and
291
+ dispatch. The TypeScript, Go, and Rust workers already remove such a task.
292
+
293
+ This includes the `:shutdown` cancellation at the end of the shutdown grace. Before, a task that
294
+ still waited then dispatched, and the batch handler ran its payload after the deadline. Now the task
295
+ goes back to its queue without a charged attempt. Before the deadline, the drain still dispatches a
296
+ lingering batch when all its tasks have arrived.
297
+
298
+ ### The worker waits longer after each not_due expiry answer
299
+
300
+ When the local timer of a deadline or an execution timeout fires, the worker asks PostgreSQL to
301
+ expire the attempt. If the clock of PostgreSQL has not reached its expiry, PostgreSQL answers
302
+ `not_due`. Before this change, the worker asked again every 5 ms. Each request locks the task row,
303
+ so the heartbeat of the same attempt answered `busy`. If the worker clock ran ahead by more than one
304
+ lease, the attempt could lose its lease before the database expiry (SM-1701).
305
+
306
+ Now the first retry waits 5 ms, and each later retry waits twice as long, to a maximum of 1 s. The
307
+ TypeScript, Go, and Rust workers already use this schedule.
308
+
309
+ ### The Ruby client measures signal and human wait values as PostgreSQL does
310
+
311
+ PostgreSQL limits a signal payload, a human wait context, and a human wait result to 65,536 bytes of
312
+ `octet_length(value::jsonb::text)`. That text has a space after each `:` and `,`, so it can be
313
+ longer than compact JSON. Before this change, the Ruby client counted the compact JSON. A value near
314
+ the limit passed that check, and then PostgreSQL refused the statement with a database error
315
+ (SM-1639).
316
+
317
+ Now `Queue#send_signal`, `Queue#complete_human_wait`, and `HandlerContext#wait_for_human` measure
318
+ the canonical JSONB text before any statement. An oversized value raises `ArgumentError`.
319
+ `HandlerContext#wait_for_human` also checks the context before its tier lookup, which can read
320
+ PostgreSQL. The limit does not change.
321
+
322
+ ### The worker warns when a maintenance pass or a claim waits longer than the stop budget
323
+
324
+ A maintenance pass or a claim can wait for a lock that another session holds. Before this change,
325
+ the worker logged nothing while it waited. A test run blocked for about one hour, and no log named
326
+ the statement (SM-1785).
327
+
328
+ Now `Worker` logs `workhorse.worker.statement_slow` once, at warn level, if a maintenance pass or a
329
+ claim waits longer than the stop budget. The stop budget is `shutdown_grace`, 25 s by default. A
330
+ grace of 0 turns the warning off. The record carries `workhorse.worker.id`,
331
+ `workhorse.worker.statement` (`maintenance` or `claim`), and `workhorse.worker.elapsed_ms`.
332
+ `workhorse.worker.elapsed_ms` is the wait at the moment of the warning. The statement continues to
333
+ wait, and no other behavior changes.
334
+
335
+ ### execution_timeout refuses a nonzero value that rounds to 0 ms
336
+
337
+ The queue rounds `execution_timeout:` to the nearest millisecond. Before this change, a positive
338
+ value below 0.0005 seconds rounded to 0, and the task had no timeout. A small negative value, such
339
+ as -0.0004, also set no timeout (SM-1714).
340
+
341
+ Now the queue raises `ArgumentError` for a nonzero value that rounds to 0 ms or is negative. `nil`
342
+ and an exact 0 still set no timeout. ADR 0114 records the rule for all seven SDKs.
343
+
344
+ ### The worker intervals refuse a value above the SQL integer limit
345
+
346
+ `register_worker_v1` takes `maintenance_interval`, `maintenance_routine_interval`, and
347
+ `registry_interval` as PostgreSQL `integer` milliseconds. Before this change, `Worker.new` accepted
348
+ a larger value, and then each registration failed (SM-1668).
349
+
350
+ Now `Worker.new` raises `ArgumentError` if one of these intervals is not from 0.1 through
351
+ 2,147,483.647 seconds. `disable_registry: true` still turns off registration. This is a behavior
352
+ change for a caller who passes a larger value. ADR 0112 records the range for all seven SDKs.
353
+
354
+ ### An anonymous error class records the name of its superclass
355
+
356
+ Ruby gives an anonymous class a nil name. Before this change, a handler that raised an instance of
357
+ `Class.new(StandardError)` recorded JSON null as the failure `name` (SM-1668).
358
+
359
+ Now the failure envelope records the name of the nearest named superclass, `StandardError` in that
360
+ example. Logs and telemetry use the same name for `error.type`.
361
+
362
+ ### retry_delay refuses a small negative value and rounds exact values exactly
363
+
364
+ `fail_v1` takes the retry delay as a PostgreSQL `integer`. Before this change, the worker rounded a
365
+ `retry_delay` to milliseconds first. Thus a negative value such as `-0.0004` became 0 and passed.
366
+ The worker also converted the value to a Float first, so `Rational(1001, 2000)`, which is exactly
367
+ 500.5 ms, became 500 ms (SM-1668).
368
+
369
+ Now the worker raises `ArgumentError` for every negative `retry_delay`, at construction for a fixed
370
+ value and before `fail_v1` for a callable result. It converts Integer, Rational, and BigDecimal
371
+ seconds exactly and rounds half up, so that value becomes 501 ms. ADR 0111 records the rule for all
372
+ seven SDKs.
373
+
374
+ ### A raising reporter callback no longer stops the worker
375
+
376
+ An application can give `on_registration_error` to the worker to send an alert. Before this change,
377
+ a registration refresh could fail, and then the callback could raise. The error left the registry
378
+ path and could stop the run. The worker also discarded a raise from `on_notification_error` without
379
+ a record (SM-1696).
380
+
381
+ Now the worker rescues a `StandardError` from either callback and continues. It writes the failure
382
+ to `logger` at the error level. Without a logger, it writes the failure to standard error. For each
383
+ callback, it writes the first failure. Then it writes at most one failure in each 60 s, with the
384
+ count of the failures that it did not write. It never calls the failed callback for its own failure.
385
+
386
+ ### Workers use the database clock for maintenance
387
+
388
+ Maintenance writes shared state: the due time of each routine and the retention cutoffs. Before
389
+ this change, the Ruby worker sent its host clock as the maintenance time. If that clock was two days
390
+ fast, other workers skipped a routine until that future time. Retention also deleted a 13-day-old
391
+ schedule occurrence under a 14-day policy (SM-1645).
392
+
393
+ Now `Worker#run` and `Worker#run_once` bind the text `now`, and PostgreSQL resolves it to its own
394
+ time at the start of the call.
395
+
396
+ The schema does not change.
397
+
398
+ ### The dashboard names an action that the host does not provide
399
+
400
+ A dashboard host answers the `enqueueTest` and `setSchedulePaused` procedures only when the
401
+ application gives a handler for them. Before this change, a writable host without that handler
402
+ answered `FORBIDDEN` with the message "This procedure is not available". Now the message is "This
403
+ dashboard host does not provide the <action> action", as in the TypeScript dashboard server. A
404
+ read-only host keeps the message "This dashboard is read-only". The code `FORBIDDEN` and the HTTP
405
+ status 403 do not change (SM-1705).
406
+
407
+ ### A lost notification subscription restarts the poll backoff
408
+
409
+ A worker without a live subscription waits 250 ms after an empty claim, and doubles that wait
410
+ toward 5,000 ms. Before this change, the empty claims of a live period counted toward that backoff.
411
+ After a loss, a worker that was idle for some time therefore waited the 5,000 ms ceiling at once,
412
+ and a new task waited up to 5 s (SM-1648).
413
+
414
+ Now the backoff restarts at 250 ms when the subscription is lost. The `subscription-recovery` case
415
+ in `protocol/v1/runtime.json` checks this rule in every SDK.
416
+
417
+ ### A polling-only worker keeps its listener connection for claims
418
+
419
+ A worker without a `cohorts` option caps its default cohort count by the connections that its pool
420
+ can lend. Before this change, a worker with `polling_only: true` also subtracted a connection for a
421
+ listener that it never opens. With a pool of 4 and concurrency 64, it used 2 cohorts instead of 3
422
+ (SM-1668).
423
+
424
+ Now a polling-only worker reserves no listener connection. A worker that listens still reserves one,
425
+ and a worker with dedicated heartbeats still reserves one more.
426
+
427
+ ### A handler error with text that jsonb cannot store settles by its retry policy
428
+
429
+ PostgreSQL `jsonb` cannot store a NUL character or an unpaired surrogate. Before this change, a
430
+ handler error with such text made `fail_v1` refuse the failure envelope. The worker stopped, and the
431
+ task stayed active until lease recovery (SM-1650).
432
+
433
+ Now `error_envelope` replaces each NUL character and each byte that is not UTF-8 with U+FFFD. It
434
+ does this in the name, the message, and the stack. The JVM and .NET workers use the same rule. The
435
+ retry policy then settles the task. All other text does not change.
436
+
437
+ The `protocol/v1/failures.json` fixtures pin the rule for every SDK.
438
+
439
+ ### A refused completion settles a due deadline or attempt timeout
440
+
441
+ A handler can return just before its deadline or attempt timeout, and the completion write can then
442
+ arrive after that boundary. PostgreSQL refuses the completion. Before this change, the worker only
443
+ tried to acknowledge a cancellation, then recorded `lease_expired` and left the task to lease
444
+ recovery. The task then settled up to one lease later with the wrong outcome. Now
445
+ `reconcile_rejected_completion` calls `expire_owned_telemetry_v1` under the attempt's fence before
446
+ the completion path returns. The attempt then ends with the outcome of the due boundary. A
447
+ cancellation request and a lost lease keep their own outcomes (SM-1649).
448
+
449
+ ### Workers on one pool renew their leases in one transaction
450
+
451
+ On the dedicated heartbeat connection, a round sent one transaction for each worker and each 100
452
+ leases. Each transaction needs at least 3 round trips to PostgreSQL. With 8 workers on one pool and
453
+ 80 ms for each round trip, workers lost leases while PostgreSQL still answered (SM-1631). Now the heartbeat thread of the pool sends one transaction for each round, with one `heartbeat_pool_v1` statement for every worker and every 100 leases. The
454
+ transaction locks the task rows of every worker in it until its `COMMIT`. A round renews all of its
455
+ leases or none. The settings statement now sets `statement_timeout` and
456
+ `idle_in_transaction_session_timeout` each to one quarter of the time that remains, not to one half
457
+ and one eighth. The statement needs no schema change. The `shared_heartbeats` opt-out does not change (SM-1633).
458
+
459
+ ### The dashboard sets the tier and the history of a queue
460
+
461
+ The Rack dashboard answers two new mutations, `setQueueTier` and `setQueueHistory`. `setQueueTier`
462
+ calls `Admin#set_queue_tier` with the authenticated actor and answers `{status: "set", tier}`. If
463
+ Workhorse refuses the switch, the dashboard answers the guard as the status: `live_tasks`,
464
+ `concurrency_policy`, or `rate_limit_policy`. Then `tier` is `null`, and the queue keeps its tier.
465
+ `setQueueHistory` calls `Admin#set_queue_history`. An absent setting keeps its current value. A
466
+ read-only dashboard refuses both mutations (SM-966).
467
+
468
+ ### `Admin` moves a queue between tiers and sets its history
469
+
470
+ `Admin#set_queue_tier(queue, tier, audit:)` moves a queue to the `:fast` or `:full` tier and returns
471
+ the tier after the change. If the tier differs and the queue holds a live task, the method raises
472
+ `FastTierUnsupportedError`.
473
+ `Admin#set_queue_history(queue, record_attempts: nil, record_claims: nil)` chooses the history a
474
+ fast-tier queue records. A `nil` setting keeps its current value. The method returns a
475
+ `QueueHistory`. The other SDKs already have these controls (SM-1417).
476
+
477
+ ### A worker on a pool of 1 polls under the opt-out
478
+
479
+ Before this change, a worker with `shared_heartbeats: true` started the shared listener on any pool.
480
+ On a pool of 1, the listener held the only connection. Registration, claims, and handler statements
481
+ then waited for the pool timeout, and no task ran (SM-1585).
482
+
483
+ Now, with `shared_heartbeats: true`, the worker starts the listener only when the pool `size` is an
484
+ Integer of at least 2. On a pool of 1, or on a pool without a `size`, the worker polls. Each run
485
+ then logs `workhorse.worker.polling_only` once at warn level. The message names
486
+ `workhorse.worker.queues` and `workhorse.worker.pool_max_size`. A worker with `polling_only: true`
487
+ logs no warning. Without the opt-out, nothing changes.
488
+
489
+ ### A contract compares a stored schema number with all its digits
490
+
491
+ Before this change, the Ruby SDK read a stored contract schema with `JSON.parse`, which reads a
492
+ fraction as a Float. Another SDK can store `"minimum": 0.30000000000000000001`, which a Float cannot
493
+ hold. The Ruby SDK then accepted `0.3`. Now a fraction that a Float cannot hold stays a Rational,
494
+ and the contract keywords compare it exactly (SM-1541, ADR 0039). `ContractSchema.new` still
495
+ accepts only JSON values.
496
+
497
+ ### A worker reserves its heartbeat connection at startup, and refuses to start without it
498
+
499
+ Before this change, a worker took the heartbeat connection of its pool only when a task arrived,
500
+ and returned it when no task remained. If the pool had no free connection then, the worker still
501
+ claimed tasks. The heartbeat waited for a connection, and the leases of those tasks could expire
502
+ (SM-1577). These changes apply to a worker without `shared_heartbeats: true`:
503
+
504
+ - **A startup reservation failure now ends the run.** This changes behavior. `run` and `run_once`
505
+ first reserve the heartbeat connection, then borrow one other connection for the compatibility
506
+ query and return it at once. Both steps must end within the larger of `heartbeat` and 5 s. This
507
+ bound includes a connect and the query itself, for every kind of pool. If a step does not end in
508
+ time, the run raises the new `Stablemates::Workhorse::WorkerStartupError` before the worker
509
+ registers. Its message names the pool, the budget, and the `shared_heartbeats: true` opt-out. A
510
+ compatibility query that fails keeps its own error type. Before this change, the same worker started and claimed tasks. A
511
+ supervisor that restarts a failed process, such as `run_worker_processes`, starts the worker
512
+ again.
513
+ - **The reservation lasts while a worker runs.** The heartbeat keeps its connection while any worker
514
+ on the pool runs, also when no task runs. The last worker that stops returns it. Thus a pool keeps
515
+ one heartbeat connection for its running workers, apart from the short replacement after a lost
516
+ connection.
517
+ - **No claim without a heartbeat connection.** While the pool has no usable heartbeat connection,
518
+ the worker claims no task. A claim checks this after it has its connection, immediately before
519
+ each attempt, and only for its own run. A fast-tier completion then settles its task with a claim
520
+ limit of 0. A rejected or busy lease does not stop claims. The heartbeat checks its connection
521
+ between rounds, also when no task runs. If the connection is lost, or a round fails, the heartbeat
522
+ asks for a new one at once. It waits for a checkout at most one round bound, and at most one
523
+ heartbeat interval between failed attempts. When the new connection is ready, the worker claims at
524
+ once.
525
+ - **One shutdown deadline.** `shutdown_grace` now starts when dispatch ends. Every wait of the stop
526
+ path ends by that deadline or one unwind window later. This includes the registration statements
527
+ and the release of the listener and the heartbeat connection. Thus a stop takes at most
528
+ `shutdown_grace` plus the unwind window. A claim that returns after the deadline starts no task,
529
+ and its leases expire. The heartbeat connection stays reserved until that claim statement
530
+ settles. A stop during the startup reservation ends the run without an error.
531
+ - **A new run waits for the stop statements of the previous run.** The next `run` or `run_once` of
532
+ the same worker waits for them within its startup budget. Then a late deregistration cannot
533
+ delete the registry row of the new run. If a statement still runs when the budget ends, the run
534
+ raises `WorkerStartupError`. A stop during this wait ends the run without an error.
535
+
536
+ ### Throttle works on a fast-tier queue
537
+
538
+ Before this change, a throttled enqueue into a fast-tier queue raised the fast-tier error with the feature
539
+ `throttle`. Throttle only decides if Workhorse accepts an enqueue request, so it needs no state of
540
+ the full tier (SM-928, ADR 0077). Migration 0075
541
+ (`0075-allow-throttle-on-fast-tier-queues.sql`) moves the schema to version 74. After it,
542
+ `enqueue_throttle_v1` and `enqueue_many_v1` accept throttle on a fast-tier queue. The outcomes are
543
+ `accepted` and `coalesced`, as on the full tier.
544
+
545
+ A coalesced request on a fast-tier task writes no `throttled` event, because a fast-tier task
546
+ records only the history that its queue enables. The SDK still emits the throttle log and the
547
+ enqueue outcome metric. In a batch with a debounce or a throttle member, a fast-tier rejection of
548
+ one member now reports the position of that member in the batch. Debounce on a fast-tier queue is
549
+ still rejected.
550
+
551
+ The migration does not raise the SDK compatibility floor. No SDK statement changes. The migration changes no stored data.
552
+
553
+ ### The retention preview reads through a dashboard function
554
+
555
+ Before this change, each dashboard host read the retention policy, sent a shared statement that
556
+ counted rows directly in the tables, and capped the counts in its own code. ADR 0040 asks for one
557
+ SQL implementation for each dashboard procedure (SM-1478). Migration 0076
558
+ (`0076-add-the-dashboard-retention-preview-function.sql`) moves the schema to version 75 and adds
559
+ `dashboard_retention_preview_v1`. The Ruby dashboard host now calls it. The `previewRetentionPolicy` response does
560
+ not change.
561
+
562
+ Every SDK statement catalogue names the new function, so the SDK compatibility floor rises from
563
+ schema 73 to 75. Run `workhorse schema migrate` before the first process of this release starts.
564
+ A 0.9 process keeps working on schema 75. The migration changes no stored data.
565
+
566
+ ### A late worker deregistration cannot delete the row of a later run
567
+
568
+ Before this change, a worker that stopped sent `deregister_worker_v1`, which deletes the registry
569
+ row by worker ID only. If the stop path was slow and the same worker ID started again, the late
570
+ deregistration could delete the row of the new run and its operator pause state. Each SDK guarded
571
+ against this in its own code (SM-1589). Migration
572
+ 0077 (`0077-fence-the-worker-deregistration-by-instance.sql`) moves the schema to version 76 and
573
+ adds `deregister_worker_v2`. It deletes the row only while the row names the instance of the run
574
+ that stops. `deregister_worker_v1` does not change.
575
+
576
+ `Stablemates::Workhorse::Worker` now calls `deregister_worker_v2` with the instance of the run that
577
+ stops.
578
+
579
+ Each SDK keeps its guard for this release: a new run still waits for the stop statements of the
580
+ previous run, or it refuses to start. Every SDK statement catalogue names the new function, so the
581
+ SDK compatibility floor rises from schema 75 to 76. Run `workhorse schema migrate` before the first
582
+ process of this release starts. A 0.9 process keeps working on schema 76. The migration changes no
583
+ stored data.
584
+
6
585
  ## 0.9.0 — 2026-10-20
7
586
 
8
587
  The npm packages, Python distribution, Go module, Rust crate, Ruby gem, and JVM artifacts release from one source commit.
data/README.md CHANGED
@@ -30,7 +30,7 @@ bundle add connection_pool
30
30
  Install the schema once, as a deployment step. The application never installs or migrates it.
31
31
 
32
32
  ```bash
33
- npx --package @stablemates/workhorse@0.9.0 workhorse schema install
33
+ npx --package @stablemates/workhorse@0.11.0 workhorse schema install
34
34
  ```
35
35
 
36
36
  The machine that runs that deployment step needs Node.js 22 or newer. The application itself needs
@@ -108,7 +108,8 @@ The gem is `stablemates-workhorse`, and its namespace is `Stablemates::Workhorse
108
108
  that was `:non_replaceable` also carries a `reason`: `:incompatible_key_mode`, `:not_pending`, or
109
109
  `:window_elapsed_pending`.
110
110
  - `Worker` takes a pool, registers handlers by task type, and runs them under a lease with a shared
111
- heartbeat connection. `run_worker_process` drains it on `TERM` or `INT`.
111
+ heartbeat connection. The pool must report a maximum size of at least 3 connections, or the
112
+ constructor raises `ArgumentError`. `run_worker_process` drains the worker on `TERM` or `INT`.
112
113
  - `HandlerContext` offers checkpoints, durable sleeps, signal and human waits, child tasks, and
113
114
  progress. PostgreSQL owns every durable decision, and an unresolved wait suspends the task.
114
115
  - `Admin` lists, inspects, and repairs tasks, dead letters, waits, workers, and queues.