durability 4.3.0 → 5.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Composable capabilities for application-owned Cloudflare Durable Objects:
4
4
 
5
- - **`Durability`** — effectively-once operations with retries, idempotency, and result lookup.
5
+ - **`Durability`** — at-least-once operations with stable idempotency keys, retries, and result lookup.
6
6
  - **`DurabilityAlarms`** — named logical alarms with retries and stable idempotency keys.
7
7
  - **`DurabilityFanout`** — durable batch acceptance and independent delivery to static consumer or storage targets.
8
8
  - **`DurabilityOutbox`** _(experimental, `durability/experimental`)_ — keyed latest-state publication: stage the newest value for a key with the state it describes, and it is republished until acknowledged.
@@ -14,7 +14,47 @@ Composable capabilities for application-owned Cloudflare Durable Objects:
14
14
 
15
15
  Durable capabilities own their storage and migrations. Routing observations are in-memory and advisory. When an object composes durable capabilities, one `DurabilityScheduler` shares its physical alarm and concurrency pool. No queue factory, generated DO class, or special consumer superclass is required.
16
16
 
17
- Durability is not an event-sourcing, replay-log, or workflow-orchestration engine. Operation methods return `Promise<void>` after registration commits; they do not wait for the handler result. Failed work is retried from later alarms. Only SQLite-backed Durable Object classes are supported.
17
+ Durability is not an event-sourcing, replay-log, or workflow-orchestration engine: [memoized](#memoize) results only last while one operation or alarm occurrence is retried, and are deleted when it settles. Operation methods return `Promise<void>` after registration commits; they do not wait for the handler result. Failed work is retried from later alarms. Only SQLite-backed Durable Object classes are supported.
18
+
19
+ ## Delivery and safety contract
20
+
21
+ Durability executes work **at least once** with a stable idempotency key per operation or named alarm. "Effectively once" is a business outcome you earn by making the receiver deduplicate that key or reconcile an uncertain prior outcome; it is never an unconditional platform guarantee. Read this before copying an example that mutates an external system.
22
+
23
+ | Concern | Guarantee |
24
+ | -------------------------- | ---------------------------------------------------------------------------- |
25
+ | Registration / scheduling | Exactly once — the durable record commits atomically or not at all. |
26
+ | Local completion record | Deduplicated by the stable key, so retries never double-record a result. |
27
+ | Handler delivery | At least once — a handler may run again after a crash, timeout, or eviction. |
28
+ | External side effect | At least once — the library cannot close a cross-system transaction. |
29
+ | "Effectively once" outcome | Your responsibility — requires an idempotent or reconciling receiver. |
30
+
31
+ The unavoidable window: a handler asks an external system to commit an effect, the system commits it, then the acknowledgement is lost or the object stops before the local completion record commits. Durability retries the same stable key. Pass that key (`DurableCall.id`, or an alarm handler's `idempotencyKey`) to the external system so it can collapse the duplicate. An HTTP success — or failure — is not proof of the committed outcome after a lost acknowledgement.
32
+
33
+ The stable key is the call `id` you choose (or an alarm name). Namespace it per logical operation (`resize:${imageId}`), keep it stable across retries, and never reuse one key for different payloads: an ID stays bound to its operation and payload for the life of the record. A receiver-side idempotency table turns at-least-once delivery into an effectively-once effect by keying committed effects on it:
34
+
35
+ ```ts
36
+ // In the receiving Durable Object, keyed on the caller's stable id.
37
+ async chargeOnce(key: string, amountCents: number) {
38
+ const prior = this.ctx.storage.sql
39
+ .exec<{ result: string }>('SELECT result FROM charges WHERE key = ?', key)
40
+ .toArray();
41
+ if (prior.length > 0) {
42
+ return JSON.parse(prior[0]!.result); // replay: return the first committed outcome
43
+ }
44
+
45
+ const result = await this.gateway.charge(amountCents);
46
+ this.ctx.storage.sql.exec(
47
+ 'INSERT INTO charges (key, result) VALUES (?, ?)',
48
+ key,
49
+ JSON.stringify(result)
50
+ );
51
+ return result;
52
+ }
53
+ ```
54
+
55
+ The sketch assumes one call per key at a time; the gateway `await` lets other requests interleave, so concurrent calls for the same key need a pending row or the gateway's own idempotency key.
56
+
57
+ When a receiver can neither deduplicate the key nor reconcile the prior outcome, do not retry the effect automatically: compensate (a saga) or require a human decision instead.
18
58
 
19
59
  ## Migrating to v3
20
60
 
@@ -128,7 +168,7 @@ Scheduling a name again replaces its pending occurrence. If that name is already
128
168
 
129
169
  Named alarms inside the scheduler's `alarmMinDelayMs` window start from an in-memory timer at their requested time. The option accepts 1 to 15 seconds and defaults to 15 seconds; the scheduler never sets the physical alarm sooner than that delay; the physical alarm remains as a durable fallback for restarts. If concurrency capacity is full when the timer fires, execution waits in the shared FIFO queue. The scheduling method resolves after persistence and never waits for the handler to finish.
130
170
 
131
- Each occurrence receives a new internal generation. Its `idempotencyKey` stays stable across retries while `attempt` increments. A successful handler removes only the occurrence it executed, so it cannot delete a replacement scheduled while it was running.
171
+ Each occurrence receives a new internal generation. Its `idempotencyKey` stays stable across retries, while `attempt` records which attempt the handler is running. The ordinal advances each time an attempt is claimed and is retry context, not a unique per-handler-entry token (see [Identity and attempt numbers](#identity-and-attempt-numbers)). A successful handler removes only the occurrence it executed, so it cannot delete a replacement scheduled while it was running.
132
172
 
133
173
  A timeout aborts the handler's `signal` and is terminal by default because the external outcome is unknown. Set `retryTimeouts: true` only when the handler's side effects use the idempotency key or reconcile their outcome before retrying. Named alarms remain at-least-once across eviction or restart.
134
174
 
@@ -167,6 +207,36 @@ export class ImageJobs extends DurableObject<Env> {
167
207
 
168
208
  A helper configured with `context` instead of `scheduler` creates a private scheduler and accepts the scheduler options (`alarmConcurrency`, `alarmMinDelayMs`, `alarmHandoffMs`) in the same object. Two helpers on the same object must share one scheduler; otherwise each would reconcile the physical alarm against only its own records.
169
209
 
210
+ ## Memoize
211
+
212
+ A retried attempt runs the whole handler again. Wrap work that must not repeat in `memoize(key, callback)`, which operation handlers, named alarm handlers, and [fanout messages](#independent-delivery) all receive. Once a key's callback succeeds, its result is persisted and every later call with that key returns it instead of running the callback:
213
+
214
+ ```ts
215
+ const deliver = async ({ payload, memoize }: DurableCall<Batch>) => {
216
+ const targets = await memoize('targets', () => lookUpTargets(payload));
217
+ const results = await Promise.allSettled(
218
+ targets.map((target) =>
219
+ memoize(`send:${target.id}`, () => send(target, payload.events))
220
+ )
221
+ );
222
+ const failed = results.filter((result) => result.status === 'rejected');
223
+ if (failed.length > 0) {
224
+ throw new AggregateError(failed.map((result) => result.reason));
225
+ }
226
+ };
227
+ ```
228
+
229
+ Here a retry sends only to the targets that failed, and does not look them up again.
230
+
231
+ - **Keys.** A key is a non-empty string that identifies one piece of work within the operation or alarm occurrence. Calling a key again returns its result, whether the call comes from a later attempt, later in the same attempt, or from inside another key's callback. A call for a key whose callback is still running rejects instead of waiting, including one made from that callback, which would otherwise wait on itself; await the first call instead.
232
+ - **Results.** A callback that throws persists nothing; the next call with that key runs it again. Results must be JSON-serializable, and `memoize` returns the persisted form even on the first run, so a replay sees the same value. A result that cannot be serialized fails the work with `DurableMemoSerializationError` and no retry.
233
+ - **Ownership.** Only the attempt that owns the record can run or persist a callback. Once its alarm is rescheduled or its record purged, a `memoize` call rejects without running, and one already running rejects instead of writing.
234
+ - **Limits.** One operation, alarm occurrence, or fanout delivery may memoize at most `maxMemosPerRecord` (1,000) keys, and each serialized result may be at most `maxMemoBytes` (1 MiB) of UTF-8. A call that would exceed either fails the work with `DurableMemoLimitError` and no retry; a key that would exceed the count is rejected before its callback runs.
235
+ - **Cleanup.** Memoized results are deleted in the transaction that settles the operation (completed or failed), the alarm occurrence, or the fanout delivery (acked or dead-lettered), when a named alarm is rescheduled, and by `purgeBefore`. Nothing is kept once the work is done.
236
+ - **Tracking.** Pending operations and alarms list their memoized keys as `state.memoized` in [`DurabilityTracker`](#tracking-work-experimental) records, in the order they were saved. Results themselves are not exposed.
237
+
238
+ Memoizing does not change delivery semantics. A process can stop after a callback's side effect succeeds but before its result commits, so the callback can run again; keep its side effects idempotent.
239
+
170
240
  ## Storage
171
241
 
172
242
  Durability stores its records in the object's SQLite database. Configure the Durable Object class with `new_sqlite_classes`:
@@ -177,7 +247,7 @@ Durability stores its records in the object's SQLite database. Configure the Dur
177
247
  }
178
248
  ```
179
249
 
180
- `Durability` creates and migrates `durability_calls`; `DurabilityAlarms` does the same for `durability_alarms`. Both histories are tracked in the shared `durability_migrations` table and only the constructed helpers' tables exist.
250
+ `Durability` creates and migrates `durability_calls`; `DurabilityAlarms` does the same for `durability_alarms`. Both also create `durability_memos`, which they share. All histories are tracked in the shared `durability_migrations` table and only the constructed helpers' tables exist. Reverting either helper's schema leaves `durability_memos` in place for the other.
181
251
 
182
252
  ### Explicit migrations
183
253
 
@@ -252,14 +322,48 @@ Each attempt receives its own `AbortSignal`. A timed-out operation follows the n
252
322
 
253
323
  Throw `NonRetryableError` to move work directly to `failed` without another attempt. Errors named `NonRetryableError` from other packages, such as `cloudflare:workflows`, are recognized too.
254
324
 
325
+ ## Identity and attempt numbers
326
+
327
+ A handler receives several identifiers; only some are stable:
328
+
329
+ - **Operation `id`** (operations) and **`idempotencyKey`** (named alarms) are the logical identity. They are stable across every attempt and retry, so they are the only safe deduplication keys for downstream services.
330
+ - **`operation` / `name`** is the handler the work was registered under.
331
+ - **`attempt`** is the one-based ordinal of the current attempt, claimed in a durable write before the handler runs. It is retry/logging context, not an execution identity.
332
+ - **Named-alarm generation** is an internal token minted per scheduled occurrence; it backs `idempotencyKey` and is not exposed directly.
333
+
334
+ `attempt` is operational context for retry policies and logging, not a record of what the handler did. It is not strictly unique per handler entry: an entry whose claim write never commits can see the same number again on recovery. Cloudflare's output gates hold that entry's outbound fetches and RPCs until its storage writes are confirmed and discard them if the write is lost, so a replayed ordinal never corresponds to an escaped external effect — but it is still not a reliable "what already happened" signal.
335
+
336
+ That is why `attempt === 1` is not a durable "first run" gate. Attempt 1 can fail or time out before or after its effect, and retries carry a higher ordinal, so gating on the number either skips the effect forever once it increments or cannot tell whether the effect already ran. Gate once-only work on durable state keyed by the stable `id` instead, preferably in the authoritative receiver:
337
+
338
+ ```ts
339
+ // Wrong: attempt is not a record of whether the effect ran. If attempt 1
340
+ // fails after charging, the retry (attempt 2) skips the charge; if it fails
341
+ // before, nothing distinguishes that from success.
342
+ if (attempt === 1) {
343
+ await chargeCard(payload);
344
+ }
345
+
346
+ // Right: the receiver dedupes on the stable id, independent of attempt.
347
+ await this.env.BILLING.chargeOnce(id, payload);
348
+ ```
349
+
255
350
  ## Delivery semantics and concurrency
256
351
 
257
- Work is delivered at least once. A process can stop after a side effect succeeds but before its completion record commits, so handlers should be idempotent and pass the call ID or alarm `idempotencyKey` to external services.
352
+ Work is delivered at least once, as the [delivery and safety contract](#delivery-and-safety-contract) describes. A process can stop after a side effect succeeds but before its completion record commits, so handlers should be idempotent and pass the call ID or alarm `idempotencyKey` to external services.
258
353
 
259
354
  `alarmConcurrency` defaults to 10 and is one FIFO limit shared by eager operation handlers, timer-driven work, and physical-alarm work attached to the same scheduler. Newly registered operations start eagerly only when a permit is immediately available. Work due before the physical-alarm minimum enters the same queue when its in-memory timer fires.
260
355
 
261
356
  Completion records are retained indefinitely so IDs remain deduplicated. The scheduler owns the Durable Object's alarm; compose unrelated scheduled work as named alarms instead of replacing the physical alarm.
262
357
 
358
+ ## Load envelope
359
+
360
+ `pnpm bench:load` runs an opt-in Workerd characterization suite (`bench/`, excluded from `pnpm test`) that drains large backlogs, a zero-delay retry storm, and a mixed operation plus named-alarm workload, emitting one machine-readable `BENCH {...}` JSON line per scenario. The numbers below come from that suite on `compatibility_date` `2026-07-07` with the default `alarmConcurrency` of 10; treat them as a tested envelope on one machine, not a platform capacity promise.
361
+
362
+ - A live object drains its backlog eagerly and through in-memory timers, so 100, 1,000, and 10,000 simultaneously due operations all complete with no lost calls, no extra terminal transitions, and nothing left pending. The physical alarm is a post-eviction fallback, not the live drain path.
363
+ - Each retained completed row costs roughly 200–260 bytes plus the serialized payload and result; 10,000 empty-payload operations occupy about 2 MB of `databaseSize`. Rows are never reclaimed automatically, so storage grows linearly until `purgeBefore` runs.
364
+ - `alarmConcurrency` is one shared budget: a mixed backlog of overlapping operations and a named alarm never exceeds 10 concurrent handlers in total, rather than 10 per category.
365
+ - A zero-delay retry storm performs exactly one attempt per retry with no extra work and settles every operation.
366
+
263
367
  ## Destructive retention purge
264
368
 
265
369
  `purgeBefore(timestamp)` on either helper deletes that helper's records created strictly before `timestamp`, regardless of status, and returns the count. Records created exactly at the cutoff are retained.
@@ -271,6 +375,104 @@ const removedAlarms = await this.alarms.purgeBefore(retentionCutoff);
271
375
 
272
376
  The purge runs in one transaction. It best-effort aborts matching active handlers, but cannot force an abort-ignoring handler to stop. Operation IDs can be reused immediately; a late old generation cannot overwrite the replacement record.
273
377
 
378
+ ## Experimental options
379
+
380
+ These options, methods, and capabilities are opt-in: none takes effect unless you set, call, or construct it, so leaving them alone keeps every behavior described above. Their names and behavior may change in a minor release. Scheduler options also apply to the private scheduler of a helper configured with `context`.
381
+
382
+ ```ts
383
+ const scheduler = new DurabilityScheduler({
384
+ context: this.ctx,
385
+ liveTimers: true,
386
+ });
387
+
388
+ const alarms = new DurabilityAlarms({
389
+ scheduler,
390
+ handlers: { tick: async () => this.enqueueDueRuns() },
391
+ });
392
+
393
+ await alarms.tick.cancel();
394
+ ```
395
+
396
+ ### `liveTimers` (scheduler)
397
+
398
+ `liveTimers?: boolean`. Due work starts from an in-memory timer at its exact scheduled time, however far beyond `alarmMinDelayMs`, and an alarm pass skips records that are already running instead of waiting for them, so a long handler no longer delays newly due work. The physical alarm stays armed at the usual bounded delay as the eviction fallback, and the handoff before the alarm wall-time limit is unchanged.
399
+
400
+ Gives up: every settled attempt runs one more reconciliation transaction to re-arm the timer, and timers do not survive eviction, so after one the work runs only when the physical alarm fires. Attempts interrupted by the eviction are counted as usual, so an operation with `maxAttempts: 1` fails instead of running again.
401
+
402
+ ### `cancel` (named alarm method)
403
+
404
+ `alarms.<name>.cancel(): Promise<boolean>`. This deletes the alarm's pending occurrence and its memos and reconciles the physical alarm in one transaction, then aborts the `signal` of a running attempt of it. That attempt's settlement can no longer retry it. It resolves `true` when a pending occurrence was deleted. Failed occurrences are left for `purgeBefore`.
405
+
406
+ Gives up: cancelling does not stop later scheduling. A call to the scheduling method after `cancel`, including one from the running handler before it returns, schedules a new occurrence.
407
+
408
+ ### `onAlarmRetry` (scheduler)
409
+
410
+ `onAlarmRetry?: (info: AlarmRetryInfo) => void | Promise<void>`, where `info` is `{ error: { name, message }, retryInMs, retryCount?, isRetry?, objectId? }`. It is called each time an alarm run fails and the scheduler re-arms its own fallback alarm (see [When a run cannot finish](#when-a-run-cannot-finish)), so the error can go to an external service. `retryInMs` is `alarmMinDelayMs`; `retryCount` and `isRetry` come from the platform's `AlarmInvocationInfo` when the invocation carried one, and `objectId` from the context.
411
+
412
+ The `durability.alarm.failed` report still happens. The hook is best effort: it runs after the fallback is armed, a returned promise is passed to `waitUntil`, and a throw or rejection is only reported as `durability.alarm.retry_hook_failed`, so it cannot change scheduling. It is not called when even the fallback cannot be armed, because the failure then reaches the platform.
413
+
414
+ ```ts
415
+ const scheduler = new DurabilityScheduler({
416
+ context: this.ctx,
417
+ onAlarmRetry: (info) => this.env.ERRORS.send(info),
418
+ });
419
+ ```
420
+
421
+ ### `DurabilitySchedule`
422
+
423
+ `DurabilitySchedule` is exported from `durability/experimental`. It runs one recurring schedule per Durable Object: each tick starts a run, runs overlap, every run is tracked on its own, and its outcome is reported once and then forgotten. It replaces a `DurabilityAlarms` tick that registers a `Durability` operation per run, which needs retained records to deduplicate and keeps them until purged.
424
+
425
+ ```ts
426
+ import { DurableObject } from 'cloudflare:workers';
427
+ import { DurabilityScheduler } from 'durability';
428
+ import { DurabilitySchedule } from 'durability/experimental';
429
+ import { nextCronTime } from './cron';
430
+
431
+ export class Cron extends DurableObject<Env> {
432
+ private readonly scheduler = new DurabilityScheduler({ context: this.ctx });
433
+ private readonly schedule = new DurabilitySchedule({
434
+ scheduler: this.scheduler,
435
+ // Durability ships no cron parser: return the first tick after `after`.
436
+ nextOccurrence: (expression, after) => nextCronTime(expression, after),
437
+ run: ({ scheduledTime, idempotencyKey, signal }) =>
438
+ this.env.TARGET.fire({ scheduledTime, idempotencyKey }, { signal }),
439
+ onOutcome: (run, outcome) => this.env.TARGET.report(run, outcome),
440
+ maxConcurrentRuns: 20,
441
+ runTimeoutMs: 15 * 60_000,
442
+ });
443
+
444
+ configure(expression: string, version: number) {
445
+ // Spread objects sharing an expression over the first 30 seconds.
446
+ return this.schedule.set({
447
+ expression,
448
+ version,
449
+ jitterMs: { maxMs: 30_000 },
450
+ });
451
+ }
452
+
453
+ remove(version: number) {
454
+ return this.schedule.cancel(version);
455
+ }
456
+
457
+ override alarm(info?: AlarmInvocationInfo) {
458
+ return this.scheduler.alarm(info);
459
+ }
460
+ }
461
+ ```
462
+
463
+ - **Versions.** `set({ expression, version })` stores the schedule and its next tick only when `version` is higher than the stored one, and resolves `true` when it did. A lower version is ignored. The same version changes nothing but reconciles the physical alarm, so resending it repairs a lost alarm. The expression is opaque; only `nextOccurrence(expression, after)` reads it, and it must return a safe integer after `after`, or `undefined` to stop ticking. `get()` returns `{ expression, version, jitterMs, nextAt }`, where `nextAt` is the next tick's nominal time, or `undefined` when no schedule is live.
464
+ - **Jitter.** `set({ expression, version, jitterMs })` delays every tick by one fixed offset, so many objects sharing a popular expression do not all fire at the same instant. `jitterMs` is either a non-negative integer, the offset itself, or `{ maxMs }`, which picks an offset uniformly in `[0, maxMs]` when the version is stored. It defaults to 0, and both forms are at most 2^31 - 1 ms. The offset is stored in the schedule's row, so it costs no write per run, and resending the same version keeps it; only a new version picks a new one. A tick's nominal time `scheduledTime` is what `nextOccurrence` returned, and it fires at `dueAt`, which is `scheduledTime + jitterMs`. `idempotencyKey` derives from `scheduledTime`, and the next tick is always searched from nominal times, so the offset never drifts. Keep `jitterMs` below the expression's interval; durability cannot know the interval of an opaque expression. A larger offset is allowed: every tick still fires once, in nominal order, each `jitterMs` after its nominal time, even when that is after the next tick's nominal time.
465
+ - **Cancel.** `cancel(version?)` stops future ticks and resolves `true` when a schedule was live. Runs already due or running still finish and report. With a version, the cancel is ignored when the stored version is higher, and is otherwise kept as a one-row tombstone so a delayed `set` of that version or lower cannot revive the schedule.
466
+ - **One tick, one run.** A tick inserts its run in the transaction that advances the stored next tick, so a replayed or duplicated alarm finds the cursor already moved and cannot fire the tick again; deduplication needs no finished-run records. When ticks were missed, for example while the object was down, they collapse into one run for the earliest missed time, and the next tick is the first whose `dueAt` is after now. A `nextOccurrence` that throws or returns an invalid time rolls the tick back and fails the alarm run, which is retried at `alarmMinDelayMs` and reported through `onAlarmRetry`.
467
+ - **Runs.** `run({ scheduledTime, dueAt, idempotencyKey, signal })` starts outside the alarm pass that recorded it, so a long run never delays later ticks. `idempotencyKey` is `<objectId>:<scheduledTime>`. At most `maxConcurrentRuns` (default 10) are in flight; later runs wait durably in tick order. This limit is separate from the scheduler's `alarmConcurrency`. A run that throws, or exceeds `runTimeoutMs` (default 15 minutes, which aborts its `signal`), is `errored`. A run is attempted once and never retried. A timed-out run keeps its slot until its handler actually settles.
468
+ - **Keeping the object alive.** While runs or outcome deliveries are in flight, the alarm pass stays pending, so an alarm invocation stays open and holds the object in memory; it fires the next ticks and outcome retries itself, at their exact times. Before the wall-time limit the scheduler hands off at `alarmHandoffMs`, and the next alarm joins the same loop. A tick started by an in-memory timer is joined by the next physical alarm. Other helpers on the same scheduler still run on each alarm pass, but work of theirs that becomes due during the hold waits for it unless the scheduler uses `liveTimers`.
469
+ - **Interrupted runs.** A run still marked running when no live instance holds it, for example after the object was evicted, reset, or redeployed mid-run, is reported as `errored` with `ScheduledRunInterruptedError` on the next alarm, and is not run again. While a run or outcome delivery is in flight, the physical alarm stays on the next tick, which is its eviction fallback, so an interrupted run is reported by the next tick at the latest, or within `alarmMinDelayMs` when no tick is left.
470
+ - **Outcomes.** `onOutcome(run, outcome)` receives `{ scheduledTime, dueAt, idempotencyKey, attempt }` and `{ status: 'complete' }` or `{ status: 'errored', error: { name, message } }`. The outcome is persisted before the first delivery, so delivery is at least once with the same outcome, even across a crash. Returning acknowledges it and deletes the run's row; nothing about the run is kept afterwards. Throwing, or exceeding `outcomeTimeoutMs` (default 60 seconds), retries it with `outcomeRetries.delay(attempt)` (default jittered exponential capped at 5 minutes) up to `outcomeRetries.maxAttempts` (default 10) deliveries. After the last, the row is deleted, `durability.schedule.outcome_dead_lettered` is reported, and a `dead_lettered` lifecycle event is emitted.
471
+ - **Storage.** Two rowid tables with no secondary indexes: `durability_schedule` holds the schedule's one row and `durability_schedule_runs` holds one small row per run from its tick until its outcome is delivered. A run writes four rows: the tick's cursor update and run insert, the outcome update, and the delete on delivery. It also arms the physical alarm once, for the next tick, and each `setAlarm()` is billed as a row written. A run that waits for a slot writes one more row, and each failed delivery one more.
472
+ - **Lifecycle events.** `onLifecycleEvent` receives `scheduled_run` events keyed by `idempotencyKey`: `due` (with `queued`), `started`, `settled` (with `outcome`, `error`, and `durationMs`), `outcome_retry`, and `dead_lettered`. Like every lifecycle hook, it is best effort.
473
+
474
+ One schedule attaches per scheduler; attaching a second throws. Construct it once per object instance, as with every helper. `DurabilitySchedule.migrate(this.ctx, null)` removes its schema.
475
+
274
476
  ## Lifecycle metrics hook
275
477
 
276
478
  Each helper accepts `onLifecycleEvent`, which receives compact events for registration or scheduling, attempt start and settlement, pre-invocation attempt exhaustion, and purges:
@@ -285,11 +487,28 @@ const durability = new Durability({
285
487
 
286
488
  The hook is best-effort and non-blocking, is not a durable journal, and may duplicate or drop events across crashes. Hook failures are reported to `console.error` and never change queue state. Returned promises are attached to `waitUntil` when the context exposes it.
287
489
 
288
- Further hooks attach after construction with `lifecycle(hook)`, which returns a function that detaches it. `Durability` hooks are typed per operation: a `completed` settlement carries the handler's `result`.
490
+ Events are built to be safe to forward to shared telemetry sinks: they carry structural fields (entity kind, operation or alarm name, idempotency `id`, `generation`, `attempt`, `durationMs`, `nextAttemptAt`) and a normalized `error` of `{ name, message }` with no stack, but **no payloads and no handler results by default**. The `id` is the raw idempotency key you chose, so hash or drop it before exporting to a metrics backend if it is high-cardinality or sensitive. Because an event is emitted per attempt, retry, and purge, treat these as counters and durations rather than an audit trail.
491
+
492
+ To include a `completed` settlement's handler `result` — for example to bridge results to a WebSocket or cache — opt in with `includeResults: true`. Enable it only when every attached hook is trusted with result content, since results may contain sensitive data:
493
+
494
+ ```ts
495
+ const durability = new Durability({
496
+ context: this.ctx,
497
+ handlers,
498
+ includeResults: true,
499
+ onLifecycleEvent: (event) => recordDurabilityMetric(event),
500
+ });
501
+ ```
502
+
503
+ Further hooks attach after construction with `lifecycle(hook)`, which returns a function that detaches it. `Durability` hooks are typed per operation: a `completed` settlement carries the handler's `result` only when `includeResults` is enabled, so the field is typed as optional.
289
504
 
290
505
  ```ts
291
506
  const detach = durability.lifecycle((event) => {
292
- if (event.type === 'attempt_settled' && event.outcome === 'completed') {
507
+ if (
508
+ event.type === 'attempt_settled' &&
509
+ event.outcome === 'completed' &&
510
+ event.result !== undefined
511
+ ) {
293
512
  publish(event.operation, event.result);
294
513
  }
295
514
  });
@@ -299,16 +518,16 @@ const detach = durability.lifecycle((event) => {
299
518
 
300
519
  Define `__DURABILITY_DEV__` as `true` at build time to log every operation the package performs. Each entry is a structured `console.debug` object whose `event` starts with `durability.dev.`:
301
520
 
302
- | `event` | Logged for |
303
- | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
304
- | `durability.dev.call` | Each public method call on every helper, including the experimental outbox and semaphores, generated operation, alarm, and socket outbound methods, and fanout `ack`, `retry`, and `deadLetter`, with its arguments. |
305
- | `durability.dev.call.return` / `.call.throw` | The call's result or serialized error, linked to the call by `call`. |
306
- | `durability.dev.storage` | Each storage method: transactions and alarm reads and writes, including those inside `transaction`. |
307
- | `durability.dev.storage.return` / `.storage.throw` | The storage call's result or serialized error, linked by `call`. |
308
- | `durability.dev.sql` | Each SQL statement, whitespace-compacted, with its bindings. |
309
- | `durability.dev.lifecycle` | Every lifecycle event, whether or not a hook is attached. |
310
- | `durability.dev.scheduler.early_alarm` | An in-memory timer that runs due work before the physical alarm. |
311
- | `durability.dev.socket.send` / `.socket.close` | Each frame sent to a browser socket, and each socket closed for a protocol error. |
521
+ | `event` | Logged for |
522
+ | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
523
+ | `durability.dev.call` | Each public method call on every helper, including the experimental outbox and semaphores, generated operation, alarm, and socket outbound methods, `memoize`, and fanout `ack`, `retry`, and `deadLetter`, with its arguments. |
524
+ | `durability.dev.call.return` / `.call.throw` | The call's result or serialized error, linked to the call by `call`. |
525
+ | `durability.dev.storage` | Each storage method: transactions and alarm reads and writes, including those inside `transaction`. |
526
+ | `durability.dev.storage.return` / `.storage.throw` | The storage call's result or serialized error, linked by `call`. |
527
+ | `durability.dev.sql` | Each SQL statement, whitespace-compacted, with its bindings. |
528
+ | `durability.dev.lifecycle` | Every lifecycle event, whether or not a hook is attached. |
529
+ | `durability.dev.scheduler.early_alarm` | An in-memory timer that runs due work before the physical alarm. |
530
+ | `durability.dev.socket.send` / `.socket.close` | Each frame sent to a browser socket, and each socket closed for a protocol error. |
312
531
 
313
532
  Every entry carries `seq`, a process-wide counter that orders entries across interleaved async work, and `at`, the `Date.now()` reading when it was logged. A deterministic simulation that controls time, randomness, and scheduling gets a replayable transcript by collecting the entries in order, for example with `vi.spyOn(console, 'debug')`.
314
533
 
@@ -361,6 +580,8 @@ export class TodoList extends DurableObject<Env> {
361
580
  return payload;
362
581
  },
363
582
  },
583
+ // Lifecycle events omit handler results unless this is set.
584
+ includeResults: true,
364
585
  });
365
586
 
366
587
  readonly socket = new DurabilitySocket({
@@ -400,7 +621,11 @@ export class TodoList extends DurableObject<Env> {
400
621
  super(ctx, env);
401
622
  // Push a durable operation's result to every subscriber once it has committed.
402
623
  this.durability.lifecycle((e) => {
403
- if (e.type === 'attempt_settled' && e.outcome === 'completed') {
624
+ if (
625
+ e.type === 'attempt_settled' &&
626
+ e.outcome === 'completed' &&
627
+ e.result !== undefined
628
+ ) {
404
629
  this.socket.created(e.result);
405
630
  }
406
631
  });
@@ -511,7 +736,7 @@ All three methods take the same query: `{ source, name, status, limit, cursor }`
511
736
  - `get(source, id)` returns one such handle, or `undefined`.
512
737
  - `records(query)` returns `{ records, cursor }` with the same data and no functions.
513
738
 
514
- Narrowing on `source` and `name` narrows `state.result` to that handler's result type.
739
+ Narrowing on `source` and `name` narrows `state.result` to that handler's result type. A pending operation or alarm also lists `state.memoized`, the keys its next attempt will [reuse](#memoize).
515
740
 
516
741
  ### Handles over RPC
517
742
 
@@ -813,7 +1038,7 @@ First deliveries are ordered by enqueue sequence per shard and target. Waiting r
813
1038
 
814
1039
  Set `batchDelayMs` to hold first deliveries for up to that many milliseconds, so messages enqueued together reach the consumer as one batch instead of one call each. Each first attempt is rounded up to the next multiple of `batchDelayMs`, so messages enqueued within the same window become due at the same time. It delays delivery, not acceptance: `enqueue` still returns once the batch commits. Retries keep their policy delay.
815
1040
 
816
- Consumer callbacks may forward the batch to any ordinary WorkerEntrypoint RPC method. Messages include `id`, `deliveryId`, `target`, `body`, `attempt`, `enqueuedAt`, and callable settlement capabilities:
1041
+ Consumer callbacks may forward the batch to any ordinary WorkerEntrypoint RPC method. Messages include `id`, `deliveryId`, `target`, `body`, `attempt`, `enqueuedAt`, a [`memoize`](#memoize) scoped to that message and target, so a retried delivery reuses work an earlier attempt finished, and callable settlement capabilities:
817
1042
 
818
1043
  - `ack()` settles this target only.
819
1044
  - `retry(10)` schedules eligibility after 10 ms, overriding the policy delay. It is not a real-time delivery guarantee.
@@ -2,22 +2,35 @@ import { type DurabilityLifecycleEvent, type LifecycleHook } from './lifecycle.j
2
2
  import { type DurabilityMigrationResult } from './migrations.js';
3
3
  import { type DurabilityAlarmMethodOptions, type ExecutionPolicyOptions } from './policy.js';
4
4
  import { type SchedulerAttachment } from './scheduler.js';
5
+ import { type DurableMemoize } from './memoize.js';
5
6
  /** Information passed to a named alarm handler for one execution attempt. */
6
7
  export type DurableAlarmInfo = {
7
8
  /** Configured name of the logical alarm. */
8
9
  name: string;
9
10
  /** Timestamp originally passed to the named alarm method. */
10
11
  scheduledTime: number;
11
- /** One-based execution attempt for this scheduled occurrence. */
12
+ /**
13
+ * One-based attempt ordinal for this scheduled occurrence, durably recorded
14
+ * before the handler runs. It is retry and logging context, not a record of
15
+ * what the handler did: never gate once-only work on it (such as
16
+ * `attempt === 1`). Key such work on durable state keyed by `idempotencyKey`
17
+ * instead. See the README identity glossary.
18
+ */
12
19
  attempt: number;
13
20
  /** Whether this scheduled occurrence has run before. */
14
21
  isRetry: boolean;
15
22
  /** Number of previous attempts for this scheduled occurrence. */
16
23
  retryCount: number;
17
- /** Stable key shared by every attempt of this scheduled occurrence. */
24
+ /**
25
+ * Stable key shared by every attempt of this scheduled occurrence. This is
26
+ * the occurrence's logical identity: pass it to downstream services for
27
+ * deduplication rather than relying on `attempt`.
28
+ */
18
29
  idempotencyKey: string;
19
30
  /** Aborts when the attempt exceeds its configured timeout. */
20
31
  signal: AbortSignal;
32
+ /** Persists a callback's result by key so a retry of this occurrence reuses it. */
33
+ memoize: DurableMemoize;
21
34
  /** Invocation information for the shared physical Durable Object alarm. */
22
35
  platform: AlarmInvocationInfo | undefined;
23
36
  };
@@ -46,7 +59,16 @@ export type DurabilityAlarmEvent = Extract<DurabilityLifecycleEvent, {
46
59
  type: 'purged';
47
60
  }>;
48
61
  /** Schedules (or replaces) the named alarm's pending occurrence. */
49
- export type DurableNamedAlarm = (scheduledTime: number) => Promise<void>;
62
+ export type DurableNamedAlarm = ((scheduledTime: number) => Promise<void>) & {
63
+ /**
64
+ * Experimental. Deletes the alarm's pending occurrence and reconciles the
65
+ * physical alarm, then aborts a running attempt of it. That attempt can no
66
+ * longer retry when it settles, but calling the scheduling method again,
67
+ * including from that handler, schedules a new occurrence. Resolves whether
68
+ * a pending occurrence was deleted.
69
+ */
70
+ cancel: () => Promise<boolean>;
71
+ };
50
72
  declare class DurabilityAlarmsCore<Names extends string> {
51
73
  #private;
52
74
  constructor(config: DurabilityAlarmsConfig<Names>);
@@ -9,14 +9,15 @@ var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (
9
9
  if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
10
10
  return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
11
11
  };
12
- var _DurabilityAlarmsCore_instances, _DurabilityAlarmsCore_engine, _DurabilityAlarmsCore_handlers, _DurabilityAlarmsCore_policy, _DurabilityAlarmsCore_lifecycle, _DurabilityAlarmsCore_emit, _DurabilityAlarmsCore_active, _DurabilityAlarmsCore_trackedSource, _DurabilityAlarmsCore_runDue, _DurabilityAlarmsCore_schedule, _DurabilityAlarmsCore_execute;
12
+ var _DurabilityAlarmsCore_instances, _DurabilityAlarmsCore_engine, _DurabilityAlarmsCore_handlers, _DurabilityAlarmsCore_policy, _DurabilityAlarmsCore_lifecycle, _DurabilityAlarmsCore_emit, _DurabilityAlarmsCore_active, _DurabilityAlarmsCore_trackedSource, _DurabilityAlarmsCore_runDue, _DurabilityAlarmsCore_schedule, _DurabilityAlarmsCore_cancel, _DurabilityAlarmsCore_execute;
13
13
  import { z } from 'zod';
14
14
  import { traceFunction, traceMethods } from './dev.js';
15
15
  import { DurableAlarmTimeoutError, DurableAttemptsExhaustedError, isErrorInstance, NonRetryableError, } from './errors.js';
16
16
  import { createLifecycleHooks, } from './lifecycle.js';
17
- import { durabilityNamedAlarmMigrations, migrateCapability, } from './migrations.js';
17
+ import { durabilityNamedAlarmMigrations, durabilityMemoMigrations, migrateCapability, } from './migrations.js';
18
18
  import { alarmMethodOptionsSchema, createPolicyResolver, } from './policy.js';
19
19
  import { engineFor, } from './scheduler.js';
20
+ import { createMemoize, deleteMemos, deleteMemosOfDroppedTable, memoizedKeys, } from './memoize.js';
20
21
  import { deleteAttempt, listCreated } from './storage.js';
21
22
  import { registerTrackedSource, } from './tracked-source.js';
22
23
  const alarmRowSchema = z.object({
@@ -37,7 +38,7 @@ const readAlarm = (sql, name) => alarmRowSchema
37
38
  .parse(sql
38
39
  .exec('SELECT * FROM durability_alarms WHERE name = ?', name)
39
40
  .toArray()[0]);
40
- const alarmRecord = (row) => ({
41
+ const alarmRecord = (sql, row) => ({
41
42
  id: row.name,
42
43
  name: row.name,
43
44
  generation: row.generation_id,
@@ -48,6 +49,7 @@ const alarmRecord = (row) => ({
48
49
  attempt: row.attempt,
49
50
  nextAttemptAt: row.next_attempt_at,
50
51
  lastError: row.last_error,
52
+ memoized: memoizedKeys(sql, alarmsTable, row.name, row.generation_id),
51
53
  }
52
54
  : {
53
55
  status: 'failed',
@@ -83,12 +85,16 @@ class DurabilityAlarmsCore {
83
85
  if (reserved.has(name)) {
84
86
  throw new Error(`"${name}" is reserved by DurabilityAlarms`);
85
87
  }
88
+ const method = Object.assign(traceFunction(`DurabilityAlarms.${name}`, (scheduledTime) => __classPrivateFieldGet(this, _DurabilityAlarmsCore_instances, "m", _DurabilityAlarmsCore_schedule).call(this, name, scheduledTime)), {
89
+ cancel: traceFunction(`DurabilityAlarms.${name}.cancel`, () => __classPrivateFieldGet(this, _DurabilityAlarmsCore_instances, "m", _DurabilityAlarmsCore_cancel).call(this, name)),
90
+ });
86
91
  Object.defineProperty(this, name, {
87
- value: traceFunction(`DurabilityAlarms.${name}`, (scheduledTime) => __classPrivateFieldGet(this, _DurabilityAlarmsCore_instances, "m", _DurabilityAlarmsCore_schedule).call(this, name, scheduledTime)),
92
+ value: method,
88
93
  enumerable: true,
89
94
  });
90
95
  }
91
96
  __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").migrateSchema('namedAlarms', durabilityNamedAlarmMigrations);
97
+ __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").migrateSchema('memos', durabilityMemoMigrations);
92
98
  __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").register({
93
99
  table: alarmsTable,
94
100
  runDue: (now, info) => __classPrivateFieldGet(this, _DurabilityAlarmsCore_instances, "m", _DurabilityAlarmsCore_runDue).call(this, now, info),
@@ -97,7 +103,9 @@ class DurabilityAlarmsCore {
97
103
  }
98
104
  /** Applies or reverts the `durability_alarms` schema; see {@link migrateCapability}. */
99
105
  static migrate(context, target) {
100
- return migrateCapability(context.storage, 'namedAlarms', durabilityNamedAlarmMigrations, target);
106
+ const result = migrateCapability(context.storage, 'namedAlarms', durabilityNamedAlarmMigrations, target);
107
+ deleteMemosOfDroppedTable(context.storage.sql, alarmsTable);
108
+ return result;
101
109
  }
102
110
  /** Runs due named alarms and re-arms the physical alarm. */
103
111
  alarm(info) {
@@ -131,7 +139,7 @@ _DurabilityAlarmsCore_engine = new WeakMap(), _DurabilityAlarmsCore_handlers = n
131
139
  return {
132
140
  get: async (name) => {
133
141
  const row = readAlarm(sql, name);
134
- return row && alarmRecord(row);
142
+ return row && alarmRecord(sql, row);
135
143
  },
136
144
  list: async ({ limit, cursor, status, name }) => {
137
145
  const page = listCreated(sql, alarmsTable, 'name', {
@@ -143,7 +151,7 @@ _DurabilityAlarmsCore_engine = new WeakMap(), _DurabilityAlarmsCore_handlers = n
143
151
  const records = alarmRowSchema
144
152
  .array()
145
153
  .parse(page.rows)
146
- .map(alarmRecord);
154
+ .map((row) => alarmRecord(sql, row));
147
155
  return page.next === undefined
148
156
  ? { records }
149
157
  : { records, cursor: page.next };
@@ -169,8 +177,13 @@ _DurabilityAlarmsCore_engine = new WeakMap(), _DurabilityAlarmsCore_handlers = n
169
177
  ORDER BY next_attempt_at ASC
170
178
  LIMIT 100`, now)
171
179
  .toArray());
180
+ // With liveTimers a running occurrence re-arms itself when it settles, so
181
+ // skipping it keeps a long handler from blocking a concurrent alarm pass.
182
+ const pending = __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").liveTimers
183
+ ? due.filter((row) => !__classPrivateFieldGet(this, _DurabilityAlarmsCore_active, "f").has(row.name))
184
+ : due;
172
185
  const failures = [];
173
- await __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").runConcurrent(due, async (row) => {
186
+ await __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").runConcurrent(pending, async (row) => {
174
187
  try {
175
188
  await __classPrivateFieldGet(this, _DurabilityAlarmsCore_instances, "m", _DurabilityAlarmsCore_execute).call(this, row, info);
176
189
  }
@@ -201,6 +214,8 @@ _DurabilityAlarmsCore_engine = new WeakMap(), _DurabilityAlarmsCore_handlers = n
201
214
  last_error = excluded.last_error,
202
215
  last_error_name = excluded.last_error_name,
203
216
  created_at = excluded.created_at`, name, generation, scheduledTime, scheduledTime, createdAt);
217
+ // Memoized results belong to the replaced occurrence, never to the new one.
218
+ deleteMemos(sql, alarmsTable, name);
204
219
  });
205
220
  // A waiter on the replaced occurrence resolves now instead of at its timeout.
206
221
  __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").notify(alarmsTable, name);
@@ -214,6 +229,28 @@ _DurabilityAlarmsCore_engine = new WeakMap(), _DurabilityAlarmsCore_handlers = n
214
229
  attempt: 0,
215
230
  scheduledTime,
216
231
  });
232
+ }, _DurabilityAlarmsCore_cancel = async function _DurabilityAlarmsCore_cancel(name) {
233
+ const cancelled = await __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").transaction((sql) => {
234
+ const removed = sql
235
+ .exec(`DELETE FROM durability_alarms
236
+ WHERE name = ? AND status = 'pending'
237
+ RETURNING name`, name)
238
+ .toArray().length > 0;
239
+ if (removed) {
240
+ deleteMemos(sql, alarmsTable, name);
241
+ }
242
+ return removed;
243
+ });
244
+ if (!cancelled) {
245
+ return false;
246
+ }
247
+ // Every settlement write is conditional on the deleted row, so a running
248
+ // attempt can no longer retry; abort it as purging does.
249
+ __classPrivateFieldGet(this, _DurabilityAlarmsCore_active, "f")
250
+ .get(name)
251
+ ?.controller?.abort(new Error(`Durable alarm "${name}" was cancelled`));
252
+ __classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").notify(alarmsTable, name);
253
+ return true;
217
254
  }, _DurabilityAlarmsCore_execute = function _DurabilityAlarmsCore_execute(row, platform) {
218
255
  const running = __classPrivateFieldGet(this, _DurabilityAlarmsCore_active, "f").get(row.name);
219
256
  if (running) {
@@ -245,6 +282,11 @@ _DurabilityAlarmsCore_engine = new WeakMap(), _DurabilityAlarmsCore_handlers = n
245
282
  retryCount: attempt - 1,
246
283
  idempotencyKey: `durability-alarm:v1:${row.generation_id}`,
247
284
  signal,
285
+ memoize: createMemoize(__classPrivateFieldGet(this, _DurabilityAlarmsCore_engine, "f").sql, alarmsTable, {
286
+ key: row.name,
287
+ generation: row.generation_id,
288
+ attempt,
289
+ }),
248
290
  platform,
249
291
  });
250
292
  },