durability 3.0.0 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,10 +6,12 @@ Composable capabilities for application-owned Cloudflare Durable Objects:
6
6
  - **`DurabilityAlarms`** — named logical alarms with retries and stable idempotency keys.
7
7
  - **`DurabilityFanout`** — durable batch acceptance and independent delivery to static consumer or storage targets.
8
8
  - **`DurabilityRouting`** — a DO-side load observer and client-side shard selector (`durability/routing`), supporting location hints and caller transforms.
9
+ - **`DurabilitySocket`** — typed WebSocket RPC and replayable event push from a Durable Object to browsers (`durability/socket`), with a dependency-free browser client (`durability/client`).
10
+ - **`DurabilityTracker`** _(experimental, `durability/experimental`)_ — lists work across capabilities and custom sources, as RPC handles that can be waited on or as plain records.
9
11
 
10
12
  Durable capabilities own their storage and migrations. Routing observations are in-memory and advisory. When an object composes durable capabilities, one `DurabilityScheduler` shares its physical alarm and concurrency pool. No queue factory, generated DO class, or special consumer superclass is required.
11
13
 
12
- Durability is not an event-sourcing, replay-log, or workflow-orchestration engine. Operation methods return `Promise<void>` after registration commits; they do not wait for the handler result. Failed work is retried from later alarms. Both SQLite-backed and KV-backed Durable Object classes are supported.
14
+ Durability is not an event-sourcing, replay-log, or workflow-orchestration engine. Operation methods return `Promise<void>` after registration commits; they do not wait for the handler result. Failed work is retried from later alarms. Only SQLite-backed Durable Object classes are supported.
13
15
 
14
16
  ## Migrating to v3
15
17
 
@@ -119,13 +121,15 @@ export class Subscription extends DurableObject<Env> {
119
121
 
120
122
  Scheduling a name again replaces its pending occurrence. If that name is already running, the current handler continues and the replacement runs afterward. Different names may run concurrently, but one name never has more than one active handler in the same instance.
121
123
 
124
+ Named alarms inside the scheduler's `alarmMinDelayMs` window start from an in-memory timer at their requested time. The option accepts 1 to 15 seconds and defaults to 15 seconds; the scheduler never sets the physical alarm sooner than that delay; the physical alarm remains as a durable fallback for restarts. If concurrency capacity is full when the timer fires, execution waits in the shared FIFO queue. The scheduling method resolves after persistence and never waits for the handler to finish.
125
+
122
126
  Each occurrence receives a new internal generation. Its `idempotencyKey` stays stable across retries while `attempt` increments. A successful handler removes only the occurrence it executed, so it cannot delete a replacement scheduled while it was running.
123
127
 
124
128
  A timeout aborts the handler's `signal` and is terminal by default because the external outcome is unknown. Set `retryTimeouts: true` only when the handler's side effects use the idempotency key or reconcile their outcome before retrying. Named alarms remain at-least-once across eviction or restart.
125
129
 
126
130
  ## Using both in one Durable Object
127
131
 
128
- A Durable Object has one physical alarm. When one object uses both helpers, create a `DurabilityScheduler` and pass it to each so they share the alarm, the storage backend, and the concurrency pool:
132
+ A Durable Object has one physical alarm. When one object uses both helpers, create a `DurabilityScheduler` and pass it to each so they share the alarm, the storage, and the concurrency pool:
129
133
 
130
134
  ```ts
131
135
  import { Durability, DurabilityAlarms, DurabilityScheduler } from 'durability';
@@ -134,6 +138,7 @@ export class ImageJobs extends DurableObject<Env> {
134
138
  private readonly scheduler = new DurabilityScheduler({
135
139
  context: this.ctx,
136
140
  alarmConcurrency: 10,
141
+ alarmMinDelayMs: 15_000,
137
142
  });
138
143
 
139
144
  private readonly durability = new Durability({
@@ -155,11 +160,11 @@ export class ImageJobs extends DurableObject<Env> {
155
160
  }
156
161
  ```
157
162
 
158
- A helper configured with `context` instead of `scheduler` creates a private scheduler and accepts the scheduler options (`storageBackend`, `alarmConcurrency`, `alarmHandoffMs`) in the same object. Two helpers on the same object must share one scheduler; otherwise each would reconcile the physical alarm against only its own records.
163
+ A helper configured with `context` instead of `scheduler` creates a private scheduler and accepts the scheduler options (`alarmConcurrency`, `alarmMinDelayMs`, `alarmHandoffMs`) in the same object. Two helpers on the same object must share one scheduler; otherwise each would reconcile the physical alarm against only its own records.
159
164
 
160
- ## Storage backends
165
+ ## Storage
161
166
 
162
- SQLite is the default. Configure the Durable Object class with `new_sqlite_classes`; no option is required:
167
+ Durability stores its records in the object's SQLite database. Configure the Durable Object class with `new_sqlite_classes`:
163
168
 
164
169
  ```jsonc
165
170
  {
@@ -169,24 +174,6 @@ SQLite is the default. Configure the Durable Object class with `new_sqlite_class
169
174
 
170
175
  `Durability` creates and migrates `durability_calls`; `DurabilityAlarms` does the same for `durability_alarms`. Both histories are tracked in the shared `durability_migrations` table and only the constructed helpers' tables exist.
171
176
 
172
- For a class backed only by the KV storage API, configure it with `new_classes` and select the backend:
173
-
174
- ```jsonc
175
- {
176
- "migrations": [{ "tag": "v1", "new_classes": ["ImageJobs"] }],
177
- }
178
- ```
179
-
180
- ```ts
181
- const durability = new Durability({
182
- context: this.ctx,
183
- handlers,
184
- storageBackend: 'kv',
185
- });
186
- ```
187
-
188
- KV mode stores records and ordered pending/created indexes in the object's private key-value storage, never touches `storage.sql`, and creates no migration records. Records are not converted between backends; deploy a separate class or copy records explicitly before switching.
189
-
190
177
  ### Explicit migrations
191
178
 
192
179
  Rollbacks are explicit because reverting the initial migration deletes records:
@@ -238,7 +225,7 @@ Throw `NonRetryableError` to move work directly to `failed` without another atte
238
225
 
239
226
  Work is delivered at least once. A process can stop after a side effect succeeds but before its completion record commits, so handlers should be idempotent and pass the call ID or alarm `idempotencyKey` to external services.
240
227
 
241
- `alarmConcurrency` defaults to 10 and is one FIFO limit shared by eager operation handlers, alarm-driven operation handlers, and named alarm handlers attached to the same scheduler. Newly registered operations start eagerly only when a permit is immediately available; otherwise the reconciled alarm picks them up.
228
+ `alarmConcurrency` defaults to 10 and is one FIFO limit shared by eager operation handlers, timer-driven work, and physical-alarm work attached to the same scheduler. Newly registered operations start eagerly only when a permit is immediately available. Work due before the physical-alarm minimum enters the same queue when its in-memory timer fires.
242
229
 
243
230
  Completion records are retained indefinitely so IDs remain deduplicated. The scheduler owns the Durable Object's alarm; compose unrelated scheduled work as named alarms instead of replacing the physical alarm.
244
231
 
@@ -251,7 +238,7 @@ const removedOperations = await this.durability.purgeBefore(retentionCutoff);
251
238
  const removedAlarms = await this.alarms.purgeBefore(retentionCutoff);
252
239
  ```
253
240
 
254
- SQLite purges in one transaction. KV purges in bounded transactions until no matching record remains, so retry if the invocation is interrupted. The purge best-effort aborts matching active handlers, but cannot force an abort-ignoring handler to stop. Operation IDs can be reused immediately; a late old generation cannot overwrite the replacement record.
241
+ The purge runs in one transaction. It best-effort aborts matching active handlers, but cannot force an abort-ignoring handler to stop. Operation IDs can be reused immediately; a late old generation cannot overwrite the replacement record.
255
242
 
256
243
  ## Lifecycle metrics hook
257
244
 
@@ -267,6 +254,261 @@ const durability = new Durability({
267
254
 
268
255
  The hook is best-effort and non-blocking, is not a durable journal, and may duplicate or drop events across crashes. Hook failures are reported to `console.error` and never change queue state. Returned promises are attached to `waitUntil` when the context exposes it.
269
256
 
257
+ Further hooks attach after construction with `lifecycle(hook)`, which returns a function that detaches it. `Durability` hooks are typed per operation: a `completed` settlement carries the handler's `result`.
258
+
259
+ ```ts
260
+ const detach = durability.lifecycle((event) => {
261
+ if (event.type === 'attempt_settled' && event.outcome === 'completed') {
262
+ publish(event.operation, event.result);
263
+ }
264
+ });
265
+ ```
266
+
267
+ ## Browser sockets
268
+
269
+ `DurabilitySocket` lets browsers call a Durable Object and receive events from it over one hibernatable WebSocket. `inbound` handlers become callable methods on the client; `outbound` events become methods on the socket that push to every subscribed connection. Nothing outside `inbound` is reachable, and no result is broadcast unless an outbound method sends it.
270
+
271
+ ```ts
272
+ import { DurableObject } from 'cloudflare:workers';
273
+ import {
274
+ Durability,
275
+ type DurableCall,
276
+ type DurabilityLifecycleEvent,
277
+ } from 'durability';
278
+ import { DurabilitySocket, event, type SocketCall } from 'durability/socket';
279
+ import { z } from 'zod';
280
+
281
+ const TodoSchema = z.object({
282
+ id: z.string(),
283
+ title: z.string(),
284
+ done: z.boolean(),
285
+ });
286
+ type Todo = z.infer<typeof TodoSchema>;
287
+ type Metadata = { userId: string };
288
+
289
+ export class TodoList extends DurableObject<Env> {
290
+ private readonly todos = new Map<string, Todo>();
291
+
292
+ readonly durability = new Durability({
293
+ context: this.ctx,
294
+ handlers: {
295
+ createTodo: async ({ payload }: DurableCall<Todo>) => {
296
+ this.todos.set(payload.id, payload);
297
+ return payload;
298
+ },
299
+ },
300
+ });
301
+
302
+ readonly socket = new DurabilitySocket({
303
+ context: this.ctx,
304
+ inbound: {
305
+ // Bare handlers take no input (or `unknown` input they validate themselves).
306
+ list: () => [...this.todos.values()],
307
+ // Typed input requires a Standard Schema; here the handler is a durable operation.
308
+ createTodo: {
309
+ input: z.object({ id: z.string(), payload: TodoSchema }),
310
+ handle: this.durability.createTodo,
311
+ },
312
+ toggle: {
313
+ input: z.string(),
314
+ handle: (id: string, call: SocketCall<Metadata>) => {
315
+ const todo = this.todos.get(id);
316
+ if (!todo) {
317
+ throw new RangeError(`No todo "${id}" for ${call.metadata.userId}`);
318
+ }
319
+ todo.done = !todo.done;
320
+ this.socket.updated(todo);
321
+ return todo;
322
+ },
323
+ },
324
+ },
325
+ outbound: {
326
+ created: event<Todo>(),
327
+ updated: event<Todo>(),
328
+ lifecycle: event<DurabilityLifecycleEvent>(),
329
+ },
330
+ metadata: (request): Metadata => ({
331
+ userId: request.headers.get('x-user-id') ?? 'anonymous',
332
+ }),
333
+ });
334
+
335
+ constructor(ctx: DurableObjectState, env: Env) {
336
+ super(ctx, env);
337
+ // Push a durable operation's result to every subscriber once it has committed.
338
+ this.durability.lifecycle((e) => {
339
+ if (e.type === 'attempt_settled' && e.outcome === 'completed') {
340
+ this.socket.created(e.result);
341
+ }
342
+ });
343
+ this.durability.lifecycle(this.socket.lifecycle);
344
+ }
345
+
346
+ fetch(request: Request) {
347
+ return this.socket.fetch(request);
348
+ }
349
+
350
+ webSocketMessage(ws: WebSocket, message: string | ArrayBuffer) {
351
+ return this.socket.webSocketMessage(ws, message);
352
+ }
353
+
354
+ webSocketClose(ws: WebSocket) {
355
+ this.socket.webSocketClose(ws);
356
+ }
357
+
358
+ alarm(info?: AlarmInvocationInfo) {
359
+ return this.durability.alarm(info);
360
+ }
361
+ }
362
+
363
+ // The Worker authenticates, then forwards the upgrade with trusted metadata.
364
+ export default {
365
+ async fetch(request: Request, env: Env) {
366
+ const listId = new URL(request.url).pathname.split('/').pop() ?? '';
367
+ const userId = await authenticate(request);
368
+ if (!userId) {
369
+ return new Response('Unauthorized', { status: 401 });
370
+ }
371
+ const upgrade = new Request(request, {
372
+ headers: { ...Object.fromEntries(request.headers), 'x-user-id': userId },
373
+ });
374
+ return env.TODO_LISTS.getByName(listId).fetch(upgrade);
375
+ },
376
+ };
377
+ ```
378
+
379
+ An inbound handler is either a bare function, which receives no input or `unknown` input it validates itself, or `{ input, handle }`, where `input` is any [Standard Schema](https://standardschema.dev) validator (zod, Valibot, ArkType, …) and `handle` declares its parameter type. A bare function that claims a typed parameter does not compile. Handlers receive `call.metadata`, the value `metadata(request)` returned at upgrade time; it is stored in the WebSocket attachment, so it survives hibernation. The fronting Worker is responsible for authentication and for routing the upgrade to the right object; the socket does no authorization of its own. `metadata` must be assignable to every `SocketCall<…>` annotation the handlers declare.
380
+
381
+ Every outbound event is persisted to `durability_socket_events` with a sequence number before it is pushed. A reconnecting client subscribes from the last sequence it saw: events still within `retention` (default 1000) are replayed in order, otherwise the client is told to resync and should rebuild its state from an inbound snapshot call. Payloads must be JSON. Keep-alive pings are answered by the runtime through `setWebSocketAutoResponse`, so idle connections never wake the object. Frames over 1 MiB are rejected; malformed frames close the connection with code `4002`.
382
+
383
+ The browser client derives every method and event type from the socket:
384
+
385
+ ```ts
386
+ import { DurabilitySocketClient, logging } from 'durability/client';
387
+ import type { TodoList } from '../worker'; // type-only: nothing from the Worker is bundled
388
+
389
+ const client = new DurabilitySocketClient<TodoList['socket']>({
390
+ url: (listId) => `wss://example.com/lists/${listId}/ws`,
391
+ }).with(logging());
392
+
393
+ const conn = client.connect('personal');
394
+ const todos = await conn.rpc.list();
395
+ await conn.rpc.createTodo({
396
+ id: crypto.randomUUID(),
397
+ payload: { id: 'todo-1', title: 'Ship it', done: false },
398
+ });
399
+ await conn.rpc.toggle('todo-1');
400
+
401
+ conn.on.updated((todo) => render(todo));
402
+ conn.on.lifecycle((e) => {
403
+ if (e.entityKind === 'operation' && e.type === 'attempt_settled') {
404
+ markSettled(e.id);
405
+ }
406
+ });
407
+ conn.on.resync(({ names }) => reload(names)); // first subscription, and after an unreplayable gap
408
+ conn.on.status((status) => showStatus(status)); // connecting | connected | disconnected | closed
409
+ conn.close();
410
+ ```
411
+
412
+ `on.resync` reports the outbound events whose subscribers have no continuous history: every mapped name on their first subscription, newly added names on a live connection, and all names after an unreplayable gap. Rebuild the affected state from an inbound snapshot call; `@durability/tanstack-db` does this for collections.
413
+
414
+ Connections open lazily on the first call or subscription and reconnect with jittered exponential backoff. Calls queued before the socket opens are sent once; calls already sent when the connection drops reject with `SocketConnectionLostError` and are never replayed, because the server may have run them. Route retried mutations through `Durability` with a stable idempotency key rather than re-calling. Calls reject with `SocketTimeoutError` after `timeoutMs` (default 2 minutes). Server-side handler failures reject with `SocketRemoteError`, carrying the server error's `name` and `message`. `with(transform)` returns a new client that wraps every call; transforms receive `{ method, input, next }` and may replace the input.
415
+
416
+ `durability/client` imports nothing from `cloudflare:workers` and is typechecked against DOM types only. The complete, typechecked version of this example lives in [`examples/realtime-todos`](../../examples/realtime-todos).
417
+
418
+ ## Tracking work (experimental)
419
+
420
+ Everything under `durability/experimental` may change or be removed in a minor release and is excluded from the semver guarantee of the root package. Pin an exact version when depending on it. Nothing from this entry point is reachable from `durability`, and it adds nothing to the public types of `Durability` or `DurabilityAlarms`.
421
+
422
+ `DurabilityTracker` lists records from several sources. Sources are keyed by names you choose; the key becomes each record's `source`.
423
+
424
+ ```ts
425
+ import { DurabilityTracker } from 'durability/experimental';
426
+
427
+ export class ImageJobs extends DurableObject<Env> {
428
+ // scheduler, durability, and alarms as above
429
+
430
+ private readonly tracker = new DurabilityTracker({
431
+ sources: { jobs: this.durability, alarms: this.alarms },
432
+ });
433
+
434
+ listWork(cursor?: string) {
435
+ return this.tracker.list({ status: 'pending', limit: 50, cursor });
436
+ }
437
+
438
+ getWork(id: string) {
439
+ return this.tracker.get('jobs', id);
440
+ }
441
+ }
442
+ ```
443
+
444
+ All three methods take the same query: `{ source, name, status, limit, cursor }`. `limit` defaults to 50 and is capped at 100. Pass `cursor` back with the same query to continue; it is present only while more records may follow. Sources are read one after another, so there is no ordering across sources.
445
+
446
+ - `list(query)` returns `{ items, cursor }`. Each item has `source`, `id`, `name`, `generation`, `createdAt`, and `state`, plus `get()`, which reads the latest state, and `wait({ timeoutMs })`, which resolves once the record is `completed` or `failed`, or returns the latest state after the timeout without polling storage. Both return `not_found` once the record is purged or its ID is reused for new work.
447
+ - `get(source, id)` returns one such handle, or `undefined`.
448
+ - `records(query)` returns `{ records, cursor }` with the same data and no functions.
449
+
450
+ Narrowing on `source` and `name` narrows `state.result` to that handler's result type.
451
+
452
+ ### Handles over RPC
453
+
454
+ Return handles from a Durable Object method. Over Workers RPC or [Cap'n Web](https://github.com/cloudflare/capnweb), the data fields are copied and `get` and `wait` become stubs, so a caller can wait without polling:
455
+
456
+ ```ts
457
+ const { items } = await env.IMAGE_JOBS.getByName('jobs').listWork();
458
+ for (const item of items) {
459
+ const result = await item.wait({ timeoutMs: 30_000 });
460
+ }
461
+ ```
462
+
463
+ Workers RPC pipelining also lets a caller wait in the same round trip that returns the handle, for example `stub.getWork(id).wait({ timeoutMs })` when the method always returns a handle. Waiting is in memory: if a deployment, eviction, or disconnect interrupts it, get a new handle by ID and wait again. Stubs follow the RPC system's disposal rules; dispose them with `using` or let the request end.
464
+
465
+ ### Records over sockets
466
+
467
+ `DurabilitySocket` frames are JSON, so functions do not survive the trip. Expose `records` as an inbound snapshot and push changes as outbound events, for example from a lifecycle hook:
468
+
469
+ ```ts
470
+ readonly socket = new DurabilitySocket({
471
+ context: this.ctx,
472
+ inbound: {
473
+ work: {
474
+ input: z.object({ cursor: z.string().optional() }),
475
+ handle: (query: { cursor?: string }) => this.tracker.records(query),
476
+ },
477
+ },
478
+ outbound: { settled: event<DurabilityOperationEvent<typeof handlers>>() },
479
+ });
480
+ ```
481
+
482
+ A complete two-object version, with push through `DurabilityFanout` and request/response through a pipelined `wait`, lives in [`examples/job-dashboard`](../../examples/job-dashboard).
483
+
484
+ ### Sources
485
+
486
+ `Durability` lists every operation. `DurabilityAlarms` lists pending and failed occurrences only, because success deletes an occurrence; a waiter still sees `completed`. Rescheduling an alarm replaces its occurrence, so waiters on the old one receive `not_found`.
487
+
488
+ Any object implementing `TrackedSource` can be a source, for example work kept in the application's own table:
489
+
490
+ ```ts
491
+ import type { TrackedSource } from 'durability/experimental';
492
+
493
+ type Approvals = { approval: { approvedBy: string } };
494
+
495
+ const approvals: TrackedSource<Approvals> = {
496
+ get: async (id) => readApproval(id),
497
+ list: async ({ limit, cursor, status, name }) =>
498
+ listApprovals(limit, cursor, status, name),
499
+ watch: (id, listener) => subscribeToApproval(id, listener),
500
+ };
501
+
502
+ new DurabilityTracker({ sources: { jobs: this.durability, approvals } });
503
+ ```
504
+
505
+ - `get(id)` and `list(query)` return records with `id`, `name`, `generation`, `createdAt`, and `state`. Keep `generation` stable unless the ID is reused for new work.
506
+ - `state.status` is `pending`, `completed`, or `failed`. Express finer outcomes, such as a rejection, as `failed` with a specific `error.name`.
507
+ - `list` may return fewer than `limit` records together with a cursor. Omit the cursor at the end.
508
+ - `watch(id, listener)` is optional and returns an unsubscribe function. Call `listener()` after a record may have settled; pass the settled record when it can no longer be read. Without `watch`, `wait` reads again only at its timeout.
509
+
510
+ New members of `TrackedSource` are always optional, so a custom source keeps compiling across releases.
511
+
270
512
  ## Routing and fanout
271
513
 
272
514
  A queue is a composition, not another exported abstraction. An application owns its DO class and attaches the capabilities it needs:
@@ -362,6 +604,8 @@ Without custom sharding, routing starts with one shard, samples two candidate sh
362
604
 
363
605
  First deliveries are ordered by enqueue sequence per shard and target. Waiting retries do not block later first deliveries; completion order is not guaranteed. Each scheduler pass advances a bounded batch per target (`maxBatchSize`, default 10) with the shared concurrency pool. Different targets progress independently subject to that capacity.
364
606
 
607
+ Set `batchDelayMs` to hold first deliveries for up to that many milliseconds, so messages enqueued together reach the consumer as one batch instead of one call each. Each first attempt is rounded up to the next multiple of `batchDelayMs`, so messages enqueued within the same window become due at the same time. It delays delivery, not acceptance: `enqueue` still returns once the batch commits. Retries keep their policy delay.
608
+
365
609
  Consumer callbacks may forward the batch to any ordinary WorkerEntrypoint RPC method. Messages include `id`, `deliveryId`, `target`, `body`, `attempt`, `enqueuedAt`, and callable settlement capabilities:
366
610
 
367
611
  - `ack()` settles this target only.
@@ -369,6 +613,18 @@ Consumer callbacks may forward the batch to any ordinary WorkerEntrypoint RPC me
369
613
  - `deadLetter()` records durable terminal intent before writing to the DLQ. Exhaustion does the same automatically.
370
614
  - Returning, throwing, or timing out leaves unsettled messages eligible for retry. A second or late settlement call rejects with `FanoutSettlementError`.
371
615
 
616
+ Each settlement call is an RPC back into the Durable Object plus a storage transaction. For consumers that only ever acknowledge the whole batch, set `ackOnReturn: true` on the target: when `deliver` returns without throwing, every message the consumer did not settle explicitly is acknowledged in one transaction, without a call per message. Explicit `retry()` and `deadLetter()` calls still apply, and a throw or timeout acknowledges nothing.
617
+
618
+ ```ts
619
+ targets: {
620
+ consumer: {
621
+ deliver: (messages) => this.env.CONSUMER.consume(messages),
622
+ ackOnReturn: true,
623
+ },
624
+ },
625
+ batchDelayMs: 100,
626
+ ```
627
+
372
628
  Await settlement calls. Timed-out RPCs cannot be forcibly cancelled, so side effects may overlap later attempts. Storage targets automatically acknowledge successful writes; failed writes retry independently of consumer targets. Their write contract must be idempotent by the supplied delivery identity.
373
629
 
374
630
  ### DLQ storage and per-target redrive
@@ -388,7 +644,7 @@ Two write invariants are the application's to uphold:
388
644
  - **Key each entry by `id` and refuse to overwrite it.** `id` is stable across delivery attempts, so a retried write collapses onto the same entry (`onlyIf: { etagDoesNotMatch: '*' }` on R2) and a redrive deletes exactly what it re-enqueued. `attempts` and `storedAt` advance between attempts, so entries are not byte-identical.
389
645
  - **Delete only after acceptance.** Reading entries back for replay or redrive is application policy, so nothing in the library lists or removes them.
390
646
 
391
- Retention is likewise external: filter on `storedAt` when selecting entries to redrive, and configure a **lifecycle deletion rule** on the prefix (for example 30 days on a DLQ prefix, unbounded for an archive) for physical expiry. A failed dead-letter write retains the terminal intent with a 60-second backoff rather than redelivering to the consumer, and omitting `dlq` retains terminal work rather than silently deleting it.
647
+ Retention is likewise external: filter on `storedAt` when selecting entries to redrive, and configure a **lifecycle deletion rule** on the prefix (for example 30 days on a DLQ prefix, unbounded for an archive) for physical expiry. A failed dead-letter write retains the terminal intent with a 60-second backoff rather than redelivering to the consumer. Omitting `dlq` drops terminal deliveries: nothing could ever accept them, and retrying would keep the object awake indefinitely. The `terminal` lifecycle event still reports each one.
392
648
 
393
649
  Redrive is explicit application code, not queue sugar. Walk your own storage, select what to replay, then enqueue only the failed target:
394
650
 
@@ -414,6 +670,126 @@ Routing uses the actual `@durability/transforms` runtime. For example, `routing.
414
670
 
415
671
  Loopback namespaces need the application DO declared in Wrangler migrations (or the platform's supported class configuration), but no redundant DO binding block is needed. Consumer transport is explicitly wired by the application, such as a service binding or loopback entrypoint. See the typechecked [email queue composition](../../examples/email-queue).
416
672
 
673
+ ## Retained log
674
+
675
+ Fanout deletes a record once every target settles it, which is what a work queue wants. A log wants the opposite: records are **retained** so a consumer can rewind, a new consumer can start from the beginning, and several consumers can read the same history at different speeds. `DurabilityLog` is that second shape, and it attaches to the same scheduler.
676
+
677
+ ```ts
678
+ import { DurabilityLog, DurabilityScheduler, type LogRecord } from 'durability';
679
+
680
+ export class Partition extends DurableObject<Env> {
681
+ private readonly scheduler = new DurabilityScheduler({ context: this.ctx });
682
+ private readonly log = new DurabilityLog<Event>({
683
+ scheduler: this.scheduler,
684
+ retention: {
685
+ maxAgeMs: 7 * 24 * 60 * 60 * 1_000,
686
+ maxRecords: 100_000,
687
+ maxBytes: 512 * 1024 * 1024,
688
+ },
689
+ // Flushed records stay readable: `write` returns a locator the log keeps,
690
+ // and `read` hands the segment back when a consumer asks for it.
691
+ cold: {
692
+ write: async ({ records, firstOffset }) => {
693
+ const locator = `segment/${this.ctx.id}/${firstOffset}`;
694
+ await this.env.COLD.put(locator, JSON.stringify(records), {
695
+ onlyIf: { etagDoesNotMatch: '*' },
696
+ });
697
+ return locator;
698
+ },
699
+ read: async (locator) => {
700
+ const stored = await this.env.COLD.get(locator);
701
+ return stored ? await stored.json<LogRecord<Event>[]>() : undefined;
702
+ },
703
+ },
704
+ });
705
+
706
+ append(events: Event[]) {
707
+ return this.log.append(events.map((body) => ({ body })));
708
+ }
709
+ }
710
+ ```
711
+
712
+ `append` mirrors `enqueue`: the batch commits atomically, definite input rejections return `success: false`, and storage or commit uncertainty throws. It returns each record's **offset** in input order. Passing a `key` makes an append idempotent — re-appending an existing key returns its original offset and consumes no new one — so a producer that retries after an uncertain commit does not duplicate records.
713
+
714
+ Consumers **pull** and own their position:
715
+
716
+ ```ts
717
+ const from = (await log.cursor('search')) ?? (await log.bounds()).oldestOffset;
718
+ const page = await log.read({ from, limit: 100 });
719
+ await index(page.records);
720
+ await log.commit('search', page.nextOffset);
721
+ ```
722
+
723
+ `commit` only ever advances, so a late or duplicated commit cannot rewind a cursor. Cursors are independent: `log.cursors()` returns every consumer's position, and `log.load()` reports the furthest-behind cursor as backlog so routing can weigh partitions by consumer lag rather than by queue depth.
724
+
725
+ ### Retention is not acknowledgement
726
+
727
+ Records leave only through `trim()`, which enforces `maxAgeMs` and `maxRecords`, oldest first. Retention is deliberately not automatic, because no capability should schedule work an application did not ask for. A `DurabilityAlarms` handler is the composition that gives it a schedule, and it keeps `alarm()` a pure delegation:
728
+
729
+ ```ts
730
+ private readonly alarms = new DurabilityAlarms({
731
+ scheduler: this.scheduler,
732
+ handlers: { compact: () => this.log.trim() },
733
+ });
734
+
735
+ alarm(info?: AlarmInvocationInfo) {
736
+ return this.scheduler.alarm(info);
737
+ }
738
+ ```
739
+
740
+ `maxBytes` is the budget that actually matters, because a Durable Object holds a hot window rather than an unbounded log. It counts payload bytes held in the object and is capped at `maxLogBytes` (1 GiB), leaving room for the other capabilities sharing that object's storage.
741
+
742
+ ### Flushing to cold storage, and reading it back
743
+
744
+ Retention has two possible destinations. Without `cold`, expiring records are **deleted**. With `cold`, they are **flushed** and stay readable:
745
+
746
+ ```ts
747
+ type LogColdStorage<Body> = {
748
+ write(segment: LogSegment<Body>): Promise<string>;
749
+ read(locator: string): Promise<LogRecord<Body>[] | undefined>;
750
+ };
751
+ ```
752
+
753
+ `write` receives one contiguous run of records, oldest first, and returns a locator — any string identifying where the segment went. The log stores that locator in a durable segment index **before** deleting the records, so a crash in between leaves a written, readable segment rather than records pointing nowhere. A failed write propagates with the records still in the object, so configured storage cannot be skipped, and writes are at-least-once, so key segments by first offset and refuse to overwrite.
754
+
755
+ `read` is what makes a flush **rehydratable**. A consumer reading a flushed offset gets its records back transparently, so falling behind the hot window is no longer fatal; `bounds()` reports `oldestOffset` (oldest readable, including flushed segments) alongside `hotOffset` (lowest offset still in the object). Cold reads cost a round trip and open input gates, so they serve only consumers that fell behind — never the hot path.
756
+
757
+ External storage expires on its own schedule, so mirror a lifecycle rule with `forgetColdBefore(offset)` to drop stale index entries. Returning `undefined` from `read` is also honoured: a segment the index outlived surfaces as truncation rather than a silent gap.
758
+
759
+ Retention still **ignores cursors**, exactly as a log should: a consumer slower than everything readable loses records. It learns loudly rather than silently skipping, because reading below the oldest readable offset throws `LogTruncatedError` carrying `requestedOffset` and `oldestOffset`. Compare `bounds()` against a cursor to detect the risk before it happens.
760
+
761
+ ### Leases for parallel consumers
762
+
763
+ `read` plus `commit` is enough for one worker per consumer. It is **not** enough when two pollers share a consumer name: both read the same page and both process it. Leases close that gap.
764
+
765
+ ```ts
766
+ const lease = await log.lease('search', 100);
767
+ if (lease) {
768
+ try {
769
+ await index(lease.records);
770
+ await log.ack('search', lease.batchId);
771
+ } catch {
772
+ await log.nack('search', lease.batchId);
773
+ }
774
+ }
775
+ ```
776
+
777
+ `lease` claims a range of offsets durably **before** returning the records, so a second caller gets `undefined` rather than the same offsets. `undefined` means nothing is available right now — either the consumer is at `maxParallelism` or the log has no unleased records — so treat it as a signal to back off.
778
+
779
+ A range is held until it is acked, nacked, or its lease expires after `leaseMs`. Redelivery always issues a **fresh `batchId`**, which fences the previous holder: its `ack` throws `LogLeaseError` rather than committing a range someone else now owns. Double settlement throws for the same reason.
780
+
781
+ `maxParallelism` lets one consumer hold several ranges at once, and `ack` commits only over an **unbroken run** of settled ranges starting at the cursor. Acking a later range while an earlier one is still in flight advances nothing, so parallelism never skips offsets. Leases are durable, so eviction mid-flight does not release a range early, and a range that retention flushed while it was held is rehydrated on redelivery.
782
+
783
+ A range that keeps failing would be redelivered forever. `maxAttempts` bounds that, but only together with `onPoison`: without somewhere to send the records, blocking the consumer beats silently dropping them. With both set, an exhausted range is marked skipped — durably, before the callback runs, so a failing handler cannot resurrect it — and the commit then runs over it like an acked one.
784
+
785
+ Leases are opt-in. Without the `leases` option no lease tables are created, `read`/`commit` behave exactly as before, and the lease methods throw.
786
+
787
+ ### What it is not
788
+
789
+ There is no per-record retry, timeout, or dead letter here: a log tracks positions, not attempts. When a single record must be retried independently of its neighbours, that is fanout's job, and the two compose on one object.
790
+
791
+ There is also no key compaction. `deduplicationKey` is unique per retained record, which is the opposite of a compaction key, and deduplication lasts only while the original record is retained — re-appending a key after its record was flushed or deleted creates a new record.
792
+
417
793
  ## Long-running alarm calls
418
794
 
419
- Alarm invocations have a 15-minute wall-time limit. If work is still pending after `alarmHandoffMs` (default 14 minutes), the scheduler retains its promise in memory, arms an immediate alarm, and returns from the current invocation. The next alarm attaches to the same promise instead of starting the handler again. If the object is evicted during a handoff, the next alarm reconstructs and executes the persisted pending work.
795
+ Alarm invocations have a 15-minute wall-time limit. If work is still pending after `alarmHandoffMs` (default 14 minutes), the scheduler retains its promise in memory, arms a fallback after `alarmMinDelayMs`, and returns from the current invocation. The next alarm attaches to the same promise instead of starting the handler again. If the object is evicted during a handoff, the next alarm reconstructs and executes the persisted pending work.
@@ -0,0 +1,94 @@
1
+ import { type SerializedError } from './errors.js';
2
+ import { type AnySocketShape, type InboundOf, type OutboundOf, type SocketInboundInput, type SocketInboundOutput, type SocketOutboundPayload } from './socket-protocol.js';
3
+ /** Rejection for calls still waiting when `close()` was called. */
4
+ export declare class SocketClosedError extends Error {
5
+ constructor();
6
+ }
7
+ /**
8
+ * Rejection for calls that were sent but unanswered when the connection
9
+ * dropped. They are never replayed: the server may or may not have run them.
10
+ */
11
+ export declare class SocketConnectionLostError extends Error {
12
+ constructor(method: string);
13
+ }
14
+ /** Rejection for calls unanswered within the configured timeout. */
15
+ export declare class SocketTimeoutError extends Error {
16
+ constructor(method: string, timeoutMs: number);
17
+ }
18
+ /** The server sent a frame this client cannot interpret; the connection is abandoned. */
19
+ export declare class SocketProtocolError extends Error {
20
+ constructor(message: string);
21
+ }
22
+ /** A handler on the server threw; `name` and `message` are the server's. */
23
+ export declare class SocketRemoteError extends Error {
24
+ constructor(error: SerializedError);
25
+ }
26
+ export type SocketStatus = 'connecting' | 'connected' | 'disconnected' | 'closed';
27
+ /** Wraps every `rpc` call; `next` forwards with the same or a replaced input. */
28
+ export type SocketTransform = (call: {
29
+ method: string;
30
+ input: unknown;
31
+ next: (input?: unknown) => Promise<unknown>;
32
+ }) => Promise<unknown>;
33
+ export type DurabilitySocketClientConfig = {
34
+ /** Builds the WebSocket URL for one Durable Object. */
35
+ url: (id: string) => string;
36
+ /** Rejects unanswered calls after this long. Defaults to 2 minutes. */
37
+ timeoutMs?: number;
38
+ /** Interval between keep-alive pings; a silent connection is dropped after two. Defaults to 30 seconds. */
39
+ pingIntervalMs?: number;
40
+ /** WebSocket implementation. Defaults to the global `WebSocket`. */
41
+ webSocket?: new (url: string) => WebSocket;
42
+ };
43
+ type RpcMethod<Input, Output> = [Input] extends [void] ? () => Promise<Output> : unknown extends Input ? (input?: unknown) => Promise<Output> : (input: Input) => Promise<Output>;
44
+ type RpcMethods<Inbound> = {
45
+ [Method in keyof Inbound]: RpcMethod<SocketInboundInput<Inbound[Method]>, SocketInboundOutput<Inbound[Method]>>;
46
+ };
47
+ type EventHandlers<Outbound> = {
48
+ [Name in keyof Outbound]: (handler: (payload: SocketOutboundPayload<Outbound[Name]>) => void) => () => void;
49
+ } & {
50
+ /** Observes connection status; the handler runs immediately with the current value. */
51
+ status: (handler: (status: SocketStatus) => void) => () => void;
52
+ /**
53
+ * Fires when subscribers to `names` have no continuous history: on their
54
+ * first subscription, and whenever missed events could not be replayed.
55
+ * Rebuild their state from a snapshot.
56
+ */
57
+ resync: (handler: (info: SocketResync) => void) => () => void;
58
+ };
59
+ export type SocketResync = {
60
+ names: string[];
61
+ };
62
+ /** One connection to one Durable Object. */
63
+ export type SocketConnection<Socket extends AnySocketShape> = {
64
+ rpc: RpcMethods<InboundOf<Socket>>;
65
+ on: EventHandlers<OutboundOf<Socket>>;
66
+ readonly status: SocketStatus;
67
+ /** Permanently closes the connection and rejects outstanding calls. */
68
+ close(): void;
69
+ };
70
+ /**
71
+ * Browser client for a {@link DurabilitySocket}. Connections open lazily,
72
+ * reconnect with jittered backoff, and resubscribe from the last seen event.
73
+ *
74
+ * @example
75
+ * ```ts
76
+ * const client = new DurabilitySocketClient<TodoList['socket']>({
77
+ * url: (id) => `wss://example.com/todos/${id}`,
78
+ * });
79
+ * const conn = client.connect('personal');
80
+ * const todos = await conn.rpc.list();
81
+ * conn.on.created((todo) => render(todo));
82
+ * conn.on.resync(() => reload());
83
+ * ```
84
+ */
85
+ export declare class DurabilitySocketClient<Socket extends AnySocketShape> {
86
+ #private;
87
+ constructor(config: DurabilitySocketClientConfig, transforms?: readonly SocketTransform[]);
88
+ /** Returns a client that also applies `transform` around every call. */
89
+ with(transform: SocketTransform): DurabilitySocketClient<Socket>;
90
+ connect(id: string): SocketConnection<Socket>;
91
+ }
92
+ /** Logs every call's input, output or error, and duration. */
93
+ export declare const logging: (log?: (entry: Record<string, unknown>) => void) => SocketTransform;
94
+ export { socketShape, type AnySocketShape, type InboundOf, type OutboundOf, type SocketInboundInput, type SocketInboundOutput, type SocketOutboundPayload, type SocketShape, } from './socket-protocol.js';