@ultimat3/jobs 8.0.0 → 10.0.0

package/CLAUDE.md

@@ -334,6 +334,29 @@ Tier 3. The `job` + `task` primitives, durable steps, transactional outbox, queu
   jumps over a page the live iteration never read. A checkpoint READ back is checked rather than
   trusted — `step.run` replays it through an unchecked `as T`, and an absent cursor is not `null`,
   so the pass would silently reopen the source at the top and walk the whole table again.
+- **`purge()` is a FACTORY over `job()` too, and it is the ONE caller every `purgeExpired()` in
+  the framework was missing** (`As of 2026-08-22`). Three stores shipped one —
+  `postgresIdempotencyStore` (`x_idempotency`), `postgresRateLimitStore` (`x_rate_limit`) and
+  `postgresAuthLimiter` (`x_auth_failures`/`x_auth_lockouts`) — each documented as "an app runs
+  this from a `task`", and a task only ENQUEUES, so there was no job for one to enqueue and every
+  row written was a row kept. `x_rate_limit` takes one upsert per HTTP request the web role serves,
+  assets included.
+
+  `PurgeTarget` is STRUCTURAL (`{ name, purgeExpired(nowMs) }`) for the reason `PgExecutor` is: two
+  of those three packages are below this one and one is beside it, and a sweep that needed their
+  types would put the HTTP pipeline on this package's import graph. `targets()` is a THUNK, read
+  once per attempt: a host declares the sweep at boot and the auth limiter does not exist yet —
+  `defineAuth` builds it when the app's modules import. One table per `step.run`, so a killed
+  attempt resumes at the table it stopped on; a purge is idempotent by nature, so the replay that
+  at-least-once guarantees deletes rows that are already gone. **One clock reading for the whole
+  pass**, handed to every target: `postgresRateLimitStore.purgeExpired(nowMs)` requires the
+  CALLER's clock, and reading the server's computed a 20,000,000-second refill against a frozen
+  test clock and deleted a bucket holding 0 of 4 tokens — a free limit reset, handed out by the
+  cleanup. Two targets under one name are refused (`X_INVARIANT`) before the first delete rather
+  than discovered as `X_STEP_DUPLICATE` after one table is already empty.
+
+  It declares no schedule of its own: `DEFAULT_PURGE_CRON` is the hourly cron a host's `task()`
+  uses, and `@ultimat3/cli`'s `dev-purge.ts` is the one that declares both halves at boot.
 - **`handle` is AT LEAST ONCE, and the ordering that makes it so is deliberate.** The body runs
   inside the step and the record is written after it returns, so an attempt killed, cancelled or
   lease-expired between the two hands that page to the next attempt — which is why the doc comment,
@@ -687,6 +710,7 @@ picture from the other side.
 | `worker-run.ts` | one claimed job, wired: its heartbeat, its slot renewal, its run signal and its span, started together and handed back in one `finally` |
 | `run-signal.ts` | the signal ONE run is cancelled by — composition that can be handed back, and that the worker can abort itself |
 | `worker-fleet-slots.ts` | the fleet slot an in-flight job holds — take, renew, hand back. The claim loop asks "may I start this one?"; this answers it across the fleet |
+| `purge.ts` | `purge()` — a factory over `job()`: the retention sweep, its structural target seam and the hourly cron a host schedules it on |
 | `task.ts` | the `task()` primitive + registry + the handle's surface + `registerTask` |
 | `scheduler.ts` | `scheduler` role: the dispatch round, catch-up, leader election, the drain |
 | `limits.ts` | per-tenant / per-queue / global concurrency + rate |

package/README.md

@@ -298,6 +298,45 @@ the new pods serve, puts the sweeps on the queue and exits, and a slow UPDATE ne
 open against a database still serving the previous build. `--all` isolates per name and continues
 past a failure, so one wedged cleanup cannot block every later one forever.
 
+## Retention sweeps are jobs too
+
+`purge()` is the **second factory over `job()`**, and it exists because three framework stores
+shipped a `purgeExpired()` with no caller — `x_idempotency`, `x_rate_limit` and the auth pair each
+kept every row they ever took. `x_rate_limit` takes one upsert per HTTP request the web role
+serves, assets included, so its growth follows total traffic and not traffic that hit a limit.
+
+```ts
+import { DEFAULT_PURGE_CRON, purge, task } from '@ultimat3/jobs';
+
+declare const store: { purgeExpired(nowMs: number): Promise<number> };
+
+export const sweep = purge({
+  name: 'x.purge',
+  // Read once per ATTEMPT, never captured: a host declares the sweep at boot, and some of the
+  // stores behind it are built later.
+  targets: () => [{ name: 'x_rate_limit', purgeExpired: (nowMs) => store.purgeExpired(nowMs) }],
+});
+
+export const hourly = task({
+  name: 'x.purge.hourly',
+  cron: DEFAULT_PURGE_CRON,
+  tz: 'UTC',
+  enqueue: () => [[sweep, {}]],
+});
+```
+
+| Rule | Why |
+|---|---|
+| `PurgeTarget` is structural | the stores live in `@ultimat3/action`, `@ultimat3/http` and `@ultimat3/auth`; importing them would put the HTTP pipeline on this package's graph |
+| one `step.run` per target | a killed attempt resumes at the table it stopped on, not at the first |
+| one clock reading per pass | `postgresRateLimitStore.purgeExpired(nowMs)` needs the CALLER's clock — the server's read a 20,000,000-second refill against a frozen one and deleted a live bucket |
+| at least once is safe here | a replayed delete removes rows that are already gone, and a row a purge deleted answers exactly as one that was never there |
+| two targets under one name | `X_INVARIANT`, before the first delete — `step.run` would raise `X_STEP_DUPLICATE` after one table was already empty |
+
+`@ultimat3/cli`'s boot declares both halves over the three tables it owns, so an app gets the sweep
+without writing any of the above. It needs a `worker` to run it and a `scheduler` to fire it: a
+deployment with neither has no background work at all, and this is one more thing it does not do.
+
 ## The deadline cancels
 
 A job's `timeout` aborts `ctx.signal` **before** it fails the attempt, because the nack that

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/jobs",
-  "version": "8.0.0",
+  "version": "10.0.0",
   "description": "Durable background work: steps, transactional outbox, cron tasks, one driver interface",
   "license": "MIT",
   "type": "module",
@@ -32,9 +32,9 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/core": "8.0.0",
-    "@ultimat3/entity": "8.0.0",
-    "@ultimat3/schema": "8.0.0",
-    "@ultimat3/time": "8.0.0"
+    "@ultimat3/core": "10.0.0",
+    "@ultimat3/entity": "10.0.0",
+    "@ultimat3/schema": "10.0.0",
+    "@ultimat3/time": "10.0.0"
   }
 }

package/src/backfill-errors.ts

@@ -6,7 +6,6 @@
 // `backfill-pending.ts` and `backfill-registry.ts`.
 
 import { UltimateError } from '@ultimat3/core';
-import { docsFor } from './errors';
 
 /**
  * The seven backfill codes below all answer one question — "why is this sweep not running?" — and
@@ -31,7 +30,6 @@ export class BackfillPendingError extends UltimateError {
       code: 'X_BACKFILL_PENDING',
       cause: `backfill "${input.backfill}" is declared and x_backfills holds no completed pass for it in ${input.environment}`,
       fix: `x db backfill ${input.backfill} --write --json`,
-      docs: docsFor('X_BACKFILL_PENDING'),
     });
   }
 }
@@ -43,7 +41,6 @@ export class BackfillAppliedError extends UltimateError {
       code: 'X_BACKFILL_APPLIED',
       cause: `backfill "${input.backfill}" completed as run ${input.runId} at ${input.completedAt}; a forced rerun writes a NEW ledger row and never edits that one`,
       fix: `x db backfill ${input.backfill} --write --force --json`,
-      docs: docsFor('X_BACKFILL_APPLIED'),
     });
   }
 }
@@ -68,7 +65,6 @@ export class BackfillEnvironmentError extends UltimateError {
         target === undefined
           ? 'x db backfill --pending --json'
           : `ULTIMATE_ENV=${target} x db backfill ${input.backfill} --write --json`,
-      docs: docsFor('X_BACKFILL_ENVIRONMENT'),
     });
   }
 }
@@ -84,7 +80,6 @@ export class BackfillMigrationPendingError extends UltimateError {
       code: 'X_BACKFILL_MIGRATION_PENDING',
       cause: `backfill "${input.backfill}" requires migration ${input.migration}, which x_migrations does not record as applied`,
       fix: 'x db migrate --json',
-      docs: docsFor('X_BACKFILL_MIGRATION_PENDING'),
     });
   }
 }
@@ -100,7 +95,6 @@ export class BackfillRunningError extends UltimateError {
       code: 'X_BACKFILL_RUNNING',
       cause: `backfill "${input.backfill}" already has a live pass queued as ${input.jobId}, and one name holds one live pass; its step trace names the batch it is on, and a pass that is not advancing is a worker that lost its lease`,
       fix: `x jobs show ${input.jobId} --json`,
-      docs: docsFor('X_BACKFILL_RUNNING'),
     });
   }
 }
@@ -116,7 +110,6 @@ export class BackfillStalledError extends UltimateError {
       code: 'X_BACKFILL_STALLED',
       cause: `backfill "${input.backfill}" swept ${input.swept} rows, exhausted its source, and count() still matches ${input.remaining} — a WHERE the sweep narrows and the count does not is what leaves rows behind`,
       fix: `make count() select on exactly what source() selects on in backfill("${input.backfill}")`,
-      docs: docsFor('X_BACKFILL_STALLED'),
     });
   }
 }
@@ -131,7 +124,6 @@ export class BackfillUnknownError extends UltimateError {
           ? `no backfill named "${input.backfill}" is declared, and this app declares none at all`
           : `no backfill named "${input.backfill}" is declared (declared: ${input.known.join(', ')})`,
       fix: 'x db backfill --pending --json',
-      docs: docsFor('X_BACKFILL_UNKNOWN'),
     });
   }
 }

package/src/backfill-pass.ts

@@ -13,7 +13,7 @@
 // in step with the work and decides where a resumed pass restarts, while the `x_backfills` row is
 // a report an operator reads and the record that a completed name has already been swept.
 
-import { appVersion, assert, logger, resolveEnvironment } from '@ultimat3/core';
+import { appVersion, assert, logger, renderThrowable, resolveEnvironment } from '@ultimat3/core';
 import type { BatchIterator } from '@ultimat3/entity';
 import type { BackfillDefinition, BackfillInput, BackfillReport } from './backfill';
 import { BackfillStalledError } from './backfill-errors';
@@ -101,7 +101,7 @@ async function markFailed(
   } catch (error) {
     logger.warn('jobs.backfill.ledger-failed', {
       runId,
-      error: error instanceof Error ? error.message : String(error),
+      error: renderThrowable(error),
     });
   }
 }

package/src/errors.ts

@@ -93,8 +93,12 @@ registerErrorRetry({
   X_BACKFILL_APPLIED: 'terminal',
 });
 
-/** Shared with `backfill-errors.ts`, which holds the seven `X_BACKFILL_*` classes. */
-export const docsFor = (code: JobErrorCode): string => `https://ultimate.dev/errors/${code}`;
+// No `docs:` on any class below, here or in `backfill-errors.ts`. `UltimateError` fills it from
+// `describeErrorCode(code).docs`, which is `@ultimat3/core`'s `ERROR_DOCS_URL` — one page for every
+// code, never one per code, because `wiki/` is the framework's only public documentation surface
+// and a code lives there in a TABLE ROW, which has no anchor. The `docsFor` that stood here built
+// `https://ultimate.dev/errors/<code>`, which answered 404, host included, on every job failure
+// this package has ever put in a dead-letter row.
 
 /** An enqueue collided with a live job holding the same idempotency key under `onConflict: 'error'`. */
 export class JobDuplicateError extends UltimateError {
@@ -103,7 +107,6 @@ export class JobDuplicateError extends UltimateError {
       code: 'X_JOB_DUPLICATE',
       cause: `job "${input.job}" already queued as ${input.existingId} with idempotencyKey "${input.idempotencyKey}"`,
       fix: 'pass onConflict: "dedupe" to enqueue, or make idempotencyKey narrower',
-      docs: docsFor('X_JOB_DUPLICATE'),
     });
   }
 }
@@ -124,7 +127,6 @@ export class JobNameTakenError extends UltimateError {
       code: 'X_JOB_DUPLICATE',
       cause: `two ${input.kind}s claim the name "${input.name}"`,
       fix: `x jobs ls --json names the one already seated; rename the other's export, or its "name:" if it declares one — a ${input.kind} name is a durable queue key and is globally unique`,
-      docs: docsFor('X_JOB_DUPLICATE'),
     });
   }
 }
@@ -151,7 +153,6 @@ export class JobRowStatusUnknownError extends UltimateError {
         `${input.table}.${input.column} holds "${input.value}", which this build does not know — ` +
         `it reads ${input.known.join(', ')}`,
       fix: `x jobs show --json   # then drain the older workers: a status this build cannot read was almost certainly written by a newer deploy`,
-      docs: docsFor('X_JOB_ROW_STATUS_UNKNOWN'),
     });
   }
 }
@@ -175,7 +176,6 @@ export class ActionJobUnbridgedError extends UltimateError {
       code: 'X_ACTION_JOB_UNBRIDGED',
       cause: `export "${input.export}" is the action projection "${input.job}", which is not a job handle and cannot be registered as one`,
       fix: `wrap it: agentJob(${input.export}, { name: '${input.export}', tenant, retry }) from @ultimat3/ai — that composes job() and returns a handle the queue accepts`,
-      docs: docsFor('X_ACTION_JOB_UNBRIDGED'),
     });
   }
 }
@@ -187,7 +187,6 @@ export class StepDuplicateError extends UltimateError {
       code: 'X_STEP_DUPLICATE',
       cause: `job "${input.job}" used step name "${input.step}" twice in one run`,
       fix: `rename one of them, e.g. step.run('${input.step}-2', ...) — step names are the replay key`,
-      docs: docsFor('X_STEP_DUPLICATE'),
     });
   }
 }
@@ -201,7 +200,6 @@ export class JobTimeoutError extends UltimateError {
           ? `job "${input.job}" exceeded its ${input.timeoutMs}ms timeout`
           : `job "${input.job}" step "${input.step}" exceeded its ${input.timeoutMs}ms timeout`,
       fix: `raise timeout on the job definition, or split the work into step.run() calls`,
-      docs: docsFor('X_JOB_TIMEOUT'),
     });
   }
 }
@@ -225,7 +223,6 @@ export class JobAbortedError extends UltimateError {
           ? `job "${input.job}" was cancelled — this attempt no longer owns the run`
           : `job "${input.job}" was cancelled before step "${input.step}" could be recorded`,
       fix: 'add throwIfAborted(ctx) before expensive work, or pass fetch(url, { signal: ctx.signal }) — the queue re-runs the job, so stop at the deadline instead of running past it',
-      docs: docsFor('X_ABORTED'),
     });
   }
 }
@@ -237,7 +234,6 @@ export class JobMaxAttemptsError extends UltimateError {
       code: 'X_JOB_MAX_ATTEMPTS',
       cause: `job "${input.job}" failed ${input.attempts} times, last error: ${input.lastError}`,
       fix: `x jobs retry ${input.jobId}`,
-      docs: docsFor('X_JOB_MAX_ATTEMPTS'),
     });
   }
 }
@@ -248,7 +244,6 @@ export class DriverUnavailableError extends UltimateError {
       code: 'X_DRIVER_UNAVAILABLE',
       cause: `jobs driver "${input.driver}" is unavailable: ${input.cause}`,
       fix: input.fix,
-      docs: docsFor('X_DRIVER_UNAVAILABLE'),
     });
   }
 }
@@ -263,7 +258,6 @@ export class IdempotencyRequiredError extends UltimateError {
       code: 'X_IDEMPOTENCY_REQUIRED',
       cause: `job "${input.job}" has no idempotencyKey — at-least-once delivery would run it twice`,
       fix: `add idempotencyKey: (input) => \`${input.job}:\${input.id}\` to the job definition`,
-      docs: docsFor('X_IDEMPOTENCY_REQUIRED'),
     });
   }
 }
@@ -290,7 +284,6 @@ export class JobTenantRequiredError extends UltimateError {
       // tenant, and the pass opens the cross-tenant scope for exactly that declaration. Half the
       // callers of this code arrive through `backfill()`, which forwards its `tenant` to `job()`.
       fix: `add tenant: (input) => input.orgId to job("${input.job}") — or tenant: 'none', which declares NO org: right for a job that touches no tenant-scoped table, and the spelling a backfill() uses to sweep every tenant`,
-      docs: docsFor('X_JOB_TENANT_REQUIRED'),
     });
   }
 }
@@ -307,7 +300,6 @@ export class LeaseLostError extends UltimateError {
       code: 'X_JOB_LEASE_LOST',
       cause: `job "${input.job}" (${input.jobId}) is no longer claimed by this worker — it was cancelled, or its visibility lease lapsed and the queue re-delivered it`,
       fix: `x jobs show ${input.jobId} --json`,
-      docs: docsFor('X_JOB_LEASE_LOST'),
     });
   }
 }
@@ -325,7 +317,6 @@ export class JobSlotLostError extends UltimateError {
       code: 'X_JOB_SLOT_LOST',
       cause: `job "${input.job}" (${input.jobId}) no longer holds fleet concurrency slot ${input.slot} — its lease expired and another worker took it`,
       fix: `x jobs show ${input.jobId} --json`,
-      docs: docsFor('X_JOB_SLOT_LOST'),
     });
   }
 }
@@ -344,7 +335,6 @@ export class JobNotCancellableError extends UltimateError {
           ? `no job ${input.jobId} exists in this queue`
           : `job ${input.jobId} is "${input.state}" and only a job that has not finished can be cancelled`,
       fix: `x jobs ls --state running --json`,
-      docs: docsFor('X_JOB_NOT_CANCELLABLE'),
     });
   }
 }
@@ -356,7 +346,6 @@ export class CancelUnsupportedError extends UltimateError {
       code: 'X_JOB_NOT_CANCELLABLE',
       cause: `the "${input.driver}" jobs driver cannot cancel a single job`,
       fix: 'call setJobDriver(createPgDriver()) at boot — only the pg driver implements introspect.cancel — then: x jobs cancel <id> --json',
-      docs: docsFor('X_JOB_NOT_CANCELLABLE'),
     });
   }
 }
@@ -373,7 +362,6 @@ export class ConcurrencyUnenforceableError extends UltimateError {
       code: 'X_JOB_CONCURRENCY_UNENFORCEABLE',
       cause: `${input.jobs.join(', ')} declare concurrency and the "${input.driver}" jobs driver has no lease store, so the cap would hold per process and the fleet would run concurrency x replicas`,
       fix: `remove concurrency from job("${input.jobs[0] ?? 'the job'}"), or call setJobDriver(createPgDriver()) at boot — the pg driver is the one with a lease store`,
-      docs: docsFor('X_JOB_CONCURRENCY_UNENFORCEABLE'),
     });
   }
 }
@@ -385,7 +373,6 @@ export class OutboxNoTxError extends UltimateError {
       code: 'X_OUTBOX_NO_TX',
       cause: `ctx.jobs.enqueue(${input.job}) ran outside a transaction with outbox: 'required'`,
       fix: 'wrap the call in ctx.tx(async (tx) => ...), or enqueue with { outbox: false }',
-      docs: docsFor('X_OUTBOX_NO_TX'),
     });
   }
 }
@@ -396,7 +383,6 @@ export class JobsNotImplementedError extends UltimateError {
       code: 'X_NOT_IMPLEMENTED',
       cause: `${input.feature} is declared but not implemented in @ultimat3/jobs`,
       fix: input.fix,
-      docs: docsFor('X_NOT_IMPLEMENTED'),
     });
   }
 }

package/src/events-pg.ts

@@ -7,7 +7,7 @@
 // resumes at 12:00:30 must still see an event published at 12:00:10.
 
 import type { Clock } from '@ultimat3/core';
-import { logger, systemClock, uuid } from '@ultimat3/core';
+import { logger, renderThrowable, systemClock, uuid } from '@ultimat3/core';
 import type { DurationInput } from './clock';
 import { nowMs, toMs } from './clock';
 import type { PgExecutor } from './driver-pg';
@@ -54,7 +54,7 @@ export function createPgEventBus(options: PgEventBusOptions): EventBus {
     void exec.query(SQL_EVENT_PURGE, []).catch((error: unknown) => {
       // Housekeeping never costs a publish: an unpurged row is filtered out of every read.
       logger.warn('jobs.event.purge-failed', {
-        error: error instanceof Error ? error.message : String(error),
+        error: renderThrowable(error),
       });
     });
     return 0;

package/src/execute.ts

@@ -10,6 +10,7 @@ import {
   anonymousActor,
   isUltimateError,
   logger,
+  renderThrowable,
   reportError,
   runWithContext,
   useContext,
@@ -22,6 +23,7 @@ import { eventBus } from './events';
 import type { AnyJobHandle } from './job';
 import type { JobStopReason } from './retry-classification';
 import { nextRetryForError, recordedFailure } from './retry-classification';
+import { createRunSignal } from './run-signal';
 import type { EventLookup, StepRecord } from './steps';
 import { createStepRunner, isStepSuspension } from './steps';
 import { jobRunActor } from './tenant';
@@ -100,7 +102,14 @@ export async function executeJob(options: ExecuteJobOptions): Promise<JobExecuti
   // where every other body does, with no jobs-only parameter to know about. Composed with the
   // caller's signal rather than replacing it: a ctx that was already going away still is.
   const cancel = new AbortController();
-  const signal = AbortSignal.any([callerSignal(options.ctx), cancel.signal]);
+  // `createRunSignal` and never `AbortSignal.any` — the second of the two sites this package's
+  // `CLAUDE.md` states the rule as absolute for. In the worker path `callerSignal` is per-run and
+  // nothing leaks; on `@ultimat3/testing`'s job-fixture path, which calls `executeJob` directly,
+  // the caller's `ctx.signal` may be process-lifetime, and a composite cannot be undone — so every
+  // job a fixture ran left a dependent signal on it for the life of the process. Disposed in the
+  // `finally` at the bottom of the try, beside `cancel.abort`.
+  const runSignal = createRunSignal([callerSignal(options.ctx), cancel.signal]);
+  const signal = runSignal.signal;
   const ctx: Ctx = Object.freeze({
     ...options.ctx,
     signal,
@@ -178,7 +187,7 @@ export async function executeJob(options: ExecuteJobOptions): Promise<JobExecuti
       });
     }
 
-    const message = error instanceof Error ? error.message : String(error);
+    const message = renderThrowable(error);
     // The ERROR decides too, not only the attempt count. A `terminal` code — a rotated password,
     // a schema mismatch, a permission denial — fails identically on every remaining attempt, so
     // spending them is a queue slot, a provider bill and, at a site that locks an account after
@@ -238,6 +247,9 @@ export async function executeJob(options: ExecuteJobOptions): Promise<JobExecuti
     // attempt already owns. The runner fences its writes on this signal. A second `abort()` keeps
     // the first reason, so a timed-out run still reports the timeout, not this.
     cancel.abort(new JobAbortedError({ job: handle.name }));
+    // AFTER the abort, so the runner's fence still sees it: `dispose` stops following the sources,
+    // it never aborts, and the signal keeps whatever state the abort above left it in.
+    runSignal.dispose();
   }
 
   // Only reachable when the BODY succeeded, and settlement is deliberately outside the catch
@@ -287,9 +299,7 @@ function raceTimeout(
         job,
         timeoutMs,
         ended,
-        ...(error === undefined
-          ? {}
-          : { error: error instanceof Error ? error.message : String(error) }),
+        ...(error === undefined ? {} : { error: renderThrowable(error) }),
       });
     };
     work.then(

package/src/heartbeat.ts

@@ -4,7 +4,7 @@
 // hardest bug in a queue to see from the outside and the easiest to name from in here.
 
 import type { Clock } from '@ultimat3/core';
-import { logger, recordLeaseLost } from '@ultimat3/core';
+import { logger, recordLeaseLost, renderThrowable } from '@ultimat3/core';
 import { nowMs } from './clock';
 import type { ClaimedJob, JobDriver } from './driver';
 import { LeaseLostError } from './errors';
@@ -76,9 +76,7 @@ export function startLeaseHeartbeat(options: LeaseHeartbeatOptions): LeaseHeartb
       attempt: claimed.attempt,
       visibilityTimeoutMs,
       reason: reason ?? 'expired',
-      ...(error === undefined
-        ? {}
-        : { error: error instanceof Error ? error.message : String(error) }),
+      ...(error === undefined ? {} : { error: renderThrowable(error) }),
     });
     recordLeaseLost(claimed.queue);
     // Cancel LAST, so the loss is logged and counted before the body it unwinds starts throwing.
@@ -134,7 +132,7 @@ export function startLeaseHeartbeat(options: LeaseHeartbeatOptions): LeaseHeartb
         job: claimed.name,
         jobId: claimed.id,
         attempt: claimed.attempt,
-        error: error instanceof Error ? error.message : String(error),
+        error: renderThrowable(error),
       });
       if (lapsed()) reportLost(error);
     } finally {

package/src/index.ts

@@ -221,6 +221,14 @@ export { DEFAULT_OUTBOX_CLAIM_LEASE_MS } from './outbox-lease';
 export type { PgOutboxOptions } from './outbox-pg';
 export { createPgOutboxStore } from './outbox-pg';
 
+export type {
+  PurgeDefinition,
+  PurgeInput,
+  PurgeReport,
+  PurgeSweep,
+  PurgeTarget,
+} from './purge';
+export { DEFAULT_PURGE_CRON, purge } from './purge';
 export type { BackoffStrategy, Random, RetryDecision, RetryPolicy } from './retry';
 export { backoffDelayMs, DEFAULT_RETRY, nextRetry, retrySchedule } from './retry';
 export type { JobRetryDecision, JobStopReason } from './retry-classification';

package/src/outbox.ts

@@ -24,7 +24,7 @@
 // test and `x dev` must enqueue with nothing wired — but it is a fallback, not the guarantee.
 
 import type { Clock } from '@ultimat3/core';
-import { currentSpanContext, logger, traceparent, uuid } from '@ultimat3/core';
+import { currentSpanContext, logger, renderThrowable, traceparent, uuid } from '@ultimat3/core';
 import type { Tx } from '@ultimat3/entity';
 import { nowMs } from './clock';
 import type { EnqueueResult, JobDriver } from './driver';
@@ -421,7 +421,7 @@ export function createOutboxRelay(options: RelayOptions): OutboxRelay {
           id: record.id,
           published,
           remaining: batch.length - published,
-          error: error instanceof Error ? error.message : String(error),
+          error: renderThrowable(error),
         });
         // Hand the rest of the batch back rather than sit on a claim nobody is publishing. The
         // claim is a lease now, so without this a single pool timeout parks every committed row
@@ -456,7 +456,7 @@ export function createOutboxRelay(options: RelayOptions): OutboxRelay {
           .then((): void => undefined)
           .catch((error: unknown) => {
             logger.error('jobs.outbox.tick-failed', {
-              error: error instanceof Error ? error.message : String(error),
+              error: renderThrowable(error),
             });
           })
           .finally(() => {

package/src/purge.ts

@@ -0,0 +1,150 @@
+// `purge()` — the framework's retention sweep, declared as a `job` and NOT as a ninth primitive.
+// Deleting expired rows on a schedule is durable background work with an input schema, a retry
+// policy, an idempotency key and a queue, which is the definition of a `job` — so this file is a
+// FACTORY over `job()`, exactly as `backfill()` is one and `llm()` is one over `action()`. That is
+// what gives a retention sweep `.enqueue()`, the worker's cancellation, the dead-letter path,
+// `x jobs show` and a manifest row without a line here.
+//
+// WHY it exists: `postgresIdempotencyStore`, `postgresRateLimitStore` and `postgresAuthLimiter`
+// each shipped a `purgeExpired()` and NOTHING called any of them, so every row those three tables
+// ever took was a row kept. `x_rate_limit` takes one upsert per HTTP request a web role serves,
+// assets included, so its growth is proportional to total traffic rather than to traffic that hit
+// a limit. A `task` could not fix it: a task only ENQUEUES, which is this package's design.
+
+import type { Clock } from '@ultimat3/core';
+import { assert, logger } from '@ultimat3/core';
+import { t } from '@ultimat3/schema';
+import type { DurationInput } from './clock';
+import { nowMs } from './clock';
+import type { JobHandle } from './job';
+import { job } from './job';
+import type { RetryPolicy } from './retry';
+import { DEFAULT_RETRY } from './retry';
+
+/**
+ * One table's worth of expired rows, behind the narrowest possible seam.
+ *
+ * Structural, exactly like `JobActor` and `PgExecutor`: the three stores this was written for live
+ * in `@ultimat3/action` (this tier), `@ultimat3/http` and `@ultimat3/auth` — none of them
+ * importable here — and a sweep that needed their types would put the whole HTTP pipeline on this
+ * package's import graph. A store satisfies this by having the method it already has.
+ */
+export interface PurgeTarget {
+  /**
+   * What this sweep is called in its durable step, its log line and its report. A table name is
+   * the natural spelling (`x_rate_limit`); a target that clears a SET of tables names their common
+   * prefix (`x_auth`, for `x_auth_failures` and `x_auth_lockouts`). Unique within one definition —
+   * the name is the step key, and two steps under one name is `X_STEP_DUPLICATE` mid-run.
+   */
+  readonly name: string;
+  /**
+   * Delete every expired row and answer how many went.
+   *
+   * `nowMs` is the JOB's clock, and a store that writes its instants from the caller MUST measure
+   * against it rather than against `now()` on the server. That mismatch is not theoretical: the
+   * http store's purge read `extract(epoch from now())` against a `last_ms` written by the caller
+   * and, on a frozen test clock, computed a 20,000,000-second refill and deleted a bucket holding
+   * 0 of 4 tokens — a free limit reset, handed out by the cleanup. A store that holds its own
+   * clock (because its host handed it one) may ignore this argument; a store that holds none
+   * may not.
+   */
+  purgeExpired(nowMs: number): Promise<number>;
+}
+
+/** What one target's sweep removed. Bounded and JSON-safe, so it survives as a step's output. */
+export interface PurgeSweep {
+  readonly name: string;
+  readonly removed: number;
+}
+
+/** What one pass reports — bounded, so `x jobs show` can print it. */
+export interface PurgeReport {
+  readonly swept: readonly PurgeSweep[];
+  readonly removed: number;
+}
+
+/**
+ * A purge decides nothing, so its payload carries nothing. Deliberately not a `force` flag like
+ * `BackfillInput`'s: a backfill is a ONE-PASS sweep whose ledger says it already ran, and this is
+ * a recurring one with no ledger and nothing to override.
+ */
+export type PurgeInput = Readonly<Record<string, never>>;
+
+export interface PurgeDefinition {
+  /**
+   * Omit it and `defineApi({ jobs })` assigns the export name. A framework-owned sweep pins one,
+   * the way `mail.send` does, because the queue key is what rows already carry.
+   */
+  readonly name?: string;
+  /**
+   * The tables to sweep, read ONCE PER ATTEMPT rather than captured at declaration. Lazy because
+   * a host declares the sweep at boot and the stores behind it are not all resolved yet — an
+   * app's `defineAuth` runs after the boot that installed the limiter factory, so the auth target
+   * does not exist until later. An empty list is a pass that removes nothing, which is the honest
+   * answer for a process whose boot has already stopped.
+   */
+  targets(): readonly PurgeTarget[];
+  /**
+   * The clock every target is measured against. Defaults to the system clock, and it must be the
+   * SAME clock the stores write their instants from — see `PurgeTarget.purgeExpired`.
+   */
+  readonly clock?: Clock;
+  readonly queue?: string;
+  readonly retry?: RetryPolicy;
+  /** Per attempt. A killed attempt resumes at the first table it had not yet checkpointed. */
+  readonly timeout?: DurationInput;
+}
+
+/** The cron a framework-shipped sweep runs on when its host has no opinion. */
+export const DEFAULT_PURGE_CRON = '23 * * * *';
+
+export function purge(definition: PurgeDefinition): JobHandle<PurgeInput> {
+  const clock = definition.clock;
+
+  return job<PurgeInput>({
+    ...(definition.name === undefined ? {} : { name: definition.name }),
+    input: t.object({}),
+    // One live sweep, forever: a second enqueue while a pass is still running is the same pass,
+    // and two deletes racing over one table buy nothing but lock contention. The scheduler's own
+    // key is occurrence-scoped on top of this, so the hourly runs are still distinct.
+    idempotencyKey: () => 'purge',
+    // Framework tables, not an org's rows. Every statement behind a target is raw SQL over the
+    // whole table, so there is no tenant-scoped read here to fail closed.
+    tenant: 'none',
+    retry: definition.retry ?? DEFAULT_RETRY,
+    ...(definition.queue === undefined ? {} : { queue: definition.queue }),
+    ...(definition.timeout === undefined ? {} : { timeout: definition.timeout }),
+    async run({ step }): Promise<PurgeReport> {
+      // ONE reading for every target in the pass. Two readings would let two tables be measured
+      // against instants a round trip apart, which is the same class of mismatch as reading the
+      // server's clock — smaller, and just as unnecessary.
+      const at = nowMs(clock);
+      const targets = definition.targets();
+      const names = new Set(targets.map((target) => target.name));
+      // Refused before the first delete, not discovered at the second step: `step.run` raises
+      // `X_STEP_DUPLICATE` on the repeat, which dead-letters a sweep AFTER it has already emptied
+      // one table. The list is lazy, so this cannot be checked at declaration.
+      assert(
+        names.size === targets.length,
+        `purge targets repeat a name: ${[...names].sort().join(', ')} across ${targets.length} targets`,
+        'give every PurgeTarget its own name — the name is the durable step key, and two steps under one name is X_STEP_DUPLICATE',
+      );
+
+      const swept: PurgeSweep[] = [];
+      for (const target of targets) {
+        // One durable step per table, so a killed attempt resumes at the table it stopped on
+        // rather than sweeping the ones already done a second time. At least once either way, and
+        // a purge is idempotent by nature: a replayed delete removes the rows that are already
+        // gone, which is none, and a row this deletes answers exactly as a row that was never
+        // there — no decision anywhere changes.
+        const removed = await step.run(target.name, () => target.purgeExpired(at));
+        swept.push({ name: target.name, removed });
+      }
+      const removed = swept.reduce((total, sweep) => total + sweep.removed, 0);
+      // Ops reads this to size the cadence: a sweep that removes hundreds of thousands every hour
+      // is a table that wants a shorter window, not a longer cron.
+      if (removed > 0) logger.info('jobs.purge.swept', { removed, tables: swept.length });
+      return { swept, removed };
+    },
+  });
+}

package/src/renewal-timer.ts

@@ -4,6 +4,8 @@
 // the interval, which does nothing to the request already on the wire. So a flag is what every
 // branch after an `await` re-reads — the shape `settleWithin`'s `decided` uses in core.
 
+import { logger, renderThrowable } from '@ultimat3/core';
+
 export interface RenewalTimer {
   /**
    * True once `stop()` has been called. Read AFTER every await in the renewal body: a clean
@@ -22,7 +24,20 @@ export function startRenewalTimer(
 ): RenewalTimer {
   let stopped = false;
   const timer = setInterval(() => {
-    void renew();
+    // `Promise.resolve().then(renew)` and never `void renew()`: a `renew` that throws SYNCHRONOUSLY
+    // escapes before any `.catch` its body chained exists. `worker-fleet-slots.ts` guards the
+    // promise chain and cannot guard this — `LeaseStore.renew` is an injected seam, and a store
+    // that throws on a closed pool throws on the call, not in the chain. Nothing sits above a
+    // `setInterval` callback, so that throw is an uncaught exception in the timer that was going
+    // to keep the lease alive. The shape `outbox.ts`'s tick loop already uses.
+    void Promise.resolve()
+      .then(renew)
+      .catch((error: unknown) => {
+        // A renewal that FAILS is each caller's own business and both handle it. Reaching here
+        // means the seam broke its contract, which is a different fact and is worth its own line —
+        // swallowed, it would be a lease that stops renewing with nothing anywhere saying so.
+        logger.error('jobs.renewal.raised', { error: renderThrowable(error) });
+      });
   }, intervalMs);
   return {
     stopped: () => stopped,

package/src/scheduler.ts

@@ -13,7 +13,7 @@
 // round rather than opening a second one over the same `lastFiredAt`.
 
 import type { Clock } from '@ultimat3/core';
-import { isUltimateError, logger, onShutdown } from '@ultimat3/core';
+import { isUltimateError, logger, onShutdown, renderThrowable } from '@ultimat3/core';
 import { instant, nextCronOccurrence } from '@ultimat3/time';
 import { nowMs } from './clock';
 import type { JobDriver } from './driver';
@@ -26,7 +26,7 @@ import { registeredTasks } from './task';
  * stable code to search on and the `fix:` to run, not a sentence.
  */
 function failureFields(error: unknown): Record<string, unknown> {
-  const message = error instanceof Error ? error.message : String(error);
+  const message = renderThrowable(error);
   return isUltimateError(error)
     ? { error: message, code: error.code, cause: error.cause, fix: error.fix }
     : { error: message };

package/src/steps.ts

@@ -7,10 +7,11 @@
 // catches it and re-queues the job for `resumeAt` instead of holding a process for three days.
 
 import type { Clock } from '@ultimat3/core';
-import { logger } from '@ultimat3/core';
+import { logger, renderThrowable } from '@ultimat3/core';
 import type { DurationInput } from './clock';
 import { nowMs, toMs } from './clock';
 import { JobAbortedError, JobTimeoutError, StepDuplicateError } from './errors';
+import { createRunSignal } from './run-signal';
 
 /**
  * The runtime list is the declaration and `StepStatus` is derived from it, the shape
@@ -283,10 +284,16 @@ export function createStepRunner(options: StepRunnerOptions): StepRunner {
     // The step's own ceiling, folded into the run's cancellation so the body reads ONE signal and
     // sees whichever deadline lands first. Composed only when there is a second one to compose.
     const deadline = new AbortController();
-    const signal =
-      options.stepTimeoutMs === undefined
-        ? runSignal
-        : AbortSignal.any([runSignal, deadline.signal]);
+    // `createRunSignal` and never `AbortSignal.any`, which is the rule this package's `CLAUDE.md`
+    // states as absolute: a composite cannot be undone, so ONE dependent signal per STEP stayed on
+    // the run's signal for the whole attempt. A `backfill()` at `batch: 1000` over 5M rows is
+    // 5,000 of them held at once, and an app whose `WorkerOptions.context()` carries a
+    // process-lifetime signal keeps them past the run. Composed only when there is a second signal
+    // to compose, and DISPOSED in the `finally` below, which is the whole reason `run-signal.ts`
+    // exists — `worker-run.ts` disposes the run's own the same way.
+    const composed =
+      options.stepTimeoutMs === undefined ? null : createRunSignal([runSignal, deadline.signal]);
+    const signal = composed?.signal ?? runSignal;
     try {
       const output = await withStepTimeout(
         fn(signal),
@@ -320,12 +327,16 @@ export function createStepRunner(options: StepRunnerOptions): StepRunner {
           status: 'failed',
           startedAt,
           attempts,
-          error: error instanceof Error ? error.message : String(error),
+          error: renderThrowable(error),
         };
         await store.put(failure);
         remember(failure);
       }
       throw error;
+    } finally {
+      // Nothing of the run's is held past the step. Idempotent, and it never aborts: a step that
+      // settled leaves its signal in whatever state it ended in.
+      composed?.dispose();
     }
   }
 

package/src/worker-fleet-slots.ts

@@ -3,7 +3,7 @@
 // Apart from `worker.ts` because the claim loop's question is "may I start this one?" — which job
 // holds which slot, and who gives it back, is bookkeeping of its own.
 
-import { logger } from '@ultimat3/core';
+import { logger, renderThrowable } from '@ultimat3/core';
 import type { ClaimedJob } from './driver';
 import { getJob } from './job';
 import type { HeldLease, LeaseStore } from './leases';
@@ -121,7 +121,7 @@ export function createFleetSlots(options: FleetSlotOptions): FleetSlots {
         logger.warn('jobs.worker.lease-release-failed', {
           workerId: options.workerId,
           jobId,
-          error: error instanceof Error ? error.message : String(error),
+          error: renderThrowable(error),
         });
       });
     },

package/src/worker.ts

@@ -4,7 +4,14 @@
 // deploy turns "at least once" into "always twice", so draining is on by default.
 
 import type { Clock, Ctx } from '@ultimat3/core';
-import { logger, onShutdown, recordJob, recordQueueDepth, uuid } from '@ultimat3/core';
+import {
+  logger,
+  onShutdown,
+  recordJob,
+  recordQueueDepth,
+  renderThrowable,
+  uuid,
+} from '@ultimat3/core';
 import { nowMs } from './clock';
 import type { ClaimedJob, JobDriver, QueueStats } from './driver';
 import { DEFAULT_QUEUE, DEFAULT_VISIBILITY_TIMEOUT_MS } from './driver';
@@ -141,7 +148,7 @@ export function createWorker(options: WorkerOptions): Worker {
       // Instrumentation never costs a tick: a queue that cannot be measured must still be worked.
       logger.warn('jobs.worker.depth-failed', {
         workerId,
-        error: error instanceof Error ? error.message : String(error),
+        error: renderThrowable(error),
       });
     }
   };
@@ -290,7 +297,7 @@ export function createWorker(options: WorkerOptions): Worker {
               workerId,
               job: job.name,
               jobId: job.id,
-              error: error instanceof Error ? error.message : String(error),
+              error: renderThrowable(error),
             });
           },
         );
@@ -332,7 +339,7 @@ export function createWorker(options: WorkerOptions): Worker {
         .catch((error: unknown) => {
           logger.error('jobs.worker.tick-failed', {
             workerId,
-            error: error instanceof Error ? error.message : String(error),
+            error: renderThrowable(error),
           });
         })
         .finally(() => {