@ultimat3/jobs 19.2.0 → 19.3.2

package/CLAUDE.md

@@ -4,7 +4,9 @@ Tier 3. The `job` + `task` primitives, durable steps, transactional outbox, queu
 
 ## Boundary
 
-- May import: `core`, `schema`, `entity`, `policy`, `cache`, `time`. Never `http`, `render`, `ui`.
+- May import: `core`, `schema`, `entity`, `policy`, `cache`, `time` — and `db`, for
+  `expectedQueryLoop` ONLY: `steps.ts` declares the per-step write one-per-step to the N+1
+  detector there, and no client is ever taken from it. Never `http`, `render`, `ui`.
 - Consumers: `action` (`<job>.enqueue`, via the ambient jobs facade), `cli`, `mcp`, `admin`.
 - External deps: none. Postgres access goes through the injected `PgExecutor`.
 
@@ -265,6 +267,16 @@ Tier 3. The `job` + `task` primitives, durable steps, transactional outbox, queu
   standstill, so start -> stop -> start stacked a second registration retaining a stopped worker's
   driver, and the next process-wide drain ran all of them. `start()` refuses while draining for the
   same reason: a claim loop back on a driver the drain is about to close.
+- **The outbox relay drains in the same two phases, `As of 2026-09`.** It registered NO hook: on
+  SIGTERM it went on claiming and publishing through every phase of the drain, and the only thing
+  that ever stopped it was `RunningRoles.stop()` in `x dev`'s release path — a caller a signal can
+  skip. Two consequences, both silent: a lease stamped on rows nothing on this pod will run (stranded
+  for the visibility window), and a row past `driver.enqueue` but short of `markPublished` published
+  a second time on the next boot, which the idempotency key collapses only while the first job is
+  still live. `accept` clears the interval and returns; `close` awaits the pass in flight under
+  `settleAllBy(…, reason.deadlineAt)`; both unregisters come back in the teardown's `finally`. The
+  poll timer is `unref`ed for `renewal-timer.ts`'s reason — a 200ms interval refed holds the event
+  loop open past every phase and makes SIGKILL the exit.
 - **A claimed job is counted with core's `beginWork()`, so the DRAIN does the waiting**
   (`As of 2026-08-23`). The wait for in-flight jobs belongs to the phase between `accept` and
   `inflight`, which exists for exactly this and is where `@ultimat3/http` already puts a request —
@@ -281,6 +293,31 @@ Tier 3. The `job` + `task` primitives, durable steps, transactional outbox, queu
   promises; `jobs.worker.drain-abandoned` names it, with the `configureLifecycle({ deadlineMs })`
   raise as its fix. **A worker always REACHES `'stopped'`**, which is what makes `stop()`'s
   `state === 'stopped'` early return an answer rather than a wedge.
+- **SIGTERM reaches the job: the `accept` hook aborts every held run's `ctx.signal`**
+  (`As of 2026-09-07`). The worker holds ONE `AbortController` (`drainSignal`), composed into every
+  run by `worker-run.ts` as the third source beside the caller's signal and the heartbeat's;
+  `stopAccepting(reason)` aborts it with a `JobDrainedError` — core's `X_DRAINING`, naming the
+  worker and the signal — and a manual `stop()` passes no reason and aborts nothing, for the same
+  line `settleAllBy` draws: a caller that asked wants its work finished. It has to be the `accept`
+  hook and not the teardown, because core runs `accept`, then waits out in-flight work (every
+  claimed job is `beginWork()`ed) under the same budget, then `close`: told in `close`, a body
+  would hear it after the in-flight wait had already spent the whole budget on it. Which is what
+  happened until this landed — the drain told nobody, so a body reading `ctx.signal` (the one
+  documented way to stop early) ran to the deadline and was abandoned there, indistinguishable
+  from one that ignores the signal; ai-maxxing measured it as every Ctrl-C paying the full 25s
+  and wrote a process-wide signal of its own to get around it. The controller is replaced with a
+  fresh one in the teardown's `finally`: aborted once stays aborted, and a restarted worker would
+  otherwise hand every job it claimed a signal born cancelled.
+  `executeJob` reads the CODE off the run signal's reason (`drainedBy`) and settles a drained
+  attempt as **`interrupted`** — `nack` with `countsAsAttempt: false`, no park, no dead letter,
+  `delayMs: 0`, the error still recorded on the row — read off the SIGNAL and not the thrown
+  error, because what a body stops WITH is not always the reason (a killed ssh child surfaces as
+  the app's own coded error) and an attempt burned per deploy is the "always twice" draining
+  exists to prevent, one layer down. Not a `retried`: `attempts: 1` would dead-letter a job the
+  process, not the job, cut short, and `jobs_total{outcome="failed"}` would spike on every
+  rollout. The first reason on a controller wins, so a timeout that fired before the drain still
+  reports as a timeout. `WorkerStats.interrupted` counts them; `jobs.worker.drain-signalled` and
+  `jobs.attempt.interrupted` are the two lines. `worker-drain-signal.test.ts` holds all of it.
 - **One teardown, joined.** `stop()` shares the in-flight teardown promise, so a SIGTERM landing
   on a manual stop waits out the same in-flight jobs instead of closing the driver underneath
   it. The promise is cleared as it settles, so a worker that started again tears down again
@@ -905,7 +942,8 @@ picture from the other side.
 | `register.ts` | `registerJobs`/`registerTasks` over a module namespace + the registrar announcements. Skips a non-job in silence — a module namespace is full of helpers — EXCEPT an `@ultimat3/action` projection (`kind: 'action-job'`), which is `X_ACTION_JOB_UNBRIDGED` |
 | `describe.ts` | the JSON projection one handle emits; `describeJobs()` is a map over it |
 | `steps.ts` | `StepStore`, `StepApi`, memoized-replay executor, `StepSuspension` |
-| `outbox.ts` | staging in a `Tx`, the relay, the ambient `JobsFacade` slot |
+| `outbox.ts` | staging in a `Tx`, the store seam, the ambient `JobsFacade` slot |
+| `outbox-relay.ts` | the relay: the poll timer, one pass, and its TWO shutdown hooks. Split off at `outbox.ts`'s 500-line ceiling |
 | `outbox-pg.ts` | `createPgOutboxStore` — `stage()` on the caller's OWN connection, claim on the pool |
 | `outbox-lease.ts` | the claim lease's one definition and its one normalisation, for both stores |
 | `leases.ts` | `LeaseStore` — fleet-wide slots, the memory one, `jobLeaseKey` |
@@ -924,7 +962,8 @@ picture from the other side.
 | `execute.ts` | `executeJob` — one claimed job run and settled, and the run's deadline/cancel |
 | `heartbeat.ts` | one claimed job's lease: the renewal interval and the loss it reports |
 | `renewal-timer.ts` | the interval a renewal runs on, and the `stopped()` latch every branch after an await re-reads |
-| `worker.ts` | `worker` role, claim loop, drain |
+| `worker.ts` | `worker` role, claim loop, drain — and the one `AbortController` SIGTERM reaches every held run through |
+| `worker-types.ts` | the worker's public contract: `WorkerOptions`, `WorkerStats`, `Worker` |
 | `drain-wait.ts` | the drain's wait, shared by both roles: everything a teardown holds, settled — or abandoned at the budget the `close` hook was handed |
 | `worker-run.ts` | one claimed job, wired: its heartbeat, its slot renewal, its run signal and its span, started together and handed back in one `finally` |
 | `run-signal.ts` | the signal ONE run is cancelled by — composition that can be handed back, and that the worker can abort itself |

package/README.md

@@ -550,6 +550,24 @@ Nothing can kill a body that ignores the signal, so the durable state is fenced:
 cancel every step write is refused with `X_ABORTED`, and a run that finishes anyway is logged
 as `jobs.timeout.abandoned` — the one way to find a handler that never reads `ctx.signal`.
 
+**SIGTERM fires the same signal** (`As of 2026-09-07`). The moment the drain begins — its `accept`
+phase, before the in-flight wait starts spending the budget — the worker aborts `ctx.signal` on
+every job it holds with `X_DRAINING`, naming the worker and the signal. A body that unwinds is
+**interrupted**, not failed: the job goes straight back to the ready bucket with the attempt
+uncounted, so `attempts: 1` survives a deploy and the worker replacing this one claims it at
+once. Whatever the body stopped with — the reason back from `fetch`, `throwIfAborted`'s
+`X_ABORTED`, an app's own error for a child the shutdown killed — the verdict is read off the
+signal, not the error. A body that ignores the signal is waited on to the deadline and abandoned
+there, as before. A manual `stop()` aborts nothing: it waits for the work it holds.
+
+| The attempt ended by | `JobOutcome` | attempt counted | `jobs_total` |
+|---|---|---|---|
+| the body returning | `completed` | — | `ok` |
+| `step.sleep` / `step.waitForEvent` | `suspended` | no | not counted |
+| the body throwing, attempts left | `retried` | yes | `failed` |
+| the body throwing, none left or `terminal` | `dead-lettered` | yes | `dead` |
+| the worker's drain (`X_DRAINING` on `ctx.signal`) | `interrupted` | **no** | not counted |
+
 Three ceilings, declared on the job and nowhere else (`As of 2026-08` — `stepTimeout` and
 `eventPoll` had been implemented in the step runner since 1.0 with no declaration able to reach
 them, so no `job()` could ask for either):
@@ -603,6 +621,11 @@ things have to be true in a process:
 | the facade is installed | `setJobsFacade(createJobsFacade({ store, driver }, currentTx))` |
 | the relay is running | `createOutboxRelay({ store, driver }).start()` |
 
+`start()` registers the same two shutdown hooks `createWorker` does — `accept` stops polling
+(the worker's also aborts every held job's `ctx.signal`), `close` waits out the pass in flight
+under the drain's deadline — and `stop()` hands both back.
+`drainOnShutdown: false` opts out, for a caller that drives its own teardown.
+
 with `store = createPgOutboxStore({ executor, txExecutor })`. `txExecutor` is what makes it
 transactional: `stage()` runs on the CALLER'S connection, never the pool. With nothing installed,
 `jobsFacade()` answers a fallback whose `currentTx` is `() => undefined` and every enqueue

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ultimat3/jobs",
-  "version": "19.2.0",
+  "version": "19.3.2",
   "description": "Durable background work: steps, transactional outbox, cron tasks, one driver interface",
   "license": "MIT",
   "type": "module",
@@ -32,9 +32,10 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@ultimat3/core": "19.2.0",
-    "@ultimat3/entity": "19.2.0",
-    "@ultimat3/schema": "19.2.0",
-    "@ultimat3/time": "19.2.0"
+    "@ultimat3/core": "19.3.2",
+    "@ultimat3/db": "19.3.2",
+    "@ultimat3/entity": "19.3.2",
+    "@ultimat3/schema": "19.3.2",
+    "@ultimat3/time": "19.3.2"
   }
 }

package/src/drain-wait.ts

@@ -6,45 +6,95 @@
 import { systemClock } from '@ultimat3/core';
 
 /**
- * Everything in `pending`, settled — or abandoned once `deadlineAt` (real monotonic ms, the clock
- * `ShutdownReason.deadlineAt` is measured on) has passed. Answers `true` when everything settled.
+ * The deadline a teardown waits under, and it is bound LATE. `undefined` while the stop is manual;
+ * a real monotonic instant (the clock `ShutdownReason.deadlineAt` is measured on) the moment a
+ * shutdown lands — before the teardown starts, or in the middle of it. A wait already in progress
+ * adopts it, which is the case a plain number could not express: `worker.stop('deploy')` starts a
+ * teardown with no budget, SIGTERM arrives, core's `close` hook JOINS the memoised teardown — and
+ * the number that teardown was started with is the number it kept. Core abandoned the hook at the
+ * deadline and moved on; the worker sat on a body that ignores `ctx.signal` with its driver open
+ * and `stopping` never settling, exactly the wedge the bound was written to end.
  *
- * `deadlineAt` is `undefined` for a MANUAL `stop()`, which waits as long as its work takes: a
- * caller that asked a role to stop has no budget to spend, and closing the queue under a live job
- * — or handing the lease back under a live dispatch — is exactly what draining exists to prevent. The bound belongs to the SIGTERM path, where the
- * budget is real and a handler that ignores `ctx.signal` would otherwise hold the teardown — and
- * with it the memoized `stopping` promise every later `stop()` joins — open forever.
+ * The EARLIEST deadline wins: a second bind can only tighten, never extend, because the budget it
+ * comes from is one process-wide grace period and not a per-caller allowance.
+ */
+export interface DrainBudget {
+  /** The deadline in force — `undefined` until a shutdown binds one. */
+  readonly deadlineAt: number | undefined;
+  /** Bind a deadline, or tighten the one held. Every wait in progress hears it at once. */
+  bind(deadlineAt: number): void;
+  /**
+   * Hear the deadline: now, when one is already bound, and again each time it tightens. Answers
+   * the unsubscribe, which a settled wait calls so a budget outlives none of its waiters.
+   */
+  watch(listener: (deadlineAt: number) => void): () => void;
+}
+
+export function createDrainBudget(deadlineAt?: number): DrainBudget {
+  let bound = deadlineAt;
+  const listeners = new Set<(deadlineAt: number) => void>();
+  return {
+    get deadlineAt() {
+      return bound;
+    },
+    bind(at) {
+      if (bound !== undefined && at >= bound) return;
+      bound = at;
+      for (const listener of listeners) listener(at);
+    },
+    watch(listener) {
+      listeners.add(listener);
+      if (bound !== undefined) listener(bound);
+      return () => {
+        listeners.delete(listener);
+      };
+    },
+  };
+}
+
+/**
+ * Everything in `pending`, settled — or abandoned once the deadline (real monotonic ms) has
+ * passed. Answers `true` when everything settled.
+ *
+ * The deadline is a number for a role whose teardown cannot be joined mid-flight, and a
+ * `DrainBudget` for one that can — the worker, whose manual `stop()` a later SIGTERM joins.
+ * `undefined` is a MANUAL `stop()`, which waits as long as its work takes: a caller that asked a
+ * role to stop has no budget to spend, and closing the queue under a live job — or handing the
+ * lease back under a live dispatch — is exactly what draining exists to prevent. The bound belongs
+ * to the SIGTERM path, where the budget is real and a handler that ignores `ctx.signal` would
+ * otherwise hold the teardown — and with it the memoized `stopping` promise every later `stop()`
+ * joins — open forever.
  *
  * `allSettled`, so work that rejected is work that finished: each caller observes its own failures
  * already, and a teardown that rethrew here would skip the close behind it.
  */
 export async function settleAllBy(
   pending: readonly Promise<unknown>[],
-  deadlineAt: number | undefined,
+  deadline: DrainBudget | number | undefined,
 ): Promise<boolean> {
   if (pending.length === 0) return true;
   const settled = Promise.allSettled(pending);
-  if (deadlineAt === undefined) {
-    await settled;
-    return true;
-  }
-  const remainingMs = Math.max(0, deadlineAt - systemClock.monotonic());
+  const budget = typeof deadline === 'object' ? deadline : createDrainBudget(deadline);
   return await new Promise<boolean>((resolve) => {
     let decided = false;
-    const timer = setTimeout(() => {
-      if (decided) return;
-      decided = true;
-      resolve(false);
-    }, remainingMs);
-    // Never the thing keeping a drained process alive — the rule `lifecycle-deadline.ts` states
-    // for its own timer. A spent budget still gives the already-settled case its turn, because a
-    // resolved promise settles on a microtask and this timer on a macrotask.
-    timer.unref?.();
-    void settled.then(() => {
+    let timer: ReturnType<typeof setTimeout> | undefined;
+    const decide = (drained: boolean): void => {
       if (decided) return;
       decided = true;
-      clearTimeout(timer);
-      resolve(true);
+      unwatch();
+      if (timer !== undefined) clearTimeout(timer);
+      resolve(drained);
+    };
+    // Re-armed on every tightening, so a wait that began with no deadline ends at the one a later
+    // shutdown bound. Never the thing keeping a drained process alive — the rule
+    // `lifecycle-deadline.ts` states for its own timer. A spent budget still gives the
+    // already-settled case its turn, because a resolved promise settles on a microtask and this
+    // timer on a macrotask.
+    const unwatch = budget.watch((deadlineAt) => {
+      if (timer !== undefined) clearTimeout(timer);
+      timer = setTimeout(() => decide(false), Math.max(0, deadlineAt - systemClock.monotonic()));
+      timer.unref?.();
     });
+    void settled.then(() => decide(true));
   });
 }

package/src/driver-memory.ts

@@ -25,7 +25,7 @@ import { JobDuplicateError } from './errors';
 import type { LeaseStore } from './leases';
 import { createMemoryLeaseStore } from './leases';
 import type { StepStore } from './steps';
-import { createMemoryStepStore } from './steps';
+import { createMemoryStepStore } from './steps-memory';
 
 export interface MemoryDriverOptions {
   readonly clock?: Clock;
@@ -146,7 +146,11 @@ export function createMemoryDriver(options: MemoryDriverOptions = {}): MemoryJob
       // `state !== 'done'`, mirroring `SQL_CANCEL`: a job that already finished has nothing to
       // stop, and cancelling it would rewrite a terminal row an operator is reading as success.
       if (existing === undefined || existing.state === 'done') return Promise.resolve(undefined);
-      update(jobId, {
+      // `settle`, not `update`: a cancellation RELEASES the claim, and `SQL_CANCEL` writes
+      // `visible_at = null, claimed_by = null` with the state. Left stamped, a cancelled row named
+      // the worker still holding it and carried that attempt's lease deadline — the pair
+      // `x jobs show` prints, and the pair the claim scan reads to decide a row was abandoned.
+      settle(jobId, {
         state: 'cancelled',
         ...(reason === undefined ? {} : { lastError: reason }),
       });
@@ -161,7 +165,11 @@ export function createMemoryDriver(options: MemoryDriverOptions = {}): MemoryJob
     leases,
     introspect,
 
-    enqueue(request: EnqueueRequest): Promise<EnqueueResult> {
+    // `async` for the reason `claim`, `list` and `deadLetters` are: `onConflict: 'error'` REJECTS
+    // here exactly as the pg driver's does, and a synchronous throw out of a method typed
+    // `Promise<…>` is a second answer to one question — caught by different code, and an
+    // unhandled exception rather than a settled promise wherever the caller holds the promise.
+    async enqueue(request: EnqueueRequest): Promise<EnqueueResult> {
       const existing = liveByKey(request.name, request.idempotencyKey, request.tenantId);
       if (existing !== undefined) {
         if (request.onConflict === 'error') {
@@ -171,7 +179,7 @@ export function createMemoryDriver(options: MemoryDriverOptions = {}): MemoryJob
             existingId: existing.id,
           });
         }
-        return Promise.resolve({ id: existing.id, runId: existing.runId, deduped: true });
+        return { id: existing.id, runId: existing.runId, deduped: true };
       }
 
       const at = nowMs(clock);
@@ -194,7 +202,7 @@ export function createMemoryDriver(options: MemoryDriverOptions = {}): MemoryJob
         ...(request.enqueuedBy === undefined ? {} : { enqueuedBy: request.enqueuedBy }),
       };
       jobs.set(record.id, record);
-      return Promise.resolve({ id: record.id, runId: record.runId, deduped: false });
+      return { id: record.id, runId: record.runId, deduped: false };
     },
 
     // `async`, so an empty queue list REJECTS here exactly as it does on the pg driver: a

package/src/driver-nats.ts

@@ -16,13 +16,22 @@ import type {
 import { JobsNotImplementedError } from './errors';
 import type { StepRecord, StepStore } from './steps';
 
-// Names the seam that actually replaces the stub, plus the runnable command for whatever is
-// already queued. NOT `jobs: { driver }` in app.config.ts, which this line said until 2026-08-20:
-// `JobsConfig.driver` has no reader anywhere (see `driver.ts`'s header), so that edit repairs
-// nothing and the reader is sent back to the same throw. #223 removes the field.
+// Names the seam that actually replaces the stub, and NOTHING ELSE — the two other repairs this
+// line has carried were both unrunnable.
+//
+// NOT `jobs: { driver }` in app.config.ts, which it said until 2026-08-20: the field had no
+// reader anywhere, so that edit repaired nothing and sent the reader back to the same throw. It
+// is deleted now.
+//
+// NOT `x jobs drain --to memory` either, which it said until 2026-09, and that one was a route
+// into data loss: the target is a Map inside the command's own process, so the drain acked every
+// durable row and lost the copy at exit. `x jobs` refuses that value by name now. There is no
+// drain to run in its place, and the reason is in this file — `enqueue` below refuses too, so
+// nothing was ever queued onto this driver and the queue is untouched.
+//
 // The nats driver lands in v2; there is no flag that turns this one on.
 const FIX =
-  'call setJobDriver(createPgDriver()) at boot instead of this driver, then move what is already queued: x jobs drain --to memory --json';
+  'call setJobDriver(createPgDriver()) at boot instead of this driver; nothing needs moving first, because enqueue here refuses too, so no job was ever written to it';
 
 const unavailable = (method: string): never => {
   throw new JobsNotImplementedError({ feature: `nats jobs driver (${method})`, fix: FIX });

package/src/driver-pg.ts

@@ -54,8 +54,10 @@ import type { HeldLease, LeaseStore } from './leases';
 import type { StepStore } from './steps';
 
 /**
- * The one thing this driver needs from the DB layer, declared structurally so this package can
- * depend on no database package at all.
+ * The one thing this driver needs from the DB layer, declared structurally so this package needs
+ * no database CLIENT: `@ultimat3/db` is imported for `expectedQueryLoop` — the marker that tells
+ * the N+1 detector a step write is one-per-step by design (`steps.ts`) — and never for a
+ * connection.
  *
  * **Not satisfied by `Bun.sql`** — verified against Bun 1.4.0: `Bun.sql.query` is `undefined`.
  * `Bun.sql` is a tagged template whose positional form is `unsafe`, so a `{ executor: Bun.sql }`

package/src/driver-redis.ts

@@ -19,13 +19,22 @@ import type {
 import { JobsNotImplementedError } from './errors';
 import type { StepRecord, StepStore } from './steps';
 
-// Names the seam that actually replaces the stub, plus the runnable command for whatever is
-// already queued. NOT `jobs: { driver }` in app.config.ts, which this line said until 2026-08-20:
-// `JobsConfig.driver` has no reader anywhere (see `driver.ts`'s header), so that edit repairs
-// nothing and the reader is sent back to the same throw. #223 removes the field.
+// Names the seam that actually replaces the stub, and NOTHING ELSE — the two other repairs this
+// line has carried were both unrunnable.
+//
+// NOT `jobs: { driver }` in app.config.ts, which it said until 2026-08-20: the field had no
+// reader anywhere, so that edit repaired nothing and sent the reader back to the same throw. It
+// is deleted now.
+//
+// NOT `x jobs drain --to memory` either, which it said until 2026-09, and that one was a route
+// into data loss: the target is a Map inside the command's own process, so the drain acked every
+// durable row and lost the copy at exit. `x jobs` refuses that value by name now. There is no
+// drain to run in its place, and the reason is in this file — `enqueue` below refuses too, so
+// nothing was ever queued onto this driver and the queue is untouched.
+//
 // The redis driver lands in v2; there is no flag that turns this one on.
 const FIX =
-  'call setJobDriver(createPgDriver()) at boot instead of this driver, then move what is already queued: x jobs drain --to memory --json';
+  'call setJobDriver(createPgDriver()) at boot instead of this driver; nothing needs moving first, because enqueue here refuses too, so no job was ever written to it';
 
 const unavailable = (method: string): never => {
   throw new JobsNotImplementedError({ feature: `redis jobs driver (${method})`, fix: FIX });

package/src/errors.ts

@@ -39,13 +39,15 @@ export const JOB_OWNED_ERROR_CODES = [
 ] as const;
 
 /**
- * `X_NOT_IMPLEMENTED` and `X_ABORTED` are `@ultimat3/core`'s. `JobsNotImplementedError` and
- * `JobAbortedError` below throw them; jobs keeps no title for either, because the copy this file
- * used to hold was a second title that nothing would have failed on once core's changed.
+ * `X_NOT_IMPLEMENTED`, `X_ABORTED` and `X_DRAINING` are `@ultimat3/core`'s. `JobsNotImplementedError`,
+ * `JobAbortedError` and `JobDrainedError` below throw them; jobs keeps no title for any of the
+ * three, because the copy this file used to hold was a second title that nothing would have failed
+ * on once core's changed. Listed here all the same, so `JobErrorCode` can name every code a job
+ * can see — `X_DRAINING` was thrown for a day before it was, and the type said it could not be.
  */
-export const JOB_BORROWED_ERROR_CODES = ['X_NOT_IMPLEMENTED', 'X_ABORTED'] as const;
+export const JOB_BORROWED_ERROR_CODES = ['X_NOT_IMPLEMENTED', 'X_ABORTED', 'X_DRAINING'] as const;
 
-/** Every code jobs can throw: the ones it owns plus the one it borrows. */
+/** Every code jobs can throw: the ones it owns plus the ones it borrows. */
 export const JOB_ERROR_CODES = [...JOB_OWNED_ERROR_CODES, ...JOB_BORROWED_ERROR_CODES] as const;
 
 export type JobOwnedErrorCode = (typeof JOB_OWNED_ERROR_CODES)[number];
@@ -271,6 +273,27 @@ export class JobAbortedError extends UltimateError {
   }
 }
 
+/**
+ * The worker holding this attempt received SIGTERM. Handed to the run as `ctx.signal`'s reason
+ * the moment the drain's `accept` phase runs — before core's in-flight wait, not at the deadline
+ * that ends it — so a body reading the one cancellation seam learns the process is going away
+ * while there is still budget to unwind in.
+ *
+ * Core's `X_DRAINING` rather than a code of jobs' own, for `JobAbortedError`'s reason: the
+ * framework already means exactly one thing by "the process is draining", it is already
+ * classified `retryable`, and `executeJob` reads the CODE off the run signal's reason to tell a
+ * drained attempt from a timed-out or lease-lost one — the drained one is handed back uncounted.
+ */
+export class JobDrainedError extends UltimateError {
+  constructor(input: { workerId: string; signal: string }) {
+    super({
+      code: 'X_DRAINING',
+      cause: `worker "${input.workerId}" is draining (${input.signal}) — this attempt is cut short and the job handed back to the queue with the attempt uncounted`,
+      fix: 'nothing on the job: another worker claims it. To unwind inside the drain budget instead of being killed at it, pass ctx.signal to every outbound call — fetch(url, { signal: ctx.signal }) — and call throwIfAborted(ctx) between steps',
+    });
+  }
+}
+
 /** Retries exhausted. The job is in the dead-letter queue, not lost. */
 export class JobMaxAttemptsError extends UltimateError {
   constructor(input: { job: string; jobId: string; attempts: number; lastError: string }) {

package/src/execute.ts

@@ -28,7 +28,13 @@ import type { EventLookup, StepRecord } from './steps';
 import { createStepRunner, isStepSuspension } from './steps';
 import { jobRunActor } from './tenant';
 
-export type JobOutcome = 'completed' | 'suspended' | 'retried' | 'dead-lettered';
+/**
+ * How one attempt ended. `interrupted` is the worker's drain cutting the attempt short: the job is
+ * back in the ready bucket with the attempt UNCOUNTED, because the process ended it and not the
+ * job — filed as `retried`, a deploy would burn an attempt per job it held, and with
+ * `attempts: 1` dead-letter it.
+ */
+export type JobOutcome = 'completed' | 'suspended' | 'retried' | 'dead-lettered' | 'interrupted';
 
 /** Stands in for a caller with nothing to cancel, so the composition below has one shape. */
 const NEVER_ABORTED = new AbortController().signal;
@@ -188,6 +194,33 @@ export async function executeJob(options: ExecuteJobOptions): Promise<JobExecuti
     }
 
     const message = renderThrowable(error);
+    if (drainedBy(signal)) {
+      // The worker's drain told this body to stop, and it did. Whatever it stopped WITH is the
+      // framework's doing — the reason itself back from `fetch`, `throwIfAborted`'s `X_ABORTED`,
+      // a fenced step write, or an app's own coded error for a child the shutdown killed — so
+      // the attempt is handed back rather than failed: `countsAsAttempt: false`, no park, no
+      // dead letter, claimable at once by the worker replacing this one. Read off the SIGNAL and
+      // not the error, because the body's error is not always the signal's reason, and an
+      // attempt burned per deploy is the "always twice" draining exists to prevent. The `error`
+      // is still recorded on the row: `x jobs show` should say why the last attempt ended.
+      await driver.nack(claimed.id, { delayMs: 0, error: message, countsAsAttempt: false });
+      logger.info('jobs.attempt.interrupted', {
+        job: handle.name,
+        jobId: claimed.id,
+        attempt: claimed.attempt,
+        error: message,
+      });
+      return settle({
+        outcome: 'interrupted',
+        jobId: claimed.id,
+        job: handle.name,
+        attempt: claimed.attempt,
+        durationMs: nowMs(options.clock) - startedAt,
+        error: message,
+        steps: [],
+        replayed: [],
+      });
+    }
     // The ERROR decides too, not only the attempt count. A `terminal` code — a rotated password,
     // a schema mismatch, a permission denial — fails identically on every remaining attempt, so
     // spending them is a queue slot, a provider bill and, at a site that locks an account after
@@ -317,6 +350,15 @@ function raceTimeout(
   });
 }
 
+/**
+ * The run was cancelled by the worker's drain: the code `JobDrainedError` carries, read off the
+ * signal. The FIRST reason wins on a controller, so a timeout that fired before the drain still
+ * reports as a timeout — the drain only claims an attempt it ended.
+ */
+function drainedBy(signal: AbortSignal): boolean {
+  return signal.aborted && isUltimateError(signal.reason) && signal.reason.code === 'X_DRAINING';
+}
+
 /** The body stopped because we cancelled it: our own reason back, or a fenced step write. */
 function isCancellation(error: unknown, reason: unknown): boolean {
   return error === reason || (isUltimateError(error) && error.code === 'X_ABORTED');

package/src/index.ts

@@ -159,6 +159,7 @@ export {
   JOB_ERROR_CODES,
   JOB_ERROR_TITLES,
   JobAbortedError,
+  JobDrainedError,
   JobDuplicateError,
   JobMaxAttemptsError,
   JobNameTakenError,
@@ -233,14 +234,11 @@ export type {
   MemoryOutboxStore,
   OutboxDeps,
   OutboxRecord,
-  OutboxRelay,
   OutboxStore,
-  RelayOptions,
 } from './outbox';
 export {
   createJobsFacade,
   createMemoryOutboxStore,
-  createOutboxRelay,
   enqueueInTx,
   jobsFacade,
   resetJobsFacade,
@@ -251,6 +249,8 @@ export {
 export { DEFAULT_OUTBOX_CLAIM_LEASE_MS } from './outbox-lease';
 export type { PgOutboxOptions } from './outbox-pg';
 export { createPgOutboxStore } from './outbox-pg';
+export type { OutboxRelay, RelayOptions } from './outbox-relay';
+export { createOutboxRelay } from './outbox-relay';
 export type {
   PurgeDefinition,
   PurgeInput,
@@ -290,7 +290,6 @@ export type {
   WaitForEventOptions,
 } from './steps';
 export {
-  createMemoryStepStore,
   createStepRunner,
   isStepStatus,
   isStepSuspension,
@@ -298,6 +297,7 @@ export {
   STEP_STATUSES,
   StepSuspension,
 } from './steps';
+export { createMemoryStepStore } from './steps-memory';
 export type {
   CatchUpPolicy,
   TaskDefinition,

package/src/metrics.ts

@@ -14,6 +14,24 @@
 
 import type { Gauge } from '@ultimat3/core';
 import { gauge } from '@ultimat3/core';
+import type { JobOutcome } from './execute';
+
+/**
+ * `JobOutcome` -> the `jobs_total` label, and `null` for the outcomes that are not one. `suspended`
+ * is deliberately unmapped: parking a run is control flow, so counting it would make every
+ * `step.sleep` read as a finished job and make the failure ratio meaningless. `interrupted` for
+ * the same reason: a deploy cutting a job short is the process's doing, and a failure ratio that
+ * spikes on every rollout is a page nobody answers. Read by the worker's one `recordJob` site.
+ */
+export const JOB_OUTCOME_LABELS = Object.freeze<
+  Record<JobOutcome, 'ok' | 'failed' | 'dead' | null>
+>({
+  completed: 'ok',
+  suspended: null,
+  retried: 'failed',
+  'dead-lettered': 'dead',
+  interrupted: null,
+});
 
 /** Seconds and not milliseconds: every Prometheus duration is seconds, and the alert is `> 300`. */
 export const queueOldestReady: Gauge = gauge('queue_oldest_ready_seconds', {