@hotmeshio/hotmesh 0.25.6 → 0.26.0

package/OPTIMIZATIONS.md

@@ -0,0 +1,128 @@
+# HotMesh needs — from long-tail
+
+A needs document from the long-tail project: the usage patterns we run at
+production scale and the engine support that would let us run them well. Each item
+states what we do, the concrete access pattern, and the engine capability or
+guarantee we need. Ordered by leverage.
+
+Context: long-tail is a durable HITL escalation platform on Postgres. Escalations
+live in `public.hmsh_escalations` (engine-created); we layer a `public.lt_escalations`
+view and a few app-owned indexes on top. Our dominant runtime shape is
+**single-role** work-queue traffic — claim / available / count / settle scoped to
+one `role` — plus a per-role escalation form the resolve UI renders.
+
+---
+
+## 1. Safe retention for terminal escalation rows
+
+**Usage.** An escalation is written once, resolved once, then read for audit. The
+pending working set stays small — a role-leading pending partial index keeps
+single-role claim/available queries `O(pending)` regardless of table size:
+
+```sql
+WHERE role = $1 AND status = 'pending'
+  AND (assigned_to IS NULL OR assigned_until <= NOW())
+  AND metadata @> $facets
+ORDER BY priority ASC, created_at ASC
+FOR UPDATE SKIP LOCKED
+```
+
+The terminal set (resolved / expired / cancelled) grows for the life of the
+deployment — the source of long-run table bloat, autovacuum load, and index growth.
+
+**Need.** A safe, engine-blessed way to age out terminal rows. Tell us what state
+the engine still reads once an escalation is terminal (replay, completion, audit),
+so we know what is inert, then one of:
+- a supported retention/archive call the engine owns; or
+- a documented contract that a row is safe to delete once its status is terminal
+  and older than a horizon; or
+- time sub-partitioning of the terminal set the engine can `DETACH`/`DROP`.
+
+`hmsh_escalations` stays a single physical table keyed by `id`/`signal_key` for
+the engine's lookups; retention touches only rows the engine confirms are inert. We
+can supply the horizon and exact DDL.
+
+---
+
+## 2. Stable ownership of app-added indexes on `hmsh_escalations`
+
+**Usage.** We add indexes to the engine table via advisory-locked
+`CREATE INDEX CONCURRENTLY IF NOT EXISTS`:
+- a role-leading pending index `(role, priority, created_at) WHERE status='pending'`
+  for the claim/available order;
+- a **pending-scoped GIN** on `metadata` (`jsonb_path_ops`) `WHERE status='pending'`
+  — keeps the GIN sized to the working set and cuts write amplification on the
+  high-churn insert→resolve path;
+- covering partial indexes for resolved-row reads.
+
+**Need.** A stable contract that app-added indexes on `hmsh_escalations` survive
+engine migrations — the engine keeps the table in place across upgrades — or a
+documented post-migration hook we can register to re-ensure them. Shipping these
+index shapes in the engine itself also works; the requirement is that they persist.
+
+---
+
+## 3. Atomic metadata write in the `condition()` Leg1 checkpoint
+
+**Usage.** A workflow surfaces a HITL wait with `condition(signalId, config)`. We
+pass the target `role`, claim/routing facets, and `metadata.schema_version` (the
+form version the resolve UI renders) in `config.metadata`. Claim/resolve-by-metadata
+and the dashboard read those facets via GIN `@>`; the resolve UI reads
+`schema_version` to render the exact form.
+
+**Need.** Guarantee — as a documented public contract — that `config.metadata` is
+written into the escalation row **atomically in the same Leg1 checkpoint** that
+creates the row (one commit, crash-safe). This is the foundation our JIT-versioned
+form pinning and metadata-driven routing stand on.
+
+---
+
+## 4. Honored long-activity timeout + reclaim safety
+
+**Usage.** We run a batch pattern: a single proxyActivity loops internally for
+~60s (poll → reconcile → act), returning one durable checkpoint per call instead of
+one per step, so replay stays compact.
+
+**Need.** A documented guarantee that:
+- `proxyActivities({ startToCloseTimeout })` is honored for long (30–120s) activities;
+- the stream reclaim window (`HMSH_XCLAIM_DELAY_MS`) holds an in-flight activity for
+  the duration of its `startToCloseTimeout`, so a long activity is not reclaimed and
+  re-dispatched (duplicate execution) before it completes. Deriving the reclaim
+  window from `startToCloseTimeout` automatically makes a 60s+ activity safe by
+  default.
+
+---
+
+## 5. Documented concurrent-condition bound for harvest fan-out
+
+**Usage.** A market-maker workflow opens N `condition()` waits and harvests them
+with `Promise.all`, chunked to stay within the safe concurrent-condition bound (a
+fast signaler can arrive before a later condition registers).
+
+**Need.** Publish the safe concurrent-condition bound as a stable number with the
+recommended chunking pattern, so we size harvest fan-out to spec rather than
+empirically — and raise it where the engine can.
+
+---
+
+## 6. Signal buffering + startChild leader-keeps-seat as public contract
+
+**Usage.** Two behaviors our crash-safe patterns build on:
+- a signal delivered to a workflow **not yet parked** on that signalId is buffered
+  and delivered on the next `condition()` await;
+- `startChild` with a **duplicate workflowId** lets the sitting execution keep its
+  seat (leader-keeps-seat), enabling self-continuing link chains (`x-l1`, `x-l2`, …)
+  as an alternative to `continueAsNew`.
+
+**Need.** Elevate both to documented, stable public guarantees (including the signal
+buffering TTL), versioned as contract.
+
+---
+
+### How we consume these
+
+Items 1–2 are the scale needs — retention for terminal rows, and durable app-owned
+indexes that keep the hot path `O(pending)`. Item 3 is the atomicity guarantee our
+versioned forms and metadata routing depend on. Items 4–6 are behaviors we already
+build on and want made explicit. We can provide `EXPLAIN` plans, row-count/latency
+profiles, and the exact DDL for any of these on request.

package/RELEASE-0.26.0.md

@@ -0,0 +1,111 @@
+# HotMesh v0.26.0 — for the long-tail project
+
+This release answers the needs document. Items 1–5 and the signal-buffering
+half of item 6 ship as published contracts with proving tests; the startChild
+contract (item 6b) is scheduled for its own follow-up release. Numbering below
+matches your document.
+
+## 1. Retention for terminal escalation rows — new API
+
+Terminal rows (`resolved`, `cancelled`, `expired`) are inert: every engine
+state transition guards on `status = 'pending'`, and the engine reads terminal
+rows only through `list()` / `get()` / `stats()`. Age them out with the new
+engine-owned call:
+
+```typescript
+let deleted: number;
+do {
+  ({ deleted } = await client.escalations.prune({ olderThan: '90 days' }));
+} while (deleted > 0);
+```
+
+- `olderThan` is a Postgres interval; rows qualify when
+  `updated_at < NOW() - olderThan`.
+- `statuses` narrows which terminal states are pruned (defaults to all three);
+  `namespace` scopes to one app; `limit` bounds rows per call (default 10,000,
+  cap 100,000) so vacuum pressure stays flat.
+- Each call is one atomic statement — candidate selection
+  (`FOR UPDATE SKIP LOCKED`) and the DELETE commit together, so concurrent
+  pruners cooperate and live waiters, claims, and signal delivery proceed
+  untouched. The proving test backdates a parked waiter 120 days, prunes, and
+  resolves it to completion.
+- If you run retention with your own tooling instead, the documented
+  safe-to-delete predicate is exactly the one `prune()` uses:
+  `status IN ('resolved','cancelled','expired') AND updated_at` older than
+  your horizon.
+- Choose a horizon at least as long as your reporting window: `stats()`
+  created/resolved counts and `get()` by id reflect the rows retained.
+
+## 2. App-added indexes survive engine migrations — published contract
+
+Engine migrations on `public.hmsh_escalations` are strictly additive
+(`CREATE TABLE / INDEX IF NOT EXISTS`, `ADD COLUMN IF NOT EXISTS`, advisory-
+locked), and the table persists in place across upgrades. The engine manages
+only indexes with the `idx_hmsh_esc_*` prefix; indexes under any other name
+are yours and persist. Your advisory-locked
+`CREATE INDEX CONCURRENTLY IF NOT EXISTS` pattern is the recommended one.
+
+Two notes on your shapes: the engine's `idx_hmsh_esc_available`
+(`(namespace, app_id, role, priority ASC, created_at ASC) WHERE
+status = 'pending'`) already serves the single-role claim/available order at
+`O(pending)`. Your pending-scoped GIN stays app-owned: the engine keeps its
+full-table GIN because resolver-merged metadata (v0.24.0) is read on resolved
+rows.
+
+## 3. Atomic metadata in the Leg1 checkpoint — published contract
+
+Every field of the `condition()` config — `metadata` facets included — is one
+INSERT committed inside the Leg1 checkpoint transaction: one commit,
+crash-safe. From the row's first visible moment it carries its complete
+routing context and metadata, so claim-by-metadata routing and
+`schema_version`-pinned forms can trust every row they read. The proving test
+tight-polls from workflow start and asserts the first sighting is complete.
+
+## 4. Long activities execute exactly once — published contract
+
+- `startToCloseTimeout` is honored for long activities (deadline enforcement
+  is covered by positive and negative tests).
+- While an activity runs, the consumer heartbeats its stream reservation at
+  half the base window, so the message stays leased for the full run —
+  your 60s batch loop (poll → reconcile → act, one checkpoint per call) is a
+  supported shape at default settings, for any duration. Redelivery happens
+  when a consumer crashes and stops heartbeating, and the collation ledger
+  settles a redelivered message as a duplicate before the activity function
+  re-executes.
+- The proving test runs a 45s activity against a 30s reservation window with
+  a second worker actively polling the same queue: exactly one execution.
+- Deployment note: standard Postgres connections have this lease extension.
+  A `securedWorker` connection (SECURITY DEFINER stored-proc mode) enforces
+  the adaptive reservation window instead — 30s base, scaling to 30 minutes
+  under load — so size activities to that window in secured mode.
+
+## 5. Concurrent-condition bound — the published number is a TTL
+
+Fan-out is sized by time, and by count it scales freely. A signal that races
+ahead of its `condition()` registration is buffered as a pending signal and
+delivered — crash-safe, in a single transaction — when the wait registers.
+The buffer holds a signal for **10 minutes** by default; pass `expire` to
+`signal()` (e.g. `'1h'`, `'30d'`) when signaling early on purpose.
+
+Chunk your harvest so every condition in a chunk registers within the TTL of
+its fastest signaler. The proving test opens 25 waits in one `Promise.all`
+and fires all 25 signals before any wait has registered: every signal is
+delivered to its own wait, in order.
+
+## 6. Contracts
+
+- **Signal buffering** is now a documented public guarantee (item 5 above,
+  TTL included).
+- **startChild leader-keeps-seat**: the sitting execution keeps its seat
+  today, and fire-and-forget `startChild` link chains (`x-l1`, `x-l2`, …)
+  work on that behavior. The *awaited* duplicate path (`executeChild` on an
+  id that already exists) gets defined semantics — a typed result rather
+  than an open wait — in a dedicated follow-up release; until then, treat
+  awaited duplicates as unspecified and build chains on the fire-and-forget
+  form.
+
+## Upgrading
+
+`npm install @hotmeshio/hotmesh@0.26.0`. Migrations are additive and run
+automatically under the existing advisory lock; your indexes and views
+persist. Full durable suite: 433 tests green.

package/build/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@hotmeshio/hotmesh",
-    "version": "0.25.6",
+    "version": "0.26.0",
     "description": "Durable Workflow",
     "main": "./build/index.js",
     "types": "./build/index.d.ts",

package/build/services/durable/workflow/condition.d.ts

@@ -46,9 +46,12 @@ import { ConditionQueueConfig } from '../../../types/hmsh_escalations';
  * ## With escalation queue config
  *
  * Pass a {@link ConditionQueueConfig} as the second argument to surface the
- * pause as a claimable row in `public.hmsh_escalations`. The INSERT is
- * committed atomically with the workflow checkpoint — one write, no
- * enrichment step, no secondary round-trip.
+ * pause as a claimable row in `public.hmsh_escalations`. The INSERT — every
+ * field of the config, including `metadata` facets — is committed atomically
+ * with the workflow checkpoint: one write, one commit, crash-safe. From the
+ * row's first visible moment it carries its complete routing context and
+ * metadata, so claim-by-metadata routing and version-pinned facets (e.g. a
+ * `schema_version` the resolver UI renders) can trust every row they read.
  *
  * ```typescript
  * const decision = await Durable.workflow.condition<{ approved: boolean }>(
@@ -81,6 +84,14 @@ import { ConditionQueueConfig } from '../../../types/hmsh_escalations';
  * SLA-gated human waits in the workflow body and let hook functions report
  * back via `signal()`.
  *
+ * ## Early signals are buffered
+ *
+ * A signal delivered before its `condition()` registers — a fast signaler,
+ * or a payload deposited before the workflow starts — is buffered as a
+ * pending signal and delivered when the wait registers. The buffer holds a
+ * signal for 10 minutes by default; pass `expire` to `signal()` (e.g.
+ * `'1h'`, `'30d'`) to hold it longer when signaling early on purpose.
+ *
  * ## Fan-in: wait for multiple signals in parallel
  *
  * ```typescript
@@ -90,6 +101,12 @@ import { ConditionQueueConfig } from '../../../types/hmsh_escalations';
  * ]);
  * ```
  *
+ * Harvest fan-out scales the same way: open N waits with `Promise.all` and
+ * signal all of them at once. Buffering covers every signal that races
+ * ahead of registration, so size fan-out by the pending-signal TTL — how
+ * long a racing signal may wait for its condition to register — with no
+ * separate bound on the number of concurrent waits.
+ *
  * ## Paired with hook: spawn work, wait for its signal
  *
  * ```typescript

package/build/services/durable/workflow/condition.js

@@ -51,9 +51,12 @@ const didRun_1 = require("./didRun");
  * ## With escalation queue config
  *
  * Pass a {@link ConditionQueueConfig} as the second argument to surface the
- * pause as a claimable row in `public.hmsh_escalations`. The INSERT is
- * committed atomically with the workflow checkpoint — one write, no
- * enrichment step, no secondary round-trip.
+ * pause as a claimable row in `public.hmsh_escalations`. The INSERT — every
+ * field of the config, including `metadata` facets — is committed atomically
+ * with the workflow checkpoint: one write, one commit, crash-safe. From the
+ * row's first visible moment it carries its complete routing context and
+ * metadata, so claim-by-metadata routing and version-pinned facets (e.g. a
+ * `schema_version` the resolver UI renders) can trust every row they read.
  *
  * ```typescript
  * const decision = await Durable.workflow.condition<{ approved: boolean }>(
@@ -86,6 +89,14 @@ const didRun_1 = require("./didRun");
  * SLA-gated human waits in the workflow body and let hook functions report
  * back via `signal()`.
  *
+ * ## Early signals are buffered
+ *
+ * A signal delivered before its `condition()` registers — a fast signaler,
+ * or a payload deposited before the workflow starts — is buffered as a
+ * pending signal and delivered when the wait registers. The buffer holds a
+ * signal for 10 minutes by default; pass `expire` to `signal()` (e.g.
+ * `'1h'`, `'30d'`) to hold it longer when signaling early on purpose.
+ *
  * ## Fan-in: wait for multiple signals in parallel
  *
  * ```typescript
@@ -95,6 +106,12 @@ const didRun_1 = require("./didRun");
  * ]);
  * ```
  *
+ * Harvest fan-out scales the same way: open N waits with `Promise.all` and
+ * signal all of them at once. Buffering covers every signal that races
+ * ahead of registration, so size fan-out by the pending-signal TTL — how
+ * long a racing signal may wait for its condition to register — with no
+ * separate bound on the number of concurrent waits.
+ *
  * ## Paired with hook: spawn work, wait for its signal
  *
  * ```typescript

package/build/services/durable/workflow/proxyActivities.d.ts

@@ -111,6 +111,20 @@ declare function wrapActivity<T>(activityName: string, options?: ActivityConfig)
  * }
  * ```
  *
+ * ## Long-running activities execute exactly once
+ *
+ * `startToCloseTimeout` bounds an activity's run and is honored for long
+ * work (a 30–120s batch loop is a supported shape: poll → reconcile → act,
+ * one durable checkpoint per call). While the activity runs, the consumer
+ * heartbeats its stream reservation at half the base window, so the message
+ * stays leased for the full run — however long — and is redelivered to
+ * another worker only when the owning consumer crashes and stops
+ * heartbeating. The collation ledger then guarantees any redelivered
+ * message settles as a duplicate before re-executing the activity. With a
+ * `securedWorker` connection (SECURITY DEFINER stored-proc mode), lease
+ * extension is unavailable and an activity must finish within the adaptive
+ * reservation window instead.
+ *
  * @template ACT - The activity type map (use `typeof activities` for inline registration).
  * @param {ActivityConfig} [options] - Activity configuration including retry policy and routing.
  * @returns {ProxyType<ACT>} A typed proxy object mapping activity names to their durable wrappers.

package/build/services/durable/workflow/proxyActivities.js

@@ -230,6 +230,20 @@ exports.wrapActivity = wrapActivity;
  * }
  * ```
  *
+ * ## Long-running activities execute exactly once
+ *
+ * `startToCloseTimeout` bounds an activity's run and is honored for long
+ * work (a 30–120s batch loop is a supported shape: poll → reconcile → act,
+ * one durable checkpoint per call). While the activity runs, the consumer
+ * heartbeats its stream reservation at half the base window, so the message
+ * stays leased for the full run — however long — and is redelivered to
+ * another worker only when the owning consumer crashes and stops
+ * heartbeating. The collation ledger then guarantees any redelivered
+ * message settles as a duplicate before re-executing the activity. With a
+ * `securedWorker` connection (SECURITY DEFINER stored-proc mode), lease
+ * extension is unavailable and an activity must finish within the adaptive
+ * reservation window instead.
+ *
  * @template ACT - The activity type map (use `typeof activities` for inline registration).
  * @param {ActivityConfig} [options] - Activity configuration including retry policy and routing.
  * @returns {ProxyType<ACT>} A typed proxy object mapping activity names to their durable wrappers.

package/build/services/escalations/client.d.ts

@@ -1,7 +1,7 @@
 import { HotMesh } from '../hotmesh';
 import { EventsConfig } from '../../types/system_events';
 import { Connection } from '../../types/durable';
-import { EscalationEntry, ClaimEscalationResult, ClaimByMetadataResult, ReleaseEscalationResult, ResolveEscalationResult, CancelEscalationResult, ListEscalationsParams, StatsEscalationsParams, EscalationStats, CreateEscalationParams, UpdateEscalationParams, AppendMilestonesParams, ClaimEscalationParams, ClaimByMetadataParams, ReleaseEscalationParams, ResolveEscalationParams, ResolveByMetadataParams, EscalateToRoleParams, MigrateEscalationParams, ClaimManyParams, EscalateManyToRoleParams, UpdateManyPriorityParams, ResolveManyParams } from '../../types/hmsh_escalations';
+import { EscalationEntry, ClaimEscalationResult, ClaimByMetadataResult, ReleaseEscalationResult, ResolveEscalationResult, CancelEscalationResult, ListEscalationsParams, StatsEscalationsParams, EscalationStats, CreateEscalationParams, UpdateEscalationParams, AppendMilestonesParams, ClaimEscalationParams, ClaimByMetadataParams, ReleaseEscalationParams, ResolveEscalationParams, ResolveByMetadataParams, EscalateToRoleParams, MigrateEscalationParams, ClaimManyParams, EscalateManyToRoleParams, UpdateManyPriorityParams, ResolveManyParams, PruneEscalationsParams, PruneEscalationsResult } from '../../types/hmsh_escalations';
 export type GetHotMeshFn = (topic: string | null, namespace?: string) => Promise<HotMesh>;
 export interface EscalationClientConfig {
     /** Postgres connection options — used when creating a standalone EscalationClient. */
@@ -93,6 +93,18 @@ export declare class EscalationClientService {
      * Uses the same filter parameters as `list()`.
      */
     count(params?: ListEscalationsParams): Promise<number>;
+    /**
+     * Retention: deletes terminal escalation rows (`resolved`/`cancelled`/`expired`)
+     * whose `updated_at` is older than the horizon (a Postgres interval string,
+     * e.g. `'90 days'`). Terminal rows are inert — every engine state transition
+     * guards on `status = 'pending'` — so pruning them is safe for live waiters,
+     * claims, and signal delivery; this call is the engine-blessed way to age
+     * out the audit backlog. Each call deletes at most `limit` rows (default
+     * 10,000) in one atomic statement; loop until `deleted` is 0 to drain a
+     * large backlog. `list()`/`get()`/`stats()` reads over windows older than
+     * the pruning horizon reflect only the rows retained.
+     */
+    prune(params: PruneEscalationsParams): Promise<PruneEscalationsResult>;
     /** Returns a single escalation row by UUID. Returns `null` if not found. */
     get(id: string, namespace?: string): Promise<EscalationEntry | null>;
     /** Looks up an escalation by `signal_key` — the value passed to `condition()`. */

package/build/services/escalations/client.js

@@ -201,6 +201,21 @@ class EscalationClientService {
         const hm = await this._engine(null, params?.namespace);
         return hm.engine.store.countEscalations(params ?? {});
     }
+    /**
+     * Retention: deletes terminal escalation rows (`resolved`/`cancelled`/`expired`)
+     * whose `updated_at` is older than the horizon (a Postgres interval string,
+     * e.g. `'90 days'`). Terminal rows are inert — every engine state transition
+     * guards on `status = 'pending'` — so pruning them is safe for live waiters,
+     * claims, and signal delivery; this call is the engine-blessed way to age
+     * out the audit backlog. Each call deletes at most `limit` rows (default
+     * 10,000) in one atomic statement; loop until `deleted` is 0 to drain a
+     * large backlog. `list()`/`get()`/`stats()` reads over windows older than
+     * the pruning horizon reflect only the rows retained.
+     */
+    async prune(params) {
+        const hm = await this._engine(null, params.namespace);
+        return hm.engine.store.pruneEscalations(params);
+    }
     /** Returns a single escalation row by UUID. Returns `null` if not found. */
     async get(id, namespace) {
         const hm = await this._engine(null, namespace);

package/build/services/store/providers/postgres/postgres.d.ts

@@ -331,5 +331,18 @@ declare class PostgresStoreService extends StoreService<ProviderClient, Provider
     escalationStats(params?: import('../../../../types/hmsh_escalations').StatsEscalationsParams): Promise<import('../../../../types/hmsh_escalations').EscalationStats>;
     listDistinctEscalationTypes(namespace?: string): Promise<string[]>;
     releaseExpiredEscalations(_namespace?: string): Promise<number>;
+    /**
+     * Deletes terminal escalation rows (`resolved`/`cancelled`/`expired`) whose
+     * `updated_at` is older than the given horizon. Terminal rows are inert —
+     * every state transition guards `status = 'pending'` — so pruning them is
+     * the engine-blessed retention path for the audit backlog.
+     *
+     * Single statement: the candidate SELECT (`FOR UPDATE SKIP LOCKED`) and the
+     * DELETE are one atomic unit, so concurrent pruners and readers never block
+     * each other and a row is either fully gone or fully present. `limit` bounds
+     * rows per call to keep vacuum pressure flat; callers loop until `deleted`
+     * returns 0.
+     */
+    pruneEscalations(params: import('../../../../types/hmsh_escalations').PruneEscalationsParams): Promise<import('../../../../types/hmsh_escalations').PruneEscalationsResult>;
 }
 export { PostgresStoreService };

package/build/services/store/providers/postgres/postgres.js

@@ -2318,5 +2318,44 @@ class PostgresStoreService extends __1.StoreService {
         // assigned_until, so no background sweep is needed to release expired claims.
         return 0;
     }
+    /**
+     * Deletes terminal escalation rows (`resolved`/`cancelled`/`expired`) whose
+     * `updated_at` is older than the given horizon. Terminal rows are inert —
+     * every state transition guards `status = 'pending'` — so pruning them is
+     * the engine-blessed retention path for the audit backlog.
+     *
+     * Single statement: the candidate SELECT (`FOR UPDATE SKIP LOCKED`) and the
+     * DELETE are one atomic unit, so concurrent pruners and readers never block
+     * each other and a row is either fully gone or fully present. `limit` bounds
+     * rows per call to keep vacuum pressure flat; callers loop until `deleted`
+     * returns 0.
+     */
+    async pruneEscalations(params) {
+        const TERMINAL = ['resolved', 'cancelled', 'expired'];
+        const statuses = params.statuses?.length
+            ? params.statuses.filter((s) => TERMINAL.includes(s))
+            : [...TERMINAL];
+        if (!statuses.length)
+            return { deleted: 0 };
+        const limit = Math.max(1, Math.min(params.limit ?? 10000, 100000));
+        const values = [statuses, params.olderThan, limit];
+        let nsClause = '';
+        if (params.namespace) {
+            values.push(params.namespace);
+            nsClause = `AND namespace = $${values.length}`;
+        }
+        const result = await this.pgClient.query(`WITH doomed AS (
+         SELECT ctid FROM public.hmsh_escalations
+         WHERE status = ANY($1::text[])
+           AND status IN ('resolved', 'cancelled', 'expired')
+           AND updated_at < NOW() - $2::interval
+           ${nsClause}
+         LIMIT $3
+         FOR UPDATE SKIP LOCKED
+       )
+       DELETE FROM public.hmsh_escalations e
+       USING doomed WHERE e.ctid = doomed.ctid`, values);
+        return { deleted: result.rowCount ?? 0 };
+    }
 }
 exports.PostgresStoreService = PostgresStoreService;

package/build/types/hmsh_escalations.d.ts

@@ -131,6 +131,29 @@ export type CancelEscalationResult = {
     ok: false;
     reason: 'not-found' | 'already-terminal';
 };
+/**
+ * Retention parameters for `pruneEscalations`. Prunes only terminal rows
+ * (`resolved`/`cancelled`/`expired`) — the statuses every engine state
+ * transition treats as final — older than the given horizon.
+ */
+export interface PruneEscalationsParams {
+    /**
+     * Age horizon as a Postgres interval string (e.g. `'90 days'`, `'12 hours'`).
+     * Rows qualify when `updated_at < NOW() - olderThan`.
+     */
+    olderThan: string;
+    /** Terminal statuses to prune. Defaults to all three; non-terminal values are ignored. */
+    statuses?: Array<'resolved' | 'cancelled' | 'expired'>;
+    namespace?: string;
+    /**
+     * Max rows deleted per call (bounds lock time and vacuum pressure).
+     * Default 10,000, capped at 100,000. Loop until `deleted` is 0 to drain.
+     */
+    limit?: number;
+}
+export interface PruneEscalationsResult {
+    deleted: number;
+}
 export interface ListEscalationsParams {
     namespace?: string;
     role?: string;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hotmeshio/hotmesh",
-  "version": "0.25.6",
+  "version": "0.26.0",
   "description": "Durable Workflow",
   "main": "./build/index.js",
   "types": "./build/index.d.ts",

package/ISSUE.md

@@ -1,64 +0,0 @@
-## EIGHTH ITEM — a claimed row's post-signal settle can fail, stranding pending state that contradicts the workflow (dashboard-visible)
-
-Two order trails, same signature (hike-mono, 0.25.5/0.7.5):
-- run A order `a0e68dba`: every station row resolved EXCEPT quality-control —
-  yet the QC stationWorker delivered (shipping minted 5s later, order
-  COMPLETED). The QC row sits `pending` + claimed forever.
-- run B order `89942462`: identical, at post-print-qa (claimed 07:34:44,
-  gluing resolved 07:34:54, order COMPLETED 07:35:11; the qa row still
-  pending+claimed 20+ min later).
-
-Trigger context (our side, fixed by us): two associates raced a shallow pond
-and BOTH booked the same order in the same second (audit rows at 07:34:44) —
-so a concurrent claim write was interleaved with the resolve.
-
-The library-side gap: the resolve path delivered the SIGNAL (workflow
-advanced, next station minted) but the row's own transition to `resolved`
-was lost against the concurrent claim — no error surfaced anywhere. The
-result is dashboard-visible corruption: a live, claimable, countdown-bearing
-work item asserting QA is owed on a COMPLETED order (real associates would
-work it). Whatever branch handled this (atomic signal_key resolve or the
-legacy handle.signal → ltResolveEscalation), signal delivery and row settle
-must be atomic — or the settle must retry until the row cannot lie.
-
-Repro shape: claim a signal_key escalation from principal A, and within the
-same second re-claim/book from principal B while principal C resolves by
-metadata. Expect: signal delivered AND row resolved, always.
-
-# RE-FILED — two reports lost when this file was replaced with your 0.25.6 reply (both found the same evening, before it)
-
-## NINTH ITEM (re-filed seventh) — a timeout wake KILLS the collator of a collated wait set; parent hangs forever
-
-`Promise.all(21 × condition(signalId, {timeout:'1320s'}))` in one parent
-(hike-mono simulationDirector, 0.25.5). Signal-path delivery through the
-collation worked (seat 18 resolved cleanly at 04:55:05, its collator_waiter
-slot consumed). At 05:07:25 UTC the FIRST batch of SLA expirations landed
-(vm0..vm4 rows flipped 'expired' in the same second) and the collator job
-(`HfSVaQSgUaYu3XGsCCjyQWy$C`) went **status -1 in that same second**
-(context={}, is_live=t). Every subsequent expiry wake then hit
-`Inactive job …$C` at CollatorService.assertJobActive (collator/index.js:13)
-via Hook.processTimeHookEvent (hook.js:650) and was dropped. End state,
-stable 10+ min: 20 live collator_waiter registry rows, all rows 'expired',
-zero pending engine_streams for the collator, parent wedged in Promise.all
-forever with no surfaced error.
-
-Repro: `await Promise.all([condition('k1',{timeout:'30s'}), condition('k2',{timeout:'30s'})])`,
-no resolver — let both expire. Expect collator -1 on first expiry, parent wedged.
-
-Our standing mitigation: each timeout-bearing wait moved into its own child
-workflow (inline wait), parent collates ExecChild completions — works, but
-collated waits + SLA should compose.
-
-## TENTH ITEM (re-filed follow-up) — second of two near-simultaneous sibling wakes lost (no crash, no restart)
-
-Clean floor, 10 orders: per order, TWO sibling printJobWorkers under one
-dispatcher park signal_key waits; a machine resolves BOTH escalations
-seconds apart. 3-for-3 wedged orders: LEFT worker woke and completed; RIGHT
-worker (resolved moments later) never woke — job row untouched since
-creation, 10+ min, stored-pending-signal path did not recover it. 14/14
-wakes delivered for the 7 orders whose resolves were spread out. Process up
-the whole time (distinct from the reclaim case). Plausibly fixed by commit
-3's atomic wake INSERT ("no interleave window") — flagging so it gets a
-sibling-burst regression test: resolve two waits of one parent within the
-same tick, both must wake. Our mitigation: 0–4s jitter between sibling
-resolves (also more realistic for print hardware).

package/RESPONSE-0.25.6.md

@@ -1,77 +0,0 @@
-# hotmesh 0.25.6 — fixes for the wake-loss and reservation reports (branch: fix/atomic-cancel-wake, three commits)
-
-Everything below is implemented and fail-first tested (each fix was watched failing
-before its change); the branch is in final full-suite verification before merge.
-
-## Commit 1 — cancellation wakes are durable
-
-**Your second addendum (3 of 6 pills slept after the 400ms cancel wave).** Your
-watchers settle waits via `cancel()`, and cancel's wake was a post-commit,
-best-effort publish whose failure was silently discarded. Now the cancellation
-wake commits inside the cancel transaction: a cancel that is durable always
-resumes its `condition()` with `null`. A native six-pill burst test pins your
-wave shape.
-
-Also adds a permanent regression test for your converge topology
-(`Promise.all([executeChild x4, sleep branch])`): the sleep's timehook mints and
-the converge settles.
-
-## Commit 2 — dead reservations reclaim; resolved waits disarm their timers
-
-**Your fifth data point (reservations never reclaimed after process death) — the
-highest-leverage fix.** The fallback poller's discovery query filtered
-`reserved_at IS NULL`, so a dead holder's reservation was invisible forever.
-It now surfaces stale reservations (threshold tracks the configured window;
-heartbeating work never matches), through dedicated index-backed branches, and
-the fixed function re-deploys to existing databases on boot. A crash or deploy
-now costs ≤ ~90 seconds of redelivery latency instead of a permanent wedge —
-and this most plausibly explains your third data point too (wake messages
-claimed by churning per-principal clients that died were never reclaimed).
-
-**Your sixth item (armed-timeout drizzle).** When the signal wins an SLA-gated
-wait, the armed timeout is now disarmed with the settled wait — root and cycled
-dimensions both covered. New timers are jid-stamped so job-scoped purges
-(terminate) remove them as well. Timers armed before the upgrade drain as the
-one-shot benign events you observed.
-
-Also hardens the pending-signal handoff and cancel wake into single atomic
-statements (safe on pooled connections, no interleave window; an in-window
-wake refreshed by a newer signal delivers the newest payload).
-
-## Commit 3 — resolve paths made single-statement; bulk resolve protects waiters
-
-- `resolve()` and `resolveByMetadata()` now use the same single-statement
-  atomic idiom as cancel: row lock, status guard, and the wake INSERT commit
-  as one unit on any connection type.
-- `resolveMany()` now excludes rows backing a live `condition()` waiter — bulk
-  resolution carries no wake, so those rows stay pending for a targeted
-  `resolve()`/`cancel()` instead of silently stranding the workflow.
-- Secured-mode worker responses stamp `jid`, so job-scoped purges address them.
-
-## Status of the remaining items
-
-- **Lost `sleep()` timer (your timer forensics):** still not reproduced — your
-  exact topology mints and converges cleanly under test, and every registration
-  path checks out atomic. The reclaim fix (commit 2) may retire this too if the
-  collator's driving message was claimed by a dying client. The decisive
-  forensic if it recurs: the wedged director's collator job
-  (`SELECT * FROM durable.jobs WHERE key LIKE '%$C'`) plus that collator jid's
-  `engine_streams` rows, no filters — it records exactly which item-cycle
-  stopped.
-- **`createClient` construction churn (your fourth data point):** your
-  per-principal memoization is the right mitigation; an upstream answer
-  (internal per-auth memoization or an explicit `client.close()`) is under
-  design as its own change.
-- **Crash windows on interrupt's parent notification** (found by our internal
-  audit, same family as your wake fixes): being engineered in a dedicated
-  follow-up PR — the straightforward implementation measurably slowed interrupt
-  dispatch, and interrupts race against live completions by design, so it is
-  getting the unhurried version.
-
-## After upgrading
-
-- Restarting workers drains any orphaned-reservation backlog automatically.
-- The settlement guard can be retired again — cancel and resolve wakes are both
-  transactional now, and dead reservations self-heal.
-- Watch for `hook-signal-pending-redelivered` (in-window wake handled) and the
-  reclaim redeliveries after any crash: both are the system working as intended.

package/wedge-capture-20260706.csv

@@ -1,19 +0,0 @@
-jid,reserved_at,retry_attempt,created_at,message
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm3,2026-07-06 20:19:07.953447+00,0,2026-07-06 20:18:07.945876+00,"{""metadata"":{""guid"":""HxfFFVmLVwtk37194ysP1qQ"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm3"",""gid"":""HAXgFtL8ZicUsOywHCKUauE"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""virtual-farm-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""printerBatch"",""arguments"":[{""machineId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm3"",""operatorId"":""0dfa0000-0000-4000-8000-000000000004"",""printSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""runsSoFar"":6,""plan"":[],""advertId"":""0a4af7d1-2dc4-4e06-9d43-791acf3c49e5""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm3-$printerBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm3-$printerBatch$1-5"",""workflowTopic"":""virtual-farm-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm4,2026-07-06 20:19:10.889217+00,0,2026-07-06 20:18:10.880925+00,"{""metadata"":{""guid"":""HXZnTUGkTix4uF7q14PuP0f"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm4"",""gid"":""HO9SAZEoZGiycaugLuwuUXv"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""virtual-farm-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""printerBatch"",""arguments"":[{""machineId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm4"",""operatorId"":""0dfa0000-0000-4000-8000-000000000005"",""printSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""runsSoFar"":6,""plan"":[],""advertId"":""a1d6d90a-a2cf-4621-a74f-e0ed1013cbc6""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm4-$printerBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm4-$printerBatch$1-5"",""workflowTopic"":""virtual-farm-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm5,2026-07-06 20:19:16.082786+00,0,2026-07-06 20:18:16.072927+00,"{""metadata"":{""guid"":""HE7nFSVKoh2oFe6-lYtCJWV"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm5"",""gid"":""H697gK8YvdS-_waP4NLiL-L"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""virtual-farm-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""printerBatch"",""arguments"":[{""machineId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm5"",""operatorId"":""0dfa0000-0000-4000-8000-000000000006"",""printSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""runsSoFar"":4,""plan"":[],""advertId"":""df441cd6-1811-41f0-b646-3c794ebf71d4""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm5-$printerBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm5-$printerBatch$1-5"",""workflowTopic"":""virtual-farm-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm1,2026-07-06 20:19:05.227375+00,0,2026-07-06 20:18:20.216145+00,"{""metadata"":{""guid"":""HTFIOwV5q0HYi8z04GwX9qj"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm1"",""gid"":""HSR3wD1ryGQ6f5vzzGcO8eV"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""virtual-farm-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""printerBatch"",""arguments"":[{""machineId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm1"",""operatorId"":""0dfa0000-0000-4000-8000-000000000002"",""printSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""runsSoFar"":13,""plan"":[],""advertId"":""e8ee662b-6de6-4a57-b8d6-ee9ed76e1f99""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm1-$printerBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm1-$printerBatch$1-5"",""workflowTopic"":""virtual-farm-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm0,2026-07-06 20:19:09.068044+00,0,2026-07-06 20:18:39.059327+00,"{""metadata"":{""guid"":""HcFhO_I7Fqn_5ftBd4uUTuZ"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm0"",""gid"":""HyVUxkj0OkvEO2r0sdgtTan"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""virtual-farm-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""printerBatch"",""arguments"":[{""machineId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm0"",""operatorId"":""0dfa0000-0000-4000-8000-000000000001"",""printSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""runsSoFar"":12,""plan"":[],""advertId"":""53bd1b3a-8ec3-4c35-a839-da93d9e23bc0""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm0-$printerBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm0-$printerBatch$1-5"",""workflowTopic"":""virtual-farm-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm2,2026-07-06 20:19:18.195319+00,0,2026-07-06 20:19:18.191286+00,"{""metadata"":{""guid"":""HWpXdbtZ-Bx5iqINxvAa6wf"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm2"",""gid"":""HWzNcP3ToTg27Qc5u5MljTT"",""dad"":"",0,16,0,0"",""aid"":""proxyer"",""topic"":""virtual-farm-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""printerBatch"",""arguments"":[{""machineId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm2"",""operatorId"":""0dfa0000-0000-4000-8000-000000000003"",""printSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""runsSoFar"":8,""plan"":[],""advertId"":""a0a84553-5031-456d-ae02-5d9746b606fc""}],""workflowDimension"":""$1"",""index"":6,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm2-$printerBatch$1-6"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-farm-vm2-$printerBatch$1-6"",""workflowTopic"":""virtual-farm-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-manager,2026-07-06 20:19:14.76557+00,0,2026-07-06 20:18:29.759824+00,"{""metadata"":{""guid"":""HzkL-DZTtXyIyDV8oZNZJxI"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-manager"",""gid"":""HCGavoqirvD8FwdwQOWgcov"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""managerBatch"",""arguments"":[{""laneId"":""9a93b820-dde3-425b-a320-260b91c08ec2"",""reviewSeconds"":3,""moveSeconds"":3,""idleTickSeconds"":3,""maxIterations"":25,""actorEmail"":""sim.manager.marco@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-manager-$managerBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-manager-$managerBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-0,2026-07-06 20:19:12.094467+00,0,2026-07-06 20:18:42.089308+00,"{""metadata"":{""guid"":""H2MwGXqBeP8-8nPX-T6ndnU"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-0"",""gid"":""H7nf5o8VvELz6NHWGWGNsXe"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_QUALITY_CONTROL"",""role"":""quality-inspector"",""toStatus"":""NEEDS_SHIPPING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":7,""unparkedRejects"":[],""troubledOrders"":[""931a1e99-2db6-4280-b034-57e3c42a9aba""],""actorEmail"":""sim.inspector.ines@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-0-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-0-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-1,2026-07-06 20:19:12.096894+00,0,2026-07-06 20:18:42.093836+00,"{""metadata"":{""guid"":""H4pQ4lScxaW8wUEiP8BQh8E"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-1"",""gid"":""HQUqSMYqzsvODyjMvRNPtvl"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_QUALITY_CONTROL"",""role"":""quality-inspector"",""toStatus"":""NEEDS_SHIPPING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":7,""unparkedRejects"":[],""troubledOrders"":[""931a1e99-2db6-4280-b034-57e3c42a9aba""],""actorEmail"":""sim.inspector.tomas@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-1-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_quality_control-1-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-draft,2026-07-06 20:19:16.150935+00,0,2026-07-06 20:18:46.146153+00,"{""metadata"":{""guid"":""HgtJMxOb1oYv6jJb2uErF0_"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-draft"",""gid"":""HN6b23imRu-wgDHA-DpxzLV"",""dad"":"",0,14,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""DRAFT"",""role"":""insole-designer"",""toStatus"":""NEEDS_MANUFACTURING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":24,""unparkedRejects"":[],""troubledOrders"":[""2b294913-f8ca-4639-9928-3abd53ea9616""],""actorEmail"":""sim.designer.dana@sim.hike.local"",""worklistStation"":""design-approval""}],""workflowDimension"":""$1"",""index"":4,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-draft-$associateBatch$1-4"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-draft-$associateBatch$1-4"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-1,2026-07-06 20:19:04.938838+00,0,2026-07-06 20:18:49.932504+00,"{""metadata"":{""guid"":""HQD-nuRyxBmH431uJ8noMBS"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-1"",""gid"":""HWM3gVxjxz9_K-EnQpCeFjY"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_FINISHING"",""role"":""finisher"",""toStatus"":""NEEDS_QUALITY_CONTROL"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":9,""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.finisher.kofi@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-1-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-1-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-0,2026-07-06 20:19:14.016307+00,0,2026-07-06 20:18:59.008759+00,"{""metadata"":{""guid"":""HHBHdHpX2Wear_cVJim70Ps"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-0"",""gid"":""HkZyt5-JhLoCNX26Q4cyqbq"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_FINISHING"",""role"":""finisher"",""toStatus"":""NEEDS_QUALITY_CONTROL"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":11,""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.finisher.ada@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-0-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_finishing-0-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_shipping,2026-07-06 20:19:15.654293+00,0,2026-07-06 20:19:00.646061+00,"{""metadata"":{""guid"":""HffnP49JTwmHAwk3BNRn22l"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_shipping"",""gid"":""HuK-iSfDdYXXBJWo2WwhCDF"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_SHIPPING"",""role"":""shipping-associate"",""toStatus"":""COMPLETED"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":12,""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.shipper.june@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_shipping-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_shipping-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-1,2026-07-06 20:19:05.381804+00,0,2026-07-06 20:19:05.3792+00,"{""metadata"":{""guid"":""Hx82JQ7g0Y4XT7nqPFK4km9"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-1"",""gid"":""HoT3fZZg04_z7BKimsTs-Ei"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_GLUING"",""role"":""gluer"",""toStatus"":""NEEDS_FINISHING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":13,""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.gluer.dev@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-1-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-1-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-0,2026-07-06 20:19:05.386589+00,0,2026-07-06 20:19:05.384446+00,"{""metadata"":{""guid"":""HvNFncRHNbqR_OSOOiQ0M12"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-0"",""gid"":""Hk7Wy8HO98wh45Clgp5-sUo"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""NEEDS_GLUING"",""role"":""gluer"",""toStatus"":""NEEDS_FINISHING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":13,""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.gluer.rosa@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-0-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-needs_gluing-0-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-1,2026-07-06 20:19:06.832852+00,0,2026-07-06 20:19:06.829652+00,"{""metadata"":{""guid"":""Hg1xKYket-ULnS-HdHaNuiD"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-1"",""gid"":""HLjxCrhPCUyqBaWbwSE0pxp"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""POST_PRINT_QA"",""role"":""post-print-qa"",""toStatus"":""NEEDS_GLUING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":14,""rejectRate"":{""pct"":15,""targetStation"":""PRINTING"",""reasons"":[""Layer separation on the arch — reprint"",""Surface finish below spec"",""Edge delamination at the heel cup"",""Warping on the medial side""],""leftQuantity"":1,""rightQuantity"":0},""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.qa.elena@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-1-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-1-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-0,2026-07-06 20:19:12.746444+00,0,2026-07-06 20:19:12.744474+00,"{""metadata"":{""guid"":""HTl4tkkOMhgso2QMlQlKYvn"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-0"",""gid"":""HY9mpKnlDbC3a1o3B4R-53h"",""dad"":"",0,15,0,0"",""aid"":""proxyer"",""topic"":""actor-crew-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""associateBatch"",""arguments"":[{""station"":""POST_PRINT_QA"",""role"":""post-print-qa"",""toStatus"":""NEEDS_GLUING"",""dwellSeconds"":4,""betweenSeconds"":2,""idleTickSeconds"":3,""maxIterations"":25,""touchesSoFar"":16,""rejectRate"":{""pct"":15,""targetStation"":""PRINTING"",""reasons"":[""Layer separation on the arch — reprint"",""Surface finish below spec"",""Edge delamination at the heel cup"",""Warping on the medial side""],""leftQuantity"":1,""rightQuantity"":0},""unparkedRejects"":[],""troubledOrders"":[],""actorEmail"":""sim.qa.marcus@sim.hike.local""}],""workflowDimension"":""$1"",""index"":5,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-0-$associateBatch$1-5"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-crew-post_print_qa-0-$associateBatch$1-5"",""workflowTopic"":""actor-crew-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"
-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-design,2026-07-06 20:19:10.594054+00,0,2026-07-06 20:18:55.585455+00,"{""metadata"":{""guid"":""HeSdgdXIms_1Lhh2nNgc4EP"",""jid"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-design"",""gid"":""HU01R82t5rnplNLKWryLJPz"",""dad"":"",0,40,0,0"",""aid"":""proxyer"",""topic"":""simulation-activity"",""wfn"":""durable.execute""},""data"":{""activityName"":""designTwinBatch"",""arguments"":[{""user"":""sim.taika.iris@sim.hike.local"",""designSeconds"":2,""idleTickSeconds"":3,""maxIterations"":10,""concurrency"":4,""designedOrders"":[""0f596ab9-f449-485d-b06d-e74864750ae7"",""a5cedbc2-f726-4411-9ac3-10fd113532d2"",""c1a09bd7-bd4b-4eb1-bd3e-91d6b45b479f"",""94d74e0e-e4ce-4db7-91f3-02c1c081c2ba"",""0ed99be7-83b6-4a91-9aa5-560d5c110c6b"",""33b987bb-9253-483f-baa4-37d4ffd0903c"",""2c4bc240-1335-423d-96af-e5855b6ccc0f"",""0e7f24bb-3fba-40e5-a5a8-666f5c91fd49"",""ad0c05b6-4528-4f2a-9569-d06c76589b54"",""de2a02e2-33ca-4c0c-843c-5ad422c7c517"",""c008abd0-8b3b-44f7-9f1c-7d314f72f587"",""04a28a34-20f3-4d10-a636-3cb8f504f56e"",""52295fa8-452b-48ca-862b-64cf3ff8cae6"",""cb1dd956-639d-4ff8-9f56-c12f9c5278e8"",""fca4a3f0-fc49-47cd-8285-bf7e11c60dbe"",""bcd42ec0-f047-4acc-8e70-33575ae7535a"",""3b73fb95-2366-4498-ac9a-3f2d9eb4d1c0"",""3c9d174f-7928-42e7-9edb-2f3b38ba0ddc"",""1e61d445-77ba-4013-af89-1a6de07ff407"",""0b066160-06ab-4681-af20-8c36866c9987"",""fa9f996c-2327-4f30-8fdc-e63239483f23"",""ff31ec18-c7e3-4907-85f0-5c4eda42841f"",""f17a825c-8905-4166-b687-54015ddeae6c"",""931a1e99-2db6-4280-b034-57e3c42a9aba"",""2b294913-f8ca-4639-9928-3abd53ea9616"",""04caa323-662a-4c28-8de0-3655d435f206"",""b2eac688-d074-45d9-8a60-e87719b18c81"",""a11092fe-31e5-4cfe-815c-230a09f2165e"",""176e3d03-0813-4816-83fd-a785798a83bb"",""e1d887bd-1ad4-4436-b728-d75e7df2c39a"",""c8a0ffdc-91b0-443a-a8f9-c01c2e3935c0"",""83cbe6eb-ea9b-492e-8657-4f138e3df05e"",""e8e56685-82a3-4f88-a804-b6a8c21f0a4c"",""89d231bf-600b-40ed-8161-e58dbcc7157c"",""6904b329-7a63-465f-aa6a-82a1095638fb"",""9ca2abe9-ae5f-459a-a003-9ea4ad5cafc0"",""cae10b57-439a-4e98-b0d0-1bb172a9d79f"",""f90af7cb-401b-4e68-9f22-a45d34581d64"",""86a1d1b2-7287-48ab-94b3-544a743fda41"",""76793c77-67a0-4148-b83b-fa50f900111c""]}],""workflowDimension"":""$3"",""index"":8,""originJobId"":""simulationDirector-H5zQndzkcfIGHhUM-QXFzcG"",""parentWorkflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-design-$designTwinBatch$3-8"",""workflowId"":""-simulationDirector-H5zQndzkcfIGHhUM-QXFzcG-design-$designTwinBatch$3-8"",""workflowTopic"":""simulation-activity"",""expire"":2592000,""backoffCoefficient"":5,""initialInterval"":1,""maximumAttempts"":3,""maximumInterval"":120}}"