@hotmeshio/hotmesh 0.21.1 → 0.22.1

package/README.md

@@ -14,7 +14,7 @@ npm install @hotmeshio/hotmesh
 - **Crash-safe execution** — Every step is a committed row. If the process dies, it picks up where it left off.
 - **Distributed state machines** — Build stateful applications where every component can [fail and recover](https://github.com/hotmeshio/sdk-typescript/blob/main/services/collator/README.md).
 - **AI and training pipelines** — Multi-step AI workloads where each stage is expensive and must not be repeated on failure. A crashed pipeline resumes from the last committed step, not from the beginning.
-- **Human-in-the-loop queues** — Suspend workflows until an operator claims and resolves the task. Pending suspensions become discoverable, claimable rows in Postgres. Concurrent workers receive exact reason codes (`conflict`, `already-resolved`, `signal-failed`) instead of opaque nulls.
+- **Human-in-the-loop workflows** — Workflow pauses that require external input are searchable, claimable rows in your database. Build approval queues, review flows, and AI handoffs without a separate task service.
 
 ## How it works in 30 seconds
 
@@ -167,35 +167,24 @@ const result = await Durable.workflow.executeChild({
 **Signals** — pause a workflow until an external event arrives.
 
 ```typescript
-// Basic: pause until any caller sends the matching signal
 const approval = await Durable.workflow.condition<{ approved: boolean }>('manager-approval');
 if (!approval.approved) return 'rejected';
 ```
 
-Pass a `ConditionQueueConfig` as the second argument to also create a claimable task record in Postgres (see [Signal queue](#signal-queue-postgres-only) below):
+**Human-in-the-loop** — When a workflow pauses waiting for input, it can write a searchable, claimable row to the database. External systems — dashboards, AI agents, APIs — query by role and metadata, claim ownership, and resolve to resume the workflow. The pause lives in `public.hmsh_escalations` alongside the rest of your workflow state.
 
 ```typescript
-// With queue config: atomically suspend the workflow AND enqueue a claimable task record
-const approval = await Durable.workflow.condition<{ approved: boolean }>('manager-approval', {
-  role: 'manager',
-  description: `Approve order ${orderId}`,
-  metadata: { orderId, region: 'us-west' }, // GIN-indexed; used for claimByMetadata queries
-  envelope: { formSchema: [...] },           // unindexed display context; safe for large blobs
-});
-// approval is false if a timeout was set and no signal arrived
-if (approval === false) return 'timed-out';
-if (!approval.approved) return 'rejected';
-```
-
-The optional first string argument is a timeout; the queue config can appear as the second or third argument:
-
-```typescript
-// With timeout AND queue config
-const approval = await Durable.workflow.condition<{ approved: boolean }>(
-  'manager-approval',
-  '72 hours',                  // workflow times out if nobody acts
-  { role: 'manager', metadata: { orderId } },
+// workflow: pause and write a claimable row
+const decision = await Durable.workflow.condition<{ approved: boolean }>(
+  'approval',
+  { role: 'manager', type: 'order-review', metadata: { orderId } },
 );
+
+// elsewhere: find pending approvals, claim one, resolve it
+const [pending] = await client.escalations.list({ role: 'manager', status: 'pending' });
+await client.escalations.claim({ id: pending.id, assignee: 'alice@company.com' });
+await client.escalations.resolve({ id: pending.id, resolverPayload: { approved: true } });
+// the workflow resumes with { approved: true }
 ```
 
 ## Retries and error handling
@@ -271,112 +260,6 @@ const exported = await handle.export({          // selective export
 });
 ```
 
-## Signal queue (Postgres only)
-
-When `condition()` is called with a `ConditionQueueConfig`, HotMesh atomically writes a row to `hotmesh_signals` inside the same Postgres transaction that suspends the workflow. The result is a durable, queryable task queue backed by the same database — no separate service, no double-write, no gap between "task created" and "workflow suspended".
-
-**Workflow side** — declare what kind of task this suspension represents:
-
-```typescript
-// workflows/approval.ts
-export async function orderApproval(orderId: string) {
-  const signalId = `approve-${orderId}`;
-
-  const result = await Durable.workflow.condition<{ approved: boolean; notes: string }>(
-    signalId,
-    {
-      role: 'pharmacist',
-      type: 'approval',
-      priority: 3,
-      description: `Review order ${orderId}`,
-      taskQueue: 'rx-approvals',
-      workflowType: 'orderApproval',
-      metadata: { orderId },               // GIN-indexed; query with claimByMetadata
-      envelope: {                          // not indexed; pass form schemas, display context
-        formSchema: [{ name: 'notes', type: 'textarea', required: true }],
-      },
-    },
-  );
-
-  if (result === false) return { status: 'timed-out' };
-  return result;
-}
-```
-
-**Resolver side** — claim and resolve from any process (API handler, worker, dashboard):
-
-```typescript
-// resolver.ts
-import { Durable } from '@hotmeshio/hotmesh';
-
-const client = new Durable.Client({ connection });
-
-// List all pending tasks for a role
-const pending = await client.signalQueue.list({ role: 'pharmacist', status: 'pending' });
-
-// Claim by metadata — atomic; concurrent workers get 'conflict', not a silent null
-const claim = await client.signalQueue.claimByMetadata({
-  key: 'orderId',
-  value: orderId,
-  assignee: 'jane@example.com',
-  durationMinutes: 30,
-});
-
-if (!claim.ok) {
-  // claim.reason: 'not-found' (nothing queued) | 'conflict' (another worker beat you)
-  return;
-}
-
-// Resolve — marks the DB record resolved AND delivers the signal to the paused workflow
-const resolution = await client.signalQueue.resolve({
-  id: claim.entry.id,
-  resolverPayload: { approved: true, notes: 'LGTM' },
-});
-
-if (!resolution.ok) {
-  if (resolution.reason === 'signal-failed') {
-    // DB updated but workflow signal not delivered — retry with resolution.signalKey
-  }
-}
-```
-
-**All `signalQueue` methods:**
-
-| Method | Description |
-|---|---|
-| `list(params)` | List signals filtered by `role`, `status`, `taskQueue` |
-| `get(id)` | Fetch a single signal record by UUID |
-| `claim({ id, assignee?, durationMinutes? })` | Claim by record ID |
-| `claimByMetadata({ key, value, assignee?, durationMinutes? })` | Claim by GIN-indexed metadata field |
-| `release({ id })` | Return a claimed signal to `pending` |
-| `releaseExpired()` | Sweep all past-expiry claims back to `pending` |
-| `resolve({ id, resolverPayload? })` | Resolve by ID and deliver signal to the workflow |
-| `resolveByMetadata({ key, value, resolverPayload? })` | Resolve by metadata field and deliver signal |
-
-**Result types** — all mutating methods return a discriminated union so callers can handle every outcome without guessing what `null` meant:
-
-```typescript
-// Claim
-type ClaimSignalResult =
-  | { ok: true; entry: SignalQueueEntry }
-  | { ok: false; reason: 'not-found' | 'conflict' };
-
-// Release
-type ReleaseSignalResult =
-  | { ok: true }
-  | { ok: false; reason: 'not-found' | 'wrong-status' };
-
-// Resolve
-type ResolveSignalResult =
-  | { ok: true }
-  | { ok: false; reason: 'not-found' | 'already-resolved' }
-  | { ok: false; reason: 'signal-failed'; signalKey: string };
-```
-
-`signal-failed` means the database record was updated but `workflow.signal()` threw (network blip, workflow already complete). The `signalKey` is returned so callers can retry signal delivery independently without re-resolving the record.
-
-> **Postgres only.** The `signalQueue.*` methods require the Postgres provider. Workflows using `condition()` without a queue config are unaffected on any provider.
-
 ## Observability
 
 There is no proprietary dashboard. Workflow state lives in Postgres, so use whatever tools you already have:

package/build/modules/utils.d.ts

@@ -13,6 +13,8 @@ export declare function getSystemHealth(): Promise<SystemHealth>;
 export declare function deepCopy<T>(obj: T): T;
 export declare function deterministicRandom(seed: number): number;
 export declare function guid(size?: number): string;
+/** Returns a standard RFC 4122 v4 UUID (e.g. for use as a DB primary key). */
+export declare function uuid(): string;
 export declare function sleepFor(ms: number): Promise<void>;
 export declare function sleepImmediate(): Promise<void>;
 export declare function XSleepFor(ms: number): {

package/build/modules/utils.js

@@ -3,7 +3,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.arrayToHash = exports.isStreamMessage = exports.parseStreamMessage = exports.normalizeRetryPolicy = exports.s = exports.isValidCron = exports.restoreHierarchy = exports.getValueByPath = exports.getIndexedHash = exports.getSymVal = exports.getSymKey = exports.formatISODate = exports.getTimeSeries = exports.getSubscriptionTopic = exports.findSubscriptionForTrigger = exports.findTopKey = exports.matchesStatus = exports.matchesStatusCode = exports.polyfill = exports.identifyProvider = exports.XSleepFor = exports.sleepImmediate = exports.sleepFor = exports.guid = exports.deterministicRandom = exports.deepCopy = exports.getSystemHealth = exports.hashOptions = void 0;
+exports.arrayToHash = exports.isStreamMessage = exports.parseStreamMessage = exports.normalizeRetryPolicy = exports.s = exports.isValidCron = exports.restoreHierarchy = exports.getValueByPath = exports.getIndexedHash = exports.getSymVal = exports.getSymKey = exports.formatISODate = exports.getTimeSeries = exports.getSubscriptionTopic = exports.findSubscriptionForTrigger = exports.findTopKey = exports.matchesStatus = exports.matchesStatusCode = exports.polyfill = exports.identifyProvider = exports.XSleepFor = exports.sleepImmediate = exports.sleepFor = exports.uuid = exports.guid = exports.deterministicRandom = exports.deepCopy = exports.getSystemHealth = exports.hashOptions = void 0;
 const os_1 = __importDefault(require("os"));
 const crypto_1 = require("crypto");
 const cron_parser_1 = require("cron-parser");
@@ -47,6 +47,11 @@ function guid(size = enums_1.HMSH_GUID_SIZE) {
     return `H` + (0, nanoid_1.nanoid)(size);
 }
 exports.guid = guid;
+/** Returns a standard RFC 4122 v4 UUID (e.g. for use as a DB primary key). */
+function uuid() {
+    return (0, crypto_1.randomUUID)();
+}
+exports.uuid = uuid;
 async function sleepFor(ms) {
     return new Promise((resolve) => setTimeout(resolve, ms));
 }
@@ -240,6 +245,9 @@ function getValueByPath(obj, path) {
     const pathParts = path.split('/');
     let currentValue = obj;
     for (const part of pathParts) {
+        if (currentValue == null || typeof currentValue !== 'object') {
+            return undefined;
+        }
         if (currentValue[part] !== undefined) {
             currentValue = currentValue[part];
         }

package/build/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@hotmeshio/hotmesh",
-    "version": "0.21.1",
+    "version": "0.22.1",
     "description": "Durable Workflow",
     "main": "./build/index.js",
     "types": "./build/index.d.ts",
@@ -50,6 +50,7 @@
         "test:durable:signal": "vitest run tests/durable/signal/postgres.test.ts",
         "test:durable:readonly": "docker compose --profile readonly up -d --build && docker compose exec hotmesh-readonly npx vitest run --config tests/durable/readonly/vitest.config.mts",
         "test:durable:unknown": "vitest run tests/durable/unknown/postgres.test.ts",
+        "test:durable:escalations": "HMSH_LOGLEVEL=info vitest run tests/durable/escalations",
         "test:durable:exporter": "HMSH_LOGLEVEL=info vitest run tests/durable/exporter",
         "test:durable:exporter:debug": "EXPORT_DEBUG=1 HMSH_LOGLEVEL=error vitest run tests/durable/basic/postgres.test.ts",
         "test:durable:codec": "vitest run tests/durable/codec/postgres.test.ts",
@@ -84,7 +85,12 @@
         "test:sub:nats": "vitest run tests/functional/sub/providers/nats/nats.test.ts",
         "test:trigger": "vitest run tests/unit/services/activities/trigger.test.ts",
         "test:virtual": "vitest run tests/virtual",
-        "test:unit": "vitest run tests/unit"
+        "test:unit": "vitest run tests/unit",
+        "prove": "docker compose exec hotmesh npx vitest run tests/durable 2>&1 | tee /tmp/hmsh-durable.txt && grep -E 'FAIL|Tests |Files ' /tmp/hmsh-durable.txt | tail -5",
+        "prove:escalations": "docker compose exec hotmesh npx vitest run tests/durable/escalations/postgres.test.ts 2>&1 | tee /tmp/hmsh-escalations.txt && grep -E 'FAIL|Tests |Files ' /tmp/hmsh-escalations.txt | tail -5",
+        "prove:functional": "docker compose exec hotmesh npx vitest run tests/functional 2>&1 | tee /tmp/hmsh-functional.txt && grep -E 'FAIL|Tests |Files ' /tmp/hmsh-functional.txt | tail -5",
+        "prove:all": "docker compose exec hotmesh npx vitest run tests/ 2>&1 | tee /tmp/hmsh-all.txt && grep -E 'FAIL|Tests |Files ' /tmp/hmsh-all.txt | tail -5",
+        "prove:file": "f() { docker compose exec hotmesh npx vitest run \"$@\" 2>&1 | tee /tmp/hmsh-file.txt && grep -E 'FAIL|Tests |Files ' /tmp/hmsh-file.txt | tail -5; }; f"
     },
     "keywords": [
         "Invisible Infrastructure",

package/build/services/activities/hook.d.ts

@@ -6,54 +6,79 @@ import { ProviderTransaction } from '../../types/provider';
 import { StreamCode, StreamStatus } from '../../types/stream';
 import { Activity } from './activity';
 /**
- * A versatile pause/resume activity that supports three distinct patterns:
- * **time hook** (sleep), **web hook** (external signal), and **passthrough**
- * (immediate transition with optional data mapping).
+ * The most flexible activity type in the HotMesh YAML DAG. Depending on
+ * its configuration it operates as one of four distinct flavors:
  *
- * The hook activity is the most flexible activity type. Depending on its
- * YAML configuration, it operates in one of the following modes:
+ * | Flavor | Key field | Behavior |
+ * |---|---|---|
+ * | **Time** | `sleep` | Pauses for a duration, then resumes |
+ * | **Signal** | `hook.topic` | Pauses until an external signal arrives |
+ * | **Cycle** | `cycle: true` | Passthrough that also accepts re-entry from a `cycle` activity |
+ * | **Passthrough** | _(none)_ | Maps data and transitions immediately |
  *
- * ## Time Hook (Sleep)
+ * ---
  *
- * Pauses the flow for a specified duration in seconds. The `sleep` value
- * can be a literal number or a `@pipe` expression for dynamic delays
- * (e.g., exponential backoff).
+ * ## Flavor 1 — Time Hook (sleep)
+ *
+ * Pauses the flow for `sleep` seconds. The value can be a literal number
+ * or a `@pipe` expression (e.g., for exponential backoff).
  *
  * ```yaml
- * app:
- *   id: myapp
- *   version: '1'
- *   graphs:
- *     - subscribes: job.start
- *       expire: 300
+ * activities:
+ *   t1:
+ *     type: trigger
  *
- *       activities:
- *         t1:
- *           type: trigger
+ *   wait_30s:
+ *     type: hook
+ *     sleep: 30                            # pause for 30 seconds
+ *     job:
+ *       maps:
+ *         paused_at: '{$self.output.metadata.ac}'
  *
- *         delay:
- *           type: hook
- *           sleep: 60                        # pause for 60 seconds
- *           job:
- *             maps:
- *               paused_at: '{$self.output.metadata.ac}'
+ *   next_step:
+ *     type: hook
  *
- *         resume:
- *           type: hook
+ * transitions:
+ *   t1:
+ *     - to: wait_30s
+ *   wait_30s:
+ *     - to: next_step
+ * ```
  *
- *       transitions:
- *         t1:
- *           - to: delay
- *         delay:
- *           - to: resume
+ * Dynamic delay with `@pipe` (exponential backoff on retry):
+ *
+ * ```yaml
+ * wait_retry:
+ *   type: hook
+ *   sleep:
+ *     '@pipe':
+ *       - ['{$self.output.data.attempt}', 2]
+ *       - ['{@math.pow}']           # 1, 2, 4, 8, 16 …
  * ```
  *
- * ## Web Hook (External Signal)
+ * ---
  *
- * Registers a webhook listener on a named topic. The flow pauses until
- * an external signal is sent to the hook's topic. The signal data becomes
- * available as `$self.hook.data`. The `hooks` section at the graph level
- * routes incoming signals to the waiting activity.
+ * ## Flavor 2 — Signal Hook (webhook)
+ *
+ * Registers a listener on a named topic. The flow pauses until an
+ * external caller delivers a signal. Signal data is available as
+ * `$self.hook.data`. The graph-level `hooks` section routes incoming
+ * signals to the waiting activity via a `conditions.match` rule.
+ *
+ * **Send the signal** from any process:
+ *
+ * ```typescript
+ * await hotMesh.signal('order.approval', { id: jobId, approved: true });
+ * ```
+ *
+ * **Claim and delete** a pending signal via the collator key:
+ *
+ * ```typescript
+ * // Ack/delete — deliver a signal and clear the hook registration
+ * await hotMesh.signal('order.approval', { id: jobId, approved: false });
+ * ```
+ *
+ * **YAML configuration:**
  *
  * ```yaml
  * app:
@@ -87,20 +112,78 @@ import { Activity } from './activity';
  *           - to: done
  *
  *       hooks:
- *         order.approval:                    # external topic that delivers the signal
+ *         order.approval:                  # topic delivering the signal
  *           - to: wait_for_approval
  *             conditions:
  *               match:
- *                 - expected: '{t1.output.data.id}'
- *                   actual: '{$self.hook.data.id}'
+ *                 - expected: '{t1.output.data.id}'   # job ID
+ *                   actual: '{$self.hook.data.id}'    # signal payload ID
+ * ```
+ *
+ * ### Signal Hook with Escalation
+ *
+ * Adding an `escalation:` block causes the hook activity to write one row
+ * to `public.hmsh_escalations` atomically inside its Leg 1 transaction —
+ * the same database commit that checkpoints job state. The row is
+ * immediately queryable and claimable by any external system.
+ *
+ * All field values support `@pipe` expressions so they can reference job
+ * data computed by earlier activities (e.g., `'{t1.output.data.region}'`).
+ *
+ * ```yaml
+ * wait_for_approval:
+ *   type: hook
+ *   hook:
+ *     type: object
+ *     properties:
+ *       approved: { type: boolean }
+ *   escalation:
+ *     role: manager                            # RBAC role that should act
+ *     type: order-approval
+ *     subtype: regional
+ *     priority: 2                              # lower = higher priority
+ *     description: Approve or reject the order
+ *     entity: '{t1.output.data.entityType}'
+ *     metadata:
+ *       orderId: '{t1.output.data.orderId}'
+ *       region: '{t1.output.data.region}'
+ *     envelope:
+ *       instructions: Review the attached order and approve or reject
+ *     expiresAt: '{t1.output.data.dueDate}'
+ *   job:
+ *     maps:
+ *       approved: '{$self.hook.data.approved}'
+ * ```
+ *
+ * **Claim and resolve the escalation** (resumes the waiting workflow):
+ *
+ * ```typescript
+ * // Find pending approvals for the manager role
+ * const [item] = await client.escalations.list({ role: 'manager', status: 'pending' });
+ *
+ * // Claim it (sets assigned_to + claim_expires_at)
+ * const claim = await client.escalations.claim({
+ *   id: item.id,
+ *   assignee: 'alice@company.com',
+ *   durationMinutes: 30,
+ * });
+ *
+ * // Resolve atomically delivers the signal and resumes the workflow
+ * await client.escalations.resolve({
+ *   id: item.id,
+ *   resolverPayload: { approved: true },
+ * });
  * ```
  *
- * ## Passthrough (No Hook)
+ * ---
+ *
+ * ## Flavor 3 — Cycle Pivot
  *
- * When neither `sleep` nor `hook` is configured, the hook activity acts
- * as a passthrough: it maps data and immediately transitions to children.
- * This is useful for data transformation, convergence points, or as a
- * cycle pivot (with `cycle: true`).
+ * A passthrough hook with `cycle: true` acts as the named re-entry point
+ * for a `cycle` activity. On first entry it behaves identically to a
+ * passthrough (maps data, transitions forward). When a `cycle` activity
+ * downstream names it as its `ancestor`, the engine routes execution back
+ * to it, allowing a controlled loop without spawning a new job.
  *
  * ```yaml
  * app:
@@ -108,6 +191,7 @@ import { Activity } from './activity';
  *   version: '1'
  *   graphs:
  *     - subscribes: job.start
+ *       expire: 120
  *
  *       activities:
  *         t1:
@@ -115,34 +199,69 @@ import { Activity } from './activity';
  *
  *         pivot:
  *           type: hook
- *           cycle: true                      # enables re-entry from a cycle activity
+ *           cycle: true                    # marks this as a loop re-entry point
+ *
+ *         do_work:
+ *           type: worker
+ *           topic: work.process
  *           output:
- *             maps:
- *               retryCount: 0
+ *             schema:
+ *               type: object
+ *               properties:
+ *                 counter: { type: number }
  *           job:
  *             maps:
- *               counter: '{$self.output.data.retryCount}'
+ *               counter: '{$self.output.data.counter}'
  *
- *         do_work:
- *           type: worker
- *           topic: work.do
+ *         loop_back:
+ *           type: cycle
+ *           ancestor: pivot               # jumps back to `pivot` when condition holds
  *
  *       transitions:
  *         t1:
  *           - to: pivot
  *         pivot:
  *           - to: do_work
+ *         do_work:
+ *           - to: loop_back
+ *             conditions:
+ *               match:
+ *                 - expected: true
+ *                   actual:
+ *                     '@pipe':
+ *                       - ['{do_work.output.data.counter}', 5]
+ *                       - ['{@conditional.less_than}']
  * ```
  *
+ * ---
+ *
+ * ## Flavor 4 — Passthrough
+ *
+ * When none of `sleep`, `hook`, or `cycle` is set, the hook activity
+ * immediately maps data and transitions to its children. Useful as a
+ * data transformation node or fan-in convergence point.
+ *
+ * ```yaml
+ * merge:
+ *   type: hook
+ *   output:
+ *     maps:
+ *       total: '{a1.output.data.subtotal}'   # copy field into activity output
+ *   job:
+ *     maps:
+ *       total: '{$self.output.data.total}'   # promote to job-level data
+ * ```
+ *
+ * ---
+ *
  * ## Execution Model
  *
- * - **With `sleep` or `hook`**: Category A (duplex). Leg 1 registers the
- *   hook and saves state. Leg 2 fires when the timer expires or the
- *   external signal arrives (via `processTimeHookEvent` or
- *   `processWebHookEvent`).
- * - **Without `sleep` or `hook`**: Category B (passthrough). Uses the
- *   crash-safe `executeLeg1StepProtocol` to map data and transition
- *   to adjacent activities.
+ * - **Time and Signal flavors** — Category A (duplex). Leg 1 registers the
+ *   hook (timer or webhook), saves state, and commits. Leg 2 fires when the
+ *   timer fires or the external signal arrives.
+ * - **Cycle and Passthrough flavors** — Category B. Uses the crash-safe
+ *   `executeLeg1StepProtocol` (GUID ledger backed) to map data and
+ *   immediately transition to adjacent activities.
  *
  * @see {@link HookActivity} for the TypeScript interface
  */
@@ -152,6 +271,7 @@ declare class Hook extends Activity {
     isConfiguredAsHook(): boolean;
     doesHook(): boolean;
     doHook(telemetry: TelemetryService): Promise<void>;
+    private addEscalationToTransaction;
     /**
      * Re-publishes a pending signal as a WEBHOOK stream message so the
      * normal leg2 dispatch path processes it. Called when leg1's