@hotmeshio/hotmesh 0.0.31 → 0.0.33

package/README.md

@@ -1,13 +1,9 @@
 # HotMesh
 ![alpha release](https://img.shields.io/badge/release-alpha-yellow)
 
-Elevate Redis from an in-memory data cache, and turn your unpredictable functions into unbreakable workflows.
+Elevate Redis from an in-memory data cache, and turn your unpredictable functions into unbreakable workflows. HotMesh is a distributed orchestration engine that governs the execution of your functions, ensuring they always complete successfully.
 
-**HotMesh** is a wrapper for Redis that exposes concepts like ‘activities’, ‘workflows’, and 'jobs'. Behind the scenes, it uses *Redis Data* (Hash, ZSet, List); *Redis Streams* (XReadGroup, XAdd, XLen, etc); and *Redis Publish/Subscribe*.
-
-It's still Redis in the background, but your functions are run as *reentrant processes* and are executed in a distributed environment, with all the benefits of a distributed system, including fault tolerance, scalability, and high availability.
-
-Write functions in your own preferred style, and let Redis govern their execution with its unmatched performance.
+*Write functions in your own preferred style, and let Redis govern their execution.*
 
 ## Install
 [![npm version](https://badge.fury.io/js/%40hotmeshio%2Fhotmesh.svg)](https://badge.fury.io/js/%40hotmeshio%2Fhotmesh)
@@ -15,102 +11,157 @@ Write functions in your own preferred style, and let Redis govern their executio
 ```sh
 npm install @hotmeshio/hotmesh
 ```
-
 ## Design
-1. Start with any ordinary class. Pay attention to unpredictable functions: those that execute slowly, cause problems at scale, or simply fail to return now and then. *Note how the `flaky` function throws an error 50% of the time. This is exactly the type of function that can be fixed using HotMesh.*
-    ```javascript
-    //myworkflow.ts
-
-    export class MyWorkflow {
 
-      async run(name: string): Promise<string> {
-        const hi = await this.flaky(name);
-        const hello = await this.greet(name);
-        return `${hi} ${hello}`;
-      }
-
-      //this function is unpredictable and will fail 50% of the time
-      async flaky(name: string): Promise<string> {
-        if (Math.random() < 0.5) {
-          throw new Error('Ooops!');
-        }
-        return `Hi, ${name}!`;
-      }
+1. Start by defining **activities**. Activities can be written in any style, using any framework, and can even be legacy functions you've already written. The only requirement is that they return a Promise. *Note how the `saludar` example throws an error 50% of the time. It doesn't matter how unpredictable your functions are, HotMesh will retry as necessary until they succeed.*
+    ```javascript
+    //activities.ts
+    export async function greet(name: string): Promise<string> {
+      return `Hello, ${name}!`;
+    }
 
-      async greet(name: string): Promise<string> {
-        return `Hello, ${name}!`;
+    export async function saludar(nombre: string): Promise<string> {
+      if (Math.random() > 0.5) throw new Error('Random error');
+      return `¡Hola, ${nombre}!`;
+    }
+    ```
+2. Define your **workflow** logic. Include conditional branching, loops, etc to control activity execution. It's vanilla code written in your own coding style. The only requirement is to use `proxyActivities`, ensuring your activities are executed with HotMesh's durability guarantee.
+    ```javascript
+    //workflows.ts
+    import { Durable } from '@hotmeshio/hotmesh';
+    import * as activities from './activities';
+
+    const { greet, saludar } = Durable.workflow
+      .proxyActivities<typeof activities>({
+        activities
+      });
+
+    export async function example(name: string, lang: string): Promise<string> {
+      if (lang === 'es') {
+        return await saludar(name);
+      } else {
+        return await greet(name);
       }
     }
     ```
-2. Import and configure `Redis` and `MeshOS` as shown. List those functions that Redis should govern as durable workflows (like `run` and `flaky`). And that's it! *Your functions don't actually change; rather, their governance does.*
+3. Instance a HotMesh **client** to invoke the workflow.
     ```javascript
-    //myworkflow.ts
-
+    //client.ts
+    import { Durable } from '@hotmeshio/hotmesh';
     import Redis from 'ioredis'; //OR `import * as Redis from 'redis';`
-    import { MeshOS } from '@hotmeshio/hotmesh';
-
-    export class MyWorkflow extends MeshOS {
-
-      //configure Redis
-      redisClass = Redis;
-      redisOptions = { host: 'localhost', port: 6379 };
-
-      //list functions to run as durable workflows
-      workflowFunctions = ['run'];
-
-      //list functions to retry and cache
-      proxyFunctions = ['flaky'];
-
-      //no need to change anything else!
-
-      async run(name: string): Promise<string> {
-        const hi = await this.flaky(name);
-        const hello = await this.greet(name);
-        return `${hi} ${hello}`;
-      }
+    import { nanoid } from 'nanoid';
 
-      async flaky(name: string): Promise<string> {
-        if (Math.random() < 0.5) {
-          throw new Error('Ooops!');
+    async function run(): Promise<string> {
+      const client = new Durable.Client({
+        connection: {
+          class: Redis,
+          options: { host: 'localhost', port: 6379 }
         }
-        return `Hi, ${name}!`;
-      }
+      });
 
-      async greet(name: string): Promise<string> {
-        return `Hello, ${name}!`;
-      }
+      const handle = await client.workflow.start({
+        args: ['HotMesh', 'es'],
+        taskQueue: 'hello-world',
+        workflowName: 'example',
+        workflowId: nanoid()
+      });
+
+      return await handle.result();
+      //returns '¡Hola, HotMesh!'
     }
     ```
-3. Invoke your class, providing a unique id (it's now an idempotent workflow and needs a GUID). Nothing changes from the outside, *but Redis now governs the end-to-end execution.* It's guaranteed to succeed, even if it breaks a few times along the way. 
+4. Finally, create a **worker** and link your workflow function. Workers listen for tasks on their assigned Redis stream and invoke your workflow function each time they receive an event.
     ```javascript
-    //mycaller.ts
-
-    const workflow = new MyWorkflow({ id: 'my123', await: true });
-    const response = await workflow.run('World');
-    //Hi, World! Hello, World!
+    //worker.ts
+    import { Durable } from '@hotmeshio/hotmesh';
+    import Redis from 'ioredis';
+    import * as workflows from './workflows';
+
+    async function run() {
+      const worker = await Durable.Worker.create({
+        connection: {
+          class: Redis,
+          options: { host: 'localhost', port: 6379 },
+        },
+        taskQueue: 'hello-world',
+        workflow: workflows.example,
+      });
+
+      await worker.run();
+    }
     ```
 
-Redis governance delivers more than just reliability. Externalizing state fundamentally changes the execution profile for your functions, allowing you to design long-running, durable workflows. The `MeshOS` base class (shown in the examples above) provides additional methods for solving the most common state management challenges.
-
- - `waitForSignal` | Pause your function and wait for external event(s) before continuing. The *waitForSignal* method will collate and cache the signals and only awaken your function once all signals have arrived.
- - `signal` | Send a signal (and optional payload) to any paused function.
- - `hook` | Redis governance converts your functions into 're-entrant processes'. Optionally use the *hook* method to spawn parallel execution threads to augment a running workflow.
- - `sleep` | Pause function execution for a ridiculous amount of time (months, years, etc). There's no risk of information loss, as Redis governs function state. When your function awakens, function state is efficiently (and automatically) restored and your function will resume right where it left off.
- - `random` | Generate a deterministic random number that can be used in a reentrant process workflow (replaces `Math.random()`).
- - `executeChild` | Call another durable function and await the response. *Design sophisticated, multi-process solutions by leveraging this command.*
- - `startChild` | Call another durable function, but do not await the response.
- - `search` | Instance a search session (e.g, `const search = MeshOS.search()`)
-    - `set` | Set one or more name/value pairs (e.g, `search.set('name1', 'value1', 'name2', 'value2')`)
-    - `get` | Get a single value by name(e.g, `search.get('name')`)
-    - `mget` | Get multiple values by name (e.g, `search.mget('name1', 'name2')`)
-    - `del` | Delete one or more entries by name and return the number deleted (e.g, `search.del('name1', 'name2')`)
-    - `incr` | Increment (or decrement) a number (e.g, `search.incr('name', -99)`)
-    - `mult` | Multiply a number (e.g, `search.mult('name', 12)`)
- - `find` | Find workflows using the native Redis [FT.*](https://redis.io/commands/ft.search/) search commands
- - `findWhere` | Find workflows using a simplified, JSON-based search syntax that overlays the native Redis FT.SEARCH syntax.
- - `createIndex` | Create a searchable index in Redis using simplified, JSON-based syntax that overlays the native Redis FT.CREATE syntax.
- - `startWorkers` | Start the workers necessary to govern your class (typically called at server startup).
- - `stopWorkers` | Stop all workers (typically called at server shutdown)
+### Workflow Extensions
+Redis governance delivers more than just reliability. Externalizing state fundamentally changes the execution profile for your functions, allowing you to design long-running, durable workflows. The `Durable.workflow` base class (shown in the examples above) provides additional methods for solving the most common state management challenges.
+
+ - `waitForSignal` Pause your function and wait for external event(s) before continuing. The *waitForSignal* method will collate and cache the signals and only awaken your function once all signals have arrived.
+   ```javascript
+    const signals = [a, b] = await Durable.workflow.waitForSignal('sig1', 'sig2')` 
+    ```
+ - `signal` Send a signal (and optional payload) to a paused function awaiting the signal.
+    ```javascript
+      await Durable.workflow.signal('sig1', {payload: 'hi!'});
+    ```
+ - `hook` Redis governance converts your functions into 're-entrant processes'. Optionally use the *hook* method to spawn parallel execution threads to augment a running workflow.
+    ```javascript
+    await Durable.workflow.hook({
+      workflowName: 'newsletter',
+      taskQueue: 'default',
+      args: []
+    });
+    ```
+ - `sleepFor` Pause function execution for a ridiculous amount of time (months, years, etc). There's no risk of information loss, as Redis governs function state. When your function awakens, function state is efficiently (and automatically) restored and your function will resume right where it left off.
+    ```javascript
+    await Durable.workflow.sleepFor('1 month');
+    ```
+ - `random` Generate a deterministic random number that can be used in a reentrant process workflow (replaces `Math.random()`).
+    ```javascript
+    const random = await Durable.workflow.random();
+    ```
+ - `executeChild` Call another durable function and await the response. *Design sophisticated, multi-process solutions by leveraging this command.*
+    ```javascript
+    const jobResponse = await Durable.workflow.executeChild({
+      workflowName: 'newsletter',
+      taskQueue: 'default',
+      args: [{ id, user_id, etc }],
+    });
+    ```
+ - `startChild` Call another durable function, but do not await the response.
+    ```javascript
+    const jobId = await Durable.workflow.startChild({
+      workflowName: 'newsletter',
+      taskQueue: 'default',
+      args: [{ id, user_id, etc }],
+    });
+    ```
+ - `search` Instance a search session
+    ```javascript
+    const search = await Durable.workflow.search();
+    ```
+    - `set` Set one or more name/value pairs
+      ```javascript
+      await search.set('name1', 'value1', 'name2', 'value2');
+      ```
+    - `get` Get a single value by name
+      ```javascript
+      const value = await search.get('name');
+      ```
+    - `mget` Get multiple values by name
+      ```javascript
+      const [val1, val2] = await search.mget('name1', 'name2');
+      ```
+    - `del` Delete one or more entries by name and return the number deleted
+      ```javascript
+      const count = await search.del('name1', 'name2');
+      ```
+    - `incr` Increment (or decrement) a number
+      ```javascript
+      const value = await search.incr('name', 12);
+      ```
+    - `mult` Multiply a number
+      ```javascript
+      const value = await search.mult('name', 12);
+      ```
 
 Refer to the [hotmeshio/samples-typescript](https://github.com/hotmeshio/samples-typescript) repo for usage examples. 
 

package/build/modules/errors.d.ts

@@ -28,6 +28,13 @@ declare class DurableSleepError extends Error {
     dimension: string;
     constructor(message: string, duration: number, index: number, dimension: string);
 }
+declare class DurableSleepForError extends Error {
+    code: number;
+    duration: number;
+    index: number;
+    dimension: string;
+    constructor(message: string, duration: number, index: number, dimension: string);
+}
 declare class DurableTimeoutError extends Error {
     code: number;
     constructor(message: string);
@@ -63,4 +70,4 @@ declare class CollationError extends Error {
     fault: CollationFaultType;
     constructor(status: number, leg: ActivityDuplex, stage: CollationStage, fault?: CollationFaultType);
 }
-export { CollationError, DurableTimeoutError, DurableMaxedError, DurableFatalError, DurableRetryError, DurableWaitForSignalError, DurableIncompleteSignalError, DurableSleepError, DuplicateJobError, GetStateError, SetStateError, MapDataError, RegisterTimeoutError, ExecActivityError };
+export { CollationError, DurableTimeoutError, DurableMaxedError, DurableFatalError, DurableRetryError, DurableWaitForSignalError, DurableIncompleteSignalError, DurableSleepError, DurableSleepForError, DuplicateJobError, GetStateError, SetStateError, MapDataError, RegisterTimeoutError, ExecActivityError };

package/build/modules/errors.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.ExecActivityError = exports.RegisterTimeoutError = exports.MapDataError = exports.SetStateError = exports.GetStateError = exports.DuplicateJobError = exports.DurableSleepError = exports.DurableIncompleteSignalError = exports.DurableWaitForSignalError = exports.DurableRetryError = exports.DurableFatalError = exports.DurableMaxedError = exports.DurableTimeoutError = exports.CollationError = void 0;
+exports.ExecActivityError = exports.RegisterTimeoutError = exports.MapDataError = exports.SetStateError = exports.GetStateError = exports.DuplicateJobError = exports.DurableSleepForError = exports.DurableSleepError = exports.DurableIncompleteSignalError = exports.DurableWaitForSignalError = exports.DurableRetryError = exports.DurableFatalError = exports.DurableMaxedError = exports.DurableTimeoutError = exports.CollationError = void 0;
 class GetStateError extends Error {
     constructor() {
         super("Error occurred while getting job state");
@@ -31,6 +31,7 @@ class DurableWaitForSignalError extends Error {
     }
 }
 exports.DurableWaitForSignalError = DurableWaitForSignalError;
+/* @deprecated */
 class DurableSleepError extends Error {
     constructor(message, duration, index, dimension) {
         super(message);
@@ -41,6 +42,16 @@ class DurableSleepError extends Error {
     }
 }
 exports.DurableSleepError = DurableSleepError;
+class DurableSleepForError extends Error {
+    constructor(message, duration, index, dimension) {
+        super(message);
+        this.duration = duration;
+        this.index = index;
+        this.dimension = dimension;
+        this.code = 592;
+    }
+}
+exports.DurableSleepForError = DurableSleepForError;
 class DurableTimeoutError extends Error {
     constructor(message) {
         super(message);

package/build/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@hotmeshio/hotmesh",
-    "version": "0.0.31",
+    "version": "0.0.33",
     "description": "Unbreakable Workflows",
     "main": "build/index.js",
     "types": "build/index.d.ts",

package/build/services/durable/client.js

@@ -76,8 +76,8 @@ class ClientService {
         };
         this.workflow = {
             start: async (options) => {
-                const taskQueueName = options.taskQueue;
-                const workflowName = options.workflowName;
+                const taskQueueName = options.entity ?? options.taskQueue;
+                const workflowName = options.entity ?? options.workflowName;
                 const trc = options.workflowTrace;
                 const spn = options.workflowSpan;
                 //topic is concat of taskQueue and workflowName
@@ -93,13 +93,12 @@ class ClientService {
                 };
                 const context = { metadata: { trc, spn }, data: {} };
                 const jobId = await hotMeshClient.pub(`${options.namespace ?? factory_1.APP_ID}.execute`, payload, context);
-                //seed search data if present
+                // Seed search data if present
                 if (jobId && options.search?.data) {
                     const searchSessionId = `-search-0`;
                     const search = new search_1.Search(jobId, hotMeshClient, searchSessionId);
-                    for (const [key, value] of Object.entries(options.search.data)) {
-                        search.set(key, value);
-                    }
+                    const entries = Object.entries(options.search.data).flat();
+                    search.set(...entries);
                 }
                 return new handle_1.WorkflowHandleService(hotMeshClient, workflowTopic, jobId);
             },

package/build/services/durable/factory.d.ts

@@ -6,7 +6,8 @@
  *
  * ERROR CODES:
  *      594: waitforsignal
- *      595: sleep
+ *      592: sleepFor
+ *      595: sleep (deprecated)
  *      596, 597, 598: fatal
  *      599: retry
  */

package/build/services/durable/factory.js

@@ -9,7 +9,8 @@ exports.DEFAULT_COEFFICIENT = exports.APP_ID = exports.APP_VERSION = exports.get
  *
  * ERROR CODES:
  *      594: waitforsignal
- *      595: sleep
+ *      592: sleepFor
+ *      595: sleep (deprecated)
  *      596, 597, 598: fatal
  *      599: retry
  */
@@ -129,6 +130,20 @@ const getWorkflowYAML = (app, version) => {
               maps:
                 duration: '{$self.output.data.duration}'
                 index: '{$self.output.data.index}'
+            592:
+              schema:
+                type: object
+                properties:
+                  duration:
+                    type: number
+                    description: sleepFor duration in seconds
+                  index:
+                    type: number
+                    description: the current index
+              maps:
+                duration: '{$self.output.data.duration}'
+                index: '{$self.output.data.index}'
+  
           job:
             maps:
               response: '{$self.output.data.response}'
@@ -231,7 +246,20 @@ const getWorkflowYAML = (app, version) => {
               maps:
                 duration: '{$self.output.data.duration}'
                 index: '{$self.output.data.index}'
-
+            592:
+              schema:
+                type: object
+                properties:
+                  duration:
+                    type: number
+                    description: sleepFor duration in seconds
+                  index:
+                    type: number
+                    description: the current index
+              maps:
+                duration: '{$self.output.data.duration}'
+                index: '{$self.output.data.index}'
+  
         siga594:
           title: Signal In - Wait for signals
           type: await
@@ -334,6 +362,19 @@ const getWorkflowYAML = (app, version) => {
             maps:
               duration: '{siga1.output.data.duration}'
 
+        siga592:
+          title: Signal In - Sleep For a duration and then cycle/goto
+          type: hook
+          sleep: '{sigw1.output.data.duration}'
+
+        sigc592:
+          title: Signal In - Goto Activity a1 after sleeping for a duration
+          type: cycle
+          ancestor: siga1
+          input:
+            maps:
+              duration: '{siga1.output.data.duration}'
+    
         siga599:
           title: Signal In - Sleep exponentially longer and retry
           type: hook
@@ -452,6 +493,19 @@ const getWorkflowYAML = (app, version) => {
             maps:
               duration: '{a1.output.data.duration}'
 
+        a592:
+          title: Sleep For a duration and then cycle/goto
+          type: hook
+          sleep: '{w1.output.data.duration}'
+
+        c592:
+          title: Goto Activity a1 after sleeping for a duration
+          type: cycle
+          ancestor: a1
+          input:
+            maps:
+              duration: '{a1.output.data.duration}'
+
         a599:
           title: Sleep exponentially longer before retrying
           type: hook
@@ -655,6 +709,9 @@ const getWorkflowYAML = (app, version) => {
           - to: siga595
             conditions:
               code: 595
+          - to: siga592
+            conditions:
+              code: 592
           - to: siga599
             conditions:
               code: 599
@@ -666,6 +723,8 @@ const getWorkflowYAML = (app, version) => {
           - to: sigc595
             conditions:
               code: 202
+        siga592:
+          - to: sigc592
         siga599:
           - to: sigc599
         a1:
@@ -677,6 +736,9 @@ const getWorkflowYAML = (app, version) => {
           - to: a595
             conditions:
               code: 595
+          - to: a592
+            conditions:
+              code: 592
           - to: a599
             conditions:
               code: 599
@@ -703,6 +765,8 @@ const getWorkflowYAML = (app, version) => {
           - to: c595
             conditions:
               code: 202
+        a592:
+          - to: c592
         a599:
           - to: c599
 

package/build/services/durable/worker.js

@@ -178,7 +178,22 @@ class WorkerService {
             }
             catch (err) {
                 //not an error...just a trigger to sleep
-                if (err instanceof errors_1.DurableSleepError) {
+                if (err instanceof errors_1.DurableSleepForError) {
+                    return {
+                        status: stream_1.StreamStatus.SUCCESS,
+                        code: err.code,
+                        metadata: { ...data.metadata },
+                        data: {
+                            code: err.code,
+                            message: JSON.stringify({ duration: err.duration, index: err.index, dimension: err.dimension }),
+                            duration: err.duration,
+                            index: err.index,
+                            dimension: err.dimension
+                        }
+                    };
+                    //deprecated format; not an error...just a trigger to sleep
+                }
+                else if (err instanceof errors_1.DurableSleepError) {
                     return {
                         status: stream_1.StreamStatus.SUCCESS,
                         code: err.code,

package/build/services/durable/workflow.d.ts

@@ -81,10 +81,21 @@ export declare class WorkflowService {
      */
     static hook(options: HookOptions): Promise<string>;
     /**
-     * Sleeps for a duration.
+     * Sleeps the workflow for a duration. As the function is reentrant,
+     * upon reentry, the function will traverse prior execution paths up
+     * until the sleep command and then resume execution from that point.
      * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
      * @returns {Promise<number>}
      */
+    static sleepFor(duration: string): Promise<number>;
+    /**
+     * Sleeps the workflow for a duration. As the function is reentrant,
+     * upon reentry, the function will traverse prior execution paths up
+     * until the sleep command and then resume execution from that point.
+     * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
+     * @returns {Promise<number>}
+     * @deprecated - use `sleepFor` instead
+     */
     static sleep(duration: string): Promise<number>;
     /**
      * Waits for a signal to awaken

package/build/services/durable/workflow.js

@@ -262,10 +262,37 @@ class WorkflowService {
         }
     }
     /**
-     * Sleeps for a duration.
+     * Sleeps the workflow for a duration. As the function is reentrant,
+     * upon reentry, the function will traverse prior execution paths up
+     * until the sleep command and then resume execution from that point.
      * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
      * @returns {Promise<number>}
      */
+    static async sleepFor(duration) {
+        const seconds = (0, ms_1.default)(duration) / 1000;
+        const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
+        const namespace = store.get('namespace');
+        const workflowTopic = store.get('workflowTopic');
+        const hotMeshClient = await worker_1.WorkerService.getHotMesh(workflowTopic, { namespace });
+        if (await WorkflowService.isSideEffectAllowed(hotMeshClient, 'sleep')) {
+            const workflowId = store.get('workflowId');
+            const workflowDimension = store.get('workflowDimension') ?? '';
+            const COUNTER = store.get('counter');
+            const execIndex = COUNTER.counter;
+            // spawn a new sleep job if error code 592 is thrown by the worker
+            // NOTE: If this message appears in the stack trace, the `.sleepFor()` method in your workflow code was NOT awaited.
+            throw new errors_1.DurableSleepForError(workflowId, seconds, execIndex, workflowDimension);
+        }
+        return seconds;
+    }
+    /**
+     * Sleeps the workflow for a duration. As the function is reentrant,
+     * upon reentry, the function will traverse prior execution paths up
+     * until the sleep command and then resume execution from that point.
+     * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
+     * @returns {Promise<number>}
+     * @deprecated - use `sleepFor` instead
+     */
     static async sleep(duration) {
         const seconds = (0, ms_1.default)(duration) / 1000;
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -284,7 +311,6 @@ class WorkflowService {
         }
         catch (e) {
             // spawn a new sleep job if error code 595 is thrown by the worker)
-            // NOTE: If this message shows up in your stack trace, you forgot to await `Durable.workflow.sleep()` in your workflow code.
             throw new errors_1.DurableSleepError(workflowId, seconds, execIndex, workflowDimension);
         }
     }

package/build/services/signaler/stream.js

@@ -225,7 +225,7 @@ class StreamSignaler {
         this.shouldConsume = false;
         this.logger.info(`stream-consumer-stopping`, this.topic ? { topic: this.topic } : undefined);
         this.cancelThrottle();
-        await (0, utils_1.sleepFor)(BLOCK_TIME_MS);
+        //await sleepFor(BLOCK_TIME_MS);
     }
     cancelThrottle() {
         if (this.currentTimerId !== undefined) {

package/build/types/durable.d.ts

@@ -46,7 +46,7 @@ type WorkflowSearchOptions = {
 };
 type WorkflowOptions = {
     namespace?: string;
-    taskQueue: string;
+    taskQueue?: string;
     args: any[];
     workflowId?: string;
     entity?: string;

package/modules/errors.ts

@@ -33,6 +33,7 @@ class DurableWaitForSignalError extends Error {
   }
 }
 
+/* @deprecated */
 class DurableSleepError extends Error {
   code: number;
   duration: number; //seconds
@@ -46,6 +47,19 @@ class DurableSleepError extends Error {
     this.code = 595;
   }
 }
+class DurableSleepForError extends Error {
+  code: number;
+  duration: number; //seconds
+  index: number;    //execution order in the workflow
+  dimension: string; //hook dimension (e.g., ',0,1,0') (uses empty string for `null`)
+  constructor(message: string, duration: number, index: number, dimension: string) {
+    super(message);
+    this.duration = duration;
+    this.index = index;
+    this.dimension = dimension;
+    this.code = 592;
+  }
+}
 class DurableTimeoutError extends Error {
   code: number;
   constructor(message: string) {
@@ -124,6 +138,7 @@ export {
   DurableWaitForSignalError,
   DurableIncompleteSignalError,
   DurableSleepError,
+  DurableSleepForError,
   DuplicateJobError,
   GetStateError,
   SetStateError,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hotmeshio/hotmesh",
-  "version": "0.0.31",
+  "version": "0.0.33",
   "description": "Unbreakable Workflows",
   "main": "build/index.js",
   "types": "build/index.d.ts",

package/services/durable/client.ts

@@ -92,8 +92,8 @@ export class ClientService {
 
   workflow = {
     start: async (options: WorkflowOptions): Promise<WorkflowHandleService> => {
-      const taskQueueName = options.taskQueue;
-      const workflowName = options.workflowName;
+      const taskQueueName = options.entity ?? options.taskQueue;
+      const workflowName = options.entity ?? options.workflowName;
       const trc = options.workflowTrace;
       const spn = options.workflowSpan;
       //topic is concat of taskQueue and workflowName
@@ -113,13 +113,12 @@ export class ClientService {
         payload,
         context as JobState
       );
-      //seed search data if present
+      // Seed search data if present
       if (jobId && options.search?.data) {
         const searchSessionId = `-search-0`;
         const search = new Search(jobId, hotMeshClient, searchSessionId);
-        for (const [key, value] of Object.entries(options.search.data)) {
-          search.set(key, value);
-        }
+        const entries = Object.entries(options.search.data).flat();
+        search.set(...entries);
       }
       return new WorkflowHandleService(hotMeshClient, workflowTopic, jobId);
     },

package/services/durable/factory.ts

@@ -6,7 +6,8 @@
  *
  * ERROR CODES:
  *      594: waitforsignal
- *      595: sleep
+ *      592: sleepFor
+ *      595: sleep (deprecated)
  *      596, 597, 598: fatal
  *      599: retry
  */
@@ -126,6 +127,20 @@ const getWorkflowYAML = (app: string, version: string) => {
               maps:
                 duration: '{$self.output.data.duration}'
                 index: '{$self.output.data.index}'
+            592:
+              schema:
+                type: object
+                properties:
+                  duration:
+                    type: number
+                    description: sleepFor duration in seconds
+                  index:
+                    type: number
+                    description: the current index
+              maps:
+                duration: '{$self.output.data.duration}'
+                index: '{$self.output.data.index}'
+  
           job:
             maps:
               response: '{$self.output.data.response}'
@@ -228,7 +243,20 @@ const getWorkflowYAML = (app: string, version: string) => {
               maps:
                 duration: '{$self.output.data.duration}'
                 index: '{$self.output.data.index}'
-
+            592:
+              schema:
+                type: object
+                properties:
+                  duration:
+                    type: number
+                    description: sleepFor duration in seconds
+                  index:
+                    type: number
+                    description: the current index
+              maps:
+                duration: '{$self.output.data.duration}'
+                index: '{$self.output.data.index}'
+  
         siga594:
           title: Signal In - Wait for signals
           type: await
@@ -331,6 +359,19 @@ const getWorkflowYAML = (app: string, version: string) => {
             maps:
               duration: '{siga1.output.data.duration}'
 
+        siga592:
+          title: Signal In - Sleep For a duration and then cycle/goto
+          type: hook
+          sleep: '{sigw1.output.data.duration}'
+
+        sigc592:
+          title: Signal In - Goto Activity a1 after sleeping for a duration
+          type: cycle
+          ancestor: siga1
+          input:
+            maps:
+              duration: '{siga1.output.data.duration}'
+    
         siga599:
           title: Signal In - Sleep exponentially longer and retry
           type: hook
@@ -449,6 +490,19 @@ const getWorkflowYAML = (app: string, version: string) => {
             maps:
               duration: '{a1.output.data.duration}'
 
+        a592:
+          title: Sleep For a duration and then cycle/goto
+          type: hook
+          sleep: '{w1.output.data.duration}'
+
+        c592:
+          title: Goto Activity a1 after sleeping for a duration
+          type: cycle
+          ancestor: a1
+          input:
+            maps:
+              duration: '{a1.output.data.duration}'
+
         a599:
           title: Sleep exponentially longer before retrying
           type: hook
@@ -652,6 +706,9 @@ const getWorkflowYAML = (app: string, version: string) => {
           - to: siga595
             conditions:
               code: 595
+          - to: siga592
+            conditions:
+              code: 592
           - to: siga599
             conditions:
               code: 599
@@ -663,6 +720,8 @@ const getWorkflowYAML = (app: string, version: string) => {
           - to: sigc595
             conditions:
               code: 202
+        siga592:
+          - to: sigc592
         siga599:
           - to: sigc599
         a1:
@@ -674,6 +733,9 @@ const getWorkflowYAML = (app: string, version: string) => {
           - to: a595
             conditions:
               code: 595
+          - to: a592
+            conditions:
+              code: 592
           - to: a599
             conditions:
               code: 599
@@ -700,6 +762,8 @@ const getWorkflowYAML = (app: string, version: string) => {
           - to: c595
             conditions:
               code: 202
+        a592:
+          - to: c592
         a599:
           - to: c599
 

package/services/durable/worker.ts

@@ -4,6 +4,7 @@ import {
   DurableMaxedError,
   DurableRetryError,
   DurableSleepError,
+  DurableSleepForError,
   DurableTimeoutError, 
   DurableWaitForSignalError} from '../../modules/errors';
 import { asyncLocalStorage } from './asyncLocalStorage';
@@ -226,7 +227,22 @@ export class WorkerService {
       } catch (err) {
 
         //not an error...just a trigger to sleep
-        if (err instanceof DurableSleepError) {
+        if (err instanceof DurableSleepForError) {
+          return {
+            status: StreamStatus.SUCCESS,
+            code: err.code,
+            metadata: { ...data.metadata },
+            data: {
+              code: err.code,
+              message: JSON.stringify({ duration: err.duration, index: err.index, dimension: err.dimension }),
+              duration: err.duration,
+              index: err.index,
+              dimension: err.dimension
+            }
+          } as StreamDataResponse;
+
+        //deprecated format; not an error...just a trigger to sleep
+        } else if (err instanceof DurableSleepError) {
           return {
             status: StreamStatus.SUCCESS,
             code: err.code,

package/services/durable/workflow.ts

@@ -3,6 +3,7 @@ import ms from 'ms';
 import {
   DurableIncompleteSignalError,
   DurableSleepError,
+  DurableSleepForError,
   DurableWaitForSignalError } from '../../modules/errors';
 import { KeyService, KeyType } from '../../modules/key';
 import { asyncLocalStorage } from './asyncLocalStorage';
@@ -289,10 +290,38 @@ export class WorkflowService {
   }
 
   /**
-   * Sleeps for a duration.
+   * Sleeps the workflow for a duration. As the function is reentrant, 
+   * upon reentry, the function will traverse prior execution paths up
+   * until the sleep command and then resume execution from that point.
    * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
    * @returns {Promise<number>}
    */
+  static async sleepFor(duration: string): Promise<number> {
+    const seconds = ms(duration) / 1000;
+    const store = asyncLocalStorage.getStore();
+    const namespace = store.get('namespace');
+    const workflowTopic = store.get('workflowTopic');
+    const hotMeshClient = await WorkerService.getHotMesh(workflowTopic, { namespace });
+    if (await WorkflowService.isSideEffectAllowed(hotMeshClient, 'sleep')) {
+      const workflowId = store.get('workflowId');
+      const workflowDimension = store.get('workflowDimension') ?? '';
+      const COUNTER = store.get('counter');
+      const execIndex = COUNTER.counter;
+      // spawn a new sleep job if error code 592 is thrown by the worker
+      // NOTE: If this message appears in the stack trace, the `.sleepFor()` method in your workflow code was NOT awaited.
+      throw new DurableSleepForError(workflowId, seconds, execIndex, workflowDimension);
+    }
+    return seconds;
+  }
+
+  /**
+   * Sleeps the workflow for a duration. As the function is reentrant, 
+   * upon reentry, the function will traverse prior execution paths up
+   * until the sleep command and then resume execution from that point.
+   * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
+   * @returns {Promise<number>}
+   * @deprecated - use `sleepFor` instead
+   */
   static async sleep(duration: string): Promise<number> {
     const seconds = ms(duration) / 1000;
 
@@ -312,7 +341,6 @@ export class WorkflowService {
       return seconds;
     } catch (e) {
       // spawn a new sleep job if error code 595 is thrown by the worker)
-      // NOTE: If this message shows up in your stack trace, you forgot to await `Durable.workflow.sleep()` in your workflow code.
       throw new DurableSleepError(workflowId, seconds, execIndex, workflowDimension);
     }
   }

package/services/signaler/stream.ts

@@ -260,7 +260,7 @@ class StreamSignaler {
     this.shouldConsume = false;
     this.logger.info(`stream-consumer-stopping`, this.topic ? { topic: this.topic } : undefined);
     this.cancelThrottle();
-    await sleepFor(BLOCK_TIME_MS);
+    //await sleepFor(BLOCK_TIME_MS);
   }
 
   cancelThrottle() {

package/types/durable.ts

@@ -54,11 +54,11 @@ type WorkflowSearchOptions = {
 
 type WorkflowOptions = {
   namespace?: string;         //'durable' is the default namespace if not provided; similar to setting `appid` in the YAML
-  taskQueue: string;
+  taskQueue?: string;         //optional if entity is provided
   args: any[];                //input arguments to pass in
   workflowId?: string;        //execution id (the job id)
   entity?: string;            //If invoking a workflow, passing 'entity' will apply the value as the workflowName, taskQueue, and prefix, ensuring the FT.SEARCH index is properly scoped. This is a convenience method but limits options.
-  workflowName?: string;      //the name of the user's workflow function
+  workflowName?: string;      //the name of the user's workflow function; optional if 'entity' is provided
   parentWorkflowId?: string;  //system reserved; the id of the parent; if present the flow will not self-clean until the parent that spawned it self-cleans
   workflowTrace?: string;
   workflowSpan?: string;
@@ -68,7 +68,7 @@ type WorkflowOptions = {
 
 type HookOptions = {
   namespace?: string;   //'durable' is the default namespace if not provided; similar to setting `appid` in the YAML
-  taskQueue?: string;
+  taskQueue?: string;   //optional if 'entity' is provided
   args: any[];          //input arguments to pass into the hook
   entity?: string;      //If invoking a hook, passing 'entity' will apply the value as the workflowName, taskQueue, and prefix, ensuring the FT.SEARCH index is properly scoped. This is a convenience method but limits options.
   workflowId?: string;   //execution id (the job id to hook into)