@hotmeshio/hotmesh 0.0.29 → 0.0.31

package/README.md

@@ -92,19 +92,25 @@ npm install @hotmeshio/hotmesh
 
 Redis governance delivers more than just reliability. Externalizing state fundamentally changes the execution profile for your functions, allowing you to design long-running, durable workflows. The `MeshOS` base class (shown in the examples above) provides additional methods for solving the most common state management challenges.
 
- - `waitForSignal` | Pause your function and wait for external event(s) before continuing. The *waitForSignal* method will collate and cache the signals and only awaken your function once they've all arrived.
+ - `waitForSignal` | Pause your function and wait for external event(s) before continuing. The *waitForSignal* method will collate and cache the signals and only awaken your function once all signals have arrived.
  - `signal` | Send a signal (and optional payload) to any paused function.
  - `hook` | Redis governance converts your functions into 're-entrant processes'. Optionally use the *hook* method to spawn parallel execution threads to augment a running workflow.
- - `sleep` | Pause function execution for a ridiculous amount of time (months, years, etc). There's no risk of information loss, as Redis governs function state. When your function awakens, function state is efficiently (and automatically) restored.
+ - `sleep` | Pause function execution for a ridiculous amount of time (months, years, etc). There's no risk of information loss, as Redis governs function state. When your function awakens, function state is efficiently (and automatically) restored and your function will resume right where it left off.
  - `random` | Generate a deterministic random number that can be used in a reentrant process workflow (replaces `Math.random()`).
  - `executeChild` | Call another durable function and await the response. *Design sophisticated, multi-process solutions by leveraging this command.*
  - `startChild` | Call another durable function, but do not await the response.
- - `set` | Set one or more name/value pairs (e.g, `set('name1', 'value1', 'name1', 'value2')`)
- - `get` | Get a single value by name(e.g, `get('name')`)
- - `mget` | Get multiple values by name (e.g, `get('name1', 'name2')`)
- - `del` | Delete one or more entries by name and return the number deleted (e.g, `del('name1', 'name1')`)
- - `incr` | Increment (or decrement) a number (e.g, `incr('name', -99)`)
- - `mult` | Multiply a number (e.g, `mult('name', 12)`)
+ - `search` | Instance a search session (e.g, `const search = MeshOS.search()`)
+    - `set` | Set one or more name/value pairs (e.g, `search.set('name1', 'value1', 'name2', 'value2')`)
+    - `get` | Get a single value by name(e.g, `search.get('name')`)
+    - `mget` | Get multiple values by name (e.g, `search.mget('name1', 'name2')`)
+    - `del` | Delete one or more entries by name and return the number deleted (e.g, `search.del('name1', 'name2')`)
+    - `incr` | Increment (or decrement) a number (e.g, `search.incr('name', -99)`)
+    - `mult` | Multiply a number (e.g, `search.mult('name', 12)`)
+ - `find` | Find workflows using the native Redis [FT.*](https://redis.io/commands/ft.search/) search commands
+ - `findWhere` | Find workflows using a simplified, JSON-based search syntax that overlays the native Redis FT.SEARCH syntax.
+ - `createIndex` | Create a searchable index in Redis using simplified, JSON-based syntax that overlays the native Redis FT.CREATE syntax.
+ - `startWorkers` | Start the workers necessary to govern your class (typically called at server startup).
+ - `stopWorkers` | Stop all workers (typically called at server shutdown)
 
 Refer to the [hotmeshio/samples-typescript](https://github.com/hotmeshio/samples-typescript) repo for usage examples. 
 
@@ -216,31 +222,31 @@ const hotMesh = await HotMesh.init({
 ```
 
 ### Observability
-Workflows and activities are run according to the rules you define, offering [Graph-Oriented](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/system_lifecycle.md#telemetry) telemetry insights into your legacy function executions.
+Workflows and activities are run according to the rules you define, offering [Graph-Oriented](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/system_lifecycle.md#telemetry) telemetry insights into your legacy function executions.
 
 ## FAQ
-Refer to the [FAQ](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/faq.md) for terminology, definitions, and an exploration of how HotMesh facilitates orchestration use cases.
+Refer to the [FAQ](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/faq.md) for terminology, definitions, and an exploration of how HotMesh facilitates orchestration use cases.
 
 ## Quick Start
-Refer to the [Quick Start](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/quickstart.md) for sample flows you can easily copy, paste, and modify to get started.
+Refer to the [Quick Start](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/quickstart.md) for sample flows you can easily copy, paste, and modify to get started.
 
 ## Developer Guide
-For more details on the complete development process, including information about schemas, APIs, and deployment, consult the [Developer Guide](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/developer_guide.md).
+For more details on the complete development process, including information about schemas, APIs, and deployment, consult the [Developer Guide](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/developer_guide.md).
 
 ## Model Driven Development
-[Model Driven Development](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/model_driven_development.md) is an established strategy for managing process-oriented tasks. Check out this guide to understand its foundational principles.
+[Model Driven Development](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/model_driven_development.md) is an established strategy for managing process-oriented tasks. Check out this guide to understand its foundational principles.
 
 ## Data Mapping
-Exchanging data between activities is central to HotMesh. For detailed information on supported functions and the functional mapping syntax (@pipes), see the [Data Mapping Overview](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/data_mapping.md).
+Exchanging data between activities is central to HotMesh. For detailed information on supported functions and the functional mapping syntax (@pipes), see the [Data Mapping Overview](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/data_mapping.md).
 
 ## Composition
-While the simplest graphs are linear, detailing a consistent sequence of non-cyclical activities, graphs can be layered to represent intricate business scenarios. Some can even be designed to accommodate long-lasting workflows that span months. For more details, check out the [Composable Workflow Guide](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/composable_workflow.md).
+While the simplest graphs are linear, detailing a consistent sequence of non-cyclical activities, graphs can be layered to represent intricate business scenarios. Some can even be designed to accommodate long-lasting workflows that span months. For more details, check out the [Composable Workflow Guide](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/composable_workflow.md).
 
 ## Distributed Orchestration
-HotMesh is a distributed orchestration engine. Refer to the [Distributed Orchestration Guide](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/distributed_orchestration.md) for a detailed breakdown of the approach.
+HotMesh is a distributed orchestration engine. Refer to the [Distributed Orchestration Guide](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/distributed_orchestration.md) for a detailed breakdown of the approach.
 
 ## System Lifecycle
-Gain insight into HotMesh's monitoring, exception handling, and alarm configurations via the [System Lifecycle Guide](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/system_lifecycle.md).
+Gain insight into HotMesh's monitoring, exception handling, and alarm configurations via the [System Lifecycle Guide](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/system_lifecycle.md).
 
 ## Alpha Release
-So what exacty is an [alpha release](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/alpha.md)?
+So what exacty is an [alpha release](https://github.com/hotmeshio/sdk-typescript/tree/main/docs/alpha.md)?

package/build/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@hotmeshio/hotmesh",
-    "version": "0.0.29",
+    "version": "0.0.31",
     "description": "Unbreakable Workflows",
     "main": "build/index.js",
     "types": "build/index.d.ts",
@@ -54,7 +54,7 @@
         "test:durable:loopactivity": "NODE_ENV=test jest ./tests/durable/loopactivity/index.test.ts --detectOpenHandles --forceExit --verbose",
         "test:durable:nested": "NODE_ENV=test jest ./tests/durable/nested/index.test.ts --detectOpenHandles --forceExit --verbose"
     },
-    "keywords": [],
+    "keywords": ["durable workflow", "hotmesh", "service mesh", "workflows", "operational data", "redis"],
     "author": "luke.birdeau@gmail.com",
     "license": "SEE LICENSE IN LICENSE",
     "dependencies": {

package/build/services/durable/client.js

@@ -1,7 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.ClientService = void 0;
-const nanoid_1 = require("nanoid");
 const factory_1 = require("./factory");
 const handle_1 = require("./handle");
 const hotmesh_1 = require("../hotmesh");
@@ -88,14 +87,14 @@ class ClientService {
                 const payload = {
                     arguments: [...options.args],
                     parentWorkflowId: options.parentWorkflowId,
-                    workflowId: options.workflowId || (0, nanoid_1.nanoid)(),
+                    workflowId: options.workflowId || hotmesh_1.HotMeshService.guid(),
                     workflowTopic: workflowTopic,
                     backoffCoefficient: options.config?.backoffCoefficient || factory_1.DEFAULT_COEFFICIENT,
                 };
                 const context = { metadata: { trc, spn }, data: {} };
                 const jobId = await hotMeshClient.pub(`${options.namespace ?? factory_1.APP_ID}.execute`, payload, context);
+                //seed search data if present
                 if (jobId && options.search?.data) {
-                    //job successfully kicked off; there is default job data to persist
                     const searchSessionId = `-search-0`;
                     const search = new search_1.Search(jobId, hotMeshClient, searchSessionId);
                     for (const [key, value] of Object.entries(options.search.data)) {
@@ -125,8 +124,17 @@ class ClientService {
                     workflowTopic,
                     backoffCoefficient: options.config?.backoffCoefficient || factory_1.DEFAULT_COEFFICIENT,
                 };
+                //seed search data if presentthe hook before entering
                 const hotMeshClient = await this.getHotMeshClient(workflowTopic, options.namespace);
-                return await hotMeshClient.hook(`${hotMeshClient.appId}.flow.signal`, payload, types_1.StreamStatus.PENDING, 202);
+                const msgId = await hotMeshClient.hook(`${hotMeshClient.appId}.flow.signal`, payload, types_1.StreamStatus.PENDING, 202);
+                if (options.search?.data) {
+                    const searchSessionId = `-search-${hotmesh_1.HotMeshService.guid()}-0`;
+                    const search = new search_1.Search(options.workflowId, hotMeshClient, searchSessionId);
+                    for (const [key, value] of Object.entries(options.search.data)) {
+                        search.set(key, value);
+                    }
+                }
+                return msgId;
             },
             getHandle: async (taskQueue, workflowName, workflowId, namespace) => {
                 const workflowTopic = `${taskQueue}-${workflowName}`;

package/build/services/durable/index.d.ts

@@ -1,12 +1,14 @@
 import { ClientService } from './client';
 import { ConnectionService } from './connection';
 import { MeshOSService } from './meshos';
+import { Search } from './search';
 import { WorkerService } from './worker';
 import { WorkflowService } from './workflow';
 import { ContextType } from '../../types/durable';
 export declare const Durable: {
     Client: typeof ClientService;
     Connection: typeof ConnectionService;
+    Search: typeof Search;
     MeshOS: typeof MeshOSService;
     Worker: typeof WorkerService;
     workflow: typeof WorkflowService;

package/build/services/durable/index.js

@@ -4,11 +4,13 @@ exports.Durable = void 0;
 const client_1 = require("./client");
 const connection_1 = require("./connection");
 const meshos_1 = require("./meshos");
+const search_1 = require("./search");
 const worker_1 = require("./worker");
 const workflow_1 = require("./workflow");
 exports.Durable = {
     Client: client_1.ClientService,
     Connection: connection_1.ConnectionService,
+    Search: search_1.Search,
     MeshOS: meshos_1.MeshOSService,
     Worker: worker_1.WorkerService,
     workflow: workflow_1.WorkflowService,

package/build/services/durable/workflow.d.ts

@@ -1,52 +1,97 @@
 import { Search } from './search';
 import { HotMeshService as HotMesh } from '../hotmesh';
-import { ActivityConfig, HookOptions, ProxyType, WorkflowOptions } from "../../types/durable";
+import { ActivityConfig, HookOptions, ProxyType, WorkflowContext, WorkflowOptions } from "../../types/durable";
 export declare class WorkflowService {
     /**
-     * Spawn a child workflow. await and return the result.
+     * Spawns a child workflow. await and return the result.
+     * @template T - the result type
+     * @param {WorkflowOptions} options - the workflow options
+     * @returns {Promise<T>} - the result of the child workflow
      */
     static executeChild<T>(options: WorkflowOptions): Promise<T>;
     /**
-     * spawn a child workflow. return the childJobId.
+     * Spawns a child workflow. return the childJobId.
+     * This method is used when the result of the child workflow is not needed.
+     * @param {WorkflowOptions} options - the workflow options
+     * @returns {Promise<string>} - the childJobId
      */
     static startChild<T>(options: WorkflowOptions): Promise<string>;
     /**
-     * wrap all activities in a proxy that will durably run them
+     * Wraps activities in a proxy that will durably run them
+     * @param {ActivityConfig} options - the activity configuration
+     * that will be used to wrap the activities. You must pass an
+     * `activities` object to this configuration. The activities object
+     * should be a key-value pair of activity names and their respective
+     * functions. This is typically done by importing the activities.
+     *
+     * @returns {ProxyType<ACT>} - a proxy object with the same keys as the
+     * activities object, but with the values replaced by a wrapped function
+     * @example
+     * // import the activities
+     * import * as activities from './activities';
+     * const proxy = WorkflowService.proxyActivities<typeof activities>({ activities });
+     *
+     * //or destructure the proxy object, as the function names are the keys
+     * const { activity1, activity2 } = WorkflowService.proxyActivities<typeof activities>({ activities });
      */
     static proxyActivities<ACT>(options?: ActivityConfig): ProxyType<ACT>;
     /**
-     * return a search session for use when reading/writing to the workflow HASH
+     * Returns a search session for use when reading/writing to the workflow HASH.
+     * The search session provides access to methods like `get`, `mget`, `set`, `del`, and `incr`.
+     * @returns {Promise<Search>} - a search session
      */
     static search(): Promise<Search>;
     /**
-     * return a handle to the hotmesh client currently running the workflow
+     * Return a handle to the hotmesh client currently running the workflow
+     * @returns {Promise<HotMesh>} - a hotmesh client
      */
     static getHotMesh(): Promise<HotMesh>;
     /**
-     * those methods that may only be called once must be protected by flagging
+     * Returns the current workflow context
+     * @returns {WorkflowContext} - the current workflow context
+     */
+    static getContext(): WorkflowContext;
+    /**
+     * Those methods that may only be called once must be protected by flagging
      * their execution with a unique key (the key is stored in the HASH alongside
      * process state and job state)
+     * @private
      */
     static isSideEffectAllowed(hotMeshClient: HotMesh, prefix: string): Promise<boolean>;
     /**
-     * returns a random number between 0 and 1. This number is deterministic
+     * Returns a random number between 0 and 1. This number is deterministic
      * and will never vary for a given seed. This is useful for randomizing
      * pathways in a workflow that can be safely replayed.
-     * @returns {number}
+     * @returns {number} - a random number between 0 and 1
      */
     static random(): number;
     /**
-     * send signal data into any other paused thread (which is paused and
+     * Sends signal data into any other paused thread (which is paused and
      * awaiting the signal) from within a hook-thread or the main-thread
+     * @param {string} signalId - the signal id
+     * @param {Record<any, any>} data - the signal data
+     * @returns {Promise<string>} - the stream id
      */
     static signal(signalId: string, data: Record<any, any>): Promise<string>;
     /**
-     * spawn a hook from either the main thread or a hook thread with
+     * Spawns a hook from either the main thread or a hook thread with
      * the provided options; worflowId/TaskQueue/Name are optional and will
      * default to the current workflowId/WorkflowTopic if not provided
+     * @param {HookOptions} options - the hook options
      */
     static hook(options: HookOptions): Promise<string>;
+    /**
+     * Sleeps for a duration.
+     * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
+     * @returns {Promise<number>}
+     */
     static sleep(duration: string): Promise<number>;
+    /**
+     * Waits for a signal to awaken
+     * @param {string[]} signals - the signals to wait for
+     * @param {Record<string, string>} options - the options
+     * @returns {Promise<Record<any, any>[]>}
+     */
     static waitForSignal(signals: string[], options?: Record<string, string>): Promise<Record<any, any>[]>;
     static wrapActivity<T>(activityName: string, options?: ActivityConfig): T;
 }

package/build/services/durable/workflow.js

@@ -17,7 +17,10 @@ const stream_1 = require("../../types/stream");
 const utils_1 = require("../../modules/utils");
 class WorkflowService {
     /**
-     * Spawn a child workflow. await and return the result.
+     * Spawns a child workflow. await and return the result.
+     * @template T - the result type
+     * @param {WorkflowOptions} options - the workflow options
+     * @returns {Promise<T>} - the result of the child workflow
      */
     static async executeChild(options) {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -28,15 +31,16 @@ class WorkflowService {
         const workflowSpan = store.get('workflowSpan');
         const COUNTER = store.get('counter');
         const execIndex = COUNTER.counter = COUNTER.counter + 1;
-        //this is risky but MUST be allowed. Users MAY set the workflowId,
-        //but if there is a naming collision, the data from the target entity will be used
-        //as there is know way of knowing if the item was generated via a prior run of the workflow
-        const childJobId = options.workflowId ?? `-${workflowId}-$${options.workflowName}${workflowDimension}-${execIndex}`;
+        //NOTE: this is the hash prefix; necessary for the search index to locate the entity
+        //if the hash is a helper, a dash begins it, so it isn't indexed
+        const entityOrEmptyString = options.entity ?? '';
+        //If the workflowId is not provided, it is generated from the entity and the workflow name
+        const childJobId = options.workflowId ?? `${entityOrEmptyString}-${workflowId}-$${options.entity ?? options.workflowName}${workflowDimension}-${execIndex}`;
         const parentWorkflowId = `${workflowId}-f`;
         const client = new client_1.ClientService({
             connection: await connection_1.ConnectionService.connect(worker_1.WorkerService.connection),
         });
-        let handle = await client.workflow.getHandle(options.taskQueue, options.workflowName, childJobId, namespace);
+        let handle = await client.workflow.getHandle(options.entity ?? options.taskQueue, options.entity ?? options.workflowName, childJobId, namespace);
         try {
             return await handle.result(true);
         }
@@ -55,7 +59,10 @@ class WorkflowService {
         }
     }
     /**
-     * spawn a child workflow. return the childJobId.
+     * Spawns a child workflow. return the childJobId.
+     * This method is used when the result of the child workflow is not needed.
+     * @param {WorkflowOptions} options - the workflow options
+     * @returns {Promise<string>} - the childJobId
      */
     static async startChild(options) {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -66,9 +73,12 @@ class WorkflowService {
         const workflowSpan = store.get('workflowSpan');
         const COUNTER = store.get('counter');
         const execIndex = COUNTER.counter = COUNTER.counter + 1;
-        const childJobId = options.workflowId ?? `-${workflowId}-$${options.workflowName}${workflowDimension}-${execIndex}`;
+        //NOTE: this is the hash prefix; necessary for the search index to locate the entity
+        const entityOrEmptyString = options.entity ?? '';
+        //If the workflowId is not provided, it is generated from the entity and the workflow name
+        const childJobId = options.workflowId ?? `${entityOrEmptyString}-${workflowId}-$${options.entity ?? options.workflowName}${workflowDimension}-${execIndex}`;
         const parentWorkflowId = `${workflowId}-f`;
-        const workflowTopic = `${options.taskQueue}-${options.workflowName}`;
+        const workflowTopic = `${options.entity ?? options.taskQueue}-${options.entity ?? options.workflowName}`;
         try {
             //get the status; if there is no error, return childJobId (what was spawned)
             const hotMeshClient = await worker_1.WorkerService.getHotMesh(workflowTopic, { namespace });
@@ -91,7 +101,22 @@ class WorkflowService {
         }
     }
     /**
-     * wrap all activities in a proxy that will durably run them
+     * Wraps activities in a proxy that will durably run them
+     * @param {ActivityConfig} options - the activity configuration
+     * that will be used to wrap the activities. You must pass an
+     * `activities` object to this configuration. The activities object
+     * should be a key-value pair of activity names and their respective
+     * functions. This is typically done by importing the activities.
+     *
+     * @returns {ProxyType<ACT>} - a proxy object with the same keys as the
+     * activities object, but with the values replaced by a wrapped function
+     * @example
+     * // import the activities
+     * import * as activities from './activities';
+     * const proxy = WorkflowService.proxyActivities<typeof activities>({ activities });
+     *
+     * //or destructure the proxy object, as the function names are the keys
+     * const { activity1, activity2 } = WorkflowService.proxyActivities<typeof activities>({ activities });
      */
     static proxyActivities(options) {
         if (options.activities) {
@@ -108,7 +133,9 @@ class WorkflowService {
         return proxy;
     }
     /**
-     * return a search session for use when reading/writing to the workflow HASH
+     * Returns a search session for use when reading/writing to the workflow HASH.
+     * The search session provides access to methods like `get`, `mget`, `set`, `del`, and `incr`.
+     * @returns {Promise<Search>} - a search session
      */
     static async search() {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -124,7 +151,8 @@ class WorkflowService {
         return new search_1.Search(workflowId, hotMeshClient, searchSessionId);
     }
     /**
-     * return a handle to the hotmesh client currently running the workflow
+     * Return a handle to the hotmesh client currently running the workflow
+     * @returns {Promise<HotMesh>} - a hotmesh client
      */
     static async getHotMesh() {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -133,9 +161,33 @@ class WorkflowService {
         return await worker_1.WorkerService.getHotMesh(workflowTopic, { namespace });
     }
     /**
-     * those methods that may only be called once must be protected by flagging
+     * Returns the current workflow context
+     * @returns {WorkflowContext} - the current workflow context
+     */
+    static getContext() {
+        const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
+        const workflowId = store.get('workflowId');
+        const workflowDimension = store.get('workflowDimension') ?? '';
+        const workflowTopic = store.get('workflowTopic');
+        const namespace = store.get('namespace');
+        const workflowTrace = store.get('workflowTrace');
+        const workflowSpan = store.get('workflowSpan');
+        const COUNTER = store.get('counter');
+        return {
+            counter: COUNTER.counter,
+            namespace,
+            workflowId,
+            workflowDimension,
+            workflowTopic,
+            workflowTrace,
+            workflowSpan,
+        };
+    }
+    /**
+     * Those methods that may only be called once must be protected by flagging
      * their execution with a unique key (the key is stored in the HASH alongside
      * process state and job state)
+     * @private
      */
     static async isSideEffectAllowed(hotMeshClient, prefix) {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -153,10 +205,10 @@ class WorkflowService {
         return guidValue === 1;
     }
     /**
-     * returns a random number between 0 and 1. This number is deterministic
+     * Returns a random number between 0 and 1. This number is deterministic
      * and will never vary for a given seed. This is useful for randomizing
      * pathways in a workflow that can be safely replayed.
-     * @returns {number}
+     * @returns {number} - a random number between 0 and 1
      */
     static random() {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -165,8 +217,11 @@ class WorkflowService {
         return (0, utils_1.deterministicRandom)(seed);
     }
     /**
-     * send signal data into any other paused thread (which is paused and
+     * Sends signal data into any other paused thread (which is paused and
      * awaiting the signal) from within a hook-thread or the main-thread
+     * @param {string} signalId - the signal id
+     * @param {Record<any, any>} data - the signal data
+     * @returns {Promise<string>} - the stream id
      */
     static async signal(signalId, data) {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -180,9 +235,10 @@ class WorkflowService {
         }
     }
     /**
-     * spawn a hook from either the main thread or a hook thread with
+     * Spawns a hook from either the main thread or a hook thread with
      * the provided options; worflowId/TaskQueue/Name are optional and will
      * default to the current workflowId/WorkflowTopic if not provided
+     * @param {HookOptions} options - the hook options
      */
     static async hook(options) {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -193,8 +249,8 @@ class WorkflowService {
             const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
             const workflowId = options.workflowId ?? store.get('workflowId');
             let workflowTopic = store.get('workflowTopic');
-            if (options.taskQueue && options.workflowName) {
-                workflowTopic = `${options.taskQueue}-${options.workflowName}`;
+            if (options.entity || (options.taskQueue && options.workflowName)) {
+                workflowTopic = `${options.entity ?? options.taskQueue}-${options.entity ?? options.workflowName}`;
             } //else this is essentially recursion as the function calls itself
             const payload = {
                 arguments: [...options.args],
@@ -205,6 +261,11 @@ class WorkflowService {
             return await hotMeshClient.hook(`${namespace}.flow.signal`, payload, stream_1.StreamStatus.PENDING, 202);
         }
     }
+    /**
+     * Sleeps for a duration.
+     * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
+     * @returns {Promise<number>}
+     */
     static async sleep(duration) {
         const seconds = (0, ms_1.default)(duration) / 1000;
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
@@ -227,6 +288,12 @@ class WorkflowService {
             throw new errors_1.DurableSleepError(workflowId, seconds, execIndex, workflowDimension);
         }
     }
+    /**
+     * Waits for a signal to awaken
+     * @param {string[]} signals - the signals to wait for
+     * @param {Record<string, string>} options - the options
+     * @returns {Promise<Record<any, any>[]>}
+     */
     static async waitForSignal(signals, options) {
         const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
         const COUNTER = store.get('counter');

package/build/services/signaler/stream.js

@@ -220,7 +220,6 @@ class StreamSignaler {
         for (const instance of [...StreamSignaler.signalers]) {
             instance.stopConsuming();
         }
-        await (0, utils_1.sleepFor)(BLOCK_TIME_MS);
     }
     async stopConsuming() {
         this.shouldConsume = false;

package/build/types/durable.d.ts

@@ -5,6 +5,36 @@ type WorkflowConfig = {
     maximumInterval?: string;
     initialInterval?: string;
 };
+type WorkflowContext = {
+    /**
+     * the reentrant semaphore, incremented in real-time as idempotent statements are re-traversed upon reentry. Indicates the current semaphore count.
+     */
+    counter: number;
+    /**
+     * the HotMesh App namespace. `durable` is the default.
+     */
+    namespace: string;
+    /**
+     * the workflow/job ID
+     */
+    workflowId: string;
+    /**
+     * the dimensional isolation for the reentrant hook, expressed in the format `0,0`, `0,1`, etc
+     */
+    workflowDimension: string;
+    /**
+     * a concatenation of the task queue and workflow name (e.g., `${taskQueueName}-${workflowName}`)
+     */
+    workflowTopic: string;
+    /**
+     * the open telemetry trace context for the workflow, used for logging and tracing. If a sink is enabled, this will be sent to the sink.
+     */
+    workflowTrace: string;
+    /**
+     * the open telemetry span context for the workflow, used for logging and tracing. If a sink is enabled, this will be sent to the sink.
+     */
+    workflowSpan: string;
+};
 type WorkflowSearchOptions = {
     index?: string;
     prefix?: string[];
@@ -19,6 +49,7 @@ type WorkflowOptions = {
     taskQueue: string;
     args: any[];
     workflowId?: string;
+    entity?: string;
     workflowName?: string;
     parentWorkflowId?: string;
     workflowTrace?: string;
@@ -30,6 +61,7 @@ type HookOptions = {
     namespace?: string;
     taskQueue?: string;
     args: any[];
+    entity?: string;
     workflowId?: string;
     workflowName?: string;
     search?: WorkflowSearchOptions;
@@ -142,4 +174,4 @@ type ActivityConfig = {
         maximumInterval: string;
     };
 };
-export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, FindWhereOptions, FindWhereQuery, HookOptions, MeshOSActivityOptions, MeshOSWorkerOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkerConfig, WorkflowConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, };
+export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, FindWhereOptions, FindWhereQuery, HookOptions, MeshOSActivityOptions, MeshOSWorkerOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkerConfig, WorkflowConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, WorkflowContext, };

package/build/types/index.d.ts

@@ -3,7 +3,7 @@ export { App, AppVID, AppTransitions, AppSubscriptions } from './app';
 export { AsyncSignal } from './async';
 export { CacheMode } from './cache';
 export { CollationFaultType, CollationStage } from './collator';
-export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, FindWhereOptions, FindWhereQuery, HookOptions, MeshOSActivityOptions, MeshOSWorkerOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkflowConfig, WorkerConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, } from './durable';
+export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, FindWhereOptions, FindWhereQuery, HookOptions, MeshOSActivityOptions, MeshOSWorkerOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkflowConfig, WorkerConfig, WorkerOptions, WorkflowContext, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, } from './durable';
 export { HookCondition, HookConditions, HookGate, HookInterface, HookRule, HookRules, HookSignal } from './hook';
 export { RedisClientType as IORedisClientType, RedisMultiType as IORedisMultiType } from './ioredisclient';
 export { ILogger } from './logger';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hotmeshio/hotmesh",
-  "version": "0.0.29",
+  "version": "0.0.31",
   "description": "Unbreakable Workflows",
   "main": "build/index.js",
   "types": "build/index.d.ts",
@@ -54,7 +54,7 @@
     "test:durable:loopactivity": "NODE_ENV=test jest ./tests/durable/loopactivity/index.test.ts --detectOpenHandles --forceExit --verbose",
     "test:durable:nested": "NODE_ENV=test jest ./tests/durable/nested/index.test.ts --detectOpenHandles --forceExit --verbose"
   },
-  "keywords": [],
+  "keywords": ["durable workflow", "hotmesh", "service mesh", "workflows", "operational data", "redis"],
   "author": "luke.birdeau@gmail.com",
   "license": "SEE LICENSE IN LICENSE",
   "dependencies": {

package/services/durable/client.ts

@@ -1,4 +1,3 @@
-import { nanoid } from 'nanoid';
 import { APP_ID, APP_VERSION, DEFAULT_COEFFICIENT, getWorkflowYAML } from './factory';
 import { WorkflowHandleService } from './handle';
 import { HotMeshService as HotMesh } from '../hotmesh';
@@ -104,7 +103,7 @@ export class ClientService {
       const payload = {
         arguments: [...options.args],
         parentWorkflowId: options.parentWorkflowId,
-        workflowId: options.workflowId || nanoid(),
+        workflowId: options.workflowId || HotMesh.guid(),
         workflowTopic: workflowTopic,
         backoffCoefficient: options.config?.backoffCoefficient || DEFAULT_COEFFICIENT,
       }
@@ -112,9 +111,10 @@ export class ClientService {
       const jobId = await hotMeshClient.pub(
         `${options.namespace ?? APP_ID}.execute`,
         payload,
-        context as JobState);
-       if (jobId && options.search?.data) {
-        //job successfully kicked off; there is default job data to persist
+        context as JobState
+      );
+      //seed search data if present
+      if (jobId && options.search?.data) {
         const searchSessionId = `-search-0`;
         const search = new Search(jobId, hotMeshClient, searchSessionId);
         for (const [key, value] of Object.entries(options.search.data)) {
@@ -146,8 +146,17 @@ export class ClientService {
         workflowTopic,
         backoffCoefficient: options.config?.backoffCoefficient || DEFAULT_COEFFICIENT,
       }
+      //seed search data if presentthe hook before entering
       const hotMeshClient = await this.getHotMeshClient(workflowTopic, options.namespace);
-      return await hotMeshClient.hook(`${hotMeshClient.appId}.flow.signal`, payload, StreamStatus.PENDING, 202);
+      const msgId = await hotMeshClient.hook(`${hotMeshClient.appId}.flow.signal`, payload, StreamStatus.PENDING, 202);
+      if (options.search?.data) {
+        const searchSessionId = `-search-${HotMesh.guid()}-0`;
+        const search = new Search(options.workflowId, hotMeshClient, searchSessionId);
+        for (const [key, value] of Object.entries(options.search.data)) {
+          search.set(key, value);
+        }
+      }
+      return msgId;
     },
 
     getHandle: async (taskQueue: string, workflowName: string, workflowId: string, namespace?: string): Promise<WorkflowHandleService> => {

package/services/durable/index.ts

@@ -1,6 +1,7 @@
 import { ClientService } from './client';
 import { ConnectionService } from './connection';
 import { MeshOSService } from './meshos';
+import { Search } from './search';
 import { WorkerService } from './worker';
 import { WorkflowService } from './workflow';
 import { ContextType } from '../../types/durable';
@@ -8,6 +9,7 @@ import { ContextType } from '../../types/durable';
 export const Durable = {
   Client: ClientService,
   Connection: ConnectionService,
+  Search,
   MeshOS: MeshOSService,
   Worker: WorkerService,
   workflow: WorkflowService,

package/services/durable/workflow.ts

@@ -16,6 +16,7 @@ import {
   ActivityConfig,
   HookOptions,
   ProxyType,
+  WorkflowContext,
   WorkflowOptions } from "../../types/durable";
 import { JobOutput, JobState } from '../../types/job';
 import { StreamStatus } from '../../types/stream';
@@ -24,7 +25,10 @@ import { deterministicRandom } from '../../modules/utils';
 export class WorkflowService {
 
   /**
-   * Spawn a child workflow. await and return the result.
+   * Spawns a child workflow. await and return the result.
+   * @template T - the result type
+   * @param {WorkflowOptions} options - the workflow options
+   * @returns {Promise<T>} - the result of the child workflow
    */
   static async executeChild<T>(options: WorkflowOptions): Promise<T> {
     const store = asyncLocalStorage.getStore();
@@ -35,10 +39,11 @@ export class WorkflowService {
     const workflowSpan = store.get('workflowSpan');
     const COUNTER = store.get('counter');
     const execIndex = COUNTER.counter = COUNTER.counter + 1;
-    //this is risky but MUST be allowed. Users MAY set the workflowId,
-    //but if there is a naming collision, the data from the target entity will be used
-    //as there is know way of knowing if the item was generated via a prior run of the workflow
-    const childJobId = options.workflowId ?? `-${workflowId}-$${options.workflowName}${workflowDimension}-${execIndex}`;
+    //NOTE: this is the hash prefix; necessary for the search index to locate the entity
+    //if the hash is a helper, a dash begins it, so it isn't indexed
+    const entityOrEmptyString = options.entity ?? '';
+    //If the workflowId is not provided, it is generated from the entity and the workflow name
+    const childJobId = options.workflowId ?? `${entityOrEmptyString}-${workflowId}-$${options.entity ?? options.workflowName}${workflowDimension}-${execIndex}`;
     const parentWorkflowId = `${workflowId}-f`;
 
     const client = new Client({
@@ -46,8 +51,8 @@ export class WorkflowService {
     });
 
     let handle = await client.workflow.getHandle(
-      options.taskQueue,
-      options.workflowName,
+      options.entity ?? options.taskQueue,
+      options.entity ?? options.workflowName,
       childJobId,
       namespace,
     );
@@ -70,7 +75,10 @@ export class WorkflowService {
   }
 
   /**
-   * spawn a child workflow. return the childJobId.
+   * Spawns a child workflow. return the childJobId.
+   * This method is used when the result of the child workflow is not needed.
+   * @param {WorkflowOptions} options - the workflow options
+   * @returns {Promise<string>} - the childJobId
    */
   static async startChild<T>(options: WorkflowOptions): Promise<string> {
     const store = asyncLocalStorage.getStore();
@@ -81,9 +89,12 @@ export class WorkflowService {
     const workflowSpan = store.get('workflowSpan');
     const COUNTER = store.get('counter');
     const execIndex = COUNTER.counter = COUNTER.counter + 1;
-    const childJobId = options.workflowId ?? `-${workflowId}-$${options.workflowName}${workflowDimension}-${execIndex}`;
+    //NOTE: this is the hash prefix; necessary for the search index to locate the entity
+    const entityOrEmptyString = options.entity ?? '';
+    //If the workflowId is not provided, it is generated from the entity and the workflow name
+    const childJobId = options.workflowId ?? `${entityOrEmptyString}-${workflowId}-$${options.entity ?? options.workflowName}${workflowDimension}-${execIndex}`;
     const parentWorkflowId = `${workflowId}-f`;
-    const workflowTopic = `${options.taskQueue}-${options.workflowName}`;
+    const workflowTopic = `${options.entity ?? options.taskQueue}-${options.entity ?? options.workflowName}`;
 
     try {
       //get the status; if there is no error, return childJobId (what was spawned)
@@ -108,7 +119,22 @@ export class WorkflowService {
   }
 
   /**
-   * wrap all activities in a proxy that will durably run them
+   * Wraps activities in a proxy that will durably run them
+   * @param {ActivityConfig} options - the activity configuration
+   * that will be used to wrap the activities. You must pass an
+   * `activities` object to this configuration. The activities object
+   * should be a key-value pair of activity names and their respective
+   * functions. This is typically done by importing the activities.
+   * 
+   * @returns {ProxyType<ACT>} - a proxy object with the same keys as the
+   * activities object, but with the values replaced by a wrapped function
+   * @example
+   * // import the activities
+   * import * as activities from './activities';
+   * const proxy = WorkflowService.proxyActivities<typeof activities>({ activities });
+   * 
+   * //or destructure the proxy object, as the function names are the keys
+   * const { activity1, activity2 } = WorkflowService.proxyActivities<typeof activities>({ activities });
    */
   static proxyActivities<ACT>(options?: ActivityConfig): ProxyType<ACT> {
     if (options.activities) {
@@ -127,7 +153,9 @@ export class WorkflowService {
   }
 
   /**
-   * return a search session for use when reading/writing to the workflow HASH
+   * Returns a search session for use when reading/writing to the workflow HASH.
+   * The search session provides access to methods like `get`, `mget`, `set`, `del`, and `incr`.
+   * @returns {Promise<Search>} - a search session
    */
   static async search(): Promise<Search> {
     const store = asyncLocalStorage.getStore();
@@ -144,7 +172,8 @@ export class WorkflowService {
   }
 
   /**
-   * return a handle to the hotmesh client currently running the workflow
+   * Return a handle to the hotmesh client currently running the workflow
+   * @returns {Promise<HotMesh>} - a hotmesh client
    */
   static async getHotMesh(): Promise<HotMesh> {
     const store = asyncLocalStorage.getStore();
@@ -154,9 +183,34 @@ export class WorkflowService {
   }
 
   /**
-   * those methods that may only be called once must be protected by flagging
+   * Returns the current workflow context
+   * @returns {WorkflowContext} - the current workflow context
+   */
+  static getContext(): WorkflowContext {
+    const store = asyncLocalStorage.getStore();
+    const workflowId = store.get('workflowId');
+    const workflowDimension = store.get('workflowDimension') ?? '';
+    const workflowTopic = store.get('workflowTopic');
+    const namespace = store.get('namespace');
+    const workflowTrace = store.get('workflowTrace');
+    const workflowSpan = store.get('workflowSpan');
+    const COUNTER = store.get('counter');
+    return {
+      counter: COUNTER.counter,
+      namespace,
+      workflowId,
+      workflowDimension,
+      workflowTopic,
+      workflowTrace,
+      workflowSpan,
+    };
+  }
+
+  /**
+   * Those methods that may only be called once must be protected by flagging
    * their execution with a unique key (the key is stored in the HASH alongside
    * process state and job state)
+   * @private
    */
   static async isSideEffectAllowed(hotMeshClient: HotMesh, prefix:string): Promise<boolean> {
     const store = asyncLocalStorage.getStore();
@@ -175,10 +229,10 @@ export class WorkflowService {
   }
 
   /**
-   * returns a random number between 0 and 1. This number is deterministic
+   * Returns a random number between 0 and 1. This number is deterministic
    * and will never vary for a given seed. This is useful for randomizing
    * pathways in a workflow that can be safely replayed.
-   * @returns {number}
+   * @returns {number} - a random number between 0 and 1
    */
   static random(): number {
     const store = asyncLocalStorage.getStore();
@@ -188,8 +242,11 @@ export class WorkflowService {
   }
 
   /**
-   * send signal data into any other paused thread (which is paused and
+   * Sends signal data into any other paused thread (which is paused and
    * awaiting the signal) from within a hook-thread or the main-thread
+   * @param {string} signalId - the signal id
+   * @param {Record<any, any>} data - the signal data
+   * @returns {Promise<string>} - the stream id
    */
   static async signal(signalId: string, data: Record<any, any>): Promise<string> {
     const store = asyncLocalStorage.getStore();
@@ -204,9 +261,10 @@ export class WorkflowService {
   }
 
   /**
-   * spawn a hook from either the main thread or a hook thread with
+   * Spawns a hook from either the main thread or a hook thread with
    * the provided options; worflowId/TaskQueue/Name are optional and will
    * default to the current workflowId/WorkflowTopic if not provided
+   * @param {HookOptions} options - the hook options
    */
   static async hook(options: HookOptions): Promise<string> {
     const store = asyncLocalStorage.getStore();
@@ -217,8 +275,8 @@ export class WorkflowService {
       const store = asyncLocalStorage.getStore();
       const workflowId = options.workflowId ?? store.get('workflowId');
       let workflowTopic = store.get('workflowTopic');
-      if (options.taskQueue && options.workflowName) {
-        workflowTopic = `${options.taskQueue}-${options.workflowName}`;
+      if (options.entity || (options.taskQueue && options.workflowName)) {
+        workflowTopic = `${options.entity ?? options.taskQueue}-${options.entity ?? options.workflowName}`;
       } //else this is essentially recursion as the function calls itself
       const payload = {
         arguments: [...options.args],
@@ -230,6 +288,11 @@ export class WorkflowService {
     }
   }
 
+  /**
+   * Sleeps for a duration.
+   * @param {string} duration - for example: '1 minute', '2 hours', '3 days'
+   * @returns {Promise<number>}
+   */
   static async sleep(duration: string): Promise<number> {
     const seconds = ms(duration) / 1000;
 
@@ -254,6 +317,12 @@ export class WorkflowService {
     }
   }
 
+  /**
+   * Waits for a signal to awaken
+   * @param {string[]} signals - the signals to wait for
+   * @param {Record<string, string>} options - the options
+   * @returns {Promise<Record<any, any>[]>}
+   */
   static async waitForSignal(signals: string[], options?: Record<string, string>): Promise<Record<any, any>[]> {
     const store = asyncLocalStorage.getStore();
     const COUNTER = store.get('counter');

package/services/signaler/stream.ts

@@ -254,7 +254,6 @@ class StreamSignaler {
     for (const instance of [...StreamSignaler.signalers]) {
       instance.stopConsuming();
     }
-    await sleepFor(BLOCK_TIME_MS);
   }
   
   async stopConsuming() {

package/types/durable.ts

@@ -7,6 +7,44 @@ type WorkflowConfig = {
   initialInterval?: string; //default 1s
 }
 
+type WorkflowContext = {
+
+  /**
+   * the reentrant semaphore, incremented in real-time as idempotent statements are re-traversed upon reentry. Indicates the current semaphore count.
+   */
+  counter: number;
+
+  /**
+   * the HotMesh App namespace. `durable` is the default.
+   */
+  namespace: string;
+
+  /**
+   * the workflow/job ID
+   */
+  workflowId: string;
+
+  /** 
+   * the dimensional isolation for the reentrant hook, expressed in the format `0,0`, `0,1`, etc
+   */
+  workflowDimension: string;
+
+  /**
+   * a concatenation of the task queue and workflow name (e.g., `${taskQueueName}-${workflowName}`)
+   */
+  workflowTopic: string;
+
+  /**
+   * the open telemetry trace context for the workflow, used for logging and tracing. If a sink is enabled, this will be sent to the sink.
+   */
+  workflowTrace: string;
+
+  /**
+   * the open telemetry span context for the workflow, used for logging and tracing. If a sink is enabled, this will be sent to the sink.
+   */
+  workflowSpan: string;
+}
+
 type WorkflowSearchOptions = {
   index?: string;         //FT index name (myapp:myindex)
   prefix?: string[];      //FT prefixes (['myapp:myindex:prefix1', 'myapp:myindex:prefix2'])
@@ -15,11 +53,12 @@ type WorkflowSearchOptions = {
 }
 
 type WorkflowOptions = {
-  namespace?: string;   //'durable' is the default namespace if not provided; similar to setting `appid` in the YAML
+  namespace?: string;         //'durable' is the default namespace if not provided; similar to setting `appid` in the YAML
   taskQueue: string;
-  args: any[];          //input arguments to pass in
-  workflowId?: string;   //execution id (the job id)
-  workflowName?: string; //the name of the user's workflow function
+  args: any[];                //input arguments to pass in
+  workflowId?: string;        //execution id (the job id)
+  entity?: string;            //If invoking a workflow, passing 'entity' will apply the value as the workflowName, taskQueue, and prefix, ensuring the FT.SEARCH index is properly scoped. This is a convenience method but limits options.
+  workflowName?: string;      //the name of the user's workflow function
   parentWorkflowId?: string;  //system reserved; the id of the parent; if present the flow will not self-clean until the parent that spawned it self-cleans
   workflowTrace?: string;
   workflowSpan?: string;
@@ -31,6 +70,7 @@ type HookOptions = {
   namespace?: string;   //'durable' is the default namespace if not provided; similar to setting `appid` in the YAML
   taskQueue?: string;
   args: any[];          //input arguments to pass into the hook
+  entity?: string;      //If invoking a hook, passing 'entity' will apply the value as the workflowName, taskQueue, and prefix, ensuring the FT.SEARCH index is properly scoped. This is a convenience method but limits options.
   workflowId?: string;   //execution id (the job id to hook into)
   workflowName?: string; //the name of the user's hook function
   search?: WorkflowSearchOptions //bind additional search terms immediately before hook reentry
@@ -188,4 +228,5 @@ export {
   WorkflowSearchOptions,
   WorkflowDataType,
   WorkflowOptions,
+  WorkflowContext,
 };

package/types/index.ts

@@ -49,6 +49,7 @@ export {
     WorkflowConfig,
     WorkerConfig,
     WorkerOptions,
+    WorkflowContext,
     WorkflowSearchOptions,
     WorkflowDataType,
     WorkflowOptions,