@hotmeshio/hotmesh 0.0.24 → 0.0.26

package/README.md

@@ -1,14 +1,11 @@
 # HotMesh
 ![alpha release](https://img.shields.io/badge/release-alpha-yellow)
 
-Elevate Redis from an in-memory data cache to a game-changing [service mesh](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/faq.md#what-is-hotmesh). Turn your unpredictable functions into unbreakable workflows.
+Elevate Redis from an in-memory data cache, and turn your unpredictable functions into unbreakable workflows.
 
-## What is HotMesh?
-HotMesh is a wrapper for Redis that exposes a higher level set of domain constructs like ‘activities’, ‘workflows’, 'jobs', etc. Behind the scenes, it uses *Redis Data* (Hash, ZSet, and List); *Redis Streams* (XReadGroup, XAdd, XLen); and *Redis Publish/Subscribe*.
+**HotMesh** is a wrapper for Redis that exposes a higher level set of domain constructs like ‘activities’, ‘workflows’, 'jobs', etc. Behind the scenes, it uses *Redis Data* (Hash, ZSet, List); *Redis Streams* (XReadGroup, XAdd, XLen, etc); and *Redis Publish/Subscribe*.
 
-The ultimate goal is to resurface Redis as a *Durable Service Mesh*, capable of running unbreakable workflows that span your microservices. The technical term for this type of durability is *Reentrant Process Engine*.
-
-It's still Redis in the background, but the information flow is fundamentally different. Your functions no longer call Redis (e.g., to cache a document) and instead are called by Redis.
+It's still Redis in the background, but the information flow is reversed. Instead of your functions calling Redis (e.g., for caching a document), Redis governs your function execution. If your microservice container goes down or your function simply fails, HotMesh will restore function state at the point of failure and retry until it succeeds.
 
 ## Install
 [![npm version](https://badge.fury.io/js/%40hotmeshio%2Fhotmesh.svg)](https://badge.fury.io/js/%40hotmeshio%2Fhotmesh)
@@ -20,34 +17,32 @@ npm install @hotmeshio/hotmesh
 ## Design
 The HotMesh SDK is designed to keep your code front-and-center. Write code as you normally would, then use HotMesh to make it durable.
 
-1. Start with any ordinary class. Pay attention to unpredictable functions: those that execute slowly, cause problems at scale, or simply fail to return. *Note how the `saludar` function throws an error 50% of the time. This is exactly the type of function to fix.*
+1. Start with any ordinary class. Pay attention to unpredictable functions: those that execute slowly, cause problems at scale, or simply fail to return. *Note how the `flaky` function throws an error 50% of the time. This is exactly the type of function that can be fixed using HotMesh.*
     ```javascript
     //myworkflow.ts
 
     export class MyWorkflow {
 
-      //main method
       async run(name: string): Promise<string> {
-        const salud = await this.saludar(name);
+        const hi = await this.flaky(name);
         const hello = await this.greet(name);
-        return `${hello} ${salud}`;
+        return `${hi} ${hello}`;
       }
 
-      //this function only succeeds 50% of the time!
-      async saludar(nombre: string): Promise<string> {
+      //this function is unpredictable and will fail 50% of the time
+      async flaky(name: string): Promise<string> {
         if (Math.random() < 0.5) {
-          throw new Error('¡No hablo español!');
+          throw new Error('Ooops!');
         }
-        return `¡Hola, ${nombre}!`;
+        return `Hi, ${name}!`;
       }
 
-      //this function always succeeds
       async greet(name: string): Promise<string> {
         return `Hello, ${name}!`;
       }
     }
     ```
-2. Import `MeshOS` and subclass it as shown. Configure `host`, `port`, etc., and list those functions that should run durably like `run` and `saludar`. Your class is now a durable workflow! *Additional helper methods include `waitForSignal`, `signal`, `hook`, `sleep` (months, years, etc), `executeChild`, `startChild`, `get`, `set`, `incr` (increment), and `mult` (multiply).*
+2. Import `Redis` and `MeshOS` and configure host, port, etc. List those functions that Redis should govern as durable workflows (like `run` and `flaky`). And that's it! *Your functions don't actually change; rather, their governance does.*
     ```javascript
     //myworkflow.ts
 
@@ -63,40 +58,52 @@ The HotMesh SDK is designed to keep your code front-and-center. Write code as yo
       workflowFunctions = ['run'];
 
       //list functions to retry and cache
-      proxyFunctions = ['saludar'];
+      proxyFunctions = ['flaky'];
 
-      //main method (Redis will now govern its execution)
       async run(name: string): Promise<string> {
-        const salud = await this.saludar(name);
+        const hi = await this.flaky(name);
         const hello = await this.greet(name);
-        return `${hello} ${salud}`;
+        return `${hi} ${hello}`;
       }
 
-      //this function will now be retried until it succeeds;
-      async saludar(nombre: string): Promise<string> {
+      //this function is now durable and will be retried until it succeeds!
+      async flaky(name: string): Promise<string> {
         if (Math.random() < 0.5) {
-          throw new Error('¡No hablo español!');
+          throw new Error('Ooops!');
         }
-        return `¡Hola, ${nombre}!`;
+        return `Hi, ${name}!`;
       }
 
-      //this vanilla function is unchanged
       async greet(name: string): Promise<string> {
         return `Hello, ${name}!`;
       }
     }
     ```
-3. Invoke your class, providing a unique GUID (it's now a workflow and it's idempotent). Nothing changes from the outside, *but Redis now governs its end-to-end execution.* It's guaranteed to succeed and return regardless of catastrophic network failure. Redis will simply inflate like a balloon and deflate as things come back online.
+3. Invoke your class, providing a unique id (it's now an idempotent workflow and needs a GUID). Nothing changes from the outside, *but Redis now governs the end-to-end execution.* It's guaranteed to succeed, even if it takes a while. 
     ```javascript
     //mycaller.ts
 
-    //...import MyWorkflow, etc...
-
-    const workflow = new MyWorkflow('unique123', { await: true }); //await the response
-    const response = await workflow.run('John');
-    //Hello, John! ¡Hola, John!
+    const workflow = new MyWorkflow({ id: 'my123', await: true });
+    const response = await workflow.run('World');
+    //Hi, World! Hello, World!
     ```
 
+Redis governance delivers more than just reliability. Externalizing state fundamentally changes the execution profile for your functions, allowing you to design long-running, durable workflows. Use the following methods to solve the most common state management challenges.
+
+ - `waitForSignal` | Pause and wait for external event(s) before continuing. The *waitForSignal* method will collate and cache the signals and only awaken your function once they've all arrived.
+ - `signal` | Send a signal (and optional payload) to any paused function.
+ - `hook` | Redis governance supercharges your functions, transforming them into 're-entrant processes'. Optionally use the *hook* method to spawn parallel execution threads within any running function.
+ - `sleep` | Pause function execution for a ridiculous amount of time (months, years, etc). There's no risk of information loss, as Redis governs function state. When your function awakens, function state is efficiently (and automatically) restored.
+ - `random` | Generate a deterministic random number that can be used in a reentrant process workflow (replaces `Math.random()`).
+ - `executeChild` | Call another durable function and await the response. *Design sophisticated, multi-process solutions by leveraging this command.*
+ - `startChild` | Call another durable function, but do not await the response.
+ - `set` | Set a value (e.g, `set('name', 'value')`)
+ - `get` | Get a value (e.g, `get('name')`)
+ - `incr` | Increment (or decrement) a number (e.g, `incr('name', -99)`)
+ - `mult` | Multiply (or divide) a number (e.g, `mult('name', 12)`)
+
+Refer to the [hotmeshio/samples-typescript](https://github.com/hotmeshio/samples-typescript) repo for usage examples. 
+
 ## Advanced Design
 HotMesh's TypeScript SDK is the easiest way to make your functions durable. But if you need full control over your function lifecycles (including high-volume, high-speed use cases), you can use HotMesh's underlying YAML models to optimize your durable workflows. The following model depicts a sequence of activities orchestrated by HotMesh. Any function you associate with a `topic` in your YAML definition is guaranteed to be durable.
 
@@ -232,4 +239,4 @@ HotMesh is a distributed orchestration engine. Refer to the [Distributed Orchest
 Gain insight into HotMesh's monitoring, exception handling, and alarm configurations via the [System Lifecycle Guide](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/system_lifecycle.md).
 
 ## Alpha Release
-So what exacty is an [alpha release](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/alpha.md)?!
+So what exacty is an [alpha release](https://github.com/hotmeshio/sdk-typescript/blob/main/docs/alpha.md)?

package/build/modules/utils.d.ts

@@ -5,6 +5,7 @@ import { RedisClient, RedisMulti } from "../types/redis";
 import { StringAnyType } from "../types/serializer";
 import { StreamCode, StreamStatus } from "../types/stream";
 export declare function sleepFor(ms: number): Promise<unknown>;
+export declare function deterministicRandom(seed: number): number;
 export declare function identifyRedisType(redisInstance: any): 'redis' | 'ioredis' | null;
 export declare const polyfill: {
     resolveActivityType(activityType: string): string;

package/build/modules/utils.js

@@ -1,10 +1,15 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.restoreHierarchy = exports.getValueByPath = exports.getIndexedHash = exports.getSymVal = exports.getSymKey = exports.formatISODate = exports.getTimeSeries = exports.getSubscriptionTopic = exports.findSubscriptionForTrigger = exports.findTopKey = exports.XSleepFor = exports.matchesStatus = exports.matchesStatusCode = exports.identifyRedisTypeFromClass = exports.polyfill = exports.identifyRedisType = exports.sleepFor = void 0;
+exports.restoreHierarchy = exports.getValueByPath = exports.getIndexedHash = exports.getSymVal = exports.getSymKey = exports.formatISODate = exports.getTimeSeries = exports.getSubscriptionTopic = exports.findSubscriptionForTrigger = exports.findTopKey = exports.XSleepFor = exports.matchesStatus = exports.matchesStatusCode = exports.identifyRedisTypeFromClass = exports.polyfill = exports.identifyRedisType = exports.deterministicRandom = exports.sleepFor = void 0;
 async function sleepFor(ms) {
     return new Promise((resolve) => setTimeout(resolve, ms));
 }
 exports.sleepFor = sleepFor;
+function deterministicRandom(seed) {
+    let x = Math.sin(seed) * 10000;
+    return x - Math.floor(x);
+}
+exports.deterministicRandom = deterministicRandom;
 function identifyRedisType(redisInstance) {
     const prototype = Object.getPrototypeOf(redisInstance);
     if ('defineCommand' in prototype || Object.keys(prototype).includes('multi')) {

package/build/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@hotmeshio/hotmesh",
-    "version": "0.0.24",
+    "version": "0.0.26",
     "description": "Unbreakable Workflows",
     "main": "build/index.js",
     "types": "build/index.d.ts",

package/build/services/durable/meshos.d.ts

@@ -1,12 +1,12 @@
 import { WorkflowHandleService } from './handle';
-import { FindOptions, MeshOSActivityOptions, MeshOSOptions, WorkflowSearchOptions } from '../../types/durable';
-import { RedisOptions, RedisClass } from '../../types/redis';
-import { StringAnyType } from '../../types';
 import { WorkflowService } from './workflow';
+import { FindOptions, FindWhereOptions, FindWhereQuery, MeshOSActivityOptions, MeshOSConfig, MeshOSOptions, MeshOSWorkerOptions, WorkflowSearchOptions } from '../../types/durable';
+import { RedisOptions, RedisClass } from '../../types/redis';
+import { StringAnyType } from '../../types/serializer';
 /**
  * The base class for running MeshOS workflows.
- * Extend and register subclass methods by name to
- * execute as durable workflows, backed by Redis.
+ * Extend this class, add your Redis config, and add functions to
+ * execute as durable `hooks`, `workflows`, and `activities`.
  */
 export declare class MeshOSService {
     /**
@@ -78,14 +78,18 @@ export declare class MeshOSService {
      */
     static createIndex(): Promise<void>;
     /**
-     * Initialize the worker(s) for the entity. This is a static
+     * stop the workers
+     * @returns {Promise<void>}
+     */
+    static stopWorkers(): Promise<void>;
+    /**
+     * Initializes the worker(s). This is a static
      * method that allows for optional task Queue targeting.
-     * NOTE: Allow List may be optionally used to only wrap
-     *       specific methods in this class.
-     * @param {string} taskQueue
-     * @param {string[]} allowList
+     * An `allowList` may be optionally provided to start
+     * specific `worker` and `hook` methods.
+     * @param {MeshOSWorkerOptions} [options]
      */
-    static startWorkers(taskQueue?: string, allowList?: Array<MeshOSOptions | string>): Promise<void>;
+    static startWorkers(options?: MeshOSWorkerOptions): Promise<void>;
     /**
      * executes the redis FT search query
      * @example '@_quantity:[89 89]'
@@ -93,6 +97,14 @@ export declare class MeshOSService {
      * @returns {string}
      */
     static find(options: FindOptions, ...args: string[]): Promise<string[] | [number]>;
+    /**
+     * Provides a JSON abstraction for the Redis FT.search command
+     * (e.g, `count`, `query`, `return`, `limit`)
+     * @param {FindWhereOptions} options
+     * @returns {Promise<string[] | [number]>}
+     */
+    static findWhere(options: FindWhereOptions): Promise<string[] | [number]>;
+    static generateSearchQuery(query: FindWhereQuery[]): string;
     /**
      * returns the workflow handle. The handle can then be
      * used to query for status, state, custom state, etc.
@@ -104,5 +116,5 @@ export declare class MeshOSService {
      * Optionally include a target taskQueue to exec the
      * workflow's call on a specific worker queue.
      */
-    constructor(id?: string, options?: Record<string, any>);
+    constructor(id?: string | MeshOSConfig, options?: MeshOSConfig);
 }

package/build/services/durable/meshos.js

@@ -2,16 +2,17 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.MeshOSService = void 0;
 const nanoid_1 = require("nanoid");
+const _1 = require(".");
+const asyncLocalStorage_1 = require("./asyncLocalStorage");
 const client_1 = require("./client");
 const search_1 = require("./search");
 const worker_1 = require("./worker");
-const _1 = require(".");
-const asyncLocalStorage_1 = require("./asyncLocalStorage");
 const workflow_1 = require("./workflow");
+const stream_1 = require("../signaler/stream");
 /**
  * The base class for running MeshOS workflows.
- * Extend and register subclass methods by name to
- * execute as durable workflows, backed by Redis.
+ * Extend this class, add your Redis config, and add functions to
+ * execute as durable `hooks`, `workflows`, and `activities`.
  */
 class MeshOSService {
     static async getHotMeshClient(redisClass, redisOptions, namespace, taskQueue) {
@@ -42,14 +43,24 @@ class MeshOSService {
         search_1.Search.configureSearchIndex(hmClient, my.search);
     }
     /**
-     * Initialize the worker(s) for the entity. This is a static
+     * stop the workers
+     * @returns {Promise<void>}
+     */
+    static async stopWorkers() {
+        await _1.Durable.Client.shutdown();
+        await _1.Durable.Worker.shutdown();
+        await stream_1.StreamSignaler.stopConsuming();
+    }
+    /**
+     * Initializes the worker(s). This is a static
      * method that allows for optional task Queue targeting.
-     * NOTE: Allow List may be optionally used to only wrap
-     *       specific methods in this class.
-     * @param {string} taskQueue
-     * @param {string[]} allowList
+     * An `allowList` may be optionally provided to start
+     * specific `worker` and `hook` methods.
+     * @param {MeshOSWorkerOptions} [options]
      */
-    static async startWorkers(taskQueue, allowList = []) {
+    static async startWorkers(options) {
+        const taskQueue = options && options.taskQueue;
+        const allowList = options && options.allowList || [];
         const my = new this();
         //helper functions
         const resolveFunctionNames = (arr) => arr.map(item => typeof item === 'string' ? item : item.name);
@@ -74,8 +85,6 @@ class MeshOSService {
             const proxiedActivities = _1.Durable.workflow.proxyActivities({
                 activities: proxyActivities
             });
-            //WATCH!: unsure if this will pollute the scope; don't think
-            //      so as activity functions are terminal in the chain.
             Object.assign(my, proxiedActivities);
         }
         const functionsToIterate = allowList.length ? resolveFunctionNames(allowList) : resolveFunctionNames([...my.workflowFunctions, ...my.hookFunctions]);
@@ -142,9 +151,8 @@ class MeshOSService {
                 class: my.redisClass,
                 options: my.redisOptions
             } });
-        //workflow name is the function name driving the workflow
         let workflowName;
-        if (options?.workflowName) {
+        if (options.workflowName) {
             workflowName = options?.workflowName;
         }
         else if (my.workflowFunctions?.length) {
@@ -156,7 +164,66 @@ class MeshOSService {
                 workflowName = target.name;
             }
         }
-        return await client.workflow.search(options?.taskQueue ?? my.taskQueue, workflowName, my.namespace, my.search.index, ...args); //[count, [id, fields[], id, fields[], id, fields[], ...]]
+        return await client.workflow.search(options.taskQueue ?? my.taskQueue, workflowName, options.namespace ?? my.namespace, options.index ?? my.search.index, ...args); //[count, [id, fields[]], [id, fields[]], [id, fields[]], ...]]
+    }
+    /**
+     * Provides a JSON abstraction for the Redis FT.search command
+     * (e.g, `count`, `query`, `return`, `limit`)
+     * @param {FindWhereOptions} options
+     * @returns {Promise<string[] | [number]>}
+     */
+    static async findWhere(options) {
+        const args = [this.generateSearchQuery(options.query)];
+        if (options.count) {
+            args.push('LIMIT', '0', '0');
+        }
+        else {
+            //limit which hash fields to return
+            if (options.return?.length) {
+                args.push('RETURN');
+                args.push(options.return.length.toString());
+                options.return.forEach(returnField => {
+                    args.push(`_${returnField}`);
+                });
+            }
+            //paginate
+            if (options.limit) {
+                args.push('LIMIT', options.limit.start.toString(), options.limit.size.toString());
+            }
+        }
+        return await this.find(options.options ?? {}, ...args);
+    }
+    static generateSearchQuery(query) {
+        const my = new this();
+        let queryString = query.map(q => {
+            const { field, is, value, type } = q;
+            const prefixedFieldName = my.search?.schema && field in my.search.schema ? `@_${field}` : `@${field}`;
+            const fieldType = my.search?.schema[field]?.type ?? type ?? 'TEXT';
+            switch (fieldType) {
+                case 'TAG':
+                    return `${prefixedFieldName}:{${value}}`;
+                case 'TEXT':
+                    return `${prefixedFieldName}:"${value}"`;
+                case 'NUMERIC':
+                    let range = '';
+                    if (is.startsWith('=')) {
+                        range = `[${value} ${value}]`;
+                    }
+                    else if (is === '<') {
+                        range = `[-inf ${value}]`;
+                    }
+                    else if (is === '>') {
+                        range = `[${value} +inf]`;
+                    }
+                    else if (is === '[]') {
+                        range = `[${value[0]} ${value[1]}]`;
+                    }
+                    return `${prefixedFieldName}:${range}`;
+                default:
+                    return '';
+            }
+        }).join(' ');
+        return queryString;
     }
     /**
      * returns the workflow handle. The handle can then be
@@ -224,7 +291,15 @@ class MeshOSService {
             password: '',
             db: 0,
         };
-        this.id = id;
+        if (typeof (id) === 'string') {
+            this.id = id;
+        }
+        else if (id?.id) {
+            this.id = id.id;
+            options = id;
+            id = undefined;
+        }
+        ;
         if (options?.taskQueue) {
             this.taskQueue = options.taskQueue;
         }

package/build/services/durable/workflow.d.ts

@@ -28,6 +28,13 @@ export declare class WorkflowService {
      * process state and job state)
      */
     static isSideEffectAllowed(hotMeshClient: HotMesh, prefix: string): Promise<boolean>;
+    /**
+     * returns a random number between 0 and 1. This number is deterministic
+     * and will never vary for a given seed. This is useful for randomizing
+     * pathways in a workflow that can be safely replayed.
+     * @returns {number}
+     */
+    static random(): number;
     /**
      * send signal data into any other paused thread (which is paused and
      * awaiting the signal) from within a hook-thread or the main-thread

package/build/services/durable/workflow.js

@@ -14,6 +14,7 @@ const factory_1 = require("./factory");
 const search_1 = require("./search");
 const worker_1 = require("./worker");
 const stream_1 = require("../../types/stream");
+const utils_1 = require("../../modules/utils");
 class WorkflowService {
     /**
      * Spawn a child workflow. await and return the result.
@@ -151,6 +152,18 @@ class WorkflowService {
         const guidValue = Number(await hotMeshClient.engine.store.exec('HINCRBYFLOAT', workflowGuid, sessionId, '1'));
         return guidValue === 1;
     }
+    /**
+     * returns a random number between 0 and 1. This number is deterministic
+     * and will never vary for a given seed. This is useful for randomizing
+     * pathways in a workflow that can be safely replayed.
+     * @returns {number}
+     */
+    static random() {
+        const store = asyncLocalStorage_1.asyncLocalStorage.getStore();
+        const COUNTER = store.get('counter');
+        const seed = COUNTER.counter = COUNTER.counter + 1;
+        return (0, utils_1.deterministicRandom)(seed);
+    }
     /**
      * send signal data into any other paused thread (which is paused and
      * awaiting the signal) from within a hook-thread or the main-thread

package/build/services/store/clients/redis.js

@@ -33,8 +33,7 @@ class RedisStoreService extends index_1.StoreService {
         };
     }
     getMulti() {
-        const multi = this.redisClient.MULTI();
-        return multi;
+        return this.redisClient.multi();
     }
     async exec(...args) {
         return await this.redisClient.sendCommand(args);

package/build/services/stream/clients/redis.js

@@ -13,7 +13,7 @@ class RedisStreamService extends index_1.StreamService {
         this.appId = appId;
     }
     getMulti() {
-        return this.redisClient.MULTI();
+        return this.redisClient.multi();
     }
     mintKey(type, params) {
         if (!this.namespace)

package/build/services/sub/clients/redis.js

@@ -13,7 +13,7 @@ class RedisSubService extends index_1.SubService {
         this.appId = appId;
     }
     getMulti() {
-        const multi = this.redisClient.MULTI();
+        const multi = this.redisClient.multi();
         return multi;
     }
     mintKey(type, params) {

package/build/types/durable.d.ts

@@ -59,15 +59,9 @@ type MeshOSClassConfig = {
     redisClass: RedisClass;
 };
 type MeshOSConfig = {
+    id?: string;
+    await?: boolean;
     taskQueue?: string;
-    index?: {
-        index: string;
-        prefix: string[];
-        schema: Record<string, {
-            type: 'TEXT' | 'NUMERIC' | 'TAG';
-            sortable: boolean;
-        }>;
-    };
 };
 type ConnectionConfig = {
     class: RedisClass;
@@ -88,12 +82,28 @@ type WorkerConfig = {
     options?: WorkerOptions;
     search?: WorkflowSearchOptions;
 };
+type FindWhereQuery = {
+    field: string;
+    is: string;
+    value: string | boolean | number | [number, number];
+    type?: string;
+};
 type FindOptions = {
     workflowName?: string;
     taskQueue?: string;
     namespace?: string;
     index?: string;
 };
+type FindWhereOptions = {
+    options?: FindOptions;
+    count?: boolean;
+    query: FindWhereQuery[];
+    return?: string[];
+    limit?: {
+        start: number;
+        size: number;
+    };
+};
 type MeshOSOptions = {
     name: string;
     options: WorkerOptions;
@@ -102,6 +112,13 @@ type MeshOSActivityOptions = {
     name: string;
     options: ActivityConfig;
 };
+type MeshOSWorkerOptions = {
+    taskQueue?: string;
+    allowList?: Array<MeshOSOptions | string>;
+    logLevel?: string;
+    maxSystemRetries?: number;
+    backoffCoefficient?: number;
+};
 type WorkerOptions = {
     logLevel?: string;
     maxSystemRetries?: number;
@@ -125,4 +142,4 @@ type ActivityConfig = {
         maximumInterval: string;
     };
 };
-export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, HookOptions, MeshOSActivityOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkerConfig, WorkflowConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, };
+export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, FindWhereOptions, FindWhereQuery, HookOptions, MeshOSActivityOptions, MeshOSWorkerOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkerConfig, WorkflowConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, };

package/build/types/index.d.ts

@@ -3,7 +3,7 @@ export { App, AppVID, AppTransitions, AppSubscriptions } from './app';
 export { AsyncSignal } from './async';
 export { CacheMode } from './cache';
 export { CollationFaultType, CollationStage } from './collator';
-export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, HookOptions, MeshOSActivityOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkflowConfig, WorkerConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, } from './durable';
+export { ActivityConfig, ActivityWorkflowDataType, ClientConfig, ContextType, ConnectionConfig, Connection, ProxyType, Registry, SignalOptions, FindOptions, FindWhereOptions, FindWhereQuery, HookOptions, MeshOSActivityOptions, MeshOSWorkerOptions, MeshOSClassConfig, MeshOSConfig, MeshOSOptions, WorkflowConfig, WorkerConfig, WorkerOptions, WorkflowSearchOptions, WorkflowDataType, WorkflowOptions, } from './durable';
 export { HookCondition, HookConditions, HookGate, HookInterface, HookRule, HookRules, HookSignal } from './hook';
 export { RedisClientType as IORedisClientType, RedisMultiType as IORedisMultiType } from './ioredisclient';
 export { ILogger } from './logger';

package/modules/utils.ts

@@ -8,6 +8,11 @@ export async function sleepFor(ms: number) {
   return new Promise((resolve) => setTimeout(resolve, ms));
 }
 
+export function deterministicRandom(seed: number): number {
+  let x = Math.sin(seed) * 10000;
+  return x - Math.floor(x);
+}
+
 export function identifyRedisType(redisInstance: any): 'redis' | 'ioredis' | null {
   const prototype = Object.getPrototypeOf(redisInstance);
   if ('defineCommand' in prototype || Object.keys(prototype).includes('multi')) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hotmeshio/hotmesh",
-  "version": "0.0.24",
+  "version": "0.0.26",
   "description": "Unbreakable Workflows",
   "main": "build/index.js",
   "types": "build/index.d.ts",

package/services/durable/meshos.ts

@@ -1,20 +1,29 @@
 import { nanoid } from 'nanoid';
 
+import { Durable } from '.';
+import { asyncLocalStorage } from './asyncLocalStorage';
 import { ClientService as Client } from './client';
 import { WorkflowHandleService } from './handle';
 import { Search } from './search';
 import { WorkerService as Worker } from './worker';
-import { FindOptions, MeshOSActivityOptions, MeshOSOptions, WorkflowSearchOptions } from '../../types/durable';
-import { RedisOptions, RedisClass } from '../../types/redis';
-import { StringAnyType } from '../../types';
-import { Durable } from '.';
-import { asyncLocalStorage } from './asyncLocalStorage';
 import { WorkflowService } from './workflow';
+import { StreamSignaler } from '../signaler/stream';
+import {
+  FindOptions,
+  FindWhereOptions,
+  FindWhereQuery,
+  MeshOSActivityOptions,
+  MeshOSConfig,
+  MeshOSOptions,
+  MeshOSWorkerOptions,
+  WorkflowSearchOptions } from '../../types/durable';
+import { RedisOptions, RedisClass } from '../../types/redis';
+import { StringAnyType } from '../../types/serializer';
 
 /**
  * The base class for running MeshOS workflows.
- * Extend and register subclass methods by name to
- * execute as durable workflows, backed by Redis.
+ * Extend this class, add your Redis config, and add functions to
+ * execute as durable `hooks`, `workflows`, and `activities`.
  */
 
 export class MeshOSService {
@@ -122,14 +131,25 @@ export class MeshOSService {
   }
 
   /**
-   * Initialize the worker(s) for the entity. This is a static
+   * stop the workers
+   * @returns {Promise<void>}
+   */
+  static async stopWorkers(): Promise<void> {
+    await Durable.Client.shutdown();
+    await Durable.Worker.shutdown();
+    await StreamSignaler.stopConsuming();
+  }
+
+  /**
+   * Initializes the worker(s). This is a static
    * method that allows for optional task Queue targeting.
-   * NOTE: Allow List may be optionally used to only wrap
-   *       specific methods in this class. 
-   * @param {string} taskQueue 
-   * @param {string[]} allowList 
+   * An `allowList` may be optionally provided to start
+   * specific `worker` and `hook` methods. 
+   * @param {MeshOSWorkerOptions} [options]
    */
-  static async startWorkers(taskQueue?: string, allowList: Array<MeshOSOptions | string> = []) {
+  static async startWorkers(options?: MeshOSWorkerOptions) {
+    const taskQueue = options && options.taskQueue;
+    const allowList = options && options.allowList || [];
     const my = new this();
 
     //helper functions
@@ -156,8 +176,6 @@ export class MeshOSService {
       const proxiedActivities = Durable.workflow.proxyActivities({
         activities: proxyActivities
       });
-      //WATCH!: unsure if this will pollute the scope; don't think
-      //      so as activity functions are terminal in the chain.
       Object.assign(my, proxiedActivities);
     }
 
@@ -229,9 +247,9 @@ export class MeshOSService {
       class: my.redisClass,
       options: my.redisOptions 
     }});
-    //workflow name is the function name driving the workflow
+
     let workflowName: string;
-    if (options?.workflowName) {
+    if (options.workflowName) {
       workflowName = options?.workflowName
     } else if(my.workflowFunctions?.length) {
       let target = my.workflowFunctions[0];
@@ -242,12 +260,70 @@ export class MeshOSService {
       }
     }
     return await client.workflow.search(
-      options?.taskQueue ?? my.taskQueue,
+      options.taskQueue ?? my.taskQueue,
       workflowName,
-      my.namespace,
-      my.search.index,
+      options.namespace ?? my.namespace,
+      options.index ?? my.search.index,
       ...args,
-    ); //[count, [id, fields[], id, fields[], id, fields[], ...]]
+    ); //[count, [id, fields[]], [id, fields[]], [id, fields[]], ...]]
+  }
+
+  /**
+   * Provides a JSON abstraction for the Redis FT.search command
+   * (e.g, `count`, `query`, `return`, `limit`)
+   * @param {FindWhereOptions} options 
+   * @returns {Promise<string[] | [number]>}
+   */
+  static async findWhere(options: FindWhereOptions): Promise<string[] | [number]> {
+    const args: string[] = [this.generateSearchQuery(options.query)];
+    if (options.count) {
+      args.push('LIMIT', '0', '0');
+    } else {
+      //limit which hash fields to return
+      if (options.return?.length) {
+        args.push('RETURN');
+        args.push(options.return.length.toString());
+        options.return.forEach(returnField => {
+          args.push(`_${returnField}`);
+        });
+      }
+      //paginate
+      if (options.limit) {
+        args.push('LIMIT', options.limit.start.toString(), options.limit.size.toString());
+      }
+    } 
+    return await this.find(options.options ?? {}, ...args);
+  }
+
+  static generateSearchQuery(query: FindWhereQuery[]) {
+    const my = new this();
+    let queryString = query.map(q => {
+      const { field, is, value, type } = q;
+      const prefixedFieldName = my.search?.schema && field in my.search.schema ? `@_${field}` : `@${field}`;
+      const fieldType = my.search?.schema[field]?.type ?? type ?? 'TEXT';
+
+      switch (fieldType) {
+        case 'TAG':
+          return `${prefixedFieldName}:{${value}}`;
+        case 'TEXT':
+          return `${prefixedFieldName}:"${value}"`;
+        case 'NUMERIC':
+          let range = '';
+          if (is.startsWith('=')) {
+            range = `[${value} ${value}]`;
+          } else if (is === '<') {
+            range = `[-inf ${value}]`;
+          } else if (is === '>') {
+            range = `[${value} +inf]`;
+          } else if (is === '[]') {
+            range = `[${value[0]} ${value[1]}]`
+          }
+          return `${prefixedFieldName}:${range}`;
+        default:
+          return '';
+      }
+    }).join(' ');
+    return queryString;
   }
 
   /**
@@ -281,8 +357,14 @@ export class MeshOSService {
    * Optionally include a target taskQueue to exec the
    * workflow's call on a specific worker queue.
    */
-  constructor(id?: string, options?: Record<string, any>) {
-    this.id = id;
+  constructor(id?: string | MeshOSConfig, options?: MeshOSConfig) {
+    if (typeof(id) === 'string') {
+      this.id = id;
+    } else if (id?.id) {
+      this.id = id.id;
+      options = id;
+      id = undefined;
+    };
     if (options?.taskQueue) {
       this.taskQueue = options.taskQueue;
     } else if (!id && !options?.taskQueue) {

package/services/durable/workflow.ts

@@ -19,6 +19,7 @@ import {
   WorkflowOptions } from "../../types/durable";
 import { JobOutput, JobState } from '../../types/job';
 import { StreamStatus } from '../../types/stream';
+import { deterministicRandom } from '../../modules/utils';
 
 export class WorkflowService {
 
@@ -173,6 +174,19 @@ export class WorkflowService {
     return guidValue === 1;
   }
 
+  /**
+   * returns a random number between 0 and 1. This number is deterministic
+   * and will never vary for a given seed. This is useful for randomizing
+   * pathways in a workflow that can be safely replayed.
+   * @returns {number}
+   */
+  static random(): number {
+    const store = asyncLocalStorage.getStore();
+    const COUNTER = store.get('counter');
+    const seed = COUNTER.counter = COUNTER.counter + 1;
+    return deterministicRandom(seed);
+  }
+
   /**
    * send signal data into any other paused thread (which is paused and
    * awaiting the signal) from within a hook-thread or the main-thread

package/services/store/clients/redis.ts

@@ -46,8 +46,7 @@ class RedisStoreService extends StoreService<RedisClientType, RedisMultiType> {
   }
 
   getMulti(): RedisMultiType {
-    const multi = this.redisClient.MULTI();
-    return multi as unknown as RedisMultiType;
+    return this.redisClient.multi() as unknown as RedisMultiType;
   }
 
   async exec(...args: any[]): Promise<string|string[]|string[][]> {

package/services/stream/clients/redis.ts

@@ -21,7 +21,7 @@ class RedisStreamService extends StreamService<RedisClientType, RedisMultiType>
   }
 
   getMulti(): RedisMultiType {
-    return this.redisClient.MULTI() as unknown as RedisMultiType;
+    return this.redisClient.multi() as unknown as RedisMultiType;
   }
 
   mintKey(type: KeyType, params: KeyStoreParams): string {

package/services/sub/clients/redis.ts

@@ -21,7 +21,7 @@ class RedisSubService extends SubService<RedisClientType, RedisMultiType> {
   }
 
   getMulti(): RedisMultiType {
-    const multi = this.redisClient.MULTI();
+    const multi = this.redisClient.multi();
     return multi as unknown as RedisMultiType;
   }
 

package/types/durable.ts

@@ -65,12 +65,9 @@ type MeshOSClassConfig = {
 }
 
 type MeshOSConfig = {
-  taskQueue?: string;
-  index?: {
-    index: string;
-    prefix: string[];
-    schema: Record<string, {type: 'TEXT' | 'NUMERIC' | 'TAG', sortable: boolean}>;
-  };
+  id?: string; //guid for the workflow when instancing
+  await?: boolean; //default is false; must explicitly send true to await the final result
+  taskQueue?: string; //optional target queue isolate for the function
 }
 
 type ConnectionConfig = {
@@ -96,6 +93,13 @@ type WorkerConfig = {
   search?: WorkflowSearchOptions;
 }
 
+type FindWhereQuery = {
+  field: string;
+  is: string;
+  value: string | boolean | number | [number, number];
+  type?: string; //default is TEXT
+}
+
 type FindOptions = {
   workflowName?: string; //also the function name
   taskQueue?: string;
@@ -103,6 +107,17 @@ type FindOptions = {
   index?: string;        //the FT search index name
 }
 
+type FindWhereOptions = {
+  options?: FindOptions;
+  count?: boolean;
+  query: FindWhereQuery[];
+  return?: string[];
+  limit?: {
+    start: number,
+    size: number
+  }
+}
+
 type MeshOSOptions = {
   name: string;
   options: WorkerOptions;
@@ -113,6 +128,14 @@ type MeshOSActivityOptions = {
   options: ActivityConfig;
 }
 
+type MeshOSWorkerOptions = {
+  taskQueue?: string; //change the default task queue
+  allowList?: Array<MeshOSOptions | string>; //limit which `hook` and `workflow` workers start
+  logLevel?: string; //debug, info, warn, error
+  maxSystemRetries?: number; //1-3 (10ms, 100ms, 1_000ms)
+  backoffCoefficient?: number; //2-10ish
+}
+
 type WorkerOptions = {
   logLevel?: string; //debug, info, warn, error
   maxSystemRetries?: number; //1-3 (10ms, 100ms, 1_000ms)
@@ -151,8 +174,11 @@ export {
   Registry,
   SignalOptions,
   FindOptions,
+  FindWhereOptions,
+  FindWhereQuery,
   HookOptions,
   MeshOSActivityOptions,
+  MeshOSWorkerOptions,
   MeshOSClassConfig,
   MeshOSConfig,
   MeshOSOptions,

package/types/index.ts

@@ -38,8 +38,11 @@ export {
     Registry,
     SignalOptions,
     FindOptions,
+    FindWhereOptions,
+    FindWhereQuery,
     HookOptions,
     MeshOSActivityOptions,
+    MeshOSWorkerOptions,
     MeshOSClassConfig,
     MeshOSConfig,
     MeshOSOptions,
@@ -49,7 +52,7 @@ export {
     WorkflowSearchOptions,
     WorkflowDataType,
     WorkflowOptions,
-  }from './durable'
+  } from './durable'
 export {
   HookCondition,
   HookConditions,