@bobtail.software/b-durable 1.0.5 → 1.0.6

package/README.md

@@ -1,232 +1,207 @@
-
-# `@bobtail.software/b-durable`: Composable, Type-Safe, Durable Workflows for TypeScript
+# `b-durable`: Composable, Type-Safe, Durable Workflows for TypeScript
 
 ![NPM Version](https://img.shields.io/npm/v/@bobtail.software/b-durable.svg)
 ![License](https://img.shields.io/npm/l/@bobtail.software/eslint-plugin-b-durable.svg)
 
-`b-durable` is a powerful system that transforms standard `async` functions into **composable, interactive, durable, and resilient workflows**. It lets you write long-running business logic—spanning hours, days, or months—as simple, linear `async/await` code. The system handles state persistence, orchestration, external events, and crash recovery, allowing you to focus on your business logic.
+`b-durable` is a production-ready system that transforms standard `async` functions into **composable, interactive, durable, and resilient workflows**. It lets you write long-running business logic—spanning hours, days, or months—as simple, linear `async/await` code. The system handles state persistence, orchestration, crash recovery, strict versioning, and observability.
 
 ## The Problem
 
 Standard `async/await` is great for short-lived operations, but it breaks down for complex, long-running processes:
 
 1.  **Fragility**: If your server restarts mid-execution, all in-memory state is lost.
-2.  **Inefficiency**: An operation like `await bSleep('7 days')` is impossible. It would hold a process hostage, consume resources, and wouldn't survive a single deployment.
-3.  **Orchestration Complexity**: Coordinating processes that involve multiple services, human-in-the-loop steps (like approvals), or external system webhooks often leads to a tangled mess of state machines, queues, and database flags.
+2.  **Inefficiency**: An operation like `await bSleep('7 days')` is impossible standard Node.js.
+3.  **Operational Blindness**: It's hard to know the state of a multi-step process running across distributed services.
+4.  **Deployment Risks**: Deploying new code while old processes are running can corrupt memory/state.
 
 ## The `b-durable` Solution
 
-`b-durable` allows you to express this complexity as a single, readable `async` function. The system automatically persists the workflow's state after each `await` step, ensuring it can resume from the exact point of interruption.
+`b-durable` allows you to express this complexity as a single, readable `async` function. The system automatically persists the workflow's state after each `await` step in Redis.
 
-Imagine orchestrating an e-commerce order. With `b-durable`, the code is as clear as the business process itself:
+### Key Capabilities
 
-```typescript
-// Define a reusable sub-workflow for handling payments
-export const paymentWorkflow = bDurable({
-  workflow: async (input: { orderId: string; amount: number }, context) => {
-    // 1. Call a service to process the payment
-    const result = await processPayment({ orderId: input.orderId, amount: input.amount });
-    if (!result.success) {
-      throw new Error('Payment failed!');
-    }
-    // 2. Pause durably for fraud checks
-    await context.bSleep('30m');
-    return { transactionId: result.transactionId };
-  },
-});
+-   **🛡️ Strict Versioning**: Prevents state corruption by ensuring running workflows only execute code compatible with their version.
+-   **💀 The Reaper (Reliability)**: Automatically detects crashed workers and recovers "lost" tasks, ensuring zero data loss.
+-   **👁️ Observability First**: Injectable Loggers, `getState` inspection API, and detailed tracking.
+-   **♻️ Dead Letter Queue (DLQ)**: Automatic retries for failed tasks; moves persistently failing tasks to a DLQ for manual inspection.
+-   **🛑 Cancellation**: Gracefully cancel running workflows with support for cleanup logic (`try/catch/finally`).
+-   **🧹 Auto-Retention**: Automatically expire completed/failed workflows from Redis to manage storage costs.
+
+## Example: E-Commerce Order
 
-// Define the main order processing workflow
-export const orderProcessingWorkflow = bDurable({
-  workflow: async (input: { orderId: string; items: Item[] }, context) => {
+```typescript
+import { bDurable } from '@bobtail.software/b-durable';
+
+// Define contracts
+interface OrderEvents { 'order.approved': { approverId: string }; }
+interface OrderSignals { 'status.update': { status: string }; }
+
+export const orderProcessingWorkflow = bDurable<
+  { orderId: string; amount: number },
+  { status: 'completed' | 'failed' },
+  OrderEvents,
+  OrderSignals
+>({
+  // VERSIONING IS MANDATORY
+  version: '1.0', 
+  workflow: async (input, context) => {
     try {
-      // 1. Call another workflow and await its result
-      const payment = await context.bExecute(paymentWorkflow, { orderId: input.orderId, amount: 99.99 });
-      context.log(`Payment successful: ${payment.transactionId}`);
+      // 1. Execute sub-workflow
+      const payment = await context.bExecute(paymentWorkflow, { amount: input.amount });
+      
+      // 2. Emit non-blocking signal
+      await context.bSignal('status.update', { status: 'paid' });
       
-      // 2. Pause and wait for an external event (e.g., from a UI or webhook)
+      // 3. Wait for external event (human approval)
       const approval = await context.bWaitForEvent('order.approved');
       context.log(`Order approved by ${approval.approverId}`);
 
-      // 3. Call the final service function
+      // 4. Schedule shipping
       await shipOrder(input.orderId);
-
+      
       return { status: 'completed' };
     } catch (error) {
-      // 4. Handle errors durably
-      await notifyCustomerOfFailure(input.orderId, error.message);
-      await cancelOrder(input.orderId);
-      return { status: 'failed', reason: error.message };
+       // 5. Handle errors (and cancellations!) durably
+       if (error.isCancellation) {
+           await releaseInventory(input.orderId); // Cleanup
+           throw error;
+       }
+       await notifyFailure(input.orderId);
+       return { status: 'failed' };
     }
   },
 });
 ```
 
-## Core Features
-
--   **Composable Orchestration**: Workflows can call other workflows using `await context.bExecute()`, allowing you to build complex processes from smaller, reusable parts. Results and errors are propagated automatically.
--   **Interactive & Event-Driven**: Pause a workflow indefinitely with `await context.bWaitForEvent()` until an external event is received, enabling human-in-the-loop patterns and webhook integrations.
--   **Durable & Resilient**: Workflows survive server restarts, crashes, and deployments, resuming exactly where they left off.
--   **Built-in Error Handling**: Use standard `try/catch` blocks to handle errors from tasks or sub-workflows. Your `catch` block will execute reliably, even if the failure occurs hours after the `try` block started.
--   **Durable Timers**: Use `await context.bSleep('30 days')` to pause workflows for extended periods without consuming server resources.
--   **Type Safety End-to-End**: Leverages TypeScript for type safety across steps, I/O, events, and workflow composition.
--   **Compiler-Powered**: A smart CLI compiler transforms your workflows into a step-by-step executable format, preserving types and ensuring runtime correctness.
-
-## How It Works: Compiler + Runtime
-
-1.  **The Smart Compiler (`b-durable-compiler`)**:
-    Analyzes your workflow files (`*.workflow.ts`). For each function wrapped in `bDurable(...)`, it:
-    -   **Maps Control Flow**: Breaks the function into steps at each `await`, analyzing `if/else` and `try/catch` blocks to build a complete state machine.
-    -   **Identifies Durable Calls**: Differentiates between a durable instruction (`context.bSleep`, `context.bExecute`) and a standard service task call.
-    -   **Generates Durable Artifacts**: Produces compiled `.mts` files that the runtime can execute step-by-step.
-
-2.  **The Durable Runtime**:
-    The engine that executes the compiled workflows.
-    -   **State Persistence**: Uses Redis to store the state, step, and context of every workflow instance.
-    -   **Orchestration Logic**: Manages parent/child workflow relationships, passing results and errors up the chain.
-    -   **Event System**: Tracks which workflows are waiting for which events.
-    -   **Task Queue & Scheduler**: Reliably executes service function calls and manages long-running timers.
-
 ## Getting Started
 
 ### 1. Installation
 
-Install the core library and its peer dependencies. We also highly recommend the ESLint plugin for the best developer experience.
-
 ```bash
-pnpm add @bobtail.software/b-durable ioredis
+pnpm add @bobtail.software/b-durable ioredis ms
 pnpm add -D @bobtail.software/eslint-plugin-b-durable
 ```
 
-### 2. Set Up ESLint (Highly Recommended)
-
-Our ESLint plugin prevents common errors by flagging unsupported code constructs (like loops) inside your workflows.
-
-**In `eslint.config.js` (Flat Config):**
-```javascript
-import bDurablePlugin from '@bobtail.software/eslint-plugin-b-durable';
-
-export default [
-  // ... your other configs
-  bDurablePlugin.configs.recommended,
-];
-```
-For legacy `.eslintrc.js` setup, see the [plugin's documentation](link-to-your-eslint-plugin-readme).
-
-### 3. Define a Workflow
+### 2. Define a Workflow
 
-Create a file ending in `.workflow.ts`. The `(input, context)` signature gives you access to durable functions.
+Create a `.workflow.ts` file. Note the mandatory `version` field.
 
 ```typescript
-// src/workflows/onboarding.workflow.ts
-import { bDurable, DurableContext } from '@bobtail.software/b-durable';
-import { createUser, sendWelcomeEmail } from '../services';
-
-interface OnboardingInput {
-  userId: string;
-  email: string;
-}
-
-export const userOnboardingWorkflow = bDurable({
-  workflow: async (input: OnboardingInput, context: DurableContext) => {
-    const user = await createUser({ id: input.userId, email: input.email });
-
-    await context.bSleep('10s');
-
-    await sendWelcomeEmail(user.email);
-
-    return { status: 'completed', userId: user.id };
+// src/workflows/user.workflow.ts
+import { bDurable } from '@bobtail.software/b-durable';
+import { sendEmail } from '../services';
+
+export const userOnboarding = bDurable({
+  name: 'userOnboarding',
+  version: '1.0', // Required for safety
+  workflow: async (input: { email: string }, context) => {
+    await context.bSleep('1 day');
+    await sendEmail(input.email, 'Welcome!');
+    return 'sent';
   },
 });
 ```
 
-### 4. Compile Workflows
-
-Add a script to your `package.json` to run the compiler.
+### 3. Compile
 
+Add to `package.json`:
 ```json
-// package.json
 "scripts": {
   "compile-workflows": "b-durable-compiler --in src/workflows --out src/generated"
 }
 ```
-Run `pnpm compile-workflows`. This generates the durable definitions in `src/generated`.
+Run `pnpm compile-workflows`.
 
-### 5. Initialize the Runtime
+### 4. Initialize the Runtime
 
-In your application's entry point, initialize the system and start a workflow.
+Initialize the system with Redis connections and configuration options.
 
 ```typescript
 // src/main.ts
 import { bDurableInitialize } from '@bobtail.software/b-durable';
 import Redis from 'ioredis';
-import durableFunctions, { userOnboardingWorkflow } from './generated';
-
-async function main() {
-  const redis = new Redis();
-  const blockingRedis = new Redis(); // Required for reliable queue operations
-
-  const durableSystem = bDurableInitialize({
-    durableFunctions,
-    sourceRoot: process.cwd(),
-    redisClient: redis,
-    blockingRedisClient: blockingRedis,
-  });
-
-  console.log('Durable system ready. Starting workflows...');
-
-  // --- Start a workflow with a system-generated ID ---
-  const workflowId1 = await durableSystem.start(userOnboardingWorkflow, {
-    input: {
-      userId: `user-${Date.now()}`,
-      email: 'test.user@example.com',
-    }
-  });
-  console.log(`Workflow ${workflowId1} started.`);
-
-  // --- Start a workflow with a predictable, user-provided ID ---
-  const orderId = `order-abc-123`;
-  const workflowId2 = await durableSystem.start(userOnboardingWorkflow, {
-    workflowId: orderId,
-    input: {
-      userId: 'user-from-order-123',
-      email: 'customer@example.com',
-    }
-  });
-  console.log(`Workflow started with predictable ID: ${workflowId2}`);
-  // Now you can easily send events using the 'orderId'
-  // durableSystem.sendEvent(..., orderId, ...);
-}
+import durableFunctions from './generated';
+import { myLogger } from './logger'; // Your Winston/Pino logger
+
+const durableSystem = bDurableInitialize({
+  durableFunctions,
+  sourceRoot: process.cwd(),
+  redisClient: new Redis(),
+  blockingRedisClient: new Redis(), // Dedicated connection for queues
+  
+  // --- Production Configuration ---
+  retention: '7 days',     // Auto-delete finished workflows after 7 days
+  pollingInterval: 5000,   // Scheduler/Heartbeat frequency (default: 5000ms)
+  logger: {                // Inject your logger for better observability
+      info: (msg, meta) => myLogger.info(msg, meta),
+      error: (msg, meta) => myLogger.error(msg, meta),
+      warn: (msg, meta) => myLogger.warn(msg, meta),
+      debug: (msg, meta) => myLogger.debug(msg, meta),
+  }
+});
+
+// Start a workflow
+const { workflowId } = await durableSystem.start(userOnboarding, {
+    input: { email: 'test@example.com' } 
+});
 
-main().catch(console.error);
+// Inspect state in real-time
+const state = await durableSystem.getState(workflowId);
+console.log(`Current step: ${state.step}, Status: ${state.status}`);
 ```
 
-### 6. Run Your Application
+## Advanced Features
+
+### Strict Versioning & Deployment
+
+When you modify a workflow, you **must** increment the `version` string (e.g., `'1.0'` -> `'1.1'`).
 
-Run your app (`node src/main.ts`). You'll see the workflow execute, pause, and resume, with all its state managed by `b-durable`.
+*   **Runtime Check**: Before executing a step, the worker checks if the database version matches the code version.
+*   **Mismatch**: If versions differ, the workflow halts with status `VERSION_MISMATCH`. This prevents "Frankenstein" workflows where step 2 of version 1 tries to run step 3 of version 2.
+*   **Strategy**: Run new workers alongside old workers (Blue/Green) or drain queues before deploying breaking changes.
 
-## Development Setup (for contributors)
+### Reliability & The Reaper
 
-The project is a `pnpm` monorepo.
+*   **Heartbeats**: Every worker sends a heartbeat to Redis every few seconds.
+*   **The Reaper**: If a worker crashes (OOM, power failure) while holding a task, the Reaper detects the missing heartbeat and automatically re-queues the task for another worker. No manual intervention required.
 
-1.  **Clone & Install**:
-    ```bash
-    git clone <repository-url>
-    cd b-durable-monorepo
-    pnpm install
-    ```
+### Error Handling & Dead Letter Queue (DLQ)
+
+*   **Retries**: Tasks are automatically retried 3 times on failure with backoff.
+*   **DLQ**: After 3 failures, the task payload and error stack are moved to the Redis list `queue:dead`.
+*   **Sub-workflows**: Failures in sub-workflows bubble up to the parent as standard JavaScript exceptions, catchable with `try/catch`.
+
+### Cancellation
+
+You can cancel a running workflow at any time.
+
+```typescript
+await durableSystem.cancel(workflowId, 'User requested cancellation');
+```
+
+Inside the workflow, this throws a `WorkflowCancellationError`. You can catch this to perform cleanup (e.g., reverting a payment) before re-throwing or returning.
+
+```typescript
+try {
+  await context.bWaitForEvent('approval');
+} catch (e) {
+  if (e.isCancellation) {
+     await refundPayment();
+     // Workflow ends here as CANCELLED
+  }
+  throw e;
+}
+```
 
-2.  **Run in Development Mode**:
-    This command builds the library, compiles example workflows, and starts the example app with hot-reloading.
-    ```bash
-    pnpm dev
-    ```
+## Architecture
 
-3.  **Run Tests**:
-    Tests use Vitest and a real Redis instance.
-    ```bash
-    pnpm --filter @bobtail.software/b-durable test
-    ```
+1.  **Compiler**: Analyzes `await` points and transforms code into a deterministic state machine.
+2.  **Redis**: Stores state, task queues, locks, and signals.
+3.  **Runtime**:
+    *   **Dispatcher**: Routes tasks to service functions.
+    *   **Scheduler**: Manages timers (`bSleep`) and the Reaper.
+    *   **Signal Bus**: Uses Redis Pub/Sub for real-time communication.
 
 ## License
 
-This project is licensed under the GPL-3.0 License. See the [LICENSE](LICENSE) file for details.
+GPL-3.0