@gravito/flux 3.0.1 → 3.0.3

package/README.md

@@ -2,14 +2,31 @@
 
 > ⚡ Platform-agnostic, high-performance workflow engine for Gravito
 
-## Features
-
-- **Pure State Machine** - No runtime dependencies, Web Standard APIs only
-- **Fluent Builder API** - Type-safe, chainable workflow definitions
-- **Storage Adapters** - Memory, SQLite (Bun), PostgreSQL (coming soon)
-- **Retry & Timeout** - Automatic retry with exponential backoff
-- **Event Hooks** - Subscribe to workflow/step lifecycle events
-- **Dual Platform** - Works with both Bun and Node.js
+## ✨ Features
+
+- 🪐 **Galaxy-Ready Workflow Engine** - Native integration with PlanetCore for universal business process orchestration.
+- 🔄 **Distributed Saga Coordination** - Reliable transaction management across multiple isolated Satellites with automatic rollback.
+- ⚡ **Pure State Machine** - High-performance engine with zero runtime dependencies beyond Web Standard APIs.
+- 🔐 **Distributed Locking** - Plasma-backed Redis locking to ensure process safety in multi-node clusters.
+- 📡 **Signal & Suspend** - Pause workflows to wait for external stimulus (Webhooks, manual approvals) and resume instantly.
+- 🛠️ **Fluent Builder API** - Fully type-safe, chainable definitions for complex logic flows.
+
+## 🌌 Role in Galaxy Architecture
+
+In the **Gravito Galaxy Architecture**, Flux acts as the **Logic Orchestrator (Cerebral Cortex)**.
+
+- **Process Master**: Manages complex, long-running business processes that span across multiple Satellites (e.g., a "Checkout" process involving `Catalog`, `Payment`, and `Notification`).
+- **Reliability Engine**: Ensures that even if a Satellite fails mid-process, the state is preserved and the appropriate compensation logic (Saga) is executed.
+- **State Persistence**: Works with `Atlas` or `Plasma` to store the execution state, allowing workflows to survive system restarts or move between nodes.
+
+```mermaid
+graph TD
+    S1[Satellite: Order] -- "Trigger Workflow" --> Flux{Flux Engine}
+    Flux -- "Step 1" --> S2[Satellite: Inventory]
+    Flux -- "Step 2" --> S3[Satellite: Payment]
+    Flux -- "Fail / Rollback" --> S2
+    Flux -.-> State[(State: Plasma/Redis)]
+```
 
 ## Installation
 
@@ -149,6 +166,14 @@ const reportWorkflow = createWorkflow('generate-report')
   })
 ```
 
+## 📚 Documentation
+
+Detailed guides and references for the Galaxy Architecture:
+
+- [🏗️ **Architecture Overview**](./README.md) — Under the hood of the state machine.
+- [🔄 **Workflow Patterns**](./doc/WORKFLOW_PATTERNS.md) — **NEW**: Sagas, Suspension, and Distributed Locking.
+- [🧪 **Testing Workflows**](./tests/README.md) — Mocking and execution testing.
+
 ## API
 
 ### `createWorkflow(name)`
@@ -237,6 +262,113 @@ Commit steps are marked to always execute, even on workflow replay:
 })
 ```
 
+## Workflow Versioning
+
+Track workflow definition versions for migration and compatibility management:
+
+### Setting Version
+
+```typescript
+const workflow = createWorkflow('order-process')
+  .version('2.0.0')
+  .input<OrderInput>()
+  .step('validate', async (ctx) => { /* ... */ })
+  .build()
+```
+
+### Filtering by Version
+
+```typescript
+// List workflows by version
+const v1Workflows = await engine.list({ version: '1.0.0' })
+
+// Combine with other filters
+const results = await engine.list({
+  name: 'order-process',
+  version: '2.0.0',
+  status: 'completed',
+  limit: 10,
+})
+```
+
+### Version Mismatch Warning
+
+When resuming a workflow with a different definition version, Flux logs a warning:
+
+```typescript
+// Original execution with v1.0.0
+const result = await engine.execute(workflowV1, input)
+
+// Resume with v2.0.0 - warns about mismatch
+await engine.resume(workflowV2, result.id, { fromStep: 1 })
+// ⚠️ Warning: version mismatch (stored: 1.0.0, current: 2.0.0)
+```
+
+## Batch Execution
+
+Execute multiple workflow instances efficiently with controlled concurrency:
+
+### Basic Usage
+
+```typescript
+const results = await engine.executeBatch(
+  orderWorkflow,
+  orders.map(o => ({ orderId: o.id })),
+  { 
+    concurrency: 10,
+    continueOnError: true,
+    onProgress: (completed, total) => console.log(`${completed}/${total}`)
+  }
+)
+
+console.log(`Succeeded: ${results.succeeded}, Failed: ${results.failed}`)
+```
+
+### Using BatchExecutor
+
+For more control, use `BatchExecutor` directly:
+
+```typescript
+import { BatchExecutor } from '@gravito/flux'
+
+const executor = new BatchExecutor(engine)
+
+// Same workflow, multiple inputs
+const result = await executor.execute(workflow, inputs, {
+  concurrency: 5,        // Max parallel executions (default: 10)
+  continueOnError: true, // Continue if one fails (default: false)
+  signal: controller.signal, // AbortSignal for cancellation
+  onProgress: (completed, total, lastResult) => {
+    updateProgressBar(completed / total)
+  }
+})
+
+// Different workflows
+const result = await executor.executeMany([
+  { workflow: orderWorkflow, input: { orderId: '1' } },
+  { workflow: notifyWorkflow, input: { userId: '2' } },
+  { workflow: orderWorkflow, input: { orderId: '3' } },
+])
+```
+
+### Result Structure
+
+```typescript
+interface BatchResult<T> {
+  total: number        // Total items processed
+  succeeded: number    // Successful executions
+  failed: number       // Failed executions
+  duration: number     // Total execution time (ms)
+  results: Array<{
+    index: number
+    input: T
+    success: boolean
+    result?: WorkflowState
+    error?: Error
+  }>
+}
+```
+
 ## Storage Adapters
 
 ### MemoryStorage (Default)
@@ -261,6 +393,42 @@ const engine = new FluxEngine({
 })
 ```
 
+### PostgreSQLStorage (Production)
+
+PostgreSQL for production deployments:
+
+```typescript
+import { FluxEngine, PostgreSQLStorage } from '@gravito/flux'
+
+const storage = new PostgreSQLStorage({
+  connectionString: 'postgresql://user:password@localhost:5432/dbname',
+  tableName: 'flux_workflows',
+  ssl: true
+})
+
+const engine = new FluxEngine({ storage })
+```
+
+Features:
+- **JSONB columns** for efficient querying of workflow data
+- **Connection pooling** via `pg` library
+- **Automatic schema migration** on init
+- **Optimized indexes** for name, status, and created_at
+
+Alternative configuration:
+
+```typescript
+const storage = new PostgreSQLStorage({
+  host: 'localhost',
+  port: 5432,
+  database: 'myapp',
+  user: 'postgres',
+  password: 'secret',
+  tableName: 'workflows',
+  ssl: { rejectUnauthorized: false }
+})
+```
+
 ### Custom Storage
 
 Implement `WorkflowStorage` interface:
@@ -276,6 +444,103 @@ interface WorkflowStorage {
 }
 ```
 
+## Distributed Locking
+
+For multi-node deployments, Flux provides a Redis-based distributed locking mechanism to prevent concurrent execution of the same workflow across multiple nodes.
+
+### RedisLockProvider
+
+Use Redis for distributed locking in production clusters:
+
+```typescript
+import { FluxEngine, RedisLockProvider } from '@gravito/flux'
+import Redis from 'ioredis'
+
+const redis = new Redis({
+  host: 'localhost',
+  port: 6379,
+})
+
+const lockProvider = new RedisLockProvider({
+  client: redis,
+  keyPrefix: 'myapp:locks:',  // Default: 'flux:lock:'
+  defaultTtl: 30000,           // Default: 30000ms (30s)
+  retryDelay: 100,             // Default: 100ms
+  maxRetries: 3,               // Default: 0 (no retries)
+})
+
+const engine = new FluxEngine({
+  storage: new PostgreSQLStorage({ /* ... */ }),
+  lockProvider,
+})
+```
+
+### Features
+
+- **Atomic Acquisition**: Uses Redis `SET NX PX` for atomic lock acquisition
+- **Safe Release**: Lua scripts ensure only the lock owner can release
+- **Auto-Expiration**: Locks automatically expire if a node crashes
+- **Retry Support**: Configurable retry with exponential backoff
+- **Idempotent**: Same owner can refresh an existing lock
+
+### Usage
+
+Locks are automatically acquired and released during workflow execution:
+
+```typescript
+// Flux automatically acquires lock before execution
+const result = await engine.execute(workflow, input)
+// Lock is automatically released after completion
+```
+
+Manual lock management:
+
+```typescript
+const lock = await lockProvider.acquire('workflow-123', 'node-1', 30000)
+
+if (lock) {
+  try {
+    // Critical section - only one node can execute
+    await doWork()
+  } finally {
+    await lock.release()
+  }
+} else {
+  console.log('Another node is processing this workflow')
+}
+```
+
+### MemoryLockProvider (Development)
+
+For single-node development environments:
+
+```typescript
+import { FluxEngine, MemoryLockProvider } from '@gravito/flux'
+
+const engine = new FluxEngine({
+  lockProvider: new MemoryLockProvider(),
+})
+```
+
+### Custom Lock Providers
+
+Implement the `LockProvider` interface for custom backends:
+
+```typescript
+interface LockProvider {
+  acquire(resourceId: string, owner: string, ttl: number): Promise<Lock | null>
+  refresh(resourceId: string, owner: string, ttl: number): Promise<boolean>
+  release(resourceId: string): Promise<void>
+}
+
+interface Lock {
+  id: string
+  owner: string
+  expiresAt: number
+  release(): Promise<void>
+}
+```
+
 ## Gravito Integration
 
 ```typescript
@@ -301,6 +566,7 @@ await flux.execute(myWorkflow, input)
 |---------|-----|---------|
 | FluxEngine | ✅ | ✅ |
 | MemoryStorage | ✅ | ✅ |
+| PostgreSQLStorage | ✅ | ✅ |
 | BunSQLiteStorage | ✅ | ❌ |
 | OrbitFlux | ✅ | ✅ |
 
@@ -320,6 +586,61 @@ bun run examples/user-signup.ts
 
 # Report generation
 bun run examples/report-generation.ts
+
+# PostgreSQL storage (requires PostgreSQL)
+POSTGRES_URL="postgresql://localhost:5432/flux_demo" bun run examples/postgresql-storage.ts
+```
+
+## Testing
+
+Flux has comprehensive test coverage with 300 total tests across 26 test files:
+
+```bash
+# Run all tests
+bun test
+
+# Run with coverage report
+bun test --coverage
+
+# Run specific test file
+bun test tests/flux.test.ts
+bun test tests/errors.test.ts
+bun test tests/workflow-builder.test.ts
+```
+
+### Coverage Metrics
+
+- **Function Coverage:** 87% (86.98%)
+- **Line Coverage:** 92% (92.49%)
+- **Total Tests:** 300 passing, 12 skipped (PostgreSQL integration tests)
+
+### What's Tested
+
+- ✅ Core workflow execution (FluxEngine, WorkflowExecutor, StepExecutor)
+- ✅ State management (StateMachine, ContextManager, StateUpdater)
+- ✅ Error handling (all 14 error factory functions, FluxError class)
+- ✅ Retry & timeout mechanisms (CompensationRetryPolicy, IdempotencyGuard)
+- ✅ Compensation & rollback (RollbackManager, RecoveryManager, Saga pattern)
+- ✅ Storage adapters (Memory, BunSQLite, PostgreSQL)
+- ✅ Parallel execution (ParallelExecutor)
+- ✅ Suspension & signals (wait/resume workflows)
+- ✅ Workflow builder API (data, describe, validate, step chaining)
+- ✅ Visualization (MermaidGenerator with all diagram variations)
+- ✅ Profiling & tracing (WorkflowProfiler, TraceEmitter, JsonFileTraceSink)
+- ✅ Gravito integration (OrbitFlux lifecycle)
+- ✅ Workflow versioning (.version(), version filtering, mismatch warnings)
+- ✅ Batch execution (BatchExecutor, executeBatch, concurrency control)
+- ✅ Redis distributed locking (RedisLockProvider with mocked Redis)
+
+### Skipped Tests
+
+The 12 skipped tests are PostgreSQL integration tests that require a running database:
+
+```bash
+# To run PostgreSQL tests locally:
+docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=test postgres:15
+export POSTGRES_URL="postgresql://postgres:test@localhost:5432/flux_test"
+bun test tests/postgresql-storage.test.ts
 ```
 
 ## License

package/bin/flux.js

@@ -1,6 +1,6 @@
 #!/usr/bin/env node
-import { createServer } from 'node:http'
 import { readFile } from 'node:fs/promises'
+import { createServer } from 'node:http'
 import { dirname, extname, join, resolve } from 'node:path'
 import { fileURLToPath } from 'node:url'
 
@@ -12,6 +12,13 @@ const help = () => {
 
 Usage:
   flux dev --trace ./.flux/trace.ndjson --port 4280
+  flux visualize --definition <path> [options]
+  
+Commands:
+  dev         Start development viewer for trace files
+  visualize   Generate Mermaid diagrams from workflow definitions
+  
+Run 'flux <command> --help' for command-specific help
 `)
 }
 
@@ -20,6 +27,23 @@ if (!command || command === '--help' || command === '-h') {
   process.exit(0)
 }
 
+if (command === 'visualize') {
+  const { fileURLToPath } = await import('node:url')
+  const { dirname, join } = await import('node:path')
+  const { spawn } = await import('node:child_process')
+  
+  const __filename = fileURLToPath(import.meta.url)
+  const __dirname = dirname(__filename)
+  const visualizeScript = join(__dirname, '..', 'dist', 'cli', 'flux-visualize.js')
+  
+  const child = spawn('node', [visualizeScript, ...args.slice(1)], {
+    stdio: 'inherit',
+  })
+  
+  child.on('exit', (code) => process.exit(code || 0))
+  await new Promise(() => {})
+}
+
 if (command !== 'dev') {
   console.error(`Unknown command: ${command}`)
   help()

package/dev/viewer/app.js

@@ -66,9 +66,7 @@ const renderStatus = (events) => {
     return
   }
 
-  const lastWorkflow = [...events]
-    .reverse()
-    .find((event) => event.type.startsWith('workflow:'))
+  const lastWorkflow = [...events].reverse().find((event) => event.type.startsWith('workflow:'))
 
   const status = lastWorkflow?.status ?? 'unknown'
   const badgeClass = status === 'completed' ? 'ok' : status === 'failed' ? 'fail' : ''
@@ -115,7 +113,9 @@ const refresh = async () => {
   const res = await fetch('/trace', { cache: 'no-store' })
   const text = await res.text()
   const hash = hashText(text)
-  if (hash === state.lastHash) return
+  if (hash === state.lastHash) {
+    return
+  }
 
   state.lastHash = hash
   state.events = parseNdjson(text)

package/dist/bun.cjs

@@ -1,7 +1,7 @@
 "use strict";Object.defineProperty(exports, "__esModule", {value: true});
 
-var _chunkSJSPR4ZUcjs = require('./chunk-SJSPR4ZU.cjs');
+var _chunkZE2RDS47cjs = require('./chunk-ZE2RDS47.cjs');
 
 
-exports.BunSQLiteStorage = _chunkSJSPR4ZUcjs.BunSQLiteStorage;
+exports.BunSQLiteStorage = _chunkZE2RDS47cjs.BunSQLiteStorage;
 //# sourceMappingURL=bun.cjs.map

package/dist/bun.d.cts

@@ -1,74 +1,113 @@
 import { Database } from 'bun:sqlite';
-import { o as WorkflowStorage, m as WorkflowState, l as WorkflowFilter } from './types-CZwYGpou.cjs';
+import { p as WorkflowStorage, a as WorkflowState, n as WorkflowFilter } from './types-CGIEQPFv.cjs';
 
 /**
- * @fileoverview Bun SQLite Storage Adapter
- *
- * High-performance storage using Bun's built-in SQLite.
- *
- * @module @gravito/flux/storage
- */
-
-/**
- * SQLite Storage Options
+ * Configuration options for the Bun SQLite storage adapter.
  */
 interface BunSQLiteStorageOptions {
-    /** Database file path (default: ':memory:') */
+    /**
+     * Path to the SQLite database file.
+     * Use ':memory:' for an ephemeral in-memory database.
+     */
     path?: string;
-    /** Table name (default: 'flux_workflows') */
+    /**
+     * Name of the table used to store workflow states.
+     */
     tableName?: string;
 }
 /**
- * Bun SQLite Storage
+ * BunSQLiteStorage provides a persistent storage backend for Flux workflows using Bun's native SQLite module.
  *
- * High-performance storage adapter using Bun's built-in SQLite.
+ * It handles automatic table creation, indexing for performance, and serialization of workflow state
+ * into a relational format.
  *
  * @example
  * ```typescript
- * const engine = new FluxEngine({
- *   storage: new BunSQLiteStorage({ path: './data/flux.db' })
- * })
+ * const storage = new BunSQLiteStorage({
+ *   path: './workflows.db',
+ *   tableName: 'my_workflows'
+ * });
+ * await storage.init();
  * ```
  */
 declare class BunSQLiteStorage implements WorkflowStorage {
     private db;
     private tableName;
     private initialized;
+    /**
+     * Creates a new instance of BunSQLiteStorage.
+     *
+     * @param options - Configuration for the database connection and table naming.
+     */
     constructor(options?: BunSQLiteStorageOptions);
     /**
-     * Initialize storage (create tables)
+     * Initializes the database schema and required indexes.
+     *
+     * This method is idempotent and will be called automatically by other operations if not invoked manually.
+     *
+     * @throws {Error} If the database schema cannot be created or indexes fail to initialize.
      */
     init(): Promise<void>;
     /**
-     * Save workflow state
+     * Persists or updates a workflow state in the database.
+     *
+     * Uses an "INSERT OR REPLACE" strategy to ensure the latest state is always stored for a given ID.
+     *
+     * @param state - The current state of the workflow to be saved.
+     * @throws {Error} If the database write operation fails or serialization errors occur.
      */
     save(state: WorkflowState): Promise<void>;
     /**
-     * Load workflow state by ID
+     * Retrieves a workflow state by its unique identifier.
+     *
+     * @param id - The unique ID of the workflow to load.
+     * @returns The reconstructed workflow state, or null if no record is found.
+     * @throws {Error} If the database query fails or deserialization of stored JSON fails.
      */
     load(id: string): Promise<WorkflowState | null>;
     /**
-     * List workflow states with optional filter
+     * Lists workflow states based on the provided filtering criteria.
+     *
+     * Results are returned in descending order of creation time.
+     *
+     * @param filter - Criteria for filtering and paginating the results.
+     * @returns An array of workflow states matching the filter.
+     * @throws {Error} If the database query fails.
      */
     list(filter?: WorkflowFilter): Promise<WorkflowState[]>;
     /**
-     * Delete workflow state
+     * Deletes a workflow state from the database.
+     *
+     * @param id - The unique ID of the workflow to delete.
+     * @throws {Error} If the database deletion fails.
      */
     delete(id: string): Promise<void>;
     /**
-     * Close database connection
+     * Closes the database connection and resets the initialization state.
+     *
+     * @throws {Error} If the database connection cannot be closed cleanly.
      */
     close(): Promise<void>;
     /**
-     * Convert SQLite row to WorkflowState
+     * Converts a raw database row into a structured WorkflowState object.
+     *
+     * @param row - The raw SQLite row data.
+     * @returns The parsed workflow state.
+     * @private
      */
     private rowToState;
     /**
-     * Get raw database (for advanced usage)
+     * Provides direct access to the underlying Bun SQLite Database instance.
+     *
+     * Useful for performing custom queries or maintenance tasks.
+     *
+     * @returns The raw Database instance.
      */
     getDatabase(): Database;
     /**
-     * Run a vacuum to optimize database
+     * Performs a VACUUM operation to reclaim unused space and defragment the database.
+     *
+     * @throws {Error} If the VACUUM operation fails.
      */
     vacuum(): void;
 }