@synergenius/flow-weaver 0.2.1 → 0.4.0

package/docs/reference/advanced-annotations.md

@@ -0,0 +1,431 @@
+---
+name: Advanced Annotations
+description: Pull execution, execution strategies, merge strategies, auto-connect, strict types, path and map sugar, node attributes, and multi-workflow files
+keywords: [pullExecution, executeWhen, mergeStrategy, autoConnect, strictTypes, path, map, sugar, attributes, expr, portOrder, portLabel, minimized, multi-workflow, CONJUNCTION, DISJUNCTION, FIRST, LAST, COLLECT, MERGE, CONCAT]
+---
+
+# Advanced Annotations
+
+This guide covers annotations that go beyond the basics in [Concepts](concepts). Each feature is fully supported by the parser, validator, and compiler.
+
+## Pull Execution
+
+Pull execution enables **lazy evaluation**. Nodes marked with `@pullExecution` don't execute eagerly — they only run when a downstream node actually consumes their output.
+
+### Node Type Level
+
+Declare a node type as pull-executed by default:
+
+```typescript
+/**
+ * @flowWeaver nodeType
+ * @expression
+ * @pullExecution execute
+ * @input value
+ * @output tripled
+ */
+function triple(value: number): { tripled: number } {
+  return { tripled: value * 3 };
+}
+```
+
+The argument (`execute`) specifies which STEP port triggers the lazy evaluation.
+
+### Instance Level Override
+
+Override pull execution per-instance in a workflow using the `[pullExecution: ...]` attribute:
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node t triple [pullExecution: execute]
+ * @connect Start.value -> t.value
+ * @connect t.tripled -> Exit.result
+ */
+```
+
+This is useful when a node type is not pull-executed by default, but a specific instance should be lazy.
+
+### How It Works
+
+1. During compilation, pull execution nodes are tracked separately
+2. Their output variables use `let` declarations (initially `undefined`)
+3. The node function is only called when a downstream node reads the output
+4. If no downstream node reads the output, the node never executes
+
+---
+
+## Execution Strategies (`@executeWhen`)
+
+Controls how a node evaluates incoming STEP signals before firing. This matters when a node has multiple STEP inputs.
+
+| Strategy | Behavior | Use Case |
+|----------|----------|----------|
+| `CONJUNCTION` | Execute when **ALL** incoming signals arrive (AND) | Default. Synchronization points |
+| `DISJUNCTION` | Execute when **ANY** signal arrives (OR) | Priority routing, first-response |
+| `CUSTOM` | Execution controlled by custom logic | Advanced patterns |
+
+### Node Type Level
+
+```typescript
+/**
+ * @flowWeaver nodeType
+ * @executeWhen DISJUNCTION
+ * @input data
+ * @output result
+ */
+function firstResponse(execute: boolean, data: string) {
+  if (!execute) return { onSuccess: false, onFailure: false, result: '' };
+  return { onSuccess: true, onFailure: false, result: data };
+}
+```
+
+### Instance Level
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node fr firstResponse [executeWhen: DISJUNCTION]
+ */
+```
+
+---
+
+## Merge Strategies
+
+When multiple connections target the same DATA input port, a merge strategy determines how the values are combined. Without a merge strategy, multiple connections to the same input produce a validation error (`MULTIPLE_CONNECTIONS_TO_INPUT`).
+
+| Strategy | Behavior | Result Type |
+|----------|----------|-------------|
+| `FIRST` | First non-undefined value | Same as port type |
+| `LAST` | Last non-undefined value | Same as port type |
+| `COLLECT` | Collect all values into an array | Array |
+| `MERGE` | Deep merge with `Object.assign` | Object |
+| `CONCAT` | Concatenate arrays/flatten | Array |
+
+### Declaring Merge Strategy
+
+Set the strategy on the port definition using `[mergeStrategy:X]`:
+
+```typescript
+/**
+ * @flowWeaver nodeType
+ * @expression
+ * @input items [mergeStrategy:COLLECT] - Collect all incoming items
+ * @output combined
+ */
+function aggregate(items: unknown[]): { combined: unknown[] } {
+  return { combined: items };
+}
+```
+
+### Generated Code
+
+The compiler generates appropriate merge expressions:
+
+```typescript
+// FIRST: returns first non-undefined value
+const merged = (() => { const __s__ = [a, b, c]; return __s__.find(v => v !== undefined); })();
+
+// COLLECT: wraps all into array
+const merged = [a, b, c];
+
+// MERGE: Object.assign
+const merged = Object.assign({}, a, b, c);
+
+// CONCAT: flatten arrays
+const merged = [a, b, c].flat();
+```
+
+---
+
+## Auto-Connect (`@autoConnect`)
+
+Enables automatic linear connection wiring. When `@autoConnect` is present and no explicit `@connect` annotations exist, nodes are wired sequentially in declaration order:
+
+```
+Start -> first @node -> second @node -> ... -> last @node -> Exit
+```
+
+Data ports are matched by name — if node A has an output named `result` and node B has an input named `result`, they are automatically connected.
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @autoConnect
+ * @node v validateRecord
+ * @node e enrichRecord
+ * @node s scoreRecord
+ * @param data - Input record
+ * @returns result - Scored record
+ */
+export function pipeline(params: { data: Record<string, unknown> }): { result: Record<string, unknown> } {
+  throw new Error('Not compiled');
+}
+```
+
+This is equivalent to manually writing:
+```
+@connect Start.data -> v.data
+@connect v.onSuccess -> e.execute
+@connect v.result -> e.data
+@connect e.onSuccess -> s.execute
+@connect e.result -> s.data
+@connect s.result -> Exit.result
+```
+
+**Important:** If any explicit `@connect` annotations are present, `@autoConnect` is disabled. It's all-or-nothing.
+
+---
+
+## Strict Types (`@strictTypes`)
+
+By default, type mismatches between connected ports produce warnings. With `@strictTypes`, they become errors.
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @strictTypes
+ * @node n myNode
+ * @connect Start.count -> n.text
+ */
+```
+
+Type compatibility levels:
+- **exact** — Same type (e.g. `STRING` → `STRING`)
+- **assignable** — Safe conversion (e.g. `NUMBER` → `ANY`)
+- **coercible** — Lossy conversion (e.g. `NUMBER` → `STRING`) — warning by default, error with `@strictTypes`
+- **incompatible** — No conversion possible — always an error
+
+---
+
+## Path Sugar (`@path`)
+
+Syntactic sugar for declaring multi-step execution routes. A `@path` annotation expands to a chain of STEP connections (execute → onSuccess).
+
+### Basic Syntax
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node v validate
+ * @node e enrich
+ * @node s score
+ * @path Start -> v -> e -> s -> Exit
+ */
+```
+
+This expands to:
+```
+@connect Start.execute -> v.execute
+@connect v.onSuccess -> e.execute
+@connect e.onSuccess -> s.execute
+@connect s.onSuccess -> Exit.execute
+```
+
+### Branching with `:ok` and `:fail`
+
+Use `:ok` or `:fail` suffixes to route through `onSuccess` or `onFailure`:
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node v validate
+ * @node router routeUrgency
+ * @node handler handle
+ * @node esc escalate
+ *
+ * @path Start -> v -> router:ok -> handler -> Exit
+ * @path Start -> v -> router:fail -> esc -> Exit
+ */
+```
+
+Without a suffix, `:ok` (onSuccess) is the default. Duplicate connections from overlapping paths are automatically deduplicated.
+
+### Path Validation
+
+The sugar optimizer validates that all nodes referenced in `@path` exist and that the expected control-flow connections are still valid. Stale paths are automatically filtered during parse-regenerate round-trips.
+
+---
+
+## Map Sugar (`@map`)
+
+Syntactic sugar for forEach iteration patterns. A `@map` expands to a synthetic iterator node type with proper scopes and connections.
+
+### Basic Syntax
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node proc doubleIt
+ * @map loop proc over Start.items
+ * @connect loop.results -> Exit.results
+ */
+```
+
+This creates a `loop` instance of a synthetic `MAP_ITERATOR` node type that:
+1. Takes `Start.items` as the array to iterate
+2. Calls `proc` (doubleIt) for each element
+3. Collects results into `loop.results`
+
+### With Explicit Port Mapping
+
+Specify which input/output ports to use on the child node:
+
+```typescript
+@map loop proc(file -> post) over scan.files
+```
+
+This maps:
+- `scan.files` array elements → `proc.file` (input)
+- `proc.post` (output) → collected into `loop.results`
+
+If ports are omitted, the first non-STEP input and first non-STEP output are used automatically.
+
+---
+
+## Node Instance Attributes
+
+Node instances in workflows support attribute brackets `[...]` for configuration. Multiple brackets can be combined.
+
+### Expression Bindings (`[expr: ...]`)
+
+Set port values via JavaScript expressions instead of connections:
+
+```typescript
+@node wait waitForEvent [expr: eventName="'app/expense.approved'", match="'data.expenseId'", timeout="'48h'"]
+```
+
+Each assignment is `portName="expression"`. Multiple assignments are comma-separated.
+
+### Port Order (`[portOrder: ...]`)
+
+Control the visual ordering of ports:
+
+```typescript
+@node myNode MyType [portOrder: input1=1, input2=2, output1=3]
+```
+
+### Port Labels (`[portLabel: ...]`)
+
+Override port display labels:
+
+```typescript
+@node myNode MyType [portLabel: execute="Start Here", onSuccess="Done"]
+```
+
+### Minimized (`[minimized]`)
+
+Display the node in a collapsed/minimized state:
+
+```typescript
+@node helper HelperNode [minimized]
+```
+
+### Size (`[size: W H]`)
+
+Custom node dimensions in the visual editor:
+
+```typescript
+@node big BigNode [size: 400 300]
+```
+
+### Color (`[color: "..."]`)
+
+Custom node color:
+
+```typescript
+@node special MyType [color: "#ff6b35"]
+```
+
+### Icon (`[icon: "..."]`)
+
+Custom node icon:
+
+```typescript
+@node db DatabaseNode [icon: "database"]
+```
+
+### Tags (`[tags: ...]`)
+
+Visual tags/badges on the instance. Each tag has a label string and optional tooltip:
+
+```typescript
+@node myNode MyType [tags: "async" "Runs asynchronously", "beta"]
+```
+
+### Combining Attributes
+
+Multiple attribute brackets can appear on the same `@node`:
+
+```typescript
+@node wait waitForEvent [expr: eventName="'approval'", timeout="'24h'"] [minimized] [color: "#3b82f6"]
+```
+
+---
+
+## Multi-Workflow Files
+
+A single TypeScript file can contain multiple `@flowWeaver workflow` annotations. Each workflow is a separate exported function.
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node v validate
+ * @path Start -> v -> Exit
+ */
+export function validatePipeline(params: { data: string }) { ... }
+
+/**
+ * @flowWeaver workflow
+ * @node e enrich
+ * @path Start -> e -> Exit
+ */
+export function enrichPipeline(params: { data: string }) { ... }
+```
+
+### Targeting a Specific Workflow
+
+Most CLI commands accept `--workflow-name` or `-w` to target a specific workflow:
+
+```bash
+flow-weaver compile multi.ts --workflow-name validatePipeline
+flow-weaver validate multi.ts -w enrichPipeline
+flow-weaver run multi.ts -w validatePipeline --params '{"data": "test"}'
+flow-weaver describe multi.ts --workflow-name enrichPipeline
+```
+
+Without this flag, all workflows in the file are processed.
+
+### Cross-Workflow References
+
+Workflows in the same file can reference each other as node types. The parser does a first-pass signature extraction, so the order of declaration doesn't matter.
+
+---
+
+## Node Type Annotations
+
+These annotations go on `@flowWeaver nodeType` blocks:
+
+| Annotation | Purpose | Example |
+|------------|---------|---------|
+| `@name` | Override display name | `@name MyCustomName` |
+| `@label` | Human-readable label | `@label Fetch with Timeout` |
+| `@description` | Node description | `@description Validates expense data` |
+| `@color` | Custom color | `@color "#ff6b35"` |
+| `@icon` | Custom icon | `@icon "database"` |
+| `@tag` | Visual tag/badge | `@tag async` or `@tag beta "Experimental"` |
+| `@scope` | Provides a named scope | `@scope processItem` |
+| `@expression` | Expression mode (simplified signature) | `@expression` |
+| `@executeWhen` | Execution strategy | `@executeWhen DISJUNCTION` |
+| `@pullExecution` | Lazy evaluation | `@pullExecution execute` |
+
+---
+
+## Related Topics
+
+- [Concepts](concepts) — Core workflow fundamentals
+- [JSDoc Grammar](jsdoc-grammar) — Formal EBNF syntax for all annotations
+- [Compilation](compilation) — How annotations affect code generation
+- [Error Codes](error-codes) — Validation errors for annotation issues
+- [CLI Reference](cli-reference) — All command flags

package/docs/reference/built-in-nodes.md

@@ -0,0 +1,225 @@
+---
+name: Built-in Nodes
+description: Built-in runtime nodes (delay, waitForEvent, invokeWorkflow) and the mock system for testing
+keywords: [delay, waitForEvent, invokeWorkflow, built-in, runtime, mock, mocks, FwMockConfig, testing, fast, events, invocations, sleep, duration, timeout]
+---
+
+# Built-in Nodes
+
+Flow Weaver provides three built-in node types for common runtime operations. All three support a **mock system** for local testing without real side effects.
+
+## delay
+
+Pauses execution for a specified duration.
+
+```typescript
+/**
+ * @flowWeaver nodeType
+ * @input duration - Duration to sleep (e.g. "30s", "5m", "1h", "2d")
+ * @output elapsed - Always true after sleep completes
+ */
+async function delay(execute: boolean, duration: string)
+```
+
+### Duration Format
+
+`<number><unit>` where unit is one of:
+
+| Unit | Meaning | Example |
+|------|---------|---------|
+| `ms` | Milliseconds | `500ms` |
+| `s` | Seconds | `30s` |
+| `m` | Minutes | `5m` |
+| `h` | Hours | `1h` |
+| `d` | Days | `2d` |
+
+### Usage in Workflow
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node wait delay [expr: duration="'30s'"]
+ * @path Start -> wait -> Exit
+ */
+```
+
+### Mock Behavior
+
+When `fast: true` is set in mock config, `delay` sleeps for 1ms instead of the real duration.
+
+---
+
+## waitForEvent
+
+Pauses execution until a named event is received. Designed for human-in-the-loop patterns and inter-workflow communication.
+
+```typescript
+/**
+ * @flowWeaver nodeType
+ * @input eventName - Event name to wait for (e.g. "app/approval.received")
+ * @input [match] - Field to match between trigger and waited event (e.g. "data.requestId")
+ * @input [timeout] - Max wait time (e.g. "24h", "7d"). Empty = no timeout
+ * @output eventData - The received event's data payload
+ */
+async function waitForEvent(execute: boolean, eventName: string, match?: string, timeout?: string)
+```
+
+### Usage in Workflow
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node wait waitForEvent [expr: eventName="'app/expense.approved'", match="'data.expenseId'", timeout="'48h'"]
+ * @path Start -> wait -> Exit
+ */
+```
+
+### Mock Behavior
+
+- If `events[eventName]` is set in mock config → returns that data via `onSuccess`
+- If no mock data for the event name → simulates timeout via `onFailure`
+
+---
+
+## invokeWorkflow
+
+Invokes another workflow (Inngest function) and waits for its result. Enables workflow composition.
+
+```typescript
+/**
+ * @flowWeaver nodeType
+ * @input functionId - Inngest function ID (e.g. "my-service/sub-workflow")
+ * @input payload - Data to pass as event.data to the invoked function
+ * @input [timeout] - Max wait time (e.g. "1h")
+ * @output result - Return value from the invoked function
+ */
+async function invokeWorkflow(execute: boolean, functionId: string, payload: object, timeout?: string)
+```
+
+### Usage in Workflow
+
+```typescript
+/**
+ * @flowWeaver workflow
+ * @node sub invokeWorkflow [expr: functionId="'my-service/payment-processor'", timeout="'5m'"]
+ * @connect Start.payload -> sub.payload
+ * @connect sub.result -> Exit.result
+ * @path Start -> sub -> Exit
+ */
+```
+
+### Mock Behavior
+
+- If `invocations[functionId]` is set in mock config → returns that result via `onSuccess`
+- If no mock data for the function ID → simulates failure via `onFailure`
+
+---
+
+## Mock System
+
+The mock system lets you test workflows with built-in nodes locally without real delays, event systems, or external workflow invocations.
+
+### FwMockConfig
+
+```typescript
+interface FwMockConfig {
+  /** Mock event data keyed by event name. Used by waitForEvent. */
+  events?: Record<string, object>;
+  /** Mock invocation results keyed by functionId. Used by invokeWorkflow. */
+  invocations?: Record<string, object>;
+  /** When true, delay nodes skip the real sleep (1ms instead of full duration). */
+  fast?: boolean;
+}
+```
+
+### CLI Usage
+
+Pass mock config directly:
+
+```bash
+flow-weaver run workflow.ts --mocks '{"fast": true, "events": {"app/approved": {"status": "ok"}}}'
+```
+
+Or from a file:
+
+```bash
+flow-weaver run workflow.ts --mocks-file mocks.json
+```
+
+**mocks.json:**
+```json
+{
+  "fast": true,
+  "events": {
+    "app/expense.approved": { "status": "approved", "approvedBy": "manager@co.com" },
+    "app/expense.withdrawn": { "status": "withdrawn" }
+  },
+  "invocations": {
+    "my-service/payment-processor": { "transactionId": "tx-123", "success": true }
+  }
+}
+```
+
+### Programmatic Usage
+
+Set mocks on `globalThis` before running the workflow:
+
+```typescript
+(globalThis as any).__fw_mocks__ = {
+  fast: true,
+  events: {
+    'app/expense.approved': { status: 'approved' }
+  },
+  invocations: {
+    'my-service/sub-workflow': { result: 'success' }
+  }
+};
+
+// Run your compiled workflow
+const result = await expenseWorkflow({ expenseId: 'exp-1', amount: 500 });
+```
+
+### Testing Patterns
+
+**Unit test with mocks:**
+
+```typescript
+import { describe, it, expect, beforeEach, afterEach } from 'vitest';
+
+describe('expense workflow', () => {
+  beforeEach(() => {
+    (globalThis as any).__fw_mocks__ = {
+      fast: true,
+      events: {
+        'app/expense.approved': { status: 'approved' }
+      }
+    };
+  });
+
+  afterEach(() => {
+    delete (globalThis as any).__fw_mocks__;
+  });
+
+  it('should process approved expense', async () => {
+    const result = await expenseWorkflow({ expenseId: 'e1', amount: 100 });
+    expect(result.onSuccess).toBe(true);
+  });
+});
+```
+
+**Testing timeout paths:**
+
+```typescript
+// Don't provide mock data for the event to simulate timeout
+(globalThis as any).__fw_mocks__ = { fast: true };
+// waitForEvent will follow onFailure path
+```
+
+---
+
+## Related Topics
+
+- [CLI Reference](cli-reference) — `run` command with `--mocks` flags
+- [Compilation](compilation) — Inngest target for durable built-in node execution
+- [Debugging](debugging) — Tracing and troubleshooting
+- [Advanced Annotations](advanced-annotations) — Expression bindings for node inputs