flowcraft 1.0.0-beta.1

package/docs/guide/best-practices/debugging.md

@@ -0,0 +1,182 @@
+# Best Practices: Debugging Workflows
+
+Debugging multi-step, asynchronous workflows can be challenging. State can change in unexpected ways, and control flow can be complex. Flowcraft is designed with debuggability in mind and provides several tools and patterns to help you pinpoint issues quickly.
+
+This guide covers the most effective techniques for debugging your workflows.
+
+## 1. Inspect Data Flow with `.tap()`
+
+> [!TIP]
+> The `.tap()` method is your best friend for non-disruptive debugging. It's the cleanest way to inspect data mid-pipeline without breaking a fluent chain.
+
+Instead of breaking your chain to insert a `console.log`, use the `.tap()` method. It receives the result of the previous step, allows you to perform a side-effect (like logging), and then passes the original result through to the next step, completely unmodified.
+
+**Scenario**: You have a chain of `.map()` calls and want to see the intermediate result.
+
+```typescript
+import { contextKey, Node } from 'flowcraft'
+
+const FINAL_RESULT = contextKey<string>('final_result')
+
+// A node that fetches a user object
+const fetchUserNode = new Node().exec(() => ({ id: 123, name: 'Alice', email: 'alice@test.com' }))
+
+const processUser = fetchUserNode
+	.map(user => ({ ...user, name: user.name.toUpperCase() }))
+// Let's inspect the data right here!
+	.tap((intermediateResult) => {
+		console.log('[DEBUG] After capitalization:', intermediateResult)
+	})
+	.map(user => `User ID: ${user.id}, Name: ${user.name}`)
+	.toContext(FINAL_RESULT)
+
+// When this runs, the debug log will print the intermediate object.
+```
+
+## 2. Trace Execution with the Logger
+
+When your problem is about *control flow* ("Why did my workflow take the wrong branch?"), the logger is your best friend. By passing a `ConsoleLogger` to your `flow.run()` call, you get a detailed, step-by-step trace of the entire execution.
+
+The logger will show:
+
+- Which node is currently running.
+- The **action** string returned by each node.
+- The successor node chosen for that action.
+- Warnings for retry attempts and errors for fallback execution.
+
+```typescript
+import { ConsoleLogger, Flow, TypedContext } from 'flowcraft'
+
+// Assume you have a conditional flow set up
+const myFlow = createMyConditionalFlow()
+const context = new TypedContext()
+
+// Run the flow with a logger enabled
+await myFlow.run(context, { logger: new ConsoleLogger() })
+```
+
+**Example Log Output**:
+
+```
+[INFO] Running flow: MyFlow
+[INFO] Running node: CheckConditionNode
+[DEBUG] Action 'action_approve' from CheckConditionNode leads to ApproveNode
+[INFO] Running node: ApproveNode
+[INFO] Flow ends: Action 'Symbol(default)' from ApproveNode has no configured successor.
+```
+
+This output makes it immediately clear that `CheckConditionNode` returned `'action_approve'`, which led to `ApproveNode`.
+
+## 3. Visualize Your Graph with `generateMermaidGraph`
+
+Sometimes the problem isn't the logic inside your nodes, but the way you've wired them together. It's easy to make a mistake with `.next()`, creating a dead end or an incorrect branch.
+
+Flowcraft includes a `generateMermaidGraph` utility that creates a visual representation of your `Flow`'s structure. You can paste the output into any Mermaid.js renderer (like the one in the GitHub or VS Code markdown preview) to see your workflow.
+
+```typescript
+import { generateMermaidGraph } from 'flowcraft'
+import { createMyComplexFlow } from './my-flows'
+
+const complexFlow = createMyComplexFlow()
+
+// Generate the Mermaid syntax
+const mermaidSyntax = generateMermaidGraph(complexFlow)
+
+console.log(mermaidSyntax)
+/*
+Outputs something like:
+graph TD
+  StartNode_0[StartNode]
+  DecisionNode_0[DecisionNode]
+  PathANode_0[PathANode]
+  PathBNode_0[PathBNode]
+  EndNode_0[EndNode]
+  StartNode_0 --> DecisionNode_0
+  DecisionNode_0 -- "go_a" --> PathANode_0
+  DecisionNode_0 -- "go_b" --> PathBNode_0
+  PathANode_0 --> EndNode_0
+  PathBNode_0 --> EndNode_0
+*/
+```
+
+This is the fastest way to verify that your graph is connected as you intend.
+
+### Visualizing `GraphBuilder` Flows
+
+You can pass a `Logger` instance to the `GraphBuilder`'s constructor. When you do, it will automatically generate and log a detailed Mermaid.js diagram of the final, "flattened" graph every time you call `.build()`. This diagram shows you the exact structure the `Executor` will run, including inlined sub-workflows and automatically generated parallel blocks.
+
+**Scenario**: You are building a complex workflow and want to see the final executable graph.
+
+```typescript
+import { ConsoleLogger, GraphBuilder } from 'flowcraft'
+
+// Assume `nodeRegistry` and `myComplexGraph` are defined
+
+// Instantiate the builder WITH a logger
+const builder = new GraphBuilder(
+	nodeRegistry,
+	{ /* dependencies */ },
+	{ /* options */ },
+	new ConsoleLogger() // <-- This enables automatic logging
+)
+
+// When you call .build(), the Mermaid graph will be logged to the console.
+const { flow } = builder.build(myComplexGraph)
+```
+
+**Example Log Output**:
+
+The builder will log a complete Mermaid diagram, which you can paste into any compatible renderer (like GitHub's markdown preview) to see the visual graph. This is invaluable for verifying complex wiring, fan-outs, and sub-workflow logic.
+
+```
+[INFO] [GraphBuilder] Flattened Graph
+[INFO] graph TD
+[INFO]   ParallelBlock_0{Parallel Block}
+[INFO]   check_sentiment_0["check_sentiment (llm-condition)"]
+[INFO]   ... and so on ...
+```
+
+> [!TIP]
+> For programmatically built flows (using `.next()`), you can still use the standalone `generateMermaidGraph` utility. However, for declarative workflows, the `GraphBuilder`'s built-in logging is the recommended approach.
+
+## 4. Isolate and Inspect Nodes
+
+If a single node is behaving incorrectly, you can debug it in isolation.
+
+### Isolate with `.run()`
+
+You can test a node's full lifecycle (`prep`, `exec`, `post`) by calling its own `.run()` method without the complexity of the entire workflow.
+
+1. Create a `TypedContext` and manually set any values the node needs.
+2. Call `node.run(context)`.
+3. Assert that the `Context` contains the expected values after the run.
+
+### Inspect with `getNodeById()`
+
+For flows built programmatically (with `.next()`), you can get a direct reference to any node instance using `flow.getNodeById()`. This is useful for inspecting its configuration before the flow runs.
+
+> [!TIP]
+> For flows built with `GraphBuilder`, always use the `nodeMap` returned by the `.build()` method for the most efficient lookup.
+
+```typescript
+const startNode = new Node().withId('start')
+const decisionNode = new Node().withId('decision')
+startNode.next(decisionNode)
+
+const myFlow = new Flow(startNode)
+
+// Get a reference to the node
+const nodeToInspect = myFlow.getNodeById('decision')
+
+// You can now inspect its properties, e.g., its successors
+console.log(nodeToInspect?.successors)
+```
+
+## Common Pitfalls
+
+If your workflow isn't behaving as expected, check for these common issues:
+
+- **Forgetting `await flow.run()`**: Since workflows are asynchronous, forgetting to `await` the top-level `run()` call will cause your script to exit before the workflow can complete.
+- **Context Key Collisions**: In large, composed flows, different sub-flows might accidentally write to the same context key, overwriting each other's data. Using descriptive, unique `ContextKey`s helps prevent this.
+- **Infinite Loops**: If you create a cycle in your graph, make sure your "decider" node has a reliable exit condition. Use the logger to trace the loop and see if the context state is changing as expected on each iteration.
+- **Mutating Objects in Context**: If you place a mutable object (like `{}`, `[]`) in the context, any node can modify it. This can lead to unexpected behavior if one node changes an object that a later node relies on. It's often safer for nodes to create new objects/arrays rather than modifying existing ones in place.

package/docs/guide/best-practices/state-management.md

@@ -0,0 +1,120 @@
+# Best Practices: State Management
+
+The `Context` is the heart of a running workflow, acting as its shared memory. Managing the state within the `Context` effectively is one of the most important skills for building clean, maintainable, and scalable workflows in Flowcraft.
+
+## 1. Always Use `ContextKey` for Type Safety
+
+> [!IMPORTANT]
+> This is the most important rule for state management. Always prefer `contextKey()` over raw strings to access the `Context`.
+
+**Why?**
+
+- **Compile-Time Safety**: The TypeScript compiler will catch typos and type mismatches. If you try to `set` a `number` to a `ContextKey<string>`, your code won't compile.
+- **No Typos**: `ctx.get('user_id')` vs. `ctx.get('userId')` is a common runtime bug. `ctx.get(USER_ID)` is checked by the compiler.
+- **Easy Refactoring**: Renaming a `ContextKey` is a simple, safe refactoring operation in any modern IDE. Renaming a string key across a large codebase is risky.
+
+```typescript
+import { contextKey, TypedContext } from 'flowcraft'
+
+// --- BAD: Using strings ---
+const ctxStrings = new TypedContext()
+ctxStrings.set('user_id', 123) // 'user_id' is just a string, no type info
+const id_bad = ctxStrings.get('user_id') // Type is 'any'
+
+// --- GOOD: Using ContextKey ---
+const USER_ID = contextKey<number>('A key for the user ID')
+
+const ctxTyped = new TypedContext()
+ctxTyped.set(USER_ID, 123)
+const id_good = ctxTyped.get(USER_ID) // Type is 'number | undefined'
+```
+
+## 2. Keep the Context Minimal
+
+The `Context` should not be a dumping ground for all data ever generated in the workflow. A lean context makes your workflow easier to debug and reason about.
+
+**Principle**: A piece of data should only be in the `Context` if it is **required by a subsequent node**.
+
+If a node generates intermediate data that is only used within that same node, it should be stored in a local variable, not written to the `Context`.
+
+```typescript
+import { Node } from 'flowcraft'
+
+class DataProcessorNode extends Node {
+	async exec({ prepRes: data }) {
+		// BAD: Don't put temporary data in the context
+		// ctx.set(TEMP_VALUE, data.a + data.b)
+		// const intermediate = ctx.get(TEMP_VALUE)
+		// return intermediate * 2
+
+		// GOOD: Use local variables for intermediate steps
+		const intermediate = data.a + data.b
+		return intermediate * 2
+	}
+}
+```
+
+## 3. Use `params` for Static Input
+
+Distinguish between *dynamic state* and *static configuration*.
+
+- **Dynamic State**: Data generated by a previous node that changes during the workflow's execution. This belongs in the **`Context`**.
+- **Static Configuration**: Data provided to a node or flow when it starts, which does not change. This belongs in **`params`**.
+
+Using `params` makes your nodes more reusable and their dependencies more explicit.
+
+```typescript
+import { contextKey, Node } from 'flowcraft'
+
+const NUMBER_TO_ADD = contextKey<number>('number_to_add')
+const CURRENT_VALUE = contextKey<number>('current_value')
+
+// --- BAD: Configuration is mixed with state ---
+// This node's behavior depends on a value that must be in the context.
+class AddNumberFromContext extends Node {
+	async exec({ ctx }) {
+		const valueToAdd = ctx.get(NUMBER_TO_ADD) // less reusable
+		return ctx.get(CURRENT_VALUE) + valueToAdd
+	}
+}
+
+// --- GOOD: Configuration is passed as a parameter ---
+// This node is self-contained. Its configuration comes from its params.
+class AddNumberFromParams extends Node {
+	async exec({ ctx, params }) {
+		const valueToAdd = params.amount // much more reusable
+		return ctx.get(CURRENT_VALUE) + valueToAdd
+	}
+}
+
+// Usage:
+const node = new AddNumberFromParams().withParams({ amount: 10 })
+```
+
+## 4. Isolate State Within Sub-Flows
+
+When you use composition (a `Flow` within a `Flow`), the sub-flow shares the same `Context` as its parent. To maintain modularity, a sub-flow should not pollute the parent's `Context` with its own internal, temporary state.
+
+**Best Practice**: A sub-flow should only write its final, essential outputs to the `Context`. If possible, use a dedicated output node within the sub-flow to aggregate results and clean up any temporary keys.
+
+The `SubWorkflowNode` in the `sandbox/4.dag` example demonstrates an advanced pattern for this. By defining explicit `input` and `output` maps, it creates a clean data boundary, preventing any leakage of temporary state. For a detailed explanation of this powerful pattern, see the guide on **[Data Flow in Sub-Workflows](./sub-workflow-data.md)**.
+
+## 5. Plan for Serialization
+
+> [!WARNING]
+> **Standard `JSON.stringify` is Lossy!**
+> While the `InMemoryExecutor` can handle any JavaScript object, your workflow's state may need to be saved or sent over a network (as in the `BullMQExecutor` example). Standard `JSON.stringify` will not correctly preserve complex data types and can lead to silent data loss or bugs.
+
+**Common data types that break `JSON.stringify`:**
+
+- `Date` objects (are converted to strings)
+- `Map` and `Set` objects (are converted to empty objects `{}`)
+- Custom class instances (lose their methods and class identity)
+- `Symbol` keys (are dropped entirely)
+
+To build robust, stateful workflows, it is a best practice to use a library that handles this automatically.
+
+> [!TIP]
+> Use a library like [`superjson`](https://github.com/blitz-js/superjson) to serialize and deserialize your context. It transparently handles all common JavaScript types, ensuring that the state you save is the same as the state you load.
+
+The **[Advanced RAG Agent](https://github.com/gorango/tree/master/sandbox/6.rag/) example** was specifically designed to demonstrate this principle. It uses `superjson` to correctly manage a `Context` containing `Map`, `Date`, and custom class instances, making its state reliable and easy to inspect.

package/docs/guide/best-practices/sub-workflow-data.md

@@ -0,0 +1,95 @@
+# Best Practices: Data Flow in Sub-Workflows
+
+When you compose workflows, managing the flow of data between the parent and child is critical for creating modular and predictable systems. By default, Flowcraft uses a **shared context**, which is simple but can lead to tight coupling.
+
+This guide describes a powerful pattern for creating an explicit data boundary between flows, as demonstrated by the `SubWorkflowNode` in the Dynamic Graph Engine example.
+
+## The Problem: A Leaky Context
+
+Because the default behavior is a shared context, a sub-workflow has access to the parent's entire state. This can cause two main problems in large systems:
+
+- **Input Pollution**: The sub-workflow has access to *everything* in the parent's context, making its dependencies implicit and hard to track. It's not clear from the sub-workflow's definition what data it actually needs to operate.
+- **Output Pollution**: Temporary, internal state from the sub-workflow can leak out and accidentally overwrite values in the parent's context, causing unexpected side effects in later stages of the parent flow.
+
+## The Solution: Explicit Input and Output Mapping
+
+To solve this, you can create a specialized `Node` (like the `SubWorkflowNode` in our examples) that acts as a gatekeeper. Instead of sharing the context, this node creates a *new, temporary context* for the sub-workflow and explicitly maps data in and out. Its configuration defines this data contract.
+
+```json
+{
+	"id": "my_sub_workflow_node",
+	"type": "sub-workflow",
+	"data": {
+		"workflowId": 201,
+		"inputs": {
+			"sub_flow_key": "parent_flow_key"
+		},
+		"outputs": {
+			"parent_flow_key_for_result": "sub_flow_output_key"
+		}
+	}
+}
+```
+
+### 1. The `inputs` Map
+
+The `inputs` map defines **what data flows from the parent context into the sub-workflow's context**.
+
+- `"sub_flow_key"`: The `ContextKey` that will be used *inside* the sub-workflow.
+- `"parent_flow_key"`: The `ContextKey` in the *parent* flow whose value will be copied.
+
+**How it works:**
+Before the sub-workflow runs, the `SubWorkflowNode` creates a new, empty context. It then iterates through the `inputs` map. For each entry, it reads the value from `parent_flow_key` in the parent context and writes it to `sub_flow_key` in the sub-workflow's fresh context.
+
+This ensures the sub-workflow only receives the exact data it needs, making it a pure, reusable component with clear dependencies.
+
+### 2. The `outputs` Map
+
+The `outputs` map defines **what data flows from the sub-workflow's context back out to the parent context**.
+
+- `"parent_flow_key_for_result"`: The `ContextKey` in the *parent* flow where the result will be stored.
+- `"sub_flow_output_key"`: The `ContextKey` in the *sub-workflow* whose value will be copied.
+
+**How it works:**
+After the sub-workflow completes, the `SubWorkflowNode` iterates through the `outputs` map. For each entry, it reads the value from `sub_flow_output_key` in the sub-workflow's context and writes it to `parent_flow_key_for_result` in the parent's context.
+
+This prevents any temporary or internal variables from the sub-workflow from leaking out and polluting the parent's state. Only the explicitly defined outputs are passed back.
+
+### Example: A User Greeter Sub-Workflow
+
+Imagine a parent flow that has a `USER_OBJECT`. We want to call a sub-workflow that generates a greeting.
+
+**Parent Flow Graph Node:**
+
+```json
+{
+	"id": "generate_greeting",
+	"type": "sub-workflow",
+	"data": {
+		"workflowId": 101,
+		"inputs": {
+			"user_to_greet": "USER_OBJECT"
+		},
+		"outputs": {
+			"FINAL_GREETING_MESSAGE": "greeting_result"
+		}
+	}
+}
+```
+
+**Sub-Workflow (ID: 101):**
+
+1. **Starts**: Receives a new context containing only `user_to_greet` (copied from the parent's `USER_OBJECT`).
+2. **Internal Node `A`**: Reads `user_to_greet.name` and creates a string: `Hello, Alice!`. It stores this in an internal key, `temp_message`.
+3. **Internal Node `B`**: Reads `temp_message`, adds an emoji, and stores the final string in `greeting_result`.
+4. **Ends**.
+
+**Data Flow:**
+
+1. The `SubWorkflowNode` copies `parentContext.get('USER_OBJECT')` into `subContext.set('user_to_greet', ...)`.
+2. Sub-workflow `101` runs with its own isolated context. The `temp_message` key exists only within this context.
+3. When it finishes, its context contains `greeting_result` with the value `"Hello, Alice! 👋"`.
+4. The `SubWorkflowNode` copies `subContext.get('greeting_result')` into `parentContext.set('FINAL_GREETING_MESSAGE', ...)`.
+5. The parent flow now has a `FINAL_GREETING_MESSAGE` key, but is completely unaware of the `temp_message` key.
+
+By enforcing this explicit data contract, you can build complex, nested workflows that are as easy to reason about as pure functions.

package/docs/guide/best-practices/testing.md

@@ -0,0 +1,187 @@
+# Best Practices: Testing Workflows
+
+Testing is crucial for building robust and reliable workflows. Flowcraft's design, which separates data preparation (`prep`), core logic (`exec`), and state updates (`post`), makes testing straightforward. This guide covers strategies for testing individual nodes and entire flows.
+
+## Testing Individual Nodes
+
+The most important part of a `Node` to test is its `exec` method, as it contains the core business logic. Because `exec` is designed to be pure—receiving all its input from `prep` and not accessing the `Context` directly—you can test it in complete isolation.
+
+### Strategy: Test `exec` Directly
+
+You can instantiate your node and call its `_exec` method (the internal method that includes retry logic) or `exec` method directly, providing mock `NodeArgs`.
+
+**Example: Testing a data transformation node**
+
+Let's assume you have this node:
+
+```typescript
+import { Node } from 'flowcraft'
+
+// src/nodes.ts
+class UserProcessorNode extends Node<{ name: string, email: string }, { fullName: string, domain: string }> {
+	async exec({ prepRes: user }) {
+		if (!user.name || !user.email) {
+			throw new Error('Invalid user data')
+		}
+		return {
+			fullName: user.name.toUpperCase(),
+			domain: user.email.split('@')[1],
+		}
+	}
+}
+```
+
+Here's how you could test it using a testing framework like Vitest or Jest:
+
+```typescript
+import { NullLogger } from 'flowcraft'
+// src/nodes.test.ts
+import { UserProcessorNode } from './nodes'
+
+describe('UserProcessorNode', () => {
+	it('should correctly process valid user data', async () => {
+		const node = new UserProcessorNode()
+		const mockArgs = {
+			prepRes: { name: 'Alice', email: 'alice@example.com' },
+			logger: new NullLogger(),
+			// other args can be mocked as needed
+		}
+
+		const result = await node.exec(mockArgs)
+
+		expect(result).toEqual({
+			fullName: 'ALICE',
+			domain: 'example.com',
+		})
+	})
+
+	it('should throw an error for invalid user data', async () => {
+		const node = new UserProcessorNode()
+		const mockArgs = {
+			prepRes: { name: '', email: 'no-name@test.com' },
+			logger: new NullLogger(),
+		}
+
+		await expect(node.exec(mockArgs)).rejects.toThrow('Invalid user data')
+	})
+})
+```
+
+## Testing `prep` and `post`
+
+To test the full lifecycle of a node, including its interaction with the `Context`, you can use the node's `.run()` method.
+
+### Strategy: Use `.run()` and inspect the Context
+
+1. Create a `TypedContext` and pre-populate it with any data your node's `prep` phase needs.
+2. Call `node.run(context)`.
+3. Assert that the `Context` contains the expected values after the run, which tests your `post` logic.
+
+**Example: Testing a node that interacts with context**
+
+```typescript
+import { contextKey, Node } from 'flowcraft'
+
+// src/nodes.ts
+const INPUT = contextKey<number>('input')
+const OUTPUT = contextKey<number>('output')
+
+class AddTenNode extends Node {
+	async prep({ ctx }) {
+		return ctx.get(INPUT) || 0
+	}
+
+	async exec({ prepRes: value }) {
+		return value + 10
+	}
+
+	async post({ ctx, execRes: result }) {
+		ctx.set(OUTPUT, result)
+	}
+}
+```
+
+And the test:
+
+```typescript
+// src/nodes.test.ts
+import { TypedContext } from 'flowcraft'
+
+describe('AddTenNode', () => {
+	it('should read from, and write to, the context correctly', async () => {
+		const node = new AddTenNode()
+		const context = new TypedContext([
+			[INPUT, 5]
+		])
+
+		await node.run(context)
+
+		expect(context.get(OUTPUT)).toBe(15)
+	})
+})
+```
+
+## Testing Flows
+
+When testing a `Flow`, you are performing an integration test to ensure that all the nodes work together correctly and that the `Context` state evolves as expected.
+
+### Strategy: Run the Flow and Assert Final Context State
+
+The approach is similar to testing a single node with `.run()`, but you do it for the entire `Flow`.
+
+```typescript
+import { contextKey, SequenceFlow, TypedContext } from 'flowcraft'
+
+const INITIAL_DATA = contextKey<string>('initial_data')
+const FINAL_RESULT = contextKey<string>('final_result')
+const SOME_INTERMEDIATE_VALUE = contextKey<any>('intermediate')
+
+describe('DataProcessingFlow', () => {
+	it('should run the full sequence and produce the correct final output', async () => {
+		// Assume NodeA, NodeB, NodeC are defined elsewhere
+		const flow = new SequenceFlow(new NodeA(), new NodeB(), new NodeC())
+		const context = new TypedContext([
+			[INITIAL_DATA, 'start']
+		])
+
+		await flow.run(context)
+
+		// Assert the final state of the context after the whole flow has run
+		expect(context.get(FINAL_RESULT)).toBe('expected-final-value')
+		expect(context.get(SOME_INTERMEDIATE_VALUE)).toBeUndefined() // Verify cleanup if applicable
+	})
+})
+```
+
+### Testing Branching Logic
+
+To test conditional branching, create separate tests for each path. In each test, set up the `Context` in a way that forces the flow to take the specific branch you want to verify.
+
+```typescript
+import { contextKey } from 'flowcraft'
+
+const IS_VALID = contextKey<boolean>('is_valid')
+const PATH_TAKEN = contextKey<string>('path_taken')
+
+it('should take the "success" path when data is valid', async () => {
+	const flow = createMyConditionalFlow()
+	const context = new TypedContext([
+		[IS_VALID, true] // Force the success path
+	])
+
+	await flow.run(context)
+
+	expect(context.get(PATH_TAKEN)).toBe('success-path')
+})
+
+it('should take the "error" path when data is invalid', async () => {
+	const flow = createMyConditionalFlow()
+	const context = new TypedContext([
+		[IS_VALID, false] // Force the error path
+	])
+
+	await flow.run(context)
+
+	expect(context.get(PATH_TAKEN)).toBe('error-path')
+})
+```