flowcraft 1.0.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (148) hide show
  1. package/.editorconfig +9 -0
  2. package/LICENSE +21 -0
  3. package/README.md +249 -0
  4. package/config/tsconfig.json +21 -0
  5. package/config/tsup.config.ts +11 -0
  6. package/config/vitest.config.ts +11 -0
  7. package/docs/.vitepress/config.ts +105 -0
  8. package/docs/api-reference/builder.md +158 -0
  9. package/docs/api-reference/fn.md +142 -0
  10. package/docs/api-reference/index.md +38 -0
  11. package/docs/api-reference/workflow.md +126 -0
  12. package/docs/guide/advanced-guides/cancellation.md +117 -0
  13. package/docs/guide/advanced-guides/composition.md +68 -0
  14. package/docs/guide/advanced-guides/custom-executor.md +180 -0
  15. package/docs/guide/advanced-guides/error-handling.md +135 -0
  16. package/docs/guide/advanced-guides/logging.md +106 -0
  17. package/docs/guide/advanced-guides/middleware.md +106 -0
  18. package/docs/guide/advanced-guides/observability.md +175 -0
  19. package/docs/guide/best-practices/debugging.md +182 -0
  20. package/docs/guide/best-practices/state-management.md +120 -0
  21. package/docs/guide/best-practices/sub-workflow-data.md +95 -0
  22. package/docs/guide/best-practices/testing.md +187 -0
  23. package/docs/guide/builders.md +157 -0
  24. package/docs/guide/functional-api.md +133 -0
  25. package/docs/guide/index.md +178 -0
  26. package/docs/guide/recipes/creating-a-loop.md +113 -0
  27. package/docs/guide/recipes/data-processing-pipeline.md +123 -0
  28. package/docs/guide/recipes/fan-out-fan-in.md +112 -0
  29. package/docs/guide/recipes/index.md +15 -0
  30. package/docs/guide/recipes/resilient-api-call.md +110 -0
  31. package/docs/guide/tooling/graph-validation.md +160 -0
  32. package/docs/guide/tooling/mermaid.md +156 -0
  33. package/docs/index.md +56 -0
  34. package/eslint.config.js +16 -0
  35. package/package.json +40 -0
  36. package/pnpm-workspace.yaml +2 -0
  37. package/sandbox/1.basic/README.md +45 -0
  38. package/sandbox/1.basic/package.json +16 -0
  39. package/sandbox/1.basic/src/flow.ts +17 -0
  40. package/sandbox/1.basic/src/main.ts +22 -0
  41. package/sandbox/1.basic/src/nodes.ts +112 -0
  42. package/sandbox/1.basic/src/utils.ts +35 -0
  43. package/sandbox/1.basic/tsconfig.json +3 -0
  44. package/sandbox/2.research/README.md +46 -0
  45. package/sandbox/2.research/package.json +16 -0
  46. package/sandbox/2.research/src/flow.ts +14 -0
  47. package/sandbox/2.research/src/main.ts +31 -0
  48. package/sandbox/2.research/src/nodes.ts +108 -0
  49. package/sandbox/2.research/src/utils.ts +45 -0
  50. package/sandbox/2.research/src/visualize.ts +29 -0
  51. package/sandbox/2.research/tsconfig.json +3 -0
  52. package/sandbox/3.parallel/README.md +65 -0
  53. package/sandbox/3.parallel/package.json +16 -0
  54. package/sandbox/3.parallel/src/main.ts +45 -0
  55. package/sandbox/3.parallel/src/nodes.ts +43 -0
  56. package/sandbox/3.parallel/src/utils.ts +25 -0
  57. package/sandbox/3.parallel/tsconfig.json +3 -0
  58. package/sandbox/4.dag/README.md +179 -0
  59. package/sandbox/4.dag/data/1.blog-post/100.json +60 -0
  60. package/sandbox/4.dag/data/1.blog-post/README.md +25 -0
  61. package/sandbox/4.dag/data/2.job-application/200.json +103 -0
  62. package/sandbox/4.dag/data/2.job-application/201.json +31 -0
  63. package/sandbox/4.dag/data/2.job-application/202.json +31 -0
  64. package/sandbox/4.dag/data/2.job-application/README.md +58 -0
  65. package/sandbox/4.dag/data/3.customer-review/300.json +141 -0
  66. package/sandbox/4.dag/data/3.customer-review/301.json +31 -0
  67. package/sandbox/4.dag/data/3.customer-review/302.json +28 -0
  68. package/sandbox/4.dag/data/3.customer-review/README.md +71 -0
  69. package/sandbox/4.dag/data/4.content-moderation/400.json +161 -0
  70. package/sandbox/4.dag/data/4.content-moderation/401.json +47 -0
  71. package/sandbox/4.dag/data/4.content-moderation/402.json +46 -0
  72. package/sandbox/4.dag/data/4.content-moderation/403.json +31 -0
  73. package/sandbox/4.dag/data/4.content-moderation/README.md +83 -0
  74. package/sandbox/4.dag/package.json +19 -0
  75. package/sandbox/4.dag/src/main.ts +73 -0
  76. package/sandbox/4.dag/src/nodes.ts +134 -0
  77. package/sandbox/4.dag/src/registry.ts +87 -0
  78. package/sandbox/4.dag/src/types.ts +25 -0
  79. package/sandbox/4.dag/src/utils.ts +42 -0
  80. package/sandbox/4.dag/tsconfig.json +3 -0
  81. package/sandbox/5.distributed/.env.example +1 -0
  82. package/sandbox/5.distributed/README.md +88 -0
  83. package/sandbox/5.distributed/data/1.blog-post/100.json +59 -0
  84. package/sandbox/5.distributed/data/1.blog-post/README.md +25 -0
  85. package/sandbox/5.distributed/data/2.job-application/200.json +103 -0
  86. package/sandbox/5.distributed/data/2.job-application/201.json +30 -0
  87. package/sandbox/5.distributed/data/2.job-application/202.json +30 -0
  88. package/sandbox/5.distributed/data/2.job-application/README.md +58 -0
  89. package/sandbox/5.distributed/data/3.customer-review/300.json +141 -0
  90. package/sandbox/5.distributed/data/3.customer-review/301.json +31 -0
  91. package/sandbox/5.distributed/data/3.customer-review/302.json +57 -0
  92. package/sandbox/5.distributed/data/3.customer-review/README.md +71 -0
  93. package/sandbox/5.distributed/data/4.content-moderation/400.json +173 -0
  94. package/sandbox/5.distributed/data/4.content-moderation/401.json +47 -0
  95. package/sandbox/5.distributed/data/4.content-moderation/402.json +46 -0
  96. package/sandbox/5.distributed/data/4.content-moderation/403.json +31 -0
  97. package/sandbox/5.distributed/data/4.content-moderation/README.md +83 -0
  98. package/sandbox/5.distributed/package.json +20 -0
  99. package/sandbox/5.distributed/src/client.ts +124 -0
  100. package/sandbox/5.distributed/src/executor.ts +69 -0
  101. package/sandbox/5.distributed/src/nodes.ts +136 -0
  102. package/sandbox/5.distributed/src/registry.ts +101 -0
  103. package/sandbox/5.distributed/src/types.ts +45 -0
  104. package/sandbox/5.distributed/src/utils.ts +69 -0
  105. package/sandbox/5.distributed/src/worker.ts +217 -0
  106. package/sandbox/5.distributed/tsconfig.json +3 -0
  107. package/sandbox/6.rag/.env.example +1 -0
  108. package/sandbox/6.rag/README.md +60 -0
  109. package/sandbox/6.rag/data/README.md +31 -0
  110. package/sandbox/6.rag/data/rag.json +58 -0
  111. package/sandbox/6.rag/documents/sample-cascade.txt +11 -0
  112. package/sandbox/6.rag/package.json +18 -0
  113. package/sandbox/6.rag/src/main.ts +52 -0
  114. package/sandbox/6.rag/src/nodes/GenerateEmbeddingsNode.ts +54 -0
  115. package/sandbox/6.rag/src/nodes/LLMProcessNode.ts +48 -0
  116. package/sandbox/6.rag/src/nodes/LoadAndChunkNode.ts +40 -0
  117. package/sandbox/6.rag/src/nodes/StoreInVectorDBNode.ts +36 -0
  118. package/sandbox/6.rag/src/nodes/VectorSearchNode.ts +53 -0
  119. package/sandbox/6.rag/src/nodes/index.ts +28 -0
  120. package/sandbox/6.rag/src/registry.ts +23 -0
  121. package/sandbox/6.rag/src/types.ts +44 -0
  122. package/sandbox/6.rag/src/utils.ts +77 -0
  123. package/sandbox/6.rag/tsconfig.json +3 -0
  124. package/sandbox/tsconfig.json +13 -0
  125. package/src/builder/collection.test.ts +287 -0
  126. package/src/builder/collection.ts +269 -0
  127. package/src/builder/graph.test.ts +406 -0
  128. package/src/builder/graph.ts +336 -0
  129. package/src/builder/graph.types.ts +104 -0
  130. package/src/builder/index.ts +3 -0
  131. package/src/context.ts +111 -0
  132. package/src/errors.ts +34 -0
  133. package/src/executor.ts +29 -0
  134. package/src/executors/in-memory.test.ts +93 -0
  135. package/src/executors/in-memory.ts +140 -0
  136. package/src/functions.test.ts +191 -0
  137. package/src/functions.ts +117 -0
  138. package/src/index.ts +5 -0
  139. package/src/logger.ts +41 -0
  140. package/src/types.ts +75 -0
  141. package/src/utils/graph.test.ts +144 -0
  142. package/src/utils/graph.ts +182 -0
  143. package/src/utils/index.ts +3 -0
  144. package/src/utils/mermaid.test.ts +239 -0
  145. package/src/utils/mermaid.ts +133 -0
  146. package/src/utils/sleep.ts +20 -0
  147. package/src/workflow.test.ts +622 -0
  148. package/src/workflow.ts +561 -0
@@ -0,0 +1,142 @@
1
+ # API Reference: Functional Helpers
2
+
3
+ This document covers the functions provided by Flowcraft for a more functional-style approach to creating `Node` instances and simple workflows. All helpers are imported from the main `flowcraft` package.
4
+
5
+ ```typescript
6
+ import {
7
+ composeContext,
8
+ contextNode,
9
+ lens,
10
+ mapNode,
11
+ pipeline,
12
+ transformNode,
13
+ } from 'flowcraft'
14
+ ```
15
+
16
+ ## `mapNode<TIn, TOut>(fn)`
17
+
18
+ Creates a `Node` from a simple, pure function that transforms an input to an output. The node's `params` object is passed as the single argument to the provided function `fn`. The result of `fn` becomes the `exec` result of the node.
19
+
20
+ ### Parameters
21
+
22
+ - `fn: (input: TIn) => TOut | Promise<TOut>`: A synchronous or asynchronous function that takes an input object and returns a result. `TIn` corresponds to the type of the node's `params`.
23
+
24
+ ### Returns
25
+
26
+ - `Node<TIn, TOut>`: A new `Node` instance that wraps the function.
27
+
28
+ ### Example
29
+
30
+ ```typescript
31
+ import { mapNode } from 'flowcraft'
32
+
33
+ // Define a reusable node that doubles the 'value' param
34
+ const doublerNode = mapNode((params: { value: number }) => params.value * 2)
35
+
36
+ // Use it in a flow
37
+ doublerNode.withParams({ value: 5 }) // This node will produce the result 10
38
+ ```
39
+
40
+ ## `contextNode<TIn, TOut>(fn)`
41
+
42
+ Creates a `Node` from a function that requires access to the shared `Context` in addition to its `params`.
43
+
44
+ ### Parameters
45
+
46
+ - `fn: (ctx: Context, input: TIn) => TOut | Promise<TOut>`: A function that takes the `Context` as its first argument and the node's `params` as its second. The result of `fn` becomes the `exec` result of the node.
47
+
48
+ ### Returns
49
+
50
+ - `Node<TIn, TOut>`: A new `Node` instance that wraps the function.
51
+
52
+ ### Example
53
+
54
+ ```typescript
55
+ import { contextKey, contextNode } from 'flowcraft'
56
+
57
+ const USER_NAME = contextKey<string>('user_name')
58
+
59
+ // A node that constructs a greeting using a value from the context
60
+ const greetingNode = contextNode(ctx => `Hello, ${ctx.get(USER_NAME)}!`)
61
+ ```
62
+
63
+ ## `transformNode(...transforms)`
64
+
65
+ Creates a `Node` that is used purely for its side effect of modifying the `Context`. It does not produce an `exec` result. Its logic runs in the `prep` phase. It is often used with `ContextTransform` functions created by a `lens`.
66
+
67
+ ### Parameters
68
+
69
+ - `...transforms: ContextTransform[]`: A sequence of `ContextTransform` functions. A `ContextTransform` is a function of the shape `(ctx: Context) => Context`.
70
+
71
+ ### Returns
72
+
73
+ - `Node`: A new `Node` instance that will apply the context transformations when it runs.
74
+
75
+ ### Example
76
+
77
+ ```typescript
78
+ import { lens, transformNode } from 'flowcraft'
79
+
80
+ const NAME = contextKey<string>('name')
81
+ const AGE = contextKey<number>('age')
82
+ const nameLens = lens(NAME)
83
+ const ageLens = lens(AGE)
84
+
85
+ // A single node that sets both name and age in the context
86
+ const setupContextNode = transformNode(
87
+ nameLens.set('Alice'),
88
+ ageLens.set(30)
89
+ )
90
+ ```
91
+
92
+ ## `pipeline(...nodes)`
93
+
94
+ A functional-style alias for the `SequenceFlow` builder. It constructs a linear `Flow` where each node executes in the order it is provided.
95
+
96
+ ### Parameters
97
+
98
+ - `...nodes: Node[]`: A sequence of `Node` instances to chain together.
99
+
100
+ ### Returns
101
+
102
+ - `Flow`: A `Flow` instance representing the linear sequence.
103
+
104
+ ## `lens<T>(key)`
105
+
106
+ Creates a `ContextLens` object, which provides a type-safe way to generate functions that interact with a specific key in the `Context`.
107
+
108
+ ### Parameters
109
+
110
+ - `key: ContextKey<T>`: The `ContextKey` to focus on.
111
+
112
+ ### Returns
113
+
114
+ - `ContextLens<T>`: An object with the following methods:
115
+ - `.get(ctx: Context): T | undefined`: Retrieves the value for the key from the context.
116
+ - `.set(value: T): ContextTransform`: Returns a function that, when called with a `Context`, will set the key to the provided `value`.
117
+ - `.update(fn: (current: T | undefined) => T): ContextTransform`: Returns a function that updates the key's value based on its current value.
118
+
119
+ ### Example
120
+
121
+ ```typescript
122
+ const NAME = contextKey<string>('name')
123
+ const nameLens = lens(NAME)
124
+
125
+ // Create a transform function that sets the name to 'Alice'
126
+ const setNameTransform = nameLens.set('Alice')
127
+
128
+ // Create a node that applies this transform
129
+ const setNode = transformNode(setNameTransform)
130
+ ```
131
+
132
+ ## `composeContext(...transforms)`
133
+
134
+ Composes multiple `ContextTransform` functions into a single `ContextTransform` function. The transformations are applied in the order they are provided.
135
+
136
+ ### Parameters
137
+
138
+ - `...transforms: ContextTransform[]`: A sequence of `ContextTransform` functions.
139
+
140
+ ### Returns
141
+
142
+ - `ContextTransform`: A single function that applies all transformations.
@@ -0,0 +1,38 @@
1
+ # API Reference
2
+
3
+ This section provides a detailed, technical breakdown of the core classes, functions, and types available in the Flowcraft framework. It is intended as a reference for when you know what component you need to use and want to understand its specific capabilities.
4
+
5
+ All components are imported from the main `flowcraft` package. For a more narrative-driven explanation of these concepts, please see the **[Guide](../guide/)**.
6
+
7
+ ## Core Workflow API
8
+
9
+ This is the main entry point for the most essential components of the framework.
10
+
11
+ - **[Core Workflow API](./workflow.md)**: Detailed documentation for the fundamental building blocks:
12
+ - `Node`: The base class for a unit of work.
13
+ - `Flow`: The orchestrator for a graph of nodes.
14
+ - `IExecutor` and `InMemoryExecutor`: The execution engine contract and its default implementation.
15
+ - `Context`, `TypedContext`, `ContextKey`, and `contextKey()`: For type-safe state management.
16
+ - `Logger` implementations: `ConsoleLogger` and `NullLogger`.
17
+ - Error types: `WorkflowError` and `AbortError`.
18
+
19
+ ## Builder API
20
+
21
+ This module contains classes that simplify the creation of common and complex workflow patterns.
22
+
23
+ - **[Builder API](./builder.md)**: Documentation for the builder classes:
24
+ - `SequenceFlow`: For creating simple, linear workflows.
25
+ - `BatchFlow`: For processing a collection of items sequentially.
26
+ - `ParallelBatchFlow`: For processing a collection of items concurrently.
27
+ - `GraphBuilder`: For constructing a `Flow` from a declarative, type-safe graph definition.
28
+
29
+ ## Functional API
30
+
31
+ This module provides a set of functions for creating nodes and pipelines in a more functional programming style.
32
+
33
+ - **[Functional API](./fn.md)**: Documentation for the functional helpers:
34
+ - `mapNode`: Creates a `Node` from a simple, pure function.
35
+ - `contextNode`: Creates a `Node` from a function that requires `Context` access.
36
+ - `pipeline`: A functional alias for creating a linear `SequenceFlow`.
37
+ - `transformNode`: Creates a `Node` for declaratively updating the `Context` using lenses.
38
+ - `lens` and `composeContext`: Utilities for functional context manipulation.
@@ -0,0 +1,126 @@
1
+ # API Reference: Core Workflow
2
+
3
+ This document covers the core classes and types exported from the main `flowcraft` entry point. These are the fundamental building blocks of any workflow.
4
+
5
+ ```typescript
6
+ import {
7
+ Context,
8
+ ContextKey,
9
+ contextKey,
10
+ Flow,
11
+ InMemoryExecutor,
12
+ Node,
13
+ TypedContext,
14
+ // ...and more
15
+ } from 'flowcraft'
16
+ ```
17
+
18
+ ## `Node<PrepRes, ExecRes, PostRes>`
19
+
20
+ The base class for a single unit of work.
21
+
22
+ ### Constructor
23
+
24
+ `new Node(options?: NodeOptions)`
25
+
26
+ - `options`: An optional object to configure the node's behavior.
27
+ - `maxRetries?: number`: Total number of `exec` attempts. Defaults to `1`.
28
+ - `wait?: number`: Milliseconds to wait between failed `exec` attempts. Defaults to `0`.
29
+
30
+ ### Lifecycle Methods
31
+
32
+ These methods are designed to be overridden in your custom `Node` subclasses.
33
+
34
+ - `async prep(args: NodeArgs): Promise<PrepRes>`: Prepares data for execution. Runs before `exec`. Ideal for reading from the context.
35
+ - `async exec(args: NodeArgs<PrepRes>): Promise<ExecRes>`: Performs the core, isolated logic. Its result is passed to `post`. This is the only phase that is retried on failure.
36
+ - `async post(args: NodeArgs<PrepRes, ExecRes>): Promise<PostRes>`: Processes results and determines the next step. Runs after `exec`. Ideal for writing to the context. Should return an action string. The default return is `DEFAULT_ACTION`.
37
+ - `async execFallback(args: NodeArgs<PrepRes>): Promise<ExecRes>`: Runs if all `exec` retries fail. If not implemented, the error will be re-thrown.
38
+
39
+ ### Fluent API Methods
40
+
41
+ > [!IMPORTANT]
42
+ > These methods are **immutable**. They return a *new* `Node` instance for creating data processing pipelines and do not modify the original node. You must chain them or assign the result to a new variable.
43
+
44
+ - `.map<NewRes>(fn)`: Transforms the `exec` result into a new type.
45
+ - `.toContext(key)`: Stores the `exec` result in the `Context` using the provided `ContextKey`.
46
+ - `.filter(predicate)`: Conditionally proceeds. Returns `DEFAULT_ACTION` if the predicate is true, `FILTER_FAILED` otherwise.
47
+ - `.tap(fn)`: Performs a side-effect with the `exec` result without modifying it.
48
+ - `.withLens(lens, value)`: Applies a context mutation using a `ContextLens` before this node's `prep` phase.
49
+
50
+ ### Other Methods
51
+
52
+ - `.next(node, action?)`: Connects this node to a successor. Returns the successor node for chaining.
53
+ - `.withParams(params)`: Sets or merges parameters for the node.
54
+ - `.run(ctx, options?)`: Runs the node as a standalone unit using an `IExecutor`.
55
+
56
+ ## `Flow`
57
+
58
+ A special `Node` that acts as a container for a graph of other nodes and their shared middleware.
59
+
60
+ `extends Node`
61
+
62
+ ### Constructor
63
+
64
+ `new Flow(startNode?: AbstractNode)`
65
+
66
+ - `startNode`: The node where the flow's execution should begin.
67
+
68
+ ### Methods
69
+
70
+ - `.use(fn: Middleware)`: Adds a middleware function that the `Executor` will apply to every node within this flow.
71
+ - `.start(node)`: Sets the starting node of the flow. Returns the start node.
72
+ - `.run(ctx, options?)`: Runs the entire flow using an `IExecutor`. This is the main entry point for executing a workflow. The method returns the action from the last node executed.
73
+ - `.exec(args)`: This lifecycle method is called when a `Flow` is used as a sub-flow (a node within another flow). It contains the logic to orchestrate the sub-flow's internal graph from start to finish.
74
+ - `.getNodeById(id: string | number): AbstractNode | undefined`: Finds a node within the flow's graph by its unique ID. This method performs a breadth-first search from the `startNode` and is useful for debugging or dynamic modifications of programmatically-built flows.
75
+
76
+ > [!WARNING]
77
+ > `.getNodeById()` traverses the graph on each call (O(V+E) complexity). For flows built with `GraphBuilder`, it is much more efficient to use the `nodeMap` returned by the `.build()` method for O(1) lookups.
78
+
79
+ ## `IExecutor` and `InMemoryExecutor`
80
+
81
+ - `IExecutor`: The interface that defines the contract for a workflow execution engine.
82
+ - `InMemoryExecutor`: The standard `IExecutor` implementation for running flows in-memory. This is the default executor if none is provided.
83
+
84
+ ### `InMemoryExecutor` Constructor
85
+
86
+ `new InMemoryExecutor()`
87
+
88
+ ## `Context` and `TypedContext`
89
+
90
+ The shared memory of a workflow.
91
+
92
+ - `Context`: The interface that defines the contract for a context object.
93
+ - `TypedContext`: The standard `Map`-based implementation of the `Context` interface.
94
+
95
+ ### `TypedContext` Constructor
96
+
97
+ `new TypedContext(initialData?)`
98
+
99
+ - `initialData`: An optional iterable (like an array of `[key, value]` pairs) to initialize the context with.
100
+
101
+ ### Methods
102
+
103
+ - `.get<T>(key)`: Retrieves a value from the context. Can be called with a `ContextKey<T>` (returns `T | undefined`) or a `string` (returns `any`).
104
+ - `.set<T>(key, value)`: Stores a value in the context.
105
+ - `.has(key)`: Checks if a key exists in the context.
106
+
107
+ ## `ContextKey` and `contextKey()`
108
+
109
+ The mechanism for type-safe access to the `Context`.
110
+
111
+ > [!IMPORTANT]
112
+ > Always prefer `contextKey()` over raw strings to access the context. This provides compile-time type checking and prevents common bugs from typos.
113
+
114
+ - `ContextKey<T>`: An opaque type representing a key for a value of type `T`.
115
+ - `contextKey<T>(description?: string)`: A factory function that creates a new, unique `ContextKey<T>`. The `description` is used for debugging and is not functionally significant.
116
+
117
+ ## Logger Interfaces
118
+
119
+ - `Logger`: The interface that any logger passed to a flow must implement (`debug`, `info`, `warn`, `error`).
120
+ - `ConsoleLogger`: A default implementation that logs messages to the `console`.
121
+ - `NullLogger`: A default implementation that performs no action. This is the framework's default if no logger is provided, ensuring it is silent by default.
122
+
123
+ ## Error Types
124
+
125
+ - `WorkflowError`: A custom error wrapper that provides context about a failure (`nodeName`, `phase`, `originalError`).
126
+ - `AbortError`: The error thrown when a workflow is cancelled via an `AbortSignal`.
@@ -0,0 +1,117 @@
1
+ # Cancellation Support
2
+
3
+ In many applications, especially those involving long-running asynchronous operations, you need a way to gracefully abort a process that is already in flight. Flowcraft provides robust cancellation support out of the box by integrating with the standard web `AbortController` and `AbortSignal` APIs.
4
+
5
+ ## How It Works
6
+
7
+ When you run a `Flow`, you can pass an `AbortController` instance in the `RunOptions`.
8
+
9
+ ```typescript
10
+ const controller = new AbortController()
11
+ flow.run(context, { controller })
12
+ ```
13
+
14
+ > [!IMPORTANT]
15
+ > Flowcraft passes the `AbortSignal` to every node, but it's your responsibility to use it. The framework cannot magically interrupt your asynchronous code. You must design your `exec` logic to listen for the abort event and stop its work, as shown in the examples below.
16
+
17
+ Calling `controller.abort()` will cause the currently running asynchronous operation to throw an `AbortError`, which immediately and cleanly halts the entire workflow.
18
+
19
+ ## When to Use Cancellation
20
+
21
+ - **User-Initiated Actions**: In a web server, a user might navigate away from a page or click a "Cancel" button while a complex background job is running. You can use the `AbortSignal` from the HTTP request to abort the workflow.
22
+ - **Timeouts**: You can implement a timeout by calling `controller.abort()` after a certain duration.
23
+ - **Resource Management**: If a parent process is shutting down, it can signal its child workflows to abort their tasks cleanly.
24
+
25
+ ## Implementing a Cancellable Node
26
+
27
+ To make a `Node` cancellable, you need to use the `signal` object provided in the `NodeArgs` within your asynchronous logic.
28
+
29
+ ### Example: A Cancellable `sleep`
30
+
31
+ The most common use case is passing the `signal` to I/O-bound operations like `fetch` or custom-built helpers. Let's create a `sleep` function that respects the `AbortSignal`.
32
+
33
+ ```typescript
34
+ import { AbortError } from 'flowcraft'
35
+
36
+ // A helper that rejects if the signal is aborted
37
+ async function sleep(ms: number, signal?: AbortSignal): Promise<void> {
38
+ return new Promise((resolve, reject) => {
39
+ // Check if the operation was already aborted
40
+ if (signal?.aborted) {
41
+ return reject(new AbortError())
42
+ }
43
+
44
+ const timeoutId = setTimeout(resolve, ms)
45
+
46
+ // Add an event listener to clean up when aborted
47
+ signal?.addEventListener('abort', () => {
48
+ clearTimeout(timeoutId)
49
+ reject(new AbortError())
50
+ })
51
+ })
52
+ }
53
+ ```
54
+
55
+ (Note: A `sleep` utility like this is included in Flowcraft).
56
+
57
+ ### Example: A Long-Running Node
58
+
59
+ Now, let's use this `sleep` function inside a `Node`.
60
+
61
+ ```typescript
62
+ import { AbortError, ConsoleLogger, Flow, Node, sleep, TypedContext } from 'flowcraft'
63
+
64
+ class LongRunningNode extends Node {
65
+ async exec({ signal, logger }) {
66
+ logger.info('Starting a very long task...')
67
+ try {
68
+ // Pass the signal to our async operation
69
+ await sleep(5000, signal)
70
+ logger.info('Task finished successfully.')
71
+ }
72
+ catch (e) {
73
+ if (e instanceof AbortError) {
74
+ logger.warn('The long task was aborted!')
75
+ }
76
+ // Re-throw the error to ensure the flow stops
77
+ throw e
78
+ }
79
+ }
80
+ }
81
+
82
+ const flow = new Flow(new LongRunningNode())
83
+ const context = new TypedContext()
84
+ const controller = new AbortController()
85
+
86
+ // Set a timeout to abort the flow after 1 second
87
+ setTimeout(() => {
88
+ console.log('>>> Aborting workflow from the outside!')
89
+ controller.abort()
90
+ }, 1000)
91
+
92
+ try {
93
+ await flow.run(context, { controller, logger: new ConsoleLogger() })
94
+ }
95
+ catch (e) {
96
+ if (e instanceof AbortError) {
97
+ console.error('Workflow execution was successfully aborted.')
98
+ }
99
+ else {
100
+ console.error('An unexpected error occurred:', e)
101
+ }
102
+ }
103
+ ```
104
+
105
+ When you run this code, the output will be:
106
+
107
+ ```
108
+ [INFO] Running node: LongRunningNode
109
+ [INFO] Starting a very long task...
110
+ >>> Aborting workflow from the outside!
111
+ [WARN] The long task was aborted!
112
+ Workflow execution was successfully aborted.
113
+ ```
114
+
115
+ The `sleep` function was interrupted, the `catch` block inside the node logged a warning, and the re-thrown `AbortError` was caught at the top level, confirming that the workflow was gracefully terminated.
116
+
117
+ This cancellation pattern is robust and extends to custom execution environments. For instance, the distributed worker example demonstrates how to bridge an external cancellation signal (from Redis) to the standard `AbortSignal`, ensuring that even long-running, distributed jobs can be gracefully terminated mid-flight.
@@ -0,0 +1,68 @@
1
+ # Composition
2
+
3
+ One of the most powerful features of Flowcraft is its composability. Because a `Flow` is itself a type of `Node`, you can treat an entire workflow as a single building block within a larger, more complex workflow.
4
+
5
+ Flowcraft supports two primary models for composition, each suited to different use cases.
6
+
7
+ ## 1. Programmatic Composition (In-Memory)
8
+
9
+ When you build workflows programmatically (by instantiating `Node` and `Flow` classes and wiring them with `.next()`), you can place one `Flow` instance inside another.
10
+
11
+ **How It Works:** The `InMemoryExecutor` understands this pattern. When its orchestration loop encounters a `Flow` node, it calls that node's `exec` method. The `Flow.exec` method then takes over and runs its *own* internal orchestration loop, executing its entire graph from start to finish. The final action from the sub-flow is returned to the parent, allowing for branching.
12
+
13
+ **When to Use:** This model is excellent for simple, in-memory workflows where you want to organize logic into reusable, testable sub-units.
14
+
15
+ ### Example: A Reusable "Math" Sub-Flow
16
+
17
+ ```typescript
18
+ // --- sub-flow.ts ---
19
+ // A sub-flow that adds 10, multiplies by 2, and returns an action.
20
+ export function createMathFlow(): Flow {
21
+ const addNode = new Node().exec(async ({ params }) => params.input + 10).toContext(MATH_VALUE)
22
+ const multiplyNode = new Node().exec(async ({ ctx }) => ctx.get(MATH_VALUE)! * 2).toContext(MATH_VALUE)
23
+ const checkNode = new CheckResultNode() // Returns 'over_50' or 'under_50'
24
+
25
+ addNode.next(multiplyNode).next(checkNode)
26
+ return new Flow(addNode)
27
+ }
28
+
29
+ // --- main.ts ---
30
+ const mathSubFlow = createMathFlow()
31
+ const handleOver50Node = new Node().exec(() => console.log('Result was over 50.'))
32
+ const handleUnder50Node = new Node().exec(() => console.log('Result was 50 or under.'))
33
+
34
+ // The parent flow starts with the sub-flow instance.
35
+ const parentFlow = new Flow(mathSubFlow)
36
+ mathSubFlow.next(handleOver50Node, 'over_50')
37
+ mathSubFlow.next(handleUnder50Node, 'under_50')
38
+
39
+ await parentFlow.withParams({ input: 20 }).run(new TypedContext()) // "Result was over 50."
40
+ ```
41
+
42
+ ## 2. Declarative Composition with `GraphBuilder`
43
+
44
+ For complex, declarative, and distributed workflows, Flowcraft uses a more powerful **"Graph Inlining"** pattern. This is the recommended approach for building scalable systems.
45
+
46
+ > [!IMPORTANT]
47
+ > **You must explicitly tell the builder which node types represent sub-workflows** by passing them in its constructor. This gives you the flexibility to use semantically rich names like `"search-workflow"` or `"process-document-pipeline"`.
48
+ >
49
+ > If the builder encounters a node with a `workflowId` property in its `data` payload whose `type` has not been registered as a sub-workflow, it will throw a configuration error.
50
+
51
+ **How It Works:** This is a **build-time** process, not a runtime one. When the `GraphBuilder` encounters a node whose type is registered as a sub-workflow, it performs "graph surgery":
52
+
53
+ 1. **Replaces the Node**: It removes the composite node from the graph.
54
+ 2. **Inlines the Graph**: It fetches the sub-workflow's graph definition and injects all of its nodes and edges into the parent graph, prefixing their IDs to prevent collisions.
55
+ 3. **Inserts Mapping Nodes**: It automatically creates two lightweight "gatekeeper" nodes:
56
+ * An **`InputMappingNode`** at the entry point, which copies data from the parent context to the keys expected by the sub-workflow (based on your `inputs` map).
57
+ * An **`OutputMappingNode`** at the exit point, which copies data from the sub-workflow's context back to the parent (based on your `outputs` map).
58
+ 4. **Re-wires Edges**: All original edges are seamlessly re-wired to these new mapping nodes.
59
+
60
+ The result is a single, unified, "flat" graph that is handed to the executor.
61
+
62
+ ### Benefits of Graph Inlining
63
+
64
+ - **Simplified Runtimes**: The executor (e.g., `BullMQExecutor`) is completely unaware of composition. It just runs a larger, pre-compiled graph, making the runtime logic much simpler and faster.
65
+ - **Non-Blocking Workers**: This is critical for distributed systems. A worker never has to block and wait for a sub-workflow to finish. It executes its single node and moves on.
66
+ - **Clear Data Contracts**: The `inputs` and `outputs` maps in your JSON definition become an explicit, declarative data contract, preventing state leakage and making data flow easy to trace. See the guide on **[Best Practices: Data Flow in Sub-Workflows](../best-practices/sub-workflow-data.md)** for more details.
67
+
68
+ This powerful pattern moves complexity from the runtime to the build step, which is a best practice for building robust, high-performance systems.
@@ -0,0 +1,180 @@
1
+ # Building a Custom Executor
2
+
3
+ The `IExecutor` pattern is one of Flowcraft's most powerful architectural features. It decouples the definition of a workflow (the graph of `Node`s) from its execution environment. While the default `InMemoryExecutor` is perfect for many use cases, you can create your own executors to run workflows in different environments, such as a distributed task queue, a test environment, or even a system that requires pausing and resuming.
4
+
5
+ This guide will walk you through the responsibilities of an executor and show you how to build a simple `DryRunExecutor` from scratch.
6
+
7
+ ## The `IExecutor` Interface
8
+
9
+ At its core, an executor is any class that implements the `IExecutor` interface. The interface is intentionally simple:
10
+
11
+ ```typescript
12
+ interface IExecutor {
13
+ run: (flow: Flow, context: Context, options?: RunOptions) => Promise<any>
14
+ }
15
+ ```
16
+
17
+ The `run` method is the main entry point. When you call `flow.run(ctx, { executor: myExecutor })`, your executor's `run` method is invoked.
18
+
19
+ ## Core Responsibilities of an Executor
20
+
21
+ An executor is responsible for the entire orchestration of a `Flow`. This involves several key tasks:
22
+
23
+ 1. **The `run` Entry Point**: This public method kicks off the workflow. It's responsible for setting up the initial state and starting the execution loop.
24
+ 2. **The Execution Loop**: It must traverse the workflow graph, executing one node at a time. This typically involves a `while` loop that continues as long as there is a `currentNode` to process.
25
+ 3. **Applying Middleware**: Before executing a node, it must apply any middleware that has been attached to the `Flow`.
26
+ 4. **Passing Arguments**: It is responsible for constructing the `NodeArgs` object and passing the correct `ctx`, `params`, `signal`, `logger`, and a reference to *itself* down to the `node._run()` method (or a middleware chain).
27
+ 5. **Handling Actions & Branching**: After a node runs, the executor must take the returned `action`, look up the correct successor node in `currentNode.successors`, and determine the next node to execute.
28
+ 6. **State Management (for distributed systems)**: If the executor runs across different processes, it is responsible for serializing the `Context` before passing it to the next job and deserializing it upon receipt.
29
+
30
+ ## Step-by-Step Example: Building a `DryRunExecutor`
31
+
32
+ To understand these responsibilities in practice, let's build a `DryRunExecutor`. This executor will traverse an entire workflow graph and log the path it would take, but it will **not** execute the core `exec()` logic of any node. This is a great tool for debugging the structure and conditional logic of a complex flow.
33
+
34
+ ### 1. The Class Structure
35
+
36
+ First, create the class and implement the `IExecutor` interface.
37
+
38
+ ```typescript
39
+ // src/executors/dry-run-executor.ts
40
+ import {
41
+ AbstractNode,
42
+ Context,
43
+ DEFAULT_ACTION,
44
+ Flow,
45
+ IExecutor,
46
+ Logger,
47
+ NullLogger,
48
+ RunOptions
49
+ } from 'flowcraft'
50
+
51
+ export class DryRunExecutor implements IExecutor {
52
+ public async run(flow: Flow, context: Context, options?: RunOptions): Promise<any> {
53
+ // Implementation will go here
54
+ }
55
+ }
56
+ ```
57
+
58
+ ### 2. The `run` Method and Orchestration Loop
59
+
60
+ The `run` method will prepare the logger and initial parameters and then enter the main orchestration loop. This is where the core logic of the executor lives.
61
+
62
+ ```typescript
63
+ // Inside DryRunExecutor class...
64
+ public async run(flow: Flow, context: Context, options?: RunOptions): Promise<any> {
65
+ const logger = options?.logger ?? new NullLogger()
66
+ const params = { ...flow.params, ...options?.params }
67
+
68
+ logger.info(`[DryRunExecutor] Starting dry run for flow: ${flow.constructor.name}`)
69
+
70
+ if (!flow.startNode) {
71
+ logger.warn('[DryRunExecutor] Flow has no start node.')
72
+ return
73
+ }
74
+
75
+ let currentNode: AbstractNode | undefined = flow.startNode
76
+ let lastAction: any
77
+
78
+ // The executor's main orchestration loop.
79
+ while (currentNode) {
80
+ logger.info(`[DryRunExecutor] --> Visiting node: ${currentNode.constructor.name}`)
81
+
82
+ // For a dry run, we simulate execution to get the next action.
83
+ // We run `prep` and `post` to see data flow and branching, but SKIP `exec`.
84
+ // This is a powerful debugging pattern.
85
+ const node = currentNode
86
+ let action
87
+
88
+ if (node instanceof Flow) {
89
+ // If the node is a sub-flow, we must run it to get its final action.
90
+ // A real executor (like InMemoryExecutor) delegates this to a helper
91
+ // method, e.g., `_orchestrateGraph(subFlow.startNode, ...)`.
92
+ // For our dry run, we can recursively call ourself.
93
+ action = await new DryRunExecutor().run(node, context, { ...options, params })
94
+ } else {
95
+ // For a regular node, run prep and post, but not exec.
96
+ await node.prep({ ctx: context, params, logger } as any)
97
+ // We call post with a null `execRes` as exec was skipped.
98
+ action = await node.post({ ctx: context, params, logger, execRes: null } as any)
99
+ }
100
+
101
+ lastAction = action
102
+
103
+ // Display the action for logging.
104
+ const actionDisplay = (typeof lastAction === 'symbol' && lastAction === DEFAULT_ACTION)
105
+ ? 'default'
106
+ : String(lastAction)
107
+
108
+ logger.info(`[DryRunExecutor] <-- Node returned action: '${actionDisplay}'`)
109
+
110
+ // Find the next node based on the action.
111
+ currentNode = node.successors.get(lastAction)
112
+ }
113
+
114
+ logger.info('[DryRunExecutor] Dry run complete.')
115
+ return lastAction
116
+ }
117
+ ```
118
+
119
+ ### 3. Using the Custom Executor
120
+
121
+ Now you can use this executor with any flow. Note that the node's `exec` logic (the `console.log`) will not run.
122
+
123
+ ```typescript
124
+ // main.ts
125
+ import { ConsoleLogger, contextKey, Flow, Node, TypedContext } from 'flowcraft'
126
+ import { DryRunExecutor } from './executors/dry-run-executor'
127
+
128
+ const VALUE = contextKey<number>('value')
129
+
130
+ // A node to set up initial state
131
+ const startNode = new Node().prep(async ({ ctx }) => ctx.set(VALUE, 15))
132
+
133
+ // A conditional node
134
+ class CheckValueNode extends Node<void, void, 'over' | 'under'> {
135
+ async post({ ctx }) {
136
+ return ctx.get(VALUE)! > 10 ? 'over' : 'under'
137
+ }
138
+ }
139
+ const checkNode = new CheckValueNode()
140
+
141
+ // Nodes for different branches
142
+ const overNode = new Node().exec(() => console.log('This should NOT be logged!'))
143
+ const underNode = new Node().exec(() => console.log('This should NOT be logged either!'))
144
+
145
+ // Wire the graph
146
+ startNode.next(checkNode)
147
+ checkNode.next(overNode, 'over')
148
+ checkNode.next(underNode, 'under')
149
+
150
+ const flow = new Flow(startNode)
151
+ const context = new TypedContext()
152
+ const logger = new ConsoleLogger()
153
+ const dryRunExecutor = new DryRunExecutor()
154
+
155
+ console.log('--- Starting Dry Run ---')
156
+ await flow.run(context, { logger, executor: dryRunExecutor })
157
+ ```
158
+
159
+ ### Expected Output
160
+
161
+ When you run this code, you'll see the executor's logs tracing the path. The `context` is modified by `prep` and `post`, so the conditional logic works, but the `console.log` messages inside the `overNode`'s `exec` method will **not** appear.
162
+
163
+ ```
164
+ --- Starting Dry Run ---
165
+ [INFO] [DryRunExecutor] Starting dry run for flow: Flow
166
+ [INFO] [DryRunExecutor] --> Visiting node: Node
167
+ [INFO] [DryRunExecutor] <-- Node returned action: 'default'
168
+ [INFO] [DryRunExecutor] --> Visiting node: CheckValueNode
169
+ [INFO] [DryRunExecutor] <-- Node returned action: 'over'
170
+ [INFO] [DryRunExecutor] --> Visiting node: Node
171
+ [INFO] [DryRunExecutor] <-- Node returned action: 'default'
172
+ [INFO] [DryRunExecutor] Dry run complete.
173
+ ```
174
+
175
+ ## Real-World Examples
176
+
177
+ This `DryRunExecutor` is a simplified example. For a complete understanding, it's highly recommended to study the source code of the official executors:
178
+
179
+ - **`InMemoryExecutor`**: The canonical implementation of a real executor. It shows the full orchestration logic, including how to correctly apply middleware. ([`src/executors/in-memory.ts`](https://github.com/gorango/flowcraft/tree/master/src/executors/in-memory.ts))
180
+ - **`BullMQExecutor`**: A full-featured distributed executor. It demonstrates a completely different execution strategy, managing a job queue instead of an in-memory loop. ([`sandbox/5.distributed/src/executor.ts`](https://github.com/gorango/flowcraft/tree/master/sandbox/5.distributed/src/executor.ts))