flowcraft 1.0.0-beta.1

package/.editorconfig

@@ -0,0 +1,9 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+indent_style = tab
+indent_size = 4
+insert_final_newline = true
+trim_trailing_whitespace = true

package/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2024 Feathers
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,249 @@
+# Flowcraft: A Workflow Framework
+
+Build complex, multi-step processes, from simple sequences to dynamic, graph-driven AI agents.
+
+## Features
+
+- **Zero Dependencies**: Lightweight and dependency-free, ensuring a small footprint and easy integration.
+
+- **Composable & Reusable**: Define workflows by chaining nodes or embedding other flows as nodes.
+
+- **Extensible Execution Engine**: A pluggable `Executor` pattern enables in-memory or distributed flows.
+
+- **Type-Safe**: Written in TypeScript to provide strong typing for your workflow definitions and context.
+
+- **Async by Default**: Built on an asynchronous foundation to handle I/O-bound and CPU-bound tasks.
+
+- **Middleware**: Intercept execution of nodes to handle cross-cutting concerns like logging or auth.
+
+- **Conditional Branching**: Direct the flow's execution path based on the results of any node.
+
+- **Retry Logic & Fallbacks**: Retry failed operations with configurable delays and fallback logic.
+
+- **Cancellation Support**: Gracefully abort running workflows using standard `AbortController`s.
+
+- **Pluggable Logging**: Use the built-in `ConsoleLogger` or bring your own (e.g., Pino, Winston).
+
+- **Dynamic Graph Engine**: Define complex, graph-based workflows as simple JSON files.
+
+- **Fluent & Functional API**: A chainable API on the `Node` class and a collection of functional helpers.
+
+## Installation
+
+```bash
+npm install flowcraft
+```
+
+## Quick Start
+
+Create and run a simple workflow in a few lines of code.
+
+```typescript
+import { Flow, Node } from 'flowcraft'
+
+const greetNode = new Node()
+// The exec method contains the core logic of the node.
+	.exec(() => 'Hello, World!')
+// Functional helpers make common tasks easy.
+	.tap(console.log)
+
+const flow = new Flow(greetNode)
+// Run using the default InMemoryExecutor.
+await flow.run()
+```
+
+## Learn by Example
+
+> [!TIP]
+> The best way to understand the framework is by exploring the included sandbox examples. They are ordered by increasing complexity, each introducing new features and demonstrating the flexibility of the core engine.
+
+### 1. Basic Sequential Flow: Article Writer
+
+A simple, linear workflow that demonstrates the core concepts of creating a sequence of nodes to perform a multi-step task like generating an article.
+
+```mermaid
+graph LR
+    A[Generate Outline] --> B[Write Content]
+    B --> C[Apply Style]
+```
+
+- **Demonstrates**: `Node` chaining, passing data via `Context`, and a simple `BatchFlow`.
+- **[Explore the Basic example »](./sandbox/1.basic/)**
+
+### 2. Conditional Branching: Research Agent
+
+A simple agent that uses a loop and conditional branching to decide whether to search the web for information or answer a question based on the current context.
+
+```mermaid
+graph TD
+    A{Decide Action} -->|"search"| B[Search Web]
+    A -->|"answer"| C[Answer Question]
+    B --> A
+```
+
+- **Demonstrates**: Conditional branching with custom actions, creating loops, and building simple state machines.
+- **[Explore the Research Agent example »](./sandbox/2.research/)**
+
+### 3. Parallel Batch Processing: Document Translator
+
+A practical example that translates a document into multiple languages concurrently. It uses the `ParallelBatchFlow` builder to showcase significant performance boosts for I/O-bound tasks.
+
+```mermaid
+graph TD
+    A[Load README.md] --> B["ParallelBatchFlow"]
+    B -- "Chinese" --> T1[TranslateNode]
+    B -- "Spanish" --> T2[TranslateNode]
+    B -- "Japanese" --> T3[TranslateNode]
+    B -- "German" --> T4[TranslateNode]
+    T1 --> S1[Save Chinese.md]
+    T2 --> S2[Save Spanish.md]
+    T3 --> S3[Save Japanese.md]
+    T4 --> S4[Save German.md]
+```
+
+- **Demonstrates**: `ParallelBatchFlow` for high-throughput concurrent processing of I/O-bound tasks.
+- **[Explore the Parallel Translation example »](./sandbox/3.parallel/)**
+
+### 4. Dynamic Graph Engine: AI Agent Runtime
+
+The most advanced example: a powerful runtime that executes complex, graph-based AI workflows defined in simple JSON-like objects. This shows how to build highly dynamic and modular AI agent systems.
+
+> [!IMPORTANT]
+> The `GraphBuilder` is fully **type-safe**. By defining a simple map of your node types to their data payloads, TypeScript can validate your entire graph at compile time, ensuring that the data provided to each node matches what its class expects. This eliminates a whole category of runtime configuration errors.
+
+```mermaid
+graph TD
+    A(User Post) --> B[check_for_pii]
+    A --> C[check_for_hate_speech]
+    A --> D[check_for_spam]
+    B --> E{triage_post}
+    C --> E
+    D --> E
+    E -- action_ban --> F["Sub-Workflow: Ban User"]
+    E -- action_redact --> G["Sub-Workflow: Redact Post"]
+    E -- action_delete_spam --> H["Sub-Workflow: Delete"]
+    E -- action_approve --> I[approve_post_branch]
+    F --> Z[final_log]
+    G --> Z
+    H --> Z
+    I --> Z
+```
+
+- **Demonstrates**:
+  - **Type-safe graph construction** from declarative definitions using `GraphBuilder`.
+  - Parallel fan-in and fan-out (mid-flow branching).
+  - Reusable, data-driven nodes (e.g., an LLM-powered router).
+  - Complex sub-workflow composition.
+- **[Explore the Dynamic AI Agent example »](./sandbox/4.dag/)**
+
+### 5. Distributed Execution: AI Agent with BullMQ
+
+This example takes the same type-safe graph definition from the previous example and runs it in a distributed environment using a custom `BullMQExecutor`. It demonstrates a client-worker architecture for building scalable and resilient background job processors.
+
+- **Demonstrates**:
+  - A pluggable `IExecutor` for distributed workflows.
+  - Client-worker architecture with state serialization.
+  - Mid-flight, distributed cancellation of long-running jobs.
+  - How business logic (the graph) remains unchanged when the execution environment changes.
+- **[Explore the Distributed AI Agent example »](./sandbox/5.distributed/)**
+
+### 6. Advanced RAG Agent: Complex Data & Serialization
+
+A complete, end-to-end Retrieval-Augmented Generation (RAG) agent that ingests a document, creates embeddings in parallel, performs a vector search, and synthesizes an answer. This example showcases how to build a sophisticated, data-driven AI workflow.
+
+```mermaid
+graph TD
+    A[Load & Chunk Document] --> B[Generate Embeddings]
+    B --> C[Store in Vector DB]
+    C --> D[Vector Search for Question]
+    D --> E[Generate Final Answer]
+```
+
+- **Demonstrates**:
+  - A full, practical RAG pipeline.
+  - A mix of custom, single-responsibility nodes.
+  - Handling complex data types (`Map`, `Date`, custom classes) in the `Context`.
+  - The necessity of robust serialization (using `superjson`) for state management.
+- **[Explore the RAG Agent example »](./sandbox/6.rag/)**
+
+## Core Concepts
+
+### Node
+
+The `Node` is the fundamental building block of a workflow. It represents a single, potentially asynchronous unit of work with a three-phase lifecycle:
+
+1. `prep(args)`: Prepare data for execution (e.g., fetch from `Context`).
+2. `exec(args)`: Perform the core logic, isolated from the context.
+3. `post(args)`: Process results, update the `Context`, and return an "action" string to determine the next step.
+
+A chainable API on the `Node` class has a set of functional helpers:
+
+- `.map(fn)`: Transform the output of a node.
+- `.filter(predicate)`: Conditionally proceed based on a node's output.
+- `.tap(fn)`: Perform a side-effect without changing the output.
+- `.toContext(key)`: Store a node's result in the context.
+
+### Flow
+
+A `Flow` is a special type of `Node` that orchestrates a sequence of other nodes. It contains the logic for traversing its own graph of operations, making it a powerful, self-contained unit of work.
+
+### Executor
+
+An `Executor` is responsible for **running** a `Flow`. It provides the top-level execution environment, handling setup and teardown. The framework is decoupled from the execution strategy, allowing you to use the default `InMemoryExecutor` or plug in custom executors for different environments (e.g., distributed queues).
+
+### Middleware
+
+A `Flow` can be configured with middleware functions that wrap the execution of every node within it. This is the ideal pattern for handling **cross-cutting concerns** like performance monitoring, centralized logging, or transaction management without cluttering your business logic.
+
+### Context
+
+The `Context` is a shared, type-safe `Map`-like object passed through every node in a flow. It acts as a shared memory space, allowing nodes to pass data and share state.
+
+### Actions & Branching
+
+A node's `post()` method returns a string called an **action**. The flow uses this action to look up the next node to execute. The default action is a `symbol`, but returning custom strings allows for conditional branching. The fluent `.filter()` method provides another powerful way to branch based on a `FILTER_FAILED` action.
+
+### Builders
+
+To simplify the creation of common and complex patterns, the framework provides a `builder` module. These helpers abstract the construction of executable `Flow`s.
+
+- **`SequenceFlow`**: Creates a linear flow from a sequence of nodes.
+- **`BatchFlow` / `ParallelBatchFlow`**: Process a collection of items sequentially or concurrently.
+- **`GraphBuilder`**: Translates a declarative graph definition into a fully executable `Flow`.
+
+## Unit Tests
+
+> [!TIP]
+> For clear, focused examples of specific, individual features (like retries, middleware, cancellation, and composition), the unit tests are an excellent resource.
+
+- Core workflow tests: [`src/workflow.test.ts`](src/workflow.test.ts)
+- Collections tests: [`src/builder/collection.test.ts`](src/builder/collection.test.ts)
+- Graph builder tests: [`src/builder/graph.test.ts`](src/builder/graph.test.ts)
+
+## API Reference
+
+### Core Classes
+
+- `Node`: The base class for a unit of work with built-in retry logic and a fluent API (`.map`, `.filter`, etc.).
+- `Flow`: Orchestrates a sequence of nodes and supports middleware via `.use()`.
+- `InMemoryExecutor`: The default `IExecutor` implementation for running flows in-memory.
+- `TypedContext`: The standard `Map`-based implementation for the `Context` interface.
+- `ConsoleLogger`, `NullLogger`: Pre-built logger implementations.
+
+### Functional Helpers
+
+A collection of functions for creating nodes and pipelines in a more functional style.
+
+- `mapNode`: Creates a `Node` from a simple, pure function.
+- `contextNode`: Creates a `Node` from a function that requires access to the `Context`.
+- `pipeline`: A functional alias for creating a linear sequence of nodes.
+
+### Builder Classes
+
+- `SequenceFlow`: A `Flow` that creates a linear flow from a sequence of nodes.
+- `BatchFlow`: A `Flow` that processes a collection of items sequentially.
+- `ParallelBatchFlow`: A `Flow` that processes a collection of items in parallel.
+- `GraphBuilder`: Constructs a `Flow` from a declarative graph definition.
+
+---
+Licensed under the [MIT License](./LICENSE).

package/config/tsconfig.json

@@ -0,0 +1,21 @@
+{
+	"compilerOptions": {
+		"target": "esnext",
+		"module": "esnext",
+		"moduleResolution": "bundler",
+		"strict": true,
+		"noImplicitAny": true,
+		"declaration": true,
+		"outDir": "dist",
+		"sourceMap": true,
+		"esModuleInterop": true,
+		"skipLibCheck": true
+	},
+	"include": [
+		"../src"
+	],
+	"exclude": [
+		"node_modules",
+		"dist"
+	]
+}

package/config/tsup.config.ts

@@ -0,0 +1,11 @@
+import { defineConfig } from 'tsup'
+
+export default defineConfig({
+	entry: ['src/index.ts'],
+	format: ['esm'],
+	dts: true,
+	splitting: false,
+	sourcemap: true,
+	clean: true,
+	minify: true,
+})

package/config/vitest.config.ts

@@ -0,0 +1,11 @@
+import { defineConfig } from 'vitest/config'
+
+export default defineConfig({
+	test: {
+		environment: 'node',
+		globals: true,
+		coverage: {
+			include: ['src/**/*.ts'],
+		},
+	},
+})

package/docs/.vitepress/config.ts

@@ -0,0 +1,105 @@
+import { defineConfig } from 'vitepress'
+import { MermaidMarkdown, MermaidPlugin } from 'vitepress-plugin-mermaid'
+
+export default defineConfig({
+	cleanUrls: true, // might need to change depending on deployment
+	title: 'Flowcraft',
+	description: 'A Workflow Framework',
+	vite: {
+		optimizeDeps: {
+			include: [
+				'mermaid',
+			],
+		},
+		plugins: [
+			MermaidPlugin(),
+		],
+	},
+	markdown: {
+		config: (md) => {
+			MermaidMarkdown(md, {})
+		},
+	},
+	themeConfig: {
+		nav: [
+			{ text: 'Guide', link: '/guide', activeMatch: '/guide' },
+			{ text: 'API', link: '/api-reference', activeMatch: '/api-reference' },
+		],
+		footer: {
+			message: 'Released under the MIT License.',
+			copyright: 'Copyright © 2025-present Goran Spasojevic',
+		},
+		sidebar: {
+			'/guide/': [
+				{
+					text: 'Guide',
+					collapsed: true,
+					items: [
+						{ text: 'Introduction', link: '/guide/' },
+						{ text: 'Builders', link: '/guide/builders' },
+						{ text: 'Functional API', link: '/guide/functional-api' },
+					],
+				},
+				{
+					text: 'Advanced Concepts',
+					collapsed: true,
+					items: [
+						{ text: 'Composition', link: '/guide/advanced-guides/composition' },
+						{ text: 'Error Handling', link: '/guide/advanced-guides/error-handling' },
+						{ text: 'Cancellation', link: '/guide/advanced-guides/cancellation' },
+						{ text: 'Middleware', link: '/guide/advanced-guides/middleware' },
+						{ text: 'Pluggable Logging', link: '/guide/advanced-guides/logging' },
+						{ text: 'Observability', link: '/guide/advanced-guides/observability' },
+						{ text: 'Custom Executor', link: '/guide/advanced-guides/custom-executor' },
+					],
+				},
+				{
+					text: 'Recipes',
+					collapsed: true,
+					items: [
+						{ text: 'Creating a Loop', link: '/guide/recipes/creating-a-loop' },
+						{ text: 'Fan-out and Fan-in', link: '/guide/recipes/fan-out-fan-in' },
+						{ text: 'Resilient API Call Node', link: '/guide/recipes/resilient-api-call' },
+						{ text: 'Data Processing Pipeline', link: '/guide/recipes/data-processing-pipeline' },
+					],
+				},
+				{
+					text: 'Best Practices',
+					collapsed: true,
+					items: [
+						{ text: 'State Management', link: '/guide/best-practices/state-management' },
+						{ text: 'Data Flow in Sub-Workflows', link: '/guide/best-practices/sub-workflow-data' },
+						{ text: 'Testing Workflows', link: '/guide/best-practices/testing' },
+						{ text: 'Debugging Workflows', link: '/guide/best-practices/debugging' },
+					],
+				},
+				{
+					text: 'Tooling',
+					collapsed: true,
+					items: [
+						{ text: 'Visualizing Workflows', link: '/guide/tooling/mermaid' },
+						{ text: 'Graph Validation', link: '/guide/tooling/graph-validation' },
+					],
+				},
+				{
+					text: 'API Reference',
+					link: '/api-reference/',
+				},
+			],
+			'/api-reference/': [
+				{
+					text: 'API Reference',
+					items: [
+						{ text: 'Introduction', link: '/api-reference/' },
+						{ text: 'Workflow', link: '/api-reference/workflow' },
+						{ text: 'Builders', link: '/api-reference/builder' },
+						{ text: 'Functional Helpers', link: '/api-reference/fn' },
+					],
+				},
+			],
+		},
+		socialLinks: [
+			{ icon: 'github', link: 'https://github.com/gorango/cascade' },
+		],
+	},
+})

package/docs/api-reference/builder.md

@@ -0,0 +1,158 @@
+# API Reference: Builders
+
+Builders are helper classes provided by Flowcraft to abstract away the manual construction of common and complex workflow patterns. They allow you to define high-level behavior, and the builder handles the underlying `Node` and `Flow` wiring for you.
+
+You can import all builders from the main `flowcraft` package.
+
+```typescript
+import {
+	BatchFlow,
+	GraphBuilder,
+	ParallelBatchFlow,
+	ParallelFlow,
+	SequenceFlow,
+} from 'flowcraft'
+```
+
+## `SequenceFlow`
+
+A `Flow` that creates a linear workflow from a sequence of nodes, automatically chaining them in order.
+
+`extends Flow`
+
+### Constructor
+
+`new SequenceFlow(...nodes: AbstractNode[])`
+
+- `...nodes`: A sequence of `Node` or `Flow` instances to be executed in order.
+
+### Example
+
+```typescript
+const linearFlow = new SequenceFlow(
+	new GetUserNode(),
+	new ProcessDataNode(),
+	new SaveResultNode()
+)
+```
+
+---
+
+## `ParallelFlow`
+
+A `Flow` that executes a collection of nodes concurrently. This is the core of the "fan-out, fan-in" pattern. After all parallel branches complete, the flow proceeds to its single successor.
+
+`extends Flow`
+
+### Constructor
+
+`new ParallelFlow(nodes: AbstractNode[])`
+
+- `nodes`: An array of `Node` or `Flow` instances to be executed in parallel.
+
+### Example
+
+```typescript
+const parallelStep = new ParallelFlow([
+	new FetchApiANode(),
+	new FetchApiBNode(),
+])
+const aggregateNode = new AggregateResultsNode()
+
+// After both API calls are complete, the aggregateNode will run.
+parallelStep.next(aggregateNode)
+```
+
+---
+
+## `BatchFlow`
+
+An **abstract** `Flow` that processes a collection of items sequentially, one by one.
+
+`extends Flow`
+
+### Abstract Members to Implement
+
+When you extend `BatchFlow`, you must implement the following:
+
+- `protected abstract nodeToRun: AbstractNode`: You must implement this property to provide the `Node` instance that will be executed for each item in the batch.
+- `async prep(args: NodeArgs): Promise<Iterable<any>>`: This method provides the list of parameter objects. The `nodeToRun` will be executed once for each of these objects, with the object's contents merged into the `params`.
+
+---
+
+## `ParallelBatchFlow`
+
+An **abstract** `Flow` that processes a collection of items concurrently. This provides a significant performance boost for I/O-bound tasks.
+
+`extends Flow`
+
+### Abstract Members to Implement
+
+When you extend `ParallelBatchFlow`, you must implement the following:
+
+- `protected abstract nodeToRun: AbstractNode`: You must implement this property to provide the `Node` instance that will be executed concurrently for each item in the batch.
+- `async prep(args: NodeArgs): Promise<Iterable<any>>`: This method provides the list of parameter objects to be processed in parallel.
+
+---
+
+## `GraphBuilder`
+
+A powerful builder that constructs an executable `Flow` from a declarative `WorkflowGraph` definition (e.g., from a JSON file). It supports a fully type-safe API for compile-time validation of graph definitions.
+
+> [!IMPORTANT]
+> To leverage compile-time type safety, you must use the `createNodeRegistry` helper and define a `NodeTypeMap` that maps your node type strings to their expected `data` payloads. This prevents configuration errors before you even run your code.
+>
+> (See how the **[Rag Agent](https://github.com/gorango/flowcraft/tree/master/sandbox/6.rag/)** implements a simple node registry; the **[Dynamic AI Agent](https://github.com/gorango/flowcraft/tree/master/sandbox/4.dag/)** even demonstrates type-safety despite using *dynamic graphs*.)
+
+### Sub-Workflow Composition (Graph Inlining)
+
+The `GraphBuilder` has built-in support for composing workflows. You must declare which node types should be treated as sub-workflows by passing them in the `options` parameter of the constructor. When the builder encounters a node of a registered sub-workflow type, it automatically performs **graph inlining**:
+
+1.  It fetches the sub-workflow's graph definition from a `WorkflowRegistry` (which must be provided in the `nodeOptionsContext`).
+2.  It injects the sub-workflow's nodes and edges into the parent graph.
+3.  It automatically creates and wires lightweight mapping nodes to handle the data contract defined in the `inputs` and `outputs` properties of the node's `data` payload.
+
+This powerful, build-time process creates a single, flattened graph, which simplifies the execution logic for both in-memory and distributed runtimes.
+
+### Constructor
+
+`new GraphBuilder(registry, nodeOptionsContext?, options?, logger?)`
+
+-   `registry: TypedNodeRegistry | NodeRegistry`: An object or `Map` where keys are `type` strings from the graph definition and values are the corresponding `Node` class constructors. For type-safety, use the `createNodeRegistry` helper.
+-   `nodeOptionsContext?: Record<string, any>`: An optional object passed to every node's constructor. This is crucial for dependency injection, such as passing a `WorkflowRegistry` instance that the builder can use to resolve sub-workflow graphs.
+-   `options?: { subWorkflowNodeTypes?: string[] }`: An optional configuration object.
+    -   `subWorkflowNodeTypes`: An array of node `type` strings that should be treated as composable sub-workflows. The builder will inline any node whose type is in this list.
+-   `logger?: Logger`: An optional `Logger` instance. If provided, the `GraphBuilder` will automatically generate and log a Mermaid.js diagram of the final, flattened graph every time `.build()` is called. This is an invaluable tool for debugging.
+
+### Methods
+
+-   `.build(graph: WorkflowGraph): BuildResult`: The main method that takes a graph definition and returns a `BuildResult` object.
+
+#### The `BuildResult` Object
+
+The `.build()` method returns an object containing:
+
+-   `flow: Flow`: The fully wired, executable `Flow` instance.
+-   `nodeMap: Map<string, AbstractNode>`: A map of all created node instances, keyed by their `id` from the graph definition.
+-   `predecessorCountMap: Map<string, number>`: A map of each node's `id` to the number of its direct predecessors. This is essential for implementing a reliable "fan-in" or "join" pattern in custom distributed executors.
+
+> [!TIP]
+> The `nodeMap` is the most efficient way to get a reference to a specific node instance within a built flow. It provides an instant, O(1) lookup, which is ideal for debugging, monitoring, or dynamic inspection.
+>
+> ```typescript
+> const { flow, nodeMap } = builder.build(myGraph)
+> const specificNode = nodeMap.get('my-node-id')
+> ```
+
+### `WorkflowGraph` Interface
+
+The data structure that `GraphBuilder` consumes.
+
+-   `nodes: GraphNode[] | TypedGraphNode[]`: An array of node definitions.
+    -   `id: string`: A unique identifier for the node.
+    -   `type: string`: The key to look up the node's class in the `NodeRegistry`.
+    -   `data?: Record<string, any>`: A flexible data object passed as options to the node's constructor. For sub-workflow nodes, this must contain a `workflowId`.
+-   `edges: GraphEdge[]`: An array of edge definitions.
+    -   `source: string`: The `id` of the source node.
+    -   `target: string`: The `id` of the target node.
+    -   `action?: string`: The action from the source node that triggers this edge. Defaults to `DEFAULT_ACTION`.