@auto-engineer/pipeline 1.147.0 → 1.149.0

package/.turbo/turbo-build.log

@@ -1,5 +1,5 @@
 
-> @auto-engineer/pipeline@1.147.0 build /home/runner/work/auto-engineer/auto-engineer/packages/pipeline
+> @auto-engineer/pipeline@1.149.0 build /home/runner/work/auto-engineer/auto-engineer/packages/pipeline
 > tsc && tsx ../../scripts/fix-esm-imports.ts
 
 Fixed ESM imports in dist/

package/.turbo/turbo-test.log

@@ -1,5 +1,5 @@
 
-> @auto-engineer/pipeline@1.146.0 test /home/runner/work/auto-engineer/auto-engineer/packages/pipeline
+> @auto-engineer/pipeline@1.148.0 test /home/runner/work/auto-engineer/auto-engineer/packages/pipeline
 > vitest run --reporter=dot
 
 
@@ -9,6 +9,6 @@
 
  Test Files  42 passed (42)
       Tests  591 passed (591)
-   Start at  19:11:56
-   Duration  46.27s (transform 9.57s, setup 0ms, collect 20.85s, tests 37.29s, environment 25ms, prepare 30.72s)
+   Start at  07:22:54
+   Duration  42.16s (transform 8.34s, setup 0ms, collect 17.10s, tests 33.88s, environment 14ms, prepare 29.63s)
 

package/.turbo/turbo-type-check.log

@@ -1,4 +1,4 @@
 
-> @auto-engineer/pipeline@1.146.0 type-check /home/runner/work/auto-engineer/auto-engineer/packages/pipeline
+> @auto-engineer/pipeline@1.148.0 type-check /home/runner/work/auto-engineer/auto-engineer/packages/pipeline
 > tsc --noEmit
 

package/CHANGELOG.md

@@ -1,5 +1,58 @@
 # @auto-engineer/pipeline
 
+## 1.149.0
+
+### Minor Changes
+
+- [`e1eebbd`](https://github.com/BeOnAuto/auto-engineer/commit/e1eebbdf4f209780e790094d2e6887c4fa809f98) Thanks [@github-actions[bot]](https://github.com/github-actions%5Bbot%5D)! - - **server-generator-apollo-emmett**: add Given state ref hints to state.ts.ejs
+  - **server-generator-apollo-emmett**: context-aware nonCommandField instructions
+  - **server-generator-apollo-emmett**: add state context instruction
+  - **server-generator-apollo-emmett**: extract shared template helpers
+  - **server-generator-apollo-emmett**: filter state refs from hasGivenEvents in decide.ts.ejs
+
+### Patch Changes
+
+- [`d38c81e`](https://github.com/BeOnAuto/auto-engineer/commit/d38c81e7bb442a39626564cf4f6d8d55b60d0a38) Thanks [@SamHatoum](https://github.com/SamHatoum)! -
+
+- Updated dependencies [[`d38c81e`](https://github.com/BeOnAuto/auto-engineer/commit/d38c81e7bb442a39626564cf4f6d8d55b60d0a38), [`e1eebbd`](https://github.com/BeOnAuto/auto-engineer/commit/e1eebbdf4f209780e790094d2e6887c4fa809f98)]:
+  - @auto-engineer/file-store@1.149.0
+  - @auto-engineer/message-bus@1.149.0
+
+## 1.148.0
+
+### Minor Changes
+
+- [`d5ba3a0`](https://github.com/BeOnAuto/auto-engineer/commit/d5ba3a0e3fb0f6a9ad7a3a8b1815590ea77a5b42) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Added state context instruction to generated decide handlers, preventing unnecessary narrowing when Given steps contain only state references
+
+- [`e0cdc4e`](https://github.com/BeOnAuto/auto-engineer/commit/e0cdc4e3363ad84d4bc49996a600ac75c97ccc38) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Added context-aware classification of non-command fields in generated decide.ts scaffolds, distinguishing between date-derived, state-derived, and not-yet-tested fields
+
+- [`9195db7`](https://github.com/BeOnAuto/auto-engineer/commit/9195db78cb707d658866cee99a1c73d34fb4efde) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Extracted shared template helpers into a dedicated module for cleaner code generation
+
+- [`abb6540`](https://github.com/BeOnAuto/auto-engineer/commit/abb6540db7196ed7935c8a8610695828f9035fc3) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Added status variant hints from Given state references to the state template, helping implementers create matching discriminated union variants
+
+- [`9195db7`](https://github.com/BeOnAuto/auto-engineer/commit/9195db78cb707d658866cee99a1c73d34fb4efde) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Extracted shared template helper functions into a dedicated module for better code reuse across generators
+  - Simplified template specs by removing inline duplicate definitions in favor of the shared helpers
+
+### Patch Changes
+
+- [`88fb1da`](https://github.com/BeOnAuto/auto-engineer/commit/88fb1da2b222de04dd4959d87657395ee960a6ce) Thanks [@github-actions[bot]](https://github.com/github-actions%5Bbot%5D)! - - **server-generator-apollo-emmett**: skip empty file plans in scaffold output
+  - **server-generator-apollo-emmett**: filter state refs from given() in decide.specs.ts.ejs
+  - **server-generator-apollo-emmett**: move CS Given states from events to states array
+  - **global**: version packages
+  - **server-generator-apollo-emmett**: mark G1+G2 ketchup plan complete
+
+- [`4255f6d`](https://github.com/BeOnAuto/auto-engineer/commit/4255f6db0d128979e573244a615886482ce799b0) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Updated ketchup plan for state reference fix in decide template generator
+  - Marked generator bug fix milestones G1 and G2 as complete
+
+- [`62f1ea3`](https://github.com/BeOnAuto/auto-engineer/commit/62f1ea3dd1b4275211574e3df9d9a6571ae9b27a) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Fixed scaffold generation to correctly distinguish between event and state references in decision handlers
+  - Prevented contradictory instructions from appearing in generated code when Given clauses contain only state references
+
+- [`ba4f5c9`](https://github.com/BeOnAuto/auto-engineer/commit/ba4f5c9749fb1c15d444e78ca9a2689817f039cb) Thanks [@rami-hatoum](https://github.com/rami-hatoum)! - - Added implementation plan for decide.ts code generation fixes in the Apollo Emmett server generator
+
+- Updated dependencies [[`88fb1da`](https://github.com/BeOnAuto/auto-engineer/commit/88fb1da2b222de04dd4959d87657395ee960a6ce), [`d5ba3a0`](https://github.com/BeOnAuto/auto-engineer/commit/d5ba3a0e3fb0f6a9ad7a3a8b1815590ea77a5b42), [`e0cdc4e`](https://github.com/BeOnAuto/auto-engineer/commit/e0cdc4e3363ad84d4bc49996a600ac75c97ccc38), [`4255f6d`](https://github.com/BeOnAuto/auto-engineer/commit/4255f6db0d128979e573244a615886482ce799b0), [`9195db7`](https://github.com/BeOnAuto/auto-engineer/commit/9195db78cb707d658866cee99a1c73d34fb4efde), [`abb6540`](https://github.com/BeOnAuto/auto-engineer/commit/abb6540db7196ed7935c8a8610695828f9035fc3), [`9195db7`](https://github.com/BeOnAuto/auto-engineer/commit/9195db78cb707d658866cee99a1c73d34fb4efde), [`62f1ea3`](https://github.com/BeOnAuto/auto-engineer/commit/62f1ea3dd1b4275211574e3df9d9a6571ae9b27a), [`ba4f5c9`](https://github.com/BeOnAuto/auto-engineer/commit/ba4f5c9749fb1c15d444e78ca9a2689817f039cb)]:
+  - @auto-engineer/file-store@1.148.0
+  - @auto-engineer/message-bus@1.148.0
+
 ## 1.147.0
 
 ### Minor Changes

package/README.md

@@ -1,14 +1,24 @@
 # @auto-engineer/pipeline
 
-Event-driven SDLC pipeline orchestration system with declarative workflow definitions.
+Event-sourced pipeline orchestration with declarative workflow definitions, reactive event handling, and real-time SSE monitoring.
 
 ---
 
 ## Purpose
 
-Without `@auto-engineer/pipeline`, you would have to manually wire event handlers, coordinate phased execution, track command completion across correlations, and implement your own SSE streaming for real-time monitoring.
+Without `@auto-engineer/pipeline`, you would have to manually wire event-to-command mappings, coordinate phased execution across ordered groups, track scatter-gather completion for parallel commands, implement settled-handler logic for fan-in aggregation, and build your own SSE streaming layer for real-time pipeline visibility.
 
-This package provides the core infrastructure for building event-sourced pipelines. It enables declarative workflow definitions using a fluent builder API, reactive event handling, phased execution, settled handlers, and real-time monitoring via SSE.
+---
+
+## Key Concepts
+
+- **Event** -- An immutable fact that has occurred (e.g. `CodeAnalyzed`, `DeploySucceeded`).
+- **Command** -- An intention to perform an action (e.g. `AnalyzeCode`, `Deploy`).
+- **Handler Descriptor** -- A declarative rule that maps events to commands. Five types exist: `emit`, `custom`, `run-await`, `foreach-phased`, and `settled`.
+- **Pipeline** -- A named collection of handler descriptors plus a `toGraph()` method that produces a `GraphIR` for visualization.
+- **PipelineRuntime** -- Indexes handler descriptors by event type and executes matching handlers when events arrive.
+- **PipelineServer** -- An Express HTTP server that wires everything together: command dispatch, event routing, settled/phased bridges, SSE broadcast, and an in-memory event store with projections.
+- **GraphIR** -- An intermediate representation of the pipeline as typed nodes (`event`, `command`, `settled`, `phased`, `await`) and edges, used for Mermaid rendering and UI visualization.
 
 ---
 
@@ -18,12 +28,14 @@ This package provides the core infrastructure for building event-sourced pipelin
 pnpm add @auto-engineer/pipeline
 ```
 
+---
+
 ## Quick Start
 
 ```typescript
 import { define, PipelineServer } from '@auto-engineer/pipeline';
 
-const pipeline = define('SDLC')
+const pipeline = define('my-sdlc')
   .on('CodePushed')
   .emit('AnalyzeCode', (e) => ({ filePath: e.data.path }))
 
@@ -36,34 +48,25 @@ const server = new PipelineServer({ port: 3000 });
 server.registerPipeline(pipeline);
 
 await server.start();
-console.log(`Server running at http://localhost:${server.port}`);
+console.log(`Pipeline server running on port ${server.port}`);
 ```
 
 ---
 
-## Key Concepts
-
-- **Events**: Immutable facts that have occurred
-- **Commands**: Intentions to perform actions
-- **Handlers**: React to events by dispatching commands
-- **Pipelines**: Declarative workflows connecting events to commands
-
----
-
 ## How-to Guides
 
-### Define a Simple Pipeline
+### Define a pipeline with emit handlers
 
 ```typescript
 import { define } from '@auto-engineer/pipeline';
 
-const pipeline = define('my-pipeline')
+const pipeline = define('simple')
   .on('TriggerEvent')
-  .emit('OutputCommand', { data: 'value' })
+  .emit('DoWork', { key: 'value' })
   .build();
 ```
 
-### Use Conditional Handlers
+### Use conditional predicates
 
 ```typescript
 const pipeline = define('conditional')
@@ -73,21 +76,42 @@ const pipeline = define('conditional')
   .build();
 ```
 
-### Use Custom Handlers
+### Use custom handlers
 
 ```typescript
 const pipeline = define('custom')
   .on('SpecialEvent')
   .handle(async (event, ctx) => {
-    await ctx.emit('NotifyUser', { message: 'Processing complete' });
-  })
+    await ctx.emit('ProcessingDone', { result: event.data.value });
+  }, { emits: ['ProcessingDone'] })
   .build();
 ```
 
-### Use Phased Execution
+### Run commands and await completion
 
 ```typescript
-const pipeline = define('builder')
+import { dispatch } from '@auto-engineer/pipeline';
+
+const pipeline = define('scatter-gather')
+  .on('FilesDiscovered')
+  .run((e) => e.data.files.map((f) => dispatch('ProcessFile', { path: f })))
+  .awaitAll('ProcessFile', (e) => e.data.path)
+  .onSuccess('AllFilesProcessed', (ctx) => ({
+    count: ctx.results.length,
+    duration: ctx.duration,
+  }))
+  .onFailure('FileProcessingFailed', (ctx) => ({
+    failures: ctx.failures,
+  }))
+  .build();
+```
+
+### Use phased execution
+
+Process items in ordered groups where each phase completes before the next begins.
+
+```typescript
+const pipeline = define('phased')
   .on('ComponentsReady')
   .forEach((e) => e.data.components)
   .groupInto(['critical', 'normal'], (c) => c.priority)
@@ -101,7 +125,9 @@ const pipeline = define('builder')
   .build();
 ```
 
-### Use Settled Handlers
+### Use settled handlers
+
+Fire logic when all specified commands have completed.
 
 ```typescript
 const pipeline = define('aggregator')
@@ -112,22 +138,55 @@ const pipeline = define('aggregator')
   .build();
 ```
 
-### Start the Pipeline Server
+### Register command handlers and start the server
 
 ```typescript
 import { PipelineServer } from '@auto-engineer/pipeline';
+import type { CommandHandlerWithMetadata } from '@auto-engineer/pipeline';
+
+const handlers: CommandHandlerWithMetadata[] = [
+  {
+    name: 'AnalyzeCode',
+    events: ['CodeAnalyzed'],
+    handle: async (cmd) => ({
+      type: 'CodeAnalyzed',
+      data: { path: cmd.data.filePath, issues: [] },
+    }),
+  },
+];
 
 const server = new PipelineServer({ port: 3000 });
-server.registerCommandHandlers([handler1, handler2]);
+server.registerCommandHandlers(handlers);
 server.registerPipeline(pipeline);
-
 await server.start();
 ```
 
-### Consume SSE Events
+### Configure concurrency control
+
+```typescript
+server.registerConcurrency('ProcessFile', {
+  strategy: 'cancel-in-progress',  // or 'queue'
+  groupKey: (data) => data.path,
+});
+```
+
+### Load plugins from packages
 
 ```typescript
-const url = new URL('/events', 'http://localhost:5555');
+import { pipelineConfig, loadPipelineConfig } from '@auto-engineer/pipeline';
+
+const config = pipelineConfig({
+  plugins: ['@auto-engineer/code-analyzer', '@auto-engineer/deployer'],
+  pipeline: myPipeline,
+});
+
+const { handlers, pipeline } = await loadPipelineConfig(config);
+```
+
+### Consume SSE events
+
+```typescript
+const url = new URL('/events', 'http://localhost:3000');
 url.searchParams.set('correlationId', 'corr-123'); // optional filter
 
 const eventSource = new EventSource(url.toString());
@@ -137,22 +196,17 @@ eventSource.onmessage = (msg) => {
 };
 ```
 
-### Handle SSE Reconnection
+### Use the v2 builder API
 
 ```typescript
-function connectWithRetry(url: string, maxRetries = 5) {
-  let retries = 0;
-  function connect() {
-    const es = new EventSource(url);
-    es.onopen = () => { retries = 0; };
-    es.onerror = () => {
-      es.close();
-      if (retries++ < maxRetries) setTimeout(connect, 3000);
-    };
-    return es;
-  }
-  return connect();
-}
+import { defineV2, toGraphV2 } from '@auto-engineer/pipeline';
+
+const pipeline = defineV2('my-pipeline')
+  .on('Start')
+  .emit('DoWork', { key: 'value' })
+  .build();
+
+const graph = toGraphV2(pipeline);
 ```
 
 ---
@@ -162,78 +216,184 @@ function connectWithRetry(url: string, maxRetries = 5) {
 ### Package Exports
 
 ```typescript
-import {
-  define,
-  dispatch,
-  PipelineRuntime,
-  PipelineServer,
-  PhasedExecutor,
-  SettledTracker,
-  AwaitTracker,
-  EventCommandMapper,
-  SSEManager,
-  EventLogger,
-} from '@auto-engineer/pipeline';
-
-import type {
-  Pipeline,
-  PipelineBuilder,
-  Command,
-  Event,
-  CommandDispatch,
-  PipelineDescriptor,
-  HandlerDescriptor,
-  GraphIR,
-  GraphNode,
-  GraphEdge,
-  CommandHandlerWithMetadata,
-} from '@auto-engineer/pipeline';
+// Builder
+export { define } from './builder/define';
+export { defineV2, toGraphV2 } from './builder/define-v2';
+
+// Core helpers
+export { dispatch } from './core/types';
+
+// Runtime
+export { PipelineRuntime } from './runtime/pipeline-runtime';
+export { EventCommandMapper } from './runtime/event-command-map';
+export { AwaitTracker } from './runtime/await-tracker';
+
+// Server
+export { PipelineServer } from './server/pipeline-server';
+export { SSEManager } from './server/sse-manager';
+
+// Logging
+export { EventLogger } from './logging/event-logger';
+
+// Engine (v2)
+export { createPipelineEngine } from './engine/pipeline-engine';
+export { createCommandDispatcher, dispatchAndStore } from './engine/command-dispatcher';
+export { createEventRouter } from './engine/event-router';
+export { createWorkflowProcessor } from './engine/workflow-processor';
+export { createPipelineStore } from './engine/sqlite-store';
+export { createConsumer } from './engine/sqlite-consumer';
+export { createAwaitWorkflow } from './engine/workflows/await-workflow';
+export { createPhasedWorkflow } from './engine/workflows/phased-workflow';
+export { createSettledWorkflow } from './engine/workflows/settled-workflow';
+
+// Engine projections
+export { itemStatusProjection } from './engine/projections/item-status';
+export { latestRunProjection } from './engine/projections/latest-run';
+export { messageLogProjection } from './engine/projections/message-log';
+export { nodeStatusProjection } from './engine/projections/node-status';
+export { statsProjection } from './engine/projections/stats';
+
+// Store
+export { createPipelineEventStore } from './store/pipeline-event-store';
+export { PipelineReadModel } from './store/pipeline-read-model';
+
+// Testing utilities
+export {
+  compareEventSequence,
+  containsSubsequence,
+  findMissingEvents,
+  findUnexpectedEvents,
+  formatSnapshotDiff,
+} from './testing/snapshot-compare';
+```
+
+### Key Types
+
+```typescript
+interface Pipeline {
+  descriptor: Readonly<PipelineDescriptor>;
+  toGraph(): GraphIR;
+}
+
+interface PipelineDescriptor {
+  name: string;
+  version?: string;
+  description?: string;
+  keys: Map<string, KeyExtractor>;
+  handlers: HandlerDescriptor[];
+}
+
+type HandlerDescriptor =
+  | EmitHandlerDescriptor
+  | RunAwaitHandlerDescriptor
+  | ForEachPhasedDescriptor
+  | CustomHandlerDescriptor
+  | SettledHandlerDescriptor
+  | AcceptsDescriptor;
+
+interface CommandDispatch<D = Record<string, unknown>> {
+  commandType: string;
+  data: D | ((event: Event) => D);
+}
+
+interface PipelineServerConfig {
+  port: number;
+  storeFileName?: string;  // SQLite file for persistent event storage
+}
+
+interface CommandHandlerWithMetadata {
+  name: string;
+  alias?: string;
+  description?: string;
+  displayName?: string;
+  fields?: Record<string, unknown>;
+  examples?: unknown[];
+  events?: EventDefinition[];
+  handle: (command: Command, context?: PipelineContext) => Promise<Event | Event[]>;
+}
+
+interface PipelineContext {
+  emit: (type: string, data: unknown) => Promise<void>;
+  sendCommand: (type: string, data: unknown, correlationId?: string) => Promise<void>;
+  correlationId: string;
+  signal?: AbortSignal;
+}
+
+interface ConcurrencyConfig {
+  strategy: 'cancel-in-progress' | 'queue';
+  groupKey?: (data: unknown) => string;
+}
+
+interface GraphIR {
+  nodes: GraphNode[];
+  edges: GraphEdge[];
+}
+
+interface GraphNode {
+  id: string;
+  type: 'event' | 'command' | 'settled' | 'phased' | 'await';
+  label: string;
+  status?: 'idle' | 'running' | 'success' | 'error';
+}
 ```
 
-### Functions
+### `define(name: string): PipelineBuilder`
+
+Entry point for the fluent builder API. Chain `.on()`, `.emit()`, `.run()`, `.forEach()`, `.settled()`, `.handle()`, and `.build()`.
 
-#### `define(name: string): PipelineBuilder`
+### `defineV2(name: string): PipelineV2Builder`
 
-Entry point for creating pipeline definitions.
+Alternative builder that produces a `PipelineV2` with flat registration arrays. Convert to graph with `toGraphV2()`.
 
-#### `dispatch(commandType: string, data: unknown): CommandDispatch`
+### `dispatch(commandType: string, data: DataOrFactory): CommandDispatch`
 
-Helper to create command dispatch objects.
+Helper to create `CommandDispatch` objects for use with `.run()`.
 
 ### PipelineServer
 
 ```typescript
 class PipelineServer {
-  constructor(config: { port: number });
+  constructor(config: PipelineServerConfig);
   registerCommandHandlers(handlers: CommandHandlerWithMetadata[]): void;
   registerPipeline(pipeline: Pipeline): void;
+  registerConcurrency(commandType: string, config: ConcurrencyConfig): void;
+  registerItemKeyExtractor(commandType: string, extractor: (data: unknown) => string | undefined): void;
+  use(handler: express.RequestHandler): this;
   start(): Promise<void>;
   stop(): Promise<void>;
   readonly port: number;
+  getHttpServer(): HttpServer;
+  getMessageBus(): MessageBus;
+  getRegisteredCommands(): string[];
+  getPipelineNames(): string[];
 }
 ```
 
 ### Server Endpoints
 
 | Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/health` | GET | Health check |
-| `/registry` | GET | List handlers |
-| `/pipeline` | GET | Pipeline graph |
-| `/pipeline/mermaid` | GET | Mermaid diagram |
-| `/pipeline/diagram` | GET | HTML diagram |
-| `/command` | POST | Dispatch command |
-| `/events` | GET | SSE stream |
-
-### Handler Types
-
-| Type | Description |
-|------|-------------|
-| `emit` | Dispatch commands when events occur |
-| `custom` | Execute arbitrary async logic |
-| `run-await` | Dispatch and await completion |
-| `foreach-phased` | Process items in ordered phases |
-| `settled` | Fire when multiple commands complete |
+|---|---|---|
+| `/health` | GET | Health check with uptime |
+| `/registry` | GET | List registered event and command handlers |
+| `/pipeline` | GET | Pipeline graph with live node status |
+| `/pipeline/mermaid` | GET | Mermaid diagram as plain text |
+| `/pipeline/diagram` | GET | Self-contained HTML page with rendered diagram |
+| `/command` | POST | Dispatch a command (returns `ack` with `correlationId`) |
+| `/execute` | POST | Dispatch and wait for synchronous result |
+| `/events` | GET | SSE stream (optional `?correlationId=` filter) |
+| `/messages` | GET | Full message log |
+| `/stats` | GET | Aggregate message statistics |
+| `/run-stats` | GET | Per-run item/node statistics |
+
+### Testing Utilities
+
+```typescript
+function compareEventSequence(expected: string[], actual: string[]): SnapshotResult;
+function containsSubsequence(sequence: string[], subsequence: string[]): boolean;
+function findMissingEvents(sequence: string[], required: string[]): string[];
+function findUnexpectedEvents(sequence: string[], allowed: string[]): string[];
+function formatSnapshotDiff(result: SnapshotResult): string;
+```
 
 ---
 
@@ -241,39 +401,108 @@ class PipelineServer {
 
 ```
 src/
-├── builder/
-├── core/
-├── graph/
-├── logging/
-├── plugins/
-├── projections/
-├── runtime/
-├── server/
-├── store/
-└── testing/
+├── builder/          # define() and defineV2() fluent builder APIs
+│   ├── define.ts             # Primary builder with graph extraction
+│   └── define-v2.ts          # Simplified v2 builder
+├── config/           # Pipeline configuration and plugin loading
+│   └── pipeline-config.ts
+├── core/             # Shared types and descriptor definitions
+│   ├── descriptors.ts        # Handler descriptor types
+│   └── types.ts              # Event, Command, CommandDispatch
+├── engine/           # v2 engine: dispatcher, router, workflows, SQLite store
+│   ├── command-dispatcher.ts
+│   ├── event-router.ts
+│   ├── pipeline-engine.ts
+│   ├── sqlite-consumer.ts
+│   ├── sqlite-store.ts
+│   ├── workflow-processor.ts
+│   ├── projections/          # Event-driven projections (item, node, run, stats)
+│   └── workflows/            # Await, phased, and settled workflow implementations
+├── graph/            # GraphIR types and filtering
+│   ├── types.ts
+│   └── filter-graph.ts
+├── logging/          # EventLogger for recording pipeline activity
+├── plugins/          # Dynamic plugin loading and handler adaptation
+│   ├── plugin-loader.ts
+│   └── handler-adapter.ts
+├── projections/      # In-memory event store projections
+│   ├── await-tracker-projection.ts
+│   ├── item-status-projection.ts
+│   ├── latest-run-projection.ts
+│   ├── message-log-projection.ts
+│   ├── node-status-projection.ts
+│   └── stats-projection.ts
+├── runtime/          # PipelineRuntime, AwaitTracker, EventCommandMapper
+├── server/           # PipelineServer, SSE, command gate, quiescence tracking
+│   ├── pipeline-server.ts
+│   ├── pipeline-server-v2.ts
+│   ├── sse-manager.ts
+│   ├── command-gate.ts
+│   ├── quiescence-tracker.ts
+│   ├── phased-bridge.ts
+│   └── v2-runtime-bridge.ts
+├── store/            # In-memory event store with Emmett projections
+│   ├── pipeline-event-store.ts
+│   └── pipeline-read-model.ts
+└── testing/          # Test helpers: mock handlers, event capture, snapshots
+    ├── event-capture.ts
+    ├── mock-handlers.ts
+    ├── snapshot-compare.ts
+    └── snapshot-sanitize.ts
 ```
 
-The following diagram shows the execution flow:
-
 ```mermaid
 flowchart TB
-    A[Event Received] --> B[Handler Matching]
-    B --> C[Handler Execution]
-    C --> D{Handler Type}
-    D -->|emit| E[Dispatch Commands]
-    D -->|custom| F[Execute Function]
-    D -->|phased| G[Phase Execution]
-    D -->|settled| H[Track Completion]
+    subgraph Builder
+        define["define() / defineV2()"]
+    end
+
+    subgraph Core
+        Pipeline["Pipeline + Descriptor"]
+        GraphIR["GraphIR"]
+    end
+
+    subgraph Runtime
+        PR["PipelineRuntime"]
+        AT["AwaitTracker"]
+        ECM["EventCommandMapper"]
+    end
+
+    subgraph Server
+        PS["PipelineServer"]
+        SSE["SSEManager"]
+        CG["CommandGate"]
+        QT["QuiescenceTracker"]
+    end
+
+    subgraph Store
+        ES["EventStore + Projections"]
+        RM["PipelineReadModel"]
+    end
+
+    define --> Pipeline
+    Pipeline --> GraphIR
+    Pipeline --> PR
+    PS --> PR
+    PS --> SSE
+    PS --> CG
+    PS --> QT
+    PS --> ES
+    ES --> RM
+    PR --> AT
+    PR --> ECM
 ```
 
-*Flow: Events arrive, handlers match, execute based on type.*
-
 ### Dependencies
 
 | Package | Usage |
-|---------|-------|
-| `@auto-engineer/message-bus` | Command/event messaging |
+|---|---|
+| `@auto-engineer/message-bus` | Command/event messaging infrastructure |
 | `@auto-engineer/file-store` | File storage utilities |
-| `@event-driven-io/emmett` | Event store |
-| `express` | HTTP server |
-| `nanoid` | ID generation |
+| `@event-driven-io/emmett` | In-memory event store with projections |
+| `@event-driven-io/emmett-sqlite` | SQLite-backed persistent event store |
+| `express` | HTTP server for REST + SSE endpoints |
+| `nanoid` | Compact unique ID generation |
+| `chokidar` | File system watching |
+| `jose` | JWT handling |
+| `get-port` | Automatic port selection |

package/package.json

@@ -15,8 +15,8 @@
     "jose": "^5.9.6",
     "nanoid": "^5.0.0",
     "uuid": "^11.0.0",
-    "@auto-engineer/file-store": "1.147.0",
-    "@auto-engineer/message-bus": "1.147.0"
+    "@auto-engineer/file-store": "1.149.0",
+    "@auto-engineer/message-bus": "1.149.0"
   },
   "devDependencies": {
     "@types/cors": "^2.8.17",
@@ -25,7 +25,7 @@
   "publishConfig": {
     "access": "public"
   },
-  "version": "1.147.0",
+  "version": "1.149.0",
   "scripts": {
     "build": "tsc && tsx ../../scripts/fix-esm-imports.ts",
     "test": "vitest run --reporter=dot",