footprintjs 9.9.0 → 9.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +9 -9
- package/CLAUDE.md +82 -753
- package/README.md +51 -20
- package/dist/advanced.js +14 -2
- package/dist/esm/advanced.d.ts +55 -0
- package/dist/esm/advanced.js +10 -1
- package/dist/esm/detach.d.ts +59 -0
- package/dist/esm/index.d.ts +202 -0
- package/dist/esm/lib/builder/FlowChartBuilder.d.ts +475 -0
- package/dist/esm/lib/builder/FlowChartBuilder.js +4 -3
- package/dist/esm/lib/builder/index.d.ts +11 -0
- package/dist/esm/lib/builder/structure/StructureRecorder.d.ts +349 -0
- package/dist/esm/lib/builder/structure/StructureRecorderDispatcher.d.ts +77 -0
- package/dist/esm/lib/builder/typedFlowChart.d.ts +25 -0
- package/dist/esm/lib/builder/types.d.ts +154 -0
- package/dist/esm/lib/capture/envelope.d.ts +169 -0
- package/dist/esm/lib/capture/index.d.ts +16 -0
- package/dist/esm/lib/capture/policies.d.ts +42 -0
- package/dist/esm/lib/capture/summarize.d.ts +65 -0
- package/dist/esm/lib/contract/defineContract.d.ts +18 -0
- package/dist/esm/lib/contract/index.d.ts +14 -0
- package/dist/esm/lib/contract/openapi.d.ts +18 -0
- package/dist/esm/lib/contract/schema.d.ts +13 -0
- package/dist/esm/lib/contract/types.d.ts +105 -0
- package/dist/esm/lib/decide/decide.d.ts +47 -0
- package/dist/esm/lib/decide/evaluator.d.ts +39 -0
- package/dist/esm/lib/decide/evidence.d.ts +22 -0
- package/dist/esm/lib/decide/index.d.ts +12 -0
- package/dist/esm/lib/decide/types.d.ts +111 -0
- package/dist/esm/lib/detach/drivers/immediate.d.ts +39 -0
- package/dist/esm/lib/detach/drivers/microtaskBatch.d.ts +57 -0
- package/dist/esm/lib/detach/drivers/sendBeacon.d.ts +38 -0
- package/dist/esm/lib/detach/drivers/setImmediate.d.ts +32 -0
- package/dist/esm/lib/detach/drivers/setTimeout.d.ts +34 -0
- package/dist/esm/lib/detach/drivers/workerThread.d.ts +50 -0
- package/dist/esm/lib/detach/flush.d.ts +62 -0
- package/dist/esm/lib/detach/handle.d.ts +83 -0
- package/dist/esm/lib/detach/registry.d.ts +82 -0
- package/dist/esm/lib/detach/runChild.d.ts +41 -0
- package/dist/esm/lib/detach/spawn.d.ts +64 -0
- package/dist/esm/lib/detach/types.d.ts +200 -0
- package/dist/esm/lib/engine/errors/errorInfo.d.ts +46 -0
- package/dist/esm/lib/engine/errors/index.d.ts +2 -0
- package/dist/esm/lib/engine/graph/StageNode.d.ts +108 -0
- package/dist/esm/lib/engine/graph/index.d.ts +2 -0
- package/dist/esm/lib/engine/handlers/ChildrenExecutor.d.ts +32 -0
- package/dist/esm/lib/engine/handlers/ContinuationResolver.d.ts +95 -0
- package/dist/esm/lib/engine/handlers/DeciderHandler.d.ts +56 -0
- package/dist/esm/lib/engine/handlers/NodeResolver.d.ts +41 -0
- package/dist/esm/lib/engine/handlers/RuntimeStructureManager.d.ts +37 -0
- package/dist/esm/lib/engine/handlers/SelectorHandler.d.ts +27 -0
- package/dist/esm/lib/engine/handlers/StageRunner.d.ts +17 -0
- package/dist/esm/lib/engine/handlers/SubflowExecutor.d.ts +34 -0
- package/dist/esm/lib/engine/handlers/SubflowExecutor.js +8 -1
- package/dist/esm/lib/engine/handlers/SubflowInputMapper.d.ts +40 -0
- package/dist/esm/lib/engine/handlers/index.d.ts +13 -0
- package/dist/esm/lib/engine/handlers/types.d.ts +32 -0
- package/dist/esm/lib/engine/index.d.ts +28 -0
- package/dist/esm/lib/engine/narrative/CombinedNarrativeRecorder.d.ts +139 -0
- package/dist/esm/lib/engine/narrative/FlowRecorderDispatcher.d.ts +47 -0
- package/dist/esm/lib/engine/narrative/NarrativeFlowRecorder.d.ts +34 -0
- package/dist/esm/lib/engine/narrative/NullControlFlowNarrativeGenerator.d.ts +27 -0
- package/dist/esm/lib/engine/narrative/index.d.ts +15 -0
- package/dist/esm/lib/engine/narrative/narrativeTypes.d.ts +196 -0
- package/dist/esm/lib/engine/narrative/recorders/AdaptiveNarrativeFlowRecorder.d.ts +25 -0
- package/dist/esm/lib/engine/narrative/recorders/ManifestFlowRecorder.d.ts +69 -0
- package/dist/esm/lib/engine/narrative/recorders/MilestoneNarrativeFlowRecorder.d.ts +24 -0
- package/dist/esm/lib/engine/narrative/recorders/ProgressiveNarrativeFlowRecorder.d.ts +30 -0
- package/dist/esm/lib/engine/narrative/recorders/RLENarrativeFlowRecorder.d.ts +25 -0
- package/dist/esm/lib/engine/narrative/recorders/SeparateNarrativeFlowRecorder.d.ts +32 -0
- package/dist/esm/lib/engine/narrative/recorders/SilentNarrativeFlowRecorder.d.ts +25 -0
- package/dist/esm/lib/engine/narrative/recorders/WindowedNarrativeFlowRecorder.d.ts +29 -0
- package/dist/esm/lib/engine/narrative/recorders/index.d.ts +9 -0
- package/dist/esm/lib/engine/narrative/types.d.ts +384 -0
- package/dist/esm/lib/engine/runtimeStageId.d.ts +103 -0
- package/dist/esm/lib/engine/traversal/FlowchartTraverser.d.ts +376 -0
- package/dist/esm/lib/engine/traversal/FlowchartTraverser.js +1 -1
- package/dist/esm/lib/engine/traversal/index.d.ts +2 -0
- package/dist/esm/lib/engine/types.d.ts +417 -0
- package/dist/esm/lib/engine/walkSubflowSpec.d.ts +95 -0
- package/dist/esm/lib/memory/DiagnosticCollector.d.ts +33 -0
- package/dist/esm/lib/memory/EventLog.d.ts +27 -0
- package/dist/esm/lib/memory/SharedMemory.d.ts +31 -0
- package/dist/esm/lib/memory/StageContext.d.ts +368 -0
- package/dist/esm/lib/memory/StageContext.js +44 -2
- package/dist/esm/lib/memory/TransactionBuffer.d.ts +183 -0
- package/dist/esm/lib/memory/TransactionBuffer.js +35 -13
- package/dist/esm/lib/memory/backtrack.d.ts +252 -0
- package/dist/esm/lib/memory/backtrack.js +79 -8
- package/dist/esm/lib/memory/commitLogUtils.d.ts +41 -0
- package/dist/esm/lib/memory/index.d.ts +14 -0
- package/dist/esm/lib/memory/pathOps.d.ts +54 -0
- package/dist/esm/lib/memory/types.d.ts +230 -0
- package/dist/esm/lib/memory/types.js +1 -1
- package/dist/esm/lib/memory/utils.d.ts +124 -0
- package/dist/esm/lib/observer-queue/deferredDispatcher.d.ts +169 -0
- package/dist/esm/lib/observer-queue/flushDriver.d.ts +124 -0
- package/dist/esm/lib/observer-queue/index.d.ts +25 -0
- package/dist/esm/lib/observer-queue/mergedQueue.d.ts +85 -0
- package/dist/esm/lib/observer-queue/ring.d.ts +99 -0
- package/dist/esm/lib/pause/index.d.ts +2 -0
- package/dist/esm/lib/pause/types.d.ts +258 -0
- package/dist/esm/lib/pause/types.js +2 -2
- package/dist/esm/lib/reactive/allowlist.d.ts +30 -0
- package/dist/esm/lib/reactive/arrayTraps.d.ts +23 -0
- package/dist/esm/lib/reactive/createTypedScope.d.ts +24 -0
- package/dist/esm/lib/reactive/index.d.ts +14 -0
- package/dist/esm/lib/reactive/pathBuilder.d.ts +31 -0
- package/dist/esm/lib/reactive/types.d.ts +159 -0
- package/dist/esm/lib/reactive/types.js +1 -1
- package/dist/esm/lib/recorder/BoundaryStateStore.d.ts +115 -0
- package/dist/esm/lib/recorder/CombinedRecorder.d.ts +215 -0
- package/dist/esm/lib/recorder/CombinedRecorder.js +2 -2
- package/dist/esm/lib/recorder/CommitRangeIndex.d.ts +147 -0
- package/dist/esm/lib/recorder/CompositeRecorder.d.ts +95 -0
- package/dist/esm/lib/recorder/ControlDepRecorder.d.ts +133 -0
- package/dist/esm/lib/recorder/EmitRecorder.d.ts +135 -0
- package/dist/esm/lib/recorder/InOutRecorder.d.ts +176 -0
- package/dist/esm/lib/recorder/KeyedStore.d.ts +70 -0
- package/dist/esm/lib/recorder/QualityRecorder.d.ts +122 -0
- package/dist/esm/lib/recorder/RecorderOperation.d.ts +35 -0
- package/dist/esm/lib/recorder/SequenceStore.d.ts +120 -0
- package/dist/esm/lib/recorder/TopologyRecorder.d.ts +165 -0
- package/dist/esm/lib/recorder/TopologyRecorder.js +1 -1
- package/dist/esm/lib/recorder/index.d.ts +9 -0
- package/dist/esm/lib/recorder/invokeHook.d.ts +32 -0
- package/dist/esm/lib/recorder/qualityTrace.d.ts +54 -0
- package/dist/esm/lib/runner/ComposableRunner.d.ts +33 -0
- package/dist/esm/lib/runner/DeferredObserverTier.d.ts +204 -0
- package/dist/esm/lib/runner/ExecutionRuntime.d.ts +162 -0
- package/dist/esm/lib/runner/ExecutionRuntime.js +16 -1
- package/dist/esm/lib/runner/FlowChartExecutor.d.ts +660 -0
- package/dist/esm/lib/runner/FlowChartExecutor.js +66 -3
- package/dist/esm/lib/runner/RunContext.d.ts +42 -0
- package/dist/esm/lib/runner/RunnableChart.d.ts +57 -0
- package/dist/esm/lib/runner/checkpointSanitize.d.ts +44 -0
- package/dist/esm/lib/runner/getSubtreeSnapshot.d.ts +70 -0
- package/dist/esm/lib/runner/index.d.ts +11 -0
- package/dist/esm/lib/runner/runId.d.ts +45 -0
- package/dist/esm/lib/runner/validateInput.d.ts +11 -0
- package/dist/esm/lib/schema/detect.d.ts +32 -0
- package/dist/esm/lib/schema/errors.d.ts +36 -0
- package/dist/esm/lib/schema/index.d.ts +14 -0
- package/dist/esm/lib/schema/validate.d.ts +33 -0
- package/dist/esm/lib/scope/ScopeFacade.d.ts +250 -0
- package/dist/esm/lib/scope/detectCircular.d.ts +47 -0
- package/dist/esm/lib/scope/index.d.ts +16 -0
- package/dist/esm/lib/scope/protection/createProtectedScope.d.ts +9 -0
- package/dist/esm/lib/scope/protection/index.d.ts +2 -0
- package/dist/esm/lib/scope/protection/readonlyInput.d.ts +26 -0
- package/dist/esm/lib/scope/protection/types.d.ts +13 -0
- package/dist/esm/lib/scope/providers/baseStateCompatible.d.ts +24 -0
- package/dist/esm/lib/scope/providers/guards.d.ts +14 -0
- package/dist/esm/lib/scope/providers/index.d.ts +6 -0
- package/dist/esm/lib/scope/providers/providers.d.ts +8 -0
- package/dist/esm/lib/scope/providers/registry.d.ts +11 -0
- package/dist/esm/lib/scope/providers/resolve.d.ts +8 -0
- package/dist/esm/lib/scope/providers/types.d.ts +40 -0
- package/dist/esm/lib/scope/recorders/DebugRecorder.d.ts +63 -0
- package/dist/esm/lib/scope/recorders/MetricRecorder.d.ts +125 -0
- package/dist/esm/lib/scope/recorders/index.d.ts +5 -0
- package/dist/esm/lib/scope/recorders/summarizeValue.d.ts +7 -0
- package/dist/esm/lib/scope/recorders/summarizeValue.js +3 -3
- package/dist/esm/lib/scope/state/installResolvers.d.ts +4 -0
- package/dist/esm/lib/scope/state/zod/defineScopeFromZod.d.ts +9 -0
- package/dist/esm/lib/scope/state/zod/index.d.ts +5 -0
- package/dist/esm/lib/scope/state/zod/resolver.d.ts +5 -0
- package/dist/esm/lib/scope/state/zod/schema/builder.d.ts +12 -0
- package/dist/esm/lib/scope/state/zod/scopeFactory.d.ts +9 -0
- package/dist/esm/lib/scope/state/zod/utils/validateHelper.d.ts +25 -0
- package/dist/esm/lib/scope/types.d.ts +142 -0
- package/dist/esm/lib/slice/elementProvenance.d.ts +74 -0
- package/dist/esm/lib/slice/elementProvenance.js +181 -0
- package/dist/esm/lib/slice/index.d.ts +16 -0
- package/dist/esm/lib/slice/index.js +16 -0
- package/dist/esm/lib/slice/keysReadSources.d.ts +51 -0
- package/dist/esm/lib/slice/keysReadSources.js +96 -0
- package/dist/esm/lib/slice/serialize.d.ts +40 -0
- package/dist/esm/lib/slice/serialize.js +97 -0
- package/dist/esm/lib/slice/sliceForKey.d.ts +47 -0
- package/dist/esm/lib/slice/sliceForKey.js +72 -0
- package/dist/esm/lib/slice/types.d.ts +240 -0
- package/dist/esm/lib/slice/types.js +20 -0
- package/dist/esm/package.json +1 -1
- package/dist/esm/recorders.d.ts +70 -0
- package/dist/esm/trace.d.ts +57 -0
- package/dist/esm/trace.js +2 -1
- package/dist/esm/zod.d.ts +18 -0
- package/dist/lib/builder/FlowChartBuilder.js +4 -3
- package/dist/lib/engine/handlers/SubflowExecutor.js +8 -1
- package/dist/lib/engine/traversal/FlowchartTraverser.js +1 -1
- package/dist/lib/memory/StageContext.js +44 -2
- package/dist/lib/memory/TransactionBuffer.js +35 -13
- package/dist/lib/memory/backtrack.js +79 -8
- package/dist/lib/memory/types.js +1 -1
- package/dist/lib/pause/types.js +2 -2
- package/dist/lib/reactive/types.js +1 -1
- package/dist/lib/recorder/CombinedRecorder.js +2 -2
- package/dist/lib/recorder/TopologyRecorder.js +1 -1
- package/dist/lib/runner/ExecutionRuntime.js +16 -1
- package/dist/lib/runner/FlowChartExecutor.js +66 -3
- package/dist/lib/scope/recorders/summarizeValue.js +3 -3
- package/dist/lib/slice/elementProvenance.js +186 -0
- package/dist/lib/slice/index.js +28 -0
- package/dist/lib/slice/keysReadSources.js +102 -0
- package/dist/lib/slice/serialize.js +102 -0
- package/dist/lib/slice/sliceForKey.js +77 -0
- package/dist/lib/slice/types.js +21 -0
- package/dist/trace.js +12 -2
- package/dist/types/advanced.d.ts +3 -0
- package/dist/types/lib/builder/FlowChartBuilder.d.ts +25 -2
- package/dist/types/lib/engine/traversal/FlowchartTraverser.d.ts +1 -1
- package/dist/types/lib/memory/StageContext.d.ts +28 -1
- package/dist/types/lib/memory/TransactionBuffer.d.ts +10 -1
- package/dist/types/lib/memory/backtrack.d.ts +26 -0
- package/dist/types/lib/memory/types.d.ts +26 -0
- package/dist/types/lib/pause/types.d.ts +24 -3
- package/dist/types/lib/reactive/types.d.ts +1 -1
- package/dist/types/lib/recorder/CombinedRecorder.d.ts +1 -1
- package/dist/types/lib/recorder/TopologyRecorder.d.ts +1 -1
- package/dist/types/lib/runner/ExecutionRuntime.d.ts +15 -1
- package/dist/types/lib/runner/FlowChartExecutor.d.ts +34 -3
- package/dist/types/lib/scope/recorders/summarizeValue.d.ts +2 -2
- package/dist/types/lib/slice/elementProvenance.d.ts +74 -0
- package/dist/types/lib/slice/index.d.ts +16 -0
- package/dist/types/lib/slice/keysReadSources.d.ts +51 -0
- package/dist/types/lib/slice/serialize.d.ts +40 -0
- package/dist/types/lib/slice/sliceForKey.d.ts +47 -0
- package/dist/types/lib/slice/types.d.ts +240 -0
- package/dist/types/trace.d.ts +3 -0
- package/package.json +75 -21
package/CLAUDE.md
CHANGED
|
@@ -1,757 +1,86 @@
|
|
|
1
|
-
|
|
1
|
+
<!-- analyzed-at: 22953d9 @ 2026-07-02 | model: fable-5 -->
|
|
2
|
+
# footprintjs — feature-work map
|
|
2
3
|
|
|
3
|
-
|
|
4
|
+
Self-explaining flowchart engine: a fluent builder emits a static chart; one DFS pass executes it while observer channels + a commit log capture everything ("collect during traversal, never post-process"). This file maps seams and blast radius; for API description read the code/README — **trust the code** where any doc disagrees.
|
|
4
5
|
|
|
5
|
-
##
|
|
6
|
+
## Module map
|
|
7
|
+
Entry points: `footprintjs` (public API) · `/recorders` (factories) · `/trace` (commitLog queries, stores, causalChain) · `/advanced` (engine internals) · `/detach` · `/zod` (opt-in; zod never imported by core). New public symbols are wired through the barrel that OWNS them (canonical path); `/advanced` re-exports only a small hand-picked trace subset — do not assume `export *` chaining.
|
|
6
8
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
## Architecture — Library of Libraries
|
|
10
|
-
|
|
11
|
-
```
|
|
12
|
-
src/lib/
|
|
13
|
-
├── capture/ → Value-capture/retention primitives (RetentionPolicy 'full'|'summary'|'off', read/write summary markers) — shared by the readTracking (#14) + writeTracking (#13c-A) dials; RFC-001 builds on it
|
|
14
|
-
├── memory/ → Transactional state (SharedMemory, StageContext, TransactionBuffer, EventLog)
|
|
15
|
-
├── schema/ → Validation abstraction (Zod optional, duck-typed detection)
|
|
16
|
-
├── builder/ → Fluent DSL (FlowChartBuilder, flowChart(), DeciderList, SelectorFnList)
|
|
17
|
-
├── scope/ → Per-stage facades + recorders + providers
|
|
18
|
-
├── reactive/ → TypedScope<T> deep Proxy (typed property access, $-methods, cycle-safe)
|
|
19
|
-
├── decide/ → decide()/select() decision evidence capture (filter + function)
|
|
20
|
-
├── recorder/ → CompositeRecorder, stores (KeyedStore/SequenceStore/BoundaryStateStore), CommitRangeIndex, composition primitives
|
|
21
|
-
├── pause/ → Pause/Resume (PauseSignal, FlowchartCheckpoint, PausableHandler)
|
|
22
|
-
├── engine/ → DFS traversal + narrative + 13 handlers
|
|
23
|
-
├── runner/ → High-level executor (FlowChartExecutor)
|
|
24
|
-
└── contract/ → I/O schema + OpenAPI generation
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
Dependency DAG: `capture (standalone leaf) <- memory <- scope <- reactive <- engine <- runner`, `schema <- engine`, `builder (standalone) -> engine`, `contract <- schema`, `decide -> scope`
|
|
28
|
-
|
|
29
|
-
Entry points:
|
|
30
|
-
- `import { ... } from 'footprintjs'` — public API
|
|
31
|
-
- `import { ... } from 'footprintjs/trace'` — execution tracing: runtimeStageId, commitLog queries, causal chain, recorder stores (`KeyedStore`/`SequenceStore`/`BoundaryStateStore`), `CommitRangeIndex`, `TopologyRecorder`/`InOutRecorder`/`QualityRecorder`
|
|
32
|
-
- `import { ... } from 'footprintjs/advanced'` — engine internals (also re-exports trace)
|
|
33
|
-
- `import { ... } from 'footprintjs/zod'` — **opt-in** zod-based scope helpers (`defineScopeFromZod`, `defineScopeSchema`, `isScopeSchema`, `createScopeProxyFromZod`, `ZodScopeResolver`). zod is an OPTIONAL peer — the core never imports it, so import these here and add `zod` to your deps. (Moved off the core barrels in 8.0.0 — see CHANGELOG.)
|
|
34
|
-
|
|
35
|
-
## Key API
|
|
36
|
-
|
|
37
|
-
### TypedScope (Recommended)
|
|
38
|
-
|
|
39
|
-
```typescript
|
|
40
|
-
import { flowChart, FlowChartExecutor } from 'footprintjs';
|
|
41
|
-
|
|
42
|
-
interface LoanState {
|
|
43
|
-
creditTier: string;
|
|
44
|
-
amount: number;
|
|
45
|
-
customer: { name: string; address: { zip: string } };
|
|
46
|
-
tags: string[];
|
|
47
|
-
approved?: boolean;
|
|
48
|
-
}
|
|
49
|
-
|
|
50
|
-
const chart = flowChart<LoanState>('Intake', async (scope) => {
|
|
51
|
-
scope.creditTier = 'A'; // typed write
|
|
52
|
-
scope.amount = 50000; // typed write
|
|
53
|
-
scope.customer.address.zip = '90210'; // deep write (updateValue)
|
|
54
|
-
scope.tags.push('vip'); // array copy-on-write (single push)
|
|
55
|
-
scope.$batchArray('tags', (arr) => { // O(1) batch: 1 clone + 1 commit
|
|
56
|
-
arr.push('vip', 'premium', 'verified');
|
|
57
|
-
});
|
|
58
|
-
scope.approved = true; // optional field
|
|
59
|
-
|
|
60
|
-
// $-prefixed escape hatches
|
|
61
|
-
scope.$debug('checkpoint', { step: 1 });
|
|
62
|
-
scope.$metric('latency', 42);
|
|
63
|
-
const args = scope.$getArgs<{ requestId: string }>();
|
|
64
|
-
const env = scope.$getEnv();
|
|
65
|
-
scope.$break(); // stop pipeline
|
|
66
|
-
}, 'intake')
|
|
67
|
-
.build();
|
|
68
|
-
|
|
69
|
-
const executor = new FlowChartExecutor(chart);
|
|
70
|
-
await executor.run({ input: { requestId: 'req-123' } });
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
### decide() / select() — Decision Evidence Capture
|
|
74
|
-
|
|
75
|
-
```typescript
|
|
76
|
-
import { decide, select } from 'footprintjs';
|
|
77
|
-
|
|
78
|
-
// Inside a decider function — auto-captures which values led to the decision
|
|
79
|
-
.addDeciderFunction('ClassifyRisk', (scope) => {
|
|
80
|
-
return decide(scope, [
|
|
81
|
-
{ when: { creditScore: { gt: 700 }, dti: { lt: 0.43 } }, then: 'approved', label: 'Good credit' },
|
|
82
|
-
{ when: (s) => s.creditScore > 600, then: 'manual-review', label: 'Marginal' },
|
|
83
|
-
], 'rejected');
|
|
84
|
-
}, 'classify-risk')
|
|
85
|
-
|
|
86
|
-
// Narrative: "It evaluated Rule 0 'Good credit': creditScore 750 gt 700, and chose approved."
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
### Builder
|
|
90
|
-
|
|
91
|
-
```typescript
|
|
92
|
-
import { flowChart, FlowChartBuilder } from 'footprintjs';
|
|
93
|
-
|
|
94
|
-
const chart = flowChart('Stage1', fn1, 'stage-1', { description: 'Description' })
|
|
95
|
-
.addFunction('Stage2', fn2, 'stage-2', 'Description')
|
|
96
|
-
.addDeciderFunction('Decide', deciderFn, 'decide', 'Route based on risk')
|
|
97
|
-
.addFunctionBranch('high', 'Reject', rejectFn)
|
|
98
|
-
.addFunctionBranch('low', 'Approve', approveFn)
|
|
99
|
-
.setDefault('high')
|
|
100
|
-
.end()
|
|
101
|
-
.build();
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
Methods: `start()`, `addFunction()`, `addStreamingFunction()`, `addDeciderFunction()`, `addSelectorFunction()`, `addListOfFunction()`, `addPausableFunction()`, `addSubFlowChart()`, `addSubFlowChartNext()`, `loopTo()`, `contract()`, `build()`, `toSpec()`, `toMermaid()`
|
|
105
|
-
|
|
106
|
-
### ScopeFacade (Internal — use TypedScope for new code)
|
|
107
|
-
|
|
108
|
-
```typescript
|
|
109
|
-
scope.getValue('key') // tracked read
|
|
110
|
-
scope.setValue('key', value) // tracked write
|
|
111
|
-
scope.getArgs<T>() // frozen readonly input (NOT tracked)
|
|
112
|
-
scope.getEnv() // frozen execution environment (NOT tracked)
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
**Three access tiers:**
|
|
116
|
-
- `getValue`/`setValue` — mutable shared state, tracked in narrative
|
|
117
|
-
- `getArgs()` — frozen business input from `run({ input })`, NOT tracked
|
|
118
|
-
- `getEnv()` — frozen infrastructure context from `run({ env })`, NOT tracked. Returns `ExecutionEnv { signal?, timeoutMs?, traceId? }`. Auto-inherited by subflows. Closed type.
|
|
119
|
-
|
|
120
|
-
### Executor
|
|
121
|
-
|
|
122
|
-
```typescript
|
|
123
|
-
const executor = new FlowChartExecutor(chart);
|
|
124
|
-
// With options (preferred over positional params):
|
|
125
|
-
const executor = new FlowChartExecutor(chart, { scopeFactory: myFactory });
|
|
126
|
-
await executor.run({ input: data, env: { traceId: 'req-123' } });
|
|
127
|
-
|
|
128
|
-
executor.attachScopeRecorder(recorder) // plug scope (data) observer
|
|
129
|
-
executor.attachFlowRecorder(r) // plug flow observer
|
|
130
|
-
executor.attachCombinedRecorder(r) // plug observer across all channels (routed by method-shape)
|
|
131
|
-
executor.attachEmitRecorder(r) // plug emit observer
|
|
132
|
-
executor.enableNarrative() // turn on the built-in combined narrative recorder
|
|
133
|
-
executor.getNarrativeEntries() // structured entries with type/depth/stageName/stageId
|
|
134
|
-
executor.getSnapshot() // full memory state (includes recorder snapshots)
|
|
135
|
-
executor.setRedactionPolicy({}) // PII protection
|
|
136
|
-
|
|
137
|
-
// Pause/Resume — human-in-the-loop
|
|
138
|
-
executor.isPaused() // true if last run paused
|
|
139
|
-
executor.getCheckpoint() // JSON-safe checkpoint (store in Redis/Postgres/etc.)
|
|
140
|
-
executor.resume(checkpoint, input) // continue from checkpoint with human's answer
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
### Pause/Resume (Human-in-the-Loop)
|
|
144
|
-
|
|
145
|
-
```typescript
|
|
146
|
-
import { flowChart, FlowChartExecutor } from 'footprintjs';
|
|
147
|
-
import type { PausableHandler } from 'footprintjs';
|
|
148
|
-
|
|
149
|
-
const handler: PausableHandler<MyState> = {
|
|
150
|
-
execute: async (scope) => {
|
|
151
|
-
// Return data = pause. Return nothing = continue.
|
|
152
|
-
return { question: `Approve $${scope.amount}?` };
|
|
153
|
-
},
|
|
154
|
-
resume: async (scope, input) => {
|
|
155
|
-
scope.approved = input.approved;
|
|
156
|
-
},
|
|
157
|
-
};
|
|
158
|
-
|
|
159
|
-
// Pausable root stage (single-stage subflows):
|
|
160
|
-
const chart = flowChart<MyState>('Approve', handler, 'approve').build();
|
|
161
|
-
|
|
162
|
-
// Or chained after other stages:
|
|
163
|
-
const chart2 = flowChart<MyState>('Seed', seedFn, 'seed')
|
|
164
|
-
.addPausableFunction('Approve', handler, 'approve')
|
|
165
|
-
.addFunction('Process', processFn, 'process')
|
|
166
|
-
.build();
|
|
167
|
-
|
|
168
|
-
const executor = new FlowChartExecutor(chart);
|
|
169
|
-
await executor.run();
|
|
170
|
-
|
|
171
|
-
if (executor.isPaused()) {
|
|
172
|
-
const checkpoint = executor.getCheckpoint(); // JSON-safe, store anywhere
|
|
173
|
-
// Later (hours, different server):
|
|
174
|
-
await executor.resume(checkpoint, { approved: true });
|
|
175
|
-
}
|
|
176
|
-
```
|
|
177
|
-
|
|
178
|
-
- `execute` returns data → pauses. Returns void → continues normally (conditional pause).
|
|
179
|
-
- Checkpoint is JSON-serializable — no functions, no class instances.
|
|
180
|
-
- Checkpoint is **deep-copied at creation** (one `structuredClone` of every field) — fully detached from engine state. Mutating a checkpoint you hold cannot corrupt the engine, and a same-executor resume cannot mutate a checkpoint you already persisted. `resume()` clones the checkpoint in too — the engine never holds a reference to your object.
|
|
181
|
-
- `getSnapshot().sharedState` is a zero-copy LIVE view in production — treat as read-only. Dev mode (`enableDevMode()`) returns a deep-frozen clone so consumer mutation throws.
|
|
182
|
-
- Checkpoints never capture recorder state: CROSS-executor resume (fresh executor from a stored checkpoint) starts with empty narrative — collect what you need before discarding the paused executor. SAME-executor resume preserves and accumulates narrative/recorder state (`preserveRecorders`). A fresh `runId` is generated for the resumed run either way.
|
|
183
|
-
- `FlowRecorder.onPause`/`onResume` and `Recorder.onPause`/`onResume` fire on both observer systems.
|
|
184
|
-
|
|
185
|
-
### ComposableRunner & Snapshot Navigation
|
|
186
|
-
|
|
187
|
-
```typescript
|
|
188
|
-
import type { ComposableRunner } from 'footprintjs';
|
|
189
|
-
import { getSubtreeSnapshot, listSubflowPaths } from 'footprintjs';
|
|
190
|
-
|
|
191
|
-
const subtree = getSubtreeSnapshot(snapshot, 'sf-payment');
|
|
192
|
-
listSubflowPaths(snapshot); // ['sf-payment', 'sf-outer/sf-inner']
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
## Observer Systems — three channels, one model
|
|
196
|
-
|
|
197
|
-
Three pluggable observer channels. All use the same dispatcher pattern
|
|
198
|
-
(`{ id, hooks } -> dispatcher -> error isolation -> attach/detach`).
|
|
199
|
-
Intentionally NOT unified into one giant interface — each channel has
|
|
200
|
-
a distinct invariant set.
|
|
201
|
-
|
|
202
|
-
**Recorder ID contract:**
|
|
203
|
-
- Every `attach*Recorder` call (`attachScopeRecorder` / `attachFlowRecorder` / `attachEmitRecorder` / `attachCombinedRecorder`) is **idempotent by ID** — same ID replaces, different IDs coexist. Prevents accidental double-counting.
|
|
204
|
-
- Built-in recorders use auto-increment default IDs (`metrics-1`, `debug-1`, ...) so multiple instances with different configs coexist naturally.
|
|
205
|
-
- Frameworks that auto-attach recorders should use a well-known ID (e.g., `new MetricRecorder('metrics')`) so the consumer can override it by passing the same ID, or add a second instance with `new MetricRecorder()` (gets unique ID).
|
|
206
|
-
|
|
207
|
-
**Delivery tiers (RFC-001):** every `attach*Recorder` accepts an options bag — `{ delivery: 'deferred', capture?, maxQueue?, overflow?, flushBudgetMs? }` (CombinedRecorder also honors the field form `{ id, delivery: 'deferred', ...hooks }`). Deferred observers are captured into ONE bounded, totally-ordered queue per executor (lazy — zero alloc without opt-in) and delivered at the next microtask checkpoint, "one beat behind"; the queue drains synchronously at run resolve/reject/pause (terminal flush) and `executor.drainObservers({timeoutMs})` settles async listeners. Capture happens strictly AFTER redaction at each dispatch site. Accounting: `snapshot.observerStats` (absent without opt-in). Default capture policy `'summary'` hands hooks a bounded `PayloadSummary` — use `'clone'` for inline-shape parity. Wiring lives in `runner/DeferredObserverTier.ts`; the pure pipeline in `lib/observer-queue/` + `lib/capture/envelope.ts` stays engine-import-free (the engine imports IT, never the reverse). Guide: [docs/guides/observers-deferred.md](docs/guides/observers-deferred.md).
|
|
208
|
-
|
|
209
|
-
**Scope Recorder** (`ScopeRecorder`; data ops — fires DURING stage execution):
|
|
210
|
-
- `onRead`, `onWrite`, `onCommit`, `onError`, `onStageStart`, `onStageEnd`, `onPause`/`onResume`, `onEmit`
|
|
211
|
-
- Built-in: `MetricRecorder`, `DebugRecorder`
|
|
212
|
-
|
|
213
|
-
**FlowRecorder** (control flow — fires AFTER stage execution):
|
|
214
|
-
- `onStageExecuted`, `onNext`, `onDecision`, `onFork`, `onSelected`, `onSubflowEntry/Exit`, `onSubflowRegistered`, `onLoop`, `onBreak`, `onError`, `onPause`/`onResume`, `onRunStart`/`onRunEnd`, `onRunFailed` (terminal counterpart to `onRunEnd` — closes the run boundary on error)
|
|
215
|
-
- All events carry `traversalContext: TraversalContext`
|
|
216
|
-
- `onDecision`/`onSelected` carry optional `evidence` from decide()/select()
|
|
217
|
-
- **`onStageExecuted` fires UNIFORMLY for every stage kind** (linear / decider / fork / selector / subflow-mount) as of v6.0+ proposal #003. Event payload carries `stageType: 'linear' | 'decider' | 'fork' | 'selector' | 'subflow-mount'`. Specialized events (`onDecision`/`onFork`/`onSelected`/`onSubflowEntry`) STILL fire — `onStageExecuted` is the universal "did this stage run" signal AFTER them.
|
|
218
|
-
- Built-in: 9 strategies (Narrative, Adaptive, Windowed, RLE, Milestone, Progressive, Separate, Manifest, Silent)
|
|
219
|
-
|
|
220
|
-
**StructureRecorder** (build-time chart shape — fires SYNCHRONOUSLY during builder operations, NOT runtime):
|
|
221
|
-
- `onStageAdded`, `onEdgeAdded`, `onLoopEdgeAdded`, `onDeciderComplete`, `onSubflowMounted`
|
|
222
|
-
- Attach via options bag: `flowChart('seed', fn, 'seed', { structureRecorders: [rec] })` OR fluent `.attachStructureRecorder(rec)`.
|
|
223
|
-
- **MOUNT-ONLY contract**: a recorder attached to a builder receives ONLY that builder's events. Subflow internals fire to THEIR builder's recorder, not the parent's. The mount event delivers the full subflow context via `subflowSpec` + `subflowPath` (proposal #001) — consumers walk it via `walkSubflowSpec` from `footprintjs/trace`.
|
|
224
|
-
- `StructureRecorder` + 6 event types now exported from the main `footprintjs` barrel (also available from `footprintjs/advanced`).
|
|
225
|
-
|
|
226
|
-
**EmitRecorder** (consumer-emitted events — third channel, see "Emit Channel" section below).
|
|
227
|
-
|
|
228
|
-
**CombinedRecorder** is a union shape that routes by runtime method-shape detection — implement only the hooks you care about across all three channels; one `attachCombinedRecorder` call. `CombinedNarrativeRecorder` is the canonical built-in; attach via `chart.recorder(narrative())` (the chart's `.recorder()` sugar) or `executor.attachCombinedRecorder(narrative())`.
|
|
229
|
-
|
|
230
|
-
**Stage type discrimination on `onStageExecuted`** — under proposal #003, consumers wanting linear-only behavior must filter:
|
|
231
|
-
```ts
|
|
232
|
-
onStageExecuted(event) {
|
|
233
|
-
if (event.stageType && event.stageType !== 'linear') return; // ignore decider/fork/selector/subflow-mount
|
|
234
|
-
// ... linear-stage logic
|
|
235
|
-
}
|
|
236
|
-
```
|
|
237
|
-
Built-in `NarrativeFlowRecorder` and `CombinedNarrativeRecorder` already gate this way — narrative output is byte-stable across the v6 transition.
|
|
238
|
-
|
|
239
|
-
## Event Ordering
|
|
240
|
-
|
|
241
|
-
```
|
|
242
|
-
1. Recorder.onStageStart — stage begins
|
|
243
|
-
2. Recorder.onRead/onWrite — DURING execution (buffered per-stage)
|
|
244
|
-
3. Recorder.onCommit — transaction flush
|
|
245
|
-
4. Recorder.onStageEnd — stage completes
|
|
246
|
-
5. FlowRecorder.onStageExecuted — CombinedNarrativeRecorder flushes buffered ops (LINEAR-only here; non-linear gated to specialized handlers — see above)
|
|
247
|
-
6. FlowRecorder.onNext/onDecision/onFork/onSelected — control flow events
|
|
248
|
-
7. FlowRecorder.onStageExecuted (with stageType !== 'linear') — fires for decider/fork/selector/subflow-mount AFTER the specialized event above
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
## Execution Tracing (`footprintjs/trace`)
|
|
252
|
-
|
|
253
|
-
Every stage execution gets a unique `runtimeStageId` — the universal key that links recorder events, commit log entries, and execution tree nodes.
|
|
254
|
-
|
|
255
|
-
**When to use:** Debugging (which stage set a value to something unexpected?), audit trails (trace every write to its source stage), custom recorders (correlate events with specific execution steps), quality trace backtracking (walk backwards to find where data quality dropped).
|
|
256
|
-
|
|
257
|
-
**Format:** `[subflowPath/]stageId#executionIndex`
|
|
258
|
-
|
|
259
|
-
```
|
|
260
|
-
seed#0 — root stage
|
|
261
|
-
call-llm#5 — 5th execution step
|
|
262
|
-
sf-tools/execute-tool-calls#8 — subflow stage
|
|
263
|
-
call-llm#9 — same stageId, different execution (loop)
|
|
264
|
-
```
|
|
265
|
-
|
|
266
|
-
**The commitLog:** An ordered array of `CommitBundle` — one per stage commit, recording what each stage wrote to shared state. Get it from `executor.getSnapshot().commitLog`. Encoding is governed by the **`commitValues: 'full' | 'delta'` dial** (#13c-B — completes the `readTracking`/`writeTracking` family, but LOSSLESS in both modes): under `'delta'`, array growth commits an `append` trace verb storing only the tail (kills the growing-history O(N²) retained heap), `deleteValue()` commits a real `delete` verb, and bundles carry one trace entry per surviving path. Replay reconstructs every step exactly. `bundle.overwrite[key]` becomes verb-qualified — use `commitValueAt` when you mean the full value. Snapshot discriminant: `getSnapshot().commitValues`.
|
|
267
|
-
|
|
268
|
-
```typescript
|
|
269
|
-
import { parseRuntimeStageId, findLastWriter, findCommit } from 'footprintjs/trace';
|
|
270
|
-
|
|
271
|
-
// Parse a runtimeStageId into components
|
|
272
|
-
parseRuntimeStageId('sf-tools/execute-tool-calls#8');
|
|
273
|
-
// → { stageId: 'execute-tool-calls', executionIndex: 8, subflowPath: 'sf-tools' }
|
|
274
|
-
|
|
275
|
-
// Get the commit log after execution
|
|
276
|
-
const snapshot = executor.getSnapshot();
|
|
277
|
-
const commitLog = snapshot.commitLog; // CommitBundle[]
|
|
278
|
-
|
|
279
|
-
// Backtrack: who last wrote 'systemPrompt' before commitLog array index 8?
|
|
280
|
-
// beforeIdx is the CommitBundle.idx (array position), NOT the executionIndex from runtimeStageId.
|
|
281
|
-
const writer = findLastWriter(commitLog, 'systemPrompt', 8);
|
|
282
|
-
// → CommitBundle | undefined (has .stage, .stageId, .runtimeStageId, .trace, .overwrite, .updates)
|
|
283
|
-
|
|
284
|
-
// Find by stageId: use findCommit when you know the stage.
|
|
285
|
-
// Use findLastWriter when you know the key but not which stage wrote it.
|
|
286
|
-
const llmCommit = findCommit(commitLog, 'call-llm', 'adapterRawResponse');
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
**Exports from `footprintjs/trace`:**
|
|
290
|
-
|
|
291
|
-
| Export | Returns | Use |
|
|
292
|
-
|--------|---------|-----|
|
|
293
|
-
| `buildRuntimeStageId(stageId, idx, subflowPath?)` | `string` | Construct an ID from components |
|
|
294
|
-
| `parseRuntimeStageId(id)` | `{ stageId, executionIndex, subflowPath }` | Decompose a runtimeStageId. `stageId` is LOCAL (not the full-prefixed form on `spec.id`) — use `splitStageId` for those |
|
|
295
|
-
| `splitStageId(prefixedStageId)` | `{ localStageId, subflowPath }` | Decompose a bare prefixed id (`spec.id`, `CommitBundle.stageId`, segment of `runtimeStageId` before `#`). Mirrors `parseRuntimeStageId`'s decomposition rule. Added in #002 |
|
|
296
|
-
| `walkSubflowSpec(spec, subflowPath, opts?)` | `Generator<WalkerItem>` | Walk a subflow spec delivered on `StructureSubflowMountedEvent.subflowSpec`. Yields `subflow-start` marker first, then `stage`/`edge`/`loop`/`subflow` items mirroring Structure event payload shapes. Auto-recurses with composed paths; `{recurse:false}` for single-level. Added in #001 |
|
|
297
|
-
| `findCommit(commitLog, stageId, key?)` | `CommitBundle \| undefined` | Find first commit by stageId |
|
|
298
|
-
| `findCommits(commitLog, stageId)` | `CommitBundle[]` | Find all commits by stageId |
|
|
299
|
-
| `findLastWriter(commitLog, key, beforeIdx?)` | `CommitBundle \| undefined` | Search backwards for who wrote a key |
|
|
300
|
-
| `commitValueAt(commitLog, idx, key)` | `unknown` | Reconstruct the FULL value of a key at a commit index — required under `commitValues: 'delta'` (#13c-B), where an `append` bundle's `overwrite[key]` holds only the tail |
|
|
301
|
-
| `causalChain` / `flattenCausalDAG` / `formatCausalChain` | functions | Backward program slicing over the commit-log DAG. RFC-003: `causalChain(log, id, keysRead, { controlDeps?, weigh?, maxDepth?, maxNodes? })` — `controlDeps` adds `kind:'control'` edges to the governing decider (labeled by the decide() rule label); `weigh` stamps consumer-computed `CausalEdge.weight` (engine never computes weights). Nodes carry `parentEdges: CausalEdge[]` (typed/keyed/weighted; `parents` kept for compat), `incompleteSources` (D2 honesty: stage also consumed args/env/unshadowed-silent → `⚠ … slice may be incomplete here`), and root `truncated?: {byDepth, byNodes}` (+ `⚠ slice truncated` footer + dev-warn) |
|
|
302
|
-
| `KeyedStore<T>` | class | **Primary** 1:1 Map store — own as a field on your recorder |
|
|
303
|
-
| `SequenceStore<T>` | class | **Primary** 1:N ordered store (has `getEntryRanges()` for O(1) time-travel) |
|
|
304
|
-
| `BoundaryStateStore<T>` | class | **Primary** transient bracket-scoped state store |
|
|
305
|
-
| `CommitRangeIndex<TLabel>` | class | Interval index over commit indices (`open`/`close`/`enclosing`/`overlapping`) |
|
|
306
|
-
| `topologyRecorder()` / `TopologyRecorder` | factory / class | Live composition graph for streaming consumers (subflow nodes + control-flow edges) |
|
|
307
|
-
| `inOutRecorder()` / `InOutRecorder` | factory / class | Chart in/out stream — `entry`/`exit` pairs at every chart boundary (top-level run + every subflow) |
|
|
308
|
-
| `controlDepRecorder()` / `ControlDepRecorder` | factory / class | RFC-003 D5: records `onDecision`/`onSelected` + the D1 runtime ancestor chain (`TraversalContext.parentRuntimeStageId`), answers "which decision allowed this stage to run?" — `ctrl.asLookup()` IS the `controlDeps` option. Nearest-decision semantics; correlates by runtime ids (never names); Convention-4 runId reset (control chains don't survive pause/resume) |
|
|
309
|
-
| `UntrackedSource` | type | `'args' \| 'env' \| 'silent'` — `CommitBundle.untrackedSources` (D2 honesty flags; absent when a stage's reads were fully tracked; silent reads shadowed by a tracked read of the same key are NOT flagged) |
|
|
310
|
-
| `QualityRecorder` + `qualityTrace`/`formatQualityTrace` | class / functions | Per-step quality scoring + Quality Stack Trace backtracking |
|
|
311
|
-
|
|
312
|
-
### TopologyRecorder — Composition Graph for Streaming Consumers
|
|
313
|
-
|
|
314
|
-
**One-liner:** reconstructs a live, queryable mini-flowchart of what your run actually traced, built from the 3 primitive recorder channels during traversal.
|
|
315
|
-
|
|
316
|
-
**Mental model:**
|
|
317
|
-
|
|
318
|
-
```
|
|
319
|
-
flowChart() builder → STATIC flowchart (design-time definition)
|
|
320
|
-
│
|
|
321
|
-
▼ executor runs it
|
|
322
|
-
Traversal emits events on 3 channels:
|
|
323
|
-
Recorder · FlowRecorder · EmitRecorder
|
|
324
|
-
│
|
|
325
|
-
▼ TopologyRecorder listens
|
|
326
|
-
DYNAMIC flowchart (runtime shape):
|
|
327
|
-
Nodes = composition points
|
|
328
|
-
(subflow / fork-branch / decision-branch)
|
|
329
|
-
Edges = transitions
|
|
330
|
-
(next / fork / decision / loop)
|
|
331
|
-
Queryable any moment — during or after run
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
**What it IS:**
|
|
335
|
-
- Live composition graph derived from 3 primitive channels
|
|
336
|
-
- Each node = one composition-significant moment (subflow entered, fork child, decision chosen)
|
|
337
|
-
- Each edge = a control-flow transition, timestamped with `runtimeStageId`
|
|
338
|
-
- Works identically during or after a run
|
|
339
|
-
|
|
340
|
-
**What it ISN'T:**
|
|
341
|
-
- Not a full execution tree — that's `StageContext` / `executor.getSnapshot()`
|
|
342
|
-
- Not per-stage data — that's `MetricRecorder` / a custom recorder composing `KeyedStore<T>`
|
|
343
|
-
- Not agent-specific — agentfootprint composes it; footprintjs owns it
|
|
344
|
-
|
|
345
|
-
**Why live consumers need it:** The executor already has the topology internally (execution tree in `StageContext`). But streaming consumers can't access that tree mid-run — they only see events. `TopologyRecorder` = "the tree, reconstructed from events, live-queryable."
|
|
346
|
-
|
|
347
|
-
Fills the gap between "post-run snapshot (full tree available)" and "live event stream (only point observations)." Attach once; query `getTopology()` anytime during or after a run.
|
|
348
|
-
|
|
349
|
-
```typescript
|
|
350
|
-
import { topologyRecorder } from 'footprintjs/trace';
|
|
351
|
-
|
|
352
|
-
const topo = topologyRecorder();
|
|
353
|
-
executor.attachCombinedRecorder(topo); // auto-routes to FlowRecorder channel
|
|
354
|
-
|
|
355
|
-
await executor.run({ input });
|
|
356
|
-
|
|
357
|
-
const { nodes, edges, activeNodeId, rootId } = topo.getTopology();
|
|
358
|
-
topo.getSubflowNodes(); // agent-centric view
|
|
359
|
-
topo.getByKind('fork-branch'); // all parallel branches
|
|
360
|
-
topo.getParallelSiblings(id); // siblings of a parallel branch
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
**Three node kinds — complete composition coverage:**
|
|
364
|
-
|
|
365
|
-
| Kind | Fires on | Represents |
|
|
366
|
-
|---|---|---|
|
|
367
|
-
| `subflow` | `onSubflowEntry` | Mounted subflow boundary (with stable `subflowId`) |
|
|
368
|
-
| `fork-branch` | `onFork` (synthesized one per child) | One branch of a parallel split — works for plain stages AND subflows |
|
|
369
|
-
| `decision-branch` | `onDecision` (synthesized for chosen) | The chosen branch of a conditional |
|
|
370
|
-
|
|
371
|
-
When a fork-branch or decision-branch target is also a subflow, the subsequent `onSubflowEntry` creates a subflow CHILD of the synthetic node. Layered shape preserves both "who branched" and "what the branch ran."
|
|
372
|
-
|
|
373
|
-
**Edges:** one per control-flow transition. `edge.kind ∈ 'next' | 'fork-branch' | 'decision-branch' | 'loop-iteration'`. Each carries `at: runtimeStageId` for time correlation.
|
|
374
|
-
|
|
375
|
-
**Correlation rules:**
|
|
376
|
-
- `onFork({ parent, children })` → N `fork-branch` nodes synthesized up-front; subsequent matching `onSubflowEntry` nests under the right fork-branch
|
|
377
|
-
- `onDecision({ chosen })` → `decision-branch` node synthesized up-front; matching `onSubflowEntry` nests under it
|
|
378
|
-
- Pending correlation clears on `onSubflowExit` so state doesn't leak across scopes
|
|
379
|
-
- `onLoop` → self-edge on the currently-active subflow (synthetic nodes don't participate)
|
|
380
|
-
- Re-entry of same `subflowId` (loop body) disambiguates via `id#n` suffix
|
|
381
|
-
|
|
382
|
-
**What it does NOT track:** plain sequential stages. Use `MetricRecorder` / `StageContext` for per-stage data. Topology is a graph of control-flow branching, not a full execution tree.
|
|
383
|
-
|
|
384
|
-
**For downstream libraries:** compose, don't duplicate. An agent-shaped recorder should wrap a `topologyRecorder()` internally and translate topology nodes into agent semantics — not re-implement subflow-stack + fork + decision tracking.
|
|
385
|
-
|
|
386
|
-
Example: [examples/runtime-features/flow-recorder/06-topology.ts](examples/runtime-features/flow-recorder/06-topology.ts)
|
|
387
|
-
|
|
388
|
-
### InOutRecorder — Chart In/Out Stream (every chart boundary, root + subflows)
|
|
389
|
-
|
|
390
|
-
**One-liner:** captures every chart execution (top-level run AND every subflow) as an `entry`/`exit` boundary pair, with the `inputMapper`/`outputMapper` payloads attached. Combined with `TopologyRecorder` (composition shape) this gives downstream layers the universal "step" primitive — `runtimeStageId` binds them.
|
|
391
|
-
|
|
392
|
-
**Mental model:**
|
|
393
|
-
|
|
394
|
-
```
|
|
395
|
-
user input ─►┌───────────────── run ─────────────────┐ ◄─ user output
|
|
396
|
-
│ __root__#0 onRunStart / onRunEnd │
|
|
397
|
-
│ │
|
|
398
|
-
│ inputMapper outputMapper │
|
|
399
|
-
│ │ │ │
|
|
400
|
-
│ parent ──►┤ subflow ├──► parent │
|
|
401
|
-
│ │ │ │
|
|
402
|
-
│ └── runtimeStageId ───┘ │
|
|
403
|
-
│ │
|
|
404
|
-
└────────────────────────────────────────┘
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
Each chart execution → 2 boundaries:
|
|
408
|
-
- **Root** — `onRunStart` / `onRunEnd` fire ONCE per `executor.run()`. `subflowId: '__root__'`, `depth: 0`, `isRoot: true`.
|
|
409
|
-
- **Subflow** — `onSubflowEntry` / `onSubflowExit` fire once per mounted subflow. Nested under root in the path tree (`['__root__', 'sf-x']`, depth 1+).
|
|
410
|
-
|
|
411
|
-
Loop re-entry produces distinct pairs because the parent stage's executionIndex increments.
|
|
412
|
-
|
|
413
|
-
**What it IS:**
|
|
414
|
-
- composes `SequenceStore<InOutEntry>` — flat ordered list + per-`runtimeStageId` index
|
|
415
|
-
- Captures the **payloads** at every chart boundary (what flowed IN and OUT)
|
|
416
|
-
- Path-aware: `subflowPath` is decomposed from the engine's path-prefixed `subflowId` and rooted under `__root__`
|
|
417
|
-
- Domain-agnostic — knows nothing about LLMs, tools, agents
|
|
418
|
-
|
|
419
|
-
**What it ISN'T:**
|
|
420
|
-
- Not a composition graph — that's `TopologyRecorder` (shape) vs this (data crossing each boundary)
|
|
421
|
-
- Not a full execution tree — that's `StageContext`
|
|
422
|
-
- Not agent-specific — domain libraries (e.g. agentfootprint) compose it; footprintjs owns it
|
|
423
|
-
|
|
424
|
-
```typescript
|
|
425
|
-
import { inOutRecorder, ROOT_SUBFLOW_ID } from 'footprintjs/trace';
|
|
426
|
-
|
|
427
|
-
const inOut = inOutRecorder();
|
|
428
|
-
executor.attachCombinedRecorder(inOut);
|
|
429
|
-
|
|
430
|
-
await executor.run({ input });
|
|
431
|
-
|
|
432
|
-
inOut.getSteps(); // entry boundaries (timeline; root is first step)
|
|
433
|
-
inOut.getBoundary(runtimeStageId); // { entry, exit } pair for one execution
|
|
434
|
-
inOut.getRootBoundary(); // { entry, exit } for the top-level run
|
|
435
|
-
inOut.getBoundaries(); // flat list (entry+exit interleaved)
|
|
436
|
-
inOut.getEntryRanges(); // O(1) per-step range index for time-travel
|
|
437
|
-
```
|
|
438
|
-
|
|
439
|
-
**`InOutEntry` shape:**
|
|
440
|
-
|
|
441
|
-
| Field | Description |
|
|
9
|
+
| src/lib/ | one job |
|
|
442
10
|
|---|---|
|
|
443
|
-
|
|
|
444
|
-
|
|
|
445
|
-
|
|
|
446
|
-
|
|
|
447
|
-
|
|
|
448
|
-
|
|
|
449
|
-
|
|
|
450
|
-
|
|
|
451
|
-
|
|
|
452
|
-
|
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
-
|
|
506
|
-
-
|
|
507
|
-
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
Gated diagnostics:
|
|
521
|
-
- **Circular-ref detection** in `ScopeFacade.setValue()` — O(n) WeakSet traversal per write
|
|
522
|
-
- **Empty-recorder warning** in `attachCombinedRecorder(r)` — catches `r` with no `on*` handler
|
|
523
|
-
- **Suspicious predicates** in `decide()` / `select()`
|
|
524
|
-
- **Snapshot integrity** in `getSubtreeSnapshot()`
|
|
525
|
-
- **Snapshot mutation guard** in `FlowChartExecutor.getSnapshot()` — `sharedState` becomes a deep-frozen CLONE (mutation throws); production returns the zero-copy live view (treat as read-only)
|
|
526
|
-
|
|
527
|
-
Convention: when adding a new dev-only check, gate on `isDevMode()` (from `scope/detectCircular.ts`). Do NOT use `process.env.NODE_ENV` inline — consumers control dev tooling centrally via `enableDevMode()`/`disableDevMode()`, and inline env checks break that contract.
|
|
528
|
-
|
|
529
|
-
## Break + Propagation
|
|
530
|
-
|
|
531
|
-
`scope.$break(reason?)` takes an optional free-form reason string that surfaces on `FlowBreakEvent.reason`. Recorders and narrative consumers see it.
|
|
532
|
-
|
|
533
|
-
By default, an inner subflow's `$break` stops ONLY the subflow; the parent continues. Opt into propagation via `SubflowMountOptions.propagateBreak: true`:
|
|
534
|
-
|
|
535
|
-
```ts
|
|
536
|
-
builder.addSubFlowChartNext('sf-escalate', escalateChart, 'Escalate', {
|
|
537
|
-
inputMapper: ..., outputMapper: ...,
|
|
538
|
-
propagateBreak: true, // ← inner $break → parent $break, with reason
|
|
539
|
-
});
|
|
540
|
-
```
|
|
541
|
-
|
|
542
|
-
Semantics:
|
|
543
|
-
- **Linear chain:** inner `$break(reason)` → parent's `breakFlag` flips → next parent stage does NOT run → `FlowBreakEvent` fires at parent-mount level with `propagatedFromSubflow` + reason.
|
|
544
|
-
- **Nested chain:** propagates through every hop that opted in. Reason survives.
|
|
545
|
-
- **outputMapper still runs** before propagation — subflow's partial state lands in parent before the break. Escape hatch: early-return `{}` from outputMapper when the break state is set.
|
|
546
|
-
- **Parallel/fan-out:** existing ChildrenExecutor rule applies — parent breaks only when ALL fork children broke. `propagateBreak: true` on a single child contributes to that count; doesn't terminate the fork alone.
|
|
547
|
-
|
|
548
|
-
Example: [examples/runtime-features/break/04-subflow-propagate.ts](examples/runtime-features/break/04-subflow-propagate.ts).
|
|
549
|
-
|
|
550
|
-
## Parallel Fan-Out Error Semantics (`failFast`)
|
|
551
|
-
|
|
552
|
-
When a selector picks ≥2 branches — or `addListOfFunction` lists several — they run **in parallel** via `ChildrenExecutor`. One branch throwing has two meanings; the **fan-out node's `failFast` flag** picks which:
|
|
553
|
-
|
|
554
|
-
- **DEFAULT** (`failFast` unset/false) → `Promise.allSettled`: best-effort. A branch error is **collected, not rethrown**; every sibling finishes, the run **resolves**, and the post-fan-out convergence stage still runs.
|
|
555
|
-
- **`failFast: true`** → `Promise.all`: the **first** branch error **rejects the whole run** (aborts before convergence). Use when every selected branch is **REQUIRED**.
|
|
556
|
-
|
|
557
|
-
**The footgun:** under the default, a *required* parallel branch that throws is **silently swallowed** — the run resolves with a half-built result (this is exactly what swallowed a failing Tools slot in the agent request-assembly fork). Set `failFast: true` so the failure surfaces.
|
|
558
|
-
|
|
559
|
-
**Set it on the fan-out node** — honored uniformly across plain-function branches, **subflow** branches (`addSubFlowChartBranch`), and `addListOfFunction`:
|
|
560
|
-
|
|
561
|
-
```ts
|
|
562
|
-
flowChartSelector('Pick', selectorFn, 'pick', { failFast: true }) // root selector
|
|
563
|
-
builder.addSelectorFunction('Pick', selectorFn, 'pick', 'desc', { failFast: true }) // mid-chain (5th arg)
|
|
564
|
-
builder.addListOfFunction([...], { failFast: true }) // bare parallel list
|
|
565
|
-
```
|
|
566
|
-
|
|
567
|
-
Defaults to `false` everywhere — existing charts are unchanged. `failFast` (a branch *threw*) is orthogonal to `$break` (a branch *chose* to stop). Full guide: [docs/guides/error-handling.md](docs/guides/error-handling.md#parallel-fan-out-error-semantics). Example: [examples/runtime-features/parallel/01-failfast.ts](examples/runtime-features/parallel/01-failfast.ts).
|
|
568
|
-
|
|
569
|
-
## Emit Channel (Phase 3)
|
|
570
|
-
|
|
571
|
-
Third observer channel alongside `Recorder` (data-flow) and `FlowRecorder` (control-flow). Consumer stage code emits structured events; `EmitRecorder.onEmit(event)` fires synchronously with auto-enriched context.
|
|
572
|
-
|
|
573
|
-
```ts
|
|
574
|
-
import type { EmitRecorder, EmitEvent } from 'footprintjs';
|
|
575
|
-
|
|
576
|
-
// Inside a stage:
|
|
577
|
-
scope.$emit('myapp.llm.tokens', { input: 100, output: 50 });
|
|
578
|
-
|
|
579
|
-
// Recorder observes:
|
|
580
|
-
const rec: EmitRecorder = {
|
|
581
|
-
id: 'token-meter',
|
|
582
|
-
onEmit: (e) => { if (e.name === 'myapp.llm.tokens') tally(e.payload); },
|
|
583
|
-
};
|
|
584
|
-
executor.attachEmitRecorder(rec);
|
|
585
|
-
```
|
|
586
|
-
|
|
587
|
-
### Semantics
|
|
588
|
-
- **Pass-through.** Delivered synchronously, in call order. Zero allocation when no recorder attached (fast-path in `ScopeFacade.emitEvent`).
|
|
589
|
-
- **Auto-enriched.** Events carry `stageName`, `runtimeStageId`, `subflowPath`, `pipelineId`, `timestamp` — parsed from `runtimeStageId` for subflow context.
|
|
590
|
-
- **Error-isolated.** A throwing `onEmit` doesn't propagate; errors route to `onError` on other recorders.
|
|
591
|
-
- **Redactable.** `RedactionPolicy.emitPatterns: RegExp[]` matches `event.name`; matched payloads become `'[REDACTED]'` before dispatch.
|
|
592
|
-
- **Buffered in narrative.** `CombinedNarrativeRecorder.onEmit` buffers alongside reads/writes; flushed in `flushOps` so emit entries appear AFTER the stage header in ordered narrative.
|
|
593
|
-
|
|
594
|
-
### Naming convention
|
|
595
|
-
Hierarchical dotted names — `<namespace>.<category>.<event>`. Examples:
|
|
596
|
-
- `'agentfootprint.llm.tokens'`, `'agentfootprint.llm.request'`
|
|
597
|
-
- `'myapp.billing.spend'`, `'myapp.auth.check'`
|
|
598
|
-
|
|
599
|
-
### Legacy primitives route through this channel
|
|
600
|
-
`$debug`, `$metric`, `$error`, `$eval`, `$log` also dispatch on the emit channel (in addition to their existing `DiagnosticCollector` side-bag writes for snapshot inclusion):
|
|
601
|
-
|
|
602
|
-
```
|
|
603
|
-
$debug(key, value) → emits 'log.debug.${key}'
|
|
604
|
-
$error(key, value) → emits 'log.error.${key}'
|
|
605
|
-
$metric(name, value) → emits 'metric.${name}'
|
|
606
|
-
$eval (name, value) → emits 'eval.${name}'
|
|
607
|
-
```
|
|
608
|
-
|
|
609
|
-
This closes the long-standing gap where `$metric` / `$debug` went to side bags no recorder observed. Backward-compat: the side bags still populate for consumers that inspect snapshots directly.
|
|
610
|
-
|
|
611
|
-
### Customizing narrative rendering
|
|
612
|
-
`NarrativeFormatter.renderEmit?(ctx)` hook renders an emit event into a narrative line. Return `string` to use, `null` to exclude, `undefined` to fall back to the default `[emit] name: payloadSummary`.
|
|
613
|
-
|
|
614
|
-
Example: [examples/runtime-features/emit/01-custom-events.ts](examples/runtime-features/emit/01-custom-events.ts).
|
|
615
|
-
|
|
616
|
-
## Detach — Fire-and-Forget Child Flowcharts (`footprintjs/detach`)
|
|
617
|
-
|
|
618
|
-
Schedule a child chart on a chosen **driver** without blocking the parent stage. Two semantics + two surfaces:
|
|
619
|
-
|
|
620
|
-
| Method | Returns | Caller |
|
|
621
|
-
|---------------------------------------|----------------|------------------------------------------|
|
|
622
|
-
| `scope.$detachAndJoinLater(d, c, i)` | `DetachHandle` | Inside a stage (refId = stage's runtimeStageId) |
|
|
623
|
-
| `scope.$detachAndForget(d, c, i)` | `void` | Inside a stage (handle discarded) |
|
|
624
|
-
| `executor.detachAndJoinLater(d, c, i)`| `DetachHandle` | Outside any chart (refId prefix `__executor__`) |
|
|
625
|
-
| `executor.detachAndForget(d, c, i)` | `void` | Outside any chart |
|
|
626
|
-
|
|
627
|
-
```ts
|
|
628
|
-
import { microtaskBatchDriver } from 'footprintjs/detach';
|
|
629
|
-
|
|
630
|
-
flowChart('process', async (scope) => {
|
|
631
|
-
scope.result = await heavyWork();
|
|
632
|
-
scope.$detachAndForget(microtaskBatchDriver, telemetryChart, { event: 'done' });
|
|
633
|
-
}, 'process').build();
|
|
634
|
-
```
|
|
635
|
-
|
|
636
|
-
**Built-in drivers** (more in v4.17.1):
|
|
637
|
-
- `microtaskBatchDriver` — coalesces N detaches into one `queueMicrotask` flush. Default for in-process.
|
|
638
|
-
- `immediateDriver` — runs sync inside `schedule()`. Test fixture / debugging aid.
|
|
639
|
-
|
|
640
|
-
**Custom drivers**: `createMicrotaskBatchDriver(runChild)` / `createImmediateDriver(runChild)` accept a custom `ChildRunner` so consumers can wrap the executor (e.g., for tracing context). Drivers are **passed explicitly** as the first arg — no library-default to keep the engine free of driver imports.
|
|
641
|
-
|
|
642
|
-
**Handle**: `{ id, status: 'queued'|'running'|'done'|'failed', result?, error?, wait() }`. NOT Promise-shaped (no `.then()` — defeats fire-and-forget). Status is sync property; `wait()` returns a CACHED Promise on every call.
|
|
643
|
-
|
|
644
|
-
**Graceful shutdown**: `flushAllDetached({ timeoutMs })` drains every in-flight handle to terminal. Use in SIGTERM handlers / test cleanup. Returns `{ done, failed, pending }` — `pending === 0` means full drain.
|
|
645
|
-
|
|
646
|
-
**Gotcha**: don't store handles in shared state — `StageContext.setValue` calls `structuredClone`, which drops the handle's class prototype (and `.wait()` method). Keep handles in closure-local variables. The builder-native `addDetachAndJoinLater` enforces this by delivering the handle to a consumer-supplied `onHandle` callback rather than to a shared-state key.
|
|
647
|
-
|
|
648
|
-
**Builder-native composition** — make detach a labeled chart stage (visible in narrative + visualizations):
|
|
649
|
-
|
|
650
|
-
```ts
|
|
651
|
-
const handles: DetachHandle[] = [];
|
|
652
|
-
const chart = flowChart('process', processFn, 'process')
|
|
653
|
-
.addDetachAndForget('telemetry', telemetryChart, {
|
|
654
|
-
driver: microtaskBatchDriver,
|
|
655
|
-
inputMapper: (scope) => ({ event: 'processed', orderId: scope.orderId }),
|
|
656
|
-
})
|
|
657
|
-
.addDetachAndJoinLater('eval', evalChart, {
|
|
658
|
-
driver: microtaskBatchDriver,
|
|
659
|
-
inputMapper: (s) => s.input,
|
|
660
|
-
onHandle: (h) => handles.push(h),
|
|
661
|
-
})
|
|
662
|
-
.addFunction('join', async (scope) => {
|
|
663
|
-
const settled = await Promise.all(handles.map((h) => h.wait()));
|
|
664
|
-
scope.results = settled.map((r) => r.result);
|
|
665
|
-
}, 'join')
|
|
666
|
-
.build();
|
|
667
|
-
```
|
|
668
|
-
|
|
669
|
-
Pure sugar over `addFunction` — zero engine changes. For server-side concurrent runs, allocate a fresh `handles` closure per run (factory-build the chart) so handles don't bleed across requests.
|
|
670
|
-
|
|
671
|
-
Examples: [examples/runtime-features/detach/](examples/runtime-features/detach/) — 7 scenarios (telemetry, fan-out, bare-executor, immediate driver, error handling, status polling, graceful shutdown).
|
|
672
|
-
|
|
673
|
-
## Combined Recorder
|
|
674
|
-
|
|
675
|
-
A `CombinedRecorder` is an observer that hooks into multiple event streams (scope data-flow, control-flow, AND emit — all three channels). One object, one `id`, one `attachCombinedRecorder()` call — the library routes to the right channels via runtime method-shape detection.
|
|
676
|
-
|
|
677
|
-
```ts
|
|
678
|
-
import type { CombinedRecorder } from 'footprintjs';
|
|
679
|
-
import { isFlowEvent } from 'footprintjs';
|
|
680
|
-
|
|
681
|
-
const audit: CombinedRecorder = {
|
|
682
|
-
id: 'audit',
|
|
683
|
-
onWrite: (e) => log('scope write', e.key), // Recorder stream
|
|
684
|
-
onDecision: (e) => log('routed to', e.chosen), // FlowRecorder stream
|
|
685
|
-
onError: (e) => {
|
|
686
|
-
// Shared method — union payload. Discriminate with isFlowEvent():
|
|
687
|
-
if (isFlowEvent(e)) log('flow error in', e.stageName);
|
|
688
|
-
else log('scope error during', e.operation);
|
|
689
|
-
},
|
|
690
|
-
};
|
|
691
|
-
|
|
692
|
-
executor.attachCombinedRecorder(audit);
|
|
693
|
-
```
|
|
694
|
-
|
|
695
|
-
Built on `CombinedRecorder`: `CombinedNarrativeRecorder` (the `executor.enableNarrative()` default). Consumers implement ONLY the events they care about — `Partial<ScopeRecorder> & Partial<FlowRecorder> & Partial<EmitRecorder>` under the hood.
|
|
696
|
-
|
|
697
|
-
**Detection rule:** only OWN event-method properties count (prototype methods are ignored for security — prevents accidental `Object.prototype` pollution from attaching handlers).
|
|
698
|
-
|
|
699
|
-
## Anti-Patterns
|
|
700
|
-
|
|
701
|
-
- Never post-process the tree — use recorders
|
|
702
|
-
- Don't use `getValue()`/`setValue()` in TypedScope stages — use typed property access
|
|
703
|
-
- Don't use `$`-prefixed state keys (e.g., `$break`) — they collide with ScopeMethods
|
|
704
|
-
- Don't use deprecated `CombinedNarrativeBuilder` — use `CombinedNarrativeRecorder`
|
|
705
|
-
- Don't extract shared base for ScopeRecorder/FlowRecorder — two instances = coincidence
|
|
706
|
-
- Don't use `getArgs()` for tracked data — use typed scope properties
|
|
707
|
-
- Don't put infrastructure data in `getArgs()` — use `getEnv()` via `run({ env })`
|
|
708
|
-
- Don't manually create `CombinedNarrativeRecorder` — `chart.recorder(narrative())` (or `executor.attachCombinedRecorder(narrative())`) handles it
|
|
709
|
-
- Don't return full arrays from `outputMapper` without `arrayMerge: ArrayMergeMode.Replace` — default `applyOutputMapping` **concatenates** arrays (`[...parent, ...subflow]`). Either return only the **delta** (new items), or set `arrayMerge: ArrayMergeMode.Replace` on `SubflowMountOptions` to overwrite instead of concatenate. Scalars are always replaced regardless.
|
|
710
|
-
|
|
711
|
-
## Project conventions (5.0+)
|
|
712
|
-
|
|
713
|
-
### Convention 1 — One purpose per recorder
|
|
714
|
-
|
|
715
|
-
A recorder owns exactly ONE concern (storage, OR event ingestion, OR state machine, OR projection). Multi-concern recorders MUST be decomposed into single-purpose pieces and composed via a thin facade.
|
|
716
|
-
|
|
717
|
-
Use composition: own a `SequenceStore<T>` / `KeyedStore<T>` / `BoundaryStateStore<T>` field, implement the relevant `ScopeRecorder` / `FlowRecorder` / `EmitRecorder` / `CombinedRecorder` interface, delegate event handling to internal helpers, delegate storage to the store. See `examples/recorders/` for canonical patterns.
|
|
718
|
-
|
|
719
|
-
Composition is the ONLY recorder model. The abstract base classes (`SequenceRecorder`, `KeyedRecorder`, `BoundaryStateTracker`) were removed in 7.0.0 — there is no inheritance path. Every recorder (including the built-ins `MetricRecorder`, `QualityRecorder`, `InOutRecorder`, `CombinedNarrativeRecorder`) owns a store as a field.
|
|
720
|
-
|
|
721
|
-
### Convention 2 — Examples are mandatory integration tests
|
|
722
|
-
|
|
723
|
-
Every library-surface change MUST include:
|
|
724
|
-
|
|
725
|
-
1. Unit tests (per-pattern coverage, all 7 test types — see Convention 3).
|
|
726
|
-
2. **Integration tests via `examples/`** — runnable end-to-end demos that exercise the feature in realistic scenarios. Each example file is treated as part of the test suite.
|
|
727
|
-
3. Documentation update (relevant README + `CLAUDE.md` if architectural).
|
|
728
|
-
|
|
729
|
-
PRs without all three are incomplete. Examples are not optional polish — they ARE the integration-test layer that catches "works in unit tests, fails in real usage" bugs.
|
|
730
|
-
|
|
731
|
-
### Convention 3 — 7 test types per feature
|
|
732
|
-
|
|
733
|
-
Every new piece (each store, each recorder, each runtime feature) ships with the following test types. One test file per type when natural, or sections in one file for tightly-scoped primitives.
|
|
734
|
-
|
|
735
|
-
| Type | Asks |
|
|
736
|
-
|---|---|
|
|
737
|
-
| **Unit** | Does this single function/class behave correctly in isolation? |
|
|
738
|
-
| **Functional** | Does this feature work end-to-end on the happy path? |
|
|
739
|
-
| **Integration** | Do multiple components cooperate correctly? |
|
|
740
|
-
| **Property** | Does the invariant hold for ANY input (randomized fuzzing)? |
|
|
741
|
-
| **Security** | Does this protect against injection, leakage, redaction bypass? |
|
|
742
|
-
| **Performance** | Is the latency / memory within budget? |
|
|
743
|
-
| **Load** | Does it sustain throughput at scale? |
|
|
744
|
-
|
|
745
|
-
### Convention 4 — `runId` for per-run scoping
|
|
746
|
-
|
|
747
|
-
Every event the engine fires carries a `runId` in `traversalContext`. Generated fresh per `executor.run()` and per `executor.resume()`; shared across all events of one run; differs across consecutive runs of the same executor. Recorders that accumulate state across runs detect "new run" via `event.traversalContext.runId !== this.lastRunId` and reset transient bookkeeping. See `examples/runtime-features/run-id/`.
|
|
748
|
-
|
|
749
|
-
## Build & Test
|
|
750
|
-
|
|
751
|
-
```bash
|
|
752
|
-
npm run build # tsc (CJS) + tsc -p tsconfig.esm.json (ESM)
|
|
753
|
-
npm test # full suite
|
|
754
|
-
npm run test:unit
|
|
755
|
-
```
|
|
756
|
-
|
|
757
|
-
Dual output: CommonJS (`dist/`) + ESM (`dist/esm/`) + types (`dist/types/`)
|
|
11
|
+
| capture/ | retention policies + bounded payload summarization (leaf; shared by tracking dials AND deferred-observer capture) |
|
|
12
|
+
| memory/ | transactional state: SharedMemory heap, StageContext frames, TransactionBuffer staging, EventLog commit log — PLUS trace analysis (backtrack.ts causalChain, commitLogUtils.ts) |
|
|
13
|
+
| schema/ | duck-typed validation (`.safeParse/.parse` ⇒ 'parseable') |
|
|
14
|
+
| builder/ | fluent DSL → FlowChartSpec → StageNode graph; build-time StructureRecorder channel. NOT standalone: imports runner's makeRunnable (FlowChartBuilder.ts:18) |
|
|
15
|
+
| scope/ | ScopeFacade per stage (tracked get/set, $emit, frozen args/env); built-in ScopeRecorders in scope/recorders/ |
|
|
16
|
+
| reactive/ | TypedScope deep Proxy over the facade |
|
|
17
|
+
| decide/ | decide()/select() evidence capture |
|
|
18
|
+
| recorder/ | stores (KeyedStore/SequenceStore/BoundaryStateStore), CombinedRecorder routing, built-ins (Topology/InOut/ControlDep/Quality) |
|
|
19
|
+
| slice/ | variable-first backward slicing (triage query layer): sliceForKey (anchor at last writer → causalChain) + arrayProvenance/elementProvenance (append-fold element births — the agent mega-key fix) + KeysReadSource strategies. Pure post-hoc queries; imports memory/ ONLY (see src/lib/slice/README.md) |
|
|
20
|
+
| pause/ | PauseSignal / FlowchartCheckpoint types |
|
|
21
|
+
| engine/ | FlowchartTraverser + handlers/; narrative/ = the whole FlowRecorder channel (dispatcher + 9 strategies), not just text |
|
|
22
|
+
| runner/ | FlowChartExecutor wiring (runtime, recorders, pause/resume, snapshot, DeferredObserverTier). RunContext.ts is a fluent run builder, NOT the threaded per-run context |
|
|
23
|
+
| observer-queue/ | pure deferred-delivery pipeline (engine imports IT, never reverse) |
|
|
24
|
+
| contract/, detach/ | I/O schema + OpenAPI; fire-and-forget child charts (drivers passed explicitly) |
|
|
25
|
+
|
|
26
|
+
## Core state & flow
|
|
27
|
+
- `SharedMemory` (memory/SharedMemory.ts:14) — run-namespaced heap; committed state is **immutable-after-swap** (applySmartMerge clones+swaps).
|
|
28
|
+
- `StageContext` (memory/StageContext.ts:29) — per-stage frame: lazy TransactionBuffer + first-touch view; `commit()` (:531) applies patch → redacted mirror → EventLog.record → onCommit.
|
|
29
|
+
- `CommitBundle`/`EventLog` — one bundle per executed stage, `bundle.idx` == array position; encoding governed by `commitValues: 'full'|'delta'`.
|
|
30
|
+
- `TraversalContext` (engine/narrative/types.ts:172) — THE per-event context (runId, runtimeStageId, parentRuntimeStageId, subflowPath, loopIteration); stamped on every FlowRecorder event.
|
|
31
|
+
- `runtimeStageId` = `[subflowPath/]stageId#executionIndex`; counter shared by reference into subflow traversers → globally unique; NOT reset on resume.
|
|
32
|
+
- Subflows get a fresh isolated ExecutionRuntime (SubflowExecutor.ts:119); inputMapper seeds, outputMapper merges back (arrays CONCAT unless `arrayMerge: Replace`).
|
|
33
|
+
|
|
34
|
+
**Event order (code-verified; older docs had 3/4 swapped):** onStageStart → onRead/onWrite (live, PRE-commit) → **onStageEnd (StageRunner.ts:81) → onCommit (StageContext.ts:587)** → FlowRecorder onDecision/onFork/onSelected/onSubflowEntry → onStageExecuted (uniform, with stageType). Error path: commit STILL happens (FlowchartTraverser.ts:1088) before onError+rethrow — a failing stage's writes land.
|
|
35
|
+
|
|
36
|
+
## Extension points
|
|
37
|
+
- **New stage kind**: boolean flag on `StageNode` (engine/graph/StageNode.ts:39 — kinds are flags, not an enum) + builder method on FlowChartBuilder (pattern: addPausableFunction :1264; fire structure events endpoint-before-edge) + phase in the hard-coded chain `executeNodeStep` (FlowchartTraverser.ts:801 — no handler registry; new handler class takes HandlerDeps, engine/types.ts:344, instantiated in traverser ctor :402-413). Must also edit 5 type-union sites: engine/types.ts:477, builder/types.ts:50, builder/types.ts:104, engine/types.ts:427, and computeNodeType's flag→type mapping (RuntimeStructureManager.ts:19-21 — the only one the compiler won't flag; miss it and the new kind silently serializes as 'stage'). Zero-engine alternative: pure sugar over addFunction (pattern: addDetachAndForget :1343).
|
|
38
|
+
- **Decider/selector**: addDeciderFunction :1433 → DeciderList (:53); addSelectorFunction :1478 → SelectorFnList (:428); runtime match on `child.branchId ?? child.id` (DeciderHandler.ts:133). Evidence via decide()/select() (decide/decide.ts:169/218).
|
|
39
|
+
- **Recorder**: channels = ScopeRecorder (scope/types.ts:137), FlowRecorder (engine/narrative/types.ts:425), EmitRecorder (recorder/EmitRecorder.ts:121 — attachEmitRecorder just delegates to the scope channel), CombinedRecorder (recorder/CombinedRecorder.ts:98, routed by method-shape), StructureRecorder (builder/structure/StructureRecorder.ts:347, build-time). Attach via executor.attach*Recorder — idempotent by id. **Sync rule: a new hook name MUST also go into `RECORDER_EVENT_METHODS`/`FLOW_…`/`EMIT_…` (CombinedRecorder.ts:198/219/247)** — they drive both combined routing AND the deferred tier's taps; miss it and deferred recorders silently never see the event. Detection accepts class-prototype methods (blocks only Object.prototype — CombinedRecorder.ts:266). New built-in = compose a store as a field (no base classes) + factory in src/recorders.ts.
|
|
40
|
+
- **Scope $-method (4-file checklist)**: ScopeFacade capability + ReactiveTarget (reactive/types.ts:20); ScopeMethods type (:74); METHOD_ROUTES (createTypedScope.ts:46); SCOPE_METHOD_NAMES (reactive/types.ts:262) — miss #4 and the name becomes a state key.
|
|
41
|
+
- **Contract**: builder.contract() (:1056) → makeRunnable toOpenAPI/toMCPTool (runner/RunnableChart.ts:83/153). BYO schema is duck-typed (schema/detect.ts:24).
|
|
42
|
+
- **Scope provider**: ProviderResolver (scope/providers/types.ts:35) via registerScopeResolver (registry.ts:15; exported from /advanced). Simpler: custom scopeFactory (engine/types.ts:73).
|
|
43
|
+
- **Detach driver**: DetachDriver (detach/types.ts:186); no registry — passed explicitly.
|
|
44
|
+
- **Observability dial** (pattern): capture/ primitive + FlowChartExecutorOptions option + snapshot discriminant — FOUR dials now: readTracking/writeTracking/commitValues/writeProvenance (#P1: `'reads-prefix'` stamps `TraceEntry.readKeys` = keys tracked-read before each write; consumed by causalChain `edgeAttribution: 'per-write'` and sliceForKey's default). Propagation is the same 6-site pattern for all four.
|
|
45
|
+
- **Closed seams that look open**: ExecutionEnv (fixed type); decide() operators (evaluator.ts:19 module-private); METHOD_ROUTES; the executeNodeStep phase chain; emit dispatcher (rides scope channel).
|
|
46
|
+
|
|
47
|
+
## Change-impact map
|
|
48
|
+
- **Trace verbs** (`set|merge|append|delete`, memory/types.ts:42) → FOUR verb-switch replicas in lockstep: applySmartMerge (utils.ts:254 — live commit AND EventLog.materialise), commitValueAt (commitLogUtils.ts:82), TransactionBuffer.toDeltaPayload/replayPathVerbs (:248-328); plus causalChain/findLastWriter readers.
|
|
49
|
+
- **StageContext.commit / dials** → dial propagation is TRIPLICATED: ExecutionRuntime.use* (:131-161), createNext/createChild inheritance (StageContext.ts:619/640), SubflowExecutor duck-push (:151-173) — miss one and subflows silently run the default. Applies to all FOUR dials incl. writeProvenance (#P1).
|
|
50
|
+
- **runtimeStageId format** (`#`/`/` delimiters) → string parsers everywhere: parseRuntimeStageId, ScopeFacade._getSubflowPath, subflowResults dual-keying, checkpoint lean-filter, narrative buffering, store keys.
|
|
51
|
+
- **Subflow id prefixing** → DUPLICATED prefixer: builder _prefixNodeTree (:1968) and traverser prefixNodeTree (FlowchartTraverser.ts:1356) are byte-twins; change both + stageMap key composition + resume drilling.
|
|
52
|
+
- **Checkpoint shape** → buildPauseCheckpoint (FlowChartExecutor.ts:987) ↔ resume() validation (:690) ↔ checkpointSanitize ↔ SubflowExecutor resume-seed (skip-inputMapper, :69-72).
|
|
53
|
+
- **Redaction** has FIVE enforcement points: facade get/set values, _stageWrites placeholder, commit-log redactPatch, redacted mirror, emitPatterns.
|
|
54
|
+
- **Traverser phases** → node-shape reads must use eff* overlay accessors (:646-703) or dynamic StageNode returns break; tail continuations must return ContinuationHops or stacks regrow; decider flat-dispatch must keep InvokerStamp or pause loses its invoker.
|
|
55
|
+
|
|
56
|
+
## End-to-end trace (seed → decider → loop branch → finish)
|
|
57
|
+
build: flowChart() → FlowChartBuilder.start → addDeciderFunction → branch {loopTo} plants stub `next={id,isLoopRef:true}` (:346) → build() → makeRunnable.
|
|
58
|
+
run: executor.run (FlowChartExecutor.ts:1455): re-entrancy guard → fresh runId → createTraverser (:336; composes scopeFactory: TypedScope + recorders + redaction :379-444) → new ExecutionRuntime (SharedMemory+EventLog+root StageContext) → traverser.execute (:501) fires onRunStart → trampoline executeNode (:738; flat hops, recursion only for forks/subflows/decider-with-next, depth cap 500).
|
|
59
|
+
per stage: executeNodeStep stamps runtimeStageId (:818) → StageRunner.run: scopeFactory → onStageStart → stage fn (writes stage into TransactionBuffer via StageContext.setObject; onWrite fires live) → onStageEnd → traverser calls context.commit() (:1094) → onCommit → onStageExecuted (narrative flushes buffered ops here).
|
|
60
|
+
decider: DeciderHandler.prepareDispatch (:89) — runs stage, **commits BEFORE branch resolution** (:124), matches branchId, fires onDecision(+evidence) → onStageExecuted('decider') → flat hop with InvokerStamp.
|
|
61
|
+
loop: Phase 6 sees isLoopRef → ContinuationResolver.resolveTarget (:107): id-map lookup, iteration guard (max 1000), onLoop → hop; zero stack, state carries forward (never rewound).
|
|
62
|
+
end: onRunEnd (throw → onRunFailed; pause → neither) → deferred terminalFlush → getSnapshot() = {sharedState (LIVE view; dev-mode frozen clone), commitLog, executionTree, subflowResults (dual-keyed), recorders}.
|
|
63
|
+
|
|
64
|
+
## Backtracking
|
|
65
|
+
Five mechanisms, no state rollback anywhere: **M1** TransactionBuffer staging + net-change commit — commit-on-error by design, explicitly NOT rollback (TransactionBuffer.ts:13-18; error path commits then rethrows). **M2** Pause/Resume checkpointing — the only resume-from-prior-point; checkpoint is one detached structuredClone; resume rebuilds the cursor from pausedStageId+subflowPath and only overrides the LEAF subflow root (2+-deep pause re-executes outer pre-mount stages). **M3** commit-log replay (EventLog.materialise, commitValueAt — required under delta mode). **M4** loopTo re-entry — forward execution over accumulated state, bounded by maxIterations + dynamicNextHops. **M5** causalChain backward slicing (read-only analysis; honesty flags for untracked reads; `edgeAttribution: 'per-write'` refines edges via `TraceEntry.readKeys` when the writeProvenance dial recorded them — worklist, subset-of-ceiling safe). **M6 (query layer)** slice/ — variable-first triage: sliceForKey + append-fold element provenance + the ONLY safe serializations (sliceToJSON/formatSlice — never JSON.stringify a slice root). Deep dive + step tables: [.claude/rules/backtracking.md](.claude/rules/backtracking.md).
|
|
66
|
+
|
|
67
|
+
## Invariants (assumed, not stated)
|
|
68
|
+
- Committed state immutable-after-swap; first-touch views hold bare references on that guarantee — in-place mutation of SharedMemory.context corrupts every in-flight stage.
|
|
69
|
+
- Reads are borrowed live references (recorders get them un-cloned) — never mutate.
|
|
70
|
+
- executionIndex globally monotonic per run; continues across resume. One CommitBundle per executed stage; empty commits are deliberate cursor stops.
|
|
71
|
+
- runtimeStageId stamped BEFORE the stage runs — the entire event-correlation model rests on scope+flow events sharing it.
|
|
72
|
+
- One executor = one run at a time (_isExecuting guard). State values must survive structuredClone.
|
|
73
|
+
- Loop-ref stubs deliberately VIOLATE node-id uniqueness — every graph search must skip `isLoopRef` first.
|
|
74
|
+
- runId regenerates per run() AND per resume(); recorders detect new runs via traversalContext.runId (engine never resets them).
|
|
75
|
+
- Recorder errors never abort traversal (isolated at invokeHook/emitEvent).
|
|
76
|
+
- Deferred-observer capture default at the executor tier is **'clone'** (DeferredObserverTier.ts:167), not 'summary'.
|
|
77
|
+
|
|
78
|
+
## Landmines
|
|
79
|
+
1. TypedScope's set trap JSON-round-trips EVERY object write (createTypedScope.ts:29-40, :363) — Date→string, Map→{}, undefined drops; `$setValue` bypasses it, so the two write paths store different bytes.
|
|
80
|
+
2. Fork "parent breaks when ALL children broke" is implemented (ChildrenExecutor.ts:53) but **unwired** — every live call site passes parentBreakFlag=undefined; fork breaks do NOT propagate in real runs.
|
|
81
|
+
3. TransactionBuffer.set stores the RAW reference in workingCopy but a CLONE in overwritePatch (:49-56) — post-write mutation changes what the stage reads back and the net-change filter, while committing stale bytes.
|
|
82
|
+
|
|
83
|
+
## Pointers
|
|
84
|
+
- [.claude/rules/backtracking.md](.claude/rules/backtracking.md) — the 5 mechanisms with step tables + pseudocode
|
|
85
|
+
- [docs/guides/](docs/guides/) (error-handling, observers-deferred) · [examples/](examples/) — mandatory integration tests (Convention 2); 7 test types per feature (Convention 3)
|
|
86
|
+
- Build/test: `npm run build` (CJS+ESM+types), `npm test`
|