@luispm/zflow-graph 0.1.0

package/docs/02-runtime.md

@@ -0,0 +1,257 @@
+# 2 · The Runtime
+
+This is the part most graph libraries don't have, and it's what makes zflow useful for real applications.
+
+## The core idea
+
+A **graph** is data. A **runtime** turns that data into computation.
+
+When you call `flow.run()`, the runtime walks every node in **topological order** and calls its `execute(ctx, inputs)` function. The return value becomes the node's output. Outputs flow through edges to downstream nodes as their inputs.
+
+```
+[Source: random] → [×2] → [if >100] ─ ok ─→ [save]
+                                    └ bad ─→ [alert]
+```
+
+If you give each box an `execute`, that diagram becomes a running program. That's it.
+
+## Minimum viable example
+
+```js
+flow.registerKind({
+  name: 'gen',
+  nin: 0, nout: 1,
+  portOut: ['value'],
+  execute: () => ({ value: Math.random() * 100 }),
+});
+
+flow.registerKind({
+  name: 'log',
+  nin: 1, nout: 0,
+  portIn: ['value'],
+  execute: (ctx, ins) => { console.log('got', ins.value); },
+});
+
+const a = flow.addNode({ kind: 'gen' });
+const b = flow.addNode({ kind: 'log' });
+flow.addEdge({ from: a, to: b });
+
+await flow.run();
+// console: "got 73.2"
+```
+
+## How values flow through edges
+
+The runtime expects `execute` to return **either** a primitive **or** an object whose keys match the kind's `portOut` labels:
+
+```js
+// Single output, one downstream port: return either form is fine.
+execute: () => 42                          // primitive
+execute: () => ({ value: 42 })             // object with 'value'
+execute: () => ({ result: 42 })            // works if portOut: ['result']
+
+// Multi-output (conditional routing): emit only the branch you want.
+execute: (ctx, ins) => ins.x > 0
+  ? { positive: ins.x }    // only this key is set → only 'positive' edge fires
+  : { negative: ins.x };
+```
+
+The downstream node receives values keyed by **its** `portIn` labels:
+
+```js
+flow.registerKind({
+  name: 'add',
+  nin: 2, nout: 1,
+  portIn:  ['a', 'b'],
+  portOut: ['sum'],
+  execute: (ctx, ins) => ({ sum: ins.a + ins.b }),
+});
+```
+
+If no port labels are declared, inputs are exposed as `in0`, `in1`, `in2`, etc. (also indexable by number: `ins[0]`).
+
+## The `ctx` object
+
+Every `execute` receives a context with:
+
+```js
+execute: async (ctx, ins) => {
+  ctx.nodeId          // this node's id
+  ctx.signal          // AbortSignal — abort if flow.stop() is called
+  ctx.params          // params set via flow.setNodeParams(id, params)
+  ctx.emit(value)     // intermediate emission (for streaming)
+  ctx.setProgress(p)  // 0..1 → drawn as a bar inside the node
+  ctx.log(...args)    // emits a 'node:log' event
+  ctx.metric(v)       // push a number to the live sparkline
+  ctx.get(otherId)    // read the latest value of another node
+  return { ... };     // final output
+}
+```
+
+## Async, retry, abort
+
+```js
+flow.registerKind({
+  name: 'fetch-user',
+  retry: { n: 3, delay: 500 },                    // 3 attempts, 500ms between
+  execute: async (ctx, ins) => {
+    ctx.setProgress(0.1);
+    const r = await fetch(`/api/users/${ins.id}`, { signal: ctx.signal });
+    if (!r.ok) throw new Error(`HTTP ${r.status}`);
+    ctx.setProgress(0.9);
+    return { user: await r.json() };
+  },
+});
+
+const p = flow.run();
+setTimeout(() => flow.stop(), 5000);   // give up after 5s
+await p;
+```
+
+If a node throws after exhausting retries, its status becomes `'error'` and the runtime emits `node:error`. By default it continues with other nodes; set `flow.options.stopOnError = true` to abort.
+
+## Streaming nodes (`async function*`)
+
+A node can emit **multiple values over time** by returning an async generator:
+
+```js
+flow.registerKind({
+  name: 'tick',
+  nin: 0, nout: 1,
+  execute: async function* (ctx) {
+    for (let i = 0; i < 10; i++) {
+      if (ctx.signal.aborted) return;
+      yield { count: i };
+      await new Promise((r) => setTimeout(r, 500));
+    }
+  },
+});
+```
+
+Each `yield` propagates through downstream nodes **before** the next yield runs. So a `tick → log` graph prints 10 times, not 1.
+
+## Conditional routing
+
+Declare `portOut` labels and emit only the branches you want:
+
+```js
+flow.registerKind({
+  name: 'threshold',
+  nin: 1, nout: 2,
+  portIn:  ['value'],
+  portOut: ['high', 'low'],
+  execute: (ctx, ins) => ins.value > 100
+    ? { high: ins.value }   // 'low' edge does NOT fire
+    : { low: ins.value },   // 'high' edge does NOT fire
+});
+```
+
+Downstream nodes on the **non-firing** branch don't execute that tick. This is how you build if/else flow.
+
+## Run modes
+
+```js
+await flow.run();                        // whole graph
+await flow.runFrom(nodeId);              // only nodeId and its descendants
+await flow.runFrame(frameId);            // only nodes inside a frame
+await flow.run({ filter: (id) => ... }); // arbitrary predicate
+
+flow.startLoop(500);                     // re-run every 500ms forever
+flow.stopLoop();
+flow.stop();                             // abort current run
+```
+
+## Memoization
+
+If your inputs don't change, why re-compute?
+
+```js
+flow.setMemoization(true);
+
+// First run executes everything
+await flow.run();
+
+// Second run skips nodes whose inputs hash matches the previous run
+await flow.run();   // → most nodes hit cache; only changed ones re-run
+```
+
+Uses FNV-1a 32-bit hash (~80µs for ~1k-entry inputs). The cache invalidates per-node when its hash changes.
+
+## Step-through debugging
+
+```js
+flow.setBreakpoint(suspectNodeId);
+
+flow.on('run:paused', ({ nodeId }) => {
+  console.log('paused at', nodeId);
+  // Inspect:
+  console.log('inputs:', flow._values.get(predecessor));
+});
+
+await flow.run();   // pauses at breakpoint
+// In response to a UI button:
+flow.stepOver();    // execute the breakpoint node and pause again at next
+flow.resume();      // exit debug mode, continue normally
+```
+
+Or step through everything:
+
+```js
+flow.setStepMode(true);
+await flow.run();           // pauses before every node
+flow.stepOver();             // advance one at a time
+```
+
+## Driving the runtime from external state
+
+Sometimes the graph has source nodes whose values you want to inject from outside:
+
+```js
+const sourceId = flow.addNode({ kind: 'gen' });
+
+flow.setNodeInput(sourceId, { value: 42 });    // injects value
+await flow.run();
+// gen's execute didn't run; the injected value flows downstream
+```
+
+This is how UI controls (sliders, text inputs) drive a running graph.
+
+## Reading the result
+
+```js
+await flow.run();
+flow.getNodeValue(sinkId);        // → { received: 42 }
+flow._values.get(sinkId);         // → same thing
+```
+
+## Events
+
+```js
+flow.on('run:start',   ({ order }) => {});
+flow.on('run:done',    ({ executed, errors, values }) => {});
+flow.on('run:paused',  ({ nodeId }) => {});
+flow.on('node:exec',   ({ id, inputs }) => {});
+flow.on('node:emit',   ({ id, outputs }) => {});     // includes streaming emissions
+flow.on('node:done',   ({ id, outputs }) => {});
+flow.on('node:error',  ({ id, error }) => {});
+flow.on('node:retry',  ({ id, attempt, error }) => {});
+flow.on('node:cached', ({ id }) => {});               // memoization skip
+flow.on('node:log',    ({ id, args }) => {});         // from ctx.log()
+```
+
+## Visual cues automatic during a run
+
+| What you see                              | Why                                        |
+| ----------------------------------------- | ------------------------------------------ |
+| Blue pulsing border on a node             | status === 'running'                       |
+| Floating bubble with the emitted value    | per emission, auto                         |
+| Edge thickens + glows blue                | data is currently flowing through it       |
+| Progress bar inside node                  | `ctx.setProgress(p)` was called            |
+| Sparkline at bottom of node               | numeric outputs accumulate over time       |
+| Status dot top-right (green/red/blue)     | `status` field                             |
+
+You can set `flow.setRunStepDelay(ms)` to pause between nodes so propagation is **visible**. Default 250ms — set to 0 for production speed.
+
+## Next
+
+→ [Designing Kinds](./03-kinds.md) — schemas, retry, and patterns for executable nodes

package/docs/03-kinds.md

@@ -0,0 +1,282 @@
+# 3 · Designing Kinds
+
+Kinds are the API surface your end-users see. Treat them like the public API of your app — name them well, give them sensible defaults, validate their inputs.
+
+## Full kind spec
+
+```js
+flow.registerKind({
+  // ── Visual ────────────────────────────────────────────────────────────
+  name: 'http-get',           // required, unique
+  color: '#5b8def',           // CSS hex, used for header + ports
+  badge: 'H',                 // 1-2 chars or emoji, shown top-left
+  w: 200, h: 80,              // default size in world units
+  shape: 'rect',              // 'rect' | 'diamond' | 'ellipse' | 'hexagon' | 'circle' | 'round' | 'subroutine'
+
+  // ── Ports ─────────────────────────────────────────────────────────────
+  nin: 1, nout: 2,            // number of input and output ports
+  portIn:  ['url'],           // labels (used as keys in execute's `ins`)
+  portOut: ['body', 'error'], // labels for outputs
+
+  // ── Schema (optional) ─────────────────────────────────────────────────
+  inputs:  [{ name: 'url',   type: 'string', required: true }],
+  outputs: [{ name: 'body',  type: 'string' },
+            { name: 'error', type: 'string' }],
+
+  // ── Runtime ───────────────────────────────────────────────────────────
+  execute: async (ctx, ins) => { /* ... */ },
+  retry:   { n: 3, delay: 500 },     // automatic retry policy
+
+  // ── HTML overlay (alternative to canvas) ──────────────────────────────
+  html: false,
+  template: null,                     // see below
+});
+```
+
+Every field is optional except `name`. Defaults are sane.
+
+## Naming conventions that won't bite you later
+
+- Use **kebab-case** for `name`: `http-get`, `sql-query`, `email-send`
+- Reserve `process` / `input` / `output` for the built-ins
+- Group with prefixes: `db-read`, `db-write`, `db-migrate`
+- The `name` is what shows up in `flow.toJSON()` — once you ship, **don't rename**, or old saved graphs break. Add a new kind and deprecate the old one.
+
+## Ports: labels vs indices
+
+Without `portIn`/`portOut`, inputs are addressable only by index:
+
+```js
+flow.registerKind({ name: 'foo', nin: 2 /* no portIn */, execute: (ctx, ins) => {
+  ins[0];   // first input
+  ins[1];   // second
+  ins.in0;  // alias
+  ins.in1;
+}});
+```
+
+With labels, you also get named access:
+
+```js
+flow.registerKind({ name: 'foo', nin: 2, portIn: ['user', 'token'], execute: (ctx, ins) => {
+  ins.user;
+  ins.token;
+  // ins[0] and ins.in0 still work
+}});
+```
+
+**Always declare labels** for kinds with more than one port. It makes the executor readable.
+
+## Schema (validation at connection time)
+
+Schemas don't change runtime behavior — they prevent invalid connections at the editor level:
+
+```js
+flow.registerKind({
+  name: 'add',
+  inputs:  [{ name: 'a', type: 'number' }, { name: 'b', type: 'number' }],
+  outputs: [{ name: 'sum', type: 'number' }],
+});
+
+flow.registerKind({
+  name: 'concat',
+  inputs:  [{ name: 'text', type: 'string' }],
+});
+
+// In the editor, dragging from add's 'sum' port to concat's 'text' port
+// shows a red toast "type mismatch: number → string" and rejects.
+```
+
+Built-in compatibility rules:
+- Same type → ok
+- Either side is `'any'` → ok
+- Numeric types (`number`, `int`, `float`, `integer`) are interchangeable
+- `string` accepts everything (string is the universal type — everything stringifies)
+- Otherwise → mismatch
+
+You can override the rules entirely with a custom validator:
+
+```js
+flow.setConnectionValidator((fromN, fp, toN, tp) => {
+  // Block self-loops
+  if (fromN === toN) return false;
+  // Otherwise allow
+  return true;
+});
+```
+
+## Patterns: source nodes
+
+Sources have `nin: 0`. They're triggered when `flow.run()` reaches them in topo order:
+
+```js
+flow.registerKind({
+  name: 'now',
+  nin: 0, nout: 1,
+  portOut: ['ms'],
+  execute: () => ({ ms: Date.now() }),
+});
+```
+
+Or driven externally:
+
+```js
+flow.registerKind({ name: 'input-slot', nin: 0, nout: 1 });   // no execute
+
+const slot = flow.addNode({ kind: 'input-slot' });
+flow.setNodeInput(slot, { value: 42 });
+// → every downstream tick sees 42 from this slot
+```
+
+## Patterns: sink nodes
+
+Sinks have `nout: 0`. They produce side effects (DB write, UI update, log):
+
+```js
+flow.registerKind({
+  name: 'console-log',
+  nin: 1, nout: 0,
+  portIn: ['value'],
+  execute: (ctx, ins) => { console.log(ins.value); /* return undefined */ },
+});
+```
+
+A sink's return value (if any) is still stored in `flow._values` so debug UI can show it, but downstream nodes don't get it (there are none).
+
+## Patterns: branching / control flow
+
+```js
+flow.registerKind({
+  name: 'switch',
+  nin: 1, nout: 3,
+  portIn:  ['value'],
+  portOut: ['gt100', 'gt50', 'else'],
+  execute: (ctx, ins) => {
+    if (ins.value > 100) return { gt100: ins.value };
+    if (ins.value > 50)  return { gt50:  ins.value };
+    return { else: ins.value };
+  },
+});
+```
+
+Downstream nodes on **non-firing** branches don't execute that tick. They sit idle.
+
+## Patterns: stateful nodes
+
+Need state across runs? Stash it on the node via `setNodeParams`:
+
+```js
+flow.registerKind({
+  name: 'counter',
+  nin: 0, nout: 1,
+  execute: (ctx) => {
+    const p = ctx.params;
+    p.count = (p.count || 0) + 1;
+    flow.setNodeParams(ctx.nodeId, p);
+    return { count: p.count };
+  },
+});
+```
+
+Or use closure-captured Maps for module-level state:
+
+```js
+const counts = new Map();
+flow.registerKind({
+  name: 'counter',
+  execute: (ctx) => {
+    counts.set(ctx.nodeId, (counts.get(ctx.nodeId) || 0) + 1);
+    return { count: counts.get(ctx.nodeId) };
+  },
+});
+```
+
+## Patterns: streaming
+
+```js
+flow.registerKind({
+  name: 'every',
+  nin: 0, nout: 1,
+  execute: async function* (ctx) {
+    while (!ctx.signal.aborted) {
+      yield { tick: Date.now() };
+      await new Promise((r) => setTimeout(r, ctx.params.intervalMs || 1000));
+    }
+  },
+});
+
+const t = flow.addNode({ kind: 'every' });
+flow.setNodeParams(t, { intervalMs: 500 });
+flow.run();   // runs forever (until flow.stop())
+```
+
+## Patterns: HTML overlay nodes
+
+When canvas isn't enough — e.g., you want a form node with input fields:
+
+```js
+flow.registerKind({
+  name: 'form',
+  html: true,
+  template: `
+    <div style="padding:12px;">
+      <input class="user-input" placeholder="name" style="width:100%;padding:6px;background:#0b0f17;border:1px solid #5b8def;border-radius:4px;color:white;">
+      <button class="submit-btn" style="margin-top:8px;width:100%;padding:6px;background:#5b8def;color:white;border:0;border-radius:4px;">Submit</button>
+    </div>
+  `,
+  w: 220, h: 110, nin: 0, nout: 1,
+  execute: (ctx) => ({ value: ctx.params.lastSubmit || null }),
+});
+
+// Hook up listeners after creating the node:
+const id = flow.addNode({ kind: 'form', x: 0, y: 0 });
+// Wait one frame for the DOM to mount, then:
+requestAnimationFrame(() => {
+  const el = flow._htmlOverlays.get(id);
+  el.querySelector('.submit-btn').onclick = () => {
+    const val = el.querySelector('.user-input').value;
+    flow.setNodeParams(id, { lastSubmit: val });
+    flow.runFrom(id);
+  };
+});
+```
+
+HTML nodes are real DOM. They participate in pan/zoom (their position is synced every frame). Events bubble normally.
+
+## Validation in `execute`
+
+Schemas catch wrong types at edit time, but you should still validate at runtime:
+
+```js
+execute: async (ctx, ins) => {
+  if (typeof ins.url !== 'string') throw new Error('url required');
+  if (!ins.url.startsWith('http')) throw new Error('url must be absolute');
+  // ...
+}
+```
+
+Errors surface in `node:error` events and color the node red.
+
+## Documenting kinds for end-users
+
+If your app is an editor for end-users, expose a palette / sidebar. Save kind metadata for tooltips:
+
+```js
+flow.registerKind({
+  name: 'http-get',
+  meta: {
+    description: 'Send an HTTP GET request and return the body.',
+    docs: 'https://yourapp.com/docs/http-get',
+  },
+});
+
+// In your sidebar UI:
+const cat = flow.kinds[flow.kindByName.get('http-get')];
+console.log(cat.meta?.description);
+```
+
+(Anything you pass to `registerKind` not in the spec list above is preserved on the kind object.)
+
+## Next
+
+→ [Performance at Scale](./04-performance.md) — make it fly with 50k+ nodes