@luispm/zflow-graph 0.1.0

package/docs/04-performance.md

@@ -0,0 +1,218 @@
+# 4 · Performance at Scale
+
+zflow-graph is engineered to keep 60 fps at 100k nodes. This isn't an accident — it's the result of specific architectural choices you should understand if you want to push it.
+
+## What runs where
+
+```
+JS heap                          WASM linear memory
+─────────────                    ──────────────────
+canvas2d ctx                     pos_x[]      ← Float32 view
+WebGL context (opt)              pos_y[]      ← Float32 view
+event listeners                  size_w[]     ← Float32 view
+plugin instances                 size_h[]     ← Float32 view
+metric history                   selected[]   ← Uint8  view
+notes, frames (small)            edge_from[]  ← Uint32 view
+                                 edge_to[]    ← Uint32 view
+                                 spatial grid ← Zig-only
+                                 undo stack   ← Zig-only
+```
+
+Reading `flow.V.posX[i]` is a **direct memory access** into WASM. There is no copy.
+
+Mutating these arrays from JS works too — `flow.V.posX[5] = 100` moves node 5. But invalidate caches with `flow._gl?.markNodeDirty(5)` if WebGL is on.
+
+## The three speeds
+
+### Speed 1: small graphs (< 1k nodes)
+Canvas2D handles everything. Pan, zoom, drag, rich text rendering — all at 60 fps. Don't even think about WebGL.
+
+### Speed 2: medium graphs (1k – 5k nodes)
+Canvas2D still works but starts to wobble with shadows and gradients. Two options:
+
+```js
+flow.options.edgeFlowSpeed = 0;     // disable particle animation
+flow.options.hoverPreview = false;  // disable popover
+```
+
+Or just turn on WebGL:
+
+```js
+await flow.enableWebGL();
+```
+
+### Speed 3: large graphs (> 5k nodes)
+WebGL is mandatory. Canvas2D for the body + edges is too slow. With WebGL:
+
+- All node bodies → **one** instanced draw call
+- All edges → **one** line draw call
+- Text/badges/ports → still Canvas2D, but **LOD-gated** (skipped below zoom 0.4)
+- Camera moves → uniform-only, **zero buffer uploads**
+
+```js
+const flow = await ZFlow.create({
+  container,
+  wasmUrl: '/zflow.wasm',
+  webglThreshold: 2000,   // auto-enable past this many nodes
+});
+```
+
+## Bulk operations
+
+Adding nodes one by one is fine for ~100. Above that, bulk:
+
+```js
+// SLOW — emits change + runs hooks + dirties everything 50,000 times
+for (let i = 0; i < 50000; i++) flow.addNode({ kind: 'process', x: i * 10, y: 0 });
+
+// FAST — same result in ~50ms
+flow.addNodesBulk(
+  Array.from({ length: 50000 }, (_, i) => ({ kind: 'process', x: i * 10, y: 0 }))
+);
+```
+
+Same for edges:
+
+```js
+flow.addEdgesBulk(specs);
+```
+
+Both return arrays of created ids in input order. They suspend event emission during the loop and emit a single `change` at the end.
+
+## LOD (Level of Detail)
+
+The library auto-degrades detail when zoom is low:
+
+| Zoom         | What renders                                  |
+| ------------ | --------------------------------------------- |
+| `> 0.4`      | Everything: text, ports, badges, shadows, descriptions, sparklines |
+| `0.35 – 0.4` | Skip text. Ports + selection visible.         |
+| `0.25 – 0.35`| Skip ports too. Bodies + edges only.          |
+| `< 0.25`     | Skip canvas overlay entirely. WebGL bodies only. |
+
+With > 5,000 nodes, the threshold is bumped to `0.55` automatically — the rationale is that 5k nodes at zoom 0.4 are unreadable anyway.
+
+You can override:
+
+```js
+// Currently the thresholds are internal constants. To override, fork the lib
+// or skip your own detail-drawing in a plugin's beforeRender hook.
+```
+
+## Viewport culling
+
+Two layers:
+
+1. **Spatial grid in Zig** — `queryRect(minX, minY, maxX, maxY)` returns node ids whose AABB intersects the rect. O(1) average, O(k) where k = visible nodes. The library uses this for any graph > 300 nodes.
+
+2. **JS frustum check** — for every returned id, verify it's actually inside the viewport. Cheap second pass.
+
+This means the cost of rendering doesn't scale with total nodes — it scales with **visible** nodes. Zoomed out so 5 nodes are visible? You pay for 5 draws (plus the GL buffer which is constant per-frame).
+
+## Memory profile
+
+At 100k nodes, the WASM uses ~40 MB of linear memory:
+
+- Node SoA: ~5 MB (8 typed arrays × 100k × varying widths)
+- Edge SoA: ~3 MB (200k edges)
+- Spatial grid: ~1 MB
+- Snapshot stack (8 deep): ~33 MB
+
+JS-side maps (titles, descriptions, colors) only allocate for nodes that actually have those fields. With `null` rich content, JS heap is < 5 MB even at 100k nodes.
+
+If you need more than 100k nodes per graph, you have two options:
+- Edit `NODE_CAP` in `src/core.zig` and rebuild WASM
+- Use multiple flow instances and link them via a parent UI
+
+## Drag performance
+
+When you drag N selected nodes, the runtime needs to mark all edges touching them as dirty (so the GL bezier buffer regenerates). The naive implementation is `O(N × edges)` — for 25 nodes × 200k edges = 5M ops per frame, unusable.
+
+zflow caches an adjacency map per-node:
+
+```js
+flow._ensureAdj();   // O(edges) once
+flow._nodeAdj[id]    // → [edgeIdx, edgeIdx, ...] for that node
+```
+
+Drag becomes `O(N × avgDegree)` which is microseconds. The cache invalidates on add/delete edge.
+
+## Memoization
+
+For grids with many sources that don't change often:
+
+```js
+flow.setMemoization(true);
+```
+
+Each node's input hash (FNV-1a) is recorded after exec. On the next run, if the hash matches, the node is skipped and `node:cached` fires. Fast (~80µs per node for 1k-entry inputs).
+
+Caveats:
+- Skipped nodes don't update sparkline / status
+- If `execute` has side effects (DB writes, network), you don't want this on
+- Memoization invalidates per-node, not graph-wide — a downstream node may still re-run if its inputs hash changed
+
+## Streaming + downstream propagation
+
+Each `yield` from an async generator propagates through downstream nodes **synchronously** before the next yield. This means 10 yields × 4 downstream nodes = 40 executions, all in sequence.
+
+If you need parallel pipelines, use separate streams:
+
+```js
+const a = flow.addNode({ kind: 'stream' });
+const b = flow.addNode({ kind: 'stream' });
+// They run independently — flow.run() will start both in parallel because
+// async generators are awaited per node, but the topo walker processes them
+// in order. For true parallelism, use Promise.all in your executor.
+```
+
+## Web Workers (manual)
+
+The library runs entirely on the main thread. For CPU-heavy executors:
+
+```js
+flow.registerKind({
+  name: 'heavy-compute',
+  execute: async (ctx, ins) => {
+    const worker = new Worker('./heavy-worker.js', { type: 'module' });
+    return new Promise((resolve, reject) => {
+      worker.onmessage = (e) => { worker.terminate(); resolve(e.data); };
+      worker.onerror = reject;
+      ctx.signal.addEventListener('abort', () => worker.terminate());
+      worker.postMessage(ins);
+    });
+  },
+});
+```
+
+(Future versions may move auto-layout and force-layout to workers automatically.)
+
+## Profiling
+
+Easy fps monitor:
+
+```js
+let frames = 0, t0 = performance.now();
+const orig = flow._loop.bind(flow);
+flow._loop = () => { frames++; orig(); };
+setInterval(() => {
+  console.log('fps:', frames * 1000 / (performance.now() - t0));
+  frames = 0; t0 = performance.now();
+}, 500);
+```
+
+Or use the Chrome DevTools Performance tab. Look for:
+- `_render` taking > 16ms per frame → CPU bound, try WebGL
+- `_drawNode` calls > 1000/frame → enable viewport culling (already on > 300 nodes)
+- Long GC pauses → check if you're allocating in a render hook
+
+## What we don't optimize (yet)
+
+- **Layout for huge graphs** — Sugiyama and force layout both run on main thread. Above 5k nodes they freeze the UI. Run them in a worker for now.
+- **Streaming with very high frequency** — yielding > 200x/sec saturates the event loop with bubble animations. Throttle inside your generator.
+- **HTML overlay nodes** — each one is a real DOM element. Don't create 1000 of them. Use canvas-rendered kinds at scale, HTML only for special-case interactive nodes.
+- **Search at very large graphs** — `flow.search(query)` is O(n). For > 10k nodes, build your own index.
+
+## Next
+
+→ [Plugins](./05-plugins.md) — extend behavior without forking

package/docs/05-plugins.md

@@ -0,0 +1,235 @@
+# 5 · Plugin System
+
+Plugins are how you extend zflow without forking. A plugin is a plain object with optional lifecycle hooks.
+
+## Anatomy
+
+```js
+const myPlugin = {
+  name: 'autosave',                     // for logging / display
+
+  init:    (flow) => { /* called once on use() */ },
+  dispose: (flow) => { /* called when removed */ },
+
+  // Render hooks
+  beforeRender: (flow, ctx) => {},
+  afterRender:  (flow, ctx) => {},
+
+  // Mutation hooks
+  onNodeAdd:    (flow, nodeId, spec) => {},
+  onNodeDelete: (flow, nodeId) => {},
+  onEdgeAdd:    (flow, edgeIdx, spec) => {},
+
+  // Runtime hooks
+  onBeforeExec: (flow, nodeId, inputs) => {},   // return false → skip exec
+  onAfterExec:  (flow, nodeId, output) => {},
+
+  // Other
+  onConnect:    (flow, fromN, fp, toN, tp) => {}, // return false → reject
+  onChange:     (flow) => {},
+  onSelectionChange: (flow, ids) => {},
+
+  // Bulk additions
+  kinds:    [{ name: 'foo', execute: ... }],
+  commands: [{ label: 'Do thing', run: () => {} }],
+  extendAPI: (flow) => { flow.myMethod = () => 42; },
+};
+
+const dispose = flow.use(myPlugin);
+// ... later ...
+dispose();   // remove the plugin
+```
+
+All fields are optional. Plugins with no hooks are valid (e.g., for bundling kinds).
+
+## Example: autosave to localStorage
+
+```js
+flow.use({
+  name: 'autosave',
+  init: (f) => {
+    const saved = localStorage.getItem('graph');
+    if (saved) f.loadJSON(JSON.parse(saved));
+  },
+  onChange: (f) => {
+    localStorage.setItem('graph', JSON.stringify(f.toJSON()));
+  },
+});
+```
+
+## Example: fps overlay
+
+```js
+flow.use({
+  name: 'fps',
+  _frames: 0, _t0: 0, _fps: 0,
+  init() { this._t0 = performance.now(); },
+  afterRender: function (f, ctx) {
+    this._frames++;
+    const now = performance.now();
+    if (now - this._t0 > 500) {
+      this._fps = Math.round(this._frames * 1000 / (now - this._t0));
+      this._frames = 0; this._t0 = now;
+    }
+    ctx.save();
+    ctx.font = '600 11px ui-monospace, Consolas, monospace';
+    ctx.fillStyle = this._fps >= 55 ? '#5bd17a' : this._fps >= 30 ? '#f0b93a' : '#e8462b';
+    ctx.fillText(`${this._fps} fps`, f.canvas.width - 80, 22);
+    ctx.restore();
+  },
+});
+```
+
+Note: hooks receive `flow` and `ctx` as the **first arguments**. If you use a method on the plugin object (with `function () {}`), `this` is the plugin itself. With arrow functions, `this` is the surrounding scope. Choose accordingly.
+
+## Example: confirmation before destructive ops
+
+```js
+flow.use({
+  name: 'confirm-delete',
+  onNodeDelete: (f, nodeId) => {
+    const title = f.titles.get(nodeId);
+    if (title === 'production-db') {
+      const ok = confirm(`Really delete '${title}'?`);
+      if (!ok) return false;   // (currently not enforced — see note below)
+    }
+  },
+});
+```
+
+> **Note:** `onNodeDelete` doesn't yet support cancellation via `return false`. The hook fires after the deletion. Wrap `flow.deleteSelection()` in your own UI code if you need to block.
+
+## Example: registering many kinds at once
+
+```js
+flow.use({
+  name: 'database-kinds',
+  kinds: [
+    { name: 'db-query',  execute: async (ctx, ins) => {/*...*/} },
+    { name: 'db-insert', execute: async (ctx, ins) => {/*...*/} },
+    { name: 'db-update', execute: async (ctx, ins) => {/*...*/} },
+  ],
+});
+```
+
+The `kinds:` array is shorthand for calling `flow.registerKind()` for each entry during `init`.
+
+## Example: extending the API
+
+```js
+flow.use({
+  name: 'graph-stats',
+  extendAPI: (f) => {
+    f.stats = () => ({
+      nodes:    f.nodeCount(),
+      edges:    f.edgeCount(),
+      avgDegree: f.edgeCount() * 2 / Math.max(1, f.nodeCount()),
+    });
+  },
+});
+
+console.log(flow.stats());
+```
+
+Consumers now use `flow.stats()` as if it were built-in.
+
+## Example: command palette entries
+
+```js
+flow.use({
+  name: 'export-pdf',
+  commands: [
+    {
+      label: 'Export to PDF',
+      hotkey: 'Ctrl+Shift+P',
+      run: () => myPDFExport(flow.toJSON()),
+    },
+    {
+      label: 'Sync to backend',
+      run: async () => {
+        await fetch('/api/save', { method: 'POST', body: JSON.stringify(flow.toJSON()) });
+      },
+    },
+  ],
+});
+```
+
+These appear in `Ctrl+K`. The `hotkey` field is display-only — wire actual hotkeys yourself with `keydown` listeners if you want them globally bound.
+
+## Example: rejecting connections
+
+```js
+flow.use({
+  name: 'no-cross-region',
+  onConnect: (f, fromN, fp, toN, tp) => {
+    const fromRegion = f.tags.get(fromN)?.find(t => t.startsWith('region:'));
+    const toRegion   = f.tags.get(toN)?.find(t => t.startsWith('region:'));
+    if (fromRegion && toRegion && fromRegion !== toRegion) {
+      console.warn('Cross-region connections forbidden');
+      return false;
+    }
+  },
+});
+```
+
+Returning `false` causes the connection to be rejected. The user sees the "type mismatch" toast (we should add a way to customize the message — TODO).
+
+## Example: instrument executor calls (poor-man's APM)
+
+```js
+flow.use({
+  name: 'apm',
+  _starts: new Map(),
+  onBeforeExec(f, id) { this._starts.set(id, performance.now()); },
+  onAfterExec(f, id) {
+    const dur = performance.now() - this._starts.get(id);
+    if (dur > 100) console.warn(`slow node ${id}: ${dur.toFixed(1)}ms`);
+  },
+});
+```
+
+## Example: a complete logging plugin
+
+```js
+function loggingPlugin(opts = {}) {
+  const prefix = opts.prefix || '[flow]';
+  return {
+    name: 'logger',
+    onNodeAdd:    (f, id, spec) => console.log(prefix, 'add node', id, spec.kind),
+    onEdgeAdd:    (f, id, spec) => console.log(prefix, 'add edge', spec.from, '→', spec.to),
+    onBeforeExec: (f, id)        => console.log(prefix, '▸', id),
+    onAfterExec:  (f, id, out)   => console.log(prefix, '✓', id, out),
+    onNodeDelete: (f, id)        => console.log(prefix, '✗', id),
+  };
+}
+
+flow.use(loggingPlugin({ prefix: '[myapp]' }));
+```
+
+You can also pass a function — it's called with `flow` and should return a plugin object:
+
+```js
+flow.use((flow) => ({
+  name: 'auto-fit',
+  onChange: () => flow.fitView(),
+}));
+```
+
+## What plugins **cannot** do
+
+- They cannot prevent rendering (only modify it via `afterRender`)
+- They cannot replace the WASM core
+- They cannot intercept events between the WASM and JS layers — the hooks only fire at the JS-side public API boundary
+- They cannot stop another plugin from running (no ordering control yet)
+
+If you need any of these, you've reached the limits of plugins — fork the source.
+
+## Security note
+
+A plugin runs with **full access** to your page. It can read DOM, send fetches, modify global state. Treat plugins like npm packages: only install what you trust.
+
+If you build a plugin marketplace where users install third-party plugins, you'd need to sandbox each plugin in a Worker or iframe — out of scope for the core lib.
+
+## Next
+
+→ [Multiplayer](./06-multiplayer.md) — real-time co-editing with Yjs

package/docs/06-multiplayer.md

@@ -0,0 +1,180 @@
+# 6 · Multiplayer with Yjs
+
+@luispm/zflow-graph ships an opt-in [Yjs](https://github.com/yjs/yjs) adapter for real-time collaborative editing. Two browser tabs (or two users on different machines) editing the same graph see each other's changes within ~30ms.
+
+## How it works
+
+- Nodes live in a `Y.Map` keyed by stable UUIDs
+- Edges live in another `Y.Map`
+- Position updates are throttled to 30 Hz
+- Awareness (cursors, selection) uses Yjs' built-in awareness protocol
+- Conflicts resolve via Yjs' CRDT semantics — last write wins per field, no merge UI needed
+
+The adapter is **opt-in**. If you don't import it, Yjs isn't loaded and `flow.toJSON()` works as a standalone snapshot.
+
+## Quick start
+
+You'll need `yjs` and a provider. For the public demo server:
+
+```bash
+npm install yjs y-websocket
+```
+
+```js
+import * as Y from 'yjs';
+import { WebsocketProvider } from 'y-websocket';
+import { ZFlow } from '@luispm/zflow-graph';
+import { bindYjs } from '@luispm/zflow-graph/adapters/yjs';
+
+const flow = await ZFlow.create({ container, wasmUrl: '/zflow.wasm' });
+
+const ydoc = new Y.Doc();
+const provider = new WebsocketProvider('wss://demos.yjs.dev/ws', 'my-room', ydoc);
+
+bindYjs(flow, ydoc, {
+  userName: 'Alice',
+  color: '#c062e8',
+  awareness: provider.awareness,
+});
+```
+
+Now open the same URL in another tab. Add a node in tab A — it appears in tab B.
+
+## What syncs
+
+| Operation                  | Synced? |
+| -------------------------- | :-----: |
+| `flow.addNode(...)`        |   ✅    |
+| `flow.addEdge(...)`        |   ✅    |
+| `flow.moveNode(id, x, y)`  |   ✅ (throttled to 30 Hz)  |
+| Drag a node                |   ✅    |
+| `flow.setNodeTitle(...)`   |   ✅    |
+| `flow.setNodeColor(...)`   |   ✅    |
+| `flow.deleteSelection()`   |   ✅    |
+| Resize a node              |   ✅    |
+| Cursor movement            |   ✅ via awareness |
+| Selection                  |   ⚠️ adapter has the hook but doesn't broadcast yet |
+| Frames / notes             |   ❌ not wired through yet |
+| Runtime state (`flow._values`) | ❌ runtime is local-only by design |
+
+## Configuration
+
+```js
+const binding = bindYjs(flow, ydoc, {
+  // Required for awareness (cursors)
+  awareness: provider.awareness,
+
+  // Your identity
+  userId:   'alice@example.com',          // unique, stable
+  userName: 'Alice',                       // displayed in remote cursors
+  color:    '#c062e8',                     // displayed in remote cursors
+});
+
+// Returned handle:
+binding.ynodes      // Y.Map of nodes
+binding.yedges      // Y.Map of edges
+binding.ymeta       // Y.Map for arbitrary app metadata
+binding.destroy()   // unhooks, restores original flow methods
+```
+
+## Providers
+
+Yjs is provider-agnostic. Common choices:
+
+- **y-websocket** — your own relay server (the simplest)
+- **y-webrtc** — peer-to-peer, no server, works through a signaling channel
+- **y-indexeddb** — local persistence (combine with one of the above for offline-first)
+- **Liveblocks**, **Hocuspocus**, **Tiptap Hub** — commercial providers with auth + history
+- **Custom** — talk directly to Yjs sync protocol over any transport
+
+```js
+import { IndexeddbPersistence } from 'y-indexeddb';
+new IndexeddbPersistence('zflow-room', ydoc);   // offline-first
+```
+
+## Naming rooms
+
+By default everyone using the public demo server shares the room `'my-room'`. Pick something specific:
+
+```js
+const ROOM = `zflow:${projectId}:${graphId}`;
+const provider = new WebsocketProvider('wss://your-relay/ws', ROOM, ydoc);
+```
+
+Or use a hash in the URL so users can share links:
+
+```js
+const ROOM = location.hash.slice(1) || 'default';
+```
+
+## Hosting your own relay
+
+The simplest setup:
+
+```bash
+npm install -g @y/y-websocket-server
+PORT=1234 npx y-websocket-server
+```
+
+```js
+new WebsocketProvider('ws://your-server:1234', 'my-room', ydoc);
+```
+
+For production, put it behind nginx/Caddy with TLS and authentication. Yjs has no built-in auth — that's the provider's job.
+
+## Local persistence
+
+Combine multiple providers — they all sync the same `Y.Doc`:
+
+```js
+new IndexeddbPersistence('local-cache', ydoc);
+new WebsocketProvider('wss://...', 'room', ydoc);
+```
+
+Now your app works offline: writes go to IndexedDB immediately, then sync to the websocket when online.
+
+## Handling auth
+
+The adapter doesn't know about auth — that's between you and your provider. Pattern:
+
+```js
+const session = await getUser();
+const provider = new WebsocketProvider('wss://...', ROOM, ydoc, {
+  params: { token: session.jwt },
+});
+provider.awareness.setLocalStateField('user', {
+  id: session.id,
+  name: session.name,
+  color: session.color,
+});
+```
+
+Your relay server validates the JWT, refuses connection if invalid.
+
+## Conflict resolution
+
+Yjs is a CRDT — it merges all concurrent changes automatically, without ever asking. Specific behaviors:
+
+- Two users edit the same node title → last write wins (per field)
+- Two users move the same node → last position wins (you'll see it snap once)
+- Two users delete the same node → both deletes are idempotent
+- Two users add nodes simultaneously → both nodes appear (no conflict)
+- One user deletes a node while another adds an edge to it → the edge ends up dangling (Yjs doesn't enforce referential integrity)
+
+For graph integrity guarantees, you'd need to layer your own validation on top.
+
+## Limits and quirks
+
+- **No history replay yet.** The adapter syncs current state. To play back history, use Yjs' `UndoManager` or a provider with versioning (Liveblocks, Hocuspocus).
+- **Position throttling is 30 Hz** — if you drag a node very fast, peers see it jump. Tweak the throttle in `src/adapters/yjs.js` if you need higher rate.
+- **No selection sync.** Each user has their own selection. The remote cursor only shows their pointer position.
+- **Memory leak risk with very long sessions.** Yjs accumulates operations. Periodically snapshot (`Y.encodeStateAsUpdate`) and start fresh if your sessions span weeks.
+- **Frames and sticky notes don't sync yet.** TODO.
+
+## Example: complete multi-user editor
+
+See [examples/multiplayer.html](../examples/multiplayer.html) for a working demo. Open it in two tabs, add nodes, drag them — they sync.
+
+## Next
+
+→ [Recipes](./07-recipes.md) — concrete worked examples