ranuts 0.4.1-alpha.1 → 0.4.1-alpha.2

package/CLAUDE.md

@@ -13,7 +13,8 @@ and Node server tooling. Multi-entry, tree-shakeable, ships ESM + UMD.
 
 - **[docs/API.md](docs/API.md)** — ⭐ generated, authoritative list of **every exported
   symbol** with its signature + one-line description, grouped by entry point. Regenerate
-  with `npm run doc:api` after changing any public API. Read this to find what exists.
+  with `npm run doc:api` after changing any public API — CI fails if you forget. Read
+  this to find what exists.
 - **This file** — orientation: which entry to import, runtime constraints, conventions,
   and gotchas. Read this to decide _where_ things go and _how_ to add them.
 
@@ -27,15 +28,17 @@ from the **subpath that owns it** (below) — not from a deep source path.
 Each subpath is an independent, tree-shakeable barrel. Import from the subpath, never from
 `ranuts/dist/...` or `@/...` (that alias is internal to the source).
 
-| Import from     | Source                      | What                                                | Runtime            |
-| --------------- | --------------------------- | --------------------------------------------------- | ------------------ |
-| `ranuts`        | `index.ts`                  | Root barrel — re-exports the utils + visual surface | browser + node     |
-| `ranuts/utils`  | `src/utils/index.ts`        | DOM/BOM, string, object, number, color, time, etc.  | browser + node\*   |
-| `ranuts/node`   | `src/node/index.ts`         | HTTP server, router, ws, fs, streams, middleware    | **node only**      |
-| `ranuts/visual` | `src/utils/visual/index.ts` | 2D rendering engine (Canvas / WebGL / WebGPU)       | **browser only**   |
-| `ranuts/i18n`   | `src/utils/i18n.ts`         | `I18nCore` / `createI18n` / `useI18n` — DOM-free    | browser + node     |
-| `ranuts/sw`     | `src/sw/index.ts`           | Cache strategies + the precache protocol's SW half  | **service worker** |
-| `ranuts/vnode`  | `src/vnode/index.ts`        | Snabbdom-style virtual DOM (`h`, `init`, modules)   | browser            |
+| Import from           | Source                      | What                                                            | Runtime            |
+| --------------------- | --------------------------- | --------------------------------------------------------------- | ------------------ |
+| `ranuts`              | `index.ts`                  | Root barrel — re-exports the utils + visual surface             | browser + node     |
+| `ranuts/utils`        | `src/utils/index.ts`        | DOM/BOM, string, object, number, color, time, etc.              | browser + node\*   |
+| `ranuts/node`         | `src/node/index.ts`         | HTTP server, router, ws, fs, streams, middleware                | **node only**      |
+| `ranuts/visual`       | `src/utils/visual/index.ts` | 2D rendering engine (Canvas / WebGL / WebGPU)                   | **browser only**   |
+| `ranuts/i18n`         | `src/utils/i18n.ts`         | `I18nCore` / `createI18n` / `useI18n` — DOM-free                | browser + node     |
+| `ranuts/sw`           | `src/sw/index.ts`           | Cache strategies + the precache protocol's SW half              | **service worker** |
+| `ranuts/vnode`        | `src/vnode/index.ts`        | Snabbdom-style virtual DOM (`h`, `init`, modules)               | browser            |
+| `ranuts/stream`       | `src/stream/index.ts`       | SSE parsing + provider-neutral model-stream fold + token budget | browser + node     |
+| `ranuts/conversation` | `src/conversation/index.ts` | Event log → renderable conversation nodes                       | browser + node     |
 
 \* `ranuts/utils` is broad: most functions are browser-oriented (touch `window`/`document`),
 but pure helpers (`str`, `obj`, `number`, `compose`, `cloneDeep`, …) run anywhere. Functions
@@ -73,10 +76,14 @@ packages/ranuts/
 │   │   ├── canvas.ts         # Canvas 2D geometry — roundRectByArc, fanShapedByArc, getLinearGradient
 │   │   ├── placement.ts      # computePlacement — flip/shift floating-panel placement (re-exported by ranui/utils/placement)
 │   │   ├── tween.ts          # easing curves (quad/cubic/quart/quint/sine/expo/circ)
+│   │   ├── scroll.ts         # createBottomFollower — follow the floor without fighting the reader
+│   │   ├── diff.ts           # diffLines — line-level diff as unified-style hunks
 │   │   ├── visual/           # ranuts/visual — 2D rendering engine (see below)
 │   │   └── totp/             # TOTP + hand-rolled SHA
 │   ├── node/                 # ranuts/node — mini HTTP framework
-│   └── vnode/                # ranuts/vnode — virtual DOM
+│   ├── vnode/                # ranuts/vnode — virtual DOM
+│   ├── stream/               # ranuts/stream — SSE + StreamChunk + accumulator + budget
+│   └── conversation/         # ranuts/conversation — event log → nodes, with publication cadence
 ├── bin/
 │   ├── build.sh              # build (tsc types + vite es/umd)
 │   └── generate-api-docs.ts  # ⭐ doc:api — emits docs/API.md from source + JSDoc
@@ -126,6 +133,137 @@ in `utils/number.ts`; `srgbToLinear`/`luma`/`blend*`/`saturation`/`cosinePalette
 
 ---
 
+## Streaming a model response
+
+`ranuts/stream` is four layers, each usable alone. The split is the point: only the
+middle one is vendor-specific, and it is the one ranuts does **not** ship.
+
+1. `parseEventStream(source)` — bytes → `ServerSentEvent`. Transport only, no model
+   concepts. Handles the parts that bite: a chunk boundary anywhere (including inside a
+   multi-byte character or between `\r` and `\n`), repeated `data:` joined with `\n`, one
+   space stripped after the colon, `:` comment keep-alives, a leading BOM, and a trailing
+   block the server never terminated.
+2. `StreamChunk` — the provider-neutral vocabulary. **You write the mapping** from your
+   provider's event shape onto it; `mapEventStream(source, map)` is the seam, and returning
+   `[]` from the mapping is how a keep-alive or a `[DONE]` sentinel is dropped.
+3. `createStreamAccumulator()` — folds chunks into blocks, so a view reads `snapshot()`,
+   `text()`, `reasoning()`, or `toolCalls()` instead of concatenating deltas itself.
+4. `estimateTokens` / `addUsage` / `planCompaction` — the budget. A chat client that skips
+   this works for a week and then stops: every turn carries the whole history, the request
+   grows monotonically, and one day the provider refuses it as a wall.
+
+Non-obvious rules the vocabulary encodes:
+
+- **Group by `index`, never by arrival order.** Reasoning and text interleave, and several
+  tool calls open at once.
+- **Tool arguments stay raw JSON text.** Half a JSON document is not a value. The
+  accumulator never parses `argumentsDelta`; parse once, after `finish`.
+- **`block-start` is optional.** Several providers open a block with its first delta, so
+  the accumulator opens one on demand. Do not require it in your mapping either.
+- **`block-end` wins.** When a provider sends an assembled block, it replaces whatever the
+  deltas accumulated.
+- **`finish` terminates.** `usage` arrives before it; nothing after it.
+
+Non-obvious rules the budget encodes:
+
+- **`estimateTokens` is an estimate, and CJK is not four characters a token.** A BPE
+  vocabulary trained mostly on Latin script rarely merges CJK, so counting it at the Latin
+  rate underestimates a Chinese conversation about fourfold — the difference between
+  compacting in time and compacting after the provider refuses. A real tokenizer is a
+  per-model dependency measured in megabytes, and this decides _when to act_, not billing.
+- **`addUsage` keeps a field one side omitted.** Providers differ in what they send, and a
+  running total that lost `outputTokens` the moment one response omitted it is worse than
+  no total.
+- **`planCompaction` counts the summary it will add.** Without that it returns "fits" for a
+  request that does not. It also never touches the protected tail, and reports `fits: false`
+  rather than compacting it anyway — the caller decides, because nothing can be done there
+  that does not lose the user's own words.
+- **It takes sizes, not messages.** What replaces a folded prefix, and which cut points a
+  wire format allows, are the caller's business.
+
+## Projecting an event log into a conversation
+
+`ranuts/conversation` turns an append-only log into the nodes a view renders. The
+alternative — a view that switches on event type and mutates a component tree — puts
+ordering, identity and partial-update reconciliation in the view, where every new kind of
+content has to be threaded through by hand.
+
+Here each kind of content is an independently registered state machine:
+
+```ts
+const engine = createConversationEngine<Event>({ definitions: [message, toolCall] });
+engine.subscribe((nodes) => render(nodes));
+engine.push(event);
+```
+
+A definition says which events are its own (`match`), folds them into its own state
+(`start` / `update`), and never learns that the others exist. Adding a kind is adding a
+definition, not editing a renderer.
+
+Semantics you need before writing one:
+
+- **Every definition sees every event.** Claims are independent; the engine does not stop
+  at the first match, so one log event can drive two nodes.
+- **Order is fixed at `start`.** A node that keeps updating stays where it opened, so a
+  streaming message does not jump to the end of the list on every delta.
+- **An `update` for an id with no open node is dropped.** That is honest when the start
+  event was trimmed from a paged window; inventing a node from a partial update would
+  render something that never existed.
+- **`publication` is the streaming throttle.** `animation-frame` coalesces every delta
+  between two paints into one notification; `immediate` is for discrete facts, where
+  waiting a frame only adds latency; `none` records without waking the view. Cadence
+  escalates and never relaxes — an `immediate` while a frame is pending fires now and
+  cancels the frame.
+- **`reader.previous(kind)` is backward-only**, so replaying the same log reproduces the
+  same view. A definition that could see nodes started after it would not.
+- **`batch(run)` for any replay.** Without it, restoring a stored conversation is
+  quadratic: every event publishes, every publication hands subscribers the whole node
+  list, and a DOM subscriber walks all of it. Measured on a 600-message transcript that was
+  5.4 seconds of blocked main thread; batched it is 158ms. A live stream does not need it —
+  one delta changes one row.
+- **A subscriber gets the keys that changed.** On a long transcript nearly every row is
+  unchanged on every frame, and rewriting all of them once per delta is the difference
+  between a transcript that streams and one that stalls. `undefined` means every node
+  changed, which is what a reset reports.
+- **`truncate(key)` is what editing, regenerating and branching are made of.** All three
+  mean "the conversation diverges here", and what follows the divergence is no longer part
+  of it. It cuts by `seq` rather than by position — a node that opened before the cut
+  survives even if its latest update came after it, which is the same rule node order
+  follows — and returns how many nodes went, because zero is the caller's cue that its own
+  idea of the conversation is stale. Without it the only way back is `reset()` and a full
+  replay, which discards every node that did not change and makes the view flash.
+
+`scheduler` is injectable, which is how the cadence tests run without a paint. The default
+uses `requestAnimationFrame` in a browser and a microtask elsewhere.
+
+`<r-conversation>` in ranui is the DOM consumer of all of this — see
+[ranui/CLAUDE.md](../ranui/CLAUDE.md#r-conversation).
+
+## Following the bottom of a scroller
+
+`createBottomFollower` (from `ranuts/utils`) keeps an append-only view — a streaming
+transcript, a log tail, a terminal — pinned to its floor without fighting the reader.
+
+The hard part is telling your own scroll writes apart from the reader's. It uses an
+**observed-top ledger**: every programmatic write records the `scrollTop` it produced, and
+a scroll event whose position deviates from the ledger is the reader. That covers wheel,
+touch, scrollbar, keyboard and stray `scrollIntoView` calls at once, without listening for
+any input device — no list of device listeners is ever complete.
+
+Using it:
+
+- Call `follow()` whenever content changes; it is a no-op while the reader is scrolled up.
+- Pass `observe: [column]` for growth that appends no node — streaming text grows an
+  existing node, so it has to be observed rather than announced.
+- Before loading older content, `captureAnchor(row)` on a row that is actually on screen,
+  and `restoreAnchor()` after the prepend lands. Resolving to `null` means the row did not
+  survive; omitting the resolver reuses the captured element.
+- A shrink-clamp never transfers ownership. A reader who had scrolled up stays unpinned
+  even when the clamp leaves them at the floor, and re-pins on their next scroll —
+  re-pinning on a clamp would take control back with no input from them.
+- Scrolling is always instant. Smooth scrolling animates toward a moving target during
+  streaming, and its intermediate positions read as reader input on the next event.
+
 ## Conventions
 
 ### Adding a function to an existing module
@@ -214,6 +352,7 @@ npm run tsc          # type-check only (tsc --noEmit)
 npm run test         # vitest run (node env)
 npm run test:coverage
 npm run doc:api      # regenerate docs/API.md from source + JSDoc
+npm run doc:api:check  # verify it is fresh without writing (what CI runs)
 ```
 
 - **Tests**: Vitest, **node environment** (no jsdom) — `vitest.config.ts` has no `environment`
@@ -222,6 +361,11 @@ npm run doc:api      # regenerate docs/API.md from source + JSDoc
   `getRgb`'s CSS fallback returns black without `document`).
 - **`onConsoleLog` throws**: the test config fails any test that logs to console — don't leave
   stray `console.*` in code under test.
+- **Never assert on unseeded randomness**: sampling the real CSPRNG can only estimate the property
+  under test, and a window tight enough to catch a real bias also fails on chance every few
+  thousand runs — which reads as a mystery CI failure on one matrix leg. Replace the sample with
+  scripted bytes and assert the mapping exactly; `test/utils/secure.test.ts` has the `withBytes`
+  helper that stubs `crypto.getRandomValues` for the length of one draw.
 
 ---
 
@@ -241,6 +385,7 @@ npm run doc:api      # regenerate docs/API.md from source + JSDoc
 | Branching on `isClient` in new code                                              | It's a module-load-time constant — wrong after SSR, unstubbable in tests. Check `typeof window === 'undefined'` inside the function.                    |
 | Reaching for `randomString` / `getRandomString` to make a token or code          | Both are `Math.random()` based and predictable. Use `secureRandomString` / `secureToken` from `secure.ts` for anything an attacker would want to guess. |
 | Comparing a secret with `===`                                                    | Returns early at the first differing byte, leaking the shared prefix through timing. Use `safeEqual`.                                                   |
+| Asserting a distribution drawn from the real CSPRNG                              | The window that catches the bias also fires on sampling noise. Script the bytes with `withBytes` (`test/utils/secure.test.ts`) and assert an equality.  |
 | Decoding chunked text per chunk                                                  | A multi-byte character straddling a boundary becomes replacement characters. `concatBytes` first, decode once.                                          |
 | `window.setTimeout` in a `ranuts/utils` module                                   | Throws outside a document. Use the bare `setTimeout`.                                                                                                   |
 | A module-level factory that keeps mutable state shared by everything it produces | Give each produced function its own state (this is why `generateThrottle` was removed).                                                                 |

package/README.md

@@ -61,6 +61,8 @@ Import as required. You can select:
 - `ranuts/visual` — 2D rendering engine (Canvas / WebGL / WebGPU, **browser only**)
 - `ranuts/sw` — cache strategies + the precache protocol's service-worker half (**service worker only**)
 - `ranuts/vnode` — Snabbdom-style virtual DOM
+- `ranuts/stream` — Server-Sent Events parsing, a provider-neutral fold of a streamed model response, and the token budget that decides when a history stops fitting
+- `ranuts/conversation` — project an append-only event log into renderable conversation nodes
 - `ranuts/i18n` — the i18n engine on its own, without the rest of `utils`
 
 ```js

package/README.zh-CN.md

@@ -60,6 +60,8 @@ npm install ranuts@latest --save
 - `ranuts/visual` —— 2D 渲染引擎（Canvas / WebGL / WebGPU，**仅浏览器**）
 - `ranuts/sw` —— 缓存策略 + 预缓存协议的 Service Worker 一侧（**仅 Service Worker**）
 - `ranuts/vnode` —— Snabbdom 风格的虚拟 DOM
+- `ranuts/stream` —— Server-Sent Events 解析、对流式模型响应的厂商中立折叠，以及判断历史何时装不下的 token 预算
+- `ranuts/conversation` —— 将只追加的事件日志投影为可渲染的对话节点
 - `ranuts/i18n` —— 单独的 i18n 引擎，不牵入 `utils` 的其余部分
 
 ```js