brookmd 0.27.0 → 0.29.0

package/CHANGELOG.md

@@ -4,6 +4,102 @@ Notable changes to brookmd (formerly `flux-md`). Format based on
 [Keep a Changelog](https://keepachangelog.com/); this project aims to follow
 [Semantic Versioning](https://semver.org/).
 
+## 0.29.0 — 2026-07-31
+
+**Sublinear resources.** Total parse work has an Ω(n) floor and stays exactly
+there — but retained memory and reopen latency don't have to be O(n), and now
+they aren't. Requires `brookmd-core` 0.26.0. Both features are **web-path**
+(WASM + TS client); the native bindings (React Native / Kotlin / Swift /
+Flutter) share every parser-core improvement but consume `allBlocks()` and so
+keep full retention — see the platform matrix in the root README.
+
+### Added
+
+- **Instant thread reopen — `getPersistable()` / `hydrate()` /
+  `sourceFingerprint()`.** A finished (or in-flight) stream can be persisted
+  as a small JSON envelope — the blocks `getSnapshot()` showed plus a source
+  length/hash and a `done` flag — and a fresh `BrookClient` hydrates from it
+  with **no worker, no WASM, no parsing**: 1 MB / 2,250 blocks re-streams in
+  ~110 ms, hydrates in **~5.5 ms** (and a real browser reopen also skips
+  worker spin-up and WASM init). To *continue* a non-finalized thread, pass
+  the original source to `beginResume()`: the UI keeps showing hydrated
+  blocks while a background re-parse catches up — reusing the existing
+  divergence/adoption machinery, so every hydrated block keeps its exact id
+  across the swap and nothing visible remounts; appends stream live once
+  caught up. The envelope is a versioned **package** format
+  (`hydrateVersion`), deliberately not the wire contract; the README's
+  "Instant thread reopen" section documents the persistence and invalidation
+  contract.
+- **`retainCommittedHtml` — the parser no longer hoards the rendered
+  document.** The core kept every committed block's HTML alive for
+  `allBlocks()` — which the web worker path never calls. The worker now
+  defaults the flag **off**: retained memory beyond the source buffer drops
+  to the open tail only (measured: **53–70% less total retention** on 1 MB
+  streams, with the flag-off overhead above the raw source staying under
+  300 bytes at every point). Patches are byte-identical either way — committed
+  blocks are emitted exactly once and never re-read. Rust-side default stays
+  ON, so native bindings and `allBlocks()` consumers are unaffected;
+  `renderToString` pins it on (it assembles from `allBlocks`). Set
+  `retainCommittedHtml: true` in `ParserConfig` if you drive the worker and
+  want `allBlocks` fidelity.
+
+## 0.28.0 — 2026-07-31
+
+**Browser-side linearity.** The parser and wire have been O(new bytes) per
+append for a while; this release makes the *DOM application* match. Before, an
+open (streaming) block was fully rebuilt on every animation frame — a 20 KB
+highlighted code block wrote ~44 million characters of `innerHTML` over its
+lifetime (≈2,150× the wire bytes), and a per-frame `replaceWith` destroyed any
+text selection or `<pre>` scroll position inside the block. Every open-block
+path now applies patches incrementally, measured at **2–3.4× the block's final
+markup, flat across sizes** (1× is the write-it-once floor), and enforced by a
+chars-written regression gate so browser-side linearity is CI-pinned like the
+parser's scaling shapes.
+
+### Performance
+
+- **Open blocks apply the wire's splice instead of rebuilding.** The delta
+  signal (`keep_units`) the wire already computed was being discarded at the
+  client; it now reaches both renderers, which rebuild only the element the
+  splice lands in (fast path fires ~94% of syncs; any ambiguity falls back to
+  a full rebuild, so correctness never depends on the fast path).
+- **Open code blocks reuse the streaming highlighter's frozen prefix**: the
+  frozen markup is appended once and never rewritten; only the bounded
+  speculative tail repaints. 20 KB highlighted stream: ~44 MB written → ~370 KB
+  (**~120× less**).
+- **Keyed list/container sync in the DOM renderer** (`blockData` on): settled
+  `<li>`s / nested container children are never re-rendered — new items append,
+  only the open last item repaints; a tight→loose flip resyncs once, keyed by
+  `(index, html)` so it cannot be silently missed. Streamed 20 KB list: 284× →
+  **2.9×**; blockquote: 318× → **3.4×**.
+- **React's keyed container path now engages by default.** It was accidentally
+  gated behind a `components` map (git history shows the gate was incidental);
+  a default-config streaming blockquote/alert now renders keyed — 22.6× and
+  growing → **3.4× flat**. React and DOM implementations were cross-checked to
+  byte-identical work counts.
+- Known documented fallback: a streamed table with `blockData: false` still
+  takes the full-rebuild path (the splice refuses table tag chains — foster
+  parenting). `blockData: true` tables were already incremental (1.7×). A
+  table-scoped splice was measured (15% improvement) and rejected as not worth
+  the parser-divergence risk.
+
+### Fixed
+
+- **Text selection and `<pre>` scroll survive streaming.** Selecting text in
+  the already-settled part of a streaming block no longer collapses on the
+  next token; a code block's horizontal scroll position is preserved. Both are
+  pinned by tests with negative controls.
+
+### Changed
+
+- The DOM inside an open, streaming `<code>` element is now two spans (frozen +
+  speculative tail) while streaming; it settles to the same single-markup form
+  as before when the block closes. CSS that targets `code > *` structurally
+  may observe the difference mid-stream only. Settled output is unchanged.
+- React's default open blockquote/alert omits inter-block whitespace text
+  nodes mid-stream (matching the DOM renderer's long-standing keyed behavior);
+  layout is unaffected and settle output is byte-identical.
+
 ## 0.27.0 — 2026-07-31
 
 ### Added

package/README.md

@@ -586,6 +586,11 @@ class BrookClient {
   whenReady(): Promise<void>;                       // resolves once WASM loaded; rejects on init failure
   subscribe(listener: () => void): () => void;      // React-friendly store
   getSnapshot(): Block[];                           // ordered current blocks
+  getPersistable(source?: string): PersistableSnapshot;     // capture the rendered doc as JSON
+  hydrate(                                                  // restore it: no worker, no parse
+    snapshot: PersistableSnapshot,
+    opts?: { source?: string },                             // source ⇒ a live thread can resume
+  ): void;
   outline(): { level: number; text: string; id: number }[]; // heading table-of-contents (works mid-stream)
   toPlaintext(): string;                            // rendered document as plain text (search / summaries)
   getMetrics(): { bytes, patches, totalParseMs, throughputKBs,
@@ -1284,6 +1289,101 @@ WASM `BrookParser`, native bindings, C ABI) keep byte-identical v1 wire by
 default and can opt in with `setWireDelta(true)` — see
 [`WIRE.md` §11](https://github.com/siinghd/brookmd/blob/main/crates/brookmd-core/WIRE.md).
 
+## Instant thread reopen — persist and hydrate
+
+Reopening a long thread normally means re-feeding its whole source through the
+parser before the first paint: O(history) work between the click and the pixels,
+which is exactly when a chat UI feels frozen.
+
+It is also unnecessary. A committed block is emitted **once** and is **final**,
+so the blocks a stream already produced are a complete description of the
+document — there is nothing to re-derive. Capture them, store them, restore them:
+
+```ts
+// When the thread closes (or at any checkpoint):
+await db.put(threadId, JSON.stringify(client.getPersistable()));
+
+// When it reopens:
+const client = new BrookClient();
+client.hydrate(JSON.parse(await db.get(threadId))); // no worker, no WASM, no parse
+// <BrookMarkdown client={client} /> paints the whole thread on the first frame.
+```
+
+Measured on a 1 MB / ~2250-block thread: **~5 ms to hydrate vs ~110 ms to
+re-stream** it in 2 KB chunks (~51 ms even for a single-shot re-parse) — before
+counting the worker spin-up and WASM init a re-stream also pays. Hydration is
+O(blocks) of JSON handling; nothing parses.
+
+Hydrated blocks are **ordinary committed blocks**: both renderers mount them
+through the same path as live ones, so a reopened thread renders byte-identical
+to the one the user closed.
+
+### What to store, and when to throw it away
+
+```ts
+interface PersistableSnapshot {
+  hydrateVersion: number; // envelope version; hydrate() refuses what it can't read
+  blocks: Block[];        // exactly what getSnapshot() showed
+  sourceLength: number;   // UTF-16 length of the markdown behind them
+  sourceHash: string;     // fingerprint of that markdown — a staleness check
+  done: boolean;          // had the stream been finalized?
+}
+```
+
+Store it as JSON beside the thread. **Invalidate it when your source moves on**:
+hydrate deliberately does *not* verify the blocks against the source, so a stale
+snapshot would paint stale HTML. That is what `sourceHash` is for:
+
+```ts
+import { sourceFingerprint } from "brookmd";
+
+if (snapshot.sourceHash === sourceFingerprint(mySource)) {
+  client.hydrate(snapshot);                        // instant
+} else {
+  client.setContent(mySource, { done: true });     // the text changed — re-stream
+}
+```
+
+Discard a snapshot whose `hydrateVersion` your build doesn't read, too. Either
+way a failed hydrate is safe to fall back from: a malformed or unreadable
+snapshot is rejected **whole**, with nothing written until every block has
+validated.
+
+> This envelope is a brookmd **package** format, *not* the parser wire contract
+> ([`WIRE.md`](https://github.com/siinghd/brookmd/blob/main/crates/brookmd-core/WIRE.md)).
+> The two are versioned independently.
+
+### Resuming a thread that was still streaming
+
+A `done: false` snapshot was captured mid-stream. Pass the original markdown as
+`source` and the thread simply continues:
+
+```ts
+client.hydrate(snapshot, { source: markdownSoFar });
+client.append(nextDelta); // or client.pipeFrom(await fetch("/api/chat"))
+```
+
+The parser's internal state cannot be serialized (it is an `Rc` graph, not
+data), so continuing a document genuinely requires re-parsing what came before
+it — but that cost moves **off the critical path**. On the first `append`
+brookmd spins up the worker, re-feeds the source in the background, and keeps
+the hydrated blocks on screen throughout: the reader scrolls and reads while the
+parse catches up. New chunks need no buffering — they queue behind the re-feed
+and land the moment it completes (they arrive at model token speed; the parser
+catches up far faster). When the re-parse lands, unchanged blocks are adopted
+**by reference and by id**, so nothing re-renders and nothing remounts at the
+swap.
+
+A `done: true` snapshot is terminal: it needs no `source`, no worker is ever
+created for it, and `append()` throws — a finished thread cannot be extended
+(`reset()` to start a new one). A `done: false` snapshot hydrated *without*
+`source` is view-only: it paints, but appending throws, because a document
+cannot be continued correctly without the text preceding it.
+
+Nothing on the hydrate path touches a worker: `whenReady()` resolves immediately
+and `client.ready` is `true`, so a readiness gate never sits over a thread that
+is already on screen.
+
 ## Security
 
 brookmd is XSS-safe by default — its HTML output is meant to be injected via

package/dist/client.d.ts

@@ -26,6 +26,50 @@ export interface OutlineEntry {
     /** Stable block id — usable as a scroll target / React key. */
     id: number;
 }
+/**
+ * A self-contained, JSON-serializable capture of everything a stream currently
+ * renders — produced by {@link BrookClient.getPersistable}, restored by
+ * {@link BrookClient.hydrate}.
+ *
+ * Reopening a long thread normally re-feeds its whole source through the parser
+ * before the first paint (O(source), and it blocks the reopen). Persist this
+ * next to the thread instead and restoring it is pure JSON handling: the blocks
+ * go straight into the store and both renderers mount them as ordinary
+ * committed blocks — **no worker, no WASM, no parse**.
+ *
+ * NOT the wire contract. The parser↔renderer wire (WIRE.md) is a separate,
+ * separately-versioned boundary; this envelope is a brookmd *package* format
+ * that only {@link BrookClient.hydrate} consumes.
+ */
+export interface PersistableSnapshot {
+    /** Envelope version. {@link BrookClient.hydrate} refuses anything it does not
+     *  know how to read, so a stored snapshot never half-restores. */
+    hydrateVersion: number;
+    /**
+     * The blocks exactly as {@link BrookClient.getSnapshot} showed them at
+     * capture: the committed history plus whatever tail was on screen. Plain
+     * JSON — a `Block` carries no hidden state (the renderers' splice bookkeeping
+     * lives in a WeakMap, never on the block), so `JSON.stringify` round-trips it.
+     */
+    blocks: Block[];
+    /** UTF-16 length of the markdown that produced `blocks`. */
+    sourceLength: number;
+    /** {@link sourceFingerprint} of that markdown. A staleness check — "is the
+     *  source I stored still the one these blocks came from?" — and deliberately
+     *  nothing more: not a checksum, not security. */
+    sourceHash: string;
+    /** Whether the captured stream had been finalized. A `done` snapshot is a
+     *  terminal document: it needs no source and can never be resumed. */
+    done: boolean;
+}
+/**
+ * 32-bit FNV-1a over a string's UTF-16 code units, as 8 hex digits. Backs
+ * {@link PersistableSnapshot.sourceHash}: cheap enough to run over a megabyte
+ * of markdown, sharp enough to notice that a stored source changed. Exported so
+ * a caller can make the same staleness decision brookmd does —
+ * `snap.sourceHash === sourceFingerprint(mySource)`. Not a security primitive.
+ */
+export declare function sourceFingerprint(source: string): string;
 export declare function applyPatch(store: BlockStore, patch: Patch): void;
 interface PoolWorker {
     worker: WorkerLike;
@@ -171,6 +215,7 @@ export declare class BrookClient {
     private recoveryBuffer;
     private recoveredLen;
     private recoveryAttempted;
+    private appendedAny;
     private coalesce;
     private rafHandle;
     private finalizePending;
@@ -182,6 +227,12 @@ export declare class BrookClient {
     /** Set by mergeStale when it had to compact a hole out of the view, so
      *  getSnapshot skips caching a view whose indices no longer track `base`. */
     private mergeDropped;
+    private hydrated;
+    private hydratedDone;
+    private hydratedLength;
+    private hydratedHash;
+    private hydratedSource;
+    private resumed;
     private appendedBytes;
     private patchCount;
     private totalParseMicros;
@@ -261,6 +312,10 @@ export declare class BrookClient {
      */
     private settleRebind;
     get ready(): boolean;
+    /** True while a hydrated document has never been handed to a parser: the
+     *  store holds restored blocks and no worker exists. Cleared once the resume
+     *  re-feed is issued (or the caller re-drives the whole document itself). */
+    private get hydrationPending();
     /**
      * The fatal error that killed this client's worker, or `null` if healthy.
      *
@@ -329,6 +384,90 @@ export declare class BrookClient {
     setContent(content: string, opts?: {
         done?: boolean;
     }): void;
+    /**
+     * Capture everything this client currently renders as a plain-JSON
+     * {@link PersistableSnapshot}: `JSON.stringify` it, store it beside the
+     * thread, and {@link hydrate} it back later to repaint with **no parse at
+     * all**. This is the persistence half of instant thread reopen.
+     *
+     * The committed wire is already a complete serialization — a committed block
+     * is emitted exactly once and is final (WIRE.md §2) — so there is nothing to
+     * re-derive: the snapshot IS the document. Cost here is one pass to
+     * fingerprint the source; cost on the way back in is a `Map` fill.
+     *
+     * @param source the markdown driven into this stream, used ONLY to compute
+     *   {@link PersistableSnapshot.sourceHash}. Optional, because a client with
+     *   `recovery` on (the default) already retains it, as does a
+     *   `setContent`-driven one; required in the single configuration that holds
+     *   neither — `recovery: false` plus manual `append()` — where omitting it
+     *   throws rather than persisting a snapshot no one can check for staleness.
+     */
+    getPersistable(source?: string): PersistableSnapshot;
+    /** The full driven document when this client happens to hold it: the recovery
+     *  buffer is exactly that whenever `recovery` is on (the default), and
+     *  setContent's baseline covers the recovery-off controlled-string mode.
+     *  `null` means genuinely unknown — reachable only with recovery off AND
+     *  manual appends. */
+    private retainedSource;
+    /**
+     * Restore a {@link PersistableSnapshot} into an untouched client. The blocks
+     * land in the store as ordinary committed blocks and `getSnapshot()` returns
+     * them immediately, so the first paint already has the whole document —
+     * **no worker is created, no WASM loads, nothing is parsed**. Reopening a
+     * thread costs O(blocks) of JSON handling instead of O(source) of parsing.
+     *
+     * Call it on a fresh client before anything is appended and, ideally, before
+     * the renderer mounts (hydrating an already-mounted client works — it
+     * notifies subscribers — but costs an extra render). Hydrating a client that
+     * already holds content throws.
+     *
+     * **Resuming a live thread.** A snapshot with `done: false` was still
+     * streaming. Pass `source` — the markdown behind the snapshot — and the first
+     * {@link append} rebuilds parser state in the background: the parser's
+     * internals are an `Rc` graph with no serialized form, so continuing a
+     * document genuinely requires re-parsing what came before it, but that cost
+     * moves OFF the critical path. The hydrated blocks stay on screen and the
+     * reader scrolls them while the worker catches up; new chunks queue behind the
+     * re-feed and land the moment it does. Without `source` the thread is
+     * view-only and appending throws — continuing a document correctly is not
+     * possible without the text that precedes it.
+     *
+     * **Hydration does not verify the blocks against the source.** The snapshot is
+     * trusted as produced; `sourceHash` exists so the CALLER can notice its stored
+     * source moved on and re-stream instead of painting stale HTML. The resume
+     * path re-checks it, lets the fresh parse win, and warns in dev.
+     *
+     * @throws if the envelope version is unknown, the snapshot is malformed, or
+     *   this client already holds content. Validation completes before anything is
+     *   written, so a rejected snapshot leaves the client exactly as it was.
+     */
+    hydrate(snapshot: PersistableSnapshot, opts?: {
+        source?: string;
+    }): void;
+    /**
+     * The first content-bearing op on a hydrated thread: give the parser back the
+     * state it could not be handed.
+     *
+     * There is no way around re-parsing — the core keeps `Rc` graphs with no
+     * `Deserialize` — but every part of that cost sits off the critical path. The
+     * hydrated blocks are already painted and STAY painted: `softReset` preserves
+     * them exactly as the setContent divergence swap does, so the reader keeps
+     * reading and scrolling while the worker chews through the history on its own
+     * thread. The caller's new chunks need no buffer of ours — `postMessage` is
+     * FIFO per worker, so they queue behind the re-feed and are parsed the instant
+     * it catches up. When the re-parse's patch lands, `mergeStale` adopts every
+     * unchanged block BY REFERENCE (same object, same id), so the swap re-renders
+     * and remounts nothing, and the live tail streams on from there.
+     */
+    private beginResume;
+    /** The resume re-feed is no longer wanted: either it has just been issued, or
+     *  the caller took over by driving the whole document itself. Releases the
+     *  retained source so one document, not two, stays in memory. */
+    private retireHydration;
+    /** Drop every trace of a hydrate. An explicit reset() is a brand-new
+     *  document, so a restored view must not keep intercepting appends or re-feed
+     *  a source that is no longer the one being streamed. */
+    private clearHydration;
     reset(): void;
     /**
      * setContent's divergence reset: rebuild the parser exactly like {@link reset},