brookmd 0.26.0 → 0.27.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,90 @@ Notable changes to brookmd (formerly `flux-md`). Format based on
4
4
  [Keep a Changelog](https://keepachangelog.com/); this project aims to follow
5
5
  [Semantic Versioning](https://semver.org/).
6
6
 
7
+ ## 0.27.0 — 2026-07-31
8
+
9
+ ### Added
10
+
11
+ - **Streaming syntax highlighting — open code blocks now highlight live, on by
12
+ default.** Since 0.19 the built-in highlighter deliberately waited for a
13
+ fence to close; a streaming block showed plain text. Open blocks now render
14
+ highlighted as they grow, and the mechanism preserves every guarantee the
15
+ deferred design existed to protect:
16
+
17
+ - **Byte-identical at settle.** All committed markup comes from seeded runs
18
+ of the same resumable tokenizer that `highlight()` uses — the frozen
19
+ prefix is only ever extended at *checkpoints* (positions provably outside
20
+ any future token: after a newline in whitespace, ≥ 3 bytes behind the
21
+ stream head, with no unterminated string/comment/template live; HTML
22
+ checkpoints after `>` instead). When the block closes, one seeded run
23
+ finishes the tail, so the final bytes equal a one-shot `highlight()` —
24
+ pinned by a ~44,000-case fuzz (all languages, chunk sizes down to 1 byte,
25
+ plus speculative-revision streams) asserting settle-identity and that
26
+ every intermediate frozen prefix is a byte-prefix of the final output.
27
+ - **Bounded work, independent of block size.** Between checkpoints, an
28
+ unterminated string/comment advances via an O(1)-state scanner over only
29
+ the new bytes; the tail re-scan is capped at 8 KB (past the cap the tail
30
+ renders plain until the next checkpoint — correctness unaffected). Total
31
+ tokenization work measures ≈ 5.7× the streamed bytes end-to-end against
32
+ the real parser, flat from 288 B to 23 KB blocks (naive re-highlighting
33
+ measures 812× more at 23 KB); a work-bound test enforces < 6×. The
34
+ 50 000-char plain-escape guard still applies and discards streaming state.
35
+ - **The tail is speculative.** The few tokens nearest the stream head may
36
+ change color as bytes arrive (an unterminated `"` reads as plain until its
37
+ closer lands) — the same speculative-tail behavior brookmd's links and
38
+ emphasis already have. Frozen output never changes.
39
+
40
+ Opt out with `streamingHighlight={false}` on `BrookMarkdown` (React) or
41
+ `streamingHighlight: false` in `mountBrookMarkdown` options (DOM, and via it
42
+ the Web Component and Vue/Svelte/Solid adapters) — that restores 0.26.1's
43
+ plain-until-close exactly. Not a `ParserConfig` field: highlighting is a
44
+ renderer concern; the parser, wire, and WASM are untouched. SSR is
45
+ unchanged (closed blocks, synchronous), and `components.CodeBlock` /
46
+ `pre` / `code` overrides still bypass the built-in highlighter entirely.
47
+ Closing a block that streamed in now also highlights near-instantly: the
48
+ close-time pass reuses the accumulated prefix instead of starting over.
49
+
50
+ ## 0.26.1 — 2026-07-30
51
+
52
+ Two fixes found by benchmarking 0.26.0 against real chat traffic. Requires
53
+ `brookmd-core` 0.25.1.
54
+
55
+ ### Fixed
56
+
57
+ - **A GFM table could not interrupt an open paragraph — anywhere.**
58
+ `scan_paragraph` had no table arm, so `item\n| a | b |\n|---|---|` swallowed
59
+ the delimiter row as paragraph text. The visible symptom was "tables inside
60
+ list items don't parse" (an item's de-indented body opens with a paragraph),
61
+ but the bug was position-independent and hit top level and blockquotes the
62
+ same way. The fix is an O(2-line) gate — header row with a pipe, delimiter
63
+ row next, matching cell counts, both rows indented ≤ 3 columns — checked once
64
+ per line as it arrives, on both the full-reparse and streaming paths, so a
65
+ scan started at a commit boundary renders byte-identically to a cold one.
66
+ Verified against GitHub's rendering for every non-pathological shape; the
67
+ deliberate exception (both rows must be ≤ 3-indented, so two exotic
68
+ mixed-indent shapes stay paragraphs) is pinned in `tests/nested_tables.rs`.
69
+ A nested table renders inside the item's `html` and — like nested lists —
70
+ carries no structured `blockData` of its own.
71
+
72
+ ### Performance
73
+
74
+ - **Close-time syntax highlighting no longer blocks the main thread.** The
75
+ built-in highlighter ran as one synchronous task when a block closed —
76
+ ~110 ms for a large fence on a mid desktop. The tokenizer loop was already
77
+ resumable at any offset with no carried state, so it now runs in ~5 ms
78
+ slices (`scheduler.yield()` where available, `MessageChannel` otherwise),
79
+ with the first slice synchronous so small blocks render highlighted in the
80
+ same tick with no flash. Output is byte-identical — pinned by a chunked ==
81
+ one-shot property test across all 20 languages at chunk sizes down to 1,
82
+ plus a 6,556-case differential fuzz against the previous implementation.
83
+ Also: the renderers now reuse the parser's already-decoded source
84
+ (`CodeBlockData.code`) when `blockData` is on instead of re-deriving it from
85
+ the HTML, and `escapeHtml` no longer concatenates per character. A 49 KB
86
+ block's longest main-thread task drops from ~38 ms to ≤ 6 ms on the same
87
+ hardware; `highlight()`'s public signature and bytes are unchanged, SSR
88
+ stays synchronous, and `components.CodeBlock`/`pre`/`code` overrides are
89
+ unaffected.
90
+
7
91
  ## 0.26.0 — 2026-07-30
8
92
 
9
93
  **Rendered HTML bytes change in this release.** Everything new below is opt-in
package/README.md CHANGED
@@ -4,7 +4,7 @@ Zero-dep streaming markdown for the browser. Rust→WASM core, one Web Worker pe
4
4
 
5
5
  Drop in a streaming-aware renderer — **React, Vue, Svelte, Solid, a framework-agnostic `<brook-markdown>` Web Component, or the vanilla DOM mount** — wire each LLM stream to a `BrookClient`, and the markdown renders incrementally off the main thread, block by block, with stable identities so unchanged blocks never re-reconcile.
6
6
 
7
- Parsing runs entirely **off the main thread** — each stream gets its own pooled Web Worker, so many concurrent LLM responses render without contending for the UI thread. On each token the parser re-parses only the **active tail**, not the whole document; patches cross the worker boundary as **verified splices** (not full re-sends, so emitted bytes stay O(n) even for one giant growing block); and heavy renderers (syntax highlighting, math, mermaid) are **deferred until a block closes**. The result is low retained memory and a main thread that stays responsive while streaming. See [the live demo](https://md.hsingh.app/).
7
+ Parsing runs entirely **off the main thread** — each stream gets its own pooled Web Worker, so many concurrent LLM responses render without contending for the UI thread. On each token the parser re-parses only the **active tail**, not the whole document; patches cross the worker boundary as **verified splices** (not full re-sends, so emitted bytes stay O(n) even for one giant growing block); and heavy renderers (math, mermaid) are **deferred until a block closes** — code fences highlight as they stream, re-tokenizing only the last line rather than the whole block per chunk. The result is low retained memory and a main thread that stays responsive while streaming. See [the live demo](https://md.hsingh.app/).
8
8
 
9
9
  > **Beyond the browser:** the same Rust core also powers experimental React
10
10
  > Native, Swift (iOS/macOS), Kotlin/Android, Flutter, and C-ABI bindings —
@@ -248,7 +248,9 @@ controlled-string helpers wrap; in vanilla you call it directly.
248
248
 
249
249
  `mountBrookMarkdown(client, container, options?)` returns `{ destroy(), refresh() }`.
250
250
  Options: `components`, `sanitize`, `virtualize`, `stickToBottom`, `highlightCode`
251
- (default true), `batch` (default true — one DOM write per `requestAnimationFrame`),
251
+ (default true), `streamingHighlight` (default true — highlight a code fence while
252
+ it is still streaming; see [Streaming syntax highlighting](#streaming-syntax-highlighting)),
253
+ `batch` (default true — one DOM write per `requestAnimationFrame`),
252
254
  `morphOpenBlocks` (default false — morph a growing generic open block's subtree in
253
255
  place instead of rebuilding it via `innerHTML`, so only the changed parts repaint
254
256
  and focus/selection in the streaming tail survive; the rendered result is
@@ -1167,6 +1169,34 @@ import { highlight } from "brookmd/highlight";
1167
1169
  const html = highlight("const x = 1;", "ts");
1168
1170
  ```
1169
1171
 
1172
+ ### Streaming syntax highlighting
1173
+
1174
+ A code fence is highlighted **while it streams**, not only once it closes. On by
1175
+ default; turn it off with `streamingHighlight={false}` (React) or
1176
+ `{ streamingHighlight: false }` (the DOM mount options / Vue / Svelte / Solid),
1177
+ which restores the plain-until-close behaviour.
1178
+
1179
+ It stays O(n) over the whole block rather than re-highlighting the fence on every
1180
+ chunk. An open block keeps a **frozen prefix** — markup for everything behind a
1181
+ checkpoint, which later bytes provably cannot rewrite — and re-tokenizes only the
1182
+ **tail** after it, about one source line's worth per patch. When the fence closes,
1183
+ the tokenizer resumes from that checkpoint instead of starting over, so a block
1184
+ that streamed in highlights near-instantly.
1185
+
1186
+ Two things worth knowing:
1187
+
1188
+ - **The tail is speculative.** A prefix of source does not tokenize like the same
1189
+ prefix of a longer source: `const s = "hello` is a stray quote plus an
1190
+ identifier until its closing quote lands, and `123.456e` is `123` plus loose
1191
+ fragments until a digit arrives. So the last line's colours can shift as bytes
1192
+ come in. Nothing behind the checkpoint ever changes.
1193
+ - **The settled markup is byte-identical** to `highlight(text, lang)` either way.
1194
+ Turning this on or off changes when colour appears, never what it is.
1195
+
1196
+ Blocks past the highlighter's 50 000-character guard, unknown languages, and any
1197
+ fence taken over by a `components.CodeBlock` / `pre` / `code` override are
1198
+ unaffected — they behave exactly as before.
1199
+
1170
1200
  ## Coverage
1171
1201
 
1172
1202
  **CommonMark 0.31: 100% (652/652 spec examples), byte-exact** — every section,
@@ -1218,8 +1248,6 @@ By design, not yet, or only partially:
1218
1248
  (`<span>`/`<div class="math …">` with `gfmMath` on) and a `Mermaid` slot, but
1219
1249
  stays zero-dep: bring your own KaTeX / mermaid pass (or a `components.MathBlock`
1220
1250
  / `components.Mermaid` override) for the actual SVG/MathML output.
1221
- - **Syntax highlighting on open code blocks** — deferred until close. This is a
1222
- deliberate perf choice.
1223
1251
 
1224
1252
  ## Performance
1225
1253
 
package/dist/dom.d.ts CHANGED
@@ -64,6 +64,17 @@ export interface MountOptions {
64
64
  /** Use the built-in code highlighter. Default true; suppressed when a
65
65
  * `components.CodeBlock` override is supplied. */
66
66
  highlightCode?: boolean;
67
+ /**
68
+ * Highlight a code fence **while it is still streaming**, instead of showing
69
+ * plain escaped text until it closes. On by default (parity with the React
70
+ * renderer's `streamingHighlight` prop).
71
+ *
72
+ * An open block keeps a frozen prefix and re-tokenizes only its tail on each
73
+ * patch, so this stays linear in the block's size. The settled markup is
74
+ * byte-identical either way — only the tail's colours are provisional, and
75
+ * they may shift as bytes arrive. Set `false` for the pre-0.27 behaviour.
76
+ */
77
+ streamingHighlight?: boolean;
67
78
  /** Coalesce patches into one DOM write per animation frame. Default true. */
68
79
  batch?: boolean;
69
80
  /**
package/dist/dom.js CHANGED
@@ -1,4 +1,5 @@
1
- import { highlight } from "./hi.js";
1
+ import { highlightDeferred } from "./hi-defer.js";
2
+ import { createInc, incHighlight, incSeed } from "./hi-inc.js";
2
3
  import { morph } from "./morph.js";
3
4
  import { blockProps, extractLang } from "./block-props.js";
4
5
  import { decorateSegments } from "./decorate.js";
@@ -28,6 +29,7 @@ function mountBrookMarkdown(client, container, options = {}) {
28
29
  const hasInlineTransforms = !!decorators || !!urlTransform;
29
30
  const hasPerf = typeof performance !== "undefined";
30
31
  const highlightCode = options.highlightCode !== false && !components?.CodeBlock;
32
+ const streamingHighlight = options.streamingHighlight !== false;
31
33
  const batch = options.batch !== false && typeof requestAnimationFrame === "function";
32
34
  const morphOpenBlocks = options.morphOpenBlocks === true;
33
35
  const root = document.createElement("div");
@@ -115,6 +117,11 @@ function mountBrookMarkdown(client, container, options = {}) {
115
117
  continue;
116
118
  }
117
119
  existing.table = void 0;
120
+ if (existing.inc && b.kind.type !== "CodeBlock") existing.inc = void 0;
121
+ if (existing.highlight) {
122
+ existing.highlight.cancel();
123
+ existing.highlight = void 0;
124
+ }
118
125
  const node = renderBlock(b, existing);
119
126
  existing.node.replaceWith(node);
120
127
  existing.node = node;
@@ -128,6 +135,8 @@ function mountBrookMarkdown(client, container, options = {}) {
128
135
  if (mounted.size > seen.size) {
129
136
  for (const [id, mb] of mounted) {
130
137
  if (!seen.has(id)) {
138
+ if (mb.highlight) mb.highlight.cancel();
139
+ mb.inc = void 0;
131
140
  mb.node.remove();
132
141
  mounted.delete(id);
133
142
  }
@@ -184,7 +193,7 @@ function mountBrookMarkdown(client, container, options = {}) {
184
193
  }
185
194
  switch (kind) {
186
195
  case "CodeBlock":
187
- if (highlightCode) return renderCodeBlock(b);
196
+ if (highlightCode) return renderCodeBlock(b, mb);
188
197
  break;
189
198
  // fall through to the generic path
190
199
  case "MathBlock":
@@ -341,10 +350,22 @@ function mountBrookMarkdown(client, container, options = {}) {
341
350
  }
342
351
  return result;
343
352
  }
344
- function renderCodeBlock(b) {
353
+ function renderCodeBlock(b, mb) {
345
354
  const lang = extractLang(b.html) || "text";
346
- const text = b.open ? "" : decodeCodeText(b.html);
347
- const highlighted = text ? highlight(text, lang) : null;
355
+ const text = b.open ? "" : codeText(b);
356
+ let openMarkup = null;
357
+ if (b.open && streamingHighlight) {
358
+ let inc = mb.inc;
359
+ if (inc === void 0 || inc.lang !== lang.toLowerCase()) {
360
+ inc = createInc(lang) ?? void 0;
361
+ mb.inc = inc;
362
+ }
363
+ if (inc !== void 0) openMarkup = incHighlight(inc, codeText(b));
364
+ }
365
+ const seed = !b.open && mb.inc !== void 0 ? incSeed(mb.inc, text, lang) : void 0;
366
+ const run = text ? highlightDeferred(text, lang, seed) : null;
367
+ if (!b.open) mb.inc = void 0;
368
+ const highlighted = openMarkup ?? (run ? run.html : null);
348
369
  const block = document.createElement("div");
349
370
  block.className = "brook-code-block" + (b.open ? " brook-streaming" : "");
350
371
  const header = document.createElement("div");
@@ -364,15 +385,8 @@ function mountBrookMarkdown(client, container, options = {}) {
364
385
  block.appendChild(header);
365
386
  const body = document.createElement("div");
366
387
  body.className = "brook-code-body";
367
- if (highlighted) {
368
- const pre = document.createElement("pre");
369
- pre.tabIndex = 0;
370
- pre.setAttribute("role", "region");
371
- pre.setAttribute("aria-label", `${lang} code`);
372
- const code = document.createElement("code");
373
- code.innerHTML = highlighted;
374
- pre.appendChild(code);
375
- body.appendChild(pre);
388
+ if (highlighted !== null) {
389
+ body.appendChild(highlightedPre(lang, highlighted));
376
390
  } else {
377
391
  const div = document.createElement("div");
378
392
  div.tabIndex = 0;
@@ -380,10 +394,29 @@ function mountBrookMarkdown(client, container, options = {}) {
380
394
  div.setAttribute("aria-label", `${lang} code`);
381
395
  div.innerHTML = b.html;
382
396
  body.appendChild(div);
397
+ if (run !== null && run.rest !== null) {
398
+ mb.highlight = run;
399
+ run.rest.then((markup) => {
400
+ if (markup === null || dead) return;
401
+ if (mb.highlight !== run || mb.node !== block) return;
402
+ mb.highlight = void 0;
403
+ body.replaceChild(highlightedPre(lang, markup), div);
404
+ });
405
+ }
383
406
  }
384
407
  block.appendChild(body);
385
408
  return block;
386
409
  }
410
+ function highlightedPre(lang, markup) {
411
+ const pre = document.createElement("pre");
412
+ pre.tabIndex = 0;
413
+ pre.setAttribute("role", "region");
414
+ pre.setAttribute("aria-label", `${lang} code`);
415
+ const code = document.createElement("code");
416
+ code.innerHTML = markup;
417
+ pre.appendChild(code);
418
+ return pre;
419
+ }
387
420
  function renderMathBlock(b) {
388
421
  const block = document.createElement("div");
389
422
  block.className = "brook-math-block" + (b.open ? " brook-streaming" : "");
@@ -476,6 +509,13 @@ function mountBrookMarkdown(client, container, options = {}) {
476
509
  frame = 0;
477
510
  }
478
511
  unsubscribe();
512
+ for (const mb of mounted.values()) {
513
+ if (mb.highlight) {
514
+ mb.highlight.cancel();
515
+ mb.highlight = void 0;
516
+ }
517
+ mb.inc = void 0;
518
+ }
479
519
  root.remove();
480
520
  },
481
521
  refresh() {
@@ -558,6 +598,10 @@ function tailOpenBlockId(snapshot) {
558
598
  const tail = snapshot.length > 0 ? snapshot[snapshot.length - 1] : void 0;
559
599
  return tail && tail.open ? tail.id : null;
560
600
  }
601
+ function codeText(b) {
602
+ const data = b.kind.data;
603
+ return typeof data?.code === "string" ? data.code : decodeCodeText(b.html);
604
+ }
561
605
  function decodeCodeText(html) {
562
606
  const m = html.match(/<pre><code[^>]*>([\s\S]*?)<\/code><\/pre>/);
563
607
  if (!m) return "";
@@ -0,0 +1,36 @@
1
+ import { type HighlightState } from "./hi.js";
2
+ /**
3
+ * Test-only: shrink the per-slice budget so a suite can force the deferred path
4
+ * deterministically instead of betting on how fast the machine is. Call with no
5
+ * argument to restore the default. Not part of the public API.
6
+ */
7
+ export declare function __setSliceMs(ms?: number): void;
8
+ export interface DeferredHighlight {
9
+ /**
10
+ * The finished markup when the whole block tokenized inside the first
11
+ * (synchronous) slice — the common case. Apply it in this same tick: no
12
+ * swap, no second paint, no flash.
13
+ */
14
+ html: string | null;
15
+ /**
16
+ * Resolves with the finished markup once the remaining slices have run, or
17
+ * with `null` if the run was {@link DeferredHighlight.cancel}led. `null` (the
18
+ * property, not the resolution) when `html` already holds the answer. Never
19
+ * rejects.
20
+ */
21
+ rest: Promise<string | null> | null;
22
+ /** Abandon the remaining slices — the block was superseded or unmounted. */
23
+ cancel(): void;
24
+ }
25
+ /**
26
+ * Tokenize `code` for at most one slice and return the finished markup, or
27
+ * `null` when it did not fit. Pure and synchronous — it schedules nothing, so a
28
+ * renderer can call it from a render pass and only reach for
29
+ * {@link highlightDeferred} when this comes back empty.
30
+ */
31
+ export declare function highlightWithin(code: string, lang: string, seed?: HighlightState): string | null;
32
+ /**
33
+ * Highlight `code` without blocking: the first slice runs here, synchronously,
34
+ * and the rest (if any) continues on later tasks. See {@link DeferredHighlight}.
35
+ */
36
+ export declare function highlightDeferred(code: string, lang: string, seed?: HighlightState): DeferredHighlight;
@@ -0,0 +1,93 @@
1
+ import { stepHighlight } from "./hi.js";
2
+ const SLICE_MS = 5;
3
+ const CHUNK = 1024;
4
+ const now = () => typeof performance !== "undefined" ? performance.now() : Date.now();
5
+ let sliceMs = SLICE_MS;
6
+ function __setSliceMs(ms) {
7
+ sliceMs = ms === void 0 ? SLICE_MS : ms;
8
+ }
9
+ const noop = () => {
10
+ };
11
+ function seeded(seed) {
12
+ return seed === void 0 ? { pos: 0, out: "" } : { pos: seed.pos, out: seed.out };
13
+ }
14
+ function highlightWithin(code, lang, seed) {
15
+ const state = seeded(seed);
16
+ return runSlice(code, lang, state) ? state.out : null;
17
+ }
18
+ function highlightDeferred(code, lang, seed) {
19
+ const state = seeded(seed);
20
+ if (runSlice(code, lang, state)) {
21
+ return { html: state.out, rest: null, cancel: noop };
22
+ }
23
+ let cancelled = false;
24
+ let channel = null;
25
+ let pending = null;
26
+ function closeChannel() {
27
+ if (channel !== null) {
28
+ channel.port1.close();
29
+ channel.port2.close();
30
+ channel = null;
31
+ }
32
+ pending = null;
33
+ }
34
+ function post() {
35
+ return new Promise((resolve) => {
36
+ if (typeof MessageChannel !== "function") {
37
+ setTimeout(resolve, 0);
38
+ return;
39
+ }
40
+ if (channel === null) {
41
+ channel = new MessageChannel();
42
+ channel.port1.onmessage = () => {
43
+ const next = pending;
44
+ pending = null;
45
+ if (next) next();
46
+ };
47
+ }
48
+ pending = resolve;
49
+ channel.port2.postMessage(0);
50
+ });
51
+ }
52
+ function nextTask() {
53
+ const scheduler = globalThis.scheduler;
54
+ if (scheduler && typeof scheduler.yield === "function") {
55
+ try {
56
+ const p = scheduler.yield();
57
+ if (p && typeof p.then === "function") return p.then(void 0, post);
58
+ } catch {
59
+ }
60
+ }
61
+ return post();
62
+ }
63
+ const rest = (async () => {
64
+ try {
65
+ for (; ; ) {
66
+ await nextTask();
67
+ if (cancelled) return null;
68
+ if (runSlice(code, lang, state)) return state.out;
69
+ }
70
+ } finally {
71
+ closeChannel();
72
+ }
73
+ })();
74
+ return {
75
+ html: null,
76
+ rest,
77
+ cancel() {
78
+ cancelled = true;
79
+ }
80
+ };
81
+ }
82
+ function runSlice(code, lang, state) {
83
+ const started = now();
84
+ for (; ; ) {
85
+ if (stepHighlight(code, lang, state, CHUNK)) return true;
86
+ if (now() - started >= sliceMs) return false;
87
+ }
88
+ }
89
+ export {
90
+ __setSliceMs,
91
+ highlightDeferred,
92
+ highlightWithin
93
+ };
@@ -0,0 +1,102 @@
1
+ import { type HighlightState } from "./hi.js";
2
+ /**
3
+ * How an unbounded opener recognizes its terminator, with the O(1) state each
4
+ * one carries between patches:
5
+ *
6
+ * - `char` — the first occurrence of `ch` closes (`[^ch]*ch` forms: Go raw
7
+ * strings, HTML/CSS attribute strings, Rust `#[…]`, Bash `'…'` and `${…}`).
8
+ * - `starSlash` — `prevStar`, for `/* … *\/`.
9
+ * - `dashGt` — `dashRun`, for `<!-- … -->`.
10
+ * - `esc` — `esc`, for backslash-escaped strings and template literals. A
11
+ * backslash immediately before a newline is {@link DEAD}, not an escape: the
12
+ * pattern's `\\.` does not match a newline, so the form can never close.
13
+ * - `triple` — `runLen`, for Python `"""` / `'''`.
14
+ * - `dbl` — `pendingQuote`, for SQL's doubled-quote escaping (`'a''b'`).
15
+ */
16
+ type PredKind = {
17
+ k: "char";
18
+ ch: string;
19
+ } | {
20
+ k: "starSlash";
21
+ } | {
22
+ k: "dashGt";
23
+ } | {
24
+ k: "esc";
25
+ ch: string;
26
+ } | {
27
+ k: "triple";
28
+ ch: string;
29
+ } | {
30
+ k: "dbl";
31
+ ch: string;
32
+ };
33
+ /** @internal */
34
+ export interface Opener {
35
+ /** Sticky regex for the OPENING delimiter only — bounded, O(1) to test. */
36
+ re: RegExp;
37
+ /** The pattern class the tokenizer emits when this form IS terminated. */
38
+ cls: string;
39
+ /** The literal terminator. */
40
+ end: string;
41
+ pred: PredKind;
42
+ }
43
+ /** @internal The carried state of one opener's continuation predicate. */
44
+ export interface OpenerScanState {
45
+ n: number;
46
+ f: boolean;
47
+ }
48
+ /** Per-block incremental state. Create with {@link createInc}, feed {@link incHighlight}. */
49
+ export interface IncState {
50
+ /** The language key this state's tables were chosen for. */
51
+ readonly lang: string;
52
+ /** Settled through here: `[0, c)` of the source will never re-tokenize. */
53
+ c: number;
54
+ /** Markup for `[0, c)`. Always a byte-prefix of the block's final markup. */
55
+ frozenHtml: string;
56
+ /** The checkpoint BEFORE `c`, and the `frozenHtml` length that went with it —
57
+ * the one step of rewind a tail revision needs (see {@link adopt}). */
58
+ c0: number;
59
+ frozenLen0: number;
60
+ /** The unbounded opener live at the tail, or `null`. */
61
+ opener: Opener | null;
62
+ /** `opener`'s carried predicate state. */
63
+ scan: OpenerScanState;
64
+ /** `opener` can never close — stop re-scanning and render the tail plain. */
65
+ sealed: boolean;
66
+ /** The source last fed in, for the append/revision guard. */
67
+ text: string;
68
+ /** Escaped source for `[c, plainUpto)` — the plain tail, extended in place. */
69
+ plain: string;
70
+ plainFrom: number;
71
+ plainUpto: number;
72
+ /** The markup last handed out, so a repeated feed of the same text is free. */
73
+ html: string | null;
74
+ }
75
+ /**
76
+ * State for a block in `lang`, or `null` when the language has no table (the
77
+ * plain-escape fallback has no token boundaries to checkpoint on).
78
+ */
79
+ export declare function createInc(lang: string): IncState | null;
80
+ /**
81
+ * Feed the block's CURRENT full source and get the markup for all of it, or
82
+ * `null` when the incremental path has bowed out (past {@link CLIFF}) and the
83
+ * caller should render the plain escaped body exactly as it does today.
84
+ *
85
+ * Calls must be append-only. Anything else — a `reset()`, a speculative tail
86
+ * revision, one block's id being reused for different content — is detected
87
+ * (`text.startsWith(prev)`, the same guard the DOM renderer's prefix-append fast
88
+ * path uses) and simply restarts the state from scratch.
89
+ */
90
+ export declare function incHighlight(st: IncState, text: string): string | null;
91
+ /**
92
+ * The `{pos, out}` a close-time `highlightWithin`/`highlightDeferred` run should
93
+ * resume from, or `undefined` when nothing was frozen or the state does not
94
+ * belong to `text`/`lang` (a revised block, a different language, a block that
95
+ * crossed the cliff). Then the close-time run is the unseeded one it always was.
96
+ */
97
+ export declare function incSeed(st: IncState, text: string, lang: string): HighlightState | undefined;
98
+ /** @internal Test-only: total source bytes re-tokenized since the last reset. */
99
+ export declare function __getIncScanned(): number;
100
+ /** @internal Test-only. */
101
+ export declare function __resetIncScanned(): void;
102
+ export {};
package/dist/hi-inc.js ADDED
@@ -0,0 +1,317 @@
1
+ import { escapeHtml, stepHighlight } from "./hi.js";
2
+ const GAP = 3;
3
+ const CAP = 8192;
4
+ const CLIFF = 5e4;
5
+ const OPEN = 0;
6
+ const CLOSED = 1;
7
+ const DEAD = 2;
8
+ const BLOCK_COMMENT = { re: /\/\*/y, cls: "com", end: "*/", pred: { k: "starSlash" } };
9
+ const JS_OPENERS = [
10
+ BLOCK_COMMENT,
11
+ { re: /`/y, cls: "str", end: "`", pred: { k: "esc", ch: "`" } }
12
+ ];
13
+ const RUST_OPENERS = [
14
+ BLOCK_COMMENT,
15
+ { re: /b?"/y, cls: "str", end: '"', pred: { k: "esc", ch: '"' } },
16
+ { re: /#!?\[/y, cls: "attr", end: "]", pred: { k: "char", ch: "]" } }
17
+ ];
18
+ const PY_OPENERS = [
19
+ { re: /[fFrRbB]{0,2}"""/y, cls: "str", end: '"""', pred: { k: "triple", ch: '"' } },
20
+ { re: /[fFrRbB]{0,2}'''/y, cls: "str", end: "'''", pred: { k: "triple", ch: "'" } }
21
+ ];
22
+ const GO_OPENERS = [
23
+ BLOCK_COMMENT,
24
+ { re: /`/y, cls: "str", end: "`", pred: { k: "char", ch: "`" } }
25
+ ];
26
+ const BASH_OPENERS = [
27
+ { re: /"/y, cls: "str", end: '"', pred: { k: "esc", ch: '"' } },
28
+ { re: /'/y, cls: "str", end: "'", pred: { k: "char", ch: "'" } },
29
+ { re: /\$\{/y, cls: "var", end: "}", pred: { k: "char", ch: "}" } }
30
+ ];
31
+ const SQL_OPENERS = [
32
+ BLOCK_COMMENT,
33
+ { re: /'/y, cls: "str", end: "'", pred: { k: "dbl", ch: "'" } },
34
+ { re: /"/y, cls: "str", end: '"', pred: { k: "dbl", ch: '"' } }
35
+ ];
36
+ const HTML_OPENERS = [
37
+ { re: /<!--/y, cls: "com", end: "-->", pred: { k: "dashGt" } },
38
+ { re: /"/y, cls: "str", end: '"', pred: { k: "char", ch: '"' } },
39
+ { re: /'/y, cls: "str", end: "'", pred: { k: "char", ch: "'" } }
40
+ ];
41
+ const CSS_OPENERS = [
42
+ BLOCK_COMMENT,
43
+ { re: /"/y, cls: "str", end: '"', pred: { k: "char", ch: '"' } },
44
+ { re: /'/y, cls: "str", end: "'", pred: { k: "char", ch: "'" } }
45
+ ];
46
+ const OPENERS = {
47
+ js: JS_OPENERS,
48
+ javascript: JS_OPENERS,
49
+ ts: JS_OPENERS,
50
+ tsx: JS_OPENERS,
51
+ jsx: JS_OPENERS,
52
+ typescript: JS_OPENERS,
53
+ rust: RUST_OPENERS,
54
+ rs: RUST_OPENERS,
55
+ py: PY_OPENERS,
56
+ python: PY_OPENERS,
57
+ go: GO_OPENERS,
58
+ bash: BASH_OPENERS,
59
+ sh: BASH_OPENERS,
60
+ shell: BASH_OPENERS,
61
+ json: [],
62
+ sql: SQL_OPENERS,
63
+ html: HTML_OPENERS,
64
+ xml: HTML_OPENERS,
65
+ css: CSS_OPENERS
66
+ };
67
+ const GT_CHECKPOINT = /* @__PURE__ */ new Set(["html", "xml"]);
68
+ function createInc(lang) {
69
+ const key = lang.toLowerCase();
70
+ if (!Object.prototype.hasOwnProperty.call(OPENERS, key)) return null;
71
+ return {
72
+ lang: key,
73
+ c: 0,
74
+ frozenHtml: "",
75
+ c0: 0,
76
+ frozenLen0: 0,
77
+ opener: null,
78
+ scan: { n: 0, f: false },
79
+ sealed: false,
80
+ text: "",
81
+ plain: "",
82
+ plainFrom: 0,
83
+ plainUpto: 0,
84
+ html: null
85
+ };
86
+ }
87
+ function divergence(a, b) {
88
+ const n = a.length < b.length ? a.length : b.length;
89
+ let i = 0;
90
+ while (i < n && a.charCodeAt(i) === b.charCodeAt(i)) i++;
91
+ return i;
92
+ }
93
+ function adopt(st, d) {
94
+ if (st.c > 0 && d >= st.c + GAP) {
95
+ dropTail(st);
96
+ return true;
97
+ }
98
+ if (st.c0 > 0 && d >= st.c0 + GAP) {
99
+ st.frozenHtml = st.frozenHtml.slice(0, st.frozenLen0);
100
+ st.c = st.c0;
101
+ st.c0 = 0;
102
+ st.frozenLen0 = 0;
103
+ dropTail(st);
104
+ return true;
105
+ }
106
+ return false;
107
+ }
108
+ function dropTail(st) {
109
+ st.opener = null;
110
+ st.scan = { n: 0, f: false };
111
+ st.sealed = false;
112
+ st.plain = "";
113
+ st.plainFrom = st.c;
114
+ st.plainUpto = st.c;
115
+ st.html = null;
116
+ }
117
+ function reset(st) {
118
+ st.c = 0;
119
+ st.frozenHtml = "";
120
+ st.c0 = 0;
121
+ st.frozenLen0 = 0;
122
+ st.opener = null;
123
+ st.scan = { n: 0, f: false };
124
+ st.sealed = false;
125
+ st.text = "";
126
+ st.plain = "";
127
+ st.plainFrom = 0;
128
+ st.plainUpto = 0;
129
+ st.html = null;
130
+ }
131
+ function incHighlight(st, text) {
132
+ if (st.text === text) return st.html;
133
+ const d = divergence(st.text, text);
134
+ const appended = d === st.text.length && text.length > st.text.length;
135
+ let from = st.text.length;
136
+ if (!appended) {
137
+ if (!adopt(st, d)) reset(st);
138
+ from = 0;
139
+ }
140
+ st.text = text;
141
+ if (text.length > CLIFF) {
142
+ reset(st);
143
+ st.text = text;
144
+ return null;
145
+ }
146
+ const live = st.opener;
147
+ if (live !== null) {
148
+ const r = st.sealed ? OPEN : feed(live.pred, st.scan, text, from, text.length);
149
+ if (r === DEAD) st.sealed = true;
150
+ if (r !== CLOSED) {
151
+ st.html = st.frozenHtml + plainTail(st, text);
152
+ return st.html;
153
+ }
154
+ st.opener = null;
155
+ }
156
+ if (text.length - st.c > CAP) {
157
+ st.html = st.frozenHtml + plainTail(st, text);
158
+ return st.html;
159
+ }
160
+ st.html = rescan(st, text);
161
+ return st.html;
162
+ }
163
+ function incSeed(st, text, lang) {
164
+ if (st.c === 0 || st.lang !== lang.toLowerCase()) return void 0;
165
+ if (text.length > CLIFF || text.length < st.c) return void 0;
166
+ if (!text.startsWith(st.text.slice(0, st.c))) return void 0;
167
+ return { pos: st.c, out: st.frozenHtml };
168
+ }
169
+ let scanned = 0;
170
+ function __getIncScanned() {
171
+ return scanned;
172
+ }
173
+ function __resetIncScanned() {
174
+ scanned = 0;
175
+ }
176
+ function plainTail(st, text) {
177
+ if (st.plainFrom !== st.c || st.plainUpto > text.length) {
178
+ st.plainFrom = st.c;
179
+ st.plain = "";
180
+ st.plainUpto = st.c;
181
+ }
182
+ if (st.plainUpto < text.length) {
183
+ st.plain += escapeHtml(text.slice(st.plainUpto));
184
+ st.plainUpto = text.length;
185
+ }
186
+ return st.plain;
187
+ }
188
+ function rescan(st, text) {
189
+ const openers = OPENERS[st.lang];
190
+ const gt = GT_CHECKPOINT.has(st.lang);
191
+ const limit = text.length - GAP;
192
+ let cp = -1;
193
+ let cpOut = 0;
194
+ let liveAt = -1;
195
+ let liveOp = null;
196
+ let liveLen = 0;
197
+ const sink = (cls, start, end, outLen) => {
198
+ if (liveAt >= 0) return;
199
+ for (let i = 0; i < openers.length; i++) {
200
+ const op = openers[i];
201
+ op.re.lastIndex = start;
202
+ const m = op.re.exec(text);
203
+ if (m === null || m.index !== start) continue;
204
+ const terminated = cls === op.cls && end - start >= m[0].length + op.end.length && text.startsWith(op.end, end - op.end.length);
205
+ if (terminated) continue;
206
+ liveAt = start;
207
+ liveOp = op;
208
+ liveLen = m[0].length;
209
+ return;
210
+ }
211
+ if (gt) {
212
+ if (cls === "pun" && end - start === 1 && text.charCodeAt(start) === 62 && end <= limit) {
213
+ cp = end;
214
+ cpOut = outLen;
215
+ }
216
+ return;
217
+ }
218
+ if (cls !== "ws") return;
219
+ const hi = end < limit ? end : limit;
220
+ if (hi <= start) return;
221
+ const nl = text.lastIndexOf("\n", hi - 1);
222
+ if (nl < start) return;
223
+ cp = nl + 1;
224
+ cpOut = outLen - (end - cp);
225
+ };
226
+ const state = { pos: st.c, out: "" };
227
+ scanned += text.length - st.c;
228
+ while (!stepHighlight(text, st.lang, state, text.length, sink)) {
229
+ }
230
+ const full = st.frozenHtml + state.out;
231
+ if (cp > st.c) {
232
+ st.c0 = st.c;
233
+ st.frozenLen0 = st.frozenHtml.length;
234
+ st.frozenHtml += state.out.slice(0, cpOut);
235
+ st.c = cp;
236
+ }
237
+ const found = liveOp;
238
+ if (found === null) {
239
+ st.opener = null;
240
+ } else {
241
+ st.scan = { n: 0, f: false };
242
+ const r = feed(found.pred, st.scan, text, liveAt + liveLen, text.length);
243
+ st.opener = r === CLOSED ? null : found;
244
+ st.sealed = r === DEAD;
245
+ }
246
+ return full;
247
+ }
248
+ function feed(p, st, s, from, to) {
249
+ switch (p.k) {
250
+ case "char": {
251
+ const i = s.indexOf(p.ch, from);
252
+ return i >= 0 && i < to ? CLOSED : OPEN;
253
+ }
254
+ case "starSlash": {
255
+ for (let i = from; i < to; i++) {
256
+ const ch = s[i];
257
+ if (st.f && ch === "/") return CLOSED;
258
+ st.f = ch === "*";
259
+ }
260
+ return OPEN;
261
+ }
262
+ case "dashGt": {
263
+ for (let i = from; i < to; i++) {
264
+ const ch = s[i];
265
+ if (ch === ">" && st.n >= 2) return CLOSED;
266
+ st.n = ch === "-" ? st.n + 1 : 0;
267
+ }
268
+ return OPEN;
269
+ }
270
+ case "esc": {
271
+ for (let i = from; i < to; i++) {
272
+ const ch = s[i];
273
+ if (st.f) {
274
+ st.f = false;
275
+ if (ch === "\n") return DEAD;
276
+ continue;
277
+ }
278
+ if (ch === "\\") {
279
+ st.f = true;
280
+ continue;
281
+ }
282
+ if (ch === p.ch) return CLOSED;
283
+ }
284
+ return OPEN;
285
+ }
286
+ case "triple": {
287
+ for (let i = from; i < to; i++) {
288
+ if (s[i] === p.ch) {
289
+ st.n++;
290
+ if (st.n === 3) return CLOSED;
291
+ } else {
292
+ st.n = 0;
293
+ }
294
+ }
295
+ return OPEN;
296
+ }
297
+ case "dbl": {
298
+ for (let i = from; i < to; i++) {
299
+ const ch = s[i];
300
+ if (st.f) {
301
+ st.f = false;
302
+ if (ch !== p.ch) return CLOSED;
303
+ continue;
304
+ }
305
+ if (ch === p.ch) st.f = true;
306
+ }
307
+ return OPEN;
308
+ }
309
+ }
310
+ }
311
+ export {
312
+ __getIncScanned,
313
+ __resetIncScanned,
314
+ createInc,
315
+ incHighlight,
316
+ incSeed
317
+ };
package/dist/hi.d.ts CHANGED
@@ -8,5 +8,46 @@
8
8
  * highlight an open (streaming) block, which avoids re-highlighting the same
9
9
  * code on every chunk — the main perf win for streaming code.
10
10
  */
11
+ export declare function escapeHtml(s: string): string;
12
+ /**
13
+ * The resumable tokenizer's cursor: `pos` is the next source index to consume,
14
+ * `out` the markup emitted so far. Start a run at `{ pos: 0, out: "" }`.
15
+ */
16
+ export interface HighlightState {
17
+ pos: number;
18
+ out: string;
19
+ }
20
+ /**
21
+ * Called once per token the tokenizer emits, AFTER its markup is appended:
22
+ * `(cls, start, end, outLen)` where `cls` is the PATTERN class (`ws`, `str`,
23
+ * `com`, `pun`, `ident`… — not the `kw`/`fn`/`ty` refinement), `[start, end)` is
24
+ * the source span, and `outLen` is `state.out.length` once the token has been
25
+ * written. The catch-all one-character fallback reports `cls === ""`.
26
+ *
27
+ * Passing no sink is the default and costs one `undefined` test per token; the
28
+ * escape-fallback path (unknown language / over the size guard) emits no tokens
29
+ * and so reports nothing.
30
+ *
31
+ * @internal The incremental streaming path (hi-inc.ts) is the only consumer —
32
+ * it needs token boundaries to pick a checkpoint that survives an append.
33
+ */
34
+ export type TokenSink = (cls: string, start: number, end: number, outLen: number) => void;
35
+ /**
36
+ * One resumable slice of {@link highlight}. Consumes WHOLE tokens from
37
+ * `state.pos` until at least `chars` source characters have been taken (or the
38
+ * input ends), appending to `state.out`; returns true once the input is fully
39
+ * consumed.
40
+ *
41
+ * The pass carries NO state between tokens beyond `pos` — every pattern is
42
+ * sticky and matched against the immutable `code` — so stopping and resuming is
43
+ * invisible: for ANY sequence of chunk sizes the final `state.out` is
44
+ * byte-identical to `highlight(code, lang)` (test/hi-chunked.test.ts proves it
45
+ * over the language corpus, down to one token per slice). That is what lets a
46
+ * renderer spread a big block's highlight across several tasks without changing
47
+ * a byte of markup.
48
+ *
49
+ * @internal Not part of the semver surface — use {@link highlight}.
50
+ */
51
+ export declare function stepHighlight(code: string, lang: string, state: HighlightState, chars: number, sink?: TokenSink): boolean;
11
52
  export declare function highlight(code: string, lang: string): string;
12
53
  export declare function supportedLangs(): string[];
package/dist/hi.js CHANGED
@@ -147,26 +147,42 @@ const LANGS = {
147
147
  css: { pats: cssPats }
148
148
  };
149
149
  function escapeHtml(s) {
150
- let out = "";
151
- for (let i = 0; i < s.length; i++) {
152
- const c = s[i];
153
- if (c === "<") out += "&lt;";
154
- else if (c === ">") out += "&gt;";
155
- else if (c === "&") out += "&amp;";
156
- else if (c === '"') out += "&quot;";
157
- else out += c;
150
+ const n = s.length;
151
+ let i = 0;
152
+ for (; i < n; i++) {
153
+ const c = s.charCodeAt(i);
154
+ if (c === 60 || c === 62 || c === 38 || c === 34) break;
158
155
  }
159
- return out;
156
+ if (i === n) return s;
157
+ let out = s.slice(0, i);
158
+ let last = i;
159
+ for (; i < n; i++) {
160
+ const c = s.charCodeAt(i);
161
+ let esc;
162
+ if (c === 60) esc = "&lt;";
163
+ else if (c === 62) esc = "&gt;";
164
+ else if (c === 38) esc = "&amp;";
165
+ else if (c === 34) esc = "&quot;";
166
+ else continue;
167
+ out += s.slice(last, i) + esc;
168
+ last = i + 1;
169
+ }
170
+ return out + s.slice(last);
160
171
  }
161
- function highlight(code, lang) {
162
- if (code.length > 5e4) return escapeHtml(code);
163
- const conf = LANGS[lang.toLowerCase()];
164
- if (!conf) return escapeHtml(code);
165
- let out = "";
166
- let pos = 0;
172
+ function stepHighlight(code, lang, state, chars, sink) {
173
+ const conf = code.length > 5e4 ? void 0 : LANGS[lang.toLowerCase()];
174
+ const stop = state.pos + (chars > 0 ? chars : 1);
175
+ if (!conf) {
176
+ const end = stop < code.length ? stop : code.length;
177
+ state.out += escapeHtml(code.slice(state.pos, end));
178
+ state.pos = end;
179
+ return state.pos >= code.length;
180
+ }
181
+ let out = state.out;
182
+ let pos = state.pos;
167
183
  const pats = conf.pats;
168
184
  const kw = conf.kw;
169
- while (pos < code.length) {
185
+ while (pos < code.length && pos < stop) {
170
186
  let matched = false;
171
187
  for (let i = 0; i < pats.length; i++) {
172
188
  const [cls, re] = pats[i];
@@ -185,6 +201,7 @@ function highlight(code, lang) {
185
201
  finalCls = "ty";
186
202
  } else {
187
203
  out += escapeHtml(text);
204
+ if (sink) sink(cls, pos, after, out.length);
188
205
  pos = after;
189
206
  matched = true;
190
207
  break;
@@ -195,21 +212,33 @@ function highlight(code, lang) {
195
212
  } else {
196
213
  out += `<span class="t-${finalCls}">${escapeHtml(text)}</span>`;
197
214
  }
215
+ if (sink) sink(cls, pos, after, out.length);
198
216
  pos = after;
199
217
  matched = true;
200
218
  break;
201
219
  }
202
220
  if (!matched) {
203
221
  out += escapeHtml(code[pos]);
222
+ if (sink) sink("", pos, pos + 1, out.length);
204
223
  pos += 1;
205
224
  }
206
225
  }
207
- return out;
226
+ state.out = out;
227
+ state.pos = pos;
228
+ return pos >= code.length;
229
+ }
230
+ function highlight(code, lang) {
231
+ const state = { pos: 0, out: "" };
232
+ while (!stepHighlight(code, lang, state, code.length)) {
233
+ }
234
+ return state.out;
208
235
  }
209
236
  function supportedLangs() {
210
237
  return Object.keys(LANGS);
211
238
  }
212
239
  export {
240
+ escapeHtml,
213
241
  highlight,
242
+ stepHighlight,
214
243
  supportedLangs
215
244
  };
package/dist/react.d.ts CHANGED
@@ -128,6 +128,22 @@ interface BrookMarkdownProps {
128
128
  * applies only to the streaming tail.
129
129
  */
130
130
  childMemo?: boolean;
131
+ /**
132
+ * Highlight a code fence **while it is still streaming**, instead of showing
133
+ * plain escaped text until it closes. On by default.
134
+ *
135
+ * An open block keeps a frozen prefix and re-tokenizes only its tail on each
136
+ * patch, so this stays linear in the block's size (it does not re-highlight
137
+ * the whole fence per chunk). The settled markup is byte-identical either way
138
+ * — only the tail's colours are provisional, and they may shift as bytes
139
+ * arrive (`"hello` is a stray quote plus an identifier until its closing quote
140
+ * lands). Set `false` for the pre-0.27 behaviour: plain body until close.
141
+ *
142
+ * No effect on SSR (the server renders closed blocks only), and none at all
143
+ * when `components.CodeBlock` / `components.pre` / `components.code` take over
144
+ * the block — an override bypasses the built-in highlighter entirely.
145
+ */
146
+ streamingHighlight?: boolean;
131
147
  /** Appended to the root's `className` (the `brook-md` class is always present). */
132
148
  className?: string;
133
149
  /** Set on the root element. */
@@ -264,6 +280,7 @@ interface BlockViewProps {
264
280
  virtualize?: boolean;
265
281
  sanitize?: (html: string) => string;
266
282
  childMemo?: boolean;
283
+ streamingHighlight?: boolean;
267
284
  onRenderMetrics?: RenderMetricsHook;
268
285
  decorators?: Decorator[];
269
286
  urlTransform?: UrlTransform;
package/dist/react.js CHANGED
@@ -52,6 +52,7 @@ function BrookMarkdownFromClient({
52
52
  stickToBottom,
53
53
  sanitize,
54
54
  childMemo,
55
+ streamingHighlight,
55
56
  className,
56
57
  id,
57
58
  role,
@@ -104,6 +105,7 @@ function BrookMarkdownFromClient({
104
105
  virtualize,
105
106
  sanitize,
106
107
  childMemo,
108
+ streamingHighlight,
107
109
  onRenderMetrics: onMetrics,
108
110
  decorators,
109
111
  urlTransform,
@@ -486,6 +488,7 @@ function renderBlockContent({
486
488
  components,
487
489
  sanitize,
488
490
  childMemo,
491
+ streamingHighlight,
489
492
  decorators,
490
493
  urlTransform
491
494
  }) {
@@ -508,7 +511,18 @@ function renderBlockContent({
508
511
  switch (kind) {
509
512
  case "CodeBlock": {
510
513
  const wantsCodeOverride = !!components && (!!components.pre || !!components.code);
511
- if (!wantsCodeOverride) return /* @__PURE__ */ jsx(CodeBlock, { html: block.html, open: block.open });
514
+ if (!wantsCodeOverride) {
515
+ const source = block.kind.data?.code;
516
+ return /* @__PURE__ */ jsx(
517
+ CodeBlock,
518
+ {
519
+ html: block.html,
520
+ open: block.open,
521
+ code: typeof source === "string" ? source : void 0,
522
+ streamingHighlight
523
+ }
524
+ );
525
+ }
512
526
  break;
513
527
  }
514
528
  case "MathBlock":
@@ -570,7 +584,7 @@ function renderBlockContent({
570
584
  }
571
585
  function blocksEqual(prev, next) {
572
586
  if (prev.block == null || next.block == null) return prev.block === next.block;
573
- return prev.block.id === next.block.id && prev.block.html === next.block.html && prev.block.open === next.block.open && prev.block.speculative === next.block.speculative && prev.components === next.components && prev.virtualize === next.virtualize && prev.sanitize === next.sanitize && prev.childMemo === next.childMemo && prev.onRenderMetrics === next.onRenderMetrics && // Identity compare: an unstable decorators/urlTransform (fresh each render)
587
+ return prev.block.id === next.block.id && prev.block.html === next.block.html && prev.block.open === next.block.open && prev.block.speculative === next.block.speculative && prev.components === next.components && prev.virtualize === next.virtualize && prev.sanitize === next.sanitize && prev.childMemo === next.childMemo && prev.streamingHighlight === next.streamingHighlight && prev.onRenderMetrics === next.onRenderMetrics && // Identity compare: an unstable decorators/urlTransform (fresh each render)
574
588
  // busts the memo so every committed block re-decorates — the O(n²) footgun
575
589
  // the dev warning calls out. A hoisted/memoized value keeps the memo holding.
576
590
  prev.decorators === next.decorators && prev.urlTransform === next.urlTransform && // Same identity rule as onRenderMetrics: an inline `onBlockError={() => …}`
@@ -1,7 +1,16 @@
1
1
  interface Props {
2
2
  html: string;
3
3
  open: boolean;
4
+ /**
5
+ * The block's DECODED source, carried by `kind.data.code` when `blockData` is
6
+ * on. Identical to `decodeText(html)` — supplying it skips that whole-body
7
+ * regex + five entity passes, which on a big fence is the same order of work
8
+ * as the highlight itself. Absent (blockData off) the HTML is decoded here.
9
+ */
10
+ code?: string;
11
+ /** Highlight the block while it is still open. Default true. */
12
+ streamingHighlight?: boolean;
4
13
  }
5
- declare function CodeBlockImpl({ html, open }: Props): import("react/jsx-runtime").JSX.Element;
14
+ declare function CodeBlockImpl({ html, open, code, streamingHighlight }: Props): import("react/jsx-runtime").JSX.Element;
6
15
  export declare const CodeBlock: import("react").MemoExoticComponent<typeof CodeBlockImpl>;
7
16
  export {};
@@ -1,19 +1,71 @@
1
1
  import { Fragment, jsx, jsxs } from "react/jsx-runtime";
2
2
  import { memo, useCallback, useEffect, useMemo, useRef, useState } from "react";
3
3
  import { highlight } from "../hi.js";
4
+ import { highlightDeferred, highlightWithin } from "../hi-defer.js";
5
+ import { createInc, incHighlight, incSeed } from "../hi-inc.js";
4
6
  import { extractLang } from "../block-props.js";
5
7
  function decodeText(html) {
6
8
  const m = html.match(/<pre><code[^>]*>([\s\S]*?)<\/code><\/pre>/);
7
9
  if (!m) return "";
8
10
  return m[1].replace(/&lt;/g, "<").replace(/&gt;/g, ">").replace(/&quot;/g, '"').replace(/&#39;/g, "'").replace(/&amp;/g, "&");
9
11
  }
10
- function CodeBlockImpl({ html, open }) {
12
+ function CodeBlockImpl({ html, open, code, streamingHighlight }) {
11
13
  const lang = extractLang(html) || "text";
12
- const text = useMemo(() => open ? "" : decodeText(html), [html, open]);
13
- const highlighted = useMemo(() => {
14
+ const text = useMemo(() => open ? "" : code ?? decodeText(html), [html, open, code]);
15
+ const streaming = open && streamingHighlight !== false;
16
+ const openText = useMemo(
17
+ () => streaming ? code ?? decodeText(html) : "",
18
+ [streaming, code, html]
19
+ );
20
+ const incRef = useRef(null);
21
+ const [inc, setInc] = useState(null);
22
+ const sync = useMemo(() => {
14
23
  if (!text) return null;
15
- return highlight(text, lang);
24
+ if (typeof window === "undefined") return highlight(text, lang);
25
+ const st = incRef.current;
26
+ return highlightWithin(text, lang, st ? incSeed(st, text, lang) : void 0);
16
27
  }, [text, lang]);
28
+ useEffect(() => {
29
+ if (!streaming || typeof window === "undefined") {
30
+ incRef.current = null;
31
+ setInc((prev) => prev === null ? prev : null);
32
+ return;
33
+ }
34
+ let st = incRef.current;
35
+ if (st === null || st.lang !== lang.toLowerCase()) {
36
+ st = createInc(lang);
37
+ incRef.current = st;
38
+ }
39
+ const markup = st === null ? null : incHighlight(st, openText);
40
+ setInc(markup === null ? null : { lang, html: markup });
41
+ }, [streaming, openText, lang]);
42
+ const [slow, setSlow] = useState(null);
43
+ useEffect(() => {
44
+ if (!text || sync !== null) {
45
+ setSlow((prev) => prev === null ? prev : null);
46
+ return;
47
+ }
48
+ const run = highlightDeferred(text, lang);
49
+ if (run.html !== null) {
50
+ setSlow({ text, lang, html: run.html });
51
+ return;
52
+ }
53
+ let live = true;
54
+ const rest = run.rest;
55
+ if (rest) {
56
+ rest.then((out) => {
57
+ if (live && out !== null) setSlow({ text, lang, html: out });
58
+ });
59
+ }
60
+ return () => {
61
+ live = false;
62
+ run.cancel();
63
+ };
64
+ }, [text, lang, sync]);
65
+ const highlighted = sync ?? (slow !== null && slow.text === text && slow.lang === lang ? slow.html : null) ?? // The streaming tail. Not gated on `openText` identity: the markup lags the
66
+ // props by one commit, and showing last patch's spans beats flashing the
67
+ // whole block back to plain every tick. A language change does invalidate it.
68
+ (streaming && inc !== null && inc.lang === lang ? inc.html : null);
17
69
  const [copied, setCopied] = useState(false);
18
70
  const timerRef = useRef(null);
19
71
  useEffect(() => {
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "brookmd",
3
- "version": "0.26.0",
3
+ "version": "0.27.0",
4
4
  "description": "Zero-dep streaming markdown for the browser. Rust→WASM core, Web Worker per stream, incremental parse with speculative closure.",
5
5
  "type": "module",
6
6
  "sideEffects": ["./dist/worker.js", "./dist/styles.css"],