brookmd 0.25.2 → 0.26.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,260 @@ Notable changes to brookmd (formerly `flux-md`). Format based on
4
4
  [Keep a Changelog](https://keepachangelog.com/); this project aims to follow
5
5
  [Semantic Versioning](https://semver.org/).
6
6
 
7
+ ## 0.26.1 — 2026-07-30
8
+
9
+ Two fixes found by benchmarking 0.26.0 against real chat traffic. Requires
10
+ `brookmd-core` 0.25.1.
11
+
12
+ ### Fixed
13
+
14
+ - **A GFM table could not interrupt an open paragraph — anywhere.**
15
+ `scan_paragraph` had no table arm, so `item\n| a | b |\n|---|---|` swallowed
16
+ the delimiter row as paragraph text. The visible symptom was "tables inside
17
+ list items don't parse" (an item's de-indented body opens with a paragraph),
18
+ but the bug was position-independent and hit top level and blockquotes the
19
+ same way. The fix is an O(2-line) gate — header row with a pipe, delimiter
20
+ row next, matching cell counts, both rows indented ≤ 3 columns — checked once
21
+ per line as it arrives, on both the full-reparse and streaming paths, so a
22
+ scan started at a commit boundary renders byte-identically to a cold one.
23
+ Verified against GitHub's rendering for every non-pathological shape; the
24
+ deliberate exception (both rows must be ≤ 3-indented, so two exotic
25
+ mixed-indent shapes stay paragraphs) is pinned in `tests/nested_tables.rs`.
26
+ A nested table renders inside the item's `html` and — like nested lists —
27
+ carries no structured `blockData` of its own.
28
+
29
+ ### Performance
30
+
31
+ - **Close-time syntax highlighting no longer blocks the main thread.** The
32
+ built-in highlighter ran as one synchronous task when a block closed —
33
+ ~110 ms for a large fence on a mid desktop. The tokenizer loop was already
34
+ resumable at any offset with no carried state, so it now runs in ~5 ms
35
+ slices (`scheduler.yield()` where available, `MessageChannel` otherwise),
36
+ with the first slice synchronous so small blocks render highlighted in the
37
+ same tick with no flash. Output is byte-identical — pinned by a chunked ==
38
+ one-shot property test across all 20 languages at chunk sizes down to 1,
39
+ plus a 6,556-case differential fuzz against the previous implementation.
40
+ Also: the renderers now reuse the parser's already-decoded source
41
+ (`CodeBlockData.code`) when `blockData` is on instead of re-deriving it from
42
+ the HTML, and `escapeHtml` no longer concatenates per character. A 49 KB
43
+ block's longest main-thread task drops from ~38 ms to ≤ 6 ms on the same
44
+ hardware; `highlight()`'s public signature and bytes are unchanged, SSR
45
+ stays synchronous, and `components.CodeBlock`/`pre`/`code` overrides are
46
+ unaffected.
47
+
48
+ ## 0.26.0 — 2026-07-30
49
+
50
+ **Rendered HTML bytes change in this release.** Everything new below is opt-in
51
+ and default-off — with the flags off those paths are byte-identical to 0.25.2 —
52
+ but the output-fidelity work is unconditional: container framing newlines,
53
+ paragraph whitespace, table line breaks, fenced-code indentation, and the
54
+ task-list checkbox form now match the CommonMark and GFM reference renderers
55
+ exactly. No API is removed, and the wire envelope, ids, and commit semantics are
56
+ untouched — but HTML snapshots taken against 0.25.x will need regenerating.
57
+ That, not the new features, is why this is a minor bump.
58
+
59
+ What it buys: **652/652 CommonMark 0.31 and 24/24 GFM, byte-exact** against the
60
+ reference renderers, where before the suites passed only after structural
61
+ normalization. Byte-exactness is now the harnesses' default floor and is pinned
62
+ in CI, so one regressed byte fails the build.
63
+
64
+ Requires `brookmd-core` 0.25.0.
65
+
66
+ ### Added
67
+
68
+ - **`softBreaks` — a soft line break renders as `<br>`.** Strict CommonMark
69
+ treats a bare `\n` inside a paragraph as whitespace, so a model that writes one
70
+ thought per line gets one reflowed blob. This is the `remark-breaks` /
71
+ chat-comment convention where one Enter is one visual line, and it is what most
72
+ chat UIs actually want. Off by default; it only ever *adds* breaks (a hard break
73
+ is `<br>` either way), so no existing output loses a line.
74
+ - **`allowSchemes` — un-block a URL scheme brookmd blocks by default.** Bare
75
+ scheme names, no colon (`allowSchemes: ["file"]`), matched case-insensitively.
76
+ It reaches exactly one tier: the *overridable*-blocked schemes, today `file:`.
77
+ The script-executing tier (`javascript:`, `data:text/html`, …) is
78
+ non-overridable — listing one is a silent no-op, not an escape hatch — and the
79
+ encoded-evasion neutralization (percent- and entity-encoded scheme prefixes) is
80
+ unchanged and runs before the check. This exists for privileged embedders —
81
+ Electron shells, extensions, editor preview panes — that intercept link clicks
82
+ instead of navigating.
83
+ - **`lenientLists` — rescue an over-indented list item from becoming a code
84
+ block.** CommonMark §5.2 says a marker followed by 5+ columns of whitespace
85
+ starts an indented code block, so a model emitting `- const value = 1;`
86
+ renders a `<pre><code>` instead of a list item. With the flag on, a marker
87
+ followed by **6 or more columns of literal spaces** absorbs the padding into the
88
+ content column and the text parses as the item's own markdown. Deliberately
89
+ narrow: exactly 5 spaces still opens code (that column is the spec boundary
90
+ itself), a fence on the marker line stays a fence, an indent on a *later* line
91
+ stays code, and **tab** padding stays code — tabs are an authoring choice, model
92
+ over-indentation is always literal spaces. Excluding tabs is what holds the
93
+ divergence to a single spec example (274) and only while the flag is on; the
94
+ conformance suites run strict and are unaffected.
95
+ - **`blockHtml` — block-level raw HTML through the safe sanitizer.** A
96
+ `<details><summary>…</summary>…</details>` block rendered as an escaped code
97
+ block before; now it renders as real elements, sanitized. Only CommonMark HTML
98
+ block **types 6 and 7** qualify — types 1–5 (`<script>`/`<pre>`/`<style>`/
99
+ `<textarea>`, comments, processing instructions, declarations, CDATA) stay
100
+ escaped by design, so the constructs that can execute or swallow the rest of
101
+ the document never take effect.
102
+ It shares its token core with the inline raw-HTML path (one policy, not two),
103
+ tracks an open-tag stack with speculative closers so a half-streamed
104
+ `<details` never leaks a broken element, carries a void-element table so `<hr>`
105
+ and friends are not pushed onto that stack, and caps nesting depth at 100.
106
+ Half-streamed tags are suppressed while open and settle at finalize. Takes
107
+ effect only when the sanitizer is engaged (`htmlAllowlist` or `dropHtmlTags`);
108
+ on its own it does nothing.
109
+ - **`CodeBlockData.meta` — the fence info string's remainder.**
110
+ ```` ```ts title="src/main.ts" ```` now yields `lang: "ts"` **and**
111
+ `meta: 'title="src/main.ts"'` on the data channel and as a `meta` prop on
112
+ `components.CodeBlock`. Always-on like `lang` (no `blockData` needed) and
113
+ omitted entirely when the fence carried none, so a fence without meta is
114
+ byte-identical to before. It is deliberately **not** in the rendered HTML —
115
+ there is no `data-meta` attribute — because a filename header is a component's
116
+ job, not the parser's. While streaming it appears only once it can no longer
117
+ change (the opening fence line terminated by a newline, or at finalize), so a
118
+ header never flickers through a half-typed `title="src/ma`.
119
+ - **`ListItemData.start` — a source byte offset per list item.** Under
120
+ `blockData`, each top-level `items[]` entry carries the document-absolute offset
121
+ of the byte where its marker begins, same origin as `Block.start` and stable as
122
+ the document grows. That is enough to build task-checkbox writeback: locate the
123
+ `[ ]` from the item's offset and flip it in the original markdown, no HTML
124
+ round-trip. **Nested** items carry no offset rather than a wrong one — a nested
125
+ list is not a separate block, it renders against a synthesized de-indented
126
+ string with no document offset — and that limitation is documented on the type.
127
+ - Together `meta` and `start` bump the **wire contract to 1.3.0** (WIRE.md §9):
128
+ purely additive optional `data` keys, byte-identical when unexercised.
129
+ - All four flags are plumbed through every surface: client/worker config, the
130
+ server renderer, the `<brook-markdown>` element (`soft-breaks`,
131
+ `lenient-lists`, `block-html`, `allow-schemes`), React Native, Kotlin, Swift,
132
+ and the Flutter hand-written config.
133
+ - **Byte-exact conformance mode in the spec harnesses**, with default ratchet
134
+ floors `CMARK_MIN_EXACT=652` / `GFM_MIN_EXACT=24` — pinned explicitly in both
135
+ the CI and publish workflows alongside the older normalized floors. The four
136
+ deliberate differences from the reference (`target`/`rel` on links, `data-lang`
137
+ on code blocks, HTML5 void `<br>`, `style="text-align:…"` instead of GFM's
138
+ deprecated `align`) are folded by a documented `canonicalize` step applied to
139
+ **both** sides — the only transform on the byte-exact path, so it can erase our
140
+ intentional extras but never hide a structural divergence.
141
+ - **An unconditional `bindings` CI job** — FFI and C-ABI crate tests plus
142
+ regeneration-freshness checks for the React Native, Kotlin, and Swift bindings.
143
+ The existing binding workflows only ran on `pull_request` behind `paths:`
144
+ filters, so a push to main or a JS-only PR gated on nothing; that is how stale
145
+ binding goldens survived.
146
+
147
+ ### Changed
148
+
149
+ - **Container inner newlines now match the reference exactly** (cmark's `cr()`
150
+ rule): `<li>` / `<blockquote>` / alert framing newlines land where the
151
+ reference puts them, an empty blockquote gets its newline, and a `<li>` opening
152
+ with a tight paragraph no longer emits a spurious `\n`.
153
+ - **Paragraph whitespace follows the spec line by line.** Leading indentation is
154
+ stripped on *every* line, not just the first; trailing spaces and tabs are
155
+ dropped before a soft break; a hard-break line sheds the *next* line's indent.
156
+ Lazy continuation lines in blockquotes and lists keep their `\n` instead of
157
+ being glued with a space.
158
+ - **GFM tables emit one element per line**, matching the reference's framing, and
159
+ task-list checkboxes emit the reference byte-form
160
+ (`checked="" disabled="" type="checkbox"`).
161
+ - **Fenced code**: the body is de-indented by the opening fence's own indent (so
162
+ a fence at columns 1–3 no longer carries that indent into every line);
163
+ significant trailing spaces inside the last line are preserved; interior blank
164
+ lines before the closer are preserved. Tab-column arithmetic is fixed after
165
+ blockquote markers, across item-content boundaries, and after list markers.
166
+ - **Document assembly is now defined and documented** ([WIRE.md
167
+ §12](../../crates/brookmd-core/WIRE.md)): a block's `html` carries **no
168
+ trailing newline** — the terminator after a top-level block belongs to the
169
+ document, not the block — and concatenating blocks is a `cr()`-join.
170
+ `renderToString` follows that rule, so a server-rendered document and a
171
+ reference render agree byte-for-byte.
172
+ - **Native `BrookConfig` gained four fields** (`soft_breaks`, `allow_schemes`,
173
+ `lenient_lists`, `block_html`), appended in a newly documented **append-only
174
+ zone**: uniffi serializes a record's fields *positionally*, so inserting a field
175
+ anywhere above shifts every later read. The wire was verified to be
176
+ exact-consumption — a binding built against the old record fails loudly on a
177
+ version mismatch rather than silently mis-decoding — and the RN, Kotlin, and
178
+ Swift bindings are regenerated, with the Flutter hand-written config synced
179
+ (it also gained the previously-missing `wire_delta`). Stale wire goldens across
180
+ all four languages were re-synced. `brookmd-ffi` and `brookmd-cabi` go to
181
+ **0.3.0**: the native library and its generated bindings must ship in lockstep.
182
+
183
+ ### Performance
184
+
185
+ Every item below is pinned by a **wall-time** regression guard in
186
+ `crates/brookmd-core/tests/scaling.rs`, each paired with a flush-control twin
187
+ that isolates the shape from its chunking. That pairing is the point: work
188
+ counters alone provably miss the allocation-class quadratics below — the parser
189
+ was doing O(n) *counted* work while re-allocating O(n) bytes per append, so only
190
+ the clock could see it.
191
+
192
+ | Shape | Measured as | Before | After |
193
+ | --- | --- | --- | --- |
194
+ | growing fence opener line | growth over an 8× input span | 50.5× | **9.1×** |
195
+ | indented continuation lines | vs the unindented control | 304.6× | **1.1×** |
196
+ | ragged chunks parked on a blank partial line | vs control | 110.2× | **1.1×** |
197
+ | the same, inside a list item | work counters | 212× | **12.8×** |
198
+ | block-HTML sanitize | vs the escaped control | 247× | **1.3×** |
199
+
200
+ - **Fence opener-line growth** re-sliced the whole opener on every append —
201
+ quadratic in the info string's length. Now linear-ish: 9.1× over an 8× span.
202
+ - **Indented continuation lines** made `is_boundary` produce *zero* cut
203
+ candidates, so every append re-rendered the entire paragraph from its start. A
204
+ new indent-led boundary rule (with a hard-break-straddle exclusion) restores
205
+ cutting; the shape now costs 1.1× its unindented control.
206
+ - **Ragged chunking that parks on a whitespace-only partial line** dropped the
207
+ paragraph cache outright, forcing a full rebuild per append. The cache now
208
+ **suspends instead of dropping**: the closed view stays byte-identical and the
209
+ parse resumes when the line completes. List items inherit the fix through the
210
+ nested parser.
211
+ - **Block-level raw HTML** would otherwise re-sanitize its whole body per append
212
+ (measured 247×); the sanitize cache folds at token boundaries, shipping at 1.3×
213
+ the cost of the escaped control. Container, table-cell, and heading caches were
214
+ proven structurally immune to the same class and are pinned so they stay that
215
+ way.
216
+
217
+ ### Security
218
+
219
+ - **The raw-HTML attribute policy is now an explicit dropped-attribute table**
220
+ rather than scheme-checking alone: `srcdoc`, `is`, `autofocus`,
221
+ `contenteditable`, the DOM-clobbering `id` / `name`, the shadow-piercing
222
+ `slot` / `part` / `exportparts`, the `form*` family, the `xmlns:` / `xlink:`
223
+ prefixes, and `ping`. `formaction` and `ping` are **dropped outright** — for a
224
+ URL carrier that re-targets a form or fires a background beacon, dropping is
225
+ strictly stronger than validating its scheme. Component-tag props stay
226
+ permissive by design (they are consumer-mediated — your component decides what
227
+ to do with them, and they never become DOM attributes); that asymmetry is now
228
+ documented rather than incidental.
229
+ - **The web component's `gfm-tagfilter` attribute was never observed.** It was in
230
+ the element's config map but missing from `observedAttributes`, so it applied at
231
+ mount and then silently ignored every later change — a security-relevant flag
232
+ that could not be turned on after the fact. Fixed, and the list now carries a
233
+ comment tying it to the map so the next flag cannot repeat it.
234
+
235
+ ### Fixed
236
+
237
+ - **A mid-stream committed-view divergence**: a cut taken after a *single*
238
+ inter-word space could straddle the hard-break lookbehind, so a paragraph
239
+ containing entity-produced spaces rendered differently mid-stream than it did
240
+ one-shot. The streamed view now matches the one-shot render at every chunk
241
+ boundary again.
242
+
243
+ ## brookmd-react-native 0.1.7 — 2026-07-30
244
+
245
+ ### Added
246
+
247
+ - `softBreaks`, `allowSchemes`, `lenientLists`, and `blockHtml` on the native
248
+ `BrookConfig`, reaching the on-device parser through the same JSI path as the
249
+ existing flags. The generated TS/C++ bindings are regenerated from
250
+ `brookmd-ffi` 0.3.0 and are now freshness-checked on every push by the new CI
251
+ `bindings` job.
252
+
253
+ ### Changed
254
+
255
+ - `brookmd` dependency range `^0.25.0` → `^0.26.0`, and the vendored native
256
+ binaries are built against `brookmd-core` 0.25.0 — so the on-device renderer
257
+ produces the same reference-exact bytes as the browser. Native and JS must move
258
+ together here: uniffi records are positional, and the four new `BrookConfig`
259
+ fields land in the record's append-only zone.
260
+
7
261
  ## 0.25.2 — 2026-07-27
8
262
 
9
263
  Performance only. Rendered output is unchanged — byte-identical mid-stream, not
package/README.md CHANGED
@@ -461,6 +461,17 @@ export default function Doc({ md }: { md: string }) {
461
461
  - **`renderToString(md, { config })`** — synchronous HTML string, **zero React
462
462
  dependency** (imports cleanly with no `react` installed).
463
463
  - **`parseToBlocks(md, { config })`** — the block array, for custom rendering.
464
+
465
+ **Document assembly.** A `Block.html` never ends with a newline — the terminator
466
+ that follows a top-level block belongs to the *document*, not the block.
467
+ `renderToString` therefore joins blocks with cmark's `cr()` rule: insert `\n`
468
+ before a block only when the output doesn't already end with one, and end the
469
+ document with one `\n`. An unconditional `"\n".join(...)` would double the
470
+ newline a raw HTML block serializes for itself. **If you assemble
471
+ `parseToBlocks` output into a document string yourself, use the same rule** —
472
+ it's what makes the output byte-identical to a reference CommonMark/GFM renderer
473
+ (652/652 CommonMark 0.31 and 24/24 GFM extension examples, byte-for-byte). See
474
+ `WIRE.md` §12.
464
475
  - **`<BrookMarkdownStatic content config components />`** (from
465
476
  `brookmd/server/react`) — synchronous React tree for **render-once** contexts;
466
477
  render it with your framework's server renderer
@@ -614,12 +625,16 @@ const client = new BrookClient({
614
625
  gfmFootnotes: true, // [^1] + [^1]: → footnote section (default false)
615
626
  gfmMath: true, // $…$ / \(…\) inline + $$…$$ / \[…\] display math (default false)
616
627
  dirAuto: true, // per-block dir="auto" for RTL/bidi text (default false)
628
+ softBreaks: true, // a single \n renders as <br> (remark-breaks / chat convention; default false)
629
+ lenientLists: true, // marker + 6+ SPACES → item text, not indented code (default false)
617
630
  a11y: true, // task-list <label> + <th scope="col"> a11y markup (default false)
618
631
  unsafeHtml: false, // pass raw HTML through (default false — keep it false for untrusted input)
619
632
  componentTags: ["Thinking", "Callout"], // BLOCK custom tags w/ markdown inside (default none)
620
633
  inlineComponentTags: ["tik", "cite"], // INLINE custom tags (chips/citations) w/ markdown inside (default none)
621
634
  htmlAllowlist: ["br", "sub", "sup"], // safe raw-HTML sanitizer: [] = allow all but dangerous; list = only those (default off)
622
635
  dropHtmlTags: [], // tags removed entirely (comments always dropped when sanitizing; default off)
636
+ blockHtml: true, // extend the sanitizer to BLOCK raw HTML (<details>…); needs a list above (default false)
637
+ allowSchemes: ["file"], // un-block a default-blocked URL scheme (default none — see "Security")
623
638
  blockData: true, // opt-in structured kind.data per block (default false — see "Structured block data")
624
639
  },
625
640
  });
@@ -640,6 +655,32 @@ When to enable each flag:
640
655
  definitions. Off by default; see the footnote streaming caveat above.
641
656
  - `dirAuto: true` — when content can be RTL / mixed-direction. Emits per-block
642
657
  `dir="auto"` so the browser detects direction independently per block.
658
+ - `lenientLists: true` — when your LLM over-indents after a list marker. Strict
659
+ CommonMark (§5.2) says a marker followed by **5 or more** columns of
660
+ whitespace starts an indented code block, so a model writing
661
+ `- const value = 1;` renders as a `<pre><code>` block instead of a list
662
+ item. This flag raises that cutoff to **6 columns of literal spaces**: at 6+
663
+ the padding is absorbed into the item's content column and the text renders
664
+ as the item's own markdown (inline formatting, links, and nested lists all
665
+ parse normally). Off by default, so strict-CommonMark output is unchanged.
666
+
667
+ Four cases stay strictly conformant **by design** — the flag is deliberately
668
+ narrow, not a general "fix my indentation" pass:
669
+
670
+ | Input | Stays | Why |
671
+ | --- | --- | --- |
672
+ | `-` + exactly 5 spaces | code block | 5 columns is the §5.2 boundary itself; relaxing it would swallow genuine one-space-past-the-minimum code |
673
+ | `` - ```js `` (fence on the marker line) | fenced code | the marker line opens a real fence — there is no over-indentation to undo |
674
+ | `-` then code indented on a **later** line | code block | the decision reads only the marker's own line; a later-line indent is unambiguous authored code |
675
+ | `-\t\tfoo` (tab padding) | code block | tab padding is a deliberate authoring choice, unlike model over-indentation which is always literal spaces |
676
+
677
+ Excluding tabs is what keeps the divergence from CommonMark down to a single
678
+ spec example (274, `1.` + 6 spaces). Everything else in the 652-example suite
679
+ renders identically with the flag on or off; the conformance suites
680
+ themselves run in strict mode and are unaffected.
681
+
682
+ The rule is a pure per-line comparison made when the marker is first scanned,
683
+ so it costs no lookahead and no re-parse while streaming.
643
684
  - `a11y: true` — opt-in accessibility markup that deviates from strict GFM
644
685
  byte-output: wraps task-list checkboxes in a `<label>` (screen-reader
645
686
  association) and adds `scope="col"` to table headers. Off by default so
@@ -664,6 +705,15 @@ When to enable each flag:
664
705
  - `htmlAllowlist` / `dropHtmlTags` — render a **safe subset of raw HTML** (e.g.
665
706
  `<br>`, `<sub>`, `<sup>`) natively without `unsafeHtml`, drop specific tags, and
666
707
  drop HTML comments. See [Safe raw HTML](#safe-raw-html).
708
+ - `blockHtml: true` — extend that sanitizer to **block-level** raw HTML, so a
709
+ `<details><summary>…</summary>…</details>` block renders as real elements
710
+ instead of an escaped code block. Needs one of the two lists above to be set;
711
+ `<script>`/`<pre>`/`<style>`/`<textarea>` blocks stay escaped. See
712
+ [Block-level raw HTML](#block-level-raw-html-blockhtml).
713
+ - `allowSchemes: ["file"]` — un-block a URL scheme brookmd blocks by default,
714
+ for privileged hosts (Electron, extensions) that intercept link clicks instead
715
+ of navigating. Script-executing schemes can never be re-enabled. See
716
+ [Un-blocking a scheme](#un-blocking-a-scheme--allowschemes).
667
717
 
668
718
  **Footnotes** (`gfmFootnotes`) work in streaming with one honest caveat: a
669
719
  `[^1]` reference renders speculatively the moment it's seen (committed blocks
@@ -763,8 +813,9 @@ attributes (with `class`→`className` and `style` as an object) plus `children`
763
813
  **Block-kind** keys (`CodeBlock`, `Mermaid`, `MathBlock`, `Alert`, `Paragraph`,
764
814
  `Heading`, `List`, `Blockquote`, `Table`, `Rule`, `Html`) replace the entire
765
815
  block. The component receives [`BlockComponentProps`](#types): `{ block, html,
766
- open, speculative }`, plus `text`/`language` for code/math blocks (the alert
767
- type is at `block.kind.data.kind`).
816
+ open, speculative }`, plus `text`/`language` for code/math blocks and `meta`,
817
+ the rest of a fence's info string (```` ```ts title="src/main.ts" ````), for a
818
+ filename header (the alert type is at `block.kind.data.kind`).
768
819
 
769
820
  > **One map, two prop contracts — the single biggest footgun.** The keys above
770
821
  > are looked up by TWO dispatchers. The block-kind dispatcher passes
@@ -866,7 +917,7 @@ byte-identical, so non-users pay nothing.
866
917
  |------|-------------------|------|-----|
867
918
  | `Table` | `{ headers, rows, aligns }`, cells `{ text, html }` | `props.table` | sort / filter / transpose / CSV / chart |
868
919
  | `Heading` | `{ level, text, id }` | `props.heading` | table of contents with anchors |
869
- | `CodeBlock` | `{ lang, code }` | `props.code` | decoded source (copy / run) |
920
+ | `CodeBlock` | `{ lang, meta?, code }` | `props.code` | decoded source (copy / run) |
870
921
  | `MathBlock` | `{ latex }` | `props.math` | LaTeX source (re-render) |
871
922
  | `List` | `{ ordered, start }` | `props.list` | ordered-list numbering |
872
923
 
@@ -1004,11 +1055,77 @@ new BrookClient({ config: { htmlAllowlist: [] } });
1004
1055
  text stays as inert text).
1005
1056
  - Every rendered tag's **attributes are sanitized**: `on*` handlers and `style`
1006
1057
  (a CSS beacon / clickjacking vector) are dropped, and dangerous URL schemes
1007
- (`javascript:`, …, including multi-encoded) become `#`.
1008
- - **Scope:** *inline* raw HTML. Block-level raw HTML stays escaped for now (use
1009
- `unsafeHtml` **without** the sanitizer to render block HTML when the sanitizer
1010
- is engaged, block HTML stays escaped even if `unsafeHtml` is also on). Tag
1011
- matching is case-insensitive.
1058
+ (`javascript:`, …, including multi-encoded) become `#` — in `href`, `src`,
1059
+ `srcset`, `poster`, `cite`, `action`, `data`, `longdesc` and `background`.
1060
+ - A further set of **DOM-hazard attributes is dropped outright** (case-insensitive)
1061
+ they neither execute nor carry a URL, but each lets authored markup reach
1062
+ past the text it should be:
1063
+ `srcdoc` (inline document injection), `is` (customized-built-in upgrade),
1064
+ `autofocus` / `contenteditable` (focus-steal, UI spoof), `id` / `name` (DOM
1065
+ clobbering — an element shadowing `document.getElementById` or a global),
1066
+ `slot` / `part` / `exportparts` (shadow-DOM injection), `form`, `formaction`,
1067
+ `formenctype`, `formmethod`, `formnovalidate`, `formtarget` (form hijack),
1068
+ `xmlns` / `xlink:*` (namespace escape hatch), and `ping` (tracking beacon).
1069
+ Some are inert *today* only because the tag that gives them meaning is already
1070
+ in the dangerous set — they are dropped anyway so the policy never depends on
1071
+ that coincidence. `class`, `title`, `alt`, `target`, `rel`, `data-*` and
1072
+ `aria-*` are unaffected.
1073
+ - **This applies to raw HTML only.** [Component tags](#component-tags) stay
1074
+ permissive: their attributes become framework *props* on `components[tag]`, so
1075
+ `<Tab id="x">` keeps `id` — the consumer's component decides whether it ever
1076
+ reaches the DOM. `on*`, `style` and dangerous URL schemes are filtered there too.
1077
+ - **Scope:** *inline* raw HTML by default. Block-level raw HTML stays escaped
1078
+ unless you also set `blockHtml` (below). Tag matching is case-insensitive.
1079
+
1080
+ #### Block-level raw HTML (`blockHtml`)
1081
+
1082
+ A model that emits a disclosure widget on its own lines —
1083
+
1084
+ ```html
1085
+ <details>
1086
+ <summary>Sources</summary>
1087
+ Three filings and a transcript.
1088
+ </details>
1089
+ ```
1090
+
1091
+ — produces an *HTML block*, not inline HTML, so the sanitizer above leaves it
1092
+ escaped. Opt in with `blockHtml` and it renders as real elements:
1093
+
1094
+ ```ts
1095
+ new BrookClient({ config: { htmlAllowlist: [], blockHtml: true } });
1096
+ // or restrict it:
1097
+ new BrookClient({ config: { htmlAllowlist: ["details", "summary"], blockHtml: true } });
1098
+ ```
1099
+
1100
+ - **Only meaningful with the sanitizer engaged.** `blockHtml` on its own does
1101
+ nothing; it extends `htmlAllowlist` / `dropHtmlTags` to block level. Default
1102
+ `false`, so existing sanitizer users keep escaped block HTML until they opt in.
1103
+ - **Same policy, no exceptions.** Tags go through the same allow / drop /
1104
+ non-overridable-dangerous decision and the same hardened attribute policy as
1105
+ inline raw HTML — a block-level `<div onclick=… id=… srcdoc=…>` renders as a
1106
+ bare `<div>`.
1107
+ - **Scope: CommonMark HTML block types 6 and 7** — a known block-level tag
1108
+ (`<details>`, `<div>`, `<table>`, `<section>`, …) or any other complete tag
1109
+ alone on its line. **Types 1–5 stay escaped/dropped**, as with the flag off:
1110
+ type 1 is the raw-text family (`<script>`, `<pre>`, `<style>`, `<textarea>`),
1111
+ where a browser reads everything after the opening tag as unparsed text — so a
1112
+ *speculative* close mid-stream is an mXSS vector — and types 2–5 (comments,
1113
+ processing instructions, CDATA, declarations) carry no renderable element at
1114
+ all. A block `<script>` is escaped in every configuration, including with
1115
+ `script` explicitly allowlisted and `unsafeHtml` also on.
1116
+ - **Streaming: speculative closers.** While the block is still arriving, every
1117
+ still-open element gets a closer appended, so the HTML you have received *so
1118
+ far* is always a complete tree — `<div>\n<b>bol` renders as
1119
+ `<div><b>bol</b></div>`, and the closers simply stop being speculative when the
1120
+ author's own `</b></div>` lands (the emitted bytes don't change). A half-arrived
1121
+ tag (`<spa`, `<a href="htt`) renders as **nothing** until it completes, the same
1122
+ pending-invisible contract as a streaming markdown link's URL; if the stream
1123
+ ends on one, it settles as escaped text. Mis-nesting is repaired rather than
1124
+ propagated: `<b><i></b>` emits `<b><i></i></b>`, and a close tag matching
1125
+ nothing open is dropped. A type-6/7 block ends at a blank line even with tags
1126
+ open — the closers land there.
1127
+ - **Markdown inside the HTML is not parsed** (the body is text + tags). That —
1128
+ full `rehype-raw` semantics — is a later stage.
1012
1129
 
1013
1130
  ### Types
1014
1131
 
@@ -1033,7 +1150,8 @@ interface BlockComponentProps {
1033
1150
  open: boolean;
1034
1151
  speculative: boolean;
1035
1152
  text?: string; // decoded source — CodeBlock / MathBlock
1036
- language?: string; // info string — CodeBlock
1153
+ language?: string; // info string, first word — CodeBlock
1154
+ meta?: string; // info string, the rest (`title="src/main.ts"`) — CodeBlock
1037
1155
  }
1038
1156
  ```
1039
1157
 
@@ -1051,14 +1169,35 @@ const html = highlight("const x = 1;", "ts");
1051
1169
 
1052
1170
  ## Coverage
1053
1171
 
1054
- **CommonMark 0.31: 100% (652/652 spec examples)** — every section, including
1055
- the hard ones (nested/loose lists, link reference definitions, link precedence,
1056
- lazy blockquote continuation). Plus GFM extensions: tables, strikethrough, task
1057
- lists, extended autolinks, GitHub alerts (`> [!NOTE]` → styled callouts),
1058
- footnotes (`[^1]` + `[^1]:`), and math (`$…$`, `$$…$$`, `\(…\)`, `\[…\]`).
1059
- Autolinks and alerts are on by default; footnotes and math are opt-in per stream
1060
- (see [Per-stream config](#per-stream-config)). See
1061
- `crates/brookmd-core/tests/{cmark_spec,gfm_spec,footnotes,math}.rs` for runners and floors.
1172
+ **CommonMark 0.31: 100% (652/652 spec examples), byte-exact** — every section,
1173
+ including the hard ones (nested/loose lists, link reference definitions, link
1174
+ precedence, lazy blockquote continuation). Plus GFM extensions, also **24/24
1175
+ byte-exact**: tables, strikethrough, task lists, extended autolinks, GitHub
1176
+ alerts (`> [!NOTE]` → styled callouts), footnotes (`[^1]` + `[^1]:`), and math
1177
+ (`$…$`, `$$…$$`, `\(…\)`, `\[…\]`). Autolinks and alerts are on by default;
1178
+ footnotes and math are opt-in per stream (see
1179
+ [Per-stream config](#per-stream-config)).
1180
+
1181
+ *Byte-exact* means the full output string matches the reference renderer's
1182
+ byte-for-byte — not a structurally normalized or whitespace-forgiving compare.
1183
+ Both counts are the harnesses' **default** floors (`CMARK_MIN_EXACT=652`,
1184
+ `GFM_MIN_EXACT=24`, pinned explicitly in the CI workflows), so a single byte of
1185
+ regression fails the build even when the normalized tally stays green.
1186
+
1187
+ The only deviations from the reference output are deliberate brookmd choices,
1188
+ folded by a documented `canonicalize` step applied to **both** sides before
1189
+ comparison — the only transform on the byte-exact path, so it can erase our
1190
+ intentional extras but never hide a structural divergence:
1191
+
1192
+ | Deliberate difference | Reference emits |
1193
+ |---|---|
1194
+ | `target="_blank" rel="noopener noreferrer nofollow"` on links (security-only) | no such attrs |
1195
+ | `data-lang="…"` on code blocks (alongside `class="language-…"`) | `class` only |
1196
+ | HTML5 void elements (`<br>`) | XHTML self-closing (`<br />`) |
1197
+ | `style="text-align:…"` on table cells | GFM's deprecated `align="…"` |
1198
+
1199
+ See `crates/brookmd-core/tests/{cmark_spec,gfm_spec,footnotes,math}.rs` for the
1200
+ runners, the `canonicalize` source, and the floors.
1062
1201
 
1063
1202
  GitHub alerts render to GitHub-compatible markup
1064
1203
  (`<div class="markdown-alert markdown-alert-note">…`), so existing markdown CSS
@@ -1125,17 +1264,61 @@ brookmd is XSS-safe by default — its HTML output is meant to be injected via
1125
1264
  - **Raw HTML is escaped** (the `unsafeHtml: true` config flag disables this;
1126
1265
  **never enable it for untrusted input without a `sanitize` hook**).
1127
1266
  - **Dangerous URL schemes are neutralized** in `<a href>` and `<img src>` —
1128
- `javascript:`, `vbscript:`, `data:text/html`, `data:text/javascript` become
1129
- `#`. The check runs on the *decoded* URL and strips characters browsers
1130
- ignore in the scheme, so obfuscations like `javascript&#58;…`,
1131
- `javascript\:…`, `&#106;avascript:…`, and embedded tabs/newlines are caught,
1132
- not just the literal form. (See `crates/brookmd-core/tests/security.rs`.)
1267
+ `javascript:`, `vbscript:`, `data:text/html`, `data:text/javascript` (and
1268
+ `file:`, see below) become `#`. The check runs on the *decoded* URL and strips
1269
+ characters browsers ignore in the scheme, so obfuscations like
1270
+ `javascript&#58;…`, `javascript\:…`, `&#106;avascript:…`, and embedded
1271
+ tabs/newlines are caught, not just the literal form. (See
1272
+ `crates/brookmd-core/tests/security.rs`.)
1273
+ - **The opt-in raw-HTML sanitizer removes DOM-hazard attributes.** Beyond `on*`,
1274
+ `style` and URL schemes, [Safe raw HTML](#safe-raw-html) drops `srcdoc`, `is`,
1275
+ `autofocus`, `contenteditable`, `id`/`name` (DOM clobbering), `slot`/`part`/
1276
+ `exportparts`, the `form*` family and `ping` from raw tags — a denylist by
1277
+ *policy*, not one that happens to be safe because some other tag is blocked.
1278
+ brookmd's own generated ids (footnote `fn-N`/`fnref-N`, heading slugs) are
1279
+ emitted by the renderer and never pass through the sanitizer, so they are
1280
+ unaffected. Component-tag props are exempt (see that section).
1133
1281
  - **`htmlToReact` defends in depth**: it drops inline `on*` event-handler
1134
1282
  attributes and runs URL attributes through the same scheme filter. It's
1135
1283
  intended for brookmd's own (already-sanitized) HTML; if you hand it arbitrary
1136
1284
  third-party HTML, these guards are your only line of defense — prefer a
1137
1285
  dedicated HTML sanitizer for genuinely hostile input.
1138
1286
 
1287
+ ### Un-blocking a scheme — `allowSchemes`
1288
+
1289
+ `file:` is blocked by default. It can't execute script, but it has no legitimate
1290
+ use in untrusted/LLM markdown, and in a **privileged** host — Electron, a browser
1291
+ extension, a page on a `file://` origin — a live `file:` href is a
1292
+ local-resource-disclosure / phishing vector. In an ordinary browser tab blocking
1293
+ it costs nothing anyway: a page simply **refuses to navigate** to a `file:` URL,
1294
+ so the href is inert there either way.
1295
+
1296
+ Some hosts genuinely need it. A coding-agent UI whose model links to local paths
1297
+ wants the href present so it can **intercept the click** and open the file in an
1298
+ editor or preview pane. `allowSchemes` is the opt-in for exactly that:
1299
+
1300
+ ```ts
1301
+ const client = new BrookClient({ config: { allowSchemes: ["file"] } });
1302
+ ```
1303
+
1304
+ - Takes **bare scheme names, no colon** (`["file"]`), matched
1305
+ case-insensitively.
1306
+ - It **never restricts** anything — this is not a general allowlist. Schemes
1307
+ outside the built-in blocklist (`vscode:`, `ftp:`, `mailto:`, …) already render
1308
+ today and are unaffected by this setting.
1309
+ - The **script-executing tier is non-overridable**: `javascript:`, `vbscript:`,
1310
+ `data:text/html`, `data:text/javascript`, and the scriptable `data:` media
1311
+ types (`data:image/svg`, `data:application/xhtml`, `data:text/xml`, …) stay
1312
+ blocked no matter what you list. Naming one is a silent no-op — the same rule
1313
+ as `htmlAllowlist`, where allowlisting `<script>` still cannot re-enable it.
1314
+ - Applies uniformly to links, URI autolinks, images, and sanitized URL
1315
+ attributes.
1316
+
1317
+ Turning it on moves the risk to you: **intercept link clicks rather than letting
1318
+ navigation happen**, and treat the path as untrusted input at the point you act
1319
+ on it. In a privileged host, whether a model-authored `file:///…` is allowed to
1320
+ reach a real file is the embedder's decision, not brookmd's.
1321
+
1139
1322
  ### Rendering untrusted / LLM HTML safely
1140
1323
 
1141
1324
  If you enable `unsafeHtml` to render HTML from an untrusted source (e.g. an LLM
@@ -38,8 +38,9 @@ function blockProps(block) {
38
38
  if (block.kind.type === "CodeBlock") {
39
39
  props.text = data?.code ?? decodeCodeText(block.html);
40
40
  props.language = data?.lang ?? "";
41
+ if (typeof data?.meta === "string") props.meta = data.meta;
41
42
  if (typeof data?.code === "string") {
42
- props.code = { lang: data.lang ?? null, code: data.code };
43
+ props.code = { lang: data.lang ?? null, meta: data.meta, code: data.code };
43
44
  }
44
45
  } else if (block.kind.type === "MathBlock") {
45
46
  props.text = data?.latex ?? decodeMathText(block.html);