@quillmark/wasm 0.108.2 → 0.109.0

package/CHANGELOG.md

@@ -1,5 +1,301 @@
 # Changelog
 
+## v0.109.0 - 2026-08-24
+
+- **breaking** content: **`Normalized` is the precondition the projections
+  require, and every codec returns one.** A projection over a container tree
+  owes totality, and `Content` alone does not say whether `normalize` has run —
+  so `to_markdown` and `emit_content` each trusted a canonical shape their
+  signature did not ask for. `Content::into_normalized` is the mint (infallible:
+  canonicalizing is total, and the codecs go on calling `validate` after it),
+  `Normalized::into_content` the way back out, and the token derefs to
+  `&Content`, so **a read-only consumer needs no change** — `.text`, `.lines`,
+  `.marks`, `.islands`, `validate()`, `is_inline()` all reach through. What
+  moves is the signatures a caller names or a value it mutates:
+
+  | Crate | 0.108 | 0.109 |
+  |---|---|---|
+  | `quillmark-content` | `from_markdown` / `from_plaintext` / `from_canonical_json` / `serial::from_canonical_value` / `serial::from_authored_value` → `Content` | → `Normalized` |
+  | | `to_markdown(&Content)` | `to_markdown(&Normalized)` |
+  | | `Content::to_canonical_json` | `Normalized::to_canonical_json` |
+  | | `serial::to_canonical_value(&Content)` | `(&Normalized)` |
+  | `quillmark-typst` | `emit::emit_content(&Content)` | `(&Normalized)` |
+  | `quillmark-core` | `Card::body() -> &Content` | `-> &Normalized` |
+  | | `Card::overwrite_body(Content)` / `overwrite_field(_, Content)` | `impl Into<Normalized>` — a `Content` still passes |
+  | | `TypedReader::get_content{,_at}` / `CardReader::get_content{,_at}` → `Option<Content>` | `Option<Normalized>` |
+
+  A consumer that *mutates* a decoded content takes the round trip, which is
+  what the codecs used to run for it silently:
+
+  ```rust
+  // 0.108
+  let mut rt = from_markdown(md)?;
+  rt.marks.push(mark);
+  rt.normalize();
+
+  // 0.109
+  let mut rt = from_markdown(md)?.into_content();
+  rt.marks.push(mark);
+  let rt = rt.into_normalized();
+  ```
+
+  The op channel needs none of that: `apply_text_delta`, `apply_mark_ops`,
+  `apply_line_ops`, `apply_island_ops` and `apply_field_change` are forwarded on
+  `Normalized` and re-establish the invariant on the error path as well as the
+  success one. `to_plaintext` still takes `&Content` and reads a token through
+  the deref, projecting `text` alone with no walk to make total.
+
+- feat(content): **`quillmark_content::traverse` is where the container walks
+  live**, `runs` (adjacent lines sharing one container instance at a depth),
+  `items` (adjacent lines whose whole container is equal) and `segment` (a line
+  plus the continuations at its own nesting). Five call sites across three
+  crates had spelled these by hand, each with its own idea of when a run ends;
+  `Span` and the walks are public so a consumer reading `Content.lines` groups
+  them the way both projections and the quill census do.
+
+- perf(content): **`serial::to_canonical_value` and `to_canonical_json` take a
+  `Normalized`.**
+  Both cloned the whole content and normalized the copy on every call, on a
+  lane whose callers — the codecs, `Card::body`, the storage DTO — were already
+  holding the canonical form. The token now carries that, so the serialize path
+  spends an encode instead of a deep clone plus a repair pass. `to_canonical_json`
+  moves from `Content` to `Normalized` with it; a caller holding a raw `Content`
+  mints first, which is what the old body did for them silently.
+
+- fix(core): **a document body that `validate` refuses is refused on write, not
+  discovered on read.** `CanonicalContent`'s `Deserialize` parsed, normalized and
+  validated; its `Serialize` validated nothing, and `Card::overwrite_body` takes a
+  caller's content on the canonical-form token alone. A store could therefore
+  accept bytes it could not read back. The serializer now validates too and fails
+  with the invariant, at the boundary that cares and while the caller still holds
+  the value that produced it.
+
+- refactor(content): **the leaf-segment walk is one loop, not two.** `traverse`
+  gains `segment` — the block-opening line plus every following one that
+  continues it at the same nesting — and `export::emit_block` and the Typst
+  emitter's `segment_end` both call it. The fifth duplicated traversal, the one
+  #1364 did not list.
+
+- fix(content): **`to_markdown` no longer aborts the process on a deeply nested
+  content.** `Normalized` states that `normalize` has run, and `normalize`
+  repairs where `validate` rejects: nothing about canonicalization brings a
+  container path under `MAX_NESTING_DEPTH`, so a hand-built `Content` mints a
+  token that `validate` refuses. `export::emit_block` recursed one frame per
+  container level and overflowed the stack a few thousand levels down — a
+  SIGABRT no caller can catch, against a Typst emitter that checks the depth up
+  front and returns `EmitError::NestingTooDeep` for the same input. The walk is
+  now an explicit frame stack, as `json_depth_exceeds` and the quill census
+  already are, so the projection is total over every token its signature
+  accepts. `Normalized`'s docs settle the half the newtype does not close: the
+  token promises canonical, not valid — the mint stays infallible, the codecs go
+  on calling `validate` after it, and a projection that takes one owes totality
+  rather than trust. Only a Rust embedder hand-building a `Content` reaches the
+  shape; every decode lane (`from_markdown`, `from_canonical_value`, storage,
+  WASM, Python) rejects the depth already.
+
+- fix(typst): **a container inside a list item no longer terminates the list.**
+  The item's continuation indent reached its leaf path only, so a quote inside
+  an item opened at column 0 — where Typst ends the enclosing list. The item's
+  later blocks came back as top-level paragraphs and the next item started a
+  fresh list, which renumbers an ordered one from the quote on. A fence and a
+  nested list escaped it by reaching that indented leaf path; a transparent
+  unknown container did not, and neither would any container added later.
+  Indentation is now the walk's rather than each construct's: one rule opens
+  every block, leaf and container alike, at the enclosing list depth, so what
+  the content nests, the markup nests.
+
+- fix(content): **a `continues` line that crosses a container boundary no longer
+  survives.** A within-block break lives inside one container, and `LineOp::Join`
+  mints the crossing shape whenever it merges two lines of differing paths — the
+  line after the seam keeps continuing across it. Both projections already read
+  the flag as dead there (`export::emit_block` and `emit::segment_end` each
+  require the depth to match before absorbing a continuation), so `normalize`
+  now clears it, which states what was already true and changes nothing
+  observable. `Content::validate` gains `Invariant::ContinuesAcrossContainers`
+  to catch a hand-built content that skipped `normalize`, and
+  `LineOp::SetContinues` refuses the *deliberate* crossing up front with
+  `ApplyError::ContinuesAcrossContainers` — the same repair-or-refuse split the
+  line-kind rule already makes. This was the one relational line invariant
+  nothing checked: `validate` is otherwise strictly per-line, while every
+  container rule is a property of a line pair.
+
+- fix(content): **two adjacent containers of one shape are no longer read as
+  one.** Container identity is the container path plus contiguity, and the path
+  carried nothing to tell one instance from the next, so two adjacent runs of
+  equal shape welded: `[Quote], [Quote]` read as a single two-paragraph quote,
+  and two one-item lists as a single item whose second line came back as an
+  unnumbered continuation paragraph — the marker gone. `Container` now carries
+  an `instance` discriminator on every arm, `Content::normalize` canonicalizes
+  it to `0` (flipping to `1` only where the adjacent preceding run would
+  otherwise weld), and the two projections read it. Four defects close with it:
+  - `from_markdown("- a\n\n<!-- -->\n\n- b")` — the CommonMark idiom for
+    spelling two lists apart — no longer destroys the second list's marker.
+  - Two adjacent ordered lists typeset with their own numbering. They reached
+    the Typst emitter as one run and `+` markers numbered the second list on
+    from the first, so `1. 2.` / `1. 2.` rendered **1 2 3 4**. The run's first
+    item now states its number, which resets Typst's running counter. Every
+    ordered run's first item therefore lowers as `N. ` where a run starting at
+    1 lowered as `+ `; the page is identical, the generated markup is not, so
+    anything diffing or golden-comparing Typst source sees it.
+  - `1. a` beside a list starting at `3` keeps that `start` through the
+    Markdown projection. CommonMark reads only a list's first number, so
+    `1. a\n\n3. b` re-imported as one list of two items and the `start` was
+    lost — breaking the round-trip fixed point `export` documents. Adjacent
+    lists now alternate their marker (`-`/`+`, `.`/`)`), which is how
+    CommonMark itself spells two lists apart, so the boundary survives the
+    projection with no comment marker in the authored file.
+  - Adjacent `Unknown` containers of equal `(tag, attrs)` round-trip as two
+    **through storage**, the lane they have: an unknown container has no
+    Markdown syntax to alternate, so it projects transparently there as it
+    always did. The open-set promise that a container this build does not know
+    survives untouched is now total rather than holding up to an adjacency
+    quotient.
+
+  An item boundary is a parent boundary: two inner lists under two outer list
+  items are two lists, so an inner run restarts its `ordinal` and needs no
+  discriminator. `ordinal` is canonicalized alongside it, to a gapless 0-based index within
+  its run, so `[5, 9]` and `[0, 1]` stop being two spellings of the same two
+  items. `instance` is written to the wire only when non-zero, so a stored row
+  that needs no discriminator — nearly all of them — keeps its exact bytes and
+  its content hash. **Breaking for Rust consumers** that match `Container`
+  exhaustively: `Quote` is now a struct variant, and `ListItem`/`Unknown` carry
+  the extra field. On the TypeScript surface `instance` is optional; a consumer
+  that never writes adjacent same-shape siblings needs no change.
+
+  The block census counts what the projections see, so two adjacent runs of one
+  shape now count two where they counted one: a quill declining `list` or
+  `quote` reports the construct at a document that has two of them where it
+  reported one, and `plate::unsupported_construct` moves with it.
+
+  A blob written here carries `instance` only where a document holds adjacent
+  same-shape siblings, and a reader that predates the field ignores the key —
+  so such a blob loads on 0.108 with the two runs welded, and re-saving there
+  drops the boundary for good. The `@0.93.0` tag is unchanged because every
+  blob written before this release re-encodes byte for byte; the forward
+  direction is the one that costs, and only for the documents that spend the
+  key.
+
+- fix(blueprint): **a variant's `object` or `array<object>` cell expands per
+  property.** The cell went through the scalar path, so it rendered as
+  `controlled_by: !must_fill # object` — a null where the schema wants a
+  mapping, with every property's description, `default:` and type annotation
+  dropped, and the marker on a path the obligation predicate never warns at.
+  A cell is a field of its container and now expands as one, like every other
+  surface already did.
+- fix(core): **a `.quillignore` pattern holding more than one `*` ignores what
+  it names.** The matcher handled exactly one wildcard and returned no match
+  for the rest, so `**/*.tmp` and `*.sublime-*` were dead lines. Patterns now
+  compile once through `glob::Pattern`, matched against the whole path and the
+  basename. Two readings tighten to gitignore's: `*` stops at `/`, and a
+  pattern spelling out a `/` anchors at the bundle root rather than matching
+  any path that opens and closes with its halves. Both narrow what a line
+  ignores, so a bundle can gain a file it used to drop: `assets/*` covers
+  `assets/logo.png` and no longer `assets/icons/logo.png`, which `assets/**`
+  or the directory line `assets/` covers. No in-tree quill spells either shape.
+  A line always ignores the
+  name it spells out as well: `[` opens a character class and is an ordinary
+  character in a filename, so `Cinzel[wght].ttf` ignores both the variable font
+  of that name and the class it describes.
+- refactor: **`RenderError::coded(code, message)` is the one constructor for a
+  single-error-diagnostic failure.** Nine sites across five crates spelled
+  `from_diag(Diagnostic::new(Severity::Error, msg).with_code(code))` by hand,
+  two of them as a per-crate `engine_err` helper the backends each carried
+  their own copy of. Additive to `quillmark-core`'s public API; no code, message
+  or shape changes.
+- refactor(pdfform): **a session holds its flattened PDF parsed, not as bytes
+  each render path reparses.** Flatten and parse now happen together in `open`
+  and `update`, the two places `field_specs` are set, so the derived flat PDF
+  moves only with the specs it comes from and `render_svg`/`render_png`/
+  `render_rgba` paint parsed pages. A malformed flatten now surfaces from the
+  call that produced it under one code, `pdfform::flat_parse_failed`, replacing
+  the per-format `pdfform::svg_parse_failed` and `pdfform::png_parse_failed`
+  raised at render time (neither documented, and both reachable only through a
+  bug in this crate's own flatten). Opening a session fails on that bug now,
+  including for a caller that only ever renders the AcroForm PDF, which is
+  stamped from the base and reads nothing flattened.
+- perf(pdf): **filling a PDF form no longer slows down with the size of its
+  background or its page count.** Reading one object from the base walks every
+  byte of it — the live copy is the last revision, so a scan cannot stop early
+  — and nothing memoized that, so a stamp or flatten pass paid O(pages) whole-
+  file scans and the live-edit path repaid them on every keystroke. The base's
+  object offsets are now collected in one pass and each read is a lookup: a
+  20-page 300 KB form stamps in 0.7 ms rather than 37 ms, flat in page count.
+
+  **breaking** in `quillmark-pdf`: `PdfUpdate::begin` and
+  `PdfUpdate::resolve_pages` take the `&ObjectIndex` the caller builds over the
+  base rather than its bytes, and `reader::find_object_bytes` /
+  `reader::object_dict` become `ObjectIndex::object_bytes` / `ObjectIndex::dict`.
+- **breaking** content: the op wire is a reading direction. `mark_op_to_value`,
+  `line_op_to_value` and `island_op_to_value` are removed from
+  `quillmark-content` — an op bundle is authored on the JS/Python side and
+  reaches Rust through `change_bundle_from_value`, so nothing in the workspace
+  ever emitted one and every wire change was made twice, once in code no product
+  path executes. The decoders are unchanged. Their round-trip tests become
+  decoder tests over literal JSON, which is what the wire actually is: an
+  encoder agreeing with its own reader never proved the shape a binding sends.
+- perf(content): **canonical serialization stops rebuilding the tree it just
+  built.** `to_canonical_value` normalized a copy — which already recursively
+  key-sorts every opaque bag reachable from it (island `props`, an unknown's
+  `attrs`) — and then ran a whole-tree `sort_keys_owned` over the encoded
+  result, re-collecting and re-allocating every object and array in the document
+  to reorder the handful of fixed keys the encoders insert themselves. The
+  encoders now emit those keys in ascending order and the terminal pass is
+  `canonicalize_keys`, which scans and returns when the tree is already
+  canonical. Canonical bytes are unchanged, byte for byte; a tree that somehow
+  arrives unsorted is still repaired rather than shipped.
+
+  The public `container_to_value` and `mark_to_value`, and the crate-internal
+  `island_to_value`, now emit their own keys in a different order. An unknown's `attrs` bag is
+  untouched, as in 0.99, and nothing hashes the op wire.
+
+<!-- seed: commits since v0.108.3, confirm the entries above cover them, then delete this comment
+- docs: the 0.108 → 0.109 release record, and two misattached doc comments
+- docs: dense-prose pass over the branch
+- docs: dense-prose pass over the branch
+- fix: close the rest of the normalized-is-not-validated gap
+- fix(typst): open every block at the enclosing list indent
+- fix(content): walk export's block tree on a frame stack
+- docs: dense-prose pass over the branch
+- fix: hold the token's guarantee on error paths, and walk census iteratively
+- feat(content): make "normalized" a type the projections require
+- refactor: consume the shared container traversal
+- feat(content): one traversal for container runs and items
+- fix(content): an item boundary is a parent boundary
+- fix(content): clear a `continues` that crosses a container boundary
+- fix(content): give a container the instance its identity was missing
+- Bulk PR integration: #1347–#1356 (#1358)
+-->
+
+
+## v0.108.3 - 2026-08-21
+
+- fix(typst): **a paragraph holding one bare `/` renders instead of failing the
+  compile.** Typst's heading `=`, list `-`/`+`/`N.`, and term `/` markers fire
+  on a space after them *or* on the line ending there; the emitter's
+  line-anchor guard tested only for the space, so a run that was one bare
+  marker reached Typst unescaped — `/` as a term list whose colon is missing
+  (`expected colon`), the other four as an empty heading, bullet or enum item.
+  The guard now takes Typst's own test, and covers a list item's body head as
+  well as column 0, that being a line start the parser reads as one.
+
+- fix(typst): **bold text, a table cell or an indented paragraph opening with
+  `-`, `=`, `+`, `/` or `N.` renders as that text.** Typst reads the head of
+  every content block `[…]` as a line start of its own, so the marker in
+  `**/ x**` or in a table cell reached it as a term list whose colon is missing
+  and failed the compile, while `**- x**` drew a bullet list inside the bold.
+  Indentation is trivia and holds that line start open behind it, so a
+  paragraph beginning `  / x` failed the same way. The line-anchor guard now
+  covers every position Typst reads as a line start.
+
+- fix(typst): **text directly after inline code, bold or an image renders when
+  it opens with `(` or `.name`.** Typst reads a `(` directly after a `#…`
+  expression as that call's arguments and a `.` before an identifier as a field
+  access, so the emitter's own `#raw(…)`, `#strong[…]` and `#image(…)` handed
+  the text behind them to Typst as code — `` `x`(y) `` became a call on
+  content, which fails the compile. Such a run now takes the same `\` prefix
+  the line-anchor guard uses. Debug builds parse every emission with Typst's
+  parser, so markup that reaches it as syntax fails a test rather than a render.
+
 ## v0.108.2 - 2026-08-20
 
 - fix(core): **storage blobs tagged `@0.81.0` and `@0.82.0` load again.** Both
@@ -53,27 +349,6 @@
   sequence item (`- {}`) and a wholly empty `$ext: {}` already used, so an
   emptied container survives the round-trip as the value a consumer stored.
 
-<!-- seed: commits since v0.108.1, confirm the entries above cover them, then delete this comment
-- docs(canon): record that @0.81.0 is the oldest tag that exists
-- docs: leave the applied migration guide untouched
-- docs: dense-prose pass on the DTO backfill
-- test(core): drop the vacuous V0_81_0 body-import test
-- fix(core): restore the V0_82_0 read shim; 0.82.0 was never yanked (#1327)
-- chore(deps): bump the cargo group with 2 updates
-- chore(deps): bump taiki-e/install-action in the actions group
-- fix(core): restore the V0_81_0 storage read shim (#1327)
-- docs: changelog entry for the simplify sweep
-- refactor(core)!: apply simplify-review cleanups to the session seam
-- docs(pdf): trim test helper doc to its contract
-- refactor(core/document): apply simplify-review cleanups
-- refactor(pdf): apply simplify-review cleanups
-- refactor(typst): apply simplify-review cleanups
-- refactor(core/quill): apply simplify-review cleanups
-- refactor(content): apply simplify-review cleanups
-- fix: an empty mapping emits as `{}` rather than losing its key
--->
-
-
 ## v0.108.1 - 2026-08-19
 
 - fix: **a content cell under `variants:` is readable at its codec.**

package/backends/pdfform/wasm.d.ts

@@ -93,11 +93,18 @@ export type ContentLineKind =
 
 /** An ancestor block a line nests inside, outermost first. Open like
  * `ContentLine.kind`: an unrecognized container round-trips with opaque `attrs`
- * and renders transparently (its lines sit at the enclosing level). */
+ * and renders transparently (its lines sit at the enclosing level).
+ *
+ * Two adjacent lines sit in the same container iff their whole path matches.
+ * `instance` is what tells one container from an adjacent sibling of identical
+ * shape — two consecutive quotes, or two consecutive lists — which contiguity
+ * alone reads as one. Omit it (or write `0`) unless a path immediately above or
+ * below is otherwise identical; a write is canonicalized to `0`/`1` on the way
+ * in, so any distinct pair of values works. */
 export type ContentContainer =
-| { container: "list_item"; ordered: boolean; start: number; ordinal: number }
-| { container: "quote" }
-| { container: string; attrs: unknown };
+| { container: "list_item"; ordered: boolean; start: number; ordinal: number; instance?: number }
+| { container: "quote"; instance?: number }
+| { container: string; attrs: unknown; instance?: number };
 
 /** A mark over char range `[start, end)` into `Content.text`. The open `type`
  * arm blocks discriminant narrowing, so read a payload-carrying arm behind its

package/backends/typst/wasm.d.ts

@@ -93,11 +93,18 @@ export type ContentLineKind =
 
 /** An ancestor block a line nests inside, outermost first. Open like
  * `ContentLine.kind`: an unrecognized container round-trips with opaque `attrs`
- * and renders transparently (its lines sit at the enclosing level). */
+ * and renders transparently (its lines sit at the enclosing level).
+ *
+ * Two adjacent lines sit in the same container iff their whole path matches.
+ * `instance` is what tells one container from an adjacent sibling of identical
+ * shape — two consecutive quotes, or two consecutive lists — which contiguity
+ * alone reads as one. Omit it (or write `0`) unless a path immediately above or
+ * below is otherwise identical; a write is canonicalized to `0`/`1` on the way
+ * in, so any distinct pair of values works. */
 export type ContentContainer =
-| { container: "list_item"; ordered: boolean; start: number; ordinal: number }
-| { container: "quote" }
-| { container: string; attrs: unknown };
+| { container: "list_item"; ordered: boolean; start: number; ordinal: number; instance?: number }
+| { container: "quote"; instance?: number }
+| { container: string; attrs: unknown; instance?: number };
 
 /** A mark over char range `[start, end)` into `Content.text`. The open `type`
  * arm blocks discriminant narrowing, so read a payload-carrying arm behind its

package/core/wasm.d.ts

@@ -93,11 +93,18 @@ export type ContentLineKind =
 
 /** An ancestor block a line nests inside, outermost first. Open like
  * `ContentLine.kind`: an unrecognized container round-trips with opaque `attrs`
- * and renders transparently (its lines sit at the enclosing level). */
+ * and renders transparently (its lines sit at the enclosing level).
+ *
+ * Two adjacent lines sit in the same container iff their whole path matches.
+ * `instance` is what tells one container from an adjacent sibling of identical
+ * shape — two consecutive quotes, or two consecutive lists — which contiguity
+ * alone reads as one. Omit it (or write `0`) unless a path immediately above or
+ * below is otherwise identical; a write is canonicalized to `0`/`1` on the way
+ * in, so any distinct pair of values works. */
 export type ContentContainer =
-| { container: "list_item"; ordered: boolean; start: number; ordinal: number }
-| { container: "quote" }
-| { container: string; attrs: unknown };
+| { container: "list_item"; ordered: boolean; start: number; ordinal: number; instance?: number }
+| { container: "quote"; instance?: number }
+| { container: string; attrs: unknown; instance?: number };
 
 /** A mark over char range `[start, end)` into `Content.text`. The open `type`
  * arm blocks discriminant narrowing, so read a payload-carrying arm behind its

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@quillmark/wasm",
-  "version": "0.108.2",
+  "version": "0.109.0",
   "description": "WebAssembly bindings for Quillmark, a schema-driven document engine",
   "type": "module",
   "license": "Apache-2.0",

package/runtime/runtime.d.ts

@@ -245,6 +245,7 @@ export declare function isListItemContainer(
 	ordered: boolean;
 	start: number;
 	ordinal: number;
+	instance?: number;
 };
 
 // The guards above answer "is this arm X". These four answer "is this a value

package/runtime/runtime.js

@@ -454,7 +454,7 @@ export function isCodeLine(line) {
 
 /**
  * @param {import('../core/wasm.js').ContentContainer} container
- * @returns {container is import('../core/wasm.js').ContentContainer & { container: 'list_item'; ordered: boolean; start: number; ordinal: number }}
+ * @returns {container is import('../core/wasm.js').ContentContainer & { container: 'list_item'; ordered: boolean; start: number; ordinal: number; instance?: number }}
  */
 export function isListItemContainer(container) {
 	return container.container === 'list_item';