@ai-matrx/content-ir 0.11.7 → 0.13.0

package/CHANGELOG.md

@@ -1,5 +1,99 @@
 # Changelog
 
+## 0.13.0 — 2026-09-25
+
+**`/source` — fixes from the RC-B3 independent verification**
+(common-docs `projects/rich-content-unification/evidence/verify-RC-B3.md`).
+Every finding got a failing test first (`__tests__/source-verify-findings.test.ts`,
+14 of the 17 non-timing tests red on 0.12.0), then the fix:
+
+- **THE nested-fence rule now lives here** (`source/fence-nesting.ts`, moved from
+  matrx-frontend `components/markdown-core/fence-nesting.ts`): inside a
+  ```` ```markdown ```` / `md` / `mdx` fence, ```` ```lang ```` opens a nested
+  fence and a bare ```` ``` ```` closes the innermost one; a nested fence still
+  open at the end of the text retries strict CommonMark. The tokenizer uses it,
+  so it ends a nested markdown fence exactly where the renderer does (real row:
+  note `f9b7e9a2-…`). Generated cross-language vectors
+  `__tests__/fence-nesting-vectors.json`; aidream matrx-ai's Python twin runs a
+  byte-identical copy and fails on drift.
+- **Islands change only through `islandEdit(island, text)`.** A plain edit must
+  carry every island inside its range back byte-identical and in order —
+  otherwise `SourceSpliceError("island_edit")` names the island. Edits are
+  diffed segment by segment between islands, so anchors inside an edited
+  paragraph map exactly.
+- **Integrity is refused by default.** `requireIntegrity` defaults to `true`;
+  `false` is documented as tooling-only.
+- **Linear time on pathological input.** Every closer search goes through an
+  occurrence index; code spans use a per-paragraph run table; the single-`$`
+  scan stops at the rule's 400-character limit; embedded `__kind` probing has a
+  per-paragraph budget; `readXmlTag` takes a finder. 2 MB of `{{`: 21 s → 41 ms.
+- **A tree's root label belongs to the tree island** (the renderer splitter's
+  `findTreeBlockStart`).
+- **`$$` follows the core's pairing rule** (`source/math-pairs.ts`, ported from
+  `@ai-matrx/kit/delimiter-guard`): tokens outside code pair in order, prose
+  pairs are not math, an unpaired `$$` is literal. `$$` at the end of a line
+  pairs with a later closer, and a lone `$$` never locks the rest of a document.
+
+### Consumer action
+
+- `spliceSave` now THROWS on an island change (`island_edit`) and on disturbed
+  integrity (`integrity`). Editors change islands with `islandEdit`; tooling
+  that wants the report passes `requireIntegrity: false`.
+- matrx-frontend: import the fence rule from `@ai-matrx/content-ir/source`; the
+  local `components/markdown-core/fence-nesting.ts` is deleted.
+
+## 0.12.1
+
+Automatic changed-only republish (docs/metadata drift since the last tag — see
+`git diff npm/content-ir-core/v0.12.0..npm/content-ir-core/v0.12.1 -- apps/shared/content-ir-core`).
+No source changes intended and no consumer action required.
+
+## 0.12.0 — 2026-09-23
+
+**`/source` — the stored text is the truth, block by block** (rich-content
+unification PLAN decision 6, register row RC-B3). New subpath
+`@ai-matrx/content-ir/source` (also re-exported from the root):
+
+- `tokenizeSource(text)` — the offset-preserving tokenizer. Blocks
+  `{ kind: "prose" | "island" | "gap", islandType, start, end, startCp, endCp,
+  raw, complete, meta, inlines }` cover the text exactly (the concatenation of
+  `raw` IS the input — nothing trimmed or normalized). Every protected
+  construct is an island: code fences (incl. special languages, the ```json
+  string-aware close), bare typed / `__kind` / directive JSON, known XML
+  regions, attribute XML (line-leading and mid-line), unknown XML containers
+  (the renderer splitter's own balance tracker, ported), raw HTML blocks and
+  tags, HTML comments and pinned anchors `<!--@a:id-->`, `$$` / `\(` / `\[`
+  math and THE single-dollar rule, `{{variables}}`, `<matrxcite/>`, embedded
+  `__kind` JSON, `[Image URL: …]` media refs, front matter, box-drawing trees.
+  Islands inside a paragraph stay in that prose block as `inlines`.
+  Streaming-tolerant: an unclosed construct is an island to the end of the
+  text with `complete: false`; never throws. Offsets in UTF-16 and code points.
+- `spliceSave(original, edits)` — save = splice. Block-aligned replacements;
+  bytes outside the edits are copied, never rebuilt; no-op edits return the
+  original string. Returns a change set narrowed to the bytes that differ
+  (UTF-16 + code points) for anchor mapping (`mapPosition`, `mapRange`,
+  ProseMirror StepMap semantics — STORE-DESIGN §3.19) and an integrity
+  report naming any island outside the edit that the edit disturbed
+  (`requireIntegrity: true` throws instead).
+- `isSingleDollarMath` / `singleDollarMathEnd` — THE single-dollar math rule,
+  moved here from matrx-frontend's math normalizer so the renderer and the
+  tokenizer can never disagree.
+- `readXmlTag`, `XmlContainerTracker`, `buildCodePointIndex`,
+  `toCodePointOffset`, `toUtf16Offset`, `joinSource`, `listIslands`,
+  `blockAt`, `blockEdit`, `blockRangeEdit`.
+
+Proof: 39 tests (fixture corpus of every construct, 2,000 adversarial random
+documents, every prefix of every fixture, 1,000 random single-block edits;
+two planted mutations turned the suite red). matrx-frontend's read-only
+`pnpm check:source-roundtrip` ran every stored row — notes, chat text parts,
+agent and template messages, message templates, skills — 135,302 rows, all
+byte-identical, all edit-local.
+
+### Consumer action
+
+None required — additive. matrx-frontend adopts `isSingleDollarMath` from
+`/source` in its math normalizer.
+
 ## 0.11.7
 
 Automatic changed-only republish (docs/metadata drift since the last tag — see

package/README.md

@@ -18,6 +18,8 @@ The root export is the full surface and always will be. Since 0.10.0 each intern
 | `@ai-matrx/content-ir/registry` | kind registry types, storage transform, the dual gate |
 | `@ai-matrx/content-ir/convert` | kind ⇄ JSON Schema, the OpenAI structured-output converter |
 | `@ai-matrx/content-ir/wire` | partial-kind events, runtime wrapper readers, emit payload |
+| `@ai-matrx/content-ir/directives` | Kind Directives: slug grammar, the ONE decoder |
+| `@ai-matrx/content-ir/source` | offset-preserving source tokenizer, splice save + anchor-mapping change set, the single-dollar math rule |
 
 ```ts
 import { createKindStreamParser, envelopeFromCompleteValue } from "@ai-matrx/content-ir/core";