@ai-matrx/content-ir 0.11.6 → 0.12.0

package/CHANGELOG.md

@@ -1,5 +1,57 @@
 # Changelog
 
+## 0.12.0 — 2026-09-23
+
+**`/source` — the stored text is the truth, block by block** (rich-content
+unification PLAN decision 6, register row RC-B3). New subpath
+`@ai-matrx/content-ir/source` (also re-exported from the root):
+
+- `tokenizeSource(text)` — the offset-preserving tokenizer. Blocks
+  `{ kind: "prose" | "island" | "gap", islandType, start, end, startCp, endCp,
+  raw, complete, meta, inlines }` cover the text exactly (the concatenation of
+  `raw` IS the input — nothing trimmed or normalized). Every protected
+  construct is an island: code fences (incl. special languages, the ```json
+  string-aware close), bare typed / `__kind` / directive JSON, known XML
+  regions, attribute XML (line-leading and mid-line), unknown XML containers
+  (the renderer splitter's own balance tracker, ported), raw HTML blocks and
+  tags, HTML comments and pinned anchors `<!--@a:id-->`, `$$` / `\(` / `\[`
+  math and THE single-dollar rule, `{{variables}}`, `<matrxcite/>`, embedded
+  `__kind` JSON, `[Image URL: …]` media refs, front matter, box-drawing trees.
+  Islands inside a paragraph stay in that prose block as `inlines`.
+  Streaming-tolerant: an unclosed construct is an island to the end of the
+  text with `complete: false`; never throws. Offsets in UTF-16 and code points.
+- `spliceSave(original, edits)` — save = splice. Block-aligned replacements;
+  bytes outside the edits are copied, never rebuilt; no-op edits return the
+  original string. Returns a change set narrowed to the bytes that differ
+  (UTF-16 + code points) for anchor mapping (`mapPosition`, `mapRange`,
+  ProseMirror StepMap semantics — STORE-DESIGN §3.19) and an integrity
+  report naming any island outside the edit that the edit disturbed
+  (`requireIntegrity: true` throws instead).
+- `isSingleDollarMath` / `singleDollarMathEnd` — THE single-dollar math rule,
+  moved here from matrx-frontend's math normalizer so the renderer and the
+  tokenizer can never disagree.
+- `readXmlTag`, `XmlContainerTracker`, `buildCodePointIndex`,
+  `toCodePointOffset`, `toUtf16Offset`, `joinSource`, `listIslands`,
+  `blockAt`, `blockEdit`, `blockRangeEdit`.
+
+Proof: 39 tests (fixture corpus of every construct, 2,000 adversarial random
+documents, every prefix of every fixture, 1,000 random single-block edits;
+two planted mutations turned the suite red). matrx-frontend's read-only
+`pnpm check:source-roundtrip` ran every stored row — notes, chat text parts,
+agent and template messages, message templates, skills — 135,302 rows, all
+byte-identical, all edit-local.
+
+### Consumer action
+
+None required — additive. matrx-frontend adopts `isSingleDollarMath` from
+`/source` in its math normalizer.
+
+## 0.11.7
+
+Automatic changed-only republish (docs/metadata drift since the last tag — see
+`git diff npm/content-ir-core/v0.11.6..npm/content-ir-core/v0.11.7 -- apps/shared/content-ir-core`).
+No source changes intended and no consumer action required.
+
 ## 0.11.6
 
 Automatic changed-only republish (docs/metadata drift since the last tag — see

package/README.md

@@ -18,6 +18,8 @@ The root export is the full surface and always will be. Since 0.10.0 each intern
 | `@ai-matrx/content-ir/registry` | kind registry types, storage transform, the dual gate |
 | `@ai-matrx/content-ir/convert` | kind ⇄ JSON Schema, the OpenAI structured-output converter |
 | `@ai-matrx/content-ir/wire` | partial-kind events, runtime wrapper readers, emit payload |
+| `@ai-matrx/content-ir/directives` | Kind Directives: slug grammar, the ONE decoder |
+| `@ai-matrx/content-ir/source` | offset-preserving source tokenizer, splice save + anchor-mapping change set, the single-dollar math rule |
 
 ```ts
 import { createKindStreamParser, envelopeFromCompleteValue } from "@ai-matrx/content-ir/core";