@ai-matrx/content-ir 0.12.0 → 0.14.0

package/CHANGELOG.md

@@ -1,5 +1,124 @@
 # Changelog
 
+## 0.14.0 — 2026-09-25
+
+**`/source` — RC-B3 polish: the re-verification residuals, and the verifier's
+attacks made permanent** (common-docs
+`projects/rich-content-unification/evidence/verify-RC-B3.md` § Re-verification).
+Each residual had a failing test first (R1–R3 red on 0.13.0; the new
+cross-language vectors fail 10 cases on the old Python twin):
+
+- **R1** — an insertion carrying an exact copy of the next island at its
+  expected offset no longer causes a false `integrity` refusal: an island's
+  candidate positions are scored by how much of the original text still
+  surrounds them.
+- **R2** — `tokenizeSource(text, { streaming: true })`: the last,
+  still-unterminated line closes no fence (a bare ```` ``` ```` there may grow
+  into ```` ```bash ````). Stored text omits the option.
+- **R3** — `listIslands` loops instead of spreading, so a block with 150k
+  inline islands saves without a `RangeError`.
+- **R4** — the fence rule defines its whitespace: `FENCE_WHITESPACE`
+  (CommonMark ASCII whitespace) and `trimFenceLine`, which every caller uses
+  in place of a language's `trim()`. The vectors carry U+FEFF, U+0085,
+  U+00A0, U+001C–U+001F and U+2028 lines; aidream matrx-ai's Python twin
+  gained `FENCE_WHITESPACE` / `trim_fence_line` and passes them.
+- **R5** — tilde and odd-whitespace fences: matrx-frontend's
+  `source-covers-renderer` property test proves the tokenizer locks at least
+  every fenced block the renderer draws (both now share `trimFenceLine`).
+- **Permanent attack suite** — `__tests__/source-attack.test.ts`: seeded fuzz
+  of rejoin identity, offsets (independently counted), surrogate safety,
+  no-edit/identity saves, splice verdicts, island refusals, misaligned spans,
+  streaming prefixes (settled islands never move) and pathological timing
+  (budget + linear scaling), bounded in CI; `SOURCE_ATTACK=long` runs the
+  verifier-scale version.
+
+**Consumer action:** fence-line classification callers trim with
+`trimFenceLine(line)` (matrx-frontend's splitters do as of this release).
+
+**`/source` — islands for the extended markdown syntax** (RC-B8, common-docs
+`projects/rich-content-unification/REGISTER.md`). The renderer now draws
+callouts, directives, front matter, footnotes and wikilinks; an editor must
+never re-serialize their grammar, so each is now an island:
+
+- `directive` (block) — a generic-directive container `:::name[label]{attrs}`
+  … `:::` with nesting counted (tabs, columns, details, figures, callouts,
+  TOC), a leaf `::name` line (`meta.leaf`), or a MkDocs admonition
+  `!!! type "Title"` / `??? type` with its indented body (`meta.syntax: "mkdocs"`).
+  An unclosed container runs to the end with `complete: false` (streaming).
+- `callout` (block) — a blockquote opening `> [!TYPE]` (GitHub alert /
+  Obsidian callout; `meta.name`, `meta.fold`).
+- `footnote_def` (block) — `[^id]: …` with its continuation lines (`meta.id`).
+- `front_matter` now also recognizes TOML (`+++`); `meta.format` is `yaml` | `toml`.
+- `wikilink` (inline) — `[[Page]]`, `[[Page|alias]]`, `![[Page]]`
+  (`meta.target`, `meta.embed`); never across a line, never inside code.
+
+CSV/TSV fences were already `fence` islands (`meta.lang`). Guard:
+`__tests__/source-markdown-syntax-islands.test.ts` (7 of 10 red on 0.13.0).
+
+**Consumer action:** an exhaustive `switch` over `IslandType` /
+`BlockIslandType` gains the new members; nothing else changes (joinSource is
+still byte-identical, the corpus check stays green).
+
+**Single-dollar math: single-letter math shapes are math** (RC-B2 verification,
+common-docs `projects/rich-content-unification/evidence/verify-RC-B2.md`).
+`isSingleDollarMath` now also accepts a shape built only from single-letter
+identifiers (primes, standard function names), digits, parentheses, commas and
+`- * /` that applies, primes or juxtaposes them — `$f(g(x))$`, `$u v$`,
+`$f'(x)$`, `$(x, y)$`, `$ln(x)$`. Words keep it text (`$Tipping (fee)$`), and
+the pandoc spacing / digit rules still keep currency (`$5 and $10`, `$5-$10`)
+as text. Guard: `__tests__/single-dollar-math.test.ts` (5 of 12 red before).
+
+**Consumer action:** none — adopt the release; matrx-frontend's math
+normalizer and `/source` tokenizer pick the rule up through the import.
+
+## 0.13.0 — 2026-09-25
+
+**`/source` — fixes from the RC-B3 independent verification**
+(common-docs `projects/rich-content-unification/evidence/verify-RC-B3.md`).
+Every finding got a failing test first (`__tests__/source-verify-findings.test.ts`,
+14 of the 17 non-timing tests red on 0.12.0), then the fix:
+
+- **THE nested-fence rule now lives here** (`source/fence-nesting.ts`, moved from
+  matrx-frontend `components/markdown-core/fence-nesting.ts`): inside a
+  ```` ```markdown ```` / `md` / `mdx` fence, ```` ```lang ```` opens a nested
+  fence and a bare ```` ``` ```` closes the innermost one; a nested fence still
+  open at the end of the text retries strict CommonMark. The tokenizer uses it,
+  so it ends a nested markdown fence exactly where the renderer does (real row:
+  note `f9b7e9a2-…`). Generated cross-language vectors
+  `__tests__/fence-nesting-vectors.json`; aidream matrx-ai's Python twin runs a
+  byte-identical copy and fails on drift.
+- **Islands change only through `islandEdit(island, text)`.** A plain edit must
+  carry every island inside its range back byte-identical and in order —
+  otherwise `SourceSpliceError("island_edit")` names the island. Edits are
+  diffed segment by segment between islands, so anchors inside an edited
+  paragraph map exactly.
+- **Integrity is refused by default.** `requireIntegrity` defaults to `true`;
+  `false` is documented as tooling-only.
+- **Linear time on pathological input.** Every closer search goes through an
+  occurrence index; code spans use a per-paragraph run table; the single-`$`
+  scan stops at the rule's 400-character limit; embedded `__kind` probing has a
+  per-paragraph budget; `readXmlTag` takes a finder. 2 MB of `{{`: 21 s → 41 ms.
+- **A tree's root label belongs to the tree island** (the renderer splitter's
+  `findTreeBlockStart`).
+- **`$$` follows the core's pairing rule** (`source/math-pairs.ts`, ported from
+  `@ai-matrx/kit/delimiter-guard`): tokens outside code pair in order, prose
+  pairs are not math, an unpaired `$$` is literal. `$$` at the end of a line
+  pairs with a later closer, and a lone `$$` never locks the rest of a document.
+
+### Consumer action
+
+- `spliceSave` now THROWS on an island change (`island_edit`) and on disturbed
+  integrity (`integrity`). Editors change islands with `islandEdit`; tooling
+  that wants the report passes `requireIntegrity: false`.
+- matrx-frontend: import the fence rule from `@ai-matrx/content-ir/source`; the
+  local `components/markdown-core/fence-nesting.ts` is deleted.
+
+## 0.12.1
+
+Automatic changed-only republish (docs/metadata drift since the last tag — see
+`git diff npm/content-ir-core/v0.12.0..npm/content-ir-core/v0.12.1 -- apps/shared/content-ir-core`).
+No source changes intended and no consumer action required.
+
 ## 0.12.0 — 2026-09-23
 
 **`/source` — the stored text is the truth, block by block** (rich-content