@matrajs/mcp 1.0.0

package/docs/engine.md

@@ -0,0 +1,315 @@
+# The Matra engine
+
+Matra runs on its own engine, called the Matra engine the way Tiptap's is
+called ProseMirror and Lexical's is Lexical: the document model, content
+expressions, transforms, position mapping, editor state, history, and the
+editable view, all under `packages/core/src/engine/`. Nothing in it depends on
+another editor framework, and no type from it appears in a public signature.
+
+This file is its history and its notes. It began as a plan to remove
+ProseMirror layer by layer, and the plan is kept as written because the
+reasoning still holds.
+
+## Removing ProseMirror
+
+Decision: Matra owns its engine. ProseMirror is being replaced layer by layer,
+not ripped out — the public API leaks no engine type, so each layer can be
+swapped without users noticing.
+
+## Method
+
+Phase 1 was a true strangler: keymap, input rules, history and list commands
+each replaced a package and each dropped a dependency, with the suite green
+throughout.
+
+**Phases 2–5 cannot work that way.** The remaining packages are mutually coupled
+through the document model:
+
+    prosemirror-transform  →  prosemirror-model
+    prosemirror-state      →  prosemirror-model, transform, view
+    prosemirror-view       →  prosemirror-model, state, transform
+
+PM's transform builds and consumes PM `Node` instances, so our model cannot be
+handed to it. Model, transform, state and view therefore land together as a
+parallel engine and flip in one cutover.
+
+That means no dependency count moves until the whole thing is done, and the
+cutover is the risky moment rather than a series of small ones. To keep it
+honest:
+
+1. Build the parallel engine under `packages/core/src/engine/`.
+2. Test each layer directly, in isolation, as it is written.
+3. Before the flip, run the entire existing suite against the new engine behind
+   a switch — both engines pass, or the flip does not happen.
+4. Keep the ProseMirror view available behind a flag until the new view has
+   survived real users on iOS Safari and Android Chrome.
+
+## Phases
+
+| Phase | Layer | Lines | gz | Status |
+|---|---|---|---|---|
+| 1 | keymap, input rules, history, list commands | ~1,700 | 21 kB | **done** |
+| 2 | model — nodes, marks, fragments, schema, content expressions, DOM parse/serialize | ~3,500 | 30 kB | **done** |
+| 3 | transform — steps, position mapping, rebasing | ~2,200 | 19 kB | **done** |
+| 4 | state — transactions, selection, plugins | ~1,000 | 9 kB | **done** |
+| 5 | view — contenteditable, IME, selection sync | ~6,000 | 59 kB | **done** |
+
+## Phase 2 notes
+
+`engine/model/content-expression.ts` is done: a tokenizer and parser for the
+content language (`paragraph block*`, `(text | image)+`, `heading{1,3}`), an NFA
+compiler, and a subset construction to a DFA of `ContentMatch` states.
+
+`fillBefore` is the piece worth pointing at. When a match cannot legally end, it
+breadth-first searches for the shortest run of fillable types that would close
+it — which is how the editor repairs a document instead of refusing an edit. A
+type marked `fillable: false` is never used to repair, so a node that needs real
+attributes is never invented out of nothing.
+
+Written since: `mark.ts` (mark sets, rank ordering, exclusion), `fragment.ts`
+(immutable runs, text joining, cutting, boundary-correct `findIndex`),
+`node.ts` (sizes, classification, text extraction, descendant walking) and
+`schema.ts` (NodeType, content compilation, `createAndFill`).
+
+Two decisions worth remembering:
+
+- **Text nodes are canonicalised on construction.** Adjacent text carrying
+  identical marks is merged and empty text is dropped, so two documents that
+  mean the same thing compare equal.
+- **`createAndFill` returns null rather than guessing.** If closing a content
+  gap would need a node whose attributes have no defaults, the caller is told
+  the edit is impossible instead of receiving a malformed document.
+
+Phase 2 is complete: `resolved-pos.ts` (ancestor chains, neighbours, marks at a
+position, shared depth, block ranges) and the DOM layer.
+
+Two behaviours in the DOM layer are deliberate and worth keeping:
+
+- **An unrecognised element is transparent.** The parser descends into it rather
+  than dropping it, so pasting from a word processor keeps the text instead of
+  losing it to a `<div>` nobody wrote a rule for.
+- **Loose inline content gets wrapped.** Pasting bare text produces inline nodes
+  with no parent block; they are wrapped in the default textblock rather than
+  discarded, because discarding them loses the paste.
+
+Next: phase 3, transform — steps and position mapping.
+
+## Phase 3 notes
+
+`step-map.ts` is the crown jewel: flat `[start, oldSize, newSize]` triples, an
+`assoc` argument deciding which side of an insertion point a position lands on,
+and `deleted` reporting when a position was inside a span that no longer exists.
+
+It is fuzzed, not just sampled — 500 deterministic seeds asserting that mapping
+never moves a position backwards past an earlier one, that inverting returns
+every position outside a change exactly, and that a chain of maps equals
+applying them one at a time.
+
+**A property the fuzz forced us to state honestly:** a deletion collapses both
+edges of its span onto one point. Deleting `[5,6)` sends both 5 and 6 to 5, and
+inverting cannot know which it came from — that information is genuinely gone.
+Round-trip is exact only for positions strictly outside the changed span; on the
+boundary, `assoc` picks a side. The first version of the test asserted a
+stronger property than reality allows and had to be corrected, not the code.
+
+`step.ts` covers replace, addMark and removeMark, each able to invert itself.
+The replacement planner handles all four shapes a cross-block range can take —
+both ends inside blocks (the blocks join, which is what backspace at a boundary
+means), one end inside, or both on boundaries. Anything deeper than one level
+of nesting returns null so the step fails loudly rather than producing a
+malformed document.
+
+Rebasing is done. `Step.map` moves a step over changes made underneath it and
+returns null when there is nothing left to act on. One rule there is worth
+keeping: a step that meant *replace this text* whose text has since been
+deleted must not degrade into *insert this text here*. Without that check, a
+rebased AI rewrite pastes itself into a paragraph the user already deleted.
+
+## Phase 4 notes
+
+`selection.ts`, `transaction.ts`, `state.ts` and `plugin.ts`.
+
+- Selections snap to positions text can actually occupy, so nothing downstream
+  has to re-check. A NodeSelection whose node is deleted degrades to a caret
+  rather than pointing at nothing.
+- A transaction remaps its own selection after every step, so the caret stays
+  where the user would expect as the document moves under it.
+- Setting the selection clears stored marks: typing after moving the caret
+  should not inherit bold from somewhere else.
+- `state.apply` returns *the same state object* when a plugin vetoes, so callers
+  can compare by identity to know whether anything happened.
+
+## Phase 5 notes — and the cutover
+
+The view is built on `beforeinput` rather than mutation reconciliation. The
+browser announces what it is about to do, the view cancels it, applies the
+equivalent change to the model, and re-renders. The DOM is therefore a
+projection of the document rather than a second source of truth that has to be
+diffed back.
+
+Composition is the deliberate exception. While an IME candidate window is open
+the browser is left completely alone — cancelling input mid-composition breaks
+Japanese, Chinese and Korean entry outright — and the affected content is read
+back when composition ends.
+
+**One behavioural difference from ProseMirror:** the view takes over the element
+it is given rather than creating a child. `editor.mount(el)` makes `el` itself
+the editable surface.
+
+### Cutover, done
+
+    dependencies: {}
+
+All four ProseMirror packages are gone and the entire suite passes on our
+engine: 178 tests, unchanged in intent from when they ran against ProseMirror.
+That was the contract, and it held.
+
+Measured at the cutover:
+
+| | before | after |
+|---|---|---|
+| runtime dependencies | 9 | **0** |
+| `@matrajs/core` gzipped | 5.9 kB | 25.9 kB |
+| full app bundle gzipped | 66.4 kB | **18.4 kB** |
+
+The core package grew because the engine is now inside it. What matters to a
+user is the last row: an app ships a fraction of what it did, because nothing
+is pulled in that the editor does not use.
+
+Extensions have landed since, so that last number is not today's. The current
+figure is whatever `pnpm size` prints — **30 kB** for the starter kit as of
+1.0 — and it is checked in CI rather than quoted from here.
+
+## What 1.0 changed underneath
+
+Measured first, then changed; the numbers are in BENCHMARKS.md. The shape of
+each change, and why it is that shape:
+
+- **Fragments past twenty-four children keep a prefix index.** `findIndex`
+  bisects it, and `replaceChild` carries it across by shifting the tail rather
+  than rebuilding it. Resolving a position near the end of a long document
+  stopped costing the document.
+- **`nodesBetween` walks only what a range touches**, jumping to the first
+  child with the index and stopping at the last. `hasMark`, `removeMark`,
+  `setBlockType` and every extension that used to `descendants` its way
+  through the document to find a selected word use it.
+- **Mark steps rebuild locally.** A block level swaps only the run of children
+  that changed, via `Fragment.replaceRange`; a textblock is rebuilt in
+  canonical form because text merges. Untouched blocks stay the same object,
+  which is what lets the renderer skip them.
+- **The command context is a class with a lazy transaction.** `isActive`,
+  `can` and decoration hooks read the state and never start one.
+- **The transaction's selection is taken as it is.** It was mapped through the
+  whole mapping a second time in `EditorState.apply`, after the transaction
+  had already moved it step by step — so `insert('X')` left the caret one
+  past its own text. Found by a probe, pinned by a test.
+- **A block inserted at a caret inside a paragraph splits the paragraph.**
+  The rule button and the `---` shortcut both asked for that and were refused.
+- **Decorations are compared after mapping**, and the span where they differ
+  joins the span the edit touched. A textblock whose runs kept their shape has
+  its text nodes updated in place. A node decoration that moved rebuilds the
+  element it left as well as the one it reached.
+- **The parser compiles each selector once** and indexes rules by tag. Inside
+  `<pre>`, and inside any node that says `code`, whitespace is literal.
+- **Two editors from one extension array share the compiled schema**, the
+  parser, the serializer, the command table and the input rules.
+- **Position markers are held weakly**, and the mapping log is swept every
+  256 changes for what no live marker still needs.
+- **Attributes can be added to nodes another extension owns**, rendered onto
+  the element and read back on parse. `textAlign` now works on the paragraph
+  in the box; before, it silently did nothing unless you wrote your own.
+
+## What typing costs
+
+A keystroke is the operation everything else is measured against, and it took
+three rounds of profiling to stop it costing the length of the document. The
+current shape, and why each piece is that shape:
+
+- **The document is rebuilt around one child, not by cutting.** An edit inside a
+  paragraph rebuilds every ancestor between it and the root. Doing that by
+  cutting the ancestor's children in two and appending the replacement back into
+  the middle walks the whole run three times and re-adds every child's size to
+  reach a total that differs from the old one by exactly one child.
+  `Fragment.replaceChild` copies the array once and does the size arithmetic in
+  a subtraction. Text is the exception — text nodes merge with their neighbours,
+  so the canonical form still has to be rebuilt when either side is text.
+- **The diff asks before it touches the DOM.** `childNodes` is a live list, and
+  the patch loop used to index it for every child before deciding whether that
+  child was inside the edit at all. On two thousand blocks, 1999 of those reads
+  were thrown away.
+- **The position map reuses its entries.** Re-recording is what a patch does to
+  every node whose subtree it kept, and a fresh entry object per node per edit is
+  garbage generated to say what the old object already said.
+- **A full position-map backlog drops the backlog, not the document.** The map
+  absorbs each edit's mapping rather than rewriting every entry, and replays the
+  backlog when a cold entry is read. Past sixty-four pending edits the replay
+  costs more than saying where everything is again — which used to mean
+  rebuilding the whole document's DOM, at the cost of a rebuild every
+  sixty-fourth keystroke and the silent loss of every mounted node view's state.
+  The re-record happens after the patch, because before it the positions are
+  still in the coordinates the edit moved away from.
+
+Measured in Node against happy-dom, that takes a keystroke on a
+2000-paragraph document from 0.464 ms to 0.062 ms, and stops it tracking the
+document's length: 0.045 ms at 20 blocks against 0.062 ms at 2000. In a browser
+it is what put Matra ahead of Lexical on the row it used to lose.
+
+The first render is a different problem with a different answer. It is within
+about 15% of the floor — the cost of the browser creating the same elements with
+no editor involved — so there is very little of it that is ours to remove. What
+was ours: building into a document fragment and attaching it once rather than
+appending block by block into a live tree, skipping the mark stack for children
+that have no marks, and taking a direct path for the `[tag, 0]` shape most nodes
+render as. Together, 0.77 ms to 0.59 ms for two hundred blocks in Node.
+
+## What is deliberately not built yet
+
+Honesty about the gaps, since "no dependencies" can read as "complete":
+
+- ~~**Collaborative editing.**~~ Done in `@matrajs/collab`: an authority, step
+  exchange, rebasing of unsent work over remote edits, and remote cursors drawn
+  as decorations, each one mapped through local steps rather than clamped.
+- ~~**Node views.**~~ Done. A node type may declare `nodeView`, returning its
+  own DOM plus an optional `contentDOM` for children. `stopEvent` keeps the
+  editor's hands off interactions inside the view.
+
+  Node views forced a real fix underneath: the renderer used to call
+  `replaceChildren()` on every keystroke, which is O(document) per character and
+  would have destroyed a view's focus, scroll position and any half-finished
+  interaction. It now patches. Because nodes are immutable, an edit inside one
+  paragraph leaves every other paragraph as literally the same object, so
+  identity alone skips most of the tree. Inline content inside a textblock is
+  still rebuilt whole — it is small, and mark wrappers make its DOM shape
+  diverge from the fragment.
+- **Decorations.** No inline highlights or widgets independent of the document.
+- **Drag and drop**, and **tables**.
+- **Deep nesting in replace.** A cross-block range nested more than one level
+  deep returns null rather than guessing; it fails loudly, but it fails.
+
+## Where the risk actually is
+
+**Phase 3 is the correctness risk.** Position mapping is what makes a late AI
+edit land on the right words. A subtle bug there corrupts documents silently,
+which is the exact failure Matra is sold against. It needs property-based tests:
+invert-and-reapply round-trips, mapping associativity, and fuzzed step sequences
+compared against a reference implementation.
+
+**Phase 5 remains the schedule risk, and shipping it does not end that.** The
+view passes its tests in happy-dom, which is not a browser. IME composition for
+CJK input, Android GBoard's after-the-fact corrections, spellcheck and
+autocorrect mutating the DOM, and browser-specific selection bugs are found by
+real users on real devices, not by unit tests. Treat the current view as
+working-but-unproven until it has survived iOS Safari and Android Chrome, and
+expect a tail of fixes there rather than a clean finish.
+
+`harness/ime` is where that gets checked: a page to open on a real phone that
+watches the document and the screen for the moment they disagree, logs the
+composition events the browser actually sent, and walks a checklist of the cases
+that break editors. Deliberately manual — the value is in the keyboards a device
+farm does not have installed.
+
+## Rules while this is in progress
+
+- No ProseMirror type may enter a public signature. `types.ts` stays clean.
+- Every phase keeps the full suite green; no phase lands with skipped tests.
+- Bundle size is measured at each phase and recorded here, not estimated.

package/docs/index.json

@@ -0,0 +1,205 @@
+[
+  {
+    "slug": "readme",
+    "title": "Matra README",
+    "description": "What Matra is, every package, the extension API, and how it compares.",
+    "source": "README.md",
+    "file": "readme.md"
+  },
+  {
+    "slug": "engine",
+    "title": "The Matra engine",
+    "description": "How the engine is built, what typing costs, and what changed in each release.",
+    "source": "ENGINE.md",
+    "file": "engine.md"
+  },
+  {
+    "slug": "design",
+    "title": "API design",
+    "description": "The principles behind the API and what differs from Tiptap.",
+    "source": "DESIGN.md",
+    "file": "design.md"
+  },
+  {
+    "slug": "benchmarks-full",
+    "title": "Benchmarks",
+    "description": "Bundle size and speed, measured, with the method and the caveats.",
+    "source": "BENCHMARKS.md",
+    "file": "benchmarks-full.md"
+  },
+  {
+    "slug": "security",
+    "title": "Security",
+    "description": "What is treated as hostile and where the rendering gate is.",
+    "source": "SECURITY.md",
+    "file": "security.md"
+  },
+  {
+    "slug": "changelog",
+    "title": "Changelog",
+    "description": "What changed in each version.",
+    "source": "CHANGELOG.md",
+    "file": "changelog.md"
+  },
+  {
+    "slug": "contributing",
+    "title": "Contributing",
+    "description": "How to set up, what gets merged, house rules.",
+    "source": "CONTRIBUTING.md",
+    "file": "contributing.md"
+  },
+  {
+    "slug": "releasing",
+    "title": "Releasing",
+    "description": "How a release is cut.",
+    "source": "RELEASING.md",
+    "file": "releasing.md"
+  },
+  {
+    "slug": "ai",
+    "title": "AI",
+    "description": "Streaming edits that survive concurrent typing: the stream you supply, askAi, and what cancel, accept and reject each do.",
+    "source": "https://matrajs.com/docs/ai",
+    "file": "docs-ai.md"
+  },
+  {
+    "slug": "api",
+    "title": "API",
+    "description": "The editor object, the core commands, and the command context, in one page.",
+    "source": "https://matrajs.com/docs/api",
+    "file": "docs-api.md"
+  },
+  {
+    "slug": "benchmarks",
+    "title": "Benchmarks",
+    "description": "How the numbers on the landing page were measured, what they leave out, and how to run the harness yourself.",
+    "source": "https://matrajs.com/docs/benchmarks",
+    "file": "docs-benchmarks.md"
+  },
+  {
+    "slug": "collab",
+    "title": "Collaboration",
+    "description": "Step exchange, rebasing and remote cursors over a central authority — with no CRDT and no dependency.",
+    "source": "https://matrajs.com/docs/collab",
+    "file": "docs-collab.md"
+  },
+  {
+    "slug": "commands",
+    "title": "Commands",
+    "description": "Commands are functions that either change the document and return true, or change nothing and return false. How they compose, how they are typed, and why the boolean matters.",
+    "source": "https://matrajs.com/docs/commands",
+    "file": "docs-commands.md"
+  },
+  {
+    "slug": "document-model",
+    "title": "Document model",
+    "description": "Your document is plain JSON: a tree of nodes you can log, diff, store and send over the wire. What positions mean, what the schema refuses, and why it is immutable.",
+    "source": "https://matrajs.com/docs/document-model",
+    "file": "docs-document-model.md"
+  },
+  {
+    "slug": "extensions",
+    "title": "Writing an extension",
+    "description": "An extension is a plain object. Nodes, marks and behaviour, with command types inferred rather than declared.",
+    "source": "https://matrajs.com/docs/extensions",
+    "file": "docs-extensions.md"
+  },
+  {
+    "slug": "first-editor",
+    "title": "Your first editor",
+    "description": "A working editor in ten lines, a toolbar wired to real commands, and the four mistakes everybody makes on the first afternoon.",
+    "source": "https://matrajs.com/docs/first-editor",
+    "file": "docs-first-editor.md"
+  },
+  {
+    "slug": "frameworks",
+    "title": "Frameworks",
+    "description": "React, Vue, Svelte and Solid have bindings. Angular, Qwik and plain JavaScript need three calls, and here they are.",
+    "source": "https://matrajs.com/docs/frameworks",
+    "file": "docs-frameworks.md"
+  },
+  {
+    "slug": "docs",
+    "title": "Introduction",
+    "description": "Matra is a headless rich text editor framework. It gives you a document model, an extension API and a command system, and stays out of your interface.",
+    "source": "https://matrajs.com/docs/",
+    "file": "docs-index.md"
+  },
+  {
+    "slug": "installation",
+    "title": "Installation",
+    "description": "Which packages exist, which are MIT, and which need a subscription.",
+    "source": "https://matrajs.com/docs/installation",
+    "file": "docs-installation.md"
+  },
+  {
+    "slug": "mcp",
+    "title": "Docs for AI tools",
+    "description": "An MCP server that serves this documentation to Claude, Cursor, Codex and anything else that speaks the protocol. Zero dependencies, one command.",
+    "source": "https://matrajs.com/docs/mcp",
+    "file": "docs-mcp.md"
+  },
+  {
+    "slug": "position-mapping",
+    "title": "Position mapping",
+    "description": "A position is an integer into the document. The moment anything changes, that integer points somewhere else. Mapping is how a position survives an edit it did not expect.",
+    "source": "https://matrajs.com/docs/position-mapping",
+    "file": "docs-position-mapping.md"
+  },
+  {
+    "slug": "react",
+    "title": "React",
+    "description": "useEditor, useEditorState and EditorContent: a toolbar whose active states stay honest.",
+    "source": "https://matrajs.com/docs/react",
+    "file": "docs-react.md"
+  },
+  {
+    "slug": "recipes",
+    "title": "Recipes",
+    "description": "Autosave, read-only, Markdown, a table of contents, search highlighting, and the things people ask for on day one.",
+    "source": "https://matrajs.com/docs/recipes",
+    "file": "docs-recipes.md"
+  },
+  {
+    "slug": "shortcuts",
+    "title": "Keyboard shortcuts",
+    "description": "Every shortcut and input rule the starter kit binds.",
+    "source": "https://matrajs.com/docs/shortcuts",
+    "file": "docs-shortcuts.md"
+  },
+  {
+    "slug": "solid",
+    "title": "Solid",
+    "description": "createMatra: the editor, a ref, and a signal that re-runs only what reads it.",
+    "source": "https://matrajs.com/docs/solid",
+    "file": "docs-solid.md"
+  },
+  {
+    "slug": "styling",
+    "title": "Styling",
+    "description": "Matra ships no appearance. What you have to style, what comes with a stylesheet you can paste, and what is state rather than UI.",
+    "source": "https://matrajs.com/docs/styling",
+    "file": "docs-styling.md"
+  },
+  {
+    "slug": "svelte",
+    "title": "Svelte",
+    "description": "A use: action, the editor, and a store that keeps a toolbar honest. Svelte 4 and 5 from one package.",
+    "source": "https://matrajs.com/docs/svelte",
+    "file": "docs-svelte.md"
+  },
+  {
+    "slug": "versions",
+    "title": "Version history",
+    "description": "Snapshots, a real word-level diff between them, and restore as one undo step.",
+    "source": "https://matrajs.com/docs/versions",
+    "file": "docs-versions.md"
+  },
+  {
+    "slug": "vue",
+    "title": "Vue",
+    "description": "useEditor, useEditorState, useEditorFocus and EditorContent for Vue 3 — first-class rather than community-maintained.",
+    "source": "https://matrajs.com/docs/vue",
+    "file": "docs-vue.md"
+  }
+]