@matrajs/mcp 1.0.0

package/docs/design.md

@@ -0,0 +1,107 @@
+# Matra — core API design
+
+Status: draft. Nothing published beyond reserved package names.
+
+## Principles
+
+1. **The engine never leaks.** No ProseMirror type appears in a public signature.
+   Raw access exists at `editor.unsafe`, is excluded from semver, and every use of
+   it is a bug report against this API.
+2. **Plain data, plain functions.** Definitions are object literals. Commands are
+   ordinary functions. No `this`, no classes, no `.extend()` inheritance chains.
+3. **Types are inferred, never declared twice.** Adding an extension to the array
+   adds its commands to `editor.commands` with full argument types. There is no
+   module augmentation step and no interface to keep in sync.
+4. **Async is a first-class problem.** Positions drift while an AI call is in
+   flight. The API makes that safe by default rather than leaving it to callers.
+
+## Three primitives
+
+```ts
+const heading = defineNode({
+  name: 'heading',
+  content: 'inline*',
+  group: 'block',
+  attrs: { level: { default: 1 } },
+  parseHTML: [{ tag: 'h1' }, { tag: 'h2' }, { tag: 'h3' }],
+  toDOM: (node) => [`h${node.attrs?.level}`, 0],
+  commands: {
+    setHeading: (ctx, level: 1 | 2 | 3) => ctx.setBlockType('heading', { level }),
+  },
+  keys: { 'Mod-Alt-1': 'setHeading' },
+})
+
+const bold = defineMark({
+  name: 'bold',
+  parseHTML: [{ tag: 'strong' }, { style: 'font-weight=bold' }],
+  toDOM: () => ['strong', 0],
+  commands: { toggleBold: (ctx) => ctx.toggleMark('bold') },
+  keys: { 'Mod-b': 'toggleBold' },
+})
+
+const editor = createEditor({ extensions: [heading, bold] })
+editor.commands.setHeading(2)   // typed
+editor.commands.setHeading(9)   // compile error
+```
+
+`defineExtension` is the third: no schema contribution, just commands, keys,
+state and lifecycle. Everything that isn't a node or a mark.
+
+## What differs from TipTap, and why
+
+| TipTap | Matra | Reason |
+|---|---|---|
+| `addCommands() { return { cmd: () => ({ commands }) => ... } }` | `commands: { cmd: (ctx, ...args) => boolean }` | Three levels of currying collapse to one function. Arguments get real types instead of being erased. |
+| `this.editor`, `this.options`, `this.storage` | everything passed as arguments | `this` is bound differently per hook and resists typing. Explicit arguments always work. |
+| `Extension.create().extend()` | plain objects, composed | Inheritance chains make it impossible to know what a definition finally contains. |
+| Commands merged into one global namespace | same, but collisions are a **compile error** | Two extensions declaring `toggleBold` should not silently shadow. |
+| `declare module` augmentation for types | inferred from the extensions array | The augmentation step is the most common source of broken types in TipTap projects. |
+| PM types in public API (`Node`, `Mark`, `EditorState`) | plain JSON `DocNode` | Swapping or upgrading the engine becomes possible without a breaking release. |
+
+## Async and position drift
+
+The hard problem in an AI editor: you send a paragraph to a model, the user keeps
+typing, the response arrives three seconds later, and every position you captured
+is now wrong. Written naively this corrupts documents.
+
+`ctx.mark()` takes a marker that maps positions through every intervening change:
+
+```ts
+const rewrite = defineExtension({
+  name: 'ai-rewrite',
+  commands: {
+    rewrite: (ctx) => {
+      const marker = ctx.mark()
+      const range = ctx.selection
+      const text = ctx.doc // read what we need now
+
+      void ai.rewrite(text).then((result) => {
+        // the user may have typed anywhere in the meantime
+        editor.commands.replaceRange(marker.mapRange(range), result)
+      })
+      return true
+    },
+  },
+})
+```
+
+This is why the AI layer belongs in core's design even though it ships as a
+separate package. Retrofitting position mapping later is not possible.
+
+## Open questions
+
+- **Collaboration.** Y.js maps positions through its own type. `PosMarker` and
+  Y.js relative positions need to be one concept, not two. Decide before 0.1.
+- **`batch()` rollback.** Rolling back when any command returns false is stated
+  in the types but interacts with input rules in ways not yet worked out.
+- **Node views.** Framework-specific by nature. Core should expose a renderer
+  interface that `@matrajs/vue` and `@matrajs/react` implement, but the shape of
+  that interface is undecided.
+- **Schema ordering.** ProseMirror's first node becomes the doc's default content.
+  Currently implicit via `priority`; may need to be explicit.
+
+## Verified
+
+`packages/core/src/types.test-d.ts` is a compile-time test. It asserts that valid
+calls typecheck and that wrong arity, wrong argument types, and unknown commands
+all fail. Run with `tsc --noEmit --strict`.

package/docs/docs-ai.md

@@ -0,0 +1,90 @@
+# AI
+
+Streaming edits that survive concurrent typing: the stream you supply, askAi, and what cancel, accept and reject each do.
+
+```
+npm i @matrajs/ai
+```
+
+`@matrajs/ai` streams a rewrite into the selected range and keeps that range correct while it arrives. Someone can carry on typing above, below or inside the selection and the streamed text still lands on the words they picked, because every chunk is applied to a **re-resolved** range rather than to the two numbers captured when the request started. That is [position mapping](https://matrajs.com/docs/position-mapping) doing the work, and it is the whole reason this package exists.
+
+> Note: This package is [commercial](https://matrajs.com/pricing). Free to evaluate, develop against, test with, teach with and use in personal projects · a subscription buys running it in production. Nothing phones home and there is no runtime licence check.
+
+## You supply the stream
+
+The extension never talks to a model. It takes an `AiStream` — any function returning an async iterable of strings — so the key stays on your server and the provider stays yours to change.
+
+```
+import { createEditor, starterKit } from '@matrajs/core'
+import { ai } from '@matrajs/ai'
+
+const editor = createEditor({
+  extensions: [
+    ...starterKit,
+    ai({
+      async *stream({ text, instruction, signal }) {
+        const response = await fetch('/api/rewrite', {
+          method: 'POST',
+          headers: { 'content-type': 'application/json' },
+          body: JSON.stringify({ text, instruction }),
+          signal,
+        })
+
+        const stream = response.body!.pipeThrough(new TextDecoderStream())
+        for await (const chunk of stream) yield chunk
+      },
+    }),
+  ] as const,
+})
+```
+
+Each chunk you yield replaces the range with *everything received so far*, so yield deltas rather than the whole answer again. `signal` aborts when the session is cancelled or the editor is destroyed · pass it to `fetch` and a cancelled rewrite stops costing tokens straight away.
+
+## Commands
+
+| Command | What it does |
+
+| `askAi(instruction)` | Starts a session over the current selection. Returns `false` on an empty selection, and `false` if a session is already streaming — one at a time. |
+
+| `cancelAi()` | Aborts the stream. **What already arrived stays in the document** · nothing further is applied. |
+
+| `acceptAi()` | Ends the session and keeps the result. The document is not touched. |
+
+| `rejectAi()` | Aborts and puts the selection back over the rewritten range. The original text comes back by undoing. |
+
+```
+editor.commands.select({ from, to })   // a rewrite needs a selection
+editor.commands.askAi('make this shorter')
+```
+
+> Note: Chunks are applied as they arrive, so the stream is several history steps rather than one. A reject followed by undo restores the paragraph · that is the path to wire to a *discard* button.
+
+## Status, for spinners and toasts
+
+`onStatus` is called on every transition, with the session as it stands.
+
+```
+ai({
+  stream,
+  onStatus: (session) => {
+    setBusy(session.status === 'streaming')
+    if (session.status === 'error') toast(session.error!.message)
+  },
+})
+```
+
+| Field | Type |  |
+
+| `id` | `number` | One per session. A late chunk from a superseded session is discarded. |
+
+| `status` | `'idle' | 'streaming' | 'done' | 'error' | 'cancelled'` |  |
+
+| `range` | `Range` | Where the rewrite is going, re-resolved as of now. |
+
+| `received` | `string` | Everything that has arrived so far. |
+
+| `error` | `Error?` | Set when `status` is `'error'`. |
+
+## What a failure does
+
+A stream that throws reports `'error'` with the thrown error, and stops. It does not throw out of the command and it does not tear the document up: whatever had already streamed stays, exactly as if the model had stopped early. A provider having a bad afternoon must not take an editor down with it.

package/docs/docs-api.md

@@ -0,0 +1,149 @@
+# API
+
+The editor object, the core commands, and the command context, in one page.
+
+Everything public, on one page. No engine type appears in any signature here · that is the contract, not a coincidence.
+
+## createEditor(options)
+
+| Option | Type |  |
+
+        {
+          options.map(([name, type, note]) => (
+
+|  |  |  |
+          ))
+        }
+
+## Editor
+
+| Member | Returns |  |
+
+        {
+          editorApi.map(([name, type, note]) => (
+
+|  |  |  |
+          ))
+        }
+
+## Core commands
+
+Always present, whatever extensions you loaded. Every one returns a boolean.
+
+| Command | Arguments |  |
+
+        {
+          coreCommands.map(([name, args, note]) => (
+
+|  |  |  |
+          ))
+        }
+
+> Note: A command never throws. Positions that are not finite integers inside the document return false · `NaN` included, which slips past naive range checks because both `NaN  size` are false.
+
+## Ctx · what a command receives
+
+        {
+          ctxApi.map(([name, note]) => (
+
+|  |  |
+          ))
+        }
+
+## What an extension may declare
+
+        commandsPlain functions of (ctx, …args) => boolean. Their types land on editor.commands and editor.can.
+        keysA binding to a command name or a function · . A binding that returns false lets the next one try.
+        inputRulesA RegExp on the text before the caret and a handler. One undo step each.
+        attributesAttributes added to nodes and marks defined elsewhere, rendered and parsed.
+        handlePaste · handleDropClaim a paste or a drop before the editor parses it. Return true to keep it.
+        filterChangeVeto a change before it lands. Nothing is applied, recorded or redrawn when it
+            returns false.
+        decorationsDraw over the document: inline attributes, node attributes, widgets. Never in the
+            document.
+        stateinit and apply, reduced on every transaction · read with editor.extensionState(name).
+        nodeViewsRender nodes defined elsewhere with your own DOM, keyed by node name.
+        onCreate · onChange · onDestroyLifecycle, each handed the editor. onCreate runs when the editor mounts.
+        priorityLoad order. Higher loads first; a later binding for the same key wins.
+        nodes · marksA node declares content, group, attrs, parseDOM, toDOM, nodeView, listItem, marks, code; a mark declares inclusive, excludes, parseDOM, toDOM.
+
+## Types worth knowing
+
+```
+type Pos = number & { readonly __pos: unique symbol }
+type Range = { from: Pos; to: Pos }
+
+interface DocNode {
+  type: string
+  attrs?: Record<string, unknown>
+  content?: DocNode[]
+  text?: string
+  marks?: DocMark[]
+}
+```
+
+`Pos` is branded so a raw number cannot be passed by accident. Cast at the boundary where you genuinely know the number is a position.
+
+## Writing a position down
+
+```
+import { pos, range } from '@matrajs/core'
+
+pos(0)         // number -> Pos
+range(1, 6)    // (number, number) -> Range
+```
+
+`Pos` is branded so that arithmetic on one does not typecheck · these are how you write a literal without a cast. Not for carrying a position across an `await`: that is ctx.mark().
+
+## Functions that need no editor
+
+```
+import { toMarkdown, fromMarkdown, tableOfContents, assignIds } from '@matrajs/core'
+
+toMarkdown(doc)          // DocNode  -> string
+fromMarkdown(source)     // string   -> DocNode
+tableOfContents(doc)     // DocNode  -> TocEntry[]
+assignIds(doc, options)  // DocNode  -> DocNode with stable block ids
+```
+
+None of these touches the DOM, so all four run on a server.
+
+    .api {
+      overflow-x: auto;
+      margin: 0 0 18px;
+    }
+    .api table {
+      border-collapse: collapse;
+      width: 100%;
+      font-size: 13.5px;
+    }
+    .api td,
+    .api th {
+      text-align: left;
+      padding: 9px 12px;
+      background-image: var(--dash-h);
+      background-size: 100% 1px;
+      background-position: 0 100%;
+      background-repeat: no-repeat;
+      vertical-align: top;
+    }
+    .api th {
+      font-family: var(--font-mono);
+      font-size: 10px;
+      letter-spacing: 0.08em;
+      text-transform: uppercase;
+      color: var(--ink-faint);
+      font-weight: 400;
+    }
+    .api td.n {
+      white-space: nowrap;
+      color: var(--ink);
+    }
+    .api td.t {
+      white-space: nowrap;
+      color: var(--indigo);
+      font-size: 12px;
+    }
+    .api td.d {
+      color: var(--ink-soft);
+    }

package/docs/docs-benchmarks.md

@@ -0,0 +1,104 @@
+# Benchmarks
+
+How the numbers on the landing page were measured, what they leave out, and how to run the harness yourself.
+
+Every figure on the landing page comes from one run of one harness with all four editors mounted in the same page. Numbers gathered from different browsers do not belong in the same table, so they are not gathered that way.
+
+## Method
+
+- WebKit, one page, one run · all four editors mounted side by side.
+
+- Median of seven samples, after five warm-up rounds.
+
+- A forced layout read every round, so the DOM work is really done.
+
+- Each published figure is the median of three such runs.
+
+That is not ceremony. A single sample moved one metric by 6× between runs on a change that could not affect it, and the keystroke row swings by about 30% between runs for every editor here. One sample is a story, not a measurement.
+
+## Milliseconds, lower is better
+
+| Operation | No editor | Matra | Tiptap | Lexical | Slate |
+
+        {
+          rows.map(([op, floor, a, b, c, d]) => (
+
+|  |  |  |  |  |  |
+          ))
+        }
+
+## These numbers are slower than the ones that were here before
+
+The machine is why. Same harness, same browser, a different day: Lexical parsed the same document in 34.7 ms one week and 76.9 ms the next, without a line of it changing. Only the comparison inside a single run means anything, which is exactly why all four editors are mounted in one page and measured in one pass. Milliseconds from two different runs are two different measurements wearing the same unit.
+
+## The keystroke row changed hands
+
+It used to be the row Matra lost, and losing it was fair: Lexical was ahead in every run. Measured back to back against the same rivals in the same session, before and after:
+
+| One keystroke | Before | After |
+
+        {
+          keystroke.map(([size, before, after]) => (
+
+|  |  | **** |
+          ))
+        }
+
+Three things were costing it, all three found by profiling rather than by reading:
+
+- **Every ancestor of an edit was rebuilt by cutting.** A paragraph changes, and each ancestor up to the document was rebuilt around it — cut the run of children in two, append the replacement, append the rest, then add up every child's size to arrive at a total that differed from the old one by exactly one child. On two thousand blocks that is four walks of two thousand children per character. Swapping the one child that moved makes it one array copy and one subtraction.
+
+- **The diff reached into the DOM for blocks it had already decided to skip.** It read `childNodes[i]` before asking whether block `i` was inside the edit at all, and for 1999 of 2000 blocks threw the answer away. Ask first, index second.
+
+- **Every sixty-fourth keystroke threw the rendered document away.** The position map absorbs each edit rather than rewriting itself, and capped its backlog at sixty-four — past which the whole document was rebuilt from scratch. It was the backlog that had gone stale, not the DOM. That one was a correctness bug in a performance bug's clothing: a rebuild also dropped the state of every mounted node view.
+
+Away from the browser's layout cost, in Node against happy-dom, the same three changes take a keystroke on two thousand paragraphs from 0.464 ms to 0.077 ms — and the cost stops tracking the length of the document: 0.052 ms at twenty blocks against 0.077 ms at two thousand.
+
+## The "No editor" column
+
+The same paragraphs, built by hand into a `contenteditable` div, with no editor in the page at all. Putting a document on screen is mostly the browser making elements and laying them out, and nothing can go faster than that. Two thirds of the 200-paragraph mount row and half of the 2,000-paragraph one is that floor, which is worth knowing before reading a 0.3 ms difference as a result.
+
+Every mount is also checked rather than trusted: the harness mounts once more outside the timing and looks for the last paragraph's text on screen. A reconciler that returns before its DOM exists is the cheapest possible way to win this row, and an earlier version of the harness reported Slate's mount as a number when nothing had been drawn at all.
+
+## The mount row was wrong, and the harness was why
+
+For a while this page said Lexical put an editor on screen faster than Matra. It did not. The harness ran each editor's teardown *inside* the timed function, and the layout read that makes a render measurement real came after it — so an editor whose teardown detaches its DOM had already taken its document off screen before the browser was asked to lay anything out. Lexical's teardown does that. Matra's does not: it drops listeners and leaves the document where it is.
+
+So Matra paid for laying out two thousand paragraphs and Lexical did not, on a row where the layout is most of the number. Teardown now runs after the clock stops, every editor pays for its own layout, and both mount rows changed hands. It is the same class of mistake the harness already refused to make for Slate, made one level up — which is the argument for a floor row and a verification pass rather than for trusting a number because it came out of a timer.
+
+## What is missing, and why
+
+Slate is measured on mount only. Its keystroke goes through a React render that has not happened by the time the timer stops, and the harness checks whether the text on screen changed during the measurement — when it did not, it reports `NOT MEASURED` instead of a number. An earlier version of the harness cheerfully reported Slate at 0.02 ms per keystroke, which was the model update with nothing drawn behind it.
+
+Each editor is driven through its own idiomatic API. That is the only fair way to do it and also the caveat worth stating: Lexical and Slate carry the rich-text behaviour their own quick-starts prescribe, which is not the same feature set as either starter kit.
+
+## Bundle size
+
+One app importing that editor and its rich-text kit, bundled and minified with esbuild, then gzipped. Same method for all four.
+
+|  | Gzipped | What is in it |
+
+        {
+          sizes.map(([name, size, note]) => (
+
+|  |  |  |
+          ))
+        }
+
+## Run it yourself
+
+The harness is in the repository, under `bench/browser`.
+
+```
+# from a scratch directory
+npm i @tiptap/core @tiptap/starter-kit @tiptap/pm \
+      lexical @lexical/rich-text @lexical/html \
+      slate slate-react slate-dom slate-history react react-dom esbuild
+cp -r matra/packages/core/dist matra-dist
+cp matra/bench/browser/{bench.src.js,index.html} .
+npx esbuild bench.src.js --bundle --format=esm --outfile=bench.js \
+  --define:process.env.NODE_ENV='"production"'
+python3 -m http.server 8899
+```
+
+Reload two or three times and take the median of the runs as well. If you get different numbers, they are your numbers · that is rather the point of shipping the harness.

package/docs/docs-collab.md

@@ -0,0 +1,128 @@
+# Collaboration
+
+Step exchange, rebasing and remote cursors over a central authority — with no CRDT and no dependency.
+
+```
+npm i @matrajs/collab
+```
+
+The protocol is the well-trodden one. A client sends the steps it has made together with the version they applied to; the authority accepts them only if that version is still current. A client whose version is stale pulls what it missed, rebases its own unsent work over it, and tries again. Neither side loses an edit.
+
+There is no CRDT here and no dependency. Rebasing already lives in the engine — mapping a step over another is what lets a local edit survive a remote one — so collaboration is a version counter and a transport on top of that.
+
+> Note: This package is [commercial](https://matrajs.com/pricing). Free to evaluate, develop against, test with, teach with and use in personal projects · a subscription buys running it in production. Nothing phones home and there is no runtime licence check.
+
+## The client
+
+```
+import { createEditor, starterKit } from '@matrajs/core'
+import { collab } from '@matrajs/collab'
+
+const editor = createEditor({
+  extensions: [...starterKit, collab({ clientId: 'nahim-1a2b' })] as const,
+})
+```
+
+| Option | Type |  |
+
+| `clientId` | `string` |
+
+| `version` | `number?` | The version this client starts from. Defaults to `0`. |
+
+## The authority
+
+`Authority` is a plain class with no server attached, so the same object works over a WebSocket, over HTTP polling, in a Durable Object, or in an in-memory channel in a test.
+
+```
+import { Authority } from '@matrajs/collab'
+
+const authority = new Authority((version) => broadcast(version))
+```
+
+| Member |  |
+
+| `version` | How many steps it holds. |
+
+| `receive(version, steps)` | Appends if the client is current, and returns whether it was. `false` means *pull, rebase, try again*. Rejecting rather than merging is what keeps the history linear and every client's version meaningful. |
+
+| `since(version)` | Everything that happened after that version.   ## The loop Two functions, and they are the whole integration.
+
+```
+import { getVersion, sendableSteps } from '@matrajs/collab'
+
+/** Send what the authority has not seen. */
+function push() {
+  const sendable = sendableSteps(editor)
+  if (!sendable) return true
+
+  const accepted = authority.receive(sendable.version, sendable.steps)
+  if (accepted) editor.commands.confirmCollabSteps(sendable.steps.length)
+  return accepted
+}
+
+/** Take what we missed and rebase onto it. */
+function pull() {
+  const missing = authority.since(getVersion(editor))
+  if (missing.length) editor.commands.receiveCollabSteps(missing)
+}
+
+editor.on('change', () => {
+  if (!push()) {
+    pull()
+    push()
+  }
+})
+```
+
+  Export |  |
+
+| `sendableSteps(editor)` | ``, or `null` when there is nothing outstanding. |
+
+| `getVersion(editor)` | The version this client believes it is on.  `receiveCollabSteps(steps)` | Applies other clients' steps. Your own are skipped — they are already in the document, and applying them twice would duplicate the edit. |
+
+| `confirmCollabSteps(count)` | The authority took this many · stop tracking them as unconfirmed. |
+
+> Note: A step that no longer applies is **dropped rather than thrown**. One malformed message from a peer must not take an editor down, and a hostile one must not be able to try.
+
+## Remote cursors
+
+A second extension, because presence is optional and costs a little to keep.
+
+```
+import { colorFor, remoteCursorCSS, remoteCursors } from '@matrajs/collab'
+
+const editor = createEditor({
+  extensions: [
+    ...starterKit,
+    collab({ clientId }),
+    remoteCursors(),
+  ] as const,
+})
+
+editor.on('selectionChange', () => {
+  const { from, to } = editor.selection
+  socket.send(JSON.stringify({
+    clientId,
+    anchor: from,
+    head: to,
+    meta: { name: 'Nahim' },
+  }))
+})
+
+socket.onmessage = (event) => editor.commands.setPresence(JSON.parse(event.data))
+```
+
+| Command or export |  |
+
+| `setPresence(presence)` | `` `removePresence(clientId)` | Someone left. |
+
+| `clearPresence()` | Everyone left · a reconnect, usually. | `colorFor(clientId)` | A stable colour derived from the id, at a fixed saturation and lightness so it stays legible on light and dark. Derived beats assigned: two clients that never speak still agree on what colour a third person is. |
+      remoteCursorCSSThe stylesheet the decorations expect. See Styling.
+
+Read the cursors for an avatar row:
+
+```
+import type { RemoteCursors } from '@matrajs/collab'
+
+const people = editor.extensionState<RemoteCursors>('remoteCursors')
+```