@jarenjs/md 0.34.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +520 -0
- package/dist/types/ast.d.ts +181 -0
- package/dist/types/bake.d.ts +61 -0
- package/dist/types/compiler.d.ts +141 -0
- package/dist/types/component/index.d.ts +101 -0
- package/dist/types/directives.d.ts +126 -0
- package/dist/types/entities.d.ts +40 -0
- package/dist/types/footnotes.d.ts +83 -0
- package/dist/types/frontmatter.d.ts +67 -0
- package/dist/types/html.d.ts +72 -0
- package/dist/types/index.d.ts +30 -0
- package/dist/types/loader.d.ts +84 -0
- package/dist/types/mdx.d.ts +45 -0
- package/dist/types/parser.d.ts +116 -0
- package/dist/types/plugins/highlight.d.ts +64 -0
- package/dist/types/plugins/index.d.ts +64 -0
- package/dist/types/plugins/mermaid.d.ts +12 -0
- package/dist/types/scanner.d.ts +240 -0
- package/dist/types/to-html.d.ts +104 -0
- package/dist/types/to-md.d.ts +23 -0
- package/dist/types/to-vnode.d.ts +161 -0
- package/dist/types/utils.d.ts +63 -0
- package/docs/LOADER.md +92 -0
- package/docs/MD-FORMAT.md +502 -0
- package/docs/PLUGINS.md +277 -0
- package/package.json +80 -0
- package/schemas/jaren-md-ast.schema.json +296 -0
- package/src/ast.js +346 -0
- package/src/bake.js +104 -0
- package/src/compiler.js +167 -0
- package/src/component/index.js +191 -0
- package/src/directives.js +371 -0
- package/src/entities.js +107 -0
- package/src/footnotes.js +180 -0
- package/src/frontmatter.js +947 -0
- package/src/html.js +281 -0
- package/src/index.js +76 -0
- package/src/loader.js +0 -0
- package/src/mdx.js +219 -0
- package/src/parser.js +1685 -0
- package/src/plugins/highlight.js +325 -0
- package/src/plugins/index.js +75 -0
- package/src/plugins/mermaid.js +14 -0
- package/src/scanner.js +832 -0
- package/src/to-html.js +425 -0
- package/src/to-md.js +396 -0
- package/src/to-vnode.js +766 -0
- package/src/utils.js +107 -0
- package/styles/md.css +238 -0
package/README.md
ADDED
|
@@ -0,0 +1,520 @@
|
|
|
1
|
+
# @jarenjs/md
|
|
2
|
+
|
|
3
|
+
Markdown + frontmatter as JSON documents — one package, **two clearly
|
|
4
|
+
separated layers**:
|
|
5
|
+
|
|
6
|
+
1. **The engine** (`@jarenjs/md`, part one). Where JTLT (in
|
|
7
|
+
[`@jarenjs/json`](../../packages/json)) turns JSON into text, the
|
|
8
|
+
engine is the inverse arrow: it parses Markdown — CommonMark core,
|
|
9
|
+
GFM tables, strikethrough, task lists, footnotes and autolink
|
|
10
|
+
literals, YAML/JSON/TOML frontmatter —
|
|
11
|
+
into a stable, serializable AST that the rest of the suite consumes
|
|
12
|
+
natively. JSLT stylesheets transform it, query documents address it,
|
|
13
|
+
[`@jarenjs/view`](../../packages/view) renders it (DOM and SSR), and
|
|
14
|
+
[`@jarenjs/app`](../../packages/app) loads it as a resource. The
|
|
15
|
+
engine is headless: it produces and prints *values* and never
|
|
16
|
+
touches a host.
|
|
17
|
+
2. **The visual component** (`@jarenjs/md/component` +
|
|
18
|
+
`@jarenjs/md/styles/md.css`, part two). The presentation layer that
|
|
19
|
+
drops the engine into a rendering host — above all an
|
|
20
|
+
`@jarenjs/app` document: a memoized, reference-stable `view()`
|
|
21
|
+
projection for viewModels, `md-load`/`md-parse` entries for the
|
|
22
|
+
effect registry, a `hydrate()` pass for browser-only plugin
|
|
23
|
+
upgrades, a default plugin set with syntax highlighting on, and the
|
|
24
|
+
component stylesheet. The [jarenjs website](../../packages/website)'s
|
|
25
|
+
playground *Markdown* tab is this component, live.
|
|
26
|
+
|
|
27
|
+
The layer boundary is a hard rule, not a convention: the engine never
|
|
28
|
+
imports from `src/component/` or ships CSS, and the component adds no
|
|
29
|
+
parsing semantics — it only packages engine output for hosts (the
|
|
30
|
+
rationale is spelled out in [ARCHITECTURE.md](ARCHITECTURE.md)). Zero
|
|
31
|
+
runtime dependencies outside the suite; no `eval`; the same
|
|
32
|
+
parse-once/compile-to-closures design as every other Jaren engine.
|
|
33
|
+
|
|
34
|
+
## The format in one glance
|
|
35
|
+
|
|
36
|
+
```js
|
|
37
|
+
import { parseMarkdown } from '@jarenjs/md';
|
|
38
|
+
|
|
39
|
+
parseMarkdown('---\ntitle: Hello\n---\n# Hi *there*');
|
|
40
|
+
// {
|
|
41
|
+
// "$md": "0.1",
|
|
42
|
+
// "frontmatter": { "title": "Hello" },
|
|
43
|
+
// "ast": [
|
|
44
|
+
// { "type": "heading", "depth": 1, "children": [
|
|
45
|
+
// { "type": "text", "value": "Hi " },
|
|
46
|
+
// { "type": "emphasis", "children": [{ "type": "text", "value": "there" }] }
|
|
47
|
+
// ] }
|
|
48
|
+
// ],
|
|
49
|
+
// "meta": { "sourceUrl": null, "hash": "8k41x2", "frontmatterLang": "yaml" }
|
|
50
|
+
// }
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
- Every node is a plain object with a `type` discriminator; containers
|
|
54
|
+
hold `children`, literals hold `value`.
|
|
55
|
+
- Frontmatter (`---` YAML subset, `---json`/leading-`{` JSON, `+++`
|
|
56
|
+
TOML) normalizes to plain JSON on the document.
|
|
57
|
+
- The whole document is JSON: stringify it, diff it with JSON Patch,
|
|
58
|
+
validate it against
|
|
59
|
+
[`schemas/jaren-md-ast.schema.json`](schemas/jaren-md-ast.schema.json),
|
|
60
|
+
transform it with JSLT, generate it under constrained decoding.
|
|
61
|
+
|
|
62
|
+
The normative contracts live in [docs/MD-FORMAT.md](docs/MD-FORMAT.md)
|
|
63
|
+
(AST + frontmatter), [docs/PLUGINS.md](docs/PLUGINS.md) (the
|
|
64
|
+
compile-time plugin system) and [docs/LOADER.md](docs/LOADER.md) (the
|
|
65
|
+
lazy URL loader).
|
|
66
|
+
|
|
67
|
+
## Usage
|
|
68
|
+
|
|
69
|
+
### Parse, print, project
|
|
70
|
+
|
|
71
|
+
```js
|
|
72
|
+
import { parseMarkdown, toMarkdown, mdToVnode, compileMarkdown } from '@jarenjs/md';
|
|
73
|
+
import { renderToString } from '@jarenjs/view';
|
|
74
|
+
|
|
75
|
+
const doc = parseMarkdown(source); // → MdDocument (plain JSON)
|
|
76
|
+
const canonical = toMarkdown(doc); // → canonical Markdown (round-trips)
|
|
77
|
+
const vnode = mdToVnode(doc); // → view vnode, content-hash keys
|
|
78
|
+
const html = renderToString(vnode); // → SSR string
|
|
79
|
+
|
|
80
|
+
// Or compile once and reuse the cached projections:
|
|
81
|
+
const md = compileMarkdown(source, { retainSource: false });
|
|
82
|
+
md.toVnode() === md.toVnode(); // true — built at most once
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
### An HTML string, in one call
|
|
86
|
+
|
|
87
|
+
```js
|
|
88
|
+
import { parseMarkdown, toHtml } from '@jarenjs/md';
|
|
89
|
+
|
|
90
|
+
toHtml(parseMarkdown(source)); // '<h1>Hi</h1><p>…</p>'
|
|
91
|
+
toHtml(doc, { wrap: 'article class="md"' }); // wrapped, like the vnode path
|
|
92
|
+
toHtml(doc, { html: 'raw' }); // TRUSTED INPUT ONLY
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
`toHtml` writes bytes; `mdToVnode` builds a patchable tree. Neither is
|
|
96
|
+
implemented in terms of the other, because their targets differ: a vnode
|
|
97
|
+
is keyed, memoized and reconciled in O(1) by `@jarenjs/view`, and its
|
|
98
|
+
safety is structural — there is no slot in it for unescaped author
|
|
99
|
+
markup. A string has one, which is why the raw-HTML corner of CommonMark
|
|
100
|
+
is reachable through `toHtml` and only through it.
|
|
101
|
+
|
|
102
|
+
The `html` option is the whole difference:
|
|
103
|
+
|
|
104
|
+
| mode | what a raw-HTML node becomes |
|
|
105
|
+
|---|---|
|
|
106
|
+
| `'escape'` (default) | escaped text — the markup is **visible**, not live |
|
|
107
|
+
| `'skip'` | dropped, as the vnode path drops it by default |
|
|
108
|
+
| `'raw'` | passed through verbatim — **trusted input only** |
|
|
109
|
+
|
|
110
|
+
The URL policy is orthogonal and runs in **every** mode: `'raw'` says
|
|
111
|
+
"this document's HTML blocks are trusted", not "trust everything", so a
|
|
112
|
+
markdown `[x](javascript:…)` still loses its `href`. No surface in this
|
|
113
|
+
repository passes `'raw'`.
|
|
114
|
+
|
|
115
|
+
For markup a vnode *can* express, the two emitters produce byte-identical
|
|
116
|
+
output — asserted over the CommonMark corpus in
|
|
117
|
+
[`test/md/to-html.test.js`](../../test/md/to-html.test.js), not on one
|
|
118
|
+
fixture.
|
|
119
|
+
|
|
120
|
+
### Heading anchors
|
|
121
|
+
|
|
122
|
+
`[see below](#the-section)` needs something to land on, so the emitter can
|
|
123
|
+
give every heading a GitHub-compatible `id` — the same slug GitHub mints,
|
|
124
|
+
so one committed README anchors identically on GitHub, in an editor
|
|
125
|
+
preview and wherever you render it:
|
|
126
|
+
|
|
127
|
+
```js
|
|
128
|
+
mdToVnode(doc, { headingIds: true });
|
|
129
|
+
// ['h2', { id: 'quick-start' }, 'Quick start']
|
|
130
|
+
|
|
131
|
+
mdToVnode(doc, { headingIds: true, headingAnchors: true });
|
|
132
|
+
// … plus a trailing ['a', { class: 'md-anchor', href: '#quick-start', … }, '#']
|
|
133
|
+
// so a reader can copy a link to the section
|
|
134
|
+
|
|
135
|
+
mdToVnode(doc, { headingIds: true, slugPrefix: 'user-content-' });
|
|
136
|
+
// every id and anchor href prefixed — set this for markdown you did not
|
|
137
|
+
// author, so its ids cannot collide with your own page's
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Repeated headings are numbered the way GitHub numbers them (`setup`,
|
|
141
|
+
`setup-1`, `setup-2`), and a heading with no slug-worthy text (`## ***`)
|
|
142
|
+
lands on `section`. Ids are **off by default on purpose**: CommonMark
|
|
143
|
+
renders a heading as `<h1>Foo</h1>`, so emitting one by default would put
|
|
144
|
+
the conformance score below at odds with what the package produces. The
|
|
145
|
+
rules are normative in [MD-FORMAT.md](docs/MD-FORMAT.md) §4.5; the slug
|
|
146
|
+
transform is `slugify` from `@jarenjs/core/string`.
|
|
147
|
+
|
|
148
|
+
The affordance styles itself from `styles/md.css` and stays quiet until
|
|
149
|
+
its heading is hovered or it takes focus. A page with a sticky header
|
|
150
|
+
sets `--md-scroll-margin` so a scrolled-to heading does not land beneath
|
|
151
|
+
it.
|
|
152
|
+
|
|
153
|
+
### Footnotes and bare links (GFM)
|
|
154
|
+
|
|
155
|
+
Both are on with `gfm` (the default) and both are **additive**: with
|
|
156
|
+
`gfm: false` the output is byte-for-byte what it was before they existed.
|
|
157
|
+
|
|
158
|
+
```md
|
|
159
|
+
A claim.[^1] Visit www.example.com or mail a@b.test.
|
|
160
|
+
|
|
161
|
+
[^1]: The source, which may hold [several](/blocks) blocks.
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
A footnote definition stays **where the author wrote it** in the AST —
|
|
165
|
+
the document is what was written, not what one renderer makes of it — and
|
|
166
|
+
the emitters collect it: references become `<sup>` links, uncited
|
|
167
|
+
definitions render nothing at all, and one `<section class="footnotes">`
|
|
168
|
+
is appended after the last block.
|
|
169
|
+
|
|
170
|
+
- **Numbering follows the first reference**, not definition order or
|
|
171
|
+
label order. Cite `[^b]` before `[^a]` and `b` is footnote 1.
|
|
172
|
+
- **An undefined `[^nope]` stays literal text**, exactly as on GitHub —
|
|
173
|
+
a citation of nothing is not a link to nothing.
|
|
174
|
+
- **Ids carry `user-content-` by default** (GitHub's own answer), so a
|
|
175
|
+
document dropped into a page you own cannot collide with its `#fn-1`.
|
|
176
|
+
`slugPrefix` replaces the prefix; `slugPrefix: ''` opts out.
|
|
177
|
+
- **A footnote cited twice gets two landing places** and two
|
|
178
|
+
back-references, so `↩` returns the reader where they left.
|
|
179
|
+
- **A footnote may cite another**, cycles included; the collection
|
|
180
|
+
terminates because each definition is rendered once.
|
|
181
|
+
- With `wrap: null` (a bare fragment) the section is appended **inside**
|
|
182
|
+
the fragment, after the last block — a consumer concatenating fragments
|
|
183
|
+
gets one footnotes section per fragment.
|
|
184
|
+
|
|
185
|
+
Literal autolinks follow GFM's extended grammar, trailing-punctuation
|
|
186
|
+
rules and all — `www.example.com/a.b.` links `www.example.com/a.b` and
|
|
187
|
+
leaves the sentence's full stop alone. The AST holds a plain `link` with
|
|
188
|
+
the scheme already inserted (`http://www.example.com/a.b`) plus an
|
|
189
|
+
`auto: true` flag, which exists for exactly one consumer: `toMarkdown`,
|
|
190
|
+
which prints it back bare instead of as `[text](url)`. Both features are
|
|
191
|
+
normative in [MD-FORMAT.md](docs/MD-FORMAT.md) §4.6 and §4.7.
|
|
192
|
+
|
|
193
|
+
### End to end: URL → frontmatter → JSLT → plugins → DOM + SSR
|
|
194
|
+
|
|
195
|
+
The full pipeline, runnable as-is in a browser module (swap the URL);
|
|
196
|
+
on the server, keep everything up to `renderToString`:
|
|
197
|
+
|
|
198
|
+
```js
|
|
199
|
+
import { loadMarkdown, createMdRenderer } from '@jarenjs/md';
|
|
200
|
+
import { highlightPlugin, mermaidPlugin } from '@jarenjs/md/plugins';
|
|
201
|
+
import { renderToString } from '@jarenjs/view';
|
|
202
|
+
import { compileJsltStylesheet } from '@jarenjs/json/jslt';
|
|
203
|
+
import { createTypeTestCompiler } from '@jarenjs/validate/query';
|
|
204
|
+
|
|
205
|
+
const plugins = [highlightPlugin(), mermaidPlugin()]; // native engine, no injected instance
|
|
206
|
+
|
|
207
|
+
// 1. Load + compile (cached by URL, AbortSignal-aware, streaming).
|
|
208
|
+
const md = await loadMarkdown('/docs/article.md', { plugins });
|
|
209
|
+
|
|
210
|
+
// 2. Frontmatter is plain JSON — and binds as JSLT externals.
|
|
211
|
+
const { title } = md.frontmatter;
|
|
212
|
+
|
|
213
|
+
// 3. A small JSLT transform of the AST: drop every h1.
|
|
214
|
+
const dropH1 = compileJsltStylesheet([
|
|
215
|
+
{ match: { path: '$.ast[*]', schema: { properties: { type: { const: 'heading' }, depth: { const: 1 } }, required: ['type', 'depth'] } },
|
|
216
|
+
body: null },
|
|
217
|
+
], { compileTypeTest: createTypeTestCompiler() }); // schema-match hook
|
|
218
|
+
const trimmed = dropH1(md.doc); // unmatched blocks stay ===
|
|
219
|
+
|
|
220
|
+
// 4. SSR: pure, deterministic (mermaid renders real inline SVG, no browser).
|
|
221
|
+
const ssr = renderToString(mdToVnode(trimmed, { plugins }));
|
|
222
|
+
|
|
223
|
+
// 5. Browser: mount and patch (mermaid's SVG is already complete — no hydrate).
|
|
224
|
+
const render = createMdRenderer({ container: document.getElementById('app'), plugins });
|
|
225
|
+
render(md);
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
`streamMarkdown(url)` yields completed top-level blocks while the body
|
|
229
|
+
is still downloading, and `createIncrementalParser()` is the same core
|
|
230
|
+
for non-URL sources — see [docs/LOADER.md](docs/LOADER.md).
|
|
231
|
+
|
|
232
|
+
### Plugins are compile-time
|
|
233
|
+
|
|
234
|
+
```js
|
|
235
|
+
import { definePlugin } from '@jarenjs/md';
|
|
236
|
+
|
|
237
|
+
const callout = definePlugin({
|
|
238
|
+
name: 'callout',
|
|
239
|
+
node: 'callout',
|
|
240
|
+
blocks: [{ chars: ':', start, continue: cont, close }],
|
|
241
|
+
render: (node, h) => h('aside', { class: 'md-callout' }, node.value),
|
|
242
|
+
});
|
|
243
|
+
parseMarkdown(source, { plugins: [callout] });
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
All extensibility — fence claims, block rules, inline rules, renderers,
|
|
247
|
+
hydrators — is declared up front and baked into dispatch tables; the
|
|
248
|
+
hot loops only do indexed lookups. Without a plugin, its syntax
|
|
249
|
+
degrades gracefully (a ` ```mermaid ` fence is just a `code` node). The
|
|
250
|
+
contract and both reference plugins are specified in
|
|
251
|
+
[docs/PLUGINS.md](docs/PLUGINS.md).
|
|
252
|
+
|
|
253
|
+
### The visual component (part two)
|
|
254
|
+
|
|
255
|
+
Everything an `@jarenjs/app` document needs, as one bundle:
|
|
256
|
+
|
|
257
|
+
```js
|
|
258
|
+
import { createApp } from '@jarenjs/app';
|
|
259
|
+
import { createMdComponent } from '@jarenjs/md/component';
|
|
260
|
+
import '@jarenjs/md/styles/md.css'; // .md rhythm + tok-* colors, light/dark
|
|
261
|
+
|
|
262
|
+
const md = createMdComponent(); // default plugins: syntax highlighting
|
|
263
|
+
|
|
264
|
+
createApp(appDoc, {
|
|
265
|
+
node,
|
|
266
|
+
effects: { ...md.effects }, // 'md-load' and 'md-parse'
|
|
267
|
+
viewModel: (state) => ({ ...state, article: md.view(state.doc) }),
|
|
268
|
+
});
|
|
269
|
+
|
|
270
|
+
// An action loads a document through the effect registry:
|
|
271
|
+
// { "effects": [{ "run": "md-load",
|
|
272
|
+
// "with": { "url": "$.url", "done": "article/loaded" } }] }
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
- `md.view(sourceOrDoc)` is **memoized and reference-stable**: the same
|
|
276
|
+
source string (or the same parsed document) returns the same vnode
|
|
277
|
+
reference, so the view patcher skips an unchanged article in O(1) —
|
|
278
|
+
the derivation contract `@jarenjs/app` viewModels rely on.
|
|
279
|
+
- `md.effects['md-load']` resolves any URL through the loader (cache,
|
|
280
|
+
abort, streaming) and dispatches the plain `MdDocument` as the
|
|
281
|
+
action payload; `md-parse` does the same for an in-state source
|
|
282
|
+
string. Failures route to an optional `error` action.
|
|
283
|
+
- `md.hydrate(container)` runs plugin `hydrate` hooks (browser-only
|
|
284
|
+
upgrades) over app-managed DOM, once per content hash. The bundled
|
|
285
|
+
plugins need none — mermaid renders complete SVG synchronously — so
|
|
286
|
+
this is a no-op until a hydrating third-party plugin is added.
|
|
287
|
+
- Pass `plugins` to extend the compiled-in set — e.g.
|
|
288
|
+
`createMdComponent({ plugins: [highlightPlugin(), mermaidPlugin()] })`.
|
|
289
|
+
|
|
290
|
+
### Forms and apps
|
|
291
|
+
|
|
292
|
+
- A document whose frontmatter declares a schema (`form:` or `$schema`)
|
|
293
|
+
feeds [`@jarenjs/forms`](../../packages/forms) through `mdToForm(doc, forms)` —
|
|
294
|
+
`{ schema, fields, data }`, with the forms module injected so this
|
|
295
|
+
package stays dependency-free.
|
|
296
|
+
- In an [`@jarenjs/app`](../../packages/app) document, a view can be a JSLT
|
|
297
|
+
stylesheet over a loaded MdDocument: register `loadMarkdown` as an
|
|
298
|
+
async effect that dispatches the plain `MdDocument` into the state,
|
|
299
|
+
and let the app's view stylesheet (`{"$apply": "$.doc.ast[*]"}`
|
|
300
|
+
rules, or simply `mdToVnode` inside a `viewModel` derivation) produce
|
|
301
|
+
the vnodes. Frontmatter members bind as externals via
|
|
302
|
+
`compiled.externals()`.
|
|
303
|
+
|
|
304
|
+
### mdx — markdown × data
|
|
305
|
+
|
|
306
|
+
`@jarenjs/md/mdx` renders a markdown TEMPLATE against a data document —
|
|
307
|
+
still a pure `(doc, data) → doc` pass over the parsed AST, so
|
|
308
|
+
`mdToVnode`, `toMarkdown` and the plugins all work unchanged on the
|
|
309
|
+
result. The template vocabulary reuses the suite's own query
|
|
310
|
+
expressions — no new mini-language:
|
|
311
|
+
|
|
312
|
+
```js
|
|
313
|
+
import { createMdx } from '@jarenjs/md/mdx';
|
|
314
|
+
import { compileJsonQuery } from '@jarenjs/json';
|
|
315
|
+
|
|
316
|
+
const mdx = createMdx({ compileQuery: compileJsonQuery });
|
|
317
|
+
const doc = mdx.transform(parseMarkdown(source), data);
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
- `{$.path}` inline in text interpolates a query expression over the
|
|
321
|
+
data (`$` is the data document; `$name` externals come from the
|
|
322
|
+
frontmatter — the same binding JSLT gets — and from `each` loops).
|
|
323
|
+
Code spans, code blocks and raw HTML never interpolate.
|
|
324
|
+
- A paragraph of exactly `{#if <expr>}` … `{/if}` keeps its section only
|
|
325
|
+
when the expression is truthy; `{#each <expr> as <name>}` … `{/each}`
|
|
326
|
+
repeats its section per item, binding `$<name>`. Sections nest, and a
|
|
327
|
+
directive must form its own paragraph (blank lines around it).
|
|
328
|
+
- The expression compiler is **injected** (`compileJsonQuery` from
|
|
329
|
+
[`@jarenjs/json`](../../packages/json)), so this package's engine layer
|
|
330
|
+
keeps its core+view-only dependency contract. A bad expression renders
|
|
331
|
+
its diagnosis in place — the pass never throws.
|
|
332
|
+
|
|
333
|
+
- `<!--mdx:$.path-->fallback<!--/mdx-->` is the **same expression through
|
|
334
|
+
the same evaluator**, carried in a comment. Use it in a document that
|
|
335
|
+
is also read raw; use `{$.path}` in one that is always rendered
|
|
336
|
+
dynamically, where it is the terser read. See Directives below.
|
|
337
|
+
|
|
338
|
+
Try it live: the `MDX` engine on
|
|
339
|
+
[Play](https://jklarenbeek.github.io/jarenjs/#/play) runs this pass over
|
|
340
|
+
an editable data pane.
|
|
341
|
+
|
|
342
|
+
### Directives — a number a machine derives and a human reads
|
|
343
|
+
|
|
344
|
+
```markdown
|
|
345
|
+
Jaren is <!--bm:jsonpath.ctsRatio-->23.1<!--/bm-->x faster on the CTS mean.
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
Every markdown renderer on earth drops HTML comments, so GitHub, an editor
|
|
349
|
+
preview and npm all show that line as one plain sentence with the figure in
|
|
350
|
+
it and no markers — static text, no runtime, correct today. A
|
|
351
|
+
directive-aware consumer reads the marker instead and can re-derive the
|
|
352
|
+
value:
|
|
353
|
+
|
|
354
|
+
```js
|
|
355
|
+
import { scanDirectives, replaceDirectives } from '@jarenjs/md/directives';
|
|
356
|
+
import { bake } from '@jarenjs/md';
|
|
357
|
+
|
|
358
|
+
scanDirectives(doc, { ns: 'bm' }); // → { directives, diagnostics }
|
|
359
|
+
replaceDirectives(doc, { ns: 'bm' }, …); // → a new doc, untouched subtrees ===
|
|
360
|
+
bake(source, { ns: 'bm', resolve }); // → { text, changed, diagnostics }
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
`bake` writes the fresh value **into the source**, which is what makes a
|
|
364
|
+
re-derivation a reviewable diff instead of a number that quietly stopped
|
|
365
|
+
being true. It splices only the spans between markers — a document with
|
|
366
|
+
no directives comes back byte-identical — because `toMarkdown` is a
|
|
367
|
+
canonicalizing printer and re-printing a hand-written README would reflow
|
|
368
|
+
every list for no reason. This repository's own figures work exactly this
|
|
369
|
+
way (`npm run docs:benchmarks`, `npm run docs:check`).
|
|
370
|
+
|
|
371
|
+
The layer never interprets the payload: `bm` puts a derivation key there,
|
|
372
|
+
`mdx` puts a query expression, and the vocabulary belongs to the
|
|
373
|
+
consumer. Unpaired markers, stray closers and same-namespace nesting come
|
|
374
|
+
back as **diagnostics** rather than being dropped — a marker nobody
|
|
375
|
+
matched is how a stale figure hides.
|
|
376
|
+
|
|
377
|
+
**One rule for authors: an inline marker must not begin a line.** A
|
|
378
|
+
comment at the start of a line opens a CommonMark HTML block, which eats
|
|
379
|
+
the rest of that line — the marker, its value and the prose after it.
|
|
380
|
+
That is CommonMark, not this package, and it bites on GitHub too. `bake`
|
|
381
|
+
reports it by name. (Six markers in this repository were written that way
|
|
382
|
+
and three sentences were disappearing from the rendered README; the gate
|
|
383
|
+
found them.)
|
|
384
|
+
|
|
385
|
+
`bake` is a build-time tool for input you control, and that is the one
|
|
386
|
+
place its trust level differs from mdx's: a baked body is spliced into
|
|
387
|
+
the source and **will** be re-parsed as markdown — a fact that is a whole
|
|
388
|
+
table is the point — whereas an mdx interpolation lands in a text node
|
|
389
|
+
and is never re-read. Normative in
|
|
390
|
+
[MD-FORMAT.md](docs/MD-FORMAT.md) §4.8.
|
|
391
|
+
|
|
392
|
+
### Untrusted Markdown
|
|
393
|
+
|
|
394
|
+
Two filters run when an AST becomes vnodes, on the same principle: the
|
|
395
|
+
vnode format has no unescaped output, so nothing authored reaches the page
|
|
396
|
+
as markup or as a live URL.
|
|
397
|
+
|
|
398
|
+
- **Raw HTML** is dropped. `options.html: 'text'` shows it as literal
|
|
399
|
+
text instead; `'vnode'` PARSES it through an allow-list
|
|
400
|
+
(`@jarenjs/md/html`), keeping the elements a document legitimately
|
|
401
|
+
uses — `<details>`, `<span class>`, `<img>`, tables — and dropping
|
|
402
|
+
everything else. What makes that offerable is structural rather than a
|
|
403
|
+
promise about filtering: the output is a vnode tree, so an
|
|
404
|
+
unrecognised element contributes only its children's text, `on*`
|
|
405
|
+
handlers and `style` never exist to begin with, `href`/`src` go
|
|
406
|
+
through the same URL policy as Markdown's own links, and a
|
|
407
|
+
`<script>`'s content is discarded rather than shown. It is not an
|
|
408
|
+
HTML5 parser — unbalanced input closes at the end of its run — and a
|
|
409
|
+
host with its own rules injects them through `options.parseHtml`.
|
|
410
|
+
- **Link and image URLs** whose scheme can execute (`javascript:`,
|
|
411
|
+
`vbscript:`) or stand in for a document (`file:`, `data:` other than a
|
|
412
|
+
raster image) lose their `href`/`src`; the element and its text stay, so
|
|
413
|
+
nothing the author wrote disappears. Relative references — `image.png`,
|
|
414
|
+
`docs/guide.md` — are not schemes and pass untouched. The AST keeps the
|
|
415
|
+
URL verbatim, so `toMarkdown` still round-trips it.
|
|
416
|
+
|
|
417
|
+
`options.sanitizeUrl` — `(url) => string | null` — replaces the URL policy
|
|
418
|
+
wholesale when a host needs a custom scheme in trusted content. It is the
|
|
419
|
+
whole guard, so widen it deliberately. Plugin `render` functions shadow the
|
|
420
|
+
core emitter and own the rule for URLs they emit; `ctx.sanitizeUrl` is the
|
|
421
|
+
active policy ([PLUGINS.md](docs/PLUGINS.md) §5, [MD-FORMAT.md](docs/MD-FORMAT.md) §4.3).
|
|
422
|
+
|
|
423
|
+
## Performance contract
|
|
424
|
+
|
|
425
|
+
Measured, not claimed — `npm run benchmark:markdown`, <!--bm:md.measured-->2026-08-11, Node v24.19.0<!--/bm-->
|
|
426
|
+
(run it yourself; micro-timings vary ±15%):
|
|
427
|
+
|
|
428
|
+
- **Parse to AST**: <!--bm:md.parseTimes-->~0.099 ms for a typical ~2 kB document, ~0.51 ms for ~10 kB, ~4.8 ms for ~100 kB<!--/bm--> — linear in input. A CPU profile puts the
|
|
429
|
+
inline phase at ~36% of that and the source hash for `meta.hash` at
|
|
430
|
+
~10%; the block scan, the obvious suspect, is ~14%. (Replacing one
|
|
431
|
+
`/\s+$/` regex at paragraph close with a scan was worth 4–17%
|
|
432
|
+
depending on how paragraph-dense the document is — measured as an A/B
|
|
433
|
+
on this corpus, because the same change looked like noise on a
|
|
434
|
+
differently shaped one.)
|
|
435
|
+
- **Parse + render to HTML** (the cross-engine row, `toHtml`): takes <!--bm:md.vsPeers-->0.6–1.1<!--/bm-->x the time `marked` and `markdown-it` take — <!--bm:md.vsPeersDetail-->faster than both at every size measured except one — `markdown-it` is ahead at ~10 kB (1.1x)<!--/bm--> — and is <!--bm:md.vsMicromark-->12.9–21.4<!--/bm-->x faster than `micromark`, on the same GFM documents.
|
|
436
|
+
Through the **vnode** path the same documents cost roughly twice that
|
|
437
|
+
— keys, memoization and a tree the patcher can reconcile are not free,
|
|
438
|
+
and the benchmark publishes that row beside this one rather than
|
|
439
|
+
quoting only the flattering half.
|
|
440
|
+
- **Where the time goes** (~100 kB, the phase split the benchmark now
|
|
441
|
+
prints and publishes): <!--bm:md.phaseSplit-->parse 35%, AST→vnode 50%, vnode→HTML 16%<!--/bm-->. The projection, not
|
|
442
|
+
the parse, is the expensive half — and <!--bm:md.keyCost-->53%<!--/bm--> of the projection is
|
|
443
|
+
computing the content-hash **keys** (<!--bm:md.unkeyedMs-->3.3 ms against 6.9 ms<!--/bm--> without them).
|
|
444
|
+
Keys are what let the patcher reorder blocks instead of rebuilding
|
|
445
|
+
them, so they are worth it for a tree that will be patched — and worth
|
|
446
|
+
nothing to a caller that renders once and throws the tree away. That
|
|
447
|
+
caller passes `keyed: false`:
|
|
448
|
+
|
|
449
|
+
```js
|
|
450
|
+
mdToVnode(doc, { keyed: false }); // SSR, a snapshot, a one-shot string
|
|
451
|
+
```
|
|
452
|
+
|
|
453
|
+
It is deliberately not inferred. A renderer cannot know whether its
|
|
454
|
+
output will be patched, and guessing wrong turns O(1) reconciliation
|
|
455
|
+
into a rebuild with no error to show for it.
|
|
456
|
+
- **The compiled fast path**: `compileMarkdown(...).toVnode()` returns
|
|
457
|
+
the cached projection in <!--bm:md.cachedNs-->39–82<!--/bm--> ns — and because block vnodes carry
|
|
458
|
+
content-hash keys and unchanged AST nodes emit reference-equal
|
|
459
|
+
vnodes, the view patcher skips unchanged blocks in O(1). A JSLT
|
|
460
|
+
identity transform returns the document by reference; a partial
|
|
461
|
+
transform keeps every unmatched subtree `===`. That pipeline — not
|
|
462
|
+
the one-shot HTML render — is what this package is optimized for.
|
|
463
|
+
- **CommonMark scorecard**, both paths, because the difference between
|
|
464
|
+
them *is* the safety boundary:
|
|
465
|
+
- `toHtml` (`html: 'raw'`, the like-for-like row): <!--bm:md.scorecard-->655 of 655 (100.0%)<!--/bm-->; for scale, <!--bm:md.scorecardPeers-->marked 620, markdown-it 655, micromark 650<!--/bm-->.
|
|
466
|
+
**No dialect gap remains on this path**: every example the spec
|
|
467
|
+
contains passes, and the round-trip suite additionally asserts that
|
|
468
|
+
all 655 survive `parseMarkdown → toMarkdown → parseMarkdown` with an
|
|
469
|
+
identical AST and an unchanged canonical form.
|
|
470
|
+
- `mdToVnode` + SSR: <!--bm:md.scorecardVnode-->593 of 655 (90.5%)<!--/bm-->. **Every** example the two
|
|
471
|
+
paths disagree on contains raw HTML — asserted, not asserted-at:
|
|
472
|
+
[`test/md/to-html.test.js`](../../test/md/to-html.test.js) checks that no
|
|
473
|
+
vnode-path failure is free of an `html` node. The spec renders raw
|
|
474
|
+
HTML verbatim, including a lone `</div>` or a never-closed tag, and a
|
|
475
|
+
vnode tree cannot hold half an element. That is a property of the
|
|
476
|
+
format, not a gap to close — it is the same property that makes the
|
|
477
|
+
vnode path safe for Markdown you did not write.
|
|
478
|
+
|
|
479
|
+
- **GFM extension scorecard**, the five extension sections of the GFM
|
|
480
|
+
specification with every engine's extensions switched on — because the
|
|
481
|
+
CommonMark corpus says nothing about any of them, and the part of the
|
|
482
|
+
dialect every engine advertises was the only part nobody measured: <!--bm:md.gfmScorecard-->22 of 24 (91.7%)<!--/bm--> through `toHtml`, and <!--bm:md.gfmScorecardVnode-->22 of 24 (91.7%)<!--/bm--> through the vnode
|
|
483
|
+
path; for scale, <!--bm:md.gfmPeers-->marked 22, markdown-it 14, micromark 23<!--/bm-->.
|
|
484
|
+
Autolink literals are <!--bm:md.gfmAutolinks-->11 of 11<!--/bm-->, ahead of every rival here. The two
|
|
485
|
+
this package does not pass are **stated boundaries, not to-do items**:
|
|
486
|
+
- **table alignment is written as `style="text-align:center"`, not the
|
|
487
|
+
deprecated `align` attribute** (1 example). Both render identically;
|
|
488
|
+
`align` was removed from HTML in 2014, and switching would change the
|
|
489
|
+
bytes every existing consumer already receives.
|
|
490
|
+
- **the "disallowed raw HTML" extension is not implemented** (1
|
|
491
|
+
example). It escapes the `<` of `<title>`, `<script>`, `<iframe>` and
|
|
492
|
+
six others when raw HTML passes through. Our raw mode is documented
|
|
493
|
+
trusted-input-only, and the two modes a host actually points at
|
|
494
|
+
untrusted Markdown — `escape` and the vnode path — already neutralize
|
|
495
|
+
those tags **and every other one**, which is a stronger guarantee
|
|
496
|
+
than a nine-tag deny-list. Implementing it would add a fourth
|
|
497
|
+
HTML policy that is safer than `raw` and weaker than the default.
|
|
498
|
+
|
|
499
|
+
Footnotes are not in this table because the GFM specification does not
|
|
500
|
+
cover them: GitHub ships them, the spec never grew a section for them,
|
|
501
|
+
so there is no reference corpus to score. They are covered by
|
|
502
|
+
hand-written tests written from GitHub's rendering
|
|
503
|
+
([`test/md/gfm.test.js`](../../test/md/gfm.test.js)).
|
|
504
|
+
|
|
505
|
+
Both scorecards run against their official spec as a git submodule,
|
|
506
|
+
QT3-style, and compare rendered meaning: whitespace that only lays
|
|
507
|
+
markup out — and the order in which a serializer happened to print a
|
|
508
|
+
tag's attributes — is normalized away on every engine's output, not
|
|
509
|
+
just this one's.
|
|
510
|
+
|
|
511
|
+
## Development
|
|
512
|
+
|
|
513
|
+
Tests live in the repository root: [`test/md/`](../../test/md)
|
|
514
|
+
(`npm run test:md`) — CommonMark-subset and GFM conformance, frontmatter,
|
|
515
|
+
plugins, structural sharing, streaming, loader caching, and AST-schema
|
|
516
|
+
validation through `@jarenjs/validate`. The benchmark methodology is
|
|
517
|
+
documented in [benchmark/README.md](../../benchmark/README.md). See the
|
|
518
|
+
repo [README](../../README.md) and [ROADMAP](../../docs/ROADMAP.md) for the
|
|
519
|
+
bigger picture, and [ARCHITECTURE.md](ARCHITECTURE.md) for the
|
|
520
|
+
internals.
|