@orkestrel/scaffold 0.0.119 → 0.0.121
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/bin/main.js +19 -19
- package/dist/bin/main.js.map +1 -1
- package/dist/host/agents/skills/orkestrel-journey/references/layer.md +10 -10
- package/dist/host/agents/skills/orkestrel-journey/references/statechart.md +1 -1
- package/dist/host/agents/skills/orkestrel-journey/references/styles.md +3 -3
- package/dist/host/claude/agents/orkestrel.md +52 -51
- package/dist/host/claude/rules/tests.md +2 -2
- package/dist/host/claude/rules/typescript.md +2 -3
- package/dist/host/guides/abort.md +0 -4
- package/dist/host/guides/agent.md +5 -5
- package/dist/host/guides/brief.md +7 -15
- package/dist/host/guides/browser.md +1 -1
- package/dist/host/guides/budget.md +1 -5
- package/dist/host/guides/capture.md +2 -2
- package/dist/host/guides/codec.md +18 -3
- package/dist/host/guides/console.md +4 -30
- package/dist/host/guides/contract.md +23 -44
- package/dist/host/guides/csv.md +86 -93
- package/dist/host/guides/database.md +0 -4
- package/dist/host/guides/emitter.md +0 -2
- package/dist/host/guides/form.md +26 -47
- package/dist/host/guides/guide.md +0 -1
- package/dist/host/guides/html.md +3 -7
- package/dist/host/guides/indexeddb.md +28 -32
- package/dist/host/guides/interpret.md +165 -182
- package/dist/host/guides/jsonrpc.md +187 -0
- package/dist/host/guides/lsp.md +26 -51
- package/dist/host/guides/markdown.md +132 -141
- package/dist/host/guides/mcp.md +220 -276
- package/dist/host/guides/middleware.md +14 -22
- package/dist/host/guides/msg.md +47 -63
- package/dist/host/guides/ndjson.md +0 -4
- package/dist/host/guides/ollama.md +1 -3
- package/dist/host/guides/pool.md +0 -4
- package/dist/host/guides/probe.md +71 -80
- package/dist/host/guides/process.md +71 -79
- package/dist/host/guides/program.md +26 -35
- package/dist/host/guides/qualifier.md +0 -6
- package/dist/host/guides/queue.md +0 -4
- package/dist/host/guides/rater.md +6 -12
- package/dist/host/guides/reason.md +30 -36
- package/dist/host/guides/relation.md +0 -2
- package/dist/host/guides/router.md +11 -18
- package/dist/host/guides/scaffold.md +148 -144
- package/dist/host/guides/sea.md +47 -51
- package/dist/host/guides/server.md +15 -19
- package/dist/host/guides/sqlite.md +0 -4
- package/dist/host/guides/sse.md +6 -10
- package/dist/host/guides/table.md +481 -493
- package/dist/host/guides/template.md +0 -4
- package/dist/host/guides/terminal.md +0 -26
- package/dist/host/guides/test.md +46 -66
- package/dist/host/guides/timeout.md +0 -6
- package/dist/host/guides/tool.md +2 -6
- package/dist/host/guides/toolbox.md +0 -10
- package/dist/host/guides/websocket.md +0 -4
- package/dist/host/guides/worker.md +57 -57
- package/dist/host/guides/workflow.md +13 -26
- package/dist/host/guides/workspace.md +15 -21
- package/dist/host/manifest.json +64 -909
- package/dist/src/core/index.cjs +128 -181
- package/dist/src/core/index.cjs.map +1 -1
- package/dist/src/core/index.d.cts +213 -213
- package/dist/src/core/index.d.ts +213 -213
- package/dist/src/core/index.js +117 -170
- package/dist/src/core/index.js.map +1 -1
- package/dist/src/server/index.cjs +35 -34
- package/dist/src/server/index.cjs.map +1 -1
- package/dist/src/server/index.d.cts +62 -58
- package/dist/src/server/index.d.ts +62 -58
- package/dist/src/server/index.js +36 -35
- package/dist/src/server/index.js.map +1 -1
- package/package.json +13 -13
|
@@ -5,9 +5,9 @@
|
|
|
5
5
|
> rewrites, folds, and streams that AST, and standalone projections that carry it out to
|
|
6
6
|
> sanitized HTML or canonical markdown source and carry an HTML AST back in.
|
|
7
7
|
|
|
8
|
-
One parse is the whole contract: every later output is a projection of the AST it produced, never a second read of the source. `
|
|
8
|
+
One parse is the whole contract: every later output is a projection of the AST it produced, never a second read of the source. `parseMarkdown` runs a block phase (headings / paragraphs / lists / GFM tables / fenced code / blockquotes / thematic breaks) then an inline phase (emphasis / inline code / links / images / hard breaks) over each block's text, and returns a render-agnostic `MarkdownDocument` — a discriminated union of node values keyed by `element` (the axis that varies: never `kind` / `type`). The AST itself is the primary contract — render-agnostic and exhaustively testable — with a from-unknown validation surface (`isInlineNode` / `isBlockNode` / `isMarkdownNode` / `isMarkdownDocument`) for when an AST arrives from outside `parseMarkdown` (a deserialized document, a value crossing a process/RPC boundary). Source: [`src/core`](../src/core). Surfaced through the `@src/core` barrel.
|
|
9
9
|
|
|
10
|
-
**Each conversion direction lives here**, because what an HTML subtree becomes in markdown — and what a markdown node becomes in HTML — is markdown-format knowledge, not HTML knowledge. `@orkestrel/html` owns the HTML AST, its total parser, its canonical serializer, and its sanitize floor; this package owns the two projections across the boundary and never asks html to know a markdown word. Outbound: `markdownToHTML` projects a `MarkdownNode` onto html's AST, `
|
|
10
|
+
**Each conversion direction lives here**, because what an HTML subtree becomes in markdown — and what a markdown node becomes in HTML — is markdown-format knowledge, not HTML knowledge. `@orkestrel/html` owns the HTML AST, its total parser, its canonical serializer, and its sanitize floor; this package owns the two projections across the boundary and never asks html to know a markdown word. Outbound: `markdownToHTML` projects a `MarkdownNode` onto html's AST, `renderMarkdownHTML` composes that projection with html's sanitizer and serializer into one sanitized string, and `renderMarkdown` writes canonical markdown source instead (§ [`renderMarkdown` round-trip](#rendermarkdown-round-trip)). Inbound: `htmlToMarkdown` folds an html `HTMLNode` back down to a `MarkdownDocument` (§ [`htmlToMarkdown` projection](#htmltomarkdown-projection)). None of them assumes its input came from a trusted parse, and none of them throws: malformed markdown degrades to literal text, while at the outbound depth cap value-bearing nodes degrade to text and structural nodes degrade to nothing; the inbound trip inherits html's own cap rather than exhausting the call stack (no ReDoS, no stack overflow).
|
|
11
11
|
|
|
12
12
|
## Surface
|
|
13
13
|
|
|
@@ -15,8 +15,6 @@ One parse is the whole contract: every later output is a projection of the AST i
|
|
|
15
15
|
|
|
16
16
|
The full node shape and workspace contract, from [`types.ts`](../src/core/types.ts). `element` is the discriminant every node carries; block nodes carry document structure, inline nodes carry the inline content of a heading / paragraph / list item / table cell. `MarkdownInterface`'s call-signature members are documented under [`## Methods`](#methods).
|
|
17
17
|
|
|
18
|
-
A `Shape` cell holds an interface's data members as bare names in braces, `?` marking an optional member and `plus` introducing its call-signature members, and a type alias's own type literal with a union's arms escaped as `\|`.
|
|
19
|
-
|
|
20
18
|
| Name | Kind | Signature | Summary |
|
|
21
19
|
| --------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
22
20
|
| `TableAlign` | type | `'left' \| 'right' \| 'center'` | Names the horizontal alignment of a GFM table column, as declared by its delimiter row (`:---` left, `---:` right, `:---:` center). A bare `---` delimiter is represented by `null` in `TableNode.align`: the positional array requires one entry per column, JSON cannot carry `undefined` in an array, and the bare delimiter is an explicit no-alignment marker rather than an omitted value. |
|
|
@@ -33,13 +31,13 @@ A `Shape` cell holds an interface's data members as bare names in braces, `?` ma
|
|
|
33
31
|
| `MarkdownSpan` | interface | `{ start, end }` | Addresses a half-open region of the original markdown string, in UTF-16 code units — `start` inclusive, `end` exclusive. The provenance a parse records for a node and `MarkdownInterface.span` reads back. |
|
|
34
32
|
| `MarkdownSegment` | interface | `{ offset, start, end }` | Maps one run of a `MarkdownSource` back to the region of the original markdown string it was taken from. |
|
|
35
33
|
| `MarkdownSource` | interface | `{ text, segments }` | Pairs a piece of derived markdown text with the runs mapping it back to the original string — what `splitLines` returns per line, so every phase downstream of it keeps original coordinates instead of reconstructing them from node values. |
|
|
36
|
-
| `
|
|
34
|
+
| `TextRun` | interface | `{ element, value }` | Represents a run of plain text — the leaf inline node. `value` is the decoded text with markdown escapes (`\*`, `\_`, …) already resolved to their literal characters; html's text encoder escapes `&`, `<`, `>` on the way out; `"` and `'` stay literal in character data. |
|
|
37
35
|
| `EmphasisNode` | interface | `{ element, strong, children }` | Represents emphasized inline content — `*italic*` / `_italic_` (`strong: false`) or `**bold**` / `__bold__` (`strong: true`). `children` are the nested inline nodes, so emphasis composes (a `**bold _and italic_**` is a strong node wrapping a text node and an emphasis node). |
|
|
38
36
|
| `CodeSpanNode` | interface | `{ element, value }` | Represents an inline code span — \`\` \`code\` \`\`. \`value\` is the verbatim span text; no inner markdown is parsed (code is literal), and the renderer HTML-escapes it inside a \`<code>\` element. |
|
|
39
37
|
| `LineBreakNode` | interface | `{ element }` | Represents a GFM hard line break — two or more trailing spaces before a newline in markdown source, a `br` element in HTML. |
|
|
40
38
|
| `LinkNode` | interface | `{ element, href, children }` | Represents an inline link — `[text](href)`. `children` are the inline nodes of the link text. At render, html's floor removes a refused `href` attribute and the link keeps its text; `htmlToMarkdown` instead stores a refused destination as `''`. |
|
|
41
39
|
| `ImageNode` | interface | `{ element, src, children }` | Represents an inline image — ``. `children` are the inline nodes of the alternative content and `src` is the image destination. |
|
|
42
|
-
| `InlineNode` | type | `
|
|
40
|
+
| `InlineNode` | type | `TextRun \| EmphasisNode \| CodeSpanNode \| LineBreakNode \| LinkNode \| ImageNode` | Represents a node that can appear inside inline content (a heading / paragraph / cell / list item / link text). |
|
|
43
41
|
| `HeadingNode` | interface | `{ element, level, children }` | Represents an ATX heading — `#` … `######`. `level` is 1–6 (the number of leading `#`), `children` the inline content of the heading text. |
|
|
44
42
|
| `ParagraphNode` | interface | `{ element, children }` | Represents a paragraph — a run of non-blank lines that is not another block; `children` its inline content. |
|
|
45
43
|
| `ListItemNode` | interface | `{ element, children }` | Represents one item of a `ListNode` — `children` the block content of the item (typically one paragraph, plus any nested list). |
|
|
@@ -56,8 +54,8 @@ A `Shape` cell holds an interface's data members as bare names in braces, `?` ma
|
|
|
56
54
|
| `MarkdownHandler<TNode, T>` | type | `(node: TNode, children: readonly T[]) => T` | Represents a fold handler for one AST element — receives the node and its children already folded to `T`, and produces the node's own `T`. The building block of a `MarkdownHandlerMap` catamorphism table. |
|
|
57
55
|
| `MarkdownHandlerMap<T>` | interface | `{ document, heading, paragraph, thematicBreak, blockquote, codeBlock, list, listItem, table, text, emphasis, codeSpan, break, link, image }` | Represents the total catamorphism table for `MarkdownInterface.fold` — one `MarkdownHandler` per AST element, keyed by its `element` discriminant. Every key is required: a fold is total over the AST, so there is no element it can skip. |
|
|
58
56
|
| `MarkdownRewriteHandler` | type | `(node: MarkdownNode) => MarkdownNode` | Represents a copy-on-write node rewrite applied bottom-up by `MarkdownInterface.map` — receives one node (its own children already rewritten) and returns its replacement (the same node, unchanged, or a new node). |
|
|
59
|
-
| `MarkdownParseResult` | type | `readonly [ document: MarkdownDocument, spans: ReadonlyMap<MarkdownNode, MarkdownSpan> ]` | Pairs a parsed document with the `MarkdownSpan` of each of its nodes — what `
|
|
60
|
-
| `MarkdownDerivation<T>` | type | `readonly [ value: T, derivations: ReadonlyMap<MarkdownNode, MarkdownNode \| undefined> ]` | Pairs a rewritten value with the input node each rewritten node was produced from — what `
|
|
57
|
+
| `MarkdownParseResult` | type | `readonly [ document: MarkdownDocument, spans: ReadonlyMap<MarkdownNode, MarkdownSpan> ]` | Pairs a parsed document with the `MarkdownSpan` of each of its nodes — what `parseMarkdownProvenance` returns, and what `parseMarkdown` projects the document out of. |
|
|
58
|
+
| `MarkdownDerivation<T>` | type | `readonly [ value: T, derivations: ReadonlyMap<MarkdownNode, MarkdownNode \| undefined> ]` | Pairs a rewritten value with the input node each rewritten node was produced from — what `rewriteMarkdownDocument` returns, so provenance survives a rewrite instead of ending at it. `T` is the rewritten value: the document for a whole-document rewrite. |
|
|
61
59
|
| `MarkdownInterface` | interface | `{ document, walk(), find(), filter(), span(), map(), reduce(), fold(), stream() }` | Represents a stateful, parsed markdown document: the typed `MarkdownDocument` AST plus the query, rewrite, and fold operations over it. |
|
|
62
60
|
|
|
63
61
|
| Name | Kind | Signature | Summary |
|
|
@@ -68,79 +66,76 @@ A `Shape` cell holds an interface's data members as bare names in braces, `?` ma
|
|
|
68
66
|
|
|
69
67
|
From [`constants.ts`](../src/core/constants.ts).
|
|
70
68
|
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
|
74
|
-
| ------------------ | ----- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
75
|
-
| `MAX_DEPTH` | const | `64` | Caps the recursion depth the parse pipeline (`parseDocument` and its `parsers.ts` helpers), the `helpers.ts` traversal / projection functions (`markdownToHTML`, `renderMarkdown`, `walkNodes`, `foldNode`, `rewriteDocument`), and the `compilers.ts` renderer (`renderHTML`) honor before degrading, at 64. It bounds blockquote nesting, inline nesting (emphasis / links), and traversal / projection recursion so pathological or hostile input cannot exhaust the call stack. `htmlToMarkdown` is the inherited exception: its fold and depth cap belong to `@orkestrel/html`. |
|
|
76
|
-
| `EMPTY_PROJECTION` | const | `MarkdownProjection` | Holds the frozen empty HTML-to-markdown projection from which projection factories default every absent field. |
|
|
69
|
+
| Name | Kind | Signature | Summary |
|
|
70
|
+
| ------------------ | ----- | -------------------- | -------------------------------------------------------------------------------------------------------------- |
|
|
71
|
+
| `EMPTY_PROJECTION` | const | `MarkdownProjection` | Holds the frozen empty HTML-to-markdown projection from which projection factories default every absent field. |
|
|
77
72
|
|
|
78
73
|
### Parsers
|
|
79
74
|
|
|
80
|
-
The block/inline parsing pipeline, from [`parsers.ts`](../src/core/parsers.ts) — the orchestration `
|
|
75
|
+
The block/inline parsing pipeline, from [`parsers.ts`](../src/core/parsers.ts) — the orchestration `parseMarkdown` composes out of `helpers.ts`'s pure scanning leaves. `parseBlocks` is the recursive spine; each parser is exported and independently testable. The construct scanners it composes (`collectTable` / `collectList`) are leaves and live in [`helpers.ts`](../src/core/helpers.ts) with the other scanners; each calls back into the phase entry above it, so the two files are mutually recursive by design.
|
|
81
76
|
|
|
82
|
-
| Name
|
|
83
|
-
|
|
|
84
|
-
| `parseBlocks`
|
|
85
|
-
| `
|
|
86
|
-
| `
|
|
87
|
-
| `parseInline`
|
|
77
|
+
| Name | Kind | Signature | Summary |
|
|
78
|
+
| ------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
79
|
+
| `parseBlocks` | function | `(lines: readonly MarkdownSource[], depth: number, spans?: Map<MarkdownNode, MarkdownSpan>, end?: number) => readonly BlockNode[]` | Parses a run of markdown lines into a block AST, recursing into nested blockquotes, list items, and depth-capped degrade paragraphs. |
|
|
80
|
+
| `parseMarkdown` | function | `(markdown: string) => MarkdownDocument` | Parses a markdown string into a typed `MarkdownDocument` AST through the block phase — the document half of what `parseMarkdownProvenance` returns. Malformed markdown degrades to literal text, so the parse never throws. |
|
|
81
|
+
| `parseMarkdownProvenance` | function | `(markdown: string) => MarkdownParseResult` | Parses a markdown string into a document and its original-source spans. Malformed markdown degrades to literal text, so the parse never throws. |
|
|
82
|
+
| `parseInline` | function | `(text: string) => readonly InlineNode[]` | Parses inline markdown text (emphasis, code spans, links, images, and hard breaks) into inline AST nodes, coalescing adjacent text runs and reading no block structure. Malformed markdown degrades to literal text, so the parse never throws. |
|
|
88
83
|
|
|
89
84
|
### Helpers
|
|
90
85
|
|
|
91
|
-
Pure, total leaves from [`helpers.ts`](../src/core/helpers.ts) — the line and character structural predicates the block and inline phases test raw lines with, the scanning functional core `parsers.ts` composes, the AST-crossing projections (`markdownToHTML`, `renderMarkdown`, `htmlToMarkdown`) plus the projection leaves they are built from, and the traversal engines `Markdown` delegates to. Every function is unit-testable in isolation; malformed input degrades to text, never throws. A predicate over a raw `string` narrows no type, so it is a leaf here rather than a guard in `validators.ts`; `
|
|
92
|
-
|
|
93
|
-
| Name
|
|
94
|
-
|
|
|
95
|
-
| `splitLines`
|
|
96
|
-
| `sliceSource`
|
|
97
|
-
| `joinSources`
|
|
98
|
-
| `
|
|
99
|
-
| `trimSource`
|
|
100
|
-
| `normalizeParagraphLine`
|
|
101
|
-
| `countIndent`
|
|
102
|
-
| `isFlankingWhitespace`
|
|
103
|
-
| `isEscapable`
|
|
104
|
-
| `isBlankLine`
|
|
105
|
-
| `isQuote`
|
|
106
|
-
| `isFenceClose`
|
|
107
|
-
| `isFenceWhitespace`
|
|
108
|
-
| `isThematicBreak`
|
|
109
|
-
| `isTableStart`
|
|
110
|
-
| `extractHeading`
|
|
111
|
-
| `extractFence`
|
|
112
|
-
| `extractListItem`
|
|
113
|
-
| `stripQuote`
|
|
114
|
-
| `splitTableRow`
|
|
115
|
-
| `splitTableSources`
|
|
116
|
-
| `delimiterToAlignments`
|
|
117
|
-
| `startsBlock`
|
|
118
|
-
| `unescapeText`
|
|
119
|
-
| `coalesceText`
|
|
120
|
-
| `scanCode`
|
|
121
|
-
| `scanLink`
|
|
122
|
-
| `scanEmphasis`
|
|
123
|
-
| `locateLink`
|
|
124
|
-
| `locateEmphasis`
|
|
125
|
-
| `scanInline`
|
|
126
|
-
| `scanInlineSource`
|
|
127
|
-
| `collectTable`
|
|
128
|
-
| `collectList`
|
|
129
|
-
| `markdownToHTML`
|
|
130
|
-
| `renderMarkdown`
|
|
131
|
-
| `htmlToMarkdown`
|
|
132
|
-
| `createProjection`
|
|
133
|
-
| `trimInlines`
|
|
134
|
-
| `normalizeInlines`
|
|
135
|
-
| `mergeProjections`
|
|
136
|
-
| `projectHTMLLeaf`
|
|
137
|
-
| `projectHTMLNode`
|
|
138
|
-
| `projectionToBlocks`
|
|
139
|
-
| `projectionToInlines`
|
|
140
|
-
| `
|
|
141
|
-
| `
|
|
142
|
-
| `
|
|
143
|
-
| `flattenText`
|
|
86
|
+
Pure, total leaves from [`helpers.ts`](../src/core/helpers.ts) — the line and character structural predicates the block and inline phases test raw lines with, the scanning functional core `parsers.ts` composes, the AST-crossing projections (`markdownToHTML`, `renderMarkdown`, `htmlToMarkdown`) plus the projection leaves they are built from, and the traversal engines `Markdown` delegates to. Every function is unit-testable in isolation; malformed input degrades to text, never throws. A predicate over a raw `string` narrows no type, so it is a leaf here rather than a guard in `validators.ts`; `renderMarkdownHTML` drives the `HTML` class, so it sits in [`compilers.ts`](../src/core/compilers.ts) instead (§ [Compilers](#compilers)). `projectHTMLLeaf`'s leaf parameter is html's own `TextNode`, written `HTMLTextNode` in the signature to keep it apart from this package's `TextRun`.
|
|
87
|
+
|
|
88
|
+
| Name | Kind | Signature | Summary |
|
|
89
|
+
| ------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
90
|
+
| `splitLines` | function | `(markdown: string) => readonly MarkdownSource[]` | Splits a markdown document into offset-bearing lines while normalizing CRLF and bare CR terminators at the line boundary. A single trailing terminator does not yield a final empty line. |
|
|
91
|
+
| `sliceSource` | function | `(source: MarkdownSource, from: number, to: number) => MarkdownSource` | Slices derived markdown text and narrows each intersecting source segment to the same text-relative range. |
|
|
92
|
+
| `joinSources` | function | `(sources: readonly MarkdownSource[], separator: string) => MarkdownSource` | Joins offset-bearing markdown sources while mapping a separator to the original region between adjacent mapped sources. |
|
|
93
|
+
| `projectMarkdownSpan` | function | `(source: MarkdownSource, from: number, to: number) => MarkdownSpan \| undefined` | Projects a derived text range through its segments to a half-open region of the original markdown string. |
|
|
94
|
+
| `trimSource` | function | `(source: MarkdownSource) => MarkdownSource` | Trims an offset-bearing source without losing the coordinates of its retained text. |
|
|
95
|
+
| `normalizeParagraphLine` | function | `(source: MarkdownSource, breaks: boolean) => MarkdownSource` | Normalizes one paragraph line while retaining the full source run consumed by a trailing-space hard break. |
|
|
96
|
+
| `countIndent` | function | `(line: string) => number` | Counts the leading space / tab characters on `line` (a tab counts as one) — the indent that decides whether a list item's continuation belongs to the item. |
|
|
97
|
+
| `isFlankingWhitespace` | function | `(character: string) => boolean` | Checks whether `character` is whitespace under the emphasis flanking rule — a space, a tab, or a newline. |
|
|
98
|
+
| `isEscapable` | function | `(character: string) => boolean` | Checks whether `character` is escapable by a leading backslash — the ASCII punctuation markdown gives meaning to (so `\*` becomes `*` but `\.` stays `\.`). |
|
|
99
|
+
| `isBlankLine` | function | `(line: string) => boolean` | Checks whether `line` is blank — empty, or containing only whitespace — the markdown definition of a blank line that block parsing uses to separate paragraphs, skip gaps, and end list continuations. |
|
|
100
|
+
| `isQuote` | function | `(line: string) => boolean` | Checks whether `line` is a blockquote line (`>` optionally indented up to three spaces) — its content is de-quoted by `stripQuote`. |
|
|
101
|
+
| `isFenceClose` | function | `(line: string, marker: string) => boolean` | Checks whether `line` closes a fence opened by `marker` — the same fence character, a run at least as long, and nothing else but surrounding whitespace. |
|
|
102
|
+
| `isFenceWhitespace` | function | `(character: string \| undefined) => boolean` | Checks whether `character` is a regex-`\s`-equivalent whitespace character — the character class `isFenceClose`'s scan treats as surrounding padding. |
|
|
103
|
+
| `isThematicBreak` | function | `(line: string) => boolean` | Checks whether `line` is a thematic break (horizontal rule) — three or more of the same marker `-`, `*`, or `_` (optionally space-separated) and nothing else (`---`, `***`, `___`, `- - -`). |
|
|
104
|
+
| `isTableStart` | function | `(header: string, delimiter: string \| undefined) => boolean` | Checks whether the pair (`header`, `delimiter`) opens a GFM table — `delimiter` is a row of `\|`-separated cells each matching `:?-+:?`, the GFM rule that a table requires a header row immediately followed by a delimiter row. |
|
|
105
|
+
| `extractHeading` | function | `(line: string) => HeadingMatch \| undefined` | Extracts an ATX heading line (`#` … `######` followed by text) into its level, trimmed text, and the text's offset inside the line. A run of more than 6 `#`s, or `#`s not followed by whitespace + text, is not a heading; an optional closing `###` run is stripped. |
|
|
106
|
+
| `extractFence` | function | `(line: string) => FenceMatch \| undefined` | Extracts a fenced-code opening line (\`\`\`\` \`\`\` \`\`\`\` or \`~~~\`, optionally with an info string) into its \`{ marker, lang }\`, or \`undefined\` when \`line\` is not a fence opener. \`marker\` is the exact fence run (the closer must match the same character + at least the same length); \`lang\` is the first word of the info string. |
|
|
107
|
+
| `extractListItem` | function | `(line: string) => ListItemMatch \| undefined` | Extracts a list-item line (`-` / `*` / `+` bullet, or `1.` / `1)` ordinal, followed by a space) into its `ListItemMatch`, or `undefined` when `line` is not a list item. `content` is the text after the marker; `marker` is the full marker-plus-space width (for measuring a continuation's indent). |
|
|
108
|
+
| `stripQuote` | function | `(source: MarkdownSource) => MarkdownSource` | Strips one level of blockquote marker (`>` plus one optional following space) from an offset-bearing blockquote line, so the de-quoted source re-parses as nested blocks without losing its original coordinates. |
|
|
109
|
+
| `splitTableRow` | function | `(row: string) => readonly string[]` | Splits one GFM table row into its cell strings — outer pipes are optional, a pipe escaped by a leading backslash inside a cell is not a separator (it becomes a literal pipe character), and the empty leading / trailing cell an outer pipe produces is dropped. Derives the string form from `splitTableSources`, which owns the escaped-pipe splitting rule. |
|
|
110
|
+
| `splitTableSources` | function | `(row: MarkdownSource) => readonly MarkdownSource[]` | Splits an offset-bearing GFM table row into offset-bearing cells, retaining the complete source spelling of an escaped pipe while exposing its literal value. |
|
|
111
|
+
| `delimiterToAlignments` | function | `(delimiter: string) => ReadonlyArray<TableAlign \| null>` | Derives the per-column `TableAlign` list from a GFM delimiter row — `:---` left, `---:` right, `:---:` center, and `---` as the explicit no-alignment marker represented by `null`. |
|
|
112
|
+
| `startsBlock` | function | `(lines: readonly string[], index: number) => boolean` | Checks whether the line at `index` starts a new block kind (heading / fence / thematic break / blockquote / list / table) — the paragraph collector stops at such a line so a block following a paragraph without a blank line still parses (a trusted-input caller writing a `##` heading directly under a paragraph, with no intervening blank line). |
|
|
113
|
+
| `unescapeText` | function | `(text: string) => string` | Resolves backslash escapes in a raw string to their literal characters — used for a link `href` (which is not otherwise inline-parsed) and any plain text run. |
|
|
114
|
+
| `coalesceText` | function | `(nodes: readonly InlineNode[], spans?: Map<MarkdownNode, MarkdownSpan>) => readonly InlineNode[]` | Merges adjacent text nodes into one — the inline scanner emits a text node per unrecognized character, so coalescing keeps the AST clean and assertion-friendly. |
|
|
115
|
+
| `scanCode` | function | `(source: string, start: number, to: number) => CodeSpanMatch \| undefined` | Scans an inline code span at `start` (a \`\` \` \`\`-run … a matching \`\` \` \`\`-run of the same length, the CommonMark rule that lets a span contain backticks). Returns the span's literal text + end index, or \`undefined\` when no matching closer exists (it then degrades to literal backticks). |
|
|
116
|
+
| `scanLink` | function | `(source: string, start: number, to: number, depth?: number) => LinkScan \| undefined` | Scans a link `[text](href)` at `start` — the text runs to a balanced `]`, then `(` must immediately follow and the destination runs to the matching `)` (both respect nested delimiters + escapes) through `locateLink`, and returns the parsed node and end index. Returns `undefined` when the shape does not hold (it then degrades to a literal `[`). |
|
|
117
|
+
| `scanEmphasis` | function | `(source: string, start: number, to: number, depth?: number) => EmphasisScan \| undefined` | Scans an emphasis run at `start` (`*` / `_`, doubled for strong) — finds the nearest matching closing run of the same marker + width while skipping complete nested runs from the other marker family, and requires non-space immediately inside both delimiters (the CommonMark flanking simplification that blocks `* x *`) through `locateEmphasis`, and returns the parsed node and end index. Returns `undefined` when no valid closer exists (it then degrades to a literal marker). |
|
|
118
|
+
| `locateLink` | function | `(source: string, start: number, to: number) => LinkBounds \| undefined` | Locates a link `[text](href)` at `start` — the text runs to a balanced `]`, then `(` must immediately follow and the destination runs to the matching `)` (both respect nested delimiters + escapes). Returns the label close and syntax end, or `undefined` when the shape does not hold (it then degrades to a literal `[`). |
|
|
119
|
+
| `locateEmphasis` | function | `(source: string, start: number, to: number) => EmphasisBounds \| undefined` | Locates an emphasis run at `start` (`*` / `_`, doubled for strong) — finds the nearest matching closing run of the same marker + width while skipping complete nested runs from the other marker family, and requires non-space immediately inside both delimiters (the CommonMark flanking simplification that blocks `* x *`). Returns the content and syntax bounds, or `undefined` when no valid closer exists (it then degrades to a literal marker). |
|
|
120
|
+
| `scanInline` | function | `(source: string, from: number, to: number, depth?: number) => readonly InlineNode[]` | Scans the window `[from, to)` of `source` into inline nodes — the single recursive engine the inline phase runs on (emphasis, link text, and image alternative content recurse through it). Linear: each character is consumed once; a failed construct emits its opening character as text and advances by one, so there is no re-scan (no ReDoS). |
|
|
121
|
+
| `scanInlineSource` | function | `(source: MarkdownSource, from: number, to: number, spans: Map<MarkdownNode, MarkdownSpan>, depth?: number) => readonly InlineNode[]` | Scans an offset-bearing inline window with the same engine as `scanInline` and records each emitted node against the original markdown string. |
|
|
122
|
+
| `collectTable` | function | `(lines: readonly MarkdownSource[], start: number, spans?: Map<MarkdownNode, MarkdownSpan>) => TableCollection` | Collects a GFM table starting at a header row, parsing the header, the alignment row, and every contiguous body row that follows. |
|
|
123
|
+
| `collectList` | function | `(lines: readonly MarkdownSource[], start: number, depth: number, spans?: Map<MarkdownNode, MarkdownSpan>, end?: number) => ListCollection` | Collects a list starting at the first item, gathering sibling items at the same indent/ordering and recursing into each item's own block content. |
|
|
124
|
+
| `markdownToHTML` | function | `(node: MarkdownNode) => HTMLDocument` | Projects a `MarkdownNode` into an unsanitized `HTMLDocument`. |
|
|
125
|
+
| `renderMarkdown` | function | `(node: MarkdownNode) => string` | Renders a `MarkdownNode` to its canonical markdown source — the inverse projection of `renderMarkdownHTML`. It is the serializer a `parse(renderMarkdown(doc))` round-trip is built on. Canonical forms: `*` / `**` emphasis at even emphasis nesting depths and `_` / `__` at odd depths, `-` bullets, `N.` sequential ordinals (from the list's `start`), `---` thematic breaks, fenced code blocks (backtick run widened past any 3+ backtick run inside the body), ATX headings, `>`-prefixed blockquote lines, GFM tables (1-space-padded cells, a backslash before each literal pipe, an alignment delimiter row), `[text](href)` links, `` images, and two-space hard breaks. A `text` node's literal content is backslash-escaped wherever it would otherwise re-parse as markup, so parsing the rendered source returns the node it was rendered from. |
|
|
126
|
+
| `htmlToMarkdown` | function | `(node: HTMLNode) => MarkdownDocument` | Projects an `@orkestrel/html` `HTMLNode` into a `MarkdownDocument` — the HTML→markdown direction, and the inverse of `markdownToHTML`. |
|
|
127
|
+
| `createProjection` | function | `(parts?: Partial<MarkdownProjection>) => MarkdownProjection` | Builds an HTML-to-markdown projection with absent fields defaulted from `EMPTY_PROJECTION` and the block/inline exclusivity invariant enforced. |
|
|
128
|
+
| `trimInlines` | function | `(nodes: readonly InlineNode[]) => readonly InlineNode[]` | Trims the whitespace at the two ends of an inline run — the leading whitespace of a leading text node and the trailing whitespace of a trailing one — dropping either node when nothing survives. |
|
|
129
|
+
| `normalizeInlines` | function | `(nodes: readonly InlineNode[], breaks: boolean) => readonly InlineNode[]` | Reduces an inline run to the shape markdown can actually write back: adjacent text coalesced, empty text dropped, and every hard break either kept as a real line ending or spent as a space. |
|
|
130
|
+
| `mergeProjections` | function | `(children: readonly MarkdownProjection[]) => MarkdownProjection` | Combines the projections of one node's children into the projection of that node — the single place inline runs become paragraphs, so no ancestor has to decide it twice. |
|
|
131
|
+
| `projectHTMLLeaf` | function | `(leaf: CommentNode \| DoctypeNode \| HTMLTextNode) => MarkdownProjection` | Projects one HTML leaf — a text node, a comment, or a doctype — to its `MarkdownProjection`. |
|
|
132
|
+
| `projectHTMLNode` | function | `(node: ElementNode \| HTMLDocument, children: readonly MarkdownProjection[]) => MarkdownProjection` | Projects one HTML container — the document root or an element — from its children's already-computed projections. The element mapping, and the only place that decides what an HTML tag becomes in markdown. |
|
|
133
|
+
| `projectionToBlocks` | function | `(projection: MarkdownProjection) => readonly BlockNode[]` | Reads a projection as block content — the view a document, a blockquote, and a list item each need. |
|
|
134
|
+
| `projectionToInlines` | function | `(projection: MarkdownProjection) => readonly InlineNode[]` | Reads a projection as inline content — the view a link, an emphasis, and a table cell each need. |
|
|
135
|
+
| `walkMarkdownNodes` | function | `(node: MarkdownNode) => Generator<MarkdownNode>` | Walks a `MarkdownNode` depth-first, pre-order, root-inclusive — yields the node itself, then recurses into its children (block children, list items, image/link inline children, table header/row cells' inline nodes) in walk order. |
|
|
136
|
+
| `foldMarkdownTree` | function | `<T>(node: MarkdownNode, handlers: MarkdownHandlerMap<T>, depth: number) => T` | Folds a `MarkdownNode` into a `T` through a total catamorphism — children are folded first (post-order), then the node's own `MarkdownHandler` is invoked with the already-folded children. |
|
|
137
|
+
| `rewriteMarkdownDocument` | function | `(document: MarkdownDocument, rewrite: MarkdownRewriteHandler) => MarkdownDerivation<MarkdownDocument>` | Rewrites a `MarkdownDocument` bottom-up (copy-on-write) — each node's children are rewritten first (post-order), then `rewrite` is applied to the node itself; the document root is never passed to `rewrite` (the `element: 'document'` invariant always holds). A table's inline cells and a list's items are rewritten too. |
|
|
138
|
+
| `flattenText` | function | `(node: MarkdownNode) => string` | Concatenates the `value` / `code` content of every descendant text / code-span / code-block node under `node`, including image alternative content, in walk order — the plain-text projection of an AST (search indexing, word counts, a text-only preview). |
|
|
144
139
|
|
|
145
140
|
The iterative engines compose these node and scan operations. Child results are supplied in traversal order; span recorders and cell-pair logs belong to the operation that calls them.
|
|
146
141
|
|
|
@@ -169,19 +164,17 @@ The iterative engines compose these node and scan operations. Child results are
|
|
|
169
164
|
|
|
170
165
|
From [`compilers.ts`](../src/core/compilers.ts) — the class-driving half of the outbound direction. `helpers.ts` owns the pure `markdownToHTML` projection and imports no implementation class; the compiler below constructs `@orkestrel/html`'s `HTML` class, so it sits above the leaves and consumes them.
|
|
171
166
|
|
|
172
|
-
| Name
|
|
173
|
-
|
|
|
174
|
-
| `
|
|
167
|
+
| Name | Kind | Signature | Summary |
|
|
168
|
+
| -------------------- | -------- | -------------------------------- | ----------------------------------------------------- |
|
|
169
|
+
| `renderMarkdownHTML` | function | `(node: MarkdownNode) => string` | Renders a `MarkdownNode` to sanitized canonical HTML. |
|
|
175
170
|
|
|
176
171
|
### Shapers
|
|
177
172
|
|
|
178
173
|
Declarative `ContractShape` values (from `@orkestrel/contract`) from [`shapers.ts`](../src/core/shapers.ts) — one shape compiles into a guard, coercing parser, JSON Schema, and seeded generator (the compilers live in `@orkestrel/contract`, invoked here through `createContract` in `factories.ts`). Only the non-recursive node types shape here; any type whose fields recurse into `BlockNode` / `InlineNode` / `MarkdownNode` stays guard-only (`validators.ts`, through `lazyOf`) — see [Relationship with @orkestrel/contract](#relationship-with-orkestrelcontract).
|
|
179
174
|
|
|
180
|
-
A `Shape` cell holds the constant's declared type.
|
|
181
|
-
|
|
182
175
|
| Name | Kind | Signature | Summary |
|
|
183
176
|
| -------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
184
|
-
| `
|
|
177
|
+
| `textRunShape` | const | `import("@orkestrel/contract").ObjectShape<{ element: import("@orkestrel/contract").LiteralShape<readonly ["text"]>; value: import("@orkestrel/contract").StringShape; }, false>` | Describes the shape of a `TextRun` — a plain-text leaf inline run. |
|
|
185
178
|
| `codeSpanShape` | const | `import("@orkestrel/contract").ObjectShape<{ element: import("@orkestrel/contract").LiteralShape<readonly ["codeSpan"]>; value: import("@orkestrel/contract").StringShape; }, false>` | Describes the shape of a `CodeSpanNode` — an inline code span (\`\` \`code\` \`\`). |
|
|
186
179
|
| `lineBreakShape` | const | `import("@orkestrel/contract").ObjectShape<{ element: import("@orkestrel/contract").LiteralShape<readonly ["break"]>; }, false>` | Describes the shape of a `LineBreakNode` — a GFM hard line-break leaf. |
|
|
187
180
|
| `codeBlockShape` | const | `import("@orkestrel/contract").ObjectShape<{ element: import("@orkestrel/contract").LiteralShape<readonly ["codeBlock"]>; lang: import("@orkestrel/contract").OptionalShape<import("@orkestrel/contract").StringShape>; code: import("@orkestrel/contract").StringShape; }, false>` | Describes the shape of a `CodeBlockNode` — a fenced code block. `lang` is optional (absent when the opening fence carries no info-string). |
|
|
@@ -193,8 +186,6 @@ A `Shape` cell holds the constant's declared type.
|
|
|
193
186
|
|
|
194
187
|
Node guards, from [`validators.ts`](../src/core/validators.ts). The `is{Element}Node` guards narrow an already-parsed `MarkdownNode` by its `element` tag; the from-unknown guards (`isInlineNode` / `isBlockNode` / `isMarkdownNode` / `isMarkdownDocument`) instead validate an arbitrary `unknown` value against the full node shape, composed from `@orkestrel/contract` combinators. Two distinct guard families: the **from-unknown boundary guards** (`isInlineNode` / `isBlockNode` / `isMarkdownNode` / `isMarkdownDocument`) take `unknown` and validate an entire untrusted value from scratch; the **narrowing guards** (`is{Element}Node`, for example `isTableNode`) take an already-typed `MarkdownNode` and narrow it to one member of the union by its `element` tag — they assume the value is already a valid node shape. The line and character structural predicates the parser tests raw strings with narrow nothing, so they are pure leaves in `helpers.ts` (§ [Helpers](#helpers)).
|
|
195
188
|
|
|
196
|
-
In a guard table a `Shape` cell holds the type the guard narrows to.
|
|
197
|
-
|
|
198
189
|
| Name | Kind | Signature | Summary |
|
|
199
190
|
| --------------------- | -------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
200
191
|
| `isHeadingNode` | function | `(node: MarkdownNode) => node is HeadingNode` | Determines whether a node is a heading block. |
|
|
@@ -204,7 +195,7 @@ In a guard table a `Shape` cell holds the type the guard narrows to.
|
|
|
204
195
|
| `isCodeBlockNode` | function | `(node: MarkdownNode) => node is CodeBlockNode` | Determines whether a node is a fenced code block. |
|
|
205
196
|
| `isBlockquoteNode` | function | `(node: MarkdownNode) => node is BlockquoteNode` | Determines whether a node is a blockquote block. |
|
|
206
197
|
| `isThematicBreakNode` | function | `(node: MarkdownNode) => node is ThematicBreakNode` | Determines whether a node is a thematic break (horizontal rule) block. |
|
|
207
|
-
| `
|
|
198
|
+
| `isTextRun` | function | `(node: MarkdownNode) => node is TextRun` | Determines whether a node is a plain text run. |
|
|
208
199
|
| `isEmphasisNode` | function | `(node: MarkdownNode) => node is EmphasisNode` | Determines whether a node is an emphasis run (`*em*` / `**strong**`). |
|
|
209
200
|
| `isCodeSpanNode` | function | `(node: MarkdownNode) => node is CodeSpanNode` | Determines whether a node is an inline code span. |
|
|
210
201
|
| `isLineBreakNode` | function | `(node: MarkdownNode) => node is LineBreakNode` | Determines whether a node is a GFM hard line break. |
|
|
@@ -213,7 +204,7 @@ In a guard table a `Shape` cell holds the type the guard narrows to.
|
|
|
213
204
|
| `isInlineNode` | const | `Guard<InlineNode>` | Determines whether an arbitrary value is a valid `InlineNode` — a text run, emphasis, code span, hard break, link, or image, recursively validated. |
|
|
214
205
|
| `isBlockNode` | const | `Guard<BlockNode>` | Determines whether an arbitrary value is a valid `BlockNode` — a heading, paragraph, list, table, code block, blockquote, or thematic break, recursively validated. |
|
|
215
206
|
| `isMarkdownNode` | const | `Guard<MarkdownNode>` | Determines whether an arbitrary value is a valid `MarkdownNode` — the `MarkdownDocument` root, a `BlockNode`, a `ListItemNode`, or an `InlineNode`, recursively validated. |
|
|
216
|
-
| `isMarkdownDocument` | const | `Guard<MarkdownDocument>` | Determines whether an arbitrary value is a valid `MarkdownDocument` — the parsed-AST root `
|
|
207
|
+
| `isMarkdownDocument` | const | `Guard<MarkdownDocument>` | Determines whether an arbitrary value is a valid `MarkdownDocument` — the parsed-AST root `parseMarkdown` returns, recursively validated. |
|
|
217
208
|
|
|
218
209
|
### Classes
|
|
219
210
|
|
|
@@ -226,7 +217,7 @@ documented in full under its own heading following this table.
|
|
|
226
217
|
|
|
227
218
|
### `Markdown`
|
|
228
219
|
|
|
229
|
-
A stateful, parsed markdown workspace, constructed from a markdown `string` (runs `
|
|
220
|
+
A stateful, parsed markdown workspace, constructed from a markdown `string` (runs `parseMarkdown`) or an already-parsed `MarkdownDocument` (adopted as-is, not re-validated). Exposes its AST through the `readonly document` member (documented here in Surface prose, per the `ContractInterface` precedent, alongside `walk` — both are part of the documented surface even though `document` carries no row in the [`## Methods`](#methods) table below, which lists only call-signature members). `walk` is the deep traversal — a lazy, depth-first, pre-order, root-inclusive generator over every node; its sync `for (const node of markdown.walk())` surface is also consumable by `for await (const node of markdown.walk())` (JavaScript accepts a sync iterable in a `for await`), so an async pipeline needs no separate iterator. Contrast with `stream`: `walk` is deep (every node) and sync; `stream` is shallow (top-level blocks only) and backpressure-respecting. Immutable — `map` never mutates the stored AST, it returns a new `Markdown`. See [`## Methods`](#methods) for its public call-signature surface.
|
|
230
221
|
|
|
231
222
|
### Factories
|
|
232
223
|
|
|
@@ -235,7 +226,7 @@ From [`factories.ts`](../src/core/factories.ts).
|
|
|
235
226
|
| Name | Kind | Signature | Summary |
|
|
236
227
|
| ----------------------------- | -------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
237
228
|
| `createMarkdown` | function | `(input: string \| MarkdownDocument) => MarkdownInterface` | Creates a stateful markdown handle from a markdown string or an already-parsed `MarkdownDocument` — a typed AST plus the query, rewrite, and fold operations `MarkdownInterface` exposes. |
|
|
238
|
-
| `createTextContract` | function | `() => ContractInterface<
|
|
229
|
+
| `createTextContract` | function | `() => ContractInterface<TextRun>` | Compiles the `textRunShape` into a `ContractInterface` for `TextRun` — a guard, coercing parser, JSON Schema, and seeded generator from one shape declaration. |
|
|
239
230
|
| `createCodeSpanContract` | function | `() => ContractInterface<CodeSpanNode>` | Compiles the `codeSpanShape` into a `ContractInterface` for `CodeSpanNode` — a guard, coercing parser, JSON Schema, and seeded generator from one shape declaration. |
|
|
240
231
|
| `createLineBreakContract` | function | `() => ContractInterface<LineBreakNode>` | Compiles the `lineBreakShape` into a `ContractInterface` for `LineBreakNode`. |
|
|
241
232
|
| `createCodeBlockContract` | function | `() => ContractInterface<CodeBlockNode>` | Compiles the `codeBlockShape` into a `ContractInterface` for `CodeBlockNode` — a guard, coercing parser, JSON Schema, and seeded generator from one shape declaration. |
|
|
@@ -280,32 +271,32 @@ A syntax hazard comes with each of them. An image is written ``, whic
|
|
|
280
271
|
- `TableNode.align` is positional — one entry per column, in column order — and JSON cannot carry `undefined` inside an array without changing the array's length on a round trip. It uses `null`, which is also the honest reading of the source: GFM's bare `---` is an explicit "no alignment here" marker written by the author, not an omitted field.
|
|
281
272
|
- `MarkdownCell.align` is a plain optional property on one cell, so absence is `undefined` there, per the ordinary rule.
|
|
282
273
|
|
|
283
|
-
`
|
|
274
|
+
`renderMarkdownHTML` emits an `align` attribute only for those literals; a `null` column emits no attribute at all. `renderMarkdown` writes `:---` / `---:` / `:---:` for them and a bare `---` for `null`, so the delimiter row round-trips exactly.
|
|
284
275
|
|
|
285
276
|
## The parse pipeline
|
|
286
277
|
|
|
287
|
-
`
|
|
278
|
+
`parseMarkdown(markdown)` runs its phases in order:
|
|
288
279
|
|
|
289
280
|
1. **Block phase** — splits the document into lines (`splitLines`, CRLF/CR normalized) and walks them, detecting fences, thematic breaks, ATX headings, blockquotes, GFM tables, and lists (`parseBlocks`, `collectTable`, `collectList`); anything left over collects into a paragraph. `startsBlock` lets a new block interrupt a paragraph without a separating blank line.
|
|
290
281
|
2. **Inline phase** — each block's raw text runs through `scanInline` (backslash escapes, code spans, links, images, emphasis, and the two-trailing-space hard break) through `parseInline`, then `coalesceText` merges adjacent text runs.
|
|
291
282
|
|
|
292
|
-
**Lines carry coordinates, not only text.** `splitLines` returns a `MarkdownSource` per line rather than a bare string: `{ text, segments }`, where `text` is the line a parser reads and `segments` are the `MarkdownSegment` runs mapping that text back to the original string. Each run is `{ offset, start, end }` — `offset` addresses `text`, `start` and `end` address the original, and the run's original length derives from `end - start` rather than being stored beside them. Every strip, trim, and join the block phase performs (`sliceSource`, `trimSource`, `stripQuote`, `joinSources`, `normalizeParagraphLine`, `splitTableSources`) narrows or remaps those runs instead of discarding them, so `
|
|
283
|
+
**Lines carry coordinates, not only text.** `splitLines` returns a `MarkdownSource` per line rather than a bare string: `{ text, segments }`, where `text` is the line a parser reads and `segments` are the `MarkdownSegment` runs mapping that text back to the original string. Each run is `{ offset, start, end }` — `offset` addresses `text`, `start` and `end` address the original, and the run's original length derives from `end - start` rather than being stored beside them. Every strip, trim, and join the block phase performs (`sliceSource`, `trimSource`, `stripQuote`, `joinSources`, `normalizeParagraphLine`, `splitTableSources`) narrows or remaps those runs instead of discarding them, so `projectMarkdownSpan` turns a derived range back into an original one by resolving the range's two boundaries against those runs. It reports `undefined` when either boundary lands in a position the runs leave uncovered: joining two abutting regions with a separator leaves that separator's derived position uncovered. It bridges an uncovered interior when both boundaries resolve. That is what lets provenance be read off the source rather than reconstructed from node values, which would be wrong the moment a value differs from its spelling (§ [Source provenance](#source-provenance)).
|
|
293
284
|
|
|
294
|
-
`new Markdown(markdown)` (or `createMarkdown(markdown)`) calls `
|
|
285
|
+
`new Markdown(markdown)` (or `createMarkdown(markdown)`) calls `parseMarkdownProvenance` internally and stores the resulting document as its `document`, alongside the span map `span` reads back. `markdownToHTML(node)`, `renderMarkdownHTML(node)`, and `renderMarkdown(node)` are **separate**, downstream, standalone projections out of an AST — never fused into parsing, so a caller can inspect, transform, or fold the AST (through `Markdown`'s `find` / `filter` / `map` / `reduce` / `fold`) before ever calling one, or never call one at all. `htmlToMarkdown(node)` is the standalone projection in, and produces the same `MarkdownDocument` shape `parseMarkdown` does, so everything downstream of a parse works identically on a projection.
|
|
295
286
|
|
|
296
|
-
**Total / never-throw.** `
|
|
287
|
+
**Total / never-throw.** `parseMarkdown`, `markdownToHTML`, `renderMarkdownHTML`, `renderMarkdown`, and `htmlToMarkdown` are all total functions: malformed markdown degrades to literal text (an unterminated `**` stays literal, a broken table falls back to a paragraph) rather than throwing, and hostile, cyclic, or pathologically deep HTML degrades rather than throwing. Inline scanning is index-based (no backtracking regex), so it is linear-time — no ReDoS on adversarial input.
|
|
297
288
|
|
|
298
289
|
### Depth degrade semantics
|
|
299
290
|
|
|
300
|
-
`MAX_DEPTH` (`64`) bounds several independent recursions, each degrading to a fixed, cheap fallback instead of recursing further:
|
|
291
|
+
`MAX_DEPTH` (`64`), which this package imports from `@orkestrel/html`, bounds several independent recursions, each degrading to a fixed, cheap fallback instead of recursing further:
|
|
301
292
|
|
|
302
293
|
- **Block recursion** (blockquote / list nesting, `parsers.ts`'s `parseBlocks`) — past the cap, the remaining lines collapse into **one literal paragraph** containing those lines joined by `\n`, instead of continuing to parse nested structure.
|
|
303
294
|
- **Inline recursion** (`scanInline` and `scanInlineSource`) — the engine recurses into `scanInlineSource` itself for a link's text, an image's alternative content, and an emphasis run's children, incrementing `depth` at each descent. Past the cap, the scan window is not scanned for markup at all; it emits as a **single literal text node**.
|
|
304
|
-
- **`markdownToHTML` / `
|
|
295
|
+
- **`markdownToHTML` / `renderMarkdownHTML` recursion** — past the cap, a node is not projected structurally; it yields a text node carrying the `value` of a node that has one (a `TextRun`, `CodeSpanNode`, …), and **nothing at all** for a node with no `value` field. A table reserves the four levels its `thead` / `tbody` / `tr` / cell scaffolding costs and contributes nothing when they would not fit, and a code block reserves the two its `pre > code` costs, so generated structure is charged to the same budget as authored structure and cannot escape the cap. The internal `switch` also carries a `default` arm, so a fabricated node with an `element` outside the exhaustive set (bypassing the type system, for example through an untyped/deserialized value) contributes nothing rather than `undefined` — the projection is total even against a hostile `MarkdownNode`.
|
|
305
296
|
- **`renderMarkdown` recursion** — the same cap and the same value-bearing-vs-empty degrade rule, applied to canonical markdown source instead of HTML.
|
|
306
|
-
- **`
|
|
307
|
-
- **`
|
|
308
|
-
- **`htmlToMarkdown` recursion — the one inherited cap.** This is the only recursion here markdown does not own: the fold is `@orkestrel/html`'s `foldNode`, so its depth bound is html's
|
|
297
|
+
- **`walkMarkdownNodes` / `foldMarkdownTree` recursion** — descent stops at the cap; the node at the cap is still yielded/folded (with an empty children list for `foldMarkdownTree`), its children are not.
|
|
298
|
+
- **`rewriteMarkdownDocument` / `Markdown.map` recursion** — the same cap, because `map` delegates to `rewriteMarkdownDocument`: at the cap, the subtree is passed through unchanged (by reference — not rebuilt, and `rewrite` is not invoked on it) instead of recursing further, so a pathologically deep adopted document cannot exhaust the call stack.
|
|
299
|
+
- **`htmlToMarkdown` recursion — the one inherited cap.** This is the only recursion here markdown does not own: the fold is `@orkestrel/html`'s `foldNode`, so its depth bound is html's own application of the same `MAX_DEPTH`. Nesting past it truncates on html's side before markdown ever sees the content, and because the projected chain can end a level or two deeper than `MAX_DEPTH`, `renderMarkdown` may then truncate the result a second time. Deeply nested HTML is therefore bounded by html's cap on the fold and then by `renderMarkdown`'s own, in sequence rather than by a single cap: the anchor law that follows holds within the depth budget, and beyond it only totality is promised.
|
|
309
300
|
|
|
310
301
|
Together these bound pathological or hostile input (deeply nested blockquotes, runaway emphasis, adversarially deep ASTs) so no parsing, projecting, or writing function can ever exhaust the call stack.
|
|
311
302
|
|
|
@@ -356,26 +347,26 @@ while the value it belongs to does not match that slice. Read provenance off the
|
|
|
356
347
|
reconstruct it from a node value.
|
|
357
348
|
|
|
358
349
|
```ts
|
|
359
|
-
import { Markdown,
|
|
350
|
+
import { Markdown, isTextRun } from '@orkestrel/markdown'
|
|
360
351
|
|
|
361
352
|
const source = 'a \\* b *c*'
|
|
362
353
|
const markdown = new Markdown(source)
|
|
363
354
|
|
|
364
|
-
const [text] = markdown.filter(
|
|
355
|
+
const [text] = markdown.filter(isTextRun)
|
|
365
356
|
text?.value // 'a * b ' — the escape decoded
|
|
366
357
|
const region = text === undefined ? undefined : markdown.span(text)
|
|
367
358
|
region // { start: 0, end: 7 } — the region slices the spelling `a \* b `, not the value
|
|
368
359
|
```
|
|
369
360
|
|
|
370
|
-
**`
|
|
361
|
+
**`parseMarkdownProvenance` is the handle-free entry point.** It returns the document and the
|
|
371
362
|
operation-owned span map together, so a caller that wants coordinates without a `Markdown` instance
|
|
372
|
-
gets the document and the map from one parse. `
|
|
363
|
+
gets the document and the map from one parse. `parseMarkdown` is its document projection, and constructing a `Markdown`
|
|
373
364
|
from a string runs it once and copies the map into the instance.
|
|
374
365
|
|
|
375
366
|
```ts
|
|
376
|
-
import {
|
|
367
|
+
import { parseMarkdownProvenance } from '@orkestrel/markdown'
|
|
377
368
|
|
|
378
|
-
const [document, spans] =
|
|
369
|
+
const [document, spans] = parseMarkdownProvenance('# Title\n\nA **bold** word.')
|
|
379
370
|
spans.get(document) // { start: 0, end: 25 } — the whole input
|
|
380
371
|
```
|
|
381
372
|
|
|
@@ -383,7 +374,7 @@ The map is keyed by node identity, so it addresses that document's nodes and no
|
|
|
383
374
|
instances over the same text hold independent maps, and a node from one reports `undefined` in the
|
|
384
375
|
other.
|
|
385
376
|
|
|
386
|
-
**A rewrite carries regions forward through its derivations.** `
|
|
377
|
+
**A rewrite carries regions forward through its derivations.** `rewriteMarkdownDocument` returns a
|
|
387
378
|
`MarkdownDerivation`, and `map` resolves each output node against the source instance's map in a
|
|
388
379
|
fixed order: an output identity that already holds a region keeps it, an output derived from one
|
|
389
380
|
input node takes that input's region, and anything else takes none. The resolution reads the direct
|
|
@@ -391,14 +382,14 @@ input the rewrite named for that output and stops there — it never follows a s
|
|
|
391
382
|
back into an earlier rewrite's input.
|
|
392
383
|
|
|
393
384
|
```ts
|
|
394
|
-
import { Markdown,
|
|
385
|
+
import { Markdown, isTextRun } from '@orkestrel/markdown'
|
|
395
386
|
|
|
396
387
|
const markdown = new Markdown('# Hi\n\nText.')
|
|
397
388
|
const lowered = markdown.map((node) =>
|
|
398
389
|
node.element === 'text' ? { element: 'text', value: node.value.toLowerCase() } : node,
|
|
399
390
|
)
|
|
400
391
|
|
|
401
|
-
const [first] = lowered.filter(
|
|
392
|
+
const [first] = lowered.filter(isTextRun)
|
|
402
393
|
first?.value // 'hi'
|
|
403
394
|
const region = first === undefined ? undefined : lowered.span(first)
|
|
404
395
|
region // { start: 2, end: 4 } — where `Hi` sits in the original source
|
|
@@ -423,7 +414,7 @@ claims the author wrote something they did not:
|
|
|
423
414
|
own region, because own-region resolution runs first.
|
|
424
415
|
|
|
425
416
|
```ts
|
|
426
|
-
import { Markdown, htmlToMarkdown,
|
|
417
|
+
import { Markdown, htmlToMarkdown, isTextRun } from '@orkestrel/markdown'
|
|
427
418
|
import { parseDocument as parseHTML } from '@orkestrel/html'
|
|
428
419
|
|
|
429
420
|
const imported = new Markdown(htmlToMarkdown(parseHTML('<div>text<p>para</p></div>')))
|
|
@@ -432,7 +423,7 @@ imported.span(imported.document) // undefined — adopted, and projected from HT
|
|
|
432
423
|
const markdown = new Markdown('a *b* c')
|
|
433
424
|
const joined = { element: 'text', value: 'joined' } as const
|
|
434
425
|
const merged = markdown.map((node) => (node.element === 'text' ? joined : node))
|
|
435
|
-
const [text] = merged.filter(
|
|
426
|
+
const [text] = merged.filter(isTextRun)
|
|
436
427
|
text === undefined ? undefined : merged.span(text) // undefined — one output, separate sources
|
|
437
428
|
```
|
|
438
429
|
|
|
@@ -461,25 +452,25 @@ emphasis === undefined ? undefined : spans.get(emphasis) // { start: 4, end: 7 }
|
|
|
461
452
|
|
|
462
453
|
Every coordinate leaf is exported for the same reason the projection leaves are: a caller writing
|
|
463
454
|
their own block or inline phase over the same lines needs `sliceSource`, `trimSource`,
|
|
464
|
-
`joinSources`, `normalizeParagraphLine`, `splitTableSources`, and `
|
|
455
|
+
`joinSources`, `normalizeParagraphLine`, `splitTableSources`, and `projectMarkdownSpan` to keep coordinates
|
|
465
456
|
the way the shipped phases do.
|
|
466
457
|
|
|
467
458
|
## Sanitization policy
|
|
468
459
|
|
|
469
460
|
There is one URL floor here, and markdown does not own it. `@orkestrel/html` owns HTML escaping, scheme judgement, and the sanitize floor; this package holds no escaper, no scheme list, and no sanitizer of its own, and composes with html's instead. A second copy would be a second thing to keep correct, and the failure mode of a sanitizer that has drifted from the one it was copied from is silent.
|
|
470
461
|
|
|
471
|
-
**`
|
|
462
|
+
**`renderMarkdownHTML` sanitizes, unconditionally.** It takes one argument and exposes no options, so there is no call shape that emits unsanitized HTML by accident:
|
|
472
463
|
|
|
473
464
|
```text
|
|
474
465
|
markdownToHTML(node) → new HTML(document).sanitize({ attributes: [...SAFE_ATTRIBUTES, 'src'] }) → renderHTML(document)
|
|
475
466
|
```
|
|
476
467
|
|
|
477
|
-
Only the middle step judges anything. `markdownToHTML` is deliberately inert: it leaves text literal and destinations unsanitized so that the projection stays a pure AST-to-AST mapping and a caller who wants a different policy can supply one. `markdownToHTML` projects only markdown's own node shapes — headings, paragraphs, links, images, emphasis, code, tables, and the rest — and never an `UNSAFE_ELEMENTS` tag such as `script`, the case the fence below exercises: raw HTML written in markdown source has no node shape of its own, so the parser leaves it as literal text and it reaches the output escaped rather than as an element. html's `UNSAFE_ELEMENTS` removal therefore has nothing to remove in this pipeline; what actually judges the projection's output is html's attribute floor, whose full refusal list — the always-stripped attributes and the hard-banned schemes — is [`guides/html.md`](./html.md)'s to state. The fence below shows one member of that floor: a `javascript:` destination stripped from a link's `href` and from an image's `src` while the element and its remaining content survive. This still runs unconditionally: `
|
|
468
|
+
Only the middle step judges anything. `markdownToHTML` is deliberately inert: it leaves text literal and destinations unsanitized so that the projection stays a pure AST-to-AST mapping and a caller who wants a different policy can supply one. `markdownToHTML` projects only markdown's own node shapes — headings, paragraphs, links, images, emphasis, code, tables, and the rest — and never an `UNSAFE_ELEMENTS` tag such as `script`, the case the fence below exercises: raw HTML written in markdown source has no node shape of its own, so the parser leaves it as literal text and it reaches the output escaped rather than as an element. html's `UNSAFE_ELEMENTS` removal therefore has nothing to remove in this pipeline; what actually judges the projection's output is html's attribute floor, whose full refusal list — the always-stripped attributes and the hard-banned schemes — is [`guides/html.md`](./html.md)'s to state. The fence below shows one member of that floor: a `javascript:` destination stripped from a link's `href` and from an image's `src` while the element and its remaining content survive. This still runs unconditionally: `renderMarkdownHTML` accepts any `MarkdownNode`, including one a caller constructed by hand, rewrote through `map`, or accepted from elsewhere, so it can never assume its input came from `parseMarkdown` on trusted markdown, and a hand-built node whose destination or attribute is hostile is still caught at this floor.
|
|
478
469
|
|
|
479
|
-
**The one widening: `src`.** html's `SAFE_ATTRIBUTES` deliberately omits resource `src`, because a sanitized page that keeps its `alt` text and loses its download is the safer default for a general HTML sanitizer. Markdown cannot accept that default — `` is syntax whose entire content is a destination — so `
|
|
470
|
+
**The one widening: `src`.** html's `SAFE_ATTRIBUTES` deliberately omits resource `src`, because a sanitized page that keeps its `alt` text and loses its download is the safer default for a general HTML sanitizer. Markdown cannot accept that default — `` is syntax whose entire content is a destination — so `renderMarkdownHTML` widens the attribute allowlist by exactly `src`, and by nothing else. The widening is narrow by construction: `src` is a member of html's `URL_ATTRIBUTES`, so every widened value still goes through `sanitizeURL`, and the hard refusals are not part of the allowlist axis at all and cannot be widened by anyone. A refused image keeps its element and its alt text and loses only the destination.
|
|
480
471
|
|
|
481
472
|
```ts
|
|
482
|
-
import {
|
|
473
|
+
import { parseMarkdown, renderMarkdownHTML } from '@orkestrel/markdown'
|
|
483
474
|
|
|
484
475
|
const source = [
|
|
485
476
|
'<script>alert(1)</script>',
|
|
@@ -491,7 +482,7 @@ const source = [
|
|
|
491
482
|
'',
|
|
492
483
|
].join('\n')
|
|
493
484
|
|
|
494
|
-
const html =
|
|
485
|
+
const html = renderMarkdownHTML(parseMarkdown(source))
|
|
495
486
|
// '<p><script>alert(1)</script></p><p><a>link</a></p><p><img alt="alt"></p><p><img src="https://x.dev/pic.png" alt="alt"></p>'
|
|
496
487
|
// — the script line has no element shape of its own and renders as escaped text, so no
|
|
497
488
|
// script tag ever reaches the output for html's UNSAFE_ELEMENTS floor to remove; the
|
|
@@ -506,13 +497,13 @@ const html = renderHTML(parseDocument(source))
|
|
|
506
497
|
- **Compact bytes.** Canonical serialization writes no whitespace between blocks: `# Hi\n\nText.` renders as `<h1>Hi</h1><p>Text.</p>`, not as two newline-separated lines.
|
|
507
498
|
- **A refused URL loses the whole attribute.** html removes a URL attribute it refuses rather than emptying it, so a hostile link renders `<a>x</a>` and a hostile image `<img alt="x">` — the words survive, the attribute does not appear at all.
|
|
508
499
|
|
|
509
|
-
**The inbound rule is different, and that asymmetry is deliberate.** Outbound, sanitizing at the very end is right: `markdownToHTML` can stay inert because `
|
|
500
|
+
**The inbound rule is different, and that asymmetry is deliberate.** Outbound, sanitizing at the very end is right: `markdownToHTML` can stay inert because `renderMarkdownHTML` is the only door to a string and it always sanitizes. Inbound there is no such door. `htmlToMarkdown` produces a `MarkdownDocument`, and the serializer that document eventually reaches — `renderMarkdown` — is not a sanitization boundary and must not become one: it writes markdown source, where a destination is content rather than an executed attribute, and where a value dropped late could not be told apart from a value the author wrote. So the projection bakes html's `sanitizeURL(value, SAFE_URL_SCHEMES)` in at projection time, on every `href` and every `src`, whether or not the AST was ever sanitized — a hand-built one never was. A refused destination empties to `''` and the link or image is kept (`[text]()`), because a bad URL is no reason to lose the words around it. Two pipelines, two last responsible moments; the rule sits at each one rather than in the same place twice.
|
|
510
501
|
|
|
511
|
-
**Wanting a stricter policy.** Because the composition is exposed rather than hidden, a caller who needs a narrower floor does not need an option on `
|
|
502
|
+
**Wanting a stricter policy.** Because the composition is exposed rather than hidden, a caller who needs a narrower floor does not need an option on `renderMarkdownHTML`: project with `markdownToHTML`, sanitize with `@orkestrel/html`'s own `HTML` class and whatever `SanitizeOptions` they want, and serialize with html's `renderHTML`. That path can narrow the element set, drop `src` again, or replace the scheme list — and it still cannot go below html's floor, which is the point. The inbound counterpart is [bringing your own element policy](#bringing-your-own-element-policy): the same composition seam placed where that direction's element mapping varies.
|
|
512
503
|
|
|
513
504
|
## `renderMarkdown` round-trip
|
|
514
505
|
|
|
515
|
-
`renderMarkdown` is the inverse of `
|
|
506
|
+
`renderMarkdown` is the inverse of `parseMarkdown`: for any `MarkdownDocument` produced by `parseMarkdown`, `parseMarkdown(renderMarkdown(doc))` deep-equals `doc`, and `renderMarkdown` is idempotent — `renderMarkdown(parseMarkdown(renderMarkdown(doc))) === renderMarkdown(doc)`. It writes every node to one canonical markdown form, never the source's original (possibly variant) spelling:
|
|
516
507
|
|
|
517
508
|
| Construct | Canonical form |
|
|
518
509
|
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
|
@@ -534,11 +525,11 @@ Sanitization is not a markdown-writing concern and does not appear in that table
|
|
|
534
525
|
|
|
535
526
|
A `text` node's literal content is backslash-escaped wherever it would otherwise re-parse as different markup (a leading `#`, a leading list marker, a leading `---` / `~~~` run, a literal `*`/`_`/`` ` ``/`[`/`]`, a trailing `!` before a link); a heading whose inline text ends in a `#` run (with or without leading whitespace) has that run's first `#` backslash-escaped so it cannot be mistaken for an ATX closing sequence on reparse — the round-trip soundness a parser owes its inverse, so parsing the rendered source returns the document it was rendered from.
|
|
536
527
|
|
|
537
|
-
This guarantee is scoped to documents `
|
|
528
|
+
This guarantee is scoped to documents `parseMarkdown` produced (or an equivalent well-formed `MarkdownDocument`). A value fabricated through `map` (or constructed by hand) that stuffs block-significant content or an embedded newline into a node field `renderMarkdown` treats as literal text (a `TextRun.value`, a `LinkNode.href`, …) has no round-trip guarantee — `renderMarkdown` still never throws, but the resulting source is not guaranteed to reparse back to the same AST.
|
|
538
529
|
|
|
539
530
|
## `htmlToMarkdown` projection
|
|
540
531
|
|
|
541
|
-
`htmlToMarkdown` is the inbound direction: an `@orkestrel/html` `HTMLNode` in, a `MarkdownDocument` out, structurally identical to one `
|
|
532
|
+
`htmlToMarkdown` is the inbound direction: an `@orkestrel/html` `HTMLNode` in, a `MarkdownDocument` out, structurally identical to one `parseMarkdown` produces. It lives here rather than in html for the same reason `markdownToHTML` does — deciding that a `<pre>` is a fenced code block, that a `<div>` is nothing at all, and that a `<td>` holding two paragraphs must become one line of text is markdown-format knowledge, and html has no business carrying it.
|
|
542
533
|
|
|
543
534
|
**The engine is borrowed, the projection is not.** The traversal is html's own `foldNode` catamorphism, driven by a total handler table: `projectHTMLNode` for the containers (`document`, `element`) and `projectHTMLLeaf` for the leaves (`text`, `comment`, `doctype`). That is a deliberate reuse rather than a rebuild: html's fold already owns bottom-up ordering, cycle termination, and depth capping over its own AST, and reimplementing them here would mean maintaining a second, subtly different traversal of somebody else's data structure. What this package contributes is the fold value and the element mapping.
|
|
544
535
|
|
|
@@ -576,7 +567,7 @@ The `pre`, list, and table rows read their own node rather than only their child
|
|
|
576
567
|
**The anchor law.** HTML is richer than markdown, so bytes cannot round-trip and the input document is the wrong fixpoint to promise. The right one is the projected AST:
|
|
577
568
|
|
|
578
569
|
```text
|
|
579
|
-
|
|
570
|
+
parseMarkdown(renderMarkdown(htmlToMarkdown(x))) deep-equals htmlToMarkdown(x)
|
|
580
571
|
```
|
|
581
572
|
|
|
582
573
|
Whatever the projection emits, markdown can write it and re-read it as the same tree. That is why the projection normalizes rather than translates literally — whitespace collapsed, edges trimmed, a blank paragraph dropped, a hard break kept only where a line can end, an emphasis's padding moved outside its markers: a shape markdown cannot write back is a shape this projection has no business producing. The law is proved over a corpus with one entry per construct the projection can emit, and again over the whole corpus concatenated into one document.
|
|
@@ -649,8 +640,8 @@ The custom policy inherits everything the default has: the same fold, the same d
|
|
|
649
640
|
|
|
650
641
|
Markdown's validation surface is a thin, purpose-built layer over `@orkestrel/contract`'s guard/combinator/shape machinery:
|
|
651
642
|
|
|
652
|
-
- **From-unknown guards for untrusted ASTs.** `isInlineNode` / `isBlockNode` / `isMarkdownNode` / `isMarkdownDocument` (`validators.ts`) are `Guard<T>` values composed from `recordOf` / `arrayOf` / `unionOf` / `literalOf` / `lazyOf` — each is total (never throws, even on cyclic or adversarially deep input) because every combinator involved is throw-contained by `@orkestrel/contract`'s guard contract. These validate a value that did **not** necessarily come from `
|
|
653
|
-
- **Leaf shapes + compiled contracts, in lockstep.** `shapers.ts` declares `ContractShape` values (`
|
|
643
|
+
- **From-unknown guards for untrusted ASTs.** `isInlineNode` / `isBlockNode` / `isMarkdownNode` / `isMarkdownDocument` (`validators.ts`) are `Guard<T>` values composed from `recordOf` / `arrayOf` / `unionOf` / `literalOf` / `lazyOf` — each is total (never throws, even on cyclic or adversarially deep input) because every combinator involved is throw-contained by `@orkestrel/contract`'s guard contract. These validate a value that did **not** necessarily come from `parseMarkdown` — a deserialized document, a value crossing a process/RPC boundary.
|
|
644
|
+
- **Leaf shapes + compiled contracts, in lockstep.** `shapers.ts` declares `ContractShape` values (`textRunShape`, `codeSpanShape`, `lineBreakShape`, `codeBlockShape`, `thematicBreakShape`, `tableAlignShape`, `listItemMatchShape`) for the AST's non-recursive node types. `factories.ts` compiles the node shapes among them through `createContract` into `ContractInterface<T>` bundles — `schema` / `is` / `parse` / `generate` derived from one declaration, so they can never drift from each other.
|
|
654
645
|
- **Why recursive nodes are guard-only.** A `ContractShape` tree has no lazy/self-referential node — it is a finite, developer-authored tree the compilers can walk exhaustively. Any AST type whose fields recurse into `BlockNode` / `InlineNode` / `MarkdownNode` (`EmphasisNode`, `LinkNode`, `ImageNode`, `HeadingNode`, `ParagraphNode`, `ListItemNode`, `ListNode`, `TableNode`, `BlockquoteNode`, `MarkdownDocument`) is therefore **not** shaped — it stays guard-only, expressed directly in `validators.ts` with `@orkestrel/contract`'s `lazyOf` (the sanctioned recursion entry point: the thunk defers construction so a self-referential guard never references itself before it exists).
|
|
655
646
|
|
|
656
647
|
## Patterns
|
|
@@ -748,7 +739,7 @@ import { Markdown } from '@orkestrel/markdown'
|
|
|
748
739
|
import type { MarkdownHandlerMap } from '@orkestrel/markdown'
|
|
749
740
|
|
|
750
741
|
// A fold is total: one handler per element, no default arm, so a new AST node is a
|
|
751
|
-
// compile error here rather than a silent omission at runtime. Reach for `
|
|
742
|
+
// compile error here rather than a silent omission at runtime. Reach for `renderMarkdownHTML`
|
|
752
743
|
// for real HTML — this table is the shape of an arbitrary projection, not a renderer.
|
|
753
744
|
const toHTML: MarkdownHandlerMap<string> = {
|
|
754
745
|
document: (_, children) => children.join('\n'),
|
|
@@ -847,35 +838,35 @@ import { parseDocument as parseHTML } from '@orkestrel/html'
|
|
|
847
838
|
import {
|
|
848
839
|
Markdown,
|
|
849
840
|
markdownToHTML,
|
|
850
|
-
|
|
841
|
+
renderMarkdownHTML,
|
|
851
842
|
renderMarkdown,
|
|
852
843
|
htmlToMarkdown,
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
844
|
+
walkMarkdownNodes,
|
|
845
|
+
foldMarkdownTree,
|
|
846
|
+
rewriteMarkdownDocument,
|
|
856
847
|
parseInline,
|
|
857
|
-
|
|
848
|
+
parseMarkdown,
|
|
858
849
|
} from '@orkestrel/markdown'
|
|
859
850
|
import type { MarkdownHandlerMap } from '@orkestrel/markdown'
|
|
860
851
|
|
|
861
852
|
const markdown = new Markdown('# Hi\n\nText.')
|
|
862
853
|
|
|
863
|
-
|
|
854
|
+
renderMarkdownHTML(markdown.document) // '<h1>Hi</h1><p>Text.</p>' — sanitized, canonical, compact
|
|
864
855
|
|
|
865
856
|
// The intermediate AST, for a caller who wants to apply their own HTML policy.
|
|
866
857
|
markdownToHTML(markdown.document) // { category: 'document', children: [...] }
|
|
867
858
|
|
|
868
|
-
// renderMarkdown round-trip:
|
|
869
|
-
const roundTripped =
|
|
859
|
+
// renderMarkdown round-trip: parseMarkdown(renderMarkdown(doc)) deep-equals doc.
|
|
860
|
+
const roundTripped = parseMarkdown(renderMarkdown(markdown.document))
|
|
870
861
|
|
|
871
862
|
// The inbound direction: HTML in, the same MarkdownDocument shape a parse produces.
|
|
872
863
|
const imported = htmlToMarkdown(parseHTML('<h1>Release notes</h1><p>Ship <b>fast</b>.</p>'))
|
|
873
864
|
renderMarkdown(imported) // '# Release notes\n\nShip **fast**.'
|
|
874
865
|
|
|
875
|
-
// The class-free path:
|
|
866
|
+
// The class-free path: walkMarkdownNodes / foldMarkdownTree / rewriteMarkdownDocument all operate on a bare MarkdownNode,
|
|
876
867
|
// no Markdown instance required.
|
|
877
868
|
const heading = markdown.document.children[0]
|
|
878
|
-
const elements = [...
|
|
869
|
+
const elements = [...walkMarkdownNodes(heading)].map((node) => node.element) // ['heading', 'text']
|
|
879
870
|
|
|
880
871
|
const countHandlers: MarkdownHandlerMap<number> = {
|
|
881
872
|
document: (_, children) => children.reduce((a, b) => a + b, 0),
|
|
@@ -894,19 +885,19 @@ const countHandlers: MarkdownHandlerMap<number> = {
|
|
|
894
885
|
link: (_, children) => 1 + children.reduce((a, b) => a + b, 0),
|
|
895
886
|
image: (_, children) => 1 + children.reduce((a, b) => a + b, 0),
|
|
896
887
|
}
|
|
897
|
-
const nodeCount =
|
|
888
|
+
const nodeCount = foldMarkdownTree(heading, countHandlers, 0) // 2
|
|
898
889
|
|
|
899
|
-
//
|
|
890
|
+
// rewriteMarkdownDocument is copy-on-write and returns a MarkdownDerivation: the rewritten value, plus
|
|
900
891
|
// the input node each rewritten node was produced from. An unchanged subtree keeps its
|
|
901
892
|
// identity, so an identity rewrite returns the very same document object and records no
|
|
902
893
|
// derivation at all.
|
|
903
|
-
const [rewritten, derivations] =
|
|
894
|
+
const [rewritten, derivations] = rewriteMarkdownDocument(markdown.document, (node) =>
|
|
904
895
|
node.element === 'text' ? { element: 'text', value: node.value.toLowerCase() } : node,
|
|
905
896
|
)
|
|
906
897
|
rewritten.children.length === markdown.document.children.length // true — same shape, lowercased text
|
|
907
898
|
derivations.size > 0 // true — the rebuilt spine, keyed by the output nodes
|
|
908
899
|
|
|
909
|
-
const [unchanged] =
|
|
900
|
+
const [unchanged] = rewriteMarkdownDocument(markdown.document, (node) => node)
|
|
910
901
|
unchanged === markdown.document // true — nothing changed, so nothing was rebuilt
|
|
911
902
|
|
|
912
903
|
const fragment = parseInline('a **bold** span') // readonly InlineNode[], no block structure
|
|
@@ -971,7 +962,7 @@ import { createTextContract } from '@orkestrel/markdown'
|
|
|
971
962
|
import { seededRandom } from '@orkestrel/contract'
|
|
972
963
|
|
|
973
964
|
const text = createTextContract()
|
|
974
|
-
text.schema // the compiled JSON Schema for
|
|
965
|
+
text.schema // the compiled JSON Schema for TextRun
|
|
975
966
|
const fixture = text.generate(seededRandom(42)) // reproducible seed data
|
|
976
967
|
text.is(fixture) // true — guard / generator stay in lockstep
|
|
977
968
|
```
|
|
@@ -980,10 +971,10 @@ text.is(fixture) // true — guard / generator stay in lockstep
|
|
|
980
971
|
|
|
981
972
|
- [`tests/guides.test.ts`](../tests/guides.test.ts) — the `## Surface` ↔ `src/core` bijection (value + type exports), the `MarkdownInterface` ↔ `Markdown` method bijection, and the equality gate: every `Summary` cell against its declaration's description paragraph, the titled `Construct from a string and narrow with a guard` fence against the `@example` block of that title (pinned so the titled pair cannot be retired silently), and the README pitch against this guide's tagline. It also runs the flagship fences and asserts the values their comments claim.
|
|
982
973
|
- [`tests/src/core/Markdown.test.ts`](../tests/src/core/Markdown.test.ts) — `walk` / `find` / `filter` / `map` / `reduce` / `fold` / `stream` behavior, construction from a string vs. an already-parsed document.
|
|
983
|
-
- [`tests/src/core/parsers.test.ts`](../tests/src/core/parsers.test.ts) — `
|
|
974
|
+
- [`tests/src/core/parsers.test.ts`](../tests/src/core/parsers.test.ts) — `parseMarkdown` / `parseInline` / `parseBlocks`, incl. degrade semantics at `MAX_DEPTH`.
|
|
984
975
|
- [`tests/src/core/validators.test.ts`](../tests/src/core/validators.test.ts) — structural predicates + per-node guards + the from-unknown AST guards (soundness on cyclic / adversarial input).
|
|
985
|
-
- [`tests/src/core/helpers.test.ts`](../tests/src/core/helpers.test.ts) — the pure line/block/inline scanning leaves, including the `collectTable` / `collectList` construct scanners; `markdownToHTML` (the unsanitized projection) and `renderMarkdown` (canonical forms, the emphasis-parity corpus, the parse↔render round-trip); the projection leaves (`trimInlines` / `normalizeInlines` / `mergeProjections` / `projectionToBlocks` / `projectionToInlines` / `projectHTMLLeaf` / `projectHTMLNode`) and `htmlToMarkdown` end to end — element mapping, adversarial and cyclic input, the round-trip anchor law over the projection corpus, and the grand markdown → HTML → markdown trip; plus `
|
|
986
|
-
- [`tests/src/core/compilers.test.ts`](../tests/src/core/compilers.test.ts) — `
|
|
976
|
+
- [`tests/src/core/helpers.test.ts`](../tests/src/core/helpers.test.ts) — the pure line/block/inline scanning leaves, including the `collectTable` / `collectList` construct scanners; `markdownToHTML` (the unsanitized projection) and `renderMarkdown` (canonical forms, the emphasis-parity corpus, the parse↔render round-trip); the projection leaves (`trimInlines` / `normalizeInlines` / `mergeProjections` / `projectionToBlocks` / `projectionToInlines` / `projectHTMLLeaf` / `projectHTMLNode`) and `htmlToMarkdown` end to end — element mapping, adversarial and cyclic input, the round-trip anchor law over the projection corpus, and the grand markdown → HTML → markdown trip; plus `walkMarkdownNodes` / `foldMarkdownTree` / `rewriteMarkdownDocument` / `flattenText`.
|
|
977
|
+
- [`tests/src/core/compilers.test.ts`](../tests/src/core/compilers.test.ts) — `renderMarkdownHTML` as the composed pipeline: structure, escaping and sanitization, the `@orkestrel/html` URL floor, the `src` widening, composed elements, and the `MAX_DEPTH` degrade arms.
|
|
987
978
|
- [`tests/src/core/shapers.test.ts`](../tests/src/core/shapers.test.ts) — per-shape guard exactness, JSON Schema essentials, seeded generate round-trips, parse rebuilds, and bidirectional `Infer` ↔ interface type parity.
|
|
988
979
|
- [`tests/src/core/factories.test.ts`](../tests/src/core/factories.test.ts) — `createMarkdown` + the compiled node contracts (`is` / `parse` / `schema` / `generate` round-trips).
|
|
989
980
|
|
|
@@ -1030,7 +1021,7 @@ rebuildMarkdownCell([], [].values(), []) // []
|
|
|
1030
1021
|
### `acceptMarkdownNode`
|
|
1031
1022
|
|
|
1032
1023
|
```ts
|
|
1033
|
-
const original:
|
|
1024
|
+
const original: TextRun = { element: 'text', value: 'hi' }
|
|
1034
1025
|
acceptMarkdownNode(original, original, { element: 'thematicBreak' }) === original // true
|
|
1035
1026
|
```
|
|
1036
1027
|
|