@markup-carve/pandoc-carve 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Mark Scherer
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,324 @@
1
+ # pandoc-carve
2
+
3
+ Bidirectional [Carve](https://github.com/markup-carve/carve) ↔ Pandoc bridge.
4
+
5
+ **Export:** converts Carve markup to Pandoc's JSON AST, unlocking every pandoc
6
+ output format for Carve documents: LaTeX, Typst, DOCX, PDF, RST, JATS, EPUB,
7
+ and dozens more. **Import:** converts anything pandoc reads (DOCX, LaTeX, RST,
8
+ Org, MediaWiki, HTML, Markdown, ...) into Carve source.
9
+
10
+ ```
11
+ .crv ──parse──▶ Carve AST ──carveToPandoc()──▶ Pandoc JSON ──pandoc -f json -t X──▶ .tex / .typ / .docx / …
12
+ .docx / .tex / … ──pandoc -t json──▶ Pandoc JSON ──pandocToCarve()──▶ Carve AST ──renderCarve──▶ .crv
13
+ ```
14
+
15
+ The bridge maps the parsed Carve AST node-by-node, so emphasis, admonitions,
16
+ tables with spans, footnotes, math, and raw passthrough all arrive correctly.
17
+ It also makes Carve's target-routed raw spans finally fire:
18
+ `` `\alpha`{=latex} `` becomes a Pandoc `RawInline` that the LaTeX writer emits
19
+ and every other writer drops - exactly the pandoc-Markdown semantics the syntax
20
+ was born from.
21
+
22
+ ## Install
23
+
24
+ ```bash
25
+ npm install @markup-carve/pandoc-carve
26
+ ```
27
+
28
+ The CLI shells out to a `pandoc` executable on your PATH (3.x; tested against
29
+ 3.10.2). Pandoc is not bundled - install it from [pandoc.org](https://pandoc.org)
30
+ or your package manager. Emitting plain JSON (`-t json`) needs no pandoc at all.
31
+
32
+ ## CLI
33
+
34
+ ```bash
35
+ # Carve -> LaTeX
36
+ pandoc-carve doc.crv -t latex -o doc.tex
37
+
38
+ # Carve -> Typst, PDF, DOCX ... any pandoc writer
39
+ pandoc-carve doc.crv -t typst -o doc.typ
40
+ pandoc-carve doc.crv -t pdf -o doc.pdf
41
+ pandoc-carve doc.crv -t docx -o doc.docx
42
+
43
+ # Standalone document (pandoc templates)
44
+ pandoc-carve doc.crv -t latex -s -o doc.tex
45
+
46
+ # Just the Pandoc JSON AST (no pandoc needed)
47
+ pandoc-carve doc.crv -t json
48
+
49
+ # Read from stdin, pass extra args through to pandoc after --
50
+ cat doc.crv | pandoc-carve - -t latex -- --toc
51
+
52
+ # IMPORT: anything pandoc reads -> Carve
53
+ pandoc-carve report.docx -f docx -o report.crv
54
+ pandoc-carve paper.tex -f latex -o paper.crv
55
+ pandoc-carve README.md -f markdown -o README.crv
56
+
57
+ # Import from a pre-made pandoc JSON AST (no pandoc needed)
58
+ pandoc -f rst -t json doc.rst | pandoc-carve - -f json
59
+
60
+ # From a SERIALIZED Carve AST (spec PART 12) - from any engine, not just carve-js
61
+ carve doc.crv --to-json | pandoc-carve - -f carve-json -t latex
62
+ ```
63
+
64
+ Anything Carve cannot map faithfully is reported on stderr as a
65
+ `pandoc-carve: degraded ...` warning - nothing degrades silently.
66
+
67
+ ## API
68
+
69
+ ```js
70
+ import { carveToPandoc, carveToPandocJson } from '@markup-carve/pandoc-carve';
71
+
72
+ const { doc, warnings } = carveToPandoc('Hello /world/!');
73
+ // doc = { 'pandoc-api-version': [1, 23, 1], meta: {...}, blocks: [...] }
74
+ // warnings = ['degraded: ...'] for lossy constructs
75
+
76
+ const json = carveToPandocJson('Hello /world/!'); // stringified doc
77
+ ```
78
+
79
+ What the converter reads is the **serialized AST of the Carve spec's PART 12**
80
+ (the shape `resources/ast-schema.json` pins), not any implementation's runtime
81
+ tree - so a document another engine already parsed converts the same way,
82
+ whether it arrives as an object or as JSON text:
83
+
84
+ ```js
85
+ import { carveAstToPandoc, carveToCarveAst } from '@markup-carve/pandoc-carve';
86
+
87
+ // A tree from carve-rs, carve-php, carve-go ... or `carve doc.crv --to-json`
88
+ const { doc, warnings } = carveAstToPandoc(serializedAstJson);
89
+
90
+ // The same exchange format, produced from source here
91
+ const ast = carveToCarveAst('Hello /world/!');
92
+ ```
93
+
94
+ The reverse direction takes a Pandoc document (object or JSON string) and
95
+ returns Carve source, serialized by carve's own `renderCarve` (the `carve fmt`
96
+ serializer), so output formatting carries fmt's guarantees:
97
+
98
+ ```js
99
+ import { pandocToCarve, pandocToCarveAst } from '@markup-carve/pandoc-carve';
100
+
101
+ const { carve, warnings } = pandocToCarve(pandocJsonString);
102
+
103
+ // Preserve structured fields that Carve 0.1 source cannot spell, including
104
+ // Pandoc's optional short figure/table caption.
105
+ const { ast, warnings: astWarnings } = pandocToCarveAst(pandocJsonString);
106
+ ```
107
+
108
+ Round-trips are tested as a hard gate: `carve -> pandoc AST -> carve` must
109
+ render byte-identical HTML to the original source across the test corpus.
110
+ For exact restoration of attribute placement (`{.lead}` on a paragraph or
111
+ list), export with `carveToPandoc(src, { roundtrip: true })` or the CLI's
112
+ `--roundtrip` flag - it stamps wrapper divs with a `carve-block` marker the
113
+ importer uses, at the cost of that marker being visible in writer output.
114
+
115
+ Pipe the JSON to pandoc yourself:
116
+
117
+ ```bash
118
+ node -e "import('@markup-carve/pandoc-carve').then(m => process.stdout.write(m.carveToPandocJson(require('fs').readFileSync('doc.crv','utf8'))))" \
119
+ | pandoc -f json -t latex
120
+ ```
121
+
122
+ ## What maps to what
123
+
124
+ | Carve | Pandoc |
125
+ |-------|--------|
126
+ | `/italic/`, `*bold*`, `/*both*/` | Emph, Strong, Strong+Emph |
127
+ | `_underline_`, `~strike~`, `=highlight=` | Underline, Strikeout, Span `.mark` |
128
+ | `{^sup^}`, `{,sub,}` | Superscript, Subscript |
129
+ | Headings + `{#id .class}` attributes | Header with Attr |
130
+ | Tables incl. rowspan/colspan and captions | Table (spans inverted to pandoc's origin-cell model) |
131
+ | `table.rowGroups` counts (PART 12 section 15) | TableHead, one TableBody per group with its RowHeadColumns and intermediate header rows, TableFoot |
132
+ | Cell attributes `\|{#id .cls k=v} text`, row attributes `\| a \|{.cls}` | the `Attr` pandoc's Cell and Row already carry |
133
+ | Footnotes (reference and inline `^[..]`) | Note |
134
+ | Math `` $`..` `` / `` $$`..` `` | Math Inline / Display |
135
+ | Images/quotes with `^ caption` lines | Figure |
136
+ | A bare `::: figure` composite (PART 9 section 4c) | Figure of nested Figures - pandoc's subfigure model - with the group caption on the outer one. An opener carrying a title or a `[label]` is not this production and stays a Div |
137
+ | `::: note` admonitions | Div `.admonition .note` (+ title paragraph) |
138
+ | Tabs / code-group panels, grouping `[label]` | Div; each `[label]` becomes a bold caption so panels stay distinguishable (graceful degradation) |
139
+ | `` `x`{=latex} `` / ```` ```=latex ```` | RawInline / RawBlock (target-routed by pandoc) |
140
+ | Citations `[@key]`, `[+@key]`, `[-@key, p. 33]` | Cite with one Citation per key (AuthorInText / SuppressAuthor / NormalCitation), the locator in the citation suffix, the verbatim source as the Cite content |
141
+ | `@mention`, `#tag`, `:ext[..]`, critic markup | classed Spans (documented degradation) |
142
+ | `[text]{.smallcaps}` | SmallCaps (pandoc's own class convention, both directions) |
143
+ | Frontmatter, nested: maps, block and flow sequences, sequences of maps | Meta, to the depth pandoc's own reader gives it |
144
+ | `::: \|` line blocks (verse) | LineBlock, one entry per line, an empty entry per stanza break |
145
+ | Ordered markers `1.` / `1)` / `a.` / `iv.` | OrderedList with the matching style and delimiter. Pandoc's example list `(@)` and its `(1)` marker have no Carve form and are reported |
146
+ | A no-break space the parser resolved (`\ `) | U+00A0 (the engines publish a private-use sentinel for it) |
147
+ | A reference link or image nothing defines (`[r][]`) | the literal source as text, which is what Carve renders, plus a warning naming the label - as an unresolved footnote already did |
148
+
149
+ The complete node-by-node contract lives in the test goldens. Worked
150
+ input/output pairs in both directions - including how interactive constructs
151
+ degrade for print formats - are in [`examples/`](examples/README.md).
152
+
153
+ ## Why a bridge, not a pandoc reader?
154
+
155
+ A native `Text.Pandoc.Readers.Carve` upstream would be a fourth full Carve
156
+ parser, written in Haskell, outside the conformance loop that keeps the three
157
+ official implementations byte-identical against the shared corpus. While the
158
+ spec is on its 0.x line and still moving, that reader would drift on pandoc's
159
+ release cadence and ship stale behavior. The bridge reuses the canonical,
160
+ conformance-tested `@markup-carve/carve` parser instead - correct by
161
+ construction, and it iterates in lockstep with the spec.
162
+
163
+ A pandoc custom Lua reader has the same drift problem (it would reimplement
164
+ the parser in Lua). Once the spec reaches 1.0 and stabilizes, contributing an
165
+ upstream reader becomes attractive - with this bridge's node map and the
166
+ conformance corpus as the oracle any port has to pass.
167
+
168
+ ## Options
169
+
170
+ ```js
171
+ carveToPandoc(src, {
172
+ roundtrip: true, // stamp attr wrappers for exact re-import
173
+ symbols: { heart: '♥' }, // resolve :name: symbols like the renderer would
174
+ listTable: false, // opt OUT: keep ::: list-table as a degraded div
175
+ citations: false, // opt OUT: read [@key] as an @mention
176
+ extensions: [], // extra Carve extensions for the parse
177
+ });
178
+ ```
179
+
180
+ CLI equivalents: `--roundtrip`, `--symbols map.json`, `--no-list-table`,
181
+ `--no-citations`.
182
+
183
+ `listTable` and `citations` default ON, because the reverse direction writes
184
+ both unprompted and a construct this bridge chooses on the way out has to be
185
+ one it recognizes on the way back. With `citations` off the parser reaches the
186
+ `@` first and `[@doe1990]` is a `.mention` span, so an imported bibliography
187
+ becomes mentions; with `listTable` off a table this bridge itself wrote as
188
+ `::: list-table` returns as a `Div` of nested lists. Turn them off to get what
189
+ a processor with neither extension enabled would render.
190
+
191
+ ## Limitations
192
+
193
+ - A `table.rowGroups` partition whose counts do not add up to the table's row
194
+ count is refused with a warning and the table converts with the implicit
195
+ head/body split instead. PART 12 section 15 requires the sum as a MUST, and
196
+ JSON Schema cannot express a cross-field sum, so a document that validates
197
+ against `resources/ast-schema.json` can still be incoherent - the bridge
198
+ checks it itself rather than trusting a green validator.
199
+ - A pandoc table with block content in a cell (a list, two paragraphs, a code
200
+ block) is imported as `::: list-table` rather than a pipe table. PART 9 §16's
201
+ pipe-table cell holds inlines, so there is no pipe form for it, and the
202
+ extension's cells are list items that hold full blocks. Structure is
203
+ preserved; three things the extension does not spell are reported instead -
204
+ per-column alignment, a foot, and a body group's intermediate header rows.
205
+ It converts back to the pandoc table by default; `listTable: false` returns
206
+ the degraded div instead.
207
+ - A pandoc table with ROW-HEAD COLUMNS takes the same route on the source path,
208
+ because `header-cols=N` is exactly pandoc's `RowHeadColumns` and a pipe table
209
+ cannot say it. Only when every body group agrees on the count: `header-cols`
210
+ is one number for the whole table, so a table whose bodies differ keeps the
211
+ pipe form rather than come back with row headers ADDED to the rows that had
212
+ none. The AST path keeps the plain table, where `rowGroups` carries the whole
213
+ partition and nothing has to be traded for it.
214
+ - Pandoc `SmallCaps` has no Carve node and is not getting one: it imports as a
215
+ `[text]{.smallcaps}` span, with a warning saying so. The span is not a dead
216
+ end, though - the export direction reads that class back as `SmallCaps`, the
217
+ same convention pandoc's own markdown reader uses, so small caps survive
218
+ Pandoc -> Carve -> Pandoc and still reach the LaTeX, Typst and DOCX writers.
219
+ Other attributes on the span are preserved around it, exactly as pandoc does.
220
+ - Pandoc `Quoted` imports as literal curly quote characters (`“…”` / `‘…’`).
221
+ Carve has no quote node, and the characters are what an author would have
222
+ typed. This one is genuinely one-way: the text re-exports as `Str`, so the
223
+ quote kind and pandoc's locale-aware quoting are gone. Reported once per
224
+ document.
225
+ - A `ColSpec`'s `ColWidth` is dropped. Carve's table model has no width slot at
226
+ any level and pipe-table source cannot spell one, so there is nowhere to put
227
+ the number and no syntax that would reproduce it - a wontfix rather than a
228
+ gap. The drop is silent on purpose: pandoc derives a `ColWidth` for every
229
+ grid and multiline table from the ASCII column widths, so a diagnostic would
230
+ fire on the ordinary case and report a value the author never chose. Column
231
+ ALIGNMENT is carried in both directions; tables leaving the bridge always
232
+ carry `ColWidthDefault`, which pandoc's writers size themselves.
233
+ - A rowspan that starts in a header row and continues into the body is clipped
234
+ to an empty body cell, with a warning. Carve is the richer model here: its
235
+ rows are one flat list, while pandoc's `TableHead` and `TableBody` hold
236
+ separate row lists and confine a cell's `rowSpan` to its own section. Moving
237
+ the head/body boundary to make the span fit would silently reclassify a row,
238
+ and duplicating the origin's content would invent a cell the author never
239
+ wrote, so the grid keeps its shape and the diagnostic reports the loss.
240
+ - Reverse conversion keeps flattening for display targets: `pandocToCarve`
241
+ serializes through Carve 0.1 source, which has no spelling for row groups or
242
+ a short caption, so those fields survive only on the `pandocToCarveAst` path.
243
+ - Citations cross as pandoc's own citeproc convention, which is lossy in one
244
+ respect: pandoc's `Citation` has no locator field, so Carve's typed
245
+ `locatorLabel`/`locatorValue` pair is serialized into `citationSuffix` behind
246
+ a `, ` and citeproc re-derives it from there. The bridge does not rebuild
247
+ those two fields on the way back - the label table is section 4.2's, it lives
248
+ in the engine, and a second copy here would drift. The locator TEXT round-
249
+ trips byte for byte, so re-parsing the emitted source with the citations
250
+ extension restores the typed pair. A group whose items mix `AuthorInText` and
251
+ `NormalCitation` cannot be spelled in Carve (the integral `+` is a property of
252
+ the whole cluster) and is imported as integral with a warning.
253
+ - Pandoc keeps its bibliography in document metadata, not in the AST, so
254
+ importing a `Cite` emits no `[@key]:` definition lines. The citation itself
255
+ round-trips: `citationNoteNum` is reproduced the way pandoc's own markdown
256
+ reader computes it (notes closed so far, plus one), so a `Cite` returns
257
+ unchanged rather than unchanged-except-one-integer.
258
+ - Block content inside metadata (pandoc's `MetaBlocks`, e.g. `abstract: |`)
259
+ round-trips as a YAML literal block scalar, the same spelling pandoc's own
260
+ markdown writer emits and its reader turns back into `MetaBlocks` - by the
261
+ FORM, so any key can carry it, not just `abstract`. The scalar keeps every
262
+ line and every blank line between them, so the paragraph structure survives
263
+ rather than flattening into one string. Every other `Meta` shape - maps,
264
+ lists, lists of maps, scalars, booleans - round-trips too.
265
+ - A `rowGroups` partition (a foot, several body groups, a body's own header
266
+ rows or row-head columns) survives into the exchange AST, so
267
+ `pandocToCarveAst` hands it on whole - but PART 9 section 16's pipe table
268
+ spells only a leading run of header rows, so `pandocToCarve` flattens the
269
+ rest into body rows and says so. Row-head columns are the exception - they
270
+ have a list-table spelling and take it (above). Use the AST entry point when
271
+ a foot or a second body group has to survive.
272
+ - A marker on a HEAD cell (`|=> Name |`) is the column's alignment and becomes
273
+ pandoc's `ColSpec`; a marker on a BODY cell (`|> 12 |`) aligns that cell alone
274
+ and becomes the cell's own `Alignment`. The two are not interchangeable: a
275
+ pandoc cell with `AlignDefault` inherits its ColSpec, so promoting a body
276
+ cell's marker to the column would align cells the author did not.
277
+ - Column alignment needs a header row to live on (`|=> Name |`). A headerless
278
+ pandoc table's column alignment is written onto each cell instead, which
279
+ renders the same; the move is reported.
280
+ - The frontmatter reader covers the YAML subset frontmatter uses, not YAML.
281
+ Anchors, tags, multi-document streams, block scalars and flow maps are out; a
282
+ line that fits no shape is reported and skipped rather than guessed at.
283
+ - Scalars are typed the way pandoc's own frontmatter reader types them: an
284
+ unquoted `true`, `yes`, `on` or `y` (and their negatives) is a `MetaBool`, a
285
+ quoted `"true"` stays text, a null scalar keeps its key as an empty
286
+ `MetaString`, and a number stays text because `Meta` has no number type. This
287
+ matters beyond tidiness - a boolean read as text is not a near miss but an
288
+ inversion, since a pandoc template testing `draft` sees any non-empty string
289
+ as true.
290
+ - Attribute order inside `{...}` is normalized to `#id .class key=val` on
291
+ round-trip - pandoc's Attr has fixed slots, so the author's original order
292
+ is not representable. Semantics are unchanged.
293
+ - Import fidelity is bounded by `renderCarve` (carve fmt): the bridge hands it
294
+ a byte-exact AST, but known fmt issues (e.g. trailing whitespace inside code
295
+ blocks, carve-js issue 340) surface in the serialized output.
296
+ - The pinned engine bounds what the source path can produce. The current git
297
+ pin exports `toAstJson`, so the engine's own exchange serializer is used;
298
+ with an older engine (any published release up to `0.1.3`),
299
+ `src/ast-json.ts` performs the PART 12 section 7 mapping instead, and a pin
300
+ that exports `toAstJson` takes over automatically.
301
+ - Tier-3 visual extensions (mermaid, chart, code-group) arrive as their
302
+ degraded block forms (code blocks / divs), same as Carve's static mode.
303
+ `list-table` is the exception: it converts to a real Pandoc table with full
304
+ block content per cell, unless `listTable: false` / `--no-list-table`.
305
+
306
+ ## Development
307
+
308
+ The test suite converts the whole shared spec corpus, which arrives as a
309
+ submodule:
310
+
311
+ ```sh
312
+ git clone --recurse-submodules https://github.com/markup-carve/pandoc-carve
313
+ # or, in an existing clone:
314
+ git submodule update --init
315
+ npm ci && npm test
316
+ ```
317
+
318
+ Without the submodule the corpus tests FAIL rather than skip - a skipped corpus
319
+ reads like a converted one, which is how a stack overflow reachable from a
320
+ two-word document stayed live on the published package.
321
+
322
+ ## License
323
+
324
+ MIT
@@ -0,0 +1,77 @@
1
+ /**
2
+ * The Carve AST exchange shape (PART 12), and the way in from a runtime tree.
3
+ *
4
+ * PART 12 section 1 is the reason this file exists: the wire form is what a
5
+ * consumer reads, and "an implementation whose internals differ MAPS on the way
6
+ * out; it does not export its internals". This package is such a consumer, so
7
+ * everything downstream of here - `convert()` and its node-type switches - keys
8
+ * on spec surface only. A tree produced by carve-rs, carve-php or carve-go
9
+ * converts exactly like one produced by carve-js, because none of them is what
10
+ * the converter reads: `spec/resources/ast-schema.json` is.
11
+ *
12
+ * Two shapes differ from the runtime tree carve-js hands out of `parse()`, and
13
+ * both differences are structural rather than cosmetic (PART 12 section 7):
14
+ *
15
+ * - frontmatter and footnote DEFINITIONS live on the runtime ROOT, as fields.
16
+ * On the wire they are block nodes in `children`, because a root field
17
+ * cannot carry the `pos` section 4 requires of every node.
18
+ * - a definition list's entries are `{terms, definitions}` objects at runtime
19
+ * and a FLAT sequence of `definition_term` / `definition_description` nodes
20
+ * on the wire, for the same reason plus a vocabulary one: under the object
21
+ * form those two names denote nothing.
22
+ *
23
+ * A published engine that exports its own serializer is authoritative and is
24
+ * used when present ({@link toCarveAst}'s `serialize` parameter, wired to the
25
+ * engine in `index.ts`), and the current git pin exports one. The mapping
26
+ * below is what runs when the installed engine predates `toAstJson` (any
27
+ * published release up to `0.1.3` does) - and it is
28
+ * measured, not assumed: every one of the 610 spec-corpus documents validates
29
+ * against the schema after passing through here (test/spec-corpus.test.mjs).
30
+ */
31
+ /** Any node in the exchange tree. Fields are read by name, per the schema. */
32
+ export interface CarveAstNode {
33
+ type: string;
34
+ [key: string]: unknown;
35
+ }
36
+ /** The document root: `type`, `children`, `srcByteLength`, nothing else. */
37
+ export interface CarveAstDocument {
38
+ type: 'document';
39
+ children: CarveAstNode[];
40
+ srcByteLength?: number;
41
+ }
42
+ /**
43
+ * A runtime document to the PART 12 exchange shape.
44
+ *
45
+ * `serialize` is the engine's own serializer (`toAstJson`) when the installed
46
+ * engine exports one. It is authoritative for the structural mapping; the
47
+ * normalization pass still runs over its result, and is a no-op on an engine
48
+ * that already speaks the current vocabulary.
49
+ */
50
+ export declare function toCarveAst(runtime: unknown, serialize?: (doc: unknown) => unknown): CarveAstDocument;
51
+ /**
52
+ * Fold a serialized tree onto the CURRENT vocabulary.
53
+ *
54
+ * The one place a spelling an older engine still writes is recognized, so the
55
+ * converter's switches stay on spec surface. A tree already written by a
56
+ * current engine passes through unchanged, and unchanged means the same
57
+ * objects, not copies of them.
58
+ */
59
+ export declare function normalizeCarveAst(doc: CarveAstDocument): CarveAstDocument;
60
+ /**
61
+ * Read a serialized AST that arrived as data - a file, stdin, another engine's
62
+ * `--to-json`.
63
+ *
64
+ * Treated as DATA, not as a trusted tree: anything that is not a document root
65
+ * is refused here with a message naming what was found, rather than converting
66
+ * to an empty pandoc document that looks like a successful run.
67
+ *
68
+ * The ROOT is all that is checked. Validating the whole tree against the schema
69
+ * would make a JSON Schema validator a runtime dependency of a converter that
70
+ * does not need one, and would refuse documents this package converts perfectly
71
+ * well - a node type newer than the schema this release shipped against, say.
72
+ * Below the root the existing contract applies instead: an unrecognized node
73
+ * degrades to its text and says so in `warnings`, and nothing is dropped
74
+ * silently. Callers that want conformance checked can validate against
75
+ * `resources/ast-schema.json` themselves; the test suite does exactly that.
76
+ */
77
+ export declare function parseCarveAst(input: CarveAstDocument | string): CarveAstDocument;
@@ -0,0 +1,255 @@
1
+ /**
2
+ * The Carve AST exchange shape (PART 12), and the way in from a runtime tree.
3
+ *
4
+ * PART 12 section 1 is the reason this file exists: the wire form is what a
5
+ * consumer reads, and "an implementation whose internals differ MAPS on the way
6
+ * out; it does not export its internals". This package is such a consumer, so
7
+ * everything downstream of here - `convert()` and its node-type switches - keys
8
+ * on spec surface only. A tree produced by carve-rs, carve-php or carve-go
9
+ * converts exactly like one produced by carve-js, because none of them is what
10
+ * the converter reads: `spec/resources/ast-schema.json` is.
11
+ *
12
+ * Two shapes differ from the runtime tree carve-js hands out of `parse()`, and
13
+ * both differences are structural rather than cosmetic (PART 12 section 7):
14
+ *
15
+ * - frontmatter and footnote DEFINITIONS live on the runtime ROOT, as fields.
16
+ * On the wire they are block nodes in `children`, because a root field
17
+ * cannot carry the `pos` section 4 requires of every node.
18
+ * - a definition list's entries are `{terms, definitions}` objects at runtime
19
+ * and a FLAT sequence of `definition_term` / `definition_description` nodes
20
+ * on the wire, for the same reason plus a vocabulary one: under the object
21
+ * form those two names denote nothing.
22
+ *
23
+ * A published engine that exports its own serializer is authoritative and is
24
+ * used when present ({@link toCarveAst}'s `serialize` parameter, wired to the
25
+ * engine in `index.ts`), and the current git pin exports one. The mapping
26
+ * below is what runs when the installed engine predates `toAstJson` (any
27
+ * published release up to `0.1.3` does) - and it is
28
+ * measured, not assumed: every one of the 610 spec-corpus documents validates
29
+ * against the schema after passing through here (test/spec-corpus.test.mjs).
30
+ */
31
+ /**
32
+ * Fields that hold child nodes, in the order a walk should follow them.
33
+ *
34
+ * Listed rather than discovered, mirroring carve-js's own serializer: a
35
+ * citation group's `items` and a definition list's `items` hold plain objects
36
+ * in the runtime tree, and walking them as nodes would rewrite data that is
37
+ * not one.
38
+ */
39
+ const CHILD_FIELDS = [
40
+ 'children',
41
+ 'items',
42
+ 'rows',
43
+ 'cells',
44
+ 'inline',
45
+ 'content',
46
+ 'caption',
47
+ 'shortCaption',
48
+ 'title',
49
+ 'target',
50
+ ];
51
+ function isObject(value) {
52
+ return typeof value === 'object' && value !== null && !Array.isArray(value);
53
+ }
54
+ /**
55
+ * Rewrite a subtree onto spec surface.
56
+ *
57
+ * Structurally shared: a branch with nothing to rewrite comes back as the SAME
58
+ * object, so the runtime tree the caller still holds is never copied and never
59
+ * mutated.
60
+ */
61
+ function normalize(node) {
62
+ if (Array.isArray(node)) {
63
+ let changed = false;
64
+ const mapped = node.map((child) => {
65
+ const next = normalize(child);
66
+ if (next !== child)
67
+ changed = true;
68
+ return next;
69
+ });
70
+ return (changed ? mapped : node);
71
+ }
72
+ if (!isObject(node))
73
+ return node;
74
+ let out;
75
+ const set = (field, value) => {
76
+ out = { ...(out ?? node), [field]: value };
77
+ };
78
+ // Depth first, so a rewritten parent is built from already-rewritten
79
+ // children rather than walked twice.
80
+ for (const field of CHILD_FIELDS) {
81
+ const value = (out ?? node)[field];
82
+ if (value === undefined || !(typeof value === 'object' && value !== null))
83
+ continue;
84
+ if (field === 'items' && (out ?? node)['type'] === 'definition_list')
85
+ continue;
86
+ const mapped = normalize(value);
87
+ if (mapped !== value)
88
+ set(field, mapped);
89
+ }
90
+ const current = out ?? node;
91
+ switch (current['type']) {
92
+ case 'definition_list': {
93
+ const items = current['items'];
94
+ if (Array.isArray(items) && items.some((item) => isObject(item) && item['type'] === undefined)) {
95
+ set('items', definitionEntriesToWire(items));
96
+ }
97
+ else if (Array.isArray(items)) {
98
+ const mapped = normalize(items);
99
+ if (mapped !== items)
100
+ set('items', mapped);
101
+ }
102
+ break;
103
+ }
104
+ case 'critic-comment':
105
+ // The engine settled the vocabulary on snake_case; the schema
106
+ // declares `critic_comment` and never the hyphen. A consumer that
107
+ // reads the wire has no reason to know the other spelling existed.
108
+ set('type', 'critic_comment');
109
+ break;
110
+ case 'footnote': {
111
+ // The pre-split inline node (carve#405), whose name COLLIDES with
112
+ // the wire `footnote` - which is the DEFINITION block, carrying
113
+ // `label` and `children`. A definition is left alone; anything else
114
+ // named `footnote` is one of the two inline forms, told apart by
115
+ // `inline` exactly as the split encodes it.
116
+ if (typeof current['label'] === 'string' && Array.isArray(current['children']))
117
+ break;
118
+ if (Array.isArray(current['inline'])) {
119
+ const { id: _id, ...rest } = out ?? { ...node };
120
+ out = { ...rest, type: 'inline_footnote' };
121
+ }
122
+ else {
123
+ const { inline: _inline, ...rest } = out ?? { ...node };
124
+ out = { ...rest, type: 'footnote_ref' };
125
+ }
126
+ break;
127
+ }
128
+ case 'heading_ref':
129
+ // PART 12 section 3a keeps the resolved target text OFF the wire:
130
+ // the heading is in the same document, so a consumer reads the text
131
+ // from there rather than from a copy in every reference.
132
+ if (current['resolvedText'] !== undefined) {
133
+ const { resolvedText: _resolvedText, ...rest } = current;
134
+ out = rest;
135
+ }
136
+ break;
137
+ default:
138
+ break;
139
+ }
140
+ return (out ?? node);
141
+ }
142
+ /**
143
+ * Runtime entries to the flat wire sequence.
144
+ *
145
+ * The grouping is recovered on the way in by the rule the renderers use: a run
146
+ * of descriptions belongs to the run of terms before it.
147
+ *
148
+ * An entry that is ALREADY a wire node passes through. That is not a
149
+ * hypothetical: the decision to convert is made per LIST, so one runtime entry
150
+ * in a list is enough to send every entry through here, and dropping the ones
151
+ * that were already converted would delete authored content silently. Same
152
+ * reasoning for a `terms` value that is not an array - a malformed entry is
153
+ * skipped rather than emitted as a term with no content.
154
+ */
155
+ function definitionEntriesToWire(items) {
156
+ const out = [];
157
+ for (const item of items) {
158
+ if (isObject(item) && typeof item['type'] === 'string') {
159
+ out.push(normalize(item));
160
+ continue;
161
+ }
162
+ const entry = (item ?? {});
163
+ for (const term of entry.terms ?? []) {
164
+ if (!Array.isArray(term))
165
+ continue;
166
+ out.push({ type: 'definition_term', children: normalize(term) });
167
+ }
168
+ for (const definition of entry.definitions ?? []) {
169
+ if (!Array.isArray(definition))
170
+ continue;
171
+ out.push({ type: 'definition_description', children: normalize(definition) });
172
+ }
173
+ }
174
+ return out;
175
+ }
176
+ /**
177
+ * A runtime document to the PART 12 exchange shape.
178
+ *
179
+ * `serialize` is the engine's own serializer (`toAstJson`) when the installed
180
+ * engine exports one. It is authoritative for the structural mapping; the
181
+ * normalization pass still runs over its result, and is a no-op on an engine
182
+ * that already speaks the current vocabulary.
183
+ */
184
+ export function toCarveAst(runtime, serialize) {
185
+ if (serialize)
186
+ return normalize(serialize(runtime));
187
+ const doc = (runtime ?? {});
188
+ const children = [];
189
+ if (doc.frontmatter) {
190
+ const node = {
191
+ type: 'frontmatter',
192
+ format: doc.frontmatter.format ?? 'yaml',
193
+ content: doc.frontmatter.content ?? '',
194
+ };
195
+ if (doc.frontmatter.pos !== undefined)
196
+ node['pos'] = doc.frontmatter.pos;
197
+ children.push(node);
198
+ }
199
+ children.push(...normalize(doc.children ?? []));
200
+ // A definition belongs to the DOCUMENT even when it was authored inside a
201
+ // container (PART 9 section 16), which is exactly how the runtime root
202
+ // already holds it - keyed by label, body lifted out.
203
+ for (const [label, body] of Object.entries(doc.footnoteDefs ?? {})) {
204
+ const node = {
205
+ type: 'footnote',
206
+ label,
207
+ children: normalize(body ?? []),
208
+ };
209
+ const pos = doc.footnoteDefPos?.[label];
210
+ if (pos !== undefined)
211
+ node['pos'] = pos;
212
+ children.push(node);
213
+ }
214
+ const out = { type: 'document', children };
215
+ if (doc.srcByteLength !== undefined)
216
+ out.srcByteLength = doc.srcByteLength;
217
+ return out;
218
+ }
219
+ /**
220
+ * Fold a serialized tree onto the CURRENT vocabulary.
221
+ *
222
+ * The one place a spelling an older engine still writes is recognized, so the
223
+ * converter's switches stay on spec surface. A tree already written by a
224
+ * current engine passes through unchanged, and unchanged means the same
225
+ * objects, not copies of them.
226
+ */
227
+ export function normalizeCarveAst(doc) {
228
+ return normalize(doc);
229
+ }
230
+ /**
231
+ * Read a serialized AST that arrived as data - a file, stdin, another engine's
232
+ * `--to-json`.
233
+ *
234
+ * Treated as DATA, not as a trusted tree: anything that is not a document root
235
+ * is refused here with a message naming what was found, rather than converting
236
+ * to an empty pandoc document that looks like a successful run.
237
+ *
238
+ * The ROOT is all that is checked. Validating the whole tree against the schema
239
+ * would make a JSON Schema validator a runtime dependency of a converter that
240
+ * does not need one, and would refuse documents this package converts perfectly
241
+ * well - a node type newer than the schema this release shipped against, say.
242
+ * Below the root the existing contract applies instead: an unrecognized node
243
+ * degrades to its text and says so in `warnings`, and nothing is dropped
244
+ * silently. Callers that want conformance checked can validate against
245
+ * `resources/ast-schema.json` themselves; the test suite does exactly that.
246
+ */
247
+ export function parseCarveAst(input) {
248
+ const value = typeof input === 'string' ? JSON.parse(input) : input;
249
+ if (!isObject(value) || value['type'] !== 'document' || !Array.isArray(value['children'])) {
250
+ const found = isObject(value) ? `type ${JSON.stringify(value['type'])}` : typeof value;
251
+ throw new Error(`not a Carve AST document: expected a root node of type "document" with a children array, found ${found}. ` +
252
+ 'The expected shape is PART 12 of the Carve spec (resources/ast-schema.json), as written by "carve --to-json".');
253
+ }
254
+ return value;
255
+ }