@uniweb/semantic-parser 1.4.0 → 1.4.1

package/AGENTS.md

@@ -53,7 +53,7 @@ The parser returns a flat content structure:
 ```js
 {
   title: '',       // Main heading
-  pretitle: '',    // `#>` label line(s) or smaller headings above the title
+  pretitle: '',    // `#>` label line(s); also smaller headings above the title
   subtitle: '',    // Line(s) one step below the title (string or array)
   paragraphs: [],
   links: [],       // All link-like entities (including buttons, documents)
@@ -161,7 +161,7 @@ Lists maintain hierarchy through nested structure. The `processListItems()` func
 
 Key patterns — the staircase rule (each heading relates to the one before it):
 
-- **Pretitle**: `#>` label lines (headings with `role: "pretitle"`), and smaller headings stacked directly above a more important one (H3→H1, H6→H5, …). String or array.
+- **Pretitle**: `#>` label lines (headings with `role: "pretitle"`) — what to write. Also smaller headings stacked directly above a more important one (H3→H1, H6→H5, …), the older spelling, read for compatibility. String or array.
 - **Subtitle**: one step below the title, directly adjacent — and each further one-step descent (or same-size repeat) is another subtitle line. String or array.
 - **Items**: a heading two or more steps below the previous one, a step back up, or any heading after body content starts a new group.
 - **Banner Pattern**: an image (with banner role or followed by a heading) at the start of the first group stays with the headline.

package/README.md

@@ -66,7 +66,8 @@ Main content fields are at the top level. The `items` array contains additional
 result = {
   // Header fields (from headings) — each is a string, or an array when the
   // part spans several lines
-  pretitle: "",             // `#>` label line(s), or smaller headings above the title
+  pretitle: "",             // `#>` label line(s) — also smaller headings above
+                            //   the title, the older spelling
   title: "Welcome",         // Main heading
   subtitle: "",             // Line(s) one step below the title
 
@@ -180,7 +181,7 @@ This is the mechanism that creates repeating content groups (cards, features, FA
 Two spellings fill the `pretitle` slot:
 
 - **A `#>` label line** — a heading with `role: "pretitle"`. It names the block that starts next, at any depth, with no level arithmetic; the hash count is carried for round-tripping and means nothing.
-- **Position** — smaller headings stacked above the most important heading of a run (H3 before H1, H6 before H5, …), in any order. `#### / ### / # Title` and `## / ### / # Title` both yield a two-line pretitle.
+- **Position** — smaller headings stacked above the most important heading of a run (H3 before H1, H6 before H5, …), in any order. `#### / ### / # Title` and `## / ### / # Title` both yield a two-line pretitle. ⚖️ This is the older spelling and is read for compatibility; a label line is what to WRITE, because it carries its meaning alone rather than by what follows it.
 
 ### Banner image
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@uniweb/semantic-parser",
-  "version": "1.4.0",
+  "version": "1.4.1",
   "description": "Semantic parser for ProseMirror/TipTap content structures",
   "type": "module",
   "main": "./src/index.js",

package/src/builders/doc.js

@@ -6,6 +6,34 @@
  *
  * Starter content uses plain strings (no HTML marks), so the conversion
  * is straightforward — no need to reverse inline HTML formatting.
+ *
+ * ## ⭐ IT ACCEPTS THE PARSER'S OWN VOCABULARY TOO — and that is a contract
+ *
+ * This builder was written with its own spelling for several concepts, and
+ * `parseContent` emits a DIFFERENT one for the same thing: a link's text is
+ * `text` here and `label` there; an image's address is `src` here and `url`
+ * there; a video's cover is a string here and `{ src | url | identifier }`
+ * there; a list item is a plain string here and a whole content GROUP there.
+ *
+ * ⛔ EVERY ONE OF THOSE MISMATCHES FAILED SILENTLY, which is why the rule is
+ * now written down. Measured 2026-09-16, all four on the shapes a caller would
+ * most naturally reach for:
+ *
+ *   { links:  [{ label, href }] }        → buildDoc returns **null** outright
+ *   { images: [{ url, alt }] }           → an ImageBlock with NO url attr
+ *   { videos: [{ coverImg: '/p.jpg' }] } → the cover parses back as ""
+ *   { lists:  parsed.lists }             → a text node whose `text` is an
+ *                                          OBJECT — an invalid document
+ *
+ * None of them threw. A generator author reaches for `label` and `url` because
+ * those are the names in `docs/reference/component-metadata.md` and the names a
+ * component actually reads off `content` — so the natural spelling was the
+ * broken one, and it produced an empty section rather than an error.
+ *
+ * ⇒ For every concept where the two sides disagree, ACCEPT BOTH SPELLINGS. The
+ * emitted document is unchanged — this only widens what may come in — so the
+ * round-trip assertions above still pin the output. A new field whose name
+ * differs from the parser's needs the same treatment, or it joins this list.
  */
 
 // --- TipTap node builders ---
@@ -27,6 +55,33 @@ function heading(level, text) {
   }
 }
 
+/**
+ * A label line — `#> Text` — which is how a pretitle is written.
+ *
+ * ⛔ NOT A SMALLER HEADING BEFORE THE TITLE. That positional form still parses,
+ * and it is what this builder emitted until 2026-09-16, but it is the old
+ * spelling and it is fragile in a way the label line is not: it means "pretitle"
+ * only by sitting before a bigger heading, so the same line moved, or left
+ * alone, becomes something else. `#>` says what it is wherever it lands.
+ *
+ * ⭐ THE `#` COUNT MEANS NOTHING. `#>`, `##>` and `###>` are the same thing —
+ * content-reader keeps the authored count in `level` purely so a file
+ * round-trips byte-for-byte. We match the level of the title it labels, which is
+ * what an author writing by hand does (`##>` above an `##` item title).
+ *
+ * On the wire it is an ordinary heading carrying `role: 'pretitle'`, so every
+ * consumer that already understands headings keeps working.
+ */
+function labelLine(level, text) {
+  if (!text) return null
+  if (Array.isArray(text)) return text.map(t => labelLine(level, t)).filter(Boolean)
+  return {
+    type: 'heading',
+    attrs: { level, role: 'pretitle' },
+    content: [textNode(text)],
+  }
+}
+
 function paragraph(text) {
   if (!text) return null
   return {
@@ -35,18 +90,23 @@ function paragraph(text) {
   }
 }
 
-function linkParagraph({ text, href, target }) {
-  if (!text || !href) return null
+function linkParagraph({ text, label, href, target }) {
+  // `label` is what parseContent emits and what a component reads off
+  // `content.links[]`; `text` is this builder's original spelling.
+  const caption = text || label
+  if (!caption || !href) return null
   const mark = { type: 'link', attrs: { href } }
   if (target) mark.attrs.target = target
   return {
     type: 'paragraph',
-    content: [{ type: 'text', text, marks: [mark] }],
+    content: [{ type: 'text', text: caption, marks: [mark] }],
   }
 }
 
-function imageBlock({ src, alt = '', caption = '', direction, role, width, height }) {
-  const attrs = { url: src, alt }
+function imageBlock({ src, url, alt = '', caption = '', direction, role, width, height }) {
+  // The node attr is `url` and so is the parsed field; `src` is this builder's
+  // original spelling. Either one in, `url` out.
+  const attrs = { url: src || url, alt }
   if (caption) attrs.caption = caption
   if (direction) attrs.direction = direction
   if (role) attrs.role = role
@@ -81,11 +141,16 @@ function iconNode({ src, url, svg, library, name, size, color }) {
   return { type: 'UniwebIcon', attrs }
 }
 
-function videoNode({ src, caption, direction, coverImg }) {
-  const attrs = { src }
+function videoNode({ src, url, caption, direction, coverImg }) {
+  const attrs = { src: src || url }
   if (caption) attrs.caption = caption
   if (direction) attrs.direction = direction
-  if (coverImg) attrs.coverImg = coverImg
+  // ⛔ The editor's `Video` node holds `coverImg` as an OBJECT — the reader is
+  // `makeAssetUrl`, which looks for `.src` / `.url` / `.identifier`. A bare
+  // string passed straight through parses back as "": `info?.src` on a string
+  // is undefined. Normalize here so a caller may write either.
+  const cover = typeof coverImg === 'string' ? { src: coverImg } : coverImg
+  if (cover && (cover.src || cover.url || cover.identifier)) attrs.coverImg = cover
   return { type: 'Video', attrs }
 }
 
@@ -93,17 +158,75 @@ function dividerBlock() {
   return { type: 'DividerBlock' }
 }
 
+/**
+ * A code block for DISPLAY — what lands in `content.snippets`.
+ *
+ * `parseContent` groups one as `{ language, code }`; the element before
+ * grouping carries `{ text, attrs }`. Accept either spelling, and a bare string
+ * for a snippet with no language.
+ */
+function codeBlock(snippet) {
+  const spec = typeof snippet === 'string' ? { code: snippet } : snippet
+  if (!spec || typeof spec !== 'object') return null
+  const text = spec.code ?? spec.text
+  if (typeof text !== 'string' || text === '') return null
+  const attrs = {}
+  if (spec.language) attrs.language = spec.language
+  if (spec.filename) attrs.filename = spec.filename
+  return { type: 'codeBlock', attrs, content: [textNode(text)] }
+}
+
+/**
+ * A tagged data block — what lands in `content.data[tag]`.
+ *
+ * ⛔ IT IS A `dataBlock`, NOT A TAGGED `codeBlock`, and the difference is not
+ * cosmetic. content-reader emits `dataBlock` when a tagged fence PARSED, with
+ * the parsed value on `attrs.data`; it falls back to a tagged `codeBlock`
+ * carrying raw TEXT only when parsing failed. A builder handed a real object
+ * has parsed data by definition, so emitting the failure shape would hand a
+ * component a string where it expects a record.
+ *
+ * ⭐ `language` IS LOAD-BEARING, not decoration. content-writer reads it back to
+ * choose the fence, and the parsed value alone cannot say whether the author
+ * wrote YAML or JSON — without it an author's YAML silently becomes JSON on the
+ * next editor sync. A builder has no author to echo, so it states one.
+ */
+function dataBlock(tag, value, language) {
+  // A null value is the same absence as no key at all — a component reading
+  // `content.data[tag]` gets null either way — so emit no fence rather than one
+  // whose whole payload is `null`.
+  if (!tag || value === undefined || value === null) return null
+  return { type: 'dataBlock', attrs: { tag, language, data: value } }
+}
+
 function bulletList(items) {
   if (!items || !items.length) return null
+  const entries = items.map(listItemText).filter(Boolean)
+  if (!entries.length) return null
   return {
     type: 'bulletList',
-    content: items.map(item => ({
+    content: entries.map(text => ({
       type: 'listItem',
-      content: [paragraph(item)].filter(Boolean),
+      content: [paragraph(text)].filter(Boolean),
     })),
   }
 }
 
+/**
+ * One list entry's text. A generator writes plain strings; `parseContent`
+ * emits a full content GROUP per entry ({ paragraphs, links, … }), and feeding
+ * that back produced a text node whose `text` was an object — an invalid
+ * document that threw nowhere.
+ */
+function listItemText(item) {
+  if (typeof item === 'string') return item
+  if (!item || typeof item !== 'object') return ''
+  if (Array.isArray(item.paragraphs) && item.paragraphs.length) {
+    return item.paragraphs.join(' ')
+  }
+  return item.title || ''
+}
+
 // --- Group builder ---
 
 /**
@@ -113,14 +236,13 @@ function bulletList(items) {
  * @param {number} titleLevel - Heading level for title (1 for main, 2 for items)
  * @returns {Array} Array of TipTap nodes
  */
-function buildGroupNodes(group, titleLevel = 1) {
+function buildGroupNodes(group, titleLevel = 1, options = {}) {
   const nodes = []
 
-  // 1. Headings: pretitle → title → subtitle
-  // Pretitle uses a higher level number (less important) than title
-  // e.g., H3 before H1 — mirrors isPreTitle() in groups.js
+  // 1. Headings: pretitle → title → subtitle.
+  // A pretitle is a LABEL LINE (`#> Text`), at the level of the title it labels.
   if (group.pretitle) {
-    const pre = heading(titleLevel + 2, group.pretitle)
+    const pre = labelLine(titleLevel, group.pretitle)
     if (Array.isArray(pre)) nodes.push(...pre)
     else if (pre) nodes.push(pre)
   }
@@ -178,6 +300,29 @@ function buildGroupNodes(group, titleLevel = 1) {
     }
   }
 
+  if (group.snippets) {
+    for (const snippet of group.snippets) {
+      const node = codeBlock(snippet)
+      if (node) nodes.push(node)
+    }
+  }
+
+  // `data` is a MAP of tag → value, not a list: one tagged fence per key, in
+  // declaration order.
+  //
+  // ⚠️ A `concept_block` (```md:faq) also lands in `content.data`, as
+  // `{ items, sequence }` — a tagged PROSE fence, not data. This does not
+  // reconstruct one: it emits the data fence, whose payload round-trips to the
+  // same value. Prose that was parsed into a document does not come back as
+  // prose here, and a caller rebuilding a concept block wants the markdown.
+  if (group.data && typeof group.data === 'object' && !Array.isArray(group.data)) {
+    const language = options.dataLanguage || 'yaml'
+    for (const [tag, value] of Object.entries(group.data)) {
+      const node = dataBlock(tag, value, language)
+      if (node) nodes.push(node)
+    }
+  }
+
   return nodes
 }
 
@@ -191,22 +336,26 @@ function buildGroupNodes(group, titleLevel = 1) {
  * roundtrips through parseContent() to yield the same structure.
  *
  * @param {Object} content - Content structure (same shape as parseContent output / starter)
+ * @param {Object} [options]
+ * @param {string} [options.dataLanguage='yaml'] - the fence a tagged data block
+ *   is written as. It is recorded on the node because content-writer reads it
+ *   back to choose the fence, and a parsed value cannot say which one it was.
  * @returns {Object|null} TipTap document { type: 'doc', content: [...] }, or null if empty
  */
-function buildDoc(content) {
+function buildDoc(content, options = {}) {
   if (!content) return null
 
   const nodes = []
 
   // Main group content (title level 1)
-  nodes.push(...buildGroupNodes(content, 1))
+  nodes.push(...buildGroupNodes(content, 1, options))
 
   // Items: separated by DividerBlock (mirrors divider-based grouping in groups.js)
   if (content.items && content.items.length > 0) {
     for (const item of content.items) {
       nodes.push(dividerBlock())
       // Item headings use level 2 (one below main H1)
-      nodes.push(...buildGroupNodes(item, 2))
+      nodes.push(...buildGroupNodes(item, 2, options))
     }
   }