@uniweb/semantic-parser 1.2.3 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -12,17 +12,14 @@ This is a semantic parser for ProseMirror/TipTap content structures. It transfor
12
12
  # Run all tests
13
13
  npm test
14
14
 
15
- # Run tests with JSON report output
16
- npm run test-report
17
-
18
15
  # Run a specific test file
19
- npx jest tests/parser.test.js
16
+ npx vitest run tests/parser.test.js
20
17
 
21
18
  # Run tests in watch mode
22
- npx jest --watch
19
+ npx vitest
23
20
 
24
21
  # Run a specific test by name
25
- npx jest -t "handles simple document structure"
22
+ npx vitest run -t "handles simple document structure"
26
23
  ```
27
24
 
28
25
  ## Architecture
@@ -33,9 +30,7 @@ The parser processes content through three distinct stages, each building on the
33
30
 
34
31
  1. **Sequence Processing** (`src/processors/sequence.js`): Flattens the ProseMirror document tree into a linear sequence of semantic elements (headings, paragraphs, images, lists, etc.)
35
32
 
36
- 2. **Groups Processing** (`src/processors/groups.js`): Transforms the sequence into semantic groups with identified main content and items. Supports two grouping modes:
37
- - Heading-based grouping (default)
38
- - Divider-based grouping (when horizontal rules are present)
33
+ 2. **Groups Processing** (`src/processors/groups.js`): Transforms the sequence into semantic groups with identified main content and items. Headings split groups via the staircase headline rule; a horizontal rule explicitly closes the current group — the two compose, there are no separate modes.
39
34
 
40
35
  3. **ByType Processing** (`src/processors/byType.js`): Organizes elements by type with positional context, enabling type-specific queries
41
36
 
@@ -47,8 +42,7 @@ const result = parseContent(doc);
47
42
  // {
48
43
  // raw: doc, // Original ProseMirror document
49
44
  // sequence: [...], // Flat sequence of elements
50
- // groups: {...}, // Semantic groups with main/items
51
- // byType: {...} // Elements organized by type
45
+ // title, pretitle, subtitle, paragraphs, items, ... // flat content fields
52
46
  // }
53
47
  ```
54
48
 
@@ -59,8 +53,8 @@ The parser returns a flat content structure:
59
53
  ```js
60
54
  {
61
55
  title: '', // Main heading
62
- pretitle: '', // Heading before main title
63
- subtitle: '', // Heading after main title
56
+ pretitle: '', // `#>` label line(s) or smaller headings above the title
57
+ subtitle: '', // Line(s) one step below the title (string or array)
64
58
  paragraphs: [],
65
59
  links: [], // All link-like entities (including buttons, documents)
66
60
  images: [],
@@ -70,7 +64,7 @@ The parser returns a flat content structure:
70
64
  quotes: [],
71
65
  snippets: [], // Fenced code — [{ language, code }]
72
66
  data: {}, // Structured data (tagged data blocks, forms, cards)
73
- headings: [], // Headings after subtitle, in document order
67
+ headings: [], // Only from nested content (quote/list bodies)
74
68
  items: [], // Child content groups (same structure recursively)
75
69
  }
76
70
  ```
@@ -106,10 +100,10 @@ FormBlock without a `schemaId` still lands at `data.form`.
106
100
 
107
101
  ### Main Content Identification
108
102
 
109
- The `identifyMainContent()` function (src/processors/groups.js:282) determines if the first group should be treated as main content:
103
+ The `identifyMainContent()` function in `src/processors/groups.js` determines if the first group should be treated as main content:
110
104
  - Single group is always main content
111
- - First group must have lower heading level than second group
112
- - Divider mode affects main content identification
105
+ - First group must have a more important (lower) heading level than the second group
106
+ - A first group with no heading at all (body before the first heading) is also promoted to main
113
107
 
114
108
  ### Special Element Detection
115
109
 
@@ -165,14 +159,15 @@ Lists maintain hierarchy through nested structure. The `processListItems()` func
165
159
 
166
160
  ## Content Writing Conventions
167
161
 
168
- Key patterns:
162
+ Key patterns — the staircase rule (each heading relates to the one before it):
169
163
 
170
- - **Pretitle Pattern**: Any heading followed by a more important heading (e.g., H3→H1, H2→H1, H6→H5, etc.)
171
- - **Banner Pattern**: Image (with banner role or followed by heading) at start of first group
172
- - **Divider Mode**: Presence of any `horizontalRule` switches entire document to divider-based grouping
173
- - **Heading Groups**: Consecutive headings are consumed together only when each is exactly one level deeper (H1→H2 yes, H1→H3 no — skipped levels start a new group)
174
- - **Main Content**: First group is main if it's the only group OR has lower heading level than second group
175
- - **Body Headings**: Headings after the title and subtitle slots are collected in `body.headings` in document order
164
+ - **Pretitle**: `#>` label lines (headings with `role: "pretitle"`), and smaller headings stacked directly above a more important one (H3→H1, H6→H5, …). String or array.
165
+ - **Subtitle**: one step below the title, directly adjacent — and each further one-step descent (or same-size repeat) is another subtitle line. String or array.
166
+ - **Items**: a heading two or more steps below the previous one, a step back up, or any heading after body content starts a new group.
167
+ - **Banner Pattern**: an image (with banner role or followed by a heading) at the start of the first group stays with the headline.
168
+ - **Dividers**: a `horizontalRule` explicitly closes the current group; it composes with the heading rules rather than replacing them.
169
+ - **Main Content**: the first group is main if it's the only group, has a more important heading level than the second group, or has no heading at all.
170
+ - **Body Headings**: `body.headings` fills only from nested content (blockquote children, list items) — a section's headline never spills there.
176
171
 
177
172
  ## Testing Structure
178
173
 
@@ -189,4 +184,4 @@ Tests are organized by processor:
189
184
  - The parser never modifies the original ProseMirror document
190
185
  - Text content can include inline HTML for formatting (bold → `<strong>`, italic → `<em>`, links → `<a>`)
191
186
  - Context information in byType includes position, previous/next elements, and nearest heading
192
- - Group splitting logic differs significantly between heading mode and divider mode
187
+ - Headline slots and group boundaries come from one walk (`readStack`), so the two cannot disagree
package/README.md CHANGED
@@ -64,10 +64,11 @@ Main content fields are at the top level. The `items` array contains additional
64
64
 
65
65
  ```js
66
66
  result = {
67
- // Header fields (from headings)
68
- pretitle: "", // Heading before main title
67
+ // Header fields (from headings) — each is a string, or an array when the
68
+ // part spans several lines
69
+ pretitle: "", // `#>` label line(s), or smaller headings above the title
69
70
  title: "Welcome", // Main heading
70
- subtitle: "", // Heading after main title
71
+ subtitle: "", // Line(s) one step below the title
71
72
 
72
73
  // Body fields
73
74
  paragraphs: ["Get started today."],
@@ -80,7 +81,7 @@ result = {
80
81
  insets: [], // Inline @Component references — { refId }
81
82
  snippets: [], // Fenced code blocks — { language, code }
82
83
  data: {}, // Structured data (tagged data blocks, forms, cards)
83
- headings: [], // Headings after subtitle, in document order
84
+ headings: [], // Only from nested content (quote/list bodies)
84
85
 
85
86
  // Additional content groups (from headings after content)
86
87
  // Each item has the SAME flat structure as the top level — title,
@@ -108,7 +109,7 @@ const content = parseContent(doc);
108
109
 
109
110
  const title = content.title;
110
111
  const description = content.paragraphs.join(" ");
111
- const image = content.banner?.url;
112
+ const image = content.images[0]?.src;
112
113
  ```
113
114
 
114
115
  ### Processing Content Sections
@@ -155,13 +156,13 @@ The parser interprets heading levels **relatively**, not absolutely. There is no
155
156
 
156
157
  A new group is started whenever a heading appears after non-heading content (paragraphs, images, links, lists, etc.). A horizontal rule (`---`) explicitly closes the current group.
157
158
 
158
- When a group begins with one or more headings, the parser consumes a contiguous **heading block** following these rules:
159
+ When a group begins with headings, the parser reads one **headline stack** with the staircase rule — each heading relates to the one before it:
159
160
 
160
- 1. **Adjacent deeper** — the next heading is exactly one level deeper (e.g., H1→H2, H2→H3). It becomes the `subtitle`. Skipping levels (H1→H3) breaks the block; the deeper heading starts a new group as an item.
161
- 2. **Pretitle promotion** — if the very first heading is followed by a *more important* (lower-numbered) heading, the first one becomes `pretitle` and the next becomes `title`. This only applies at the start of the block.
162
- 3. **Same-level continuation** — consecutive headings at the same level merge into an array (multi-line title or subtitle). This stops once a deeper level has been reached: after going deeper, a same-level heading starts a new group instead of merging.
161
+ 1. **Same size** — another line of the same part: a split title, or another subtitle line. Merged slots become arrays.
162
+ 2. **One step smaller** — the next line of the headline: the subtitle first, then further subtitle lines (`subtitle` is a string for one line, an array for several — a three-line header like name / role / affiliation fits in one headline).
163
+ 3. **Two or more steps smaller, or a step back up** — the stack ends; the rest of the run starts a new group (an item), read the same way.
163
164
 
164
- Anything else (going back up after going deeper, or any other gap) breaks the block.
165
+ Before the title, `#>` label lines (headings carrying `role: "pretitle"`) and smaller headings ascending to a more important one all join the `pretitle`. A label line anywhere names the block it sits in; a label with no following heading labels an untitled block.
165
166
 
166
167
  ### Main content vs items
167
168
 
@@ -176,14 +177,10 @@ This is the mechanism that creates repeating content groups (cards, features, FA
176
177
 
177
178
  ### Pretitle detection
178
179
 
179
- A heading followed by a more important heading at the start of a group is detected as `pretitle`:
180
+ Two spellings fill the `pretitle` slot:
180
181
 
181
- - H3 before H1 → pretitle
182
- - H2 before H1 → pretitle
183
- - H6 before H5 → pretitle
184
- - H4 before H2 → pretitle
185
-
186
- Pretitle is detected only between the first two headings of a heading block (before the title is set).
182
+ - **A `#>` label line** — a heading with `role: "pretitle"`. It names the block that starts next, at any depth, with no level arithmetic; the hash count is carried for round-tripping and means nothing.
183
+ - **Position** — headings followed by a more important heading at the start of a group (H3 before H1, H6 before H5, …). Stacked smaller headings all join: `#### / ### / # Title` yields a two-line pretitle.
187
184
 
188
185
  ### Banner image
189
186
 
@@ -193,7 +190,7 @@ If a section's very first element is an image (or an image followed immediately
193
190
 
194
191
  A horizontal rule (`---`) explicitly closes the current group and starts a new one. Dividers compose with the heading rules above — they don't replace them — and are useful in three situations:
195
192
 
196
- - **Resolving ambiguity.** When the heading rules would group content one way but you want it grouped another way. Most common case: forcing items without a subtitle. Without a divider, `# Our Stats` followed by `## 15,000+` makes `## 15,000+` the subtitle (one level deeper). Adding `---` between them closes the title group so `## 15,000+` becomes `items[0].title` instead.
193
+ - **Resolving ambiguity.** When the heading rules would group content one way but you want it grouped another way. Most common case: forcing items without a subtitle. Without a divider, `# Our Stats` followed by `## 15,000+` makes `## 15,000+` the subtitle (one step down). Adding `---` between them closes the headline so `## 15,000+` becomes `items[0].title` instead — as does writing the entries two steps down (`### 15,000+`), the divider-free spelling.
197
194
  - **Forcing splits within same-level runs.** Same-level headings that would otherwise merge into a multi-line title/subtitle become separate groups when separated by a divider.
198
195
  - **Personal preference.** Even when the heading rules would already produce the desired structure, authors can use `---` as an explicit visual separator between groups. It never changes a structure that's already correct — it just makes the boundary obvious in the markdown source.
199
196
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@uniweb/semantic-parser",
3
- "version": "1.2.3",
3
+ "version": "1.3.0",
4
4
  "description": "Semantic parser for ProseMirror/TipTap content structures",
5
5
  "type": "module",
6
6
  "main": "./src/index.js",
package/src/index.js CHANGED
@@ -1,6 +1,7 @@
1
1
  import { processSequence, resolveAssetUrl, ASSET_SLOTS } from "./processors/sequence.js";
2
2
  import { processGroups } from "./processors/groups.js";
3
3
  import { buildDoc } from "./builders/doc.js";
4
+ import { lintContent } from "./lint.js";
4
5
 
5
6
  /**
6
7
  * Parse ProseMirror/TipTap content into semantic structure
@@ -50,4 +51,7 @@ function parseContent(doc, options = {}) {
50
51
  // parse (the editor, a kit component) reaches THIS implementation rather than
51
52
  // writing a second one. A duplicated selector is this repo's most-repeated
52
53
  // defect; one exported function is the cheapest guard against it.
53
- export { parseContent, buildDoc, resolveAssetUrl, ASSET_SLOTS };
54
+ // `lintContent` diagnoses grouping near-misses (see src/lint.js). It never
55
+ // affects a parse — parsing stays deterministic; the judgment lives here so
56
+ // the CLI, the dev server, and an editor surface the same findings.
57
+ export { parseContent, buildDoc, resolveAssetUrl, ASSET_SLOTS, lintContent };
package/src/lint.js ADDED
@@ -0,0 +1,140 @@
1
+ /**
2
+ * @fileoverview Content lint — diagnose grouping near-misses.
3
+ *
4
+ * The grouping rules are deterministic, but a handful of authored shapes are
5
+ * genuinely ambiguous: the same markdown can mean two things, and whichever
6
+ * reading the parser takes, an author who meant the other gets a silently
7
+ * reshaped section. Parsing never guesses; THIS module is where the judgment
8
+ * lives — it looks at a document the way a reviewer would and names the shapes
9
+ * that usually indicate a misfire, without ever changing the parse.
10
+ *
11
+ * Severities:
12
+ * - "warning" — the shape almost always means a misfire (measured across the
13
+ * intent corpus and real site content; see tests/lint.test.js).
14
+ * - "hint" — the shape is ambiguous by construction. The message says what the
15
+ * parser did and how to spell the other meaning; correct content simply
16
+ * ignores it. Hints trade a known false-fire rate for catching the most
17
+ * common naive spellings — the corpus records both sides.
18
+ *
19
+ * Environment-free on purpose: pure functions over the document, no I/O, so
20
+ * the CLI, the dev server, and an editor can all surface the same findings.
21
+ */
22
+
23
+ import { processSequence } from "./processors/sequence.js";
24
+ import { splitBySlices, readStack } from "./processors/groups.js";
25
+
26
+ const stripTags = (s) => String(s ?? "").replace(/<[^>]+>/g, "");
27
+ const excerpt = (s) => {
28
+ const t = stripTags(s).trim();
29
+ return t.length > 40 ? t.slice(0, 39) + "…" : t;
30
+ };
31
+
32
+ /**
33
+ * Lint a ProseMirror document for grouping near-misses.
34
+ *
35
+ * @param {Object} doc ProseMirror document
36
+ * @param {Object} [options] The same options a `parseContent` call would use —
37
+ * findings must describe the parse the consumer actually gets.
38
+ * @returns {Array<{rule: string, severity: "warning"|"hint", message: string}>}
39
+ */
40
+ export function lintContent(doc, options = {}) {
41
+ const sequence = processSequence(doc, options);
42
+ const groups = splitBySlices(sequence, options);
43
+ const stacks = groups.map((g) =>
44
+ g[0]?.type === "heading" ? readStack(g, 0, options) : null
45
+ );
46
+ const findings = [];
47
+
48
+ // ---- label-as-text: a short bare line above the first heading ----------
49
+ // `**New**` above `# Title` turns the section into an untitled main with
50
+ // the title demoted to items[0] — the most destructive naive spelling.
51
+ // Real intro text is left alone: it runs longer and ends like a sentence.
52
+ if (stacks[0] === null && stacks.length > 1) {
53
+ const first = groups[0];
54
+ const laterTitled = stacks.some((s) => s && s.title.length);
55
+ if (first.length === 1 && first[0].type === "paragraph" && laterTitled) {
56
+ const text = stripTags(first[0].text).trim();
57
+ const words = text.split(/\s+/).filter(Boolean);
58
+ if (words.length > 0 && words.length <= 6 && !/[.!?]$/.test(text)) {
59
+ findings.push({
60
+ rule: "label-as-text",
61
+ severity: "warning",
62
+ message:
63
+ `"${excerpt(text)}" sits above the first heading, so the section became an ` +
64
+ `untitled body and the heading below it starts the items. If that line is ` +
65
+ `a label for the title, write it as a label line: #> ${text}`,
66
+ });
67
+ }
68
+ }
69
+ }
70
+
71
+ // ---- per-group checks against the groups that follow -------------------
72
+ for (let i = 0; i < stacks.length; i++) {
73
+ const stack = stacks[i];
74
+ if (!stack) continue;
75
+
76
+ const laterTitleLevels = new Set(
77
+ stacks
78
+ .slice(i + 1)
79
+ .filter(Boolean)
80
+ .map((s) => s.title[0]?.level)
81
+ .filter((l) => l != null)
82
+ );
83
+
84
+ const subLevels = stack.sub.map((el) => el.level);
85
+
86
+ // entry-joined-headline: the headline absorbed two or more lines and
87
+ // at least one sits at the same size as a later block's title. Every
88
+ // measured real-world instance of this shape was a misfire (the first
89
+ // stat, feature, testimonial, or job folded into the headline).
90
+ if (
91
+ stack.sub.length >= 2 &&
92
+ subLevels.some((l) => laterTitleLevels.has(l))
93
+ ) {
94
+ const absorbed = stack.sub[stack.sub.length - 1];
95
+ findings.push({
96
+ rule: "entry-joined-headline",
97
+ severity: "warning",
98
+ message:
99
+ `The headline under "${excerpt(stack.title[0]?.text)}" absorbed ` +
100
+ `${stack.sub.length} lines, ending with "${excerpt(absorbed.text)}", while ` +
101
+ `later headings of the same size start items. If those lines are entries, ` +
102
+ `step them two sizes below the headline, or put a lead paragraph (or ---) ` +
103
+ `after it.`,
104
+ });
105
+ continue; // one finding per stack — the hint below would repeat it
106
+ }
107
+
108
+ // first-entry-or-subtitle: a single line one step under the title,
109
+ // with same-size headings starting items later. Ambiguous by
110
+ // construction — a platform hero reads exactly like a question list —
111
+ // so this only states what happened and how to spell the other intent.
112
+ if (stack.sub.length === 1 && laterTitleLevels.has(subLevels[0])) {
113
+ findings.push({
114
+ rule: "first-entry-or-subtitle",
115
+ severity: "hint",
116
+ message:
117
+ `"${excerpt(stack.sub[0].text)}" joined the headline as the subtitle, while ` +
118
+ `later headings of the same size start items. If it is the subtitle, ignore ` +
119
+ `this. If it was meant as the first entry, step the entries two sizes below ` +
120
+ `the headline or put a lead paragraph before the first one.`,
121
+ });
122
+ }
123
+
124
+ // merged-title-run: three or more bodiless same-size headings read as
125
+ // one multi-line title. Two lines is the classic split headline; three
126
+ // or more usually means a bare list (names, logos, tags).
127
+ if (stack.title.length >= 3) {
128
+ findings.push({
129
+ rule: "merged-title-run",
130
+ severity: "hint",
131
+ message:
132
+ `${stack.title.length} same-size headings merged into one multi-line title ` +
133
+ `starting "${excerpt(stack.title[0]?.text)}". If these are separate items, ` +
134
+ `give each one body content, separate them with ---, or write them as a list.`,
135
+ });
136
+ }
137
+ }
138
+
139
+ return findings;
140
+ }
@@ -46,8 +46,8 @@ function flattenGroup(group) {
46
46
  * render time and at editor time — and nothing derived is ever stored, so a
47
47
  * change to a grouping rule retroactively re-reads every document that already
48
48
  * exists. Leaking these suppressions into the default path would silently
49
- * reshape every section body ever authored. `tests/grouping-default-path.test.js`
50
- * pins the default against that.
49
+ * reshape every section body ever authored.
50
+ * `tests/processors/always-items.test.js` pins the default against that.
51
51
  *
52
52
  * @param {Array} sequence Flat sequence of elements
53
53
  * @param {Object} options Parsing options
@@ -79,7 +79,9 @@ function processGroups(sequence, options = {}) {
79
79
  const groups = splitBySlices(sequence, options);
80
80
 
81
81
  // Process each group's structure (still nested internally)
82
- const processedGroups = groups.map((group) => processGroupContent(group));
82
+ const processedGroups = groups.map((group) =>
83
+ processGroupContent(group, options)
84
+ );
83
85
 
84
86
  // Determine main vs items
85
87
  let mainGroup = null;
@@ -148,21 +150,23 @@ function splitBySlices(sequence, options = {}) {
148
150
  // we do NOT close the group. We let the heading merge with the image.
149
151
  const isBannerMerge = i === 1 && isBannerImage(sequence, 0);
150
152
 
151
- // A new Heading Group starts a new visual block.
153
+ // A new headline stack starts a new visual block.
152
154
  // If we have gathered content in the current group, close it now.
153
155
  if (currentGroup.length > 0 && !isBannerMerge) {
154
156
  groups.push(currentGroup);
155
157
  currentGroup = [];
156
158
  }
157
159
 
158
- // Consume the entire semantic heading block (Title + Subtitles)
159
- // We reuse your smart readHeadingGroup logic here!
160
- const headingBlock = readHeadingGroup(sequence, i, options);
161
- currentGroup.push(...headingBlock);
160
+ // Consume exactly ONE headline stack (pretitle + title + subtitle
161
+ // lines). A longer contiguous run re-enters here on the next
162
+ // iteration and opens the next group.
163
+ const stack = readStack(sequence, i, options);
164
+ const consumed = Math.max(stack.length, 1);
165
+ currentGroup.push(...sequence.slice(i, i + consumed));
162
166
 
163
167
  // Advance the index by the number of headings consumed
164
- // (Loop increments i by 1, so we add length - 1)
165
- i += headingBlock.length - 1;
168
+ // (Loop increments i by 1, so we add consumed - 1)
169
+ i += consumed - 1;
166
170
  } else {
167
171
  // 3. Handle Content (Body)
168
172
  // Paragraphs, images, lists, etc. just append to the current slice.
@@ -179,16 +183,11 @@ function splitBySlices(sequence, options = {}) {
179
183
  }
180
184
 
181
185
  /**
182
- * Check if this is a pretitle - any heading followed by a more important heading
183
- * (e.g., H3→H1, H2→H1, H6→H5, etc.)
186
+ * A `#>` label line — a heading carrying `role: "pretitle"`. It names the
187
+ * block it opens (or sits in) and never competes for the title slot.
184
188
  */
185
- function isPreTitle(sequence, i) {
186
- return (
187
- i + 1 < sequence.length &&
188
- sequence[i].type === "heading" &&
189
- sequence[i + 1].type === "heading" &&
190
- sequence[i].level > sequence[i + 1].level // Smaller heading before larger
191
- );
189
+ function isLabel(el) {
190
+ return el?.type === "heading" && el.attrs?.role === "pretitle";
192
191
  }
193
192
 
194
193
  function isBannerImage(sequence, i) {
@@ -200,69 +199,95 @@ function isBannerImage(sequence, i) {
200
199
  );
201
200
  }
202
201
 
203
- function readHeadingGroup(sequence, startIdx, options = {}) {
204
- const elements = [sequence[startIdx]];
205
- let hasGoneDeeper = false;
206
-
207
- // Iterate starting from the next element
208
- for (let i = startIdx + 1; i < sequence.length; i++) {
209
- const element = sequence[i];
210
- const previousElement = elements[elements.length - 1];
211
-
212
- if (element.type !== "heading") {
213
- break;
214
- }
202
+ /**
203
+ * Read ONE headline stack from a contiguous heading run — the staircase rule:
204
+ * each heading relates to the one before it. The same size extends the current
205
+ * part; one step smaller is the next line of the headline (the subtitle, then
206
+ * further subtitle lines); anything else — two or more steps down, or a step
207
+ * back up — ends the stack, and the rest of the run starts a new block.
208
+ *
209
+ * Before the title: `#>` label lines, and any smaller headings ascending to a
210
+ * more important one, are the pretitle. A label line anywhere in the stack
211
+ * also names the block — it never competes for the title.
212
+ *
213
+ * `alwaysItems` (a tagged concept block's grouping) suppresses the same-size
214
+ * continuation: two bodiless same-level headings are two ITEMS, not one
215
+ * heading split across visual lines. That suppression must happen HERE, at
216
+ * group formation — no amount of suppressing title promotion downstream can
217
+ * split a group that was never split. Descents still join: a concept-block
218
+ * item keeps its subtitle.
219
+ *
220
+ * @returns {{ pre: Array, title: Array, sub: Array, length: number }}
221
+ * The slot elements and how many sequence elements the stack consumed.
222
+ */
223
+ function readStack(sequence, startIdx, options = {}) {
224
+ const pre = [];
225
+ const title = [];
226
+ const sub = [];
227
+ let i = startIdx;
228
+
229
+ // Leading `#>` label lines belong to this block's pretitle.
230
+ while (i < sequence.length && isLabel(sequence[i])) {
231
+ pre.push(sequence[i]);
232
+ i++;
233
+ }
215
234
 
216
- // Case 1: Adjacent Deeper (Standard Subtitle/Deep Header)
217
- // Only group headings that are exactly one level apart.
218
- // e.g. H1 -> H2 groups, but H1 -> H3 does not (skipped level
219
- // signals a structural tier change, not a title-subtitle pair).
220
- if (element.level - previousElement.level === 1) {
221
- hasGoneDeeper = true;
222
- elements.push(element);
223
- continue;
224
- }
235
+ // Ascending positional pretitle: smaller headings stacked directly above
236
+ // a more important one (e.g. #### / ### / # Title) all join the pretitle.
237
+ while (
238
+ i + 1 < sequence.length &&
239
+ sequence[i].type === "heading" &&
240
+ sequence[i + 1].type === "heading" &&
241
+ !isLabel(sequence[i + 1]) &&
242
+ sequence[i + 1].level < sequence[i].level
243
+ ) {
244
+ pre.push(sequence[i]);
245
+ i++;
246
+ }
225
247
 
226
- // Case 2: Pretitle Promotion (Small -> Big)
227
- // Only allowed if we haven't gone deep yet (length is 1)
228
- // e.g. H2 -> H1
229
- if (elements.length === 1 && element.level < previousElement.level) {
230
- elements.push(element);
231
- continue;
232
- }
248
+ // A label-only stack (labels followed by body, or nothing) is legal: the
249
+ // block has a pretitle and no title.
250
+ if (i >= sequence.length || sequence[i].type !== "heading") {
251
+ return { pre, title, sub, length: i - startIdx };
252
+ }
233
253
 
234
- // Case 3: Same Level Continuation (multi-line heading)
235
- // Only before going deeper — once a subtitle level is reached,
236
- // same-level headings are new sections, not continuations.
237
- // e.g. H1 -> H1 → merged into title array
238
- // but H1 -> H2 -> H2 → second H2 starts a new group (items)
239
- //
240
- // Suppression 2 of 2 (`alwaysItems`): in a tagged concept block two
241
- // same-level headings are two ITEMS, not one heading split across
242
- // visual lines. Merging them is why bodiless headings collapse into a
243
- // single group carrying a title ARRAY — and no amount of suppressing
244
- // title promotion downstream can split what was never split here, which
245
- // is what makes this the second half of a fix that looks like one line.
246
- if (
247
- element.level === previousElement.level &&
248
- !hasGoneDeeper &&
249
- !options.alwaysItems
250
- ) {
251
- elements.push(element);
252
- continue;
254
+ title.push(sequence[i]);
255
+ let prevLevel = sequence[i].level;
256
+ let part = title;
257
+ i++;
258
+
259
+ while (i < sequence.length && sequence[i].type === "heading") {
260
+ const el = sequence[i];
261
+ if (isLabel(el)) {
262
+ // A label inside the stack still names this block.
263
+ pre.push(el);
264
+ i++;
265
+ } else if (el.level === prevLevel && !options.alwaysItems) {
266
+ part.push(el); // another line of the same part
267
+ i++;
268
+ } else if (el.level === prevLevel + 1) {
269
+ sub.push(el); // the next line of the headline, one step down
270
+ part = sub;
271
+ prevLevel = el.level;
272
+ i++;
273
+ } else {
274
+ break; // two+ steps down, or back up: a new block starts here
253
275
  }
254
-
255
- // Otherwise (New Section — went deeper then back up, or
256
- // same-level after going deeper), stop.
257
- break;
258
276
  }
259
- return elements;
277
+ return { pre, title, sub, length: i - startIdx };
278
+ }
279
+
280
+ /** Collapse a slot's texts: none → '', one → string, several → array. */
281
+ function flattenSlot(texts) {
282
+ if (texts.length === 0) return "";
283
+ if (texts.length === 1) return texts[0];
284
+ return texts;
260
285
  }
261
286
 
262
287
  /**
263
288
  * Process a group's content to identify its structure
264
289
  */
265
- function processGroupContent(elements) {
290
+ function processGroupContent(elements, options = {}) {
266
291
  const header = {
267
292
  pretitle: "",
268
293
  title: "",
@@ -298,46 +323,33 @@ function processGroupContent(elements) {
298
323
  metadata,
299
324
  };
300
325
 
301
- // Track last assigned heading slot and level for same-level merging
302
- let lastSlot = null;
303
- let lastLevel = null;
304
-
305
- for (let i = 0; i < elements.length; i++) {
306
- //We shuold only set pretitle once
307
- if (isPreTitle(elements, i) && !header.pretitle) {
308
- header.pretitle = elements[i].text;
309
- i++; // move to known next heading (H1 or h2)
326
+ // The group's leading heading run is its headline stack: read the slots
327
+ // with the same staircase rule that decided the group boundary, so the
328
+ // two walks cannot disagree.
329
+ let start = 0;
330
+ if (elements[0]?.type === "heading") {
331
+ const stack = readStack(elements, 0, options);
332
+ header.pretitle = flattenSlot(stack.pre.map((el) => el.text));
333
+ header.title = flattenSlot(stack.title.map((el) => el.text));
334
+ header.subtitle = flattenSlot(stack.sub.map((el) => el.text));
335
+ metadata.level = stack.title[0]?.level ?? null;
336
+ for (const el of [...stack.pre, ...stack.title, ...stack.sub]) {
337
+ if (el.children && Array.isArray(el.children))
338
+ processInlineElements(el.children, body);
310
339
  }
340
+ start = Math.max(stack.length, 1);
341
+ }
311
342
 
343
+ for (let i = start; i < elements.length; i++) {
312
344
  const element = elements[i];
313
345
 
314
346
  if (element.type === "heading") {
347
+ // Only reachable for nested content (blockquote children, list
348
+ // items), where headings can follow body inside one element
349
+ // array — section content is split into groups before this runs.
315
350
  if (element.children && Array.isArray(element.children))
316
351
  processInlineElements(element.children, body);
317
-
318
- //We shuold set the group level to the highest one instead of the first one.
319
- metadata.level ??= element.level;
320
-
321
- // Same level as last assigned → merge into same slot as array
322
- if (lastLevel !== null && element.level === lastLevel && lastSlot) {
323
- const current = header[lastSlot];
324
- if (Array.isArray(current)) {
325
- current.push(element.text);
326
- } else {
327
- header[lastSlot] = [current, element.text];
328
- }
329
- } else if (!header.title) {
330
- header.title = element.text;
331
- lastSlot = 'title';
332
- } else if (!header.subtitle) {
333
- header.subtitle = element.text;
334
- lastSlot = 'subtitle';
335
- } else {
336
- // After subtitle, remaining headings go to body
337
- body.headings.push(element.text);
338
- lastSlot = null;
339
- }
340
- lastLevel = element.level;
352
+ body.headings.push(element.text);
341
353
  } else if (element.type === "list") {
342
354
  const listItems = element.children;
343
355
 
@@ -578,4 +590,8 @@ function processInlineElements(children, body) {
578
590
  });
579
591
  }
580
592
 
581
- export { processGroups };
593
+ // `splitBySlices` and `readStack` are exported for `lintContent` (src/lint.js),
594
+ // which diagnoses grouping near-misses and must see the SAME walk that decided
595
+ // the groups — a private reimplementation there would drift. They are not part
596
+ // of the package's public API.
597
+ export { processGroups, splitBySlices, readStack };