@uniweb/semantic-parser 1.0.3 → 1.0.8

package/AGENTS.md

@@ -0,0 +1,136 @@
+# AGENTS.md
+
+This file provides guidance for AI assistants working with code in this repository.
+
+## Project Overview
+
+This is a semantic parser for ProseMirror/TipTap content structures. It transforms rich text editor content into structured, semantic groups that web components can consume. The parser bridges the gap between natural content writing and component-based web development.
+
+## Development Commands
+
+```bash
+# Run all tests
+npm test
+
+# Run tests with JSON report output
+npm run test-report
+
+# Run a specific test file
+npx jest tests/parser.test.js
+
+# Run tests in watch mode
+npx jest --watch
+
+# Run a specific test by name
+npx jest -t "handles simple document structure"
+```
+
+## Architecture
+
+### Three-Stage Processing Pipeline
+
+The parser processes content through three distinct stages, each building on the previous:
+
+1. **Sequence Processing** (`src/processors/sequence.js`): Flattens the ProseMirror document tree into a linear sequence of semantic elements (headings, paragraphs, images, lists, etc.)
+
+2. **Groups Processing** (`src/processors/groups.js`): Transforms the sequence into semantic groups with identified main content and items. Supports two grouping modes:
+   - Heading-based grouping (default)
+   - Divider-based grouping (when horizontal rules are present)
+
+3. **ByType Processing** (`src/processors/byType.js`): Organizes elements by type with positional context, enabling type-specific queries
+
+The main entry point (`src/index.js`) returns all three views:
+```js
+import { parseContent } from './src/index.js';
+
+const result = parseContent(doc);
+// {
+//   raw: doc,        // Original ProseMirror document
+//   sequence: [...], // Flat sequence of elements
+//   groups: {...},   // Semantic groups with main/items
+//   byType: {...}    // Elements organized by type
+// }
+```
+
+### Content Group Structure
+
+Groups follow a specific structure defined in `processGroupContent()`:
+
+```js
+{
+  header: {
+    pretitle: '',  // H3 before main title
+    title: '',     // Main heading (H1 or H2)
+    subtitle: ''   // Heading after main title
+  },
+  body: {
+    imgs: [],
+    icons: [],
+    videos: [],
+    paragraphs: [],
+    links: [],
+    lists: [],
+    buttons: [],
+    properties: [],
+    propertyBlocks: [],
+    cards: [],
+    headings: []
+  },
+  banner: null,    // Image with banner role or image before heading
+  metadata: {
+    level: null,   // Heading level that started this group
+    contentTypes: Set()
+  }
+}
+```
+
+### Main Content Identification
+
+The `identifyMainContent()` function (src/processors/groups.js:282) determines if the first group should be treated as main content:
+- Single group is always main content
+- First group must have lower heading level than second group
+- Divider mode affects main content identification
+
+### Special Element Detection
+
+The sequence processor identifies several special element types by inspecting paragraph content:
+- **Links**: Paragraphs containing only a single link mark
+- **Images**: Paragraphs with single image (role: 'image' or 'banner')
+- **Icons**: Paragraphs with single image (role: 'icon')
+- **Buttons**: Paragraphs with single text node having button mark
+- **Videos**: Paragraphs with single image (role: 'video')
+
+These are extracted into dedicated element types for easier downstream processing.
+
+### List Processing
+
+Lists maintain hierarchy through nested structure. The `processListItems()` function in sequence.js handles nested lists, while `processListContent()` in groups.js applies full group content processing to each list item, allowing lists to contain rich content (images, paragraphs, nested lists, etc.).
+
+## Content Writing Conventions
+
+The parser implements the semantic conventions documented in `docs/guide.md`. Key patterns:
+
+- **Pretitle Pattern**: Any heading followed by a more important heading (e.g., H3→H1, H2→H1, H6→H5, etc.)
+- **Banner Pattern**: Image (with banner role or followed by heading) at start of first group
+- **Divider Mode**: Presence of any `horizontalRule` switches entire document to divider-based grouping
+- **Heading Groups**: Consecutive headings with increasing levels are consumed together
+- **Main Content**: First group is main if it's the only group OR has lower heading level than second group
+- **Body Headings**: Headings that overflow the header slots (title, subtitle, subtitle2) are automatically collected in `body.headings`
+
+## Testing Structure
+
+Tests are organized by processor:
+- `tests/parser.test.js` - Integration tests
+- `tests/processors/sequence.test.js` - Sequence processing
+- `tests/processors/groups.test.js` - Groups processing
+- `tests/processors/byType.test.js` - ByType processing
+- `tests/utils/role.test.js` - Role utilities
+- `tests/fixtures/` - Shared test documents
+
+## Important Implementation Notes
+
+- The parser never modifies the original ProseMirror document
+- Text content can include inline HTML for formatting (bold → `<strong>`, italic → `<em>`, links → `<a>`)
+- The `processors_old/` directory contains legacy implementations - do not modify
+- Context information in byType includes position, previous/next elements, and nearest heading
+- Group splitting logic differs significantly between heading mode and divider mode

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@uniweb/semantic-parser",
-  "version": "1.0.3",
+  "version": "1.0.8",
   "description": "Semantic parser for ProseMirror/TipTap content structures",
   "type": "module",
   "main": "./src/index.js",

package/src/processors/byType.js

@@ -44,7 +44,8 @@ function processByType(sequence) {
         break;
 
       case "image": {
-        const role = element.role || "content";
+        // Support both attrs.role and top-level role for backwards compatibility
+        const role = element.attrs?.role || element.role || "content";
         if (!collections.images[role]) {
           collections.images[role] = [];
         }

package/src/processors/groups.js

@@ -320,9 +320,11 @@ function identifyMainContent(groups) {
 
 function processInlineElements(children, body) {
     children.forEach((item) => {
-        //Handle icons only for now
         if (item.type === "icon") {
             body.icons.push(item.attrs);
+        } else if (item.type === "link") {
+            // Handle inline links extracted from paragraph text nodes
+            body.links.push(item.attrs);
         }
     });
 }

package/src/processors/sequence.js

@@ -102,10 +102,10 @@ function createSequenceElement(node, options = {}) {
                 attrs: parseImgBlock(attrs),
             };
         case "image":
-            // Standard ProseMirror image node - spread attrs at top level
+            // Standard ProseMirror image node - wrap attrs like ImageBlock
             return {
                 type: "image",
-                ...attrs,
+                attrs: attrs || {},
             };
         case "Video":
             return {
@@ -302,6 +302,18 @@ function processInlineElements(content) {
             });
         } else if (item.type === "math-inline") {
             items.push(item);
+        } else if (item.type === "text" && item.marks) {
+            // Extract links from text nodes with link marks
+            const linkMark = item.marks.find((m) => m.type === "link");
+            if (linkMark) {
+                items.push({
+                    type: "link",
+                    attrs: {
+                        href: linkMark.attrs?.href,
+                        label: item.text || "",
+                    },
+                });
+            }
         }
     }