@wix/web5-core 1.56.0

package/dist/cjs/patternValidator/README.md

@@ -0,0 +1,757 @@
+# Pattern Validator
+
+Validates if a sequence of block elements matches a pattern rule using finite automaton theory.
+
+## Usage
+
+### Validate Pattern Syntax
+
+```typescript
+import { validatePatternSyntax } from './patternValidator';
+
+// Check if pattern syntax is valid
+validatePatternSyntax('h2 > t') 
+// { valid: true }
+
+validatePatternSyntax('h2 > invalid') 
+// { valid: false, error: 'Invalid element: invalid' }
+
+validatePatternSyntax('h2 >')
+// { valid: false, error: 'Unexpected token after pattern' }
+```
+
+### Validate Block Sequence
+
+```typescript
+import { validatePattern } from './patternValidator';
+
+// Simple sequence
+validatePattern(['h2', 't'], 'h2 > t') // true
+
+// Quantifiers
+validatePattern(['h2', 't', 't'], 'h2 > t+') // true
+validatePattern(['h2'], 'h2 > t*') // true
+validatePattern(['h2', 't', 't'], 'h2 > t[2]') // true
+
+// Alternation
+validatePattern(['h2', 't'], 'h2 > (t | image)') // true
+
+// Groups
+validatePattern(['h2', 't', 'h2', 't'], '(h2 > t)+') // true
+
+// Lists in repetitive patterns (✨ NEW - fully supported!)
+validatePattern(
+  ['t', 'list{[t]}', 't', 'list{[t]}'],
+  '(t > list)*'
+) // true
+
+validatePattern(
+  ['h2', 'html_comment', 'link', 'list{[t]}', 'html_comment', 'link', 'list{[t]}'],
+  'h2 > (html_comment > link > list)*'
+) // true
+
+// Lists with item structure validation
+validatePattern(['list{[t, t]}'], 'list{item[1]:t > t}') // true - 1 item with 2 t's
+validatePattern(['list{[t], [t]}'], 'list{item[2]:t}') // true - 2 items, each with 1 t
+validatePattern(['list{[t], [t], [t]}'], 'list{item[-2]:t}') // false - 3 items exceeds limit
+
+// Complex: item with quantifiers (✨ NEW!)
+validatePattern(
+  ['list{[t, t, t, link, link]}'],
+  'list{item[1]:t+ > link[2]}'
+) // true - 1 item: 1+ t's, then exactly 2 links
+
+// Variable item counts with quantifiers (✨ NEW!)
+validatePattern(
+  ['list{[t, t]}', 'list{[t, t], [t, t], [t, t]}'],
+  'list{item+:t+}'
+) // true - 1+ items, each with 1+ t's
+
+validatePattern(
+  ['list{[t, t], [t, t], [t, t]}'],
+  'list{item[1-]:t > t}'
+) // true - 1 or more items, each with t > t
+
+// Real-world complex pattern
+validatePattern(
+  ['h2', 'link', 't', 'link', 'html_comment', 'link', 'image', 't', 'list{[t, t]}'],
+  'h2 > (link > t)* > link > (html_comment > link > image > t > list{item[1]:t > t})*'
+) // true
+```
+
+## Pattern Syntax
+
+- `>` - Sequence (followed by)
+- `|` - Alternation (or)
+- `*` - 0 or more
+- `+` - 1 or more
+- `?` - Optional (0 or 1)
+- `[N]` - Exactly N
+- `[N-M]` - Between N and M
+- `[N-]` - N or more (open-ended)
+- `[-M]` - Up to M (0 to M)
+- `()` - Grouping
+
+### List Item Quantifiers
+
+- `item+` - 1 or more items
+- `item*` - 0 or more items
+- `item?` - 0 or 1 item
+- `item[N]` - Exactly N items
+- `item[N-M]` - Between N and M items
+- `item[N-]` - N or more items
+- `item[-M]` - Up to M items
+
+## Architecture
+
+1. **Lexer** - Tokenizes pattern strings
+2. **Parser** - Builds AST from tokens
+3. **Automaton Builder** - Converts AST to NFA using Thompson's Construction
+4. **Matcher** - Validates sequences using NFA simulation with epsilon closure
+
+# Advanced Pattern Matching - Complete Syntax Guide
+
+A powerful pattern matching system for validating markdown block element sequences using a custom DSL (Domain Specific Language).
+
+## Table of Contents
+- [Overview](#overview)
+- [Block Elements](#block-elements)
+- [HTML Comment Attributes](#html-comment-attributes)
+- [Basic Syntax](#basic-syntax)
+- [Quantifiers](#quantifiers)
+- [Sequences](#sequences)
+- [Alternation](#alternation)
+- [Groups](#groups)
+- [List Patterns](#list-patterns)
+- [Complex Examples](#complex-examples)
+- [Error Messages](#error-messages)
+- [Usage](#usage)
+
+---
+
+## Overview
+
+The pattern validator allows you to define rules for markdown block element sequences. Think of it as "regex for markdown structure" - instead of matching characters, you match markdown elements like headings, paragraphs, links, images, and lists.
+
+**Use Cases:**
+- Validate document structure (e.g., "must start with h1, then paragraphs")
+- Enforce content templates (e.g., "blog posts must have h2 > intro > image > body")
+- Check list formatting (e.g., "lists must have 2-5 items, each with text and a link")
+
+---
+
+## Block Elements
+
+Block elements are the basic units that patterns match against:
+
+| Element | Description | Example in Markdown |
+|---------|-------------|---------------------|
+| `h1` | Heading level 1 | `# Title` |
+| `h2` | Heading level 2 | `## Section` |
+| `h3` | Heading level 3 | `### Subsection` |
+| `h4` | Heading level 4 | `#### Sub-subsection` |
+| `h5` | Heading level 5 | `##### Level 5` |
+| `h6` | Heading level 6 | `###### Level 6` |
+| `t` | Text/Paragraph | `Regular paragraph text` |
+| `link` | Hyperlink | `[text](url)` |
+| `image` | Image | `![alt](url)` |
+| `list` | List (ordered/unordered) | `- item` or `1. item` |
+| `blockquote` | Blockquote | `> quote` |
+| `table` | Table | Markdown table |
+| `html_comment` | HTML comment | `<!-- comment -->` |
+
+---
+
+## HTML Comment Attributes
+
+HTML comments can include metadata that can be matched using CSS selector-like attribute syntax. This allows you to match specific HTML comments based on their metadata properties.
+
+### Attribute Syntax
+
+HTML comments can contain metadata in the format `<!-- key: value -->` or `<!-- key=value -->`, which can be matched using attribute selectors:
+
+| Syntax | Operator | Description | Example |
+|-------|---------|-------------|---------|
+| `html_comment[key]` | `exists` | Matches if key exists | `html_comment[type]` |
+| `html_comment[key=value]` | `equals` | Exact match | `html_comment[type=solution]` |
+| `html_comment[key*=value]` | `contains` | Substring match | `html_comment[type*=sol]` |
+| `html_comment[key^=value]` | `startsWith` | Prefix match | `html_comment[type^=sol]` |
+| `html_comment[key$=value]` | `endsWith` | Suffix match | `html_comment[type$=tion]` |
+
+### Examples
+
+```typescript
+// Match HTML comment with type=solution metadata
+validatePattern(
+  ['html_comment[type=solution]', 'link', 'image'],
+  'html_comment[type=solution] > link > image'
+) // ✓ true
+
+// Match HTML comment with section=search_results metadata
+validatePattern(
+  ['html_comment[section=search_results]', 'h2'],
+  'html_comment[section=search_results] > h2'
+) // ✓ true
+
+// Match HTML comment with type containing substring
+validatePattern(
+  ['html_comment[type=solution]', 'link'],
+  'html_comment[type*=sol] > link'
+) // ✓ true
+
+// Match HTML comment with search attribute (exists check)
+validatePattern(
+  ['html_comment[search]', 'h2'],
+  'html_comment[search] > h2'
+) // ✓ true
+
+// Optional HTML comment with attributes
+validatePattern(
+  ['link', 'image'],
+  'html_comment[type=solution]? > link > image'
+) // ✓ true - comment is optional
+
+// Multiple HTML comments with attributes
+validatePattern(
+  ['html_comment[section=search_results]', 'h2', 'html_comment[search]', 'html_comment[search]'],
+  'html_comment[section=search_results] > h2 > html_comment[search]*'
+) // ✓ true
+```
+
+### HTML Comment Metadata Format
+
+HTML comments should contain metadata in one of these formats:
+
+```markdown
+<!-- type: solution -->
+<!-- type=solution -->
+<!-- section: search_results -->
+<!-- section=search_results -->
+```
+
+The parser extracts key-value pairs from HTML comments and makes them available for pattern matching.
+
+---
+
+## Basic Syntax
+
+### Single Element
+
+Match a single block element:
+
+```
+h1          // Matches exactly one h1
+t           // Matches exactly one text block
+link        // Matches exactly one link
+```
+
+**Example:**
+```typescript
+validatePattern(['h1'], 'h1')  // ✓ true
+validatePattern(['h2'], 'h1')  // ✗ false
+```
+
+---
+
+## Quantifiers
+
+Quantifiers specify how many times an element can appear:
+
+| Quantifier | Syntax | Description | Matches |
+|------------|--------|-------------|---------|
+| **Star** | `*` | Zero or more | 0, 1, 2, 3, ... |
+| **Plus** | `+` | One or more | 1, 2, 3, ... |
+| **Optional** | `?` | Zero or one | 0, 1 |
+| **Exact** | `[N]` | Exactly N | N |
+| **Range** | `[N-M]` | Between N and M | N, N+1, ..., M |
+| **Up to** | `[-M]` | Up to M | 0, 1, 2, ..., M |
+| **At least** | `[N-]` | N or more | N, N+1, N+2, ... |
+
+### Examples
+
+```typescript
+// Zero or more images
+validatePattern(['image', 'image'], 'image*')  // ✓ true
+validatePattern([], 'image*')                   // ✓ true
+
+// One or more links
+validatePattern(['link', 'link'], 'link+')      // ✓ true
+validatePattern([], 'link+')                    // ✗ false
+
+// Optional text
+validatePattern(['t'], 't?')                    // ✓ true
+validatePattern([], 't?')                       // ✓ true
+
+// Exactly 3 links
+validatePattern(['link', 'link', 'link'], 'link[3]')  // ✓ true
+validatePattern(['link', 'link'], 'link[3]')          // ✗ false
+
+// Between 2 and 4 headings
+validatePattern(['h2', 'h2', 'h2'], 'h2[2-4]')  // ✓ true
+
+// Up to 3 images
+validatePattern(['image', 'image'], 'image[-3]')  // ✓ true
+
+// At least 2 paragraphs
+validatePattern(['t', 't', 't'], 't[2-]')       // ✓ true
+```
+
+---
+
+## Sequences
+
+Use `>` to match elements in sequence:
+
+```
+h1 > t              // h1 followed by text
+h2 > t > link       // h2, then text, then link
+h1 > t > image > t  // h1, text, image, text
+```
+
+**Examples:**
+```typescript
+validatePattern(['h1', 't'], 'h1 > t')                  // ✓ true
+validatePattern(['h1', 't', 'link'], 'h1 > t > link')   // ✓ true
+validatePattern(['h1', 'link'], 'h1 > t')               // ✗ false - missing 't'
+```
+
+### Sequences with Quantifiers
+
+```
+h1 > t+             // h1 followed by one or more text blocks
+h2 > image* > t     // h2, optional images, then text
+t > link[2] > t     // text, exactly 2 links, then text
+```
+
+**Examples:**
+```typescript
+validatePattern(['h1', 't', 't'], 'h1 > t+')            // ✓ true
+validatePattern(['h2', 'image', 'image', 't'], 'h2 > image* > t')  // ✓ true
+validatePattern(['h2', 't'], 'h2 > image* > t')         // ✓ true - image* matches 0
+```
+
+---
+
+## Alternation
+
+Use `|` to match one of several options:
+
+```
+t | image           // Either text OR image
+h1 | h2             // Either h1 OR h2
+link | image | t    // link OR image OR text
+```
+
+**Examples:**
+```typescript
+validatePattern(['t'], 't | image')         // ✓ true
+validatePattern(['image'], 't | image')     // ✓ true
+validatePattern(['link'], 't | image')      // ✗ false
+
+validatePattern(['h1', 't'], 'h1 > (t | image)')  // ✓ true
+validatePattern(['h1', 'image'], 'h1 > (t | image)')  // ✓ true
+```
+
+### Alternation with Quantifiers
+
+```
+(t | image)+        // One or more text or image blocks
+(h1 | h2)?          // Optional h1 or h2
+(link | image)*     // Zero or more links or images
+```
+
+**Examples:**
+```typescript
+validatePattern(['t', 'image', 't'], '(t | image)+')  // ✓ true
+validatePattern(['link', 'image', 'link'], '(link | image)*')  // ✓ true
+```
+
+---
+
+## Groups
+
+Use parentheses `()` to group patterns and apply quantifiers to the entire group:
+
+```
+(h2 > t)*           // Zero or more repetitions of (h2 followed by t)
+(t > link)+         // One or more repetitions of (text followed by link)
+(h1 > t > image)?   // Optional group
+```
+
+**Examples:**
+```typescript
+// Repeating pattern
+validatePattern(['h2', 't', 'h2', 't'], '(h2 > t)+')  // ✓ true
+
+// Nested groups
+validatePattern(['h1', 't', 'link', 't'], 'h1 > (t > (link | image) > t)')  // ✓ true
+
+// Optional groups
+validatePattern(['h1', 't'], 'h1 > (image > t)?')  // ✓ true - group is optional
+validatePattern(['h1'], 'h1 > (image > t)?')       // ✓ true
+```
+
+---
+
+## List Patterns
+
+Lists have special syntax for validating both the number of items and the structure of each item:
+
+### Basic List Syntax
+
+```
+list                // Any list
+list*               // Zero or more lists
+list+               // One or more lists
+```
+
+### List with Item Count Validation
+
+```
+list{item[N]:...}           // Exactly N items
+list{item[N-M]:...}         // Between N and M items
+list{item[-M]:...}          // Up to M items
+list{item[N-]:...}          // At least N items
+list{item+:...}             // One or more items (equivalent to [1-])
+```
+
+### List with Item Structure Validation
+
+Validate what each item contains:
+
+```
+list{item[2]:t}                    // 2 items, each containing only text
+list{item[1-3]:t > link}          // 1-3 items, each: text then link
+list{item+:image > t+}            // 1+ items, each: image then 1+ text blocks
+list{item[-5]:t > image? > link*} // Up to 5 items, complex structure
+```
+
+### List Pattern Examples
+
+```typescript
+// Simple list validation
+validatePattern(['list{[t]}'], 'list')  // ✓ true
+
+// Validate item count
+validatePattern(['list{[t], [t]}'], 'list{item[2]:t}')  // ✓ true - 2 items
+validatePattern(['list{[t]}'], 'list{item[2]:t}')       // ✗ false - only 1 item
+
+// Validate item structure
+const blocks = ['list{[t, link], [t, link]}'];
+validatePattern(blocks, 'list{item[2]:t > link}')  // ✓ true
+
+// Complex item patterns
+const complexList = ['list{[t, t, link, image]}'];
+validatePattern(complexList, 'list{item[1]:t+ > link > image}')  // ✓ true
+
+// Multiple items with quantifiers
+const variedList = ['list{[t], [t, link], [t, link, link]}'];
+validatePattern(variedList, 'list{item[3]:t > link*}')  // ✓ true
+
+// Up to 2 items
+const shortList = ['list{[t]}'];
+validatePattern(shortList, 'list{item[-2]:t}')  // ✓ true
+
+// At least 2 items
+const longList = ['list{[t], [t], [t], [t]}'];
+validatePattern(longList, 'list{item[2-]:t}')  // ✓ true
+```
+
+### List Format Notes
+
+**Important:** List items are formatted as:
+- Single item: `list{[element1, element2]}`
+- Multiple items: `list{[item1_el1, item1_el2], [item2_el1], [item3_el1, item3_el2, item3_el3]}`
+
+Each item is enclosed in square brackets and separated by commas.
+
+---
+
+## Complex Examples
+
+### Example 1: Blog Post Structure
+
+```typescript
+// Blog post must have: h1, intro paragraph, optional image, multiple sections
+const pattern = 'h1 > t > image? > (h2 > t+)+';
+
+const validPost = [
+  'h1',
+  't',
+  'image',
+  'h2',
+  't', 't',
+  'h2',
+  't'
+];
+
+validatePattern(validPost, pattern);  // ✓ true
+```
+
+### Example 2: Product Page with Lists
+
+```typescript
+// Product page: title, description, feature list, optional gallery, specs list
+const pattern = 'h2 > t > list{item[3-5]:t > link?} > image* > list{item+:t}';
+
+const productPage = [
+  'h2',
+  't',
+  'list{[t, link], [t], [t, link]}',  // 3 features
+  'image', 'image',                    // Gallery
+  'list{[t], [t], [t]}'               // 3 specs
+];
+
+validatePattern(productPage, pattern);  // ✓ true
+```
+
+### Example 3: Documentation with Nested Patterns
+
+```typescript
+// Documentation: h1, sections with optional code/images, lists with structured items
+const pattern = 'h1 > (h2 > (t | image)+ > list{item[-3]:t > link*})*';
+
+const docs = [
+  'h1',
+  'h2',
+  't', 'image', 't',
+  'list{[t, link], [t]}',
+  'h2',
+  't',
+  'list{[t]}'
+];
+
+validatePattern(docs, pattern);  // ✓ true
+```
+
+### Example 4: Article with Alternating Content Types
+
+```typescript
+// Article: h1, then sections that can be text-heavy OR image-heavy
+const pattern = 'h1 > (h2 > t+ > list{item+:t} | h2 > image+ > list{item+:image > t})+';
+
+const textSection = [
+  'h1',
+  'h2',
+  't', 't', 't',
+  'list{[t], [t]}'
+];
+
+const imageSection = [
+  'h1',
+  'h2',
+  'image', 'image',
+  'list{[image, t], [image, t]}'
+];
+
+validatePattern(textSection, pattern);   // ✓ true
+validatePattern(imageSection, pattern);  // ✓ true
+```
+
+### Example 5: Complex Real-World Pattern
+
+```typescript
+// Complete article: header, intro, multiple sections, each with various content
+const pattern = 'h1 > html_comment > t[2] > image[2] > (h2 > (t | link)+ > list{item+:t+ > link*} > blockquote? > image*)+';
+
+const article = [
+  'h1',
+  'html_comment',
+  't', 't',
+  'image', 'image',
+  'h2',
+  't', 'link', 't',
+  'list{[t, t, link]}',
+  'blockquote',
+  'h2',
+  't', 't',
+  'list{[t], [t, link]}'
+];
+
+validatePattern(article, pattern);  // ✓ true
+```
+
+---
+
+## Error Messages
+
+When validation fails, detailed error messages help you debug:
+
+### Error Types
+
+#### 1. Item Count Mismatch
+```typescript
+const blocks = ['h2', 'list{[t], [t], [t]}'];
+const pattern = 'h2 > list{item[2]:t}';
+
+const result = validatePattern(blocks, pattern, true);
+// Error: "Expected exactly 2 items in list, but got 3 items"
+```
+
+#### 2. Item Structure Mismatch
+```typescript
+const blocks = ['h2', 'list{[t, link, link]}'];
+const pattern = 'h2 > list{item[1]:t > link}';
+
+const result = validatePattern(blocks, pattern, true);
+// Error: "Item #1 [t, link, link] does not match expected pattern: t > link"
+```
+
+#### 3. Unexpected Element
+```typescript
+const blocks = ['h2', 'image', 't'];
+const pattern = 'h2 > t > link';
+
+const result = validatePattern(blocks, pattern, true);
+// Error: "Unexpected element 'image' at position 2. Pattern does not match."
+```
+
+#### 4. Incomplete Pattern
+```typescript
+const blocks = ['h2', 't'];
+const pattern = 'h2 > t > link > image';
+
+const result = validatePattern(blocks, pattern, true);
+// Error: "Pattern matching incomplete. Expected more elements after position 2."
+```
+
+#### 5. Unexpected List
+```typescript
+const blocks = ['h2', 'list{[t]}'];
+const pattern = 'h2 > t';
+
+const result = validatePattern(blocks, pattern, true);
+// Error: "Unexpected list at position 2. Pattern does not expect a list here."
+```
+
+---
+
+## Usage
+
+### Import
+
+```typescript
+import { validatePattern, validatePatternSyntax } from './patternValidator';
+```
+
+### Basic Validation (Boolean)
+
+```typescript
+const blocks = ['h1', 't', 'link'];
+const pattern = 'h1 > t > link';
+
+const isValid = validatePattern(blocks, pattern);
+console.log(isValid);  // true or false
+```
+
+### Validation with Error Details
+
+```typescript
+const blocks = ['h1', 'image'];
+const pattern = 'h1 > t';
+
+const result = validatePattern(blocks, pattern, true);
+if (!result.valid) {
+  console.error('Validation failed:', result.error);
+  // Output: "Validation failed: Unexpected element 'image' at position 2..."
+}
+```
+
+### Validate Pattern Syntax
+
+```typescript
+const patternSyntax = validatePatternSyntax('h2 > t+ > list{item[-5]:t > link}');
+
+if (!patternSyntax.valid) {
+  console.error('Invalid syntax:', patternSyntax.error);
+}
+```
+
+### Type Definitions
+
+```typescript
+type BlockElement = string;  // 'h1', 'h2', 't', 'link', etc.
+
+type PatternValidationResult = {
+  valid: boolean;
+  error?: string;
+};
+
+// Boolean return
+validatePattern(blocks: BlockElement[], pattern: string): boolean;
+
+// Detailed return
+validatePattern(blocks: BlockElement[], pattern: string, returnDetails: true): PatternValidationResult;
+```
+
+---
+
+## Pattern Syntax Quick Reference
+
+| Feature | Syntax | Example |
+|---------|--------|---------|
+| **Single element** | `element` | `h1`, `t`, `link` |
+| **Sequence** | `>` | `h1 > t > link` |
+| **Alternation** | `\|` | `t \| image` |
+| **Group** | `(...)` | `(h2 > t)` |
+| **Zero or more** | `*` | `image*` |
+| **One or more** | `+` | `link+` |
+| **Optional** | `?` | `t?` |
+| **Exact count** | `[N]` | `link[3]` |
+| **Range** | `[N-M]` | `t[2-5]` |
+| **Up to** | `[-M]` | `image[-3]` |
+| **At least** | `[N-]` | `h2[2-]` |
+| **List** | `list` | `list` |
+| **List with items** | `list{item[N]:...}` | `list{item[2]:t > link}` |
+| **List with range** | `list{item[N-M]:...}` | `list{item[1-5]:t}` |
+| **HTML comment attribute (exists)** | `html_comment[key]` | `html_comment[type]` |
+| **HTML comment attribute (equals)** | `html_comment[key=value]` | `html_comment[type=solution]` |
+| **HTML comment attribute (contains)** | `html_comment[key*=value]` | `html_comment[type*=sol]` |
+| **HTML comment attribute (starts with)** | `html_comment[key^=value]` | `html_comment[type^=sol]` |
+| **HTML comment attribute (ends with)** | `html_comment[key$=value]` | `html_comment[type$=tion]` |
+
+---
+
+## Best Practices
+
+1. **Start Simple**: Begin with basic patterns and add complexity as needed
+2. **Use Groups**: Parentheses help organize complex patterns
+3. **Test Incrementally**: Validate patterns with small test cases first
+4. **Leverage Error Messages**: Use `returnDetails: true` during development
+5. **Document Patterns**: Add comments explaining complex validation rules
+6. **Consider Variations**: Use alternation and optionals for flexible matching
+
+---
+
+## Testing Your Patterns
+
+The package includes comprehensive test coverage. See `validator.test.ts` for:
+- ✅ 40+ test cases
+- ✅ Simple to super complex patterns
+- ✅ List validation examples
+- ✅ Error message validation
+
+---
+
+## Technical Implementation
+
+Built using:
+- **Lexer**: Tokenizes pattern strings
+- **Parser**: Builds Abstract Syntax Tree (AST)
+- **Automaton**: Uses Thompson's Construction for NFA
+- **Matcher**: Simulates NFA with epsilon closure
+
+This implementation supports:
+- ✅ Full regex-like quantifiers
+- ✅ Nested groups and alternation
+- ✅ Advanced list validation with item structure
+- ✅ HTML comment attribute matching (CSS selector-like syntax)
+- ✅ Detailed error messages with position tracking
+- ✅ Clean, maintainable architecture
+
+---
+
+## License
+
+Part of the Web5.0 POC project.
+