documentation-hub 5.7.2

package/docs/analysis/FIELD_PRESERVATION_ANALYSIS.md

@@ -0,0 +1,1200 @@
+# Field Preservation Analysis - Critical Bug Report
+
+**Date**: November 14, 2025
+**Analysis Type**: Deep Codebase Investigation
+**Status**: 🚨 **CRITICAL BUG IDENTIFIED**
+**Severity**: HIGH - Data Loss Issue
+
+---
+
+## Executive Summary
+
+After thoroughly analyzing both the **docXMLater framework** (document processing library) and **dochub-app application** (main application using the framework), I have identified **critical bugs in field preservation** that explain why fields are inconsistently preserved during DOCX file processing.
+
+### Key Findings
+
+1. ✅ **Simple fields (`<w:fldSimple>`) ARE parsed** - Code exists in DocumentParser
+2. ❌ **Simple fields MAY NOT be serialized** - Conditional preservation bug
+3. ❌ **Complex fields (`<w:fldChar>`) are COMPLETELY IGNORED** - Not parsed at all
+4. ⚠️ **Order preservation is unreliable** - Metadata generation has edge cases
+
+---
+
+## Architecture Overview
+
+### System Flow
+
+```
+dochub-app (Main Application)
+    ↓
+WordDocumentProcessor.ts
+    ↓
+DocXMLaterProcessor.ts (thin wrapper)
+    ↓
+docXMLater Framework
+    ├─ DocumentParser.ts (Load: XML → Objects)
+    ├─ Document.ts (In-memory document)
+    └─ DocumentGenerator.ts (Save: Objects → XML)
+```
+
+### Document Processing Pipeline
+
+```
+1. LOAD PHASE
+   ┌─────────────────────────────────────┐
+   │ Word DOCX File (ZIP archive)       │
+   └──────────────┬──────────────────────┘
+                  ↓
+   ┌─────────────────────────────────────┐
+   │ ZipHandler.load()                  │
+   │ - Extracts word/document.xml       │
+   └──────────────┬──────────────────────┘
+                  ↓
+   ┌─────────────────────────────────────┐
+   │ DocumentParser.parseDocument()     │
+   │ - Parses XML to structured objects │
+   │ - Creates Paragraph, Run, etc.     │
+   └──────────────┬──────────────────────┘
+                  ↓
+   ┌─────────────────────────────────────┐
+   │ Document (in-memory)               │
+   │ - bodyElements: Paragraph[]        │
+   └─────────────────────────────────────┘
+
+2. PROCESSING PHASE
+   ┌─────────────────────────────────────┐
+   │ WordDocumentProcessor              │
+   │ - Updates hyperlinks               │
+   │ - Applies styles                   │
+   │ - Formats tables                   │
+   └─────────────────────────────────────┘
+
+3. SAVE PHASE
+   ┌─────────────────────────────────────┐
+   │ Document.save()                    │
+   │ - Calls Paragraph.toXML()          │
+   │ - Generates word/document.xml      │
+   └──────────────┬──────────────────────┘
+                  ↓
+   ┌─────────────────────────────────────┐
+   │ ZipHandler.save()                  │
+   │ - Writes ZIP archive to disk       │
+   └─────────────────────────────────────┘
+```
+
+---
+
+## 🐛 BUG #1: Complex Fields Are Completely Ignored
+
+### Description
+
+**Complex fields** (using `<w:fldChar>` structure) are **NOT parsed at all**. They are silently dropped during document loading.
+
+### Field Types in Word Documents
+
+Word documents use two field structures:
+
+#### 1. Simple Fields (`<w:fldSimple>`)
+
+```xml
+<w:p>
+  <w:fldSimple w:instr=" PAGE \* MERGEFORMAT ">
+    <w:t>1</w:t>
+  </w:fldSimple>
+</w:p>
+```
+
+#### 2. Complex Fields (`<w:fldChar>`) - ❌ **NOT HANDLED**
+
+```xml
+<w:p>
+  <w:r><w:fldChar w:fldCharType="begin"/></w:r>
+  <w:r><w:instrText> PAGE \* MERGEFORMAT </w:instrText></w:r>
+  <w:r><w:fldChar w:fldCharType="separate"/></w:r>
+  <w:r><w:t>1</w:t></w:r>
+  <w:r><w:fldChar w:fldCharType="end"/></w:r>
+</w:p>
+```
+
+### Affected Field Types
+
+Complex fields are used for:
+
+- ✅ Table of Contents (TOC)
+- ✅ Cross-references (REF)
+- ✅ Page numbers (PAGE)
+- ✅ Date/Time fields (DATE, TIME)
+- ✅ Document properties (AUTHOR, TITLE, etc.)
+- ✅ Conditional fields (IF)
+- ✅ Mail merge fields (MERGEFIELD)
+
+### Code Evidence
+
+**File**: `docXMLater/src/core/DocumentParser.ts`
+
+**Parsing Logic** (lines ~238-310):
+
+```typescript
+// In parseParagraphWithOrder() method:
+if (orderedChildren && orderedChildren.length > 0) {
+  for (const childInfo of orderedChildren) {
+    const elementType = childInfo.type;
+
+    if (elementType === 'w:r') {
+      // ✅ Runs are parsed
+    } else if (elementType === 'w:hyperlink') {
+      // ✅ Hyperlinks are parsed
+    } else if (elementType === 'w:fldSimple') {
+      // ✅ Simple fields are parsed
+    }
+    // ❌ NO HANDLING FOR w:fldChar - complex fields are ignored!
+  }
+}
+```
+
+**Parsing Method** (lines ~580-616):
+
+```typescript
+private parseSimpleFieldFromObject(fieldObj: any): Field | null {
+  // ✅ This method exists and works
+  const instruction = fieldObj["@_w:instr"];
+  const type = (typeMatch?.[1] || 'PAGE') as FieldType;
+  return Field.create({ type, instruction, formatting });
+}
+
+// ❌ NO METHOD FOR parseComplexFieldFromObject()
+// ❌ NO CODE TO DETECT w:fldChar elements
+```
+
+**Result**: Complex fields are **silently dropped** during parsing. The XML contains them, but they never make it into the in-memory Document object.
+
+### Impact
+
+**Hit or Miss Behavior**:
+
+- Documents with **simple fields** (`<w:fldSimple>`) → ✅ Preserved (sometimes, see Bug #2)
+- Documents with **complex fields** (`<w:fldChar>`) → ❌ Always lost
+- Mixed documents → ⚠️ Partial loss (simple preserved, complex lost)
+
+This explains the "hit or miss" nature - it depends on which field structure Word used when creating the field.
+
+### Why Word Uses Different Structures
+
+Word decides between simple and complex fields based on:
+
+- **Simple**: Basic fields with no special formatting or nested content
+- **Complex**: Fields with special formatting, nested fields, or complex instructions
+
+The choice is **automatic and invisible to users**, which is why the bug appears random.
+
+---
+
+## 🐛 BUG #2: Simple Field Preservation Depends on `_orderedChildren` Metadata
+
+### Description
+
+Simple fields (`<w:fldSimple>`) are only preserved if the **XMLParser generates `_orderedChildren` metadata**. This metadata is **conditionally generated**, leading to inconsistent field preservation.
+
+### Root Cause
+
+**File**: `docXMLater/src/xml/XMLParser.ts`
+
+**Lines ~560-575** (coalesceChildren method):
+
+```typescript
+// Build ordered children metadata to preserve document order
+const orderedChildren: Array<{ type: string; index: number }> = [];
+
+// ... build orderedChildren array ...
+
+// ❌ BUG: Only adds metadata if multiple child types exist
+if (uniqueTypes.length > 1 && orderedChildren.length > 0) {
+  result['_orderedChildren'] = orderedChildren;
+}
+```
+
+### The Problem
+
+**Scenario 1**: Paragraph with runs, hyperlinks, AND fields
+
+```xml
+<w:p>
+  <w:r><w:t>Text</w:t></w:r>
+  <w:hyperlink><w:r><w:t>Link</w:t></w:r></w:hyperlink>
+  <w:fldSimple w:instr="PAGE"><w:t>1</w:t></w:fldSimple>
+</w:p>
+```
+
+- `uniqueTypes.length = 3` (w:r, w:hyperlink, w:fldSimple)
+- ✅ `_orderedChildren` is created
+- ✅ Field is parsed in correct order
+- ✅ **Field PRESERVED**
+
+**Scenario 2**: Paragraph with ONLY fields
+
+```xml
+<w:p>
+  <w:fldSimple w:instr="PAGE"><w:t>1</w:t></w:fldSimple>
+</w:p>
+```
+
+- `uniqueTypes.length = 1` (only w:fldSimple)
+- ❌ `_orderedChildren` is **NOT created** (fails `uniqueTypes.length > 1` check)
+- ❌ Falls back to non-ordered parsing
+- ⚠️ **Field MAY be lost** (depends on fallback behavior)
+
+**Scenario 3**: Paragraph with fields and runs of same type
+
+```xml
+<w:p>
+  <w:r><w:t>Page </w:t></w:r>
+  <w:fldSimple w:instr="PAGE"><w:t>1</w:t></w:fldSimple>
+  <w:r><w:t> of </w:t></w:r>
+  <w:fldSimple w:instr="NUMPAGES"><w:t>10</w:t></w:fldSimple>
+</w:p>
+```
+
+- `uniqueTypes.length = 2` (w:r, w:fldSimple)
+- ✅ `_orderedChildren` is created
+- ✅ **Fields PRESERVED**
+
+### Fallback Behavior Analysis
+
+**File**: `docXMLater/src/core/DocumentParser.ts` (lines ~280-310)
+
+```typescript
+} else {
+  // Fallback to sequential processing if no order metadata
+  // Handle runs (w:r)
+  const runs = pElement["w:r"];
+  // ...process runs...
+
+  // Handle hyperlinks (w:hyperlink)
+  const hyperlinks = pElement["w:hyperlink"];
+  // ...process hyperlinks...
+
+  // Handle simple fields (w:fldSimple)
+  const fields = pElement["w:fldSimple"];
+  const fieldChildren = Array.isArray(fields) ? fields : (fields ? [fields] : []);
+
+  for (const fieldObj of fieldChildren) {
+    const field = this.parseSimpleFieldFromObject(fieldObj);
+    if (field) {
+      paragraph.addField(field);  // ✅ Field IS added in fallback
+    }
+  }
+}
+```
+
+**Conclusion**: In the fallback path, fields **ARE processed** and added to the paragraph. However, they are processed **AFTER runs and hyperlinks**, which means:
+
+- ✅ Fields are preserved
+- ⚠️ Field **ORDER** may be wrong (always appear last instead of in correct position)
+
+### Why This Causes "Hit or Miss"
+
+**Working Case** (multi-type paragraph):
+
+- Paragraph has runs + fields → `_orderedChildren` created → Fields in correct order ✅
+
+**Broken Case** (fields only):
+
+- Paragraph has only fields → No `_orderedChildren` → Fallback parsing → Fields at wrong position ⚠️
+- If document structure depends on field order (e.g., TOC), this breaks functionality
+
+---
+
+## 🐛 BUG #3: Field Serialization Does Not Preserve Complex Fields
+
+### Description
+
+Even if complex fields WERE parsed (they aren't per Bug #1), the serialization code in `Paragraph.toXML()` cannot properly reconstruct them.
+
+### Code Evidence
+
+**File**: `docXMLater/src/elements/Paragraph.ts` (lines ~600-650)
+
+```typescript
+// Add content (runs, fields, hyperlinks, revisions, shapes, textboxes)
+for (let i = 0; i < this.content.length; i++) {
+  const item = this.content[i];
+
+  if (item instanceof Field) {
+    // ❌ BUG: Fields are wrapped in a run - converts to <w:fldSimple>
+    paragraphChildren.push(XMLBuilder.w('r', undefined, [item.toXML()]));
+  } else if (item instanceof Hyperlink) {
+    // ✅ Hyperlinks are standalone elements
+    paragraphChildren.push(item.toXML());
+  } else if (item) {
+    paragraphChildren.push(item.toXML());
+  }
+}
+```
+
+**Field.toXML()** from `Field.ts`:
+
+```typescript
+toXML(): XMLElement {
+  // ...
+  return {
+    name: 'w:fldSimple',  // ❌ Always generates fldSimple, never fldChar
+    attributes: {
+      'w:instr': this.instruction,
+    },
+    children,
+  };
+}
+```
+
+### The Problem
+
+The current code **always serializes fields as `<w:fldSimple>`**, even if they were originally complex fields. This means:
+
+1. Complex fields can't be represented in the object model
+2. Even if parsing were fixed, serialization would convert them to simple fields
+3. Word may reject or incorrectly render the simplified fields
+
+### Impact
+
+- Complex field formatting is lost
+- Nested fields are flattened
+- TOC fields may not update correctly in Word
+- Cross-references lose their specialized behavior
+
+---
+
+## 🐛 BUG #4: Runs with `w:fldChar` Elements Are Treated as Regular Text Runs
+
+### Description
+
+During parsing, runs that contain `<w:fldChar>` elements (field markers) are processed as **regular text runs**, losing the field structure entirely.
+
+### Code Evidence
+
+**File**: `docXMLater/src/core/DocumentParser.ts` (lines ~450-550)
+
+```typescript
+private parseRunFromObject(runObj: any): Run | null {
+  // Extract all run content elements (text, tabs, breaks, etc.)
+  const content: RunContent[] = [];
+
+  if (runObj["_orderedChildren"]) {
+    for (const child of runObj["_orderedChildren"]) {
+      const elementType = child.type;
+
+      switch (elementType) {
+        case 'w:t':
+          // ✅ Text is handled
+          break;
+        case 'w:tab':
+          // ✅ Tabs are handled
+          break;
+        case 'w:br':
+          // ✅ Breaks are handled
+          break;
+        // ❌ NO CASE FOR 'w:fldChar'
+        // ❌ NO CASE FOR 'w:instrText'
+      }
+    }
+  }
+
+  // Create run from content elements - returns a regular Run
+  const run = Run.createFromContent(content, { cleanXmlFromText: false });
+  return run;
+}
+```
+
+### What Should Happen
+
+When a run contains `<w:fldChar>`, it's part of a **complex field structure**:
+
+```xml
+<!-- Field begin marker -->
+<w:r><w:fldChar w:fldCharType="begin"/></w:r>
+
+<!-- Field instruction -->
+<w:r><w:instrText> PAGE \* MERGEFORMAT </w:instrText></w:r>
+
+<!-- Field separator -->
+<w:r><w:fldChar w:fldCharType="separate"/></w:r>
+
+<!-- Field result (actual displayed value) -->
+<w:r><w:t>1</w:t></w:r>
+
+<!-- Field end marker -->
+<w:r><w:fldChar w:fldCharType="end"/></w:r>
+```
+
+These runs should be **grouped together** and converted to a single Field object.
+
+### What Actually Happens
+
+Each run is parsed **independently**:
+
+- Run with `fldChar="begin"` → Parsed as empty Run (no text) → ✅ Preserved as empty run
+- Run with `instrText` → Parsed as Run with text " PAGE \* MERGEFORMAT " → ❌ Shows as literal text
+- Run with `fldChar="separate"` → Parsed as empty Run → ✅ Preserved as empty run
+- Run with actual value → Parsed as Run with text "1" → ✅ Preserved
+- Run with `fldChar="end"` → Parsed as empty Run → ✅ Preserved as empty run
+
+**Result**: The field structure is **lost**. The instruction text appears as **literal visible text** in the document instead of being executed as a field.
+
+---
+
+## 🐛 BUG #5: Field Order Can Be Scrambled During Serialization
+
+### Description
+
+Even when fields ARE preserved during parsing, they may be serialized in the wrong order relative to runs and hyperlinks.
+
+### Code Evidence
+
+**File**: `docXMLater/src/elements/Paragraph.ts` (lines ~600-650)
+
+```typescript
+// Add content (runs, fields, hyperlinks, revisions, shapes, textboxes)
+for (let i = 0; i < this.content.length; i++) {
+  const item = this.content[i];
+
+  if (item instanceof Field) {
+    paragraphChildren.push(XMLBuilder.w('r', undefined, [item.toXML()]));
+  } else if (item instanceof Hyperlink) {
+    paragraphChildren.push(item.toXML());
+  } else if (item instanceof Revision) {
+    paragraphChildren.push(item.toXML());
+  } else if (item instanceof RangeMarker) {
+    paragraphChildren.push(item.toXML());
+  } else if (item) {
+    paragraphChildren.push(item.toXML());
+  }
+}
+```
+
+### The Problem
+
+The serialization iterates through `this.content[]` in order, which **should** preserve order. However:
+
+1. **During parsing**, the order depends on whether `_orderedChildren` exists (Bug #2)
+2. **During processing**, the `content[]` array may be modified by dochub-app operations
+3. **No validation** ensures the order remains correct
+
+### Potential Scenario
+
+```
+Original: Text [FIELD: PAGE] Link
+Parsed:   [Run: "Text"] [Field: PAGE] [Hyperlink: "Link"]
+After processing: [Run: "Text"] [Hyperlink: "Link"] [Field: PAGE]
+Saved:    Text Link [FIELD: PAGE]  ❌ Wrong order!
+```
+
+While this is **theoretically possible**, the current code doesn't modify `content[]` order during processing, so this is **low risk** compared to Bugs #1-4.
+
+---
+
+## 📊 Summary Table: Field Preservation Matrix
+
+| Field Structure      | Paragraph Content     | `_orderedChildren`? | Parsing Result     | Serialization    | Final Result           |
+| -------------------- | --------------------- | ------------------- | ------------------ | ---------------- | ---------------------- |
+| Simple (`fldSimple`) | Runs + Fields + Links | ✅ Yes (3 types)    | ✅ Parsed in order | ✅ Correct order | ✅ **PRESERVED**       |
+| Simple (`fldSimple`) | Runs + Fields         | ✅ Yes (2 types)    | ✅ Parsed in order | ✅ Correct order | ✅ **PRESERVED**       |
+| Simple (`fldSimple`) | Fields only           | ❌ No (1 type)      | ⚠️ Fallback path   | ⚠️ Wrong order   | ⚠️ **ORDER BROKEN**    |
+| Complex (`fldChar`)  | Any combination       | N/A                 | ❌ Not parsed      | ❌ No object     | ❌ **COMPLETELY LOST** |
+
+---
+
+## 🔍 Root Cause Analysis
+
+### Why These Bugs Exist
+
+#### 1. **Incomplete Implementation**
+
+The docXMLater framework has:
+
+- ✅ `Field` class defined in `elements/Field.ts`
+- ✅ `ComplexField` class defined in `elements/Field.ts`
+- ❌ **No parsing code** for `ComplexField` in `DocumentParser.ts`
+- ❌ **No detection logic** for `w:fldChar` elements
+
+**Evidence**: The `Field.ts` file has complete classes for both simple and complex fields, but `DocumentParser.ts` only has code to parse simple fields.
+
+#### 2. **Design Flaw in Order Preservation**
+
+The `_orderedChildren` metadata is meant to preserve element order, but the condition `uniqueTypes.length > 1` is **too restrictive**:
+
+```typescript
+// ❌ FLAWED LOGIC: Assumes order only matters with multiple types
+if (uniqueTypes.length > 1 && orderedChildren.length > 0) {
+  result['_orderedChildren'] = orderedChildren;
+}
+
+// ✅ CORRECT LOGIC: Order always matters
+if (orderedChildren.length > 0) {
+  result['_orderedChildren'] = orderedChildren;
+}
+```
+
+The assumption that "single-type content doesn't need ordering" is **FALSE**. Consider:
+
+- Multiple fields in sequence → Order matters (e.g., "Page 1 of 10")
+- Multiple runs → Order matters for text flow
+
+#### 3. **Architectural Mismatch**
+
+The framework was designed with a **run-centric model**:
+
+- Paragraphs contain **Runs, Hyperlinks, Fields**
+- Each is a separate object type
+
+But Word's XML has a **run-based structure** where **complex fields ARE runs**:
+
+- Complex fields are **sequences of special runs**
+- Each `<w:r>` can contain `<w:fldChar>` or `<w:instrText>`
+
+The framework needs **stateful parsing** to group these runs into Field objects, but it uses **stateless element-by-element parsing**.
+
+---
+
+## 💡 Impact on User Experience
+
+### Symptoms Users See
+
+1. **"Fields disappear"** after processing
+   - User inserts PAGE field in Word
+   - Processes document in dochub-app
+   - Opens result → `PAGE` shows as literal text or is missing
+
+2. **"Sometimes it works, sometimes it doesn't"**
+   - Document A: Simple fields → Works ✅
+   - Document B: Complex fields → Fails ❌
+   - Same operation, different field types → Appears random
+
+3. **"Table of Contents is broken"**
+   - TOC uses complex fields
+   - Always lost during processing
+   - Document needs manual TOC recreation after processing
+
+4. **"Page numbers disappear"**
+   - Header/footer page numbers often use complex fields
+   - Lost during document processing
+   - Users must manually re-insert fields
+
+### Real-World Scenarios
+
+**Scenario A: Legal Documents**
+
+```
+Original: Contract dated [DATE] on page [PAGE]
+After:    Contract dated DATE on page PAGE
+```
+
+- Field codes appear as literal text
+- Professional documents look broken
+
+**Scenario B: Reports**
+
+```
+Original: [TOC with hyperlinked entries]
+After:    [Empty TOC or visible field codes]
+```
+
+- TOC must be manually regenerated
+- Cross-references broken
+
+**Scenario C: Templates**
+
+```
+Original: Author: [AUTHOR], Modified: [SAVEDATE]
+After:    Author: AUTHOR, Modified: SAVEDATE
+```
+
+- Dynamic fields converted to static text
+- Document loses its template functionality
+
+---
+
+## 🛠️ Recommended Fixes
+
+### Fix Priority
+
+1. **CRITICAL** - BUG #1: Add complex field parsing
+2. **HIGH** - BUG #4: Detect and group `w:fldChar` runs
+3. **MEDIUM** - BUG #2: Always generate `_orderedChildren`
+4. **LOW** - BUG #3: Add ComplexField serialization
+5. **LOW** - BUG #5: Validate content order preservation
+
+### Fix #1: Add Complex Field Parsing
+
+**File**: `docXMLater/src/core/DocumentParser.ts`
+
+**Location**: `parseParagraphWithOrder()` method (around line 250)
+
+**Current Code**:
+
+```typescript
+} else if (elementType === "w:fldSimple") {
+  // Parse simple fields
+}
+```
+
+**Add After**:
+
+```typescript
+} else if (elementType === "w:r") {
+  // Check if this run contains field characters
+  const run = runArray[elementIndex];
+  if (run && (run["w:fldChar"] || run["w:instrText"])) {
+    // This is part of a complex field - add to pending field parser
+    this.addToComplexField(run);
+  } else {
+    // Regular run
+    const parsedRun = this.parseRunFromObject(run);
+    if (parsedRun) paragraph.addRun(parsedRun);
+  }
+}
+```
+
+**Add New Method**:
+
+```typescript
+private complexFieldBuffer: any[] = [];
+
+private addToComplexField(run: any): void {
+  this.complexFieldBuffer.push(run);
+
+  // Check if we've completed a field (found "end" marker)
+  if (run["w:fldChar"]?.["@_w:fldCharType"] === "end") {
+    const field = this.parseComplexFieldFromBuffer();
+    if (field) {
+      // Add field to current paragraph
+    }
+    this.complexFieldBuffer = [];
+  }
+}
+
+private parseComplexFieldFromBuffer(): ComplexField | null {
+  // Parse the buffered runs into a ComplexField object
+  // Extract instruction from w:instrText
+  // Extract result from runs between separate and end
+  // Return ComplexField instance
+}
+```
+
+### Fix #2: Always Generate `_orderedChildren`
+
+**File**: `docXMLater/src/xml/XMLParser.ts`
+
+**Location**: `coalesceChildren()` method (line ~573)
+
+**Current Code**:
+
+```typescript
+// ❌ BUG: Only adds metadata if multiple child types exist
+if (uniqueTypes.length > 1 && orderedChildren.length > 0) {
+  result['_orderedChildren'] = orderedChildren;
+}
+```
+
+**Fixed Code**:
+
+```typescript
+// ✅ FIX: Always add metadata to preserve element order
+if (orderedChildren.length > 0) {
+  result['_orderedChildren'] = orderedChildren;
+}
+```
+
+**Impact**: This single-line change ensures field order is always preserved, even for paragraphs with only one element type.
+
+### Fix #3: Handle `w:fldChar` in Run Parsing
+
+**File**: `docXMLater/src/core/DocumentParser.ts`
+
+**Location**: `parseRunFromObject()` method (around line 500)
+
+**Add to switch statement**:
+
+```typescript
+switch (elementType) {
+  case 'w:t':
+    // Existing text handling
+    break;
+
+  case 'w:tab':
+    // Existing tab handling
+    break;
+
+  case 'w:fldChar':
+    // ✅ NEW: Handle field character markers
+    const fldChar = runObj['w:fldChar'];
+    const fldCharType = fldChar?.['@_w:fldCharType'];
+    content.push({
+      type: 'fieldCharacter',
+      value: fldCharType, // 'begin', 'separate', or 'end'
+    });
+    break;
+
+  case 'w:instrText':
+    // ✅ NEW: Handle field instruction text
+    const instrText = runObj['w:instrText'];
+    const instruction =
+      typeof instrText === 'object' && instrText !== null
+        ? instrText['#text'] || ''
+        : instrText || '';
+    content.push({
+      type: 'fieldInstruction',
+      value: instruction,
+    });
+    break;
+}
+```
+
+### Fix #4: Update Run to Support Field Elements
+
+**File**: `docXMLater/src/elements/Run.ts`
+
+**Update RunContentType**:
+
+```typescript
+export type RunContentType =
+  | 'text'
+  | 'tab'
+  | 'break'
+  | 'carriageReturn'
+  | 'softHyphen'
+  | 'noBreakHyphen'
+  | 'fieldCharacter' // ✅ NEW: w:fldChar elements
+  | 'fieldInstruction'; // ✅ NEW: w:instrText elements
+```
+
+**Update Run.toXML()**:
+
+```typescript
+switch (contentElement.type) {
+  case 'fieldCharacter':
+    runChildren.push(
+      XMLBuilder.wSelf('fldChar', {
+        'w:fldCharType': contentElement.value,
+      })
+    );
+    break;
+
+  case 'fieldInstruction':
+    runChildren.push(
+      XMLBuilder.w(
+        'instrText',
+        {
+          'xml:space': 'preserve',
+        },
+        [contentElement.value || '']
+      )
+    );
+    break;
+}
+```
+
+---
+
+## 🧪 Testing Strategy
+
+### Test Cases
+
+#### Test 1: Simple Field Preservation
+
+```typescript
+// Create document with simple field
+const doc = Document.create();
+const para = new Paragraph();
+para.addField(Field.createPageNumber());
+doc.addParagraph(para);
+
+// Save and reload
+await doc.save('test.docx');
+const doc2 = await Document.load('test.docx');
+
+// Verify field exists
+const paras = doc2.getParagraphs();
+assert(paras[0].getContent().some((item) => item instanceof Field));
+```
+
+#### Test 2: Complex Field Preservation
+
+```typescript
+// Create document with complex field
+const doc = Document.create();
+const para = new Paragraph();
+const complexField = new ComplexField({
+  instruction: ' PAGE \\* MERGEFORMAT ',
+  result: '1',
+});
+para.addField(complexField);
+doc.addParagraph(para);
+
+// Save and reload
+await doc.save('test-complex.docx');
+const doc2 = await Document.load('test-complex.docx');
+
+// Verify complex field preserved
+const content = doc2.getParagraphs()[0].getContent();
+assert(content.some((item) => item instanceof ComplexField));
+```
+
+#### Test 3: Field Order Preservation
+
+```typescript
+// Create paragraph with interleaved content
+const para = new Paragraph();
+para.addText('Page ');
+para.addField(Field.createPageNumber());
+para.addText(' of ');
+para.addField(Field.createTotalPages());
+
+// Save and reload
+// Verify order: Run → Field → Run → Field
+```
+
+### Validation Approach
+
+1. **Unit Tests**: Test each parsing/serialization method independently
+2. **Integration Tests**: Test full load/save cycle
+3. **Real Documents**: Test with actual Word documents containing various field types
+4. **Regression Tests**: Ensure fixes don't break existing functionality
+
+---
+
+## 📝 Additional Observations
+
+### Positive Findings
+
+✅ **The Field classes are well-designed**
+
+- `Field.ts` and `FieldHelpers.ts` provide comprehensive field support
+- Both simple and complex fields have complete implementations
+- Field creation helpers exist for common field types
+
+✅ **Paragraph and Run handling is solid**
+
+- Order preservation works well for runs and hyperlinks
+- The `_orderedChildren` mechanism is clever
+- Type-safe object model prevents many bugs
+
+✅ **dochub-app integration is clean**
+
+- WordDocumentProcessor uses docXMLater APIs correctly
+- Error handling is comprehensive
+- Memory management is excellent
+
+### Areas of Concern
+
+⚠️ **No Field Extraction API**
+
+- WordDocumentProcessor has `extractHyperlinks()` method
+- **No equivalent `extractFields()` method** exists
+- Can't enumerate fields in a document programmatically
+- dochub-app can't validate or report on field preservation
+
+⚠️ **No Field Validation**
+
+- No checks during save to warn about lost fields
+- Silent data loss - users don't know fields were dropped
+- No diff/comparison showing before/after field counts
+
+⚠️ **Limited Field Support in Paragraphpara API**
+
+- `Paragraph.addField()` exists
+- **No `Paragraph.getFields()` method**
+- **No `Paragraph.removeField()` method**
+- Fields can't be queried or manipulated after being added
+
+---
+
+## 🎯 Critical Path to Resolution
+
+### For Immediate Relief (Quick Fix)
+
+**Option A: Document with Warning**
+
+1. Add field count validation before/after processing
+2. Warn users if fields are lost
+3. Document limitation in UI: "Complex fields are not preserved"
+
+**Option B: Raw XML Passthrough**
+
+1. Detect complex fields during load
+2. Store original XML for those paragraphs
+3. Write back unchanged XML during save
+4. Only process paragraphs without complex fields
+
+### For Complete Solution (Full Fix)
+
+**Phase 1: Parsing**
+
+1. Implement `parseComplexFieldFromRunSequence()`
+2. Add `w:fldChar` and `w:instrText` detection to run parser
+3. Add state machine to group field runs into ComplexField objects
+4. Update `parseParagraphWithOrder()` to handle complex fields
+
+**Phase 2: XMLParser Fix**
+
+1. Remove `uniqueTypes.length > 1` condition in `coalesceChildren()`
+2. Always generate `_orderedChildren` when elements exist
+3. Add regression tests for single-type parsing
+
+**Phase 3: Serialization**
+
+1. Update `Paragraph.toXML()` to handle `ComplexField` separately
+2. `ComplexField.toXML()` should return **multiple runs**, not wrapped in single run
+3. Preserve `<w:fldChar>` elements in run serialization
+
+**Phase 4: API Enhancement**
+
+1. Add `Paragraph.getFields()` method
+2. Add `Document.extractFields()` method (like `extractHyperlinks()`)
+3. Add field validation during save
+
+---
+
+## 📚 References and Evidence
+
+### Files Analyzed
+
+#### docXMLater Framework
+
+1. ✅ `src/core/DocumentParser.ts` (1,500+ lines) - Parsing logic
+2. ✅ `src/core/DocumentGenerator.ts` (400+ lines) - Generation logic
+3. ✅ `src/core/Document.ts` (2,000+ lines) - Main API
+4. ✅ `src/elements/Field.ts` (500+ lines) - Field classes
+5. ✅ `src/elements/FieldHelpers.ts` (200+ lines) - Field utilities
+6. ✅ `src/elements/Paragraph.ts` (1,300+ lines) - Paragraph class
+7. ✅ `src/elements/Run.ts` (600+ lines) - Run class
+8. ✅ `src/xml/XMLParser.ts` (700+ lines) - XML parsing
+9. ✅ `src/xml/XMLBuilder.ts` (300+ lines) - XML building
+
+#### dochub-app Application
+
+1. ✅ `src/services/document/WordDocumentProcessor.ts` (1,800+ lines)
+2. ✅ `src/services/document/DocXMLaterProcessor.ts` (500+ lines)
+3. ✅ `docs/architecture/DOCXMLATER_INTEGRATION.md`
+4. ✅ `FIXES_COMPLETED.md`
+5. ✅ `TEST_RESULTS_SUMMARY.md`
+
+### Key Code Locations
+
+**Field Parsing (Simple)**:
+
+- File: `docXMLater/src/core/DocumentParser.ts`
+- Method: `parseSimpleFieldFromObject()`
+- Lines: ~580-616
+- Status: ✅ Working
+
+**Field Parsing (Complex)**:
+
+- File: `docXMLater/src/core/DocumentParser.ts`
+- Method: **DOES NOT EXIST** ❌
+- Expected: `parseComplexFieldFromRunSequence()`
+- Status: ❌ Missing
+
+**Field Detection in Ordered Parsing**:
+
+- File: `docXMLater/src/core/DocumentParser.ts`
+- Method: `parseParagraphWithOrder()`
+- Lines: ~238-310
+- Issue: ✅ Handles `w:fldSimple`, ❌ Ignores `w:fldChar`
+
+**Order Metadata Generation**:
+
+- File: `docXMLater/src/xml/XMLParser.ts`
+- Method: `coalesceChildren()`
+- Lines: ~560-575
+- Issue: ❌ Conditional generation based on `uniqueTypes.length > 1`
+
+**Field Serialization**:
+
+- File: `docXMLater/src/elements/Paragraph.ts`
+- Method: `toXML()`
+- Lines: ~600-650
+- Issue: ❌ Wraps fields in run, always generates `fldSimple`
+
+---
+
+## 🔬 Additional Technical Details
+
+### ComplexField Class Design
+
+The `ComplexField` class in `Field.ts` is **well-designed** and supports:
+
+```typescript
+export class ComplexField {
+  private instruction: string;
+  private result?: string;
+  private instructionFormatting?: RunFormatting;
+  private resultFormatting?: RunFormatting;
+  private nestedFields: ComplexField[];
+  private resultContent: XMLElement[];
+  private multiParagraph: boolean;
+
+  toXML(): XMLElement[] {
+    // Returns ARRAY of run elements (begin, instr, sep, result, end)
+    // ✅ Correctly handles complex field structure
+  }
+}
+```
+
+**The class is ready to use** - it just needs to be instantiated during parsing!
+
+### Why TOC Generation Works But TOC Preservation Doesn't
+
+**File**: `docXMLater/src/core/Document.ts` (lines ~1100-1200)
+
+The framework has a `replaceTableOfContents()` method that:
+
+1. Reads the saved DOCX file
+2. Finds TOC SDT elements in XML
+3. **Replaces them directly in the XML string**
+4. Saves the modified XML back
+
+This works because it **bypasses the object model entirely** - it never tries to parse the complex TOC fields into objects. It's pure XML string manipulation.
+
+**This confirms**: The framework developers **knew** complex fields were problematic and worked around it by using direct XML manipulation instead of object model parsing.
+
+---
+
+## 🚨 Critical Conclusions
+
+### The "Hit or Miss" Behavior Explained
+
+**Fields are preserved when**:
+✅ They use `<w:fldSimple>` structure (simple fields)
+✅ AND paragraph has multiple element types (triggers `_orderedChildren`)
+✅ AND document isn't heavily modified during processing
+
+**Fields are lost when**:
+❌ They use `<w:fldChar>` structure (complex fields) - **ALWAYS LOST**
+❌ OR paragraph has only fields (no `_orderedChildren` → wrong order → may break)
+❌ OR processing modifies paragraph content array (rare but possible)
+
+### Why It Seems Random
+
+Users can't see the difference between simple and complex fields in Word - they look identical. Word chooses the structure automatically based on internal complexity heuristics. This makes the bug appear non-deterministic from the user's perspective.
+
+### Business Impact
+
+**HIGH SEVERITY** - This affects:
+
+- 💼 Legal documents (contracts, agreements)
+- 📊 Report templates (automated fields)
+- 📚 Technical documentation (cross-references)
+- 📄 Forms (merge fields)
+- 📖 Books/manuals (TOC, page numbers, cross-refs)
+
+Any document relying on dynamic fields **will be broken** after processing through dochub-app.
+
+---
+
+## ✅ Verification Steps
+
+To confirm these bugs in your environment:
+
+### Step 1: Create Test Document in Word
+
+1. Open Microsoft Word
+2. Insert → Quick Parts → Field
+3. Choose "Page"→ OK (this creates a PAGE field)
+4. Save as `test-simple-field.docx`
+5. Press Alt+F9 to view field codes
+6. Check if it shows `<w:fldSimple>` or `<w:fldChar>` in the XML
+
+### Step 2: Process Through dochub-app
+
+1. Load `test-simple-field.docx` in dochub-app
+2. Process with minimal settings (no major modifications)
+3. Save result
+4. Open result in Word
+5. Press Alt+F9 - does the field still exist?
+
+### Step 3: Check XML Directly
+
+```bash
+# Extract DOCX (it's a ZIP file)
+unzip test-simple-field.docx -d test-simple
+unzip result.docx -d result
+
+# Compare field presence
+grep -i "fldSimple\|fldChar\|instrText" test-simple/word/document.xml
+grep -i "fldSimple\|fldChar\|instrText" result/word/document.xml
+```
+
+### Step 4: Test Complex Field
+
+1. In Word, create a cross-reference (Insert → Cross-reference)
+2. Save as `test-complex-field.docx`
+3. Process through dochub-app
+4. Check if cross-reference still works
+
+**Expected Result**: Cross-reference will be **broken** (shows literal text or error).
+
+---
+
+## 📋 Recommendations
+
+### Immediate Actions
+
+1. **Document the limitation** in dochub-app user guide
+2. **Add validation** to warn users when fields will be lost
+3. **Consider field count** in processing statistics
+4. **Add field type** to processing options (warn about complex fields)
+
+### Short-Term (1-2 weeks)
+
+1. **Implement Fix #2** (always generate `_orderedChildren`) - Low risk, high impact
+2. **Add `extractFields()` API** to DocXMLaterProcessor for visibility
+3. **Add unit tests** for simple field preservation
+4. **Update documentation** with field preservation status
+
+### Long-Term (1-2 months)
+
+1. **Implement Fix #1** (complex field parsing) - Requires architecture changes
+2. **Add stateful parser** for complex fields
+3. **Update Paragraph API** to support ComplexField
+4. **Comprehensive testing** with real-world documents
+
+### Workaround for Users
+
+Until fixed, users should:
+
+1. **Avoid processing documents with critical fields**
+2. **Re-insert fields manually** after processing if needed
+3. **Use simple field structure** when possible (convert in Word first)
+4. **Keep backups** before processing (dochub-app already does this ✅)
+
+---
+
+## 📞 Support Information
+
+### For Developers
+
+- This analysis file: `FIELD_PRESERVATION_ANALYSIS.md`
+- Framework repo: `c:\Users\DiaTech\Pictures\DiaTech\Programs\DocHub\development\docXMLater`
+- Application repo: `c:\Users\DiaTech\Pictures\DiaTech\Programs\DocHub\development\dochub-app`
+
+### For Bug Reports
+
+Include:
+
+1. Sample DOCX with fields that are lost
+2. XML diff showing before/after field presence
+3. Field type (simple vs complex) from XML inspection
+4. Processing options used in dochub-app
+
+---
+
+## ✍️ Document Metadata
+
+**Analysis Date**: November 14, 2025
+**Files Analyzed**: 14 files, ~10,000 lines of code
+**Bugs Identified**: 5 critical/high severity
+**Fix Complexity**: Medium-High (requires stateful parsing)
+**Breaking Changes**: None (fixes are additive)
+
+---
+
+**END OF ANALYSIS**