@ansonlai/docx-redline-js 0.8.3 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. package/ARCHITECTURE.md +221 -12
  2. package/CHANGELOG.md +803 -586
  3. package/README.md +1018 -901
  4. package/core/current-properties.js +136 -0
  5. package/core/paragraph-text.js +211 -209
  6. package/core/types.js +9 -3
  7. package/core/word-revision-semantics.js +256 -0
  8. package/dist/docx-redline-js.esm.js +4551 -967
  9. package/dist/docx-redline-js.esm.js.map +4 -4
  10. package/dist/docx-redline-js.esm.min.js +112 -106
  11. package/dist/docx-redline-js.esm.min.js.map +4 -4
  12. package/dist/docx-redline.bundle.cjs +4690 -1106
  13. package/dist/docx-redline.bundle.cjs.map +4 -4
  14. package/dist/docx-redline.bundle.js +4690 -1106
  15. package/dist/docx-redline.bundle.js.map +4 -4
  16. package/docs/AGENT_FAST_START.md +2 -0
  17. package/docs/AGENT_KNOWLEDGE_BASE.md +23 -4
  18. package/docs/SKILL_AUTHORING.md +165 -150
  19. package/docs/TESTING.md +1142 -1059
  20. package/docs/schemas/document-operations.schema.json +221 -168
  21. package/document/docx-document.js +1022 -1010
  22. package/engine/oxml-engine.js +567 -560
  23. package/engine/paragraph-mark-revisions.js +292 -0
  24. package/engine/reconstruction-mapper.js +312 -311
  25. package/engine/reconstruction-mode.js +183 -183
  26. package/engine/reconstruction-writer.js +67 -10
  27. package/engine/rpr-helpers.js +42 -0
  28. package/engine/run-builders.js +0 -73
  29. package/engine/surgical-diff-application.js +604 -605
  30. package/engine/table-mode.js +63 -63
  31. package/index.d.ts +51 -1
  32. package/index.js +47 -3
  33. package/node/cli-help.js +1 -0
  34. package/node/cli.js +992 -974
  35. package/orchestration/list-structural-fallback.js +67 -9
  36. package/package.json +1 -1
  37. package/pipeline/diff-engine.js +130 -1
  38. package/pipeline/ingestion-export.js +227 -243
  39. package/pipeline/ingestion-paragraph.js +93 -98
  40. package/pipeline/list-generation.js +415 -419
  41. package/pipeline/pipeline.js +206 -204
  42. package/pipeline/serialization.js +111 -94
  43. package/scripts/export-validation-fixtures.mjs +19 -6
  44. package/scripts/generate-word-revision-semantics-matrix.mjs +134 -0
  45. package/scripts/lib/word-coverage-catalogue.mjs +7 -2
  46. package/scripts/render-multilevel-bullet-visual.ps1 +2 -2
  47. package/scripts/word-com-corpus-visual-suite.ps1 +2 -2
  48. package/scripts/word-com-differential.ps1 +443 -335
  49. package/scripts/word-com-visual-suite.ps1 +2 -2
  50. package/scripts/word-revision-semantics-matrix.ps1 +108 -0
  51. package/services/batch-operation-orchestrator.js +26 -0
  52. package/services/capture-engine.js +228 -226
  53. package/services/document-operation-applier.js +215 -0
  54. package/services/document-operation-contract.js +628 -537
  55. package/services/document-operation-mutations.js +789 -46
  56. package/services/error-recovery.js +17 -0
  57. package/services/list-conversion-diagnostics.js +296 -0
  58. package/services/list-format-change.js +586 -0
  59. package/services/list-format-mutation.js +416 -0
  60. package/services/list-format.js +305 -0
  61. package/services/numbering-allocation.js +642 -0
  62. package/services/numbering-helpers.js +354 -524
  63. package/services/operation-batch-compiler.js +612 -436
  64. package/services/operation-heuristics.js +197 -173
  65. package/services/operation-preflight.js +1 -0
  66. package/services/receipt-collector.js +288 -288
  67. package/services/revision-comment-management.js +634 -594
  68. package/services/revision-fidelity-verifier.js +498 -0
  69. package/services/standalone-docx-plumbing.js +627 -631
  70. package/services/standalone-operation-runner.d.ts +606 -427
  71. package/services/table-reconciliation.js +103 -49
package/docs/TESTING.md CHANGED
@@ -1,1059 +1,1142 @@
1
- # Testing Guide
2
-
3
- This project uses several test lanes because no single oracle can prove that a
4
- tracked-change document is correct. JavaScript assertions can verify exact XML
5
- and return contracts; Microsoft Word can verify what the primary consumer
6
- actually accepts and rejects; real documents expose structures that small
7
- fixtures rarely contain.
8
-
9
- ## Test lanes at a glance
10
-
11
- | Lane | Command | What it proves | What it does not prove |
12
- |---|---|---|---|
13
- | Automated regression suite | `npm test` | API behavior, exact text invariants, OOXML structure, deterministic fuzz cases, and catalogue integrity | That desktop Word accepts the generated package |
14
- | Isolation, types, and lint | `npm run test:isolation`, `npm run check:types`, `npm run lint` | Runtime boundaries, declaration alignment, and static repository rules | Document correctness |
15
- | JavaScript coverage | `npm run test:coverage` | Which source lines, functions, and branches the automated suite executes | That an executed path is correct |
16
- | Synthetic Word differential | `npm run test:word` | Word opens generated packages, sees revisions, and produces the intended text after Accept All and Reject All | The diversity of real-world packages |
17
- | Word visual evidence | `npm run test:word:visual` | Word renders layout-sensitive fixtures in All Markup, Accept All, and Reject All views and writes a pending review manifest | That the rendered pages are visually correct until a reviewer inspects them |
18
- | SuperDoc Word corpus | `npm run test:corpus:word` | The same Word differential on 60 scenarios drawn from 23 reviewed, pinned real English legal/administrative documents while untouched package parts remain byte-identical | Every possible DOCX producer or document type |
19
- | SuperDoc visual evidence | `npm run test:corpus:word:visual` | Word renders 40 focused list, table, long-document, and page-header scenarios in All Markup, Accept All, and Reject All views | Human visual sign-off |
20
- | Visual evidence inspection | `npm run test:visual:inspect` | Automated inspection of rendered Word PDFs across synthetic and SuperDoc suites (page counts, PDF integrity, anomaly detection) | Human visual sign-off |
21
- | Visual failure regressions | `node tests/visual_failure_regression_tests.mjs` | Semantic OOXML guards against visual failures (formatting leaks, font resets, ghost bullets, table cell destruction) | Visual rendering proof in Word |
22
- | Multimodal LLM visual spot check | On-demand / sampled | Evaluates rendered real-document pages with vision models for layout, table alignment, and typography regressions | Full-corpus automated coverage (intentionally decoupled and sampled due to cost/time) |
23
- | XSD and LibreOffice | See [Release validation and independent oracles](#release-validation-and-independent-oracles) | Schema conformance and acceptance by a second consumer | Word-specific revision semantics |
24
- | Agent inspection and package facade | `node tests/document_inspection_tests.mjs`, `node tests/docx_package_facade_tests.mjs` | Canonical text, comment/list resolution, package-scoped IDs, untouched-part preservation, and atomic rollback | Desktop Word rendering |
25
- | Agent CLI | `node tests/agent_cli_tests.mjs`, `node tests/agent_cli_protocol_tests.mjs`, `node tests/agent_cli_help_tests.mjs`, `node tests/agent_cli_inspection_context_tests.mjs`, `node tests/agent_cli_output_budget_tests.mjs` | JSON contracts, command-specific help, contextual pagination, composable progressive/atomic profiles, deduplicated receipts, compact stdout, file/stdin Unicode fidelity, complete-success exits, source safety, command families, and schema readability | Cross-platform CI beyond the current runner |
26
- | Example agent session | `node tests/agent_session_example_tests.mjs`, `node tests/localized_edit_example_tests.mjs` | A non-package sample wrapper delegates to the Node facade, binds and refreshes opaque handles across package revisions, returns context windows, compiles exact localized edits, preserves accepted/rejected lifecycle and atomic rollback, and supports review resolution | LLM/provider latency or a production tool-server transport |
27
- | Batch compilation and recovery | `node tests/batch_source_binding_tests.mjs`, `node tests/error_recovery_contract_tests.mjs`, `node tests/capture_dependency_graph_tests.mjs` | Batch-start source identity survives structural index drift and savepoint restoration; true overlap and capture fan-out fail before mutation; recovery envelopes, retry bases, bounded CLI diagnostics, and complete-success exits remain stable | Whether a model follows the returned recovery action correctly |
28
- | Agent documentation contract | `node tests/agent_documentation_contract_tests.mjs` | The ordinary fast start stays within its 40–60 line/600-word budget, examples lead with scoped contextual extraction, and packaged skill-authoring guidance separates invariants, policies, transports, recovery, and presentation while the development wrapper stays outside package files | Whether a particular model reads or follows the instructions |
29
- | Agent edge cases | `node tests/canonical_paragraph_text_tests.mjs`, `node tests/document_inspection_edge_tests.mjs`, `node tests/docx_package_transaction_edge_tests.mjs`, `node tests/node_zip_archive_tests.mjs`, `node tests/agent_cli_edge_tests.mjs` | Revision-view semantics, cross-paragraph anchors, nested numbering, transaction reuse, multi-author cleanup, malformed ZIP handling, and destructive CLI safeguards | Desktop Word rendering and non-Windows CI |
30
- | Performance boundary regression | `node tests/performance_phase2_boundary_tests.mjs` | Stable facade re-exports, leaf imports, session rollback, isolated context commit, and comment-first scheduling | The Phase 1 one-parse/one-serialize performance target |
31
- | Live-session accuracy and instrumentation | `node tests/performance_phase1_session_tests.mjs` | One full parse/serialization, sequential semantic equivalence, exact accepted/rejected text, valid revisions, list/table/comment/highlight preservation, savepoint no-ops, and zero-serialization rollback | Desktop Word rendering |
32
- | Target hot-path parity | `node tests/performance_phase3_target_hot_paths_tests.mjs` | Cached and uncached resolution metadata parity, duplicate safety, snapshots, session invalidation, and pointer-based revision-ID seeding | Machine-independent latency thresholds |
33
- | List and text-walker parity | `node tests/performance_phase4_list_and_text_parity_tests.mjs` | One shared marker vocabulary and canonical/specialized visible-text agreement across structural characters and revision views | That all specialized mappings are interchangeable |
34
- | Universal document facade & ZIP | `node tests/universal_docx_document_tests.mjs`, `node tests/universal_zip_archive_tests.mjs`, `node tests/universal_facade_edge_cases_tests.mjs` | Universal `Uint8Array` binary handling, `fflate` compression/decompression, optimistic concurrency, and complete-DOCX operations without Node built-ins | Desktop Word rendering |
35
- | Sandbox bundle isolation | `node tests/sandbox_bundle_isolation_tests.mjs` | Zero-dependency standalone ESM/CJS bundles execute in restricted VM sandboxes without `process`, `Buffer`, or `require`, with zero Node built-in imports | Web Worker latency |
36
- | Cross-author revision slicing | `node tests/cross_author_slicing_fixtures_tests.mjs`, `node tests/cross_author_carrier_splitting_tests.mjs`, `node tests/cross_author_slicing_synthetic_tests.mjs`, `node tests/cross_author_slicing_real_tests.mjs`, `node tests/cross_author_slicing_hyperlink_roundtrip_tests.mjs`, `node tests/cross_author_slicing_insertion_stress_tests.mjs` | Word-native `<w:ins>` splitting, `<w:del>` nesting, multi-author stacking, straddle boundaries, synthetic matrix (SYN-01..12d), strict package differential replay (PKG-01..06), exact hyperlink/NBSP round trips, and 76 insertion stress scenarios | Non-Word consumers beyond OpenXML XSD validation |
37
-
38
- The package-facade regression opens a real ZIP buffer, adds a comment beside an
39
- existing high ID, validates OPC wiring, and checks an unrelated binary part is
40
- unchanged after extraction. Its failure case requires byte-exact rollback. The
41
- inspection regression covers revisions, structural characters, heading
42
- context, comment joining, and a numbering start override.
43
-
44
- The CLI regression executes the recommended extract → preflight → apply →
45
- validate flow and also covers accept, reject, comment deletion, output collision
46
- refusal, and source-byte preservation. CLI stdout is captured and parsed as one
47
- JSON value so diagnostic logging cannot corrupt agent output.
48
-
49
- The edge suites deliberately use structures omitted from the basic happy-path
50
- fixtures: `w:moveFrom`/`w:moveTo`, deleted text, structural breaks and hyphens,
51
- comments crossing paragraph boundaries, orphan definitions, outline headings,
52
- footnote/endnote references, nested Roman/alphabetic numbering, sequential
53
- commit-then-rollback calls, selective comment authors, existing high anchor
54
- IDs, encrypted/unsupported ZIP entries, `--force`, and `--in-place`.
55
-
56
- The Phase 2 boundary regression is intentionally structural as well as
57
- behavioral. It prevents the public runner from accumulating implementation
58
- logic again, verifies that internal operation code does not import the root
59
- entry point, and checks that failed atomic work can retain the exact source
60
- string while working maps and sets remain isolated until commit.
61
-
62
- The checked post-Phase-5 2026-09-04 coverage run reports 89.81%
63
- statements/lines, 77.34% branches, and 93.04% functions repository-wide; this
64
- is above every recorded pre-refactor dimension. Targeting performance can be
65
- measured in Node with `npm run benchmark:targeting` and against the native DOM
66
- by serving the repository locally and opening
67
- `scripts/benchmark-targeting-browser.html`; its `paragraphs`, `operations`, and
68
- `iterations` query parameters reproduce the checked fixture sizes.
69
- The Node surface reports 99.61% statements/lines; `document-inspection.js`
70
- reports 97.64% statements/lines and 100% functions; `paragraph-text.js` reports
71
- 100% statements/lines/functions.
72
-
73
- Run the observational live-session benchmark with:
74
-
75
- ```powershell
76
- npm run benchmark:session
77
- ```
78
-
79
- It writes `tmp/benchmarks/operation-session-latest.json`. The checked
80
- 1,000-paragraph/10-operation run reduced full-document parse/serialization from
81
- 10/10 to 1/1 and measured a 1.58x median speedup. Timing and heap figures are
82
- not correctness gates. Per-operation DOM savepoints are retained because
83
- redline accuracy, no-op isolation, and rollback fidelity take precedence over
84
- the aspirational speed target.
85
-
86
- Run the observational agent-protocol benchmark with:
87
-
88
- ```powershell
89
- npm run benchmark:agent
90
- ```
91
-
92
- It writes `tmp/benchmarks/agent-workflow-latest.json` and compares canonical
93
- stateless Node, legacy file-based CLI, compact stdin/profile CLI, and the
94
- development-only session example. It records native/in-process CLI time, heap,
95
- serialized request bytes, protocol calls, instruction words/bytes, exact
96
- accepted/rejected text, comment preservation, ordering, recovery, and
97
- cross-author attribution. It does not measure or estimate LLM reasoning, exact
98
- tokens, tool transport, Claude, or OpenCode wall time; collect those separately
99
- in the calling harness. The checked WP-07 results are summarized in the
100
- [agent protocol rollout audit](validation-reports/2026-09-12-agent-protocol-rollout.md).
101
-
102
- Run the CLI discovery/output benchmark with:
103
-
104
- ```powershell
105
- npm run benchmark:agent-cli
106
- ```
107
-
108
- It records command-specific help size, broad inspect/search response size,
109
- context-window call count, pagination metadata, case-insensitive match parity,
110
- and native elapsed time in `tmp/benchmarks/agent-cli-discovery-latest.json`.
111
- The checked WP-00 through WP-02 before/after measurements are in the
112
- [agent CLI discovery baseline](validation-reports/2026-09-12-agent-cli-discovery-baseline.md).
113
- WP-03 through WP-05 profile, output-size, workflow, documentation, and package
114
- results are in the
115
- [agent CLI efficiency rollout audit](validation-reports/2026-09-13-agent-cli-efficiency-rollout.md).
116
-
117
- ## Cross-author revision slicing test suite
118
-
119
- The cross-author revision slicing subsystem introduces six complementary test lanes:
120
-
121
- 1. **Native Word Desktop Fixtures (`tests/cross_author_slicing_fixtures_tests.mjs`)**:
122
- Verifies 36 reference `.docx` and `.xml` files generated via Microsoft Word Desktop 365 COM automation across 6 golden scenarios (`insert-interior`, `delete-interior`, `delete-boundary-start`, `delete-boundary-end`, `delete-straddle-baseline-insertion`, and `multi-author-stacked`) in pending, accepted, and rejected states. Proves Word-native OOXML patterns (carrier `<w:ins>` splitting for insertions, direct `<w:del>` nesting inside `<w:ins>` for deletions) and lifecycle parity under `acceptTrackedChangesInOoxml` and `rejectTrackedChangesInOoxml`.
123
- 2. **Carrier Splitting Unit Suite (`tests/cross_author_carrier_splitting_tests.mjs`)**:
124
- Tests `splitTrackChangeCarrier` across interior and boundary cuts, multi-run formatting preservation, stable metadata cloning, independent revision ID allocation via `RevisionIdAllocator`, and immutable input handling.
125
- 3. **Synthetic Boundary Matrix (`tests/cross_author_slicing_synthetic_tests.mjs`)**:
126
- Executes the full SYN-01 through SYN-12d test matrix covering pure interior inserts, pure interior deletes, boundary start/end cuts, full-content annihilation, baseline/insertion straddles, multi-carrier straddles, formatting preservation, paired replacements, 3-author stacked revisions, and selective author accept/reject lifecycle oracles.
127
- 4. **Checked-In Word Package Differential Suite (`tests/cross_author_slicing_real_tests.mjs`)**:
128
- Reconstructs the pre-mutation document packages, applies `slice-cross-author` via the strict `openDocx` package facade, enforces atomic package validation, and compares engine `AcceptAll` and `RejectAll` output against Word Desktop-generated golden packages (PKG-01 through PKG-06).
129
- 5. **Hyperlink and Exact-Text Regression (`tests/cross_author_slicing_hyperlink_roundtrip_tests.mjs`)**:
130
- Reproduces the repeated-phrase, hyperlink-boundary, and NBSP-to-space failure through both the low-level engine and document operation runner. It requires exact accepted-view equality, preserves both hyperlink relationship containers, and proves that an unreconstructable slicing mutation returns `PATCH_ROUNDTRIP_MISMATCH` with the original OOXML unchanged.
131
- 6. **Insertion Stress Matrix (`tests/cross_author_slicing_insertion_stress_tests.mjs`)**:
132
- Runs 76 deterministic scenarios across carrier start/end/interior positions, repeated phrases, multi-run and formatted carriers, spaces/tabs/NBSP, XML-sensitive characters, emoji/combining Unicode, hyperlink interiors and boundaries, bookmarks, comments, nested deletions, adjacent authors, mixed same/foreign-author carriers, multiple structural containers, consecutive reviewer rounds, and the atomic document runner. Every supported case requires exact current-view, Accept-All, Reject-Current, schema, metadata, and non-empty-wrapper invariants; unsupported nested structures must fail without throwing or returning changed output.
133
-
134
- ## Coverage matrix and test selection
135
-
136
- Run the deterministic task-by-structure report before choosing the next Word
137
- case:
138
-
139
- ```powershell
140
- npm run report:word:coverage
141
- npm run report:word:coverage -- --json
142
- npm run report:test:dashboard
143
- ```
144
-
145
- The dashboard command writes a self-contained interactive report to
146
- `docs/test-comparison-dashboard.html`. It compares task/structure cells,
147
- synthetic and real-document coverage, independent oracles, visual-render
148
- eligibility, and planned high-priority gaps from the live catalogues. It also
149
- embeds source, tracked, accepted, and rejected packages for every synthetic
150
- fixture and, when the pinned corpus is downloaded, all 60 reviewed real legal
151
- and administrative scenarios. Its `docx-preview` workbench supports arbitrary left/right states,
152
- comparison presets, synchronized scrolling, revision metadata, expected text,
153
- and local downloads; no fixture upload or file picker is required. Real cases
154
- are labeled in the document selector and selected by default when available.
155
- Use **Hide sidebar** beside the dashboard filters to remove the coverage-detail
156
- column and expand the comparison workbench to the full browser width. The
157
- button becomes **Show sidebar**, and the preference is stored locally so the
158
- wide document view survives a reload.
159
- Without the local corpus, generation remains offline-safe and embeds only the
160
- synthetic previews.
161
-
162
- The report combines all synthetic and reviewed SuperDoc scenarios. Synthetic
163
- metadata is declared in `tests/fixtures/word-task-coverage.mjs`; SuperDoc
164
- metadata is declared in `tests/corpus/superdoc-word-coverage.json`. Descriptive
165
- `task`, `shape`, and `coverage` labels remain on the original cases as review
166
- notes, while the matrix uses shared task, structure, and oracle vocabularies.
167
-
168
- Catalogue tests reject unknown vocabulary labels, missing oracle or manual-review
169
- metadata, duplicate identities, structural claims unsupported by fixture XML or
170
- reviewed corpus labels, and uncovered high-priority cells without a recorded
171
- plan or exclusion. The dispositions live in
172
- `tests/fixtures/coverage-matrix-priorities.json` and require both a reason and a
173
- dependency.
174
-
175
- Choose new cases from uncovered high-priority cells first, then prefer a new
176
- task/structure combination over another case in a dense cell. A higher count is
177
- not itself a reason to add a fixture. Adding a catalogue case automatically
178
- changes the report because it reads the live metadata.
179
-
180
- ## Reliability improvement process used for this test expansion
181
-
182
- The current corpus, visual runners, and comparison dashboard were built as one
183
- feedback loop rather than as separate test features. The working sequence is:
184
-
185
- 1. inventory the claimed behavior and existing test lanes;
186
- 2. add realistic cases that expose weak structures;
187
- 3. assert the smallest machine-checkable invariant at the engine level;
188
- 4. package the result into the original real document without changing
189
- unrelated parts;
190
- 5. ask Word to accept and reject the revisions;
191
- 6. render All Markup, Accept All, and Reject All views;
192
- 7. inspect the actual pages at both contact-sheet and full-page scale; and
193
- 8. turn every visual defect into a focused regression plus a corpus assertion.
194
-
195
- This ordering matters. A valid `<w:ins>` or `<w:del>` tree does not establish
196
- that the inserted run inherited the correct font, list indentation, hyperlink
197
- wrapper, table-cell formatting, or header styling. Conversely, a good-looking
198
- PDF does not prove that Reject All restores the exact source. Each stage answers
199
- a different question, and a case is useful only when its intended claim is
200
- explicit.
201
-
202
- ### Start from coverage claims, not fixture count
203
-
204
- Read the task-by-structure matrix and select a real document that adds a missing
205
- or weak combination. The expansion deliberately favored bullets, nested lists,
206
- table cells, headers, long documents, and atomic multi-change batches because
207
- these structures cross more OOXML boundaries than a single plain paragraph.
208
- Several scenarios reuse a pinned source when a new batch exercises a genuinely
209
- different structure; this increases behavioral coverage without pretending
210
- that another copy of the same document is a new source.
211
-
212
- For long real documents, record structural minimums such as word count, table
213
- count, list-paragraph count, section count, and related header/footer parts.
214
- `scripts/prepare-superdoc-word-corpus.mjs` checks these declarations against the
215
- source package. This prevents a scenario description such as “table-heavy” or
216
- “multi-section” from silently drifting away from the actual fixture.
217
-
218
- ### Use multi-change batches to exercise interaction effects
219
-
220
- Single replacements remain valuable for isolating engine behavior, but real
221
- editing sessions usually contain several changes. The larger corpus scenarios
222
- therefore use `operations` to apply independent edits atomically across one
223
- document. Targets are chosen from different list items or table cells when
224
- possible. This catches failures caused by earlier replacements shifting later
225
- anchors, revision-ID allocation across a batch, and formatting state leaking
226
- between operations.
227
-
228
- The batch runner must either produce all requested revisions or roll back the
229
- document. Word then checks the complete accepted and rejected story, not merely
230
- the first target. Untouched ZIP parts are hashed and compared with the pinned
231
- source, so success cannot come from rebuilding or normalizing the rest of the
232
- package.
233
-
234
- ### Expand weak list and table cells with structural operations
235
-
236
- The list/table expansion was driven directly by the task-by-structure matrix;
237
- it did not add more plain replacement cases. The current catalogue contains 109
238
- Word cases: 49 synthetic fixtures and 60 reviewed real-document scenarios. The
239
- `List Change × List` cell now contains 27 cases; the
240
- `Table Reconciliation × Table` cell contains 11 cases.
241
-
242
- The added synthetic cases isolate one structural decision at a time:
243
-
244
- - append one list item;
245
- - append several adjacent items;
246
- - add a nested child;
247
- - insert into the middle of an explicit list range;
248
- - preserve upper- and lower-Roman numbering;
249
- - preserve upper-letter, parenthesized lower-letter, dash, and symbol bullets;
250
- - expand a manually numbered heading whose `w:numId="0"` suppresses numbering
251
- into separate, positively numbered list paragraphs;
252
- - update one table cell;
253
- - insert a table row;
254
- - delete a table row; and
255
- - update several cells in one reconciliation.
256
-
257
- The mixed-attachment fixture replaces one anchor with explicit headings,
258
- ordinary paragraphs, a Markdown table, and a numbered list. It verifies real
259
- `w:pStyle`, `w:tbl`, and positive-numbering structures plus independent Word
260
- Accept All and Reject All results; literal table pipes are forbidden.
261
-
262
- The real-document cases repeat those claims against native numbering and table
263
- markup from municipal notices, board agendas, a long zoning ordinance, PPG
264
- minutes, action tables, and invoice tables. They include consecutive and nested
265
- list insertions, a long-document definition insertion, single- and multi-cell
266
- updates, row insertion/deletion, and invoice row changes. Reusing a reviewed,
267
- hash-pinned source is intentional when the scenario exercises a different
268
- operation; the unit of coverage is the behavioral claim, not the download.
269
-
270
- These cases declare `operation.type` as `list-change` or
271
- `table-reconciliation`. Their accepted/rejected expectations use explicit
272
- contains/absent assertions when a Markdown table payload or a structural list
273
- edit cannot be compared meaningfully to the raw Word source as one exact text
274
- string. Structural expectations still verify real properties such as native
275
- list paragraphs, tables, headers/footers, sections, and minimum document size.
276
- The catalogue tests require these expectations and reject unknown operation or
277
- coverage labels.
278
-
279
- List-style cases do not infer their claim from visible text alone. Synthetic
280
- fixtures declare the required `w:numFmt` values and use complete, schema-ordered
281
- numbering parts: every `w:abstractNum` must precede the first concrete `w:num`
282
- mapping. That ordering is regression-tested because Word otherwise opens the
283
- package but silently substitutes its fallback bullet. Real scenarios declare a
284
- `listStyleExpectation` with the source level, number format, and, where useful,
285
- the exact `w:lvlText`. Corpus preparation resolves the target paragraph's
286
- `w:numId` through `word/numbering.xml` and fails before editing if the pinned
287
- source no longer has that native style. The current real cases exercise upper
288
- and lower Roman, upper and lower letter, parenthesized lower letter, dash, and
289
- symbol bullets across bylaws, agendas, healthcare minutes, and long council
290
- minutes.
291
-
292
- ### Diagnose visual failures in the generated OOXML first
293
-
294
- When a page looks wrong, compare the relevant source and tracked
295
- `word/document.xml` or related part before changing the renderer. Locate the
296
- visible text, inspect its nearest `w:r`, `w:rPr`, wrapper, paragraph properties,
297
- and neighboring runs, then compare those properties with the source. This
298
- separates an engine defect from a Word, PDF, or `docx-preview` display issue.
299
-
300
- Three defects illustrate the approach:
301
-
302
- - The council header date replacement inherited `w:vertAlign="superscript"`
303
- from the trailing ordinal suffix. The fix anchors replacement formatting at
304
- the beginning of the deleted range, and the regression explicitly forbids
305
- `vertAlign` on the inserted runs.
306
- - The prospectus filing-date replacement crossed normal runs and a hyperlink.
307
- A forward-only property lookup had already moved past the beginning of the
308
- deletion when the insertion asked for its formatting, so the new run had no
309
- `w:rPr`. Word and SuperDoc correctly fell back from the surrounding 10 pt to
310
- the document default of 12 pt. The lookup now supports that deliberate
311
- backward query. The focused regression requires `w:sz="20"` and
312
- `w:szCs="20"`, and the real prospectus scenario requires those values on
313
- every inserted text run.
314
- - A wholly inserted list item originally tracked only its text run. Accept All
315
- looked correct and text-only Reject All assertions passed, but Word retained
316
- the untracked paragraph mark as an empty bullet or number. Inserted list
317
- paragraphs now track both their text and paragraph mark. The regression
318
- rejects the change and asserts the exact original paragraph count, text, and
319
- numbering, while the real Word PDFs confirm that no ghost marker remains.
320
-
321
- These assertions test the semantic cause rather than a screenshot pixel. The
322
- visual render remains necessary to confirm that the corrected properties
323
- produce the intended page. The automated test suite in
324
- `tests/visual_failure_regression_tests.mjs` codifies these guards directly in
325
- the regression test pipeline (`npm test`) to prevent regressions before DOCX
326
- packages reach Word.
327
-
328
- ### Regenerate packages; do not reason from stale outputs
329
-
330
- Source downloads are content-addressed and may be reused. Generated tracked,
331
- accepted, rejected, PDF, and dashboard artifacts are outputs and must be
332
- rebuilt after an engine or scenario change. A `Verified cached ...` line from
333
- the corpus fetcher means only that the pinned source `.docx` already matches its
334
- recorded hash; it does not mean the edited package was reused.
335
-
336
- Use this sequence for a corpus-affecting engine change:
337
-
338
- ```powershell
339
- npm test
340
- npm run check:types
341
- npm run lint
342
- npm run test:isolation
343
- npm run test:corpus:word
344
- npm run test:corpus:word:visual
345
- npm run report:test:dashboard
346
- ```
347
-
348
- For faster iteration, `prepare-superdoc-word-corpus.mjs` accepts
349
- `--input-dir` and `--output-dir`, and the visual PowerShell runner accepts
350
- `-Case <scenario-key>` plus custom fixture/output directories. Run the complete
351
- lane before handoff even when a focused render passed.
352
-
353
- ### Treat Word COM cleanup separately from render success
354
-
355
- Word sometimes disconnects a COM document proxy after successfully exporting
356
- a PDF and raises `RPC_E_DISCONNECTED` during `Document.Close()`. The visual
357
- runner uses safe close/quit helpers: a disconnect during post-export cleanup is
358
- reported but does not discard a valid, non-empty PDF; a disconnect during the
359
- actual render causes Word to restart and the case to retry once. Other COM
360
- exceptions still fail the run. This distinction avoids both false failures and
361
- false passes.
362
-
363
- Each visual invocation uses a process-specific fixture directory. That keeps a
364
- Word process left behind by an earlier disconnect from locking or contaminating
365
- the next set of generated DOCX files.
366
-
367
- ### Review both the whole document and the changed page
368
-
369
- For a long document, create low-resolution contact sheets for every page in all
370
- three revision states and inspect them for pagination changes, blank pages,
371
- clipping, table displacement, or large typography shifts. Then inspect every
372
- changed page at full resolution. Contact sheets are good at document-wide
373
- layout; they are not reliable for a 10 pt versus 12 pt difference or a leaked
374
- superscript flag.
375
-
376
- Record the exact scenario, Word version/build, revision view, page, observed
377
- fact, and whether the judgment is certain. The visual manifest remains pending
378
- until a reviewer makes that judgment; successful PDF export alone is not a
379
- visual pass.
380
-
381
- ### Keep the dashboard generated and test its controls
382
-
383
- `scripts/generate-test-dashboard.mjs` owns the dashboard markup, styles, and
384
- behavior. Do not hand-edit `docs/test-comparison-dashboard.html`; rebuild it
385
- with `npm run report:test:dashboard`. The report embeds the four DOCX states so
386
- it remains self-contained and can show the exact generated packages without a
387
- file picker or server.
388
-
389
- Dashboard changes require generator-level assertions in
390
- `tests/test_dashboard_report_tests.mjs`. The sidebar-width control, for
391
- example, is checked for its accessible relationship, full-width CSS state, and
392
- persisted preference. DOCX package parsing remains covered separately by
393
- `tests/docxjs_dashboard_rendering_tests.mjs`. When browser control is available,
394
- also click the control in the generated page and verify that the sidebar is
395
- hidden, both document panes widen, the label changes, and reloading preserves
396
- the selected state.
397
-
398
- ### Definition of done for a discovered real-document defect
399
-
400
- A visual defect is complete only when all of the following are true:
401
-
402
- - the source and faulty generated OOXML have been compared;
403
- - the engine cause is understood rather than masked in CSS or the renderer;
404
- - a minimal automated regression reproduces the structural boundary;
405
- - Accept All yields the requested text and Reject All restores the source;
406
- - the affected real corpus scenario has an appropriate structural or
407
- formatting assertion;
408
- - its source, tracked, accepted, and rejected packages have been regenerated;
409
- - desktop Word differential passes;
410
- - the changed page and the document-wide three-view render have been inspected;
411
- - the self-contained comparison dashboard has been rebuilt; and
412
- - the full automated, isolation, type, lint, and whitespace checks pass.
413
-
414
- Prepare, but do not approve, a human review sample with:
415
-
416
- ```powershell
417
- npm run review:word:prepare -- --cycle=0
418
- ```
419
-
420
- The ignored `tmp/word-manual-review/review-manifest-cycle-0.json` selects all
421
- catalogue families changed in the worktree, a rotating 20% synthetic sample,
422
- and one legal plus one administrative SuperDoc case. Every All Markup, Accept
423
- All, Reject All, and human sign-off field starts as `pending`; the helper cannot
424
- turn them into passes. Increment `--cycle` between releases to rotate the
425
- unchanged sample.
426
-
427
- ## Automated JavaScript tests
428
-
429
- Files matching `tests/*.mjs` are discovered by `scripts/run-tests.mjs`. Tests
430
- use `assert/strict` and each runs in a separate Node process. The runner uses up
431
- to four workers by default while preserving filename-sorted reporting. Set
432
- `DOCX_TEST_CONCURRENCY=1` for deterministic serial reproduction, or another
433
- positive integer for an explicit worker bound. Word, visual, and corpus lanes
434
- remain separate serial commands. Shared OOXML assertions belong in
435
- `tests/helpers/ooxml-assertions.mjs`; XML-provider setup belongs in
436
- `tests/setup-xml-provider.mjs`.
437
-
438
- Measure serial and parallel execution on the same machine with:
439
-
440
- ```powershell
441
- npm run benchmark:tests
442
- ```
443
-
444
- The benchmark performs paired warmups/runs and writes its JSON result to
445
- `tmp/benchmarks/test-runner-latest.json`. The checked Node 24/Windows x64 run
446
- reduced median time from 18.02 seconds to 7.96 seconds (55.80%). Twenty
447
- consecutive four-worker runs completed without a failure.
448
-
449
- For function-level gap work, run:
450
-
451
- ```powershell
452
- npm run test:coverage
453
- npm run coverage:gaps
454
- npm run coverage:gaps -- --json
455
- ```
456
-
457
- The coverage command emits both the text summary and detailed Istanbul JSON.
458
- The gap report includes only runtime production roots, lists each uncovered
459
- function with its file and declaration line, assigns the Phase 3 P0/P1/P2
460
- priority, and fails if a targeted file drops below the checked covered-function
461
- or covered-branch baseline in `tests/coverage-data/phase3-baseline.json`. Because V8
462
- discovers new branch sites when a formerly cold function first executes, review
463
- covered counts and behavior assertions alongside percentages. Do not add ignore
464
- annotations or call private code merely to improve a score; classify a retained
465
- gap with a reachability or environment reason.
466
-
467
- For an engine regression:
468
-
469
- 1. Add the smallest fixed case that reproduces the bug and asserts the exact
470
- result or structured error.
471
- 2. Assert both sides of tracked changes: accepting must produce the requested
472
- text and rejecting must restore the original text exactly.
473
- 3. Assert structural invariants with `validateRedlineOoxml` and the shared OOXML
474
- helpers. Do not use a whitespace-collapsing reader as the oracle.
475
- 4. If engine behavior changed, add the same shape to the deterministic fuzz
476
- corpus so nearby inputs are exercised too.
477
- 5. Run `npm test`, isolation, types, and lint.
478
-
479
- For a mixed attachment regression, also run the content through
480
- `planStructuredReplacement(...)`. Assert the declared block sequence, reject
481
- malformed table grammar, and verify that the result contains real `w:tbl`,
482
- heading styles, positive list numbering IDs, valid non-nested revisions, and
483
- independent Accept/Reject outcomes. Do not treat absence of literal pipe text as
484
- sufficient proof of a table.
485
-
486
- ## Synthetic Microsoft Word tests
487
-
488
- The runtime library contains no COM or Word dependency. Word automation exists
489
- only in Windows development scripts and is intentionally excluded from package
490
- runtime paths.
491
-
492
- The synthetic catalogue is `tests/fixtures/word-task-cases.mjs`. During
493
- `npm run test:word`:
494
-
495
- 1. `scripts/export-validation-fixtures.mjs` applies each operation, runs the
496
- library's structural validator, checks any case-specific `requiredElements`,
497
- and builds a minimal `.docx` under ignored `tmp/word-validation/` storage.
498
- 2. Expected Accept All and Reject All text is derived from the test's edit
499
- intent, not from the library's own revision-management functions.
500
- 3. `scripts/word-com-differential.ps1` opens each fixture through desktop Word
501
- without a repair dialog and confirms Word sees at least one revision.
502
- 4. Word accepts every revision in one fresh document and rejects every revision
503
- in another. Both resulting texts are compared with the intent-derived
504
- expectations.
505
-
506
- To add a synthetic Word case:
507
-
508
- 1. Add a unique lowercase-hyphenated entry to `WORD_TASK_CASES` with
509
- `category: 'legal'` or `'administrative'`, a distinct `task`, and `original`
510
- and `modified` text.
511
- 2. Use `sourceDocumentXml` when the case needs bookmarks, hyperlinks, tables,
512
- content controls, prior revisions, or another deliberate structure.
513
- 3. Add `requiredElements`, such as `{ bookmarkStart: 1, bookmarkEnd: 1 }`, when
514
- retaining a structure is part of the claim. These are minimum namespace-aware
515
- element counts checked before Word runs.
516
- 4. Supply `expectedAcceptedText` and `expectedRejectedText` when the source is a
517
- multi-paragraph document or Word exposes structural separators such as table
518
- row boundaries. Exact comparison is the default.
519
- 5. Run `node tests/word_task_catalog_tests.mjs`, then `npm run test:word` on a
520
- Windows machine with desktop Word installed.
521
-
522
- ### Related-part fixture schema
523
-
524
- The script-only packager supports optional numbering plus comments, footnotes,
525
- endnotes, headers, footers, and external hyperlinks. Add them to a catalogue
526
- case through `packageParts`:
527
-
528
- ```js
529
- {
530
- sourceDocumentXml,
531
- packageParts: {
532
- commentsXml,
533
- footnotesXml,
534
- endnotesXml,
535
- headers: [{
536
- partName: 'header1.xml',
537
- relationshipId: 'rIdHeader1',
538
- xml: headerXml
539
- }],
540
- footers: [{
541
- partName: 'footer1.xml',
542
- relationshipId: 'rIdFooter1',
543
- xml: footerXml
544
- }],
545
- externalHyperlinks: [{
546
- relationshipId: 'rIdPolicy',
547
- target: 'https://example.com/policy'
548
- }]
549
- }
550
- }
551
- ```
552
-
553
- `createCommentsPart`, `createNotesPart`, and `createHeaderFooterPart` in
554
- `tests/fixtures/word-package-parts.mjs` provide small escaped constructors for
555
- the common XML parts. Header/footer `partName` and `relationshipId` values have
556
- deterministic defaults, but explicit values make the corresponding
557
- `w:headerReference`, `w:footerReference`, or `w:hyperlink` easier to audit.
558
-
559
- Before ZIP emission, the packager rejects malformed XML, duplicate relationship
560
- IDs or part names, missing relationship targets, undefined comment/note IDs,
561
- and note parts without separator IDs `-1` and `0`. It generates the required
562
- content-type overrides and document relationships. Every supplied related part
563
- is compared byte-for-byte with the packaged entry, recorded as SHA-256 in the
564
- case sidecar, and rechecked by the Word differential before opening the DOCX.
565
- These helpers remain under `scripts/` and `tests/`; none are shipped through the
566
- runtime entry point.
567
-
568
- ### What the automated Word differential proves
569
-
570
- The Word process is real desktop Microsoft Word, but it is driven invisibly
571
- through COM. For each fixture, the script:
572
-
573
- 1. starts `Word.Application` with alerts and the window disabled;
574
- 2. opens the generated package with `OpenNoRepairDialog`;
575
- 3. fails if Word cannot open it or sees zero tracked revisions;
576
- 4. accepts all revisions and reads `Document.Content.Text`;
577
- 5. closes without saving, reopens the untouched fixture, rejects all revisions,
578
- and reads the text again; and
579
- 6. compares both results with expectations that were not calculated by this
580
- library's own accept/reject implementation.
581
-
582
- Exact comparison is the default. The harness removes Word's terminal paragraph
583
- mark, normalizes CR/LF representation, and removes the characters Word exposes
584
- as table-cell and footnote/endnote reference boundaries; case-specific normalized comparison requires an
585
- explicit reason. Documents are never saved by the automated differential.
586
-
587
- For synthetic fixtures, expectations come directly from `original`, `modified`,
588
- and any explicit full-document expectations in the case. For SuperDoc fixtures,
589
- Word first reads the declared story from the original pinned source document;
590
- every target in a single or multi-change scenario must occur exactly once, and
591
- the accepted expectation is formed by applying those replacements to Word's own
592
- source text. Header scenarios use Word's header stories rather than body text.
593
- The corpus packager hashes every package part other than the one intentionally
594
- replaced by the scenario.
595
-
596
- This proves that Word can consume the package, recognizes the revision markup,
597
- and resolves Accept All and Reject All to the intended text. It does **not**
598
- prove that the document looks right on the page. `Content.Text` cannot detect
599
- bad pagination, awkward revision balloons, shifted table widths, broken tab
600
- alignment, font substitution, changed list indentation, clipped headers,
601
- visually stale fields, or a comment/footnote marker that is technically present
602
- but poorly placed.
603
-
604
- ### Microsoft Word visual review guide
605
-
606
- Human visual review complements—not replaces—the automated differential. The
607
- automated differential proves Word revision semantics by comparing text after
608
- Accept All and Reject All, but this review checks the layout, typography, and
609
- interaction details that `Document.Content.Text` cannot see.
610
-
611
- Review is required for:
612
-
613
- - every new or materially changed synthetic Word case;
614
- - the first case for a new structure, operation type, or related package part;
615
- - any engine change affecting reconstruction, formatting, lists, tables,
616
- fields, tabs/breaks, comments, notes, headers/footers, or revision metadata;
617
- - any case that needs normalized rather than exact text comparison; and
618
- - any automated Word failure whose cause is not immediately textual.
619
-
620
- #### Prepare the review set
621
-
622
- Run the automated lanes first:
623
-
624
- ```powershell
625
- npm run test:word
626
- npm run test:corpus:word
627
- npm run report:word:coverage
628
- npm run review:word:prepare -- --cycle=0
629
- ```
630
-
631
- The last command writes a pending review manifest under ignored
632
- `tmp/word-manual-review/` containing changed catalogue families, the rotating
633
- 20% synthetic release sample, and legal/administrative corpus representatives.
634
- It only prepares the selection: it never records a visual pass or human
635
- sign-off. Use a new cycle number for each release rotation.
636
-
637
- Synthetic documents are generated under `tmp/word-validation/`; reviewed
638
- SuperDoc results are under `tmp/superdoc-word-fixtures/`. Do not commit generated
639
- or downloaded `.docx` files.
640
-
641
- Selection rules:
642
- - every new or changed case;
643
- - every case required by the triggers above;
644
- - at least 20% of unchanged synthetic cases, rotating from the prior release (so
645
- every retained synthetic case receives a human review over five release cycles);
646
- - at least one legal and one administrative SuperDoc result; and
647
- - representative list, table, formatted-run, and anchor/field/content-control
648
- structures.
649
-
650
- A major release or a change to package assembly requires a full visual sweep of
651
- affected structure families.
652
-
653
- Record the installed Word version and build from **File → Account → About Word**.
654
- Differences in rendering can be version-specific.
655
-
656
- #### Configure Word
657
-
658
- For the tracked-change inspection:
659
-
660
- 1. Open the generated fixture directly in desktop Word.
661
- 2. On **Review**, select **All Markup**.
662
- 3. Under **Show Markup**, enable insertions/deletions, formatting, comments, and
663
- all reviewers relevant to the case.
664
- 4. Use the expected balloons/inline display for the document and enable
665
- paragraph marks when checking whitespace, tabs, breaks, lists, and empty
666
- paragraphs.
667
- 5. Do not overwrite the generated fixture. Work on disposable copies if a
668
- saved accepted or rejected view is useful.
669
-
670
- #### Inspect each case (3-view checklist)
671
-
672
- ##### 1. Tracked-change view
673
-
674
- - The document opens without a repair, conversion, or unreadable-content prompt.
675
- - Word shows the expected revision count and author attribution.
676
- - Insertions and deletions are anchored at the intended words or paragraph
677
- marks; a small edit has not become an unexplained whole-paragraph rewrite.
678
- - Untargeted text and surrounding revisions remain unchanged.
679
- - Existing bold, italic, underline, highlighting, fonts, styles, and language
680
- settings remain visually consistent.
681
- - Spaces, tabs, manual breaks, paragraph spacing, indentation, and alignment
682
- look intentional with formatting marks visible.
683
- - Lists retain numbering, levels, continuation, indentation, and marker style.
684
- - Tables retain widths, borders, merged cells, row heights, and alignment.
685
- - Bookmarks, hyperlinks, fields, content controls, comments, and note references
686
- remain in the correct visible location and still behave when activated.
687
- - Headers, footers, section boundaries, page breaks, and pagination remain
688
- stable around the edit.
689
- - Revision balloons and comment balloons point to the correct content and do
690
- not obscure or displace unrelated layout unexpectedly.
691
-
692
- ##### 2. Accept All view
693
-
694
- On a disposable copy, choose **Accept All Changes** and verify:
695
-
696
- - The resulting visible text expresses the intended edit.
697
- - No deletion residue, empty revision wrapper, unexpected blank line, or stale
698
- formatting remains.
699
- - Lists, tables, fields, links, comments, notes, and page layout still work.
700
- - Untargeted content is visually unchanged.
701
-
702
- Close the copy without replacing the generated fixture.
703
-
704
- ##### 3. Reject All view
705
-
706
- Reopen a fresh copy, choose **Reject All Changes**, and verify:
707
-
708
- - The original visible text and formatting are restored.
709
- - Original list numbering, table layout, fields, anchors, and pagination return.
710
- - No content introduced by the edit remains.
711
-
712
- #### Record the visual review result
713
-
714
- Keep the review report with release-validation artifacts. Screenshots may be
715
- stored under ignored `tmp/word-manual-review/<date>/` when useful, but do not
716
- commit corpus document images or document contents without checking rights and
717
- sensitivity.
718
-
719
- Suggested report template:
720
-
721
- ```markdown
722
- # Word visual review — <release/date>
723
-
724
- - Reviewer:
725
- - Review date:
726
- - Word version/build:
727
- - Automated synthetic result:
728
- - Automated corpus result:
729
- - Review type: Human sign-off | AI visual preflight
730
-
731
- | Case | Why selected | All Markup | Accept All | Reject All | Result | Notes |
732
- |---|---|---|---|---|---|---|
733
- | example-case | New table structure | Pass | Pass | Pass | Pass | No layout shift |
734
-
735
- ## Failures or follow-ups
736
-
737
- - None.
738
- ```
739
-
740
- A **Pass** requires all three views to pass. Record **Fail** if rendering or
741
- interaction is wrong even when automated text comparison passes. Turn a failure
742
- into a fixed regression case when possible; otherwise record the exact harness
743
- or Word-version limitation in the active reliability plan.
744
-
745
-
746
- ### AI-assisted Word visual preflight
747
-
748
- Generate repeatable three-view PDF evidence for every layout-sensitive
749
- synthetic fixture with:
750
-
751
- ```powershell
752
- npm run test:word:visual
753
- ```
754
-
755
- The command writes PDFs and `manifest.json` under ignored
756
- `tmp/word-visual-review/rendered/` storage. It verifies that Word opened each
757
- fixture and produced a non-empty, paginated rendering, but leaves the manifest
758
- certification and every visual judgment pending. To render only named cases,
759
- invoke `scripts/word-com-visual-suite.ps1 -Case case-one,case-two` directly.
760
-
761
- An AI agent with Windows computer control may also open the generated fixtures
762
- in the installed desktop Word application, switch among All Markup, Accept All,
763
- and Reject All views, capture screenshots, and inspect them for visible
764
- regressions. This is a useful intermediate oracle because it exercises the same
765
- real UI a person sees rather than only `Document.Content.Text`.
766
-
767
- Run an AI visual preflight:
768
-
769
- - after a phase that changes reconstruction, formatting, list/table behavior,
770
- package assembly, or revision display;
771
- - after every five to ten new Word fixtures;
772
- - for all newly introduced structure families;
773
- - when automated Word passes but the XML change is unusually broad; and
774
- - before asking a human to perform the release sample, so obvious failures are
775
- found first.
776
-
777
- For the focused real-document additions, run:
778
-
779
- ```powershell
780
- npm run test:corpus:word:visual
781
- ```
782
-
783
- This first runs the exact Word differential, then writes 120 PDFs and a pending
784
- manifest under ignored `tmp/superdoc-word-visual-review/rendered/` storage: three
785
- views for 22 list-focused cases, 16 table-focused cases, and two page-header
786
- cases. The long-document set includes 6,000+ word council minutes,
787
- an 8,000+ word zoning resolution, and a 59,000+ word prospectus with 180 tables.
788
- Each invocation packages its fixtures in a process-specific directory so
789
- a Word process left behind by an RPC disconnect cannot lock the next run's
790
- inputs. The runner ignores `RPC_E_DISCONNECTED` during post-export cleanup and
791
- restarts Word once when a disconnect interrupts an actual render. After Word
792
- finishes, the command rebuilds `docs/test-comparison-dashboard.html` from that
793
- exact process-specific fixture directory. The `Verified cached ...` messages
794
- refer only to the pinned source-document downloads; edited comparison DOCX
795
- files are regenerated on every run.
796
-
797
- The AI should use the same selection rules and checklist as a human, inspect at
798
- least the changed cases plus representative legal and administrative samples,
799
- and save screenshots under ignored `tmp/word-manual-review/<date>/` storage.
800
- Its report must identify itself as **AI visual preflight**, record the Word
801
- version/build when available, list the exact cases and views inspected, and
802
- separate observed facts from uncertain visual judgments.
803
-
804
- AI visual review is advisory. It can catch missing or misplaced revisions,
805
- unexpected whole-paragraph markup, obvious pagination shifts, broken tables,
806
- lost indentation, clipped headers, and visibly misplaced anchors. It may miss
807
- subtle font metrics, accessibility issues, field behavior that requires domain
808
- knowledge, or a legally meaningful formatting distinction. An AI pass does not
809
- satisfy the human release sign-off, and sensitive/private documents must not be
810
- opened for AI review without explicit authorization.
811
-
812
- ### Multimodal LLM visual inspection (real-document spot checks)
813
-
814
- Evaluating subtle visual defects—such as shifted table borders, unnatural run
815
- breaks, font substitution, clipping headers, or awkward revision balloons—requires
816
- interpreting rendered document pages visually rather than relying solely on
817
- string assertions or raw XML checks. A multimodal Large Language Model (MLLM)
818
- with vision capabilities can review rendered page images across the three
819
- revision views (`allMarkup`, `acceptAll`, and `rejectAll`) and detect regressions
820
- that are invisible to plain-text diffs.
821
-
822
- #### Why multimodal inspection is kept separate
823
-
824
- Passing high-resolution, multi-page document images through a vision-capable LLM
825
- is **expensive, computationally intensive, and time-consuming**:
826
-
827
- - Each document scenario produces three full-document rendering views (often 1
828
- to 20+ pages per view).
829
- - Sending dozens of high-resolution page images consumes significant token
830
- budgets and model context.
831
- - Latency per case is orders of magnitude higher than programmatic XML parsing
832
- or COM automation.
833
-
834
- For these reasons, multimodal visual inspection **forms a separate check from the
835
- rest of the test suite**. It is never executed on every commit, local test run, or
836
- full batch pipeline.
837
-
838
- #### Sampling strategy: Random real-document spot checks
839
-
840
- Instead of exhaustive or continuous execution, multimodal visual inspection is
841
- conducted as a **random spot check of real documents here and there only**:
842
-
843
- 1. **Focus on authentic real documents:** Spot checks prioritize the
844
- [SuperDoc real-document corpus](#superdoc-real-document-word-tests) rather than
845
- simple synthetic fixtures. Real documents contain organic complexities—such
846
- as multi-column headers, nested bullet styles, non-uniform table borders, and
847
- mixed-font legal numbering—where visual layout errors are most likely to hide.
848
- 2. **Small random sampling:** A typical visual spot-check run selects a random
849
- sample of **2 to 5 real document scenarios** (for example, one administrative
850
- board agenda, one municipal minutes multi-table document, and one complex
851
- zoning ordinance).
852
- 3. **Execution cadence:** Spot checks are run on-demand:
853
- - periodically during active engine refactoring;
854
- - when introducing major changes to run splitting, table reconciliation, or
855
- list reconstruction; or
856
- - as an advisory audit prior to a release cycle.
857
-
858
- #### Multimodal inspection workflow
859
-
860
- When performing a multimodal visual spot check:
861
-
862
- 1. **Render target pages to images:** Render the selected scenario in Word to
863
- PDFs using `scripts/word-com-corpus-visual-suite.ps1 -Case <scenario-name>`,
864
- then convert the relevant pages (or contact sheets) to PNG images using the
865
- visual inspection helper:
866
- ```powershell
867
- node scripts/inspect-visual-evidence.mjs --contact-sheets
868
- ```
869
- 2. **Submit to the multimodal model:** Provide the vision model with the side-by-side
870
- page renders of `allMarkup`, `acceptAll`, and `rejectAll` along with the target
871
- diff and context.
872
- 3. **Targeted visual checklist:** Prompt the model to evaluate specific layout risks:
873
- - **Table cell integrity:** Did column widths shift? Are table borders intact?
874
- Did multi-line cell text collapse or overlap?
875
- - **Typography & Font metrics:** Did any run revert to default font size (e.g.
876
- dropping from 14pt Georgia to 12pt Normal)? Did bold, italic, or underline
877
- bleed into neighboring text?
878
- - **List alignment:** Are bullet indents consistent with existing items? Are
879
- there ghost markers or blank list rows?
880
- - **Page apparatus:** Are headers, footers, and page numbers in their expected
881
- margins without clipping or wrapping anomalies?
882
- 4. **Triaging findings:** If the multimodal model flags a visual abnormality,
883
- treat it as an advisory signal:
884
- - Verify the defect manually in Microsoft Word.
885
- - Trace the visual defect to its underlying OOXML cause (per the
886
- [visual failure diagnosis guidelines](#diagnose-visual-failures-in-the-generated-ooxml-first)).
887
- - Add a minimal semantic regression test in
888
- `tests/visual_failure_regression_tests.mjs` to ensure the issue is permanently
889
- prevented in the automated suite (`npm test`).
890
-
891
- ## SuperDoc real-document Word tests
892
-
893
- The corpus lane references selected documents from SuperDoc's
894
- [docx-corpus](https://docxcorp.us/) under ODC-By 1.0. Source documents are never
895
- committed. `tests/corpus/superdoc-english-legal-administrative.json` pins each
896
- reviewed source and observed SHA-256; `tests/corpus/superdoc-word-scenarios.json`
897
- defines named deterministic scenarios and records the structural coverage they
898
- add. A source may support multiple scenarios when each one adds a distinct
899
- behavioral claim.
900
-
901
- To add a corpus case:
902
-
903
- 1. Select an English legal or administrative document that adds a structure or
904
- task not already represented. Review its content and record why it is
905
- suitable.
906
- 2. Add an explicit pinned reference and observed hash to the manifest. Do not
907
- use a floating or bulk corpus download.
908
- 3. Fetch only that reference with
909
- `npm run corpus:fetch:superdoc -- --id <pinned-id>`.
910
- 4. Inspect the document in Word and choose a target that occurs exactly once in
911
- Word's source text.
912
- 5. Add the deterministic operation, shape, coverage labels, and review note to
913
- `superdoc-word-scenarios.json`. Use `key` plus `sourceId` when adding another
914
- scenario for an already pinned source. Use `operations` for a multi-change
915
- atomic batch, or `part: "word/header<N>.xml"` for a page-header revision.
916
- 6. Run `npm test` for manifest/catalogue checks and `npm run test:corpus:word`
917
- for package hashing plus the Word Accept All/Reject All differential.
918
-
919
- The corpus packager starts from the original `.docx`, replaces only the declared
920
- revision part (`word/document.xml` by default, or a named header part), and
921
- verifies every untouched ZIP part byte-for-byte before Word opens the result.
922
- The Word oracle derives multi-change expectations by applying each independently
923
- declared replacement to Word's source text and reads header-story text separately
924
- ## Release validation and independent oracles
925
-
926
- This package works on OOXML strings and intentionally leaves `.docx` zip
927
- packaging to consumers (release *tooling* assembles minimal `.docx` fixtures
928
- with a script-local zip writer; the published library has no zip dependency).
929
-
930
- The test suite verifies the accept/reject round-trip invariant using the
931
- library's own transforms. Because a shared misconception between the
932
- generator and the resolver would pass those tests silently, release
933
- validation adds **independent oracles**: Microsoft Word, LibreOffice, and
934
- the ECMA-376 schemas.
935
-
936
- ### Pre-release automated check sequence
937
-
938
- ```bash
939
- npm test # includes tests/roundtrip_fuzz_tests.mjs (seeded, deterministic)
940
- npm run test:isolation
941
- npm run check:types
942
- npm run lint
943
- ```
944
-
945
- The fuzz harness generates random paragraph structures and edits, asserting the
946
- round-trip invariant plus `validateRedlineOoxml` on each case. Tune or reproduce with:
947
-
948
- ```bash
949
- FUZZ_SEED=<seed> FUZZ_ITERATIONS=<n> node tests/roundtrip_fuzz_tests.mjs
950
- ```
951
-
952
- A failing case prints its exact reproduction command.
953
-
954
- ### Runtime guardrail
955
-
956
- `validateRedlineOoxml(oxml)` (exported from `index.js`) runs structural
957
- invariants at runtime and returns `{ valid, issues }`. Downstream packagers
958
- should call it before writing engine output into `word/document.xml`.
959
-
960
- ### Export fixtures
961
-
962
- ```bash
963
- node scripts/export-validation-fixtures.mjs
964
- ```
965
-
966
- Writes to `tmp/validation-docx/`, per case:
967
-
968
- - `<name>.document.xml` — generated `word/document.xml` payload
969
- - `<name>.docx` — minimal assembled package
970
- - `<name>.expected.json` — expected accept-all / reject-all plain text,
971
- derived from edit *intent* (not from this library's transforms), so
972
- external consumers act as independent oracles.
973
-
974
- ### Schema validation (ECMA-376 transitional XSD)
975
-
976
- ```bash
977
- node scripts/export-validation-fixtures.mjs
978
- bash scripts/validate-fixtures-xsd.sh
979
- ```
980
-
981
- Downloads (and caches in `.cache/ooxml-schemas/`) the transitional
982
- wordprocessingml schemas from ECMA-376 Part 4, patches the `xml:` namespace
983
- import to resolve offline, and validates every `*.document.xml` fixture with
984
- `xmllint`. Requires `curl`, `unzip`, and `xmllint` (`libxml2-utils` on
985
- Debian/Ubuntu; available on Windows via conda/msys).
986
-
987
- ### LibreOffice consumer check
988
-
989
- ```bash
990
- cd tmp/validation-docx
991
- soffice --headless --convert-to pdf --outdir converted *.docx
992
- ```
993
-
994
- Verifies that a second independent OOXML consumer parses and converts the
995
- fixtures without error.
996
-
997
- ### Continuous validation
998
-
999
- `.github/workflows/validation.yml` runs nightly (and on demand via
1000
- `workflow_dispatch`):
1001
-
1002
- 1. **xsd-schema** — exports fixtures and validates them against the
1003
- ECMA-376 transitional `wml.xsd`.
1004
- 2. **libreoffice** — exports fixtures and converts them with headless
1005
- LibreOffice.
1006
- 3. **fuzz-extended** — 20,000 fuzz round-trip cases with a date-derived
1007
- seed, so every night explores new inputs. A failure log includes the
1008
- exact `FUZZ_SEED` reproduction command.
1009
-
1010
- The Word differential check stays manual because it requires desktop Word;
1011
- run it before tagging a release.
1012
-
1013
- ### JavaScript coverage baselines
1014
-
1015
- ```bash
1016
- npm run test:coverage
1017
- ```
1018
-
1019
- The command runs the complete JavaScript test suite under c8 and prints a
1020
- per-file report. Coverage answers "which implementation paths did the automated
1021
- JavaScript tests execute?" It does not prove that executed paths are correct,
1022
- that generated OOXML opens in Word, or that the real-document corpus is broad
1023
- enough. Treat it as a map for finding thinly tested code; Word, schema,
1024
- LibreOffice, fuzz, and corpus checks provide different evidence.
1025
-
1026
- Historical reference snapshots:
1027
-
1028
- - **Initial Phase 7 baseline (2026-08-29)**:
1029
- | Metric | Coverage |
1030
- |---|---:|
1031
- | Lines / statements | 79.57% |
1032
- | Functions | 80.07% |
1033
- | Branches | 69.22% |
1034
-
1035
- - **Post-Phase-5 snapshot (2026-08-30)**:
1036
- 79.96% lines/statements, 80.95% functions, and 69.52% branches.
1037
-
1038
- - **Post-Phase-5 snapshot (2026-09-04)**:
1039
- 89.81% statements/lines, 93.04% functions, and 77.34% branches repo-wide.
1040
- The Node surface reports 99.61% statements/lines; `document-inspection.js`
1041
- reports 97.64% statements/lines and 100% functions; `paragraph-text.js` reports
1042
- 100% statements/lines/functions.
1043
-
1044
- Preserve snapshots so progress is visible over time rather than presenting
1045
- coverage as a single pass/fail quality score.
1046
-
1047
- ## Choosing where a new test belongs
1048
-
1049
- - A fixed bug always gets an automated regression.
1050
- - A broad engine rule also gets deterministic fuzz coverage.
1051
- - A claim about what Word renders or accepts gets a synthetic Word case.
1052
- - A claim involving package complexity or realistic authoring structures gets
1053
- a reviewed corpus case.
1054
- - A structural XML rule should be checked by runtime validation and, where
1055
- practical, the ECMA-376 XSD lane.
1056
-
1057
- Keep expectations independent of the code under test. In particular, never use
1058
- this library's Accept/Reject helpers to calculate the expected result for the
1059
- Word differential.
1
+ # Testing Guide
2
+
3
+ This project uses several test lanes because no single oracle can prove that a
4
+ tracked-change document is correct. JavaScript assertions can verify exact XML
5
+ and return contracts; Microsoft Word can verify what the primary consumer
6
+ actually accepts and rejects; real documents expose structures that small
7
+ fixtures rarely contain.
8
+
9
+ ## Test lanes at a glance
10
+
11
+ | Lane | Command | What it proves | What it does not prove |
12
+ |---|---|---|---|
13
+ | Automated regression suite | `npm test` | API behavior, exact text invariants, OOXML structure, deterministic fuzz cases, and catalogue integrity | That desktop Word accepts the generated package |
14
+ | Isolation, types, and lint | `npm run test:isolation`, `npm run check:types`, `npm run lint` | Runtime boundaries, declaration alignment, and static repository rules | Document correctness |
15
+ | JavaScript coverage | `npm run test:coverage` | Which source lines, functions, and branches the automated suite executes | That an executed path is correct |
16
+ | Synthetic Word differential | `npm run test:word` | Word opens generated packages, sees revisions, and produces the intended text after Accept All and Reject All | The diversity of real-world packages |
17
+ | Word visual evidence | `npm run test:word:visual` | Word renders layout-sensitive fixtures in All Markup, Accept All, and Reject All views and writes a pending review manifest | That the rendered pages are visually correct until a reviewer inspects them |
18
+ | SuperDoc Word corpus | `npm run test:corpus:word` | The same Word differential on 60 scenarios drawn from 23 reviewed, pinned real English legal/administrative documents while untouched package parts remain byte-identical | Every possible DOCX producer or document type |
19
+ | SuperDoc visual evidence | `npm run test:corpus:word:visual` | Word renders 40 focused list, table, long-document, and page-header scenarios in All Markup, Accept All, and Reject All views | Human visual sign-off |
20
+ | Visual evidence inspection | `npm run test:visual:inspect` | Automated inspection of rendered Word PDFs across synthetic and SuperDoc suites (page counts, PDF integrity, anomaly detection) | Human visual sign-off |
21
+ | Visual failure regressions | `node tests/visual_failure_regression_tests.mjs` | Semantic OOXML guards against visual failures (formatting leaks, font resets, ghost bullets, table cell destruction) | Visual rendering proof in Word |
22
+ | Multimodal LLM visual spot check | On-demand / sampled | Evaluates rendered real-document pages with vision models for layout, table alignment, and typography regressions | Full-corpus automated coverage (intentionally decoupled and sampled due to cost/time) |
23
+ | XSD and LibreOffice | See [Release validation and independent oracles](#release-validation-and-independent-oracles) | Schema conformance and acceptance by a second consumer | Word-specific revision semantics (see the Word revision semantics row) |
24
+ | Word revision semantics | `node tests/word_revision_semantics_tests.mjs` | The Reject All / Accept All projection in `core/word-revision-semantics.js` and the library's reject/accept agree with recorded desktop Word 16 results for each paragraph-mark revision shape (inserted/deleted marks mid-document, last paragraph, table cells, rows, lists, `w:pPrChange`, mixed authors); no Word needed to run | That Word still behaves as recorded (regenerate the expectations) |
25
+ | Revision fidelity verifier | `node tests/revision_fidelity_verifier_tests.mjs` | Post-apply Reject All invariants (`REJECT_*`), `ACCEPT_NEIGHBORS` locality and `UNTRACKED_CREATED_REVISION` for untracked edits, accept-first scoping, modes, baseline cache, and the `REVISION_FIDELITY_VIOLATION` payload through the bare runner and `openDocx` | Desktop Word behavior beyond the recorded semantics |
26
+ | Revision fidelity view cache | `node tests/revision_fidelity_view_cache_tests.mjs` | Session reuse of verified projections as the next baseline: an untracked edit between tracked ones, a refused operation and an unverified commit verify exactly like uncached single operations | Performance numbers (see `scripts/benchmark-operation-session.mjs`) |
27
+ | Paragraph-mark and property layering | `node tests/paragraph_mark_single_site_tests.mjs`, `node tests/current_properties_usage_tests.mjs` | Tracked paragraph-mark `w:ins`/`w:del` is constructed only in `engine/paragraph-mark-revisions.js`; current properties are read only through `core/current-properties.js` (allowlists carry written reasons) | Behavior of the allowed sites |
28
+ | Agent inspection and package facade | `node tests/document_inspection_tests.mjs`, `node tests/docx_package_facade_tests.mjs` | Canonical text, comment/list resolution, package-scoped IDs, untouched-part preservation, and atomic rollback | Desktop Word rendering |
29
+ | Agent CLI | `node tests/agent_cli_tests.mjs`, `node tests/agent_cli_protocol_tests.mjs`, `node tests/agent_cli_help_tests.mjs`, `node tests/agent_cli_inspection_context_tests.mjs`, `node tests/agent_cli_output_budget_tests.mjs` | JSON contracts, command-specific help, contextual pagination, composable progressive/atomic profiles, deduplicated receipts, compact stdout, file/stdin Unicode fidelity, complete-success exits, source safety, command families, and schema readability | Cross-platform CI beyond the current runner |
30
+ | Example agent session | `node tests/agent_session_example_tests.mjs`, `node tests/localized_edit_example_tests.mjs` | A non-package sample wrapper delegates to the Node facade, binds and refreshes opaque handles across package revisions, returns context windows, compiles exact localized edits, preserves accepted/rejected lifecycle and atomic rollback, and supports review resolution | LLM/provider latency or a production tool-server transport |
31
+ | Batch compilation and recovery | `node tests/batch_source_binding_tests.mjs`, `node tests/error_recovery_contract_tests.mjs`, `node tests/capture_dependency_graph_tests.mjs` | Batch-start source identity survives structural index drift and savepoint restoration; true overlap and capture fan-out fail before mutation; recovery envelopes, retry bases, bounded CLI diagnostics, and complete-success exits remain stable | Whether a model follows the returned recovery action correctly |
32
+ | Agent documentation contract | `node tests/agent_documentation_contract_tests.mjs` | The ordinary fast start stays within its 40–60 line/600-word budget, examples lead with scoped contextual extraction, and packaged skill-authoring guidance separates invariants, policies, transports, recovery, and presentation while the development wrapper stays outside package files | Whether a particular model reads or follows the instructions |
33
+ | Agent edge cases | `node tests/canonical_paragraph_text_tests.mjs`, `node tests/document_inspection_edge_tests.mjs`, `node tests/docx_package_transaction_edge_tests.mjs`, `node tests/node_zip_archive_tests.mjs`, `node tests/agent_cli_edge_tests.mjs` | Revision-view semantics, cross-paragraph anchors, nested numbering, transaction reuse, multi-author cleanup, malformed ZIP handling, and destructive CLI safeguards | Desktop Word rendering and non-Windows CI |
34
+ | Performance boundary regression | `node tests/performance_phase2_boundary_tests.mjs` | Stable facade re-exports, leaf imports, session rollback, isolated context commit, and comment-first scheduling | The Phase 1 one-parse/one-serialize performance target |
35
+ | Live-session accuracy and instrumentation | `node tests/performance_phase1_session_tests.mjs` | One full parse/serialization, sequential semantic equivalence, exact accepted/rejected text, valid revisions, list/table/comment/highlight preservation, savepoint no-ops, and zero-serialization rollback | Desktop Word rendering |
36
+ | Target hot-path parity | `node tests/performance_phase3_target_hot_paths_tests.mjs` | Cached and uncached resolution metadata parity, duplicate safety, snapshots, session invalidation, and pointer-based revision-ID seeding | Machine-independent latency thresholds |
37
+ | List and text-walker parity | `node tests/performance_phase4_list_and_text_parity_tests.mjs` | One shared marker vocabulary and canonical/specialized visible-text agreement across structural characters and revision views | That all specialized mappings are interchangeable |
38
+ | Universal document facade & ZIP | `node tests/universal_docx_document_tests.mjs`, `node tests/universal_zip_archive_tests.mjs`, `node tests/universal_facade_edge_cases_tests.mjs` | Universal `Uint8Array` binary handling, `fflate` compression/decompression, optimistic concurrency, and complete-DOCX operations without Node built-ins | Desktop Word rendering |
39
+ | Sandbox bundle isolation | `node tests/sandbox_bundle_isolation_tests.mjs` | Zero-dependency standalone ESM/CJS bundles execute in restricted VM sandboxes without `process`, `Buffer`, or `require`, with zero Node built-in imports | Web Worker latency |
40
+ | Cross-author revision slicing | `node tests/cross_author_slicing_fixtures_tests.mjs`, `node tests/cross_author_carrier_splitting_tests.mjs`, `node tests/cross_author_slicing_synthetic_tests.mjs`, `node tests/cross_author_slicing_real_tests.mjs`, `node tests/cross_author_slicing_hyperlink_roundtrip_tests.mjs`, `node tests/cross_author_slicing_insertion_stress_tests.mjs` | Word-native `<w:ins>` splitting, `<w:del>` nesting, multi-author stacking, straddle boundaries, synthetic matrix (SYN-01..12d), strict package differential replay (PKG-01..06), exact hyperlink/NBSP round trips, and 76 insertion stress scenarios | Non-Word consumers beyond OpenXML XSD validation |
41
+
42
+ The package-facade regression opens a real ZIP buffer, adds a comment beside an
43
+ existing high ID, validates OPC wiring, and checks an unrelated binary part is
44
+ unchanged after extraction. Its failure case requires byte-exact rollback. The
45
+ inspection regression covers revisions, structural characters, heading
46
+ context, comment joining, and a numbering start override.
47
+
48
+ The CLI regression executes the recommended extract → preflight → apply →
49
+ validate flow and also covers accept, reject, comment deletion, output collision
50
+ refusal, and source-byte preservation. CLI stdout is captured and parsed as one
51
+ JSON value so diagnostic logging cannot corrupt agent output.
52
+
53
+ The edge suites deliberately use structures omitted from the basic happy-path
54
+ fixtures: `w:moveFrom`/`w:moveTo`, deleted text, structural breaks and hyphens,
55
+ comments crossing paragraph boundaries, orphan definitions, outline headings,
56
+ footnote/endnote references, nested Roman/alphabetic numbering, sequential
57
+ commit-then-rollback calls, selective comment authors, existing high anchor
58
+ IDs, encrypted/unsupported ZIP entries, `--force`, and `--in-place`.
59
+
60
+ The Phase 2 boundary regression is intentionally structural as well as
61
+ behavioral. It prevents the public runner from accumulating implementation
62
+ logic again, verifies that internal operation code does not import the root
63
+ entry point, and checks that failed atomic work can retain the exact source
64
+ string while working maps and sets remain isolated until commit.
65
+
66
+ The checked post-Phase-5 2026-09-04 coverage run reports 89.81%
67
+ statements/lines, 77.34% branches, and 93.04% functions repository-wide; this
68
+ is above every recorded pre-refactor dimension. Targeting performance can be
69
+ measured in Node with `npm run benchmark:targeting` and against the native DOM
70
+ by serving the repository locally and opening
71
+ `scripts/benchmark-targeting-browser.html`; its `paragraphs`, `operations`, and
72
+ `iterations` query parameters reproduce the checked fixture sizes.
73
+ The Node surface reports 99.61% statements/lines; `document-inspection.js`
74
+ reports 97.64% statements/lines and 100% functions; `paragraph-text.js` reports
75
+ 100% statements/lines/functions.
76
+
77
+ Run the observational live-session benchmark with:
78
+
79
+ ```powershell
80
+ npm run benchmark:session
81
+ ```
82
+
83
+ It writes `tmp/benchmarks/operation-session-latest.json`. The checked
84
+ 1,000-paragraph/10-operation run reduced full-document parse/serialization from
85
+ 10/10 to 1/1 and measured a 1.58x median speedup. Timing and heap figures are
86
+ not correctness gates. Per-operation DOM savepoints are retained because
87
+ redline accuracy, no-op isolation, and rollback fidelity take precedence over
88
+ the aspirational speed target.
89
+
90
+ Run the observational agent-protocol benchmark with:
91
+
92
+ ```powershell
93
+ npm run benchmark:agent
94
+ ```
95
+
96
+ It writes `tmp/benchmarks/agent-workflow-latest.json` and compares canonical
97
+ stateless Node, legacy file-based CLI, compact stdin/profile CLI, and the
98
+ development-only session example. It records native/in-process CLI time, heap,
99
+ serialized request bytes, protocol calls, instruction words/bytes, exact
100
+ accepted/rejected text, comment preservation, ordering, recovery, and
101
+ cross-author attribution. It does not measure or estimate LLM reasoning, exact
102
+ tokens, tool transport, Claude, or OpenCode wall time; collect those separately
103
+ in the calling harness. The checked WP-07 results are summarized in the
104
+ [agent protocol rollout audit](validation-reports/2026-09-12-agent-protocol-rollout.md).
105
+
106
+ Run the CLI discovery/output benchmark with:
107
+
108
+ ```powershell
109
+ npm run benchmark:agent-cli
110
+ ```
111
+
112
+ It records command-specific help size, broad inspect/search response size,
113
+ context-window call count, pagination metadata, case-insensitive match parity,
114
+ and native elapsed time in `tmp/benchmarks/agent-cli-discovery-latest.json`.
115
+ The checked WP-00 through WP-02 before/after measurements are in the
116
+ [agent CLI discovery baseline](validation-reports/2026-09-12-agent-cli-discovery-baseline.md).
117
+ WP-03 through WP-05 profile, output-size, workflow, documentation, and package
118
+ results are in the
119
+ [agent CLI efficiency rollout audit](validation-reports/2026-09-13-agent-cli-efficiency-rollout.md).
120
+
121
+ ## Cross-author revision slicing test suite
122
+
123
+ The cross-author revision slicing subsystem introduces six complementary test lanes:
124
+
125
+ 1. **Native Word Desktop Fixtures (`tests/cross_author_slicing_fixtures_tests.mjs`)**:
126
+ Verifies 36 reference `.docx` and `.xml` files generated via Microsoft Word Desktop 365 COM automation across 6 golden scenarios (`insert-interior`, `delete-interior`, `delete-boundary-start`, `delete-boundary-end`, `delete-straddle-baseline-insertion`, and `multi-author-stacked`) in pending, accepted, and rejected states. Proves Word-native OOXML patterns (carrier `<w:ins>` splitting for insertions, direct `<w:del>` nesting inside `<w:ins>` for deletions) and lifecycle parity under `acceptTrackedChangesInOoxml` and `rejectTrackedChangesInOoxml`.
127
+ 2. **Carrier Splitting Unit Suite (`tests/cross_author_carrier_splitting_tests.mjs`)**:
128
+ Tests `splitTrackChangeCarrier` across interior and boundary cuts, multi-run formatting preservation, stable metadata cloning, independent revision ID allocation via `RevisionIdAllocator`, and immutable input handling.
129
+ 3. **Synthetic Boundary Matrix (`tests/cross_author_slicing_synthetic_tests.mjs`)**:
130
+ Executes the full SYN-01 through SYN-12d test matrix covering pure interior inserts, pure interior deletes, boundary start/end cuts, full-content annihilation, baseline/insertion straddles, multi-carrier straddles, formatting preservation, paired replacements, 3-author stacked revisions, and selective author accept/reject lifecycle oracles.
131
+ 4. **Checked-In Word Package Differential Suite (`tests/cross_author_slicing_real_tests.mjs`)**:
132
+ Reconstructs the pre-mutation document packages, applies `slice-cross-author` via the strict `openDocx` package facade, enforces atomic package validation, and compares engine `AcceptAll` and `RejectAll` output against Word Desktop-generated golden packages (PKG-01 through PKG-06).
133
+ 5. **Hyperlink and Exact-Text Regression (`tests/cross_author_slicing_hyperlink_roundtrip_tests.mjs`)**:
134
+ Reproduces the repeated-phrase, hyperlink-boundary, and NBSP-to-space failure through both the low-level engine and document operation runner. It requires exact accepted-view equality, preserves both hyperlink relationship containers, and proves that an unreconstructable slicing mutation returns `PATCH_ROUNDTRIP_MISMATCH` with the original OOXML unchanged.
135
+ 6. **Insertion Stress Matrix (`tests/cross_author_slicing_insertion_stress_tests.mjs`)**:
136
+ Runs 76 deterministic scenarios across carrier start/end/interior positions, repeated phrases, multi-run and formatted carriers, spaces/tabs/NBSP, XML-sensitive characters, emoji/combining Unicode, hyperlink interiors and boundaries, bookmarks, comments, nested deletions, adjacent authors, mixed same/foreign-author carriers, multiple structural containers, consecutive reviewer rounds, and the atomic document runner. Every supported case requires exact current-view, Accept-All, Reject-Current, schema, metadata, and non-empty-wrapper invariants; unsupported nested structures must fail without throwing or returning changed output.
137
+
138
+ ## Coverage matrix and test selection
139
+
140
+ Run the deterministic task-by-structure report before choosing the next Word
141
+ case:
142
+
143
+ ```powershell
144
+ npm run report:word:coverage
145
+ npm run report:word:coverage -- --json
146
+ npm run report:test:dashboard
147
+ ```
148
+
149
+ The dashboard command writes a self-contained interactive report to
150
+ `docs/test-comparison-dashboard.html`. It compares task/structure cells,
151
+ synthetic and real-document coverage, independent oracles, visual-render
152
+ eligibility, and planned high-priority gaps from the live catalogues. It also
153
+ embeds source, tracked, accepted, and rejected packages for every synthetic
154
+ fixture and, when the pinned corpus is downloaded, all 60 reviewed real legal
155
+ and administrative scenarios. Its `docx-preview` workbench supports arbitrary left/right states,
156
+ comparison presets, synchronized scrolling, revision metadata, expected text,
157
+ and local downloads; no fixture upload or file picker is required. Real cases
158
+ are labeled in the document selector and selected by default when available.
159
+ Use **Hide sidebar** beside the dashboard filters to remove the coverage-detail
160
+ column and expand the comparison workbench to the full browser width. The
161
+ button becomes **Show sidebar**, and the preference is stored locally so the
162
+ wide document view survives a reload.
163
+ Without the local corpus, generation remains offline-safe and embeds only the
164
+ synthetic previews.
165
+
166
+ The report combines all synthetic and reviewed SuperDoc scenarios. Synthetic
167
+ metadata is declared in `tests/fixtures/word-task-coverage.mjs`; SuperDoc
168
+ metadata is declared in `tests/corpus/superdoc-word-coverage.json`. Descriptive
169
+ `task`, `shape`, and `coverage` labels remain on the original cases as review
170
+ notes, while the matrix uses shared task, structure, and oracle vocabularies.
171
+
172
+ Catalogue tests reject unknown vocabulary labels, missing oracle or manual-review
173
+ metadata, duplicate identities, structural claims unsupported by fixture XML or
174
+ reviewed corpus labels, and uncovered high-priority cells without a recorded
175
+ plan or exclusion. The dispositions live in
176
+ `tests/fixtures/coverage-matrix-priorities.json` and require both a reason and a
177
+ dependency.
178
+
179
+ Choose new cases from uncovered high-priority cells first, then prefer a new
180
+ task/structure combination over another case in a dense cell. A higher count is
181
+ not itself a reason to add a fixture. Adding a catalogue case automatically
182
+ changes the report because it reads the live metadata.
183
+
184
+ ## Reliability improvement process used for this test expansion
185
+
186
+ The current corpus, visual runners, and comparison dashboard were built as one
187
+ feedback loop rather than as separate test features. The working sequence is:
188
+
189
+ 1. inventory the claimed behavior and existing test lanes;
190
+ 2. add realistic cases that expose weak structures;
191
+ 3. assert the smallest machine-checkable invariant at the engine level;
192
+ 4. package the result into the original real document without changing
193
+ unrelated parts;
194
+ 5. ask Word to accept and reject the revisions;
195
+ 6. render All Markup, Accept All, and Reject All views;
196
+ 7. inspect the actual pages at both contact-sheet and full-page scale; and
197
+ 8. turn every visual defect into a focused regression plus a corpus assertion.
198
+
199
+ This ordering matters. A valid `<w:ins>` or `<w:del>` tree does not establish
200
+ that the inserted run inherited the correct font, list indentation, hyperlink
201
+ wrapper, table-cell formatting, or header styling. Conversely, a good-looking
202
+ PDF does not prove that Reject All restores the exact source. Each stage answers
203
+ a different question, and a case is useful only when its intended claim is
204
+ explicit.
205
+
206
+ ### Start from coverage claims, not fixture count
207
+
208
+ Read the task-by-structure matrix and select a real document that adds a missing
209
+ or weak combination. The expansion deliberately favored bullets, nested lists,
210
+ table cells, headers, long documents, and atomic multi-change batches because
211
+ these structures cross more OOXML boundaries than a single plain paragraph.
212
+ Several scenarios reuse a pinned source when a new batch exercises a genuinely
213
+ different structure; this increases behavioral coverage without pretending
214
+ that another copy of the same document is a new source.
215
+
216
+ For long real documents, record structural minimums such as word count, table
217
+ count, list-paragraph count, section count, and related header/footer parts.
218
+ `scripts/prepare-superdoc-word-corpus.mjs` checks these declarations against the
219
+ source package. This prevents a scenario description such as “table-heavy” or
220
+ “multi-section” from silently drifting away from the actual fixture.
221
+
222
+ ### Use multi-change batches to exercise interaction effects
223
+
224
+ Single replacements remain valuable for isolating engine behavior, but real
225
+ editing sessions usually contain several changes. The larger corpus scenarios
226
+ therefore use `operations` to apply independent edits atomically across one
227
+ document. Targets are chosen from different list items or table cells when
228
+ possible. This catches failures caused by earlier replacements shifting later
229
+ anchors, revision-ID allocation across a batch, and formatting state leaking
230
+ between operations.
231
+
232
+ The batch runner must either produce all requested revisions or roll back the
233
+ document. Word then checks the complete accepted and rejected story, not merely
234
+ the first target. Untouched ZIP parts are hashed and compared with the pinned
235
+ source, so success cannot come from rebuilding or normalizing the rest of the
236
+ package.
237
+
238
+ ### Expand weak list and table cells with structural operations
239
+
240
+ The list/table expansion was driven directly by the task-by-structure matrix;
241
+ it did not add more plain replacement cases. The current catalogue contains 125
242
+ Word cases: 65 synthetic fixtures and 60 reviewed real-document scenarios. The
243
+ `List Change × List` cell now contains 27 cases; the
244
+ `Table Reconciliation × Table` cell contains 11 cases.
245
+
246
+ The added synthetic cases isolate one structural decision at a time:
247
+
248
+ - append one list item;
249
+ - append several adjacent items;
250
+ - add a nested child;
251
+ - insert into the middle of an explicit list range;
252
+ - preserve upper- and lower-Roman numbering;
253
+ - preserve upper-letter, parenthesized lower-letter, dash, and symbol bullets;
254
+ - expand a manually numbered heading whose `w:numId="0"` suppresses numbering
255
+ into separate, positively numbered list paragraphs;
256
+ - update one table cell;
257
+ - insert a table row;
258
+ - delete a table row; and
259
+ - update several cells in one reconciliation.
260
+
261
+ The mixed-attachment fixture replaces one anchor with explicit headings,
262
+ ordinary paragraphs, a Markdown table, and a numbered list. It verifies real
263
+ `w:pStyle`, `w:tbl`, and positive-numbering structures plus independent Word
264
+ Accept All and Reject All results; literal table pipes are forbidden.
265
+
266
+ The real-document cases repeat those claims against native numbering and table
267
+ markup from municipal notices, board agendas, a long zoning ordinance, PPG
268
+ minutes, action tables, and invoice tables. They include consecutive and nested
269
+ list insertions, a long-document definition insertion, single- and multi-cell
270
+ updates, row insertion/deletion, and invoice row changes. Reusing a reviewed,
271
+ hash-pinned source is intentional when the scenario exercises a different
272
+ operation; the unit of coverage is the behavioral claim, not the download.
273
+
274
+ These cases declare `operation.type` as `list-change` or
275
+ `table-reconciliation`. Their accepted/rejected expectations use explicit
276
+ contains/absent assertions when a Markdown table payload or a structural list
277
+ edit cannot be compared meaningfully to the raw Word source as one exact text
278
+ string. Structural expectations still verify real properties such as native
279
+ list paragraphs, tables, headers/footers, sections, and minimum document size.
280
+ The catalogue tests require these expectations and reject unknown operation or
281
+ coverage labels.
282
+
283
+ List-style cases do not infer their claim from visible text alone. Synthetic
284
+ fixtures declare the required `w:numFmt` values and use complete, schema-ordered
285
+ numbering parts: every `w:abstractNum` must precede the first concrete `w:num`
286
+ mapping. That ordering is regression-tested because Word otherwise opens the
287
+ package but silently substitutes its fallback bullet. Real scenarios declare a
288
+ `listStyleExpectation` with the source level, number format, and, where useful,
289
+ the exact `w:lvlText`. Corpus preparation resolves the target paragraph's
290
+ `w:numId` through `word/numbering.xml` and fails before editing if the pinned
291
+ source no longer has that native style. The current real cases exercise upper
292
+ and lower Roman, upper and lower letter, parenthesized lower letter, dash, and
293
+ symbol bullets across bylaws, agendas, healthcare minutes, and long council
294
+ minutes.
295
+
296
+ ### Diagnose visual failures in the generated OOXML first
297
+
298
+ When a page looks wrong, compare the relevant source and tracked
299
+ `word/document.xml` or related part before changing the renderer. Locate the
300
+ visible text, inspect its nearest `w:r`, `w:rPr`, wrapper, paragraph properties,
301
+ and neighboring runs, then compare those properties with the source. This
302
+ separates an engine defect from a Word, PDF, or `docx-preview` display issue.
303
+
304
+ Three defects illustrate the approach:
305
+
306
+ - The council header date replacement inherited `w:vertAlign="superscript"`
307
+ from the trailing ordinal suffix. The fix anchors replacement formatting at
308
+ the beginning of the deleted range, and the regression explicitly forbids
309
+ `vertAlign` on the inserted runs.
310
+ - The prospectus filing-date replacement crossed normal runs and a hyperlink.
311
+ A forward-only property lookup had already moved past the beginning of the
312
+ deletion when the insertion asked for its formatting, so the new run had no
313
+ `w:rPr`. Word and SuperDoc correctly fell back from the surrounding 10 pt to
314
+ the document default of 12 pt. The lookup now supports that deliberate
315
+ backward query. The focused regression requires `w:sz="20"` and
316
+ `w:szCs="20"`, and the real prospectus scenario requires those values on
317
+ every inserted text run.
318
+ - A wholly inserted list item originally tracked only its text run. Accept All
319
+ looked correct and text-only Reject All assertions passed, but Word retained
320
+ the untracked paragraph mark as an empty bullet or number. Inserted list
321
+ paragraphs now track both their text and paragraph mark. The regression
322
+ rejects the change and asserts the exact original paragraph count, text, and
323
+ numbering, while the real Word PDFs confirm that no ghost marker remains.
324
+
325
+ These assertions test the semantic cause rather than a screenshot pixel. The
326
+ visual render remains necessary to confirm that the corrected properties
327
+ produce the intended page. The automated test suite in
328
+ `tests/visual_failure_regression_tests.mjs` codifies these guards directly in
329
+ the regression test pipeline (`npm test`) to prevent regressions before DOCX
330
+ packages reach Word.
331
+
332
+ ### Regenerate packages; do not reason from stale outputs
333
+
334
+ Source downloads are content-addressed and may be reused. Generated tracked,
335
+ accepted, rejected, PDF, and dashboard artifacts are outputs and must be
336
+ rebuilt after an engine or scenario change. A `Verified cached ...` line from
337
+ the corpus fetcher means only that the pinned source `.docx` already matches its
338
+ recorded hash; it does not mean the edited package was reused.
339
+
340
+ Use this sequence for a corpus-affecting engine change:
341
+
342
+ ```powershell
343
+ npm test
344
+ npm run check:types
345
+ npm run lint
346
+ npm run test:isolation
347
+ npm run test:corpus:word
348
+ npm run test:corpus:word:visual
349
+ npm run report:test:dashboard
350
+ ```
351
+
352
+ For faster iteration, `prepare-superdoc-word-corpus.mjs` accepts
353
+ `--input-dir` and `--output-dir`, and the visual PowerShell runner accepts
354
+ `-Case <scenario-key>` plus custom fixture/output directories. Run the complete
355
+ lane before handoff even when a focused render passed.
356
+
357
+ ### Treat Word COM cleanup separately from render success
358
+
359
+ Word sometimes disconnects a COM document proxy after successfully exporting
360
+ a PDF and raises `RPC_E_DISCONNECTED` during `Document.Close()`. The visual
361
+ runner uses safe close/quit helpers: a disconnect during post-export cleanup is
362
+ reported but does not discard a valid, non-empty PDF; a disconnect during the
363
+ actual render causes Word to restart and the case to retry once. Other COM
364
+ exceptions still fail the run. This distinction avoids both false failures and
365
+ false passes.
366
+
367
+ Each visual invocation uses a process-specific fixture directory. That keeps a
368
+ Word process left behind by an earlier disconnect from locking or contaminating
369
+ the next set of generated DOCX files.
370
+
371
+ ### Review both the whole document and the changed page
372
+
373
+ For a long document, create low-resolution contact sheets for every page in all
374
+ three revision states and inspect them for pagination changes, blank pages,
375
+ clipping, table displacement, or large typography shifts. Then inspect every
376
+ changed page at full resolution. Contact sheets are good at document-wide
377
+ layout; they are not reliable for a 10 pt versus 12 pt difference or a leaked
378
+ superscript flag.
379
+
380
+ Record the exact scenario, Word version/build, revision view, page, observed
381
+ fact, and whether the judgment is certain. The visual manifest remains pending
382
+ until a reviewer makes that judgment; successful PDF export alone is not a
383
+ visual pass.
384
+
385
+ ### Keep the dashboard generated and test its controls
386
+
387
+ `scripts/generate-test-dashboard.mjs` owns the dashboard markup, styles, and
388
+ behavior. Do not hand-edit `docs/test-comparison-dashboard.html`; rebuild it
389
+ with `npm run report:test:dashboard`. The report embeds the four DOCX states so
390
+ it remains self-contained and can show the exact generated packages without a
391
+ file picker or server.
392
+
393
+ Dashboard changes require generator-level assertions in
394
+ `tests/test_dashboard_report_tests.mjs`. The sidebar-width control, for
395
+ example, is checked for its accessible relationship, full-width CSS state, and
396
+ persisted preference. DOCX package parsing remains covered separately by
397
+ `tests/docxjs_dashboard_rendering_tests.mjs`. When browser control is available,
398
+ also click the control in the generated page and verify that the sidebar is
399
+ hidden, both document panes widen, the label changes, and reloading preserves
400
+ the selected state.
401
+
402
+ ### Definition of done for a discovered real-document defect
403
+
404
+ A visual defect is complete only when all of the following are true:
405
+
406
+ - the source and faulty generated OOXML have been compared;
407
+ - the engine cause is understood rather than masked in CSS or the renderer;
408
+ - a minimal automated regression reproduces the structural boundary;
409
+ - Accept All yields the requested text and Reject All restores the source;
410
+ - the affected real corpus scenario has an appropriate structural or
411
+ formatting assertion;
412
+ - its source, tracked, accepted, and rejected packages have been regenerated;
413
+ - desktop Word differential passes;
414
+ - the changed page and the document-wide three-view render have been inspected;
415
+ - the self-contained comparison dashboard has been rebuilt; and
416
+ - the full automated, isolation, type, lint, and whitespace checks pass.
417
+
418
+ Prepare, but do not approve, a human review sample with:
419
+
420
+ ```powershell
421
+ npm run review:word:prepare -- --cycle=0
422
+ ```
423
+
424
+ The ignored `tmp/word-manual-review/review-manifest-cycle-0.json` selects all
425
+ catalogue families changed in the worktree, a rotating 20% synthetic sample,
426
+ and one legal plus one administrative SuperDoc case. Every All Markup, Accept
427
+ All, Reject All, and human sign-off field starts as `pending`; the helper cannot
428
+ turn them into passes. Increment `--cycle` between releases to rotate the
429
+ unchanged sample.
430
+
431
+ ## Automated JavaScript tests
432
+
433
+ Files matching `tests/*.mjs` are discovered by `scripts/run-tests.mjs`. Tests
434
+ use `assert/strict` and each runs in a separate Node process. The runner uses up
435
+ to four workers by default while preserving filename-sorted reporting. Set
436
+ `DOCX_TEST_CONCURRENCY=1` for deterministic serial reproduction, or another
437
+ positive integer for an explicit worker bound. Word, visual, and corpus lanes
438
+ remain separate serial commands. Shared OOXML assertions belong in
439
+ `tests/helpers/ooxml-assertions.mjs`; XML-provider setup belongs in
440
+ `tests/setup-xml-provider.mjs`.
441
+
442
+ Measure serial and parallel execution on the same machine with:
443
+
444
+ ```powershell
445
+ npm run benchmark:tests
446
+ ```
447
+
448
+ The benchmark performs paired warmups/runs and writes its JSON result to
449
+ `tmp/benchmarks/test-runner-latest.json`. The checked Node 24/Windows x64 run
450
+ reduced median time from 18.02 seconds to 7.96 seconds (55.80%). Twenty
451
+ consecutive four-worker runs completed without a failure.
452
+
453
+ For function-level gap work, run:
454
+
455
+ ```powershell
456
+ npm run test:coverage
457
+ npm run coverage:gaps
458
+ npm run coverage:gaps -- --json
459
+ ```
460
+
461
+ The coverage command emits both the text summary and detailed Istanbul JSON.
462
+ The gap report includes only runtime production roots, lists each uncovered
463
+ function with its file and declaration line, assigns the Phase 3 P0/P1/P2
464
+ priority, and fails if a targeted file drops below the checked covered-function
465
+ or covered-branch baseline in `tests/coverage-data/phase3-baseline.json`. Because V8
466
+ discovers new branch sites when a formerly cold function first executes, review
467
+ covered counts and behavior assertions alongside percentages. Do not add ignore
468
+ annotations or call private code merely to improve a score; classify a retained
469
+ gap with a reachability or environment reason.
470
+
471
+ For an engine regression:
472
+
473
+ 1. Add the smallest fixed case that reproduces the bug and asserts the exact
474
+ result or structured error.
475
+ 2. Assert both sides of tracked changes: accepting must produce the requested
476
+ text and rejecting must restore the original text exactly.
477
+ 3. Assert structural invariants with `validateRedlineOoxml` and the shared OOXML
478
+ helpers. Do not use a whitespace-collapsing reader as the oracle.
479
+ 4. If engine behavior changed, add the same shape to the deterministic fuzz
480
+ corpus so nearby inputs are exercised too.
481
+ 5. Run `npm test`, isolation, types, and lint.
482
+
483
+ For a mixed attachment regression, also run the content through
484
+ `planStructuredReplacement(...)`. Assert the declared block sequence, reject
485
+ malformed table grammar, and verify that the result contains real `w:tbl`,
486
+ heading styles, positive list numbering IDs, valid non-nested revisions, and
487
+ independent Accept/Reject outcomes. Do not treat absence of literal pipe text as
488
+ sufficient proof of a table.
489
+
490
+ ## Synthetic Microsoft Word tests
491
+
492
+ Run only one Word job at a time. `scripts/word-com-suite.ps1` and
493
+ `scripts/word-com-corpus-suite.ps1` begin by force-killing every windowless
494
+ `WINWORD` process to clear orphans from aborted runs, which also kills the
495
+ invisible Word instance of any other COM job running concurrently.
496
+
497
+ The runtime library contains no COM or Word dependency. Word automation exists
498
+ only in Windows development scripts and is intentionally excluded from package
499
+ runtime paths.
500
+
501
+ The synthetic catalogue is `tests/fixtures/word-task-cases.mjs`. During
502
+ `npm run test:word`:
503
+
504
+ 1. `scripts/export-validation-fixtures.mjs` applies each operation, runs the
505
+ library's structural validator, checks any case-specific `requiredElements`,
506
+ and builds a minimal `.docx` under ignored `tmp/word-validation/` storage.
507
+ 2. Expected Accept All and Reject All text is derived from the test's edit
508
+ intent, not from the library's own revision-management functions.
509
+ 3. `scripts/word-com-differential.ps1` opens each fixture through desktop Word
510
+ without a repair dialog and confirms Word sees at least one revision.
511
+ 4. Word accepts every revision in one fresh document and rejects every revision
512
+ in another. Both resulting texts are compared with the intent-derived
513
+ expectations.
514
+
515
+ To add a synthetic Word case:
516
+
517
+ 1. Add a unique lowercase-hyphenated entry to `WORD_TASK_CASES` with
518
+ `category: 'legal'` or `'administrative'`, a distinct `task`, and `original`
519
+ and `modified` text.
520
+ 2. Use `sourceDocumentXml` when the case needs bookmarks, hyperlinks, tables,
521
+ content controls, prior revisions, or another deliberate structure.
522
+ 3. Add `requiredElements`, such as `{ bookmarkStart: 1, bookmarkEnd: 1 }`, when
523
+ retaining a structure is part of the claim. These are minimum namespace-aware
524
+ element counts checked before Word runs.
525
+ 4. Supply `expectedAcceptedText` and `expectedRejectedText` when the source is a
526
+ multi-paragraph document or Word exposes structural separators such as table
527
+ row boundaries.
528
+ 5. Run `node tests/word_task_catalog_tests.mjs`, then `npm run test:word` on a
529
+ Windows machine with desktop Word installed.
530
+
531
+ ### Related-part fixture schema
532
+
533
+ The script-only packager supports optional numbering plus comments, footnotes,
534
+ endnotes, headers, footers, and external hyperlinks. Add them to a catalogue
535
+ case through `packageParts`:
536
+
537
+ ```js
538
+ {
539
+ sourceDocumentXml,
540
+ packageParts: {
541
+ commentsXml,
542
+ footnotesXml,
543
+ endnotesXml,
544
+ headers: [{
545
+ partName: 'header1.xml',
546
+ relationshipId: 'rIdHeader1',
547
+ xml: headerXml
548
+ }],
549
+ footers: [{
550
+ partName: 'footer1.xml',
551
+ relationshipId: 'rIdFooter1',
552
+ xml: footerXml
553
+ }],
554
+ externalHyperlinks: [{
555
+ relationshipId: 'rIdPolicy',
556
+ target: 'https://example.com/policy'
557
+ }]
558
+ }
559
+ }
560
+ ```
561
+
562
+ `createCommentsPart`, `createNotesPart`, and `createHeaderFooterPart` in
563
+ `tests/fixtures/word-package-parts.mjs` provide small escaped constructors for
564
+ the common XML parts. Header/footer `partName` and `relationshipId` values have
565
+ deterministic defaults, but explicit values make the corresponding
566
+ `w:headerReference`, `w:footerReference`, or `w:hyperlink` easier to audit.
567
+
568
+ Before ZIP emission, the packager rejects malformed XML, duplicate relationship
569
+ IDs or part names, missing relationship targets, undefined comment/note IDs,
570
+ and note parts without separator IDs `-1` and `0`. It generates the required
571
+ content-type overrides and document relationships. Every supplied related part
572
+ is compared byte-for-byte with the packaged entry, recorded as SHA-256 in the
573
+ case sidecar, and rechecked by the Word differential before opening the DOCX.
574
+ These helpers remain under `scripts/` and `tests/`; none are shipped through the
575
+ runtime entry point.
576
+
577
+ List cases may declare `expectedAcceptedListStrings` / `expectedRejectedListStrings` (per-paragraph
578
+ `ListFormat.ListString`) and `expectedAcceptedListGroupSizes` (per-paragraph
579
+ `ListFormat.List.ListParagraphs.Count`, 0 for a non-list paragraph). The group sizes tell one list from two lists
580
+ that merely read `A.` and `B.` through start values (R2).
581
+
582
+ ### What the automated Word differential proves
583
+
584
+ The Word process is real desktop Microsoft Word, but it is driven invisibly
585
+ through COM. For each fixture, the script:
586
+
587
+ 1. starts `Word.Application` with alerts and the window disabled;
588
+ 2. opens the generated package with `OpenNoRepairDialog`;
589
+ 3. fails if Word cannot open it or sees zero tracked revisions;
590
+ 4. accepts all revisions and reads `Document.Content.Text`;
591
+ 5. closes without saving, reopens the untouched fixture, rejects all revisions,
592
+ and reads the text again; and
593
+ 6. compares both results with expectations that were not calculated by this
594
+ library's own accept/reject implementation.
595
+
596
+ Where the case has a source package the script also compares the paragraph count after
597
+ Reject All with the source package's paragraph count and fails on a mismatch, so
598
+ a leftover or extra paragraph mark is caught even when the text matches.
599
+
600
+ Accept All and Reject All are issued through `Document.Revisions.AcceptAll()` /
601
+ `RejectAll()` (per-range `Revisions` for header stories). Do not use
602
+ `Document.AcceptAllRevisions()` / `RejectAllRevisions()` in COM scripts: when
603
+ the last body paragraph has a tracked-inserted mark they can leave that mark and
604
+ an extra empty paragraph behind, unlike the ribbon command and Word users.
605
+
606
+ Exact comparison is the default. The harness removes Word's terminal paragraph
607
+ mark, normalizes CR/LF representation, and removes the characters Word exposes
608
+ as table-cell and footnote/endnote reference boundaries; case-specific normalized comparison requires an
609
+ explicit reason. Documents are never saved by the automated differential.
610
+
611
+ For synthetic fixtures, expectations come directly from `original`, `modified`,
612
+ and any explicit full-document expectations in the case. For SuperDoc fixtures,
613
+ Word first reads the declared story from the original pinned source document;
614
+ every target in a single or multi-change scenario must occur exactly once, and
615
+ the accepted expectation is formed by applying those replacements to Word's own
616
+ source text. Header scenarios use Word's header stories rather than body text.
617
+ The corpus packager hashes every package part other than the one intentionally
618
+ replaced by the scenario.
619
+
620
+ This proves that Word can consume the package, recognizes the revision markup,
621
+ and resolves Accept All and Reject All to the intended text. It does **not**
622
+ prove that the document looks right on the page. `Content.Text` cannot detect
623
+ bad pagination, awkward revision balloons, shifted table widths, broken tab
624
+ alignment, font substitution, changed list indentation, clipped headers,
625
+ visually stale fields, or a comment/footnote marker that is technically present
626
+ but poorly placed.
627
+
628
+ ### Microsoft Word visual review guide
629
+
630
+ Human visual review complements—not replaces—the automated differential. The
631
+ automated differential proves Word revision semantics by comparing text after
632
+ Accept All and Reject All, but this review checks the layout, typography, and
633
+ interaction details that `Document.Content.Text` cannot see.
634
+
635
+ Review is required for:
636
+
637
+ - every new or materially changed synthetic Word case;
638
+ - the first case for a new structure, operation type, or related package part;
639
+ - any engine change affecting reconstruction, formatting, lists, tables,
640
+ fields, tabs/breaks, comments, notes, headers/footers, or revision metadata;
641
+ - any case that needs normalized rather than exact text comparison; and
642
+ - any automated Word failure whose cause is not immediately textual.
643
+
644
+ #### Prepare the review set
645
+
646
+ Run the automated lanes first:
647
+
648
+ ```powershell
649
+ npm run test:word
650
+ npm run test:corpus:word
651
+ npm run report:word:coverage
652
+ npm run review:word:prepare -- --cycle=0
653
+ ```
654
+
655
+ The last command writes a pending review manifest under ignored
656
+ `tmp/word-manual-review/` containing changed catalogue families, the rotating
657
+ 20% synthetic release sample, and legal/administrative corpus representatives.
658
+ It only prepares the selection: it never records a visual pass or human
659
+ sign-off. Use a new cycle number for each release rotation.
660
+
661
+ Synthetic documents are generated under `tmp/word-validation/`; reviewed
662
+ SuperDoc results are under `tmp/superdoc-word-fixtures/`. Do not commit generated
663
+ or downloaded `.docx` files.
664
+
665
+ Selection rules:
666
+ - every new or changed case;
667
+ - every case required by the triggers above;
668
+ - at least 20% of unchanged synthetic cases, rotating from the prior release (so
669
+ every retained synthetic case receives a human review over five release cycles);
670
+ - at least one legal and one administrative SuperDoc result; and
671
+ - representative list, table, formatted-run, and anchor/field/content-control
672
+ structures.
673
+
674
+ A major release or a change to package assembly requires a full visual sweep of
675
+ affected structure families.
676
+
677
+ Record the installed Word version and build from **File → Account → About Word**.
678
+ Differences in rendering can be version-specific.
679
+
680
+ #### Configure Word
681
+
682
+ For the tracked-change inspection:
683
+
684
+ 1. Open the generated fixture directly in desktop Word.
685
+ 2. On **Review**, select **All Markup**.
686
+ 3. Under **Show Markup**, enable insertions/deletions, formatting, comments, and
687
+ all reviewers relevant to the case.
688
+ 4. Use the expected balloons/inline display for the document and enable
689
+ paragraph marks when checking whitespace, tabs, breaks, lists, and empty
690
+ paragraphs.
691
+ 5. Do not overwrite the generated fixture. Work on disposable copies if a
692
+ saved accepted or rejected view is useful.
693
+
694
+ #### Inspect each case (3-view checklist)
695
+
696
+ ##### 1. Tracked-change view
697
+
698
+ - The document opens without a repair, conversion, or unreadable-content prompt.
699
+ - Word shows the expected revision count and author attribution.
700
+ - Insertions and deletions are anchored at the intended words or paragraph
701
+ marks; a small edit has not become an unexplained whole-paragraph rewrite.
702
+ - Untargeted text and surrounding revisions remain unchanged.
703
+ - Existing bold, italic, underline, highlighting, fonts, styles, and language
704
+ settings remain visually consistent.
705
+ - Spaces, tabs, manual breaks, paragraph spacing, indentation, and alignment
706
+ look intentional with formatting marks visible.
707
+ - Lists retain numbering, levels, continuation, indentation, and marker style.
708
+ - Tables retain widths, borders, merged cells, row heights, and alignment.
709
+ - Bookmarks, hyperlinks, fields, content controls, comments, and note references
710
+ remain in the correct visible location and still behave when activated.
711
+ - Headers, footers, section boundaries, page breaks, and pagination remain
712
+ stable around the edit.
713
+ - Revision balloons and comment balloons point to the correct content and do
714
+ not obscure or displace unrelated layout unexpectedly.
715
+
716
+ ##### 2. Accept All view
717
+
718
+ On a disposable copy, choose **Accept All Changes** and verify:
719
+
720
+ - The resulting visible text expresses the intended edit.
721
+ - No deletion residue, empty revision wrapper, unexpected blank line, or stale
722
+ formatting remains.
723
+ - Lists, tables, fields, links, comments, notes, and page layout still work.
724
+ - Untargeted content is visually unchanged.
725
+
726
+ Close the copy without replacing the generated fixture.
727
+
728
+ ##### 3. Reject All view
729
+
730
+ Reopen a fresh copy, choose **Reject All Changes**, and verify:
731
+
732
+ - The original visible text and formatting are restored.
733
+ - Original list numbering, table layout, fields, anchors, and pagination return.
734
+ - No content introduced by the edit remains.
735
+
736
+ #### Record the visual review result
737
+
738
+ Keep the review report with release-validation artifacts. Screenshots may be
739
+ stored under ignored `tmp/word-manual-review/<date>/` when useful, but do not
740
+ commit corpus document images or document contents without checking rights and
741
+ sensitivity.
742
+
743
+ Suggested report template:
744
+
745
+ ```markdown
746
+ # Word visual review — <release/date>
747
+
748
+ - Reviewer:
749
+ - Review date:
750
+ - Word version/build:
751
+ - Automated synthetic result:
752
+ - Automated corpus result:
753
+ - Review type: Human sign-off | AI visual preflight
754
+
755
+ | Case | Why selected | All Markup | Accept All | Reject All | Result | Notes |
756
+ |---|---|---|---|---|---|---|
757
+ | example-case | New table structure | Pass | Pass | Pass | Pass | No layout shift |
758
+
759
+ ## Failures or follow-ups
760
+
761
+ - None.
762
+ ```
763
+
764
+ A **Pass** requires all three views to pass. Record **Fail** if rendering or
765
+ interaction is wrong even when automated text comparison passes. Turn a failure
766
+ into a fixed regression case when possible; otherwise record the exact harness
767
+ or Word-version limitation in the active reliability plan.
768
+
769
+
770
+ ### AI-assisted Word visual preflight
771
+
772
+ Generate repeatable three-view PDF evidence for every layout-sensitive
773
+ synthetic fixture with:
774
+
775
+ ```powershell
776
+ npm run test:word:visual
777
+ ```
778
+
779
+ The command writes PDFs and `manifest.json` under ignored
780
+ `tmp/word-visual-review/rendered/` storage. It verifies that Word opened each
781
+ fixture and produced a non-empty, paginated rendering, but leaves the manifest
782
+ certification and every visual judgment pending. To render only named cases,
783
+ invoke `scripts/word-com-visual-suite.ps1 -Case case-one,case-two` directly.
784
+
785
+ An AI agent with Windows computer control may also open the generated fixtures
786
+ in the installed desktop Word application, switch among All Markup, Accept All,
787
+ and Reject All views, capture screenshots, and inspect them for visible
788
+ regressions. This is a useful intermediate oracle because it exercises the same
789
+ real UI a person sees rather than only `Document.Content.Text`.
790
+
791
+ Run an AI visual preflight:
792
+
793
+ - after a phase that changes reconstruction, formatting, list/table behavior,
794
+ package assembly, or revision display;
795
+ - after every five to ten new Word fixtures;
796
+ - for all newly introduced structure families;
797
+ - when automated Word passes but the XML change is unusually broad; and
798
+ - before asking a human to perform the release sample, so obvious failures are
799
+ found first.
800
+
801
+ For the focused real-document additions, run:
802
+
803
+ ```powershell
804
+ npm run test:corpus:word:visual
805
+ ```
806
+
807
+ This first runs the exact Word differential, then writes 120 PDFs and a pending
808
+ manifest under ignored `tmp/superdoc-word-visual-review/rendered/` storage: three
809
+ views for 22 list-focused cases, 16 table-focused cases, and two page-header
810
+ cases. The long-document set includes 6,000+ word council minutes,
811
+ an 8,000+ word zoning resolution, and a 59,000+ word prospectus with 180 tables.
812
+ Each invocation packages its fixtures in a process-specific directory so
813
+ a Word process left behind by an RPC disconnect cannot lock the next run's
814
+ inputs. The runner ignores `RPC_E_DISCONNECTED` during post-export cleanup and
815
+ restarts Word once when a disconnect interrupts an actual render. After Word
816
+ finishes, the command rebuilds `docs/test-comparison-dashboard.html` from that
817
+ exact process-specific fixture directory. The `Verified cached ...` messages
818
+ refer only to the pinned source-document downloads; edited comparison DOCX
819
+ files are regenerated on every run.
820
+
821
+ The AI should use the same selection rules and checklist as a human, inspect at
822
+ least the changed cases plus representative legal and administrative samples,
823
+ and save screenshots under ignored `tmp/word-manual-review/<date>/` storage.
824
+ Its report must identify itself as **AI visual preflight**, record the Word
825
+ version/build when available, list the exact cases and views inspected, and
826
+ separate observed facts from uncertain visual judgments.
827
+
828
+ AI visual review is advisory. It can catch missing or misplaced revisions,
829
+ unexpected whole-paragraph markup, obvious pagination shifts, broken tables,
830
+ lost indentation, clipped headers, and visibly misplaced anchors. It may miss
831
+ subtle font metrics, accessibility issues, field behavior that requires domain
832
+ knowledge, or a legally meaningful formatting distinction. An AI pass does not
833
+ satisfy the human release sign-off, and sensitive/private documents must not be
834
+ opened for AI review without explicit authorization.
835
+
836
+ ### Multimodal LLM visual inspection (real-document spot checks)
837
+
838
+ Evaluating subtle visual defects—such as shifted table borders, unnatural run
839
+ breaks, font substitution, clipping headers, or awkward revision balloons—requires
840
+ interpreting rendered document pages visually rather than relying solely on
841
+ string assertions or raw XML checks. A multimodal Large Language Model (MLLM)
842
+ with vision capabilities can review rendered page images across the three
843
+ revision views (`allMarkup`, `acceptAll`, and `rejectAll`) and detect regressions
844
+ that are invisible to plain-text diffs.
845
+
846
+ #### Why multimodal inspection is kept separate
847
+
848
+ Passing high-resolution, multi-page document images through a vision-capable LLM
849
+ is **expensive, computationally intensive, and time-consuming**:
850
+
851
+ - Each document scenario produces three full-document rendering views (often 1
852
+ to 20+ pages per view).
853
+ - Sending dozens of high-resolution page images consumes significant token
854
+ budgets and model context.
855
+ - Latency per case is orders of magnitude higher than programmatic XML parsing
856
+ or COM automation.
857
+
858
+ For these reasons, multimodal visual inspection **forms a separate check from the
859
+ rest of the test suite**. It is never executed on every commit, local test run, or
860
+ full batch pipeline.
861
+
862
+ #### Sampling strategy: Random real-document spot checks
863
+
864
+ Instead of exhaustive or continuous execution, multimodal visual inspection is
865
+ conducted as a **random spot check of real documents here and there only**:
866
+
867
+ 1. **Focus on authentic real documents:** Spot checks prioritize the
868
+ [SuperDoc real-document corpus](#superdoc-real-document-word-tests) rather than
869
+ simple synthetic fixtures. Real documents contain organic complexities—such
870
+ as multi-column headers, nested bullet styles, non-uniform table borders, and
871
+ mixed-font legal numbering—where visual layout errors are most likely to hide.
872
+ 2. **Small random sampling:** A typical visual spot-check run selects a random
873
+ sample of **2 to 5 real document scenarios** (for example, one administrative
874
+ board agenda, one municipal minutes multi-table document, and one complex
875
+ zoning ordinance).
876
+ 3. **Execution cadence:** Spot checks are run on-demand:
877
+ - periodically during active engine refactoring;
878
+ - when introducing major changes to run splitting, table reconciliation, or
879
+ list reconstruction; or
880
+ - as an advisory audit prior to a release cycle.
881
+
882
+ #### Multimodal inspection workflow
883
+
884
+ When performing a multimodal visual spot check:
885
+
886
+ 1. **Render target pages to images:** Render the selected scenario in Word to
887
+ PDFs using `scripts/word-com-corpus-visual-suite.ps1 -Case <scenario-name>`,
888
+ then convert the relevant pages (or contact sheets) to PNG images using the
889
+ visual inspection helper:
890
+ ```powershell
891
+ node scripts/inspect-visual-evidence.mjs --contact-sheets
892
+ ```
893
+ 2. **Submit to the multimodal model:** Provide the vision model with the side-by-side
894
+ page renders of `allMarkup`, `acceptAll`, and `rejectAll` along with the target
895
+ diff and context.
896
+ 3. **Targeted visual checklist:** Prompt the model to evaluate specific layout risks:
897
+ - **Table cell integrity:** Did column widths shift? Are table borders intact?
898
+ Did multi-line cell text collapse or overlap?
899
+ - **Typography & Font metrics:** Did any run revert to default font size (e.g.
900
+ dropping from 14pt Georgia to 12pt Normal)? Did bold, italic, or underline
901
+ bleed into neighboring text?
902
+ - **List alignment:** Are bullet indents consistent with existing items? Are
903
+ there ghost markers or blank list rows?
904
+ - **Page apparatus:** Are headers, footers, and page numbers in their expected
905
+ margins without clipping or wrapping anomalies?
906
+ 4. **Triaging findings:** If the multimodal model flags a visual abnormality,
907
+ treat it as an advisory signal:
908
+ - Verify the defect manually in Microsoft Word.
909
+ - Trace the visual defect to its underlying OOXML cause (per the
910
+ [visual failure diagnosis guidelines](#diagnose-visual-failures-in-the-generated-ooxml-first)).
911
+ - Add a minimal semantic regression test in
912
+ `tests/visual_failure_regression_tests.mjs` to ensure the issue is permanently
913
+ prevented in the automated suite (`npm test`).
914
+
915
+ ## SuperDoc real-document Word tests
916
+
917
+ The corpus lane references selected documents from SuperDoc's
918
+ [docx-corpus](https://docxcorp.us/) under ODC-By 1.0. Source documents are never
919
+ committed. `tests/corpus/superdoc-english-legal-administrative.json` pins each
920
+ reviewed source and observed SHA-256; `tests/corpus/superdoc-word-scenarios.json`
921
+ defines named deterministic scenarios and records the structural coverage they
922
+ add. A source may support multiple scenarios when each one adds a distinct
923
+ behavioral claim.
924
+
925
+ To add a corpus case:
926
+
927
+ 1. Select an English legal or administrative document that adds a structure or
928
+ task not already represented. Review its content and record why it is
929
+ suitable.
930
+ 2. Add an explicit pinned reference and observed hash to the manifest. Do not
931
+ use a floating or bulk corpus download.
932
+ 3. Fetch only that reference with
933
+ `npm run corpus:fetch:superdoc -- --id <pinned-id>`.
934
+ 4. Inspect the document in Word and choose a target that occurs exactly once in
935
+ Word's source text.
936
+ 5. Add the deterministic operation, shape, coverage labels, and review note to
937
+ `superdoc-word-scenarios.json`. Use `key` plus `sourceId` when adding another
938
+ scenario for an already pinned source. Use `operations` for a multi-change
939
+ atomic batch, or `part: "word/header<N>.xml"` for a page-header revision.
940
+ 6. Run `npm test` for manifest/catalogue checks and `npm run test:corpus:word`
941
+ for package hashing plus the Word Accept All/Reject All differential.
942
+
943
+ The corpus packager starts from the original `.docx`, replaces only the declared
944
+ revision part (`word/document.xml` by default, or a named header part), and
945
+ verifies every untouched ZIP part byte-for-byte before Word opens the result.
946
+ The Word oracle derives multi-change expectations by applying each independently
947
+ declared replacement to Word's source text and reads header-story text separately
948
+ ## Release validation and independent oracles
949
+
950
+ This package works on OOXML strings and intentionally leaves `.docx` zip
951
+ packaging to consumers (release *tooling* assembles minimal `.docx` fixtures
952
+ with a script-local zip writer; the published library has no zip dependency).
953
+
954
+ The test suite verifies the accept/reject round-trip invariant using the
955
+ library's own transforms. Because a shared misconception between the
956
+ generator and the resolver would pass those tests silently, release
957
+ validation adds **independent oracles**: Microsoft Word, LibreOffice, and
958
+ the ECMA-376 schemas.
959
+
960
+ ### Pre-release automated check sequence
961
+
962
+ ```bash
963
+ npm test # includes tests/roundtrip_fuzz_tests.mjs (seeded, deterministic)
964
+ npm run test:isolation
965
+ npm run check:types
966
+ npm run lint
967
+ ```
968
+
969
+ The fuzz harness generates random paragraph structures and edits, asserting the
970
+ round-trip invariant plus `validateRedlineOoxml` on each case. Tune or reproduce with:
971
+
972
+ ```bash
973
+ FUZZ_SEED=<seed> FUZZ_ITERATIONS=<n> node tests/roundtrip_fuzz_tests.mjs
974
+ ```
975
+
976
+ A failing case prints its exact reproduction command.
977
+
978
+ ### Reject-roundtrip matrix
979
+
980
+ `tests/reject_roundtrip_matrix_tests.mjs` (generators in
981
+ `tests/helpers/operation-generators.mjs`, fixtures in
982
+ `tests/helpers/reject-roundtrip-fixtures.mjs`) applies seeded, generated
983
+ operations (word/sentence/paragraph/range replace, insert-after, delete,
984
+ `replacements`, format, paragraph-format, plain-to-list, last-paragraph) to the
985
+ nested-lists fixture and synthetic documents (hyperlinks, breaks/tabs, simple
986
+ fields, tables, multi-author revisions, empty list items). Each case runs
987
+ through both `applyOperationsToDocumentXml` and `openDocx` with default options.
988
+ A `REVISION_FIDELITY_VIOLATION` refusal, a rejected view that differs from the
989
+ source, a missing intended accepted text, or bare/packaged divergence fails the
990
+ run; other refusals are counted as skipped. The default run (fixed seed, 150
991
+ cases, about 3 s) is part of `npm test`. Extended lane and replay:
992
+
993
+ ```bash
994
+ FUZZ_SEED=<seed> MATRIX_ITERATIONS=<n> node tests/reject_roundtrip_matrix_tests.mjs
995
+ ```
996
+
997
+ `MATRIX_VERBOSE=1` prints each skipped refusal; `MATRIX_MAX_REPORTED=<n>` raises
998
+ the failure print limit. Known unfixable cases go in
999
+ `tests/fixtures/reject-roundtrip-allowlist.json` (`{"entries":[{"fixture","kind","path","reason"}]}`);
1000
+ the file does not exist while the allowlist is empty.
1001
+
1002
+ The nightly `matrix-extended` job in `.github/workflows/validation.yml` runs 1000 cases with a date-derived `FUZZ_SEED`;
1003
+ a failure prints the seed, fixture, kind, operation and verifier violations. `tests/list_parser_diagnostics_tests.mjs` asserts that the
1004
+ list conversion paths emit no `console.warn`/`console.error` parser diagnostics under the pure-JS fallback and the explicit xmldom provider.
1005
+
1006
+ ### Word revision semantics matrix
1007
+
1008
+ `tests/fixtures/word-revision-semantics/` holds small packages (`cases.json`
1009
+ lists them) covering each paragraph-mark revision shape, and
1010
+ `expectations.json`, what desktop Word 16 produced on Reject All and Accept All
1011
+ for each. `tests/word_revision_semantics_tests.mjs` checks the projection and
1012
+ the library's reject/accept against those recorded results and needs no Word.
1013
+ `KNOWN_DIVERGENCES` in that test lists current differences from Word; the suite
1014
+ fails when a listed divergence disappears or an unlisted one appears.
1015
+
1016
+ Regenerate the packages and re-record expectations after changing the case list
1017
+ or when Word's behavior may have changed (requires desktop Word; use `pwsh`,
1018
+ PowerShell 7, because Windows PowerShell 5.1 hangs in `SaveAs2` over COM here):
1019
+
1020
+ ```bash
1021
+ node scripts/generate-word-revision-semantics-matrix.mjs
1022
+ pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/word-revision-semantics-matrix.ps1
1023
+ ```
1024
+
1025
+ Review the `expectations.json` diff before committing it.
1026
+
1027
+ ### Revision fidelity verification in tests
1028
+
1029
+ Every tracked operation is verified after it is applied (default `enforce`).
1030
+ Diagnostic lanes may override the default without code changes by setting
1031
+ `DOCX_REDLINE_REVISION_FIDELITY=warn` (apply and report) or `off`. An explicit
1032
+ `verifyRevisionFidelity` option or CLI `--revision-fidelity` flag wins over the
1033
+ environment variable. Use it only to investigate a refusal; the reject
1034
+ round-trip matrix and `tests/revision_fidelity_verifier_tests.mjs` run with the
1035
+ default.
1036
+
1037
+ ### Runtime guardrail
1038
+
1039
+ `validateRedlineOoxml(oxml)` (exported from `index.js`) runs structural
1040
+ invariants at runtime and returns `{ valid, issues }`. Downstream packagers
1041
+ should call it before writing engine output into `word/document.xml`.
1042
+
1043
+ ### Export fixtures
1044
+
1045
+ ```bash
1046
+ node scripts/export-validation-fixtures.mjs
1047
+ ```
1048
+
1049
+ Writes to `tmp/validation-docx/`, per case:
1050
+
1051
+ - `<name>.document.xml` — generated `word/document.xml` payload
1052
+ - `<name>.docx` — minimal assembled package
1053
+ - `<name>.expected.json` — expected accept-all / reject-all plain text,
1054
+ derived from edit *intent* (not from this library's transforms), so
1055
+ external consumers act as independent oracles.
1056
+
1057
+ ### Schema validation (ECMA-376 transitional XSD)
1058
+
1059
+ ```bash
1060
+ node scripts/export-validation-fixtures.mjs
1061
+ bash scripts/validate-fixtures-xsd.sh
1062
+ ```
1063
+
1064
+ Downloads (and caches in `.cache/ooxml-schemas/`) the transitional
1065
+ wordprocessingml schemas from ECMA-376 Part 4, patches the `xml:` namespace
1066
+ import to resolve offline, and validates every `*.document.xml` fixture with
1067
+ `xmllint`. Requires `curl`, `unzip`, and `xmllint` (`libxml2-utils` on
1068
+ Debian/Ubuntu; available on Windows via conda/msys).
1069
+
1070
+ ### LibreOffice consumer check
1071
+
1072
+ ```bash
1073
+ cd tmp/validation-docx
1074
+ soffice --headless --convert-to pdf --outdir converted *.docx
1075
+ ```
1076
+
1077
+ Verifies that a second independent OOXML consumer parses and converts the
1078
+ fixtures without error.
1079
+
1080
+ ### Continuous validation
1081
+
1082
+ `.github/workflows/validation.yml` runs nightly (and on demand via
1083
+ `workflow_dispatch`):
1084
+
1085
+ 1. **xsd-schema** — exports fixtures and validates them against the
1086
+ ECMA-376 transitional `wml.xsd`.
1087
+ 2. **libreoffice** — exports fixtures and converts them with headless
1088
+ LibreOffice.
1089
+ 3. **fuzz-extended** — 20,000 fuzz round-trip cases with a date-derived
1090
+ seed, so every night explores new inputs. A failure log includes the
1091
+ exact `FUZZ_SEED` reproduction command.
1092
+
1093
+ The Word differential check stays manual because it requires desktop Word;
1094
+ run it before tagging a release.
1095
+
1096
+ ### JavaScript coverage baselines
1097
+
1098
+ ```bash
1099
+ npm run test:coverage
1100
+ ```
1101
+
1102
+ The command runs the complete JavaScript test suite under c8 and prints a
1103
+ per-file report. Coverage answers "which implementation paths did the automated
1104
+ JavaScript tests execute?" It does not prove that executed paths are correct,
1105
+ that generated OOXML opens in Word, or that the real-document corpus is broad
1106
+ enough. Treat it as a map for finding thinly tested code; Word, schema,
1107
+ LibreOffice, fuzz, and corpus checks provide different evidence.
1108
+
1109
+ Historical reference snapshots:
1110
+
1111
+ - **Initial Phase 7 baseline (2026-08-29)**:
1112
+ | Metric | Coverage |
1113
+ |---|---:|
1114
+ | Lines / statements | 79.57% |
1115
+ | Functions | 80.07% |
1116
+ | Branches | 69.22% |
1117
+
1118
+ - **Post-Phase-5 snapshot (2026-08-30)**:
1119
+ 79.96% lines/statements, 80.95% functions, and 69.52% branches.
1120
+
1121
+ - **Post-Phase-5 snapshot (2026-09-04)**:
1122
+ 89.81% statements/lines, 93.04% functions, and 77.34% branches repo-wide.
1123
+ The Node surface reports 99.61% statements/lines; `document-inspection.js`
1124
+ reports 97.64% statements/lines and 100% functions; `paragraph-text.js` reports
1125
+ 100% statements/lines/functions.
1126
+
1127
+ Preserve snapshots so progress is visible over time rather than presenting
1128
+ coverage as a single pass/fail quality score.
1129
+
1130
+ ## Choosing where a new test belongs
1131
+
1132
+ - A fixed bug always gets an automated regression.
1133
+ - A broad engine rule also gets deterministic fuzz coverage.
1134
+ - A claim about what Word renders or accepts gets a synthetic Word case.
1135
+ - A claim involving package complexity or realistic authoring structures gets
1136
+ a reviewed corpus case.
1137
+ - A structural XML rule should be checked by runtime validation and, where
1138
+ practical, the ECMA-376 XSD lane.
1139
+
1140
+ Keep expectations independent of the code under test. In particular, never use
1141
+ this library's Accept/Reject helpers to calculate the expected result for the
1142
+ Word differential.