@ansonlai/docx-redline-js 0.8.4 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +221 -12
- package/CHANGELOG.md +176 -0
- package/README.md +125 -12
- package/core/current-properties.js +136 -0
- package/core/paragraph-text.js +211 -209
- package/core/types.js +9 -3
- package/core/word-revision-semantics.js +256 -0
- package/dist/docx-redline-js.esm.js +4189 -767
- package/dist/docx-redline-js.esm.js.map +4 -4
- package/dist/docx-redline-js.esm.min.js +111 -108
- package/dist/docx-redline-js.esm.min.js.map +4 -4
- package/dist/docx-redline.bundle.cjs +4292 -870
- package/dist/docx-redline.bundle.cjs.map +4 -4
- package/dist/docx-redline.bundle.js +4292 -870
- package/dist/docx-redline.bundle.js.map +4 -4
- package/docs/AGENT_FAST_START.md +2 -0
- package/docs/AGENT_KNOWLEDGE_BASE.md +19 -3
- package/docs/SKILL_AUTHORING.md +165 -150
- package/docs/TESTING.md +1142 -1059
- package/docs/schemas/document-operations.schema.json +54 -1
- package/document/docx-document.js +1022 -1010
- package/engine/oxml-engine.js +736 -749
- package/engine/paragraph-mark-revisions.js +292 -0
- package/engine/reconstruction-mapper.js +312 -311
- package/engine/reconstruction-writer.js +67 -10
- package/engine/rpr-helpers.js +42 -0
- package/engine/run-builders.js +0 -73
- package/engine/surgical-diff-application.js +604 -605
- package/engine/table-mode.js +63 -63
- package/index.d.ts +47 -1
- package/index.js +11 -2
- package/node/cli-help.js +1 -0
- package/node/cli.js +992 -975
- package/orchestration/list-structural-fallback.js +67 -9
- package/package.json +1 -1
- package/pipeline/diff-engine.js +130 -1
- package/pipeline/ingestion-export.js +227 -243
- package/pipeline/ingestion-paragraph.js +93 -98
- package/pipeline/list-generation.js +415 -419
- package/pipeline/pipeline.js +206 -204
- package/scripts/export-validation-fixtures.mjs +19 -6
- package/scripts/generate-word-revision-semantics-matrix.mjs +134 -0
- package/scripts/lib/word-coverage-catalogue.mjs +7 -2
- package/scripts/render-multilevel-bullet-visual.ps1 +2 -2
- package/scripts/word-com-corpus-visual-suite.ps1 +2 -2
- package/scripts/word-com-differential.ps1 +443 -335
- package/scripts/word-com-visual-suite.ps1 +2 -2
- package/scripts/word-revision-semantics-matrix.ps1 +108 -0
- package/services/batch-operation-orchestrator.js +23 -3
- package/services/capture-engine.js +228 -226
- package/services/document-operation-applier.js +213 -0
- package/services/document-operation-contract.js +75 -0
- package/services/document-operation-mutations.js +3346 -2721
- package/services/error-recovery.js +17 -0
- package/services/list-conversion-diagnostics.js +296 -0
- package/services/list-format-change.js +586 -0
- package/services/list-format-mutation.js +416 -0
- package/services/list-format.js +305 -0
- package/services/numbering-allocation.js +642 -0
- package/services/numbering-helpers.js +354 -524
- package/services/operation-batch-compiler.js +183 -11
- package/services/operation-preflight.js +1 -0
- package/services/receipt-collector.js +288 -288
- package/services/revision-comment-management.js +38 -5
- package/services/revision-fidelity-verifier.js +498 -0
- package/services/standalone-docx-plumbing.js +627 -631
- package/services/standalone-operation-runner.d.ts +182 -5
- package/services/table-reconciliation.js +59 -7
package/docs/TESTING.md
CHANGED
|
@@ -1,1059 +1,1142 @@
|
|
|
1
|
-
# Testing Guide
|
|
2
|
-
|
|
3
|
-
This project uses several test lanes because no single oracle can prove that a
|
|
4
|
-
tracked-change document is correct. JavaScript assertions can verify exact XML
|
|
5
|
-
and return contracts; Microsoft Word can verify what the primary consumer
|
|
6
|
-
actually accepts and rejects; real documents expose structures that small
|
|
7
|
-
fixtures rarely contain.
|
|
8
|
-
|
|
9
|
-
## Test lanes at a glance
|
|
10
|
-
|
|
11
|
-
| Lane | Command | What it proves | What it does not prove |
|
|
12
|
-
|---|---|---|---|
|
|
13
|
-
| Automated regression suite | `npm test` | API behavior, exact text invariants, OOXML structure, deterministic fuzz cases, and catalogue integrity | That desktop Word accepts the generated package |
|
|
14
|
-
| Isolation, types, and lint | `npm run test:isolation`, `npm run check:types`, `npm run lint` | Runtime boundaries, declaration alignment, and static repository rules | Document correctness |
|
|
15
|
-
| JavaScript coverage | `npm run test:coverage` | Which source lines, functions, and branches the automated suite executes | That an executed path is correct |
|
|
16
|
-
| Synthetic Word differential | `npm run test:word` | Word opens generated packages, sees revisions, and produces the intended text after Accept All and Reject All | The diversity of real-world packages |
|
|
17
|
-
| Word visual evidence | `npm run test:word:visual` | Word renders layout-sensitive fixtures in All Markup, Accept All, and Reject All views and writes a pending review manifest | That the rendered pages are visually correct until a reviewer inspects them |
|
|
18
|
-
| SuperDoc Word corpus | `npm run test:corpus:word` | The same Word differential on 60 scenarios drawn from 23 reviewed, pinned real English legal/administrative documents while untouched package parts remain byte-identical | Every possible DOCX producer or document type |
|
|
19
|
-
| SuperDoc visual evidence | `npm run test:corpus:word:visual` | Word renders 40 focused list, table, long-document, and page-header scenarios in All Markup, Accept All, and Reject All views | Human visual sign-off |
|
|
20
|
-
| Visual evidence inspection | `npm run test:visual:inspect` | Automated inspection of rendered Word PDFs across synthetic and SuperDoc suites (page counts, PDF integrity, anomaly detection) | Human visual sign-off |
|
|
21
|
-
| Visual failure regressions | `node tests/visual_failure_regression_tests.mjs` | Semantic OOXML guards against visual failures (formatting leaks, font resets, ghost bullets, table cell destruction) | Visual rendering proof in Word |
|
|
22
|
-
| Multimodal LLM visual spot check | On-demand / sampled | Evaluates rendered real-document pages with vision models for layout, table alignment, and typography regressions | Full-corpus automated coverage (intentionally decoupled and sampled due to cost/time) |
|
|
23
|
-
| XSD and LibreOffice | See [Release validation and independent oracles](#release-validation-and-independent-oracles) | Schema conformance and acceptance by a second consumer | Word-specific revision semantics |
|
|
24
|
-
|
|
|
25
|
-
|
|
|
26
|
-
|
|
|
27
|
-
|
|
|
28
|
-
| Agent
|
|
29
|
-
| Agent
|
|
30
|
-
|
|
|
31
|
-
|
|
|
32
|
-
|
|
|
33
|
-
|
|
|
34
|
-
|
|
|
35
|
-
|
|
|
36
|
-
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
```
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
and
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
-
|
|
249
|
-
-
|
|
250
|
-
-
|
|
251
|
-
|
|
252
|
-
-
|
|
253
|
-
-
|
|
254
|
-
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
- The
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
`
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
npm
|
|
344
|
-
npm run
|
|
345
|
-
npm run
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
- the
|
|
407
|
-
|
|
408
|
-
-
|
|
409
|
-
-
|
|
410
|
-
- the
|
|
411
|
-
|
|
412
|
-
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
`
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
```
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
`
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
and
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
relationshipId: '
|
|
547
|
-
|
|
548
|
-
}]
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
-
|
|
643
|
-
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
-
|
|
700
|
-
-
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
-
|
|
709
|
-
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
717
|
-
|
|
718
|
-
|
|
719
|
-
|
|
720
|
-
|
|
721
|
-
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
-
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
763
|
-
|
|
764
|
-
|
|
765
|
-
|
|
766
|
-
|
|
767
|
-
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
-
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
|
|
778
|
-
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
|
|
782
|
-
|
|
783
|
-
|
|
784
|
-
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
797
|
-
|
|
798
|
-
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
|
|
802
|
-
|
|
803
|
-
|
|
804
|
-
|
|
805
|
-
|
|
806
|
-
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
811
|
-
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
-
|
|
830
|
-
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
|
|
842
|
-
|
|
843
|
-
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
|
|
863
|
-
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
869
|
-
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
873
|
-
|
|
874
|
-
|
|
875
|
-
|
|
876
|
-
|
|
877
|
-
|
|
878
|
-
-
|
|
879
|
-
|
|
880
|
-
-
|
|
881
|
-
|
|
882
|
-
|
|
883
|
-
|
|
884
|
-
|
|
885
|
-
|
|
886
|
-
|
|
887
|
-
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
|
|
901
|
-
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
909
|
-
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
|
|
915
|
-
|
|
916
|
-
|
|
917
|
-
|
|
918
|
-
|
|
919
|
-
|
|
920
|
-
|
|
921
|
-
|
|
922
|
-
|
|
923
|
-
|
|
924
|
-
|
|
925
|
-
|
|
926
|
-
|
|
927
|
-
|
|
928
|
-
|
|
929
|
-
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
|
|
934
|
-
the
|
|
935
|
-
|
|
936
|
-
|
|
937
|
-
|
|
938
|
-
|
|
939
|
-
|
|
940
|
-
npm run test:
|
|
941
|
-
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
|
|
951
|
-
|
|
952
|
-
|
|
953
|
-
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
960
|
-
###
|
|
961
|
-
|
|
962
|
-
```bash
|
|
963
|
-
|
|
964
|
-
|
|
965
|
-
|
|
966
|
-
|
|
967
|
-
|
|
968
|
-
|
|
969
|
-
|
|
970
|
-
-
|
|
971
|
-
|
|
972
|
-
|
|
973
|
-
|
|
974
|
-
|
|
975
|
-
|
|
976
|
-
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
|
|
981
|
-
|
|
982
|
-
|
|
983
|
-
|
|
984
|
-
`
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
991
|
-
|
|
992
|
-
|
|
993
|
-
|
|
994
|
-
|
|
995
|
-
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
|
|
999
|
-
|
|
1000
|
-
|
|
1001
|
-
|
|
1002
|
-
|
|
1003
|
-
|
|
1004
|
-
|
|
1005
|
-
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
|
|
1016
|
-
|
|
1017
|
-
|
|
1018
|
-
|
|
1019
|
-
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
|
|
1023
|
-
|
|
1024
|
-
|
|
1025
|
-
|
|
1026
|
-
|
|
1027
|
-
|
|
1028
|
-
|
|
1029
|
-
|
|
1030
|
-
|
|
1031
|
-
|
|
1032
|
-
|
|
1033
|
-
|
|
1034
|
-
|
|
1035
|
-
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1046
|
-
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
-
|
|
1052
|
-
-
|
|
1053
|
-
|
|
1054
|
-
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1
|
+
# Testing Guide
|
|
2
|
+
|
|
3
|
+
This project uses several test lanes because no single oracle can prove that a
|
|
4
|
+
tracked-change document is correct. JavaScript assertions can verify exact XML
|
|
5
|
+
and return contracts; Microsoft Word can verify what the primary consumer
|
|
6
|
+
actually accepts and rejects; real documents expose structures that small
|
|
7
|
+
fixtures rarely contain.
|
|
8
|
+
|
|
9
|
+
## Test lanes at a glance
|
|
10
|
+
|
|
11
|
+
| Lane | Command | What it proves | What it does not prove |
|
|
12
|
+
|---|---|---|---|
|
|
13
|
+
| Automated regression suite | `npm test` | API behavior, exact text invariants, OOXML structure, deterministic fuzz cases, and catalogue integrity | That desktop Word accepts the generated package |
|
|
14
|
+
| Isolation, types, and lint | `npm run test:isolation`, `npm run check:types`, `npm run lint` | Runtime boundaries, declaration alignment, and static repository rules | Document correctness |
|
|
15
|
+
| JavaScript coverage | `npm run test:coverage` | Which source lines, functions, and branches the automated suite executes | That an executed path is correct |
|
|
16
|
+
| Synthetic Word differential | `npm run test:word` | Word opens generated packages, sees revisions, and produces the intended text after Accept All and Reject All | The diversity of real-world packages |
|
|
17
|
+
| Word visual evidence | `npm run test:word:visual` | Word renders layout-sensitive fixtures in All Markup, Accept All, and Reject All views and writes a pending review manifest | That the rendered pages are visually correct until a reviewer inspects them |
|
|
18
|
+
| SuperDoc Word corpus | `npm run test:corpus:word` | The same Word differential on 60 scenarios drawn from 23 reviewed, pinned real English legal/administrative documents while untouched package parts remain byte-identical | Every possible DOCX producer or document type |
|
|
19
|
+
| SuperDoc visual evidence | `npm run test:corpus:word:visual` | Word renders 40 focused list, table, long-document, and page-header scenarios in All Markup, Accept All, and Reject All views | Human visual sign-off |
|
|
20
|
+
| Visual evidence inspection | `npm run test:visual:inspect` | Automated inspection of rendered Word PDFs across synthetic and SuperDoc suites (page counts, PDF integrity, anomaly detection) | Human visual sign-off |
|
|
21
|
+
| Visual failure regressions | `node tests/visual_failure_regression_tests.mjs` | Semantic OOXML guards against visual failures (formatting leaks, font resets, ghost bullets, table cell destruction) | Visual rendering proof in Word |
|
|
22
|
+
| Multimodal LLM visual spot check | On-demand / sampled | Evaluates rendered real-document pages with vision models for layout, table alignment, and typography regressions | Full-corpus automated coverage (intentionally decoupled and sampled due to cost/time) |
|
|
23
|
+
| XSD and LibreOffice | See [Release validation and independent oracles](#release-validation-and-independent-oracles) | Schema conformance and acceptance by a second consumer | Word-specific revision semantics (see the Word revision semantics row) |
|
|
24
|
+
| Word revision semantics | `node tests/word_revision_semantics_tests.mjs` | The Reject All / Accept All projection in `core/word-revision-semantics.js` and the library's reject/accept agree with recorded desktop Word 16 results for each paragraph-mark revision shape (inserted/deleted marks mid-document, last paragraph, table cells, rows, lists, `w:pPrChange`, mixed authors); no Word needed to run | That Word still behaves as recorded (regenerate the expectations) |
|
|
25
|
+
| Revision fidelity verifier | `node tests/revision_fidelity_verifier_tests.mjs` | Post-apply Reject All invariants (`REJECT_*`), `ACCEPT_NEIGHBORS` locality and `UNTRACKED_CREATED_REVISION` for untracked edits, accept-first scoping, modes, baseline cache, and the `REVISION_FIDELITY_VIOLATION` payload through the bare runner and `openDocx` | Desktop Word behavior beyond the recorded semantics |
|
|
26
|
+
| Revision fidelity view cache | `node tests/revision_fidelity_view_cache_tests.mjs` | Session reuse of verified projections as the next baseline: an untracked edit between tracked ones, a refused operation and an unverified commit verify exactly like uncached single operations | Performance numbers (see `scripts/benchmark-operation-session.mjs`) |
|
|
27
|
+
| Paragraph-mark and property layering | `node tests/paragraph_mark_single_site_tests.mjs`, `node tests/current_properties_usage_tests.mjs` | Tracked paragraph-mark `w:ins`/`w:del` is constructed only in `engine/paragraph-mark-revisions.js`; current properties are read only through `core/current-properties.js` (allowlists carry written reasons) | Behavior of the allowed sites |
|
|
28
|
+
| Agent inspection and package facade | `node tests/document_inspection_tests.mjs`, `node tests/docx_package_facade_tests.mjs` | Canonical text, comment/list resolution, package-scoped IDs, untouched-part preservation, and atomic rollback | Desktop Word rendering |
|
|
29
|
+
| Agent CLI | `node tests/agent_cli_tests.mjs`, `node tests/agent_cli_protocol_tests.mjs`, `node tests/agent_cli_help_tests.mjs`, `node tests/agent_cli_inspection_context_tests.mjs`, `node tests/agent_cli_output_budget_tests.mjs` | JSON contracts, command-specific help, contextual pagination, composable progressive/atomic profiles, deduplicated receipts, compact stdout, file/stdin Unicode fidelity, complete-success exits, source safety, command families, and schema readability | Cross-platform CI beyond the current runner |
|
|
30
|
+
| Example agent session | `node tests/agent_session_example_tests.mjs`, `node tests/localized_edit_example_tests.mjs` | A non-package sample wrapper delegates to the Node facade, binds and refreshes opaque handles across package revisions, returns context windows, compiles exact localized edits, preserves accepted/rejected lifecycle and atomic rollback, and supports review resolution | LLM/provider latency or a production tool-server transport |
|
|
31
|
+
| Batch compilation and recovery | `node tests/batch_source_binding_tests.mjs`, `node tests/error_recovery_contract_tests.mjs`, `node tests/capture_dependency_graph_tests.mjs` | Batch-start source identity survives structural index drift and savepoint restoration; true overlap and capture fan-out fail before mutation; recovery envelopes, retry bases, bounded CLI diagnostics, and complete-success exits remain stable | Whether a model follows the returned recovery action correctly |
|
|
32
|
+
| Agent documentation contract | `node tests/agent_documentation_contract_tests.mjs` | The ordinary fast start stays within its 40–60 line/600-word budget, examples lead with scoped contextual extraction, and packaged skill-authoring guidance separates invariants, policies, transports, recovery, and presentation while the development wrapper stays outside package files | Whether a particular model reads or follows the instructions |
|
|
33
|
+
| Agent edge cases | `node tests/canonical_paragraph_text_tests.mjs`, `node tests/document_inspection_edge_tests.mjs`, `node tests/docx_package_transaction_edge_tests.mjs`, `node tests/node_zip_archive_tests.mjs`, `node tests/agent_cli_edge_tests.mjs` | Revision-view semantics, cross-paragraph anchors, nested numbering, transaction reuse, multi-author cleanup, malformed ZIP handling, and destructive CLI safeguards | Desktop Word rendering and non-Windows CI |
|
|
34
|
+
| Performance boundary regression | `node tests/performance_phase2_boundary_tests.mjs` | Stable facade re-exports, leaf imports, session rollback, isolated context commit, and comment-first scheduling | The Phase 1 one-parse/one-serialize performance target |
|
|
35
|
+
| Live-session accuracy and instrumentation | `node tests/performance_phase1_session_tests.mjs` | One full parse/serialization, sequential semantic equivalence, exact accepted/rejected text, valid revisions, list/table/comment/highlight preservation, savepoint no-ops, and zero-serialization rollback | Desktop Word rendering |
|
|
36
|
+
| Target hot-path parity | `node tests/performance_phase3_target_hot_paths_tests.mjs` | Cached and uncached resolution metadata parity, duplicate safety, snapshots, session invalidation, and pointer-based revision-ID seeding | Machine-independent latency thresholds |
|
|
37
|
+
| List and text-walker parity | `node tests/performance_phase4_list_and_text_parity_tests.mjs` | One shared marker vocabulary and canonical/specialized visible-text agreement across structural characters and revision views | That all specialized mappings are interchangeable |
|
|
38
|
+
| Universal document facade & ZIP | `node tests/universal_docx_document_tests.mjs`, `node tests/universal_zip_archive_tests.mjs`, `node tests/universal_facade_edge_cases_tests.mjs` | Universal `Uint8Array` binary handling, `fflate` compression/decompression, optimistic concurrency, and complete-DOCX operations without Node built-ins | Desktop Word rendering |
|
|
39
|
+
| Sandbox bundle isolation | `node tests/sandbox_bundle_isolation_tests.mjs` | Zero-dependency standalone ESM/CJS bundles execute in restricted VM sandboxes without `process`, `Buffer`, or `require`, with zero Node built-in imports | Web Worker latency |
|
|
40
|
+
| Cross-author revision slicing | `node tests/cross_author_slicing_fixtures_tests.mjs`, `node tests/cross_author_carrier_splitting_tests.mjs`, `node tests/cross_author_slicing_synthetic_tests.mjs`, `node tests/cross_author_slicing_real_tests.mjs`, `node tests/cross_author_slicing_hyperlink_roundtrip_tests.mjs`, `node tests/cross_author_slicing_insertion_stress_tests.mjs` | Word-native `<w:ins>` splitting, `<w:del>` nesting, multi-author stacking, straddle boundaries, synthetic matrix (SYN-01..12d), strict package differential replay (PKG-01..06), exact hyperlink/NBSP round trips, and 76 insertion stress scenarios | Non-Word consumers beyond OpenXML XSD validation |
|
|
41
|
+
|
|
42
|
+
The package-facade regression opens a real ZIP buffer, adds a comment beside an
|
|
43
|
+
existing high ID, validates OPC wiring, and checks an unrelated binary part is
|
|
44
|
+
unchanged after extraction. Its failure case requires byte-exact rollback. The
|
|
45
|
+
inspection regression covers revisions, structural characters, heading
|
|
46
|
+
context, comment joining, and a numbering start override.
|
|
47
|
+
|
|
48
|
+
The CLI regression executes the recommended extract → preflight → apply →
|
|
49
|
+
validate flow and also covers accept, reject, comment deletion, output collision
|
|
50
|
+
refusal, and source-byte preservation. CLI stdout is captured and parsed as one
|
|
51
|
+
JSON value so diagnostic logging cannot corrupt agent output.
|
|
52
|
+
|
|
53
|
+
The edge suites deliberately use structures omitted from the basic happy-path
|
|
54
|
+
fixtures: `w:moveFrom`/`w:moveTo`, deleted text, structural breaks and hyphens,
|
|
55
|
+
comments crossing paragraph boundaries, orphan definitions, outline headings,
|
|
56
|
+
footnote/endnote references, nested Roman/alphabetic numbering, sequential
|
|
57
|
+
commit-then-rollback calls, selective comment authors, existing high anchor
|
|
58
|
+
IDs, encrypted/unsupported ZIP entries, `--force`, and `--in-place`.
|
|
59
|
+
|
|
60
|
+
The Phase 2 boundary regression is intentionally structural as well as
|
|
61
|
+
behavioral. It prevents the public runner from accumulating implementation
|
|
62
|
+
logic again, verifies that internal operation code does not import the root
|
|
63
|
+
entry point, and checks that failed atomic work can retain the exact source
|
|
64
|
+
string while working maps and sets remain isolated until commit.
|
|
65
|
+
|
|
66
|
+
The checked post-Phase-5 2026-09-04 coverage run reports 89.81%
|
|
67
|
+
statements/lines, 77.34% branches, and 93.04% functions repository-wide; this
|
|
68
|
+
is above every recorded pre-refactor dimension. Targeting performance can be
|
|
69
|
+
measured in Node with `npm run benchmark:targeting` and against the native DOM
|
|
70
|
+
by serving the repository locally and opening
|
|
71
|
+
`scripts/benchmark-targeting-browser.html`; its `paragraphs`, `operations`, and
|
|
72
|
+
`iterations` query parameters reproduce the checked fixture sizes.
|
|
73
|
+
The Node surface reports 99.61% statements/lines; `document-inspection.js`
|
|
74
|
+
reports 97.64% statements/lines and 100% functions; `paragraph-text.js` reports
|
|
75
|
+
100% statements/lines/functions.
|
|
76
|
+
|
|
77
|
+
Run the observational live-session benchmark with:
|
|
78
|
+
|
|
79
|
+
```powershell
|
|
80
|
+
npm run benchmark:session
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
It writes `tmp/benchmarks/operation-session-latest.json`. The checked
|
|
84
|
+
1,000-paragraph/10-operation run reduced full-document parse/serialization from
|
|
85
|
+
10/10 to 1/1 and measured a 1.58x median speedup. Timing and heap figures are
|
|
86
|
+
not correctness gates. Per-operation DOM savepoints are retained because
|
|
87
|
+
redline accuracy, no-op isolation, and rollback fidelity take precedence over
|
|
88
|
+
the aspirational speed target.
|
|
89
|
+
|
|
90
|
+
Run the observational agent-protocol benchmark with:
|
|
91
|
+
|
|
92
|
+
```powershell
|
|
93
|
+
npm run benchmark:agent
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
It writes `tmp/benchmarks/agent-workflow-latest.json` and compares canonical
|
|
97
|
+
stateless Node, legacy file-based CLI, compact stdin/profile CLI, and the
|
|
98
|
+
development-only session example. It records native/in-process CLI time, heap,
|
|
99
|
+
serialized request bytes, protocol calls, instruction words/bytes, exact
|
|
100
|
+
accepted/rejected text, comment preservation, ordering, recovery, and
|
|
101
|
+
cross-author attribution. It does not measure or estimate LLM reasoning, exact
|
|
102
|
+
tokens, tool transport, Claude, or OpenCode wall time; collect those separately
|
|
103
|
+
in the calling harness. The checked WP-07 results are summarized in the
|
|
104
|
+
[agent protocol rollout audit](validation-reports/2026-09-12-agent-protocol-rollout.md).
|
|
105
|
+
|
|
106
|
+
Run the CLI discovery/output benchmark with:
|
|
107
|
+
|
|
108
|
+
```powershell
|
|
109
|
+
npm run benchmark:agent-cli
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
It records command-specific help size, broad inspect/search response size,
|
|
113
|
+
context-window call count, pagination metadata, case-insensitive match parity,
|
|
114
|
+
and native elapsed time in `tmp/benchmarks/agent-cli-discovery-latest.json`.
|
|
115
|
+
The checked WP-00 through WP-02 before/after measurements are in the
|
|
116
|
+
[agent CLI discovery baseline](validation-reports/2026-09-12-agent-cli-discovery-baseline.md).
|
|
117
|
+
WP-03 through WP-05 profile, output-size, workflow, documentation, and package
|
|
118
|
+
results are in the
|
|
119
|
+
[agent CLI efficiency rollout audit](validation-reports/2026-09-13-agent-cli-efficiency-rollout.md).
|
|
120
|
+
|
|
121
|
+
## Cross-author revision slicing test suite
|
|
122
|
+
|
|
123
|
+
The cross-author revision slicing subsystem introduces six complementary test lanes:
|
|
124
|
+
|
|
125
|
+
1. **Native Word Desktop Fixtures (`tests/cross_author_slicing_fixtures_tests.mjs`)**:
|
|
126
|
+
Verifies 36 reference `.docx` and `.xml` files generated via Microsoft Word Desktop 365 COM automation across 6 golden scenarios (`insert-interior`, `delete-interior`, `delete-boundary-start`, `delete-boundary-end`, `delete-straddle-baseline-insertion`, and `multi-author-stacked`) in pending, accepted, and rejected states. Proves Word-native OOXML patterns (carrier `<w:ins>` splitting for insertions, direct `<w:del>` nesting inside `<w:ins>` for deletions) and lifecycle parity under `acceptTrackedChangesInOoxml` and `rejectTrackedChangesInOoxml`.
|
|
127
|
+
2. **Carrier Splitting Unit Suite (`tests/cross_author_carrier_splitting_tests.mjs`)**:
|
|
128
|
+
Tests `splitTrackChangeCarrier` across interior and boundary cuts, multi-run formatting preservation, stable metadata cloning, independent revision ID allocation via `RevisionIdAllocator`, and immutable input handling.
|
|
129
|
+
3. **Synthetic Boundary Matrix (`tests/cross_author_slicing_synthetic_tests.mjs`)**:
|
|
130
|
+
Executes the full SYN-01 through SYN-12d test matrix covering pure interior inserts, pure interior deletes, boundary start/end cuts, full-content annihilation, baseline/insertion straddles, multi-carrier straddles, formatting preservation, paired replacements, 3-author stacked revisions, and selective author accept/reject lifecycle oracles.
|
|
131
|
+
4. **Checked-In Word Package Differential Suite (`tests/cross_author_slicing_real_tests.mjs`)**:
|
|
132
|
+
Reconstructs the pre-mutation document packages, applies `slice-cross-author` via the strict `openDocx` package facade, enforces atomic package validation, and compares engine `AcceptAll` and `RejectAll` output against Word Desktop-generated golden packages (PKG-01 through PKG-06).
|
|
133
|
+
5. **Hyperlink and Exact-Text Regression (`tests/cross_author_slicing_hyperlink_roundtrip_tests.mjs`)**:
|
|
134
|
+
Reproduces the repeated-phrase, hyperlink-boundary, and NBSP-to-space failure through both the low-level engine and document operation runner. It requires exact accepted-view equality, preserves both hyperlink relationship containers, and proves that an unreconstructable slicing mutation returns `PATCH_ROUNDTRIP_MISMATCH` with the original OOXML unchanged.
|
|
135
|
+
6. **Insertion Stress Matrix (`tests/cross_author_slicing_insertion_stress_tests.mjs`)**:
|
|
136
|
+
Runs 76 deterministic scenarios across carrier start/end/interior positions, repeated phrases, multi-run and formatted carriers, spaces/tabs/NBSP, XML-sensitive characters, emoji/combining Unicode, hyperlink interiors and boundaries, bookmarks, comments, nested deletions, adjacent authors, mixed same/foreign-author carriers, multiple structural containers, consecutive reviewer rounds, and the atomic document runner. Every supported case requires exact current-view, Accept-All, Reject-Current, schema, metadata, and non-empty-wrapper invariants; unsupported nested structures must fail without throwing or returning changed output.
|
|
137
|
+
|
|
138
|
+
## Coverage matrix and test selection
|
|
139
|
+
|
|
140
|
+
Run the deterministic task-by-structure report before choosing the next Word
|
|
141
|
+
case:
|
|
142
|
+
|
|
143
|
+
```powershell
|
|
144
|
+
npm run report:word:coverage
|
|
145
|
+
npm run report:word:coverage -- --json
|
|
146
|
+
npm run report:test:dashboard
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
The dashboard command writes a self-contained interactive report to
|
|
150
|
+
`docs/test-comparison-dashboard.html`. It compares task/structure cells,
|
|
151
|
+
synthetic and real-document coverage, independent oracles, visual-render
|
|
152
|
+
eligibility, and planned high-priority gaps from the live catalogues. It also
|
|
153
|
+
embeds source, tracked, accepted, and rejected packages for every synthetic
|
|
154
|
+
fixture and, when the pinned corpus is downloaded, all 60 reviewed real legal
|
|
155
|
+
and administrative scenarios. Its `docx-preview` workbench supports arbitrary left/right states,
|
|
156
|
+
comparison presets, synchronized scrolling, revision metadata, expected text,
|
|
157
|
+
and local downloads; no fixture upload or file picker is required. Real cases
|
|
158
|
+
are labeled in the document selector and selected by default when available.
|
|
159
|
+
Use **Hide sidebar** beside the dashboard filters to remove the coverage-detail
|
|
160
|
+
column and expand the comparison workbench to the full browser width. The
|
|
161
|
+
button becomes **Show sidebar**, and the preference is stored locally so the
|
|
162
|
+
wide document view survives a reload.
|
|
163
|
+
Without the local corpus, generation remains offline-safe and embeds only the
|
|
164
|
+
synthetic previews.
|
|
165
|
+
|
|
166
|
+
The report combines all synthetic and reviewed SuperDoc scenarios. Synthetic
|
|
167
|
+
metadata is declared in `tests/fixtures/word-task-coverage.mjs`; SuperDoc
|
|
168
|
+
metadata is declared in `tests/corpus/superdoc-word-coverage.json`. Descriptive
|
|
169
|
+
`task`, `shape`, and `coverage` labels remain on the original cases as review
|
|
170
|
+
notes, while the matrix uses shared task, structure, and oracle vocabularies.
|
|
171
|
+
|
|
172
|
+
Catalogue tests reject unknown vocabulary labels, missing oracle or manual-review
|
|
173
|
+
metadata, duplicate identities, structural claims unsupported by fixture XML or
|
|
174
|
+
reviewed corpus labels, and uncovered high-priority cells without a recorded
|
|
175
|
+
plan or exclusion. The dispositions live in
|
|
176
|
+
`tests/fixtures/coverage-matrix-priorities.json` and require both a reason and a
|
|
177
|
+
dependency.
|
|
178
|
+
|
|
179
|
+
Choose new cases from uncovered high-priority cells first, then prefer a new
|
|
180
|
+
task/structure combination over another case in a dense cell. A higher count is
|
|
181
|
+
not itself a reason to add a fixture. Adding a catalogue case automatically
|
|
182
|
+
changes the report because it reads the live metadata.
|
|
183
|
+
|
|
184
|
+
## Reliability improvement process used for this test expansion
|
|
185
|
+
|
|
186
|
+
The current corpus, visual runners, and comparison dashboard were built as one
|
|
187
|
+
feedback loop rather than as separate test features. The working sequence is:
|
|
188
|
+
|
|
189
|
+
1. inventory the claimed behavior and existing test lanes;
|
|
190
|
+
2. add realistic cases that expose weak structures;
|
|
191
|
+
3. assert the smallest machine-checkable invariant at the engine level;
|
|
192
|
+
4. package the result into the original real document without changing
|
|
193
|
+
unrelated parts;
|
|
194
|
+
5. ask Word to accept and reject the revisions;
|
|
195
|
+
6. render All Markup, Accept All, and Reject All views;
|
|
196
|
+
7. inspect the actual pages at both contact-sheet and full-page scale; and
|
|
197
|
+
8. turn every visual defect into a focused regression plus a corpus assertion.
|
|
198
|
+
|
|
199
|
+
This ordering matters. A valid `<w:ins>` or `<w:del>` tree does not establish
|
|
200
|
+
that the inserted run inherited the correct font, list indentation, hyperlink
|
|
201
|
+
wrapper, table-cell formatting, or header styling. Conversely, a good-looking
|
|
202
|
+
PDF does not prove that Reject All restores the exact source. Each stage answers
|
|
203
|
+
a different question, and a case is useful only when its intended claim is
|
|
204
|
+
explicit.
|
|
205
|
+
|
|
206
|
+
### Start from coverage claims, not fixture count
|
|
207
|
+
|
|
208
|
+
Read the task-by-structure matrix and select a real document that adds a missing
|
|
209
|
+
or weak combination. The expansion deliberately favored bullets, nested lists,
|
|
210
|
+
table cells, headers, long documents, and atomic multi-change batches because
|
|
211
|
+
these structures cross more OOXML boundaries than a single plain paragraph.
|
|
212
|
+
Several scenarios reuse a pinned source when a new batch exercises a genuinely
|
|
213
|
+
different structure; this increases behavioral coverage without pretending
|
|
214
|
+
that another copy of the same document is a new source.
|
|
215
|
+
|
|
216
|
+
For long real documents, record structural minimums such as word count, table
|
|
217
|
+
count, list-paragraph count, section count, and related header/footer parts.
|
|
218
|
+
`scripts/prepare-superdoc-word-corpus.mjs` checks these declarations against the
|
|
219
|
+
source package. This prevents a scenario description such as “table-heavy” or
|
|
220
|
+
“multi-section” from silently drifting away from the actual fixture.
|
|
221
|
+
|
|
222
|
+
### Use multi-change batches to exercise interaction effects
|
|
223
|
+
|
|
224
|
+
Single replacements remain valuable for isolating engine behavior, but real
|
|
225
|
+
editing sessions usually contain several changes. The larger corpus scenarios
|
|
226
|
+
therefore use `operations` to apply independent edits atomically across one
|
|
227
|
+
document. Targets are chosen from different list items or table cells when
|
|
228
|
+
possible. This catches failures caused by earlier replacements shifting later
|
|
229
|
+
anchors, revision-ID allocation across a batch, and formatting state leaking
|
|
230
|
+
between operations.
|
|
231
|
+
|
|
232
|
+
The batch runner must either produce all requested revisions or roll back the
|
|
233
|
+
document. Word then checks the complete accepted and rejected story, not merely
|
|
234
|
+
the first target. Untouched ZIP parts are hashed and compared with the pinned
|
|
235
|
+
source, so success cannot come from rebuilding or normalizing the rest of the
|
|
236
|
+
package.
|
|
237
|
+
|
|
238
|
+
### Expand weak list and table cells with structural operations
|
|
239
|
+
|
|
240
|
+
The list/table expansion was driven directly by the task-by-structure matrix;
|
|
241
|
+
it did not add more plain replacement cases. The current catalogue contains 125
|
|
242
|
+
Word cases: 65 synthetic fixtures and 60 reviewed real-document scenarios. The
|
|
243
|
+
`List Change × List` cell now contains 27 cases; the
|
|
244
|
+
`Table Reconciliation × Table` cell contains 11 cases.
|
|
245
|
+
|
|
246
|
+
The added synthetic cases isolate one structural decision at a time:
|
|
247
|
+
|
|
248
|
+
- append one list item;
|
|
249
|
+
- append several adjacent items;
|
|
250
|
+
- add a nested child;
|
|
251
|
+
- insert into the middle of an explicit list range;
|
|
252
|
+
- preserve upper- and lower-Roman numbering;
|
|
253
|
+
- preserve upper-letter, parenthesized lower-letter, dash, and symbol bullets;
|
|
254
|
+
- expand a manually numbered heading whose `w:numId="0"` suppresses numbering
|
|
255
|
+
into separate, positively numbered list paragraphs;
|
|
256
|
+
- update one table cell;
|
|
257
|
+
- insert a table row;
|
|
258
|
+
- delete a table row; and
|
|
259
|
+
- update several cells in one reconciliation.
|
|
260
|
+
|
|
261
|
+
The mixed-attachment fixture replaces one anchor with explicit headings,
|
|
262
|
+
ordinary paragraphs, a Markdown table, and a numbered list. It verifies real
|
|
263
|
+
`w:pStyle`, `w:tbl`, and positive-numbering structures plus independent Word
|
|
264
|
+
Accept All and Reject All results; literal table pipes are forbidden.
|
|
265
|
+
|
|
266
|
+
The real-document cases repeat those claims against native numbering and table
|
|
267
|
+
markup from municipal notices, board agendas, a long zoning ordinance, PPG
|
|
268
|
+
minutes, action tables, and invoice tables. They include consecutive and nested
|
|
269
|
+
list insertions, a long-document definition insertion, single- and multi-cell
|
|
270
|
+
updates, row insertion/deletion, and invoice row changes. Reusing a reviewed,
|
|
271
|
+
hash-pinned source is intentional when the scenario exercises a different
|
|
272
|
+
operation; the unit of coverage is the behavioral claim, not the download.
|
|
273
|
+
|
|
274
|
+
These cases declare `operation.type` as `list-change` or
|
|
275
|
+
`table-reconciliation`. Their accepted/rejected expectations use explicit
|
|
276
|
+
contains/absent assertions when a Markdown table payload or a structural list
|
|
277
|
+
edit cannot be compared meaningfully to the raw Word source as one exact text
|
|
278
|
+
string. Structural expectations still verify real properties such as native
|
|
279
|
+
list paragraphs, tables, headers/footers, sections, and minimum document size.
|
|
280
|
+
The catalogue tests require these expectations and reject unknown operation or
|
|
281
|
+
coverage labels.
|
|
282
|
+
|
|
283
|
+
List-style cases do not infer their claim from visible text alone. Synthetic
|
|
284
|
+
fixtures declare the required `w:numFmt` values and use complete, schema-ordered
|
|
285
|
+
numbering parts: every `w:abstractNum` must precede the first concrete `w:num`
|
|
286
|
+
mapping. That ordering is regression-tested because Word otherwise opens the
|
|
287
|
+
package but silently substitutes its fallback bullet. Real scenarios declare a
|
|
288
|
+
`listStyleExpectation` with the source level, number format, and, where useful,
|
|
289
|
+
the exact `w:lvlText`. Corpus preparation resolves the target paragraph's
|
|
290
|
+
`w:numId` through `word/numbering.xml` and fails before editing if the pinned
|
|
291
|
+
source no longer has that native style. The current real cases exercise upper
|
|
292
|
+
and lower Roman, upper and lower letter, parenthesized lower letter, dash, and
|
|
293
|
+
symbol bullets across bylaws, agendas, healthcare minutes, and long council
|
|
294
|
+
minutes.
|
|
295
|
+
|
|
296
|
+
### Diagnose visual failures in the generated OOXML first
|
|
297
|
+
|
|
298
|
+
When a page looks wrong, compare the relevant source and tracked
|
|
299
|
+
`word/document.xml` or related part before changing the renderer. Locate the
|
|
300
|
+
visible text, inspect its nearest `w:r`, `w:rPr`, wrapper, paragraph properties,
|
|
301
|
+
and neighboring runs, then compare those properties with the source. This
|
|
302
|
+
separates an engine defect from a Word, PDF, or `docx-preview` display issue.
|
|
303
|
+
|
|
304
|
+
Three defects illustrate the approach:
|
|
305
|
+
|
|
306
|
+
- The council header date replacement inherited `w:vertAlign="superscript"`
|
|
307
|
+
from the trailing ordinal suffix. The fix anchors replacement formatting at
|
|
308
|
+
the beginning of the deleted range, and the regression explicitly forbids
|
|
309
|
+
`vertAlign` on the inserted runs.
|
|
310
|
+
- The prospectus filing-date replacement crossed normal runs and a hyperlink.
|
|
311
|
+
A forward-only property lookup had already moved past the beginning of the
|
|
312
|
+
deletion when the insertion asked for its formatting, so the new run had no
|
|
313
|
+
`w:rPr`. Word and SuperDoc correctly fell back from the surrounding 10 pt to
|
|
314
|
+
the document default of 12 pt. The lookup now supports that deliberate
|
|
315
|
+
backward query. The focused regression requires `w:sz="20"` and
|
|
316
|
+
`w:szCs="20"`, and the real prospectus scenario requires those values on
|
|
317
|
+
every inserted text run.
|
|
318
|
+
- A wholly inserted list item originally tracked only its text run. Accept All
|
|
319
|
+
looked correct and text-only Reject All assertions passed, but Word retained
|
|
320
|
+
the untracked paragraph mark as an empty bullet or number. Inserted list
|
|
321
|
+
paragraphs now track both their text and paragraph mark. The regression
|
|
322
|
+
rejects the change and asserts the exact original paragraph count, text, and
|
|
323
|
+
numbering, while the real Word PDFs confirm that no ghost marker remains.
|
|
324
|
+
|
|
325
|
+
These assertions test the semantic cause rather than a screenshot pixel. The
|
|
326
|
+
visual render remains necessary to confirm that the corrected properties
|
|
327
|
+
produce the intended page. The automated test suite in
|
|
328
|
+
`tests/visual_failure_regression_tests.mjs` codifies these guards directly in
|
|
329
|
+
the regression test pipeline (`npm test`) to prevent regressions before DOCX
|
|
330
|
+
packages reach Word.
|
|
331
|
+
|
|
332
|
+
### Regenerate packages; do not reason from stale outputs
|
|
333
|
+
|
|
334
|
+
Source downloads are content-addressed and may be reused. Generated tracked,
|
|
335
|
+
accepted, rejected, PDF, and dashboard artifacts are outputs and must be
|
|
336
|
+
rebuilt after an engine or scenario change. A `Verified cached ...` line from
|
|
337
|
+
the corpus fetcher means only that the pinned source `.docx` already matches its
|
|
338
|
+
recorded hash; it does not mean the edited package was reused.
|
|
339
|
+
|
|
340
|
+
Use this sequence for a corpus-affecting engine change:
|
|
341
|
+
|
|
342
|
+
```powershell
|
|
343
|
+
npm test
|
|
344
|
+
npm run check:types
|
|
345
|
+
npm run lint
|
|
346
|
+
npm run test:isolation
|
|
347
|
+
npm run test:corpus:word
|
|
348
|
+
npm run test:corpus:word:visual
|
|
349
|
+
npm run report:test:dashboard
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
For faster iteration, `prepare-superdoc-word-corpus.mjs` accepts
|
|
353
|
+
`--input-dir` and `--output-dir`, and the visual PowerShell runner accepts
|
|
354
|
+
`-Case <scenario-key>` plus custom fixture/output directories. Run the complete
|
|
355
|
+
lane before handoff even when a focused render passed.
|
|
356
|
+
|
|
357
|
+
### Treat Word COM cleanup separately from render success
|
|
358
|
+
|
|
359
|
+
Word sometimes disconnects a COM document proxy after successfully exporting
|
|
360
|
+
a PDF and raises `RPC_E_DISCONNECTED` during `Document.Close()`. The visual
|
|
361
|
+
runner uses safe close/quit helpers: a disconnect during post-export cleanup is
|
|
362
|
+
reported but does not discard a valid, non-empty PDF; a disconnect during the
|
|
363
|
+
actual render causes Word to restart and the case to retry once. Other COM
|
|
364
|
+
exceptions still fail the run. This distinction avoids both false failures and
|
|
365
|
+
false passes.
|
|
366
|
+
|
|
367
|
+
Each visual invocation uses a process-specific fixture directory. That keeps a
|
|
368
|
+
Word process left behind by an earlier disconnect from locking or contaminating
|
|
369
|
+
the next set of generated DOCX files.
|
|
370
|
+
|
|
371
|
+
### Review both the whole document and the changed page
|
|
372
|
+
|
|
373
|
+
For a long document, create low-resolution contact sheets for every page in all
|
|
374
|
+
three revision states and inspect them for pagination changes, blank pages,
|
|
375
|
+
clipping, table displacement, or large typography shifts. Then inspect every
|
|
376
|
+
changed page at full resolution. Contact sheets are good at document-wide
|
|
377
|
+
layout; they are not reliable for a 10 pt versus 12 pt difference or a leaked
|
|
378
|
+
superscript flag.
|
|
379
|
+
|
|
380
|
+
Record the exact scenario, Word version/build, revision view, page, observed
|
|
381
|
+
fact, and whether the judgment is certain. The visual manifest remains pending
|
|
382
|
+
until a reviewer makes that judgment; successful PDF export alone is not a
|
|
383
|
+
visual pass.
|
|
384
|
+
|
|
385
|
+
### Keep the dashboard generated and test its controls
|
|
386
|
+
|
|
387
|
+
`scripts/generate-test-dashboard.mjs` owns the dashboard markup, styles, and
|
|
388
|
+
behavior. Do not hand-edit `docs/test-comparison-dashboard.html`; rebuild it
|
|
389
|
+
with `npm run report:test:dashboard`. The report embeds the four DOCX states so
|
|
390
|
+
it remains self-contained and can show the exact generated packages without a
|
|
391
|
+
file picker or server.
|
|
392
|
+
|
|
393
|
+
Dashboard changes require generator-level assertions in
|
|
394
|
+
`tests/test_dashboard_report_tests.mjs`. The sidebar-width control, for
|
|
395
|
+
example, is checked for its accessible relationship, full-width CSS state, and
|
|
396
|
+
persisted preference. DOCX package parsing remains covered separately by
|
|
397
|
+
`tests/docxjs_dashboard_rendering_tests.mjs`. When browser control is available,
|
|
398
|
+
also click the control in the generated page and verify that the sidebar is
|
|
399
|
+
hidden, both document panes widen, the label changes, and reloading preserves
|
|
400
|
+
the selected state.
|
|
401
|
+
|
|
402
|
+
### Definition of done for a discovered real-document defect
|
|
403
|
+
|
|
404
|
+
A visual defect is complete only when all of the following are true:
|
|
405
|
+
|
|
406
|
+
- the source and faulty generated OOXML have been compared;
|
|
407
|
+
- the engine cause is understood rather than masked in CSS or the renderer;
|
|
408
|
+
- a minimal automated regression reproduces the structural boundary;
|
|
409
|
+
- Accept All yields the requested text and Reject All restores the source;
|
|
410
|
+
- the affected real corpus scenario has an appropriate structural or
|
|
411
|
+
formatting assertion;
|
|
412
|
+
- its source, tracked, accepted, and rejected packages have been regenerated;
|
|
413
|
+
- desktop Word differential passes;
|
|
414
|
+
- the changed page and the document-wide three-view render have been inspected;
|
|
415
|
+
- the self-contained comparison dashboard has been rebuilt; and
|
|
416
|
+
- the full automated, isolation, type, lint, and whitespace checks pass.
|
|
417
|
+
|
|
418
|
+
Prepare, but do not approve, a human review sample with:
|
|
419
|
+
|
|
420
|
+
```powershell
|
|
421
|
+
npm run review:word:prepare -- --cycle=0
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
The ignored `tmp/word-manual-review/review-manifest-cycle-0.json` selects all
|
|
425
|
+
catalogue families changed in the worktree, a rotating 20% synthetic sample,
|
|
426
|
+
and one legal plus one administrative SuperDoc case. Every All Markup, Accept
|
|
427
|
+
All, Reject All, and human sign-off field starts as `pending`; the helper cannot
|
|
428
|
+
turn them into passes. Increment `--cycle` between releases to rotate the
|
|
429
|
+
unchanged sample.
|
|
430
|
+
|
|
431
|
+
## Automated JavaScript tests
|
|
432
|
+
|
|
433
|
+
Files matching `tests/*.mjs` are discovered by `scripts/run-tests.mjs`. Tests
|
|
434
|
+
use `assert/strict` and each runs in a separate Node process. The runner uses up
|
|
435
|
+
to four workers by default while preserving filename-sorted reporting. Set
|
|
436
|
+
`DOCX_TEST_CONCURRENCY=1` for deterministic serial reproduction, or another
|
|
437
|
+
positive integer for an explicit worker bound. Word, visual, and corpus lanes
|
|
438
|
+
remain separate serial commands. Shared OOXML assertions belong in
|
|
439
|
+
`tests/helpers/ooxml-assertions.mjs`; XML-provider setup belongs in
|
|
440
|
+
`tests/setup-xml-provider.mjs`.
|
|
441
|
+
|
|
442
|
+
Measure serial and parallel execution on the same machine with:
|
|
443
|
+
|
|
444
|
+
```powershell
|
|
445
|
+
npm run benchmark:tests
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
The benchmark performs paired warmups/runs and writes its JSON result to
|
|
449
|
+
`tmp/benchmarks/test-runner-latest.json`. The checked Node 24/Windows x64 run
|
|
450
|
+
reduced median time from 18.02 seconds to 7.96 seconds (55.80%). Twenty
|
|
451
|
+
consecutive four-worker runs completed without a failure.
|
|
452
|
+
|
|
453
|
+
For function-level gap work, run:
|
|
454
|
+
|
|
455
|
+
```powershell
|
|
456
|
+
npm run test:coverage
|
|
457
|
+
npm run coverage:gaps
|
|
458
|
+
npm run coverage:gaps -- --json
|
|
459
|
+
```
|
|
460
|
+
|
|
461
|
+
The coverage command emits both the text summary and detailed Istanbul JSON.
|
|
462
|
+
The gap report includes only runtime production roots, lists each uncovered
|
|
463
|
+
function with its file and declaration line, assigns the Phase 3 P0/P1/P2
|
|
464
|
+
priority, and fails if a targeted file drops below the checked covered-function
|
|
465
|
+
or covered-branch baseline in `tests/coverage-data/phase3-baseline.json`. Because V8
|
|
466
|
+
discovers new branch sites when a formerly cold function first executes, review
|
|
467
|
+
covered counts and behavior assertions alongside percentages. Do not add ignore
|
|
468
|
+
annotations or call private code merely to improve a score; classify a retained
|
|
469
|
+
gap with a reachability or environment reason.
|
|
470
|
+
|
|
471
|
+
For an engine regression:
|
|
472
|
+
|
|
473
|
+
1. Add the smallest fixed case that reproduces the bug and asserts the exact
|
|
474
|
+
result or structured error.
|
|
475
|
+
2. Assert both sides of tracked changes: accepting must produce the requested
|
|
476
|
+
text and rejecting must restore the original text exactly.
|
|
477
|
+
3. Assert structural invariants with `validateRedlineOoxml` and the shared OOXML
|
|
478
|
+
helpers. Do not use a whitespace-collapsing reader as the oracle.
|
|
479
|
+
4. If engine behavior changed, add the same shape to the deterministic fuzz
|
|
480
|
+
corpus so nearby inputs are exercised too.
|
|
481
|
+
5. Run `npm test`, isolation, types, and lint.
|
|
482
|
+
|
|
483
|
+
For a mixed attachment regression, also run the content through
|
|
484
|
+
`planStructuredReplacement(...)`. Assert the declared block sequence, reject
|
|
485
|
+
malformed table grammar, and verify that the result contains real `w:tbl`,
|
|
486
|
+
heading styles, positive list numbering IDs, valid non-nested revisions, and
|
|
487
|
+
independent Accept/Reject outcomes. Do not treat absence of literal pipe text as
|
|
488
|
+
sufficient proof of a table.
|
|
489
|
+
|
|
490
|
+
## Synthetic Microsoft Word tests
|
|
491
|
+
|
|
492
|
+
Run only one Word job at a time. `scripts/word-com-suite.ps1` and
|
|
493
|
+
`scripts/word-com-corpus-suite.ps1` begin by force-killing every windowless
|
|
494
|
+
`WINWORD` process to clear orphans from aborted runs, which also kills the
|
|
495
|
+
invisible Word instance of any other COM job running concurrently.
|
|
496
|
+
|
|
497
|
+
The runtime library contains no COM or Word dependency. Word automation exists
|
|
498
|
+
only in Windows development scripts and is intentionally excluded from package
|
|
499
|
+
runtime paths.
|
|
500
|
+
|
|
501
|
+
The synthetic catalogue is `tests/fixtures/word-task-cases.mjs`. During
|
|
502
|
+
`npm run test:word`:
|
|
503
|
+
|
|
504
|
+
1. `scripts/export-validation-fixtures.mjs` applies each operation, runs the
|
|
505
|
+
library's structural validator, checks any case-specific `requiredElements`,
|
|
506
|
+
and builds a minimal `.docx` under ignored `tmp/word-validation/` storage.
|
|
507
|
+
2. Expected Accept All and Reject All text is derived from the test's edit
|
|
508
|
+
intent, not from the library's own revision-management functions.
|
|
509
|
+
3. `scripts/word-com-differential.ps1` opens each fixture through desktop Word
|
|
510
|
+
without a repair dialog and confirms Word sees at least one revision.
|
|
511
|
+
4. Word accepts every revision in one fresh document and rejects every revision
|
|
512
|
+
in another. Both resulting texts are compared with the intent-derived
|
|
513
|
+
expectations.
|
|
514
|
+
|
|
515
|
+
To add a synthetic Word case:
|
|
516
|
+
|
|
517
|
+
1. Add a unique lowercase-hyphenated entry to `WORD_TASK_CASES` with
|
|
518
|
+
`category: 'legal'` or `'administrative'`, a distinct `task`, and `original`
|
|
519
|
+
and `modified` text.
|
|
520
|
+
2. Use `sourceDocumentXml` when the case needs bookmarks, hyperlinks, tables,
|
|
521
|
+
content controls, prior revisions, or another deliberate structure.
|
|
522
|
+
3. Add `requiredElements`, such as `{ bookmarkStart: 1, bookmarkEnd: 1 }`, when
|
|
523
|
+
retaining a structure is part of the claim. These are minimum namespace-aware
|
|
524
|
+
element counts checked before Word runs.
|
|
525
|
+
4. Supply `expectedAcceptedText` and `expectedRejectedText` when the source is a
|
|
526
|
+
multi-paragraph document or Word exposes structural separators such as table
|
|
527
|
+
row boundaries.
|
|
528
|
+
5. Run `node tests/word_task_catalog_tests.mjs`, then `npm run test:word` on a
|
|
529
|
+
Windows machine with desktop Word installed.
|
|
530
|
+
|
|
531
|
+
### Related-part fixture schema
|
|
532
|
+
|
|
533
|
+
The script-only packager supports optional numbering plus comments, footnotes,
|
|
534
|
+
endnotes, headers, footers, and external hyperlinks. Add them to a catalogue
|
|
535
|
+
case through `packageParts`:
|
|
536
|
+
|
|
537
|
+
```js
|
|
538
|
+
{
|
|
539
|
+
sourceDocumentXml,
|
|
540
|
+
packageParts: {
|
|
541
|
+
commentsXml,
|
|
542
|
+
footnotesXml,
|
|
543
|
+
endnotesXml,
|
|
544
|
+
headers: [{
|
|
545
|
+
partName: 'header1.xml',
|
|
546
|
+
relationshipId: 'rIdHeader1',
|
|
547
|
+
xml: headerXml
|
|
548
|
+
}],
|
|
549
|
+
footers: [{
|
|
550
|
+
partName: 'footer1.xml',
|
|
551
|
+
relationshipId: 'rIdFooter1',
|
|
552
|
+
xml: footerXml
|
|
553
|
+
}],
|
|
554
|
+
externalHyperlinks: [{
|
|
555
|
+
relationshipId: 'rIdPolicy',
|
|
556
|
+
target: 'https://example.com/policy'
|
|
557
|
+
}]
|
|
558
|
+
}
|
|
559
|
+
}
|
|
560
|
+
```
|
|
561
|
+
|
|
562
|
+
`createCommentsPart`, `createNotesPart`, and `createHeaderFooterPart` in
|
|
563
|
+
`tests/fixtures/word-package-parts.mjs` provide small escaped constructors for
|
|
564
|
+
the common XML parts. Header/footer `partName` and `relationshipId` values have
|
|
565
|
+
deterministic defaults, but explicit values make the corresponding
|
|
566
|
+
`w:headerReference`, `w:footerReference`, or `w:hyperlink` easier to audit.
|
|
567
|
+
|
|
568
|
+
Before ZIP emission, the packager rejects malformed XML, duplicate relationship
|
|
569
|
+
IDs or part names, missing relationship targets, undefined comment/note IDs,
|
|
570
|
+
and note parts without separator IDs `-1` and `0`. It generates the required
|
|
571
|
+
content-type overrides and document relationships. Every supplied related part
|
|
572
|
+
is compared byte-for-byte with the packaged entry, recorded as SHA-256 in the
|
|
573
|
+
case sidecar, and rechecked by the Word differential before opening the DOCX.
|
|
574
|
+
These helpers remain under `scripts/` and `tests/`; none are shipped through the
|
|
575
|
+
runtime entry point.
|
|
576
|
+
|
|
577
|
+
List cases may declare `expectedAcceptedListStrings` / `expectedRejectedListStrings` (per-paragraph
|
|
578
|
+
`ListFormat.ListString`) and `expectedAcceptedListGroupSizes` (per-paragraph
|
|
579
|
+
`ListFormat.List.ListParagraphs.Count`, 0 for a non-list paragraph). The group sizes tell one list from two lists
|
|
580
|
+
that merely read `A.` and `B.` through start values (R2).
|
|
581
|
+
|
|
582
|
+
### What the automated Word differential proves
|
|
583
|
+
|
|
584
|
+
The Word process is real desktop Microsoft Word, but it is driven invisibly
|
|
585
|
+
through COM. For each fixture, the script:
|
|
586
|
+
|
|
587
|
+
1. starts `Word.Application` with alerts and the window disabled;
|
|
588
|
+
2. opens the generated package with `OpenNoRepairDialog`;
|
|
589
|
+
3. fails if Word cannot open it or sees zero tracked revisions;
|
|
590
|
+
4. accepts all revisions and reads `Document.Content.Text`;
|
|
591
|
+
5. closes without saving, reopens the untouched fixture, rejects all revisions,
|
|
592
|
+
and reads the text again; and
|
|
593
|
+
6. compares both results with expectations that were not calculated by this
|
|
594
|
+
library's own accept/reject implementation.
|
|
595
|
+
|
|
596
|
+
Where the case has a source package the script also compares the paragraph count after
|
|
597
|
+
Reject All with the source package's paragraph count and fails on a mismatch, so
|
|
598
|
+
a leftover or extra paragraph mark is caught even when the text matches.
|
|
599
|
+
|
|
600
|
+
Accept All and Reject All are issued through `Document.Revisions.AcceptAll()` /
|
|
601
|
+
`RejectAll()` (per-range `Revisions` for header stories). Do not use
|
|
602
|
+
`Document.AcceptAllRevisions()` / `RejectAllRevisions()` in COM scripts: when
|
|
603
|
+
the last body paragraph has a tracked-inserted mark they can leave that mark and
|
|
604
|
+
an extra empty paragraph behind, unlike the ribbon command and Word users.
|
|
605
|
+
|
|
606
|
+
Exact comparison is the default. The harness removes Word's terminal paragraph
|
|
607
|
+
mark, normalizes CR/LF representation, and removes the characters Word exposes
|
|
608
|
+
as table-cell and footnote/endnote reference boundaries; case-specific normalized comparison requires an
|
|
609
|
+
explicit reason. Documents are never saved by the automated differential.
|
|
610
|
+
|
|
611
|
+
For synthetic fixtures, expectations come directly from `original`, `modified`,
|
|
612
|
+
and any explicit full-document expectations in the case. For SuperDoc fixtures,
|
|
613
|
+
Word first reads the declared story from the original pinned source document;
|
|
614
|
+
every target in a single or multi-change scenario must occur exactly once, and
|
|
615
|
+
the accepted expectation is formed by applying those replacements to Word's own
|
|
616
|
+
source text. Header scenarios use Word's header stories rather than body text.
|
|
617
|
+
The corpus packager hashes every package part other than the one intentionally
|
|
618
|
+
replaced by the scenario.
|
|
619
|
+
|
|
620
|
+
This proves that Word can consume the package, recognizes the revision markup,
|
|
621
|
+
and resolves Accept All and Reject All to the intended text. It does **not**
|
|
622
|
+
prove that the document looks right on the page. `Content.Text` cannot detect
|
|
623
|
+
bad pagination, awkward revision balloons, shifted table widths, broken tab
|
|
624
|
+
alignment, font substitution, changed list indentation, clipped headers,
|
|
625
|
+
visually stale fields, or a comment/footnote marker that is technically present
|
|
626
|
+
but poorly placed.
|
|
627
|
+
|
|
628
|
+
### Microsoft Word visual review guide
|
|
629
|
+
|
|
630
|
+
Human visual review complements—not replaces—the automated differential. The
|
|
631
|
+
automated differential proves Word revision semantics by comparing text after
|
|
632
|
+
Accept All and Reject All, but this review checks the layout, typography, and
|
|
633
|
+
interaction details that `Document.Content.Text` cannot see.
|
|
634
|
+
|
|
635
|
+
Review is required for:
|
|
636
|
+
|
|
637
|
+
- every new or materially changed synthetic Word case;
|
|
638
|
+
- the first case for a new structure, operation type, or related package part;
|
|
639
|
+
- any engine change affecting reconstruction, formatting, lists, tables,
|
|
640
|
+
fields, tabs/breaks, comments, notes, headers/footers, or revision metadata;
|
|
641
|
+
- any case that needs normalized rather than exact text comparison; and
|
|
642
|
+
- any automated Word failure whose cause is not immediately textual.
|
|
643
|
+
|
|
644
|
+
#### Prepare the review set
|
|
645
|
+
|
|
646
|
+
Run the automated lanes first:
|
|
647
|
+
|
|
648
|
+
```powershell
|
|
649
|
+
npm run test:word
|
|
650
|
+
npm run test:corpus:word
|
|
651
|
+
npm run report:word:coverage
|
|
652
|
+
npm run review:word:prepare -- --cycle=0
|
|
653
|
+
```
|
|
654
|
+
|
|
655
|
+
The last command writes a pending review manifest under ignored
|
|
656
|
+
`tmp/word-manual-review/` containing changed catalogue families, the rotating
|
|
657
|
+
20% synthetic release sample, and legal/administrative corpus representatives.
|
|
658
|
+
It only prepares the selection: it never records a visual pass or human
|
|
659
|
+
sign-off. Use a new cycle number for each release rotation.
|
|
660
|
+
|
|
661
|
+
Synthetic documents are generated under `tmp/word-validation/`; reviewed
|
|
662
|
+
SuperDoc results are under `tmp/superdoc-word-fixtures/`. Do not commit generated
|
|
663
|
+
or downloaded `.docx` files.
|
|
664
|
+
|
|
665
|
+
Selection rules:
|
|
666
|
+
- every new or changed case;
|
|
667
|
+
- every case required by the triggers above;
|
|
668
|
+
- at least 20% of unchanged synthetic cases, rotating from the prior release (so
|
|
669
|
+
every retained synthetic case receives a human review over five release cycles);
|
|
670
|
+
- at least one legal and one administrative SuperDoc result; and
|
|
671
|
+
- representative list, table, formatted-run, and anchor/field/content-control
|
|
672
|
+
structures.
|
|
673
|
+
|
|
674
|
+
A major release or a change to package assembly requires a full visual sweep of
|
|
675
|
+
affected structure families.
|
|
676
|
+
|
|
677
|
+
Record the installed Word version and build from **File → Account → About Word**.
|
|
678
|
+
Differences in rendering can be version-specific.
|
|
679
|
+
|
|
680
|
+
#### Configure Word
|
|
681
|
+
|
|
682
|
+
For the tracked-change inspection:
|
|
683
|
+
|
|
684
|
+
1. Open the generated fixture directly in desktop Word.
|
|
685
|
+
2. On **Review**, select **All Markup**.
|
|
686
|
+
3. Under **Show Markup**, enable insertions/deletions, formatting, comments, and
|
|
687
|
+
all reviewers relevant to the case.
|
|
688
|
+
4. Use the expected balloons/inline display for the document and enable
|
|
689
|
+
paragraph marks when checking whitespace, tabs, breaks, lists, and empty
|
|
690
|
+
paragraphs.
|
|
691
|
+
5. Do not overwrite the generated fixture. Work on disposable copies if a
|
|
692
|
+
saved accepted or rejected view is useful.
|
|
693
|
+
|
|
694
|
+
#### Inspect each case (3-view checklist)
|
|
695
|
+
|
|
696
|
+
##### 1. Tracked-change view
|
|
697
|
+
|
|
698
|
+
- The document opens without a repair, conversion, or unreadable-content prompt.
|
|
699
|
+
- Word shows the expected revision count and author attribution.
|
|
700
|
+
- Insertions and deletions are anchored at the intended words or paragraph
|
|
701
|
+
marks; a small edit has not become an unexplained whole-paragraph rewrite.
|
|
702
|
+
- Untargeted text and surrounding revisions remain unchanged.
|
|
703
|
+
- Existing bold, italic, underline, highlighting, fonts, styles, and language
|
|
704
|
+
settings remain visually consistent.
|
|
705
|
+
- Spaces, tabs, manual breaks, paragraph spacing, indentation, and alignment
|
|
706
|
+
look intentional with formatting marks visible.
|
|
707
|
+
- Lists retain numbering, levels, continuation, indentation, and marker style.
|
|
708
|
+
- Tables retain widths, borders, merged cells, row heights, and alignment.
|
|
709
|
+
- Bookmarks, hyperlinks, fields, content controls, comments, and note references
|
|
710
|
+
remain in the correct visible location and still behave when activated.
|
|
711
|
+
- Headers, footers, section boundaries, page breaks, and pagination remain
|
|
712
|
+
stable around the edit.
|
|
713
|
+
- Revision balloons and comment balloons point to the correct content and do
|
|
714
|
+
not obscure or displace unrelated layout unexpectedly.
|
|
715
|
+
|
|
716
|
+
##### 2. Accept All view
|
|
717
|
+
|
|
718
|
+
On a disposable copy, choose **Accept All Changes** and verify:
|
|
719
|
+
|
|
720
|
+
- The resulting visible text expresses the intended edit.
|
|
721
|
+
- No deletion residue, empty revision wrapper, unexpected blank line, or stale
|
|
722
|
+
formatting remains.
|
|
723
|
+
- Lists, tables, fields, links, comments, notes, and page layout still work.
|
|
724
|
+
- Untargeted content is visually unchanged.
|
|
725
|
+
|
|
726
|
+
Close the copy without replacing the generated fixture.
|
|
727
|
+
|
|
728
|
+
##### 3. Reject All view
|
|
729
|
+
|
|
730
|
+
Reopen a fresh copy, choose **Reject All Changes**, and verify:
|
|
731
|
+
|
|
732
|
+
- The original visible text and formatting are restored.
|
|
733
|
+
- Original list numbering, table layout, fields, anchors, and pagination return.
|
|
734
|
+
- No content introduced by the edit remains.
|
|
735
|
+
|
|
736
|
+
#### Record the visual review result
|
|
737
|
+
|
|
738
|
+
Keep the review report with release-validation artifacts. Screenshots may be
|
|
739
|
+
stored under ignored `tmp/word-manual-review/<date>/` when useful, but do not
|
|
740
|
+
commit corpus document images or document contents without checking rights and
|
|
741
|
+
sensitivity.
|
|
742
|
+
|
|
743
|
+
Suggested report template:
|
|
744
|
+
|
|
745
|
+
```markdown
|
|
746
|
+
# Word visual review — <release/date>
|
|
747
|
+
|
|
748
|
+
- Reviewer:
|
|
749
|
+
- Review date:
|
|
750
|
+
- Word version/build:
|
|
751
|
+
- Automated synthetic result:
|
|
752
|
+
- Automated corpus result:
|
|
753
|
+
- Review type: Human sign-off | AI visual preflight
|
|
754
|
+
|
|
755
|
+
| Case | Why selected | All Markup | Accept All | Reject All | Result | Notes |
|
|
756
|
+
|---|---|---|---|---|---|---|
|
|
757
|
+
| example-case | New table structure | Pass | Pass | Pass | Pass | No layout shift |
|
|
758
|
+
|
|
759
|
+
## Failures or follow-ups
|
|
760
|
+
|
|
761
|
+
- None.
|
|
762
|
+
```
|
|
763
|
+
|
|
764
|
+
A **Pass** requires all three views to pass. Record **Fail** if rendering or
|
|
765
|
+
interaction is wrong even when automated text comparison passes. Turn a failure
|
|
766
|
+
into a fixed regression case when possible; otherwise record the exact harness
|
|
767
|
+
or Word-version limitation in the active reliability plan.
|
|
768
|
+
|
|
769
|
+
|
|
770
|
+
### AI-assisted Word visual preflight
|
|
771
|
+
|
|
772
|
+
Generate repeatable three-view PDF evidence for every layout-sensitive
|
|
773
|
+
synthetic fixture with:
|
|
774
|
+
|
|
775
|
+
```powershell
|
|
776
|
+
npm run test:word:visual
|
|
777
|
+
```
|
|
778
|
+
|
|
779
|
+
The command writes PDFs and `manifest.json` under ignored
|
|
780
|
+
`tmp/word-visual-review/rendered/` storage. It verifies that Word opened each
|
|
781
|
+
fixture and produced a non-empty, paginated rendering, but leaves the manifest
|
|
782
|
+
certification and every visual judgment pending. To render only named cases,
|
|
783
|
+
invoke `scripts/word-com-visual-suite.ps1 -Case case-one,case-two` directly.
|
|
784
|
+
|
|
785
|
+
An AI agent with Windows computer control may also open the generated fixtures
|
|
786
|
+
in the installed desktop Word application, switch among All Markup, Accept All,
|
|
787
|
+
and Reject All views, capture screenshots, and inspect them for visible
|
|
788
|
+
regressions. This is a useful intermediate oracle because it exercises the same
|
|
789
|
+
real UI a person sees rather than only `Document.Content.Text`.
|
|
790
|
+
|
|
791
|
+
Run an AI visual preflight:
|
|
792
|
+
|
|
793
|
+
- after a phase that changes reconstruction, formatting, list/table behavior,
|
|
794
|
+
package assembly, or revision display;
|
|
795
|
+
- after every five to ten new Word fixtures;
|
|
796
|
+
- for all newly introduced structure families;
|
|
797
|
+
- when automated Word passes but the XML change is unusually broad; and
|
|
798
|
+
- before asking a human to perform the release sample, so obvious failures are
|
|
799
|
+
found first.
|
|
800
|
+
|
|
801
|
+
For the focused real-document additions, run:
|
|
802
|
+
|
|
803
|
+
```powershell
|
|
804
|
+
npm run test:corpus:word:visual
|
|
805
|
+
```
|
|
806
|
+
|
|
807
|
+
This first runs the exact Word differential, then writes 120 PDFs and a pending
|
|
808
|
+
manifest under ignored `tmp/superdoc-word-visual-review/rendered/` storage: three
|
|
809
|
+
views for 22 list-focused cases, 16 table-focused cases, and two page-header
|
|
810
|
+
cases. The long-document set includes 6,000+ word council minutes,
|
|
811
|
+
an 8,000+ word zoning resolution, and a 59,000+ word prospectus with 180 tables.
|
|
812
|
+
Each invocation packages its fixtures in a process-specific directory so
|
|
813
|
+
a Word process left behind by an RPC disconnect cannot lock the next run's
|
|
814
|
+
inputs. The runner ignores `RPC_E_DISCONNECTED` during post-export cleanup and
|
|
815
|
+
restarts Word once when a disconnect interrupts an actual render. After Word
|
|
816
|
+
finishes, the command rebuilds `docs/test-comparison-dashboard.html` from that
|
|
817
|
+
exact process-specific fixture directory. The `Verified cached ...` messages
|
|
818
|
+
refer only to the pinned source-document downloads; edited comparison DOCX
|
|
819
|
+
files are regenerated on every run.
|
|
820
|
+
|
|
821
|
+
The AI should use the same selection rules and checklist as a human, inspect at
|
|
822
|
+
least the changed cases plus representative legal and administrative samples,
|
|
823
|
+
and save screenshots under ignored `tmp/word-manual-review/<date>/` storage.
|
|
824
|
+
Its report must identify itself as **AI visual preflight**, record the Word
|
|
825
|
+
version/build when available, list the exact cases and views inspected, and
|
|
826
|
+
separate observed facts from uncertain visual judgments.
|
|
827
|
+
|
|
828
|
+
AI visual review is advisory. It can catch missing or misplaced revisions,
|
|
829
|
+
unexpected whole-paragraph markup, obvious pagination shifts, broken tables,
|
|
830
|
+
lost indentation, clipped headers, and visibly misplaced anchors. It may miss
|
|
831
|
+
subtle font metrics, accessibility issues, field behavior that requires domain
|
|
832
|
+
knowledge, or a legally meaningful formatting distinction. An AI pass does not
|
|
833
|
+
satisfy the human release sign-off, and sensitive/private documents must not be
|
|
834
|
+
opened for AI review without explicit authorization.
|
|
835
|
+
|
|
836
|
+
### Multimodal LLM visual inspection (real-document spot checks)
|
|
837
|
+
|
|
838
|
+
Evaluating subtle visual defects—such as shifted table borders, unnatural run
|
|
839
|
+
breaks, font substitution, clipping headers, or awkward revision balloons—requires
|
|
840
|
+
interpreting rendered document pages visually rather than relying solely on
|
|
841
|
+
string assertions or raw XML checks. A multimodal Large Language Model (MLLM)
|
|
842
|
+
with vision capabilities can review rendered page images across the three
|
|
843
|
+
revision views (`allMarkup`, `acceptAll`, and `rejectAll`) and detect regressions
|
|
844
|
+
that are invisible to plain-text diffs.
|
|
845
|
+
|
|
846
|
+
#### Why multimodal inspection is kept separate
|
|
847
|
+
|
|
848
|
+
Passing high-resolution, multi-page document images through a vision-capable LLM
|
|
849
|
+
is **expensive, computationally intensive, and time-consuming**:
|
|
850
|
+
|
|
851
|
+
- Each document scenario produces three full-document rendering views (often 1
|
|
852
|
+
to 20+ pages per view).
|
|
853
|
+
- Sending dozens of high-resolution page images consumes significant token
|
|
854
|
+
budgets and model context.
|
|
855
|
+
- Latency per case is orders of magnitude higher than programmatic XML parsing
|
|
856
|
+
or COM automation.
|
|
857
|
+
|
|
858
|
+
For these reasons, multimodal visual inspection **forms a separate check from the
|
|
859
|
+
rest of the test suite**. It is never executed on every commit, local test run, or
|
|
860
|
+
full batch pipeline.
|
|
861
|
+
|
|
862
|
+
#### Sampling strategy: Random real-document spot checks
|
|
863
|
+
|
|
864
|
+
Instead of exhaustive or continuous execution, multimodal visual inspection is
|
|
865
|
+
conducted as a **random spot check of real documents here and there only**:
|
|
866
|
+
|
|
867
|
+
1. **Focus on authentic real documents:** Spot checks prioritize the
|
|
868
|
+
[SuperDoc real-document corpus](#superdoc-real-document-word-tests) rather than
|
|
869
|
+
simple synthetic fixtures. Real documents contain organic complexities—such
|
|
870
|
+
as multi-column headers, nested bullet styles, non-uniform table borders, and
|
|
871
|
+
mixed-font legal numbering—where visual layout errors are most likely to hide.
|
|
872
|
+
2. **Small random sampling:** A typical visual spot-check run selects a random
|
|
873
|
+
sample of **2 to 5 real document scenarios** (for example, one administrative
|
|
874
|
+
board agenda, one municipal minutes multi-table document, and one complex
|
|
875
|
+
zoning ordinance).
|
|
876
|
+
3. **Execution cadence:** Spot checks are run on-demand:
|
|
877
|
+
- periodically during active engine refactoring;
|
|
878
|
+
- when introducing major changes to run splitting, table reconciliation, or
|
|
879
|
+
list reconstruction; or
|
|
880
|
+
- as an advisory audit prior to a release cycle.
|
|
881
|
+
|
|
882
|
+
#### Multimodal inspection workflow
|
|
883
|
+
|
|
884
|
+
When performing a multimodal visual spot check:
|
|
885
|
+
|
|
886
|
+
1. **Render target pages to images:** Render the selected scenario in Word to
|
|
887
|
+
PDFs using `scripts/word-com-corpus-visual-suite.ps1 -Case <scenario-name>`,
|
|
888
|
+
then convert the relevant pages (or contact sheets) to PNG images using the
|
|
889
|
+
visual inspection helper:
|
|
890
|
+
```powershell
|
|
891
|
+
node scripts/inspect-visual-evidence.mjs --contact-sheets
|
|
892
|
+
```
|
|
893
|
+
2. **Submit to the multimodal model:** Provide the vision model with the side-by-side
|
|
894
|
+
page renders of `allMarkup`, `acceptAll`, and `rejectAll` along with the target
|
|
895
|
+
diff and context.
|
|
896
|
+
3. **Targeted visual checklist:** Prompt the model to evaluate specific layout risks:
|
|
897
|
+
- **Table cell integrity:** Did column widths shift? Are table borders intact?
|
|
898
|
+
Did multi-line cell text collapse or overlap?
|
|
899
|
+
- **Typography & Font metrics:** Did any run revert to default font size (e.g.
|
|
900
|
+
dropping from 14pt Georgia to 12pt Normal)? Did bold, italic, or underline
|
|
901
|
+
bleed into neighboring text?
|
|
902
|
+
- **List alignment:** Are bullet indents consistent with existing items? Are
|
|
903
|
+
there ghost markers or blank list rows?
|
|
904
|
+
- **Page apparatus:** Are headers, footers, and page numbers in their expected
|
|
905
|
+
margins without clipping or wrapping anomalies?
|
|
906
|
+
4. **Triaging findings:** If the multimodal model flags a visual abnormality,
|
|
907
|
+
treat it as an advisory signal:
|
|
908
|
+
- Verify the defect manually in Microsoft Word.
|
|
909
|
+
- Trace the visual defect to its underlying OOXML cause (per the
|
|
910
|
+
[visual failure diagnosis guidelines](#diagnose-visual-failures-in-the-generated-ooxml-first)).
|
|
911
|
+
- Add a minimal semantic regression test in
|
|
912
|
+
`tests/visual_failure_regression_tests.mjs` to ensure the issue is permanently
|
|
913
|
+
prevented in the automated suite (`npm test`).
|
|
914
|
+
|
|
915
|
+
## SuperDoc real-document Word tests
|
|
916
|
+
|
|
917
|
+
The corpus lane references selected documents from SuperDoc's
|
|
918
|
+
[docx-corpus](https://docxcorp.us/) under ODC-By 1.0. Source documents are never
|
|
919
|
+
committed. `tests/corpus/superdoc-english-legal-administrative.json` pins each
|
|
920
|
+
reviewed source and observed SHA-256; `tests/corpus/superdoc-word-scenarios.json`
|
|
921
|
+
defines named deterministic scenarios and records the structural coverage they
|
|
922
|
+
add. A source may support multiple scenarios when each one adds a distinct
|
|
923
|
+
behavioral claim.
|
|
924
|
+
|
|
925
|
+
To add a corpus case:
|
|
926
|
+
|
|
927
|
+
1. Select an English legal or administrative document that adds a structure or
|
|
928
|
+
task not already represented. Review its content and record why it is
|
|
929
|
+
suitable.
|
|
930
|
+
2. Add an explicit pinned reference and observed hash to the manifest. Do not
|
|
931
|
+
use a floating or bulk corpus download.
|
|
932
|
+
3. Fetch only that reference with
|
|
933
|
+
`npm run corpus:fetch:superdoc -- --id <pinned-id>`.
|
|
934
|
+
4. Inspect the document in Word and choose a target that occurs exactly once in
|
|
935
|
+
Word's source text.
|
|
936
|
+
5. Add the deterministic operation, shape, coverage labels, and review note to
|
|
937
|
+
`superdoc-word-scenarios.json`. Use `key` plus `sourceId` when adding another
|
|
938
|
+
scenario for an already pinned source. Use `operations` for a multi-change
|
|
939
|
+
atomic batch, or `part: "word/header<N>.xml"` for a page-header revision.
|
|
940
|
+
6. Run `npm test` for manifest/catalogue checks and `npm run test:corpus:word`
|
|
941
|
+
for package hashing plus the Word Accept All/Reject All differential.
|
|
942
|
+
|
|
943
|
+
The corpus packager starts from the original `.docx`, replaces only the declared
|
|
944
|
+
revision part (`word/document.xml` by default, or a named header part), and
|
|
945
|
+
verifies every untouched ZIP part byte-for-byte before Word opens the result.
|
|
946
|
+
The Word oracle derives multi-change expectations by applying each independently
|
|
947
|
+
declared replacement to Word's source text and reads header-story text separately
|
|
948
|
+
## Release validation and independent oracles
|
|
949
|
+
|
|
950
|
+
This package works on OOXML strings and intentionally leaves `.docx` zip
|
|
951
|
+
packaging to consumers (release *tooling* assembles minimal `.docx` fixtures
|
|
952
|
+
with a script-local zip writer; the published library has no zip dependency).
|
|
953
|
+
|
|
954
|
+
The test suite verifies the accept/reject round-trip invariant using the
|
|
955
|
+
library's own transforms. Because a shared misconception between the
|
|
956
|
+
generator and the resolver would pass those tests silently, release
|
|
957
|
+
validation adds **independent oracles**: Microsoft Word, LibreOffice, and
|
|
958
|
+
the ECMA-376 schemas.
|
|
959
|
+
|
|
960
|
+
### Pre-release automated check sequence
|
|
961
|
+
|
|
962
|
+
```bash
|
|
963
|
+
npm test # includes tests/roundtrip_fuzz_tests.mjs (seeded, deterministic)
|
|
964
|
+
npm run test:isolation
|
|
965
|
+
npm run check:types
|
|
966
|
+
npm run lint
|
|
967
|
+
```
|
|
968
|
+
|
|
969
|
+
The fuzz harness generates random paragraph structures and edits, asserting the
|
|
970
|
+
round-trip invariant plus `validateRedlineOoxml` on each case. Tune or reproduce with:
|
|
971
|
+
|
|
972
|
+
```bash
|
|
973
|
+
FUZZ_SEED=<seed> FUZZ_ITERATIONS=<n> node tests/roundtrip_fuzz_tests.mjs
|
|
974
|
+
```
|
|
975
|
+
|
|
976
|
+
A failing case prints its exact reproduction command.
|
|
977
|
+
|
|
978
|
+
### Reject-roundtrip matrix
|
|
979
|
+
|
|
980
|
+
`tests/reject_roundtrip_matrix_tests.mjs` (generators in
|
|
981
|
+
`tests/helpers/operation-generators.mjs`, fixtures in
|
|
982
|
+
`tests/helpers/reject-roundtrip-fixtures.mjs`) applies seeded, generated
|
|
983
|
+
operations (word/sentence/paragraph/range replace, insert-after, delete,
|
|
984
|
+
`replacements`, format, paragraph-format, plain-to-list, last-paragraph) to the
|
|
985
|
+
nested-lists fixture and synthetic documents (hyperlinks, breaks/tabs, simple
|
|
986
|
+
fields, tables, multi-author revisions, empty list items). Each case runs
|
|
987
|
+
through both `applyOperationsToDocumentXml` and `openDocx` with default options.
|
|
988
|
+
A `REVISION_FIDELITY_VIOLATION` refusal, a rejected view that differs from the
|
|
989
|
+
source, a missing intended accepted text, or bare/packaged divergence fails the
|
|
990
|
+
run; other refusals are counted as skipped. The default run (fixed seed, 150
|
|
991
|
+
cases, about 3 s) is part of `npm test`. Extended lane and replay:
|
|
992
|
+
|
|
993
|
+
```bash
|
|
994
|
+
FUZZ_SEED=<seed> MATRIX_ITERATIONS=<n> node tests/reject_roundtrip_matrix_tests.mjs
|
|
995
|
+
```
|
|
996
|
+
|
|
997
|
+
`MATRIX_VERBOSE=1` prints each skipped refusal; `MATRIX_MAX_REPORTED=<n>` raises
|
|
998
|
+
the failure print limit. Known unfixable cases go in
|
|
999
|
+
`tests/fixtures/reject-roundtrip-allowlist.json` (`{"entries":[{"fixture","kind","path","reason"}]}`);
|
|
1000
|
+
the file does not exist while the allowlist is empty.
|
|
1001
|
+
|
|
1002
|
+
The nightly `matrix-extended` job in `.github/workflows/validation.yml` runs 1000 cases with a date-derived `FUZZ_SEED`;
|
|
1003
|
+
a failure prints the seed, fixture, kind, operation and verifier violations. `tests/list_parser_diagnostics_tests.mjs` asserts that the
|
|
1004
|
+
list conversion paths emit no `console.warn`/`console.error` parser diagnostics under the pure-JS fallback and the explicit xmldom provider.
|
|
1005
|
+
|
|
1006
|
+
### Word revision semantics matrix
|
|
1007
|
+
|
|
1008
|
+
`tests/fixtures/word-revision-semantics/` holds small packages (`cases.json`
|
|
1009
|
+
lists them) covering each paragraph-mark revision shape, and
|
|
1010
|
+
`expectations.json`, what desktop Word 16 produced on Reject All and Accept All
|
|
1011
|
+
for each. `tests/word_revision_semantics_tests.mjs` checks the projection and
|
|
1012
|
+
the library's reject/accept against those recorded results and needs no Word.
|
|
1013
|
+
`KNOWN_DIVERGENCES` in that test lists current differences from Word; the suite
|
|
1014
|
+
fails when a listed divergence disappears or an unlisted one appears.
|
|
1015
|
+
|
|
1016
|
+
Regenerate the packages and re-record expectations after changing the case list
|
|
1017
|
+
or when Word's behavior may have changed (requires desktop Word; use `pwsh`,
|
|
1018
|
+
PowerShell 7, because Windows PowerShell 5.1 hangs in `SaveAs2` over COM here):
|
|
1019
|
+
|
|
1020
|
+
```bash
|
|
1021
|
+
node scripts/generate-word-revision-semantics-matrix.mjs
|
|
1022
|
+
pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/word-revision-semantics-matrix.ps1
|
|
1023
|
+
```
|
|
1024
|
+
|
|
1025
|
+
Review the `expectations.json` diff before committing it.
|
|
1026
|
+
|
|
1027
|
+
### Revision fidelity verification in tests
|
|
1028
|
+
|
|
1029
|
+
Every tracked operation is verified after it is applied (default `enforce`).
|
|
1030
|
+
Diagnostic lanes may override the default without code changes by setting
|
|
1031
|
+
`DOCX_REDLINE_REVISION_FIDELITY=warn` (apply and report) or `off`. An explicit
|
|
1032
|
+
`verifyRevisionFidelity` option or CLI `--revision-fidelity` flag wins over the
|
|
1033
|
+
environment variable. Use it only to investigate a refusal; the reject
|
|
1034
|
+
round-trip matrix and `tests/revision_fidelity_verifier_tests.mjs` run with the
|
|
1035
|
+
default.
|
|
1036
|
+
|
|
1037
|
+
### Runtime guardrail
|
|
1038
|
+
|
|
1039
|
+
`validateRedlineOoxml(oxml)` (exported from `index.js`) runs structural
|
|
1040
|
+
invariants at runtime and returns `{ valid, issues }`. Downstream packagers
|
|
1041
|
+
should call it before writing engine output into `word/document.xml`.
|
|
1042
|
+
|
|
1043
|
+
### Export fixtures
|
|
1044
|
+
|
|
1045
|
+
```bash
|
|
1046
|
+
node scripts/export-validation-fixtures.mjs
|
|
1047
|
+
```
|
|
1048
|
+
|
|
1049
|
+
Writes to `tmp/validation-docx/`, per case:
|
|
1050
|
+
|
|
1051
|
+
- `<name>.document.xml` — generated `word/document.xml` payload
|
|
1052
|
+
- `<name>.docx` — minimal assembled package
|
|
1053
|
+
- `<name>.expected.json` — expected accept-all / reject-all plain text,
|
|
1054
|
+
derived from edit *intent* (not from this library's transforms), so
|
|
1055
|
+
external consumers act as independent oracles.
|
|
1056
|
+
|
|
1057
|
+
### Schema validation (ECMA-376 transitional XSD)
|
|
1058
|
+
|
|
1059
|
+
```bash
|
|
1060
|
+
node scripts/export-validation-fixtures.mjs
|
|
1061
|
+
bash scripts/validate-fixtures-xsd.sh
|
|
1062
|
+
```
|
|
1063
|
+
|
|
1064
|
+
Downloads (and caches in `.cache/ooxml-schemas/`) the transitional
|
|
1065
|
+
wordprocessingml schemas from ECMA-376 Part 4, patches the `xml:` namespace
|
|
1066
|
+
import to resolve offline, and validates every `*.document.xml` fixture with
|
|
1067
|
+
`xmllint`. Requires `curl`, `unzip`, and `xmllint` (`libxml2-utils` on
|
|
1068
|
+
Debian/Ubuntu; available on Windows via conda/msys).
|
|
1069
|
+
|
|
1070
|
+
### LibreOffice consumer check
|
|
1071
|
+
|
|
1072
|
+
```bash
|
|
1073
|
+
cd tmp/validation-docx
|
|
1074
|
+
soffice --headless --convert-to pdf --outdir converted *.docx
|
|
1075
|
+
```
|
|
1076
|
+
|
|
1077
|
+
Verifies that a second independent OOXML consumer parses and converts the
|
|
1078
|
+
fixtures without error.
|
|
1079
|
+
|
|
1080
|
+
### Continuous validation
|
|
1081
|
+
|
|
1082
|
+
`.github/workflows/validation.yml` runs nightly (and on demand via
|
|
1083
|
+
`workflow_dispatch`):
|
|
1084
|
+
|
|
1085
|
+
1. **xsd-schema** — exports fixtures and validates them against the
|
|
1086
|
+
ECMA-376 transitional `wml.xsd`.
|
|
1087
|
+
2. **libreoffice** — exports fixtures and converts them with headless
|
|
1088
|
+
LibreOffice.
|
|
1089
|
+
3. **fuzz-extended** — 20,000 fuzz round-trip cases with a date-derived
|
|
1090
|
+
seed, so every night explores new inputs. A failure log includes the
|
|
1091
|
+
exact `FUZZ_SEED` reproduction command.
|
|
1092
|
+
|
|
1093
|
+
The Word differential check stays manual because it requires desktop Word;
|
|
1094
|
+
run it before tagging a release.
|
|
1095
|
+
|
|
1096
|
+
### JavaScript coverage baselines
|
|
1097
|
+
|
|
1098
|
+
```bash
|
|
1099
|
+
npm run test:coverage
|
|
1100
|
+
```
|
|
1101
|
+
|
|
1102
|
+
The command runs the complete JavaScript test suite under c8 and prints a
|
|
1103
|
+
per-file report. Coverage answers "which implementation paths did the automated
|
|
1104
|
+
JavaScript tests execute?" It does not prove that executed paths are correct,
|
|
1105
|
+
that generated OOXML opens in Word, or that the real-document corpus is broad
|
|
1106
|
+
enough. Treat it as a map for finding thinly tested code; Word, schema,
|
|
1107
|
+
LibreOffice, fuzz, and corpus checks provide different evidence.
|
|
1108
|
+
|
|
1109
|
+
Historical reference snapshots:
|
|
1110
|
+
|
|
1111
|
+
- **Initial Phase 7 baseline (2026-08-29)**:
|
|
1112
|
+
| Metric | Coverage |
|
|
1113
|
+
|---|---:|
|
|
1114
|
+
| Lines / statements | 79.57% |
|
|
1115
|
+
| Functions | 80.07% |
|
|
1116
|
+
| Branches | 69.22% |
|
|
1117
|
+
|
|
1118
|
+
- **Post-Phase-5 snapshot (2026-08-30)**:
|
|
1119
|
+
79.96% lines/statements, 80.95% functions, and 69.52% branches.
|
|
1120
|
+
|
|
1121
|
+
- **Post-Phase-5 snapshot (2026-09-04)**:
|
|
1122
|
+
89.81% statements/lines, 93.04% functions, and 77.34% branches repo-wide.
|
|
1123
|
+
The Node surface reports 99.61% statements/lines; `document-inspection.js`
|
|
1124
|
+
reports 97.64% statements/lines and 100% functions; `paragraph-text.js` reports
|
|
1125
|
+
100% statements/lines/functions.
|
|
1126
|
+
|
|
1127
|
+
Preserve snapshots so progress is visible over time rather than presenting
|
|
1128
|
+
coverage as a single pass/fail quality score.
|
|
1129
|
+
|
|
1130
|
+
## Choosing where a new test belongs
|
|
1131
|
+
|
|
1132
|
+
- A fixed bug always gets an automated regression.
|
|
1133
|
+
- A broad engine rule also gets deterministic fuzz coverage.
|
|
1134
|
+
- A claim about what Word renders or accepts gets a synthetic Word case.
|
|
1135
|
+
- A claim involving package complexity or realistic authoring structures gets
|
|
1136
|
+
a reviewed corpus case.
|
|
1137
|
+
- A structural XML rule should be checked by runtime validation and, where
|
|
1138
|
+
practical, the ECMA-376 XSD lane.
|
|
1139
|
+
|
|
1140
|
+
Keep expectations independent of the code under test. In particular, never use
|
|
1141
|
+
this library's Accept/Reject helpers to calculate the expected result for the
|
|
1142
|
+
Word differential.
|