arrolio 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (272) hide show
  1. checksums.yaml +7 -0
  2. data/.rspec +2 -0
  3. data/.rubocop.yml +343 -0
  4. data/CHANGELOG.md +18 -0
  5. data/CLAUDE.md +154 -0
  6. data/Gemfile +15 -0
  7. data/LICENSE +24 -0
  8. data/README.adoc +95 -0
  9. data/Rakefile +21 -0
  10. data/TODO.complete/01-true-type-font-metrics.md +56 -0
  11. data/TODO.complete/02-font-embedding.md +51 -0
  12. data/TODO.complete/03-knuth-plass-breaker.md +42 -0
  13. data/TODO.complete/04-table-flowable.md +60 -0
  14. data/TODO.complete/05-list-flowable.md +50 -0
  15. data/TODO.complete/06-notes-and-examples.md +41 -0
  16. data/TODO.complete/07-odd-even-templates-and-page-numbering.md +63 -0
  17. data/TODO.complete/08-image-flowable.md +45 -0
  18. data/TODO.complete/09-toc-and-leaders.md +66 -0
  19. data/TODO.complete/10-hyperlinks-and-destinations.md +46 -0
  20. data/TODO.complete/11-pdf-outline-bookmarks.md +47 -0
  21. data/TODO.complete/12-metadata-info-dict.md +42 -0
  22. data/TODO.complete/13-xmp-metadata.md +43 -0
  23. data/TODO.complete/14-footnotes.md +43 -0
  24. data/TODO.complete/15-color-management.md +33 -0
  25. data/TODO.complete/16-pdfua-struct-tree.md +35 -0
  26. data/TODO.complete/17-alt-text-and-actual-text.md +28 -0
  27. data/TODO.complete/18-math-rendering.md +41 -0
  28. data/TODO.complete/19-preformatted-and-code.md +31 -0
  29. data/TODO.complete/20-page-break-controls.md +46 -0
  30. data/TODO.complete/21-svg-renderer.md +40 -0
  31. data/TODO.complete/22-definition-lists.md +36 -0
  32. data/TODO.complete/23-multi-column.md +34 -0
  33. data/TODO.complete/24-cover-page-fidelity.md +40 -0
  34. data/TODO.complete/25-i18n-and-multi-language.md +28 -0
  35. data/TODO.complete/26-pixel-diff-harness.md +36 -0
  36. data/TODO.complete/27-pdfrb-only-text-extraction.md +37 -0
  37. data/TODO.complete/28-restore-nested-module-style.md +41 -0
  38. data/TODO.complete/29-font-embedding-wiring-debug.md +72 -0
  39. data/TODO.complete/30-adapter-decomposition.md +76 -0
  40. data/TODO.complete/31-subscript-superscript-inline-runs.md +56 -0
  41. data/TODO.complete/32-debug-output-cleanup.md +59 -0
  42. data/TODO.complete/33-comprehensive-adapter-specs.md +55 -0
  43. data/TODO.complete/34-table-column-auto-layout.md +50 -0
  44. data/TODO.complete/35-image-path-resolution.md +66 -0
  45. data/TODO.complete/36-xsl-converter-completeness.md +40 -0
  46. data/TODO.complete/37-content-model-completeness.md +54 -0
  47. data/TODO.complete/38-pipeline-config-externalization.md +55 -0
  48. data/TODO.complete/39-two-pass-cross-reference-resolution.md +66 -0
  49. data/TODO.complete/40-vertical-writing-modes.md +63 -0
  50. data/TODO.complete/41-pdf-annotations-hyperlinks.md +53 -0
  51. data/TODO.complete/42-font-manifest-configuration.md +75 -0
  52. data/TODO.complete/43-shrink-to-fit-containers.md +52 -0
  53. data/TODO.complete/44-back-of-book-index-generation.md +51 -0
  54. data/TODO.complete/45-pdf-forms-acroform.md +52 -0
  55. data/TODO.complete/46-pdf-digital-signatures.md +47 -0
  56. data/TODO.complete/47-ocp-flavor-separation.md +50 -0
  57. data/TODO.complete/48-oiml-as-pure-configuration.md +80 -0
  58. data/TODO.complete/49-oiml-flowbuilder-as-configuration.md +39 -0
  59. data/TODO.complete/50-delete-oiml-ruby-code.md +53 -0
  60. data/TODO.complete/51-xsl-as-authoritative-source.md +68 -0
  61. data/TODO.complete/52-flavor-packaging-separation.md +49 -0
  62. data/TODO.complete/53-second-flavor-spike.md +74 -0
  63. data/TODO.complete/54-flavor-manifest-contract.md +72 -0
  64. data/TODO.complete/55-header-footer-style-configuration.md +57 -0
  65. data/TODO.complete/56-generic-adapter-mece-decomposition.md +56 -0
  66. data/TODO.complete/57-strict-section-typing.md +49 -0
  67. data/TODO.complete/58-flavor-packaging-gem-convention.md +47 -0
  68. data/TODO.complete/59-xsl-converter-profiles.md +76 -0
  69. data/TODO.complete/60-adapter-fidelity-footnotes-notes-examples.md +62 -0
  70. data/TODO.complete/61-error-model-and-strict-mode.md +62 -0
  71. data/TODO.complete/62-font-resolution-and-strict-fonts.md +66 -0
  72. data/TODO.complete/63-purge-hardcoded-vocabulary-from-generic-core.md +68 -0
  73. data/TODO.complete/64-selector-driven-adapter-specs.md +46 -0
  74. data/TODO.complete/66-semantic-content-types.md +171 -0
  75. data/TODO.complete/README.md +48 -0
  76. data/TODO.implementation/01-foundation.md +46 -0
  77. data/TODO.implementation/02-content-contract.md +47 -0
  78. data/TODO.implementation/03-layoutspec-contract.md +46 -0
  79. data/TODO.implementation/04-style-system.md +43 -0
  80. data/TODO.implementation/05-visitor-pattern.md +40 -0
  81. data/TODO.implementation/06-afm-data-and-parser.md +37 -0
  82. data/TODO.implementation/07-afm-metrics.md +41 -0
  83. data/TODO.implementation/08-truetype-tables-and-metrics.md +48 -0
  84. data/TODO.implementation/09-font-registry-and-glyph-measurer.md +49 -0
  85. data/TODO.implementation/10-break-opportunity-and-inline-run.md +47 -0
  86. data/TODO.implementation/11-greedy-line-breaker.md +48 -0
  87. data/TODO.implementation/12-knuth-plass-line-breaker.md +46 -0
  88. data/TODO.implementation/13-line-and-alignment.md +45 -0
  89. data/TODO.implementation/14-knuth-elements-and-breaker.md +56 -0
  90. data/TODO.implementation/15-refactor-text-layout-on-knuth.md +47 -0
  91. data/TODO.implementation/16-flowable-and-frame.md +52 -0
  92. data/TODO.implementation/17-flow-context.md +50 -0
  93. data/TODO.implementation/18-text-flowable.md +52 -0
  94. data/TODO.implementation/19-spacer-and-page-break.md +52 -0
  95. data/TODO.implementation/20-engine-paged.md +63 -0
  96. data/TODO.implementation/21-output-tree.md +56 -0
  97. data/TODO.implementation/22-pdf-renderer.md +77 -0
  98. data/TODO.implementation/23-end-to-end-smoke.md +53 -0
  99. data/TODO.implementation/24-page-template-and-region.md +51 -0
  100. data/TODO.implementation/25-page-sequence-master.md +54 -0
  101. data/TODO.implementation/26-two-pass-layout.md +66 -0
  102. data/TODO.implementation/27-table-model.md +48 -0
  103. data/TODO.implementation/28-table-layouts.md +53 -0
  104. data/TODO.implementation/29-table-renderer.md +55 -0
  105. data/TODO.implementation/30-table-flowable.md +53 -0
  106. data/TODO.implementation/31-list-model.md +45 -0
  107. data/TODO.implementation/32-list-flowable.md +54 -0
  108. data/TODO.implementation/33-image-flowable.md +59 -0
  109. data/TODO.implementation/34-svg-renderer.md +58 -0
  110. data/TODO.implementation/35-svg-as-form-xobject.md +38 -0
  111. data/TODO.implementation/36-inline-builder.md +50 -0
  112. data/TODO.implementation/37-hyperlink.md +53 -0
  113. data/TODO.implementation/38-sub-superscript.md +44 -0
  114. data/TODO.implementation/39-leader-and-field-runs.md +60 -0
  115. data/TODO.implementation/40-bookmark-outline.md +45 -0
  116. data/TODO.implementation/41-destinations.md +52 -0
  117. data/TODO.implementation/42-toc-builder.md +52 -0
  118. data/TODO.implementation/43-footnotes.md +51 -0
  119. data/TODO.implementation/44-multi-column.md +45 -0
  120. data/TODO.implementation/45-page-numbering-and-section-tracking.md +50 -0
  121. data/TODO.implementation/46-color-management.md +47 -0
  122. data/TODO.implementation/47-metadata-and-xmp.md +41 -0
  123. data/TODO.implementation/48-struct-tree-and-marked-content.md +49 -0
  124. data/TODO.implementation/49-alt-text-and-actual-text.md +43 -0
  125. data/TODO.implementation/50-composer-facade.md +59 -0
  126. data/TODO.implementation/51-style-presets.md +44 -0
  127. data/TODO.implementation/52-oiml-adapter-and-e2e-pipeline.md +120 -0
  128. data/TODO.implementation/53-oiml-layout-spec.md +121 -0
  129. data/TODO.implementation/54-oiml-pipeline.md +97 -0
  130. data/TODO.implementation/55-pdf-comparator.md +98 -0
  131. data/TODO.implementation/56-diff-formatters.md +95 -0
  132. data/TODO.implementation/57-path-locator-and-enrichment.md +74 -0
  133. data/TODO.implementation/58-rspec-matchers-and-harness-api.md +124 -0
  134. data/TODO.implementation/59-pixel-diff-mode.md +93 -0
  135. data/TODO.implementation/60-yard-documentation.md +43 -0
  136. data/TODO.implementation/61-benchmarks-and-stress-tests.md +50 -0
  137. data/TODO.implementation/62-validation-framework.md +107 -0
  138. data/TODO.implementation/63-layout-rules-profile.md +110 -0
  139. data/TODO.implementation/64-pdfa-and-pdfua-conformance.md +74 -0
  140. data/TODO.implementation/65-feature-extractor.md +64 -0
  141. data/TODO.implementation/66-auto-fixer.md +75 -0
  142. data/TODO.implementation/README.md +169 -0
  143. data/data/arrolio/afm/Courier-Bold.afm +342 -0
  144. data/data/arrolio/afm/Courier-BoldOblique.afm +342 -0
  145. data/data/arrolio/afm/Courier-Oblique.afm +342 -0
  146. data/data/arrolio/afm/Courier.afm +342 -0
  147. data/data/arrolio/afm/Helvetica-Bold.afm +2827 -0
  148. data/data/arrolio/afm/Helvetica-BoldOblique.afm +2827 -0
  149. data/data/arrolio/afm/Helvetica-Oblique.afm +3051 -0
  150. data/data/arrolio/afm/Helvetica.afm +3051 -0
  151. data/data/arrolio/afm/Symbol.afm +213 -0
  152. data/data/arrolio/afm/Times-Bold.afm +2588 -0
  153. data/data/arrolio/afm/Times-BoldItalic.afm +2384 -0
  154. data/data/arrolio/afm/Times-Italic.afm +2667 -0
  155. data/data/arrolio/afm/Times-Roman.afm +2419 -0
  156. data/data/arrolio/afm/ZapfDingbats.afm +225 -0
  157. data/data/arrolio/glyphlist.txt +4306 -0
  158. data/exe/arrolio2pdf +28 -0
  159. data/lib/arrolio/asset_resolver.rb +36 -0
  160. data/lib/arrolio/color.rb +107 -0
  161. data/lib/arrolio/column_set.rb +64 -0
  162. data/lib/arrolio/composer.rb +10 -0
  163. data/lib/arrolio/config_driven_pipeline.rb +144 -0
  164. data/lib/arrolio/content/bibliography_item.rb +42 -0
  165. data/lib/arrolio/content/builder.rb +108 -0
  166. data/lib/arrolio/content/document.rb +60 -0
  167. data/lib/arrolio/content/example.rb +43 -0
  168. data/lib/arrolio/content/figure_group.rb +39 -0
  169. data/lib/arrolio/content/footnote.rb +46 -0
  170. data/lib/arrolio/content/form_field.rb +47 -0
  171. data/lib/arrolio/content/formula.rb +34 -0
  172. data/lib/arrolio/content/heading.rb +42 -0
  173. data/lib/arrolio/content/hyperlink.rb +42 -0
  174. data/lib/arrolio/content/image.rb +36 -0
  175. data/lib/arrolio/content/index_entry.rb +52 -0
  176. data/lib/arrolio/content/inline_run.rb +51 -0
  177. data/lib/arrolio/content/list.rb +60 -0
  178. data/lib/arrolio/content/note.rb +48 -0
  179. data/lib/arrolio/content/page_break.rb +28 -0
  180. data/lib/arrolio/content/paragraph.rb +38 -0
  181. data/lib/arrolio/content/preformatted.rb +47 -0
  182. data/lib/arrolio/content/section.rb +39 -0
  183. data/lib/arrolio/content/table.rb +117 -0
  184. data/lib/arrolio/content/term_entry.rb +43 -0
  185. data/lib/arrolio/content.rb +27 -0
  186. data/lib/arrolio/data_dir.rb +28 -0
  187. data/lib/arrolio/engine/cross_reference_registry.rb +69 -0
  188. data/lib/arrolio/engine/paged.rb +186 -0
  189. data/lib/arrolio/engine.rb +9 -0
  190. data/lib/arrolio/error.rb +70 -0
  191. data/lib/arrolio/flavor/manifest.rb +111 -0
  192. data/lib/arrolio/flavor/registry.rb +54 -0
  193. data/lib/arrolio/flavor.rb +23 -0
  194. data/lib/arrolio/flow_context.rb +37 -0
  195. data/lib/arrolio/flowable.rb +66 -0
  196. data/lib/arrolio/flowables/footnote_marker_flowable.rb +28 -0
  197. data/lib/arrolio/flowables/heading_flowable.rb +23 -0
  198. data/lib/arrolio/flowables/image_flowable.rb +53 -0
  199. data/lib/arrolio/flowables/list_flowable.rb +117 -0
  200. data/lib/arrolio/flowables/note_flowable.rb +19 -0
  201. data/lib/arrolio/flowables/page_break.rb +15 -0
  202. data/lib/arrolio/flowables/page_sequence_start.rb +28 -0
  203. data/lib/arrolio/flowables/spacer.rb +34 -0
  204. data/lib/arrolio/flowables/table_flowable.rb +147 -0
  205. data/lib/arrolio/flowables/text_flowable.rb +95 -0
  206. data/lib/arrolio/flowables/toc_line_flowable.rb +35 -0
  207. data/lib/arrolio/flowables/two_column_block.rb +56 -0
  208. data/lib/arrolio/flowables.rb +19 -0
  209. data/lib/arrolio/font/embedder.rb +197 -0
  210. data/lib/arrolio/font/manifest.rb +114 -0
  211. data/lib/arrolio/font/resolver.rb +161 -0
  212. data/lib/arrolio/font/text_encoder.rb +28 -0
  213. data/lib/arrolio/font.rb +10 -0
  214. data/lib/arrolio/font_metrics/afm_metrics.rb +91 -0
  215. data/lib/arrolio/font_metrics/registry.rb +55 -0
  216. data/lib/arrolio/font_metrics/true_type_metrics.rb +95 -0
  217. data/lib/arrolio/font_metrics.rb +10 -0
  218. data/lib/arrolio/frame.rb +36 -0
  219. data/lib/arrolio/generic_adapter.rb +672 -0
  220. data/lib/arrolio/generic_flow_builder.rb +338 -0
  221. data/lib/arrolio/glyph_measurer.rb +71 -0
  222. data/lib/arrolio/harness/pdf_diff.rb +104 -0
  223. data/lib/arrolio/harness/pixel_diff.rb +122 -0
  224. data/lib/arrolio/harness/text_diff.rb +53 -0
  225. data/lib/arrolio/harness/text_extractor.rb +58 -0
  226. data/lib/arrolio/harness.rb +10 -0
  227. data/lib/arrolio/inline_run.rb +59 -0
  228. data/lib/arrolio/layout_spec/flow.rb +17 -0
  229. data/lib/arrolio/layout_spec/loader.rb +117 -0
  230. data/lib/arrolio/layout_spec/page_template.rb +127 -0
  231. data/lib/arrolio/layout_spec/page_template_selector.rb +58 -0
  232. data/lib/arrolio/layout_spec/region.rb +38 -0
  233. data/lib/arrolio/layout_spec/scale_length.rb +34 -0
  234. data/lib/arrolio/layout_spec.rb +90 -0
  235. data/lib/arrolio/logger.rb +54 -0
  236. data/lib/arrolio/output/page.rb +69 -0
  237. data/lib/arrolio/output/placed_box.rb +68 -0
  238. data/lib/arrolio/output/region.rb +32 -0
  239. data/lib/arrolio/output.rb +10 -0
  240. data/lib/arrolio/renderer/accessibility_tagger.rb +70 -0
  241. data/lib/arrolio/renderer/font_registry.rb +32 -0
  242. data/lib/arrolio/renderer/link_annotator.rb +62 -0
  243. data/lib/arrolio/renderer/outline_builder.rb +102 -0
  244. data/lib/arrolio/renderer/pdf.rb +493 -0
  245. data/lib/arrolio/renderer/signature_config.rb +59 -0
  246. data/lib/arrolio/renderer/structure_tree_builder.rb +117 -0
  247. data/lib/arrolio/renderer/xmp_builder.rb +92 -0
  248. data/lib/arrolio/renderer.rb +17 -0
  249. data/lib/arrolio/style/definition.rb +98 -0
  250. data/lib/arrolio/style/loader.rb +25 -0
  251. data/lib/arrolio/style/registry.rb +53 -0
  252. data/lib/arrolio/style.rb +9 -0
  253. data/lib/arrolio/table/auto_layout.rb +91 -0
  254. data/lib/arrolio/table.rb +9 -0
  255. data/lib/arrolio/text_direction.rb +81 -0
  256. data/lib/arrolio/text_layout/break_opportunity.rb +103 -0
  257. data/lib/arrolio/text_layout/greedy.rb +110 -0
  258. data/lib/arrolio/text_layout/knuth_plass/breaker.rb +265 -0
  259. data/lib/arrolio/text_layout/knuth_plass/item.rb +84 -0
  260. data/lib/arrolio/text_layout/knuth_plass/item_builder.rb +103 -0
  261. data/lib/arrolio/text_layout/knuth_plass.rb +12 -0
  262. data/lib/arrolio/text_layout/line.rb +72 -0
  263. data/lib/arrolio/text_layout.rb +11 -0
  264. data/lib/arrolio/toc_builder.rb +31 -0
  265. data/lib/arrolio/version.rb +5 -0
  266. data/lib/arrolio/writing_mode.rb +75 -0
  267. data/lib/arrolio.rb +42 -0
  268. data/script/smoke.rb +32 -0
  269. data/scripts/write_content_extras.py +227 -0
  270. data/scripts/xsl_profiles/standoc.yml +218 -0
  271. data/scripts/xsl_to_config.rb +391 -0
  272. metadata +394 -0
@@ -0,0 +1,98 @@
1
+ ---
2
+ priority: P0
3
+ phase: 17
4
+ depends_on: [54]
5
+ layer: harness
6
+ est: 5d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ Modeled on `Canon::Comparison`, Arrolio needs a semantic PDF
13
+ comparator: take two PDFs (ours vs FOP reference), produce a
14
+ `DiffReport` of `DiffNode`s classified by **dimension** (text,
15
+ font, position, colour, image, vector, structure) and by
16
+ **severity** (normative vs informative formatting drift).
17
+
18
+ This is the **primary** diff mode. Pixel diff (TODO 59) is one
19
+ optional input; RSpec matchers (TODO 58) and formatters (TODO 56)
20
+ consume the report.
21
+
22
+ ## Approach
23
+
24
+ Inspired by `~/src/lutaml/canon/lib/canon/comparison/` — same shape,
25
+ different domain.
26
+
27
+ ### Files under `lib/arrolio/harness/comparator/`
28
+
29
+ - `base_comparator.rb` — abstract: takes two `Pdfrb::Document`
30
+ instances, produces `DiffReport`.
31
+ - `pdf_comparator.rb` — concrete comparator for PDF vs PDF.
32
+ - `dimension.rb` — base class for a diff dimension. Each subclass
33
+ compares one aspect of two corresponding elements.
34
+ - `dimension_set.rb` — registry of active dimensions per profile.
35
+ - `dimensions/` — one file per dimension:
36
+ - `text_content.rb` — string-equality on extracted text per
37
+ PlacedBox.
38
+ - `text_order.rb` — reading order across the page.
39
+ - `font.rb` — font family, size, weight, colour.
40
+ - `position.rb` — x/y/width/height within a page (with tolerance).
41
+ - `colour.rb` — fill/stroke colour match.
42
+ - `image_content.rb` — perceptual hash (pHash) for embedded images.
43
+ - `vector_path.rb` — path-data comparison for line/curve shapes.
44
+ - `structure.rb` — page count, region count, element nesting.
45
+ - `metadata.rb` — `/Info`, `/Metadata` XMP.
46
+ - `classifier.rb` — marks each `DiffNode` as `:normative` (changes
47
+ reader-visible meaning) or `:informative` (formatting drift within
48
+ tolerance). Used by RSpec matchers to decide pass/fail.
49
+ - `match_options.rb` — per-dimension tolerances (e.g. position
50
+ tolerance 0.5pt; phash hamming distance ≤ 5).
51
+ - `profile.rb` — presets:
52
+ - `:strict` — zero tolerance, all dimensions active.
53
+ - `:content_only` — text + structure only.
54
+ - `:visual_equiv` — text + image + colour, ignore exact position.
55
+ - `:oiml_regression` — OIML-tuned: position tolerance 1pt,
56
+ font-metric exact, ignore running-header date.
57
+
58
+ ### Algorithm
59
+
60
+ ```
61
+ 1. Parse both PDFs via Pdfrb.
62
+ 2. Walk pages pairwise (different page counts → structural diff).
63
+ 3. Per page, walk regions pairwise.
64
+ 4. Per region, walk PlacedBoxes pairwise (matching by reading order).
65
+ 5. For each pair, run every active dimension; collect DiffNodes.
66
+ 6. Classifier labels each DiffNode.
67
+ 7. Return DiffReport.
68
+ ```
69
+
70
+ Pairing across pages/regions uses reading-order + position heuristics
71
+ (canonical: top-to-bottom, left-to-right). When pairing is ambiguous,
72
+ emit a `:structure` diff with both candidates.
73
+
74
+ ### DiffReport
75
+
76
+ ```ruby
77
+ class Arrolio::Harness::DiffReport
78
+ attr_reader :differences, :profile, :stats
79
+
80
+ def normative_diffs; differences.select(&:normative?); end
81
+ def informative_diffs; differences.reject(&:normative?); end
82
+ def equivalent?(profile); normative_diffs.empty?; end
83
+ def similarity -> Float in [0, 1]
84
+ end
85
+ ```
86
+
87
+ ## Done-When
88
+
89
+ - [ ] Two identical PDFs produce `equivalent?(:strict) == true`.
90
+ - [ ] PDFs differing in one paragraph's text produce one
91
+ `:text_content` DiffNode, classified normative.
92
+ - [ ] PDFs differing in font size by 0.1pt produce one `:font`
93
+ DiffNode, classified informative (within default tolerance).
94
+ - [ ] PDFs with different page counts produce a `:structure` diff.
95
+ - [ ] Different embedded images produce `:image_content` diff with
96
+ phash distance.
97
+ - [ ] Profile selection correctly activates/deactivates dimensions.
98
+ - [ ] Specs cover each dimension + classifier + at least 2 profiles.
@@ -0,0 +1,95 @@
1
+ ---
2
+ priority: P0
3
+ phase: 17
4
+ depends_on: [55]
5
+ layer: harness
6
+ est: 3d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ `DiffReport` is data; users need it rendered. Different audiences
13
+ need different views:
14
+
15
+ - **CI / regression test**: machine-readable JSON + pass/fail summary.
16
+ - **Developer local**: pretty terminal output with colour and context.
17
+ - **Designer review**: HTML report with side-by-side pages and
18
+ highlighted diff regions (paired with pixel-diff images from
19
+ TODO 59 when available).
20
+ - **Audit log**: by-element dump with canonical paths.
21
+
22
+ Modeled on `Canon::DiffFormatter::*` (by_line, by_object,
23
+ pretty_diff_formatter, diff_detail_formatter).
24
+
25
+ ## Approach
26
+
27
+ Files under `lib/arrolio/harness/formatters/`:
28
+
29
+ - `base_formatter.rb` — abstract: takes `DiffReport`, returns String
30
+ (or writes to IO).
31
+
32
+ - `summary_formatter.rb` — one-line pass/fail + similarity %.
33
+ Example: `✓ equivalent (97.3% similarity, 4 informative diffs)`.
34
+ Used as the first line of every other format.
35
+
36
+ - `pretty_formatter.rb` — coloured terminal output. Groups diffs by
37
+ severity (normative first), then by page. Each diff shows:
38
+ - Dimension icon (📝 text, 🔤 font, 📍 position, 🎨 colour, 🖼 image).
39
+ - Canonical path (TODO 57).
40
+ - Before/after snippet (truncated to 60 chars).
41
+ - Reason phrase.
42
+
43
+ - `by_page_formatter.rb` — per-page section; lists every diff on
44
+ that page. Good for "what's wrong with page 7?".
45
+
46
+ - `by_element_formatter.rb` — per-element-namespace section. Lists
47
+ every diff affecting e.g. all tables, all headings.
48
+
49
+ - `json_formatter.rb` — machine-readable JSON for CI integration.
50
+ Schema:
51
+ ```json
52
+ {
53
+ "equivalent": false,
54
+ "similarity": 0.873,
55
+ "profile": "oiml_regression",
56
+ "normative_diffs": [...],
57
+ "informative_diffs": [...]
58
+ }
59
+ ```
60
+
61
+ - `html_formatter.rb` — standalone HTML report. Embeds:
62
+ - Page thumbnails (left: reference, right: ours) with diff regions
63
+ boxed.
64
+ - Per-diff detail panel.
65
+ - Filter controls (hide informative, hide by dimension).
66
+ - Uses pixel-diff images (TODO 59) when available.
67
+
68
+ - `junit_formatter.rb` — JUnit XML for CI systems that consume it
69
+ (Jenkins, GitLab CI).
70
+
71
+ - `legend.rb` — explains icons, colours, dimension names. Shared
72
+ across formatters.
73
+
74
+ - `theme.rb` — ANSI colour palette + HTML CSS variables.
75
+
76
+ ## Approach for diffs within a diff
77
+
78
+ Use `Diff::LCS.sdiff` on the relevant extracted text (per page or
79
+ per element) to produce line-level hunks. Wrap each hunk as a
80
+ `DiffBlock` (modeled on `Canon::Diff::DiffBlock`). The formatter
81
+ renders DiffBlocks with `+`/`-`/` ` prefixes and optional colour.
82
+
83
+ ## Done-When
84
+
85
+ - [ ] `SummaryFormatter.(report)` returns a one-line pass/fail.
86
+ - [ ] `PrettyFormatter.(report)` produces coloured terminal output
87
+ that fits in 80 columns.
88
+ - [ ] `JsonFormatter.(report)` produces valid JSON parseable by
89
+ `JSON.parse`.
90
+ - [ ] `HtmlFormatter.(report)` produces a valid HTML file openable
91
+ in a browser.
92
+ - [ ] `JUnitFormatter.(report)` produces valid JUnit XML.
93
+ - [ ] Each formatter has at least one fixture-based spec.
94
+ - [ ] A CLI flag (`--format pretty|json|html|junit|summary`) selects
95
+ the formatter.
@@ -0,0 +1,74 @@
1
+ ---
2
+ priority: P1
3
+ phase: 17
4
+ depends_on: [55]
5
+ layer: harness
6
+ est: 2d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ Every `DiffNode` should carry a **canonical path** so users can
13
+ pinpoint exactly where the difference is. Modeled on canon's
14
+ `PathBuilder`, `SourceLocator`, `DiffNodeEnricher`, `DiffCharRange`,
15
+ `DiffLine`, `DiffContext`.
16
+
17
+ A canonical path looks like:
18
+ ```
19
+ page[2].region[:body].flowable[3].line[1].run[0].text_content
20
+ ```
21
+
22
+ Plus enrichment: each DiffNode should carry `serialized_before` /
23
+ `serialized_after` snippets, `line_range_before` / `line_range_after`
24
+ (line ranges in the source PDFs' content streams), and `char_ranges`
25
+ (character-level diff within a changed text run).
26
+
27
+ ## Approach
28
+
29
+ Files under `lib/arrolio/harness/`:
30
+
31
+ - `path_builder.rb` — constructs canonical paths. Walks the DiffNode's
32
+ source location chain (page → region → flowable → line → run →
33
+ property) and builds the dotted path. Each segment includes an
34
+ ordinal index for stable ordering.
35
+
36
+ - `source_locator.rb` — given a `Pdfrb::Model::Object` reference,
37
+ returns its source location: `(page_number, oid, gen,
38
+ content_stream_offset)`.
39
+
40
+ - `diff_node_enricher.rb` — post-processing pass over a `DiffNode[]`:
41
+ 1. Compute canonical path via `PathBuilder`.
42
+ 2. Serialize before/after content (text run, font dict, image
43
+ stream).
44
+ 3. Compute line ranges in the source PDFs.
45
+ 4. For text differences, run `Diff::LCS` on character level →
46
+ `DiffCharRange[]`.
47
+
48
+ - `diff_char_range.rb` — value object: `start_before`, `end_before`,
49
+ `start_after`, `end_after`, `kind` (`:insert`, `:delete`,
50
+ `:replace`). Used by PrettyFormatter to highlight just the changed
51
+ characters within a run.
52
+
53
+ - `diff_line.rb`, `diff_block.rb`, `diff_context.rb` — line-level
54
+ diff artifacts (same as canon's). Used by line-diff formatters.
55
+
56
+ - `diff_classifier.rb` — refines the basic normative/informative
57
+ classification from TODO 55 with heuristics:
58
+ - Position drift under 1pt → informative.
59
+ - Font size drift under 0.1pt → informative.
60
+ - Colour drift under ΔE 2.0 (perceptual) → informative.
61
+ - Text content change of any kind → normative.
62
+ - Image phash distance ≤ 5 → informative.
63
+
64
+ ## Done-When
65
+
66
+ - [ ] Every DiffNode has a non-nil `path` after enrichment.
67
+ - [ ] Paths are stable across runs (same input → same paths).
68
+ - [ ] `serialized_before` / `serialized_after` are non-nil for text
69
+ diffs; truncation is consistent (60 chars + ellipsis).
70
+ - [ ] Character ranges highlight exactly the changed characters in a
71
+ text run.
72
+ - [ ] Classifier correctly labels a 0.05pt font size drift as
73
+ informative.
74
+ - [ ] Specs cover path building, enrichment, classifier thresholds.
@@ -0,0 +1,124 @@
1
+ ---
2
+ priority: P0
3
+ phase: 17
4
+ depends_on: [55, 56, 57]
5
+ layer: harness
6
+ est: 2d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ For tests and CI, users need a fluent RSpec API. Two matchers cover
13
+ the common cases:
14
+
15
+ ```ruby
16
+ expect(our_pdf_bytes).to match_pdf(ref_path) # strict
17
+ expect(our_pdf_bytes).to be_equivalent_to_pdf(ref_path) # ignores informative
18
+ ```
19
+
20
+ Plus a programmatic `Arrolio::Harness.diff` for non-RSpec use.
21
+
22
+ ## Approach
23
+
24
+ ### RSpec matchers
25
+
26
+ File: `lib/arrolio/harness/rspec.rb` (auto-registered into RSpec
27
+ when required from `spec_helper.rb`).
28
+
29
+ ```ruby
30
+ RSpec::Matchers.define :match_pdf do |reference_path|
31
+ match do |actual_bytes|
32
+ report = Arrolio::Harness.diff(
33
+ ours: actual_bytes,
34
+ reference: File.binread(reference_path),
35
+ profile: @profile || :strict
36
+ )
37
+ report.equivalent?(profile: @profile || :strict).tap do
38
+ @failure_report = report
39
+ end
40
+ end
41
+
42
+ chain :with_profile do |profile|
43
+ @profile = profile
44
+ end
45
+
46
+ failure_message do
47
+ Arrolio::Harness::Formatters::PrettyFormatter.(@failure_report)
48
+ end
49
+ end
50
+
51
+ RSpec::Matchers.define :be_equivalent_to_pdf do |reference_path|
52
+ # Same as match_pdf but profile: :content_only by default —
53
+ # ignores font-size drift, position drift under 1pt, etc.
54
+ end
55
+ ```
56
+
57
+ ### Programmatic API
58
+
59
+ File: `lib/arrolio/harness.rb`.
60
+
61
+ ```ruby
62
+ module Arrolio::Harness
63
+ module_function
64
+
65
+ def diff(ours:, reference:, profile: :strict)
66
+ ours_doc = Pdfrb::Document.new(io: StringIO.new(ours))
67
+ ref_doc = Pdfrb::Document.new(io: StringIO.new(reference))
68
+ comparator = Comparator::PdfComparator.new(profile:)
69
+ report = comparator.compare(ours_doc, ref_doc)
70
+ DiffNodeEnricher.enrich(report)
71
+ report
72
+ end
73
+
74
+ def equivalent?(ours:, reference:, profile: :strict)
75
+ diff(ours:, reference:, profile:).equivalent?(profile:)
76
+ end
77
+
78
+ def format(report, format: :pretty)
79
+ Formatters.from_name(format).(report)
80
+ end
81
+ end
82
+ ```
83
+
84
+ ### CLI integration
85
+
86
+ Extend `exe/arrolio-oiml` (TODO 54):
87
+
88
+ ```sh
89
+ # Diff our rendering of OIML XML against FOP's:
90
+ arrolio-oiml diff document.presentation.xml \
91
+ --reference=document.fop.pdf \
92
+ --profile=oiml_regression \
93
+ --format=pretty|json|html|junit
94
+
95
+ # Diff two pre-rendered PDFs:
96
+ arrolio-harness diff ours.pdf ref.pdf --profile=strict
97
+ ```
98
+
99
+ ### Spec wiring
100
+
101
+ In `spec/spec_helper.rb`:
102
+ ```ruby
103
+ require "arrolio/harness/rspec"
104
+ ```
105
+
106
+ Then `spec/arrolio/oiml/e2e_spec.rb` becomes:
107
+ ```ruby
108
+ it "#{base} matches FOP reference" do
109
+ out = StringIO.new
110
+ Arrolio::Oiml::Pipeline.(xml, io: out)
111
+ expect(out.string).to be_equivalent_to_pdf(ref_pdf)
112
+ .with_profile(:oiml_regression)
113
+ end
114
+ ```
115
+
116
+ ## Done-When
117
+
118
+ - [ ] `require "arrolio/harness/rspec"` registers both matchers.
119
+ - [ ] `match_pdf` fails with a useful pretty-formatted message.
120
+ - [ ] `be_equivalent_to_pdf` ignores informative diffs by default.
121
+ - [ ] `Arrolio::Harness.diff(...)` returns a `DiffReport`.
122
+ - [ ] `Arrolio::Harness.equivalent?(...)` returns a bool.
123
+ - [ ] CLI `arrolio-oiml diff ...` exits 0 on equivalent, 1 on diff.
124
+ - [ ] CI-friendly: JSON formatter output stable across runs.
@@ -0,0 +1,93 @@
1
+ ---
2
+ priority: P1
3
+ phase: 17
4
+ depends_on: [55, 57]
5
+ layer: harness
6
+ est: 2d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ The semantic comparator (TODO 55) catches missing words and
13
+ structural changes. But some visual regressions slip through — a
14
+ missing table border, a 0.5pt colour shift, an anti-aliasing change.
15
+ Pixel diff rasterises both PDFs and compares images, catching
16
+ anything semantic diff misses.
17
+
18
+ Pixel diff is one **mode** within the harness, not the whole diff
19
+ strategy. It's expensive (~1s/page at 150 DPI; ~10s/page at 300 DPI)
20
+ so it's opt-in via profile, not on by default.
21
+
22
+ ## Approach
23
+
24
+ Files under `lib/arrolio/harness/pixel/`:
25
+
26
+ - `rasteriser.rb` — wraps `pdftoppm` (poppler) to convert a PDF to a
27
+ PNG per page at configurable DPI. Falls back gracefully if
28
+ `pdftoppm` isn't installed.
29
+
30
+ - `image_comparator.rb` — uses ImageMagick `compare` to compute a
31
+ per-page diff image and a similarity metric (0..1). Optionally
32
+ uses libvips for faster batch comparison.
33
+
34
+ - `pixel_diff_report.rb` — value object: `per_page_similarity`
35
+ (Array of Float), `overall_similarity` (Float),
36
+ `diff_image_paths` (Array of paths to per-page diff PNGs).
37
+
38
+ - `pixel_dimension.rb` — bridges into the comparator subsystem: a
39
+ `:pixel` dimension that contributes a `DiffNode` when similarity
40
+ drops below threshold (default 0.98 per page).
41
+
42
+ - `html_embedder.rb` — when the HTML formatter (TODO 56) runs,
43
+ embeds the per-page diff images as base64 PNGs in the report.
44
+
45
+ ### Pipeline integration
46
+
47
+ Pixel diff plugs into the comparator (TODO 55) as one more dimension:
48
+
49
+ ```ruby
50
+ comparator = Comparator::PdfComparator.new(
51
+ profile: :visual_strict
52
+ )
53
+ # profile: :visual_strict includes :pixel dimension
54
+ ```
55
+
56
+ ### Heuristic thresholds
57
+
58
+ - 1.0 similarity → identical; no diff.
59
+ - ≥ 0.99 → informative (anti-aliasing noise).
60
+ - 0.95..0.99 → flagged; usually a real but minor visual change.
61
+ - < 0.95 → normative; almost certainly a visible regression.
62
+
63
+ ### CLI
64
+
65
+ ```sh
66
+ # Run pixel diff alone
67
+ arrolio-harness pixel-diff ours.pdf ref.pdf --dpi=150
68
+
69
+ # Combined semantic + pixel diff
70
+ arrolio-harness diff ours.pdf ref.pdf --profile=visual_strict
71
+ ```
72
+
73
+ ### CI integration
74
+
75
+ - Rake task `rake harness:pixel_diff[fixture]` — runs against one
76
+ fixture.
77
+ - Rake task `rake harness:pixel_diff_all` — runs against every
78
+ fixture; uploads the HTML report as a CI artifact.
79
+ - Weekly CI job (not per-commit) due to runtime cost.
80
+
81
+ ## Done-When
82
+
83
+ - [ ] Two identical PDFs produce overall similarity 1.0.
84
+ - [ ] Two PDFs with one word different produce overall similarity
85
+ > 0.99 with a visible diff blob in the report.
86
+ - [ ] Two PDFs with a colour change produce overall similarity >
87
+ 0.95 with the changed region highlighted.
88
+ - [ ] HTML report renders in a browser; diff images embedded as
89
+ base64.
90
+ - [ ] The `:pixel` dimension integrates cleanly with the comparator.
91
+ - [ ] If `pdftoppm`/ImageMagick isn't installed, the dimension
92
+ skips cleanly with a warning (no crash).
93
+ - [ ] Profile `:visual_strict` includes pixel diff; `:strict` doesn't.
@@ -0,0 +1,43 @@
1
+ ---
2
+ priority: P3
3
+ phase: 18
4
+ depends_on: []
5
+ layer: polish
6
+ est: 2d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ Arrolio needs API documentation for users who didn't write it. YARD
13
+ is the standard Ruby doc tool. Every public class and method should
14
+ have YARD docstrings; the docs should render cleanly to HTML.
15
+
16
+ ## Approach
17
+
18
+ - Add `yard` gem to the development group.
19
+ - `.yardopts` with `--no-private`, `--markup markdown`,
20
+ `--files CHANGELOG.md,LICENSE`, `--output-dir doc/yard`.
21
+ - Add YARD docstrings to every public class and method:
22
+ - Module purpose statement at the top of each `lib/arrolio/*.rb`.
23
+ - Per-method: `# Brief description.` then `@param`, `@return`,
24
+ `@example`.
25
+ - Cross-link related classes with `{Class::Name}`.
26
+ - Rake task `yard` to generate; `rake yard:serve` for live preview.
27
+
28
+ Modules to document first (in order of API stability):
29
+ 1. `Arrolio` (top-level — the autoload tree, the `compose` shortcut).
30
+ 2. `Arrolio::Content::*` — content contract.
31
+ 3. `Arrolio::LayoutSpec::*` — layout spec.
32
+ 4. `Arrolio::Engine::Paged` — engine entry point.
33
+ 5. `Arrolio::Renderer::Pdf` — render entry point.
34
+ 6. `Arrolio::Composer` — high-level facade.
35
+ 7. `Arrolio::Oiml::*` — OIML pipeline (the main use case).
36
+
37
+ ## Done-When
38
+
39
+ - [ ] `bundle exec yard` produces HTML in `doc/yard/`.
40
+ - [ ] Every public method has a docstring.
41
+ - [ ] No "Missing documentation" warnings from YARD on public API.
42
+ - [ ] `Arrolio::Composer` is documented with a working example.
43
+ - [ ] The README's quick-start links into the YARD docs.
@@ -0,0 +1,50 @@
1
+ ---
2
+ priority: P3
3
+ phase: 18
4
+ depends_on: [54]
5
+ layer: polish
6
+ est: 2d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ Need performance baselines to detect regressions and find hot paths.
13
+ Also need to know Arrolio's behaviour on stress cases (very long
14
+ documents, deep tables, thousands of paragraphs) before claiming
15
+ production-ready.
16
+
17
+ ## Approach
18
+
19
+ Files:
20
+
21
+ - `benchmarks/bench.rb` — runs against a generated test document
22
+ (10, 50, 200 pages) and reports wall-clock per phase:
23
+ - Content tree construction.
24
+ - LayoutSpec construction.
25
+ - Flowable list construction.
26
+ - Engine::Paged layout.
27
+ - Renderer::Pdf render.
28
+ - Total.
29
+ - Compares against `HexaPDF` (if installed) and FOP (if installed)
30
+ on the equivalent source.
31
+
32
+ - `spec/arrolio/stress_spec.rb` — tagged `:stress`:
33
+ - 1000-page document lays out without crash.
34
+ - Memory after layout is < 500 MB.
35
+ - A 100-row table lays out across 5+ pages.
36
+ - A 100-item list lays out correctly.
37
+ - A document with 100 inline runs per paragraph lays out correctly.
38
+
39
+ - Profiling: `stackprof` integration; `rake bench:profile` produces
40
+ a flamegraph.
41
+
42
+ ## Done-When
43
+
44
+ - [ ] `rake bench` runs and prints per-phase timings.
45
+ - [ ] A 100-page OIML-style document renders in < 30 seconds.
46
+ - [ ] Memory stays bounded: 1000-page doc uses < 500 MB.
47
+ - [ ] Hot paths identified; obvious inefficiencies fixed.
48
+ - [ ] Stress specs pass under default RSpec config (not just `:stress`).
49
+ - [ ] Benchmark history tracked in `benchmarks/history.csv` so
50
+ regressions are visible across commits.
@@ -0,0 +1,107 @@
1
+ ---
2
+ priority: P1
3
+ phase: 19
4
+ depends_on: [22, 21]
5
+ layer: conformance
6
+ est: 3d
7
+ status: pending
8
+ ---
9
+
10
+ ## Problem
11
+
12
+ Modeled on `veraPDF/pdfa/validation/profiles/`, Arrolio needs a
13
+ generic **rule-based validation framework**. A rule is `(id,
14
+ applies_to, test_expression, severity, spec_reference)`. A profile
15
+ is a collection of rules. Validation runs every rule over the
16
+ document and produces `TestAssertion[]` (pass/fail per rule per
17
+ location).
18
+
19
+ Two consumers:
20
+
21
+ 1. **Self-validation** (Phase 19 layout rules, TODO 63): did
22
+ Arrolio itself produce a document free of layout regressions?
23
+ 2. **Conformance validation** (TODO 64): does the rendered PDF meet
24
+ PDF/A or PDF/UA spec requirements?
25
+
26
+ ## Approach
27
+
28
+ Files under `lib/arrolio/conformance/`:
29
+
30
+ - `rule.rb` — frozen value object:
31
+ ```ruby
32
+ Rule = Struct.new(:id, :applies_to, :test, :severity,
33
+ :spec_reference, :tags, keyword_init: true)
34
+ ```
35
+ - `applies_to` — Symbol naming the target type
36
+ (`:page`, `:flowable`, `:table`, `:image`, `:document`).
37
+ - `test` — a `Proc` or DSL expression that returns `true`/`false`
38
+ given the target + context.
39
+ - `severity` — `:error`, `:warning`, `:info`.
40
+
41
+ - `profile.rb` — collection of rules + metadata (name, version,
42
+ spec_link).
43
+
44
+ - `test_assertion.rb` — frozen value object:
45
+ ```ruby
46
+ TestAssertion = Struct.new(:rule_id, :location, :status,
47
+ :message, :evidence, keyword_init: true)
48
+ ```
49
+ - `status` — `:passed`, `:failed`, `:skipped`.
50
+ - `location` — `Location` (reuses TODO 57 source locator).
51
+
52
+ - `validation_result.rb` — frozen value object:
53
+ - `profile` — the Profile applied.
54
+ - `assertions` — Array of TestAssertion.
55
+ - `compliant?` — true if no `:error` severity failed.
56
+ - `summary` — counts by status + severity.
57
+
58
+ - `validator.rb` — runs a Profile over an `Output::Page[]` (or a
59
+ Pdfrb::Document):
60
+ ```ruby
61
+ class Arrolio::Conformance::Validator
62
+ def initialize(profile)
63
+ def validate(target) -> ValidationResult
64
+ end
65
+ ```
66
+
67
+ - `profile_registry.rb` — name → Profile lookup. Built-in profiles
68
+ register at load.
69
+
70
+ - `profile_loader.rb` — loads profiles from YAML or Ruby DSL.
71
+
72
+ ### YAML profile format
73
+
74
+ ```yaml
75
+ name: oiml-render
76
+ version: 1.0.0
77
+ spec_reference: "OIML R-series style guide v3"
78
+ rules:
79
+ - id: no-orphan-headings
80
+ applies_to: flowable
81
+ severity: error
82
+ test: |
83
+ target.kind == :heading && target.level <= 6 && !target.last_on_page?
84
+ - id: page-count-positive
85
+ applies_to: document
86
+ severity: error
87
+ test: "target.pages.length > 0"
88
+ - id: every-page-has-footer
89
+ applies_to: page
90
+ severity: warning
91
+ test: "target.region(:after).placed_boxes.any?"
92
+ ```
93
+
94
+ The `test` expression is evaluated in a sandboxed context (no
95
+ `eval`, no `send`); it's a small expression language with comparison,
96
+ boolean, and method-call operators only.
97
+
98
+ ## Done-When
99
+
100
+ - [ ] A profile with 1 trivial rule (page-count-positive) passes on
101
+ a valid doc, fails on an empty doc.
102
+ - [ ] YAML loader produces equivalent Profile to Ruby DSL.
103
+ - [ ] `Validator.validate(output_pages)` returns a ValidationResult.
104
+ - [ ] `result.compliant?` is true iff no error-severity failures.
105
+ - [ ] Each failed TestAssertion carries a Location pointing at the
106
+ offending page/box.
107
+ - [ ] Spec coverage for happy path + every severity + YAML loading.