arrolio 0.1.0 → 0.1.2

data/TODO.complete/87-odd-even-page-templates.md

@@ -0,0 +1,55 @@
+---
+priority: P2
+impact: medium
+depends_on: [76]
+layer: render
+status: pending
+est: 1d
+---
+
+## Problem
+
+The OIML reference PDF uses distinct page templates for odd vs even
+pages:
+- Odd pages: header right-aligned
+- Even pages: header left-aligned (or centered when title_complementary
+  is present)
+
+Our engine produces correct page-count parity but doesn't apply
+distinct templates based on page number. The header alignment shifts
+but the underlying template (margins, region extents) is the same.
+
+## Current state
+
+- `header_align_for(page_number)` in `Engine::Paged` returns `:left`
+  for even, `:right` for odd — SHIPPED and now REACHABLE
+  (2026-08-25): a nil `header_align` flows through
+  PageSequenceStart/Output::Page as "no flavor opinion"; the builder
+  no longer defaults to :right, and the OIML flavor dropped its
+  explicit `header_align: right` pins. Explicit values still win.
+- Page templates: single `body` template, no odd/even variants.
+
+## Approach
+
+1. **Add odd/even template variants to LayoutSpec.** Each
+   `page_template` entry can specify `odd` and `even` sub-templates
+   with different margins, headers, footers.
+
+2. **Wire PageSequenceMaster.** The XSL uses
+   `<xsl:conditional-page-master-reference>` with
+   `odd-or-even` condition. Map this to the engine's page-open logic.
+
+3. **Apply template per page.** `Engine::Paged#open_page` should
+   select the template based on `page_number.odd?` / `page_number.even?`.
+
+## Done-When
+
+- [ ] Odd and even pages use different templates when configured
+- [ ] Header alignment matches reference on all pages
+- [ ] Blank pages inserted for even-page chapter starts
+- [ ] Specs cover odd/even template selection
+
+## Measurement
+
+Minor visual impact; mostly affects header position consistency.
+Last measured: 2026-08-08.

data/TODO.complete/88-hyphenation-support.md

@@ -0,0 +1,52 @@
+---
+priority: P2
+impact: medium
+depends_on: [70, 79]
+layer: text
+status: pending
+est: 3d
+---
+
+## Problem
+
+Long words at line boundaries get pushed to the next line, leaving the
+previous line underfull. FOP uses TeX hyphenation patterns to break
+words at valid hyphenation points. Our KP breaker has no hyphenation.
+
+## Impact
+
+Visible on paragraphs with long technical terms:
+- "characteristics" → pushes to next line
+- "classification" → underfull line before
+- "metrological" → gap at line end
+
+## Approach
+
+1. **Choose a hyphenation gem.** Options:
+   - `text-hyphen` — pure Ruby, TeX patgen patterns
+   - `ruby-hyphen` — C extension, faster
+   - Inline TeX pattern data file (no dependency)
+
+2. **Integrate into ItemBuilder.** When a word exceeds remaining line
+   width, emit `Box + Penalty(flagged=true)` at each valid hyphenation
+   point. The KP breaker treats flagged penalties as valid break
+   opportunities with a demerit.
+
+3. **Language-aware patterns.** Use the document's `language` metadata
+   to select the correct pattern file (en, fr, de, etc.).
+
+4. **Minimum word length.** Don't hyphenate words shorter than 5 chars
+   (TeX default).
+
+## Done-When
+
+- [ ] Long words break at valid hyphenation points
+- [ ] Hyphen character (`-`) rendered at break point
+- [ ] Language detection from document metadata
+- [ ] No overfull lines from hyphenation
+- [ ] Specs cover hyphenation edge cases
+
+## Measurement
+
+Affects ~20-30 body paragraphs with long technical terms.
+Last measured: 2026-08-08.

data/TODO.complete/89-table-rowspan.md

@@ -0,0 +1,49 @@
+---
+priority: P1
+impact: high
+depends_on: [72]
+layer: flowable
+status: done
+est: 2d
+---
+
+## Problem
+
+Table cells with `rowspan` (cells spanning multiple rows) were not
+supported: covered cells shifted into the wrong columns (Table 5's
+"Influence factor" column rendered under the p_LC column) and tall
+table-cell text was crammed into one overfull line.
+
+## Solution (2026-08-17)
+
+- `Table::Grid` — occupancy grid over a Content::Table honoring
+  colspan AND rowspan. Answers cell placement (which column each
+  cell really occupies) and which rows are welded together by a
+  vertical span. Shared by AutoLayout (natural widths), TableFlowable
+  (row heights, placement, splitting).
+- `TableFlowable` places every cell at its grid column; a rowspan
+  cell is drawn once in its start row with the combined spanned-row
+  height, `valign: middle` centers its content (Table 5's "1.0").
+- Row heights: single-row cells drive their row; a spanning cell that
+  does not fit its span distributes the deficit equally over the
+  spanned rows.
+- `do_split` splits only at welded-group boundaries — a rowspan cell
+  is never cut by a page break. When the first group does not fit
+  the remainder, no head is returned and the whole table moves to
+  the next page (FOP semantics).
+- `Engine::Paged#place` propagates the page state after a
+  whole-flowable move (previously the caller kept filling the old
+  page and content fell out of document order).
+
+## Done-When
+
+- [x] `<td rowspan="2">` renders as a cell spanning 2 rows
+- [x] Subsequent rows skip cells covered by rowspan
+- [x] Cell content vertically centered in spanned area
+- [x] Specs cover rowspan + colspan combinations
+- [x] OIML Table 4 (rowspan header + colspan) and Table 5 (rowspan 9)
+      render correctly
+
+## Measurement
+
+Table 5's page similarity 38% → 86% (2026-08-17).

data/TODO.complete/90-mathml-stacked-layout.md

@@ -0,0 +1,65 @@
+---
+priority: P2
+impact: medium
+depends_on: [75]
+layer: flowable
+status: in progress
+est: 1d
+---
+
+## Problem
+
+MathML stacked layout (mfrac, msqrt) renders flat.
+
+## Measured nuance (2026-08-19)
+
+Do NOT expand line height for every line carrying sub/superscript
+runs: the reference keeps ordinary symbol sub/sups compact (the
+terminology region's E_min/D_max/v_min lines are single-height and
+were aligned). Only the parenthesized display-style expressions
+(e.g. 5.3.2's ": (0.3 <= pLC <= 0.8)," after a colon) lay out stacked
+- whole expression ~10pt below the text baseline, parens at the
+text level, box two lines tall. A global 1.9x on math lines was
+tested and overshoots the document by 2 pages (29 vs 28). The
+discriminator between compact sub/sup and stacked expressions is in
+the base XSL's stem handling (not the OIML extension) - find it
+before implementing.
+
+
+MathML fractions (`<mfrac>`) render as "numerator denominator" on one
+line instead of a stacked fraction. Other stacked elements (mroot,
+msqrt) also need vertical layout.
+
+## Current state
+
+- MathML integration: SHIPPED (plurimath/mml)
+- `ELEMENT_HANDLERS` registry: mi, mn, mo, msub, msup, msubsup, mfrac
+- 10 specs covering basic elements
+
+## Approach
+
+1. **Stacked fraction flowable.** Create a `MathFractionFlowable` that
+   renders numerator centered above denominator with a horizontal rule.
+
+2. **Vertical layout in renderer.** The renderer needs to handle
+   `Output::PlacedBox` with `data[:type] == :math_fraction`:
+   - Draw numerator at top
+   - Draw horizontal rule at middle
+   - Draw denominator at bottom
+
+3. **Mroot/msqrt.** Render radical sign with content.
+
+4. **Mtable.** Render simple matrices.
+
+## Done-When
+
+- [ ] `<mfrac>` renders as stacked fraction with dividing line
+- [ ] `<msqrt>` renders with radical sign
+- [ ] `<mroot>` renders with index
+- [ ] Math content vertically centered in line
+- [ ] Specs cover fraction, sqrt, root
+
+## Measurement
+
+Affects ~5 formula references in OIML body text.
+Last measured: 2026-08-08.

data/TODO.complete/91-parity-diff-diagnostics.md

@@ -0,0 +1,50 @@
+---
+priority: P1
+impact: high
+depends_on: [80]
+layer: harness
+status: in_progress
+est: 1d
+---
+
+## Problem
+
+The parity check reports overall similarity but doesn't show
+per-element breakdowns. When a page has low similarity, we can't tell
+if the issue is spacing, text content, image absence, or table layout.
+
+## Current state
+
+- `rake parity:check` renders and diffs — SHIPPED
+- Per-page similarity: SHIPPED
+- Per-element diff: NOT implemented
+
+## Approach
+
+1. **Visual diff overlay.** Generate a side-by-side PNG comparison
+   of each page (reference vs ours) with differences highlighted.
+
+2. **Text diff per page.** Extract text from both PDFs per page and
+   show unified diff (added/removed lines).
+
+3. **Element classification.** Tag each difference:
+   - `spacing` — same text, different position
+   - `content` — different text
+   - `missing` — element in reference but not in ours
+   - `extra` — element in ours but not in reference
+
+4. **Parity leaderboard.** Track parity % per commit so we can see
+   which changes helped or hurt.
+
+## Done-When
+
+- [ ] `rake parity:diff PAGE=5` shows detailed diff for page 5
+- [ ] Visual overlay PNGs generated to `tmp/parity_overlays/`
+- [ ] Text diff shows exact added/removed words
+- [ ] Element classification helps identify root cause
+- [ ] CI reports parity delta on each PR
+
+## Measurement
+
+Critical for iterating on parity. Without this, fixes are guesswork.
+Last measured: 2026-08-08.

data/TODO.complete/92-table-cell-height-parity.md

@@ -0,0 +1,66 @@
+---
+priority: P1
+impact: high
+depends_on: [89]
+layer: flowable
+status: done
+est: 2d
+---
+
+## Problem
+
+Tables rendered with incorrect cell heights and padding (rows 16pt vs
+the reference's 25.45pt pitch), table-cell footnotes inlined into the
+header cell (inflating the header and the column widths), SVG figures
+a third taller than the reference, and captions that could be
+orphaned at a page bottom. Tables spanned the wrong pages and shifted
+content on pages 19-27.
+
+## Root causes (measured against mn2pdf v2.55, 2026-08-17)
+
+1. The flavor XSL `table-row-style` sets `min-height: 8.3mm` — with
+   border/padding allowance the reference row pitch is 25.45pt. Our
+   rows had no floor (content + 4pt).
+2. `<fn>` inside a table cell is a TABLE FOOTNOTE in the FOP model:
+   only a superscript marker stays in the cell, the body renders
+   below the table. The adapter inlined the whole body into the cell.
+3. FOP sizes viewport-less SVGs (viewBox only) at CSS pixels → PDF
+   points (×72/96). We treated viewBox units as points.
+4. `do_split` ignored the caption and repeated header rows in the
+   head budget, over-filling the page.
+5. Table captions were separate flowables and could be orphaned.
+
+## Solution
+
+- Flavor-configurable table geometry in `flow_rules.yml`:
+  `table: { min_row_height, cell_padding, footnote_font_size }` —
+  OIML calibrated to 25.45 / 3.3 / 9.0 from the XSL + reference.
+- Adapter: table-cell `<fn>` elements contribute a superscript marker
+  run to the cell (selector `footnote_marker`) and their bodies to
+  `Content::Table#footnotes`; the inline run collector takes an
+  `exclude:` selector so footnote text never leaks into cells.
+- `TableFlowable` renders table footnotes below the last row (marker
+  + body, 9pt, 10pt gap) and carries them across splits to the part
+  holding the final rows.
+- Caption emitted inside `TableFlowable` (full caption on the head,
+  "Table N (continued)" on continuations) — never orphaned.
+- SVG user-unit dimensions converted px → pt (×0.75) in both the
+  adapter (inline SVG viewBox) and the flow builder (external SVG).
+- `Content::Table::Cell` gained `valign`; `align`/`valign` parsed
+  from the XML cell attributes.
+
+## Done-When
+
+- [x] Row pitch matches the reference (25.45pt floor from config)
+- [x] Table footnotes render below the table with an in-cell
+      superscript marker
+- [x] Figures render at FOP's px→pt scale
+- [x] Caption + header + first row group budgeted in do_split
+- [x] Continuation caption appears when a table spans pages
+- [x] Specs cover footnote extraction, geometry, viewBox scaling
+
+## Measurement
+
+Overall similarity 62.7% → 64.07% (2026-08-17). Table region:
+p19 34→61, p21 38→68, p22 42→61, p23 38→86.
+Remaining drift documented in TODO 96.

data/TODO.complete/93-figure-text-searchability.md

@@ -0,0 +1,49 @@
+---
+priority: P2
+impact: medium
+depends_on: [84]
+layer: render
+status: pending
+est: 3d
+---
+
+## Problem
+
+SVG figures rendered via rsvg-convert are rasterized to PNG. Text
+inside the figures (labels, annotations) becomes pixels and is not
+searchable in the PDF. The reference mn2pdf/FOP renders SVG as vector
+content, preserving text searchability.
+
+This accounts for ~1,500 characters of the remaining text content gap
+(53,472 vs 55,534 reference). Figure labels like "Peripheral devices",
+"Keyboard to operate", "Maximum capacity Emax", etc. are visible in
+the reference's pdftotext output but absent from ours.
+
+## Approach
+
+1. **Vector SVG embedding.** Instead of rasterizing via rsvg-convert,
+   embed the SVG as a PDF XObject (Form XObject). This preserves text
+   as actual PDF text operators.
+
+2. **Text overlay.** Alternatively, rasterize the SVG for visual
+   fidelity but add an invisible text layer with the SVG's text
+   content. The text layer is marked as invisible but searchable.
+
+3. **SVG text extraction.** Parse the SVG XML to extract all `<text>`
+   elements with their positions. Use these to create searchable text
+   in the PDF at the corresponding coordinates.
+
+Option 3 is the most pragmatic: it doesn't require vector rendering
+support but makes figure text searchable.
+
+## Done-When
+
+- [ ] Figure text content appears in pdftotext output
+- [ ] Text positions match visual figure positions
+- [ ] No visual regression in figure rendering
+- [ ] Text content gap reduced from ~2,000 to <500 chars
+
+## Measurement
+
+`pdftotext` char count comparison. Currently 53,472 vs 55,534.
+Last measured: 2026-08-09.

data/TODO.complete/94-fo-space-resolution.md

@@ -0,0 +1,63 @@
+---
+priority: P0
+impact: critical
+depends_on: [81]
+layer: engine
+status: done
+est: 2d
+---
+
+## Problem
+
+Our engine ADDS consecutive flowable margins (space_after of previous
++ space_before of current). FOP/FO uses space RESOLUTION: takes the
+MAX of the two, not the sum. This causes every spacing change to
+cascade incorrectly.
+
+## Evidence
+
+The reference PDF has ~8pt paragraph spacing between consecutive body
+paragraphs (measured from bbox: consecutive paragraph lines are 21.2pt
+apart vs 13.2pt normal line height = 8pt extra).
+
+Adding margin_bottom: 8 to the body style caused page explosion (27 →
+33 pages) because the margin cascaded to ALL child styles and ALL
+paragraphs (including those inside notes, lists, terms).
+
+## Approach
+
+1. **Implement FO space resolution in Engine::Paged.** When placing
+   flowable B after flowable A, the gap should be:
+   ```
+   gap = max(A.space_after, B.space_before)
+   ```
+   Not `A.space_after + B.space_before`.
+
+2. **Track previous flowable's space_after.** The engine needs to know
+   the previous flowable's margin_bottom to resolve the space.
+
+3. **Apply to section-level flowables only.** Flowables inside
+   containers (NoteFlowable, ListFlowable) handle their own internal
+   spacing. Space resolution applies between sibling flowables in the
+   main flow.
+
+## Impact
+
+With FO space resolution + body margin_bottom: 8:
+- Standalone body paragraphs get 8pt gap (correct)
+- Note body paragraphs still get 0 gap (correct — notes have their
+  own margin_bottom on the container)
+- Heading-to-body gap: max(heading.space_after, body.space_before) =
+  max(12, 0) = 12pt (correct, no double-counting)
+
+## Done-When
+
+- [ ] Engine resolves consecutive flowable spaces via max()
+- [ ] body margin_bottom: 8 produces correct inter-paragraph spacing
+- [ ] No cascade to note/list/term internal paragraphs
+- [ ] Overall parity > 58%
+
+## Measurement
+
+`bundle exec rake parity:check` — currently 53.99%.
+Last measured: 2026-08-10.

data/TODO.complete/95-title-header-placement.md

@@ -0,0 +1,55 @@
+---
+priority: P1
+impact: high
+depends_on: [71]
+layer: engine
+status: done
+est: 2d
+---
+
+## Problem
+
+The document title block ("Part 1 - Metrological and technical
+requirements") is rendered as BODY flow content, consuming ~58pt of
+body region space on page 5. The reference (mn2pdf/FOP) renders it
+in the HEADER area above the body region, getting it "for free".
+
+## Evidence
+
+Reference page 5:
+- Title at y=71-95 (ABOVE body region)
+- "1 Introduction" at y=130 (body region start)
+
+Our page 5:
+- Title at y=126-144 (INSIDE body region)
+- "1 Introduction" at y=183 (pushed down by title)
+
+Net effect: our page 5 has ~58pt less content than reference. Over
+the document, this accumulates to ~1 page of offset.
+
+## Approach
+
+1. **Static content per page sequence.** The PageSequenceStart
+   flowable should support a `title_template` that renders in the
+   header region on the first page of the sequence.
+
+2. **Renderer support.** The renderer draws the title at a fixed
+   position in the header area (y = margin_top to body_top) when the
+   page is the first of a sequence with a title template.
+
+3. **Flow builder change.** `append_title_block` should NOT emit a
+   body TextFlowable. Instead, pass the title to the
+   PageSequenceStart flowable.
+
+## Done-When
+
+- [ ] Title renders above body region (y < 126 on A4 with 26.5mm/18mm)
+- [ ] Body content starts at same y as reference (y=130 for heading)
+- [ ] Page 5 contains Introduction + section 2 content (matching ref)
+- [ ] Overall parity > 65%
+
+## Measurement
+
+`bundle exec rake parity:check` — currently 62.7%. Page 5: 99.6%,
+page 6: 66.7%, page 7: 50.0% (mostly empty due to figure push).
+Last measured: 2026-08-16.