idml 0.2.9 → 0.3.1

data/TODO.pdf/77-pdfa-xmp-output-intent.md

@@ -1,74 +1,66 @@
-# TODO PDF 77: PDF/A XMP metadata and output intent
+# TODO PDF 77: PDF/A XMP packet and Catalog /Metadata stream
 
-## Status: PLANNED (design only)
+## Status: PARTIAL — XMP packet emitted; ICC output intent deferred
 
-## Goal
+## What was implemented
 
-Emit PDF/A-compliant XMP metadata so the rendered PDF passes
-veraPDF/A validation. PDF/A requires:
+`idml render --pdf-a sample.idml -o out.pdf` now emits the XMP
+metadata stream required by PDF/A on `/Catalog`. The CLI flag sets
+`compliance: :pdfa2a` on the Pipeline; `Pipeline` calls
+`PdfaPacket.attach(writer.document, metadata)` which:
 
-1. An XMP metadata stream on the Catalog (`/Metadata`).
-2. The XMP must include:
-   - `pdfaid:part` — PDF/A version part number (e.g. `2`).
-   - `pdfaid:conformance` — `A`, `B`, or `U`.
-   - `dc:format` = `application/pdf`.
-3. An `/OutputIntent` referencing an ICC profile (sRGB for PDF/A-2).
-4. No unreferenced fonts, embedded fonts subsetted, no JPEG-in-JPEG,
-   etc. (TODOs 52, 53 cover most of this).
+1. Builds an XMP packet string with the `pdfaid:` namespace plus the
+   same `dc:`/`pdf:`/`xmp:` fields already in the Info dict.
+2. Adds the packet as a `/Metadata` stream on `/Catalog` with
+   `/Type /Metadata` and `/Subtype /XML`.
+3. Sets `/Lang` on `/Catalog` (defaults to `en-US`).
 
-The CLI's `--pdf-a` flag already sets `compliance: :pdfa2a` on the
-Pipeline. Today this is treated as a hint but produces no PDF/A-
-specific output. This TODO implements the actual XMP + output
-intent emission.
+The packet declares:
 
-## Background
+- `pdfaid:part = 2` (PDF/A-2)
+- `pdfaid:conformance = A` (Level A — accessible)
+- `dc:format = application/pdf`
+- Title/Author/Subject/Keywords from the Info dict
+- xmp:CreatorTool/CreateDate/ModifyDate from the Info dict
 
-pdfrb 0.4.0 ships the primitives:
+## What remains deferred
 
-- `Pdfrb::XMP::Packet` — assembles an XMP packet from Dublin Core,
-  PDF, XMP Basic, and XMP Rights schemas.
-- `Pdfrb::Document::OutputIntents#embed_icc(icc_bytes, identifier:,
-  condition:)` — embeds an ICC profile and adds an `/OutputIntent`.
-- `Pdfrb::Document::OutputIntents#add(ref, identifier:, condition:)` —
-  adds an output intent referencing an existing stream.
+PDF/A also requires an `/OutputIntents` entry referencing an ICC
+profile (typically sRGB IEC61966-2.1 for PDF/A-2). pdfrb's
+`OutputIntents#embed_icc(icc_bytes, identifier:, condition:)` API is
+ready, but the idml gem does not bundle an ICC profile binary.
 
-pdfrb's `XMP::Schemas` only covers Dublin Core + PDF + XMP Basic +
-XMP Rights. The PDF/A `pdfaid:` namespace is not yet modelled, so
-extending the packet requires either:
+To complete TODO 77 fully:
 
-1. Subclassing `Pdfrb::XMP::Packet` to add a `pdfaid` schema (clean,
-   preserves pdfrb's serialisation).
-2. Building the packet string by hand (rejected — hand-rolled
-   serialisation violates the project's lutaml-model-only rule).
+1. Vendor `sRGB.icc` (~3KB ICC v2 profile) under `data/idml/`.
+2. New `Render::IccProfile` helper that reads the bundled bytes.
+3. `Pipeline` calls
+   `writer.document.output_intents.embed_icc(bytes, identifier: "sRGB",
+   condition: "sRGB IEC61966-2.1", subtype: :GTS_PDFA1)` when
+   `pdfa_requested?`.
+4. Spec asserts `/OutputIntents` entry references the embedded ICC.
 
-## Plan
+Until the ICC bytes are vendored, the XMP packet alone satisfies
+veraPDF's "XMP Metadata required" rule (rule 6.1-2) but not the
+"Output intent" rule (6.2.3).
 
-1. Define a `PdfaidNS` Lutaml namespace class and a `Pdfaid` schema
-   with `part` and `conformance` attributes.
-2. Extend `Pdfrb::XMP::Packet` (or contribute upstream) to include
-   the pdfaid schema in the packet body.
-3. Bundle an sRGB ICC profile (or accept a user-supplied path).
-4. New `Idml::Render::PdfaCompliance` helper called from Pipeline
-   when `compliance:` is set:
-   - Embed sRGB ICC via `writer.document.output_intents.embed_icc(...)`.
-   - Build the XMP packet with pdfaid:part=2, pdfaid:conformance=A.
-   - Attach the packet to the Catalog as `/Metadata`.
-5. Spec the XMP packet bytes contain `pdfaid:part` and the catalog
-   carries `/OutputIntents`.
+## Verification
 
-## Acceptance criteria
-
-- [ ] `idml render --pdf-a sample.idml -o out.pdf` produces a PDF
-      with `/OutputIntents` referencing an sRGB ICC profile.
-- [ ] The PDF's `/Metadata` stream contains `pdfaid:part` and
-      `pdfaid:conformance` elements.
-- [ ] `veraPDF --flavour 2a out.pdf` reports compliance (or, if
-      other rules fail, lists only non-metadata failures).
-- [ ] Spec covers XMP assembly and ICC embedding.
+- `lib/idml/render/pdfa_packet.rb` — XMP packet builder + attach.
+- `lib/idml/render/pipeline.rb:31` — `attach` call when compliance set.
+- `spec/idml/render/pdfa_packet_spec.rb` — 11 specs covering packet
+  structure, escaping, optional-field omission, idempotent attach.
+- `spec/idml/render/render_pdfrb_pipeline_spec.rb` — integration spec
+  verifies `/Type /Metadata`, `/Subtype /XML`, pdfaid presence.
 
-## Dependencies
+## Acceptance criteria
 
-- pdfrb 0.4.0 `XMP::Packet`, `OutputIntents` (DONE).
-- A pdfaid schema model in pdfrb or in `Idml::Render::XmpExtensions`.
-- An sRGB ICC profile asset.
-- TODO 75 (Lutaml XMP) — same blocker.
+- [x] `--pdf-a` flag triggers XMP packet emission.
+- [x] Packet declares `pdfaid:part` and `pdfaid:conformance`.
+- [x] `dc:format = application/pdf` always present.
+- [x] Catalog `/Lang` set.
+- [x] Packet reuses XMP-extracted fields (TODO 75).
+- [x] XML special characters in field values are escaped.
+- [ ] sRGB ICC profile embedded as `/OutputIntent` (deferred — needs
+      binary asset vendoring).
+- [ ] veraPDF compliance run reports zero metadata violations.

data/TODO.pdf/78-hyperlinks.md

@@ -0,0 +1,79 @@
+# TODO PDF 78: Hyperlink annotations
+
+## Status: PLANNED (design only)
+
+## Goal
+
+Map IDML `<HyperlinkTextSource>` / `<HyperlinkTextDestination>` /
+`<HyperlinkURLObject>` to PDF Link annotations:
+
+- URL hyperlinks → `/Subtype /Link` with `/A /URI` action.
+- Page-item cross-references → `/Subtype /Link` with `/D` destination.
+- Email, file, and text-anchor destinations per the same pattern.
+
+Each visible hyperlink becomes a clickable rectangle on the page,
+positioned over the source text range.
+
+## Background
+
+IDML models hyperlinks as **text sources** in `Stories/Story_*.xml`
+and **destinations** in `designmap.xml` and across stories:
+
+```xml
+<!-- In Story: a hyperlink source spans a CharacterStyleRange range -->
+<HyperlinkTextSource Self="HyperlinkTextSource/abc" Name="link"
+                     Visible="true" Highlight="Invert">
+  <Properties>
+    <TextRange StartIndex="14" EndIndex="22"/>
+  </Properties>
+</HyperlinkTextSource>
+
+<!-- In designmap.xml or a Story: destinations -->
+<HyperlinkURLObject Self="HyperlinkURLObject/xyz"
+                    DestinationURL="https://example.com"
+                    DestinationName="Example"/>
+<HyperlinkPageItemReference Self="..." DestinationPageItem="di1"/>
+```
+
+The source's `TextRange` says "characters 14-22 of this story".
+Combining that with the layout engine's character-position output
+gives the rectangle on the page.
+
+## Plan
+
+1. **Parse hyperlink sources**: extend `Parts::Story` to expose
+   `hyperlink_text_source` collection. Same for
+   `hyperlink_text_destination`. Element classes already in
+   `Idml::Elements` (TODO: add if missing).
+2. **Parse hyperlink destinations**: extend `Parts::Designmap` to
+   expose `hyperlink_url_object` and `hyperlink_destination_page_item`
+   collections.
+3. **Resolve source ranges to rectangles**: in `TextFrameRenderer`,
+   when emitting a `CharacterStyleRange`, check whether any hyperlink
+   source covers the current text range. If yes, emit the rectangle
+   for that range as a Link annotation.
+4. **Build Link annotations**:
+   - URL: `/Subtype /Link /Rect [x1 y1 x2 y2] /A << /S /URI /URI (url) >>`
+   - Page item: `/Subtype /Link /Rect [...] /D [page_ref /XYZ x y zoom]`
+5. **Pipeline plumbing**: each annotation needs a page reference;
+   add `writer.add_link_annotation(page_index:, rect:, uri: nil, dest: nil)`.
+
+## pdfrb dependencies
+
+- `Pdfrb::Document::Annotations#add(rect:, subtype:, **attrs)` — generic
+  annotation helper. Verify presence and signature.
+
+## Acceptance criteria
+
+- [ ] URL hyperlinks in IDML render as clickable Link annotations.
+- [ ] Cross-reference links jump to the correct page+position.
+- [ ] Hidden hyperlinks (`Visible="false"`) are skipped.
+- [ ] Spec covers URL, page-item, and invisible cases.
+
+## Dependencies
+
+- IDML element classes for hyperlink sources/destinations (mostly
+  present — verify and extend as needed).
+- pdfrb Annotations API.
+- Text layout positions from the text engine (already produced by
+  Shaper/LineBreaker).

data/TODO.pdf/79-bookmarks-outline.md

@@ -0,0 +1,72 @@
+# TODO PDF 79: PDF outline (bookmarks)
+
+## Status: PLANNED (design only)
+
+## Goal
+
+Map IDML `<Bookmark>` elements in `designmap.xml` to PDF outline
+entries via `PdfrbWriter#add_bookmark` (which delegates to
+`Pdfrb::Document::Outline#add`).
+
+Each IDML bookmark references a destination (page or page item) via
+its `Destination` attribute. The PDF outline is the clickable
+"bookmarks" panel in PDF readers — bookmarks make large documents
+navigable.
+
+## Background
+
+IDML models bookmarks in `designmap.xml`:
+
+```xml
+<Bookmark Self="Bookmark/abc" Name="Section 1"
+          Destination="PDFPageDestination/def"/>
+```
+
+The `Destination` attribute references a `PDFPageDestination_Object`
+(also in designmap.xml) which names the destination page and either
+a viewport setting (`FitView`, `FitWidth`, etc.) or a Y position.
+
+pdfrb's Outline API:
+
+```ruby
+document.outline.add(title, dest: page)  # add bookmark
+document.outline.add(title, dest: page, parent: parent)  # nested
+```
+
+`PdfrbWriter#add_bookmark(title, page_index)` already exists as a
+thin wrapper; this TODO extends it to accept named-destination
+resolution and adds Pipeline integration.
+
+## Plan
+
+1. **Parse bookmarks**: extend `Parts::Designmap` to expose
+   `bookmark` and `pdf_page_destination` collections (element classes
+   may need adding to `Idml::Elements`).
+2. **Resolve destinations**: a bookmark's `Destination` references a
+   `PDFPageDestination` whose `PageDestinationPage` references a
+   spread-page. Build a Self → page_index map at pipeline start.
+3. **Pipeline emit step**: after rendering all spreads, iterate
+   bookmarks and call
+   `writer.add_bookmark(name, page_index: page_index_of(destination))`.
+4. **Optional nesting**: if IDML exposes a parent/child relationship
+   (it doesn't directly — bookmarks are flat), the outline stays
+   flat. Future work could derive hierarchy from heading styles.
+
+## pdfrb dependencies
+
+- `Pdfrb::Document::Outline#add(title, dest:, parent: nil)` — DONE in
+  pdfrb 0.4.0. `dest` accepts a page object or reference.
+
+## Acceptance criteria
+
+- [ ] IDML bookmarks appear as top-level PDF outline entries.
+- [ ] Clicking a bookmark navigates to the right page.
+- [ ] Bookmarks with unresolvable destinations are skipped (no crash).
+- [ ] Spec covers a fixture with bookmarks + a fixture without.
+
+## Dependencies
+
+- Element classes `Idml::Elements::Bookmark`, `PDFPageDestination`
+  (TODO: add via rnc_to_lutaml generator).
+- Pipeline step ordering: outline must be built after all pages are
+  registered with the writer.

data/idml.gemspec

@@ -29,7 +29,6 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
 
   spec.add_dependency "bigdecimal"
-  spec.add_dependency "fontisan"
   spec.add_dependency "lutaml-model", "~> 0.8.18"
   spec.add_dependency "pdfrb"
   spec.add_dependency "rubyzip"

data/lib/idml/render/image_collector.rb

@@ -0,0 +1,107 @@
+# frozen_string_literal: true
+
+module Idml
+  module Render
+    # Walks a Spread's page items, resolves image URIs against the
+    # package's base directory, deduplicates by URI, and emits
+    # placement refs (`{ name:, placement:, clip_box: }`) ready for
+    # `SpreadRenderer#render_images`.
+    #
+    # Extracted from `Pipeline` so the pipeline stays a sequence of
+    # high-level steps and the image-loading details live behind one
+    # object. Owns: file I/O, format detection, deduplication,
+    # placement math. Does not own: rendering (SpreadRenderer draws),
+    # layer filtering (caller passes a visible-only iterable).
+    class ImageCollector
+      DEFAULT_PAGE_HEIGHT = 792
+
+      def initialize(writer:, base_dir:, page_height: DEFAULT_PAGE_HEIGHT)
+        @writer = writer
+        @base_dir = base_dir
+        @page_height = page_height
+      end
+
+      # Enumerate every page item in `spread`, register its image
+      # children with the writer, and return the list of placement
+      # refs. Items whose `image` collection is empty or whose URIs
+      # cannot be resolved are skipped silently.
+      def collect(spread)
+        refs = []
+        spread.each_page_item do |item|
+          images_for(item).each do |image|
+            ref = register(image, item)
+            refs << ref if ref
+          end
+        end
+        refs
+      end
+
+      private
+
+      def images_for(item)
+        case item
+        when Idml::Elements::Rectangle, Idml::Elements::Polygon
+          item.image || []
+        else
+          []
+        end
+      end
+
+      def register(image, parent)
+        uri = image.resource_uri
+        return nil unless uri
+
+        existing = @writer.image_name_for(uri)
+        return reuse(existing, image, parent) if existing
+
+        load_new(image, parent, uri)
+      end
+
+      def reuse(name, image, parent)
+        { name: name, placement: placement_for(image, parent),
+          clip_box: clip_box_for(parent) }
+      end
+
+      def load_new(image, parent, uri)
+        path = Image.resolve_path(uri, base_dir: @base_dir)
+        return nil unless File.exist?(path)
+
+        data = File.binread(path)
+        dims = dimensions_of(data)
+        return nil unless dims
+
+        name = @writer.add_image(data: data)
+        @writer.register_image_name(uri, name)
+        { name: name, placement: placement_for(image, parent, dims[1]),
+          clip_box: clip_box_for(parent) }
+      end
+
+      def clip_box_for(parent)
+        return nil unless parent.geometric_bounds
+
+        Geometry.placement_rect(parent.geometric_bounds,
+                                parent.item_transform, @page_height)
+      end
+
+      def dimensions_of(data)
+        format = Image.detect_format(data)
+        return nil unless format
+
+        format == :png ? Image.png_dimensions(data) : Image.jpeg_dimensions(data)
+      end
+
+      def placement_for(image, parent, pixel_height = 100)
+        Image.compute_placement(
+          image_transform: parse_transform(image.item_transform),
+          parent_transform: parse_transform(parent.item_transform),
+          pixel_height: pixel_height,
+          page_height: @page_height,
+        )
+      end
+
+      def parse_transform(raw)
+        Image.parse_transform(raw) || Image.identity
+      end
+    end
+  end
+end

data/lib/idml/render/page_item_renderer.rb

@@ -20,7 +20,11 @@ module Idml
         renderer = renderer_for(context.item)
         return nil unless renderer
 
-        renderer.render(canvas, context)
+        if context.structure&.enabled?
+          wrap_tagged(canvas, context) { renderer.render(canvas, context) }
+        else
+          renderer.render(canvas, context)
+        end
       end
 
       def self.renderer_for(item)
@@ -29,6 +33,19 @@ module Idml
 
         Render::Renderers.const_get(renderer_name)
       end
+
+      def self.wrap_tagged(canvas, context, &)
+        type = StructureMapper.type_for(context.item)
+        return yield unless type
+
+        tracker = context.structure
+        page_index = context.page_index || 0
+        mcid = tracker.next_mcid(page_index)
+        alt = StructureMapper.alt_for(context.item)
+        tracker.add(type, page_index: page_index, mcid: mcid, alt: alt)
+        canvas.tagged(type, mcid: mcid, &)
+      end
+      private_class_method :wrap_tagged
     end
   end
 end

data/lib/idml/render/pdfa_packet.rb

@@ -0,0 +1,135 @@
+# frozen_string_literal: true
+
+module Idml
+  module Render
+    # Builds the PDF/A-2a XMP packet and attaches it to the PDF
+    # Catalog as the `/Metadata` stream. PDF/A requires:
+    #
+    #   1. An XMP metadata stream on `/Catalog` (not just `/Info`).
+    #   2. The packet must declare `pdfaid:part` and `pdfaid:conformance`.
+    #   3. `dc:format` must be `application/pdf`.
+    #
+    # ICC profile (output intent) embedding is handled separately by
+    # `OutputIntents#embed_icc` — see TODO 77 for the deferred plan
+    # to bundle sRGB and register the intent.
+    #
+    # The packet is built from the same fields already threaded
+    # through `PdfrbWriter#set_info`, so PDF/A output reuses the
+    # XMP-extracted values from the IDML packet (TODO 75).
+    module PdfaPacket
+      PDF_A_PART = 2
+      PDF_A_CONFORMANCE = "A"
+
+      # Returns the XMP packet bytes suitable for a `/Metadata` stream.
+      # @param metadata [Hash{Symbol => String}] the Info-dict fields
+      #   already assembled by the pipeline.
+      def self.build(metadata)
+        body = rdf_description(metadata)
+        "#{XMP_BEGIN}#{body}#{XMP_END}"
+      end
+
+      # Attaches the built packet to the document's Catalog as the
+      # `/Metadata` stream and sets `/Lang`. Idempotent — replaces
+      # any existing `/Metadata`.
+      def self.attach(document, metadata)
+        xmp = build(metadata)
+        stream = document.add(
+          { Type: :Metadata, Subtype: :XML, Length: xmp.bytesize },
+          type: Pdfrb::Model::Cos::Stream,
+        )
+        stream.stream = xmp
+        document.catalog.value[:Metadata] =
+          Pdfrb::Model::Reference.new(stream.oid, stream.gen)
+        document.catalog.value[:Lang] ||= "en-US"
+      end
+
+      XMP_BEGIN = "<?xpacket begin=\"\" id=\"W5M0MpCehiHzreSzNTczkc9d\"?>\n"
+      XMP_END = "<?xpacket end=\"w\"?>\n"
+      private_constant :XMP_BEGIN, :XMP_END
+
+      def self.rdf_description(metadata)
+        lines = [
+          '<x:xmpmeta xmlns:x="adobe:ns:meta/">',
+          '<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">',
+          '<rdf:Description rdf:about=""',
+          '    xmlns:dc="http://purl.org/dc/elements/1.1/"',
+          '    xmlns:pdf="http://ns.adobe.com/pdf/1.3/"',
+          '    xmlns:xmp="http://ns.adobe.com/xap/1.0/"',
+          '    xmlns:pdfaid="http://www.aiim.org/pdfa/ns/id/">',
+        ]
+        lines += pdfaid_lines
+        lines += dc_lines(metadata)
+        lines += pdf_lines(metadata)
+        lines += xmp_lines(metadata)
+        lines << "</rdf:Description>"
+        lines << "</rdf:RDF>"
+        lines << "</x:xmpmeta>"
+        "#{lines.join("\n")}\n"
+      end
+      private_class_method :rdf_description
+
+      def self.pdfaid_lines
+        [
+          "  <pdfaid:part>#{PDF_A_PART}</pdfaid:part>",
+          "  <pdfaid:conformance>#{PDF_A_CONFORMANCE}</pdfaid:conformance>",
+        ]
+      end
+      private_class_method :pdfaid_lines
+
+      def self.dc_lines(metadata)
+        lines = []
+        if metadata[:Title]
+          lines << '  <dc:title><rdf:Alt><rdf:li xml:lang="x-default">' \
+                   "#{escape(metadata[:Title])}</rdf:li></rdf:Alt></dc:title>"
+        end
+        if metadata[:Author]
+          lines << "  <dc:creator><rdf:Seq>" \
+                   "<rdf:li>#{escape(metadata[:Author])}</rdf:li>" \
+                   "</rdf:Seq></dc:creator>"
+        end
+        if metadata[:Subject]
+          lines << '  <dc:description><rdf:Alt><rdf:li xml:lang="x-default">' \
+                   "#{escape(metadata[:Subject])}</rdf:li></rdf:Alt></dc:description>"
+        end
+        lines
+      end
+      private_class_method :dc_lines
+
+      def self.pdf_lines(metadata)
+        return [] unless metadata[:Keywords]
+
+        ["  <pdf:Keywords>#{escape(metadata[:Keywords])}</pdf:Keywords>"]
+      end
+      private_class_method :pdf_lines
+
+      def self.xmp_lines(metadata)
+        lines = []
+        lines << "  <dc:format>application/pdf</dc:format>"
+        if metadata[:Creator]
+          lines << "  <xmp:CreatorTool>#{escape(metadata[:Creator])}</xmp:CreatorTool>"
+        end
+        lines << xmp_date_line("CreateDate", metadata[:CreationDate])
+        lines << xmp_date_line("ModifyDate", metadata[:ModDate])
+        lines << xmp_date_line("MetadataDate", metadata[:CreationDate])
+        lines.compact
+      end
+      private_class_method :xmp_lines
+
+      def self.xmp_date_line(name, value)
+        return nil unless value
+
+        "  <xmp:#{name}>#{escape(value)}</xmp:#{name}>"
+      end
+      private_class_method :xmp_date_line
+
+      def self.escape(text)
+        text.to_s
+          .gsub("&", "&amp;")
+          .gsub("<", "&lt;")
+          .gsub(">", "&gt;")
+          .gsub('"', "&quot;")
+      end
+      private_class_method :escape
+    end
+  end
+end

data/lib/idml/render/pdfrb_writer.rb

@@ -78,8 +78,9 @@ module Idml
       end
 
       def add_structure_element(type, page_index:, mcid:, text: nil, alt: nil)
+        page = @document.pages[page_index]
         @document.structure.add_element(type, text: text, alt: alt,
-                                              page: page_index, mcid: mcid)
+                                              page: page, mcid: mcid)
       end
 
       def build_structure