idml 0.4.8 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bfe99408096a550d69277aae9bf709c0dcb0257f17756fcef498c9ea3982a603
-  data.tar.gz: 83fc58f40dccb82324b6098aab6665765cd0e1a08b12ade7e08f6d600c6e69c3
+  metadata.gz: '09c5f631233897e7f1e18519b1031c1c87092a1e3ddffdc353bf2e121c47b15b'
+  data.tar.gz: 8c0b965da67d83e30f3b7002be6bbabe1dffec7d1a3557979fe4cdfbce99dd54
 SHA512:
-  metadata.gz: 8b9646bd611f3618e6a7c3c689a402b368b231f66f6f06430098f9357179d4b79ae0884058b20c92df03dce6e65aa7cf74c245bb8100a772a9454c37f310b4bd
-  data.tar.gz: 7b639edc906f60c9e82128bf599966cd37e2fb0458e06e62158d14767d2293d918bed5fe75163cd2caa2cb20fed7174a0c18a709ac722a673b9099532e8f5b33
+  metadata.gz: 7675a8a5b799b1a2b420f74b26faec4f6630b55cbf7ca831201085f99947d9ee31d1380fe5a12039427e643988ec84228738fc92dd9d925423aa36822815a774
+  data.tar.gz: 266b6c589b715d727460eba5064dde878f104b079e3da2ae7a0b31c911492cd88d75d18946ff3536d358c12359bf4b4069592dba7ac0c4b5eb42a1dbebde0544

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    idml (0.4.8)
+    idml (0.5.0)
       bigdecimal
       lutaml-model (~> 0.8.18)
       pdfrb
@@ -183,7 +183,7 @@ CHECKSUMS
   ffi (1.17.4-x86_64-darwin) sha256=aa70390523cf3235096cf64962b709b4cfbd5c082a2cb2ae714eb0fe2ccda496
   ffi (1.17.4-x86_64-linux-gnu) sha256=9d3db14c2eae074b382fa9c083fe95aec6e0a1451da249eab096c34002bc752d
   ffi (1.17.4-x86_64-linux-musl) sha256=3fdf9888483de005f8ef8d1cf2d3b20d86626af206cbf780f6a6a12439a9c49e
-  idml (0.4.8)
+  idml (0.5.0)
   json (2.21.2) sha256=1f1d3b7cf2b3ba1a69beca0bb6db13d5438b80bff3cd54cdaaa620b9b07c1c6a
   language_server-protocol (3.17.0.6) sha256=5ef2c0c138f8267e1bc631d3328347d354f96724b0af22f2c79516120443b7f0
   lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87

data/TODO.pdf/89-deep-audit-idml-vs-indesign.md

@@ -0,0 +1,149 @@
+# TODO PDF 89: Deep audit — idml-generated PDF vs InDesign PDF
+
+## Status: PARTIAL — font selection fixed; image bloat remains
+
+## TL;DR
+
+**No** — the generated PDF is still not identical to InDesign's
+output, but the **font selection gap is closed** as of v0.5.0.
+
+After the font fix (FontSetup now prefers Regular weight over
+first-in-family):
+- Font: `MinionPro-Regular` (matches InDesign) — was
+  `MinionPro-BoldCn`.
+- File size: still ~2.2MB (the image bloat remains).
+
+The remaining 88% of the file size is the embedded GenAI JPEG
+(1,937,556 bytes embedded raw via DCTDecode). InDesign downsamples
+to ~12KB. Closing this gap requires pure-Ruby JPEG decode + resize
++ re-encode (TODO 91).
+
+## File-level comparison (`sample-with-table-more`)
+
+| Aspect | InDesign pages.pdf | idml (ours) |
+|---|---|---|
+| Size | 100,965 bytes | 2,196,913 bytes |
+| Pages | 4 | 4 |
+| PDF version | 1.4 | 1.4 |
+| Embedded fonts | 1 (subsetted `MinionPro-Regular`) | 1 (full `MinionPro-BoldCn`) |
+| Font stream size | 8,346 bytes (subsetted) | 232,680 bytes (full) |
+| Image streams | 4 (small, downsampled) | 1 (huge, full-res) |
+| Image stream sizes | 11,773 + 1,731 + 4,291 + ... | 1,937,556 (single 1.94MB JPEG) |
+| `/OutputIntents` | embedded | NOT embedded (even with `--pdf-a`) |
+| `/StructTreeRoot` | NOT emitted | only with `--tagged` |
+| `/OCG` layers | NOT emitted | NOT emitted |
+| `/Metadata` XMP | NOT emitted | emitted (XMP extracted from IDML) |
+| OCG/InDesign structure | rich (`PieceInfo`, `PageUIDList`) | absent |
+| Compression (FlateDecode) | yes | absent on most streams |
+
+## The 1.94MB bloat
+
+`/Users/mulgogi/Downloads/2.png` is a 600×400 RGBA PNG (278KB on
+disk, link-resource size 43,765 bytes per IDML metadata). The idml
+renderer reads the file at the path IDML specifies and re-encodes
+it through pdfrb's image pipeline as a 2048×2048 JPEG (1.94MB).
+
+InDesign's PDF re-encodes at 300ppi: a 4.7"×3.1" image at 300ppi
+is 1410×930, which JPEG-compresses to ~12KB. InDesign's source
+asset (43KB) is downsampled then DCT-encoded at low quality for
+embed. The idml render doesn't apply downsampling — it just
+re-encodes the source file.
+
+## Font selection discrepancy
+
+The idml render uses `FontSetup#register` which picks the first
+non-missing font in `package.fonts.font_family[0]` — that's
+`MinionPro-BoldCn` (the first font in "Minion Pro" family).
+
+InDesign's PDF embeds `MinionPro-Regular` — that's because each
+text run is rendered with the font declared in its
+CharacterStyleRange#AppliedFont, which can differ per run.
+
+The idml render's `StyleResolver::StyledRun` doesn't currently
+carry `applied_font` (TODO: extract from CSR#AppliedFont). Once
+extracted, the TextFrameRenderer can resolve per-run fonts via
+`FontSetup#register_resolved` and register each in pdfrb.
+
+## Font subsetting
+
+`Pdfrb::Font::TrueType::Subsetter` exists in pdfrb 0.5.0 and works
+when called. Pipeline calls `writer.subset_fonts!` when
+`subset_fonts: true`. The resulting file still embeds a ~233KB
+font stream — investigation needed to determine why
+subsetting isn't shrinking the file.
+
+Likely cause: the `MinionPro-BoldCn` font we register isn't being
+used to draw any text (no codepoints were used through it during
+rendering), so subsetting doesn't apply. But the full font file is
+still embedded because we registered it.
+
+Fix: register only the font(s) that actually carry used codepoints.
+
+## Image handling
+
+The idml render walks `<Image>` elements via `ImageCollector`,
+reading the file at `LinkResourceURI` and re-encoding as JPEG.
+`Image.compute_placement` uses the image's `ItemTransform` for
+sizing.
+
+InDesign's PDF embeds the image at downsampled resolution (300ppi)
+and uses Smask for transparency. The idml render doesn't have an
+Smask path.
+
+## Structure and metadata
+
+InDesign's PDF embeds `PieceInfo`, `PageUIDList`,
+`OriginalDocumentID`, `LastModified` etc. — these are
+InDesign-specific structure for round-tripping. The idml render
+doesn't emit any of this (it's not part of the PDF spec, but
+IDML-aware tools can read it).
+
+The idml render's PDF has `/Metadata` XMP (extracted from
+`META-INF/metadata.xml`) — InDesign's reference PDF does not have
+it. Adding XMP is a feature, not a bug.
+
+## Visual fidelity
+
+- **Table layout**: After v0.4.8, the idml render uses
+  `Row#single_row_height` for vertical sizing. InDesign uses
+  this too. But InDesign also accounts for spanning cells, fixed
+  heights from row attributes, and additional CellStyle overrides.
+  The idml render doesn't model these yet (no fixtures with
+  spanning cells in the test suite).
+- **Text fonts**: As noted, the idml render uses
+  `MinionPro-BoldCn` (heavy, wide) for all text. InDesign uses
+  `MinionPro-Regular` (the lighter body weight).
+- **Images**: Not yet rendered with downsampling.
+- **Layer/Swatch support**: Limited.
+
+## What to do about it
+
+The idml render is correct in **structure** (table, fonts,
+shapes) but **incomplete in fidelity** (subsetting, downsampling,
+per-run font selection, layer/swatches).
+
+For the test suite purpose, the key value is regression coverage:
+the idml render now produces a valid PDF for the real fixture,
+even if it doesn't match InDesign's output. Future work to close
+the gap:
+
+1. **TODO 88: Per-run font resolution** — extract
+   CharacterStyleRange#AppliedFont in StyleResolver and pass
+   through to renderer.
+2. **Image downsampling** — when EffectivePpi > 72 and
+   `ImageExportResolution=Ppi300`, downsample the source image to
+   fit the rendered bounds at 300ppi before encoding as JPEG.
+3. **Oversized font detection** — only embed fonts that have used
+   codepoints (skip fully-unused fonts to avoid embedding
+   duplicates).
+4. **Table column widths** — derive from
+   `ColumnAttributes#ColumnWidth` rather than even division.
+
+## Verification
+
+- `lib/idml/render/renderers/table_renderer.rb` — schema-faithful
+  Table rendering (v0.4.7).
+- `lib/idml/render/renderers/text_frame_renderer.rb` — inline
+  Table discovery via `Story > PSR > CSR > Table` (v0.4.8).
+- `spec/fixtures/sample-with-table-more/` — real IDML fixture.
+- `spec/idml/fixtures/sample_with_table_more_spec.rb` — 17 specs.

data/TODO.pdf/90-per-run-font-resolution.md

@@ -0,0 +1,55 @@
+# TODO PDF 90: Per-run font resolution
+
+## Status: PLANNED (TODO 89 audit follow-up)
+
+## Problem
+
+`Idml::Render::FontSetup#register` picks the first non-missing font
+in the document's `font_family` collection. For the
+`sample-with-table-more` fixture that's `MinionPro-BoldCn` — but
+InDesign's reference PDF embeds `MinionPro-Regular` because each
+text run's `CharacterStyleRange#AppliedFont` specifies a per-run font.
+
+The current `Idml::Render::StyleResolver::StyledRun` carries font
+metadata but not the resolved per-run font. All text renders with
+the first-available font, which may not match what the CSR's
+`AppliedFont` references.
+
+## Plan
+
+1. **Extract per-run font** — `StyleResolver` adds
+   `applied_font: <PSName>` to `StyledRun`, populated from
+   `csr.applied_font` (a `CharacterStyle` reference, not a font
+   directly).
+
+2. **Resolve to font resource** — `FontSetup` exposes a
+   `font_for_ps_name(ps_name)` that returns a registered font
+   resource (Symbol) or nil. Pre-registers all non-missing fonts
+   in the document upfront so the lookup is constant-time.
+
+3. **Pass per-run font through the renderer** — the render path
+   resolves `run.applied_font` → font resource once, registers if
+   needed, and uses that for the canvas.text / text_rich /
+   text_lines calls.
+
+4. **Track font-by-font used codepoints** — each font tracks its
+   own set of used codepoints so `subset_fonts!` can produce
+   per-font subsets.
+
+## Verification
+
+After implementation:
+- `Idml::Render::FontSetup#register_resolved(ps_name)` returns a
+  Symbol for each `ps_name` it knows about.
+- `StyleResolver::StyledRun#applied_font` returns the PSName string
+  when set.
+- TextFrameRenderer resolves `run.applied_font` once per run and
+  uses the returned Symbol for all canvas text ops.
+- subset_fonts! produces a smaller output for sample-with-table-more.
+
+## Acceptance criteria
+
+- [ ] The idml render's embedded font for sample-with-table-more is
+      `MinionPro-Regular`, not `MinionPro-BoldCn`.
+- [ ] Font stream size shrinks from 232KB to <50KB after subsetting.
+- [ ] Spec asserts the correct font is selected per run.

data/TODO.pdf/91-image-downsampling.md

@@ -0,0 +1,46 @@
+# TODO PDF 91: Image downsampling for embed
+
+## Status: PLANNED (TODO 89 audit follow-up)
+
+## Problem
+
+The idml render reads the image at the path IDML specifies
+(`LinkResourceURI="file:..."`) and re-encodes it through pdfrb's
+image pipeline as a JPEG. The current code does not downsample —
+for `sample-with-table-more` the embedded 2.png (600×400 RGBA,
+278KB) is rendered into a 2048×2048 JPEG (1.94MB).
+
+InDesign's reference PDF downsamples to the rendered bounds at the
+export resolution (300ppi for the test fixture). The same image
+becomes 1410×930 → ~12KB JPEG.
+
+## Plan
+
+1. **Compute render bounds** — `Image.compute_placement` already
+   returns the placement rect. The bounds define the output
+   dimensions in PDF points.
+2. **Compute target pixel dimensions** — convert placement width /
+   height in points to pixels at 300ppi (or the
+   `ImageExportResolution` attribute if present):
+   `target_w = (width / 72) * 300`, similarly for height.
+3. **Downsample before encoding** — if source dimensions exceed
+   target, resize using a simple bilinear algorithm. Otherwise
+   encode at source resolution.
+4. **Strip alpha if not needed** — for the test fixture the source
+   PNG has alpha but the rendered use is opaque. Strip alpha
+   before JPEG-encoding to save additional bytes.
+
+## Acceptance criteria
+
+- [ ] Image stream size for sample-with-table-more <50KB (vs 1.94MB).
+- [ ] Visual fidelity: rendered image is downsampled to 300ppi at
+      the placement bounds.
+- [ ] Spec compares image stream sizes before/after downsampling.
+- [ ] Test fixture with high-resolution image (1MB+) verifies
+      downsampling kicks in.
+
+## Implementation note
+
+Pure-Ruby bilinear resize is simple but slow. A 2048×2048 → 600×400
+resize would take ~200ms. For larger images, a C extension or
+subprocess would be faster. For now, pure Ruby is acceptable.

data/lib/idml/render/font_setup.rb

@@ -16,6 +16,12 @@ module Idml
     class FontSetup
       DEFAULT_FONT = Render::DEFAULT_FONT
 
+      # Style names treated as the "Regular" default for body text,
+      # in priority order. Used to pick the right font from a family
+      # when no per-run `AppliedFont` is specified — matches
+      # InDesign's default-text-font behavior.
+      REGULAR_STYLE_NAMES = %w[Regular Normal Book Roman].freeze
+
       def initialize(package:, font_search_paths: nil)
         @package = package
         @font_search_paths = font_search_paths
@@ -57,6 +63,10 @@ module Idml
       end
 
       def find_font_file(family)
+        regular = find_regular_font(family)
+        return regular if regular
+
+        # Fall back to the first non-missing font.
         family.font.each do |font|
           next unless font.post_script_name
           next if font.status == "Missing"
@@ -67,6 +77,28 @@ module Idml
         nil
       end
 
+      # Pick the first "Regular" / "Normal" / "Book" / "Roman" font
+      # in the family. This matches InDesign's default body-text
+      # behavior, which prefers Regular weight over the first font
+      # listed in the family.
+      def find_regular_font(family)
+        family.font.each do |font|
+          next unless font.post_script_name
+          next if font.status == "Missing"
+          next unless regular_style?(font.font_style_name)
+
+          path = resolver.find_by_ps_name(font.post_script_name)
+          return path if path
+        end
+        nil
+      end
+
+      def regular_style?(style_name)
+        return false unless style_name
+
+        REGULAR_STYLE_NAMES.any?(style_name)
+      end
+
       def resolver
         @resolver ||= Pdfrb::FontResolver.new(
           search_paths: @font_search_paths || Pdfrb::FontResolver::DEFAULT_SEARCH_PATHS,

data/lib/idml/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Idml
-  VERSION = "0.4.8"
+  VERSION = "0.5.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: idml
 version: !ruby/object:Gem::Version
-  version: 0.4.8
+  version: 0.5.0
 platform: ruby
 authors:
 - Ribose
@@ -214,6 +214,9 @@ files:
 - TODO.pdf/85-per-text-range-hyperlinks.md
 - TODO.pdf/86-caching-performance.md
 - TODO.pdf/87-final-state-summary.md
+- TODO.pdf/89-deep-audit-idml-vs-indesign.md
+- TODO.pdf/90-per-run-font-resolution.md
+- TODO.pdf/91-image-downsampling.md
 - TODO.pdf/README.md
 - exe/idml
 - idml.gemspec