idml 0.5.1 → 0.5.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d4b32bcaed6562461191096ac7bbcd0ff8a14c151056bdbf122c439a711dc869
-  data.tar.gz: 30cff3491f9588716cfb676b00509f728c7c2af21b0efe9cb5a6ac4f2b77fe0b
+  metadata.gz: b61e00a5f26f6e7266f1f512362e22453623b09ea06dd002d5e0a79ba23b52d0
+  data.tar.gz: 2177e3e9371f985bd06fe889d02db20a03c90f31fdbcaa162709d2494f37fd24
 SHA512:
-  metadata.gz: a97de15a173bca917a1e10aa4fc32eb553b2ee2bd88fc33f49953437513e923b90acdaa02a734b74bc3d51493ba464aadbfbb51f86ede65369c0cf2a7a779911
-  data.tar.gz: 55448f8fb8d465fe0fbc2d078b4530ef80843721055856717788d7a77f6984f5e20a44dcaa41a8e2741a66b17b616b174f322ed0a770b3236c8617b6545054c4
+  metadata.gz: a85eed35528a6da8709a6f54c7b7a01f42334646f110adc5ff6fff572b250c26a011a3744682da3216476973f97a45cf9e9850b1bf398df7985fd85ec013eb0b
+  data.tar.gz: 405859c6e2efe2e62ae56ea3079799e0d5a822e3c7c0b15d04a904969408d7f2cd0aa76fa20faf5bb1dd7640c7c4b98fcfe0185e9f18d0c041168e7b55af5e9c

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    idml (0.5.1)
+    idml (0.5.2)
       bigdecimal
       lutaml-model (~> 0.8.18)
       pdfrb
@@ -183,7 +183,7 @@ CHECKSUMS
   ffi (1.17.4-x86_64-darwin) sha256=aa70390523cf3235096cf64962b709b4cfbd5c082a2cb2ae714eb0fe2ccda496
   ffi (1.17.4-x86_64-linux-gnu) sha256=9d3db14c2eae074b382fa9c083fe95aec6e0a1451da249eab096c34002bc752d
   ffi (1.17.4-x86_64-linux-musl) sha256=3fdf9888483de005f8ef8d1cf2d3b20d86626af206cbf780f6a6a12439a9c49e
-  idml (0.5.1)
+  idml (0.5.2)
   json (2.21.2) sha256=1f1d3b7cf2b3ba1a69beca0bb6db13d5438b80bff3cd54cdaaa620b9b07c1c6a
   language_server-protocol (3.17.0.6) sha256=5ef2c0c138f8267e1bc631d3328347d354f96724b0af22f2c79516120443b7f0
   lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87

data/TODO.pdf/89-deep-audit-idml-vs-indesign.md

@@ -65,16 +65,82 @@ non-image streams.
 - **Font subsetting to fewer glyphs than used** (lossless subsetting
   IS done by pdfrb — removes only unused glyphs).
 
-## Future audit work
-
-- **Stream-by-stream comparison** with `qpdf --qdf` to decompose both
-  PDFs into canonical form and diff.
-- **Font subsetting investigation** — pdfrb's `subset_fonts!` is
-  called but the font stream is still 232KB. The likely cause is
-  that no codepoints are recorded as "used" for the registered font
-  (the renderer uses a default font that doesn't get text drawn
-  through it). Future fix: register the font actually used to draw
-  text, so `used_codepoints` populates correctly.
+## Additional findings (stream-level audit via `mutool`)
+
+### xref byte-count bug (TODO 92)
+
+pdfrb emits 21-byte xref entries (extra space before `\r\n`). PDF
+spec §7.5.4 requires exactly 20. `mutool` reports:
+```
+format error: expected trailer marker
+warning: trying to repair broken xref
+```
+**Workaround**: use `compress: true` — switches to XRef stream,
+bypassing the traditional xref table entirely.
+
+### Font naming
+
+- InDesign: `OZNMOQ+MinionPro-Regular` (5-char subset prefix per
+  PDF spec convention for subsetted fonts).
+- Ours: `FileFont-MinionPro-Regular` (non-standard prefix).
+
+pdfrb should use the `[A-Z]{6}+` prefix convention for subsetted
+fonts.
+
+### Font type
+
+- InDesign: `/Subtype /Type1` with `/FontFile3` (CFF/Type1C).
+  InDesign converts OTF → CFF subset.
+- Ours: `/Subtype /TrueType` with `/FontFile2`. We embed the raw
+  TTF/OTF bytes as-is.
+
+Both are valid PDF but Type1/CFF is typically more compact.
+
+### Page Resources
+
+- InDesign page 1 Resources: `/ColorSpace` (ICCBased),
+  `/ExtGState` (graphics states for transparency),
+  `/Font` (T1_0), `/ProcSet`, `/XObject` (Form XObjects for master
+  items + Images).
+- Our page 1 Resources: `/Font` only. No ColorSpace, no ExtGState,
+  no XObject/Form.
+
+Missing: ICC color space, extended graphics states, Form XObjects
+for master-spread content.
+
+### Image count
+
+- InDesign: 3 images per page (downsampled to 300ppi, with ICC
+  profiles).
+- Ours: 1 image total (full-resolution GenAI JPEG, no ICC).
+
+The difference is because InDesign renders the same source asset
+on multiple pages via Form XObject reference, while we embed it
+once globally.
+
+### Info dictionary
+
+- InDesign Info: clean key/value pairs.
+- Ours: includes `/Type/Metadata` — incorrect. Info dictionaries
+  should not have a `/Type` key. This is a pdfrb issue.
+
+## Summary of all differences
+
+| Category | InDesign | Ours | Status |
+|---|---|---|---|
+| File size | 100KB | 2.1MB | Cannot close without data loss (image) |
+| PDF version | 1.7 | 1.4 | Minor |
+| Font selection | MinionPro-Regular | MinionPro-Regular | **Closed** (v0.5.0) |
+| Font subtype | Type1/CFF | TrueType | pdfrb upstream |
+| Font subset prefix | `OZNMOQ+` | `FileFont-` | pdfrb upstream |
+| FlateDecode | 24 streams | 8 streams | **Closed** (v0.5.1, opt-in) |
+| xref compliance | valid | 21-byte entries | **Workaround**: `compress: true` |
+| ICC color space | embedded | absent | TODO |
+| ExtGState | per-page | absent | TODO |
+| Form XObjects | master items | absent | TODO |
+| Image count | 3/page (downsampled) | 1 total (full-res) | Cannot close without data loss |
+| Info dict | clean | has spurious /Type/Metadata | pdfrb upstream |
+| Object count | 215 | 17 | InDesign richer structure |
 
 ## Verification
 

data/TODO.pdf/92-xref-entry-byte-count.md

@@ -0,0 +1,74 @@
+# TODO PDF 92: XRef entry byte-count compliance (21 vs 20 bytes)
+
+## Status: IDENTIFIED — pdfrb upstream bug
+
+## Problem
+
+PDF spec §7.5.4 requires each cross-reference entry to be exactly
+**20 bytes** including the 2-character end-of-line marker. Three
+valid EOL formats:
+
+- `nnnnnnnnnn ggggg n\r\n` (CRLF)
+- `nnnnnnnnnn ggggg n \n` (space + LF)
+- `nnnnnnnnnn ggggg n \r` (space + CR)
+
+pdfrb's xref writer emits `nnnnnnnnnn ggggg n \r\n` — **21 bytes**
+because of the extra space between the status character and the
+`\r\n`. Strict PDF readers (MuPDF, `mutool`) reject this:
+
+```
+format error: expected trailer marker
+warning: trying to repair broken xref
+warning: repairing PDF document
+```
+
+Lenient readers (`pdftk`, Apple Preview, Chrome) tolerate it.
+
+## Verification
+
+```python
+# In a generated PDF:
+entry = b'0000000015 00000 n \r\n'  # 21 bytes
+assert len(entry) == 21  # BUG: should be 20
+```
+
+Compare with InDesign's output (compliant):
+```python
+entry = b'0000000015 00000 n \r\n'  # should be 20 bytes
+# InDesign uses object streams + xref stream, so no traditional
+# xref table in the test fixture's pages.pdf
+```
+
+## Fix
+
+In pdfrb's writer, remove the space before `\r\n` in each xref
+entry. The format should be:
+```ruby
+"%010d %05d %s\r\n" % [offset, generation, status]
+```
+Not:
+```ruby
+"%010d %05d %s \r\n" % [offset, generation, status]
+```
+
+## Impact
+
+Without the fix, strict validators flag our PDFs as broken. The
+PDFs still open in most readers because the xref can be rebuilt
+from the object scan, but:
+
+- `mutool` prints repair warnings.
+- veraPDF may reject the file.
+- PDF/A validators may flag non-compliance.
+
+## Workaround
+
+Use `compress: true` — pdfrb then uses an XRef stream
+(`writer.use_xref_stream: true`) instead of a traditional xref
+table, bypassing the byte-counting issue entirely.
+
+## Acceptance criteria
+
+- [ ] pdfrb xref writer emits 20-byte entries.
+- [ ] `mutool info` reports no repair warnings.
+- [ ] PDF/A validators accept the output without xref warnings.

data/lib/idml/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Idml
-  VERSION = "0.5.1"
+  VERSION = "0.5.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: idml
 version: !ruby/object:Gem::Version
-  version: 0.5.1
+  version: 0.5.2
 platform: ruby
 authors:
 - Ribose
@@ -217,6 +217,7 @@ files:
 - TODO.pdf/89-deep-audit-idml-vs-indesign.md
 - TODO.pdf/90-per-run-font-resolution.md
 - TODO.pdf/91-image-downsampling.md
+- TODO.pdf/92-xref-entry-byte-count.md
 - TODO.pdf/README.md
 - exe/idml
 - idml.gemspec