stationery 0.12.0 → 0.12.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '08814231457b8700e5c8a03e29c1234d219adcfbf29cbc49de819f05667bf994'
-  data.tar.gz: 0cdb872927ead42db1e41ab558c828acabd73d7cba89c121edf46a58aafbf26d
+  metadata.gz: ab0f207d1a92bc13494a61e91f50d1af6df807978f878f4cf613536d96f072b6
+  data.tar.gz: feb1a51466ae5020b1afcc0f5e7be4fd0389c8217e14b06d18044f61a5f5e67b
 SHA512:
-  metadata.gz: 583090cf9f95cecf4e53ed6a535c9e71d8c62797e455228f70036497554e0b0e594a8b3b74d60212cef2734334611379b8ac301a1411c6eee7597bcf58d720a0
-  data.tar.gz: 42541eb732431081e0ff9a00d66e6d0071c46bed006402b5ebee3fc87d7cd679098b77f03692b00b275aa04f20eeefc82a2e50fb949af9ac0e27664dc472bde2
+  metadata.gz: 2707d1a55f7bef5d7f22a6a6117b45bb5f73c41641da1ae9786b37c463ff68323c24ec75d5cf3ab88274b5b8d600cde5fe49d456f776a29afdbf152624d15a28
+  data.tar.gz: f6954e7b30c7e211c0d9f83055b9cdac5cb628773c8aba34ec38fce37ba0bb39ac958e0a245fb840200be0da43722175ecb3615a48278958b300aa562fb91a6a

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## Unreleased
+
+- A tagged render is written with object streams (#206): its structure elements, and every other object that is not a stream, are packed 200 to a deflated `/Type /ObjStm`, with a cross-reference stream in place of the table and trailer (ISO 32000-1, 7.5.7 and 7.5.8). A table of 300 rows, 8 pages, is 115 KB tagged where it was 501 KB, 36 objects where it was 4,544; the tagged examples are 9 to 65% smaller. It is on by default for a render that is tagged (`tagged`, `to_pdf(tagged: true)`, `conformance :pdf_ua1`), and `object_streams false` at class level or `to_pdf(object_streams: false)` writes the classic table for a reader that cannot read object streams; `to_pdf(object_streams: true)` packs an untagged render too. Streams, the `/Encrypt` dictionary and a signature's dictionary stay out, so an encrypted, a signed and an incremental render are packed too, and PDF/A-2b, PDF/A-3b and PDF/UA-1 pass veraPDF packed. `to_png` and `to_zpl` refuse `object_streams:` as a PDF option. `Document#page_count` is how many pages the last render laid out, which `stationery render` now reports. In `PDF::Writer` (`object_streams:`, `add_unpacked`) and `PDF::ObjectStreams`.
+- **Behaviour changes:** a tagged render is packed: its catalog, pages, annotations and structure elements are inside deflated object streams, so a test or tool that searches the file's bytes for them (`/StructParent`, `/Type /Page`) finds nothing (`document.page_count` has the pages), and the file ends in a cross-reference stream, not `xref` and `trailer`. A reader of PDF 1.4 and earlier cannot open it; `object_streams: false` writes it as 0.12.0 did, byte for byte. A tagged render streamed to a block is a few bytes longer than the String, as each flush closes an object stream. Untagged documents are byte for byte what they were in 0.12.0.
+
 ## 0.12.0 (2026-09-29)
 
 Links that conform; labels and printing (page sizes in units, print hints, monochrome, page limits, `to_png`, JPEG pixels, `to_zpl` and barcodes); floats that keep to their page; speed and memory, with tables read as pages reach them; and AI first: a skill that ships with the gem, examples with their code, the docs over MCP, and `stationery inspect`.

data/README.md

@@ -514,6 +514,8 @@ render Callout.new(color: "#F3F4F6") { text "Amount due" }
   are included. `to_pdf(strict: true)`, or `strict` at class level, raises `Stationery::WarningsError`
   (with `#warnings`) instead of writing a PDF that produced any; `to_pdf(strict: false)` opts one
   render out again.
+- After a render, `document.page_count` is how many pages it laid out: a tagged file's page
+  dictionaries are packed into object streams, where a search of its bytes does not find them.
 
 ### Printing
 
@@ -672,7 +674,7 @@ ShelfLabel.new.to_png(monochrome: { dpi: 203, snap: true }) # one bit to a dot,
 - **Options.** `dpi:` (96, or the monochrome dpi), `pages:`, `monochrome:`, `debug:` (the layout
   rectangles), `strict:`, `shaper:` and `max_pages:` as `to_pdf` has them. What only a PDF has (`sign:`, `encrypt:`,
   `conformance:`, `attachments:`, `print:`, `tagged:`, `page_labels:`, `xmp:`, `factur_x:`,
-  `incremental:`, `missing_glyphs:`) raises `ArgumentError` when it is passed, and is left alone when
+  `incremental:`, `missing_glyphs:`, `object_streams:`) raises `ArgumentError` when it is passed, and is left alone when
   the class declares it, so a signed or encrypted document still has pictures.
 - **Speed.** A 100 × 150 mm label at 203 dpi (800 × 1200 dots) takes about 30 ms, a one-page invoice
   at 96 dpi about 0.1 s, a page of dithered photographs at 203 dpi about 0.7 s, a label with a
@@ -875,6 +877,13 @@ and across page breaks: a paragraph continued on the next page stays one `P`.
   `canvas.tag(element) { … }`.
 - A missing `lang` is a `Warnings::MissingLanguage`. All four are warnings, so `strict` catches them,
   and `conformance :pdf_ua1` raises on the first three.
+- A tagged render packs its structure elements, and every other object that is not a stream, into
+  deflated object streams (`/Type /ObjStm`, 200 objects each) with a cross-reference stream in place
+  of the table and trailer (ISO 32000-1, 7.5.7 and 7.5.8). A table of 300 rows is 115 KB instead of
+  501 KB. Streams, the `/Encrypt` dictionary and a signature dictionary stay out; PDF/A-2, PDF/A-3 and
+  PDF/UA-1 allow them. `object_streams false` at class level, or `to_pdf(object_streams: false)`,
+  writes the classic table for a reader that cannot read them; `to_pdf(object_streams: true)` packs
+  an untagged render too.
 - Untagged documents (the default) are written exactly as before.
 
 Check the tree in tests with `have_structure` and `have_tagged_content` (see [Testing](#testing)):

data/lib/stationery/cli/render.rb

@@ -67,7 +67,7 @@ module Stationery
         output = @options[:zpl] ? to_zpl(document) : to_pdf(document)
         return FAILURE unless warnings_ok?(document)
 
-        write(output, target)
+        write(output, target, document)
         pictures(document, target) if @options[:png]
         OK
       end
@@ -149,20 +149,21 @@ module Stationery
         false
       end
 
-      def write(pdf, target)
+      def write(pdf, target, document)
         if target == "-"
           @out.binmode
           @out.write(pdf)
         else
           File.binwrite(target, pdf)
-          @out.puts "wrote #{target} (#{count(pdf)}, #{pdf.bytesize} bytes)"
+          @out.puts "wrote #{target} (#{count(pdf, document)}, #{pdf.bytesize} bytes)"
         end
       end
 
-      # "1 page", "3 labels": what the file holds.
-      def count(output)
+      # "1 page", "3 labels": what the file holds. The pages are the
+      # render's: a tagged file's are packed, and encrypted too with it.
+      def count(output, document)
         name = @options[:zpl] ? "label" : "page"
-        count = output.scan(@options[:zpl] ? "^XA" : %r{/Type /Page\b}).size
+        count = @options[:zpl] ? output.scan("^XA").size : document.page_count
         "#{count} #{name}#{"s" unless count == 1}"
       end
 

data/lib/stationery/document.rb

@@ -131,6 +131,15 @@ module Stationery
         config[:tagged] = value
       end
 
+      # Packs every object that is not a stream (the structure elements of a
+      # tagged PDF, most of all) into deflated object streams, with a
+      # cross-reference stream: PDF 1.5, which PDF/A-2, PDF/A-3 and PDF/UA-1
+      # allow. On for a tagged render unless set; `object_streams false`
+      # writes the classic table for a reader that cannot read them.
+      def object_streams(value = true) # rubocop:disable Style/OptionalBooleanParameter
+        config[:object_streams] = value
+      end
+
       # Claims PDF/A-2b, PDF/A-3b and/or PDF/UA-1 (`conformance :pdf_a3b,
       # :pdf_ua1`) and writes what the level asks for; a render that cannot
       # keep the claim raises. See PDF::Conformance. `missing_glyphs:` is what
@@ -197,8 +206,9 @@ module Stationery
       end
     end
 
-    # `fields` is every form field's name and value from the last render.
-    attr_reader :warnings, :fields
+    # `fields` is every form field's name and value from the last render;
+    # `page_count` is how many pages it laid out.
+    attr_reader :warnings, :fields, :page_count
 
     def page_options = self.class.config[:page]
     def metadata = self.class.config[:metadata]
@@ -225,13 +235,17 @@ module Stationery
     #
     # `max_pages:` replaces the class's limit (see .max_pages) for one
     # render; nil takes it away.
+    #
+    # `object_streams:` packs what is not a stream into object streams (see
+    # .object_streams); nil, the default, packs a tagged render.
     def to_pdf(target = nil, strict: self.class.config[:strict], debug: false, encrypt: self.class.config[:encrypt],
                tagged: self.class.config[:tagged], page_labels: self.class.config[:page_labels], attachments: [],
                xmp: metadata[:xmp] != false, conformance: self.class.config[:conformance],
                factur_x: self.class.config[:factur_x], sign: self.class.config[:sign],
                shaper: self.class.config[:shaping][:shaper], incremental: self.class.config[:incremental],
                print: PDF::PrintHints::NONE, missing_glyphs: self.class.config[:missing_glyphs],
-               monochrome: self.class.config[:monochrome], max_pages: self.class.config[:max_pages], &block)
+               monochrome: self.class.config[:monochrome], max_pages: self.class.config[:max_pages],
+               object_streams: self.class.config[:object_streams], &block)
       invoice = PDF::FacturX.for(factur_x, self)
       attachments = PDF::Attachments.merge(self.class.config[:attachments], attachments, invoice&.attachment)
       conformance = PDF::Conformance.for(invoice ? invoice.conformance(conformance) : conformance, missing_glyphs:)
@@ -243,6 +257,7 @@ module Stationery
       raise ArgumentError, "pass a target or a block, not both" if target && block
 
       options = { strict:, debug:, encrypt:, tagged: tagged || conformance&.pdf_ua?, page_labels:, attachments:,
+                  object_streams: packed?(object_streams, tagged, conformance),
                   xmp: xmp || !conformance.nil?, conformance:, invoice:, signature:, sink: block, shaper:, print:,
                   incremental: incremental && !conformance && !signature && !(strict && block),
                   monochrome: Monochrome.for(self.class.config[:monochrome], monochrome), max_pages: }
@@ -297,6 +312,7 @@ module Stationery
       destinations = Structure.resolve(pages, warnings:, book:, canvases:)
       yield builder.outline.resolve(destinations) if block_given?
       @warnings = warnings
+      @page_count = pages.size
       pages
     end
 
@@ -359,8 +375,8 @@ module Stationery
     end
 
     def assemble(pages, resources, outline, encrypt:, tagging:, page_labels:, attachments:, xmp:, conformance:,
-                 invoice:, signature:, sink:, writer:, print:)
-      assembler = PDF::Assembler.new(pages:, resources:, info:, outline:, tagging:, xmp:, writer:,
+                 invoice:, signature:, sink:, writer:, print:, object_streams:)
+      assembler = PDF::Assembler.new(pages:, resources:, info:, outline:, tagging:, xmp:, writer:, object_streams:,
                                      encryption: writer ? nil : encryption(encrypt),
                                      lang: metadata[:lang], page_labels: PDF::PageLabels.entries(page_labels),
                                      attachments:, conformance:, xmp_extensions: invoice&.xmp_extensions || {},
@@ -370,6 +386,13 @@ module Stationery
       end
     end
 
+    # Object streams when asked for, else when the render is tagged.
+    def packed?(object_streams, tagged, conformance)
+      return object_streams unless object_streams.nil?
+
+      tagged || conformance&.pdf_ua? || false
+    end
+
     def byte_count(pdf) = pdf.is_a?(String) ? pdf.bytesize : pdf
     def encryption(options) = options && PDF::Encryption::StandardSecurity.new(**options)
 
@@ -377,8 +400,10 @@ module Stationery
     # every body at once, to the writer the rest of the file follows on; any
     # other seals the pages nothing paints on again (no header, footer or
     # page template, and no conformance audit to read them).
-    def sealer_for(incremental, conformance, encrypt:, sink:, **)
-      return PDF::PageSealer.new(writer: PDF::Writer.new(encryption: encryption(encrypt), sink:)) if incremental
+    def sealer_for(incremental, conformance, encrypt:, sink:, object_streams:, **)
+      if incremental
+        return PDF::PageSealer.new(writer: PDF::Writer.new(encryption: encryption(encrypt), sink:, object_streams:))
+      end
 
       config = self.class.config
       PDF::PageSealer.new(final: conformance.nil? && config[:templates].empty? && config[:regions].empty?)

data/lib/stationery/forms/acro_form.rb

@@ -93,7 +93,7 @@ module Stationery
       # known to exist; nil without a signature.
       def signed
         return unless @signature
-        return @writer.add(@signature.dictionary) if @widgets.any? { |widget| signs?(widget) }
+        return @writer.add_unpacked(@signature.dictionary) if @widgets.any? { |widget| signs?(widget) }
 
         raise ArgumentError, %(sign field: names "#{@signature.field}", but the document has no such signature_field)
       end

data/lib/stationery/pdf/assembler.rb

@@ -17,13 +17,15 @@ module Stationery
       # finished file, so it cannot be streamed. `writer:` is the Writer that
       # already holds the bodies of pages sealed as they were painted; it
       # brings its own encryption and sink. `print:` are the hints of
-      # PrintHints for the catalog.
+      # PrintHints for the catalog. `object_streams:` (true) packs what is not
+      # a stream into object streams (see Writer).
       def initialize(pages:, resources:, info: {}, outline: [], encryption: nil, tagging: nil, lang: nil,
                      page_labels: nil, attachments: [], xmp: true, xmp_extensions: {}, xmp_schemas: [],
-                     conformance: nil, signature: nil, sink: nil, writer: nil, print: nil)
+                     conformance: nil, signature: nil, sink: nil, writer: nil, print: nil, object_streams: false)
         raise ArgumentError, "a signed document cannot be streamed: sign needs the whole file" if signature && sink
 
         @writer = writer
+        @object_streams = object_streams
         @print = print
         @sink = sink
         @xmp_schemas = xmp_schemas
@@ -44,7 +46,7 @@ module Stationery
 
       # The file as a String, or the number of bytes streamed to the sink.
       def render
-        writer = @writer || Writer.new(encryption: @encryption, sink: @sink)
+        writer = @writer || Writer.new(encryption: @encryption, sink: @sink, object_streams: @object_streams)
         tree = writer.reserve
         refs = @resources.build(writer)
         @form = Forms::AcroForm.new(writer, fonts: refs.fetch(:Font), signature: @signature,

data/lib/stationery/pdf/object_streams.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Stationery
+  module PDF
+    # The two stream objects of a compressed file (ISO 32000-1, 7.5.7 and
+    # 7.5.8): an object stream holding objects that are not streams, and the
+    # cross-reference stream that replaces the table and the trailer.
+    module ObjectStreams
+      # Objects in one object stream: a few hundred deflate to a tenth of
+      # their size, and a reader inflates the whole stream to reach one.
+      CAPACITY = 200
+      # Bytes of an entry's type, offset or object stream number, and
+      # generation or index.
+      WIDTHS = [1, 4, 2].freeze
+      FREE = [0, 0, 0xFFFF].freeze
+
+      module_function
+
+      # The object stream of `batch`, [number, value] pairs, its values
+      # written without encryption: the stream is encrypted whole.
+      def pack(batch)
+        header = String.new(encoding: Encoding::BINARY)
+        body = String.new(encoding: Encoding::BINARY)
+        batch.each do |number, value|
+          header << "#{number} #{body.bytesize} "
+          body << Serializer.dump(value) << "\n"
+        end
+        Stream.new(header + body, { Type: :ObjStm, N: batch.size, First: header.bytesize })
+      end
+
+      # The cross-reference stream of `locations`, one per object from 1: an
+      # Integer offset in the file, or [object stream, index] for a packed
+      # object. `trailer` is what the trailer dictionary would hold.
+      def xref(locations, trailer)
+        fields = FREE.dup
+        locations.each { |at| at.is_a?(Integer) ? fields.push(1, at, 0) : fields.push(2, at[0], at[1]) }
+        Stream.new(fields.pack("CNn" * (fields.size / 3)), trailer.merge(Type: :XRef, W: WIDTHS))
+      end
+    end
+  end
+end

data/lib/stationery/pdf/writer.rb

@@ -5,7 +5,8 @@ require "digest/md5"
 module Stationery
   module PDF
     # Collects numbered objects and writes them as a PDF file with a classic
-    # cross-reference table. `render` returns the file as a String with the
+    # cross-reference table, or with `object_streams: true` packed into
+    # object streams with a cross-reference stream (see ObjectStreams). `render` returns the file as a String with the
     # objects in numbered order. With a `sink:` (anything answering
     # `call(bytes)`) the file goes there in pieces instead: `flush` writes every
     # object set so far and drops it, so objects leave in the order they were
@@ -16,8 +17,14 @@ module Stationery
 
       # `encryption` (an Encryption::StandardSecurity) encrypts every string
       # and stream except those of its own /Encrypt dictionary.
-      def initialize(encryption: nil, sink: nil)
+      #
+      # With `object_streams: true` every object that is not a stream goes
+      # into a deflated object stream, but for the Encrypt dictionary and an
+      # object added with #add_unpacked (a signature dictionary, which is
+      # filled in once the file is written and found by its bytes).
+      def initialize(encryption: nil, sink: nil, object_streams: false)
         @objects = []
+        @loose = [] if object_streams
         @offsets = []
         @encryption = encryption
         @sink = sink
@@ -45,6 +52,13 @@ module Stationery
         set(reserve, value)
       end
 
+      # Adds an object that stays out of the object streams.
+      def add_unpacked(value)
+        ref = add(value)
+        @loose << ref.id if @loose
+        ref
+      end
+
       # With a sink: writes every object set and not yet written, lowest number
       # first, and drops it. References to objects still unset stay valid, as
       # the cross-reference table is written last. Without a sink it does
@@ -66,9 +80,10 @@ module Stationery
         end
 
         write_pending
-        xref = @size
         id = HexString.new(file_id)
+        return write_xref_stream(root, info, id) if @loose
 
+        xref = @size
         emit("xref\n0 #{@objects.size + 1}\n0000000000 65535 f \n")
         @offsets.each { |offset| emit(format("%010d 00000 n \n", offset)) }
         emit("trailer\n#{Serializer.dump(trailer(root, info, id))}")
@@ -79,6 +94,8 @@ module Stationery
       private
 
       def write_pending
+        return pack_pending if @loose
+
         @objects.each_with_index do |object, index|
           next if object.nil? || object.equal?(WRITTEN)
 
@@ -87,14 +104,51 @@ module Stationery
         end
       end
 
+      # Streams, the Encrypt dictionary and what was added with #add_unpacked
+      # are written as they are, the rest into object streams of at most
+      # ObjectStreams::CAPACITY objects; the last one is closed with the flush.
+      def pack_pending
+        batch = []
+        @objects.each_with_index do |object, index|
+          next if object.nil? || object.equal?(WRITTEN)
+
+          number = index + 1
+          if object.is_a?(Stream) || @loose.include?(number) || number == @encrypt&.id
+            write_object(number, object)
+          else
+            batch << [number, object]
+            batch = write_object_stream(batch) if batch.size == ObjectStreams::CAPACITY
+          end
+          @objects[index] = WRITTEN
+        end
+        write_object_stream(batch) if batch.any?
+      end
+
+      def write_object_stream(batch)
+        @objects << WRITTEN
+        number = @objects.size
+        batch.each_with_index { |(packed, _), index| @offsets[packed - 1] = [number, index] }
+        write_object(number, ObjectStreams.pack(batch))
+        []
+      end
+
+      # The cross-reference stream, its own entry among them, never encrypted.
+      def write_xref_stream(root, info, id)
+        @objects << WRITTEN
+        number = @objects.size
+        xref = @offsets[number - 1] = @size
+        write_object(number, ObjectStreams.xref(@offsets, trailer(root, info, id)), nil)
+        emit("startxref\n#{xref}\n%%EOF\n")
+        @buffer || @size
+      end
+
       def trailer(root, info, id)
         trailer = { Size: @objects.size + 1, Root: root, Info: info, ID: [id, id] }
         @encrypt ? trailer.merge(Encrypt: @encrypt) : trailer
       end
 
-      def write_object(number, object)
+      def write_object(number, object, crypt = crypt_for(number))
         @offsets[number - 1] = @size
-        crypt = crypt_for(number)
         emit("#{number} 0 obj\n")
         if object.is_a?(Stream)
           data = crypt ? crypt.call(object.data) : object.data

data/lib/stationery/raster/render.rb

@@ -16,7 +16,7 @@ module Stationery
       # What only a PDF has: refused when asked for, left alone when the
       # class declares it.
       PDF_ONLY = %i[sign encrypt conformance attachments print tagged page_labels xmp factur_x incremental
-                    missing_glyphs].freeze
+                    missing_glyphs object_streams].freeze
       GREY = "\x01-\xFE".b
       DARK = "\x00-\x7F".b
       LIGHT = "\x80-\xFF".b

data/lib/stationery/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Stationery
-  VERSION = "0.12.0"
+  VERSION = "0.12.1"
 end

data/lib/stationery.rb

@@ -11,6 +11,7 @@ require_relative "stationery/pdf/encryption/aes"
 require_relative "stationery/pdf/encryption/revision4"
 require_relative "stationery/pdf/encryption/revision6"
 require_relative "stationery/pdf/encryption/standard_security"
+require_relative "stationery/pdf/object_streams"
 require_relative "stationery/pdf/writer"
 require_relative "stationery/fonts/sfnt_metrics"
 require_relative "stationery/fonts/woff"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: stationery
 version: !ruby/object:Gem::Version
-  version: 0.12.0
+  version: 0.12.1
 platform: ruby
 authors:
 - Mikael Henriksson
@@ -254,6 +254,7 @@ files:
 - lib/stationery/pdf/factur_x.rb
 - lib/stationery/pdf/icc/LICENSE.md
 - lib/stationery/pdf/icc/sRGB2014.icc
+- lib/stationery/pdf/object_streams.rb
 - lib/stationery/pdf/outline_writer.rb
 - lib/stationery/pdf/page_labels.rb
 - lib/stationery/pdf/page_sealer.rb