stationery 0.10.1 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 94c8fd60adf62c2f1ac3dd92b4a45b5b4254d11284c255c37c86f4d7887d0f14
-  data.tar.gz: 704273654b825478068079526b21ec8b4f32c9b1b93d4f5c4854a47b22e6f0b7
+  metadata.gz: 6ebbefcf328d01eacf608d65fe46e05ac3561d335ae3ea255805afcb4fef66d2
+  data.tar.gz: 37b7e8db370b6f1d82c9b2fd493300f4194be46006c6bd29456a3002774af3f3
 SHA512:
-  metadata.gz: 07175d385d770227a5d5af7f542f608d86215ecf3172852a7f75738c426273dfe75fc272db81b929136be496e9b0b7394e1febb5b2a326ec2e06d53b012242e7
-  data.tar.gz: 864623a98fc783adecd1f8e51a7ae85bbeb1a2bd7c393a6cf72fcf8a548d48221333e29d6ca1336a100bda62576869ccae1fce084ec57b3393758226fd85ac4f
+  metadata.gz: 21b371d2b1cd3a829ac46991185c0e9b4406b67089984d5ea35dc8e60d9bacb7e0f97e2cec744e37f2c9bed39264b32af83f75e765eb2053d81834082c8aae38
+  data.tar.gz: b2aa8f17f1d896c7e9796ff960cf2777db76f20270f05ffce4dc28da9aa370973e4c6ed7aa84ccfe20da1b8579aa9183ecc6922ac9ddecf7da282007faac6a19

data/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.11.0 (2026-09-28)
+
+Text wrap around images, balanced columns, lossless WebP, a shaper hook for complex scripts, and far less memory for long documents.
+
+- Floats: `image(source, float: :left | :right, margin:)` and `box(float:, margin:, width:) { … }` take an image or a box to the edge of the flow it is written in (page body, box, column, cell), and what follows wraps beside it: every line of a paragraph takes the width left at its own top, aligned and justified in it, and the full width again below the float, mid-paragraph. `margin:` is a number (kept on the sides that face the text) or names the sides as `padding:` does; a floated box needs a `width:` (points, a fraction or `:auto`). Tables, boxes, rows, list items and other blocks go beside the float in the width that is left when they fit, else below it. A second float goes beside the first when it fits, else below; left and right floats coexist; a paragraph whose widest word does not fit starts below them. The flow is as tall as its floats. A float never splits: it moves to the next page with the content that wraps around it, and the lines a paragraph carries over a page break are wrapped again at the full width. In a tagged PDF the float stays where it was written, a `Figure` or the box's `role:`. `html` reads `float: left|right` on `img` and `<img align="left|right">`, with the image's CSS `margin` or `styles: { img: { float_margin: 8 } }`; `float` on any other element is still reported as `Warnings::UnsupportedCss`. `examples/article.rb` shows a floated photo and a pull quote. Documents without floats are byte for byte what they were, and allocate what they did.
+- `columns(count: 2, gap: 12, balance: true, rule: nil) { … }`: one flow poured through columns, newspaper style (a `row` is columns side by side with content of their own). The content is laid out at the column width and cut where a page break would cut it: between children, between the lines of a paragraph (`orphans:`/`widows:`), inside boxes and tables by their own rules, never inside `break_inside: :avoid`, and `keep_with_next` keeps a heading in the column of what follows it. `balance: true` ends the columns at nearly the same height where the content ends (the last page of the block, and before a `page_break` inside it) by searching for the shortest column height that holds everything; `balance: false` fills each column before the next. A block that does not fit fills the height left on the page and continues on the next; below other content it starts only when every column takes something, and otherwise moves to the next page. What follows the block starts below its tallest column. `rule: true | { color:, width: }` draws a line between the columns (an artifact in a tagged PDF). Works inside boxes and inside another `columns`; a tagged PDF reads column 1, then column 2, with a paragraph split across columns as one `P`. `html` maps `column-count`, `columns: <n>` and `column-gap` on `div`-like containers, `p`, headings, lists, tables, block quotes and `pre` to it. `examples/newsletter.rb`. `Layout::Columns`, `Layout::Columns::Pour`, `Layout::Columns::Balancer`.
+- Lossless WebP: `image "photo.webp"` embeds a lossless WebP (VP8L, in a plain or an extended `VP8X` container, whose colour profile and metadata chunks are skipped), decoded in Ruby with no dependency: prefix codes, backward references, the colour cache and the predictor, cross-colour, subtract-green and colour-indexing transforms. It is embedded like a PNG, RGB as a Flate stream with the alpha channel as a soft mask when a pixel is not opaque, and takes everything `image` offers, `downscale: true` included. A 1000 × 1000 px image decodes in 0.2 to 0.4 s, once per process. Lossy and animated WebP still raise `UnsupportedImage`, now naming which it is, as do truncated and damaged files and images of more than 33 megapixels (2^25 pixels). `image.stationery` reports `format: "WebP"`; the message for an unknown format reads "only JPEG, PNG and lossless WebP images are supported".
+- Shaper hook: `shaper MyShaper` at class level (inherited; `shaper nil` takes it away) or `to_pdf(shaper:)` hands every text to an object answering `call(text, font, size:, features:, language:)`, so an application with HarfBuzz can set Arabic, Hebrew, Indic or Thai text. It answers `Stationery::Shaper::Glyph`s (`gid`, `advance`, `x_offset`, `y_offset` in font units, `cluster`) in visual order, or `nil` to decline a text; `font` is a `Stationery::Shaper::Face` (`path`, `index`, `data`, `units_per_em`, `postscript_name`, `glyph_count`). Line breaking measures words through the shaper, then every stretch of a line in one font and style is shaped whole, measured and drawn from that one answer: advances as `TJ` adjustments against the font's own widths, offsets as pen movements and text rise. Shaped glyphs are embedded whatever they map to, and glyphs the ToUnicode map cannot speak for (out of logical order, several for one cluster, standing for another text already) are shown in a `Span` with the characters as `ActualText`. Glyph 0 is a `Warnings::MissingGlyph`; an answer that cannot be drawn raises `Stationery::ShaperError`. The gem shapes nothing itself and has no new dependency: there is no reordering across stretches or lines, justification widens spaces only, form fields are not shaped, and poppler and MuPDF return a right-to-left `ActualText` reversed. `examples/shaping/harfbuzz_shaper.rb` is an adapter for the `harfbuzz-ruby` gem. The memo of shaped runs starts over after 64 KB of text, like the font's own. Without a shaper every render is byte for byte what it was.
+- Long documents hold far less while they render, and every form of `to_pdf` writes the bytes it wrote. A page's nodes are let go once it is painted (the root flow, the builder and `FontBook`'s record of split runs no longer keep them), the fonts' shape and metrics memos start over after 64 KB of text instead of keeping every line drawn, a table without rowspans measures a row when a page reaches it (`Node#height_within`), a flow copies the children still to come once a page instead of once a child, and a page nothing paints on again keeps its deflated stream instead of its operators. Peak resident memory of 1,000 pages of text 391 → 111 MB and of 5,001 pages 1,556 → 350 MB; a table of 33,000 rows 901 → 653 MB and one of 165,000 rows 3,792 → 2,952 MB (what is alive when pagination ends: 216 → 17, 1,062 → 41, 494 → 28 and 2,437 → 65 MB). What is left is the document as it was built, which for a table is a node for every cell. `bundle exec rake memory` measures it.
+- `to_pdf(incremental: true)`, or `incremental` at class level, writes each page's content as soon as the page is painted, to a block before the next page is painted. For documents with headers, footers, page templates or a table of contents, which otherwise hold every page's operators until the page count is known: 5,189 pages with a footer peak at 446 MB instead of 614 MB (1,643 MB before), with 20 MB alive at the end of pagination instead of 180 MB. The file has its page contents first and fonts, page tree and catalog last, and what is painted once every page is known is a content stream of its own under or over the body, so it is a little larger (1.5 → 1.7 MB for 1,039 pages). A render with `conformance:` or `sign:`, and a `strict` one that goes to a block, take the usual path, since they must be checked before a byte is written; to a block an error on a late page leaves the start of a file. A document without headers, footers, templates or contents gains nothing from it.
+- A character no font has raises under a conformance level: text may not reference `.notdef` under PDF/UA-1 (ISO 14289-1, 7.21.8) nor under PDF/A-2b and PDF/A-3b (ISO 19005-2/3, 6.2.11.8; veraPDF rules 7.21.8-1 and 6.2.11.8-1), so `conformance` now raises `Stationery::ConformanceError` with an issue per character and family (`"☃" (U+2603) is in no font of Brand: add a font or font_fallbacks that covers it`) instead of writing a file that claims a level it does not keep. Body text, page templates and form field values are all covered. Whitespace a font lacks draws as a blank and is accepted, and without `conformance` a missing glyph stays a `MissingGlyph` warning.
+- Form field appearances keep characters no font has, as body text does since 0.10.1: a text field's value (single line, `multiline:` and `comb:` cells), a select's value and a signature field's label wrap each run of `.notdef` glyphs in a `/Span` with `/ActualText`, inside the text object and inside `/Tx BMC … EMC`, so `text_field "name", value: "日本語"` extracts as `日本語`. The appearances were already written this way; it is now specified and documented. The `MissingGlyph` warning and the bytes of values the fonts cover are unchanged. A field made without a font book (`Forms::Field.new` on a bare canvas) still writes a character outside Windows-1252 as `?` without `ActualText`: that is a real glyph, and `/V` holds the value.
+- **Behaviour changes:** a render with `conformance` that draws a character no font has now raises `ConformanceError` (it wrote a file veraPDF rejected), under PDF/A-2b and PDF/A-3b as well as PDF/UA-1. In `html`, `<img align="left|right">` floats the image, where it drew a block of its own. The lines a paragraph carries over a page break are wrapped again when the width they continue at is another. `image` and `box` raise `ArgumentError` for `margin:` without `float:` and for `float:` with `at:`. The message of `UnsupportedImage` for an unknown format names lossless WebP, and a lossy or animated WebP is named as such. `rake verify:conformance` also validates `examples/article.rb` and `examples/newsletter.rb`.
+
 ## 0.10.1 (2026-09-28)
 
 - Extracted text keeps characters no font has: each run of `.notdef` glyphs is wrapped in a `/Span` with `/ActualText` holding the written characters, and ToUnicode maps glyph 0 to U+FFFD instead of the first missing character drawn, so `日本語` set in a font without those glyphs copies out as `日本語`, not `日日日`. The `MissingGlyph` warning and the bytes of text the fonts cover are unchanged.

data/README.md

@@ -3,7 +3,7 @@
 Pure-Ruby PDF documents built from Phlex-style components. Describe the page
 with rows, columns, boxes, tables, text and images; a box-layout engine
 measures, places and paginates them; a small PDF writer embeds subsetted
-TrueType and OpenType fonts, JPEG/PNG images and SVG drawings.
+TrueType and OpenType fonts, JPEG, PNG and lossless WebP images and SVG drawings.
 
 **Documentation: [stationery.zoolutions.llc](https://stationery.zoolutions.llc)**
 
@@ -70,7 +70,7 @@ InvoicePdf.new(invoice).to_pdf          # => "%PDF-1.7…" (binary String)
 InvoicePdf.new(invoice).to_pdf("a.pdf") # also writes a path or an IO
 ```
 
-`examples/` has a complete, runnable invoice, annual report, letter, packing slip, fillable form, postcard collage, event flyer and Factur-X e-invoice
+`examples/` has a complete, runnable invoice, annual report, letter, packing slip, fillable form, postcard collage, magazine article, event flyer, two-column newsletter and Factur-X e-invoice
 ([previews and live PDFs](https://stationery.zoolutions.llc/docs/examples)); `bundle exec rake examples`
 renders them all, or render one with `stationery render examples/report.rb`.
 
@@ -81,10 +81,11 @@ renders them all, or render one with `stationery render examples/report.rb`.
 | `text(string, **style)` | A paragraph. Plain strings are literal. |
 | `text(string, markup: true)` | Reads `<b> <i> <u> <strikethrough> <sub> <sup> <br> <color rgb=""> <font size="" name=""> <link href="">`; decodes numeric (`&#39;`, `&#x27;`) and HTML 4 named entities once, so escape user data (`ERB::Util.html_escape`) before wrapping it in your own tags: `&lt;b&gt;` stays literal text. |
 | `text { b "Total"; plain " due" }` | Styled runs in Ruby. Take a block argument (`{ \|t\| t.b @x }`) to keep your own `self`. |
-| `box(padding:, background:, border:, radius:, width:, height:, min_height:, overflow:, at:, link:, outset:, break_inside:, decoration:, rotate:, shadow:) { }` | A container. Moves to the next page whole when it fits there and continues across pages when it does not; `break_inside: :auto` splits it at any page break, `:avoid` never splits it. At a cut, `decoration: :slice` (default) drops the padding and border, `:clone` keeps the padding. `overflow: :truncate` or `:shrink_to_fit` for fixed heights; a fixed `height:` never splits. `min_height:` is a floor that still splits: the first fragment keeps as much of it as the page holds, the next carries the rest (not combinable with `height:`). `at: [x, y]` pins it to a page position. `link:` makes the whole box clickable. `outset:` bleeds the background past the box (e.g. into the page margins). `rotate: -3` turns the painted box around its centre (layout box unchanged, never splits; a `link:` keeps its unrotated rectangle). `shadow: true` or `{ offset: [0, 4], blur: 8, color:, opacity: 0.15 }` paints a soft drop shadow under it, taking no space. `overflow: :hidden` clips the content to the rounded outline. |
+| `box(padding:, background:, border:, radius:, width:, height:, min_height:, overflow:, at:, link:, outset:, break_inside:, decoration:, rotate:, shadow:, float:, margin:) { }` | A container. Moves to the next page whole when it fits there and continues across pages when it does not; `break_inside: :auto` splits it at any page break, `:avoid` never splits it. At a cut, `decoration: :slice` (default) drops the padding and border, `:clone` keeps the padding. `overflow: :truncate` or `:shrink_to_fit` for fixed heights; a fixed `height:` never splits. `min_height:` is a floor that still splits: the first fragment keeps as much of it as the page holds, the next carries the rest (not combinable with `height:`). `at: [x, y]` pins it to a page position. `link:` makes the whole box clickable. `outset:` bleeds the background past the box (e.g. into the page margins). `rotate: -3` turns the painted box around its centre (layout box unchanged, never splits; a `link:` keeps its unrotated rectangle). `shadow: true` or `{ offset: [0, 4], blur: 8, color:, opacity: 0.15 }` paints a soft drop shadow under it, taking no space. `overflow: :hidden` clips the content to the rounded outline. `float: :left` or `:right` with a `width:` takes it to that side, `margin:` away from the text that wraps beside it: see [Floats](#floats). |
 | `row(gap:, align:, break_inside:) { column(width:) { } }` | Columns side by side. `width:` is points, a fraction (`0.5`), `:auto` or `nil` (equal share). Splits across pages like a box, every column at once; a row with a fixed-height column never splits; columns with `min_height:` do. |
+| `columns(count:, gap:, balance:, rule:) { }` | One flow poured through `count` columns, newspaper style: column 1 top to bottom, then column 2. Breaks where a page would (between lines with `orphans:`/`widows:`, never inside `break_inside: :avoid`, `keep_with_next` honoured). `balance: true` (default) ends the columns at nearly the same height where the content ends; `false` fills each before the next. Continues across pages; `page_break` inside ends the page; `rule: true \| { color:, width: }` draws a line between columns. |
 | `table(rows, widths:, width:, header:, split_rows:, cell:) { \|t\| }` | Tables. Cells are strings, layout nodes, procs built with the DSL (`-> { image logo }`) or components. Style with `t.row(0)`, `t.rows(-1)`, `t.column(1)`, `t.columns(1..)`, chained, plus `t.zebra`. Header rows repeat after a page break. A cell may be `{ content:, colspan:, rowspan: }` plus any cell option; rows list only the cells they start, as in HTML, and pages never break through a rowspan. Spans are set in the rows, not through selections. A row taller than the page continues on the next page, cut through its cells, with the header repeated; `split_rows: true` cuts any row that reaches the page bottom instead of moving it whole. |
-| `image(path_or_io, width:, height:, fit:, align:, radius:, rotate:, max_ppi:, downscale:)` | JPEG or PNG, aspect preserved. `fit: [w, h]` scales to fit inside; `fit: :cover` fills `width:` × `height:` and crops around the centre. `radius:` rounds the corners; `rotate:` turns it (degrees, clockwise) without changing the space it takes. Drawn at more than twice `max_ppi:` (300) it is reported as oversized; `downscale: true` resamples a PNG to that resolution instead. |
+| `image(path_or_io, width:, height:, fit:, align:, radius:, rotate:, max_ppi:, downscale:, float:, margin:)` | JPEG, PNG or lossless WebP, aspect preserved. `fit: [w, h]` scales to fit inside; `fit: :cover` fills `width:` × `height:` and crops around the centre. `radius:` rounds the corners; `rotate:` turns it (degrees, clockwise) without changing the space it takes. Drawn at more than twice `max_ppi:` (300) it is reported as oversized; `downscale: true` resamples a PNG or WebP to that resolution instead. |
 | `svg(source_or_path, width:, height:, color:, align:)` | Vector icons and drawings; `currentColor` takes `color:` (or the `color` an element sets). Linear and radial gradients (`fill="url(#id)"`, `href` chains, both gradient units); `text`/`tspan` in the document's fonts; `<style>` stylesheets (element, class, id and `*` selectors); `use`, `symbol` sprites and nested `svg` viewports (`viewBox`, `preserveAspectRatio`); `clipPath` (both `clipPathUnits`). |
 | `wrap(gap:, row_gap:, align:) { }` | Children side by side at their own widths, wrapping onto new rows (chips, tags). |
 | `stack(gap:, align:) { }` | A base with layers painted over it: ordinary children set the height, `layer` children float over them and take no space. Moves to the next page whole. |
@@ -112,6 +113,47 @@ Text style options: `font`, `size`, `weight` (`:regular`, `:bold`), `style` (`:i
 ending in a newline and lines without spaces stay left-aligned (tabs are never stretched).
 Colours are `"#RRGGBB"`, `"RRGGBB"`, `"#RGB"`, `[r, g, b]` (0-255) or `[c, m, y, k]` (0-100).
 
+### Floats
+
+`float: :left` or `:right` on an `image` or a `box` takes it to that edge of the flow it is written
+in (the page body, a box, a column, a table cell); what follows it in that flow starts at its top
+and wraps beside it, back to the full width below it, in the middle of a paragraph if need be.
+
+```ruby
+image "bay.jpg", float: :left, width: 0.4, margin: 12, alt: "The bay at dawn"
+text story, align: :justify                     # beside the photo, then below it
+
+box(float: :right, width: 170, margin: { left: 16, bottom: 6 }, padding: 10, role: :blockquote) do
+  text "The view is everywhere.", size: 13, style: :italic
+end
+text more                                       # around the pull quote
+```
+
+- `margin:` is the space between the float and the text: a number is kept on the sides that face
+  the text (the inner side and the bottom); a Hash (`{ left: 16, bottom: 6 }`, `x:`, `y:`) or an
+  Array names the sides as `padding:` does. A floated box needs a `width:`: points, a fraction of
+  the flow's width or `:auto`. Anything but `:left` and `:right` raises `ArgumentError`.
+- Text (paragraphs, headings, the text inside a `group` or an `html` block) wraps: every line takes
+  the width left at its own top and is aligned and justified in it. A paragraph whose widest word
+  does not fit beside the floats starts below them.
+- Anything else (a `box`, `row`, `columns`, `table`, list item, `rule`, `image`, `svg`, form field) is a block:
+  it goes beside the float in the width that is left, and keeps that width all the way down, when
+  its own width (or the least its content takes) fits there; else it starts below the float. A
+  `spacer` takes its height beside the float; a `page_break` ends the page and the float with it.
+- Several floats: the next one goes beside those already there when it fits, else below them, and
+  never above one written before it. Left and right floats share a line with the text between them.
+- The flow that holds a float is at least as tall as the float, so what follows a box, a column or a
+  cell starts below the floats inside it.
+- Across pages a float never splits. When it does not fit what is left of the page, or the content
+  after it could not start beside it there, it moves to the next page with that content. A
+  paragraph beside a float splits between lines as always (`orphans:`, `widows:`); the lines carried
+  over are wrapped again at the full width, because the float stayed behind.
+- In a tagged PDF the float is where it was written: an image is a `Figure`, a box has its `role:`.
+
+A line beside a float is taken to be as tall as a line of the paragraph's own style when its width
+is looked up, so one much taller word may reach a little past a float's bottom edge before the text
+widens.
+
 ### HTML and Markdown
 
 `html` and `markdown` render user content with the elements above; their parsers load on first use.
@@ -133,7 +175,7 @@ end
   monospace family for inline and block code, otherwise the text font is used), `pre` and
   `blockquote` (box options), `hr` (rule options), `table` (`cell:` options, `header:` text style),
   `ul` and `ol` (list options: `gap:`, `indent:`, `marker_gap:`, `marker_color:`, plus `style:` for
-  `ul` and `format:`/`suffix:` for `ol`), `li` (text style) and `img` (`max_width:`).
+  `ul` and `format:`/`suffix:` for `ol`), `li` (text style) and `img` (`max_width:`, and `float_margin:` for a floated image without a CSS margin).
 - Links are written only for `http`, `https`, `mailto` and `tel` hrefs (and `#anchor`); anything
   else (`javascript:`, `data:`, a relative path) keeps its text without a link and is reported as a
   `DroppedLink` warning. `links: %w[http https]` changes the list, `links: :all` keeps every href
@@ -173,11 +215,14 @@ ignored and reported. At-rules are skipped, except that rules inside `@media pri
 | `margin`, `margin-top`, `margin-bottom` | lengths in `px` or `pt`; top and bottom only, added to `gap:` | blocks |
 | `border` | `1px solid #ccc` in any order, `none` | `table` (every cell), `td`, `th` |
 | `width` | `px`, `pt`, `%`, `auto` | `img`, `table`, and `td`/`th` (column widths, when every cell of the first row has one) |
+| `float` | `left`, `right`, `none` | `img` only: the text that follows wraps beside it. Its `margin` (all four sides) is kept around it, else `styles: { img: { float_margin: 8 } }` towards the text |
 | `page-break-before`, `page-break-after`, `break-before`, `break-after` | `always`, `page`, `auto` | blocks |
 | `page-break-inside`, `break-inside` | `avoid`, `auto` | blocks |
+| `column-count`, `columns` | a number of columns, `auto` (a column width is not read) | `div` and other containers, `p`, headings, lists, tables, `blockquote`, `pre` |
+| `column-gap` | a length in `px` or `pt`, `normal` | the same |
 
-`<font color size>` and `<center>` are read the same way. Everything else (`display`, `float`,
-`position`, `font-family`, `line-height`, `em` lengths outside `font-size`, `url()` values,
+`<font color size>`, `<center>` and `<img align="left|right">` (a float) are read the same way.
+Everything else (`display`, `float` on anything but an image, `position`, `font-family`, `line-height`, `em` lengths outside `font-size`, `url()` values,
 inline backgrounds) is ignored and named in the `UnsupportedCss` warning, so `strict` catches
 content that expects more than this. No value is ever fetched: `url()` and `@import` are dropped.
 
@@ -284,7 +329,8 @@ signature_field "signature", label: "Signature of the applicant"
 - Every widget carries its own appearance, so the form looks the same in every viewer. Text is set
   in the text style around the field, in the document's own fonts: embedded, with the fallbacks
   applied per character, so a value in any script the fonts cover is drawn (`/V` holds it as
-  Unicode). Check marks and radio dots are paths. `NeedAppearances` is set too, so viewers redraw
+  Unicode). A character no font has is drawn as `.notdef` inside a `Span` whose `ActualText` is the
+  character, as in any other text, so the appearance still extracts as written. Check marks and radio dots are paths. `NeedAppearances` is set too, so viewers redraw
   edited values; ZapfDingbats is listed for the ones that redraw a button's mark, never embedded
   and never used by the appearances themselves.
 - A field that can be edited keeps printable ASCII and Latin-1 in its font beyond the value it shows
@@ -496,10 +542,14 @@ mislabelled: `ArgumentError` for options that contradict the level, `Stationery:
   field has a `/TU`, and `NeedAppearances` and ZapfDingbats are left out. Only a field made without
   a font book (`Forms::Field.new` placed with `canvas.widget`) raises, since it draws with the
   standard Helvetica.
+- A character no font has raises at every level, naming the character and the family: it draws as
+  `.notdef`, which text may not reference under PDF/A (6.2.11.8) or PDF/UA (7.21.8). Add a font or
+  `font_fallbacks` that covers it. Whitespace a font lacks draws as a blank and is accepted.
 - Without `conformance` nothing changes: the output is byte for byte what it was.
 
 `bundle exec rake verify:conformance` renders `examples/invoice.rb` as PDF/A-3b, and
-`examples/report.rb` and `examples/form.rb` as PDF/A-3b plus PDF/UA-1, and validates them with
+`examples/report.rb`, `examples/form.rb`, `examples/article.rb` (floats) and `examples/newsletter.rb`
+(columns) as PDF/A-3b plus PDF/UA-1, and validates them with
 [veraPDF](https://verapdf.org) through Docker (`verapdf/cli`); CI runs it on every push. Validate
 your own documents the same way:
 
@@ -634,17 +684,71 @@ gem "stationery", require: "stationery/rails"
 
 ### Large documents
 
+A render holds what it still has to paint. The nodes of a page are let go once the page is
+painted, a table measures its rows as pages reach them, the fonts forget the lines they shaped
+after 64 KB of text, and a page nothing paints on again (no header, footer or page template, no
+contents page number waiting on it) keeps its deflated content stream instead of its operators.
+None of that changes a byte of the file.
+
 `to_pdf { |chunk| … }` streams the file to the block in pieces as it is written and answers the
-number of bytes: the first bytes leave sooner and no output buffer is built. Peak memory does
-not drop. Layout runs in full before the first byte (pagination has to see every page) and the
-laid-out pages are what a render holds: a 938-page document peaks at 187 MB as a String and
-185 MB streamed. The streamed file lists its objects in the order they were written, a page's
-before the next page's and fonts, the structure tree and the catalog last; the String keeps them
-in numbered order. Both are the same document. A signed document cannot go to a block, since the
-signature covers every byte (`ArgumentError`). `to_pdf`, `to_pdf(path)` and `to_pdf(io)` build the
-String and return it. In a controller with `ActionController::Live`,
-`document.to_pdf { |chunk| response.stream.write(chunk) }` streams a download; `send_pdf` and
-`render pdf:` buffer.
+number of bytes: the first bytes leave sooner and no output buffer is built. Every page is laid
+out and painted before the first byte. The streamed file lists its objects in the order they
+were written, a page's before the next page's and fonts, the structure tree and the catalog
+last; the String keeps them in numbered order. Both are the same document. A signed document
+cannot go to a block, since the signature covers every byte (`ArgumentError`). `to_pdf`,
+`to_pdf(path)` and `to_pdf(io)` build the String and return it. In a controller with
+`ActionController::Live`, `document.to_pdf { |chunk| response.stream.write(chunk) }` streams a
+download; `send_pdf` and `render pdf:` buffer.
+
+`to_pdf(incremental: true)`, or `incremental` at class level, writes each page's content as soon
+as the page is painted and lets go of it; with a block it is handed over before the next page is
+painted. It is for long documents with headers, footers, page templates or a table of contents:
+those are painted once every page is known, so without it such a document holds the operators
+of every page until then.
+
+```ruby
+class StatementPdf < Stationery::Document
+  incremental                                   # or to_pdf(incremental: true) for one render
+  footer { |page| text "Page #{page.number} of #{page.count}" }
+end
+
+StatementPdf.new(account).to_pdf { |chunk| response.stream.write(chunk) }
+```
+
+- The file is the same document written another way: page contents first, then fonts, the page
+  tree, the outline and the catalog. What is painted once every page is known is a content
+  stream of its own, under or over the page's body, so a page has up to three and the file is
+  larger (1.5 → 1.7 MB for 1,039 pages with a footer).
+- To a block, bytes leave while pages are painted: an error raised on a late page leaves the
+  block with the start of a file.
+- A render that has to be checked before anything is written takes the usual path, as if
+  `incremental:` were not given: one with `conformance:` (PDF/A, PDF/UA, Factur-X: the audit
+  reads the painted pages), one with `sign:` (the signature covers the finished file), and a
+  `strict` one that goes to a block (a warning on the last page has to stop the first byte).
+  `strict` to a String, a path or an IO is incremental and raises before anything is written.
+- A document with none of them is written the same either way and gains nothing in memory.
+
+Peak resident memory of one render in a fresh process, and the megabytes still alive when
+pagination ends (`bundle exec rake memory`; Apple M2 Max, Ruby 3.4.2 +YJIT, on a busy machine:
+two runs of the same render peak up to a fifth apart, what is alive repeats):
+
+| Document | Pages | | Before | Now | `incremental: true` |
+|---|---:|---|---:|---:|---:|
+| Headings and paragraphs | 1,000 | peak | 391 MB | 111 MB | 108 MB |
+| | | alive | 216 MB | 17 MB | 14 MB |
+| | 5,001 | peak | 1,556 MB | 350 MB | 415 MB |
+| | | alive | 1,062 MB | 41 MB | 25 MB |
+| The same with a footer and a table of contents | 1,039 | peak | 397 MB | 145 MB | 106 MB |
+| | | alive | 222 MB | 44 MB | 11 MB |
+| | 5,189 | peak | 1,643 MB | 614 MB | 446 MB |
+| | | alive | 1,080 MB | 180 MB | 20 MB |
+| One table of 33,000 rows | 1,000 | peak | 901 MB | 653 MB | 648 MB |
+| | | alive | 494 MB | 28 MB | 25 MB |
+| One table of 165,000 rows | 5,000 | peak | 3,792 MB | 2,952 MB | 2,955 MB |
+| | | alive | 2,437 MB | 65 MB | 49 MB |
+
+What is left is the document as it was built: every node exists before the first page is
+painted, and a table resolves its column widths from every cell.
 
 Controllers gain `render pdf:` and `send_pdf`:
 
@@ -794,9 +898,9 @@ a figure space, a narrow no-break space, …) is drawn as a blank of the
 character's conventional width, never as `.notdef`. Any other glyph no font
 has is drawn as the family's `.notdef` and reported as a
 `Warnings::MissingGlyph` counting each drawn occurrence (so `strict` raises
-on it). The characters themselves travel as the `ActualText` of a `Span`
-around the glyphs, so the text still extracts, copies and reads aloud as
-written. Fallback covers every text element,
+on it, and a `conformance` level raises `ConformanceError`). The characters
+themselves travel as the `ActualText` of a `Span` around the glyphs, so the
+text still extracts, copies and reads aloud as written. Fallback covers every text element,
 table cell, list marker, table of contents entry and page template text;
 direct `canvas.text` calls draw with the font they are given.
 
@@ -842,19 +946,109 @@ drawn, and between ideographic characters (CJK ideographs, kana, Hangul), keepin
 a closing mark such as 。」 on the line before it and an opening bracket with what
 follows, so Japanese, Chinese and Korean text wraps without spaces.
 
-Images are JPEG (grey, RGB, CMYK) and PNG (every colour type, alpha as a soft
-mask). Parsed fonts and images are cached per process.
+Images are JPEG (grey, RGB, CMYK), PNG (every colour type, alpha as a soft
+mask) and lossless WebP (decoded in Ruby and embedded like a PNG, alpha as a
+soft mask; lossy and animated WebP raise `UnsupportedImage`). Parsed fonts and
+images are cached per process.
 
 A bitmap keeps its pixels, so a 1600 px photo drawn 160 pt wide ships all
 1600 px at 720 ppi. An image drawn at more than twice the document's
 `max_ppi` (300, or `images max_ppi: 220` at class level, `nil` to switch the
 check off) is reported as a `Warnings::OversizedImage` naming the file, its
 pixel width and the resolution it lands at. `images downscale: true` (or
-`image(..., downscale: true)`) resamples a PNG to `max_ppi` at its drawn size
+`image(..., downscale: true)`) resamples a PNG or a WebP to `max_ppi` at its drawn size
 before embedding it, alpha included; a JPEG is embedded byte for byte, so
 resize it before you embed it (an ActiveStorage variant per drawn size,
 preprocessed, keeps a render to a download).
 
+### Complex scripts: the shaper hook
+
+Stationery places glyphs itself: one per character, the font's ligatures and single substitutions,
+pair kerning. Arabic, Hebrew, Indic and Thai text need a shaper, and the gem does not have one. An
+application that does (HarfBuzz) plugs it in, for a document class or for one render:
+
+```ruby
+class Invoice < Stationery::Document
+  shaper HarfBuzzShaper.new            # inherited; `shaper nil` takes an inherited one away
+end
+
+Invoice.new.to_pdf(shaper: my_shaper)  # this render only
+```
+
+A shaper is anything that answers `call`:
+
+```ruby
+def call(text, font, size:, features:, language:, **)
+  # => [Stationery::Shaper::Glyph.new(gid:, advance:, cluster:, x_offset: 0, y_offset: 0), …] or nil
+end
+```
+
+| Argument | What it is |
+|---|---|
+| `text` | One stretch of a line in one font and style |
+| `font` | A `Stationery::Shaper::Face`: `path` (the font file, nil when there is none), `index` (the face of a collection), `data` (the sfnt bytes; a WOFF is already unpacked), `units_per_em`, `postscript_name`, `glyph_count` |
+| `size:` | The size drawn at, in points |
+| `features:` | A frozen Hash of OpenType tags to switches: `"kern"` and `"liga"` always, following `kerning:` and `ligatures:` (letter spacing switches `liga` off), and the style's `features:` as `true` |
+| `language:` | The document's `metadata lang:`, or nil |
+
+The answer is the glyphs in **visual order**, left to right. `gid` is the glyph id in the font,
+`advance` and the offsets are in font units, and `cluster` is the index, in characters, of the first
+character of `text` the glyph stands for; glyphs of the same cluster stand together for the
+characters up to the next cluster. A Hash of the same fields does for a `Glyph`. Answering `nil`
+declines a text, which is then drawn as without a shaper. Anything else (a glyph id the font does
+not have, a cluster outside the text, an advance that is not a number) raises
+`Stationery::ShaperError`. Direction and script are not passed, because stationery knows neither:
+the shaper works them out from the text.
+
+What is shaped, and when:
+
+- Line breaking works on the text as written and measures each word and each stretch of spaces on
+  its own. Each line is then cut where the font or the style changes, and every such stretch is
+  shaped whole. That one answer is the stretch's width (alignment, underline, link rectangle) and
+  what is drawn, so the two always agree. The width a break was decided on is the sum of the words',
+  which can differ a little from the shaped line's, as kerning across a space already does.
+- The shaper is asked once per text, size and set of features in a render, and must answer the
+  same for the same.
+- Every text a document draws goes through it: `text`, table cells, lists, `html`, `markdown`,
+  headers and footers, the table of contents, text in `svg`. Form fields do not: a viewer that
+  redraws a field after an edit would not shape it either.
+
+In the PDF the glyph ids are written as the shaper gave them. The font's widths stay its own, so an
+advance the shaper changed is a `TJ` adjustment, an x offset moves the pen before the glyph and back
+after it, and a y offset is a text rise (`Ts`) around it, which keeps marks in place under a
+synthetic oblique, a superscript or letter spacing. Glyphs the shaper placed are embedded whether or
+not a character maps to them. Where the ToUnicode map cannot give the text (glyphs out of logical
+order, several glyphs for one cluster, a glyph that already stands for another text, glyph 0) the
+glyphs are shown in a `Span` whose `ActualText` is the characters in logical order. Glyph 0 is
+reported as a `Warnings::MissingGlyph`, like any other.
+
+The limits of a hook:
+
+- The shaper reorders within the stretch it is given and nowhere else. Stretches on a line are placed
+  left to right in the order written and lines break in that order, so there is no bidirectional
+  reordering across a change of font or style; a right-to-left paragraph is set with `align: :right`,
+  and the last line of a justified one is set left.
+- Fallback fonts are chosen per character from the fonts' cmaps before anything is shaped. Give the
+  text the family that covers its script, or digits and punctuation the first family has are drawn
+  from it, as stretches of their own.
+- Justification widens U+0020 spaces only (no kashida). Letter spacing is added after each cluster,
+  so a mark stays on its base, but it pulls cursive letters apart.
+- Hyphenation and the breaking of a word wider than the line cut the text as written; the pieces are
+  shaped as separate words.
+- Vertical advances are not read: text runs horizontally.
+- A reader that takes `ActualText` for the text (the gem's `Inspector`) gets it as written. poppler
+  (`pdftotext` 26.09) and MuPDF (1.28) run their own reordering over it and return a right-to-left
+  stretch reversed. pdf-reader, and so `Inspector#text`, drops a `Span` whose first glyph has no
+  advance (a mark drawn first); `Inspector#structure` does not.
+
+[`examples/shaping/harfbuzz_shaper.rb`](https://github.com/zoolutions/stationery/blob/main/examples/shaping/harfbuzz_shaper.rb)
+is an adapter for HarfBuzz through the [`harfbuzz-ruby`](https://github.com/ydah/harfbuzz) gem
+(`gem "harfbuzz-ruby"`, `require "harfbuzz"`; not the older `harfbuzz` gem, which answers to the same
+`require`), about a hundred lines to copy into an application; stationery does not depend on it. It
+was run with harfbuzz-ruby 1.1.0 and HarfBuzz 14.5.0 against Noto Sans Arabic and draws joined,
+right-to-left Arabic with its marks and with left-to-right digits inside it. It cuts a stretch into
+runs of one direction by its letters alone, not by the Unicode bidirectional algorithm.
+
 ## Testing
 
 `stationery/rspec` and `stationery/minitest` read a rendered PDF back for
@@ -1023,19 +1217,32 @@ table render under StackProf (wall mode; `MODE=cpu` or `MODE=object` for the oth
 `SGHTMLTOPDF=0 bundle exec rake bench` leaves the third engine out, as does a machine
 without its gem.
 
+A lossless WebP is decoded in Ruby when it is first loaded, which a JPEG or an opaque PNG
+(both passed through) never is: a 1000 × 1000 px image takes 0.2 to 0.4 s on the machine
+above, by what is in it, and the image cache keeps it for the renders that follow.
+
 Time depends on the machine, so CI holds what does not: `bundle exec rake metrics`
 renders six fixed documents and compares the objects each render allocates, its
 page count and its bytes with `benchmark/baseline.json` (allocations may grow 3%,
 bytes 1%, pages not at all). A change that moves them on purpose records a new
 baseline with `bundle exec rake metrics:update` and says why in the commit.
 
+`bundle exec rake memory` reports what long documents hold while they render
+(`PAGES=5000` for more than its 1,000 pages): the peak resident set size of a render in
+a fresh process, and what is alive when building, pagination and writing end. It
+moves with the machine and gates nothing; the figures are under
+[Large documents](#large-documents).
+
 ## Limitations
 
 Fonts: no variable fonts (including CFF2) and no WOFF2 (it needs Brotli; convert to `.ttf` or
 `.woff`); shaping stops at pair kerning and single or ligature substitutions (`liga` by default,
 `smcp`, `onum`, `tnum`, `ss01`… on request), so contextual alternates (`calt`, `clig`, `frac`) do
-nothing and scripts that need contextual shaping (Arabic, Indic, Thai) draw glyph by glyph, and colour or emoji glyphs no font
-in the chain has are drawn as `.notdef` and reported. Text runs left to right (CJK text wraps between
+nothing and scripts that need contextual shaping (Arabic, Indic, Thai) draw glyph by glyph unless the
+application brings a shaper (`shaper`, see [the shaper hook](#complex-scripts-the-shaper-hook): the gem
+shapes none of them itself), and colour or emoji glyphs no font
+in the chain has are drawn as `.notdef` and reported. Text runs left to right: a shaper orders the glyphs
+within a stretch of one font and style, nothing reorders stretches or lines (CJK text wraps between
 ideographs, but there is no vertical layout); hyphenation
 patterns are bundled for English, German and Swedish only (a soft hyphen works in any language),
 and justification only widens spaces.
@@ -1044,15 +1251,23 @@ SVG covers the shapes, gradients, text, stylesheets, `use`/`symbol` sprites and
 sets and exports use, nothing else: `image`, `mask`, `pattern`, `filter` and `textPath` are skipped
 and reported, text inside a `clipPath` does not clip, and the shapes of a clip path join into one
 path, so overlapping shapes wound in opposite directions cancel where they overlap.
-Images are JPEG and PNG (non-interlaced) and never fetched from a URL; a JPEG is embedded at its
-source resolution (only PNGs can be downscaled), so an oversized one is reported, not resized. `html` reads a fixed subset of CSS (colours, sizes, weights, alignment, margins, padding, table
+Images are JPEG, PNG (non-interlaced) and lossless WebP (not lossy or animated WebP, and no more
+than 33 megapixels) and never fetched from a URL; a JPEG is embedded at its
+source resolution (only PNG and WebP can be downscaled), so an oversized one is reported, not resized. `html` reads a fixed subset of CSS (colours, sizes, weights, alignment, margins, padding, table
 borders and widths, page breaks; see [What CSS is read](#what-css-is-read)), not a layout
-engine's worth: no `display`, floats, positioning, `font-family` or selectors with combinators.
+engine's worth: no `display`, positioning, `font-family` or selectors with combinators, and floats
+for images only.
 `markdown` reads no CSS, and raw HTML inside Markdown stays literal text.
 
 Layout: a box with a fixed `height:` never splits (use `min_height:` for a floor that can); a row
-splits only when every column can; a rotated box and a `stack` move to the next page whole. Text
-does not wrap around images. Link and form-widget rectangles stay in page space inside `rotate`
+splits only when every column can; a rotated box and a `stack` move to the next page whole.
+`columns` balances to the shortest height that holds the content and fills the columns in order
+(ten lines in three columns are 4, 4 and 2), has columns of one width, nothing spanning them
+(end the block, write the full-width content, start another) and no column break of its own; a
+spacer that lands at the top of a column keeps its height. Text
+wraps around floated images and boxes, along their rectangles, never along a shape; beside a float
+a table, box, row or list is a block of the width that is left, all the way down (see
+[Floats](#floats)). Link and form-widget rectangles stay in page space inside `rotate`
 and `transform`, and `shadow:` is stacked rectangles, not a blur.
 
 PDF: PDF/A-2b, PDF/A-3b and PDF/UA-1 only (no PDF/A-1, no level A or U, no PDF/UA-2, no PDF/X) and

data/lib/stationery/builder.rb

@@ -56,6 +56,15 @@ module Stationery
       node
     end
 
+    # Hands the finished root over and forgets it: the paginator is then the
+    # only holder of the nodes, so those of a painted page can be collected.
+    def release
+      root = @root
+      @root = nil
+      @containers = []
+      root
+    end
+
     def within(container)
       @containers << container
       yield

data/lib/stationery/canvas/text.rb

@@ -9,14 +9,20 @@ module Stationery
       # Draws `string` with its baseline at (x, y) in top-left coordinates and
       # returns its advance width. Standard ligatures form unless `ligatures:`
       # is false or letter spacing is set; `features:` are further OpenType
-      # feature tags (Strings) to apply.
+      # feature tags (Strings) to apply. A font with a shaper draws the glyphs
+      # the shaper placed (see Shaper).
       def text(string, x:, y:, font:, size:, color: "#000000", letter_spacing: 0, rise: 0, opacity: nil,
                synthetic_bold: false, synthetic_oblique: false, underline: false, strikethrough: false,
                kerning: false, ligatures: true, features: Fonts::Font::NO_FEATURES, word_spacing: 0)
         return 0 if string.empty?
 
         color = Color.parse(color)
-        run = font.glyph_run(string, kerning:, ligatures: ligatures && letter_spacing.zero?, features:)
+        run = font.shaped(string, size, letter_spacing:, kerning:, ligatures:, features:)&.with(rise:)
+        return 0 if run&.glyphs&.empty?
+
+        # A shaped run carries its letter spacing between its clusters.
+        letter_spacing = 0 if run
+        run ||= font.glyph_run(string, kerning:, ligatures: ligatures && letter_spacing.zero?, features:)
         run = run.with_word_spacing(word_spacing, size) unless word_spacing.zero?
         width = run.width(size, letter_spacing:)
         graphics(opacity:) do |ops|