@zumer/snapdom-plugins 3.0.0-beta.0 → 4.0.0

package/pdf/src/mega/README.md

@@ -0,0 +1,47 @@
+# pdfmega — the "pro pro" layer
+
+The modules that took `@zumer/snapdom-pdf` from *exact capture in a PDF* to
+*document generator*: identity, diagnostics, flow control, folios, fillable
+forms, and the accessible-structure edge. They are part of the plugin — bundled
+by `build.mjs` like everything in `src/` — and live in their own folder because
+each one is a bounded contract that `src/index.js` consumes, not a place where
+page drawing happens.
+
+| Module | Feature | Public surface |
+|---|---|---|
+| `meta.js` | `/Info`, XMP packet, `DisplayDocTitle` | `meta` export option |
+| `diag.js` | stable warning codes + report | `onReport` export option |
+| `flow.js` | progress + cancellation | `onProgress`, `signal` export options |
+| `labels.js` | `/PageLabels` agreeing with the printed folio | automatic |
+| `forms.js` | fillable AcroForm fields from captured controls | `forms` (capture), `fields` (export) |
+| `fonts.js` | `@font-face` rules → fetched, parsed, subset, embeddable faces | `embedFonts`, `vectorText` export options |
+
+The writer half of these features — trailer `/Info`/`/ID`, `%PDF-1.7`, `/OBJR`
+and table attributes in the structure tree — lives in `src/writer`,
+because bytes are its job. The capture half of forms — control records measured
+under the same visibility, clip and transform rules as every text run — lives in
+`src/text-layer.js`, because measurement is its job. What is in this folder is
+the part in between: planning and byte-shaping.
+
+Decisions that are settled (see the triage memory and REFERENCE.md):
+
+- **The `pdfuaid` claim is earned, never configured.** Fonts embed from the
+  capture's own `@font-face` (`fonts.js` + `pdf-writer/font.js`), subset to the
+  glyphs the document drew (`pdf-writer/subset.js`; `embedFonts: 'full'` opts
+  out, and CFF faces always travel whole). The claim is written
+  exactly when every text operation rode an embedded face AND the export is
+  tagged, has a language and displays a title; otherwise the blockers are named
+  (`ua-withheld`). Fillable fields block it: their `/DA` types in unembedded
+  Helvetica, so a UA-claiming export pairs with `fields: false`.
+- **Dates are never invented.** `meta.created`/`modified` are written when the
+  caller passes them. The trailer `/ID` is a content hash, not a random number.
+  Byte determinism survives every feature in this folder.
+- **A password value never reaches the file.** The field exists, flagged,
+  empty.
+- **No JavaScript actions, no submit URLs** in generated forms.
+
+Verification: `node test/megacheck.mjs` (fields, identity, labels, structure
+edge, determinism, abort — against `test/fixtures/mega.html`), plus the main
+suite `node test/verify.mjs`, whose text-layer fixtures export with
+`fields: false` because they assert the layer the values deliberately leave.
+Demo: `demo/pdfmega.html` under `node test/serve.mjs`.

package/pdf/src/mega/diag.js

@@ -0,0 +1,148 @@
+/**
+ * Structured diagnostics: the same honesty the console already gets, made
+ * machine-readable.
+ *
+ * Every fidelity loss this exporter can name is already a sentence somewhere in
+ * the pipeline — clipped runs, unresolved links, a thead too tall to repeat.
+ * What a CI gate needs is not a different sentence but a STABLE NAME for it, so
+ * "fail the build when the PDF lost annotations" is a filter and not a regex an
+ * integrator maintains against our prose.
+ *
+ * The names are assigned here, in one table, by classifying the finished
+ * message. Classifying beats threading a code through thirty call sites for one
+ * reason: the sentence and the code can never disagree about which warning this
+ * is, because the sentence IS the key. The cost is that rewording a message
+ * means touching its pattern here — which is exactly the reminder a stable
+ * contract needs.
+ *
+ * `code` values are the public contract and follow `area-what`:
+ *
+ *   page-fallback      an unknown paper name fell back to A4
+ *   layout-split       a block or field had to be split across a page break
+ *   layout-break       a declared page break could not be honoured
+ *   layout-repeat      a table header could not repeat
+ *   text-clipped       runs/links dropped by clip or overflow — fail-safe
+ *   text-invisible     runs/links dropped as invisible or degenerate
+ *   text-effects       runs/links omitted under unmappable visual effects
+ *   text-off-page      runs land outside the page box
+ *   text-overflow      the Type0 font ran out of codepoints
+ *   link-unsafe        malformed URL or unsafe protocol
+ *   link-unresolved    fragment with no destination in the capture
+ *   link-cropped       annotation cropped to its visible area
+ *   dest-collision     a matter page reused a body destination name
+ *   geometry-approx    transforms/zoom/SVG placed approximately
+ *   shadow-closed      unreachable closed shadow roots
+ *   content-skipped    content-visibility may have skipped content
+ *   raster-limit       the capture exceeds browser bitmap ceilings
+ *   image-off          image:false omitted an element band or watermark
+ *   fit-furniture      furniture/matter options that need pagination on 'fit'
+ *   toc-empty          a contents page was asked for with no headings
+ *   toc-raster         index lines drawn from the heading's own pixels
+ *   furniture-script   base-14 furniture cannot represent its script
+ *   quality-noop       lowered quality reached nothing (Flate won)
+ *   flate-off          Flate unavailable in this runtime
+ *   no-lang            tagged output without a document language
+ *   field-skipped      a control did not become a fillable field
+ *   field-password     password fields written empty, flagged, never valued
+ *   field-appearance   a field's appearance is approximate or viewer-drawn
+ *   field-name         duplicate field names were uniquified
+ *   font-embedded      the faces whose whole file travels in the document
+ *   font-fallback      a face that could not be embedded, and its reason
+ *   font-coverage      runs whose embedded face lacks some glyph — fallback
+ *   vector-painted     runs drawn as real vector glyphs from the embedded fonts
+ *   vector-missed      vector text asked for but a run could not be painted
+ *   ua-claimed         the export verified and wrote its PDF/UA-1 identifier
+ *   ua-withheld        fonts are clean but a named blocker stops the claim
+ *   option-misplaced   a capture-time option was passed at export
+ *   option-unknown     an option or property this plugin does not have
+ *   capture-empty      the capture has no layout box
+ *   other              a warning this table does not name yet
+ *
+ * An unmatched message is a bug in this table, not in the caller; it still
+ * reaches the report as `other` rather than being dropped.
+ */
+
+const RULES = [
+  [/the document is encrypted with AES-256/, 'encrypted'],
+  [/is not a permission — ignored/, 'encrypt-option'],
+  [/is not a usable size|unknown page size/, 'page-fallback'],
+  [/block\(s\) had to be split|field annotation\(s\) straddle/, 'layout-split'],
+  [/page break\(s\) could not be honoured/, 'layout-break'],
+  [/header is more than half a page tall/, 'layout-repeat'],
+  [/crossing the snapdom clip edge|dropped because an overflow clip/, 'text-clipped'],
+  [/dropped because their paint is invisible/, 'text-invisible'],
+  [/omitted because a visual effect/, 'text-effects'],
+  [/land outside the page box/, 'text-off-page'],
+  [/more than 65,534 distinct codepoints/, 'text-overflow'],
+  [/URL is malformed or uses an unsafe protocol/, 'link-unsafe'],
+  [/point outside the final captured tree|got no annotation/, 'link-unresolved'],
+  [/link annotation\(s\) cropped/, 'link-cropped'],
+  [/destination name collision/, 'dest-collision'],
+  [/non-identity transform|off-centre origin|without a screen CTM|upright glyphs|zoom on an ancestor|axis-aligned/, 'geometry-approx'],
+  [/closed shadow root/, 'shadow-closed'],
+  [/content-visibility/, 'content-skipped'],
+  [/raster pixels wide|image limit/, 'raster-limit'],
+  [/omitted because image:false/, 'image-off'],
+  [/need a paginated page size|does nothing on page:'fit'/, 'fit-furniture'],
+  [/no headings to put in one/, 'toc-empty'],
+  [/heading's own pixels/, 'toc-raster'],
+  [/furniture could not be drawn|cannot represent its script/, 'furniture-script'],
+  [/quality .* changed nothing/, 'quality-noop'],
+  [/Flate|CompressionStream/, 'flate-off'],
+  [/no `lang`/, 'no-lang'],
+  [/did not become a fillable field|field\(s\) skipped/, 'field-skipped'],
+  [/password field\(s\) written empty/, 'field-password'],
+  [/appearance|NeedAppearances/, 'field-appearance'],
+  [/duplicate field name/, 'field-name'],
+  [/^fonts embedded:/, 'font-embedded'],
+  [/none reached the capture/, 'font-fallback'],
+  [/stays searchable but is not embedded/, 'font-fallback'],
+  [/characters their embedded face lacks/, 'font-coverage'],
+  [/painted as real vector glyphs/, 'vector-painted'],
+  [/could not be painted as vector/, 'vector-missed'],
+  [/^document claims PDF\/UA-1/, 'ua-claimed'],
+  [/^PDF\/UA-1 claim withheld/, 'ua-withheld'],
+  [/decided when the capture measures/, 'option-misplaced'],
+  [/is not a PDF option|is not a document property/, 'option-unknown'],
+  [/capture element has no layout box/, 'capture-empty'],
+]
+
+/** One warning sentence → its stable code. */
+export function classify(message) {
+  for (const [pattern, code] of RULES) {
+    if (pattern.test(message)) return code
+  }
+  return 'other'
+}
+
+/**
+ * The object `onReport` receives — everything a caller can gate on, and nothing
+ * that varies between identical exports. Timing is deliberately absent: two
+ * identical exports must produce two identical reports, or the report cannot sit
+ * in a snapshot test.
+ *
+ * @param {object} args
+ * @param {string[]} args.messages  every warning, in emission order
+ * @param {number} args.pages  page count, front and back matter included
+ * @param {number} args.bytes  the finished blob's size
+ * @param {number} args.fields  fillable fields written
+ * @param {number} args.annotations  link annotations written
+ */
+export function buildReport({
+  messages, pages, bytes, fields = 0, annotations = 0, fontsEmbedded = [], pdfua = false,
+  vectorText = 0,
+}) {
+  const warnings = messages.map(message => ({ code: classify(message), message }))
+  const counts = {}
+  for (const w of warnings) counts[w.code] = (counts[w.code] || 0) + 1
+  return { warnings, counts, pages, bytes, fields, annotations, fontsEmbedded, pdfua, vectorText }
+}
+
+/** Validate the option before any expensive work runs. */
+export function reportSpec(onReport) {
+  if (onReport == null) return null
+  if (typeof onReport !== 'function') {
+    throw new TypeError('[snapdom-pdf] onReport must be a function')
+  }
+  return onReport
+}

package/pdf/src/mega/flow.js

@@ -0,0 +1,56 @@
+/**
+ * Progress and cancellation for an export that is no longer instant.
+ *
+ * A 40-page report rasterizes forty page regions through an image codec; that is
+ * seconds, and seconds behind a mute button is how users learn to distrust a
+ * feature. `onProgress` narrates, `signal` aborts — the same two primitives the
+ * platform already standardised, so nothing here invents a vocabulary.
+ *
+ * Cancellation is checked BETWEEN pages, not inside codec calls: an abort lands
+ * within one page's worth of work, and the capture's own caches stay coherent
+ * because nothing is torn down mid-object. An aborted export throws the
+ * platform's own `AbortError` — the one shape `fetch` taught every caller to
+ * catch — and produces no blob, no download, no report.
+ *
+ * Phases, in the order a caller will see them:
+ *
+ *   measure     the capture's final artifact is being measured (first export only)
+ *   page        one body page drawn; done/total are page counts
+ *   matter      cover, contents or closing page drawn
+ *   assemble    objects, structure and metadata being written
+ *
+ * `total` is the body page count once pagination has run and `null` before
+ * anything knows it. A progress bar treats null as indeterminate, which it is.
+ */
+
+/** Validate both options before any expensive work, and bind them into the two
+ * calls the pipeline actually makes. */
+export function flowSpec({ signal, onProgress } = {}) {
+  if (signal != null && !(typeof AbortSignal !== 'undefined' && signal instanceof AbortSignal)) {
+    throw new TypeError('[snapdom-pdf] signal must be an AbortSignal')
+  }
+  if (onProgress != null && typeof onProgress !== 'function') {
+    throw new TypeError('[snapdom-pdf] onProgress must be a function')
+  }
+  if (!signal && !onProgress) return null
+  return {
+    /** Throws the platform AbortError if the caller has cancelled. */
+    check() {
+      if (!signal?.aborted) return
+      throw signal.reason instanceof Error
+        ? signal.reason
+        : new DOMException('[snapdom-pdf] export aborted', 'AbortError')
+    },
+    /** One progress heartbeat. A throwing callback must not kill the export —
+     * progress is narration, not control flow — so it is reported and dropped. */
+    tick(phase, done = null, total = null) {
+      if (!onProgress) return
+      try {
+        onProgress({ phase, done, total })
+      } catch (error) {
+        console.warn(`[snapdom-pdf] onProgress threw: ${error?.message || error}`)
+        onProgress = null
+      }
+    },
+  }
+}

package/pdf/src/mega/fonts.js

@@ -0,0 +1,287 @@
+/**
+ * Which font FILES stand behind the capture's text, and the bytes of each.
+ *
+ * The path from a CSS `font-family` stack to an embeddable binary has four
+ * hops, and every one can fail in its own way:
+ *
+ *   stack → family     the first non-generic name with a readable @font-face
+ *   family → rule      CSS matching: style first, nearest weight, then the
+ *                      unicode-range subset that covers what this style paints
+ *   rule → bytes       a data: URI inlined by snapdom, or the src URL fetched
+ *                      force-cache (the page already downloaded it to paint)
+ *   bytes → sfnt       WOFF unpacks; WOFF2, collections and license-restricted
+ *                      faces refuse, each with its reason
+ *
+ * Whatever survives becomes ONE parsed face per underlying file, shared by
+ * every style that matched it. Whatever does not is a named diagnostic, never
+ * a silent substitution: a system font has no bytes a page can hand over, and
+ * saying so is the difference between "not embeddable" and "forgot".
+ *
+ * Coverage is settled LATER, per run: a face travels bound to a style even
+ * when its cmap lacks some character that style paints, because the emitter
+ * checks every run against the real cmap and falls back run-by-run. Refusing
+ * the whole binding for one missing glyph (what the SVG channel does for its
+ * textLength contract) would throw away a page of embeddable Latin over one
+ * stray dingbat.
+ *
+ * The sibling vector product solves the same problem the same way — same rule
+ * reading, same matching order — so the two channels embed the same file for
+ * the same page or disagree loudly, never quietly.
+ */
+
+import { sfntFromBytes, parseFont } from '../writer/font.js'
+
+/** Families that name a CATEGORY, not a file. Nothing behind them to embed. */
+const GENERIC = new Set([
+  'sans-serif', 'serif', 'monospace', 'cursive', 'fantasy', 'system-ui', 'math',
+  'ui-sans-serif', 'ui-serif', 'ui-monospace', 'ui-rounded', 'emoji', 'fangsong',
+  '-apple-system', 'blinkmacsystemfont',
+])
+
+const unquote = (s) => String(s || '').trim().replace(/^["']|["']$/g, '').trim()
+
+/** Split at top-level `sep`, parens and quotes respected. */
+function splitTop(s, sep) {
+  const out = []
+  let depth = 0
+  let quote = null
+  let at = 0
+  for (let i = 0; i < s.length; i++) {
+    const c = s[i]
+    if (quote) {
+      if (c === quote) quote = null
+      continue
+    }
+    if (c === '"' || c === "'") quote = c
+    else if (c === '(') depth++
+    else if (c === ')') depth = Math.max(0, depth - 1)
+    else if (c === sep && depth === 0) {
+      out.push(s.slice(at, i))
+      at = i + 1
+    }
+  }
+  out.push(s.slice(at))
+  return out
+}
+
+/** `font-weight` descriptor → [lo, hi]. A single value pins both ends. */
+function weightRange(value) {
+  const words = String(value || '').trim().toLowerCase()
+    .replace(/\bnormal\b/g, '400').replace(/\bbold\b/g, '700').split(/\s+/)
+  const lo = parseFloat(words[0])
+  const hi = parseFloat(words[1])
+  if (!Number.isFinite(lo)) return [400, 400]
+  return [lo, Number.isFinite(hi) ? hi : lo]
+}
+
+/** `unicode-range` → [[lo,hi],…], or null for "everything". */
+function parseUnicodeRange(value) {
+  const src = String(value || '').trim()
+  if (!src) return null
+  const out = []
+  for (const part of src.split(',')) {
+    const m = /^\s*U\+([0-9a-f?]+)(?:-([0-9a-f]+))?\s*$/i.exec(part)
+    if (!m) continue
+    if (m[1].includes('?')) {
+      out.push([parseInt(m[1].replace(/\?/g, '0'), 16), parseInt(m[1].replace(/\?/g, 'f'), 16)])
+    } else {
+      const lo = parseInt(m[1], 16)
+      out.push([lo, m[2] ? parseInt(m[2], 16) : lo])
+    }
+  }
+  return out.length ? out : null
+}
+
+function dataUriBytes(uri) {
+  const comma = uri.indexOf(',')
+  if (comma < 0) return null
+  const head = uri.slice(0, comma)
+  const payload = uri.slice(comma + 1)
+  try {
+    if (/;base64/i.test(head)) {
+      const bin = atob(payload)
+      const out = new Uint8Array(bin.length)
+      for (let i = 0; i < bin.length; i++) out[i] = bin.charCodeAt(i)
+      return out
+    }
+    return new TextEncoder().encode(decodeURIComponent(payload))
+  } catch {
+    return null
+  }
+}
+
+/** Formats an engine would load; anything else in `format()` never painted. */
+const LOADABLE = new Set([
+  'woff2', 'woff', 'truetype', 'opentype',
+  'woff2-variations', 'woff-variations', 'truetype-variations', 'opentype-variations',
+])
+
+/** `src:` in source order — the browser takes the first loadable entry. */
+function srcCandidates(src, base) {
+  const out = []
+  for (const item of splitTop(String(src || ''), ',')) {
+    if (/^\s*local\s*\(/i.test(item)) continue
+    const u = /url\(\s*(?:"([^"]*)"|'([^']*)'|([^)\s]*))\s*\)/i.exec(item)
+    if (!u) continue
+    const raw = u[1] || u[2] || u[3] || ''
+    if (!raw) continue
+    const f = /format\(\s*(?:"([^"]*)"|'([^']*)'|([^)\s]*))\s*\)/i.exec(item)
+    const format = (f ? (f[1] || f[2] || f[3] || '') : '').toLowerCase()
+    if (format && !LOADABLE.has(format)) continue
+    let url = raw
+    if (!/^data:/i.test(raw)) {
+      try { url = new URL(raw, base).href } catch { continue }
+    }
+    out.push(url)
+  }
+  return out
+}
+
+/**
+ * Every `@font-face` the measurement document can show. Runs while the isolated
+ * frame is ALIVE — snapdom serialized the capture's stylesheets into it, fonts
+ * usually inlined as data: URIs — because this CSSOM is gone the moment the
+ * frame is torn down, and the export that wants the bytes runs later.
+ */
+export function collectFaceRules(frameDoc) {
+  const rules = []
+  const base = frameDoc.baseURI
+  const walk = (list, sheetBase) => {
+    for (const rule of list) {
+      if (rule.type === 5) {
+        const family = unquote(rule.style.getPropertyValue('font-family'))
+        const src = rule.style.getPropertyValue('src')
+        if (family && src) {
+          rules.push({
+            family,
+            src,
+            weight: rule.style.getPropertyValue('font-weight') || '',
+            style: rule.style.getPropertyValue('font-style') || '',
+            unicodeRange: rule.style.getPropertyValue('unicode-range') || '',
+            base: sheetBase,
+          })
+        }
+        continue
+      }
+      if (rule.cssRules) {
+        try { walk(rule.cssRules, sheetBase) } catch { /* nested cross-origin */ }
+      }
+    }
+  }
+  for (const sheet of frameDoc.styleSheets) {
+    try { walk(sheet.cssRules, sheet.href || base) } catch { /* cross-origin: nothing to read */ }
+  }
+  return rules
+}
+
+/** Style first, then nearest weight — the browser's own ordering, compressed. */
+function matchScore(rule, weight, italic) {
+  const [lo, hi] = weightRange(rule.weight)
+  const distance = weight < lo ? lo - weight : weight > hi ? weight - hi : 0
+  const ruleItalic = /^(italic|oblique)/.test((rule.style || 'normal').trim())
+  return (ruleItalic === italic ? 1000 : 0) - distance
+}
+
+function coversAll(rule, points) {
+  const ranges = parseUnicodeRange(rule.unicodeRange)
+  if (!ranges) return true
+  for (const cp of points) {
+    let hit = false
+    for (const [lo, hi] of ranges) {
+      if (cp >= lo && cp <= hi) { hit = true; break }
+    }
+    if (!hit) return false
+  }
+  return true
+}
+
+async function fetchBytes(url, timeout) {
+  const controller = typeof AbortController === 'function' ? new AbortController() : null
+  const timer = controller && timeout > 0 ? setTimeout(() => controller.abort(), timeout) : null
+  try {
+    const res = await fetch(url, { signal: controller?.signal, cache: 'force-cache' })
+    if (!res.ok) return null
+    return new Uint8Array(await res.arrayBuffer())
+  } catch {
+    return null
+  } finally {
+    if (timer) clearTimeout(timer)
+  }
+}
+
+/**
+ * Resolve every font key the layer measured to a parsed, embeddable face — or
+ * to `null` and a sentence.
+ *
+ * @param {Array<{stack: string, weight: number, italic: boolean}>} fontKeys
+ * @param {Array<Set<number>>} pointsOfFk  code points each key's runs paint
+ * @param {Array<object>} rules  `collectFaceRules` output, captured at measure
+ * @param {Map} cache  capture-lifetime: url/data-uri → {parsed, sfnt}|{error}
+ * @param {(msg: string) => void} report
+ * @returns {Promise<Array<{parsed, sfnt, label}|null>>} parallel to fontKeys;
+ *   entries are SHARED where two keys resolved to one file.
+ */
+export async function resolveFaces(fontKeys, pointsOfFk, rules, cache, report) {
+  const byFamily = new Map()
+  for (const rule of rules) {
+    const key = rule.family.toLowerCase()
+    if (!byFamily.has(key)) byFamily.set(key, [])
+    byFamily.get(key).push(rule)
+  }
+
+  const faceByUrl = new Map()
+  const out = []
+  const failed = new Map() // one sentence per distinct failure, however many keys hit it
+
+  for (let i = 0; i < fontKeys.length; i++) {
+    const fk = fontKeys[i]
+    out.push(null)
+    const points = pointsOfFk[i] || new Set()
+    for (const name of splitTop(fk.stack, ',').map(unquote)) {
+      if (!name || GENERIC.has(name.toLowerCase())) continue
+      const candidates = byFamily.get(name.toLowerCase())
+      if (!candidates) continue
+      const scored = candidates
+        .map((rule, at) => ({ rule, at, score: matchScore(rule, fk.weight, fk.italic) }))
+        .sort((a, b) => (b.score - a.score) || (a.at - b.at))
+      const best = scored.filter(s => s.score === scored[0].score)
+      const winner = (best.find(s => coversAll(s.rule, points)) || best[0]).rule
+      const label = `"${winner.family}" ${fk.weight}${fk.italic ? ' italic' : ''}`
+
+      let face = null
+      for (const url of srcCandidates(winner.src, winner.base)) {
+        if (faceByUrl.has(url)) { face = faceByUrl.get(url); break }
+        let entry = cache.get(url)
+        if (!entry) {
+          const bytes = /^data:/i.test(url) ? dataUriBytes(url) : await fetchBytes(url, 8000)
+          if (!bytes || !bytes.length) {
+            entry = { error: 'its bytes could not be read' }
+          } else {
+            const container = await sfntFromBytes(bytes)
+            if (container.error) {
+              entry = { error: container.error }
+            } else {
+              const parsed = parseFont(container.sfnt)
+              entry = parsed.error ? { error: parsed.error } : { parsed, sfnt: container.sfnt }
+            }
+          }
+          cache.set(url, entry)
+        }
+        if (!entry.error) {
+          face = { parsed: entry.parsed, sfnt: entry.sfnt, label }
+          faceByUrl.set(url, face)
+          break
+        }
+        failed.set(`${label}: ${entry.error}`, true)
+      }
+      if (face) out[i] = face
+      break // first non-generic family with rules decides, matching the browser
+    }
+  }
+
+  for (const sentence of failed.keys()) {
+    report(`font ${sentence} — text in that face stays searchable but is not embedded, and the ` +
+      'export cannot claim conformance.')
+  }
+  return out
+}