safe_image 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cb95e7819270c76a02c8ca67321d234f945cb572098e80055e43b44fac0c84ce
-  data.tar.gz: 3886ea56e99336f15cc1afd124ac57e200052120b2f61dd98bb17240fe1d3d04
+  metadata.gz: ce9a76b504fa826aef4164c5748c800a95828ef7fda6bda3a00e7e85f0134f38
+  data.tar.gz: 61c5ebac14806e8e3445c3e7aa73bb1ab5191cc414e1af311b752ad0be492bc7
 SHA512:
-  metadata.gz: e5da964df60b257eb3d72df410379c9ae52b66375d8b75f165a0ad49953099e7f41c894f896d3d8c2371fff4337d53d096cb78657dfba2c7f455f14c1d9bb873
-  data.tar.gz: 201b2f6499be3cc1bb10141c55688564022dca6f133760ecaccf6bb0502b536eb6e1ba2d1bc3824c9909d61474ac9ba5db35f812a85326b7c8cf387d4d947719
+  metadata.gz: 5d15f34ca4de05f1ca4618695bc3993a000ce38a7de54afde86ac87497c0288ad0ed37493ca31431669341be10991bbcdb5192796ea6bdb3af016943adfac2bc
+  data.tar.gz: e71a59800452d4d396741e4e28134c47403490100233e803644a90a0ca967abd74fb5572a936a7af247411cb2055378982bccad89085085184aa92c66ca94d0d

data/CHANGELOG.md

@@ -0,0 +1,351 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.3.0 - 2026-06-12]
+
+### Changed (breaking)
+
+- **`sanitize_svg!` now requires `id_namespace:`.** The argument forces a
+  deliberate choice of where the output may be used, removing the footgun of a
+  silently-wrong default. Pass `:standalone` for output served only as an
+  external `<img>`/CSS-url/file, or a stable per-document String to make it safe
+  to inline (see below). Omitting it (or passing `nil`/`""`) raises
+  `ArgumentError`. Callers must update: `sanitize_svg!(path)` →
+  `sanitize_svg!(path, id_namespace: :standalone)`.
+
+### Added
+
+- **`sanitize_svg!` can produce output safe to inline into an HTML DOM.** Pass
+  `id_namespace:` a stable per-document value (e.g. the upload sha) and the
+  sanitizer prefixes every `id` and every reference to it (`href`/`xlink:href`
+  fragments, `url(#…)` in attributes and CSS, ARIA IDREF attributes like
+  `aria-labelledby`/`aria-controls`, and every `class` token plus the matching
+  `.class` selectors) with the namespace, and scopes every `<style>` selector
+  under a `<ns>-scope` class added to the root `<svg>`. Namespacing classes stops
+  an inlined SVG from invoking the host page's framework CSS (a bare
+  `class="modal fixed"` overlay vector). `var()`/`env()`/`attr()` in presentation
+  attributes are rejected outright — they resolve against the host page.
+  Inlined into a page, the preserved `<style>` can no longer reach the host
+  cascade (`*{visibility:hidden}`, `#header{display:none}`) and ids cannot
+  clobber host ids — including references written `URL(#x)`, `url('#x')`, or
+  `url("#x")`, which are namespaced like the unquoted form. In this mode the root
+  `<svg>`'s `overflow` is also dropped so it clips to its declared viewport (a
+  tiny viewport with `overflow:visible` and oversized content would otherwise
+  paint a full-page overlay). With `id_namespace: :standalone` the output is the
+  document-safe form (no namespacing). The namespace must be a valid ident (a
+  letter followed by letters/digits/`_`/`-`); malformed tokens are rejected
+  rather than coerced, so two distinct values can never collapse to one. The
+  transform is idempotent per namespace. `style=""` attributes are
+  element-scoped and inline-safe either way.
+- **`<style>` elements now fail closed on any at-rule.** Previously an at-rule
+  block followed by a valid rule (`@font-face{…}.ok{…}`) could keep the trailing
+  rule; a stylesheet containing `@` anywhere is now rejected whole, matching the
+  documented guarantee.
+- **The SVG sanitizer keeps a safe CSS subset instead of stripping all CSS.**
+  `style` attributes (as written by Inkscape) and `<style>` elements (as
+  written by Illustrator) now survive sanitisation when they parse against a
+  constructed allowlist grammar: properties mirroring the allowed
+  presentation attributes, type/class/id selectors, numeric/keyword/color
+  values, and `url(#fragment)` references only. Output is reassembled from
+  validated tokens — CSS escapes, quotes/strings, at-rules, comments, and
+  unknown properties, functions, or selectors drop the declaration, rule, or
+  whole stylesheet rather than being interpreted. A single at-rule or nested
+  block fails the whole `<style>` element closed. `!important` and modern
+  `rgb()/hsl()` slash-alpha (`rgb(R G B / A)`) are preserved; both are parsed
+  structurally and re-emitted, and admitting `/` for the alpha keeps CSS
+  comments impossible because `*` remains excluded from the value charset.
+- **The presentation-attribute allowlist covers common editor output.** Added
+  the safe, widely-emitted SVG presentation properties (and their CSS twins):
+  `stroke-dasharray`/`stroke-dashoffset`, `vector-effect`, `marker`/`marker-*`
+  (with the `<marker>` element and its geometry attributes), `color`,
+  `display`/`visibility`/`overflow`, `paint-order`/`mix-blend-mode`/`isolation`,
+  the `*-rendering` hints, and the longhand text properties (`font-style`,
+  `font-variant`, `font-stretch`, `text-decoration`, `letter-spacing`,
+  `word-spacing`, `dominant-baseline`, `baseline-shift`, `writing-mode`,
+  `direction`). The only additions carrying a URL — `marker*` — are constrained
+  to `url(#fragment)` like the existing paint and clip/mask references. Filters
+  remain out of scope.
+
+### Changed
+
+- **Sandboxed operations are served by a pool of resident zygote workers —
+  warm Landlock overhead drops from ~100ms to ~3–8ms per operation, with
+  near-linear concurrency.** Previously every sandboxed call exec'd a fresh
+  Ruby that re-paid interpreter boot, rubygems, the gem's requires, and
+  libvips init. A zygote boots that once, then forks a child per operation;
+  the child applies rlimits, its per-operation Landlock policy (filesystem
+  allowlist, all TCP denied on ABI ≥ 4, abstract-unix-socket/signal scopes on
+  ABI ≥ 6), and — when the installed `landlock` gem exposes
+  `seccomp_deny_network!` — the helper's deny-all-network seccomp filter
+  (blocking sockets of every family, closing the UDP gap the in-process
+  Landlock policy alone leaves open), *before* touching untrusted input, and
+  exits after the operation. Workers are pooled so N threads run N sandboxed
+  operations at once (the single zygote would serialise them): the pool grows
+  on demand to `SAFE_IMAGE_ZYGOTE_WORKERS` (default 8) and offered concurrency
+  past the cap blocks until a worker frees, bounding concurrent libvips
+  memory. An idle worker exits after `Zygote::IDLE_SECONDS` (300, overridable
+  via `SAFE_IMAGE_ZYGOTE_IDLE_SECONDS`; idling holds ~16MB private memory and
+  no CPU), exits immediately when its parent process does, and `configure!`
+  always retires the pool, so no stale process outlives its parent or a
+  reconfigure. Forking is sound because the zygote never runs operations
+  itself: libvips is initialised but quiescent (zero native threads) at every
+  fork — verified empirically. Outputs are byte-identical to both the exec
+  worker and unsandboxed operation. `SAFE_IMAGE_ZYGOTE=0` falls back to the
+  exec-per-operation worker. Pool correctness under concurrent reconfigure,
+  worker death, and fork is enforced by a per-worker generation token (a
+  worker checked out when `configure!` lands is retired on return, never
+  reused under stale config), channel-health (not process-liveness) reuse
+  decisions (a worker whose pipe broke is discarded, never re-pooled), a
+  `PR_SET_PDEATHSIG` so an operation child dies with its zygote, and
+  parent-side tmp-root cleanup that survives a SIGKILLed worker — all
+  exercised by a multithreaded chaos stress test (reconfigure + worker kills)
+  asserting no wrong output, no process/fd/tmpdir leak, and no deadlock.
+
+- **External tools run with `MALLOC_ARENA_MAX=2`.** Multithreaded optimizer
+  tools (oxipng's rayon pool, ImageMagick's OpenMP) otherwise have glibc
+  reserve a 64MB malloc arena per thread; combined with the sandbox's
+  `RLIMIT_AS` memory cap, that *address-space* reservation spuriously fails
+  the tool under concurrency even though real memory use is tiny. Bounding the
+  arena count is the standard mitigation and is free for these compute-bound
+  tools. Found by stress-testing concurrent sandboxed `convert`.
+
+- **rexml loads on first SVG use instead of at `require "safe_image"`.**
+  rexml costs ~27ms to parse and only the SVG paths need it, so every boot of
+  the gem — and in particular every Landlock sandbox worker, which is a fresh
+  Ruby process per operation — was paying it on operations that never touch
+  SVG. Measured per-op sandbox cost drops from ~102ms to ~85ms on the vips
+  backend and ~75ms to ~58ms on the imagemagick backend.
+
+- **Remote fetches reject bad responses from the headers alone.** The
+  `Content-Type` allowlist and content-type/extension agreement checks now run
+  before any body bytes are read (previously the body was downloaded first and
+  rejected afterwards), and the first bytes of the body must be compatible
+  with the claimed format's magic bytes — an obviously mislabeled body is
+  dropped after the first chunk instead of being downloaded to `max_bytes`.
+- **Remote metadata helpers download only what the answer needs.**
+  `remote_size`, `remote_type`, `remote_info` and `remote_animated?` now probe
+  the partially-downloaded file at growing thresholds (64KB, 256KB, ...) and
+  abort the transfer once the answer is final, instead of always downloading
+  up to `max_bytes`. Early answers are only trusted when more data cannot
+  change them: "not animated" still requires the complete file (truncated
+  animations undercount frames), SVG metadata still downloads the whole
+  document so the SVG size cap keeps its meaning, and any prefix probe
+  failure falls back to the full download with unchanged validation and
+  error behaviour. `fetch_remote` and `remote_dominant_color` still download
+  the complete body.
+
+### Fixed
+
+- **Lossy PNG optimisation no longer fails when pngquant declines to
+  quantise.** pngquant signals "quantised result not used" through exit
+  status 98 (`--skip-if-larger`) and 99 (`--quality` not met) — for example
+  on low-bit-depth grayscale PNGs its RGBA-palette output cannot beat —
+  and `optimize(mode: :lossy)` raised `CommandError` instead of keeping the
+  original and continuing to oxipng. Found by running 10 random Wikimedia
+  Commons images through the optimizer.
+
+- **`optimize` no longer ships sideways JPEGs.** With `strip_metadata: true`
+  (the default), stripping deleted the EXIF orientation tag without applying
+  the rotation, so an oriented camera photo came out rendered 90/180° wrong.
+  `optimize` now bakes the rotation into the pixels first via jpegtran's
+  lossless transforms: `-perfect` when the dimensions are MCU-aligned, else
+  `-trim`, which drops the partial edge blocks (under one MCU, at most 15px)
+  instead of hiding a lossy re-encode. The result hash gains `rotated_from:`
+  and `trimmed:` so the trim is reported, never silent — image_optim's jhead
+  worker (Discourse's `FileHelper.optimize_image!`) does the same transform
+  but trims silently. Without jpegtran an oriented JPEG raises in strict mode
+  and is left untouched otherwise; reading the tag goes through the configured
+  backend, so `optimize` now also enforces the pixel cap before touching an
+  oriented JPEG. Internal callers optimising output the gem just encoded skip
+  the check (`assume_upright:`).
+
+- **JPEGs with an EXIF orientation no longer fail on the libvips path once
+  they outgrow the sequential readahead window (~512px — every real camera
+  photo).** `resize`, `crop`, `convert`/`convert_to_jpeg` and the
+  `fix_orientation` re-encode tier loaded input with `access: sequential` and
+  then autorotated, and the rotation's out-of-order row reads raised
+  `VipsJpeg: out of order read`. Oriented images are now reloaded with random
+  access before autorotation; upright images keep the streaming sequential
+  load, and the pixel cap still runs before any decode.
+
+### Security
+
+- **Presentation-attribute `url()` references fail closed unless they are a
+  canonical same-document fragment.** A single validation/rewrite grammar now
+  governs both the keep decision and the namespace rewrite, so external URLs,
+  mismatched quotes, and unterminated forms (`url(#id`, `url(http://evil`) are
+  dropped rather than kept on browser parse-error leniency — and no bare,
+  un-namespaced reference can survive in inline (`id_namespace:` String) output.
+- **Attribute values containing CSS escapes are rejected outright.**
+  Browsers feed SVG presentation attributes through their CSS value parsers,
+  where an escape can re-form a token after the sanitizer's pattern checks
+  (`ur\6c(...)` is `url(...)`). No allowlisted attribute legitimately
+  contains a backslash, so any attribute value with one is now dropped.
+- **SVG parsing rejects encodings the byte-level guards cannot see through.**
+  The DOCTYPE/processing-instruction guards are ASCII byte scans; a UTF-16
+  document interleaves NUL bytes between the ASCII characters, so a
+  `<!DOCTYPE` (and an entity payload behind it) could slip past them while
+  REXML still decoded and honoured it. SVG documents must now be UTF-8 (BOM
+  allowed) or declare a single-byte ASCII-transparent charset (US-ASCII,
+  ISO-8859-*, Windows-125x): UTF-16/32 BOMs, embedded NUL bytes, and declared
+  multi-byte or transforming encodings (Shift_JIS, GBK, EUC-*, ISO-2022-*,
+  UTF-7) raise `InvalidImageError`. Declared names that fit the allowed shape
+  but resolve to no real encoding (e.g. `utf8`, `windows-1259`) also fail
+  closed as `InvalidImageError` instead of surfacing REXML's bare
+  `ArgumentError`.
+
+## [0.2.0] - 2026-06-10
+
+The host's whole image-processing posture is now decided in one place, once,
+at boot. There is no per-call backend fidelity and no automatic routing
+between backends.
+
+### Added
+
+- **`SafeImage.configure!(backend:, landlock:, max_pixels:)` is mandatory**:
+  every operation before it raises the new `SafeImage::NotConfiguredError`.
+  `backend:` (`:vips` or `:imagemagick`) picks the decoder for all untrusted
+  bytes; `landlock:` decides sandboxing; `max_pixels:` sets the default
+  decompression-bomb ceiling (128MP, still overridable per call). Validation
+  is eager — a missing libvips, ImageMagick binary, or Landlock support fails
+  at boot, not on the first request. Calling `configure!` again replaces the
+  configuration atomically, so initializer reloads are safe.
+- `SafeImage.config` (frozen current configuration) and
+  `SafeImage.configured?`.
+
+### Changed
+
+- **Backend selection is strict.** `:auto` is gone; nothing ever falls back
+  from libvips to ImageMagick. A format the configured backend cannot decode
+  (e.g. ICO transform input on `:vips`, HEIC on a libvips build without
+  libheif) raises `UnsupportedFormatError`. Pure-Ruby format handling (SVG
+  metadata/sanitising, ICO metadata) still works on either backend.
+- **cjpegli is availability-driven**, like the optimizer tools: installed
+  means used for JPEG output on the `:vips` backend, absent means libvips
+  encodes. It is no longer a per-call or configuration choice.
+- Sandbox worker processes inherit the parent's backend and pixel-ceiling
+  configuration through the request payload (landlock is forced off inside
+  the worker, so sandboxed operations never nest).
+- `dominant_color` on the `:imagemagick` backend now decodes ICO through
+  ImageMagick; the pure-Ruby ICO decoder serves the `:vips` backend.
+
+### Removed
+
+- Per-call `backend:` keyword on `resize`, `crop`, `downsize`, `convert`,
+  `thumbnail`, `letter_avatar`, `fix_orientation` and `dominant_color`.
+- Per-call `encoder:` keyword on `convert`, `convert_to_jpeg`, `thumbnail`,
+  `resize`, `crop` and `downsize`.
+- `execution:` keyword on `thumbnail` (`:sandbox` / `:sandbox_if_available`)
+  — sandboxing is decided by `configure!(landlock:)`.
+- `SafeImage.enable_sandbox!`, `SafeImage.disable_sandbox!`,
+  `SafeImage.sandbox_enabled?` and `SafeImage.sandbox_call` —
+  `SafeImage.sandbox_available?` remains as the pre-configuration probe.
+
+## [0.1.1] - 2026-06-10
+
+### Added
+
+- `SafeImage.dominant_color` and `SafeImage.remote_dominant_color`: alpha-weighted
+  average colour as an `RRGGBB` hex string, computed natively through libvips,
+  with an ImageMagick histogram backend (`backend: :imagemagick`) matching the
+  command Discourse runs.
+- Pure-Ruby ICO support: directory parsing for `probe`/`frame_count`,
+  largest-entry favicon extraction for `convert_favicon_to_png` (embedded PNG
+  payloads are sanitised by re-encoding; legacy DIB payloads decode 1/4/8/24/32bpp
+  with the AND mask), and decompression-bomb caps enforced from the container
+  and IHDR headers before any decode.
+- Native GIF decode through libvips' bundled libnsgif loader (first frame,
+  matching the ImageMagick `[0]` semantics) and GIF output through cgif.
+- JPEG XL support end to end: native libvips loader/saver, ImageMagick coder
+  and policy allowlisting, remote content-type/extension handling, and a
+  committed test fixture.
+- Native letter avatars rendered through libvips' Pango text support with the
+  glyph blended in one linear transform; the gem now bundles DejaVu Sans (see
+  `lib/safe_image/fonts/DEJAVU-LICENSE`) so the default font renders
+  identically on every host with no font packages installed.
+- Header-only native metadata reads: `frame_count`/`animated?` from the
+  n-pages field and `orientation` from the orientation field — no pixel decode
+  and no `identify` subprocess for natively supported formats.
+- `fix_orientation` lossless tier: MCU-aligned JPEGs are transformed with
+  `jpegtran` (zero generation loss) when installed, falling back to a libvips
+  re-encode with a new `quality:` keyword (default 95).
+- Native `convert` tier: decode through the allowlisted loaders,
+  auto-orient, flatten transparency onto white for JPEG targets (matching the
+  ImageMagick path), re-encode; default JPEG quality 92 when unspecified.
+- `backend:` keyword (`:auto` / `:vips` / `:imagemagick`) across
+  `resize`, `crop`, `downsize`, `convert`, `thumbnail`, `letter_avatar`,
+  `fix_orientation` and `dominant_color`. `:auto` prefers the native path and
+  uses ImageMagick only for capabilities libvips cannot serve; `:vips` fails
+  closed; `:imagemagick` pins the compatibility pipeline.
+- Graceful operation without libvips: the new `SafeImage::VipsUnavailableError`
+  (a subclass of `UnsupportedFormatError`) makes `backend: :auto` route through
+  ImageMagick on hosts without the library, while explicit `backend: :vips`
+  calls fail closed. `SafeImage::VipsGlue.available?` reports the state.
+- `docker/run.sh`: containerised validation against Debian bookworm's packaged
+  libvips 8.14 with no toolchain installed.
+
+### Changed
+
+- **The compiled C extension is gone.** libvips is now bound at runtime
+  through a minimal Fiddle binding (`SafeImage::VipsGlue`) that exposes only
+  the operations the gem invokes. Nothing compiles at gem install time; the
+  only gem dependencies are `fiddle` and `rexml`. Minimum libvips is 8.13
+  (Debian bookworm's 8.14 package is tested); `SAFE_IMAGE_LIBVIPS` overrides
+  the library name.
+- All transform defaults are now native-first: `resize`, `crop`, `downsize`,
+  `convert` and `thumbnail` default to `backend: :auto` (previously
+  ImageMagick for the first three and `:vips` fail-closed for `thumbnail`).
+  Pin `backend: :imagemagick` where byte-similar output with previously
+  generated thumbnails matters.
+- The default letter avatar font is now `DejaVu-Sans` (bundled), and the
+  native renderer centres the glyph's ink box optically; the ImageMagick
+  path's baseline placement (which could clip descenders) remains available
+  via `backend: :imagemagick`. Regenerate cached avatars when switching.
+- `convert_favicon_to_png` extracts the largest ICO entry rather than the
+  last one, and `probe` on ICO reports the largest entry's dimensions from a
+  pure-Ruby directory parse.
+- libvips' GLib warnings about rejected input (e.g. "Not a PNG file") are
+  suppressed by default — failures already surface as exceptions; set
+  `SAFE_IMAGE_VIPS_WARNINGS=1` to restore them for debugging.
+
+### Fixed
+
+- `resize`/`crop`/`downsize`/`convert` with `optimize: true` no longer raise
+  for output formats the optimizer tools cannot handle (GIF, JPEG XL); the
+  optimize pass is skipped instead.
+- In-place `fix_orientation` writes through a sibling tempfile and renames,
+  so libvips never streams its input into itself.
+- Garbage EXIF orientation tags clamp to the valid 1–8 range instead of
+  leaking raw values.
+- The Landlock sandbox grants read access to Ruby's `libdir`, so workers no
+  longer fail to start under `--enable-shared` Rubies installed outside the
+  default read roots (e.g. GitHub Actions' hostedtoolcache builds). Sandbox
+  failures now include the child's stderr in the error message.
+
+### Security
+
+- The bundled ImageMagick `policy.xml` gained write-only `HISTOGRAM`/`INFO`
+  coders (for the dominant-colour backend) and the `JXL` coder, plus a
+  regression test asserting the policy parses completely — ImageMagick's
+  hand-rolled tokenizer silently truncates the file on a stray backtick or
+  apostrophe in a comment.
+- The libjxl loader/saver are deliberately re-enabled from libvips'
+  untrusted-operation block: JPEG XL is part of the supported input surface,
+  and inputs still pass extension routing, pixel caps and (optionally) the
+  Landlock sandbox.
+- The Fiddle binding doubles as an operation allowlist, and a leak-loop test
+  guards GObject reference handling.
+
+## [0.1.0] - 2026-06-09
+
+Initial release: hardened image-processing boundary for untrusted uploads.
+Probing and metadata helpers, thumbnails/resize/crop/downsize/convert through
+a native libvips fast path with an ImageMagick compatibility backend under a
+restrictive bundled policy, JPEG/PNG optimisation (jpegoptim/oxipng/pngquant),
+optional Jpegli encoding, allowlist-based SVG sanitising, SSRF-hardened remote
+fetching with DNS pinning, symlink-safe path handling, 128MP default pixel
+caps, and optional atomic Landlock sandboxing.