image_pack 0.2.0 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 96839f17672b82deb6a21685649f5f577fedeb8d7f07b79410ddce761de9d40d
-  data.tar.gz: a806d3ff58c3a32d17e20df100511e31ad6d582facb9508c202ab928cc7cf7f7
+  metadata.gz: 301f10cb975250ef59e3bf1beae23208aeb3e40d6598721323b93e4ea8d2eee8
+  data.tar.gz: cc8b9139d2a16ebe94f2c32a047202ea9e9fdf04a608c1bc4608eb5d9e53d589
 SHA512:
-  metadata.gz: cd8d2f768961d288cbfede915ee0a30f3fe538508c9d88f48d37cf27bf3f77f603e47ab60d0d4d044f298d6bb35336fad15df8fffabc034ecf04e71eb6d6fe44
-  data.tar.gz: dbe7c27f803a23e9181276c8b4efda825ed14cb172385bf86c48591278e0776a63f4c4c2f1664e2dc6547745addc0268613860bd3e1bf3761ac6ce000022566e
+  metadata.gz: 2f9bbde96c6269da00a9514aa6eb7660024435ddcb05ae10e57e02338ce69b8a3465b6f7380c551986e0b0e067db041c34fc640074b86e3da189442486e2c111
+  data.tar.gz: 906df1d615b6ae0fb0fbb05636fcb8916ff671886b7319aad70801099974bc742da54444fcb858293c4777ffe89a87390c8d4de78d83c1a613314132dbf1661a

data/CHANGELOG.md

@@ -1,5 +1,57 @@
 # Changelog
 
+## 0.2.2
+
+- Fixed native cleanup safety: native contexts are now released through `rb_ensure`, including Ruby exception paths from argument coercion, config reads, native status errors, output `String` allocation, and file-output failures.
+- Fixed libjpeg `longjmp` cleanup for encode/decode/luma-decode transient buffers.
+- Fixed `max_output_size = 0` semantics; all size/dimension limits now consistently treat `0` as disabled.
+- Fixed `compress_pixels(min_ssim:)`: it now uses the raw pixel buffer as the reference instead of a temporary quality-95 seed JPEG, and it respects the requested execution mode.
+- Fixed metadata preservation drift: `strip_metadata: false` now preserves markers across fast, size, and SSIM paths.
+- Fixed EXIF Orientation stripping: when metadata is stripped, orientation is applied to decoded pixels before output.
+- Fixed `progressive:` for `algo: :mozjpeg`; `progressive: false` no longer silently inherits the max-compression progressive scan script.
+- Added decode/luma-decode cancellation checkpoints.
+- Reduced avoidable copying for explicit `execution: :direct` String/pixel inputs.
+- Batched RGBA→RGB scanline encoding instead of writing one line at a time.
+- Decoded SSIM candidate luma directly as grayscale instead of RGB + manual luma conversion.
+- Made output-path writes safer by writing to a temporary file, checking `fclose`, and renaming into place.
+- Added `compress_bytes`, `compress_file`, `optimize_jpeg`, `optimize_bytes`, `optimize_file`, and `build_info`.
+- Added coefficient-level lossless JPEG optimization through `jpeg_read_coefficients` / `jpeg_write_coefficients`; this avoids decode→pixels→re-encode when only optimizing an existing JPEG.
+- Made private native entrypoints private class methods.
+- Added `-fvisibility=hidden` and `IMAGE_PACK_REQUIRE_SIMD=1` build guard.
+
+## 0.2.1
+
+- `ip_compute_ssim_luma_buffer`: rewrote inner accumulators from `double` to
+  `int32_t`. For an 8x8 luma window all partial sums (sum, sum-of-squares,
+  cross-product) fit in 32 bits. GCC was already auto-vectorizing the
+  `double` version on AVX2 (4 lanes × fp64), but int32 doubles the lane
+  count (8 lanes × i32) and uses cheaper integer multiplies. Split the
+  kernel into a fixed-size 8x8 specialization plus a variable edge kernel.
+  Top hot symbol in the SSIM-guarded perf profile.
+- `ip_build_luma_buffer`: split the runtime-strided BT.601 loop into
+  channels==3 and channels==4 specializations with `__restrict__` pointers
+  so the compiler can vectorize per-pixel work.
+- `prepare_encode_row` (RGBA→RGB): `__restrict__` + hoisted width to enable
+  vectorization by the compiler.
+- `ip_malloc_hot`: new helper for buffers we touch in tight loops right
+  after allocation. On Linux it issues `madvise(MADV_HUGEPAGE)` for
+  allocations >= 256 KiB to remove the per-cacheline minor page faults
+  that previously appeared inside `jsimd_*_avx2` hot loops in the perf
+  profile. No-op on macOS / non-Linux. Used for the decoded pixel buffer,
+  the SSIM luma buffer, and the pixel input buffer.
+- SSIM-guarded path: reference and candidate decodes both now use
+  `fast_decode_mode=1` (no fancy upsampling, no block smoothing). The
+  comparison stays apples-to-apples since both sides use the same decode
+  pipeline; ~30% reduction in candidate-decode cost.
+- `extconf.rb`: opt-in `IMAGE_PACK_MARCH=<arch>` env knob for tuned
+  builds (`native`, `x86-64-v3`, etc). Default build stays portable.
+  Also added `-fno-math-errno -fno-trapping-math` to remove libm-related
+  vectorization barriers without changing semantics for the integer hot
+  paths.
+- CI: Linux x86_64 jobs now compile with `IMAGE_PACK_MARCH=x86-64-v3`
+  (AVX2 baseline; covers all current GitHub-hosted runner generations).
+  arm64 / macOS unchanged.
+
 ## 0.2.0
 
 - Added `min_ssim:` to `ImagePack.compress` for SSIM-guarded JPEG compression.

data/README.md

@@ -2,7 +2,7 @@
 
 `image_pack` is a Ruby-native JPEG compressor prototype for Ruby `>= 3.4.0`.
 
-Current version: `0.2.0`.
+Current version: `0.2.2`.
 
 This pure-C variant intentionally removes Jpegli and any C++ toolchain requirement.
 The native layer is written in C and links against vendored MozJPEG, which is libjpeg-compatible and based on libjpeg-turbo.
@@ -14,8 +14,8 @@ ImagePack.compress(input, algo: :jpeg_turbo)
 ImagePack.compress(input, algo: :mozjpeg)
 ```
 
-- `:jpeg_turbo` — fast libjpeg-compatible mode with MozJPEG-specific size optimizations disabled.
-- `:mozjpeg` — size-oriented mode using progressive/optimized Huffman coding through the MozJPEG/libjpeg API.
+- `:jpeg_turbo` / `:fast` — fast libjpeg-compatible mode with MozJPEG-specific size optimizations disabled.
+- `:mozjpeg` / `:size` — size-oriented mode using optimized Huffman coding and optional progressive output through the MozJPEG/libjpeg API.
 
 Both modes produce ordinary `.jpg` files.
 
@@ -52,6 +52,14 @@ ImagePack.compress(jpeg, algo: :mozjpeg, min_ssim: 0.985)
 ImagePack.compress(jpeg, algo: :mozjpeg, quality: 75, min_ssim: 0.985)
 
 ImagePack.compress("photo.jpg", output: "photo.optimized.jpg", algo: :mozjpeg)
+ImagePack.compress_file("photo.jpg", output: "photo.optimized.jpg", algo: :mozjpeg)
+ImagePack.compress_bytes(jpeg, algo: :fast, quality: 82)
+
+# Lossless coefficient-level JPEG optimization/transcode.
+# This rewrites JPEG coefficients without decoding/re-encoding pixels.
+ImagePack.optimize_jpeg(jpeg, progressive: true, strip_metadata: false)
+ImagePack.optimize_file("photo.jpg", output: "photo.lossless.jpg")
+ImagePack.optimize_bytes(jpeg)
 
 ImagePack.compress_pixels(rgb_buffer,
   width: 1920,
@@ -69,19 +77,32 @@ ImagePack.inspect_image(jpeg)
 `compress(input, ...)` accepts JPEG input only:
 
 - binary `String` (`ASCII-8BIT`) — JPEG bytes;
-- non-binary `String` — file path;
+- non-binary `String` that points to an existing file — file path;
 - `Pathname` — file path;
 - `IO::Buffer` — JPEG bytes, copied before native no-GVL execution for safety.
 
+For non-ambiguous call sites prefer `compress_bytes(bytes, ...)` or `compress_file(path, ...)`.
+
 `compress_pixels(buffer, ...)` accepts raw pixels as binary `String` or `IO::Buffer`.
-`channels` must be `1`, `3`, or `4`. Alpha in RGBA input is dropped in v0.2.0.
+`channels` must be `1`, `3`, or `4`. Alpha in RGBA input is dropped in v0.2.2; pass `drop_alpha: true` to make that explicit, or `drop_alpha: false` to reject it.
+
+`output: nil` returns binary JPEG bytes. `output: String/Pathname` writes through a temporary file and renames it into place.
+Streaming output is intentionally not supported in v0.2.2.
+
+## Lossless JPEG optimize
 
-`output: nil` returns binary JPEG bytes. `output: String/Pathname` writes to path and returns `true`.
-Streaming output is intentionally not supported in v0.2.0.
+`optimize_jpeg`, `optimize_bytes`, and `optimize_file` are coefficient-level JPEG transcode helpers. They use `jpeg_read_coefficients` / `jpeg_write_coefficients`, so pixels are not decoded and re-encoded. This is the preferred path when you only want to rewrite an existing JPEG with optimized Huffman tables and optional progressive scans.
+
+Defaults are intentionally conservative:
+
+- `progressive: true` creates a progressive optimized JPEG;
+- `strip_metadata: false` preserves APP/COM metadata by default because this path is meant to be visually/losslessly safe.
+
+If `strip_metadata: true` is requested and the source JPEG has EXIF Orientation, `optimize_jpeg` raises `UnsupportedError` instead of silently removing the orientation tag and changing how viewers display the image. Use `strip_metadata: false` for coefficient-level optimization, or `compress(..., strip_metadata: true)` if you want pixel-level orientation normalization.
 
 ## SSIM guard
 
-`compress` accepts `min_ssim:` for JPEG input. This enables a native guarded path:
+`compress` and `compress_pixels` accept `min_ssim:`. This enables a native guarded path:
 
 1. decode the original JPEG to reference pixels;
 2. encode trial JPEG candidates with the existing MozJPEG/libjpeg backend;
@@ -96,8 +117,9 @@ the encoder raises quality only if the candidate violates the SSIM floor.
 If no quality can satisfy the requested floor, `ImagePack::QualityConstraintError`
 is raised. With `execution: :auto`, guarded compression uses the no-GVL/offload
 path instead of the small-image direct path because it may encode/decode several
-candidates. `min_ssim` is intentionally not supported by `compress_pixels` in
-this release.
+candidates. For `compress_pixels`, the reference is the raw pixel buffer itself, not a seed JPEG. RGBA + `min_ssim` is rejected because JPEG cannot represent alpha.
+
+The metric is a fast native luma SSIM-like guard based on 8x8 blocks, not a full Wang-style Gaussian-window reference implementation. It is intended as a compression guard, not as a general-purpose image quality benchmark.
 
 ## Execution modes
 
@@ -115,8 +137,19 @@ ImagePack.compress(jpeg, execution: :auto)
 
 ## Cancellation
 
-`cancellable: true` is supported only for `algo: :mozjpeg` and `execution: :nogvl`, `:offload`, or `:auto`.
-Cancellation is cooperative at encoder scanline checkpoints, not instant.
+`cancellable: true` is supported for `execution: :nogvl`, `:offload`, or `:auto`.
+Cancellation is cooperative at decode, encode, and SSIM-search checkpoints, not instant.
+
+## Metadata / EXIF orientation
+
+`strip_metadata: true` removes metadata. If the source JPEG contains EXIF Orientation, `image_pack` applies that orientation to decoded pixels before stripping metadata, so the visual orientation is preserved. With `strip_metadata: false`, APP/COM markers are preserved across both fast and size-oriented paths.
+
+## Build info
+
+```ruby
+ImagePack.build_info
+# => { version: "0.2.2", mozjpeg: "4.1.5", simd: true }
+```
 
 ## Vendoring
 
@@ -130,11 +163,18 @@ bundle exec rake compile
 
 `rake vendor` pins MozJPEG `v4.1.5`.
 
+## Current limitations
+
+- Pixel-level `compress` rejects CMYK/YCCK JPEG input because it decodes to RGB/gray before re-encoding. Use `optimize_jpeg` for coefficient-level lossless optimization of existing CMYK/YCCK JPEGs.
+- Arithmetic-coded JPEG support is disabled in the vendored `jconfig.h` for v0.2.2.
+- The SSIM guard is a fast 8x8 luma block metric and still assumes quality/SSIM monotonicity during binary search.
+- `compress(input, ...)` still has a legacy path-vs-bytes heuristic for non-binary `String`; prefer `compress_bytes` / `compress_file` and `optimize_bytes` / `optimize_file` in new code.
+
 ## What is intentionally not included
 
 - Jpegli / C++ code
 - AVIF/WebP/PNG
 - FFI
 - shell-out
-- tempfiles by default
+- external tempfiles for in-memory output
 - byte-size targets / max-bytes policy

data/ext/image_pack/extconf.rb

@@ -186,10 +186,17 @@ def detect_simd_arch
 end
 
 def find_nasm
-  ENV["AS_NASM"].presence_or_nil ||
+  presence(ENV["AS_NASM"]) ||
     %w[nasm yasm].find { |bin| system("which #{bin} >/dev/null 2>&1") }
 end
 
+def presence(string)
+  return nil if string.nil?
+  return nil if string.empty?
+
+  string
+end
+
 def write_mozjpeg_config_headers!(build_dir, with_simd:)
   with_simd_flag = with_simd ? "#define WITH_SIMD 1" : "#undef WITH_SIMD"
 
@@ -287,7 +294,11 @@ def select_simd_backend(mozjpeg_dir, arch)
 
   when :x86_64
     if find_nasm.nil?
-      warn "image_pack: x86_64 detected but NASM/YASM not found on PATH. Falling back to scalar; install nasm for ~3-4x speedup."
+      message = "image_pack: x86_64 detected but NASM/YASM not found on PATH."
+      if ENV["IMAGE_PACK_REQUIRE_SIMD"] == "1"
+        abort "#{message} Install nasm/yasm or unset IMAGE_PACK_REQUIRE_SIMD."
+      end
+      warn "#{message} Falling back to scalar; install nasm for ~3-4x speedup. Set IMAGE_PACK_REQUIRE_SIMD=1 to fail instead."
       { kind: :none }
     elsif !X86_64_C_SOURCES.all? { |rel| File.exist?(File.join(mozjpeg_dir, rel)) }
       warn "image_pack: x86_64 SIMD sources missing under #{mozjpeg_dir}/simd/x86_64/. Falling back to scalar."
@@ -350,21 +361,6 @@ def write_neon_compat_header!(mozjpeg_dir)
   target
 end
 
-class String
-  unless instance_methods.include?(:presence_or_nil)
-    def presence_or_nil
-      empty? ? nil : self
-    end
-  end
-end
-class NilClass
-  unless instance_methods.include?(:presence_or_nil)
-    def presence_or_nil
-      nil
-    end
-  end
-end
-
 def configure_vendored_mozjpeg(vendor_dir)
   versions = File.read(File.join(vendor_dir, ".vendored"))
   mozjpeg_dir = File.join(vendor_dir, "mozjpeg")
@@ -497,6 +493,14 @@ $warnflags = ""
 
 unless msvc?
   $CFLAGS += " -O3 -Wall -Wextra -Wno-unused-parameter -Wno-sign-compare -std=gnu11"
+  $CFLAGS += " -fno-math-errno -fno-trapping-math -fvisibility=hidden"
+
+  march = ENV["IMAGE_PACK_MARCH"].to_s
+  unless march.empty?
+    $CFLAGS += " -march=#{march}"
+    $CFLAGS += " -mtune=#{march}" if march == "native"
+    puts "image_pack: using -march=#{march}"
+  end
 else
   $CFLAGS += " -O2"
 end