pure_jpeg 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e09848a734582d7635ff6a8c3d85b166da4f2c5b496d5c4f4e30105af4834e33
-  data.tar.gz: 052b8e0a21d58eb9aa9e169e06b7cfbde44f3610775da6fed86e05875dceaea3
+  metadata.gz: 66b5d6fe1b663128f62aae8111b55a9b2ddbe9739d501fcf1146c459286433da
+  data.tar.gz: 02ae6cdc25f520221fee4adfe9d6070c4a975602e5de2b34a0b9ab8b4e005829
 SHA512:
-  metadata.gz: 750ea3d65bf2ae6c272998b2ef7b814954eb1b576f6df83036d931543f30b99481281e8a12ef63fc388fdc60db0c54815fc8219b66782415844b3ed8721a5975
-  data.tar.gz: 16475c5174b009a7a45eec5ee288799ea7965e4e10ed02ab20d55b15de5ebbdf92e9c0196d58053b6539dfea4601d7e22016c636ee471d2e73300feb86d97f07
+  metadata.gz: f476b8fec25f1f0402f297d534f52887aed90778ddf1217668a0889755856436dae8a0d8e4e9d648bc2a33879165b32fa0560e5b506549467c21a648fd6ecf29
+  data.tar.gz: 5438c161519149458fad8cd28dea1f9f073a2646c5f91f619a357d375b77456ed5f366dd3ed78e75f5c0d2cbe4156f4ec8a5a6dbaa9e19d86198dcd8a2fcacfc

data/CHANGELOG.md

@@ -1,5 +1,47 @@
 # Changelog
 
+## 0.3.0
+
+New features:
+
+- `PureJPEG.info` for reading dimensions and metadata without full decode
+- ICC color profile extraction (available on `Info` and `Image`)
+- Optional image-specific optimized Huffman tables (`optimize_huffman: true`)
+
+Fixes:
+
+- Decoder validates Huffman table, quantization table, and component references with clear error messages
+- Color decoding looks up Y/Cb/Cr components by ID instead of assuming SOF array order
+- Support for non-standard component IDs (e.g. 0, 1, 2 as used by some Adobe tools)
+- Explicit error for unsupported component counts (e.g. CMYK)
+- Encoder no longer holds file handle open during encoding
+
+## 0.2.0
+
+New features:
+
+- Progressive JPEG decoding (SOF2) with spectral selection and successive approximation
+- `Image#each_rgb` for iterating pixels without per-pixel struct allocation
+- `PureJPEG::DecodeError` exception class for all decoding errors
+- Validation of custom quantization tables (length and value range)
+
+Performance:
+
+- Packed integer pixel storage in `Image` eliminates per-pixel object allocation on decode (~6x faster decode)
+- Fast path for encoder pixel extraction from packed sources (`ChunkyPNGSource`, `Image`)
+- `BitReader#read_bits` fast path when buffer already has enough bits
+- `BitWriter` builds a `String` directly instead of `Array` + `pack`
+- `Huffman.build_table` returns an `Array` for O(1) lookup instead of `Hash`
+- Faster scan data extraction using `String#index`
+
+Fixes:
+
+- JPEG data detection uses SOI marker check instead of null-byte heuristic
+- `RawSource` pixels default to black instead of `nil`
+- `BitReader` bounds check for truncated 0xFF sequences
+- `JFIFReader` bounds check when reading past end of data
+- Fixed dead tautological check in AC encoding EOB logic
+
 ## 0.1.0
 
 Initial release.

data/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2025 Peter Cooper
+Copyright (c) 2026 Peter Cooper
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,6 +1,15 @@
-# PureJPEG
+<p align="center">
+  <img src="purejpeg.jpg" width="480" alt="PureJPEG">
+</p>
 
-Pure Ruby JPEG encoder and decoder. Implements baseline JPEG (DCT, Huffman, 4:2:0 chroma subsampling) and exposes a variety of encoding options to adjust parts of the JPEG pipeline not normally available (I needed this to recreate the JPEG compression styles of older digital cameras - don't ask..)
+# PureJPEG - Pure Ruby JPEG encoder and decoder library
+
+Convert PNG or other pixel data to JPEG. Or the other way! Implements baseline JPEG encoding (DCT, Huffman, 4:2:0 chroma subsampling) and decodes both baseline and progressive JPEGs. Exposes a variety of encoding options to adjust parts of the JPEG pipeline not normally available (I needed this to recreate the JPEG compression styles of older digital cameras - don't ask..)
+
+It works on CRuby 3.0+, TruffleRuby 33.0, and JRuby 10.0.
+
+> [!NOTE]
+> Rubyists might find the [AI Disclosure](#ai-disclosure) section below of interest.
 
 ## Installation
 
@@ -14,7 +23,7 @@ gem "pure_jpeg"
 gem install pure_jpeg
 ```
 
-There are no runtime dependencies. [ChunkyPNG](https://github.com/wvanbergen/chunky_png) is optional and for if you want to use `from_chunky_png`. I have a pure PNG encoder/decoder not far behind this that will ultimately plug in nicely too to get pure Ruby graphical bliss ;-)
+There are no runtime dependencies. [ChunkyPNG](https://github.com/wvanbergen/chunky_png) is optional (though quite useful) if you want to use `from_chunky_png`. I have a pure PNG encoder/decoder not far behind this that will ultimately plug in nicely too to get 100% pure Ruby graphical bliss ;-)
 
 `examples/` contains some useful example scripts for basic JPEG to PNG and PNG to JPEG conversion if you want to do some quick tests without writing code.
 
@@ -70,13 +79,42 @@ PureJPEG.encode(source,
   luminance_table: nil,           # custom 64-element quantization table for Y
   chrominance_table: nil,         # custom 64-element quantization table for Cb/Cr
   quantization_modifier: nil,     # proc(table, :luminance/:chrominance) -> modified table
-  scramble_quantization: false    # intentionally misordered quant tables (creative effect)
+  scramble_quantization: false,   # intentionally misordered quant tables (creative effect)
+  optimize_huffman: false         # slower 2-pass encode, usually smaller files
 )
 ```
 
 See [CREATIVE.md](CREATIVE.md) for detailed examples of the creative encoding options.
 
-Each stage of the JPEG pipeline is a separate module, so individual components (DCT, quantization, Huffman coding) can be replaced or extended independently which is kinda my plan here as I made this to play around with effects.
+Here's a quick example of sort of the "old digital camera" effect I was looking for though:
+
+<table>
+<tr>
+<td align="center"><strong>Normal</strong></td>
+<td align="center"><strong>Scrambled quantization</strong></td>
+</tr>
+<tr>
+<td><img src="examples/peppers.jpg" width="360"></td>
+<td><img src="examples/peppers-funky.jpg" width="360"></td>
+</tr>
+</table>
+
+And here's what happens when you convert a PNG with transparency — JPEG doesn't support alpha, so the hidden RGB data behind transparent pixels bleeds through:
+
+<table>
+<tr>
+<td align="center"><strong>PNG with transparency</strong></td>
+<td align="center"><strong>Converted to JPEG</strong></td>
+</tr>
+<tr>
+<td><img src="examples/dice.png" width="360"></td>
+<td><img src="examples/dice.jpg" width="360"></td>
+</tr>
+</table>
+
+I consider this a feature but you may consider it a deficiency and that a default background of white should be applied. This may be something I'll add if anyone wants it!
+
+Note that each stage of the JPEG pipeline is a separate module, so individual components (DCT, quantization, Huffman coding) can be replaced or extended independently which is kinda my plan here as I made this to play around with effects.
 
 ## Decoding (reading JPEGs!)
 
@@ -98,6 +136,16 @@ pixel.b  # => 97
 image = PureJPEG.read(jpeg_bytes)
 ```
 
+### Read dimensions and metadata only
+
+```ruby
+info = PureJPEG.info("photo.jpg")
+info.width           # => 1024
+info.height          # => 768
+info.component_count # => 3
+info.progressive     # => false
+```
+
 ### Iterating pixels
 
 ```ruby
@@ -134,27 +182,30 @@ Encoding:
 - 8-bit precision
 - Grayscale (1 component) and YCbCr color (3 components)
 - 4:2:0 chroma subsampling (color) or no subsampling (grayscale)
-- Standard Huffman tables (Annex K)
+- Standard Huffman tables (Annex K) by default
+- Optional image-specific optimized Huffman tables
 
 Decoding:
-- Baseline DCT (SOF0)
+- Baseline DCT (SOF0) and Progressive DCT (SOF2)
 - 8-bit precision
 - 1-component (grayscale) and 3-component (YCbCr) images
 - Any chroma subsampling factor (4:4:4, 4:2:2, 4:2:0, etc.)
 - Restart markers (DRI/RST)
 
-Not supported: progressive JPEG (SOF2), arithmetic coding, 12-bit precision, multi-scan, EXIF/ICC profile preservation. Largely because I don't need these, but they are all do-able, especially with how loosely coupled this library is internally. Raise an issue if you really care about them!
+Not supported: arithmetic coding, 12-bit precision, EXIF/ICC profile preservation, adding a default background for transparent sources (see what happens above!). Largely because I don't need these, but they are all do-able, especially with how loosely coupled this library is internally. Raise an issue if you really care about them!
+
+Possible future improvements: AAN/fixed-point DCT (but it's a LOT of work), ICC profile rendering/conversion.
 
 ## Performance
 
-On a 1024x1024 image (Ruby 3.4 on my M1 Max):
+On a 1024x1024 image (Ruby 4.0.1 on my M1 Max):
 
 | Operation | Time |
 |-----------|------|
-| Encode (color, q85) | ~2.8s |
-| Decode (color) | ~12s |
+| Encode (color, q85) | ~1.7s |
+| Decode (color) | ~1.8s |
 
-The encoder uses a separable DCT with a precomputed cosine matrix and reuses all per-block buffers to minimize GC pressure (more on the optimizations below).
+Both the encoder and decoder use a separable DCT with a precomputed cosine matrix and reuse all per-block buffers to minimize GC pressure. Pixel data is stored as packed integers internally to avoid per-pixel object allocation.
 
 ## Some useful `rake` tasks
 
@@ -167,13 +218,19 @@ rake profile     # CPU profile with StackProf (requires the stackprof gem)
 
 ## AI Disclosure
 
-Claude Code did the majority of the work. However, it did require a lot of guidance as it was quite naive in its approach at first with its JPEG outputs looking very akin to those of my Kodak digital camera from 2001! It turns out it got something wrong which, amusingly, it seems devices of those era also got wrong (specifically not using the zigzag approach during quanitization).
+**Claude Code did the majority of the work.** The math of JPEG encoding/decoding is beyond me, except 'getting it' at a high level. I understand it like I understand the engine in my car :-) *Later update: OpenAI Codex is also reviewing and adding features now. It feels stronger in many areas.*
+
+**I have read all of the code produced up to v0.2.0.** The algorithms are above my paygrade, but I'm OK with what has been produced, and I manually fixed a variety of stylistic things along the way. For example, CC seems to like wrapping entire functions in `if` statements rather than bailing on the opposite condition. *Later update: I have not read the ICC and optimized Huffman code yet, but it is heavily tested.*
+
+**CC needed a lot of guidance.** Its initial JPEG algorithm was somewhat naive and output odd looking JPEGs akin to those of my Kodak digital camera from 2001. After some back and forth and image comparisons, we figured out it was doing the quantization entirely wrong (specifically not using the zigzag approach during quanitization but just going in raster order). I *like* this aesthetic, but fixed it up so that it works as a generally usable JPEG library, while adding ways to customize things so you can recreate the effect, if preferred (see `CREATIVE.md` for more on that).
+
+**CC is lazy.** The initial implementation was VERY SLOW. It took 15 seconds to turn a 1024x1024 PNG into a JPEG, so we went down the profiling rabbit hole and found many optimizations to make it ~6x faster. CC is poor at considering the role of Ruby's GC when implementing low level algorithms and needs some prodding to make the correct optimizations. CC is also lazy to the point of recommending that you just use another language (e.g. Go or Rust) rather than do a pure Ruby version of something - despite it being possible with some extra work.
 
-The initial implementation was also VERY SLOW. It took about 15 seconds just to turn a 1024x1024 PNG into a JPEG, so some profiling was necessary which ended up finding a lot of possible optimizations to make it about 6x faster.
+**CC's testing and cleanliness leaves a bit to be desired.** The CC-created tests were superficial, so I worked on getting them beefed up to tackle a variety of edge cases. They could still get better. It also didn't do RDoc comments, use Minitest, and a variety of other things I coerced it into working on. A good `CLAUDE.md` file could probably avoid many of these problems. I worked without one.
 
-The tests were also a bit superficial, so I worked on getting them beefed up to tackle a variety of edge cases, although they could still be better. It also didn't do RDoc comments, use Minitest, and a variety of other things I had to coerce it into finishing.
+**The overall experience was good.** I enjoyed this project, but CC clearly requires an experienced developer to keep it on the rails and to not end up with a bunch of buggy half-working crap. Getting to the basic 'turn a PNG into a JPEG' took only twenty minutes, but the rest of making it actually widely useful took several hours more.
 
-I have read all of the code produced. A lot of the internals are above my paygrade but I'm generally OK with what has been produced and fixed a variety of stylistic things along the way.
+**The final 10% still takes 90% of the time.** As mentioned above, the first run was quick, but getting things right has taken much longer. v0.1->0.2 has taken longer than 0.1 did! But we now have progressive JPEG support, even more optimizations, better tests, etc. etc.
 
 ## License
 

data/lib/pure_jpeg/bit_reader.rb

@@ -18,6 +18,12 @@ module PureJPEG
 
     def read_bits(n)
       return 0 if n == 0
+      # Fast path: enough bits already in the buffer
+      if @bits_in_buffer >= n
+        @bits_in_buffer -= n
+        return (@buffer >> @bits_in_buffer) & ((1 << n) - 1)
+      end
+      # Slow path: need to refill
       value = 0
       n.times { value = (value << 1) | read_bit }
       value
@@ -43,10 +49,11 @@ module PureJPEG
     private
 
     def fill_buffer
-      raise "Unexpected end of scan data" if @pos >= @length
+      raise PureJPEG::DecodeError, "Unexpected end of scan data" if @pos >= @length
       byte = @data.getbyte(@pos)
       @pos += 1
       if byte == 0xFF
+        raise PureJPEG::DecodeError, "Unexpected end of scan data" if @pos >= @length
         next_byte = @data.getbyte(@pos)
         @pos += 1
         # 0xFF 0x00 is a stuffed 0xFF byte

data/lib/pure_jpeg/bit_writer.rb

@@ -3,7 +3,7 @@
 module PureJPEG
   class BitWriter
     def initialize
-      @data = []
+      @data = String.new(capacity: 4096, encoding: Encoding::BINARY)
       @buffer = 0
       @bits_in_buffer = 0
     end
@@ -17,8 +17,8 @@ module PureJPEG
       while @bits_in_buffer >= 8
         @bits_in_buffer -= 8
         byte = (@buffer >> @bits_in_buffer) & 0xFF
-        @data << byte
-        @data << 0x00 if byte == 0xFF  # byte stuffing
+        @data << byte.chr
+        @data << "\x00".b if byte == 0xFF  # byte stuffing
       end
 
       @buffer &= (1 << @bits_in_buffer) - 1
@@ -32,7 +32,7 @@ module PureJPEG
     end
 
     def bytes
-      @data.pack("C*")
+      @data
     end
   end
 end