omnizip 0.3.18 → 0.3.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 689a5e44cef4327034541bbcdca28c15e75da2d6cb2452a0617365e107bf1597
4
- data.tar.gz: d9bd5f70930e7efdb4fcaaba779927c0b0898e163e79e8e059a990ccc4083c31
3
+ metadata.gz: dcad8b150c8f11ed823e029a054b9c7fb8efcd169482dacad067d6a9ea703d3b
4
+ data.tar.gz: af2b5cec7a79078f1068a148c2c7dc128bb1ec4241aef5a6f7c37058eee10adb
5
5
  SHA512:
6
- metadata.gz: c2ec5fa49e00586c54e7a267de4da9fdcccacd0545e664e81608a5a697ad5430e95143ffd08afa0bafc41b44e17073d0780d694330748e7e35949991a77c9643
7
- data.tar.gz: f79484e4aaa748152c1ea5abca31a08adcfe22207037608e3714e6fac57222c2d65763c16a0b26ef434a9f22ba8122ce4f27cddf9e5d4dc8e0db0c1b9abfe020
6
+ metadata.gz: 3db9493020536b160e63d7519610761f18fb1e77624cb8ebc1ca941bd58be86872e05eafc3985ee53f5767ad84cc095b6013d8a2e25ed3fe22024ba85321b3f0
7
+ data.tar.gz: 83196c0c7630c1cbe05859ca027c7ea5245fcfee56eae2de85e2f7a484c1513d4cc3017885536d22d5fe38fcdcfd84727ed19f313201e7aa3d5190ccc9fe6667
data/CHANGELOG.md CHANGED
@@ -5,6 +5,100 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [Unreleased]
9
+
10
+ ## [0.3.19] - 2026-08-25
11
+
12
+ ### Fixed
13
+ - 7-Zip LZMA2 encoder corrupted every chunk after the first: pos_state was
14
+ computed from the chunk-relative position while decoders count positions
15
+ continuously across chunk boundaries. The stale duplicated encoder is now a
16
+ thin subclass of the fixed XZ Utils encoder (also picking up its state-reset,
17
+ 64 KiB chunk-cap and symbol-queue fixes).
18
+
19
+ ## [0.3.18] - 2026-08-25
20
+
21
+ ### Added
22
+ - Zstandard long-distance matching (LDM): a sparse hash table over the whole
23
+ frame finds matches beyond the 128 KiB block window at levels >= 19 for
24
+ multi-block inputs, bringing zstd-22 output to reference parity
25
+ (0.146 vs omnizip-rs 0.1456 on the benchmark corpus).
26
+
27
+ ## [0.3.17] - 2026-08-25
28
+
29
+ ### Changed
30
+ - Zstandard greedy parser: rep0 fast-path and backward match extension into
31
+ the pending literals (default-level text ratio 0.177 -> 0.167).
32
+
33
+ ## [0.3.16] - 2026-08-24
34
+
35
+ ### Added
36
+ - Zstandard Compressed_Blocks now carry a sequences section: LZ77 match
37
+ finder (greedy/lazy/lazy2) and LL/OF/ML FSE sequence encoding with
38
+ Predefined/FSE_Compressed table selection. Text ratio 0.49 -> ~0.17.
39
+
40
+ ### Changed
41
+ - Migrated ConversionOptions, ParallelOptions, ProgressOptions and EtaResult
42
+ models to lutaml-model (breaking change for hand-rolled hash access).
43
+
44
+ ## [0.3.15] - 2026-08-24
45
+
46
+ ### Fixed
47
+ - Zstandard codec rebuilt per RFC 8878: FSE tables from stream, FSE Huffman
48
+ weights, treeless literals, repeat mode, XXH64 frame checksums, window
49
+ descriptor; `compress` with default options now really compresses instead
50
+ of emitting stored frames (issues #27, BUGREPORT 01-10).
51
+ - XZ LZMA2 encoder: chunking, repeat-offset state and wire format repaired
52
+ (issue #26; missing `tempfile` require, symbol-buffer overflow at ~10 MB).
53
+
54
+ ## [0.3.14] - 2026-07-25
55
+
56
+ ### Changed
57
+ - Documented the Algorithms vs Implementations boundary; deepened the
58
+ Algorithm base class with a class-level facade and IO::Source/Sink.
59
+ - Extracted `Omnizip::Parallel::Engine`.
60
+
61
+ ## [0.3.13] - 2026-07-24
62
+
63
+ ### Changed
64
+ - Reverted the v0.3.12 require-omnizip-everywhere band-aid in favor of
65
+ per-file entry-point requires.
66
+
67
+ ## [0.3.12] - 2026-07-24
68
+
69
+ ### Fixed
70
+ - Circular dependency at gemspec load (`version.rb` no longer requires
71
+ omnizip); missing entry-point requires at the top of internal files.
72
+
73
+ ## [0.3.11] - 2026-07-22
74
+
75
+ ### Fixed
76
+ - Format files broke when required directly by external code; LinkHandler
77
+ symlink gating; BCJ filter autoload.
78
+
79
+ ## [0.3.10] - 2026-07-21
80
+
81
+ ### Changed
82
+ - Migrated CompressionOptions and AlgorithmMetadata to lutaml-model;
83
+ decoupled Convenience from ZIP via ArchiveHandler; replaced `respond_to?`
84
+ type checks with `is_a?`.
85
+
86
+ ## [0.3.9] - 2026-03-24
87
+
88
+ ### Fixed
89
+ - Windows unrar detection; 7-Zip SDK version validation (major version only).
90
+
91
+ ## [0.3.8] - 2026-02-24
92
+
93
+ ### Fixed
94
+ - Formats now auto-register when their files are loaded directly.
95
+
96
+ ## [0.3.7] - 2026-02-23
97
+
98
+ ### Fixed
99
+ - XZ dictionary-size bounds checking (OOM prevention); block header parser
100
+ autoload.
101
+
8
102
  ## [0.3.6] - 2026-02-23
9
103
 
10
104
  ### Changed
@@ -18,8 +112,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
18
112
  - Fixed RAR format `verify` and `repair` convenience methods
19
113
  - Fixed library loading to ensure convenience methods are available at startup
20
114
 
21
- ## [Unreleased]
22
-
23
115
  ### Added
24
116
  - **XAR Format Support**: Full read/write support for XAR (eXtensible ARchive) format
25
117
  - XAR is primarily used on macOS for software packages (.pkg files) and installers
@@ -594,4 +686,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
594
686
  - Native compression support documented
595
687
  - All 6 compression methods explained
596
688
  - Performance characteristics detailed
597
- - Real-world usage examples
689
+ - Real-world usage examples
@@ -20,506 +20,35 @@
20
20
  # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
21
21
  # DEALINGS IN THE SOFTWARE.
22
22
 
23
- require "stringio"
24
-
25
23
  module Omnizip
26
24
  module Implementations
27
25
  module SevenZip
28
26
  module LZMA2
29
- # 7-Zip SDK LZMA2 encoder implementation.
30
- #
31
- # This encoder produces LZMA2 compressed data compatible with 7-Zip format.
32
- # It uses the same LZMA encoding logic as XZ Utils, but with 7-Zip
33
- # format requirements (no EOS marker, no padding).
27
+ # 7-Zip LZMA2 encoder.
34
28
  #
35
- # Key differences from XZ Utils implementation:
36
- # - No EOS marker (raw LZMA2 data ends with 0x00 control byte)
37
- # - No chunk padding (XZ pads to 4-byte boundary)
38
- # - No LZMA2 property byte in data stream (method ID only in container)
29
+ # The LZMA2 stream itself is format-identical to the raw LZMA2
30
+ # layer of the XZ Utils encoder (chunks + 0x00 end marker); the
31
+ # 7-Zip container carries the method properties, so no property
32
+ # byte is written into the data stream (standalone: false).
39
33
  #
40
- # Based on LZMA SDK by Igor Pavlov
41
- # Reference: https://www.7-zip.org/sdk.html
42
- #
43
- # LZMA2 format (as used by 7-Zip):
44
- # - Control byte specifies chunk type and dictionary reset
45
- # - Dictionary size follows in some chunk types
46
- # - Uncompressed size follows in some chunk types
47
- # - Compressed data follows
48
- class Encoder < Base::LZMA2EncoderBase
49
- include Omnizip::Algorithms::LZMA2Const
50
-
51
- # Maximum chunk sizes (from LZMA2 specification)
52
- MAX_UNCOMPRESSED_CHUNK = 2 * 1024 * 1024 # 2MB
53
- MAX_COMPRESSED_CHUNK = 64 * 1024 # 64KB
54
-
55
- # Encoding constants
56
- UINT32_MAX = 0xFFFFFFFF
57
- REPS = 4
58
- MATCH_LEN_MIN = 2
59
-
60
- attr_reader :dict_size, :lc, :lp, :pb, :standalone
61
-
62
- # Initialize 7-Zip SDK LZMA2 encoder
63
- #
64
- # @param dict_size [Integer] Dictionary size (must be power of 2)
65
- # @param lc [Integer] Literal context bits (0-8)
66
- # @param lp [Integer] Literal position bits (0-4)
67
- # @param pb [Integer] Position bits (0-4)
68
- # @param standalone [Boolean] Include property byte (false for 7-Zip)
69
- def initialize(dict_size:, lc: 3, lp: 0, pb: 2, standalone: false)
70
- super
71
-
72
- # Initialize shared state across all chunks
73
- # Using XZ Utils components (tested and working)
74
-
75
- @dictionary = Omnizip::Algorithms::LZMA::Dictionary.new(dict_size)
76
- @state = Omnizip::Algorithms::LZMA::LZMAState.new(0)
77
- @models = Omnizip::Algorithms::LZMA::XzProbabilityModels.new(lc,
78
- lp, pb)
79
- @match_finder = Omnizip::Algorithms::LZMA::MatchFinder.new(@dictionary)
80
- @optimal = Omnizip::Algorithms::LZMA::OptimalEncoder.new(mode: :fast)
81
-
82
- # Track previous byte for literal context
83
- @prev_byte = 0
84
-
85
- # First chunk always resets dictionary (7-Zip compatibility)
86
- @need_dictionary_reset = true
87
- @need_state_reset = false
88
- @need_properties = true
89
- end
90
-
91
- # Encode data with LZMA2 compression
92
- #
93
- # @param data [String] Input data to compress
94
- # @return [String] LZMA2 compressed data (7-Zip format)
95
- def encode(data)
96
- return "" if data.empty?
97
-
98
- output = StringIO.new
99
- output.set_encoding(Encoding::BINARY)
100
-
101
- # Write property byte if standalone mode
102
- if @standalone
103
- prop_byte = encode_dict_size(@dict_size)
104
- output.putc(prop_byte)
105
- end
106
-
107
- # Reset match finder state for each encoding session
108
- @match_finder.reset
109
-
110
- # Process in chunks
111
- input = StringIO.new(data)
112
- input.set_encoding(Encoding::BINARY)
113
-
114
- while !input.eof?
115
- chunk_data = input.read(MAX_UNCOMPRESSED_CHUNK)
116
- break if chunk_data.nil? || chunk_data.empty?
117
-
118
- chunk = encode_chunk(chunk_data)
119
- output.write(chunk)
120
-
121
- # Update reset flags for next chunk
122
- @need_dictionary_reset = false
123
- @need_state_reset = false
124
- @need_properties = false
125
- end
126
-
127
- # End of stream marker (0x00)
128
- output.write(Omnizip::Algorithms::LZMA2::LZMA2Chunk.end_chunk.to_bytes)
129
-
130
- output.string
131
- end
132
-
133
- # Get implementation identifier
34
+ # This is a thin subclass rather than a copy: the XZ Utils
35
+ # encoder carries the fixes for global pos_state across
36
+ # chunks, announced state resets, the 64 KiB compressed-chunk
37
+ # cap (16-bit size field) and the range-encoder symbol-queue
38
+ # drain, which a duplicated implementation silently missed.
39
+ class Encoder < Implementations::XZUtils::LZMA2::Encoder
40
+ # 7-Zip callers pass the same option keys; only the property
41
+ # byte default differs (the container describes the coder).
42
+ def initialize(options = {})
43
+ super({ standalone: false }.merge(options))
44
+ end
45
+
46
+ # Get implementation identifier.
134
47
  #
135
48
  # @return [Symbol] :seven_zip_sdk
136
49
  def implementation_name
137
50
  :seven_zip_sdk
138
51
  end
139
-
140
- private
141
-
142
- # Encode a single chunk with LZMA2 compression
143
- #
144
- # Uses XZ Utils encoding logic (tested and compatible)
145
- def encode_chunk(uncompressed_data)
146
- compressed = try_compress(uncompressed_data)
147
-
148
- # Decide: compressed vs uncompressed
149
- # Use compressed if it's actually smaller
150
- if compressed.bytesize >= uncompressed_data.bytesize
151
- # Use uncompressed chunk
152
- chunk = Omnizip::Algorithms::LZMA2::LZMA2Chunk.new(
153
- chunk_type: :uncompressed,
154
- uncompressed_data: uncompressed_data,
155
- compressed_data: "",
156
- need_dict_reset: @need_dictionary_reset,
157
- need_state_reset: false,
158
- need_props: false,
159
- )
160
- # After uncompressed chunk, next chunk needs state reset
161
- @need_state_reset = true
162
- else
163
- # Use compressed chunk
164
- chunk_properties = (((@pb * 5) + @lp) * 9) + @lc
165
- chunk = Omnizip::Algorithms::LZMA2::LZMA2Chunk.new(
166
- chunk_type: :compressed,
167
- uncompressed_data: uncompressed_data,
168
- compressed_data: compressed,
169
- compressed_size: compressed.bytesize,
170
- properties: chunk_properties,
171
- need_dict_reset: @need_dictionary_reset,
172
- need_state_reset: @need_state_reset,
173
- need_props: true,
174
- )
175
- end
176
-
177
- # Update dictionary with the chunk data
178
- @dictionary.append(uncompressed_data)
179
-
180
- # Update prev_byte for next chunk
181
- if uncompressed_data.bytesize.positive?
182
- @prev_byte = uncompressed_data.getbyte(uncompressed_data.bytesize - 1)
183
- end
184
-
185
- chunk.to_bytes
186
- end
187
-
188
- # Try to compress data using LZMA
189
- #
190
- # Uses XZ Utils encoding components (tested and working)
191
- def try_compress(data)
192
- # Create output buffer
193
- output_buffer = StringIO.new
194
- output_buffer.set_encoding(Encoding::BINARY)
195
-
196
- # Create range encoder
197
- encoder = Omnizip::Algorithms::LZMA::XzRangeEncoder.new(output_buffer)
198
-
199
- # Feed all data to match finder first
200
- @match_finder.feed(data)
201
-
202
- # Initialize hash table
203
- match_len_max = 2
204
- end_pos = [
205
- @dictionary.buffer.bytesize + data.bytesize - match_len_max, 0
206
- ].max
207
- @match_finder.skip(end_pos)
208
-
209
- # Position in match finder's buffer for encoding
210
- start_pos = @dictionary.buffer.bytesize
211
- @current_start_pos = start_pos
212
-
213
- pos = 0
214
- while pos < data.bytesize
215
- # Encode queued symbols if buffer getting full
216
- if encoder.count > 20
217
- encode_queued_symbols(encoder, output_buffer)
218
- end
219
-
220
- # Find matches at current position
221
- match_pos = start_pos + pos
222
- @match_finder.find_matches(match_pos)
223
-
224
- # Get optimal encoding choice
225
- distance, length = @optimal.find_optimal(
226
- match_pos,
227
- @match_finder,
228
- @state,
229
- @state.reps,
230
- @models,
231
- )
232
-
233
- # Encode based on choice
234
- if distance == UINT32_MAX || length == 1
235
- encode_literal(data.getbyte(pos), encoder, pos)
236
- pos += 1
237
- elsif distance < REPS
238
- encode_repeated_match(distance, length, encoder, pos, match_pos)
239
- pos += length
240
- else
241
- actual_distance = distance - REPS
242
- encode_match(actual_distance, length, encoder, pos, match_pos,
243
- data)
244
- pos += length
245
- end
246
- end
247
-
248
- # Flush encoder
249
- encode_queued_symbols(encoder, output_buffer)
250
- encoder.queue_flush
251
- encode_queued_symbols(encoder, output_buffer)
252
-
253
- output_buffer.string
254
- end
255
-
256
- # Encode queued symbols to output
257
- def encode_queued_symbols(encoder, output)
258
- return if encoder.none?
259
-
260
- temp_buffer = "\0" * 10000
261
- out_pos = Omnizip::Algorithms::LZMA::IntRef.new(0)
262
-
263
- size_before = output.size
264
-
265
- encoder.encode_symbols(temp_buffer, out_pos, 10000)
266
-
267
- if out_pos.value.positive?
268
- output.write(StringCompat.byteslice(temp_buffer, 0,
269
- out_pos.value))
270
- end
271
-
272
- output.size - size_before
273
- end
274
-
275
- # Compatibility helper for Ruby 3.0-3.1
276
- module StringCompat
277
- if "".respond_to?(:byteslice)
278
- def self.byteslice(string, start, length)
279
- string.byteslice(start, length)
280
- end
281
- else
282
- def self.byteslice(string, start, length)
283
- string.bytes[start, length]&.pack("C*") || ""
284
- end
285
- end
286
- end
287
-
288
- # Encode literal byte
289
- def encode_literal(symbol, encoder, pos)
290
- pos_state = pos & ((1 << @pb) - 1)
291
-
292
- prob_is_match = @models.is_match[@state.value][pos_state]
293
- encoder.queue_bit(prob_is_match, 0)
294
-
295
- literal_offset = get_literal_state(pos, @prev_byte)
296
- use_matched = @state.use_matched_literal?
297
-
298
- @state.update_literal!
299
-
300
- if use_matched
301
- match_pos = @current_start_pos + pos
302
- match_byte_pos = match_pos - @state.reps[0] - 1
303
- match_byte = @match_finder.buffer.getbyte(match_byte_pos) if match_byte_pos >= 0 && match_byte_pos < @match_finder.buffer.bytesize
304
-
305
- if match_byte.nil?
306
- encode_normal_literal(literal_offset, symbol, encoder)
307
- else
308
- encode_matched_literal(literal_offset, match_byte, symbol,
309
- encoder)
310
- end
311
- else
312
- encode_normal_literal(literal_offset, symbol, encoder)
313
- end
314
-
315
- @prev_byte = symbol
316
- end
317
-
318
- # Encode normal match
319
- def encode_match(distance, length, encoder, pos, match_pos,
320
- _input_data)
321
- pos_state = pos & ((1 << @pb) - 1)
322
-
323
- prob_is_match = @models.is_match[@state.value][pos_state]
324
- encoder.queue_bit(prob_is_match, 1)
325
-
326
- prob_is_rep = @models.is_rep[@state.value]
327
- encoder.queue_bit(prob_is_rep, 0)
328
-
329
- # Reps are stored 0-based (distance - 1), matching the decoder's
330
- # rep0 convention.
331
- @state.update_match!(distance - 1)
332
-
333
- encode_match_length(length, pos_state, encoder)
334
- encode_distance(distance, length, encoder)
335
-
336
- last_byte_pos = match_pos - distance + length - 1
337
- @prev_byte = @match_finder.buffer.getbyte(last_byte_pos) if last_byte_pos >= 0 && last_byte_pos < @match_finder.buffer.bytesize
338
- end
339
-
340
- # Encode repeated match
341
- def encode_repeated_match(rep, length, encoder, pos, match_pos)
342
- pos_state = pos & ((1 << @pb) - 1)
343
-
344
- prob_is_match = @models.is_match[@state.value][pos_state]
345
- encoder.queue_bit(prob_is_match, 1)
346
-
347
- prob_is_rep = @models.is_rep[@state.value]
348
- encoder.queue_bit(prob_is_rep, 1)
349
-
350
- prob_is_rep0 = @models.is_rep0[@state.value]
351
- if rep.zero?
352
- encoder.queue_bit(prob_is_rep0, 0)
353
-
354
- prob_is_rep0_long = @models.is_rep0_long[@state.value][pos_state]
355
- encoder.queue_bit(prob_is_rep0_long, length == 1 ? 0 : 1)
356
- else
357
- encoder.queue_bit(prob_is_rep0, 1)
358
-
359
- prob_is_rep1 = @models.is_rep1[@state.value]
360
- if rep == 1
361
- encoder.queue_bit(prob_is_rep1, 0)
362
- else
363
- encoder.queue_bit(prob_is_rep1, 1)
364
-
365
- prob_is_rep2 = @models.is_rep2[@state.value]
366
- encoder.queue_bit(prob_is_rep2, rep - 2)
367
- end
368
-
369
- # XZ Utils rep_match: capture the selected distance BEFORE
370
- # rotating, then shift reps down.
371
- distance = @state.reps[rep]
372
- rep.downto(1) { |i| @state.reps[i] = @state.reps[i - 1] }
373
- @state.reps[0] = distance
374
- end
375
-
376
- if length == 1
377
- @state.update_short_rep!
378
- else
379
- # Rep matches use their own length models (decoder's
380
- # rep_length_coder).
381
- encode_match_length(length, pos_state, encoder,
382
- @models.rep_len_encoder)
383
- @state.update_long_rep!
384
- end
385
-
386
- # Last byte of the match region in the stream
387
- last_byte_pos = match_pos + length - 1
388
- @prev_byte = @match_finder.buffer.getbyte(last_byte_pos) if last_byte_pos >= 0 && last_byte_pos < @match_finder.buffer.bytesize
389
- end
390
-
391
- def get_literal_state(pos, prev_byte)
392
- literal_mask = (0x100 << @lp) - (0x100 >> @lc)
393
- # Factor of 3: each literal context owns a 0x300-sized subcoder
394
- # (XZ Utils literal_subcoder macro); must match the decoder's
395
- # base_offset = 3 * (lit_state << lc).
396
- 3 * ((((pos << 8) + prev_byte) & literal_mask) << @lc)
397
- end
398
-
399
- def encode_normal_literal(literal_offset, symbol, encoder)
400
- context = 1
401
- 8.downto(1) do |i|
402
- bit = (symbol >> (i - 1)) & 1
403
- encoder.queue_bit(@models.literal[literal_offset + context], bit)
404
- context = (context << 1) | bit
405
- end
406
- end
407
-
408
- def encode_matched_literal(literal_offset, match_byte, symbol,
409
- encoder)
410
- offset = 0x100
411
- symbol += 0x100
412
-
413
- while symbol < 0x10000
414
- match_byte <<= 1
415
- match_bit = match_byte & offset
416
- subcoder_index = offset + match_bit + (symbol >> 8)
417
- bit = (symbol >> 7) & 1
418
-
419
- encoder.queue_bit(
420
- @models.literal[literal_offset + subcoder_index], bit
421
- )
422
-
423
- symbol <<= 1
424
- offset &= ~(match_byte ^ symbol)
425
- end
426
- end
427
-
428
- # len_encoder selects the length model set: match_len_encoder for
429
- # normal matches, rep_len_encoder for repeated matches.
430
- def encode_match_length(length, pos_state, encoder,
431
- len_encoder = @models.match_len_encoder)
432
- len = length - 2
433
-
434
- if len < 8
435
- encoder.queue_bit(len_encoder.choice, 0)
436
- encode_bittree(len_encoder.low[pos_state], 3, len, encoder)
437
- elsif len < 16
438
- encoder.queue_bit(len_encoder.choice, 1)
439
- encoder.queue_bit(len_encoder.choice2, 0)
440
- encode_bittree(len_encoder.mid[pos_state], 3, len - 8, encoder)
441
- else
442
- encoder.queue_bit(len_encoder.choice, 1)
443
- encoder.queue_bit(len_encoder.choice2, 1)
444
- encode_bittree(len_encoder.high, 8, len - 16, encoder)
445
- end
446
- end
447
-
448
- def encode_distance(distance, length, encoder)
449
- # get_dist_slot expects a 0-based distance (xz fastpos.h:
450
- # dist 4 -> slot 4, dist 5 -> slot 4).
451
- dist0 = distance - 1
452
- dist_slot = get_dist_slot(dist0)
453
- len_state = [length - 2, 3].min
454
-
455
- encode_bittree(@models.dist_slot[len_state], 6, dist_slot, encoder)
456
-
457
- if dist_slot >= 4
458
- footer_bits = (dist_slot >> 1) - 1
459
- base = (2 | (dist_slot & 1)) << footer_bits
460
- dist_reduced = dist0 - base
461
-
462
- if dist_slot < 14
463
- encode_bittree_reverse(@models.dist_special, dist_reduced,
464
- footer_bits, base - dist_slot - 1, encoder)
465
- else
466
- direct_bits = footer_bits - 4
467
- encoder.queue_direct_bits(dist_reduced >> 4, direct_bits)
468
- align_mask = (1 << 4) - 1
469
- encode_bittree_reverse(@models.dist_align,
470
- dist_reduced & align_mask, 4, 0, encoder)
471
- end
472
- end
473
- end
474
-
475
- def encode_bittree(probs, num_bits, value, encoder)
476
- context = 1
477
- num_bits.downto(1) do |i|
478
- bit = (value >> (i - 1)) & 1
479
- encoder.queue_bit(probs[context], bit)
480
- context = (context << 1) | bit
481
- end
482
- end
483
-
484
- def encode_bittree_reverse(probs, value, num_bits, offset, encoder)
485
- context = 1
486
- num_bits.times do |i|
487
- bit = (value >> i) & 1
488
- encoder.queue_bit(probs[offset + context], bit)
489
- context = (context << 1) | bit
490
- end
491
- end
492
-
493
- def get_dist_slot(distance)
494
- if distance < 4
495
- distance
496
- else
497
- slot = 0
498
- dist = distance
499
- while dist > 3
500
- dist >>= 1
501
- slot += 2
502
- end
503
- slot + dist
504
- end
505
- end
506
-
507
- def encode_dict_size(dict_size)
508
- d = [dict_size, DICT_SIZE_MIN].max
509
-
510
- log2_size = 0
511
- temp = d
512
- while temp > 1
513
- log2_size += 1
514
- temp >>= 1
515
- end
516
-
517
- if d == (1 << log2_size)
518
- [(log2_size - 12) * 2, 40].min
519
- else
520
- [((log2_size - 12) * 2) + 1, 40].min
521
- end
522
- end
523
52
  end
524
53
  end
525
54
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Omnizip
4
- VERSION = "0.3.18"
4
+ VERSION = "0.3.20"
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: omnizip
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.18
4
+ version: 0.3.20
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ribose Inc.
@@ -298,8 +298,6 @@ files:
298
298
  - lib/omnizip/algorithms/ppmd8/restoration_method.rb
299
299
  - lib/omnizip/algorithms/ppmd_base.rb
300
300
  - lib/omnizip/algorithms/registration.rb
301
- - lib/omnizip/algorithms/sevenzip_lzma2.rb
302
- - lib/omnizip/algorithms/xz_lzma2.rb
303
301
  - lib/omnizip/algorithms/zstandard.rb
304
302
  - lib/omnizip/algorithms/zstandard/constants.rb
305
303
  - lib/omnizip/algorithms/zstandard/decoder.rb
@@ -1,119 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- # Copyright (C) 2025 Ribose Inc.
4
- #
5
- # Permission is hereby granted, free of charge, to any person obtaining a
6
- # copy of this software and associated documentation files (the "Software"),
7
- # to deal in the Software without restriction, including without limitation
8
- # the rights to use, copy, modify, merge, publish, distribute, sublicense,
9
- # and/or sell copies of the Software, and to permit persons to whom the
10
- # Software is furnished to do so, subject to the following conditions:
11
- #
12
- # The above copyright notice and this permission notice shall be included in
13
- # all copies or substantial portions of the Software.
14
- #
15
- # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20
- # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
21
- # DEALINGS IN THE SOFTWARE.
22
-
23
- module Omnizip
24
- module Algorithms
25
- # 7-Zip LZMA2 compression algorithm.
26
- #
27
- # This algorithm uses the 7-Zip SDK-compatible implementation of LZMA2,
28
- # which provides compatibility with 7-Zip command-line tools.
29
- #
30
- # @example Compress data
31
- # algorithm = Omnizip::AlgorithmRegistry.get(:sevenzip_lzma2).new(
32
- # dict_size: 8 * 1024 * 1024
33
- # )
34
- # algorithm.compress(input_io, output_io)
35
- #
36
- # @example Decompress data
37
- # algorithm = Omnizip::AlgorithmRegistry.get(:sevenzip_lzma2).new
38
- # algorithm.decompress(input_io, output_io)
39
- class SevenZipLZMA2 < Algorithm
40
- # Get algorithm metadata.
41
- #
42
- # @return [Models::AlgorithmMetadata] Algorithm information
43
- def self.metadata
44
- Models::AlgorithmMetadata.new.tap do |meta|
45
- meta.name = "sevenzip_lzma2"
46
- meta.description = "LZMA2 compression (7-Zip SDK implementation)"
47
- meta.version = "1.0.0"
48
- end
49
- end
50
-
51
- # Compress data using 7-Zip LZMA2.
52
- #
53
- # @param input [IO, String] Input data
54
- # @param output [IO] Output stream
55
- # @return [void]
56
- def compress(input, output)
57
- # Apply filter if set
58
- input_data = if input.is_a?(String)
59
- if @filter
60
- @filter.encode(input)
61
- else
62
- input
63
- end
64
- elsif input.respond_to?(:read)
65
- data = input.read
66
- if @filter
67
- @filter.encode(data)
68
- else
69
- data
70
- end
71
- else
72
- raise ArgumentError, "Input must be a String or IO"
73
- end
74
-
75
- # Get encoding options
76
- dict_size = @options.fetch(:dict_size, 8 * 1024 * 1024)
77
- lc = @options.fetch(:lc, 3)
78
- lp = @options.fetch(:lp, 0)
79
- pb = @options.fetch(:pb, 2)
80
-
81
- # Use existing LZMA2Encoder which wraps SimpleLZMA2Encoder
82
- encoder = LZMA2::LZMA2Encoder.new(
83
- dict_size: dict_size,
84
- lc: lc,
85
- lp: lp,
86
- pb: pb,
87
- standalone: false,
88
- )
89
-
90
- compressed = encoder.encode(input_data)
91
- output.write(compressed)
92
- end
93
-
94
- # Decompress data using 7-Zip LZMA2.
95
- #
96
- # @param input [IO] Input stream
97
- # @param output [IO] Output stream
98
- # @return [void]
99
- def decompress(input, output)
100
- dict_size = @options.fetch(:dict_size, 8 * 1024 * 1024)
101
-
102
- # Use existing LZMA2::Decoder
103
- decoder = LZMA2::Decoder.new(input, raw_mode: true,
104
- dict_size: dict_size)
105
- decompressed = decoder.decode_stream
106
-
107
- # Reverse filter if set
108
- if @filter
109
- decompressed = @filter.decode(decompressed)
110
- end
111
-
112
- output.write(decompressed)
113
- end
114
- end
115
- end
116
- end
117
-
118
- # Register the algorithm
119
- Omnizip::Algorithms::SevenZipLZMA2.register_algorithm
@@ -1,114 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- # Copyright (C) 2025 Ribose Inc.
4
- #
5
- # Permission is hereby granted, free of charge, to any person obtaining a
6
- # copy of this software and associated documentation files (the "Software"),
7
- # to deal in the Software without restriction, including without limitation
8
- # the rights to use, copy, modify, merge, publish, distribute, sublicense,
9
- # and/or sell copies of the Software, and to permit persons to whom the
10
- # Software is furnished to do so, subject to the following conditions:
11
- #
12
- # The above copyright notice and this permission notice shall be included in
13
- # all copies or substantial portions of the Software.
14
- #
15
- # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20
- # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
21
- # DEALINGS IN THE SOFTWARE.
22
-
23
- module Omnizip
24
- module Algorithms
25
- # XZ Utils LZMA2 compression algorithm.
26
- #
27
- # This algorithm uses the XZ Utils implementation of LZMA2,
28
- # which provides full compatibility with xz command-line tools.
29
- #
30
- # @example Compress data
31
- # algorithm = Omnizip::AlgorithmRegistry.get(:xz_lzma2).new(
32
- # dict_size: 8 * 1024 * 1024
33
- # )
34
- # algorithm.compress(input_io, output_io)
35
- #
36
- # @example Decompress data
37
- # algorithm = Omnizip::AlgorithmRegistry.get(:xz_lzma2).new
38
- # algorithm.decompress(input_io, output_io)
39
- class XZLZMA2 < Algorithm
40
- # Get algorithm metadata.
41
- #
42
- # @return [Models::AlgorithmMetadata] Algorithm information
43
- def self.metadata
44
- Models::AlgorithmMetadata.new.tap do |meta|
45
- meta.name = "xz_lzma2"
46
- meta.description = "LZMA2 compression (XZ Utils implementation)"
47
- meta.version = "1.0.0"
48
- end
49
- end
50
-
51
- # Compress data using XZ Utils LZMA2.
52
- #
53
- # @param input [IO, String] Input data
54
- # @param output [IO] Output stream
55
- # @return [void]
56
- def compress(input, output)
57
- input_data = if input.is_a?(String)
58
- input
59
- elsif input.respond_to?(:read)
60
- input.read
61
- else
62
- raise ArgumentError, "Input must be a String or IO"
63
- end
64
-
65
- # Apply filter if set
66
- if @filter
67
- input_data = @filter.encode(input_data)
68
- end
69
-
70
- # Get encoding options
71
- dict_size = @options.fetch(:dict_size, 8 * 1024 * 1024)
72
- lc = @options.fetch(:lc, 3)
73
- lp = @options.fetch(:lp, 0)
74
- pb = @options.fetch(:pb, 2)
75
- standalone = @options.fetch(:standalone, false)
76
-
77
- # Create encoder and compress
78
- encoder = Implementations::XZUtils::LZMA2::Encoder.new(
79
- dict_size: dict_size,
80
- lc: lc,
81
- lp: lp,
82
- pb: pb,
83
- standalone: standalone,
84
- )
85
-
86
- compressed = encoder.encode(input_data)
87
- output.write(compressed)
88
- end
89
-
90
- # Decompress data using XZ Utils LZMA2.
91
- #
92
- # @param input [IO] Input stream
93
- # @param output [IO] Output stream
94
- # @return [void]
95
- def decompress(input, output)
96
- dict_size = @options.fetch(:dict_size, 8 * 1024 * 1024)
97
-
98
- decoder = Implementations::XZUtils::LZMA2::Decoder.new(input,
99
- raw_mode: true, dict_size: dict_size)
100
- decompressed = decoder.decode_stream
101
-
102
- # Reverse filter if set
103
- if @filter
104
- decompressed = @filter.decode(decompressed)
105
- end
106
-
107
- output.write(decompressed)
108
- end
109
- end
110
- end
111
- end
112
-
113
- # Register the algorithm
114
- Omnizip::Algorithms::XZLZMA2.register_algorithm