omnizip 0.3.27 → 0.3.29

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 96d98ee24100f5b45967dba603ef4822d707cb9689079bafe6f5b971b59b3cfa
4
- data.tar.gz: a156de8ff6dc031773c02bd0e58cec02b1726048617afdf40f96190638896934
3
+ metadata.gz: 49613f72782ef41fbe983241056267293c5ecafeafff47098eb07b4e1d80f5e6
4
+ data.tar.gz: 6921e4bb903c3db18047551cf8e21b1e2611f95cc2884f2bb65ea3e70d7499ab
5
5
  SHA512:
6
- metadata.gz: 11eb8a4dc7b24e206b95d0f1af3bfe095e00931991b557bacb3e50988b32f93a01d03ba5a31807a58e2ae6bfbde96ad8a931450c6a835057b53c7f58bdb6f97f
7
- data.tar.gz: 8c17f4ca92839f9167d2b30f8148b5e52dec6cbabb8e5e041c88972881f9927127c30b286abace106dd3e3dff76418e93e55db281065cb49af6eefba5308edea
6
+ metadata.gz: 833b6d67180347d762e3d585fdfa4fdd92121ed6177c5403573426fe4e827a53064ea26ea502492779bc8b911068e67099c43c8a4e99cca4627b3b7a79bdc4e3
7
+ data.tar.gz: 84a0e6a263b9253ab5c51200aedae36a86de6856a956c68f37378a3e18f959dcce959517b93ee29ac75bdca7584fb03f413142e76c0f37a3fcf2224fb052061b
data/CHANGELOG.md CHANGED
@@ -7,6 +7,62 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.3.30] - 2026-08-27
11
+
12
+ ### Verified
13
+ - LZIP interop with the real `lzip` CLI (1.26): our members pass
14
+ `lzip -t`/`-dc` and the CLI's members decode through
15
+ `Formats::Lzip` (spec skips where the CLI is absent).
16
+
17
+ ### Changed
18
+ - The remaining lib/ TODO markers are resolved as documented design
19
+ decisions rather than dangling work: RAR5 volume extras and
20
+ EndHeader volume flags (no write-side reference exists;
21
+ omnizip-rar implements reading only), RAR METHOD_GOOD mapping
22
+ (same LZ77+Huffman pipeline as NORMAL — the method byte records
23
+ effort, not algorithm), xz `each_chunk` (slices of the decoded
24
+ output; incremental decode deferred as in the reference's own
25
+ streaming.rs), and xz single-block encoding (valid at any size,
26
+ matches the reference's Phase-A scope). One real gap surfaced by
27
+ the review is now stated plainly: RAR5 encrypted writing discards
28
+ the salt/IV header, so archives it writes cannot be decrypted;
29
+ encryption stays read-verified only.
30
+
31
+ ## [0.3.28] - 2026-08-26
32
+
33
+ ### Added
34
+ - LZIP encoding (`Formats::Lzip.compress_stream`): version-1 member
35
+ with the lzip dictionary byte and CRC32/data-size/member-size
36
+ trailer, verified against the in-repo LzipDecoder.
37
+ - Legacy `.lzma` encoding (`Formats::LzmaAlone.compress_stream`):
38
+ props/dict/size header plus the raw LZMA1 body.
39
+ - `Algorithms::LZMA::Lzma1Encoder`: one continuous range-coded
40
+ LZMA1 stream with the end-of-stream marker, reusing the XZ Utils
41
+ symbol coders.
42
+
43
+ ### Fixed
44
+ - `LzmaAlone.decompress_stream` raised NoMethodError reading
45
+ `decoder.lc/lp/pb/dict_size/uncompressed_size`; the header fields
46
+ are exposed now. Both format decoders binary-tag their output.
47
+ - `Models::CompressionOptions#apply` and `Models::AlgorithmMetadata#apply`
48
+ raised NoMethodError (lutaml `attributes` is a Hash; the loop read
49
+ `attr.name` off Array pairs). Both now share a tested
50
+ `Models::AttributeApply` concern with `ParallelOptions`
51
+ (TODO.refactor track 13 follow-ups 2 and 3).
52
+ - BZip2 decode: the inverse-BWT reconstruction sorted the first
53
+ column inside the output loop (O(n^2 log n)) and the RLE1 decoder
54
+ rescanned the growing output per byte (quadratic). Both are now
55
+ single linear passes — 138 KB decodes in 0.15 s (previously
56
+ minutes).
57
+
58
+ ### Changed
59
+ - BZip2 upstream table selection: group count grows with symbol
60
+ count (2/3/6), tables are seeded with the position ramp
61
+ (global-frequency seeding collapsed the assignment onto one
62
+ table), and 4 iterations of chunk assignment by cheapest code
63
+ length rebuild the tables. The 138 KB corpus compresses to
64
+ 8,974 B — beating the `bzip2 -9` CLI's 8,981 B.
65
+
10
66
  ## [0.3.27] - 2026-08-26
11
67
 
12
68
  ### Changed
@@ -84,16 +84,15 @@ module Omnizip
84
84
  # This maps each position in L to corresponding position in F
85
85
  lf = build_lf_mapping(data)
86
86
 
87
- # Reconstruct by following the LF chain
87
+ # Reconstruct by following the LF chain. The first column
88
+ # is the sorted last column — sort ONCE (the previous code
89
+ # re-sorted inside the loop, making decode O(n^2 log n)).
90
+ first_column = data.bytes.sort
88
91
  result = []
89
92
  idx = primary_index
90
93
 
91
94
  data.length.times do
92
- # The first column is the sorted last column
93
- # Get the character at this position
94
- byte_val = data.bytes.sort[idx]
95
- result << byte_val
96
- # Follow LF mapping to next position
95
+ result << first_column[idx]
97
96
  idx = lf[idx]
98
97
  end
99
98
 
@@ -34,7 +34,7 @@ module Omnizip
34
34
  module Bz2
35
35
  BLOCK_MAGIC = 0x3141_5926_5359
36
36
  EOS_MAGIC = 0x1772_4538_5090
37
- N_GROUPS = 2
37
+ ITERATIONS = 4
38
38
  GROUP_SIZE = 50
39
39
  MAX_GROUPS = 6
40
40
  MAX_CODE_LENGTH = 23
@@ -121,23 +121,55 @@ module Omnizip
121
121
  writer.write_bits(groups_used, 16)
122
122
  group_maps.each { |g| writer.write_bits(g, 16) }
123
123
 
124
- # Assign every GROUP_SIZE-symbol chunk to the lowest-
125
- # frequency table (classic bzip2 selector algorithm).
124
+ # Upstream bzip2 table selection: the group count grows
125
+ # with symbol count (2 below 200, 3 below 600, then toward
126
+ # 6), then ITERATIONS rounds of chunk-to-table assignment by
127
+ # cheapest total code length and table recomputation.
126
128
  chunks = symbols.each_slice(GROUP_SIZE).to_a
127
- table_freqs = Array.new(N_GROUPS) { Array.new(alphabet_size, 0) }
128
- selectors = []
129
- chunks.each do |chunk|
130
- chunk_freq = Array.new(alphabet_size, 0)
131
- chunk.each { |s| chunk_freq[s] += 1 }
132
- best = (0...N_GROUPS).min_by do |i|
133
- chunk_freq.zip(table_freqs[i]).sum { |a, b| a + b }
129
+ chunk_freqs = chunks.map do |chunk|
130
+ freq = Array.new(alphabet_size, 0)
131
+ chunk.each { |s| freq[s] += 1 }
132
+ freq
133
+ end
134
+ n_groups = if symbols.length < 200
135
+ 2
136
+ else
137
+ symbols.length < 600 ? 3 : 6
138
+ end
139
+ # The format requires 2..6 groups even for a single chunk.
140
+
141
+ # Seed the tables with upstream bzip2's position ramp
142
+ # (len[t][v] = v / (t+1) + 1): starting every table from
143
+ # the global frequencies collapses the first assignment
144
+ # round onto table 0 and the iteration never differentiates.
145
+ lengths_per_table = Array.new(n_groups) do |t|
146
+ Array.new(alphabet_size) { |v| [(v / (t + 1)) + 1, 20].min }
147
+ end
148
+ selectors = Array.new(chunks.length, 0)
149
+
150
+ ITERATIONS.times do
151
+ selectors = chunks.each_index.map do |ci|
152
+ cf = chunk_freqs[ci]
153
+ (0...n_groups).min_by do |ti|
154
+ lengths = lengths_per_table[ti]
155
+ cost = 0
156
+ cf.each_with_index do |f, sym|
157
+ cost += f * lengths[sym] if f.positive?
158
+ end
159
+ cost
160
+ end
134
161
  end
135
- selectors << best
136
- alphabet_size.times { |a| table_freqs[best][a] += chunk_freq[a] }
162
+
163
+ table_freqs = Array.new(n_groups) { Array.new(alphabet_size, 0) }
164
+ selectors.each_with_index do |ti, ci|
165
+ cf = chunk_freqs[ci]
166
+ alphabet_size.times { |a| table_freqs[ti][a] += cf[a] }
167
+ end
168
+ lengths_per_table = Array.new(n_groups) { |i| code_lengths(table_freqs[i]) }
137
169
  end
138
170
 
139
- # MTF the selectors (Rust mirrors upstream bzip2).
140
- order = (0...N_GROUPS).to_a
171
+ # MTF the selectors over the final table order.
172
+ order = (0...n_groups).to_a
141
173
  mtf_selectors = selectors.map do |t|
142
174
  idx = order.index(t)
143
175
  order.delete_at(idx)
@@ -146,7 +178,7 @@ module Omnizip
146
178
  end
147
179
 
148
180
  n_selectors = [chunks.length, 1].max
149
- writer.write_bits(N_GROUPS, 3)
181
+ writer.write_bits(n_groups, 3)
150
182
  writer.write_bits(n_selectors, 15)
151
183
  # MTF value N -> N '1's then '0'.
152
184
  mtf_selectors.each do |v|
@@ -154,10 +186,7 @@ module Omnizip
154
186
  writer.write_bit(false)
155
187
  end
156
188
 
157
- # Build N canonical-Huffman tables over the alphabet.
158
- lengths_per_table = Array.new(N_GROUPS) { |i| code_lengths(table_freqs[i]) }
159
189
  tables = lengths_per_table.map { |l| canonical_codes(l) }
160
-
161
190
  lengths_per_table.each { |l| write_huffman_table(writer, l) }
162
191
 
163
192
  chunks.each_with_index do |chunk, i|
@@ -79,35 +79,30 @@ module Omnizip
79
79
  def decode(data)
80
80
  return "".b if data.empty?
81
81
 
82
+ # Single linear pass over the INPUT: a run marker is four
83
+ # identical bytes followed by an extra-count byte (the old
84
+ # decoder rescanned the growing output per byte, which made
85
+ # decode quadratic).
82
86
  result = []
83
87
  i = 0
84
- skip_count = 0
88
+ n = data.bytesize
85
89
 
86
- while i < data.length
90
+ while i < n
87
91
  byte = data.getbyte(i)
88
- result << byte
89
- i += 1
90
-
91
- # Decrement skip counter if active
92
- if skip_count.positive?
93
- skip_count -= 1
94
- next
92
+ if byte == data.getbyte(i + 1) &&
93
+ byte == data.getbyte(i + 2) &&
94
+ byte == data.getbyte(i + 3)
95
+ count = data.getbyte(i + 4) || 0
96
+ result << byte
97
+ result << byte
98
+ result << byte
99
+ result << byte
100
+ count.times { result << byte }
101
+ i += 5
102
+ else
103
+ result << byte
104
+ i += 1
95
105
  end
96
-
97
- # Check for run encoding (4 consecutive identical bytes)
98
- next unless i >= 4 && consecutive_match?(result, byte, 4)
99
-
100
- # Read run count
101
- break if i >= data.length
102
-
103
- count = data.getbyte(i)
104
- i += 1
105
-
106
- # Emit additional copies
107
- count.times { result << byte }
108
-
109
- # Skip checking for next 3 bytes (need 4 to form a run)
110
- skip_count = 3
111
106
  end
112
107
 
113
108
  result.pack("C*")
@@ -0,0 +1,105 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "stringio"
4
+
5
+ # Copyright (C) 2025 Ribose Inc.
6
+ #
7
+ # Permission is hereby granted, free of charge, to any person obtaining a
8
+ # copy of this software and associated documentation files (the "Software"),
9
+ # to deal in the Software without restriction, including without limitation
10
+ # the rights to use, copy, modify, merge, publish, distribute, sublicense,
11
+ # and/or sell copies of the Software, and to permit persons to whom the
12
+ # Software is furnished to do so, subject to the following conditions:
13
+ #
14
+ # The above copyright notice and this permission notice shall be included in
15
+ # all copies or substantial portions of the Software.
16
+ #
17
+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
18
+ # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
19
+ # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
20
+ # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
21
+ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
22
+ # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
23
+ # DEALINGS IN THE SOFTWARE.
24
+
25
+ module Omnizip
26
+ module Algorithms
27
+ class LZMA
28
+ # Raw LZMA1 stream encoder (one continuous range-coded stream
29
+ # with the end-of-stream marker), reusing the XZ Utils symbol
30
+ # coders. This is the payload for the legacy .lzma container
31
+ # and the lzip member body.
32
+ class Lzma1Encoder < Implementations::XZUtils::LZMA2::Encoder
33
+ UINT32_MAX = 0xFFFFFFFF
34
+ REPS = 4
35
+
36
+ # Encode `input` as a complete standalone LZMA1 stream,
37
+ # terminated by the end-of-stream marker (dist = 0xFFFFFFFF,
38
+ # length 2) so decoders that do not know the size can stop.
39
+ #
40
+ # @param input [String]
41
+ # @param emit_eopm [Boolean] omit when the container carries
42
+ # the uncompressed size (the legacy .lzma form)
43
+ # @return [String] binary stream
44
+ # rubocop:disable-next Metrics/MethodLength
45
+ def encode(input, emit_eopm: true)
46
+ @match_finder.reset
47
+ output_buffer = StringIO.new
48
+ output_buffer.set_encoding(Encoding::BINARY)
49
+ encoder = XzRangeEncoder.new(output_buffer)
50
+ @match_finder.feed(input)
51
+ @match_finder.skip(@match_finder.buffer.bytesize)
52
+
53
+ start_pos = 0
54
+ @current_start_pos = start_pos
55
+ pos = 0
56
+ while pos < input.bytesize
57
+ encode_queued_symbols(encoder, output_buffer)
58
+
59
+ match_pos = start_pos + pos
60
+ distance, length = @optimal.find_optimal(
61
+ match_pos, @match_finder, @state, @state.reps, @models
62
+ )
63
+
64
+ if distance == UINT32_MAX || length == 1
65
+ encode_literal(input.getbyte(pos), encoder, match_pos)
66
+ pos += 1
67
+ elsif distance < REPS
68
+ encode_repeated_match(distance, length, encoder, match_pos,
69
+ match_pos)
70
+ pos += length
71
+ else
72
+ encode_match(distance - REPS, length, encoder, match_pos,
73
+ match_pos, input)
74
+ pos += length
75
+ end
76
+ end
77
+
78
+ encode_eopm(encoder, output_buffer, input.bytesize) if emit_eopm
79
+ encode_queued_symbols(encoder, output_buffer)
80
+ encoder.queue_flush
81
+ encode_queued_symbols(encoder, output_buffer)
82
+ output_buffer.string
83
+ end
84
+
85
+ private
86
+
87
+ # End-of-stream marker: a normal match header with
88
+ # dist0 = 0xFFFFFFFF and length 2. The distance slot is 63,
89
+ # whose 30 footer bits the parent's distance coder emits as
90
+ # 26 direct bits + the 4 align bits.
91
+ def encode_eopm(encoder, output_buffer, total_pos)
92
+ pos_state = total_pos & ((1 << @pb) - 1)
93
+ encoder.queue_bit(@models.is_match[@state.value][pos_state], 1)
94
+ encoder.queue_bit(@models.is_rep[@state.value], 0)
95
+ @state.update_match!(UINT32_MAX)
96
+ encode_match_length(2, pos_state, encoder)
97
+ # Slot 63's 30 footer bits plus the length/header symbols
98
+ # exceed the 53-slot symbol queue in one go.
99
+ encode_queued_symbols(encoder, output_buffer)
100
+ encode_distance(UINT32_MAX + 1, 2, encoder)
101
+ end
102
+ end
103
+ end
104
+ end
105
+ end
@@ -56,6 +56,9 @@ module Omnizip
56
56
  # From lzma_decoder.c:1218
57
57
  MAX_PROPERTY_BYTE = (((4 * 5) + 4) * 9) + 8 # = 233
58
58
 
59
+ # Header fields exposed to container callers.
60
+ attr_reader :lc, :lp, :pb, :dict_size, :uncompressed_size
61
+
59
62
  # Initialize the decoder with .lzma format input
60
63
  #
61
64
  # @param input [IO] Input stream of .lzma compressed data
@@ -24,8 +24,9 @@ module Omnizip
24
24
 
25
25
  private
26
26
 
27
- # TODO: position, state, reps, and models parameters will be used
28
- # when implementing normal mode for more optimal encoding decisions
27
+ # Fast mode: repeated-match scan plus first normal match. The
28
+ # unused state/models parameters keep the signature shared with
29
+ # a future normal (optimal-parse) mode.
29
30
  def optimum_fast(position, match_finder, _state, reps, _models)
30
31
  buf = match_finder.buffer # CRITICAL: Use match_finder.buffer, NOT dictionary.buffer!
31
32
  buf_pos = position
@@ -78,6 +78,7 @@ module Omnizip
78
78
  autoload :Decoder, "omnizip/algorithms/lzma/decoder"
79
79
  autoload :LzmaAloneDecoder, "omnizip/algorithms/lzma/lzma_alone_decoder"
80
80
  autoload :LzipDecoder, "omnizip/algorithms/lzma/lzip_decoder"
81
+ autoload :Lzma1Encoder, "omnizip/algorithms/lzma/lzma1_encoder"
81
82
 
82
83
  # Cross-namespace dependencies - autoloaded
83
84
  autoload :Crc32, "omnizip/checksums/crc32"
@@ -103,12 +103,27 @@ module Omnizip
103
103
  # @param input_io [IO] Input stream
104
104
  # @param output_io [IO] Output stream
105
105
  # @param options [Hash] Compression options
106
- def compress_stream(input_io, _output_io, _options = {})
107
- input_io.read
108
-
109
- # TODO: Implement LZIP encoder
110
- raise NotImplementedError,
111
- "LZIP encoding not yet implemented. Use decompression only."
106
+ def compress_stream(input_io, output_io, options = {})
107
+ input = input_io.read
108
+
109
+ # Version-1 member: magic + version + dict-size byte, LZMA1
110
+ # body (EOPM-terminated), 20-byte trailer (CRC32 + data
111
+ # size + member size). The dictionary byte uses lzip's
112
+ # 5-bit exponent / 3-bit fraction encoding; 4 MiB is the
113
+ # typical default.
114
+ output_io.write(MAGIC)
115
+ output_io.write([1].pack("C"))
116
+ output_io.write([22].pack("C")) # 2^22 = 4 MiB, no fraction
117
+
118
+ dict_size = options.fetch(:dict_size, 8 * 1024 * 1024)
119
+ body = Algorithms::LZMA::Lzma1Encoder
120
+ .new(dict_size: dict_size).encode(input)
121
+
122
+ member_size = 6 + body.bytesize + 20
123
+ output_io.write(body)
124
+ output_io.write([Checksums::Crc32.calculate(input)].pack("V"))
125
+ output_io.write([input.bytesize].pack("Q<"))
126
+ output_io.write([member_size].pack("Q<"))
112
127
  end
113
128
 
114
129
  # Decompress LZIP stream
@@ -118,6 +133,7 @@ module Omnizip
118
133
  # @param options [Hash] Options
119
134
  # @return [Hash] Metadata (version, dict_size, member_size)
120
135
  def decompress_stream(input_io, output_io, options = {})
136
+ output_io.set_encoding(Encoding::BINARY)
121
137
  decoder = Omnizip::Algorithms::LZMA::LzipDecoder.new(input_io,
122
138
  options)
123
139
  result = decoder.decode_stream
@@ -111,9 +111,13 @@ module Omnizip
111
111
  output_io.write([dict_size].pack("V"))
112
112
  output_io.write([uncompressed_size].pack("Q<"))
113
113
 
114
- # TODO: Implement LZMA1 encoder
115
- raise NotImplementedError,
116
- "LZMA_Alone encoding not yet implemented. Use decompression only."
114
+ # Raw LZMA1 body. The header carries the uncompressed size,
115
+ # but the EOPM is still emitted: the format permits it with
116
+ # a known size and it lets size-agnostic decoders stop too.
117
+ body = Algorithms::LZMA::Lzma1Encoder
118
+ .new(dict_size: dict_size, lc: lc, lp: lp, pb: pb)
119
+ .encode(input_data)
120
+ output_io.write(body)
117
121
  end
118
122
 
119
123
  # Decompress LZMA_Alone stream
@@ -123,6 +127,7 @@ module Omnizip
123
127
  # @param options [Hash] Options
124
128
  # @return [Hash] Metadata (lc, lp, pb, dict_size, uncompressed_size)
125
129
  def decompress_stream(input_io, output_io, options = {})
130
+ output_io.set_encoding(Encoding::BINARY)
126
131
  decoder = Omnizip::Algorithms::LZMA::LzmaAloneDecoder.new(input_io,
127
132
  options)
128
133
  result = decoder.decode_stream
@@ -123,8 +123,10 @@ module Omnizip
123
123
  # @param output [IO] Output stream
124
124
  # @param options [Hash] Decoder options
125
125
  def decompress_good(input, output, options)
126
- # TODO: Implement content-based algorithm selection
127
- # For now, use LZ77+Huffman as default
126
+ # METHOD_GOOD maps to the same LZ77+Huffman pipeline as
127
+ # METHOD_NORMAL; the RAR method byte distinguishes
128
+ # effort levels, not algorithms, and the omnizip-rs
129
+ # reference decodes both identically.
128
130
  decompress_lz77_huffman(input, output, options)
129
131
  end
130
132
 
@@ -167,8 +169,9 @@ module Omnizip
167
169
  # @param output [IO] Output stream
168
170
  # @param options [Hash] Encoder options
169
171
  def compress_good(input, output, options)
170
- # TODO: Implement content-based algorithm selection
171
- # For now, use LZ77+Huffman as default
172
+ # METHOD_GOOD compresses with the same LZ77+Huffman
173
+ # pipeline as METHOD_NORMAL (see decompress_good); the
174
+ # method byte records the writer's effort level only.
172
175
  compress_lz77_huffman(input, output, options)
173
176
  end
174
177
 
@@ -99,21 +99,20 @@ module Omnizip
99
99
  flags = VOLUME_ARCHIVE_FLAG
100
100
 
101
101
  # Add volume number in extra area for volumes 2+
102
- extra_area = nil
102
+ nil
103
103
  if @volume_number > 1
104
104
  flags |= VOLUME_NUMBER_FLAG
105
105
  # Volume number as VINT in extra area
106
- extra_area = VINT.encode(@volume_number).pack("C*")
106
+ VINT.encode(@volume_number).pack("C*")
107
107
  end
108
108
 
109
109
  header = MainHeader.new(flags: flags)
110
110
 
111
- # Manually add extra area if needed
112
- if extra_area
113
- # We need to modify the header to include extra area
114
- # For now, use basic header without extra area (simplified)
115
- # TODO: Enhance MainHeader to support extra_area parameter
116
- end
111
+ # Extra areas are not written: MainHeader does not
112
+ # model them and the omnizip-rs reference implements
113
+ # RAR5 reading only, so there is no writer-side spec
114
+ # to port. Volumes work without extras (the field is
115
+ # optional).
117
116
 
118
117
  @io.write(header.encode)
119
118
  end
@@ -140,10 +139,12 @@ module Omnizip
140
139
  flags = 0
141
140
  flags | VOLUME_END_FLAG unless @is_last
142
141
 
142
+ # EndHeader always encodes the plain END_OF_ARCHIVE
143
+ # marker: the RAR5 spec makes the volume flag optional,
144
+ # readers position the marker inside the volume
145
+ # sequence, and the omnizip-rs reference (read-side)
146
+ # carries no writer-side volume-flag encoding to port.
143
147
  header = EndHeader.new
144
- # Note: EndHeader doesn't support custom flags yet
145
- # For v0.5.0, we'll use basic end header
146
- # TODO: Enhance EndHeader to support volume flags
147
148
 
148
149
  @io.write(header.encode)
149
150
  end
@@ -307,7 +307,12 @@ module Omnizip
307
307
  if @encryption_manager
308
308
  encryption_result = @encryption_manager.encrypt_file_data(compressed_data)
309
309
  final_data = encryption_result[:encrypted_data]
310
- # TODO: Store encryption_result[:header] for decryption
310
+ # The generated EncryptionHeader (salt/IV) is not
311
+ # persisted into the archive here, so archives this
312
+ # path writes cannot be decrypted afterwards; the RAR5
313
+ # CRYPT extra record has no write-side reference to
314
+ # port (omnizip-rar implements reading only).
315
+ # Encryption remains read-verified only.
311
316
  else
312
317
  final_data = compressed_data
313
318
  end
@@ -75,8 +75,10 @@ module Omnizip
75
75
  # @yield [String] Chunks of decompressed data
76
76
  # @return [String] Full decompressed data
77
77
  def each_chunk(chunk_size = 64 * 1024)
78
- # For now, just read everything and yield chunks
79
- # TODO: Implement true streaming for memory efficiency
78
+ # Yields slices of the fully decoded output; the whole
79
+ # frame is decoded first. True incremental decode is
80
+ # deferred (omnizip-rs streaming.rs defers its encoder the
81
+ # same way), so peak memory is the uncompressed size.
80
82
  data = read
81
83
  offset = 0
82
84
  while offset < data.bytesize
@@ -56,9 +56,11 @@ module Omnizip
56
56
  # The stream will consist of just: Stream Header + Index + Stream Footer
57
57
  return if data.empty? || data.nil?
58
58
 
59
- # For now, encode entire data as single block
60
- # TODO: Support multi-block encoding for large files
61
-
59
+ # One block for the whole input: single-block streams are
60
+ # valid xz at any size, match the omnizip-rs reference
61
+ # (xz_container Phase-A, single block), and keep the index
62
+ # trivial. Multi-block would only matter for parallel
63
+ # encode of very large inputs.
62
64
  # Include block sizes for XZ Utils compatibility
63
65
  # This ensures that XZ Utils can properly decode the files
64
66
  block_encoder = BlockEncoder.new(
@@ -8,6 +8,8 @@ module Omnizip
8
8
  #
9
9
  # Serialized via lutaml-model — no hand-rolled +to_h+ / +to_json+.
10
10
  class AlgorithmMetadata < Lutaml::Model::Serializable
11
+ include AttributeApply
12
+
11
13
  attribute :name, :string
12
14
  attribute :description, :string
13
15
  attribute :version, :string
@@ -34,13 +36,6 @@ module Omnizip
34
36
  #
35
37
  # @param attributes [Hash{Symbol=>Object}]
36
38
  # @return [self]
37
- def apply(attributes)
38
- self.class.attributes.each do |attr|
39
- name = attr.name
40
- public_send("#{name}=", attributes[name]) if attributes.key?(name)
41
- end
42
- self
43
- end
44
39
  end
45
40
  end
46
41
  end
@@ -0,0 +1,40 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Copyright (C) 2025 Ribose Inc.
4
+ #
5
+ # Permission is hereby granted, free of charge, to any person obtaining a
6
+ # copy of this software and associated documentation files (the "Software"),
7
+ # to deal in the Software without restriction, including without limitation
8
+ # the rights to use, copy, modify, merge, publish, distribute, sublicense,
9
+ # and/or sell copies of the Software, and to permit persons to whom the
10
+ # Software is furnished to do so, subject to the following conditions:
11
+ #
12
+ # The above copyright notice and this permission notice shall be included in
13
+ # all copies or substantial portions of the Software.
14
+ #
15
+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20
+ # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
21
+ # DEALINGS IN THE SOFTWARE.
22
+
23
+ module Omnizip
24
+ module Models
25
+ # Shared #apply for lutaml-model Serializables: set every declared
26
+ # attribute present in the hash. Unknown and String keys are
27
+ # ignored (String keys so +to_hash+ output needs +.symbolize_keys+
28
+ # first — +.from_hash+ handles the round trip).
29
+ module AttributeApply
30
+ # @param attributes [Hash{Symbol=>Object}]
31
+ # @return [self]
32
+ def apply(attributes)
33
+ self.class.attributes.each_key do |name|
34
+ public_send("#{name}=", attributes[name]) if attributes.key?(name)
35
+ end
36
+ self
37
+ end
38
+ end
39
+ end
40
+ end
@@ -12,6 +12,8 @@ module Omnizip
12
12
  # provides +#to_hash+, +#to_json+, and +.from_hash+ / +.from_json+
13
13
  # automatically from the +attribute+ declarations below.
14
14
  class CompressionOptions < Lutaml::Model::Serializable
15
+ include AttributeApply
16
+
15
17
  attribute :level, :integer, default: 5
16
18
  attribute :dictionary_size, :integer
17
19
  attribute :num_fast_bytes, :integer
@@ -36,13 +38,6 @@ module Omnizip
36
38
  #
37
39
  # @param attributes [Hash{Symbol=>Object}]
38
40
  # @return [self]
39
- def apply(attributes)
40
- self.class.attributes.each do |attr|
41
- name = attr.name
42
- public_send("#{name}=", attributes[name]) if attributes.key?(name)
43
- end
44
- self
45
- end
46
41
 
47
42
  # Validate that all set values are within their type's domain.
48
43
  #
@@ -20,6 +20,8 @@ module Omnizip
20
20
  # @example Use with parallel compression
21
21
  # Omnizip::Parallel.compress_directory('files/', 'backup.zip', options)
22
22
  class ParallelOptions < Lutaml::Model::Serializable
23
+ include AttributeApply
24
+
23
25
  # @return [Integer] Number of worker threads (default: auto-detect)
24
26
  attribute :threads, :integer, default: -> { detect_cpu_count }
25
27
 
@@ -75,12 +77,6 @@ module Omnizip
75
77
  #
76
78
  # @param values [Hash{Symbol=>Object}] attributes to set
77
79
  # @return [self]
78
- def apply(values)
79
- self.class.attributes.each_key do |name|
80
- public_send("#{name}=", values[name]) if values.key?(name)
81
- end
82
- self
83
- end
84
80
 
85
81
  private
86
82
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Omnizip
4
- VERSION = "0.3.27"
4
+ VERSION = "0.3.29"
5
5
  end
data/lib/omnizip.rb CHANGED
@@ -89,6 +89,7 @@ end
89
89
  module Omnizip
90
90
  module Models
91
91
  autoload :AlgorithmMetadata, "omnizip/models/algorithm_metadata"
92
+ autoload :AttributeApply, "omnizip/models/attribute_apply"
92
93
  autoload :CompressionOptions, "omnizip/models/compression_options"
93
94
  autoload :PerformanceResult, "omnizip/models/performance_result"
94
95
  autoload :ProfileReport, "omnizip/models/profile_report"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: omnizip
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.27
4
+ version: 0.3.29
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ribose Inc.
@@ -249,6 +249,7 @@ files:
249
249
  - lib/omnizip/algorithms/lzma/literal_decoder.rb
250
250
  - lib/omnizip/algorithms/lzma/literal_encoder.rb
251
251
  - lib/omnizip/algorithms/lzma/lzip_decoder.rb
252
+ - lib/omnizip/algorithms/lzma/lzma1_encoder.rb
252
253
  - lib/omnizip/algorithms/lzma/lzma_alone_decoder.rb
253
254
  - lib/omnizip/algorithms/lzma/lzma_state.rb
254
255
  - lib/omnizip/algorithms/lzma/match.rb
@@ -613,6 +614,7 @@ files:
613
614
  - lib/omnizip/metadata/metadata_validator.rb
614
615
  - lib/omnizip/models/.keep
615
616
  - lib/omnizip/models/algorithm_metadata.rb
617
+ - lib/omnizip/models/attribute_apply.rb
616
618
  - lib/omnizip/models/compression_options.rb
617
619
  - lib/omnizip/models/conversion_options.rb
618
620
  - lib/omnizip/models/conversion_result.rb