omnizip 0.4.9-arm-linux → 0.4.11-arm-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (94) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop-oss.yml +814 -0
  3. data/.rubocop.yml +15 -1
  4. data/.rubocop_todo.yml +7 -1
  5. data/CHANGELOG.md +17 -0
  6. data/Gemfile +23 -1
  7. data/README.adoc +101 -1282
  8. data/docs/compatibility.adoc +1 -2
  9. data/docs/lychee.toml +1 -7
  10. data/lib/omnizip/algorithms/deflate/constants.rb +5 -3
  11. data/lib/omnizip/implementations/rust/archive.rb +9 -41
  12. data/lib/omnizip/implementations/rust/ffi_library.rb +127 -0
  13. data/lib/omnizip/implementations/rust/fiddle_library.rb +144 -0
  14. data/lib/omnizip/implementations/rust/library.rb +39 -104
  15. data/lib/omnizip/implementations/rust.rb +2 -0
  16. data/lib/omnizip/version.rb +1 -1
  17. data/lib/tasks/docs.rake +146 -0
  18. data/readme-docs/rar-archives.adoc +505 -0
  19. data/scripts/generate_rar_oracle_fixtures.rb +177 -0
  20. data/vendor/libomnizip_ffi.so +0 -0
  21. metadata +21 -74
  22. data/TODO.refactor/00-overview.md +0 -67
  23. data/TODO.refactor/01-registry-base-class.md +0 -81
  24. data/TODO.refactor/02-error-hierarchy.md +0 -57
  25. data/TODO.refactor/03-autoload-entry-point.md +0 -69
  26. data/TODO.refactor/04-filter-base-consolidation.md +0 -49
  27. data/TODO.refactor/05-send-private-methods.md +0 -64
  28. data/TODO.refactor/06-instance-variable-access.md +0 -69
  29. data/TODO.refactor/07-respond-to-replacement.md +0 -237
  30. data/TODO.refactor/08-spec-doubles.md +0 -27
  31. data/TODO.refactor/09-cli-shared-module.md +0 -35
  32. data/TODO.refactor/10-format-detector-ocp.md +0 -34
  33. data/TODO.refactor/11-convenience-decoupling.md +0 -38
  34. data/TODO.refactor/12-thread-safety.md +0 -8
  35. data/TODO.refactor/13-lutaml-model-migration.md +0 -158
  36. data/TODO.refactor/14-add-missing-specs.md +0 -25
  37. data/docs/.github/workflows/docs.yml +0 -142
  38. data/docs/.gitignore +0 -21
  39. data/docs/Gemfile +0 -13
  40. data/docs/README.md +0 -101
  41. data/docs/_config.yml +0 -112
  42. data/docs/comparison/feature-matrix.adoc +0 -694
  43. data/docs/comparison/index.adoc +0 -113
  44. data/docs/comparison/vs-7zip.adoc +0 -309
  45. data/docs/comparison/vs-peazip.adoc +0 -77
  46. data/docs/comparison/vs-rubyzip.adoc +0 -342
  47. data/docs/comparison/vs-winrar.adoc +0 -100
  48. data/docs/concepts/index.adoc +0 -129
  49. data/docs/developer/architecture.adoc +0 -256
  50. data/docs/developer/contributing.adoc +0 -158
  51. data/docs/developer/index.adoc +0 -25
  52. data/docs/developer/testing.adoc +0 -212
  53. data/docs/getting-started/basic-usage.adoc +0 -263
  54. data/docs/getting-started/index.adoc +0 -42
  55. data/docs/getting-started/installation.adoc +0 -138
  56. data/docs/getting-started/quick-start.adoc +0 -183
  57. data/docs/getting-started/your-first-archive.adoc +0 -219
  58. data/docs/guides/advanced-features/encryption.adoc +0 -277
  59. data/docs/guides/advanced-features/index.adoc +0 -59
  60. data/docs/guides/advanced-features/parallel-processing.adoc +0 -144
  61. data/docs/guides/advanced-features/progress-tracking.adoc +0 -124
  62. data/docs/guides/advanced-features/streaming.adoc +0 -142
  63. data/docs/guides/archive-formats/gzip-format.adoc +0 -105
  64. data/docs/guides/archive-formats/index.adoc +0 -160
  65. data/docs/guides/archive-formats/ole-format.adoc +0 -151
  66. data/docs/guides/archive-formats/rar-format.adoc +0 -104
  67. data/docs/guides/archive-formats/rar5.adoc +0 -521
  68. data/docs/guides/archive-formats/rpm-format.adoc +0 -108
  69. data/docs/guides/archive-formats/seven-zip-format.adoc +0 -35
  70. data/docs/guides/archive-formats/tar-format.adoc +0 -106
  71. data/docs/guides/archive-formats/xz-format.adoc +0 -118
  72. data/docs/guides/archive-formats/zip-format.adoc +0 -35
  73. data/docs/guides/compression-algorithms/bzip2.adoc +0 -114
  74. data/docs/guides/compression-algorithms/deflate.adoc +0 -313
  75. data/docs/guides/compression-algorithms/index.adoc +0 -190
  76. data/docs/guides/compression-algorithms/lzma.adoc +0 -394
  77. data/docs/guides/compression-algorithms/lzma2.adoc +0 -244
  78. data/docs/guides/compression-algorithms/ppmd.adoc +0 -313
  79. data/docs/guides/compression-algorithms/zstandard.adoc +0 -296
  80. data/docs/guides/creating-archives.adoc +0 -348
  81. data/docs/guides/extracting-archives.adoc +0 -53
  82. data/docs/guides/format-conversion.adoc +0 -64
  83. data/docs/guides/index.adoc +0 -49
  84. data/docs/guides/migration-rubyzip.adoc +0 -217
  85. data/docs/guides/parity-archives.adoc +0 -605
  86. data/docs/guides/performance-tuning.adoc +0 -88
  87. data/docs/index.adoc +0 -228
  88. data/docs/reference/api/overview.adoc +0 -187
  89. data/docs/reference/cli/compress-command.adoc +0 -114
  90. data/docs/reference/cli/overview.adoc +0 -140
  91. data/docs/reference/index.adoc +0 -26
  92. data/docs/resources/faq.adoc +0 -185
  93. data/docs/resources/quick-reference.adoc +0 -222
  94. data/docs/troubleshooting/index.adoc +0 -207
data/README.adoc CHANGED
@@ -1,1327 +1,146 @@
1
1
  = Omnizip
2
2
 
3
- https://github.com/metanorma/omnizip[image:https://img.shields.io/github/stars/metanorma/omnizip.svg?style=social[GitHub Stars]]
4
- https://github.com/metanorma/omnizip[image:https://img.shields.io/github/forks/metanorma/omnizip.svg?style=social[GitHub Forks]]
5
- image:https://img.shields.io/github/license/metanorma/omnizip.svg[License]
6
- image:https://img.shields.io/github/actions/workflow/status/metanorma/omnizip/test.yml?branch=main[Build Status]
7
- image:https://img.shields.io/gem/v/omnizip.svg[RubyGems Version]
3
+ Compression and archive toolkit for Ruby: LZMA/LZMA2, Zstandard, BZip2,
4
+ PPMd7/8, Deflate/Deflate64 codecs; ZIP, 7z, RAR4/RAR5, TAR, CPIO, ISO, XAR,
5
+ RPM, OLE, PAR2 formats; deterministic creation, AES encryption, PAR2
6
+ recovery records, format conversion and repair. Where a prebuilt binary
7
+ exists the codecs and archives run on the
8
+ link:https://github.com/omnizip/omnizip-rs[omnizip-rs] Rust engine — with
9
+ zero compilation for the user; everywhere else the gem stays fully
10
+ functional on its pure-Ruby core.
8
11
 
9
- == Purpose
12
+ == Installation
10
13
 
11
- Omnizip is a comprehensive pure Ruby implementation of compression algorithms and archive formats. Built on an extensible, registry-based architecture with clean object-oriented design, Omnizip provides full-featured compression capabilities without external dependencies.
14
+ ....
15
+ gem install omnizip # picks a prebuilt Rust cdylib per platform
16
+ ....
12
17
 
13
- This implementation supports multiple compression algorithms (LZMA, LZMA2, BZip2, PPMd7/8, Deflate, Deflate64, Zstandard), preprocessing filters (BCJ variants, Delta), encryption (AES-256), and complete support for multiple archive formats (.7z, ZIP, RAR, TAR, ISO, CPIO, GZIP, XZ, BZIP2).
14
-
15
- It provides both a command-line interface and a programmatic Ruby API, making it ideal for applications requiring portable, dependency-free compression without relying on system-level libraries or external binaries.
16
-
17
- **Pure Ruby Implementation:** Works on all Ruby platforms (MRI, JRuby, TruffleRuby) with zero external dependencies. Performance is 10-60x slower than native implementations, which is an acceptable trade-off for maximum portability.
18
-
19
- == Features
20
-
21
- === Compression Algorithms
22
-
23
- * **LZMA/LZMA2** - High compression ratio with dictionary-based encoding
24
- * **BZip2** - Burrows-Wheeler Transform compression
25
- * **PPMd7/PPMd8** - Prediction by Partial Matching for text
26
- * **Deflate/Deflate64** - ZIP-compatible compression (32KB/64KB windows)
27
- * **Zstandard** - Modern fast compression
28
-
29
- See link:readme-docs/compression-algorithms.adoc[Compression Algorithms Guide] for detailed information.
30
-
31
- ==== Rust Acceleration (prebuilt, zero compilation)
32
-
33
- Where the platform gem exists, `gem install omnizip` ships a prebuilt
34
- `omnizip-ffi` cdylib and the codec + archive tiers route through it
35
- automatically — no Rust toolchain, no compilation:
36
-
37
- * x86_64-linux, aarch64-linux, arm-linux (glibc and musl)
38
- * arm64-darwin, x86_64-darwin
39
- * x64-mingw-ucrt, aarch64-mingw-ucrt, x64-mingw32
40
-
41
- Everywhere else — including JRuby/TruffleRuby and any platform without
42
- a binary — the gem stays fully functional on its pure-Ruby core (the
43
- acceleration is strictly optional). Three escape hatches:
18
+ Supported prebuilt platforms: x86_64/aarch64/arm Linux (glibc + musl),
19
+ arm64/x86_64 macOS, x64/arm64 Windows (ucrt). JRuby and TruffleRuby bind
20
+ through the bundled `ffi` gem; MRI uses stdlib Fiddle. Escape hatches:
44
21
 
45
22
  * `OMNIZIP_NO_RUST=1` — force the pure-Ruby core everywhere.
46
23
  * `OMNIZIP_FFI_DYLIB=/path/to/libomnizip_ffi` — load a specific build.
47
- * `rake rust:build` — build and vendor the cdylib yourself from a
48
- checkout (developers).
49
-
50
- Any Rust-side failure also falls back to the Ruby core mid-operation,
51
- so `auto` mode is never worse than pure Ruby.
52
-
53
- ==== SDK-Compatible LZMA Mode (v0.2.0)
54
-
55
- Omnizip now provides SDK-compatible LZMA encoding and decoding for full XZ/LZMA tool compatibility.
56
-
57
- **Usage:**
58
-
59
- [source,ruby]
60
- ----
61
- require 'omnizip'
62
-
63
- # SDK-compatible encoding
64
- File.open('output.lzma', 'wb') do |f|
65
- encoder = Omnizip::Algorithms::LZMA::Encoder.new(f, sdk_compatible: true)
66
- encoder.encode_stream("Hello, World!")
67
- end
68
-
69
- # SDK-compatible decoding
70
- output = StringIO.new
71
- File.open('output.lzma', 'rb') do |f|
72
- decoder = Omnizip::Algorithms::LZMA::Decoder.new(f, sdk_compatible: true)
73
- decoder.decode_stream(output)
74
- end
75
- ----
76
-
77
- **Configuration:**
78
-
79
- [source,ruby]
80
- ----
81
- encoder = Omnizip::Algorithms::LZMA::Encoder.new(output,
82
- sdk_compatible: true, # Enable SDK mode
83
- lc: 3, # Literal context bits (0-8, default: 3)
84
- lp: 0, # Literal position bits (0-4, default: 0)
85
- pb: 2, # Position bits (0-4, default: 2)
86
- dict_size: 65536, # Dictionary size in bytes (default: 64KB)
87
- level: 5 # Compression level (0-9, default: 5)
88
- )
89
- ----
90
-
91
- **Compatibility:**
92
-
93
- SDK-compatible mode produces output that can be decoded by:
94
-
95
- * XZ Utils (`xz` command-line tool)
96
- * LZMA Utils (`lzma` command-line tool)
97
- * 7-Zip
98
- * Any LZMA SDK-based application
99
-
100
- And can decode files created by these tools.
101
-
102
- **Performance:**
103
-
104
- Pure Ruby implementation is 10-60x slower than native, an acceptable trade-off for maximum portability across all Ruby platforms (MRI, JRuby, TruffleRuby).
105
-
106
- ==== XZ Format Support (v0.3.0)
107
-
108
- Omnizip provides complete XZ container format (`.xz`) support with LZMA2 compression and decompression. The implementation is based on a port of the XZ Utils liblzma LZMA2 encoder and decoder, achieving full bidirectional compatibility with XZ Utils.
109
-
110
- **Status**: ✅ **Full Support** - Bidirectional XZ Utils compatibility achieved
111
-
112
- **Test Results**: 265/265 tests passing (100%)
113
- * ✅ XZ Official Test Suite: 31/31 tests passing (100%)
114
- * ✅ XZ Utils Reference Tests: 64/64 tests passing (100%)
115
- * ✅ XZ Utils Test Suite: 111/111 tests passing (100%)
116
- * ✅ XZ Filter Support: 30/30 tests passing (100%)
117
- * ✅ XZ Encoding Compatibility: 7/7 tests passing (100%)
118
-
119
- **What Works**:
120
-
121
- * ✅ XZ container format (Stream Header, Stream Footer, Index)
122
- * ✅ LZMA2 encoder and decoder (fully functional, XZ Utils compatible)
123
- * ✅ Block headers with VLI encoding and correct padding
124
- * ✅ All integrity checks: CRC32, CRC64, SHA256, None
125
- * ✅ Multi-block support (encoding and decoding)
126
- * ✅ Decoding all official XZ test fixtures
127
- * ✅ Encoding produces files that XZ Utils can decode (bidirectional compatibility)
128
- * ✅ ARM64 BCJ filter (both start_offset=0 and non-zero values)
129
- * ✅ All BCJ filters (x86, PowerPC, IA-64, ARM, ARM Thumb, SPARC, RISC-V)
130
- * ✅ Delta filter (single and multiple filter chains)
131
- * ✅ Empty files and single-byte files
132
- * ✅ Large files (>100 bytes)
133
-
134
- **Usage**:
135
-
136
- [source,ruby]
137
- ----
138
- require 'omnizip'
139
-
140
- # Compress a file to XZ format
141
- Omnizip::Formats::Xz.create_file('input.txt', level: 6)
142
-
143
- # Compress in-memory data (returns the .xz bytes)
144
- compressed = Omnizip::Formats::Xz.create("Hello, World!")
145
-
146
- # Decompress (accepts a path or the .xz bytes)
147
- decompressed = Omnizip::Formats::Xz.decompress(compressed)
148
-
149
- # Or use the Reader API
150
- reader = Omnizip::Formats::Xz::Reader.new('file.xz')
151
- data = reader.read
152
- ----
153
-
154
- **Advanced Options**:
155
-
156
- [source,ruby]
157
- ----
158
- # Configure checksum type
159
- compressed = Omnizip::Formats::Xz.create(data, check_type: :crc64)
160
- # Options: :crc32 (default), :crc64, :sha256, :none
161
-
162
- # Using Builder API for multi-part data
163
- compressed = Omnizip::Formats::Xz.create do |builder|
164
- builder.add_data("Part 1")
165
- builder.add_data("Part 2")
166
- builder.add_data("Part 3")
167
- end
168
- ----
169
-
170
- **Compatibility**:
171
-
172
- * ✅ **Bidirectional Compatibility**: Files created by Omnizip can be decoded by XZ Utils, and vice versa
173
- * All official XZ test fixtures (22 good-*.xz files) decode successfully
174
- * All compression levels supported
175
- * All checksum types supported (CRC32, CRC64, SHA256, None)
176
- * All BCJ filters working (x86, ARM, ARM64, PowerPC, IA-64, SPARC, RISC-V)
177
- * Delta filter working with single and multiple filter chains
178
- * Multi-block streams fully supported
179
- * ARM64 BCJ with non-zero start_offset works correctly
180
-
181
- **Testing**:
182
-
183
- All XZ test suites pass (265/265, 100%):
184
-
185
- * ✅ Decoding official files: 31/31 tests passing
186
- * ✅ Structure validation: All tests passing
187
- * ✅ LZMA2 encoder and decoder: Fully functional
188
- * ✅ Encoding compatibility: 7/7 tests passing (Omnizip → XZ Utils)
189
- * ✅ Filter support: 30/30 tests passing (BCJ, Delta)
190
-
191
- **Performance**:
192
-
193
- Pure Ruby implementation is 10-30x slower than native XZ Utils, which is an acceptable trade-off for maximum portability across all Ruby platforms (MRI, JRuby, TruffleRuby).
194
-
195
- **Architecture**:
196
-
197
- The XZ implementation follows clean object-oriented design with separation of concerns:
198
-
199
- * `Formats::Xz::Reader` - Public API for reading XZ files
200
- * `Formats::XzImpl::StreamDecoder` - Orchestrates stream decoding
201
- * `Formats::XzImpl::StreamEncoder` - Handles stream-level encoding
202
- * `Formats::XzImpl::BlockDecoder` - Decodes blocks with LZMA2 integration
203
- * `Formats::XzImpl::BlockEncoder` - Encodes blocks with LZMA2
204
- * `Formats::XzImpl::VLI` - Variable-length integer codec
205
- * `Formats::XzImpl::StreamHeaderParser` - Stream header parsing
206
- * `Formats::XzImpl::StreamFooterParser` - Stream footer parsing
207
- * `Formats::XzImpl::BlockHeaderParser` - Block header parsing
208
- * `Formats::XzImpl::IndexDecoder` - Index metadata parsing
209
- * `Checksums::Verifier` - Checksum verification utilities
210
-
211
- ==== 7-Zip Format Support (v0.3.0)
212
-
213
- Omnizip provides complete 7-Zip container format (`.7z`) support with multiple compression algorithms and encryption. The implementation is based on the 7-Zip format specification.
214
-
215
- **Status**: ✅ **Full Support** - Complete 7-Zip compatibility achieved
216
-
217
- **Test Results**: 50/50 tests passing (100%)
218
- * ✅ 7-Zip Official Test Suite: 50/50 tests passing (100%)
219
- * ✅ Archive creation and extraction: All tests passing
220
- * ✅ Solid compression: Fully functional
221
- * ✅ Multi-volume archives: Fully functional
222
- * ✅ Header encryption: Fully functional
223
-
224
- **What Works**:
225
-
226
- * ✅ 7-Zip container format (headers, metadata, structure)
227
- * ✅ Multiple compression algorithms (LZMA, LZMA2, DEFLATE, PPMD, BZip2)
228
- * ✅ Solid compression for improved ratios
229
- * ✅ Multi-volume split archives
230
- * ✅ Password protection (AES-256)
231
- * ✅ File attributes and timestamps
232
- * ✅ Archive creation and extraction
233
- * ✅ Directory structures
234
-
235
- **Usage**:
236
-
237
- [source,ruby]
238
- ----
239
- require 'omnizip'
240
-
241
- # Create 7z archive
242
- Omnizip::Formats::SevenZip.create('archive.7z') do |sz|
243
- sz.add_file('document.pdf')
244
- sz.add_directory('photos/')
245
- end
246
-
247
- # Extract 7z archive
248
- Omnizip::Formats::SevenZip.open('archive.7z') do |sz|
249
- sz.extract_all('output/')
250
- end
251
-
252
- # List contents
253
- Omnizip::Formats::SevenZip.open('archive.7z') do |sz|
254
- sz.list_files.each do |entry|
255
- puts "#{entry.name}: #{entry.size} bytes"
256
- end
257
- end
258
- ----
259
-
260
- **Advanced Options**:
261
-
262
- [source,ruby]
263
- ----
264
- # With compression options
265
- Omnizip::Formats::SevenZip.create('archive.7z',
266
- algorithm: :lzma2,
267
- level: 9,
268
- solid: true
269
- ) do |sz|
270
- sz.add_directory('data/')
271
- end
272
-
273
- # With password encryption
274
- Omnizip::Formats::SevenZip.create('secure.7z',
275
- password: 'secret123',
276
- encrypt_headers: true
277
- ) do |sz|
278
- sz.add_file('confidential.doc')
279
- end
280
- ----
281
-
282
- **Compatibility**:
283
-
284
- * ✅ **Full 7-Zip compatibility**: Archives created by Omnizip can be opened by 7-Zip
285
- * ✅ **7-Zip format specification**: Follows the official 7z format specification
286
- * ✅ **All compression methods**: LZMA, LZMA2, DEFLATE, PPMD, BZip2
287
- * ✅ **Solid compression**: For improved compression ratios on similar files
288
- * ✅ **Multi-volume**: Split archives across multiple files
289
- * ✅ **Encryption**: AES-256 password protection
290
-
291
- **Testing**:
292
-
293
- All 7-Zip test suites pass (50/50, 100%):
294
-
295
- * ✅ Archive creation: All tests passing
296
- * ✅ Archive extraction: All tests passing
297
- * ✅ Solid compression: All tests passing
298
- * ✅ Multi-volume: All tests passing
299
- * ✅ Header encryption: All tests passing
300
- * ✅ File attributes: All tests passing
301
-
302
- **Performance**:
303
-
304
- Pure Ruby implementation is 10-60x slower than native 7-Zip, which is an acceptable trade-off for maximum portability across all Ruby platforms (MRI, JRuby, TruffleRuby).
305
-
306
- ==== XAR Format Support (v0.4.0)
307
-
308
- Omnizip provides complete XAR (eXtensible ARchive) format support. XAR is primarily used on macOS for software packages (.pkg files), OS installers, and software distribution.
309
-
310
- **Status**: ✅ **Full Support** - Complete XAR format implementation
311
-
312
- **What Works**:
313
-
314
- * ✅ XAR container format (binary header, compressed XML TOC, heap)
315
- * ✅ Multiple compression algorithms (gzip, bzip2, lzma, xz, none)
316
- * ✅ Multiple checksum algorithms (MD5, SHA1, SHA256, SHA384, SHA512)
317
- * ✅ Extended attributes (xattrs)
318
- * ✅ Hardlinks and symlinks
319
- * ✅ Device nodes and FIFOs
320
- * ✅ Directory structures
321
- * ✅ File metadata (permissions, timestamps, ownership)
322
- * ✅ libarchive compatibility
323
-
324
- **Usage**:
325
-
326
- [source,ruby]
327
- ----
328
- require 'omnizip'
329
-
330
- # Create XAR archive
331
- Omnizip::Formats::Xar.create('archive.xar') do |xar|
332
- xar.add_file('document.pdf')
333
- xar.add_directory('resources/')
334
- end
335
-
336
- # Create with options
337
- Omnizip::Formats::Xar.create('archive.xar',
338
- compression: 'gzip', # Options: gzip, bzip2, lzma, xz, none
339
- toc_checksum: 'sha1', # Options: sha1, md5, sha256, none
340
- file_checksum: 'sha1' # Options: sha1, md5, sha256, none
341
- ) do |xar|
342
- xar.add_data("content", "file.txt")
343
- end
344
-
345
- # Extract XAR archive
346
- Omnizip::Formats::Xar.extract('archive.xar', 'output/')
347
-
348
- # List contents
349
- entries = Omnizip::Formats::Xar.list('archive.xar')
350
- entries.each { |e| puts "#{e.name} (#{e.size} bytes)" }
351
-
352
- # Get archive info
353
- info = Omnizip::Formats::Xar.info('archive.xar')
354
- puts "Format: XAR version #{info[:header][:version]}"
355
- puts "Files: #{info[:file_count]}"
356
- ----
357
-
358
- **Architecture**:
359
-
360
- The XAR implementation follows clean object-oriented design:
361
-
362
- * `Formats::Xar::Reader` - Public API for reading XAR files
363
- * `Formats::Xar::Writer` - Public API for writing XAR files
364
- * `Formats::Xar::Header` - Binary header parsing and generation
365
- * `Formats::Xar::Toc` - XML Table of Contents handling
366
- * `Formats::Xar::Entry` - File entry model with metadata
367
-
368
- **XAR Format Structure**:
369
-
370
- ```
371
- +-------------------+
372
- | Header (28 bytes) | Magic, sizes, checksum type
373
- +-------------------+
374
- | Compressed TOC | GZIP-compressed XML
375
- +-------------------+
376
- | TOC Checksum | SHA1 (20 bytes) or MD5 (16 bytes)
377
- +-------------------+
378
- | File Data Heap | Compressed file contents
379
- +-------------------+
380
- ```
381
-
382
- **libarchive Compatibility**:
383
-
384
- All libarchive XAR test cases pass, including:
385
-
386
- * ✅ Regular files with various compression methods
387
- * ✅ Hardlinks and symlinks
388
- * ✅ Character and block devices
389
- * ✅ Directories and FIFOs
390
- * ✅ Extended attributes
391
- * ✅ Various checksum algorithms
392
-
393
- ==== RPM Format Support (v0.4.0)
394
-
395
- Omnizip provides complete RPM package format support for reading and writing RPM packages.
396
-
397
- **Status**: ✅ **Full Support** - Complete RPM reading and writing
398
-
399
- **What Works**:
400
-
401
- * ✅ RPM lead parsing (magic, version, name, architecture)
402
- * ✅ Header parsing with tag extraction (NAME, VERSION, RELEASE, etc.)
403
- * ✅ File list extraction (basenames, directories, permissions)
404
- * ✅ Dependency information (requires, provides, conflicts)
405
- * ✅ Payload extraction with multiple compression formats
406
- * ✅ gzip, bzip2, xz, zstd decompression support
407
- * ✅ RPM package creation with CPIO payload
408
- * ✅ Multiple compression options for package creation
409
-
410
- **Reading RPM Packages**:
411
-
412
- [source,ruby]
413
- ----
414
- require 'omnizip'
415
-
416
- # Read RPM package metadata
417
- Omnizip::Formats::Rpm.open('package.rpm') do |rpm|
418
- puts "Name: #{rpm.name}"
419
- puts "Version: #{rpm.version}"
420
- puts "Release: #{rpm.release}"
421
- puts "Architecture: #{rpm.architecture}"
422
- puts "Files: #{rpm.files.count}"
423
- end
424
-
425
- # Extract RPM contents
426
- Omnizip::Formats::Rpm.extract('package.rpm', 'output/')
427
-
428
- # List files in RPM
429
- files = Omnizip::Formats::Rpm.list('package.rpm')
430
- files.each { |f| puts f }
431
-
432
- # Get package information
433
- info = Omnizip::Formats::Rpm.info('package.rpm')
434
- puts "#{info[:name]}-#{info[:version]}-#{info[:release]}"
435
- ----
436
-
437
- **Writing RPM Packages**:
438
-
439
- [source,ruby]
440
- ----
441
- require 'omnizip'
442
-
443
- # Create RPM package with gzip-compressed payload (default)
444
- rpm = Omnizip::Formats::Rpm::Writer.new(
445
- name: 'mypackage', version: '1.0', release: '1', arch: 'noarch'
446
- )
447
-
448
- # Add payload entries (content is passed as a string)
449
- rpm.add_file('/usr/bin/myapp', File.binread('path/to/myapp'), mode: 0o755)
450
- rpm.add_file('/etc/myapp.conf', File.binread('path/to/config'))
451
- rpm.add_directory('/var/lib/myapp')
452
-
453
- rpm.write('mypackage-1.0-1.noarch.rpm')
454
-
455
- # Supported compression types: :gzip (default), :bzip2, :xz, :zstd, :none
456
- Omnizip::Formats::Rpm::Writer.new(
457
- name: 'mypackage', version: '1.0', release: '1',
458
- arch: 'x86_64', compression: :xz
459
- )
460
- ----
461
-
462
- **Architecture**:
463
-
464
- * `Formats::Rpm::Reader` - Public API for reading RPM packages
465
- * `Formats::Rpm::Writer` - Public API for writing RPM packages
466
- * `Formats::Rpm::Lead` - 96-byte lead parser
467
- * `Formats::Rpm::Header` - Header structure with tag extraction
468
- * `Formats::Rpm::Entry` - File entry model
469
-
470
- ==== OLE Format Support (v0.4.0)
471
-
472
- Omnizip provides complete OLE (Object Linking and Embedding) compound document format support for reading and writing Microsoft compound files.
473
-
474
- **Status**: ✅ **Full Support** - Complete OLE reading and writing
475
-
476
- **What Works**:
477
-
478
- * ✅ OLE compound document header parsing
479
- * ✅ Block allocation tables (BAT, SBAT, XBAT)
480
- * ✅ Directory entry navigation
481
- * ✅ File stream extraction
482
- * ✅ Support for .doc, .xls, .ppt, .msi files
483
- * ✅ Property set storage
484
- * ✅ OLE compound document creation
485
- * ✅ Stream writing with BAT/SBAT management
486
-
487
- **Reading OLE Documents**:
488
-
489
- [source,ruby]
490
- ----
491
- require 'omnizip'
492
-
493
- # Open OLE compound document
494
- Omnizip::Formats::Ole.open('document.doc') do |ole|
495
- # List all streams in the document
496
- ole.each_entry do |entry|
497
- puts "#{entry.name} (#{entry.size} bytes)"
498
- end
499
-
500
- # Read a specific stream
501
- data = ole.read_stream('WordDocument')
502
- end
503
-
504
- # Extract all streams
505
- Omnizip::Formats::Ole.extract('document.doc', 'output/')
506
- ----
507
-
508
- **Writing OLE Documents**:
509
-
510
- [source,ruby]
511
- ----
512
- require 'omnizip'
513
-
514
- # Create new OLE compound document
515
- Omnizip::Formats::Ole.write('output.doc') do |ole|
516
- # Add streams (files) to the document
517
- ole.add_stream('WordDocument', word_data)
518
- ole.add_stream('\x01CompObj', compobj_data)
519
- ole.add_stream('\x05SummaryInformation', summary_data)
520
-
521
- # Add nested storage (directory)
522
- ole.add_storage('MyStorage')
523
- end
524
-
525
- # Create from existing files
526
- Omnizip::Formats::Ole.write('output.msi') do |ole|
527
- ole.add_stream_from_file('BinaryFile', 'path/to/file.bin')
528
- ole.add_stream('\x05DigitalSignature', signature_data)
529
- end
530
- ----
531
-
532
- **Architecture**:
533
-
534
- * `Formats::Ole::Storage` - Core storage implementation
535
- * `Formats::Ole::Writer` - Public API for writing OLE documents
536
- * `Formats::Ole::Header` - 512-byte header parser
537
- * `Formats::Ole::AllocationTable` - BAT/SBAT management
538
- * `Formats::Ole::Dirent` - 128-byte directory entry
539
- * `Formats::Ole::RangesIO` - Range-based IO wrapper
540
- * `Formats::Ole::Types` - Type serialization (Variant, Lpstr, FileTime, etc.)
541
-
542
- === Preprocessing Filters
543
-
544
- * **BCJ Filters** - Branch-Call-Jump filters for executables (x86, ARM, ARM64, PPC, SPARC, IA-64)
545
- * **BCJ2** - Advanced 4-stream x86 filter
546
- * **Delta** - Delta encoding for multimedia/databases
547
-
548
- See link:readme-docs/preprocessing-filters.adoc[Preprocessing Filters Guide] for details.
549
-
550
- === Archive Formats
551
-
552
- * **.7z** - Full read/write with solid compression, multi-volume support
553
- * **ZIP** - Full read/write with ZIP64, WinZip AES encryption
554
- * **RAR4** - Read support for real WinRAR archives (STORE natively; compressed methods via unrar), write with STORE verified against unrar, FASTEST through BEST for Omnizip round-trip (v0.3.0)
555
- * **RAR5** - Read/write with STORE (unrar-verified), AES-256 encryption, multi-volume; compressed-method writing falls back to STORE (v0.3.0)
556
- * **TAR** - Full read/write with POSIX extensions
557
- * **ISO 9660** - Full read/write with Rock Ridge/Joliet
558
- * **CPIO** - Full read/write (newc, CRC formats) with RPM payload support (v0.4.0)
559
- * **RPM** - Full read/write support with metadata extraction, gzip/bzip2/xz/zstd payload compression (v0.4.0)
560
- * **XAR** - Full read/write with XML TOC, gzip/bzip2/lzma compression (v0.4.0)
561
- * **OLE** - Full read/write support for Microsoft compound documents (.doc, .xls, .ppt, .msi) (v0.4.0)
562
- * **GZIP/XZ/BZIP2** - Single file compression formats
563
-
564
- See link:readme-docs/archive-formats.adoc[Archive Formats Documentation] for complete details.
565
-
566
- === PAR2 Parity Archives
567
-
568
- Full support for PAR2 (Parity Archive Volume 2) error correction using Reed-Solomon codes over GF(2^16):
569
-
570
- * Detect data corruption at block level using MD5 checksums
571
- * Verify file integrity without unpacking
572
- * Repair corrupted or missing files automatically
573
- * Protect multiple files in a single archive set
574
- * Configurable redundancy from 0-100%
575
- * Full par2cmdline compatibility (v0.2.0)
576
-
577
- See link:readme-docs/par2-archives.adoc[PAR2 Parity Archives Guide] for comprehensive documentation.
578
-
579
- === Advanced Features
580
-
581
- * **Compression Profiles** - Smart algorithm selection based on file type
582
- * **Format Converter** - Convert between ZIP and 7z formats
583
- * **Performance Profiler** - Identify bottlenecks and optimize
584
- * **Progress Tracking** - Real-time progress with ETA calculation
585
- * **Selective Extraction** - Glob, regex, and predicate-based extraction
586
- * **Parallel Processing** - Multi-threaded compression using Ractors
587
- * **Encryption** - AES-256 password protection with SHA-256 key derivation
588
- * **Checksums** - CRC32/CRC64 integrity verification
589
- * **Enumerable Collections** - All archive and result classes support Ruby's Enumerable interface
24
+ * `rake rust:build` — build and vendor the cdylib from a checkout.
590
25
 
591
- See link:readme-docs/advanced-features.adoc[Advanced Features Guide] for details.
26
+ A Rust-side failure falls back to the Ruby core mid-operation, so
27
+ acceleration is never worse than pure Ruby.
592
28
 
593
- === Test Coverage (v0.4.0)
29
+ External code must `require "omnizip"`. Files under `lib/omnizip/**` are
30
+ internal — requiring them directly bypasses the autoload chain.
594
31
 
595
- Omnizip maintains comprehensive test coverage:
32
+ == Command line
596
33
 
597
- * **Total Tests**: 3540+ examples
598
- * **Pass Rate**: 100% (0 failures, 0 pending)
599
- * **Coverage**: All compression algorithms, archive formats, and features
600
- * **Integration**: Full round-trip verification for all formats
601
- * **Reference Tests**: libarchive RAR4/RAR5 compatibility verified (103 test files)
602
- * **New Formats**: RPM (21 tests), CPIO (25 tests), OLE (36 tests), XAR (17 tests)
603
-
604
- == Quick Start
605
-
606
- === Installation
607
-
608
- [source,sh]
609
- ----
610
- # Via Bundler
611
- gem 'omnizip'
612
-
613
- # Via gem command
614
- gem install omnizip
615
- ----
616
-
617
- External code must `require "omnizip"` to use the gem. Files under
618
- `lib/omnizip/**` are internal — requiring them directly (e.g.
619
- `require "omnizip/formats/cpio"`) bypasses the autoload chain that
620
- wires the namespace modules, and lookups for sibling constants may
621
- fail with `NameError`.
622
-
623
- See link:readme-docs/installation.adoc[Installation Guide] for complete instructions.
624
-
625
- === Command Line
626
-
627
- [source,sh]
628
- ----
629
- # Compress a file
34
+ ....
630
35
  omnizip compress input.txt output.lzma --level 9
631
-
632
- # Create a .7z archive
633
36
  omnizip archive create backup.7z documents/ photos/
634
-
635
- # Extract an archive
636
37
  omnizip archive extract backup.7z output/
637
-
638
- # List archive contents
639
38
  omnizip archive list backup.7z
640
- ----
641
-
642
- See link:readme-docs/cli-usage.adoc[CLI Usage Guide] for all commands and options.
643
-
644
- === Ruby API
39
+ omnizip archive verify backup.rar
40
+ omnizip archive repair damaged.rar
41
+ omnizip parity create backup.rar --percent 10
42
+ omnizip parity verify backup.par2
43
+ omnizip parity repair backup.par2
44
+ omnizip profile list
45
+ omnizip metadata archive.rar
46
+ ....
645
47
 
646
- ==== Simple Convenience Methods
48
+ == Ruby API
647
49
 
648
- [source,ruby]
649
- ----
650
- require 'omnizip'
50
+ === One-liners
651
51
 
652
- # One-liners for common operations
653
- Omnizip.compress_file('input.txt', 'output.zip')
654
- Omnizip.extract_archive('archive.zip', 'output/')
655
- Omnizip.list_archive('archive.zip')
656
- ----
52
+ ....
53
+ require "omnizip"
657
54
 
658
- ==== In-Memory Archives
55
+ Omnizip.compress_file("input.txt", "output.zip")
56
+ Omnizip.extract_archive("archive.zip", "output/")
57
+ Omnizip.list_archive("archive.zip")
58
+ ....
659
59
 
660
- `Omnizip::Buffer` creates and reads ZIP and 7z archives entirely in
661
- memory (7z bridges through a temporary file internally):
60
+ === In-memory archives
662
61
 
663
- [source,ruby]
664
- ----
665
- require 'omnizip'
62
+ `Omnizip::Buffer` creates and reads ZIP and 7z archives entirely in memory:
666
63
 
667
- # Create in memory
64
+ ....
668
65
  buffer = Omnizip::Buffer.create(:zip) do |archive|
669
- archive.add('readme.txt', 'Hello World')
670
- archive.add('docs/', '') # directory entry
671
- end
672
-
673
- # Same API for 7z
674
- buffer7z = Omnizip::Buffer.create(:seven_zip) do |archive|
675
- archive.add('data.json', '{"key": "value"}')
676
- end
677
-
678
- # Read back (format auto-detected)
679
- files = Omnizip::Buffer.extract_to_memory(buffer.string)
680
- # => {"readme.txt" => "Hello World"}
681
-
682
- Omnizip::Buffer.open(buffer.string) do |archive|
683
- archive.each_entry do |entry|
684
- puts "#{entry.name}: #{entry.size} bytes" unless entry.directory?
685
- end
686
- end
687
- ----
688
-
689
- ==== Rubyzip Compatibility Mode
690
-
691
- [source,ruby]
692
- ----
693
- require 'omnizip/rubyzip_compat'
694
-
695
- # Drop-in replacement for rubyzip
696
- Zip::File.open('archive.zip', create: true) do |zip|
697
- zip.add('file.txt') { 'Content' }
698
- end
699
-
700
- Zip::File.open('archive.zip') do |zip|
701
- content = zip.read('file.txt')
66
+ archive.add("readme.txt", "Hello World")
702
67
  end
703
- ----
68
+ names = Omnizip::Backends.archive_entry_names(buffer.to_s)
69
+ ....
704
70
 
705
- ==== Native Omnizip API
71
+ === Archive files
706
72
 
707
- [source,ruby]
708
- ----
709
- require 'omnizip'
710
-
711
- # Full control with algorithm registry
712
- algorithm = Omnizip::AlgorithmRegistry.get(:lzma2).new(level: 9)
713
- File.open('input.txt', 'rb') do |input|
714
- File.open('output.lzma', 'wb') do |output|
715
- algorithm.compress(input, output)
716
- end
717
- end
718
-
719
- # .7z archive operations
720
- Omnizip::Formats::SevenZip.create('archive.7z') do |sz|
721
- sz.add_file('document.pdf')
73
+ ....
74
+ Omnizip::Zip::File.open("in.zip") { |z| z.extract("a.txt", "out.txt") }
75
+ Omnizip::Zip::File.open("out.zip", create: true) do |z|
76
+ z.add_file("a.txt")
722
77
  end
723
- ----
724
-
725
- ==== RAR Archives
726
-
727
- ==== RAR4 Archives (v0.3.0)
728
-
729
- RAR4 reading and writing against the on-disk format unrar accepts:
730
- headers carry the low 16 bits of a CRC32, HEAD_SIZE counts the whole
731
- block, and the dictionary mask 0xE0 marks directory entries. STORE
732
- archives round-trip through official unrar byte-identically.
733
-
734
- ====== Reading RAR4 Archives
735
-
736
- [source,ruby]
737
- ----
738
- require 'omnizip'
739
-
740
- reader = Omnizip::Formats::Rar::Reader.new('archive.rar')
741
- reader.open
742
-
743
- reader.list_files.each do |entry|
744
- puts "#{entry.name}: #{entry.size} bytes (#{entry.compressed_size} compressed)"
745
- puts " Directory: #{entry.is_dir}"
746
- puts " Modified: #{entry.mtime}"
747
- end
748
-
749
- # STORE data decodes natively; WinRAR-compressed entries fall back to
750
- # the unrar command after a CRC check rejects foreign streams
751
- reader.extract_entry('test.txt', 'output/test.txt')
752
- reader.extract_all('output/')
753
- ----
754
-
755
- **RAR4 Reader Features:**
756
-
757
- * ✅ Block header parsing (MAIN/FILE/ENDARC), directories, empty files
758
- * ✅ Native STORE decompression with stored-CRC32 verification
759
- * ✅ unrar fallback for real WinRAR-compressed entries (byte-exact)
760
- * ✅ DOS-style backslash name normalization to forward slashes
761
- * ✅ Multi-volume archive detection
762
- * ✅ Parsing verified against libarchive WinRAR fixtures
763
78
 
764
- ====== Writing RAR4 Archives
79
+ names = Omnizip::Backends.archive_entry_names("archive.rar")
80
+ data = Omnizip::Backends.archive_read_entry("archive.rar", "doc.txt",
81
+ password: "secret")
82
+ ....
765
83
 
766
- [source,ruby]
767
- ----
768
- require 'omnizip'
84
+ === Codecs directly
769
85
 
770
- # STORE output is verified interoperable with unrar
771
- writer = Omnizip::Formats::Rar::Writer.new('archive.rar',
772
- compression: :store
773
- )
774
- writer.add_file('document.txt')
775
- writer.add_file('image.png')
776
- writer.add_directory('photos/')
777
- writer.write
778
- ----
86
+ ....
87
+ compressed = Omnizip::Algorithms::Lzma.compress(data, level: 9)
88
+ plain = Omnizip::Algorithms::Lzma.decompress(compressed)
779
89
 
780
- Writer options: `compression:` (see below), `test_after_create: true`
781
- (reads the archive back and verifies every file). RAR4 encryption is
782
- not implemented — `password:` raises `NotImplementedError`. The
783
- `solid:`/`recovery:`/`volume_size:` options are accepted but not
784
- implemented; they are ignored with a warning and never advertised in
785
- header flags.
90
+ Omnizip::Algorithms::Zstandard.compress(data, level: 19)
91
+ Omnizip::Algorithms::Bzip2.compress(data)
92
+ Omnizip::Algorithms::Ppmd7.compress(data, order: 8)
93
+ ....
786
94
 
787
- ====== RAR4 Compression Methods
95
+ === SDK-compatible LZMA
788
96
 
789
- [cols="2,2,2,4",options="header"]
790
- |===
791
- |Method |Speed |unrar interop |Notes
792
- |`:store` |Instant |✅ verified |Byte-identical round-trip incl. directories
793
- |`:fastest` |Very Fast |❌ Omnizip-only |LZ77+Huffman, readable by Omnizip
794
- |`:normal` (default) |Fast |❌ Omnizip-only |LZ77+Huffman, readable by Omnizip
795
- |`:good` |Medium |❌ Omnizip-only |LZ77+Huffman, readable by Omnizip
796
- |`:best` |Slow |❌ Omnizip-only |PPMd, readable by Omnizip
797
- |===
798
-
799
- Official RAR LZ/PPMd streams are proprietary; methods above `:store`
800
- produce archives Omnizip can read back but unrar reports as checksum
801
- errors. Use `:store` for interoperability.
802
-
803
- ===== RAR5 Archives (v0.3.0)
804
-
805
- Read/write support for RAR5 archives with STORE compression,
806
- AES-256 encryption, and multi-volume archives — all verified against
807
- official unrar.
808
-
809
- ====== Reading RAR5 Archives
810
-
811
- [source,ruby]
812
- ----
813
- require 'omnizip'
814
-
815
- reader = Omnizip::Formats::Rar5::Reader.new
816
- File.open('archive.rar', 'rb') do |io|
817
- entries = reader.read_archive(io)
818
-
819
- entries.each do |entry|
820
- puts "#{entry.name}: #{entry.uncompressed_size} bytes"
821
- puts " CRC32: #{entry.crc32.to_s(16)}" if entry.crc32
822
- puts " Modified: #{entry.modified_time}" if entry.modified_time
823
- end
97
+ ....
98
+ File.open("out.lzma", "wb") do |f|
99
+ Omnizip::Algorithms::LZMA::Encoder.new(f, sdk_compatible: true)
100
+ .encode_stream("Hello, World!")
824
101
  end
825
- ----
826
-
827
- **RAR5 Reader Features:**
828
-
829
- * ✅ STORE file data, header CRC32 verification
830
- * ✅ 32-bit Unix mtime decoding (matches unrar; nil when absent)
831
- * ✅ Unicode filenames (UTF-8)
832
- * ✅ Multi-file archives
833
- * ✅ VInt (variable-length integer) parsing
834
- * ✅ Graceful error handling for truncated/invalid files
835
- * ✅ libarchive compatibility fixtures
836
-
837
- ====== Writing RAR5 Archives
838
-
839
- [source,ruby]
840
- ----
841
- require 'omnizip'
842
-
843
- # STORE compression (default) — unrar-verified
844
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar')
845
- writer.add_file('document.txt')
846
- writer.add_file('image.png')
847
- writer.write
848
-
849
- # AES-256 encryption
850
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
851
- password: 'secret'
852
- )
853
- writer.add_file('data.json')
854
- writer.write
855
-
856
- # Multi-volume archives split at 64 KB boundaries
857
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
858
- multi_volume: true,
859
- volume_size: 65536
860
- )
861
- writer.add_file('big1.bin')
862
- writer.add_file('big2.bin')
863
- writer.write # writes archive.part1.rar, archive.part2.rar, ...
864
- ----
865
-
866
- ====== RAR5 Compression Methods
867
-
868
- [cols="2,2,4",options="header"]
869
- |===
870
- |Method |Interop |Notes
871
- |`:store` (default) |✅ unrar-verified |Uncompressed passthrough
872
- |`:lzss` / `:lzma` |falls back to STORE |Encoder is not official-RAR compatible; warns and stores
873
- |`:auto` |STORE |Size-based selection
874
- |===
875
-
876
- ====== RAR5 Optional Fields
877
-
878
- The writer can embed metadata that readers use without decompressing:
879
-
880
- [source,ruby]
881
- ----
882
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
883
- include_mtime: true, # 32-bit Unix mtime per file header
884
- include_crc32: true # CRC32 per file header
885
- )
886
- ----
887
-
888
- ====== RAR5 Optional Fields
889
-
890
- RAR5 supports optional metadata fields for enhanced archive information:
891
-
892
- [source,ruby]
893
- ----
894
- require 'omnizip/formats/rar/rar5/writer'
895
-
896
- # Include modification time in archive
897
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
898
- include_mtime: true # Preserves file modification timestamps
899
- )
900
- writer.add_file('document.txt')
901
- writer.write
902
-
903
- # Include CRC32 checksums (STORE compression only)
904
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
905
- compression: :store,
906
- include_crc32: true # Only works with STORE compression
907
- )
908
- writer.add_file('data.bin')
909
- writer.write
910
-
911
- # IMPORTANT: CRC32 with LZMA is automatically disabled
912
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
913
- compression: :lzma,
914
- level: 5,
915
- include_crc32: true # Will be auto-disabled, no error
916
- )
917
- writer.add_file('document.txt')
918
- writer.write
919
- # CRC32 will be silently disabled - archive uses BLAKE2sp only
920
- ----
921
-
922
- **CRC32 Limitation Explained:**
923
-
924
- RAR5's optional CRC32 field is incompatible with compression algorithms. The RAR5 format specification requires that when compression is used (LZMA, LZMA2), only the BLAKE2sp checksum (always present in the main file header) should be used for integrity verification. The optional CRC32 field is designed for uncompressed (STORE) files only.
925
-
926
- When you request `include_crc32: true` with LZMA compression, Omnizip automatically disables CRC32 to ensure format compliance and compatibility with official unrar tools.
927
-
928
- ====== RAR5 Multi-Volume Archives
929
-
930
- Create split archives when file size exceeds volume limit. Volumes are automatically created and numbered according to the chosen naming pattern.
931
-
932
- [source,ruby]
933
- ----
934
- require 'omnizip/formats/rar/rar5/writer'
935
-
936
- # Create multi-volume archive with default settings
937
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
938
- multi_volume: true,
939
- volume_size: '10M' # Human-readable size
940
- )
941
- writer.add_directory('data/')
942
- volumes = writer.write # Returns array of volume paths
943
- # => ['archive.part1.rar', 'archive.part2.rar', ...]
944
-
945
- # Custom volume naming pattern
946
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup.rar',
947
- multi_volume: true,
948
- volume_size: '100M',
949
- volume_naming: 'volume' # backup.volume1.rar, backup.volume2.rar
950
- )
951
- writer.add_directory('large_dataset/')
952
- volumes = writer.write
953
-
954
- # Numeric naming pattern
955
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('data.rar',
956
- multi_volume: true,
957
- volume_size: '50M',
958
- volume_naming: 'numeric' # data.001.rar, data.002.rar
959
- )
960
- writer.add_file('huge_file.bin')
961
- volumes = writer.write
962
- ----
963
-
964
- **Volume Naming Patterns:**
965
-
966
- * `part` (default): archive.part1.rar, archive.part2.rar, ...
967
- * `volume`: archive.volume1.rar, archive.volume2.rar, ...
968
- * `numeric`: archive.001.rar, archive.002.rar, ...
969
-
970
- **Human-Readable Sizes:**
971
-
972
- * Bytes: `1024`, `2048`
973
- * Kilobytes: `10K`, `100KB`
974
- * Megabytes: `10M`, `100MB`
975
- * Gigabytes: `1G`, `5GB`
102
+ ....
976
103
 
977
- **Minimum volume size:** 64 KB (65,536 bytes)
104
+ === Profiles, conversion, streaming, repair
978
105
 
979
- ====== RAR5 Solid Compression
980
-
981
- Compress multiple files with a shared dictionary for significantly better compression ratios. Ideal for similar files such as source code, logs, or document collections.
982
-
983
- [source,ruby]
984
- ----
985
- require 'omnizip/formats/rar/rar5/writer'
986
-
987
- # Enable solid compression (default: 10-30% better ratio)
988
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
989
- compression: :lzma,
990
- level: 5,
991
- solid: true # Enable solid mode
992
- )
993
- writer.add_directory('project/')
994
- writer.write
995
-
996
- # Combine with high compression level
997
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup.rar',
998
- compression: :lzma,
999
- level: 5, # Best compression
1000
- solid: true # Shared dictionary
1001
- )
1002
- writer.add_file('log1.txt')
1003
- writer.add_file('log2.txt')
1004
- writer.add_file('log3.txt')
1005
- writer.write
1006
- ----
1007
-
1008
- **Benefits:**
1009
-
1010
- * 10-30% better compression ratios for similar files
1011
- * Larger LZMA dictionaries (16-64 MB vs 1-16 MB)
1012
- * Particularly effective for:
1013
- - Source code repositories
1014
- - Log files and text documents
1015
- - Similar structured data
1016
-
1017
- **Trade-offs:**
1018
-
1019
- * Cannot extract individual files without decompressing entire solid block
1020
- * Corruption in one file may affect subsequent files in the block
1021
- * Slightly longer extraction time for single files
1022
- * Best for archiving complete collections
1023
-
1024
- ====== RAR5 AES-256 Encryption
1025
-
1026
- Protect archives with industry-standard AES-256-CBC encryption and PBKDF2-HMAC-SHA256 key derivation.
1027
-
1028
- [source,ruby]
1029
- ----
1030
- require 'omnizip/formats/rar/rar5/writer'
1031
-
1032
- # Basic password protection
1033
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
1034
- compression: :lzma,
1035
- password: 'SecurePassword123!'
1036
- )
1037
- writer.add_file('confidential.pdf')
1038
- writer.write
1039
-
1040
- # Custom key derivation iterations
1041
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('secure.rar',
1042
- compression: :lzma,
1043
- level: 5,
1044
- password: 'VerySecurePassword2025!',
1045
- kdf_iterations: 524_288 # Higher = more secure, slower
1046
- )
1047
- writer.add_directory('sensitive_data/')
1048
- writer.write
1049
-
1050
- # Minimum security (faster but less secure)
1051
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('quick.rar',
1052
- password: 'FastPassword',
1053
- kdf_iterations: 65_536 # Minimum allowed
1054
- )
1055
- writer.add_file('temp.txt')
1056
- writer.write
1057
- ----
1058
-
1059
- **Security Features:**
1060
-
1061
- * **AES-256-CBC encryption** with PKCS#7 padding
1062
- * **PBKDF2-HMAC-SHA256** key derivation function
1063
- * **Configurable KDF iterations:**
1064
- - Minimum: 65,536 (2^16) - fast but less secure
1065
- - Default: 262,144 (2^18) - balanced security/performance
1066
- - Maximum: 1,048,576 (2^20) - maximum security
1067
- * **Per-file IV generation** for enhanced security
1068
- * **Password verification** before decryption attempts
1069
-
1070
- **Performance Impact:**
1071
-
1072
- * Encryption overhead: < 2x slower than unencrypted
1073
- * KDF computation time varies with iteration count:
1074
- - 65,536 iterations: ~50-100ms
1075
- - 262,144 iterations: ~200-400ms
1076
- - 1,048,576 iterations: ~800-1600ms
1077
-
1078
- ====== RAR5 PAR2 Recovery Records
1079
-
1080
- Generate PAR2 parity files for archive recovery and error correction using Reed-Solomon codes.
1081
-
1082
- [source,ruby]
1083
- ----
1084
- require 'omnizip/formats/rar/rar5/writer'
1085
-
1086
- # Enable recovery with default 5% redundancy
1087
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
1088
- compression: :lzma,
1089
- recovery: true
1090
- )
1091
- writer.add_directory('important_data/')
1092
- files = writer.write
1093
- # => ['archive.rar', 'archive.par2', 'archive.vol00+01.par2', ...]
1094
-
1095
- # Custom redundancy percentage
1096
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup.rar',
1097
- compression: :lzma,
1098
- level: 5,
1099
- recovery: true,
1100
- recovery_percent: 10 # 10% redundancy (can recover up to 10% data loss)
1101
- )
1102
- writer.add_file('critical.db')
1103
- files = writer.write
1104
-
1105
- # Maximum redundancy for critical data
1106
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('critical.rar',
1107
- compression: :lzma,
1108
- recovery: true,
1109
- recovery_percent: 50 # 50% redundancy (maximum protection)
1110
- )
1111
- writer.add_directory('mission_critical/')
1112
- files = writer.write
1113
- ----
1114
-
1115
- **Recovery Capabilities:**
1116
-
1117
- * **Detect corruption** at block level
1118
- * **Repair damaged archives** automatically
1119
- * **Recover from partial data loss** up to redundancy percentage
1120
- * **Works with all features:**
1121
- - Multi-volume archives
1122
- - Solid compression
1123
- - Encrypted archives
1124
- * **Reed-Solomon error correction** over GF(2^16)
1125
-
1126
- **Redundancy Guidelines:**
1127
-
1128
- * 5% (default): Suitable for general backups
1129
- * 10%: Recommended for important data
1130
- * 20-30%: High-value data requiring extra protection
1131
- * 50-100%: Critical data with maximum recovery needs
1132
-
1133
- **PAR2 File Size:**
1134
-
1135
- PAR2 files add approximately the redundancy percentage to total archive size. For example, a 100MB archive with 10% redundancy will generate ~10MB of PAR2 files.
1136
-
1137
- ====== RAR5 Combined Features
1138
-
1139
- All RAR5 features can be used together for comprehensive archive protection:
1140
-
1141
- [source,ruby]
1142
- ----
1143
- require 'omnizip/formats/rar/rar5/writer'
1144
-
1145
- # Complete feature demonstration
1146
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('complete.rar',
1147
- # Compression
1148
- compression: :lzma,
1149
- level: 5, # Best compression
1150
- solid: true, # Shared dictionary for better ratios
1151
-
1152
- # Security
1153
- password: 'SecureBackup2025!',
1154
- kdf_iterations: 524_288, # Enhanced security
1155
-
1156
- # Multi-volume
1157
- multi_volume: true,
1158
- volume_size: '100M',
1159
- volume_naming: 'part',
1160
-
1161
- # Recovery
1162
- recovery: true,
1163
- recovery_percent: 10,
1164
-
1165
- # Optional fields
1166
- include_mtime: true
1167
- )
1168
-
1169
- writer.add_directory('/critical/data')
1170
- files = writer.write
1171
- # => ['complete.part1.rar', 'complete.part2.rar', ...,
1172
- # 'complete.par2', 'complete.vol00+01.par2', ...]
1173
- ----
1174
-
1175
- **Best Practices:**
1176
-
1177
- 1. **Solid + LZMA level 5** for maximum compression on similar files
1178
- 2. **10-20% PAR2** for important data protection
1179
- 3. **262,144 KDF iterations** for balanced security/performance
1180
- 4. **Multi-volume** for large archives or optical media
1181
- 5. **Always include mtime** to preserve file timestamps
1182
-
1183
- **Example: Secure Backup Archive**
1184
-
1185
- [source,ruby]
1186
- ----
1187
- # Production-ready backup configuration
1188
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup_2025-12-24.rar',
1189
- compression: :lzma,
1190
- level: 5,
1191
- solid: true, # 10-30% better compression
1192
- password: ENV['BACKUP_PASSWORD'] || 'DefaultSecure123!',
1193
- kdf_iterations: 262_144, # Balanced security
1194
- multi_volume: true,
1195
- volume_size: '4G', # DVD-sized volumes
1196
- recovery: true,
1197
- recovery_percent: 15, # 15% redundancy
1198
- include_mtime: true
1199
- )
1200
-
1201
- writer.add_directory('/home/user/documents')
1202
- writer.add_directory('/home/user/projects')
1203
- files = writer.write
1204
-
1205
- puts "Backup created: #{files.size} files"
1206
- puts "Total size: #{files.sum { |f| File.size(f) } / 1024 / 1024}MB"
1207
- ----
1208
-
1209
- **Important:**
1210
- - Ensure you have set the `BACKUP_PASSWORD` environment variable before running the secure backup example.
1211
- - This example assumes a Linux/Unix environment; file paths may need adjustments for Windows.
1212
-
1213
- **Security Note:**
1214
- - Use a strong, complex password for `BACKUP_PASSWORD`.
1215
- - Consider using a password manager to store and retrieve your backup password securely.
1216
- - If using this code in production, review the security implications and adjust as needed.
1217
-
1218
- **Performance Note:**
1219
- - Encryption and KDF computations can be CPU-intensive.
1220
- - The `kdf_iterations` value affects security; higher values are more secure but slower.
1221
- - The `volume_naming` option can impact the efficiency and naming of multi-volume archives.
1222
-
1223
- **Error Handling:**
1224
- - Enhance this example by adding error handling for file operations and encryption failures.
1225
-
1226
- Dresses your ruby file as README.md (see https://guides.github.com/features/mastering-markdown/).
106
+ Content-class profiles pick the codec automatically
107
+ (`Omnizip::Profile::DEFAULT` and friends); `Omnizip::Converter` moves
108
+ archives between formats; the chunked reader/writer processes inputs far
109
+ larger than RAM; `Omnizip::ArchiveRepairCommand` routes structural
110
+ repair. See the guides below.
1227
111
 
1228
112
  == Documentation
1229
113
 
1230
- === Core Documentation
1231
-
1232
- * **link:readme-docs/installation.adoc[Installation Guide]** - Setup and system requirements
1233
- * **link:readme-docs/cli-usage.adoc[CLI Usage]** - Command-line interface reference
1234
- * **link:readme-docs/api-usage.adoc[Library API]** - Programmatic Ruby API guide
1235
- * **link:readme-docs/compression-algorithms.adoc[Compression Algorithms]** - LZMA, BZip2, PPMd, Deflate, Zstandard
1236
- * **link:readme-docs/preprocessing-filters.adoc[Preprocessing Filters]** - BCJ, BCJ2, Delta filters
1237
- * **link:readme-docs/encryption-checksums.adoc[Encryption & Checksums]** - AES-256, CRC32/CRC64
1238
- * **link:readme-docs/architecture.adoc[Architecture]** - System design, patterns, extensibility
1239
-
1240
- === Format Documentation
1241
-
1242
- * **link:readme-docs/archive-formats.adoc[Archive Formats]** - .7z, ZIP, RAR, TAR, ISO, CPIO
1243
- * **link:readme-docs/par2-archives.adoc[PAR2 Parity Archives]** - Error correction and repair
1244
-
1245
- === Advanced Topics
1246
-
1247
- * **link:readme-docs/compression-profiles.adoc[Compression Profiles]** - Smart algorithm selection
1248
- * **link:readme-docs/format-converter.adoc[Format Converter]** - Convert between formats
1249
- * **link:readme-docs/performance-profiler.adoc[Performance Profiler]** - Benchmarking and optimization
1250
- * **link:readme-docs/advanced-features.adoc[Advanced Features]** - Progress, ETA, parallel processing
1251
-
1252
- === Additional Resources
1253
-
1254
- * link:old-docs/MIGRATION_GUIDE.md[Migration from Rubyzip]
1255
- * link:old-docs/COMPATIBILITY_MATRIX.md[Compatibility Matrix]
1256
- * link:old-docs/BASELINE.md[Performance Baseline]
1257
- * link:old-docs/CONTRIBUTING.md[Contributing Guidelines]
114
+ * link:readme-docs/installation.adoc[Installation] ·
115
+ link:readme-docs/cli-usage.adoc[CLI usage] ·
116
+ link:readme-docs/api-usage.adoc[API usage]
117
+ * link:readme-docs/compression-algorithms.adoc[Compression algorithms] ·
118
+ link:readme-docs/compression-profiles.adoc[Profiles] ·
119
+ link:readme-docs/format-converter.adoc[Format converter] ·
120
+ link:readme-docs/advanced-features.adoc[Progress, ETA, parallel] ·
121
+ link:readme-docs/performance-profiler.adoc[Performance profiler]
122
+ * link:readme-docs/archive-formats.adoc[Archive formats] ·
123
+ link:readme-docs/rar-archives.adoc[RAR4/RAR5 guide] ·
124
+ link:readme-docs/par2-archives.adoc[PAR2 parity archives] ·
125
+ link:readme-docs/encryption-checksums.adoc[Encryption and checksums]
126
+ * link:readme-docs/architecture.adoc[Architecture] ·
127
+ link:docs/rust-backend.md[Rust backend] ·
128
+ link:docs/compatibility.adoc[Compatibility] ·
129
+ link:docs/RAR_WRITE_SUPPORT.md[RAR write support]
1258
130
 
1259
131
  == Development
1260
132
 
1261
- === Running Tests
1262
-
1263
- [source,sh]
1264
- ----
1265
- # Run all tests
1266
- bundle exec rspec
1267
-
1268
- # Run specific test file
1269
- bundle exec rspec spec/omnizip/algorithms/lzma_spec.rb
1270
-
1271
- # Run with documentation format
1272
- bundle exec rspec --format documentation
1273
- ----
1274
-
1275
- === Running Linters
1276
-
1277
- [source,sh]
1278
- ----
1279
- # Run RuboCop
1280
- bundle exec rubocop
1281
-
1282
- # Auto-correct offenses
1283
- bundle exec rubocop -A
1284
-
1285
- # Generate config for new offenses
1286
- bundle exec rubocop -A --auto-gen-config
1287
- ----
1288
-
1289
- === Performance Benchmarks
1290
-
1291
- [source,sh]
1292
- ----
1293
- # Run performance benchmarks
1294
- ruby benchmark/run_benchmarks.rb
1295
-
1296
- # View baseline results
1297
- cat benchmark/results/v1.0_baseline.txt
1298
- ----
1299
-
1300
- == Contributing
1301
-
1302
- Contributions are welcome! Please read link:old-docs/CONTRIBUTING.md[CONTRIBUTING.md] for details on our code of conduct, development process, and how to submit pull requests.
1303
-
1304
- **Quick start:**
1305
-
1306
- . Fork the repository
1307
- . Create your feature branch (`git checkout -b feature/my-new-feature`)
1308
- . Make your changes and add tests
1309
- . Run the test suite (`bundle exec rspec`)
1310
- . Run RuboCop (`bundle exec rubocop -A`)
1311
- . Commit your changes with semantic commit messages
1312
- . Push to the branch (`git push origin feature/my-new-feature`)
1313
- . Create a new Pull Request
1314
-
1315
- == Acknowledgments
1316
-
1317
- Omnizip is a completely independent, clean-room implementation of compression algorithms and archive formats. The compression algorithms (LZMA, LZMA2, BZip2, PPMd, Deflate64, etc.) are implemented from publicly available specifications and mathematical descriptions.
1318
-
1319
- Archive formats (7z, ZIP, RAR, TAR, ISO, CPIO) are implemented based on their public format specifications. Similar to libarchive's independent implementations, Omnizip provides open-source, unencumbered implementations of these formats.
1320
-
1321
- IMPORTANT: Compression algorithms themselves are mathematical concepts and cannot be patented. Omnizip's implementations are original work based on algorithm specifications, not derivative of any existing codebase.
1322
-
1323
- == Copyright and License
133
+ ....
134
+ bundle install
135
+ bundle exec rspec # suite
136
+ bundle exec rspec spec/omnizip/algorithms/lzma_spec.rb # one file
137
+ bundle exec rubocop # lint
138
+ rake rust:build # vendor the Rust cdylib locally
139
+ rake docs:build # render docs/site via Coradoc
140
+ rake docs:links # lychee link check
141
+ ....
1324
142
 
1325
- Copyright 2026 https://www.ribose.com[Ribose Inc.]
143
+ == Copyright and license
1326
144
 
1327
- See the link:COPYING[COPYING] file and link:LICENSE[LICENSE] file for the complete text of the licenses.
145
+ Copyright (c) Ribose Inc. Released under the MIT License; see the LICENSE
146
+ file. The Rust engine (omnizip-rs) is MIT OR Apache-2.0.