omnizip 0.4.10-arm-linux → 0.4.11-arm-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (84) hide show
  1. checksums.yaml +4 -4
  2. data/Gemfile +13 -0
  3. data/README.adoc +101 -1284
  4. data/docs/compatibility.adoc +1 -2
  5. data/docs/lychee.toml +1 -7
  6. data/lib/omnizip/version.rb +1 -1
  7. data/lib/tasks/docs.rake +146 -0
  8. data/readme-docs/rar-archives.adoc +505 -0
  9. data/scripts/generate_rar_oracle_fixtures.rb +177 -0
  10. data/vendor/libomnizip_ffi.so +0 -0
  11. metadata +4 -74
  12. data/TODO.refactor/00-overview.md +0 -67
  13. data/TODO.refactor/01-registry-base-class.md +0 -81
  14. data/TODO.refactor/02-error-hierarchy.md +0 -57
  15. data/TODO.refactor/03-autoload-entry-point.md +0 -69
  16. data/TODO.refactor/04-filter-base-consolidation.md +0 -49
  17. data/TODO.refactor/05-send-private-methods.md +0 -64
  18. data/TODO.refactor/06-instance-variable-access.md +0 -69
  19. data/TODO.refactor/07-respond-to-replacement.md +0 -237
  20. data/TODO.refactor/08-spec-doubles.md +0 -27
  21. data/TODO.refactor/09-cli-shared-module.md +0 -35
  22. data/TODO.refactor/10-format-detector-ocp.md +0 -34
  23. data/TODO.refactor/11-convenience-decoupling.md +0 -38
  24. data/TODO.refactor/12-thread-safety.md +0 -8
  25. data/TODO.refactor/13-lutaml-model-migration.md +0 -158
  26. data/TODO.refactor/14-add-missing-specs.md +0 -25
  27. data/docs/.github/workflows/docs.yml +0 -142
  28. data/docs/.gitignore +0 -21
  29. data/docs/Gemfile +0 -13
  30. data/docs/README.md +0 -101
  31. data/docs/_config.yml +0 -112
  32. data/docs/comparison/feature-matrix.adoc +0 -694
  33. data/docs/comparison/index.adoc +0 -113
  34. data/docs/comparison/vs-7zip.adoc +0 -309
  35. data/docs/comparison/vs-peazip.adoc +0 -77
  36. data/docs/comparison/vs-rubyzip.adoc +0 -342
  37. data/docs/comparison/vs-winrar.adoc +0 -100
  38. data/docs/concepts/index.adoc +0 -129
  39. data/docs/developer/architecture.adoc +0 -256
  40. data/docs/developer/contributing.adoc +0 -158
  41. data/docs/developer/index.adoc +0 -25
  42. data/docs/developer/testing.adoc +0 -212
  43. data/docs/getting-started/basic-usage.adoc +0 -263
  44. data/docs/getting-started/index.adoc +0 -42
  45. data/docs/getting-started/installation.adoc +0 -138
  46. data/docs/getting-started/quick-start.adoc +0 -183
  47. data/docs/getting-started/your-first-archive.adoc +0 -219
  48. data/docs/guides/advanced-features/encryption.adoc +0 -277
  49. data/docs/guides/advanced-features/index.adoc +0 -59
  50. data/docs/guides/advanced-features/parallel-processing.adoc +0 -144
  51. data/docs/guides/advanced-features/progress-tracking.adoc +0 -124
  52. data/docs/guides/advanced-features/streaming.adoc +0 -142
  53. data/docs/guides/archive-formats/gzip-format.adoc +0 -105
  54. data/docs/guides/archive-formats/index.adoc +0 -160
  55. data/docs/guides/archive-formats/ole-format.adoc +0 -151
  56. data/docs/guides/archive-formats/rar-format.adoc +0 -104
  57. data/docs/guides/archive-formats/rar5.adoc +0 -521
  58. data/docs/guides/archive-formats/rpm-format.adoc +0 -108
  59. data/docs/guides/archive-formats/seven-zip-format.adoc +0 -35
  60. data/docs/guides/archive-formats/tar-format.adoc +0 -106
  61. data/docs/guides/archive-formats/xz-format.adoc +0 -118
  62. data/docs/guides/archive-formats/zip-format.adoc +0 -35
  63. data/docs/guides/compression-algorithms/bzip2.adoc +0 -114
  64. data/docs/guides/compression-algorithms/deflate.adoc +0 -313
  65. data/docs/guides/compression-algorithms/index.adoc +0 -190
  66. data/docs/guides/compression-algorithms/lzma.adoc +0 -394
  67. data/docs/guides/compression-algorithms/lzma2.adoc +0 -244
  68. data/docs/guides/compression-algorithms/ppmd.adoc +0 -313
  69. data/docs/guides/compression-algorithms/zstandard.adoc +0 -296
  70. data/docs/guides/creating-archives.adoc +0 -348
  71. data/docs/guides/extracting-archives.adoc +0 -53
  72. data/docs/guides/format-conversion.adoc +0 -64
  73. data/docs/guides/index.adoc +0 -49
  74. data/docs/guides/migration-rubyzip.adoc +0 -217
  75. data/docs/guides/parity-archives.adoc +0 -605
  76. data/docs/guides/performance-tuning.adoc +0 -88
  77. data/docs/index.adoc +0 -228
  78. data/docs/reference/api/overview.adoc +0 -187
  79. data/docs/reference/cli/compress-command.adoc +0 -114
  80. data/docs/reference/cli/overview.adoc +0 -140
  81. data/docs/reference/index.adoc +0 -26
  82. data/docs/resources/faq.adoc +0 -185
  83. data/docs/resources/quick-reference.adoc +0 -222
  84. data/docs/troubleshooting/index.adoc +0 -207
data/README.adoc CHANGED
@@ -1,1329 +1,146 @@
1
1
  = Omnizip
2
2
 
3
- https://github.com/metanorma/omnizip[image:https://img.shields.io/github/stars/metanorma/omnizip.svg?style=social[GitHub Stars]]
4
- https://github.com/metanorma/omnizip[image:https://img.shields.io/github/forks/metanorma/omnizip.svg?style=social[GitHub Forks]]
5
- image:https://img.shields.io/github/license/metanorma/omnizip.svg[License]
6
- image:https://img.shields.io/github/actions/workflow/status/metanorma/omnizip/test.yml?branch=main[Build Status]
7
- image:https://img.shields.io/gem/v/omnizip.svg[RubyGems Version]
3
+ Compression and archive toolkit for Ruby: LZMA/LZMA2, Zstandard, BZip2,
4
+ PPMd7/8, Deflate/Deflate64 codecs; ZIP, 7z, RAR4/RAR5, TAR, CPIO, ISO, XAR,
5
+ RPM, OLE, PAR2 formats; deterministic creation, AES encryption, PAR2
6
+ recovery records, format conversion and repair. Where a prebuilt binary
7
+ exists the codecs and archives run on the
8
+ link:https://github.com/omnizip/omnizip-rs[omnizip-rs] Rust engine — with
9
+ zero compilation for the user; everywhere else the gem stays fully
10
+ functional on its pure-Ruby core.
8
11
 
9
- == Purpose
12
+ == Installation
10
13
 
11
- Omnizip is a comprehensive pure Ruby implementation of compression algorithms and archive formats. Built on an extensible, registry-based architecture with clean object-oriented design, Omnizip provides full-featured compression capabilities without external dependencies.
14
+ ....
15
+ gem install omnizip # picks a prebuilt Rust cdylib per platform
16
+ ....
12
17
 
13
- This implementation supports multiple compression algorithms (LZMA, LZMA2, BZip2, PPMd7/8, Deflate, Deflate64, Zstandard), preprocessing filters (BCJ variants, Delta), encryption (AES-256), and complete support for multiple archive formats (.7z, ZIP, RAR, TAR, ISO, CPIO, GZIP, XZ, BZIP2).
14
-
15
- It provides both a command-line interface and a programmatic Ruby API, making it ideal for applications requiring portable, dependency-free compression without relying on system-level libraries or external binaries.
16
-
17
- **Pure Ruby Implementation:** Works on all Ruby platforms (MRI, JRuby, TruffleRuby) with zero external dependencies. Performance is 10-60x slower than native implementations, which is an acceptable trade-off for maximum portability.
18
-
19
- == Features
20
-
21
- === Compression Algorithms
22
-
23
- * **LZMA/LZMA2** - High compression ratio with dictionary-based encoding
24
- * **BZip2** - Burrows-Wheeler Transform compression
25
- * **PPMd7/PPMd8** - Prediction by Partial Matching for text
26
- * **Deflate/Deflate64** - ZIP-compatible compression (32KB/64KB windows)
27
- * **Zstandard** - Modern fast compression
28
-
29
- See link:readme-docs/compression-algorithms.adoc[Compression Algorithms Guide] for detailed information.
30
-
31
- ==== Rust Acceleration (prebuilt, zero compilation)
32
-
33
- Where the platform gem exists, `gem install omnizip` ships a prebuilt
34
- `omnizip-ffi` cdylib and the codec + archive tiers route through it
35
- automatically — no Rust toolchain, no compilation:
36
-
37
- * x86_64-linux, aarch64-linux, arm-linux (glibc and musl)
38
- * arm64-darwin, x86_64-darwin
39
- * x64-mingw-ucrt, aarch64-mingw-ucrt, x64-mingw32
40
-
41
- Everywhere else — including any platform without a binary — the gem
42
- stays fully functional on its pure-Ruby core (the acceleration is
43
- strictly optional). JRuby and TruffleRuby get acceleration too
44
- whenever a cdylib is available: they bind through the bundled `ffi`
45
- gem (MRI uses stdlib Fiddle). Three escape hatches:
18
+ Supported prebuilt platforms: x86_64/aarch64/arm Linux (glibc + musl),
19
+ arm64/x86_64 macOS, x64/arm64 Windows (ucrt). JRuby and TruffleRuby bind
20
+ through the bundled `ffi` gem; MRI uses stdlib Fiddle. Escape hatches:
46
21
 
47
22
  * `OMNIZIP_NO_RUST=1` — force the pure-Ruby core everywhere.
48
23
  * `OMNIZIP_FFI_DYLIB=/path/to/libomnizip_ffi` — load a specific build.
49
- * `rake rust:build` — build and vendor the cdylib yourself from a
50
- checkout (developers).
51
-
52
- Any Rust-side failure also falls back to the Ruby core mid-operation,
53
- so `auto` mode is never worse than pure Ruby.
54
-
55
- ==== SDK-Compatible LZMA Mode (v0.2.0)
56
-
57
- Omnizip now provides SDK-compatible LZMA encoding and decoding for full XZ/LZMA tool compatibility.
58
-
59
- **Usage:**
60
-
61
- [source,ruby]
62
- ----
63
- require 'omnizip'
64
-
65
- # SDK-compatible encoding
66
- File.open('output.lzma', 'wb') do |f|
67
- encoder = Omnizip::Algorithms::LZMA::Encoder.new(f, sdk_compatible: true)
68
- encoder.encode_stream("Hello, World!")
69
- end
70
-
71
- # SDK-compatible decoding
72
- output = StringIO.new
73
- File.open('output.lzma', 'rb') do |f|
74
- decoder = Omnizip::Algorithms::LZMA::Decoder.new(f, sdk_compatible: true)
75
- decoder.decode_stream(output)
76
- end
77
- ----
78
-
79
- **Configuration:**
80
-
81
- [source,ruby]
82
- ----
83
- encoder = Omnizip::Algorithms::LZMA::Encoder.new(output,
84
- sdk_compatible: true, # Enable SDK mode
85
- lc: 3, # Literal context bits (0-8, default: 3)
86
- lp: 0, # Literal position bits (0-4, default: 0)
87
- pb: 2, # Position bits (0-4, default: 2)
88
- dict_size: 65536, # Dictionary size in bytes (default: 64KB)
89
- level: 5 # Compression level (0-9, default: 5)
90
- )
91
- ----
92
-
93
- **Compatibility:**
94
-
95
- SDK-compatible mode produces output that can be decoded by:
96
-
97
- * XZ Utils (`xz` command-line tool)
98
- * LZMA Utils (`lzma` command-line tool)
99
- * 7-Zip
100
- * Any LZMA SDK-based application
101
-
102
- And can decode files created by these tools.
103
-
104
- **Performance:**
105
-
106
- Pure Ruby implementation is 10-60x slower than native, an acceptable trade-off for maximum portability across all Ruby platforms (MRI, JRuby, TruffleRuby).
107
-
108
- ==== XZ Format Support (v0.3.0)
109
-
110
- Omnizip provides complete XZ container format (`.xz`) support with LZMA2 compression and decompression. The implementation is based on a port of the XZ Utils liblzma LZMA2 encoder and decoder, achieving full bidirectional compatibility with XZ Utils.
111
-
112
- **Status**: ✅ **Full Support** - Bidirectional XZ Utils compatibility achieved
113
-
114
- **Test Results**: 265/265 tests passing (100%)
115
- * ✅ XZ Official Test Suite: 31/31 tests passing (100%)
116
- * ✅ XZ Utils Reference Tests: 64/64 tests passing (100%)
117
- * ✅ XZ Utils Test Suite: 111/111 tests passing (100%)
118
- * ✅ XZ Filter Support: 30/30 tests passing (100%)
119
- * ✅ XZ Encoding Compatibility: 7/7 tests passing (100%)
120
-
121
- **What Works**:
122
-
123
- * ✅ XZ container format (Stream Header, Stream Footer, Index)
124
- * ✅ LZMA2 encoder and decoder (fully functional, XZ Utils compatible)
125
- * ✅ Block headers with VLI encoding and correct padding
126
- * ✅ All integrity checks: CRC32, CRC64, SHA256, None
127
- * ✅ Multi-block support (encoding and decoding)
128
- * ✅ Decoding all official XZ test fixtures
129
- * ✅ Encoding produces files that XZ Utils can decode (bidirectional compatibility)
130
- * ✅ ARM64 BCJ filter (both start_offset=0 and non-zero values)
131
- * ✅ All BCJ filters (x86, PowerPC, IA-64, ARM, ARM Thumb, SPARC, RISC-V)
132
- * ✅ Delta filter (single and multiple filter chains)
133
- * ✅ Empty files and single-byte files
134
- * ✅ Large files (>100 bytes)
135
-
136
- **Usage**:
137
-
138
- [source,ruby]
139
- ----
140
- require 'omnizip'
141
-
142
- # Compress a file to XZ format
143
- Omnizip::Formats::Xz.create_file('input.txt', level: 6)
144
-
145
- # Compress in-memory data (returns the .xz bytes)
146
- compressed = Omnizip::Formats::Xz.create("Hello, World!")
147
-
148
- # Decompress (accepts a path or the .xz bytes)
149
- decompressed = Omnizip::Formats::Xz.decompress(compressed)
150
-
151
- # Or use the Reader API
152
- reader = Omnizip::Formats::Xz::Reader.new('file.xz')
153
- data = reader.read
154
- ----
155
-
156
- **Advanced Options**:
157
-
158
- [source,ruby]
159
- ----
160
- # Configure checksum type
161
- compressed = Omnizip::Formats::Xz.create(data, check_type: :crc64)
162
- # Options: :crc32 (default), :crc64, :sha256, :none
163
-
164
- # Using Builder API for multi-part data
165
- compressed = Omnizip::Formats::Xz.create do |builder|
166
- builder.add_data("Part 1")
167
- builder.add_data("Part 2")
168
- builder.add_data("Part 3")
169
- end
170
- ----
171
-
172
- **Compatibility**:
173
-
174
- * ✅ **Bidirectional Compatibility**: Files created by Omnizip can be decoded by XZ Utils, and vice versa
175
- * All official XZ test fixtures (22 good-*.xz files) decode successfully
176
- * All compression levels supported
177
- * All checksum types supported (CRC32, CRC64, SHA256, None)
178
- * All BCJ filters working (x86, ARM, ARM64, PowerPC, IA-64, SPARC, RISC-V)
179
- * Delta filter working with single and multiple filter chains
180
- * Multi-block streams fully supported
181
- * ARM64 BCJ with non-zero start_offset works correctly
182
-
183
- **Testing**:
184
-
185
- All XZ test suites pass (265/265, 100%):
186
-
187
- * ✅ Decoding official files: 31/31 tests passing
188
- * ✅ Structure validation: All tests passing
189
- * ✅ LZMA2 encoder and decoder: Fully functional
190
- * ✅ Encoding compatibility: 7/7 tests passing (Omnizip → XZ Utils)
191
- * ✅ Filter support: 30/30 tests passing (BCJ, Delta)
192
-
193
- **Performance**:
194
-
195
- Pure Ruby implementation is 10-30x slower than native XZ Utils, which is an acceptable trade-off for maximum portability across all Ruby platforms (MRI, JRuby, TruffleRuby).
196
-
197
- **Architecture**:
198
-
199
- The XZ implementation follows clean object-oriented design with separation of concerns:
200
-
201
- * `Formats::Xz::Reader` - Public API for reading XZ files
202
- * `Formats::XzImpl::StreamDecoder` - Orchestrates stream decoding
203
- * `Formats::XzImpl::StreamEncoder` - Handles stream-level encoding
204
- * `Formats::XzImpl::BlockDecoder` - Decodes blocks with LZMA2 integration
205
- * `Formats::XzImpl::BlockEncoder` - Encodes blocks with LZMA2
206
- * `Formats::XzImpl::VLI` - Variable-length integer codec
207
- * `Formats::XzImpl::StreamHeaderParser` - Stream header parsing
208
- * `Formats::XzImpl::StreamFooterParser` - Stream footer parsing
209
- * `Formats::XzImpl::BlockHeaderParser` - Block header parsing
210
- * `Formats::XzImpl::IndexDecoder` - Index metadata parsing
211
- * `Checksums::Verifier` - Checksum verification utilities
212
-
213
- ==== 7-Zip Format Support (v0.3.0)
214
-
215
- Omnizip provides complete 7-Zip container format (`.7z`) support with multiple compression algorithms and encryption. The implementation is based on the 7-Zip format specification.
216
-
217
- **Status**: ✅ **Full Support** - Complete 7-Zip compatibility achieved
218
-
219
- **Test Results**: 50/50 tests passing (100%)
220
- * ✅ 7-Zip Official Test Suite: 50/50 tests passing (100%)
221
- * ✅ Archive creation and extraction: All tests passing
222
- * ✅ Solid compression: Fully functional
223
- * ✅ Multi-volume archives: Fully functional
224
- * ✅ Header encryption: Fully functional
225
-
226
- **What Works**:
227
-
228
- * ✅ 7-Zip container format (headers, metadata, structure)
229
- * ✅ Multiple compression algorithms (LZMA, LZMA2, DEFLATE, PPMD, BZip2)
230
- * ✅ Solid compression for improved ratios
231
- * ✅ Multi-volume split archives
232
- * ✅ Password protection (AES-256)
233
- * ✅ File attributes and timestamps
234
- * ✅ Archive creation and extraction
235
- * ✅ Directory structures
236
-
237
- **Usage**:
238
-
239
- [source,ruby]
240
- ----
241
- require 'omnizip'
242
-
243
- # Create 7z archive
244
- Omnizip::Formats::SevenZip.create('archive.7z') do |sz|
245
- sz.add_file('document.pdf')
246
- sz.add_directory('photos/')
247
- end
248
-
249
- # Extract 7z archive
250
- Omnizip::Formats::SevenZip.open('archive.7z') do |sz|
251
- sz.extract_all('output/')
252
- end
253
-
254
- # List contents
255
- Omnizip::Formats::SevenZip.open('archive.7z') do |sz|
256
- sz.list_files.each do |entry|
257
- puts "#{entry.name}: #{entry.size} bytes"
258
- end
259
- end
260
- ----
261
-
262
- **Advanced Options**:
263
-
264
- [source,ruby]
265
- ----
266
- # With compression options
267
- Omnizip::Formats::SevenZip.create('archive.7z',
268
- algorithm: :lzma2,
269
- level: 9,
270
- solid: true
271
- ) do |sz|
272
- sz.add_directory('data/')
273
- end
274
-
275
- # With password encryption
276
- Omnizip::Formats::SevenZip.create('secure.7z',
277
- password: 'secret123',
278
- encrypt_headers: true
279
- ) do |sz|
280
- sz.add_file('confidential.doc')
281
- end
282
- ----
283
-
284
- **Compatibility**:
285
-
286
- * ✅ **Full 7-Zip compatibility**: Archives created by Omnizip can be opened by 7-Zip
287
- * ✅ **7-Zip format specification**: Follows the official 7z format specification
288
- * ✅ **All compression methods**: LZMA, LZMA2, DEFLATE, PPMD, BZip2
289
- * ✅ **Solid compression**: For improved compression ratios on similar files
290
- * ✅ **Multi-volume**: Split archives across multiple files
291
- * ✅ **Encryption**: AES-256 password protection
292
-
293
- **Testing**:
294
-
295
- All 7-Zip test suites pass (50/50, 100%):
296
-
297
- * ✅ Archive creation: All tests passing
298
- * ✅ Archive extraction: All tests passing
299
- * ✅ Solid compression: All tests passing
300
- * ✅ Multi-volume: All tests passing
301
- * ✅ Header encryption: All tests passing
302
- * ✅ File attributes: All tests passing
303
-
304
- **Performance**:
305
-
306
- Pure Ruby implementation is 10-60x slower than native 7-Zip, which is an acceptable trade-off for maximum portability across all Ruby platforms (MRI, JRuby, TruffleRuby).
307
-
308
- ==== XAR Format Support (v0.4.0)
309
-
310
- Omnizip provides complete XAR (eXtensible ARchive) format support. XAR is primarily used on macOS for software packages (.pkg files), OS installers, and software distribution.
311
-
312
- **Status**: ✅ **Full Support** - Complete XAR format implementation
313
-
314
- **What Works**:
315
-
316
- * ✅ XAR container format (binary header, compressed XML TOC, heap)
317
- * ✅ Multiple compression algorithms (gzip, bzip2, lzma, xz, none)
318
- * ✅ Multiple checksum algorithms (MD5, SHA1, SHA256, SHA384, SHA512)
319
- * ✅ Extended attributes (xattrs)
320
- * ✅ Hardlinks and symlinks
321
- * ✅ Device nodes and FIFOs
322
- * ✅ Directory structures
323
- * ✅ File metadata (permissions, timestamps, ownership)
324
- * ✅ libarchive compatibility
325
-
326
- **Usage**:
327
-
328
- [source,ruby]
329
- ----
330
- require 'omnizip'
331
-
332
- # Create XAR archive
333
- Omnizip::Formats::Xar.create('archive.xar') do |xar|
334
- xar.add_file('document.pdf')
335
- xar.add_directory('resources/')
336
- end
337
-
338
- # Create with options
339
- Omnizip::Formats::Xar.create('archive.xar',
340
- compression: 'gzip', # Options: gzip, bzip2, lzma, xz, none
341
- toc_checksum: 'sha1', # Options: sha1, md5, sha256, none
342
- file_checksum: 'sha1' # Options: sha1, md5, sha256, none
343
- ) do |xar|
344
- xar.add_data("content", "file.txt")
345
- end
346
-
347
- # Extract XAR archive
348
- Omnizip::Formats::Xar.extract('archive.xar', 'output/')
349
-
350
- # List contents
351
- entries = Omnizip::Formats::Xar.list('archive.xar')
352
- entries.each { |e| puts "#{e.name} (#{e.size} bytes)" }
353
-
354
- # Get archive info
355
- info = Omnizip::Formats::Xar.info('archive.xar')
356
- puts "Format: XAR version #{info[:header][:version]}"
357
- puts "Files: #{info[:file_count]}"
358
- ----
359
-
360
- **Architecture**:
361
-
362
- The XAR implementation follows clean object-oriented design:
363
-
364
- * `Formats::Xar::Reader` - Public API for reading XAR files
365
- * `Formats::Xar::Writer` - Public API for writing XAR files
366
- * `Formats::Xar::Header` - Binary header parsing and generation
367
- * `Formats::Xar::Toc` - XML Table of Contents handling
368
- * `Formats::Xar::Entry` - File entry model with metadata
369
-
370
- **XAR Format Structure**:
371
-
372
- ```
373
- +-------------------+
374
- | Header (28 bytes) | Magic, sizes, checksum type
375
- +-------------------+
376
- | Compressed TOC | GZIP-compressed XML
377
- +-------------------+
378
- | TOC Checksum | SHA1 (20 bytes) or MD5 (16 bytes)
379
- +-------------------+
380
- | File Data Heap | Compressed file contents
381
- +-------------------+
382
- ```
383
-
384
- **libarchive Compatibility**:
385
-
386
- All libarchive XAR test cases pass, including:
387
-
388
- * ✅ Regular files with various compression methods
389
- * ✅ Hardlinks and symlinks
390
- * ✅ Character and block devices
391
- * ✅ Directories and FIFOs
392
- * ✅ Extended attributes
393
- * ✅ Various checksum algorithms
394
-
395
- ==== RPM Format Support (v0.4.0)
396
-
397
- Omnizip provides complete RPM package format support for reading and writing RPM packages.
398
-
399
- **Status**: ✅ **Full Support** - Complete RPM reading and writing
400
-
401
- **What Works**:
402
-
403
- * ✅ RPM lead parsing (magic, version, name, architecture)
404
- * ✅ Header parsing with tag extraction (NAME, VERSION, RELEASE, etc.)
405
- * ✅ File list extraction (basenames, directories, permissions)
406
- * ✅ Dependency information (requires, provides, conflicts)
407
- * ✅ Payload extraction with multiple compression formats
408
- * ✅ gzip, bzip2, xz, zstd decompression support
409
- * ✅ RPM package creation with CPIO payload
410
- * ✅ Multiple compression options for package creation
411
-
412
- **Reading RPM Packages**:
413
-
414
- [source,ruby]
415
- ----
416
- require 'omnizip'
417
-
418
- # Read RPM package metadata
419
- Omnizip::Formats::Rpm.open('package.rpm') do |rpm|
420
- puts "Name: #{rpm.name}"
421
- puts "Version: #{rpm.version}"
422
- puts "Release: #{rpm.release}"
423
- puts "Architecture: #{rpm.architecture}"
424
- puts "Files: #{rpm.files.count}"
425
- end
426
-
427
- # Extract RPM contents
428
- Omnizip::Formats::Rpm.extract('package.rpm', 'output/')
429
-
430
- # List files in RPM
431
- files = Omnizip::Formats::Rpm.list('package.rpm')
432
- files.each { |f| puts f }
433
-
434
- # Get package information
435
- info = Omnizip::Formats::Rpm.info('package.rpm')
436
- puts "#{info[:name]}-#{info[:version]}-#{info[:release]}"
437
- ----
438
-
439
- **Writing RPM Packages**:
440
-
441
- [source,ruby]
442
- ----
443
- require 'omnizip'
444
-
445
- # Create RPM package with gzip-compressed payload (default)
446
- rpm = Omnizip::Formats::Rpm::Writer.new(
447
- name: 'mypackage', version: '1.0', release: '1', arch: 'noarch'
448
- )
449
-
450
- # Add payload entries (content is passed as a string)
451
- rpm.add_file('/usr/bin/myapp', File.binread('path/to/myapp'), mode: 0o755)
452
- rpm.add_file('/etc/myapp.conf', File.binread('path/to/config'))
453
- rpm.add_directory('/var/lib/myapp')
454
-
455
- rpm.write('mypackage-1.0-1.noarch.rpm')
456
-
457
- # Supported compression types: :gzip (default), :bzip2, :xz, :zstd, :none
458
- Omnizip::Formats::Rpm::Writer.new(
459
- name: 'mypackage', version: '1.0', release: '1',
460
- arch: 'x86_64', compression: :xz
461
- )
462
- ----
463
-
464
- **Architecture**:
465
-
466
- * `Formats::Rpm::Reader` - Public API for reading RPM packages
467
- * `Formats::Rpm::Writer` - Public API for writing RPM packages
468
- * `Formats::Rpm::Lead` - 96-byte lead parser
469
- * `Formats::Rpm::Header` - Header structure with tag extraction
470
- * `Formats::Rpm::Entry` - File entry model
471
-
472
- ==== OLE Format Support (v0.4.0)
473
-
474
- Omnizip provides complete OLE (Object Linking and Embedding) compound document format support for reading and writing Microsoft compound files.
475
-
476
- **Status**: ✅ **Full Support** - Complete OLE reading and writing
477
-
478
- **What Works**:
479
-
480
- * ✅ OLE compound document header parsing
481
- * ✅ Block allocation tables (BAT, SBAT, XBAT)
482
- * ✅ Directory entry navigation
483
- * ✅ File stream extraction
484
- * ✅ Support for .doc, .xls, .ppt, .msi files
485
- * ✅ Property set storage
486
- * ✅ OLE compound document creation
487
- * ✅ Stream writing with BAT/SBAT management
488
-
489
- **Reading OLE Documents**:
490
-
491
- [source,ruby]
492
- ----
493
- require 'omnizip'
494
-
495
- # Open OLE compound document
496
- Omnizip::Formats::Ole.open('document.doc') do |ole|
497
- # List all streams in the document
498
- ole.each_entry do |entry|
499
- puts "#{entry.name} (#{entry.size} bytes)"
500
- end
501
-
502
- # Read a specific stream
503
- data = ole.read_stream('WordDocument')
504
- end
505
-
506
- # Extract all streams
507
- Omnizip::Formats::Ole.extract('document.doc', 'output/')
508
- ----
509
-
510
- **Writing OLE Documents**:
511
-
512
- [source,ruby]
513
- ----
514
- require 'omnizip'
515
-
516
- # Create new OLE compound document
517
- Omnizip::Formats::Ole.write('output.doc') do |ole|
518
- # Add streams (files) to the document
519
- ole.add_stream('WordDocument', word_data)
520
- ole.add_stream('\x01CompObj', compobj_data)
521
- ole.add_stream('\x05SummaryInformation', summary_data)
522
-
523
- # Add nested storage (directory)
524
- ole.add_storage('MyStorage')
525
- end
526
-
527
- # Create from existing files
528
- Omnizip::Formats::Ole.write('output.msi') do |ole|
529
- ole.add_stream_from_file('BinaryFile', 'path/to/file.bin')
530
- ole.add_stream('\x05DigitalSignature', signature_data)
531
- end
532
- ----
533
-
534
- **Architecture**:
535
-
536
- * `Formats::Ole::Storage` - Core storage implementation
537
- * `Formats::Ole::Writer` - Public API for writing OLE documents
538
- * `Formats::Ole::Header` - 512-byte header parser
539
- * `Formats::Ole::AllocationTable` - BAT/SBAT management
540
- * `Formats::Ole::Dirent` - 128-byte directory entry
541
- * `Formats::Ole::RangesIO` - Range-based IO wrapper
542
- * `Formats::Ole::Types` - Type serialization (Variant, Lpstr, FileTime, etc.)
543
-
544
- === Preprocessing Filters
545
-
546
- * **BCJ Filters** - Branch-Call-Jump filters for executables (x86, ARM, ARM64, PPC, SPARC, IA-64)
547
- * **BCJ2** - Advanced 4-stream x86 filter
548
- * **Delta** - Delta encoding for multimedia/databases
549
-
550
- See link:readme-docs/preprocessing-filters.adoc[Preprocessing Filters Guide] for details.
551
-
552
- === Archive Formats
553
-
554
- * **.7z** - Full read/write with solid compression, multi-volume support
555
- * **ZIP** - Full read/write with ZIP64, WinZip AES encryption
556
- * **RAR4** - Read support for real WinRAR archives (STORE natively; compressed methods via unrar), write with STORE verified against unrar, FASTEST through BEST for Omnizip round-trip (v0.3.0)
557
- * **RAR5** - Read/write with STORE (unrar-verified), AES-256 encryption, multi-volume; compressed-method writing falls back to STORE (v0.3.0)
558
- * **TAR** - Full read/write with POSIX extensions
559
- * **ISO 9660** - Full read/write with Rock Ridge/Joliet
560
- * **CPIO** - Full read/write (newc, CRC formats) with RPM payload support (v0.4.0)
561
- * **RPM** - Full read/write support with metadata extraction, gzip/bzip2/xz/zstd payload compression (v0.4.0)
562
- * **XAR** - Full read/write with XML TOC, gzip/bzip2/lzma compression (v0.4.0)
563
- * **OLE** - Full read/write support for Microsoft compound documents (.doc, .xls, .ppt, .msi) (v0.4.0)
564
- * **GZIP/XZ/BZIP2** - Single file compression formats
565
-
566
- See link:readme-docs/archive-formats.adoc[Archive Formats Documentation] for complete details.
567
-
568
- === PAR2 Parity Archives
569
-
570
- Full support for PAR2 (Parity Archive Volume 2) error correction using Reed-Solomon codes over GF(2^16):
571
-
572
- * Detect data corruption at block level using MD5 checksums
573
- * Verify file integrity without unpacking
574
- * Repair corrupted or missing files automatically
575
- * Protect multiple files in a single archive set
576
- * Configurable redundancy from 0-100%
577
- * Full par2cmdline compatibility (v0.2.0)
578
-
579
- See link:readme-docs/par2-archives.adoc[PAR2 Parity Archives Guide] for comprehensive documentation.
580
-
581
- === Advanced Features
582
-
583
- * **Compression Profiles** - Smart algorithm selection based on file type
584
- * **Format Converter** - Convert between ZIP and 7z formats
585
- * **Performance Profiler** - Identify bottlenecks and optimize
586
- * **Progress Tracking** - Real-time progress with ETA calculation
587
- * **Selective Extraction** - Glob, regex, and predicate-based extraction
588
- * **Parallel Processing** - Multi-threaded compression using Ractors
589
- * **Encryption** - AES-256 password protection with SHA-256 key derivation
590
- * **Checksums** - CRC32/CRC64 integrity verification
591
- * **Enumerable Collections** - All archive and result classes support Ruby's Enumerable interface
24
+ * `rake rust:build` — build and vendor the cdylib from a checkout.
592
25
 
593
- See link:readme-docs/advanced-features.adoc[Advanced Features Guide] for details.
26
+ A Rust-side failure falls back to the Ruby core mid-operation, so
27
+ acceleration is never worse than pure Ruby.
594
28
 
595
- === Test Coverage (v0.4.0)
29
+ External code must `require "omnizip"`. Files under `lib/omnizip/**` are
30
+ internal — requiring them directly bypasses the autoload chain.
596
31
 
597
- Omnizip maintains comprehensive test coverage:
32
+ == Command line
598
33
 
599
- * **Total Tests**: 3540+ examples
600
- * **Pass Rate**: 100% (0 failures, 0 pending)
601
- * **Coverage**: All compression algorithms, archive formats, and features
602
- * **Integration**: Full round-trip verification for all formats
603
- * **Reference Tests**: libarchive RAR4/RAR5 compatibility verified (103 test files)
604
- * **New Formats**: RPM (21 tests), CPIO (25 tests), OLE (36 tests), XAR (17 tests)
605
-
606
- == Quick Start
607
-
608
- === Installation
609
-
610
- [source,sh]
611
- ----
612
- # Via Bundler
613
- gem 'omnizip'
614
-
615
- # Via gem command
616
- gem install omnizip
617
- ----
618
-
619
- External code must `require "omnizip"` to use the gem. Files under
620
- `lib/omnizip/**` are internal — requiring them directly (e.g.
621
- `require "omnizip/formats/cpio"`) bypasses the autoload chain that
622
- wires the namespace modules, and lookups for sibling constants may
623
- fail with `NameError`.
624
-
625
- See link:readme-docs/installation.adoc[Installation Guide] for complete instructions.
626
-
627
- === Command Line
628
-
629
- [source,sh]
630
- ----
631
- # Compress a file
34
+ ....
632
35
  omnizip compress input.txt output.lzma --level 9
633
-
634
- # Create a .7z archive
635
36
  omnizip archive create backup.7z documents/ photos/
636
-
637
- # Extract an archive
638
37
  omnizip archive extract backup.7z output/
639
-
640
- # List archive contents
641
38
  omnizip archive list backup.7z
642
- ----
643
-
644
- See link:readme-docs/cli-usage.adoc[CLI Usage Guide] for all commands and options.
645
-
646
- === Ruby API
39
+ omnizip archive verify backup.rar
40
+ omnizip archive repair damaged.rar
41
+ omnizip parity create backup.rar --percent 10
42
+ omnizip parity verify backup.par2
43
+ omnizip parity repair backup.par2
44
+ omnizip profile list
45
+ omnizip metadata archive.rar
46
+ ....
647
47
 
648
- ==== Simple Convenience Methods
48
+ == Ruby API
649
49
 
650
- [source,ruby]
651
- ----
652
- require 'omnizip'
50
+ === One-liners
653
51
 
654
- # One-liners for common operations
655
- Omnizip.compress_file('input.txt', 'output.zip')
656
- Omnizip.extract_archive('archive.zip', 'output/')
657
- Omnizip.list_archive('archive.zip')
658
- ----
52
+ ....
53
+ require "omnizip"
659
54
 
660
- ==== In-Memory Archives
55
+ Omnizip.compress_file("input.txt", "output.zip")
56
+ Omnizip.extract_archive("archive.zip", "output/")
57
+ Omnizip.list_archive("archive.zip")
58
+ ....
661
59
 
662
- `Omnizip::Buffer` creates and reads ZIP and 7z archives entirely in
663
- memory (7z bridges through a temporary file internally):
60
+ === In-memory archives
664
61
 
665
- [source,ruby]
666
- ----
667
- require 'omnizip'
62
+ `Omnizip::Buffer` creates and reads ZIP and 7z archives entirely in memory:
668
63
 
669
- # Create in memory
64
+ ....
670
65
  buffer = Omnizip::Buffer.create(:zip) do |archive|
671
- archive.add('readme.txt', 'Hello World')
672
- archive.add('docs/', '') # directory entry
673
- end
674
-
675
- # Same API for 7z
676
- buffer7z = Omnizip::Buffer.create(:seven_zip) do |archive|
677
- archive.add('data.json', '{"key": "value"}')
678
- end
679
-
680
- # Read back (format auto-detected)
681
- files = Omnizip::Buffer.extract_to_memory(buffer.string)
682
- # => {"readme.txt" => "Hello World"}
683
-
684
- Omnizip::Buffer.open(buffer.string) do |archive|
685
- archive.each_entry do |entry|
686
- puts "#{entry.name}: #{entry.size} bytes" unless entry.directory?
687
- end
688
- end
689
- ----
690
-
691
- ==== Rubyzip Compatibility Mode
692
-
693
- [source,ruby]
694
- ----
695
- require 'omnizip/rubyzip_compat'
696
-
697
- # Drop-in replacement for rubyzip
698
- Zip::File.open('archive.zip', create: true) do |zip|
699
- zip.add('file.txt') { 'Content' }
700
- end
701
-
702
- Zip::File.open('archive.zip') do |zip|
703
- content = zip.read('file.txt')
66
+ archive.add("readme.txt", "Hello World")
704
67
  end
705
- ----
68
+ names = Omnizip::Backends.archive_entry_names(buffer.to_s)
69
+ ....
706
70
 
707
- ==== Native Omnizip API
71
+ === Archive files
708
72
 
709
- [source,ruby]
710
- ----
711
- require 'omnizip'
712
-
713
- # Full control with algorithm registry
714
- algorithm = Omnizip::AlgorithmRegistry.get(:lzma2).new(level: 9)
715
- File.open('input.txt', 'rb') do |input|
716
- File.open('output.lzma', 'wb') do |output|
717
- algorithm.compress(input, output)
718
- end
719
- end
720
-
721
- # .7z archive operations
722
- Omnizip::Formats::SevenZip.create('archive.7z') do |sz|
723
- sz.add_file('document.pdf')
73
+ ....
74
+ Omnizip::Zip::File.open("in.zip") { |z| z.extract("a.txt", "out.txt") }
75
+ Omnizip::Zip::File.open("out.zip", create: true) do |z|
76
+ z.add_file("a.txt")
724
77
  end
725
- ----
726
-
727
- ==== RAR Archives
728
-
729
- ==== RAR4 Archives (v0.3.0)
730
-
731
- RAR4 reading and writing against the on-disk format unrar accepts:
732
- headers carry the low 16 bits of a CRC32, HEAD_SIZE counts the whole
733
- block, and the dictionary mask 0xE0 marks directory entries. STORE
734
- archives round-trip through official unrar byte-identically.
735
-
736
- ====== Reading RAR4 Archives
737
-
738
- [source,ruby]
739
- ----
740
- require 'omnizip'
741
-
742
- reader = Omnizip::Formats::Rar::Reader.new('archive.rar')
743
- reader.open
744
-
745
- reader.list_files.each do |entry|
746
- puts "#{entry.name}: #{entry.size} bytes (#{entry.compressed_size} compressed)"
747
- puts " Directory: #{entry.is_dir}"
748
- puts " Modified: #{entry.mtime}"
749
- end
750
-
751
- # STORE data decodes natively; WinRAR-compressed entries fall back to
752
- # the unrar command after a CRC check rejects foreign streams
753
- reader.extract_entry('test.txt', 'output/test.txt')
754
- reader.extract_all('output/')
755
- ----
756
-
757
- **RAR4 Reader Features:**
758
-
759
- * ✅ Block header parsing (MAIN/FILE/ENDARC), directories, empty files
760
- * ✅ Native STORE decompression with stored-CRC32 verification
761
- * ✅ unrar fallback for real WinRAR-compressed entries (byte-exact)
762
- * ✅ DOS-style backslash name normalization to forward slashes
763
- * ✅ Multi-volume archive detection
764
- * ✅ Parsing verified against libarchive WinRAR fixtures
765
78
 
766
- ====== Writing RAR4 Archives
79
+ names = Omnizip::Backends.archive_entry_names("archive.rar")
80
+ data = Omnizip::Backends.archive_read_entry("archive.rar", "doc.txt",
81
+ password: "secret")
82
+ ....
767
83
 
768
- [source,ruby]
769
- ----
770
- require 'omnizip'
84
+ === Codecs directly
771
85
 
772
- # STORE output is verified interoperable with unrar
773
- writer = Omnizip::Formats::Rar::Writer.new('archive.rar',
774
- compression: :store
775
- )
776
- writer.add_file('document.txt')
777
- writer.add_file('image.png')
778
- writer.add_directory('photos/')
779
- writer.write
780
- ----
86
+ ....
87
+ compressed = Omnizip::Algorithms::Lzma.compress(data, level: 9)
88
+ plain = Omnizip::Algorithms::Lzma.decompress(compressed)
781
89
 
782
- Writer options: `compression:` (see below), `test_after_create: true`
783
- (reads the archive back and verifies every file). RAR4 encryption is
784
- not implemented — `password:` raises `NotImplementedError`. The
785
- `solid:`/`recovery:`/`volume_size:` options are accepted but not
786
- implemented; they are ignored with a warning and never advertised in
787
- header flags.
90
+ Omnizip::Algorithms::Zstandard.compress(data, level: 19)
91
+ Omnizip::Algorithms::Bzip2.compress(data)
92
+ Omnizip::Algorithms::Ppmd7.compress(data, order: 8)
93
+ ....
788
94
 
789
- ====== RAR4 Compression Methods
95
+ === SDK-compatible LZMA
790
96
 
791
- [cols="2,2,2,4",options="header"]
792
- |===
793
- |Method |Speed |unrar interop |Notes
794
- |`:store` |Instant |✅ verified |Byte-identical round-trip incl. directories
795
- |`:fastest` |Very Fast |❌ Omnizip-only |LZ77+Huffman, readable by Omnizip
796
- |`:normal` (default) |Fast |❌ Omnizip-only |LZ77+Huffman, readable by Omnizip
797
- |`:good` |Medium |❌ Omnizip-only |LZ77+Huffman, readable by Omnizip
798
- |`:best` |Slow |❌ Omnizip-only |PPMd, readable by Omnizip
799
- |===
800
-
801
- Official RAR LZ/PPMd streams are proprietary; methods above `:store`
802
- produce archives Omnizip can read back but unrar reports as checksum
803
- errors. Use `:store` for interoperability.
804
-
805
- ===== RAR5 Archives (v0.3.0)
806
-
807
- Read/write support for RAR5 archives with STORE compression,
808
- AES-256 encryption, and multi-volume archives — all verified against
809
- official unrar.
810
-
811
- ====== Reading RAR5 Archives
812
-
813
- [source,ruby]
814
- ----
815
- require 'omnizip'
816
-
817
- reader = Omnizip::Formats::Rar5::Reader.new
818
- File.open('archive.rar', 'rb') do |io|
819
- entries = reader.read_archive(io)
820
-
821
- entries.each do |entry|
822
- puts "#{entry.name}: #{entry.uncompressed_size} bytes"
823
- puts " CRC32: #{entry.crc32.to_s(16)}" if entry.crc32
824
- puts " Modified: #{entry.modified_time}" if entry.modified_time
825
- end
97
+ ....
98
+ File.open("out.lzma", "wb") do |f|
99
+ Omnizip::Algorithms::LZMA::Encoder.new(f, sdk_compatible: true)
100
+ .encode_stream("Hello, World!")
826
101
  end
827
- ----
828
-
829
- **RAR5 Reader Features:**
830
-
831
- * ✅ STORE file data, header CRC32 verification
832
- * ✅ 32-bit Unix mtime decoding (matches unrar; nil when absent)
833
- * ✅ Unicode filenames (UTF-8)
834
- * ✅ Multi-file archives
835
- * ✅ VInt (variable-length integer) parsing
836
- * ✅ Graceful error handling for truncated/invalid files
837
- * ✅ libarchive compatibility fixtures
838
-
839
- ====== Writing RAR5 Archives
840
-
841
- [source,ruby]
842
- ----
843
- require 'omnizip'
844
-
845
- # STORE compression (default) — unrar-verified
846
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar')
847
- writer.add_file('document.txt')
848
- writer.add_file('image.png')
849
- writer.write
850
-
851
- # AES-256 encryption
852
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
853
- password: 'secret'
854
- )
855
- writer.add_file('data.json')
856
- writer.write
857
-
858
- # Multi-volume archives split at 64 KB boundaries
859
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
860
- multi_volume: true,
861
- volume_size: 65536
862
- )
863
- writer.add_file('big1.bin')
864
- writer.add_file('big2.bin')
865
- writer.write # writes archive.part1.rar, archive.part2.rar, ...
866
- ----
867
-
868
- ====== RAR5 Compression Methods
869
-
870
- [cols="2,2,4",options="header"]
871
- |===
872
- |Method |Interop |Notes
873
- |`:store` (default) |✅ unrar-verified |Uncompressed passthrough
874
- |`:lzss` / `:lzma` |falls back to STORE |Encoder is not official-RAR compatible; warns and stores
875
- |`:auto` |STORE |Size-based selection
876
- |===
877
-
878
- ====== RAR5 Optional Fields
879
-
880
- The writer can embed metadata that readers use without decompressing:
881
-
882
- [source,ruby]
883
- ----
884
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
885
- include_mtime: true, # 32-bit Unix mtime per file header
886
- include_crc32: true # CRC32 per file header
887
- )
888
- ----
889
-
890
- ====== RAR5 Optional Fields
891
-
892
- RAR5 supports optional metadata fields for enhanced archive information:
893
-
894
- [source,ruby]
895
- ----
896
- require 'omnizip/formats/rar/rar5/writer'
897
-
898
- # Include modification time in archive
899
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
900
- include_mtime: true # Preserves file modification timestamps
901
- )
902
- writer.add_file('document.txt')
903
- writer.write
904
-
905
- # Include CRC32 checksums (STORE compression only)
906
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
907
- compression: :store,
908
- include_crc32: true # Only works with STORE compression
909
- )
910
- writer.add_file('data.bin')
911
- writer.write
912
-
913
- # IMPORTANT: CRC32 with LZMA is automatically disabled
914
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
915
- compression: :lzma,
916
- level: 5,
917
- include_crc32: true # Will be auto-disabled, no error
918
- )
919
- writer.add_file('document.txt')
920
- writer.write
921
- # CRC32 will be silently disabled - archive uses BLAKE2sp only
922
- ----
923
-
924
- **CRC32 Limitation Explained:**
925
-
926
- RAR5's optional CRC32 field is incompatible with compression algorithms. The RAR5 format specification requires that when compression is used (LZMA, LZMA2), only the BLAKE2sp checksum (always present in the main file header) should be used for integrity verification. The optional CRC32 field is designed for uncompressed (STORE) files only.
927
-
928
- When you request `include_crc32: true` with LZMA compression, Omnizip automatically disables CRC32 to ensure format compliance and compatibility with official unrar tools.
929
-
930
- ====== RAR5 Multi-Volume Archives
931
-
932
- Create split archives when file size exceeds volume limit. Volumes are automatically created and numbered according to the chosen naming pattern.
933
-
934
- [source,ruby]
935
- ----
936
- require 'omnizip/formats/rar/rar5/writer'
937
-
938
- # Create multi-volume archive with default settings
939
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
940
- multi_volume: true,
941
- volume_size: '10M' # Human-readable size
942
- )
943
- writer.add_directory('data/')
944
- volumes = writer.write # Returns array of volume paths
945
- # => ['archive.part1.rar', 'archive.part2.rar', ...]
946
-
947
- # Custom volume naming pattern
948
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup.rar',
949
- multi_volume: true,
950
- volume_size: '100M',
951
- volume_naming: 'volume' # backup.volume1.rar, backup.volume2.rar
952
- )
953
- writer.add_directory('large_dataset/')
954
- volumes = writer.write
955
-
956
- # Numeric naming pattern
957
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('data.rar',
958
- multi_volume: true,
959
- volume_size: '50M',
960
- volume_naming: 'numeric' # data.001.rar, data.002.rar
961
- )
962
- writer.add_file('huge_file.bin')
963
- volumes = writer.write
964
- ----
965
-
966
- **Volume Naming Patterns:**
967
-
968
- * `part` (default): archive.part1.rar, archive.part2.rar, ...
969
- * `volume`: archive.volume1.rar, archive.volume2.rar, ...
970
- * `numeric`: archive.001.rar, archive.002.rar, ...
971
-
972
- **Human-Readable Sizes:**
973
-
974
- * Bytes: `1024`, `2048`
975
- * Kilobytes: `10K`, `100KB`
976
- * Megabytes: `10M`, `100MB`
977
- * Gigabytes: `1G`, `5GB`
102
+ ....
978
103
 
979
- **Minimum volume size:** 64 KB (65,536 bytes)
104
+ === Profiles, conversion, streaming, repair
980
105
 
981
- ====== RAR5 Solid Compression
982
-
983
- Compress multiple files with a shared dictionary for significantly better compression ratios. Ideal for similar files such as source code, logs, or document collections.
984
-
985
- [source,ruby]
986
- ----
987
- require 'omnizip/formats/rar/rar5/writer'
988
-
989
- # Enable solid compression (default: 10-30% better ratio)
990
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
991
- compression: :lzma,
992
- level: 5,
993
- solid: true # Enable solid mode
994
- )
995
- writer.add_directory('project/')
996
- writer.write
997
-
998
- # Combine with high compression level
999
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup.rar',
1000
- compression: :lzma,
1001
- level: 5, # Best compression
1002
- solid: true # Shared dictionary
1003
- )
1004
- writer.add_file('log1.txt')
1005
- writer.add_file('log2.txt')
1006
- writer.add_file('log3.txt')
1007
- writer.write
1008
- ----
1009
-
1010
- **Benefits:**
1011
-
1012
- * 10-30% better compression ratios for similar files
1013
- * Larger LZMA dictionaries (16-64 MB vs 1-16 MB)
1014
- * Particularly effective for:
1015
- - Source code repositories
1016
- - Log files and text documents
1017
- - Similar structured data
1018
-
1019
- **Trade-offs:**
1020
-
1021
- * Cannot extract individual files without decompressing entire solid block
1022
- * Corruption in one file may affect subsequent files in the block
1023
- * Slightly longer extraction time for single files
1024
- * Best for archiving complete collections
1025
-
1026
- ====== RAR5 AES-256 Encryption
1027
-
1028
- Protect archives with industry-standard AES-256-CBC encryption and PBKDF2-HMAC-SHA256 key derivation.
1029
-
1030
- [source,ruby]
1031
- ----
1032
- require 'omnizip/formats/rar/rar5/writer'
1033
-
1034
- # Basic password protection
1035
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
1036
- compression: :lzma,
1037
- password: 'SecurePassword123!'
1038
- )
1039
- writer.add_file('confidential.pdf')
1040
- writer.write
1041
-
1042
- # Custom key derivation iterations
1043
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('secure.rar',
1044
- compression: :lzma,
1045
- level: 5,
1046
- password: 'VerySecurePassword2025!',
1047
- kdf_iterations: 524_288 # Higher = more secure, slower
1048
- )
1049
- writer.add_directory('sensitive_data/')
1050
- writer.write
1051
-
1052
- # Minimum security (faster but less secure)
1053
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('quick.rar',
1054
- password: 'FastPassword',
1055
- kdf_iterations: 65_536 # Minimum allowed
1056
- )
1057
- writer.add_file('temp.txt')
1058
- writer.write
1059
- ----
1060
-
1061
- **Security Features:**
1062
-
1063
- * **AES-256-CBC encryption** with PKCS#7 padding
1064
- * **PBKDF2-HMAC-SHA256** key derivation function
1065
- * **Configurable KDF iterations:**
1066
- - Minimum: 65,536 (2^16) - fast but less secure
1067
- - Default: 262,144 (2^18) - balanced security/performance
1068
- - Maximum: 1,048,576 (2^20) - maximum security
1069
- * **Per-file IV generation** for enhanced security
1070
- * **Password verification** before decryption attempts
1071
-
1072
- **Performance Impact:**
1073
-
1074
- * Encryption overhead: < 2x slower than unencrypted
1075
- * KDF computation time varies with iteration count:
1076
- - 65,536 iterations: ~50-100ms
1077
- - 262,144 iterations: ~200-400ms
1078
- - 1,048,576 iterations: ~800-1600ms
1079
-
1080
- ====== RAR5 PAR2 Recovery Records
1081
-
1082
- Generate PAR2 parity files for archive recovery and error correction using Reed-Solomon codes.
1083
-
1084
- [source,ruby]
1085
- ----
1086
- require 'omnizip/formats/rar/rar5/writer'
1087
-
1088
- # Enable recovery with default 5% redundancy
1089
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('archive.rar',
1090
- compression: :lzma,
1091
- recovery: true
1092
- )
1093
- writer.add_directory('important_data/')
1094
- files = writer.write
1095
- # => ['archive.rar', 'archive.par2', 'archive.vol00+01.par2', ...]
1096
-
1097
- # Custom redundancy percentage
1098
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup.rar',
1099
- compression: :lzma,
1100
- level: 5,
1101
- recovery: true,
1102
- recovery_percent: 10 # 10% redundancy (can recover up to 10% data loss)
1103
- )
1104
- writer.add_file('critical.db')
1105
- files = writer.write
1106
-
1107
- # Maximum redundancy for critical data
1108
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('critical.rar',
1109
- compression: :lzma,
1110
- recovery: true,
1111
- recovery_percent: 50 # 50% redundancy (maximum protection)
1112
- )
1113
- writer.add_directory('mission_critical/')
1114
- files = writer.write
1115
- ----
1116
-
1117
- **Recovery Capabilities:**
1118
-
1119
- * **Detect corruption** at block level
1120
- * **Repair damaged archives** automatically
1121
- * **Recover from partial data loss** up to redundancy percentage
1122
- * **Works with all features:**
1123
- - Multi-volume archives
1124
- - Solid compression
1125
- - Encrypted archives
1126
- * **Reed-Solomon error correction** over GF(2^16)
1127
-
1128
- **Redundancy Guidelines:**
1129
-
1130
- * 5% (default): Suitable for general backups
1131
- * 10%: Recommended for important data
1132
- * 20-30%: High-value data requiring extra protection
1133
- * 50-100%: Critical data with maximum recovery needs
1134
-
1135
- **PAR2 File Size:**
1136
-
1137
- PAR2 files add approximately the redundancy percentage to total archive size. For example, a 100MB archive with 10% redundancy will generate ~10MB of PAR2 files.
1138
-
1139
- ====== RAR5 Combined Features
1140
-
1141
- All RAR5 features can be used together for comprehensive archive protection:
1142
-
1143
- [source,ruby]
1144
- ----
1145
- require 'omnizip/formats/rar/rar5/writer'
1146
-
1147
- # Complete feature demonstration
1148
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('complete.rar',
1149
- # Compression
1150
- compression: :lzma,
1151
- level: 5, # Best compression
1152
- solid: true, # Shared dictionary for better ratios
1153
-
1154
- # Security
1155
- password: 'SecureBackup2025!',
1156
- kdf_iterations: 524_288, # Enhanced security
1157
-
1158
- # Multi-volume
1159
- multi_volume: true,
1160
- volume_size: '100M',
1161
- volume_naming: 'part',
1162
-
1163
- # Recovery
1164
- recovery: true,
1165
- recovery_percent: 10,
1166
-
1167
- # Optional fields
1168
- include_mtime: true
1169
- )
1170
-
1171
- writer.add_directory('/critical/data')
1172
- files = writer.write
1173
- # => ['complete.part1.rar', 'complete.part2.rar', ...,
1174
- # 'complete.par2', 'complete.vol00+01.par2', ...]
1175
- ----
1176
-
1177
- **Best Practices:**
1178
-
1179
- 1. **Solid + LZMA level 5** for maximum compression on similar files
1180
- 2. **10-20% PAR2** for important data protection
1181
- 3. **262,144 KDF iterations** for balanced security/performance
1182
- 4. **Multi-volume** for large archives or optical media
1183
- 5. **Always include mtime** to preserve file timestamps
1184
-
1185
- **Example: Secure Backup Archive**
1186
-
1187
- [source,ruby]
1188
- ----
1189
- # Production-ready backup configuration
1190
- writer = Omnizip::Formats::Rar::Rar5::Writer.new('backup_2025-12-24.rar',
1191
- compression: :lzma,
1192
- level: 5,
1193
- solid: true, # 10-30% better compression
1194
- password: ENV['BACKUP_PASSWORD'] || 'DefaultSecure123!',
1195
- kdf_iterations: 262_144, # Balanced security
1196
- multi_volume: true,
1197
- volume_size: '4G', # DVD-sized volumes
1198
- recovery: true,
1199
- recovery_percent: 15, # 15% redundancy
1200
- include_mtime: true
1201
- )
1202
-
1203
- writer.add_directory('/home/user/documents')
1204
- writer.add_directory('/home/user/projects')
1205
- files = writer.write
1206
-
1207
- puts "Backup created: #{files.size} files"
1208
- puts "Total size: #{files.sum { |f| File.size(f) } / 1024 / 1024}MB"
1209
- ----
1210
-
1211
- **Important:**
1212
- - Ensure you have set the `BACKUP_PASSWORD` environment variable before running the secure backup example.
1213
- - This example assumes a Linux/Unix environment; file paths may need adjustments for Windows.
1214
-
1215
- **Security Note:**
1216
- - Use a strong, complex password for `BACKUP_PASSWORD`.
1217
- - Consider using a password manager to store and retrieve your backup password securely.
1218
- - If using this code in production, review the security implications and adjust as needed.
1219
-
1220
- **Performance Note:**
1221
- - Encryption and KDF computations can be CPU-intensive.
1222
- - The `kdf_iterations` value affects security; higher values are more secure but slower.
1223
- - The `volume_naming` option can impact the efficiency and naming of multi-volume archives.
1224
-
1225
- **Error Handling:**
1226
- - Enhance this example by adding error handling for file operations and encryption failures.
1227
-
1228
- Dresses your ruby file as README.md (see https://guides.github.com/features/mastering-markdown/).
106
+ Content-class profiles pick the codec automatically
107
+ (`Omnizip::Profile::DEFAULT` and friends); `Omnizip::Converter` moves
108
+ archives between formats; the chunked reader/writer processes inputs far
109
+ larger than RAM; `Omnizip::ArchiveRepairCommand` routes structural
110
+ repair. See the guides below.
1229
111
 
1230
112
  == Documentation
1231
113
 
1232
- === Core Documentation
1233
-
1234
- * **link:readme-docs/installation.adoc[Installation Guide]** - Setup and system requirements
1235
- * **link:readme-docs/cli-usage.adoc[CLI Usage]** - Command-line interface reference
1236
- * **link:readme-docs/api-usage.adoc[Library API]** - Programmatic Ruby API guide
1237
- * **link:readme-docs/compression-algorithms.adoc[Compression Algorithms]** - LZMA, BZip2, PPMd, Deflate, Zstandard
1238
- * **link:readme-docs/preprocessing-filters.adoc[Preprocessing Filters]** - BCJ, BCJ2, Delta filters
1239
- * **link:readme-docs/encryption-checksums.adoc[Encryption & Checksums]** - AES-256, CRC32/CRC64
1240
- * **link:readme-docs/architecture.adoc[Architecture]** - System design, patterns, extensibility
1241
-
1242
- === Format Documentation
1243
-
1244
- * **link:readme-docs/archive-formats.adoc[Archive Formats]** - .7z, ZIP, RAR, TAR, ISO, CPIO
1245
- * **link:readme-docs/par2-archives.adoc[PAR2 Parity Archives]** - Error correction and repair
1246
-
1247
- === Advanced Topics
1248
-
1249
- * **link:readme-docs/compression-profiles.adoc[Compression Profiles]** - Smart algorithm selection
1250
- * **link:readme-docs/format-converter.adoc[Format Converter]** - Convert between formats
1251
- * **link:readme-docs/performance-profiler.adoc[Performance Profiler]** - Benchmarking and optimization
1252
- * **link:readme-docs/advanced-features.adoc[Advanced Features]** - Progress, ETA, parallel processing
1253
-
1254
- === Additional Resources
1255
-
1256
- * link:old-docs/MIGRATION_GUIDE.md[Migration from Rubyzip]
1257
- * link:old-docs/COMPATIBILITY_MATRIX.md[Compatibility Matrix]
1258
- * link:old-docs/BASELINE.md[Performance Baseline]
1259
- * link:old-docs/CONTRIBUTING.md[Contributing Guidelines]
114
+ * link:readme-docs/installation.adoc[Installation] ·
115
+ link:readme-docs/cli-usage.adoc[CLI usage] ·
116
+ link:readme-docs/api-usage.adoc[API usage]
117
+ * link:readme-docs/compression-algorithms.adoc[Compression algorithms] ·
118
+ link:readme-docs/compression-profiles.adoc[Profiles] ·
119
+ link:readme-docs/format-converter.adoc[Format converter] ·
120
+ link:readme-docs/advanced-features.adoc[Progress, ETA, parallel] ·
121
+ link:readme-docs/performance-profiler.adoc[Performance profiler]
122
+ * link:readme-docs/archive-formats.adoc[Archive formats] ·
123
+ link:readme-docs/rar-archives.adoc[RAR4/RAR5 guide] ·
124
+ link:readme-docs/par2-archives.adoc[PAR2 parity archives] ·
125
+ link:readme-docs/encryption-checksums.adoc[Encryption and checksums]
126
+ * link:readme-docs/architecture.adoc[Architecture] ·
127
+ link:docs/rust-backend.md[Rust backend] ·
128
+ link:docs/compatibility.adoc[Compatibility] ·
129
+ link:docs/RAR_WRITE_SUPPORT.md[RAR write support]
1260
130
 
1261
131
  == Development
1262
132
 
1263
- === Running Tests
1264
-
1265
- [source,sh]
1266
- ----
1267
- # Run all tests
1268
- bundle exec rspec
1269
-
1270
- # Run specific test file
1271
- bundle exec rspec spec/omnizip/algorithms/lzma_spec.rb
1272
-
1273
- # Run with documentation format
1274
- bundle exec rspec --format documentation
1275
- ----
1276
-
1277
- === Running Linters
1278
-
1279
- [source,sh]
1280
- ----
1281
- # Run RuboCop
1282
- bundle exec rubocop
1283
-
1284
- # Auto-correct offenses
1285
- bundle exec rubocop -A
1286
-
1287
- # Generate config for new offenses
1288
- bundle exec rubocop -A --auto-gen-config
1289
- ----
1290
-
1291
- === Performance Benchmarks
1292
-
1293
- [source,sh]
1294
- ----
1295
- # Run performance benchmarks
1296
- ruby benchmark/run_benchmarks.rb
1297
-
1298
- # View baseline results
1299
- cat benchmark/results/v1.0_baseline.txt
1300
- ----
1301
-
1302
- == Contributing
1303
-
1304
- Contributions are welcome! Please read link:old-docs/CONTRIBUTING.md[CONTRIBUTING.md] for details on our code of conduct, development process, and how to submit pull requests.
1305
-
1306
- **Quick start:**
1307
-
1308
- . Fork the repository
1309
- . Create your feature branch (`git checkout -b feature/my-new-feature`)
1310
- . Make your changes and add tests
1311
- . Run the test suite (`bundle exec rspec`)
1312
- . Run RuboCop (`bundle exec rubocop -A`)
1313
- . Commit your changes with semantic commit messages
1314
- . Push to the branch (`git push origin feature/my-new-feature`)
1315
- . Create a new Pull Request
1316
-
1317
- == Acknowledgments
1318
-
1319
- Omnizip is a completely independent, clean-room implementation of compression algorithms and archive formats. The compression algorithms (LZMA, LZMA2, BZip2, PPMd, Deflate64, etc.) are implemented from publicly available specifications and mathematical descriptions.
1320
-
1321
- Archive formats (7z, ZIP, RAR, TAR, ISO, CPIO) are implemented based on their public format specifications. Similar to libarchive's independent implementations, Omnizip provides open-source, unencumbered implementations of these formats.
1322
-
1323
- IMPORTANT: Compression algorithms themselves are mathematical concepts and cannot be patented. Omnizip's implementations are original work based on algorithm specifications, not derivative of any existing codebase.
1324
-
1325
- == Copyright and License
133
+ ....
134
+ bundle install
135
+ bundle exec rspec # suite
136
+ bundle exec rspec spec/omnizip/algorithms/lzma_spec.rb # one file
137
+ bundle exec rubocop # lint
138
+ rake rust:build # vendor the Rust cdylib locally
139
+ rake docs:build # render docs/site via Coradoc
140
+ rake docs:links # lychee link check
141
+ ....
1326
142
 
1327
- Copyright 2026 https://www.ribose.com[Ribose Inc.]
143
+ == Copyright and license
1328
144
 
1329
- See the link:COPYING[COPYING] file and link:LICENSE[LICENSE] file for the complete text of the licenses.
145
+ Copyright (c) Ribose Inc. Released under the MIT License; see the LICENSE
146
+ file. The Rust engine (omnizip-rs) is MIT OR Apache-2.0.