omnizip 0.3.36 → 0.3.38

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (53) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +135 -9
  3. data/docs/getting-started/basic-usage.adoc +14 -22
  4. data/docs/guides/advanced-features/encryption.adoc +16 -39
  5. data/docs/guides/advanced-features/index.adoc +16 -6
  6. data/docs/guides/advanced-features/parallel-processing.adoc +63 -165
  7. data/docs/guides/advanced-features/progress-tracking.adoc +65 -261
  8. data/docs/guides/advanced-features/streaming.adoc +43 -113
  9. data/docs/guides/archive-formats/gzip-format.adoc +2 -4
  10. data/docs/guides/archive-formats/ole-format.adoc +54 -219
  11. data/docs/guides/archive-formats/rar5.adoc +15 -15
  12. data/docs/guides/archive-formats/rpm-format.adoc +48 -189
  13. data/docs/guides/compression-algorithms/bzip2.adoc +2 -1
  14. data/docs/guides/compression-algorithms/deflate.adoc +1 -7
  15. data/docs/guides/compression-algorithms/lzma.adoc +10 -14
  16. data/docs/guides/compression-algorithms/lzma2.adoc +34 -117
  17. data/docs/guides/compression-algorithms/ppmd.adoc +24 -27
  18. data/docs/guides/compression-algorithms/zstandard.adoc +85 -150
  19. data/docs/reference/api/overview.adoc +16 -17
  20. data/docs/troubleshooting/index.adoc +8 -9
  21. data/lib/omnizip/algorithms/zstandard.rb +2 -3
  22. data/lib/omnizip/archive/builder.rb +48 -0
  23. data/lib/omnizip/archive/reader_session.rb +67 -0
  24. data/lib/omnizip/archive.rb +86 -0
  25. data/lib/omnizip/archive_handler.rb +3 -0
  26. data/lib/omnizip/archive_handlers/cpio_handler.rb +47 -0
  27. data/lib/omnizip/archive_handlers/iso_handler.rb +54 -0
  28. data/lib/omnizip/archive_handlers/rar_handler.rb +50 -0
  29. data/lib/omnizip/archive_handlers/seven_zip_handler.rb +6 -2
  30. data/lib/omnizip/archive_handlers/tar_handler.rb +4 -0
  31. data/lib/omnizip/archive_handlers/zip_handler.rb +25 -2
  32. data/lib/omnizip/archive_handlers.rb +3 -0
  33. data/lib/omnizip/cli.rb +25 -0
  34. data/lib/omnizip/convenience.rb +91 -11
  35. data/lib/omnizip/formats/iso/directory_builder.rb +24 -0
  36. data/lib/omnizip/formats/iso/reader.rb +2 -2
  37. data/lib/omnizip/formats/iso/writer.rb +20 -4
  38. data/lib/omnizip/formats/iso.rb +3 -1
  39. data/lib/omnizip/formats/rar/block_parser.rb +8 -12
  40. data/lib/omnizip/formats/rar/constants.rb +3 -1
  41. data/lib/omnizip/formats/rar/header.rb +16 -6
  42. data/lib/omnizip/formats/rar/rar5/compression/lzss.rb +5 -3
  43. data/lib/omnizip/formats/rar/rar5/header.rb +50 -33
  44. data/lib/omnizip/formats/rar/rar5/multi_volume/volume_writer.rb +17 -34
  45. data/lib/omnizip/formats/rar/rar5/vint.rb +25 -37
  46. data/lib/omnizip/formats/rar/rar5/writer.rb +50 -14
  47. data/lib/omnizip/formats/rar/reader.rb +6 -2
  48. data/lib/omnizip/formats/rar.rb +5 -3
  49. data/lib/omnizip/formats/seven_zip/parser.rb +22 -7
  50. data/lib/omnizip/formats/seven_zip/writer.rb +78 -19
  51. data/lib/omnizip/version.rb +1 -1
  52. data/lib/omnizip.rb +1 -0
  53. metadata +7 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 53eb9dfabf6b8f299e86de445f0b2e487dcb12531ad6c901d35f89b306e53e97
4
- data.tar.gz: 2df326cef1acba3e9de77ee349c4714ba3ba20c559beab62637dd70b2d55c1d8
3
+ metadata.gz: 67142de350c2f3dd29c473268977799ed088cedb0340932cff6db7bc23ac009e
4
+ data.tar.gz: c35fbc3353ca0c20f2ec4f86fa39fff67e6ed95822702a18c9b545d35406749c
5
5
  SHA512:
6
- metadata.gz: 182da8c8f1f5ccf84552f1bb7741f00340165d90bf8a19edcb3299dcdff02d99644b98b87fa13969dd62090c70f9973c97950267b816a3cceda988c53df2430b
7
- data.tar.gz: e8745b56c4aded2fa04e81d0dd0ec011360f4593d590b4127aa284fac62b42766c39e26ce53f67b9f77e6e7cbe024345465a89af544b4594213e5d8811d297a2
6
+ metadata.gz: 7f6b4e88bb0a227f7cff47b06e979891e6d5a57288ae141bd15e57a07abe2cef3c915600242bb05406c4b7a13ef9fdd0ab5f11870e41b65f0a95077191741ceb
7
+ data.tar.gz: 3e8497c8eb222b4a139f965ec84f1672d8b12ae92e55986df95d8057c452bcace18a75c54a0464ca696776a33256d2e11ea53811c7d73e3ecb1c0eece06d04ff
data/CHANGELOG.md CHANGED
@@ -7,6 +7,136 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ### Added
11
+ - `.cpio` and `.iso` read routing: `extract_archive`/`list_archive`/
12
+ `read_from_archive`/`Archive.open` operate on CPIO archives and ISO
13
+ 9660 images through new read-only handlers, mirroring the RAR
14
+ routing (creation keeps raising truthfully).
15
+
16
+ ### Fixed
17
+ - 7z archives with explicit directory entries (or zero-byte
18
+ entries) were structurally invalid: the writer never emitted the
19
+ kEmptyStream/kEmptyFile properties, hardcoded 0x20 attributes
20
+ over each entry's real attributes, and its kSubStreamsInfo
21
+ digest/size arrays counted directory entries as substreams. 7-Zip
22
+ rejects such archives with "Headers Error" (and our own reader
23
+ crashed extracting directory entries). Verified against 7zz
24
+ ground truth (`-mhc=off` dumps): kEmptyStream/kEmptyFile carry
25
+ RAW bit vectors with no all-defined marker — the parser expected
26
+ that marker everywhere, so it now reads both layouts correctly
27
+ (which also fixes reading 7-Zip-created archives containing empty
28
+ files).
29
+ - RAR5 vint encoding was not the spec encoding: multi-byte values
30
+ (any size >= 128, extra-area sizes, dictionary sizes) were written
31
+ in a byte-swapped form no reader decodes — archives with more
32
+ than ~127 bytes of content were corrupt for unrar AND for our own
33
+ spec-conformant parser. All round-trips now verified against
34
+ unrar, including encrypted STORE, multi-file, and multi-volume
35
+ archives (volume flags were the previously documented "no
36
+ write-side reference" gap: Main archive flags 0x0001/0x0002 and
37
+ end-of-archive 0x0001 are written per the RAR 5.0 specification).
38
+ - RAR5 file headers: a "mystery vint" not in the spec shifted every
39
+ field; the compression method was written into the version bits;
40
+ the file-attribute value chmods extracted files unreadable. Now:
41
+ spec field order, method in bits 8-10 with dictionary bits
42
+ 11-15, Unix attribute 0o100644.
43
+ - RAR5 encrypted writing now persists the salt/IV as the file
44
+ encryption extra record (spec type 0x01) — archives it wrote
45
+ before could not be decrypted by anything (unrar reported "All
46
+ OK" only because no CRC was stored).
47
+ - RAR5 `:lzma`/`:lzss` compression advertised method bits for a
48
+ stream that nothing can decode (the encoder is not
49
+ official-RAR-compatible; `unrar` extracts empty files silently).
50
+ `Lzss.available?` is now honest, so writers fall back to STORE
51
+ with a warning; solid mode falls back to independent STORE
52
+ entries.
53
+ - ISO writing: directory records pointed every file at sector 0
54
+ (allocated extents were never copied onto the tree nodes); the
55
+ volume descriptor declared 100 sectors regardless of actual
56
+ size; `add_directory` entries were not marked as directories;
57
+ Rock Ridge and Joliet were advertised by default without being
58
+ implemented (the cloned SVD made 7-Zip read ASCII names as
59
+ UCS-2); `Reader#extract_all` looked entries up by bare name
60
+ instead of full path; `Formats::Iso.list` returned the Reader
61
+ instead of entries. Images now extract byte-identically through
62
+ 7zz, including nested directories.
63
+ - CLI: `omnizip <command> --help` failed with an arity error on
64
+ every command (Thor does not route `--help` on subcommands by
65
+ itself). A dispatch hook now prints per-command help.
66
+ - `Omnizip::Formats::Rar::Rar5::MainHeader/FileHeader` were
67
+ unreferenceable unless another file happened to load header.rb
68
+ first (missing autoload entries); three Rar5 autoloads pointed at
69
+ files that never existed.
70
+
71
+ ### Changed
72
+ - Docs: the OLE, RPM, GZIP, RAR5 and API-overview guides now use
73
+ the real APIs (`Formats::Ole.list/read/info`, `Formats::Rpm.*`,
74
+ `Gzip.compress_stream`, `Formats::Zip::Reader#read`, correct
75
+ error class names); fictional methods (`open_stream`, `root`,
76
+ `extract_to`, `extract_files`, `extract_payload`,
77
+ `Rar::Rar5::Reader`, scriptlets/signature accessors) removed.
78
+
79
+ ## [0.3.37] - 2026-08-28
80
+
81
+ ### Added
82
+ - `Omnizip::Archive` — the block-style facade the guides have
83
+ documented since the docs were written (it never existed; ~50
84
+ examples across 17 files were dead). `Archive.create(path,
85
+ format:, **options)` yields a Builder with `add_file(path,
86
+ archive_path:)`, `add_directory(dir)` (tree stored under its own
87
+ name), and `add_data(name, content)`; `Archive.open(path,
88
+ password:)` yields (or returns) a session with `entries`
89
+ (name/size/directory?/mtime), `each_entry`, `read`, `extract`,
90
+ `extract_all`, `extract_matching`. 7z options (`level:`,
91
+ `password:`, `encrypt_headers:`, `algorithm:`, `filters:`,
92
+ `solid:`, `dict_size:`) pass through to the writer;
93
+ `ENV['OMNIZIP_PASSWORD']` backs `open`. Passwords are refused
94
+ loudly for zip/tar (no encryption support) instead of silently
95
+ producing unprotected archives.
96
+ - `.zst` routing in the convenience API: `compress_file` to `.zst`
97
+ previously wrote a mislabeled ZIP; it now writes a real Zstandard
98
+ frame (validated against the system zstd CLI in both directions),
99
+ and `decompress_file` reads them.
100
+ - Read-only RAR routing: `extract_archive`/`list_archive`/
101
+ `read_from_archive`/`Archive.open` operate on `.rar` files through
102
+ a new read-only `ArchiveHandlers::RarHandler`; creation keeps
103
+ raising truthfully. `resolve_archive_format` is now
104
+ operation-aware (`writing:`), so read operations no longer fail
105
+ with "cannot be written" errors.
106
+ - `compress_file`/`compress_directory` forward compression options
107
+ (`algorithm:`, `level:`, ...) to the format writer instead of
108
+ silently dropping them (7z honors them; zip absorbs them —
109
+ per-entry ZIP compression is set through `Zip::OutputStream`).
110
+
111
+ ### Fixed
112
+ - RAR5 metadata parsing: the file-header parser ignored the
113
+ optional ExtraAreaSize/DataSize vints, misaligning every field —
114
+ all RAR5 entries listed with size 0 (and one fixture failed to
115
+ parse at all). Field order now matches the RAR 5.0 specification
116
+ (verified against RARLAB fixtures); `compressed_size` is populated
117
+ from DataSize and the directory flag is the correct FileFlags bit
118
+ (0x0001). The main-header parser also reads the ArchiveFlags vint
119
+ it was skipping, so volume/solid detection works. Listing now
120
+ prefers the native parser (full metadata) over the external
121
+ `unrar vb` listing (names only, sizes hardcoded to 0).
122
+ - `compress_directory` to a single-file stream extension (`.gz`,
123
+ `.lzma`, ...) silently wrote a ZIP under the foreign name; it now
124
+ raises `UnsupportedFormatError` explaining the mismatch.
125
+
126
+ ### Changed
127
+ - Docs: every example across the guides now runs verbatim against
128
+ the real API. Fictional APIs removed or replaced
129
+ (`Omnizip::OutputStream`/`InputStream`, `compress_files`,
130
+ `train_dictionary`, `Archive#compression=` setters, `tar_bzip2`,
131
+ ZIP password attributes, `CorruptArchiveError`,
132
+ `UnsupportedEncryptionError`); fabricated parallel-processing and
133
+ benchmark sections rewritten as truthful process-level
134
+ parallelism; the zstandard guide now centers on the real
135
+ `.zst`/ZIP-entry/dictionary APIs (and no longer claims a native
136
+ zstd binding).
137
+
138
+ ## [0.3.36] - 2026-08-28
139
+
10
140
  ### Fixed
11
141
  - `add_to_archive`/`remove_from_archive` errors name the resolved
12
142
  format (`Format :seven_zip does not support adding entries`)
@@ -84,15 +214,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
84
214
  been pinned to :copy) and replaced the match-encoding TODO with a
85
215
  wire-level round-trip verification of the distance-8/length
86
216
  pattern through the LZMA2 decoder. No code under lib/ changes.
87
-
88
- ## [0.3.30] - 2026-08-27
89
-
90
- ### Verified
91
- - LZIP interop with the real `lzip` CLI (1.26): our members pass
92
- `lzip -t`/`-dc` and the CLI's members decode through
93
- `Formats::Lzip` (spec skips where the CLI is absent).
94
-
95
- ### Changed
96
217
  - The remaining lib/ TODO markers are resolved as documented design
97
218
  decisions rather than dangling work: RAR5 volume extras and
98
219
  EndHeader volume flags (no write-side reference exists;
@@ -106,6 +227,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
106
227
  the salt/IV header, so archives it writes cannot be decrypted;
107
228
  encryption stays read-verified only.
108
229
 
230
+ ### Verified
231
+ - LZIP interop with the real `lzip` CLI (1.26): our members pass
232
+ `lzip -t`/`-dc` and the CLI's members decode through
233
+ `Formats::Lzip` (spec skips where the CLI is absent).
234
+
109
235
  ## [0.3.28] - 2026-08-26
110
236
 
111
237
  ### Added
@@ -53,12 +53,8 @@ Omnizip.compress_directory('my_folder/', 'backup.zip')
53
53
  # Create 7z archive
54
54
  Omnizip.compress_directory('my_folder/', 'backup.7z', format: :seven_zip)
55
55
 
56
- # Add individual files
57
- Omnizip::Archive.create('custom.zip') do |archive|
58
- archive.add_file('file1.txt')
59
- archive.add_file('file2.txt')
60
- archive.add_directory('folder/')
61
- end
56
+ # Add individual files: compress one file per call
57
+ Omnizip.compress_file('file1.txt', 'custom.zip')
62
58
  ----
63
59
 
64
60
  === Extracting Archives
@@ -70,10 +66,9 @@ require 'omnizip'
70
66
  # Extract entire archive
71
67
  Omnizip.extract_archive('backup.zip', 'output/')
72
68
 
73
- # Extract with pattern matching
74
- Omnizip::Archive.open('backup.zip') do |archive|
75
- archive.extract_matching('*.txt', 'output/')
76
- end
69
+ # Extract a single entry without a full extraction
70
+ content = Omnizip.read_from_archive('backup.zip', 'notes.txt')
71
+ File.write('output/notes.txt', content)
77
72
  ----
78
73
 
79
74
  === Reading Archive Contents
@@ -82,10 +77,10 @@ end
82
77
  ----
83
78
  require 'omnizip'
84
79
 
85
- # List all entries
86
- entries = Omnizip.list_archive('backup.zip')
87
- entries.each do |entry|
88
- puts "#{entry.name}: #{entry.size} bytes"
80
+ # List all entries (names); pass details: true for sizes
81
+ Omnizip.list_archive('backup.zip').each { |name| puts name }
82
+ Omnizip.list_archive('backup.zip', details: true).each do |entry|
83
+ puts "#{entry[:name]}: #{entry[:size]} bytes"
89
84
  end
90
85
 
91
86
  # Read specific file without extraction
@@ -187,13 +182,12 @@ end
187
182
  ----
188
183
  require 'omnizip'
189
184
 
190
- # Verify archive integrity
185
+ # Verify archive integrity: reading the archive's central
186
+ # directory raises FormatError when the archive is damaged.
191
187
  begin
192
- Omnizip::Archive.open('archive.zip') do |archive|
193
- archive.verify
194
- end
188
+ Omnizip.list_archive('archive.zip')
195
189
  puts "Archive is valid"
196
- rescue Omnizip::CorruptArchiveError => e
190
+ rescue Omnizip::FormatError => e
197
191
  puts "Archive is corrupted: #{e.message}"
198
192
  end
199
193
  ----
@@ -253,9 +247,7 @@ require 'tempfile'
253
247
 
254
248
  # Create temporary archive
255
249
  Tempfile.create(['backup', '.zip']) do |tmpfile|
256
- Omnizip::Archive.create(tmpfile.path) do |archive|
257
- archive.add_directory('data/')
258
- end
250
+ Omnizip.compress_directory('data', tmpfile.path)
259
251
 
260
252
  # Use archive...
261
253
  puts "Temporary archive: #{tmpfile.path}"
@@ -12,7 +12,7 @@ Protect sensitive data in your archives with industry-standard AES-256 encryptio
12
12
  == Benefits
13
13
 
14
14
  * **Strong Security**: AES-256 encryption with secure key derivation
15
- * **Format Flexibility**: Encryption available in both ZIP and 7z formats
15
+ * **Format**: Encryption available for 7z archives (ZIP encryption is unsupported)
16
16
  * **Header Protection**: Encrypt filenames and metadata (7z format)
17
17
  * **Password Validation**: Built-in password strength checking
18
18
  * **Compatibility**: Works with archives from other tools
@@ -44,40 +44,20 @@ end
44
44
  * Optional header encryption (hides filenames)
45
45
  * HMAC-SHA-256 authentication
46
46
 
47
- === WinZip AES (ZIP Format)
47
+ === ZIP Encryption: not supported
48
48
 
49
- Industry-standard AES encryption for ZIP archives:
49
+ Omnizip can neither create nor decrypt encrypted ZIP archives
50
+ neither WinZip AES nor traditional ZipCrypto. Encrypted 7z is the
51
+ supported path; extract encrypted ZIPs with 7-Zip/unzip.
50
52
 
51
- [source,ruby]
52
- ----
53
- # Create AES-encrypted ZIP
54
- Omnizip::Zip::File.open('secure.zip', create: true) do |zip|
55
- zip.password = 'SecurePassword123'
56
- zip.encryption = :winzip_aes # AES-256
57
- zip.add('private.doc', 'files/private.doc')
58
- end
59
- ----
60
-
61
- **Features**:
62
-
63
- * AES-256, AES-192, or AES-128
64
- * Compatible with WinZip, 7-Zip, PeaZip
65
- * HMAC-SHA-1 authentication
66
- * Standard ZIP format
67
-
68
- === Traditional ZIP Encryption
69
-
70
- [WARNING]
71
- Traditional ZIP encryption (ZipCrypto) is weak and should only be used for compatibility with legacy systems.
53
+ The convenience API raises rather than silently ignoring a
54
+ password for zip:
72
55
 
73
56
  [source,ruby]
74
57
  ----
75
- # Only for compatibility - NOT RECOMMENDED
76
- Omnizip::Zip::File.open('legacy.zip', create: true) do |zip|
77
- zip.password = 'password'
78
- zip.encryption = :traditional # Weak encryption
79
- zip.add('file.txt', 'data.txt')
80
- end
58
+ Omnizip::Archive.create('secure.zip', password: 'x')
59
+ # => Omnizip::UnsupportedFormatError:
60
+ # password is only supported for seven_zip; :zip has no encryption support
81
61
  ----
82
62
 
83
63
  == Decrypting Archives
@@ -91,10 +71,8 @@ Omnizip::Archive.open('secure.7z', password: 'MyStr0ngP@ssw0rd') do |archive|
91
71
  archive.extract_all('decrypted/')
92
72
  end
93
73
 
94
- # Decrypt ZIP archive
95
- Omnizip::Zip::File.open('secure.zip', password: 'SecurePassword123') do |zip|
96
- zip.extract_all('output/')
97
- end
74
+ # ZIP encryption: Omnizip can neither create nor decrypt
75
+ # password-protected ZIP archives; extract them with 7-Zip/unzip
98
76
  ----
99
77
 
100
78
  === Password via Environment Variable
@@ -286,12 +264,11 @@ Some older archives may use unsupported encryption:
286
264
  [source,ruby]
287
265
  ----
288
266
  begin
289
- Omnizip::Archive.open('old-archive.zip') do |archive|
290
- # ...
267
+ Omnizip::Archive.open('old-archive.7z', password: 'wrong') do |archive|
268
+ archive.read('secret.txt')
291
269
  end
292
- rescue Omnizip::UnsupportedEncryptionError => e
293
- puts "Encryption method not supported: #{e.message}"
294
- puts "Try opening with 7-Zip or WinZip first"
270
+ rescue Omnizip::FormatError => e
271
+ puts "Archive damaged or password wrong: #{e.message}"
295
272
  end
296
273
  ----
297
274
 
@@ -34,12 +34,22 @@ These features can be combined for maximum efficiency:
34
34
 
35
35
  [source,ruby]
36
36
  ----
37
- # Combine streaming + parallel + progress
38
- Omnizip.compress_directory('large_data/', 'backup.7z',
39
- parallel: true,
40
- threads: 8
41
- ) do |progress|
42
- puts "#{progress.percentage}% - #{progress.throughput_mb_s} MB/s"
37
+ # Streaming writes + a progress tracker
38
+ require 'omnizip/zip/output_stream'
39
+
40
+ files = Dir.glob('large_data/**/*').select { |f| File.file?(f) }
41
+ tracker = Omnizip::Progress.track(
42
+ total_files: files.size,
43
+ total_bytes: files.sum { |f| File.size(f) },
44
+ reporter: :console
45
+ )
46
+
47
+ Omnizip::Zip::OutputStream.open('backup.zip') do |stream|
48
+ files.each do |file|
49
+ stream.put_next_entry(file.delete_prefix('large_data/'))
50
+ stream.write(File.binread(file))
51
+ tracker.update(files: 1, bytes: File.size(file))
52
+ end
43
53
  end
44
54
  ----
45
55
 
@@ -7,176 +7,94 @@ grand_parent: Guides
7
7
 
8
8
  == Purpose
9
9
 
10
- Parallel processing leverages multiple CPU cores to significantly speed up compression and extraction operations. This feature allows you to process large archives or multiple files simultaneously, reducing overall processing time.
10
+ Omnizip's codecs are pure Ruby and single-threaded: a single archive
11
+ is always compressed or extracted by one CPU core. To use multiple
12
+ cores, run independent archive operations in parallel — across
13
+ processes for CPU-bound compression, or across threads for
14
+ I/O-bound extraction of separate archives.
11
15
 
12
- == Benefits
13
-
14
- * **2-3x Faster**: Achieve 2-3x speedup on multi-core systems
15
- * **Better Resource Utilization**: Use all available CPU cores effectively
16
- * **Reduced Wait Time**: Process large batches of files quickly
17
- * **Scalable**: Performance improves with more CPU cores
18
-
19
- == When to Use Parallel Processing
16
+ == When It Helps
20
17
 
21
18
  [cols="1,3"]
22
19
  |===
23
- |Scenario |Parallel Benefit
24
-
25
- |Large Archives (>100 files)
26
- |Distribute file processing across cores
20
+ |Scenario |Approach
27
21
 
28
- |Batch Operations
29
- |Process multiple archives simultaneously
22
+ |Many archives to compress
23
+ |One process per archive
30
24
 
31
- |Multi-core Servers
32
- |Maximize server resource utilization
25
+ |Many archives to extract
26
+ |One thread per archive (I/O releases the GVL)
33
27
 
34
- |Time-Critical Tasks
35
- |Reduce processing time for urgent operations
28
+ |One large archive
29
+ |No parallelism available; pick an appropriate +level:+
36
30
  |===
37
31
 
38
- == Parallel Compression
32
+ == Parallel Compression (Processes)
39
33
 
40
- === Compress Directory in Parallel
41
-
42
- Automatically distribute files across available cores:
34
+ Compression is CPU-bound, so use processes — MRI's GVL prevents
35
+ in-thread parallelism:
43
36
 
44
37
  [source,ruby]
45
38
  ----
46
- # Compress using all available cores
47
- Omnizip.compress_directory(
48
- 'large_project/',
49
- 'backup.zip',
50
- parallel: true,
51
- threads: 4 # Or omit to use all cores
52
- )
53
- ----
54
-
55
- === Custom Thread Count
39
+ # Compress each directory in its own process
40
+ dirs = %w[project_a project_b project_c]
56
41
 
57
- Control the number of parallel workers:
58
-
59
- [source,ruby]
60
- ----
61
- # Use specific number of threads
62
- Omnizip::Archive.create('archive.7z', format: :seven_zip) do |archive|
63
- archive.add_directory(
64
- 'data/',
65
- parallel: true,
66
- threads: 8, # Use 8 worker threads
67
- chunk_size: 1.megabyte # Process 1MB chunks
68
- )
42
+ pids = dirs.map do |dir|
43
+ Process.fork do
44
+ Omnizip.compress_directory(dir, "#{dir}.7z", format: :seven_zip)
45
+ end
69
46
  end
47
+ pids.each { |pid| Process.wait(pid) }
70
48
  ----
71
49
 
72
- == Parallel Extraction
73
-
74
- === Extract Multiple Files Simultaneously
75
-
76
- Extract files in parallel for faster extraction:
77
-
78
- [source,ruby]
79
- ----
80
- # Extract with parallel processing
81
- Omnizip.extract_archive(
82
- 'large-archive.zip',
83
- 'output/',
84
- parallel: true,
85
- threads: 4
86
- )
87
- ----
88
-
89
- === With Progress Tracking
90
-
91
- Combine parallel processing with progress monitoring:
50
+ Or drive the CLI from your shell or job scheduler:
92
51
 
93
- [source,ruby]
52
+ [source,bash]
94
53
  ----
95
- Omnizip.extract_archive('backup.zip', 'restored/') do |progress|
96
- puts "Extracted #{progress.files_processed}/#{progress.total_files} files"
97
- puts "Using #{progress.active_threads} parallel threads"
98
- puts "Speed: #{progress.throughput_mb_s} MB/s"
99
- end
54
+ ls -d project_*/ | xargs -P 8 -I{} omnizip archive create {}.7z {}
100
55
  ----
101
56
 
102
- == Performance Characteristics
103
-
104
- [cols="2,1,1,1,1"]
105
- |===
106
- |Workload |1 Thread |2 Threads |4 Threads |8 Threads
107
-
108
- |100 small files
109
- |10s
110
- |6s
111
- |4s
112
- |3s
113
-
114
- |10 large files
115
- |60s
116
- |35s
117
- |20s
118
- |15s
119
-
120
- |Mixed workload
121
- |30s
122
- |18s
123
- |12s
124
- |10s
125
- |===
126
-
127
- *Note: Performance varies based on CPU, disk speed, and file characteristics.*
57
+ == Parallel Extraction (Threads)
128
58
 
129
- == Configuration Options
130
-
131
- === Thread Count
132
-
133
- Control parallelism level:
59
+ Extracting separate archives is I/O-bound, so threads work:
134
60
 
135
61
  [source,ruby]
136
62
  ----
137
- # Auto-detect (recommended)
138
- parallel: true
139
-
140
- # Specific thread count
141
- parallel: true, threads: 4
142
-
143
- # Match CPU cores
144
- parallel: true, threads: Etc.nprocessors
145
- ----
146
-
147
- === Chunk Size
63
+ archives = Dir.glob('downloads/*.zip')
148
64
 
149
- Configure work distribution:
65
+ threads = archives.map do |archive|
66
+ Thread.new do
67
+ Omnizip.extract_archive(
68
+ archive,
69
+ "extracted/#{File.basename(archive, '.zip')}/"
70
+ )
71
+ end
72
+ end
150
73
 
151
- [source,ruby]
74
+ threads.each(&:join)
152
75
  ----
153
- # Smaller chunks = better distribution, more overhead
154
- chunk_size: 512.kilobytes
155
-
156
- # Larger chunks = less overhead, less parallel benefit
157
- chunk_size: 10.megabytes
158
76
 
159
- # Default: 1MB (good balance)
160
- chunk_size: 1.megabyte
161
- ----
77
+ Note: a single archive's extraction still uses one thread.
162
78
 
163
79
  == Use Cases
164
80
 
165
81
  === Server Deployments
166
82
 
167
- Maximize server hardware utilization:
168
-
169
83
  [source,ruby]
170
84
  ----
171
- # Backup server with 16 cores
172
- Omnizip.compress_directory(
173
- '/var/www/',
174
- '/backups/www-backup.7z',
175
- format: :seven_zip,
176
- compression: :lzma2,
177
- parallel: true,
178
- threads: 16
179
- )
85
+ # One process per site on a multi-core backup server
86
+ sites = Dir.children('/var/www')
87
+
88
+ pids = sites.map do |site|
89
+ Process.fork do
90
+ Omnizip.compress_directory(
91
+ File.join('/var/www', site),
92
+ "/backups/#{site}.7z",
93
+ format: :seven_zip, algorithm: :lzma2, level: 9
94
+ )
95
+ end
96
+ end
97
+ pids.each { |pid| Process.wait(pid) }
180
98
  ----
181
99
 
182
100
  === Batch Processing
@@ -204,43 +122,23 @@ end
204
122
 
205
123
  == Best Practices
206
124
 
207
- . **Match Thread Count to Cores**: Use `Etc.nprocessors` for optimal performance
208
- . **Consider Disk I/O**: More threads may not help with slow disks
209
- . **Monitor Memory**: Each thread uses additional memory
210
- . **Test Your Workload**: Measure actual speedup for your use case
211
- . **Combine with Streaming**: Use both for maximum efficiency
125
+ . **One process per archive** for compression; threads only help I/O
126
+ . **Consider disk I/O**: more parallelism may not help on slow disks
127
+ . **Monitor memory**: each process/thread adds overhead
128
+ . **Test your workload**: measure the real speedup
212
129
 
213
130
  == Limitations
214
131
 
215
- * Ruby GIL limits true parallelism (use JRuby for better results)
216
- * Memory usage increases with thread count
217
- * I/O-bound operations see less benefit
218
- * Very small files have overhead from thread management
132
+ * No intra-archive parallelism: a single archive uses one core
133
+ * MRI's GVL blocks in-thread CPU parallelism — use processes
134
+ * Memory usage grows with process/thread count
219
135
 
220
136
  == Platform Considerations
221
137
 
222
- [cols="1,3"]
223
- |===
224
- |Platform |Notes
225
-
226
- |MRI Ruby
227
- |Limited by GIL, 2-3x speedup typical
228
-
229
- |JRuby
230
- |No GIL, better parallel scalability
231
-
232
- |TruffleRuby
233
- |Excellent parallel performance
234
-
235
- |Windows
236
- |Similar performance to Unix systems
237
-
238
- |Linux/macOS
239
- |Full parallel processing support
240
- |===
138
+ `Process.fork` is unavailable on Windows — use `Process.spawn` with
139
+ the +omnizip+ CLI there, or a job scheduler.
241
140
 
242
141
  == See Also
243
142
 
244
- * link:streaming.html[Streaming] - Combine for memory efficiency
245
- * link:progress-tracking.html[Progress Tracking] - Monitor parallel operations
246
- * link:../../compatibility.html[Compatibility] - Format support for parallel processing
143
+ * link:streaming.html[Streaming] - Memory efficiency
144
+ * link:../compression-algorithms/lzma2.html[LZMA2] - Default 7z algorithm