omnizip 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.rspec +3 -0
- data/.rubocop.yml +32 -0
- data/.rubocop_todo.yml +754 -0
- data/COPYING +502 -0
- data/Gemfile +17 -0
- data/LICENSE +12 -0
- data/README.adoc +1045 -0
- data/Rakefile +12 -0
- data/benchmark/README.md +260 -0
- data/benchmark/benchmark_suite.rb +125 -0
- data/benchmark/compression_bench.rb +181 -0
- data/benchmark/filter_bench.rb +180 -0
- data/benchmark/models/benchmark_result.rb +59 -0
- data/benchmark/models/comparison_result.rb +69 -0
- data/benchmark/profile_suite.rb +167 -0
- data/benchmark/reporter.rb +150 -0
- data/benchmark/run_benchmarks.rb +66 -0
- data/benchmark/test_data.rb +137 -0
- data/config/formats/rar3_spec.yml +91 -0
- data/config/formats/rar5_spec.yml +102 -0
- data/docs/.github/workflows/docs.yml +142 -0
- data/docs/.gitignore +21 -0
- data/docs/.lychee.toml +67 -0
- data/docs/Gemfile +13 -0
- data/docs/RAR_WRITE_SUPPORT.md +26 -0
- data/docs/README.md +101 -0
- data/docs/_config.yml +112 -0
- data/docs/assets/logo.svg +1 -0
- data/docs/assets/omnizip-logo.pdf +1540 -11
- data/docs/comparison/feature-matrix.adoc +694 -0
- data/docs/comparison/index.adoc +113 -0
- data/docs/comparison/vs-7zip.adoc +309 -0
- data/docs/comparison/vs-peazip.adoc +77 -0
- data/docs/comparison/vs-rubyzip.adoc +342 -0
- data/docs/comparison/vs-winrar.adoc +100 -0
- data/docs/compatibility.adoc +579 -0
- data/docs/concepts/index.adoc +129 -0
- data/docs/developer/architecture.adoc +256 -0
- data/docs/developer/contributing.adoc +158 -0
- data/docs/developer/index.adoc +25 -0
- data/docs/developer/testing.adoc +212 -0
- data/docs/getting-started/basic-usage.adoc +271 -0
- data/docs/getting-started/index.adoc +42 -0
- data/docs/getting-started/installation.adoc +138 -0
- data/docs/getting-started/quick-start.adoc +185 -0
- data/docs/getting-started/your-first-archive.adoc +218 -0
- data/docs/guides/advanced-features/encryption.adoc +300 -0
- data/docs/guides/advanced-features/index.adoc +49 -0
- data/docs/guides/advanced-features/parallel-processing.adoc +246 -0
- data/docs/guides/advanced-features/progress-tracking.adoc +320 -0
- data/docs/guides/advanced-features/streaming.adoc +212 -0
- data/docs/guides/archive-formats/gzip-format.adoc +107 -0
- data/docs/guides/archive-formats/index.adoc +130 -0
- data/docs/guides/archive-formats/rar-format.adoc +104 -0
- data/docs/guides/archive-formats/rar5.adoc +521 -0
- data/docs/guides/archive-formats/seven-zip-format.adoc +35 -0
- data/docs/guides/archive-formats/tar-format.adoc +106 -0
- data/docs/guides/archive-formats/xz-format.adoc +118 -0
- data/docs/guides/archive-formats/zip-format.adoc +35 -0
- data/docs/guides/compression-algorithms/bzip2.adoc +113 -0
- data/docs/guides/compression-algorithms/deflate.adoc +319 -0
- data/docs/guides/compression-algorithms/index.adoc +190 -0
- data/docs/guides/compression-algorithms/lzma.adoc +398 -0
- data/docs/guides/compression-algorithms/lzma2.adoc +327 -0
- data/docs/guides/compression-algorithms/ppmd.adoc +316 -0
- data/docs/guides/compression-algorithms/zstandard.adoc +361 -0
- data/docs/guides/creating-archives.adoc +354 -0
- data/docs/guides/extracting-archives.adoc +53 -0
- data/docs/guides/format-conversion.adoc +64 -0
- data/docs/guides/index.adoc +49 -0
- data/docs/guides/migration-rubyzip.adoc +217 -0
- data/docs/guides/parity-archives.adoc +605 -0
- data/docs/guides/performance-tuning.adoc +88 -0
- data/docs/index.adoc +218 -0
- data/docs/lychee.toml +67 -0
- data/docs/reference/api/overview.adoc +188 -0
- data/docs/reference/cli/compress-command.adoc +114 -0
- data/docs/reference/cli/overview.adoc +140 -0
- data/docs/reference/index.adoc +26 -0
- data/docs/resources/faq.adoc +185 -0
- data/docs/resources/quick-reference.adoc +222 -0
- data/docs/troubleshooting/index.adoc +208 -0
- data/examples/api_comparison.rb +205 -0
- data/examples/deflate64_example.rb +96 -0
- data/examples/par2_demo.rb +121 -0
- data/examples/quick_start_native.rb +150 -0
- data/examples/quick_start_rubyzip.rb +115 -0
- data/examples/rubyzip_compatibility_demo.rb +194 -0
- data/exe/omnizip +27 -0
- data/lib/omnizip/algorithm.rb +130 -0
- data/lib/omnizip/algorithm_registry.rb +86 -0
- data/lib/omnizip/algorithms/.keep +0 -0
- data/lib/omnizip/algorithms/bzip2/bwt.rb +225 -0
- data/lib/omnizip/algorithms/bzip2/decoder.rb +193 -0
- data/lib/omnizip/algorithms/bzip2/encoder.rb +237 -0
- data/lib/omnizip/algorithms/bzip2/huffman.rb +206 -0
- data/lib/omnizip/algorithms/bzip2/mtf.rb +101 -0
- data/lib/omnizip/algorithms/bzip2/rle.rb +151 -0
- data/lib/omnizip/algorithms/bzip2.rb +130 -0
- data/lib/omnizip/algorithms/deflate/constants.rb +28 -0
- data/lib/omnizip/algorithms/deflate/decoder.rb +38 -0
- data/lib/omnizip/algorithms/deflate/encoder.rb +46 -0
- data/lib/omnizip/algorithms/deflate.rb +128 -0
- data/lib/omnizip/algorithms/deflate64/constants.rb +45 -0
- data/lib/omnizip/algorithms/deflate64/decoder.rb +153 -0
- data/lib/omnizip/algorithms/deflate64/encoder.rb +98 -0
- data/lib/omnizip/algorithms/deflate64/huffman_coder.rb +354 -0
- data/lib/omnizip/algorithms/deflate64/lz77_encoder.rb +142 -0
- data/lib/omnizip/algorithms/deflate64.rb +109 -0
- data/lib/omnizip/algorithms/lzma/bit_model.rb +120 -0
- data/lib/omnizip/algorithms/lzma/constants.rb +112 -0
- data/lib/omnizip/algorithms/lzma/decoder.rb +148 -0
- data/lib/omnizip/algorithms/lzma/dictionary.rb +69 -0
- data/lib/omnizip/algorithms/lzma/distance_coder.rb +415 -0
- data/lib/omnizip/algorithms/lzma/encoder.rb +142 -0
- data/lib/omnizip/algorithms/lzma/length_coder.rb +260 -0
- data/lib/omnizip/algorithms/lzma/literal_decoder.rb +320 -0
- data/lib/omnizip/algorithms/lzma/literal_encoder.rb +210 -0
- data/lib/omnizip/algorithms/lzma/lzip_decoder.rb +341 -0
- data/lib/omnizip/algorithms/lzma/lzma_alone_decoder.rb +192 -0
- data/lib/omnizip/algorithms/lzma/lzma_state.rb +128 -0
- data/lib/omnizip/algorithms/lzma/match.rb +32 -0
- data/lib/omnizip/algorithms/lzma/match_finder.rb +205 -0
- data/lib/omnizip/algorithms/lzma/match_finder_config.rb +142 -0
- data/lib/omnizip/algorithms/lzma/match_finder_factory.rb +88 -0
- data/lib/omnizip/algorithms/lzma/optimal_encoder.rb +130 -0
- data/lib/omnizip/algorithms/lzma/probability_models.rb +72 -0
- data/lib/omnizip/algorithms/lzma/range_coder.rb +85 -0
- data/lib/omnizip/algorithms/lzma/range_decoder.rb +434 -0
- data/lib/omnizip/algorithms/lzma/range_encoder.rb +194 -0
- data/lib/omnizip/algorithms/lzma/state.rb +127 -0
- data/lib/omnizip/algorithms/lzma/xz_buffered_range_encoder.rb +325 -0
- data/lib/omnizip/algorithms/lzma/xz_encoder.rb +426 -0
- data/lib/omnizip/algorithms/lzma/xz_encoder_fast.rb +645 -0
- data/lib/omnizip/algorithms/lzma/xz_match_finder_adapter.rb +227 -0
- data/lib/omnizip/algorithms/lzma/xz_price_calculator.rb +169 -0
- data/lib/omnizip/algorithms/lzma/xz_probability_models.rb +261 -0
- data/lib/omnizip/algorithms/lzma/xz_range_encoder.rb +223 -0
- data/lib/omnizip/algorithms/lzma/xz_range_encoder_exact.rb +331 -0
- data/lib/omnizip/algorithms/lzma/xz_state.rb +116 -0
- data/lib/omnizip/algorithms/lzma/xz_utils_decoder.rb +2055 -0
- data/lib/omnizip/algorithms/lzma.rb +238 -0
- data/lib/omnizip/algorithms/lzma2/chunk_manager.rb +182 -0
- data/lib/omnizip/algorithms/lzma2/constants.rb +41 -0
- data/lib/omnizip/algorithms/lzma2/encoder.rb +147 -0
- data/lib/omnizip/algorithms/lzma2/lzma2_chunk.rb +161 -0
- data/lib/omnizip/algorithms/lzma2/properties.rb +179 -0
- data/lib/omnizip/algorithms/lzma2/simple_lzma2_encoder.rb +127 -0
- data/lib/omnizip/algorithms/lzma2/xz_encoder_adapter.rb +85 -0
- data/lib/omnizip/algorithms/lzma2.rb +141 -0
- data/lib/omnizip/algorithms/ppmd7/constants.rb +74 -0
- data/lib/omnizip/algorithms/ppmd7/context.rb +154 -0
- data/lib/omnizip/algorithms/ppmd7/decoder.rb +126 -0
- data/lib/omnizip/algorithms/ppmd7/encoder.rb +163 -0
- data/lib/omnizip/algorithms/ppmd7/model.rb +248 -0
- data/lib/omnizip/algorithms/ppmd7/symbol_state.rb +57 -0
- data/lib/omnizip/algorithms/ppmd7.rb +116 -0
- data/lib/omnizip/algorithms/ppmd8/constants.rb +61 -0
- data/lib/omnizip/algorithms/ppmd8/context.rb +34 -0
- data/lib/omnizip/algorithms/ppmd8/decoder.rb +107 -0
- data/lib/omnizip/algorithms/ppmd8/encoder.rb +138 -0
- data/lib/omnizip/algorithms/ppmd8/model.rb +250 -0
- data/lib/omnizip/algorithms/ppmd8/restoration_method.rb +78 -0
- data/lib/omnizip/algorithms/ppmd8.rb +82 -0
- data/lib/omnizip/algorithms/ppmd_base.rb +138 -0
- data/lib/omnizip/algorithms/sevenzip_lzma2.rb +123 -0
- data/lib/omnizip/algorithms/xz_lzma2.rb +118 -0
- data/lib/omnizip/algorithms/zstandard/constants.rb +25 -0
- data/lib/omnizip/algorithms/zstandard/decoder.rb +46 -0
- data/lib/omnizip/algorithms/zstandard/encoder.rb +51 -0
- data/lib/omnizip/algorithms/zstandard.rb +138 -0
- data/lib/omnizip/buffer/memory_archive.rb +251 -0
- data/lib/omnizip/buffer/memory_extractor.rb +224 -0
- data/lib/omnizip/buffer.rb +176 -0
- data/lib/omnizip/checksum_registry.rb +114 -0
- data/lib/omnizip/checksums/crc32.rb +100 -0
- data/lib/omnizip/checksums/crc64.rb +101 -0
- data/lib/omnizip/checksums/crc_base.rb +158 -0
- data/lib/omnizip/checksums/verifier.rb +131 -0
- data/lib/omnizip/chunked/memory_manager.rb +194 -0
- data/lib/omnizip/chunked/reader.rb +78 -0
- data/lib/omnizip/chunked/writer.rb +120 -0
- data/lib/omnizip/chunked.rb +129 -0
- data/lib/omnizip/cli/output_formatter.rb +104 -0
- data/lib/omnizip/cli.rb +572 -0
- data/lib/omnizip/commands/.keep +0 -0
- data/lib/omnizip/commands/archive_create_command.rb +427 -0
- data/lib/omnizip/commands/archive_extract_command.rb +272 -0
- data/lib/omnizip/commands/archive_list_command.rb +218 -0
- data/lib/omnizip/commands/archive_repair_command.rb +131 -0
- data/lib/omnizip/commands/archive_verify_command.rb +117 -0
- data/lib/omnizip/commands/compress_command.rb +117 -0
- data/lib/omnizip/commands/decompress_command.rb +120 -0
- data/lib/omnizip/commands/list_command.rb +53 -0
- data/lib/omnizip/commands/metadata_command.rb +153 -0
- data/lib/omnizip/commands/parity_create_command.rb +122 -0
- data/lib/omnizip/commands/parity_repair_command.rb +122 -0
- data/lib/omnizip/commands/parity_verify_command.rb +124 -0
- data/lib/omnizip/commands/profile_list_command.rb +56 -0
- data/lib/omnizip/commands/profile_show_command.rb +44 -0
- data/lib/omnizip/convenience.rb +359 -0
- data/lib/omnizip/converter/conversion_registry.rb +49 -0
- data/lib/omnizip/converter/conversion_strategy.rb +121 -0
- data/lib/omnizip/converter/seven_zip_to_zip_strategy.rb +97 -0
- data/lib/omnizip/converter/zip_to_seven_zip_strategy.rb +112 -0
- data/lib/omnizip/converter.rb +105 -0
- data/lib/omnizip/crypto/aes256/cipher.rb +100 -0
- data/lib/omnizip/crypto/aes256/constants.rb +28 -0
- data/lib/omnizip/crypto/aes256/key_derivation.rb +101 -0
- data/lib/omnizip/crypto/aes256.rb +102 -0
- data/lib/omnizip/error.rb +106 -0
- data/lib/omnizip/eta/exponential_smoothing_estimator.rb +98 -0
- data/lib/omnizip/eta/moving_average_estimator.rb +99 -0
- data/lib/omnizip/eta/rate_calculator.rb +104 -0
- data/lib/omnizip/eta/sample_history.rb +143 -0
- data/lib/omnizip/eta/time_estimator.rb +106 -0
- data/lib/omnizip/eta.rb +63 -0
- data/lib/omnizip/extraction/filter_chain.rb +177 -0
- data/lib/omnizip/extraction/glob_pattern.rb +140 -0
- data/lib/omnizip/extraction/pattern_matcher.rb +70 -0
- data/lib/omnizip/extraction/predicate_pattern.rb +52 -0
- data/lib/omnizip/extraction/regex_pattern.rb +50 -0
- data/lib/omnizip/extraction/selective_extractor.rb +240 -0
- data/lib/omnizip/extraction.rb +111 -0
- data/lib/omnizip/file_type/mime_classifier.rb +144 -0
- data/lib/omnizip/file_type.rb +113 -0
- data/lib/omnizip/filter.rb +139 -0
- data/lib/omnizip/filter_pipeline.rb +108 -0
- data/lib/omnizip/filter_registry.rb +166 -0
- data/lib/omnizip/filters/bcj.rb +279 -0
- data/lib/omnizip/filters/bcj2/constants.rb +53 -0
- data/lib/omnizip/filters/bcj2/decoder.rb +200 -0
- data/lib/omnizip/filters/bcj2/encoder.rb +61 -0
- data/lib/omnizip/filters/bcj2/stream_data.rb +93 -0
- data/lib/omnizip/filters/bcj2.rb +99 -0
- data/lib/omnizip/filters/bcj_arm.rb +176 -0
- data/lib/omnizip/filters/bcj_arm64.rb +244 -0
- data/lib/omnizip/filters/bcj_ia64.rb +196 -0
- data/lib/omnizip/filters/bcj_ppc.rb +190 -0
- data/lib/omnizip/filters/bcj_sparc.rb +176 -0
- data/lib/omnizip/filters/bcj_x86.rb +193 -0
- data/lib/omnizip/filters/delta.rb +196 -0
- data/lib/omnizip/filters/filter_base.rb +72 -0
- data/lib/omnizip/filters/registry.rb +123 -0
- data/lib/omnizip/filters/xz_delta.rb +258 -0
- data/lib/omnizip/format_detector.rb +162 -0
- data/lib/omnizip/format_registry.rb +59 -0
- data/lib/omnizip/formats/.keep +0 -0
- data/lib/omnizip/formats/bzip2_file.rb +172 -0
- data/lib/omnizip/formats/cpio/constants.rb +55 -0
- data/lib/omnizip/formats/cpio/entry.rb +385 -0
- data/lib/omnizip/formats/cpio/reader.rb +196 -0
- data/lib/omnizip/formats/cpio/writer.rb +234 -0
- data/lib/omnizip/formats/cpio.rb +140 -0
- data/lib/omnizip/formats/format_spec_loader.rb +230 -0
- data/lib/omnizip/formats/gzip.rb +238 -0
- data/lib/omnizip/formats/iso/directory_builder.rb +297 -0
- data/lib/omnizip/formats/iso/directory_record.rb +152 -0
- data/lib/omnizip/formats/iso/joliet.rb +204 -0
- data/lib/omnizip/formats/iso/path_table.rb +125 -0
- data/lib/omnizip/formats/iso/reader.rb +197 -0
- data/lib/omnizip/formats/iso/rock_ridge.rb +349 -0
- data/lib/omnizip/formats/iso/volume_builder.rb +320 -0
- data/lib/omnizip/formats/iso/volume_descriptor.rb +168 -0
- data/lib/omnizip/formats/iso/writer.rb +530 -0
- data/lib/omnizip/formats/iso.rb +140 -0
- data/lib/omnizip/formats/lzip.rb +175 -0
- data/lib/omnizip/formats/lzma_alone.rb +171 -0
- data/lib/omnizip/formats/rar/archive_repairer.rb +243 -0
- data/lib/omnizip/formats/rar/archive_verifier.rb +195 -0
- data/lib/omnizip/formats/rar/block_parser.rb +243 -0
- data/lib/omnizip/formats/rar/compression/bit_stream.rb +180 -0
- data/lib/omnizip/formats/rar/compression/dispatcher.rb +217 -0
- data/lib/omnizip/formats/rar/compression/lz77_huffman/decoder.rb +216 -0
- data/lib/omnizip/formats/rar/compression/lz77_huffman/encoder.rb +158 -0
- data/lib/omnizip/formats/rar/compression/lz77_huffman/huffman_builder.rb +217 -0
- data/lib/omnizip/formats/rar/compression/lz77_huffman/huffman_coder.rb +189 -0
- data/lib/omnizip/formats/rar/compression/lz77_huffman/match_finder.rb +135 -0
- data/lib/omnizip/formats/rar/compression/lz77_huffman/sliding_window.rb +165 -0
- data/lib/omnizip/formats/rar/compression/ppmd/context.rb +105 -0
- data/lib/omnizip/formats/rar/compression/ppmd/decoder.rb +219 -0
- data/lib/omnizip/formats/rar/compression/ppmd/encoder.rb +262 -0
- data/lib/omnizip/formats/rar/compression_method_registry.rb +106 -0
- data/lib/omnizip/formats/rar/constants.rb +82 -0
- data/lib/omnizip/formats/rar/decompressor.rb +238 -0
- data/lib/omnizip/formats/rar/external_writer.rb +312 -0
- data/lib/omnizip/formats/rar/header.rb +192 -0
- data/lib/omnizip/formats/rar/license_validator.rb +109 -0
- data/lib/omnizip/formats/rar/models/rar_archive.rb +77 -0
- data/lib/omnizip/formats/rar/models/rar_entry.rb +65 -0
- data/lib/omnizip/formats/rar/models/rar_volume.rb +56 -0
- data/lib/omnizip/formats/rar/parity_handler.rb +292 -0
- data/lib/omnizip/formats/rar/rar5/compression/lzma.rb +202 -0
- data/lib/omnizip/formats/rar/rar5/compression/lzss.rb +578 -0
- data/lib/omnizip/formats/rar/rar5/compression/store.rb +60 -0
- data/lib/omnizip/formats/rar/rar5/crc32.rb +39 -0
- data/lib/omnizip/formats/rar/rar5/encryption/aes256_cbc.rb +97 -0
- data/lib/omnizip/formats/rar/rar5/encryption/encryption_header.rb +114 -0
- data/lib/omnizip/formats/rar/rar5/encryption/encryption_manager.rb +166 -0
- data/lib/omnizip/formats/rar/rar5/encryption/key_derivation.rb +97 -0
- data/lib/omnizip/formats/rar/rar5/header.rb +187 -0
- data/lib/omnizip/formats/rar/rar5/models/encryption_options.rb +74 -0
- data/lib/omnizip/formats/rar/rar5/models/recovery_options.rb +63 -0
- data/lib/omnizip/formats/rar/rar5/models/solid_options.rb +63 -0
- data/lib/omnizip/formats/rar/rar5/models/volume_options.rb +74 -0
- data/lib/omnizip/formats/rar/rar5/multi_volume/ARCHITECTURE.md +290 -0
- data/lib/omnizip/formats/rar/rar5/multi_volume/volume_manager.rb +264 -0
- data/lib/omnizip/formats/rar/rar5/multi_volume/volume_splitter.rb +155 -0
- data/lib/omnizip/formats/rar/rar5/multi_volume/volume_writer.rb +194 -0
- data/lib/omnizip/formats/rar/rar5/solid/solid_encoder.rb +109 -0
- data/lib/omnizip/formats/rar/rar5/solid/solid_manager.rb +142 -0
- data/lib/omnizip/formats/rar/rar5/solid/solid_stream.rb +121 -0
- data/lib/omnizip/formats/rar/rar5/vint.rb +65 -0
- data/lib/omnizip/formats/rar/rar5/writer.rb +466 -0
- data/lib/omnizip/formats/rar/rar_format_base.rb +241 -0
- data/lib/omnizip/formats/rar/reader.rb +366 -0
- data/lib/omnizip/formats/rar/recovery_record.rb +245 -0
- data/lib/omnizip/formats/rar/volume_manager.rb +168 -0
- data/lib/omnizip/formats/rar/writer.rb +431 -0
- data/lib/omnizip/formats/rar.rb +205 -0
- data/lib/omnizip/formats/rar3/compressor.rb +73 -0
- data/lib/omnizip/formats/rar3/decompressor.rb +66 -0
- data/lib/omnizip/formats/rar3/reader.rb +386 -0
- data/lib/omnizip/formats/rar3/writer.rb +219 -0
- data/lib/omnizip/formats/rar5/compressor.rb +73 -0
- data/lib/omnizip/formats/rar5/decompressor.rb +66 -0
- data/lib/omnizip/formats/rar5/reader.rb +342 -0
- data/lib/omnizip/formats/rar5/writer.rb +214 -0
- data/lib/omnizip/formats/seven_zip/coder_chain.rb +150 -0
- data/lib/omnizip/formats/seven_zip/constants.rb +126 -0
- data/lib/omnizip/formats/seven_zip/encoded_header.rb +114 -0
- data/lib/omnizip/formats/seven_zip/encrypted_header.rb +142 -0
- data/lib/omnizip/formats/seven_zip/file_collector.rb +144 -0
- data/lib/omnizip/formats/seven_zip/header.rb +106 -0
- data/lib/omnizip/formats/seven_zip/header_encryptor.rb +134 -0
- data/lib/omnizip/formats/seven_zip/header_writer.rb +466 -0
- data/lib/omnizip/formats/seven_zip/models/coder_info.rb +30 -0
- data/lib/omnizip/formats/seven_zip/models/file_entry.rb +58 -0
- data/lib/omnizip/formats/seven_zip/models/folder.rb +69 -0
- data/lib/omnizip/formats/seven_zip/models/stream_info.rb +42 -0
- data/lib/omnizip/formats/seven_zip/parser.rb +660 -0
- data/lib/omnizip/formats/seven_zip/reader.rb +458 -0
- data/lib/omnizip/formats/seven_zip/split_archive_reader.rb +632 -0
- data/lib/omnizip/formats/seven_zip/split_archive_writer.rb +315 -0
- data/lib/omnizip/formats/seven_zip/stream_compressor.rb +151 -0
- data/lib/omnizip/formats/seven_zip/stream_decompressor.rb +162 -0
- data/lib/omnizip/formats/seven_zip/writer.rb +740 -0
- data/lib/omnizip/formats/seven_zip.rb +93 -0
- data/lib/omnizip/formats/tar/constants.rb +73 -0
- data/lib/omnizip/formats/tar/entry.rb +94 -0
- data/lib/omnizip/formats/tar/header.rb +168 -0
- data/lib/omnizip/formats/tar/reader.rb +121 -0
- data/lib/omnizip/formats/tar/writer.rb +216 -0
- data/lib/omnizip/formats/tar.rb +84 -0
- data/lib/omnizip/formats/xz/reader.rb +116 -0
- data/lib/omnizip/formats/xz.rb +237 -0
- data/lib/omnizip/formats/xz_impl/block_decoder.rb +754 -0
- data/lib/omnizip/formats/xz_impl/block_encoder.rb +306 -0
- data/lib/omnizip/formats/xz_impl/block_header.rb +210 -0
- data/lib/omnizip/formats/xz_impl/block_header_parser.rb +186 -0
- data/lib/omnizip/formats/xz_impl/constants.rb +49 -0
- data/lib/omnizip/formats/xz_impl/index_decoder.rb +174 -0
- data/lib/omnizip/formats/xz_impl/index_encoder.rb +122 -0
- data/lib/omnizip/formats/xz_impl/stream_decoder.rb +468 -0
- data/lib/omnizip/formats/xz_impl/stream_encoder.rb +99 -0
- data/lib/omnizip/formats/xz_impl/stream_footer.rb +81 -0
- data/lib/omnizip/formats/xz_impl/stream_footer_parser.rb +117 -0
- data/lib/omnizip/formats/xz_impl/stream_header.rb +55 -0
- data/lib/omnizip/formats/xz_impl/stream_header_parser.rb +108 -0
- data/lib/omnizip/formats/xz_impl/vli.rb +128 -0
- data/lib/omnizip/formats/xz_impl/writer.rb +421 -0
- data/lib/omnizip/formats/zip/central_directory_header.rb +195 -0
- data/lib/omnizip/formats/zip/constants.rb +69 -0
- data/lib/omnizip/formats/zip/end_of_central_directory.rb +133 -0
- data/lib/omnizip/formats/zip/local_file_header.rb +138 -0
- data/lib/omnizip/formats/zip/reader.rb +250 -0
- data/lib/omnizip/formats/zip/unix_extra_field.rb +153 -0
- data/lib/omnizip/formats/zip/writer.rb +375 -0
- data/lib/omnizip/formats/zip/zip64_end_of_central_directory.rb +104 -0
- data/lib/omnizip/formats/zip/zip64_end_of_central_directory_locator.rb +66 -0
- data/lib/omnizip/formats/zip/zip64_extra_field.rb +114 -0
- data/lib/omnizip/formats/zip.rb +50 -0
- data/lib/omnizip/implementations/base/lzma2_decoder_base.rb +75 -0
- data/lib/omnizip/implementations/base/lzma2_encoder_base.rb +128 -0
- data/lib/omnizip/implementations/base/lzma_decoder_base.rb +83 -0
- data/lib/omnizip/implementations/base/lzma_encoder_base.rb +108 -0
- data/lib/omnizip/implementations/base/state_machine_base.rb +182 -0
- data/lib/omnizip/implementations/seven_zip/lzma/decoder.rb +421 -0
- data/lib/omnizip/implementations/seven_zip/lzma/encoder.rb +465 -0
- data/lib/omnizip/implementations/seven_zip/lzma/match_finder.rb +288 -0
- data/lib/omnizip/implementations/seven_zip/lzma/range_decoder.rb +200 -0
- data/lib/omnizip/implementations/seven_zip/lzma/range_encoder.rb +197 -0
- data/lib/omnizip/implementations/seven_zip/lzma/state_machine.rb +141 -0
- data/lib/omnizip/implementations/seven_zip/lzma2/encoder.rb +519 -0
- data/lib/omnizip/implementations/xz_utils/lzma2/decoder.rb +723 -0
- data/lib/omnizip/implementations/xz_utils/lzma2/encoder.rb +750 -0
- data/lib/omnizip/io/buffered_input.rb +146 -0
- data/lib/omnizip/io/buffered_output.rb +105 -0
- data/lib/omnizip/io/stream_manager.rb +115 -0
- data/lib/omnizip/link_handler/hard_link.rb +79 -0
- data/lib/omnizip/link_handler/symbolic_link.rb +74 -0
- data/lib/omnizip/link_handler.rb +124 -0
- data/lib/omnizip/metadata/archive_metadata.rb +114 -0
- data/lib/omnizip/metadata/entry_metadata.rb +146 -0
- data/lib/omnizip/metadata/metadata_editor.rb +171 -0
- data/lib/omnizip/metadata/metadata_registry.rb +64 -0
- data/lib/omnizip/metadata/metadata_validator.rb +99 -0
- data/lib/omnizip/metadata.rb +57 -0
- data/lib/omnizip/models/.keep +0 -0
- data/lib/omnizip/models/algorithm_metadata.rb +73 -0
- data/lib/omnizip/models/compression_options.rb +71 -0
- data/lib/omnizip/models/conversion_options.rb +87 -0
- data/lib/omnizip/models/conversion_result.rb +135 -0
- data/lib/omnizip/models/eta_result.rb +46 -0
- data/lib/omnizip/models/extraction_rule.rb +115 -0
- data/lib/omnizip/models/filter_chain.rb +144 -0
- data/lib/omnizip/models/filter_config.rb +183 -0
- data/lib/omnizip/models/match_result.rb +124 -0
- data/lib/omnizip/models/optimization_suggestion.rb +91 -0
- data/lib/omnizip/models/parallel_options.rb +104 -0
- data/lib/omnizip/models/performance_result.rb +79 -0
- data/lib/omnizip/models/profile_report.rb +82 -0
- data/lib/omnizip/models/progress_options.rb +38 -0
- data/lib/omnizip/models/split_options.rb +116 -0
- data/lib/omnizip/optimization_registry.rb +81 -0
- data/lib/omnizip/parallel/job_queue.rb +209 -0
- data/lib/omnizip/parallel/job_scheduler.rb +203 -0
- data/lib/omnizip/parallel/parallel_compressor.rb +347 -0
- data/lib/omnizip/parallel/parallel_extractor.rb +329 -0
- data/lib/omnizip/parallel/worker_pool.rb +223 -0
- data/lib/omnizip/parallel.rb +149 -0
- data/lib/omnizip/parity/chunked_block_processor.rb +196 -0
- data/lib/omnizip/parity/galois16.rb +145 -0
- data/lib/omnizip/parity/models/creator_packet.rb +73 -0
- data/lib/omnizip/parity/models/file_description_packet.rb +133 -0
- data/lib/omnizip/parity/models/ifsc_packet.rb +123 -0
- data/lib/omnizip/parity/models/main_packet.rb +128 -0
- data/lib/omnizip/parity/models/packet.rb +156 -0
- data/lib/omnizip/parity/models/packet_registry.rb +109 -0
- data/lib/omnizip/parity/models/recovery_slice_packet.rb +78 -0
- data/lib/omnizip/parity/par2_creator.rb +531 -0
- data/lib/omnizip/parity/par2_repairer.rb +407 -0
- data/lib/omnizip/parity/par2_verifier.rb +364 -0
- data/lib/omnizip/parity/par2cmdline_algorithm.rb +110 -0
- data/lib/omnizip/parity/par2cmdline_coefficients.rb +78 -0
- data/lib/omnizip/parity/reed_solomon_decoder.rb +266 -0
- data/lib/omnizip/parity/reed_solomon_encoder.rb +111 -0
- data/lib/omnizip/parity/reed_solomon_matrix.rb +342 -0
- data/lib/omnizip/parity.rb +186 -0
- data/lib/omnizip/password/encryption_registry.rb +65 -0
- data/lib/omnizip/password/encryption_strategy.rb +96 -0
- data/lib/omnizip/password/password_validator.rb +129 -0
- data/lib/omnizip/password/winzip_aes_strategy.rb +192 -0
- data/lib/omnizip/password/zip_crypto_strategy.rb +141 -0
- data/lib/omnizip/password.rb +87 -0
- data/lib/omnizip/pipe/stream_compressor.rb +124 -0
- data/lib/omnizip/pipe/stream_decompressor.rb +174 -0
- data/lib/omnizip/pipe.rb +121 -0
- data/lib/omnizip/platform/ntfs_streams.rb +201 -0
- data/lib/omnizip/platform.rb +189 -0
- data/lib/omnizip/profile/archive_profile.rb +39 -0
- data/lib/omnizip/profile/balanced_profile.rb +33 -0
- data/lib/omnizip/profile/binary_profile.rb +36 -0
- data/lib/omnizip/profile/compression_profile.rb +158 -0
- data/lib/omnizip/profile/custom_profile.rb +157 -0
- data/lib/omnizip/profile/fast_profile.rb +33 -0
- data/lib/omnizip/profile/maximum_profile.rb +33 -0
- data/lib/omnizip/profile/profile_detector.rb +110 -0
- data/lib/omnizip/profile/profile_registry.rb +161 -0
- data/lib/omnizip/profile/text_profile.rb +36 -0
- data/lib/omnizip/profile.rb +190 -0
- data/lib/omnizip/profiler/memory_profiler.rb +66 -0
- data/lib/omnizip/profiler/method_profiler.rb +49 -0
- data/lib/omnizip/profiler/report_generator.rb +169 -0
- data/lib/omnizip/profiler.rb +204 -0
- data/lib/omnizip/progress/callback_reporter.rb +36 -0
- data/lib/omnizip/progress/console_reporter.rb +62 -0
- data/lib/omnizip/progress/log_reporter.rb +91 -0
- data/lib/omnizip/progress/operation_progress.rb +118 -0
- data/lib/omnizip/progress/progress_bar.rb +156 -0
- data/lib/omnizip/progress/progress_reporter.rb +40 -0
- data/lib/omnizip/progress/progress_tracker.rb +190 -0
- data/lib/omnizip/progress/silent_reporter.rb +24 -0
- data/lib/omnizip/progress.rb +127 -0
- data/lib/omnizip/rubyzip_compat.rb +63 -0
- data/lib/omnizip/temp/safe_extract.rb +168 -0
- data/lib/omnizip/temp/temp_file.rb +124 -0
- data/lib/omnizip/temp/temp_file_pool.rb +109 -0
- data/lib/omnizip/temp.rb +181 -0
- data/lib/omnizip/version.rb +5 -0
- data/lib/omnizip/zip/entry.rb +156 -0
- data/lib/omnizip/zip/file.rb +485 -0
- data/lib/omnizip/zip/input_stream.rb +273 -0
- data/lib/omnizip/zip/output_stream.rb +324 -0
- data/lib/omnizip.rb +156 -0
- data/readme-docs/advanced-features.adoc +515 -0
- data/readme-docs/api-usage.adoc +444 -0
- data/readme-docs/architecture.adoc +449 -0
- data/readme-docs/archive-formats.adoc +479 -0
- data/readme-docs/cli-usage.adoc +222 -0
- data/readme-docs/compression-algorithms.adoc +442 -0
- data/readme-docs/compression-profiles.adoc +247 -0
- data/readme-docs/encryption-checksums.adoc +328 -0
- data/readme-docs/format-converter.adoc +325 -0
- data/readme-docs/installation.adoc +228 -0
- data/readme-docs/par2-archives.adoc +608 -0
- data/readme-docs/performance-profiler.adoc +389 -0
- data/readme-docs/preprocessing-filters.adoc +280 -0
- data/xz-file-format-1.2.1.txt +1174 -0
- metadata +617 -0
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
begin
|
|
4
|
+
require "lutaml/model"
|
|
5
|
+
rescue LoadError, ArgumentError
|
|
6
|
+
# lutaml-model not available, using simple classes
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
module Omnizip
|
|
10
|
+
module Formats
|
|
11
|
+
module Rar
|
|
12
|
+
module Rar5
|
|
13
|
+
module Models
|
|
14
|
+
# Encryption options for RAR5 archives
|
|
15
|
+
#
|
|
16
|
+
# This model configures password-based encryption using
|
|
17
|
+
# AES-256-CBC with PBKDF2-HMAC-SHA256 key derivation.
|
|
18
|
+
#
|
|
19
|
+
# @example Enable encryption with default settings
|
|
20
|
+
# options = EncryptionOptions.new(
|
|
21
|
+
# enabled: true,
|
|
22
|
+
# password: "SecurePassword123"
|
|
23
|
+
# )
|
|
24
|
+
#
|
|
25
|
+
# @example Custom KDF iterations
|
|
26
|
+
# options = EncryptionOptions.new(
|
|
27
|
+
# enabled: true,
|
|
28
|
+
# password: "SecurePassword123",
|
|
29
|
+
# kdf_iterations: 524_288 # 2^19, higher security
|
|
30
|
+
# )
|
|
31
|
+
class EncryptionOptions < Lutaml::Model::Serializable
|
|
32
|
+
# Enable encryption (default: false)
|
|
33
|
+
attribute :enabled, :boolean, default: false
|
|
34
|
+
|
|
35
|
+
# Encryption password
|
|
36
|
+
attribute :password, :string, default: nil
|
|
37
|
+
|
|
38
|
+
# PBKDF2 iteration count (default: 262,144 = 2^18)
|
|
39
|
+
attribute :kdf_iterations, :integer, default: 262_144
|
|
40
|
+
|
|
41
|
+
# Validate options
|
|
42
|
+
#
|
|
43
|
+
# @raise [ArgumentError] If validation fails
|
|
44
|
+
def validate!
|
|
45
|
+
if enabled? && (password.nil? || password.empty?)
|
|
46
|
+
raise ArgumentError,
|
|
47
|
+
"Password required when encryption is enabled"
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
if kdf_iterations < 65_536 || kdf_iterations > 1_048_576
|
|
51
|
+
raise ArgumentError,
|
|
52
|
+
"KDF iterations must be between 65,536 and 1,048,576"
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# Check if encryption is enabled
|
|
57
|
+
#
|
|
58
|
+
# @return [Boolean] true if enabled
|
|
59
|
+
def enabled?
|
|
60
|
+
enabled == true
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
# Check if password is set
|
|
64
|
+
#
|
|
65
|
+
# @return [Boolean] true if password provided
|
|
66
|
+
def has_password?
|
|
67
|
+
!password.nil? && !password.empty?
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
end
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
begin
|
|
4
|
+
require "lutaml/model"
|
|
5
|
+
rescue LoadError, ArgumentError
|
|
6
|
+
# lutaml-model not available, using simple classes
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
module Omnizip
|
|
10
|
+
module Formats
|
|
11
|
+
module Rar
|
|
12
|
+
module Rar5
|
|
13
|
+
module Models
|
|
14
|
+
# Recovery (PAR2) options for RAR5 archives
|
|
15
|
+
#
|
|
16
|
+
# This model configures PAR2 parity file generation for error
|
|
17
|
+
# correction and recovery of corrupted or missing archive data.
|
|
18
|
+
#
|
|
19
|
+
# @example Enable recovery with default settings
|
|
20
|
+
# options = RecoveryOptions.new(enabled: true)
|
|
21
|
+
#
|
|
22
|
+
# @example Custom redundancy percentage
|
|
23
|
+
# options = RecoveryOptions.new(
|
|
24
|
+
# enabled: true,
|
|
25
|
+
# redundancy: 10 # 10% redundancy
|
|
26
|
+
# )
|
|
27
|
+
class RecoveryOptions < Lutaml::Model::Serializable
|
|
28
|
+
# Enable PAR2 recovery (default: false)
|
|
29
|
+
attribute :enabled, :boolean, default: false
|
|
30
|
+
|
|
31
|
+
# Redundancy percentage (0-100, default: 5)
|
|
32
|
+
attribute :redundancy, :integer, default: 5
|
|
33
|
+
|
|
34
|
+
# Block size for PAR2 (default: 16384)
|
|
35
|
+
attribute :block_size, :integer, default: 16_384
|
|
36
|
+
|
|
37
|
+
# Validate options
|
|
38
|
+
#
|
|
39
|
+
# @raise [ArgumentError] If validation fails
|
|
40
|
+
def validate!
|
|
41
|
+
if redundancy.negative? || redundancy > 100
|
|
42
|
+
raise ArgumentError,
|
|
43
|
+
"Redundancy must be 0-100, got #{redundancy}"
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
if block_size <= 0 || (block_size % 4) != 0
|
|
47
|
+
raise ArgumentError,
|
|
48
|
+
"Block size must be positive and divisible by 4"
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# Check if recovery is enabled
|
|
53
|
+
#
|
|
54
|
+
# @return [Boolean] true if enabled
|
|
55
|
+
def enabled?
|
|
56
|
+
enabled == true
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
begin
|
|
4
|
+
require "lutaml/model"
|
|
5
|
+
rescue LoadError, ArgumentError
|
|
6
|
+
# lutaml-model not available, using simple classes
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
module Omnizip
|
|
10
|
+
module Formats
|
|
11
|
+
module Rar
|
|
12
|
+
module Rar5
|
|
13
|
+
module Models
|
|
14
|
+
# Solid compression options
|
|
15
|
+
#
|
|
16
|
+
# This model configures solid compression behavior, including
|
|
17
|
+
# whether to enable solid mode and block size limits.
|
|
18
|
+
#
|
|
19
|
+
# @example Enable solid compression
|
|
20
|
+
# options = SolidOptions.new(enabled: true)
|
|
21
|
+
#
|
|
22
|
+
# @example Configure solid block size
|
|
23
|
+
# options = SolidOptions.new(enabled: true, max_block_size: 100 * 1024 * 1024)
|
|
24
|
+
class SolidOptions < Lutaml::Model::Serializable
|
|
25
|
+
# Enable solid compression (default: false)
|
|
26
|
+
attribute :enabled, :boolean, default: false
|
|
27
|
+
|
|
28
|
+
# Maximum solid block size in bytes (default: unlimited)
|
|
29
|
+
# When set, files are grouped into blocks not exceeding this size
|
|
30
|
+
attribute :max_block_size, :integer, default: nil
|
|
31
|
+
|
|
32
|
+
# Whether to split by file extension (default: false)
|
|
33
|
+
# When true, files with different extensions use separate solid blocks
|
|
34
|
+
attribute :split_by_extension, :boolean, default: false
|
|
35
|
+
|
|
36
|
+
# Validate options
|
|
37
|
+
#
|
|
38
|
+
# @raise [ArgumentError] if max_block_size is too small
|
|
39
|
+
def validate!
|
|
40
|
+
if max_block_size && max_block_size < 1_048_576 # 1 MB minimum
|
|
41
|
+
raise ArgumentError, "Solid block size must be at least 1 MB"
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# Check if solid compression is enabled
|
|
46
|
+
#
|
|
47
|
+
# @return [Boolean] true if enabled
|
|
48
|
+
def enabled?
|
|
49
|
+
enabled == true
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# Check if block size is limited
|
|
53
|
+
#
|
|
54
|
+
# @return [Boolean] true if max_block_size set
|
|
55
|
+
def block_size_limited?
|
|
56
|
+
!max_block_size.nil?
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
begin
|
|
4
|
+
require "lutaml/model"
|
|
5
|
+
rescue LoadError, ArgumentError
|
|
6
|
+
# lutaml-model not available, using simple classes
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
module Omnizip
|
|
10
|
+
module Formats
|
|
11
|
+
module Rar
|
|
12
|
+
module Rar5
|
|
13
|
+
module Models
|
|
14
|
+
# Volume options for multi-volume archives
|
|
15
|
+
#
|
|
16
|
+
# This model configures split archive behavior, including
|
|
17
|
+
# maximum volume size and naming convention.
|
|
18
|
+
#
|
|
19
|
+
# @example Create with default options
|
|
20
|
+
# options = VolumeOptions.new
|
|
21
|
+
# options.max_volume_size # => 104857600 (100 MB)
|
|
22
|
+
#
|
|
23
|
+
# @example Create with custom size
|
|
24
|
+
# options = VolumeOptions.new(max_volume_size: 10 * 1024 * 1024)
|
|
25
|
+
# options.max_volume_size # => 10485760 (10 MB)
|
|
26
|
+
class VolumeOptions < Lutaml::Model::Serializable
|
|
27
|
+
# Maximum size per volume in bytes (default: 100 MB)
|
|
28
|
+
attribute :max_volume_size, :integer, default: 104_857_600
|
|
29
|
+
|
|
30
|
+
# Volume naming pattern (default: 'part')
|
|
31
|
+
# Results in: archive.part1.rar, archive.part2.rar, etc.
|
|
32
|
+
attribute :volume_naming, :string, default: "part"
|
|
33
|
+
|
|
34
|
+
# Validate options
|
|
35
|
+
#
|
|
36
|
+
# @raise [ArgumentError] if max_volume_size is too small
|
|
37
|
+
def validate!
|
|
38
|
+
if max_volume_size < 65_536 # 64 KB minimum
|
|
39
|
+
raise ArgumentError, "Volume size must be at least 64 KB"
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# Parse human-readable size string
|
|
44
|
+
#
|
|
45
|
+
# @param size_str [String] Size with suffix (e.g., "10M", "1G")
|
|
46
|
+
# @return [Integer] Size in bytes
|
|
47
|
+
def self.parse_size(size_str)
|
|
48
|
+
return size_str if size_str.is_a?(Integer)
|
|
49
|
+
|
|
50
|
+
match = size_str.match(/^(\d+(?:\.\d+)?)\s*([KMGT])?$/i)
|
|
51
|
+
unless match
|
|
52
|
+
raise ArgumentError,
|
|
53
|
+
"Invalid size format: #{size_str}"
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
value = match[1].to_f
|
|
57
|
+
suffix = match[2]&.upcase
|
|
58
|
+
|
|
59
|
+
multiplier = case suffix
|
|
60
|
+
when "K" then 1024
|
|
61
|
+
when "M" then 1024**2
|
|
62
|
+
when "G" then 1024**3
|
|
63
|
+
when "T" then 1024**4
|
|
64
|
+
else 1
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
(value * multiplier).to_i
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
end
|
|
@@ -0,0 +1,290 @@
|
|
|
1
|
+
# Multi-Volume Archive Architecture
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
The multi-volume archive feature enables splitting large RAR5 archives into multiple smaller volume files. This is essential for distributing large datasets across size-constrained media or network uploads.
|
|
6
|
+
|
|
7
|
+
## Key Components
|
|
8
|
+
|
|
9
|
+
### 1. VolumeOptions Model
|
|
10
|
+
**File**: [`models/volume_options.rb`](../models/volume_options.rb)
|
|
11
|
+
|
|
12
|
+
**Purpose**: Configuration for multi-volume archives using Lutaml::Model
|
|
13
|
+
|
|
14
|
+
**Attributes**:
|
|
15
|
+
- `max_volume_size` (Integer): Maximum size per volume in bytes (default: 100 MB)
|
|
16
|
+
- `volume_naming` (String): Naming pattern - "part", "volume", or "numeric"
|
|
17
|
+
|
|
18
|
+
**Key Methods**:
|
|
19
|
+
- `validate!`: Ensures volume size >= 64 KB minimum
|
|
20
|
+
- `parse_size(str)`: Converts "10M", "1G" to bytes
|
|
21
|
+
|
|
22
|
+
### 2. VolumeSplitter
|
|
23
|
+
**File**: [`volume_splitter.rb`](volume_splitter.rb)
|
|
24
|
+
|
|
25
|
+
**Purpose**: Handles data splitting logic and file distribution calculation
|
|
26
|
+
|
|
27
|
+
**Responsibilities**:
|
|
28
|
+
- Calculate optimal file distribution across volumes
|
|
29
|
+
- Track volume boundaries and remaining space
|
|
30
|
+
- Ensure atomic file placement (no mid-file splits in v0.5.0)
|
|
31
|
+
- Reserve header overhead (1 KB per volume)
|
|
32
|
+
|
|
33
|
+
**Key Methods**:
|
|
34
|
+
- `can_fit_in_current_volume?(size)`: Check space availability
|
|
35
|
+
- `calculate_file_distribution(files)`: Optimize file placement
|
|
36
|
+
- `needs_splitting?(total, max)`: Determine if splitting required
|
|
37
|
+
|
|
38
|
+
**Algorithm**:
|
|
39
|
+
```
|
|
40
|
+
For each file:
|
|
41
|
+
If file fits in current volume with headers:
|
|
42
|
+
Add to current volume
|
|
43
|
+
Else:
|
|
44
|
+
Finalize current volume
|
|
45
|
+
Start new volume with this file
|
|
46
|
+
Return: Array of volume assignments
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
### 3. VolumeWriter
|
|
50
|
+
**File**: [`volume_writer.rb`](volume_writer.rb)
|
|
51
|
+
|
|
52
|
+
**Purpose**: Write individual .rar volume files with proper headers
|
|
53
|
+
|
|
54
|
+
**Responsibilities**:
|
|
55
|
+
- Write RAR5 signature to each volume
|
|
56
|
+
- Add volume-specific Main header flags
|
|
57
|
+
- Write file entries
|
|
58
|
+
- Add End header with continuation flags
|
|
59
|
+
|
|
60
|
+
**RAR5 Volume Flags**:
|
|
61
|
+
```ruby
|
|
62
|
+
# Main header
|
|
63
|
+
VOLUME_ARCHIVE_FLAG = 0x0001 # Indicates multi-volume archive
|
|
64
|
+
VOLUME_NUMBER_FLAG = 0x0002 # Volume number in extra area
|
|
65
|
+
|
|
66
|
+
# End header
|
|
67
|
+
VOLUME_END_FLAG = 0x0001 # More volumes follow (not last)
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
**Volume Naming**:
|
|
71
|
+
- **part**: `archive.part1.rar`, `archive.part2.rar`, ... (default)
|
|
72
|
+
- **volume**: `archive.vol1.rar`, `archive.vol2.rar`, ...
|
|
73
|
+
- **numeric**: `archive.rar`, `archive.r00`, `archive.r01`, ...
|
|
74
|
+
|
|
75
|
+
### 4. VolumeManager
|
|
76
|
+
**File**: [`volume_manager.rb`](volume_manager.rb)
|
|
77
|
+
|
|
78
|
+
**Purpose**: Coordinate entire multi-volume archive creation
|
|
79
|
+
|
|
80
|
+
**Responsibilities**:
|
|
81
|
+
- Accept file additions
|
|
82
|
+
- Compress all files upfront
|
|
83
|
+
- Calculate optimal volume distribution
|
|
84
|
+
- Delegate to VolumeWriter for each volume
|
|
85
|
+
- Return array of created volume paths
|
|
86
|
+
|
|
87
|
+
**Workflow**:
|
|
88
|
+
```
|
|
89
|
+
1. User adds files via add_file() or add_directory()
|
|
90
|
+
2. User calls create_volumes()
|
|
91
|
+
3. Manager compresses all files
|
|
92
|
+
4. Calculate file distribution across volumes
|
|
93
|
+
5. For each volume:
|
|
94
|
+
a. Create VolumeWriter
|
|
95
|
+
b. Write signature + main header
|
|
96
|
+
c. Write assigned file entries
|
|
97
|
+
d. Write end header
|
|
98
|
+
6. Return volume paths
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Integration with Existing Writer
|
|
102
|
+
|
|
103
|
+
The [`Writer`](../writer.rb) class is enhanced with multi-volume support:
|
|
104
|
+
|
|
105
|
+
**New Options**:
|
|
106
|
+
- `multi_volume: true` - Enable multi-volume mode
|
|
107
|
+
- `volume_size: Integer` - Maximum volume size (accepts human-readable)
|
|
108
|
+
|
|
109
|
+
**API**:
|
|
110
|
+
```ruby
|
|
111
|
+
# Single archive (existing)
|
|
112
|
+
writer = Writer.new('archive.rar', compression: :lzma)
|
|
113
|
+
writer.add_file('file.txt')
|
|
114
|
+
writer.write
|
|
115
|
+
|
|
116
|
+
# Multi-volume (new)
|
|
117
|
+
writer = Writer.new('archive.rar',
|
|
118
|
+
multi_volume: true,
|
|
119
|
+
volume_size: '10M', # or 10_485_760
|
|
120
|
+
compression: :lzma
|
|
121
|
+
)
|
|
122
|
+
writer.add_file('largefile.dat')
|
|
123
|
+
writer.write # Returns: ['archive.part1.rar', 'archive.part2.rar', ...]
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
**Implementation Strategy**:
|
|
127
|
+
- Check `multi_volume` option in `write()`
|
|
128
|
+
- If enabled, delegate to VolumeManager
|
|
129
|
+
- If disabled, use existing single-file logic
|
|
130
|
+
- Clean separation - no complex conditionals
|
|
131
|
+
|
|
132
|
+
## Data Flow
|
|
133
|
+
|
|
134
|
+
### Single Archive (Existing)
|
|
135
|
+
```
|
|
136
|
+
Files → Writer → Compress → Single .rar file
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### Multi-Volume Archive (New)
|
|
140
|
+
```
|
|
141
|
+
Files → Writer (multi_volume=true)
|
|
142
|
+
↓
|
|
143
|
+
VolumeManager
|
|
144
|
+
↓
|
|
145
|
+
Compress all files
|
|
146
|
+
↓
|
|
147
|
+
VolumeSplitter (calculate distribution)
|
|
148
|
+
↓
|
|
149
|
+
For each volume:
|
|
150
|
+
VolumeWriter → part1.rar, part2.rar, ...
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
## Volume Format Specification
|
|
154
|
+
|
|
155
|
+
### Volume File Structure
|
|
156
|
+
```
|
|
157
|
+
Volume 1 (archive.part1.rar):
|
|
158
|
+
[RAR5 Signature: 8 bytes]
|
|
159
|
+
[Main Header: VOLUME_ARCHIVE_FLAG set]
|
|
160
|
+
[File1 Header + Data]
|
|
161
|
+
[File2 Header + Data]
|
|
162
|
+
...
|
|
163
|
+
[End Header: VOLUME_END_FLAG set]
|
|
164
|
+
|
|
165
|
+
Volume 2 (archive.part2.rar):
|
|
166
|
+
[RAR5 Signature: 8 bytes]
|
|
167
|
+
[Main Header: VOLUME_ARCHIVE_FLAG + VOLUME_NUMBER_FLAG]
|
|
168
|
+
[File5 Header + Data]
|
|
169
|
+
...
|
|
170
|
+
[End Header: VOLUME_END_FLAG set]
|
|
171
|
+
|
|
172
|
+
Last Volume (archive.partN.rar):
|
|
173
|
+
[RAR5 Signature: 8 bytes]
|
|
174
|
+
[Main Header: VOLUME_ARCHIVE_FLAG + VOLUME_NUMBER_FLAG]
|
|
175
|
+
[FileX Header + Data]
|
|
176
|
+
...
|
|
177
|
+
[End Header: VOLUME_END_FLAG NOT set]
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
## Limitations (v0.5.0)
|
|
181
|
+
|
|
182
|
+
1. **No File Spanning**: Individual files cannot span multiple volumes
|
|
183
|
+
- Large files must fit in a single volume
|
|
184
|
+
- Trade-off: Simplicity vs flexibility
|
|
185
|
+
- Future: Implement file spanning in v0.6.0+
|
|
186
|
+
|
|
187
|
+
2. **Sequential Creation**: Volumes created sequentially, not in parallel
|
|
188
|
+
- Simpler implementation
|
|
189
|
+
- Future: Parallel volume writing with Ractors
|
|
190
|
+
|
|
191
|
+
3. **Fixed Boundaries**: Volume splits at file boundaries only
|
|
192
|
+
- Predictable behavior
|
|
193
|
+
- May result in inefficient space usage
|
|
194
|
+
|
|
195
|
+
## Testing Strategy
|
|
196
|
+
|
|
197
|
+
### Unit Tests
|
|
198
|
+
- VolumeOptions: Validation, size parsing
|
|
199
|
+
- VolumeSplitter: Distribution algorithm, space calculation
|
|
200
|
+
- VolumeWriter: Header generation, filename creation
|
|
201
|
+
- VolumeManager: File preparation, volume coordination
|
|
202
|
+
|
|
203
|
+
### Integration Tests
|
|
204
|
+
- Small archive (< volume size): Single volume
|
|
205
|
+
- Large archive (> volume size): Multiple volumes
|
|
206
|
+
- Many small files: Optimal distribution
|
|
207
|
+
- Few large files: One per volume
|
|
208
|
+
- Round-trip: Extract with unrar, verify integrity
|
|
209
|
+
|
|
210
|
+
### Compatibility Tests
|
|
211
|
+
- Extract with official unrar
|
|
212
|
+
- Extract with 7-Zip
|
|
213
|
+
- Verify all volume flags correct
|
|
214
|
+
- Verify volume sequence correct
|
|
215
|
+
|
|
216
|
+
## Performance Considerations
|
|
217
|
+
|
|
218
|
+
### Memory Usage
|
|
219
|
+
- All files compressed before splitting → Memory = sum of compressed sizes
|
|
220
|
+
- For 100 MB archives, reasonable memory footprint
|
|
221
|
+
- Future: Streaming compression for > 1 GB archives
|
|
222
|
+
|
|
223
|
+
### Disk I/O
|
|
224
|
+
- Sequential writes to multiple files
|
|
225
|
+
- No random access required
|
|
226
|
+
- Single pass through data
|
|
227
|
+
|
|
228
|
+
### Optimization Opportunities
|
|
229
|
+
1. Parallel compression of files (Ractors)
|
|
230
|
+
2. Streaming compression to reduce memory
|
|
231
|
+
3. Intelligent file ordering (similar files together for solid compression)
|
|
232
|
+
|
|
233
|
+
## Error Handling
|
|
234
|
+
|
|
235
|
+
### Volume Size Too Small
|
|
236
|
+
```ruby
|
|
237
|
+
VolumeOptions#validate!
|
|
238
|
+
# Raises ArgumentError if < 64 KB
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
### File Too Large
|
|
242
|
+
```ruby
|
|
243
|
+
VolumeSplitter#calculate_file_distribution
|
|
244
|
+
# Places large file in dedicated volume
|
|
245
|
+
# Logs warning if file > max_volume_size
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
### Disk Space Exhausted
|
|
249
|
+
```ruby
|
|
250
|
+
VolumeWriter#write
|
|
251
|
+
# Standard Ruby File I/O exceptions propagate
|
|
252
|
+
# Partial volumes cleaned up on failure (future)
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
## Future Enhancements (Post-v0.5.0)
|
|
256
|
+
|
|
257
|
+
1. **File Spanning**: Split large files across volumes
|
|
258
|
+
2. **Parallel Processing**: Compress files in parallel
|
|
259
|
+
3. **Streaming Mode**: Compress directly to volumes
|
|
260
|
+
4. **Resume Support**: Continue interrupted volume creation
|
|
261
|
+
5. **Volume Verification**: CRC checks for each volume
|
|
262
|
+
6. **Automatic Recovery**: Generate PAR2 for each volume
|
|
263
|
+
|
|
264
|
+
## References
|
|
265
|
+
|
|
266
|
+
- RAR5 Format Specification: https://www.rarlab.com/technote.htm
|
|
267
|
+
- RAR Volume Format Details: Section 4.3
|
|
268
|
+
- Volume Header Flags: Section 3.1
|
|
269
|
+
|
|
270
|
+
## Design Decisions
|
|
271
|
+
|
|
272
|
+
### Why Upfront Compression?
|
|
273
|
+
**Decision**: Compress all files before splitting
|
|
274
|
+
**Rationale**: Need exact compressed size to calculate distribution
|
|
275
|
+
**Trade-off**: Higher memory usage vs accurate splitting
|
|
276
|
+
|
|
277
|
+
### Why Atomic Files?
|
|
278
|
+
**Decision**: No file spanning in v0.5.0
|
|
279
|
+
**Rationale**: Simpler implementation, faster delivery
|
|
280
|
+
**Trade-off**: Potential wasted space vs implementation complexity
|
|
281
|
+
|
|
282
|
+
### Why Sequential Writing?
|
|
283
|
+
**Decision**: Write volumes one at a time
|
|
284
|
+
**Rationale**: Simpler error handling, predictable behavior
|
|
285
|
+
**Trade-off**: Slower for very large archives vs complexity
|
|
286
|
+
|
|
287
|
+
### Why Lutaml::Model?
|
|
288
|
+
**Decision**: Use Lutaml::Model for VolumeOptions
|
|
289
|
+
**Rationale**: Consistent with project architecture, serialization support
|
|
290
|
+
**Benefit**: YAML/JSON configuration files for volume settings
|