omnizip 0.4.7-arm-linux

data/docs/guides/compression-algorithms/ppmd.adoc

@@ -0,0 +1,313 @@
+---
+title: PPMd
+nav_order: 6
+parent: Compression Algorithms
+grand_parent: Guides
+---
+
+== Purpose
+
+PPMd7 (Prediction by Partial Matching, variant H revision 7) is a statistical compression algorithm that excels at compressing text files. It achieves excellent compression ratios on source code, documents, and repetitive text.
+
+== Key Characteristics
+
+[cols="1,3"]
+|===
+|Property |Value
+
+|Compression Ratio
+|Excellent (best for text)
+
+|Compression Speed
+|Slow
+
+|Decompression Speed
+|Slow
+
+|Memory Usage
+|High (configurable)
+
+|Best For
+|Text files, source code, XML/JSON, logs
+|===
+
+== When to Use PPMd7
+
+**Choose PPMd7 when**:
+
+* Compressing primarily text data
+* Source code archives
+* XML, JSON, or structured text files
+* Log file archives
+* Maximum text compression is needed
+
+**Avoid PPMd7 when**:
+
+* Working with binary data (use LZMA2)
+* Working with already-compressed files
+* Speed is more important than ratio
+* Memory is very limited
+
+== Basic Usage
+
+=== Compress Text Files
+
+[source,ruby]
+----
+# Compress source code with PPMd7
+Omnizip::Archive.create('code.7z', format: :seven_zip,
+                        algorithm: :ppmd7) do |archive|
+  archive.add_directory('src/')
+end
+----
+
+=== Memory and Order
+
+PPMd memory (2^mem_size bytes) and context order are encoder
+parameters; the writer currently emits PPMd with default settings
+(empty properties, which readers interpret as defaults), so there
+are no +mem_size:/+order: knobs yet:
+
+[source,ruby]
+----
+Omnizip::Archive.create('text.7z', format: :seven_zip,
+                        algorithm: :ppmd7) do |archive|
+  archive.add_directory('documents/')
+end
+----
+
+== Configuration Parameters
+
+The writer currently emits PPMd with default memory/order settings;
+the tables below describe the algorithm's parameters for reference
+when choosing PPMd.
+
+=== Memory Size
+
+Controls context tree size (2^mem_size bytes):
+
+[cols="1,1,2,2"]
+|===
+|mem_size |Memory |Compression |Best For
+
+|20
+|1 MB
+|Good
+|Small files, limited memory
+
+|22
+|4 MB
+|Very Good
+|Most text files
+
+|24
+|16 MB
+|Excellent
+|**Default** - large texts
+
+|26
+|64 MB
+|Best
+|Very large text corpora
+
+|28
+|256 MB
+|Maximum
+|Huge text archives
+|===
+
+=== Context Order
+
+Number of previous characters used for prediction:
+
+[cols="1,2,2"]
+|===
+|Order |Compression |Speed
+
+|4
+|Good
+|Faster
+
+|6
+|Excellent
+|**Default**
+
+|8
+|Best
+|Slower
+
+|16
+|Maximum
+|Very slow
+|===
+
+.Configuration Example
+[source,ruby]
+----
+# For large source code archive
+Omnizip::Archive.create('linux-kernel.7z', format: :seven_zip,
+                        algorithm: :ppmd7) do |archive|
+  archive.add_directory('linux-6.0/')
+end
+----
+
+== Performance Characteristics
+
+=== Compression Ratios (Text Files)
+
+[cols="2,1,1,1"]
+|===
+|Content Type |Deflate |LZMA2 |PPMd7
+
+|Source Code (.rb, .js)
+|65%
+|72%
+|78%
+
+|XML/JSON
+|68%
+|75%
+|82%
+
+|Plain Text
+|62%
+|70%
+|80%
+
+|Logs (repetitive)
+|70%
+|76%
+|85%
+|===
+
+=== Speed Comparison
+
+For 10 MB text file:
+
+[cols="2,1,1,1"]
+|===
+|Algorithm |Compress Time |Decompress Time |Ratio
+
+|**PPMd7**
+|25s
+|18s
+|80%
+
+|LZMA2
+|15s
+|2s
+|75%
+
+|Deflate
+|3s
+|1s
+|65%
+
+|Zstandard
+|1s
+|0.5s
+|68%
+|===
+
+== Common Use Cases
+
+=== Source Code Archives
+
+[source,ruby]
+----
+# Archive project source code
+Omnizip::Archive.create("project-v1.0.7z", format: :seven_zip,
+                        algorithm: :ppmd7) do |archive|
+  archive.add_directory('src/')
+  archive.add_directory('tests/')
+  archive.add_file('README.md')
+end
+----
+
+=== Log File Compression
+
+[source,ruby]
+----
+# Compress application logs with maximum ratio
+log_files = Dir.glob('/var/log/app/*.log')
+
+Omnizip::Archive.create("logs_#{Date.today}.7z",
+                        format: :seven_zip,
+                        algorithm: :ppmd7) do |archive|
+  log_files.each { |log| archive.add_file(log) }
+end
+----
+
+=== Documentation Archives
+
+[source,ruby]
+----
+# Compress documentation and markup files
+Omnizip::Archive.create('docs.7z', format: :seven_zip,
+                        algorithm: :ppmd7) do |archive|
+  archive.add_directory('docs/')
+  archive.add_directory('wiki/')
+  archive.add_directory('examples/')
+end
+----
+
+== PPMd7 vs PPMd8
+
+Both are variants of PPMd with slight differences:
+
+[cols="2,1,1,2"]
+|===
+|Characteristic |PPMd7 |PPMd8 |Recommendation
+
+|Compression Ratio
+|Excellent
+|Excellent
+|Similar quality
+
+|Speed
+|Slow
+|Slow
+|Similar performance
+
+|Memory Usage
+|High
+|High
+|Similar requirements
+
+|Compatibility
+|Wider
+|Limited
+|Use PPMd7 for compatibility
+
+|Default in 7-Zip
+|Yes
+|No
+|PPMd7 is standard
+|===
+
+**Recommendation**: Use PPMd7 unless you have specific reasons to use PPMd8.
+
+== Optimization Tips
+
+. **Match Memory to File Size**: Use larger memory for larger files
+. **Higher Order for Repetitive Text**: Increase order for highly structured text
+. **Use Solid Archives**: PPMd benefits greatly from solid compression
+. **Combine with Preprocessing**: Generally not needed for text
+. **Monitor Memory Usage**: Ensure sufficient RAM available
+
+== Technical Details
+
+**Algorithm Type**: Statistical context modeling
+
+**Prediction Method**: Partial Matching with escape codes
+
+**Context Length**: Configurable (order parameter)
+
+**Memory Model**: Explicit memory size configuration
+
+**Compatibility**: Standard PPMd variant H implementation
+
+== See Also
+
+* link:lzma2.html[LZMA2] - General-purpose alternative
+* link:../../compatibility.html[Compatibility] - PPMd support across tools

data/docs/guides/compression-algorithms/zstandard.adoc

@@ -0,0 +1,296 @@
+---
+title: Zstandard
+nav_order: 8
+parent: Compression Algorithms
+grand_parent: Guides
+---
+
+== Purpose
+
+Zstandard (zstd) is a modern, fast compression algorithm developed by Facebook that offers an excellent balance between compression ratio and speed. It significantly outperforms Deflate in both speed and compression ratio.
+
+== Key Characteristics
+
+[cols="1,3"]
+|===
+|Property |Value
+
+|Compression Ratio
+|Very good (better than Deflate)
+
+|Compression Speed
+|Very fast (3-4x faster than Deflate)
+
+|Decompression Speed
+|Extremely fast (4-5x faster than Deflate)
+
+|Memory Usage
+|Low to medium
+
+|Best For
+|Real-time compression, high-throughput applications, modern systems
+|===
+
+== When to Use Zstandard
+
+**Choose Zstandard when**:
+
+* Need fast compression with good ratios
+* Real-time compression requirements
+* High throughput is critical
+* Working with modern systems and tools
+* Want better performance than Deflate without LZMA's slowness
+
+**Consider alternatives when**:
+
+* Maximum compatibility is required (use Deflate)
+* Best compression ratio needed (use LZMA2)
+* Working with legacy systems
+
+== Basic Usage
+
+=== Compress a File (.zst)
+
+[source,ruby]
+----
+# Writes a real Zstandard frame (validated by the system zstd CLI)
+Omnizip.compress_file('logfile.log', 'logfile.log.zst')
+
+# Compression level (generic 0-9 scale; values above 9 pass through
+# as native zstd levels 1-22)
+Omnizip.compress_file('data.bin', 'data.bin.zst', level: 9)
+
+Omnizip.decompress_file('logfile.log.zst', 'logfile.log')
+----
+
+=== Zstandard Entries in ZIP
+
+ZIP supports Zstandard as a per-entry method (method 93). Requires a
+modern reader (e.g. 7-Zip 21+):
+
+[source,ruby]
+----
+require 'omnizip/zip/output_stream'
+
+Omnizip::Zip::OutputStream.open('fast.zip') do |zip|
+  zip.put_next_entry('data.bin', compression: :zstandard, level: 3)
+  zip.write(data)
+end
+----
+
+== Compression Levels
+
+Zstandard offers levels 1-22 with fine-grained control:
+
+[cols="1,2,2,2"]
+|===
+|Level Range |Speed |Ratio |Typical Use
+
+|1-3
+|Very Fast
+|Good
+|Real-time, temporary files
+
+|3-7
+|Fast
+|Very Good
+|**Default** - general use
+
+|8-15
+|Medium
+|Excellent
+|Distribution, important data
+
+|16-22
+|Slow
+|Excellent+
+|Archival (approaches LZMA)
+|===
+
+.Level Performance Example
+[source,ruby]
+----
+# Compare different Zstandard levels (native zstd scale)
+[1, 12, 19].each do |level|
+  time = Benchmark.measure do
+    Omnizip.compress_file('data.bin', "zstd-#{level}.zst", level: level)
+  end
+
+  size = File.size("zstd-#{level}.zst")
+  puts "Level #{level}: #{size / 1024} KB in #{time.real.round(3)}s"
+end
+----
+
+== Performance Comparison
+
+No benchmark tables here: Omnizip's Zstandard is a pure Ruby
+implementation, so absolute throughput differs substantially from
+the C library. The algorithmic trade-offs hold — faster than LZMA
+per level, better ratio than Deflate — but measure on your own
+workloads.
+
+== Advanced Features
+
+=== Dictionary Compression
+
+A raw dictionary (shared prefix) dramatically improves ratios on
+small, repetitive inputs. Dictionary *training* is not implemented;
+prepare dictionary content yourself and load it with +from_raw+:
+
+[source,ruby]
+----
+dict = Omnizip::Algorithms::Zstandard::Dictionary.from_raw(
+  42, File.binread('shared-samples.bin')
+)
+
+zstd = Omnizip::Algorithms::Zstandard.new
+File.open('data.json', 'rb') do |input|
+  File.open('data.json.zstd', 'wb') do |output|
+    zstd.compress_with_dict(input, output, dict)
+  end
+end
+
+File.open('data.json.zstd', 'rb') do |input|
+  File.open('data.restored', 'wb') do |output|
+    zstd.decompress_with_dict(input, output, dict)
+  end
+end
+----
+
+=== Streaming
+
+The codec is stream-oriented:
+
+[source,ruby]
+----
+zstd = Omnizip::AlgorithmRegistry.get(:zstandard).new
+File.open('input.log', 'rb') do |input|
+  File.open('input.log.zst', 'wb') do |output|
+    zstd.compress(input, output, level: 3)
+  end
+end
+----
+
+== Use Cases
+
+=== Log File Compression
+
+Perfect for compressing logs quickly:
+
+[source,ruby]
+----
+# Compress today's logs rapidly
+Dir.glob('/var/log/app/*.log').each do |log|
+  Omnizip.compress_file(log, "#{log}.zst", level: 3)
+end
+----
+
+=== Web Application Responses
+
+Compress API responses on-the-fly:
+
+[source,ruby]
+----
+require 'stringio'
+
+class ExportController < ApplicationController
+  def download
+    data = generate_export_data # a String
+
+    zstd = Omnizip::AlgorithmRegistry.get(:zstandard).new
+    output = StringIO.new
+    zstd.compress(StringIO.new(data), output, level: 3)
+
+    send_data output.string,
+      filename: 'export.zst',
+      type: 'application/zstd'
+  end
+end
+----
+
+=== Database Dumps
+
+Stream a dump into a .zst frame:
+
+[source,ruby]
+----
+# Stream database dump to compressed file
+require 'open3'
+
+zstd = Omnizip::AlgorithmRegistry.get(:zstandard).new
+File.open('db-backup.sql.zst', 'wb') do |output|
+  # Stream pg_dump output directly
+  Open3.popen3('pg_dump mydb') do |_stdin, stdout, _stderr, _wait_thr|
+    zstd.compress(stdout, output, level: 7)
+  end
+end
+----
+
+== Memory Efficiency
+
+Zstandard is memory efficient:
+
+[cols="1,1,1"]
+|===
+|Level |Compression RAM |Decompression RAM
+
+|1-3
+|~1 MB
+|<1 MB
+
+|4-7
+|~8 MB
+|<1 MB
+
+|8-15
+|~30 MB
+|<1 MB
+
+|16-22
+|~100 MB
+|<1 MB
+|===
+
+*Note: Decompression always requires minimal memory*
+
+== Best Practices
+
+. **Use Level 3-5 for General Use**: Excellent speed/ratio balance
+. **Level 1 for Real-Time**: When speed is critical
+. **Level 10+ for Archival**: When time permits better compression
+. **Train Dictionaries**: For repetitive data types (JSON, logs)
+. **Monitor Performance**: Should achieve 50-200 MB/s compression
+
+== Command-Line Usage
+
+[source,bash]
+----
+$ omnizip compress data.bin data.bin.zst --algorithm zstandard
+
+# The output is a standard Zstandard frame, readable by the system
+# zstd CLI
+----
+
+== Technical Details
+
+**Zstandard Features**:
+
+* **Fast Entropy Encoding**: Finite State Entropy (FSE)
+* **Adaptive Window**: Dynamically sized based on level
+* **Dictionary Compression**: Raw-dictionary priming (entropy-table
+  preloading from RFC dictionaries is not implemented)
+* **Frames**: Interoperable with the reference zstd CLI (verified
+  both directions)
+
+**Implementation**: Pure Ruby, part of Omnizip. No native binding.
+
+**Not available**: Zstandard is not offered as a 7z coder method;
+in 7z archives use LZMA2/PPMd/BZip2. Multi-threading is not
+implemented (single-threaded encoder).
+
+== See Also
+
+* link:deflate.html[Deflate] - Traditional fast compression
+* link:lzma2.html[LZMA2] - Better ratio, slower speed
+* link:../advanced-features/streaming.html[Streaming] - Zstandard excels at streaming
+* link:../performance-tuning.html[Performance Tuning] - Optimize for speed