omnizip 0.3.1

data/readme-docs/encryption-checksums.adoc

@@ -0,0 +1,328 @@
+= Encryption & Checksums Guide
+:toc:
+:toclevels: 3
+
+== Purpose
+
+This document covers encryption and checksum features in Omnizip for securing archives and verifying data integrity.
+
+== AES-256 Encryption
+
+=== General
+
+Omnizip provides password-protected archive encryption using AES-256-CBC with SHA-256 key derivation. This ensures strong security for sensitive data in archives.
+
+=== How It Works
+
+. **Key Derivation:** Password is hashed using SHA-256 to create encryption key
+. **Encryption:** Data is encrypted using AES-256 in CBC (Cipher Block Chaining) mode
+. **Initialization Vector (IV):** Random IV is generated for each archive
+. **Storage:** Encrypted data and IV are stored in archive metadata
+
+=== Security Features
+
+* **AES-256:** Industry-standard strong encryption (256-bit key)
+* **CBC Mode:** Cipher Block Chaining prevents pattern analysis
+* **SHA-256:** Secure hash function for key derivation
+* **Random IV:** Each archive has unique initialization vector
+* **Salt:** Password is salted before hashing (prevents rainbow tables)
+
+=== Usage with .7z Archives
+
+[source,ruby]
+----
+# Create encrypted archive
+writer = Omnizip::Formats::SevenZip::Writer.new(
+  'secure_archive.7z',
+  password: "my_secure_password"
+)
+
+writer.add_file('sensitive_document.pdf')
+writer.add_file('confidential_data.xlsx')
+writer.close
+
+# Extract encrypted archive
+reader = Omnizip::Formats::SevenZip::Reader.new(
+  'secure_archive.7z',
+  password: "my_secure_password"
+)
+
+reader.extract_all('output/')
+reader.close
+----
+
+=== Password Strength Recommendations
+
+**Weak (avoid):**
+* Single words
+* Common passwords
+* < 8 characters
+* No special characters
+
+**Strong (recommended):**
+* 12+ characters
+* Mix of uppercase, lowercase, numbers, symbols
+* Not based on dictionary words
+* Unique for each archive
+
+**Example strong passwords:**
+* `mK9$pL2#nQ5@vR8!`
+* `Correct-Horse-Battery-Staple-2024`
+* `Th1s!sMyS3cur3P@ssw0rd`
+
+=== Security Considerations
+
+**What encryption protects:**
+* File contents from unauthorized access
+* Archive contents from modification
+* Data confidentiality during storage/transfer
+
+**What encryption does NOT protect:**
+* File names (visible in archive)
+* File sizes (visible in archive)
+* Directory structure (visible in archive)
+* Metadata (creation dates, etc.)
+
+**Important notes:**
+* Store passwords securely (password managers recommended)
+* Lost passwords cannot be recovered
+* Stronger passwords require more time to crack
+* Consider using key files for additional security
+
+=== Performance Impact
+
+Encryption adds minimal overhead:
+
+* **Encryption:** ~5-10% slower than unencrypted
+* **Decryption:** ~5-10% slower than unencrypted
+* **Memory:** No significant increase
+
+The security benefits far outweigh the small performance cost.
+
+== CRC32 Checksums
+
+=== General
+
+CRC32 (Cyclic Redundancy Check 32-bit) provides fast integrity verification for compressed data. It detects accidental corruption but is not cryptographically secure.
+
+=== Characteristics
+
+* **Size:** 32-bit (4 bytes)
+* **Speed:** Very fast
+* **Collision resistance:** Moderate
+* **Use case:** Detect accidental corruption
+
+=== Usage
+
+[source,ruby]
+----
+# Calculate CRC32 of data
+data = File.binread('file.txt')
+crc32 = Omnizip::ChecksumRegistry.get(:crc32).calculate(data)
+
+puts "CRC32: #{crc32.to_s(16).upcase}"  # Hexadecimal format
+----
+
+=== When to Use CRC32
+
+**Good for:**
+* ZIP archives (standard checksum)
+* Quick integrity checks
+* Detecting transmission errors
+* Low-overhead verification
+
+**Not suitable for:**
+* Security-critical applications
+* Detecting intentional tampering
+* Cryptographic purposes
+
+== CRC64 Checksums
+
+=== General
+
+CRC64 (Cyclic Redundancy Check 64-bit) provides stronger integrity verification than CRC32 with very low collision probability. Used in .7z archives for reliable data verification.
+
+=== Characteristics
+
+* **Size:** 64-bit (8 bytes)
+* **Speed:** Very fast
+* **Collision resistance:** Very high
+* **Use case:** Reliable corruption detection
+
+=== Usage
+
+[source,ruby]
+----
+# Calculate CRC64 of data
+data = File.binread('large_file.dat')
+crc64 = Omnizip::ChecksumRegistry.get(:crc64).calculate(data)
+
+puts "CRC64: #{crc64.to_s(16).upcase}"
+----
+
+=== When to Use CRC64
+
+**Good for:**
+* .7z archives (standard checksum)
+* Large files (> 4GB)
+* High-reliability verification
+* Minimal collision risk needed
+
+**Advantages over CRC32:**
+* Much lower collision probability
+* Better for large datasets
+* More reliable for critical data
+
+## Checksum Comparison
+
+[cols="20,20,20,20,20",options="header"]
+|===
+|Feature |CRC32 |CRC64 |MD5 |SHA-256
+
+|Size
+|4 bytes
+|8 bytes
+|16 bytes
+|32 bytes
+
+|Speed
+|⭐⭐⭐⭐⭐
+|⭐⭐⭐⭐⭐
+|⭐⭐⭐⭐
+|⭐⭐⭐
+
+|Collision Resistance
+|⭐⭐
+|⭐⭐⭐⭐
+|⭐⭐⭐
+|⭐⭐⭐⭐⭐
+
+|Cryptographic Security
+|❌ No
+|❌ No
+|⚠️ Weak
+|✅ Yes
+
+|Best Use Case
+|ZIP archives
+|.7z archives
+|Legacy systems
+|Security-critical
+|===
+
+**Note:** MD5 and SHA-256 are not currently implemented in Omnizip core, but can be added if needed.
+
+== Archive Integrity Verification
+
+=== Automatic Verification
+
+Omnizip automatically verifies checksums during extraction:
+
+[source,ruby]
+----
+# Extraction automatically verifies checksums
+reader = Omnizip::Formats::SevenZip::Reader.new('archive.7z')
+
+begin
+  reader.extract_all('output/')
+  puts "✓ All files verified successfully"
+rescue Omnizip::ChecksumMismatchError => e
+  puts "✗ Checksum verification failed: #{e.message}"
+  puts "File may be corrupted: #{e.filename}"
+end
+
+reader.close
+----
+
+=== Manual Verification
+
+You can manually verify checksums without extraction:
+
+[source,ruby]
+----
+# Verify archive integrity without extracting
+reader = Omnizip::Formats::SevenZip::Reader.new('archive.7z')
+
+reader.entries.each do |entry|
+  # Get stored checksum
+  stored_crc = entry.crc
+  
+  # Read and calculate actual checksum
+  data = reader.read_entry_data(entry)
+  calculated_crc = Omnizip::ChecksumRegistry.get(:crc64).calculate(data)
+  
+  if stored_crc == calculated_crc
+    puts "✓ #{entry.name}: OK"
+  else
+    puts "✗ #{entry.name}: CORRUPTED"
+  end
+end
+
+reader.close
+----
+
+== Combined Security: Encryption + Checksums
+
+Omnizip uses both encryption and checksums for maximum security and integrity:
+
+[source,ruby]
+----
+# Create secure archive with both encryption and checksums
+writer = Omnizip::Formats::SevenZip::Writer.new(
+  'secure_archive.7z',
+  password: "strong_password_here",
+  checksum: :crc64  # Default, can be explicit
+)
+
+writer.add_file('important_data.xlsx')
+writer.close
+
+# Extraction verifies both encryption and checksums
+reader = Omnizip::Formats::SevenZip::Reader.new(
+  'secure_archive.7z',
+  password: "strong_password_here"
+)
+
+begin
+  reader.extract_all('output/')
+  puts "✓ Password correct, checksums valid, data intact"
+rescue Omnizip::WrongPasswordError
+  puts "✗ Incorrect password"
+rescue Omnizip::ChecksumMismatchError
+  puts "✗ Data corrupted (password correct but checksum failed)"
+end
+
+reader.close
+----
+
+== Best Practices
+
+=== For Security
+
+. **Use strong passwords** (12+ characters, mixed case, numbers, symbols)
+. **Don't reuse passwords** across different archives
+. **Store passwords securely** (password manager recommended)
+. **Use AES-256** for sensitive data
+. **Combine with file permissions** for defense in depth
+
+=== For Data Integrity
+
+. **Always verify checksums** during extraction
+. **Use CRC64** for large or critical files
+. **Test archives** after creation
+. **Keep backup copies** of important archives
+. **Verify archives periodically** (detect bit rot)
+
+=== For Performance
+
+. **Encryption overhead is minimal** (~5-10%) - always use for sensitive data
+. **CRC64 is very fast** - overhead is negligible
+. **Larger files benefit more** from CRC64 over CRC32
+. **Consider compression level** before encryption for best balance
+
+== See Also
+
+* link:archive-formats.adoc[Archive Formats]
+* link:api-usage.adoc[Library API Usage]
+* link:par2-archives.adoc[PAR2 Error Correction]
+* link:../README.adoc[Main README]

data/readme-docs/format-converter.adoc

@@ -0,0 +1,325 @@
+= Format Converter
+:toc:
+:toclevels: 3
+
+== Purpose
+
+The Format Converter provides seamless conversion between different archive formats, preserving metadata and allowing batch processing operations.
+
+== Supported Conversions
+
+[cols="30,30,40",options="header"]
+|===
+|Source Format |Target Format |Status
+
+|ZIP
+|7z
+|✅ Full support
+
+|7z
+|ZIP
+|✅ Full support
+
+|RAR
+|ZIP/7z
+|🔵 Via extract + re-compress
+
+|TAR
+|ZIP/7z
+|🔵 Via extract + re-compress
+|===
+
+== Basic Conversion
+
+=== Single File Conversion
+
+[source,ruby]
+----
+# Convert ZIP to 7z
+Omnizip::Converter.convert('archive.zip', 'archive.7z')
+
+# Convert 7z to ZIP
+Omnizip::Converter.convert('archive.7z', 'archive.zip')
+----
+
+=== With Options
+
+[source,ruby]
+----
+# Convert with compression settings
+Omnizip::Converter.convert('source.zip', 'target.7z',
+  algorithm: :lzma2,
+  level: 9,
+  solid: true
+)
+
+# Convert and delete source
+Omnizip::Converter.convert('source.zip', 'target.7z',
+  delete_source: true
+)
+
+# Convert with progress tracking
+Omnizip::Converter.convert('large.zip', 'large.7z',
+  progress: ->(pct) { puts "Progress: #{pct}%" }
+)
+----
+
+== Batch Conversion
+
+=== Convert Multiple Archives
+
+[source,ruby]
+----
+# Batch convert all ZIP files to 7z
+sources = Dir.glob('backups/*.zip')
+
+Omnizip::Converter.batch_convert(
+  sources,
+  target_format: :seven_zip,
+  level: 9
+) do |result|
+  puts "Converted: #{result.source} -> #{result.target}"
+  puts "  Time: #{result.duration}s"
+  puts "  Original: #{result.source_size} bytes"
+  puts "  Converted: #{result.target_size} bytes"
+  puts "  Savings: #{result.compression_improvement}%"
+end
+----
+
+== Conversion Strategies
+
+=== ZIP to 7z Strategy
+
+Converts ZIP archives to 7z format with better compression:
+
+[source,ruby]
+----
+strategy = Omnizip::Converter::ZipToSevenZipStrategy.new(
+  'source.zip',
+  'target.7z',
+  algorithm: :lzma2,
+  level: 9,
+  solid: true
+)
+
+result = strategy.convert
+puts "Compression improved by #{result.compression_improvement}%"
+----
+
+=== 7z to ZIP Strategy
+
+Converts 7z archives to ZIP for better compatibility:
+
+[source,ruby]
+----
+strategy = Omnizip::Converter::SevenZipToZipStrategy.new(
+  'source.7z',
+  'target.zip',
+  algorithm: :deflate,
+  level: 6
+)
+
+result = strategy.convert
+----
+
+== Conversion Options
+
+=== Available Options
+
+[source,ruby]
+----
+options = Omnizip::Models::ConversionOptions.new(
+  # Target compression settings
+  algorithm: :lzma2,          # Compression algorithm
+  level: 9,                   # Compression level (1-9)
+  solid: true,                # Solid compression (7z only)
+  
+  # Filters
+  filter: :bcj_x86,          # Preprocessing filter
+  
+  # Behavior options
+  delete_source: false,       # Delete source after conversion
+  overwrite: false,           # Overwrite existing target
+  preserve_timestamps: true,  # Keep original timestamps
+  
+  # Progress tracking
+  progress: nil              # Progress callback
+)
+
+Omnizip::Converter.convert_with_options('source.zip', 'target.7z', options)
+----
+
+== Conversion Results
+
+=== Result Object
+
+Each conversion returns a result object with detailed information:
+
+[source,ruby]
+----
+result = Omnizip::Converter.convert('archive.zip', 'archive.7z')
+
+# Conversion details
+result.success?                    # => true/false
+result.source                      # => 'archive.zip'
+result.target                      # => 'archive.7z'
+result.duration                    # => 2.5 (seconds)
+
+# Size information
+result.source_size                 # => 1048576 (bytes)
+result.target_size                 # => 524288 (bytes)
+result.compression_improvement     # => 50.0 (%)
+
+# Files processed
+result.files_processed             # => 42
+result.files_failed                # => 0
+
+# Error information (if failed)
+result.error_message               # => nil or error string
+----
+
+== Examples
+
+=== Example 1: Optimize Storage with 7z
+
+[source,ruby]
+----
+# Convert all ZIP backups to 7z for better compression
+def optimize_backups(backup_dir)
+  zip_files = Dir.glob("#{backup_dir}/*.zip")
+  total_saved = 0
+  
+  zip_files.each do |zip_file|
+    sevenz_file = zip_file.sub('.zip', '.7z')
+    
+    result = Omnizip::Converter.convert(zip_file, sevenz_file,
+      algorithm: :lzma2,
+      level: 9,
+      solid: true,
+      delete_source: true
+    )
+    
+    saved = result.source_size - result.target_size
+    total_saved += saved
+    
+    puts "#{File.basename(zip_file)}: saved #{saved / 1024}KB"
+  end
+  
+  puts "\nTotal space saved: #{total_saved / (1024 * 1024)}MB"
+end
+
+optimize_backups('backups/')
+----
+
+=== Example 2: Convert to ZIP for Compatibility
+
+[source,ruby]
+----
+# Convert 7z archives to ZIP for compatibility with other systems
+def convert_for_compatibility(source_dir, target_dir)
+  FileUtils.mkdir_p(target_dir)
+  
+  Dir.glob("#{source_dir}/*.7z").each do |sevenz_file|
+    basename = File.basename(sevenz_file, '.7z')
+    zip_file = File.join(target_dir, "#{basename}.zip")
+    
+    puts "Converting: #{basename}"
+    result = Omnizip::Converter.convert(sevenz_file, zip_file,
+      algorithm: :deflate,
+      level: 6
+    )
+    
+    if result.success?
+      puts "  ✓ Success (#{result.files_processed} files)"
+    else
+      puts "  ✗ Failed: #{result.error_message}"
+    end
+  end
+end
+
+convert_for_compatibility('internal/', 'external/')
+----
+
+=== Example 3: Smart Conversion with Analysis
+
+[source,ruby]
+----
+# Analyze and convert only if beneficial
+def smart_convert(source_file)
+  target_file = source_file.sub(/\.(zip|7z)$/, '.optimized.7z')
+  
+  # Try conversion with maximum compression
+  result = Omnizip::Converter.convert(source_file, target_file,
+    algorithm: :lzma2,
+    level: 9,
+    solid: true
+  )
+  
+  # Keep conversion only if it saves significant space
+  improvement = result.compression_improvement
+  
+  if improvement > 10.0
+    # Good savings, delete source
+    File.delete(source_file)
+    File.rename(target_file, source_file.sub(/\.(zip|7z)$/, '.7z'))
+    puts "Optimized: #{improvement.round(1)}% savings"
+  else
+    # Not worth it, delete converted file
+    File.delete(target_file)
+    puts "Skipped: only #{improvement.round(1)}% improvement"
+  end
+end
+----
+
+== Checking Conversion Support
+
+[source,ruby]
+----
+# Check if conversion is supported
+if Omnizip::Converter.supported?('archive.zip', 'archive.7z')
+  puts "ZIP to 7z conversion is supported"
+end
+
+# List available strategies
+Omnizip::Converter.strategies.each do |strategy_class|
+  puts strategy_class.name
+end
+----
+
+== Performance Considerations
+
+=== Conversion Speed
+
+[cols="30,30,40",options="header"]
+|===
+|Conversion |Speed |Notes
+
+|ZIP → 7z (LZMA2)
+|Slow
+|10-15x slower than ZIP extraction alone
+
+|7z → ZIP (Deflate)
+|Medium
+|Faster due to simpler Deflate compression
+
+|With solid compression
+|Very Slow
+|Solid mode requires recompression of entire archive
+
+|With Store (no compression)
+|Fast
+|Just format conversion, minimal processing
+|===
+
+=== Memory Usage
+
+- **Streaming mode**: Processes files one at a time (lower memory)
+- **Batch mode**: May load multiple files in memory
+- **Large archives**: Consider chunked processing for >1GB archives
+
+== See Also
+
+* link:../README.adoc#seven-zip-format[7z Format Documentation]
+* link:../README.adoc#zip-format[ZIP Format Documentation]
+* link:compression-profiles.adoc[Compression Profiles]
+* link:performance-profiler.adoc[Performance Profiler]