omnizip 0.3.35 → 0.3.37

data/docs/guides/advanced-features/progress-tracking.adoc

@@ -7,314 +7,118 @@ grand_parent: Guides
 
 == Purpose
 
-Real-time progress tracking provides visibility into long-running archive operations. Monitor file counts, bytes processed, processing rate, and estimated time remaining.
+Long compression and extraction jobs benefit from progress feedback.
+Omnizip ships a standalone tracker — +Omnizip::Progress+ — with
+percentage, throughput, and ETA calculation. The convenience methods
+do not emit progress themselves: wrap your own loop with a tracker.
 
-== Benefits
+== Basic Tracking
 
-* **User Feedback**: Show progress to users during long operations
-* **Better UX**: Prevent "frozen" appearance during processing
-* **Performance Monitoring**: Track processing rates and bottlenecks
-* **Time Estimation**: Know how long operations will take
-* **Debugging**: Identify which files cause slowdowns
-
-== Progress Callbacks
-
-=== Basic Progress Monitoring
-
-Get updates during compression or extraction:
+Create a tracker, update it as work completes, read live stats:
 
 [source,ruby]
 ----
-Omnizip.compress_directory('large_project/', 'backup.zip') do |progress|
-  puts "Progress: #{progress.percentage}%"
-  puts "Current file: #{progress.current_file}"
-  puts "Files: #{progress.files_processed}/#{progress.total_files}"
-end
-----
-
-=== Detailed Progress Information
+tracker = Omnizip::Progress.track(total_files: 100, total_bytes: 1_000_000)
 
-Access comprehensive progress data:
+tracker.update(files: 10, bytes: 100_000, current_file: "file.txt")
 
-[source,ruby]
-----
-Omnizip.extract_archive('archive.zip', 'output/') do |progress|
-  puts "Files: #{progress.files_processed}/#{progress.total_files}"
-  puts "Bytes: #{progress.bytes_processed}/#{progress.total_bytes}"
-  puts "Speed: #{progress.throughput_mb_s} MB/s"
-  puts "Elapsed: #{progress.elapsed_seconds}s"
-  puts "ETA: #{progress.eta_seconds}s"
-  puts "Percentage: #{progress.percentage}%"
-end
+tracker.percentage       # => 10.0
+tracker.elapsed_seconds
+tracker.eta_seconds
+tracker.eta_formatted    # => "2m 30s"
 ----
 
-== Console Progress Bar
+== Reporters
 
-=== Automatic Progress Bar
+=== Callback Reporter
 
-Enable automatic progress bar for terminal output:
+Yield progress to a block:
 
 [source,ruby]
 ----
-Omnizip.compress_directory(
-  'data/',
-  'backup.zip',
-  progress: :console  # Show progress bar
-)
-
-# Output:
-# [=========>          ] 45% (23/50 files) - 2.5 MB/s - ETA: 12s
-----
-
-=== Custom Progress Bar
-
-Configure progress bar appearance:
-
-[source,ruby]
-----
-options = {
-  progress: :console,
-  progress_format: :detailed,  # or :simple, :verbose
-  update_interval: 0.5,  # Update every 500ms
-  show_rate: true,
-  show_eta: true
-}
-
-Omnizip.compress_directory('files/', 'backup.7z', **options)
-----
-
-== Web Application Integration
-
-=== Rails Real-Time Updates
-
-Send progress updates to the browser via WebSocket:
-
-[source,ruby]
-----
-class ArchiveJob < ApplicationJob
-  def perform(user_id, file_paths)
-    ActionCable.server.broadcast("archive_#{user_id}", {
-      status: 'starting',
-      total_files: file_paths.count
-    })
-
-    Omnizip.compress_files(file_paths, 'output.zip') do |progress|
-      ActionCable.server.broadcast("archive_#{user_id}", {
-        status: 'processing',
-        percentage: progress.percentage,
-        files_done: progress.files_processed,
-        current_file: progress.current_file,
-        eta: progress.eta_seconds
-      })
-    end
-
-    ActionCable.server.broadcast("archive_#{user_id}", {
-      status: 'complete',
-      download_url: archive_url
-    })
-  end
+tracker = Omnizip::Progress.track(total_files: 100,
+                                  total_bytes: 1_000_000) do |progress|
+  puts "#{progress.percentage}% complete"
 end
 ----
 
-=== API Progress Endpoint
+=== Console Reporter
 
-Provide progress via polling endpoint:
+Render a terminal progress bar:
 
 [source,ruby]
 ----
-# Start background job
-def create_archive
-  job_id = SecureRandom.uuid
-
-  Thread.new do
-    Omnizip.compress_directory('data/', 'output.zip') do |progress|
-      # Store progress in Redis/cache
-      Rails.cache.write("archive_progress_#{job_id}", {
-        percentage: progress.percentage,
-        files: progress.files_processed,
-        total: progress.total_files,
-        eta: progress.eta_seconds
-      })
-    end
-  end
-
-  render json: { job_id: job_id }
-end
-
-# Check progress
-def check_progress
-  progress = Rails.cache.read("archive_progress_#{params[:job_id]}")
-  render json: progress
-end
-----
-
-== Progress Metrics
-
-=== Available Metrics
-
-[cols="2,3"]
-|===
-|Metric |Description
-
-|`percentage`
-|Overall completion percentage (0-100)
-
-|`files_processed`
-|Number of files processed so far
-
-|`total_files`
-|Total number of files to process
-
-|`bytes_processed`
-|Bytes processed so far
-
-|`total_bytes`
-|Total bytes to process
-
-|`current_file`
-|Name of file currently being processed
-
-|`throughput_mb_s`
-|Processing speed in MB/second
-
-|`elapsed_seconds`
-|Time elapsed since start
-
-|`eta_seconds`
-|Estimated seconds until completion
-
-|`active_threads`
-|Number of active worker threads (parallel mode)
-|===
-
-=== Computed Properties
-
-[source,ruby]
-----
-progress.percentage          # => 45.2
-progress.files_remaining     # => 27
-progress.bytes_remaining     # => 125829120
-progress.elapsed_formatted   # => "2m 15s"
-progress.eta_formatted       # => "3m 05s"
-progress.completion_time     # => 2024-01-15 14:32:10
+tracker = Omnizip::Progress.track(
+  total_files: 100,
+  total_bytes: 1_000_000,
+  reporter: :console
+)
 ----
 
-== Performance Monitoring
-
-=== Track Processing Rate
+=== Log Reporter
 
-Monitor compression/decompression speed:
+Write updates to a log file (pass the reporter instance):
 
 [source,ruby]
 ----
-rates = []
-
-Omnizip.compress_directory('files/', 'backup.zip') do |progress|
-  rates << progress.throughput_mb_s
-
-  # Alert if speed drops significantly
-  if rates.size > 10 && progress.throughput_mb_s < rates[-10..-1].average * 0.5
-    puts "WARNING: Processing slowed down significantly"
-    puts "Current file: #{progress.current_file}"
-  end
-end
+reporter = Omnizip::Progress::LogReporter.new(
+  log_file: 'compress.log', verbose: true
+)
 
-puts "Average rate: #{rates.average.round(2)} MB/s"
-puts "Peak rate: #{rates.max.round(2)} MB/s"
+tracker = Omnizip::Progress.track(
+  total_files: 100,
+  total_bytes: 1_000_000,
+  reporter: reporter
+)
 ----
 
-=== Identify Slow Files
+== Wrapping a Compression Job
 
-Find files that take longest to process:
+Track a directory compression file by file:
 
 [source,ruby]
 ----
-slow_files = []
+files = Dir.glob('data/**/*').select { |f| File.file?(f) }
+tracker = Omnizip::Progress.track(
+  total_files: files.size,
+  total_bytes: files.sum { |f| File.size(f) },
+  reporter: :console
+)
 
-Omnizip.compress_directory('data/', 'backup.zip') do |progress|
-  if progress.current_file_duration > 5.0  # More than 5 seconds
-    slow_files << {
-      name: progress.current_file,
-      duration: progress.current_file_duration,
-      size: progress.current_file_size
-    }
+Omnizip::Archive.create('backup.zip') do |archive|
+  files.each do |file|
+    archive.add_file(file, file.delete_prefix('data/'))
+    tracker.update(files: 1, bytes: File.size(file), current_file: file)
   end
 end
-
-slow_files.sort_by { |f| -f[:duration] }.first(10).each do |f|
-  puts "#{f[:name]}: #{f[:duration]}s (#{f[:size]} bytes)"
-end
 ----
 
-== Custom Progress Reporters
-
-=== Log File Reporter
+== Web Application Integration
 
-Write progress to log file:
+Broadcast progress from a background job:
 
 [source,ruby]
 ----
-File.open('archive_progress.log', 'w') do |log|
-  Omnizip.compress_directory('data/', 'backup.zip') do |progress|
-    log.puts "[#{Time.now}] #{progress.percentage}% - #{progress.current_file}"
-    log.flush
-  end
-end
-----
-
-=== JSON Progress Reporter
-
-Output machine-readable progress:
+class ArchiveJob < ApplicationJob
+  def perform(user_id, file_paths)
+    tracker = Omnizip::Progress.track(total_files: file_paths.size)
 
-[source,ruby]
-----
-Omnizip.compress_directory('data/', 'backup.zip') do |progress|
-  json = {
-    timestamp: Time.now.iso8601,
-    percentage: progress.percentage,
-    files_done: progress.files_processed,
-    files_total: progress.total_files,
-    current: progress.current_file,
-    rate_mbs: progress.throughput_mb_s,
-    eta_seconds: progress.eta_seconds
-  }
+    file_paths.each_with_index do |path, i|
+      process_file(path)
+      tracker.update(files: 1, current_file: path)
 
-  puts json.to_json
+      ActionCable.server.broadcast("archive_#{user_id}", {
+        status: 'processing',
+        files_done: i + 1,
+        total_files: file_paths.size,
+        percentage: tracker.percentage
+      })
+    end
+  end
 end
 ----
 
-== CLI Progress Display
-
-[source,bash]
-----
-# Default progress bar
-$ omnizip archive create backup.zip files/
-[=========>          ] 45% (230/512 files) - 2.5 MB/s - ETA: 1m 23s
-
-# Verbose progress
-$ omnizip archive create backup.zip files/ --progress verbose
-Processing: src/app.rb (230/512 files, 45%)
-Size: 45 KB -> 12 KB (73% compression)
-Rate: 2.5 MB/s, Elapsed: 1m 12s, ETA: 1m 23s
-
-# Quiet mode (no progress)
-$ omnizip archive create backup.zip files/ --quiet
-
-# JSON progress output
-$ omnizip archive create backup.zip files/ --progress json
-{"percentage":45.0,"files":230,"total":512,"eta":83}
-----
-
-== Best Practices
-
-. **Update Interval**: Don't update too frequently (0.5-1s is good)
-. **Non-Blocking**: Use threads/async for long operations
-. **User Experience**: Always show progress for operations >5 seconds
-. **Error Handling**: Continue tracking even if individual files fail
-. **Resource Cleanup**: Ensure progress callbacks don't leak memory
-
 == See Also
 
-* link:parallel-processing.html[Parallel Processing] - Monitor multi-threaded operations
-* link:streaming.html[Streaming] - Track streaming operations
-* link:../performance-tuning.html[Performance Tuning] - Optimize based on progress data
+* link:streaming.html[Streaming Operations] - Memory-efficient processing
+* link:parallel-processing.html[Parallel Processing] - Multi-core strategies

data/docs/guides/advanced-features/streaming.adoc

@@ -7,47 +7,23 @@ grand_parent: Guides
 
 == Purpose
 
-Streaming operations allow you to process archives without loading entire files into memory. This is essential for handling large archives, web applications, and memory-constrained environments.
-
-== Benefits
-
-* **Memory Efficient**: Process multi-GB archives with minimal memory footprint
-* **Fast Response**: Start processing immediately without waiting for complete download
-* **Scalable**: Handle archives of any size on servers with limited resources
-* **Web-Friendly**: Perfect for Rails applications and API services
-
-== When to Use Streaming
-
-[cols="1,3"]
-|===
-|Scenario |Streaming Benefit
-
-|Large Archives (>100MB)
-|Avoid loading entire archive into memory
-
-|Web Applications
-|Process uploads without temporary files
-
-|API Services
-|Stream responses directly to clients
+== Purpose
 
-|Limited Memory
-|Process on resource-constrained servers
+Streaming operations allow you to process archives without loading entire files into memory. This is essential for handling large archives, web applications, and memory-constrained environments.
 
-|Network Transfers
-|Start processing while downloading
-|===
+Omnizip's streaming classes live under +Omnizip::Zip+ (streaming is a ZIP feature; the 7z and TAR writers accept buffered adds through the link:../creating-archives.html[Archive facade]).
 
 == Creating Archives with Streaming
 
 === Stream to File
 
-Create archives by streaming entries one at a time:
+Create ZIP archives by streaming entries one at a time:
 
 [source,ruby]
 ----
-Omnizip::OutputStream.open('backup.zip', format: :zip) do |stream|
-  # Add files as you go - no need to collect them first
+require 'omnizip/zip/output_stream'
+
+Omnizip::Zip::OutputStream.open('backup.zip') do |stream|
   Dir.glob('logs/*.log').each do |file|
     stream.put_next_entry(File.basename(file))
     File.open(file, 'rb') do |f|
@@ -57,22 +33,31 @@ Omnizip::OutputStream.open('backup.zip', format: :zip) do |stream|
 end
 ----
 
+Per-entry compression method and level are supported:
+
+[source,ruby]
+----
+stream.put_next_entry('data.bin', compression: :zstandard, level: 3)
+----
+
 === Stream to Memory
 
-Create archives in memory for immediate use:
+Create ZIP archives in memory with +Omnizip::Buffer+:
 
 [source,ruby]
 ----
-zip_data = Omnizip::OutputStream.write_buffer(format: :zip) do |stream|
-  stream.put_next_entry('report.txt')
-  stream.write(generate_report)
-
-  stream.put_next_entry('data.json')
-  stream.write(export_data.to_json)
+zip_data = Omnizip::Buffer.create(:zip) do |archive|
+  archive.add('report.txt', generate_report)
+  archive.add('data.json', export_data.to_json)
 end
 
-# Send directly over HTTP
-send_data zip_data.string, filename: 'export.zip'
+send_data zip_data.read, filename: 'export.zip'  # StringIO
+
+# Or straight from a Hash
+zip_data = Omnizip::Buffer.create_from_hash(
+  'report.txt' => generate_report,
+  'data.json' => export_data.to_json
+)
 ----
 
 == Reading Archives with Streaming
@@ -83,11 +68,12 @@ Process each file as it's read from the archive:
 
 [source,ruby]
 ----
-Omnizip::InputStream.open('large-archive.zip') do |stream|
-  while entry = stream.get_next_entry
+require 'omnizip/zip/input_stream'
+
+Omnizip::Zip::InputStream.open('large-archive.zip') do |stream|
+  while (entry = stream.get_next_entry)
     next if entry.directory?
 
-    # Process each file immediately
     case entry.name
     when /\.csv$/
       import_csv_data(stream.read)
@@ -98,21 +84,16 @@ Omnizip::InputStream.open('large-archive.zip') do |stream|
 end
 ----
 
-=== Selective Extraction
+=== Extract to Memory
 
-Extract only specific files without reading the entire archive:
+Read all entries into a Hash without touching the filesystem:
 
 [source,ruby]
 ----
-Omnizip::InputStream.open('backup.zip') do |stream|
-  while entry = stream.get_next_entry
-    # Only extract configuration files
-    if entry.name.start_with?('config/')
-      File.open(entry.name, 'wb') do |f|
-        f.write(stream.read)
-      end
-    end
-  end
+files = Omnizip::Buffer.extract_to_memory(File.binread('backup.zip'))
+
+files.each do |filename, content|
+  process_file(filename, content)
 end
 ----
 
@@ -120,19 +101,15 @@ end
 
 === Rails Upload Processing
 
-Process uploaded archives without temporary files:
-
 [source,ruby]
 ----
 class ArchiveController < ApplicationController
   def upload
     uploaded = params[:archive]
 
-    # Process upload stream directly
-    files = Omnizip::Buffer.extract_all_to_memory(uploaded.read)
+    files = Omnizip::Buffer.extract_to_memory(uploaded.read)
 
     files.each do |filename, content|
-      # Process each file
       process_file(filename, content)
     end
 
@@ -143,70 +120,23 @@ end
 
 === API Response Streaming
 
-Generate and stream archives in API responses:
-
 [source,ruby]
 ----
 get '/export' do
   content_type 'application/zip'
   attachment 'export.zip'
 
-  stream do |out|
-    buffer = Omnizip::OutputStream.write_buffer do |zip|
-      User.find_each do |user|
-        zip.put_next_entry("#{user.id}.json")
-        zip.write(user.to_json)
-      end
+  zip_data = Omnizip::Buffer.create(:zip) do |archive|
+    User.find_each do |user|
+      archive.add("#{user.id}.json", user.to_json)
     end
-
-    out << buffer.string
   end
+
+  zip_data.read
 end
 ----
 
-== Performance Characteristics
-
-[cols="2,1,1,1"]
-|===
-|Operation |Traditional |Streaming |Improvement
-
-|100MB Archive
-|100MB RAM
-|~10MB RAM
-|90% reduction
-
-|1GB Archive
-|1GB RAM
-|~10MB RAM
-|99% reduction
-
-|Web Upload
-|Temp file required
-|No temp file
-|Faster response
-
-|Large Extraction
-|Full parse first
-|Start immediately
-|Better UX
-|===
-
-== Best Practices
-
-. **Use Streaming for Large Files**: Any archive >50MB should use streaming
-. **Process Incrementally**: Don't collect all data before processing
-. **Handle Errors Gracefully**: Stream operations can fail mid-process
-. **Clean Up Resources**: Always use blocks to ensure cleanup
-. **Monitor Memory**: Verify memory stays constant during processing
-
-== Limitations
-
-* Cannot seek backwards in stream (forward-only processing)
-* Some operations require random access (use file-based API for those)
-* Central directory must fit in memory (usually small)
-
 == See Also
 
-* link:parallel-processing.html[Parallel Processing] - Combine with streaming for maximum performance
-* link:progress-tracking.html[Progress Tracking] - Monitor streaming operations
-* link:../../compatibility.html[Compatibility] - Streaming support across formats
+* link:parallel-processing.html[Parallel Processing] - Multi-core strategies
+* link:../creating-archives.html[Creating Archives] - Block-style archive facade

data/docs/guides/compression-algorithms/bzip2.adoc

@@ -59,7 +59,8 @@ Omnizip.compress_file('data.txt', 'data.txt.bz2',
 )
 
 # Create archive with BZip2 compression
-Omnizip::Archive.create('backup.tar.bz2', format: :tar_bzip2) do |archive|
+Omnizip::Archive.create('backup.7z', format: :seven_zip,
+                        algorithm: :bzip2) do |archive|
   archive.add_directory('project/')
 end
 ----

data/docs/guides/compression-algorithms/deflate.adoc

@@ -59,10 +59,8 @@ Deflate is the standard ZIP compression algorithm, offering a good balance betwe
 # Deflate is the default for ZIP
 Omnizip.compress_directory('project/', 'backup.zip')
 
-# Explicit Deflate configuration
+# ZIP entries use Deflate by default
 Omnizip::Archive.create('archive.zip', format: :zip) do |archive|
-  archive.compression = :deflate
-  archive.level = 6  # Default level
   archive.add_directory('files/')
 end
 ----
@@ -250,8 +248,6 @@ Omnizip.compress_directory(
 ----
 # Package for distribution (broad compatibility)
 Omnizip::Archive.create('release-v1.0.zip', format: :zip) do |archive|
-  archive.compression = :deflate
-  archive.level = 9  # Best compression
   archive.add_directory('dist/')
 end
 ----
@@ -265,8 +261,6 @@ Process uploads quickly:
 # Handle user uploads efficiently
 uploaded_files.each do |file|
   Omnizip::Archive.create("#{user_id}-upload.zip", format: :zip) do |archive|
-    archive.compression = :deflate
-    archive.level = 6  # Balanced
     archive.add_data(file.original_filename, file.read)
   end
 end