omnizip 0.3.37 → 0.3.38

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: dbb80e9d06123c538d5823fd0263b38c8a1a386ce0ba107f1aefd6320b2f3618
-  data.tar.gz: 53a147f76a0290e7b8dc211c2040cb090799f196fbc57c24cd4ff44290ef53f1
+  metadata.gz: 67142de350c2f3dd29c473268977799ed088cedb0340932cff6db7bc23ac009e
+  data.tar.gz: c35fbc3353ca0c20f2ec4f86fa39fff67e6ed95822702a18c9b545d35406749c
 SHA512:
-  metadata.gz: 8f1fae0da5256104bc0ce6871ebf6f85ec4e60865d513004a55c30389eca1dd3ba47bc3d94c6c0f419717930933920428eb3755e2a3f237014094012638953b7
-  data.tar.gz: 27eb53ffd94c880bfa429202cd8b9bdc9094e83ccc382a96479a6dd0c8f090997081f41d5b3463c6f63bb7e9d06fff0dc7b44c45425c0bdd85cc503998559763
+  metadata.gz: 7f6b4e88bb0a227f7cff47b06e979891e6d5a57288ae141bd15e57a07abe2cef3c915600242bb05406c4b7a13ef9fdd0ab5f11870e41b65f0a95077191741ceb
+  data.tar.gz: 3e8497c8eb222b4a139f965ec84f1672d8b12ae92e55986df95d8057c452bcace18a75c54a0464ca696776a33256d2e11ea53811c7d73e3ecb1c0eece06d04ff

data/CHANGELOG.md

@@ -7,6 +7,77 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- `.cpio` and `.iso` read routing: `extract_archive`/`list_archive`/
+  `read_from_archive`/`Archive.open` operate on CPIO archives and ISO
+  9660 images through new read-only handlers, mirroring the RAR
+  routing (creation keeps raising truthfully).
+
+### Fixed
+- 7z archives with explicit directory entries (or zero-byte
+  entries) were structurally invalid: the writer never emitted the
+  kEmptyStream/kEmptyFile properties, hardcoded 0x20 attributes
+  over each entry's real attributes, and its kSubStreamsInfo
+  digest/size arrays counted directory entries as substreams. 7-Zip
+  rejects such archives with "Headers Error" (and our own reader
+  crashed extracting directory entries). Verified against 7zz
+  ground truth (`-mhc=off` dumps): kEmptyStream/kEmptyFile carry
+  RAW bit vectors with no all-defined marker — the parser expected
+  that marker everywhere, so it now reads both layouts correctly
+  (which also fixes reading 7-Zip-created archives containing empty
+  files).
+- RAR5 vint encoding was not the spec encoding: multi-byte values
+  (any size >= 128, extra-area sizes, dictionary sizes) were written
+  in a byte-swapped form no reader decodes — archives with more
+  than ~127 bytes of content were corrupt for unrar AND for our own
+  spec-conformant parser. All round-trips now verified against
+  unrar, including encrypted STORE, multi-file, and multi-volume
+  archives (volume flags were the previously documented "no
+  write-side reference" gap: Main archive flags 0x0001/0x0002 and
+  end-of-archive 0x0001 are written per the RAR 5.0 specification).
+- RAR5 file headers: a "mystery vint" not in the spec shifted every
+  field; the compression method was written into the version bits;
+  the file-attribute value chmods extracted files unreadable. Now:
+  spec field order, method in bits 8-10 with dictionary bits
+  11-15, Unix attribute 0o100644.
+- RAR5 encrypted writing now persists the salt/IV as the file
+  encryption extra record (spec type 0x01) — archives it wrote
+  before could not be decrypted by anything (unrar reported "All
+  OK" only because no CRC was stored).
+- RAR5 `:lzma`/`:lzss` compression advertised method bits for a
+  stream that nothing can decode (the encoder is not
+  official-RAR-compatible; `unrar` extracts empty files silently).
+  `Lzss.available?` is now honest, so writers fall back to STORE
+  with a warning; solid mode falls back to independent STORE
+  entries.
+- ISO writing: directory records pointed every file at sector 0
+  (allocated extents were never copied onto the tree nodes); the
+  volume descriptor declared 100 sectors regardless of actual
+  size; `add_directory` entries were not marked as directories;
+  Rock Ridge and Joliet were advertised by default without being
+  implemented (the cloned SVD made 7-Zip read ASCII names as
+  UCS-2); `Reader#extract_all` looked entries up by bare name
+  instead of full path; `Formats::Iso.list` returned the Reader
+  instead of entries. Images now extract byte-identically through
+  7zz, including nested directories.
+- CLI: `omnizip <command> --help` failed with an arity error on
+  every command (Thor does not route `--help` on subcommands by
+  itself). A dispatch hook now prints per-command help.
+- `Omnizip::Formats::Rar::Rar5::MainHeader/FileHeader` were
+  unreferenceable unless another file happened to load header.rb
+  first (missing autoload entries); three Rar5 autoloads pointed at
+  files that never existed.
+
+### Changed
+- Docs: the OLE, RPM, GZIP, RAR5 and API-overview guides now use
+  the real APIs (`Formats::Ole.list/read/info`, `Formats::Rpm.*`,
+  `Gzip.compress_stream`, `Formats::Zip::Reader#read`, correct
+  error class names); fictional methods (`open_stream`, `root`,
+  `extract_to`, `extract_files`, `extract_payload`,
+  `Rar::Rar5::Reader`, scriptlets/signature accessors) removed.
+
+## [0.3.37] - 2026-08-28
+
 ### Added
 - `Omnizip::Archive` — the block-style facade the guides have
   documented since the docs were written (it never existed; ~50
@@ -143,15 +214,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   been pinned to :copy) and replaced the match-encoding TODO with a
   wire-level round-trip verification of the distance-8/length
   pattern through the LZMA2 decoder. No code under lib/ changes.
-
-## [0.3.30] - 2026-08-27
-
-### Verified
-- LZIP interop with the real `lzip` CLI (1.26): our members pass
-  `lzip -t`/`-dc` and the CLI's members decode through
-  `Formats::Lzip` (spec skips where the CLI is absent).
-
-### Changed
 - The remaining lib/ TODO markers are resolved as documented design
   decisions rather than dangling work: RAR5 volume extras and
   EndHeader volume flags (no write-side reference exists;
@@ -165,6 +227,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   the salt/IV header, so archives it writes cannot be decrypted;
   encryption stays read-verified only.
 
+### Verified
+- LZIP interop with the real `lzip` CLI (1.26): our members pass
+  `lzip -t`/`-dc` and the CLI's members decode through
+  `Formats::Lzip` (spec skips where the CLI is absent).
+
 ## [0.3.28] - 2026-08-26
 
 ### Added

data/docs/guides/archive-formats/gzip-format.adoc

@@ -92,10 +92,8 @@ GZIP supports streaming for large files:
 ----
 # Stream compress large file
 File.open('large.bin', 'rb') do |input|
-  Omnizip::Gzip::Writer.open('large.bin.gz') do |output|
-    while chunk = input.read(65536)
-      output.write(chunk)
-    end
+  File.open('large.bin.gz', 'wb') do |output|
+    Omnizip::Formats::Gzip.compress_stream(input, output)
   end
 end
 ----

data/docs/guides/archive-formats/ole-format.adoc

@@ -8,37 +8,13 @@ grand_parent: Guides
 [[ole-format]]
 == Purpose
 
-OLE (Object Linking and Embedding) Compound Document format is used by Microsoft for storing structured data in a single file. Common OLE files include:
-- MSI (Windows Installer packages)
-- DOC (legacy Word documents)
-- XLS (legacy Excel spreadsheets)
-- PPT (legacy PowerPoint presentations)
-- THM (Microsoft Theme files)
-- MSG (Outlook messages)
+OLE (Object Linking and Embedding) Compound Document format is used
+by Microsoft for storing structured data in a single file. Common OLE
+files include MSI (Windows Installer packages), DOC (legacy Word),
+XLS (legacy Excel), PPT (legacy PowerPoint) and THM (theme) files.
 
-OLE uses a FAT-like file system within a single file, supporting up to 4GB of structured data.
-
-== Key Characteristics
-
-[cols="1,3"]
-|===
-|Property |Value
-
-|Max Size
-|2GB (standard), 4GB (with 4KB sectors)
-
-|Sector Sizes
-|512 bytes (standard), 4KB (large)
-
-|Streams
-|Multiple streams and sub-storages
-
-|Encryption
-|Optional (document-specific)
-
-|Best For
-|Windows compound files, legacy Office documents, MSI installers
-|===
+The API is stream-oriented: open the compound document, list
+streams and storages by path, and read the streams you need.
 
 == Basic Usage
 
@@ -46,189 +22,104 @@ OLE uses a FAT-like file system within a single file, supporting up to 4GB of st
 
 [source,ruby]
 ----
-# Open OLE compound document
-ole = Omnizip::Ole.open('document.doc')
-
-# Access root storage
-root = ole.root
-puts "Root storage: #{root.name}"
-
-# List streams and storages
-root.each do |entry|
-  puts "#{entry.name} (#{entry.size} bytes)" if entry.file?
-  puts "#{entry.name}/" if entry.directory?
+# Open OLE compound document (block form closes automatically)
+Omnizip::Formats::Ole.open('document.doc') do |ole|
+  puts ole.list.inspect
 end
 
+# Non-block form: close explicitly
+ole = Omnizip::Formats::Ole.open('document.doc')
 ole.close
 ----
 
+=== List Entries
+
+[source,ruby]
+----
+# Top-level streams and storages
+Omnizip::Formats::Ole.list('document.doc')
+
+# Inside a nested storage
+Omnizip::Formats::Ole.list('document.doc', '/ObjectPool')
+----
+
 === Read Stream Data
 
 [source,ruby]
 ----
-# Read a specific stream
-ole = Omnizip::Ole.open('document.doc')
-
-# Access \x01SummaryInformation stream (document metadata)
-summary = ole.open_stream("\x01SummaryInformation")
-if summary
-  data = summary.read
-  puts "Summary size: #{data.size} bytes"
-end
+# \x01SummaryInformation holds document metadata
+summary = Omnizip::Formats::Ole.read('document.doc',
+                                     "\x01SummaryInformation")
+puts "Summary size: #{summary.bytesize} bytes"
 
-# Access WordDocument stream (main content)
-word_stream = ole.open_stream("WordDocument")
-if word_stream
-  content = word_stream.read
-  puts "Document content size: #{content.size} bytes"
-end
+# WordDocument stream (main content)
+content = Omnizip::Formats::Ole.read('document.doc', 'WordDocument')
+puts "Document content size: #{content.bytesize} bytes"
 
-ole.close
+# Write a stream to disk
+File.binwrite('workbook.bin',
+              Omnizip::Formats::Ole.read('spreadsheet.xls', 'Workbook'))
 ----
 
-=== Extract All Files
+=== Entry Introspection
 
 [source,ruby]
 ----
-# Extract all streams from OLE to files
-Omnizip::Ole.open('document.doc') do |ole|
-  ole.extract_to('/output/directory/')
-end
+info = Omnizip::Formats::Ole.info('document.doc', 'WordDocument')
+puts info.inspect
 
-# Extract specific stream
-Omnizip::Ole.open('spreadsheet.xls') do |ole|
-  workbook = ole.open_stream('Workbook')
-  File.binwrite('workbook.bin', workbook.read)
-end
+Omnizip::Formats::Ole.exist?('document.doc', 'WordDocument') # => true
+Omnizip::Formats::Ole.file?('document.doc', 'WordDocument')  # => true
+Omnizip::Formats::Ole.directory?('document.doc', 'ObjectPool') # => true
 ----
 
 == MSI Package Inspection
 
-MSI files are OLE compound documents containing Windows Installer data:
+MSI files are OLE compound documents; the installer tables live in
+storages and streams you can walk with +list+:
 
 [source,ruby]
 ----
-# Open MSI package
-msi = Omnizip::Ole.open('installer.msi')
-
-# List MSI tables (storages)
-tables = msi.root.select { |e| e.directory? }
-puts "MSI Tables:"
-tables.each { |t| puts "  #{t.name}" }
-
-# Access \x05SummaryInformation
-summary = msi.open_stream("\x05SummaryInformation")
-if summary
-  puts "Summary information found"
+Omnizip::Formats::Ole.open('installer.msi') do |msi|
+  msi.list('/').each { |entry| puts entry }
+  summary = msi.read("\x05SummaryInformation") rescue nil
+  tables = msi.read('_Tables') rescue nil
+  puts "Tables stream size: #{tables&.bytesize}" if tables
 end
-
-# Access _Tables stream (list of all tables)
-tables_stream = msi.open_stream("_Tables")
-if tables_stream
-  puts "Tables stream size: #{tables_stream.size}"
-end
-
-msi.close
 ----
 
 == Storage Structure
 
 OLE files have a hierarchical structure like a file system:
 
-[source]
 ----
 Root Storage/
 ├── \x01SummaryInformation     (document metadata)
-├── \x01DocumentSummaryInformation
 ├── WordDocument              (main content)
-├── 1Table                    (data tables)
 ├── Data/
-│   ├── Mso1Table
-│   └── Mso0Table
+│   └── Mso1Table
 └── ObjectPool/
-    └── _1000000000/
-        └── Ole10Native
-----
-
-[source,ruby]
-----
-# Navigate storage hierarchy
-ole = Omnizip::Ole.open('document.doc')
-
-# Access nested storage
-pool = ole.root.find("ObjectPool")
-if pool
-  pool.each do |entry|
-    puts "ObjectPool/#{entry.name}"
-  end
-end
-
-ole.close
-----
-
-== Summary Information
-
-OLE documents contain standard metadata streams:
-
-[source,ruby]
+    └── Ole10Native
 ----
-# Read summary information
-ole = Omnizip::Ole.open('document.doc')
-
-# Access summary stream
-summary_stream = ole.open_stream("\x01SummaryInformation")
-if summary_stream
-  # Summary information is in a specific binary format
-  # Omnizip provides helpers to decode it
-  summary = ole.summary_information
-  if summary
-    puts "Title: #{summary.title}"
-    puts "Author: #{summary.author}"
-    puts "Subject: #{summary.subject}"
-    puts "Keywords: #{summary.keywords}"
-    puts "Created: #{summary.create_time}"
-    puts "Modified: #{summary.last_save_time}"
-  end
-end
 
-ole.close
-----
+Stream paths use `/` separators (e.g. +"ObjectPool/Ole10Native"+).
 
 == File Format Detection
 
-OLE files can be identified by their signature:
+OLE files carry the signature +D0 CF 11 E0 A1 B1 1A E1+ in the first
+8 bytes:
 
 [source,ruby]
 ----
-# Check if file is OLE
-File.open('document.doc', 'rb') do |f|
-  header = f.read(8)
-  if Omnizip::Ole.ole_signature?(header)
-    puts "This is an OLE compound document"
-  end
-end
-
-# Get OLE file type
-ole = Omnizip::Ole.open('document.doc')
-case ole.file_type
-when :word_document
-  puts "Microsoft Word document"
-when :excel_spreadsheet
-  puts "Microsoft Excel spreadsheet"
-when :powerpoint
-  puts "Microsoft PowerPoint presentation"
-when :msi_package
-  puts "Windows Installer package"
-else
-  puts "Unknown OLE type: #{ole.file_type}"
+File.open('document.doc', "rb") do |f|
+  magic = f.read(8)
+  ole = magic == "\xD0\xCF\x11\xE0\xA1\xB1\x1A\xE1".b
+  puts(ole ? "OLE compound document" : "not OLE")
 end
-ole.close
 ----
 
 == Supported File Types
 
-Omnizip supports reading the following OLE-based formats:
-
 [cols="2,1,1"]
 |===
 |Format |Extension |Read Support
@@ -249,67 +140,11 @@ Omnizip supports reading the following OLE-based formats:
 |.msi
 |Yes
 
-|Microsoft Theme
-|.thm
-|Yes
-
-|Outlook Message
-|.msg
-|Yes
-
 |Generic OLE
 |.ole
 |Yes
 |===
 
-== Low-Level Access
-
-For advanced use cases, you can access the OLE internals directly:
-
-[source,ruby]
-----
-# Access BAT (Block Allocation Table)
-ole = Omnizip::Ole.open('document.doc')
-
-# Get sector size
-puts "Sector size: #{ole.sector_size}"
-
-# Access BAT entries
-bat = ole.bat
-puts "BAT entries: #{bat.size}"
-
-# Access SBAT (Small Block Allocation Table)
-sbat = ole.sbat
-puts "SBAT entries: #{sbat.size}" if sbat
-
-# Access directory entries
-ole.root.each do |entry|
-  puts "#{entry.name}: starting_sector=#{entry.starting_sector}, size=#{entry.size}"
-end
-
-ole.close
-----
-
-== Performance Considerations
-
-OLE files can be large. For efficient reading:
-
-[source,ruby]
-----
-# Stream read large OLE files efficiently
-Omnizip::Ole.open('large.msi') do |ole|
-  # Only read streams you need
-  summary = ole.open_stream("\x05SummaryInformation")
-  if summary
-    # Read in chunks for large streams
-    chunk_size = 65536
-    while chunk = summary.read(chunk_size)
-      process(chunk)
-    end
-  end
-end
-----
-
 == See Also
 
 * link:seven-zip-format.html[7z Format] - Can contain OLE files

data/docs/guides/archive-formats/rar5.adoc

@@ -493,24 +493,24 @@ volumes.each { |v| puts "  - #{File.basename(v)}" }
 
 [source,ruby]
 ----
-# Convert ZIP to RAR5
-def zip_to_rar5(zip_file, rar_file)
-  zip = Omnizip::Formats::Zip::Reader.new(zip_file)
-
-  Omnizip::Formats::Rar::Rar5::Writer.new(rar_file,
-    compression: :lzma,
-    level: 5
-  ) do |writer|
-    zip.each_entry do |entry|
-      content = zip.read_entry(entry)
-      writer.add_data(entry.name, content)
-    end
+# Convert ZIP to RAR5 (extract first: the RAR5 writer adds files
+# from disk)
+def zip_to_rar5(zip_file, rar_file, workdir)
+  require "fileutils"
+
+  # Reader#read parses the central directory and returns self
+  zip = Omnizip::Formats::Zip::Reader.new(zip_file).read
+  zip.extract_all(workdir)
+
+  writer = Omnizip::Formats::Rar::Rar5::Writer.new(rar_file,
+                                                  compression: :store)
+  Dir.glob(File.join(workdir, "**", "*")).each do |path|
+    writer.add_file(path, path.delete_prefix("#{workdir}/")) if File.file?(path)
   end
-
-  zip.close
+  writer.write
 end
 
-zip_to_rar5('archive.zip', 'archive.rar')
+zip_to_rar5('archive.zip', 'archive.rar', 'workdir')
 ----
 
 === See Also