zip_kit 6.3.5 → 6.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ci.yml +14 -0
- data/CHANGELOG.md +9 -0
- data/README.md +19 -0
- data/Rakefile +8 -0
- data/lib/zip_kit/block_deflate.rb +1 -1
- data/lib/zip_kit/block_write.rb +12 -5
- data/lib/zip_kit/rack_chunked_body.rb +1 -1
- data/lib/zip_kit/size_estimator.rb +10 -0
- data/lib/zip_kit/streamer/entry.rb +1 -1
- data/lib/zip_kit/streamer.rb +154 -13
- data/lib/zip_kit/version.rb +1 -1
- data/lib/zip_kit/zip_writer.rb +14 -6
- data/rbi/zip_kit.rbi +121 -13
- data/rbi/zip_kit.rbs +91 -5
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 67a468ca21123613319181955fd418268f661dc8beaee1c5eab2054395759478
|
|
4
|
+
data.tar.gz: 33047d74b3e2caaa47ce7ef1868bb5b2dba260e3dc97e116fa0b7fba4c6ec07d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 219f3a08a9413835762ee7af7108324e1d2819f3a4a4e4d35c1451b2ad60453f09b8e5783070c879fa644b87199c4e825d40978894489ded72f90ddc99b73a79
|
|
7
|
+
data.tar.gz: 532ca543f7dc346385336ba11e876f14667825d5ef67b6bd1b308ef91c1306591a203adf638894b8a720e1d0abeed0af9ac66ff941dc5b41229261719d47fd21
|
data/.github/workflows/ci.yml
CHANGED
|
@@ -37,6 +37,20 @@ jobs:
|
|
|
37
37
|
RUBYOPT: "--enable=frozen-string-literal --debug=frozen-string-literal"
|
|
38
38
|
run: "bundle exec rspec --backtrace --fail-fast"
|
|
39
39
|
|
|
40
|
+
test_ractors:
|
|
41
|
+
name: "Ractor tests (Ruby 4.0)"
|
|
42
|
+
runs-on: ubuntu-22.04
|
|
43
|
+
steps:
|
|
44
|
+
- name: Checkout
|
|
45
|
+
uses: actions/checkout@v4
|
|
46
|
+
- name: Setup Ruby
|
|
47
|
+
uses: ruby/setup-ruby@v1
|
|
48
|
+
with:
|
|
49
|
+
ruby-version: '4.0'
|
|
50
|
+
bundler-cache: true
|
|
51
|
+
- name: "Tests"
|
|
52
|
+
run: "bundle exec rspec --backtrace --fail-fast spec/zip_kit/ractor_spec.rb"
|
|
53
|
+
|
|
40
54
|
lint_baseline_ruby: # We need to use syntax appropriate for the minimum supported Ruby version
|
|
41
55
|
name: Lint (Ruby 2.6 syntax)
|
|
42
56
|
runs-on: ubuntu-22.04
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
## Unreleased
|
|
2
2
|
|
|
3
|
+
* Add `Streamer.new(io, ocf: true)` for writing EPUB (OCF) and OpenDocument containers. The first entry must then be a stored `mimetype` file with its sizes in the local header, and `Streamer::OCFViolation` is raised otherwise. That entry is written without the extended timestamp extra field, which those formats forbid - before, they could not be produced with ZipKit at all. File names get checked against the OCF rules as well: valid UTF-8, no forbidden characters, segments of at most 255 bytes not ending with a `.`, and no two names in a directory which only differ by case or Unicode normalization.
|
|
4
|
+
* Add `Streamer#write_mimetype_file` and `SizeEstimator#add_mimetype_entry`
|
|
5
|
+
* Add the `extended_timestamp:` keyword argument to `ZipWriter#write_local_file_header` and `ZipWriter#write_central_directory_file_header`
|
|
6
|
+
|
|
7
|
+
## 6.3.6
|
|
8
|
+
|
|
9
|
+
* Fix the end of central directory record counting entries discarded by `rollback!`, which made archives with a rolled back entry unreadable for readers which check the entry count (including `ZipKit::FileReader`).
|
|
10
|
+
* Make ZipKit usable from within Ractors: freeze the computed String constants in `ZipWriter`, `BlockDeflate` and `RackChunkedBody`, and stop defining the raising `BlockWrite` methods with `define_method`. Ractor tests run on Ruby 4.0 in CI.
|
|
11
|
+
|
|
3
12
|
## 6.3.5
|
|
4
13
|
|
|
5
14
|
* Make `WriteBuffer#<<` much faster for lots of tiny writes, like the XML fragments written by libraries such as caxlsx. Strings are no longer copied with `String#b` before being appended: on Ruby 3.4+ `String#append_as_bytes` is used, on older Rubies the string is converted only if Ruby refuses to append it. Mixing binary strings and non-ASCII strings still works, and the writable still always receives binary strings.
|
data/README.md
CHANGED
|
@@ -228,6 +228,25 @@ end
|
|
|
228
228
|
Check out the `examples/` directory at the root of the project. This will give you a good idea
|
|
229
229
|
of various use cases the library supports.
|
|
230
230
|
|
|
231
|
+
### Writing EPUB and OpenDocument files
|
|
232
|
+
|
|
233
|
+
EPUB books (OCF containers) and OpenDocument files are ZIP archives which must start with a `mimetype`
|
|
234
|
+
entry that is stored, has its sizes in the local header and has no extra fields - readers find the media type
|
|
235
|
+
at byte 38 of the file. Pass `ocf: true` to make the Streamer enforce that, and write the entry with `write_mimetype_file`:
|
|
236
|
+
|
|
237
|
+
```ruby
|
|
238
|
+
ZipKit::Streamer.open(epub_file, ocf: true) do |zip|
|
|
239
|
+
zip.write_mimetype_file("application/epub+zip")
|
|
240
|
+
zip.write_file("META-INF/container.xml") { |sink| sink << container_xml }
|
|
241
|
+
zip.write_file("OEBPS/package.opf") { |sink| sink << package_opf }
|
|
242
|
+
end
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
If the first entry would make the archive non-conformant, `ZipKit::Streamer::OCFViolation` gets raised
|
|
246
|
+
before anything is written. The same happens for file names those formats do not allow: names which are not
|
|
247
|
+
UTF-8, contain characters like `:`, `?` or `*`, have a segment longer than 255 bytes or ending with a `.`,
|
|
248
|
+
or which only differ from another name in the same directory by case or Unicode normalization.
|
|
249
|
+
|
|
231
250
|
### Computing the CRC32 value of a large file
|
|
232
251
|
|
|
233
252
|
`BlockCRC32` computes the CRC32 checksum of an IO in a streaming fashion.
|
data/Rakefile
CHANGED
|
@@ -16,6 +16,14 @@ RSpec::Core::RakeTask.new(:spec)
|
|
|
16
16
|
task :generate_typedefs do
|
|
17
17
|
`bundle exec sord rbi/zip_kit.rbi`
|
|
18
18
|
`bundle exec sord rbi/zip_kit.rbs`
|
|
19
|
+
|
|
20
|
+
# Sord inlines the VERSION literal, which would make every version bump produce a typedef diff
|
|
21
|
+
rbi_path = "rbi/zip_kit.rbi"
|
|
22
|
+
rbi = File.read(rbi_path).sub(/^(\s*)VERSION = T\.let\(.+\)$/, '\1VERSION = T.let(T.unsafe(nil), String)')
|
|
23
|
+
File.write(rbi_path, rbi)
|
|
24
|
+
rbs_path = "rbi/zip_kit.rbs"
|
|
25
|
+
rbs = File.read(rbs_path).sub(/^(\s*)VERSION: untyped$/, '\1VERSION: String')
|
|
26
|
+
File.write(rbs_path, rbs)
|
|
19
27
|
end
|
|
20
28
|
|
|
21
29
|
task default: [:spec, :standard, :generate_typedefs]
|
|
@@ -47,7 +47,7 @@
|
|
|
47
47
|
|
|
48
48
|
class ZipKit::BlockDeflate
|
|
49
49
|
DEFAULT_BLOCKSIZE = 1_024 * 1024 * 5
|
|
50
|
-
END_MARKER = [3, 0].pack("C*")
|
|
50
|
+
END_MARKER = [3, 0].pack("C*").freeze
|
|
51
51
|
# Zlib::NO_COMPRESSION..
|
|
52
52
|
VALID_COMPRESSIONS = (Zlib::DEFAULT_COMPRESSION..Zlib::BEST_COMPRESSION).to_a.freeze
|
|
53
53
|
# Write the end marker (\x3\x0) to the given IO.
|
data/lib/zip_kit/block_write.rb
CHANGED
|
@@ -27,11 +27,18 @@ class ZipKit::BlockWrite
|
|
|
27
27
|
@block = block
|
|
28
28
|
end
|
|
29
29
|
|
|
30
|
-
# Make sure those methods raise outright
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
30
|
+
# Make sure those methods raise outright. These are not created with define_method,
|
|
31
|
+
# because methods defined with a block can't be called from a non-main Ractor
|
|
32
|
+
def seek(*)
|
|
33
|
+
raise "seek not supported - this IO adapter is non-rewindable"
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def pos=(*)
|
|
37
|
+
raise "pos= not supported - this IO adapter is non-rewindable"
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def to_s(*)
|
|
41
|
+
raise "to_s not supported - this IO adapter is non-rewindable"
|
|
35
42
|
end
|
|
36
43
|
|
|
37
44
|
# Sends a string through to the block stored in the BlockWrite.
|
|
@@ -81,6 +81,16 @@ class ZipKit::SizeEstimator
|
|
|
81
81
|
self
|
|
82
82
|
end
|
|
83
83
|
|
|
84
|
+
# Add the `mimetype` entry an EPUB (OCF) or OpenDocument container starts with,
|
|
85
|
+
# see {Streamer#write_mimetype_file}.
|
|
86
|
+
#
|
|
87
|
+
# @param media_type [String] the media type, like "application/epub+zip"
|
|
88
|
+
# @return self
|
|
89
|
+
def add_mimetype_entry(media_type:)
|
|
90
|
+
@streamer.write_mimetype_file(media_type)
|
|
91
|
+
self
|
|
92
|
+
end
|
|
93
|
+
|
|
84
94
|
# Add an empty directory to the archive.
|
|
85
95
|
#
|
|
86
96
|
# @param dirname [String] the name of the directory
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
# Normally you will not have to use this class directly
|
|
5
5
|
class ZipKit::Streamer::Entry < Struct.new(:filename, :crc32, :compressed_size,
|
|
6
6
|
:uncompressed_size, :storage_mode, :mtime,
|
|
7
|
-
:use_data_descriptor, :local_header_offset, :bytes_used_for_local_header, :bytes_used_for_data_descriptor, :unix_permissions)
|
|
7
|
+
:use_data_descriptor, :local_header_offset, :bytes_used_for_local_header, :bytes_used_for_data_descriptor, :unix_permissions, :extended_timestamp)
|
|
8
8
|
def initialize(*)
|
|
9
9
|
super
|
|
10
10
|
filename.force_encoding(Encoding::UTF_8)
|
data/lib/zip_kit/streamer.rb
CHANGED
|
@@ -85,6 +85,22 @@ require "set"
|
|
|
85
85
|
# zip.close
|
|
86
86
|
#
|
|
87
87
|
# Calling {Streamer#close} **will not** call `#close` on the underlying IO object.
|
|
88
|
+
#
|
|
89
|
+
# ## Writing EPUB and OpenDocument containers
|
|
90
|
+
#
|
|
91
|
+
# EPUB (via its Open Container Format, OCF) and OpenDocument are ZIP files with an extra rule: the
|
|
92
|
+
# first entry must be a file called `mimetype`, stored without compression and with no extra fields
|
|
93
|
+
# in its local header. Readers check this by looking at the raw bytes at fixed offsets: the media type
|
|
94
|
+
# must start at byte 38 of the file. Pass `ocf: true` and write the `mimetype` file first:
|
|
95
|
+
#
|
|
96
|
+
# ZipKit::Streamer.open(epub_file, ocf: true) do |zip|
|
|
97
|
+
# zip.write_mimetype_file("application/epub+zip")
|
|
98
|
+
# zip.write_file("META-INF/container.xml") { |sink| sink << container_xml }
|
|
99
|
+
# ...
|
|
100
|
+
# end
|
|
101
|
+
#
|
|
102
|
+
# With `ocf: true` the Streamer will raise {OCFViolation} if the first entry it is asked to write
|
|
103
|
+
# would make the archive non-conformant, or if a file name is not allowed in such a container.
|
|
88
104
|
class ZipKit::Streamer
|
|
89
105
|
autoload :DeflatedWriter, File.dirname(__FILE__) + "/streamer/deflated_writer.rb"
|
|
90
106
|
autoload :Writable, File.dirname(__FILE__) + "/streamer/writable.rb"
|
|
@@ -99,12 +115,28 @@ class ZipKit::Streamer
|
|
|
99
115
|
DEFLATED = 8
|
|
100
116
|
|
|
101
117
|
EntryBodySizeMismatch = Class.new(StandardError)
|
|
118
|
+
OCFViolation = Class.new(StandardError)
|
|
102
119
|
InvalidOutput = Class.new(ArgumentError)
|
|
103
120
|
Overflow = Class.new(StandardError)
|
|
104
121
|
UnknownMode = Class.new(StandardError)
|
|
105
122
|
OffsetOutOfSync = Class.new(StandardError)
|
|
106
123
|
|
|
107
|
-
|
|
124
|
+
# Characters which may not appear in file names inside an OCF container, see
|
|
125
|
+
# https://www.w3.org/TR/epub-33/#sec-container-filenames - a trailing "." is not allowed either
|
|
126
|
+
OCF_FORBIDDEN_CHARACTERS = begin
|
|
127
|
+
ranges = [0x22, 0x2A, 0x3A, 0x3C, 0x3E, 0x3F, 0x5C, 0x7C].map { |c| [c, c] }
|
|
128
|
+
ranges << [0x00, 0x1F] # C0
|
|
129
|
+
ranges << [0x7F, 0x9F] # DEL and C1
|
|
130
|
+
ranges << [0xE000, 0xF8FF] # Private Use Area
|
|
131
|
+
ranges << [0xFDD0, 0xFDEF] # noncharacters
|
|
132
|
+
ranges << [0xFFF0, 0xFFFF] # Specials, includes the U+FFFE and U+FFFF noncharacters
|
|
133
|
+
ranges += (1..14).map { |plane| [plane * 0x10000 + 0xFFFE, plane * 0x10000 + 0xFFFF] } # noncharacters
|
|
134
|
+
ranges << [0xF0000, 0x10FFFF] # Supplementary Private Use Areas
|
|
135
|
+
Regexp.new("[" + ranges.map { |from, to| format("\\u{%X}-\\u{%X}", from, to) }.join + "]").freeze
|
|
136
|
+
end
|
|
137
|
+
OCF_MAX_FILENAME_BYTES = 255
|
|
138
|
+
|
|
139
|
+
private_constant :STORED, :DEFLATED, :OCF_FORBIDDEN_CHARACTERS, :OCF_MAX_FILENAME_BYTES
|
|
108
140
|
|
|
109
141
|
# Creates a new Streamer on top of the given IO-ish object and yields it. Once the given block
|
|
110
142
|
# returns, the Streamer will have it's `close` method called, which will write out the central
|
|
@@ -127,7 +159,14 @@ class ZipKit::Streamer
|
|
|
127
159
|
# @param auto_rename_duplicate_filenames[Boolean] whether duplicate filenames, when encountered,
|
|
128
160
|
# should be suffixed with (1), (2) etc. Default value is `false` - if
|
|
129
161
|
# dupliate names are used an exception will be raised
|
|
130
|
-
|
|
162
|
+
# @param ocf[Boolean] whether the archive must be an OCF container (EPUB) or an OpenDocument package.
|
|
163
|
+
# When set, the first entry must be a stored `mimetype` file without a data descriptor - use
|
|
164
|
+
# {#write_mimetype_file} for it - and {OCFViolation} gets raised otherwise. That entry will be
|
|
165
|
+
# written without the extended timestamp extra field, which those formats forbid. All file names
|
|
166
|
+
# get checked against the OCF rules too: valid UTF-8, no forbidden characters, no names longer
|
|
167
|
+
# than 255 bytes or ending with a ".", and no two names in a directory which only differ by case
|
|
168
|
+
# or Unicode normalization.
|
|
169
|
+
def initialize(writable, writer: create_writer, auto_rename_duplicate_filenames: false, ocf: false)
|
|
131
170
|
raise InvalidOutput, "The writable must respond to #<< or #write" unless writable.respond_to?(:<<) || writable.respond_to?(:write)
|
|
132
171
|
|
|
133
172
|
@out = ZipKit::WriteAndTell.new(writable)
|
|
@@ -135,6 +174,8 @@ class ZipKit::Streamer
|
|
|
135
174
|
@path_set = ZipKit::PathSet.new
|
|
136
175
|
@writer = writer
|
|
137
176
|
@dedupe_filenames = auto_rename_duplicate_filenames
|
|
177
|
+
@ocf = ocf
|
|
178
|
+
@ocf_folded_paths = {}
|
|
138
179
|
end
|
|
139
180
|
|
|
140
181
|
# Writes a part of a zip entry body (actual binary data of the entry) into the output stream.
|
|
@@ -211,6 +252,31 @@ class ZipKit::Streamer
|
|
|
211
252
|
@out.tell
|
|
212
253
|
end
|
|
213
254
|
|
|
255
|
+
# Writes the `mimetype` file which EPUB (OCF) and OpenDocument containers have to start with.
|
|
256
|
+
# The entry is stored, has no data descriptor and no extra fields, so that the media type
|
|
257
|
+
# ends up at byte 38 of the archive if this is the first entry.
|
|
258
|
+
#
|
|
259
|
+
# @param media_type[String] the media type, like "application/epub+zip"
|
|
260
|
+
# @param modification_time [Time] the modification time of the file in the archive
|
|
261
|
+
# @return [Integer] the offset the output IO is at after writing the entry
|
|
262
|
+
def write_mimetype_file(media_type, modification_time: Time.now.utc)
|
|
263
|
+
unless media_type.ascii_only? && media_type.match?(/\A[[:graph:]]+\z/)
|
|
264
|
+
raise OCFViolation, "The media type must be ASCII without whitespace or padding, but was #{media_type.inspect}"
|
|
265
|
+
end
|
|
266
|
+
media_type = media_type.b
|
|
267
|
+
add_file_and_write_local_header(filename: "mimetype",
|
|
268
|
+
modification_time: modification_time,
|
|
269
|
+
crc32: Zlib.crc32(media_type),
|
|
270
|
+
storage_mode: STORED,
|
|
271
|
+
compressed_size: media_type.bytesize,
|
|
272
|
+
uncompressed_size: media_type.bytesize,
|
|
273
|
+
unix_permissions: nil,
|
|
274
|
+
use_data_descriptor: false,
|
|
275
|
+
extended_timestamp: false)
|
|
276
|
+
@out << media_type
|
|
277
|
+
@out.tell
|
|
278
|
+
end
|
|
279
|
+
|
|
214
280
|
# Adds an empty directory to the archive with a size of 0 and permissions of 755.
|
|
215
281
|
#
|
|
216
282
|
# @param dirname [String] the name of the directory in the archive
|
|
@@ -412,11 +478,10 @@ class ZipKit::Streamer
|
|
|
412
478
|
# Record the central directory offset, so that it can be written into the EOCD record
|
|
413
479
|
cdir_starts_at = @out.tell
|
|
414
480
|
|
|
415
|
-
# Write out the central directory entries, one for each file
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
481
|
+
# Write out the central directory entries, one for each file.
|
|
482
|
+
# Skip fillers which are standing in for broken/incomplete files
|
|
483
|
+
entries = @files.reject(&:filler?)
|
|
484
|
+
entries.each do |entry|
|
|
420
485
|
@writer.write_central_directory_file_header(io: @out,
|
|
421
486
|
local_file_header_location: entry.local_header_offset,
|
|
422
487
|
gp_flags: entry.gp_flags,
|
|
@@ -426,7 +491,8 @@ class ZipKit::Streamer
|
|
|
426
491
|
mtime: entry.mtime,
|
|
427
492
|
crc32: entry.crc32,
|
|
428
493
|
filename: entry.filename,
|
|
429
|
-
unix_permissions: entry.unix_permissions
|
|
494
|
+
unix_permissions: entry.unix_permissions,
|
|
495
|
+
extended_timestamp: entry.extended_timestamp)
|
|
430
496
|
end
|
|
431
497
|
|
|
432
498
|
# Record the central directory size, for the EOCDR
|
|
@@ -436,11 +502,12 @@ class ZipKit::Streamer
|
|
|
436
502
|
@writer.write_end_of_central_directory(io: @out,
|
|
437
503
|
start_of_central_directory_location: cdir_starts_at,
|
|
438
504
|
central_directory_size: cdir_size,
|
|
439
|
-
num_files_in_archive:
|
|
505
|
+
num_files_in_archive: entries.length)
|
|
440
506
|
|
|
441
507
|
# Clear the files so that GC will not have to trace all the way to here to deallocate them
|
|
442
508
|
@files.clear
|
|
443
509
|
@path_set.clear
|
|
510
|
+
@ocf_folded_paths.clear
|
|
444
511
|
|
|
445
512
|
# and return the final offset
|
|
446
513
|
@out.tell
|
|
@@ -510,8 +577,11 @@ class ZipKit::Streamer
|
|
|
510
577
|
|
|
511
578
|
# Recreate the path set from remaining entries (PathSet does not support cheap deletes yet)
|
|
512
579
|
@path_set.clear
|
|
580
|
+
@ocf_folded_paths.clear
|
|
513
581
|
@files.each do |e|
|
|
514
|
-
|
|
582
|
+
next if e.filler?
|
|
583
|
+
@path_set.add_directory_or_file_path(e.filename)
|
|
584
|
+
register_ocf_folded_path(e.filename) if @ocf
|
|
515
585
|
end
|
|
516
586
|
|
|
517
587
|
# Create filler for the truncated or unusable local file entry that did get written into the output
|
|
@@ -573,7 +643,8 @@ class ZipKit::Streamer
|
|
|
573
643
|
compressed_size:,
|
|
574
644
|
uncompressed_size:,
|
|
575
645
|
use_data_descriptor:,
|
|
576
|
-
unix_permissions
|
|
646
|
+
unix_permissions:,
|
|
647
|
+
extended_timestamp: true
|
|
577
648
|
)
|
|
578
649
|
# Set state needed for proper rollback later. If write_local_file_header
|
|
579
650
|
# does manage to write _some_ bytes, but fails later (we write in tiny bits sometimes)
|
|
@@ -586,10 +657,21 @@ class ZipKit::Streamer
|
|
|
586
657
|
raise UnknownMode, "Unknown compression mode #{storage_mode}" unless [STORED, DEFLATED].include?(storage_mode)
|
|
587
658
|
raise Overflow, "Filename is too long" if filename.bytesize > 0xFFFF
|
|
588
659
|
|
|
660
|
+
# Going by the offset and not by the entries, since a rolled back `mimetype` leaves a filler behind
|
|
661
|
+
if @ocf && @out.tell.zero?
|
|
662
|
+
verify_ocf_mimetype_entry!(filename, storage_mode, use_data_descriptor)
|
|
663
|
+
extended_timestamp = false
|
|
664
|
+
elsif @ocf && (@files.empty? || @files.first.filler?)
|
|
665
|
+
raise OCFViolation, "The \"mimetype\" entry of an OCF container must start at offset 0, " \
|
|
666
|
+
"but #{@out.tell} bytes have already been output"
|
|
667
|
+
end
|
|
668
|
+
|
|
589
669
|
# If we need to massage filenames to enforce uniqueness,
|
|
590
670
|
# do so before we check for file/directory conflicts
|
|
591
671
|
filename = ZipKit::UniquifyFilename.call(filename, @path_set) if @dedupe_filenames
|
|
592
672
|
|
|
673
|
+
verify_ocf_filename!(filename) if @ocf
|
|
674
|
+
|
|
593
675
|
# Make sure there is no file/directory clobbering (conflicts), or - if deduping is disabled -
|
|
594
676
|
# no duplicate filenames/paths
|
|
595
677
|
if filename.end_with?("/")
|
|
@@ -597,6 +679,7 @@ class ZipKit::Streamer
|
|
|
597
679
|
else
|
|
598
680
|
@path_set.add_file_path(filename)
|
|
599
681
|
end
|
|
682
|
+
register_ocf_folded_path(filename) if @ocf
|
|
600
683
|
|
|
601
684
|
if use_data_descriptor
|
|
602
685
|
crc32 = 0
|
|
@@ -616,7 +699,8 @@ class ZipKit::Streamer
|
|
|
616
699
|
_local_file_header_offset = local_header_starts_at,
|
|
617
700
|
_bytes_used_for_local_header = 0,
|
|
618
701
|
_bytes_used_for_data_descriptor = 0,
|
|
619
|
-
unix_permissions
|
|
702
|
+
unix_permissions,
|
|
703
|
+
extended_timestamp)
|
|
620
704
|
|
|
621
705
|
@writer.write_local_file_header(io: @out,
|
|
622
706
|
gp_flags: e.gp_flags,
|
|
@@ -625,7 +709,8 @@ class ZipKit::Streamer
|
|
|
625
709
|
uncompressed_size: e.uncompressed_size,
|
|
626
710
|
mtime: e.mtime,
|
|
627
711
|
filename: e.filename,
|
|
628
|
-
storage_mode: e.storage_mode
|
|
712
|
+
storage_mode: e.storage_mode,
|
|
713
|
+
extended_timestamp: e.extended_timestamp)
|
|
629
714
|
|
|
630
715
|
e.bytes_used_for_local_header = @out.tell - e.local_header_offset
|
|
631
716
|
|
|
@@ -633,6 +718,62 @@ class ZipKit::Streamer
|
|
|
633
718
|
@remove_last_file_at_rollback = true
|
|
634
719
|
end
|
|
635
720
|
|
|
721
|
+
def verify_ocf_mimetype_entry!(filename, storage_mode, use_data_descriptor)
|
|
722
|
+
problem = if filename != "mimetype"
|
|
723
|
+
"it is #{filename.inspect}"
|
|
724
|
+
elsif storage_mode != STORED
|
|
725
|
+
"it is compressed"
|
|
726
|
+
elsif use_data_descriptor
|
|
727
|
+
"it uses a data descriptor (sizes are not known upfront)"
|
|
728
|
+
end
|
|
729
|
+
return unless problem
|
|
730
|
+
|
|
731
|
+
message = "The first entry of an OCF container must be a stored \"mimetype\" file without a data descriptor, " \
|
|
732
|
+
"but #{problem}. Use `write_mimetype_file` to write it."
|
|
733
|
+
raise OCFViolation, message
|
|
734
|
+
end
|
|
735
|
+
|
|
736
|
+
def verify_ocf_filename!(filename)
|
|
737
|
+
path = filename.dup.force_encoding(Encoding::UTF_8)
|
|
738
|
+
raise OCFViolation, "File names in an OCF container must be UTF-8, but #{filename.inspect} is not" unless path.valid_encoding?
|
|
739
|
+
|
|
740
|
+
path.chomp("/").split("/", -1).each do |name|
|
|
741
|
+
problem = if name.empty?
|
|
742
|
+
"has an empty path segment"
|
|
743
|
+
elsif name.bytesize > OCF_MAX_FILENAME_BYTES
|
|
744
|
+
"has a segment longer than #{OCF_MAX_FILENAME_BYTES} bytes"
|
|
745
|
+
elsif name.end_with?(".")
|
|
746
|
+
"has a segment ending with a \".\""
|
|
747
|
+
elsif (char = name[OCF_FORBIDDEN_CHARACTERS])
|
|
748
|
+
"contains the forbidden character U+%04X" % char.ord
|
|
749
|
+
end
|
|
750
|
+
raise OCFViolation, "The file name #{path.inspect} is not allowed in an OCF container: it #{problem}" if problem
|
|
751
|
+
end
|
|
752
|
+
|
|
753
|
+
each_ocf_folded_prefix(path) do |prefix, folded|
|
|
754
|
+
existing = @ocf_folded_paths[folded]
|
|
755
|
+
if existing && existing != prefix
|
|
756
|
+
raise OCFViolation, "The file name #{path.inspect} is not allowed in an OCF container: " \
|
|
757
|
+
"#{prefix.inspect} only differs from #{existing.inspect} by case or Unicode normalization"
|
|
758
|
+
end
|
|
759
|
+
end
|
|
760
|
+
end
|
|
761
|
+
|
|
762
|
+
def register_ocf_folded_path(filename)
|
|
763
|
+
each_ocf_folded_prefix(filename.dup.force_encoding(Encoding::UTF_8)) do |prefix, folded|
|
|
764
|
+
@ocf_folded_paths[folded] ||= prefix
|
|
765
|
+
end
|
|
766
|
+
end
|
|
767
|
+
|
|
768
|
+
# Directories have to be unique too, so "OEBPS/a.xhtml" and "oebps/b.xhtml" clash on "OEBPS"
|
|
769
|
+
def each_ocf_folded_prefix(path)
|
|
770
|
+
segments = path.chomp("/").split("/")
|
|
771
|
+
segments.each_index do |i|
|
|
772
|
+
prefix = segments[0..i].join("/")
|
|
773
|
+
yield(prefix, prefix.unicode_normalize(:nfc).downcase(:fold))
|
|
774
|
+
end
|
|
775
|
+
end
|
|
776
|
+
|
|
636
777
|
def remove_backslash(filename)
|
|
637
778
|
filename.tr("\\", "_")
|
|
638
779
|
end
|
data/lib/zip_kit/version.rb
CHANGED
data/lib/zip_kit/zip_writer.rb
CHANGED
|
@@ -29,7 +29,7 @@ require "stringio"
|
|
|
29
29
|
class ZipKit::ZipWriter
|
|
30
30
|
FOUR_BYTE_MAX_UINT = 0xFFFFFFFF
|
|
31
31
|
TWO_BYTE_MAX_UINT = 0xFFFF
|
|
32
|
-
ZIP_KIT_COMMENT = "Written using ZipKit %<version>s" % {version: ZipKit::VERSION}
|
|
32
|
+
ZIP_KIT_COMMENT = ("Written using ZipKit %<version>s" % {version: ZipKit::VERSION}).freeze
|
|
33
33
|
VERSION_MADE_BY = 52
|
|
34
34
|
VERSION_NEEDED_TO_EXTRACT = 20
|
|
35
35
|
VERSION_NEEDED_TO_EXTRACT_ZIP64 = 45
|
|
@@ -40,7 +40,7 @@ class ZipKit::ZipWriter
|
|
|
40
40
|
MADE_BY_SIGNATURE = begin
|
|
41
41
|
# A combination of the VERSION_MADE_BY low byte and the OS type high byte
|
|
42
42
|
os_type = 3 # UNIX
|
|
43
|
-
[VERSION_MADE_BY, os_type].pack("CC")
|
|
43
|
+
[VERSION_MADE_BY, os_type].pack("CC").freeze
|
|
44
44
|
end
|
|
45
45
|
|
|
46
46
|
C_UINT4 = "V" # Encode a 4-byte unsigned little-endian uint
|
|
@@ -72,8 +72,13 @@ class ZipKit::ZipWriter
|
|
|
72
72
|
# @param mtime[Time] the modification time to be recorded in the ZIP
|
|
73
73
|
# @param gp_flags[Integer] bit-packed general purpose flags
|
|
74
74
|
# @param storage_mode[Integer] 8 for deflated, 0 for stored...
|
|
75
|
+
# @param extended_timestamp[Boolean] whether to add the extended timestamp ("UT") extra field. The DOS date
|
|
76
|
+
# and time get written regardless, but they carry local time with 2-second precision - which is why the
|
|
77
|
+
# extra field is there by default. Some formats built on top of ZIP forbid extra fields on certain entries
|
|
78
|
+
# though: EPUB (OCF) and OpenDocument require their `mimetype` entry to have none. Normally `Streamer`
|
|
79
|
+
# decides this for you, see the `ocf:` option of {Streamer#initialize}
|
|
75
80
|
# @return [void]
|
|
76
|
-
def write_local_file_header(io:, filename:, compressed_size:, uncompressed_size:, crc32:, gp_flags:, mtime:, storage_mode:)
|
|
81
|
+
def write_local_file_header(io:, filename:, compressed_size:, uncompressed_size:, crc32:, gp_flags:, mtime:, storage_mode:, extended_timestamp: true)
|
|
77
82
|
requires_zip64 = compressed_size > FOUR_BYTE_MAX_UINT || uncompressed_size > FOUR_BYTE_MAX_UINT
|
|
78
83
|
|
|
79
84
|
# local file header signature 4 bytes (0x04034b50)
|
|
@@ -121,7 +126,7 @@ class ZipKit::ZipWriter
|
|
|
121
126
|
if requires_zip64
|
|
122
127
|
extra_fields << zip_64_extra_for_local_file_header(compressed_size: compressed_size, uncompressed_size: uncompressed_size)
|
|
123
128
|
end
|
|
124
|
-
extra_fields << timestamp_extra_for_local_file_header(mtime)
|
|
129
|
+
extra_fields << timestamp_extra_for_local_file_header(mtime) if extended_timestamp
|
|
125
130
|
|
|
126
131
|
# extra field length 2 bytes
|
|
127
132
|
io << [extra_fields.size].pack(C_UINT2)
|
|
@@ -144,6 +149,8 @@ class ZipKit::ZipWriter
|
|
|
144
149
|
# @param mtime[Time] the modification time to be recorded in the ZIP
|
|
145
150
|
# @param gp_flags[Integer] bit-packed general purpose flags
|
|
146
151
|
# @param unix_permissions[Integer] the permissions for the file, or nil for the default to be used
|
|
152
|
+
# @param extended_timestamp[Boolean] whether to add the extended timestamp ("UT") extra field,
|
|
153
|
+
# see {#write_local_file_header}
|
|
147
154
|
# @return [void]
|
|
148
155
|
def write_central_directory_file_header(io:,
|
|
149
156
|
local_file_header_location:,
|
|
@@ -154,7 +161,8 @@ class ZipKit::ZipWriter
|
|
|
154
161
|
mtime:,
|
|
155
162
|
crc32:,
|
|
156
163
|
filename:,
|
|
157
|
-
unix_permissions: nil
|
|
164
|
+
unix_permissions: nil,
|
|
165
|
+
extended_timestamp: true)
|
|
158
166
|
# At this point if the header begins somewhere beyound 0xFFFFFFFF we _have_ to record the offset
|
|
159
167
|
# of the local file header as a zip64 extra field, so we give up, give in, you loose, love will always win...
|
|
160
168
|
add_zip64 = (local_file_header_location > FOUR_BYTE_MAX_UINT) ||
|
|
@@ -203,7 +211,7 @@ class ZipKit::ZipWriter
|
|
|
203
211
|
compressed_size: compressed_size,
|
|
204
212
|
uncompressed_size: uncompressed_size)
|
|
205
213
|
end
|
|
206
|
-
extra_fields << timestamp_extra_for_central_directory_entry(mtime)
|
|
214
|
+
extra_fields << timestamp_extra_for_central_directory_entry(mtime) if extended_timestamp
|
|
207
215
|
|
|
208
216
|
# extra field length 2 bytes
|
|
209
217
|
io << [extra_fields.size].pack(C_UINT2)
|
data/rbi/zip_kit.rbi
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# typed: strong
|
|
2
2
|
module ZipKit
|
|
3
|
-
VERSION = T.let(
|
|
3
|
+
VERSION = T.let(T.unsafe(nil), String)
|
|
4
4
|
|
|
5
5
|
class Railtie < Rails::Railtie
|
|
6
6
|
end
|
|
@@ -186,15 +186,44 @@ module ZipKit
|
|
|
186
186
|
# zip.close
|
|
187
187
|
#
|
|
188
188
|
# Calling {Streamer#close} **will not** call `#close` on the underlying IO object.
|
|
189
|
+
#
|
|
190
|
+
# ## Writing EPUB and OpenDocument containers
|
|
191
|
+
#
|
|
192
|
+
# EPUB (via its Open Container Format, OCF) and OpenDocument are ZIP files with an extra rule: the
|
|
193
|
+
# first entry must be a file called `mimetype`, stored without compression and with no extra fields
|
|
194
|
+
# in its local header. Readers check this by looking at the raw bytes at fixed offsets: the media type
|
|
195
|
+
# must start at byte 38 of the file. Pass `ocf: true` and write the `mimetype` file first:
|
|
196
|
+
#
|
|
197
|
+
# ZipKit::Streamer.open(epub_file, ocf: true) do |zip|
|
|
198
|
+
# zip.write_mimetype_file("application/epub+zip")
|
|
199
|
+
# zip.write_file("META-INF/container.xml") { |sink| sink << container_xml }
|
|
200
|
+
# ...
|
|
201
|
+
# end
|
|
202
|
+
#
|
|
203
|
+
# With `ocf: true` the Streamer will raise {OCFViolation} if the first entry it is asked to write
|
|
204
|
+
# would make the archive non-conformant, or if a file name is not allowed in such a container.
|
|
189
205
|
class Streamer
|
|
190
206
|
include ZipKit::WriteShovel
|
|
191
207
|
STORED = T.let(0, T.untyped)
|
|
192
208
|
DEFLATED = T.let(8, T.untyped)
|
|
193
209
|
EntryBodySizeMismatch = T.let(Class.new(StandardError), T.untyped)
|
|
210
|
+
OCFViolation = T.let(Class.new(StandardError), T.untyped)
|
|
194
211
|
InvalidOutput = T.let(Class.new(ArgumentError), T.untyped)
|
|
195
212
|
Overflow = T.let(Class.new(StandardError), T.untyped)
|
|
196
213
|
UnknownMode = T.let(Class.new(StandardError), T.untyped)
|
|
197
214
|
OffsetOutOfSync = T.let(Class.new(StandardError), T.untyped)
|
|
215
|
+
OCF_FORBIDDEN_CHARACTERS = T.let(begin
|
|
216
|
+
ranges = [0x22, 0x2A, 0x3A, 0x3C, 0x3E, 0x3F, 0x5C, 0x7C].map { |c| [c, c] }
|
|
217
|
+
ranges << [0x00, 0x1F] # C0
|
|
218
|
+
ranges << [0x7F, 0x9F] # DEL and C1
|
|
219
|
+
ranges << [0xE000, 0xF8FF] # Private Use Area
|
|
220
|
+
ranges << [0xFDD0, 0xFDEF] # noncharacters
|
|
221
|
+
ranges << [0xFFF0, 0xFFFF] # Specials, includes the U+FFFE and U+FFFF noncharacters
|
|
222
|
+
ranges += (1..14).map { |plane| [plane * 0x10000 + 0xFFFE, plane * 0x10000 + 0xFFFF] } # noncharacters
|
|
223
|
+
ranges << [0xF0000, 0x10FFFF] # Supplementary Private Use Areas
|
|
224
|
+
Regexp.new("[" + ranges.map { |from, to| format("\\u{%X}-\\u{%X}", from, to) }.join + "]").freeze
|
|
225
|
+
end, T.untyped)
|
|
226
|
+
OCF_MAX_FILENAME_BYTES = T.let(255, T.untyped)
|
|
198
227
|
|
|
199
228
|
# sord omit - no YARD return type given, using untyped
|
|
200
229
|
# Creates a new Streamer on top of the given IO-ish object and yields it. Once the given block
|
|
@@ -215,8 +244,17 @@ module ZipKit
|
|
|
215
244
|
# _@param_ `writer` — the object to be used as the writer. Defaults to an instance of ZipKit::ZipWriter, normally you won't need to override it
|
|
216
245
|
#
|
|
217
246
|
# _@param_ `auto_rename_duplicate_filenames` — whether duplicate filenames, when encountered, should be suffixed with (1), (2) etc. Default value is `false` - if dupliate names are used an exception will be raised
|
|
218
|
-
|
|
219
|
-
|
|
247
|
+
#
|
|
248
|
+
# _@param_ `ocf` — whether the archive must be an OCF container (EPUB) or an OpenDocument package. When set, the first entry must be a stored `mimetype` file without a data descriptor - use {#write_mimetype_file} for it - and {OCFViolation} gets raised otherwise. That entry will be written without the extended timestamp extra field, which those formats forbid. All file names get checked against the OCF rules too: valid UTF-8, no forbidden characters, no names longer than 255 bytes or ending with a ".", and no two names in a directory which only differ by case or Unicode normalization.
|
|
249
|
+
sig do
|
|
250
|
+
params(
|
|
251
|
+
writable: T.untyped,
|
|
252
|
+
writer: ZipKit::ZipWriter,
|
|
253
|
+
auto_rename_duplicate_filenames: T::Boolean,
|
|
254
|
+
ocf: T::Boolean
|
|
255
|
+
).void
|
|
256
|
+
end
|
|
257
|
+
def initialize(writable, writer: create_writer, auto_rename_duplicate_filenames: false, ocf: false); end
|
|
220
258
|
|
|
221
259
|
# Writes a part of a zip entry body (actual binary data of the entry) into the output stream.
|
|
222
260
|
#
|
|
@@ -303,6 +341,18 @@ module ZipKit
|
|
|
303
341
|
end
|
|
304
342
|
def add_stored_entry(filename:, modification_time: Time.now.utc, size: 0, crc32: 0, unix_permissions: nil, use_data_descriptor: false); end
|
|
305
343
|
|
|
344
|
+
# Writes the `mimetype` file which EPUB (OCF) and OpenDocument containers have to start with.
|
|
345
|
+
# The entry is stored, has no data descriptor and no extra fields, so that the media type
|
|
346
|
+
# ends up at byte 38 of the archive if this is the first entry.
|
|
347
|
+
#
|
|
348
|
+
# _@param_ `media_type` — the media type, like "application/epub+zip"
|
|
349
|
+
#
|
|
350
|
+
# _@param_ `modification_time` — the modification time of the file in the archive
|
|
351
|
+
#
|
|
352
|
+
# _@return_ — the offset the output IO is at after writing the entry
|
|
353
|
+
sig { params(media_type: String, modification_time: Time).returns(Integer) }
|
|
354
|
+
def write_mimetype_file(media_type, modification_time: Time.now.utc); end
|
|
355
|
+
|
|
306
356
|
# Adds an empty directory to the archive with a size of 0 and permissions of 755.
|
|
307
357
|
#
|
|
308
358
|
# _@param_ `dirname` — the name of the directory in the archive
|
|
@@ -554,6 +604,7 @@ module ZipKit
|
|
|
554
604
|
# sord omit - no YARD type given for "uncompressed_size:", using untyped
|
|
555
605
|
# sord omit - no YARD type given for "use_data_descriptor:", using untyped
|
|
556
606
|
# sord omit - no YARD type given for "unix_permissions:", using untyped
|
|
607
|
+
# sord omit - no YARD type given for "extended_timestamp:", using untyped
|
|
557
608
|
# sord omit - no YARD return type given, using untyped
|
|
558
609
|
sig do
|
|
559
610
|
params(
|
|
@@ -564,10 +615,34 @@ module ZipKit
|
|
|
564
615
|
compressed_size: T.untyped,
|
|
565
616
|
uncompressed_size: T.untyped,
|
|
566
617
|
use_data_descriptor: T.untyped,
|
|
567
|
-
unix_permissions: T.untyped
|
|
618
|
+
unix_permissions: T.untyped,
|
|
619
|
+
extended_timestamp: T.untyped
|
|
568
620
|
).returns(T.untyped)
|
|
569
621
|
end
|
|
570
|
-
def add_file_and_write_local_header(filename:, modification_time:, crc32:, storage_mode:, compressed_size:, uncompressed_size:, use_data_descriptor:, unix_permissions:); end
|
|
622
|
+
def add_file_and_write_local_header(filename:, modification_time:, crc32:, storage_mode:, compressed_size:, uncompressed_size:, use_data_descriptor:, unix_permissions:, extended_timestamp: true); end
|
|
623
|
+
|
|
624
|
+
# sord omit - no YARD type given for "filename", using untyped
|
|
625
|
+
# sord omit - no YARD type given for "storage_mode", using untyped
|
|
626
|
+
# sord omit - no YARD type given for "use_data_descriptor", using untyped
|
|
627
|
+
# sord omit - no YARD return type given, using untyped
|
|
628
|
+
sig { params(filename: T.untyped, storage_mode: T.untyped, use_data_descriptor: T.untyped).returns(T.untyped) }
|
|
629
|
+
def verify_ocf_mimetype_entry!(filename, storage_mode, use_data_descriptor); end
|
|
630
|
+
|
|
631
|
+
# sord omit - no YARD type given for "filename", using untyped
|
|
632
|
+
# sord omit - no YARD return type given, using untyped
|
|
633
|
+
sig { params(filename: T.untyped).returns(T.untyped) }
|
|
634
|
+
def verify_ocf_filename!(filename); end
|
|
635
|
+
|
|
636
|
+
# sord omit - no YARD type given for "filename", using untyped
|
|
637
|
+
# sord omit - no YARD return type given, using untyped
|
|
638
|
+
sig { params(filename: T.untyped).returns(T.untyped) }
|
|
639
|
+
def register_ocf_folded_path(filename); end
|
|
640
|
+
|
|
641
|
+
# sord omit - no YARD type given for "path", using untyped
|
|
642
|
+
# sord omit - no YARD return type given, using untyped
|
|
643
|
+
# Directories have to be unique too, so "OEBPS/a.xhtml" and "oebps/b.xhtml" clash on "OEBPS"
|
|
644
|
+
sig { params(path: T.untyped).returns(T.untyped) }
|
|
645
|
+
def each_ocf_folded_prefix(path); end
|
|
571
646
|
|
|
572
647
|
# sord omit - no YARD type given for "filename", using untyped
|
|
573
648
|
# sord omit - no YARD return type given, using untyped
|
|
@@ -648,6 +723,10 @@ module ZipKit
|
|
|
648
723
|
# Returns the value of attribute unix_permissions
|
|
649
724
|
sig { returns(Object) }
|
|
650
725
|
attr_accessor :unix_permissions
|
|
726
|
+
|
|
727
|
+
# Returns the value of attribute extended_timestamp
|
|
728
|
+
sig { returns(Object) }
|
|
729
|
+
attr_accessor :extended_timestamp
|
|
651
730
|
end
|
|
652
731
|
|
|
653
732
|
# Is used internally by Streamer to keep track of entries in the archive during writing.
|
|
@@ -942,7 +1021,7 @@ module ZipKit
|
|
|
942
1021
|
class ZipWriter
|
|
943
1022
|
FOUR_BYTE_MAX_UINT = T.let(0xFFFFFFFF, T.untyped)
|
|
944
1023
|
TWO_BYTE_MAX_UINT = T.let(0xFFFF, T.untyped)
|
|
945
|
-
ZIP_KIT_COMMENT = T.let("Written using ZipKit %<version>s" % {version: ZipKit::VERSION}, T.untyped)
|
|
1024
|
+
ZIP_KIT_COMMENT = T.let(("Written using ZipKit %<version>s" % {version: ZipKit::VERSION}).freeze, T.untyped)
|
|
946
1025
|
VERSION_MADE_BY = T.let(52, T.untyped)
|
|
947
1026
|
VERSION_NEEDED_TO_EXTRACT = T.let(20, T.untyped)
|
|
948
1027
|
VERSION_NEEDED_TO_EXTRACT_ZIP64 = T.let(45, T.untyped)
|
|
@@ -953,7 +1032,7 @@ module ZipKit
|
|
|
953
1032
|
MADE_BY_SIGNATURE = T.let(begin
|
|
954
1033
|
# A combination of the VERSION_MADE_BY low byte and the OS type high byte
|
|
955
1034
|
os_type = 3 # UNIX
|
|
956
|
-
[VERSION_MADE_BY, os_type].pack("CC")
|
|
1035
|
+
[VERSION_MADE_BY, os_type].pack("CC").freeze
|
|
957
1036
|
end, T.untyped)
|
|
958
1037
|
C_UINT4 = T.let("V", T.untyped)
|
|
959
1038
|
C_UINT2 = T.let("v", T.untyped)
|
|
@@ -979,6 +1058,8 @@ end, T.untyped)
|
|
|
979
1058
|
# _@param_ `gp_flags` — bit-packed general purpose flags
|
|
980
1059
|
#
|
|
981
1060
|
# _@param_ `storage_mode` — 8 for deflated, 0 for stored...
|
|
1061
|
+
#
|
|
1062
|
+
# _@param_ `extended_timestamp` — whether to add the extended timestamp ("UT") extra field. The DOS date and time get written regardless, but they carry local time with 2-second precision - which is why the extra field is there by default. Some formats built on top of ZIP forbid extra fields on certain entries though: EPUB (OCF) and OpenDocument require their `mimetype` entry to have none. Normally `Streamer` decides this for you, see the `ocf:` option of {Streamer#initialize}
|
|
982
1063
|
sig do
|
|
983
1064
|
params(
|
|
984
1065
|
io: T.untyped,
|
|
@@ -988,10 +1069,11 @@ end, T.untyped)
|
|
|
988
1069
|
crc32: Integer,
|
|
989
1070
|
gp_flags: Integer,
|
|
990
1071
|
mtime: Time,
|
|
991
|
-
storage_mode: Integer
|
|
1072
|
+
storage_mode: Integer,
|
|
1073
|
+
extended_timestamp: T::Boolean
|
|
992
1074
|
).void
|
|
993
1075
|
end
|
|
994
|
-
def write_local_file_header(io:, filename:, compressed_size:, uncompressed_size:, crc32:, gp_flags:, mtime:, storage_mode:); end
|
|
1076
|
+
def write_local_file_header(io:, filename:, compressed_size:, uncompressed_size:, crc32:, gp_flags:, mtime:, storage_mode:, extended_timestamp: true); end
|
|
995
1077
|
|
|
996
1078
|
# sord duck - #<< looks like a duck type, replacing with untyped
|
|
997
1079
|
# sord omit - no YARD type given for "local_file_header_location:", using untyped
|
|
@@ -1014,6 +1096,8 @@ end, T.untyped)
|
|
|
1014
1096
|
# _@param_ `gp_flags` — bit-packed general purpose flags
|
|
1015
1097
|
#
|
|
1016
1098
|
# _@param_ `unix_permissions` — the permissions for the file, or nil for the default to be used
|
|
1099
|
+
#
|
|
1100
|
+
# _@param_ `extended_timestamp` — whether to add the extended timestamp ("UT") extra field, see {#write_local_file_header}
|
|
1017
1101
|
sig do
|
|
1018
1102
|
params(
|
|
1019
1103
|
io: T.untyped,
|
|
@@ -1025,10 +1109,11 @@ end, T.untyped)
|
|
|
1025
1109
|
mtime: Time,
|
|
1026
1110
|
crc32: Integer,
|
|
1027
1111
|
filename: String,
|
|
1028
|
-
unix_permissions: T.nilable(Integer)
|
|
1112
|
+
unix_permissions: T.nilable(Integer),
|
|
1113
|
+
extended_timestamp: T::Boolean
|
|
1029
1114
|
).void
|
|
1030
1115
|
end
|
|
1031
|
-
def write_central_directory_file_header(io:, local_file_header_location:, gp_flags:, storage_mode:, compressed_size:, uncompressed_size:, mtime:, crc32:, filename:, unix_permissions: nil); end
|
|
1116
|
+
def write_central_directory_file_header(io:, local_file_header_location:, gp_flags:, storage_mode:, compressed_size:, uncompressed_size:, mtime:, crc32:, filename:, unix_permissions: nil, extended_timestamp: true); end
|
|
1032
1117
|
|
|
1033
1118
|
# sord duck - #<< looks like a duck type, replacing with untyped
|
|
1034
1119
|
# Writes the data descriptor following the file data for a file whose local file header
|
|
@@ -1164,6 +1249,20 @@ end, T.untyped)
|
|
|
1164
1249
|
sig { params(block: T.proc.params(bytes: String).void).void }
|
|
1165
1250
|
def initialize(&block); end
|
|
1166
1251
|
|
|
1252
|
+
# sord omit - no YARD return type given, using untyped
|
|
1253
|
+
# Make sure those methods raise outright. These are not created with define_method,
|
|
1254
|
+
# because methods defined with a block can't be called from a non-main Ractor
|
|
1255
|
+
sig { returns(T.untyped) }
|
|
1256
|
+
def seek; end
|
|
1257
|
+
|
|
1258
|
+
# sord omit - no YARD return type given, using untyped
|
|
1259
|
+
sig { returns(T.untyped) }
|
|
1260
|
+
def pos=; end
|
|
1261
|
+
|
|
1262
|
+
# sord omit - no YARD return type given, using untyped
|
|
1263
|
+
sig { returns(T.untyped) }
|
|
1264
|
+
def to_s; end
|
|
1265
|
+
|
|
1167
1266
|
# Sends a string through to the block stored in the BlockWrite.
|
|
1168
1267
|
#
|
|
1169
1268
|
# _@param_ `buf` — the string to write. Note that a zero-length String will not be forwarded to the block, as it has special meaning when used with chunked encoding (it indicates the end of the stream).
|
|
@@ -1879,7 +1978,7 @@ end, T.untyped)
|
|
|
1879
1978
|
# compressed_string = ZipKit::BlockDeflate.deflate_chunk(big_string)
|
|
1880
1979
|
class BlockDeflate
|
|
1881
1980
|
DEFAULT_BLOCKSIZE = T.let(1_024 * 1024 * 5, T.untyped)
|
|
1882
|
-
END_MARKER = T.let([3, 0].pack("C*"), T.untyped)
|
|
1981
|
+
END_MARKER = T.let([3, 0].pack("C*").freeze, T.untyped)
|
|
1883
1982
|
VALID_COMPRESSIONS = T.let((Zlib::DEFAULT_COMPRESSION..Zlib::BEST_COMPRESSION).to_a.freeze, T.untyped)
|
|
1884
1983
|
|
|
1885
1984
|
# Write the end marker (\x3\x0) to the given IO.
|
|
@@ -2016,6 +2115,15 @@ end, T.untyped)
|
|
|
2016
2115
|
end
|
|
2017
2116
|
def add_deflated_entry(filename:, uncompressed_size:, compressed_size:, use_data_descriptor: false); end
|
|
2018
2117
|
|
|
2118
|
+
# Add the `mimetype` entry an EPUB (OCF) or OpenDocument container starts with,
|
|
2119
|
+
# see {Streamer#write_mimetype_file}.
|
|
2120
|
+
#
|
|
2121
|
+
# _@param_ `media_type` — the media type, like "application/epub+zip"
|
|
2122
|
+
#
|
|
2123
|
+
# _@return_ — self
|
|
2124
|
+
sig { params(media_type: String).returns(T.untyped) }
|
|
2125
|
+
def add_mimetype_entry(media_type:); end
|
|
2126
|
+
|
|
2019
2127
|
# Add an empty directory to the archive.
|
|
2020
2128
|
#
|
|
2021
2129
|
# _@param_ `dirname` — the name of the directory
|
|
@@ -2225,7 +2333,7 @@ end, T.untyped)
|
|
|
2225
2333
|
# carry, so we copy it into our code.
|
|
2226
2334
|
class RackChunkedBody
|
|
2227
2335
|
TERM = T.let("\r\n", T.untyped)
|
|
2228
|
-
TAIL = T.let("0
|
|
2336
|
+
TAIL = T.let("0\r\n", T.untyped)
|
|
2229
2337
|
|
|
2230
2338
|
# sord duck - #each looks like a duck type, replacing with untyped
|
|
2231
2339
|
# _@param_ `body` — the enumerable that yields bytes, usually a `OutputEnumerator`
|
data/rbi/zip_kit.rbs
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
module ZipKit
|
|
2
|
-
VERSION:
|
|
2
|
+
VERSION: String
|
|
3
3
|
|
|
4
4
|
class Railtie < Rails::Railtie
|
|
5
5
|
end
|
|
@@ -177,15 +177,34 @@ module ZipKit
|
|
|
177
177
|
# zip.close
|
|
178
178
|
#
|
|
179
179
|
# Calling {Streamer#close} **will not** call `#close` on the underlying IO object.
|
|
180
|
+
#
|
|
181
|
+
# ## Writing EPUB and OpenDocument containers
|
|
182
|
+
#
|
|
183
|
+
# EPUB (via its Open Container Format, OCF) and OpenDocument are ZIP files with an extra rule: the
|
|
184
|
+
# first entry must be a file called `mimetype`, stored without compression and with no extra fields
|
|
185
|
+
# in its local header. Readers check this by looking at the raw bytes at fixed offsets: the media type
|
|
186
|
+
# must start at byte 38 of the file. Pass `ocf: true` and write the `mimetype` file first:
|
|
187
|
+
#
|
|
188
|
+
# ZipKit::Streamer.open(epub_file, ocf: true) do |zip|
|
|
189
|
+
# zip.write_mimetype_file("application/epub+zip")
|
|
190
|
+
# zip.write_file("META-INF/container.xml") { |sink| sink << container_xml }
|
|
191
|
+
# ...
|
|
192
|
+
# end
|
|
193
|
+
#
|
|
194
|
+
# With `ocf: true` the Streamer will raise {OCFViolation} if the first entry it is asked to write
|
|
195
|
+
# would make the archive non-conformant, or if a file name is not allowed in such a container.
|
|
180
196
|
class Streamer
|
|
181
197
|
include ZipKit::WriteShovel
|
|
182
198
|
STORED: untyped
|
|
183
199
|
DEFLATED: untyped
|
|
184
200
|
EntryBodySizeMismatch: untyped
|
|
201
|
+
OCFViolation: untyped
|
|
185
202
|
InvalidOutput: untyped
|
|
186
203
|
Overflow: untyped
|
|
187
204
|
UnknownMode: untyped
|
|
188
205
|
OffsetOutOfSync: untyped
|
|
206
|
+
OCF_FORBIDDEN_CHARACTERS: untyped
|
|
207
|
+
OCF_MAX_FILENAME_BYTES: untyped
|
|
189
208
|
|
|
190
209
|
# sord omit - no YARD return type given, using untyped
|
|
191
210
|
# Creates a new Streamer on top of the given IO-ish object and yields it. Once the given block
|
|
@@ -205,7 +224,14 @@ module ZipKit
|
|
|
205
224
|
# _@param_ `writer` — the object to be used as the writer. Defaults to an instance of ZipKit::ZipWriter, normally you won't need to override it
|
|
206
225
|
#
|
|
207
226
|
# _@param_ `auto_rename_duplicate_filenames` — whether duplicate filenames, when encountered, should be suffixed with (1), (2) etc. Default value is `false` - if dupliate names are used an exception will be raised
|
|
208
|
-
|
|
227
|
+
#
|
|
228
|
+
# _@param_ `ocf` — whether the archive must be an OCF container (EPUB) or an OpenDocument package. When set, the first entry must be a stored `mimetype` file without a data descriptor - use {#write_mimetype_file} for it - and {OCFViolation} gets raised otherwise. That entry will be written without the extended timestamp extra field, which those formats forbid. All file names get checked against the OCF rules too: valid UTF-8, no forbidden characters, no names longer than 255 bytes or ending with a ".", and no two names in a directory which only differ by case or Unicode normalization.
|
|
229
|
+
def initialize: (
|
|
230
|
+
untyped writable,
|
|
231
|
+
?writer: ZipKit::ZipWriter,
|
|
232
|
+
?auto_rename_duplicate_filenames: bool,
|
|
233
|
+
?ocf: bool
|
|
234
|
+
) -> void
|
|
209
235
|
|
|
210
236
|
# Writes a part of a zip entry body (actual binary data of the entry) into the output stream.
|
|
211
237
|
#
|
|
@@ -284,6 +310,17 @@ module ZipKit
|
|
|
284
310
|
?use_data_descriptor: bool
|
|
285
311
|
) -> Integer
|
|
286
312
|
|
|
313
|
+
# Writes the `mimetype` file which EPUB (OCF) and OpenDocument containers have to start with.
|
|
314
|
+
# The entry is stored, has no data descriptor and no extra fields, so that the media type
|
|
315
|
+
# ends up at byte 38 of the archive if this is the first entry.
|
|
316
|
+
#
|
|
317
|
+
# _@param_ `media_type` — the media type, like "application/epub+zip"
|
|
318
|
+
#
|
|
319
|
+
# _@param_ `modification_time` — the modification time of the file in the archive
|
|
320
|
+
#
|
|
321
|
+
# _@return_ — the offset the output IO is at after writing the entry
|
|
322
|
+
def write_mimetype_file: (String media_type, ?modification_time: Time) -> Integer
|
|
323
|
+
|
|
287
324
|
# Adds an empty directory to the archive with a size of 0 and permissions of 755.
|
|
288
325
|
#
|
|
289
326
|
# _@param_ `dirname` — the name of the directory in the archive
|
|
@@ -504,6 +541,7 @@ module ZipKit
|
|
|
504
541
|
# sord omit - no YARD type given for "uncompressed_size:", using untyped
|
|
505
542
|
# sord omit - no YARD type given for "use_data_descriptor:", using untyped
|
|
506
543
|
# sord omit - no YARD type given for "unix_permissions:", using untyped
|
|
544
|
+
# sord omit - no YARD type given for "extended_timestamp:", using untyped
|
|
507
545
|
# sord omit - no YARD return type given, using untyped
|
|
508
546
|
def add_file_and_write_local_header: (
|
|
509
547
|
filename: untyped,
|
|
@@ -513,9 +551,29 @@ module ZipKit
|
|
|
513
551
|
compressed_size: untyped,
|
|
514
552
|
uncompressed_size: untyped,
|
|
515
553
|
use_data_descriptor: untyped,
|
|
516
|
-
unix_permissions: untyped
|
|
554
|
+
unix_permissions: untyped,
|
|
555
|
+
?extended_timestamp: untyped
|
|
517
556
|
) -> untyped
|
|
518
557
|
|
|
558
|
+
# sord omit - no YARD type given for "filename", using untyped
|
|
559
|
+
# sord omit - no YARD type given for "storage_mode", using untyped
|
|
560
|
+
# sord omit - no YARD type given for "use_data_descriptor", using untyped
|
|
561
|
+
# sord omit - no YARD return type given, using untyped
|
|
562
|
+
def verify_ocf_mimetype_entry!: (untyped filename, untyped storage_mode, untyped use_data_descriptor) -> untyped
|
|
563
|
+
|
|
564
|
+
# sord omit - no YARD type given for "filename", using untyped
|
|
565
|
+
# sord omit - no YARD return type given, using untyped
|
|
566
|
+
def verify_ocf_filename!: (untyped filename) -> untyped
|
|
567
|
+
|
|
568
|
+
# sord omit - no YARD type given for "filename", using untyped
|
|
569
|
+
# sord omit - no YARD return type given, using untyped
|
|
570
|
+
def register_ocf_folded_path: (untyped filename) -> untyped
|
|
571
|
+
|
|
572
|
+
# sord omit - no YARD type given for "path", using untyped
|
|
573
|
+
# sord omit - no YARD return type given, using untyped
|
|
574
|
+
# Directories have to be unique too, so "OEBPS/a.xhtml" and "oebps/b.xhtml" clash on "OEBPS"
|
|
575
|
+
def each_ocf_folded_prefix: (untyped path) -> untyped
|
|
576
|
+
|
|
519
577
|
# sord omit - no YARD type given for "filename", using untyped
|
|
520
578
|
# sord omit - no YARD return type given, using untyped
|
|
521
579
|
def remove_backslash: (untyped filename) -> untyped
|
|
@@ -578,6 +636,9 @@ module ZipKit
|
|
|
578
636
|
|
|
579
637
|
# Returns the value of attribute unix_permissions
|
|
580
638
|
attr_accessor unix_permissions: Object
|
|
639
|
+
|
|
640
|
+
# Returns the value of attribute extended_timestamp
|
|
641
|
+
attr_accessor extended_timestamp: Object
|
|
581
642
|
end
|
|
582
643
|
|
|
583
644
|
# Is used internally by Streamer to keep track of entries in the archive during writing.
|
|
@@ -873,6 +934,8 @@ module ZipKit
|
|
|
873
934
|
# _@param_ `gp_flags` — bit-packed general purpose flags
|
|
874
935
|
#
|
|
875
936
|
# _@param_ `storage_mode` — 8 for deflated, 0 for stored...
|
|
937
|
+
#
|
|
938
|
+
# _@param_ `extended_timestamp` — whether to add the extended timestamp ("UT") extra field. The DOS date and time get written regardless, but they carry local time with 2-second precision - which is why the extra field is there by default. Some formats built on top of ZIP forbid extra fields on certain entries though: EPUB (OCF) and OpenDocument require their `mimetype` entry to have none. Normally `Streamer` decides this for you, see the `ocf:` option of {Streamer#initialize}
|
|
876
939
|
def write_local_file_header: (
|
|
877
940
|
io: untyped,
|
|
878
941
|
filename: String,
|
|
@@ -881,7 +944,8 @@ module ZipKit
|
|
|
881
944
|
crc32: Integer,
|
|
882
945
|
gp_flags: Integer,
|
|
883
946
|
mtime: Time,
|
|
884
|
-
storage_mode: Integer
|
|
947
|
+
storage_mode: Integer,
|
|
948
|
+
?extended_timestamp: bool
|
|
885
949
|
) -> void
|
|
886
950
|
|
|
887
951
|
# sord duck - #<< looks like a duck type, replacing with untyped
|
|
@@ -905,6 +969,8 @@ module ZipKit
|
|
|
905
969
|
# _@param_ `gp_flags` — bit-packed general purpose flags
|
|
906
970
|
#
|
|
907
971
|
# _@param_ `unix_permissions` — the permissions for the file, or nil for the default to be used
|
|
972
|
+
#
|
|
973
|
+
# _@param_ `extended_timestamp` — whether to add the extended timestamp ("UT") extra field, see {#write_local_file_header}
|
|
908
974
|
def write_central_directory_file_header: (
|
|
909
975
|
io: untyped,
|
|
910
976
|
local_file_header_location: untyped,
|
|
@@ -915,7 +981,8 @@ module ZipKit
|
|
|
915
981
|
mtime: Time,
|
|
916
982
|
crc32: Integer,
|
|
917
983
|
filename: String,
|
|
918
|
-
?unix_permissions: Integer
|
|
984
|
+
?unix_permissions: Integer?,
|
|
985
|
+
?extended_timestamp: bool
|
|
919
986
|
) -> void
|
|
920
987
|
|
|
921
988
|
# sord duck - #<< looks like a duck type, replacing with untyped
|
|
@@ -1038,6 +1105,17 @@ module ZipKit
|
|
|
1038
1105
|
# _@param_ `block` — The block that will be called when this object receives the `<<` message
|
|
1039
1106
|
def initialize: () ?{ (String bytes) -> void } -> void
|
|
1040
1107
|
|
|
1108
|
+
# sord omit - no YARD return type given, using untyped
|
|
1109
|
+
# Make sure those methods raise outright. These are not created with define_method,
|
|
1110
|
+
# because methods defined with a block can't be called from a non-main Ractor
|
|
1111
|
+
def seek: () -> untyped
|
|
1112
|
+
|
|
1113
|
+
# sord omit - no YARD return type given, using untyped
|
|
1114
|
+
def pos=: () -> untyped
|
|
1115
|
+
|
|
1116
|
+
# sord omit - no YARD return type given, using untyped
|
|
1117
|
+
def to_s: () -> untyped
|
|
1118
|
+
|
|
1041
1119
|
# Sends a string through to the block stored in the BlockWrite.
|
|
1042
1120
|
#
|
|
1043
1121
|
# _@param_ `buf` — the string to write. Note that a zero-length String will not be forwarded to the block, as it has special meaning when used with chunked encoding (it indicates the end of the stream).
|
|
@@ -1769,6 +1847,14 @@ module ZipKit
|
|
|
1769
1847
|
?use_data_descriptor: bool
|
|
1770
1848
|
) -> untyped
|
|
1771
1849
|
|
|
1850
|
+
# Add the `mimetype` entry an EPUB (OCF) or OpenDocument container starts with,
|
|
1851
|
+
# see {Streamer#write_mimetype_file}.
|
|
1852
|
+
#
|
|
1853
|
+
# _@param_ `media_type` — the media type, like "application/epub+zip"
|
|
1854
|
+
#
|
|
1855
|
+
# _@return_ — self
|
|
1856
|
+
def add_mimetype_entry: (media_type: String) -> untyped
|
|
1857
|
+
|
|
1772
1858
|
# Add an empty directory to the archive.
|
|
1773
1859
|
#
|
|
1774
1860
|
# _@param_ `dirname` — the name of the directory
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: zip_kit
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 6.
|
|
4
|
+
version: 6.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Julik Tarkhanov
|
|
@@ -11,7 +11,7 @@ authors:
|
|
|
11
11
|
- Felix Bünemann
|
|
12
12
|
bindir: exe
|
|
13
13
|
cert_chain: []
|
|
14
|
-
date: 2026-
|
|
14
|
+
date: 2026-10-08 00:00:00.000000000 Z
|
|
15
15
|
dependencies:
|
|
16
16
|
- !ruby/object:Gem::Dependency
|
|
17
17
|
name: bundler
|