ziperu 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +73 -0
- data/Cargo.lock +1 -1
- data/README.md +56 -5
- data/ext/ziperu/Cargo.toml +1 -1
- data/ext/ziperu/src/archive.rs +85 -54
- data/ext/ziperu/src/errors.rs +76 -14
- data/ext/ziperu/src/extract.rs +92 -33
- data/ext/ziperu/src/lib.rs +12 -9
- data/ext/ziperu/src/reader.rs +211 -13
- data/ext/ziperu/src/sink.rs +48 -8
- data/ext/ziperu/src/source.rs +2 -2
- data/ext/ziperu/src/stream.rs +541 -0
- data/ext/ziperu/src/writer.rs +128 -5
- data/lib/ziperu/entry/source.rb +1 -1
- data/lib/ziperu/entry.rb +8 -1
- data/lib/ziperu/errors.rb +5 -0
- data/lib/ziperu/file.rb +24 -2
- data/lib/ziperu/input_stream.rb +167 -11
- data/lib/ziperu/output_stream.rb +72 -1
- data/lib/ziperu/remote_io.rb +278 -0
- data/lib/ziperu/replace_rubyzip.rb +67 -0
- data/lib/ziperu/version.rb +1 -1
- data/lib/ziperu.rb +4 -0
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c150e0c8fc9674dbd9d76291707efdfab0904b3438ebe0c53c3f790db4fd8c81
|
|
4
|
+
data.tar.gz: d149abb6aacaddc8f2992462225f5e16aa2ff5e32247d7488312c2e3a726acc7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 2858d1912c604e90132efa21fc26cd73fcca7df0254b1a63d0183e2f4c41d313ce723bd3a4d5a56e358f9426ba9d3596279ba5fa9a302d459dba04f9fad2cb95
|
|
7
|
+
data.tar.gz: eaaf2c7c5a217e17d705d5d72acef3d1af2631f77a9750207133883c92a78a84a83945a420e7e273b9805db43c0985324ee52bc419ee580e50e91a300a38cba5
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,78 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.2.2 (2026-11-11)
|
|
4
|
+
|
|
5
|
+
Fixed:
|
|
6
|
+
|
|
7
|
+
- Duplicate entry names no longer get around the open-time limits. The
|
|
8
|
+
checks for `max_entries`, `max_total_size`, `max_entry_size` and
|
|
9
|
+
`max_compression_ratio` counted only the last record under each name,
|
|
10
|
+
while listing, `InputStream` and `extract_all` reach every one: a
|
|
11
|
+
zip bomb behind a harmless entry of the same name went through. Every
|
|
12
|
+
record now counts, in one pass.
|
|
13
|
+
- A large `read` interrupted (`Timeout`, `Thread#raise`) and then
|
|
14
|
+
rescued no longer loses data: what it had decompressed is returned by
|
|
15
|
+
the next read. After any other failure part-way through an entry,
|
|
16
|
+
further reads raise until `rewind`, instead of returning a short entry
|
|
17
|
+
with no error.
|
|
18
|
+
- `OutputStream.new(io, stream: true)` whose IO raises, with the error
|
|
19
|
+
rescued and writing carried on: the writer refuses every later call
|
|
20
|
+
rather than lose track of which entries the archive holds (which gave
|
|
21
|
+
a `NoMethodError` on close, or an entry silently overwritten), and no
|
|
22
|
+
longer writes again what it had handed the IO before it raised.
|
|
23
|
+
- `File#add`, `add_stored` and `replace` take a `Pathname` as the path it
|
|
24
|
+
is, not an IO (`Rails.root.join(...)` raised `NoMethodError`).
|
|
25
|
+
- On Windows, `File.open(dir, create: true)` and a commit no longer move
|
|
26
|
+
a directory at that path aside to put the archive in its place; the
|
|
27
|
+
commit raises, as rubyzip's does.
|
|
28
|
+
- Adding to an archive that is read-only (or has other hard links)
|
|
29
|
+
rewrites it instead of failing to append in place.
|
|
30
|
+
- Errors that came out as a bare `IOError` are `Ziperu::Error`s: corrupt
|
|
31
|
+
zstd and LZMA data (`DecompressionError`, as for Deflate), a local
|
|
32
|
+
header past the end of the archive or with a bad signature, a forged
|
|
33
|
+
zip64 offset no file can seek to. A NUL in a name raises
|
|
34
|
+
`ArgumentError` from the native `extract_all`, as from Ruby's own file
|
|
35
|
+
calls.
|
|
36
|
+
- `add_files(threads:)` (and the parallel commit) can be interrupted: it
|
|
37
|
+
stopped only once the whole batch, up to 256 MB, was compressed. It
|
|
38
|
+
still adds the batch whole or not at all.
|
|
39
|
+
- A local header that names a different compression method from the
|
|
40
|
+
central directory is refused, rather than decoded one way or the other
|
|
41
|
+
depending on `validate_entry_sizes`.
|
|
42
|
+
- An interrupted `extract_all` removes the file it was writing, so a
|
|
43
|
+
second run is not stopped by `DestinationExistsError`.
|
|
44
|
+
- Directory times restored by `extract_all` are set once the files in
|
|
45
|
+
the directory are written, which would otherwise move them (and every
|
|
46
|
+
directory entry is made before any file, wherever it is listed).
|
|
47
|
+
- An entry kept across a commit that moved it (the one `remove` returns,
|
|
48
|
+
say) still answers `time`, `extra` and the like from what it holds.
|
|
49
|
+
- `InputStream.open("missing.zip")` names the file in its `ENOENT`;
|
|
50
|
+
`File.open(nil)` raises `TypeError`.
|
|
51
|
+
|
|
52
|
+
Added:
|
|
53
|
+
|
|
54
|
+
- `require "ziperu/replace_rubyzip"`: ziperu answers for rubyzip
|
|
55
|
+
(`require "zip"`, `"rubyzip"`, `"zip/..."`) even without Bundler, where
|
|
56
|
+
RubyGems would otherwise load a rubyzip that a dependency asks for.
|
|
57
|
+
- `InputStream.open(pipe, streaming: true)`: a pipe or socket read in one
|
|
58
|
+
pass, local header by local header, in an entry's worth of memory, with
|
|
59
|
+
the limits counted as the bytes come. Entries whose sizes follow their
|
|
60
|
+
data (how streaming writers write) are read when Deflated.
|
|
61
|
+
- `Ziperu::RemoteIO`: a seekable IO over HTTP Range requests (or a block
|
|
62
|
+
that fetches byte ranges), so an archive on S3 or a web server can be
|
|
63
|
+
listed and have entries read without downloading it.
|
|
64
|
+
- `OutputStream#rollback_entry`: takes the entry being written back out
|
|
65
|
+
of the archive, so one whose data could not be had can be skipped. On
|
|
66
|
+
a stream it works until the entry's first bytes are written (the entry
|
|
67
|
+
is begun only then); on a file or seekable IO, at any point.
|
|
68
|
+
- `InputStream#seek` and `pos=` in a Stored entry that is not encrypted
|
|
69
|
+
(`get_input_stream(...).seek`): random access without reading what
|
|
70
|
+
comes before.
|
|
71
|
+
|
|
72
|
+
Changed:
|
|
73
|
+
|
|
74
|
+
- `open_buffer` on a String copies the archive once, not twice.
|
|
75
|
+
|
|
3
76
|
## 0.2.1 (2026-11-10)
|
|
4
77
|
|
|
5
78
|
Fixed:
|
data/Cargo.lock
CHANGED
data/README.md
CHANGED
|
@@ -165,6 +165,51 @@ Ziperu::InputStream.open(socket) do |zis| # any IO, no seeking needed
|
|
|
165
165
|
process(entry.name, zis.read)
|
|
166
166
|
end
|
|
167
167
|
end
|
|
168
|
+
|
|
169
|
+
# A pipe or an upload read in one pass, an entry's worth of memory at a
|
|
170
|
+
# time, rather than taken in whole to reach the central directory:
|
|
171
|
+
Ziperu::InputStream.open(s3_object.body, streaming: true) do |zis|
|
|
172
|
+
while (entry = zis.get_next_entry)
|
|
173
|
+
zis.each_line { |line| ... }
|
|
174
|
+
end
|
|
175
|
+
end
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Read an archive in place on a server, without downloading it: only the
|
|
179
|
+
central directory and the entries read are fetched, with HTTP Range
|
|
180
|
+
requests (S3, GCS, a CDN; or a block that returns byte ranges):
|
|
181
|
+
|
|
182
|
+
```ruby
|
|
183
|
+
io = Ziperu::RemoteIO.new(url) # "https://example.com/dataset.zip"
|
|
184
|
+
Ziperu::File.open_buffer(io, write: false) do |zf|
|
|
185
|
+
zf.extract("docs/guide.txt", "guide.txt") # a few requests, not 5 GB
|
|
186
|
+
end
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Leave out an entry whose data could not be had, and go on with the rest
|
|
190
|
+
(a download that failed half-way through a streamed zip, say):
|
|
191
|
+
|
|
192
|
+
```ruby
|
|
193
|
+
body = Ziperu::OutputStream.body do |zos|
|
|
194
|
+
files.each do |name, url|
|
|
195
|
+
zos.put_next_entry(name)
|
|
196
|
+
begin
|
|
197
|
+
fetch(url) { |chunk| zos << chunk }
|
|
198
|
+
rescue Net::HTTPError
|
|
199
|
+
zos.rollback_entry # before its first byte on a stream; any time on a file
|
|
200
|
+
end
|
|
201
|
+
end
|
|
202
|
+
end
|
|
203
|
+
[200, { "content-type" => "application/zip" }, body]
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Seek about in a Stored entry, for readers that want random access (a PDF
|
|
207
|
+
stored in an archive):
|
|
208
|
+
|
|
209
|
+
```ruby
|
|
210
|
+
Ziperu::File.open("scans.zip") do |zf|
|
|
211
|
+
zf.get_input_stream("scan.pdf") { |io| io.seek(-1024, IO::SEEK_END); io.read(1024) }
|
|
212
|
+
end
|
|
168
213
|
```
|
|
169
214
|
|
|
170
215
|
Passwords, both ways:
|
|
@@ -209,11 +254,17 @@ path, so `require "zip"` loads ziperu and those gems run on it; rubyzip
|
|
|
209
254
|
stays installed and unused. Checked with Bundler 2.3, 2.5 and 4.0, in
|
|
210
255
|
either Gemfile order.
|
|
211
256
|
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
257
|
+
Without Bundler, RubyGems loads whichever gem asks for `zip` first. To
|
|
258
|
+
have ziperu answer for rubyzip there too, require this before anything
|
|
259
|
+
else loads it (the top of a script, or `config/boot.rb`):
|
|
260
|
+
|
|
261
|
+
```ruby
|
|
262
|
+
require "ziperu/replace_rubyzip" # every require "zip" / "zip/..." loads ziperu
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
It raises if rubyzip is already loaded, rather than mix the two. Short of
|
|
266
|
+
that, use `require "ziperu"` and `Ziperu::` for your own code and leave
|
|
267
|
+
`Zip::` to rubyzip.
|
|
217
268
|
|
|
218
269
|
ziperu tracks the rubyzip 3.x API (`Ziperu::RUBYZIP_COMPAT_VERSION`), and
|
|
219
270
|
diverges on purpose where rubyzip is lax: `InputStream` reads through the
|
data/ext/ziperu/Cargo.toml
CHANGED
data/ext/ziperu/src/archive.rs
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
use std::cell::{RefCell, RefMut};
|
|
8
8
|
use std::collections::HashMap;
|
|
9
9
|
use std::fs;
|
|
10
|
-
use std::io::{self, Read
|
|
10
|
+
use std::io::{self, Read};
|
|
11
11
|
|
|
12
12
|
use magnus::{
|
|
13
13
|
gc::Marker, typed_data::DataTypeFunctions, value::Opaque, Error, RArray, RHash, RString, Ruby,
|
|
@@ -17,7 +17,7 @@ use zip::read::{ZipFileEntry, ZipReadOptions};
|
|
|
17
17
|
use zip::result::{ZipError, ZipResult};
|
|
18
18
|
use zip::ZipArchive;
|
|
19
19
|
|
|
20
|
-
use crate::errors::{busy_error, read_error, zip_read_error, IntoRuby};
|
|
20
|
+
use crate::errors::{busy_error, header_error, read_error, zip_read_error, IntoRuby};
|
|
21
21
|
use crate::gvl::{maybe_without_gvl, Cancellable};
|
|
22
22
|
use crate::rb::bytes_of;
|
|
23
23
|
use crate::reader::{open_local_entry, EntryInfo, EntryReader};
|
|
@@ -56,7 +56,11 @@ impl Archive {
|
|
|
56
56
|
}
|
|
57
57
|
|
|
58
58
|
pub fn open_bytes(ruby: &Ruby, bytes: RString) -> Result<Self, Error> {
|
|
59
|
-
|
|
59
|
+
// SAFETY: copied out at once into the Arc, with the GVL held and
|
|
60
|
+
// no Ruby call in between: one copy, rather than a Vec and then
|
|
61
|
+
// the Arc it is moved into (twice the archive's size, briefly).
|
|
62
|
+
let shared: std::sync::Arc<[u8]> = std::sync::Arc::from(unsafe { bytes.as_slice() });
|
|
63
|
+
Self::from_source(ruby, Source::memory(shared))
|
|
60
64
|
}
|
|
61
65
|
|
|
62
66
|
/// Any seekable IO-like Ruby object; read lazily, never copied.
|
|
@@ -130,50 +134,45 @@ impl Archive {
|
|
|
130
134
|
Ok(rb_self.archive(ruby)?.has_duplicate_names())
|
|
131
135
|
}
|
|
132
136
|
|
|
133
|
-
///
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
///
|
|
145
|
-
|
|
146
|
-
pub fn max_entry_size(ruby: &Ruby, rb_self: &Self) -> Result<u64, Error> {
|
|
147
|
-
let archive = rb_self.archive(ruby)?;
|
|
148
|
-
let meta = archive.metadata();
|
|
149
|
-
let mut largest = 0;
|
|
150
|
-
for i in 0..meta.len() {
|
|
151
|
-
largest = largest.max(meta.entry(i).rb(ruby)?.size());
|
|
152
|
-
}
|
|
153
|
-
Ok(largest)
|
|
154
|
-
}
|
|
155
|
-
|
|
156
|
-
/// The highest declared uncompressed-to-compressed size ratio among
|
|
157
|
-
/// the entries, for `Ziperu.max_compression_ratio`. An entry declaring
|
|
158
|
-
/// data but no compressed bytes is infinite. Also the bound on the
|
|
159
|
-
/// archive's overall ratio, which can be no higher than its highest
|
|
160
|
-
/// entry's.
|
|
161
|
-
pub fn max_compression_ratio(ruby: &Ruby, rb_self: &Self) -> Result<f64, Error> {
|
|
137
|
+
/// What the open-time limits look at, in one pass:
|
|
138
|
+
/// `[records, total_size, largest_size, highest_ratio]`.
|
|
139
|
+
///
|
|
140
|
+
/// Every record counts, including several under one name. The crate's
|
|
141
|
+
/// parsed view keeps only the last record of each name, but `entries`,
|
|
142
|
+
/// `InputStream` and `extract_all` reach every one of them, so the
|
|
143
|
+
/// limits must too: otherwise a bomb hides behind a harmless entry of
|
|
144
|
+
/// the same name.
|
|
145
|
+
///
|
|
146
|
+
/// An entry declaring data but no compressed bytes has an infinite
|
|
147
|
+
/// ratio. The archive's overall ratio can be no higher than its
|
|
148
|
+
/// highest entry's, so that bounds it too.
|
|
149
|
+
pub fn limits(ruby: &Ruby, rb_self: &Self) -> Result<(usize, u64, u64, f64), Error> {
|
|
162
150
|
let archive = rb_self.archive(ruby)?;
|
|
163
|
-
let
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
let
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
151
|
+
let mut summary = LimitSummary::default();
|
|
152
|
+
if archive.has_duplicate_names() {
|
|
153
|
+
let dir_start = archive.central_directory_start();
|
|
154
|
+
let mut raw = archive.clone().into_inner();
|
|
155
|
+
drop(archive);
|
|
156
|
+
let records = maybe_without_gvl(rb_self.needs_gvl, || {
|
|
157
|
+
walk_central_directory(&mut raw, dir_start)
|
|
158
|
+
})
|
|
159
|
+
.map_err(|e| header_error(ruby, e))?;
|
|
160
|
+
for record in &records {
|
|
161
|
+
summary.add(record.size, record.compressed_size);
|
|
170
162
|
}
|
|
171
|
-
|
|
172
|
-
|
|
163
|
+
} else {
|
|
164
|
+
let meta = archive.metadata();
|
|
165
|
+
for i in 0..meta.len() {
|
|
166
|
+
let entry = meta.entry(i).rb(ruby)?;
|
|
167
|
+
summary.add(entry.size(), entry.compressed_size());
|
|
173
168
|
}
|
|
174
|
-
worst = worst.max(size as f64 / csize as f64);
|
|
175
169
|
}
|
|
176
|
-
Ok(
|
|
170
|
+
Ok((
|
|
171
|
+
summary.records,
|
|
172
|
+
summary.total,
|
|
173
|
+
summary.largest,
|
|
174
|
+
summary.ratio,
|
|
175
|
+
))
|
|
177
176
|
}
|
|
178
177
|
|
|
179
178
|
/// Inflates entry `index` to nowhere so the crate checks its CRC (and
|
|
@@ -216,6 +215,7 @@ impl Archive {
|
|
|
216
215
|
size,
|
|
217
216
|
compressed_size,
|
|
218
217
|
crc32: crc,
|
|
218
|
+
method: None,
|
|
219
219
|
};
|
|
220
220
|
maybe_without_gvl(rb_self.needs_gvl, || {
|
|
221
221
|
let mut entry = open_local_entry(&mut source, &info, password.as_deref(), size)?;
|
|
@@ -267,7 +267,7 @@ impl Archive {
|
|
|
267
267
|
let records = maybe_without_gvl(rb_self.needs_gvl, || {
|
|
268
268
|
walk_central_directory(&mut raw, dir_start)
|
|
269
269
|
})
|
|
270
|
-
.
|
|
270
|
+
.map_err(|e| header_error(ruby, e))?;
|
|
271
271
|
let mut by_offset = HashMap::with_capacity(meta.len());
|
|
272
272
|
for i in 0..meta.len() {
|
|
273
273
|
let entry = meta.entry(i).rb(ruby)?;
|
|
@@ -288,7 +288,10 @@ impl Archive {
|
|
|
288
288
|
})
|
|
289
289
|
.collect::<ZipResult<Vec<_>>>()
|
|
290
290
|
})
|
|
291
|
-
.
|
|
291
|
+
.map_err(|e| match e {
|
|
292
|
+
ZipError::Io(e) => header_error(ruby, e),
|
|
293
|
+
other => crate::errors::zip_error(ruby, other),
|
|
294
|
+
})?;
|
|
292
295
|
for (i, record) in records.iter().enumerate() {
|
|
293
296
|
let entry = meta.entry(i).rb(ruby)?;
|
|
294
297
|
emit(entry_hash(ruby, &keys, i, &entry, record)?)?;
|
|
@@ -303,7 +306,8 @@ impl Archive {
|
|
|
303
306
|
let mut raw = small_source(&archive);
|
|
304
307
|
drop(archive);
|
|
305
308
|
let entry = meta.entry(index).rb(ruby)?;
|
|
306
|
-
let record = raw_central_record(&mut raw, entry.central_header_start())
|
|
309
|
+
let record = raw_central_record(&mut raw, entry.central_header_start())
|
|
310
|
+
.map_err(|e| header_error(ruby, e))?;
|
|
307
311
|
entry_hash(ruby, &MetaKeys::new(ruby), index, &entry, &record)
|
|
308
312
|
}
|
|
309
313
|
|
|
@@ -325,7 +329,8 @@ impl Archive {
|
|
|
325
329
|
let out = ruby.ary_new_capa(end.saturating_sub(index));
|
|
326
330
|
for i in index..end {
|
|
327
331
|
let entry = meta.entry(i).rb(ruby)?;
|
|
328
|
-
let record = raw_central_record(&mut raw, entry.central_header_start())
|
|
332
|
+
let record = raw_central_record(&mut raw, entry.central_header_start())
|
|
333
|
+
.map_err(|e| header_error(ruby, e))?;
|
|
329
334
|
out.push(entry_hash(ruby, &keys, i, &entry, &record)?)?;
|
|
330
335
|
}
|
|
331
336
|
Ok(out)
|
|
@@ -339,7 +344,7 @@ impl Archive {
|
|
|
339
344
|
let header_start = archive.metadata().entry(index).rb(ruby)?.header_start();
|
|
340
345
|
let mut raw = small_source(&archive);
|
|
341
346
|
drop(archive);
|
|
342
|
-
let extra = raw_local_extra(&mut raw, header_start).
|
|
347
|
+
let extra = raw_local_extra(&mut raw, header_start).map_err(|e| header_error(ruby, e))?;
|
|
343
348
|
Ok(ruby.str_from_slice(&extra))
|
|
344
349
|
}
|
|
345
350
|
|
|
@@ -350,7 +355,7 @@ impl Archive {
|
|
|
350
355
|
header_offset: u64,
|
|
351
356
|
) -> Result<RString, Error> {
|
|
352
357
|
let mut raw = rb_self.small_source(ruby)?;
|
|
353
|
-
let extra = raw_local_extra(&mut raw, header_offset).
|
|
358
|
+
let extra = raw_local_extra(&mut raw, header_offset).map_err(|e| header_error(ruby, e))?;
|
|
354
359
|
Ok(ruby.str_from_slice(&extra))
|
|
355
360
|
}
|
|
356
361
|
|
|
@@ -399,6 +404,7 @@ impl Archive {
|
|
|
399
404
|
size,
|
|
400
405
|
compressed_size,
|
|
401
406
|
crc32: crc,
|
|
407
|
+
method: None,
|
|
402
408
|
};
|
|
403
409
|
EntryReader::open_at(
|
|
404
410
|
ruby,
|
|
@@ -446,6 +452,31 @@ impl Archive {
|
|
|
446
452
|
}
|
|
447
453
|
}
|
|
448
454
|
|
|
455
|
+
/// The figures `Archive::limits` returns.
|
|
456
|
+
#[derive(Default)]
|
|
457
|
+
struct LimitSummary {
|
|
458
|
+
records: usize,
|
|
459
|
+
total: u64,
|
|
460
|
+
largest: u64,
|
|
461
|
+
ratio: f64,
|
|
462
|
+
}
|
|
463
|
+
|
|
464
|
+
impl LimitSummary {
|
|
465
|
+
fn add(&mut self, size: u64, compressed_size: u64) {
|
|
466
|
+
self.records += 1;
|
|
467
|
+
self.total = self.total.saturating_add(size);
|
|
468
|
+
self.largest = self.largest.max(size);
|
|
469
|
+
if size > 0 {
|
|
470
|
+
let ratio = if compressed_size == 0 {
|
|
471
|
+
f64::INFINITY
|
|
472
|
+
} else {
|
|
473
|
+
size as f64 / compressed_size as f64
|
|
474
|
+
};
|
|
475
|
+
self.ratio = self.ratio.max(ratio);
|
|
476
|
+
}
|
|
477
|
+
}
|
|
478
|
+
}
|
|
479
|
+
|
|
449
480
|
/// A central directory record read straight from the bytes.
|
|
450
481
|
///
|
|
451
482
|
/// Two fields are read this way for every entry, where the crate's parsed
|
|
@@ -496,7 +527,7 @@ fn walk_central_directory(
|
|
|
496
527
|
source: &mut Source,
|
|
497
528
|
dir_start: u64,
|
|
498
529
|
) -> io::Result<Vec<RawCentralRecord>> {
|
|
499
|
-
source.
|
|
530
|
+
source.seek_to(dir_start)?;
|
|
500
531
|
let mut records = Vec::new();
|
|
501
532
|
let mut offset = dir_start;
|
|
502
533
|
while let Some(record) = read_central_record(source, offset)? {
|
|
@@ -604,7 +635,7 @@ fn raw_local_extra(source: &mut Source, header_offset: u64) -> io::Result<Vec<u8
|
|
|
604
635
|
const SIGNATURE: u32 = 0x0403_4b50;
|
|
605
636
|
const HEADER_LEN: usize = 30;
|
|
606
637
|
|
|
607
|
-
source.
|
|
638
|
+
source.seek_to(header_offset)?;
|
|
608
639
|
let mut header = [0u8; HEADER_LEN];
|
|
609
640
|
source.read_exact(&mut header)?;
|
|
610
641
|
if u32::from_le_bytes([header[0], header[1], header[2], header[3]]) != SIGNATURE {
|
|
@@ -613,9 +644,9 @@ fn raw_local_extra(source: &mut Source, header_offset: u64) -> io::Result<Vec<u8
|
|
|
613
644
|
"local file header signature not found",
|
|
614
645
|
));
|
|
615
646
|
}
|
|
616
|
-
let name_len = u16::from_le_bytes([header[26], header[27]]) as
|
|
647
|
+
let name_len = u16::from_le_bytes([header[26], header[27]]) as u64;
|
|
617
648
|
let extra_len = u16::from_le_bytes([header[28], header[29]]) as usize;
|
|
618
|
-
source.
|
|
649
|
+
source.seek_to(header_offset + HEADER_LEN as u64 + name_len)?;
|
|
619
650
|
let mut extra = vec![0u8; extra_len];
|
|
620
651
|
source.read_exact(&mut extra)?;
|
|
621
652
|
Ok(extra)
|
|
@@ -738,7 +769,7 @@ fn record_hash(
|
|
|
738
769
|
/// `to_u16` is deprecated in favour of matching on constants, but it is the
|
|
739
770
|
/// only public way to recover the raw code, which is what we hand to Ruby.
|
|
740
771
|
#[allow(deprecated)]
|
|
741
|
-
fn compression_code(method: zip::CompressionMethod) -> u16 {
|
|
772
|
+
pub(crate) fn compression_code(method: zip::CompressionMethod) -> u16 {
|
|
742
773
|
method.to_u16()
|
|
743
774
|
}
|
|
744
775
|
|
data/ext/ziperu/src/errors.rs
CHANGED
|
@@ -27,7 +27,7 @@ exception!(pub ERROR, "Ziperu::Error");
|
|
|
27
27
|
exception!(DECOMPRESSION_ERROR, "Ziperu::DecompressionError");
|
|
28
28
|
exception!(NATIVE_SIZE_LIMIT_ERROR, "Ziperu::Native::SizeLimitError");
|
|
29
29
|
exception!(NATIVE_PASSWORD_ERROR, "Ziperu::Native::PasswordError");
|
|
30
|
-
exception!(NATIVE_UNSUPPORTED_ERROR, "Ziperu::Native::UnsupportedError");
|
|
30
|
+
exception!(pub NATIVE_UNSUPPORTED_ERROR, "Ziperu::Native::UnsupportedError");
|
|
31
31
|
exception!(pub DESTINATION_EXISTS_ERROR, "Ziperu::DestinationExistsError");
|
|
32
32
|
exception!(pub ENTRY_SIZE_ERROR, "Ziperu::EntrySizeError");
|
|
33
33
|
exception!(pub NATIVE_DUPLICATE_ENTRY_ERROR, "Ziperu::Native::DuplicateEntryError");
|
|
@@ -96,10 +96,61 @@ fn invalid_archive_message(msg: &str) -> String {
|
|
|
96
96
|
if msg.contains("EOCD") {
|
|
97
97
|
"Zip end of central directory signature not found".to_string()
|
|
98
98
|
} else {
|
|
99
|
-
|
|
99
|
+
// Some of the crate's messages carry a placeholder it never fills.
|
|
100
|
+
msg.replace(" of {properties_size}", "")
|
|
100
101
|
}
|
|
101
102
|
}
|
|
102
103
|
|
|
104
|
+
/// `io_error` for a failure while reading an archive's own structure (a
|
|
105
|
+
/// central record, a local header): bytes that end early or make no
|
|
106
|
+
/// sense are a broken archive, `Ziperu::Error`, not an `IOError`.
|
|
107
|
+
pub fn header_error(ruby: &Ruby, err: io::Error) -> Error {
|
|
108
|
+
if is_foreign(&err) || crate::gvl::is_cancelled(&err) {
|
|
109
|
+
return io_error(ruby, err);
|
|
110
|
+
}
|
|
111
|
+
match err.kind() {
|
|
112
|
+
io::ErrorKind::UnexpectedEof => Error::new(
|
|
113
|
+
ruby.get_inner(&ERROR),
|
|
114
|
+
"invalid Zip archive: a header runs past the end of the data",
|
|
115
|
+
),
|
|
116
|
+
io::ErrorKind::InvalidData | io::ErrorKind::InvalidInput => Error::new(
|
|
117
|
+
ruby.get_inner(&ERROR),
|
|
118
|
+
format!("invalid Zip archive: {err}"),
|
|
119
|
+
),
|
|
120
|
+
_ => io_error(ruby, err),
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
/// An error that carries a Ruby exception or one of the crate's own.
|
|
125
|
+
fn is_foreign(err: &io::Error) -> bool {
|
|
126
|
+
err.get_ref()
|
|
127
|
+
.is_some_and(|e| e.is::<RubyError>() || e.is::<ZipError>())
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
/// The kinds that are about the file system or the machine rather than
|
|
131
|
+
/// the bytes being decoded.
|
|
132
|
+
fn is_system(kind: io::ErrorKind) -> bool {
|
|
133
|
+
use io::ErrorKind::*;
|
|
134
|
+
matches!(
|
|
135
|
+
kind,
|
|
136
|
+
NotFound
|
|
137
|
+
| PermissionDenied
|
|
138
|
+
| AlreadyExists
|
|
139
|
+
| IsADirectory
|
|
140
|
+
| NotADirectory
|
|
141
|
+
| StorageFull
|
|
142
|
+
| WriteZero
|
|
143
|
+
| Interrupted
|
|
144
|
+
| OutOfMemory
|
|
145
|
+
| BrokenPipe
|
|
146
|
+
| ResourceBusy
|
|
147
|
+
| ReadOnlyFilesystem
|
|
148
|
+
| QuotaExceeded
|
|
149
|
+
| FileTooLarge
|
|
150
|
+
| TimedOut
|
|
151
|
+
)
|
|
152
|
+
}
|
|
153
|
+
|
|
103
154
|
/// Raised when a thread calls into a native object another thread is
|
|
104
155
|
/// using with the GVL released (an inflate or deflate in progress).
|
|
105
156
|
pub fn busy_error(ruby: &Ruby, what: &str) -> Error {
|
|
@@ -139,24 +190,35 @@ pub fn io_error(ruby: &Ruby, err: io::Error) -> Error {
|
|
|
139
190
|
io::ErrorKind::NotADirectory => Error::new(ruby.get_inner(&ENOTDIR), err.to_string()),
|
|
140
191
|
_ if is_size_limit(&err) => size_limit_error(ruby),
|
|
141
192
|
io::ErrorKind::InvalidData => decompression_error(ruby, &err.to_string()),
|
|
193
|
+
// An archive that ends early, or an offset no file can seek to
|
|
194
|
+
// (a forged zip64 field): a broken archive.
|
|
195
|
+
io::ErrorKind::UnexpectedEof => Error::new(
|
|
196
|
+
ruby.get_inner(&ERROR),
|
|
197
|
+
format!("invalid Zip archive: unexpected end of data ({err})"),
|
|
198
|
+
),
|
|
199
|
+
io::ErrorKind::InvalidInput => Error::new(
|
|
200
|
+
ruby.get_inner(&ERROR),
|
|
201
|
+
format!("invalid Zip archive: {err}"),
|
|
202
|
+
),
|
|
142
203
|
_ => Error::new(ruby.exception_io_error(), err.to_string()),
|
|
143
204
|
}
|
|
144
205
|
}
|
|
145
206
|
|
|
146
|
-
/// `io_error` for a failure while reading an entry's data
|
|
147
|
-
///
|
|
148
|
-
///
|
|
149
|
-
/// is
|
|
207
|
+
/// `io_error` for a failure while reading an entry's data. The decoders
|
|
208
|
+
/// each report corrupt input their own way (zlib as `InvalidInput`, a
|
|
209
|
+
/// truncated stream as `UnexpectedEof`, zstd and lzma as `Other` with a
|
|
210
|
+
/// message): all of it is rubyzip's `DecompressionError`, which is what
|
|
211
|
+
/// callers rescue around a read. A file system error, an exception from a
|
|
212
|
+
/// Ruby IO, an interrupt and the crate's own errors keep their own class.
|
|
150
213
|
pub fn read_error(ruby: &Ruby, err: io::Error) -> Error {
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
}
|
|
158
|
-
_ => io_error(ruby, err),
|
|
214
|
+
if is_foreign(&err)
|
|
215
|
+
|| crate::gvl::is_cancelled(&err)
|
|
216
|
+
|| is_size_limit(&err)
|
|
217
|
+
|| is_system(err.kind())
|
|
218
|
+
{
|
|
219
|
+
return io_error(ruby, err);
|
|
159
220
|
}
|
|
221
|
+
decompression_error(ruby, &err.to_string())
|
|
160
222
|
}
|
|
161
223
|
|
|
162
224
|
/// Work cut short because Ruby interrupted the thread (see `gvl`): the
|