ziperu 0.2.2-aarch64-linux-gnu → 0.2.3-aarch64-linux-gnu
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +46 -0
- data/README.md +22 -30
- data/lib/ziperu/3.1/ziperu.so +0 -0
- data/lib/ziperu/3.2/ziperu.so +0 -0
- data/lib/ziperu/3.3/ziperu.so +0 -0
- data/lib/ziperu/3.4/ziperu.so +0 -0
- data/lib/ziperu/4.0/ziperu.so +0 -0
- data/lib/ziperu/entry/source.rb +6 -0
- data/lib/ziperu/entry.rb +58 -7
- data/lib/ziperu/file.rb +40 -4
- data/lib/ziperu/input_stream.rb +24 -8
- data/lib/ziperu/output_stream.rb +10 -0
- data/lib/ziperu/streamable_stream.rb +110 -10
- data/lib/ziperu/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: fc523b92d085a00db7c6c8c0d791ef38b536ab76a05bc40cac4b035c44988a31
|
|
4
|
+
data.tar.gz: b0bbc08f5320631828ea7d2081704bfab4f7018e661e46b27c8641efcb9976c5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1e1b48ff1d095cdf3b4ef74677199ef9a846e6c67348bac777fc4dad8f26ca363b9a923731990b1b2b893dd29c1b19b733ac2796fe2e532b1176ad0ac4c6ddf6
|
|
7
|
+
data.tar.gz: d363ffaca6f3ef09f59d63a26d1478300aa994511b3a8448b36786e0a4ca18f26037d6bb41e270f3123c070811ba7ab2387c121273af9ffae82ab78a0ad23966
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,51 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.2.3 (2026-11-11)
|
|
4
|
+
|
|
5
|
+
Faster (Windows, against 0.2.2):
|
|
6
|
+
|
|
7
|
+
- Writing many small entries to a file: the header each entry patches
|
|
8
|
+
at its end is now still in the write buffer, so it costs no system
|
|
9
|
+
call. 1000 entries of 8 KB: 46 to 32 ms; 10,000 tiny entries: 315 to
|
|
10
|
+
180 ms. A seekable Ruby IO (`write_buffer`) gets the same, in 64 KB
|
|
11
|
+
chunks.
|
|
12
|
+
- `File#get_output_stream` keeps an entry in memory until the commit
|
|
13
|
+
rather than opening a tempfile for each one; past 8 MB held between
|
|
14
|
+
an archive's pending entries, the rest go to tempfiles as before.
|
|
15
|
+
1000 entries and a commit: 981 to 58 ms.
|
|
16
|
+
- `Entry#time` reads the modification time from the central directory,
|
|
17
|
+
as rubyzip does, rather than the entry's local header; the access and
|
|
18
|
+
creation times still come from there. Listing names and times of
|
|
19
|
+
10,000 entries: 153 to 78 ms.
|
|
20
|
+
- Appending to a large archive no longer reads every entry's local
|
|
21
|
+
header, reads the central directory through a buffer, and keeps the
|
|
22
|
+
entries it had after the commit (same objects) rather than building
|
|
23
|
+
them all again. One entry added to a 10,000-entry archive: 337 to
|
|
24
|
+
98 ms.
|
|
25
|
+
- `InputStream#get_next_entry` opens the entry's data when it is first
|
|
26
|
+
read, so walking only the names reads no local header: 103 to 40 ms
|
|
27
|
+
for 10,000 entries.
|
|
28
|
+
- Reading a whole entry of 64 KB or more (`File#read`, and
|
|
29
|
+
`get_input_stream`'s `read` of the rest of one, which is what roo and
|
|
30
|
+
caxlsx do) decompresses straight into the returned String: no second
|
|
31
|
+
copy, and half the peak memory. 64 MB: 58 to 17 ms; a 64 MB Stored
|
|
32
|
+
entry: 62 to 18 ms. An interrupted read rescued still loses nothing.
|
|
33
|
+
- Files added by path (`add`, `add_stored`, `replace`) are read by the
|
|
34
|
+
native core at commit instead of through Ruby: a 64 MB `add_stored`
|
|
35
|
+
and commit, 141 to 34 ms.
|
|
36
|
+
|
|
37
|
+
Fixed:
|
|
38
|
+
|
|
39
|
+
- Native work could not be interrupted, and kept every other thread
|
|
40
|
+
waiting, whenever Ruby declined to release the GVL for it because an
|
|
41
|
+
interrupt flag was set (often just its thread-scheduling timer): it
|
|
42
|
+
then ran to the end with the GVL held. Now a pending `Thread#raise`,
|
|
43
|
+
`Timeout` or Ctrl-C stops it at once, and otherwise the GVL is
|
|
44
|
+
released on another try.
|
|
45
|
+
- `get_input_stream.read` of an entry whose central record declares it
|
|
46
|
+
empty but gives a CRC of some data returned `""`; it raises, as
|
|
47
|
+
`File#read` does.
|
|
48
|
+
|
|
3
49
|
## 0.2.2 (2026-11-11)
|
|
4
50
|
|
|
5
51
|
Fixed:
|
data/README.md
CHANGED
|
@@ -19,6 +19,19 @@ the [`zip`](https://github.com/zip-rs/zip2) crate through
|
|
|
19
19
|
[Magnus](https://github.com/matsadler/magnus). `require "zip"` aliases
|
|
20
20
|
`Zip = Ziperu`, so existing code runs unchanged.
|
|
21
21
|
|
|
22
|
+
On GitHub's Linux runners (Ruby 3.4, median of 5), against rubyzip 3.7.0:
|
|
23
|
+
|
|
24
|
+
| scenario | rubyzip | ziperu | speedup |
|
|
25
|
+
|---|---:|---:|---:|
|
|
26
|
+
| read 1000 x 8 KB deflated entries | 80 ms | 14 ms | 5.8x |
|
|
27
|
+
| open and list 10000 entries | 135 ms | 58 ms | 2.3x |
|
|
28
|
+
| deflate one 64 MB entry | 188 ms | 77 ms | 2.4x |
|
|
29
|
+
| extract_all 1000 entries | 169 ms | 80 ms | 2.1x |
|
|
30
|
+
| add 1000 files from disk (`threads: 4`) | 214 ms | 60 ms | 3.6x |
|
|
31
|
+
|
|
32
|
+
macOS and Windows, every scenario, and which calls release the GVL (what
|
|
33
|
+
Puma users ask about) are in [docs/benchmarks.md](docs/benchmarks.md).
|
|
34
|
+
|
|
22
35
|
- **Precompiled** for Linux (glibc and musl), macOS and Windows, Ruby 3.1 to 4.0. No toolchain, no runtime dependencies.
|
|
23
36
|
- **Fast**: entries inflate and deflate in Rust with the GVL released, the central directory is parsed once, unchanged entries are copied byte for byte on commit, and a commit that only adds entries appends them in place.
|
|
24
37
|
- **Reads everything**: Stored, Deflate, Deflate64, bzip2, zstd, LZMA, XZ, PPMd and the PKZIP 1.x methods; AES and ZipCrypto. Writes Deflate, Stored, bzip2, zstd, XZ, PPMd, AES and ZipCrypto.
|
|
@@ -40,25 +53,8 @@ Or on its own:
|
|
|
40
53
|
gem install ziperu
|
|
41
54
|
```
|
|
42
55
|
|
|
43
|
-
|
|
44
|
-
(
|
|
45
|
-
`aarch64-linux-musl`, `arm-linux-musl`, `x86_64-darwin`, `arm64-darwin`,
|
|
46
|
-
`x64-mingw-ucrt`, `aarch64-mingw-ucrt`), each holding the extension for
|
|
47
|
-
Ruby 3.1 through 4.0. The glibc gems need RubyGems 3.3.22 or later (Ruby
|
|
48
|
-
3.1.4 ships it); the musl gems need `libgcc` (`apk add libgcc`, already in
|
|
49
|
-
the `ruby:3.2-alpine` and later images). Anywhere else the source gem
|
|
50
|
-
builds with Rust 1.88+.
|
|
51
|
-
|
|
52
|
-
Which Rubies, and how firmly:
|
|
53
|
-
|
|
54
|
-
| Ruby | Status |
|
|
55
|
-
| --- | --- |
|
|
56
|
-
| 3.3, 3.4, 4.0 | Supported. The suite runs on Linux, macOS and Windows on every change; binaries ship. |
|
|
57
|
-
| 3.1, 3.2 | Best effort. Past end of life upstream; binaries still ship and the suite runs on 3.1, but a fix that needs a newer Ruby may drop them, in a minor release and with a changelog entry. |
|
|
58
|
-
| ruby-head | Watched. A non-blocking CI job runs the suite nightly, so a new Ruby gets binaries the day it is released. |
|
|
59
|
-
|
|
60
|
-
JRuby and TruffleRuby are not supported: the core is a native extension
|
|
61
|
-
against the CRuby API. rubyzip remains the library for those.
|
|
56
|
+
Precompiled gems ship for Linux, macOS and Windows; see
|
|
57
|
+
[Platforms](#platforms) for the details.
|
|
62
58
|
|
|
63
59
|
## Quick start
|
|
64
60
|
|
|
@@ -284,18 +280,14 @@ dump, reads entries in large chunks, copies unchanged entries byte for
|
|
|
284
280
|
byte on `commit`, and releases the GVL while inflating from a file or
|
|
285
281
|
String buffer and while deflating to a file, so other Ruby threads keep running.
|
|
286
282
|
|
|
287
|
-
|
|
283
|
+
## Platforms
|
|
288
284
|
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
| add 1000 files from disk (`threads: 4`) | 214 ms | 60 ms | 3.6x |
|
|
296
|
-
|
|
297
|
-
macOS and Windows, every scenario, and which calls release the GVL (what
|
|
298
|
-
Puma users ask about) are in [docs/benchmarks.md](docs/benchmarks.md).
|
|
285
|
+
- **Ruby 3.3 to 4.0** are supported and tested on Linux, macOS and Windows.
|
|
286
|
+
3.1 and 3.2 still get binaries, best effort.
|
|
287
|
+
- **Precompiled** for x86_64 and aarch64 Linux (glibc and musl; musl needs
|
|
288
|
+
`libgcc`), Intel and Apple Silicon macOS, and x64 and arm64 Windows.
|
|
289
|
+
Elsewhere the source gem builds with Rust 1.88+.
|
|
290
|
+
- **CRuby only**: on JRuby or TruffleRuby, use rubyzip.
|
|
299
291
|
|
|
300
292
|
## Development
|
|
301
293
|
|
data/lib/ziperu/3.1/ziperu.so
CHANGED
|
Binary file
|
data/lib/ziperu/3.2/ziperu.so
CHANGED
|
Binary file
|
data/lib/ziperu/3.3/ziperu.so
CHANGED
|
Binary file
|
data/lib/ziperu/3.4/ziperu.so
CHANGED
|
Binary file
|
data/lib/ziperu/4.0/ziperu.so
CHANGED
|
Binary file
|
data/lib/ziperu/entry/source.rb
CHANGED
|
@@ -109,6 +109,12 @@ module Ziperu
|
|
|
109
109
|
@index ? archive.local_extra(@index) : archive.local_extra_at(offset)
|
|
110
110
|
end
|
|
111
111
|
|
|
112
|
+
# Moves the record to `native`, which holds it at the same index:
|
|
113
|
+
# the archive reopened after entries were appended to it.
|
|
114
|
+
def rebind(native)
|
|
115
|
+
@native = native
|
|
116
|
+
end
|
|
117
|
+
|
|
112
118
|
def detached
|
|
113
119
|
NONE
|
|
114
120
|
end
|
data/lib/ziperu/entry.rb
CHANGED
|
@@ -108,6 +108,17 @@ module Ziperu
|
|
|
108
108
|
end
|
|
109
109
|
private :parsed_extra
|
|
110
110
|
|
|
111
|
+
# The fields as far as the central directory has them (with any local
|
|
112
|
+
# ones already merged in, and edits), or nil when it has none: no
|
|
113
|
+
# local header is read.
|
|
114
|
+
def central_extra
|
|
115
|
+
return @extra if @extra
|
|
116
|
+
return nil if @extra_raw.nil? || @extra_raw.empty?
|
|
117
|
+
|
|
118
|
+
parsed_extra
|
|
119
|
+
end
|
|
120
|
+
private :central_extra
|
|
121
|
+
|
|
111
122
|
# Whether the extra fields differ from what the archive holds. Such an
|
|
112
123
|
# edit is written whenever the archive is next committed, but does not
|
|
113
124
|
# call for a commit by itself, as in rubyzip: `FileSystem#chown` edits
|
|
@@ -129,6 +140,12 @@ module Ziperu
|
|
|
129
140
|
|
|
130
141
|
attr_writer :size # :nodoc:
|
|
131
142
|
|
|
143
|
+
# Reads from `native` from now on, which holds the entry's record at
|
|
144
|
+
# the same index (see Ziperu::File#refresh_after_append).
|
|
145
|
+
def rebind_native(native) # :nodoc:
|
|
146
|
+
@source.rebind(native)
|
|
147
|
+
end
|
|
148
|
+
|
|
132
149
|
# Index of the entry in the archive's central directory, or nil for an
|
|
133
150
|
# entry that is not (yet) in an archive, or that the native core cannot
|
|
134
151
|
# address by index: one of several records under the same name, which
|
|
@@ -249,12 +266,20 @@ module Ziperu
|
|
|
249
266
|
|
|
250
267
|
# Get a timestamp component of this entry. Returns modification time by
|
|
251
268
|
# default.
|
|
269
|
+
#
|
|
270
|
+
# The modification time comes from the central directory's fields,
|
|
271
|
+
# which carry it in full, as rubyzip reads it; the access and creation
|
|
272
|
+
# times are only in the local header's, which is read for them. Listing
|
|
273
|
+
# the times of a large archive so reads no local header.
|
|
252
274
|
def time(component: :mtime)
|
|
275
|
+
fields = component == :mtime ? central_extra : extra
|
|
253
276
|
time =
|
|
254
|
-
if
|
|
255
|
-
|
|
256
|
-
elsif
|
|
257
|
-
|
|
277
|
+
if fields.nil?
|
|
278
|
+
nil
|
|
279
|
+
elsif fields[:universaltime]
|
|
280
|
+
fields[:universaltime].send(component)
|
|
281
|
+
elsif fields[:ntfs]
|
|
282
|
+
fields[:ntfs].send(component)
|
|
258
283
|
end
|
|
259
284
|
|
|
260
285
|
time || (stored_time if component == :mtime)
|
|
@@ -307,7 +332,8 @@ module Ziperu
|
|
|
307
332
|
|
|
308
333
|
# Does this entry return time fields with accurate timezone information?
|
|
309
334
|
def absolute_time?
|
|
310
|
-
|
|
335
|
+
fields = central_extra
|
|
336
|
+
!fields.nil? && (fields.member?(:universaltime) || fields.member?(:ntfs))
|
|
311
337
|
end
|
|
312
338
|
|
|
313
339
|
# Returns STORED if the entry is a directory or the compression level is 0.
|
|
@@ -554,8 +580,12 @@ module Ziperu
|
|
|
554
580
|
zip_output_stream.put_next_entry(self)
|
|
555
581
|
when :restream
|
|
556
582
|
zip_output_stream.put_next_entry(self)
|
|
557
|
-
|
|
558
|
-
|
|
583
|
+
if (path = native_source_path)
|
|
584
|
+
zip_output_stream.write_file(path)
|
|
585
|
+
else
|
|
586
|
+
get_input_stream do |is|
|
|
587
|
+
::Ziperu::IOExtras.copy_stream(zip_output_stream, is)
|
|
588
|
+
end
|
|
559
589
|
end
|
|
560
590
|
when :copy_encrypted
|
|
561
591
|
zip_output_stream.copy_encrypted_entry(self)
|
|
@@ -572,6 +602,19 @@ module Ziperu
|
|
|
572
602
|
# its own headers, `:copy_raw` otherwise. Raises, with the reason, for
|
|
573
603
|
# an entry that would have to be restreamed but cannot be (encrypted,
|
|
574
604
|
# or a compression method the writer lacks).
|
|
605
|
+
# The file on disk a regular file entry added from a path is written
|
|
606
|
+
# from, when the native core can open it by name; nil otherwise.
|
|
607
|
+
def native_source_path # :nodoc:
|
|
608
|
+
return nil unless pending_data? && ftype == :file
|
|
609
|
+
|
|
610
|
+
path = @source.path
|
|
611
|
+
return nil unless path
|
|
612
|
+
|
|
613
|
+
path = path.to_path if path.respond_to?(:to_path)
|
|
614
|
+
path = path.to_s
|
|
615
|
+
path if path.ascii_only? || (path.encoding == ::Encoding::UTF_8 && path.valid_encoding?)
|
|
616
|
+
end
|
|
617
|
+
|
|
575
618
|
def write_plan # :nodoc:
|
|
576
619
|
# A symlink's target goes into the header call, whether the link is
|
|
577
620
|
# new or a copy that has to be written afresh (an edit, or extra
|
|
@@ -615,6 +658,14 @@ module Ziperu
|
|
|
615
658
|
restreamable? && (extra.keys & RAW_COPY_LOSES).any?
|
|
616
659
|
end
|
|
617
660
|
|
|
661
|
+
# Whether appending to the archive, which leaves this entry's local
|
|
662
|
+
# header alone but has the crate write its central record again,
|
|
663
|
+
# would lose fields of that record.
|
|
664
|
+
def central_fields_lost_on_append? # :nodoc:
|
|
665
|
+
fields = central_extra
|
|
666
|
+
!fields.nil? && restreamable? && (fields.keys & RAW_COPY_LOSES).any?
|
|
667
|
+
end
|
|
668
|
+
|
|
618
669
|
# Whether the writer can decompress and rewrite the entry: it is not
|
|
619
670
|
# encrypted and its compression method is one the writer has.
|
|
620
671
|
def restreamable? # :nodoc:
|
data/lib/ziperu/file.rb
CHANGED
|
@@ -288,7 +288,7 @@ module Ziperu
|
|
|
288
288
|
"cannot open stream to directory entry - '#{new_entry}'"
|
|
289
289
|
end
|
|
290
290
|
new_entry.unix_perms = permissions
|
|
291
|
-
zip_streamable_entry = StreamableStream.new(new_entry)
|
|
291
|
+
zip_streamable_entry = StreamableStream.new(new_entry, @spool_budget ||= SpoolIO::Budget.new)
|
|
292
292
|
@cdir << zip_streamable_entry
|
|
293
293
|
@dirty = true
|
|
294
294
|
zip_streamable_entry.get_output_stream(&a_proc)
|
|
@@ -556,6 +556,7 @@ module Ziperu
|
|
|
556
556
|
::Ziperu::OutputStream.open(@name, append: true, **stream_options) do |zos|
|
|
557
557
|
zos.put_entries(appended, threads: @threads)
|
|
558
558
|
end
|
|
559
|
+
was_append = true
|
|
559
560
|
else
|
|
560
561
|
on_success_replace do |tmp_file|
|
|
561
562
|
::Ziperu::OutputStream.open(tmp_file, **stream_options) do |zos|
|
|
@@ -576,9 +577,41 @@ module Ziperu
|
|
|
576
577
|
@name = @write_to
|
|
577
578
|
@write_to = nil
|
|
578
579
|
end
|
|
579
|
-
initialize_cdir(@name)
|
|
580
|
+
initialize_cdir(@name) unless was_append && refresh_after_append(appended)
|
|
580
581
|
end
|
|
581
582
|
|
|
583
|
+
# After an append: the entries already there are where they were, so
|
|
584
|
+
# they are kept (pointed at the archive reopened) and only the new
|
|
585
|
+
# records are read, rather than every entry built again. False, with
|
|
586
|
+
# nothing changed, when the archive does not hold what was expected;
|
|
587
|
+
# `initialize_cdir` then reads it all.
|
|
588
|
+
def refresh_after_append(appended)
|
|
589
|
+
held = @native.entry_count
|
|
590
|
+
native = InputStream.open_archive(@name, @settings)
|
|
591
|
+
unless native.entry_count == held + appended.size && !native.duplicate_names?
|
|
592
|
+
native.close
|
|
593
|
+
return false
|
|
594
|
+
end
|
|
595
|
+
|
|
596
|
+
fresh = native.entry_metas(held, appended.size).map do |meta|
|
|
597
|
+
Entry.from_meta(native, meta, self, metadata_encoding: @metadata_encoding, settings: @settings)
|
|
598
|
+
end
|
|
599
|
+
unless fresh.map { |e| e.name.b } == appended.map { |e| e.name.b }
|
|
600
|
+
native.close
|
|
601
|
+
return false
|
|
602
|
+
end
|
|
603
|
+
|
|
604
|
+
kept = @cdir.entries.first(held)
|
|
605
|
+
kept.each { |e| e.rebind_native(native) }
|
|
606
|
+
fresh.each { |e| @cdir << e }
|
|
607
|
+
@native.close
|
|
608
|
+
@native = native
|
|
609
|
+
@dirty = false
|
|
610
|
+
@file_permissions = ::File.stat(@name).mode
|
|
611
|
+
true
|
|
612
|
+
end
|
|
613
|
+
private :refresh_after_append
|
|
614
|
+
|
|
582
615
|
# The entries to append, when the pending changes are additions and
|
|
583
616
|
# nothing else: every entry the archive holds is still listed, in
|
|
584
617
|
# place, unchanged, and the comment is as it was. Nil otherwise, and
|
|
@@ -589,9 +622,12 @@ module Ziperu
|
|
|
589
622
|
|
|
590
623
|
all = @cdir.entries
|
|
591
624
|
held = @native.entry_count
|
|
625
|
+
# The local headers there stay as they are; the crate writes the
|
|
626
|
+
# central directory again, which loses some fields, so those decide
|
|
627
|
+
# (without reading every local header, as `needs_restream?` would).
|
|
592
628
|
return nil if all.size <= held || all.first(held).each_with_index.any? do |e, i|
|
|
593
|
-
!(e.index == i && e.backed_by?(@native) && !e.dirty? && !e.
|
|
594
|
-
!e.
|
|
629
|
+
!(e.index == i && e.backed_by?(@native) && !e.dirty? && !e.changed? &&
|
|
630
|
+
!e.central_fields_lost_on_append? && !@cdir.shadowed?(e))
|
|
595
631
|
end
|
|
596
632
|
|
|
597
633
|
all.drop(held)
|
data/lib/ziperu/input_stream.rb
CHANGED
|
@@ -110,6 +110,7 @@ module Ziperu
|
|
|
110
110
|
def close
|
|
111
111
|
@reader&.close
|
|
112
112
|
@reader = nil
|
|
113
|
+
@lazy_open = false
|
|
113
114
|
@stream&.close
|
|
114
115
|
@stream = nil
|
|
115
116
|
@archive&.close if @owns_archive
|
|
@@ -130,6 +131,7 @@ module Ziperu
|
|
|
130
131
|
# entry again.
|
|
131
132
|
@reader&.close
|
|
132
133
|
@reader = nil
|
|
134
|
+
@lazy_open = false
|
|
133
135
|
flush
|
|
134
136
|
return @current_entry = nil
|
|
135
137
|
end
|
|
@@ -137,7 +139,9 @@ module Ziperu
|
|
|
137
139
|
@current_entry = Entry.from_meta(@archive, meta_at(@next_index),
|
|
138
140
|
metadata_encoding: @metadata_encoding, settings: @settings)
|
|
139
141
|
@next_index += 1
|
|
140
|
-
|
|
142
|
+
# The entry's data is opened when it is first read: a walk that only
|
|
143
|
+
# looks at the entries reads no local header.
|
|
144
|
+
open_entry(lazy: true)
|
|
141
145
|
@current_entry
|
|
142
146
|
end
|
|
143
147
|
|
|
@@ -187,7 +191,7 @@ module Ziperu
|
|
|
187
191
|
|
|
188
192
|
@lineno = 0
|
|
189
193
|
@pos = 0
|
|
190
|
-
wrap_errors {
|
|
194
|
+
wrap_errors { reader.rewind }
|
|
191
195
|
flush
|
|
192
196
|
end
|
|
193
197
|
|
|
@@ -200,7 +204,7 @@ module Ziperu
|
|
|
200
204
|
# Not in rubyzip.
|
|
201
205
|
def seek(offset, whence = ::IO::SEEK_SET)
|
|
202
206
|
check_open!
|
|
203
|
-
raise Error, 'no entry to seek in' if
|
|
207
|
+
raise Error, 'no entry to seek in' if reader.nil?
|
|
204
208
|
raise Error, 'an entry read with streaming: true cannot be seeked in' if @stream
|
|
205
209
|
|
|
206
210
|
target =
|
|
@@ -212,7 +216,7 @@ module Ziperu
|
|
|
212
216
|
end
|
|
213
217
|
raise Errno::EINVAL, "seek to #{target}" if target.negative?
|
|
214
218
|
|
|
215
|
-
wrap_errors {
|
|
219
|
+
wrap_errors { reader.seek(target) }
|
|
216
220
|
flush
|
|
217
221
|
@pos = [target, @current_entry.size].min
|
|
218
222
|
0
|
|
@@ -392,7 +396,7 @@ module Ziperu
|
|
|
392
396
|
|
|
393
397
|
protected
|
|
394
398
|
|
|
395
|
-
def open_entry # :nodoc:
|
|
399
|
+
def open_entry(lazy: false) # :nodoc:
|
|
396
400
|
entry = @current_entry
|
|
397
401
|
unless READABLE_COMPRESSION_METHODS.include?(entry.compression_method)
|
|
398
402
|
raise CompressionMethodError, entry.compression_method
|
|
@@ -402,6 +406,19 @@ module Ziperu
|
|
|
402
406
|
raise Error, PASSWORD_REQUIRED_MESSAGE
|
|
403
407
|
end
|
|
404
408
|
|
|
409
|
+
@reader&.close
|
|
410
|
+
@reader = nil
|
|
411
|
+
@lazy_open = true
|
|
412
|
+
reader unless lazy
|
|
413
|
+
flush
|
|
414
|
+
end
|
|
415
|
+
|
|
416
|
+
# The native reader of the current entry, opened on first use.
|
|
417
|
+
def reader # :nodoc:
|
|
418
|
+
return @reader unless @lazy_open
|
|
419
|
+
|
|
420
|
+
@lazy_open = false
|
|
421
|
+
entry = @current_entry
|
|
405
422
|
password = @decrypter&.password&.b
|
|
406
423
|
@reader = wrap_errors do
|
|
407
424
|
if entry.index
|
|
@@ -411,7 +428,6 @@ module Ziperu
|
|
|
411
428
|
entry.crc, password, @enforce_size)
|
|
412
429
|
end
|
|
413
430
|
end
|
|
414
|
-
flush
|
|
415
431
|
end
|
|
416
432
|
|
|
417
433
|
def produce_input(maxlen = CHUNK_SIZE) # :nodoc:
|
|
@@ -420,8 +436,8 @@ module Ziperu
|
|
|
420
436
|
raise TypeError, "no implicit conversion of #{maxlen.class} into Integer"
|
|
421
437
|
end
|
|
422
438
|
raise ArgumentError, "negative length #{maxlen} given" if maxlen&.negative?
|
|
423
|
-
return nil if @reader.nil?
|
|
424
439
|
return produce_stream_input(maxlen) if @stream
|
|
440
|
+
return nil if reader.nil?
|
|
425
441
|
|
|
426
442
|
wrap_errors { @reader.read(maxlen) }
|
|
427
443
|
end
|
|
@@ -430,7 +446,7 @@ module Ziperu
|
|
|
430
446
|
# A closed stream is not "at EOF": reading it raises, as with an IO.
|
|
431
447
|
return false if closed?
|
|
432
448
|
|
|
433
|
-
|
|
449
|
+
reader.nil? || @reader.eof?
|
|
434
450
|
end
|
|
435
451
|
|
|
436
452
|
private
|
data/lib/ziperu/output_stream.rb
CHANGED
|
@@ -396,6 +396,16 @@ module Ziperu
|
|
|
396
396
|
self
|
|
397
397
|
end
|
|
398
398
|
|
|
399
|
+
# Writes the file at `path` to the entry in progress, as `<<` of its
|
|
400
|
+
# contents would, without reading them into Ruby.
|
|
401
|
+
def write_file(path) # :nodoc:
|
|
402
|
+
raise IOError, 'closed stream' if @closed || @current_entry.nil?
|
|
403
|
+
|
|
404
|
+
start_deferred
|
|
405
|
+
@writer.write_file(path)
|
|
406
|
+
self
|
|
407
|
+
end
|
|
408
|
+
|
|
399
409
|
private
|
|
400
410
|
|
|
401
411
|
def finish_archive
|
|
@@ -4,12 +4,14 @@
|
|
|
4
4
|
|
|
5
5
|
module Ziperu
|
|
6
6
|
# An entry whose contents are being written through `get_output_stream`:
|
|
7
|
-
# the data
|
|
7
|
+
# the data is held until the archive is committed. It starts in memory
|
|
8
|
+
# and moves to a tempfile once the archive's entries hold more than
|
|
9
|
+
# `SpoolIO::BUDGET` bytes in memory between them; rubyzip opens a
|
|
10
|
+
# tempfile for every entry, which costs more than writing a small one.
|
|
8
11
|
class StreamableStream < DelegateClass(Entry) # :nodoc:all
|
|
9
|
-
def initialize(entry)
|
|
10
|
-
super
|
|
11
|
-
@temp_file =
|
|
12
|
-
@temp_file.binmode
|
|
12
|
+
def initialize(entry, budget = SpoolIO::Budget.new)
|
|
13
|
+
super(entry)
|
|
14
|
+
@temp_file = SpoolIO.new(::File.basename(name), budget)
|
|
13
15
|
end
|
|
14
16
|
|
|
15
17
|
def get_output_stream
|
|
@@ -33,16 +35,15 @@ module Ziperu
|
|
|
33
35
|
raise StandardError, "cannot open entry for reading while its open for writing - #{name}"
|
|
34
36
|
end
|
|
35
37
|
|
|
36
|
-
@temp_file.
|
|
37
|
-
@temp_file.binmode
|
|
38
|
+
io = @temp_file.reader
|
|
38
39
|
if block_given?
|
|
39
40
|
begin
|
|
40
|
-
yield(
|
|
41
|
+
yield(io)
|
|
41
42
|
ensure
|
|
42
|
-
|
|
43
|
+
io.close
|
|
43
44
|
end
|
|
44
45
|
else
|
|
45
|
-
|
|
46
|
+
io
|
|
46
47
|
end
|
|
47
48
|
end
|
|
48
49
|
|
|
@@ -56,4 +57,103 @@ module Ziperu
|
|
|
56
57
|
@temp_file.unlink
|
|
57
58
|
end
|
|
58
59
|
end
|
|
60
|
+
|
|
61
|
+
# What `get_output_stream` hands out: a StringIO that becomes a Tempfile
|
|
62
|
+
# (with what was written so far) when its archive's in-memory budget
|
|
63
|
+
# runs out. Everything else is the IO's own.
|
|
64
|
+
class SpoolIO < SimpleDelegator # :nodoc:all
|
|
65
|
+
# Bytes the pending `get_output_stream` entries of one archive may hold
|
|
66
|
+
# in memory between them.
|
|
67
|
+
BUDGET = 8 * 1024 * 1024
|
|
68
|
+
|
|
69
|
+
# The bytes left to the entries of one archive.
|
|
70
|
+
class Budget
|
|
71
|
+
attr_accessor :left
|
|
72
|
+
|
|
73
|
+
def initialize(left = BUDGET)
|
|
74
|
+
@left = left
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def initialize(basename, budget)
|
|
79
|
+
@basename = basename
|
|
80
|
+
@budget = budget
|
|
81
|
+
@held = 0
|
|
82
|
+
@file = nil
|
|
83
|
+
super(::StringIO.new(+''.b))
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
# Every call goes to the IO; afterwards, the memory it holds is
|
|
87
|
+
# counted, and past the budget moved to a tempfile.
|
|
88
|
+
def method_missing(name, *args, &block)
|
|
89
|
+
result = super
|
|
90
|
+
account unless @file
|
|
91
|
+
result.equal?(__getobj__) ? self : result
|
|
92
|
+
end
|
|
93
|
+
ruby2_keywords(:method_missing)
|
|
94
|
+
|
|
95
|
+
def respond_to_missing?(name, include_private = false)
|
|
96
|
+
__getobj__.respond_to?(name, include_private)
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# An IO over what was written, from its start.
|
|
100
|
+
def reader
|
|
101
|
+
if @file
|
|
102
|
+
@file.open
|
|
103
|
+
@file.binmode
|
|
104
|
+
@file
|
|
105
|
+
else
|
|
106
|
+
::StringIO.new(__getobj__.string, 'rb')
|
|
107
|
+
end
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
def unlink
|
|
111
|
+
if @file
|
|
112
|
+
@file.close
|
|
113
|
+
@file.unlink
|
|
114
|
+
else
|
|
115
|
+
__getobj__.string.clear unless __getobj__.string.frozen?
|
|
116
|
+
@budget.left += @held
|
|
117
|
+
@held = 0
|
|
118
|
+
end
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
def path
|
|
122
|
+
spill
|
|
123
|
+
@file.path
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
private
|
|
127
|
+
|
|
128
|
+
def account
|
|
129
|
+
size = __getobj__.string.bytesize
|
|
130
|
+
grown = size - @held
|
|
131
|
+
return if grown.zero?
|
|
132
|
+
|
|
133
|
+
if grown > @budget.left
|
|
134
|
+
spill
|
|
135
|
+
else
|
|
136
|
+
@budget.left -= grown
|
|
137
|
+
@held = size
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
|
|
141
|
+
def spill
|
|
142
|
+
return if @file
|
|
143
|
+
|
|
144
|
+
io = __getobj__
|
|
145
|
+
file = ::Tempfile.new(@basename)
|
|
146
|
+
file.binmode
|
|
147
|
+
file.write(io.string)
|
|
148
|
+
if io.closed?
|
|
149
|
+
file.close
|
|
150
|
+
else
|
|
151
|
+
file.pos = io.pos
|
|
152
|
+
end
|
|
153
|
+
@budget.left += @held
|
|
154
|
+
@held = 0
|
|
155
|
+
@file = file
|
|
156
|
+
__setobj__(file)
|
|
157
|
+
end
|
|
158
|
+
end
|
|
59
159
|
end
|
data/lib/ziperu/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ziperu
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.2.
|
|
4
|
+
version: 0.2.3
|
|
5
5
|
platform: aarch64-linux-gnu
|
|
6
6
|
authors:
|
|
7
7
|
- Maor Caplan
|
|
8
8
|
bindir: bin
|
|
9
9
|
cert_chain: []
|
|
10
|
-
date: 2026-11-
|
|
10
|
+
date: 2026-11-12 00:00:00.000000000 Z
|
|
11
11
|
dependencies: []
|
|
12
12
|
description: Zip reading and writing for Ruby with rubyzip's API under the Ziperu
|
|
13
13
|
namespace, implemented on the Rust `zip` crate via Magnus. `require "zip"` aliases
|