zip_kit 6.4.1 → 6.4.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0c9a2d7c76b813cfcbfb622fce1164f1566e42ef1487f8f44095d2464b76f282
-  data.tar.gz: 45ef948824bebe50569949c030cbf2d2c720568e5c87e011ac124d4dccaccc55
+  metadata.gz: c94f6db106a2c4fb28f882cdbaf3411a2fb4142a3691f337fd3adbefc8cf566b
+  data.tar.gz: e5eef7ee71cb2a2b7c5a091399308f97ff69e9f64d350540de9fdf908e77da08
 SHA512:
-  metadata.gz: 45d55f2c2038c0d19c3da829a113b44c5cea973a9f54b7118b6c21c0a75f6edf14d6d31e55ce27c6713b4e7d04cab4e752fb1fd62264b72440e88147c0d9fd7c
-  data.tar.gz: 9740f7b322e6a6b32a3e1ff2c38e0b07fb20b92cdac2b30863c65bead2e04d05646d28c881edb993aee0b192009a38f30b04dd06ebb3174fc85bc31daf80f847
+  metadata.gz: 11236fe274f019b9f4c0a0c90814578ed61f074d911dc2c110289455507e6728c0b3e9bd8e9f820f06bfd3075cb1136d5bd6ea6812b0b3fa22ad9a674904555e
+  data.tar.gz: cdf1dfe461549d4df702a7b058e0fce429102edf1a1721df2c7ce769d4dfa67fc04e7d6cd8576427ded83e9529a818c1fa4104b5bab81a883882fb9565517829

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 ## Unreleased
 
+## 6.4.2
+
+* Write every ZIP record with a single `Array#pack` and a single `<<` to the destination, instead of one per field. Archives with lots of entries get written about 1.5x faster.
+* Unpack the fixed fields of local and central directory headers in one go in `FileReader`, and read 4KB upfront for a local file header instead of the maximum possible 128KB. `FileReader.read_zip_structure` with local headers gets about 5x faster on archives with many entries, and parsing the central directory alone about 2x.
+* Fix `ZipWriter` converting the given `mtime` to UTC in place. The local file header got the DOS time in local time and the central directory header got it in UTC, and the caller was left with a Time object in UTC.
+* Remove the `C_CHAR` and `C_INT4` constants from `ZipWriter`, which were never meant to be public (the other packspec constants were private already).
+* Add `"cache-control" => "no-transform"` to `OutputEnumerator.streaming_http_headers`, and add `no-transform` to the `Cache-Control` of the response in `zip_kit_stream` (keeping whatever the controller has set there). `Rack::Deflater` only leaves a response alone if it has `no-transform` or a content encoding other than `identity`, so the `content-encoding: identity` we were sending did not stop it - it gzipped the ZIP all the same, and with `sync: false` buffered all of it to do so. If you replace our `cache-control` with your own, keep `no-transform` in it.
+
 ## 6.4.1
 
 * Use lowercase names for the HTTP headers ZipKit produces, since Rack 3 requires them (`Rack::Lint` rejects uppercase header names). `OutputEnumerator.streaming_http_headers` now returns `"content-type"`, `"last-modified"`, `"content-encoding"` and `"x-accel-buffering"`. This is a breaking change if you look those headers up or override them in the returned Hash using the capitalised names: `headers["Content-Type"]` is now `nil`, and `headers["Content-Type"] = "application/epub+zip"` adds a second content type instead of replacing ours. Use the lowercase names instead. `zip_kit_stream` in Rails is not affected: it now merges our headers into the response regardless of case, so headers Rails sets itself (like the content type) still take precedence and do not get sent twice.

data/CODE_OF_CONDUCT.md

@@ -1,4 +1,8 @@
-# Contributor Covenant Code of Conduct
+# Code of Conduct
+
+## Relationship to CONTRIBUTING
+
+This Code of Conduct works together with [CONTRIBUTING.md](CONTRIBUTING.md). Where the two differ, CONTRIBUTING.md takes precedence.
 
 ## Our Pledge
 
@@ -24,23 +28,23 @@ Examples of unacceptable behavior by participants include:
 
 ## Our Responsibilities
 
-Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+The maintainer is responsible for clarifying the standards of acceptable behavior and is expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
 
-Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviours that they deem inappropriate, threatening, offensive, or harmful.
+The maintainer has the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviours that the maintainer deems inappropriate, threatening, offensive, or harmful.
 
 ## Scope
 
-This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+This Code of Conduct applies within project spaces: the repository, its issues, pull requests and discussions, and any other channels the project runs.
 
-## Enforcement
+Conduct outside project spaces is not considered, either for or against anyone, as described in [What happens elsewhere stays elsewhere](CONTRIBUTING.md#what-happens-elsewhere-stays-elsewhere).
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at me@julik.nl. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+## Enforcement
 
-Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the maintainer at me@julik.nl. The maintainer will review and investigate all complaints, and will respond in a way the maintainer deems appropriate to the circumstances. The maintainer is obligated to maintain confidentiality with regard to the reporter of an incident. Enforcement decisions are final.
 
 ## Attribution
 
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, with modifications, available at [http://contributor-covenant.org/version/1/4][version]
 
 [homepage]: http://contributor-covenant.org
 [version]: http://contributor-covenant.org/version/1/4/

data/CONTRIBUTING.md

@@ -1,153 +1,89 @@
 # Contributing to zip_kit
 
-Please take a moment to review this document in order to make the contribution
-process easy and effective for everyone involved.
+Contributions are welcome! Please take a moment to review this document so that
+contributing stays easy and effective for everyone involved.
 
-Following these guidelines helps to communicate that you respect the time of
-the developers managing and developing this open source project. In return,
-they should reciprocate that respect in addressing your issue or assessing
-patches and features.
+I'd especially love to get:
 
-## What do I need to know to help?
+* Reports of archives that a specific unarchiver refuses to open or extracts incorrectly
+* Performance work
+* Documentation fixes
 
-If you are already familiar with the [Ruby Programming Language](https://www.ruby-lang.org/) you can start contributing code right away, otherwise look for issues labeled with *documentation* or *good first issue* to get started.
+## What do I need to know to help?
 
-If you are interested in contributing code and would like to learn more about the technologies that we use, check out the (non-exhaustive) list below. You can also get in touch with us via an issue or email to get additional information.
+If you would like to learn more about the technologies zip_kit builds on, check out the (non-exhaustive) list below. You can also get in touch with me via an issue or email.
 
  - [ruby](https://ruby-doc.org)
- - [rubyzip](https://github.com/rubyzip/rubyzip) as we like to test "our implementation against theirs"
- - [rspec](http://rspec.info/) for testing
- - [zip files](https://en.wikipedia.org/wiki/Zip_(file_format))
-
-# How do I make a contribution?
-
-## Using the issue tracker
-
-The issue tracker is the preferred channel for [bug reports](#bug-reports),
-[feature requests](#feature-requests) and [submitting pull
-requests](#pull-requests), but please respect the following restrictions:
-
-* Please **do not** derail or troll issues. Keep the discussion on topic and respect the opinions of others. Adhere to the principles set out in the [Code of Conduct](https://github.com/WeTransfer/zip_kit/blob/main/CODE_OF_CONDUCT.md).
-
-## Bug reports
-
-A bug is a _demonstrable problem_ that is caused by code in the repository.
-
-Good bug reports are extremely helpful-thank you!
+ - [ZIP files](https://en.wikipedia.org/wiki/Zip_(file_format)) and the [APPNOTE.TXT](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) specification
+ - [rubyzip](https://github.com/rubyzip/rubyzip), as we like to test "our implementation against theirs"
+ - [rspec](https://rspec.info/) for testing
 
-Guidelines for bug reports:
+## Issues and pull requests
 
-1. **Use the GitHub issue search** — check if the issue has already been
-   reported.
+* Opening an issue first is often the fastest way to get something in. I can shape it into something that fits the library. Please do this before any significant PR (new features, refactoring), so you don't spend time on something that won't be merged.
+* Search the issues first, and check whether the problem still happens on the latest `main`.
+* The most useful bug report is a small script that produces the broken archive (or the archive itself), plus the name, version and OS of the tool that fails to open it.
+* I may close a PR and implement the idea differently. That's not a rejection of the idea, and you'll be credited (`Co-authored-by`).
+* Feel free to propose opinionated changes, but be prepared that your opinion won't be shared. It is a give-and-take.
+* Keep PRs focused, without unrelated commits.
 
-2. **Check if the issue has been fixed** — try to reproduce it using the
-   latest `main` branch in the repository.
+## Code changes
 
-3. **Isolate the problem** — create a [reduced test
-   case](http://css-tricks.com/reduced-test-cases/) and a live example.
-
-A good bug report shouldn't leave others needing to chase you up for more
-information. Please try to be as detailed as possible in your report. What is
-your environment? What steps will reproduce the issue? What tool(s) or OS will
-experience the problem? What would you expect to be the outcome? All these
-details will help people to fix any potential bugs.
-
-Example:
-
-> Short and descriptive example bug report title
->
-> A summary of the issue and the OS environment in which it occurs. If
-> suitable, include the steps required to reproduce the bug.
->
-> 1. This is the first step
-> 2. This is the second step
-> 3. Further steps, etc.
->
-> `<url>` - a link to the reduced test case, if possible. Feel free to use a [Gist](https://gist.github.com).
->
-> Any other information you want to share that is relevant to the issue being
-> reported. This might include the lines of code that you have identified as
-> causing the bug, and potential solutions (and your opinions on their
-> merits).
-
-
-## Feature requests
-
-Feature requests are welcome. But take a moment to find out whether your idea
-fits with the scope and aims of the project. It's up to *you* to make a strong
-case to convince the project's developers of the merits of this feature. Please
-provide as much detail and context as possible.
+Please adhere to the coding conventions used throughout the project (indentation,
+accurate and terse comments, etc.) and any other requirements (such as test coverage).
 
+Run the `rake` default task before submitting. It runs the test suite (`bundle exec rspec`),
+checks formatting with [standard](https://github.com/standardrb/standard) (`bundle exec rake format`
+fixes most of it), and regenerates the type signatures in `rbi/`. Commit the regenerated signatures.
 
-## Pull requests
+zip_kit supports Ruby 2.6 and up, and CI runs the tests and the linter on 2.6. Don't use syntax or
+stdlib features newer than that. CI also runs the suite with frozen string literals enforced, so don't
+mutate string literals.
 
-Good pull requests-patches, improvements, new features-are a fantastic
-help. They should remain focused in scope and avoid containing unrelated
-commits.
+## Basic expectations
 
-**Please ask first** before embarking on any significant pull request (e.g.
-implementing features, refactoring code, porting to a different language),
-otherwise you risk spending a lot of time working on something that the
-project's developers might not want to merge into the project.
+This project is something created with love, and I - as an author - expect you, as a contributor, to return the favor. The ground rules:
 
-Please adhere to the coding conventions used throughout the project (indentation,
-accurate comments, etc.) and any other requirements (such as test coverage).
+* Do your best to be kind.
+* Be polite and direct. No abrasiveness, no passive-aggression.
+* [Low-context](https://www.ebsco.com/research-starters/communication-and-mass-media/high-context-and-low-context-cultures) communication is preferred and permitted
+* No ad-hominems, also in code comments
+* No sexualized or aggressive language
 
-The project uses Rubocop which can be run using `bundle exec rubocop`. The test
-suite can be run with `bundle exec rspec`. You are also encouraged to use the
-script in the `testing` directory to create test files that you can then verify
-with various zip/unzip utilities. Further instructions are [here](https://github.com/WeTransfer/zip_kit/blob/main/testing/README_TESTING.md).  
+These rules sit alongside the [Code of Conduct](CODE_OF_CONDUCT.md), which applies to everyone taking part in the project.
 
-Follow this process if you'd like your work considered for inclusion in the
-project:
+As a maintainer, I reserve the right to deny any contribution, discussion post, comment or pull request I find inappropriate or that is not aligned with the Code of Conduct.
 
-1. [Fork](http://help.github.com/fork-a-repo/) the project, clone your fork,
-   and configure the remotes:
+Similarly, I reserve the right to not accept any contribution that doesn't fit the goals or spirit of the project (or my own), without explaining why it has been refused.
 
-   ```bash
-   # Clone your fork of the repo into the current directory
-   git clone git@github.com:julik/zip_kit.git
-   # Navigate to the newly cloned directory
-   cd zip_kit
-   # Assign the original repo to a remote called "upstream"
-   git remote add upstream git@github.com:julik/zip_kit.git
-   ```
+## Decisions are final
 
-2. If you cloned a while ago, get the latest changes from upstream:
+You're welcome to make your case once, clearly. After that, please don't reopen closed issues, start meta-threads about the decision, or argue about the wording of this document. This is a list of expectations, not a contract to be lawyered. If something isn't covered, the maintainer's judgment applies.
 
-   ```bash
-   git checkout <dev-branch>
-   git pull upstream <dev-branch>
-   ```
+This applies to Code of Conduct enforcement too. Reporting a violation is always welcome, privately, at the address given in the [Code of Conduct](CODE_OF_CONDUCT.md#enforcement), but the outcome is final.
 
-3. Create a new topic branch (off the main project development branch) to
-   contain your feature, change, or fix:
+## What happens elsewhere stays elsewhere
 
-   ```bash
-   git checkout -b <topic-branch-name>
-   ```
+Contributions are judged on what they are, not on who sent them. What anyone says or does outside this project (on social media, at their job, in other repos, in politics) is not a topic here and will not be considered, either for or against them. Please don't bring it in: no call-outs, no "are you aware this person…" threads, no requests to ban someone over things that didn't happen here.
 
-4. Commit your changes in logical chunks and/or squash them for readability and
-   conciseness. Check out [this post](https://chris.beams.io/posts/git-commit/) or
-   [this other post](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html) for some tips re: writing good commit messages.
+The [Code of Conduct](CODE_OF_CONDUCT.md#scope) applies within project spaces only, and defers to this document where the two differ.
 
-5. Locally merge (or rebase) the upstream development branch into your topic branch:
+## AI-generated issues and code
 
-   ```bash
-   git pull [--rebase] upstream <dev-branch>
-   ```
+This project uses LLMs to produce code and artifacts, and it's fine if you do the same when contributing. LLM use in this project is settled: threads about whether it should be used (utility, harm, societal implications etc.) will be closed.
 
-6. Push your topic branch up to your fork:
+If you require LLM-untainted code for your needs, feel free to use zip_kit prior to and including commit 2f123ae41cffe7255f1ff0824af44420d5dbfaa7 - but I am not going to maintain an "LLM-free" branch of zip_kit just for your needs.
+ 
+Please disclose LLM use in every issue and PR: say which model or tool you used and what for (e.g. "Claude wrote the fix, I wrote the test"). This is for transparency, not for judgment. Disclosed LLM use is treated the same as no LLM use.
 
-   ```bash
-   git push origin <topic-branch-name>
-   ```
+However, the following is on you:
 
-   Make sure to run `bundle exec appraisal rspec` to make sure your changes are compatible with all the Rails versions we support.
+* Make your LLM output _terse._ Nobody likes reading slop, even my agents. Make it get to the point.
+* Make sure what you submit actually ran. Don't file issues about behavior or APIs your LLM imagined.
+* zip_kit has no runtime dependencies. Do not add any (or have your LLM do it) unless absolutely necessary
+* If you file a PR, make at least one pass with "Make this at least 50% less code"
+* Code generated by your LLM will be considered as written by you, with the same requirements
 
-7. [Open a Pull Request](https://help.github.com/articles/using-pull-requests/)
-    with a clear title and description.
+## License
 
-**IMPORTANT**: By submitting a patch, you agree to allow the project owner to
-license your work under the same license as that used by the project, which you
-can see by clicking [here](LICENSE.txt). 
+By submitting a patch, you agree to allow the project owner to license your work under the same license as that used by the project, which you can see [here](LICENSE.txt).

data/lib/zip_kit/file_reader.rb

@@ -291,9 +291,8 @@ class ZipKit::FileReader
 
   # Parse the local header entry and get the offset in the IO at which the
   # actual compressed data of the file starts within the ZIP.
-  # The method will eager-read the entire local header for the file
-  # (the maximum size the local header may use), starting at the given offset,
-  # and will then compute its size. That size plus the local header offset
+  # The method will read the local header for the file, starting at the
+  # given offset, and will then compute its size. That size plus the local header offset
   # given will be the compressed data offset of the entry (read starting at
   # this offset to get the data).
   #
@@ -303,28 +302,43 @@ class ZipKit::FileReader
   def read_local_file_header(io:)
     local_file_header_offset = io.tell
 
-    # Reading in bulk is cheaper - grab the maximum length of the local header,
-    # including any headroom for extra fields etc.
-    local_file_header_str_plus_headroom = io.read(MAX_LOCAL_HEADER_SIZE)
-    raise ReadError if local_file_header_str_plus_headroom.nil? # reached EOF
+    # Reading in bulk is cheaper - grab enough for most local headers, including
+    # some headroom for the filename and extra fields. Reading the maximum possible
+    # size (about 128KB) for every entry costs more than the parsing itself.
+    header = io.read(LOCAL_HEADER_READ_SIZE)
+    if header.nil? || header.bytesize < SIZE_OF_LOCAL_HEADER_FIXED_FIELDS
+      raise ReadError, "Expected to read at least #{SIZE_OF_LOCAL_HEADER_FIXED_FIELDS} bytes of the local file header, but the IO was at the end"
+    end
 
-    io_starting_at_local_header = StringIO.new(local_file_header_str_plus_headroom)
+    signature,
+      version_needed_to_extract,
+      gp_flags,
+      storage_mode,
+      dos_time,
+      dos_date,
+      crc32,
+      compressed_size,
+      uncompressed_size,
+      filename_size,
+      extra_size = header.unpack(LOCAL_HEADER_FIXED_FIELDS_UNPACKSPEC)
+    assert_signature_value(signature, 0x04034b50)
+
+    # ...and if the filename and extras did not fit, read the rest
+    size_of_header = SIZE_OF_LOCAL_HEADER_FIXED_FIELDS + filename_size + extra_size
+    header << read_n(io, size_of_header - header.bytesize) if header.bytesize < size_of_header
 
-    assert_signature(io_starting_at_local_header, 0x04034b50)
     e = ZipEntry.new
-    e.version_needed_to_extract = read_2b(io_starting_at_local_header) # Version needed to extract
-    e.gp_flags = read_2b(io_starting_at_local_header) # gp flags
-    e.storage_mode = read_2b(io_starting_at_local_header) # storage mode
-    e.dos_time = read_2b(io_starting_at_local_header) # dos time
-    e.dos_date = read_2b(io_starting_at_local_header) # dos date
-    e.crc32 = read_4b(io_starting_at_local_header) # CRC32
-    e.compressed_size = read_4b(io_starting_at_local_header) # Comp size
-    e.uncompressed_size = read_4b(io_starting_at_local_header) # Uncomp size
-
-    filename_size = read_2b(io_starting_at_local_header)
-    extra_size = read_2b(io_starting_at_local_header)
-    e.filename = read_n(io_starting_at_local_header, filename_size)
-    extra_fields_str = read_n(io_starting_at_local_header, extra_size)
+    e.version_needed_to_extract = version_needed_to_extract
+    e.gp_flags = gp_flags
+    e.storage_mode = storage_mode
+    e.dos_time = dos_time
+    e.dos_date = dos_date
+    e.crc32 = crc32
+    e.compressed_size = compressed_size
+    e.uncompressed_size = uncompressed_size
+
+    e.filename = header.byteslice(SIZE_OF_LOCAL_HEADER_FIXED_FIELDS, filename_size)
+    extra_fields_str = header.byteslice(SIZE_OF_LOCAL_HEADER_FIXED_FIELDS + filename_size, extra_size)
 
     # Parse out the extra fields
     extra_table = parse_out_extra_fields(extra_fields_str)
@@ -351,16 +365,14 @@ class ZipKit::FileReader
       e.compressed_size = read_8b(zip64_extra) if e.compressed_size == 0xFFFFFFFF
     end
 
-    offset = local_file_header_offset + io_starting_at_local_header.tell
-    e.compressed_data_offset = offset
+    e.compressed_data_offset = local_file_header_offset + size_of_header
 
     e
   end
 
   # Get the offset in the IO at which the actual compressed data of the file
-  # starts within the ZIP. The method will eager-read the entire local header
-  # for the file (the maximum size the local header may use), starting at the
-  # given offset, and will then compute its size. That size plus the local
+  # starts within the ZIP. The method will read the local header for the file,
+  # starting at the given offset, and will then compute its size. That size plus the local
   # header offset given will be the compressed data offset of the entry
   # (read starting at this offset to get the data).
   #
@@ -431,7 +443,10 @@ class ZipKit::FileReader
   end
 
   def assert_signature(io, signature_magic_number)
-    readback = read_4b(io)
+    assert_signature_value(read_4b(io), signature_magic_number)
+  end
+
+  def assert_signature_value(readback, signature_magic_number)
     if readback != signature_magic_number
       expected = "0x0" + signature_magic_number.to_s(16)
       actual = "0x0" + readback.to_s(16)
@@ -475,24 +490,39 @@ class ZipKit::FileReader
 
   def read_cdir_entry(io)
     # read_cdir_entry is too high. [45.66/15]
-    assert_signature(io, 0x02014b50)
+    signature,
+      made_by,
+      version_needed_to_extract,
+      gp_flags,
+      storage_mode,
+      dos_time,
+      dos_date,
+      crc32,
+      compressed_size,
+      uncompressed_size,
+      filename_size,
+      extra_size,
+      comment_len,
+      disk_number_start,
+      internal_attrs,
+      external_attrs,
+      local_file_header_offset = read_n(io, SIZE_OF_CDIR_ENTRY_FIXED_FIELDS).unpack(CDIR_ENTRY_FIXED_FIELDS_UNPACKSPEC)
+    assert_signature_value(signature, 0x02014b50)
+
     ZipEntry.new.tap do |e|
-      e.made_by = read_2b(io)
-      e.version_needed_to_extract = read_2b(io)
-      e.gp_flags = read_2b(io)
-      e.storage_mode = read_2b(io)
-      e.dos_time = read_2b(io)
-      e.dos_date = read_2b(io)
-      e.crc32 = read_4b(io)
-      e.compressed_size = read_4b(io)
-      e.uncompressed_size = read_4b(io)
-      filename_size = read_2b(io)
-      extra_size = read_2b(io)
-      comment_len = read_2b(io)
-      e.disk_number_start = read_2b(io)
-      e.internal_attrs = read_2b(io)
-      e.external_attrs = read_4b(io)
-      e.local_file_header_offset = read_4b(io)
+      e.made_by = made_by
+      e.version_needed_to_extract = version_needed_to_extract
+      e.gp_flags = gp_flags
+      e.storage_mode = storage_mode
+      e.dos_time = dos_time
+      e.dos_date = dos_date
+      e.crc32 = crc32
+      e.compressed_size = compressed_size
+      e.uncompressed_size = uncompressed_size
+      e.disk_number_start = disk_number_start
+      e.internal_attrs = internal_attrs
+      e.external_attrs = external_attrs
+      e.local_file_header_offset = local_file_header_offset
       e.filename = read_n(io, filename_size)
 
       # Extra fields
@@ -660,6 +690,16 @@ class ZipKit::FileReader
   C_UINT2 = "v"
   C_UINT8 = "Q<"
 
+  # The fixed-size fields get unpacked in one go, instead of one `read` and `unpack` per field.
+  # Signature, version needed, gp flags, storage mode, DOS time and date, CRC32, both sizes and
+  # the filename and extra field lengths
+  LOCAL_HEADER_FIXED_FIELDS_UNPACKSPEC = "VvvvvvVVVvv"
+  SIZE_OF_LOCAL_HEADER_FIXED_FIELDS = 30
+  # Same as the local header, but with "made by" after the signature and the comment length,
+  # disk number, internal and external attributes and the local header offset at the end
+  CDIR_ENTRY_FIXED_FIELDS_UNPACKSPEC = "VvvvvvvVVVvvvvvVV"
+  SIZE_OF_CDIR_ENTRY_FIXED_FIELDS = 46
+
   # To prevent too many tiny reads, read the maximum possible size of end of
   # central directory record upfront (all the fixed fields + at most 0xFFFF
   # bytes of the archive comment)
@@ -673,22 +713,9 @@ class ZipKit::FileReader
     2 + # The comment size
     0xFFFF # Maximum comment size
 
-  # To prevent too many tiny reads, read the maximum possible size of the local file header upfront.
-  # The maximum size is all the usual items, plus the maximum size
-  # of the filename (0xFFFF bytes) and the maximum size of the extras (0xFFFF bytes)
-  MAX_LOCAL_HEADER_SIZE = 4 + # signature
-    2 + # Version needed to extract
-    2 + # gp flags
-    2 + # storage mode
-    2 + # dos time
-    2 + # dos date
-    4 + # CRC32
-    4 + # Comp size
-    4 + # Uncomp size
-    2 + # Filename size
-    2 + # Extra fields size
-    0xFFFF + # Maximum filename size
-    0xFFFF # Maximum extra fields size
+  # How much to read upfront for a local file header. Filenames and extra fields rarely
+  # take more than a few hundred bytes, the rest gets read separately if needed
+  LOCAL_HEADER_READ_SIZE = 4 * 1024
 
   SIZE_OF_USABLE_EOCD_RECORD = 4 + # Signature
     2 + # Number of this disk
@@ -716,7 +743,9 @@ class ZipKit::FileReader
   end
 
   private_constant :C_UINT4, :C_UINT2, :C_UINT8, :MAX_END_OF_CENTRAL_DIRECTORY_RECORD_SIZE,
-    :MAX_LOCAL_HEADER_SIZE, :SIZE_OF_USABLE_EOCD_RECORD
+    :LOCAL_HEADER_FIXED_FIELDS_UNPACKSPEC, :SIZE_OF_LOCAL_HEADER_FIXED_FIELDS,
+    :CDIR_ENTRY_FIXED_FIELDS_UNPACKSPEC, :SIZE_OF_CDIR_ENTRY_FIXED_FIELDS,
+    :LOCAL_HEADER_READ_SIZE, :SIZE_OF_USABLE_EOCD_RECORD
 
   # Is provided as a stub to be overridden in a subclass if you need it. Will report
   # during various stages of reading. The log message is contained in the return value

data/lib/zip_kit/output_enumerator.rb

@@ -116,9 +116,12 @@ class ZipKit::OutputEnumerator
   #
   # * `last-modified` will be set to "now" so that the response is considered "fresh" by `Rack::ETag`. This is done so that `Rack::ETag` won't try to
   #      calculate a lax ETag value and thus won't start buffering your response out of nowhere
-  # * `content-encoding` will be set to `identity`. This is so that proxies or the Rack middleware that applies compression to the response (like gzip)
-  #      is not going to try to compress your response. It also tells the receiving browsers (or downstream proxies) that they should not attempt to
-  #      open or uncompress the response before saving it or passing it onwards.
+  # * `cache-control` will be set to `no-transform`. This is so that `Rack::Deflater` and proxies which compress responses (like gzip)
+  #      are not going to try to compress your response again (and, with `Rack::Deflater` set to `sync: false`, buffer all of it to do so).
+  #      It says nothing about caching, so add your own directives to it if you need them, and keep `no-transform` in there
+  # * `content-encoding` will be set to `identity`. This tells the receiving browsers (or downstream proxies) that they should not attempt to
+  #      open or uncompress the response before saving it or passing it onwards. Note that it does _not_ stop `Rack::Deflater`, which compresses
+  #      responses with `content-encoding: identity` just the same
   # * `x-accel-buffering` will be set to `no` - this tells both nginx and the Google Cloud load balancer that the response should not be buffered
   #
   # These header values are known to get as close as possible to guaranteeing streaming on most environments where Ruby web applications may be hosted.
@@ -134,8 +137,9 @@ class ZipKit::OutputEnumerator
       # Set this even when not streaming for consistency. The fact that there would be
       # a weak ETag generated would mean that the middleware buffers, so we have tests for that.
       "last-modified" => Time.now.httpdate,
-      # Make sure Rack::Deflater does not touch our response body either, see
-      # https://github.com/felixbuenemann/xlsxtream/issues/14#issuecomment-529569548
+      # Make sure Rack::Deflater does not touch our response body either. It only skips
+      # responses which have `no-transform` or a content-encoding other than `identity`
+      "cache-control" => "no-transform",
       "content-encoding" => "identity",
       # Disable buffering for both nginx and Google Load Balancer, see
       # https://cloud.google.com/appengine/docs/flexible/how-requests-are-handled?tab=python#x-accel-buffering

data/lib/zip_kit/rails_streaming.rb

@@ -45,9 +45,16 @@ module ZipKit::RailsStreaming
     # names would send some headers twice
     present_header_names = response.headers.keys.map(&:downcase)
     ZipKit::OutputEnumerator.streaming_http_headers.each_pair do |name, value|
+      next if name == "cache-control"
       response.headers[name] = value unless present_header_names.include?(name)
     end
 
+    # Rails rewrites Cache-Control when the response gets committed, and before 7.1 it only
+    # looks for it under that name - a lowercase one would go out next to its own. no-transform
+    # is what keeps Rack::Deflater away, so it goes on top of whatever the controller has set
+    cache_control = response.headers["Cache-Control"].to_s.split(/,\s*/)
+    response.headers["Cache-Control"] = (cache_control | ["no-transform"]).join(", ")
+
     # The output enumerator yields chunks of bytes generated from the Streamer,
     # with some buffering. See OutputEnumerator docs for more.
     rack_zip_body = ZipKit::OutputEnumerator.new(**output_enumerator_options, &zip_streaming_blk)

data/lib/zip_kit/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ZipKit
-  VERSION = "6.4.1"
+  VERSION = "6.4.2"
 end