zip-peek 0.5.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +21 -261
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,257 +1,34 @@
|
|
|
1
1
|
# zip-peek
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
`zip-peek` lets browser applications load files from a remote `.zip` archive by only requesting the needed range of bytes without downloading and extracting the full ZIP file.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
It registers a service worker that intercepts requests like:
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
## The setup every platform team knows
|
|
10
|
-
|
|
11
|
-
At our company, we serve the same packaged content to two clients: a **mobile app** and a **web app**. For a long time, that meant **two identical buckets** on our CDN—same assets, same structure, duplicated storage and ops:
|
|
12
|
-
|
|
13
|
-
| Bucket | Consumer | Why it existed |
|
|
14
|
-
|--------|----------|----------------|
|
|
15
|
-
| **Zipped** | Mobile | Users can download the package once and open it offline later. A single archive is the right shape for “take this home.” |
|
|
16
|
-
| **Unzipped** | Web | Loose files under a normal folder path—`base/slides/slide-1/image.png`—because that’s what browsers and our frontend already expected. |
|
|
17
|
-
|
|
18
|
-
Same content. Two pipelines. Two bills. Two places for drift to creep in (“did we deploy the zip *and* the folder?”).
|
|
19
|
-
|
|
20
|
-
Nobody loved it, but it felt *correct*. Mobile gets archives. Web gets directories. That’s just how the web works, isn’t it?
|
|
21
|
-
|
|
22
|
-
### Before → After
|
|
23
|
-
|
|
24
|
-
| **Before** | **After** |
|
|
25
|
-
|------------|-----------|
|
|
26
|
-
| Mobile → `session.zip` | Mobile → `session.zip` |
|
|
27
|
-
| Web → `session/` (unzipped) | Web → `session.zip` (same bucket) |
|
|
28
|
-
| Duplicate storage & deploys | One source of truth |
|
|
29
|
-
|
|
30
|
-
```mermaid
|
|
31
|
-
flowchart LR
|
|
32
|
-
subgraph before [Before]
|
|
33
|
-
M1[Mobile] --> Z[session.zip]
|
|
34
|
-
W1[Web] --> U[session/]
|
|
35
|
-
end
|
|
36
|
-
subgraph after [After]
|
|
37
|
-
M2[Mobile] --> Z2[session.zip]
|
|
38
|
-
W2[Web] --> Z2
|
|
39
|
-
end
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
---
|
|
43
|
-
|
|
44
|
-
## What we believed about browsers and ZIP files
|
|
45
|
-
|
|
46
|
-
When my manager said we should **remove the unzipped bucket** and have the **web app use the zipped bucket**, my first reaction was honest: *this is madness.*
|
|
47
|
-
|
|
48
|
-
The mental model was simple and brutal:
|
|
49
|
-
|
|
50
|
-
1. To use a file inside a ZIP in the browser, you’d have to **download the whole archive**.
|
|
51
|
-
2. Then **unzip it in JavaScript**—CPU, memory, time on the main thread or a worker.
|
|
52
|
-
3. For a large session package (slides, images, audio), that would **destroy first load**, **hurt low-end devices**, and **waste bandwidth** on assets the user never opens.
|
|
53
|
-
|
|
54
|
-
Mobile can download `session.zip`, stash it, and read entries locally. The web, we assumed, **cannot** download a ZIP and unzip it without a performance cliff—so we **had** to keep the unzipped bucket for the web app.
|
|
55
|
-
|
|
56
|
-
That belief wasn’t lazy. It matched how most teams ship: ZIP on mobile, folder tree on CDN for web. It’s the industry default.
|
|
57
|
-
|
|
58
|
-
So when the task landed on my desk—*make the web app work from the zip bucket, performantly*—it felt like being asked to disprove physics.
|
|
59
|
-
|
|
60
|
-
**Spoiler alert: it can be done wonderfully.** But not by downloading and unpacking the whole ZIP in the page. By changing *where* the work happens.
|
|
61
|
-
|
|
62
|
-
| ❌ Naive approach | ✓ zip-peek |
|
|
63
|
-
|-------------------|------------|
|
|
64
|
-
| Download entire 200 MB ZIP | Range: ~40 KB for one PNG |
|
|
65
|
-
| Unzip in JS — CPU + memory | One asset at a time |
|
|
66
|
-
| Browser chokes | Fast, normal responses |
|
|
67
|
-
|
|
68
|
-
---
|
|
69
|
-
|
|
70
|
-
## The insight: browsers don’t need the whole ZIP—only the bytes for one file
|
|
71
|
-
|
|
72
|
-
ZIP isn’t a black box you must swallow whole. At the end of every archive is a **central directory**: an index of filenames, compressed sizes, and byte offsets. With **HTTP range requests**, you can:
|
|
73
|
-
|
|
74
|
-
1. Fetch a small slice from the **end** of the file to find that index.
|
|
75
|
-
2. Fetch only the **central directory** bytes.
|
|
76
|
-
3. For each asset request, fetch **only the byte range** for that one entry.
|
|
77
|
-
4. Inflate that chunk if it’s deflated, return a normal `Response`, and **cache** the result.
|
|
78
|
-
|
|
79
|
-
No 200 MB download for a 40 KB PNG. No app-wide unzip loop. No rewriting every screen to understand “archive semantics.”
|
|
80
|
-
|
|
81
|
-
The web doesn’t need a second bucket. It needs something in front of the zip bucket that speaks **URL paths** on the outside and **byte ranges** on the inside.
|
|
82
|
-
|
|
83
|
-
That something is what I built: **`zip-peek`**.
|
|
84
|
-
|
|
85
|
-
**What happens (in order):**
|
|
86
|
-
|
|
87
|
-
1. App requests `session.zip/slides/hero.png`
|
|
88
|
-
2. Service worker intercepts — not a normal loose file
|
|
89
|
-
3. Range: last 64 KB of ZIP → locate End of Central Directory
|
|
90
|
-
4. Range: fetch central directory only (the index)
|
|
91
|
-
5. Range: fetch bytes for `hero.png` only → inflate if needed
|
|
92
|
-
6. Cache extracted asset; return `200 image/png` to the app
|
|
93
|
-
|
|
94
|
-
```mermaid
|
|
95
|
-
sequenceDiagram
|
|
96
|
-
participant App
|
|
97
|
-
participant SW as zip-peek SW
|
|
98
|
-
participant CDN
|
|
99
|
-
App->>SW: GET session.zip/.../hero.png
|
|
100
|
-
SW->>CDN: Range tail 64KB
|
|
101
|
-
CDN-->>SW: 206
|
|
102
|
-
SW->>CDN: Range central directory
|
|
103
|
-
CDN-->>SW: 206
|
|
104
|
-
SW->>CDN: Range entry bytes only
|
|
105
|
-
CDN-->>SW: 206
|
|
106
|
-
SW-->>App: 200 image/png
|
|
7
|
+
```text
|
|
8
|
+
https://cdn.example.com/packages/session.zip/path/to/asset.png
|
|
107
9
|
```
|
|
108
10
|
|
|
109
|
-
|
|
11
|
+
The service worker fetches only the byte range needed for the requested ZIP entry, inflates the file when required, caches the extracted response, and returns it to the browser as a normal asset response.
|
|
110
12
|
|
|
111
|
-
##
|
|
13
|
+
## When Is This Package Useful?
|
|
112
14
|
|
|
113
|
-
|
|
15
|
+
Use this package when your app receives a base path and later appends folders and file names to fetch assets from that path.
|
|
114
16
|
|
|
115
|
-
|
|
116
|
-
https://cdn.example.com/packages/session.zip/slides/slide-1/hero.png
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
Our web app **keeps the same URL-building pattern** it used for the unzipped bucket—only the base path changes from a folder to a `.zip` URL:
|
|
17
|
+
For example, your app may normally treat the base path as a directory:
|
|
120
18
|
|
|
121
19
|
```ts
|
|
122
|
-
// Before (unzipped bucket)
|
|
123
20
|
const basePath = "https://cdn.example.com/packages/session";
|
|
124
|
-
const imageUrl = `${basePath}/slides/slide-1/
|
|
125
|
-
|
|
126
|
-
// After (zip bucket — same pattern)
|
|
127
|
-
const basePath = "https://cdn.example.com/packages/session.zip";
|
|
128
|
-
const imageUrl = `${basePath}/slides/slide-1/hero.png`;
|
|
21
|
+
const imageUrl = `${basePath}/slides/slide-1/image.png`;
|
|
129
22
|
```
|
|
130
23
|
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
- Parses the ZIP index from the remote archive (range requests on the tail, then the central directory).
|
|
134
|
-
- Resolves the inner path (`slides/slide-1/hero.png`).
|
|
135
|
-
- Fetches **only** that entry’s bytes.
|
|
136
|
-
- Decompresses when needed (stored or deflated entries).
|
|
137
|
-
- Serves a normal cached response to the browser.
|
|
138
|
-
|
|
139
|
-
From product code’s perspective, the zip bucket **behaves like a virtual folder**. The “impossible” migration was mostly: point `basePath` at `.zip`, initialize the worker, ensure the CDN supports `Accept-Ranges: bytes`, and configure **CORS** (S3 `ExposeHeaders` for `Content-Range` / `Accept-Ranges`, plus CloudFront `GET`/`HEAD`/`OPTIONS` with CORS preflight and **CORS-S3Origin** when applicable).
|
|
140
|
-
|
|
141
|
-
**Simple flow:**
|
|
142
|
-
|
|
143
|
-
1. **Your app** — `fetch()` or `<img src>` — no ZIP code.
|
|
144
|
-
2. **zip-peek (service worker)** — sees a path into a `.zip` file, fetches only that asset from the archive.
|
|
145
|
-
3. **CDN** — sends a small byte range from `session.zip`, not the whole file.
|
|
146
|
-
4. **Back to your app** — a normal image response, as if the PNG lived in a folder.
|
|
147
|
-
|
|
148
|
-
*The URL looks like a folder path. The service worker makes it work.*
|
|
149
|
-
|
|
150
|
-
---
|
|
151
|
-
|
|
152
|
-
## The real win: shared engines, zero code changes
|
|
153
|
-
|
|
154
|
-
The performance story mattered—but the reason this package was **so** valuable on our stack is structural.
|
|
155
|
-
|
|
156
|
-
We run **engines** that power both the web app and the mobile app. They are the same code paths: load a package, resolve assets, render slides, play audio, and so on. The host passes in a **`packageBasePath`**, and the engine builds URLs the same way every time:
|
|
24
|
+
If the same assets are sometimes delivered as a ZIP file instead:
|
|
157
25
|
|
|
158
26
|
```ts
|
|
159
|
-
const
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
The engine does not branch on web vs mobile. It **`fetch`es** what looks like an ordinary absolute URL and expects the platform to return bytes.
|
|
163
|
-
|
|
164
|
-
**Mobile and web prepare that base differently**—but the engine never sees the difference in code:
|
|
165
|
-
|
|
166
|
-
| Host | How the host prepares assets | What the engine receives as `packageBasePath` |
|
|
167
|
-
|------|------------------------------|-----------------------------------------------|
|
|
168
|
-
| **Mobile** | Download `session.zip` → uncompress on device → **localhost** server over the extracted folder | `http://127.0.0.1:3847/` |
|
|
169
|
-
| **Web (before)** | Unzipped CDN bucket | `https://cdn.example.com/packages/session` |
|
|
170
|
-
| **Web (after zip-peek)** | Zip on CDN + `initZipPeek()` | `https://cdn.example.com/packages/session.zip` |
|
|
171
|
-
|
|
172
|
-
On mobile, the engine always talked to a **real folder**—just served locally:
|
|
173
|
-
|
|
174
|
-
```text
|
|
175
|
-
http://127.0.0.1:3847/slides/slide-1/image.png
|
|
176
|
-
```
|
|
177
|
-
|
|
178
|
-
Mobile never passed the zip URL into the engine. The app shell downloaded the archive, unpacked it, stood up localhost, and handed the engine an unzipped base path.
|
|
179
|
-
|
|
180
|
-
**After `zip-peek`**, web could use the **same zip bucket** mobile downloads from. Web passes:
|
|
181
|
-
|
|
182
|
-
```text
|
|
183
|
-
https://cdn.example.com/packages/session.zip/slides/slide-1/image.png
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
Only the **service worker** knows that URL points *into* a remote ZIP.
|
|
187
|
-
|
|
188
|
-
- **Mobile:** unchanged (download zip → unzip → localhost → directory `packageBasePath`).
|
|
189
|
-
- **Web:** inject the zip URL and wire up `initZipPeek()` once.
|
|
190
|
-
- **Engine:** zero changes.
|
|
191
|
-
|
|
192
|
-
```mermaid
|
|
193
|
-
flowchart TB
|
|
194
|
-
subgraph engine["Shared engine — unchanged"]
|
|
195
|
-
U["`${packageBasePath}/asset.png`"]
|
|
196
|
-
F[fetch]
|
|
197
|
-
U --> F
|
|
198
|
-
end
|
|
199
|
-
subgraph mobile["Mobile host"]
|
|
200
|
-
CDN1[(CDN session.zip)]
|
|
201
|
-
DL[Download and unzip]
|
|
202
|
-
LH[Localhost server]
|
|
203
|
-
PM["packageBasePath = http://127.0.0.1:…/"]
|
|
204
|
-
CDN1 --> DL --> LH --> PM --> engine
|
|
205
|
-
end
|
|
206
|
-
subgraph web["Web host"]
|
|
207
|
-
CDN2[(CDN session.zip)]
|
|
208
|
-
P["packageBasePath = …/session.zip"]
|
|
209
|
-
SW[zip-peek service worker]
|
|
210
|
-
CDN2 --> P --> engine
|
|
211
|
-
F -. intercepted .-> SW
|
|
212
|
-
SW --> CDN2
|
|
213
|
-
end
|
|
27
|
+
const basePath = "https://cdn.example.com/packages/session.zip";
|
|
28
|
+
const imageUrl = `${basePath}/slides/slide-1/image.png`;
|
|
214
29
|
```
|
|
215
30
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
## From “pure madness” to one source of truth
|
|
219
|
-
|
|
220
|
-
| Before | After |
|
|
221
|
-
|--------|--------|
|
|
222
|
-
| Two buckets, same content | **One zip bucket** for mobile *and* web |
|
|
223
|
-
| Web tied to unzipped CDN layout | Web uses **path-under-zip** URLs |
|
|
224
|
-
| Fear of full-archive download + JS unzip | **Range-based peek** per asset + Cache API |
|
|
225
|
-
| “Browsers can’t do ZIP performantly” | **Browsers can**, with the right network layer |
|
|
226
|
-
|
|
227
|
-
Mobile still downloads the zip, uncompresses it, and serves it on localhost for offline—that workflow unchanged. Web now reads from the **same zip on the CDN**, on demand via zip-peek, without downloading the full archive.
|
|
228
|
-
|
|
229
|
-
---
|
|
230
|
-
|
|
231
|
-
## What had to be true on the infrastructure side
|
|
232
|
-
|
|
233
|
-
- The zip origin must support **HTTP byte ranges** (`Accept-Ranges: bytes`, `206 Partial Content`).
|
|
234
|
-
- **CORS** must allow cross-origin `Range` requests: S3 `ExposeHeaders` must include **`Content-Range`** and **`Accept-Ranges`**; CloudFront behaviors must allow **`GET`**, **`HEAD`**, and **`OPTIONS`**, use **CORS-S3Origin**, and enable **CORS with preflight** on the response headers policy.
|
|
235
|
-
- The service worker script must be served **same-origin** (with `Service-Worker-Allowed` when scope is wider than the worker path).
|
|
236
|
-
- Package URLs should end with **`.zip`**.
|
|
237
|
-
|
|
238
|
-
We also support **presigned URLs**, **allow-lists** for zip URLs, and **cache TTL** for extracted assets. See the [README](../README.md#documentation) for details.
|
|
239
|
-
|
|
240
|
-
---
|
|
241
|
-
|
|
242
|
-
## A note to the next person who hears “use the zip bucket on web”
|
|
243
|
-
|
|
244
|
-
If your gut says *browsers will choke*, you’re half right: **browsers will choke if you download and unzip the whole archive in the client.**
|
|
245
|
-
|
|
246
|
-
If your gut says *therefore we need a parallel unzipped bucket forever*, that’s the part we overturned.
|
|
247
|
-
|
|
248
|
-
The performant model is **peek, don’t gulp**: index the archive remotely, pull one entry at a time, cache what you’ve already extracted.
|
|
249
|
-
|
|
250
|
-
**It can be done wonderfully.**
|
|
251
|
-
|
|
252
|
-
---
|
|
253
|
-
|
|
254
|
-
## Documentation
|
|
31
|
+
`zip-peek` lets the app keep using the same URL-building pattern. The app does not need to know whether the base path points to a normal directory or a ZIP file, and the ZIP-specific work stays inside the service worker.
|
|
255
32
|
|
|
256
33
|
## What It Does
|
|
257
34
|
|
|
@@ -479,25 +256,19 @@ In the S3 console, open the ZIP bucket → **Permissions** → **Cross-origin re
|
|
|
479
256
|
{
|
|
480
257
|
"AllowedHeaders": ["*"],
|
|
481
258
|
"AllowedMethods": ["GET", "HEAD"],
|
|
482
|
-
"AllowedOrigins": [
|
|
483
|
-
|
|
484
|
-
"http://localhost:3000"
|
|
485
|
-
],
|
|
486
|
-
"ExposeHeaders": [
|
|
487
|
-
"Content-Range",
|
|
488
|
-
"Accept-Ranges"
|
|
489
|
-
]
|
|
259
|
+
"AllowedOrigins": ["https://your-app.example.com", "http://localhost:3000"],
|
|
260
|
+
"ExposeHeaders": ["Content-Range", "Accept-Ranges"]
|
|
490
261
|
}
|
|
491
262
|
]
|
|
492
263
|
```
|
|
493
264
|
|
|
494
265
|
**Required for zip-peek:**
|
|
495
266
|
|
|
496
|
-
| Setting
|
|
497
|
-
|
|
498
|
-
| `AllowedHeaders: ["*"]`
|
|
499
|
-
| `AllowedMethods`: `GET`, `HEAD`
|
|
500
|
-
| `AllowedOrigins`
|
|
267
|
+
| Setting | Why |
|
|
268
|
+
| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
|
269
|
+
| `AllowedHeaders: ["*"]` | Allows the `Range` request header (triggers preflight). |
|
|
270
|
+
| `AllowedMethods`: `GET`, `HEAD` | Manifest and asset reads. |
|
|
271
|
+
| `AllowedOrigins` | Must include your app origin(s). Use `*` only for quick local testing. |
|
|
501
272
|
| `ExposeHeaders`: **`Content-Range`**, **`Accept-Ranges`** | The service worker reads `Content-Range` when parsing the ZIP index. These two headers are **required** in `ExposeHeaders`. |
|
|
502
273
|
|
|
503
274
|
S3 answers `OPTIONS` preflight automatically when CORS is configured; you do not list `OPTIONS` in `AllowedMethods`.
|
|
@@ -638,10 +409,7 @@ For example, after resolving the emitted filename:
|
|
|
638
409
|
|
|
639
410
|
```ts
|
|
640
411
|
await initZipPeek({
|
|
641
|
-
workerUrl: new URL(
|
|
642
|
-
`/assets/${zipServiceWorkerFilename}`,
|
|
643
|
-
window.location.origin,
|
|
644
|
-
).href,
|
|
412
|
+
workerUrl: new URL(`/assets/${zipServiceWorkerFilename}`, window.location.origin).href,
|
|
645
413
|
scopeUrl: new URL("/", window.location.origin).href,
|
|
646
414
|
});
|
|
647
415
|
```
|
|
@@ -718,11 +486,3 @@ For a detailed explanation of the modular service worker (`src/zipServiceWorker.
|
|
|
718
486
|
- Manifest entry fallback prepends the ZIP basename only (e.g. `session/icon.png` for `session.zip`); generic `*/filename` suffix matching is not used.
|
|
719
487
|
- Multi-range client requests are not supported (416).
|
|
720
488
|
- The package is browser-only and depends on Service Worker, Cache API, and HTTP range request support.
|
|
721
|
-
|
|
722
|
-
---
|
|
723
|
-
|
|
724
|
-
## Further reading
|
|
725
|
-
|
|
726
|
-
- [Medium-style article (HTML)](./docs/MEDIUM_POST.html) — animated diagrams
|
|
727
|
-
- [Article (Markdown)](./docs/MEDIUM_POST.md) — same story as Markdown
|
|
728
|
-
- [Zip service worker flow](./docs/ZIP_SERVICE_WORKER_FLOW.md)
|