@africanpilot/next-snapshot 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +149 -0
- package/cli.mjs +60 -0
- package/lib/browser.mjs +63 -0
- package/lib/bundle.mjs +213 -0
- package/lib/capture.mjs +522 -0
- package/lib/config.mjs +106 -0
- package/lib/key.js +29 -0
- package/lib/rewrite.mjs +195 -0
- package/lib/runtime/frame.js +404 -0
- package/lib/runtime/shell.js +426 -0
- package/lib/server.mjs +86 -0
- package/lib/verify.mjs +170 -0
- package/package.json +54 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 africanpilot
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# next-snapshot
|
|
2
|
+
|
|
3
|
+
Capture a running Next.js app and bundle it into **one HTML file** that opens
|
|
4
|
+
from disk — double-click, `file://`, network off — and still behaves like the
|
|
5
|
+
app: client components run, links and router navigation work, selects and tabs
|
|
6
|
+
work, and every page shows the data it showed when captured.
|
|
7
|
+
|
|
8
|
+
## Install
|
|
9
|
+
|
|
10
|
+
Requires **Node 22+** and **Google Chrome** (or any Chromium — see `browser`
|
|
11
|
+
below). Run it without installing:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
npx @africanpilot/next-snapshot all --config my-app.config.mjs --screens
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
or add it to a project, where the command is `next-snapshot`:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
npm install --save-dev @africanpilot/next-snapshot
|
|
21
|
+
npx next-snapshot all --config my-app.config.mjs
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Write a config for your app, starting from
|
|
25
|
+
[`examples/basic.config.mjs`](https://github.com/africanpilot/next-snapshot/blob/main/examples/basic.config.mjs)
|
|
26
|
+
(or [`roles.config.mjs`](https://github.com/africanpilot/next-snapshot/blob/main/examples/roles.config.mjs)
|
|
27
|
+
for an app with sign-in). The commands:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
next-snapshot all --config my-app.config.mjs # capture, bundle, verify
|
|
31
|
+
next-snapshot capture --config my-app.config.mjs # crawl the app (starts it if configured)
|
|
32
|
+
next-snapshot bundle --config my-app.config.mjs # capture dir -> one .html
|
|
33
|
+
next-snapshot verify --config my-app.config.mjs # open the .html offline, check it
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`verify` checks a sample — two URLs per route per variant — unless given
|
|
37
|
+
`--full`. `--screens` saves a screenshot of each page it checks.
|
|
38
|
+
|
|
39
|
+
Two examples: `examples/basic.config.mjs` (start the app, crawl from a couple of
|
|
40
|
+
entry points) and `examples/roles.config.mjs` (every role as a variant, the
|
|
41
|
+
app's own sign-in screen as the switcher, a POST emulated offline).
|
|
42
|
+
|
|
43
|
+
## How it works
|
|
44
|
+
|
|
45
|
+
A Next app cannot simply be inlined: its pages are rendered by a server (server
|
|
46
|
+
components, cookies, redirects), its client fetches more from that server as you
|
|
47
|
+
navigate, and a `file://` page cannot `fetch`, cannot load module scripts from
|
|
48
|
+
disk, and cannot `pushState` to a new path. So the tool does not transform the
|
|
49
|
+
source. It **records the real app and replays it**.
|
|
50
|
+
|
|
51
|
+
1. **Capture** (`lib/capture.mjs`). Starts the app (`next start`), then crawls it
|
|
52
|
+
in headless Chrome, once per *variant* (e.g. per role). It records the HTML of
|
|
53
|
+
every page exactly as served, every asset and client GET the page triggers,
|
|
54
|
+
and every redirect. It follows links, the URLs Next prefetches, and the URLs
|
|
55
|
+
a `<select>` produces when changed. Every non-GET request is aborted in the
|
|
56
|
+
browser, so the crawl cannot write to the app. Bodies are content-addressed,
|
|
57
|
+
so a chunk shared by 200 pages is stored once.
|
|
58
|
+
2. **Bundle** (`lib/bundle.mjs`). Rewrites every `location` reference in the
|
|
59
|
+
app's JS to a virtual location (esbuild `define`, so strings and local
|
|
60
|
+
variables are untouched), turns asset URLs in HTML and CSS into tokens, and
|
|
61
|
+
gzips + base64s every body into an inert `<script type="text/plain">`. One
|
|
62
|
+
file out, with a Content-Security-Policy that forbids all network access.
|
|
63
|
+
3. **Replay** (`lib/runtime/`). The outer page decodes assets into `blob:` URLs
|
|
64
|
+
and shows each page in a **fresh srcdoc iframe**. A shim runs first in every
|
|
65
|
+
frame: it supplies the virtual `location`, answers `fetch`/XHR from the
|
|
66
|
+
snapshot, maps runtime-created script/style/img URLs to their blobs (and
|
|
67
|
+
reports the original back to chunk loaders that look themselves up), and
|
|
68
|
+
intercepts links and forms. Next's RSC requests are deliberately answered
|
|
69
|
+
with a non-RSC response: Next then falls back to a full navigation, which
|
|
70
|
+
becomes a fresh frame of that page's captured HTML. Every navigation is
|
|
71
|
+
therefore a clean boot of exactly what the server sent — no framework state
|
|
72
|
+
to reconcile, no partial payloads to match.
|
|
73
|
+
4. **Verify** (`lib/verify.mjs`). Opens the file from `file://` with the network
|
|
74
|
+
off, loads every captured page, checks that React hydrated, clicks a link on
|
|
75
|
+
a few pages, and reports console errors, shim reports, and any attempt to
|
|
76
|
+
reach the network.
|
|
77
|
+
|
|
78
|
+
The address bar hash carries the page: `my-app.html#editor:/reports?year=2026`
|
|
79
|
+
opens that page as that variant, and back/forward work.
|
|
80
|
+
|
|
81
|
+
## Config
|
|
82
|
+
|
|
83
|
+
A config is an ES module; relative paths resolve against it.
|
|
84
|
+
|
|
85
|
+
| key | default | |
|
|
86
|
+
|---|---|---|
|
|
87
|
+
| `app.cwd`, `app.start`, `app.build`, `app.port` | — | How to run the app. `{port}` is substituted into `start`. `build` runs when there is no `.next/BUILD_ID`, or with `--build`. Omit `app` and set `url` to capture a server you started yourself. |
|
|
88
|
+
| `url` | `http://localhost:{port}` | Origin to capture. `localhost`, because Next builds redirect URLs on it and a session cookie set on `127.0.0.1` is not sent to `localhost`. |
|
|
89
|
+
| `aliases` | loopback spellings | Other origins that are the same app (a canonical host it redirects to); URLs on them are treated as the app's own. `localhost`, `127.0.0.1` and `[::1]` are always aliases of each other. |
|
|
90
|
+
| `out` | `./<name>.html` | Output file. The capture directory, bundle report, verify report and screenshots sit next to it. |
|
|
91
|
+
| `title` | — | Title shown while the file opens. |
|
|
92
|
+
| `start` | `/` | Page to open when the file has no hash. |
|
|
93
|
+
| `seeds` | `["/"]` | Where the crawl starts. Add any URL links and selects do not reach. |
|
|
94
|
+
| `exclude`, `include` | `/_next/`, `/api/` excluded | Regexes on the path+query key. |
|
|
95
|
+
| `maxPages` | 500 | Per variant. |
|
|
96
|
+
| `variants` | one | `[{ id, label, login({context, request, origin}) }]`. One crawl per variant; the file can switch between them. |
|
|
97
|
+
| `defaultVariant` | first | |
|
|
98
|
+
| `explore.selects` | `true` | Try each option of each visible `<select>` once per page path. |
|
|
99
|
+
| `explore.tabs` | `true` | Click each tab-like control once per page path: `[role=tab]`, and "button bars" — an element whose children are two or more buttons and nothing else. A URL the click writes (router push/replace, or a bare `history.replaceState`) is captured. |
|
|
100
|
+
| `explore.click` | `[]` | Extra CSS selectors to click the same way. |
|
|
101
|
+
| `explore.denyText` | sign out, delete, approve, submit, save… | Controls whose label matches are never clicked. Writes are blocked at the network anyway; this protects the session and client-side state. |
|
|
102
|
+
| `explore.custom` | — | `async ({page, key, variant, discover}) => {}` for app-specific discovery (clicking tabs that change the URL, etc). |
|
|
103
|
+
| `offline.post` | `{}` | `{ "/path": (fields, {variant, key}) => ({ variant?, location?, message? }) }` — emulate a POST in the browser. Must be an arrow or `function` expression; it is serialised into the file. Any POST without a handler is refused as read-only. |
|
|
104
|
+
| `offline.css` | `""` | CSS added to every page — for hiding what has no meaning offline (a sign-out button, a user menu). |
|
|
105
|
+
| `offline.missingLinks` | `"show"` | Links to pages the snapshot does not hold: `"show"` (they open a "not in snapshot" page), `"disable"` (dimmed, not clickable) or `"hide"`. With `include` narrowing the crawl, `"hide"` removes the nav entries for everything left out. |
|
|
106
|
+
| `offline.badge` | `"bottom-right"` | The "Offline snapshot" pill; `false` to hide. |
|
|
107
|
+
| `offline.switcher` | `true` | A variant `<select>` in the badge. |
|
|
108
|
+
| `includeStatic` | `true` | Also embed every file under `.next/static`, so lazily-loaded chunks the crawl never triggered are present. |
|
|
109
|
+
| `viewport`, `locale`, `timezoneId`, `browser` | | Passed to Chrome. `browser.executablePath` if Chrome is not installed. |
|
|
110
|
+
|
|
111
|
+
## What it cannot do
|
|
112
|
+
|
|
113
|
+
- **Anything not captured is not there.** A URL no link, prefetch, select or tab
|
|
114
|
+
led to shows a "not in this snapshot" page listing what was captured. Add it to
|
|
115
|
+
`seeds`, or teach `explore.custom` how to reach it. One exception: a URL the
|
|
116
|
+
app writes itself while you use the file (a tab calling `replaceState`) is
|
|
117
|
+
remembered, and returning to it re-serves the page it came from at that URL.
|
|
118
|
+
- **Exploration is one option at a time.** Each select option and each tab is
|
|
119
|
+
tried once per page path, from the first URL of that path the crawl reached —
|
|
120
|
+
not every combination. Seed the combinations you need.
|
|
121
|
+
- **Writes.** Forms and fetches that POST/PUT/DELETE are refused unless an
|
|
122
|
+
`offline.post` handler emulates them. Server Actions fail the same way.
|
|
123
|
+
- **Soft navigation.** Every navigation is a full page boot, so in-memory client
|
|
124
|
+
state (a React context, an open panel) does not survive a page change.
|
|
125
|
+
`localStorage` does, where the browser allows it on `file://`.
|
|
126
|
+
- **Size.** The file carries every page's HTML, including the data server
|
|
127
|
+
components embedded in it. Pages that differ only a little still compress
|
|
128
|
+
separately. Check the bundle report's "largest bodies".
|
|
129
|
+
- **Runtime URLs built inside CSS-in-JS** (`url()` in a style string set from
|
|
130
|
+
JS) are not remapped; they are reported as blocked by `verify`.
|
|
131
|
+
- **Other frameworks.** Nothing here is Next-specific except the RSC fallback
|
|
132
|
+
and the `.next/static` default; a plain SPA or a Remix/Nuxt app should capture,
|
|
133
|
+
but only Next App Router is tested.
|
|
134
|
+
|
|
135
|
+
## Security
|
|
136
|
+
|
|
137
|
+
A config is code: `app.build`, `app.start`, `login` hooks and `explore.custom`
|
|
138
|
+
run with your privileges. A snapshot contains every page it captured, for every
|
|
139
|
+
variant — treat it like access to the app. See [SECURITY.md](SECURITY.md),
|
|
140
|
+
including how to report a vulnerability.
|
|
141
|
+
|
|
142
|
+
## Development
|
|
143
|
+
|
|
144
|
+
`npm test` runs the unit and end-to-end suites; `npm run test:next` snapshots a
|
|
145
|
+
real Next.js app. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
146
|
+
|
|
147
|
+
## License
|
|
148
|
+
|
|
149
|
+
MIT — see [LICENSE](LICENSE).
|
package/cli.mjs
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// next-snapshot — capture a running Next.js app and bundle it into one HTML file
|
|
3
|
+
// that opens from disk with no network.
|
|
4
|
+
//
|
|
5
|
+
// node cli.mjs all --config app.config.mjs capture, bundle, verify
|
|
6
|
+
// node cli.mjs capture --config app.config.mjs crawl the app (starts it if configured)
|
|
7
|
+
// node cli.mjs bundle --config app.config.mjs capture dir -> one .html
|
|
8
|
+
// node cli.mjs verify --config app.config.mjs open the .html offline, walk every page
|
|
9
|
+
//
|
|
10
|
+
// Options: --build (rebuild the app first) --screens (verify saves screenshots)
|
|
11
|
+
// --limit N / --variant ID (verify a subset) --no-verify (all: skip verify)
|
|
12
|
+
|
|
13
|
+
import { parseArgs } from "node:util";
|
|
14
|
+
|
|
15
|
+
import { bundle } from "./lib/bundle.mjs";
|
|
16
|
+
import { capture } from "./lib/capture.mjs";
|
|
17
|
+
import { loadConfig } from "./lib/config.mjs";
|
|
18
|
+
import { withServer } from "./lib/server.mjs";
|
|
19
|
+
import { verify } from "./lib/verify.mjs";
|
|
20
|
+
|
|
21
|
+
const { values, positionals } = parseArgs({
|
|
22
|
+
allowPositionals: true,
|
|
23
|
+
options: {
|
|
24
|
+
config: { type: "string", short: "c" },
|
|
25
|
+
build: { type: "boolean" },
|
|
26
|
+
screens: { type: "boolean" },
|
|
27
|
+
full: { type: "boolean" },
|
|
28
|
+
limit: { type: "string" },
|
|
29
|
+
variant: { type: "string" },
|
|
30
|
+
"no-verify": { type: "boolean" },
|
|
31
|
+
help: { type: "boolean", short: "h" },
|
|
32
|
+
},
|
|
33
|
+
});
|
|
34
|
+
|
|
35
|
+
const cmd = positionals[0] ?? "all";
|
|
36
|
+
if (values.help || !["all", "capture", "bundle", "verify"].includes(cmd)) {
|
|
37
|
+
console.log(
|
|
38
|
+
"usage: next-snapshot [all|capture|bundle|verify] --config app.config.mjs [--build] [--screens] [--full] [--limit N] [--variant ID] [--no-verify]",
|
|
39
|
+
);
|
|
40
|
+
process.exit(values.help ? 0 : 2);
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
const log = (s = "") => console.log(s);
|
|
44
|
+
const cfg = await loadConfig(values.config);
|
|
45
|
+
const verifyOpts = { screens: values.screens, full: values.full, limit: values.limit ? +values.limit : 0, variant: values.variant };
|
|
46
|
+
|
|
47
|
+
try {
|
|
48
|
+
if (cmd === "capture" || cmd === "all") {
|
|
49
|
+
await withServer(cfg, { build: values.build }, log, () => capture(cfg, log));
|
|
50
|
+
}
|
|
51
|
+
if (cmd === "bundle" || cmd === "all") await bundle(cfg, log);
|
|
52
|
+
if (cmd === "verify" || (cmd === "all" && !values["no-verify"])) {
|
|
53
|
+
const r = await verify(cfg, verifyOpts, log);
|
|
54
|
+
if (r.failed || r.leaks || r.clickFailures) process.exitCode = 1;
|
|
55
|
+
}
|
|
56
|
+
} catch (e) {
|
|
57
|
+
console.error(`\nnext-snapshot: ${e.message}`);
|
|
58
|
+
if (process.env.DEBUG) console.error(e.stack);
|
|
59
|
+
process.exit(1);
|
|
60
|
+
}
|
package/lib/browser.mjs
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
// Find a Chromium to drive. Installed Chrome first (no download needed), then
|
|
2
|
+
// any browser Playwright has cached, then whatever the config names.
|
|
3
|
+
|
|
4
|
+
import fs from "node:fs";
|
|
5
|
+
import os from "node:os";
|
|
6
|
+
import path from "node:path";
|
|
7
|
+
import { chromium } from "playwright-core";
|
|
8
|
+
|
|
9
|
+
export async function launch(cfg) {
|
|
10
|
+
const b = cfg.browser ?? {};
|
|
11
|
+
const tries = [];
|
|
12
|
+
if (b.executablePath) tries.push({ executablePath: b.executablePath });
|
|
13
|
+
if (b.channel !== false) tries.push({ channel: b.channel ?? "chrome" });
|
|
14
|
+
for (const p of cachedChromiums()) tries.push({ executablePath: p });
|
|
15
|
+
|
|
16
|
+
let last;
|
|
17
|
+
for (const t of tries) {
|
|
18
|
+
try {
|
|
19
|
+
return await chromium.launch({ headless: b.headless ?? true, ...t });
|
|
20
|
+
} catch (e) {
|
|
21
|
+
last = e;
|
|
22
|
+
}
|
|
23
|
+
}
|
|
24
|
+
throw new Error(
|
|
25
|
+
"No Chromium could be launched. Install Google Chrome, or set browser.executablePath in the config.\n" +
|
|
26
|
+
(last?.message ?? ""),
|
|
27
|
+
);
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
function cachedChromiums() {
|
|
31
|
+
const roots = [
|
|
32
|
+
process.env.PLAYWRIGHT_BROWSERS_PATH,
|
|
33
|
+
path.join(os.homedir(), "Library", "Caches", "ms-playwright"),
|
|
34
|
+
path.join(os.homedir(), ".cache", "ms-playwright"),
|
|
35
|
+
path.join(os.homedir(), "AppData", "Local", "ms-playwright"),
|
|
36
|
+
].filter(Boolean);
|
|
37
|
+
const rels = [
|
|
38
|
+
"chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing",
|
|
39
|
+
"chrome-mac/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing",
|
|
40
|
+
"chrome-mac-arm64/Chromium.app/Contents/MacOS/Chromium",
|
|
41
|
+
"chrome-mac/Chromium.app/Contents/MacOS/Chromium",
|
|
42
|
+
"chrome-linux64/chrome",
|
|
43
|
+
"chrome-linux/chrome",
|
|
44
|
+
"chrome-win64/chrome.exe",
|
|
45
|
+
"chrome-win/chrome.exe",
|
|
46
|
+
];
|
|
47
|
+
const out = [];
|
|
48
|
+
for (const root of roots) {
|
|
49
|
+
let names;
|
|
50
|
+
try {
|
|
51
|
+
names = fs.readdirSync(root);
|
|
52
|
+
} catch {
|
|
53
|
+
continue;
|
|
54
|
+
}
|
|
55
|
+
for (const n of names.filter((n) => /^chromium-\d+$/.test(n)).sort().reverse()) {
|
|
56
|
+
for (const rel of rels) {
|
|
57
|
+
const p = path.join(root, n, rel);
|
|
58
|
+
if (fs.existsSync(p)) out.push(p);
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
return out;
|
|
63
|
+
}
|
package/lib/bundle.mjs
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
1
|
+
// Turn a capture directory into one HTML file.
|
|
2
|
+
//
|
|
3
|
+
// JS location references are rewritten to a virtual location object.
|
|
4
|
+
// CSS url(...) and @import references become asset tokens.
|
|
5
|
+
// HTML asset-bearing attributes become tokens; a <base> naming the page's
|
|
6
|
+
// real URL and a placeholder for the frame shim go first in <head>.
|
|
7
|
+
// (The transformations themselves live in rewrite.mjs.)
|
|
8
|
+
//
|
|
9
|
+
// Every body is gzipped and base64'd into its own inert <script type=text/plain>.
|
|
10
|
+
// The runtime (runtime/shell.js) decodes assets once into blob: URLs, replaces
|
|
11
|
+
// tokens with them, and decodes a page only when it is navigated to.
|
|
12
|
+
|
|
13
|
+
import crypto from "node:crypto";
|
|
14
|
+
import fs from "node:fs/promises";
|
|
15
|
+
import path from "node:path";
|
|
16
|
+
import { fileURLToPath } from "node:url";
|
|
17
|
+
import zlib from "node:zlib";
|
|
18
|
+
|
|
19
|
+
import { urlKey } from "./key.js";
|
|
20
|
+
import { createRewriter, escHTML, serialisePost, virtualiseLocation } from "./rewrite.mjs";
|
|
21
|
+
|
|
22
|
+
const HERE = path.dirname(fileURLToPath(import.meta.url));
|
|
23
|
+
|
|
24
|
+
// Nothing in the file may reach the network. Anything that tries is refused by
|
|
25
|
+
// the browser and shows up as a CSP violation, which `verify` reports.
|
|
26
|
+
export const CSP = [
|
|
27
|
+
"default-src 'none'",
|
|
28
|
+
"script-src 'unsafe-inline' 'unsafe-eval' blob: data:",
|
|
29
|
+
"style-src 'unsafe-inline' blob: data:",
|
|
30
|
+
"img-src blob: data:",
|
|
31
|
+
"font-src blob: data:",
|
|
32
|
+
"media-src blob: data:",
|
|
33
|
+
"connect-src blob: data:",
|
|
34
|
+
"worker-src blob:",
|
|
35
|
+
"frame-src blob: data: about:",
|
|
36
|
+
"form-action 'none'",
|
|
37
|
+
].join("; ");
|
|
38
|
+
|
|
39
|
+
export async function bundle(cfg, log) {
|
|
40
|
+
const M = JSON.parse(await fs.readFile(path.join(cfg.captureDir, "manifest.json"), "utf8"));
|
|
41
|
+
const origin = M.origin;
|
|
42
|
+
const readBody = (sha) => fs.readFile(path.join(cfg.captureDir, "bodies", sha));
|
|
43
|
+
const t0 = Date.now();
|
|
44
|
+
|
|
45
|
+
// --- asset table ---------------------------------------------------------
|
|
46
|
+
const assets = [];
|
|
47
|
+
for (const [k, e] of Object.entries(M.assets)) assets.push({ k, v: null, ...e });
|
|
48
|
+
for (const [v, map] of Object.entries(M.variantAssets)) for (const [k, e] of Object.entries(map)) assets.push({ k, v, ...e });
|
|
49
|
+
const index = new Map(assets.map((a, i) => [`${a.v ?? ""}\n${a.k}`, i]));
|
|
50
|
+
const lookup = (key, v) => {
|
|
51
|
+
if (!key) return -1;
|
|
52
|
+
let i = index.get(`${v ?? ""}\n${key}`) ?? index.get(`\n${key}`);
|
|
53
|
+
if (i == null && key.includes("?") && key.startsWith(cfg.staticPrefix)) i = index.get(`\n${key.split("?")[0]}`);
|
|
54
|
+
return i ?? -1;
|
|
55
|
+
};
|
|
56
|
+
|
|
57
|
+
// --- per-page CSS ----------------------------------------------------------
|
|
58
|
+
const cssRules = [cfg.offline.css ?? ""];
|
|
59
|
+
if (cfg.offline.missingLinks === "hide") cssRules.push("a[data-no-missing],area[data-no-missing]{display:none!important}");
|
|
60
|
+
if (cfg.offline.missingLinks === "disable") {
|
|
61
|
+
cssRules.push("a[data-no-missing],area[data-no-missing]{opacity:.45;pointer-events:none;cursor:not-allowed}");
|
|
62
|
+
}
|
|
63
|
+
const pageCSS = cssRules.join("\n").trim()
|
|
64
|
+
? `<style data-no-css>${cssRules.join("\n").replace(/<\/style/gi, "<\\/style")}</style>`
|
|
65
|
+
: "";
|
|
66
|
+
|
|
67
|
+
const missing = new Map(); // referenced but never captured -> count
|
|
68
|
+
const { rewriteCSS, rewriteHTML } = createRewriter({
|
|
69
|
+
origin,
|
|
70
|
+
aliases: cfg.aliases,
|
|
71
|
+
lookup,
|
|
72
|
+
pageCSS,
|
|
73
|
+
onMiss: (k) => missing.set(k, (missing.get(k) ?? 0) + 1),
|
|
74
|
+
});
|
|
75
|
+
|
|
76
|
+
// --- bodies ---------------------------------------------------------------
|
|
77
|
+
const out = []; // id -> base64(gzip)
|
|
78
|
+
const outIndex = new Map(); // content hash -> id
|
|
79
|
+
const sizes = []; // {what, raw, packed}
|
|
80
|
+
function emit(buf, what) {
|
|
81
|
+
const h = crypto.createHash("sha256").update(buf).digest("hex");
|
|
82
|
+
let id = outIndex.get(h);
|
|
83
|
+
if (id == null) {
|
|
84
|
+
const packed = zlib.gzipSync(buf, { level: 9 }).toString("base64");
|
|
85
|
+
id = out.length;
|
|
86
|
+
out.push(packed);
|
|
87
|
+
outIndex.set(h, id);
|
|
88
|
+
sizes.push({ what, raw: buf.length, packed: packed.length });
|
|
89
|
+
}
|
|
90
|
+
return id;
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
const jsCache = new Map();
|
|
94
|
+
let jsRewritten = 0;
|
|
95
|
+
const isJS = (a) => /javascript|ecmascript/i.test(a.type) || /\.m?js$/.test(a.k.split("?")[0]);
|
|
96
|
+
const isCSS = (a) => /text\/css/i.test(a.type) || /\.css$/.test(a.k.split("?")[0]);
|
|
97
|
+
|
|
98
|
+
for (const a of assets) {
|
|
99
|
+
let buf = await readBody(a.body);
|
|
100
|
+
if (isJS(a)) {
|
|
101
|
+
if (!jsCache.has(a.body)) {
|
|
102
|
+
try {
|
|
103
|
+
jsCache.set(a.body, Buffer.from(await virtualiseLocation(buf.toString("utf8"))));
|
|
104
|
+
} catch (e) {
|
|
105
|
+
log(` warn: esbuild could not parse ${a.k}; left as-is, location in it is NOT virtualised (${e.message.split("\n")[0]})`);
|
|
106
|
+
jsCache.set(a.body, buf);
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
buf = jsCache.get(a.body);
|
|
110
|
+
jsRewritten++;
|
|
111
|
+
} else if (isCSS(a)) {
|
|
112
|
+
const base = a.k.startsWith("/") ? origin + a.k : a.k;
|
|
113
|
+
buf = Buffer.from(rewriteCSS(buf.toString("utf8"), base, a.v));
|
|
114
|
+
}
|
|
115
|
+
a.b = emit(buf, `asset ${a.k}`);
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
// --- pages ------------------------------------------------------------------
|
|
119
|
+
const pagesOut = {};
|
|
120
|
+
let pageCount = 0;
|
|
121
|
+
for (const [v, P] of Object.entries(M.pages)) {
|
|
122
|
+
pagesOut[v] = {};
|
|
123
|
+
for (const [k, e] of Object.entries(P)) {
|
|
124
|
+
if (e.redirect != null) pagesOut[v][k] = { r: e.redirect };
|
|
125
|
+
else if (e.file != null) {
|
|
126
|
+
const i = lookup(e.file, v);
|
|
127
|
+
if (i >= 0) pagesOut[v][k] = { a: i };
|
|
128
|
+
} else {
|
|
129
|
+
const html = (await readBody(e.body)).toString("utf8");
|
|
130
|
+
pagesOut[v][k] = { b: emit(Buffer.from(rewriteHTML(html, k, v)), `page ${v} ${k}`), s: e.status };
|
|
131
|
+
pageCount++;
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
// --- runtime + output ---------------------------------------------------------
|
|
137
|
+
const frameSrc = await fs.readFile(path.join(HERE, "runtime", "frame.js"), "utf8");
|
|
138
|
+
const shellSrc = (await fs.readFile(path.join(HERE, "runtime", "shell.js"), "utf8"))
|
|
139
|
+
.replace("/*@URLKEY@*/null", () => `(${urlKey.toString()})`)
|
|
140
|
+
.replace('/*@FRAME@*/""', () => JSON.stringify(frameSrc).replace(/</g, "\\u003c"))
|
|
141
|
+
.replace("/*@POST@*/{}", () => serialisePost(cfg.offline.post));
|
|
142
|
+
|
|
143
|
+
const manifestOut = {
|
|
144
|
+
v: 1,
|
|
145
|
+
origin,
|
|
146
|
+
aliases: cfg.aliases,
|
|
147
|
+
title: cfg.title ?? M.title ?? null,
|
|
148
|
+
createdAt: M.createdAt,
|
|
149
|
+
variants: M.variants,
|
|
150
|
+
defaultVariant: M.defaultVariant,
|
|
151
|
+
start: M.start,
|
|
152
|
+
badge: cfg.offline.badge,
|
|
153
|
+
switcher: cfg.offline.switcher,
|
|
154
|
+
missingLinks: cfg.offline.missingLinks,
|
|
155
|
+
assets: assets.map((a) => [a.k, a.v, a.b, a.type, a.status ?? 200]),
|
|
156
|
+
pages: pagesOut,
|
|
157
|
+
};
|
|
158
|
+
|
|
159
|
+
const title = escHTML(manifestOut.title ?? "Offline snapshot");
|
|
160
|
+
const parts = [
|
|
161
|
+
`<!doctype html>\n<html lang="en"><head><meta charset="utf-8">`,
|
|
162
|
+
`<meta name="viewport" content="width=device-width, initial-scale=1">`,
|
|
163
|
+
`<meta http-equiv="Content-Security-Policy" content="${CSP}">`,
|
|
164
|
+
`<meta name="generator" content="next-snapshot">`,
|
|
165
|
+
`<title>${title}</title>`,
|
|
166
|
+
`<style>${SHELL_CSS}</style></head><body>`,
|
|
167
|
+
`<noscript>This offline snapshot needs JavaScript enabled.</noscript>`,
|
|
168
|
+
`<div id="no-loading">Opening ${title}…</div>`,
|
|
169
|
+
`<script type="application/json" id="no-manifest">${JSON.stringify(manifestOut).replace(/</g, "\\u003c")}</script>`,
|
|
170
|
+
];
|
|
171
|
+
for (let i = 0; i < out.length; i++) parts.push(`<script type="text/plain" id="no-b${i}">${out[i]}</script>`);
|
|
172
|
+
parts.push(`<script>${shellSrc.replace(/<\/script/gi, "<\\/script").replace(/<!--/g, "<\\!--")}</script>`);
|
|
173
|
+
parts.push(`</body></html>\n`);
|
|
174
|
+
|
|
175
|
+
await fs.mkdir(path.dirname(cfg.out), { recursive: true });
|
|
176
|
+
const html = parts.join("\n");
|
|
177
|
+
await fs.writeFile(cfg.out, html);
|
|
178
|
+
|
|
179
|
+
// --- report ----------------------------------------------------------------------
|
|
180
|
+
const mb = (n) => (n / 1024 / 1024).toFixed(2) + " MB";
|
|
181
|
+
const pageBytes = sizes.filter((s) => s.what.startsWith("page")).reduce((n, s) => n + s.packed, 0);
|
|
182
|
+
const assetBytes = sizes.filter((s) => s.what.startsWith("asset")).reduce((n, s) => n + s.packed, 0);
|
|
183
|
+
log("");
|
|
184
|
+
log(`bundle finished in ${((Date.now() - t0) / 1000).toFixed(1)}s -> ${path.relative(process.cwd(), cfg.out)}`);
|
|
185
|
+
log(` file size ${mb(html.length)} (pages ${mb(pageBytes)}, assets ${mb(assetBytes)})`);
|
|
186
|
+
log(` pages ${pageCount} captured, ${out.length} unique bodies after dedupe`);
|
|
187
|
+
log(` assets ${assets.length} (${jsRewritten} scripts with location virtualised)`);
|
|
188
|
+
const biggest = [...sizes].sort((a, b) => b.packed - a.packed).slice(0, 6);
|
|
189
|
+
log(` largest bodies ${biggest.map((s) => `${s.what.slice(0, 70)} ${mb(s.packed)}`).join("\n ")}`);
|
|
190
|
+
if (missing.size) {
|
|
191
|
+
const top = [...missing.entries()].sort((a, b) => b[1] - a[1]).slice(0, 10);
|
|
192
|
+
log(` NOT CAPTURED ${missing.size} referenced URL(s) will fail offline:`);
|
|
193
|
+
for (const [k, n] of top) log(` ${k} (${n}x)`);
|
|
194
|
+
}
|
|
195
|
+
const report = { file: cfg.out, bytes: html.length, pageCount, bodies: out.length, missing: Object.fromEntries(missing), sizes };
|
|
196
|
+
await fs.writeFile(cfg.out.replace(/\.html?$/, "") + ".bundle.json", JSON.stringify(report, null, 1));
|
|
197
|
+
return report;
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
const SHELL_CSS = `
|
|
201
|
+
html,body{margin:0;height:100%;overflow:hidden;background:#fff}
|
|
202
|
+
.no-frame{position:fixed;inset:0;width:100%;height:100%;border:0;display:block;background:#fff}
|
|
203
|
+
.no-frame.loading{visibility:hidden}
|
|
204
|
+
#no-loading{position:fixed;inset:0;display:flex;align-items:center;justify-content:center;font:14px system-ui,-apple-system,"Segoe UI",sans-serif;color:#666;background:#fff;z-index:1}
|
|
205
|
+
#no-badge{position:fixed;z-index:2147483646;display:flex;align-items:center;gap:8px;padding:5px 10px;border-radius:999px;background:rgba(20,20,20,.82);color:#fff;font:500 11px/1.3 system-ui,-apple-system,"Segoe UI",sans-serif;opacity:.5;transition:opacity .15s;box-shadow:0 2px 8px rgba(0,0,0,.2)}
|
|
206
|
+
#no-badge:hover,#no-badge:focus-within{opacity:1}
|
|
207
|
+
#no-badge.bottom-right{right:12px;bottom:12px}#no-badge.bottom-left{left:12px;bottom:12px}#no-badge.top-right{right:12px;top:12px}#no-badge.top-left{left:12px;top:12px}
|
|
208
|
+
#no-badge select{font:inherit;color:#fff;background:rgba(255,255,255,.12);border:1px solid rgba(255,255,255,.25);border-radius:6px;padding:1px 4px}
|
|
209
|
+
#no-badge select option{color:#000}
|
|
210
|
+
#no-badge .dot{width:7px;height:7px;border-radius:50%;background:#e9b949;flex:none}
|
|
211
|
+
#no-toast{position:fixed;left:50%;bottom:24px;transform:translateX(-50%);z-index:2147483647;max-width:min(520px,calc(100vw - 32px));padding:10px 14px;border-radius:8px;background:#1f1f1f;color:#fff;font:13px/1.4 system-ui,-apple-system,"Segoe UI",sans-serif;box-shadow:0 6px 24px rgba(0,0,0,.25);opacity:0;transition:opacity .2s;pointer-events:none}
|
|
212
|
+
#no-toast.show{opacity:1}
|
|
213
|
+
`;
|