@runeditrun/rer 0.2.44 → 0.2.46
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +87 -1
- package/cli/README.md +8 -0
- package/cli/companion/api.mjs +27 -0
- package/cli/companion/auth.mjs +34 -0
- package/cli/companion/command.mjs +73 -0
- package/cli/companion/commands.mjs +22 -0
- package/cli/companion/event-payload.mjs +7 -0
- package/cli/companion/grants.mjs +30 -0
- package/cli/companion/no-turn-evidence.mjs +49 -0
- package/cli/companion/providers/claude.mjs +31 -0
- package/cli/companion/providers/codex.mjs +55 -0
- package/cli/companion/providers/jsonl-rpc.mjs +95 -0
- package/cli/companion/providers/sandbox.mjs +58 -0
- package/cli/companion/providers/secrets.mjs +36 -0
- package/cli/companion/recovery.mjs +67 -0
- package/cli/companion/runner.mjs +121 -0
- package/cli/companion/setup.mjs +71 -0
- package/cli/companion/state.mjs +10 -0
- package/cli/companion/workspace.mjs +101 -0
- package/cli/deploy.mjs +1 -1
- package/cli/local.mjs +6 -2
- package/cli/main.mjs +1 -0
- package/cli/port.mjs +10 -11
- package/cli/server-limit-file.mjs +21 -0
- package/cli/server-registry.mjs +13 -6
- package/cli/server-resources.mjs +13 -2
- package/global/CHANGELOG.md +14 -0
- package/global/README.md +87 -1
- package/global/launch/capacity.mjs +9 -4
- package/global/launch/facts.mjs +11 -19
- package/global/main.mjs +3 -0
- package/modules/demo-host/README.md +3 -0
- package/modules/demo-host/build.mjs +3 -1
- package/modules/demo-host/native/config.mjs +2 -0
- package/modules/demo-host/native/prepare.mjs +3 -1
- package/modules/demo-host/native/visitor.mjs +20 -7
- package/modules/demo-host/native/warm.mjs +49 -0
- package/modules/demo-host/native/worker.mjs +4 -2
- package/modules/demo-host/pages.mjs +6 -1
- package/modules/demo-host/worker.mjs +3 -2
- package/modules/deploy-kit/README.md +15 -1
- package/modules/deploy-kit/deploy.mjs +5 -3
- package/modules/deploy-kit/diagnostics.mjs +8 -0
- package/modules/deploy-kit/migration-ledger.mjs +28 -8
- package/modules/deploy-kit/migration-retry.mjs +32 -16
- package/modules/deploy-kit/migrations.mjs +10 -4
- package/modules/deploy-kit/owner-cli.mjs +1 -0
- package/modules/deploy-kit/owner.mjs +2 -1
- package/onboarding/port-probe.mjs +23 -1
- package/onboarding/process-diagnostics.mjs +3 -2
- package/package.json +1 -1
package/global/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.2.46
|
|
4
|
+
|
|
5
|
+
- **`rer companion` pairs your computer with a Run Edit Run app and runs its change requests here.** The local companion is now a `rer` verb: `map` approves one Git checkout whose `origin` is exactly the app's GitHub repository, `pair` redeems the one-use code from the app's Devices page, `codex-auth` or `claude-auth` sets up a dedicated executor home, `commands` approves setup and verify argv lists, and `run` takes jobs from the app. Each job starts from the exact commit the app names in a separate detached checkout, and the provider runs inside a macOS `sandbox-exec` boundary with no GitHub, Cloudflare or MCP credentials; on other systems `run` refuses before it asks for work. It makes outbound HTTPS requests only and opens no local server. State lives in `~/.rer-local` (`RER_LOCAL_HOME` picks another folder), the device key in a 0600 file. `status` prints the paired device, mapped projects, active job and pending events, never the key. Revoke a device in the app under Settings › Devices (#202).
|
|
6
|
+
- **A native demo frees its capacity place before it erases the visitor's storage.** Cancel, sign-out, expiry and a failed start now stop the copy and release its place first, then erase; a retry alarm repeats the erasure until it finishes. Before, a failing or slow erase (an upload still streaming into storage) held the place for up to an hour, so the next visitor could be refused. A start also re-checks that it was not ended while it extended its reservation (#213).
|
|
7
|
+
|
|
8
|
+
## 0.2.45
|
|
9
|
+
|
|
10
|
+
- **`RER_SERVER_LIMIT_FILE` sets the server limit from a file.** A start that sees it reads the positive integer in the named file and uses that as the limit, over its own `RER_SERVER_LIMIT` and over the limits running servers recorded, in both the launcher's check and the dispatcher's registry. A fleet's launcher points every run at one file, so a long-lived run that still exports an older `RER_SERVER_LIMIT` can no longer cap the rest. An unreadable file or a non-integer refuses the start naming the file; the limit refusal names it when it set the cap. Without the variable nothing changes (#205).
|
|
11
|
+
- **`rer deploy` retries a migration that D1 rolled back, when a reviewed rewrite replaced its bytes.** If a hosted D1 import fails, D1 rolls it back, but rer's ledger keeps an `applying` claim on the old bytes. Before, a reviewed rewrite in `migrations/rewrites.json` covered only an `applied` row, so `rer deploy` refused with `changed-or-uncertain-migration-history`. Now, holding its deploy state lock, `rer deploy` reruns a claim whose bytes are a declared `rewrites[].from`, using the current bytes. It does so only after the same proof that none of the effects are present, and it prints `Migrations: retried <name>: the earlier attempt was rolled back`. A claim on any other bytes is still refused, and the refusal now gives the exact `wrangler d1 execute` command, filled in, and says when running it is safe. A table the migration rebuilds no longer blocks the retry. The absence check also now fits within D1's limit on compound SELECT terms (#208).
|
|
12
|
+
- **A seed can declare the host a fresh checkout binds.** A seed's `commandAdapter({ host })` names `127.0.0.1`, `localhost` or `::1`, and a fresh checkout binds that host. Any other value is refused. A saved origin keeps its own host, and `--host` and the front door still take precedence. Auth declares `localhost`, because WebAuthn refuses an IP address as a passkey relying-party ID. In package mode Auth no longer needs the `pnpm patch` of `@runeditrun/rer`, which `rer check` SC-13 refused as a patched pin (#204).
|
|
13
|
+
- **`RER_SERVERS_DISK_FLOOR_GIB` sets `rer dev`'s free-disk floor.** The value is a number of GiB (decimals allowed, 0 for no floor). Unset, the floor is 15, as before. An invalid value refuses with exit 4 and names the variable. The disk refusal states the floor it used, the free space it measured and the path it measured, and names the variable. CI or managed build runners on small disks may set it lower, for example 4 (#210).
|
|
14
|
+
- **The hosted demo start page prefetches the console's chunks while the visitor waits.** `rer deploy --demo` writes an ordered list of the seed's hashed `/assets` chunks into the native demo bundle. The root and console layouts come first, then the console's own views, capped at 1 MiB gzip. The start page names them as `<link rel="prefetch">`. The first visit to each console view then finds its chunks already at the edge (Mango's first-visit navigation p50 went from about 380 ms to about 90 ms on a cold colo). The start request, the sign-in redirect and the CSP are unchanged (#207).
|
|
15
|
+
- **`rer dev` and `rer setup` no longer give up on a free port because the computer is busy.** Before, both free-port searches stopped on a wall-clock deadline. Under heavy load, `rer dev` could stop with "Could not find a free port … in time (system busy)" even with free ports ahead. Now both try the next 1000 ports and wait for each bind's own answer. They give up only when every candidate was refused, and the message names the range and the count (#211).
|
|
16
|
+
|
|
3
17
|
## 0.2.44
|
|
4
18
|
|
|
5
19
|
- **Under `rer dev`, a seed can link to another local seed over http.** The dependencies module accepts a provider address of `http://127.0.0.1`, `http://[::1]` or `http://localhost`, on any port, for the RER auth preset, but only when the app runs under `rer dev`. That lets a local Balm link to a local Auth. Every other `http` address, and every hosted deploy, still requires https (#196).
|
package/global/README.md
CHANGED
|
@@ -181,11 +181,17 @@ naming their PIDs.
|
|
|
181
181
|
|
|
182
182
|
Servers started through `rer` (`rer dev`, `rer preview`) get a 10-server limit and a 12-hour idle timeout
|
|
183
183
|
unless you set `RER_SERVER_LIMIT`, `RER_SERVER_IDLE_SECONDS` or `--idle-timeout`.
|
|
184
|
+
`RER_SERVER_LIMIT_FILE=PATH` names a file holding the limit as a positive integer,
|
|
185
|
+
read at every start; it overrides `RER_SERVER_LIMIT` and the limits running servers
|
|
186
|
+
recorded, so a fleet's one current value cannot be capped by an older run's
|
|
187
|
+
environment. An unreadable or non-integer file refuses the start, naming the file.
|
|
184
188
|
`rer` supplies them only to checkouts whose pinned copy applies lower defaults
|
|
185
189
|
or does not report its own (`rer describe` reports `defaults`), so a current
|
|
186
190
|
checkout never records a limit you did not set.
|
|
187
191
|
A start is never refused for machine load alone unless `RER_SERVERS_LOAD_THRESHOLD`
|
|
188
|
-
or `RER_JOBS_LOAD_THRESHOLD` is set; memory and disk guards stay.
|
|
192
|
+
or `RER_JOBS_LOAD_THRESHOLD` is set; memory and disk guards stay. The disk guard
|
|
193
|
+
needs 15 GiB free; `RER_SERVERS_DISK_FLOOR_GIB=N` changes it (CI runners may set
|
|
194
|
+
it lower, for example 4; people should not).
|
|
189
195
|
|
|
190
196
|
## The front door
|
|
191
197
|
|
|
@@ -426,3 +432,83 @@ The four deterministic adapter implementations and their native output formats
|
|
|
426
432
|
are described in [the adapter guide](cli/business/vendors/README.md). Their
|
|
427
433
|
presence does not enable export until the seed import authority and remaining
|
|
428
434
|
transport contracts are connected.
|
|
435
|
+
|
|
436
|
+
## Local engineering companion
|
|
437
|
+
|
|
438
|
+
`rer companion` pairs your own computer with a Run Edit Run app, then runs the
|
|
439
|
+
change requests the app sends it with your own Codex or Claude, here. It makes
|
|
440
|
+
outbound HTTPS requests to the app only and opens no local server. It needs
|
|
441
|
+
Node 22 or later; `run` needs macOS, because each provider runs inside a macOS
|
|
442
|
+
`sandbox-exec` boundary. On other systems `run` refuses before it asks the app
|
|
443
|
+
for work: it never runs a provider without that boundary.
|
|
444
|
+
|
|
445
|
+
```sh
|
|
446
|
+
npx -y @runeditrun/rer companion map PROJECT_ID <owner>/<repo> /path/to/your/checkout
|
|
447
|
+
npx -y @runeditrun/rer companion pair https://app.runeditrun.com ABCD-2345
|
|
448
|
+
npx -y @runeditrun/rer companion codex-auth # or: claude-auth
|
|
449
|
+
npx -y @runeditrun/rer companion run
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
With `rer` installed, drop `npx -y @runeditrun/rer`. Run `rer companion --help`
|
|
453
|
+
for every verb.
|
|
454
|
+
|
|
455
|
+
- `map PROJECT_ID <owner>/<repo> CHECKOUT` approves one local Git checkout for an
|
|
456
|
+
app. Its `origin` must be exactly that GitHub repository. `PROJECT_ID` is the
|
|
457
|
+
app's id, shown in the command on its Devices page. On a Managed app, if you
|
|
458
|
+
cloned with the temporary clone URL the app gives you, reset `origin` first
|
|
459
|
+
with `git remote set-url origin https://github.com/<owner>/<repo>.git`: `map`
|
|
460
|
+
refuses a URL that carries a token, and the reset keeps the expired token out
|
|
461
|
+
of your `.git/config`.
|
|
462
|
+
- `pair URL CODE` redeems the one-use code from the app's Devices page (the
|
|
463
|
+
8-character code or the full code; both expire after 10 minutes). The app
|
|
464
|
+
must name the repository you mapped, or this computer stays unpaired.
|
|
465
|
+
`--project ID`, `--provider codex|claude` and `--name NAME` set the rest; in
|
|
466
|
+
a terminal it asks for a missing project (and offers to map it) and provider.
|
|
467
|
+
With exactly one mapped project it uses that one, and the provider defaults
|
|
468
|
+
to `codex`. The earlier form `pair URL CODE PROJECT_ID [PROVIDER] [NAME]`
|
|
469
|
+
still works.
|
|
470
|
+
- `codex-auth [AUTH_FILE]` copies only Codex's `auth.json`
|
|
471
|
+
(`$CODEX_HOME/auth.json`, else `~/.codex/auth.json`) into a dedicated
|
|
472
|
+
executor home. Codex configuration and MCP servers are not imported.
|
|
473
|
+
- `claude-auth` runs Claude Code's own sign-in with the dedicated executor home
|
|
474
|
+
(add `--console` for a Console account). Host Claude configuration and
|
|
475
|
+
history are not copied. To use API billing instead, set
|
|
476
|
+
`RER_COMPANION_ANTHROPIC_API_KEY`; a host-wide `ANTHROPIC_API_KEY` is never
|
|
477
|
+
passed to the provider.
|
|
478
|
+
- `commands PROJECT_ID FILE` approves argv lists for setup and verification,
|
|
479
|
+
for example `{"setup":[["pnpm","install","--frozen-lockfile"]],"verify":[["pnpm","verify"]]}`.
|
|
480
|
+
They run in the same boundary, with network access, a home of their own and
|
|
481
|
+
no provider or deployment credentials. A job from the browser cannot change
|
|
482
|
+
them.
|
|
483
|
+
- `run [--once]` takes jobs from the app and runs them. `status` prints the
|
|
484
|
+
paired device, mapped projects, the active job and pending events, never the
|
|
485
|
+
device key.
|
|
486
|
+
|
|
487
|
+
How a job runs: each job starts from the exact commit the app names, fetched
|
|
488
|
+
into a separate detached checkout with its own Git metadata. Fetch that commit
|
|
489
|
+
into your checkout first. Your branch, index, dirty and untracked files stay
|
|
490
|
+
untouched. Git runs without global or system configuration, hooks or
|
|
491
|
+
templates. The provider (Codex `app-server`, or `claude`) can write only that
|
|
492
|
+
checkout and the dedicated home, and read only them, its own installation and
|
|
493
|
+
the system runtime; other files, including through symlinks, are denied. Its
|
|
494
|
+
environment has no GitHub, Cloudflare or MCP credentials. A request to widen
|
|
495
|
+
that boundary can only be declined from the app; it is never approved
|
|
496
|
+
remotely. Known provider credentials are redacted from relayed progress and
|
|
497
|
+
block a result that contains them. Events wait on disk and are retried with the
|
|
498
|
+
same ids until the app accepts them. A job that was running when the companion
|
|
499
|
+
stopped is never restarted on its own: resume or cancel it from the app. The
|
|
500
|
+
provider's own network access is not fenced.
|
|
501
|
+
|
|
502
|
+
State lives in `~/.rer-local` (`RER_LOCAL_HOME` picks another private folder).
|
|
503
|
+
The device key is stored there in a 0600 file; the app keeps only its hash.
|
|
504
|
+
|
|
505
|
+
**Revoke a device** in the app: Settings › Devices › Revoke. A running
|
|
506
|
+
companion stops at its next request to the app. Then delete the state folder
|
|
507
|
+
on that computer.
|
|
508
|
+
|
|
509
|
+
If setup failed before any provider turn and you have confirmed no provider or
|
|
510
|
+
setup process is still running, `confirm-no-turn-started JOB_ID
|
|
511
|
+
--confirm-no-provider-turn-started` lets the app resume it. It refuses if the
|
|
512
|
+
checkout changed or the provider recorded a session for it. It reads the
|
|
513
|
+
checkout with Git inside the same boundary, because setup could have changed
|
|
514
|
+
the checkout's Git configuration.
|
|
@@ -8,6 +8,8 @@ import { existsSync } from 'node:fs';
|
|
|
8
8
|
import { join } from 'node:path';
|
|
9
9
|
import { pathToFileURL } from 'node:url';
|
|
10
10
|
import { liveServers, stopCommand } from '../registry.mjs';
|
|
11
|
+
import { RerError } from '../../cli/errors.mjs';
|
|
12
|
+
import { serverLimitFile } from '../../cli/server-limit-file.mjs';
|
|
11
13
|
import { LaunchFailure } from './failure.mjs';
|
|
12
14
|
|
|
13
15
|
export const personDefaults = { limit: 10, idleSeconds: 43200 };
|
|
@@ -44,7 +46,8 @@ export function personEnvironment(env, args, pin) {
|
|
|
44
46
|
const listing = record => ` ${record.mode} on port ${record.port ?? '(starting)'} in ${record.checkout} (PID ${record.owner?.pid})\n stop it with: ${stopCommand(record)}`;
|
|
45
47
|
|
|
46
48
|
// Refuse in seconds, before any install or setup, when the pinned dispatcher
|
|
47
|
-
// would refuse the start for the server limit.
|
|
49
|
+
// would refuse the start for the server limit. RER_SERVER_LIMIT_FILE, when set,
|
|
50
|
+
// is the cap alone, as in the dispatcher's registry. Returns this checkout's own
|
|
48
51
|
// running server when there is one.
|
|
49
52
|
export async function checkCapacity(checkout, env, pin) {
|
|
50
53
|
const running = await liveServers(env);
|
|
@@ -52,8 +55,9 @@ export async function checkCapacity(checkout, env, pin) {
|
|
|
52
55
|
if (own) return { own };
|
|
53
56
|
if (env.RER_SERVER_JOURNEY === '1') return {};
|
|
54
57
|
const counted = running.filter(record => !record.journey);
|
|
55
|
-
const
|
|
56
|
-
const
|
|
58
|
+
const file = await serverLimitFile(env).catch(error => { throw error instanceof RerError ? new LaunchFailure(error.exitCode, error.message) : error; });
|
|
59
|
+
const limit = file ? file.limit : Number(env.RER_SERVER_LIMIT ?? pin.limit);
|
|
60
|
+
const binding = file ? [] : pin.explicitOnly ? counted.filter(record => record.explicit) : counted;
|
|
57
61
|
const cap = Math.min(limit, ...binding.map(record => record.limit));
|
|
58
62
|
if (counted.length < cap) return {};
|
|
59
63
|
const holders = cap < limit ? binding.filter(record => record.limit === cap) : [];
|
|
@@ -67,6 +71,7 @@ export async function checkCapacity(checkout, env, pin) {
|
|
|
67
71
|
throw new LaunchFailure(4, [
|
|
68
72
|
`Not started: ${counted.length} dev or preview ${counted.length === 1 ? 'server is' : 'servers are'} running on this computer and the limit is ${cap}.`,
|
|
69
73
|
'Running now:', ...counted.map(listing),
|
|
70
|
-
`To make room, stop one with its command above, or
|
|
74
|
+
file ? `The limit is ${cap} because ${file.path} (RER_SERVER_LIMIT_FILE) says ${cap}. To make room, stop one with its command above, or raise the number in that file.`
|
|
75
|
+
: `To make room, stop one with its command above, or start this one with a higher limit, for example RER_SERVER_LIMIT=${cap * 2} rer dev.`,
|
|
71
76
|
].join('\n'));
|
|
72
77
|
}
|
package/global/launch/facts.mjs
CHANGED
|
@@ -3,8 +3,7 @@ import { execFile } from 'node:child_process';
|
|
|
3
3
|
import { stat } from 'node:fs/promises';
|
|
4
4
|
import { join } from 'node:path';
|
|
5
5
|
import { promisify } from 'node:util';
|
|
6
|
-
import {
|
|
7
|
-
import { probePort } from '../../onboarding/port-probe.mjs';
|
|
6
|
+
import { probePort, searchPort } from '../../onboarding/port-probe.mjs';
|
|
8
7
|
import { LaunchFailure } from './failure.mjs';
|
|
9
8
|
|
|
10
9
|
const run = promisify(execFile);
|
|
@@ -69,24 +68,17 @@ export async function portTaken(port) {
|
|
|
69
68
|
return Boolean((await probePort('127.0.0.1', port, { hosts: ['127.0.0.1', '::1'], anyAddress: true })).listener);
|
|
70
69
|
}
|
|
71
70
|
|
|
72
|
-
// The first port from 8787 (RER_DEV_PORT_FROM moves the start)
|
|
73
|
-
// `held` (by a registered server, or saved
|
|
74
|
-
// nothing listens on, binds on both
|
|
75
|
-
// ::1
|
|
76
|
-
// another launch choosing it at the
|
|
71
|
+
// The first port from 8787 (RER_DEV_PORT_FROM moves the start), among the next
|
|
72
|
+
// `searchCandidates` ports, that is not `held` (by a registered server, or saved
|
|
73
|
+
// by another checkout that is stopped), nothing listens on, binds on both
|
|
74
|
+
// 127.0.0.1 and ::1 (a browser that tries ::1 first never reaches another
|
|
75
|
+
// server) and that `claim` secures against another launch choosing it at the
|
|
76
|
+
// same time. The search is bounded by that count, never by time or load.
|
|
77
77
|
export async function freePort(ports, claim, env = process.env) {
|
|
78
78
|
const from = Number(env.RER_DEV_PORT_FROM ?? 8787);
|
|
79
79
|
if (!Number.isInteger(from) || from < 1 || from > 65535) throw new LaunchFailure(2, 'RER_DEV_PORT_FROM must be a port number.');
|
|
80
|
-
const held = new Set(ports.filter(Boolean))
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
if (!result.available) {
|
|
85
|
-
if (performance.now() >= context.deadline) throw new LaunchFailure(4, `Could not find a free port from ${from} in time (system busy). Run rer dev again or pass rer dev --port PORT with a free port.`);
|
|
86
|
-
continue;
|
|
87
|
-
}
|
|
88
|
-
const claimed = await claim(port);
|
|
89
|
-
if (claimed) return { port, ...claimed };
|
|
90
|
-
}
|
|
91
|
-
throw new LaunchFailure(4, `No free port from ${from} upward. Pass one with rer dev --port PORT.`);
|
|
80
|
+
const held = new Set(ports.filter(Boolean));
|
|
81
|
+
const found = await searchPort({ host: '127.0.0.1', from, hosts: ['127.0.0.1', '::1'], skip: port => held.has(port), claim, verb: 'dev' });
|
|
82
|
+
if (found.port !== undefined) return found;
|
|
83
|
+
throw new LaunchFailure(4, `No free port from ${found.range[0]} to ${found.range[1]}: all ${found.tried} ports tried were in use. Stop something that holds one, or pass rer dev --port PORT with a free port.`);
|
|
92
84
|
}
|
package/global/main.mjs
CHANGED
|
@@ -29,6 +29,8 @@ pins (installed first by any verb when node_modules holds another), else its ven
|
|
|
29
29
|
launch them, whether they are set up, and their running URL. --json for data.
|
|
30
30
|
rer business ... Pair a Business device, store local vendor keys, inspect tasks and disconnect.
|
|
31
31
|
rer business --help lists the commands and current protocol limitations.
|
|
32
|
+
rer companion ... Pair this computer with a Run Edit Run app and run its change requests here
|
|
33
|
+
with your own Codex or Claude. rer companion --help lists the commands.
|
|
32
34
|
rer door ... The opt-in front door: every checkout at http://NAME.localhost:PORT, started
|
|
33
35
|
on its first visit and stopped when idle. rer door --help for its verbs.
|
|
34
36
|
rer cli status This rer: version, contents digest, install channel, launcher and other rer
|
|
@@ -63,6 +65,7 @@ export async function main(argv) {
|
|
|
63
65
|
delete process.env[localDefaults];
|
|
64
66
|
if (verb === 'update') return (await import('./update/command.mjs')).update(argv.slice(1));
|
|
65
67
|
if (verb === 'business') return (await import('../cli/business/command.mjs')).businessCommand(argv.slice(1));
|
|
68
|
+
if (verb === 'companion') return (await import('../cli/companion/command.mjs')).companionCommand(argv.slice(1));
|
|
66
69
|
const notice = await import('./update/notice.mjs');
|
|
67
70
|
if (verb !== 'doctor' && notice.wantsNotice(argv)) await notice.updateNotice();
|
|
68
71
|
if (!verb || verb === '--help' || verb === '-h' || verb === 'help') { process.stdout.write(help); return 0; }
|
|
@@ -12,6 +12,9 @@ host. See [adoption and deployment](../../export/DEMO.md#hosting-the-demo).
|
|
|
12
12
|
outbound requests return to the same visitor, including declared secondary hosts.
|
|
13
13
|
- `worker.mjs`, `pages.mjs`, `start-script.mjs`: Turnstile start, progress,
|
|
14
14
|
cancellation and visitor routing. Static assets are served at the edge.
|
|
15
|
+
- `native/warm.mjs`: the start page's prefetch list, built from `dist/client`
|
|
16
|
+
and TanStack Start's route manifest, the console's views first, capped by
|
|
17
|
+
compressed size.
|
|
15
18
|
- `capacity.mjs`: concurrent visitor admission and operator diagnostics. This
|
|
16
19
|
is used by the native host as well as retained during legacy upgrades.
|
|
17
20
|
- `upgrade.mjs`: verifies ownership, removes the old Container application and
|
|
@@ -40,7 +40,9 @@ export async function buildDemo(root, output, declaration) {
|
|
|
40
40
|
await rm(output, { recursive: true, force: true });
|
|
41
41
|
await mkdir(join(output, 'app'), { recursive: true });
|
|
42
42
|
await cp(join(root, 'dist'), join(output, 'app/dist'), { recursive: true });
|
|
43
|
-
|
|
43
|
+
// Where the console is mounted orders what the start page prefetches (native/warm.mjs).
|
|
44
|
+
const { operatorPath } = JSON.parse(await readFile(join(root, 'seed.json'), 'utf8').catch(() => '{}')).deploy ?? {};
|
|
45
|
+
await writeFile(join(output, 'app/demo.json'), JSON.stringify({ config: workerConfig, migrations, operatorPath }) + '\n');
|
|
44
46
|
}
|
|
45
47
|
|
|
46
48
|
if (process.argv[1] === import.meta.filename) {
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
import { parseCron } from './cron.mjs';
|
|
2
2
|
|
|
3
3
|
export const RUNTIME_FILES = ['config', 'worker', 'visitor', 'loader', 'd1', 'sql', 'r2', 'r2-store', 'queues', 'resources', 'cron', 'bridge', 'alarms'];
|
|
4
|
+
// Hashed build output: served immutable, and what the start page may prefetch.
|
|
5
|
+
export const HASHED_ASSET = /^\/assets\/.*-[\w-]{8,}\.[\w]+$/;
|
|
4
6
|
export const ISOLATE_FILES = ['bindings', 'namespace', 'r2-client', 'd1', 'sql', 'seed-object'];
|
|
5
7
|
export function nativeBindings(config) {
|
|
6
8
|
for (const kind of ['kv_namespaces', 'services', 'workflows', 'ai', 'vectorize', 'hyperdrive', 'analytics_engine_datasets', 'browser', 'images', 'send_email', 'dispatch_namespaces', 'mtls_certificates', 'pipelines', 'secrets_store_secrets', 'unsafe', 'vpc_services', 'vpc_networks', 'ratelimits', 'worker_loaders', 'containers', 'ai_search_namespaces', 'ai_search', 'agent_memory', 'connect', 'artifacts', 'unsafe_hello_world', 'flagship', 'logfwdr', 'version_metadata']) {
|
|
@@ -4,6 +4,7 @@ import { createHash } from 'node:crypto';
|
|
|
4
4
|
import { collectBuildModules } from '../../deploy-kit/build-modules.mjs';
|
|
5
5
|
import { collectFiles, moduleType } from '../../deploy-kit/artifact.mjs';
|
|
6
6
|
import { nativeBindings, ISOLATE_FILES, objectExports } from './config.mjs';
|
|
7
|
+
import { warmList } from './warm.mjs';
|
|
7
8
|
|
|
8
9
|
// The Worker Loader module kind for each upload type deploy-kit gives a build
|
|
9
10
|
// module, so a demo carries what a normal deploy uploads. Binary kinds travel
|
|
@@ -32,7 +33,8 @@ export async function prepareNative(folder) {
|
|
|
32
33
|
const assets = await collectFiles(join(app, 'dist/client'));
|
|
33
34
|
const bundle = { compatibilityDate: config.compatibility_date, compatibilityFlags: [...new Set([...(config.compatibility_flags ?? []), 'nodejs_compat'])], modules,
|
|
34
35
|
bindings, migrations: manifest.migrations, vars: config.vars ?? {}, crons: config.triggers?.crons ?? [],
|
|
35
|
-
assetPaths: assets.filter(file => !file.path.split('/').some(part => part.startsWith('.'))).map(file => '/' + file.path)
|
|
36
|
+
assetPaths: assets.filter(file => !file.path.split('/').some(part => part.startsWith('.'))).map(file => '/' + file.path),
|
|
37
|
+
warm: (await warmList(assets, root, manifest.operatorPath)).paths };
|
|
36
38
|
bundle.digest = createHash('sha256').update(JSON.stringify(bundle)).digest('hex');
|
|
37
39
|
await cp(join(app, 'dist/client'), join(folder, 'assets'), { recursive: true });
|
|
38
40
|
await writeFile(join(folder, 'modules/demo-host/native/bundle.mjs'), `export default ${JSON.stringify(bundle)};\n`);
|
|
@@ -56,8 +56,10 @@ export class NativeDemo extends DurableObject {
|
|
|
56
56
|
const seeded = await app.fetch(new Request(origin + '/health', { headers: { 'cf-connecting-ip': clientIp, cookie: cookies.map(cookie => cookie.split(';')[0]).join('; ') } }));
|
|
57
57
|
await seeded.arrayBuffer();
|
|
58
58
|
if (!seeded.ok) throw new Error(`demo-seed-${seeded.status}`);
|
|
59
|
-
if (this.state?.phase !== 'starting') throw new Error('demo-ended-during-start');
|
|
60
59
|
if (!await this.capacity().extend(id, ends)) throw new Error('start-reservation-expired');
|
|
60
|
+
// Checked after the extend, right before ready: an end that ran meanwhile
|
|
61
|
+
// released this place, and the copy must not outlive it.
|
|
62
|
+
if (this.state?.phase !== 'starting') throw new Error('demo-ended-during-start');
|
|
61
63
|
this.state = { ...this.state, phase: 'ready', cookies, location: claimed.headers.get('location'), timing: { migrationsMs: migrated - at, claimAndSeedMs: Date.now() - migrated, totalMs: Date.now() - started } };
|
|
62
64
|
await this.save();
|
|
63
65
|
await this.schedule(await nextQueue(this));
|
|
@@ -125,8 +127,9 @@ export class NativeDemo extends DurableObject {
|
|
|
125
127
|
if (!current || current > at) await this.ctx.storage.setAlarm(at);
|
|
126
128
|
}
|
|
127
129
|
async alarm() {
|
|
128
|
-
|
|
129
|
-
|
|
130
|
+
// An ending copy's alarm finishes its erasure; a failed attempt keeps the retry alarm.
|
|
131
|
+
if (this.state?.phase === 'ending') { await this.end(); return this.erasing; }
|
|
132
|
+
try { await this.active(); } catch { return this.erasing; }
|
|
130
133
|
try {
|
|
131
134
|
if (this.state.phase === 'ready') {
|
|
132
135
|
const now = Date.now(), minute = Math.floor(now / 60_000);
|
|
@@ -143,18 +146,28 @@ export class NativeDemo extends DurableObject {
|
|
|
143
146
|
}
|
|
144
147
|
} finally { await this.schedule(await nextQueue(this)); }
|
|
145
148
|
}
|
|
149
|
+
// Cancel, sign-out, expiry and a failed start all end here. The copy stops
|
|
150
|
+
// serving and frees its place first, so the next visitor is admitted at once
|
|
151
|
+
// however long erasing its storage takes (an upload still streaming into R2
|
|
152
|
+
// holds that). Concurrent ends share one release; a later one finds the copy
|
|
153
|
+
// already erasing. Erasure follows, and the retry alarm repeats it until done.
|
|
146
154
|
end() {
|
|
147
|
-
if (
|
|
155
|
+
if (this.erasing) return Promise.resolve();
|
|
156
|
+
this.ending ??= this.retire().finally(() => { this.ending = undefined; });
|
|
148
157
|
return this.ending;
|
|
149
158
|
}
|
|
150
|
-
async
|
|
159
|
+
async retire() {
|
|
151
160
|
if (this.state) { this.state.phase = 'ending'; await this.save(); }
|
|
152
|
-
// Keep a retry alarm until external objects have actually been erased.
|
|
153
161
|
await this.ctx.storage.setAlarm(Date.now() + 10_000);
|
|
162
|
+
await this.capacity().release(this.ctx.id.toString());
|
|
163
|
+
this.erasing = this.erase()
|
|
164
|
+
.catch(error => console.error(JSON.stringify({ event: 'native-demo-erase-failed', message: error.message })))
|
|
165
|
+
.finally(() => { this.erasing = undefined; });
|
|
166
|
+
}
|
|
167
|
+
async erase() {
|
|
154
168
|
await eraseResources(this);
|
|
155
169
|
await this.ctx.storage.deleteAll();
|
|
156
170
|
await this.ctx.storage.deleteAlarm();
|
|
157
171
|
this.state = undefined;
|
|
158
|
-
await this.capacity().release(this.ctx.id.toString());
|
|
159
172
|
}
|
|
160
173
|
}
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
import { readdir } from 'node:fs/promises';
|
|
2
|
+
import { join } from 'node:path';
|
|
3
|
+
import { pathToFileURL } from 'node:url';
|
|
4
|
+
import { gzipSync } from 'node:zlib';
|
|
5
|
+
import { HASHED_ASSET } from './config.mjs';
|
|
6
|
+
|
|
7
|
+
// The start page prefetches the console's hashed JS and CSS while the visitor
|
|
8
|
+
// waits for the human check and sign-in, so each view's first visit finds its
|
|
9
|
+
// chunks in the browser's cache, and the colo holds them for the next visitor.
|
|
10
|
+
// At most WARM_BYTES of gzip-compressed transfer, the console first: the
|
|
11
|
+
// root and console layout, the console's own views (what its nav links to),
|
|
12
|
+
// its deeper views, the other routes, then the remaining chunks.
|
|
13
|
+
export const WARM_BYTES = 1024 * 1024;
|
|
14
|
+
|
|
15
|
+
const segments = path => path.split('/').filter(Boolean);
|
|
16
|
+
const dynamic = path => segments(path).filter(part => part.startsWith('$')).length;
|
|
17
|
+
const within = (path, root) => path === root || path.startsWith(root.endsWith('/') ? root : root + '/');
|
|
18
|
+
|
|
19
|
+
// TanStack Start's route manifest from the server build: each route's CSS and
|
|
20
|
+
// the chunks it preloads.
|
|
21
|
+
async function startRoutes(server) {
|
|
22
|
+
const directory = join(server, 'assets');
|
|
23
|
+
const file = (await readdir(directory).catch(() => [])).find(name => /^_tanstack-start-manifest_v-[\w-]+\.js$/.test(name));
|
|
24
|
+
if (!file) return {};
|
|
25
|
+
const { tsrStartManifest } = await import(pathToFileURL(join(directory, file)));
|
|
26
|
+
return tsrStartManifest().routes ?? {};
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
function routeOrder(routes, consolePath) {
|
|
30
|
+
const ids = Object.keys(routes).filter(id => id !== '__root__');
|
|
31
|
+
const views = id => within(id, consolePath) && id !== consolePath;
|
|
32
|
+
const nav = id => views(id) && !dynamic(id) && segments(id).length === segments(consolePath).length + 1 && !id.endsWith('_');
|
|
33
|
+
const rank = id => [consolePath, consolePath + '/'].includes(id) ? 0 : nav(id) ? 1 : views(id) ? 2 : 3;
|
|
34
|
+
const sorted = ids.sort((a, b) => rank(a) - rank(b) || dynamic(a) - dynamic(b) || segments(a).length - segments(b).length || a.localeCompare(b));
|
|
35
|
+
return ['__root__', ...sorted].filter(id => routes[id]);
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/** The ordered, size-capped paths the start page prefetches, from dist/client's files and the server build. */
|
|
39
|
+
export async function warmList(files, server, consolePath = '/app') {
|
|
40
|
+
const sizes = new Map(files.map(file => ['/' + file.path, file]).filter(([path]) => HASHED_ASSET.test(path) && /\.(js|css)$/.test(path))
|
|
41
|
+
.map(([path, file]) => [path, gzipSync(Buffer.from(file.base64, 'base64')).length]));
|
|
42
|
+
const routes = await startRoutes(server);
|
|
43
|
+
const ordered = new Set();
|
|
44
|
+
for (const id of routeOrder(routes, consolePath)) for (const path of [...(routes[id].css ?? []), ...(routes[id].preloads ?? [])]) if (sizes.has(path)) ordered.add(path);
|
|
45
|
+
for (const path of [...sizes.keys()].sort((a, b) => sizes.get(a) - sizes.get(b))) ordered.add(path);
|
|
46
|
+
const warm = []; let bytes = 0;
|
|
47
|
+
for (const path of ordered) if (bytes + sizes.get(path) <= WARM_BYTES) { warm.push(path); bytes += sizes.get(path); }
|
|
48
|
+
return { paths: warm, bytes, of: sizes.size };
|
|
49
|
+
}
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import front from '../worker.mjs';
|
|
2
2
|
import { bundle } from './loader.mjs';
|
|
3
|
+
import { HASHED_ASSET } from './config.mjs';
|
|
3
4
|
export { HostBridge } from './bridge.mjs';
|
|
4
5
|
export { DemoAssets, DemoEgress } from './loader.mjs';
|
|
5
6
|
export { NativeDemo } from './visitor.mjs';
|
|
@@ -13,10 +14,11 @@ export default {
|
|
|
13
14
|
if (['GET', 'HEAD'].includes(request.method) && assets.has(url.pathname)) {
|
|
14
15
|
const response = await env.ASSETS.fetch(request);
|
|
15
16
|
const headers = new Headers(response.headers);
|
|
16
|
-
if (
|
|
17
|
+
if (HASHED_ASSET.test(url.pathname)) headers.set('cache-control', 'public, max-age=31536000, immutable');
|
|
17
18
|
headers.set('x-rer-demo-assets', 'edge');
|
|
18
19
|
return new Response(response.body, { status: response.status, headers });
|
|
19
20
|
}
|
|
20
|
-
|
|
21
|
+
// The start page prefetches the console's chunks while the visitor waits.
|
|
22
|
+
return front.fetch(request, { ...env, DEMO_WARM: bundle.warm }, ctx);
|
|
21
23
|
},
|
|
22
24
|
};
|
|
@@ -24,10 +24,15 @@ ol.steps{display:none;list-style:none;margin:22px auto 0;padding:0;width:max-con
|
|
|
24
24
|
@media(min-width:600px) and (max-width:1023px){main{width:calc(100% - 64px);max-width:666px;margin:100px auto 48px}.card{padding:36px}h1{font-size:38px;line-height:46px}.intro{font-size:18px;line-height:27px}}
|
|
25
25
|
@media(max-width:599px){main,.ended main{width:calc(100% - 32px);margin:32px auto}.ended .card{padding:26px 22px}.ended h1{font-size:29px;line-height:36px}.ended .intro{font-size:17px;line-height:25px;margin-bottom:28px}.card{padding:26px 22px;border-radius:16px}.brand{font-size:16px}h1{font-size:34px;line-height:41px;letter-spacing:-.8px}.intro{font-size:17px;line-height:25px}ul{font-size:16px;line-height:25px;margin:22px 0 26px}li{padding-left:22px;margin-bottom:9px}li:before{left:2px;top:9px}.button{font-size:18px;margin-top:22px}.note{font-size:13px;line-height:20px}.steps{font-size:15px}.back{margin-top:24px;font-size:14px}}`;
|
|
26
26
|
|
|
27
|
+
// The console's chunks, fetched at idle priority while the visitor waits for
|
|
28
|
+
// the check and sign-in (native/warm.mjs). The browser ignores a failed one,
|
|
29
|
+
// and one with prefetching turned off fetches none.
|
|
30
|
+
const warm = env => (env.DEMO_WARM ?? []).map(path => `<link rel="prefetch" href="${escape(path)}">`).join('');
|
|
31
|
+
|
|
27
32
|
function html(env, { page, title, body, status = 200, headers = {}, turnstile = false, refresh = false }) {
|
|
28
33
|
const nonce = crypto.randomUUID().replaceAll('-', ''), site = new URL(env.DEMO_SITE);
|
|
29
34
|
const policy = `default-src 'none'; style-src 'nonce-${nonce}'; img-src 'self'; form-action 'self'; base-uri 'none'; frame-ancestors 'none'${turnstile ? `; script-src 'nonce-${nonce}' https://challenges.cloudflare.com; frame-src https://challenges.cloudflare.com; connect-src 'self' https://challenges.cloudflare.com` : ''}`;
|
|
30
|
-
return new Response(`<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta name="robots" content="noindex"><title>${escape(title)}</title>${refresh ? '<noscript><meta http-equiv="refresh" content="2;url=/__demo/status"></noscript>' : ''}<style nonce="${nonce}">${STYLE}</style>${turnstile ? `<noscript><style nonce="${nonce}">#note{display:none}</style></noscript><script nonce="${nonce}">${START_SCRIPT}</script><script nonce="${nonce}" src="https://challenges.cloudflare.com/turnstile/v0/api.js" async defer></script
|
|
35
|
+
return new Response(`<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta name="robots" content="noindex"><title>${escape(title)}</title>${refresh ? '<noscript><meta http-equiv="refresh" content="2;url=/__demo/status"></noscript>' : ''}<style nonce="${nonce}">${STYLE}</style>${turnstile ? `<noscript><style nonce="${nonce}">#note{display:none}</style></noscript><script nonce="${nonce}">${START_SCRIPT}</script><script nonce="${nonce}" src="https://challenges.cloudflare.com/turnstile/v0/api.js" async defer></script>${warm(env)}` : ''}</head>
|
|
31
36
|
<body class="${page}"><main><section class="card" aria-labelledby="title"><div class="mark" aria-hidden="true"></div><p class="brand">${escape(env.DEMO_NAME)}</p>${body}</section><nav class="back" aria-label="Return"><a href="${escape(site.origin)}/">Back to ${escape(site.host)}</a></nav></main></body></html>`,
|
|
32
37
|
{ status, headers: { 'content-type': 'text/html; charset=utf-8', 'cache-control': 'no-store', 'content-security-policy': policy, 'referrer-policy': 'strict-origin', 'x-content-type-options': 'nosniff', ...headers } });
|
|
33
38
|
}
|
|
@@ -130,8 +130,9 @@ async function start(request, env, ctx) {
|
|
|
130
130
|
}
|
|
131
131
|
|
|
132
132
|
// The visitor explicitly abandons this copy before running a fresh human check.
|
|
133
|
-
// Wait
|
|
134
|
-
// so another click can finish it
|
|
133
|
+
// Wait until the copy has stopped and released its place (its storage is erased
|
|
134
|
+
// after); a failure must retain the cookie so another click can finish it
|
|
135
|
+
// instead of silently leaking a reservation.
|
|
135
136
|
async function cancel(request, env) {
|
|
136
137
|
if (request.headers.get('origin') !== env.DEMO_ORIGIN) return refusal(request, env, 403, 'Start the demo from this page.');
|
|
137
138
|
const id = demoId(request, env);
|
|
@@ -203,7 +203,7 @@ No Single Redirect (dynamic redirect ruleset) permission is needed. Base's refer
|
|
|
203
203
|
|
|
204
204
|
SQL files have ordered unique names and SHA-256 identities. `d1_migrations` holds controller history with ordered IDs, names, timestamps, SHA-256 hashes and states. Native archives already exclude this installation table. Standard Wrangler name-only inserts remain possible, but their missing hashes/states block subsequent Base deployment pending reviewed adoption. Applied entries must be an unchanged prefix of the desired set. A new migration gets an `applying` claim before its SQL and completion marker are submitted together through the [Cloudflare D1 SQL import API](https://developers.cloudflare.com/api/resources/d1/subresources/database/methods/import/), using the same whole-file protocol as Wrangler to preserve compound trigger bodies and atomic SQL/completion. Uploads use bounded HTTPS R2 destinations, without account credentials; polling is bounded and signed URLs are never diagnostic output. A failed import keeps Cloudflare's own codes and messages on the error (`providerRefusal`, which `rer deploy` prints, with credentials and URLs removed and each message cut to 300 characters) and names the migration it was running.
|
|
205
205
|
|
|
206
|
-
An import that did not complete leaves its `applying` claim. The next deploy of the same migration bytes runs it again only when it proves none of it applied (`migration-retry.mjs`): the import is atomic, so the migration's effects that would make a second run fail (a CREATE without IF NOT EXISTS, ADD COLUMN, RENAME) and that last to the end of the file must all be absent. Then no attempt committed, and one that commits late makes the re-run fail whole instead of applying twice. A migration with no such effect, or with any of them present, stays refused as `uncertain-migration-claim`; the message names the migration, what is present, and the exact step: undo what is present and deploy again, or confirm by hand and delete or complete its claim with `wrangler d1 execute`. The kit never re-runs a migration whose effects may be present, rolls back user data, or marks an unknown outcome successful. Any other unconfirmed or changed history is still `changed-or-uncertain-migration-history`. Readback checks every final identity.
|
|
206
|
+
An import that did not complete leaves its `applying` claim. The next deploy of the same migration bytes runs it again only when it proves none of it applied (`migration-retry.mjs`): the import is atomic, so the migration's effects that would make a second run fail (a CREATE without IF NOT EXISTS, ADD COLUMN, RENAME) and that last to the end of the file must all be absent. Then no attempt committed, and one that commits late makes the re-run fail whole instead of applying twice. A name the file drops or renames away existed before it ran, so a table it rebuilds under the same name, and that table's indexes, are not checked. A migration with no such effect, or with any of them present, stays refused as `uncertain-migration-claim`; the message names the migration, what is present, and the exact step: undo what is present and deploy again, or confirm by hand and delete or complete its claim with `wrangler d1 execute`. The kit never re-runs a migration whose effects may be present, rolls back user data, or marks an unknown outcome successful. Any other unconfirmed or changed history is still `changed-or-uncertain-migration-history`. Readback checks every final identity.
|
|
207
207
|
|
|
208
208
|
A seed that had to rewrite a migration it already released declares the rewrite
|
|
209
209
|
in `migrations/rewrites.json`: `{ "version": 1, "rewrites": [{ "migration":
|
|
@@ -216,6 +216,20 @@ migration in the set, and `to` is that migration's current bytes; every other
|
|
|
216
216
|
change to applied history is still refused. Keep the record for as long as any
|
|
217
217
|
installation may hold the released digest.
|
|
218
218
|
|
|
219
|
+
An `applying` claim on `from`, left when D1 rolled back an import of the released
|
|
220
|
+
bytes, is retried with the current bytes by `rer deploy`, which runs migrations
|
|
221
|
+
inside its `<state file>.lock` (One run at a time), so no other deploy of that
|
|
222
|
+
install is running. The same proof as above must hold for the current bytes, and the
|
|
223
|
+
completion marker moves the claim to the current digest only as that import
|
|
224
|
+
commits. The deploy prints `Migrations: retried <name>: the earlier attempt was
|
|
225
|
+
rolled back (D1 <database>).` A caller without that lock keeps the refusal. An
|
|
226
|
+
`applying` claim on any other bytes stays `changed-or-uncertain-migration-history`,
|
|
227
|
+
and the message gives the `wrangler d1 execute ... DELETE FROM d1_migrations WHERE
|
|
228
|
+
name=... AND sha256=... AND state='applying'` command with the database and
|
|
229
|
+
migration filled in. It is safe once that attempt has ended and no other deploy of
|
|
230
|
+
the environment is running: the claim and the migration commit together, so a claim
|
|
231
|
+
still `applying` means none of it applied.
|
|
232
|
+
|
|
219
233
|
Completion preserves the original SQL bytes and always starts on a new line.
|
|
220
234
|
Comment-only migrations are rejected before any provider request or ledger claim.
|
|
221
235
|
The controller adds a separator only when the final non-comment token is not a
|
|
@@ -15,8 +15,10 @@ import { verifyHealth } from './health.mjs';
|
|
|
15
15
|
* mode creates owned resources itself first, then passes false. `uploadWithoutReceipt`
|
|
16
16
|
* re-uploads an identical build when no receipt survived, instead of refusing the no-op.
|
|
17
17
|
* `declaredSecrets` (default `plan.secrets`) are the secrets the fingerprint names: owner mode
|
|
18
|
-
* passes the declaration's, so retiring the spent setup token leaves the build unchanged.
|
|
19
|
-
|
|
18
|
+
* passes the declaration's, so retiring the spent setup token leaves the build unchanged.
|
|
19
|
+
* `locked`: the caller holds the install's deploy lock (owner mode), so migrate may retry a
|
|
20
|
+
* claim on bytes a reviewed rewrite replaced. */
|
|
21
|
+
export async function deploy({api,plan,artifact,artifactSha256,previousReceipt,repeatOnly=false,createResources=!previousReceipt,uploadWithoutReceipt=false,onReceipt,onHealth,readiness={},env={},fetch=globalThis.fetch,healthOptions={},declaredSecrets=plan.secrets,locked=false}) {
|
|
20
22
|
validatePlan(plan); validateArtifact(artifact);
|
|
21
23
|
await verifyServiceGrants(api, plan);
|
|
22
24
|
for (const item of plan.d1) validateMigrations(item.migrations);
|
|
@@ -43,7 +45,7 @@ export async function deploy({api,plan,artifact,artifactSha256,previousReceipt,r
|
|
|
43
45
|
}
|
|
44
46
|
const migrations = [];
|
|
45
47
|
// A migration failure names its database, so the owner knows which one to inspect.
|
|
46
|
-
for (const item of plan.d1) migrations.push({binding:item.binding,...await migrate(api,resources.d1.find(x => x.binding === item.binding).id,item.migrations,{apply:!unchanged,upgrade:true}).catch(error => { error.resource ??= `d1:${item.name}`; throw error; })});
|
|
48
|
+
for (const item of plan.d1) migrations.push({binding:item.binding,...await migrate(api,resources.d1.find(x => x.binding === item.binding).id,item.migrations,{apply:!unchanged,upgrade:true,locked}).catch(error => { error.resource ??= `d1:${item.name}`; throw error; })});
|
|
47
49
|
const window = readiness.window ?? readinessWindow({...(healthOptions.now ? {clock:healthOptions.now} : {}), ...(healthOptions.sleep ? {sleep:healthOptions.sleep} : {}), ...readiness});
|
|
48
50
|
let upload;
|
|
49
51
|
if (!unchanged) { window.start(); upload = await uploadWorker(api,{plan,artifact,bindings,secrets,migration,fingerprint}); }
|
|
@@ -40,6 +40,14 @@ export function migrationFailure(error, again = 'rer deploy') {
|
|
|
40
40
|
? `Some of its changes are present (${present.join(', ')}). Undo them so the database is as it was before ${name}, then run ${again} again: rer checks again and runs it.`
|
|
41
41
|
: `It creates no table, index, view, trigger or column, so rer cannot check whether it applied. If none of its changes are present, delete its claim with ${run('DELETE FROM d1_migrations')}, then run ${again} again.`} If it did apply in full, record that instead with ${run("UPDATE d1_migrations SET state='applied'")}.`;
|
|
42
42
|
}
|
|
43
|
+
const unconfirmed = error?.unconfirmedClaim;
|
|
44
|
+
if (typeof unconfirmed?.name === 'string' && /^[a-f0-9]{64}$/.test(unconfirmed.sha256 ?? '')) {
|
|
45
|
+
const name = safe(unconfirmed.name).replaceAll(' ','?');
|
|
46
|
+
const run = `npx wrangler d1 execute ${database ?? '<database>'} --remote --command "DELETE FROM d1_migrations WHERE name='${name}' AND sha256='${unconfirmed.sha256}' AND state='applying'"`;
|
|
47
|
+
return `Migration ${name} in ${database ? `D1 ${database}` : 'its D1 database'} was started on bytes this build does not have (SHA-256 ${unconfirmed.sha256.slice(0,12)}) and never confirmed. ${unconfirmed.rewritten
|
|
48
|
+
? `A reviewed rewrite in migrations/rewrites.json replaces those bytes; rer deploy retries it under its deploy lock, and this run does not hold one.`
|
|
49
|
+
: `No reviewed rewrite in migrations/rewrites.json replaces those bytes, so rer will not run it again.`} D1 imports a migration and its completion marker as one atomic file, so a claim still applying after that attempt ended means none of it applied. Deleting the claim is safe once that attempt has ended (it failed, or its run was stopped) and no other deploy of this environment is running: ${run}, then run ${again} again. While a deploy may still be running, leave it.`;
|
|
50
|
+
}
|
|
43
51
|
if (typeof error?.migration === 'string') return `Migration ${safe(error.migration)} did not apply; the next ${again} runs it again once rer proves none of it applied.`;
|
|
44
52
|
return undefined;
|
|
45
53
|
}
|
|
@@ -44,10 +44,26 @@ export const recorded = (row, migration) => row.name === migration?.name && (row
|
|
|
44
44
|
const validPrefix = (rows, migrations) => rows.every((row, index) => recorded(row, migrations[index]) && row.state === 'applied');
|
|
45
45
|
const same = (a, b) => a.length === b.length && a.every((row, index) => row.name === b[index].name && row.sha256 === b[index].sha256 && row.state === b[index].state);
|
|
46
46
|
|
|
47
|
-
|
|
47
|
+
/** History this build does not match. When it ends in an unconfirmed claim after a
|
|
48
|
+
* valid prefix, the error names that claim (`unconfirmedClaim`) so the refusal can
|
|
49
|
+
* give the exact command that clears it; `rewritten` says a reviewed rewrite
|
|
50
|
+
* replaced the claimed bytes and only a deploy holding the install's lock retries it. */
|
|
51
|
+
function unmatched(rows, migrations) {
|
|
52
|
+
const error = new Error('changed-or-uncertain-migration-history'), last = rows.at(-1);
|
|
53
|
+
if (last?.state === 'applying' && /^[a-f0-9]{64}$/.test(last.sha256 ?? '') && validPrefix(rows.slice(0, -1), migrations)) {
|
|
54
|
+
const next = migrations[rows.length - 1];
|
|
55
|
+
error.unconfirmedClaim = { name:last.name, sha256:last.sha256, rewritten:last.name === next?.name && (next.rewrites ?? []).some(rewrite => rewrite.from === last.sha256) };
|
|
56
|
+
}
|
|
57
|
+
return error;
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
/** `rewritten`: the caller holds the install's deploy lock (owner.mjs withStateLock),
|
|
61
|
+
* so no other deploy of it is running; a claim on bytes a reviewed rewrite replaced
|
|
62
|
+
* is then this build's to retry. */
|
|
63
|
+
export async function readLedger(api, id, migrations, query, { rewritten = false } = {}) {
|
|
48
64
|
const read = async sql => (await query(api, id, sql))[0].results;
|
|
49
65
|
const tables = (await read(`SELECT name FROM sqlite_master WHERE type='table' AND name IN ('${legacy}','${ledger}')`)).map(row => row.name);
|
|
50
|
-
let old = [], applied = [], transition = '', claimed = false;
|
|
66
|
+
let old = [], applied = [], transition = '', claimed = false, claimedFrom;
|
|
51
67
|
if (tables.includes(legacy)) {
|
|
52
68
|
const columns = await read(`SELECT name FROM pragma_table_info('${legacy}')`);
|
|
53
69
|
assert(columns.length === 3 && columns.every(row => ['name','sha256','state'].includes(row.name)), 'invalid-migration-ledger');
|
|
@@ -59,11 +75,15 @@ export async function readLedger(api, id, migrations, query) {
|
|
|
59
75
|
assert(['id','name','applied_at'].every(name => columns.includes(name)), 'invalid-migration-ledger');
|
|
60
76
|
if (columns.includes('sha256') && columns.includes('state')) {
|
|
61
77
|
applied = await read(`SELECT name, sha256, state FROM ${ledger} ORDER BY id`);
|
|
62
|
-
// The last row may claim this build's next migration
|
|
63
|
-
//
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
78
|
+
// The last row may claim this build's next migration whose import did not
|
|
79
|
+
// complete: its exact bytes, or (`rewritten`) bytes a reviewed rewrite replaced.
|
|
80
|
+
// migrate re-runs it only on proof (migration-retry.mjs).
|
|
81
|
+
const last = applied.at(-1), next = migrations[applied.length - 1];
|
|
82
|
+
if (!tables.includes(legacy) && last?.state === 'applying' && last.name === next?.name) {
|
|
83
|
+
if (last.sha256 === next.sha256) claimed = Boolean(applied.pop());
|
|
84
|
+
else if (rewritten && (next.rewrites ?? []).some(rewrite => rewrite.from === last.sha256)) { claimedFrom = last.sha256; claimed = Boolean(applied.pop()); }
|
|
85
|
+
}
|
|
86
|
+
if (!validPrefix(applied, migrations)) throw unmatched(applied, migrations);
|
|
67
87
|
if (tables.includes(legacy)) assert(same(applied, old), 'changed-or-uncertain-migration-history');
|
|
68
88
|
} else {
|
|
69
89
|
assert(!columns.includes('sha256') && !columns.includes('state'), 'invalid-migration-ledger');
|
|
@@ -82,7 +102,7 @@ export async function readLedger(api, id, migrations, query) {
|
|
|
82
102
|
applied = old;
|
|
83
103
|
}
|
|
84
104
|
if (tables.includes(legacy)) transition += `DROP TABLE ${legacy};`;
|
|
85
|
-
return { applied, transition, claimed, exists:tables.includes(ledger) || tables.includes(legacy) };
|
|
105
|
+
return { applied, transition, claimed, claimedFrom, exists:tables.includes(ledger) || tables.includes(legacy) };
|
|
86
106
|
}
|
|
87
107
|
function insert(row) {
|
|
88
108
|
return `INSERT INTO ${ledger}(name,sha256,state) VALUES (${literal(row.name)},${literal(row.sha256)},'applied');\n`;
|