@runeditrun/rer 0.2.117 → 0.2.119

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/README.md +0 -6
  2. package/cli/README.md +11 -0
  3. package/cli/business/command.mjs +5 -8
  4. package/cli/business/connect.mjs +6 -104
  5. package/cli/business/local-commands.mjs +2 -2
  6. package/cli/business/state.mjs +3 -3
  7. package/cli/contract.mjs +2 -2
  8. package/cli/deploy-command.mjs +13 -7
  9. package/cli/deploy-inputs.mjs +1 -4
  10. package/cli/deploy-settings.mjs +15 -9
  11. package/cli/deploy.mjs +1 -1
  12. package/cli/inspect.mjs +8 -3
  13. package/cli/managed-data-schema.mjs +2 -0
  14. package/cli/parse.mjs +1 -1
  15. package/cli/portability-worker.mjs +3 -1
  16. package/cli/portability.mjs +1 -1
  17. package/global/CHANGELOG.md +41 -0
  18. package/global/README.md +0 -6
  19. package/modules/deploy-kit/owner-cli.mjs +6 -3
  20. package/modules/deploy-kit/owner.mjs +21 -3
  21. package/modules/portability/README.md +8 -0
  22. package/modules/portability/archive-step.mjs +11 -0
  23. package/modules/portability/export.mjs +34 -25
  24. package/modules/portability/import-safety.mjs +4 -3
  25. package/modules/portability/import-staging.mjs +4 -4
  26. package/modules/portability/import.mjs +14 -5
  27. package/modules/portability/index.mjs +2 -1
  28. package/modules/portability/object-inventory.mjs +45 -0
  29. package/modules/portability/storage.mjs +4 -3
  30. package/operations/README.md +14 -1
  31. package/operations/operations.mjs +2 -0
  32. package/operations/project-operations.mjs +8 -1
  33. package/package.json +1 -1
  34. package/scripts/publish-check/README.md +5 -1
  35. package/scripts/publish-check/checksum_maps.py +9 -5
  36. package/scripts/publish-check/hit_location.py +20 -9
  37. package/scripts/publish-check/json_identity.py +15 -14
package/README.md CHANGED
@@ -531,12 +531,6 @@ different origin or Business: disconnect first or use a separate
531
531
  `RER_BUSINESS_STATE_DIR`. It reports this computer's plain name, working folder,
532
532
  agent and OS (including WSL) for the owner to recognise.
533
533
 
534
- For older Business pages, `rer business connect --code ABCD-1234 --origin URL`
535
- keeps the browser-created code and six-digit confirmation-number flow unchanged
536
- in 0.2.98. It cannot be combined with `--business` or `--timeout`. Removal is
537
- planned for a later release after the production rollback target uses device pairing.
538
- The install.sh form still passes either connect command through unchanged.
539
-
540
534
  The owner chooses three things on the computer's row in the browser: "Add rules
541
535
  so Claude Code never reads your secret files" (Claude Code user Read deny
542
536
  rules), "Keep this computer awake while it works" (recorded; nothing keeps the computer awake now that rer runs no exports) and "Install the background service" (a launchd or systemd user
package/cli/README.md CHANGED
@@ -292,6 +292,17 @@ anything is built or uploaded. The seed owns what an installation does with mail
292
292
  off, and turning mail on later is a `rer deploy --set-secret` of the provider
293
293
  inputs the seed documents.
294
294
 
295
+ The adapter must apply that selection to the runtime as well as pass preflight.
296
+ For example, the foundation declares optional `MAIL_PROVIDER` and `MAIL_FROM`
297
+ environment-to-binding mappings in `deploy.json`, and makes `MAIL_API_KEY`
298
+ optional there; production preflight still requires all enabled-provider inputs.
299
+ Its mail facade returns no sender for `none`, so email requests report
300
+ `mail_unavailable` rather than claiming delivery. Independently authored adapters
301
+ must implement their own matching contract. Declarative operations loops can
302
+ offer `none` for optional mail; required delivery loops continue to fail while off.
303
+
304
+ — Codex, 2026-11-15T03:31:15Z
305
+
295
306
  The production preflight also reads the product's `artifact.specInputs` in
296
307
  `rer-packaging.json`, `packaging.json` or `scripts/delivery/base/packaging.json`,
297
308
  and `management/integration/deployment-spec.json`. Public inputs come from
@@ -8,7 +8,8 @@ import { describe } from './errors.mjs';
8
8
  import { identifier, originUrl } from './api.mjs';
9
9
  import { appAddress, appSignInHelp } from '../app-address.mjs';
10
10
 
11
- const usage = `Set up your business: rer business connect
11
+ const usage = `Set up your business: rer business connect --business ID
12
+ The Business Connect page names the ID.
12
13
  Business keeps your business name, logo and brand colour together for the apps
13
14
  that support shared Business settings. Choose what to share in your browser.
14
15
 
@@ -19,7 +20,6 @@ Advanced commands (Usage: rer business VERB)
19
20
  scan [DIR...] [--json] Names-only local discovery
20
21
  connect --business ID [--origin URL] [--timeout SECONDS]
21
22
  Print a code and link for the owner to approve in a signed-in browser
22
- Older Business pages: connect --code CODE [--origin URL]
23
23
  Connect carries the Business name into local apps
24
24
  run [--once] Work this computer's tasks: the folder scan the owner allows. Says what
25
25
  it is doing and what it waits for; applies the owner's choices at each
@@ -72,13 +72,12 @@ the code's 10 minutes), then keeps the credential in OS credential storage.
72
72
  --origin URL Business platform address (default: ${appAddress}).
73
73
  --timeout SECONDS Wait at most this long, then exit 75 with the pairing kept;
74
74
  run the same command again to keep waiting (same code).
75
- --code CODE Older Business pages: redeem a code made in the browser.
76
75
  Exits 0 connected, 1 ended (refused, expired or failed), 2 usage, 75 still waiting.
77
76
  `,
78
77
  token: `Usage: rer business token
79
78
  Print an MCP Authorization header object for this computer's paired Business.
80
79
  Send this headersHelper output directly to the MCP client; it contains a credential.
81
- Requires an existing pairing: rer business connect.
80
+ Requires an existing pairing: rer business connect --business ID (the Connect page names the ID).
82
81
  `,
83
82
  };
84
83
 
@@ -93,11 +92,9 @@ export async function businessCommand(argv) {
93
92
  const options = {};
94
93
  for (let index = 0; index < args.length; index += 2) {
95
94
  const name = args[index];
96
- if (!['--code', '--origin', '--business', '--timeout'].includes(name) || !args[index + 1] || args[index + 1].startsWith('--') || Object.hasOwn(options, name.slice(2))) return invalid();
95
+ if (!['--origin', '--business', '--timeout'].includes(name) || !args[index + 1] || args[index + 1].startsWith('--') || Object.hasOwn(options, name.slice(2))) return invalid();
97
96
  options[name.slice(2)] = args[index + 1];
98
97
  }
99
- if (options.code !== undefined && !/^[A-Za-z0-9]{4}-?[A-Za-z0-9]{4}$/.test(options.code)) return invalid();
100
- if (options.code !== undefined && (options.business !== undefined || options.timeout !== undefined)) return invalid();
101
98
  if (options.timeout !== undefined) {
102
99
  if (!/^\d{1,3}$/.test(options.timeout) || Number(options.timeout) > 900) return invalid();
103
100
  options.timeout = Number(options.timeout);
@@ -106,7 +103,7 @@ export async function businessCommand(argv) {
106
103
  if (options.business !== undefined) identifier(options.business);
107
104
  if (options.origin !== undefined && options.business !== undefined) originUrl(options.origin);
108
105
  } catch { return invalid(); }
109
- if (options.origin !== undefined && options.code === undefined && options.business === undefined) originUrl(options.origin);
106
+ if (options.origin !== undefined && options.business === undefined) originUrl(options.origin);
110
107
  await connect(options);
111
108
  } else if (verb === 'done' && args.length === 3) await done(...args);
112
109
  else if (verb === 'answer' && [3, 4].includes(args.length)) await answer(...args);
@@ -17,113 +17,15 @@ const approvalWindow = 600_000;
17
17
  export const capabilities = ['discover.local'];
18
18
 
19
19
  const clock = time => new Date(time).toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' });
20
- const nothingSent = 'Nothing was sent, so the pairing code still works.';
21
-
22
- /** Pair (or resume a saved pairing) and wait in the foreground until the owner
23
- * approves or refuses in the browser, or the code expires. Exit 0 means the
24
- * computer is active: its credential is stored and the server has seen it.
25
- */
20
+ /** Pair (or resume a saved device pairing) and wait for the owner to approve. */
26
21
  export async function connect(options = {}) {
27
22
  if (options.business !== undefined) return connectDevice(options);
28
23
  const pending = await loadPending();
29
- // Approval can outlive the original code deadline. A legacy command must
30
- // not discard a device pairing that is still within its collection window.
31
- if (pending?.flow === 'device' && options.code !== undefined && Date.parse(pending.keep_until) > Date.now()) throw Error('Business is already waiting for a device pairing. Resume that command, or run business disconnect before using another code.');
32
- if (pending?.flow === 'device' && options.code === undefined) return connectDevice(options);
33
- return connectLegacy(options);
34
- }
35
-
36
- // Removal of the browser-created --code flow is planned for a later release,
37
- // once the production rollback target uses device pairing.
38
- async function connectLegacy({ code, origin } = {}) {
24
+ if (pending?.flow === 'device') return connectDevice(options);
39
25
  const stored = await loadState();
40
- let pending = await loadPending();
41
- origin = originUrl(origin ?? stored?.origin ?? pending?.origin);
42
- if (stored) {
43
- if (stored.revoked) throw Error(`This computer was ${revokedText(stored)}\nThen connect with a new code from ${stored.origin}/business.`);
44
- if (stored.origin !== origin) throw Error(`This computer is already connected to Business at ${stored.origin}. Run ${rerCommand()} business disconnect before connecting it elsewhere.`);
45
- const credential = await createCredentialStore().get(deviceCredentialHandle(stored));
46
- if (!credential) throw Error(`This computer's Business credential is missing from OS credential storage. Run ${rerCommand()} business disconnect, then connect again with a new code from ${origin}/business.`);
47
- await connected(new Api(origin, credential), stored, { reconnecting: true });
48
- return;
49
- }
50
- if (pending && Date.parse(pending.expires_at) <= Date.now()) { await clearPending(); pending = null; }
51
- if (pending && (pending.origin !== origin || code && pending.code !== code)) throw Error(`Business is already waiting for approval of another code${pending.origin === origin ? '' : ` at ${pending.origin}`}. Run ${rerCommand()} business connect to resume it, or ${rerCommand()} business disconnect to drop it before using another code.`);
52
- if (!pending && !code) throw Error(`Set up your business: open ${origin}/business and take the prompt from the Business Connect page. It names the Business to join with --business.`);
53
- // Refuse before redeeming the one-use code or collecting an approval when this
54
- // session cannot keep the pairing or store the device credential.
55
- await checkStateWritable();
56
- const store = createCredentialStore();
57
- try { await store.check(); }
58
- catch (error) {
59
- const outside = error.sandboxRefused ? ` Approve this command outside the sandbox: ${rerCommand()} business connect${code ? ` --code ${code}` : ''} --origin ${origin}. Every other business command runs inside the sandbox.` : '';
60
- throw Error(`${error.message}${outside} ${pending ? 'The saved pairing was kept; run connect again once this is fixed.' : nothingSent}`);
61
- }
62
-
63
- // Ctrl-C or a stopped tool call never strands a pairing: the redeemed pairing
64
- // is saved before waiting, and a collected credential is stored before exit.
65
- const stop = new AbortController();
66
- const stopWith = signal => () => stop.abort(signal);
67
- const onInterrupt = stopWith('SIGINT'), onTerminate = stopWith('SIGTERM');
68
- process.on('SIGINT', onInterrupt); process.on('SIGTERM', onTerminate);
69
- try {
70
- const api = new Api(origin);
71
- if (!pending) {
72
- // The computer's plain name and the folder connect runs in help the owner
73
- // recognise this computer and prefill the scan folder; each is left out when empty.
74
- const name = deviceName(), folder = workingFolder();
75
- const response = await api.post('/api/business/executors/pair', {
76
- code, ...(name ? { device_name: name } : {}), ...(folder ? { working_folder: folder } : {}),
77
- os: process.platform === 'darwin' ? 'macos' : 'linux', agent_kind: agentKind(), capabilities,
78
- }).catch(error => { throw error.status === 401 ? Error(`Business did not accept code ${code}: it has expired or was already used${error.reason ? ` (the server said: ${error.reason})` : ''}. Create a new code in ${origin}/business.`) : error; });
79
- const expires = Math.min(Date.now() + 600_000, Date.parse(response?.expires_at));
80
- if (response?.status !== 'pending_approval' || !/^\d{6}$/.test(response.confirmation_number) || typeof response.pairing_token !== 'string' || !response.pairing_token || !Number.isFinite(expires) || expires <= Date.now()) throw Error('Business server returned an invalid pairing response.');
81
- pending = { origin, code, pairing_token: response.pairing_token, confirmation_number: response.confirmation_number, code_expires_at: new Date(expires).toISOString(), expires_at: new Date(expires + approvalWindow).toISOString() };
82
- await savePending(pending);
83
- }
84
- const resume = `${rerCommand()} business connect --origin ${origin}`;
85
- const until = pending.code_expires_at ? ` until ${clock(pending.code_expires_at)}, when the code expires` : ' until the code expires';
86
- process.stdout.write(`Confirmation number: ${pending.confirmation_number}\nCompare this number in the browser that created the code, then approve there.\nWaiting for approval in ${origin}/business${until}. Keep this command running.\nIf it stops, run ${resume} to resume this pairing.\n`);
87
- while (Date.now() < Date.parse(pending.expires_at)) {
88
- if (stop.signal.aborted) break;
89
- let result;
90
- try { result = await api.post('/api/business/executors/pair', { pairing_token: pending.pairing_token }, { signal: stop.signal }); }
91
- catch (error) {
92
- if (stop.signal.aborted) break;
93
- if ([401, 403, 410].includes(error.status)) {
94
- await clearPending();
95
- throw Error(`Business pairing ended without approval: the owner refused it, or the code expired${error.reason ? ` (the server said: ${error.reason})` : ''}. Create a new code in ${origin}/business and run connect with it.`);
96
- }
97
- throw error;
98
- }
99
- if (result?.status === 'paired') {
100
- if (typeof result.device_credential !== 'string') throw Error('Business server returned an invalid device credential.');
101
- const folder = workingFolder();
102
- const state = { version: 1, credentialVersion: 2, origin, businessId: identifier(result.business_id), executorId: identifier(result.executor_id), vendorHandles: [], awakeConsent: false, ...(folder ? { workingFolder: folder } : {}) };
103
- const handle = credentialHandle(origin, state.businessId);
104
- // No authenticated request (including activation) before both durable writes.
105
- await store.set(handle, result.device_credential);
106
- try { await saveState(state); } catch (error) { await store.delete(handle); throw error; }
107
- await clearPending();
108
- process.stdout.write('Approved. Device credential stored in OS credential storage.\n');
109
- await connected(new Api(origin, result.device_credential), state);
110
- return;
111
- }
112
- if (['denied', 'refused', 'expired'].includes(result?.status)) {
113
- await clearPending();
114
- throw Error(`Business pairing was ${result.status === 'expired' ? 'not approved before the code expired' : 'refused in the browser'}. Create a new code in ${origin}/business and run connect with it.`);
115
- }
116
- if (result?.status !== 'authorization_pending') throw Error('Business server returned an invalid pairing status.');
117
- try { await pause(Math.max(1, Math.min(5, Number(result.interval) || 5)) * 1000, stop.signal); } catch { break; }
118
- }
119
- if (stop.signal.aborted) {
120
- throw new Stopped(`Stopped waiting for approval. The pairing is saved: run ${resume} to resume it${pending.code_expires_at ? ` before ${clock(pending.code_expires_at)}` : ''}.`, stop.signal.reason === 'SIGTERM' ? 143 : 130);
121
- }
122
- await clearPending();
123
- throw Error(`Business pairing code expired before it was approved. Create a new code in ${origin}/business and run connect with it.`);
124
- } finally {
125
- process.off('SIGINT', onInterrupt); process.off('SIGTERM', onTerminate);
126
- }
26
+ if (stored) return connectDevice({ ...options, business: stored.businessId });
27
+ const origin = originUrl(options.origin ?? pending?.origin);
28
+ throw Error(`Set up your business: open ${origin}/business and take the prompt from the Business Connect page. It names the ID for ${rerCommand()} business connect --business ID.`);
127
29
  }
128
30
 
129
31
  const shortCode = /^[2-9A-HJKMNP-Z]{4}-[2-9A-HJKMNP-Z]{4}$/;
@@ -153,7 +55,7 @@ async function connectDevice({ business, origin, timeout } = {}) {
153
55
  process.on('SIGINT', interrupt); process.on('SIGTERM', terminate);
154
56
  let pending, resume, timer, unlock;
155
57
  try {
156
- await checkStateWritable({ codeExists: false });
58
+ await checkStateWritable();
157
59
  unlock = await lockPairing(stop.signal);
158
60
  const stored = await loadState();
159
61
  pending = await loadPending();
@@ -13,10 +13,10 @@ import { supportsTaskKind } from './supervisor.mjs';
13
13
 
14
14
  export async function paired() {
15
15
  const state = await loadState();
16
- if (!state?.origin || !state.businessId || !state.executorId) throw Error(`This computer is not paired. Run ${rerCommand()} business connect --code CODE.`);
16
+ if (!state?.origin || !state.businessId || !state.executorId) throw Error(`This computer is not paired. Run ${rerCommand()} business connect --business ID; the Business Connect page names the ID.`);
17
17
  if (state.revoked) throw Error(`This computer was ${revokedText(state)}`);
18
18
  const credential = await createCredentialStore().get(deviceCredentialHandle(state));
19
- if (!credential) throw Error(`This computer's Business credential is missing from OS credential storage. Run ${rerCommand()} business disconnect, then connect again with a new code from ${state.origin}/business.`);
19
+ if (!credential) throw Error(`This computer's Business credential is missing from OS credential storage. Run ${rerCommand()} business disconnect, then run ${rerCommand()} business connect --business ID; the Connect page at ${state.origin}/business names the ID.`);
20
20
  return { state, credential };
21
21
  }
22
22
 
@@ -57,12 +57,12 @@ async function saveJson(name, value) {
57
57
  } finally { await unlink(temporary).catch(error => { if (error.code !== 'ENOENT') throw error; }); }
58
58
  }
59
59
 
60
- /** Before a one-use code is spent or an approval collected, prove the state
60
+ /** Before a pairing starts or an approval is collected, prove the state
61
61
  * folder takes the same private create, write and replace the pairing needs.
62
62
  * Folders made only for the check are removed again, so a refused connect
63
63
  * leaves nothing behind.
64
64
  */
65
- export async function checkStateWritable({ codeExists = true } = {}) {
65
+ export async function checkStateWritable() {
66
66
  const directory = stateDirectory(), probe = join(directory, `${randomUUID()}.tmp`), created = [];
67
67
  for (let path = directory; path !== dirname(path); path = dirname(path)) {
68
68
  try { await lstat(path); break; } catch (error) { if (error.code !== 'ENOENT') break; created.push(path); }
@@ -78,7 +78,7 @@ export async function checkStateWritable({ codeExists = true } = {}) {
78
78
  const fix = chosenStateDirectory()
79
79
  ? 'Allow writes to that folder in this session, or set RER_BUSINESS_STATE_DIR to a folder this session can write'
80
80
  : 'Allow writes to that folder, or set RER_BUSINESS_STATE_DIR to a folder this session can write (in Codex, one inside the working folder, such as RER_BUSINESS_STATE_DIR="$PWD/.rer-business")';
81
- throw Error(`Business cannot save its state in ${directory}: ${localCause(error) ?? error.message}. ${codeExists ? 'Nothing was sent, so the pairing code still works.' : 'Nothing was sent.'} ${fix}, then run connect again.`);
81
+ throw Error(`Business cannot save its state in ${directory}: ${localCause(error) ?? error.message}. Nothing was sent. ${fix}, then run connect again.`);
82
82
  } finally {
83
83
  for (const path of created) await rmdir(path).catch(() => {});
84
84
  }
package/cli/contract.mjs CHANGED
@@ -12,8 +12,8 @@ const definitions = {
12
12
  verify: [[], 'Run the complete internal verification target.'],
13
13
  reset: [['--yes'], 'Confirmed removal of the declared local inventory only.'],
14
14
  deploy: [['--owner-name', '--environment', '--dry-run', '--site', '--demo', '--host', '--env-file', '--image', '--bind-address', '--http-port', '--https-port', '--remove', '--yes', '--set-secret', '--unset-secret', '--rotate-secret', '--reopen-installation-key'], 'Owner-account deployment and health; dry run builds and validates the local deployment plan, printing target names, bindings and migration names. No Cloudflare credentials or requests are needed and nothing changes remotely. Valid fixture senders on .test, .example, .invalid, .localhost and example.com, example.net or example.org (including subdomains) are accepted only in a dry run; real deploys refuse placeholder senders. Dry run never prompts for or requires a missing owner name, saves no name and bypasses custom deploy hooks and their owner-email preflight. Target secrets, permissions and remote readiness are checked only by a real deploy. Builds and deploy hooks run with the checkout\'s local settings files moved aside, and a build output holding one (.dev.vars, .env) is refused. Declared secrets come from the environment or --env-file (an owned 0600 KEY=VALUE file); a new Worker gets the local setup owner and fresh generated secrets when they are not given. rer keeps the secrets it generates in .rer/deploy/ENVIRONMENT.secrets.env (0600) and reuses them on every redeploy; it refuses to generate a replacement for one a live Worker holds, and --rotate-secret NAME[,NAME] replaces them on purpose. After a healthy deploy rer claims the setup owner on the hosted app with the saved setup owner name and the one-time setup token, then removes the token. For a real deploy, the managed adapter checks its hosted owner email before rer resolves the owner name, so its placeholder-email refusal comes first. If the installer declares an owner-name field and no name is saved, deploy asks once and saves it; --non-interactive requires --owner-name NAME before any upload. --set-secret NAME[,NAME] (each value from the environment or --env-file; repeat the flag or join names with commas) and --unset-secret NAME[,NAME] change those secrets of the deployed Worker without a build, all of them in one new Worker version or none, and record that version; a missing value refuses before any change. --reopen-installation-key KID temporarily binds a retired installation key from private custody, reseals restored dependency keys under the current key, verifies them and removes the previous binding. --remove --yes deletes what this environment\'s deploy record says rer created (Worker, D1, R2, KV, queues), reads back that each is gone, and touches nothing else. --site deploys the run site instead: an assets-only Worker on its custom domain with www redirected; with --dry-run it builds the site and prints the plan with no provider request. --demo deploys the hosted demo instead (deploy-demo.json): a front Worker on demo.<domain> giving each visitor their own signed-in, throwaway native Worker isolate of this seed with real public reads/checks and simulated external writes; --dry-run writes it to .rer/demo-host with no provider request; --demo --remove --yes deletes that demo (its Worker, demo.<domain> Custom Domain, each only while the record its deploy left proves it the demo\'s), reads back that each is gone, and is a no-op once it is.'],
15
- export: [['--target', '--output', '--origin', '--session-file'], 'For owner move-in, run rer dev and open /app/import: export people there, then upload through the destination door. Developer-only installation backup: complete supported archive of the local copy on this computer, with integrity and exclusions, written to --output. The hosted install cannot be exported yet: --origin or --target remote refuses (exit 2) before any effect and names the command that saves the hosted database.'],
16
- import: [['--target', '--input', '--journal', '--origin', '--session-file', '--yes', '--legacy-v1', '--acknowledge-unverified-metadata', '--acknowledge-unverified-schema-objects', '--output'], 'For owner move-in, run rer dev and open /app/import: upload, review and apply through the door. Developer-only installation recovery: confirmed atomic restore into an empty destination; --journal LATER_ARCHIVE also fences rows that installation erased since. Backup-only local v1 recovery requires --legacy-v1 --acknowledge-unverified-metadata and a new --output v2 archive; --journal requires normal v2 import. A v2 archive without schemaObjects requires --acknowledge-unverified-schema-objects, an empty destination and compatible destination migration history; its receipt records schemaUnverified. A fresh export from current rer is preferred; current archives need no acknowledgment.'],
15
+ export: [['--target', '--output', '--origin', '--session-file'], 'For owner move-in, run rer dev and open the app\'s signed-in Move in page (/app/import in the Base reference; the project\'s own docs name its path): export people there, then upload through the destination door. Developer-only installation backup: complete supported archive of the local copy on this computer, with integrity and exclusions, written to --output. The hosted install cannot be exported yet: --origin or --target remote refuses (exit 2) before any effect and names the command that saves the hosted database.'],
16
+ import: [['--target', '--input', '--journal', '--origin', '--session-file', '--yes', '--legacy-v1', '--acknowledge-unverified-metadata', '--acknowledge-unverified-schema-objects', '--output'], 'For owner move-in, run rer dev and open the app\'s signed-in Move in page (/app/import in the Base reference; the project\'s own docs name its path): upload, review and apply through the door. Developer-only installation recovery: confirmed atomic restore into an empty destination; --journal LATER_ARCHIVE also fences rows that installation erased since. Backup-only local v1 recovery requires --legacy-v1 --acknowledge-unverified-metadata and a new --output v2 archive; --journal requires normal v2 import. A v2 archive without schemaObjects requires --acknowledge-unverified-schema-objects, an empty destination and compatible destination migration history; its receipt records schemaUnverified. A fresh export from current rer is preferred; current archives need no acknowledgment.'],
17
17
  doctor: [[], 'Read-only installed tool, configuration and state diagnostics.', 'never'],
18
18
  describe: [[], 'Read-only description of actual command capabilities.'],
19
19
  check: [['--profile'], 'Read-only structural command contract and operations validation.'],
@@ -26,6 +26,10 @@ import { keepHookSecrets } from './deploy-secrets.mjs';
26
26
  import { withoutLocalSettings, refuseLocalEnvOutput } from './local-env.mjs';
27
27
  import { deployProductDirectory, deployProductArguments, DeployProductError } from './deploy-product.mjs';
28
28
 
29
+ // A hook may wrap this controller. Its needs return to the caller, which renders
30
+ // the combined dry-run summary once after every local plan step has succeeded.
31
+ const deferDryRunSummary = Symbol('defer dry-run summary');
32
+
29
33
  export function validateDeployOptions(options) {
30
34
  const changes = [options.remove && '--remove', options.setSecret && '--set-secret', options.unsetSecret && '--unset-secret', options.rotateSecret && '--rotate-secret', options.reopenInstallationKey && '--reopen-installation-key'].filter(Boolean);
31
35
  if (changes.length > 1 || changes.length && options.dryRun) throw invalid(`Use ${changes.join(', ')}${options.dryRun ? ' and --dry-run' : ''} on separate runs.`);
@@ -88,16 +92,18 @@ export async function deployCommand(context) {
88
92
  const plan = result?.plan ?? (!planned ? undefined : (await owner(['local-plan', '--declaration', path, '--environment', options.environment ?? 'production', ...await deployProductArguments(root, deployProductDirectory(declaration))], production.env, { cwd: root, localOnlyKeys })).plan);
89
93
  result = { ...result, dryRun: true, deployed: false, providerChecked: false, plan };
90
94
  const { signIn, mailOff } = await requireProductionAuth({ root, declaration, env: production.env, setupManifest, setupSeed: context.setupSeed, adapter, declarationPath: path, options }) ?? {};
91
- if (mailOff) store.report(mailOffNotice(mailOff, options.environment ?? 'production'), { names: mailOff.credentialNames });
92
- // The one completion line: rer's own plan step (or its deliberate skip) succeeded. A hook's own line says only that the hook finished.
93
- store.report('deploy: local dry run complete; no credentials were needed, no Cloudflare requests were made and nothing was changed remotely. Target secrets and remote readiness were not checked.');
94
95
  const needed = [...new Set([...neededForRealDeploy({ declaration, env: production.env, signIn, secretsChecked: false }), ...(result.neededForRealDeploy ?? []), ...(extraNeeded ?? [])])];
95
- reportNeeded(store, needed);
96
- if (declaration.secrets?.some(item => item.binding === 'INSTALLATION_KEY')) store.report('deploy: INSTALLATION_KEY is a private generated binding; INSTALLATION_KEY_PREVIOUS is optional and managed only during rotation or an explicit retired-key reopen. Values are withheld.');
96
+ if (!context[deferDryRunSummary]) {
97
+ if (mailOff) store.report(mailOffNotice(mailOff, options.environment ?? 'production'), { names: mailOff.credentialNames });
98
+ // The one completion line: rer's own plan step (or its deliberate skip) succeeded. A hook's own line says only that the hook finished.
99
+ store.report('deploy: local dry run complete; no credentials were needed, no Cloudflare requests were made and nothing was changed remotely. Target secrets and remote readiness were not checked.');
100
+ reportNeeded(store, needed);
101
+ if (declaration.secrets?.some(item => item.binding === 'INSTALLATION_KEY')) store.report('deploy: INSTALLATION_KEY is a private generated binding; INSTALLATION_KEY_PREVIOUS is optional and managed only during rotation or an explicit retired-key reopen. Values are withheld.');
102
+ }
97
103
  return needed.length && result && typeof result === 'object' ? { ...result, neededForRealDeploy: needed } : result;
98
104
  };
99
105
  const runHook = hookContext => options.dryRun ? withLocalDeployPlan(async () => {
100
- try { return await adapter.deploy(hookContext); }
106
+ try { return await adapter.deploy({ ...hookContext, [deferDryRunSummary]: true }); }
101
107
  catch (error) {
102
108
  // Production-only owner/input preflights may refuse before the hook builds.
103
109
  // An offline plan defers those inputs, retaining the refusal as guidance.
@@ -123,7 +129,7 @@ export async function deployCommand(context) {
123
129
  const redaction = createRedaction();
124
130
  const kinds = await settingKinds({ declaration, setupManifest });
125
131
  const addSecret = value => { redaction.addSecrets([value]); store.addSecrets?.([value]); };
126
- for (const value of [...kinds.privateValues(env), env.CLOUDFLARE_ACCOUNT_ID, ...(declaration.secrets ?? []).map(item => env[item.environment])].filter(Boolean)) addSecret(value);
132
+ for (const value of kinds.privateValues(env)) addSecret(value);
127
133
  const environment = options.environment ?? 'production';
128
134
  // The hook uploads every value it is given: a generated secret it gets is the one the Worker keeps.
129
135
  if (!options.dryRun) await keepHookSecrets({ root, environment, declaration, setupManifest, env, generated, reused, rotated, again: deployAgain(options), report: (line, more) => store.report(line, more) });
@@ -77,11 +77,8 @@ export async function withDeployEnvironment(project, options, out, run) {
77
77
  const { declaredSettings, settingKinds, undeclaredNotice } = await import('./deploy-settings.mjs');
78
78
  const declared = await declaredSettings(project), kinds = await settingKinds(declared);
79
79
  const redaction = createRedaction();
80
- // Provider names are public enum values. Masking "resend" would also hide
81
- // RESEND_API_KEY in the preflight's instruction for the missing credential.
82
- const publicProvider = (key, value) => key === 'MAIL_PROVIDER' && /^(resend|sendgrid|cloudflare|rer|later|none)$/.test(value);
83
80
  const supplied = Object.entries(env).filter(([key, value]) => saved[key] !== value);
84
- redaction.addSecrets([...kinds.privateValues(env), ...supplied.filter(([key, value]) => !kinds.shown(key, value) && !publicProvider(key, value)).map(([, value]) => value)]);
81
+ redaction.addSecrets([...kinds.privateValues(env), ...supplied.filter(([key, value]) => !kinds.shown(key, value)).map(([, value]) => value)]);
85
82
  const undeclared = declared.declaration ? kinds.undeclared(supplied.map(([key]) => key)) : [];
86
83
  if (undeclared.length) {
87
84
  const message = undeclaredNotice(undeclared, declared.path);
@@ -3,24 +3,30 @@ import { shared } from './shared.mjs';
3
3
  // What a deployment's declaration says each value is. A value prints in full only
4
4
  // when the declaration names it a non-secret setting: a declared var, the origin
5
5
  // variable and its setting, a setup field that is not a secret, or a name in
6
- // "publicSettings". Anything the declaration calls a secret, any credential-named
7
- // key and any other --env-file value stays redacted.
6
+ // "publicSettings". Secret transport alone does not make an explicitly public
7
+ // setting private. Credential bindings, private setup fields and undeclared
8
+ // --env-file values stay redacted.
8
9
  const PRIVATE_KEYS = ['CLOUDFLARE_ACCOUNT_ID'];
9
10
 
10
11
  export async function settingKinds({ declaration = {}, setupManifest } = {}) {
11
- const { credentialName, credentialUrl, credentialValues } = await shared('redaction.mjs');
12
+ const { credentialName, credentialValue, credentialValues } = await shared('redaction.mjs');
12
13
  const secrets = Array.isArray(declaration.secrets) ? declaration.secrets : [];
13
14
  const names = list => (Array.isArray(list) ? list : []).filter(name => typeof name === 'string');
14
15
  const fields = Array.isArray(setupManifest?.fields) ? setupManifest.fields : [];
15
- const secret = new Set([...PRIVATE_KEYS, ...secrets.flatMap(item => [item?.binding, item?.environment]), ...fields.filter(field => field?.kind === 'secret').map(field => field.name)].filter(Boolean));
16
- const open = new Set([...Object.keys(declaration.vars ?? {}), declaration.originVar, declaration.publicOriginSetting, ...names(declaration.publicSettings), ...fields.filter(field => field?.kind !== 'secret').map(field => field?.name)].filter(name => name && !secret.has(name)));
17
- // A declared setting is exempt from the key-name heuristic (AUTH_ORIGIN is a
18
- // name, not a credential), never when its value carries credentials itself.
19
- const shown = (key, value) => open.has(key) && typeof value === 'string' && !credentialUrl(value);
16
+ const privateFields = new Set([...PRIVATE_KEYS, ...fields.filter(field => field?.kind === 'secret').map(field => field.name)]);
17
+ // Select fields describe public choices. Other transported inputs need an
18
+ // explicit publicSettings declaration; an owner email is still private.
19
+ const publicTransport = new Set([...names(declaration.publicSettings), ...fields.filter(field => field?.kind === 'select').map(field => field.name)]);
20
+ const publicBindings = secrets.filter(item => publicTransport.has(item?.binding) &&
21
+ [item.binding, item.environment].every(name => name && !privateFields.has(name) && !credentialName(name)));
22
+ const secret = new Set([...privateFields, ...secrets.filter(item => !publicBindings.includes(item)).flatMap(item => [item?.binding, item?.environment])].filter(Boolean));
23
+ const open = new Set([...Object.keys(declaration.vars ?? {}), declaration.originVar, declaration.publicOriginSetting, ...names(declaration.publicSettings), ...fields.filter(field => field?.kind !== 'secret').map(field => field?.name), ...publicBindings.flatMap(item => [item.binding, item.environment])].filter(name => name && !secret.has(name)));
24
+ // Public names never exempt a credential-shaped value from masking.
25
+ const shown = (key, value) => open.has(key) && typeof value === 'string' && !credentialValue(value);
20
26
  return {
21
27
  shown,
22
28
  // The values to keep out of output, from every key that is not shown.
23
- privateValues: env => Object.entries(env).flatMap(([key, value]) => typeof value !== 'string' || shown(key, value) ? [] : secret.has(key) ? (value ? [value] : []) : credentialValues({ [key]: value })),
29
+ privateValues: env => Object.entries(env).flatMap(([key, value]) => typeof value !== 'string' || shown(key, value) ? [] : (secret.has(key) || credentialValue(value)) ? (value ? [value] : []) : credentialValues({ [key]: value })),
24
30
  // Keys the deploy hides by default and the person may declare.
25
31
  undeclared: keys => keys.filter(key => !open.has(key) && !secret.has(key) && !credentialName(key)),
26
32
  };
package/cli/deploy.mjs CHANGED
@@ -61,7 +61,7 @@ export function ownerDeploy(declarationPath, lifecycle) {
61
61
  const { main } = await import('../modules/deploy-kit/owner-cli.mjs');
62
62
  const { createRedaction } = await shared('redaction.mjs');
63
63
  const redaction = createRedaction();
64
- redaction.addSecrets([...(await settingKinds({ declaration, setupManifest })).privateValues(env), env.CLOUDFLARE_ACCOUNT_ID, ...(declaration.secrets ?? []).map(item => env[item.environment]).filter(Boolean)]);
64
+ redaction.addSecrets((await settingKinds({ declaration, setupManifest })).privateValues(env));
65
65
  const claim = setupManifest?.steps?.find(item => item.type === 'owner-claim');
66
66
  const declared = binding => (Array.isArray(declaration.secrets) ? declaration.secrets : []).find(item => item?.binding === binding);
67
67
  const owner = async (command, extra = [], values = env, more = {}) => main([command, '--declaration', declarationPath, '--environment', environment, ...(['dry-run', 'local-plan', 'deploy', 'health'].includes(command) ? await deployProductArguments(root, productDirectory) : []), ...extra], values, { cwd: root, localOnlyKeys, ...more });
package/cli/inspect.mjs CHANGED
@@ -253,9 +253,14 @@ export async function doctor(project, store, signal) {
253
253
  const firstParty = project.profile === 'first-party';
254
254
  const workflowInputs = firstParty ? diagnostic.missingEnvironment.filter(name => buildWorkflowInput.test(name)) : [];
255
255
  const deploymentInputs = firstParty ? diagnostic.missingEnvironment.filter(name => !buildWorkflowInput.test(name) && !inputs.includes(name) && !(typeof values[name] === 'string' && values[name].trim())) : [];
256
+ // A required loop that waits only on inputs a deployment supplies (a hosted mail provider's key, say) is ready for
257
+ // local use: the local app suppresses that delivery. Its inputs are named with the other deploy inputs.
258
+ const deployOnlyLoop = loop => firstParty && loop.status === 'missing-inputs' && (loop.missingEnvironment ?? []).every(name => !inputs.includes(name));
259
+ const loopDeploymentInputs = (diagnostic.loops ?? []).filter(deployOnlyLoop).flatMap(loop => loop.missingEnvironment ?? []);
256
260
  if (firstParty) {
257
261
  diagnostic.missingEnvironment = diagnostic.missingEnvironment.filter(name => inputs.includes(name));
258
- diagnostic.ok = diagnostic.commands.every(command => command.available) && !diagnostic.missingEnvironment.length && diagnostic.configuration === 'valid' && (diagnostic.loops ?? []).every(loop => ['ready', 'optional-disabled'].includes(loop.status));
262
+ diagnostic.ok = diagnostic.commands.every(command => command.available) && !diagnostic.missingEnvironment.length && diagnostic.configuration === 'valid' && (diagnostic.loops ?? []).every(loop => ['ready', 'optional-disabled'].includes(loop.status) || deployOnlyLoop(loop));
263
+ diagnostic.loops = (diagnostic.loops ?? []).map(loop => deployOnlyLoop(loop) ? { ...loop, deploymentOnly: true } : loop);
259
264
  }
260
265
  const setupNeeded = firstParty ? !(await localState(project)).prepared : false;
261
266
  const warnings = project.profile === 'first-party' ? await cacheWarnings(project.root) : [];
@@ -263,7 +268,7 @@ export async function doctor(project, store, signal) {
263
268
  node: { actual: process.versions.node, expected: expectedNode },
264
269
  pnpm: project.profile === 'first-party' ? { actual: pnpm, expected: expectedPnpm ?? null, ...(pnpmIssues.length ? { issue: [...new Set(pnpmIssues)].join('\n') } : {}) } : undefined,
265
270
  ai: { issues: aiIssues },
266
- missingInputs: [...new Set([...diagnostic.missingEnvironment, ...inputs])], deploymentInputs, workflowInputs, setupNeeded: setupNeeded || (!firstParty && diagnostic.missingEnvironment.length > 0),
271
+ missingInputs: [...new Set([...diagnostic.missingEnvironment, ...inputs])], deploymentInputs: [...new Set([...deploymentInputs, ...loopDeploymentInputs])], workflowInputs, setupNeeded: setupNeeded || (!firstParty && diagnostic.missingEnvironment.length > 0),
267
272
  sources: project.descriptor.environmentSources ?? {}, state: { path: project.adapter?.state, safe: stateValid, ...(stateIssue ? { issue: stateIssue } : {}) }, scope: 'local-inputs-only' };
268
273
  }
269
274
 
@@ -282,7 +287,7 @@ export function doctorSummary(result) {
282
287
  const unavailable = result.operations.commands.filter(command => !command.available);
283
288
  if (unavailable.length) lines.push('Unavailable command inputs:', ...unavailable.map(command => ` ${command.name}: restore its declared input files.`));
284
289
  if (result.operations.configuration === 'invalid') lines.push('Local configuration is invalid. Check the declared configuration module; private validator details are suppressed.');
285
- for (const loop of result.operations.loops ?? []) if (!['ready', 'optional-disabled'].includes(loop.status)) lines.push(`Loop ${loop.id}: ${loop.status}${loop.missingEnvironment?.length ? `; missing ${loop.missingEnvironment.join(', ')}` : ''}.`);
290
+ for (const loop of result.operations.loops ?? []) if (!['ready', 'optional-disabled'].includes(loop.status) && !loop.deploymentOnly) lines.push(`Loop ${loop.id}: ${loop.status}${loop.missingEnvironment?.length ? `; missing ${loop.missingEnvironment.join(', ')}` : ''}.`);
286
291
  if (!result.state.safe) lines.push(`Local state: ${result.state.issue}`);
287
292
  if (result.browser?.issue) lines.push(`Browser dependencies: ${result.browser.issue}`);
288
293
  else if (result.browser?.note) lines.push(result.browser.note);
@@ -5,6 +5,7 @@ import { excludedTable } from '../modules/portability/catalogue.mjs';
5
5
  import { schemaSQL } from '../modules/portability/schema-identity.mjs';
6
6
  import { ensurePortabilitySchema, portabilityPlan } from '../modules/container/portability-schema.mjs';
7
7
  import { ensureImportLeases } from '../modules/portability/import-lease.mjs';
8
+ import { ensureObjectInventory } from '../modules/portability/object-inventory.mjs';
8
9
  import { splitStatements } from '../modules/portability/sql-text.mjs';
9
10
  import { createPortability } from '../modules/portability/index.mjs';
10
11
  import { migrationReplay } from '../modules/portability/migration-history.mjs';
@@ -55,6 +56,7 @@ export async function managedSchema(db, plan, policy, ledger) {
55
56
  const native = nativeDatabase(scratch);
56
57
  await ensurePortabilitySchema(native, await portabilityPlan(splitStatements));
57
58
  await ensureImportLeases(native);
59
+ await ensureObjectInventory(native);
58
60
  previous = new Map((await all(native, query)).map(object => [key(object), object]));
59
61
  const inventory = [], groups = new Map();
60
62
  const account = (object, disposition, migration) => {
package/cli/parse.mjs CHANGED
@@ -111,7 +111,7 @@ export function parse(argv, cwd = process.cwd(), { independent = false, deferTai
111
111
  export function help(verb) {
112
112
  if (!verb) return `Usage: rer VERB [FLAGS]\nCommands: ${Object.keys({...verbs, ...authoringVerbs}).join(', ')}, signin, size, release, listing, rehearse\nRun rer VERB --help for flags.\n${appSignInHelp}\n`;
113
113
  const known = verbs[verb] ?? authoringVerbs[verb] ?? (verb === 'signin' ? signinVerb : verb === 'size' ? sizeVerb : undefined);
114
- const door = ['export', 'import'].includes(verb) ? 'To move records, start the app and open its signed-in owner import door:\n\n rer dev\n\nOpen /app/import, then export or upload and review supported records there.\n\nTo back up or restore this installation:\n' : '';
114
+ const door = ['export', 'import'].includes(verb) ? 'To move records, start the app and open its signed-in owner import door:\n\n rer dev\n\nOpen its Move in page (/app/import in the Base reference; the project\'s own docs name its path), then export or upload and review supported records there.\n\nTo back up or restore this installation:\n' : '';
115
115
  return `${door}Usage: rer ${verb}${verb === 'refresh-base' ? ' --ref FULL_COMMIT' : ''} [FLAGS]\nFlags: ${(known?.flags ?? commonFlags).join(', ')}\n${known?.effects ?? 'Execute the declared OPERATIONS v2 argv with explicit arguments after --.'}\n`;
116
116
  }
117
117
 
@@ -1,3 +1,4 @@
1
+ import { failureStep } from '../modules/portability/archive-step.mjs';
1
2
  import { installD1Preflight } from '../modules/container/adapters/d1-workerd-setup.mjs';
2
3
  import { portabilityPlan, ensurePortabilitySchema } from '../modules/container/portability-schema.mjs';
3
4
  import { exportArchive, importArchive, fileRecords } from '../modules/portability/files.mjs';
@@ -94,7 +95,8 @@ catch (error) {
94
95
  const identityTable = error instanceof PortabilityError && error.code.startsWith('identity_table_undeclared:') ? error.code.slice('identity_table_undeclared:'.length) : undefined;
95
96
  const safety = error instanceof PortabilityError && error.code.startsWith('import_row_') ? error.code.slice(error.code.indexOf(':')+1) : undefined;
96
97
  // Anything that is not a portability refusal names its class and code, never its message: that can hold row data.
98
+ const step=failureStep(error) ?? (process.argv[2] === 'import' ? undefined : process.argv[2]);
97
99
  const native = error instanceof PortabilityError || error.message === 'archive_json' ? undefined : nativeFailure(error);
98
100
  process.exitCode = exitCode;
99
- await reply({ ok: false, code: exitCode === 2 ? 'invalid_archive' : exitCode === 4 ? 'restore_refused' : 'transfer_failed', reason, ...(reExport ? { reExport: true } : {}), ...(migration ? { migration } : {}), ...(unapplied ? { unapplied } : {}), ...(occupied ? { occupied } : {}), ...(identityTable ? { identityTable } : {}), ...(safety ? { safety } : {}), ...(native ? { native } : {}), exitCode });
101
+ await reply({ ok: false, code: exitCode === 2 ? 'invalid_archive' : exitCode === 4 ? 'restore_refused' : 'transfer_failed', reason, step, ...(reExport ? { reExport: true } : {}), ...(migration ? { migration } : {}), ...(unapplied ? { unapplied } : {}), ...(occupied ? { occupied } : {}), ...(identityTable ? { identityTable } : {}), ...(safety ? { safety } : {}), ...(native ? { native } : {}), exitCode });
100
102
  }
@@ -110,7 +110,7 @@ async function nativeWorker(store,direction,input,signal) {
110
110
  : receipt.reason?.startsWith('migration_ledger:') ? ledgerRefusal(receipt)
111
111
  : receipt.migration ? `Restore refused (schema_mismatch:migration_data): migration ${receipt.migration} changed data after this archive was made, so its rows would skip that change. Restore an archive made after it.`
112
112
  : receipt.occupied ? occupiedRefusal(input,receipt.reason)
113
- : `The portability module refused or failed the archive operation (${receipt.reason ?? 'incomplete'}${receipt.native ? `: ${receipt.native.class}${receipt.native.code && receipt.native.code !== receipt.reason ? `, code ${receipt.native.code}` : ''}` : ''}).${receipt.native && receipt.exitCode === 1 ? ' Run the same command again; it starts over.' : ''}`, receipt.exitCode ?? 1);
113
+ : `The portability module refused or failed the archive operation (${receipt.reason ?? 'incomplete'}${receipt.native ? `: ${receipt.native.class}${receipt.native.code && receipt.native.code !== receipt.reason ? `, code ${receipt.native.code}` : ''}` : ''})${receipt.step ? ` during ${receipt.step}` : ''}.${receipt.native && receipt.exitCode === 1 ? ' Run the same command again; it starts over.' : ''}`, receipt.exitCode ?? 1);
114
114
  delete receipt.ok;
115
115
  return receipt;
116
116
  }
@@ -1,5 +1,46 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.2.119
4
+
5
+ ### Large installation archive inventories
6
+
7
+ - Archive export and restore store large object inventories in bounded internal rows instead of one oversized job-metadata value, addressing the D1 cell limit reached by a company-copy export with 13,879 objects. Small jobs keep their existing metadata representation. Archive parts can group multiple small objects, and restore reuses a shared part while reading its objects. The archive format and version are unchanged.
8
+ - Managed intake accounts for the internal inventory table. Native archive failures name the transfer stage without printing SQL, object keys, row values or filesystem paths.
9
+
10
+ ### Publication scan copy pointers
11
+
12
+ - Publication scans locate Gitleaks full matches within their reported lines when directory-fragment columns are relative to a fragment beginning mid-line. Every possible occurrence must satisfy the existing copy-pointer or checksum content proof; unresolved matches and unproved occurrences retain their finding.
13
+
14
+ ### Deploy output and timing checks
15
+
16
+ - Declared public deployment settings, including `MAIL_PROVIDER`, remain readable even when carried through secret transport. Credential-named bindings and aliases, private setup fields and credential-shaped values stay masked.
17
+ - Nested deploy hooks defer their dry-run completion and needs summary to the outer command, so the combined summary prints once after local planning succeeds.
18
+ - The real 20-second listing-backoff test still uses the native timer and high-resolution timestamps, observes the configured delay and compares elapsed time at Node's millisecond timer precision to avoid a fractional early-wake failure. Runtime backoff behavior is unchanged.
19
+
20
+ These changes become available after adopting the released package or refreshed source. This preparation establishes no company-copy export result, seed adoption, deployment or production result. The actual diff changes portability inventory storage and archive-part grouping, adds no product migrations, changes no import-kit source and preserves the archive format identity.
21
+
22
+ ## 0.2.118
23
+
24
+ ### Local diagnostics and Move in guidance
25
+
26
+ - After a clean `rer dev`, `rer doctor` treats required loops waiting only on deployment inputs as ready for local use and lists those inputs as optional locally and needed for deployment. It still checks local configuration, commands and required local inputs.
27
+ - Import and export help directs owners to the app's signed-in Move in page; `/app/import` is the Base reference path, and each project's own documentation names its path.
28
+
29
+ ### Business device pairing only
30
+
31
+ - Removed `rer business connect --code` and its browser-created pairing flow, help and recovery guidance. Pair with `rer business connect --business ID`; the Business Connect page names the ID. Saved device pairings can still resume.
32
+
33
+ ### Explicit outbound mail off
34
+
35
+ - Foundations accept an explicit `MAIL_PROVIDER=none` through declared environment-to-Worker bindings. Provider and sender bindings, and the mail key at the binding layer, are optional; mail off needs no sender or key and binds neither `MAIL_FROM` nor `MAIL_API_KEY`. An absent provider does not select off, and enabled providers still require their real sender and credentials. Conflicting mail-off credentials are refused.
36
+ - `rer deploy --dry-run` validates this choice, says outbound email is off and names `rer signin` for owner entry. Email sign-in links and invitations remain unavailable. A declared optional operations mail loop reports `optional-disabled`; a required delivery loop still fails when mail is off.
37
+
38
+ ### Recorded app target for owner sign-in
39
+
40
+ - `rer signin` selects the app Worker from the held ownership record, preserving its recorded account, project, environment and origin instead of recomputing the default Worker name. The selected identity must match the caller and live deployment. Wrong account, project or environment, missing or replaced Worker or receipt, and substitution of a public-page Worker refuse without issuing an owner grant.
41
+
42
+ These changes become available to seeds after adopting the released package or refreshed source. This preparation establishes no seed adoption, provider delivery, deployment or production result. The actual diff adds no migrations and changes neither modules/import-kit nor modules/portability.
43
+
3
44
  ## 0.2.117
4
45
 
5
46
  ### Size queue isolation and timeout attribution
package/global/README.md CHANGED
@@ -531,12 +531,6 @@ different origin or Business: disconnect first or use a separate
531
531
  `RER_BUSINESS_STATE_DIR`. It reports this computer's plain name, working folder,
532
532
  agent and OS (including WSL) for the owner to recognise.
533
533
 
534
- For older Business pages, `rer business connect --code ABCD-1234 --origin URL`
535
- keeps the browser-created code and six-digit confirmation-number flow unchanged
536
- in 0.2.98. It cannot be combined with `--business` or `--timeout`. Removal is
537
- planned for a later release after the production rollback target uses device pairing.
538
- The install.sh form still passes either connect command through unchanged.
539
-
540
534
  The owner chooses three things on the computer's row in the browser: "Add rules
541
535
  so Claude Code never reads your secret files" (Claude Code user Read deny
542
536
  rules), "Keep this computer awake while it works" (recorded; nothing keeps the computer awake now that rer runs no exports) and "Install the background service" (a launchd or systemd user
@@ -9,7 +9,7 @@ import { assert, hexId } from './validation.mjs';
9
9
  import { regularBytes } from './artifact.mjs';
10
10
  import { packageBuild } from './prepare.mjs';
11
11
  import { cloudflare } from './provider.mjs';
12
- import { ownerTarget, ownerWorkerName, ownerPlan, ownerDeploy, ownerHealth, settleOwnerState, sameDeployment, localOnlyBinding, withStateLock, readOwnerState, writeOwnerState, workerSettings, withScopeHints } from './owner.mjs';
12
+ import { ownerTarget, recordedOwnerTarget, ownerWorkerName, ownerPlan, ownerDeploy, ownerHealth, settleOwnerState, sameDeployment, localOnlyBinding, withStateLock, readOwnerState, writeOwnerState, workerSettings, withScopeHints } from './owner.mjs';
13
13
  import { activeWorker, secretBindings } from './worker.mjs';
14
14
  import { ownerSecret } from './owner-secret.mjs';
15
15
  import { ownerRemove } from './owner-remove.mjs';
@@ -97,11 +97,14 @@ export async function main(argv = process.argv.slice(2), env = process.env, {fet
97
97
  };
98
98
  // Transport must see the same outer signal when deciding whether to retry.
99
99
  const api = deploymentDiagnostics(cloudflare({accountId:env.CLOUDFLARE_ACCOUNT_ID,token:env.CLOUDFLARE_API_TOKEN,fetch:guardedRequest,requestSignal,budgetMigrations:command === 'deploy',...(command === 'remove' ? {maxRequests:Infinity,removal:true} : {})}),{onCreated});
100
+ const statePath = values.state ? resolve(root,values.state) : join(root,'.rer','deploy',`${values.environment}.json`);
101
+ // Signin uses only the selected app record; never discover a sibling page Worker.
100
102
  // Removal needs the recorded account/name identity, not a workers.dev route.
101
- const target = command === 'remove'
103
+ const target = command === 'signin'
104
+ ? await recordedOwnerTarget(api,{project:declaration.project,environment:values.environment,statePath})
105
+ : command === 'remove'
102
106
  ? {accountId:api.accountId,project:declaration.project,environment:values.environment,worker:ownerWorkerName(declaration.project,values.environment)}
103
107
  : await ownerTarget(api,{project:declaration.project,environment:values.environment});
104
- const statePath = values.state ? resolve(root,values.state) : join(root,'.rer','deploy',`${target.environment}.json`);
105
108
  if (command === 'remove') return {ok:true,...await ownerRemove({api,target,statePath}).catch(error => { throw api.annotateFailure(error,'remove'); })};
106
109
  if (command === 'rekey') return ownerRekey({api,target,statePath,declaration,env,notice});
107
110
  if (command === 'secret') {
@@ -100,15 +100,33 @@ async function listedAfterCreate(api, kind, item, {windowMs = LIST_WINDOW_MS, cl
100
100
  }
101
101
  }
102
102
  /** Reads the account's workers.dev subdomain; no other provider request. */
103
- export async function ownerTarget(provider, {project, environment}) {
103
+ async function workerOrigin(provider, worker) {
104
104
  const api = withScopeHints(provider);
105
- const worker = ownerWorkerName(project, environment);
106
105
  let result;
107
106
  try { result = await api.request('/workers/subdomain'); }
108
107
  catch (error) { if (error.status === 404) throw new Error('workers-subdomain-missing'); throw error; }
109
108
  assert(typeof result?.subdomain === 'string' && result.subdomain, 'workers-subdomain-missing');
110
109
  assert(/^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$/.test(result.subdomain),'invalid-account-subdomain');
111
- return {accountId:api.accountId,project,environment,worker,origin:`https://${worker}.${result.subdomain}.workers.dev`};
110
+ return `https://${worker}.${result.subdomain}.workers.dev`;
111
+ }
112
+ export async function ownerTarget(api, {project, environment}) {
113
+ const worker = ownerWorkerName(project, environment);
114
+ return {accountId:api.accountId,project,environment,worker,origin:await workerOrigin(api,worker)};
115
+ }
116
+
117
+ /** Signin follows the selected app record, including native uploaders' custom names.
118
+ * The caller still supplies the account, project and environment to validate it.
119
+ * Later state reads and the live provider check must corroborate this identity. */
120
+ export async function recordedOwnerTarget(api, {project, environment, statePath}) {
121
+ ownerWorkerName(project, environment); // Validate the caller's project and environment as usual.
122
+ let recorded;
123
+ try { recorded = JSON.parse(await regularBytes(statePath, 4 * 1024 * 1024)); }
124
+ catch (error) { throw new Error(error.code === 'ENOENT' ? 'missing-owner-receipt' : 'invalid-owner-state'); }
125
+ assert(resourceName(recorded?.worker), 'invalid-owner-state');
126
+ const target = {accountId:api.accountId,project,environment,worker:recorded.worker};
127
+ const state = await readOwnerState(statePath,target);
128
+ assert(state.receipt, 'missing-owner-receipt');
129
+ return {...target,origin:state.receipt.origin ?? await workerOrigin(api,target.worker)};
112
130
  }
113
131
 
114
132
  function entries(declaration, key) {
@@ -1133,6 +1133,8 @@ modules/portability/sqlite-tokens.mjs
1133
1133
  modules/portability/bookkeeping.mjs
1134
1134
  modules/portability/snapshot-objects.mjs
1135
1135
  modules/portability/storage.mjs
1136
+ modules/portability/object-inventory.mjs
1137
+ modules/portability/archive-step.mjs
1136
1138
  modules/portability/copies.mjs
1137
1139
  modules/portability/derived.mjs
1138
1140
  modules/portability/dev-data.mjs
@@ -1268,3 +1270,9 @@ parameters of at most 2 MiB each. It refuses compressed or encrypted entries,
1268
1270
  unsafe paths, duplicate or unexpected members, missing ledgers, other versions,
1269
1271
  container transport, later-journal and legacy-recovery options. Refusals use
1270
1272
  constant `managed_data_*` reasons and do not print source values.
1273
+
1274
+ Large object inventories are stored in bounded internal rows so D1 cell and row
1275
+ limits do not prevent backups. Native CLI failures name the transfer stage (for
1276
+ example `export_snapshot`) without printing SQL, values or absolute paths. Keep
1277
+ the stopped source and retry after resolving the named stage; changing source
1278
+ objects or deleting data is not a remedy for an inventory storage failure.
@@ -0,0 +1,11 @@
1
+ // Stage names are fixed by the implementation. Never send native messages,
2
+ // SQL, object keys, row values or filesystem paths to the CLI.
3
+ const steps=new Set(['export_object_inventory','export_snapshot','export_part']);
4
+ export async function archiveStep(step,operation) {
5
+ try{return await operation();}
6
+ catch(error){
7
+ if(error && typeof error==='object' && !error.portabilityStep)error.portabilityStep=step;
8
+ throw error;
9
+ }
10
+ }
11
+ export const failureStep=error=>steps.has(error?.portabilityStep)?error.portabilityStep:undefined;
@@ -1,4 +1,6 @@
1
- import { statement, state, guard, job, save, partStatements, parts, readPart, cleanCopies, quote, stream, insist, run } from './storage.mjs';
1
+ import { prepareInventory, metadataJson, inventoryStatements } from './object-inventory.mjs';
2
+ import { archiveStep } from './archive-step.mjs';
3
+ import { statement, state, guard, job, save, partStatements, parts, readPart, cleanCopies, quote, stream, insist } from './storage.mjs';
2
4
  import { catalogue, encodedRow } from './catalogue.mjs';
3
5
  import { checkReferences } from './references.mjs';
4
6
  import { createCopy,readCopy,normalizeCopy } from './copies.mjs';
@@ -16,7 +18,7 @@ export async function createExport(context,actor='direct') {
16
18
  for(const [binding,spec] of Object.entries(config.buckets).sort()) {
17
19
  let cursor;
18
20
  do {
19
- const page=await buckets[binding].list({prefix:spec.prefix??'',cursor,include:['httpMetadata','customMetadata']});
21
+ const page=await archiveStep('export_object_inventory',()=>buckets[binding].list({prefix:spec.prefix??'',cursor,include:['httpMetadata','customMetadata']}));
20
22
  for(const object of page.objects) objects.push({bucket:binding,key:object.key,size:object.size,etag:object.etag,httpMetadata:object.httpMetadata??{},customMetadata:object.customMetadata??{}});
21
23
  cursor=page.truncated?page.cursor:undefined;
22
24
  }while(cursor);
@@ -27,9 +29,10 @@ export async function createExport(context,actor='direct') {
27
29
  const manifest={kind:'manifest',format:FORMAT,version:2,product:config.product,exportedAt:new Date().toISOString(),sourceIdentity:s.source_identity,schema:cat.identity,tables:cat.tables,schemaObjects:cat.schemaObjects,objects,payloads:[],excludedTables:cat.excludedTables,excludedColumns:config.secretColumns,requiredConfiguration:config.requiredConfiguration,partCount:0,epoch:s.epoch,journal:await journal(db,s.source_identity),...(migrations?{migrations}:{}),...(origin?{migrationLedger:origin}:{})};
28
30
  const row={id,kind:'export',actor,phase:'building',completed:0,metadata:{manifest,copies,cursor:{table:0,row:0,binary:0,offset:0},partBytes:config.partBytes}};
29
31
  const audit=context.onExport?context.onExport({id,actor:actor==='direct'?null:actor,scope:'installation'}):[];
30
- await db.batch([guard(db,'EXISTS(SELECT 1 FROM rer_portability_state WHERE id=1 AND applying=0 AND source_identity=? AND epoch=?)',s.source_identity,s.epoch),
31
- ...cat.tables.flatMap(t=>createCopy(db,t,copies[t.name],true)),statement(db,'INSERT INTO rer_portability_jobs(id,kind,actor,phase,metadata) VALUES(?,?,?,?,?)',id,'export',actor,row.phase,canonicalJson(row.metadata)),db.prepare('DELETE FROM rer_portability_guard'),
32
- ...audit.map(([sql,args])=>statement(db,sql,...args))]);
32
+ prepareInventory(row);
33
+ await archiveStep('export_snapshot',()=>db.batch([guard(db,'EXISTS(SELECT 1 FROM rer_portability_state WHERE id=1 AND applying=0 AND source_identity=? AND epoch=?)',s.source_identity,s.epoch),
34
+ ...cat.tables.flatMap(t=>createCopy(db,t,copies[t.name],true)),statement(db,'INSERT INTO rer_portability_jobs(id,kind,actor,phase,metadata) VALUES(?,?,?,?,?)',id,'export',actor,row.phase,metadataJson(row.metadata)),...inventoryStatements(db,row),db.prepare('DELETE FROM rer_portability_guard'),
35
+ ...audit.map(([sql,args])=>statement(db,sql,...args))]));
33
36
  try {
34
37
  await dropDevData(db,cat.tables,copies);
35
38
  await dropProviderData(db,cat.tables,copies);
@@ -38,7 +41,7 @@ export async function createExport(context,actor='direct') {
38
41
  for(const [name,producer] of Object.entries(config.extraPayloads).sort()) {
39
42
  const descriptor=await capturePayload(db,id,manifest.payloads.length,await producer.produce());manifest.payloads.push({name,...descriptor});
40
43
  }
41
- await run(db,'UPDATE rer_portability_jobs SET metadata=? WHERE id=?',canonicalJson(row.metadata),id);
44
+ await save(db,row).run();
42
45
  await checkReferences(context,copies,objects);
43
46
  return id;
44
47
  }catch(e){await abortExport(context,id);throw e;}
@@ -68,24 +71,30 @@ async function nextItems(context,row) {
68
71
  }
69
72
  const entries=[...manifest.objects,...manifest.payloads];
70
73
  if(cursor.binary===entries.length)return null;
71
- const entry=entries[cursor.binary],length=Math.min(entry.size-cursor.offset,Math.max(1,Math.floor((partBytes-200-utf8(canonicalJson({bucket:entry.bucket??null,key:entry.key??null,name:entry.name??null})).length)*3/4)));
72
- let bytes,item;
73
- if(cursor.binary<manifest.objects.length) {
74
- const object=await buckets[entry.bucket].get(entry.key,length?{range:{offset:cursor.offset,length}}:undefined);
75
- insist(object && object.etag===entry.etag,'object_changed:'+entry.bucket+':'+entry.key);
76
- bytes=new Uint8Array(await object.arrayBuffer());insist(bytes.length===length,'object_changed:'+entry.key);
77
- item={kind:'object',bucket:entry.bucket,key:entry.key,offset:cursor.offset,data:base64(bytes)};
78
- }else{
79
- bytes=await payloadRange(db,row.id,cursor.binary-manifest.objects.length,cursor.offset,length);
80
- item={kind:'payload',name:entry.name,offset:cursor.offset,data:base64(bytes)};
74
+ const items=[];let used=200;
75
+ while(cursor.binary<entries.length && items.length<1000 && partBytes-used>512) {
76
+ const entry=entries[cursor.binary],header=utf8(canonicalJson({bucket:entry.bucket??null,key:entry.key??null,name:entry.name??null})).length;
77
+ if(used+200+header>=partBytes){insist(items.length,'binary_part_too_large');break;}
78
+ const length=Math.min(entry.size-cursor.offset,Math.max(1,Math.floor((partBytes-used-200-header)*3/4)));
79
+ let bytes,item;
80
+ if(cursor.binary<manifest.objects.length) {
81
+ const object=await buckets[entry.bucket].get(entry.key,length?{range:{offset:cursor.offset,length}}:undefined);
82
+ insist(object && object.etag===entry.etag,'object_changed:'+entry.bucket+':'+entry.key);
83
+ bytes=new Uint8Array(await object.arrayBuffer());insist(bytes.length===length,'object_changed:'+entry.key);
84
+ item={kind:'object',bucket:entry.bucket,key:entry.key,offset:cursor.offset,data:base64(bytes)};
85
+ }else{
86
+ bytes=await payloadRange(db,row.id,cursor.binary-manifest.objects.length,cursor.offset,length);
87
+ item={kind:'payload',name:entry.name,offset:cursor.offset,data:base64(bytes)};
88
+ }
89
+ const hash=new StreamHash();if(cursor.hash){hash.h=Uint32Array.from(cursor.hash.h);hash.block=Uint8Array.from(cursor.hash.block);hash.size=cursor.hash.size;hash.used=cursor.hash.used;}
90
+ hash.update(bytes);cursor.offset+=length;
91
+ if(cursor.offset===entry.size) {
92
+ const digest=hash.digest();if(entry.sha256)insist(entry.sha256===digest,'payload_changed');entry.sha256=digest;delete entry.etag;
93
+ cursor.binary++;cursor.offset=0;delete cursor.hash;
94
+ }else cursor.hash={h:[...hash.h],block:[...hash.block],size:hash.size,used:hash.used};
95
+ items.push(item);used+=utf8(canonicalJson(item)).length+1;
81
96
  }
82
- const hash=new StreamHash();if(cursor.hash){hash.h=Uint32Array.from(cursor.hash.h);hash.block=Uint8Array.from(cursor.hash.block);hash.size=cursor.hash.size;hash.used=cursor.hash.used;}
83
- hash.update(bytes);cursor.offset+=length;
84
- if(cursor.offset===entry.size) {
85
- const digest=hash.digest();if(entry.sha256)insist(entry.sha256===digest,'payload_changed');entry.sha256=digest;delete entry.etag;
86
- cursor.binary++;cursor.offset=0;delete cursor.hash;
87
- }else cursor.hash={h:[...hash.h],block:[...hash.block],size:hash.size,used:hash.used};
88
- return [item];
97
+ return items;
89
98
  }
90
99
  export async function advanceExport(context,id,{expectedPart,maxParts=1}={}) {
91
100
  insist(Number.isSafeInteger(maxParts) && maxParts>0 && maxParts<=100,'advance_limit');
@@ -94,10 +103,10 @@ export async function advanceExport(context,id,{expectedPart,maxParts=1}={}) {
94
103
  if(row.phase==='complete')return exportStatus(context,id);
95
104
  insist(row.phase==='building','export_phase',409);
96
105
  if(expectedPart!==undefined && row.completed!==expectedPart)return exportStatus(context,id);
97
- const before=row.completed,items=await nextItems(context,row);
106
+ const before=row.completed,binary=row.metadata.cursor.binary,items=await archiveStep('export_part',()=>nextItems(context,row));
98
107
  if(items) {
99
108
  const part=await makePart(before,items),size=await checkPart(part,before,row.metadata.partBytes);row.completed++;row.metadata.bytes=(row.metadata.bytes??0)+size+1;
100
- try{await context.db.batch([guard(context.db,'(SELECT completed FROM rer_portability_jobs WHERE id=?)=?',id,before),...partStatements(context.db,id,part),save(context.db,row),context.db.prepare('DELETE FROM rer_portability_guard')]);}
109
+ try{await context.db.batch([guard(context.db,'(SELECT completed FROM rer_portability_jobs WHERE id=?)=?',id,before),...partStatements(context.db,id,part),...inventoryStatements(context.db,row,binary,row.metadata.cursor.binary),save(context.db,row),context.db.prepare('DELETE FROM rer_portability_guard')]);}
101
110
  catch(e){if((await job(context.db,id)).completed===before)throw e;}
102
111
  if(expectedPart!==undefined)expectedPart++;
103
112
  }else{
@@ -1,3 +1,4 @@
1
+ import { metadataJson } from './object-inventory.mjs';
1
2
  import { encodedRow, validateRow, bindCell } from './catalogue.mjs';
2
3
  import { readCopy, copyRowid } from './copies.mjs';
3
4
  import { canonicalJson } from './hash.mjs';
@@ -44,7 +45,7 @@ const guardSql=`INSERT INTO rer_portability_guard(value) SELECT CASE WHEN (
44
45
  ) THEN 1 ELSE 0 END`;
45
46
  const checkpoint=(row,before,metadata,queries)=>[
46
47
  {sql:guardSql,params:[row.id,before,row.id]},...queries,
47
- {sql:'UPDATE rer_portability_jobs SET phase=?,completed=?,metadata=? WHERE id=?',params:[row.phase,row.completed,canonicalJson(metadata),row.id]},
48
+ {sql:'UPDATE rer_portability_jobs SET phase=?,completed=?,metadata=? WHERE id=?',params:[row.phase,row.completed,metadataJson(metadata),row.id]},
48
49
  {sql:'DELETE FROM rer_portability_guard'},
49
50
  ];
50
51
  // Each page's staged updates, cursor and value-free change counts commit together.
@@ -55,10 +56,10 @@ export async function normalizeStaged(context,row) {
55
56
  const copy=row.metadata.copies[table.name],count=row.metadata.stagedCounts?.[table.name]??0;
56
57
  let offset=row.metadata.safetyProgress?.[table.name]??0;
57
58
  while(offset<count) {
58
- let before=canonicalJson(row.metadata),queries=[];
59
+ let before=metadataJson(row.metadata),queries=[];
59
60
  const commit=async()=>{
60
61
  await db.batch(checkpoint(row,before,row.metadata,queries).map(q=>statement(db,q.sql,...(q.params??[]))));
61
- before=canonicalJson(row.metadata);queries=[];
62
+ before=metadataJson(row.metadata);queries=[];
62
63
  };
63
64
  for(const {row:raw,meta} of await readCopy(db,table,copy,offset,100,true)) {
64
65
  const original=encodedRow(table,raw),after=mapImportRow(config,table,original);
@@ -1,5 +1,5 @@
1
+ import { metadataJson } from './object-inventory.mjs';
1
2
  import { RPC_LIMIT, rpcSize, rpcSizeBytes } from '../d1-transport/index.mjs';
2
- import { canonicalJson } from './hash.mjs';
3
3
  import { statement, save, readPart, insist } from './storage.mjs';
4
4
  import { checkPart } from './format.mjs';
5
5
 
@@ -18,11 +18,11 @@ export async function stagePart(context,row,part,queries,stagedCounts) {
18
18
  insist(!pending || (pending.index===part.index && pending.sha256===part.sha256),'part_conflict',409);
19
19
  let next=pending?.next??0;
20
20
  while(next<queries.length) {
21
- const before=canonicalJson(row.metadata),chunk=[];
21
+ const before=metadataJson(row.metadata),chunk=[];
22
22
  // Reserve the largest cursor for this part before admitting any data.
23
23
  row.metadata.pendingPart={index:part.index,sha256:part.sha256,next:queries.length};
24
24
  const guard={sql:guardSql,params:[row.id,row.completed,before,row.id]};
25
- const checkpoint={sql:'UPDATE rer_portability_jobs SET phase=?,completed=?,metadata=? WHERE id=?',params:[row.phase,row.completed,canonicalJson(row.metadata),row.id]};
25
+ const checkpoint={sql:'UPDATE rer_portability_jobs SET phase=?,completed=?,metadata=? WHERE id=?',params:[row.phase,row.completed,metadataJson(row.metadata),row.id]};
26
26
  let size=rpcSize([guard,checkpoint,{sql:'DELETE FROM rer_portability_guard'}]),end=next;
27
27
  for(;end<queries.length;end++) {
28
28
  const candidate=combine(size,rpcSize([queries[end]]));
@@ -37,7 +37,7 @@ export async function stagePart(context,row,part,queries,stagedCounts) {
37
37
  // Authenticate the reassembled stored bytes before acknowledging the part.
38
38
  const stored=await readPart(db,row.id,part.index);
39
39
  await checkPart(stored,part.index,config.partBytes);insist(stored.sha256===part.sha256,'part_checksum');
40
- const before=canonicalJson(row.metadata),completed=row.completed;
40
+ const before=metadataJson(row.metadata),completed=row.completed;
41
41
  delete row.metadata.pendingPart;row.metadata.stagedCounts=stagedCounts;row.completed++;
42
42
  await db.batch([statement(db,guardSql,row.id,completed,before,row.id),
43
43
  statement(db,'INSERT INTO rer_portability_parts VALUES(?,?,?)',row.id,part.index,part.sha256),
@@ -1,3 +1,4 @@
1
+ import { prepareInventory, metadataJson, inventoryStatements } from './object-inventory.mjs';
1
2
  import { libraryMigrations } from './library-history.mjs';
2
3
  import { all, statement, state, guard, job, save, partQueries, parts, readPart, dropCopyStatements, cleanCopyStatements, quote, stream, insist, run } from './storage.mjs';
3
4
  import { stagePart } from './import-staging.mjs';
@@ -78,11 +79,12 @@ export async function createImport(context,manifest,actor='direct',legacyRecover
78
79
  }
79
80
  const copies=Object.fromEntries(targets.map((t,i)=>[t.name,'rer_portability_stage_'+id.replaceAll('-','')+'_'+i]));
80
81
  const row={id,kind:'import',actor,phase:'staging',completed:0,metadata:{manifest:original,...(unverified?{schemaUnverified:{reason:'acknowledged recovery of a version 2 archive without schemaObjects',sourceIdentity:original.sourceIdentity,exportedAt:original.exportedAt,acknowledgedAt:new Date().toISOString()}}:{}),...(skippedTables.length?{restoreManifest:manifest,skippedTables}:{}),copies,targets,triggers:cat.triggers,...(bridge.fromSchema?{bridge}:{}),...(replays?{replays}:{}),...(legacyRecovery?{legacyRecovery}: {}),...(ledgers?{ledgers}:{})}};
82
+ prepareInventory(row);
81
83
  try {
82
84
  await db.batch([guard(db,'(SELECT import_job FROM rer_portability_state WHERE id=1) IS NULL'),
83
85
  statement(db,'UPDATE rer_portability_state SET import_job=?,applying=? WHERE id=1',id,unverified?1:0),
84
86
  ...targets.flatMap(t=>createCopy(db,t,copies[t.name])),
85
- statement(db,'INSERT INTO rer_portability_jobs(id,kind,actor,phase,metadata) VALUES(?,?,?,?,?)',id,'import',actor,row.phase,canonicalJson(row.metadata)),createImportLease(db,id),db.prepare('DELETE FROM rer_portability_guard')]);
87
+ statement(db,'INSERT INTO rer_portability_jobs(id,kind,actor,phase,metadata) VALUES(?,?,?,?,?)',id,'import',actor,row.phase,metadataJson(row.metadata)),...inventoryStatements(db,row),createImportLease(db,id),db.prepare('DELETE FROM rer_portability_guard')]);
86
88
  }catch(e){const running=(await state(db)).import_job;if(running)throw await importRunning(db,running);throw e;}
87
89
  return id;
88
90
  }
@@ -136,10 +138,10 @@ async function uploadLeasedPart(context,id,index,part) {
136
138
  return importStatus(context,id,{includeLease:false});
137
139
  }
138
140
  const binaryKey=entry=>canonicalJson(entry.bucket===undefined?['payload',entry.name]:['object',entry.bucket,entry.key]);
139
- function binaryStream(context,id,entry,indexes) {
141
+ function binaryStream(entry,indexes,read) {
140
142
  const selected=indexes.get(binaryKey(entry));insist(selected,'binary_inventory');
141
143
  return stream((async function*(){
142
- for(const index of selected)for(const item of (await readPart(context.db,id,index)).items) {
144
+ for(const index of selected)for(const item of (await read(index)).items) {
143
145
  if(entry.bucket===undefined?item.kind==='payload' && item.name===entry.name:item.kind==='object' && item.bucket===entry.bucket && item.key===entry.key)yield unbase64(item.data);
144
146
  }
145
147
  })());
@@ -218,17 +220,24 @@ async function finalizeLeasedImport(context,id,{contentHash,archiveHash,laterJou
218
220
  }
219
221
  row.phase='applying';
220
222
  await db.batch([guard(db,'(SELECT import_job FROM rer_portability_state WHERE id=1)=?',id),db.prepare('UPDATE rer_portability_state SET applying=1 WHERE id=1'),save(db,row),db.prepare('DELETE FROM rer_portability_guard')]);
223
+ // Grouped objects often share a part. Keep one immutable part during this
224
+ // leased finalize, rather than rereading its bytes for every small object.
225
+ let cached;
226
+ const readBinaryPart=async index=>{
227
+ if(cached?.index!==index)cached=await readPart(db,id,index);
228
+ return cached;
229
+ };
221
230
  try {
222
231
  for(const object of manifest.objects) {
223
232
  await run(db,'INSERT OR IGNORE INTO rer_portability_owned VALUES(?,?,?)',id,object.bucket,object.key);
224
- await writeObject(context.buckets[object.bucket],object,binaryStream(context,id,object,binaryIndexes));
233
+ await writeObject(context.buckets[object.bucket],object,binaryStream(object,binaryIndexes,readBinaryPart));
225
234
  const stored=await context.buckets[object.bucket].get(object.key);insist(stored,'object_write_failed');
226
235
  // Read back independently, including multipart results, before the commit.
227
236
  const hash=new (await import('./hash.mjs')).StreamHash();const reader=stored.body.getReader();
228
237
  for(;;){const {value,done}=await reader.read();if(done)break;hash.update(value);}reader.releaseLock();
229
238
  insist(hash.digest()===object.sha256,'object_write_checksum');
230
239
  }
231
- for(const payload of manifest.payloads)await config.extraPayloads[payload.name].consume(binaryStream(context,id,payload,binaryIndexes),payload);
240
+ for(const payload of manifest.payloads)await config.extraPayloads[payload.name].consume(binaryStream(payload,binaryIndexes,readBinaryPart),payload);
232
241
  const derived=await rebuildDerivedStatements(db,config);
233
242
  row.phase='complete';row.metadata.contentHash=digest;
234
243
  if(row.metadata.schemaUnverified)row.metadata.schemaUnverified.archiveHash=archiveHash;
@@ -1,3 +1,4 @@
1
+ import { ensureObjectInventory } from './object-inventory.mjs';
1
2
  import { ensureImportLeases } from './import-lease.mjs';
2
3
  import { policy, catalogue } from './catalogue.mjs';
3
4
  import { wireDatabase } from '../d1-transport/index.mjs';
@@ -14,7 +15,7 @@ export { createPortabilityHandler } from './http.mjs';
14
15
  export async function createPortability({db,buckets={},policy:input,migrations,onExport,migrationRoot}) {
15
16
  insist(onExport===undefined || typeof onExport==='function','invalid_on_export');
16
17
  db=wireDatabase(db);
17
- const config=policy(input);await catalogue(db,config);
18
+ const config=policy(input);await ensureObjectInventory(db);await catalogue(db,config);
18
19
  for(const name of Object.keys(config.buckets))insist(buckets[name] && typeof buckets[name].get==='function','missing_bucket:'+name);
19
20
  await ensureImportLeases(db);
20
21
  const context={db,buckets,config,migrations,onExport};
@@ -0,0 +1,45 @@
1
+ import { canonicalJson, utf8 } from './hash.mjs';
2
+
3
+ const table='rer_portability_object_inventory';
4
+ const chunkSize=50;
5
+ export async function ensureObjectInventory(db) {
6
+ await db.prepare(`CREATE TABLE IF NOT EXISTS ${table}(job TEXT NOT NULL REFERENCES rer_portability_jobs(id) ON DELETE CASCADE, chunk INTEGER NOT NULL, data TEXT NOT NULL, PRIMARY KEY(job,chunk))`).run();
7
+ }
8
+ // Keep large manifests out of D1's per-value and per-row limits. Only the
9
+ // storage representation changes; archive manifests still contain every object.
10
+ export function prepareInventory(row) {
11
+ if(utf8(canonicalJson(row.metadata.manifest.objects)).length>256*1024)
12
+ row.metadata.objectInventory=Math.ceil(row.metadata.manifest.objects.length/chunkSize);
13
+ }
14
+ export function metadataJson(metadata) {
15
+ if(!metadata.objectInventory)return canonicalJson(metadata);
16
+ const compact={...metadata,manifest:{...metadata.manifest,objects:[]}};
17
+ if(metadata.restoreManifest)compact.restoreManifest={...metadata.restoreManifest,objects:[]};
18
+ return canonicalJson(compact);
19
+ }
20
+ export function inventoryStatements(db,row,start=0,end=row.metadata.manifest.objects.length) {
21
+ if(!row.metadata.objectInventory)return [];
22
+ const objects=row.metadata.manifest.objects,statements=[];
23
+ for(let chunk=Math.floor(start/chunkSize);chunk<Math.ceil(end/chunkSize);chunk++)
24
+ statements.push(db.prepare(`INSERT OR REPLACE INTO ${table}(job,chunk,data) VALUES(?,?,?)`).bind(row.id,chunk,canonicalJson(objects.slice(chunk*chunkSize,(chunk+1)*chunkSize))));
25
+ return statements;
26
+ }
27
+ export async function loadInventory(db,row) {
28
+ if(!row.metadata.objectInventory)return row;
29
+ const objects=[];
30
+ // Read bounded pages: a complete inventory can exceed the RPC result limit.
31
+ for(let offset=0;offset<row.metadata.objectInventory;offset+=16) {
32
+ const {results}=await db.prepare(`SELECT data FROM ${table} WHERE job=? ORDER BY chunk LIMIT 16 OFFSET ?`).bind(row.id,offset).all();
33
+ for(const item of results)objects.push(...JSON.parse(item.data));
34
+ }
35
+ row.metadata.manifest.objects=objects;
36
+ if(row.metadata.restoreManifest)row.metadata.restoreManifest.objects=objects;
37
+ return row;
38
+ }
39
+ export function clearInventory(db,row) {
40
+ if(!row.metadata.objectInventory)return [];
41
+ delete row.metadata.objectInventory;
42
+ row.metadata.manifest.objects=[];
43
+ if(row.metadata.restoreManifest)row.metadata.restoreManifest.objects=[];
44
+ return [db.prepare(`DELETE FROM ${table} WHERE job=?`).bind(row.id)];
45
+ }
@@ -1,3 +1,4 @@
1
+ import { metadataJson, loadInventory, clearInventory } from './object-inventory.mjs';
1
2
  import { canonicalJson } from './hash.mjs';
2
3
  // Installed helpers and copied runtimes can coexist in one Worker. Share only
3
4
  // constructor-issued identities, so arbitrary objects with code/status fields
@@ -26,10 +27,10 @@ export async function state(db) {
26
27
  export async function job(db,id,kind) {
27
28
  const row=await statement(db,'SELECT * FROM rer_portability_jobs WHERE id=?',id).first();
28
29
  insist(row && (!kind || row.kind===kind),'job_not_found',404);
29
- return {...row, metadata:JSON.parse(row.metadata)};
30
+ return loadInventory(db,{...row, metadata:JSON.parse(row.metadata)});
30
31
  }
31
32
  export function save(db,row) {
32
- return statement(db,'UPDATE rer_portability_jobs SET phase=?,completed=?,metadata=? WHERE id=?',row.phase,row.completed,canonicalJson(row.metadata),row.id);
33
+ return statement(db,'UPDATE rer_portability_jobs SET phase=?,completed=?,metadata=? WHERE id=?',row.phase,row.completed,metadataJson(row.metadata),row.id);
33
34
  }
34
35
  export function partStatements(db,id,part) {
35
36
  return partQueries(id,part).map(({sql,params})=>statement(db,sql,...params));
@@ -64,7 +65,7 @@ export function dropCopyStatements(db,row) {
64
65
  return transferTables(row).map(name=>db.prepare(`DROP TABLE IF EXISTS ${quote(name)}`));
65
66
  }
66
67
  export function cleanCopyStatements(db,row) {
67
- return [...dropCopyStatements(db,row),statement(db,'DELETE FROM rer_portability_parts WHERE job=?',row.id),
68
+ return [...dropCopyStatements(db,row),...clearInventory(db,row),statement(db,'DELETE FROM rer_portability_parts WHERE job=?',row.id),
68
69
  statement(db,'DELETE FROM rer_portability_segments WHERE job=?',row.id)];
69
70
  }
70
71
  export async function cleanCopies(db,row) {
@@ -194,7 +194,7 @@ exclusive. For example:
194
194
  ```
195
195
 
196
196
  Every environment name must match `[A-Z][A-Z0-9_]*`. A nonempty `providers` map
197
- supports `sendgrid`, `resend`, `cloudflare` and `development` only. Every provider
197
+ supports `sendgrid`, `resend`, `cloudflare`, `development` and explicit `none`. Every provider
198
198
  entry declares `requiredEnvironment` (an empty array is allowed). Hosted entries
199
199
  also require `apiKeyEnvironment`; Cloudflare additionally requires
200
200
  `accountIdEnvironment`, checked as 32 hex characters. Those account declarations
@@ -204,6 +204,19 @@ configuration module remains responsible for its native sender and credential
204
204
  validation. A required mail loop still fails when foundation-only configuration
205
205
  is valid but mail has not been configured.
206
206
 
207
+ An optional mail loop may declare `"none": { "requiredEnvironment": [] }` in
208
+ its `binding.providers`. Only an explicit selector value of `none` turns that
209
+ loop off; its report says `optional-disabled` with `mailOff: true`, and the CLI
210
+ explains that no delivery is available and a declared provider must be configured
211
+ to enable it. A required loop selected to `none` still fails (`disabled-required`).
212
+ The off entry cannot declare credentials, sender requirements or development
213
+ settings, and cannot be a `whenUnset` default. The application's configuration
214
+ validator must honor the same deliberate off state. This inspection does not
215
+ send mail or verify a deployment; rer's deploy preflight independently checks
216
+ the mail-free sign-in declaration and rejects conflicting sending credentials.
217
+
218
+ — Codex, 2026-11-15T03:31:15Z
219
+
207
220
  Optional `whenUnset` declares a hosted provider and its inputs with the same
208
221
  shape. It applies only if the selector is absent (`undefined`), never for blank,
209
222
  unknown or non-string selectors. It cannot select development. Without
@@ -126,6 +126,8 @@ async function checkSummary(result, descriptor, root) {
126
126
  if (declaredInputs.size) lines.push(`Check the values you set for these declared input names: ${[...declaredInputs].sort().join(', ')}.`);
127
127
  }
128
128
  const blockedLoops = (result.loops ?? []).filter(loop => !['ready', 'optional-disabled'].includes(loop.status));
129
+ const mailOff = (result.loops ?? []).filter(loop => loop.mailOff);
130
+ if (mailOff.length) lines.push(`Outbound mail is off for ${mailOff.map(loop => loop.id).join(', ')}. No delivery is available; select and configure a declared mail provider to enable it.`);
129
131
  if (blockedLoops.length) lines.push(`Loops needing attention: ${blockedLoops.map(loop => `${loop.id} (${loop.status})`).join(', ')}.`);
130
132
  if (result.ok) lines.push('Remote deployment, provider access, and product acceptance remain unchecked.');
131
133
  else {
@@ -18,6 +18,11 @@ function fields(value, allowed) {
18
18
  }
19
19
 
20
20
  function mailInputs(provider, declaration, fallback = false) {
21
+ if (provider === 'none' && !fallback) {
22
+ fields(declaration, ['requiredEnvironment']);
23
+ if (!Array.isArray(declaration.requiredEnvironment) || declaration.requiredEnvironment.length) throw new Error('Mail off cannot require delivery inputs.');
24
+ return { provider, names: [] };
25
+ }
21
26
  fields(declaration, ['apiKeyEnvironment', 'requiredEnvironment', 'accountIdEnvironment', 'development', ...(fallback ? ['provider'] : [])]);
22
27
  if (![...hostedProviders, ...(fallback ? [] : ['development'])].includes(provider)
23
28
  || !Array.isArray(declaration.requiredEnvironment) || declaration.requiredEnvironment.some(name => !environmentName(name))) throw new Error('Invalid mail binding declaration.');
@@ -145,7 +150,9 @@ export function inspectRequiredLoops(descriptor, env) {
145
150
  const { providerEnvironment, providers, whenUnset } = loop.binding;
146
151
  const provider = env[providerEnvironment];
147
152
  const selected = provider === undefined ? whenUnset : typeof provider === 'string' && has(providers, provider) ? providers[provider] : undefined;
148
- result = selected ? inspectMailInputs(selected, env) : { status: 'invalid-mail-provider', missingEnvironment: [providerEnvironment] };
153
+ result = selected?.provider === 'none'
154
+ ? { status: loop.required ? 'disabled-required' : 'optional-disabled', missingEnvironment: [], mailOff: true }
155
+ : selected ? inspectMailInputs(selected, env) : { status: 'invalid-mail-provider', missingEnvironment: [providerEnvironment] };
149
156
  } else {
150
157
  result = inspectMailInputs(loop.selected, env);
151
158
  // Preserve the fixed-provider contract's truthy-selector mismatch behavior.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@runeditrun/rer",
3
- "version": "0.2.117",
3
+ "version": "0.2.119",
4
4
  "description": "The rer command: install, set up, run and launch Run Edit Run checkouts.",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -148,7 +148,11 @@ entry that does not prove, a 40-hex value, a key that names no tree file, or a h
148
148
  value with no file-name key keeps the native finding. Historical matches use only
149
149
  that commit's blobs and working-tree preimages reject linked paths. There is no
150
150
  path, filename, hash-value or repository exemption. `hit_location.py` resolves
151
- the native match's exact bytes for both proofs.
151
+ the native match's exact bytes for both proofs. Gitleaks directory fragments can
152
+ start mid-line without reporting their starting column. The resolver therefore
153
+ locates full matches within the reported LF-delimited lines. An unresolved match
154
+ retains its finding; repeated matches must all satisfy the structural and digest
155
+ proofs above, even if a reported column appears to identify just one of them.
152
156
 
153
157
  `trust_bundle.py` classifies an email match as public CA certificate metadata
154
158
  only when all of these hold: the file's path is an artifact listed by a committed
@@ -35,18 +35,22 @@ class ChecksumMaps:
35
35
  def matches(self, data, hit):
36
36
  if hit.get('RuleID') not in ('generic-api-key', 'hex-api-key'):
37
37
  return False
38
+ spans = match_spans(data, hit)
39
+ if not spans:
40
+ return False
38
41
  entries = list(ENTRY.finditer(data))
39
- for left, right in match_spans(data, hit):
42
+ for left, right in spans:
40
43
  keyed = [i for i, e in enumerate(entries)
41
44
  if left <= e.start('digest') and e.end('digest') <= right
42
45
  and e.group('digest').decode() == hit.get('Secret')]
43
46
  if len(keyed) != 1:
44
- continue
47
+ return False
45
48
  first = last = keyed[0]
46
49
  while first > 0 and SEPARATOR.fullmatch(data, entries[first - 1].end(), entries[first].start()):
47
50
  first -= 1
48
51
  while last + 1 < len(entries) and SEPARATOR.fullmatch(data, entries[last].end(), entries[last + 1].start()):
49
52
  last += 1
50
- return all(self.names_file((e.group('quoted') or e.group('bare')).decode('utf-8', 'replace'),
51
- e.group('digest').decode()) for e in entries[first:last + 1])
52
- return False
53
+ if not all(self.names_file((e.group('quoted') or e.group('bare')).decode('utf-8', 'replace'),
54
+ e.group('digest').decode()) for e in entries[first:last + 1]):
55
+ return False
56
+ return True
@@ -1,18 +1,29 @@
1
- """Resolve a native Gitleaks hit to the exact byte spans of its match."""
1
+ """Resolve a native Gitleaks hit to its possible full-match byte spans."""
2
2
 
3
3
 
4
4
  def match_spans(data, hit):
5
- lines = data.splitlines(keepends=True)
6
5
  try:
6
+ # Gitleaks counts LF bytes, not Python's broader set of line breaks.
7
+ lines = data.split(b'\n')
7
8
  first, last = hit['StartLine'] - 1, hit['EndLine'] - 1
8
9
  if not (0 <= first <= last < len(lines)):
9
10
  return []
10
- start = sum(map(len, lines[:first])) + hit['StartColumn'] - 1
11
- end = sum(map(len, lines[:last])) + hit['EndColumn']
12
- # Gitleaks 8.30.1 includes the preceding newline in columns
13
- # after fragment line one; Git fragments may begin mid-file.
14
- # Resolve that one-byte difference only against the exact match.
15
- return [(start - shift, end - shift) for shift in (0, 1)
16
- if data[start - shift:end - shift] == hit['Match'].encode('utf-8')]
11
+ match = hit['Match'].encode('utf-8')
12
+ if not match:
13
+ return []
14
+ left = sum(len(line) + 1 for line in lines[:first])
15
+ right = left + sum(len(line) + 1 for line in lines[first:last + 1])
16
+ # Directory fragments can start mid-line. Their columns are relative
17
+ # to that fragment, whose starting column is absent from the report.
18
+ # Later fragment lines also count the preceding LF in their columns.
19
+ # Bind the full match within the reported lines instead. Even an
20
+ # apparently exact column cannot disambiguate repeated occurrences.
21
+ # Callers must prove every candidate, never just the first one.
22
+ spans = []
23
+ start = data.find(match, left, right)
24
+ while start >= 0:
25
+ spans.append((start, start + len(match)))
26
+ start = data.find(match, start + 1, right)
27
+ return spans
17
28
  except (KeyError, TypeError, IndexError, UnicodeError):
18
29
  return []
@@ -80,19 +80,20 @@ class Document:
80
80
  secret = hit.get('Secret', '').encode('utf-8')
81
81
  if not secret:
82
82
  return False
83
- located = set()
84
- for left, right in match_spans(self.data, hit):
83
+ spans = match_spans(self.data, hit)
84
+ if not spans:
85
+ return False
86
+ allowed = [self.spans[pointer] for pointer in pointers if pointer in self.spans]
87
+ for left, right in spans:
85
88
  start = self.data.find(secret, left, right)
86
89
  if start < 0 or self.data.find(secret, start + 1, right) >= 0:
87
- continue
88
- for pointer in pointers:
89
- span = self.spans.get(pointer)
90
- if span:
91
- a, b, value = span
92
- # Native fragments can truncate a digest capture. Bind the
93
- # captured secret's bytes, including a partial capture, to
94
- # this exact unescaped JSON value.
95
- if (a + 1 <= start and start + len(secret) <= b - 1
96
- and self.data[a + 1:b - 1] == value.encode('ascii')):
97
- located.add((a, b))
98
- return len(located) == 1
90
+ return False
91
+ # Location alone is not proof. Every possible native occurrence
92
+ # must bind its captured bytes (even a truncated capture) to an
93
+ # exact unescaped JSON value at a proved pointer. An identical
94
+ # credential elsewhere on the line therefore retains the finding.
95
+ if not any(a + 1 <= start and start + len(secret) <= b - 1
96
+ and self.data[a + 1:b - 1] == value.encode('ascii')
97
+ for a, b, value in allowed):
98
+ return False
99
+ return True