sproutboat 0.27.0 → 0.29.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/CHANGELOG.md +44 -0
  2. package/SURFACE.md +4 -4
  3. package/THIRD_PARTY_NOTICES.md +9 -0
  4. package/package.json +8 -8
  5. package/src/build.ts +12 -2
  6. package/src/compile.ts +23 -1
  7. package/src/main.ts +13 -2
  8. package/src/native-restore-sources.ts +48 -0
  9. package/src/native-restore.ts +155 -0
  10. package/src/surface.ts +3 -3
  11. package/vendor/ltx-restore/LICENSE +202 -0
  12. package/vendor/ltx-restore/NOTICE +32 -0
  13. package/vendor/ltx-restore/crc64-native.ts.txt +7 -0
  14. package/vendor/ltx-restore/errors.ts.txt +74 -0
  15. package/vendor/ltx-restore/file-io.ts.txt +54 -0
  16. package/vendor/ltx-restore/ltx/byte-reader.ts.txt +46 -0
  17. package/vendor/ltx-restore/ltx/bytes.ts.txt +87 -0
  18. package/vendor/ltx-restore/ltx/checksum.ts.txt +46 -0
  19. package/vendor/ltx-restore/ltx/constants.ts.txt +71 -0
  20. package/vendor/ltx-restore/ltx/crc64.ts.txt +73 -0
  21. package/vendor/ltx-restore/ltx/decoder.ts.txt +285 -0
  22. package/vendor/ltx-restore/ltx/header.ts.txt +126 -0
  23. package/vendor/ltx-restore/ltx/lz4.ts.txt +260 -0
  24. package/vendor/ltx-restore/ltx/page-index.ts.txt +140 -0
  25. package/vendor/ltx-restore/ltx/page.ts.txt +80 -0
  26. package/vendor/ltx-restore/ltx/sink.ts.txt +33 -0
  27. package/vendor/ltx-restore/ltx/trailer.ts.txt +58 -0
  28. package/vendor/ltx-restore/ltx/types.ts.txt +152 -0
  29. package/vendor/ltx-restore/lz4js/README.md +39 -0
  30. package/vendor/ltx-restore/lz4js/lz4.js.txt +560 -0
  31. package/vendor/ltx-restore/lz4js/package.json +36 -0
  32. package/vendor/ltx-restore/lz4js/util.js.txt +54 -0
  33. package/vendor/ltx-restore/lz4js/xxh32.js.txt +102 -0
  34. package/vendor/ltx-restore/program.ts.txt +3 -0
  35. package/vendor/ltx-restore/restore-native.ts.txt +120 -0
  36. package/vendor/ltx-restore/restore.ts.txt +96 -0
package/CHANGELOG.md CHANGED
@@ -10,6 +10,50 @@ maintained going forward by the `release` skill.
10
10
 
11
11
  ## Unreleased
12
12
 
13
+ ## [0.29.0] - 2026-10-23
14
+
15
+ Built on `@sproutboat/runtime` 0.28.0, `@sproutboat/toolchain` 0.15.0 and
16
+ `@sproutboat/wire` 0.17.0.
17
+
18
+ ### Added
19
+
20
+ - Offline restore of a captured D1 database, inside the app's own executable.
21
+ `sproutboat build --standalone --with-restore` adds an `ltx-restore` command:
22
+ `./my-app ltx-restore --generation DIR --output NEW_DB [--txid N]` rebuilds a
23
+ SQLite database from one generation of LTX files written by native capture.
24
+ The target needs no Node, Bun or SQLite CLI. The restore checks every
25
+ checksum and TXID in the chain, then runs SQLite's integrity check on the
26
+ result. It refuses to run while capture is still writing, and never
27
+ overwrites an existing file or touches the destination's `-wal`/`-shm`
28
+ sidecars. On success it prints the TXID, checksum and size as JSON. See
29
+ `docs/native-ltx-restore.md`.
30
+ - The restore code is compiled separately from the app and runs before any app
31
+ code starts. Builds without `--with-restore` are unchanged, and there an
32
+ `ltx-restore` argument does nothing.
33
+
34
+ ## [0.28.0] - 2026-10-23
35
+
36
+ Built on `@sproutboat/runtime` 0.28.0, `@sproutboat/toolchain` 0.14.0 and
37
+ `@sproutboat/wire` 0.17.0.
38
+
39
+ ### Added
40
+
41
+ - Request bodies stream from compatibility date `2026-10-27`, as in Workers.
42
+ The handler runs as soon as a request's headers arrive, and `request.body`
43
+ hands out the bytes as they come in. The server pauses the connection while
44
+ more than 1 MiB waits unread, so a 128 MiB upload read through the stream
45
+ peaks at 38 MiB. `text()`, `json()`, `arrayBuffer()` and `bytes()` still read
46
+ the whole body (as Promises). New example: `examples/upload`.
47
+ - From the same date `SB_REQUEST_BODY_MAX` defaults to 100 MiB (it stays 1 MiB
48
+ before it, where the body is held whole). A larger `Content-Length` gets
49
+ `413 Payload Too Large` before the handler runs.
50
+ - On the platform, `fetch()` and service bindings send a `ReadableStream`
51
+ request body as it is read, instead of reading it to the end first. A
52
+ streamed body can't follow a 307 or 308 redirect, as in Workers.
53
+
54
+ Earlier dates are unchanged: the handler gets the whole body, and
55
+ `request.text()` answers at once.
56
+
13
57
  ## [0.27.0] - 2026-10-22
14
58
 
15
59
  Built on `@sproutboat/runtime` 0.27.1.
package/SURFACE.md CHANGED
@@ -3,7 +3,7 @@
3
3
  > Generated by `src/surface.test.ts` from `src/surface.ts` + the pinned
4
4
  > toolchain constants. Do not edit by hand — run `UPDATE_SURFACE=1 bun test`.
5
5
 
6
- **Package:** `sproutboat` 0.27.0
6
+ **Package:** `sproutboat` 0.29.0
7
7
 
8
8
  ## Commands
9
9
 
@@ -13,7 +13,7 @@
13
13
  | `check` | `[project-dir]` | Validate the config and entry point without building. |
14
14
  | `types` | `[project-dir]` | Write sproutboat-env.d.ts from the config, so an editor knows what `env` holds. `dev` and `deploy` refresh it when the config is newer. |
15
15
  | `dev` | `[project-dir] [--port <n>] [--no-watch]` | Run the project on this machine against a real broker, rebuilding on save. |
16
- | `build` | `[project-dir] [--target host] [--standalone] [--emit-only]` | Cross-compile the native-fetch sprout (Porffor + Zig). `--target host` builds for this machine instead, to run locally — not deployable. `--standalone` emits one executable carrying its own bindings, with SQLite compiled in (~2 MB) and no broker process. `--emit-only` stops after writing `.sproutboat/porffor-units/<shape>/sprout.generated.js`, the module Porffor would compile, and prints its path: every check still runs, nothing is compiled. |
16
+ | `build` | `[project-dir] [--target host] [--standalone] [--with-restore] [--emit-only]` | Cross-compile the native-fetch sprout (Porffor + Zig). `--target host` builds for this machine instead, to run locally, not deployable. `--standalone` emits one executable carrying its own bindings, with SQLite compiled in (~2 MB) and no broker process. `--with-restore` adds an offline `APP ltx-restore --generation DIR --output NEW_DB [--txid N]` startup command to a standalone executable; it runs before app initialization. `--emit-only` stops after writing `.sproutboat/porffor-units/<shape>/sprout.generated.js`, the module Porffor would compile, and prints its path: every check still runs, nothing is compiled. |
17
17
  | `toolchain` | `doctor [--json]` | Report the selected compiler inputs, cache locations, overrides, and host prerequisites without downloading or compiling anything. |
18
18
  | `deploy` | `[project-dir] [--dry-run] [--artifact <dir>] [--no-wait] [--no-provision]` | Build (unless --artifact), auto-provision id-less storage bindings and pin their ids into sproutboat.jsonc, print the report, upload, wait until the URL serves. The control plane skips an upload that matches the live artifact byte-for-byte. --dry-run stops before upload; --no-wait skips the health check; --no-provision leaves id-less bindings as ephemeral deploy-scoped stores. |
19
19
  | `versions` | `<list | view <version-id>> [project-dir]` | List the project's deployed versions, or show one version's artifact and bindings. |
@@ -32,7 +32,7 @@
32
32
  | `complete` | `[bash | zsh | fish]` | Print a shell completion script for the top-level commands (defaults to $SHELL). |
33
33
 
34
34
  ```
35
- usage: sproutboat <init [name] | check [project-dir] | types [project-dir] | dev [project-dir] [--port <n>] [--no-watch] | build [project-dir] [--target host] [--standalone] [--emit-only] | toolchain doctor [--json] | deploy [project-dir] [--dry-run] [--artifact <dir>] [--no-wait] [--no-provision] | versions <list | view <version-id>> [project-dir] | rollback <version-id> [project-dir] | tail [project-dir] [--sprout] | kv <list | create <name> | info <name> | rename <name> <new> | delete <name>> | <key | bulk | export> ... | d1 <list | create <name> | info <name> | rename <name> <new> | delete <name>> | r2 <list | create <name> | info <name> | rename <name> <new> | delete <name>> | queues <list | create <name> | info <name> | rename <name> <new> | delete <name>> | domains <list | add <host> | verify <host> | delete <host>> [project-dir] | secrets <list | put <NAME> [--value <value>] | delete <NAME>> [project-dir] | delete [project-dir] [--name <project>] --yes | login [--api-url <url>] [--token <token>] | logout [--api-url <url>] | whoami | complete [bash | zsh | fish]>
35
+ usage: sproutboat <init [name] | check [project-dir] | types [project-dir] | dev [project-dir] [--port <n>] [--no-watch] | build [project-dir] [--target host] [--standalone] [--with-restore] [--emit-only] | toolchain doctor [--json] | deploy [project-dir] [--dry-run] [--artifact <dir>] [--no-wait] [--no-provision] | versions <list | view <version-id>> [project-dir] | rollback <version-id> [project-dir] | tail [project-dir] [--sprout] | kv <list | create <name> | info <name> | rename <name> <new> | delete <name>> | <key | bulk | export> ... | d1 <list | create <name> | info <name> | rename <name> <new> | delete <name>> | r2 <list | create <name> | info <name> | rename <name> <new> | delete <name>> | queues <list | create <name> | info <name> | rename <name> <new> | delete <name>> | domains <list | add <host> | verify <host> | delete <host>> [project-dir] | secrets <list | put <NAME> [--value <value>] | delete <NAME>> [project-dir] | delete [project-dir] [--name <project>] --yes | login [--api-url <url>] [--token <token>] | logout [--api-url <url>] | whoami | complete [bash | zsh | fish]>
36
36
  ```
37
37
 
38
38
  ## Environment variables
@@ -67,7 +67,7 @@ usage: sproutboat <init [name] | check [project-dir] | types [project-dir] | dev
67
67
  | `SPROUTBOAT_DATA` | Data directory for a standalone binary, after --data and SB_DATA_DIR, before the ./<name>.data default (#15). |
68
68
  | `SB_CA_BUNDLE` | PEM bundle of extra certificate authorities a standalone binary should trust, on top of the compiled-in Mozilla root set (#15). Adds trust; nothing disables verification. |
69
69
  | `SB_FETCH_MAX_BYTES` | Largest outbound `fetch()` response body a sprout will read, in bytes (default 33554432). The size is the remote host's choice and the body is held whole, so this is the ceiling that stops one upstream exhausting memory. |
70
- | `SB_REQUEST_BODY_MAX` | Largest inbound request body the runtime accepts, in bytes (default 1048576). Anything larger is refused with 413 before the handler runs. |
70
+ | `SB_REQUEST_BODY_MAX` | Largest inbound request body the runtime accepts, in bytes. From compatibility date 2026-10-27 bodies stream and the default is 104857600 (100 MiB); before it the body is held whole and the default is 1048576. A larger Content-Length is refused with 413 before the handler runs; a chunked body that grows past it closes the connection. |
71
71
  | `SB_EXTRA_CFLAGS` | Flags added to Porffor's native-fetch compile step, set by the build so inline C can include BearSSL's header (#15). |
72
72
  | `SB_EXTRA_LINK` | Objects to add to Porffor's native-fetch link line, set by the build so a standalone binary links SQLite and BearSSL (#15). |
73
73
 
@@ -25,3 +25,12 @@ to its upstream source.
25
25
  The complete Apache-2.0 notice for uWebSockets and uSockets is retained in the
26
26
  embedded source archive. Porffor and BearSSL retain their MIT notices in the
27
27
  source archives acquired at the revisions above.
28
+
29
+ ## Optional native LTX restore
30
+
31
+ The trusted restore codec is adapted from `platformatic/sqlite-js-stream` at
32
+ `a5b830d5f1e408c54a04285c2199019b40154c25` (Apache-2.0), with streaming/checksum
33
+ and native integration changes by OpenAI Codex (GPT-6.1). See
34
+ `vendor/ltx-restore/LICENSE` and `NOTICE` for Platformatic, Litestream and LTX
35
+ attribution. LZ4JS 0.2.0 declares the ISC license in its retained package metadata under
36
+ `vendor/ltx-restore/lz4js/`.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "sproutboat",
3
- "version": "0.27.0",
3
+ "version": "0.29.0",
4
4
  "description": "Wrangler-shaped CLI for Sproutboat. Deploys workers to any control plane via --api-url / SPROUTBOAT_API_URL.",
5
5
  "keywords": [
6
6
  "cli",
@@ -64,9 +64,9 @@
64
64
  "@sproutboat/artifact": "^0.3.0",
65
65
  "@sproutboat/assets": "^0.3.0",
66
66
  "@sproutboat/config": "^0.5.0",
67
- "@sproutboat/runtime": "^0.27.1",
68
- "@sproutboat/toolchain": "^0.13.0",
69
- "@sproutboat/wire": "^0.16.0"
67
+ "@sproutboat/runtime": "^0.28.0",
68
+ "@sproutboat/toolchain": "^0.15.0",
69
+ "@sproutboat/wire": "^0.17.0"
70
70
  },
71
71
  "devDependencies": {
72
72
  "@oxlint/plugins": "1.85.0",
@@ -78,10 +78,10 @@
78
78
  "typescript": "7.0.2"
79
79
  },
80
80
  "optionalDependencies": {
81
- "@sproutboat/cli-darwin-arm64": "0.27.0",
82
- "@sproutboat/cli-darwin-x64": "0.27.0",
83
- "@sproutboat/cli-linux-arm64": "0.27.0",
84
- "@sproutboat/cli-linux-x64": "0.27.0"
81
+ "@sproutboat/cli-darwin-arm64": "0.29.0",
82
+ "@sproutboat/cli-darwin-x64": "0.29.0",
83
+ "@sproutboat/cli-linux-arm64": "0.29.0",
84
+ "@sproutboat/cli-linux-x64": "0.29.0"
85
85
  },
86
86
  "engines": {
87
87
  "node": ">=18"
package/src/build.ts CHANGED
@@ -1,4 +1,5 @@
1
1
  import { createHash, randomUUID } from "node:crypto";
2
+ import { NATIVE_RESTORE_IDENTITY } from "./native-restore";
2
3
  import { cp, mkdir, readFile, readdir, rename, rm, stat, writeFile } from "node:fs/promises";
3
4
  import { dirname, resolve } from "node:path";
4
5
  import { readAssetRules, walkAssets, type AssetManifest } from "./assets";
@@ -54,6 +55,8 @@ export type BuildInput = {
54
55
  outputDirectory?: string;
55
56
  /** Stable compiler input used only by the local dev loop. */
56
57
  generatedPath?: string;
58
+ /** Include trusted offline restore startup in a standalone executable. */
59
+ withRestore?: boolean;
57
60
  };
58
61
 
59
62
  export type BuildOutput = {
@@ -79,7 +82,7 @@ async function integrationIdentity(): Promise<string> {
79
82
  const executable = await stat(process.execPath);
80
83
  return `release:${cliVersion}:${executable.size}:${executable.mtimeMs}`;
81
84
  }
82
- const files = ["build.ts", "compile.ts", "sqlite.ts", "bearssl.ts", "toolchain.ts"].map((file) =>
85
+ const files = ["build.ts", "compile.ts", "sqlite.ts", "bearssl.ts", "toolchain.ts", "native-restore.ts"].map((file) =>
83
86
  resolve(import.meta.dir, file),
84
87
  );
85
88
  // Hash the patch implementation and its local modules so a Unicode patch
@@ -118,6 +121,10 @@ async function acquireArtifactLock(artifactDir: string): Promise<string> {
118
121
  * exactly where `buildArtifact` would.
119
122
  */
120
123
  async function prepareBuild(input: BuildInput) {
124
+ if (input.withRestore && input.reuseSproutPath)
125
+ throw new Error("--with-restore cannot reuse a previously compiled sprout");
126
+ if (input.withRestore && input.transport !== "embedded")
127
+ throw new Error("--with-restore is available only with --standalone");
121
128
  const source = input.source ?? (await readFile(input.sourcePath));
122
129
  const sourceHash = digest(source);
123
130
  const target = input.target ?? "linux-x86_64";
@@ -247,7 +254,7 @@ function unitDirFor(input: BuildInput): string {
247
254
  return resolve(
248
255
  input.projectDir,
249
256
  ".sproutboat/porffor-units",
250
- `${input.target ?? "linux-x86_64"}-${input.transport ?? "broker"}-${input.optimize ?? "release"}`,
257
+ `${input.target ?? "linux-x86_64"}-${input.transport ?? "broker"}-${input.optimize ?? "release"}${input.withRestore ? "-restore" : ""}`,
251
258
  );
252
259
  }
253
260
 
@@ -280,6 +287,7 @@ export async function emitGenerated(input: BuildInput): Promise<{ generatedPath:
280
287
  optimize: input.optimize,
281
288
  versionMetadata,
282
289
  emitOnly: true,
290
+ withRestore: input.withRestore,
283
291
  });
284
292
  } finally {
285
293
  if (unitLock) await rm(unitLock, { recursive: true, force: true });
@@ -318,6 +326,7 @@ export async function buildArtifact(input: BuildInput): Promise<BuildOutput> {
318
326
  porffor: [PORFFOR_COMMIT_FULL, PORFFOR_ARCHIVE_SHA256],
319
327
  native: embedded ? [sqliteStamp(), bearsslStamp()] : [],
320
328
  integration: await integrationIdentity(),
329
+ nativeRestore: input.withRestore ? NATIVE_RESTORE_IDENTITY : null,
321
330
  }),
322
331
  ).slice("sha256:".length);
323
332
  const artifactId = versionMetadata
@@ -356,6 +365,7 @@ export async function buildArtifact(input: BuildInput): Promise<BuildOutput> {
356
365
  extraCflags: tls ? ["-I", tls.includeDir] : [],
357
366
  optimize: input.optimize,
358
367
  versionMetadata,
368
+ withRestore: input.withRestore,
359
369
  });
360
370
  } finally {
361
371
  if (unitLock) await rm(unitLock, { recursive: true, force: true });
package/src/compile.ts CHANGED
@@ -14,6 +14,7 @@ import { chmod, mkdir, readdir, readFile, rm, stat, writeFile } from "node:fs/pr
14
14
  import { homedir } from "node:os";
15
15
  import { dirname, resolve } from "node:path";
16
16
  import { ensurePorfforPatched } from "./patch-porffor";
17
+ import { buildNativeRestore } from "./native-restore";
17
18
  import { ensureUWebSockets, ensureUWebSocketsHost, resolveHostCompiler, UwsUnavailableError } from "./toolchain";
18
19
  import { ensurePorffor } from "./porffor-toolchain";
19
20
  // The prelude + transports live in @sproutboat/runtime (next to wrap.ts, which
@@ -100,6 +101,8 @@ export type CompileInput = {
100
101
  * binary, such as a qualification run that only inspects the source.
101
102
  */
102
103
  emitOnly?: boolean;
104
+ /** Trusted offline restore startup, standalone builds only. Never a tenant compiler flag. */
105
+ withRestore?: boolean;
103
106
  };
104
107
 
105
108
  /**
@@ -242,6 +245,8 @@ async function writeGenerated(input: CompileInput): Promise<string> {
242
245
  * the wrapper, never Porffor itself.
243
246
  */
244
247
  export async function compileSprout(input: CompileInput): Promise<void> {
248
+ if (input.withRestore && (input.transport !== "embedded" || input.emitOnly))
249
+ throw new Error("--with-restore requires a standalone native compile, without --emit-only");
245
250
  if (input.emitOnly) {
246
251
  await writeGenerated(input);
247
252
  return;
@@ -298,13 +303,30 @@ export async function compileSprout(input: CompileInput): Promise<void> {
298
303
  ? [process.execPath, "__porffor", launcher]
299
304
  : [process.execPath, resolve(import.meta.dir, "main.ts"), "__porffor", launcher];
300
305
 
306
+ const restoreObjects = input.withRestore
307
+ ? await buildNativeRestore({
308
+ directory: dirname(generatedPath),
309
+ prelude: await loadPrelude("embedded"),
310
+ porfforCommand: command,
311
+ cc: hostCompiler?.cc ?? [input.zigBin!, "cc", "-target", "x86_64-linux-musl"],
312
+ cxx: hostCompiler?.cxx ?? [input.zigBin!, "c++", "-target", "x86_64-linux-musl"],
313
+ cflags: input.extraCflags ?? [],
314
+ macho: input.target === "host" && process.platform === "darwin",
315
+ })
316
+ : [];
317
+
301
318
  const child = Bun.spawn(
302
319
  [...command, ...porfforArgs(generatedPath, input.outPath, input.target, input.optimize, hostCompiler?.usingZig)],
303
320
  {
304
321
  cwd: outDir,
305
322
  stdout: "pipe",
306
323
  stderr: "pipe",
307
- env: compileEnv(path, input.extraLink, input.extraCflags, hostCompiler),
324
+ env: compileEnv(
325
+ path,
326
+ [...(input.extraLink ?? []), ...restoreObjects],
327
+ [...(input.extraCflags ?? []), ...(input.withRestore ? ["-Dmain=sb_app_main"] : [])],
328
+ hostCompiler,
329
+ ),
308
330
  },
309
331
  );
310
332
  let timedOut = false;
package/src/main.ts CHANGED
@@ -335,7 +335,11 @@ async function build(directory?: string, target: "linux-x86_64" | "host" = "linu
335
335
  * *this* machine either way in phase 0 — cross-compiling both halves comes with
336
336
  * the embedded backend, which removes the launcher entirely.
337
337
  */
338
- async function buildStandaloneBinary(directory: string | undefined, target: "linux-x86_64" | "host") {
338
+ async function buildStandaloneBinary(
339
+ directory: string | undefined,
340
+ target: "linux-x86_64" | "host",
341
+ withRestore = false,
342
+ ) {
339
343
  const project = await readProject(directory);
340
344
  console.log(dim(`Building a standalone ${project.config.name} (${target})…`));
341
345
  try {
@@ -345,6 +349,7 @@ async function buildStandaloneBinary(directory: string | undefined, target: "lin
345
349
  sourcePath: project.sourcePath,
346
350
  source: project.bundle.code,
347
351
  target,
352
+ withRestore,
348
353
  });
349
354
  console.log(ok(`built ${result.outPath} (${(result.bytes / 1_000_000).toFixed(1)} MB)`));
350
355
  console.log(dim(` run it: ${result.outPath} · state lands in ./${project.config.name}.data`));
@@ -1339,11 +1344,17 @@ switch (command) {
1339
1344
  await dev(args);
1340
1345
  break;
1341
1346
  case "build": {
1347
+ const withRestore = args.includes("--with-restore");
1348
+ if (withRestore && (!args.includes("--standalone") || args.includes("--emit-only")))
1349
+ usageError(
1350
+ "--with-restore requires --standalone without --emit-only",
1351
+ "build --standalone --with-restore [--target host] [project-dir]",
1352
+ );
1342
1353
  const hostBuild = args.includes("--target") && args[args.indexOf("--target") + 1] === "host";
1343
1354
  const directory = args.find((arg) => !arg.startsWith("--") && arg !== "host");
1344
1355
  const target = hostBuild ? "host" : "linux-x86_64";
1345
1356
  if (args.includes("--emit-only")) await emitOnly(directory, target, args.includes("--standalone"));
1346
- else if (args.includes("--standalone")) await buildStandaloneBinary(directory, target);
1357
+ else if (args.includes("--standalone")) await buildStandaloneBinary(directory, target, withRestore);
1347
1358
  else await build(directory, target);
1348
1359
  break;
1349
1360
  }
@@ -0,0 +1,48 @@
1
+ // Text assets are embedded in the packaged CLI and compiled only as trusted restore source.
2
+ import source0 from "../vendor/ltx-restore/crc64-native.ts.txt" with { type: "text" };
3
+ import source1 from "../vendor/ltx-restore/errors.ts.txt" with { type: "text" };
4
+ import source2 from "../vendor/ltx-restore/file-io.ts.txt" with { type: "text" };
5
+ import source3 from "../vendor/ltx-restore/ltx/byte-reader.ts.txt" with { type: "text" };
6
+ import source4 from "../vendor/ltx-restore/ltx/bytes.ts.txt" with { type: "text" };
7
+ import source5 from "../vendor/ltx-restore/ltx/checksum.ts.txt" with { type: "text" };
8
+ import source6 from "../vendor/ltx-restore/ltx/constants.ts.txt" with { type: "text" };
9
+ import source7 from "../vendor/ltx-restore/ltx/crc64.ts.txt" with { type: "text" };
10
+ import source8 from "../vendor/ltx-restore/ltx/decoder.ts.txt" with { type: "text" };
11
+ import source9 from "../vendor/ltx-restore/ltx/header.ts.txt" with { type: "text" };
12
+ import source10 from "../vendor/ltx-restore/ltx/lz4.ts.txt" with { type: "text" };
13
+ import source11 from "../vendor/ltx-restore/ltx/page-index.ts.txt" with { type: "text" };
14
+ import source12 from "../vendor/ltx-restore/ltx/page.ts.txt" with { type: "text" };
15
+ import source13 from "../vendor/ltx-restore/ltx/sink.ts.txt" with { type: "text" };
16
+ import source14 from "../vendor/ltx-restore/ltx/trailer.ts.txt" with { type: "text" };
17
+ import source15 from "../vendor/ltx-restore/ltx/types.ts.txt" with { type: "text" };
18
+ import source16 from "../vendor/ltx-restore/lz4js/lz4.js.txt" with { type: "text" };
19
+ import source17 from "../vendor/ltx-restore/lz4js/util.js.txt" with { type: "text" };
20
+ import source18 from "../vendor/ltx-restore/lz4js/xxh32.js.txt" with { type: "text" };
21
+ import source19 from "../vendor/ltx-restore/program.ts.txt" with { type: "text" };
22
+ import source20 from "../vendor/ltx-restore/restore-native.ts.txt" with { type: "text" };
23
+ import source21 from "../vendor/ltx-restore/restore.ts.txt" with { type: "text" };
24
+
25
+ export const NATIVE_RESTORE_SOURCES = {
26
+ "crc64-native.ts": source0,
27
+ "errors.ts": source1,
28
+ "file-io.ts": source2,
29
+ "ltx/byte-reader.ts": source3,
30
+ "ltx/bytes.ts": source4,
31
+ "ltx/checksum.ts": source5,
32
+ "ltx/constants.ts": source6,
33
+ "ltx/crc64.ts": source7,
34
+ "ltx/decoder.ts": source8,
35
+ "ltx/header.ts": source9,
36
+ "ltx/lz4.ts": source10,
37
+ "ltx/page-index.ts": source11,
38
+ "ltx/page.ts": source12,
39
+ "ltx/sink.ts": source13,
40
+ "ltx/trailer.ts": source14,
41
+ "ltx/types.ts": source15,
42
+ "lz4js/lz4.js": source16,
43
+ "lz4js/util.js": source17,
44
+ "lz4js/xxh32.js": source18,
45
+ "program.ts": source19,
46
+ "restore-native.ts": source20,
47
+ "restore.ts": source21,
48
+ } satisfies Record<string, string>;
@@ -0,0 +1,155 @@
1
+ import { createHash } from "node:crypto";
2
+ import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
3
+ import { dirname, resolve } from "node:path";
4
+ import { NATIVE_RESTORE_SOURCES } from "./native-restore-sources";
5
+
6
+ /** Included in the native artifact cache key, including in the packaged CLI. */
7
+ export const NATIVE_RESTORE_IDENTITY = createHash("sha256")
8
+ .update(JSON.stringify(NATIVE_RESTORE_SOURCES))
9
+ .digest("hex");
10
+
11
+ const CONTROLLER_HEAD = String.raw`
12
+ static int sb_cli_restore_stdout = -1;
13
+ `;
14
+ const DISPATCH = String.raw`
15
+ int sb_cli_restore_dispatch(int argc, char **argv) {
16
+ if (argc < 2 || strcmp(argv[1], "ltx-restore")) return -1;
17
+ const char *source = NULL, *output = NULL, *target = "";
18
+ const char *usage = "usage: APP ltx-restore --generation DIR --output NEW_DB [--txid DECIMAL_TXID]\n";
19
+ if (argc == 3 && (!strcmp(argv[2], "--help") || !strcmp(argv[2], "-h"))) { fputs(usage, stdout); return 0; }
20
+ for (int i = 2; i < argc; i += 2) {
21
+ if (i + 1 >= argc || !argv[i + 1][0]) { fputs(usage, stderr); return 2; }
22
+ if (!strcmp(argv[i], "--generation") && !source) source = argv[i + 1];
23
+ else if (!strcmp(argv[i], "--output") && !output) output = argv[i + 1];
24
+ else if (!strcmp(argv[i], "--txid") && !target[0]) target = argv[i + 1];
25
+ else { fputs(usage, stderr); return 2; }
26
+ }
27
+ if (!source || !output || strlen(source) > 4095 || strlen(output) > 4095) { fputs(usage, stderr); return 2; }
28
+ if (target[0]) {
29
+ uint64_t value = 0;
30
+ for (const unsigned char *p = (const unsigned char *)target; *p; p++) {
31
+ unsigned int digit = (unsigned int)(*p - '0');
32
+ if (digit > 9 || value > (UINT64_MAX - digit) / 10) { fputs(usage, stderr); return 2; }
33
+ value = value * 10 + digit;
34
+ }
35
+ if (!value) { fputs(usage, stderr); return 2; }
36
+ }
37
+ if (setenv("SB_LTX_RESTORE_GENERATION", source, 1) || setenv("SB_LTX_RESTORE_OUTPUT", output, 1) ||
38
+ setenv("SB_LTX_RESTORE_TXID", target, 1)) { perror("restore arguments"); return 1; }
39
+ sb_cli_restore_stdout = dup(STDOUT_FILENO);
40
+ if (sb_cli_restore_stdout < 0) { perror("restore stdout"); return 1; }
41
+ porf_native_fetch_runtime_init();
42
+ fputs("restore startup returned without completing\n", stderr);
43
+ return 1;
44
+ }
45
+ `;
46
+
47
+ const STARTUP = String.raw`
48
+ extern "C" int sb_cli_restore_dispatch(int, char **);
49
+ int sb_app_main(void);
50
+ int main(int argc, char **argv) {
51
+ const int result = sb_cli_restore_dispatch(argc, argv);
52
+ return result >= 0 ? result : sb_app_main();
53
+ }
54
+ `;
55
+
56
+ async function run(command: string[]): Promise<string> {
57
+ const child = Bun.spawn(command, { stdout: "pipe", stderr: "pipe" });
58
+ const timer = setTimeout(() => child.kill("SIGKILL"), 600_000);
59
+ try {
60
+ const [code, stdout, stderr] = await Promise.all([
61
+ child.exited,
62
+ new Response(child.stdout).text(),
63
+ new Response(child.stderr).text(),
64
+ ]);
65
+ if (code !== 0) throw new Error(`native restore build failed (${command[0]}): ${stderr || stdout}`);
66
+ return stdout;
67
+ } finally {
68
+ clearTimeout(timer);
69
+ }
70
+ }
71
+
72
+ /** Prefix only defined external symbols. Undefined libc/SQLite/server imports stay external. */
73
+ export function restoreSymbolDefines(symbols: string, macho: boolean): string {
74
+ const names = new Set<string>();
75
+ for (const line of symbols.split("\n")) {
76
+ const match = /^\s*[0-9a-fA-F]+\s+[A-Za-z]\s+([A-Za-z_][A-Za-z_0-9]*)\s*$/.exec(line);
77
+ if (!match) continue;
78
+ const name = macho ? match[1].slice(1) : match[1];
79
+ if (name !== "sb_cli_restore_dispatch") names.add(name);
80
+ }
81
+ if (!names.has("porf_mem") || !names.has("porf_native_fetch_runtime_init"))
82
+ throw new Error("native restore compiler symbol inventory changed");
83
+ return [...names]
84
+ .sort()
85
+ .map((name) => `#define ${name} sb_ltx_trusted_${name}\n`)
86
+ .join("");
87
+ }
88
+
89
+ /** Trusted source only. No project source, bindings, configuration or secrets enter this compile. */
90
+ export async function buildNativeRestore(input: {
91
+ directory: string;
92
+ prelude: string;
93
+ porfforCommand: string[];
94
+ cc: string[];
95
+ cxx: string[];
96
+ cflags: string[];
97
+ macho: boolean;
98
+ }): Promise<string[]> {
99
+ const root = resolve(input.directory, "native-restore");
100
+ await mkdir(root, { recursive: true });
101
+ for (const [name, source] of Object.entries(NATIVE_RESTORE_SOURCES)) {
102
+ const file = resolve(root, name);
103
+ await mkdir(dirname(file), { recursive: true });
104
+ await writeFile(file, source);
105
+ }
106
+ const bundled = await Bun.build({ entrypoints: [resolve(root, "program.ts")], target: "browser", format: "esm" });
107
+ if (!bundled.success) throw new Error(`cannot bundle trusted restore: ${bundled.logs.join("\n")}`);
108
+ // Full released transport supplies the existing CRC kernel and SQLite declarations.
109
+ const generated = resolve(root, "trusted.generated.js");
110
+ const { wrapNativeFetchHandler, EMPTY_BINDINGS } = await import("./wrap");
111
+ const program = await bundled.outputs[0].text();
112
+ const wrapped = wrapNativeFetchHandler(
113
+ program,
114
+ input.prelude + "\nPorffor.c`" + CONTROLLER_HEAD + "`;\n",
115
+ {},
116
+ EMPTY_BINDINGS,
117
+ undefined,
118
+ "2026-10-22",
119
+ "ltx-restore",
120
+ undefined,
121
+ "embedded",
122
+ );
123
+ await writeFile(generated, wrapped);
124
+ const emitted = resolve(root, "emitted");
125
+ await rm(emitted, { recursive: true, force: true });
126
+ await run([...input.porfforCommand, "c", generated, "-o", emitted, "--sb-native-bytes-v1"]);
127
+ const c = (await readFile(resolve(emitted, "app.c"), "utf8")) + DISPATCH;
128
+ const source = resolve(root, "trusted.c"),
129
+ probe = resolve(root, "probe.o");
130
+ const compileFlags = ["-std=gnu11", "-w", ...input.cflags];
131
+ await writeFile(source, c);
132
+ await rm(probe, { force: true });
133
+ await run([...input.cc, ...compileFlags, "-O0", "-c", source, "-o", probe]);
134
+ const nm = Bun.which("nm");
135
+ if (!nm) throw new Error("native restore builds require nm from the native compiler tools");
136
+ const defines = restoreSymbolDefines(await run([nm, "-g", "-U", probe]), input.macho);
137
+ await writeFile(source, defines + c);
138
+ const object = resolve(root, "trusted.o");
139
+ await rm(object, { force: true });
140
+ await run([...input.cc, ...compileFlags, "-O3", "-c", source, "-o", object]);
141
+ const symbols = await run([nm, "-g", "-U", object]);
142
+ for (const line of symbols.split("\n")) {
143
+ const match = /^\s*[0-9a-fA-F]+\s+[A-Za-z]\s+(\S+)\s*$/.exec(line);
144
+ if (!match) continue;
145
+ const name = input.macho ? match[1].slice(1) : match[1];
146
+ if (name !== "sb_cli_restore_dispatch" && !name.startsWith("sb_ltx_trusted_"))
147
+ throw new Error(`native restore external symbol was not namespaced: ${name}`);
148
+ }
149
+ const startup = resolve(root, "startup.cpp"),
150
+ startupObject = resolve(root, "startup.o");
151
+ await writeFile(startup, STARTUP);
152
+ await rm(startupObject, { force: true });
153
+ await run([...input.cxx, "-O3", "-c", startup, "-o", startupObject]);
154
+ return [object, startupObject];
155
+ }
package/src/surface.ts CHANGED
@@ -97,10 +97,10 @@ export const COMMANDS: readonly Command[] = [
97
97
  name: "build",
98
98
  group: "Develop",
99
99
  emoji: "🔨",
100
- args: "[project-dir] [--target host] [--standalone] [--emit-only]",
100
+ args: "[project-dir] [--target host] [--standalone] [--with-restore] [--emit-only]",
101
101
  brief: "[project-dir]",
102
102
  summary:
103
- "Cross-compile the native-fetch sprout (Porffor + Zig). `--target host` builds for this machine instead, to run locally — not deployable. `--standalone` emits one executable carrying its own bindings, with SQLite compiled in (~2 MB) and no broker process. `--emit-only` stops after writing `.sproutboat/porffor-units/<shape>/sprout.generated.js`, the module Porffor would compile, and prints its path: every check still runs, nothing is compiled.",
103
+ "Cross-compile the native-fetch sprout (Porffor + Zig). `--target host` builds for this machine instead, to run locally, not deployable. `--standalone` emits one executable carrying its own bindings, with SQLite compiled in (~2 MB) and no broker process. `--with-restore` adds an offline `APP ltx-restore --generation DIR --output NEW_DB [--txid N]` startup command to a standalone executable; it runs before app initialization. `--emit-only` stops after writing `.sproutboat/porffor-units/<shape>/sprout.generated.js`, the module Porffor would compile, and prints its path: every check still runs, nothing is compiled.",
104
104
  },
105
105
  {
106
106
  name: "toolchain",
@@ -311,7 +311,7 @@ export const ENV_VARS: readonly EnvVar[] = [
311
311
  {
312
312
  name: "SB_REQUEST_BODY_MAX",
313
313
  purpose:
314
- "Largest inbound request body the runtime accepts, in bytes (default 1048576). Anything larger is refused with 413 before the handler runs.",
314
+ "Largest inbound request body the runtime accepts, in bytes. From compatibility date 2026-10-27 bodies stream and the default is 104857600 (100 MiB); before it the body is held whole and the default is 1048576. A larger Content-Length is refused with 413 before the handler runs; a chunked body that grows past it closes the connection.",
315
315
  },
316
316
  {
317
317
  name: "SB_EXTRA_CFLAGS",