@frockbot/applet-sdk 0.5.1 → 0.6.0

package/README.md

@@ -2,11 +2,13 @@
 
 The SDK a FrockBot Applet is written against: a schema-first Durable Object
 server, a TanStack DB client over one real-time socket, a precompiled component
-kit on the theme tokens, a linter, and the `applet` CLI.
+kit on the theme tokens, a linter, and the build pipeline the cloud build
+service runs.
 
-See [ADR 0022](../../docs/adr/0022-applets-as-instance-packages.md) for why an
-Applet's state is a Durable Object facet the kernel owns the lifecycle of, and
-`docs/plans/applets.md` §8 for this package's place in the build.
+An Applet is authored with the `applet_*` tools, built by `apps/applet-build`,
+and mounted as a Durable Object facet from an immutable artifact. There is no
+CLI: nothing outside the service builds an Applet, and no Computer is involved
+at any point.
 
 ## Entry points
 
@@ -17,29 +19,24 @@ Applet's state is a Durable Object facet the kernel owns the lifecycle of, and
 | `@frockbot/applet-sdk/kit`      | the fourteen components (`src/kit/README.md`)            |
 | `@frockbot/applet-sdk/lint`     | the flat ESLint config and the five custom rules         |
 | `@frockbot/applet-sdk/protocol` | wire protocol v1, for the kernel and for tests           |
+| `@frockbot/applet-sdk/build`    | `runAppletBuildV1` — the five stages, for the service    |
 
-## The CLI
+## The build
 
-```sh
-applet new "Weekly Todos" # scaffold from template/
-applet check              # tsc + lint; path:line:col message; non-zero on error
-applet build              # dist/{server.js,ui.html,manifest.json}
-applet dev                # Miniflare on a local port; prints a URL, opens nothing
-```
+`runAppletBuildV1(directory, { mode })` is five named stages over one
+directory: `descriptor`, `typecheck`, `lint`, `bundle`, `describe`. `check`
+stops after the linter; `build` goes on to the artifacts. A stage that fails
+stops the run and names itself, and every failure is a list of
+`{file, line, column, message, severity}`.
 
-`applet build` derives `manifest.json`'s tool declarations by mounting the built
-`dist/server.js` in Miniflare and calling `health()` — the same question the
-kernel asks the facet before it admits a generation, so the manifest cannot
-disagree with the code.
+`manifest.json`'s tool declarations are derived by mounting the built
+`server.js` in Miniflare and calling `health()` — the same question the kernel
+asks the facet before it admits a generation, so the manifest cannot disagree
+with the code.
 
-**The published CLI runs under Node.** `prepublishOnly` bundles
-`src/cli/main.ts` to `dist/cli.mjs`, and the package's `bin` points there. The
-Computer's `applets` provisioning phase installs this package and its runtime
-once under the shared Computer runtime; an in-place runtime update repairs that
-installation when it is missing. An Applet project deliberately has no
-`node_modules` of its own. The checker and bundler resolve SDK, React, and
-TanStack imports from the shared installation, while project dependency trees
-remain reproducible scratch and never enter the durable-root sync.
+`template/` is the scaffold a new Applet starts as.
+`scripts/build-applets-assets.ts` turns it into `applets/template.generated.ts`,
+which `applet_create` writes through the Workspace.
 
 ## What runs where
 
@@ -50,9 +47,9 @@ page served from the anonymous artifact origin into a sandboxed iframe, which
 receives its theme tokens and a short-lived viewer token through the host's
 `init` message and opens exactly one WebSocket back to the facet.
 
-The Cloudflare programming model is not hidden and ADR 0022 says so: an Applet
-is a Durable Object with SQLite and hibernating sockets. What the SDK does hide
-is every binding name — an author sees `tables`, `tools`, and `this.db`.
+The Cloudflare programming model is not hidden: an Applet is a Durable Object
+with SQLite and hibernating sockets. What the SDK does hide is every binding
+name — an author sees `tables`, `tools`, and `this.db`.
 
 ## Wire protocol v1
 
@@ -77,5 +74,5 @@ bun test test spike
 
 Pure modules and the client are tested in `bun test`: the store runs against
 `bun:sqlite`, and `test/loopback.ts` joins the real protocol server to the real
-client transport through a pair of fake sockets. `test/cli.test.ts` and
-`spike/` run the built Applet in Miniflare for real.
+client transport through a pair of fake sockets. `test/build.test.ts` and
+`spike/` run the real pipeline and the built Applet in Miniflare.

package/package.json

@@ -1,9 +1,9 @@
 {
   "name": "@frockbot/applet-sdk",
-  "version": "0.5.1",
+  "version": "0.6.0",
   "private": false,
   "type": "module",
-  "description": "Authoring SDK for FrockBot Applets: schema-first Durable Object server, TanStack DB client, component kit, linter, and the `applet` CLI.",
+  "description": "Authoring SDK for FrockBot Applets: schema-first Durable Object server, TanStack DB client, component kit, linter, and the build pipeline.",
   "license": "UNLICENSED",
   "exports": {
     "./server": "./src/server/index.ts",
@@ -11,21 +11,16 @@
     "./kit": "./src/kit/index.tsx",
     "./lint": "./src/lint/index.ts",
     "./protocol": "./src/protocol/index.ts",
+    "./build": "./src/build/pipeline.ts",
     "./package.json": "./package.json"
   },
-  "bin": {
-    "applet": "./dist/cli.mjs"
-  },
   "files": [
-    "dist",
     "src",
     "types",
     "template",
     "README.md"
   ],
   "scripts": {
-    "build": "bun scripts/build-cli.ts",
-    "prepublishOnly": "bun scripts/build-cli.ts",
     "test": "bun test test spike",
     "typecheck": "tsc --noEmit -p tsconfig.json"
   },
@@ -52,7 +47,7 @@
   "repository": {
     "type": "git",
     "url": "git+https://github.com/timoconnellaus/frockbot.git",
-    "directory": "packages/applet-sdk"
+    "directory": "applets/sdk"
   },
   "frockbot": {
     "npm": true

package/src/{cli/build.ts → build/artifacts.ts}

@@ -1,12 +1,12 @@
 /**
- * `applet build` — three immutable artifacts from two source files.
+ * Three immutable artifacts from two source files.
  *
- * `dist/server.js`  one ESM file whose only import is `cloudflare:workers`,
- *                   exporting `Applet`, which is the name the kernel mounts.
- * `dist/ui.html`    one self-contained page: React, TanStack DB, the kit, and
- *                   the app inlined. No external URL, because the artifact
- *                   origin serves it into a sandbox with no network of its own.
- * `dist/manifest.json` `{ contract, tools, hashes }`.
+ * `server.js`      one ESM file whose only import is `cloudflare:workers`,
+ *                  exporting `Applet`, which is the name the kernel mounts.
+ * `ui.html`        one self-contained page: React, TanStack DB, the kit, and
+ *                  the app inlined. No external URL, because the artifact
+ *                  origin serves it into a sandbox with no network of its own.
+ * `manifest.json`  `{ contract, tools, hashes }`.
  *
  * The tool declarations come from mounting the built server in Miniflare and
  * calling `health()`, not from reading the source. Static analysis would be a
@@ -14,33 +14,91 @@
  * the kernel actually asks — and the kernel admits a generation by comparing
  * the manifest to the facet's own `health()`, so any disagreement is a failed
  * publish. Running the code is the only derivation that cannot drift.
+ *
+ * Nothing here touches a filesystem beyond the source it is given: the build
+ * service returns these three strings over its contract, and the app stores
+ * them under their content hashes.
  */
 
 import { createHash, randomUUID } from "node:crypto";
-import { mkdir, writeFile } from "node:fs/promises";
-import { join } from "node:path";
+import { realpathSync } from "node:fs";
+import { relative, resolve } from "node:path";
 
-import { build as esbuild } from "esbuild";
+import { build as esbuild, type Metafile } from "esbuild";
 
 import type { AppletDescriptionV1 } from "../server/applet.js";
 import { readDescriptor, type AppletBuildManifestV1 } from "./manifest.js";
-import { bundlerNodePaths, SDK_ENTRIES } from "./paths.js";
+import { bundlerNodePaths, SDK_ENTRIES, SDK_ROOT } from "./paths.js";
 import { startAppletRuntime } from "./runtime.js";
 
-export interface AppletBuildResult {
-  directory: string;
-  serverPath: string;
-  uiPath: string;
-  manifestPath: string;
+export interface AppletArtifactsV1 {
   manifest: AppletBuildManifestV1;
+  server: string;
+  ui: string;
 }
 
 function sha256(text: string): string {
   return createHash("sha256").update(text, "utf8").digest("hex");
 }
 
-function alias(): Record<string, string> {
-  return { ...SDK_ENTRIES };
+/** esbuild reports real paths; a temp directory is often a symlink to one. */
+function real(path: string): string {
+  try {
+    return realpathSync(resolve(path));
+  } catch {
+    return resolve(path);
+  }
+}
+
+/**
+ * Where a bundled module came from, said the same way everywhere.
+ *
+ * An Applet's directory, the SDK's directory and the working directory are all
+ * different on an author's machine and in the build container — and esbuild
+ * writes each module's path into the unminified output as a comment. Left
+ * alone, identical source bundled in two places produces two different files
+ * and therefore two different content hashes, which would give R2 a new object
+ * on every publish of unchanged code.
+ */
+function moduleLabel(absolute: string, appletRoot: string): string {
+  for (const [root, prefix] of [
+    [appletRoot, ""],
+    [real(SDK_ROOT), "applet-sdk/"],
+  ] as const) {
+    const inside = relative(root, absolute);
+    if (!inside.startsWith("..") && inside !== "") return prefix + inside;
+  }
+  const dependency = absolute.lastIndexOf("node_modules/");
+  return dependency === -1 ? absolute : absolute.slice(dependency);
+}
+
+/**
+ * Rewrites esbuild's module comments to those stable labels.
+ *
+ * The paths come from the metafile rather than from a pattern over the output,
+ * so only lines esbuild actually wrote are touched and a comment in someone's
+ * own source is left alone.
+ */
+function stableModulePaths(
+  text: string,
+  metafile: Metafile,
+  appletRoot: string,
+): string {
+  const root = real(appletRoot);
+  const labels = new Map(
+    Object.keys(metafile.inputs).map((input) => [
+      input,
+      moduleLabel(real(input), root),
+    ]),
+  );
+  return text
+    .split("\n")
+    .map((line) => {
+      if (!line.startsWith("// ")) return line;
+      const label = labels.get(line.slice(3));
+      return label === undefined ? line : `// ${label}`;
+    })
+    .join("\n");
 }
 
 async function bundle(options: {
@@ -68,15 +126,16 @@ async function bundle(options: {
     minify: options.minify,
     legalComments: "none",
     external: options.external,
-    alias: alias(),
+    alias: { ...SDK_ENTRIES },
     nodePaths: bundlerNodePaths(),
     conditions: ["import", "module", "browser", "default"],
     define: { "process.env.NODE_ENV": '"production"' },
+    metafile: true,
     logLevel: "silent",
   });
   const file = result.outputFiles?.[0];
   if (!file) throw new Error("The bundler produced no output");
-  return file.text;
+  return stableModulePaths(file.text, result.metafile, options.resolveDir);
 }
 
 function page(title: string, script: string): string {
@@ -130,12 +189,12 @@ export async function readDescription(
   }
 }
 
-export async function buildApplet(
+/** The two bundles, in the order that lets a bundle failure precede a boot. */
+export async function bundleAppletArtifacts(
   directory: string,
-): Promise<AppletBuildResult> {
+): Promise<{ descriptorId: string; server: string; ui: string }> {
   const descriptor = await readDescriptor(directory);
-
-  const serverCode = await bundle({
+  const server = await bundle({
     // `Applet` is the export name the kernel's facet mount looks up; the author
     // writes an ordinary default export and never learns that name.
     stdin:
@@ -147,7 +206,6 @@ export async function buildApplet(
     minify: false,
     loaderName: "applet-server-entry.ts",
   });
-
   const uiScript = await bundle({
     stdin: 'import "./ui";\n',
     resolveDir: directory,
@@ -157,27 +215,34 @@ export async function buildApplet(
     minify: true,
     loaderName: "applet-ui-entry.tsx",
   });
-  const html = page(descriptor.displayName, uiScript);
+  return {
+    descriptorId: descriptor.id,
+    server,
+    ui: page(descriptor.displayName, uiScript),
+  };
+}
 
-  const description = await readDescription(serverCode, descriptor.id);
-  const manifest: AppletBuildManifestV1 = {
+/** The manifest for artifacts whose declarations have been read. */
+export function appletManifest(
+  tools: AppletDescriptionV1["tools"],
+  artifacts: { server: string; ui: string },
+): AppletBuildManifestV1 {
+  return {
     contract: 1,
-    tools: description.tools,
-    hashes: { server: sha256(serverCode), ui: sha256(html) },
+    tools,
+    hashes: { server: sha256(artifacts.server), ui: sha256(artifacts.ui) },
   };
+}
 
-  const dist = join(directory, "dist");
-  await mkdir(dist, { recursive: true });
-  const serverPath = join(dist, "server.js");
-  const uiPath = join(dist, "ui.html");
-  const manifestPath = join(dist, "manifest.json");
-  await writeFile(serverPath, serverCode, "utf8");
-  await writeFile(uiPath, html, "utf8");
-  await writeFile(
-    manifestPath,
-    `${JSON.stringify(manifest, null, 2)}\n`,
-    "utf8",
-  );
-
-  return { directory, serverPath, uiPath, manifestPath, manifest };
+/** Bundle, boot, describe. The whole artifact derivation, with no output. */
+export async function buildAppletArtifacts(
+  directory: string,
+): Promise<AppletArtifactsV1> {
+  const { descriptorId, server, ui } = await bundleAppletArtifacts(directory);
+  const description = await readDescription(server, descriptorId);
+  return {
+    manifest: appletManifest(description.tools, { server, ui }),
+    server,
+    ui,
+  };
 }

package/src/{cli → build}/check.ts

@@ -1,5 +1,5 @@
 /**
- * `applet check` — the type checker and the linter, one diagnostic list.
+ * The check stage — the type checker and the linter, one diagnostic list.
  *
  * The Applet has no `node_modules`: it is source at a durable root. So the
  * compiler options are built here from `paths.ts` rather than from a tsconfig
@@ -106,7 +106,7 @@ export async function typeCheckApplet(
     });
 }
 
-/** Everything `applet check` reports, in source order. */
+/** Everything the check stage reports, in source order. */
 export async function checkApplet(
   directory: string,
 ): Promise<AppletDiagnostic[]> {

package/src/{cli → build}/manifest.ts

@@ -7,7 +7,7 @@ import { APPLET_CONTRACT_VERSION } from "../protocol/index.js";
 import type { AppletToolDeclarationV1 } from "../server/applet.js";
 
 export interface AppletDescriptorV1 {
-  /** `/^[a-z][a-z0-9-]{0,31}$/`; the directory name `applet new` creates. */
+  /** `/^[a-z][a-z0-9-]{0,31}$/`; the scaffold's directory name. */
   id: string;
   displayName: string;
   contract: 1;
@@ -50,9 +50,7 @@ export async function readDescriptor(
   try {
     text = await readFile(path, "utf8");
   } catch {
-    throw new Error(
-      `No applet.json in ${directory}; run \`applet new <name>\` first`,
-    );
+    throw new Error(`No applet.json in ${directory}`);
   }
   return decodeDescriptor(JSON.parse(text));
 }

package/src/{cli → build}/paths.ts

@@ -17,11 +17,12 @@ const require = createRequire(import.meta.url);
  * Found by walking up to this package's own `package.json`, not by counting
  * directories.
  *
- * The same module runs from two depths: `src/cli/paths.ts` under Bun, and the
- * bundled `dist/cli.mjs` under Node on the Computer, which has no Bun. A fixed
- * `../../` is right for one and silently wrong for the other — it would resolve
- * the SDK's entries to a directory that does not exist and every Applet import
- * would fail to type-check with no explanation.
+ * The same module runs from three depths: `src/build/paths.ts` under Bun, the
+ * bundled `dist/cli.mjs` under Node, and the build service's own bundle, which
+ * is emitted into `dist/` for exactly this reason. A fixed `../../` is right
+ * for one and silently wrong for the others — it would resolve the SDK's
+ * entries to a directory that does not exist and every Applet import would
+ * fail to type-check with no explanation.
  */
 function findSdkRoot(): string {
   let directory = dirname(fileURLToPath(import.meta.url));
@@ -92,7 +93,7 @@ export function typeCheckerPaths(): Record<string, string[]> {
     paths["react/jsx-runtime"] = [join(types, "jsx-runtime.d.ts")];
     paths["react/*"] = [join(types, "*")];
   } catch {
-    // No React declarations available; `applet check` reports the import.
+    // No React declarations available; the check stage reports the import.
   }
   return paths;
 }

package/src/build/pipeline.ts

@@ -0,0 +1,96 @@
+/**
+ * The whole Applet build, as five named stages over one directory.
+ *
+ * `applet_check` and `applet_publish` are the two ways in, and both reach
+ * these stages through the same build service. There is one implementation of
+ * each stage, so a check and the publish that follows it cannot disagree about
+ * whether the code is admissible or about what it hashes to.
+ *
+ * A stage that fails stops the run and names itself, which is what turns a
+ * wall of diagnostics into "your `server.ts` does not type-check". The two
+ * stages that produce no diagnostic list of their own — `bundle` and
+ * `describe` — report the thrown message as one diagnostic against
+ * `applet.json`, so a caller has one shape to render.
+ */
+
+import { lintApplet, type AppletDiagnostic } from "../lint/index.js";
+import { buildAppletArtifacts, type AppletArtifactsV1 } from "./artifacts.js";
+import { typeCheckApplet } from "./check.js";
+import { readDescriptor } from "./manifest.js";
+
+export type AppletBuildStage =
+  "descriptor" | "typecheck" | "lint" | "bundle" | "describe";
+
+export type AppletBuildOutcome =
+  | { status: "checked" }
+  | ({ status: "built" } & AppletArtifactsV1)
+  | {
+      status: "failed";
+      stage: AppletBuildStage;
+      diagnostics: AppletDiagnostic[];
+    };
+
+/** A thrown stage failure, reported at `applet.json` for want of a position. */
+function thrown(error: unknown): AppletDiagnostic[] {
+  return [
+    {
+      file: "applet.json",
+      line: 1,
+      column: 1,
+      message: error instanceof Error ? error.message : String(error),
+      severity: "error",
+    },
+  ];
+}
+
+function hasError(diagnostics: AppletDiagnostic[]): boolean {
+  return diagnostics.some((diagnostic) => diagnostic.severity === "error");
+}
+
+export interface AppletBuildPipelineOptions {
+  /** `check` stops after the linter; `build` goes on to the artifacts. */
+  mode: "check" | "build";
+}
+
+export async function runAppletBuildV1(
+  directory: string,
+  options: AppletBuildPipelineOptions,
+): Promise<AppletBuildOutcome> {
+  try {
+    await readDescriptor(directory);
+  } catch (error) {
+    return {
+      status: "failed",
+      stage: "descriptor",
+      diagnostics: thrown(error),
+    };
+  }
+
+  const types = await typeCheckApplet(directory);
+  if (hasError(types)) {
+    return { status: "failed", stage: "typecheck", diagnostics: types };
+  }
+  const lint = await lintApplet(directory);
+  if (hasError(lint)) {
+    return { status: "failed", stage: "lint", diagnostics: lint };
+  }
+  if (options.mode === "check") return { status: "checked" };
+
+  let artifacts: AppletArtifactsV1;
+  try {
+    artifacts = await buildAppletArtifacts(directory);
+  } catch (error) {
+    // `describe` is the only stage that boots the built module, and both of
+    // its failures say so in their own words; anything else thrown here came
+    // out of the bundler.
+    const message = error instanceof Error ? error.message : String(error);
+    return {
+      status: "failed",
+      stage: /^The Applet (failed to mount|could not describe)/.test(message)
+        ? "describe"
+        : "bundle",
+      diagnostics: thrown(error),
+    };
+  }
+  return { status: "built", ...artifacts };
+}

package/src/{cli → build}/runtime.ts

@@ -3,8 +3,8 @@
  *
  * Miniflare gives the built `dist/server.js` the one thing no fake can: a
  * SQLite-backed Durable Object with hibernating WebSockets, which is exactly
- * what the loader gives it in production. `applet dev` serves the page from
- * it, and `applet build` uses the same runtime to ask the mounted class what
+ * what the loader gives it in production. The build uses it to ask the
+ * mounted class what
  * tools it declares rather than guessing from the source.
  */
 

package/src/client/index.ts

@@ -17,7 +17,7 @@
  * The page never opens the socket itself: the host sends an `init` postMessage
  * carrying the theme tokens and a short-lived viewer token, and `createApplet`
  * connects from that. `connect(init)` is the same path, called by hand, which
- * is what `applet dev` and the tests use.
+ * is what the tests use.
  */
 
 import type { Collection } from "@tanstack/db";

package/src/lint/index.ts

@@ -1,6 +1,6 @@
 /**
  * `@frockbot/applet-sdk/lint` — the flat config, the rules, and the one call
- * `applet check` makes.
+ * the check stage makes.
  *
  * A diagnostic is the SDK's whole answer to "what did I do wrong": the CLI
  * prints `path:line:col message` and nothing else, so what a Bot must remember

package/src/server/applet.ts

@@ -61,7 +61,7 @@ export interface AppletHealthV1 {
   schemaRevision: number;
 }
 
-/** The build-time description `applet build` writes into `dist/manifest.json`. */
+/** The build-time description the build writes into the manifest. */
 export interface AppletDescriptionV1 {
   contract: 1;
   tools: AppletToolDeclarationV1[];
@@ -250,7 +250,7 @@ export abstract class Applet<
     };
   }
 
-  /** The tool declarations, for `applet build` to write into the manifest. */
+  /** The tool declarations, for the build to write into the manifest. */
   describe(): AppletDescriptionV1 {
     return {
       contract: APPLET_CONTRACT_VERSION,

package/template/README.md

@@ -9,14 +9,11 @@ A FrockBot Applet. Two files are yours:
 
 ## The loop
 
-```sh
-applet check # type-check and lint; every problem prints as path:line:col message
-applet build # dist/server.js, dist/ui.html, dist/manifest.json
-applet dev   # serves the built Applet; prints a URL, opens nothing
-```
-
-Open the printed URL in the Computer's browser to look at it. Publish with
-`applet_publish` once `applet check` is clean.
+`applet_files` and `applet_read_file` to see what is here, `applet_write_file`
+to change it, then `applet_check` — it type-checks, lints, bundles and boots
+your server, and answers either with every problem as `path:line:col message`
+or with the tools it declares and a URL for the page. Publish with
+`applet_publish` once the check is clean.
 
 ## Rules the linter enforces