executor 2.0.0-beta.4 → 2.0.0-beta.40

package/README.md

@@ -1,27 +1,62 @@
 # Executor 2 beta
 
-Install with `npm i -g executor@beta`. Requires Node 24.14 or newer.
-Run `executor` to start the server and open the dashboard. Use `executor serve`
-for headless operation and `executor pair` to get a new browser connection link.
-
-The package includes the dashboard, framework, runtime compiler, pinned Bun,
-Git, and the Motel collector. It does not need a source checkout, Bun, or
-1Password on the user machine. Native runtimes are selected by npm for macOS
-(Apple Silicon and Intel), Linux (arm64 and x64), and Windows (x64).
+Install with npm, `npm i -g executor@beta`, on Node 14.18 or newer. Without Node, run `curl -fsSL https://executor.sh/docs/install.sh | sh`
+on macOS or Linux, or `irm https://executor.sh/docs/install.ps1 | iex` in
+PowerShell on Windows. Each release also attaches its archives and `SHA256SUMS`.
+Run `executor` to start the background Executor, or join the one already
+running, and open the dashboard. The background Executor keeps running until
+`executor service stop`; nothing starts it with the OS. `executor` checks npm
+daily for a newer release and suggests `executor upgrade`; set
+`EXECUTOR_NO_UPDATE_CHECK=1` to skip it. Use `executor service start` to start it
+without a browser, `executor serve` to run it in the foreground, and
+`executor pair` to get a new browser connection link.
+`executor upgrade` installs the newest release on your channel the way Executor
+was installed: an install-script copy downloads and checks the release and keeps
+the previous version for `executor upgrade --rollback`; an npm, pnpm, Bun or
+Yarn install runs its package manager. Either way it stops the background
+Executor before replacing its files and starts it again on the new version; a
+release from before the background service, which a rollback can return to, has
+no `executor service`, so run `executor serve` there instead.
+
+Executor is one executable, built with Bun, with the files it runs or reads
+beside it: the dashboard, the app framework and compiler references, workerd,
+Git, the native credential-store binding and the Motel collector. It needs no
+source checkout, Node, Bun or 1Password on the user machine. The `executor` npm
+package is a launcher. The first time a version starts, it downloads that
+release's archive for the platform, for macOS (Apple Silicon and Intel), Linux
+(arm64 and x64, glibc) or Windows (x64), from the public GitHub release, checks it
+against the SHA-256 the package records, and keeps it in
+`~/.executor/v2/launcher/<version>` with the version before it. The same archive
+is the release download the install scripts and Homebrew use.
+`EXECUTOR_DOWNLOAD_URL` names a mirror of the releases for all of them.
 
 Beta uses `~/.executor/v2/cli` for its data. This path stays the same at the
-stable release. It is separate from Executor 1 data. Do not run two servers
-against the same directory. `EXECUTOR_DATA_DIR` selects another directory and
-`EXECUTOR_PORT` changes the default port of 4312.
+stable release. It is separate from Executor 1 data. One server runs per data
+directory; its `.server.lock` stops a second. `EXECUTOR_DATA_DIR` selects another
+directory, and `EXECUTOR_PORT` or `executor service set port` changes the default
+port of 4312. The desktop app uses the same directory and background Executor.
+Its first launch moves the data of a desktop release from before the background
+service into that directory, once. CLI data already there moves to
+`~/.executor/v2/backups/cli-data-<time>`, which the desktop app names; nothing
+is merged or deleted. The CLI's settings carry over, except its port and the
+`env` variables that belong to its data.
 
 On first launch, Executor saves its API and encryption keys in the OS credential
 store: macOS Keychain, Windows Credential Manager, or Linux Secret Service.
-Linux requires a persistent Secret Service such as GNOME Keyring. There is no
-plaintext or in-memory fallback. A locked, unavailable, missing or invalid
-credential stops startup; existing installations never receive replacement keys.
-
-Back up the data directory, including `installation.json`, and the matching OS
-credential (`com.usefulsoftware.executor.v2`, account ID from that file).
+Linux uses a persistent Secret Service such as GNOME Keyring. If there is no
+store on a directory's first start, the keys are saved to `keys.json` in the
+data directory (mode `0600`) and the path is printed to stderr. If a store exists
+but access is denied or it is locked, startup stops without a key file and the
+next start asks again. Over SSH, unlock the store first, or set
+`EXECUTOR_KEY_STORAGE=file` before a directory's first start to use `keys.json`
+(`os` requires the store instead). A directory that already uses the OS store stops when it
+is locked or unavailable. A missing or
+invalid credential or key file stops startup; existing installations never
+receive replacement keys.
+
+Back up the data directory, including `installation.json` and any `keys.json`,
+and the matching OS credential (`com.usefulsoftware.executor.v2`, account ID
+from that file) when the directory uses one.
 Copying only the data directory to another machine is not enough to recover it.
 For a new headless install, supply both `EXECUTOR_API_KEY` (32+ characters) and
 `EXECUTOR_ENCRYPTION_KEY` (64 hexadecimal characters) through a secret manager.
@@ -38,29 +73,67 @@ version publishes to `latest` only through an explicitly selected stable release
 
 ```sh
 bun install --frozen-lockfile
-bun run release:cli
+EXECUTOR_BUN_RUNTIME=/path/to/locked-down/bun bun run release:cli
 bun run release:wrapper
 ```
 
 Native artifacts are under `.local/releases/<version>-<platform>-<arch>/`.
-The npm launcher is under `.local/releases/<version>/wrapper/` and uses exact
-native version aliases. These build commands do not publish anything.
-The native package bundles the CLI and desktop server together. The app compiler
-and framework are prepared for workerd at build time. Native libraries, Git,
-Motel and authoring references remain included. No runtime download is needed.
-Each compressed native archive must fit within 180 MiB before publication.
-This is a release budget with margin, not a documented npm registry limit.
+`release:cli` bundles the runtime twice. `package/` holds the JavaScript runtime
+the desktop app runs with its own Node. `standalone/` holds the release install:
+the `executor` executable (`bun build --compile`) and the files beside it, which
+`executor-<version>-<platform>-<arch>.tgz` packs as the platform's release
+archive. The executable takes `runtime/cli.mjs` in its install directory as
+every module's location, so resources, `require` and `import.meta.resolve` find
+the same files beside it wherever the archive is unpacked, through any launcher
+symlink. Packages that must stay on disk load from
+`runtime/node_modules`; Git is in `runtime/git`.
+The npm launcher is under `.local/releases/<version>/wrapper/`. Its
+package.json records the SHA-256 of each platform archive found under
+`.local/releases/<version>-<platform>-<arch>/`, or under the directory
+`release:wrapper` is given; publishing requires all five. The runtimes are never
+published to npm. Through 2.0.0-beta.36 each was published as a version of the
+`executor` package, such as `2.0.0-beta.36-win32-x64`, which semver ranks above
+the launcher, so a range update such as `bun update -g` replaced the launcher
+with a runtime. Those versions are deprecated so package managers skip them.
+These build commands do not publish anything.
+The app compiler and framework are prepared for workerd at build time, so the
+executable downloads nothing more. Most of each archive is the executable,
+workerd and the bundled Git, which keeps only what the runtime runs (`keptGit`
+in `runtime-bundle.ts`).
 External source maps stay in the separate CI artifact for release diagnostics.
 
+Stock Bun lets any local process run its own code inside a compiled executable,
+through `BUN_OPTIONS`, `BUN_BE_BUN`, `BUN_INSPECT` and similar variables that
+Bun reads before the executable's first line. The release executable is built
+on a Bun runtime patched to ignore them, from
+[`bun-runtime/`](bun-runtime/README.md). `EXECUTOR_BUN_RUNTIME` names it; CI and
+publishing builds refuse stock Bun, and a local build without it prints that it
+is a development build. The release workflow fetches the runtime from the
+`bun-runtime-<bun version>-<patch hash>` release, building it first when the
+patch changed.
+
+The install scripts are `apps/docs/public/install.sh` and `install.ps1`, served
+at `https://executor.sh/docs/`. The publish job writes the Homebrew formula
+(`scripts/releases/homebrew.ts`) to the tap named by the `HOMEBREW_TAP`
+repository variable once it is set.
+
 The prepared packages retain dependency licenses. Public releases also include
 checksum-pinned source archives for the bundled Git distribution.
 
 ## macOS signing
 
-`node scripts/releases/desktop.ts <version>` is unsigned by default, which is a
-development build: Gatekeeper rejects it until someone overrides it by hand.
+`node scripts/releases/desktop.ts <version>` and `node scripts/releases/package.ts`
+are unsigned by default, which is a development build: Gatekeeper rejects it
+until someone overrides it by hand.
 Add `--sign` to sign with a Developer ID Application identity, or `--notarize`
-to sign and submit the result to Apple. Both modes keep electron-builder's
+to sign and submit the result to Apple. A publishing build of the CLI on macOS
+(`EXECUTOR_RELEASE_PUBLISH=true`) fails unless it notarizes. For the CLI, every
+Mach-O file in the install is signed with the hardened runtime, the executable
+last as `com.usefulsoftware.executor.v2.cli` with
+`entitlements.standalone.plist`: JIT for Bun and workerd, and library validation
+left on, so it loads only libraries signed with the same identity. The whole
+install is notarized as one archive; a bare executable cannot hold a stapled
+ticket, so Gatekeeper checks it online. Both modes keep electron-builder's
 identity auto-discovery off, so a build never picks up a stray keychain identity.
 
 Signing accepts these environment variables and keeps plaintext keys off ordinary disk:

package/bin/executor

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+import("../bin.mjs");

package/bin/package.json

@@ -0,0 +1 @@
+{"type":"commonjs"}

package/bin.mjs

@@ -1,15 +1,245 @@
 #!/usr/bin/env node
-const name = "executor-" + process.platform + "-" + process.arch;
-const supported = ["executor-darwin-arm64","executor-darwin-x64","executor-linux-x64","executor-linux-arm64","executor-win32-x64"];
-if (!supported.includes(name)) {
-  console.error("Executor does not support " + process.platform + "/" + process.arch + ".");
-  process.exitCode = 1;
-} else {
-  let entry;
-  try { entry = import.meta.resolve(name + "/bin.mjs"); }
-  catch {
-    console.error("Executor's native runtime is missing. Reinstall with: npm i -g executor@beta --include=optional");
-    process.exitCode = 1;
+// The `executor` npm package's command. The package holds no runtime: on first use of a version it
+// downloads that release's archive for this platform from the public GitHub release, checks it
+// against the SHA-256 this package's package.json records, and unpacks it into
+// ~/.executor/v2/launcher/<version>. It then starts the Executor executable there, passes signals
+// on, and exits as it exits.
+//
+// npm, Bun, pnpm and Yarn resolve `executor` alone, so an update can only ever land on another
+// launcher; the running executable lives outside node_modules, so an update never has to replace a
+// file in use. The executable carries its own runtime, so this file uses only what every Node since
+// 14 has.
+import { spawn, spawnSync } from "child_process";
+import { createHash, randomBytes } from "crypto";
+import {
+  createWriteStream,
+  existsSync,
+  mkdirSync,
+  readdirSync,
+  renameSync,
+  rmSync,
+  statSync,
+  utimesSync,
+} from "fs";
+import { request as httpRequest } from "http";
+import { request as httpsRequest } from "https";
+import { createRequire } from "module";
+import { homedir } from "os";
+import { join } from "path";
+import { fileURLToPath } from "url";
+
+const require = createRequire(import.meta.url);
+const manifest = require("./package.json");
+const version = manifest.version;
+const target = process.platform + "-" + process.arch;
+const digest = (manifest.runtimes || {})[target];
+const channel = version.indexOf("-beta.") === -1 ? "" : "@beta";
+const executableName = process.platform === "win32" ? "executor.exe" : "executor";
+const archive = "executor-" + version + "-" + target + ".tgz";
+// The same mirror setting the install scripts and `executor upgrade` read.
+const downloads = (
+  process.env.EXECUTOR_DOWNLOAD_URL ||
+  "https://github.com/UsefulSoftwareCo/executor/releases/download"
+).replace(/\/+$/, "");
+const url = downloads + "/executor@" + version + "/" + archive;
+const root = join(homedir(), ".executor", "v2", "launcher");
+const installed = join(root, version);
+const executable = join(installed, executableName);
+
+const fail = (lines) => {
+  for (const line of lines) console.error(line);
+  process.exit(1);
+};
+
+if (typeof digest !== "string")
+  fail(["Executor does not support " + process.platform + "/" + process.arch + "."]);
+
+/** GET `address`, following redirects; resolves to the response of the final location. */
+const get = (address, redirects) =>
+  new Promise((resolve, reject) => {
+    const request = (address.startsWith("http:") ? httpRequest : httpsRequest)(
+      address,
+      { headers: { "user-agent": "executor-launcher/" + version } },
+      (response) => {
+        const status = response.statusCode || 0;
+        if (status >= 300 && status < 400 && response.headers.location) {
+          response.resume();
+          if (redirects >= 10) return reject(new Error("too many redirects"));
+          return resolve(get(new URL(response.headers.location, address).href, redirects + 1));
+        }
+        if (status !== 200) {
+          response.resume();
+          return reject(new Error("HTTP " + status));
+        }
+        resolve(response);
+      },
+    );
+    // A connection that stops sending for a minute is not coming back.
+    request.setTimeout(60000, () => request.destroy(new Error("the connection stalled")));
+    request.on("error", reject);
+    request.end();
+  });
+
+/** Download the archive to `file`, hashing it as it arrives; resolves to its SHA-256. */
+const download = async (file) => {
+  const response = await get(url, 0);
+  const total = Number(response.headers["content-length"]) || 0;
+  const hash = createHash("sha256");
+  const out = createWriteStream(file);
+  const tty = process.stderr.isTTY;
+  let received = 0;
+  let shown = -1;
+  const label = "Downloading Executor " + version + " for " + target;
+  if (!tty) console.error(label + " (first run of this version)");
+  await new Promise((resolve, reject) => {
+    response.on("data", (chunk) => {
+      hash.update(chunk);
+      received += chunk.length;
+      const step = total ? Math.floor((received * 100) / total) : Math.floor(received / 1048576);
+      if (tty && step !== shown) {
+        shown = step;
+        const amount = (received / 1048576).toFixed(1) + " MB";
+        process.stderr.write(
+          "\r\x1b[2K" + label + (total ? " " + step + "%" : "") + " (" + amount + ")",
+        );
+      }
+    });
+    response.on("error", reject);
+    out.on("error", reject);
+    out.on("finish", resolve);
+    response.pipe(out);
+  });
+  if (tty) process.stderr.write("\r\x1b[2K");
+  return hash.digest("hex");
+};
+
+/**
+ * Keep this version and the most recently installed other one, which a background Executor started
+ * before an update may still run from, as the install scripts keep `previous`. Downloads that
+ * stopped part way are removed after a day. Anything in use, which Windows refuses to delete, stays
+ * for a later download to remove.
+ */
+const prune = () => {
+  const day = 24 * 60 * 60 * 1000;
+  const entries = readdirSync(root)
+    .filter((name) => name !== version)
+    .map((name) => ({ name, time: statSync(join(root, name)).mtimeMs }));
+  const versions = entries.filter((entry) => !entry.name.startsWith("."));
+  versions.sort((left, right) => right.time - left.time);
+  const stale = versions
+    .slice(1)
+    .concat(entries.filter((entry) => entry.name.startsWith(".") && Date.now() - entry.time > day));
+  for (const entry of stale) {
+    try {
+      rmSync(join(root, entry.name), { recursive: true, force: true });
+    } catch {
+      // In use; a later download removes it.
+    }
   }
-  if (entry !== undefined) await import(entry);
-}
+};
+
+const install = async () => {
+  mkdirSync(root, { recursive: true });
+  const staging = join(root, ".download-" + process.pid + "-" + randomBytes(4).toString("hex"));
+  mkdirSync(staging);
+  // `fail` exits at once, without running `finally`.
+  const cleanup = () => rmSync(staging, { recursive: true, force: true });
+  process.on("exit", cleanup);
+  try {
+    let actual;
+    try {
+      actual = await download(join(staging, archive));
+    } catch (error) {
+      fail([
+        "Executor could not download its runtime from " + url + ": " + error.message + ".",
+        "Check your connection and try again. EXECUTOR_DOWNLOAD_URL names a mirror of the releases.",
+      ]);
+    }
+    if (actual !== digest)
+      fail([
+        archive +
+          " does not match the SHA-256 this package records (expected " +
+          digest +
+          ", got " +
+          actual +
+          ").",
+        "Nothing was installed. Check EXECUTOR_DOWNLOAD_URL, or reinstall with: npm i -g executor" +
+          channel,
+      ]);
+    // The system's tar, as the install scripts use: Windows' own bsdtar, since Git's GNU tar on
+    // PATH reads a drive letter as a remote host. The archive path is relative for the same reason.
+    const tar =
+      process.platform === "win32"
+        ? join(process.env.SystemRoot || "C:\\Windows", "System32", "tar.exe")
+        : "tar";
+    // Nothing reaches stdout, which an MCP client starting `executor mcp` reads as the protocol.
+    const unpacked = spawnSync(tar, ["-xzf", archive], {
+      cwd: staging,
+      stdio: ["ignore", "ignore", "inherit"],
+    });
+    if (unpacked.status !== 0)
+      fail([
+        "Executor could not unpack " +
+          archive +
+          " with " +
+          tar +
+          (unpacked.error ? ": " + unpacked.error.message : "."),
+      ]);
+    try {
+      renameSync(join(staging, "package"), installed);
+    } catch (error) {
+      // Another `executor` command installed this version first.
+      if (!existsSync(executable))
+        fail([
+          "Executor could not install its runtime in " + installed + ": " + error.message,
+          "Remove " + installed + " and try again.",
+        ]);
+    }
+    // The version's time orders it for pruning; the archive's own times are from 1985.
+    const now = new Date();
+    utimesSync(installed, now, now);
+  } finally {
+    process.removeListener("exit", cleanup);
+    cleanup();
+  }
+  try {
+    prune();
+  } catch {
+    // Pruning never stops Executor from starting.
+  }
+};
+
+if (!existsSync(executable)) await install();
+
+const child = spawn(executable, process.argv.slice(2), {
+  stdio: "inherit",
+  env: Object.assign({}, process.env, {
+    // How Executor was installed, for `executor upgrade` and its update notice.
+    EXECUTOR_LAUNCHER: fileURLToPath(import.meta.url),
+    EXECUTOR_LAUNCHER_NODE: process.versions.node,
+  }),
+});
+
+// A terminal's Ctrl-C already reaches the whole process group, and on Windows every process on
+// the console. A service manager signals only this process, so the signal is passed on. Windows
+// has no signals to pass: killing the child there would end it at once, before its shutdown.
+const signals =
+  process.platform === "win32" ? ["SIGINT", "SIGBREAK"] : ["SIGINT", "SIGTERM", "SIGHUP"];
+const forward = (signal) => {
+  if (process.platform !== "win32" && child.exitCode === null && child.signalCode === null)
+    child.kill(signal);
+};
+for (const signal of signals) process.on(signal, forward);
+
+child.on("error", (error) =>
+  fail([
+    "Executor could not start " + executable + ": " + error.message,
+    "Remove " + installed + " to download it again.",
+  ]),
+);
+child.on("exit", (code, signal) => {
+  for (const name of signals) process.removeListener(name, forward);
+  // Ending by the same signal tells the shell what stopped Executor.
+  if (signal !== null) process.kill(process.pid, signal);
+  else process.exit(code === null ? 1 : code);
+});

package/package.json

@@ -1,9 +1,9 @@
 {
   "name": "executor",
-  "version": "2.0.0-beta.4",
+  "version": "2.0.0-beta.40",
   "private": false,
   "description": "Run Executor locally and connect your agents to your apps.",
-  "homepage": "https://v2.executor.sh",
+  "homepage": "https://executor.sh",
   "license": "MIT",
   "repository": {
     "type": "git",
@@ -11,13 +11,15 @@
   },
   "type": "module",
   "engines": {
-    "node": ">=24.14.0"
+    "node": ">=14.18.0"
   },
   "bin": {
     "executor": "bin.mjs"
   },
   "files": [
     "bin.mjs",
+    "bin/executor",
+    "bin/package.json",
     "README.md",
     "LICENSE"
   ],
@@ -25,11 +27,11 @@
     "access": "public",
     "tag": "beta"
   },
-  "optionalDependencies": {
-    "executor-darwin-arm64": "npm:executor@2.0.0-beta.4-darwin-arm64",
-    "executor-darwin-x64": "npm:executor@2.0.0-beta.4-darwin-x64",
-    "executor-linux-x64": "npm:executor@2.0.0-beta.4-linux-x64",
-    "executor-linux-arm64": "npm:executor@2.0.0-beta.4-linux-arm64",
-    "executor-win32-x64": "npm:executor@2.0.0-beta.4-win32-x64"
+  "runtimes": {
+    "darwin-arm64": "85ae709121e94e10216e3a8e7b6e547fa2fb1f40077507d3e6c2fa21776da07a",
+    "darwin-x64": "6ec9505e1aeae66bb13370577a1446fbabdaf7ad7a3947f377de0513e44c8bc5",
+    "linux-x64": "ee4e11927b3067e80948f1ffb9187ad52f9bb39fee2ae7c22fa39933d8aa0cac",
+    "linux-arm64": "23791582608c449cd070668cebe024df2dbb11b914a32aec4abbd2abb7c0e080",
+    "win32-x64": "cd7802407951ed968aa150f301a0727763697f13fcf4f512f62e2eac15e7ab19"
   }
 }