@forgezero/agent 0.1.38 → 0.1.40

package/README.md

@@ -15,7 +15,10 @@ fz --help
 
 The one public package installs both commands. Keeping bootstrap and the daemon
 in one version prevents a newly installed `fz` from provisioning a different
-agent protocol.
+agent protocol. Both executable artifacts bundle their ForgeZero runtime
+dependencies, so the constrained SSH bootstrap can transfer the reviewed pair
+to a new host without relying on a checkout, `node_modules`, or a second remote
+package-resolution step. Library subpath exports remain modular packages.
 
 The operator command starts browser-first platform genesis, inspects status,
 wraps processes, and installs the daemon. Genesis opens the one-use founder
@@ -24,7 +27,6 @@ authorization, then continues custody in the browser. Passkey PRF output and
 both recovery phrases never enter the terminal:
 
 ```bash
-fz keys
 fz status
 fz genesis --mode 2-of-3 --api https://api.example --app https://console.example
 fz unlock --phrase-file /secure/offline-phrase.txt # run once per participating custodian
@@ -43,13 +45,51 @@ script and no second implementation for tenant or platform machines:
 
 ```bash
 fz bootstrap platform                         # interactive plan, no mutation
-sudo fz bootstrap platform --apply            # initial DB+API or API-only host
+sudo fz bootstrap platform prepare --bootstrap-config ./platform.json --apply
+# Register the printed read-only deploy key. For genesis, run --apply on all
+# three Agency members concurrently; later agency:none joiners run alone.
+sudo fz bootstrap platform --bootstrap-config ./platform.json --apply
 sudo fz bootstrap tenant --bootstrap-config ./tenant.json --apply
 sudo fz bootstrap metal --bootstrap-config ./metal.json --apply
 sudo fz bootstrap status
 sudo fz bootstrap repair --bootstrap-config ./original.json --apply
 ```
 
+Interactive bootstrap prompts only for missing typed inputs and validates every
+answer before it changes the host. It must then render a secret-free plan, apply
+idempotently, and verify the external and local result. Initial platform bring-up
+has exactly three attended owner gates:
+
+1. SMTP host, port, sender, optional user, and an owner-only password file; the
+   resulting provider must pass its configured health/send acceptance.
+2. The separate Cloudflare phase consumes owner-only management and KV-runtime
+   token files, reconciles Tunnel/hostname state, and emits the node handoff; the
+   normal API/Agent never receives the management token.
+3. `prepare` prints that machine's read-only Git deploy public key. The owner adds
+   it to the repository and confirms completion; bootstrap then proves repository
+   access before the first deployment.
+
+Browser authentication is deliberately not automated away. If npm or another
+provider prints an authentication URL, leave the command attached to its TTY,
+open the URL, complete the passkey flow, and let the same command continue.
+
+Platform database participation is explicit in the private JSON configuration.
+The three genesis nodes use `database.agency: "member"`; later DB+API joiners
+normally use `database.agency: "none"`, which starts a Coordinator and DBServer
+without adding a fourth Agency member. Enrolment is likewise explicit:
+`enrolment.source: "genesis-derived"` only for the offline genesis fleet, or
+`"api-token"` with `tokenFile` for every API-authorized later node. No behavior
+is inferred from a hostname or numeric suffix.
+
+Repair is not a second installation path: the CLI detects the installed
+platform/tenant or metal state and dispatches to that profile's validator. It
+fails closed if both state formats are present or neither is present. A private
+intent journal binds an interrupted first apply, and later repair refuses a
+config whose immutable host identity differs from the installed profile. Status checks the persisted
+identity record, both Agent sockets, profile-owned systemd units and, for a
+platform host, the active API health endpoint, nginx configuration and live
+Coordinator-mode evidence.
+
 Platform bootstrap supports an elastic Community ArangoDB 3.11.14 fleet. The
 first three database-capable computes establish the writable cluster; later
 database joiners and API-only computes use the same command with their typed
@@ -70,21 +110,87 @@ but remains separate from the root host install so its management token never
 enters the API or Agent service environment:
 
 ```bash
-chmod 600 cloudflare-bootstrap.json cloudflare-management.token
+chmod 600 cloudflare-bootstrap.json cloudflare-management.token cloudflare-runtime-kv.token
 fz bootstrap platform cloudflare --bootstrap-config ./cloudflare-bootstrap.json
 fz bootstrap platform cloudflare --bootstrap-config ./cloudflare-bootstrap.json --apply
+# After every host is bootstrapped and publishing healthy KV state:
+fz bootstrap platform cloudflare verify --bootstrap-config ./cloudflare-handoff.json
 ```
 
+Tenant-owned public origins use the identical reviewed request and handoff with
+`fz bootstrap tenant cloudflare ...`; `fz bootstrap tenant` consumes only that
+tenant node's handoff. Both host kinds seal the connector and supplied KV token
+and can repair after the plaintext handoff has been removed.
+
 The reviewed JSON has `format: 1`, kind
 `forgezero-cloudflare-bootstrap-request`, a `checkpointPath`, the typed
-`coordinates`, and `tokenFiles` containing file paths only. `coordinates.nodes`
-explicitly lists each node name, origin hostname, loopback service, Tunnel name
-and Access application name. Relative token, checkpoint and Worker project
-paths resolve beside the config file. Plan is offline. Apply creates/reuses one
-shared KV namespace, Worker and Access service token plus one Tunnel, Access
-application, ingress rule and DNS record per node. The command prints only
-resource IDs; connector, Access and requested runtime-token values stay in the
-atomically updated owner-only checkpoint for direct systemd-credential handoff.
+`coordinates`, and token **file paths only**. The recommended token split is:
+
+```json
+{
+  "format": 1,
+  "kind": "forgezero-cloudflare-bootstrap-request",
+  "checkpointPath": "./cloudflare-handoff.json",
+  "coordinates": {
+    "accountId": "0123456789abcdef0123456789abcdef",
+    "zoneId": "0123456789abcdef0123456789abcdef",
+    "hostname": "dev-api-n1.example.com",
+    "service": "http://127.0.0.1:3000",
+    "tunnelName": "dev-api-n1",
+    "kvNamespaceId": "0123456789abcdef0123456789abcdef",
+    "nodes": [
+      {
+        "nodeName": "dev-api-n1",
+        "hostname": "dev-api-n1.example.com",
+        "service": "http://127.0.0.1:3000",
+        "tunnelName": "dev-api-n1"
+      }
+    ]
+  },
+  "tokenFiles": {
+    "managementApiTokenFile": "./cloudflare-management.token",
+    "runtimeApiTokenFile": "./cloudflare-runtime-kv.token"
+  }
+}
+```
+
+The management token has exact-account **Cloudflare Tunnel Write** and
+exact-zone **DNS Write**. The runtime token has exact-account **Workers KV
+Storage Write**. Apply creates or adopts one remotely managed Tunnel per node,
+checkpoints its connector token, configures the public-hostname ingress, and
+reconciles the exact proxied CNAME. DNS reconciliation lists the hostname
+without a type filter: it updates one existing CNAME, creates only when absent,
+and refuses ambiguity or an incompatible A/AAAA/other owner before changing
+Tunnel ingress. It does not create a Worker, KV namespace, Access application,
+Access service token, or Cloudflare API token.
+
+The Worker and its KV/DO bindings are deployed separately through Cloudflare's
+Git integration. Durable Objects are reached through the Worker's binding, not
+with a Cloudflare REST token. The complete 0600 checkpoint stays on the
+operator machine; `<checkpoint>.hosts/<node>.json` contains only that node's
+connector plus the supplied KV runtime token. Host bootstrap seals them as the
+`cloudflared-token` and `cloudflare-kv-token` systemd credentials and removes
+the plaintext node handoff. The API resolves `cloudflare.kvApiToken` from the
+platform Vault first and falls back to the same systemd credential while the
+Vault is locked or unavailable. Management credentials are never persisted or
+loaded into the API/Agent service.
+
+Platform and tenant bootstrap configs may declare a bounded
+`deploymentCredentials` map from the exact pipeline secret name to its
+encrypted `.cred` source, for example
+`{"SMTP_PASSWORD":"/etc/forgezero/creds/SMTP_PASSWORD.cred"}`. The supervised
+Agent loads only those named credentials. A deployment asks for the same name;
+the Agent reads its project-scoped in-memory Vault replica first and uses the
+same-name systemd credential only until the replica can supply it. There is no
+directory scan, cross-project lookup, or differently named fallback.
+
+This is a versioned runtime schema, not a naming convention. The enrolled
+binding supplies the exact project and environment; `credential-schema.ts`
+maps a requested name to that environment's Vault cache coordinate. Platform
+and tenant computes use this Vault-first policy. Physical metal has no project
+Vault and loads only its fixed `metal-agent-seed` systemd credential. Attended
+operator Cloudflare/bootstrap tokens are owner-only input files and are never a
+runtime fallback. The schema is exported as `@forgezero/agent/credential-schema`.
 
 ## What changes for an application
 

package/dist/agent-heartbeat.d.ts

@@ -11,6 +11,11 @@ export interface AgentObservation {
     architecture: string;
     mode: 'attested' | 'enrolled';
     update?: AgentUpdateReceipt;
+    deploymentIntake?: {
+        generation: number;
+        state: 'running' | 'paused';
+        appliedAtTs: number;
+    };
 }
 export interface AgentHeartbeatResponse {
     ok: true;
@@ -22,6 +27,10 @@ export interface AgentHeartbeatResponse {
         leaseExpiresAtTs: number;
         release: AgentRelease;
     };
+    deploymentIntake?: {
+        generation: number;
+        state: 'running' | 'paused';
+    };
 }
 export interface AgentHeartbeatOptions {
     apiUrl: string;
@@ -33,6 +42,10 @@ export interface AgentHeartbeatOptions {
     fetch?: (input: URL, init: RequestInit) => Promise<Response>;
     requestTimeoutMs?: number;
     observation?: () => AgentObservation;
+    /** Last applied server-owned deployment-intake generation, reported on the next heartbeat. */
+    deploymentIntakeObservation?: () => AgentObservation['deploymentIntake'];
+    /** Apply the authenticated desired state before any new deployment claim is allowed. */
+    applyDeploymentIntake?: (directive: NonNullable<AgentHeartbeatResponse['deploymentIntake']>) => Promise<void> | void;
     receiptPath?: string;
     now?: () => number;
     /** Stop new lifecycle/deploy claims and await current jobs before replacement. */

package/dist/agent-heartbeat.js

@@ -93,6 +93,8 @@ var checked = async (run, input, label) => {
   return result;
 };
 async function validateReleaseDirectory(directory, release, run) {
+  chmodSync(directory, 493);
+  chmodSync(join(directory, "dist"), 493);
   const manifest = JSON.parse(readFileSync(join(directory, "package.json"), "utf8"));
   if (manifest.name !== release.package || manifest.version !== release.version) {
     throw new Error("agent update manifest does not match the selected release");
@@ -143,19 +145,19 @@ async function stageAgentRelease(releaseInput, options) {
     if (!timingSafeEqual(actual, expected))
       throw new Error("agent update integrity mismatch");
     writeFileSync(archive, bytes, { mode: 384, flag: "wx" });
-    await checked(run, {
-      command: "/usr/bin/tar",
-      args: [
-        "-xzf",
-        archive,
-        "-C",
-        unpacked,
-        "--strip-components=1",
-        "package/package.json",
-        "package/dist/fz-agent.js",
-        "package/dist/fz.js"
-      ]
-    }, "agent update extraction");
+    for (const [member, relative] of [
+      ["package/package.json", "package.json"],
+      ["package/dist/fz-agent.js", "dist/fz-agent.js"],
+      ["package/dist/fz.js", "dist/fz.js"]
+    ]) {
+      const extracted = await checked(run, {
+        command: "/usr/bin/tar",
+        args: ["-xOzf", archive, member]
+      }, `agent update extraction of ${member}`);
+      const destination = join(unpacked, relative);
+      mkdirSync(dirname(destination), { recursive: true, mode: 448 });
+      writeFileSync(destination, extracted.output, { mode: 384, flag: "wx" });
+    }
     await validateReleaseDirectory(unpacked, release, run);
     if (!existsSync(finalDirectory)) {
       renameSync(unpacked, finalDirectory);
@@ -744,7 +746,7 @@ async function postSignedNode(options, path, body) {
 }
 
 // src/version.ts
-var VERSION3 = "0.1.38";
+var VERSION3 = "0.1.40";
 
 // src/agent-heartbeat.ts
 var unquote = (value) => value.replace(/^['"]|['"]$/g, "");
@@ -770,7 +772,15 @@ async function heartbeatAgentOnce(options) {
   const update = observation.update ?? readAgentUpdateReceipt(options.receiptPath ?? AGENT_UPDATE_RECEIPT);
   if (update)
     observation = { ...observation, update };
+  const deploymentIntake = options.deploymentIntakeObservation?.();
+  if (deploymentIntake)
+    observation = { ...observation, deploymentIntake };
   const response = await postSignedNode(options, "v1/node/heartbeat", observation);
+  if (response.deploymentIntake !== undefined) {
+    if (!Number.isSafeInteger(response.deploymentIntake.generation) || response.deploymentIntake.generation < 0 || response.deploymentIntake.state !== "running" && response.deploymentIntake.state !== "paused")
+      throw new Error("agent heartbeat deployment-intake directive is invalid");
+    await options.applyDeploymentIntake?.(response.deploymentIntake);
+  }
   const desired = response.desiredAgentUpdate ?? (response.desiredAgentRelease ? {
     attemptId: `legacy:${response.desiredAgentRelease.version}`,
     leaseExpiresAtTs: Number.MAX_SAFE_INTEGER,

package/dist/agent-update-helper.js

@@ -93,6 +93,8 @@ var checked = async (run, input, label) => {
   return result;
 };
 async function validateReleaseDirectory(directory, release, run) {
+  chmodSync(directory, 493);
+  chmodSync(join(directory, "dist"), 493);
   const manifest = JSON.parse(readFileSync(join(directory, "package.json"), "utf8"));
   if (manifest.name !== release.package || manifest.version !== release.version) {
     throw new Error("agent update manifest does not match the selected release");
@@ -143,19 +145,19 @@ async function stageAgentRelease(releaseInput, options) {
     if (!timingSafeEqual(actual, expected))
       throw new Error("agent update integrity mismatch");
     writeFileSync(archive, bytes, { mode: 384, flag: "wx" });
-    await checked(run, {
-      command: "/usr/bin/tar",
-      args: [
-        "-xzf",
-        archive,
-        "-C",
-        unpacked,
-        "--strip-components=1",
-        "package/package.json",
-        "package/dist/fz-agent.js",
-        "package/dist/fz.js"
-      ]
-    }, "agent update extraction");
+    for (const [member, relative] of [
+      ["package/package.json", "package.json"],
+      ["package/dist/fz-agent.js", "dist/fz-agent.js"],
+      ["package/dist/fz.js", "dist/fz.js"]
+    ]) {
+      const extracted = await checked(run, {
+        command: "/usr/bin/tar",
+        args: ["-xOzf", archive, member]
+      }, `agent update extraction of ${member}`);
+      const destination = join(unpacked, relative);
+      mkdirSync(dirname(destination), { recursive: true, mode: 448 });
+      writeFileSync(destination, extracted.output, { mode: 384, flag: "wx" });
+    }
     await validateReleaseDirectory(unpacked, release, run);
     if (!existsSync(finalDirectory)) {
       renameSync(unpacked, finalDirectory);

package/dist/agent-update.js

@@ -93,6 +93,8 @@ var checked = async (run, input, label) => {
   return result;
 };
 async function validateReleaseDirectory(directory, release, run) {
+  chmodSync(directory, 493);
+  chmodSync(join(directory, "dist"), 493);
   const manifest = JSON.parse(readFileSync(join(directory, "package.json"), "utf8"));
   if (manifest.name !== release.package || manifest.version !== release.version) {
     throw new Error("agent update manifest does not match the selected release");
@@ -143,19 +145,19 @@ async function stageAgentRelease(releaseInput, options) {
     if (!timingSafeEqual(actual, expected))
       throw new Error("agent update integrity mismatch");
     writeFileSync(archive, bytes, { mode: 384, flag: "wx" });
-    await checked(run, {
-      command: "/usr/bin/tar",
-      args: [
-        "-xzf",
-        archive,
-        "-C",
-        unpacked,
-        "--strip-components=1",
-        "package/package.json",
-        "package/dist/fz-agent.js",
-        "package/dist/fz.js"
-      ]
-    }, "agent update extraction");
+    for (const [member, relative] of [
+      ["package/package.json", "package.json"],
+      ["package/dist/fz-agent.js", "dist/fz-agent.js"],
+      ["package/dist/fz.js", "dist/fz.js"]
+    ]) {
+      const extracted = await checked(run, {
+        command: "/usr/bin/tar",
+        args: ["-xOzf", archive, member]
+      }, `agent update extraction of ${member}`);
+      const destination = join(unpacked, relative);
+      mkdirSync(dirname(destination), { recursive: true, mode: 448 });
+      writeFileSync(destination, extracted.output, { mode: 384, flag: "wx" });
+    }
     await validateReleaseDirectory(unpacked, release, run);
     if (!existsSync(finalDirectory)) {
       renameSync(unpacked, finalDirectory);

package/dist/bootstrap.d.ts

@@ -4,6 +4,8 @@ import { type PlatformSharedEnvironment } from './platform-bootstrap-runtime';
 export declare const PLATFORM_BOOTSTRAP_PROFILES: readonly ["platform-db-api", "platform-api"];
 export type PlatformBootstrapProfile = (typeof PLATFORM_BOOTSTRAP_PROFILES)[number];
 export type DatabaseBootstrapRole = 'master' | 'joiner' | 'none';
+export type DatabaseAgencyParticipation = 'member' | 'none';
+export type PlatformEnrolmentSource = 'genesis-derived' | 'api-token';
 export type BootstrapEnvironment = 'production' | 'development';
 export interface PlatformBootstrapConfig {
     kind: 'platform';
@@ -17,8 +19,16 @@ export interface PlatformBootstrapConfig {
     branch: string;
     deployRoot?: string;
     telemetryEndpoint: string;
+    /**
+     * Pipeline secret name -> encrypted systemd credential source. The Agent asks
+     * its project Vault replica first and uses only the same-name entry here while
+     * that replica is unavailable or has not received the secret yet.
+     */
+    deploymentCredentials?: Record<string, string>;
     database: {
         role: DatabaseBootstrapRole;
+        /** Genesis master/joiners are Agency members; elastic joiners normally use `none`. */
+        agency: DatabaseAgencyParticipation;
         /** Cluster mode must remain default. Bootstrap verifies it and never mutates it. */
         serverMode: 'default';
         address?: string;
@@ -27,8 +37,12 @@ export interface PlatformBootstrapConfig {
         /** Owner/root-only file containing the shared 64-hex bootstrap code. */
         bootstrapSecretFile: string;
     };
-    /** Required for every post-genesis compute; initial n1..n3 derive their one-use capability. */
-    platformEnrolTokenFile?: string;
+    /** Explicit authority source; never inferred from a node number. */
+    enrolment: {
+        source: PlatformEnrolmentSource;
+        /** Required only for an API-issued post-genesis capability. */
+        tokenFile?: string;
+    };
     runtime: {
         environment: PlatformSharedEnvironment;
         serviceUser: string;
@@ -49,9 +63,9 @@ export interface PlatformBootstrapConfig {
     };
     /** Install the connector binary only. Resource creation belongs to attended Cloudflare bootstrap. */
     installCloudflared?: boolean;
-    /** Completed attended resource checkpoint; root seals only this node's runtime capabilities. */
+    /** Node-specific attended handoff; a host must never receive the fleet checkpoint. */
     cloudflareHandoff?: {
-        checkpointFile: string;
+        handoffFile: string;
         nodeName: string;
     };
 }
@@ -61,6 +75,8 @@ export interface TenantBootstrapConfig {
     realm: string;
     nodeHostname: string;
     telemetryEndpoint: string;
+    /** Same Vault-first, same-name systemd fallback contract as platform deploys. */
+    deploymentCredentials?: Record<string, string>;
     enrolTokenFile: string;
     repository?: string;
     branch?: string;
@@ -68,6 +84,11 @@ export interface TenantBootstrapConfig {
     deployRoot?: string;
     software?: SoftwareRequirement[];
     installCloudflared?: boolean;
+    /** Optional node-specific attended Tunnel/KV handoff, identical to platform hosts. */
+    cloudflareHandoff?: {
+        handoffFile: string;
+        nodeName: string;
+    };
     /** Local-only runner authority. The API receives only the resulting node scope. */
     bootstrapRunner?: {
         sshPrivateKeyFile: string;
@@ -97,6 +118,13 @@ export interface BootstrapResult {
         inviteUrl: string;
     };
 }
+export interface PlatformBootstrapPreparation {
+    kind: 'platform';
+    prepared: true;
+    identityDigest: string;
+    gitPublicKey: string;
+    next: 'register this read-only deploy key, then run --apply concurrently on all three genesis Agency members';
+}
 export interface BootstrapStatus {
     initialized: boolean;
     kind?: BootstrapConfig['kind'];
@@ -104,6 +132,12 @@ export interface BootstrapStatus {
     services: Record<string, boolean>;
     problems: string[];
 }
+export type InstalledBootstrapKind = 'compute' | 'metal';
+/** One host has one bootstrap authority. Dual state is never resolved by precedence. */
+export declare function resolveInstalledBootstrapKind(states: {
+    compute: boolean;
+    metal: boolean;
+}): InstalledBootstrapKind | undefined;
 export interface BootstrapHost {
     uid(): number;
     exists(path: string): boolean;
@@ -122,11 +156,22 @@ export interface BootstrapHost {
     exec(argv: readonly string[], options?: {
         stdin?: string;
     }): Promise<SoftwareCommandResult>;
+    sleep?(milliseconds: number): Promise<void>;
     ensureSoftware(requirements: readonly SoftwareRequirement[]): Promise<unknown>;
     installAgent(config: BootstrapConfig, enrolTokenSourcePath?: string): Promise<ProvisionPlan>;
 }
+export declare const BOOTSTRAP_STATE_PATH = "/var/lib/forgezero/bootstrap.json";
 export declare function validateBootstrapConfig(value: BootstrapConfig): BootstrapConfig;
 export declare function planBootstrap(input: BootstrapConfig, initialized?: boolean): BootstrapPlan;
+export declare function bootstrapIdentityDigest(config: BootstrapConfig): string;
+/**
+ * Install only the common Agent boundary and generate its machine-local Git
+ * identity. Private repositories cannot be deployed until the owner registers
+ * this public key, so pretending one monolithic first command can do both is a
+ * sequencing bug. The bound intent makes the later apply resumable and refuses
+ * a different config after this first privileged mutation.
+ */
+export declare function preparePlatformBootstrap(input: PlatformBootstrapConfig, host?: BootstrapHost): Promise<PlatformBootstrapPreparation>;
 export declare function bootstrapStatus(host?: BootstrapHost): Promise<BootstrapStatus>;
 export declare function applyBootstrap(input: BootstrapConfig, host?: BootstrapHost): Promise<BootstrapResult>;
 export declare function readBootstrapConfig(path: string): BootstrapConfig;