@checkstack/satellite-backend 0.5.1 → 0.6.1

package/CHANGELOG.md

@@ -1,5 +1,104 @@
 # @checkstack/satellite-backend
 
+## 0.6.1
+
+### Patch Changes
+
+- Updated dependencies [13373ce]
+  - @checkstack/common@0.14.0
+  - @checkstack/backend-api@0.21.1
+  - @checkstack/queue-api@0.3.10
+  - @checkstack/automation-backend@0.5.1
+  - @checkstack/automation-common@0.4.1
+  - @checkstack/gitops-backend@0.5.1
+  - @checkstack/gitops-common@0.6.1
+  - @checkstack/healthcheck-backend@1.6.1
+  - @checkstack/healthcheck-common@1.5.1
+  - @checkstack/satellite-common@0.8.1
+  - @checkstack/script-packages-backend@0.3.1
+  - @checkstack/script-packages-common@0.3.1
+  - @checkstack/secrets-backend@0.2.1
+  - @checkstack/secrets-common@0.2.1
+  - @checkstack/signal-common@0.2.7
+
+## 0.6.0
+
+### Minor Changes
+
+- 9dcc848: Layered OS-level script sandbox, secure and fail-closed by default (epic #247).
+
+  Script and shell health checks and the `run_shell` / `run_script` automation actions now run inside a layered OS-level sandbox by default. The sandbox lives in `core/backend-api/src/script-sandbox/` (the single source of truth) and is enforced inside the shared runners, so it applies wherever a job runs.
+
+  Layers:
+
+  - Resource caps (CPU / memory / PID / FD / file-size, via `prlimit` on capable Linux; ESM JS-heap cap via `--max-old-space-size`; portable wall-clock timeout) and an OOM-safe streaming output cap.
+  - Privilege drop via a NON-ROOT supervisor model: the shipped images run the supervisor as non-root uid `65532`, so every sandboxed script inherits non-root and can never be host-root; filesystem + network confinement is delivered by ROOTLESS `bwrap`/`nsjail` via unprivileged user namespaces. `enforced.privilege` is truthful (true only when the child cannot run as host-root). Runners no longer pass `uid`/`gid` to `Bun.spawn` (a silent no-op and a forward-compat hazard).
+  - Filesystem isolation (`scratch-only` / `scratch-plus-ro`) confining the child to its per-run scratch dir over a read-only base; the interpreter path is RO-bound so the runtime execs, and `TMPDIR` is pinned to the in-namespace tmpfs.
+  - Network egress control: `deny` (routeless loopback-only netns), `allowlist` (real plumbed egress via macvlan OR rootless slirp4netns + an in-kernel nftables filter), and an always-on metadata / link-local block (`169.254.0.0/16`, `fe80::/10`, `fc00::/7`). No-blackhole invariant: `enforced.network` is never true when egress is actually severed or unfiltered; unpluggable egress degrades to surfaced host net.
+  - Per-run fork-bomb containment via RLIMIT*NPROC inside the fresh per-run user+PID namespace; a centralized forbidden-env denylist (`LD_PRELOAD`, `LD_LIBRARY_PATH`, `DYLD*_`, `NODE*OPTIONS`, `BUN*_`, caller `PATH` overrides).
+  - A validated tuned seccomp profile (`deploy/seccomp/checkstack-userns.json`) and a live `clone(CLONE_NEWUSER|CLONE_NEWNET)` capability probe (not the static sysctl), shipped by default in both Dockerfiles, `docker-compose.yml`, and `deploy/k8s/checkstack-sandbox.yaml`.
+
+  Global policy and operator surface:
+
+  - The global sandbox policy lives in ONE durable row owned by `script-packages` (its `ConfigService` row in shared `plugin_configs`). A single process-wide provider serves every runner; the two script plugins no longer register competing providers. A dedicated admin-only `script-sandbox.manage` permission gates both reading and writing the policy. New `getSandboxPolicy` / `setSandboxPolicy` endpoints and a Settings -> Script Sandbox admin UI (`enabled`, `onUnavailable`, network/filesystem/privilege modes, allow list, metadata block, resource caps). The startup capability/readiness log is emitted in-process by `script-packages-backend` (no fragile init-order RPC self-loop), and on a host that cannot enforce a layer a one-time startup warning explains the two local-dev paths (Docker, or set the global policy to `degrade`).
+  - Satellite relay: the WS protocol carries the resolved policy in the `authenticated` message and a `sandbox_policy` push-on-change; a satellite caches the last relayed policy and resolves every run through it.
+
+  BREAKING CHANGES (platform in BETA, shipped as minor):
+
+  - Scripts run sandboxed by default. The shipped global default is FAIL-CLOSED (`onUnavailable: "fail"`): when a requested layer cannot be enforced the run is REFUSED (clean `exitCode: -1`, never an unsandboxed spawn) rather than silently degrading. Deployments on hosts that cannot enforce a layer (no bubblewrap, user namespaces blocked, no `/proc` unmask) must run the official images with the documented runtime flags (the bundled seccomp profile + `systempaths=unconfined`, or k8s `procMount: Unmasked`), or set the global policy to `degrade`. On macOS / restricted containers the strong layers degrade to the portable subset and are surfaced per run.
+  - Default network posture is deny-egress (`allowlist` with an empty allow list, which resolves to the routeless `deny` path). Scripts calling external endpoints fail until those destinations are allowlisted in the global default. The always-on metadata / link-local block applies even under looser modes.
+  - The per-action / per-check `sandbox` config override and the transport `ScriptRequest.sandbox` field are removed; policy is global-only, so an automation/check author can no longer weaken the sandbox on their own item. Stored configs carrying a stray `sandbox` key are tolerated (stripped on parse).
+  - The shared runners' `run()` no longer accepts a `sandbox` option; callers rely on the global policy provider.
+  - A satellite fails closed (most restrictive profile) until it receives the first relayed policy; a relay-read failure or an older core keeps it fail-closed. A relay failure can never loosen a satellite's sandbox.
+
+  State and scale: the global policy is a single durable Postgres row read identically on every pod. Capability detection is per-process, deterministic from the host kernel, and surfaced per run via the `EffectiveSandbox` report (a Linux pod and a macOS satellite may legitimately differ). `CHECKSTACK_SANDBOX_UID/GID` and macvlan addressing are genuinely per-host infrastructure, surfaced per run, not the queryable policy. The satellite's policy cache is satellite-local transport state. No new pod-local current-state.
+
+  This is a beta minor.
+
+- 9dcc848: Align workspace dependency versions and migrate React Router to v7.
+
+  BREAKING CHANGES (React Router v7): All frontend packages now depend on `react-router-dom@^7.16.0`. Previously the workspace declared four divergent ranges (`^6.20.0`, `^6.22.0`, `^7.1.1`, `^7.14.2`), which resolved both `react-router@6` and `react-router@7` into a single bundle. Everything is now unified on v7. The public imports the app uses (`BrowserRouter`, `Routes`, `Route`, `Link`, `NavLink`, `MemoryRouter`, `useNavigate`, `useParams`, `useSearchParams`, `useLocation`) are unchanged between v6 and v7, so no source rewrites were required - but any out-of-tree plugin still on react-router v6 should upgrade to v7 (see the React Router v6 -> v7 upgrade guide) to share the host's single router instance via the import map.
+
+  Other unified ranges (no API change): `react` -> `^18.3.1`, the `@orpc/*` family (`contract`, `server`, `client`, `tanstack-query`, `openapi`, `zod`) -> `^1.14.4`, and `better-auth` -> `^1.6.13`.
+
+  Removed the pre-rename `@orpc/react-query` leftover from `@checkstack/frontend-api`; its `createRouterUtils` / `RouterUtils` / `ProcedureUtils` now come from `@orpc/tanstack-query` (the package already in use).
+
+  Stale in-range runtime deps pulled up to current published versions: `hono` `^4.12.23`, `@tanstack/react-query` (+devtools) `^5.100.14`, `date-fns` `^4.4.0`, `jose` `^6.2.3`, `tar` `^7.5.16`, `semver` `^7.8.1`, `@xyflow/react` `^12.11.0`.
+
+### Patch Changes
+
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+- Updated dependencies [9dcc848]
+  - @checkstack/backend-api@0.21.0
+  - @checkstack/healthcheck-backend@1.6.0
+  - @checkstack/healthcheck-common@1.5.0
+  - @checkstack/automation-backend@0.5.0
+  - @checkstack/automation-common@0.4.0
+  - @checkstack/common@0.13.0
+  - @checkstack/script-packages-common@0.3.0
+  - @checkstack/script-packages-backend@0.3.0
+  - @checkstack/satellite-common@0.8.0
+  - @checkstack/gitops-backend@0.5.0
+  - @checkstack/gitops-common@0.6.0
+  - @checkstack/secrets-backend@0.2.0
+  - @checkstack/secrets-common@0.2.0
+  - @checkstack/queue-api@0.3.9
+  - @checkstack/signal-common@0.2.6
+
 ## 0.5.1
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@checkstack/satellite-backend",
-  "version": "0.5.1",
+  "version": "0.6.1",
   "license": "Elastic-2.0",
   "type": "module",
   "main": "src/index.ts",
@@ -15,29 +15,29 @@
     "test": "bun test"
   },
   "dependencies": {
-    "@checkstack/backend-api": "0.18.0",
-    "@checkstack/automation-backend": "0.2.0",
-    "@checkstack/automation-common": "0.2.0",
-    "@checkstack/satellite-common": "0.6.0",
-    "@checkstack/healthcheck-common": "1.3.0",
-    "@checkstack/signal-common": "0.2.5",
-    "@checkstack/healthcheck-backend": "1.3.0",
-    "@checkstack/script-packages-common": "0.1.0",
-    "@checkstack/script-packages-backend": "0.1.0",
-    "@checkstack/secrets-common": "0.0.1",
-    "@checkstack/secrets-backend": "0.0.1",
-    "@checkstack/gitops-backend": "0.3.7",
-    "@checkstack/gitops-common": "0.4.2",
-    "@checkstack/common": "0.12.0",
-    "@checkstack/queue-api": "0.3.6",
+    "@checkstack/backend-api": "0.21.0",
+    "@checkstack/automation-backend": "0.5.0",
+    "@checkstack/automation-common": "0.4.0",
+    "@checkstack/satellite-common": "0.8.0",
+    "@checkstack/healthcheck-common": "1.5.0",
+    "@checkstack/signal-common": "0.2.6",
+    "@checkstack/healthcheck-backend": "1.6.0",
+    "@checkstack/script-packages-common": "0.3.0",
+    "@checkstack/script-packages-backend": "0.3.0",
+    "@checkstack/secrets-common": "0.2.0",
+    "@checkstack/secrets-backend": "0.2.0",
+    "@checkstack/gitops-backend": "0.5.0",
+    "@checkstack/gitops-common": "0.6.0",
+    "@checkstack/common": "0.13.0",
+    "@checkstack/queue-api": "0.3.9",
     "drizzle-orm": "^0.45.0",
     "zod": "^4.2.1",
-    "@orpc/server": "^1.13.2"
+    "@orpc/server": "^1.14.4"
   },
   "devDependencies": {
     "@checkstack/drizzle-helper": "0.0.5",
-    "@checkstack/scripts": "0.3.4",
-    "@checkstack/test-utils-backend": "0.1.31",
+    "@checkstack/scripts": "0.4.0",
+    "@checkstack/test-utils-backend": "0.1.34",
     "@checkstack/tsconfig": "0.0.7",
     "@types/bun": "^1.0.0",
     "@types/pg": "^8.20.0",

package/src/heartbeat-monitor.it.test.ts

@@ -1,7 +1,7 @@
 /**
  * Integration test (real Postgres): cross-pod heartbeat-lost detection.
  *
- * This is the DETERMINISTIC backstop for `.agent/rules/state-and-scale.md` that
+ * This is the DETERMINISTIC backstop for `.claude/rules/state-and-scale.md` that
  * the single-process unit suite structurally cannot provide for the heartbeat
  * monitor. The prior fix made connection STATUS durable but left the
  * online→offline transition DETECTION pod-local (an in-memory `previousStatuses`

package/src/index.ts

@@ -10,7 +10,10 @@ import {
 import { HealthCheckApi } from "@checkstack/healthcheck-common";
 import { healthCheckHooks } from "@checkstack/healthcheck-backend";
 import { ScriptPackagesApi } from "@checkstack/script-packages-common";
-import { scriptPackagesChangedHook } from "@checkstack/script-packages-backend";
+import {
+  scriptPackagesChangedHook,
+  sandboxPolicyChangedHook,
+} from "@checkstack/script-packages-backend";
 import { secretResolverRef } from "@checkstack/secrets-backend";
 import { resolveSatelliteRunSecrets } from "./run-secret-resolver";
 import { SatelliteService } from "./service";
@@ -256,6 +259,17 @@ export default createBackendPlugin({
                 resolver: secretResolver,
               }),
           },
+          {
+            // Global sandbox-policy relay: carry the resolved cluster-wide
+            // policy in the `authenticated` payload so a satellite enforces it
+            // from its first run, and push it on change. A satellite stays
+            // FAIL-CLOSED (deny egress) until this first relay arrives, so a
+            // read failure here can never loosen its sandbox.
+            getCurrentPolicy: async () => {
+              const spClient = rpcClient.forPlugin(ScriptPackagesApi);
+              return spClient.getSandboxPolicy();
+            },
+          },
         );
 
         // Register satellite WebSocket endpoint via the scoped WS registry
@@ -343,6 +357,18 @@ export default createBackendPlugin({
           { mode: "broadcast" },
         );
 
+        // Fan the global sandbox-policy change out to THIS instance's connected
+        // satellites (push-on-change relay). Broadcast mode so every core pod
+        // pushes to its own satellites; offline satellites converge via the
+        // policy carried in `authenticated` on reconnect.
+        onHook(
+          sandboxPolicyChangedHook,
+          async ({ policy }) => {
+            wsHandler.pushSandboxPolicyToAll(policy);
+          },
+          { mode: "broadcast" },
+        );
+
         logger.debug("✅ Satellite Backend afterPluginsReady complete.");
       },
     });

package/src/satellite-ws-handler.test.ts

@@ -529,4 +529,89 @@ describe("SatelliteWsHandler", () => {
       expect(mirrors).toHaveLength(0);
     });
   });
+
+  describe("global sandbox-policy relay", () => {
+    const POLICY = {
+      enabled: true,
+      onUnavailable: "degrade" as const,
+      resources: { cpuSeconds: 33 },
+      filesystem: { mode: "scratch-plus-ro" as const },
+      network: {
+        mode: "deny" as const,
+        allow: [] as string[],
+        denyLinkLocalAndMetadata: true,
+      },
+      privilege: { mode: "drop-to-uid" as const },
+    };
+
+    function makeSandboxSink() {
+      return {
+        sink: {
+          getCurrentPolicy: mock(async () => POLICY),
+        },
+      };
+    }
+
+    async function authedHandlerWithSandboxSink() {
+      const { sink } = makeSandboxSink();
+      const h = new SatelliteWsHandler(
+        service,
+        configRelay,
+        resultHandler,
+        logger,
+        undefined,
+        undefined,
+        undefined,
+        sink,
+      );
+      const ws = createMockWs();
+      const { onMessage } = h.onConnection(ws);
+      await onMessage(
+        JSON.stringify({
+          type: "authenticate",
+          clientId: "sat-1",
+          token: "csat_valid-token",
+        }),
+      );
+      return { h, ws };
+    }
+
+    it("carries the resolved policy in the authenticated payload", async () => {
+      const { ws } = await authedHandlerWithSandboxSink();
+      const auth = JSON.parse(ws.messages[0]);
+      expect(auth.type).toBe("authenticated");
+      expect(auth.sandboxPolicy.network.mode).toBe("deny");
+      expect(auth.sandboxPolicy.resources.cpuSeconds).toBe(33);
+    });
+
+    it("omits sandboxPolicy entirely when no sink is wired (version-skew safe)", async () => {
+      const ws = createMockWs();
+      const { onMessage } = handler.onConnection(ws);
+      await onMessage(
+        JSON.stringify({
+          type: "authenticate",
+          clientId: "sat-1",
+          token: "csat_valid-token",
+        }),
+      );
+      const auth = JSON.parse(ws.messages[0]);
+      expect("sandboxPolicy" in auth).toBe(false);
+    });
+
+    it("pushes sandbox_policy to every connected satellite on change", async () => {
+      const { h, ws } = await authedHandlerWithSandboxSink();
+      ws.messages.length = 0;
+
+      h.pushSandboxPolicyToAll({
+        ...POLICY,
+        network: { mode: "allowlist", allow: ["10.0.0.1"], denyLinkLocalAndMetadata: true },
+      });
+
+      expect(ws.messages).toHaveLength(1);
+      const msg = JSON.parse(ws.messages[0]);
+      expect(msg.type).toBe("sandbox_policy");
+      expect(msg.policy.network.mode).toBe("allowlist");
+      expect(msg.policy.network.allow).toEqual(["10.0.0.1"]);
+    });
+  });
 });

package/src/satellite-ws-handler.ts

@@ -14,6 +14,7 @@ import {
   type ResultMessage,
   type SatelliteWithStatus,
 } from "@checkstack/satellite-common";
+import type { SandboxPolicy } from "@checkstack/common";
 
 /**
  * Optional plug-point for driving a satellite connection lifecycle edge into
@@ -90,6 +91,18 @@ export interface SatelliteSecretSink {
   }): Promise<Record<string, string>>;
 }
 
+/**
+ * Optional plug-point for relaying the GLOBAL script-sandbox policy to
+ * satellites. Wired from `afterPluginsReady` against the script-packages RPC.
+ * When absent, the `authenticated` message omits `sandboxPolicy` (version-skew
+ * safe) and the satellite stays FAIL-CLOSED until a policy arrives, so a
+ * missing sink can never loosen a satellite's sandbox.
+ */
+export interface SatelliteSandboxPolicySink {
+  /** The current resolved global sandbox policy to relay to satellites. */
+  getCurrentPolicy(): Promise<SandboxPolicy>;
+}
+
 /**
  * Active satellite connection tracking.
  */
@@ -138,6 +151,12 @@ export class SatelliteWsHandler implements WebSocketRouteHandler {
      * unset, such a request is answered with an error.
      */
     private secretSink?: SatelliteSecretSink,
+    /**
+     * Optional. When set, the `authenticated` message carries the resolved
+     * global sandbox policy and the handler can push `sandbox_policy` on change.
+     * When unset, the field is omitted and the satellite stays fail-closed.
+     */
+    private sandboxPolicySink?: SatelliteSandboxPolicySink,
   ) {}
 
   /**
@@ -220,6 +239,7 @@ export class SatelliteWsHandler implements WebSocketRouteHandler {
           await this.configRelay.getAssignmentsForSatellite(satellite.id);
         const scriptPackagesLockfileHash =
           await this.resolveDesiredLockfileHash();
+        const sandboxPolicy = await this.resolveSandboxPolicy();
 
         this.sendMessage(ws, {
           type: "authenticated",
@@ -228,6 +248,7 @@ export class SatelliteWsHandler implements WebSocketRouteHandler {
           ...(scriptPackagesLockfileHash === undefined
             ? {}
             : { scriptPackagesLockfileHash }),
+          ...(sandboxPolicy === undefined ? {} : { sandboxPolicy }),
         });
 
         this.logger.info(
@@ -420,6 +441,38 @@ export class SatelliteWsHandler implements WebSocketRouteHandler {
     );
   }
 
+  /**
+   * Push the new global sandbox policy to EVERY connected satellite. Called by
+   * the `script-sandbox.policy-changed` broadcast handler so each core instance
+   * fans the change out to its own satellites (push-on-change relay).
+   * Best-effort liveness; the policy carried in `authenticated` on (re)connect
+   * is the durable backstop.
+   */
+  pushSandboxPolicyToAll(policy: SandboxPolicy): void {
+    for (const conn of this.connections.values()) {
+      this.sendMessage(conn.ws, { type: "sandbox_policy", policy });
+    }
+    this.logger.debug(
+      `Pushed sandbox_policy to ${this.connections.size} satellite(s)`,
+    );
+  }
+
+  /**
+   * Resolve the current global sandbox policy for the `authenticated` payload.
+   * Returns `undefined` when the sink isn't wired or its read throws, so the
+   * field is omitted (version-skew safe) and the satellite stays FAIL-CLOSED
+   * (denies egress) - a relay failure must never loosen a satellite's sandbox.
+   */
+  private async resolveSandboxPolicy(): Promise<SandboxPolicy | undefined> {
+    if (!this.sandboxPolicySink) return undefined;
+    try {
+      return await this.sandboxPolicySink.getCurrentPolicy();
+    } catch (error) {
+      this.logger.error("Failed to resolve global sandbox policy:", error);
+      return undefined;
+    }
+  }
+
   /**
    * Resolve the desired lockfile hash for assignment payloads. Returns
    * `undefined` when the sink isn't wired (so the field is omitted entirely