@questpie/sandbox 3.16.0 → 3.18.0

package/README.md

@@ -1,15 +1,18 @@
 # @questpie/sandbox
 
-Hardened sandboxed code execution for QUESTPIE executor workloads. The package ships the HTTP adapter used by a QUESTPIE app and the Deno supervisor source that executes untrusted code in a separate process with capability-scoped network, import, file, and app bindings.
+Hardened isolated-script execution for QUESTPIE. The package ships the HTTP
+adapter used by a QUESTPIE app and the Deno supervisor source that executes each
+untrusted script in a fresh, capability-scoped subprocess.
 
 ## Features
 
-- **HTTP executor adapter** - connect `ctx.executor.run()` to a standalone sandbox service.
-- **Deno supervisor** - run guest code outside the main Bun/Node app process.
-- **Capability-scoped bindings** - expose only declared collections, stores, files, and fetch access.
-- **SSRF protection** - validate literal IPs, DNS rebinding, redirects, and private/metadata ranges.
-- **Network isolation support** - Linux network namespace/firewall planning for defense in depth.
-- **Structured failures** - timeouts, non-JSON responses, and network failures return predictable results.
+- HTTP `ExecutorAdapter` for `ctx.executor.run()`
+- fresh Deno subprocess with bounded wall time and memory
+- capability-scoped collections, stores, files, and network bindings
+- explicitly released custom MCP tools through a separate guest namespace
+- SSRF, redirect, private-address, and metadata-address protection
+- optional Linux network namespace/firewall defense in depth
+- generic, consumer-supplied workload authorization
 
 ## Installation
 
@@ -17,10 +20,9 @@ Hardened sandboxed code execution for QUESTPIE executor workloads. The package s
 bun add @questpie/sandbox
 ```
 
-## Configure The Adapter
+## Trusted framework execution
 
 ```ts
-// config/app.ts
 import { httpSandboxAdapter } from "@questpie/sandbox/adapter";
 import { appConfig } from "questpie";
 
@@ -28,30 +30,213 @@ export default appConfig({
 	executor: {
 		sandboxed: httpSandboxAdapter({
 			url: process.env.SANDBOX_URL,
+			hostAdmissionSecret: process.env.SANDBOX_HOST_ADMISSION_SECRET,
 		}),
 	},
 });
 ```
 
-Guest bindings require a host-side broker route in the app or control-plane layer. Compute-only sandboxed execution only needs the adapter above.
+The adapter and supervisor must share the same random
+`SANDBOX_HOST_ADMISSION_SECRET` of at least 32 bytes. Omitting the credential
+fails closed. `SANDBOX_URL` must be a canonical HTTP(S) origin with no userinfo,
+fragment, query, or path prefix. The adapter never follows `/run` redirects, so
+the host admission credential cannot cross origins. Broker coordinates are still minted by QUESTPIE's core executor
+and remain capability-scoped. Any execution with bindings also requires the
+supervisor's canonical `SANDBOX_BROKER_URL`; missing, malformed, or non-matching
+protocol, host, effective port, path, or query is denied before spawn. Userinfo
+and fragments are forbidden; only one trailing path slash is treated as
+equivalent. The supervisor fetches the validated canonical URL, never the raw
+request spelling.
+
+Bindings guests always run with `--allow-net=[]`. Their declared `net`
+allowlist remains host-side and is used only by the resolve/validate/pin HTTP
+broker. The backwards-compatible compute-only path may still receive a direct
+Deno network grant.
+
+## Generic workload authorization
+
+A remote consumer supplies its own opaque envelope and authorizer. QUESTPIE does
+not prescribe, inspect, or persist the envelope. The authorizer returns the
+complete sandbox policy for one execution; no source, input, process capability,
+secret, or broker binding can be supplied beside the envelope.
 
-## Run The Supervisor
+```ts
+import { httpSandboxAdapter } from "@questpie/sandbox";
+
+const adapter = httpSandboxAdapter({
+	url: process.env.SANDBOX_URL,
+	workload: {
+		admission: {
+			keyId: process.env.SANDBOX_WORKLOAD_ADMISSION_KEY_ID!,
+			secret: new TextEncoder().encode(
+				process.env.SANDBOX_WORKLOAD_ADMISSION_SECRET!,
+			),
+			instanceId: process.env.SANDBOX_INSTANCE_ID!,
+		},
+		authorize: async (opaqueEnvelope, { phase, signal }) => {
+			const authorization = await consumerAuthority.authorize(opaqueEnvelope, {
+				phase,
+				signal,
+			});
+			if (!authorization) return null;
+
+			return {
+				source: authorization.source,
+				input: authorization.input,
+				capabilities: {
+					net: authorization.networkHosts,
+					import: authorization.importHosts,
+					timeoutMs: 5_000,
+					memoryMb: 128,
+				},
+				secrets: {},
+				validUntil: authorization.validUntil,
+			};
+		},
+		audit: writeRedactedSandboxDecision,
+	},
+});
+
+const result = await adapter.runWorkload({ envelope: consumerEnvelope });
+```
+
+The authorizer is called at preparation and immediately before dispatch. Both
+normalized policies must be identical. Authorization and audit callbacks have
+hard deadlines, caller cancellation is propagated, and malformed/missing
+authorization fails closed. Policy input, secrets, bindings, source, combined
+egress hosts, wall time, memory, and the final request body all have hard bounds.
+Cancellation after dispatch aborts broker relays and terminates the guest with a
+bounded `SIGTERM`/`SIGKILL` sequence.
+
+The subprocess boundary uses one 5 MiB decoded HTTP body budget. Its
+method-aware request, response, frame, and cumulative-output limits are derived
+from the exact base64 expansion plus bounded JSON-envelope headroom. Native
+binding results and custom-tool results remain capped at 768 KiB. Stderr is
+independently capped at 256 KiB, binding calls at 256 per run, and concurrent
+broker relays at 16. Crossing a limit kills the guest and aborts outstanding
+relays. Broker and supervisor HTTP responses are streamed through byte caps and
+parsed against exact schemas; broker-provided error text is replaced by
+supervisor-owned stable messages before it can enter the guest. Broker and
+`/run` redirects are rejected. Any broker denial or malformed response latches
+a terminal run failure; guest `catch` code cannot turn it back into success. A
+valid result frame is terminal immediately, and a result racing an in-flight
+RPC is rejected.
+
+The adapter then signs a minimal, product-neutral transport admission. It is
+bound to the exact request body and one supervisor instance, expires within five
+seconds, and is consumed once. The supervisor rejects missing, forged, expired,
+replayed, wrong-instance, and body-mismatched admissions before spawning a
+subprocess. Consumer audit events distinguish authorization decisions from the
+terminal transport outcome and include only decision plus phase/reason. The
+supervisor's transport log additionally includes its own instance id. Neither
+surface contains the opaque envelope, source, inputs, secrets, or broker token.
+
+`SANDBOX_INSTANCE_ID` is a process incarnation, not a deployment or replica id.
+It must be newly generated for every supervisor process and restart. The
+adapter's `SANDBOX_URL` must route directly to the one process named by its
+configured instance id; do not put this admission path behind an unpinned
+round-robin load balancer. Reusing one instance id across replicas is forbidden.
+Replay state is deliberately process-local because the admission is
+cryptographically bound to that unique process incarnation.
+
+## Custom tools
+
+Custom `mcpTool(...)` definitions can be exposed to a sandbox without turning
+MCP CRUD into the sandbox data API. Collections, files, and stores keep using
+their native broker primitives; only explicitly released custom tools enter the
+separate `questpie.tools` namespace.
+
+Configure the host-side consumer authorization port in `questpie.config.ts`:
+
+```ts
+import { sandboxCustomTools } from "@questpie/sandbox";
+import { runtimeConfig } from "questpie/app";
+
+export default runtimeConfig({
+	// db, executor, …
+	sandboxCustomTools: sandboxCustomTools({
+		authorizer: consumerToolAuthority,
+		contextBinder: consumerQuestpieContextBinder,
+		evidence: async (event) => {
+			// Product-neutral: phase, tool name, result, reason, duration only.
+			await evidenceSink.write(event);
+		},
+	}),
+});
+```
+
+The tool itself must be in the released MCP custom-tool catalog and declare an
+explicit workload requirement. Omitted exposure or workload policy remains
+invisible and uncallable. Every call repeats consumer authorization, binds a
+fresh user-mode QUESTPIE context, rechecks the tool access/scope policy, parses
+the original input schema, and invokes the original handler.
+
+The host supplies only an opaque envelope for the run:
+
+```ts
+const result = await ctx.executor.run({
+	isolation: "sandboxed",
+	source: `
+		export default async () => {
+			const { tools } = await globalThis.questpie.tools.list();
+			return globalThis.questpie.tools.call("reports.generate", {
+				period: "week",
+			});
+		}
+	`,
+	brokerUrl: "https://app.internal/api/sandbox/rpc",
+	sandboxTools: { envelope: consumerEnvelope },
+	capabilities: {
+		net: [],
+		import: [],
+		timeoutMs: 5_000,
+		memoryMb: 128,
+	},
+});
+```
+
+The adapter keeps the envelope in the app host. The broker endpoint and token
+are sent only to the trusted supervisor through the existing bindings
+coordinate; none of them is exposed to guest code or `globalThis.questpie`.
+Discovery, arguments, results, operation count, wall time, and concurrency are
+bounded. Evidence callbacks receive no envelope, arguments, result body,
+bearer credential, app instance, database handle, or request context.
+
+Defaults/hard maxima are: 64/256 discovered tools, 256 KiB/768 KiB discovery
+payload, 32 KiB/63 KiB arguments, 256 KiB/768 KiB result, 64/256 operations,
+5 s/30 s per operation, 4/16 concurrent operations, and 250 ms/1 s for
+observational evidence. Overrides must be positive safe integers at or below
+the hard maximum; `sandboxCustomTools(...)` rejects invalid configuration.
+Denied operations consume the call budget. A timed-out handler keeps its
+concurrency slot until the underlying MCP call settles, even if it ignores
+cancellation.
 
-The Deno supervisor source is published with the package:
+## Run the supervisor
 
 ```bash
-deno run --allow-net --allow-env --allow-run --allow-read --allow-write=$TMPDIR \
+export SANDBOX_WORKLOAD_ADMISSION_KEY_ID=sandbox-workload-v1
+export SANDBOX_WORKLOAD_ADMISSION_SECRET='replace-with-at-least-32-random-bytes'
+export SANDBOX_HOST_ADMISSION_SECRET='replace-with-another-32-byte-secret'
+# Generate a new value for every supervisor process/restart.
+export SANDBOX_INSTANCE_ID=sandbox_instance_01HXYZ
+# Required whenever a policy can receive QUESTPIE broker bindings.
+export SANDBOX_BROKER_URL='https://app.internal/api/sandbox/rpc'
+
+deno run --allow-net --allow-env --allow-run --allow-read \
+	--allow-write=$TMPDIR \
 	node_modules/@questpie/sandbox/src/sandbox-server.ts
 ```
 
-Set `SANDBOX_URL` in the app to the supervisor URL.
+Set `SANDBOX_URL` in the app to the directly addressed URL of that supervisor
+instance.
 
 ## Exports
 
-| Entry Point                 | Purpose                                    |
-| --------------------------- | ------------------------------------------ |
-| `@questpie/sandbox`         | Adapter, validation, and binding utilities |
-| `@questpie/sandbox/adapter` | `httpSandboxAdapter()` only                |
+| Entry point                         | Purpose                                       |
+| ----------------------------------- | --------------------------------------------- |
+| `@questpie/sandbox`                 | Adapter, authorization, validation, bindings  |
+| `@questpie/sandbox/adapter`         | HTTP adapter and workload authorization types |
+| `@questpie/sandbox/modules/sandbox` | Generated broker route module                 |
 
 ## License
 

package/dist/adapter-http-CCmYxOrN.d.mts

@@ -0,0 +1,259 @@
+import { a as SandboxCustomToolsRunOptions } from "./custom-tools-CbaRJ1ei.mjs";
+import { ExecutorAdapter, ExecutorRunOptions, ExecutorRunResult } from "questpie/executor";
+
+//#region src/workload-admission.d.ts
+interface SandboxWorkloadAdmissionClaims {
+  readonly kind: "sandbox_workload_admission";
+  readonly version: 1;
+  readonly admissionId: string;
+  readonly supervisorInstanceId: string;
+  readonly expiresAt: string;
+  readonly requestSha256: string;
+}
+interface SandboxWorkloadAdmissionKey {
+  readonly keyId: string;
+  readonly secret: Uint8Array;
+  /** Unique per supervisor process; rotate on every restart. */
+  readonly instanceId: string;
+}
+type SandboxWorkloadAdmissionDenialReason = "invalid" | "wrong_instance" | "expired" | "body_mismatch";
+type SandboxWorkloadAdmissionVerification = {
+  readonly ok: true;
+  readonly claims: SandboxWorkloadAdmissionClaims;
+} | {
+  readonly ok: false;
+  readonly reason: SandboxWorkloadAdmissionDenialReason;
+  readonly claims?: SandboxWorkloadAdmissionClaims;
+};
+//#endregion
+//#region src/types.d.ts
+/**
+ * Wire contract between the `executor` adapter (Bun/Node, main app) and the
+ * standalone Deno sandbox server. Kept intentionally minimal: the main app
+ * never imports Deno; it only speaks this JSON over HTTP.
+ */
+/**
+ * Per-request guest capabilities. `net` and `import` are INDEPENDENT axes
+ * (Deno enforces them separately — `--allow-net` gates `fetch`, `--allow-import`
+ * gates module imports). The richer app-binding capabilities (`data`, `services`,
+ * etc.) live on the core `ExecutorCapabilities` type and are NOT enforced by the
+ * sandbox server — they are the later bindings-broker's job.
+ */
+interface SandboxCapabilities {
+  /** Runtime `fetch()` allowlist as `host[:port]`. Empty = no network. */
+  net: string[];
+  /** Module import allowlist as `host[:port]` (e.g. `esm.sh:443`). Empty = no remote imports. */
+  import: string[];
+  /** Hard wall-clock timeout; the subprocess is force-killed on exceed. */
+  timeoutMs: number;
+  /** Hard per-guest memory bound (`--v8-flags=--max-old-space-size`). */
+  memoryMb: number;
+}
+/**
+ * App-bindings wiring for the UNTRUSTED path. When present, the supervisor
+ * exposes a capability-scoped `globalThis.questpie` proxy to the guest whose
+ * methods RPC (over STDIO, never the network) to the host broker at {@link url},
+ * authenticated by the per-run scoped {@link token}.
+ *
+ * The token is OPAQUE and is held by the SUPERVISOR only — it is never written
+ * into the guest subprocess, so guest code cannot read or forge it to widen
+ * scope. The guest only ever sees the typed proxy.
+ *
+ * Critically, the broker is reached SERVER-TO-SERVER (supervisor → app), NOT by
+ * the guest's `fetch`: the guest's egress allowlist rejects loopback/private IPs
+ * (the SSRF defense), so a localhost broker is unreachable from guest code by
+ * design — we do not poke a hole in that.
+ */
+interface SandboxBindings {
+  /** Absolute URL of the host broker endpoint (e.g. `https://app/sandbox/rpc`). */
+  url: string;
+  /** Per-run opaque scoped token. Supervisor-only; never exposed to the guest. */
+  token: string;
+}
+/** POST /run request body. */
+interface SandboxRunRequest {
+  /** Explicit transport admission. The supervisor rejects omitted/unknown modes. */
+  mode: "workload" | "host";
+  /** Guest TypeScript source. Must `export default` a `function(input)`. */
+  source: string;
+  /** Payload passed to the guest entry function. */
+  input: unknown;
+  capabilities: SandboxCapabilities;
+  /**
+   * Secrets injected into the guest (NOT in source). Surfaced to the guest as
+   * `globalThis.__secrets`. The sandbox never persists these.
+   */
+  secrets?: Record<string, string>;
+  /**
+   * App-bindings broker wiring. When omitted, the guest gets no `questpie`
+   * proxy (the legacy compute-only path). When present, the supervisor brokers
+   * the guest's binding RPCs to the host. See {@link SandboxBindings}.
+   */
+  bindings?: SandboxBindings;
+}
+/** POST /run response body — also the `ExecutorAdapter.run` result shape. */
+type SandboxRunResult = {
+  ok: true;
+  output?: unknown;
+  logs: string[];
+  error?: never;
+  timedOut?: never;
+  /** Wall-clock duration in ms (server-measured). */
+  ms?: number;
+} | {
+  ok: false;
+  error: string;
+  logs: string[];
+  output?: never;
+  timedOut?: never;
+  /** Wall-clock duration in ms (server-measured). */
+  ms?: number;
+} | {
+  ok: false;
+  error: string;
+  logs: string[];
+  output?: never;
+  /** True when the run was killed by the wall-clock timeout. */
+  timedOut: true;
+  /** Wall-clock duration in ms (server-measured). */
+  ms?: number;
+};
+/** Line prefix marking a framed protocol message on guest stdout. */
+declare const FRAME_MARKER = "__QP_SANDBOX_MSG__";
+/**
+ * Header carrying the per-run scoped token on the supervisor → broker request.
+ * Mirrors `questpie`'s `BINDINGS_TOKEN_HEADER`. Supervisor-only; never set by or
+ * exposed to guest code.
+ */
+declare const BINDINGS_TOKEN_HEADER$1 = "x-questpie-sandbox-token";
+/** Host-issued admission binding an authorized workload to one exact request. */
+declare const WORKLOAD_ADMISSION_HEADER = "x-questpie-workload-admission";
+/** Trusted service identity for the structurally separate direct-host path. */
+declare const HOST_ADMISSION_HEADER = "x-questpie-sandbox-host-admission";
+/** Structured error returned by the broker (mirrors `questpie` `BindingError`). */
+interface SandboxBindingError {
+  code: string;
+  message: string;
+  correlationId?: string;
+}
+/** Guest → supervisor: a binding RPC awaiting a brokered result. */
+interface GuestRpcMessage {
+  type: "rpc";
+  id: number;
+  method: string;
+  args: unknown;
+}
+/** Guest → supervisor: the run's final result. */
+interface GuestResultMessage {
+  type: "result";
+  ok: boolean;
+  output?: unknown;
+  error?: string;
+  logs: string[];
+}
+/** Any message the guest emits on stdout (framed). */
+type GuestMessage = GuestRpcMessage | GuestResultMessage;
+/** Supervisor → guest: the brokered outcome of a prior {@link GuestRpcMessage}. */
+interface HostRpcResultMessage {
+  type: "rpc-result";
+  id: number;
+  ok: boolean;
+  value?: unknown;
+  error?: SandboxBindingError;
+}
+//#endregion
+//#region src/workload.d.ts
+declare const SANDBOX_WORKLOAD_DENIAL_MESSAGE = "The workload is not authorized for this sandbox operation.";
+declare class SandboxWorkloadDeniedError extends Error {
+  readonly code: "sandbox_authority_denied";
+  constructor();
+}
+interface SandboxWorkloadPolicy {
+  readonly source: string;
+  readonly input?: unknown;
+  readonly capabilities: SandboxCapabilities;
+  readonly secrets?: Readonly<Record<string, string>>;
+  readonly bindings?: SandboxBindings;
+  /** Host-only custom-tool authority. Never serialized to the supervisor. */
+  readonly sandboxTools?: SandboxCustomToolsRunOptions;
+  /** Optional consumer deadline; transport admission is additionally capped at 5s. */
+  readonly validUntil?: string;
+}
+type SandboxWorkloadAuthorizationPhase = "prepare" | "dispatch";
+interface SandboxWorkloadAuthorizationContext {
+  readonly phase: SandboxWorkloadAuthorizationPhase;
+  readonly signal: AbortSignal;
+}
+type SandboxWorkloadAuthorizer = (envelope: unknown, context: SandboxWorkloadAuthorizationContext) => SandboxWorkloadPolicy | null | Promise<SandboxWorkloadPolicy | null>;
+interface SandboxWorkloadAuditEvent {
+  readonly boundary: "sandbox.workload";
+  readonly phase: SandboxWorkloadAuthorizationPhase | "transport";
+  readonly decision: "allowed" | "denied";
+  readonly reason: "authorization_failed" | "authorization_timed_out" | "cancelled" | "egress_denied" | "policy_changed" | "policy_invalid" | "authorization_succeeded" | "transport_completed" | "transport_denied" | "transport_cancelled" | "transport_timed_out" | "transport_invalid_response" | "transport_failed";
+}
+interface SandboxWorkloadAuditContext {
+  readonly signal: AbortSignal;
+}
+interface SandboxWorkloadRunOptions {
+  /** Consumer-owned opaque value. QUESTPIE never parses or persists it. */
+  readonly envelope: unknown;
+  readonly signal?: AbortSignal;
+}
+//#endregion
+//#region src/adapter-http.d.ts
+interface WorkloadHttpSandboxOptions {
+  readonly authorize: SandboxWorkloadAuthorizer;
+  readonly admission: SandboxWorkloadAdmissionKey;
+  /** Hard bound around each consumer authorization read. Default 2s, max 5s. */
+  readonly authorizationTimeoutMs?: number;
+  /** Redacted audit seam. Allowed decisions fail closed when it cannot settle. */
+  readonly audit?: (event: SandboxWorkloadAuditEvent, context: SandboxWorkloadAuditContext) => void | Promise<void>;
+  /** Hard bound around the audit callback. Default 250ms, max 1s. */
+  readonly auditTimeoutMs?: number;
+}
+interface HttpSandboxAdapterOptions {
+  /**
+   * Base URL of the standalone Deno sandbox service (`sandbox-server.ts`).
+   * Defaults to `process.env.SANDBOX_URL`. Throws on `run` if neither is set.
+   */
+  url?: string;
+  /**
+   * Per-request HTTP fetch timeout (ms). A safety margin above the guest
+   * wall-timeout so the server's own kill fires first; if the server is
+   * unreachable this bounds the client wait. Default: guest timeout + 10s.
+   */
+  fetchTimeoutMs?: number;
+  /** Optional custom `fetch` (e.g. for auth headers / mTLS). */
+  fetch?: typeof fetch;
+  /**
+   * Validate net/import hosts against the private-IP policy BEFORE sending to
+   * the sandbox (defense-in-depth; the server validates too). Default: true.
+   */
+  validateEgress?: boolean;
+  /** Consumer-owned fail-closed admission for opaque remote workloads. */
+  workload?: WorkloadHttpSandboxOptions;
+  /** Trusted host credential for the direct framework `run()` path. */
+  hostAdmissionSecret?: string;
+}
+/**
+ * Production sandboxed `ExecutorAdapter` — POSTs the run to the standalone Deno
+ * sandbox service over HTTP. The main app (Bun/Node) holds NO Deno; it only
+ * speaks this JSON contract.
+ *
+ * Use as the `sandboxed` adapter in the executor config:
+ *   executor: { sandboxed: httpSandboxAdapter({ url: process.env.SANDBOX_URL }) }
+ */
+declare class HttpSandboxAdapter implements ExecutorAdapter {
+  private readonly options;
+  private readonly fetchImpl;
+  private readonly validateEgress;
+  constructor(options?: HttpSandboxAdapterOptions);
+  private resolveUrl;
+  private postRun;
+  run(options: ExecutorRunOptions): Promise<ExecutorRunResult>;
+  runWorkload(options: SandboxWorkloadRunOptions): Promise<ExecutorRunResult>;
+}
+/** Factory for the production sandboxed adapter. */
+declare function httpSandboxAdapter(options?: HttpSandboxAdapterOptions): HttpSandboxAdapter;
+//#endregion
+export { SandboxWorkloadAdmissionVerification as A, SandboxCapabilities as C, SandboxWorkloadAdmissionClaims as D, WORKLOAD_ADMISSION_HEADER as E, SandboxWorkloadAdmissionDenialReason as O, SandboxBindings as S, SandboxRunResult as T, GuestResultMessage as _, SANDBOX_WORKLOAD_DENIAL_MESSAGE as a, HostRpcResultMessage as b, SandboxWorkloadAuthorizationContext as c, SandboxWorkloadDeniedError as d, SandboxWorkloadPolicy as f, GuestMessage as g, FRAME_MARKER as h, httpSandboxAdapter as i, SandboxWorkloadAdmissionKey as k, SandboxWorkloadAuthorizationPhase as l, BINDINGS_TOKEN_HEADER$1 as m, HttpSandboxAdapterOptions as n, SandboxWorkloadAuditContext as o, SandboxWorkloadRunOptions as p, WorkloadHttpSandboxOptions as r, SandboxWorkloadAuditEvent as s, HttpSandboxAdapter as t, SandboxWorkloadAuthorizer as u, GuestRpcMessage as v, SandboxRunRequest as w, SandboxBindingError as x, HOST_ADMISSION_HEADER as y };