pi-roundtable-sandbox 0.7.9 → 0.7.11

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 0.7.11
+
+- `precheckScriptRunner` runs agents' precheck scripts (pi-roundtable 0.7.11) in a sealed container per run: no network, a read-only root, no capabilities, a non-root user, an empty workspace, and a per-run broker (`precheckBroker`) that forwards only single `tools/call` requests of the tools `grant(scope)` allows, with the host's credential (`PrecheckMcpServer`). The container runs `worker/precheck-main.ts` (`PRECHECK_ENTRYPOINT`, `PrecheckWorkerInput`), which gives the script `mcp.call`, `mcp.json`, `firedAt`, `timeZone`, `today`, and `schedule`.
+- A script's workspace is mounted read-only (`ContainerSpec.workspaceReadOnly`), so it writes only to the container's bounded `/tmp`. The broker refuses an answer that carries the credential plainly, escaped, inside JSON text, base64-encoded, or as a key; joins an SSE event's `data:` lines; and refuses a second call while one is pending (409) without spending the budget. The script's console output goes to stderr and never into its answer, and `grant` receives the creator's `tier`.
+- `ContainerSpec` takes `entrypoint`, and `DockerContainerDriver.exec` runs a sealed container with any input and a bounded output (`PrecheckContainerDriver`); `worker/Dockerfile` includes the precheck worker.
+
+## 0.7.10
+
+- Compact Pi worker sessions with the core's tiers (300,000 tokens through a host compactor, Pi's summary past 500,000), through the new `compaction` option (`PiCompactor`: `engine`, `compact(request, { channel, signal })`, `timeoutMs`, `maxRequestBytes`) and the protocol types `PiCompactRequest`, `PiCompaction`, `PiCompactResponse`, `PiCompactionReport`, `PiCompactMessage` and `PiWorkerConfig`; the host logs every compaction and fallback per channel.
+- A failed `runTurn` keeps its cause (`AgentRunError` message and `cause`, whether it timed out) and logs it; the broker logs upstream failures with channel, status, latency and the error body; a timed-out worker's last 200 log lines are logged before its container is removed (`PiContainerDriver.logs`).
+- Behavior change: Pi containers log to journald tagged `sandbox/<channel>` by default, instead of two 10 MiB `local` files, so the Docker daemon must run with journald (pass `PiDockerContainerDriver` a `log` option for another driver) (`PiContainerSpec.log`, `PiDockerContainerDriver` option `log`, `defaultContainerLog`), and the worker logs model, broker, tool and compaction failures.
+
 ## 0.7.5
 
 - `ScopedSandboxDelegator` limits a title to 200 characters by default again, and accepts `maxTitleChars`, `maxReportChars` and `diagnosticChars`.

package/README.md

@@ -226,7 +226,8 @@ Other model transports are not implicitly proxied by this broker.
 | `timeZone` | Explicit worker time zone; default UTC. |
 | `containerPrefix`, `labelChannel`, `labelProfile` | Operator compatibility names for existing containers, not guest data. |
 | `driver` | Trusted local Docker driver or offline fixture; no remote bind mounts. |
-| `logger` | Host logger; package failures avoid credentials and upstream payloads. |
+| `logger` | Host logger: failed turns with their cause, upstream failures (channel, status, latency, error body cut to 2,000 characters with credentials masked), a timed-out worker's last 200 log lines, and every compaction. |
+| `compaction` | Optional host compactor (`PiCompactor`), see [Compaction](#compaction). |
 | `startTimeoutMs`, `turnTimeoutMs` | 1–600 seconds; defaults 90 and 600 seconds. |
 | `maxCalls`, `maxOutputTokens` | Default 128 credential-bearing calls and 128,000 output tokens per model call; configurable 1–1,000 calls and 1,025–200,000 tokens. |
 
@@ -238,6 +239,37 @@ Callbacks must still validate input and enforce per-person quotas, memory author
 Optional person/notes/moments tools can use an existing PostgreSQL store without copying the connection, schema, migration ledger or owner memory into the image.
 Schedules can be host tools with a declared channel-local background target and host-bound author; no scheduler is enabled automatically.
 
+### Compaction
+
+Worker sessions compact with the core's tiers, as the host's own sessions do: a model whose window leaves more than 300,000 tokens compacts at 300,000 through the host compactor, and past 500,000 through Pi's own summary, whose model calls go through the broker like any other.
+A compaction that left the context within 50,000 tokens of its threshold moves the next one to the hard ceiling, so it does not repeat on the next request.
+
+The host compactor runs on the host, where the worker cannot reach (a remote compaction service, say):
+
+```ts
+new PiSandboxRuntime({
+  // ...
+  compaction: {
+    engine: "my-compaction", // recorded as details.engine of its compactions
+    compact: async (request, { channel, signal }) => {
+      // request: PiCompactRequest; return a PiCompaction, or undefined for Pi's summary
+    },
+    timeoutMs: 120_000, // optional; default the smaller of 120 s and a third of turnTimeoutMs
+    maxRequestBytes: 32 * 1024 * 1024, // optional; default 32 MiB
+  },
+});
+```
+
+`PiCompactRequest` carries Pi's preparation: `reason`, `tokensBefore`, `firstKeptEntryId`, `isSplitTurn`, `messagesToSummarize`, `turnPrefixMessages`, `keptMessages` (the messages from `firstKeptEntryId` on), `previousSummary?`, `customInstructions?`, `readFiles` and `modifiedFiles`.
+A `PiCompaction` is Pi's `CompactionResult`: `summary`, `firstKeptEntryId` (the request's), `tokensBefore`, `estimatedTokensAfter?` and `details?` (an object; the broker sets its `engine`).
+A compactor that returns `undefined`, throws, answers out of shape, outlasts `timeoutMs` (its `signal` aborts), or gets a request over `maxRequestBytes` falls back to Pi's summary, and the host logs the reason.
+The timeout may be at most half of `turnTimeoutMs`, so Pi's summary keeps time to run.
+A turn may ask the host for three compactions and send sixteen compaction reports; later compactions fall back to Pi's summary, and while a compactor that ignored its signal still runs, a new request falls back too.
+Without `compaction` the worker registers no compaction handler; the tiers and Pi's summary still apply.
+Compactions are written to the session file in the channel workspace, so they survive container removal and restarts.
+
+The host logs, per channel: `conversation compacted` (`trigger`, `engine` `extension` or `pi`, `tokensBefore`, `tokensAfter`, `nextCompactionAt`), `compaction failed`, `compaction skips the extension for Pi's summary` past the ceiling, and `compaction falls back to Pi's summary` with its `fallback` reason.
+
 ### Worker content and files
 
 The trusted image exports `workerContent(profile): PiWorkerContent` from `/app/worker/content.ts`.
@@ -268,7 +300,10 @@ Never include real auth files or host source in those content directories.
 ### Pi isolation and continuity
 
 Pi containers remain network-none, non-root, read-only-root, capability-free and `no-new-privileges`.
-They have 1.5 GiB RAM with no additional swap, one CPU, 256 PIDs, a 512 MiB temporary filesystem, and bounded Docker logs (two 10 MiB local log files).
+They have 1.5 GiB RAM with no additional swap, one CPU, 256 PIDs, and a 512 MiB temporary filesystem.
+Their logs go to journald tagged `sandbox/<channel>`, so they outlive the container; the Docker daemon must then run under systemd with journald, or containers fail to start.
+journald bounds the lines through its own rate limit and `SystemMaxUse`, not per container; `PiDockerContainerDriver`'s third argument, `{ log: { driver, options } }` (or a function of the channel), names another Docker log driver.
+The worker logs model errors, broker call failures, tool failures and compactions to stdout as JSON lines.
 Only the channel workspace (writable) and host-created broker directory (read-only) are mounted.
 Worker-initiated `/worker/ready`, `/worker/next` and `/worker/result` requests transport turns and byte-backed replies; the host never connects to a guest-created socket or reads guest reply paths.
 Only one idle long poll, one admitted turn and one incoming reply packet per channel are allowed.
@@ -329,6 +364,53 @@ One of `search` with a fetch option, or `tools`, is required.
 Host search/model credentials stay in the trusted host process, and report delivery must remain in the declared guest channel.
 These hooks broaden the sealed threat model: review every adapter, apply provider spend limits and host quotas, and never substitute an unrestricted default delegation worker.
 
+## Precheck scripts
+
+`precheckScriptRunner` lets agents write a schedule's precheck themselves (pi-roundtable 0.7.11 or later): a short JavaScript module that decides, before the scheduled turn, whether the agent is woken at all.
+The core stores and parses the script but never runs it; this runner runs it in a sealed container, one per run.
+Register it from a trusted plugin of your own:
+
+```ts
+import { PRECHECKS, definePlugin } from "pi-roundtable";
+import { precheckScriptRunner } from "pi-roundtable-sandbox";
+
+export const precheckScripts = definePlugin({
+  name: "precheck-scripts",
+  setup: ({ services }) => {
+    services.get(PRECHECKS).useScriptRunner(precheckScriptRunner({
+      image: "pi-roundtable-sandbox:pi",
+      runRoot: "/tmp/roundtable-precheck",
+      // The host's non-root user, who owns runRoot; root is refused.
+      uid: 1000,
+      gid: 1000,
+      // Never more than the schedule's agent may call itself; [] leaves the script no way out.
+      grant: ({ channel, target }) => grantsFor(channel, target),
+    }));
+    return {};
+  },
+});
+```
+
+`grant({ channel, target })` returns the `PrecheckMcpServer`s a script for that schedule may call: `{ name, url, tools, token? }`.
+The script calls a server by `name`, only the listed `tools`, and only with single `tools/call` requests; every other server, tool, method, query, and extra credential is refused, and refused calls count against the run's budget (`maxCalls`, default 16).
+`url` is a fixed HTTPS Streamable HTTP endpoint answering with JSON or one SSE event; `token` is read on the host for each call and inserted by the broker. An answer that carries it, plainly, inside JSON text, or base64-encoded, is refused; this catches an echo, not an upstream set on leaking it, so grant only upstreams you trust.
+Calls run one at a time; a second call while one is pending is refused.
+Neither the endpoint nor the credential reaches the container.
+
+Each run gets:
+
+- The same sealed `docker run` as a guest turn: `--network none`, a read-only root, all capabilities dropped, `no-new-privileges`, the host's non-root user, 256 MiB, one CPU, and 32 processes by default (`limits`).
+- A fresh broker socket and an empty workspace under `runRoot`, both removed afterwards; the workspace is mounted read-only, so a script writes only to the container's 64 MiB `/tmp`.
+- When the host stops, the scheduler aborts running scripts and waits for their containers to be removed.
+- The worker `worker/precheck-main.ts` as its entrypoint (`PRECHECK_ENTRYPOINT`, the Pi image's layout; pass `entrypoint` for another image, such as `["bun", "/app/worker/precheck-main.ts"]` for `worker/Dockerfile`'s).
+- At most `timeoutMs` (default 60 seconds); on timeout the container is killed and the turn wakes with the error.
+
+The script is `export default async ({ mcp, firedAt, timeZone, today, schedule }) => result`, where `result` is `{ wake: false, note? }` or `{ wake: true, context }`.
+`today` is the date in the host's time zone, so a script never reads a UTC date by mistake.
+`mcp.call(server, tool, args)` returns the MCP tool result and `mcp.json(server, tool, args)` its structured content or its first text content parsed as JSON; a tool error throws.
+A throw, a wrong answer, or a timeout wakes the turn with the error.
+The runner's `describe` tells the model this contract and what it may call, through `schedule_list`.
+
 ## Development and verification
 
 From the pi-roundtable repository root:

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "pi-roundtable-sandbox",
-	"version": "0.7.9",
+	"version": "0.7.11",
 	"description": "Sealed guest channels and an allow-listed credential broker for pi-roundtable",
 	"type": "module",
 	"license": "MIT",
@@ -52,7 +52,7 @@
 		"@biomejs/biome": "2.5.15",
 		"@types/bun": "1.4.2",
 		"discord.js": "14.27.0",
-		"pi-roundtable": "0.7.9",
+		"pi-roundtable": "0.7.11",
 		"typebox": "1.3.34",
 		"typescript": "7.0.2"
 	}

package/src/container-driver.ts

@@ -17,6 +17,10 @@ export interface ContainerSpec {
 	memoryMb?: number;
 	cpus?: number;
 	pids?: number;
+	/** Runs this command instead of the image's own, such as a precheck script's runner. */
+	entrypoint?: readonly string[];
+	/** Mounts the workspace read-only, for a run that may write nothing to the host. */
+	workspaceReadOnly?: boolean;
 }
 export interface ContainerDriver {
 	run(
@@ -64,6 +68,15 @@ export function containerRunArgs(spec: ContainerSpec): string[] {
 		pids < 8
 	)
 		throw new Error("invalid resource limits");
+	const [command, ...args] = spec.entrypoint ?? [];
+	if (
+		spec.entrypoint &&
+		(!command ||
+			spec.entrypoint.some(
+				(part) => !part || part.startsWith("-") || /[\0\r\n]/.test(part),
+			))
+	)
+		throw new Error("invalid entrypoint");
 	return [
 		"run",
 		"--rm",
@@ -99,10 +112,12 @@ export function containerRunArgs(spec: ContainerSpec): string[] {
 		"--mount",
 		`type=bind,src=${spec.runDir},dst=/broker,readonly,bind-propagation=rprivate`,
 		"--mount",
-		`type=bind,src=${spec.workspaceDir},dst=/workspace,bind-propagation=rprivate`,
+		`type=bind,src=${spec.workspaceDir},dst=/workspace,${spec.workspaceReadOnly ? "readonly," : ""}bind-propagation=rprivate`,
 		"--env",
 		"HOME=/tmp/home",
+		...(command ? ["--entrypoint", command] : []),
 		spec.image,
+		...args,
 	];
 }
 
@@ -117,6 +132,30 @@ export class DockerContainerDriver implements ContainerDriver {
 		turn: SandboxTurn,
 		signal: AbortSignal,
 	): Promise<SandboxReply> {
+		const output = await this.exec(spec, JSON.stringify(turn), signal, {
+			stdoutBytes: 2 * 1024 * 1024,
+		});
+		const reply: unknown = JSON.parse(output);
+		if (
+			!isRecord(reply) ||
+			typeof reply.ok !== "boolean" ||
+			typeof reply.text !== "string" ||
+			reply.text.length > 100_000
+		)
+			throw new Error("invalid sandbox reply");
+		return { ok: reply.ok, text: reply.text };
+	}
+
+	/**
+	 * Runs one sealed container with `input` on its stdin and returns its stdout, bounded; a
+	 * nonzero exit or an abort throws, and the container is force-removed either way.
+	 */
+	async exec(
+		spec: ContainerSpec,
+		input: string,
+		signal: AbortSignal,
+		limits: { stdoutBytes: number },
+	): Promise<string> {
 		// Canonical paths prevent symlinks in operator configuration selecting a different mount.
 		const args = containerRunArgs({
 			...spec,
@@ -133,24 +172,16 @@ export class DockerContainerDriver implements ContainerDriver {
 		const abort = () => child.kill("SIGKILL");
 		signal.addEventListener("abort", abort, { once: true });
 		try {
-			child.stdin.write(JSON.stringify(turn));
+			child.stdin.write(input);
 			child.stdin.end();
 			const [output, , code] = await Promise.all([
-				boundedText(child.stdout, 2 * 1024 * 1024),
+				boundedText(child.stdout, limits.stdoutBytes),
 				boundedText(child.stderr, 128 * 1024),
 				child.exited,
 			]);
 			signal.throwIfAborted();
 			if (code !== 0) throw new Error("sandbox worker failed");
-			const reply: unknown = JSON.parse(output);
-			if (
-				!isRecord(reply) ||
-				typeof reply.ok !== "boolean" ||
-				typeof reply.text !== "string" ||
-				reply.text.length > 100_000
-			)
-				throw new Error("invalid sandbox reply");
-			return { ok: reply.ok, text: reply.text };
+			return output;
 		} finally {
 			signal.removeEventListener("abort", abort);
 			child.kill("SIGKILL");

package/src/index.ts

@@ -38,13 +38,18 @@ export {
 	type PiAttachmentOptions,
 } from "./pi-attachments.ts";
 export {
+	PI_COMPACT_LIMITS,
 	type PiBrokerOptions,
+	type PiCompactor,
 	type PiHostContext,
 	type PiMcpServer,
 	PiSandboxBroker,
 } from "./pi-broker.ts";
 export {
+	defaultContainerLog,
+	demuxDockerLog,
 	type PiContainerDriver,
+	type PiContainerLog,
 	type PiContainerSpec,
 	type PiContainerStatus,
 	PiDockerContainerDriver,
@@ -60,6 +65,11 @@ export {
 	PI_RUN_DIR,
 	PI_THINKING_LEVELS,
 	PI_WORKSPACE,
+	type PiCompaction,
+	type PiCompactionReport,
+	type PiCompactMessage,
+	type PiCompactRequest,
+	type PiCompactResponse,
 	type PiMcpDiscovery,
 	type PiReplyFile,
 	type PiThinkingLevel,
@@ -67,7 +77,10 @@ export {
 	type PiTurnContext,
 	type PiTurnRequest,
 	type PiTurnResponse,
+	type PiWorkerConfig,
 	safeFileName,
+	validateCompactionReport,
+	validateCompactRequest,
 	validateImages,
 	validateReplyFiles,
 } from "./pi-protocol.ts";
@@ -84,6 +97,15 @@ export {
 	type SandboxService,
 	sandbox,
 } from "./plugin.ts";
+export {
+	PRECHECK_ENTRYPOINT,
+	type PrecheckContainerDriver,
+	type PrecheckMcpServer,
+	type PrecheckScriptRunnerOptions,
+	type PrecheckWorkerInput,
+	precheckBroker,
+	precheckScriptRunner,
+} from "./precheck-runner.ts";
 export type { SandboxReply, SandboxTurn, ToolSpec } from "./protocol.ts";
 export {
 	type ResearchTools,