@nylorun/runtime 0.15.1-beta → 0.16.0-beta

package/CHANGELOG.md

@@ -1,5 +1,87 @@
 # Changelog
 
+## 0.16.0-beta
+
+### Major Changes
+
+- b352feb: **File artifacts, message parts and capability links (protocol 6).** Upgrade these with the Runtime: they speak protocol 6, which this Runtime serves beside 4 and 5. See MIGRATION.md.
+
+  - **`@nylorun/runtime`: file artifacts.** A file is an artifact: an `af_` id, a name and numbered immutable versions, with rows in the new `artifacts` and `artifact_versions` tables (migration `0004_artifacts`) and bytes in the Object store (`BlobStore`). `POST /v1/artifacts?name=&sessionId=` uploads a file in one streamed request and `POST /v1/artifacts/{id}/versions` adds a version, within the Tenant's limits (`GET`/`PUT /v1/tenant/artifacts`: 100 MiB per file and 10 GiB in all by default); a body past either is `413 limit_exceeded`, refused mid-stream with nothing stored. `GET /v1/artifacts` lists them (by session), `GET /v1/artifacts/{id}` reads one with its versions, `GET /v1/artifacts/{id}/versions/{n|latest}/content` downloads through the Runtime with HTTP Range (`206`, `Content-Range`, `416`), and `DELETE /v1/artifacts/{id}` deletes it with its bytes. A session's artifacts go with it on a sessions reset, and appear in its history as `artifact.created`, `artifact.version.created` and `artifact.deleted` (new in the event catalog). A subject reaches only the artifacts of their own sessions.
+  - **`@nylorun/runtime`: capability links.** `POST /v1/artifacts/{id}/links` mints a short-lived path, `/v1/artifact-links/<token>`, that downloads one version (with Range) with no credential and no `Nylorun-Protocol`: an ES256 JWT (`typ: nylorun-artifact+jwt`) signed with the Tenant's signing key, at most 15 minutes, and dead once the artifact is deleted. The Host logs its path without the token.
+  - **Breaking (`@nylorun/core`, `@nylorun/runtime`): message `parts`.** A user message may carry `parts`: `text`, and `file` by `artifactId` (and `version`, else the latest, pinned when the message is accepted). Model-gate reads the file from the Object store: an image goes to the model as image input, a text file as text, and any other file fails the call (`invalid_request`). The record and the transcript hold only the reference. The gateway builds its `BlobStore` from `NYLORUN_OBJECT_STORE_*` (the Tenant's `fs` store without it). `PROTOCOL_VERSION` is 6; `HOST_PROTOCOL` is 4–6 and protocol 6 clients require the feature `artifacts`.
+  - **`@nylorun/runtime`: `save_artifact`.** A session with a sandbox gets the `nylorun.artifacts` capability, whose `save_artifact` tool saves a sandbox file (`path`) or text (`content`) as an artifact of the session, carrying the turn and tool call on its event.
+  - **Breaking (`@nylorun/runtime`): `MediaStore` is removed.** `MediaStore` and `localMedia` leave `@nylorun/runtime/node`, and `piModel` takes `files` (a resolver from an artifact reference to its bytes) instead of `media`; images are file artifacts. The image checks (`decodeImageBase64`, `validateImageBytes`, `IMAGE_MEDIA_TYPES`, `MAX_IMAGE_BYTES`) stay.
+  - **`@nylorun/agents`:** `client.artifacts` (`upload`, `uploadVersion`, `list`, `get`, `download` with a range, `link`, `delete`), also on `BrowserClient`, and `session.inputParts(parts)`. A request with a body of its own type keeps it.
+  - `@nylorun/admin`, `@nylorun/cli`, `nylorun`, `@nylorun/studio`: speak protocol 6.
+
+### Minor Changes
+
+- e58e6ec: **egress-gate: pod sandboxes reach only the hosts their spec allows (F7.2).** A new `egress` service runs in the gateway's process (`--service gates,keys,egress`) on `NYLORUN_EGRESS_LISTEN_PORT` (default 4200). It is a CONNECT proxy: each tunnel needs an egress token (`Proxy-Authorization`, Basic with the token as password or Bearer) for a live sandbox at its current host epoch, a host name (no IP literal) that the sandbox's `network.allow` names exactly or by `*.suffix`, port 443 or 80, and a name that resolves to a public address; the gate connects to that checked address and pipes bytes, with no TLS interception and no credential injection. Plain HTTP is answered `405`; a sandbox holds at most 64 tunnels; idle tunnels close after 5 minutes; refusals are logged, never emitted as events. Egress tokens (`typ: nylorun-egress+jwt`), minted with each host token at a pod's join, are accepted nowhere else; egress-gate also refuses a token of a sandbox that is deleted, lost or expired, or of a pod other than the one that joined. Sandbox spec resolution lets `pod` sandboxes allow `*.suffix` hosts within the Tenant's ceiling (virtual ones keep exact names). With sandboxes enabled, `nylorun` runs the gateway with `egress` and publishes it on `NYLORUN_SANDBOX_BIND` at `NYLORUN_SANDBOX_EGRESS_PORT`, next to the gates.
+- 186ec3f: **A local Tenant has an Object store: RustFS, behind the Runtime's new `BlobStore` seam.** `nylorun start` adds a `rustfs` container (RustFS 1.0.1, single node and single drive, pinned by digest) on the `nylorun-<tenant>-rustfs` volume, unpublished and without its console. Its secret key is generated once into `docker/.env` (`NYLORUN_OBJECT_STORE_SECRET_KEY`), and only the `runtime` and `gateway` containers receive the credential (`NYLORUN_OBJECT_STORE_ENDPOINT`, `_ACCESS_KEY`, `_SECRET_KEY`); the runtime creates the bucket at boot. The Runtime reaches the store through `BlobStore` (put with a streamed body and a size cap, get with a byte range, head, delete, list by prefix) with an `s3` adapter over the plain S3 API and an `fs` adapter, which a Runtime without `NYLORUN_OBJECT_STORE_ENDPOINT` (embedded, ephemeral, tests) uses under the Tenant directory's `blobs/`. Nothing stores files there yet; file artifacts build on it.
+- 6077272: **Folder artifacts and the turn-end outputs export (F8.2).** Part of protocol 6, with file artifacts.
+
+  - **`@nylorun/runtime`: the turn-end export.** When an agent's turn completes, the Runtime reads `/workspace/outputs` of the session's sandbox and keeps it as a version of the session's folder artifact `outputs`: the first export creates it, and each later turn whose outputs changed adds a version (`artifact.created` / `artifact.version.created` with `kind: "folder"`, `source: "export"`, `fileCount` and `claimed: true`, since the listing and bytes are what the sandbox supplied). Nothing is exported without a sandbox or without outputs. An export past 10,000 files, 1 GiB, the per-file limit or the Tenant total stores nothing and records `artifact.export.skipped` with its reason; a failure records `artifact.export.failed`. Neither fails the turn. Core reads the workspace through one seam, `WorkspaceReader`, which the Harness API and pod sandboxes will implement later.
+  - **`@nylorun/runtime`: folder artifacts.** A folder version is a manifest of paths to content-addressed files: each file's bytes are stored once at `blobs/sha256/<hex>`, so an unchanged file is never stored again, and the Tenant total (`GET /v1/tenant/artifacts` `usedBytes`) counts it once. New routes: `GET /v1/artifacts/{id}/versions/{n|latest}/tree` (the manifest), `…/files/{path}` (one file by its percent-encoded path, with Range), `…/diff?from=n` (files added, removed and changed) and `…/zip` (a streamed zip). `POST /v1/artifacts/{id}/links` takes `file` to link one file of a folder; a folder's link without it opens the zip. A folder's `/content`, a new version uploaded to a folder, and a message part naming a folder are `400`. Deleting a folder removes the files no other version names. Migration `0006_folder_artifacts` allows the `folder` kind and adds the `artifact_content` table.
+  - **`@nylorun/core`:** `ArtifactKindSchema` (`file`, `folder`), the `export` source, `FolderEntrySchema`, `FolderManifestSchema`, `ArtifactTreeSchema`, `ArtifactDiffSchema`, the `artifact.export.skipped` and `artifact.export.failed` events, and `fileCount` and `claimed` on `artifact.created` / `artifact.version.created`.
+  - **`@nylorun/agents`:** `client.artifacts.tree()`, `file()`, `diff()` and `zip()`, and `link(id, { file })`.
+
+- a5f9cf5: **Gate trust (F5): run tokens and two gate credentials.** A model or MCP call through the gates service reaches only its own run, and the gate takes the call's scope from a token instead of believing the request body.
+
+  - **Run tokens.** Each advance that takes a session's lease mints a run token (`tenant/run-token.ts`): an ES256 JWT of `typ` `nylorun-run+jwt` and `aud` `nylorun-gates`, signed by the keys service with the Tenant's current signing key, naming the session (`sub`), its active turn (`trn`), its root agent (`agt`) and the lease epoch (`epc`), for 15 minutes. The lease heartbeat re-mints it when less than 5 minutes remain, and it is dropped when the advance ends or loses the lease. The tokens are internal: not in `openapi.json`, and the JWKS is unchanged.
+  - **Two credentials at the gateway.** Model calls and their cancels accept only a run token. Tool calls, their cancels and the MCP connect, list and close routes accept a run token, or `NYLORUN_GATES_TOKEN` (now core's credential) for requests made outside a run. Keys, `/deliveries` and endpoint pings accept only core's credential.
+  - **Scope from the token.** The model-call body no longer carries `sessionId`, `turnId` or `agentId` (a body naming them is refused with `400`), and MCP requests under a run token leave out `server.sessionId`: ledger rows, turn and agent caps, and MCP manifest and vault-grant lookup use the token's claims.
+  - **Stale tokens die.** Every call under a run token checks that the session's epoch and active turn are still the token's and that it is not cancelled, and answers `409 run_stale` otherwise. A keyed call joins a call already running at the gate only for the same session at the same or a newer epoch, so the new owner's re-send after a takeover keeps working (P1.2, F4.1) and the old owner's is refused. A cancel stops only its own session's call (`403 gate_forbidden` otherwise).
+  - In process nothing changes: embedding, `startEphemeralRuntime` and Runtimes without the gates service mint no token. No protocol bump: the public API, protocol 5, the Action endpoint wire and `docker/.env` are unchanged. See `DEPLOYMENT.md`.
+
+- b6bf1f5: **Harness API v1, in process (F6.1).** Every segment now runs in a harness: the advance takes the session's lease and offers the segment as a run, and the Tenant's own harness, in the same process, runs the engine and reports how it ended. Core settles it exactly as before. Nothing changes on the wire: protocol 5, durable checkpoint 1, engine `hosted-3` and the Action endpoint wire are the same.
+
+  - `@nylorun/core/harness-api`: the protocol (messages, Zod schemas, the effect request hash, transcript edits, an RPC channel with an in-process memory transport).
+  - `@nylorun/harness/api`: `createHarness({ channel, executors })`, a harness that leases runs, renews their leases, replays a run's recorded outcomes without asking, keeps transcripts by record cursor, and runs model, MCP and sandbox calls through the executors it is given.
+  - `@nylorun/runtime`: the Harness API server per Tenant (`TenantHandle.attachHarness`), the in-process harness, and the journal as the Record seam. A model call's journal row now stores the request's hash without its prompt, so a replay never sends a prompt twice. `NYLORUN_HARNESS_API=0` runs the engine in the advance as before, until F6.2 removes it. A Runtime older than this one may fail a turn that was in flight across a downgrade with drift.
+
+- 8ed4ea6: **Harness service over WebSocket, with the workspace capability (F6.2).** A Runtime started with `NYLORUN_HARNESS=remote` runs no harness of its own: it opens the Harness API listener (`NYLORUN_HARNESS_LISTEN_HOST`/`_PORT`, default port 4200, `NYLORUN_HARNESS_ALLOWED_HOSTS`), which accepts only the harness credential (`NYLORUN_HARNESS_TOKEN`) on `/nylorun/harness/v1`. The runtime image's `--service harness` connects to it (`NYLORUN_HARNESS_URL`, `NYLORUN_HARNESS_TOKEN`, `NYLORUN_GATES_URL`, `NYLORUN_HARNESS_ROOT`) and runs the Tenant's segments, MCP servers and sandboxes with no store; it refuses to start with a database, the gates' or keys' credential, or Restate settings, and presents only run tokens at the gates. The in-process harness stays the default.
+
+  - `@nylorun/core/harness-api`: the `workspace.*` requests core sends to a harness that serves workspaces, `tenantId` in the `hello` answer, a workspace record on `sandbox.state` claims, and `TurnStart.options.holdMs`. Tenant and admin status report the Tenant's harnesses (`harness`).
+  - `@nylorun/harness/api`: `createHarness` declares capabilities, reports grants (`onGrant`) and the `hello` answer, readies MCP through `executors.prepare` (`session.mcp`), and holds a run while its Action is pending until core sends the outcome (`effect.resolved`).
+  - `@nylorun/runtime`: the WebSocket listener and client, `--service harness`, the workspace capability (`ctx.sandbox` is a `WorkspacePort`; sandbox tool routes, `save_artifact`, sweep and reset reach the harness's workspaces), the SandboxManager's records port, and held runs (`actionHoldMs`, default 5 minutes). `save_artifact` runs in core. `NYLORUN_HARNESS_API` and the engine run in the advance are removed; tests run with `NYLORUN_TEST_HARNESS=memory|json|ws`.
+
+- 678e085: **Pod sandboxes (F7.2, second part).** With `nylorun sandbox enable`, a sandbox resource of kind `pod` (`PUT /v1/sandboxes/{id}` with `kind: "pod"`, an `image`, `*.suffix` hosts, `storage` and `lifecycle.ttl`) is an agent-sandbox pod on the Tenant's cluster, created at once. The turns of the sessions attached to it run in the pod: the engine is copied from the Runtime image into the pod (the Runtime image now carries tini for it), waits until the pod's NetworkPolicy is in force, exchanges its join token for a host token at the Harness API listener (`POST /nylorun/harness/v1/host/join`, published for pods on the Docker host's address, with the gates), and serves its sandbox alone. New: `POST /v1/sandboxes/{id}/stop` and `/reset`; the Tenant's `limits.ttl`, `lifecycle.onExpiry`, `lifecycle.stopGrace` and `placement`; idle stop; lifecycle events `sandbox.running`, `.suspended`, `.expired`, `.relaunched`, `.lost`, `.reset` and `.failed`; error codes `placement_refused`, `sandbox_lost` and `sandbox_expired`; `cluster` in `GET /v1/tenant/sandbox`; Host feature `sandbox-pods` (additive, protocol unchanged). Without a cluster, kind `pod` is `409 sandbox_unavailable` (it was 400). Migration `0007_sandbox_pods` adds the pod lifecycle columns to `sandbox_resources`.
+- 926711b: **Sandboxes are a resource (F7.1, blueprint D39; Host feature `sandboxes`).** A sandbox has its own id, a kind, a spec and labels, and outlives the sessions attached to it. Additive: protocol 5 is unchanged.
+
+  - `PUT /v1/sandboxes/{id}` creates a sandbox or finds the one with that id (get-or-create in one call); `GET /v1/sandboxes/{id}`, `GET /v1/sandboxes?label=key=value` (repeatable), `GET /v1/sandboxes/{id}/events` and `DELETE /v1/sandboxes/{id}`. Ids are `/`-separated segments (`team-a/proj-42`), sent percent-encoded as one path segment. Only kind `virtual` runs; `pod` is refused with `sandbox_unavailable`. The spec is resolved against the Tenant's limits and fixed once the sandbox exists; labels can change.
+  - A session attaches with `sandbox: { id }` and shares the sandbox's `/workspace` with every other session attached to it. Turns are serial per sandbox: a second session's turn is refused with `409 sandbox_busy` while another runs. Deleting a session (a sessions reset) only detaches it. Deleting a sandbox is refused while a turn runs in it; afterwards an attached session's next turn is refused with `sandbox_unavailable` until a sandbox with that id exists again.
+  - Subject tokens carry an `sbx` claim: `POST /v1/tokens` takes `sandboxes`, exact ids or prefixes ending in `/*` (at most 16). A token reaches only the sandboxes they match, checked when a session attaches and at every turn start (`403 sandbox_not_granted`); any other sandbox is the 404 of a missing one. The new scope `sandboxes:write` lets a role create and delete the sandboxes its grants reach. Application keys reach every sandbox.
+  - The Tenant holds at most `limits.sandboxes` sandboxes (`PUT /v1/tenant/sandbox`, default 100); one more is `409 limit_exceeded`.
+  - Lifecycle events (`sandbox.created`, `sandbox.attached`, `sandbox.detached`, `sandbox.deleted`) go to the sandbox's own stream in the record, through the record module; the session's log records `sandbox.attached`. The sandbox stream is not relayed to S2.
+  - New error codes `sandbox_not_granted`, `sandbox_busy` and `sandbox_unavailable`; the session view gains `sandboxId` and `sandboxSource: "sandbox"`. Migration `0004_sandbox_resources` adds `sandbox_resources`, `nylorun_streams.sandbox_events` and the sessions' `sandbox_id` column.
+  - `@nylorun/agents`: `client.sandboxes` with `ensure(id, spec)`, `get`, `list({ labels })`, `delete`, `events`, and `forSession({ session, spec })`, which creates a sandbox for one session, opens the session on it, and deletes it with `release()`. It replaces sharing through another session (`sandbox: { session }` and the view's `sandboxOwnerId`, now deprecated). `client.tokens.create` takes `sandboxes`.
+  - `nylorun sandbox ls [--label key=value]... [--json]` and `nylorun sandbox rm <id>` list and delete the running local Tenant's sandboxes.
+
+- db045cd: **The sandboxes service and `nylorun sandbox enable` (F7.2, first part).** A new image, `ghcr.io/nylorun/sandboxes`, versioned with the Runtime, drives agent-sandbox v1.0.5 Sandboxes in one namespace per Tenant: `PUT`, `GET ?wait=` and `DELETE /v1/pods/{name}` with an operation id, `/ready` and `/v1/info`, behind a bearer token only the runtime container holds. `nylorun sandbox enable --context <name>` installs into that kubeconfig context only (the pinned controller when absent, the namespace `nylorun-sbx-<tenant>`, a ServiceAccount whose Role covers Sandbox lifecycle and join Secrets, no NetworkPolicy or exec rights), refuses a cluster whose NetworkPolicy it cannot prove enforced, records `<Host root>/sandboxes/cluster.json` and the token, and adds the `sandboxes` service to the Tenant; `nylorun sandbox disable` and `nylorun sandbox status` remove and report it. Sessions do not run in pods yet: that comes with the Harness API and egress-gate.
+
+### Patch Changes
+
+- 3aaf640: **Closing a Tenant no longer hangs on work that ignores its abort.** `MemoryExecution.stop()` and `RestateExecution.stop()` now abort running handlers and wait for them only up to a stop grace (`stopGraceMs`, default `DEFAULT_STOP_GRACE_MS`, 30 s), then abandon the ones still running. Before, `stop()` waited without a limit, so `TenantRuntime.close()` hung in its first step (`detach`) before its bounded `idle` step could run. Shutdown semantics are unchanged: work is aborted as a `shutdown`, and an abandoned advance's lease lapses so the next advance takes the session over. A Tenant that owns its execution uses its advance grace period as the stop grace; the Restate Worker logs a warning when it abandons handlers and drops their connections, so Restate retries them.
+- 167d01a: **A run core stops while it asks about a pending Action is no longer held.** When core cancelled a run, or stopped it for a shutdown, while the harness waited for core's answer to an Action's `effect.intent`, the harness then held the run for the whole `holdMs` (5 minutes by default): it listened for an abort that had already happened. The run kept its lease, and closing the Tenant waited for it. The harness now gives such a run back at once.
+- Pin core to the tested release.
+- Pin harness to the tested release.
+- Updated dependencies [b352feb]
+- Updated dependencies [6077272]
+- Updated dependencies [b6bf1f5]
+- Updated dependencies [8ed4ea6]
+- Updated dependencies [167d01a]
+- Updated dependencies [678e085]
+- Updated dependencies [926711b]
+- Updated dependencies
+  - @nylorun/core@0.12.0-beta
+  - @nylorun/harness@0.22.0-beta
+
+## 0.15.2-beta
+
+### Patch Changes
+
+- 0cbd5c9: The stream relay's lag is never negative: an idle relay confirms one byte past `pg_current_wal_lsn()`, which reported a lag of -1 and failed `/v1/admin/status` validation (400), so Studio showed an idle, healthy Tenant as unavailable.
+
 ## 0.15.1-beta
 
 ### Patch Changes

package/README.md

@@ -50,6 +50,18 @@ them. A local Tenant packs `core,loop` into the `runtime` container and
 api|worker|all` is the deprecated name of `--service core`, `loop` and
 `core,loop`.
 
+`harness` runs alone: agent turns, stdio MCP servers and workspaces, apart from
+core. With `NYLORUN_HARNESS=remote`, core runs no turn itself and opens the
+Harness API listener (`NYLORUN_HARNESS_LISTEN_HOST`, `NYLORUN_HARNESS_LISTEN_PORT`,
+default 4200, `NYLORUN_HARNESS_ALLOWED_HOSTS`), which accepts only
+`NYLORUN_HARNESS_TOKEN`; `in-process` (the default outside Compose) runs turns in
+core's process. A `harness` process needs `NYLORUN_HARNESS_URL`,
+`NYLORUN_HARNESS_TOKEN`, `NYLORUN_GATES_URL` and `NYLORUN_HARNESS_ROOT` (default
+`/harness`), answers `/health` on `127.0.0.1:4300`, and refuses to start with a
+database, Restate, keys or gates credential in its environment. A local Tenant
+runs it in the `harness` container, on a network with only the runtime and the
+gateway ([DEPLOYMENT.md](../DEPLOYMENT.md#the-harness-agent-turns-mcp-servers-and-workspaces)).
+
 The container is configured by its environment, which a local Tenant's Compose file
 sets: `NYLORUN_DATABASE_URL` (required), `NYLORUN_RESTATE_INGRESS_URL`,
 `NYLORUN_RESTATE_ADMIN_URL`, `NYLORUN_WORKER_URL` and
@@ -59,12 +71,22 @@ sets: `NYLORUN_DATABASE_URL` (required), `NYLORUN_RESTATE_INGRESS_URL`,
 `NYLORUN_BROWSER_ACCESS` (`on` or `off`) and the operator listener
 (`NYLORUN_ADMIN_LISTEN_PORT`, `NYLORUN_ADMIN_LISTEN_HOST`,
 `NYLORUN_ADMIN_ALLOWED_HOSTS`). A process that runs `loop` sends its model
-calls to the gate at `NYLORUN_GATES_URL` with `NYLORUN_GATES_TOKEN`; in a
-container it refuses to start without them. A `gates` process needs only
+calls to the gate at `NYLORUN_GATES_URL`; in a container it refuses to start
+without it and `NYLORUN_GATES_TOKEN`. That token is core's credential: the gate
+accepts only it for vault writes, token signing and Action deliveries. Model and
+remote MCP calls carry a run token instead, which the loop mints for each
+session it advances, and the gate takes the call's session, turn and agent from
+it; a token whose turn was cancelled or whose session another process took over
+is refused with `409 run_stale`. A `gates` process needs only
 `NYLORUN_DATABASE_URL`, `NYLORUN_GATES_TOKEN`, its listener
 (`NYLORUN_GATES_LISTEN_HOST`, `NYLORUN_GATES_LISTEN_PORT`, default 4100, and
 `NYLORUN_GATES_ALLOWED_HOSTS`) and the Host's `tenant/` directory, which it
-never writes; it serves the database's one Tenant. `NYLORUN_PACKING` (`combined` or `split`) is logged at startup.
+never writes; it serves the database's one Tenant. With `egress`
+(`--service gates,keys,egress`, for pod sandboxes) the same process runs
+egress-gate on `NYLORUN_EGRESS_LISTEN_HOST`:`NYLORUN_EGRESS_LISTEN_PORT`
+(default `0.0.0.0:4200`): a CONNECT proxy that admits a pod's egress token and
+tunnels only to the hosts its sandbox spec allows, on 443 or 80, never to a
+private address. `NYLORUN_PACKING` (`combined` or `split`) is logged at startup.
 
 With an operator listener (a local Tenant's default: container port 4001), the Host
 serves two ports. The public one serves the Tenant API, to browsers too when
@@ -97,17 +119,22 @@ The Host root is `NYLORUN_HOME` or `~/.nylorun` (for a local Tenant,
 | `POST /v1/admin/host/shutdown` | admin key | Host-private; not in `@nylorun/admin` |
 | `GET /v1/admin/openapi.json` | admin key | The Admin API's OpenAPI 3.2 document |
 | `/v1/*` Tenant routes | application key, subject token or delivery token | Require `Nylorun-Protocol`; nothing names the Tenant |
+| `GET /v1/artifact-links/{token}` | the link itself | A capability link to one artifact version (protocol 6): no credential, no `Nylorun-Protocol`, Range supported; a folder's link opens its zip, or one of its files |
 
 Every route checks `Host` first (`421 host_rejected`) and rejects non-JSON bodies
-with `415 unsupported_media_type`. An `Origin` is `403 origin_rejected` on
+with `415 unsupported_media_type`, except an artifact upload (`POST /v1/artifacts`,
+`POST /v1/artifacts/{id}/versions`), whose body is the file in any media type. An `Origin` is `403 origin_rejected` on
 `/health`, `/ready`, admin routes, and everywhere when browser access is off.
 With browser access on (feature `browser-access`: a local Tenant's default, or
 `browserAccess` in `host.json`), the Host answers preflights for browser routes
 from the route alone, and the Tenant admits an `Origin` only with a publishable
 key (`Nylorun-Key`) that lists it, adding CORS headers only then; Tenant keys
 and delivery tokens are refused from browsers before they are looked up. Missing or
-unsupported protocol → `426` before authentication (the Host serves protocols 4 and 5).
-Protocol 5 clients name no Tenant. A protocol 4 client's `Nylorun-Tenant` naming
+unsupported protocol → `426` before authentication (the Host serves protocols 4, 5 and 6;
+protocol 6 adds file and folder artifacts and message `parts`; at each turn's end the
+Runtime exports `/workspace/outputs` of the session's sandbox as a version of its
+`outputs` folder, read at `/v1/artifacts/{id}/versions/{n|latest}/tree`, `/files/{path}`,
+`/diff?from=` and `/zip`). Protocol 5 and 6 clients name no Tenant. A protocol 4 client's `Nylorun-Tenant` naming
 another Tenant (or malformed), a publishable key of another Tenant, a Tenant that
 could not be opened and rejected credentials → opaque `404` with identical body. A path or method no route serves is `404 Route not found`
 once the caller is known.

package/dist/adapters/execution/restate.d.ts

@@ -1,4 +1,4 @@
-import { type DurableExecution, type StuckInvocation, type Wake, type WorkerHandlers } from "../../execution/types.js";
+import { type DurableExecution, type SandboxSignal, type StuckInvocation, type Wake, type WorkerHandlers } from "../../execution/types.js";
 export interface RestateTimeouts {
     /**
      * How long an invocation may run without journal progress before Restate
@@ -66,6 +66,12 @@ export interface RestateExecutionOptions {
     forceRegistration?: boolean;
     /** How long `start` keeps retrying registration while Restate comes up. Default 60000. */
     registrationTimeoutMs?: number;
+    /**
+     * How long `stop` waits for running handlers after aborting them; the ones still running
+     * then are abandoned and their attempts end with the dropped connections, so Restate
+     * retries them. Default `DEFAULT_STOP_GRACE_MS`.
+     */
+    stopGraceMs?: number;
     /** Replaces the SDK's console logging. */
     logger?: (level: string, message: string) => void;
 }
@@ -91,9 +97,11 @@ export declare class RestateExecution implements DurableExecution {
         tenant: string;
         timer: string;
         action: string;
+        sandbox: string;
     };
     wake(tenantId: string, sessionId: string, wake: Wake): Promise<void>;
     deliver(tenantId: string, actionId: string): Promise<void>;
+    sandbox(tenantId: string, sandboxId: string, signal: SandboxSignal): Promise<void>;
     timer(tenantId: string, key: string, at: Date): Promise<void>;
     armSweep(tenantId: string): Promise<void>;
     disarmSweep(tenantId: string): Promise<void>;

package/dist/adapters/execution/restate.js

@@ -22,13 +22,19 @@
  *   no state and makes no `ctx.run`: its only journal entry is the delayed
  *   self-send after a `retry` result, so an endpoint that is down never spends
  *   the retry budget, which is for infrastructure errors.
+ * - `<prefix>NylorunSandbox`, key `<tenantId>:<sandboxId>` (F7.2). Its
+ *   exclusive handlers serialize a pod sandbox's reconciles and timers:
+ *   `reconcile` calls `WorkerHandlers.sandbox`; `arm` records a timer's latest
+ *   time (`idle`, `ttl`, or the reconcile's own `retry`) and sends a delayed
+ *   `fire`; a `fire` for an older time is a no-op, as the timer object's.
+ *   What a run answers arms its timers and its retry.
  *
  * Wakes, deliveries, timers and sweep arming go through the ingress as one-way
  * sends, so an API node can call them without ever calling `start`.
  */
 import { createServer } from "node:http2";
 import * as restate from "@restatedev/restate-sdk";
-import { WAKE_REASONS, parseSessionKey, sessionKey, } from "../../execution/types.js";
+import { DEFAULT_STOP_GRACE_MS, WAKE_REASONS, parseSessionKey, sessionKey, settleWithin, } from "../../execution/types.js";
 const HOUR_MS = 60 * 60 * 1000;
 export function createRestateExecution(options) {
     return new RestateExecution(options);
@@ -55,6 +61,7 @@ export class RestateExecution {
             tenant: `${prefix}NylorunTenant`,
             timer: `${prefix}NylorunTimer`,
             action: `${prefix}NylorunAction`,
+            sandbox: `${prefix}NylorunSandbox`,
         };
         this.ingressUrl = options.ingressUrl.replace(/\/+$/, "");
         this.adminUrl = options.adminUrl.replace(/\/+$/, "");
@@ -72,6 +79,16 @@ export class RestateExecution {
     async deliver(tenantId, actionId) {
         await this.send(this.names.action, sessionKey(tenantId, actionId), "deliver", {});
     }
+    async sandbox(tenantId, sandboxId, signal) {
+        const key = sessionKey(tenantId, sandboxId);
+        if (signal.kind === "reconcile")
+            await this.send(this.names.sandbox, key, "reconcile", {});
+        else
+            await this.send(this.names.sandbox, key, "arm", {
+                timer: signal.timer,
+                at: signal.at,
+            });
+    }
     async timer(tenantId, key, at) {
         await this.send(this.names.timer, sessionKey(tenantId, key), "set", {
             at: at.getTime(),
@@ -143,8 +160,9 @@ export class RestateExecution {
         this.stopping = true;
         for (const controller of this.controllers)
             controller.abort(new Error("Worker stopping"));
-        while (this.inflight.size > 0)
-            await Promise.allSettled(this.inflight);
+        const abandoned = await settleWithin(this.inflight, this.options.stopGraceMs ?? DEFAULT_STOP_GRACE_MS);
+        if (abandoned > 0)
+            this.options.logger?.("warn", `Worker stopped with ${abandoned} handler(s) still running; abandoned`);
         this.handlers = undefined;
         const server = this.server;
         this.server = undefined;
@@ -259,7 +277,48 @@ export class RestateExecution {
             },
             options: serviceOptions,
         });
-        return [session, tenant, timer, action];
+        /** Sets a sandbox timer: the latest time wins; `fire` checks it. */
+        const armSandbox = async (ctx, input) => {
+            ctx.set(`at:${input.timer}`, input.at);
+            const now = await ctx.date.now();
+            ctx
+                .objectSendClient({ name: names.sandbox }, ctx.key)
+                .fire(input, restate.rpc.sendOpts({ delay: Math.max(0, input.at - now) }));
+        };
+        const runSandbox = async (ctx, trigger) => {
+            const { tenantId, sessionId: sandboxId } = parseSessionKey(ctx.key);
+            const result = await this.runAborted(ctx.request().attemptCompletedSignal, (handlers, signal) => {
+                if (!handlers.sandbox)
+                    throw new Error("WorkerHandlers.sandbox is required for sandbox reconciles");
+                return handlers.sandbox(tenantId, sandboxId, trigger, signal);
+            });
+            for (const item of result.arm ?? [])
+                await armSandbox(ctx, item);
+            if (result.retryAfterMs !== undefined)
+                await armSandbox(ctx, {
+                    timer: "retry",
+                    at: (await ctx.date.now()) + Math.max(0, result.retryAfterMs),
+                });
+        };
+        const sandbox = restate.object({
+            name: names.sandbox,
+            handlers: {
+                reconcile: async (ctx) => {
+                    await runSandbox(ctx, "reconcile");
+                },
+                arm: async (ctx, input) => {
+                    await armSandbox(ctx, input);
+                },
+                fire: async (ctx, input) => {
+                    if ((await ctx.get(`at:${input.timer}`)) !== input.at)
+                        return;
+                    ctx.clear(`at:${input.timer}`);
+                    await runSandbox(ctx, input.timer === "retry" ? "reconcile" : input.timer);
+                },
+            },
+            options: serviceOptions,
+        });
+        return [session, tenant, timer, action, sandbox];
     }
     /**
      * Runs a Worker handler with a signal aborted by `stop` or by the end of the Restate
@@ -365,7 +424,7 @@ export async function listStuckInvocations(options) {
     const prefix = options.servicePrefix ?? "";
     if (!/^[A-Za-z0-9_]*$/.test(prefix))
         throw new Error(`Invalid Restate service prefix: ${prefix}`);
-    const services = ["NylorunSession", "NylorunTenant", "NylorunTimer", "NylorunAction"].map((name) => `'${prefix}${name}'`);
+    const services = ["NylorunSession", "NylorunTenant", "NylorunTimer", "NylorunAction", "NylorunSandbox"].map((name) => `'${prefix}${name}'`);
     const clauses = [
         `target_service_name IN (${services.join(", ")})`,
         `status IN ('paused', 'backing-off')`,

package/dist/adapters/replication/pgoutput.js

@@ -145,7 +145,9 @@ export function createPgoutputSource(options) {
             ]);
         },
         async lag() {
-            const { rows } = await pool.query(`SELECT pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)::text AS lag
+            // An idle relay confirms one byte past the last record, ahead of pg_current_wal_lsn():
+            // that is no lag, not -1 (the Admin status refuses a negative lag).
+            const { rows } = await pool.query(`SELECT GREATEST(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn), 0)::text AS lag
          FROM pg_replication_slots WHERE slot_name = $1`, [slot]);
             return rows[0]?.lag == null ? undefined : Number(rows[0].lag);
         },

package/dist/adapters/sandbox/local.d.ts

@@ -0,0 +1,10 @@
+import type { SandboxBackend } from "../../sandbox/types.js";
+export interface LocalBackendOptions {
+    /** Default `/workspace`. */
+    readonly workspace?: string;
+    /** The proxy variables of the moment (the egress token rotates); default none. */
+    readonly proxyEnv?: () => Readonly<Record<string, string>>;
+    /** The base environment of commands (the engine's allowlisted one); `NYLORUN_*` is dropped. */
+    readonly env: Readonly<Record<string, string | undefined>>;
+}
+export declare function localBackend(options: LocalBackendOptions): SandboxBackend;

package/dist/adapters/sandbox/local.js

@@ -0,0 +1,195 @@
+/**
+ * Local backend (F7.2): the workspace of the pod sandbox the engine runs in. The pod is the
+ * boundary (its own user, no capabilities, a read-only root, a NetworkPolicy that lets it reach
+ * only the Harness API, the gates and egress-gate), so commands run as plain child processes:
+ * `/bin/sh -c <command>` in `/workspace`, its own process group, killed whole on a timeout or a
+ * cancel. Files are read and written with `fs`. Egress goes through egress-gate: every command
+ * gets `HTTPS_PROXY`/`HTTP_PROXY` with the sandbox's current egress token as the proxy password
+ * (`proxyEnv`), and nothing of the engine's own environment (`NYLORUN_*`).
+ *
+ * One pod holds one sandbox: every key opens the same `/workspace`. Removing a workspace is the
+ * sandbox's reset or delete (a new volume), so `remove` and `list` do nothing here.
+ */
+import { spawn } from "node:child_process";
+import { accessSync, constants } from "node:fs";
+import { lstat, readdir, readFile, stat, writeFile } from "node:fs/promises";
+import { posix } from "node:path";
+import { SANDBOX_WORKSPACE } from "@nylorun/core/define";
+import { SandboxFileTooLargeError } from "../../sandbox/types.js";
+/** The most of each stream kept from one command (the tools keep 30 000 characters of it). */
+const STREAM_LIMIT = 4 * 1024 * 1024;
+/** This process's environment without the engine's own variables. */
+function baseEnv(env, workspace) {
+    const out = {};
+    for (const [name, value] of Object.entries(env))
+        if (value !== undefined && !name.startsWith("NYLORUN_") && !/_PROXY$/i.test(name))
+            out[name] = value;
+    return { PATH: "/usr/local/bin:/usr/bin:/bin", LANG: "C.UTF-8", TERM: "dumb", ...out, HOME: workspace };
+}
+/** Collects a stream, keeping at most `STREAM_LIMIT` bytes (head first). */
+function collector() {
+    const chunks = [];
+    let size = 0;
+    let dropped = 0;
+    return {
+        push(chunk) {
+            if (size >= STREAM_LIMIT) {
+                dropped += chunk.length;
+                return;
+            }
+            const room = STREAM_LIMIT - size;
+            const kept = chunk.length > room ? chunk.subarray(0, room) : chunk;
+            chunks.push(kept);
+            size += kept.length;
+            dropped += chunk.length - kept.length;
+        },
+        text() {
+            const text = Buffer.concat(chunks).toString("utf8");
+            return dropped > 0 ? `${text}\n… [${dropped} bytes dropped]` : text;
+        },
+    };
+}
+export function localBackend(options) {
+    const workspace = options.workspace ?? SANDBOX_WORKSPACE;
+    const env = baseEnv(options.env, workspace);
+    const exec = (request, signal) => new Promise((resolve, reject) => {
+        if (signal.aborted)
+            return reject(signal.reason ?? new Error("Turn cancelled"));
+        const child = spawn("/bin/sh", ["-c", request.command], {
+            cwd: request.cwd,
+            env: { ...env, ...(options.proxyEnv?.() ?? {}) },
+            stdio: ["ignore", "pipe", "pipe"],
+            // Its own process group, so a timeout or a cancel kills everything it started.
+            detached: true,
+        });
+        const stdout = collector();
+        const stderr = collector();
+        child.stdout.on("data", (chunk) => stdout.push(chunk));
+        child.stderr.on("data", (chunk) => stderr.push(chunk));
+        let killed = false;
+        let timedOut = false;
+        const kill = () => {
+            if (killed || child.pid === undefined)
+                return;
+            killed = true;
+            try {
+                process.kill(-child.pid, "SIGKILL");
+            }
+            catch {
+                child.kill("SIGKILL");
+            }
+        };
+        const timer = setTimeout(() => {
+            timedOut = true;
+            kill();
+        }, request.timeoutMs);
+        const onAbort = () => kill();
+        signal.addEventListener("abort", onAbort, { once: true });
+        child.once("error", (error) => {
+            clearTimeout(timer);
+            signal.removeEventListener("abort", onAbort);
+            reject(error);
+        });
+        child.once("close", (code, by) => {
+            clearTimeout(timer);
+            signal.removeEventListener("abort", onAbort);
+            resolve({
+                exitCode: killed ? (timedOut ? 124 : 130) : (code ?? (by ? 128 : 1)),
+                stdout: stdout.text(),
+                stderr: stderr.text(),
+                killed,
+                timedOut,
+            });
+        });
+    });
+    const missing = (error) => error?.code === "ENOENT" || error?.code === "ENOTDIR";
+    const handle = {
+        workspace,
+        exec,
+        async readFile(path) {
+            try {
+                return await readFile(path, "utf8");
+            }
+            catch (error) {
+                if (missing(error))
+                    return undefined;
+                throw error;
+            }
+        },
+        async readBytes(path, maxBytes) {
+            let size;
+            try {
+                size = (await stat(path)).size;
+            }
+            catch (error) {
+                if (missing(error))
+                    return undefined;
+                throw error;
+            }
+            if (size > maxBytes)
+                throw new SandboxFileTooLargeError(path, size, maxBytes);
+            return new Uint8Array(await readFile(path));
+        },
+        async listFiles(dir, maxEntries) {
+            try {
+                if (!(await lstat(dir)).isDirectory())
+                    return undefined;
+            }
+            catch (error) {
+                if (missing(error))
+                    return undefined;
+                throw error;
+            }
+            const entries = [];
+            const walk = async (absolute, relative) => {
+                for (const name of (await readdir(absolute)).sort()) {
+                    const path = posix.join(absolute, name);
+                    const inFolder = relative === "" ? name : `${relative}/${name}`;
+                    const info = await lstat(path);
+                    if (info.isSymbolicLink())
+                        continue;
+                    if (info.isDirectory()) {
+                        if (!(await walk(path, inFolder)))
+                            return false;
+                    }
+                    else if (info.isFile()) {
+                        entries.push({ path: inFolder, size: info.size });
+                        if (entries.length > maxEntries)
+                            return false;
+                    }
+                }
+                return true;
+            };
+            const complete = await walk(dir, "");
+            return { entries, truncated: !complete };
+        },
+        async writeFile(path, content) {
+            await writeFile(path, content, "utf8");
+        },
+        async stop() { },
+    };
+    return {
+        name: "local",
+        isolation: "container",
+        async probe() {
+            const base = { name: "local", isolation: "container" };
+            try {
+                accessSync(workspace, constants.W_OK);
+                return { ...base, available: true, reason: `the pod sandbox's ${workspace}` };
+            }
+            catch (error) {
+                return {
+                    ...base,
+                    available: false,
+                    reason: `${workspace} is not writable: ${error instanceof Error ? error.message : String(error)}`,
+                };
+            }
+        },
+        unmet: () => undefined,
+        open: async () => handle,
+        async remove() { },
+        async list() {
+            return [];
+        },
+    };
+}

package/dist/adapters/sandbox/virtual.js

@@ -4,8 +4,9 @@
  * It is not a VM boundary; it exists so the first run and CI work on any machine.
  */
 import { mkdirSync, readdirSync, rmSync } from "node:fs";
-import { join } from "node:path";
+import { join, posix } from "node:path";
 import { SANDBOX_WORKSPACE } from "@nylorun/core/define";
+import { SandboxFileTooLargeError } from "../../sandbox/types.js";
 const ENV = Object.freeze({
     PATH: "/usr/local/bin:/usr/bin:/bin",
     HOME: SANDBOX_WORKSPACE,
@@ -104,6 +105,40 @@ export function virtualBackend(options) {
                         return undefined;
                     return fs.readFile(path, "utf8");
                 },
+                async readBytes(path, maxBytes) {
+                    if (!(await fs.exists(path)))
+                        return undefined;
+                    const stat = await fs.stat(path);
+                    if (stat.size > maxBytes)
+                        throw new SandboxFileTooLargeError(path, stat.size, maxBytes);
+                    return fs.readFileBuffer(path);
+                },
+                async listFiles(dir, maxEntries) {
+                    if (!(await fs.exists(dir)) || !(await fs.lstat(dir)).isDirectory)
+                        return undefined;
+                    const entries = [];
+                    const walk = async (absolute, relative) => {
+                        for (const name of (await fs.readdir(absolute)).sort()) {
+                            const path = posix.join(absolute, name);
+                            const inFolder = relative === "" ? name : `${relative}/${name}`;
+                            const stat = await fs.lstat(path);
+                            if (stat.isSymbolicLink)
+                                continue;
+                            if (stat.isDirectory) {
+                                if (!(await walk(path, inFolder)))
+                                    return false;
+                            }
+                            else if (stat.isFile) {
+                                entries.push({ path: inFolder, size: stat.size });
+                                if (entries.length > maxEntries)
+                                    return false;
+                            }
+                        }
+                        return true;
+                    };
+                    const complete = await walk(dir, "");
+                    return { entries, truncated: !complete };
+                },
                 async writeFile(path, content) {
                     await fs.writeFile(path, content, "utf8");
                 },