@nylorun/runtime 0.24.0-beta → 0.25.0-beta

package/CHANGELOG.md

@@ -1,5 +1,109 @@
 # Changelog
 
+## 0.25.0-beta
+
+### Minor Changes
+
+- 3a32a57: **Files by hash and images by digest (Q6, Q14).** An Environment's `files` entry may name files by content hash, and a pod Environment runs its image by digest once a pod ran it.
+
+  - **Files by hash.** `{ path, files: { "<relative path>": "sha256:<hex>" } }` copies each definition file the Tenant holds (`PUT /v1/files/sha256:<hex>`, at most 10 MiB each, at most 1000 per entry) under `path`, before `setup`. A file the Tenant does not hold is refused when the manifest is saved or the Environment made. The engine reads them as a folder through `host/files/{index}`. `@nylorun/agents`: `environmentFiles(path, contents)` hashes them, and `manifests.save(manifest, { files })` and `plan()` upload those the Runtime lacks.
+  - **Images by digest.** The sandboxes service reports the workload container's `imageID`; the first time a pod is ready the Runtime records it (`pod.imageDigest` in the view) and every later apply runs that digest, so a stop, relaunch or reset never pulls another image under the same tag. A fork pulls afresh. Migration 0027.
+  - **Studio** lists files by hash on the Environment page.
+
+- af90d1e: **Forks: how an Environment gets a new manifest version (Q15, Q17).** `POST /v1/environments/{id}/fork` makes a new Environment `id` from a manifest version (default the source's manifest at its latest), its disk started from a version of the source's (`disk`: `latest`, the default, snapshots first; a version; or `empty`) and the secrets the new manifest declares copied inside the Runtime (`secrets: "none"` copies none): each value is unsealed and sealed again under the new vault, with the slot's binding. The source keeps what it runs, its disk and its sessions, and gets `environment.forked`; the new Environment's view has `forkedFrom`. `dryRun: true` answers the plan: the disk, the secrets copied and left behind, required slots still missing, the changes and the sessions that stay. Migration 0026.
+
+  - **Setup on a forked disk.** A pod fork onto other setup runs the new version's setup on the copied disk before its first turn: the engine connects first (the join answer's `afterSeed`), core writes the disk version through it, then the engine runs the setup, and turns wait until it reports it done. A fork onto the same setup, a restore and a lost-volume recovery skip it, as before.
+  - `@nylorun/core`: `ForkEnvironmentRequestSchema`, `EnvironmentForkPlanSchema`, `ForkEnvironmentResponseSchema`, `EnvironmentView.forkedFrom`, the `environment.forked` event and `forkedFrom` on `environment.created`.
+  - `@nylorun/agents`: `client.environments.fork(id, { id, manifest?, disk?, secrets?, labels?, ownerUserId? })` and `planFork()`.
+
+- 104f24f: **Environment manifests, and Environments that never change (protocol 14, D63 to D65).** An Environment's definition can now be saved like an agent: a named, versioned environment manifest, written as a JSON file with a published schema and checked with a plan before it is saved. An Environment is made from one version and keeps it for life; a save changes no Environment. The Runtime speaks protocol 14 only; sessions, Environments and vaults carry over. See MIGRATION.md.
+
+  - **Environment manifests.** `PUT /v1/environment-manifests/{name}` saves the next version, or answers the latest `unchanged`; `If-Match: <version>` refuses a save over a newer one (`412 manifest_changed`), and `?dryRun=true` answers the plan (changes, errors, warnings, the Environments behind) and saves nothing. Each version records `savedBy`. `GET` (latest), `GET /v1/environment-manifests`, `.../versions`, `.../versions/{n}` and `DELETE` (refused while an Environment names it, `409 manifest_in_use`). Migration 0025.
+  - **Pinned Environments.** `PUT /v1/environments/{id}` takes `manifest: "name"` (the latest at creation) or `"name@n"`. On an existing Environment only `labels` and `ownerUserId` may change; fields that match are accepted, and anything else is `409 environment_immutable` with `details.fields`. The view's `recipe` is now `manifest`, with `manifestName`, `manifestVersion`, `manifestHash` and `slots`; `version` and `appliedVersion` no longer move. `environment.created` names the manifest.
+  - **Secret slots.** A manifest's (or inline) `secrets` declares each secret's name, type, binding, `required` and `description`, never a value. On such an Environment a secret write sends only `{ value }` (or `{ headers }`), the binding comes from the slot, and an undeclared name is `409 secret_not_in_manifest`.
+  - **`requires`.** Agent manifest v7 takes `requires: { sandbox?: "none" | "any" | "pod", secrets? }` (`.requires()` in `@nylorun/core/define`), checked when a session opens (`409 environment_unsuitable`, `details.missing`). Turn variants keep it.
+  - **Dry runs.** `PUT /v1/agents/{id}?dryRun=true` checks a definition and saves nothing (`dryRun: true` in the answer).
+  - `@nylorun/core`: `PROTOCOL_VERSION` 14, `HOST_PROTOCOL` `{ min: 14, max: 14 }`, required feature `environment-manifests`; `EnvironmentManifestSchema`, `EnvironmentSecretSlotSchema`, `parseEnvironmentManifestRef` and the view, plan and save schemas; JSON Schemas at `@nylorun/core/schemas/environment-manifest.json` and `@nylorun/core/schemas/agent-manifest.json` (`$id` `https://docs.nylorun.com/schemas/environment-manifest/v1.json` and `.../agent-manifest/v7.json`).
+  - `@nylorun/agents`: `client.environments.manifests` (`save`, `plan`, `get`, `list`, `versions`, `delete`), `environmentManifest()`, `EnvironmentInput` (was `EnvironmentRecipeInput`), value-only `secrets.put`, and `saveAgent(agent, { dryRun })`.
+  - **Studio** edits only an Environment's labels and owner guard, shows the manifest it was made from, and its setup and files are read-only.
+  - `nylorun environments put` help says what it runs never changes.
+
+- c96b9a8: **An agent manifest is flat: manifest v7, a `runtime` block the Runtime writes, and tools known by name (protocol 13, Harness API 3).** A manifest listed capabilities, each with its own instructions, tools, skills and servers, and the Runtime added its own as more capabilities. Now the agent's parts sit at the top level, a plugin's skills and servers say which plugin brought them, and what the Runtime adds to a session's agent is one `runtime` block. A fresh start: the Runtime speaks protocol 13 only, so save your agents again; sessions from protocol 12 do not carry over. See MIGRATION.md.
+
+  - **Manifest v7.** `{ manifestSchemaVersion: 7, id, name?, description?, metadata?, outputSchema?, instructions?, tools?, skills?, mcpServers?, runtime? }`. Versions 3 to 6 and a `capabilities` field are refused with a message to rebuild. `SkillManifest.plugin` and `McpServerManifest.plugin` are `{ name, version? }`, set by `.plugin(dir)` from `plugin.json`.
+  - **The `runtime` block.** `{ sandbox?, tools?, instructions? }`, written only by the Runtime when a session pins the agent: the sandbox spec and tools, `save_artifact`, `read_artifact`, `load_skill` and `read_skill_resource`, the skill catalog and the notes that go with them. A definition or a turn variant that sets it is refused. `runtimeManifestFor` in `@nylorun/core/define` replaces `sandboxCapabilityManifest`, `artifactsCapabilityManifest` and `toolsCapabilityManifest`. `.skills()` and `.plugin()` no longer add the skill tools or the catalog.
+  - **Names.** An agent's tool may not take a Runtime tool's name (`RUNTIME_TOOL_NAMES`) or contain `__`, and names are unique across tools, skills and servers.
+  - **Turn variants** may remove tools, tighten MCP servers and change instructions, name, description and metadata. Skills are fixed, and a variant never sets `runtime` or changes `outputSchema`.
+  - **Tools by name.** `capabilityId` is gone from `tool.completed`, `mcp.discovered`, tool call rows, `EffectIntent`, gate requests, checkpoints and `SessionToolRef`. An HTTP tool's `server` is its URL's host. `Implementations` is `{ tools }`, keyed by tool name (`Agent.from(json, { tools })`), `AgentBinding.declarations` is `parts`, and observe events say `partId`.
+  - **Harness API 3.** `hello.capabilities` is `features`, and the `harness` pseudo-capability is gone: a native tool is known by its name, and one whose name is taken fails the turn at its start. `createHarness({ features })` in `@nylorun/harness`. `examples/harness-minimal` speaks it.
+  - **Studio.** The Agent tab shows instructions, tools (HTTP and agent badges), skills and MCP servers, each skill and server with its plugin's badge and a filter by plugin, and a session's Runtime section: its sandbox, tools and instructions.
+  - `@nylorun/core`: `PROTOCOL_VERSION` 13, `HOST_PROTOCOL` `{ min: 13, max: 13 }`, required feature `flat-manifest`, `HARNESS_API_VERSION` 3; `PluginLabel`, `RuntimeManifest`, `RUNTIME_TOOL_NAMES`, `isRuntimeToolName`, `runtimeToolOf`, `isSkillTool`, `SKILLS_USAGE` and `formatSkillCatalog` in `/define`.
+  - `@nylorun/agents`, `@nylorun/admin`, `@nylorun/studio` and `nylorun` send protocol 13.
+
+- 94bf329: **Each model says what it reads, a file it does not read is refused before a turn starts, and a file is an artifact the model reads again after its turn (protocol 12).** A message could name any file, and a model that could not read it failed the turn at its model call; every later turn sent the file's bytes again. Now the Runtime knows what the Tenant's model reads, checks a message's files when it is accepted, and sends a file's bytes only in the turn that brought it. Protocol 12 only adds to 11: the Runtime serves both.
+
+  - **Model inputs.** A model reads `text`, and `image`, `pdf`, `office`, `audio` or `video` when its catalog says so or the operator adds it where its API carries it natively (images everywhere; PDFs on the Anthropic, OpenAI, Azure and Google APIs; Office documents on the OpenAI Responses APIs; audio and video on Google's APIs and OpenAI-compatible completions). `GET /v1/tenant/models` gives each model `inputs` and `addableInputs`, `GET /v1/tenant/providers` each provider's `inputs`, and **`PUT /v1/tenant/models/inputs`** (the Management API; `admin.models.setInputs({ provider, model, inputs })`) sets the added ones, kept in the Tenant setting `model.inputs`. The Model Gate gives the provider adapter a model with an added image, so it is sent.
+  - **A custom endpoint reads text only by default.** It used to be sent images. Set `settings.inputs` on `PUT /v1/tenant/model` (`image`, `pdf`, `audio`, `video`) for what yours reads; `office` is refused there. See MIGRATION.md.
+  - **The accept check.** A message naming a file of a type the Tenant's model does not read is `422 attachment_unsupported`, with `details.parts` (`artifactId`, `name`, `mediaType`, `reason`), and no turn starts. Only the type is checked, never the size. Without a model in the vault nothing is refused.
+  - **Files as artifacts.** After its turn a file is a reference line, `[Uploaded artifact "name" · <artifactId> v<n> · read_artifact to view]` (a tool's: `[Tool <name> returned artifact …]`). Our harness sends the turn's own files once more, as media, in one user message after the conversation, so a message reads the same in every later call. At the gate, an image the model reads is an image, a text file a `<file>` block, and any other file, or one that cannot be read, a note: no file fails a call. PDFs, Office documents, audio and video are notes until the gate sends them natively.
+  - **`read_artifact` for every agent.** Every agent's session has it (an agent that declares its own keeps it), so every agent session pins its definition with it added. It takes a `version`, reads the Tenant-wide artifacts the session's messages named, and shows any file the Tenant's model reads; one it does not read fails `artifact.unsupported`.
+  - **Compaction** keeps the summarized part's reference lines (`files` on the compaction entry), and counts the turn's files by type and size in its token estimate.
+  - `@nylorun/core`: `PROTOCOL_VERSION` 12, required feature `model-inputs`, `HOST_PROTOCOL` `{ min: 11, max: 12 }`, error code `attachment_unsupported`; `MODEL_INPUTS`, `ModelInputSchema`, `PutModelInputsRequestSchema`, `ModelInputsViewSchema`, `CustomModelSettings.inputs`; `mediaReferenceLine` and `mediaReferenceOf` in `/define`; `TranscriptCompactionEntry.files`.
+  - `@nylorun/agents`, `@nylorun/studio` and `nylorun` send protocol 12.
+
+- bd80e14: **PDFs, Office documents, audio and video reach the model, and files in AG-UI and A2A messages.**
+
+  - **Native blocks.** A file the Tenant's model reads is sent in its API's own block, no longer as a note: a PDF as an Anthropic `document`, an OpenAI Responses `input_file`, an OpenAI-compatible completions `file` part or Google `inlineData`; Office documents as an OpenAI Responses `input_file`; audio as completions `input_audio` (WAV and MP3 only) or Google `inlineData`; video as completions `video_url` or Google `inlineData`. The Runtime converts and resizes nothing. A file the request cannot carry (audio other than WAV or MP3 on completions, say) fails the model call with `invalid_request` before anything is sent, and the call is not retried. The bytes are only in the provider request, never in the reported call or the transcript.
+  - **Files in AG-UI messages.** A user message's image, audio, video and document parts are files: bytes sent inline (a `data` source) are stored as an artifact of the thread's session, and a `file` source with provider `nylorun` names an artifact the caller may read. A thread's messages give a file back as a media part whose source is `{ type: "file", provider: "nylorun", value: <artifact id> }`.
+  - **Files in A2A messages.** A message may carry `raw` file parts beside its text, stored the same way in the context's session. The Agent Card's `defaultInputModes` add the media types the Tenant's model reads (`image/png`, `image/jpeg`, `image/webp`, `image/gif`, `application/pdf`, the three Office Open XML types, `audio/*`, `video/*`).
+  - Each file holds at most 20 MiB, and the two endpoints take bodies up to 28 MiB. The message then goes through the session's usual command path as text and file parts, so a file the model does not read is refused like any other (`attachment_unsupported`): a `RUN_ERROR` in the AG-UI stream, `ContentTypeNotSupportedError` over A2A, and the files that message stored are deleted. A retried message (the same AG-UI message id or A2A `messageId`) reuses the files it stored the first time. A file by URL is refused (AG-UI `400`, A2A `ContentTypeNotSupportedError`): the Runtime fetches nothing.
+
+- aff9bb8: **Sessions on one Environment run their sandbox commands at the same time, and each version of a skill has its own folder (D66).** The SandboxManager ran every call on a workspace one at a time, whichever session sent it, so parallel turns would still have queued at every command. Now each session's calls run in the order it sends them and different sessions' calls overlap; opening, stopping and removing a workspace run alone, after the calls running.
+
+  - **Skill folders.** A skill's files are mounted at `/skills/<name>@<hash>/` (`skillFolderOf` in `@nylorun/core/define`, the hash from its canonical file map), written once and never replaced, so agents with different versions of one skill share an Environment. An agent used as a tool gets its own version too. The instructions, `load_skill`'s `sandboxPath` and skill attribution on `tool.completed` follow the folder.
+
+- f44c018: **Sessions on one Environment run their turns at the same time, and egress is recorded on the Environment (protocol 14, D66, D67).** A second session's turn on an Environment was refused with `409 environment_busy` until the first ended. Now any session on it may start a turn while others run; one turn per session stays. Which sessions work on one disk, and when, is the app's call. See MIGRATION.md, "Parallel turns, and egress on the Environment".
+
+  - **Turns.** `checkEnvironmentTurn` no longer refuses a turn because another session's turn runs, and the flow-tree exception is gone. `environment_busy` still answers stop, reset, snapshot and restore while any turn runs, and a turn while a disk copy runs.
+  - **Egress.** Every connection through egress-gate, allowed or refused, is `environment.egress` on the Environment's stream, with `runningTurns: [{ sessionId, turnId }]`, the turns running when it opened (at most 16). `egress.connection` leaves the session event catalog, and turn traces carry no `CONNECT` span.
+  - **Reading it.** `GET /v1/environments/{id}/events` takes `types` (comma-separated), `since` and `until` (ISO times); `client.environments.events(id, { types, since, until })` in `@nylorun/agents`, and `nylorun environments events <id> --types --since --until`.
+  - `@nylorun/core`: `EgressRunningTurnSchema`, `EgressRunningTurn`; `EgressConnectionPayloadSchema` gains `runningTurns`.
+
+- c8e640b: **A tool call is traced from the log alone to its skill or MCP server and the plugin that brought it.** No protocol change beyond 13. See MIGRATION.md.
+
+  - **`tool.completed`** gains `skill` and `plugin` (`{ name, version? }`), and tool call rows (`GET /v1/sessions/{id}/calls/tool`, `session.toolCalls()`) gain `skill` and `plugin`, `null` when there is none. An MCP call gets its server's plugin. `load_skill` and `read_skill_resource` get the skill they name, and a sandbox tool whose input names `/skills/<name>/` gets that skill; both get the skill's plugin.
+  - **`kind: "harness"`** (`TOOL_CALL_KINDS`): a tool the turn's harness declared in `hello.features.tools`. Such a call said `platform` before.
+  - **`mcp.discovered`** names each server's `plugin`.
+  - `@nylorun/core`: `skillOfToolCall`, `pluginOf` and `ToolCallTrace` in `/define`.
+  - **Studio** shows each tool call's kind, server or skill, and a plugin badge, in the chat and the event details and summaries, and the plugin of each server `mcp.discovered` lists.
+
+### Patch Changes
+
+- 912336a: **A `500 internal_error` is logged.** The Tenant API, the gates listener and the keys service log an unexpected error through the Tenant's or the gate's logger as `request_failed` (`keys_request_failed` for a keys operation), with the method, the route's pattern and the error's message and stack, never a request body, header or argument. The answers are unchanged.
+- a4c431a: **One command, `nylorun`, grouped by the API each command uses.** `nylo` became part of `nylorun`, and the commands that are not the installation's lifecycle moved into groups: `environments` (the Runtime API, an application key), `tenant` (the Management API, a management key) and `local` (Docker and kubectl on this machine). See MIGRATION.md.
+
+  - **`nylorun`:** `up`, `down`, `start`, `stop`, `status`, `studio` and `doctor` stay at the top. `tenant reset|keys|signing-keys|models set|mcp inspect`; `environments list|delete`; `local ls|delete|logs|reset|sandbox|telemetry`; `doctor sandbox`. The old spellings (`nylorun logs`, `reset`, `key`, `mcp`, `sandbox`, `configure`, …) exit 2 naming their replacement.
+  - **Remote installations:** without `--tenant`, `NYLORUN_RUNTIME_URL` with `NYLORUN_MANAGEMENT_KEY` points `status`, `doctor sandbox` and `tenant` at a remote installation, and with `NYLORUN_SERVER_KEY` points `environments` there. `tenant keys` then manages application keys through the Management API; `tenant mcp inspect` and `environments` no longer need a local Tenant.
+  - **`nylo`:** for one release, each command exits 2 naming the `nylorun` command that replaced it. `nylo env` has no replacement: the SDKs read the Project link themselves.
+  - **`@nylorun/cli`:** its `nylo` still runs nylorun's, which now names the replacement.
+  - **`@nylorun/runtime`:** messages that named `nylo configure`, `nylorun sandbox enable|status` or `nylorun key put` name `nylorun tenant models set`, `nylorun local sandbox enable|status` and `nylorun tenant keys put`.
+
+- 1a31e47: **A pod's new volume is seeded through its new engine, never the old one.** A restore or a lost volume moves the pod's host epoch, and the old engine's connection closes only when that reaches every process. A seed that started in between could write the version into the old pod, and the new volume came up without the restored files. A seed now waits for an engine that joined at the Environment's epoch or later, and writes through it.
+- Pin core to the tested release.
+- Pin harness to the tested release.
+- Updated dependencies [3a32a57]
+- Updated dependencies [af90d1e]
+- Updated dependencies [104f24f]
+- Updated dependencies [c96b9a8]
+- Updated dependencies [94bf329]
+- Updated dependencies [b5ea2b5]
+- Updated dependencies [aff9bb8]
+- Updated dependencies [f44c018]
+- Updated dependencies [c8e640b]
+- Updated dependencies
+  - @nylorun/core@0.21.0-beta
+  - @nylorun/harness@0.27.0-beta
+
 ## 0.24.0-beta
 
 ### Minor Changes

package/README.md

@@ -160,6 +160,7 @@ The Host root is `NYLORUN_HOME` or `~/.nylorun` (for a local Tenant,
 | `/v1/*` Runtime API routes (all but `/v1/tenant/*`) | application key or trusted issuer's token | Require `Nylorun-Protocol`; nothing names the Tenant. A management key is `403 key_role_mismatch`, except on `/v1/me` and the public `GET /v1/access/jwks` |
 | `/v1/tenant/*` Management API routes | management key | Require `Nylorun-Protocol`. An application key, alone or acting for a subject, is `403 key_role_mismatch`; a management key with `Nylorun-Subject` or `Nylorun-Scopes` is `403 subject_invalid` |
 | `PUT /v1/tenant/keys/{keyId}` | management key | Creates application key `keyId` or rotates it, and returns it once; `GET /v1/tenant/keys` lists every key (id, role, when issued, never the keys) and `DELETE` deletes one. `studio`, `bootstrap` and management keys are refused |
+| `PUT /v1/tenant/models/inputs` | management key | What a catalog model reads beyond its catalog's inputs (protocol 12): only inputs in its `addableInputs`; an empty list removes them. `GET /v1/tenant/models` gives each model `inputs` and `addableInputs`. A custom endpoint sets `settings.inputs` on `PUT /v1/tenant/model` (text only by default) |
 | `GET /v1/artifact-links/{token}` | the link itself | A capability link to one artifact version (protocol 6): no credential, no `Nylorun-Protocol`, Range supported; a folder's link opens its zip, or one of its files |
 
 Every route checks `Host` first (`421 host_rejected`) and rejects non-JSON bodies
@@ -171,7 +172,12 @@ are refused from browsers before they are looked up. The Runtime sends no CORS h
 `OPTIONS` with `204` and `Allow` only: the operator's reverse proxy answers preflights
 ([DEPLOYMENT.md](../../guides/DEPLOYMENT.md#calling-the-runtime-from-browsers-and-apps)). Missing or
 unsupported protocol → `426` before authentication, unless the request sends neither
-`Nylorun-Protocol` nor `Authorization` (the Host serves protocol 11 only; protocol 11 replaces
+`Nylorun-Protocol` nor `Authorization` (the Host serves protocol 13 only; protocol 13 makes the
+manifest flat, manifest 7: no `capabilities`, tools known by name, the Runtime's own tools in the
+pinned manifest's `runtime` block and no `capabilityId` on events, tool call rows or the Harness
+API, now 3; it closes every session opened before it; protocol 12 adds
+model inputs, the `422 attachment_unsupported` accept check and `read_artifact` for every agent,
+and changes nothing a protocol 11 client sends; protocol 11 replaces
 sandbox resources, the Tenant's default sandbox, inline sandboxes and user and installation
 vaults with Environments, each one sandbox and one vault, under `/v1/environments`, and
 removes the Management API's vault routes, so a protocol 10 client gets `426`; protocol 10 removes
@@ -283,7 +289,8 @@ an epoch; a Worker that takes over after a crash marks in-flight model effects
 `uncertain`, and sends an interrupted MCP or HTTP tool call again with the same key (R5). Vault ciphertext needs that Tenant's own KEK.
 
 A session that names an Environment of kind `virtual` or `pod` gets Runtime-executed
-sandbox tools; the agent declares nothing. The Tenant owns each Environment; backend
+sandbox tools, pinned in its manifest's `runtime` block with `save_artifact`; the agent declares
+nothing, and none of its own tools may take a Runtime tool's name. The Tenant owns each Environment; backend
 names are prefixed `nylorun-<tenant-id>-`.
 
 ## Local Project workflow
@@ -304,6 +311,7 @@ the Project link (outside a project it runs the Tenant `default`); the project's
 | `database-layout-old` | The database holds `tenant_<id>` schemas of an older Runtime, or the `schema_version` tables of a pre-release one: point the Runtime at a new database (locally, a new Tenant: `nylorun start --tenant <new name>`); the old one is left as it is |
 | `426 protocol_unsupported` | Upgrade clients or Host to a compatible set |
 | `421 host_rejected` / `403 origin_rejected` | In a container, list the `Host` in `NYLORUN_ALLOWED_HOSTS`. From a browser, send a trusted issuer's token, never a key |
+| `422 attachment_unsupported` | A message names a file the Tenant's model does not read (`details.parts` says which and why): choose a model that reads it, add the input with `PUT /v1/tenant/models/inputs` (or a custom endpoint's `settings.inputs`), or send it as another type |
 | `403 key_role_mismatch` | The key belongs to the other API: use a management key on `/v1/tenant/*` (`@nylorun/admin`) and an application key everywhere else (`@nylorun/agents`) |
 | `503` for a Tenant | Postgres or Restate is unreachable; `GET /ready` names which |
 | Port in use | Change `NYLORUN_PORT` in `~/.nylorun/tenants/<name>/docker/.env` and run `nylorun start` |

package/dist/api/a2a/card.d.ts

@@ -12,4 +12,14 @@ export interface CardDefinition {
     };
     readonly implementationVersion: string;
 }
-export declare function agentCard(agentId: string, definition: CardDefinition): Record<string, unknown>;
+import type { ModelInput } from "@nylorun/core/contracts";
+/**
+ * The input modes for a model that reads `inputs`: text and JSON, then the media types of the
+ * files it reads. No inputs known (no model in the vault): text and JSON.
+ */
+export declare function inputModesFor(inputs: readonly ModelInput[] | undefined): string[];
+/**
+ * `inputModes`: the media types the agent takes, when its model reads files; default text and
+ * JSON. The output modes stay text and JSON.
+ */
+export declare function agentCard(agentId: string, definition: CardDefinition, inputModes?: readonly string[]): Record<string, unknown>;

package/dist/api/a2a/card.js

@@ -4,8 +4,32 @@
  * `supportedInterfaces` is empty: whoever publishes the card (the gateway in v1) adds its own
  * URL, provider and security schemes.
  */
+import { MODEL_IMAGE_TYPES } from "../../artifacts/media-types.js";
 const MODES = ["text/plain", "application/json"];
-export function agentCard(agentId, definition) {
+/** The media types a card advertises for each file input a model reads. */
+const INPUT_MODES = {
+    image: MODEL_IMAGE_TYPES,
+    pdf: ["application/pdf"],
+    office: [
+        "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+        "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+        "application/vnd.openxmlformats-officedocument.presentationml.presentation",
+    ],
+    audio: ["audio/*"],
+    video: ["video/*"],
+};
+/**
+ * The input modes for a model that reads `inputs`: text and JSON, then the media types of the
+ * files it reads. No inputs known (no model in the vault): text and JSON.
+ */
+export function inputModesFor(inputs) {
+    return [...MODES, ...(inputs ?? []).flatMap((input) => (input === "text" ? [] : INPUT_MODES[input]))];
+}
+/**
+ * `inputModes`: the media types the agent takes, when its model reads files; default text and
+ * JSON. The output modes stay text and JSON.
+ */
+export function agentCard(agentId, definition, inputModes = MODES) {
     const { manifest } = definition;
     const name = typeof manifest.name === "string" && manifest.name !== "" ? manifest.name : agentId;
     const description = typeof manifest.description === "string" && manifest.description !== ""
@@ -17,7 +41,7 @@ export function agentCard(agentId, definition) {
         supportedInterfaces: [],
         version: definition.implementationVersion,
         capabilities: { streaming: false, pushNotifications: false, extendedAgentCard: false },
-        defaultInputModes: MODES,
+        defaultInputModes: [...inputModes],
         defaultOutputModes: MODES,
         skills: [{ id: agentId, name, description, tags: [] }],
     };

package/dist/api/a2a/endpoint.js

@@ -1,5 +1,6 @@
 import { z } from "zod";
 import { readText } from "../http/body.js";
+import { INLINE_BODY_MAX_BYTES } from "../inbound-files.js";
 import { TraceHeaders, tenantRoute } from "../http/define.js";
 import { traceContextOf } from "../../traces/ids.js";
 import { jsonResponse } from "../http/respond.js";
@@ -56,7 +57,7 @@ export function a2aRoutes(api) {
         path: "/v1/a2a/agents/{agentId}",
         tags: TAGS,
         summary: "Call an agent over A2A",
-        description: "A2A 1.0 JSON-RPC, for a subject: `SendMessage` (blocking up to 5 minutes, or `returnImmediately`), `GetTask` and `CancelTask`. A context is one session per subject, agent and `contextId`; a task is one turn. JSON-RPC errors are answered with 200; limits, scopes and unavailable streams stay HTTP errors.",
+        description: "A2A 1.0 JSON-RPC, for a subject: `SendMessage` (blocking up to 5 minutes, or `returnImmediately`), `GetTask` and `CancelTask`. A message is text, one data part, or text and `raw` files (at most 20 MiB each, stored as artifacts of the context's session); a file `url` is `ContentTypeNotSupportedError`. A context is one session per subject, agent and `contextId`; a task is one turn. JSON-RPC errors are answered with 200; limits, scopes and unavailable streams stay HTTP errors.",
         externalDocs: A2A_SPECIFICATION,
         request: {
             params: agentId,
@@ -77,7 +78,8 @@ export function a2aRoutes(api) {
         const version = c.env.incoming.headers["a2a-version"];
         const trace = traceContextOf(c.req.raw.headers);
         return jsonResponse(200, await a2aCall(c.env.tenant, scope, c.req.param("agentId"), {
-            body: () => readText(c.req.raw),
+            // A message may carry files inline.
+            body: () => readText(c.req.raw, INLINE_BODY_MAX_BYTES),
             version: typeof version === "string" ? version : (c.req.query("A2A-Version") ?? null),
             ...(trace ? { trace } : {}),
         }, c.env.outgoing));

package/dist/api/a2a/protocol.d.ts

@@ -1,7 +1,7 @@
 /**
  * A2A 1.0 over JSON-RPC: the wire types the Runtime answers with, the envelope, the error
  * codes and the parameter checks (Host feature `a2a-endpoint`). Only what v1 serves is typed:
- * text and data parts, tasks, the card. Field names are the spec's ProtoJSON (camelCase,
+ * text, data and file parts, tasks, the card. Field names are the spec's ProtoJSON (camelCase,
  * enum values such as `TASK_STATE_WORKING`). No Tenant state is read here.
  */
 /** The A2A version this endpoint speaks (`A2A-Version`, Major.Minor). */
@@ -16,6 +16,16 @@ export type Part = {
     data: unknown;
     mediaType?: string;
     metadata?: Record<string, unknown>;
+} | {
+    raw: string;
+    mediaType?: string;
+    filename?: string;
+    metadata?: Record<string, unknown>;
+} | {
+    url: string;
+    mediaType?: string;
+    filename?: string;
+    metadata?: Record<string, unknown>;
 };
 export interface Message {
     messageId: string;
@@ -120,11 +130,24 @@ export declare function parseEnvelope(text: string): {
 };
 /** `A2A-Version` as Major.Minor; empty means 0.3 (spec §3.6.2). */
 export declare function checkVersion(value: string | null | undefined): void;
-/** A client's message reduced to what the Runtime runs: text, or one JSON value. */
+/** A file sent inline (`raw`, base64): stored as an artifact of the context's session. */
+export interface RawFile {
+    readonly raw: string;
+    readonly mediaType?: string;
+    readonly filename?: string;
+}
+/**
+ * A client's message reduced to what the Runtime runs: text, one JSON value, or text and files
+ * sent inline.
+ */
 export type MessageInput = {
     content: string;
 } | {
     data: unknown;
+} | {
+    files: readonly ({
+        text: string;
+    } | RawFile)[];
 };
 export interface SendParams {
     messageId: string;

package/dist/api/a2a/protocol.js

@@ -1,7 +1,7 @@
 /**
  * A2A 1.0 over JSON-RPC: the wire types the Runtime answers with, the envelope, the error
  * codes and the parameter checks (Host feature `a2a-endpoint`). Only what v1 serves is typed:
- * text and data parts, tasks, the card. Field names are the spec's ProtoJSON (camelCase,
+ * text, data and file parts, tasks, the card. Field names are the spec's ProtoJSON (camelCase,
  * enum values such as `TASK_STATE_WORKING`). No Tenant state is read here.
  */
 /** The A2A version this endpoint speaks (`A2A-Version`, Major.Minor). */
@@ -136,7 +136,12 @@ function partKind(part, field) {
         return a2aFail("invalidParams", `${field} must contain exactly one of text, raw, url, data`, {}, field);
     if (kinds[0] === "text" && typeof part.text !== "string")
         return a2aFail("invalidParams", `${field}.text must be a string`, {}, `${field}.text`);
-    return kinds[0] === "raw" || kinds[0] === "url" ? "file" : kinds[0];
+    if (kinds[0] === "raw" && typeof part.raw !== "string")
+        return a2aFail("invalidParams", `${field}.raw must be base64 bytes`, {}, `${field}.raw`);
+    for (const key of ["mediaType", "filename"])
+        if (part[key] !== undefined && typeof part[key] !== "string")
+            return a2aFail("invalidParams", `${field}.${key} must be a string`, {}, `${field}.${key}`);
+    return kinds[0] === "raw" ? "raw" : kinds[0] === "url" ? "url" : kinds[0];
 }
 /** `SendMessage` params (spec §3.2.1), checked in the order a client can fix them. */
 export function parseSendParams(params) {
@@ -166,8 +171,20 @@ export function parseSendParams(params) {
         input = { content: parts.map((part) => part.text).join("\n") };
     else if (kinds.length === 1 && kinds[0] === "data")
         input = { data: parts[0].data };
+    else if (kinds.includes("url"))
+        return a2aFail("contentTypeNotSupported", "File URLs are not fetched yet; send the bytes inline as raw", { accepted: "raw" });
+    else if (kinds.includes("data"))
+        return a2aFail("contentTypeNotSupported", "This agent accepts text and file parts, or a single data part", { accepted: "text/plain,application/json" });
     else
-        return a2aFail("contentTypeNotSupported", "This agent accepts text parts, or a single data part; files are not supported", { accepted: "text/plain,application/json" });
+        input = {
+            files: parts.map((part, index) => kinds[index] === "text"
+                ? { text: part.text }
+                : {
+                    raw: part.raw,
+                    ...(part.mediaType !== undefined ? { mediaType: part.mediaType } : {}),
+                    ...(part.filename !== undefined ? { filename: part.filename } : {}),
+                }),
+        };
     if ("content" in input && input.content.trim() === "")
         a2aFail("invalidParams", "The message has no text", {}, "message.parts");
     return {

package/dist/api/a2a/routes.d.ts

@@ -6,7 +6,10 @@ import type { TraceContext } from "../../traces/ids.js";
  * task is returned `WORKING` and keeps running; the caller polls `GetTask`.
  */
 export declare const A2A_BLOCKING_WAIT_MS: number;
-/** `GET /v1/a2a/agents/:agent/card`: the agent's card, for a caller who may use the agent. */
+/**
+ * `GET /v1/a2a/agents/:agent/card`: the agent's card, for a caller who may use the agent. Its
+ * input modes name the files the Tenant's model reads.
+ */
 export declare function a2aCard(ctx: TenantContext, scope: AuthScope, agentId: string): Promise<unknown>;
 /**
  * `POST /v1/a2a/agents/:agent`: one A2A JSON-RPC call, for a subject. JSON-RPC answers (errors

package/dist/api/a2a/routes.js

@@ -9,13 +9,20 @@
  * caller's scope (limits and owner checks apply), sessions derived from the caller's own
  * subject, history from Durable Streams. A task id from the client only selects a context and
  * a turn within the caller's own sessions, so another subject's task is `TaskNotFoundError`.
+ *
+ * A message's `raw` file parts are stored as artifacts of the context's session once it is open,
+ * and the message goes in as `parts` (`../inbound-files.ts`): a file the model does not read is
+ * `ContentTypeNotSupportedError`, and the files of a refused message are deleted.
  */
 import { randomUUID } from "node:crypto";
 import { SessionCommandSchema, } from "@nylorun/core/contracts";
 import { commandKey } from "../../core/flow-host.js";
-import { agentCard } from "./card.js";
+import { contentTypeFor } from "../../artifacts/media-types.js";
+import { discardFiles, inlineFileName, messageParts, } from "../inbound-files.js";
+import { agentCard, inputModesFor } from "./card.js";
 import { A2aError, a2aFail, checkVersion, jsonRpcError, jsonRpcResult, parseEnvelope, parseSendParams, parseTaskParams, } from "./protocol.js";
 import { SETTLED_EVENTS, buildTask, contextSessionId, newContextId, parseTaskId, pendingInteraction, taskIdOf, } from "./tasks.js";
+import { currentModelInputs } from "../../model/catalog.js";
 import { accessOf } from "../../tenant/auth.js";
 import { command } from "../../tenant/commands.js";
 import { loadSession, } from "../../tenant/context.js";
@@ -27,9 +34,14 @@ import { putSession } from "../../tenant/sessions.js";
  * task is returned `WORKING` and keeps running; the caller polls `GetTask`.
  */
 export const A2A_BLOCKING_WAIT_MS = 5 * 60_000;
-/** `GET /v1/a2a/agents/:agent/card`: the agent's card, for a caller who may use the agent. */
+/**
+ * `GET /v1/a2a/agents/:agent/card`: the agent's card, for a caller who may use the agent. Its
+ * input modes name the files the Tenant's model reads.
+ */
 export async function a2aCard(ctx, scope, agentId) {
-    return agentCard(agentId, await definitionFor(ctx, agentId, accessOf(scope)));
+    const definition = await definitionFor(ctx, agentId, accessOf(scope));
+    // The files a caller may send: those the Tenant's model reads now.
+    return agentCard(agentId, definition, inputModesFor(await currentModelInputs(ctx)));
 }
 /**
  * `POST /v1/a2a/agents/:agent`: one A2A JSON-RPC call, for a subject. JSON-RPC answers (errors
@@ -60,7 +72,7 @@ export async function a2aCall(ctx, scope, agentId, input, response) {
         if (error instanceof A2aError)
             return jsonRpcError(id, error);
         // Limits, scopes and unavailable streams stay HTTP answers, as on every Tenant route.
-        if (error instanceof HttpError && [403, 429, 503].includes(error.status))
+        if (error instanceof HttpError && [403, 413, 429, 503].includes(error.status))
             throw error;
         ctx.config.logger.warn("a2a request failed", {
             agentId,
@@ -141,7 +153,14 @@ conflict) {
         return { turnId: accepted.turnId, cursor: accepted.cursor };
     }
     catch (error) {
-        if (!(error instanceof HttpError) || error.status !== 409)
+        if (!(error instanceof HttpError))
+            throw error;
+        // A file the agent's model does not read (`attachment_unsupported`).
+        if (error.status === 422)
+            return a2aFail("contentTypeNotSupported", error.message, {
+                reason: error.rejection.code ?? "request_rejected",
+            });
+        if (error.status !== 409)
             throw error;
         if (error.message.startsWith("Idempotency key"))
             return a2aFail("invalidParams", "message.messageId was already used for a different message", {}, "message.messageId");
@@ -162,13 +181,17 @@ async function openContext(caller, sessionId) {
 async function sendMessage(caller, params, signal) {
     const send = parseSendParams(params);
     const idempotencyKey = `a2a:${send.messageId}`;
-    const value = "content" in send.input ? send.input.content : send.input.data;
     let contextId;
     let sessionId;
     let accepted;
     if (send.taskId !== undefined) {
         // A reply on an existing task: only a question it is waiting on can take it.
         const ids = parseTaskId(send.taskId) ?? taskNotFound(send.taskId);
+        const value = "content" in send.input
+            ? send.input.content
+            : "data" in send.input
+                ? send.input.data
+                : a2aFail("contentTypeNotSupported", "A reply takes text or one data part; send files in a new message", { accepted: "text/plain,application/json" });
         if (send.contextId !== undefined && send.contextId !== ids.contextId)
             a2aFail("invalidParams", "message.contextId does not match the task's context", {}, "message.contextId");
         contextId = ids.contextId;
@@ -198,17 +221,21 @@ async function sendMessage(caller, params, signal) {
         sessionId = contextSessionId(caller.access.owner, caller.agentId, contextId);
         await openContext(caller, sessionId);
         const context = contextId;
+        const { body, stored } = await messageBody(caller, sessionId, idempotencyKey, send.input);
         accepted = await run(caller, sessionId, {
             type: "message",
             requestId: randomUUID(),
             idempotencyKey,
-            ...send.input,
+            ...body,
         }, async () => {
             const active = (await contextSession(caller, sessionId))?.activeTurnId;
             const taskId = active ? taskIdOf(context, active) : undefined;
             return taskId
                 ? new A2aError("unsupportedOperation", `The context has a task in progress (${taskId}); reply to it or cancel it`, { taskId })
                 : new A2aError("unsupportedOperation", "The context has unresolved work; start a new context");
+        }).catch(async (error) => {
+            await discardFiles(caller.ctx, stored, caller.access);
+            throw error;
         });
     }
     if (!send.returnImmediately)
@@ -216,6 +243,37 @@ async function sendMessage(caller, params, signal) {
     const { task } = await ownedTask(caller, taskIdOf(contextId, accepted.turnId), send.historyLength);
     return { task };
 }
+/**
+ * A new message's body: its text, its JSON value, or parts with its `raw` files stored as
+ * artifacts of the session (`stored`: the ones this call created). A file the Runtime cannot
+ * store is `InvalidParamsError`; one past a limit stays an HTTP `413`.
+ */
+async function messageBody(caller, sessionId, idempotencyKey, input) {
+    if (!("files" in input))
+        return { body: input, stored: [] };
+    let files = 0;
+    const inbound = input.files.map((part) => {
+        if ("text" in part)
+            return { type: "text", text: part.text };
+        const index = files++;
+        const mediaType = part.mediaType ?? (part.filename !== undefined ? contentTypeFor(part.filename) : "application/octet-stream");
+        return {
+            type: "inline",
+            name: part.filename ?? inlineFileName(mediaType, index),
+            mediaType,
+            data: part.raw,
+        };
+    });
+    try {
+        const { parts, stored } = await messageParts(caller.ctx, sessionId, idempotencyKey, inbound, caller.access);
+        return { body: { parts }, stored };
+    }
+    catch (error) {
+        if (error instanceof HttpError && error.status === 400)
+            return a2aFail("invalidParams", error.message, {}, "message.parts");
+        throw error;
+    }
+}
 /** The interaction a reply answers: the task must be active and paused on a question. */
 async function replyTarget(caller, session, sessionId, turnId, taskId) {
     if (session.activeTurnId !== turnId) {

package/dist/api/ag-ui/endpoint.js

@@ -2,6 +2,7 @@ import { RESPONSE_ALREADY_SENT } from "@hono/node-server/utils/response";
 import { z } from "zod";
 import { EventSchemas, MessageSchema, RunAgentInputSchema } from "@ag-ui/core/schemas";
 import { readJson } from "../http/body.js";
+import { INLINE_BODY_MAX_BYTES } from "../inbound-files.js";
 import { TraceHeaders, tenantRoute } from "../http/define.js";
 import { traceContextOf } from "../../traces/ids.js";
 import { jsonResponse } from "../http/respond.js";
@@ -50,7 +51,7 @@ export function agUiRoutes(api) {
         path: "/v1/ag-ui/agents/{agentId}",
         tags: TAGS,
         summary: "Run an agent",
-        description: "For a person, named by a trusted issuer's token or by `Nylorun-Subject`. The thread is one session per person, agent and thread, created on its first run with `forwardedProps.nylorun.session`. The trailing user message starts a turn (its id is the idempotency key); `resume` answers interrupts. A busy session is a `RUN_ERROR` in the stream.",
+        description: "For a person, named by a trusted issuer's token or by `Nylorun-Subject`. The thread is one session per person, agent and thread, created on its first run with `forwardedProps.nylorun.session`. The trailing user message starts a turn (its id is the idempotency key); its media parts are files: bytes sent inline (`data`, at most 20 MiB each) are stored as artifacts of the session, a `file` source with provider `nylorun` names an artifact, and a URL is refused. `resume` answers interrupts. A busy session, or a file the model does not read, is a `RUN_ERROR` in the stream.",
         request: {
             params: agentId,
             body: {
@@ -62,7 +63,9 @@ export function agUiRoutes(api) {
         responses: { 200: events("The run's events, from RUN_STARTED to its end") },
     }, async (c) => {
         const { scope, subject, access } = callerOf(c);
-        await startRun(c.env.tenant, scope, c.req.param("agentId"), { subject, access }, await readJson(c.req.raw), c.env.outgoing, traceContextOf(c.req.raw.headers));
+        await startRun(c.env.tenant, scope, c.req.param("agentId"), { subject, access }, 
+        // A message may carry files inline.
+        await readJson(c.req.raw, INLINE_BODY_MAX_BYTES), c.env.outgoing, traceContextOf(c.req.raw.headers));
         return RESPONSE_ALREADY_SENT;
     });
     tenantRoute(api, FOR_A_PERSON, {

package/dist/api/ag-ui/history.d.ts

@@ -1,7 +1,12 @@
 /**
  * An AG-UI transcript rebuilt from a session's log, with the ids a live run gave each
- * message (see `translate.ts`), usable as `HttpAgent`'s `initialMessages`.
+ * message (see `translate.ts`), usable as `HttpAgent`'s `initialMessages`. A user message with
+ * files has array content: its text parts, and each file as a media part whose source is the
+ * artifact (`{ type: "file", provider: "nylorun", value: <artifact id> }`), typed by the media
+ * type its `artifact.created` event recorded.
  */
 import type { Message } from "@ag-ui/core";
 import { type LiveEvent } from "@nylorun/core/contracts";
+/** The `provider` of a file source that names one of the Tenant's artifacts. */
+export declare const NYLORUN_FILE_PROVIDER = "nylorun";
 export declare function messagesFromEvents(items: readonly LiveEvent[]): Message[];

package/dist/api/ag-ui/history.js

@@ -1,10 +1,44 @@
 import { isEmptyAssistantMessage, parseTranscriptEvent, } from "@nylorun/core/contracts";
 import { outputText, toolResultOf } from "./translate.js";
+/** The `provider` of a file source that names one of the Tenant's artifacts. */
+export const NYLORUN_FILE_PROVIDER = "nylorun";
+/** A file part as an AG-UI media part, its kind from its media type when the log has it. */
+function filePart(artifactId, mediaType) {
+    const top = mediaType?.split("/")[0];
+    const type = top === "image" || top === "audio" || top === "video" ? top : "document";
+    return {
+        type,
+        source: {
+            type: "file",
+            value: artifactId,
+            provider: NYLORUN_FILE_PROVIDER,
+            ...(mediaType ? { mimeType: mediaType } : {}),
+        },
+    };
+}
+/** A message's parts (protocol 6): its text, or text and media parts when it has files. */
+function partsContent(parts, mediaTypes) {
+    if (!parts.some((part) => part.type === "file"))
+        return parts.flatMap((part) => (typeof part.text === "string" ? [part.text] : [])).join("\n");
+    return parts.flatMap((part) => part.type === "file" && typeof part.artifactId === "string"
+        ? [filePart(part.artifactId, mediaTypes.get(part.artifactId))]
+        : typeof part.text === "string"
+            ? [{ type: "text", text: part.text }]
+            : []);
+}
 export function messagesFromEvents(items) {
     const messages = [];
     const results = new Set();
+    // Each artifact's media type, from its events, for the files a message names.
+    const mediaTypes = new Map();
     let sawAssistant = false;
     for (const raw of items) {
+        if (raw.type === "artifact.created" || raw.type === "artifact.version.created") {
+            const p = (raw.payload ?? {});
+            if (typeof p.artifactId === "string" && typeof p.contentType === "string")
+                mediaTypes.set(p.artifactId, p.contentType);
+            continue;
+        }
         if (raw.type === "command.message") {
             const p = (raw.payload ?? {});
             messages.push({
@@ -14,8 +48,7 @@ export function messagesFromEvents(items) {
                 content: typeof p.content === "string"
                     ? p.content
                     : Array.isArray(p.parts)
-                        ? // A message with parts (protocol 6): its text; files are artifacts, not text.
-                            p.parts.flatMap((part) => (typeof part.text === "string" ? [part.text] : [])).join("\n")
+                        ? partsContent(p.parts, mediaTypes)
                         : JSON.stringify(p.data),
             });
             sawAssistant = false;