insika 0.0.1 → 0.2.0

data/docs/RUNNING-LOCAL.md

@@ -0,0 +1,153 @@
+---
+title: Running locally
+parent: Build an agent
+nav_order: 8
+permalink: /running-local/
+---
+
+# Running the Insika locally
+
+Boots the engine single-process, serving `/studio` and `/v1/*` against a demo
+agent (the `bia` persona on DeepSeek). Every message runs the **same**
+`send_message` the API runs — real tools, skills, and memory.
+
+## Boot
+
+```bash
+cd insika
+DEEPSEEK_API_KEY=sk-... bundle exec ruby scripts/serve_real.rb
+```
+
+> Use **`bundle exec`** (bundler isolation matters — the optional OpenTelemetry gem
+> is in the Gemfile). It is single-process: `Ctrl-C` frees the port immediately.
+
+Open `http://localhost:9292`:
+
+| URL | What |
+|-----|------|
+| `/studio` | management UI (log in with the token; default `local-demo`) |
+| `/studio/chats` | chat with the demo agent (`agent: bia`, `session_id: web`, multi-turn ready) |
+| `/studio/tasks` | tasks / approvals console |
+| `/v1/responses` | OpenAI-Responses ingress (Bearer) — the drop-in API contract |
+| `/v1/agents` | provisioning by definition/pack (Bearer) — `POST` imports, `DELETE /:id` removes |
+| `/v1/messages` | `send_message` sugar (Bearer; SSE when `?stream` is set) |
+
+`POST /v1/messages?stream=false` answers the aggregated turn as JSON. It is also
+the only HTTP surface that can report a **coalesced** message, so it is the only
+one on which an agent configured with `queue_mode: "collect"`
+([Agents](AGENTS.md#queue_mode--when-a-message-arrives-while-the-agent-is-busy))
+actually coalesces:
+
+```jsonc
+// this call owns the reply
+{ "task_id": "9f3c…", "content": "…" }
+
+// this one joined a turn already waiting — deliver NOTHING, no stream is opened
+{ "task_id": "9f3c…", "merged": true }
+```
+
+`/v1/responses` and any open stream never coalesce: their bodies have nowhere to
+put that verdict, and a caller that cannot hear it would send the same answer once
+per fragment. Those requests fall back to one turn per message.
+
+Every `/v1` and `/a2a` route needs the Bearer. The exceptions are `/up` and, when
+`INSIKA_ONBOARDING` is on, `/start.md`, `/models.json` and `/docs*` — a route is closed
+unless it is on the allowlist in `lib/insika/server/app.rb`. With no token configured at all the
+whole surface answers `503`, never open by omission.
+
+## Variables (all optional)
+
+| Env | Default | Effect |
+|-----|---------|--------|
+| `INSIKA_DB` | — (ephemeral memory) | SQLite path → config + execution survive a restart |
+| `BIND` | `http://localhost:9292` | host:port |
+| `ADMIN_TOKEN` | `local-demo` | token for `/studio` |
+| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN` | Bearer for the whole `/v1` + `/a2a` surface |
+| `DEEPSEEK_MODEL` | `deepseek-v4-flash` | model |
+
+With persistence:
+
+```bash
+DEEPSEEK_API_KEY=sk-... INSIKA_DB=./insika.db bundle exec ruby scripts/serve_real.rb
+```
+
+## Pointing a Responses client at the local engine
+
+Anything that speaks the OpenAI Responses contract can drive the local engine —
+that is the whole point of the `/v1/responses` drop-in. Point your client's base
+URL at `http://localhost:9292`, send the API Bearer, and address an agent by id as
+the `model`:
+
+```bash
+curl -N http://localhost:9292/v1/responses \
+  -H "Authorization: Bearer local-demo" \
+  -H "Content-Type: application/json" \
+  -d '{ "model": "bia", "user": "web", "stream": true, "input": "hello" }'
+```
+
+`user` is the session id (any stable id for a multi-turn conversation).
+
+## Wiring an agent's tools back to your backend
+
+If the agent has **data-tools** that call your own HTTP backend (see
+[Tools](TOOLS.md)), and both the engine and that backend run on your machine, the
+tools target `http://localhost:<port>/…` — plain `http` on a loopback address,
+which the egress guard blocks by default (SSRF defense). Enable the opt-in
+**pinned to your backend's host** when you boot:
+
+```bash
+INSIKA_EGRESS_ALLOW_HTTP=1 INSIKA_EGRESS_ALLOW_PRIVATE=1 \
+INSIKA_EGRESS_HOSTS=localhost,127.0.0.1 \
+DEEPSEEK_API_KEY=sk-... bundle exec ruby scripts/serve_real.rb
+```
+
+> `INSIKA_EGRESS_HOSTS` restricts the opened egress to just the internal host
+> (defense-in-depth) — without it, `ALLOW_PRIVATE` opens *any* private destination.
+> These `ALLOW_*` vars are for the fully-local loop only; never set them in the
+> cloud. See [Security](SECURITY.md#egress-the-ssrf-boundary).
+
+## Provisioning an agent
+
+An agent is created from a **definition** — a folder ("pack") with an agent config,
+prompt files, skills, and one data-tool per file:
+
+```
+<pack>/
+  agent.config.json     # { id, model, provider, memory, metadata }
+  *.md                  # prompt files (identity, tools notes, …)
+  skills/<name>/SKILL.md
+  tools/<tool>.json     # one data-tool per file
+```
+
+> **Data-tool URLs must be literal on the pack path.** The pack import does not
+> resolve `{{env.*}}` — bake the backend base URL into each `tools/*.json` at
+> generation time. (Only the *manifest* path resolves `{{env.*}}`.) See
+> [Tools](TOOLS.md#the-one-gotcha-env-templating-is-manifest-only).
+
+Provision it (runs as a client against the live server; the internal token comes
+from the environment, never disk):
+
+```bash
+INSIKA_URL=http://localhost:9292 OPENCLAW_GATEWAY_TOKEN=local-demo \
+  bundle exec ruby scripts/import_pack.rb /path/to/pack
+```
+
+…or `POST /v1/agents` directly, or build the agent by hand in the `/studio`. All
+paths land on the same import. See [Agents](AGENTS.md) for the from-scratch flow
+and the `Insika.agent { … }` DSL.
+
+## Observability (OpenTelemetry, opt-in)
+
+OpenTelemetry is **off by default** (the gems do not even load). To turn it on, see
+traces in a local collector (a one-line Jaeger), and read the attribute convention
+and the production config, see [OBSERVABILITY.md](OBSERVABILITY.md). In short:
+`INSIKA_OTEL=1` + `OTEL_EXPORTER_OTLP_ENDPOINT=…`, and every turn becomes a
+`insika.turn` trace with `insika.tool` / `insika.data_tool` children, plus counters
+and histograms (`insika.turns`, `insika.turn.duration`, `insika.tokens`,
+`insika.cost`) you can chart without aggregating spans.
+
+## See also
+
+- [Agents](AGENTS.md) — create and configure an agent.
+- [Tools](TOOLS.md) — define data-tools and troubleshoot egress.
+- [Deploy](DEPLOY.md) — running the same image durably in a container.

data/docs/SANDBOX.md

@@ -0,0 +1,114 @@
+---
+title: Sandbox
+parent: Ship it
+nav_order: 2
+permalink: /sandbox/
+---
+
+# Sandbox — confined execution
+
+`Insika::Sandbox` is the engine primitive for **confined execution**: a single,
+pluggable interface a tool holds to touch the filesystem and run commands inside
+a bounded environment. It follows the principle of the *narrowest sandbox that
+supports the task* — cheap in-process confinement by default, real container
+isolation when the task warrants it, chosen by configuration rather than code.
+
+It has two halves:
+
+- **FS confinement (`Boundary`) — always on, always host-side.** Every path a
+  tool resolves is proven to live inside a single root *before any IO happens*.
+- **Command exec — via a swappable provider.** `local` (in-process, the default)
+  or `docker` (isolated container). The provider is selected by data on the agent
+  profile, not by branching in tool code.
+
+## The FS boundary
+
+`Insika::Sandbox::Boundary` confines every path to one root directory. It
+rejects, before touching disk:
+
+- `..` traversal that escapes the root (path normalized, then string-contained on
+  a separator boundary — so `/ws-evil` does not pass for root `/ws`);
+- absolute paths outside the root;
+- symlinks: the final path component may never be a symlink (an `lstat` check that
+  also catches a *broken* symlink whose target does not yet exist), and any
+  existing target's real path (`File.realpath`) must also be contained — a symlink
+  inside the root pointing outside is refused rather than followed.
+
+An escape raises `Insika::Sandbox::Escape`; tools rescue it and return a
+structured `{ error: ... }` to the model, never a crashed turn.
+
+The boundary is **always host-side**, for both providers. With `docker`, the FS
+tools still read and write host files (confined by the boundary); the container
+sees the same bytes through a bind mount. Only the risky part — shell exec — is
+isolated. That is the "narrowest sandbox" line: don't containerize a file read.
+
+## Exec providers
+
+`sandbox.exec(command)` returns a `Insika::Sandbox::Result`
+(`exit_status`, `output`, `timed_out`). Both providers enforce a **hard-kill
+wall-clock timeout**: on the deadline the process — and, via its own process
+group, any children — is force-killed and the partial output is returned with
+`timed_out: true`. (The engine's fiber contract forbids `Timeout.timeout`; the
+deadline is enforced with a bounded `Thread#join` and a process-group kill.)
+
+### `local` (default)
+
+Runs the command in-process (`/bin/bash -c`) with the working directory pinned to
+the root. Cheap, no daemon, no image pull. It is **not** an isolation boundary for
+a shell (a command can still read absolute paths or `cd ..`); shell tools stay
+approval-gated, and untrusted execution should use `docker`.
+
+### `docker`
+
+Runs the command inside a throwaway container (`docker run --rm`) with the root
+bind-mounted at a fixed workdir. Conservative defaults: `--network none`, a memory
+cap, a cpu cap, and a minimal image — all overridable. The container is named so
+the timeout teardown can `docker kill` the exact container.
+
+## Declaring a sandbox (config-over-code)
+
+The provider and its policy are **data on the agent profile**, under the
+`sandbox` key — never a branch in tool code:
+
+```ruby
+Insika::AgentProfile.build(
+  id: "coder",
+  # ...
+  sandbox: {
+    provider: "docker",       # "local" (default) | "docker"
+    root:     "/srv/project", # confinement root (default: cwd)
+    timeout:  120,            # per-exec wall-clock seconds
+    image:    "ruby:3.3",     # docker only
+    network:  "none",         # docker only (default: none)
+    memory:   "512m",         # docker only
+    cpus:     "1.0"           # docker only
+  }
+)
+```
+
+A deployment turns that config into the object every tool holds:
+
+```ruby
+sandbox = Insika::Sandbox.build(profile.sandbox)   # => Insika::Sandbox::Env
+
+sandbox.resolve("src/app.rb")        # host path, guaranteed inside the root
+sandbox.exec("bundle exec rspec")    # => Result(exit_status:, output:, timed_out:)
+```
+
+The config round-trips through the JSON store (Studio edits), so profile and
+runtime never drift. Absent (`nil`) config means a deployment builds a `local`
+sandbox by default.
+
+## Reference deployment
+
+`plugins/insika-code` (a tier-2 code plugin — see [Plugins](PLUGINS.md)) is the
+reference consumer: the FS/shell toolset
+(`read_file`, `list_dir`, `grep`, `write_file`, `edit_file`, `bash`) is built on
+this primitive, and `examples/insika-code/boot.rb` declares the `sandbox` block
+on its profile while the plugin builds the matching `Sandbox` from the same
+config. See [`examples/insika-code/README.md`](https://github.com/guizaols/insika/blob/main/examples/insika-code/README.md).
+
+The sandbox is one of two independent controls on the high-risk tools; the other
+is the engine's human-approval gate (`approvals_required` + the `ToolEnvelope`
+suspend/resume path). They compose: confinement bounds *where* a tool can act;
+approval bounds *whether* it acts at all.

data/docs/SECURITY.md

@@ -0,0 +1,375 @@
+---
+title: Security
+parent: Ship it
+nav_order: 1
+permalink: /security/
+---
+
+# Security
+
+An agent runtime runs untrusted input through a model that can call tools and
+touch the outside world. Insika treats that as the core problem, not an add-on.
+Every control below is **built into the engine**, **configured as data** (not
+hand-rolled per agent), and composes with the others. This page is the map;
+each section links to the deeper guide.
+
+The layers, from the edge inward:
+
+0. **[The Bearer gate](#the-bearer-gate)** — nothing but the health probe answers without a token.
+1. **[Edge limits](#edge-limits)** — stop a flood before it costs anything.
+2. **[Input guardrails](#guardrails)** — refuse injection/abuse without a model turn.
+3. **[Human approval](#human-approval)** — gate high-risk tool calls on an operator.
+4. **[Egress guard](#egress-the-ssrf-boundary)** — bound where a tool can reach.
+5. **[Sandbox](#sandbox-confined-execution)** — bound where code can run.
+6. **[Output guardrails](#guardrails)** — moderate and redact what streams back.
+7. **[Secrets](#secrets-live-only-in-the-environment)** — never on disk, never in the model.
+
+## The Bearer gate
+
+The `/v1` and `/a2a` surface answers only with
+`Authorization: Bearer <OPENCLAW_GATEWAY_TOKEN>` (which falls back to `ADMIN_TOKEN`).
+The check runs in the router, **before** any dispatch, against an **allowlist** of
+public routes — so a route added later is closed until someone deliberately publishes
+it. Only these answer without a token:
+
+| Route | Why |
+|---|---|
+| `GET /up` | health/readiness probe; touches no store |
+| `GET /start.md`, `/models.json`, `/docs`, `/docs/<name>.md` | the onboarding surface, opt-in via `INSIKA_ONBOARDING`: it exists to be read by a coding agent that has no credential yet |
+| `GET /.well-known/agent-card.json` | A2A discovery — the card is the advertisement |
+
+**With no token configured, the surface is not open — it is `503`.** Fail-closed by
+construction, the same posture as `/studio`, which denies login without `ADMIN_TOKEN`.
+`insika doctor` warns when neither is set.
+
+This matters most for `POST /v1/commands/<type>`, the generic Command ingress: it can
+dispatch **any** registered authoring Command (`write_agent_file`, `write_data_tool`,
+`upsert_llm_provider`, `update_settings`, `delete_agent`). Treat that token as
+operator-grade — it is not a read key, and a leak is agent takeover, not just usage.
+
+### Multi-tenant mode (`INSIKA_TENANCY=multi_tenant`)
+
+A single operator-grade token cannot host N stores. With `INSIKA_TENANCY=multi_tenant`
+the Bearer is resolved to a **principal** before the routes:
+
+- **Per-tenant tokens** (`POST /v1/commands/issue_tenant_token`) scope a caller to
+  one tenant; **operator** tokens (and the legacy gateway token — an existing
+  deployment switching modes keeps its credential) have the run of the deployment.
+- Tokens are stored **only as SHA-256 hashes**; the plaintext is shown exactly once,
+  at issue time. Rotation (`rotate_tenant_token`) and revocation (`revoke_token`)
+  are operator commands — revoking one tenant's token never touches another's.
+- A tenant principal reaches only its **own runtime surfaces** (`/v1/sessions`,
+  `/v1/messages`, `/v1/responses`, workflow runs, and its own session/task/event
+  reads). Every authoring/provisioning/config surface answers `403` to a tenant.
+- Isolation is the key, not a convention: a tenant's sessions live under
+  `<tenant>:<session-id>`, its commands carry `meta.tenant` (memory scoping,
+  event tagging), and reading another tenant's session/task reads as `404`.
+
+`single_tenant` (the default) is exactly the classic behavior above — one
+operator credential, no principal, no stamping.
+
+## The `/v1` contract is versioned by date
+
+Every `/v1` route reads an optional `Insika-Version: YYYY-MM-DD` header, checked
+before the Bearer gate above. Absent header means today's (only) behaviour; an
+unknown value is a `400`, not a silent fallback — a caller that pins a version
+finds out immediately that it does not exist, rather than being served whatever
+happens to be current. `/a2a` and `/channels/<id>/…` are versioned by their own
+contracts (JSON-RPC, the platform's own shape) and never read this header.
+
+## Channels authenticate themselves
+
+`/channels/<id>/…` is the one route family that does **not** answer to the gateway
+token — and it is not an exception to the rule above, it is the same rule with a
+different credential. A messaging platform has no way to send your gateway token,
+and neither has a visitor's browser; what they can send is their own scheme (a
+shared secret for the [relay](CHANNELS.md), an origin for the widget, an HMAC
+signature for a channel you write yourself). So the channel does the check, and the router refuses before
+parsing anything:
+
+| The channel says | The route answers |
+|---|---|
+| `:ok` | the turn is dispatched |
+| `:unauthorized` | `401` |
+| `:disabled` — no credential configured | `503` |
+| the channel has no `authenticate` at all | `503` |
+
+There is no path to an open channel route. A relay with no `INSIKA_RELAY_TOKEN` is
+not mounted at all (`404`); one that is mounted always has a secret. That is
+deliberate: a public inbound route with an LLM behind it is a money faucet, and
+[edge limits](#edge-limits) are the second line, not the first.
+
+The routes are enumerated in the router, not prefix-matched, so a channel route
+added tomorrow is gated by default and publishing it is a deliberate edit.
+
+Two more things a relay operator owns:
+
+- **The callback URL is egress.** The delivery POST goes through the
+  [egress guard](#egress-the-ssrf-boundary) on every call, not once at boot — a
+  hostname that resolved publicly yesterday can resolve to `169.254.169.254`
+  today, and that POST carries a customer's conversation.
+- **`event_id` is a safety property, not an optimization.** Without it a retried
+  webhook is a second turn you pay for and a second message the customer reads.
+
+### A public channel: the web widget
+
+The [widget](CHANNELS.md#the-web-widget) is different from every other surface here
+in one way that changes the whole posture: **the caller is an anonymous browser, so
+there is no secret to check.** Three controls stand in for the missing credential,
+and it is worth being precise about which of them is actually load-bearing.
+
+- **The rate limit is the real defense, and it is mandatory.** The widget answers
+  `503` until a chat rate limit exists — the platform's `edge.chat_rate_limit` or a
+  per-agent `limits.chat_rate_limit` on every published agent. This is the only
+  place in the engine that refuses to serve rather than warn, because the failure
+  mode is a bill rather than an error. The bucket is the minted session id, and it
+  is checked *before* the input guardrail so a flood cannot even spend the
+  moderator. See [edge limits](#edge-limits).
+- **The agent allowlist is a real boundary.** `INSIKA_WIDGET_AGENTS` is what an
+  anonymous visitor may address. Editing `data-agent` in devtools to name an
+  internal agent gets a `422`, not that agent.
+- **The origin allowlist is a browser courtesy, not a control.** `Access-Control-
+  Allow-Origin` is enforced by the browser, and curl sends whatever origin it
+  likes. It stops another *site* from embedding your widget; it does not stop a
+  script. Configure it (exact match, no wildcards — `https://shop.example` does not
+  admit `https://a.shop.example`) and then do not count on it.
+
+Two more properties worth knowing:
+
+- **The engine issues session ids; the client never proposes one.** `POST
+  /channels/web/messages` with an unminted id is a `404`. Create-on-write on an
+  anonymous endpoint means anyone who guesses an id joins someone else's
+  conversation, and the ids are 128 random bits for the same reason. A session also
+  belongs to exactly one channel — a widget visitor cannot stream a relay
+  customer's conversation by pasting its id.
+- **What the visitor types is data, at the most cuttable priority.** Untrusted
+  input from a public channel enters at `REQUEST` (40) like any other turn content,
+  and the [guardrails](#guardrails) run before the model. A channel may refuse a
+  request; it can never widen what the agent is allowed to do.
+
+## Edge limits
+
+The next gate. The edge limiter wraps a turn **before** the input guardrail,
+so a flood cannot even spend an LLM moderator call. Two independent, opt-in
+ceilings (nil/0 = off):
+
+- **`chat_rate_limit`** — turn attempts per session per `chat_rate_window`.
+  Counted on entry (blocked attempts still count). Keyed by session id, so id
+  rotation defeats the per-session limit — the per-agent token ceiling is the
+  backstop.
+- **`agent_token_ceiling`** — total tokens per agent per `agent_token_window`.
+  Checked on entry against a ledger, recorded after the turn. Advisory under
+  concurrency (overshoot ≈ in-flight turns).
+
+On breach: a graceful halt returning a configurable `limit_response` with **zero
+LLM calls**. A *resumed* turn is never re-counted.
+
+> ⚠️ Windows live at the platform level; the token window defaults to **86400
+> (daily)**. "500k tokens per hour" means `agent_token_ceiling = 500000` **and**
+> `agent_token_window = 3600`. A per-agent ceiling that is *present but nil* reads
+> as OFF for that agent — leave the key **absent** to inherit the platform value,
+> `0` to explicitly disable. A malformed value in the Studio raises a validation
+> error rather than silently disabling a production limit.
+
+See [Agents §Layer 4](AGENTS.md#layer-4-edge-limits-flood-and-spend-control).
+
+## Guardrails
+
+Content safety runs on both sides of a turn, **opt-in per agent**. An agent that
+configures nothing gets a conservative default: deterministic detectors on, LLM
+moderator off. See [`examples/guardrails/`](https://github.com/guizaols/insika/tree/main/examples/guardrails/).
+
+- **Input** — deterministic detectors (prompt-injection, abuse) run *before* the
+  model. A flagged input gets a **safe refusal without burning a model turn** — an
+  injection or a flood never reaches the provider. An LLM moderator can be layered
+  on top. The moderator is **fail-open**: an error or an unparseable reply never
+  blocks a legitimate customer — but silence is not a negative. That third state
+  surfaces as a `:guardrail_flagged` event with category `moderator_unavailable`,
+  so a degraded tier is distinguishable from a healthy one in the audit stream.
+- **Output** — moderation plus PII/secret redaction on the streamed response, and
+  a post-turn validator.
+
+```jsonc
+{
+  "input":  true,
+  "output": true,
+  "moderator": "provider/model",     // optional LLM moderator; omit for detectors only
+  "strictness": "low | medium | high",
+  "responses": { "injection": "safe reply…", "default": "…" }
+}
+```
+
+Strictness selects the detector categories (`low` = injection only; `medium`
+(default) and `high` add sexual and abuse). Safe-reply lookup falls back per
+category: the agent's category reply → the agent's default → the builtin
+category → the builtin default. All of it is editable in the Studio Configuration
+form. See [Agents §Layer 3](AGENTS.md#layer-3-guardrails-content-safety).
+
+## Human approval
+
+Some tool calls should not happen unattended. Mark them with
+`approvals_required: [tool names]`: the approval policy *tags* those tools, and
+when the model tries to call one, the turn **suspends** and waits for an operator
+to approve or reject it in the Studio. Approval and confinement are independent
+and compose — approval bounds *whether* a tool acts; the sandbox bounds *where* it
+can act.
+
+The wait is bounded by `approval_timeout` (default ~1h), and a turn that is
+waiting on approval is *not* killed by the ordinary turn timeout. The suspend/
+resume path is durable: an approval that arrives after a restart still resumes the
+turn from its checkpoint (see
+[Architecture](ARCHITECTURE.md#durability-checkpoints-and-resume)).
+
+## Egress: the SSRF boundary
+
+Every outbound HTTP call from a data tool passes through the **EgressGuard**, a
+Server-Side Request Forgery defense. The default posture is **strict: public
+`https` only** — private and loopback addresses and plain `http` are refused
+unless explicitly opted in.
+
+| Env | Effect |
+|-----|--------|
+| `INSIKA_EGRESS_HOSTS` | host allowlist (CSV) — the safe, specific way to permit a backend |
+| `INSIKA_EGRESS_ALLOW_HTTP=1` | permit plain `http` — **loopback dev only** |
+| `INSIKA_EGRESS_ALLOW_PRIVATE=1` | permit private/loopback IPs — **dev only** |
+
+Restricting `INSIKA_EGRESS_HOSTS` to exactly the hosts a tool needs is
+defense-in-depth: without it, `ALLOW_PRIVATE` opens *any* private destination.
+**Never set the `ALLOW_*` vars in a cloud deployment** — a public backend over
+`https` already passes the strict default.
+
+> ⚠️ A blocked egress **fails silently** — the tool returns an error to the model,
+> the request never leaves the process, and the conversation *looks* fine. Verify
+> tool health by the Studio session **trace** (a healthy call shows the backend's
+> `200`), never by the model's reply. Full detail in
+> [Tools §Egress](TOOLS.md#egress-the-ssrf-guard-and-its-silent-failure).
+
+## Sandbox: confined execution
+
+Code tools that touch the filesystem or run commands do so through a **sandbox** —
+a single, pluggable primitive chosen by config, following the principle of the
+*narrowest sandbox that supports the task*:
+
+- **Filesystem confinement is always on, host-side.** Every path is proven to
+  live inside one root *before any IO happens* — `..` escapes, absolute paths
+  outside the root, and symlinks pointing out are all refused. An escape returns a
+  structured error, never a crashed turn.
+- **Command execution is via a swappable provider.** `local` (in-process, the
+  default — cheap, but *not* an isolation boundary for a shell) or `docker` (a
+  throwaway container with `--network none`, memory/cpu caps, and a minimal image).
+  Untrusted execution should use `docker`.
+
+Both providers enforce a hard-kill wall-clock timeout that also reaps child
+processes. The provider is **data on the agent profile** (a `sandbox` block), not
+a branch in tool code. Full detail — including the config shape — in
+[Sandbox](SANDBOX.md).
+
+## Secrets live only in the environment
+
+Secrets never live on disk in agent or tool definitions, and are never exposed to
+the model:
+
+- A data tool references a secret with `{{secret.*}}`, allowed **only** inside a
+  header named in `secret_headers`; the real value is injected at provision time
+  and stored masked. A stray `{{secret.*}}` anywhere else is rejected. See
+  [Tools](TOOLS.md#data-tools-a-tool-is-a-row).
+- Full tool results are stored **masked** in the trace store; the copy persisted
+  into the transcript is capped.
+- Provider keys and the API bearer token come from the environment (see
+  [Deploy](DEPLOY.md)). Rotating the API bearer requires updating both the runtime
+  and every consumer in the same step.
+
+## Grading sends text to a judge (evals)
+
+A rubric is scored by a model, so running an eval sends that case's **user turns and
+the assistant reply** to every judge in the panel ([Evals](EVALS.md)). Two
+consequences worth stating out loud:
+
+- **The judges are a second provider surface.** Configure them deliberately: a panel
+  of three models is three vendors seeing those conversations. Judges are opt-in and
+  empty by default — with none configured, only the deterministic assertions run and
+  nothing leaves the deployment.
+- **A case is curated text, not live traffic.** The corpus is authored from real
+  conversations with PII removed at curation time. That masking is a human step, not
+  an automatic one: treat a golden case as something that WILL be read by an external
+  model and reviewed in a pull request.
+
+## Reading traffic back (refinement)
+
+A refinement run reads an agent's own transcripts and tool traces to report what
+broke ([Refinement](REFINEMENT.md)). Two properties keep that from becoming a
+second copy of your customers' data:
+
+- **It quotes as little as possible, redacted.** Only the `repetition` finding
+  carries customer words, and every snippet goes through the same detectors that
+  redact a customer-facing turn (formatted CPF/CNPJ, API secrets). Tool arguments
+  and results are never copied — only the normalized error signature is. That is
+  not a general PII scrubber: a phone number typed into a chat can survive into a
+  snippet, so the page sits behind the Studio login like the transcripts do.
+- **Provenance is ids.** A run record stores session ids, never their contents, and
+  the events it emits carry counts only.
+
+### Editing an agent from its traffic
+
+A run can also propose a change to the agent's instructions, and that path is opt-in
+per agent (`refinement.mode`), off by default, and bounded by construction rather
+than by instruction:
+
+| Surface | Reachable? | Why |
+|---|---|---|
+| the files listed in `refinement.files` | **yes** | text you already edit by hand, versioned, one-click restore |
+| skill bodies and descriptions | **yes** | same trust level, same history |
+| guardrails, the safety corpus | **no** | a constrained thing does not edit its own constraints |
+| tool definitions and schemas | **no** | tools are authored by a person; a wrong schema is theirs to fix |
+| policies, approvals, denied tools | **no** | authorization is not a prompt concern |
+| model pins, limits, edge config | **no** | cost and latency are the operator's decisions |
+| the system preamble the engine assembles | **no** | the fixed frame of a turn |
+
+There is no code path to the "no" rows — not a rule in a prompt. Three more
+properties are worth stating because each one is a way this could have gone wrong:
+
+- **An edit is verified by running it, not by asking a model.** The candidate is
+  applied to a throwaway clone of the agent and the golden set is replayed against
+  it; any regression disqualifies it. An agent with no cases, with no recorded
+  baseline, or with a baseline in which nothing passes, **cannot be edited at all** —
+  the gate refuses instead of passing vacuously. That last case is the subtle one: a
+  regression is measured against a case that was passing, so an all-red baseline
+  cannot produce one and would wave everything through.
+- **A human approves.** A gate pass parks the proposal for review; nothing applies
+  itself. Approving writes through the versioned file store, so undo is the Restore
+  button that was already there.
+- **Prompt injection buys nothing.** Evidence reaches a proposer as quoted, masked
+  data, and whatever comes back is validated against the candidate schema and the
+  file allowlist. The worst an injected instruction can achieve is a proposal that
+  gets dropped or fails the gate.
+- **The proposer reads the allowlisted files and nothing else.** Those it gets
+  verbatim and unmasked, which sends a model nothing it was not already sent on every
+  turn — they are the agent's own instructions. Masking them would break anchoring
+  (a `before` copied from a masked view never matches the real file) and protect
+  nothing. Files outside the allowlist are not shown at all.
+
+**Know what the gate does not measure.** It catches an edit that breaks a case you
+wrote; it cannot catch one that breaks something no case covers. Two of its blind
+spots are this engine working correctly rather than gaps — an edit cannot remove a
+tool (availability is `tools_allow`, not prose) and it cannot make a reply leak PII
+(the output guardrail redacts first) — but the general point stands, and it is
+[stated plainly in Refinement](REFINEMENT.md#what-the-gate-can-and-cannot-catch).
+The structural limits above hold regardless; the gate is the layer that has to be
+earned with cases.
+
+## Config discipline
+
+Configuration is validated against a schema of known keys at boot. An unknown key
+in the `INSIKA_` namespace (a typo the runtime would otherwise ignore) or a
+wrong-typed value is **surfaced**, and `INSIKA_CONFIG_STRICT=1` turns findings
+into a boot refusal. By default the engine warns and boots on last-known-good — a
+rotated key or a typo never takes the whole service down. The `insika doctor`
+command runs the same checks on demand against a live database. See
+[Deploy](DEPLOY.md#strict-config-and-insika-doctor).
+
+## See also
+
+- [Agents](AGENTS.md) — the five access layers per agent.
+- [Tools](TOOLS.md) — egress and secret placeholders in depth.
+- [Sandbox](SANDBOX.md) — the confinement primitive.
+- [Deploy](DEPLOY.md) — tokens, rotation, and strict config.