insika 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5c2b2fbe064af45aa132f93de00954b24dc504e052db2e059b4f6daa151e248b
-  data.tar.gz: 50cabcc10d82eabdfd08de562bd61feddf1a9de68ee6427343c80c5cf6ad9d8c
+  metadata.gz: ec89eb1dae94e5a57733d0dd88a19d92ff98eb7e2ae0b907792e2a4ce8d508aa
+  data.tar.gz: cb882c8015e00c0c905e5945a05a5a465c938dedda762914c3737a64a9f0d4fb
 SHA512:
-  metadata.gz: 3004e30a234349bcb91db185fcc31f9b52cbdeb7660c613b1fc229adf6bbbe717d9abc8ebcdc5bf9ecc125986694c0d2f3a82e1d48eb83105ff74db0467e4d91
-  data.tar.gz: ca4bec6a8b8cd03ee1a5c2df627e7da778d3dd157f30f848c70f8077ac467570284bf934e3a4c8040fb223503855d9cdcad111b9139ed093c702b6a8850ea1ff
+  metadata.gz: e2acb07f2ae896ee762cd3f1113dca3d462f80ea0bb32caea69f5db3fd6fbd31821cf9a5140b32c64c26b702b29b78aaf9fec0ca8f59653178dcb37afe9eca38
+  data.tar.gz: f8c98a2b76c802a75e18a77e0762ef42c650aa956afb3f131e1e9ccacfb2a56de87540b7389c740021008b45a174fa371d7a6dbb0a5a7cf83f248ce41b696770

data/CHANGELOG.md

@@ -8,6 +8,197 @@ it is released. Entries land with the pull request that makes the change.
 
 ## [Unreleased]
 
+## [0.9.0] - 2026-09-07
+
+The commerce-agent wave: the engine now enforces what the prompt used to ask for —
+a write names only IDs a lookup returned, the model picks which cards to show and
+the engine shows them, third-party text can be fenced before the model reads it,
+and an eval case can start from a seeded snapshot. Additive on the wire: `/v1`
+gains frames and keys, none change meaning.
+
+### Added
+
+- **Write provenance** — `requires_evidence` on HTTP data tools checks declared
+  ID parameters against the session evidence ledger before approval or execution.
+  Unknown IDs and an unavailable ledger return `blocked` with gate `provenance`.
+  Studio editing, Doctor diagnostics, session traces, `tool_blocked`, the
+  `insika.tool.blocked` counter and `insika tools:report` expose the gate.
+  See [Tools](docs/TOOLS.md#provenance-checking-ids-before-a-write).
+- **Snapshot evals** — cases can seed evidence IDs, memory, user/assistant history
+  and briefing before the first turn. The authenticated seed endpoint is disabled
+  by default (`evals.seeding`); Doctor warns when enabled. New assertions cover
+  forbidden/alternative/first calls, call limits, reply text and blocked gates.
+  Responses SSE now carries tool arguments, completion status and the provider's
+  `call_id` on both item frames, so a consumer pairs `added` with `done`.
+  See [Evals](docs/EVALS.md#state--a-case-starts-from-a-snapshot).
+- **Presentation tools** — data tools with `presentation` instead of `request`
+  select evidence cards by known ID — this turn's, or the last 64 the session
+  ledger kept (snapshots seed them as `evidence.cards`). They report shown/dropped IDs,
+  emit `insika.ui` (web channel: `ui`) and select the outbox attachments. Evidence
+  cards carry IDs; empty selections suppress automatic card delivery. Includes
+  `ui_components` / `no_ui` eval assertions and Doctor diagnostics.
+  See [Tools](docs/TOOLS.md#presentation-tools-the-model-picks-ids-the-engine-shows-the-cards).
+- **Fencing** — opt-in per agent through the DSL, payload or Studio. Sanitizes
+  ordinary tool result strings, memory text and injected knowledge summaries, and adds a fixed
+  identity-layer notice. The tool-string cap `fencing.max_chars` defaults to 12,000; the agent flag
+  defaults to off — and stays off in this release: flipping the default waits for the
+  golden-case comparison with fencing on ([Releasing](docs/RELEASING.md), gate 4).
+  Doctor warns for unfenced agents exposed by relay/widget env configuration.
+  See [Agents](docs/AGENTS.md#fencing--third-party-text-is-data-never-instructions).
+
+### Changed
+
+- **Cache boundary** — explicit Anthropic cache control ends at the identity layer;
+  volatile system text follows in a separate block. Prefix fingerprints cover
+  identity and tool schemas; memory/knowledge/request changes retain separate
+  diagnostic digests without reporting identity-prefix invalidation.
+  See [Context](docs/CONTEXT.md#the-provider-prefix-cache).
+- **Serial writes in parallel batches** — tools marked `side_effect` acquire a
+  per-session serial gate before the shared concurrency slot. Independent reads
+  can overlap writes; queued writes do not consume that shared slot. MCP tools
+  annotated `readOnlyHint` are reads.
+- Memory and knowledge extraction read only nonblank user/assistant text, preserving
+  original transcript indexes. Direct tool messages are excluded; assistant
+  paraphrases can still reach extraction. This applies regardless of `fencing`.
+- A tool's own `{"status":"blocked"}` answer is a call that ran: only the engine's
+  gate refusal reports `blocked` on `:tool_result`, the trace and `tools:report`.
+- A Studio save that changes a tool's HTTP method re-derives `side_effect`.
+
+### Fixed
+
+- Data tools declaring `evidence` no longer fail when the envelope parses an
+  already-parsed `Evidence::Spec`.
+- Studio tool saves preserve `evidence` and `presentation` declarations.
+- `scripts/serve_real.rb` passes channel and settings stores to the edge, enabling
+  configured relay/widget routes and the eval seeding setting in the demo server.
+- Fencing: the notice ships with the flag regardless of the `context_providers`
+  allowlist; briefing values and the compaction summary are sanitized too; evidence
+  item ids stay byte-exact for the model (only the line is sanitized).
+- Provenance: a schema-optional parameter the model omits is not checked; a blocked
+  call on a one-shot turn no longer fails emitting `tool_blocked`.
+- `POST /v1/conversations/:id/seed` decodes a percent-encoded conversation id.
+- Widget `ui` cards link only `http(s)` urls.
+- The evidence ledger reads the session row once per turn, not once per gated call.
+- Studio reads its templates as UTF-8 regardless of the process locale: a host with
+  no `LANG` answered 500 on the login page (found by the release install proof).
+
+
+## [0.8.0] - 2026-08-31
+
+The post-consolidation hardening wave: the last two OpenClaw-era names are gone
+from the wire (**breaking** — `insika:<agent>` / `X-Insika-Agent` and
+`INSIKA_GATEWAY_TOKEN`, no aliases), and the engine learned to manage its own
+long conversations — in-session compaction (RFC-0044), a tool budget that warns
+before it kills, goal recitation at the tail of the context, and a tool-usage
+audit report. Update `/v1` consumers and the gateway-token variable together
+with this upgrade.
+
+### Added
+
+- **In-session compaction (RFC-0044).** When a session's uncompacted
+  transcript grows past `compact_after` messages (default 40), everything but
+  the last `keep_last` (default 20) is summarized by a cheap model
+  (`compaction.model` → platform `utility_model`) into one
+  `<conversation_summary>` history fragment; the tail stays verbatim and the
+  boundary is stable between compactions, so the prompt cache holds after it.
+  Runs post-turn, off the critical path, best-effort; persisted on the session
+  record with a monotonic boundary. Opt-in via the long-reserved Settings
+  `compaction` hash (the Studio general form is back), `insika doctor` warns
+  when it is enabled with no model, and every compaction is observable:
+  `:context_compacted` event, `insika.context.compacted` counter, `{upto,
+  runs}` + its own category in the context trace. See
+  [Context](docs/CONTEXT.md).
+
+- **`insika tools:report` — the tool audit.** The per-session trace never
+  aggregated, so "which tools does this agent carry and never use?" had no
+  answer. The new CLI report reads the stored traces per agent (tasks →
+  sessions → `tool_traces`) and flags allowlisted-but-never-called tools,
+  tools over 30% errors in the window, and tools not called in N days
+  (default 14). Read-only — the operator removes. `--agent`, `--days`,
+  `--json`. See [Tools](docs/TOOLS.md).
+
+- **Two series promoted to OTEL metrics.** `insika.cache.hit_rate` (histogram,
+  `%` — cache reads over the billed prompt, per turn, same arithmetic as the
+  Studio's per-agent series) and `insika.tool.loop_intervened` (counter — the
+  loop detector's one-shot warning, labelled by tool). Both computed in the
+  telemetry bridge from events that already existed: zero cost when OTEL is
+  off, no engine change. See [Observability](docs/OBSERVABILITY.md).
+
+- **The doctor warns on a prompt file that outgrew a prompt.** The
+  `prompt-files` sweep now WARNs (never errors) past ~6 000 estimated tokens
+  or 600 lines, with the actual counts — the LLM-generated merchant-pack
+  shape that costs 20%+ extra tokens per turn. Corrupted-wrapper detection
+  is unchanged.
+
+- **The tool budget is announced before it kills.** `max_tool_calls` was
+  enforced but silent: the model met the ceiling only when the turn died with
+  `stage: :tool_limit`, having delivered nothing. At **10 / 5 / 2** calls
+  remaining the engine now appends a short, escalating notice at the next
+  tool-batch boundary and emits `:tool_budget_warned`. It is a `user` message,
+  never a system one, so the cacheable system prefix stays byte-stable.
+  Verified live (DeepSeek): a turn asking for 30 lookups on a budget of 20 used
+  to die at `tool_limit`; it now stops at 18 and answers with what it has. The
+  turn's counter, the notices and the abort are one object (`TurnBudget`), and
+  the batch-boundary arithmetic it shares with the loop detector moved into
+  `ToolBatch`.
+
+### Changed
+
+- **The briefing recites the goal at the TAIL of the context.** `<briefing>`
+  used to carry known facts, the still-missing list and the next step in one
+  `:system` block at the top. The goal half was **moved** (not copied) into a
+  new `<recitation>` block rendered after the whole history, as the last thing
+  the model reads before the current message — attention is strongest at the
+  end, and a goal stated only at the top is the first thing a 30-call turn
+  forgets. New `:tail` fragment placement; the head keeps the durable facts.
+
+- **The `/v1/responses` wire names are insika-native**: the agent arrives as
+  `model: "insika:<agent>"` and the fallback header is `X-Insika-Agent`. Hard
+  rename, no alias — a consumer still sending `openclaw:<agent>` gets
+  "model/agent missing". Update every `/v1` consumer together with this upgrade.
+- **`INSIKA_GATEWAY_TOKEN` replaces `OPENCLAW_GATEWAY_TOKEN`** as the Bearer for
+  `/v1` + `/a2a`. Hard rename, no alias: the old name was a leftover of the
+  OpenClaw migration and is now a foreign var the engine ignores. Update the
+  variable on every deployment before upgrading. The dead `OPENCLAW_AGENTS_DIR`
+  env spec (nothing read it) is gone.
+
+### Fixed
+
+- **The Studio home charts went blank for three hours every day.** The activity
+  buckets key on calendar parts (date, hour) of `updated_at`, which the engine
+  writes in UTC, but `render_home` read `Time.now` in the host's local zone. On a
+  UTC-3 host, from 21:00 local onward "today" was already tomorrow in UTC: the
+  14-day chart matched no session and the 24h sparkline's floor was set three
+  hours in the future, dropping the newest buckets. Stamps are now read through
+  one `utc_time` helper and `now` is UTC. Instant comparisons (`active_now`)
+  never had the bug — only the calendar arithmetic did.
+- **The Conversations trend showed "−1" for the day's first conversation.** The
+  14-day series is oldest-first and ends at today, so its last pair reads
+  `[yesterday, today]`; it was destructured the other way round, inverting the
+  sign on every conversations delta. The Messages card was always correct.
+- **Declaring a tool allow/deny list now opts the profile into the policy.** The
+  lists are applied by exactly one policy — the builtin `tool_allowlist` — and
+  the policy engine runs only the policies a profile *names*. So an agent with
+  `tools_allow: ["a", "b"]` and no `policies` sent **every registered tool** to
+  the model, silently: 508 tools / 52 405 tokens of schemas per request against
+  an allowlist of 2, observed on a real run. `tools_deny` was ignored the same
+  way. `AgentProfile.build` now appends `tool_allowlist` whenever `tools_allow`,
+  `tools_deny` or `tools_allow_groups` is declared — presence, not emptiness, so
+  `tools_allow: []` still means "no tools". A stored record left in the old
+  state is repaired on read but stays wrong on disk, so `doctor` gained a
+  `tool-allowlist` **error** naming each agent to re-save.
+- **A DSL agent could write its briefing and never read it back.** The
+  `Insika.agent`/`Insika.system` runtime wired every context provider except
+  `Briefing`, so `update_briefing` / `set_next_step` persisted state that never
+  reached a prompt. Now wired, before `Session`, like `config/wiring.rb`.
+- **Docker image shipped without `docs/`, so `GET /start.md` was a 500 with
+  `INSIKA_ONBOARDING=1`.** `.dockerignore` excluded the whole tree; the onboarding
+  surface reads `docs/onboarding/start.md`, `docs/prompts/` and the public
+  `docs/*.md` from disk at request time. Only the Jekyll build output and the
+  internal notes are excluded now, and a spec asserts `.dockerignore` never drops
+  a file `Insika::Onboarding` serves. Found on the first Railway deploy with
+  onboarding on; the gem was never affected (its payload keeps `docs/`).
+
 ## [0.7.0] - 2026-08-29
 
 The pre-1.0 consolidation: MCP servers as first-class tools, a template gallery,

data/README.md

@@ -18,7 +18,9 @@ deployment.
 - **Drop-in `/v1/responses`** — SSE streaming and usage, the shape existing clients already speak.
 - **Durable & resumable** — every turn checkpoints; a crash resumes without repeating side-effects.
 - **Agents as data** — agents, prompts, skills and tools are editable at runtime (UI or API), no redeploy.
-- **Tools & skills** — code tools, data-defined tools, MCP import; skills load on demand.
+- **Tools & skills** — code, HTTP data and live MCP tools; skills load on demand.
+- **Evidence-backed actions** — ID checks before writes, serial side effects per session, and selected evidence cards ([Tools](docs/TOOLS.md)).
+- **Snapshot evals** — seed a case's starting state and check calls, refusals, replies and cards ([Evals](docs/EVALS.md)).
 - **Safe by default** — content-safety guardrails, an egress guard, confined execution, approvals, edge limits.
 - **Observable** — an event stream, per-session tool-call traces, optional OpenTelemetry.
 - **~0.4 ms of engine per turn** — p50 overhead on a neutral, key-free benchmark you can rerun yourself ([methodology](docs/BENCHMARK.md)).
@@ -106,7 +108,7 @@ navigates by.
 
 - [Agents](docs/AGENTS.md) — the AgentProfile and every key on it; create and edit at runtime.
 - [Limits and policy](docs/POLICY.md) — the five layers: allowed tools, approvals, guardrails, edge limits, reasoning.
-- [Tools](docs/TOOLS.md) — code vs data vs MCP tools, manifests, egress troubleshooting.
+- [Tools](docs/TOOLS.md) — code, HTTP, MCP and presentation tools; evidence gates and serial writes.
 - [Skills](docs/SKILLS.md) — the SKILL.md format and progressive loading.
 - [Context](docs/CONTEXT.md) — what fills a turn's prompt; budget, eviction, memory.
 - [Workflows](docs/WORKFLOWS.md) — deterministic orchestration of several agents: the five patterns, and when to let the model choose instead.
@@ -140,10 +142,10 @@ navigates by.
 
 **Improve**
 
-- [Evals](docs/EVALS.md) — the cases that grade an agent: rubrics, the judge panel, and the pre-merge gate.
+- [Evals](docs/EVALS.md) — snapshot cases, tool/UI assertions, rubrics, the judge panel and the pre-merge gate.
 - [Refinement](docs/REFINEMENT.md) — read an agent's own traffic back as a ranked report of what broke.
 - [Outcomes and follow-ups](docs/OUTCOMES.md) — what the traffic was worth in business terms, and the tool that comes back on a promise.
-- [Knowledge](docs/KNOWLEDGE.md) — extract durable concepts from finished conversations into a per-agent knowledge base; provenance-stamped, PII-redacted, best-effort with a re-scan recovery path (retrieval and consolidation are not shipped yet).
+- [Knowledge](docs/KNOWLEDGE.md) — extract durable concepts from finished conversations into a per-agent knowledge base; provenance-stamped, PII-redacted, best-effort with a re-scan recovery path.
 - [Facts](docs/FACTS.md) — distill finished customer conversations into proposed facts; a human approves them into memory (nothing is ever applied automatically).
 - [Harvest](docs/HARVEST.md) — mine real traffic for SKILL proposals; the negative list, the evidence-ledger grounding filter and the double gate hold every proposal to a human approval and the append-only log (nothing is ever applied automatically).
 
@@ -176,5 +178,6 @@ and [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). Vulnerabilities go through
 
 ## Status
 
-Pre-release — APIs may still change, nothing is tagged yet
-([CHANGELOG.md](CHANGELOG.md)). Licensed **MIT** ([LICENSE](LICENSE)).
+Pre-1.0 — published on [RubyGems](https://rubygems.org/gems/insika); APIs may
+still change between minor versions ([CHANGELOG.md](CHANGELOG.md)). Licensed
+**MIT** ([LICENSE](LICENSE)).

data/bin/insika

@@ -32,6 +32,7 @@ module Insika
       when "harvest" then harvest(argv)
       when "harvest:negative" then harvest_negative_import(argv)
       when "harvest:criterion" then harvest_criterion_check(argv)
+      when "tools:report" then tools_report(argv)
       when "knowledge:backfill" then knowledge_backfill(argv)
       when "knowledge:export" then knowledge_export(argv)
       when "soak"    then exit(Insika::Soak::Runner.main(argv, stdout: $stdout, stderr: $stderr))
@@ -65,6 +66,11 @@ module Insika
                                     seed the profile's negative list from a rules file
           harvest:criterion check --file F
                                     strict-load the frozen conversion criterion
+          tools:report [--agent ID] [--days N] [--json]
+                                    tool usage per agent over the stored traces:
+                                    allowlisted-but-never-called, error rate > 30%
+                                    in the window, and stale (not called in N days).
+                                    Read-only — the operator removes
           knowledge:backfill --agent ID [--since ISO8601] [--json]
                                     the recovery path for post-turn extraction: replays
                                     the agent's stored sessions through the same
@@ -337,7 +343,7 @@ def evals_simulate(argv)
   require "fileutils"
   require "time"
   opts = { base_url: ENV["INSIKA_URL"] || "http://localhost:9292",
-           token: ENV["OPENCLAW_GATEWAY_TOKEN"] || ENV["ADMIN_TOKEN"] || "local-demo",
+           token: ENV["INSIKA_GATEWAY_TOKEN"] || ENV["ADMIN_TOKEN"] || "local-demo",
            timeout: 120, persona_model: ENV["EVAL_PERSONA_MODEL"],
            judge_model: ENV["EVAL_JUDGE_MODEL"], judge_provider: ENV["EVAL_JUDGE_PROVIDER"] }
   OptionParser.new do |o|
@@ -345,7 +351,7 @@ def evals_simulate(argv)
     o.on("--persona FILE", "the persona YAML — a case with a `persona:` key (required)") { |v| opts[:persona] = v }
     o.on("--target T", "agent id (over --base-url) or an A2A http(s) URL (required)") { |v| opts[:target] = v }
     o.on("--base-url URL", "deployment base URL (default #{opts[:base_url]})") { |v| opts[:base_url] = v }
-    o.on("--token T", "gateway token (default OPENCLAW_GATEWAY_TOKEN/local-demo)") { |v| opts[:token] = v }
+    o.on("--token T", "gateway token (default INSIKA_GATEWAY_TOKEN/local-demo)") { |v| opts[:token] = v }
     o.on("--timeout N", Integer, "per-turn read timeout, and the A2A poll budget in seconds (default 120)") { |v| opts[:timeout] = v }
     o.on("--staging", "the target is a staging deployment (side effects acceptable)") { opts[:staging] = true }
     o.on("--eval-profile", "the target runs an eval profile (side-effect tools swapped for fakes)") { opts[:eval_profile] = true }
@@ -384,7 +390,7 @@ def evals_simulate(argv)
   # paired comparison, which it silently invalidates.
   require "securerandom"
   conv = opts[:conv] || "sim-#{golden.id}-#{SecureRandom.hex(4)}"
-  run = simulator.run(persona: golden.persona, agent: target_agent, conv: conv)
+  run = simulator.run(persona: golden.persona, agent: target_agent, conv: conv, state: golden.state)
 
   verdict = judge && judge.score_conversation(
     rubric: golden.rubric, transcript: run.transcript, policy: golden.policy,
@@ -1162,6 +1168,41 @@ end
       "last #{n} session(s)"
     end
 
+    # insika tools:report [--agent ID] [--days N] [--json]
+    #
+    # The tool audit: reads the SAME durable backend as `doctor` (tasks →
+    # sessions → tool_traces), no app boot. Read-only — it names never-called
+    # allowlisted tools, high-error tools and stale tools; the operator removes.
+    def tools_report(argv)
+      opts = { json: false, days: Insika::ToolUsageReport::WINDOW_DAYS }
+      OptionParser.new do |o|
+        o.banner = "Usage: insika tools:report [--agent ID] [--days N] [--json]"
+        o.on("--agent ID", "narrow the report to one stored agent") { |v| opts[:agent] = v }
+        o.on("--days N", Integer, "window for error-rate and staleness (default #{opts[:days]})") { |v| opts[:days] = v }
+        o.on("--json", "emit the report as JSON") { opts[:json] = true }
+      end.parse!(argv)
+
+      backend = Insika::Wiring::Graph.backend_from_env
+      config_store = Insika::ConfigStore.new(store: backend)
+      report = Insika::ToolUsageReport.new(
+        task_store: Insika::TaskStore.new(store: backend),
+        tool_trace_store: Insika::ToolTraceStore.new(store: backend),
+        profile_source: Insika::StoredProfileSource.new(config_store: config_store)
+      ).generate(days: opts[:days], agent: opts[:agent])
+
+      if opts[:agent] && report.agents.empty?
+        warn "insika tools:report: no stored agent '#{opts[:agent]}'"
+        exit 2
+      end
+
+      if opts[:json]
+        require "json"
+        puts JSON.pretty_generate(report.to_h)
+      else
+        puts report.to_s
+      end
+    end
+
     # Config stores over the real durable backend — no app boot / no DEEPSEEK.
     def build_doctor
       backend = Insika::Wiring::Graph.backend_from_env

data/docs/AGENTS.md

@@ -32,9 +32,10 @@ All three land on the **same** config-over-code import path — they differ only
 ergonomics, not in what they produce.
 
 - **DSL** — `Insika.agent("id") { … }` builds an agent definition and imports it
-  into the durable store. Best for code-defined agents and examples. The DSL
-  auto-enables the tool- and skill-allowlist policies. `model` is optional — a nil
-  model resolves the platform `default_model` at turn start.
+  into the durable store. Best for code-defined agents and examples. The DSL also
+  names the **skill**-allowlist policy for you (the tool one is implied by any
+  declared tool list, on every path). `model` is optional — a nil model resolves
+  the platform `default_model` at turn start.
 - **API** — `POST /v1/agents` with a definition (a "pack": an agent config plus its
   prompt files, skills, and data-tools). The import is **idempotent** and
   **authoritative** — what leaves the definition leaves the agent, so a
@@ -108,6 +109,14 @@ DEFAULT_LIMITS = {
 
 `build` merges your overrides over these — you set only the deltas.
 
+`max_tool_calls` is announced before it kills. At **10 / 5 / 2** calls remaining
+the engine appends a short user message at the next tool-batch boundary
+("2 tool calls left — consolidate what you already have and answer now"), so a
+long turn converges on an answer instead of dying with `stage: :tool_limit` and
+delivering nothing. It is a `user` message, never a system one: the system prefix
+stays byte-stable and `prompt_caching` keeps hitting. Each threshold fires at
+most once per turn and emits `:tool_budget_warned` on the event stream.
+
 `max_tool_repeat` is the loop guard: the same tool called with **identical
 arguments** that many times in a row gets ONE in-turn warning (a user message at
 the next tool-batch boundary: "you already ran this, answer with what you
@@ -135,9 +144,44 @@ the thing worth declaring:
 tool_persistence false   # remove the block for this agent
 ```
 
-The block is a byte-stable constant, so `prompt_caching` stays effective: the
-deploy that introduces it costs one cache write per agent, and every turn after
-that hits as before.
+The block is constant, so it does not invalidate the identity prefix between turns.
+
+### `prompt_caching` — explicit cache control
+
+Set `"prompt_caching": true` in the agent payload, or enable it in the Studio
+agent configuration. There is no dedicated DSL method for this flag.
+
+For Anthropic, this puts the cache breakpoint after the identity layer and leaves
+volatile system text below it. Other providers receive a plain system string and
+may cache automatically. See [Context](CONTEXT.md#the-provider-prefix-cache) for
+layering, fallback behavior and token accounting.
+
+### `fencing` — third-party text is data, never instructions
+
+```ruby
+fencing true
+```
+
+Off by default. When enabled, the engine sanitizes ordinary tool result string
+leaves after evidence processing, memory facts/notes, briefing values and the next
+step, the compaction summary, and the knowledge names and descriptions injected in
+the context block. It normalizes Unicode (NFKC), removes invisible format characters,
+replaces control characters except tabs/newlines/carriage returns, strips known
+engine/transcript tag shapes and defuses forged role markers. Tool result strings
+are also capped at `fencing.max_chars` (platform default 12,000); memory and knowledge
+context sanitization does not apply that cap. Tool error and halt results bypass
+this sanitizer; tool Hash keys, evidence item `id`s and attachment captions are
+unchanged (an id is a key the ledger holds byte-exact).
+`load_knowledge` returns the full body without fencing.
+
+A fixed notice beneath the identity tells the model to treat contextual material
+as data. It ships with the flag — the `context_providers` allowlist does not govern
+it. It is stable across turns; it does not guarantee prompt-injection resistance.
+
+Memory/knowledge extraction reads only nonblank user/assistant text regardless
+of the flag. Direct tool messages are excluded, but text repeated by the
+assistant can still reach extraction. `insika doctor` warns for unfenced agents
+exposed by the configured relay/widget environment settings.
 
 ### Why some limits are missing from that list
 
@@ -164,17 +208,16 @@ So the two groups read differently on purpose:
 ### `tool_concurrency` — parallel tool calls
 
 When the model asks for several tools in one step, they run **one at a time by
-default**. Raise `tool_concurrency` and they run together, capped at that number
-in flight:
+default**. Raise `tool_concurrency` to overlap independent reads, capped at that
+number in flight. Tools marked `side_effect` still execute one at a time per session:
 
 ```ruby
 limit :tool_concurrency, 4   # nil / 0 / 1 = serial (the default); N = at most N at once
 ```
 
 One number is both the switch and the cap. It pays off only when a turn issues
-several **slow, independent** calls (data tools waiting on HTTP) — the wall-clock
-becomes the slowest call instead of the sum. It buys nothing for fast in-process
-tools, and it is the *model* that decides the fan-out, which is why the cap is not
+several **slow, independent reads** (data tools waiting on HTTP). It buys nothing
+for fast in-process tools, and it is the *model* that decides the fan-out, which is why the cap is not
 optional: an uncapped batch of 15 data tools is 15 simultaneous requests to the
 same backend.
 
@@ -382,13 +425,23 @@ The same three-state rule governs tools, skills, context providers, and workflow
 For tools, a paired deny list (`tools_deny`) **always wins**, and
 `tools_allow_groups` unions a per-group allowlist on top of `tools_allow`.
 
+**Declaring a list is opting into it.** The lists are applied by one policy, the
+builtin `tool_allowlist`, and the policy engine runs only the policies a profile
+names — so declaring `tools_allow` without naming that policy used to mean the
+model got *every* registered tool. The engine now adds `tool_allowlist` for you
+the moment any of `tools_allow`, `tools_deny` or `tools_allow_groups` is
+declared, on every creation path. Presence is the trigger, not content:
+`tools_allow: []` means "no tools" and enforces just as hard. If a stored agent
+predates this, `insika doctor` names it — the engine repairs it on read, but the
+stored record stays wrong until you re-save it.
+
 Three capabilities invert the default — `nil`/absent means **OFF**, not "all":
 `subagents`, `memory`, and `guardrails` (each defaults to off or a conservative
 setting, never "everything on"). `tool_output_compression` is a fourth: opt-in
 mechanical dedupe of repeated tool results in the history (see
-[Context](CONTEXT.md#compaction-is-not-wired--except-the-mechanical-dedupe)),
-off by default because it changes what the model sees. And one flag inverts the
-other way: `tool_persistence` is **ON unless you set it to `false`** (see
+[Context](CONTEXT.md#budget-and-eviction--the-actual-compaction)),
+off by default because it changes what the model sees; `fencing` is a fifth (see
+[`fencing`](#fencing--third-party-text-is-data-never-instructions)), off by default. One flag inverts the other way: `tool_persistence` is **ON unless you set it to `false`** (see
 [`tool_persistence`](#tool_persistence--dont-give-up-on-the-first-empty-result)).
 
 ### Declaring what this deployment has
@@ -673,6 +726,13 @@ optionally shared outside the Studio via an expiring signed link
 saving agent — never a parameter the model types. Artifact content is LLM
 output and is served as untrusted. See [Artifacts](ARTIFACTS.md).
 
+## Review before enabling writes
+
+Does each ID come from the server? Declare `evidence` on the lookup tool and
+`requires_evidence` on the write tool so the engine enforces session provenance.
+Mark writes as `side_effect` so parallel batches serialize them and recovery skips
+completed calls. See [Tools](TOOLS.md#provenance-checking-ids-before-a-write).
+
 ## See also
 
 - [Tools](TOOLS.md) — define, register, and troubleshoot tools.

data/docs/API.md

@@ -21,6 +21,7 @@ two cannot drift (a spec pins the gate's version to the date below).
 | `POST /v1/agents` | provisioning by definition/pack (idempotent, authoritative) |
 | `GET/DELETE /v1/agents` | list / remove agents |
 | `POST /v1/sessions` | create a session explicitly |
+| `POST /v1/conversations/:id/seed` | load an eval snapshot before the first turn; see [Seeding](#seeding-an-eval-conversation) |
 | `POST /v1/outcomes` | record a business outcome (`conversion`, `escalation`, any label) |
 | `GET /v1/outcomes` | last outcome per agent + the per-day series |
 | `POST /v1/commands/...` | operator commands (`forget_customer`, `delete_tenant_data`, the follow-up mutations) |
@@ -30,6 +31,78 @@ two cannot drift (a spec pins the gate's version to the date below).
 The turn endpoints speak the OpenAI `response.completed` wire format; the
 `Insika-Version` header declares the compatibility vintage of a request.
 
+## Tool and presentation SSE events
+
+`POST /v1/responses` emits a tool name on `response.output_item.added`, with
+`arguments` as a JSON string when available. `response.output_item.done` reports
+how it ended. These `ok` / `error` / `blocked` statuses and optional `gate` are
+Insika extensions, not OpenAI function-call lifecycle statuses.
+
+```text
+event: response.output_item.added
+data: {"type":"response.output_item.added","item":{"type":"function_call","name":"add_to_cart","call_id":"toolu_01","arguments":"{\"product_id\":\"SKU-1\"}"}}
+
+event: response.output_item.done
+data: {"type":"response.output_item.done","item":{"type":"function_call","name":"add_to_cart","call_id":"toolu_01","status":"blocked","gate":"provenance"}}
+```
+
+`added` and `done` are two frames of **one** call, paired by `call_id` (the provider's
+tool-call id, present whenever the provider gave one). A consumer that counts tool
+calls counts `added` frames; `done` only says how each ended.
+
+The completion frame does not contain the tool result body. Use stored session
+traces for a per-call audit of what a tool returned.
+
+A presentation call also emits a namespaced event:
+
+```text
+event: insika.ui
+data: {"type":"insika.ui","component":"product_cards","title":"Selected products","items":[{"type":"card","id":"SKU-1","url":"https://shop.example/products/1","caption":"Dark chocolate"}],"count":1,"dropped":[{"id":"SKU-2","reason":"no_card"}]}
+```
+
+`count` is the number shown; `dropped` reasons are `unknown`, `no_card` or `max`.
+An empty selection still emits the event with `count: 0`. Clients that render
+cards consume this event; text-only clients can ignore it. See
+[Presentation tools](TOOLS.md#presentation-tools-the-model-picks-ids-the-engine-shows-the-cards).
+
+## Seeding an eval conversation
+
+Enable `evals.seeding` in platform settings on the eval deployment. For example,
+an operator can `POST /v1/commands/update_settings` with
+`{"patch":{"evals":{"seeding":true}}}`. Turn it off after the run.
+
+`POST /v1/conversations/:id/seed` uses the same Bearer and tenant ID namespacing
+as a turn. Its body is the snapshot itself, plus optional `customer`; do not wrap
+it in `state`:
+
+```json
+{
+  "customer": "eval-customer-1",
+  "evidence": {
+    "ids": ["SKU-1"],
+    "cards": [{ "type": "card", "id": "SKU-1", "url": "https://shop.example/1", "caption": "Dark chocolate" }]
+  },
+  "history": [
+    { "role": "assistant", "content": "The dark chocolate is SKU-1." }
+  ],
+  "memory": { "facts": { "preference": "dark chocolate" }, "notes": [] },
+  "briefing": { "fields": { "delivery_day": "Friday" } }
+}
+```
+
+It creates a missing session and returns `200` with `{ "session": … }`; seeding
+does not run a model or create a task. Disabled seeding returns `403`, a session
+with messages returns `409`, and invalid supported field shapes return `422`.
+History accepts only `user` / `assistant` roles and is stamped `origin: engine`.
+
+Memory uses `[tenant:]customer` when `customer` is supplied; subsequent turns
+must carry the same customer. Without one, memory is written to the tenant cell,
+or `chat:<session-id>` when there is no tenant. Use an isolated eval deployment:
+a fresh conversation ID alone does not isolate tenant memory. Evidence IDs seed
+only the runtime ledger, never model-visible text; `evidence.cards` seeds the cards
+a presentation tool can show.
+See [Snapshot evals](EVALS.md#state--a-case-starts-from-a-snapshot).
+
 ## The compatibility rule
 
 **Request and response shapes are stable; changes are additive-only.** A new