insika 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5c2b2fbe064af45aa132f93de00954b24dc504e052db2e059b4f6daa151e248b
-  data.tar.gz: 50cabcc10d82eabdfd08de562bd61feddf1a9de68ee6427343c80c5cf6ad9d8c
+  metadata.gz: 345e57c18a277e8ecd49b2481300c1474c60b2e0c3c384a086740e0de5ec3a2e
+  data.tar.gz: 800794431e279fec34f135e4e203a0488b04066b667bc691806ea5f575d81530
 SHA512:
-  metadata.gz: 3004e30a234349bcb91db185fcc31f9b52cbdeb7660c613b1fc229adf6bbbe717d9abc8ebcdc5bf9ecc125986694c0d2f3a82e1d48eb83105ff74db0467e4d91
-  data.tar.gz: ca4bec6a8b8cd03ee1a5c2df627e7da778d3dd157f30f848c70f8077ac467570284bf934e3a4c8040fb223503855d9cdcad111b9139ed093c702b6a8850ea1ff
+  metadata.gz: 199cc16b9ea651ec2da1d2e6f7ddea5020194882da00cf96981e8d5d44086233862873d14640d11cd1d445f9d202992ed46691210f9fb1699b1a3d56a99b3145
+  data.tar.gz: 44d24bc53036da3803ee6eba6203d1addc8ccf0bd27c1aa15d1e9acfe763bca807ede55c2b7136ee2c245e5cd725541c719baa018c99e09638e964803bf688a9

data/CHANGELOG.md

@@ -8,6 +8,122 @@ it is released. Entries land with the pull request that makes the change.
 
 ## [Unreleased]
 
+## [0.8.0] - 2026-08-31
+
+The post-consolidation hardening wave: the last two OpenClaw-era names are gone
+from the wire (**breaking** — `insika:<agent>` / `X-Insika-Agent` and
+`INSIKA_GATEWAY_TOKEN`, no aliases), and the engine learned to manage its own
+long conversations — in-session compaction (RFC-0044), a tool budget that warns
+before it kills, goal recitation at the tail of the context, and a tool-usage
+audit report. Update `/v1` consumers and the gateway-token variable together
+with this upgrade.
+
+### Added
+
+- **In-session compaction (RFC-0044).** When a session's uncompacted
+  transcript grows past `compact_after` messages (default 40), everything but
+  the last `keep_last` (default 20) is summarized by a cheap model
+  (`compaction.model` → platform `utility_model`) into one
+  `<conversation_summary>` history fragment; the tail stays verbatim and the
+  boundary is stable between compactions, so the prompt cache holds after it.
+  Runs post-turn, off the critical path, best-effort; persisted on the session
+  record with a monotonic boundary. Opt-in via the long-reserved Settings
+  `compaction` hash (the Studio general form is back), `insika doctor` warns
+  when it is enabled with no model, and every compaction is observable:
+  `:context_compacted` event, `insika.context.compacted` counter, `{upto,
+  runs}` + its own category in the context trace. See
+  [Context](docs/CONTEXT.md).
+
+- **`insika tools:report` — the tool audit.** The per-session trace never
+  aggregated, so "which tools does this agent carry and never use?" had no
+  answer. The new CLI report reads the stored traces per agent (tasks →
+  sessions → `tool_traces`) and flags allowlisted-but-never-called tools,
+  tools over 30% errors in the window, and tools not called in N days
+  (default 14). Read-only — the operator removes. `--agent`, `--days`,
+  `--json`. See [Tools](docs/TOOLS.md).
+
+- **Two series promoted to OTEL metrics.** `insika.cache.hit_rate` (histogram,
+  `%` — cache reads over the billed prompt, per turn, same arithmetic as the
+  Studio's per-agent series) and `insika.tool.loop_intervened` (counter — the
+  loop detector's one-shot warning, labelled by tool). Both computed in the
+  telemetry bridge from events that already existed: zero cost when OTEL is
+  off, no engine change. See [Observability](docs/OBSERVABILITY.md).
+
+- **The doctor warns on a prompt file that outgrew a prompt.** The
+  `prompt-files` sweep now WARNs (never errors) past ~6 000 estimated tokens
+  or 600 lines, with the actual counts — the LLM-generated merchant-pack
+  shape that costs 20%+ extra tokens per turn. Corrupted-wrapper detection
+  is unchanged.
+
+- **The tool budget is announced before it kills.** `max_tool_calls` was
+  enforced but silent: the model met the ceiling only when the turn died with
+  `stage: :tool_limit`, having delivered nothing. At **10 / 5 / 2** calls
+  remaining the engine now appends a short, escalating notice at the next
+  tool-batch boundary and emits `:tool_budget_warned`. It is a `user` message,
+  never a system one, so the cacheable system prefix stays byte-stable.
+  Verified live (DeepSeek): a turn asking for 30 lookups on a budget of 20 used
+  to die at `tool_limit`; it now stops at 18 and answers with what it has. The
+  turn's counter, the notices and the abort are one object (`TurnBudget`), and
+  the batch-boundary arithmetic it shares with the loop detector moved into
+  `ToolBatch`.
+
+### Changed
+
+- **The briefing recites the goal at the TAIL of the context.** `<briefing>`
+  used to carry known facts, the still-missing list and the next step in one
+  `:system` block at the top. The goal half was **moved** (not copied) into a
+  new `<recitation>` block rendered after the whole history, as the last thing
+  the model reads before the current message — attention is strongest at the
+  end, and a goal stated only at the top is the first thing a 30-call turn
+  forgets. New `:tail` fragment placement; the head keeps the durable facts.
+
+- **The `/v1/responses` wire names are insika-native**: the agent arrives as
+  `model: "insika:<agent>"` and the fallback header is `X-Insika-Agent`. Hard
+  rename, no alias — a consumer still sending `openclaw:<agent>` gets
+  "model/agent missing". Update every `/v1` consumer together with this upgrade.
+- **`INSIKA_GATEWAY_TOKEN` replaces `OPENCLAW_GATEWAY_TOKEN`** as the Bearer for
+  `/v1` + `/a2a`. Hard rename, no alias: the old name was a leftover of the
+  OpenClaw migration and is now a foreign var the engine ignores. Update the
+  variable on every deployment before upgrading. The dead `OPENCLAW_AGENTS_DIR`
+  env spec (nothing read it) is gone.
+
+### Fixed
+
+- **The Studio home charts went blank for three hours every day.** The activity
+  buckets key on calendar parts (date, hour) of `updated_at`, which the engine
+  writes in UTC, but `render_home` read `Time.now` in the host's local zone. On a
+  UTC-3 host, from 21:00 local onward "today" was already tomorrow in UTC: the
+  14-day chart matched no session and the 24h sparkline's floor was set three
+  hours in the future, dropping the newest buckets. Stamps are now read through
+  one `utc_time` helper and `now` is UTC. Instant comparisons (`active_now`)
+  never had the bug — only the calendar arithmetic did.
+- **The Conversations trend showed "−1" for the day's first conversation.** The
+  14-day series is oldest-first and ends at today, so its last pair reads
+  `[yesterday, today]`; it was destructured the other way round, inverting the
+  sign on every conversations delta. The Messages card was always correct.
+- **Declaring a tool allow/deny list now opts the profile into the policy.** The
+  lists are applied by exactly one policy — the builtin `tool_allowlist` — and
+  the policy engine runs only the policies a profile *names*. So an agent with
+  `tools_allow: ["a", "b"]` and no `policies` sent **every registered tool** to
+  the model, silently: 508 tools / 52 405 tokens of schemas per request against
+  an allowlist of 2, observed on a real run. `tools_deny` was ignored the same
+  way. `AgentProfile.build` now appends `tool_allowlist` whenever `tools_allow`,
+  `tools_deny` or `tools_allow_groups` is declared — presence, not emptiness, so
+  `tools_allow: []` still means "no tools". A stored record left in the old
+  state is repaired on read but stays wrong on disk, so `doctor` gained a
+  `tool-allowlist` **error** naming each agent to re-save.
+- **A DSL agent could write its briefing and never read it back.** The
+  `Insika.agent`/`Insika.system` runtime wired every context provider except
+  `Briefing`, so `update_briefing` / `set_next_step` persisted state that never
+  reached a prompt. Now wired, before `Session`, like `config/wiring.rb`.
+- **Docker image shipped without `docs/`, so `GET /start.md` was a 500 with
+  `INSIKA_ONBOARDING=1`.** `.dockerignore` excluded the whole tree; the onboarding
+  surface reads `docs/onboarding/start.md`, `docs/prompts/` and the public
+  `docs/*.md` from disk at request time. Only the Jekyll build output and the
+  internal notes are excluded now, and a spec asserts `.dockerignore` never drops
+  a file `Insika::Onboarding` serves. Found on the first Railway deploy with
+  onboarding on; the gem was never affected (its payload keeps `docs/`).
+
 ## [0.7.0] - 2026-08-29
 
 The pre-1.0 consolidation: MCP servers as first-class tools, a template gallery,

data/README.md

@@ -143,7 +143,7 @@ navigates by.
 - [Evals](docs/EVALS.md) — the cases that grade an agent: rubrics, the judge panel, and the pre-merge gate.
 - [Refinement](docs/REFINEMENT.md) — read an agent's own traffic back as a ranked report of what broke.
 - [Outcomes and follow-ups](docs/OUTCOMES.md) — what the traffic was worth in business terms, and the tool that comes back on a promise.
-- [Knowledge](docs/KNOWLEDGE.md) — extract durable concepts from finished conversations into a per-agent knowledge base; provenance-stamped, PII-redacted, best-effort with a re-scan recovery path (retrieval and consolidation are not shipped yet).
+- [Knowledge](docs/KNOWLEDGE.md) — extract durable concepts from finished conversations into a per-agent knowledge base; provenance-stamped, PII-redacted, best-effort with a re-scan recovery path.
 - [Facts](docs/FACTS.md) — distill finished customer conversations into proposed facts; a human approves them into memory (nothing is ever applied automatically).
 - [Harvest](docs/HARVEST.md) — mine real traffic for SKILL proposals; the negative list, the evidence-ledger grounding filter and the double gate hold every proposal to a human approval and the append-only log (nothing is ever applied automatically).
 
@@ -176,5 +176,6 @@ and [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). Vulnerabilities go through
 
 ## Status
 
-Pre-release — APIs may still change, nothing is tagged yet
-([CHANGELOG.md](CHANGELOG.md)). Licensed **MIT** ([LICENSE](LICENSE)).
+Pre-1.0 — published on [RubyGems](https://rubygems.org/gems/insika); APIs may
+still change between minor versions ([CHANGELOG.md](CHANGELOG.md)). Licensed
+**MIT** ([LICENSE](LICENSE)).

data/bin/insika

@@ -32,6 +32,7 @@ module Insika
       when "harvest" then harvest(argv)
       when "harvest:negative" then harvest_negative_import(argv)
       when "harvest:criterion" then harvest_criterion_check(argv)
+      when "tools:report" then tools_report(argv)
       when "knowledge:backfill" then knowledge_backfill(argv)
       when "knowledge:export" then knowledge_export(argv)
       when "soak"    then exit(Insika::Soak::Runner.main(argv, stdout: $stdout, stderr: $stderr))
@@ -65,6 +66,11 @@ module Insika
                                     seed the profile's negative list from a rules file
           harvest:criterion check --file F
                                     strict-load the frozen conversion criterion
+          tools:report [--agent ID] [--days N] [--json]
+                                    tool usage per agent over the stored traces:
+                                    allowlisted-but-never-called, error rate > 30%
+                                    in the window, and stale (not called in N days).
+                                    Read-only — the operator removes
           knowledge:backfill --agent ID [--since ISO8601] [--json]
                                     the recovery path for post-turn extraction: replays
                                     the agent's stored sessions through the same
@@ -337,7 +343,7 @@ def evals_simulate(argv)
   require "fileutils"
   require "time"
   opts = { base_url: ENV["INSIKA_URL"] || "http://localhost:9292",
-           token: ENV["OPENCLAW_GATEWAY_TOKEN"] || ENV["ADMIN_TOKEN"] || "local-demo",
+           token: ENV["INSIKA_GATEWAY_TOKEN"] || ENV["ADMIN_TOKEN"] || "local-demo",
            timeout: 120, persona_model: ENV["EVAL_PERSONA_MODEL"],
            judge_model: ENV["EVAL_JUDGE_MODEL"], judge_provider: ENV["EVAL_JUDGE_PROVIDER"] }
   OptionParser.new do |o|
@@ -345,7 +351,7 @@ def evals_simulate(argv)
     o.on("--persona FILE", "the persona YAML — a case with a `persona:` key (required)") { |v| opts[:persona] = v }
     o.on("--target T", "agent id (over --base-url) or an A2A http(s) URL (required)") { |v| opts[:target] = v }
     o.on("--base-url URL", "deployment base URL (default #{opts[:base_url]})") { |v| opts[:base_url] = v }
-    o.on("--token T", "gateway token (default OPENCLAW_GATEWAY_TOKEN/local-demo)") { |v| opts[:token] = v }
+    o.on("--token T", "gateway token (default INSIKA_GATEWAY_TOKEN/local-demo)") { |v| opts[:token] = v }
     o.on("--timeout N", Integer, "per-turn read timeout, and the A2A poll budget in seconds (default 120)") { |v| opts[:timeout] = v }
     o.on("--staging", "the target is a staging deployment (side effects acceptable)") { opts[:staging] = true }
     o.on("--eval-profile", "the target runs an eval profile (side-effect tools swapped for fakes)") { opts[:eval_profile] = true }
@@ -1162,6 +1168,41 @@ end
       "last #{n} session(s)"
     end
 
+    # insika tools:report [--agent ID] [--days N] [--json]
+    #
+    # The tool audit: reads the SAME durable backend as `doctor` (tasks →
+    # sessions → tool_traces), no app boot. Read-only — it names never-called
+    # allowlisted tools, high-error tools and stale tools; the operator removes.
+    def tools_report(argv)
+      opts = { json: false, days: Insika::ToolUsageReport::WINDOW_DAYS }
+      OptionParser.new do |o|
+        o.banner = "Usage: insika tools:report [--agent ID] [--days N] [--json]"
+        o.on("--agent ID", "narrow the report to one stored agent") { |v| opts[:agent] = v }
+        o.on("--days N", Integer, "window for error-rate and staleness (default #{opts[:days]})") { |v| opts[:days] = v }
+        o.on("--json", "emit the report as JSON") { opts[:json] = true }
+      end.parse!(argv)
+
+      backend = Insika::Wiring::Graph.backend_from_env
+      config_store = Insika::ConfigStore.new(store: backend)
+      report = Insika::ToolUsageReport.new(
+        task_store: Insika::TaskStore.new(store: backend),
+        tool_trace_store: Insika::ToolTraceStore.new(store: backend),
+        profile_source: Insika::StoredProfileSource.new(config_store: config_store)
+      ).generate(days: opts[:days], agent: opts[:agent])
+
+      if opts[:agent] && report.agents.empty?
+        warn "insika tools:report: no stored agent '#{opts[:agent]}'"
+        exit 2
+      end
+
+      if opts[:json]
+        require "json"
+        puts JSON.pretty_generate(report.to_h)
+      else
+        puts report.to_s
+      end
+    end
+
     # Config stores over the real durable backend — no app boot / no DEEPSEEK.
     def build_doctor
       backend = Insika::Wiring::Graph.backend_from_env

data/docs/AGENTS.md

@@ -32,9 +32,10 @@ All three land on the **same** config-over-code import path — they differ only
 ergonomics, not in what they produce.
 
 - **DSL** — `Insika.agent("id") { … }` builds an agent definition and imports it
-  into the durable store. Best for code-defined agents and examples. The DSL
-  auto-enables the tool- and skill-allowlist policies. `model` is optional — a nil
-  model resolves the platform `default_model` at turn start.
+  into the durable store. Best for code-defined agents and examples. The DSL also
+  names the **skill**-allowlist policy for you (the tool one is implied by any
+  declared tool list, on every path). `model` is optional — a nil model resolves
+  the platform `default_model` at turn start.
 - **API** — `POST /v1/agents` with a definition (a "pack": an agent config plus its
   prompt files, skills, and data-tools). The import is **idempotent** and
   **authoritative** — what leaves the definition leaves the agent, so a
@@ -108,6 +109,14 @@ DEFAULT_LIMITS = {
 
 `build` merges your overrides over these — you set only the deltas.
 
+`max_tool_calls` is announced before it kills. At **10 / 5 / 2** calls remaining
+the engine appends a short user message at the next tool-batch boundary
+("2 tool calls left — consolidate what you already have and answer now"), so a
+long turn converges on an answer instead of dying with `stage: :tool_limit` and
+delivering nothing. It is a `user` message, never a system one: the system prefix
+stays byte-stable and `prompt_caching` keeps hitting. Each threshold fires at
+most once per turn and emits `:tool_budget_warned` on the event stream.
+
 `max_tool_repeat` is the loop guard: the same tool called with **identical
 arguments** that many times in a row gets ONE in-turn warning (a user message at
 the next tool-batch boundary: "you already ran this, answer with what you
@@ -382,6 +391,16 @@ The same three-state rule governs tools, skills, context providers, and workflow
 For tools, a paired deny list (`tools_deny`) **always wins**, and
 `tools_allow_groups` unions a per-group allowlist on top of `tools_allow`.
 
+**Declaring a list is opting into it.** The lists are applied by one policy, the
+builtin `tool_allowlist`, and the policy engine runs only the policies a profile
+names — so declaring `tools_allow` without naming that policy used to mean the
+model got *every* registered tool. The engine now adds `tool_allowlist` for you
+the moment any of `tools_allow`, `tools_deny` or `tools_allow_groups` is
+declared, on every creation path. Presence is the trigger, not content:
+`tools_allow: []` means "no tools" and enforces just as hard. If a stored agent
+predates this, `insika doctor` names it — the engine repairs it on read, but the
+stored record stays wrong until you re-save it.
+
 Three capabilities invert the default — `nil`/absent means **OFF**, not "all":
 `subagents`, `memory`, and `guardrails` (each defaults to off or a conservative
 setting, never "everything on"). `tool_output_compression` is a fourth: opt-in

data/docs/ARTIFACTS.md

@@ -67,6 +67,48 @@ No script, no external fetch, no forms. The model writes HTML with **inline
 SVG** for charts — that is a skill instruction (palette, tables, pure-SVG bars),
 not engine code. A "real" charting need is a plugin.
 
+## Reasoning effort on a report turn
+
+A report turn is not one shape of work — it plans, then it mines, then it
+writes. `thinking` is one value per agent for the whole turn, so an agent set to
+`high` pays deliberation on every one of the 30–50 tool calls a real report
+makes, and that is where the 300 s turn timeout gets spent.
+
+Split the phases across agents instead, which the engine already supports today:
+
+```ruby
+Insika.system do
+  # The miners: one narrow question each, no judgement to make.
+  agent("sales_miner") do
+    model "deepseek-v4-flash"
+    params thinking: "low"
+    tools %w[query_sales]
+    instructions "Answer ONE question about sales from the store data. Numbers, no prose."
+  end
+
+  # The orchestrator: it plans the report and writes it. This is the turn
+  # that deserves the deliberation.
+  agent("reporter") do
+    model "deepseek-v4-flash"
+    params thinking: "high"
+    tools %w[save_artifact]
+    subagents "sales_miner"
+    instructions "Plan the report, call spawn_subagents ONCE for every number you need, then write the page and save_artifact it."
+  end
+end
+```
+
+Two things make this work: a child inherits the *environment* (model, thinking)
+only as a **default**, so its own `params thinking:` wins; and `spawn_subagents`
+runs the children in parallel, so wall-clock is the slowest miner rather than
+the sum. Each child also mines in its own isolated context, which is what keeps
+the orchestrator's context from filling with raw rows.
+
+Measure it before reaching for anything cleverer: the numbers that matter are
+the turn's wall-clock, the timeout rate, and the judge score on the same report.
+Per-phase effort *inside* a single turn is a real idea, but it is only worth
+building once this recipe is shown not to be enough.
+
 ## Limits and retention
 
 - **Size cap** — `INSIKA_ARTIFACT_MAX_BYTES` (default 1 MB): an artifact is a

data/docs/CONTEXT.md

@@ -28,9 +28,10 @@ into a deterministic prompt:
 | **Skill trigger** | `<active_skill>` | 85 | volatile | Level-2 bodies: the agent's `skills_eager` set, plus the ones whose `triggers:` match the message — see [Skills](SKILLS.md). |
 | **Knowledge** | `<knowledge>` | 77 | volatile | Level-1 top-K learned concepts for the turn's message (+ one-hop `[[links]]`), only if `knowledge.retrieve` is on. Cuttable — see [Knowledge](KNOWLEDGE.md). |
 | **Memory** | `<memory>` | 75 | volatile | Durable facts + recent notes, only if `memory` is on. Cuttable. |
-| **Briefing** | `<briefing>` | 65 | volatile | The session's working state (known fields, still-missing list, next step) — only if the pack declared `briefing_fields`. Cuttable. |
+| **Briefing** | `<briefing>` | 65 | volatile | The session's working state — the *known* fields only. Only if the pack declared `briefing_fields`. Cuttable. |
 | **Session** | history | 60–79 | volatile | The running transcript; priority scales with recency. |
 | **Request** | `<request_context>` | 40 | volatile | Turn variables + tenant. Most cuttable; sits last. |
+| **Briefing (tail)** | `<recitation>` | 95 | volatile | The still-missing list + next step, rendered **after the whole history** as a `user` message — the last thing the model reads before the current message. |
 
 The ordering is deliberate: the render order is **identity layer first, volatile
 layer after** — nothing volatile can sit above the cache boundary, whatever its
@@ -53,14 +54,37 @@ cacheable prefix byte-stable (see the prefix cache below).
 > returns empty turns, raise `context_budget` (e.g. to `60000`) before looking
 > anywhere else. See [Agents](AGENTS.md#default-limits).
 
-### Compaction is not wired — except the mechanical dedupe
-
-There is a settings stub for LLM-summarization compaction (`enabled: false`,
-`keep_last`, a reserved utility-model slot), but **nothing consumes it today**
-— and the Studio no longer shows a form for it, so the setting cannot be
-switched on by accident. Size is managed purely by hard budget eviction.
-Do not rely on compaction to shrink a bloated agent: tune `context_budget` and
-keep the identity lean.
+### In-session compaction (opt-in, platform Settings)
+
+When enabled (Studio → Settings → General, or the `compaction` settings hash),
+the engine summarizes old turns *inside* the session instead of losing them to
+eviction:
+
+- **Trigger:** after a turn commits, if the session's *uncompacted* message
+  count exceeds `compact_after` (default 40), everything but the last
+  `keep_last` messages (default 20) is summarized by a cheap model
+  (`compaction.model`, falling back to the platform `utility_model`; neither
+  set = the feature is inert and `insika doctor` warns). Runs off the critical
+  path — the customer already has the answer.
+- **Read path:** the Session provider replaces the compacted prefix with ONE
+  history fragment — a `user` message wrapped in `<conversation_summary>` tags,
+  priority 59 (one step below the oldest verbatim message), source
+  `compaction` (its own category in the context trace). The tail stays
+  verbatim and the boundary is **stable** between compactions, so the prompt
+  cache holds after it.
+- **What survives:** the default prompt orders the summary to preserve customer
+  facts (sizes, CEP, order numbers), the assistant's commitments, the still-open
+  questions and the decisions already made; on re-compaction the previous
+  summary is folded in, so a fact from turn 3 survives every later batch.
+  A platform `compaction.prompt` replaces the default wholesale.
+- **Scope:** store-sourced history only — a checkpoint resume replays its own
+  tape and an explicit `history` is the caller's contract; neither is
+  rewritten. Observability: the `:context_compacted` event, the
+  `insika.context.compacted` counter and `{upto, runs}` in the context trace.
+
+Compaction does not replace the budget: eviction stays as the hard backstop
+for a single oversized turn. Tune `context_budget` and keep the identity lean
+regardless.
 
 One cheap half **is** wired, opt-in per agent: `tool_output_compression` (DSL
 `tool_output_compression`, or `"tool_output_compression": true` in the pack).
@@ -121,19 +145,37 @@ pack:
 briefing_fields "size", "budget", "delivery_day"
 ```
 
-With fields declared, the turn's `:system` context gains a `<briefing>` block
-(priority 65 — below identity/skill/memory so it never breaks the cacheable
+With fields declared, the briefing renders in **two places**, and the split is
+deliberate.
+
+The durable half — what is already known — sits in the `:system` context
+(priority 65: below identity/skill/memory so it never breaks the cacheable
 prefix, above the turn's own `<request_context>`):
 
 ```
 <briefing>
 known:
   size: M
+</briefing>
+```
+
+The half that is a *goal* — what is still missing and the agreed next step — is
+**recited at the tail**, after the whole history, as the last thing the model
+reads before the current user message:
+
+```
+<recitation>
 still missing: budget, delivery_day
 next step: send the payment link tomorrow at 10
-</briefing>
+</recitation>
 ```
 
+Attention is strongest at the end of the context: a goal stated only at the top
+is the first thing a 30-call turn forgets. So the recitation was **moved** there,
+not copied — the head never repeats it, and the turn pays for it once. It rides
+as a `user` message, like every other engine append inside a turn, so the system
+prefix stays byte-stable and the cache breakpoint at its end keeps hitting.
+
 The `still missing` list is the point: the *model* sees which declared fields are
 still unanswered, so it stops re-asking for something already given. Stored keys
 that the pack no longer declares are never rendered. The Studio session screen

data/docs/DEPLOY.md

@@ -25,7 +25,7 @@ is durable SQLite (WAL) at `INSIKA_DB`; mount a volume and point it inside.
 docker build -t insika .
 docker run -p 9292:9292 -v insika-data:/data \
   -e DEEPSEEK_API_KEY=sk-... \
-  -e OPENCLAW_GATEWAY_TOKEN=change-me \
+  -e INSIKA_GATEWAY_TOKEN=change-me \
   insika
 curl localhost:9292/up      # {"status":"ok"}
 ```
@@ -115,7 +115,7 @@ this section is the single source of truth for what changing it means.
 | `INSIKA_DRAIN_TIMEOUT` | `20` | seconds a stopping worker waits for in-flight turns before abandoning them to the next boot's recovery (process model, item 4). The entrypoint sizes Falcon's `--graceful-stop` from it; on Railway also set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS` ≥ drain + 10 |
 | `INSIKA_TICK_INTERVAL` | `60` | seconds between tick passes — outbox drain + stale recovery sweep (process model, item 5). `0` disables |
 | `INSIKA_TICK_STALE_AFTER` | `900` | seconds a `:queued`/`:running` task must sit untouched before the tick sweeps it. Must exceed the largest `turn_timeout` of the deployment |
-| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN` | Bearer for `/v1/responses` and `/v1/agents` (the API contract) |
+| `INSIKA_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN` | Bearer for `/v1/responses` and `/v1/agents` (the API contract) |
 | `ADMIN_TOKEN` | `local-demo` | login token for `/studio` (**change in production**) |
 | `DEEPSEEK_API_KEY` | — | provider key. **Without it the engine still boots** (`/up` green), but turns fail until it is configured (env or Studio → LLM providers) — cloud resilience |
 | `DEEPSEEK_MODEL` | `deepseek-v4-flash` | model |
@@ -157,7 +157,7 @@ values (the API token falling back to `ADMIN_TOKEN` is a dev convenience only):
 - **`ADMIN_TOKEN`** — the `/studio` login (cookie auth). This is the **operator**
   surface (just you). Rotating it is **safe and independent**: change it, redeploy,
   log in with the new value. It does not affect any API consumer.
-- **`OPENCLAW_GATEWAY_TOKEN`** — the Bearer for `/v1/responses` and `/v1/agents`.
+- **`INSIKA_GATEWAY_TOKEN`** — the Bearer for `/v1/responses` and `/v1/agents`.
   This is the **contract with your API consumers**. Rotating it means **changing
   both sides together** (or the integration breaks): update the runtime var **and**
   each consumer's token in the same step.
@@ -187,15 +187,24 @@ insika doctor            # colored report; exits != 0 on any error
 insika doctor --json     # machine-readable (CI / monitoring)
 insika doctor --fix      # applies the safe autofixes and re-diagnoses
 insika env               # lists known keys + current values (secrets masked)
+insika tools:report      # tool audit over the stored traces: never-called
+                         # allowlisted tools, error rate > 30%, stale tools —
+                         # read-only, the operator removes ([--agent ID] [--days N] [--json])
 ```
 
 Checks: env (the schema above), settings schema version (a pending migration →
 `--fix` applies it), a missing platform `default_model` (`--fix` seeds it from
 `DEEPSEEK_MODEL`), durable vs ephemeral backend, LLM provider configured,
-`ADMIN_TOKEN` set, data-tool definitions still valid, **prompt files that hold
+`ADMIN_TOKEN` set, data-tool definitions still valid, **a stored agent whose
+declared tool allow/deny list does not name the `tool_allowlist` policy** (the
+engine repairs it on read, but as stored the list is inert — see
+[Agents](AGENTS.md#the-allowlist-convention)), **prompt files that hold
 text rather than a serialized object** (a file whose content is a stringified Hash
 serves a mangled prompt on every turn while looking perfectly healthy — present,
-non-empty, and the agent still answers), and **skill drift** — a shared skill whose
+non-empty, and the agent still answers), **a prompt file that outgrew a prompt**
+(WARN past ~6 000 estimated tokens or 600 lines — the LLM-generated pack shape
+that costs 20%+ extra tokens per turn for no better instruction-following), and
+**skill drift** — a shared skill whose
 body names one store, a prompt file routing to a skill the agent cannot load, a broken
 companion pair, a stale `eager:` key (see
 [Skills](SKILLS.md#drift-guards)). Settings-schema migrations are **explicit**
@@ -227,7 +236,7 @@ healthcheck, and a restart policy.
 2. **Volume**: mount it at `/data` (the default `INSIKA_DB` points there) —
    without a volume, SQLite is ephemeral and recovery resumes nothing after a
    redeploy.
-3. **Vars**: `DEEPSEEK_API_KEY`, `OPENCLAW_GATEWAY_TOKEN`, `CONSUMER_INTERNAL_URL`,
+3. **Vars**: `DEEPSEEK_API_KEY`, `INSIKA_GATEWAY_TOKEN`, `CONSUMER_INTERNAL_URL`,
    `INSIKA_EGRESS_HOSTS`. **Leave `WEB_CONCURRENCY` at its default of 1** unless
    you run [`insika-router`](ROUTER.md) in front. Railway's own docs say it
    "does not support sticky sessions" and randomly distributes traffic across
@@ -402,7 +411,7 @@ cache hits, P50/P95, error rate. Runs against local or a remote deployment. See
 [LOADTEST.md](LOADTEST.md).
 
 ```bash
-INSIKA_URL=http://localhost:9292 OPENCLAW_GATEWAY_TOKEN=xxx \
+INSIKA_URL=http://localhost:9292 INSIKA_GATEWAY_TOKEN=xxx \
   bundle exec ruby scripts/loadtest.rb --agents assistant --concurrency 16 --iterations 3
 ```
 

data/docs/LOADTEST.md

@@ -67,7 +67,7 @@ frame that carries it.
 
 ```bash
 INSIKA_URL=http://localhost:9292 \
-OPENCLAW_GATEWAY_TOKEN=xxx \
+INSIKA_GATEWAY_TOKEN=xxx \
 bundle exec ruby scripts/loadtest.rb \
   --agents demo,my-store --concurrency 16 --iterations 3 \
   --message "hi, how are you?"
@@ -80,7 +80,7 @@ Runs against a local server **or** a remote one (e.g. Railway) — just point
 
 | Flag | Default | Meaning |
 |------|---------|---------|
-| `--agents a,b,c` | `demo` | comma-separated agent ids (mapped to `model: openclaw:<agent>`) |
+| `--agents a,b,c` | `demo` | comma-separated agent ids (mapped to `model: insika:<agent>`) |
 | `--concurrency N` | `8` | concurrent turns per wave |
 | `--iterations N` | `1` | number of waves per agent |
 | `--message TEXT` | greeting | user message sent every turn |
@@ -95,14 +95,14 @@ Runs against a local server **or** a remote one (e.g. Railway) — just point
 | Env | Default | Meaning |
 |-----|---------|---------|
 | `INSIKA_URL` | `http://localhost:9292` | base URL of the engine |
-| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN`, then `local-demo` | Bearer for `/v1/responses` |
+| `INSIKA_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN`, then `local-demo` | Bearer for `/v1/responses` |
 | `DEEPSEEK_API_KEY` | — | must be configured **on the server** for real turns (not read by the client) |
 
 Use `--dry-run` to sanity-check your flags/URL/token before firing real traffic
 (and to confirm the request body without needing a running server):
 
 ```bash
-INSIKA_URL=http://localhost:9292 OPENCLAW_GATEWAY_TOKEN=xxx \
+INSIKA_URL=http://localhost:9292 INSIKA_GATEWAY_TOKEN=xxx \
   bundle exec ruby scripts/loadtest.rb --agents demo --concurrency 16 --dry-run
 ```
 
@@ -133,7 +133,7 @@ DEEPSEEK_API_KEY=sk-... ./scripts/loadtest-local.sh [WORKERS] [CONCURRENCY]
 | Env | Default | Meaning |
 |-----|---------|---------|
 | `DEEPSEEK_API_KEY` | — (required) | real turns hit the provider; also auto-sourced from `.env.local` |
-| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN`, then `local-demo` | Bearer for the sweep |
+| `INSIKA_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN`, then `local-demo` | Bearer for the sweep |
 | `PORT` | `9299` | bind port for the local Falcon |
 | `AGENT` | `demo` | agent id to load |
 
@@ -158,30 +158,18 @@ the engine, once at the gateway (its `/v1/responses` speaks the same protocol).
 Keep `--agents`, `--concurrency`, `--iterations` and `--message` identical, and use
 matching agents on both sides. Compare the printed TTFB/total/cache/error lines.
 
-### 4b. Reuse OpenClaw's `loadtest-gateway.mjs` unmodified
+### 4b. Comparing against an OpenClaw gateway
 
-`loadtest.rb` is the Ruby port of OpenClaw's `loadtest-gateway.mjs`. You do **not**
-need to change that script to point it at the engine — because the engine is a
-drop-in for the gateway, you only change **where it points**:
+The engine speaks its own wire names (`model: insika:<agent>`, header
+`X-Insika-Agent`), so OpenClaw's `loadtest-gateway.mjs` cannot be pointed at it
+unmodified. For a shadow comparison, run `loadtest.rb` (section 4a) against each
+side with identical `--agents`, `--concurrency`, `--iterations` and `--message`,
+and diff the reports. What you still need in hand:
 
-```bash
-# In the OpenClaw checkout, run its gateway loadtest against the HARNESS:
-OPENCLAW_GATEWAY_URL=http://localhost:9292 \
-OPENCLAW_GATEWAY_TOKEN=<same bearer the engine accepts> \
-node scripts/loadtest-gateway.mjs --agents demo --concurrency 16 --iterations 3
-```
-
-Then run the exact same command with `OPENCLAW_GATEWAY_URL` pointing at the real
-gateway, and diff the two reports. This is the shadow comparison the pilot needs.
-
-**What the operator must have in hand** (this repo does not vendor OpenClaw):
-
-- The OpenClaw checkout containing `scripts/loadtest-gateway.mjs` and Node installed.
-- A **bearer token accepted by both** sides. For the engine that is
-  `OPENCLAW_GATEWAY_TOKEN` (see DEPLOY.md); point the gateway run at its own token.
-- **The same agent id provisioned on both** sides (e.g. `demo`) so `model:
-  openclaw:<agent>` resolves on each. On the engine, provision via
-  `scripts/import_pack.rb`.
+- A bearer token accepted by each side — `INSIKA_GATEWAY_TOKEN` for the engine
+  (see DEPLOY.md), the gateway's own token for the gateway run.
+- **The same agent id provisioned on both** sides (e.g. `demo`). On the engine,
+  provision via `scripts/import_pack.rb`.
 - The **same provider** (or an equivalent-latency one) behind each, otherwise you
   are comparing providers, not engines.
 - Both endpoints reachable from where you run the client, warmed up (hit `/up` on

data/docs/MEDIA.md

@@ -87,7 +87,7 @@ receive it (`channel.capabilities` on the request):
 
 ```bash
 curl -X POST /v1/responses -H "Authorization: Bearer $TOKEN" -d '{
-  "model": "openclaw:store-support", "user": "chat-7",
+  "model": "insika:store-support", "user": "chat-7",
   "input": "manda a foto do sofá da promoção",
   "channel": { "capabilities": ["image_output", "audio_output"] }
 }'