insika 0.8.0 → 1.0.1

data/docs/EVALS.md

@@ -42,6 +42,117 @@ expect:
 Turns replay **in order** under one conversation, so a case can build context ("what
 about the shipping?" after "I want the 70% bar"); the assertions run on the last turn.
 
+### State — a case starts from a snapshot
+
+Every case above starts from an **empty** conversation. To test "the customer already
+saw three products and says *add the second one*", such a case would first have to
+replay the search turn — which makes it depend on the model's first answer, costs a
+turn, and cannot reproduce a messy state (a contradiction from six turns ago, a
+stored preference). `state:` is the precondition, loaded into the conversation
+**before turn 1**:
+
+```yaml
+id: loja-chocolates-add-seen
+agent: loja-chocolates
+requires:
+  tools: [add_to_cart]
+state:
+  evidence:
+    ids: ["SKU-70-DARK"]
+    cards:
+      - { type: card, url: "https://shop.example/70", id: "SKU-70-DARK", caption: "Dark chocolate" }
+  memory: { facts: { preference: "dark chocolate" }, notes: [] }
+  history:
+    - { role: user, content: "quero chocolate amargo" }
+    - { role: assistant, content: "O chocolate 70% é SKU-70-DARK." }
+  briefing: { fields: { cep: "01311-000" } }
+turns:
+  - user: "adiciona uma unidade desse chocolate 70% no carrinho"
+expect:
+  tools_called: [add_to_cart]
+  never_calls: [search_products]
+  reply_omits: ["SKU-70-DARK"]
+```
+
+Only those four `state` keys are accepted by the case loader. A persona case may
+carry `state` too. Adapt the tool names, arguments and briefing fields to the agent.
+
+The evidence ledger is a runtime precondition, not a list shown to the model:
+include the relevant product and ID in `history` or the user's request. `cards`
+seeds the cards a search would have returned (each needs a `url` and an `id`; the
+id counts as seen), so a presentation case needs no lookup in the turn.
+
+With the HTTP transport, the eval stays a **client**: the runner never writes a
+store. Seeding goes through
+`POST /v1/conversations/:id/seed` (same Bearer as the turn, same id namespacing for a
+tenant), and the deployment accepts it **only while the platform setting
+`evals.seeding` is on** — off by default, because a seeded conversation is a
+fabricated precondition, and the doctor warns while it is on. With the setting off
+the replay Runner **skips** seeded cases with a reason; it also skips when its
+transport has no seed support. Other seed errors fail the replay. In-process
+`GraphTransport` dispatches `seed_session` directly and bypasses `evals.seeding`;
+it needs a runtime exposing the graph. The persona
+Simulator surfaces seed refusal as an error instead of the Runner's skip result.
+
+A session with messages returns `409`. Replays default to `eval-<case-id>`; pass
+`--conv-map FILE` with a JSON mapping such as
+`{"loja-chocolates-add-seen":"eval-add-seen-run-2"}` to use a fresh ID on reruns.
+Seeded memory may share a tenant cell even with fresh IDs; see
+[the seed API](API.md#seeding-an-eval-conversation) for scope and payload details.
+
+#### Graders — what a turn's calls and reply are checked against
+
+The turn's SSE stream reports each tool call with its arguments and how it ended
+(`ok`, `error`, or `blocked` plus the gate that held it), so the deterministic
+layer can check more than "was the tool called". All optional; each is its own
+check, and the report names the one that failed.
+
+| Key | Checks |
+|-----|--------|
+| `tools_called: [name, name?]` | each required tool was called on the LAST turn (`?` = optional, never fails) |
+| `turns[i].tools_called: [names]` | the calls one specific turn must make — a two-turn case where turn one claimed an action and turn two repaired the store passes the key above and fails this one |
+| `never_calls: [names]` | none of these was called — the negative every `tools_called` needs |
+| `calls_one_of: [names]` | at least one of these was called |
+| `first_tool: name` | the first call's name |
+| `max_tool_calls: N` | a ceiling on the turn's calls |
+| `reply_includes: [substrings]` | each appears in the published answer (case-insensitive) |
+| `reply_omits: [substrings]` | none appears — where an internal id, a CPF or a raw tag leaking into the customer's text is pinned |
+| `blocked_gates: ["tool:gate"]` | each pair appears among the turn's blocked calls |
+| `ui_components: [names]` | a presentation tool showed at least one card of each component (the `insika.ui` frames) |
+| `no_ui: true` | the turn showed nothing — the negative of `ui_components` |
+| `must_not: [detectors]` | the negative detectors (`pii_leak`, `tool_error`, `phantom_action`, …); a blocked call is not a tool error |
+| `claims: { tool: pattern }` | what `phantom_action` reads — per mutation, the words a reply uses to say it did it (a regex, case-insensitive); the store's language, so it lives in the case |
+
+**Every positive has a negative.** A case that only says `tools_called: [add_to_cart]`
+passes an agent that also re-searched, or that echoed the SKU to the customer. Pin
+what a correct turn does *not* do — `never_calls`, `reply_omits`, `max_tool_calls` —
+in the same case. Snapshots avoid the model calls needed to reconstruct the setup;
+use the simulator when later customer messages need to branch on the replies.
+
+`tools_called` confirms an attempted call, not a successful backend mutation.
+
+`must_not: phantom_action` reads the other direction: a reply that SAYS it did
+something ("adicionei", "removi", "fechei o pedido") on a turn where the matching
+tool never ran, errored, or was blocked. It is cumulative over the conversation, so
+"já adicionei" on turn two is true when turn one did it, and it only pins the words
+against the calls — a cart that ended up right by accident is still a lie to the
+customer. The words come from the case's `claims:` map, in the store's own
+language, past tense on purpose: "quer adicionar?" is a question.
+
+```yaml
+expect:
+  must_not: [tool_error, phantom_action]
+  claims:
+    add_to_cart: '\b(?:adicion(?:ei|ad[oa]s?)|coloquei)\b'
+    create_order: '\b(?:fechei|finalizei|pedido (?:criado|fechado))\b'
+```
+Arguments are collected for inspection; there is no generic argument or cart-state
+grader. `blocked_gates` needs completion statuses from the HTTP transport. The
+in-process `GraphTransport` records tool names and UI events, but not completion
+statuses or arguments. Neither transport counts `load_skill` / `load_knowledge` as
+tool calls. `ui_components` requires a nonempty selection; `no_ui`
+accepts absent UI events and events with zero items.
+
 ### `requires` — a case that cannot run here is skipped, not failed
 
 Deployments differ. Some stores have order tracking wired, some do not; some run
@@ -90,12 +201,13 @@ judge is told the rule instead of guessing it.
 
 ```yaml
 expect:
-  policy: ask_once
+  policy: ask_once                    # the strict reading: one question, full stop
+  policy: { ask_once: { max: 2 } }    # a store that greets its customers
 ```
 
 | policy | the check | the judge is told |
 |---|---|---|
-| `ask_once` | no reply asks more than one question | two questions in one message is a failure |
+| `ask_once` | no reply asks more than `max` questions (default 1) | at most `max` per reply, and a greeting formula is courtesy rather than a question |
 | `investigate_first` | turn 1 asks something and calls no tool | ask on a vague request, don't search immediately |
 | `act_fast` | turn 1 calls a tool | asking what a search would answer is a failure |
 
@@ -108,6 +220,16 @@ so a tracking link's query string is not read as the agent asking something. It
 policy signal, not grammar — and crude was enough to catch an agent breaking a rule
 written in its own prompt, twice, with no model in the loop.
 
+**`max` is the store's, and it matters more than it looks.** The default of 1 comes from
+a real store's rule, quoted down to the punctuation: *"Máximo 1 pergunta principal por
+mensagem = 1 ponto de interrogação no fim"*. A store that greets its customers spends
+one of those on courtesy — measured across six harnesses answering the same "oi", `max:
+1` failed every reply containing "tudo bem?" and passed every reply without it, and
+nothing else. That grades a greeting habit, not the rule the policy exists for, which is
+the form: *"qual seu nome? qual o número do pedido? e o motivo?"*. Set `max` to what
+your store actually tolerates; the judge is told the same number, so the two halves
+never grade different rules.
+
 ### `reference` — compared against the system you want to replace
 
 `min_score: 0.7` says a reply cleared a bar you invented. It says nothing about
@@ -269,14 +391,11 @@ The model only sees `case_id`, enumerated with the ids the tool can actually run
 (never a free string it could invent) — every **simulated** case in the store, the same
 `persona:` shape as above.
 
-**Safety is derived here too, but there is no swap yet.** The tool computes the
-target's reachable side-effect tools (`Evals::EvalProfile`, the same derivation the CLI
-uses) and **refuses outright** — naming the tools — if that list is non-empty. Unlike
-the CLI, nothing here actually swaps a side-effect tool for a dry-run: `Evals::
-EvalProfile.registry` (the overlay) exists for exactly that, but nothing calls it yet.
-So `run_persona_eval` only runs against **read-only** target agents today; wiring the
-overlay into an in-process turn (so a target WITH a write tool can be tested safely) is
-follow-up work, not something this tool claims to do.
+**Side effects are replaced for local graph runs.** The tool derives the target's
+reachable side-effect tools and builds a throwaway executor with recorder tools
+under those names. Read-only tools and the other graph collaborators are shared.
+If side effects are reachable but no graph was supplied, it refuses the run.
+This substitution prevents real writes; it does not prove a backend mutation.
 
 **Budget**: the persona model + judge model calls are the cost of running the eval,
 charged to the **calling** agent's own turn — never the target's (the target's own

data/docs/FACTS.md

@@ -38,6 +38,10 @@ decides; the engine never applies its own proposal.
 5. Approved facts join the customer's memory cell and are injected by the
    Memory provider on the next turn of any session of that customer.
 
+The extraction transcript contains only nonblank `user` and `assistant` text,
+with original message indexes preserved. Tool payloads are excluded regardless
+of `fencing`; an assistant's repetition of tool text can still be included.
+
 ## Enabling it — the `distill:` block
 
 Distillation is pack data on the agent, exactly like `refinement:` or
@@ -66,12 +70,14 @@ The engine never writes store vocabulary.
 
 The **Facts** page (operate group, next to Follow-ups) shows:
 
-- **Pending** — oldest first, because evidence ages. Each card is the fact,
-  its confidence, its scope (`tenant:customer`), and the evidence excerpt read
-  from the transcript at request time (evidence is a link, never a copy). Three
-  buttons: **Approve & save to memory**, **Reject** (with an optional reason,
-  shown on the card), **Dismiss** (ghost — labelled "will not be proposed
-  again").
+- **Pending** — oldest first, because evidence ages. Select a fact to review its
+  proposed value, model confidence and conversation evidence. Long excerpts scroll
+  within the evidence panel; **Open conversation** shows their source. Keys and
+  scope/session IDs live under **Technical details**. **Approve & save to memory**
+  saves the value; **Reject or dismiss** expands the alternatives, including an
+  optional rejection reason. Both prevent the same key and value from being proposed
+  again. Decisions refresh the queue and open the next pending fact in the selected
+  agent/scope.
 - **Stale** — the CAS-lost re-present (below): the proposed value struck
   through next to the operator's current value, resolved by dismissal.
 - **Recent** — every resolved proposal, most recent first, with operator and

data/docs/HARVEST.md

@@ -35,8 +35,10 @@ A mine reads ONLY durable, finished data — never an in-flight turn, never a
 live prompt prefix (the "fork" is structural: the mining writes nothing to the
 sessions it read, so a customer turn's cache is untouched by construction). The
 miner gets the transcript slices (masked), the session's evidence-ledger ids,
-and the agent's current skill names so it does not re-propose them. Then the
-filters, each drop counted and logged:
+the agent's current skill names and the candidates still open (name and
+description), so it does not re-propose an idea under another name.
+Simulator and eval sessions (`sim-…`, `eval-…`) are never mined: they are
+not customers. Then the filters, each drop counted and logged:
 
 1. **The negative list** — the versioned rules file (seed, imported
    per store by `insika harvest:negative import --agent ID --file F`) and the
@@ -111,6 +113,9 @@ back, re-promoted is three readable rows.
 - `insika harvest:criterion check --file F` — strict-load the frozen conversion criterion (the hook before any promotion).
 - The **Harvest page** in the Studio: the human's inbox (each candidate with
   its evidence excerpt, the eval report, the conversion card, promote/reject),
-  the gated-but-blocked rows with the named ruler hole, the pending list, the
+  the gated-but-blocked rows with the named ruler hole and a "Run checks
+  again" once the hole is fixed, the pending list (every row with the skill
+  text, the rationale and a link to its origin conversation), a notice when
+  the loop skips the agent (harvest off, or no grounding matcher), the
   append-only promoted log with the rollback mirror, the negative list with
   per-rule rejection counts, and the criterion block read-only.

data/docs/KNOWLEDGE.md

@@ -27,6 +27,13 @@ and an operator can see, edit and resolve all of it in the Studio. Only the
 optional FTS5 index remains, deferred with a measured trigger — see
 [What's not here yet](#whats-not-here-yet).
 
+Extraction reads only nonblank `user` and `assistant` text, retaining original
+message indexes. Direct tool payloads are excluded regardless of `fencing`;
+assistant paraphrases can still reach the extractor. The names and descriptions
+in the injected `<knowledge>` block are sanitized when
+[fencing](AGENTS.md#fencing--third-party-text-is-data-never-instructions) is on.
+The full body returned by `load_knowledge` is not fenced.
+
 ## The concept format
 
 One concept is one record — a markdown document with a YAML frontmatter
@@ -88,6 +95,13 @@ end
 recalling (write-only, inspected by hand), or recall without learning
 (curate every concept by hand in the Studio, `provenance: policy`).
 
+The same settings are available in **Studio → Agents → select an agent →
+Configuration → Memory & knowledge**: learn from conversations, use knowledge in
+replies, concepts per turn, concept types, learning model and instructions.
+Knowledge reranking has its own switch and settings, independent of memory
+reranking, and defaults to off. Options authored in a pack but not exposed in the
+form, such as `index`, are preserved when saving.
+
 ## The write path
 
 After a turn completes, off the turn's critical path (the customer already has

data/docs/MEDIA.md

@@ -39,6 +39,8 @@ curl -X POST /v1/messages?stream=false -H "Authorization: Bearer $TOKEN" \
   the usage flows like any ask. The first image URL is also
   `{{ctx.image_url}}` for data tools — photo analysis outside the prompt, the
   tool's own egress applying when it fetches.
+  DeepSeek images require RubyLLM's [image-attachment fix](https://github.com/crmne/ruby_llm/pull/972),
+  included in this deployment's pinned revision, and a vision-capable model.
 - **Documents** (a prescription, a recipe, an invoice — most often a PDF)
   attach the same way images do; the first document URL is
   `{{ctx.document_url}}`. A model without document support fails the ask at
@@ -117,8 +119,10 @@ end
 - **Seams, not magic.** The generator is injectable per kind (specs stub it);
   the defaults are lazy: images via RubyLLM (paint), speech via a thin POST to
   the OpenAI-compatible `/audio/speech` endpoint using the same provider
-  config the chat uses — RubyLLM as of 1.16.0 has no speech API. A generated
-  part over 8 MB refuses loudly, never silently truncates.
+  config the chat uses. RubyLLM 2 has a speech API, but its inspected native
+  transport buffers non-audio/error responses without yielding chunks. Insika
+  retains its bounded HTTP transport until native limits cover those responses.
+  A generated part over 8 MB refuses loudly, never silently truncates.
 - **Not here:** what the generated image *means* — a fitting room, a product
   mockup — is a skill on top. The engine transports bytes and cost.
 

data/docs/OBSERVABILITY.md

@@ -5,7 +5,44 @@ nav_order: 1
 permalink: /observability/
 ---
 
-# Observability — OpenTelemetry (opt-in)
+# Observability — native model diagnostics and OpenTelemetry
+
+## Studio Models
+
+Open the operator-authenticated `/studio/models` page to filter native
+requests by the last 24 hours, 7 days, or 30 days, agent, provider, and model. It shows
+request volume, failures, retries, reported USD cost and unknown-cost coverage,
+p50/p90/p95 latency, separate input/output/cache-read/cache-write/thinking token
+subtotals, and the 20 slowest measured requests linked to their tasks.
+
+Charts show request volume, reported cost, and p50/p90/p95 latency over time,
+plus a ranked cost comparison by model. They share the page filters. The 24-hour
+view uses hourly intervals, 7 days uses six-hour intervals, and 30 days uses
+daily intervals, all in UTC. Each chart has an expandable table of exact values.
+Missing cost or duration leaves a gap; no-call intervals have zero requests.
+The page is a snapshot: apply the filters again to refresh it.
+
+Request latency includes native retries. Retries count usage attempts beyond the
+first within a request, not Insika task executions. Percentiles use completed
+requests with measured durations. Cost and token values sum only reported usage;
+when coverage is incomplete, they are subtotals, and a dash means no value was
+reported. An unknown-cost failed attempt followed by a priced success leaves
+partial cost coverage even when the subtotal is known.
+
+Every turn also leaves one row (agent, model, provider, status and its
+windows), whatever `INSIKA_TURN_TIMING` says. A failed turn keeps its row with
+the stage that failed and a total that runs to the failure, so timeouts stay in
+the p95 and the page shows the turn failure rate by stage; that flag only decides what the
+response exposes. **Turn latency** shows p50/p95 of ttft (the provider's first
+token), total, first balloon (what a channel customer waits), queue (debounce
+and FIFO wait before the turn starts), prep and tools. **Tool health** lists
+every tool call in those turns: calls, ok %, p50/p95 ms. A call is a failure
+when its result carries `error`; tools ms sums calls, so parallel calls count
+more than their wall time.
+
+The page reads content-free summaries stored independently of the capped task
+trace. History starts when the summaries are deployed; earlier requests are not
+backfilled. Task deletion, retention, and tenant purge remove their summaries.
 
 Insika already has an observability spine: the **event stream**. Every turn emits
 structured events (`task_started`, `tool_call`/`tool_result`, `data_tool_call`,
@@ -25,15 +62,17 @@ authoring writes (`:golden_written`, `:agent_file_written`, …), queue bookkeep
 channel delivery (`:channel_delivered` — see [Channels](CHANNELS.md))
 travel the same stream and are **ignored** by the bridge: they open no span and
 touch no instrument, because they are not part of a turn's latency or cost. Any
-other subscriber still sees them. The one mid-turn event the bridge does consume
-is `:tool_loop_intervened` — it feeds the `insika.tool.loop_intervened` counter
-(no span: the intervention is a fact about the turn, not a timed operation).
+other subscriber still sees them. Additional events feed counters without opening
+spans: `:tool_loop_intervened`, `:tool_blocked` and `:context_compacted`.
 
 They are worth subscribing to even so, because each is the ONLY record of
 something that left no task of its own behind:
 
 | Event | Data | What it answers |
 |---|---|---|
+| `:tool_blocked` | `name`, `gate`, `param` | a provenance gate refused a call before execution; no rejected value in this event |
+| `:ui` | `component`, `title`, `items`, `count`, `dropped` | the presentation selection, including customer-facing card content; see [API](API.md#tool-and-presentation-sse-events) |
+| `:session_seeded` | `session_id`, `tenant`, `keys` | an eval snapshot was loaded; no snapshot contents |
 | `:turn_coalesced` | `task_id`, `merged`, `arrivals[]` | the fragments a customer typed in a row arrived as separate messages, and when |
 | `:turn_steered` | `task_id`, `count`, `total` | a message arrived mid-run and was appended to the turn in flight |
 | `:turn_steer_released` | `task_id`, `released_as`, `count` | the run could not absorb it, so it became the turn `released_as` |
@@ -60,31 +99,52 @@ something that left no task of its own behind:
 | `:knowledge_deleted` | `name`, `agent` | an operator removed a concept from the Studio |
 | `:knowledge_retrieved` | `name`, `agent` | the model called `load_knowledge` — the adoption signal (retrieval calls per conversation), not the `<knowledge>` injection itself |
 | `:knowledge_backfilled` | `agent`, `sessions`, `concepts`, `conflicts`, `dropped{}` | the recovery re-scan (`insika knowledge:backfill`) finished replaying an agent's stored sessions |
+| `:retrieval_reranked` | `provider`, `candidate_count`, `selected_count` | an opt-in memory or knowledge rerank selected candidates; no candidate text |
+| `:provider_warning` | `provider`, `message` | a rerank failed or returned invalid indexes; fallback context remains available |
 
 `delivery_failed` and `breaker_open` are the two the operator config is pointed at
 (`alerts.webhook` on the profile): each only fires when something durable did
 not land. `:ttft` is additive debug, absent unless `INSIKA_TURN_TIMING` is set.
 
+Rerank requests have `operation: "rerank"` in native model diagnostics. Their
+reported cost is tracked separately from chat and included once in the total.
+If any rerank request lacks usage, the rerank and total cost remain unknown.
+Reported native rerank tokens, including reported failed attempts, count once
+against agent token ceilings and calendar budgets, separately from chat usage.
+Unknown token fields stay unknown. Diagnostics carry the task and captured turn.
+Candidate and selected counts measure narrowing, not recall: records excluded
+by the lexical index or recent-note window cannot appear in those counts.
+
 `:channel_delivered` is the one worth alerting on: a turn can be `:task_completed`
 and correct while the customer got nothing, because delivery is a separate,
 retried, out-of-band step. `status: "failed"` means the reply is sitting in the
 outbox and the customer is still waiting.
 
-Counts, ids and times only — never message content. The text lives in the
-transcript, which is the surface that is allowed to carry it.
+Operational audit events use metadata rather than transcript bodies. Presentation
+`:ui` events intentionally contain customer-facing card content; tool-call events
+can contain arguments. Do not treat the whole event stream as content-free.
 
 The bridge speaks the standard the market already runs on: point any OTLP backend
 at Insika and a real turn shows up as a full trace, next to counters and histograms
 you can chart without touching a span.
 
-**This page is a convention, not an integration.** Insika ships no dashboard, no
+**The OpenTelemetry section is a convention, not an integration.** Insika ships no OTEL dashboard, no
 backend config, no vendor file. It ships a stable set of attribute and instrument
 names, and the recipes below tell you what to chart against them — in whatever you
 already run.
 
+### Blocked tool calls
+
+`tool_blocked` carries `name`, `gate`, and `param`, with task/session correlation
+in event metadata. It never carries the rejected value. The `insika.tool.blocked`
+counter (unit `{call}`) uses the turn's agent/tenant/command labels plus
+`insika.tool` and `insika.gate`. Session traces keep the `gate` field alongside the
+masked result; `insika tools:report` lists blocked calls separately from errors.
+
 ## Contents
 
 - [Turning it on](#turning-it-on-opt-in-parity-when-off)
+- [Native model diagnostics](#native-model-diagnostics)
 - [Traces: the span reference](#traces-the-span-reference)
 - [Metrics: the instrument reference](#metrics-the-instrument-reference)
 - [Attribute reference](#attribute-reference)
@@ -112,12 +172,43 @@ stay on; `OTEL_METRIC_EXPORT_INTERVAL` (ms) sets the export period. If the metri
 SDK is not in the bundle at all, the bridge degrades to traces only rather than
 failing to boot.
 
-**Off (the default):** `Insika::Telemetry.setup` returns `nil`, the OTEL gems are
-**never loaded** (lazy `require`, like the LLM client), and nothing is
-instrumented — zero overhead. This is enforced by a test
+**Off (the default):** `Insika::Telemetry.setup` returns `nil` and the OTEL gems are
+**never loaded** (lazy `require`, like the LLM client). No OTEL spans or metrics
+are exported; native model diagnostics remain available in Studio. The lazy
+load is enforced by a test
 (`spec/insika/load_guard_spec.rb`: "require insika does not load
 OpenTelemetry").
 
+A failed export (a 404 from an ingest that wants a header, a wrong key) is
+logged to stderr as `ERROR -- otel: OpenTelemetry error: OTLP exporter received
+http.code=…`. `OTEL_EXPORTER_OTLP_HEADERS` is `key=value[,key=value]` and each
+value is URL-decoded: a `+` in a token becomes a space, so write it as `%2B`
+(and `%` as `%25`).
+
+## Native model diagnostics
+
+The Studio task page shows `llm_request` and `llm_usage` rows from RubyLLM. A
+request is one logical model operation, including RubyLLM's transport retries;
+each attempt has its own usage row. A successful task can therefore show a failed
+attempt followed by a successful attempt under one request. These are not extra
+task executions. Each request has a locally generated `request_id` that links its
+rows; it is not the provider's HTTP request ID. The rows show operation,
+provider, model, outcome, and five token buckets (input, output, cache read,
+cache write, thinking). Only the request has a measured duration. Attempts have
+no measured duration. Cost is RubyLLM's reported attempt cost, or the
+`INSIKA_MODEL_PRICING` estimate when RubyLLM reports none: unknown
+remains an em dash, while a reported zero remains zero. These diagnostic values
+do not add to turn usage, `insika.tokens`, or `insika.cost`.
+
+The diagnostic store retains the latest 200 events per task. Studio marks a
+truncated tail; deleting the task deletes its diagnostics. Insika stores and
+exports only allowlisted scalar fields: event type, timestamp, turn number,
+operation, provider, model, outcome, request ID, duration, exception class,
+token counts, and cost. It excludes prompts, messages, tool arguments, responses,
+URLs, exception messages, and arbitrary RubyLLM payload fields. This restriction
+applies to Insika's diagnostics; an application-supplied RubyLLM instrumenter
+still receives RubyLLM's original payload under the host's own policy.
+
 ## Traces: the span reference
 
 One span per **turn**, with **child spans** per tool, correlated by `task_id`.
@@ -128,16 +219,29 @@ Latency is the span duration, reconstructed from the events' real `at` timestamp
 | `insika.turn` | one per turn, opened on `task_started`, closed on the terminal event | root |
 | `insika.tool` | one per tool call, `tool_call` → `tool_result` (FIFO-correlated) | `insika.turn` |
 | `insika.data_tool` | one per data-tool call — point-in-time (the engine emits a single event) | `insika.turn` |
+| `insika.llm.request` | one per retained native request with measured duration, exported when the turn ends | `insika.turn` |
 
 A turn that ends with `task_failed` also carries the OTEL **error status**, with
-the failure message.
+the exception class (or `Task failed` when the class is unavailable), never the
+exception message. A failed native request uses the same class-only rule.
+
+Request spans carry `insika.llm.request_id`, `insika.model`, `insika.provider`,
+`insika.operation`, `insika.status`, `insika.llm.duration_ms`,
+`insika.llm.attempts`, and `insika.llm.failures`. Each retained attempt adds
+`insika.llm.attempt.N.status` and, when known,
+`insika.llm.attempt.N.input_tokens`, `output_tokens`, `cache_read_tokens`,
+`cache_write_tokens`, `thinking_tokens`, and `cost`. The `N` starts at 1. Unknown
+values are omitted. Request spans wait until the turn ends so usage emitted after
+request completion is included. When the 200-event buffer overflows,
+`insika.llm.truncated=true` appears on the turn and request spans: their attempt
+counts describe only the retained tail.
 
 ## Metrics: the instrument reference
 
 The same events feed instruments, so volume, latency, tokens and cost are chartable
 **without aggregating spans** — which not every backend does, and none does cheaply
-at retention. Metrics are recorded on the *terminal* event, so every point already
-knows its outcome.
+at retention. Turn metrics are recorded on the *terminal* event, when its outcome
+is known. Native attempt counters are recorded as usage arrives.
 
 | Instrument | Type | Unit | Recorded when |
 |------------|------|------|----------------|
@@ -145,11 +249,20 @@ knows its outcome.
 | `insika.turn.duration` | histogram | `s` | same, when both timestamps are known |
 | `insika.tokens` | counter | `{token}` | the turn reported usage |
 | `insika.cost` | counter | `{USD}` | the turn's model is priced (see below) |
+| `insika.tool.blocked` | counter | `{call}` | a `tool_blocked` gate refusal |
+| `insika.confirmation` | counter | `{hold}` | a customer confirmation is requested, confirmed, cancelled or expires (`insika.outcome`) |
 | `insika.tool.calls` | counter | `{call}` | a tool call completes |
 | `insika.tool.duration` | histogram | `s` | a `tool_call`/`tool_result` pair completes |
 | `insika.cache.hit_rate` | histogram | `%` | a turn reported billed prompt tokens (see below) |
 | `insika.tool.loop_intervened` | counter | `{intervention}` | the loop detector delivered its one-shot warning |
 | `insika.context.compacted` | counter | `{compaction}` | an in-session compaction was persisted (RFC-0044) |
+| `insika.llm.attempts` | counter | `{attempt}` | each native usage event, including retries |
+| `insika.llm.failures` | counter | `{attempt}` | each native usage event with failed status |
+
+The native attempt counters count every received attempt, even when the span
+buffer has discarded earlier rows. They use the turn's agent/tenant/command
+labels plus `insika.model`, `insika.provider`, `insika.operation`, and
+`insika.status`; neither task nor request ID becomes a metric label.
 
 `insika.tool.duration` is deliberately **not** recorded for data-tools: those are a
 single point-in-time event, so there is no measured duration to report. A tool left
@@ -189,10 +302,11 @@ metric attribute with per-turn cardinality is how you destroy a metrics backend.
 | `insika.status` | string | `turn` | turn instruments | `ok` / `error` / `cancelled` / `abandoned` |
 | `insika.model` | string | `turn` | all except tool | model id the provider reported |
 | `insika.model_source` | string | `turn` | — | which config layer resolved the model (chat / agent / model / global) |
-| `insika.tokens.input` | int | `turn` | — | input tokens (**includes** the cached ones) |
+| `insika.provider` | string | `turn` | — | provider of the turn's final model selection (the fallback's, when one answered) |
+| `insika.tokens.input` | int | `turn` | — | non-cached input tokens |
 | `insika.tokens.output` | int | `turn` | — | output tokens |
-| `insika.tokens.total` | int | `turn` | — | input + output |
-| `insika.tokens.cached` | int | `turn` | — | cache **reads**, a subset of `tokens.input` |
+| `insika.tokens.total` | int | `turn` | — | non-cached input + output subtotal; cache buckets are separate |
+| `insika.tokens.cached` | int | `turn` | — | cache **reads**, separate from `tokens.input` |
 | `insika.tokens.cache_creation` | int | `turn` | — | cache **writes**, *not* inside `tokens.input` |
 | `insika.token.type` | string | — | `insika.tokens` | `input` / `output` / `cached` / `cache_creation` |
 | `insika.cost.usd` | double | `turn` | — | estimated cost of the turn (span attribute; the metric is the `insika.cost` counter) |
@@ -219,12 +333,27 @@ attributes are there for it.
 
 ## Estimated cost
 
-Insika ships **no prices**. They change weekly, differ per contract and per region,
-and a stale table inside the engine would be worse than no number. You declare the
-rates; Insika multiplies. Unset, no cost is reported anywhere.
+RubyLLM supplies attempt-time usage and cost. Insika aggregates native chat usage
+across tool rounds and retries, including recorded usage before a failure, and
+exports the recorded `cost_usd`. Unknown costs remain unknown, not zero.
+Insika maintains no competing provider price catalog.
+
+Internal token buckets are disjoint. The `total_tokens` field retains its historical
+non-cached input/output subtotal; budgets and evals add the cache buckets once.
+The Responses HTTP adapter projects cache-inclusive `input_tokens` and `total_tokens`.
 
-Set `INSIKA_MODEL_PRICING` to a JSON object of model id → rates in **USD per
-million tokens**:
+RubyLLM prices from its model catalog. Insika keeps that catalog in its store
+and refreshes it every `INSIKA_MODEL_REFRESH_HOURS` (default 24; 0 keeps the
+gem's bundled catalog): one worker downloads it, the others reload it, and a
+restart starts from the last download instead of the catalog shipped with the gem.
+
+`INSIKA_MODEL_PRICING` optionally fills in cost where RubyLLM reports none (a
+model missing from its registry, a negotiated contract) — everywhere: Studio
+Models, task traces, turn usage and telemetry. RubyLLM's cost always wins, so the
+table never goes stale against a priced model. Rates are a JSON object of model
+id to **USD per million tokens**.
+Each request is priced under its own model, so a turn that ran a failed primary,
+a fallback and the router costs each at its own rate:
 
 ```bash
 INSIKA_MODEL_PRICING='{
@@ -237,13 +366,17 @@ INSIKA_MODEL_PRICING='{
   `provider/` prefix — `deepseek/deepseek-v4-flash` and `deepseek-v4-flash` both hit the
   same entry.
 - `input` / `output` are required (one of them is enough for the entry to load).
-- `cached_input`, when given, bills cache **reads** at that rate and subtracts them
-  from the fresh input. Omit it and cached tokens simply stay at the input rate.
+- `cached_input`, when given, bills cache **reads** separately at that rate.
+  Omit it and cache reads use the input rate; they are never subtracted from fresh input.
 - `cache_write`, when given, bills cache **creation** tokens at that rate. Omit it
   and they are billed at the input rate.
-- An **unpriced model reports nothing** — no attribute, no metric point. A missing
-  price is not a zero cost, and a dashboard should show the gap.
-- A malformed table degrades to "no cost". Telemetry config can never stop a boot.
+- A model RubyLLM prices never reads the table. If neither supplies a cost,
+  there is no attribute or metric point. A missing price is not zero.
+- A malformed table disables the override. Telemetry config cannot stop a boot.
+
+Routing (`operation: "routing"`) and summarizing compaction
+(`operation: "compaction"`) requests are recorded like the chat's, so they show
+on Studio Models next to it.
 
 The number is an **estimate for trend and attribution**, not a bill. Reconcile
 against your provider's invoice, never the other way round.
@@ -291,7 +424,10 @@ climbing while `input` stays flat.
 `insika.tokens` filtered to `insika.token.type="cached"` over the same counter
 filtered to `input`. This is the number that moves your bill. For the per-turn
 distribution (does every turn hit, or do fleet averages hide cold agents?), chart
-`insika.cache.hit_rate` — p50 by `insika.agent`; a healthy agent sits near 100.
+`insika.cache.hit_rate` — p50 by `insika.agent`. Compare with that agent's baseline:
+cache eligibility, expiry and changing history affect the ratio. The
+[identity-prefix fingerprint](CONTEXT.md#the-observable-cache-fingerprints-and-the-invalidation-reason)
+explains identity/tool-schema changes; volatile category digests are separate.
 
 **Loop interventions**
 `insika.tool.loop_intervened`, rate, grouped by `insika.agent` and `insika.tool`.
@@ -374,7 +510,7 @@ Set the OTEL envs on the deployment and every worker exports to your collector:
 INSIKA_OTEL=1
 OTEL_EXPORTER_OTLP_ENDPOINT=https://<your-collector>:4318
 OTEL_SERVICE_NAME=insika        # optional; names the service
-INSIKA_MODEL_PRICING='{...}'    # optional; unlocks the cost attribute + counter
+INSIKA_MODEL_PRICING='{...}'    # optional negotiated-rate override for telemetry
 # plus any standard OTEL_EXPORTER_OTLP_HEADERS your backend needs
 ```