insika 0.0.1 → 0.2.0

data/docs/SKILLS.md

@@ -0,0 +1,284 @@
+---
+title: Skills
+parent: Build an agent
+nav_order: 3
+permalink: /skills/
+---
+
+# Skills
+
+A **skill** is a named playbook an agent loads **on demand**. It is a directory
+with a `SKILL.md` file: YAML frontmatter (`name` + `description`) followed by a
+Markdown body. The agent always sees the *name and description* of each skill it
+is allowed; it pulls the full *body* into context only when a turn actually calls
+for it. That is **progressive loading** — an agent can "know" twenty skills exist
+while paying for the text of only the ones it opens.
+
+See [`examples/skills/`](https://github.com/guizaols/insika/tree/main/examples/skills/) for a runnable one.
+
+## Format
+
+```markdown
+---
+name: refunds                       # must equal the directory name
+description: When and how to process a refund   # the Level-1 trigger text
+triggers: [refund, money back]      # optional: deterministic activation (below)
+companions: [refund-policy]         # optional: skills this one cannot work without
+---
+
+<the full playbook body — loaded only on demand>
+```
+
+Frontmatter is parsed tolerantly. A skill's canonical name is its directory name,
+which the `name:` field must match.
+
+## Progressive loading: two levels
+
+- **Level 1 — metadata only.** A context provider injects an `<available_skills>`
+  list into the system prompt — the name, the one-line description and the
+  `triggers:` of every allowed skill — telling the model to load a skill before
+  acting on it. Cheap, and always present for allowed skills. **This is the routing
+  table, and it is generated:** it cannot disagree with the allowlist, so do not
+  hand-write one in a prompt file (see [Drift guards](#drift-guards)).
+- **Level 2 — the full body.** A built-in `load_skill` tool returns the skill body
+  on demand. It enforces the agent's skill allowlist and is wired **automatically**
+  whenever the agent has any allowed skills — you do not add it to `tools_allow`.
+- **Deterministic activation — `triggers:`.** When the user message contains one
+  of the skill's `triggers`, the body is injected for that turn — no model
+  decision, no `load_skill` call. Only matched skills, only that turn. Use it for
+  skills that MUST fire on known phrases; model loading stays as the fallback for
+  everything else. Matching is on **whole words**, case-insensitive and
+  accent-folded: `presente` fires on *um presente* and *presénte*, never inside
+  *apresente*.
+
+The Level-1 list is budgeted like any other context fragment
+(see [Context](CONTEXT.md)); the Level-2 body only costs tokens on the turns that
+open it.
+
+### Only trigger a skill that can finish the turn alone
+
+A `triggers:` match is not a hint — the body lands in the prompt with the
+authority of an instruction. So put triggers only on a skill that is
+**self-sufficient** for the turn it fires on.
+
+The failure mode is counter-intuitive: injecting a skill that is only *part* of
+the answer is **worse than injecting nothing**. Give the model a reference table
+whose procedure lives in a companion skill, and it now holds a plausible
+half-recipe — so it never calls `load_skill` for the other half, and improvises
+the missing part. A precise trigger on the wrong kind of skill still breaks the
+turn.
+
+Reference tables, vocabularies and lookup maps are the skills to leave on
+level 1. Whole procedures ("run this journey", "recover from this error") are the
+ones worth triggering.
+
+## Always-on skills: `skills_eager`
+
+A skill that every turn needs — output format, the marker vocabulary, how to
+recover from a failed tool — should not depend on the model choosing to load it.
+Name it on the **agent** and its body is in the prompt on every turn:
+
+```ruby
+Insika.agent("consultant") do
+  skills_eager "recommendation-formatting", "tool-error-recovery"
+  # skills_eager        # or: every allowed skill (a corpus that fits the budget)
+  # skills_eager false  # or: none — the default
+end
+```
+
+An eager skill also **leaves level 1**: it is absent from `<available_skills>` and
+`load_skill` refuses to serve it. There is no level 2 left to fetch, and a catalog
+pointing at a body already in the prompt only invites a call that pays for a
+duplicate.
+
+### Why the agent decides, and not the skill
+
+Eagerness used to be an `eager: true` key in the `SKILL.md` frontmatter. That put
+the decision on the wrong object: **skills are shared.** `escalation-to-human`,
+`recommendation-formatting` and `tool-error-recovery` each sit in several agents'
+allowlists, and one flag on the skill forced one decision onto every agent holding
+it — with no way to be always-on for the agent that needs it and discretionary for
+the one that does not.
+
+`skills_eager` is a per-agent list, so the same shared skill can be both. The
+frontmatter key is **ignored** — `insika doctor` flags any skill still carrying it,
+and names the agent setting that replaced it.
+
+A name that is not in the agent's `skills` allowlist is a no-op (eagerness is
+intersected with what the agent is allowed to see); `doctor` flags that too.
+
+### Keep the discretionary skills on the load path
+
+Making everything eager is a trap, and the reason is not the tokens: **it costs you
+the signal**. When every body is present on every turn, "which skills were active"
+is always "all of them", and you can no longer tell which one the model reached for.
+The `load_skill` call is the only record of that choice — it is a persisted tool
+message, so it shows up in the transcript on its own.
+
+So the split is: **eager for what the turn always needs, `load_skill` for what the
+turn might need.** The second group is where you want the model's choice on the
+record, because that is the group where a wrong choice is worth seeing.
+
+The token trade is real but smaller than it looks: eager bodies sit at a fixed
+position ahead of the history, so they belong to the **cacheable prefix**, and they
+are still evictable under budget pressure, unlike the pinned identity. Conditional
+injection is what breaks that prefix, on exactly the turns it fires.
+
+## Seeing which skills were active, and why
+
+The load path is legible for free: `load_skill` is a tool, so the call is a
+persisted message and shows up in the transcript on its own. The deterministic paths
+are not a call, so the engine reports them itself — **with a reason per skill**:
+
+| reason | what it means |
+|---|---|
+| `eager` | the agent's `skills_eager` names it, so every turn gets it |
+| `trigger:<phrase>` | this message matched that `triggers:` entry — the phrase as **authored**, so you can find the line to edit |
+| `pack` | a plugin's own context provider supplied the body |
+
+Where it shows up, per turn, in the Studio session screen:
+
+- an **activation card in the transcript thread**, placed at the top of its turn and
+  in the same visual language as a tool result, so a context-injected skill and a
+  model-loaded one read the same way;
+- the **Context card**, next to that category's token count, for the after-the-fact
+  audit;
+- the `skill_activated` **event** (`skills: [{name, reason}]`, `source: "context"`),
+  with full task/session correlation.
+
+All three are computed from what actually reached the prompt **after the budget
+cut**: a body the budget evicted is reported as an eviction, never as an activation.
+A turn that mixes both paths is labelled `mixed`, and each line keeps its own reason.
+
+## Where skills live: the store, over a disk seed
+
+- Skills live as **rows in SQLite** — one row per skill, holding the entire
+  `SKILL.md`, versioned (recent revisions are retained).
+- On-disk `SKILL.md` files in configured roots are loaded as a **seed**, then the
+  store is **overlaid on top — the store wins**. A reload swaps the index
+  atomically, so edits take effect **without a restart**.
+
+> ⚠️ **Committing a `.md` file to the repo does not make a skill show up on a
+> running deployment.** The on-disk file is only a seed for a *fresh* box; a live
+> box serves the store, and a deploy does not rewrite the database. Editing is a
+> runtime operation (Studio / API / DSL), not a commit. See
+> [Context](CONTEXT.md#the-volume).
+
+## Pairs that must not break: `companions:`
+
+Injecting *part* of an answer is worse than injecting nothing. Give the model a line
+map whose query-construction rules live in another skill and it holds a plausible
+half-recipe — so it never calls `load_skill` for the other half, and improvises the
+missing part. Measured on a real pack: the map arrived by trigger, the rules did not,
+and the searches came out malformed. Twice.
+
+Declare the dependency and it travels with whatever brought it — a trigger match, the
+agent's eager set, or a `load_skill` call (which returns both bodies in the one call):
+
+```yaml
+companions: [query-construction]
+```
+
+Two deliberate limits:
+
+- **One level, no transitive walk.** A cycle would be a hang and a chain a budget
+  blowout, and "cannot work without" is a direct relationship.
+- **Never widens an allowlist.** A companion the agent is not allowed to load is
+  simply absent; `insika doctor` flags the declaration instead.
+
+## Specializing a shared skill for one agent
+
+Skills are shared on purpose: `escalation-to-human` belongs in several agents'
+allowlists. But sometimes one agent needs a different version of the same skill —
+its own return policy, its own store name — and forking it under a second name
+throws the sharing away and leaves two things to keep in step.
+
+So the store has a second scope, and resolution is a **precedence chain** with one
+more dimension:
+
+```
+for agent A, skill <name>:   (A, <name>) in the agent scope,
+                             then <name> in the shared scope
+```
+
+Three cases fall out of that one rule:
+
+| case | what exists in the store |
+|---|---|
+| **shared** | only the shared record — every agent gets the same body |
+| **override** | both — the agent's wins, for that agent only |
+| **agent-private** | only the agent record — invisible elsewhere, and the name may collide freely |
+
+The **name never changes.** An override keeps saying `name: escalation-to-human`
+inside, because it *is* that skill, specialized; the allowlist, the `<available_skills>`
+list, `load_skill` and the activation card all keep showing the bare name. What
+decides which record you get is its **position in the store**, never the frontmatter —
+otherwise an override would clobber the shared skill for everybody.
+
+Write one with `agent:`, and remove it the same way (which un-specializes, leaving
+the shared skill in place):
+
+```ruby
+dispatch(:write_skill,  { name: "escalation-to-human", agent: "store-cacau", content: md })
+dispatch(:delete_skill, { name: "escalation-to-human", agent: "store-cacau" })
+```
+
+In the Studio: **Skills → specialize for this agent**, which seeds the override from
+the shared body.
+
+## Making a new skill "show up"
+
+For an agent to actually use a skill, **both** conditions must hold:
+
+1. The skill exists as a row in the store (a `SKILL.md` written into it).
+2. The skill is in that agent's `skills` allowlist
+   (`nil` = all, `[]` = none, `[names]` = those — see
+   [Agents](AGENTS.md#the-allowlist-convention)).
+
+Miss either and the skill is invisible: not in the store → nothing to load; not
+in the allowlist → the model never sees it in `<available_skills>`.
+
+Two ways to satisfy both:
+
+- **Via a definition/pack import.** The import writes each skill directory into
+  the store and sets the agent's `skills` allowlist **authoritatively** from the
+  skills present — so a re-import that drops a skill also removes it. Keep the
+  definition complete.
+- **Directly (Studio / API / DSL).** Write the skill (upserts the row and reloads
+  the catalog atomically — live immediately), then attach it to the agent(s) by
+  adding its name to the `skills` allowlist.
+
+## Verify it showed up
+
+- In the Studio agent's Skills section, the skill is listed and allowed.
+- In a turn, the skill appears in the `<available_skills>` list and the model can
+  `load_skill` it (the body loads on demand).
+- If the model never mentions it → check the allowlist (condition 2). If
+  `load_skill` errors → the store row is missing or misnamed (condition 1); the
+  `name:` frontmatter must equal the directory name.
+
+## Drift guards
+
+A skill catalog drifts against the prose that routes to it, and every way it happened
+on the pilot was silent — found by reading a customer conversation days later. So the
+routing table is **generated** (above), and `insika doctor` reports the residue the
+generator cannot remove. Every check takes mechanical inputs only — names, allowlists,
+agent identities — because one false positive is enough for an operator to stop
+reading the doctor:
+
+| finding | what it means |
+|---|---|
+| a prompt file names a skill outside that agent's allowlist | leftover hand-written routing: the model is told to use something it cannot load |
+| a shared skill's body names one of its own holders | specialized text in shared clothing — the other holders are served that store's policy as their own. Specialize it instead |
+| a body references another catalog skill without declaring it a companion | the pair can still arrive apart |
+| a declared companion is outside an agent's allowlist | the pair cannot travel for that agent, and the engine will not widen the allowlist |
+| a skill still declares `eager:` in its frontmatter | the key is ignored; the decision moved to the agent |
+| an agent marks a skill eager that it does not allow | the name is a no-op |
+
+## See also
+
+- [Context](CONTEXT.md) — how the skills list is budgeted into a turn.
+- [Agents](AGENTS.md) — the skills allowlist.
+- [Tools](TOOLS.md) — `load_skill` and deferred-tool progressive disclosure.
+- [Plugins](PLUGINS.md) — shipping skills inside a plugin, and the two extension tiers.
+- [`examples/skills/`](https://github.com/guizaols/insika/tree/main/examples/skills/) — progressive loading, runnable.

data/docs/TOOLS.md

@@ -0,0 +1,302 @@
+---
+title: Tools
+parent: Build an agent
+nav_order: 2
+permalink: /tools/
+---
+
+# Tools
+
+A **tool** is a function the model can call inside a turn. Insika has three
+kinds, and the distinction that matters is **who can change one at runtime**:
+
+| | **Code tool** | **Data tool** | **MCP tool** |
+|---|---|---|---|
+| What | a Ruby class (`< RubyLLM::Tool`) | an HTTP call described by config, no Ruby | an MCP server's tool, ingested |
+| Lives | in the deployment image | as a row in SQLite | as data-tool rows in SQLite |
+| Editable at runtime | no (shipped in the image) | **yes** (DSL / API / manifest / Studio) | **yes** (re-ingest) |
+| Reach for it when | logic must run in-process (file edit, shell, subagent) | calling an external HTTP API | adopting a whole MCP toolset at once |
+
+**MCP tools are not a separate runtime type.** An MCP ingestor discovers an MCP
+server's tools and turns each into an HTTP **data tool** that posts a JSON-RPC
+`tools/call`, tagged with a `group` naming the source instance. (Only
+HTTP-transport MCP servers are ingestible; stdio is rejected.)
+
+Code tools **win name collisions** — you cannot register a data tool whose name
+shadows a code tool.
+
+## Data tools: a tool is a row
+
+A data tool is defined entirely by config — this is the operator-facing kind, and
+the one you create and change without a rebuild. See
+[`examples/data-tool/`](https://github.com/guizaols/insika/tree/main/examples/data-tool/) for a runnable one.
+
+```jsonc
+{
+  "name": "search_products",              // /\A[a-z][a-z0-9_]*\z/
+  "description": "Search the catalog",     // required — this is what the model reads
+  "parameters": { /* JSON Schema, safe subset */ },
+  "request": {
+    "method": "POST",                       // GET | HEAD | POST | PUT | PATCH | DELETE
+    "url": "https://api.example.com/search",
+    "headers": { "X-Session": "{{ctx.chat_id}}",
+                 "Authorization": "Bearer {{secret.api_token}}" },
+    "query": {}, "body": "…"
+  },
+  "response": { "extract": "json_path", "path": "$.results" },
+  "secret_headers": ["Authorization"],
+  "side_effect": true, "timeout": 30, "group": "catalog", "tags": []
+}
+```
+
+### Parameters: the schema is the contract
+
+`parameters` is **JSON Schema**, and it reaches the provider verbatim — it is the only
+thing telling the model what shape to send. The engine never fills a gap in it.
+
+For simple params there is a flat sugar (what the Studio's textarea and a hand-written
+manifest accept), one line per param:
+
+```
+cep      | string       | required | The ZIP code to look up
+tags     | array:string | optional | Labels to filter by
+quantity | integer      | required | How many
+```
+
+Types are `string`, `number`, `integer`, `boolean`, and `array:<scalar>` for a list.
+There is **no bare `array`**: a list without an item type is an incomplete declaration,
+and it is rejected instead of being guessed at. A list of **objects** — the common
+`[{query, filters}]` shape — cannot be written in the flat form at all; write the JSON
+Schema, which is what the Studio field reads when the text starts with `{`:
+
+```jsonc
+{ "type": "object",
+  "properties": {
+    "query_filter_pairs": {
+      "type": "array",
+      "items": { "type": "object",
+                 "properties": { "query":   { "type": "string" },
+                                 "filters": { "type": "object", "properties": {} } },
+                 "required": ["query"] } } },
+  "required": ["query_filter_pairs"] }
+```
+
+**Arguments are checked against the schema at call time.** A call the schema does not
+allow never becomes a request: it returns an `{ error: … }` naming the path
+(`query_filter_pairs[0]: expected an object, got a string`), which the model reads and
+retries against. Structure is strict; a scalar may arrive in its lossless string form
+(`"2"`, `"true"`) and is never coerced — what the model sent is what the request carries.
+
+**Placeholders** are resolved at turn time:
+
+- `{{param}}` — a declared top-level parameter, filled from the model's call.
+- `{{ctx.*}}` — turn context set **server-side, never by the model**: a closed set
+  of `chat_id`, `store_id`, `agent_id`, `tenant`. This is how a tool knows *which*
+  session/agent it is acting for without trusting the model.
+- `{{secret.*}}` — allowed **only** inside a header named in `secret_headers`.
+  A secret placeholder anywhere else is rejected (it would leak unmasked). The
+  real secret value is injected at provision time and never lives on disk.
+
+**Validation** happens on ingestion. Common rejections:
+
+- `url` must be `http`/`https` — anything else is a 422.
+- `parameters` is a **safe subset** of JSON Schema
+  (`object/array/string/number/integer/boolean`); `oneOf`/`anyOf`/`allOf`/`$ref`/
+  `if`/`then`/`else` are forbidden (not every provider supports them).
+- `side_effect` defaults from the method (GET/HEAD → false, else true) and drives
+  checkpoint/replay semantics (a completed side-effecting tool is not re-run on
+  resume — see [Architecture](ARCHITECTURE.md#durability-checkpoints-and-resume)).
+
+### `halt_when`: when the answer is already out
+
+Some tools do the work **and** deliver the news. A backend that subscribes a customer
+and sends its own confirmation over the channel has already said everything there is to
+say: if the model then writes "all set, you're subscribed!", the person gets the message
+twice. The usual patch is to ask the model to stay quiet in the tool's instructions —
+which works until the turn it doesn't, and the failure lands in front of a customer.
+
+`halt_when` moves the decision from the prompt to the engine. It reads the tool's own
+**response**, and when it matches, the turn ends right there — no further provider call:
+
+```jsonc
+{ "name": "subscribe_to_learning_path",
+  "request": { "method": "POST", "url": "https://app.example/subscribe" },
+  "halt_when": { "json_path": "tool_result.status", "equals": ["SUBSCRIBED"] } }
+```
+
+By **result**, not by tool. The same call that goes silent on `SUBSCRIBED` must let the
+model explain a `SUBSCRIPTION_FAILED` ("you are already enrolled") — one tool, two
+endings, decided by what the backend actually returned.
+
+- `json_path` is a dotted path into the parsed response body, and `equals` a list of
+  values compared **as strings** (a status is a label; JSON types vary by backend).
+- It reads the **body**, independently of `response.extract` — which shapes what the
+  *model* sees, not what the engine decides on.
+- It only fires on a **2xx**. An error response that happens to carry the value is a
+  failure, and a failure must reach the model.
+- A non-JSON body or a missing path simply does not match: a turn never ends on a guess.
+
+A halted turn keeps whatever the model had already streamed *before* the call (usually a
+"let me get that for you") and adds nothing after it.
+
+#### `say`: what the customer gets when the model wrote nothing first
+
+The model does not always introduce the call. Then the lead-in is empty, and the turn
+used to publish **nothing** — measured on a real store, two escalation turns in a row
+delivered silence to the customer. `say` is the answer for that turn, and only that
+turn: when there **is** a lead-in it still wins, because two messages for one
+escalation is what `halt_when` exists to prevent.
+
+It cannot be inferred. `json_path` + `equals` cannot supply it either — the matched
+value is by definition one of the `equals` tokens, so publishing it would ship
+`SUBSCRIBED` to a person as often as it ships a sentence. So you name it, in one of two
+shapes:
+
+```jsonc
+// the sentence the backend itself returned
+"halt_when": { "json_path": "tool_result.status", "equals": ["SUBSCRIBED"],
+               "say": { "json_path": "tool_result.message" } }
+
+// a literal the CHANNEL knows how to resolve
+"halt_when": { "json_path": "tool_result", "equals": ["…"],
+               "say": { "text": "CALL_SUPPORT" } }
+```
+
+The literal form replaces the usual workaround: instructing the model to emit a control
+token and parsing it downstream. The token now comes from the **tool's contract**,
+deterministically, instead of depending on the model complying with a sentence in a
+prompt.
+
+- Exactly one of `text` or `json_path` — two answers to "what does the customer get" is
+  a configuration nobody can read, so both (or neither) is refused at load.
+- A `json_path` that does not resolve to a **string** publishes nothing: a hash or a
+  number reaching a customer as the answer is never what someone meant.
+- Omit `say` and the behaviour is unchanged — a halt with no lead-in completes empty,
+  which is what a channel consumer drops.
+
+`say` is declared on the **tool**, because what a backend answers is a property of that
+backend, not of whoever calls it. Every agent sharing the tool gets the same value.
+
+> The Studio's tool editor does not render this field (nor `group`/`tags`), but a save
+> there **preserves** it — the form carries the stored values through instead of
+> replacing the record with only what it shows.
+
+## Registering a tool
+
+A tool appears in the Studio panel and enters an agent's tool-loop when it is
+**registered** in the catalog **and** allowed by the agent's policy allowlist.
+Four ways to write a data tool into the store — all **hot** (registry and catalog
+reload, no restart):
+
+1. **DSL** — `data_tool(name:, …)` in a `Insika.agent { … }` block.
+2. **Studio** — the Tools panel editor.
+3. **Manifest** — `POST /v1/tools/manifest`. Partial failure is isolated: one
+   malformed tool becomes an `errors[]` entry; only a structural manifest error
+   fails the whole request. The response reports `{ version, created, updated, errors }`.
+4. **MCP ingestion** — import a server; each of its tools becomes a data tool.
+
+### The one gotcha: env templating is manifest-only
+
+`{{env.*}}` (and `{{secret.*}}`) are substituted **at ingestion, on the manifest
+path**. Other write paths do **not** resolve `{{env.*}}` — a literal
+`{{env.API_URL}}` there fails the `http`/`https` URL check and 422s. Rule:
+**manifest tools may template the URL with `{{env.*}}`; tools written any other
+way must ship a literal URL.** `{{ctx.*}}` and `{{param}}` work everywhere (they
+resolve at turn time, not ingestion).
+
+## Making it appear — and enter the tool-loop
+
+1. **Panel visibility** = registered in the catalog. Data tools are marked
+   editable; code tools are allow/deny only.
+2. **Per-agent exposure** is set from the same panel, or by the agent's allowlist.
+3. **Entering the tool-loop** is decided by the **policy allowlist**, not by tool
+   type: deny wins, otherwise the agent sees `tools_allow ∪ tools_allow_groups`
+   (or all, when both are absent). See [Agents](AGENTS.md#the-allowlist-convention).
+4. **Deferred tools** (`tools_deferred`) are *not* offered directly — they appear
+   as a short "available tools" list and the model must call `tool_search` to
+   enable one. This is progressive disclosure for large toolsets — see
+   [Context](CONTEXT.md).
+
+## Parallel tool calls
+
+A model can ask for several tools in one step. By default the engine runs them one
+at a time. Set `limits[:tool_concurrency]` above 1 (see
+[Agents](AGENTS.md#tool_concurrency--parallel-tool-calls)) and the calls in that
+batch run concurrently, **at most N in flight**, on the turn's own reactor — so
+the wall-clock of a batch of slow data tools approaches the slowest call rather
+than their sum. The cap covers every enveloped tool of the turn, including the
+ones `tool_search` promotes mid-turn.
+
+It applies only to what the *model* fans out. Two primitives already parallelize
+deterministically and are unaffected: `spawn_subagents` (capped at 8 children) and
+`Insika::Tools::Concurrency.gather` (fan-out inside one tool). System tools —
+`tool_search`, `load_skill`, `remember`, `spawn_subagent` — are not enveloped and
+so are not gated by the cap; they are trivial or capped on their own.
+
+Turning it on changes three things, all of them worth knowing before you do:
+
+- **`max_tool_calls` becomes approximate.** The limit is checked per call, but a
+  call that trips it does not stop its siblings — the whole batch finishes and the
+  turn then fails. With a cap of 4, up to 3 extra tools may have executed. The turn
+  still fails at the right boundary; the count is just no longer exact.
+- **The transcript records results in completion order.** Providers key results by
+  `tool_call_id`, so the wire stays valid and persistence is faithful to what was
+  sent — but a replayed transcript no longer reads in call order.
+- **`turn_timeout` can overrun by up to `tool_timeout`.** A turn deadline does not
+  cancel a tool call already in flight in a sibling fiber; it waits for it. Each
+  call is still bounded by its own `tool_timeout`, which is what bounds the
+  overrun. Serial execution is unaffected (there, the deadline lands directly in
+  the fiber running the tool).
+
+Approvals and concurrency are mutually exclusive per turn — the approval gate wins
+and the turn goes serial. That is a deadlock avoided, not a preference.
+
+## Egress: the SSRF guard (and its silent failure)
+
+Data tools make outbound HTTP, so every call passes through the **EgressGuard**, a
+Server-Side Request Forgery defense. The default posture is **strict: public
+`https` only.** Three env vars widen it:
+
+| Env | Effect |
+|-----|--------|
+| `INSIKA_EGRESS_HOSTS` | allowlist of hosts (CSV). The safe way to permit a specific backend. |
+| `INSIKA_EGRESS_ALLOW_HTTP=1` | permit plain `http` — **loopback dev only** |
+| `INSIKA_EGRESS_ALLOW_PRIVATE=1` | permit private/loopback IPs — **dev only** |
+
+> ⚠️ **Egress failures are silent.** When a tool targets a blocked host (e.g. a
+> plain-`http` localhost backend without the opt-ins), the guard turns the block
+> into a `{ error: … }` returned **to the model** — the request never leaves the
+> process, yet the stream still emits a tool call, so the model narrates a
+> plausible failure and the conversation *looks* like it worked. You will not see
+> an exception.
+>
+> **Always verify by the trace, never by the reply:** open the Studio session
+> viewer — a healthy call shows the request, args, and the backend's `200`; a
+> missing or errored call is almost always egress (host not in the allowlist, or
+> `http`/private without the opt-in).
+
+Egress is **orthogonal** to registration and allowlisting: a tool can be
+registered, allowed, offered to the model, and still blocked at call time.
+
+## Troubleshooting: "the tool is missing"
+
+Work down this checklist:
+
+1. **Registered?** Is it in the catalog (Studio Tools panel)? If not, the write
+   or import failed — check the manifest `errors[]`, and run `insika doctor`: a stored
+   definition that no longer validates is dropped from the catalog, and the
+   `data-tools` check is the only place that says so.
+2. **Allowed for this agent?** In `tools_allow` (or an allowed group), and not in
+   `tools_deny`?
+3. **Egress?** If it *appears and is called* but "fails", open the trace — a
+   blocked call is ~99% egress.
+4. **URL literal?** For non-manifest tools, an unresolved `{{env.*}}` would have
+   422'd at import — re-check the definition.
+
+## See also
+
+- [Agents](AGENTS.md) — allowlists, groups, and per-agent tool exposure.
+- [Plugins](PLUGINS.md) — where a code tool comes from, and how to package one.
+- [Security](SECURITY.md) — egress, sandbox, and approval gating together.
+- [Architecture](ARCHITECTURE.md) — the tool-loop and side-effect checkpointing.
+- [`examples/data-tool/`](https://github.com/guizaols/insika/tree/main/examples/data-tool/) — a runnable data tool + the egress note.