insika 0.2.0 → 0.7.0

data/docs/DEPLOY.md

@@ -37,7 +37,9 @@ The image boots **N Falcon worker processes over one SQLite file**
 knob**: it decides which engine semantics hold cluster-wide and which are
 per-worker. The default is 1 because the per-worker semantics are the product
 — the queue modes (`collect`/`steer`) and FIFO ordering need one
-session actor; raise N only with sticky routing per session in front. The
+session actor; raise N only with sticky routing per session in front. It is
+also a [Soak](SOAK.md) precondition: with more than one worker the hourly RSS
+series is a random process per sample, and a soak refuses to start. The
 contract:
 
 > Everything here describes N workers of **one** deployment — one graph, replicated.
@@ -55,7 +57,14 @@ contract:
    worker that holds the session's actor. The engine does **not** promise them
    across workers. A deploy that needs those semantics for a session must route
    that session's traffic to one worker (sticky routing) — or accept per-worker
-   best-effort.
+   best-effort. **On Railway there is no sticky-routing option at any layer** —
+   confirmed against Railway's own docs, which distribute traffic randomly and
+   explicitly do not support sticky sessions — so N>1 there is not "per-worker
+   best-effort," it is a guaranteed cross-session leak the first time two
+   requests for the same session land on different workers. See the Railway
+   section below; `insika doctor` errors on `WEB_CONCURRENCY>1` there — unless
+   [`insika-router`](ROUTER.md), the session-sticky proxy, is in front,
+   which is what makes N>1 safe on Railway too (N local workers behind it).
 3. **Recovery is part of boot, in every wiring.** Every worker boots through
    `Server::Boot`, which runs recovery **before the listen**. The per-record
    sweeps (undelivered outbox records, undelivered delegation results) run in
@@ -112,10 +121,11 @@ this section is the single source of truth for what changing it means.
 | `DEEPSEEK_MODEL` | `deepseek-v4-flash` | model |
 | `CONSUMER_INTERNAL_URL` | — | base URL for data-tools calling back a consumer's internal API (see below) |
 | `INSIKA_EGRESS_HOSTS` | — | outbound host allowlist (SSRF guard) |
-| `INSIKA_EGRESS_ALLOW_HTTP` / `_ALLOW_PRIVATE` | off | for `http`/loopback callbacks only (**never in cloud**) |
+| `INSIKA_EGRESS_ALLOW_HTTP` / `_ALLOW_PRIVATE` | off | for `http`/loopback callbacks only (**never in cloud**). Also opens inbound **media** fetches (audio/image URLs) to `http`/private targets — a local run serving media over `http://` needs it |
 | `INSIKA_RELAY_TOKEN` | — | **mounts the relay channel** at `POST /channels/relay/events`, and is the Bearer it requires. Empty = the route does not exist (`404`). See [Channels](CHANNELS.md) |
 | `INSIKA_RELAY_DELIVER_URL` | — | your callback; the engine POSTs each reply there. Goes through the egress guard |
 | `INSIKA_RELAY_DELIVER_TOKEN` | — | Bearer the engine sends **to** your callback (optional) |
+| `INSIKA_RELAY_DELIVERY` | `at_end` | how the relay flushes the outbox: `at_end` (one POST) or `progressive` (one POST per balloon — [delivery policy](CHANNELS.md#delivery-policy)) |
 | `INSIKA_WIDGET_ORIGINS` | — | exact-match origins allowed to embed the [web widget](CHANNELS.md#the-web-widget), comma-separated. No wildcards. **Half the switch**: with `INSIKA_WIDGET_AGENTS` unset, nothing is mounted (`404`) |
 | `INSIKA_WIDGET_AGENTS` | — | agent ids a widget visitor may address, comma-separated. The other half of the switch. **A chat rate limit is also required** or the widget answers `503` |
 | `LITESTREAM_REPLICA_URL` | — | **enables Litestream** (backup/DR). Empty = disabled (default). See below |
@@ -218,11 +228,64 @@ healthcheck, and a restart policy.
    without a volume, SQLite is ephemeral and recovery resumes nothing after a
    redeploy.
 3. **Vars**: `DEEPSEEK_API_KEY`, `OPENCLAW_GATEWAY_TOKEN`, `CONSUMER_INTERNAL_URL`,
-   `INSIKA_EGRESS_HOSTS` (and `WEB_CONCURRENCY` to match your plan/CPU).
+   `INSIKA_EGRESS_HOSTS`. **Leave `WEB_CONCURRENCY` at its default of 1** unless
+   you run [`insika-router`](ROUTER.md) in front. Railway's own docs say it
+   "does not support sticky sessions" and randomly distributes traffic across
+   replicas/workers — there is no way on this platform to satisfy the "sticky
+   routing per session in front" precondition item 2 of the process model
+   requires, at any layer (Railway replicas or Falcon's own `--count` workers
+   within one container). Raising `WEB_CONCURRENCY` without a router in front
+   is not a throughput knob, it is a guaranteed way to leak a reply across
+   sessions the first time two requests for the same session land on different
+   workers — `insika doctor` errors on this combination for exactly that
+   reason. `insika-router` is the fix: N local Falcon workers
+   behind one sticky proxy, still one Railway replica.
 4. The healthcheck hits `/up`.
 5. Point your consumer at the service's public URL, with a matching API token
    (see [RUNNING-LOCAL.md](RUNNING-LOCAL.md)).
 
+## Kubernetes
+
+Same contract as Railway, for the same reason, verified against
+ingress-nginx's own docs: **run one pod, `WEB_CONCURRENCY=1`.** Scale
+vertically (bigger pod) for now.
+
+Why a standard K8s setup does not get you out of this: the session id lives
+in the **JSON body** of `POST /v1/responses` (the `user` field), not in a
+header, cookie, or URL segment. That rules out every sticky-routing
+mechanism K8s gives you for free:
+
+- A vanilla `Service` (`ClusterIP`) load-balances across pod endpoints with
+  no notion of session at all — same failure mode as Railway's replicas,
+  one layer down.
+- `Service.spec.sessionAffinity: ClientIP` does not help either: the caller
+  is the consumer's own backend (achei-b2b, a server-to-server call), not
+  the end customer's browser, so many different customers' sessions arrive
+  from the same source IP and would collide on the same pod instead of
+  spreading — the opposite of what you want, and it still does not restore
+  a real per-session guarantee.
+- ingress-nginx's `nginx.ingress.kubernetes.io/upstream-hash-by` (the
+  standard sticky/consistent-hash annotation) hashes on nginx *variables* —
+  headers, cookies, `$request_uri`, client IP — not on a field parsed out of
+  a POST body. Reaching into the JSON body needs a custom Lua/OpenResty
+  snippet (or an Envoy filter) that parses the request and extracts `user`
+  before hashing. That is real, unbuilt engineering work, not a K8s
+  annotation to flip.
+
+If real horizontal throughput is ever needed, two paths get there, neither
+of them "just add replicas":
+1. **Build the body-aware sticky layer** above (Lua/Envoy consistent-hash on
+   the `user` field) in front of N pods, each still `WEB_CONCURRENCY=1`.
+2. **Move the session id into the URL**, the way the web widget transport
+   already does (`POST /api/widget/sessions/:token/messages` carries the
+   session token in the path) — that surface, unlike `/v1/responses`, *is*
+   sticky-routable today with a plain `upstream-hash-by: $request_uri`, no
+   custom scripting required.
+
+Neither is urgent while a single pod's throughput is enough — this section
+exists so scaling this deployment does not silently reintroduce the same
+cross-session leak the Railway incident above already found once.
+
 ## Backup / DR — Litestream (opt-in, configurable)
 
 A single volume is the **one point of total loss** between a pilot and production
@@ -297,6 +360,12 @@ per pod + **sticky-by-agent** routing (shard by tenant), or **LiteFS**, or an
 optional **Postgres** adapter. **Litestream** (above) for backup/DR from day one —
 orthogonal to topology.
 
+For the *session-routing* half specifically — as opposed to the storage
+topology above — see [`insika-router`](ROUTER.md): N pods behind a
+headless `Service`, one `insika-router` Deployment in front doing the
+consistent-hash routing ingress-nginx's own `upstream-hash-by` cannot (it
+hashes nginx variables, never a field parsed out of a POST body).
+
 ---
 
 ## Measuring performance / load
@@ -348,6 +417,8 @@ DEEPSEEK_API_KEY=sk-... ./scripts/loadtest-local.sh 4 24
 
 ## See also
 
+- [ROUTER.md](ROUTER.md) — `insika-router`, the session-sticky proxy for
+  scaling past one worker.
 - [RUNNING-LOCAL.md](RUNNING-LOCAL.md) — run the engine locally, single-process.
 - [Security](SECURITY.md) — tokens, egress, strict config.
 - [BENCHMARK.md](BENCHMARK.md) — the neutral, key-free engine benchmark.

data/docs/EMBEDDING.md

@@ -1,6 +1,6 @@
 ---
 title: Embedding
-parent: Ship it
+parent: Integrate
 nav_order: 4
 permalink: /embedding/
 ---

data/docs/EVALS.md

@@ -1,7 +1,7 @@
 ---
 title: Evals
-parent: Operate & prove it
-nav_order: 4
+parent: Improve
+nav_order: 1
 permalink: /evals/
 ---
 
@@ -157,6 +157,82 @@ Three rules make the number worth quoting:
 Cost: **two provider calls per judge per case**, which is why it is opt-in and never
 part of the gate.
 
+## Simulated users — conversations the agent never had
+
+A golden case fixes the user's turns in advance. Real customers branch — they answer
+the agent's question, or ignore it, or send an order number three messages later. A
+case with frozen turns can only ever test the path the author imagined.
+
+A **simulated** case generates the conversation instead: a persona model (the cheap
+platform `utility_model`) plays a customer with the persona as its **whole
+instruction**, and the target agent answers through the same transport a replay uses.
+They talk until the persona's `max_turns`, or until the persona emits a stop marker —
+`<<goal_met>>` when its goal is served, `<<gave_up>>` when it abandons. Both are
+recorded: "gave up at turn 3" is a finding.
+
+```yaml
+id: loja-objetivo-difuso
+agent: loja-chocolates
+persona:                          # the SIMULATED customer (alternative to `turns:`)
+  goal: "descobrir o que comprar de presente; não sabe nomes de produto"
+  style: "mensagens curtas, responde o que perguntam, desiste se tiver que explicar duas vezes"
+  opens_with: "oi, queria um presente"
+  knows:                          # the ONLY facts the persona may assert
+    orcamento: "até R$ 100"
+    ocasião: "aniversário"
+  max_turns: 8
+expect:
+  policy: investigate_first
+  rubric: |
+    Descobre o objetivo antes de recomendar (uma ou duas perguntas, não um formulário),
+    recomenda do catálogo real e fecha com um próximo passo claro. Reprova se despejar
+    catálogo antes de entender, ou se ficar perguntando sem nunca buscar.
+  min_score: 0.7
+```
+
+**The anti-invention rule is the soul of the feature.** The persona's prompt contains
+only the `knows` facts and the rule that it may assert exactly those and nothing else
+— asked about anything it does not have, it answers with ignorance ("não sei", "não
+tenho isso aqui"), like a real customer without that fact. A simulator that invents an
+order number produces a conversation the agent could never have had, and a case that
+tests nothing.
+
+```bash
+insika evals:simulate --persona persona.yml --target loja-chocolates --staging
+```
+
+- `--target <agent|url>` — an agent id over the deployment's `/v1/responses`, or a full
+  A2A URL (an agent that only speaks A2A rides the same Simulator through a thin A2A
+  transport; it polls once per second, bounded by `--timeout`, since a remote agent
+  takes seconds to finish a task).
+- `--staging` or `--eval-profile` — **one is required**. A simulated conversation
+  marks its transcript `simulated: true`; it does NOT disarm the agent's tools. So the
+  Simulator only runs against (a) a target the operator declares is a staging
+  deployment (`--staging`), or (b) an eval profile where the agent's side-effect tools
+  are swapped for dry-runs (`--eval-profile`).
+- `--eval-profile` is a **verified declaration, not a trust-me flag**. The CLI derives
+  the target's side-effect tools — the deployment's own registry over
+  `GET /v1/agents/:id` (`side_effect_tools`), or the local store when `INSIKA_DB`
+  points at it — and refuses unless `--eval-tools <a,b,c>` names every one of them.
+  The swap itself happens where the tools run (the deployment's eval profile; the
+  in-process overlay for a local graph, as in `examples/agent-tester/`); the
+  derivation is what keeps the client honest about what must be covered. An A2A target
+  has no reachable registry, so the explicit `--eval-tools` list is required there.
+- `--persona-model` — the model playing the customer (default: the platform
+  `utility_model`). The judge that scores the transcript comes from the same
+  panel configuration as a replay.
+- `--conv` — the conversation id. The default is unique per run
+  (`sim-<case id>-<random>`), because two runs on the same conv share the
+  deployment session: the second inherits the first's history — including
+  across the arms of an A/B comparison — and the grade stops measuring the
+  agent. Pass `--conv` only when continuing a session is the point.
+
+Every simulated run is **`simulated: true`** — the transcript carries the flag end to
+end, so a report never mixes a generated conversation with real traffic. A simulated
+case (`persona:`) is one of the two shapes of a case; the replay Runner **skips** it
+(the turns do not exist until a Simulator generates them) and only the simulate CLI
+drives it.
+
 Cases live in two places, and it is the same YAML in both:
 
 - **`evals/golden/**`** in the repo — the curated corpus, reviewable in a pull request,
@@ -178,6 +254,55 @@ A case whose stored YAML no longer validates is **listed as broken** on the Eval
 rather than skipped in silence — a test suite that quietly shrinks is worse than a red
 one.
 
+### `run_persona_eval` — a QA agent that tests other agents
+
+A tool (`lib/insika/tools/run_persona_eval.rb`), not a CLI: a QA agent picks one of its
+authored simulated cases and runs it **in-process**, against the case's declared
+target agent — the same Simulator + Judge machinery `evals:simulate` drives over HTTP,
+minus the network hop.
+
+```ruby
+tools %w[run_persona_eval save_artifact]
+```
+
+The model only sees `case_id`, enumerated with the ids the tool can actually run
+(never a free string it could invent) — every **simulated** case in the store, the same
+`persona:` shape as above.
+
+**Safety is derived here too, but there is no swap yet.** The tool computes the
+target's reachable side-effect tools (`Evals::EvalProfile`, the same derivation the CLI
+uses) and **refuses outright** — naming the tools — if that list is non-empty. Unlike
+the CLI, nothing here actually swaps a side-effect tool for a dry-run: `Evals::
+EvalProfile.registry` (the overlay) exists for exactly that, but nothing calls it yet.
+So `run_persona_eval` only runs against **read-only** target agents today; wiring the
+overlay into an in-process turn (so a target WITH a write tool can be tested safely) is
+follow-up work, not something this tool claims to do.
+
+**Budget**: the persona model + judge model calls are the cost of running the eval,
+charged to the **calling** agent's own turn — never the target's (the target's own
+turns bill normally, through the ordinary edge limiter, exactly as a real customer's
+would). A hard cap on the calling agent skips the run — visibly, in the tool's own
+result (`{skipped: true, reason: "budget", window: "daily"|"monthly"}`) — before a cent
+is spent; a soft cap just runs (the ledger's own alert already warns).
+
+Each run gets a **fresh session** (`eval-<case>-<random>`), never a reused one — a
+reused session lets the target agent "remember" earlier runs, which is not what a
+persona case is testing. See `examples/agent-tester/qa_scheduled.rb` for the full
+loop: `schedule` fires the QA agent, it calls `run_persona_eval`, then
+`save_artifact` publishes the quality report.
+
+**Tenant isolation.** A golden case carries a `tenant` (`platform` — the
+single-tenant default — unless the case declares one, the same convention
+`save_artifact` uses for its own binding tenant). `run_persona_eval` only ever
+lists or runs cases in the *calling* agent's own tenant: a case authored for
+another tenant is invisible, not merely refused — the model cannot even learn
+its id from the enum, and running it by a guessed id gets the same "unknown or
+invalid" error a nonexistent id would (the two must not be distinguishable).
+This is what makes "one QA agent per store" an actual boundary
+in a deployment where several stores' persona cases live in the same
+`GoldenStore` — without it, `qa-store-a` could enumerate and run
+`qa-store-b`'s persona and read its `knows` in the transcript.
+
 ## Running
 
 ```bash
@@ -232,6 +357,15 @@ That is what you run before merging a prompt, tool or model change. A case with
 baseline entry never blocks: it shows as failing in the report, but a brand-new case is
 not a regression.
 
+The same gate machinery guards the two automated loops — [Refinement](REFINEMENT.md)
+(cloned-agent replay of a proposed edit) and [Harvest](HARVEST.md) (cloned-agent
+replay of a mined skill). **Judges are mandatory for those gates** in exactly three
+shapes: a gate without a recorded baseline refuses, an all-red
+baseline refuses, and a baseline recorded WITH judge scores, replayed with no judge
+configured, refuses — a rubric'd case with no verdict reads as a pass, so the
+candidate would beat a measurement it never took. A store with no golden cases cannot
+gate, and cannot promote.
+
 ## Honest limits
 
 - **A judge is a model.** It has taste and it has bad days; that is why the
@@ -271,3 +405,13 @@ needs to score a candidate agent with the same judge, and a second copy of the j
 would be the worst possible outcome. It stays a **client** even so: it reaches a running
 deployment over HTTP through `POST /v1/responses` and never reads a store directly.
 `evals/run.rb` is a thin CLI over it.
+
+## Parity — the shadow criterion
+
+The shadow experiment's rule lives in a **deployment-side criterion file** —
+the path `INSIKA_PARITY_CRITERION` points at, prose a human reads and a `yaml`
+block the machine applies, in one file, so there is exactly one place to edit.
+The file's whole bytes are hashed; every shadow pair records that hash,
+and a window whose pairs disagree produces `:invalid`, never a verdict. Editing
+the criterion mid-experiment is *caught*, not averaged away. The fold itself is
+`lib/insika/parity/*` (see [Channels](CHANNELS.md#shadow-mode)).

data/docs/FACTS.md

@@ -0,0 +1,135 @@
+---
+title: Facts
+parent: Improve
+nav_order: 5
+permalink: /facts/
+---
+
+# Facts — distilled customer memory, human-gated
+
+Finished conversations teach a shop things — the customer's size, their budget,
+how they like to pay — and today that knowledge dies with the session. The
+engine can read it back out of the transcripts: **distillation** turns an idle,
+finished customer conversation into a list of proposed facts, a human approves
+or rejects them in the Studio, and an approved fact lands in the customer's
+memory cell — the same cell the `<memory>` block injects on every later turn.
+
+The loop has one hard rule: **nothing is ever applied automatically.** Zero
+facts reach the store without a click. Distillation proposes; the operator
+decides; the engine never applies its own proposal.
+
+## The loop
+
+1. A customer conversation (a session tagged with a `customer`, per
+   [Context](CONTEXT.md#memory)) goes idle — nobody has written for the
+   configured hours.
+2. The engine's distillation duty picks the session, sends its transcript to
+   the platform `utility_model` (off the turn path, on its own worker fiber),
+   and the model answers with a JSON list of durable facts: `name`, `value`,
+   an optional `confidence`, and the transcript message indexes that support
+   the fact (the **evidence**).
+3. The engine filters the answer against a safe subset (no invented scopes, no
+   oversized values, no out-of-range evidence), dedups it against the ledger,
+   and writes the survivors as **proposals**.
+4. The **Facts** page in the Studio shows the pending proposals with their
+   evidence excerpt. The operator **approves** (the fact is written to the
+   customer's memory, stamped with its origin), **rejects** (optionally with a
+   reason) or **dismisses** (it will never be proposed again).
+5. Approved facts join the customer's memory cell and are injected by the
+   Memory provider on the next turn of any session of that customer.
+
+## Enabling it — the `distill:` block
+
+Distillation is pack data on the agent, exactly like `refinement:` or
+`followup:` — absent = the feature is off for that agent, byte-identical engine:
+
+```ruby
+agent = Insika.agent("store-support") do
+  instructions "…"
+  distill enabled: true,
+          idle_hours: 6,        # how idle a session must be before it distills
+          min_messages: 3,      # a shorter session distills noise, not facts
+          max_proposals: 10     # cap per session pass
+  # prompt: "<what counts as a fact for THIS store>" — the pack-authored half;
+  #   absent = the engine's generic prompt. `model:` (absent = the platform
+  #   utility_model) can name the distiller explicitly.
+end
+```
+
+The same keys work in a pack's `agent.config.json`. `idle_hours` /
+`min_messages` / `max_proposals` are per-agent data; `prompt` is the store's
+half — what counts as a fact for a fashion store ("size, preference, budget")
+is not what counts for a logistics one ("address, delivery window, carrier").
+The engine never writes store vocabulary.
+
+## The human gate, precisely
+
+The **Facts** page (operate group, next to Follow-ups) shows:
+
+- **Pending** — oldest first, because evidence ages. Each card is the fact,
+  its confidence, its scope (`tenant:customer`), and the evidence excerpt read
+  from the transcript at request time (evidence is a link, never a copy). Three
+  buttons: **Approve & save to memory**, **Reject** (with an optional reason,
+  shown on the card), **Dismiss** (ghost — labelled "will not be proposed
+  again").
+- **Stale** — the CAS-lost re-present (below): the proposed value struck
+  through next to the operator's current value, resolved by dismissal.
+- **Recent** — every resolved proposal, most recent first, with operator and
+  note.
+
+**The latch** — a dismissed *or* rejected `(name, value)` tuple is never
+proposed again. The proposal rows ARE the ledger: a human saw that tuple and
+said no, and re-proposing it would train the operator to stop reading. An
+unanswered proposal is never piled on, either. A *different* value for the same
+name is a different tuple — "wears M" dismissed does not block "wears L".
+
+**The CAS guarantee** — approval never silently overwrites an operator edit.
+At distill time the engine records the target fact's existence and revision; at
+approve time it writes through the store's optimistic compare-and-swap. A fact
+the operator moved in between flips the proposal to `stale` with both values
+visible — the operator's edit always wins, never a silent overwrite.
+
+## Provenance
+
+An approved fact is written with `origin: "distilled:<session_ref>"` — the
+The provenance discipline: `"engine"` (the `remember` tool), `"operator"`
+(Studio edits), `"legacy"`, and `"distilled"` (+ the session that produced it).
+The Closed loop reads the same cell every later turn injects, and the
+distillation ledger suppresses re-proposing a fact that is already applied with
+a `distilled:` origin.
+
+## LGPD
+
+Distilled facts are personal data, and the engine treats them like it:
+
+- **Forget a customer** — `forget_customer` purges the customer's proposals
+  (every status) along with their memory cell and sessions.
+- **Delete a tenant** — `delete_tenant_data` purges the tenant's proposals.
+- **Retention** — proposals age out under the same `retention_days` sweep as
+  the rest of the footprint (pending included — a proposal is evidence of a
+  transcript, and when the transcript dies the pending fact is stale). The
+  session marker dies with its proposals: **an unreviewed proposal that ages
+  out is expired, not lost-locked** — the session is re-distillable, and a
+  duplicate survivor is filtered by the ledger, never applied.
+
+## The honest limits
+
+- **Best-effort extraction, re-scan recovery.** There is no distillation
+  queue. A crash mid-pass leaves the session unmarked; the next pass re-scans
+  it, and the ledger filters any duplicate proposal. Exactly-once is not
+  claimed — facts are re-derivable, and a duplicate is filtered, never applied.
+- **Precision is a forge audit.** The engine guarantees the *gates* (schema,
+  dedup, CAS, human approval); it cannot guarantee the *model's judgment*.
+  "Is this fact true and durable?" is audited on real traffic, per store — the
+  pack prompt is where that judgment is tuned.
+- **Scope comes from the session, never the model.** The proposal's landing
+  cell is assembled by the engine from the session's tenant and customer; the
+  schema rejects a model-authored scope outright (a cross-tenant escape), and
+  an untagged session is never distilled.
+- **Sessions only.** Distillation targets customer-tagged sessions; a
+  session without a customer has no landing zone and is skipped.
+- **One stamp, once.** A session receives `vars["agent"]` on the same write
+  that stamps its `customer` — the first tagged turn. The stamp is
+  idempotent: a session that already carries a `customer` (a pre-upgrade
+  conversation, or one that since moved to this agent) never gains `agent`
+  on its own, so it can never participate in distillation.

data/docs/HARVEST.md

@@ -0,0 +1,116 @@
+---
+title: Harvest
+parent: Improve
+nav_order: 6
+permalink: /harvest/
+---
+
+# Harvest — skills from real traffic, promoted only if eval AND conversion hold
+
+The harvest is the loop that makes a store smarter with its own traffic: it
+reads the store's **finished** conversations, asks a model to propose SKILLS
+for the agent's playbook, filters every proposal through two pre-registered
+lists (the negative list, the evidence ledger), scores it with a double gate,
+and — **only after a human approves** — lands it as a live skill. Nothing is
+ever applied automatically, and a skill that fails the gate is terminal: the
+same finding must re-surface with new evidence, there is no silent retry.
+
+Per-store data (`harvest:` on the agent), absent = the loop is off for that
+agent:
+
+```ruby
+harvest enabled: true,
+        negative_list: [ { rule: "no-competitor-prices", pattern: "concorrente" } ],
+        miner: { model: "deepseek-v4-flash",    # absent = the platform utility_model
+                 window: { last_sessions: 200 },
+                 max_proposals: 10,
+                 budget: { tokens: 100_000 } },
+        idle_hours: 24,      # how idle a session must be before it mines
+        min_messages: 3      # a shorter session mines noise
+```
+
+## The pass
+
+A mine reads ONLY durable, finished data — never an in-flight turn, never a
+live prompt prefix (the "fork" is structural: the mining writes nothing to the
+sessions it read, so a customer turn's cache is untouched by construction). The
+miner gets the transcript slices (masked), the session's evidence-ledger ids,
+and the agent's current skill names so it does not re-propose them. Then the
+filters, each drop counted and logged:
+
+1. **The negative list** — the versioned rules file (seed, imported
+   per store by `insika harvest:negative import --agent ID --file F`) and the
+   profile's hot-editable `harvest.negative_list`. A rule is a phrase or a
+   regex; phrases match case/accent-folded at word boundaries. Every rejected
+   candidate is logged with the rule id.
+2. **The grounding filter** — every product reference in a proposal must be
+   in the union of the origin sessions' evidence ids (the evidence ledger).
+   A store without `grounding.matcher.sku` does not mine at all: product
+   claims that cannot be verified are blocked by refusal, not by prompt.
+3. **Dedup** — an open `(agent, name)` tuple or a skill the store already has.
+
+## The double gate
+
+A candidate that survives mining is scored by RUNNING it — the eval gate
+clones the agent, writes the candidate skill into the clone's agent-scoped
+store, enables it on the clone's allowlist, replays the golden set over the
+ordinary public surface and compares to the accepted baseline. **Any
+regression disqualifies** — the gate is a veto, never a score to argue with.
+Judges are mandatory in exactly the shapes the refinement gate already
+refuses: no recorded baseline, an all-red baseline, and a judged baseline
+replayed with no judge (a rubric'd case with no verdict would count as a
+pass, see [Evals](EVALS.md)).
+
+The conversion gate is the second ruler: the store's funnel metric over the
+criterion's window, compared to the **frozen baseline** (`freeze_funnel_baseline`). Outcome is evidence — this gate can only say "the
+store is measurably worse than the accepted state" or "there is nothing to
+compare against". It refuses on missing data, never passes: no frozen
+baseline, no criterion, no funnel store, a fold that has not converged — each
+named on the Harvest page.
+
+Both passed → the candidate awaits **a human**. Promotion is snapshot-first
+(the pre-promotion content + allowlist), then the two existing write commands,
+then an append-only log row carrying `skill`, `origin`, `eval_ref`,
+`conversion_ref`, `approver`, `snapshot_ref` and the criterion's sha. The
+conversion ruler is re-read at the moment the skill lands — a store that
+dipped below its frozen baseline since gating parks the promotion with the
+current numbers, and a criterion file that changed since boot is a criterion
+nobody froze.
+
+## Rollback
+
+One click, deterministic: the snapshot is restored — `WriteSkill` with the
+pre-promotion bytes (or `DeleteSkill` when the skill did not exist), the
+allowlist restored to the snapshot's set, and the promotion row stamped
+`rolled_back_at`. The log stays the single ledger: a skill promoted, rolled
+back, re-promoted is three readable rows.
+
+## The honest limits
+
+- **A store with no golden cases cannot gate, and cannot promote.** The gate's
+  strength is entirely the golden set.
+- **Grounding first.** A store without a matcher does not mine — the product
+  loop is blocked until the evidence ledger is live (by refusal, not by
+  warning).
+- **A promoted skill is live text**, like any skill: the SkillStore's version
+  history plus the harvest snapshot plus the log make the rollback path
+  deterministic.
+- **The first-10 audit is human.** The negative list can carry a false
+  restriction (a phrase too broad) — that is exactly the Hermes failure, and
+  it is caught by the audit, not by the engine: the list grows and the loop
+  stops (`harvest.enabled: false`), both by data.
+- **The harvest costs provider money**: one miner call per window plus one
+  full golden replay per gated candidate. The `budget` cap, the recorded run
+  cost and the manual trigger bound it; the automated loop mines one session
+  per claim window.
+
+## The operator surface
+
+- `insika harvest --agent ID [--last-sessions N] [--since ISO] [--full]` — mine one window.
+- `insika harvest:negative import --agent ID --file F` — seed the profile's list from a rules file.
+- `insika harvest:criterion check --file F` — strict-load the frozen conversion criterion (the hook before any promotion).
+- The **Harvest page** in the Studio: the human's inbox (each candidate with
+  its evidence excerpt, the eval report, the conversion card, promote/reject),
+  the gated-but-blocked rows with the named ruler hole, the pending list, the
+  append-only promoted log with the rollback mirror, the negative list with
+  per-rule rejection counts, and the criterion block read-only.