insika 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (280) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +199 -5
  3. data/README.md +8 -2
  4. data/bin/insika +231 -13
  5. data/docs/AGENTS.md +505 -6
  6. data/docs/API.md +56 -0
  7. data/docs/CHANNELS.md +100 -10
  8. data/docs/CONTEXT.md +147 -19
  9. data/docs/DEPLOY.md +34 -11
  10. data/docs/EMBEDDING.md +11 -7
  11. data/docs/EVALS.md +20 -1
  12. data/docs/FACTS.md +135 -0
  13. data/docs/HARVEST.md +117 -0
  14. data/docs/LOADTEST.md +17 -10
  15. data/docs/OBSERVABILITY.md +65 -2
  16. data/docs/REFINEMENT.md +9 -9
  17. data/docs/RELEASING.md +34 -7
  18. data/docs/RUNNING-LOCAL.md +4 -4
  19. data/docs/SECURITY.md +85 -11
  20. data/docs/SKILLS.md +189 -3
  21. data/docs/SOAK.md +127 -0
  22. data/docs/TOOLS.md +70 -2
  23. data/docs/WHY.md +1 -1
  24. data/docs/WORKFLOWS.md +2 -2
  25. data/docs/domain.md +115 -0
  26. data/docs/index.md +2 -2
  27. data/docs/onboarding/start.md +1 -1
  28. data/lib/insika/agent_profile.rb +228 -26
  29. data/lib/insika/alert_dispatcher.rb +139 -0
  30. data/lib/insika/balloon_splitter.rb +102 -0
  31. data/lib/insika/baseline_store.rb +2 -2
  32. data/lib/insika/budget_ledger.rb +166 -0
  33. data/lib/insika/cache_series_store.rb +49 -0
  34. data/lib/insika/channel_delivery.rb +132 -24
  35. data/lib/insika/channel_registry.rb +1 -1
  36. data/lib/insika/channels/relay.rb +80 -6
  37. data/lib/insika/channels/web/widget.js +2 -2
  38. data/lib/insika/channels/web.rb +9 -9
  39. data/lib/insika/channels/webhook.rb +58 -0
  40. data/lib/insika/chat_builder.rb +145 -13
  41. data/lib/insika/checkpoint_store.rb +16 -0
  42. data/lib/insika/circuit_state.rb +114 -0
  43. data/lib/insika/coercion.rb +8 -0
  44. data/lib/insika/commands/agent_payload.rb +6 -4
  45. data/lib/insika/commands/cancel_followup.rb +49 -0
  46. data/lib/insika/commands/create_agent.rb +2 -2
  47. data/lib/insika/commands/create_session.rb +1 -1
  48. data/lib/insika/commands/delete_llm_provider.rb +1 -1
  49. data/lib/insika/commands/delete_skill.rb +43 -0
  50. data/lib/insika/commands/delete_tenant_data.rb +95 -0
  51. data/lib/insika/commands/export_customer_memory.rb +48 -0
  52. data/lib/insika/commands/forget_customer.rb +117 -0
  53. data/lib/insika/commands/freeze_funnel_baseline.rb +113 -0
  54. data/lib/insika/commands/gate_harvest.rb +138 -0
  55. data/lib/insika/commands/gate_refinement.rb +12 -12
  56. data/lib/insika/commands/import_mcp_tools.rb +1 -1
  57. data/lib/insika/commands/import_tools.rb +4 -4
  58. data/lib/insika/commands/issue_tenant_token.rb +41 -0
  59. data/lib/insika/commands/judge_shadow_pairs.rb +124 -0
  60. data/lib/insika/commands/memory_forget_fact.rb +20 -4
  61. data/lib/insika/commands/memory_put_fact.rb +23 -4
  62. data/lib/insika/commands/promote_harvest.rb +130 -0
  63. data/lib/insika/commands/record_outcome.rb +46 -0
  64. data/lib/insika/commands/record_shadow_reply.rb +68 -0
  65. data/lib/insika/commands/reject_harvest.rb +38 -0
  66. data/lib/insika/commands/resolve_proposal.rb +108 -0
  67. data/lib/insika/commands/resolve_refinement.rb +1 -1
  68. data/lib/insika/commands/revoke_contact.rb +49 -0
  69. data/lib/insika/commands/revoke_token.rb +39 -0
  70. data/lib/insika/commands/rollback_harvest.rb +86 -0
  71. data/lib/insika/commands/rotate_tenant_token.rb +43 -0
  72. data/lib/insika/commands/run_distillation.rb +186 -0
  73. data/lib/insika/commands/run_harvest.rb +393 -0
  74. data/lib/insika/commands/run_refinement.rb +5 -5
  75. data/lib/insika/commands/send_message.rb +112 -15
  76. data/lib/insika/commands/session_purge.rb +67 -0
  77. data/lib/insika/commands/set_agent_tools.rb +1 -1
  78. data/lib/insika/commands/set_skill_agents.rb +60 -19
  79. data/lib/insika/commands/trigger_workflow.rb +1 -1
  80. data/lib/insika/commands/update_agent.rb +1 -1
  81. data/lib/insika/commands/write_data_tool.rb +1 -1
  82. data/lib/insika/commands/write_golden.rb +1 -1
  83. data/lib/insika/commands/write_skill.rb +19 -9
  84. data/lib/insika/config_store.rb +8 -4
  85. data/lib/insika/contact_store.rb +183 -0
  86. data/lib/insika/context/builder.rb +23 -5
  87. data/lib/insika/context/fragment.rb +31 -3
  88. data/lib/insika/context/priority.rb +6 -2
  89. data/lib/insika/context/provider.rb +17 -3
  90. data/lib/insika/context/providers/briefing.rb +96 -0
  91. data/lib/insika/context/providers/memory.rb +16 -7
  92. data/lib/insika/context/providers/prompt.rb +30 -2
  93. data/lib/insika/context/providers/request.rb +1 -1
  94. data/lib/insika/context/providers/session.rb +17 -2
  95. data/lib/insika/context/providers/skill.rb +7 -1
  96. data/lib/insika/context/providers/skill_trigger.rb +128 -0
  97. data/lib/insika/context/providers/tool_search.rb +2 -0
  98. data/lib/insika/context_trace_store.rb +128 -0
  99. data/lib/insika/delegation_store.rb +2 -2
  100. data/lib/insika/distill.rb +224 -0
  101. data/lib/insika/distill_engine.rb +169 -0
  102. data/lib/insika/doctor.rb +962 -7
  103. data/lib/insika/dsl/runtime.rb +20 -11
  104. data/lib/insika/dsl/server_boot.rb +74 -4
  105. data/lib/insika/dsl/system.rb +1 -1
  106. data/lib/insika/dsl.rb +152 -15
  107. data/lib/insika/edge_limiter.rb +167 -8
  108. data/lib/insika/egress_guard.rb +3 -3
  109. data/lib/insika/env_schema.rb +22 -12
  110. data/lib/insika/errors.rb +72 -5
  111. data/lib/insika/evals/assertions.rb +15 -14
  112. data/lib/insika/evals/baseline.rb +3 -3
  113. data/lib/insika/evals/golden.rb +8 -8
  114. data/lib/insika/evals/judge.rb +7 -7
  115. data/lib/insika/evals/pairwise.rb +21 -9
  116. data/lib/insika/evals/report.rb +2 -2
  117. data/lib/insika/evals/runner.rb +6 -6
  118. data/lib/insika/evals/transport.rb +2 -2
  119. data/lib/insika/event_stream.rb +23 -5
  120. data/lib/insika/evidence.rb +183 -0
  121. data/lib/insika/executor.rb +1092 -160
  122. data/lib/insika/followup_engine.rb +207 -0
  123. data/lib/insika/followup_policy.rb +221 -0
  124. data/lib/insika/followup_store.rb +306 -0
  125. data/lib/insika/frontmatter.rb +1 -1
  126. data/lib/insika/funnel_declaration.rb +106 -0
  127. data/lib/insika/funnel_fold.rb +179 -0
  128. data/lib/insika/funnel_store.rb +163 -0
  129. data/lib/insika/golden_store.rb +3 -3
  130. data/lib/insika/grounding/matcher.rb +69 -0
  131. data/lib/insika/grounding.rb +44 -0
  132. data/lib/insika/harvest/conversion_gate.rb +159 -0
  133. data/lib/insika/harvest/criterion.rb +98 -0
  134. data/lib/insika/harvest/gate.rb +194 -0
  135. data/lib/insika/harvest/negative_list.rb +199 -0
  136. data/lib/insika/harvest.rb +241 -0
  137. data/lib/insika/harvest_engine.rb +193 -0
  138. data/lib/insika/harvest_store.rb +548 -0
  139. data/lib/insika/http_client.rb +3 -3
  140. data/lib/insika/inbound_log.rb +1 -1
  141. data/lib/insika/llm_configurator.rb +3 -3
  142. data/lib/insika/loop_detector.rb +143 -0
  143. data/lib/insika/mcp_http_client.rb +4 -4
  144. data/lib/insika/mcp_tool_ingestor.rb +6 -6
  145. data/lib/insika/media.rb +298 -0
  146. data/lib/insika/memory_audit_store.rb +85 -0
  147. data/lib/insika/memory_store.rb +264 -23
  148. data/lib/insika/message_origin.rb +8 -3
  149. data/lib/insika/model_resolver.rb +1 -1
  150. data/lib/insika/model_selection.rb +5 -4
  151. data/lib/insika/model_visible.rb +87 -0
  152. data/lib/insika/model_visible_trace_store.rb +66 -0
  153. data/lib/insika/onboarding.rb +8 -3
  154. data/lib/insika/outbox_store.rb +44 -6
  155. data/lib/insika/outcome_store.rb +147 -0
  156. data/lib/insika/overlay_tool_registry.rb +3 -4
  157. data/lib/insika/pack.rb +3 -3
  158. data/lib/insika/pack_importer.rb +17 -15
  159. data/lib/insika/packaging.rb +163 -0
  160. data/lib/insika/parity/criterion.rb +79 -0
  161. data/lib/insika/parity/verdict.rb +318 -0
  162. data/lib/insika/pending_action_store.rb +1 -1
  163. data/lib/insika/plugin/loader.rb +2 -2
  164. data/lib/insika/policy/policy.rb +1 -1
  165. data/lib/insika/prefix_fingerprint.rb +58 -0
  166. data/lib/insika/profile_source.rb +34 -7
  167. data/lib/insika/proposal_store.rb +271 -0
  168. data/lib/insika/provider_error_classifier.rb +160 -0
  169. data/lib/insika/queue_policy.rb +6 -3
  170. data/lib/insika/recovery.rb +47 -6
  171. data/lib/insika/refinement/candidate.rb +4 -4
  172. data/lib/insika/refinement/evidence_collector.rb +6 -6
  173. data/lib/insika/refinement/gate.rb +7 -7
  174. data/lib/insika/refinement/panel.rb +7 -7
  175. data/lib/insika/refinement/proposer.rb +10 -10
  176. data/lib/insika/refinement_store.rb +12 -12
  177. data/lib/insika/reliability.rb +211 -0
  178. data/lib/insika/retention.rb +281 -0
  179. data/lib/insika/routing.rb +101 -0
  180. data/lib/insika/safety/config.rb +46 -6
  181. data/lib/insika/safety/corpus.rb +255 -0
  182. data/lib/insika/safety/detectors.rb +34 -115
  183. data/lib/insika/safety/factory.rb +18 -5
  184. data/lib/insika/safety/grounding_enforcer.rb +59 -0
  185. data/lib/insika/safety/grounding_validator.rb +49 -0
  186. data/lib/insika/safety/input_guardrail.rb +20 -5
  187. data/lib/insika/safety/moderator.rb +19 -11
  188. data/lib/insika/safety/output_filter.rb +10 -6
  189. data/lib/insika/safety/output_validator.rb +13 -7
  190. data/lib/insika/safety/safe_responses.rb +1 -1
  191. data/lib/insika/sandbox/boundary.rb +2 -2
  192. data/lib/insika/sandbox.rb +1 -1
  193. data/lib/insika/schema_guard.rb +35 -0
  194. data/lib/insika/server/app.rb +366 -54
  195. data/lib/insika/server/boot.rb +4 -4
  196. data/lib/insika/server/rack_app.rb +31 -7
  197. data/lib/insika/server/responses.rb +58 -9
  198. data/lib/insika/server/tenant_auth.rb +61 -0
  199. data/lib/insika/session_actor.rb +11 -7
  200. data/lib/insika/session_store.rb +66 -3
  201. data/lib/insika/settings_store.rb +15 -5
  202. data/lib/insika/shadow_pair_store.rb +258 -0
  203. data/lib/insika/shutdown.rb +4 -4
  204. data/lib/insika/skill_catalog.rb +131 -20
  205. data/lib/insika/skill_store.rb +70 -22
  206. data/lib/insika/soak/envelope.rb +140 -0
  207. data/lib/insika/soak/report.rb +392 -0
  208. data/lib/insika/soak/runner.rb +554 -0
  209. data/lib/insika/steer_injector.rb +1 -1
  210. data/lib/insika/store.rb +11 -2
  211. data/lib/insika/stores/memory.rb +6 -0
  212. data/lib/insika/stores/sqlite.rb +8 -0
  213. data/lib/insika/studio/app.rb +1058 -75
  214. data/lib/insika/studio/assets/dist/application.css +1 -1
  215. data/lib/insika/studio/assets/dist/application.js +27 -26
  216. data/lib/insika/studio/assets/dist/favicon.svg +6 -0
  217. data/lib/insika/studio/forms.rb +274 -22
  218. data/lib/insika/studio/nav_icons.rb +7 -2
  219. data/lib/insika/studio/views/_message.erb +2 -2
  220. data/lib/insika/studio/views/agent_detail.erb +629 -86
  221. data/lib/insika/studio/views/agents.erb +11 -7
  222. data/lib/insika/studio/views/approvals.erb +4 -1
  223. data/lib/insika/studio/views/chats.erb +4 -1
  224. data/lib/insika/studio/views/customer.erb +94 -0
  225. data/lib/insika/studio/views/customers.erb +32 -0
  226. data/lib/insika/studio/views/evals.erb +4 -1
  227. data/lib/insika/studio/views/facts.erb +133 -0
  228. data/lib/insika/studio/views/followups.erb +125 -0
  229. data/lib/insika/studio/views/funnel.erb +106 -0
  230. data/lib/insika/studio/views/harvest.erb +234 -0
  231. data/lib/insika/studio/views/home.erb +2 -1
  232. data/lib/insika/studio/views/layout.erb +1 -0
  233. data/lib/insika/studio/views/parity.erb +147 -0
  234. data/lib/insika/studio/views/playground.erb +7 -1
  235. data/lib/insika/studio/views/refinement.erb +4 -4
  236. data/lib/insika/studio/views/session.erb +133 -3
  237. data/lib/insika/studio/views/settings.erb +9 -12
  238. data/lib/insika/studio/views/skills.erb +66 -12
  239. data/lib/insika/studio/views/system_files.erb +1 -1
  240. data/lib/insika/studio/views/task.erb +13 -0
  241. data/lib/insika/studio/views/tasks.erb +4 -1
  242. data/lib/insika/studio/views/tools.erb +0 -1
  243. data/lib/insika/subagent_graph.rb +3 -3
  244. data/lib/insika/task_actor.rb +3 -3
  245. data/lib/insika/task_store.rb +22 -2
  246. data/lib/insika/telemetry/pricing.rb +3 -3
  247. data/lib/insika/telemetry/recorder.rb +1 -1
  248. data/lib/insika/telemetry.rb +2 -2
  249. data/lib/insika/testing/store_contract.rb +54 -33
  250. data/lib/insika/tick.rb +146 -0
  251. data/lib/insika/token_store.rb +168 -0
  252. data/lib/insika/tool_assembly.rb +5 -5
  253. data/lib/insika/tool_definition.rb +25 -15
  254. data/lib/insika/tool_envelope.rb +70 -1
  255. data/lib/insika/tool_manifest.rb +11 -7
  256. data/lib/insika/tool_output_compressor.rb +100 -0
  257. data/lib/insika/tool_store.rb +1 -1
  258. data/lib/insika/tool_trace_store.rb +1 -1
  259. data/lib/insika/tools/concurrency.rb +2 -2
  260. data/lib/insika/tools/data_defined_tool.rb +14 -5
  261. data/lib/insika/tools/generate_image.rb +44 -0
  262. data/lib/insika/tools/load_skill.rb +61 -3
  263. data/lib/insika/tools/schedule_followup.rb +164 -0
  264. data/lib/insika/tools/stuck_signal.rb +44 -0
  265. data/lib/insika/tools/subagent.rb +4 -4
  266. data/lib/insika/tools/subagents.rb +1 -1
  267. data/lib/insika/tools/tts.rb +47 -0
  268. data/lib/insika/tools/update_briefing.rb +126 -0
  269. data/lib/insika/turn_output.rb +2 -2
  270. data/lib/insika/turn_state.rb +54 -13
  271. data/lib/insika/turn_timing.rb +24 -4
  272. data/lib/insika/usage_ledger.rb +1 -1
  273. data/lib/insika/version.rb +1 -1
  274. data/lib/insika/vitals.rb +84 -0
  275. data/lib/insika/wiring/graph.rb +372 -34
  276. data/lib/insika/workflow.rb +1 -1
  277. data/lib/insika/workflow_registry.rb +1 -1
  278. data/lib/insika.rb +122 -16
  279. metadata +95 -2
  280. data/lib/insika/server/admin_auth.rb +0 -29
data/docs/CHANNELS.md CHANGED
@@ -181,16 +181,19 @@ whole turn and read the answer off it — or the relay, where the engine acks in
181
181
  milliseconds and POSTs the answer to you when it exists.
182
182
 
183
183
  The instinct is that streaming gets the customer their reply sooner, and that the
184
- relay trades that away. **It does not, and the reason is structural:** the engine
185
- publishes `:content` as the ANSWER, whole, after the turn's hooks
186
- ([what crosses the edge](/architecture/#what-crosses-the-edge)). During the turn the
187
- stream carries tool activity; the text arrives in one piece at the end. Measured on
188
- a real store agent, the text frames span **0 ms** — there is nothing to deliver
189
- progressively, on either path.
184
+ relay trades that away. **For the default relay it does not, and the reason is
185
+ structural:** the engine publishes `:content` as the ANSWER, whole, after the
186
+ turn's hooks ([what crosses the edge](/architecture/#what-crosses-the-edge)).
187
+ During the turn the stream carries tool activity; the text arrives in one piece at
188
+ the end. Measured on a real store agent, the text frames span **0 ms** — there is
189
+ nothing to deliver progressively. That is the `:at_end` fact, true of `/v1/responses`
190
+ and of a relay that never opted into [progressive delivery](#delivery-policy).
191
+ The relay can opt out of it per channel — the opt-in is below, and it is the one
192
+ thing that changes the 0 ms span.
190
193
 
191
194
  | | drop-in `/v1/responses` | relay |
192
195
  |---|---|---|
193
- | What the customer receives | one message, at the end | one message, at the end |
196
+ | What the customer receives | one message, at the end | one message at the end (`:at_end`), or one WhatsApp balloon per paragraph with the first one as soon as the answer exists (`delivery: :progressive`) |
194
197
  | Your app's request | held open for the whole turn (seconds) | acked in **milliseconds** |
195
198
  | A turn that outlives your HTTP timeout | your problem | already handled — the answer arrives later |
196
199
  | Retry on a failed handover | yours to build | the engine's outbox, bounded, at-most-once |
@@ -259,7 +262,7 @@ Content-Type: application/json
259
262
  "external_id": "5511999998888", // required — YOUR key for this conversation
260
263
  "message": "queria saber do pedido", // required
261
264
  "event_id": "wamid.HBg…", // optional but strongly recommended (dedup)
262
- "vars": { "store": "ocean-drop" } // optional — session vars on first contact
265
+ "vars": { "store": "demo-store" } // optional — session vars on first contact
263
266
  }
264
267
  ```
265
268
 
@@ -285,12 +288,14 @@ with the default `followup` you will only see `202` and `duplicate`.
285
288
 
286
289
  ### Outbound
287
290
 
288
- One POST per reply, to the URL you configured:
291
+ One POST per **balloon**, to the URL you configured. For the default `:at_end`
292
+ delivery a turn emits exactly one; a progressive turn may emit several — `task_id`
293
+ is the correlation, `index` is the order.
289
294
 
290
295
  ```jsonc
291
296
  POST <INSIKA_RELAY_DELIVER_URL>
292
297
  Authorization: Bearer <INSIKA_RELAY_DELIVER_TOKEN> // omitted if unset
293
- X-Insika-Delivery: 0f2c… // stable idempotency key
298
+ X-Insika-Delivery: 0f2c… // stable idempotency key, PER balloon
294
299
  Content-Type: application/json
295
300
 
296
301
  {
@@ -310,6 +315,46 @@ stays internal unless the agent opts in. That contract is
310
315
  [the edge contract](/architecture/#what-crosses-the-edge), and it is why you can
311
316
  forward `content` straight to the customer.
312
317
 
318
+ ### Delivery policy
319
+
320
+ A relay declares **how the outbox flushes** — a property of the surface
321
+ (WhatsApp balloons), not of the model:
322
+
323
+ - **`:at_end`** (the default, also `INSIKA_RELAY_DELIVERY=at_end` or unset): one
324
+ POST at the end with the whole answer. Byte-identical to the contract above.
325
+ - **`:progressive`** (`INSIKA_RELAY_DELIVERY=progressive`): the answer is split
326
+ into balloons at paragraph boundaries (a soft 600-char cap splits a single long
327
+ paragraph on sentences; fenced code blocks are atomic), and each balloon is its
328
+ own POST, in order, starting as soon as the answer exists. The consumer sends
329
+ each POST as its own platform message.
330
+
331
+ A progressive POST carries two additive fields **only when the turn split into
332
+ more than one balloon** — a one-balloon progressive turn is indistinguishable
333
+ from `:at_end` on the wire:
334
+
335
+ ```jsonc
336
+ {
337
+ "external_id": "5511999998888",
338
+ "session_id": "relay:5511999998888",
339
+ "task_id": "…",
340
+ "content": "Seu pedido saiu para entrega hoje",
341
+ "index": 0, // this balloon's position, 0-based
342
+ "final": false // true on the LAST balloon of this task_id
343
+ }
344
+ ```
345
+
346
+ When `index`/`final` are present, this POST is **one balloon of several for the
347
+ same `task_id`** — forward `content` as its own platform message and honor
348
+ `X-Insika-Delivery` per balloon. If you only forward `content` and ignore the new
349
+ fields, progressive turns still read as N messages in arrival order (which is
350
+ index order — the engine dispatches the chain sequentially); single-balloon turns
351
+ behave exactly as before. A consumer that ignores unknown keys keeps working.
352
+
353
+ The engine measures the win in-process: every channel turn records
354
+ `first_balloon_ms` (inbound receipt → first outbox flush) on the task record and
355
+ the terminal event, so the Studio task page shows whether the 2 s target is being
356
+ hit without toggling any flag.
357
+
313
358
  ### Deduplication
314
359
 
315
360
  Send `event_id` and a retried webhook costs you nothing: the engine recognizes the
@@ -344,6 +389,7 @@ Three environment variables on the engine:
344
389
  INSIKA_RELAY_TOKEN=<a long random secret> # the switch AND the credential
345
390
  INSIKA_RELAY_DELIVER_URL=https://you.example/insika/deliver
346
391
  INSIKA_RELAY_DELIVER_TOKEN=<another secret> # optional; what we send to you
392
+ INSIKA_RELAY_DELIVERY=progressive # optional; "at_end" (the default) = one POST
347
393
  ```
348
394
 
349
395
  `INSIKA_RELAY_TOKEN` is the switch: without it the channel is not mounted and
@@ -367,6 +413,50 @@ nothing at all).
367
413
  A runnable consumer in ~40 lines lives in
368
414
  [`examples/relay-channel/`](https://github.com/guizaols/insika/tree/main/examples/relay-channel).
369
415
 
416
+ ## Shadow mode
417
+
418
+ Shadow mode (RFC-0025) lets one channel run every turn **end to end and deliver
419
+ nothing** — the experiment that answers "can we replace the incumbent?" before
420
+ any customer is handed over. The incumbent keeps answering; the engine records
421
+ what it *would* have answered, and the two replies are judged pairwise against a
422
+ **frozen criterion** — the file `INSIKA_PARITY_CRITERION` points at.
423
+
424
+ ```bash
425
+ INSIKA_RELAY_SHADOW=1 # the switch
426
+ # INSIKA_PARITY_CRITERION — required in shadow mode
427
+ ```
428
+
429
+ Three things change when it is on:
430
+
431
+ - The turn still runs; the reply is recorded as a **pair** and never reaches the
432
+ customer. Zero outbox records, ever — and `Relay#deliver` refuses loudly if one
433
+ somehow exists.
434
+ - The inbound ack becomes `200 {"task_id": …, "shadow": true}` instead of `202`,
435
+ so a consumer wired to "202 means a reply is coming" cannot be misled.
436
+ - `event_id` becomes **required** — it is the correlation key both halves of the
437
+ pair are built from.
438
+
439
+ The incumbent's reply enters the same pair through one of two shapes: alongside
440
+ the mirror call itself (`"incumbent_reply": "…"` on `POST /channels/relay/events`),
441
+ or as a follow-up when the consumer answers first:
442
+
443
+ ```jsonc
444
+ POST /channels/relay/shadow-reply
445
+ Authorization: Bearer <INSIKA_RELAY_TOKEN>
446
+ { "external_id": "5511999998888", "event_id": "wamid.HBg…",
447
+ "reply": "Claro! Me passa o número do pedido?", "at": "2026-…Z" }
448
+ → 202 { "pair_id": "9f2c…", "status": "open" }
449
+ ```
450
+
451
+ Both shapes land in one command; a retried reply is ignored (first write wins —
452
+ the customer received one reply, and a retry must not rewrite evidence).
453
+
454
+ **No criterion, no shadow.** Boot refuses when shadow is on and
455
+ the criterion file is missing or unparseable — a number nobody pre-registered
456
+ does not count. The Studio's Parity page folds the running verdict on demand
457
+ from the pair store; `insika doctor` reports the shadow configuration before
458
+ boot does.
459
+
370
460
  ## Sessions
371
461
 
372
462
  The engine mints the session id: `relay:<your external_id>`. Namespacing is not
data/docs/CONTEXT.md CHANGED
@@ -20,18 +20,21 @@ Providers are chosen by a double gate (the provider opts in for the profile **an
20
20
  the agent's `context_providers` allowlist permits it), then assembled by priority
21
21
  into a deterministic prompt:
22
22
 
23
- | Provider | Block | Priority | Notes |
24
- |----------|-------|:--------:|-------|
25
- | **Identity** | system | **100 — pinned** | The agent's prompt files (global system files first). Never cut. |
26
- | **Skills** | `<available_skills>` | 80 | Level-1 skill list — see [Skills](SKILLS.md). |
27
- | **Memory** | `<memory>` | 75 | Durable facts + recent notes, only if `memory` is on. Cuttable. |
28
- | **Tool search** | `<available_tools>` | 70 | Level-1 list of deferred tools — see [Tools](TOOLS.md). |
29
- | **Session** | history | 60–79 | The running transcript; priority scales with recency. |
30
- | **Request** | `<request_context>` | 40 | Turn variables + tenant. Most cuttable; sits last. |
31
-
32
- The ordering is deliberate: the **stable identity sits first**, the **volatile
33
- request context sits last**. That keeps the cacheable prefix byte-stable (see the
34
- prefix cache below).
23
+ | Provider | Block | Priority | Layer | Notes |
24
+ |----------|-------|:--------:|-------|-------|
25
+ | **Identity** | system | **100 — pinned** | identity | The agent's prompt files (global system files first). Never cut. |
26
+ | **Skills** | `<available_skills>` | 80 | identity | Level-1 skill list, minus whatever is already eager — see [Skills](SKILLS.md). |
27
+ | **Tool search** | `<available_tools>` | 70 | identity | Level-1 list of deferred tools see [Tools](TOOLS.md). |
28
+ | **Skill trigger** | `<active_skill>` | 85 | volatile | Level-2 bodies: the agent's `skills_eager` set, plus the ones whose `triggers:` match the message — see [Skills](SKILLS.md). |
29
+ | **Memory** | `<memory>` | 75 | volatile | Durable facts + recent notes, only if `memory` is on. Cuttable. |
30
+ | **Briefing** | `<briefing>` | 65 | volatile | The session's working state (known fields, still-missing list, next step) — only if the pack declared `briefing_fields`. Cuttable. |
31
+ | **Session** | history | 60–79 | volatile | The running transcript; priority scales with recency. |
32
+ | **Request** | `<request_context>` | 40 | volatile | Turn variables + tenant. Most cuttable; sits last. |
33
+
34
+ The ordering is deliberate: the render order is **identity layer first, volatile
35
+ layer after** — nothing volatile can sit above the cache boundary, whatever its
36
+ priority — and within each layer the priority sort above holds. That keeps the
37
+ cacheable prefix byte-stable (see the prefix cache below).
35
38
 
36
39
  ## Budget and eviction — the actual "compaction"
37
40
 
@@ -48,32 +51,111 @@ prefix cache below).
48
51
  > returns empty turns, raise `context_budget` (e.g. to `60000`) before looking
49
52
  > anywhere else. See [Agents](AGENTS.md#default-limits).
50
53
 
51
- ### Compaction is not wired
54
+ ### Compaction is not wired — except the mechanical dedupe
52
55
 
53
56
  There is a settings stub for LLM-summarization compaction (`enabled: false`,
54
- `keep_last`, a reserved utility-model slot), and the Studio exposes it, but
55
- **nothing consumes it today** size is managed purely by hard budget eviction.
57
+ `keep_last`, a reserved utility-model slot), but **nothing consumes it today**
58
+ and the Studio no longer shows a form for it, so the setting cannot be
59
+ switched on by accident. Size is managed purely by hard budget eviction.
56
60
  Do not rely on compaction to shrink a bloated agent: tune `context_budget` and
57
- keep the identity lean. Turning the setting on does nothing yet.
61
+ keep the identity lean.
62
+
63
+ One cheap half **is** wired, opt-in per agent: `tool_output_compression` (DSL
64
+ `tool_output_compression`, or `"tool_output_compression": true` in the pack).
65
+ When on, byte-identical repeated **tool results** in the replayed history
66
+ collapse to a compact back-reference (the first occurrence stays full, with a
67
+ one-line summary) — no LLM involved. It changes what the model sees, so it is
68
+ never a default: an older detail is only in the first occurrence, and a model
69
+ that wants it re-calls the tool. Reach for it when a tool keeps returning the
70
+ same body (a catalog page, a status) and history is the fragment blowing the
71
+ budget first.
72
+
73
+ > The **Studio session screen** shows what the builder assembled per turn —
74
+ > tokens per category (identity, history, memory, …), the tools-schema estimate
75
+ > and the budget verdict (`used / cap`, evicted sources). Counts only, never
76
+ > fragment content.
58
77
 
59
78
  ## Memory
60
79
 
61
80
  With `memory` enabled, an agent gains a built-in `remember` tool for durable
62
81
  facts, and those facts (plus recent notes) are injected back into the prompt on
63
82
  later turns — **including turns in a different session**. Memory is scoped per
64
- agent. This is distinct from *session history*, which is the transcript of one
83
+ agent, per `(tenant, customer)` when the message carries a `customer`, and per
84
+ session otherwise — a session's own memory lives in a marked `memory:chat:<session id>`
85
+ cell, never a bare one, so the Customers drill cannot read a conversation as a
86
+ customer. This is distinct from *session history*, which is the transcript of one
65
87
  conversation; memory is the small set of facts that should outlive any single
66
88
  conversation. Facts and notes are editable from the Studio agent page. See
67
89
  [`examples/memory/`](https://github.com/guizaols/insika/tree/main/examples/memory/) for a runnable cross-session example.
68
90
 
91
+ Facts carry **provenance metadata** (RFC-0031): every fact record stores `origin`
92
+ (who wrote it — `"engine"`, `"operator"`, `"legacy"` or `"distilled"`),
93
+ `created_at` / `updated_at` timestamps, and an optional `expires_at` (ISO8601) —
94
+ **an expired fact is never injected**, even before the daily sweep prunes it. The
95
+ Studio Customers drill reads and edits the same cell the next turn reads (injection
96
+ unchanged), and every operator mutation lands in the content-free audit trail
97
+ (digests, never values). The sweep honors the `memory_ttl_days` setting on its own
98
+ knob — see [Security](SECURITY.md#memory-and-the-right-to-be-forgotten-lgpd-rfc-0031).
99
+
100
+ An **approved distilled fact** (RFC-0034 — see [Facts](FACTS.md)) lands in the
101
+ same cell this provider injects, stamped `distilled:<session_ref>` — approved on
102
+ the Studio Facts page, never applied automatically.
103
+
104
+ ## Briefing — the session's working state
105
+
106
+ The **briefing** is the per-conversation working state the agent keeps and asks
107
+ for: which facts it already learned (size, budget, delivery day) and the agreed
108
+ next step. It is **engine-owned data** — one `"briefing"` key on the session
109
+ record, written only by the agent through tools — whose *fields* come from the
110
+ pack:
111
+
112
+ ```jsonc
113
+ // pack agent.config.json
114
+ { "id": "store-support", "briefing_fields": ["size", "budget", "delivery_day"] }
115
+ ```
116
+
117
+ ```ruby
118
+ # or the DSL — [] = the feature is off (no block, no tools)
119
+ briefing_fields "size", "budget", "delivery_day"
120
+ ```
121
+
122
+ With fields declared, the turn's `:system` context gains a `<briefing>` block
123
+ (priority 65 — below identity/skill/memory so it never breaks the cacheable
124
+ prefix, above the turn's own `<request_context>`):
125
+
126
+ ```
127
+ <briefing>
128
+ known:
129
+ size: M
130
+ still missing: budget, delivery_day
131
+ next step: send the payment link tomorrow at 10
132
+ </briefing>
133
+ ```
134
+
135
+ The `still missing` list is the point: the *model* sees which declared fields are
136
+ still unanswered, so it stops re-asking for something already given. Stored keys
137
+ that the pack no longer declares are never rendered. The Studio session screen
138
+ shows the persisted state (known fields + next step), read-only.
139
+
140
+ The agent writes the briefing through two built-in tools, wired only when the
141
+ pack declared fields:
142
+
143
+ - `update_briefing(field:, value:)` — records a field. An undeclared `field`
144
+ returns an envelope error (`unknown field '…'; declared: …`) and nothing is
145
+ persisted; a blank `value` clears the field.
146
+ - `set_next_step(text:)` — records the agreed next step; a blank `text` clears it.
147
+
148
+ Both are deterministic in-process writes (never enveloped) and survive across
149
+ turns and resumes — a resumed conversation re-opens with the briefing intact.
150
+
69
151
  ## The provider prefix cache
70
152
 
71
153
  Two distinct caching mechanisms — don't conflate them:
72
154
 
73
155
  - **Automatic server-side prefix cache.** Some providers prefix-cache a stable
74
156
  system prefix automatically, at no cost to configure. This works **only because**
75
- the identity is at the top of the system block and the volatile
76
- `<request_context>` is at the bottom, keeping the cacheable prefix byte-stable.
157
+ the engine renders the system in two layers (below) and the volatile half sits
158
+ **under** the identity boundary, keeping the cacheable prefix byte-stable.
77
159
  Anything that injects volatile content high in the system block breaks the cache.
78
160
  - **Manual cache breakpoints (opt-in).** With `prompt_caching` on **and** a
79
161
  provider that supports explicit cache control, the builder sets one cache
@@ -83,6 +165,52 @@ Two distinct caching mechanisms — don't conflate them:
83
165
  Cache accounting surfaces as `cached_tokens` (reads) and `cache_creation_tokens`
84
166
  (writes), visible in telemetry and the Studio tokens chip.
85
167
 
168
+ ### The two layers (RFC-0030)
169
+
170
+ The system block is partitioned into two cache layers:
171
+
172
+ - **Identity** — bytes that change only on deploy/config edit: the persona
173
+ prompt (`Prompt`), the level-1 skill list (`Skill`) and the deferred-tool
174
+ catalog (`ToolSearch`). This is the cacheable prefix.
175
+ - **Volatile** — bytes that may change per turn: memory, session history,
176
+ triggered skill bodies, the `<request_context>`. Everything else.
177
+
178
+ The layer is a **provider-class contract**, not profile data: `ContextProvider`
179
+ declares `def layer = :volatile` (conservative — nothing gets pinned by
180
+ accident) and the three identity builtins override to `:identity`. A pack does
181
+ not set it — a pack reorganizes *which content goes into the Prompt provider vs
182
+ the volatile providers*. The Builder stamps the layer on every fragment at
183
+ production, and the render order is **identity first, volatile after** — a
184
+ volatile block can never land above the cache boundary, whatever its priority.
185
+ Within each partition the existing priority sort is untouched.
186
+
187
+ The engine's own `doctor` check verifies the declaration: an engine-known
188
+ volatile provider (Memory, Session, Request, SkillTrigger) that overrides to
189
+ `:identity` is an **error** (guaranteed cache kill); any other custom
190
+ `:identity` provider is a **warning** (purity unverifiable from outside — the
191
+ output must be byte-stable across turns).
192
+
193
+ ### The observable cache: fingerprints and the invalidation reason
194
+
195
+ Each turn, the Executor hashes the rendered prefix into a PII-free fingerprint
196
+ chain — one SHA-256 per system category in render order, one for the tool
197
+ schemas, one cumulative `prefix` — and compares it against the previous turn's
198
+ entry. The **invalidation reason** is the first category whose bytes changed (or
199
+ vanished); a turn whose prefix held reports nothing. History is deliberately
200
+ excluded: a new user message is a divergence every turn, which would be noise,
201
+ not a reason.
202
+
203
+ The Studio surfaces it in two places: the **session Context card** shows the
204
+ turn's cache-hit percentage and the `broke: <category>` line (plus the
205
+ `identity` marker on the category rows), and the **agent detail** carries a
206
+ cache tab with the per-agent hit series over time. The per-agent series lives
207
+ in its own capped store, because a session does not stamp its author — the
208
+ per-session trace cannot answer "cache-hit over time for *this* agent".
209
+
210
+ With the prefix stable by construction, the existing `prompt_caching` breakpoint
211
+ sits on bytes that stay put — the first (write) turn of a deployment pays the
212
+ cache write once, every subsequent turn reads.
213
+
86
214
  ## The volume
87
215
 
88
216
  Agents, prompts, skills, and tools are **data in SQLite**, not files on a volume.
data/docs/DEPLOY.md CHANGED
@@ -33,9 +33,14 @@ curl localhost:9292/up # {"status":"ok"}
33
33
  ## The process model
34
34
 
35
35
  The image boots **N Falcon worker processes over one SQLite file**
36
- (`WEB_CONCURRENCY`, default 2). That number is a **contract input, not a tuning
36
+ (`WEB_CONCURRENCY`, default 1). That number is a **contract input, not a tuning
37
37
  knob**: it decides which engine semantics hold cluster-wide and which are
38
- per-worker. The contract:
38
+ per-worker. The default is 1 because the per-worker semantics are the product
39
+ — the queue modes (`collect`/`steer`) and FIFO ordering need one
40
+ session actor; raise N only with sticky routing per session in front. It is
41
+ also a [Soak](SOAK.md) precondition: with more than one worker the hourly RSS
42
+ series is a random process per sample, and a soak refuses to start. The
43
+ contract:
39
44
 
40
45
  > Everything here describes N workers of **one** deployment — one graph, replicated.
41
46
  > N *graphs* inside one process is a different contract, and it is
@@ -76,6 +81,18 @@ per-worker. The contract:
76
81
  that buffer defaults to 0** — SIGKILL right after SIGTERM, which cancels the
77
82
  whole drain — so set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS=30` on the
78
83
  service.
84
+ 5. **A periodic tick closes the gap between boots.** Serving workers run a tick
85
+ every `INSIKA_TICK_INTERVAL` (default **60s**, `0` disables) as a child of
86
+ the turn supervisor: it re-drives any outbox record left `:pending` (each
87
+ carries its own claim — every worker may drain), and sweeps orphaned tasks
88
+ **without waiting for a deploy** — which is what recovers the orphans of a
89
+ worker respawned mid-generation (item 3's hole). One worker per window
90
+ sweeps (a single transactional claim), and only `:queued`/`:running` tasks
91
+ untouched for `INSIKA_TICK_STALE_AFTER` (default **900s**) are candidates:
92
+ a live turn is bounded by `turn_timeout`, so anything older cannot be alive.
93
+ **If you raise `turn_timeout` past it, raise `INSIKA_TICK_STALE_AFTER` too** —
94
+ the threshold must exceed the largest `turn_timeout` of the deployment.
95
+ `:waiting`/`:paused` tasks are idle by nature and stay boot recovery's.
79
96
 
80
97
  `deploy/entrypoint.sh` sets `WEB_CONCURRENCY` next to a pointer to this section;
81
98
  this section is the single source of truth for what changing it means.
@@ -86,19 +103,22 @@ this section is the single source of truth for what changing it means.
86
103
  |-----|---------|--------|
87
104
  | `INSIKA_DB` | `/data/insika.db` (in the image) | durable SQLite path (**mount a volume!**) |
88
105
  | `PORT` | `9292` | HTTP bind port |
89
- | `WEB_CONCURRENCY` | `2` | number of Falcon worker processes — a contract input, see [The process model](#the-process-model) |
106
+ | `WEB_CONCURRENCY` | `1` | number of Falcon worker processes — a contract input, see [The process model](#the-process-model) |
90
107
  | `INSIKA_BOOT_ID` | set by `deploy/entrypoint.sh` | boot generation id; the recovery **task sweep** runs once per id (process model, item 3). Unset = every boot sweeps (single-process default) |
91
108
  | `INSIKA_DRAIN_TIMEOUT` | `20` | seconds a stopping worker waits for in-flight turns before abandoning them to the next boot's recovery (process model, item 4). The entrypoint sizes Falcon's `--graceful-stop` from it; on Railway also set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS` ≥ drain + 10 |
109
+ | `INSIKA_TICK_INTERVAL` | `60` | seconds between tick passes — outbox drain + stale recovery sweep (process model, item 5). `0` disables |
110
+ | `INSIKA_TICK_STALE_AFTER` | `900` | seconds a `:queued`/`:running` task must sit untouched before the tick sweeps it. Must exceed the largest `turn_timeout` of the deployment |
92
111
  | `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN` | Bearer for `/v1/responses` and `/v1/agents` (the API contract) |
93
112
  | `ADMIN_TOKEN` | `local-demo` | login token for `/studio` (**change in production**) |
94
113
  | `DEEPSEEK_API_KEY` | — | provider key. **Without it the engine still boots** (`/up` green), but turns fail until it is configured (env or Studio → LLM providers) — cloud resilience |
95
- | `DEEPSEEK_MODEL` | `deepseek-chat` | model |
96
- | `ACHEI_INTERNAL_URL` | — | base URL for data-tools calling back a consumer's internal API (see below) |
114
+ | `DEEPSEEK_MODEL` | `deepseek-v4-flash` | model |
115
+ | `CONSUMER_INTERNAL_URL` | — | base URL for data-tools calling back a consumer's internal API (see below) |
97
116
  | `INSIKA_EGRESS_HOSTS` | — | outbound host allowlist (SSRF guard) |
98
- | `INSIKA_EGRESS_ALLOW_HTTP` / `_ALLOW_PRIVATE` | off | for `http`/loopback callbacks only (**never in cloud**) |
117
+ | `INSIKA_EGRESS_ALLOW_HTTP` / `_ALLOW_PRIVATE` | off | for `http`/loopback callbacks only (**never in cloud**). Also opens inbound **media** fetches (audio/image URLs) to `http`/private targets — a local run serving media over `http://` needs it |
99
118
  | `INSIKA_RELAY_TOKEN` | — | **mounts the relay channel** at `POST /channels/relay/events`, and is the Bearer it requires. Empty = the route does not exist (`404`). See [Channels](CHANNELS.md) |
100
119
  | `INSIKA_RELAY_DELIVER_URL` | — | your callback; the engine POSTs each reply there. Goes through the egress guard |
101
120
  | `INSIKA_RELAY_DELIVER_TOKEN` | — | Bearer the engine sends **to** your callback (optional) |
121
+ | `INSIKA_RELAY_DELIVERY` | `at_end` | how the relay flushes the outbox: `at_end` (one POST) or `progressive` (one POST per balloon — [RFC-0027](CHANNELS.md#delivery-policy)) |
102
122
  | `INSIKA_WIDGET_ORIGINS` | — | exact-match origins allowed to embed the [web widget](CHANNELS.md#the-web-widget), comma-separated. No wildcards. **Half the switch**: with `INSIKA_WIDGET_AGENTS` unset, nothing is mounted (`404`) |
103
123
  | `INSIKA_WIDGET_AGENTS` | — | agent ids a widget visitor may address, comma-separated. The other half of the switch. **A chat rate limit is also required** or the widget answers `503` |
104
124
  | `LITESTREAM_REPLICA_URL` | — | **enables Litestream** (backup/DR). Empty = disabled (default). See below |
@@ -165,10 +185,13 @@ insika env # lists known keys + current values (secrets masked)
165
185
  Checks: env (the schema above), settings schema version (a pending migration →
166
186
  `--fix` applies it), a missing platform `default_model` (`--fix` seeds it from
167
187
  `DEEPSEEK_MODEL`), durable vs ephemeral backend, LLM provider configured,
168
- `ADMIN_TOKEN` set, data-tool definitions still valid, and **prompt files that hold
188
+ `ADMIN_TOKEN` set, data-tool definitions still valid, **prompt files that hold
169
189
  text rather than a serialized object** (a file whose content is a stringified Hash
170
190
  serves a mangled prompt on every turn while looking perfectly healthy — present,
171
- non-empty, and the agent still answers). Settings-schema migrations are **explicit**
191
+ non-empty, and the agent still answers), and **skill drift** — a shared skill whose
192
+ body names one store, a prompt file routing to a skill the agent cannot load, a broken
193
+ companion pair, a stale `eager:` key (see
194
+ [Skills](SKILLS.md#drift-guards)). Settings-schema migrations are **explicit**
172
195
  — no Studio save silently reinterprets old-shape data.
173
196
 
174
197
  ### Data-tool callbacks to a backend — via a tunnel
@@ -178,8 +201,8 @@ cloud** and your backend **on your machine** (`:3000`), expose it over a public
178
201
  `https` tunnel and point the engine at it:
179
202
 
180
203
  ```bash
181
- # in the tool/manifest: base_url = {{env.ACHEI_INTERNAL_URL}}
182
- ACHEI_INTERNAL_URL=https://your-tunnel.example.dev
204
+ # in the tool/manifest: base_url = {{env.CONSUMER_INTERNAL_URL}}
205
+ CONSUMER_INTERNAL_URL=https://your-tunnel.example.dev
183
206
  INSIKA_EGRESS_HOSTS=your-tunnel.example.dev
184
207
  ```
185
208
 
@@ -197,7 +220,7 @@ healthcheck, and a restart policy.
197
220
  2. **Volume**: mount it at `/data` (the default `INSIKA_DB` points there) —
198
221
  without a volume, SQLite is ephemeral and recovery resumes nothing after a
199
222
  redeploy.
200
- 3. **Vars**: `DEEPSEEK_API_KEY`, `OPENCLAW_GATEWAY_TOKEN`, `ACHEI_INTERNAL_URL`,
223
+ 3. **Vars**: `DEEPSEEK_API_KEY`, `OPENCLAW_GATEWAY_TOKEN`, `CONSUMER_INTERNAL_URL`,
201
224
  `INSIKA_EGRESS_HOSTS` (and `WEB_CONCURRENCY` to match your plan/CPU).
202
225
  4. The healthcheck hits `/up`.
203
226
  5. Point your consumer at the service's public URL, with a matching API token
data/docs/EMBEDDING.md CHANGED
@@ -21,7 +21,7 @@ makes your responsibility.
21
21
  # config/initializers/insika.rb
22
22
  INSIKA = Insika.embed(backend: Insika::Stores::SQLite.new(path: Rails.root.join("storage/insika.sqlite3").to_s)) do
23
23
  agent "support" do
24
- model "deepseek-chat"
24
+ model "deepseek-v4-flash"
25
25
  provider :deepseek
26
26
  api_key ENV.fetch("DEEPSEEK_API_KEY")
27
27
  instructions "You answer questions about orders. Be brief."
@@ -57,7 +57,7 @@ Rails' `mount` (and `Rack::URLMap`, and anything else that moves the prefix into
57
57
 
58
58
  ## Why the store is an argument
59
59
 
60
- Until RFC-0017 the engine was not one process — it was one *program*. Two graphs
60
+ The engine was not always one process — it was one *program*. Two graphs
61
61
  built in the same Ruby process shared things they never declared:
62
62
 
63
63
  | What | What actually happened |
@@ -163,15 +163,19 @@ was given a backend never looks at it.
163
163
  | The reactor / `supervised` | You, matching your server |
164
164
  | Recovery of orphaned turns at boot | You, if you want it — the sweep is `Insika::Recovery`, wired by `Insika::Server::Boot` for the standalone deployment, not by `embed` |
165
165
 
166
- ### Embedding is not multi-tenancy
166
+ ### Embedding is not the multi-tenant server
167
167
 
168
168
  Two graphs stop corrupting each other. That is all this contract says. **Who is
169
169
  allowed to talk to which graph** is authorization, and it is not here: `token:` is
170
170
  a single Bearer gating the whole mounted app, exactly as it does for the
171
- standalone server the written decision is [one deployment, one token;
172
- multi-tenancy belongs to the host](SECURITY.md#the-bearer-gate). If your app has
173
- users, put the mounted app behind your own authentication and pass `session:`
174
- yourself do not hand the mount point to the browser.
171
+ standalone server in the default mode. Multi-tenancy is a property of the
172
+ *standalone* server, not of an embed: with `INSIKA_TENANCY=multi_tenant` the
173
+ Bearer resolves to a principal before the routes per-tenant tokens scoped to
174
+ their own sessions, tasks and events, operator tokens with the run of the
175
+ deployment (see [Security](SECURITY.md#the-bearer-gate)). An embedded graph keeps
176
+ no token store, so an embed's tenancy belongs to the host: put the mounted app
177
+ behind your own authentication and pass `session:` yourself — do not hand the
178
+ mount point to the browser.
175
179
 
176
180
  ---
177
181
 
data/docs/EVALS.md CHANGED
@@ -132,7 +132,7 @@ reference:
132
132
  ```
133
133
 
134
134
  ```bash
135
- ruby evals/run.rb --agent ocean-drop --pairwise
135
+ ruby evals/run.rb --agent loja-chocolates --pairwise
136
136
  ```
137
137
 
138
138
  Three outcomes — `better`, `comparable`, `worse` — and two more the panel can produce
@@ -232,6 +232,15 @@ That is what you run before merging a prompt, tool or model change. A case with
232
232
  baseline entry never blocks: it shows as failing in the report, but a brand-new case is
233
233
  not a regression.
234
234
 
235
+ The same gate machinery guards the two automated loops — [Refinement](REFINEMENT.md)
236
+ (cloned-agent replay of a proposed edit) and [Harvest](HARVEST.md) (cloned-agent
237
+ replay of a mined skill). **Judges are mandatory for those gates** in exactly three
238
+ shapes, the P18 lesson: a gate without a recorded baseline refuses, an all-red
239
+ baseline refuses, and a baseline recorded WITH judge scores, replayed with no judge
240
+ configured, refuses — a rubric'd case with no verdict reads as a pass, so the
241
+ candidate would beat a measurement it never took. A store with no golden cases cannot
242
+ gate, and cannot promote.
243
+
235
244
  ## Honest limits
236
245
 
237
246
  - **A judge is a model.** It has taste and it has bad days; that is why the
@@ -271,3 +280,13 @@ needs to score a candidate agent with the same judge, and a second copy of the j
271
280
  would be the worst possible outcome. It stays a **client** even so: it reaches a running
272
281
  deployment over HTTP through `POST /v1/responses` and never reads a store directly.
273
282
  `evals/run.rb` is a thin CLI over it.
283
+
284
+ ## Parity — the shadow criterion
285
+
286
+ The shadow experiment's rule lives in a **deployment-side criterion file** —
287
+ the path `INSIKA_PARITY_CRITERION` points at, prose a human reads and a `yaml`
288
+ block the machine applies, in one file, so there is exactly one place to edit.
289
+ The file's whole bytes are hashed; every shadow pair records that hash,
290
+ and a window whose pairs disagree produces `:invalid`, never a verdict. Editing
291
+ the criterion mid-experiment is *caught*, not averaged away. The fold itself is
292
+ `lib/insika/parity/*` (see [Channels](CHANNELS.md#shadow-mode)).