insika 0.3.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (204) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +296 -0
  3. data/README.md +48 -12
  4. data/bin/insika +725 -0
  5. data/bin/insika-router +87 -0
  6. data/docs/AGENTS.md +116 -406
  7. data/docs/API.md +5 -5
  8. data/docs/ARCHITECTURE.md +3 -2
  9. data/docs/ARTIFACTS.md +137 -0
  10. data/docs/BENCHMARK.md +2 -2
  11. data/docs/CHANNELS.md +14 -14
  12. data/docs/CONTEXT.md +63 -19
  13. data/docs/DEMO.md +80 -0
  14. data/docs/DEPLOY.md +87 -10
  15. data/docs/EMBEDDING.md +1 -1
  16. data/docs/EVALS.md +128 -3
  17. data/docs/FACTS.md +3 -3
  18. data/docs/HARVEST.md +5 -6
  19. data/docs/KNOWLEDGE.md +290 -0
  20. data/docs/LOADTEST.md +17 -29
  21. data/docs/MEDIA.md +128 -0
  22. data/docs/OBSERVABILITY.md +46 -12
  23. data/docs/OUTCOMES.md +137 -0
  24. data/docs/PLUGINS.md +51 -6
  25. data/docs/POLICY.md +222 -0
  26. data/docs/REFINEMENT.md +14 -9
  27. data/docs/RELEASING.md +4 -4
  28. data/docs/ROUTER.md +213 -0
  29. data/docs/RUNNING-LOCAL.md +5 -5
  30. data/docs/SCHEDULING.md +121 -0
  31. data/docs/SECURITY.md +23 -7
  32. data/docs/SKILLS.md +11 -2
  33. data/docs/SOAK.md +3 -3
  34. data/docs/TEMPLATES.md +134 -0
  35. data/docs/TOOLS.md +176 -27
  36. data/docs/WHY.md +1 -1
  37. data/docs/WORKFLOWS.md +2 -2
  38. data/docs/_includes/head_custom.html +5 -0
  39. data/docs/_includes/title.html +13 -0
  40. data/docs/_sass/color_schemes/insika.scss +32 -0
  41. data/docs/_sass/custom/custom.scss +199 -0
  42. data/docs/_sass/custom/setup.scss +26 -0
  43. data/docs/assets/img/favicon.svg +7 -0
  44. data/docs/assets/img/insika-mark.svg +7 -0
  45. data/docs/core-concepts.md +21 -0
  46. data/docs/domain.md +4 -4
  47. data/docs/improve.md +20 -0
  48. data/docs/index.md +8 -5
  49. data/docs/integrate.md +20 -0
  50. data/docs/operate.md +13 -6
  51. data/docs/prompts/ADD-TOOL.md +118 -0
  52. data/docs/prompts/DIAGNOSE-TURN.md +65 -0
  53. data/docs/prompts/GO-LIVE.md +138 -0
  54. data/docs/prompts/RUN-EXAMPLES.md +70 -0
  55. data/docs/reference.md +19 -0
  56. data/docs/ship.md +10 -2
  57. data/docs/start-here.md +18 -0
  58. data/lib/insika/agent_profile.rb +99 -17
  59. data/lib/insika/artifact_signing.rb +82 -0
  60. data/lib/insika/artifact_store.rb +160 -0
  61. data/lib/insika/channel_delivery.rb +1 -1
  62. data/lib/insika/chat_builder.rb +50 -19
  63. data/lib/insika/commands/agent_payload.rb +2 -2
  64. data/lib/insika/commands/backfill_knowledge.rb +145 -0
  65. data/lib/insika/commands/delete_artifact.rb +35 -0
  66. data/lib/insika/commands/delete_concept.rb +34 -0
  67. data/lib/insika/commands/delete_mcp.rb +6 -2
  68. data/lib/insika/commands/delete_tenant_data.rb +15 -3
  69. data/lib/insika/commands/gate_refinement.rb +1 -1
  70. data/lib/insika/commands/refresh_mcp_tools.rb +47 -0
  71. data/lib/insika/commands/restore_concept.rb +34 -0
  72. data/lib/insika/commands/seed_demo_data.rb +31 -0
  73. data/lib/insika/commands/upsert_mcp.rb +6 -3
  74. data/lib/insika/commands/write_concept.rb +57 -0
  75. data/lib/insika/compaction.rb +196 -0
  76. data/lib/insika/context/builder.rb +6 -2
  77. data/lib/insika/context/fragment.rb +4 -1
  78. data/lib/insika/context/priority.rb +8 -0
  79. data/lib/insika/context/providers/briefing.rb +53 -24
  80. data/lib/insika/context/providers/knowledge.rb +108 -0
  81. data/lib/insika/context/providers/prompt.rb +30 -24
  82. data/lib/insika/context/providers/session.rb +46 -10
  83. data/lib/insika/context_trace_store.rb +11 -1
  84. data/lib/insika/cron.rb +189 -0
  85. data/lib/insika/demo/agent_attrs.rb +43 -0
  86. data/lib/insika/demo/golden_cases.rb +81 -0
  87. data/lib/insika/demo/seeder.rb +336 -0
  88. data/lib/insika/doctor.rb +280 -17
  89. data/lib/insika/dsl/definition.rb +3 -2
  90. data/lib/insika/dsl/runtime.rb +64 -79
  91. data/lib/insika/dsl/server_boot.rb +23 -1
  92. data/lib/insika/dsl/system.rb +10 -2
  93. data/lib/insika/dsl.rb +103 -2
  94. data/lib/insika/env_schema.rb +21 -7
  95. data/lib/insika/evals/golden.rb +41 -4
  96. data/lib/insika/evals/judge.rb +47 -2
  97. data/lib/insika/evals/pairwise.rb +11 -0
  98. data/lib/insika/evals/persona.rb +98 -0
  99. data/lib/insika/evals/runner.rb +9 -0
  100. data/lib/insika/evals/simulator.rb +225 -0
  101. data/lib/insika/evals/transport.rb +84 -2
  102. data/lib/insika/event_stream.rb +10 -0
  103. data/lib/insika/executor.rb +295 -55
  104. data/lib/insika/followup_policy.rb +2 -25
  105. data/lib/insika/golden_store.rb +16 -1
  106. data/lib/insika/grounding/matcher.rb +1 -1
  107. data/lib/insika/knowledge.rb +680 -0
  108. data/lib/insika/knowledge_store.rb +140 -0
  109. data/lib/insika/loop_detector.rb +5 -34
  110. data/lib/insika/mcp_client.rb +94 -0
  111. data/lib/insika/mcp_json.rb +74 -0
  112. data/lib/insika/mcp_live_tool.rb +43 -0
  113. data/lib/insika/mcp_store.rb +98 -26
  114. data/lib/insika/mcp_tool_ingestor.rb +30 -8
  115. data/lib/insika/mcp_tool_registry.rb +100 -0
  116. data/lib/insika/media.rb +115 -31
  117. data/lib/insika/message_origin.rb +1 -1
  118. data/lib/insika/middleware.rb +9 -0
  119. data/lib/insika/onboarding.rb +17 -1
  120. data/lib/insika/outcome_store.rb +1 -1
  121. data/lib/insika/overlay_tool_registry.rb +37 -17
  122. data/lib/insika/packaging.rb +2 -2
  123. data/lib/insika/profile_source.rb +15 -1
  124. data/lib/insika/prompt_catalog.rb +10 -0
  125. data/lib/insika/retention.rb +36 -1
  126. data/lib/insika/router/app.rb +157 -0
  127. data/lib/insika/router/backend_pool.rb +98 -0
  128. data/lib/insika/router/hash_ring.rb +55 -0
  129. data/lib/insika/router/proxy_body.rb +34 -0
  130. data/lib/insika/router/session_key.rb +54 -0
  131. data/lib/insika/router.rb +18 -0
  132. data/lib/insika/schedule.rb +177 -0
  133. data/lib/insika/schedule_engine.rb +314 -0
  134. data/lib/insika/schedule_store.rb +208 -0
  135. data/lib/insika/server/app.rb +105 -15
  136. data/lib/insika/server/rack_app.rb +5 -1
  137. data/lib/insika/server/responses.rb +5 -5
  138. data/lib/insika/session_store.rb +34 -4
  139. data/lib/insika/settings_store.rb +8 -1
  140. data/lib/insika/skill_catalog.rb +12 -0
  141. data/lib/insika/soak/runner.rb +4 -4
  142. data/lib/insika/steer_injector.rb +21 -10
  143. data/lib/insika/studio/app.rb +591 -47
  144. data/lib/insika/studio/assets/dist/application.css +1 -1
  145. data/lib/insika/studio/assets/dist/application.js +21 -21
  146. data/lib/insika/studio/forms.rb +57 -5
  147. data/lib/insika/studio/nav_icons.rb +14 -1
  148. data/lib/insika/studio/views/_agent_tab_cache.erb +25 -0
  149. data/lib/insika/studio/views/_agent_tab_config.erb +514 -0
  150. data/lib/insika/studio/views/_agent_tab_history.erb +24 -0
  151. data/lib/insika/studio/views/_agent_tab_loops.erb +54 -0
  152. data/lib/insika/studio/views/_agent_tab_memory.erb +51 -0
  153. data/lib/insika/studio/views/_agent_tab_outcomes.erb +31 -0
  154. data/lib/insika/studio/views/_agent_tab_prompts.erb +108 -0
  155. data/lib/insika/studio/views/_agent_tab_skills.erb +38 -0
  156. data/lib/insika/studio/views/_agents_master.erb +44 -0
  157. data/lib/insika/studio/views/_message.erb +49 -32
  158. data/lib/insika/studio/views/agent_detail.erb +61 -820
  159. data/lib/insika/studio/views/agents.erb +70 -57
  160. data/lib/insika/studio/views/artifact.erb +23 -0
  161. data/lib/insika/studio/views/artifacts.erb +59 -0
  162. data/lib/insika/studio/views/evals.erb +2 -2
  163. data/lib/insika/studio/views/facts.erb +1 -1
  164. data/lib/insika/studio/views/funnel.erb +1 -1
  165. data/lib/insika/studio/views/home.erb +106 -67
  166. data/lib/insika/studio/views/knowledge.erb +123 -0
  167. data/lib/insika/studio/views/layout.erb +14 -11
  168. data/lib/insika/studio/views/mcp.erb +174 -80
  169. data/lib/insika/studio/views/session.erb +231 -177
  170. data/lib/insika/studio/views/settings.erb +50 -1
  171. data/lib/insika/studio/views/skills.erb +1 -1
  172. data/lib/insika/studio/views/tools.erb +24 -9
  173. data/lib/insika/telemetry/recorder.rb +49 -1
  174. data/lib/insika/templates/browser-agent/README.md +36 -0
  175. data/lib/insika/templates/browser-agent/agent.rb +49 -0
  176. data/lib/insika/templates/daily-digest/README.md +47 -0
  177. data/lib/insika/templates/daily-digest/agent.rb +77 -0
  178. data/lib/insika/templates/repo-explorer/README.md +36 -0
  179. data/lib/insika/templates/repo-explorer/agent.rb +45 -0
  180. data/lib/insika/templates/research-analyst/README.md +26 -0
  181. data/lib/insika/templates/research-analyst/agent.rb +68 -0
  182. data/lib/insika/templates/review-panel/README.md +20 -0
  183. data/lib/insika/templates/review-panel/agent.rb +50 -0
  184. data/lib/insika/templates/travel-planner/README.md +35 -0
  185. data/lib/insika/templates/travel-planner/agent.rb +87 -0
  186. data/lib/insika/templates.rb +112 -0
  187. data/lib/insika/tick.rb +24 -12
  188. data/lib/insika/timezone.rb +45 -0
  189. data/lib/insika/tool_batch.rb +67 -0
  190. data/lib/insika/tool_usage_report.rb +162 -0
  191. data/lib/insika/tools/generate_image.rb +52 -7
  192. data/lib/insika/tools/load_knowledge.rb +74 -0
  193. data/lib/insika/tools/run_persona_eval.rb +328 -0
  194. data/lib/insika/tools/save_artifact.rb +95 -0
  195. data/lib/insika/turn_budget.rb +91 -0
  196. data/lib/insika/turn_output.rb +1 -1
  197. data/lib/insika/turn_state.rb +15 -4
  198. data/lib/insika/version.rb +1 -1
  199. data/lib/insika/wiring/graph.rb +184 -12
  200. data/lib/insika/wiring/graph_chat.rb +102 -0
  201. data/lib/insika.rb +64 -0
  202. metadata +109 -5
  203. data/docs/build.md +0 -14
  204. data/docs/understand.md +0 -10
data/docs/TOOLS.md CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  title: Tools
3
- parent: Build an agent
4
- nav_order: 2
3
+ parent: Core concepts
4
+ nav_order: 3
5
5
  permalink: /tools/
6
6
  ---
7
7
 
@@ -12,15 +12,16 @@ kinds, and the distinction that matters is **who can change one at runtime**:
12
12
 
13
13
  | | **Code tool** | **Data tool** | **MCP tool** |
14
14
  |---|---|---|---|
15
- | What | a Ruby class (`< RubyLLM::Tool`) | an HTTP call described by config, no Ruby | an MCP server's tool, ingested |
16
- | Lives | in the deployment image | as a row in SQLite | as data-tool rows in SQLite |
17
- | Editable at runtime | no (shipped in the image) | **yes** (DSL / API / manifest / Studio) | **yes** (re-ingest) |
18
- | Reach for it when | logic must run in-process (file edit, shell, subagent) | calling an external HTTP API | adopting a whole MCP toolset at once |
15
+ | What | a Ruby class (`< RubyLLM::Tool`) | an HTTP call described by config, no Ruby | an MCP server's tool, called LIVE |
16
+ | Lives | in the deployment image | as a row in SQLite | on the MCP server, behind a live client |
17
+ | Editable at runtime | no (shipped in the image) | **yes** (DSL / API / manifest / Studio) | **yes** — enable/edit the *instance* (DSL / CLI / API / JSON import / Studio); the server owns its own tools |
18
+ | Reach for it when | logic must run in-process (file edit, shell, subagent) | calling an external HTTP API | adopting a whole external MCP server's toolset |
19
19
 
20
- **MCP tools are not a separate runtime type.** An MCP ingestor discovers an MCP
21
- server's tools and turns each into an HTTP **data tool** that posts a JSON-RPC
22
- `tools/call`, tagged with a `group` naming the source instance. (Only
23
- HTTP-transport MCP servers are ingestible; stdio is rejected.)
20
+ **MCP tools are not data tools.** Configuring an enabled MCP **instance** (any
21
+ surface below) is enough — its tools appear automatically, tagged
22
+ `mcp:<instance>`, and each CALL goes straight to the server through a live,
23
+ held client (stdio process / Streamable HTTP / SSE, with the full protocol
24
+ handshake) — never a frozen snapshot. See [MCP servers](#mcp-servers) below.
24
25
 
25
26
  Code tools **win name collisions** — you cannot register a data tool whose name
26
27
  shadows a code tool.
@@ -49,6 +50,11 @@ the one you create and change without a rebuild. See
49
50
  }
50
51
  ```
51
52
 
53
+ `{{secret.api_token}}` above is only real coming through the **manifest**
54
+ write path (`POST /v1/tools/manifest`) — writing this same shape via the DSL
55
+ or Studio needs the literal header value instead; see
56
+ "[The one gotcha](#the-one-gotcha-envsecret-templating-is-manifest-only)" below.
57
+
52
58
  ### Parameters: the schema is the contract
53
59
 
54
60
  `parameters` is **JSON Schema**, and it reaches the provider verbatim — it is the only
@@ -87,17 +93,25 @@ allow never becomes a request: it returns an `{ error: … }` naming the path
87
93
  retries against. Structure is strict; a scalar may arrive in its lossless string form
88
94
  (`"2"`, `"true"`) and is never coerced — what the model sent is what the request carries.
89
95
 
90
- **Placeholders** are resolved at turn time:
96
+ **Placeholders**. Two of these resolve at turn time; `{{secret.*}}` resolves
97
+ once, at ingestion — see the gotcha below before reaching for it:
91
98
 
92
- - `{{param}}` — a declared top-level parameter, filled from the model's call.
99
+ - `{{param}}` — a declared top-level parameter, filled from the model's call,
100
+ every turn.
93
101
  - `{{ctx.*}}` — turn context set **server-side, never by the model**: a closed set
94
102
  of `chat_id`, `store_id`, `agent_id`, `tenant`, `image_url`. This is how a tool knows *which*
95
103
  session/agent it is acting for without trusting the model. `image_url` is the
96
104
  first image part on the message (a photo for analysis outside the prompt);
97
- absent when the turn carried none.
98
- - `{{secret.*}}` — allowed **only** inside a header named in `secret_headers`.
99
- A secret placeholder anywhere else is rejected (it would leak unmasked). The
100
- real secret value is injected at provision time and never lives on disk.
105
+ absent when the turn carried none. Resolved every turn, like `{{param}}`.
106
+ - `{{secret.*}}` — **only resolved on the manifest ingestion path**
107
+ (`POST /v1/tools/manifest`; see "[The one gotcha](#the-one-gotcha-envsecret-templating-is-manifest-only)"
108
+ below), and only once — the resolved value is what gets stored, the token
109
+ itself never lives on disk and is never re-read per turn. Allowed **only**
110
+ inside a header named in `secret_headers`. Written any other way — DSL,
111
+ Studio, or anywhere outside a `secret_headers` header — a
112
+ `{{secret.*}}` is not a credential the engine knows how to fill; it is an
113
+ undeclared parameter, and tool registration refuses it exactly like it
114
+ refuses any other unknown placeholder.
101
115
 
102
116
  **Validation** happens on ingestion. Common rejections:
103
117
 
@@ -183,7 +197,7 @@ backend, not of whoever calls it. Every agent sharing the tool gets the same val
183
197
  > there **preserves** it — the form carries the stored values through instead of
184
198
  > replacing the record with only what it shows.
185
199
 
186
- ## Evidence: the lean envelope and grounding (RFC-0029)
200
+ ## Evidence: the lean envelope and grounding
187
201
 
188
202
  A catalog tool returns products; the model should only ever quote the ones the tool
189
203
  actually returned — the store dies of a SKU the model invented. `evidence` is the
@@ -261,16 +275,117 @@ reload, no restart):
261
275
  3. **Manifest** — `POST /v1/tools/manifest`. Partial failure is isolated: one
262
276
  malformed tool becomes an `errors[]` entry; only a structural manifest error
263
277
  fails the whole request. The response reports `{ version, created, updated, errors }`.
264
- 4. **MCP ingestion** — import a server; each of its tools becomes a data tool.
265
-
266
- ### The one gotcha: env templating is manifest-only
267
-
268
- `{{env.*}}` (and `{{secret.*}}`) are substituted **at ingestion, on the manifest
269
- path**. Other write paths do **not** resolve `{{env.*}}` — a literal
270
- `{{env.API_URL}}` there fails the `http`/`https` URL check and 422s. Rule:
271
- **manifest tools may template the URL with `{{env.*}}`; tools written any other
272
- way must ship a literal URL.** `{{ctx.*}}` and `{{param}}` work everywhere (they
273
- resolve at turn time, not ingestion).
278
+ 4. ~~MCP ingestion~~ — retired. An MCP server's tools are no
279
+ longer written into this store at all; see [MCP servers](#mcp-servers).
280
+
281
+ ### The one gotcha: env/secret templating is manifest-only
282
+
283
+ `{{env.*}}` and `{{secret.*}}` are substituted **at ingestion, on the manifest
284
+ path, once** — the resolved literal is what gets stored; the token itself
285
+ never survives to a turn. Other write paths (DSL, Studio) do
286
+ **not** resolve either: a literal `{{env.API_URL}}` in a URL fails the
287
+ `http`/`https` check and 422s; a literal `{{secret.X}}` anywhere — including
288
+ inside a header named in `secret_headers` — fails tool registration the same
289
+ way an unknown parameter would (`ToolDefinition.build`'s placeholder check
290
+ does not special-case it). Rule: **manifest tools may template a URL with
291
+ `{{env.*}}` and a `secret_headers` header with `{{secret.*}}`; tools written
292
+ any other way must ship literal values** — a real URL, and a real (masked on
293
+ read) header value. `{{ctx.*}}` and `{{param}}` work everywhere (they resolve
294
+ at turn time, not ingestion).
295
+
296
+ ## MCP servers
297
+
298
+ An MCP **instance** is durable config — transport, target, credentials, an
299
+ `enabled` flag — held in its own store, separate from data tools. Once an
300
+ instance is enabled, its tools appear in the catalog automatically (group
301
+ `mcp:<instance>`, `side_effect: true`), and every call goes straight to the
302
+ server through a live, held client — the runtime never converts an MCP tool
303
+ into a stored data tool.
304
+
305
+ **Three transports**, picked by `transport:`:
306
+
307
+ | Transport | Target | Notes |
308
+ |---|---|---|
309
+ | `stdio` | `command` + `args`, run as a child process, `env` is its process environment | requires `INSIKA_MCP_STDIO=1` — see below |
310
+ | `http` | `url` + `headers` (Streamable HTTP, the modern default) | egress-guarded like any outbound URL |
311
+ | `sse` | `url` + `headers` | same egress guard as `http` |
312
+
313
+ **The stdio gate.** A stdio instance is arbitrary command execution by
314
+ config — it saves, but refuses to start ("stdio disabled by env") until the
315
+ operator sets `INSIKA_MCP_STDIO=1` (config-over-convention, the same pattern
316
+ as the egress envs). `http`/`sse` need no such gate; their URL is checked by
317
+ the normal egress allowlist instead.
318
+
319
+ **Credentials are never visible in plaintext.** `env` (stdio) and `headers`
320
+ (http/sse) mask every value as `__OCULTO__` on read, everywhere (CLI, API,
321
+ Studio). On write, sending the sentinel back **preserves** the stored value; a
322
+ new string **replaces** it; `""` (or omitting the key) **clears** it — the
323
+ same per-key reconciliation `llm_providers` api keys use.
324
+
325
+ **Discovery vs execution.** `insika mcp refresh <name>` (or `POST
326
+ /v1/mcp/:name/import`, kept as that action's route since before the live
327
+ registry) connects live, lists the server's tools, and caches the result
328
+ (`tools_cache`) purely for display — the Studio panel and `insika doctor`.
329
+ **Execution never reads that cache**: a live turn always goes through the
330
+ held client, which does its own discovery on first use regardless of whether
331
+ `refresh` ever ran.
332
+
333
+ ### Configuring an instance
334
+
335
+ 1. **DSL** — inside `Insika.system { … }` or a single `Insika.agent { … }`:
336
+
337
+ ```ruby
338
+ mcp "tavily", transport: :http, url: "https://mcp.tavily.com/mcp",
339
+ headers: { "Authorization" => "Bearer #{ENV['TAVILY_KEY']}" }
340
+ mcp "filesystem", transport: :stdio, command: "npx",
341
+ args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
342
+ ```
343
+
344
+ Code is the **template**: transport/command/args/url/description always
345
+ follow the declaration on every boot. But once the instance exists, its
346
+ `enabled` flag and its credentials are the **operator's** — a Studio/CLI/API
347
+ edit made after boot is never clobbered back by the next restart.
348
+
349
+ 2. **CLI** — `insika mcp list | add | remove | import <file.json> | test <name> |
350
+ refresh <name>`. `add` takes `--name`, `--transport`, `--command`/`--arg`
351
+ (repeatable) or `--url`/`--header "Name: value"` (repeatable)/`--env
352
+ "KEY=value"` (repeatable), `--description`, `--disabled`. `test` connects
353
+ live and prints the discovered tools (or the error) without any special
354
+ setup; `refresh` does the same and additionally updates `tools_cache`.
355
+
356
+ 3. **JSON import/export** — the same `mcpServers` shape every MCP client
357
+ (Claude Desktop, Cursor, …) already uses:
358
+
359
+ ```jsonc
360
+ {
361
+ "mcpServers": {
362
+ "tavily": { "url": "https://mcp.tavily.com/mcp", "headers": { "Authorization": "Bearer …" } },
363
+ "filesystem": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"] }
364
+ }
365
+ }
366
+ ```
367
+
368
+ `insika mcp import FILE.json` upserts every entry (a bare `command` implies
369
+ `stdio`; a bare `url` implies `http`; add `"transport": "sse"` explicitly
370
+ for SSE — the bare format has no other way to spell it). The same parser
371
+ backs `PUT /v1/mcp` and Studio's "Import JSON" box; `export` produces the
372
+ document back with secrets masked as `__OCULTO__`, so round-tripping an
373
+ export never wipes a stored credential.
374
+
375
+ 4. **HTTP API** (operator-only, gateway Bearer):
376
+ - `GET /v1/mcp` — every instance, masked.
377
+ - `GET /v1/mcp/:name` — one instance, masked.
378
+ - `PUT /v1/mcp` — upsert (body = the instance attrs, `name` required).
379
+ - `DELETE /v1/mcp/:name` — remove (idempotent).
380
+ - `POST /v1/mcp/:name/import` — refresh (connect live, list tools, cache).
381
+
382
+ 5. **Studio** — the `/studio/mcp` panel (create/edit/delete). The form is
383
+ transport-aware (stdio shows command/args/env, http/sse shows
384
+ url/headers); each instance shows a status chip ("N tool(s)", "untested",
385
+ "stdio disabled", or "off") and its discovered tools from `tools_cache`; a
386
+ "Test connection" button dispatches the same `refresh_mcp_tools` seam as
387
+ `insika mcp test`; an "Import JSON" box takes a `mcpServers` document and
388
+ fans it out into one `upsert_mcp` per entry.
274
389
 
275
390
  ## Making it appear — and enter the tool-loop
276
391
 
@@ -361,9 +476,43 @@ Work down this checklist:
361
476
  4. **URL literal?** For non-manifest tools, an unresolved `{{env.*}}` would have
362
477
  422'd at import — re-check the definition.
363
478
 
479
+ ## The `save_artifact` built-in
480
+
481
+ `save_artifact` is a **registry tool** — it obeys the same per-agent
482
+ `tools_allow` as any data tool, and an agent that did not name it cannot call
483
+ it (`tools_allow: %w[save_artifact]`). The agent hands in `title` + `content`
484
+ and gets the URL back; the tenant is bound from the turn, never a parameter the
485
+ model types. See [Artifacts](ARTIFACTS.md) for the tool contract, the serving
486
+ routes, the signed link and the retention/LGPD reach.
487
+
488
+ ## The usage report — `insika tools:report`
489
+
490
+ The per-session trace answers "what did this conversation call"; nothing used to
491
+ answer "what does this agent carry and never use". The report aggregates the
492
+ stored traces per agent (tasks → sessions → `tool_traces`, the same read the
493
+ Studio does) and flags three shapes:
494
+
495
+ - **`never_called`** — in `tools_allow`, zero calls in any stored trace. Dead
496
+ weight: its schema ships on every request and buys nothing.
497
+ - **`error_rate`** — over 30% conventional errors (the trace's `ok` flag) inside
498
+ the window (default 14 days). Either the tool is broken or the model cannot
499
+ hold its contract.
500
+ - **`stale`** — called at some point, but not once inside the window.
501
+
502
+ ```bash
503
+ insika tools:report # every stored agent
504
+ insika tools:report --agent store-support # one agent
505
+ insika tools:report --days 30 --json # wider window, machine-readable
506
+ ```
507
+
508
+ Read-only by design: the report names candidates, the **operator** removes — a
509
+ flagged tool may still be the one a rare but critical flow needs. Counts are
510
+ "at least", never exact: the trace keeps a capped tail per session.
511
+
364
512
  ## See also
365
513
 
366
514
  - [Agents](AGENTS.md) — allowlists, groups, and per-agent tool exposure.
515
+ - [Artifacts](ARTIFACTS.md) — the report destination: the tool, the routes, the signed link.
367
516
  - [Plugins](PLUGINS.md) — where a code tool comes from, and how to package one.
368
517
  - [Security](SECURITY.md) — egress, sandbox, and approval gating together.
369
518
  - [Architecture](ARCHITECTURE.md) — the tool-loop and side-effect checkpointing.
data/docs/WHY.md CHANGED
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  title: Why Insika
3
- parent: Understand the idea
3
+ parent: Start here
4
4
  nav_order: 1
5
5
  permalink: /why/
6
6
  ---
data/docs/WORKFLOWS.md CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  title: Workflows
3
- parent: Build an agent
4
- nav_order: 5
3
+ parent: Core concepts
4
+ nav_order: 6
5
5
  permalink: /workflows/
6
6
  ---
7
7
 
@@ -0,0 +1,5 @@
1
+ {%- comment -%}
2
+ Appended inside <head> on every page by the theme. The theme's own favicon
3
+ include only looks for a legacy /favicon.ico, so the SVG icon is declared here.
4
+ {%- endcomment -%}
5
+ <link rel="icon" href="{{ '/assets/img/favicon.svg' | relative_url }}" type="image/svg+xml">
@@ -0,0 +1,13 @@
1
+ {%- comment -%}
2
+ Overrides the theme's title.html so the sidebar shows the pillar mark next to
3
+ the wordmark. The theme's own `site.logo` path swaps the title for a single
4
+ background image, which would drop the text — and the text is what the
5
+ browser tab, the skip link and screen readers rely on.
6
+ {%- endcomment -%}
7
+ <span class="site-title-mark" aria-hidden="true">
8
+ <svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg" focusable="false">
9
+ <rect x="3" y="3" width="18" height="3.6" rx="1"/>
10
+ <rect x="8.7" y="7.6" width="6.6" height="8.8" class="shaft"/>
11
+ <rect x="3" y="17.4" width="18" height="3.6" rx="1"/>
12
+ </svg>
13
+ </span>{{ site.title }}
@@ -0,0 +1,32 @@
1
+ // Insika's colour scheme for Just the Docs.
2
+ //
3
+ // Selected by `color_scheme: insika` in _config.yml; the theme picks the file up
4
+ // by name and needs no other wiring. This file only maps the palette tokens from
5
+ // _sass/custom/setup.scss onto the theme's own `!default` variables — anything
6
+ // that is not a theme variable belongs in _sass/custom/custom.scss.
7
+ //
8
+ // The palette is warm stone with a terracotta accent: the name is Zulu for the
9
+ // pillar that carries a structure, and the docs should read like one — quiet
10
+ // neutrals, a single load-bearing colour. It is deliberately unlike the blue and
11
+ // purple that every other Ruby docs site defaults to.
12
+
13
+ $color-scheme: insika;
14
+
15
+ $body-background-color: $white;
16
+ $body-heading-color: $ink-900;
17
+ $body-text-color: $ink-700;
18
+ $link-color: $terracotta-100;
19
+ $nav-child-link-color: $ink-700;
20
+ $sidebar-color: $stone-050;
21
+ $border-color: $stone-200;
22
+ $base-button-color: $stone-100;
23
+ $btn-primary-color: $terracotta-100;
24
+ $code-background-color: $stone-100;
25
+ $feedback-color: darken($sidebar-color, 3%);
26
+ $table-background-color: $white;
27
+ $search-background-color: $white;
28
+ $search-result-preview-color: $ink-500;
29
+
30
+ // The theme ships accessible-pygments; github-light is the one that sits calmly
31
+ // on a warm background instead of fighting it.
32
+ @import "./vendor/accessible-pygments/github-light";
@@ -0,0 +1,199 @@
1
+ // Insika's own styling on top of Just the Docs.
2
+ //
3
+ // Everything that is NOT one of the theme's `!default` variables lives here;
4
+ // the palette itself is in _sass/color_schemes/insika.scss. Kept deliberately
5
+ // small: the theme already handles layout, search and responsiveness, and every
6
+ // rule below exists because a specific page needed it.
7
+ //
8
+ // IMPORTANT: these pages are read in three places — this site, GitHub, and the
9
+ // raw-markdown API (`GET /docs/<name>.md`). Styling is therefore driven by
10
+ // kramdown attribute lists (`{: .card-grid }`) on ORDINARY markdown, never by
11
+ // raw HTML blocks: a list still reads as a list everywhere else.
12
+
13
+ // ---------------------------------------------------------------------------
14
+ // Typography
15
+ // ---------------------------------------------------------------------------
16
+
17
+ // The theme's 1.6 content line-height is tight for pages this long.
18
+ .main-content {
19
+ line-height: 1.65;
20
+
21
+ h1,
22
+ h2,
23
+ h3 {
24
+ letter-spacing: -0.01em;
25
+ }
26
+
27
+ // A rule above every h2 turns a long page into visible sections. h2 is the
28
+ // level the search index already uses as a landmark, so it is the honest one
29
+ // to draw.
30
+ h2 {
31
+ padding-top: $sp-5;
32
+ margin-top: $sp-7;
33
+ border-top: $border $border-color;
34
+ }
35
+
36
+ h2:first-of-type {
37
+ margin-top: $sp-4;
38
+ padding-top: 0;
39
+ border-top: 0;
40
+ }
41
+
42
+ // Inline code appears in nearly every sentence here (env vars, method names).
43
+ // A tinted chip separates it from prose without shouting.
44
+ p > code,
45
+ li > code,
46
+ td > code,
47
+ h2 > code,
48
+ h3 > code,
49
+ h4 > code {
50
+ padding: 0.12em 0.32em;
51
+ background-color: $stone-100;
52
+ border: $border $stone-200;
53
+ border-radius: 3px;
54
+ font-size: 0.85em;
55
+ }
56
+
57
+ blockquote {
58
+ margin-left: 0;
59
+ padding: $sp-2 $sp-4;
60
+ border-left: 3px solid $terracotta-000;
61
+ background-color: $stone-050;
62
+ color: $ink-700;
63
+
64
+ > :first-child { margin-top: 0; }
65
+ > :last-child { margin-bottom: 0; }
66
+ }
67
+ }
68
+
69
+ // ---------------------------------------------------------------------------
70
+ // Home hero
71
+ // ---------------------------------------------------------------------------
72
+
73
+ // `{: .hero-tagline }` on the one line under the h1.
74
+ .hero-tagline {
75
+ max-width: 34rem;
76
+ margin-bottom: $sp-6;
77
+ color: $ink-700;
78
+ font-size: $font-size-6;
79
+ font-weight: 300;
80
+ line-height: 1.45;
81
+ }
82
+
83
+ // ---------------------------------------------------------------------------
84
+ // Card grid
85
+ // ---------------------------------------------------------------------------
86
+
87
+ // `{: .card-grid }` on a markdown list whose items read
88
+ // `**[Title](page.md)** — what the page answers.`
89
+ //
90
+ // The em dash is the split point: the bold link becomes the card title and the
91
+ // remainder becomes the card's body. No HTML in the markdown, so GitHub and the
92
+ // raw-markdown API still see a plain, readable list.
93
+ .main-content ul.card-grid {
94
+ display: grid;
95
+ grid-template-columns: repeat(auto-fit, minmax(15rem, 1fr));
96
+ gap: $sp-3;
97
+ margin: $sp-5 0;
98
+ padding: 0;
99
+ list-style: none;
100
+
101
+ > li {
102
+ margin: 0;
103
+ padding: $sp-4;
104
+ border: $border $border-color;
105
+ border-radius: 6px;
106
+ background-color: $body-background-color;
107
+ line-height: 1.5;
108
+ transition: border-color 150ms ease, background-color 150ms ease;
109
+
110
+ &::before { content: none; }
111
+
112
+ &:hover {
113
+ border-color: $terracotta-000;
114
+ background-color: $stone-050;
115
+ }
116
+
117
+ // The bold-wrapped link is the card's title.
118
+ > strong {
119
+ display: block;
120
+ margin-bottom: $sp-1;
121
+ font-size: $font-size-5;
122
+ font-weight: 600;
123
+
124
+ > a {
125
+ color: $body-heading-color;
126
+ text-decoration: none;
127
+ background-image: none;
128
+
129
+ &:hover { color: $terracotta-100; }
130
+ }
131
+ }
132
+
133
+ color: $ink-500;
134
+ font-size: $font-size-4;
135
+ }
136
+ }
137
+
138
+ // ---------------------------------------------------------------------------
139
+ // Navigation
140
+ // ---------------------------------------------------------------------------
141
+
142
+ .site-title {
143
+ font-weight: 600;
144
+ letter-spacing: -0.02em;
145
+ }
146
+
147
+ // The sidebar is six sections deep; a hairline between top-level entries makes
148
+ // the grouping legible at a glance instead of one 40-item column.
149
+ .site-nav > .nav-list > .nav-list-item + .nav-list-item {
150
+ border-top: $border $border-color;
151
+ }
152
+
153
+ .nav-list .nav-list-item .nav-list-link.active {
154
+ font-weight: 600;
155
+ box-shadow: inset 2px 0 0 $terracotta-100;
156
+ }
157
+
158
+ // ---------------------------------------------------------------------------
159
+ // Tables
160
+ // ---------------------------------------------------------------------------
161
+
162
+ // Reference tables here are wide (env var, default, meaning). Zebra striping
163
+ // and a tinted header make a 20-row table scannable.
164
+ .main-content table {
165
+ th {
166
+ background-color: $stone-100;
167
+ font-size: $font-size-3;
168
+ text-transform: uppercase;
169
+ letter-spacing: 0.04em;
170
+ }
171
+
172
+ tbody tr:nth-child(even) {
173
+ background-color: $stone-050;
174
+ }
175
+ }
176
+
177
+ // ---------------------------------------------------------------------------
178
+ // Site title
179
+ // ---------------------------------------------------------------------------
180
+
181
+ // _includes/title.html puts the pillar mark inline before the wordmark.
182
+ .site-title {
183
+ display: inline-flex;
184
+ align-items: center;
185
+ gap: $sp-2;
186
+ }
187
+
188
+ .site-title-mark {
189
+ display: inline-flex;
190
+ flex: 0 0 auto;
191
+
192
+ svg {
193
+ width: 1.05em;
194
+ height: 1.05em;
195
+ fill: $terracotta-100;
196
+
197
+ .shaft { fill: $terracotta-000; }
198
+ }
199
+ }
@@ -0,0 +1,26 @@
1
+ // Insika's palette tokens.
2
+ //
3
+ // This file is the ONE place the raw colours are written. It lives in
4
+ // custom/setup rather than in the colour scheme because Just the Docs imports
5
+ // it before every scheme it compiles — including its own stock light and dark
6
+ // stylesheets, which also pull in _sass/custom/custom.scss and would otherwise
7
+ // fail on an undefined variable.
8
+ //
9
+ // _sass/color_schemes/insika.scss maps these onto the theme's own variables.
10
+
11
+ // Warm neutrals. The greys carry a little red so that text on the off-white
12
+ // sidebar does not read as blue-grey next to the terracotta accent.
13
+ $ink-900: #16191d !default; // headings
14
+ $ink-700: #3d4148 !default; // body copy
15
+ $ink-500: #6b6a68 !default; // muted
16
+ $stone-050: #faf8f5 !default; // sidebar
17
+ $stone-100: #f6f3ef !default; // code blocks, table stripes
18
+ $stone-200: #e8e2d9 !default; // borders
19
+
20
+ // The single accent. #a8431e clears 4.5:1 on both #fff and the sidebar, which
21
+ // the lighter, prettier terracottas do not — links are the one thing here that
22
+ // cannot trade contrast for warmth.
23
+ $terracotta-000: #c2542a !default;
24
+ $terracotta-100: #a8431e !default;
25
+ $terracotta-200: #8a3617 !default;
26
+ $terracotta-300: #6b2911 !default;
@@ -0,0 +1,7 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" role="img" aria-label="Insika">
2
+ <!-- A pillar seen head-on: capital, shaft, base. It doubles as the "I" of
3
+ Insika, which is why the shaft is centred and the slabs overhang. -->
4
+ <rect x="3" y="3" width="18" height="3.6" rx="1" fill="#a8431e"/>
5
+ <rect x="8.7" y="7.6" width="6.6" height="8.8" fill="#c2542a"/>
6
+ <rect x="3" y="17.4" width="18" height="3.6" rx="1" fill="#a8431e"/>
7
+ </svg>
@@ -0,0 +1,7 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" role="img" aria-label="Insika">
2
+ <!-- A pillar seen head-on: capital, shaft, base. It doubles as the "I" of
3
+ Insika, which is why the shaft is centred and the slabs overhang. -->
4
+ <rect x="3" y="3" width="18" height="3.6" rx="1" fill="#a8431e"/>
5
+ <rect x="8.7" y="7.6" width="6.6" height="8.8" fill="#c2542a"/>
6
+ <rect x="3" y="17.4" width="18" height="3.6" rx="1" fill="#a8431e"/>
7
+ </svg>
@@ -0,0 +1,21 @@
1
+ ---
2
+ title: Core concepts
3
+ nav_order: 3
4
+ has_children: true
5
+ permalink: /core-concepts/
6
+ ---
7
+
8
+ # Core concepts
9
+
10
+ An agent is data: a profile, what it is allowed to do, its tools, its skills, and
11
+ what fills its prompt. These six pages are the vocabulary everything else on this
12
+ site assumes. Each one has a runnable counterpart under
13
+ [`examples/`](https://github.com/guizaols/insika/tree/main/examples/).
14
+
15
+ - **[Agents](AGENTS.md)** — the profile, the three ways to create one, and every key on it.
16
+ - **[Limits and policy](POLICY.md)** — the five layers that decide what an agent may do and what stops it.
17
+ - **[Tools](TOOLS.md)** — code tools, data-defined tools, MCP servers, and why a tool call goes missing.
18
+ - **[Skills](SKILLS.md)** — playbooks the agent loads only when the conversation calls for them.
19
+ - **[Context](CONTEXT.md)** — what fills a turn's prompt, the token budget, and cross-session memory.
20
+ - **[Workflows](WORKFLOWS.md)** — when the order of work belongs in Ruby instead of a prompt.
21
+ {: .card-grid }
data/docs/domain.md CHANGED
@@ -1,13 +1,13 @@
1
1
  ---
2
2
  title: The domain-free core
3
- parent: Operate & prove it
4
- nav_order: 9
3
+ parent: Reference
4
+ nav_order: 1
5
5
  permalink: /domain/
6
6
  ---
7
7
 
8
8
  # The domain-free core — what ships, what a deployment declares, and how to clear it
9
9
 
10
- The engine is domain-free by construction (RFC-0036): the gem carries no store
10
+ The engine is domain-free by construction: the gem carries no store
11
11
  vocabulary, no persona, and no fixed conversation language. This page is the
12
12
  removability map — for every artifact that could make a deployment look like
13
13
  "the Brazilian store harness", here is what ships, what the doctor reports, and
@@ -90,7 +90,7 @@ metadata domain: "e-commerce-pt-BR"
90
90
  ```
91
91
 
92
92
  - **Outcome funnel** — `funnel:` on the profile (see
93
- [Outcomes](AGENTS.md#outcomes--business-results-over-real-traffic-ws7)).
93
+ [Outcomes](OUTCOMES.md#outcomes--business-results-over-real-traffic)).
94
94
  Vocabulary note: in the gem and the doctor output it is an **outcome
95
95
  funnel**, never "conversion" — the stage names are the deployment's, and a
96
96
  bare install shows no funnel and no stage names at all.
data/docs/improve.md ADDED
@@ -0,0 +1,20 @@
1
+ ---
2
+ title: Improve
3
+ nav_order: 7
4
+ has_children: true
5
+ permalink: /improve/
6
+ ---
7
+
8
+ # Improve
9
+
10
+ The loops that make an agent better than it was last month — measure it, read
11
+ its own traffic back, and turn finished conversations into something the next
12
+ conversation can use. Every one of them ends at a human approval.
13
+
14
+ - **[Evals](EVALS.md)** — the cases that grade an agent, and the gate that stops a regression.
15
+ - **[Refinement](REFINEMENT.md)** — production traffic read back as a ranked report of what broke.
16
+ - **[Outcomes and follow-ups](OUTCOMES.md)** — what the traffic was worth, and coming back on a promise.
17
+ - **[Knowledge](KNOWLEDGE.md)** — durable concepts extracted from finished conversations.
18
+ - **[Facts](FACTS.md)** — distilled customer memory, approved one fact at a time.
19
+ - **[Harvest](HARVEST.md)** — skills mined from real traffic, promoted only if two gates hold.
20
+ {: .card-grid }