insika 0.3.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +296 -0
- data/README.md +48 -12
- data/bin/insika +725 -0
- data/bin/insika-router +87 -0
- data/docs/AGENTS.md +116 -406
- data/docs/API.md +5 -5
- data/docs/ARCHITECTURE.md +3 -2
- data/docs/ARTIFACTS.md +137 -0
- data/docs/BENCHMARK.md +2 -2
- data/docs/CHANNELS.md +14 -14
- data/docs/CONTEXT.md +63 -19
- data/docs/DEMO.md +80 -0
- data/docs/DEPLOY.md +87 -10
- data/docs/EMBEDDING.md +1 -1
- data/docs/EVALS.md +128 -3
- data/docs/FACTS.md +3 -3
- data/docs/HARVEST.md +5 -6
- data/docs/KNOWLEDGE.md +290 -0
- data/docs/LOADTEST.md +17 -29
- data/docs/MEDIA.md +128 -0
- data/docs/OBSERVABILITY.md +46 -12
- data/docs/OUTCOMES.md +137 -0
- data/docs/PLUGINS.md +51 -6
- data/docs/POLICY.md +222 -0
- data/docs/REFINEMENT.md +14 -9
- data/docs/RELEASING.md +4 -4
- data/docs/ROUTER.md +213 -0
- data/docs/RUNNING-LOCAL.md +5 -5
- data/docs/SCHEDULING.md +121 -0
- data/docs/SECURITY.md +23 -7
- data/docs/SKILLS.md +11 -2
- data/docs/SOAK.md +3 -3
- data/docs/TEMPLATES.md +134 -0
- data/docs/TOOLS.md +176 -27
- data/docs/WHY.md +1 -1
- data/docs/WORKFLOWS.md +2 -2
- data/docs/_includes/head_custom.html +5 -0
- data/docs/_includes/title.html +13 -0
- data/docs/_sass/color_schemes/insika.scss +32 -0
- data/docs/_sass/custom/custom.scss +199 -0
- data/docs/_sass/custom/setup.scss +26 -0
- data/docs/assets/img/favicon.svg +7 -0
- data/docs/assets/img/insika-mark.svg +7 -0
- data/docs/core-concepts.md +21 -0
- data/docs/domain.md +4 -4
- data/docs/improve.md +20 -0
- data/docs/index.md +8 -5
- data/docs/integrate.md +20 -0
- data/docs/operate.md +13 -6
- data/docs/prompts/ADD-TOOL.md +118 -0
- data/docs/prompts/DIAGNOSE-TURN.md +65 -0
- data/docs/prompts/GO-LIVE.md +138 -0
- data/docs/prompts/RUN-EXAMPLES.md +70 -0
- data/docs/reference.md +19 -0
- data/docs/ship.md +10 -2
- data/docs/start-here.md +18 -0
- data/lib/insika/agent_profile.rb +99 -17
- data/lib/insika/artifact_signing.rb +82 -0
- data/lib/insika/artifact_store.rb +160 -0
- data/lib/insika/channel_delivery.rb +1 -1
- data/lib/insika/chat_builder.rb +50 -19
- data/lib/insika/commands/agent_payload.rb +2 -2
- data/lib/insika/commands/backfill_knowledge.rb +145 -0
- data/lib/insika/commands/delete_artifact.rb +35 -0
- data/lib/insika/commands/delete_concept.rb +34 -0
- data/lib/insika/commands/delete_mcp.rb +6 -2
- data/lib/insika/commands/delete_tenant_data.rb +15 -3
- data/lib/insika/commands/gate_refinement.rb +1 -1
- data/lib/insika/commands/refresh_mcp_tools.rb +47 -0
- data/lib/insika/commands/restore_concept.rb +34 -0
- data/lib/insika/commands/seed_demo_data.rb +31 -0
- data/lib/insika/commands/upsert_mcp.rb +6 -3
- data/lib/insika/commands/write_concept.rb +57 -0
- data/lib/insika/compaction.rb +196 -0
- data/lib/insika/context/builder.rb +6 -2
- data/lib/insika/context/fragment.rb +4 -1
- data/lib/insika/context/priority.rb +8 -0
- data/lib/insika/context/providers/briefing.rb +53 -24
- data/lib/insika/context/providers/knowledge.rb +108 -0
- data/lib/insika/context/providers/prompt.rb +30 -24
- data/lib/insika/context/providers/session.rb +46 -10
- data/lib/insika/context_trace_store.rb +11 -1
- data/lib/insika/cron.rb +189 -0
- data/lib/insika/demo/agent_attrs.rb +43 -0
- data/lib/insika/demo/golden_cases.rb +81 -0
- data/lib/insika/demo/seeder.rb +336 -0
- data/lib/insika/doctor.rb +280 -17
- data/lib/insika/dsl/definition.rb +3 -2
- data/lib/insika/dsl/runtime.rb +64 -79
- data/lib/insika/dsl/server_boot.rb +23 -1
- data/lib/insika/dsl/system.rb +10 -2
- data/lib/insika/dsl.rb +103 -2
- data/lib/insika/env_schema.rb +21 -7
- data/lib/insika/evals/golden.rb +41 -4
- data/lib/insika/evals/judge.rb +47 -2
- data/lib/insika/evals/pairwise.rb +11 -0
- data/lib/insika/evals/persona.rb +98 -0
- data/lib/insika/evals/runner.rb +9 -0
- data/lib/insika/evals/simulator.rb +225 -0
- data/lib/insika/evals/transport.rb +84 -2
- data/lib/insika/event_stream.rb +10 -0
- data/lib/insika/executor.rb +295 -55
- data/lib/insika/followup_policy.rb +2 -25
- data/lib/insika/golden_store.rb +16 -1
- data/lib/insika/grounding/matcher.rb +1 -1
- data/lib/insika/knowledge.rb +680 -0
- data/lib/insika/knowledge_store.rb +140 -0
- data/lib/insika/loop_detector.rb +5 -34
- data/lib/insika/mcp_client.rb +94 -0
- data/lib/insika/mcp_json.rb +74 -0
- data/lib/insika/mcp_live_tool.rb +43 -0
- data/lib/insika/mcp_store.rb +98 -26
- data/lib/insika/mcp_tool_ingestor.rb +30 -8
- data/lib/insika/mcp_tool_registry.rb +100 -0
- data/lib/insika/media.rb +115 -31
- data/lib/insika/message_origin.rb +1 -1
- data/lib/insika/middleware.rb +9 -0
- data/lib/insika/onboarding.rb +17 -1
- data/lib/insika/outcome_store.rb +1 -1
- data/lib/insika/overlay_tool_registry.rb +37 -17
- data/lib/insika/packaging.rb +2 -2
- data/lib/insika/profile_source.rb +15 -1
- data/lib/insika/prompt_catalog.rb +10 -0
- data/lib/insika/retention.rb +36 -1
- data/lib/insika/router/app.rb +157 -0
- data/lib/insika/router/backend_pool.rb +98 -0
- data/lib/insika/router/hash_ring.rb +55 -0
- data/lib/insika/router/proxy_body.rb +34 -0
- data/lib/insika/router/session_key.rb +54 -0
- data/lib/insika/router.rb +18 -0
- data/lib/insika/schedule.rb +177 -0
- data/lib/insika/schedule_engine.rb +314 -0
- data/lib/insika/schedule_store.rb +208 -0
- data/lib/insika/server/app.rb +105 -15
- data/lib/insika/server/rack_app.rb +5 -1
- data/lib/insika/server/responses.rb +5 -5
- data/lib/insika/session_store.rb +34 -4
- data/lib/insika/settings_store.rb +8 -1
- data/lib/insika/skill_catalog.rb +12 -0
- data/lib/insika/soak/runner.rb +4 -4
- data/lib/insika/steer_injector.rb +21 -10
- data/lib/insika/studio/app.rb +591 -47
- data/lib/insika/studio/assets/dist/application.css +1 -1
- data/lib/insika/studio/assets/dist/application.js +21 -21
- data/lib/insika/studio/forms.rb +57 -5
- data/lib/insika/studio/nav_icons.rb +14 -1
- data/lib/insika/studio/views/_agent_tab_cache.erb +25 -0
- data/lib/insika/studio/views/_agent_tab_config.erb +514 -0
- data/lib/insika/studio/views/_agent_tab_history.erb +24 -0
- data/lib/insika/studio/views/_agent_tab_loops.erb +54 -0
- data/lib/insika/studio/views/_agent_tab_memory.erb +51 -0
- data/lib/insika/studio/views/_agent_tab_outcomes.erb +31 -0
- data/lib/insika/studio/views/_agent_tab_prompts.erb +108 -0
- data/lib/insika/studio/views/_agent_tab_skills.erb +38 -0
- data/lib/insika/studio/views/_agents_master.erb +44 -0
- data/lib/insika/studio/views/_message.erb +49 -32
- data/lib/insika/studio/views/agent_detail.erb +61 -820
- data/lib/insika/studio/views/agents.erb +70 -57
- data/lib/insika/studio/views/artifact.erb +23 -0
- data/lib/insika/studio/views/artifacts.erb +59 -0
- data/lib/insika/studio/views/evals.erb +2 -2
- data/lib/insika/studio/views/facts.erb +1 -1
- data/lib/insika/studio/views/funnel.erb +1 -1
- data/lib/insika/studio/views/home.erb +106 -67
- data/lib/insika/studio/views/knowledge.erb +123 -0
- data/lib/insika/studio/views/layout.erb +14 -11
- data/lib/insika/studio/views/mcp.erb +174 -80
- data/lib/insika/studio/views/session.erb +231 -177
- data/lib/insika/studio/views/settings.erb +50 -1
- data/lib/insika/studio/views/skills.erb +1 -1
- data/lib/insika/studio/views/tools.erb +24 -9
- data/lib/insika/telemetry/recorder.rb +49 -1
- data/lib/insika/templates/browser-agent/README.md +36 -0
- data/lib/insika/templates/browser-agent/agent.rb +49 -0
- data/lib/insika/templates/daily-digest/README.md +47 -0
- data/lib/insika/templates/daily-digest/agent.rb +77 -0
- data/lib/insika/templates/repo-explorer/README.md +36 -0
- data/lib/insika/templates/repo-explorer/agent.rb +45 -0
- data/lib/insika/templates/research-analyst/README.md +26 -0
- data/lib/insika/templates/research-analyst/agent.rb +68 -0
- data/lib/insika/templates/review-panel/README.md +20 -0
- data/lib/insika/templates/review-panel/agent.rb +50 -0
- data/lib/insika/templates/travel-planner/README.md +35 -0
- data/lib/insika/templates/travel-planner/agent.rb +87 -0
- data/lib/insika/templates.rb +112 -0
- data/lib/insika/tick.rb +24 -12
- data/lib/insika/timezone.rb +45 -0
- data/lib/insika/tool_batch.rb +67 -0
- data/lib/insika/tool_usage_report.rb +162 -0
- data/lib/insika/tools/generate_image.rb +52 -7
- data/lib/insika/tools/load_knowledge.rb +74 -0
- data/lib/insika/tools/run_persona_eval.rb +328 -0
- data/lib/insika/tools/save_artifact.rb +95 -0
- data/lib/insika/turn_budget.rb +91 -0
- data/lib/insika/turn_output.rb +1 -1
- data/lib/insika/turn_state.rb +15 -4
- data/lib/insika/version.rb +1 -1
- data/lib/insika/wiring/graph.rb +184 -12
- data/lib/insika/wiring/graph_chat.rb +102 -0
- data/lib/insika.rb +64 -0
- metadata +109 -5
- data/docs/build.md +0 -14
- data/docs/understand.md +0 -10
data/docs/ARCHITECTURE.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Architecture
|
|
3
|
-
parent:
|
|
4
|
-
nav_order:
|
|
3
|
+
parent: Start here
|
|
4
|
+
nav_order: 3
|
|
5
5
|
permalink: /architecture/
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -319,6 +319,7 @@ validator as the after-task hook, so both roots enforce content safety identical
|
|
|
319
319
|
| Tools (data/manifest/MCP) | `lib/insika/tool_definition.rb`, `tool_manifest.rb`, `mcp_tool_ingestor.rb` |
|
|
320
320
|
| Plugin loading (boot) | `lib/insika/plugin.rb`, `lib/insika/plugin/loader.rb` |
|
|
321
321
|
| Refinement (traffic → report) | `lib/insika/refinement/*`, `lib/insika/refinement_store.rb` |
|
|
322
|
+
| Post-turn learning (facts, skills, knowledge — extracted from finished conversations) | `lib/insika/distill.rb`, `lib/insika/harvest.rb`, `lib/insika/knowledge.rb`, `lib/insika/knowledge_store.rb`; the per-turn hook lives in `Executor#persist_turn`, next to `finalize_delegation` |
|
|
322
323
|
| Evals (cases, judges, gate) | `lib/insika/evals/*`, `lib/insika/golden_store.rb`; `evals/run.rb` is the CLI |
|
|
323
324
|
| HTTP/SSE surface | `lib/insika/server/*` |
|
|
324
325
|
|
data/docs/ARTIFACTS.md
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Artifacts
|
|
3
|
+
parent: Operate
|
|
4
|
+
nav_order: 3
|
|
5
|
+
permalink: /artifacts/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Artifacts — a report the agent can hand you a URL to
|
|
9
|
+
|
|
10
|
+
A channel message is ephemeral, unformatted and capped. A scheduled report turn
|
|
11
|
+
(see [Schedules](SCHEDULING.md)) produces something no message can carry: a page
|
|
12
|
+
— tables, sections, inline charts. An **artifact** is that page as a thing that
|
|
13
|
+
exists afterwards: listable, linkable, and — because it carries customer PII —
|
|
14
|
+
deletable on the same terms as everything else the engine stores.
|
|
15
|
+
|
|
16
|
+
Deliberately small: a store, a tool, a route. Not a CMS.
|
|
17
|
+
|
|
18
|
+
## The `save_artifact` tool
|
|
19
|
+
|
|
20
|
+
A registry tool, allowlisted per agent like every tool — **the allowlist IS the
|
|
21
|
+
switch**:
|
|
22
|
+
|
|
23
|
+
```ruby
|
|
24
|
+
agent = Insika.agent("reporter") do
|
|
25
|
+
instructions "…"
|
|
26
|
+
tools_allow %w[save_artifact] # without this, the tool is not even offered
|
|
27
|
+
end
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
The agent hands in `title` + `content` (default mime `text/html`; also
|
|
31
|
+
`text/markdown` and `image/svg+xml`) and gets the URL back, which it can include
|
|
32
|
+
in a channel message ("today's report: <url>"). When a signing key is
|
|
33
|
+
configured (below), the result also carries a `signed_url` that expires.
|
|
34
|
+
|
|
35
|
+
```jsonc
|
|
36
|
+
{ "id": "…", "url": "https://…/studio/artifacts/<id>/content",
|
|
37
|
+
"signed_url": "https://…/studio/artifacts/s/<id>?exp=…&sig=…" } // only with a key
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The tenant binding is **inherited, never chosen**: an artifact belongs to the
|
|
41
|
+
tenant of the agent that saved it — a binding of the tool instance, never a
|
|
42
|
+
parameter the model types. Store A's report can never appear in, or be linked
|
|
43
|
+
from, store B.
|
|
44
|
+
|
|
45
|
+
## Serving
|
|
46
|
+
|
|
47
|
+
- `GET /studio/artifacts` — the Studio's per-agent list (the listing IS the
|
|
48
|
+
history; no versioning — one report per run).
|
|
49
|
+
- `GET /studio/artifacts/:id` — the preview page, rendered **inside a sandboxed
|
|
50
|
+
iframe** (no scripts, no same-origin, no forms).
|
|
51
|
+
- `GET /studio/artifacts/:id/content` — the raw page (authenticated).
|
|
52
|
+
- `GET /studio/artifacts/s/:id?exp=…&sig=…` — the **signed link**: the only
|
|
53
|
+
artifact route that works without a Studio session. HMAC-SHA256 over
|
|
54
|
+
`(id, expiry)` with `INSIKA_ARTIFACT_SIGNING_KEY`, verified in constant time.
|
|
55
|
+
Expired or bad signatures **404 (never 403 — no oracle)**. Rotating the key
|
|
56
|
+
invalidates every outstanding link — the documented behavior, not a bug.
|
|
57
|
+
Without `INSIKA_ARTIFACT_SIGNING_KEY` there is no signed surface at all.
|
|
58
|
+
|
|
59
|
+
**Artifact content is untrusted.** It is LLM output. Both content routes send:
|
|
60
|
+
|
|
61
|
+
```http
|
|
62
|
+
Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'; img-src data:
|
|
63
|
+
X-Content-Type-Options: nosniff
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
No script, no external fetch, no forms. The model writes HTML with **inline
|
|
67
|
+
SVG** for charts — that is a skill instruction (palette, tables, pure-SVG bars),
|
|
68
|
+
not engine code. A "real" charting need is a plugin.
|
|
69
|
+
|
|
70
|
+
## Reasoning effort on a report turn
|
|
71
|
+
|
|
72
|
+
A report turn is not one shape of work — it plans, then it mines, then it
|
|
73
|
+
writes. `thinking` is one value per agent for the whole turn, so an agent set to
|
|
74
|
+
`high` pays deliberation on every one of the 30–50 tool calls a real report
|
|
75
|
+
makes, and that is where the 300 s turn timeout gets spent.
|
|
76
|
+
|
|
77
|
+
Split the phases across agents instead, which the engine already supports today:
|
|
78
|
+
|
|
79
|
+
```ruby
|
|
80
|
+
Insika.system do
|
|
81
|
+
# The miners: one narrow question each, no judgement to make.
|
|
82
|
+
agent("sales_miner") do
|
|
83
|
+
model "deepseek-v4-flash"
|
|
84
|
+
params thinking: "low"
|
|
85
|
+
tools %w[query_sales]
|
|
86
|
+
instructions "Answer ONE question about sales from the store data. Numbers, no prose."
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
# The orchestrator: it plans the report and writes it. This is the turn
|
|
90
|
+
# that deserves the deliberation.
|
|
91
|
+
agent("reporter") do
|
|
92
|
+
model "deepseek-v4-flash"
|
|
93
|
+
params thinking: "high"
|
|
94
|
+
tools %w[save_artifact]
|
|
95
|
+
subagents "sales_miner"
|
|
96
|
+
instructions "Plan the report, call spawn_subagents ONCE for every number you need, then write the page and save_artifact it."
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Two things make this work: a child inherits the *environment* (model, thinking)
|
|
102
|
+
only as a **default**, so its own `params thinking:` wins; and `spawn_subagents`
|
|
103
|
+
runs the children in parallel, so wall-clock is the slowest miner rather than
|
|
104
|
+
the sum. Each child also mines in its own isolated context, which is what keeps
|
|
105
|
+
the orchestrator's context from filling with raw rows.
|
|
106
|
+
|
|
107
|
+
Measure it before reaching for anything cleverer: the numbers that matter are
|
|
108
|
+
the turn's wall-clock, the timeout rate, and the judge score on the same report.
|
|
109
|
+
Per-phase effort *inside* a single turn is a real idea, but it is only worth
|
|
110
|
+
building once this recipe is shown not to be enough.
|
|
111
|
+
|
|
112
|
+
## Limits and retention
|
|
113
|
+
|
|
114
|
+
- **Size cap** — `INSIKA_ARTIFACT_MAX_BYTES` (default 1 MB): an artifact is a
|
|
115
|
+
page, not an attachment. The mime allowlist is `text/html`, `text/markdown`,
|
|
116
|
+
`image/svg+xml`; no binaries, no uploads.
|
|
117
|
+
- **Expiry** — the settings key `artifact_ttl_days` (Integer days; absent = OFF)
|
|
118
|
+
ages artifacts out on the retention sweep's own daily pass, **independent of
|
|
119
|
+
`retention_days`**: a deployment that keeps its conversations forever must
|
|
120
|
+
still expire the reports. This is the guarantee that PII inside a report
|
|
121
|
+
expires — the honest reach, because no reader can see inside the opaque HTML.
|
|
122
|
+
|
|
123
|
+
## Privacy
|
|
124
|
+
|
|
125
|
+
- `delete_tenant_data` deletes the tenant's artifacts (the tenant binding is
|
|
126
|
+
the isolation boundary).
|
|
127
|
+
- `forget_customer` **cannot** know which artifacts mention a customer (content
|
|
128
|
+
is opaque HTML), so per-customer redaction inside a report is not pretended to
|
|
129
|
+
exist; the `artifact_ttl_days` knob is the guarantee that a report's PII
|
|
130
|
+
expires.
|
|
131
|
+
|
|
132
|
+
## See also
|
|
133
|
+
|
|
134
|
+
- [Schedules](SCHEDULING.md) — the recurring turns whose output lands here.
|
|
135
|
+
- [Tools](TOOLS.md) — how a tool enters the per-agent allowlist.
|
|
136
|
+
- [`examples/scheduled-report/`](https://github.com/guizaols/insika/tree/main/examples/scheduled-report/)
|
|
137
|
+
— schedule + skill + data tool + artifact, tenant-bound, end to end.
|
data/docs/BENCHMARK.md
CHANGED
data/docs/CHANNELS.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Channels
|
|
3
|
-
parent:
|
|
4
|
-
nav_order:
|
|
3
|
+
parent: Integrate
|
|
4
|
+
nav_order: 2
|
|
5
5
|
permalink: /channels/
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -176,14 +176,14 @@ again.
|
|
|
176
176
|
## Relay or the drop-in API?
|
|
177
177
|
|
|
178
178
|
If you already own your messaging platform, you can reach the engine two ways: the
|
|
179
|
-
drop-in [`POST /v1/responses`](
|
|
179
|
+
drop-in [`POST /v1/responses`](ARCHITECTURE.md) — you hold an SSE connection for the
|
|
180
180
|
whole turn and read the answer off it — or the relay, where the engine acks in
|
|
181
181
|
milliseconds and POSTs the answer to you when it exists.
|
|
182
182
|
|
|
183
183
|
The instinct is that streaming gets the customer their reply sooner, and that the
|
|
184
184
|
relay trades that away. **For the default relay it does not, and the reason is
|
|
185
185
|
structural:** the engine publishes `:content` as the ANSWER, whole, after the
|
|
186
|
-
turn's hooks ([what crosses the edge](
|
|
186
|
+
turn's hooks ([what crosses the edge](ARCHITECTURE.md#what-crosses-the-edge)).
|
|
187
187
|
During the turn the stream carries tool activity; the text arrives in one piece at
|
|
188
188
|
the end. Measured on a real store agent, the text frames span **0 ms** — there is
|
|
189
189
|
nothing to deliver progressively. That is the `:at_end` fact, true of `/v1/responses`
|
|
@@ -204,7 +204,7 @@ That last row is the one that cannot be had the other way. `/v1/responses` answe
|
|
|
204
204
|
the request it was given, so a message that arrives while a turn is running is a
|
|
205
205
|
second turn — the engine has no way to tell you "this joined the previous one". The
|
|
206
206
|
relay's `merged` / `steered` acks exist precisely to say that, which is why
|
|
207
|
-
[the inbound queue](
|
|
207
|
+
[the inbound queue](AGENTS.md#queue_mode--when-a-message-arrives-while-the-agent-is-busy)
|
|
208
208
|
is only reachable from here.
|
|
209
209
|
|
|
210
210
|
**Measured, so you can judge it rather than take our word:** same agent, same
|
|
@@ -282,7 +282,7 @@ answers, and **they are four different facts**:
|
|
|
282
282
|
answer two or three times.** That is the one contract mistake that is visible to
|
|
283
283
|
the end user, so it is worth a line of code: only deliver for a `202`.
|
|
284
284
|
|
|
285
|
-
`merged` and `steered` come from the [inbound queue](
|
|
285
|
+
`merged` and `steered` come from the [inbound queue](AGENTS.md#queue_mode--when-a-message-arrives-while-the-agent-is-busy)
|
|
286
286
|
(`limits[:queue_mode]`). They only ever occur if you turned that on for the agent;
|
|
287
287
|
with the default `followup` you will only see `202` and `duplicate`.
|
|
288
288
|
|
|
@@ -312,7 +312,7 @@ short backoff, and then the delivery is marked `failed` and stops.
|
|
|
312
312
|
`content` is the turn's **answer** — one message, whole. The model's narration on
|
|
313
313
|
the way to an answer ("vou verificar o cardápio…") does not come through here; it
|
|
314
314
|
stays internal unless the agent opts in. That contract is
|
|
315
|
-
[the edge contract](
|
|
315
|
+
[the edge contract](ARCHITECTURE.md#what-crosses-the-edge), and it is why you can
|
|
316
316
|
forward `content` straight to the customer.
|
|
317
317
|
|
|
318
318
|
### Delivery policy
|
|
@@ -379,7 +379,7 @@ your side: you already have the customer's conversation, and `GET /v1/tasks/:id`
|
|
|
379
379
|
tells you the turn's terminal state.
|
|
380
380
|
|
|
381
381
|
A turn that **failed** delivers nothing — an error string is not an answer. Watch
|
|
382
|
-
`GET /v1/tasks/:id` or the [event stream](
|
|
382
|
+
`GET /v1/tasks/:id` or the [event stream](OBSERVABILITY.md) for those.
|
|
383
383
|
|
|
384
384
|
## Setting up the relay
|
|
385
385
|
|
|
@@ -397,7 +397,7 @@ INSIKA_RELAY_DELIVERY=progressive # optional; "at_end" (the defaul
|
|
|
397
397
|
without a credential — a public inbound route with an LLM behind it is a money
|
|
398
398
|
faucet, so it fails closed by construction.
|
|
399
399
|
|
|
400
|
-
The delivery POST goes through the same [egress guard](
|
|
400
|
+
The delivery POST goes through the same [egress guard](SECURITY.md#egress-the-ssrf-boundary) as
|
|
401
401
|
data-tools: **https only**, and private/loopback destinations blocked. For local
|
|
402
402
|
development, where your consumer is on `localhost`:
|
|
403
403
|
|
|
@@ -415,7 +415,7 @@ A runnable consumer in ~40 lines lives in
|
|
|
415
415
|
|
|
416
416
|
## Shadow mode
|
|
417
417
|
|
|
418
|
-
Shadow mode
|
|
418
|
+
Shadow mode lets one channel run every turn **end to end and deliver
|
|
419
419
|
nothing** — the experiment that answers "can we replace the incumbent?" before
|
|
420
420
|
any customer is handed over. The incumbent keeps answering; the engine records
|
|
421
421
|
what it *would* have answered, and the two replies are judged pairwise against a
|
|
@@ -470,7 +470,7 @@ redirect its own conversation.
|
|
|
470
470
|
## Writing your own channel
|
|
471
471
|
|
|
472
472
|
A channel is a plain object — no base class. Register it from a plugin
|
|
473
|
-
(see [Plugins](
|
|
473
|
+
(see [Plugins](PLUGINS.md)) with `contracts: { channels: [<id>] }` in the manifest,
|
|
474
474
|
and it mounts under `/channels/<id>/`.
|
|
475
475
|
|
|
476
476
|
Two members are always there; the rest of the object decides which shape you get.
|
|
@@ -536,8 +536,8 @@ command carries `transport: "channel:<id>"`.
|
|
|
536
536
|
|
|
537
537
|
## See also
|
|
538
538
|
|
|
539
|
-
- [Security](
|
|
539
|
+
- [Security](SECURITY.md) — the tokens, the egress guard, and why the rate limit
|
|
540
540
|
matters for anything public.
|
|
541
|
-
- [Agents](
|
|
541
|
+
- [Agents](AGENTS.md) — `limits[:queue_mode]`, which is what produces `merged` and
|
|
542
542
|
`steered`.
|
|
543
|
-
- [Observability](
|
|
543
|
+
- [Observability](OBSERVABILITY.md) — the event stream and OpenTelemetry.
|
data/docs/CONTEXT.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Context
|
|
3
|
-
parent:
|
|
4
|
-
nav_order:
|
|
3
|
+
parent: Core concepts
|
|
4
|
+
nav_order: 5
|
|
5
5
|
permalink: /context/
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -26,10 +26,12 @@ into a deterministic prompt:
|
|
|
26
26
|
| **Skills** | `<available_skills>` | 80 | identity | Level-1 skill list, minus whatever is already eager — see [Skills](SKILLS.md). |
|
|
27
27
|
| **Tool search** | `<available_tools>` | 70 | identity | Level-1 list of deferred tools — see [Tools](TOOLS.md). |
|
|
28
28
|
| **Skill trigger** | `<active_skill>` | 85 | volatile | Level-2 bodies: the agent's `skills_eager` set, plus the ones whose `triggers:` match the message — see [Skills](SKILLS.md). |
|
|
29
|
+
| **Knowledge** | `<knowledge>` | 77 | volatile | Level-1 top-K learned concepts for the turn's message (+ one-hop `[[links]]`), only if `knowledge.retrieve` is on. Cuttable — see [Knowledge](KNOWLEDGE.md). |
|
|
29
30
|
| **Memory** | `<memory>` | 75 | volatile | Durable facts + recent notes, only if `memory` is on. Cuttable. |
|
|
30
|
-
| **Briefing** | `<briefing>` | 65 | volatile | The session's working state
|
|
31
|
+
| **Briefing** | `<briefing>` | 65 | volatile | The session's working state — the *known* fields only. Only if the pack declared `briefing_fields`. Cuttable. |
|
|
31
32
|
| **Session** | history | 60–79 | volatile | The running transcript; priority scales with recency. |
|
|
32
33
|
| **Request** | `<request_context>` | 40 | volatile | Turn variables + tenant. Most cuttable; sits last. |
|
|
34
|
+
| **Briefing (tail)** | `<recitation>` | 95 | volatile | The still-missing list + next step, rendered **after the whole history** as a `user` message — the last thing the model reads before the current message. |
|
|
33
35
|
|
|
34
36
|
The ordering is deliberate: the render order is **identity layer first, volatile
|
|
35
37
|
layer after** — nothing volatile can sit above the cache boundary, whatever its
|
|
@@ -41,7 +43,8 @@ cacheable prefix byte-stable (see the prefix cache below).
|
|
|
41
43
|
- The cap is `profile.limits[:context_budget]`, **default 8000 tokens**.
|
|
42
44
|
- To fit the budget, the builder cuts **non-pinned** fragments
|
|
43
45
|
lowest-priority-first (ties broken by oldest history first). Under pressure you
|
|
44
|
-
lose
|
|
46
|
+
lose request context first, then old history, then briefing, then memory, then
|
|
47
|
+
learned knowledge, then the skill/tool-search level-1 lists — **the pinned
|
|
45
48
|
identity is never truncated**.
|
|
46
49
|
- A **pinned** fragment (the identity) that *alone* exceeds the budget raises an
|
|
47
50
|
error — the turn fails rather than shipping a truncated identity.
|
|
@@ -51,14 +54,37 @@ cacheable prefix byte-stable (see the prefix cache below).
|
|
|
51
54
|
> returns empty turns, raise `context_budget` (e.g. to `60000`) before looking
|
|
52
55
|
> anywhere else. See [Agents](AGENTS.md#default-limits).
|
|
53
56
|
|
|
54
|
-
###
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
57
|
+
### In-session compaction (opt-in, platform Settings)
|
|
58
|
+
|
|
59
|
+
When enabled (Studio → Settings → General, or the `compaction` settings hash),
|
|
60
|
+
the engine summarizes old turns *inside* the session instead of losing them to
|
|
61
|
+
eviction:
|
|
62
|
+
|
|
63
|
+
- **Trigger:** after a turn commits, if the session's *uncompacted* message
|
|
64
|
+
count exceeds `compact_after` (default 40), everything but the last
|
|
65
|
+
`keep_last` messages (default 20) is summarized by a cheap model
|
|
66
|
+
(`compaction.model`, falling back to the platform `utility_model`; neither
|
|
67
|
+
set = the feature is inert and `insika doctor` warns). Runs off the critical
|
|
68
|
+
path — the customer already has the answer.
|
|
69
|
+
- **Read path:** the Session provider replaces the compacted prefix with ONE
|
|
70
|
+
history fragment — a `user` message wrapped in `<conversation_summary>` tags,
|
|
71
|
+
priority 59 (one step below the oldest verbatim message), source
|
|
72
|
+
`compaction` (its own category in the context trace). The tail stays
|
|
73
|
+
verbatim and the boundary is **stable** between compactions, so the prompt
|
|
74
|
+
cache holds after it.
|
|
75
|
+
- **What survives:** the default prompt orders the summary to preserve customer
|
|
76
|
+
facts (sizes, CEP, order numbers), the assistant's commitments, the still-open
|
|
77
|
+
questions and the decisions already made; on re-compaction the previous
|
|
78
|
+
summary is folded in, so a fact from turn 3 survives every later batch.
|
|
79
|
+
A platform `compaction.prompt` replaces the default wholesale.
|
|
80
|
+
- **Scope:** store-sourced history only — a checkpoint resume replays its own
|
|
81
|
+
tape and an explicit `history` is the caller's contract; neither is
|
|
82
|
+
rewritten. Observability: the `:context_compacted` event, the
|
|
83
|
+
`insika.context.compacted` counter and `{upto, runs}` in the context trace.
|
|
84
|
+
|
|
85
|
+
Compaction does not replace the budget: eviction stays as the hard backstop
|
|
86
|
+
for a single oversized turn. Tune `context_budget` and keep the identity lean
|
|
87
|
+
regardless.
|
|
62
88
|
|
|
63
89
|
One cheap half **is** wired, opt-in per agent: `tool_output_compression` (DSL
|
|
64
90
|
`tool_output_compression`, or `"tool_output_compression": true` in the pack).
|
|
@@ -88,16 +114,16 @@ conversation; memory is the small set of facts that should outlive any single
|
|
|
88
114
|
conversation. Facts and notes are editable from the Studio agent page. See
|
|
89
115
|
[`examples/memory/`](https://github.com/guizaols/insika/tree/main/examples/memory/) for a runnable cross-session example.
|
|
90
116
|
|
|
91
|
-
Facts carry **provenance metadata
|
|
117
|
+
Facts carry **provenance metadata**: every fact record stores `origin`
|
|
92
118
|
(who wrote it — `"engine"`, `"operator"`, `"legacy"` or `"distilled"`),
|
|
93
119
|
`created_at` / `updated_at` timestamps, and an optional `expires_at` (ISO8601) —
|
|
94
120
|
**an expired fact is never injected**, even before the daily sweep prunes it. The
|
|
95
121
|
Studio Customers drill reads and edits the same cell the next turn reads (injection
|
|
96
122
|
unchanged), and every operator mutation lands in the content-free audit trail
|
|
97
123
|
(digests, never values). The sweep honors the `memory_ttl_days` setting on its own
|
|
98
|
-
knob — see [Security](SECURITY.md#memory-and-the-right-to-be-forgotten-lgpd
|
|
124
|
+
knob — see [Security](SECURITY.md#memory-and-the-right-to-be-forgotten-lgpd).
|
|
99
125
|
|
|
100
|
-
An **approved distilled fact** (
|
|
126
|
+
An **approved distilled fact** (see [Facts](FACTS.md)) lands in the
|
|
101
127
|
same cell this provider injects, stamped `distilled:<session_ref>` — approved on
|
|
102
128
|
the Studio Facts page, never applied automatically.
|
|
103
129
|
|
|
@@ -119,19 +145,37 @@ pack:
|
|
|
119
145
|
briefing_fields "size", "budget", "delivery_day"
|
|
120
146
|
```
|
|
121
147
|
|
|
122
|
-
With fields declared, the
|
|
123
|
-
|
|
148
|
+
With fields declared, the briefing renders in **two places**, and the split is
|
|
149
|
+
deliberate.
|
|
150
|
+
|
|
151
|
+
The durable half — what is already known — sits in the `:system` context
|
|
152
|
+
(priority 65: below identity/skill/memory so it never breaks the cacheable
|
|
124
153
|
prefix, above the turn's own `<request_context>`):
|
|
125
154
|
|
|
126
155
|
```
|
|
127
156
|
<briefing>
|
|
128
157
|
known:
|
|
129
158
|
size: M
|
|
159
|
+
</briefing>
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
The half that is a *goal* — what is still missing and the agreed next step — is
|
|
163
|
+
**recited at the tail**, after the whole history, as the last thing the model
|
|
164
|
+
reads before the current user message:
|
|
165
|
+
|
|
166
|
+
```
|
|
167
|
+
<recitation>
|
|
130
168
|
still missing: budget, delivery_day
|
|
131
169
|
next step: send the payment link tomorrow at 10
|
|
132
|
-
</
|
|
170
|
+
</recitation>
|
|
133
171
|
```
|
|
134
172
|
|
|
173
|
+
Attention is strongest at the end of the context: a goal stated only at the top
|
|
174
|
+
is the first thing a 30-call turn forgets. So the recitation was **moved** there,
|
|
175
|
+
not copied — the head never repeats it, and the turn pays for it once. It rides
|
|
176
|
+
as a `user` message, like every other engine append inside a turn, so the system
|
|
177
|
+
prefix stays byte-stable and the cache breakpoint at its end keeps hitting.
|
|
178
|
+
|
|
135
179
|
The `still missing` list is the point: the *model* sees which declared fields are
|
|
136
180
|
still unanswered, so it stops re-asking for something already given. Stored keys
|
|
137
181
|
that the pack no longer declares are never rendered. The Studio session screen
|
|
@@ -165,7 +209,7 @@ Two distinct caching mechanisms — don't conflate them:
|
|
|
165
209
|
Cache accounting surfaces as `cached_tokens` (reads) and `cache_creation_tokens`
|
|
166
210
|
(writes), visible in telemetry and the Studio tokens chip.
|
|
167
211
|
|
|
168
|
-
### The two layers
|
|
212
|
+
### The two layers
|
|
169
213
|
|
|
170
214
|
The system block is partitioned into two cache layers:
|
|
171
215
|
|
data/docs/DEMO.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Demo data
|
|
3
|
+
parent: Start here
|
|
4
|
+
nav_order: 4
|
|
5
|
+
permalink: /demo/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Demo data — see every loop working at once
|
|
9
|
+
|
|
10
|
+
Most of what makes Insika worth looking at only shows up after data has
|
|
11
|
+
accumulated: a [funnel](OUTCOMES.md#the-outcome-funnel) with a frozen
|
|
12
|
+
baseline needs weeks of folded outcomes, a [refinement](REFINEMENT.md)
|
|
13
|
+
proposal needs a run that actually found something, an
|
|
14
|
+
[approval](POLICY.md#layer-2-policies-and-approvals) needs a tool call
|
|
15
|
+
someone is waiting on. A fresh instance shows none of that — every one of
|
|
16
|
+
those pages renders its empty state, which makes it hard to tell "nothing
|
|
17
|
+
happened yet" from "this doesn't work."
|
|
18
|
+
|
|
19
|
+
`insika demo:seed` closes that gap in one shot: it provisions a single
|
|
20
|
+
fictional agent (`demo-store`, an e-commerce support bot) and writes enough
|
|
21
|
+
realistic-looking data to see every loop at once.
|
|
22
|
+
|
|
23
|
+
## What it creates
|
|
24
|
+
|
|
25
|
+
| Page | What you'll see |
|
|
26
|
+
|------|------------------|
|
|
27
|
+
| [Funnel](OUTCOMES.md#the-outcome-funnel) — `/studio/funnel?agent=demo-store` | 40 days of folded outcomes (`greeted → browsing → cart_started → checkout_started → purchased`) and a **frozen baseline** |
|
|
28
|
+
| [Follow-ups](OUTCOMES.md#follow-ups--the-seller-who-comes-back) — `/studio/followups?agent=demo-store` | one record in each state: `pending`, `fired` (one per arm, so the A/B card has something to compare), `cancelled`, `blocked` |
|
|
29
|
+
| [Refinement](REFINEMENT.md) — `/studio/refinement?agent=demo-store` | four runs across the lifecycle: `awaiting_approval`, `applied`, `rejected` (gate failed), `no_findings` |
|
|
30
|
+
| [Approvals](POLICY.md#layer-2-policies-and-approvals) — `/studio/approvals` | two pending tool calls waiting on a human, one already resolved |
|
|
31
|
+
| [Facts](FACTS.md) — `/studio/facts` | three distillation proposals (`pending`, `approved` — with the resulting memory fact, `rejected`) |
|
|
32
|
+
| [Evals](EVALS.md) — `/studio/evals?agent=demo-store` | six golden cases and a baseline run with a mix of passes and one failure |
|
|
33
|
+
|
|
34
|
+
Every record is written through the same store APIs a real turn would use
|
|
35
|
+
(`OutcomeStore#create` + the funnel fold, `FollowupStore#create` + its
|
|
36
|
+
transitions, and so on) — there is no bulk-insert shortcut, and no bundled
|
|
37
|
+
`.rb` script outside `lib/` (nothing here needs a checkout; it ships in the
|
|
38
|
+
gem).
|
|
39
|
+
|
|
40
|
+
## Running it
|
|
41
|
+
|
|
42
|
+
From the CLI, against whichever store the rest of your commands already use
|
|
43
|
+
([Running locally](RUNNING-LOCAL.md#variables-all-optional) — `INSIKA_DB`
|
|
44
|
+
unset means an ephemeral, in-memory store, which is a fine place to try this):
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
insika demo:seed
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
A second run is a safe no-op once `demo-store` exists; pass `--force` to seed
|
|
51
|
+
another batch on top (the funnel baseline recomputes cleanly, but follow-ups,
|
|
52
|
+
refinement runs, approvals, proposals and goldens accumulate rather than
|
|
53
|
+
reset — none of those stores expose a per-agent bulk-delete that a shared
|
|
54
|
+
"platform" tenant could call without risking another agent's data).
|
|
55
|
+
|
|
56
|
+
From the Studio, open **Settings → Demo data** and click **Seed demo data**.
|
|
57
|
+
It dispatches the exact same command the CLI runs — the Studio never writes a
|
|
58
|
+
store directly, here or anywhere else.
|
|
59
|
+
|
|
60
|
+
**This writes into whatever store the running instance already has open.**
|
|
61
|
+
There's no separate demo database and no isolation: point `INSIKA_DB` at a
|
|
62
|
+
scratch file (or leave it unset, for an ephemeral store) before seeding —
|
|
63
|
+
never at a deployment holding real tenant data.
|
|
64
|
+
|
|
65
|
+
## Then look around
|
|
66
|
+
|
|
67
|
+
Once seeded, the CLI prints the same six paths listed above. If Studio isn't
|
|
68
|
+
running yet, boot it the way you already do — see [Running
|
|
69
|
+
locally](RUNNING-LOCAL.md#boot) for a checkout, or [Embedding](EMBEDDING.md)
|
|
70
|
+
for `Insika.agent { … }.serve` — pointed at the same `INSIKA_DB`. Studio reads
|
|
71
|
+
every agent's profile from the same config store, so it will show
|
|
72
|
+
`demo-store` next to whatever agent you're actually building, no matter which
|
|
73
|
+
one the running process itself defines.
|
|
74
|
+
|
|
75
|
+
## See also
|
|
76
|
+
|
|
77
|
+
- [Refinement](REFINEMENT.md), [Facts](FACTS.md), [Evals](EVALS.md) — what
|
|
78
|
+
each seeded page actually means.
|
|
79
|
+
- [Running locally](RUNNING-LOCAL.md) — booting Studio against a durable
|
|
80
|
+
`INSIKA_DB`.
|