insika 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +69 -3
- data/README.md +1 -1
- data/bin/insika +22 -7
- data/docs/AGENTS.md +129 -5
- data/docs/CHANNELS.md +1 -1
- data/docs/CONTEXT.md +22 -5
- data/docs/DEPLOY.md +30 -10
- data/docs/EMBEDDING.md +11 -7
- data/docs/EVALS.md +1 -1
- data/docs/LOADTEST.md +3 -2
- data/docs/OBSERVABILITY.md +11 -2
- data/docs/REFINEMENT.md +6 -6
- data/docs/RELEASING.md +7 -7
- data/docs/RUNNING-LOCAL.md +1 -1
- data/docs/SECURITY.md +24 -11
- data/docs/SKILLS.md +189 -3
- data/docs/WHY.md +1 -1
- data/docs/WORKFLOWS.md +2 -2
- data/docs/index.md +1 -1
- data/docs/onboarding/start.md +1 -1
- data/lib/insika/agent_profile.rb +89 -22
- data/lib/insika/alert_dispatcher.rb +139 -0
- data/lib/insika/baseline_store.rb +2 -2
- data/lib/insika/budget_ledger.rb +135 -0
- data/lib/insika/channel_delivery.rb +14 -11
- data/lib/insika/channel_registry.rb +1 -1
- data/lib/insika/channels/relay.rb +3 -3
- data/lib/insika/channels/web/widget.js +2 -2
- data/lib/insika/channels/web.rb +7 -7
- data/lib/insika/channels/webhook.rb +58 -0
- data/lib/insika/chat_builder.rb +62 -13
- data/lib/insika/circuit_state.rb +114 -0
- data/lib/insika/coercion.rb +8 -0
- data/lib/insika/commands/agent_payload.rb +5 -3
- data/lib/insika/commands/create_agent.rb +2 -2
- data/lib/insika/commands/create_session.rb +1 -1
- data/lib/insika/commands/delete_llm_provider.rb +1 -1
- data/lib/insika/commands/delete_skill.rb +43 -0
- data/lib/insika/commands/gate_refinement.rb +12 -12
- data/lib/insika/commands/import_mcp_tools.rb +1 -1
- data/lib/insika/commands/import_tools.rb +4 -4
- data/lib/insika/commands/issue_tenant_token.rb +41 -0
- data/lib/insika/commands/resolve_refinement.rb +1 -1
- data/lib/insika/commands/revoke_token.rb +39 -0
- data/lib/insika/commands/rotate_tenant_token.rb +43 -0
- data/lib/insika/commands/run_refinement.rb +5 -5
- data/lib/insika/commands/send_message.rb +9 -9
- data/lib/insika/commands/set_agent_tools.rb +1 -1
- data/lib/insika/commands/set_skill_agents.rb +60 -19
- data/lib/insika/commands/trigger_workflow.rb +1 -1
- data/lib/insika/commands/update_agent.rb +1 -1
- data/lib/insika/commands/write_data_tool.rb +1 -1
- data/lib/insika/commands/write_golden.rb +1 -1
- data/lib/insika/commands/write_skill.rb +19 -9
- data/lib/insika/config_store.rb +8 -4
- data/lib/insika/context/builder.rb +2 -2
- data/lib/insika/context/fragment.rb +27 -3
- data/lib/insika/context/priority.rb +3 -2
- data/lib/insika/context/providers/memory.rb +1 -1
- data/lib/insika/context/providers/request.rb +1 -1
- data/lib/insika/context/providers/session.rb +17 -2
- data/lib/insika/context/providers/skill.rb +5 -1
- data/lib/insika/context/providers/skill_trigger.rb +128 -0
- data/lib/insika/context_trace_store.rb +92 -0
- data/lib/insika/delegation_store.rb +2 -2
- data/lib/insika/doctor.rb +250 -5
- data/lib/insika/dsl/runtime.rb +12 -9
- data/lib/insika/dsl/server_boot.rb +4 -3
- data/lib/insika/dsl/system.rb +1 -1
- data/lib/insika/dsl.rb +72 -15
- data/lib/insika/edge_limiter.rb +144 -6
- data/lib/insika/egress_guard.rb +3 -3
- data/lib/insika/env_schema.rb +13 -10
- data/lib/insika/errors.rb +61 -5
- data/lib/insika/evals/assertions.rb +12 -12
- data/lib/insika/evals/baseline.rb +3 -3
- data/lib/insika/evals/golden.rb +8 -8
- data/lib/insika/evals/judge.rb +7 -7
- data/lib/insika/evals/pairwise.rb +3 -3
- data/lib/insika/evals/report.rb +2 -2
- data/lib/insika/evals/runner.rb +6 -6
- data/lib/insika/evals/transport.rb +2 -2
- data/lib/insika/event_stream.rb +23 -5
- data/lib/insika/executor.rb +423 -108
- data/lib/insika/frontmatter.rb +1 -1
- data/lib/insika/golden_store.rb +2 -2
- data/lib/insika/http_client.rb +3 -3
- data/lib/insika/inbound_log.rb +1 -1
- data/lib/insika/llm_configurator.rb +3 -3
- data/lib/insika/loop_detector.rb +143 -0
- data/lib/insika/mcp_http_client.rb +4 -4
- data/lib/insika/mcp_tool_ingestor.rb +6 -6
- data/lib/insika/message_origin.rb +2 -2
- data/lib/insika/model_resolver.rb +1 -1
- data/lib/insika/model_selection.rb +5 -4
- data/lib/insika/onboarding.rb +2 -2
- data/lib/insika/outbox_store.rb +2 -2
- data/lib/insika/overlay_tool_registry.rb +3 -4
- data/lib/insika/pack.rb +3 -3
- data/lib/insika/pack_importer.rb +17 -15
- data/lib/insika/pending_action_store.rb +1 -1
- data/lib/insika/plugin/loader.rb +2 -2
- data/lib/insika/policy/policy.rb +1 -1
- data/lib/insika/profile_source.rb +12 -6
- data/lib/insika/provider_error_classifier.rb +160 -0
- data/lib/insika/queue_policy.rb +2 -2
- data/lib/insika/recovery.rb +47 -6
- data/lib/insika/refinement/candidate.rb +4 -4
- data/lib/insika/refinement/evidence_collector.rb +6 -6
- data/lib/insika/refinement/gate.rb +7 -7
- data/lib/insika/refinement/panel.rb +7 -7
- data/lib/insika/refinement/proposer.rb +9 -9
- data/lib/insika/refinement_store.rb +12 -12
- data/lib/insika/reliability.rb +185 -0
- data/lib/insika/safety/config.rb +2 -2
- data/lib/insika/safety/detectors.rb +5 -5
- data/lib/insika/safety/factory.rb +3 -3
- data/lib/insika/safety/input_guardrail.rb +19 -4
- data/lib/insika/safety/moderator.rb +19 -11
- data/lib/insika/safety/output_filter.rb +2 -2
- data/lib/insika/safety/output_validator.rb +2 -2
- data/lib/insika/safety/safe_responses.rb +1 -1
- data/lib/insika/sandbox/boundary.rb +2 -2
- data/lib/insika/sandbox.rb +1 -1
- data/lib/insika/server/app.rb +223 -51
- data/lib/insika/server/boot.rb +4 -4
- data/lib/insika/server/rack_app.rb +15 -7
- data/lib/insika/server/responses.rb +18 -8
- data/lib/insika/server/tenant_auth.rb +61 -0
- data/lib/insika/session_actor.rb +3 -3
- data/lib/insika/session_store.rb +1 -1
- data/lib/insika/settings_store.rb +5 -5
- data/lib/insika/shutdown.rb +4 -4
- data/lib/insika/skill_catalog.rb +127 -20
- data/lib/insika/skill_store.rb +70 -22
- data/lib/insika/steer_injector.rb +1 -1
- data/lib/insika/store.rb +1 -1
- data/lib/insika/studio/app.rb +183 -61
- data/lib/insika/studio/assets/dist/application.js +25 -24
- data/lib/insika/studio/forms.rb +13 -18
- data/lib/insika/studio/nav_icons.rb +1 -1
- data/lib/insika/studio/views/_message.erb +2 -2
- data/lib/insika/studio/views/agent_detail.erb +2 -2
- data/lib/insika/studio/views/agents.erb +1 -1
- data/lib/insika/studio/views/refinement.erb +4 -4
- data/lib/insika/studio/views/session.erb +78 -3
- data/lib/insika/studio/views/settings.erb +7 -12
- data/lib/insika/studio/views/skills.erb +67 -12
- data/lib/insika/subagent_graph.rb +3 -3
- data/lib/insika/task_actor.rb +3 -3
- data/lib/insika/task_store.rb +1 -1
- data/lib/insika/telemetry/pricing.rb +3 -3
- data/lib/insika/telemetry/recorder.rb +1 -1
- data/lib/insika/telemetry.rb +2 -2
- data/lib/insika/testing/store_contract.rb +27 -27
- data/lib/insika/tick.rb +122 -0
- data/lib/insika/token_store.rb +168 -0
- data/lib/insika/tool_assembly.rb +5 -5
- data/lib/insika/tool_definition.rb +8 -8
- data/lib/insika/tool_envelope.rb +1 -1
- data/lib/insika/tool_manifest.rb +6 -6
- data/lib/insika/tool_output_compressor.rb +100 -0
- data/lib/insika/tool_store.rb +1 -1
- data/lib/insika/tool_trace_store.rb +1 -1
- data/lib/insika/tools/concurrency.rb +2 -2
- data/lib/insika/tools/data_defined_tool.rb +4 -5
- data/lib/insika/tools/load_skill.rb +61 -3
- data/lib/insika/tools/stuck_signal.rb +44 -0
- data/lib/insika/tools/subagent.rb +4 -4
- data/lib/insika/tools/subagents.rb +1 -1
- data/lib/insika/turn_output.rb +2 -2
- data/lib/insika/turn_state.rb +17 -13
- data/lib/insika/turn_timing.rb +2 -2
- data/lib/insika/usage_ledger.rb +1 -1
- data/lib/insika/version.rb +1 -1
- data/lib/insika/wiring/graph.rb +77 -26
- data/lib/insika/workflow.rb +1 -1
- data/lib/insika/workflow_registry.rb +1 -1
- data/lib/insika.rb +32 -15
- metadata +19 -2
- data/lib/insika/server/admin_auth.rb +0 -29
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9e7913378bd8be19a7d9422524e62c6786c2abc8d26b5194ef8b1db8fef82be5
|
|
4
|
+
data.tar.gz: f1a279b7284d9756d7ad3dcef56329314062ef77c0fbb3d24df98e10c9bcb4ce
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: dcbf5403a982de5ee9dc8ed0c5fde2b21b5b63667cde0b47d2b9985b99fe7a9708d53fb82d43ce8464f46c38989d5b2b261156a84c8245e16199017515d24191
|
|
7
|
+
data.tar.gz: 349d532c16f5baed05fcd79318f3f68056e40f2a931e5e33ff12c339791edaa6fcdfe811d419cf3aff490118bd46698486470f234d9e820f0e568bb875e8b2cc
|
data/CHANGELOG.md
CHANGED
|
@@ -8,12 +8,78 @@ it is released. Entries land with the pull request that makes the change.
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
-
Nothing
|
|
12
|
-
|
|
11
|
+
Nothing yet.
|
|
12
|
+
|
|
13
|
+
## [0.2.0] - 2026-08-13
|
|
14
|
+
|
|
15
|
+
The workstreams between the first release and the one the gem actually became:
|
|
16
|
+
multi-tenancy at the edge (WS1), calendar budgets (WS2), provider reliability
|
|
17
|
+
(WS3), the stuck signal (WS5), operator alerts + live TTFB (WS6), and the
|
|
18
|
+
failure-classification core (B9) — plus the two fix rounds that made them
|
|
19
|
+
safe to ship.
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- **Multi-tenant at the edge (WS1)** — `INSIKA_TENANCY=multi_tenant` resolves the
|
|
24
|
+
Bearer to a principal before the routes: per-tenant + operator tokens stored
|
|
25
|
+
only as SHA-256 hashes, a tenant's sessions/tasks/streams living under its own
|
|
26
|
+
`<tenant>:` namespace (fail-closed: another tenant's reads as `404`), and every
|
|
27
|
+
authoring/config surface refused to a tenant.
|
|
28
|
+
- **Calendar budgets (WS2)** — `AgentProfile#budget` caps the billed spend
|
|
29
|
+
(input + output + cached + cache-creation) per calendar day/month and
|
|
30
|
+
(tenant, agent): HARD (default) fails the turn with the typed
|
|
31
|
+
`Insika::BudgetExceeded` + `retry_after`; `soft: true` runs the turn and warns
|
|
32
|
+
once per window — with the `alert_at` (`0.8`) crossing and the real cap
|
|
33
|
+
crossing as separate events.
|
|
34
|
+
- **Reliability (WS3)** — retries with backoff, mid-turn rotation to the
|
|
35
|
+
fallback chain, a per-`(tenant, provider/model)` circuit breaker with
|
|
36
|
+
half-open trials (a failed trial reopens), and a per-attempt `timeout`
|
|
37
|
+
(default 30s) counted as retryable. A `:fatal` provider error is never
|
|
38
|
+
retried.
|
|
39
|
+
- **Stuck signal (WS5)** — an agent declared stuck ends its turn with
|
|
40
|
+
`outcome: "stuck"` on the envelope and a dedicated `:turn_stuck` event — the
|
|
41
|
+
deterministic point a consumer escalates on.
|
|
42
|
+
- **Operator alerts + live TTFB (WS6)** — `budget_warning`, `breaker_open` and
|
|
43
|
+
`delivery_failed` POSTed to a per-agent `alerts.webhook` over the at-most-once
|
|
44
|
+
outbox pipeline (boot-recoverable); under `INSIKA_TURN_TIMING` the first
|
|
45
|
+
content chunk emits a live `:ttft` on the streaming envelope.
|
|
46
|
+
- **Failure classification (B9)** — provider/transport failures classified by
|
|
47
|
+
action (`:fatal` / `:retryable` / `:rate_limited_*`) and wrapped with the
|
|
48
|
+
provider's `retry_after`; mechanical tool-output dedupe (C3) back-references a
|
|
49
|
+
byte-identical repeat only when the reference is genuinely shorter.
|
|
50
|
+
- **The periodic tick** — durability no longer waits for a reboot. Serving
|
|
51
|
+
workers run a tick every `INSIKA_TICK_INTERVAL` (default 60s, `0` disables)
|
|
52
|
+
as a child of the turn supervisor: it re-drives outbox records left `:pending`
|
|
53
|
+
and sweeps orphaned `:queued`/`:running` tasks untouched past
|
|
54
|
+
`INSIKA_TICK_STALE_AFTER` (default 900s) — the orphans of a worker respawned
|
|
55
|
+
mid-generation are recovered without a deploy. One worker per window sweeps (a
|
|
56
|
+
single transactional claim); a task someone alive owns is skipped, never
|
|
57
|
+
failed. `:waiting`/`:paused` stay boot recovery's.
|
|
58
|
+
|
|
59
|
+
### Fixed
|
|
60
|
+
|
|
61
|
+
- **WS2/WS3/WS6 criticals** — the budget alert marker no longer returns inside
|
|
62
|
+
the store transaction (a leaked `BEGIN IMMEDIATE` locked SQLite on the 2nd
|
|
63
|
+
over-threshold turn); the monthly reset is December-safe and UTC-aligned; an
|
|
64
|
+
unset reliability timeout is 30s, not 1s, and a timeout retries/rotates
|
|
65
|
+
instead of dying as "fatal"; a failed half-open trial reopens the circuit;
|
|
66
|
+
webhook deliveries pass the egress guard (SSRF); `:ttft` is emitted once per
|
|
67
|
+
turn; webhook channels pre-register so the boot sweep recovers pending alerts;
|
|
68
|
+
the alert dispatcher subscribes typed and re-subscribes on overflow.
|
|
69
|
+
- **WS1** — `#revoke` rides the store transaction; a `tenant_id` containing
|
|
70
|
+
`:` is refused (the session-namespace delimiter); `POST /v1/sessions` mints a
|
|
71
|
+
tenant's session under its own prefix.
|
|
72
|
+
- **WS2/WS3 softs** — a failed turn's consumed tokens count against the budget;
|
|
73
|
+
`:breaker_open` alerts only on the closed→open transition; the fallback chain
|
|
74
|
+
dedupes `"model"` vs `"provider/model"` spellings.
|
|
75
|
+
|
|
76
|
+
## [0.1.0] - 2026-08-10
|
|
77
|
+
|
|
78
|
+
The first release: `gem install insika`.
|
|
13
79
|
|
|
14
80
|
### Added
|
|
15
81
|
|
|
16
|
-
- **A publishable core
|
|
82
|
+
- **A publishable core** — `gem install insika` gives every shape: `reply`
|
|
17
83
|
in-process, `serve`, `Insika::Server.rack_app` mounted, and `embed(backend:)`. The
|
|
18
84
|
server and the Studio moved under `lib/insika/` and ship in the gem; the exported
|
|
19
85
|
store contract (`lib/insika/testing/store_contract.rb`) is what a third-party
|
data/README.md
CHANGED
data/bin/insika
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env ruby
|
|
2
2
|
# frozen_string_literal: true
|
|
3
3
|
|
|
4
|
-
# `insika` CLI
|
|
4
|
+
# `insika` CLI — strict config + doctor --fix. The operator's
|
|
5
5
|
# DX front door to the strict-config discipline:
|
|
6
6
|
#
|
|
7
7
|
# insika doctor [--fix] [--json] diagnose the deployment config; --fix applies
|
|
@@ -45,9 +45,9 @@ module Insika
|
|
|
45
45
|
Usage: insika <command> [options]
|
|
46
46
|
|
|
47
47
|
Commands:
|
|
48
|
-
doctor [--fix] [--json] diagnose the deployment configuration
|
|
48
|
+
doctor [--fix] [--json] diagnose the deployment configuration
|
|
49
49
|
env [--json] list known config keys + current (masked) values
|
|
50
|
-
refine --agent ID [...] report what broke in an agent's real traffic
|
|
50
|
+
refine --agent ID [...] report what broke in an agent's real traffic
|
|
51
51
|
evals:import [--dir D] load the golden corpus into the store (editable in Studio)
|
|
52
52
|
evals:export [--dir D] write the stored cases back out as YAML
|
|
53
53
|
evals:baseline <sub> the accepted state per agent: show | import | export
|
|
@@ -108,9 +108,9 @@ module Insika
|
|
|
108
108
|
|
|
109
109
|
# insika refine --agent ID [--last-sessions N] [--since ISO8601] [--full] [--json]
|
|
110
110
|
#
|
|
111
|
-
#
|
|
111
|
+
# reads the agent's own traffic and records a ranked failure
|
|
112
112
|
# report. Read-only with respect to the agent — it writes only the run record.
|
|
113
|
-
# There is no scheduler in the engine
|
|
113
|
+
# There is no scheduler in the engine: this command and the Studio button
|
|
114
114
|
# are how a run starts, and a cron that wants one calls this.
|
|
115
115
|
def refine(argv)
|
|
116
116
|
opts = { json: false, full: false }
|
|
@@ -149,7 +149,7 @@ module Insika
|
|
|
149
149
|
# insika evals:import [--dir evals/golden] [--keep-existing]
|
|
150
150
|
#
|
|
151
151
|
# The corpus on disk is the SEED and the export format; the store is what a
|
|
152
|
-
# deployment runs and what the Studio edits
|
|
152
|
+
# deployment runs and what the Studio edits. Importing is how a
|
|
153
153
|
# fresh deploy gets the cases, and how a broken edit is undone.
|
|
154
154
|
def evals_import(argv)
|
|
155
155
|
opts = { dir: File.expand_path("../evals/golden", __dir__), overwrite: true }
|
|
@@ -189,7 +189,7 @@ end
|
|
|
189
189
|
#
|
|
190
190
|
# The accepted state of a golden set used to live ONLY in `evals/baseline.json`,
|
|
191
191
|
# which is fine for the CLI (it runs from a checkout) and useless for the refinement
|
|
192
|
-
# gate
|
|
192
|
+
# gate, which runs inside a deployment and has to know what "no
|
|
193
193
|
# regression" means without one. So it became a per-agent record.
|
|
194
194
|
#
|
|
195
195
|
# `import` splits the one file into those records, resolving each case's agent from
|
|
@@ -333,10 +333,25 @@ end
|
|
|
333
333
|
# Prompt-file corruption is invisible from the outside (the file is there, the
|
|
334
334
|
# agent answers) — the sweep is the only thing that names it.
|
|
335
335
|
agent_file_store: Insika::AgentFileStore.new(config_store: config_store),
|
|
336
|
+
# Skills + profiles: the eagerness/drift checks are about the RELATION between
|
|
337
|
+
# a skill and the agents holding it, so neither half alone is enough. The
|
|
338
|
+
# catalog brings the DISK seeds into the sweep (deploy/skills in a deployment,
|
|
339
|
+
# skills/ in the minimal wiring — whichever exists here); without it a stale
|
|
340
|
+
# `eager:` or a Natura-in-shared-body sitting in a seed pack is invisible.
|
|
341
|
+
skill_store: Insika::SkillStore.new(config_store: config_store),
|
|
342
|
+
skill_catalog: Insika::SkillCatalog.new(skill_roots),
|
|
343
|
+
profile_source: Insika::StoredProfileSource.new(config_store: config_store),
|
|
336
344
|
backend: backend
|
|
337
345
|
)
|
|
338
346
|
end
|
|
339
347
|
|
|
348
|
+
# The conventional seed roots, relative to where the operator runs the CLI —
|
|
349
|
+
# same "no app boot" rule as the rest of `doctor`, so the composition root that
|
|
350
|
+
# actually declares them is never loaded.
|
|
351
|
+
def skill_roots
|
|
352
|
+
%w[deploy/skills skills].map { |p| File.join(Dir.pwd, p) }.select { |p| Dir.exist?(p) }
|
|
353
|
+
end
|
|
354
|
+
|
|
340
355
|
def display_value(spec, raw)
|
|
341
356
|
return "(unset)" if raw.nil?
|
|
342
357
|
|
data/docs/AGENTS.md
CHANGED
|
@@ -18,7 +18,7 @@ keeps the profile it captured when it started; the next turn sees the new one.
|
|
|
18
18
|
>
|
|
19
19
|
> ```ruby
|
|
20
20
|
> agent = Insika.agent("assistant") do
|
|
21
|
-
> model "deepseek-
|
|
21
|
+
> model "deepseek-v4-flash"
|
|
22
22
|
> provider :deepseek
|
|
23
23
|
> instructions "You are a concise, friendly assistant."
|
|
24
24
|
> end
|
|
@@ -84,13 +84,19 @@ lookups.
|
|
|
84
84
|
```ruby
|
|
85
85
|
DEFAULT_LIMITS = {
|
|
86
86
|
turn_timeout: 300, tool_timeout: 60, provider_timeout: 5,
|
|
87
|
-
context_budget: 8_000, max_tool_calls: 50,
|
|
88
|
-
tool_concurrency: 1
|
|
87
|
+
context_budget: 8_000, max_tool_calls: 50, max_tool_repeat: 3,
|
|
88
|
+
approval_timeout: 3_600, tool_concurrency: 1
|
|
89
89
|
}
|
|
90
90
|
```
|
|
91
91
|
|
|
92
92
|
`build` merges your overrides over these — you set only the deltas.
|
|
93
93
|
|
|
94
|
+
`max_tool_repeat` is the loop guard: the same tool called with **identical
|
|
95
|
+
arguments** that many times in a row gets ONE in-turn warning (a user message at
|
|
96
|
+
the next tool-batch boundary: "you already ran this, answer with what you
|
|
97
|
+
have"). A repeat after the warning aborts the turn like `max_tool_calls` does.
|
|
98
|
+
Set it below 2 to switch it off.
|
|
99
|
+
|
|
94
100
|
### Why some limits are missing from that list
|
|
95
101
|
|
|
96
102
|
`chat_rate_limit`, `agent_token_ceiling`, `queue_mode`, `debounce_ms`,
|
|
@@ -304,7 +310,7 @@ What "abandoned" means, exactly:
|
|
|
304
310
|
visible to an *operator* (its `tool_call`/`tool_result` events and the trace), not to
|
|
305
311
|
the model — a half batch in the history would be an invalid prompt.
|
|
306
312
|
|
|
307
|
-
> **No grace knob.**
|
|
313
|
+
> **No grace knob.** An `interrupt_grace_ms` was sketched; it is not
|
|
308
314
|
> implemented, and would buy nothing here. The new turn is queued behind the abandoned
|
|
309
315
|
> one either way (one turn at a time per session is the invariant), and waiting for a
|
|
310
316
|
> boundary inside the request would break the ack-fast rule that put the debounce
|
|
@@ -330,7 +336,10 @@ For tools, a paired deny list (`tools_deny`) **always wins**, and
|
|
|
330
336
|
|
|
331
337
|
Three capabilities invert the default — `nil`/absent means **OFF**, not "all":
|
|
332
338
|
`subagents`, `memory`, and `guardrails` (each defaults to off or a conservative
|
|
333
|
-
setting, never "everything on").
|
|
339
|
+
setting, never "everything on"). `tool_output_compression` is a fourth: opt-in
|
|
340
|
+
mechanical dedupe of repeated tool results in the history (see
|
|
341
|
+
[Context](CONTEXT.md#compaction-is-not-wired--except-the-mechanical-dedupe)),
|
|
342
|
+
off by default because it changes what the model sees.
|
|
334
343
|
|
|
335
344
|
### Declaring what this deployment has
|
|
336
345
|
|
|
@@ -386,6 +395,86 @@ per agent (blank inherits the platform value, `0` explicitly disables it).
|
|
|
386
395
|
> *present but nil* reads as OFF for that agent — leave the key **absent** to
|
|
387
396
|
> inherit. See [Security](SECURITY.md#edge-limits).
|
|
388
397
|
|
|
398
|
+
#### Calendar budgets — the daily/monthly cost wall
|
|
399
|
+
|
|
400
|
+
A third, opt-in ceiling for the *billing* shape the windows above cannot express:
|
|
401
|
+
a spend cap over a CALENDAR day or month, per `(tenant, agent)` when
|
|
402
|
+
multi-tenant. Data on the profile (DSL `budget` or the pack's `budget` key):
|
|
403
|
+
|
|
404
|
+
```ruby
|
|
405
|
+
budget daily: 100_000, monthly: 2_000_000, soft: false # or soft: true
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
- Tokens count the **billed** spend — `input + output + cached + cache_creation`
|
|
409
|
+
(the cached prefix is the bulk of the bill, not an afterthought).
|
|
410
|
+
- **Hard** (the default, `soft` absent/false): a turn that arrives with the
|
|
411
|
+
window's spend already at/over the cap **fails** with the typed
|
|
412
|
+
`Insika::BudgetExceeded` — the envelope reads `budget_exceeded` +
|
|
413
|
+
`retry_after` (seconds until the window rolls). It is NOT a customer reply; it
|
|
414
|
+
is an operator signal.
|
|
415
|
+
- **Soft** (`soft: true`): the same turn RUNS — crossing the cap emits one
|
|
416
|
+
`budget_warning` event per window and injects a note into the context (the
|
|
417
|
+
model sees it, the transcript does not).
|
|
418
|
+
- Either way, crossing `alert_at` (default `0.8` of the cap) fires the same
|
|
419
|
+
warning **before** the wall, once per window.
|
|
420
|
+
|
|
421
|
+
> ⚠️ Unlike the ceilings above, the cap that counts is per **calendar** window —
|
|
422
|
+
> a `daily` budget rolls at UTC midnight, a `monthly` one on the 1st, whatever
|
|
423
|
+
> the sun. `agent_token_window` is a fixed seconds window and cannot express
|
|
424
|
+
> "the day resets at midnight".
|
|
425
|
+
|
|
426
|
+
#### Reliability — retries, fallback, circuit breaker (WS3)
|
|
427
|
+
|
|
428
|
+
The provider interaction is a single attempt by default (RubyLLM's own 2
|
|
429
|
+
transport retries aside). For a store that cannot have a dead model take the
|
|
430
|
+
chat down, the reliability policy is DATA on the profile:
|
|
431
|
+
|
|
432
|
+
```ruby
|
|
433
|
+
reliability retries: 2, backoff: "exponential",
|
|
434
|
+
fallback: ["openai/gpt-4o-mini"],
|
|
435
|
+
circuit_breaker: { after: 10, within: 60, cooldown: 300 }
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
- **Retries** — transient failures (`:retryable` / `:rate_limited_*` per the
|
|
439
|
+
error classification) retry with exponential backoff, up to `retries`.
|
|
440
|
+
A `:fatal` (auth, billing, bad request) is NEVER retried or rotated. Each
|
|
441
|
+
attempt runs on a fresh chat — the customer-visible answer comes only from
|
|
442
|
+
the attempt that returns.
|
|
443
|
+
- **Fallback** — after a node's retries, the turn ROTATES to the next model in
|
|
444
|
+
the chain: the profile's `fallback` refs first, then the platform
|
|
445
|
+
`fallback_models`. The turn's usage is attributed to the model that actually
|
|
446
|
+
spoke (`model_source: "fallback"`).
|
|
447
|
+
- **Circuit breaker** — per `(tenant, provider/model)`: `after` failures within
|
|
448
|
+
`within` seconds open the circuit; while open, the turn fail-fasts with the
|
|
449
|
+
typed `circuit_open` + `retry_after` (remaining cooldown) and the provider is
|
|
450
|
+
never touched. After `cooldown` a half-open trial closes the circuit on
|
|
451
|
+
success or reopens it on failure.
|
|
452
|
+
- **`timeout`** — per-attempt ceiling (default 30s), counted as a retryable
|
|
453
|
+
failure.
|
|
454
|
+
|
|
455
|
+
Absent `reliability` = the plain single attempt, byte-for-byte today's
|
|
456
|
+
behavior.
|
|
457
|
+
|
|
458
|
+
#### Operator alerts — the webhook (WS6)
|
|
459
|
+
|
|
460
|
+
Three operational events — `budget_warning`, `breaker_open`, `delivery_failed` —
|
|
461
|
+
can be answered per agent with a webhook:
|
|
462
|
+
|
|
463
|
+
```ruby
|
|
464
|
+
alerts webhook: "https://ops.example.com/insika-alerts"
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
When present, each such event is POSTed to the URL as JSON (the event's
|
|
468
|
+
type/data/meta, plus the agent). Delivery rides the same outbox + claim +
|
|
469
|
+
bounded-retry pipeline as channel answers — at-most-once, crashed deliveries
|
|
470
|
+
recovered at boot. The engine transports the event and does not interpret it: a
|
|
471
|
+
Slack/CRM adapter is the consumer's. Absent `alerts` = nothing is sent.
|
|
472
|
+
|
|
473
|
+
Separately, with `INSIKA_TURN_TIMING`, the provider's **live TTFB** is carried in
|
|
474
|
+
the streaming envelope: the first content chunk emits an `insika.ttft` frame
|
|
475
|
+
(`ttft_ms`) on `/v1/responses`, alongside the per-turn `timing` breakdown on the
|
|
476
|
+
final `response.completed`. Additive and opt-in — absent by default.
|
|
477
|
+
|
|
389
478
|
### Layer 5: Reasoning (thinking)
|
|
390
479
|
|
|
391
480
|
Controls the model's thinking budget, resolved by precedence
|
|
@@ -422,6 +511,41 @@ and each opted-in channel gets its own frame type, never the answer's. See
|
|
|
422
511
|
> reads. That is the operator's call, which is why it is neither a default nor a
|
|
423
512
|
> global.
|
|
424
513
|
|
|
514
|
+
## The stuck signal — "I cannot proceed" (WS5)
|
|
515
|
+
|
|
516
|
+
The engine doesn't decide what "I can't help you" means — the consumer does. What
|
|
517
|
+
the engine provides is the deterministic signal, so that a product wanting
|
|
518
|
+
**human escalation** can act on it instead of regexing the answer text:
|
|
519
|
+
|
|
520
|
+
```ruby
|
|
521
|
+
stuck_signal true
|
|
522
|
+
```
|
|
523
|
+
|
|
524
|
+
With `stuck_signal` on, the model may call `signal_stuck(reason:, message:)` when it
|
|
525
|
+
determines it cannot proceed (out of scope, missing data, a case a human must take
|
|
526
|
+
over). The turn then **ends** — a final message is published (the model's lead-in,
|
|
527
|
+
or the tool's `message` when it wrote none) — and the contract carries the signal
|
|
528
|
+
twice:
|
|
529
|
+
|
|
530
|
+
- the terminal event `task_completed` gains an additive sibling
|
|
531
|
+
`"outcome": "stuck"` (and the OpenAI `response.completed` frame too), so a
|
|
532
|
+
consumer that only reads the response can react;
|
|
533
|
+
- a dedicated `:turn_stuck` event is published with the `agent`, the `reason`, and
|
|
534
|
+
the final `message` — the subscription point for an operator inbox.
|
|
535
|
+
|
|
536
|
+
```jsonc
|
|
537
|
+
{ "type": "turn_stuck",
|
|
538
|
+
"data": { "agent": "store-support", "reason": "order outside my scope",
|
|
539
|
+
"message": "I'll transfer you to our human team." } }
|
|
540
|
+
```
|
|
541
|
+
|
|
542
|
+
Nothing about handing off, pausing, or resuming is in scope here — escalation is a
|
|
543
|
+
consumer concern. How a human joins the conversation is exactly what
|
|
544
|
+
`MessageOrigin.operator` ([Refinement](REFINEMENT.md#who-wrote-a-message)) stamps
|
|
545
|
+
an imported transcript with; this workstream provides the point at which that
|
|
546
|
+
handoff is *triggered*. Off by default (parity): without `stuck_signal`, the tool
|
|
547
|
+
is not wired and the outcome never appears.
|
|
548
|
+
|
|
425
549
|
## Refinement
|
|
426
550
|
|
|
427
551
|
`refinement` configures how an agent's own traffic is read back as a report — what
|
data/docs/CHANNELS.md
CHANGED
|
@@ -259,7 +259,7 @@ Content-Type: application/json
|
|
|
259
259
|
"external_id": "5511999998888", // required — YOUR key for this conversation
|
|
260
260
|
"message": "queria saber do pedido", // required
|
|
261
261
|
"event_id": "wamid.HBg…", // optional but strongly recommended (dedup)
|
|
262
|
-
"vars": { "store": "
|
|
262
|
+
"vars": { "store": "demo-store" } // optional — session vars on first contact
|
|
263
263
|
}
|
|
264
264
|
```
|
|
265
265
|
|
data/docs/CONTEXT.md
CHANGED
|
@@ -23,7 +23,8 @@ into a deterministic prompt:
|
|
|
23
23
|
| Provider | Block | Priority | Notes |
|
|
24
24
|
|----------|-------|:--------:|-------|
|
|
25
25
|
| **Identity** | system | **100 — pinned** | The agent's prompt files (global system files first). Never cut. |
|
|
26
|
-
| **
|
|
26
|
+
| **Skill trigger** | `<active_skill>` | 85 | Level-2 bodies: the agent's `skills_eager` set, plus the ones whose `triggers:` match the message — see [Skills](SKILLS.md). |
|
|
27
|
+
| **Skills** | `<available_skills>` | 80 | Level-1 skill list, minus whatever is already eager — see [Skills](SKILLS.md). |
|
|
27
28
|
| **Memory** | `<memory>` | 75 | Durable facts + recent notes, only if `memory` is on. Cuttable. |
|
|
28
29
|
| **Tool search** | `<available_tools>` | 70 | Level-1 list of deferred tools — see [Tools](TOOLS.md). |
|
|
29
30
|
| **Session** | history | 60–79 | The running transcript; priority scales with recency. |
|
|
@@ -48,13 +49,29 @@ prefix cache below).
|
|
|
48
49
|
> returns empty turns, raise `context_budget` (e.g. to `60000`) before looking
|
|
49
50
|
> anywhere else. See [Agents](AGENTS.md#default-limits).
|
|
50
51
|
|
|
51
|
-
### Compaction is not wired
|
|
52
|
+
### Compaction is not wired — except the mechanical dedupe
|
|
52
53
|
|
|
53
54
|
There is a settings stub for LLM-summarization compaction (`enabled: false`,
|
|
54
|
-
`keep_last`, a reserved utility-model slot),
|
|
55
|
-
|
|
55
|
+
`keep_last`, a reserved utility-model slot), but **nothing consumes it today**
|
|
56
|
+
— and the Studio no longer shows a form for it, so the setting cannot be
|
|
57
|
+
switched on by accident. Size is managed purely by hard budget eviction.
|
|
56
58
|
Do not rely on compaction to shrink a bloated agent: tune `context_budget` and
|
|
57
|
-
keep the identity lean.
|
|
59
|
+
keep the identity lean.
|
|
60
|
+
|
|
61
|
+
One cheap half **is** wired, opt-in per agent: `tool_output_compression` (DSL
|
|
62
|
+
`tool_output_compression`, or `"tool_output_compression": true` in the pack).
|
|
63
|
+
When on, byte-identical repeated **tool results** in the replayed history
|
|
64
|
+
collapse to a compact back-reference (the first occurrence stays full, with a
|
|
65
|
+
one-line summary) — no LLM involved. It changes what the model sees, so it is
|
|
66
|
+
never a default: an older detail is only in the first occurrence, and a model
|
|
67
|
+
that wants it re-calls the tool. Reach for it when a tool keeps returning the
|
|
68
|
+
same body (a catalog page, a status) and history is the fragment blowing the
|
|
69
|
+
budget first.
|
|
70
|
+
|
|
71
|
+
> The **Studio session screen** shows what the builder assembled per turn —
|
|
72
|
+
> tokens per category (identity, history, memory, …), the tools-schema estimate
|
|
73
|
+
> and the budget verdict (`used / cap`, evicted sources). Counts only, never
|
|
74
|
+
> fragment content.
|
|
58
75
|
|
|
59
76
|
## Memory
|
|
60
77
|
|
data/docs/DEPLOY.md
CHANGED
|
@@ -33,9 +33,12 @@ curl localhost:9292/up # {"status":"ok"}
|
|
|
33
33
|
## The process model
|
|
34
34
|
|
|
35
35
|
The image boots **N Falcon worker processes over one SQLite file**
|
|
36
|
-
(`WEB_CONCURRENCY`, default
|
|
36
|
+
(`WEB_CONCURRENCY`, default 1). That number is a **contract input, not a tuning
|
|
37
37
|
knob**: it decides which engine semantics hold cluster-wide and which are
|
|
38
|
-
per-worker. The
|
|
38
|
+
per-worker. The default is 1 because the per-worker semantics are the product
|
|
39
|
+
— the queue modes (`collect`/`steer`) and FIFO ordering need one
|
|
40
|
+
session actor; raise N only with sticky routing per session in front. The
|
|
41
|
+
contract:
|
|
39
42
|
|
|
40
43
|
> Everything here describes N workers of **one** deployment — one graph, replicated.
|
|
41
44
|
> N *graphs* inside one process is a different contract, and it is
|
|
@@ -76,6 +79,18 @@ per-worker. The contract:
|
|
|
76
79
|
that buffer defaults to 0** — SIGKILL right after SIGTERM, which cancels the
|
|
77
80
|
whole drain — so set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS=30` on the
|
|
78
81
|
service.
|
|
82
|
+
5. **A periodic tick closes the gap between boots.** Serving workers run a tick
|
|
83
|
+
every `INSIKA_TICK_INTERVAL` (default **60s**, `0` disables) as a child of
|
|
84
|
+
the turn supervisor: it re-drives any outbox record left `:pending` (each
|
|
85
|
+
carries its own claim — every worker may drain), and sweeps orphaned tasks
|
|
86
|
+
**without waiting for a deploy** — which is what recovers the orphans of a
|
|
87
|
+
worker respawned mid-generation (item 3's hole). One worker per window
|
|
88
|
+
sweeps (a single transactional claim), and only `:queued`/`:running` tasks
|
|
89
|
+
untouched for `INSIKA_TICK_STALE_AFTER` (default **900s**) are candidates:
|
|
90
|
+
a live turn is bounded by `turn_timeout`, so anything older cannot be alive.
|
|
91
|
+
**If you raise `turn_timeout` past it, raise `INSIKA_TICK_STALE_AFTER` too** —
|
|
92
|
+
the threshold must exceed the largest `turn_timeout` of the deployment.
|
|
93
|
+
`:waiting`/`:paused` tasks are idle by nature and stay boot recovery's.
|
|
79
94
|
|
|
80
95
|
`deploy/entrypoint.sh` sets `WEB_CONCURRENCY` next to a pointer to this section;
|
|
81
96
|
this section is the single source of truth for what changing it means.
|
|
@@ -86,14 +101,16 @@ this section is the single source of truth for what changing it means.
|
|
|
86
101
|
|-----|---------|--------|
|
|
87
102
|
| `INSIKA_DB` | `/data/insika.db` (in the image) | durable SQLite path (**mount a volume!**) |
|
|
88
103
|
| `PORT` | `9292` | HTTP bind port |
|
|
89
|
-
| `WEB_CONCURRENCY` | `
|
|
104
|
+
| `WEB_CONCURRENCY` | `1` | number of Falcon worker processes — a contract input, see [The process model](#the-process-model) |
|
|
90
105
|
| `INSIKA_BOOT_ID` | set by `deploy/entrypoint.sh` | boot generation id; the recovery **task sweep** runs once per id (process model, item 3). Unset = every boot sweeps (single-process default) |
|
|
91
106
|
| `INSIKA_DRAIN_TIMEOUT` | `20` | seconds a stopping worker waits for in-flight turns before abandoning them to the next boot's recovery (process model, item 4). The entrypoint sizes Falcon's `--graceful-stop` from it; on Railway also set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS` ≥ drain + 10 |
|
|
107
|
+
| `INSIKA_TICK_INTERVAL` | `60` | seconds between tick passes — outbox drain + stale recovery sweep (process model, item 5). `0` disables |
|
|
108
|
+
| `INSIKA_TICK_STALE_AFTER` | `900` | seconds a `:queued`/`:running` task must sit untouched before the tick sweeps it. Must exceed the largest `turn_timeout` of the deployment |
|
|
92
109
|
| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN` | Bearer for `/v1/responses` and `/v1/agents` (the API contract) |
|
|
93
110
|
| `ADMIN_TOKEN` | `local-demo` | login token for `/studio` (**change in production**) |
|
|
94
111
|
| `DEEPSEEK_API_KEY` | — | provider key. **Without it the engine still boots** (`/up` green), but turns fail until it is configured (env or Studio → LLM providers) — cloud resilience |
|
|
95
|
-
| `DEEPSEEK_MODEL` | `deepseek-
|
|
96
|
-
| `
|
|
112
|
+
| `DEEPSEEK_MODEL` | `deepseek-v4-flash` | model |
|
|
113
|
+
| `CONSUMER_INTERNAL_URL` | — | base URL for data-tools calling back a consumer's internal API (see below) |
|
|
97
114
|
| `INSIKA_EGRESS_HOSTS` | — | outbound host allowlist (SSRF guard) |
|
|
98
115
|
| `INSIKA_EGRESS_ALLOW_HTTP` / `_ALLOW_PRIVATE` | off | for `http`/loopback callbacks only (**never in cloud**) |
|
|
99
116
|
| `INSIKA_RELAY_TOKEN` | — | **mounts the relay channel** at `POST /channels/relay/events`, and is the Bearer it requires. Empty = the route does not exist (`404`). See [Channels](CHANNELS.md) |
|
|
@@ -165,10 +182,13 @@ insika env # lists known keys + current values (secrets masked)
|
|
|
165
182
|
Checks: env (the schema above), settings schema version (a pending migration →
|
|
166
183
|
`--fix` applies it), a missing platform `default_model` (`--fix` seeds it from
|
|
167
184
|
`DEEPSEEK_MODEL`), durable vs ephemeral backend, LLM provider configured,
|
|
168
|
-
`ADMIN_TOKEN` set, data-tool definitions still valid,
|
|
185
|
+
`ADMIN_TOKEN` set, data-tool definitions still valid, **prompt files that hold
|
|
169
186
|
text rather than a serialized object** (a file whose content is a stringified Hash
|
|
170
187
|
serves a mangled prompt on every turn while looking perfectly healthy — present,
|
|
171
|
-
non-empty, and the agent still answers)
|
|
188
|
+
non-empty, and the agent still answers), and **skill drift** — a shared skill whose
|
|
189
|
+
body names one store, a prompt file routing to a skill the agent cannot load, a broken
|
|
190
|
+
companion pair, a stale `eager:` key (see
|
|
191
|
+
[Skills](SKILLS.md#drift-guards)). Settings-schema migrations are **explicit**
|
|
172
192
|
— no Studio save silently reinterprets old-shape data.
|
|
173
193
|
|
|
174
194
|
### Data-tool callbacks to a backend — via a tunnel
|
|
@@ -178,8 +198,8 @@ cloud** and your backend **on your machine** (`:3000`), expose it over a public
|
|
|
178
198
|
`https` tunnel and point the engine at it:
|
|
179
199
|
|
|
180
200
|
```bash
|
|
181
|
-
# in the tool/manifest: base_url = {{env.
|
|
182
|
-
|
|
201
|
+
# in the tool/manifest: base_url = {{env.CONSUMER_INTERNAL_URL}}
|
|
202
|
+
CONSUMER_INTERNAL_URL=https://your-tunnel.example.dev
|
|
183
203
|
INSIKA_EGRESS_HOSTS=your-tunnel.example.dev
|
|
184
204
|
```
|
|
185
205
|
|
|
@@ -197,7 +217,7 @@ healthcheck, and a restart policy.
|
|
|
197
217
|
2. **Volume**: mount it at `/data` (the default `INSIKA_DB` points there) —
|
|
198
218
|
without a volume, SQLite is ephemeral and recovery resumes nothing after a
|
|
199
219
|
redeploy.
|
|
200
|
-
3. **Vars**: `DEEPSEEK_API_KEY`, `OPENCLAW_GATEWAY_TOKEN`, `
|
|
220
|
+
3. **Vars**: `DEEPSEEK_API_KEY`, `OPENCLAW_GATEWAY_TOKEN`, `CONSUMER_INTERNAL_URL`,
|
|
201
221
|
`INSIKA_EGRESS_HOSTS` (and `WEB_CONCURRENCY` to match your plan/CPU).
|
|
202
222
|
4. The healthcheck hits `/up`.
|
|
203
223
|
5. Point your consumer at the service's public URL, with a matching API token
|
data/docs/EMBEDDING.md
CHANGED
|
@@ -21,7 +21,7 @@ makes your responsibility.
|
|
|
21
21
|
# config/initializers/insika.rb
|
|
22
22
|
INSIKA = Insika.embed(backend: Insika::Stores::SQLite.new(path: Rails.root.join("storage/insika.sqlite3").to_s)) do
|
|
23
23
|
agent "support" do
|
|
24
|
-
model "deepseek-
|
|
24
|
+
model "deepseek-v4-flash"
|
|
25
25
|
provider :deepseek
|
|
26
26
|
api_key ENV.fetch("DEEPSEEK_API_KEY")
|
|
27
27
|
instructions "You answer questions about orders. Be brief."
|
|
@@ -57,7 +57,7 @@ Rails' `mount` (and `Rack::URLMap`, and anything else that moves the prefix into
|
|
|
57
57
|
|
|
58
58
|
## Why the store is an argument
|
|
59
59
|
|
|
60
|
-
|
|
60
|
+
The engine was not always one process — it was one *program*. Two graphs
|
|
61
61
|
built in the same Ruby process shared things they never declared:
|
|
62
62
|
|
|
63
63
|
| What | What actually happened |
|
|
@@ -163,15 +163,19 @@ was given a backend never looks at it.
|
|
|
163
163
|
| The reactor / `supervised` | You, matching your server |
|
|
164
164
|
| Recovery of orphaned turns at boot | You, if you want it — the sweep is `Insika::Recovery`, wired by `Insika::Server::Boot` for the standalone deployment, not by `embed` |
|
|
165
165
|
|
|
166
|
-
### Embedding is not multi-
|
|
166
|
+
### Embedding is not the multi-tenant server
|
|
167
167
|
|
|
168
168
|
Two graphs stop corrupting each other. That is all this contract says. **Who is
|
|
169
169
|
allowed to talk to which graph** is authorization, and it is not here: `token:` is
|
|
170
170
|
a single Bearer gating the whole mounted app, exactly as it does for the
|
|
171
|
-
standalone server
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
171
|
+
standalone server in the default mode. Multi-tenancy is a property of the
|
|
172
|
+
*standalone* server, not of an embed: with `INSIKA_TENANCY=multi_tenant` the
|
|
173
|
+
Bearer resolves to a principal before the routes — per-tenant tokens scoped to
|
|
174
|
+
their own sessions, tasks and events, operator tokens with the run of the
|
|
175
|
+
deployment (see [Security](SECURITY.md#the-bearer-gate)). An embedded graph keeps
|
|
176
|
+
no token store, so an embed's tenancy belongs to the host: put the mounted app
|
|
177
|
+
behind your own authentication and pass `session:` yourself — do not hand the
|
|
178
|
+
mount point to the browser.
|
|
175
179
|
|
|
176
180
|
---
|
|
177
181
|
|
data/docs/EVALS.md
CHANGED
data/docs/LOADTEST.md
CHANGED
|
@@ -23,6 +23,7 @@ There are three scripts, each answering a different question:
|
|
|
23
23
|
| `scripts/bench_store.rb` | Does SQLite (WAL) hold up N processes writing the same file? | No |
|
|
24
24
|
| `scripts/loadtest.rb` | End-to-end: TTFB/total/tokens/cache/error against `/v1/responses` | Yes |
|
|
25
25
|
| `scripts/loadtest-local.sh` | Single-proc baseline vs N-worker multi-proc on one box | Yes |
|
|
26
|
+
| `scripts/loadtest_session.rb` | A full multi-message session (CEP, searches, FAQ) under C concurrent sessions — direct to the engine (`--surface engine`, stream vs steer) or through the consumer's real ingress (`--surface web`, the consumer's widget API) | Yes |
|
|
26
27
|
|
|
27
28
|
All three take `--help` / `-h`.
|
|
28
29
|
|
|
@@ -54,7 +55,7 @@ sits ~100× below this ceiling. See DEPLOY.md for the measured numbers.
|
|
|
54
55
|
## 2. `loadtest.rb` — end-to-end against `/v1/responses` (with provider)
|
|
55
56
|
|
|
56
57
|
Hits `POST /v1/responses` (SSE) directly — the production path
|
|
57
|
-
(
|
|
58
|
+
(a consumer app, e.g. WhatsApp, → engine). Standard library only. Fires `agents × concurrency ×
|
|
58
59
|
iterations` turns in waves of `concurrency`, and per turn records TTFB (time to
|
|
59
60
|
first SSE byte), total time, and the `usage` block (tokens + cache hit) of the last
|
|
60
61
|
frame that carries it.
|
|
@@ -109,7 +110,7 @@ and TTFB drop. Run both to bracket cold vs hot behaviour.
|
|
|
109
110
|
|
|
110
111
|
---
|
|
111
112
|
|
|
112
|
-
## 3. `loadtest-local.sh` — baseline vs multi-worker on one box
|
|
113
|
+
## 3. `loadtest-local.sh` — baseline vs multi-worker on one box
|
|
113
114
|
|
|
114
115
|
Boots Falcon with `--count 1` (single-process baseline), runs the sweep, then boots
|
|
115
116
|
`--count N` (multi-process) over the **same** SQLite file (WAL), runs the sweep
|
data/docs/OBSERVABILITY.md
CHANGED
|
@@ -36,7 +36,16 @@ something that left no task of its own behind:
|
|
|
36
36
|
| `:turn_steered` | `task_id`, `count`, `total` | a message arrived mid-run and was appended to the turn in flight |
|
|
37
37
|
| `:turn_steer_released` | `task_id`, `released_as`, `count` | the run could not absorb it, so it became the turn `released_as` |
|
|
38
38
|
| `:turn_interrupted` | `task_id`, `replaced_by` | the turn was abandoned mid-run, and which turn replaced it |
|
|
39
|
+
| `:turn_stuck` | `task_id`, `agent`, `reason`, `message` | the agent declared it could not proceed (`signal_stuck`, WS5) — the deterministic signal a consumer escalates on |
|
|
39
40
|
| `:channel_delivered` | `channel`, `outbox_id`, `status`, `attempts`, `error` | the answer reached the platform (or did not) — the turn completing says nothing about that |
|
|
41
|
+
| `:delivery_failed` | `channel`, `outbox_id`, `status`, `attempts`, `error` | a delivery exhausted its bounded retries — the alert face of the row above (WS6) |
|
|
42
|
+
| `:budget_warning` | `agent`, `tenant`, `window`, `spent`, `cap` | a calendar budget crossed its threshold (`alert_at` or a soft cap) — once per window (WS2) |
|
|
43
|
+
| `:breaker_open` | `agent`, `ref`, `tenant` | the reliability circuit breaker tripped for a `(tenant, provider/model)` — further turns fail fast until the cooldown (WS3/WS6) |
|
|
44
|
+
| `:ttft` | `task_id`, `session_id`, `ttft_ms` | the provider's time-to-first-token on the streaming envelope — only under `INSIKA_TURN_TIMING`, once per turn (WS6) |
|
|
45
|
+
|
|
46
|
+
`delivery_failed` and `breaker_open` are the two the operator config is pointed at
|
|
47
|
+
(`alerts.webhook` on the profile): each only fires when something durable did
|
|
48
|
+
not land. `:ttft` is additive debug, absent unless `INSIKA_TURN_TIMING` is set.
|
|
40
49
|
|
|
41
50
|
`:channel_delivered` is the one worth alerting on: a turn can be `:task_completed`
|
|
42
51
|
and correct while the customer got nothing, because delivery is a separate,
|
|
@@ -181,13 +190,13 @@ million tokens**:
|
|
|
181
190
|
|
|
182
191
|
```bash
|
|
183
192
|
INSIKA_MODEL_PRICING='{
|
|
184
|
-
"deepseek-
|
|
193
|
+
"deepseek-v4-flash": {"input": 0.27, "output": 1.10, "cached_input": 0.07},
|
|
185
194
|
"claude-sonnet-4-5": {"input": 3.00, "output": 15.00, "cached_input": 0.30, "cache_write": 3.75}
|
|
186
195
|
}'
|
|
187
196
|
```
|
|
188
197
|
|
|
189
198
|
- A key matches the model id the provider reports, **with or without** the
|
|
190
|
-
`provider/` prefix — `deepseek/deepseek-
|
|
199
|
+
`provider/` prefix — `deepseek/deepseek-v4-flash` and `deepseek-v4-flash` both hit the
|
|
191
200
|
same entry.
|
|
192
201
|
- `input` / `output` are required (one of them is enough for the entry to load).
|
|
193
202
|
- `cached_input`, when given, bills cache **reads** at that rate and subtracts them
|