insika 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +199 -5
- data/README.md +8 -2
- data/bin/insika +231 -13
- data/docs/AGENTS.md +505 -6
- data/docs/API.md +56 -0
- data/docs/CHANNELS.md +100 -10
- data/docs/CONTEXT.md +147 -19
- data/docs/DEPLOY.md +34 -11
- data/docs/EMBEDDING.md +11 -7
- data/docs/EVALS.md +20 -1
- data/docs/FACTS.md +135 -0
- data/docs/HARVEST.md +117 -0
- data/docs/LOADTEST.md +17 -10
- data/docs/OBSERVABILITY.md +65 -2
- data/docs/REFINEMENT.md +9 -9
- data/docs/RELEASING.md +34 -7
- data/docs/RUNNING-LOCAL.md +4 -4
- data/docs/SECURITY.md +85 -11
- data/docs/SKILLS.md +189 -3
- data/docs/SOAK.md +127 -0
- data/docs/TOOLS.md +70 -2
- data/docs/WHY.md +1 -1
- data/docs/WORKFLOWS.md +2 -2
- data/docs/domain.md +115 -0
- data/docs/index.md +2 -2
- data/docs/onboarding/start.md +1 -1
- data/lib/insika/agent_profile.rb +228 -26
- data/lib/insika/alert_dispatcher.rb +139 -0
- data/lib/insika/balloon_splitter.rb +102 -0
- data/lib/insika/baseline_store.rb +2 -2
- data/lib/insika/budget_ledger.rb +166 -0
- data/lib/insika/cache_series_store.rb +49 -0
- data/lib/insika/channel_delivery.rb +132 -24
- data/lib/insika/channel_registry.rb +1 -1
- data/lib/insika/channels/relay.rb +80 -6
- data/lib/insika/channels/web/widget.js +2 -2
- data/lib/insika/channels/web.rb +9 -9
- data/lib/insika/channels/webhook.rb +58 -0
- data/lib/insika/chat_builder.rb +145 -13
- data/lib/insika/checkpoint_store.rb +16 -0
- data/lib/insika/circuit_state.rb +114 -0
- data/lib/insika/coercion.rb +8 -0
- data/lib/insika/commands/agent_payload.rb +6 -4
- data/lib/insika/commands/cancel_followup.rb +49 -0
- data/lib/insika/commands/create_agent.rb +2 -2
- data/lib/insika/commands/create_session.rb +1 -1
- data/lib/insika/commands/delete_llm_provider.rb +1 -1
- data/lib/insika/commands/delete_skill.rb +43 -0
- data/lib/insika/commands/delete_tenant_data.rb +95 -0
- data/lib/insika/commands/export_customer_memory.rb +48 -0
- data/lib/insika/commands/forget_customer.rb +117 -0
- data/lib/insika/commands/freeze_funnel_baseline.rb +113 -0
- data/lib/insika/commands/gate_harvest.rb +138 -0
- data/lib/insika/commands/gate_refinement.rb +12 -12
- data/lib/insika/commands/import_mcp_tools.rb +1 -1
- data/lib/insika/commands/import_tools.rb +4 -4
- data/lib/insika/commands/issue_tenant_token.rb +41 -0
- data/lib/insika/commands/judge_shadow_pairs.rb +124 -0
- data/lib/insika/commands/memory_forget_fact.rb +20 -4
- data/lib/insika/commands/memory_put_fact.rb +23 -4
- data/lib/insika/commands/promote_harvest.rb +130 -0
- data/lib/insika/commands/record_outcome.rb +46 -0
- data/lib/insika/commands/record_shadow_reply.rb +68 -0
- data/lib/insika/commands/reject_harvest.rb +38 -0
- data/lib/insika/commands/resolve_proposal.rb +108 -0
- data/lib/insika/commands/resolve_refinement.rb +1 -1
- data/lib/insika/commands/revoke_contact.rb +49 -0
- data/lib/insika/commands/revoke_token.rb +39 -0
- data/lib/insika/commands/rollback_harvest.rb +86 -0
- data/lib/insika/commands/rotate_tenant_token.rb +43 -0
- data/lib/insika/commands/run_distillation.rb +186 -0
- data/lib/insika/commands/run_harvest.rb +393 -0
- data/lib/insika/commands/run_refinement.rb +5 -5
- data/lib/insika/commands/send_message.rb +112 -15
- data/lib/insika/commands/session_purge.rb +67 -0
- data/lib/insika/commands/set_agent_tools.rb +1 -1
- data/lib/insika/commands/set_skill_agents.rb +60 -19
- data/lib/insika/commands/trigger_workflow.rb +1 -1
- data/lib/insika/commands/update_agent.rb +1 -1
- data/lib/insika/commands/write_data_tool.rb +1 -1
- data/lib/insika/commands/write_golden.rb +1 -1
- data/lib/insika/commands/write_skill.rb +19 -9
- data/lib/insika/config_store.rb +8 -4
- data/lib/insika/contact_store.rb +183 -0
- data/lib/insika/context/builder.rb +23 -5
- data/lib/insika/context/fragment.rb +31 -3
- data/lib/insika/context/priority.rb +6 -2
- data/lib/insika/context/provider.rb +17 -3
- data/lib/insika/context/providers/briefing.rb +96 -0
- data/lib/insika/context/providers/memory.rb +16 -7
- data/lib/insika/context/providers/prompt.rb +30 -2
- data/lib/insika/context/providers/request.rb +1 -1
- data/lib/insika/context/providers/session.rb +17 -2
- data/lib/insika/context/providers/skill.rb +7 -1
- data/lib/insika/context/providers/skill_trigger.rb +128 -0
- data/lib/insika/context/providers/tool_search.rb +2 -0
- data/lib/insika/context_trace_store.rb +128 -0
- data/lib/insika/delegation_store.rb +2 -2
- data/lib/insika/distill.rb +224 -0
- data/lib/insika/distill_engine.rb +169 -0
- data/lib/insika/doctor.rb +962 -7
- data/lib/insika/dsl/runtime.rb +20 -11
- data/lib/insika/dsl/server_boot.rb +74 -4
- data/lib/insika/dsl/system.rb +1 -1
- data/lib/insika/dsl.rb +152 -15
- data/lib/insika/edge_limiter.rb +167 -8
- data/lib/insika/egress_guard.rb +3 -3
- data/lib/insika/env_schema.rb +22 -12
- data/lib/insika/errors.rb +72 -5
- data/lib/insika/evals/assertions.rb +15 -14
- data/lib/insika/evals/baseline.rb +3 -3
- data/lib/insika/evals/golden.rb +8 -8
- data/lib/insika/evals/judge.rb +7 -7
- data/lib/insika/evals/pairwise.rb +21 -9
- data/lib/insika/evals/report.rb +2 -2
- data/lib/insika/evals/runner.rb +6 -6
- data/lib/insika/evals/transport.rb +2 -2
- data/lib/insika/event_stream.rb +23 -5
- data/lib/insika/evidence.rb +183 -0
- data/lib/insika/executor.rb +1092 -160
- data/lib/insika/followup_engine.rb +207 -0
- data/lib/insika/followup_policy.rb +221 -0
- data/lib/insika/followup_store.rb +306 -0
- data/lib/insika/frontmatter.rb +1 -1
- data/lib/insika/funnel_declaration.rb +106 -0
- data/lib/insika/funnel_fold.rb +179 -0
- data/lib/insika/funnel_store.rb +163 -0
- data/lib/insika/golden_store.rb +3 -3
- data/lib/insika/grounding/matcher.rb +69 -0
- data/lib/insika/grounding.rb +44 -0
- data/lib/insika/harvest/conversion_gate.rb +159 -0
- data/lib/insika/harvest/criterion.rb +98 -0
- data/lib/insika/harvest/gate.rb +194 -0
- data/lib/insika/harvest/negative_list.rb +199 -0
- data/lib/insika/harvest.rb +241 -0
- data/lib/insika/harvest_engine.rb +193 -0
- data/lib/insika/harvest_store.rb +548 -0
- data/lib/insika/http_client.rb +3 -3
- data/lib/insika/inbound_log.rb +1 -1
- data/lib/insika/llm_configurator.rb +3 -3
- data/lib/insika/loop_detector.rb +143 -0
- data/lib/insika/mcp_http_client.rb +4 -4
- data/lib/insika/mcp_tool_ingestor.rb +6 -6
- data/lib/insika/media.rb +298 -0
- data/lib/insika/memory_audit_store.rb +85 -0
- data/lib/insika/memory_store.rb +264 -23
- data/lib/insika/message_origin.rb +8 -3
- data/lib/insika/model_resolver.rb +1 -1
- data/lib/insika/model_selection.rb +5 -4
- data/lib/insika/model_visible.rb +87 -0
- data/lib/insika/model_visible_trace_store.rb +66 -0
- data/lib/insika/onboarding.rb +8 -3
- data/lib/insika/outbox_store.rb +44 -6
- data/lib/insika/outcome_store.rb +147 -0
- data/lib/insika/overlay_tool_registry.rb +3 -4
- data/lib/insika/pack.rb +3 -3
- data/lib/insika/pack_importer.rb +17 -15
- data/lib/insika/packaging.rb +163 -0
- data/lib/insika/parity/criterion.rb +79 -0
- data/lib/insika/parity/verdict.rb +318 -0
- data/lib/insika/pending_action_store.rb +1 -1
- data/lib/insika/plugin/loader.rb +2 -2
- data/lib/insika/policy/policy.rb +1 -1
- data/lib/insika/prefix_fingerprint.rb +58 -0
- data/lib/insika/profile_source.rb +34 -7
- data/lib/insika/proposal_store.rb +271 -0
- data/lib/insika/provider_error_classifier.rb +160 -0
- data/lib/insika/queue_policy.rb +6 -3
- data/lib/insika/recovery.rb +47 -6
- data/lib/insika/refinement/candidate.rb +4 -4
- data/lib/insika/refinement/evidence_collector.rb +6 -6
- data/lib/insika/refinement/gate.rb +7 -7
- data/lib/insika/refinement/panel.rb +7 -7
- data/lib/insika/refinement/proposer.rb +10 -10
- data/lib/insika/refinement_store.rb +12 -12
- data/lib/insika/reliability.rb +211 -0
- data/lib/insika/retention.rb +281 -0
- data/lib/insika/routing.rb +101 -0
- data/lib/insika/safety/config.rb +46 -6
- data/lib/insika/safety/corpus.rb +255 -0
- data/lib/insika/safety/detectors.rb +34 -115
- data/lib/insika/safety/factory.rb +18 -5
- data/lib/insika/safety/grounding_enforcer.rb +59 -0
- data/lib/insika/safety/grounding_validator.rb +49 -0
- data/lib/insika/safety/input_guardrail.rb +20 -5
- data/lib/insika/safety/moderator.rb +19 -11
- data/lib/insika/safety/output_filter.rb +10 -6
- data/lib/insika/safety/output_validator.rb +13 -7
- data/lib/insika/safety/safe_responses.rb +1 -1
- data/lib/insika/sandbox/boundary.rb +2 -2
- data/lib/insika/sandbox.rb +1 -1
- data/lib/insika/schema_guard.rb +35 -0
- data/lib/insika/server/app.rb +366 -54
- data/lib/insika/server/boot.rb +4 -4
- data/lib/insika/server/rack_app.rb +31 -7
- data/lib/insika/server/responses.rb +58 -9
- data/lib/insika/server/tenant_auth.rb +61 -0
- data/lib/insika/session_actor.rb +11 -7
- data/lib/insika/session_store.rb +66 -3
- data/lib/insika/settings_store.rb +15 -5
- data/lib/insika/shadow_pair_store.rb +258 -0
- data/lib/insika/shutdown.rb +4 -4
- data/lib/insika/skill_catalog.rb +131 -20
- data/lib/insika/skill_store.rb +70 -22
- data/lib/insika/soak/envelope.rb +140 -0
- data/lib/insika/soak/report.rb +392 -0
- data/lib/insika/soak/runner.rb +554 -0
- data/lib/insika/steer_injector.rb +1 -1
- data/lib/insika/store.rb +11 -2
- data/lib/insika/stores/memory.rb +6 -0
- data/lib/insika/stores/sqlite.rb +8 -0
- data/lib/insika/studio/app.rb +1058 -75
- data/lib/insika/studio/assets/dist/application.css +1 -1
- data/lib/insika/studio/assets/dist/application.js +27 -26
- data/lib/insika/studio/assets/dist/favicon.svg +6 -0
- data/lib/insika/studio/forms.rb +274 -22
- data/lib/insika/studio/nav_icons.rb +7 -2
- data/lib/insika/studio/views/_message.erb +2 -2
- data/lib/insika/studio/views/agent_detail.erb +629 -86
- data/lib/insika/studio/views/agents.erb +11 -7
- data/lib/insika/studio/views/approvals.erb +4 -1
- data/lib/insika/studio/views/chats.erb +4 -1
- data/lib/insika/studio/views/customer.erb +94 -0
- data/lib/insika/studio/views/customers.erb +32 -0
- data/lib/insika/studio/views/evals.erb +4 -1
- data/lib/insika/studio/views/facts.erb +133 -0
- data/lib/insika/studio/views/followups.erb +125 -0
- data/lib/insika/studio/views/funnel.erb +106 -0
- data/lib/insika/studio/views/harvest.erb +234 -0
- data/lib/insika/studio/views/home.erb +2 -1
- data/lib/insika/studio/views/layout.erb +1 -0
- data/lib/insika/studio/views/parity.erb +147 -0
- data/lib/insika/studio/views/playground.erb +7 -1
- data/lib/insika/studio/views/refinement.erb +4 -4
- data/lib/insika/studio/views/session.erb +133 -3
- data/lib/insika/studio/views/settings.erb +9 -12
- data/lib/insika/studio/views/skills.erb +66 -12
- data/lib/insika/studio/views/system_files.erb +1 -1
- data/lib/insika/studio/views/task.erb +13 -0
- data/lib/insika/studio/views/tasks.erb +4 -1
- data/lib/insika/studio/views/tools.erb +0 -1
- data/lib/insika/subagent_graph.rb +3 -3
- data/lib/insika/task_actor.rb +3 -3
- data/lib/insika/task_store.rb +22 -2
- data/lib/insika/telemetry/pricing.rb +3 -3
- data/lib/insika/telemetry/recorder.rb +1 -1
- data/lib/insika/telemetry.rb +2 -2
- data/lib/insika/testing/store_contract.rb +54 -33
- data/lib/insika/tick.rb +146 -0
- data/lib/insika/token_store.rb +168 -0
- data/lib/insika/tool_assembly.rb +5 -5
- data/lib/insika/tool_definition.rb +25 -15
- data/lib/insika/tool_envelope.rb +70 -1
- data/lib/insika/tool_manifest.rb +11 -7
- data/lib/insika/tool_output_compressor.rb +100 -0
- data/lib/insika/tool_store.rb +1 -1
- data/lib/insika/tool_trace_store.rb +1 -1
- data/lib/insika/tools/concurrency.rb +2 -2
- data/lib/insika/tools/data_defined_tool.rb +14 -5
- data/lib/insika/tools/generate_image.rb +44 -0
- data/lib/insika/tools/load_skill.rb +61 -3
- data/lib/insika/tools/schedule_followup.rb +164 -0
- data/lib/insika/tools/stuck_signal.rb +44 -0
- data/lib/insika/tools/subagent.rb +4 -4
- data/lib/insika/tools/subagents.rb +1 -1
- data/lib/insika/tools/tts.rb +47 -0
- data/lib/insika/tools/update_briefing.rb +126 -0
- data/lib/insika/turn_output.rb +2 -2
- data/lib/insika/turn_state.rb +54 -13
- data/lib/insika/turn_timing.rb +24 -4
- data/lib/insika/usage_ledger.rb +1 -1
- data/lib/insika/version.rb +1 -1
- data/lib/insika/vitals.rb +84 -0
- data/lib/insika/wiring/graph.rb +372 -34
- data/lib/insika/workflow.rb +1 -1
- data/lib/insika/workflow_registry.rb +1 -1
- data/lib/insika.rb +122 -16
- metadata +95 -2
- data/lib/insika/server/admin_auth.rb +0 -29
data/docs/FACTS.md
ADDED
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Facts
|
|
3
|
+
parent: Operate & prove it
|
|
4
|
+
nav_order: 6
|
|
5
|
+
permalink: /facts/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Facts — distilled customer memory, human-gated
|
|
9
|
+
|
|
10
|
+
Finished conversations teach a shop things — the customer's size, their budget,
|
|
11
|
+
how they like to pay — and today that knowledge dies with the session. The
|
|
12
|
+
engine can read it back out of the transcripts: **distillation** turns an idle,
|
|
13
|
+
finished customer conversation into a list of proposed facts, a human approves
|
|
14
|
+
or rejects them in the Studio, and an approved fact lands in the customer's
|
|
15
|
+
memory cell — the same cell the `<memory>` block injects on every later turn.
|
|
16
|
+
|
|
17
|
+
The loop has one hard rule: **nothing is ever applied automatically.** Zero
|
|
18
|
+
facts reach the store without a click. Distillation proposes; the operator
|
|
19
|
+
decides; the engine never applies its own proposal.
|
|
20
|
+
|
|
21
|
+
## The loop
|
|
22
|
+
|
|
23
|
+
1. A customer conversation (a session tagged with a `customer`, per
|
|
24
|
+
[Context](CONTEXT.md#memory)) goes idle — nobody has written for the
|
|
25
|
+
configured hours.
|
|
26
|
+
2. The engine's distillation duty picks the session, sends its transcript to
|
|
27
|
+
the platform `utility_model` (off the turn path, on its own worker fiber),
|
|
28
|
+
and the model answers with a JSON list of durable facts: `name`, `value`,
|
|
29
|
+
an optional `confidence`, and the transcript message indexes that support
|
|
30
|
+
the fact (the **evidence**).
|
|
31
|
+
3. The engine filters the answer against a safe subset (no invented scopes, no
|
|
32
|
+
oversized values, no out-of-range evidence), dedups it against the ledger,
|
|
33
|
+
and writes the survivors as **proposals**.
|
|
34
|
+
4. The **Facts** page in the Studio shows the pending proposals with their
|
|
35
|
+
evidence excerpt. The operator **approves** (the fact is written to the
|
|
36
|
+
customer's memory, stamped with its origin), **rejects** (optionally with a
|
|
37
|
+
reason) or **dismisses** (it will never be proposed again).
|
|
38
|
+
5. Approved facts join the customer's memory cell and are injected by the
|
|
39
|
+
Memory provider on the next turn of any session of that customer.
|
|
40
|
+
|
|
41
|
+
## Enabling it — the `distill:` block
|
|
42
|
+
|
|
43
|
+
Distillation is pack data on the agent, exactly like `refinement:` or
|
|
44
|
+
`followup:` — absent = the feature is off for that agent, byte-identical engine:
|
|
45
|
+
|
|
46
|
+
```ruby
|
|
47
|
+
agent = Insika.agent("store-support") do
|
|
48
|
+
instructions "…"
|
|
49
|
+
distill enabled: true,
|
|
50
|
+
idle_hours: 6, # how idle a session must be before it distills
|
|
51
|
+
min_messages: 3, # a shorter session distills noise, not facts
|
|
52
|
+
max_proposals: 10 # cap per session pass
|
|
53
|
+
# prompt: "<what counts as a fact for THIS store>" — the pack-authored half;
|
|
54
|
+
# absent = the engine's generic prompt. `model:` (absent = the platform
|
|
55
|
+
# utility_model) can name the distiller explicitly.
|
|
56
|
+
end
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The same keys work in a pack's `agent.config.json`. `idle_hours` /
|
|
60
|
+
`min_messages` / `max_proposals` are per-agent data; `prompt` is the store's
|
|
61
|
+
half — what counts as a fact for a fashion store ("size, preference, budget")
|
|
62
|
+
is not what counts for a logistics one ("address, delivery window, carrier").
|
|
63
|
+
The engine never writes store vocabulary.
|
|
64
|
+
|
|
65
|
+
## The human gate, precisely
|
|
66
|
+
|
|
67
|
+
The **Facts** page (operate group, next to Follow-ups) shows:
|
|
68
|
+
|
|
69
|
+
- **Pending** — oldest first, because evidence ages. Each card is the fact,
|
|
70
|
+
its confidence, its scope (`tenant:customer`), and the evidence excerpt read
|
|
71
|
+
from the transcript at request time (evidence is a link, never a copy). Three
|
|
72
|
+
buttons: **Approve & save to memory**, **Reject** (with an optional reason,
|
|
73
|
+
shown on the card), **Dismiss** (ghost — labelled "will not be proposed
|
|
74
|
+
again").
|
|
75
|
+
- **Stale** — the CAS-lost re-present (below): the proposed value struck
|
|
76
|
+
through next to the operator's current value, resolved by dismissal.
|
|
77
|
+
- **Recent** — every resolved proposal, most recent first, with operator and
|
|
78
|
+
note.
|
|
79
|
+
|
|
80
|
+
**The latch** — a dismissed *or* rejected `(name, value)` tuple is never
|
|
81
|
+
proposed again. The proposal rows ARE the ledger: a human saw that tuple and
|
|
82
|
+
said no, and re-proposing it would train the operator to stop reading. An
|
|
83
|
+
unanswered proposal is never piled on, either. A *different* value for the same
|
|
84
|
+
name is a different tuple — "wears M" dismissed does not block "wears L".
|
|
85
|
+
|
|
86
|
+
**The CAS guarantee** — approval never silently overwrites an operator edit.
|
|
87
|
+
At distill time the engine records the target fact's existence and revision; at
|
|
88
|
+
approve time it writes through the store's optimistic compare-and-swap. A fact
|
|
89
|
+
the operator moved in between flips the proposal to `stale` with both values
|
|
90
|
+
visible — the operator's edit always wins, never a silent overwrite.
|
|
91
|
+
|
|
92
|
+
## Provenance
|
|
93
|
+
|
|
94
|
+
An approved fact is written with `origin: "distilled:<session_ref>"` — the
|
|
95
|
+
RFC-0031 provenance discipline: `"engine"` (the `remember` tool), `"operator"`
|
|
96
|
+
(Studio edits), `"legacy"`, and `"distilled"` (+ the session that produced it).
|
|
97
|
+
The Closed loop reads the same cell every later turn injects, and the
|
|
98
|
+
distillation ledger suppresses re-proposing a fact that is already applied with
|
|
99
|
+
a `distilled:` origin.
|
|
100
|
+
|
|
101
|
+
## LGPD
|
|
102
|
+
|
|
103
|
+
Distilled facts are personal data, and the engine treats them like it:
|
|
104
|
+
|
|
105
|
+
- **Forget a customer** — `forget_customer` purges the customer's proposals
|
|
106
|
+
(every status) along with their memory cell and sessions.
|
|
107
|
+
- **Delete a tenant** — `delete_tenant_data` purges the tenant's proposals.
|
|
108
|
+
- **Retention** — proposals age out under the same `retention_days` sweep as
|
|
109
|
+
the rest of the footprint (pending included — a proposal is evidence of a
|
|
110
|
+
transcript, and when the transcript dies the pending fact is stale). The
|
|
111
|
+
session marker dies with its proposals: **an unreviewed proposal that ages
|
|
112
|
+
out is expired, not lost-locked** — the session is re-distillable, and a
|
|
113
|
+
duplicate survivor is filtered by the ledger, never applied.
|
|
114
|
+
|
|
115
|
+
## The honest limits
|
|
116
|
+
|
|
117
|
+
- **Best-effort extraction, re-scan recovery.** There is no distillation
|
|
118
|
+
queue. A crash mid-pass leaves the session unmarked; the next pass re-scans
|
|
119
|
+
it, and the ledger filters any duplicate proposal. Exactly-once is not
|
|
120
|
+
claimed — facts are re-derivable, and a duplicate is filtered, never applied.
|
|
121
|
+
- **Precision is a forge audit.** The engine guarantees the *gates* (schema,
|
|
122
|
+
dedup, CAS, human approval); it cannot guarantee the *model's judgment*.
|
|
123
|
+
"Is this fact true and durable?" is audited on real traffic, per store — the
|
|
124
|
+
pack prompt is where that judgment is tuned.
|
|
125
|
+
- **Scope comes from the session, never the model.** The proposal's landing
|
|
126
|
+
cell is assembled by the engine from the session's tenant and customer; the
|
|
127
|
+
schema rejects a model-authored scope outright (a cross-tenant escape), and
|
|
128
|
+
an untagged session is never distilled.
|
|
129
|
+
- **Sessions only.** Distillation targets customer-tagged sessions; a
|
|
130
|
+
session without a customer has no landing zone and is skipped.
|
|
131
|
+
- **One stamp, once.** A session receives `vars["agent"]` on the same write
|
|
132
|
+
that stamps its `customer` — the first tagged turn. The stamp is
|
|
133
|
+
idempotent: a session that already carries a `customer` (a pre-upgrade
|
|
134
|
+
conversation, or one that since moved to this agent) never gains `agent`
|
|
135
|
+
on its own, so it can never participate in distillation.
|
data/docs/HARVEST.md
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Harvest
|
|
3
|
+
parent: Operate & prove it
|
|
4
|
+
nav_order: 7
|
|
5
|
+
permalink: /harvest/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Harvest — skills from real traffic, promoted only if eval AND conversion hold
|
|
9
|
+
|
|
10
|
+
The harvest is the loop that makes a store smarter with its own traffic: it
|
|
11
|
+
reads the store's **finished** conversations, asks a model to propose SKILLS
|
|
12
|
+
for the agent's playbook, filters every proposal through two pre-registered
|
|
13
|
+
lists (the negative list, the evidence ledger), scores it with a double gate,
|
|
14
|
+
and — **only after a human approves** — lands it as a live skill. Nothing is
|
|
15
|
+
ever applied automatically, and a skill that fails the gate is terminal: the
|
|
16
|
+
same finding must re-surface with new evidence, there is no silent retry.
|
|
17
|
+
|
|
18
|
+
Per-store data (`harvest:` on the agent), absent = the loop is off for that
|
|
19
|
+
agent:
|
|
20
|
+
|
|
21
|
+
```ruby
|
|
22
|
+
harvest enabled: true,
|
|
23
|
+
negative_list: [ { rule: "no-competitor-prices", pattern: "concorrente" } ],
|
|
24
|
+
miner: { model: "deepseek-v4-flash", # absent = the platform utility_model
|
|
25
|
+
window: { last_sessions: 200 },
|
|
26
|
+
max_proposals: 10,
|
|
27
|
+
budget: { tokens: 100_000 } },
|
|
28
|
+
idle_hours: 24, # how idle a session must be before it mines
|
|
29
|
+
min_messages: 3 # a shorter session mines noise
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
## The pass
|
|
33
|
+
|
|
34
|
+
A mine reads ONLY durable, finished data — never an in-flight turn, never a
|
|
35
|
+
live prompt prefix (the "fork" is structural: the mining writes nothing to the
|
|
36
|
+
sessions it read, so a customer turn's cache is untouched by construction). The
|
|
37
|
+
miner gets the transcript slices (masked), the session's evidence-ledger ids,
|
|
38
|
+
and the agent's current skill names so it does not re-propose them. Then the
|
|
39
|
+
filters, each drop counted and logged:
|
|
40
|
+
|
|
41
|
+
1. **The negative list** — the versioned rules file (seed, imported
|
|
42
|
+
per store by `insika harvest:negative import --agent ID --file F`) and the
|
|
43
|
+
profile's hot-editable `harvest.negative_list`. A rule is a phrase or a
|
|
44
|
+
regex; phrases match case/accent-folded at word boundaries. Every rejected
|
|
45
|
+
candidate is logged with the rule id.
|
|
46
|
+
2. **The grounding filter** — every product reference in a proposal must be
|
|
47
|
+
in the union of the origin sessions' evidence ids (RFC-0029's ledger).
|
|
48
|
+
A store without `grounding.matcher.sku` does not mine at all: product
|
|
49
|
+
claims that cannot be verified are blocked by refusal, not by prompt.
|
|
50
|
+
3. **Dedup** — an open `(agent, name)` tuple or a skill the store already has.
|
|
51
|
+
|
|
52
|
+
## The double gate
|
|
53
|
+
|
|
54
|
+
A candidate that survives mining is scored by RUNNING it — the eval gate
|
|
55
|
+
clones the agent, writes the candidate skill into the clone's agent-scoped
|
|
56
|
+
store, enables it on the clone's allowlist, replays the golden set over the
|
|
57
|
+
ordinary public surface and compares to the accepted baseline. **Any
|
|
58
|
+
regression disqualifies** — the gate is a veto, never a score to argue with.
|
|
59
|
+
Judges are mandatory in exactly the shapes the refinement gate already
|
|
60
|
+
refuses: no recorded baseline, an all-red baseline, and a judged baseline
|
|
61
|
+
replayed with no judge (a rubric'd case with no verdict would count as a
|
|
62
|
+
pass — the P18 lesson, see [Evals](EVALS.md)).
|
|
63
|
+
|
|
64
|
+
The conversion gate is the second ruler: the store's funnel metric over the
|
|
65
|
+
criterion's window, compared to the **frozen baseline** (the RFC-0032
|
|
66
|
+
`freeze_funnel_baseline`). Outcome is evidence — this gate can only say "the
|
|
67
|
+
store is measurably worse than the accepted state" or "there is nothing to
|
|
68
|
+
compare against". It refuses on missing data, never passes: no frozen
|
|
69
|
+
baseline, no criterion, no funnel store, a fold that has not converged — each
|
|
70
|
+
named on the Harvest page.
|
|
71
|
+
|
|
72
|
+
Both passed → the candidate awaits **a human**. Promotion is snapshot-first
|
|
73
|
+
(the pre-promotion content + allowlist), then the two existing write commands,
|
|
74
|
+
then an append-only log row carrying `skill`, `origin`, `eval_ref`,
|
|
75
|
+
`conversion_ref`, `approver`, `snapshot_ref` and the criterion's sha. The
|
|
76
|
+
conversion ruler is re-read at the moment the skill lands — a store that
|
|
77
|
+
dipped below its frozen baseline since gating parks the promotion with the
|
|
78
|
+
current numbers, and a criterion file that changed since boot is a criterion
|
|
79
|
+
nobody froze.
|
|
80
|
+
|
|
81
|
+
## Rollback
|
|
82
|
+
|
|
83
|
+
One click, deterministic: the snapshot is restored — `WriteSkill` with the
|
|
84
|
+
pre-promotion bytes (or `DeleteSkill` when the skill did not exist), the
|
|
85
|
+
allowlist restored to the snapshot's set, and the promotion row stamped
|
|
86
|
+
`rolled_back_at`. The log stays the single ledger: a skill promoted, rolled
|
|
87
|
+
back, re-promoted is three readable rows.
|
|
88
|
+
|
|
89
|
+
## The honest limits
|
|
90
|
+
|
|
91
|
+
- **A store with no golden cases cannot gate, and cannot promote.** The gate's
|
|
92
|
+
strength is entirely the golden set.
|
|
93
|
+
- **Grounding first.** A store without a matcher does not mine — the product
|
|
94
|
+
loop is blocked until the evidence ledger is live (by refusal, not by
|
|
95
|
+
warning).
|
|
96
|
+
- **A promoted skill is live text**, like any skill: the SkillStore's version
|
|
97
|
+
history plus the harvest snapshot plus the log make the rollback path
|
|
98
|
+
deterministic.
|
|
99
|
+
- **The first-10 audit is human.** The negative list can carry a false
|
|
100
|
+
restriction (a phrase too broad) — that is exactly the Hermes failure, and
|
|
101
|
+
it is caught by the audit, not by the engine: the list grows and the loop
|
|
102
|
+
stops (`harvest.enabled: false`), both by data.
|
|
103
|
+
- **The harvest costs provider money**: one miner call per window plus one
|
|
104
|
+
full golden replay per gated candidate. The `budget` cap, the recorded run
|
|
105
|
+
cost and the manual trigger bound it; the automated loop mines one session
|
|
106
|
+
per claim window.
|
|
107
|
+
|
|
108
|
+
## The operator surface
|
|
109
|
+
|
|
110
|
+
- `insika harvest --agent ID [--last-sessions N] [--since ISO] [--full]` — mine one window.
|
|
111
|
+
- `insika harvest:negative import --agent ID --file F` — seed the profile's list from a rules file.
|
|
112
|
+
- `insika harvest:criterion check --file F` — strict-load the frozen conversion criterion (the hook before any promotion).
|
|
113
|
+
- The **Harvest page** in the Studio: the human's inbox (each candidate with
|
|
114
|
+
its evidence excerpt, the eval report, the conversion card, promote/reject),
|
|
115
|
+
the gated-but-blocked rows with the named ruler hole, the pending list, the
|
|
116
|
+
append-only promoted log with the rollback mirror, the negative list with
|
|
117
|
+
per-rule rejection counts, and the criterion block read-only.
|
data/docs/LOADTEST.md
CHANGED
|
@@ -16,15 +16,21 @@ The whole point: the engine exposes `POST /v1/responses` as an **SSE drop-in** o
|
|
|
16
16
|
the OpenClaw gateway. Same contract → the same load tools work against either side,
|
|
17
17
|
so you can measure the engine you are about to ship against the gateway it replaces.
|
|
18
18
|
|
|
19
|
-
There are
|
|
19
|
+
There are four scripts, each answering a different question:
|
|
20
20
|
|
|
21
21
|
| Script | Question it answers | Needs a provider? |
|
|
22
22
|
|--------|---------------------|-------------------|
|
|
23
23
|
| `scripts/bench_store.rb` | Does SQLite (WAL) hold up N processes writing the same file? | No |
|
|
24
24
|
| `scripts/loadtest.rb` | End-to-end: TTFB/total/tokens/cache/error against `/v1/responses` | Yes |
|
|
25
25
|
| `scripts/loadtest-local.sh` | Single-proc baseline vs N-worker multi-proc on one box | Yes |
|
|
26
|
+
| `scripts/loadtest_session.rb` | A full multi-message session (CEP, searches, FAQ) under C concurrent sessions — direct to the engine (`--surface engine`, stream vs steer) or through the consumer's real ingress (`--surface web`, the consumer's widget API) | Yes |
|
|
27
|
+
| `insika soak` | Does the deploy degrade over 72 h of steady load? | Yes |
|
|
26
28
|
|
|
27
|
-
|
|
29
|
+
The first four take `--help` / `-h`; the soak is a shipped command (`insika soak --help`) rather
|
|
30
|
+
than a repo script. Bursts and uptime are different questions: a wave driver
|
|
31
|
+
measures a burst, and the soak's arrival process measures *degradation over
|
|
32
|
+
uptime* — the load-test table above deliberately stops where
|
|
33
|
+
[Soak](SOAK.md) begins.
|
|
28
34
|
|
|
29
35
|
---
|
|
30
36
|
|
|
@@ -54,7 +60,7 @@ sits ~100× below this ceiling. See DEPLOY.md for the measured numbers.
|
|
|
54
60
|
## 2. `loadtest.rb` — end-to-end against `/v1/responses` (with provider)
|
|
55
61
|
|
|
56
62
|
Hits `POST /v1/responses` (SSE) directly — the production path
|
|
57
|
-
(
|
|
63
|
+
(a consumer app, e.g. WhatsApp, → engine). Standard library only. Fires `agents × concurrency ×
|
|
58
64
|
iterations` turns in waves of `concurrency`, and per turn records TTFB (time to
|
|
59
65
|
first SSE byte), total time, and the `usage` block (tokens + cache hit) of the last
|
|
60
66
|
frame that carries it.
|
|
@@ -63,7 +69,7 @@ frame that carries it.
|
|
|
63
69
|
INSIKA_URL=http://localhost:9292 \
|
|
64
70
|
OPENCLAW_GATEWAY_TOKEN=xxx \
|
|
65
71
|
bundle exec ruby scripts/loadtest.rb \
|
|
66
|
-
--agents
|
|
72
|
+
--agents demo,my-store --concurrency 16 --iterations 3 \
|
|
67
73
|
--message "hi, how are you?"
|
|
68
74
|
```
|
|
69
75
|
|
|
@@ -74,7 +80,7 @@ Runs against a local server **or** a remote one (e.g. Railway) — just point
|
|
|
74
80
|
|
|
75
81
|
| Flag | Default | Meaning |
|
|
76
82
|
|------|---------|---------|
|
|
77
|
-
| `--agents a,b,c` | `
|
|
83
|
+
| `--agents a,b,c` | `demo` | comma-separated agent ids (mapped to `model: openclaw:<agent>`) |
|
|
78
84
|
| `--concurrency N` | `8` | concurrent turns per wave |
|
|
79
85
|
| `--iterations N` | `1` | number of waves per agent |
|
|
80
86
|
| `--message TEXT` | greeting | user message sent every turn |
|
|
@@ -97,7 +103,7 @@ Use `--dry-run` to sanity-check your flags/URL/token before firing real traffic
|
|
|
97
103
|
|
|
98
104
|
```bash
|
|
99
105
|
INSIKA_URL=http://localhost:9292 OPENCLAW_GATEWAY_TOKEN=xxx \
|
|
100
|
-
bundle exec ruby scripts/loadtest.rb --agents
|
|
106
|
+
bundle exec ruby scripts/loadtest.rb --agents demo --concurrency 16 --dry-run
|
|
101
107
|
```
|
|
102
108
|
|
|
103
109
|
### `--same-user` and the cache
|
|
@@ -109,7 +115,7 @@ and TTFB drop. Run both to bracket cold vs hot behaviour.
|
|
|
109
115
|
|
|
110
116
|
---
|
|
111
117
|
|
|
112
|
-
## 3. `loadtest-local.sh` — baseline vs multi-worker on one box
|
|
118
|
+
## 3. `loadtest-local.sh` — baseline vs multi-worker on one box
|
|
113
119
|
|
|
114
120
|
Boots Falcon with `--count 1` (single-process baseline), runs the sweep, then boots
|
|
115
121
|
`--count N` (multi-process) over the **same** SQLite file (WAL), runs the sweep
|
|
@@ -129,7 +135,7 @@ DEEPSEEK_API_KEY=sk-... ./scripts/loadtest-local.sh [WORKERS] [CONCURRENCY]
|
|
|
129
135
|
| `DEEPSEEK_API_KEY` | — (required) | real turns hit the provider; also auto-sourced from `.env.local` |
|
|
130
136
|
| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN`, then `local-demo` | Bearer for the sweep |
|
|
131
137
|
| `PORT` | `9299` | bind port for the local Falcon |
|
|
132
|
-
| `AGENT` | `
|
|
138
|
+
| `AGENT` | `demo` | agent id to load |
|
|
133
139
|
|
|
134
140
|
The final block prints the lock counts; the expected reading is `0` for both:
|
|
135
141
|
|
|
@@ -162,7 +168,7 @@ drop-in for the gateway, you only change **where it points**:
|
|
|
162
168
|
# In the OpenClaw checkout, run its gateway loadtest against the HARNESS:
|
|
163
169
|
OPENCLAW_GATEWAY_URL=http://localhost:9292 \
|
|
164
170
|
OPENCLAW_GATEWAY_TOKEN=<same bearer the engine accepts> \
|
|
165
|
-
node scripts/loadtest-gateway.mjs --agents
|
|
171
|
+
node scripts/loadtest-gateway.mjs --agents demo --concurrency 16 --iterations 3
|
|
166
172
|
```
|
|
167
173
|
|
|
168
174
|
Then run the exact same command with `OPENCLAW_GATEWAY_URL` pointing at the real
|
|
@@ -173,7 +179,7 @@ gateway, and diff the two reports. This is the shadow comparison the pilot needs
|
|
|
173
179
|
- The OpenClaw checkout containing `scripts/loadtest-gateway.mjs` and Node installed.
|
|
174
180
|
- A **bearer token accepted by both** sides. For the engine that is
|
|
175
181
|
`OPENCLAW_GATEWAY_TOKEN` (see DEPLOY.md); point the gateway run at its own token.
|
|
176
|
-
- **The same agent id provisioned on both** sides (e.g. `
|
|
182
|
+
- **The same agent id provisioned on both** sides (e.g. `demo`) so `model:
|
|
177
183
|
openclaw:<agent>` resolves on each. On the engine, provision via
|
|
178
184
|
`scripts/import_pack.rb`.
|
|
179
185
|
- The **same provider** (or an equivalent-latency one) behind each, otherwise you
|
|
@@ -223,6 +229,7 @@ Work top-down and **measure before assuming** — avoid premature topology optim
|
|
|
223
229
|
| 4 | Insika vs gateway, identical knobs | §4 (either method) | Is the engine at parity with the engine it replaces before cut-over? |
|
|
224
230
|
| 5 | Cold vs hot conversation (cache) | `loadtest.rb` with/without `--same-user 1` | Expected steady-state cost/latency once conversations warm up. |
|
|
225
231
|
| 6 | Remote (Railway) vs local | `loadtest.rb` with `INSIKA_URL` remote | Network/deploy overhead of the real environment. |
|
|
232
|
+
| 7 | Degradation over uptime (72 h) | `insika soak` (see [Soak](SOAK.md)) | The cut argument: does latency or memory grow with uptime? Run it last — it needs the topology settled first. |
|
|
226
233
|
|
|
227
234
|
**Reaching for horizontal scale is only justified after 1–3 show the single box is
|
|
228
235
|
the limit.** If it is, the paths are: sharding-by-tenant +
|
data/docs/OBSERVABILITY.md
CHANGED
|
@@ -36,7 +36,27 @@ something that left no task of its own behind:
|
|
|
36
36
|
| `:turn_steered` | `task_id`, `count`, `total` | a message arrived mid-run and was appended to the turn in flight |
|
|
37
37
|
| `:turn_steer_released` | `task_id`, `released_as`, `count` | the run could not absorb it, so it became the turn `released_as` |
|
|
38
38
|
| `:turn_interrupted` | `task_id`, `replaced_by` | the turn was abandoned mid-run, and which turn replaced it |
|
|
39
|
+
| `:turn_stuck` | `task_id`, `agent`, `reason`, `message` | the agent declared it could not proceed (`signal_stuck`, WS5) — the deterministic signal a consumer escalates on |
|
|
39
40
|
| `:channel_delivered` | `channel`, `outbox_id`, `status`, `attempts`, `error` | the answer reached the platform (or did not) — the turn completing says nothing about that |
|
|
41
|
+
| `:delivery_failed` | `channel`, `outbox_id`, `status`, `attempts`, `error` | a delivery exhausted its bounded retries — the alert face of the row above (WS6) |
|
|
42
|
+
| `:budget_warning` | `agent`, `tenant`, `window`, `spent`, `cap` | a calendar budget crossed its threshold (`alert_at` or a soft cap) — once per window (WS2) |
|
|
43
|
+
| `:breaker_open` | `agent`, `ref`, `tenant` | the reliability circuit breaker tripped for a `(tenant, provider/model)` — further turns fail fast until the cooldown (WS3/WS6) |
|
|
44
|
+
| `:provider_failure` | `agent`, `ref`, `error`, `kind` | one attempt against `ref` failed and spent a retry — emitted with or without a circuit breaker (WS3) |
|
|
45
|
+
| `:provider_fallback` | `agent`, `from`, `to`, `error`, `kind` | the turn ROTATED mid-flight to the next node of the fallback chain, and the error that caused it (WS3) |
|
|
46
|
+
| `:ttft` | `task_id`, `session_id`, `ttft_ms` | the provider's time-to-first-token on the streaming envelope — only under `INSIKA_TURN_TIMING`, once per turn (WS6) |
|
|
47
|
+
| `:distillation_completed` | `session_ref`, `agent`, `proposals`, `dropped{}`, `deduped`, `cost{}` | a session's traffic was distilled into N proposals (RFC-0034) — counts and ids only, never a fact value |
|
|
48
|
+
| `:proposal_approved` | `proposal_id`, `status`, `operator` | a human approved a distilled fact on the Facts page — written to memory via CAS |
|
|
49
|
+
| `:proposal_rejected` | `proposal_id`, `status`, `operator` | a human rejected it (the reason is on the proposal record, not in the event) |
|
|
50
|
+
| `:proposal_dismissed` | `proposal_id`, `status`, `operator` | a human dismissed it — the tuple is latched, never proposed again |
|
|
51
|
+
| `:proposal_stale` | `proposal_id`, `status`, `operator` | the CAS lost: the fact moved after distillation — re-presented with both values on the wiki, never silently overwritten |
|
|
52
|
+
| `:harvest_mined` | `agent`, `run_id`, `candidates`, `rejected{}`, `cost{}` | a mining pass finished (RFC-0035) — counts and rule ids only, never a skill body |
|
|
53
|
+
| `:harvest_gated` | `run_id`, `candidate_id`, `agent`, `eval_passed`, `conversion_passed`, `reason` | the double gate's verdict on one candidate — ids and verdicts only |
|
|
54
|
+
| `:skill_promoted` | `agent`, `skill`, `candidate_id`, `snapshot_ref`, `promotion_ref`, `approver` | a human approved a mined skill — it is live for the store, with the snapshot for rollback |
|
|
55
|
+
| `:skill_rolled_back` | `snapshot_ref`, `skill`, `agent`, `operator` | the snapshot was restored — the promotion row carries the `rolled_back_at` stamp |
|
|
56
|
+
|
|
57
|
+
`delivery_failed` and `breaker_open` are the two the operator config is pointed at
|
|
58
|
+
(`alerts.webhook` on the profile): each only fires when something durable did
|
|
59
|
+
not land. `:ttft` is additive debug, absent unless `INSIKA_TURN_TIMING` is set.
|
|
40
60
|
|
|
41
61
|
`:channel_delivered` is the one worth alerting on: a turn can be `:task_completed`
|
|
42
62
|
and correct while the customer got nothing, because delivery is a separate,
|
|
@@ -181,13 +201,13 @@ million tokens**:
|
|
|
181
201
|
|
|
182
202
|
```bash
|
|
183
203
|
INSIKA_MODEL_PRICING='{
|
|
184
|
-
"deepseek-
|
|
204
|
+
"deepseek-v4-flash": {"input": 0.27, "output": 1.10, "cached_input": 0.07},
|
|
185
205
|
"claude-sonnet-4-5": {"input": 3.00, "output": 15.00, "cached_input": 0.30, "cache_write": 3.75}
|
|
186
206
|
}'
|
|
187
207
|
```
|
|
188
208
|
|
|
189
209
|
- A key matches the model id the provider reports, **with or without** the
|
|
190
|
-
`provider/` prefix — `deepseek/deepseek-
|
|
210
|
+
`provider/` prefix — `deepseek/deepseek-v4-flash` and `deepseek-v4-flash` both hit the
|
|
191
211
|
same entry.
|
|
192
212
|
- `input` / `output` are required (one of them is enough for the entry to load).
|
|
193
213
|
- `cached_input`, when given, bills cache **reads** at that rate and subtracts them
|
|
@@ -357,6 +377,49 @@ consumer that fell far behind would have its subscription closed (telemetry stop
|
|
|
357
377
|
the turn does not). Span and instrument operations are cheap, so there's ample
|
|
358
378
|
headroom.
|
|
359
379
|
|
|
380
|
+
## Process vitals — `GET /v1/vitals`
|
|
381
|
+
|
|
382
|
+
OTel carries turn/tool telemetry; it says nothing about the **process**. For the
|
|
383
|
+
questions a soak (or any operator) asks — *which process is this, how long has it
|
|
384
|
+
been up, how much memory does it hold, and what is the Ruby heap doing?* — there
|
|
385
|
+
is one read-only route:
|
|
386
|
+
|
|
387
|
+
```bash
|
|
388
|
+
curl -H "Authorization: Bearer $TOKEN" https://<target>/v1/vitals
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
```jsonc
|
|
392
|
+
{
|
|
393
|
+
"boot_id": "20260820T09-…", // one per container start; a change = a restart
|
|
394
|
+
"pid": 42,
|
|
395
|
+
"started_at": "2026-08-20T09:00:00Z",
|
|
396
|
+
"uptime_s": 259200,
|
|
397
|
+
"version": "0.2.0",
|
|
398
|
+
"ruby": "ruby 3.4.1 …",
|
|
399
|
+
"yjit": true,
|
|
400
|
+
"rss_bytes": 536870912, // nil when unreadable — never a guess
|
|
401
|
+
"gc": { "heap_live_slots": …, "major_gc_count": …, "malloc_increase_bytes": … },
|
|
402
|
+
"threads": 8,
|
|
403
|
+
"in_flight": 1, // the executor's in-flight turns
|
|
404
|
+
"db_bytes": { "db": …, "wal": …, "shm": … },
|
|
405
|
+
"at": "2026-08-20T09:00:00Z"
|
|
406
|
+
}
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
The two fields that make it a **restart detector**: `boot_id` (one per container
|
|
410
|
+
start, exported by the entrypoint and shared by every worker) and `pid`. A
|
|
411
|
+
`boot_id` change is a container restart; a `pid` change under the same `boot_id`
|
|
412
|
+
is a worker respawn — the event a platform metrics API cannot see.
|
|
413
|
+
|
|
414
|
+
- **Operator-only.** The route is not in the public allowlist (no bearer →
|
|
415
|
+
unauthorized) and not on the tenant surface, so only an operator reads
|
|
416
|
+
process internals. `/up` stays the public health probe and carries no
|
|
417
|
+
process data.
|
|
418
|
+
- **Reads no store.** Pure OS/VM readings — safe to poll at any rate, and it
|
|
419
|
+
cannot contend with turns.
|
|
420
|
+
- **The soak's sampler.** [Soak](SOAK.md) polls it hourly; a `nil` RSS reads as
|
|
421
|
+
missing coverage, never as zero.
|
|
422
|
+
|
|
360
423
|
---
|
|
361
424
|
|
|
362
425
|
*Packaging note.* Today the bridge lives in the Insika repo as an opt-in core
|
data/docs/REFINEMENT.md
CHANGED
|
@@ -32,11 +32,11 @@ it reaches anyone. Every part of that is below, including what it cannot catch.
|
|
|
32
32
|
From the CLI, against the same database the engine uses:
|
|
33
33
|
|
|
34
34
|
```bash
|
|
35
|
-
INSIKA_DB=insika.db bin/insika refine --agent
|
|
35
|
+
INSIKA_DB=insika.db bin/insika refine --agent demo
|
|
36
36
|
```
|
|
37
37
|
|
|
38
38
|
```text
|
|
39
|
-
|
|
39
|
+
demo — completed (last 200 session(s))
|
|
40
40
|
tool_error ×24 shipping_quote failed: cep is required
|
|
41
41
|
sessions: 9f2c1a04…, 4b7e5590…, c1d0aa31…
|
|
42
42
|
repetition ×7 customer repeated themselves
|
|
@@ -149,7 +149,7 @@ it did before, so no transcript needs migrating.
|
|
|
149
149
|
|
|
150
150
|
`repetition` counts only what a customer said and `safe_reply` reads only what the
|
|
151
151
|
engine said, both from this field. A message that declares nothing falls back to the
|
|
152
|
-
old guess (an injected fragment opens with its own tag, `<
|
|
152
|
+
old guess (an injected fragment opens with its own tag, `<store_cep_required> …`,
|
|
153
153
|
which no customer types) — that heuristic now runs only on messages that made no
|
|
154
154
|
claim about themselves.
|
|
155
155
|
|
|
@@ -173,8 +173,8 @@ an agent with no configuration at all can be run. The optional block on the agen
|
|
|
173
173
|
sets the defaults:
|
|
174
174
|
|
|
175
175
|
```ruby
|
|
176
|
-
Insika.agent "
|
|
177
|
-
model "deepseek-
|
|
176
|
+
Insika.agent "demo" do
|
|
177
|
+
model "deepseek-v4-flash"
|
|
178
178
|
refine window: { last_sessions: 200 }, max_findings: 20
|
|
179
179
|
end
|
|
180
180
|
```
|
|
@@ -186,8 +186,8 @@ end
|
|
|
186
186
|
| `exclude_sessions` | none | session-id prefixes to drop |
|
|
187
187
|
| `mode` | `"report"` | `report` reads and writes nothing. `propose` allows a gated, human-approved edit (below). `auto_apply` lets a gate-passing edit land unattended — off by default, and read [what it costs you](#applying-without-a-human) first. A mode the engine does not know is refused, never silently downgraded |
|
|
188
188
|
| `files` | none | the ONLY files a proposal may edit. Empty means report-only |
|
|
189
|
-
| `proposer` | the platform `utility_model` | which model writes the candidate (`"deepseek/deepseek-
|
|
190
|
-
| `proposers` | falls back to `proposer` | a **panel**: several models, each writing its own candidate. `["deepseek/deepseek-
|
|
189
|
+
| `proposer` | the platform `utility_model` | which model writes the candidate (`"deepseek/deepseek-v4-flash"` or a bare model name). Neither set means no proposal — the engine never picks a model to spend your budget on |
|
|
190
|
+
| `proposers` | falls back to `proposer` | a **panel**: several models, each writing its own candidate. `["deepseek/deepseek-v4-flash", {model: "gpt-5-mini", provider: "openai"}]` — either syntax |
|
|
191
191
|
| `budget.tokens` | unlimited | what one run may spend across every proposal and every gate replay |
|
|
192
192
|
| `max_edits` | 3 | edits a single proposal may carry |
|
|
193
193
|
| `auto_apply_max_edits` | 1 | edits an **unattended** apply may carry. A bigger diff waits for a person |
|
|
@@ -306,7 +306,7 @@ first question worth asking is whether the finding it addresses is behaviour at
|
|
|
306
306
|
|
|
307
307
|
```ruby
|
|
308
308
|
refine mode: "propose", files: %w[TOOLS.md],
|
|
309
|
-
proposers: ["deepseek/deepseek-
|
|
309
|
+
proposers: ["deepseek/deepseek-v4-flash", "gpt-5-mini"],
|
|
310
310
|
budget: { tokens: 200_000 }
|
|
311
311
|
```
|
|
312
312
|
|
|
@@ -440,7 +440,7 @@ not reasoned about.
|
|
|
440
440
|
rest is detail. A regression is "a case that was passing now fails" — so an edit
|
|
441
441
|
that breaks something no case covers passes cleanly. Two shallow cases wave almost
|
|
442
442
|
anything through. If you want the loop to protect a behaviour, there has to be a
|
|
443
|
-
case for that behaviour; that is the price [
|
|
443
|
+
case for that behaviour; that is the price [the gate](#what-the-gate-needs) is charging,
|
|
444
444
|
and it is charged in curation work, not in configuration.
|
|
445
445
|
|
|
446
446
|
Three things it will **not** catch, and two of them are the engine working correctly:
|
data/docs/RELEASING.md
CHANGED
|
@@ -8,7 +8,7 @@ permalink: /releasing/
|
|
|
8
8
|
# Releasing
|
|
9
9
|
|
|
10
10
|
How an `insika` gem release is cut, and how the install is **proven** before the
|
|
11
|
-
push. The rule that matters
|
|
11
|
+
push. The rule that matters: a green suite is not a green gem —
|
|
12
12
|
the suite resolves everything by path, so the entire class of packaging failure
|
|
13
13
|
is invisible to it. Do not publish on rspec alone.
|
|
14
14
|
|
|
@@ -18,7 +18,7 @@ is invisible to it. Do not publish on rspec alone.
|
|
|
18
18
|
2. `lib/insika/version.rb` carries the version being published.
|
|
19
19
|
3. Every new `lib/` file is **tracked in git**. The gemspec's `files` come from
|
|
20
20
|
`git ls-files`: an untracked file builds without a warning and the installed
|
|
21
|
-
gem fails at `require` — this is exactly the failure
|
|
21
|
+
gem fails at `require` — this is exactly the failure this proof exists to catch.
|
|
22
22
|
|
|
23
23
|
## Cut the gem
|
|
24
24
|
|
|
@@ -26,7 +26,7 @@ is invisible to it. Do not publish on rspec alone.
|
|
|
26
26
|
gem build insika.gemspec # -> insika-<version>.gem
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
## Prove the install
|
|
29
|
+
## Prove the install — from OUTSIDE the repo
|
|
30
30
|
|
|
31
31
|
Install into a clean gem home and run the four shapes from a directory that is
|
|
32
32
|
not the checkout, with the repo's `lib/` nowhere on the load path:
|
|
@@ -39,7 +39,7 @@ cd "$T"
|
|
|
39
39
|
# 1. reply in-process
|
|
40
40
|
GEM_HOME="$T/gemhome" GEM_PATH="$T/gemhome" ruby -e '
|
|
41
41
|
require "insika"
|
|
42
|
-
agent = Insika.agent("assistant") { model "deepseek-
|
|
42
|
+
agent = Insika.agent("assistant") { model "deepseek-v4-flash"; provider :deepseek }
|
|
43
43
|
puts agent.reply("hi")' # needs DEEPSEEK_API_KEY
|
|
44
44
|
|
|
45
45
|
# 2. serve — /studio login 200, /v1/responses streams, /start.md 200
|
|
@@ -50,7 +50,7 @@ GEM_HOME="$T/gemhome" GEM_PATH="$T/gemhome" ruby -e '
|
|
|
50
50
|
Shapes 2–4 are the ones that fail when a file is missing from the gem (the
|
|
51
51
|
Studio's `views/`, `assets/dist/`, the onboarding docs); run all four.
|
|
52
52
|
|
|
53
|
-
Then the load guard, from the **installed** gem
|
|
53
|
+
Then the load guard, from the **installed** gem — no test double:
|
|
54
54
|
|
|
55
55
|
```bash
|
|
56
56
|
GEM_HOME="$T/gemhome" GEM_PATH="$T/gemhome" ruby -e '
|
|
@@ -59,6 +59,33 @@ GEM_HOME="$T/gemhome" GEM_PATH="$T/gemhome" ruby -e '
|
|
|
59
59
|
puts "clean"'
|
|
60
60
|
```
|
|
61
61
|
|
|
62
|
+
### The 1.0 release gate — the RFC-0036 install proof (E1)
|
|
63
|
+
|
|
64
|
+
For the 1.0 release the proof above is scripted and its **installed-bytes**
|
|
65
|
+
half is asserted by the domain-boundary suite (C1) on the artifact, not the
|
|
66
|
+
repo. Run the runbook, with a key (the smoke turn is one `reply` through the
|
|
67
|
+
installed gem):
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
DEEPSEEK_API_KEY=sk-... scripts/install_proof/install_proof.sh # prints PASS
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The script builds, installs into a FRESH `GEM_HOME`, asserts `gem contents
|
|
74
|
+
insika` carries no `deploy/ packs/ examples/ plugins/ evals/ scripts/ spec/`
|
|
75
|
+
path and no demo-persona-name string, and answers one turn from an app dir that follows
|
|
76
|
+
only the public docs. Archive the PASS output with the release notes — it is
|
|
77
|
+
the 1.0 exit criterion "install proof by the docs alone".
|
|
78
|
+
|
|
79
|
+
The same gate writes the freeze date: a breaking `/v1` change needs a new
|
|
80
|
+
`Insika-Version` entry (server/app.rb), a compatibility branch (RFC-0016 A5)
|
|
81
|
+
and a rewritten `**Frozen as of:**` line in `docs/API.md` — the version-gate
|
|
82
|
+
spec pins the two together, and the 1.0 release writes the date at release
|
|
83
|
+
time.
|
|
84
|
+
|
|
85
|
+
Catalog submission checklist (RFC §6): verify the best-of-Agent-Harnesses
|
|
86
|
+
catalog size at submission time (161 vs 154 — the counts diverge across the
|
|
87
|
+
catalog's own pages) and cite the conformance suite as the `durable` evidence.
|
|
88
|
+
|
|
62
89
|
## Publish
|
|
63
90
|
|
|
64
91
|
```bash
|
|
@@ -66,5 +93,5 @@ gem push insika-<version>.gem
|
|
|
66
93
|
```
|
|
67
94
|
|
|
68
95
|
Publishing is irreversible in practice — a yanked 0.1.0 is a bad first
|
|
69
|
-
impression. The version number is cheap; the name is not.
|
|
70
|
-
nothing else.
|
|
96
|
+
impression. The version number is cheap; the name is not. The install proof before
|
|
97
|
+
push, and nothing else.
|
data/docs/RUNNING-LOCAL.md
CHANGED
|
@@ -8,7 +8,7 @@ permalink: /running-local/
|
|
|
8
8
|
# Running the Insika locally
|
|
9
9
|
|
|
10
10
|
Boots the engine single-process, serving `/studio` and `/v1/*` against a demo
|
|
11
|
-
agent (the `
|
|
11
|
+
agent (the `demo` agent on DeepSeek). Every message runs the **same**
|
|
12
12
|
`send_message` the API runs — real tools, skills, and memory.
|
|
13
13
|
|
|
14
14
|
## Boot
|
|
@@ -26,7 +26,7 @@ Open `http://localhost:9292`:
|
|
|
26
26
|
| URL | What |
|
|
27
27
|
|-----|------|
|
|
28
28
|
| `/studio` | management UI (log in with the token; default `local-demo`) |
|
|
29
|
-
| `/studio/chats` | chat with the demo agent (`agent:
|
|
29
|
+
| `/studio/chats` | chat with the demo agent (`agent: demo`, `session_id: web`, multi-turn ready) |
|
|
30
30
|
| `/studio/tasks` | tasks / approvals console |
|
|
31
31
|
| `/v1/responses` | OpenAI-Responses ingress (Bearer) — the drop-in API contract |
|
|
32
32
|
| `/v1/agents` | provisioning by definition/pack (Bearer) — `POST` imports, `DELETE /:id` removes |
|
|
@@ -63,7 +63,7 @@ whole surface answers `503`, never open by omission.
|
|
|
63
63
|
| `BIND` | `http://localhost:9292` | host:port |
|
|
64
64
|
| `ADMIN_TOKEN` | `local-demo` | token for `/studio` |
|
|
65
65
|
| `OPENCLAW_GATEWAY_TOKEN` | falls back to `ADMIN_TOKEN` | Bearer for the whole `/v1` + `/a2a` surface |
|
|
66
|
-
| `DEEPSEEK_MODEL` | `deepseek-
|
|
66
|
+
| `DEEPSEEK_MODEL` | `deepseek-v4-flash` | model |
|
|
67
67
|
|
|
68
68
|
With persistence:
|
|
69
69
|
|
|
@@ -82,7 +82,7 @@ the `model`:
|
|
|
82
82
|
curl -N http://localhost:9292/v1/responses \
|
|
83
83
|
-H "Authorization: Bearer local-demo" \
|
|
84
84
|
-H "Content-Type: application/json" \
|
|
85
|
-
-d '{ "model": "
|
|
85
|
+
-d '{ "model": "demo", "user": "web", "stream": true, "input": "hello" }'
|
|
86
86
|
```
|
|
87
87
|
|
|
88
88
|
`user` is the session id (any stable id for a multi-turn conversation).
|