insika 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +295 -0
- data/LICENSE +21 -0
- data/README.md +136 -2
- data/bin/insika +351 -0
- data/docs/AGENTS.md +494 -0
- data/docs/ARCHITECTURE.md +333 -0
- data/docs/BENCHMARK.md +114 -0
- data/docs/CHANNELS.md +453 -0
- data/docs/CONTEXT.md +100 -0
- data/docs/DEPLOY.md +334 -0
- data/docs/EMBEDDING.md +194 -0
- data/docs/EVALS.md +273 -0
- data/docs/LOADTEST.md +231 -0
- data/docs/OBSERVABILITY.md +365 -0
- data/docs/PLUGINS.md +211 -0
- data/docs/REFINEMENT.md +477 -0
- data/docs/RELEASING.md +70 -0
- data/docs/RUNNING-LOCAL.md +153 -0
- data/docs/SANDBOX.md +114 -0
- data/docs/SECURITY.md +362 -0
- data/docs/SKILLS.md +98 -0
- data/docs/TOOLS.md +302 -0
- data/docs/WHY.md +137 -0
- data/docs/WORKFLOWS.md +225 -0
- data/docs/build.md +14 -0
- data/docs/index.md +68 -0
- data/docs/onboarding/start.md +126 -0
- data/docs/operate.md +12 -0
- data/docs/ship.md +10 -0
- data/docs/understand.md +10 -0
- data/lib/insika/agent_file_store.rb +125 -0
- data/lib/insika/agent_profile.rb +188 -0
- data/lib/insika/allowlist.rb +28 -0
- data/lib/insika/baseline_store.rb +74 -0
- data/lib/insika/capability/resolved_tool.rb +34 -0
- data/lib/insika/capability_registry.rb +112 -0
- data/lib/insika/channel_delivery.rb +150 -0
- data/lib/insika/channel_registry.rb +30 -0
- data/lib/insika/channels/relay.rb +178 -0
- data/lib/insika/channels/web/widget.js +283 -0
- data/lib/insika/channels/web.rb +211 -0
- data/lib/insika/chat_builder.rb +254 -0
- data/lib/insika/checkpoint.rb +13 -0
- data/lib/insika/checkpoint_store.rb +153 -0
- data/lib/insika/coercion.rb +50 -0
- data/lib/insika/command.rb +32 -0
- data/lib/insika/command_bus.rb +39 -0
- data/lib/insika/commands/agent_payload.rb +41 -0
- data/lib/insika/commands/approve_action.rb +46 -0
- data/lib/insika/commands/cancel_task.rb +33 -0
- data/lib/insika/commands/create_agent.rb +54 -0
- data/lib/insika/commands/create_session.rb +67 -0
- data/lib/insika/commands/delete_agent.rb +33 -0
- data/lib/insika/commands/delete_agent_file.rb +50 -0
- data/lib/insika/commands/delete_data_tool.rb +33 -0
- data/lib/insika/commands/delete_llm_provider.rb +36 -0
- data/lib/insika/commands/delete_mcp.rb +30 -0
- data/lib/insika/commands/delete_system_file.rb +29 -0
- data/lib/insika/commands/gate_refinement.rb +245 -0
- data/lib/insika/commands/import_mcp_tools.rb +48 -0
- data/lib/insika/commands/import_tools.rb +81 -0
- data/lib/insika/commands/memory_add_note.rb +32 -0
- data/lib/insika/commands/memory_forget_fact.rb +32 -0
- data/lib/insika/commands/memory_put_fact.rb +35 -0
- data/lib/insika/commands/pause_task.rb +29 -0
- data/lib/insika/commands/resolve_refinement.rb +126 -0
- data/lib/insika/commands/restore_agent_file.rb +36 -0
- data/lib/insika/commands/restore_data_tool.rb +34 -0
- data/lib/insika/commands/restore_system_file.rb +31 -0
- data/lib/insika/commands/resume_task.rb +85 -0
- data/lib/insika/commands/run_refinement.rb +133 -0
- data/lib/insika/commands/send_message.rb +150 -0
- data/lib/insika/commands/set_agent_tools.rb +39 -0
- data/lib/insika/commands/set_skill_agents.rb +71 -0
- data/lib/insika/commands/trigger_workflow.rb +80 -0
- data/lib/insika/commands/update_agent.rb +49 -0
- data/lib/insika/commands/update_settings.rb +33 -0
- data/lib/insika/commands/upsert_llm_provider.rb +34 -0
- data/lib/insika/commands/upsert_mcp.rb +32 -0
- data/lib/insika/commands/write_agent_file.rb +57 -0
- data/lib/insika/commands/write_data_tool.rb +43 -0
- data/lib/insika/commands/write_golden.rb +58 -0
- data/lib/insika/commands/write_skill.rb +50 -0
- data/lib/insika/commands/write_system_file.rb +31 -0
- data/lib/insika/config_store.rb +85 -0
- data/lib/insika/context/builder.rb +166 -0
- data/lib/insika/context/catalog_provider.rb +23 -0
- data/lib/insika/context/fragment.rb +19 -0
- data/lib/insika/context/priority.rb +29 -0
- data/lib/insika/context/provider.rb +19 -0
- data/lib/insika/context/providers/memory.rb +60 -0
- data/lib/insika/context/providers/prompt.rb +105 -0
- data/lib/insika/context/providers/request.rb +32 -0
- data/lib/insika/context/providers/session.rb +108 -0
- data/lib/insika/context/providers/skill.rb +20 -0
- data/lib/insika/context/providers/tool_search.rb +20 -0
- data/lib/insika/delegation_store.rb +153 -0
- data/lib/insika/doctor.rb +294 -0
- data/lib/insika/dsl/definition.rb +55 -0
- data/lib/insika/dsl/runtime.rb +379 -0
- data/lib/insika/dsl/server_boot.rb +97 -0
- data/lib/insika/dsl/system.rb +93 -0
- data/lib/insika/dsl/workflow_adapter.rb +59 -0
- data/lib/insika/dsl.rb +307 -0
- data/lib/insika/edge_limiter.rb +130 -0
- data/lib/insika/egress_guard.rb +75 -0
- data/lib/insika/env_schema.rb +246 -0
- data/lib/insika/errors.rb +145 -0
- data/lib/insika/evals/assertions.rb +247 -0
- data/lib/insika/evals/baseline.rb +69 -0
- data/lib/insika/evals/golden.rb +172 -0
- data/lib/insika/evals/judge.rb +225 -0
- data/lib/insika/evals/pairwise.rb +178 -0
- data/lib/insika/evals/report.rb +115 -0
- data/lib/insika/evals/runner.rb +141 -0
- data/lib/insika/evals/transport.rb +178 -0
- data/lib/insika/event.rb +18 -0
- data/lib/insika/event_stream.rb +114 -0
- data/lib/insika/executor.rb +1680 -0
- data/lib/insika/frontmatter.rb +42 -0
- data/lib/insika/golden_store.rb +145 -0
- data/lib/insika/hooks.rb +48 -0
- data/lib/insika/http_client.rb +63 -0
- data/lib/insika/inbound_log.rb +84 -0
- data/lib/insika/llm_configurator.rb +99 -0
- data/lib/insika/llm_provider_store.rb +83 -0
- data/lib/insika/mcp_http_client.rb +67 -0
- data/lib/insika/mcp_store.rb +115 -0
- data/lib/insika/mcp_tool_ingestor.rb +143 -0
- data/lib/insika/memory_store.rb +93 -0
- data/lib/insika/message_origin.rb +76 -0
- data/lib/insika/middleware.rb +36 -0
- data/lib/insika/model_policy.rb +52 -0
- data/lib/insika/model_resolver.rb +176 -0
- data/lib/insika/model_selection.rb +114 -0
- data/lib/insika/onboarding.rb +208 -0
- data/lib/insika/outbox_store.rb +166 -0
- data/lib/insika/overlay_tool_registry.rb +103 -0
- data/lib/insika/pack.rb +102 -0
- data/lib/insika/pack_importer.rb +121 -0
- data/lib/insika/pending_action_store.rb +120 -0
- data/lib/insika/plugin/loader.rb +356 -0
- data/lib/insika/plugin.rb +35 -0
- data/lib/insika/policy/engine.rb +83 -0
- data/lib/insika/policy/policy.rb +120 -0
- data/lib/insika/policy_registry.rb +23 -0
- data/lib/insika/profile_source.rb +137 -0
- data/lib/insika/prompt_catalog.rb +61 -0
- data/lib/insika/queue_policy.rb +167 -0
- data/lib/insika/recovery.rb +127 -0
- data/lib/insika/refinement/candidate.rb +159 -0
- data/lib/insika/refinement/evidence_collector.rb +371 -0
- data/lib/insika/refinement/gate.rb +234 -0
- data/lib/insika/refinement/panel.rb +222 -0
- data/lib/insika/refinement/proposer.rb +262 -0
- data/lib/insika/refinement_store.rb +295 -0
- data/lib/insika/registry.rb +59 -0
- data/lib/insika/safety/config.rb +109 -0
- data/lib/insika/safety/detectors.rb +176 -0
- data/lib/insika/safety/factory.rb +102 -0
- data/lib/insika/safety/input_guardrail.rb +87 -0
- data/lib/insika/safety/moderator.rb +86 -0
- data/lib/insika/safety/output_filter.rb +79 -0
- data/lib/insika/safety/output_validator.rb +101 -0
- data/lib/insika/safety/safe_responses.rb +47 -0
- data/lib/insika/sandbox/boundary.rb +93 -0
- data/lib/insika/sandbox/docker.rb +74 -0
- data/lib/insika/sandbox/local.rb +33 -0
- data/lib/insika/sandbox/runner.rb +80 -0
- data/lib/insika/sandbox.rb +85 -0
- data/lib/insika/schema_guard.rb +147 -0
- data/lib/insika/secret_masking.rb +34 -0
- data/lib/insika/server/a2a/agent_card.rb +27 -0
- data/lib/insika/server/a2a/app.rb +112 -0
- data/lib/insika/server/a2a/client.rb +101 -0
- data/lib/insika/server/a2a/errors.rb +32 -0
- data/lib/insika/server/a2a/http.rb +42 -0
- data/lib/insika/server/a2a/message.rb +27 -0
- data/lib/insika/server/a2a/protocol.rb +45 -0
- data/lib/insika/server/a2a/remotes.rb +25 -0
- data/lib/insika/server/a2a/task_projection.rb +40 -0
- data/lib/insika/server/admin_auth.rb +29 -0
- data/lib/insika/server/app.rb +850 -0
- data/lib/insika/server/boot.rb +119 -0
- data/lib/insika/server/rack_app.rb +110 -0
- data/lib/insika/server/responses.rb +155 -0
- data/lib/insika/server/sse_body.rb +96 -0
- data/lib/insika/session_actor.rb +162 -0
- data/lib/insika/session_store.rb +143 -0
- data/lib/insika/settings_store.rb +154 -0
- data/lib/insika/shutdown.rb +125 -0
- data/lib/insika/skill_catalog.rb +113 -0
- data/lib/insika/skill_store.rb +79 -0
- data/lib/insika/steer_injector.rb +110 -0
- data/lib/insika/store.rb +52 -0
- data/lib/insika/stores/memory.rb +123 -0
- data/lib/insika/stores/sqlite.rb +183 -0
- data/lib/insika/studio/app.rb +1571 -0
- data/lib/insika/studio/assets/dist/application.css +1 -0
- data/lib/insika/studio/assets/dist/application.js +69 -0
- data/lib/insika/studio/forms.rb +340 -0
- data/lib/insika/studio/nav_icons.rb +31 -0
- data/lib/insika/studio/views/_message.erb +44 -0
- data/lib/insika/studio/views/agent_detail.erb +285 -0
- data/lib/insika/studio/views/agents.erb +63 -0
- data/lib/insika/studio/views/approvals.erb +41 -0
- data/lib/insika/studio/views/chats.erb +34 -0
- data/lib/insika/studio/views/evals.erb +83 -0
- data/lib/insika/studio/views/home.erb +72 -0
- data/lib/insika/studio/views/layout.erb +94 -0
- data/lib/insika/studio/views/login.erb +17 -0
- data/lib/insika/studio/views/mcp.erb +91 -0
- data/lib/insika/studio/views/not_found.erb +5 -0
- data/lib/insika/studio/views/playground.erb +47 -0
- data/lib/insika/studio/views/refinement.erb +234 -0
- data/lib/insika/studio/views/session.erb +62 -0
- data/lib/insika/studio/views/settings.erb +173 -0
- data/lib/insika/studio/views/skills.erb +86 -0
- data/lib/insika/studio/views/system_files.erb +65 -0
- data/lib/insika/studio/views/task.erb +105 -0
- data/lib/insika/studio/views/tasks.erb +33 -0
- data/lib/insika/studio/views/tool_edit.erb +107 -0
- data/lib/insika/studio/views/tools.erb +89 -0
- data/lib/insika/subagent_graph.rb +96 -0
- data/lib/insika/system_file_store.rb +96 -0
- data/lib/insika/task_actor.rb +128 -0
- data/lib/insika/task_store.rb +250 -0
- data/lib/insika/telemetry/pricing.rb +104 -0
- data/lib/insika/telemetry/recorder.rb +228 -0
- data/lib/insika/telemetry.rb +127 -0
- data/lib/insika/testing/store_contract.rb +270 -0
- data/lib/insika/token_estimator.rb +16 -0
- data/lib/insika/tool_assembly.rb +140 -0
- data/lib/insika/tool_catalog.rb +89 -0
- data/lib/insika/tool_definition.rb +518 -0
- data/lib/insika/tool_envelope.rb +140 -0
- data/lib/insika/tool_manifest.rb +218 -0
- data/lib/insika/tool_registry.rb +21 -0
- data/lib/insika/tool_store.rb +135 -0
- data/lib/insika/tool_trace_store.rb +92 -0
- data/lib/insika/tools/a2a_remote.rb +48 -0
- data/lib/insika/tools/agent_enum.rb +68 -0
- data/lib/insika/tools/concurrency.rb +54 -0
- data/lib/insika/tools/data_defined_tool.rb +220 -0
- data/lib/insika/tools/load_skill.rb +41 -0
- data/lib/insika/tools/remember.rb +53 -0
- data/lib/insika/tools/subagent.rb +75 -0
- data/lib/insika/tools/subagents.rb +77 -0
- data/lib/insika/tools/tool_search.rb +94 -0
- data/lib/insika/turn_output.rb +139 -0
- data/lib/insika/turn_state.rb +158 -0
- data/lib/insika/turn_timing.rb +56 -0
- data/lib/insika/usage_ledger.rb +47 -0
- data/lib/insika/version.rb +3 -1
- data/lib/insika/wiring/graph.rb +198 -0
- data/lib/insika/workflow.rb +185 -0
- data/lib/insika/workflow_registry.rb +33 -0
- data/lib/insika.rb +203 -4
- metadata +395 -8
data/docs/SKILLS.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Skills
|
|
3
|
+
parent: Build an agent
|
|
4
|
+
nav_order: 3
|
|
5
|
+
permalink: /skills/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Skills
|
|
9
|
+
|
|
10
|
+
A **skill** is a named playbook an agent loads **on demand**. It is a directory
|
|
11
|
+
with a `SKILL.md` file: YAML frontmatter (`name` + `description`) followed by a
|
|
12
|
+
Markdown body. The agent always sees the *name and description* of each skill it
|
|
13
|
+
is allowed; it pulls the full *body* into context only when a turn actually calls
|
|
14
|
+
for it. That is **progressive loading** — an agent can "know" twenty skills exist
|
|
15
|
+
while paying for the text of only the ones it opens.
|
|
16
|
+
|
|
17
|
+
See [`examples/skills/`](https://github.com/guizaols/insika/tree/main/examples/skills/) for a runnable one.
|
|
18
|
+
|
|
19
|
+
## Format
|
|
20
|
+
|
|
21
|
+
```markdown
|
|
22
|
+
---
|
|
23
|
+
name: refunds # must equal the directory name
|
|
24
|
+
description: When and how to process a refund # the Level-1 trigger text
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
<the full playbook body — loaded only on demand>
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Frontmatter is parsed tolerantly. A skill's canonical name is its directory name,
|
|
31
|
+
which the `name:` field must match.
|
|
32
|
+
|
|
33
|
+
## Progressive loading: two levels
|
|
34
|
+
|
|
35
|
+
- **Level 1 — metadata only.** A context provider injects an `<available_skills>`
|
|
36
|
+
list (the name + one-line description of every allowed skill) into the system
|
|
37
|
+
prompt, telling the model to load a skill before acting on it. Cheap, and always
|
|
38
|
+
present for allowed skills.
|
|
39
|
+
- **Level 2 — the full body.** A built-in `load_skill` tool returns the skill body
|
|
40
|
+
on demand. It enforces the agent's skill allowlist and is wired **automatically**
|
|
41
|
+
whenever the agent has any allowed skills — you do not add it to `tools_allow`.
|
|
42
|
+
|
|
43
|
+
The Level-1 list is budgeted like any other context fragment
|
|
44
|
+
(see [Context](CONTEXT.md)); the Level-2 body only costs tokens on the turns that
|
|
45
|
+
open it.
|
|
46
|
+
|
|
47
|
+
## Where skills live: the store, over a disk seed
|
|
48
|
+
|
|
49
|
+
- Skills live as **rows in SQLite** — one row per skill, holding the entire
|
|
50
|
+
`SKILL.md`, versioned (recent revisions are retained).
|
|
51
|
+
- On-disk `SKILL.md` files in configured roots are loaded as a **seed**, then the
|
|
52
|
+
store is **overlaid on top — the store wins**. A reload swaps the index
|
|
53
|
+
atomically, so edits take effect **without a restart**.
|
|
54
|
+
|
|
55
|
+
> ⚠️ **Committing a `.md` file to the repo does not make a skill show up on a
|
|
56
|
+
> running deployment.** The on-disk file is only a seed for a *fresh* box; a live
|
|
57
|
+
> box serves the store, and a deploy does not rewrite the database. Editing is a
|
|
58
|
+
> runtime operation (Studio / API / DSL), not a commit. See
|
|
59
|
+
> [Context](CONTEXT.md#the-volume).
|
|
60
|
+
|
|
61
|
+
## Making a new skill "show up"
|
|
62
|
+
|
|
63
|
+
For an agent to actually use a skill, **both** conditions must hold:
|
|
64
|
+
|
|
65
|
+
1. The skill exists as a row in the store (a `SKILL.md` written into it).
|
|
66
|
+
2. The skill is in that agent's `skills` allowlist
|
|
67
|
+
(`nil` = all, `[]` = none, `[names]` = those — see
|
|
68
|
+
[Agents](AGENTS.md#the-allowlist-convention)).
|
|
69
|
+
|
|
70
|
+
Miss either and the skill is invisible: not in the store → nothing to load; not
|
|
71
|
+
in the allowlist → the model never sees it in `<available_skills>`.
|
|
72
|
+
|
|
73
|
+
Two ways to satisfy both:
|
|
74
|
+
|
|
75
|
+
- **Via a definition/pack import.** The import writes each skill directory into
|
|
76
|
+
the store and sets the agent's `skills` allowlist **authoritatively** from the
|
|
77
|
+
skills present — so a re-import that drops a skill also removes it. Keep the
|
|
78
|
+
definition complete.
|
|
79
|
+
- **Directly (Studio / API / DSL).** Write the skill (upserts the row and reloads
|
|
80
|
+
the catalog atomically — live immediately), then attach it to the agent(s) by
|
|
81
|
+
adding its name to the `skills` allowlist.
|
|
82
|
+
|
|
83
|
+
## Verify it showed up
|
|
84
|
+
|
|
85
|
+
- In the Studio agent's Skills section, the skill is listed and allowed.
|
|
86
|
+
- In a turn, the skill appears in the `<available_skills>` list and the model can
|
|
87
|
+
`load_skill` it (the body loads on demand).
|
|
88
|
+
- If the model never mentions it → check the allowlist (condition 2). If
|
|
89
|
+
`load_skill` errors → the store row is missing or misnamed (condition 1); the
|
|
90
|
+
`name:` frontmatter must equal the directory name.
|
|
91
|
+
|
|
92
|
+
## See also
|
|
93
|
+
|
|
94
|
+
- [Context](CONTEXT.md) — how the skills list is budgeted into a turn.
|
|
95
|
+
- [Agents](AGENTS.md) — the skills allowlist.
|
|
96
|
+
- [Tools](TOOLS.md) — `load_skill` and deferred-tool progressive disclosure.
|
|
97
|
+
- [Plugins](PLUGINS.md) — shipping skills inside a plugin, and the two extension tiers.
|
|
98
|
+
- [`examples/skills/`](https://github.com/guizaols/insika/tree/main/examples/skills/) — progressive loading, runnable.
|
data/docs/TOOLS.md
ADDED
|
@@ -0,0 +1,302 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Tools
|
|
3
|
+
parent: Build an agent
|
|
4
|
+
nav_order: 2
|
|
5
|
+
permalink: /tools/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Tools
|
|
9
|
+
|
|
10
|
+
A **tool** is a function the model can call inside a turn. Insika has three
|
|
11
|
+
kinds, and the distinction that matters is **who can change one at runtime**:
|
|
12
|
+
|
|
13
|
+
| | **Code tool** | **Data tool** | **MCP tool** |
|
|
14
|
+
|---|---|---|---|
|
|
15
|
+
| What | a Ruby class (`< RubyLLM::Tool`) | an HTTP call described by config, no Ruby | an MCP server's tool, ingested |
|
|
16
|
+
| Lives | in the deployment image | as a row in SQLite | as data-tool rows in SQLite |
|
|
17
|
+
| Editable at runtime | no (shipped in the image) | **yes** (DSL / API / manifest / Studio) | **yes** (re-ingest) |
|
|
18
|
+
| Reach for it when | logic must run in-process (file edit, shell, subagent) | calling an external HTTP API | adopting a whole MCP toolset at once |
|
|
19
|
+
|
|
20
|
+
**MCP tools are not a separate runtime type.** An MCP ingestor discovers an MCP
|
|
21
|
+
server's tools and turns each into an HTTP **data tool** that posts a JSON-RPC
|
|
22
|
+
`tools/call`, tagged with a `group` naming the source instance. (Only
|
|
23
|
+
HTTP-transport MCP servers are ingestible; stdio is rejected.)
|
|
24
|
+
|
|
25
|
+
Code tools **win name collisions** — you cannot register a data tool whose name
|
|
26
|
+
shadows a code tool.
|
|
27
|
+
|
|
28
|
+
## Data tools: a tool is a row
|
|
29
|
+
|
|
30
|
+
A data tool is defined entirely by config — this is the operator-facing kind, and
|
|
31
|
+
the one you create and change without a rebuild. See
|
|
32
|
+
[`examples/data-tool/`](https://github.com/guizaols/insika/tree/main/examples/data-tool/) for a runnable one.
|
|
33
|
+
|
|
34
|
+
```jsonc
|
|
35
|
+
{
|
|
36
|
+
"name": "search_products", // /\A[a-z][a-z0-9_]*\z/
|
|
37
|
+
"description": "Search the catalog", // required — this is what the model reads
|
|
38
|
+
"parameters": { /* JSON Schema, safe subset */ },
|
|
39
|
+
"request": {
|
|
40
|
+
"method": "POST", // GET | HEAD | POST | PUT | PATCH | DELETE
|
|
41
|
+
"url": "https://api.example.com/search",
|
|
42
|
+
"headers": { "X-Session": "{{ctx.chat_id}}",
|
|
43
|
+
"Authorization": "Bearer {{secret.api_token}}" },
|
|
44
|
+
"query": {}, "body": "…"
|
|
45
|
+
},
|
|
46
|
+
"response": { "extract": "json_path", "path": "$.results" },
|
|
47
|
+
"secret_headers": ["Authorization"],
|
|
48
|
+
"side_effect": true, "timeout": 30, "group": "catalog", "tags": []
|
|
49
|
+
}
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
### Parameters: the schema is the contract
|
|
53
|
+
|
|
54
|
+
`parameters` is **JSON Schema**, and it reaches the provider verbatim — it is the only
|
|
55
|
+
thing telling the model what shape to send. The engine never fills a gap in it.
|
|
56
|
+
|
|
57
|
+
For simple params there is a flat sugar (what the Studio's textarea and a hand-written
|
|
58
|
+
manifest accept), one line per param:
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
cep | string | required | The ZIP code to look up
|
|
62
|
+
tags | array:string | optional | Labels to filter by
|
|
63
|
+
quantity | integer | required | How many
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Types are `string`, `number`, `integer`, `boolean`, and `array:<scalar>` for a list.
|
|
67
|
+
There is **no bare `array`**: a list without an item type is an incomplete declaration,
|
|
68
|
+
and it is rejected instead of being guessed at. A list of **objects** — the common
|
|
69
|
+
`[{query, filters}]` shape — cannot be written in the flat form at all; write the JSON
|
|
70
|
+
Schema, which is what the Studio field reads when the text starts with `{`:
|
|
71
|
+
|
|
72
|
+
```jsonc
|
|
73
|
+
{ "type": "object",
|
|
74
|
+
"properties": {
|
|
75
|
+
"query_filter_pairs": {
|
|
76
|
+
"type": "array",
|
|
77
|
+
"items": { "type": "object",
|
|
78
|
+
"properties": { "query": { "type": "string" },
|
|
79
|
+
"filters": { "type": "object", "properties": {} } },
|
|
80
|
+
"required": ["query"] } } },
|
|
81
|
+
"required": ["query_filter_pairs"] }
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
**Arguments are checked against the schema at call time.** A call the schema does not
|
|
85
|
+
allow never becomes a request: it returns an `{ error: … }` naming the path
|
|
86
|
+
(`query_filter_pairs[0]: expected an object, got a string`), which the model reads and
|
|
87
|
+
retries against. Structure is strict; a scalar may arrive in its lossless string form
|
|
88
|
+
(`"2"`, `"true"`) and is never coerced — what the model sent is what the request carries.
|
|
89
|
+
|
|
90
|
+
**Placeholders** are resolved at turn time:
|
|
91
|
+
|
|
92
|
+
- `{{param}}` — a declared top-level parameter, filled from the model's call.
|
|
93
|
+
- `{{ctx.*}}` — turn context set **server-side, never by the model**: a closed set
|
|
94
|
+
of `chat_id`, `store_id`, `agent_id`, `tenant`. This is how a tool knows *which*
|
|
95
|
+
session/agent it is acting for without trusting the model.
|
|
96
|
+
- `{{secret.*}}` — allowed **only** inside a header named in `secret_headers`.
|
|
97
|
+
A secret placeholder anywhere else is rejected (it would leak unmasked). The
|
|
98
|
+
real secret value is injected at provision time and never lives on disk.
|
|
99
|
+
|
|
100
|
+
**Validation** happens on ingestion. Common rejections:
|
|
101
|
+
|
|
102
|
+
- `url` must be `http`/`https` — anything else is a 422.
|
|
103
|
+
- `parameters` is a **safe subset** of JSON Schema
|
|
104
|
+
(`object/array/string/number/integer/boolean`); `oneOf`/`anyOf`/`allOf`/`$ref`/
|
|
105
|
+
`if`/`then`/`else` are forbidden (not every provider supports them).
|
|
106
|
+
- `side_effect` defaults from the method (GET/HEAD → false, else true) and drives
|
|
107
|
+
checkpoint/replay semantics (a completed side-effecting tool is not re-run on
|
|
108
|
+
resume — see [Architecture](ARCHITECTURE.md#durability-checkpoints-and-resume)).
|
|
109
|
+
|
|
110
|
+
### `halt_when`: when the answer is already out
|
|
111
|
+
|
|
112
|
+
Some tools do the work **and** deliver the news. A backend that subscribes a customer
|
|
113
|
+
and sends its own confirmation over the channel has already said everything there is to
|
|
114
|
+
say: if the model then writes "all set, you're subscribed!", the person gets the message
|
|
115
|
+
twice. The usual patch is to ask the model to stay quiet in the tool's instructions —
|
|
116
|
+
which works until the turn it doesn't, and the failure lands in front of a customer.
|
|
117
|
+
|
|
118
|
+
`halt_when` moves the decision from the prompt to the engine. It reads the tool's own
|
|
119
|
+
**response**, and when it matches, the turn ends right there — no further provider call:
|
|
120
|
+
|
|
121
|
+
```jsonc
|
|
122
|
+
{ "name": "subscribe_to_learning_path",
|
|
123
|
+
"request": { "method": "POST", "url": "https://app.example/subscribe" },
|
|
124
|
+
"halt_when": { "json_path": "tool_result.status", "equals": ["SUBSCRIBED"] } }
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
By **result**, not by tool. The same call that goes silent on `SUBSCRIBED` must let the
|
|
128
|
+
model explain a `SUBSCRIPTION_FAILED` ("you are already enrolled") — one tool, two
|
|
129
|
+
endings, decided by what the backend actually returned.
|
|
130
|
+
|
|
131
|
+
- `json_path` is a dotted path into the parsed response body, and `equals` a list of
|
|
132
|
+
values compared **as strings** (a status is a label; JSON types vary by backend).
|
|
133
|
+
- It reads the **body**, independently of `response.extract` — which shapes what the
|
|
134
|
+
*model* sees, not what the engine decides on.
|
|
135
|
+
- It only fires on a **2xx**. An error response that happens to carry the value is a
|
|
136
|
+
failure, and a failure must reach the model.
|
|
137
|
+
- A non-JSON body or a missing path simply does not match: a turn never ends on a guess.
|
|
138
|
+
|
|
139
|
+
A halted turn keeps whatever the model had already streamed *before* the call (usually a
|
|
140
|
+
"let me get that for you") and adds nothing after it.
|
|
141
|
+
|
|
142
|
+
#### `say`: what the customer gets when the model wrote nothing first
|
|
143
|
+
|
|
144
|
+
The model does not always introduce the call. Then the lead-in is empty, and the turn
|
|
145
|
+
used to publish **nothing** — measured on a real store, two escalation turns in a row
|
|
146
|
+
delivered silence to the customer. `say` is the answer for that turn, and only that
|
|
147
|
+
turn: when there **is** a lead-in it still wins, because two messages for one
|
|
148
|
+
escalation is what `halt_when` exists to prevent.
|
|
149
|
+
|
|
150
|
+
It cannot be inferred. `json_path` + `equals` cannot supply it either — the matched
|
|
151
|
+
value is by definition one of the `equals` tokens, so publishing it would ship
|
|
152
|
+
`SUBSCRIBED` to a person as often as it ships a sentence. So you name it, in one of two
|
|
153
|
+
shapes:
|
|
154
|
+
|
|
155
|
+
```jsonc
|
|
156
|
+
// the sentence the backend itself returned
|
|
157
|
+
"halt_when": { "json_path": "tool_result.status", "equals": ["SUBSCRIBED"],
|
|
158
|
+
"say": { "json_path": "tool_result.message" } }
|
|
159
|
+
|
|
160
|
+
// a literal the CHANNEL knows how to resolve
|
|
161
|
+
"halt_when": { "json_path": "tool_result", "equals": ["…"],
|
|
162
|
+
"say": { "text": "CALL_SUPPORT" } }
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
The literal form replaces the usual workaround: instructing the model to emit a control
|
|
166
|
+
token and parsing it downstream. The token now comes from the **tool's contract**,
|
|
167
|
+
deterministically, instead of depending on the model complying with a sentence in a
|
|
168
|
+
prompt.
|
|
169
|
+
|
|
170
|
+
- Exactly one of `text` or `json_path` — two answers to "what does the customer get" is
|
|
171
|
+
a configuration nobody can read, so both (or neither) is refused at load.
|
|
172
|
+
- A `json_path` that does not resolve to a **string** publishes nothing: a hash or a
|
|
173
|
+
number reaching a customer as the answer is never what someone meant.
|
|
174
|
+
- Omit `say` and the behaviour is unchanged — a halt with no lead-in completes empty,
|
|
175
|
+
which is what a channel consumer drops.
|
|
176
|
+
|
|
177
|
+
`say` is declared on the **tool**, because what a backend answers is a property of that
|
|
178
|
+
backend, not of whoever calls it. Every agent sharing the tool gets the same value.
|
|
179
|
+
|
|
180
|
+
> The Studio's tool editor does not render this field (nor `group`/`tags`), but a save
|
|
181
|
+
> there **preserves** it — the form carries the stored values through instead of
|
|
182
|
+
> replacing the record with only what it shows.
|
|
183
|
+
|
|
184
|
+
## Registering a tool
|
|
185
|
+
|
|
186
|
+
A tool appears in the Studio panel and enters an agent's tool-loop when it is
|
|
187
|
+
**registered** in the catalog **and** allowed by the agent's policy allowlist.
|
|
188
|
+
Four ways to write a data tool into the store — all **hot** (registry and catalog
|
|
189
|
+
reload, no restart):
|
|
190
|
+
|
|
191
|
+
1. **DSL** — `data_tool(name:, …)` in a `Insika.agent { … }` block.
|
|
192
|
+
2. **Studio** — the Tools panel editor.
|
|
193
|
+
3. **Manifest** — `POST /v1/tools/manifest`. Partial failure is isolated: one
|
|
194
|
+
malformed tool becomes an `errors[]` entry; only a structural manifest error
|
|
195
|
+
fails the whole request. The response reports `{ version, created, updated, errors }`.
|
|
196
|
+
4. **MCP ingestion** — import a server; each of its tools becomes a data tool.
|
|
197
|
+
|
|
198
|
+
### The one gotcha: env templating is manifest-only
|
|
199
|
+
|
|
200
|
+
`{{env.*}}` (and `{{secret.*}}`) are substituted **at ingestion, on the manifest
|
|
201
|
+
path**. Other write paths do **not** resolve `{{env.*}}` — a literal
|
|
202
|
+
`{{env.API_URL}}` there fails the `http`/`https` URL check and 422s. Rule:
|
|
203
|
+
**manifest tools may template the URL with `{{env.*}}`; tools written any other
|
|
204
|
+
way must ship a literal URL.** `{{ctx.*}}` and `{{param}}` work everywhere (they
|
|
205
|
+
resolve at turn time, not ingestion).
|
|
206
|
+
|
|
207
|
+
## Making it appear — and enter the tool-loop
|
|
208
|
+
|
|
209
|
+
1. **Panel visibility** = registered in the catalog. Data tools are marked
|
|
210
|
+
editable; code tools are allow/deny only.
|
|
211
|
+
2. **Per-agent exposure** is set from the same panel, or by the agent's allowlist.
|
|
212
|
+
3. **Entering the tool-loop** is decided by the **policy allowlist**, not by tool
|
|
213
|
+
type: deny wins, otherwise the agent sees `tools_allow ∪ tools_allow_groups`
|
|
214
|
+
(or all, when both are absent). See [Agents](AGENTS.md#the-allowlist-convention).
|
|
215
|
+
4. **Deferred tools** (`tools_deferred`) are *not* offered directly — they appear
|
|
216
|
+
as a short "available tools" list and the model must call `tool_search` to
|
|
217
|
+
enable one. This is progressive disclosure for large toolsets — see
|
|
218
|
+
[Context](CONTEXT.md).
|
|
219
|
+
|
|
220
|
+
## Parallel tool calls
|
|
221
|
+
|
|
222
|
+
A model can ask for several tools in one step. By default the engine runs them one
|
|
223
|
+
at a time. Set `limits[:tool_concurrency]` above 1 (see
|
|
224
|
+
[Agents](AGENTS.md#tool_concurrency--parallel-tool-calls)) and the calls in that
|
|
225
|
+
batch run concurrently, **at most N in flight**, on the turn's own reactor — so
|
|
226
|
+
the wall-clock of a batch of slow data tools approaches the slowest call rather
|
|
227
|
+
than their sum. The cap covers every enveloped tool of the turn, including the
|
|
228
|
+
ones `tool_search` promotes mid-turn.
|
|
229
|
+
|
|
230
|
+
It applies only to what the *model* fans out. Two primitives already parallelize
|
|
231
|
+
deterministically and are unaffected: `spawn_subagents` (capped at 8 children) and
|
|
232
|
+
`Insika::Tools::Concurrency.gather` (fan-out inside one tool). System tools —
|
|
233
|
+
`tool_search`, `load_skill`, `remember`, `spawn_subagent` — are not enveloped and
|
|
234
|
+
so are not gated by the cap; they are trivial or capped on their own.
|
|
235
|
+
|
|
236
|
+
Turning it on changes three things, all of them worth knowing before you do:
|
|
237
|
+
|
|
238
|
+
- **`max_tool_calls` becomes approximate.** The limit is checked per call, but a
|
|
239
|
+
call that trips it does not stop its siblings — the whole batch finishes and the
|
|
240
|
+
turn then fails. With a cap of 4, up to 3 extra tools may have executed. The turn
|
|
241
|
+
still fails at the right boundary; the count is just no longer exact.
|
|
242
|
+
- **The transcript records results in completion order.** Providers key results by
|
|
243
|
+
`tool_call_id`, so the wire stays valid and persistence is faithful to what was
|
|
244
|
+
sent — but a replayed transcript no longer reads in call order.
|
|
245
|
+
- **`turn_timeout` can overrun by up to `tool_timeout`.** A turn deadline does not
|
|
246
|
+
cancel a tool call already in flight in a sibling fiber; it waits for it. Each
|
|
247
|
+
call is still bounded by its own `tool_timeout`, which is what bounds the
|
|
248
|
+
overrun. Serial execution is unaffected (there, the deadline lands directly in
|
|
249
|
+
the fiber running the tool).
|
|
250
|
+
|
|
251
|
+
Approvals and concurrency are mutually exclusive per turn — the approval gate wins
|
|
252
|
+
and the turn goes serial. That is a deadlock avoided, not a preference.
|
|
253
|
+
|
|
254
|
+
## Egress: the SSRF guard (and its silent failure)
|
|
255
|
+
|
|
256
|
+
Data tools make outbound HTTP, so every call passes through the **EgressGuard**, a
|
|
257
|
+
Server-Side Request Forgery defense. The default posture is **strict: public
|
|
258
|
+
`https` only.** Three env vars widen it:
|
|
259
|
+
|
|
260
|
+
| Env | Effect |
|
|
261
|
+
|-----|--------|
|
|
262
|
+
| `INSIKA_EGRESS_HOSTS` | allowlist of hosts (CSV). The safe way to permit a specific backend. |
|
|
263
|
+
| `INSIKA_EGRESS_ALLOW_HTTP=1` | permit plain `http` — **loopback dev only** |
|
|
264
|
+
| `INSIKA_EGRESS_ALLOW_PRIVATE=1` | permit private/loopback IPs — **dev only** |
|
|
265
|
+
|
|
266
|
+
> ⚠️ **Egress failures are silent.** When a tool targets a blocked host (e.g. a
|
|
267
|
+
> plain-`http` localhost backend without the opt-ins), the guard turns the block
|
|
268
|
+
> into a `{ error: … }` returned **to the model** — the request never leaves the
|
|
269
|
+
> process, yet the stream still emits a tool call, so the model narrates a
|
|
270
|
+
> plausible failure and the conversation *looks* like it worked. You will not see
|
|
271
|
+
> an exception.
|
|
272
|
+
>
|
|
273
|
+
> **Always verify by the trace, never by the reply:** open the Studio session
|
|
274
|
+
> viewer — a healthy call shows the request, args, and the backend's `200`; a
|
|
275
|
+
> missing or errored call is almost always egress (host not in the allowlist, or
|
|
276
|
+
> `http`/private without the opt-in).
|
|
277
|
+
|
|
278
|
+
Egress is **orthogonal** to registration and allowlisting: a tool can be
|
|
279
|
+
registered, allowed, offered to the model, and still blocked at call time.
|
|
280
|
+
|
|
281
|
+
## Troubleshooting: "the tool is missing"
|
|
282
|
+
|
|
283
|
+
Work down this checklist:
|
|
284
|
+
|
|
285
|
+
1. **Registered?** Is it in the catalog (Studio Tools panel)? If not, the write
|
|
286
|
+
or import failed — check the manifest `errors[]`, and run `insika doctor`: a stored
|
|
287
|
+
definition that no longer validates is dropped from the catalog, and the
|
|
288
|
+
`data-tools` check is the only place that says so.
|
|
289
|
+
2. **Allowed for this agent?** In `tools_allow` (or an allowed group), and not in
|
|
290
|
+
`tools_deny`?
|
|
291
|
+
3. **Egress?** If it *appears and is called* but "fails", open the trace — a
|
|
292
|
+
blocked call is ~99% egress.
|
|
293
|
+
4. **URL literal?** For non-manifest tools, an unresolved `{{env.*}}` would have
|
|
294
|
+
422'd at import — re-check the definition.
|
|
295
|
+
|
|
296
|
+
## See also
|
|
297
|
+
|
|
298
|
+
- [Agents](AGENTS.md) — allowlists, groups, and per-agent tool exposure.
|
|
299
|
+
- [Plugins](PLUGINS.md) — where a code tool comes from, and how to package one.
|
|
300
|
+
- [Security](SECURITY.md) — egress, sandbox, and approval gating together.
|
|
301
|
+
- [Architecture](ARCHITECTURE.md) — the tool-loop and side-effect checkpointing.
|
|
302
|
+
- [`examples/data-tool/`](https://github.com/guizaols/insika/tree/main/examples/data-tool/) — a runnable data tool + the egress note.
|
data/docs/WHY.md
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Why Insika
|
|
3
|
+
parent: Understand the idea
|
|
4
|
+
nav_order: 1
|
|
5
|
+
permalink: /why/
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Why Insika
|
|
9
|
+
|
|
10
|
+
**Ruby is ready for production AI.** The narrative that you must reach for Python
|
|
11
|
+
to ship serious LLM applications is out of date. [RubyLLM](https://rubyllm.com)
|
|
12
|
+
gave the language a first-class, provider-agnostic LLM client — chat, tools,
|
|
13
|
+
streaming, embeddings, moderation — a genuinely valuable foundation that closed
|
|
14
|
+
the "Ruby can't do AI" gap. Insika is the next layer up: it takes those
|
|
15
|
+
primitives and makes an agent *dependable in production*.
|
|
16
|
+
|
|
17
|
+
Because **an agent without a harness is just a chat loop.** The model does the
|
|
18
|
+
reasoning, but everything that makes an agent survive contact with real traffic —
|
|
19
|
+
durable state, tools, guardrails, evals, operations — lives in the engine around
|
|
20
|
+
it. Those harnesses exist, mature, for other ecosystems. Ruby teams have mostly
|
|
21
|
+
been told to glue libraries together. They shouldn't have to.
|
|
22
|
+
|
|
23
|
+
Insika is an **agent runtime for Ruby**: the turn pipeline, the operational
|
|
24
|
+
surface, and the safety layer, in one deployable piece — behind an
|
|
25
|
+
OpenAI-Responses-compatible API. It stands on RubyLLM for the provider layer and
|
|
26
|
+
adds everything between a chat call and a production agent.
|
|
27
|
+
|
|
28
|
+
And it does it *ergonomically*. A complete, running agent is a few lines of Ruby —
|
|
29
|
+
the same speed-to-first-agent story Python teams tell, without leaving Ruby:
|
|
30
|
+
|
|
31
|
+
```ruby
|
|
32
|
+
require "insika"
|
|
33
|
+
|
|
34
|
+
agent = Insika.agent("assistant") do
|
|
35
|
+
model "deepseek-chat"
|
|
36
|
+
provider :deepseek
|
|
37
|
+
instructions "You are a concise, friendly assistant."
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
puts agent.reply("hi, what can you do?") # one turn, in-process
|
|
41
|
+
agent.serve # ...or a full server: /studio + /v1/responses
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Every capability below is reachable the same way — see [`examples/`](https://github.com/guizaols/insika/tree/main/examples/),
|
|
45
|
+
one small runnable project per capability.
|
|
46
|
+
|
|
47
|
+
## Four ways to run an agent
|
|
48
|
+
|
|
49
|
+
There are, roughly, four ways to put an LLM agent into production. They're not
|
|
50
|
+
wrong — they're different amounts of "build it yourself":
|
|
51
|
+
|
|
52
|
+
- **Raw SDK loop.** Call the provider SDK directly and hand-roll the
|
|
53
|
+
reason→act→observe loop. Minimal to start; everything operational is on you.
|
|
54
|
+
- **Assemble a framework.** Compose library pieces — a tool abstraction here, a
|
|
55
|
+
memory store there, your own persistence and moderation. Flexible, but you own
|
|
56
|
+
the integration and the gaps between the parts.
|
|
57
|
+
- **Hosted agent gateway.** A managed service runs the agent behind an API. You
|
|
58
|
+
get operations for free, but your agents, prompts, and conversation data live in
|
|
59
|
+
someone else's system.
|
|
60
|
+
- **An agent runtime (this project).** The loop, durability, tools, safety, and an
|
|
61
|
+
operations UI as one thing you deploy in your own infrastructure.
|
|
62
|
+
|
|
63
|
+
How they compare on what production actually demands:
|
|
64
|
+
|
|
65
|
+
| | Raw SDK loop | Assemble a framework | Hosted gateway | **Insika (runtime)** |
|
|
66
|
+
|---|:---:|:---:|:---:|:---:|
|
|
67
|
+
| Get a first agent talking | ✅ | ⚠️ | ✅ | ✅ |
|
|
68
|
+
| Add/change a tool without a redeploy | ❌ | ❌ | ⚠️ | ✅ |
|
|
69
|
+
| Durable, resumable turns (crash mid-conversation) | ❌ | ⚠️ | ✅ | ✅ |
|
|
70
|
+
| Content-safety guardrails built in | ❌ | ⚠️ | ⚠️ | ✅ |
|
|
71
|
+
| Evals / regression gating as a primitive | ❌ | ⚠️ | ⚠️ | ✅ |
|
|
72
|
+
| Operations UI included | ❌ | ❌ | ✅ | ✅ |
|
|
73
|
+
| Runs in your infrastructure (you own the data) | ✅ | ✅ | ❌ | ✅ |
|
|
74
|
+
| One container, no orchestrator to start | ✅ | ⚠️ | — | ✅ |
|
|
75
|
+
|
|
76
|
+
✅ built in · ⚠️ possible, but you build/integrate it · ❌ not addressed · — n/a
|
|
77
|
+
|
|
78
|
+
## What we do differently
|
|
79
|
+
|
|
80
|
+
- **Tools are data, not code.** Define a tool with a JSON Schema manifest and a
|
|
81
|
+
declarative binding, and the running agent picks it up — no rebuild, no redeploy.
|
|
82
|
+
Import whole toolsets from an MCP server or a manifest at runtime. In most setups
|
|
83
|
+
a new tool is a code change; here it's a row.
|
|
84
|
+
- **Durability is the default.** Every turn checkpoints; sessions and tasks recover
|
|
85
|
+
across restarts; continuous replication is one env var away, with a documented
|
|
86
|
+
restore drill. If the box dies mid-conversation, the conversation doesn't.
|
|
87
|
+
- **Safety is built in, not bolted on.** Input guardrails (prompt-injection and
|
|
88
|
+
abuse handling that fails gracefully *without* burning a model turn), content
|
|
89
|
+
moderation, PII/secret redaction in the output stream, and a post-turn validator
|
|
90
|
+
— configured, not hand-rolled, and opt-in per agent.
|
|
91
|
+
- **Evals are a primitive.** Golden conversations, LLM-as-judge, and baseline
|
|
92
|
+
gating run against the *same* API your users hit — so a prompt or model change
|
|
93
|
+
that regresses behavior fails loudly before it ships.
|
|
94
|
+
- **An operations UI is included.** Studio lets an operator manage agents, tools,
|
|
95
|
+
approvals, and traces — pause a task, approve a sensitive action, inspect a
|
|
96
|
+
tool-call trace — without writing code. Headless when you want it, operable when
|
|
97
|
+
you need it.
|
|
98
|
+
- **One container, one file.** SQLite in WAL mode with streaming replication. No
|
|
99
|
+
queue cluster, no orchestrator, no managed database required to start — and it
|
|
100
|
+
runs a real production workload today.
|
|
101
|
+
- **Fibers, not a thread per request.** An LLM turn is almost all *waiting* on the
|
|
102
|
+
provider. Insika runs on the [Async](https://github.com/socketry/async) fiber
|
|
103
|
+
scheduler and serves under [Falcon](https://github.com/socketry/falcon), so one
|
|
104
|
+
process handles thousands of concurrent turns on a handful of connections instead
|
|
105
|
+
of pinning a heavyweight thread per call. This is exactly the model
|
|
106
|
+
[RubyLLM's async guide](https://rubyllm.com/async/) recommends — *"Falcon is a
|
|
107
|
+
Ruby application server built on fibers; with Falcon, async just works"* — and it
|
|
108
|
+
composes for free, because RubyLLM's HTTP cooperates with Ruby's fiber scheduler.
|
|
109
|
+
It's a big part of why one box goes so far.
|
|
110
|
+
- **The engine gets out of the way.** On top of that concurrency, all the machinery
|
|
111
|
+
above adds **well under a millisecond of overhead per turn**; a single process
|
|
112
|
+
sustains thousands of turns per second of pure engine work. A neutral,
|
|
113
|
+
provider-free benchmark in the repo reproduces it with one command — see
|
|
114
|
+
[BENCHMARK.md](BENCHMARK.md). The rest of a turn's latency is the model, not us.
|
|
115
|
+
*(That number is engine overhead only — end-to-end latency is provider-bound and
|
|
116
|
+
not claimed here.)*
|
|
117
|
+
|
|
118
|
+
## When a lighter tool is the right call
|
|
119
|
+
|
|
120
|
+
Insika is for when an agent has to run **unattended, durably, and safely, in
|
|
121
|
+
production**. If that's not you yet, reach for less:
|
|
122
|
+
|
|
123
|
+
- If you only need LLM calls with tools in Ruby and nothing operational around
|
|
124
|
+
them, **[RubyLLM](https://rubyllm.com) alone is excellent** — and it's exactly
|
|
125
|
+
what Insika builds on. Reach for Insika when "nothing operational around them"
|
|
126
|
+
stops being true: when the agent has to run unattended, recover from crashes,
|
|
127
|
+
enforce safety, and be operable by someone who isn't you.
|
|
128
|
+
- If your team is committed to another language ecosystem, use a runtime native to
|
|
129
|
+
it — the ideas here travel, the code doesn't.
|
|
130
|
+
- If you want a minimal terminal coding assistant rather than a product platform,
|
|
131
|
+
a single-purpose CLI agent will be simpler.
|
|
132
|
+
|
|
133
|
+
## See also
|
|
134
|
+
|
|
135
|
+
- [README](https://github.com/guizaols/insika#readme) — quickstart and the drop-in `/v1/responses` API.
|
|
136
|
+
- [BENCHMARK.md](BENCHMARK.md) — the neutral, reproducible engine benchmark.
|
|
137
|
+
- [OBSERVABILITY.md](OBSERVABILITY.md) — OpenTelemetry tracing (opt-in).
|