silas 0.4.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (59) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +211 -0
  3. data/DEPLOY.md +111 -0
  4. data/README.md +87 -250
  5. data/app/controllers/silas/api/v1/approvals_controller.rb +10 -0
  6. data/app/controllers/silas/inbox/invocations_controller.rb +8 -0
  7. data/app/helpers/silas/inbox/trace_helper.rb +20 -5
  8. data/app/jobs/silas/channel_delivery_job.rb +15 -0
  9. data/app/models/concerns/silas/inbox/broadcastable.rb +12 -0
  10. data/app/models/silas/compaction.rb +32 -0
  11. data/app/models/silas/tool_invocation.rb +29 -3
  12. data/app/views/layouts/silas/inbox.html.erb +89 -30
  13. data/app/views/silas/inbox/invocations/_approval_card.html.erb +36 -13
  14. data/app/views/silas/inbox/invocations/_invocation.html.erb +17 -5
  15. data/app/views/silas/inbox/sessions/_row.html.erb +14 -0
  16. data/app/views/silas/inbox/sessions/index.html.erb +16 -15
  17. data/app/views/silas/inbox/sessions/show.html.erb +9 -0
  18. data/config/routes.rb +2 -0
  19. data/db/migrate/20260725000002_create_silas_compactions.rb +26 -0
  20. data/docs/agents.md +81 -0
  21. data/docs/budgets.md +67 -0
  22. data/docs/cancellation.md +41 -0
  23. data/docs/channels.md +290 -0
  24. data/docs/configuration.md +106 -0
  25. data/docs/connections.md +55 -0
  26. data/docs/conventions.md +161 -0
  27. data/docs/evals.md +95 -0
  28. data/docs/guarantees.md +76 -0
  29. data/docs/inbox-and-api.md +84 -0
  30. data/docs/memory.md +35 -0
  31. data/docs/sandbox.md +44 -0
  32. data/docs/tools.md +77 -0
  33. data/docs/tutorial.md +268 -0
  34. data/docs/vs-eve.md +93 -0
  35. data/docs/why-silas.md +87 -0
  36. data/lib/generators/silas/channel/channel_generator.rb +72 -0
  37. data/lib/generators/silas/channel/templates/channel.rb.tt +48 -0
  38. data/lib/generators/silas/channel/templates/controller.rb.tt +66 -0
  39. data/lib/generators/silas/install/install_generator.rb +11 -1
  40. data/lib/generators/silas/install/templates/claude_skill.md +136 -0
  41. data/lib/generators/silas/install/templates/ruby_llm.rb +4 -1
  42. data/lib/silas/adapters/ruby_llm.rb +102 -46
  43. data/lib/silas/channel.rb +35 -0
  44. data/lib/silas/compactor.rb +178 -0
  45. data/lib/silas/configuration.rb +16 -0
  46. data/lib/silas/eval/dsl.rb +7 -2
  47. data/lib/silas/instrumentation.rb +7 -3
  48. data/lib/silas/ledger.rb +2 -2
  49. data/lib/silas/log_subscriber.rb +5 -0
  50. data/lib/silas/message_builder.rb +19 -0
  51. data/lib/silas/registry.rb +5 -2
  52. data/lib/silas/schedule.rb +44 -15
  53. data/lib/silas/slack.rb +8 -5
  54. data/lib/silas/step_runner.rb +5 -0
  55. data/lib/silas/tools/ask_question.rb +26 -0
  56. data/lib/silas/version.rb +1 -1
  57. data/lib/silas/webhook.rb +47 -0
  58. data/lib/silas.rb +3 -0
  59. metadata +30 -1
data/docs/evals.md ADDED
@@ -0,0 +1,95 @@
1
+ # Evals
2
+
3
+ Agent evals answer the question tests can't: *given these model decisions, does
4
+ the durable machinery do the right thing?* A scenario scripts the **model's
5
+ decisions** step by step; everything else is real — the real registry resolves
6
+ your real tools, the real Ledger enforces effect modes and approvals, and the
7
+ assertions read the genuine durable transcript, not a mock.
8
+
9
+ They run keyless and deterministically, which makes them a **deploy gate**:
10
+
11
+ ```sh
12
+ bin/rails silas:eval # loads <eval_dir>/**/*_eval.rb; exits 1 on failure
13
+ ```
14
+
15
+ The installer wires this into `bin/ci` (or tells you to add it to yours), and
16
+ the application template generates a working example in
17
+ `test/agent_evals/refund_desk_eval.rb`.
18
+
19
+ ## A scenario
20
+
21
+ ```ruby
22
+ Silas::Eval.scenario "over the gate: a £64 refund holds at the signal" do
23
+ input "The walnut monitor stand (order R-1002) arrived cracked."
24
+
25
+ on_step 0, call: { name: "lookup_order", arguments: { number: "R-1002" } }
26
+ on_step 1, call: { name: "issue_refund",
27
+ arguments: { number: "R-1002", amount_pence: 6400, reason: "arrived cracked" } }
28
+
29
+ expect do
30
+ assert_parked tool: "issue_refund" # the turn holds at zero compute
31
+ assert_no_tool_called "issue_refund" # and the refund row does NOT exist
32
+ end
33
+ end
34
+ ```
35
+
36
+ `on_step index, ...` is the script: what the "model" returns on that model
37
+ call. `text:` is an assistant text block, `call:`/`calls:` are tool calls. A
38
+ step index with no entry ends the scripted conversation.
39
+
40
+ ## The DSL
41
+
42
+ | Call | Meaning |
43
+ |---|---|
44
+ | `input "…"` | The turn's input. Required. |
45
+ | `on_step i, text:, call:, calls:, data:` | The scripted model output for step `i`. `data:` scripts a **structured answer** (the `final_answer` schema case) — it becomes the same block the real adapter persists, so `assert_answer_data` reads it back. |
46
+ | `approve tool: "name"` | When the turn parks on this tool, approve it (recorded as approved by `"eval"` — the same `approve!` the inbox uses) and resume. This is how you assert exactly-once **across** a hold. |
47
+ | `stub_tool "name", effect_mode:, approval: { \|**args\| … }` | Replace a real tool with a stub for a side-effect-free scenario. Unstubbed tools stay real. |
48
+ | `mode :real` | Drive a real model instead of the script. **Skipped automatically when `ANTHROPIC_API_KEY` is unset**, so the gate stays green offline. |
49
+ | `max_steps n` | Override the per-turn step cap for this scenario. |
50
+ | `metadata h` / tags on `scenario "…", tags: [...]` | Stamped on the session / used for filtering. |
51
+ | `expect { … }` | The assertion block. Required. Runs after the turn reaches its resting state. |
52
+
53
+ ## Assertions
54
+
55
+ All assertions read only the durable rows, and they **collect** failures — one
56
+ run reports every miss, not just the first.
57
+
58
+ | Assertion | Checks |
59
+ |---|---|
60
+ | `assert_tool_called "name", times: 1` | The tool actually executed (`times:` for exactly-N — the exactly-once assertion). |
61
+ | `assert_no_tool_called "name"` | No execution happened (e.g. because the turn parked first). |
62
+ | `assert_tool_arg "name", :key, value` (or a block predicate) | An argument the model passed. |
63
+ | `assert_parked tool: "name"` | The turn is waiting, held on that tool's approval (or in-doubt). |
64
+ | `assert_approved tool: "name"` | The invocation's approval state is `approved`. |
65
+ | `assert_turn_completed` / `assert_turn_failed(reason: "…")` | Terminal state. |
66
+ | `assert_final_matches(/…/)` | The final answer text. |
67
+ | `assert_answer_data(key: :verdict, value: "approve")` | The structured `final_answer` payload (whole-Hash and predicate forms too). |
68
+ | `assert_no_hallucinated_price(allowed: [])` | Every money amount in the final answer traces to a number the agent actually saw (tool results or the input), allowing pence↔pounds scaling. |
69
+ | `assert_rubric "…" ` | LLM-graded check against `config.eval_grader` — **skips offline** rather than failing the gate (set `SILAS_EVAL_STRICT=1` to make skips fail). |
70
+
71
+ ## How a scenario runs (so you can trust it)
72
+
73
+ The driver starts a real session (`Silas.agent.start`), runs the real
74
+ `AgentLoopJob` inline on the `:test` queue adapter, and — for each `approve
75
+ tool:` — performs the same `approve!` an operator would, then resumes the
76
+ loop. Scripted mode swaps only the inference adapter; `mode :real` uses your
77
+ configured one.
78
+
79
+ Two consequences worth knowing:
80
+
81
+ - **Rows persist.** Scenarios commit ordinary durable rows to the database of
82
+ the environment you run them in (they need real seeds for tools that read
83
+ your tables — the template's CI runs `db:seed silas:eval`). Run against a
84
+ scratch database if residue bothers you.
85
+ - **Config is restored per run**, but scenarios execute sequentially in one
86
+ process — don't have two scenarios fight over global state you set yourself.
87
+
88
+ ## Configuration
89
+
90
+ ```ruby
91
+ Silas.configure do |c|
92
+ c.eval_dir = "test/agent_evals" # where *_eval.rb files live
93
+ # c.eval_grader = ->(prompt) { … } # custom LLM grader for assert_rubric
94
+ end
95
+ ```
@@ -0,0 +1,76 @@
1
+ # Guarantees
2
+
3
+ Durability in Silas is a contract, not a slide. Everything on this page is
4
+ verified by the in-repo chaos harness (`chaos_host/bin/chaos`), which
5
+ `kill -9`s live agents hundreds of times per release — the current gate is
6
+ 100/100 completions per mode across a 295-run matrix, zero duplicate effects,
7
+ byte-identical replay, on SQLite and Postgres. Results live in
8
+ `chaos_host/results/` and every run is reproducible.
9
+
10
+ ## Turns survive hard process death
11
+
12
+ Worker `kill -9`, whole-tree `kill -9`, SIGTERM deploys — a turn resumes from
13
+ its last completed step. Each step checkpoints through Active Job
14
+ Continuations; the resume replays completed work **from rows**, never by
15
+ re-calling the model or re-running settled tools.
16
+
17
+ ## `transactional!` tools execute exactly once
18
+
19
+ The tool's database writes and the ledger's record of "this ran" commit or
20
+ roll back **in one transaction**:
21
+
22
+ - **Crash before commit** → both roll back; resume runs the tool from a clean
23
+ slate. No orphan effect.
24
+ - **Crash after commit** → the ledger row says `completed`; resume skips the
25
+ step. No second effect.
26
+
27
+ No idempotency key is required, because the dedup lives in your database — the
28
+ same transaction as the effect. This is the guarantee that requires the ledger
29
+ to live *inside* your app: a runtime outside your database can retry and
30
+ reconcile, but it can't make its record and your effect one atomic commit.
31
+
32
+ ## Ambiguity parks instead of guessing
33
+
34
+ The default effect mode is `at_most_once!`: when a crash makes an execution
35
+ ambiguous ("did the email send?"), the invocation parks **in doubt** for a
36
+ human verdict rather than re-firing blind. `idempotent!` is the explicit
37
+ opt-in to automatic re-runs. Never double-pay; sometimes ask.
38
+
39
+ ## Approvals park at zero compute
40
+
41
+ A turn awaiting a human exits the worker entirely — no held thread, no polling
42
+ loop. Approving enqueues a fresh job that replays from rows. Parks expire
43
+ (default 7 days, `config.approval_ttl`) rather than ghosting forever.
44
+
45
+ ## Errors can't strand a turn
46
+
47
+ Transient model errors (rate limits, overloads, timeouts) back off and retry
48
+ from the checkpoint. Exhausted retries and permanent rejections (bad key, bad
49
+ request) expire pending approvals and fail the turn loudly. And a turn can
50
+ never sit in `running` forever: the recurring `DeadJobRescuerJob` retries jobs
51
+ failed by dead-worker reaping and fails turns stranded by a loop job that died
52
+ outside the retry list. The rescuer is part of the contract — keep it in
53
+ `config/recurring.yml`, and monitor worker liveness (the rescuer can requeue
54
+ work; it cannot conjure a consumer).
55
+
56
+ ## Deploys can't corrupt a run
57
+
58
+ Instructions are snapshotted per turn. Tools, skills, connections, and the
59
+ final-answer schema are model-visible state, captured in a definitions digest —
60
+ a deploy that changes them while a turn is parked fails that turn loudly on
61
+ resume (`NondeterminismError`) instead of quietly resuming into a different
62
+ agent. Settle parked turns before shipping agent changes.
63
+
64
+ ## Compaction can't corrupt a replay
65
+
66
+ Long conversations summarise past `config.compact_at` — but a summary is a
67
+ **persisted, exactly-once effect** (claimed compare-and-swap, written once),
68
+ never a rebuild-time computation. The message array a resumed turn sees is
69
+ byte-identical to the one the crashed turn saw.
70
+
71
+ ## The one rule you owe the contract
72
+
73
+ Run agents on Solid Queue (or `:inline` for scripts) — never ActiveJob's
74
+ in-process `:async` adapter, which runs a re-enqueued continuation
75
+ concurrently with the original and double-executes steps. Silas raises on it
76
+ in production; `bin/rails silas:doctor` flags it everywhere.
@@ -0,0 +1,84 @@
1
+ # Inbox & API
2
+
3
+ Everything an operator needs ships in the gem — mount the engine (the
4
+ installer does this) and it's there. No dashboard to build, no second console
5
+ to deploy.
6
+
7
+ ## The inbox — `/silas/inbox`
8
+
9
+ A session rail grouped **Held / Working / Filed**, web chat (start a session
10
+ or reply from the browser — same durable loop), a live step-trace that
11
+ streams tokens over Turbo as the agent runs, approval and question cards
12
+ hoisted to the top of the session, a full audit trail (every tool call's
13
+ arguments and result, who cleared what and why), cancel, raise-budget, and
14
+ per-session token/cost accounting priced from RubyLLM's model registry.
15
+
16
+ It's **deny-by-default** — invisible until you wire auth:
17
+
18
+ ```ruby
19
+ Silas.configure do |c|
20
+ # The lambda DENIES by rendering (or head-ing) and PASSES by not rendering.
21
+ c.inbox_auth = ->(controller) { controller.head :not_found unless controller.current_user&.admin? }
22
+ # c.inbox_public_read = true # read-only demo mode; writes stay gated
23
+ # c.model_prices["your-fine-tune"] = { in: 3000, out: 15_000 } # price overrides
24
+ end
25
+ ```
26
+
27
+ `config.inbox_actor` names the identity recorded on approvals (defaults to
28
+ `current_user&.email || "inbox"`). Live streaming activates automatically when
29
+ the host app has `turbo-rails` (every default Rails app does); without it the
30
+ trace falls back to a polling refresh. The gem takes no turbo dependency.
31
+
32
+ ## The JSON API — `/silas/api/v1`
33
+
34
+ The same surface over HTTP, also deny-by-default (`config.api_auth`, same
35
+ contract as the inbox; `config.api_actor` names the API identity):
36
+
37
+ ```sh
38
+ curl -X POST .../silas/api/v1/sessions -d "input=Refund order 42, £12.50"
39
+ curl .../silas/api/v1/sessions/1?trace=1 # turns + steps + tool calls
40
+ curl .../silas/api/v1/sessions/1/approvals # what's held
41
+ curl -X POST .../silas/api/v1/approvals/7/approve # the same approve! as the inbox
42
+ curl -X POST .../silas/api/v1/approvals/9/answer -d "answer=Use the June invoice"
43
+ curl -X POST .../silas/api/v1/sessions/1/turns -d "input=Now email them" # 409 if busy
44
+ curl -X POST .../silas/api/v1/turns/9/cancel
45
+ curl -N .../silas/api/v1/sessions/1/stream # server-sent events
46
+ ```
47
+
48
+ The stream is SSE at **row granularity** — turn, completed-step, and
49
+ invocation changes, at-least-once with `Last-Event-ID` resume (ids are
50
+ epoch-ms watermarks). `?poll=1` returns the backlog and closes, curl-friendly;
51
+ streams close themselves after `config.api_stream_max_duration` and clients
52
+ reconnect. Per-token streaming is deliberately the browser/Turbo feature:
53
+ deltas live in the worker process, and the gem requires no cross-process bus.
54
+
55
+ ## Streaming — decoration over durable rows
56
+
57
+ Turns stream everywhere it helps: `silas:chat` prints tokens as they arrive,
58
+ and the inbox renders them live (coalesced to ~10Hz). Deltas are never
59
+ persisted, never fed back to the model, and absent from replays — a replayed
60
+ step renders from its row — so streaming adds zero risk to the durability
61
+ contract. Custom sinks subscribe to the `"delta.silas"` notification
62
+ (`{ session_id:, turn_id:, step_id:, step_index:, text: }`, where `text` is
63
+ the accumulated string so far; filter by ids — notifications are
64
+ process-global).
65
+
66
+ ## Structured answers
67
+
68
+ Give the turn's final answer a schema in `agent.yml`:
69
+
70
+ ```yaml
71
+ final_answer:
72
+ type: object
73
+ properties:
74
+ verdict: { type: string }
75
+ amount_pence: { type: integer }
76
+ required: [verdict]
77
+ ```
78
+
79
+ `Turn#answer_data` returns the parsed Hash (`answer_text` stays for prose
80
+ agents); the API carries it as `answer_data`, and evals assert on it with
81
+ `assert_answer_data(key: :verdict, value: "approve")`. Rendered through
82
+ RubyLLM's `with_schema`, so each provider's native structured-output mode is
83
+ used. The schema is model-visible state — changing it mid-turn fails the turn
84
+ loudly rather than resuming into a different contract.
data/docs/memory.md ADDED
@@ -0,0 +1,35 @@
1
+ # Memory
2
+
3
+ Silas memory is **graph-shaped, not a graph database**: facts stored as
4
+ `subject · attribute · content` triples with provenance and supersession —
5
+ "author:jane · report_format: prefers CSV", and a new value retires the old
6
+ one instead of piling up beside it.
7
+
8
+ ## Approval-gated by default
9
+
10
+ The first time the model calls `remember`, nothing persists — the memory
11
+ **parks as a card in your inbox** and you approve or decline it
12
+ (`config.memory_approval = :always`, the default). An agent that can silently
13
+ write its own long-term memory is an agent that can silently drift;
14
+ the gate keeps a person in that loop. `:never` auto-approves once you trust
15
+ an agent's judgment, and `config.memory = false` removes the tools entirely.
16
+
17
+ ## Recall
18
+
19
+ A handful of the most recent relevant memories (default 8,
20
+ `config.memory_injection_limit`) are injected into each turn automatically;
21
+ the `recall` tool digs deeper on demand. Memories are private per agent, or
22
+ `shared: true` for the whole staff.
23
+
24
+ ## What belongs here — and what doesn't
25
+
26
+ Memory is for the fuzzy residue with no natural home: preferences, standing
27
+ context, things a colleague would jot in a notebook. Your **domain data does
28
+ not belong here** — it belongs in your own tables, which your tools already
29
+ read. If it has a schema, it's a model; if it's a remark, it's a memory.
30
+
31
+ ## The fine print
32
+
33
+ Memory tools are model-visible state, so toggling `config.memory` (like any
34
+ builtin) changes the definitions digest — settle parked turns before flipping
35
+ it, or they fail loudly on resume ([guarantees](guarantees.md)).
data/docs/sandbox.md ADDED
@@ -0,0 +1,44 @@
1
+ # Sandbox
2
+
3
+ Code execution is **off by default** (`config.sandbox = :none`). Configure a
4
+ sandbox and the `run_code` tool is advertised to the model automatically —
5
+ always `at_most_once!`, because an exec is an external effect.
6
+
7
+ ## Built-in: `:docker`
8
+
9
+ A hardened container seam — resource-capped, network-off by default, with
10
+ knobs for image, memory, CPUs, pids, and timeout
11
+ ([configuration](configuration.md)). It's honest-but-interim: a container is
12
+ weaker than a microVM, and it runs on the same host as your ledger and
13
+ `RAILS_MASTER_KEY`. Treat it as suitable for code **you** wrote, not code the
14
+ model wrote.
15
+
16
+ ## Real isolation: hermetic
17
+
18
+ For untrusted or model-generated code, the companion gem
19
+ [hermetic](https://github.com/danielstpaul/hermetic) drops straight in:
20
+
21
+ ```ruby
22
+ # Gemfile: gem "hermetic" (zero runtime deps)
23
+ Silas.configure do |c|
24
+ c.sandbox = Hermetic.gvisor(image: "python:3.12-slim")
25
+ # or .docker / .firecracker(kernel:, rootfs:) / .hosted(:e2b, api_key:)
26
+ end
27
+ ```
28
+
29
+ Two properties carry through the seam:
30
+
31
+ - **The trust axis is visible.** Every hermetic backend exposes `trust`
32
+ (`:vendor` / `:remote` / `:vm` / `:host`) and `off_host?`, so you can refuse
33
+ to run untrusted code on the box that holds your secrets — and pair any
34
+ local backend with `executor:` to push execution to a dedicated sandbox
35
+ host.
36
+ - **The ledger guard is auto-armed.** Configuring a hermetic backend loads its
37
+ Silas shim, so a sandbox exec attempted inside a ledger transaction fails
38
+ loudly (sandbox-backed tools must be `at_most_once!`, never
39
+ `transactional!`).
40
+
41
+ ## Bring your own
42
+
43
+ `config.sandbox` accepts any object responding to `#run` — the seam is the
44
+ contract, the backends are interchangeable.
data/docs/tools.md ADDED
@@ -0,0 +1,77 @@
1
+ # Tools & approvals
2
+
3
+ A tool is one file in `app/agent/tools/`. The filename is the tool's identity;
4
+ the keyword signature of `#call` is the schema the model sees. No registry, no
5
+ wiring — add a file, restart, and the model can use it.
6
+
7
+ ```ruby
8
+ # app/agent/tools/issue_refund.rb -> the "issue_refund" tool
9
+ class Agent::Tools::IssueRefund < Silas::Tool
10
+ description "Refund part or all of an order."
11
+ param :amount_pence, :integer, desc: "Amount in pence (1800 = £18.00)"
12
+ approval ->(session:, input:) { input[:amount_pence] > 2_500 ? :user_approval : :approved }
13
+ transactional!
14
+
15
+ def call(number:, amount_pence:, reason:)
16
+ order = Order.find_by!(number: number)
17
+ refund = order.refunds.create!(amount_pence:, reason:)
18
+ { refunded_pence: refund.amount_pence, order: order.number }
19
+ end
20
+ end
21
+ ```
22
+
23
+ The rules:
24
+
25
+ - **Keywords only** — the keyword signature *is* the schema; `param` refines
26
+ types and descriptions.
27
+ - **Return a Hash** (anything else is wrapped as `{"value" => ...}`). Raising
28
+ records a failed invocation the model sees — don't rescue-and-swallow.
29
+ - `session` is available inside `call` (the `Silas::Session` row), so tools
30
+ can read channel metadata or scope queries.
31
+
32
+ ## Effect modes — decide by where the side effect lives
33
+
34
+ | The tool… | Declare | What you get |
35
+ |---|---|---|
36
+ | writes this app's database | `transactional!` | effect + ledger commit atomically → **exactly-once**, even through `kill -9` |
37
+ | calls anything external (email, HTTP, Slack…) | `at_most_once!` (the default) | a crash mid-call leaves it **in doubt** → parks for a human verdict, never re-fires blind |
38
+ | only reads, safe to repeat | `idempotent!` | crash replays may re-run it freely |
39
+
40
+ Never mark an external call `transactional!` — the ledger cannot roll back a
41
+ sent email. Model money as rows in your own database wherever you can; that's
42
+ what upgrades the guarantee to exactly-once
43
+ ([guarantees](guarantees.md)).
44
+
45
+ ## Approval — who holds the lever
46
+
47
+ ```ruby
48
+ approval :never # default — runs without asking
49
+ approval :always # every call parks for a human
50
+ approval :once # one approval per identical (tool, arguments) pair per session
51
+ approval ->(session:, input:) { ... } # decide from the arguments
52
+ ```
53
+
54
+ A lambda returns `:user_approval`, `:approved`, `:not_applicable`, or
55
+ `{denied: "reason"}` (the denial rides back to the model as the tool result).
56
+ Parking costs nothing — the turn **holds at the signal** at zero compute until
57
+ someone clears it from the inbox, Slack, a signed email link, or the JSON API,
58
+ all calling the same `approve!`/`decline!`. Parks expire after
59
+ `config.approval_ttl` (default 7 days).
60
+
61
+ The built-in **`ask_question`** tool is the reverse direction: the agent parks
62
+ to ask the operator something — information, not permission — and their typed
63
+ answer resumes the turn as the tool result.
64
+
65
+ ## Skills — playbooks, loaded on demand
66
+
67
+ A skill is a markdown file in `app/agent/skills/` with a `description:`
68
+ frontmatter line. The description is always visible to the model as a routing
69
+ hint; the body loads only when the model asks for it (`load_skill`), keeping
70
+ the always-on prompt small. Put procedures in skills; put identity in
71
+ `instructions.md`.
72
+
73
+ ## Remote tools
74
+
75
+ `app/agent/connections/*.yml` plugs a remote MCP server's tools in under the
76
+ same ledger, namespaced `<connection>__<tool>` — see
77
+ [connections](connections.md).
data/docs/tutorial.md ADDED
@@ -0,0 +1,268 @@
1
+ # Tutorial: build the desk
2
+
3
+ One running app, eleven short chapters, one new primitive each. You start with
4
+ a working refund-desk agent and finish with a scheduled, Slack-connected,
5
+ memory-having, staff-employing one — with evals gating every change and every
6
+ consequential action holding at the signal until a person clears it.
7
+
8
+ No API key is needed until chapter 3 (and even then it's optional — everything
9
+ except "talk to a real model" works keyless).
10
+
11
+ ## 1 · Open the desk
12
+
13
+ ```bash
14
+ rails new desk -m https://raw.githubusercontent.com/danielstpaul/silas/main/templates/desk.rb
15
+ cd desk && bin/dev
16
+ ```
17
+
18
+ Open <http://localhost:3000> — the **signal board** — then *operator inbox →*
19
+ and start a session:
20
+
21
+ > The walnut monitor stand (order R-1002) arrived cracked.
22
+
23
+ Watch the trace: `lookup_order` clears, then `issue_refund` **holds** — an
24
+ amber card at the top of the session, and the turn costs nothing while it
25
+ waits. Approve it. The turn resumes exactly where it stopped, notifies the
26
+ customer, and answers. Now check the till:
27
+
28
+ ```bash
29
+ bin/rails runner 'puts Refund.count' # => 1
30
+ ```
31
+
32
+ Try the £18 story too (`order R-1001`) — under the gate, it never holds.
33
+
34
+ What you just saw is the whole thesis: a **turn** (one request to its answer)
35
+ made of **steps** (model calls) whose tool effects go through a **ledger**;
36
+ anything gated **parks at zero compute** until a human verdict; and a crash
37
+ anywhere in that story resumes from the last completed step. Kill `bin/dev`
38
+ mid-run and restart it if you want to test that claim right now.
39
+
40
+ ## 2 · Read the agent (it's a directory)
41
+
42
+ ```
43
+ app/agent/
44
+ instructions.md # the persona — plain markdown, ERB allowed
45
+ agent.yml # model + per-turn limits; data only
46
+ tools/
47
+ lookup_order.rb # idempotent! — read-only, replays freely
48
+ issue_refund.rb # transactional! — DB effect: exactly-once, gated over £25
49
+ notify_customer.rb # at_most_once! — external effect: parks IN DOUBT on a crash
50
+ skills/ schedules/ channels/
51
+ ```
52
+
53
+ Open the three tools. Each declares an **effect mode**, and the mode is the
54
+ entire durability decision:
55
+
56
+ | The tool… | Declare | Because |
57
+ |---|---|---|
58
+ | writes this app's database | `transactional!` | effect + ledger commit atomically → **exactly-once** |
59
+ | calls anything external | `at_most_once!` (default) | an ambiguous crash **parks for a human**, never re-fires blind |
60
+ | only reads | `idempotent!` | replays may re-run it freely |
61
+
62
+ Never mark an external call `transactional!` — the ledger cannot roll back a
63
+ sent email. And note `issue_refund`'s approval lambda: policy lives **on the
64
+ tool**, next to the code it gates.
65
+
66
+ ## 3 · Change it: your first tool
67
+
68
+ The desk can't answer "what did Ada buy this year?" Give it history. Create
69
+ `app/agent/tools/order_history.rb`:
70
+
71
+ ```ruby
72
+ class Agent::Tools::OrderHistory < Silas::Tool
73
+ description "List a customer's orders by email, newest first."
74
+ idempotent!
75
+
76
+ def call(email:)
77
+ orders = Order.where(email: email.to_s.strip.downcase).order(created_at: :desc)
78
+ return { error: "no orders for #{email}" } if orders.none?
79
+
80
+ { orders: orders.map { |o| { number: o.number, item: o.item,
81
+ amount_pence: o.amount_pence, status: o.status } } }
82
+ end
83
+ end
84
+ ```
85
+
86
+ The keyword signature of `#call` **is** the schema the model sees; the
87
+ filename is the tool's name. Restart `bin/dev` (files register at boot), then:
88
+
89
+ ```bash
90
+ bin/rails silas:doctor # "tools — 4 tool(s) validate"
91
+ bin/rails silas:chat # you> what has ada@example.com ordered?
92
+ ```
93
+
94
+ (Keyless, the scripted stand-in won't improvise about your new tool — export
95
+ `ANTHROPIC_API_KEY` and restart to watch a real model use it. Everything else
96
+ in this tutorial stays keyless-friendly.)
97
+
98
+ ## 4 · Prove it: evals as the deploy gate
99
+
100
+ Open `test/agent_evals/refund_desk_eval.rb` — three scenarios already assert
101
+ the desk's contract, including *the refund holds* and *clearing it executes
102
+ exactly once*. Add one for your tool:
103
+
104
+ ```ruby
105
+ Silas::Eval.scenario "order history is grounded in rows" do
106
+ input "What has ada@example.com ordered?"
107
+
108
+ on_step 0, call: { name: "order_history", arguments: { email: "ada@example.com" } }
109
+ on_step 1, text: "Ada has two orders: the field notebook (£18.00) and the walnut monitor stand (£64.00)."
110
+
111
+ expect do
112
+ assert_tool_called "order_history", times: 1
113
+ assert_turn_completed
114
+ assert_no_hallucinated_price # every £ in the answer must trace to data the agent saw
115
+ end
116
+ end
117
+ ```
118
+
119
+ ```bash
120
+ bin/rails silas:eval # 4 scenarios, 0 failing
121
+ bin/ci # tests + evals — the deploy gate
122
+ ```
123
+
124
+ You script the **model's decisions**; the **real ledger** runs your real
125
+ tools. That's why `assert_parked` and `times: 1` are trustworthy — they read
126
+ durable rows, not mocks. Details: [evals.md](evals.md).
127
+
128
+ ## 5 · Give it a clock: schedules
129
+
130
+ A schedule is a markdown file whose body becomes the turn input. Create
131
+ `app/agent/schedules/daily_digest.md`:
132
+
133
+ ```markdown
134
+ ---
135
+ cron: "0 9 * * *"
136
+ ---
137
+ Summarize yesterday's refunds: count, total pence, and anything still held at
138
+ the signal. Keep it to five lines.
139
+ ```
140
+
141
+ ```bash
142
+ bin/rails silas:schedules
143
+ ```
144
+
145
+ That **compiles** schedules into `config/recurring.yml` — cron that fires real
146
+ work stays a reviewable git diff, and each tick is a normal durable turn (it
147
+ can hold for approval like any other). Prefer code? Drop a `.rb` subclassing
148
+ `Silas::Schedule::Handler` in the same directory.
149
+
150
+ ## 6 · Give it a doorway: Slack
151
+
152
+ The installer already scaffolded `app/agent/channels/slack.rb`. Wire
153
+ credentials and it's live:
154
+
155
+ ```bash
156
+ bin/rails credentials:edit # silas: { slack: { signing_secret: ..., bot_token: ... } }
157
+ ```
158
+
159
+ Point your Slack app's events at `/silas/channels/slack/events` (the mounted
160
+ engine verifies signatures). A new thread starts a session; replies continue
161
+ it; and a held refund renders as **Approve/Decline buttons in Slack** that
162
+ call the exact same `approve!` as the inbox. Any other transport:
163
+
164
+ ```bash
165
+ bin/rails g silas:channel whatsapp
166
+ ```
167
+
168
+ scaffolds the signature-verifying webhook and the outbound half. See
169
+ [channels.md](channels.md).
170
+
171
+ ## 7 · Let it ask: `ask_question`
172
+
173
+ Approvals are the human saying yes/no. `ask_question` is the reverse — the
174
+ agent needs *information*, not permission. It's built in; instruct it in
175
+ `app/agent/instructions.md`:
176
+
177
+ ```markdown
178
+ - If the customer's request is ambiguous (which order? partial or full?), use
179
+ ask_question to ask the operator before touching money.
180
+ ```
181
+
182
+ When the model calls it, the turn parks the same way an approval does — zero
183
+ compute, amber card — but the card has a **text box**. Your typed answer
184
+ becomes the tool result and the turn resumes with it. Same TTL, same audit
185
+ trail, same API (`POST .../approvals/:id/answer`).
186
+
187
+ ## 8 · Let it keep notes: memory
188
+
189
+ Memory is for the fuzzy residue with no natural home in your tables —
190
+ "ada@example.com prefers replacement over refund" — stored as
191
+ `subject · attribute · content` triples with provenance and supersession.
192
+
193
+ It's already on. Tell the agent when to use it (instructions again):
194
+
195
+ ```markdown
196
+ - When a customer states a durable preference, remember it.
197
+ ```
198
+
199
+ The first time the model calls `remember`, the memory **parks as a card in
200
+ your inbox** — nothing persists until you approve it
201
+ (`config.memory_approval = :always` is the default). Approved memories inject
202
+ into future turns automatically; `recall` digs deeper on demand. Your *domain*
203
+ data stays in your tables, where your tools already read it.
204
+
205
+ ## 9 · Hire staff: named agents and handoffs
206
+
207
+ One desk, many specialists. A named agent is the same directory shape under
208
+ `app/agents/<name>/`:
209
+
210
+ ```
211
+ app/agents/escalations/
212
+ instructions.md # "You handle disputes the desk hands you…"
213
+ agent.yml
214
+ tools/ # its own toolset — the desk's tools are NOT inherited
215
+ schedules/ # its own clock, ticking IT — not the root agent
216
+ ```
217
+
218
+ Restart, and the desk can delegate durably: the built-in `handoff` tool files
219
+ a **self-contained brief** that starts a linked session for the named agent —
220
+ exactly-once-guarded and cycle-checked — instead of two models chatting
221
+ freely (a cost and audit hazard, deliberately unblessed). Talk to a
222
+ specialist directly with `Silas.agent("escalations").start(input: "…")` or
223
+ `bin/rails silas:chat AGENT=escalations`; the inbox filters by agent.
224
+
225
+ ## 10 · Fit the governors: budgets and compaction
226
+
227
+ Open `app/agent/agent.yml`:
228
+
229
+ ```yaml
230
+ limits:
231
+ max_steps: 8 # runaway guard — breaching this FAILS the turn
232
+ max_cost: 0.25 # dollars per turn — breaching PARKS it
233
+ timeout: 300 # active seconds — held time never counts
234
+ ```
235
+
236
+ A parked budget breach is an amber card with a **raise-budget** control: the
237
+ top-up is recorded per turn, so one exception never loosens the standing
238
+ limits ([budgets.md](budgets.md)). And long conversations don't die at the
239
+ context window: past `config.compact_at` (default 90% of the model's window)
240
+ Silas summarises prior turns — as a **persisted, exactly-once effect**, so a
241
+ crash replay rebuilds byte-identical messages. You configure nothing; it's a
242
+ default.
243
+
244
+ ## 11 · Ship it
245
+
246
+ ```bash
247
+ bin/rails silas:doctor
248
+ ```
249
+
250
+ Work through its checklist for production:
251
+
252
+ 1. A real provider key in the production environment.
253
+ 2. Real inbox auth — replace the template's dev-only lambda in
254
+ `config/initializers/silas.rb` with your `current_user` check.
255
+ 3. Keep `silas_dead_job_rescuer` in `config/recurring.yml` — it's part of the
256
+ crash-recovery contract — and **monitor worker liveness**: the rescuer can
257
+ requeue work; it cannot conjure a consumer.
258
+ 4. `bin/ci` in your pipeline — the evals you wrote are now the gate that stops
259
+ a deploy from changing the desk's behavior unnoticed.
260
+ 5. Deploy with Kamal as usual. The hard-won operational notes (what the chaos
261
+ harness taught us about dead workers, why you never hand-delete
262
+ `solid_queue_processes` rows) are in
263
+ [DEPLOY.md](https://github.com/danielstpaul/silas/blob/main/DEPLOY.md).
264
+
265
+ Then delete the desk — models, tools, seeds — and build your own agent in the
266
+ hole it leaves. The shape you learned is the whole framework: **a directory of
267
+ plain files, a ledger that makes effects land exactly once, and a signal that
268
+ holds anything consequential until a person clears it.**