samagotchi 0.97.0 → 0.99.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +50 -1
  3. data/docs/architecture.md +11 -3
  4. data/docs/bundle-authoring.md +5 -3
  5. data/docs/cli.md +8 -3
  6. data/docs/configuration.md +8 -0
  7. data/docs/guardrails.md +8 -0
  8. data/docs/hooks.md +64 -0
  9. data/docs/internals/anthropic.md +79 -8
  10. data/docs/internals/bundles.md +1 -1
  11. data/docs/internals/engine.md +31 -3
  12. data/docs/internals/llm-context-forget.md +2 -1
  13. data/docs/internals/prompt-caching.md +9 -0
  14. data/docs/plugins.md +24 -2
  15. data/docs/sessions.md +11 -1
  16. data/docs/testing.md +27 -0
  17. data/lib/samagotchi/answer_refs.rb +16 -9
  18. data/lib/samagotchi/bundle_command.rb +2 -2
  19. data/lib/samagotchi/bundles/dev/manifest.yml +2 -2
  20. data/lib/samagotchi/bundles/source-links/hooks/source_links.rb +39 -5
  21. data/lib/samagotchi/bundles/source-links/manifest.yml +5 -5
  22. data/lib/samagotchi/bundles/system/manifest.yml +1 -1
  23. data/lib/samagotchi/engine.rb +91 -10
  24. data/lib/samagotchi/hooks/bundle_loader.rb +18 -15
  25. data/lib/samagotchi/hooks.rb +4 -1
  26. data/lib/samagotchi/kernel_loop.rb +15 -2
  27. data/lib/samagotchi/llm/anthropic_messages.rb +85 -4
  28. data/lib/samagotchi/llm/anthropic_wire.rb +90 -11
  29. data/lib/samagotchi/llm/chat_loop.rb +76 -7
  30. data/lib/samagotchi/llm/dialect.rb +8 -5
  31. data/lib/samagotchi/llm/errors.rb +51 -17
  32. data/lib/samagotchi/llm/refusal_memo.rb +6 -0
  33. data/lib/samagotchi/llm/turn_settings.rb +13 -5
  34. data/lib/samagotchi/llm_context_stale.rb +2 -2
  35. data/lib/samagotchi/memory_bundle/bundle_hook.rb +28 -9
  36. data/lib/samagotchi/memory_bundle/manifest.rb +17 -0
  37. data/lib/samagotchi/message_parts.rb +2 -1
  38. data/lib/samagotchi/output_formatter.rb +8 -2
  39. data/lib/samagotchi/pending_question.rb +7 -3
  40. data/lib/samagotchi/plugin/api.rb +6 -1
  41. data/lib/samagotchi/plugin/context.rb +21 -2
  42. data/lib/samagotchi/prompt.rb +3 -0
  43. data/lib/samagotchi/question_asked.rb +83 -0
  44. data/lib/samagotchi/question_desk.rb +3 -2
  45. data/lib/samagotchi/testing/bundle.rb +20 -1
  46. data/lib/samagotchi/testing/host.rb +1 -0
  47. data/lib/samagotchi/testing/recorder.rb +13 -2
  48. data/lib/samagotchi/tool_loads.rb +62 -0
  49. data/lib/samagotchi/tool_response.rb +5 -1
  50. data/lib/samagotchi/tool_runner.rb +29 -7
  51. data/lib/samagotchi/tools/builtin_calls.rb +16 -0
  52. data/lib/samagotchi/tools/deferred_tools.rb +80 -0
  53. data/lib/samagotchi/tools/registry.rb +7 -2
  54. data/lib/samagotchi/tools/tool_index.rb +88 -6
  55. data/lib/samagotchi/version.rb +1 -1
  56. data/lib/samagotchi/web/app.rb +30 -9
  57. data/lib/samagotchi/web/public/app.js +44 -11
  58. data/lib/samagotchi/web/public/index.html +36 -13
  59. data/lib/samagotchi/web/public/info_bar_model.js +0 -2
  60. data/lib/samagotchi/web/public/lists_view.js +32 -8
  61. data/lib/samagotchi/web/public/markdown_place.js +25 -8
  62. data/lib/samagotchi/web/public/question_cards.js +23 -6
  63. data/lib/samagotchi/web/public/session_rename.js +88 -15
  64. data/lib/samagotchi/web/public/session_title.js +6 -0
  65. data/lib/samagotchi/web/public/stage_frame.js +24 -14
  66. data/lib/samagotchi/web/public/stage_model.js +4 -2
  67. data/lib/samagotchi/web/public/stage_slots.js +41 -9
  68. data/lib/samagotchi/web/public/stage_view.js +149 -8
  69. data/lib/samagotchi/web/public/stream_handlers.js +5 -1
  70. data/lib/samagotchi/web/public/turn_model.js +20 -0
  71. data/lib/samagotchi/web/public/turn_view.js +22 -9
  72. metadata +4 -1
data/docs/plugins.md CHANGED
@@ -433,7 +433,9 @@ This is a bundle hook, the same as a `hooks/*.rb` file. See
433
433
  [hooks.md](hooks.md#hook-events) for the events; on `:after_turn`,
434
434
  `event[:present]` sets how the answer is shown in the web
435
435
  ([Presenting the answer](hooks.md#presenting-the-answer-display-only)) and `event[:refs]`
436
- attaches refs to it ([Refs on the answer](hooks.md#refs-on-the-answer-display-only)). Its label is
436
+ attaches refs to it ([Refs on the answer](hooks.md#refs-on-the-answer-display-only)); on
437
+ `:question_asked`, `event[:refs]` attaches refs to the reply that asks a question
438
+ ([Refs on a question](hooks.md#refs-on-a-question-display-only)). Its label is
437
439
  `plugin.rb (bundle my-bundle)`. The block may take only the event. If it
438
440
  raises, the error is logged and the hook is skipped.
439
441
 
@@ -629,6 +631,17 @@ and after a reload. `tool: call_tool` rules see MCP calls and misses only. A
629
631
  `call_tool` itself is removed, is refused by its name before the guardrails
630
632
  (no hook, no card).
631
633
 
634
+ **On a Messages host the model calls them by name too.** With `api:
635
+ anthropic` (not Fireworks' dialect), every tool `call_tool` reaches goes in
636
+ the request's tools deferred (`defer_loading`: its name only, until
637
+ something loads it): a built-in by its name, a source's tool by its
638
+ `acts_as` (`mcp_fs_read`; one with a refusal, an acts_as two tools share, or
639
+ a name another tool has stays out). A direct call of one runs whether
640
+ anything loaded it or not: a built-in as itself (as if declared), a
641
+ source's tool as the `call_tool` call it stands for, so its rules,
642
+ approvals, hooks and row are `call_tool`'s. Elsewhere such a call is
643
+ refused with the `call_tool` route, as before.
644
+
632
645
  ```ruby
633
646
  @source = chi.tool_source(label: "mcp", prefix: "mcp_",
634
647
  call: ->(tool, args, ctx) { run(tool, args, ctx) }, # the result, as a tool block's
@@ -757,6 +770,7 @@ full label, `plugin.rb (bundle my-bundle)`.
757
770
  | `ctx.toolkit` | the name of the session's toolkit (`full` for a session from before toolkits); nil with no session |
758
771
  | `ctx.declared_tools` | the tool names the session's next turn declares (its toolkit resolved, with its own adds and removes): ask this, not the name, whether a tool is there |
759
772
  | `ctx.use_toolkit(name)` | put the session on toolkit `name` (yours from `chi.toolkit`, or chi's): saved, applied at the next turn's start and announced (`toolkit> toolkit: NAME from the next turn's start`, the web's tools chip follows). Only before the session's first model reply and outside a turn (the declared tools start every request, and a plugin doesn't ask first); otherwise it returns false and changes nothing: tell the user to run `/toolkit NAME`. An unknown name raises ArgumentError. Needs chi 0.66.0 |
773
+ | `ctx.load_tools(names)` | load tools the session reaches but doesn't declare, so the model calls them by name: built-ins (`"delegate"`, `"chi/delegate"`), MCP tools (`"mcp_fs_read"`, `"fs/read"`). Returns the names it staged (the wire names); a declared tool, one loaded before, a removed or unknown one is skipped (an unknown one logged), so calling it every turn is fine. From a `before_turn` hook it loads for that turn; anywhere else (mid-turn too) from the next turn's start. Call it from a hook: before the session's first turn its toolkit isn't applied yet and nothing resolves. **A hint, not access**: the tools keep their guardrails and approvals, and only a host that takes tool changes (`api: anthropic`, not Fireworks) gets one, as a system message after the user's (`[tools loaded: delegate (your-bundle)]` in the session, a notice line live); elsewhere nothing is sent or saved, and the tools stay reachable with `call_tool`. A hook may switch the model after you call it, so the return value can't promise delivery. Raises ArgumentError for anything but an Array. Needs chi 0.99.0 |
760
774
  | `ctx.context` | attach outside text to this session and list it: see [Attached context](#attached-context-ctxcontext) |
761
775
 
762
776
  The Engine itself is never handed to a plugin.
@@ -1159,6 +1173,7 @@ require "samagotchi/testing"
1159
1173
  bundle = Samagotchi::Testing::Bundle.load("path/to/my-bundle", settings: { "every" => 3 },
1160
1174
  host: { session_id: "s1", cwd: Dir.pwd })
1161
1175
  bundle.fire(:after_turn, status: "completed") # → the event, after the hooks
1176
+ bundle.question_asked("Fix JIRA-1?", question: { question: "Go?" }) # → the refs, as chi's seams fire it
1162
1177
  event = bundle.before_tool_call_event({ name: "read", path: "a.txt" }, params: "read a.txt")
1163
1178
  bundle.command("/mine stop") # → what the command returned
1164
1179
  bundle.tool("my_tool", path: "a.txt") # → the text the model reads
@@ -1173,6 +1188,7 @@ bundle.http.stub(:post, "http://127.0.0.1:9000/classify", json: { label: "safe"
1173
1188
  bundle.set(running: false) # the session's values: cwd:, messages:, cancelled: …
1174
1189
  bundle.set(cost: 2.5) # ctx.cost: reported USD, or { reported:, estimated:, side:, known: }
1175
1190
  bundle.set(session_settings: { "cap" => 1 }) # the session's --set values (Bundle.load takes session_settings: too)
1191
+ bundle.tool_loads.last.names # ctx.load_tools calls; set(load_tools_staged: [...]) limits what they stage
1176
1192
  bundle.shutdown
1177
1193
  ```
1178
1194
 
@@ -1212,6 +1228,10 @@ bundle.shutdown
1212
1228
  lays them over `settings:` through chi's own `Context#settings`; the plugin's `initialize(settings)` gets
1213
1229
  `settings:` only. `bundle.set(session_settings:)` moves the session to a new id (`test-session-2`, or the
1214
1230
  `session_id:` given beside it): chi never changes them within a session, and builds `ctx.settings` once per id.
1231
+ - **`question_asked(text, kind: "question", question: {}, cwd: nil)`** fires `:question_asked` through
1232
+ `QuestionAsked.fire`, as chi's two seams do: a step whose text is `text` asks a question (or `kind: "approval"`)
1233
+ with the `question` fields, and the refs the hooks attach come back checked (the question caps, a file ref under
1234
+ `cwd:`, default the session's folder), as the step and its pending question keep them; `[]` for none.
1215
1235
  - **A hook that raises fails the call** (`Testing::HookFailed`, with the hook's error) where chi would log it and
1216
1236
  go on; `bundle.lenient { … }` collects them in `bundle.errors` instead.
1217
1237
  - **Sandbox.** A test runs with temp config, state, data and cache homes (config.yml, memories, `ctx.data_dir`, the
@@ -1269,7 +1289,9 @@ by one: the bundle publishes them into chi's tool index
1269
1289
  ([chi.tool_source](#chitool_sourcecall-acts_as-label-wait-prefix), a group
1270
1290
  per server), and the model has its two tools, `find_tools` and `call_tool` (~250 tokens in
1271
1291
  every request plus a line per server, whatever the servers have; `/mcp`
1272
- says how many). Stdio servers only, for now.
1292
+ says how many). On `api: anthropic` each tool also goes out deferred as
1293
+ `mcp_<server>_<tool>`, so a `find_tools` hit loads it and the model calls it
1294
+ by that name, as `call_tool` would run it. Stdio servers only, for now.
1273
1295
 
1274
1296
  ```yaml
1275
1297
  # config.yml
data/docs/sessions.md CHANGED
@@ -221,7 +221,7 @@ chi send --new --continues last:3fa2 -m "new day" # the next link of 3fa2's
221
221
 
222
222
  A session is named by its first message until you give it a title: `/title Coordinator` (in the REPL, an attached terminal, the web composer or `chi send -m "/title Coordinator" ID`; a parent agent may name its delegates the same way). The title is saved in the session file (`title`), whitespace collapsed and cut at 80 characters, and runs mid-turn too, at once. `/title` alone shows it (`title: none (shows the first message: hey chi …)` without one); `/title --clear` drops it, and the first message names the session again. A session whose first line was the command itself (`chi send --new -m "/title X"`) takes its preview from its first real message.
223
223
 
224
- The title names the session wherever its first message did: `chi sessions list` (plain, `--live`, tsv and json's `desc`, before the recap; json has `title` too), `chi sessions delete`'s line, the `list_sessions` tool, `chi broadcast`, the desktop helper, a parent's children (the coordinator bundle's `/children` card, `ctx.sessions.children`; `ctx.sessions.read` too), and in the web the cards, the families and chain rows, the top bar and the browser tab, the notifications and the search; the terminal's status row too (the REPL's and an attached one's, first, see [Status Line](cli.md#status-line)). The web's info bar has `✎` (Rename this session): a small input prefilled with the title; Enter or a click elsewhere sends `/title TEXT` when it changed (empty: `/title --clear`), Esc closes it unsent. A session a chi REPL owns has no `✎`. A continue's new link keeps the title (see [Session chains](#session-chains)). An older `chi web` (or any chi from before the field) that saves a session drops its title: restart `chi web` after upgrading.
224
+ The title names the session wherever its first message did: `chi sessions list` (plain, `--live`, tsv and json's `desc`, before the recap; json has `title` too), `chi sessions delete`'s line, the `list_sessions` tool, `chi broadcast`, the desktop helper, a parent's children (the coordinator bundle's `/children` card, `ctx.sessions.children`; `ctx.sessions.read` too), and in the web the cards, the families and chain rows, the top bar and the browser tab, the notifications and the search; the terminal's status row too (the REPL's and an attached one's, first, see [Status Line](cli.md#status-line)). In the web, `✎` (Rename this session) sits next to the session's title, on its active strip card or after the top bar's title: a small input over the title, prefilled with it; Enter or a click elsewhere sends `/title TEXT` when it changed (empty: `/title --clear`), Esc closes it unsent, and so does the title leaving the screen (All sessions opened, the card swiped out of a phone's strip). A session a chi REPL owns has no `✎`, nor one that shows no title yet (`/title` still works from the composer). A continue's new link keeps the title (see [Session chains](#session-chains)). An older `chi web` (or any chi from before the field) that saves a session drops its title: restart `chi web` after upgrading.
225
225
 
226
226
  ## Session chains
227
227
 
@@ -253,6 +253,16 @@ declares every tool (full). The file stores only the name: a kit's tools and pro
253
253
  `/toolkit` changes it, saved at once, from the next turn's start. See
254
254
  [Toolkits](configuration.md#toolkits).
255
255
 
256
+ A `find_tools` result on a chat host saves the tools its hit loads as `loaded: ["list_sessions", "mcp_fs_read"]` on
257
+ its `tool_response` entry (wire names: a built-in's, an MCP tool's `mcp_<server>_<tool>`), whatever the host: a
258
+ later turn on `api: anthropic` (after a `/model` switch, a resume) sends the load as references, any other host
259
+ ignores it. The names stay deferred for the session's life, across respawns, `--resume` and forks.
260
+
261
+ A plugin's `ctx.load_tools` leaves a system entry `{"role": "system", "kind": "tool_change", "content": "[tools loaded:
262
+ delegate (bundle)]", "tool_change": {"add": ["delegate"], "by": "bundle"}}` after the turn's user message, only on a
263
+ host that takes tool changes (`api: anthropic`). It is never the system head, counts as conversation, is hidden in
264
+ the web and TUI histories, and goes to no other wire; its names stay deferred for the session's life like a load's.
265
+
256
266
  ## Parent answers
257
267
 
258
268
  A session file's `parent_answers` (`hard`, `standard`, `soft`, or `null` for none) is the session's own
data/docs/testing.md CHANGED
@@ -219,3 +219,30 @@ each column; `--model GLOB`, `--since DATE` and `--min-steps N` filter, `--json`
219
219
  no session text (ids, model names, note names and digests, dates, numbers), and the suite runs it only on small
220
220
  synthetic sessions. It is how a note is judged on and off (`chi --mute model_notes_<name>`) for the same kind of
221
221
  task: see [memory.md](memory.md) ("Model notes").
222
+
223
+ ## Dependency check
224
+
225
+ `bundle exec rake deps:check` says what our dependencies did since the last check; `rake "deps:check[json]"` gives
226
+ the same as JSON. It reads only and always exits 0: it informs (a tech-debt audit round runs it first, and its NEW
227
+ lines go into that round's follow-ups), it doesn't gate, and CI doesn't run it. Three sections:
228
+
229
+ - **Engines:** each `hosts:` entry of your config.yml on a loopback or private address (an IP, `localhost`, a
230
+ single-label name or `.local`/`.lan`/`.internal`/`.home.arpa`; a public API is never probed) gets `/props`
231
+ (`build_info`: llama.cpp) and `/status` (`identity` and `chat_template`: Splash), 2 s each. A host that doesn't
232
+ answer prints "down, skipped"; one that is neither is left out. Each line carries the newest upstream release
233
+ (GitHub, no auth) with its URL, for a model to read the notes: llama.cpp's newest `b<N>` tag and how many builds
234
+ behind the host is, Splash's newest release.
235
+ - **Ruby gems:** `bundle outdated --only-explicit` (direct dependencies, with the Gemfile's requirement).
236
+ - **npm:** `npm outdated` (direct) and `npm audit`'s advisories by severity.
237
+
238
+ A section that can't be read (offline, a GitHub rate limit) says why and the others go on. What a run saw goes to
239
+ `$XDG_STATE_HOME/samagotchi/deps-check.json` (`~/.local/state/...`; `DEPS_CHECK_STATE` overrides it), outside the
240
+ repo since the engines are this machine's; the next run prints `NEW` on each line that changed since (the first run:
241
+ all), NEW lines first in each section.
242
+
243
+ **The Splash version** isn't in Splash's `/status`, so you record it: read `splash --version` on the engine host,
244
+ then `rake "deps:splash_version[1.3.2]"` (every Splash host) or `rake "deps:splash_version[host:port,1.3.2]"`. It is
245
+ kept with the build id `/status` shows (`identity.cache.build_id`); when that changes, the check asks you to confirm
246
+ the version and record it again. This step goes once a Splash release carries its version in `/status`.
247
+
248
+ The code is `script/deps/deps_check.rb`; its spec stubs every request and command (`spec/deps_check_spec.rb`).
@@ -52,7 +52,9 @@ module Samagotchi
52
52
  #
53
53
  # The target is AnswerDisplay's: the stored conversation's last message
54
54
  # when it is the model's. The list starts from the refs it already has
55
- # (re-validated), so a later hook run never loses them.
55
+ # (re-validated), so a later hook run never loses them. QuestionAsked
56
+ # uses it too, on the step that asks a question (+messages+ that one
57
+ # step, lower caps).
56
58
  class AnswerRefs
57
59
  KEY = :display_refs
58
60
  MAX_REFS = 200
@@ -63,12 +65,13 @@ module Samagotchi
63
65
  # rebuilt and checked again, the caps applied; what isn't valid any
64
66
  # more is dropped. Never raises: a reload must show the conversation.
65
67
  # A relative file path is dropped (it was stored absolute), and with
66
- # +cwd+ (the session's folder) a file outside it.
68
+ # +cwd+ (the session's folder) a file outside it. +max_refs+ and
69
+ # +max_bytes+: the caps (a question's are lower, QuestionAsked).
67
70
  # @return [Array<Hash>] the refs to send ({match:, kind:, target:, …})
68
- def self.for_display(raw, cwd: nil)
71
+ def self.for_display(raw, cwd: nil, max_refs: MAX_REFS, max_bytes: MAX_BYTES)
69
72
  return [] if raw.nil?
70
73
 
71
- kept = new([{ role: "model", KEY => raw }], cwd: cwd).to_a
74
+ kept = new([{ role: "model", KEY => raw }], cwd: cwd, max_refs: max_refs, max_bytes: max_bytes).to_a
72
75
  dropped = raw.is_a?(Array) ? raw.length - kept.length : 1
73
76
  Log.debug(:web, "display_refs_dropped", count: dropped) if dropped.positive?
74
77
  kept
@@ -81,10 +84,14 @@ module Samagotchi
81
84
 
82
85
  # @param messages [Array<Hash>] the conversation the turn stored
83
86
  # @param cwd [String, nil] the session's folder, for relative file paths
84
- def initialize(messages, cwd:)
87
+ # @param max_refs [Integer] the list's cap in refs
88
+ # @param max_bytes [Integer] the list's cap in JSON bytes
89
+ def initialize(messages, cwd:, max_refs: MAX_REFS, max_bytes: MAX_BYTES)
85
90
  last = Array(messages).last
86
91
  @target = last if last.is_a?(Hash) && (last[:role] || last["role"]).to_s == "model"
87
92
  @roots = RowRefs.roots(cwd)
93
+ @max_refs = max_refs
94
+ @max_bytes = max_bytes
88
95
  @refs = []
89
96
  @seen = {}
90
97
  @bytes = 2 # the list's []
@@ -143,10 +150,10 @@ module Samagotchi
143
150
 
144
151
  entry = AnswerRef.from(value, roots: @roots).to_h
145
152
  return true if @seen.key?(entry)
146
- return over_cap(hook, "#{MAX_REFS} refs") if @refs.length >= MAX_REFS
153
+ return over_cap(hook, "#{@max_refs} refs") if @refs.length >= @max_refs
147
154
 
148
155
  size = JSON.generate(entry).bytesize + (@refs.empty? ? 0 : 1)
149
- return over_cap(hook, "#{MAX_BYTES / 1024} KiB") if @bytes + size > MAX_BYTES
156
+ return over_cap(hook, "#{@max_bytes / 1024} KiB") if @bytes + size > @max_bytes
150
157
 
151
158
  push(entry, size)
152
159
  @added = true
@@ -160,10 +167,10 @@ module Samagotchi
160
167
  # @return [Boolean] kept
161
168
  def keep(raw)
162
169
  entry = AnswerRef.from(raw, roots: @roots, relative: false).to_h
163
- return false if @seen.key?(entry) || @refs.length >= MAX_REFS
170
+ return false if @seen.key?(entry) || @refs.length >= @max_refs
164
171
 
165
172
  size = JSON.generate(entry).bytesize + (@refs.empty? ? 0 : 1)
166
- return false if @bytes + size > MAX_BYTES
173
+ return false if @bytes + size > @max_bytes
167
174
 
168
175
  push(entry, size)
169
176
  true
@@ -476,7 +476,7 @@ module Samagotchi
476
476
  hooks.each do |hk, hook|
477
477
  hook_path = File.join(Samagotchi::MemoryBundle::Provenance.new(name: name).hooks_dir, hk)
478
478
  exists = File.exist?(hook_path) ? "ok" : "missing"
479
- @stdout.puts " #{hk}: event=#{hook.event} on_error=#{hook.on_error} priority=#{hook.priority} [#{exists}]"
479
+ @stdout.puts " #{hk}: #{hook.events_field} on_error=#{hook.on_error} priority=#{hook.priority} [#{exists}]"
480
480
  end
481
481
  @stdout.puts " requires_chi: #{st[:provenance].requires_chi}" if st[:hooks_requires_failure]
482
482
  @stdout.puts " not loaded: #{st[:hooks_requires_failure]}" if st[:hooks_requires_failure]
@@ -658,7 +658,7 @@ module Samagotchi
658
658
  end
659
659
 
660
660
  def hook_metadata_line(hook)
661
- "--- metadata: event=#{hook.event} on_error=#{hook.on_error} priority=#{hook.priority} sha256=#{hook.sha256}"
661
+ "--- metadata: #{hook.events_field} on_error=#{hook.on_error} priority=#{hook.priority} sha256=#{hook.sha256}"
662
662
  end
663
663
 
664
664
  def build(rest)
@@ -1,10 +1,10 @@
1
1
  ---
2
2
  name: dev
3
- version: 0.7.0
3
+ version: 0.7.1
4
4
  scope: system
5
5
  description: "Developer tools"
6
6
  trust_level: reviewed
7
- requires_chi: ">= 0.72.0"
7
+ requires_chi: ">= 0.98.0"
8
8
  includes:
9
9
  - known-names
10
10
  - mcp
@@ -4,8 +4,8 @@
4
4
  require "open3"
5
5
  require "samagotchi/ref"
6
6
 
7
- # An after_turn hook that links the source refs the model's answer
8
- # mentions — a JIRA ticket, a GitHub issue, an internal wiki page. In the
7
+ # An after_turn (and question_asked) hook that links the source refs the
8
+ # model's answer mentions — a JIRA ticket, a GitHub issue, an internal wiki page. In the
9
9
  # web, each ref in the answer becomes a link (`[JIRA-123](https://…)`,
10
10
  # through event[:present]: display only, the model's text stays as it was,
11
11
  # and it survives a reload). Every UI also gets one line right after the
@@ -58,6 +58,14 @@ require "samagotchi/ref"
58
58
  # action, "Open in <name>", the ref's URL. A URL that isn't http(s) stays a
59
59
  # markdown link only. The web's ref menu shows the action.
60
60
  #
61
+ # The hook is on :question_asked too (the manifest's events:, chi newer than
62
+ # 0.97.0): while a question of the model's (or an approval of its call)
63
+ # waits, the refs in the reply that asks it, and in the question's text and
64
+ # options, are refs on that reply, the same as an answer's. The web pairs
65
+ # them in the stage's asking slot, on the question's card and on that step
66
+ # in the history (a ref only in the question's text pairs there alone). No
67
+ # links in the text and no note line for it: the turn's end has those.
68
+ #
61
69
  # The note is not stored in the conversation: it is an event, replayed by a
62
70
  # UI with the session's cards (a reload keeps it, after the worker stopped
63
71
  # too: cards.json). The links are stored as the answer's display. A ref in
@@ -120,10 +128,20 @@ class SourceLinks
120
128
  end
121
129
 
122
130
  def call(event)
123
- return unless event.is_a?(Hash) && event[:type] == :after_turn
124
- return unless event[:status].to_s == "completed"
131
+ return unless event.is_a?(Hash)
125
132
  return if @sources.empty?
126
133
 
134
+ case event[:type]
135
+ when :after_turn then after_turn(event)
136
+ when :question_asked then question_asked(event)
137
+ end
138
+ end
139
+
140
+ private
141
+
142
+ def after_turn(event)
143
+ return unless event[:status].to_s == "completed"
144
+
127
145
  text = last_model_text(event[:messages])
128
146
  return if text.nil? || text.empty?
129
147
 
@@ -141,7 +159,23 @@ class SourceLinks
141
159
  event[:notify]&.call(line(found), level: :info, fallback_for: covered ? :display : nil)
142
160
  end
143
161
 
144
- private
162
+ # A question of the model's (or an approval of its call) is about to
163
+ # open: the refs in the reply that asks it and in the question's own text
164
+ # and options, each scanned on its own (a code span never runs from one
165
+ # into the next), as refs on that reply. No present (a mid-turn text has
166
+ # no display) and no note line (the after-turn one stays the terminals').
167
+ def question_asked(event)
168
+ refs = event[:refs]
169
+ return unless refs
170
+
171
+ question = event[:question].is_a?(Hash) ? event[:question] : {}
172
+ field = ->(key) { question.key?(key) ? question[key] : question[key.to_s] }
173
+ parts = [event[:text], field.call(:question), *Array(field.call(:options))]
174
+ parts.each do |part|
175
+ text = part.is_a?(String) ? part : ""
176
+ add_refs(refs, occurrences(text[0, MAX_SCAN])) unless text.empty?
177
+ end
178
+ end
145
179
 
146
180
  # Links the refs in the answer as shown: the model's text unless an
147
181
  # earlier hook changed it (then its offsets differ, so it is scanned
@@ -1,15 +1,15 @@
1
1
  ---
2
2
  name: source-links
3
- version: 0.5.1
3
+ version: 0.6.0
4
4
  scope: system
5
- description: Links source refs (JIRA tickets, GitHub issues, …) in the model's answer — inline links in the web, and a one-line note after the turn (note false turns it off); sources are configured patterns
5
+ description: Links source refs (JIRA tickets, GitHub issues, …) in the model's answer — inline links in the web, refs in a waiting question's reply, and a one-line note after the turn (note false turns it off); sources are configured patterns
6
6
  trust_level: reviewed
7
- requires_chi: ">= 0.61.0"
7
+ requires_chi: ">= 0.98.0"
8
8
  files:
9
9
  source_links.md: sha256:ace01a0f7c308a0ae54ece714c31c0e6d7a49c372d98111030e6d268320bed3a
10
10
  hooks:
11
11
  source_links.rb:
12
- sha256: sha256:c41249ec0bbb708ec7827e1d0f66689dfbbc415c1da5fe33f5d3a8ec8e5eaa2f
13
- event: after_turn
12
+ sha256: sha256:84c0bef064fb1ea76749a61e758eba612662aaf0f05f18a18ab4228497078fa2
13
+ events: [after_turn, question_asked]
14
14
  on_error: log
15
15
  priority: 90
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: samagotchi-system
3
- version: 0.97.0
3
+ version: 0.99.0
4
4
  scope: system
5
5
  description: Default system memories — identity, self map, config modification protocol, memory guide and the delegated-session rules
6
6
  files:
@@ -39,6 +39,7 @@ require_relative "llm/openai_chat"
39
39
  require_relative "llm/turn_settings"
40
40
  require_relative "session"
41
41
  require_relative "toolkit_binding"
42
+ require_relative "tool_loads"
42
43
  require_relative "answer_tail"
43
44
  require_relative "archive_store"
44
45
  require_relative "session_observer"
@@ -77,6 +78,7 @@ require_relative "answer_display"
77
78
  require_relative "answer_refs"
78
79
  require_relative "edit_preview"
79
80
  require_relative "question_desk"
81
+ require_relative "question_asked"
80
82
  require_relative "child_reports"
81
83
  require_relative "relay_desk"
82
84
  require_relative "guardrail_wiring"
@@ -276,6 +278,9 @@ module Samagotchi
276
278
  # The tools plugins publish without declaring (chi.tool_source), and
277
279
  # the search and call tools that reach them (Tools::ToolIndex).
278
280
  @tool_index = Tools::ToolIndex.new
281
+ # The tools plugins load (ctx.load_tools), staged until a turn's
282
+ # messages are built (#append_tool_loads).
283
+ @tool_loads = ToolLoads.new
279
284
  # The context sources the peer bundles declare as a peer agent's
280
285
  # answers (chi.peer_source); the worker's ContextAbsorber asks it.
281
286
  @peer_sources = PeerSources.new
@@ -364,7 +369,8 @@ module Samagotchi
364
369
  model_key: -> { @model_key },
365
370
  model_name: -> { guardrail_model_name },
366
371
  cancelled: -> { !!active_cancel_controller&.cancelled? },
367
- ask: ->(fields) { open_question(fields) },
372
+ # An approval of the model's call: :question_asked first (QuestionAsked).
373
+ ask: ->(fields) { open_question(question_asked(QuestionAsked::KIND_APPROVAL, fields)) },
368
374
  notify: ->(message) { guardrail_notify(message) }
369
375
  )
370
376
  end
@@ -1576,13 +1582,35 @@ module Samagotchi
1576
1582
 
1577
1583
  # Ask the model's question (ask_user_question): the kernel's
1578
1584
  # question_handler, called on the turn thread with the payload
1579
- # Tools::AskUserQuestion.validate made (QuestionDesk#request).
1585
+ # Tools::AskUserQuestion.validate made (QuestionDesk#request), after
1586
+ # the :question_asked hooks (QuestionAsked) gave the step its refs; a
1587
+ # --non-interactive run has no one to ask, so they don't fire there.
1580
1588
  # @param payload [Hash] {question:, options:, header:, multi_select:, allow_freeform:}
1581
1589
  # @return [String] normalized answer JSON
1582
1590
  def request_question(payload)
1591
+ payload = question_asked(QuestionAsked::KIND_QUESTION, payload) unless interface == :non_interactive
1583
1592
  @question_desk.request(payload)
1584
1593
  end
1585
1594
 
1595
+ # The :question_asked hooks for a question of the model's about to
1596
+ # open: the fields with the asking step's refs (QuestionAsked.fire).
1597
+ # @return [Hash]
1598
+ def question_asked(kind, fields)
1599
+ QuestionAsked.fire(hooks: @hooks, conversation: live_conversation, kind: kind, fields: fields,
1600
+ cwd: @session&.working_directory, synchronize: ->(&block) { synchronize_events(&block) },
1601
+ visible: ->(content) { question_step_text(content) })
1602
+ end
1603
+ private :question_asked
1604
+
1605
+ # A step's text as the asking slot shows it: the native loop's raw
1606
+ # response without its thoughts and tool-call bodies (a chat step has
1607
+ # neither), then the markup's tokens (OutputFormatter.strip_markup).
1608
+ def question_step_text(content)
1609
+ parser = @kernel.parser
1610
+ OutputFormatter.strip_markup(parser.strip_tool_calls(parser.strip_thought(content)))
1611
+ end
1612
+ private :question_step_text
1613
+
1586
1614
  # Open a question for the UIs and wait for its answer
1587
1615
  # (QuestionDesk#open_question: BLOCKS until answered or cancelled).
1588
1616
  # @param fields [Hash] question:, options:, header:, multi_select:, allow_freeform:, …
@@ -2091,12 +2119,13 @@ module Samagotchi
2091
2119
  })
2092
2120
  end
2093
2121
 
2094
- return if turn.continue
2095
-
2096
- user_message = { role: "user", content: turn.prompt, turn_id: turn.id }
2097
- user_message[:images] = image_refs unless image_refs.empty?
2098
- turn.messages << user_message
2099
- session.last_prompt = turn.prompt
2122
+ unless turn.continue
2123
+ user_message = { role: "user", content: turn.prompt, turn_id: turn.id }
2124
+ user_message[:images] = image_refs unless image_refs.empty?
2125
+ turn.messages << user_message
2126
+ session.last_prompt = turn.prompt
2127
+ end
2128
+ append_tool_loads(turn)
2100
2129
  end
2101
2130
  private :turn_messages
2102
2131
 
@@ -2113,8 +2142,13 @@ module Samagotchi
2113
2142
  # through the kernel without its #run: tag those dumps with this
2114
2143
  # turn's model, not the last native one.
2115
2144
  explained = model.llm_context_explained(@model_settings.llm_context_session_args)
2145
+ model_backend = backend(model)
2146
+ # A host that loads tools runs a direct call of one the session
2147
+ # reaches but doesn't declare (ToolRunner, KernelLoop#dispatch).
2148
+ loads_tools = model_backend.respond_to?(:loads_tools?) && model_backend.loads_tools?(bare_for_backend)
2116
2149
  @kernel.turn_settings = turn.settings.with(model_name: bare_for_backend, window_setting: model.window_setting,
2117
- llm_context: explained&.resolved, price: model.price)
2150
+ llm_context: explained&.resolved, price: model.price,
2151
+ loads_tools: loads_tools)
2118
2152
  log_llm_context(explained)
2119
2153
 
2120
2154
  turn.limit = @no_interrupt ? IterationLimit.for(no_interrupt: true) : max_iterations || IterationLimit.for
@@ -2122,7 +2156,7 @@ module Samagotchi
2122
2156
  # end keeps it, so a Continue resumes this run.
2123
2157
  run_start = turn.messages.length
2124
2158
  reasoning_since = resumed_run_start(turn, run_start)
2125
- result = backend(model).complete(
2159
+ result = model_backend.complete(
2126
2160
  messages: turn.messages,
2127
2161
  max_iterations: turn.limit,
2128
2162
  on_stream_event: build_stream_event_handler(turn.on_event, cancel_controller: turn.controller,
@@ -2531,6 +2565,52 @@ module Samagotchi
2531
2565
 
2532
2566
  public :tools, :toolkit_kits, :resolved_toolkit, :toolkit_explained, :toolkit_summary
2533
2567
 
2568
+ # A plugin's ctx.load_tools: stage the reachable tools +names+ names
2569
+ # (Tools::ToolIndex#deferred_name) that the session neither declares
2570
+ # nor loaded with an earlier tool change, for bundle +bundle+. A name
2571
+ # that names no tool at all is logged; a declared or removed one is
2572
+ # skipped quietly.
2573
+ # @return [Array<String>] the wire names staged
2574
+ def load_tools(names, bundle)
2575
+ added = ToolLoads.added(@session&.messages)
2576
+ wanted = names.filter_map do |name|
2577
+ wire = @tool_index.deferred_name(name)
2578
+ Log.info(:plugins, "load_tools_skipped", bundle: bundle, tool: name) unless wire || @tools.key?(name)
2579
+ wire
2580
+ end
2581
+ @tool_loads.stage(wanted - added, by: bundle)
2582
+ end
2583
+ public :load_tools
2584
+
2585
+ # The staged tool loads, appended to +turn+'s messages (after its user
2586
+ # message, or the trailing results and notes of a turn with none) as
2587
+ # one ToolLoads entry per bundle, when the turn's host takes tool
2588
+ # changes (ChatLoop#tool_changes?). Elsewhere they are dropped, logged.
2589
+ # A conversation that ends with a reply keeps them staged (a system
2590
+ # change must follow a user turn).
2591
+ def append_tool_loads(turn)
2592
+ return unless @tool_loads.any?
2593
+ return if turn.messages.last&.dig(:role).to_s == "model"
2594
+
2595
+ model = effective_model
2596
+ host = backend(model)
2597
+ staged = @tool_loads.take
2598
+ added = ToolLoads.added(turn.messages)
2599
+ unless host.respond_to?(:tool_changes?) && host.tool_changes?(bare_model_name(model.name))
2600
+ Log.info(:plugins, "tool_loads_not_sent", tools: staged.values.flatten.join(","), model: model.name.to_s)
2601
+ return
2602
+ end
2603
+
2604
+ staged.each do |bundle, names|
2605
+ names -= added
2606
+ next if names.empty?
2607
+
2608
+ turn.messages << ToolLoads.entry(names, by: bundle)
2609
+ hook_notify("tools loaded: #{names.join(", ")}", :info, bundle)
2610
+ end
2611
+ end
2612
+ private :append_tool_loads
2613
+
2534
2614
  # A plugin's ctx.use_toolkit (ToolkitBinding#use).
2535
2615
  # @return [Boolean] whether it was staged
2536
2616
  # @raise [ArgumentError] an unknown toolkit
@@ -2584,6 +2664,7 @@ module Samagotchi
2584
2664
  toolkit: -> { @toolkit_binding.name },
2585
2665
  declared_tools: -> { toolkit_explained.declared.map(&:name) },
2586
2666
  use_toolkit: ->(name) { use_toolkit(name) },
2667
+ load_tools: ->(names, bundle) { load_tools(names, bundle) },
2587
2668
  cancel_controller: ->(handler: false) { plugin_cancel_controller(handler: handler) },
2588
2669
  ask_model: lambda { |request, timeout:, max_tokens:, cancel_controller:, model: nil, source: nil|
2589
2670
  ask_side_model(request, timeout: timeout, max_tokens: max_tokens, cancel_controller: cancel_controller,
@@ -69,9 +69,9 @@ module Samagotchi
69
69
 
70
70
  hook = MemoryBundle::BundleHook.parse(raw_meta)
71
71
  # A hook with no declared event cannot be auto-registered.
72
- next if hook.event.empty?
72
+ next if hook.events.empty?
73
73
 
74
- fail_closed = hook.on_error == "fail_closed" && hook.event == "before_tool_call"
74
+ fail_closed = hook.events.any? { |event| hook.fail_closed?(event) }
75
75
  what = "hook #{basename} (bundle #{bundle_name})"
76
76
  file = hooks_dir && File.join(hooks_dir, basename)
77
77
  unless file && File.exist?(file)
@@ -98,9 +98,10 @@ module Samagotchi
98
98
  loaded
99
99
  end
100
100
 
101
- # Build one hook file and register it in +registry+ under its
102
- # event, priority and error policy (a fail_closed before_tool_call
103
- # one denies when it raises). No missing-file, sha or requires_chi
101
+ # Build one hook file and register it in +registry+ under each of
102
+ # its events (one instance for all: its state is shared), with its
103
+ # priority and error policy (a fail_closed before_tool_call one
104
+ # denies when it raises). No missing-file, sha or requires_chi
104
105
  # check: #load makes them; the test kit loads a bundle's source
105
106
  # this way.
106
107
  # @param hook [MemoryBundle::BundleHook] its manifest entry
@@ -109,17 +110,19 @@ module Samagotchi
109
110
  # @raise [Exception] what loading or building the file raised;
110
111
  # nothing is registered then
111
112
  def load_hook_file(bundle_name, basename, file, hook, registry, settings: {}, namespace: nil)
112
- event_sym = hook.event.to_sym
113
- fail_closed = hook.on_error == "fail_closed" && event_sym == :before_tool_call
114
113
  plugin = instantiate(bundle_name, basename, file, settings: settings, namespace: namespace)
115
- policy = if fail_closed then :deny
116
- elsif hook.on_error == "log" then :log
117
- else :skip
118
- end
119
- label = "#{"guardrail " if fail_closed}#{basename} (bundle #{bundle_name})"
120
- handler = Hooks.wrap(label: label, event: event_sym, policy: policy, log: [:hooks, "bundle_hook_failed"],
121
- fields: { bundle: bundle_name }) { |event| plugin.call(event) }
122
- registry.register_bundle(bundle_name, event_sym, hook_name: basename, priority: hook.priority, &handler)
114
+ hook.events.each do |event_name|
115
+ event_sym = event_name.to_sym
116
+ fail_closed = hook.fail_closed?(event_sym)
117
+ policy = if fail_closed then :deny
118
+ elsif hook.on_error == "log" then :log
119
+ else :skip
120
+ end
121
+ label = "#{"guardrail " if fail_closed}#{basename} (bundle #{bundle_name})"
122
+ handler = Hooks.wrap(label: label, event: event_sym, policy: policy, log: [:hooks, "bundle_hook_failed"],
123
+ fields: { bundle: bundle_name }) { |event| plugin.call(event) }
124
+ registry.register_bundle(bundle_name, event_sym, hook_name: basename, priority: hook.priority, &handler)
125
+ end
123
126
  nil
124
127
  end
125
128
 
@@ -24,12 +24,15 @@ module Samagotchi
24
24
  # :before_tool_call — before a tool is dispatched: vote with event[:guardrail].deny!/ask!
25
25
  # (or the older event[:blocked]=true + event[:block_reason]); the
26
26
  # event also has context: and targets: (docs/guardrails.md)
27
+ # :question_asked — a question of the model's (ask_user_question) or an approval of its
28
+ # call is about to open: event[:refs] attaches refs to the asking
29
+ # reply (QuestionAsked)
27
30
  # :after_tool_call — after tool execution, before result injection
28
31
  module Hooks
29
32
  # Every event a hook can be registered for, in a turn's order
30
33
  # (docs/hooks.md). Plugin::Api#on takes any symbol: `chi bundle check`
31
34
  # warns on one not here.
32
35
  EVENTS = %i[session_start before_turn before_generation generation_progress after_generation before_tool_call
33
- after_tool_call after_turn session_end].freeze
36
+ question_asked after_tool_call after_turn session_end].freeze
34
37
  end
35
38
  end
@@ -1139,6 +1139,14 @@ module Samagotchi
1139
1139
  # through call_tool (+via+) of a tool the toolkit declares is told to
1140
1140
  # call it directly: the model's call_tool(tool: "chi/read") with read
1141
1141
  # declared, or a hook's rewrite (a direct call can't carry via:).
1142
+ # The built-in +name+ names when the turn's host loads tools and the
1143
+ # toolkit reaches it (Registry#indexed): a direct call runs it.
1144
+ def loaded_entry(name)
1145
+ return nil unless @turn_settings&.loads_tools && @tools.respond_to?(:indexed)
1146
+
1147
+ @tools.indexed(name, layers: llm_context_layers)
1148
+ end
1149
+
1142
1150
  def unknown_tool_error(name, layers, via: nil)
1143
1151
  names = @tools.names(layers: layers)
1144
1152
  toolkit = @tools.toolkit&.name
@@ -1160,11 +1168,13 @@ module Samagotchi
1160
1168
 
1161
1169
  def dispatch(call)
1162
1170
  # A built-in reached through call_tool runs only while the toolkit
1163
- # reaches it that way (Registry#indexed); any other call by its name.
1171
+ # reaches it that way (Registry#indexed); any other call by its name,
1172
+ # on a host that loads tools (TurnSettings#loads_tools) a reachable
1173
+ # one too.
1164
1174
  entry = if call[:via]
1165
1175
  @tools.respond_to?(:indexed) ? @tools.indexed(call[:name], layers: llm_context_layers) : nil
1166
1176
  else
1167
- @tools.offered(call[:name], layers: llm_context_layers)
1177
+ @tools.offered(call[:name], layers: llm_context_layers) || loaded_entry(call[:name])
1168
1178
  end
1169
1179
  unless entry
1170
1180
  result = unknown_tool_error(call[:name], llm_context_layers, via: call[:via])
@@ -1198,6 +1208,9 @@ module Samagotchi
1198
1208
  dispatched[:ref_path] = result.ref_path
1199
1209
  dispatched[:ref_stderr_path] = result.stderr_path if result.stderr_path
1200
1210
  end
1211
+ # The tools a find_tools hit loads (Tools::ToolIndex::FoundText),
1212
+ # saved with a chat result (ToolResponse.single).
1213
+ dispatched[:loaded] = result.loaded if result.respond_to?(:loaded) && !Array(result.loaded).empty?
1201
1214
  if result.respond_to?(:images) && !Array(result.images).empty?
1202
1215
  dispatched[:images] = Array(result.images)
1203
1216
  dispatched[:image_only] = true if result.respond_to?(:image_only?) && result.image_only?