samagotchi 0.36.0 → 0.37.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1a970676ddf9dacd7bd644f66dcdf28f68a49c6cb24caf2bc785656cb6a2d286
4
- data.tar.gz: 8babf6d8562a213f07c3053b7efde79aae43ba4d51fef8ce16b933d17894acd0
3
+ metadata.gz: 15887ec5fe31be3f32d3e1bfa15b9cf5ca73b03e2101f6416eb2cfad4c2f477a
4
+ data.tar.gz: 2226f5343806c3171babc66d075fc57498af03d313f180458e4f6ee1801c7384
5
5
  SHA512:
6
- metadata.gz: 9727a14cff5153a0a061d9e787aebd0e0ea7494781e63386405b261fb4391d04bef56d4fefa346c9d0f2081d49fe7eddeadc3f6bb1b4ef8edc6478f5d273db6f
7
- data.tar.gz: 3dc9840ef7eba85ea3160bb2ac84412e9be6219c4e9bdbabc89796d2ee677adeb7903d4ef1002d82cae12daefac3ca6545f1f6519466bcfda164e2c126fd699e
6
+ metadata.gz: 8986ea963bda07138d6874b9024a031c5c6c000d03bad8281c9c24f57a079234fab5de68efe08305dd38ea4f138bf7ef7b02637e310e69f061182bb17d01d8a3
7
+ data.tar.gz: d9e07e7a5bacd114b8916a538fa2265bbd5b43ed78373fc306766455788b75ef5c1b0eb21d054288887483c93385063293150339abf4e3d02284bc1320cd9509
data/CHANGELOG.md CHANGED
@@ -8,6 +8,29 @@ and commands may change between minor versions. How releases are made:
8
8
 
9
9
  ## [Unreleased]
10
10
 
11
+ ## [0.37.0] - 2026-10-07
12
+
13
+ ### Added
14
+
15
+ - For plugin authors: a tool's `targets:` may return `acts_as:` (the tool a call stands for, like an MCP tool behind
16
+ a dispatcher), `args:` (the arguments it acts with) and `label:` (the name the approval question shows). A guardrail
17
+ rule's `tool:` then matches either name, and "allow this call" is keyed by both names and those arguments. It can't
18
+ name one of chi's own tools or another bundle's (dropped and logged). See [docs/plugins.md](docs/plugins.md#guardrails).
19
+ - `chi broadcast` asks a small triage model about the sessions no tag matched, instead of skipping them: it reads the
20
+ note and each session's scope card and answers yes or no ("model: yes (p 0.91)" when the host gives logprobs,
21
+ graded by `broadcast.threshold`). The model is `broadcast.triage_model` (with `triage_host_ref` or
22
+ `triage_base_url`), else the recap's, else `default.model`; a ~9B instruct model or larger works well. Up to
23
+ `broadcast.triage_parallel` (4) requests run at once, all within `broadcast.triage_deadline` (20 s): a session not
24
+ judged by then, or whose request failed, gets the note anyway, and the summary line counts it ("1 unchecked: triage
25
+ deadline"). A first line naming one project ("shopfront/checkout") keeps the note to that project's sessions.
26
+ `--dry-run` runs triage too. See [docs/broadcast.md](docs/broadcast.md#triage).
27
+ - `chi broadcast` prints the broadcast's id and logs each recipient's verdict (`~/.local/state/samagotchi/broadcast/log.jsonl`,
28
+ rotated at 2 MiB). `chi broadcast log [--last N] [--format json]` shows the last broadcasts and who got them, and
29
+ `chi broadcast deliver BROADCAST_ID ID...` gives one to sessions it skipped, after all. See
30
+ [docs/broadcast.md](docs/broadcast.md#the-triage-log).
31
+
32
+ Update with `chi update` (mcp 0.5.1).
33
+
11
34
  ## [0.36.0] - 2026-10-07
12
35
 
13
36
  ### Added
@@ -1782,7 +1805,8 @@ and long-lived sessions.
1782
1805
  - A macOS desktop helper (`chi desktop install`): a "Send to chi" Service and a
1783
1806
  hotkey panel that send selected text or the clipboard to your sessions.
1784
1807
 
1785
- [Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.36.0...HEAD
1808
+ [Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.37.0...HEAD
1809
+ [0.37.0]: https://github.com/dm1try/samagotchi/compare/v0.36.0...v0.37.0
1786
1810
  [0.36.0]: https://github.com/dm1try/samagotchi/compare/v0.35.0...v0.36.0
1787
1811
  [0.35.0]: https://github.com/dm1try/samagotchi/compare/v0.34.0...v0.35.0
1788
1812
  [0.34.0]: https://github.com/dm1try/samagotchi/compare/v0.33.0...v0.34.0
data/README.md CHANGED
@@ -120,7 +120,7 @@ chi web --open # web UI: this project's sessions (--s
120
120
  chi web --web-host lan # the web UI on your phone too: scan the QR code it prints
121
121
  chi sessions list # this project's saved sessions (--scope=all: every one)
122
122
  pbpaste | chi note --source slack <id> # background context for a session (no turn)
123
- chi broadcast -m "payments API is down" # the same for every session it concerns (a shared ticket, PR or link)
123
+ chi broadcast -m "payments API is down" # the same for every session it concerns (a shared tag, or a triage model says so)
124
124
  chi context add <PR URL> <id> # attached context: chi keeps it fresh, the agent hears when it changes
125
125
  pbpaste | chi send -m "same bug?" <id> # a message to a session, the clipboard quoted above it
126
126
  chi send --new --wait -m "review feat/x" # a new session you can watch in the web; prints the answer
@@ -203,7 +203,7 @@ once, for the session, for the repo, or for the whole rule in the repo.
203
203
  - [Memory](docs/memory.md): scopes and model-specific overlays
204
204
  - [Sessions](docs/sessions.md): storage, retention, `chi sessions`
205
205
  - [Attached context](docs/context.md): `chi context`, sources, scopes, waking, the github-pr bundle, safety
206
- - [Broadcast](docs/broadcast.md): `chi broadcast`, who a note reaches, tags, what a session gets
206
+ - [Broadcast](docs/broadcast.md): `chi broadcast`, who a note reaches, tags, triage, what a session gets, the triage log
207
207
  - [chi as a sub-agent](docs/sub-agent.md): `chi send --wait --format json`, `chi answer`, instructions for a parent agent
208
208
  - [Desktop helper](docs/desktop.md): `chi desktop`, the macOS "Send to chi" Service and hotkey
209
209
  - [Guardrails](docs/guardrails.md): allow / ask / deny for tool calls, rules, approvals
data/bin/chi CHANGED
@@ -97,7 +97,7 @@ parser = OptionParser.new do |opts|
97
97
  chi web [--port PORT] [--open] [--scope=all] the web UI
98
98
  chi sessions <list|stop|archive|unarchive|delete|prune|clean>
99
99
  chi note [--source NAME] [-m TEXT] ID...|--all background context for sessions
100
- chi broadcast [-m TEXT] [--all] [--dry-run] a note for every session it may concern
100
+ chi broadcast [-m TEXT] [--all] [--dry-run] | log | deliver a note for every session it may concern
101
101
  chi context <add|push|ls|show|refresh|rm|mute|unmute> live external text attached to sessions
102
102
  chi send [-m TEXT] ID...|--new a message into sessions, as if typed there
103
103
  chi answer ID --question QID --option N answer a question a session waits on
data/docs/broadcast.md CHANGED
@@ -12,6 +12,8 @@ chi broadcast -m "payments API returns 500 since 14:00 (PAY-123)"
12
12
  pbpaste | chi broadcast # the note from stdin
13
13
  chi broadcast --dry-run -m "…" # who would get it and why; nothing is delivered
14
14
  chi broadcast --all -m "deploy freeze until 18:00" # every recipient, tag or not
15
+ chi broadcast log # the last broadcasts and who got them
16
+ chi broadcast deliver b-7f3a1c9e c0ffee12 # one it skipped, after all
15
17
  ```
16
18
 
17
19
  To tell one session or a few you picked, use `chi note ID...` instead.
@@ -47,30 +49,69 @@ so `acme/shop#42` doesn't match `acme/other#42`; `PR #42` with no repo matches
47
49
  pull request 42 of any repo. Words the default ticket pattern would take that
48
50
  are no ticket (`UTF-8`, `SHA-256`, `ISO-8601`, `GPT-4`, …) are left out.
49
51
 
50
- A session that shares no tag is skipped for now, listed as "no tag match (not
51
- checked: no triage yet)": a small triage model that reads the note and each
52
- session's scope card is the next step. Until then, `--all` reaches them all.
52
+ A session that shares no tag goes to triage.
53
+
54
+ ## Triage
55
+
56
+ A small model reads the note and each other recipient's scope card (below)
57
+ and answers yes or no: would the agent working there want to know? A yes
58
+ delivers the note, a no skips the session ("model: no"). `--all` skips tags
59
+ and triage and reaches every recipient.
60
+
61
+ - **The model:** `broadcast.triage_model` (with `broadcast.triage_host_ref` or
62
+ `broadcast.triage_base_url`, as the `recap.*` settings take them); with none
63
+ of them set, the [recap's](sessions.md) model, else `default.model`. A ~9B
64
+ dense instruct model or larger works well (qwen3.5-9b missed no session in
65
+ our tests and sent about one extra note per broadcast of 20 sessions);
66
+ 4B-class models say yes to nearly everything. A local model keeps the
67
+ cards (your prompts, recaps) on your machine; a remote triage host is your
68
+ explicit choice. Thinking is off, and each answer is a word.
69
+ - **A score, when the host gives one:** with logprobs (OpenAI, OpenRouter
70
+ for some models), the verdict is P(yes) of the answer's first token, shown as
71
+ "model: yes (p 0.83)", and a session needs `broadcast.threshold` (0.5). A
72
+ higher threshold drops real "this blocks you" notes first. A host that
73
+ refuses logprobs (llama.cpp's, some gateways) is asked again without them:
74
+ the plain answer counts as 1 or 0.
75
+ - **A scope line:** when the note's first line names exactly one of the
76
+ recipients' projects as a word (`shopfront/checkout`, `infra / ci`), the
77
+ other projects' sessions, and sessions outside any project, are skipped:
78
+ "scope line names shopfront". Anything fuzzier ("for payments folks") is the
79
+ model's job; it sees the line too. A one-line note counts as its own first
80
+ line: with a project named after a common word (`web`, `docs`), "the web
81
+ composer docs moved" names it and keeps the note there.
82
+ - **Fail open:** at most `broadcast.triage_parallel` (4) requests run at a
83
+ time, all within `broadcast.triage_deadline` (20 s; a local server's first
84
+ request after a pause can take 10 s). A session not judged by then, one
85
+ whose request failed, or one the model answered with neither yes nor no
86
+ gets the note anyway, "unchecked: triage deadline", and the summary line
87
+ counts them: a missed session is worse than an extra note. Without a
88
+ triage model to ask (settings that don't resolve) every such session gets
89
+ it unchecked, and chi says why.
53
90
 
54
91
  ## What it prints
55
92
 
56
93
  One line per recipient, the ones that got it first, then a summary:
57
94
 
58
95
  ```
59
- broadcast "payments API returns 500 since 14:00 (PAY-123)"
96
+ broadcast b-7f3a1c9e "payments API returns 500 since 14:00 (PAY-123)"
60
97
  3f2a1c9e delivered ticket PAY-123 matches (branch)
61
98
  91ab02c4 delivered link notion.so/team/checkout-v2 matches (messages); waits for its next start
62
- c0ffee12 skipped no tag match (not checked: no triage yet)
99
+ 5e1f0a77 delivered model: yes (p 0.91)
100
+ 77aa0b3c delivered unchecked: triage deadline
101
+ c0ffee12 skipped model: no
63
102
  d00dad00 skipped open in a chi REPL
64
- delivered 2 · skipped 2
103
+ delivered 4 · skipped 2 · 1 unchecked: triage deadline
65
104
  ```
66
105
 
67
- A session a worker runs adds the note within a few seconds (after a running
106
+ The first line names the broadcast (`b-7f3a1c9e`): `chi broadcast log` and
107
+ `chi broadcast deliver` take it. A session a worker runs adds the note within a few seconds (after a running
68
108
  turn); one with no worker gets it at its next start ("waits for its next
69
109
  start"). Nothing starts a turn anywhere.
70
110
 
71
111
  `--dry-run` prints the note's tags, each recipient's verdict ("would get it" or
72
- "skipped") and its scope card: what chi knows about it from local state, and
73
- what a triage model will read.
112
+ "skipped"; triage runs) and its scope card: what chi knows about it from local
113
+ state, and what the triage model reads. A line before them names the triage model
114
+ and the setting it came from.
74
115
 
75
116
  ```
76
117
  3f2a1c9e would get it ticket PAY-123 matches (branch)
@@ -83,7 +124,8 @@ what a triage model will read.
83
124
  ```
84
125
 
85
126
  Exit status: 0 when it ran (skipped sessions included), 1 when it was refused,
86
- no session is active or a delivery failed, 2 for a usage error.
127
+ no session is active or a delivery failed, 2 for a usage error. `chi broadcast
128
+ deliver` exits 0 when every session named has the note now, 1 otherwise.
87
129
 
88
130
  ## What a session gets
89
131
 
@@ -96,7 +138,8 @@ payments API returns 500 since 14:00 (PAY-123)
96
138
  [END NOTE]
97
139
  ```
98
140
 
99
- With `--all` the line says "with every active session". The date is there
141
+ With `--all` the line says "with every active session"; one that triage
142
+ delivered says "with the sessions it may concern." and no reason. The date is there
100
143
  because a session with no worker may read it days later. The system prompt
101
144
  tells the model what a note from `broadcast` is: if it affects its current
102
145
  work, it says so briefly in its next answer, otherwise it ignores it, and it
@@ -107,9 +150,37 @@ plain note can't pass for a broadcast.
107
150
  The note and its line together are capped at 16 KiB like any note; the text
108
151
  may take up to 15872 bytes.
109
152
 
153
+ ## The triage log
154
+
155
+ Each broadcast (not a `--dry-run`) leaves one line in
156
+ `$XDG_STATE_HOME/samagotchi/broadcast/log.jsonl` (`~/.local/state/…`): its
157
+ id, time, your text, the note's tags and each recipient's verdict (result,
158
+ reason, P(yes), who decided, the session's tags). Scope cards are not kept:
159
+ they quote your prompts. Past 2 MiB the file becomes `log.jsonl.1`, one old
160
+ file kept. Sessions don't read it: a broadcast stays fire-and-forget.
161
+
162
+ ```sh
163
+ chi broadcast log # the last 5 broadcasts
164
+ chi broadcast log --last 20 --format json
165
+ ```
166
+
167
+ ```
168
+ b-7f3a1c9e 2026-10-06 14:20 "payments API returns 500 since 14:00 (PAY-123)"
169
+ 3f2a1c9e delivered ticket PAY-123 matches (branch)
170
+ c0ffee12 skipped model: no
171
+ delivered 1 · skipped 1
172
+ 2026-10-06 14:31 delivered anyway: c0ffee12
173
+ ```
174
+
175
+ When triage skipped a session that needed the note, `chi broadcast deliver
176
+ BROADCAST_ID ID...` (an id or the start of one, for both) gives it the logged
177
+ text, dated when it was shared and saying you passed it on. A session that
178
+ got it already is left alone. The log keeps it as a correction: labels for
179
+ tuning `broadcast.threshold`, or for a classifier later.
180
+
110
181
  ## For you, not for an agent
111
182
 
112
- `chi broadcast` refuses to run inside a chi session (where
183
+ `chi broadcast` (`log` and `deliver` too) refuses to run inside a chi session (where
113
184
  `SAMAGOTCHI_PARENT_SESSION` is set): "chi broadcast is for your user, not an
114
185
  agent". With the guardrails bundle, the `chi-broadcast` rule also asks before
115
186
  any shell command that runs it, once at a time, and only you may allow it
@@ -121,9 +192,17 @@ any shell command that runs it, once at a time, and only you may allow it
121
192
  |---|---|---|
122
193
  | `broadcast.active_hours` | `8` | A session with no worker or REPL counts when its last turn ended within this many hours. |
123
194
  | `broadcast.ticket_pattern` | `\b[A-Z][A-Z0-9]+-\d+\b` | A Ruby regex for ticket ids; an invalid one warns and the default is used. |
195
+ | `broadcast.triage_model` | recap's, else `default.model` | The triage model, a model ref (`box:qwen3.5-9b`). |
196
+ | `broadcast.triage_host_ref` | none | A `hosts:` name for it. |
197
+ | `broadcast.triage_base_url` | none | An OpenAI API base for it instead. |
198
+ | `broadcast.triage_parallel` | `4` | Triage requests at a time. |
199
+ | `broadcast.triage_deadline` | `20` | Seconds triage may take in all; the rest get the note unchecked. |
200
+ | `broadcast.threshold` | `0.5` | The P(yes) a session needs when the host gives logprobs. |
124
201
 
125
202
  ```yaml
126
203
  broadcast:
127
204
  active_hours: 12
128
205
  ticket_pattern: '\b(?:PAY|OPS)-\d+\b'
206
+ triage_model: box:qwen3.5-9b
207
+ triage_deadline: 30
129
208
  ```
@@ -742,7 +742,8 @@ One record is one line, plus indented payload lines at debug level:
742
742
  control characters (terminal colours in tool output) are escaped.
743
743
  - Tags: `turn` (a session's event trail), `http` (model requests),
744
744
  `worker`, `bridge`, `web`, `attached`, `repl`, `idle`, `recap`, `hooks`,
745
- `plugins`, `guardrails`, `config`, `memory`, `model` (debug dumps), `context` (attached context fetches).
745
+ `plugins`, `guardrails`, `config`, `memory`, `model` (debug dumps), `context` (attached context fetches),
746
+ `broadcast` (chi broadcast's triage).
746
747
  - The format is parsed by `Samagotchi::LogLine` (`parse`, `each_record`);
747
748
  keep tools that read it on that parser.
748
749
 
@@ -884,6 +885,12 @@ described in their own sections.
884
885
  | `recap.sentences` | `2-4` | yes | Recap length, `N` or `N-M` (1–10). |
885
886
  | `broadcast.active_hours` | `8` | | `chi broadcast` reaches a session a worker or a chi REPL runs, or whose last turn ended within this many hours. See [Broadcast](broadcast.md). |
886
887
  | `broadcast.ticket_pattern` | `\b[A-Z][A-Z0-9]+-\d+\b` | | A Ruby regex for ticket ids; one in the note and in a session's branch (any case) or prompts delivers the broadcast there. See [Broadcast](broadcast.md). |
888
+ | `broadcast.triage_model` | recap's | | The model that judges a recipient no tag matched (a model ref, as `recap.model`); unset: the recap's model, else `default.model`. A ~9B dense instruct model or larger; 4B-class ones say yes to nearly everything. See [Broadcast](broadcast.md#triage). |
889
+ | `broadcast.triage_host_ref` | none | | A `hosts:` name for the triage model. |
890
+ | `broadcast.triage_base_url` | none | | An OpenAI API base for the triage model instead (`http://h:8081/v1`). |
891
+ | `broadcast.triage_parallel` | `4` | | Triage requests at a time. |
892
+ | `broadcast.triage_deadline` | `20` | | Seconds triage may take in all; a recipient not judged by then gets the note unchecked. |
893
+ | `broadcast.threshold` | `0.5` | | The P(yes) a recipient needs, when the triage host gives logprobs; a plain yes or no is 1 or 0. |
887
894
  | `session.shared` | `true` | | Plain `chi` runs its session in a worker and attaches; `--no-shared` per run. |
888
895
  | `session.idle_exit_minutes` | `30` | yes | An unused worker exits after this; `0` = never. |
889
896
  | `session.keep_empty` | `false` | | Keep sessions nothing happened in. |
data/docs/guardrails.md CHANGED
@@ -292,6 +292,21 @@ A plugin tool whose `targets:` name no command or path (an MCP tool) is asked
292
292
  about with its arguments (`mcp_x_sum: a=20 b=22`), and an approval of "this
293
293
  call" is keyed by them.
294
294
 
295
+ A plugin tool that runs other tools for the model (a dispatcher such as
296
+ `mcp_call`) can say, in its `targets:`, which tool a call acts as (`acts_as:`)
297
+ and with which arguments (`args:`). A rule's `tool:` then matches the call's
298
+ own name **or** the tool it acts as: `tool: mcp_github_*` fires for an
299
+ `mcp_call` that acts as `mcp_github_merge_pull_request`, and `tool: mcp_call`
300
+ fires whatever it acts as. The question names the tool it acts as (by the
301
+ `label:` its `targets:` gives, if any: display only) and shows its arguments, and an approval of "this call" is keyed by the real name, the
302
+ alias and those arguments (`/guardrails` lists it as
303
+ `mcp_call>mcp_github_merge_pull_request:number=7 …`), so it never covers a
304
+ direct call of that tool, nor a direct call's approval it. A tool can't act as
305
+ one of chi's own tools or another bundle's tool: that `acts_as:` is dropped
306
+ (and logged as `plugin_acts_as_dropped`), and the call is matched by its own
307
+ name only. Hooks and loop-guard's `ignore_tools` still see the call's own
308
+ name.
309
+
295
310
  ```yaml
296
311
  - id: mcp-ask
297
312
  tool: "mcp_*" # every MCP tool (the mcp bundle's mcp_<server>_<tool>)
data/docs/plugins.md CHANGED
@@ -162,8 +162,8 @@ in the chat path's `tools:`, after chi's own tools.
162
162
  it raises, the default is shown. Both are saved with the call's result, so
163
163
  a web page reloaded later shows the same row: the web server doesn't run
164
164
  plugins.
165
- - `targets`: `->(args) { { paths: [...], command: "…", cwd: "…" } }`, each
166
- key optional, says what a call acts on, for [guardrails](#guardrails).
165
+ - `targets`: `->(args) { { paths: [...], command: "…", cwd: "…", acts_as: "…", args: {…}, label: "…" } }`,
166
+ each key optional, says what a call acts on, for [guardrails](#guardrails).
167
167
 
168
168
  The block returns the result text. If it starts with `Error:`, it counts as
169
169
  a failure. If it raises, the model gets `Error: <message>`. To return images
@@ -234,6 +234,27 @@ is what `targets:` says:
234
234
  paths (chi's config, …) match them.
235
235
  - `command:`: a shell command the call runs; `command:` rules match it.
236
236
  - `cwd:`: where it runs, for the repo root and relative paths.
237
+ - `acts_as:`: the tool a call stands for, when the tool runs other tools
238
+ (the mcp bundle's `mcp_call`). Rules keyed by that tool's name match the
239
+ call too, as well as rules on the tool's own name. It can't be one of
240
+ chi's own tools or another bundle's tool: that is dropped (and logged).
241
+ - `args:`: the arguments it acts with (a Hash), when they aren't the call's
242
+ own (a dispatcher's inner arguments). A call with no `command:` or
243
+ `paths:` is asked about with them, and "allow this call" is keyed by them.
244
+ - `label:`: what the approval question names the call by (`github: x`), in
245
+ place of the tool's `label` (or, with `acts_as:`, that tool's). Display
246
+ only: rules and approvals don't read it. One line of at most 80
247
+ characters, else it is ignored (and logged).
248
+
249
+ ```ruby
250
+ targets: ->(args) { { acts_as: "mcp_#{args["server"]}_#{args["tool"]}", args: args["args"],
251
+ label: "#{args["server"]}: #{args["tool"]}" } }
252
+ ```
253
+
254
+ An approval of an `acts_as:` call keys on the tool's own name as well
255
+ (`mcp_call>mcp_github_x:…`), so it never stands in for an approval of the
256
+ tool it acts as, nor that one for it. Hooks see the call's own name, and so
257
+ does loop-guard's `ignore_tools`.
237
258
 
238
259
  A tool without `targets:` is matched by its name only. A `targets:` that
239
260
  raises counts as nothing (it is logged). See [guardrails.md](guardrails.md).
@@ -0,0 +1,267 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative "../idle_client"
4
+ require_relative "../idle_target"
5
+ require_relative "../cancellation_controller"
6
+ require_relative "../log"
7
+ require_relative "tags"
8
+
9
+ module Samagotchi
10
+ module Broadcast
11
+ # Whether a broadcast concerns a recipient its tags didn't match: a
12
+ # small model reads the note and the session's ScopeCard and answers
13
+ # yes or no (Triage::LLM). Many recipients are judged in parallel under
14
+ # one deadline (Triage.judge_all); whatever isn't judged by then, or
15
+ # fails, is delivered unchecked (fail open: a missed recipient is worse
16
+ # than an extra note).
17
+ module Triage
18
+ # One recipient's verdict.
19
+ # @!attribute relevant [Boolean] it gets the note
20
+ # @!attribute p [Float, nil] the model's P(yes) from its logprobs,
21
+ # 1.0 or 0.0 from a plain answer, nil when no model judged it
22
+ # @!attribute reason [String] why, for the output and the log
23
+ # @!attribute by [String] who decided: "tags" (a shared tag),
24
+ # "model", "fallback" (delivered unchecked: deadline, error, an
25
+ # answer that is no yes or no), "scope" (the note's first line
26
+ # names another project), "all" (--all) or "repl" (a chi REPL owns
27
+ # it: no notes)
28
+ # @!attribute match [Match, nil] the shared tag (by "tags")
29
+ Verdict = Data.define(:relevant, :p, :reason, :by, :match) do
30
+ def initialize(match: nil, **) = super
31
+
32
+ def unchecked? = by == "fallback"
33
+ end
34
+
35
+ # The spike's prompt (plain yes/no beat JSON with a p and a reason
36
+ # for every model tried: writing a reason pushes towards "relevant").
37
+ SYSTEM = <<~TEXT
38
+ You route short notes that a developer broadcasts to their running coding-agent sessions.
39
+ You get one note and one session's card (project, branch, title, what the session was started for, its latest recap, the last request).
40
+ Decide whether the note may affect THIS session's current work: would the agent working on it want to know?
41
+ A note can start with an informal scope line (a project, a component, a team); use it as a hint.
42
+ Notes about things unrelated to the session's work (social, other projects, other components) are not relevant.
43
+ Answer with exactly one word: yes or no.
44
+ TEXT
45
+
46
+ DEFAULT_PARALLEL = 4
47
+ DEFAULT_DEADLINE = 20.0
48
+ DEFAULT_THRESHOLD = 0.5
49
+ # How long threads still asking get to stop once the deadline passed.
50
+ JOIN_GRACE = 1.0
51
+
52
+ module_function
53
+
54
+ # @return [Verdict] delivered unchecked, +why+ in the reason
55
+ def unchecked(why) = Verdict.new(relevant: true, p: nil, reason: "unchecked: #{why}", by: "fallback")
56
+
57
+ # The messages a model judges +note+ for +card+ by.
58
+ def messages(note, card)
59
+ [{ role: "system", content: SYSTEM }, { role: "user", content: "NOTE:\n#{note}\n\nSESSION CARD:\n#{card}" }]
60
+ end
61
+
62
+ # The one project the note's first line names, as a whole word (any
63
+ # case), among +projects+ (the recipients' project names); nil when it
64
+ # names none or several. A scope line ("shopfront/checkout", "agentd
65
+ # web") then keeps the note to that project's sessions; anything
66
+ # fuzzier is the model's job.
67
+ # @param projects [Array<String>]
68
+ # @return [String, nil]
69
+ def scope_project(note, projects)
70
+ line = note.to_s.lines.first.to_s.downcase
71
+ named = projects.compact.uniq.select do |project|
72
+ line.match?(/(?<![[:alnum:]_-])#{Regexp.escape(project.downcase)}(?![[:alnum:]_-])/)
73
+ end
74
+ named.one? ? named.first : nil
75
+ end
76
+
77
+ # Every card's verdict: a shared tag delivers (Tags.match); a note
78
+ # whose first line names one of the projects (scope_project) skips
79
+ # the other projects' sessions; the model judges the rest
80
+ # (judge_all, +judge+ its keywords).
81
+ # @param cards [Array<ScopeCard>]
82
+ # @param note_tags [Array<Tag>]
83
+ # @return [Hash{String => Verdict}] by card id, in +cards+' order
84
+ def verdicts(note, cards, note_tags:, new_backend:, **judge)
85
+ scope = scope_project(note, cards.map(&:project))
86
+ decided = cards.to_h do |card|
87
+ match = Tags.match(note_tags, card.tags)
88
+ verdict = if match
89
+ Verdict.new(relevant: true, p: nil, reason: match.reason, by: "tags", match: match)
90
+ elsif scope && card.project != scope
91
+ Verdict.new(relevant: false, p: nil, reason: "scope line names #{scope}", by: "scope")
92
+ end
93
+ [card.id, verdict]
94
+ end
95
+ judged = judge_all(note, cards.reject { |card| decided[card.id] }, new_backend: new_backend, **judge)
96
+ cards.to_h { |card| [card.id, decided[card.id] || judged.fetch(card.id)] }
97
+ end
98
+
99
+ # Judges +cards+ with at most +parallel+ backends at a time, one per
100
+ # thread (a backend keeps per-host state), all under +deadline+
101
+ # seconds. A card not judged by then is delivered unchecked; the
102
+ # requests still running are cancelled.
103
+ # @param new_backend [#call] (CancellationController) → a backend:
104
+ # #judge(note, card) → Verdict
105
+ # @param cards [Array<ScopeCard>]
106
+ # @return [Hash{String => Verdict}] by card id
107
+ def judge_all(note, cards, new_backend:, parallel: DEFAULT_PARALLEL, deadline: DEFAULT_DEADLINE,
108
+ clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) })
109
+ return {} if cards.empty?
110
+
111
+ queue = Queue.new
112
+ cards.each { |card| queue << card }
113
+ queue.close
114
+ results = {}
115
+ lock = Mutex.new
116
+ changed = ConditionVariable.new
117
+ cancel = CancellationController.new
118
+ ends = clock.call + deadline.to_f
119
+ finished = 0
120
+ threads = Array.new(parallel.to_i.clamp(1, cards.size)) do
121
+ Thread.new do
122
+ card = nil
123
+ backend = new_backend.call(cancel)
124
+ while (card = queue.pop)
125
+ verdict = judge_one(backend, note, card)
126
+ lock.synchronize do
127
+ results[card.id] = verdict unless cancel.cancelled?
128
+ changed.broadcast
129
+ end
130
+ card = nil
131
+ end
132
+ rescue StandardError => e
133
+ # No backend (or a bug around it): this thread's card and the
134
+ # ones it takes from here on are delivered unchecked at once,
135
+ # so nobody waits for the deadline.
136
+ Log.exception(:broadcast, "triage_thread_failed", e)
137
+ failed = unchecked("triage failed (#{e.class})")
138
+ lock.synchronize { results[card.id] ||= failed if card }
139
+ while (card = queue.pop)
140
+ lock.synchronize { results[card.id] ||= failed }
141
+ end
142
+ ensure
143
+ lock.synchronize do
144
+ finished += 1
145
+ changed.broadcast
146
+ end
147
+ end
148
+ end
149
+ lock.synchronize do
150
+ loop do
151
+ break if results.size == cards.size || finished == threads.size
152
+
153
+ left = ends - clock.call
154
+ break if left <= 0
155
+
156
+ changed.wait(lock, left)
157
+ end
158
+ cancel.cancel!(:deadline)
159
+ queue.clear
160
+ end
161
+ threads.each { |thread| thread.join(JOIN_GRACE) || thread.kill }
162
+ cards.to_h { |card| [card.id, results[card.id] || unchecked("triage deadline")] }
163
+ end
164
+
165
+ def judge_one(backend, note, card)
166
+ backend.judge(note, card)
167
+ rescue Samagotchi::LLM::RequestCancelled
168
+ unchecked("triage deadline")
169
+ rescue StandardError => e
170
+ Log.exception(:broadcast, "triage_failed", e)
171
+ unchecked("triage failed (#{e.class})")
172
+ end
173
+ private_class_method :judge_one
174
+
175
+ # A triage model on an OpenAI-compatible host (an IdleTarget), asked
176
+ # through its own IdleClient: thinking off, a few tokens, the first
177
+ # token's logprobs when the host gives them. P(yes) / (P(yes) +
178
+ # P(no)) from those is graded against +threshold+; a plain yes or no
179
+ # is 1.0 or 0.0.
180
+ class LLM
181
+ MAX_TOKENS = 16
182
+ TOP_LOGPROBS = 5
183
+ LOGPROB_OPTIONS = { logprobs: true, top_logprobs: TOP_LOGPROBS }.freeze
184
+ # Hosts (base urls) that refused logprobs with a 400: asked without
185
+ # them from then on, in this process.
186
+ @no_logprobs = Set.new
187
+ @no_logprobs_lock = Mutex.new
188
+
189
+ class << self
190
+ def logprobs?(base_url) = @no_logprobs_lock.synchronize { !@no_logprobs.include?(base_url) }
191
+
192
+ def no_logprobs!(base_url) = @no_logprobs_lock.synchronize { @no_logprobs << base_url }
193
+
194
+ # Specs: forget the hosts that refused.
195
+ def reset_logprobs! = @no_logprobs_lock.synchronize { @no_logprobs.clear }
196
+ end
197
+
198
+ # @param target [IdleTarget]
199
+ # @param timeout [Numeric] each request's
200
+ # @param client [IdleClient, nil] specs inject one
201
+ def initialize(target:, timeout:, threshold: DEFAULT_THRESHOLD, cancel_controller: nil, client: nil)
202
+ @target = target
203
+ @threshold = threshold.to_f
204
+ @cancel = cancel_controller
205
+ @client = client || IdleClient.for(target, timeout: timeout, purpose: "broadcast")
206
+ end
207
+
208
+ # @return [Verdict]
209
+ # @raise [Samagotchi::LLM::RequestCancelled] the deadline passed
210
+ def judge(note, card)
211
+ answer = ask(Triage.messages(note, card))
212
+ word = self.class.word(answer.text)
213
+ return Triage.unchecked("the model answered #{answer.text.to_s.strip[0, 40].inspect}") unless word
214
+
215
+ p = self.class.logprob_p(answer.top_logprobs, word: word)
216
+ relevant = p ? p >= @threshold : word == "yes"
217
+ reason = "model: #{relevant ? "yes" : "no"}#{format(" (p %.2f)", p) if p}"
218
+ Verdict.new(relevant: relevant, p: p || (word == "yes" ? 1.0 : 0.0), reason: reason, by: "model")
219
+ rescue IdleClient::SummarizeError => e
220
+ Triage.unchecked("the model failed (#{e.message[0, 120]})")
221
+ end
222
+
223
+ # "yes" or "no": the answer's first such word (thinking left out),
224
+ # nil when it has neither.
225
+ def self.word(text)
226
+ text.to_s.gsub(%r{<think>.*?</think>}m, "").strip.downcase[/\A\W*(yes|no)\b/, 1]
227
+ end
228
+
229
+ # P(yes) / (P(yes) + P(no)) over the first token's alternatives
230
+ # ("Yes", " yes" count as yes), when that token is the answer +word+
231
+ # itself: its likeliest alternative is yes or no and agrees with
232
+ # +word+. nil otherwise (an answer starting "**", "<think>" or a
233
+ # quote: those alternatives say nothing about yes or no).
234
+ # @param top [Array<Samagotchi::LLM::TokenLogprob>]
235
+ # @return [Float, nil]
236
+ def self.logprob_p(top, word:)
237
+ first = top.max_by(&:logprob)
238
+ return nil unless first && first.token.strip.downcase == word
239
+
240
+ sum = ->(want) { top.select { |t| t.token.strip.downcase == want }.sum { |t| Math.exp(t.logprob) } }
241
+ yes = sum.call("yes")
242
+ no = sum.call("no")
243
+ (yes / (yes + no)).round(4)
244
+ end
245
+
246
+ private
247
+
248
+ def ask(messages)
249
+ base_url = @target.base_url
250
+ options = self.class.logprobs?(base_url) ? LOGPROB_OPTIONS : {}
251
+ begin
252
+ @client.ask(messages, max_tokens: MAX_TOKENS, cancel_controller: @cancel, kind: "broadcast", options: options)
253
+ rescue IdleClient::SummarizeError => e
254
+ raise unless !options.empty? && e.cause.is_a?(Samagotchi::LLM::BadRequest)
255
+
256
+ # The host won't give logprobs (a server that refuses "n and
257
+ # logprobs"): ask again without, and leave them out from now on.
258
+ self.class.no_logprobs!(base_url)
259
+ Log.info(:broadcast, "logprobs_refused", model: @target.label, detail: e.cause.message.to_s[0, 200])
260
+ options = {}
261
+ retry
262
+ end
263
+ end
264
+ end
265
+ end
266
+ end
267
+ end