samagotchi 0.36.0 → 0.37.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +25 -1
- data/README.md +2 -2
- data/bin/chi +1 -1
- data/docs/broadcast.md +91 -12
- data/docs/configuration.md +8 -1
- data/docs/guardrails.md +15 -0
- data/docs/plugins.md +23 -2
- data/lib/samagotchi/broadcast/triage.rb +267 -0
- data/lib/samagotchi/broadcast/triage_log.rb +165 -0
- data/lib/samagotchi/broadcast/triage_model.rb +56 -0
- data/lib/samagotchi/broadcast_command.rb +273 -69
- data/lib/samagotchi/bundles/mcp/manifest.yml +2 -2
- data/lib/samagotchi/bundles/mcp/plugin.rb +35 -23
- data/lib/samagotchi/bundles/system/manifest.yml +1 -1
- data/lib/samagotchi/cli/command.rb +27 -3
- data/lib/samagotchi/config.rb +9 -0
- data/lib/samagotchi/context_command.rb +0 -17
- data/lib/samagotchi/engine.rb +4 -6
- data/lib/samagotchi/guardrail_wiring.rb +7 -2
- data/lib/samagotchi/guardrails/approval.rb +5 -2
- data/lib/samagotchi/guardrails/approvals.rb +9 -5
- data/lib/samagotchi/guardrails/rules.rb +4 -2
- data/lib/samagotchi/guardrails/targets.rb +76 -11
- data/lib/samagotchi/idle_client.rb +48 -28
- data/lib/samagotchi/idle_recap.rb +6 -5
- data/lib/samagotchi/idle_target.rb +95 -0
- data/lib/samagotchi/llm/openai_chat.rb +29 -3
- data/lib/samagotchi/log_line.rb +1 -1
- data/lib/samagotchi/note_command.rb +0 -21
- data/lib/samagotchi/recap_setup.rb +38 -52
- data/lib/samagotchi/send_command.rb +0 -22
- data/lib/samagotchi/version.rb +1 -1
- data/lib/samagotchi/web/public/question_card.js +4 -3
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 15887ec5fe31be3f32d3e1bfa15b9cf5ca73b03e2101f6416eb2cfad4c2f477a
|
|
4
|
+
data.tar.gz: 2226f5343806c3171babc66d075fc57498af03d313f180458e4f6ee1801c7384
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8986ea963bda07138d6874b9024a031c5c6c000d03bad8281c9c24f57a079234fab5de68efe08305dd38ea4f138bf7ef7b02637e310e69f061182bb17d01d8a3
|
|
7
|
+
data.tar.gz: d9e07e7a5bacd114b8916a538fa2265bbd5b43ed78373fc306766455788b75ef5c1b0eb21d054288887483c93385063293150339abf4e3d02284bc1320cd9509
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,29 @@ and commands may change between minor versions. How releases are made:
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [0.37.0] - 2026-10-07
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- For plugin authors: a tool's `targets:` may return `acts_as:` (the tool a call stands for, like an MCP tool behind
|
|
16
|
+
a dispatcher), `args:` (the arguments it acts with) and `label:` (the name the approval question shows). A guardrail
|
|
17
|
+
rule's `tool:` then matches either name, and "allow this call" is keyed by both names and those arguments. It can't
|
|
18
|
+
name one of chi's own tools or another bundle's (dropped and logged). See [docs/plugins.md](docs/plugins.md#guardrails).
|
|
19
|
+
- `chi broadcast` asks a small triage model about the sessions no tag matched, instead of skipping them: it reads the
|
|
20
|
+
note and each session's scope card and answers yes or no ("model: yes (p 0.91)" when the host gives logprobs,
|
|
21
|
+
graded by `broadcast.threshold`). The model is `broadcast.triage_model` (with `triage_host_ref` or
|
|
22
|
+
`triage_base_url`), else the recap's, else `default.model`; a ~9B instruct model or larger works well. Up to
|
|
23
|
+
`broadcast.triage_parallel` (4) requests run at once, all within `broadcast.triage_deadline` (20 s): a session not
|
|
24
|
+
judged by then, or whose request failed, gets the note anyway, and the summary line counts it ("1 unchecked: triage
|
|
25
|
+
deadline"). A first line naming one project ("shopfront/checkout") keeps the note to that project's sessions.
|
|
26
|
+
`--dry-run` runs triage too. See [docs/broadcast.md](docs/broadcast.md#triage).
|
|
27
|
+
- `chi broadcast` prints the broadcast's id and logs each recipient's verdict (`~/.local/state/samagotchi/broadcast/log.jsonl`,
|
|
28
|
+
rotated at 2 MiB). `chi broadcast log [--last N] [--format json]` shows the last broadcasts and who got them, and
|
|
29
|
+
`chi broadcast deliver BROADCAST_ID ID...` gives one to sessions it skipped, after all. See
|
|
30
|
+
[docs/broadcast.md](docs/broadcast.md#the-triage-log).
|
|
31
|
+
|
|
32
|
+
Update with `chi update` (mcp 0.5.1).
|
|
33
|
+
|
|
11
34
|
## [0.36.0] - 2026-10-07
|
|
12
35
|
|
|
13
36
|
### Added
|
|
@@ -1782,7 +1805,8 @@ and long-lived sessions.
|
|
|
1782
1805
|
- A macOS desktop helper (`chi desktop install`): a "Send to chi" Service and a
|
|
1783
1806
|
hotkey panel that send selected text or the clipboard to your sessions.
|
|
1784
1807
|
|
|
1785
|
-
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.
|
|
1808
|
+
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.37.0...HEAD
|
|
1809
|
+
[0.37.0]: https://github.com/dm1try/samagotchi/compare/v0.36.0...v0.37.0
|
|
1786
1810
|
[0.36.0]: https://github.com/dm1try/samagotchi/compare/v0.35.0...v0.36.0
|
|
1787
1811
|
[0.35.0]: https://github.com/dm1try/samagotchi/compare/v0.34.0...v0.35.0
|
|
1788
1812
|
[0.34.0]: https://github.com/dm1try/samagotchi/compare/v0.33.0...v0.34.0
|
data/README.md
CHANGED
|
@@ -120,7 +120,7 @@ chi web --open # web UI: this project's sessions (--s
|
|
|
120
120
|
chi web --web-host lan # the web UI on your phone too: scan the QR code it prints
|
|
121
121
|
chi sessions list # this project's saved sessions (--scope=all: every one)
|
|
122
122
|
pbpaste | chi note --source slack <id> # background context for a session (no turn)
|
|
123
|
-
chi broadcast -m "payments API is down" # the same for every session it concerns (a shared
|
|
123
|
+
chi broadcast -m "payments API is down" # the same for every session it concerns (a shared tag, or a triage model says so)
|
|
124
124
|
chi context add <PR URL> <id> # attached context: chi keeps it fresh, the agent hears when it changes
|
|
125
125
|
pbpaste | chi send -m "same bug?" <id> # a message to a session, the clipboard quoted above it
|
|
126
126
|
chi send --new --wait -m "review feat/x" # a new session you can watch in the web; prints the answer
|
|
@@ -203,7 +203,7 @@ once, for the session, for the repo, or for the whole rule in the repo.
|
|
|
203
203
|
- [Memory](docs/memory.md): scopes and model-specific overlays
|
|
204
204
|
- [Sessions](docs/sessions.md): storage, retention, `chi sessions`
|
|
205
205
|
- [Attached context](docs/context.md): `chi context`, sources, scopes, waking, the github-pr bundle, safety
|
|
206
|
-
- [Broadcast](docs/broadcast.md): `chi broadcast`, who a note reaches, tags, what a session gets
|
|
206
|
+
- [Broadcast](docs/broadcast.md): `chi broadcast`, who a note reaches, tags, triage, what a session gets, the triage log
|
|
207
207
|
- [chi as a sub-agent](docs/sub-agent.md): `chi send --wait --format json`, `chi answer`, instructions for a parent agent
|
|
208
208
|
- [Desktop helper](docs/desktop.md): `chi desktop`, the macOS "Send to chi" Service and hotkey
|
|
209
209
|
- [Guardrails](docs/guardrails.md): allow / ask / deny for tool calls, rules, approvals
|
data/bin/chi
CHANGED
|
@@ -97,7 +97,7 @@ parser = OptionParser.new do |opts|
|
|
|
97
97
|
chi web [--port PORT] [--open] [--scope=all] the web UI
|
|
98
98
|
chi sessions <list|stop|archive|unarchive|delete|prune|clean>
|
|
99
99
|
chi note [--source NAME] [-m TEXT] ID...|--all background context for sessions
|
|
100
|
-
chi broadcast [-m TEXT] [--all] [--dry-run] a note for every session it may concern
|
|
100
|
+
chi broadcast [-m TEXT] [--all] [--dry-run] | log | deliver a note for every session it may concern
|
|
101
101
|
chi context <add|push|ls|show|refresh|rm|mute|unmute> live external text attached to sessions
|
|
102
102
|
chi send [-m TEXT] ID...|--new a message into sessions, as if typed there
|
|
103
103
|
chi answer ID --question QID --option N answer a question a session waits on
|
data/docs/broadcast.md
CHANGED
|
@@ -12,6 +12,8 @@ chi broadcast -m "payments API returns 500 since 14:00 (PAY-123)"
|
|
|
12
12
|
pbpaste | chi broadcast # the note from stdin
|
|
13
13
|
chi broadcast --dry-run -m "…" # who would get it and why; nothing is delivered
|
|
14
14
|
chi broadcast --all -m "deploy freeze until 18:00" # every recipient, tag or not
|
|
15
|
+
chi broadcast log # the last broadcasts and who got them
|
|
16
|
+
chi broadcast deliver b-7f3a1c9e c0ffee12 # one it skipped, after all
|
|
15
17
|
```
|
|
16
18
|
|
|
17
19
|
To tell one session or a few you picked, use `chi note ID...` instead.
|
|
@@ -47,30 +49,69 @@ so `acme/shop#42` doesn't match `acme/other#42`; `PR #42` with no repo matches
|
|
|
47
49
|
pull request 42 of any repo. Words the default ticket pattern would take that
|
|
48
50
|
are no ticket (`UTF-8`, `SHA-256`, `ISO-8601`, `GPT-4`, …) are left out.
|
|
49
51
|
|
|
50
|
-
A session that shares no tag
|
|
51
|
-
|
|
52
|
-
|
|
52
|
+
A session that shares no tag goes to triage.
|
|
53
|
+
|
|
54
|
+
## Triage
|
|
55
|
+
|
|
56
|
+
A small model reads the note and each other recipient's scope card (below)
|
|
57
|
+
and answers yes or no: would the agent working there want to know? A yes
|
|
58
|
+
delivers the note, a no skips the session ("model: no"). `--all` skips tags
|
|
59
|
+
and triage and reaches every recipient.
|
|
60
|
+
|
|
61
|
+
- **The model:** `broadcast.triage_model` (with `broadcast.triage_host_ref` or
|
|
62
|
+
`broadcast.triage_base_url`, as the `recap.*` settings take them); with none
|
|
63
|
+
of them set, the [recap's](sessions.md) model, else `default.model`. A ~9B
|
|
64
|
+
dense instruct model or larger works well (qwen3.5-9b missed no session in
|
|
65
|
+
our tests and sent about one extra note per broadcast of 20 sessions);
|
|
66
|
+
4B-class models say yes to nearly everything. A local model keeps the
|
|
67
|
+
cards (your prompts, recaps) on your machine; a remote triage host is your
|
|
68
|
+
explicit choice. Thinking is off, and each answer is a word.
|
|
69
|
+
- **A score, when the host gives one:** with logprobs (OpenAI, OpenRouter
|
|
70
|
+
for some models), the verdict is P(yes) of the answer's first token, shown as
|
|
71
|
+
"model: yes (p 0.83)", and a session needs `broadcast.threshold` (0.5). A
|
|
72
|
+
higher threshold drops real "this blocks you" notes first. A host that
|
|
73
|
+
refuses logprobs (llama.cpp's, some gateways) is asked again without them:
|
|
74
|
+
the plain answer counts as 1 or 0.
|
|
75
|
+
- **A scope line:** when the note's first line names exactly one of the
|
|
76
|
+
recipients' projects as a word (`shopfront/checkout`, `infra / ci`), the
|
|
77
|
+
other projects' sessions, and sessions outside any project, are skipped:
|
|
78
|
+
"scope line names shopfront". Anything fuzzier ("for payments folks") is the
|
|
79
|
+
model's job; it sees the line too. A one-line note counts as its own first
|
|
80
|
+
line: with a project named after a common word (`web`, `docs`), "the web
|
|
81
|
+
composer docs moved" names it and keeps the note there.
|
|
82
|
+
- **Fail open:** at most `broadcast.triage_parallel` (4) requests run at a
|
|
83
|
+
time, all within `broadcast.triage_deadline` (20 s; a local server's first
|
|
84
|
+
request after a pause can take 10 s). A session not judged by then, one
|
|
85
|
+
whose request failed, or one the model answered with neither yes nor no
|
|
86
|
+
gets the note anyway, "unchecked: triage deadline", and the summary line
|
|
87
|
+
counts them: a missed session is worse than an extra note. Without a
|
|
88
|
+
triage model to ask (settings that don't resolve) every such session gets
|
|
89
|
+
it unchecked, and chi says why.
|
|
53
90
|
|
|
54
91
|
## What it prints
|
|
55
92
|
|
|
56
93
|
One line per recipient, the ones that got it first, then a summary:
|
|
57
94
|
|
|
58
95
|
```
|
|
59
|
-
broadcast "payments API returns 500 since 14:00 (PAY-123)"
|
|
96
|
+
broadcast b-7f3a1c9e "payments API returns 500 since 14:00 (PAY-123)"
|
|
60
97
|
3f2a1c9e delivered ticket PAY-123 matches (branch)
|
|
61
98
|
91ab02c4 delivered link notion.so/team/checkout-v2 matches (messages); waits for its next start
|
|
62
|
-
|
|
99
|
+
5e1f0a77 delivered model: yes (p 0.91)
|
|
100
|
+
77aa0b3c delivered unchecked: triage deadline
|
|
101
|
+
c0ffee12 skipped model: no
|
|
63
102
|
d00dad00 skipped open in a chi REPL
|
|
64
|
-
delivered
|
|
103
|
+
delivered 4 · skipped 2 · 1 unchecked: triage deadline
|
|
65
104
|
```
|
|
66
105
|
|
|
67
|
-
|
|
106
|
+
The first line names the broadcast (`b-7f3a1c9e`): `chi broadcast log` and
|
|
107
|
+
`chi broadcast deliver` take it. A session a worker runs adds the note within a few seconds (after a running
|
|
68
108
|
turn); one with no worker gets it at its next start ("waits for its next
|
|
69
109
|
start"). Nothing starts a turn anywhere.
|
|
70
110
|
|
|
71
111
|
`--dry-run` prints the note's tags, each recipient's verdict ("would get it" or
|
|
72
|
-
"skipped") and its scope card: what chi knows about it from local
|
|
73
|
-
what
|
|
112
|
+
"skipped"; triage runs) and its scope card: what chi knows about it from local
|
|
113
|
+
state, and what the triage model reads. A line before them names the triage model
|
|
114
|
+
and the setting it came from.
|
|
74
115
|
|
|
75
116
|
```
|
|
76
117
|
3f2a1c9e would get it ticket PAY-123 matches (branch)
|
|
@@ -83,7 +124,8 @@ what a triage model will read.
|
|
|
83
124
|
```
|
|
84
125
|
|
|
85
126
|
Exit status: 0 when it ran (skipped sessions included), 1 when it was refused,
|
|
86
|
-
no session is active or a delivery failed, 2 for a usage error.
|
|
127
|
+
no session is active or a delivery failed, 2 for a usage error. `chi broadcast
|
|
128
|
+
deliver` exits 0 when every session named has the note now, 1 otherwise.
|
|
87
129
|
|
|
88
130
|
## What a session gets
|
|
89
131
|
|
|
@@ -96,7 +138,8 @@ payments API returns 500 since 14:00 (PAY-123)
|
|
|
96
138
|
[END NOTE]
|
|
97
139
|
```
|
|
98
140
|
|
|
99
|
-
With `--all` the line says "with every active session"
|
|
141
|
+
With `--all` the line says "with every active session"; one that triage
|
|
142
|
+
delivered says "with the sessions it may concern." and no reason. The date is there
|
|
100
143
|
because a session with no worker may read it days later. The system prompt
|
|
101
144
|
tells the model what a note from `broadcast` is: if it affects its current
|
|
102
145
|
work, it says so briefly in its next answer, otherwise it ignores it, and it
|
|
@@ -107,9 +150,37 @@ plain note can't pass for a broadcast.
|
|
|
107
150
|
The note and its line together are capped at 16 KiB like any note; the text
|
|
108
151
|
may take up to 15872 bytes.
|
|
109
152
|
|
|
153
|
+
## The triage log
|
|
154
|
+
|
|
155
|
+
Each broadcast (not a `--dry-run`) leaves one line in
|
|
156
|
+
`$XDG_STATE_HOME/samagotchi/broadcast/log.jsonl` (`~/.local/state/…`): its
|
|
157
|
+
id, time, your text, the note's tags and each recipient's verdict (result,
|
|
158
|
+
reason, P(yes), who decided, the session's tags). Scope cards are not kept:
|
|
159
|
+
they quote your prompts. Past 2 MiB the file becomes `log.jsonl.1`, one old
|
|
160
|
+
file kept. Sessions don't read it: a broadcast stays fire-and-forget.
|
|
161
|
+
|
|
162
|
+
```sh
|
|
163
|
+
chi broadcast log # the last 5 broadcasts
|
|
164
|
+
chi broadcast log --last 20 --format json
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
```
|
|
168
|
+
b-7f3a1c9e 2026-10-06 14:20 "payments API returns 500 since 14:00 (PAY-123)"
|
|
169
|
+
3f2a1c9e delivered ticket PAY-123 matches (branch)
|
|
170
|
+
c0ffee12 skipped model: no
|
|
171
|
+
delivered 1 · skipped 1
|
|
172
|
+
2026-10-06 14:31 delivered anyway: c0ffee12
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
When triage skipped a session that needed the note, `chi broadcast deliver
|
|
176
|
+
BROADCAST_ID ID...` (an id or the start of one, for both) gives it the logged
|
|
177
|
+
text, dated when it was shared and saying you passed it on. A session that
|
|
178
|
+
got it already is left alone. The log keeps it as a correction: labels for
|
|
179
|
+
tuning `broadcast.threshold`, or for a classifier later.
|
|
180
|
+
|
|
110
181
|
## For you, not for an agent
|
|
111
182
|
|
|
112
|
-
`chi broadcast` refuses to run inside a chi session (where
|
|
183
|
+
`chi broadcast` (`log` and `deliver` too) refuses to run inside a chi session (where
|
|
113
184
|
`SAMAGOTCHI_PARENT_SESSION` is set): "chi broadcast is for your user, not an
|
|
114
185
|
agent". With the guardrails bundle, the `chi-broadcast` rule also asks before
|
|
115
186
|
any shell command that runs it, once at a time, and only you may allow it
|
|
@@ -121,9 +192,17 @@ any shell command that runs it, once at a time, and only you may allow it
|
|
|
121
192
|
|---|---|---|
|
|
122
193
|
| `broadcast.active_hours` | `8` | A session with no worker or REPL counts when its last turn ended within this many hours. |
|
|
123
194
|
| `broadcast.ticket_pattern` | `\b[A-Z][A-Z0-9]+-\d+\b` | A Ruby regex for ticket ids; an invalid one warns and the default is used. |
|
|
195
|
+
| `broadcast.triage_model` | recap's, else `default.model` | The triage model, a model ref (`box:qwen3.5-9b`). |
|
|
196
|
+
| `broadcast.triage_host_ref` | none | A `hosts:` name for it. |
|
|
197
|
+
| `broadcast.triage_base_url` | none | An OpenAI API base for it instead. |
|
|
198
|
+
| `broadcast.triage_parallel` | `4` | Triage requests at a time. |
|
|
199
|
+
| `broadcast.triage_deadline` | `20` | Seconds triage may take in all; the rest get the note unchecked. |
|
|
200
|
+
| `broadcast.threshold` | `0.5` | The P(yes) a session needs when the host gives logprobs. |
|
|
124
201
|
|
|
125
202
|
```yaml
|
|
126
203
|
broadcast:
|
|
127
204
|
active_hours: 12
|
|
128
205
|
ticket_pattern: '\b(?:PAY|OPS)-\d+\b'
|
|
206
|
+
triage_model: box:qwen3.5-9b
|
|
207
|
+
triage_deadline: 30
|
|
129
208
|
```
|
data/docs/configuration.md
CHANGED
|
@@ -742,7 +742,8 @@ One record is one line, plus indented payload lines at debug level:
|
|
|
742
742
|
control characters (terminal colours in tool output) are escaped.
|
|
743
743
|
- Tags: `turn` (a session's event trail), `http` (model requests),
|
|
744
744
|
`worker`, `bridge`, `web`, `attached`, `repl`, `idle`, `recap`, `hooks`,
|
|
745
|
-
`plugins`, `guardrails`, `config`, `memory`, `model` (debug dumps), `context` (attached context fetches)
|
|
745
|
+
`plugins`, `guardrails`, `config`, `memory`, `model` (debug dumps), `context` (attached context fetches),
|
|
746
|
+
`broadcast` (chi broadcast's triage).
|
|
746
747
|
- The format is parsed by `Samagotchi::LogLine` (`parse`, `each_record`);
|
|
747
748
|
keep tools that read it on that parser.
|
|
748
749
|
|
|
@@ -884,6 +885,12 @@ described in their own sections.
|
|
|
884
885
|
| `recap.sentences` | `2-4` | yes | Recap length, `N` or `N-M` (1–10). |
|
|
885
886
|
| `broadcast.active_hours` | `8` | | `chi broadcast` reaches a session a worker or a chi REPL runs, or whose last turn ended within this many hours. See [Broadcast](broadcast.md). |
|
|
886
887
|
| `broadcast.ticket_pattern` | `\b[A-Z][A-Z0-9]+-\d+\b` | | A Ruby regex for ticket ids; one in the note and in a session's branch (any case) or prompts delivers the broadcast there. See [Broadcast](broadcast.md). |
|
|
888
|
+
| `broadcast.triage_model` | recap's | | The model that judges a recipient no tag matched (a model ref, as `recap.model`); unset: the recap's model, else `default.model`. A ~9B dense instruct model or larger; 4B-class ones say yes to nearly everything. See [Broadcast](broadcast.md#triage). |
|
|
889
|
+
| `broadcast.triage_host_ref` | none | | A `hosts:` name for the triage model. |
|
|
890
|
+
| `broadcast.triage_base_url` | none | | An OpenAI API base for the triage model instead (`http://h:8081/v1`). |
|
|
891
|
+
| `broadcast.triage_parallel` | `4` | | Triage requests at a time. |
|
|
892
|
+
| `broadcast.triage_deadline` | `20` | | Seconds triage may take in all; a recipient not judged by then gets the note unchecked. |
|
|
893
|
+
| `broadcast.threshold` | `0.5` | | The P(yes) a recipient needs, when the triage host gives logprobs; a plain yes or no is 1 or 0. |
|
|
887
894
|
| `session.shared` | `true` | | Plain `chi` runs its session in a worker and attaches; `--no-shared` per run. |
|
|
888
895
|
| `session.idle_exit_minutes` | `30` | yes | An unused worker exits after this; `0` = never. |
|
|
889
896
|
| `session.keep_empty` | `false` | | Keep sessions nothing happened in. |
|
data/docs/guardrails.md
CHANGED
|
@@ -292,6 +292,21 @@ A plugin tool whose `targets:` name no command or path (an MCP tool) is asked
|
|
|
292
292
|
about with its arguments (`mcp_x_sum: a=20 b=22`), and an approval of "this
|
|
293
293
|
call" is keyed by them.
|
|
294
294
|
|
|
295
|
+
A plugin tool that runs other tools for the model (a dispatcher such as
|
|
296
|
+
`mcp_call`) can say, in its `targets:`, which tool a call acts as (`acts_as:`)
|
|
297
|
+
and with which arguments (`args:`). A rule's `tool:` then matches the call's
|
|
298
|
+
own name **or** the tool it acts as: `tool: mcp_github_*` fires for an
|
|
299
|
+
`mcp_call` that acts as `mcp_github_merge_pull_request`, and `tool: mcp_call`
|
|
300
|
+
fires whatever it acts as. The question names the tool it acts as (by the
|
|
301
|
+
`label:` its `targets:` gives, if any: display only) and shows its arguments, and an approval of "this call" is keyed by the real name, the
|
|
302
|
+
alias and those arguments (`/guardrails` lists it as
|
|
303
|
+
`mcp_call>mcp_github_merge_pull_request:number=7 …`), so it never covers a
|
|
304
|
+
direct call of that tool, nor a direct call's approval it. A tool can't act as
|
|
305
|
+
one of chi's own tools or another bundle's tool: that `acts_as:` is dropped
|
|
306
|
+
(and logged as `plugin_acts_as_dropped`), and the call is matched by its own
|
|
307
|
+
name only. Hooks and loop-guard's `ignore_tools` still see the call's own
|
|
308
|
+
name.
|
|
309
|
+
|
|
295
310
|
```yaml
|
|
296
311
|
- id: mcp-ask
|
|
297
312
|
tool: "mcp_*" # every MCP tool (the mcp bundle's mcp_<server>_<tool>)
|
data/docs/plugins.md
CHANGED
|
@@ -162,8 +162,8 @@ in the chat path's `tools:`, after chi's own tools.
|
|
|
162
162
|
it raises, the default is shown. Both are saved with the call's result, so
|
|
163
163
|
a web page reloaded later shows the same row: the web server doesn't run
|
|
164
164
|
plugins.
|
|
165
|
-
- `targets`: `->(args) { { paths: [...], command: "…", cwd: "…" } }`,
|
|
166
|
-
key optional, says what a call acts on, for [guardrails](#guardrails).
|
|
165
|
+
- `targets`: `->(args) { { paths: [...], command: "…", cwd: "…", acts_as: "…", args: {…}, label: "…" } }`,
|
|
166
|
+
each key optional, says what a call acts on, for [guardrails](#guardrails).
|
|
167
167
|
|
|
168
168
|
The block returns the result text. If it starts with `Error:`, it counts as
|
|
169
169
|
a failure. If it raises, the model gets `Error: <message>`. To return images
|
|
@@ -234,6 +234,27 @@ is what `targets:` says:
|
|
|
234
234
|
paths (chi's config, …) match them.
|
|
235
235
|
- `command:`: a shell command the call runs; `command:` rules match it.
|
|
236
236
|
- `cwd:`: where it runs, for the repo root and relative paths.
|
|
237
|
+
- `acts_as:`: the tool a call stands for, when the tool runs other tools
|
|
238
|
+
(the mcp bundle's `mcp_call`). Rules keyed by that tool's name match the
|
|
239
|
+
call too, as well as rules on the tool's own name. It can't be one of
|
|
240
|
+
chi's own tools or another bundle's tool: that is dropped (and logged).
|
|
241
|
+
- `args:`: the arguments it acts with (a Hash), when they aren't the call's
|
|
242
|
+
own (a dispatcher's inner arguments). A call with no `command:` or
|
|
243
|
+
`paths:` is asked about with them, and "allow this call" is keyed by them.
|
|
244
|
+
- `label:`: what the approval question names the call by (`github: x`), in
|
|
245
|
+
place of the tool's `label` (or, with `acts_as:`, that tool's). Display
|
|
246
|
+
only: rules and approvals don't read it. One line of at most 80
|
|
247
|
+
characters, else it is ignored (and logged).
|
|
248
|
+
|
|
249
|
+
```ruby
|
|
250
|
+
targets: ->(args) { { acts_as: "mcp_#{args["server"]}_#{args["tool"]}", args: args["args"],
|
|
251
|
+
label: "#{args["server"]}: #{args["tool"]}" } }
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
An approval of an `acts_as:` call keys on the tool's own name as well
|
|
255
|
+
(`mcp_call>mcp_github_x:…`), so it never stands in for an approval of the
|
|
256
|
+
tool it acts as, nor that one for it. Hooks see the call's own name, and so
|
|
257
|
+
does loop-guard's `ignore_tools`.
|
|
237
258
|
|
|
238
259
|
A tool without `targets:` is matched by its name only. A `targets:` that
|
|
239
260
|
raises counts as nothing (it is logged). See [guardrails.md](guardrails.md).
|
|
@@ -0,0 +1,267 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../idle_client"
|
|
4
|
+
require_relative "../idle_target"
|
|
5
|
+
require_relative "../cancellation_controller"
|
|
6
|
+
require_relative "../log"
|
|
7
|
+
require_relative "tags"
|
|
8
|
+
|
|
9
|
+
module Samagotchi
|
|
10
|
+
module Broadcast
|
|
11
|
+
# Whether a broadcast concerns a recipient its tags didn't match: a
|
|
12
|
+
# small model reads the note and the session's ScopeCard and answers
|
|
13
|
+
# yes or no (Triage::LLM). Many recipients are judged in parallel under
|
|
14
|
+
# one deadline (Triage.judge_all); whatever isn't judged by then, or
|
|
15
|
+
# fails, is delivered unchecked (fail open: a missed recipient is worse
|
|
16
|
+
# than an extra note).
|
|
17
|
+
module Triage
|
|
18
|
+
# One recipient's verdict.
|
|
19
|
+
# @!attribute relevant [Boolean] it gets the note
|
|
20
|
+
# @!attribute p [Float, nil] the model's P(yes) from its logprobs,
|
|
21
|
+
# 1.0 or 0.0 from a plain answer, nil when no model judged it
|
|
22
|
+
# @!attribute reason [String] why, for the output and the log
|
|
23
|
+
# @!attribute by [String] who decided: "tags" (a shared tag),
|
|
24
|
+
# "model", "fallback" (delivered unchecked: deadline, error, an
|
|
25
|
+
# answer that is no yes or no), "scope" (the note's first line
|
|
26
|
+
# names another project), "all" (--all) or "repl" (a chi REPL owns
|
|
27
|
+
# it: no notes)
|
|
28
|
+
# @!attribute match [Match, nil] the shared tag (by "tags")
|
|
29
|
+
Verdict = Data.define(:relevant, :p, :reason, :by, :match) do
|
|
30
|
+
def initialize(match: nil, **) = super
|
|
31
|
+
|
|
32
|
+
def unchecked? = by == "fallback"
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# The spike's prompt (plain yes/no beat JSON with a p and a reason
|
|
36
|
+
# for every model tried: writing a reason pushes towards "relevant").
|
|
37
|
+
SYSTEM = <<~TEXT
|
|
38
|
+
You route short notes that a developer broadcasts to their running coding-agent sessions.
|
|
39
|
+
You get one note and one session's card (project, branch, title, what the session was started for, its latest recap, the last request).
|
|
40
|
+
Decide whether the note may affect THIS session's current work: would the agent working on it want to know?
|
|
41
|
+
A note can start with an informal scope line (a project, a component, a team); use it as a hint.
|
|
42
|
+
Notes about things unrelated to the session's work (social, other projects, other components) are not relevant.
|
|
43
|
+
Answer with exactly one word: yes or no.
|
|
44
|
+
TEXT
|
|
45
|
+
|
|
46
|
+
DEFAULT_PARALLEL = 4
|
|
47
|
+
DEFAULT_DEADLINE = 20.0
|
|
48
|
+
DEFAULT_THRESHOLD = 0.5
|
|
49
|
+
# How long threads still asking get to stop once the deadline passed.
|
|
50
|
+
JOIN_GRACE = 1.0
|
|
51
|
+
|
|
52
|
+
module_function
|
|
53
|
+
|
|
54
|
+
# @return [Verdict] delivered unchecked, +why+ in the reason
|
|
55
|
+
def unchecked(why) = Verdict.new(relevant: true, p: nil, reason: "unchecked: #{why}", by: "fallback")
|
|
56
|
+
|
|
57
|
+
# The messages a model judges +note+ for +card+ by.
|
|
58
|
+
def messages(note, card)
|
|
59
|
+
[{ role: "system", content: SYSTEM }, { role: "user", content: "NOTE:\n#{note}\n\nSESSION CARD:\n#{card}" }]
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
# The one project the note's first line names, as a whole word (any
|
|
63
|
+
# case), among +projects+ (the recipients' project names); nil when it
|
|
64
|
+
# names none or several. A scope line ("shopfront/checkout", "agentd
|
|
65
|
+
# web") then keeps the note to that project's sessions; anything
|
|
66
|
+
# fuzzier is the model's job.
|
|
67
|
+
# @param projects [Array<String>]
|
|
68
|
+
# @return [String, nil]
|
|
69
|
+
def scope_project(note, projects)
|
|
70
|
+
line = note.to_s.lines.first.to_s.downcase
|
|
71
|
+
named = projects.compact.uniq.select do |project|
|
|
72
|
+
line.match?(/(?<![[:alnum:]_-])#{Regexp.escape(project.downcase)}(?![[:alnum:]_-])/)
|
|
73
|
+
end
|
|
74
|
+
named.one? ? named.first : nil
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# Every card's verdict: a shared tag delivers (Tags.match); a note
|
|
78
|
+
# whose first line names one of the projects (scope_project) skips
|
|
79
|
+
# the other projects' sessions; the model judges the rest
|
|
80
|
+
# (judge_all, +judge+ its keywords).
|
|
81
|
+
# @param cards [Array<ScopeCard>]
|
|
82
|
+
# @param note_tags [Array<Tag>]
|
|
83
|
+
# @return [Hash{String => Verdict}] by card id, in +cards+' order
|
|
84
|
+
def verdicts(note, cards, note_tags:, new_backend:, **judge)
|
|
85
|
+
scope = scope_project(note, cards.map(&:project))
|
|
86
|
+
decided = cards.to_h do |card|
|
|
87
|
+
match = Tags.match(note_tags, card.tags)
|
|
88
|
+
verdict = if match
|
|
89
|
+
Verdict.new(relevant: true, p: nil, reason: match.reason, by: "tags", match: match)
|
|
90
|
+
elsif scope && card.project != scope
|
|
91
|
+
Verdict.new(relevant: false, p: nil, reason: "scope line names #{scope}", by: "scope")
|
|
92
|
+
end
|
|
93
|
+
[card.id, verdict]
|
|
94
|
+
end
|
|
95
|
+
judged = judge_all(note, cards.reject { |card| decided[card.id] }, new_backend: new_backend, **judge)
|
|
96
|
+
cards.to_h { |card| [card.id, decided[card.id] || judged.fetch(card.id)] }
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# Judges +cards+ with at most +parallel+ backends at a time, one per
|
|
100
|
+
# thread (a backend keeps per-host state), all under +deadline+
|
|
101
|
+
# seconds. A card not judged by then is delivered unchecked; the
|
|
102
|
+
# requests still running are cancelled.
|
|
103
|
+
# @param new_backend [#call] (CancellationController) → a backend:
|
|
104
|
+
# #judge(note, card) → Verdict
|
|
105
|
+
# @param cards [Array<ScopeCard>]
|
|
106
|
+
# @return [Hash{String => Verdict}] by card id
|
|
107
|
+
def judge_all(note, cards, new_backend:, parallel: DEFAULT_PARALLEL, deadline: DEFAULT_DEADLINE,
|
|
108
|
+
clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) })
|
|
109
|
+
return {} if cards.empty?
|
|
110
|
+
|
|
111
|
+
queue = Queue.new
|
|
112
|
+
cards.each { |card| queue << card }
|
|
113
|
+
queue.close
|
|
114
|
+
results = {}
|
|
115
|
+
lock = Mutex.new
|
|
116
|
+
changed = ConditionVariable.new
|
|
117
|
+
cancel = CancellationController.new
|
|
118
|
+
ends = clock.call + deadline.to_f
|
|
119
|
+
finished = 0
|
|
120
|
+
threads = Array.new(parallel.to_i.clamp(1, cards.size)) do
|
|
121
|
+
Thread.new do
|
|
122
|
+
card = nil
|
|
123
|
+
backend = new_backend.call(cancel)
|
|
124
|
+
while (card = queue.pop)
|
|
125
|
+
verdict = judge_one(backend, note, card)
|
|
126
|
+
lock.synchronize do
|
|
127
|
+
results[card.id] = verdict unless cancel.cancelled?
|
|
128
|
+
changed.broadcast
|
|
129
|
+
end
|
|
130
|
+
card = nil
|
|
131
|
+
end
|
|
132
|
+
rescue StandardError => e
|
|
133
|
+
# No backend (or a bug around it): this thread's card and the
|
|
134
|
+
# ones it takes from here on are delivered unchecked at once,
|
|
135
|
+
# so nobody waits for the deadline.
|
|
136
|
+
Log.exception(:broadcast, "triage_thread_failed", e)
|
|
137
|
+
failed = unchecked("triage failed (#{e.class})")
|
|
138
|
+
lock.synchronize { results[card.id] ||= failed if card }
|
|
139
|
+
while (card = queue.pop)
|
|
140
|
+
lock.synchronize { results[card.id] ||= failed }
|
|
141
|
+
end
|
|
142
|
+
ensure
|
|
143
|
+
lock.synchronize do
|
|
144
|
+
finished += 1
|
|
145
|
+
changed.broadcast
|
|
146
|
+
end
|
|
147
|
+
end
|
|
148
|
+
end
|
|
149
|
+
lock.synchronize do
|
|
150
|
+
loop do
|
|
151
|
+
break if results.size == cards.size || finished == threads.size
|
|
152
|
+
|
|
153
|
+
left = ends - clock.call
|
|
154
|
+
break if left <= 0
|
|
155
|
+
|
|
156
|
+
changed.wait(lock, left)
|
|
157
|
+
end
|
|
158
|
+
cancel.cancel!(:deadline)
|
|
159
|
+
queue.clear
|
|
160
|
+
end
|
|
161
|
+
threads.each { |thread| thread.join(JOIN_GRACE) || thread.kill }
|
|
162
|
+
cards.to_h { |card| [card.id, results[card.id] || unchecked("triage deadline")] }
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
def judge_one(backend, note, card)
|
|
166
|
+
backend.judge(note, card)
|
|
167
|
+
rescue Samagotchi::LLM::RequestCancelled
|
|
168
|
+
unchecked("triage deadline")
|
|
169
|
+
rescue StandardError => e
|
|
170
|
+
Log.exception(:broadcast, "triage_failed", e)
|
|
171
|
+
unchecked("triage failed (#{e.class})")
|
|
172
|
+
end
|
|
173
|
+
private_class_method :judge_one
|
|
174
|
+
|
|
175
|
+
# A triage model on an OpenAI-compatible host (an IdleTarget), asked
|
|
176
|
+
# through its own IdleClient: thinking off, a few tokens, the first
|
|
177
|
+
# token's logprobs when the host gives them. P(yes) / (P(yes) +
|
|
178
|
+
# P(no)) from those is graded against +threshold+; a plain yes or no
|
|
179
|
+
# is 1.0 or 0.0.
|
|
180
|
+
class LLM
|
|
181
|
+
MAX_TOKENS = 16
|
|
182
|
+
TOP_LOGPROBS = 5
|
|
183
|
+
LOGPROB_OPTIONS = { logprobs: true, top_logprobs: TOP_LOGPROBS }.freeze
|
|
184
|
+
# Hosts (base urls) that refused logprobs with a 400: asked without
|
|
185
|
+
# them from then on, in this process.
|
|
186
|
+
@no_logprobs = Set.new
|
|
187
|
+
@no_logprobs_lock = Mutex.new
|
|
188
|
+
|
|
189
|
+
class << self
|
|
190
|
+
def logprobs?(base_url) = @no_logprobs_lock.synchronize { !@no_logprobs.include?(base_url) }
|
|
191
|
+
|
|
192
|
+
def no_logprobs!(base_url) = @no_logprobs_lock.synchronize { @no_logprobs << base_url }
|
|
193
|
+
|
|
194
|
+
# Specs: forget the hosts that refused.
|
|
195
|
+
def reset_logprobs! = @no_logprobs_lock.synchronize { @no_logprobs.clear }
|
|
196
|
+
end
|
|
197
|
+
|
|
198
|
+
# @param target [IdleTarget]
|
|
199
|
+
# @param timeout [Numeric] each request's
|
|
200
|
+
# @param client [IdleClient, nil] specs inject one
|
|
201
|
+
def initialize(target:, timeout:, threshold: DEFAULT_THRESHOLD, cancel_controller: nil, client: nil)
|
|
202
|
+
@target = target
|
|
203
|
+
@threshold = threshold.to_f
|
|
204
|
+
@cancel = cancel_controller
|
|
205
|
+
@client = client || IdleClient.for(target, timeout: timeout, purpose: "broadcast")
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
# @return [Verdict]
|
|
209
|
+
# @raise [Samagotchi::LLM::RequestCancelled] the deadline passed
|
|
210
|
+
def judge(note, card)
|
|
211
|
+
answer = ask(Triage.messages(note, card))
|
|
212
|
+
word = self.class.word(answer.text)
|
|
213
|
+
return Triage.unchecked("the model answered #{answer.text.to_s.strip[0, 40].inspect}") unless word
|
|
214
|
+
|
|
215
|
+
p = self.class.logprob_p(answer.top_logprobs, word: word)
|
|
216
|
+
relevant = p ? p >= @threshold : word == "yes"
|
|
217
|
+
reason = "model: #{relevant ? "yes" : "no"}#{format(" (p %.2f)", p) if p}"
|
|
218
|
+
Verdict.new(relevant: relevant, p: p || (word == "yes" ? 1.0 : 0.0), reason: reason, by: "model")
|
|
219
|
+
rescue IdleClient::SummarizeError => e
|
|
220
|
+
Triage.unchecked("the model failed (#{e.message[0, 120]})")
|
|
221
|
+
end
|
|
222
|
+
|
|
223
|
+
# "yes" or "no": the answer's first such word (thinking left out),
|
|
224
|
+
# nil when it has neither.
|
|
225
|
+
def self.word(text)
|
|
226
|
+
text.to_s.gsub(%r{<think>.*?</think>}m, "").strip.downcase[/\A\W*(yes|no)\b/, 1]
|
|
227
|
+
end
|
|
228
|
+
|
|
229
|
+
# P(yes) / (P(yes) + P(no)) over the first token's alternatives
|
|
230
|
+
# ("Yes", " yes" count as yes), when that token is the answer +word+
|
|
231
|
+
# itself: its likeliest alternative is yes or no and agrees with
|
|
232
|
+
# +word+. nil otherwise (an answer starting "**", "<think>" or a
|
|
233
|
+
# quote: those alternatives say nothing about yes or no).
|
|
234
|
+
# @param top [Array<Samagotchi::LLM::TokenLogprob>]
|
|
235
|
+
# @return [Float, nil]
|
|
236
|
+
def self.logprob_p(top, word:)
|
|
237
|
+
first = top.max_by(&:logprob)
|
|
238
|
+
return nil unless first && first.token.strip.downcase == word
|
|
239
|
+
|
|
240
|
+
sum = ->(want) { top.select { |t| t.token.strip.downcase == want }.sum { |t| Math.exp(t.logprob) } }
|
|
241
|
+
yes = sum.call("yes")
|
|
242
|
+
no = sum.call("no")
|
|
243
|
+
(yes / (yes + no)).round(4)
|
|
244
|
+
end
|
|
245
|
+
|
|
246
|
+
private
|
|
247
|
+
|
|
248
|
+
def ask(messages)
|
|
249
|
+
base_url = @target.base_url
|
|
250
|
+
options = self.class.logprobs?(base_url) ? LOGPROB_OPTIONS : {}
|
|
251
|
+
begin
|
|
252
|
+
@client.ask(messages, max_tokens: MAX_TOKENS, cancel_controller: @cancel, kind: "broadcast", options: options)
|
|
253
|
+
rescue IdleClient::SummarizeError => e
|
|
254
|
+
raise unless !options.empty? && e.cause.is_a?(Samagotchi::LLM::BadRequest)
|
|
255
|
+
|
|
256
|
+
# The host won't give logprobs (a server that refuses "n and
|
|
257
|
+
# logprobs"): ask again without, and leave them out from now on.
|
|
258
|
+
self.class.no_logprobs!(base_url)
|
|
259
|
+
Log.info(:broadcast, "logprobs_refused", model: @target.label, detail: e.cause.message.to_s[0, 200])
|
|
260
|
+
options = {}
|
|
261
|
+
retry
|
|
262
|
+
end
|
|
263
|
+
end
|
|
264
|
+
end
|
|
265
|
+
end
|
|
266
|
+
end
|
|
267
|
+
end
|