samagotchi 0.30.0 → 0.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +46 -1
- data/docs/configuration.md +4 -0
- data/docs/internals/engine.md +1 -1
- data/docs/sessions.md +9 -2
- data/lib/samagotchi/bridge/card_store.rb +1 -1
- data/lib/samagotchi/bridge/turn_notice.rb +1 -1
- data/lib/samagotchi/bundles/system/delegated.md +1 -0
- data/lib/samagotchi/bundles/system/manifest.yml +3 -3
- data/lib/samagotchi/bundles/system/self_map.md +2 -1
- data/lib/samagotchi/child_reports.rb +214 -0
- data/lib/samagotchi/client.rb +35 -4
- data/lib/samagotchi/config.rb +5 -0
- data/lib/samagotchi/engine.rb +2 -0
- data/lib/samagotchi/idle_recap.rb +4 -0
- data/lib/samagotchi/kernel_loop.rb +44 -5
- data/lib/samagotchi/llm/chat_loop.rb +2 -1
- data/lib/samagotchi/llm/errors.rb +9 -0
- data/lib/samagotchi/log_subscriber.rb +1 -1
- data/lib/samagotchi/prompt_warmup.rb +30 -5
- data/lib/samagotchi/reply_wait.rb +39 -8
- data/lib/samagotchi/session.rb +26 -5
- data/lib/samagotchi/session_inbox.rb +34 -1
- data/lib/samagotchi/session_manager.rb +32 -3
- data/lib/samagotchi/steer.rb +39 -7
- data/lib/samagotchi/terminal_ui/attached_loop.rb +4 -2
- data/lib/samagotchi/terminal_ui/event_renderer.rb +3 -1
- data/lib/samagotchi/terminal_ui/formatting.rb +31 -4
- data/lib/samagotchi/terminal_ui.rb +9 -0
- data/lib/samagotchi/tool_call_parser.rb +40 -0
- data/lib/samagotchi/tool_declarations.rb +7 -7
- data/lib/samagotchi/tools/delegate.rb +13 -4
- data/lib/samagotchi/tools/delegate_cursor.rb +91 -0
- data/lib/samagotchi/tools/delegate_wait.rb +50 -31
- data/lib/samagotchi/tools/task_runtime.rb +21 -2
- data/lib/samagotchi/tools/task_wait.rb +2 -2
- data/lib/samagotchi/version.rb +1 -1
- data/lib/samagotchi/web/app.rb +8 -2
- data/lib/samagotchi/web/public/app.js +12 -4
- data/lib/samagotchi/web/public/format.js +8 -1
- data/lib/samagotchi/web/public/notify.js +4 -2
- data/lib/samagotchi/web/public/turn_events.js +26 -8
- data/lib/samagotchi/web/public/turn_view.js +23 -4
- data/lib/samagotchi/worker.rb +217 -8
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b18cad295a3d093e74497e8d271c571b12531797f0345bc7f929cc3e1d10d663
|
|
4
|
+
data.tar.gz: e72c996d2692f6a91a69f03943ca4e985ba777fea18ba4cb08728fe32a744c72
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8878011a9051c4d815dc24e0785fd91772155e770d39974841c2d85b0d0d5f9a57d23f6f862d49f449e8b9436171a99b91e2fad5105ce9863ba5ce3a76e9fe5d
|
|
7
|
+
data.tar.gz: 11086d6eb896ea62961f87d70ecd7e6eaaacab9fefc519ef11249b643fbdcb3becdc28054447b23be10fbf898c599bd7da746a2a259504e54e9e9656f8d92d86
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,50 @@ and commands may change between minor versions. How releases are made:
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [0.31.0] - 2026-10-06
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- Delegate reports: a child started with `delegate` (`wait: false`) brings its reply to its parent by itself when
|
|
16
|
+
it ends its turn, fails or asks the parent something. A parent mid-turn gets it at the next step; an idle parent
|
|
17
|
+
runs a turn for it and tells its user, waking its worker when that had exited. The report carries the reply, so
|
|
18
|
+
the model doesn't poll with `delegate_result`. The web shows a "delegate report" bubble, the attached terminal
|
|
19
|
+
`delegate report> <child> answered: …`, and the bell rings when such a turn ends. A stopped or archived parent
|
|
20
|
+
isn't woken: its reports join its next turn. New settings `session.delegate_reports` (`wake`, `queue`, `off`) and
|
|
21
|
+
`session.max_wakes` (default 10 turns in a row with no human input; then reports wait for your next message).
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
|
|
25
|
+
- The parent's record of which delegate replies its model already had is kept on disk (`delegates.json`), so a
|
|
26
|
+
restarted worker no longer hands the model a reply again; `delegate_result` asked again about a question it
|
|
27
|
+
already reported waits for its answer instead of repeating it.
|
|
28
|
+
- A delegated child does only what its task asks: a question gets a report with the change it would make, not
|
|
29
|
+
file edits; it edits files only when the task asks for a change, and then only the files that takes.
|
|
30
|
+
- `task_get`, `task_stop` and `task_wait` say they take a task id (from `task_create` or `task_list`, with an example),
|
|
31
|
+
not a session id: models passed session ids to them.
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- A message sent while a reminder turn ran was lost when that reminder turn was the session worker's first turn
|
|
36
|
+
(a fresh or restarted worker): the turn took the message and dropped it. It now joins the turn as steering does.
|
|
37
|
+
- `chi -p … --non-interactive` (and a fresh REPL session's first turn without a worker) could not use the `delegate`
|
|
38
|
+
tool: the session was not saved until its turn ended, so the child failed with "Session not found". A fresh
|
|
39
|
+
session is now saved before its first turn.
|
|
40
|
+
- `task_get`, `task_stop` and `task_wait` given a session id say so: `task not found: <id> (that is a session id; task
|
|
41
|
+
ids come from task_create, and task_list lists them)`.
|
|
42
|
+
- `task_get`, `task_stop` and `task_wait` take only an id shaped like task_create's (`20261005093000-1a2b3c4d`): an id
|
|
43
|
+
such as `../x` could read a `task.json` outside `tmp/tasks`. Anything else is `task not found`.
|
|
44
|
+
- `task_wait` for an unknown task said `Error: Error: task not found: …`; it now says it once.
|
|
45
|
+
- A corrupt Gemma 4 generation on a llama.cpp host (a tool call never closed, or a fresh thought header after the
|
|
46
|
+
answer text) was saved as the answer, and a delegate report passed it on to the parent. A poisoned llama.cpp prompt
|
|
47
|
+
cache produced one with another session's answer in it (ggml-org/llama.cpp#27148). chi now logs
|
|
48
|
+
`generation_malformed`, shows `↻ malformed answer, asking again (1/1)` and asks once more without the prompt cache;
|
|
49
|
+
a second corrupt generation fails the turn ("malformed generation from host …"). The half tool call never runs.
|
|
50
|
+
- On llama.cpp, a parent's turn-end warm-up no longer waits behind a delegate child generating on the same slot, and
|
|
51
|
+
its next turn no longer waits behind that warm-up: the warm-up is skipped when `/slots` says the slot is busy.
|
|
52
|
+
|
|
53
|
+
Update with `chi update` and restart `chi web`.
|
|
54
|
+
|
|
11
55
|
## [0.30.0] - 2026-10-05
|
|
12
56
|
|
|
13
57
|
### Added
|
|
@@ -1552,7 +1596,8 @@ and long-lived sessions.
|
|
|
1552
1596
|
- A macOS desktop helper (`chi desktop install`): a "Send to chi" Service and a
|
|
1553
1597
|
hotkey panel that send selected text or the clipboard to your sessions.
|
|
1554
1598
|
|
|
1555
|
-
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.
|
|
1599
|
+
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.31.0...HEAD
|
|
1600
|
+
[0.31.0]: https://github.com/dm1try/samagotchi/compare/v0.30.0...v0.31.0
|
|
1556
1601
|
[0.30.0]: https://github.com/dm1try/samagotchi/compare/v0.29.0...v0.30.0
|
|
1557
1602
|
[0.29.0]: https://github.com/dm1try/samagotchi/compare/v0.28.0...v0.29.0
|
|
1558
1603
|
[0.28.0]: https://github.com/dm1try/samagotchi/compare/v0.27.0...v0.28.0
|
data/docs/configuration.md
CHANGED
|
@@ -72,6 +72,8 @@ session:
|
|
|
72
72
|
# idle_exit_minutes: 30 # an unused worker exits after this (0 = never)
|
|
73
73
|
# keep_empty: false # true keeps sessions nothing happened in (default: deleted when left)
|
|
74
74
|
# max_children: 4 # running sessions one session may have delegated at a time (the delegate tool)
|
|
75
|
+
# delegate_reports: wake # a child's reply reaches its parent by itself: wake (an idle parent runs a turn), queue, off
|
|
76
|
+
# max_wakes: 10 # turns in a row a parent runs for delegate reports with no human input
|
|
75
77
|
|
|
76
78
|
# Baseline memories preloaded into the system prompt (same shape as --memory);
|
|
77
79
|
# name entries you have, or chi warns at each start.
|
|
@@ -884,6 +886,8 @@ described in their own sections.
|
|
|
884
886
|
| `session.idle_exit_minutes` | `30` | yes | An unused worker exits after this; `0` = never. |
|
|
885
887
|
| `session.keep_empty` | `false` | | Keep sessions nothing happened in. |
|
|
886
888
|
| `session.max_children` | `4` | | Running delegated sessions one session may have. |
|
|
889
|
+
| `session.delegate_reports` | `wake` | | When a delegate child ends a turn, its reply reaches the parent: `wake` runs an idle parent's turn for it, `queue` waits for the parent's next turn, `off` leaves it to `delegate_result`. |
|
|
890
|
+
| `session.max_wakes` | `10` | | Turns a parent runs for delegate reports in a row with no human input; past it reports wait for the next turn. |
|
|
887
891
|
| `session.retention_days` | `14` | yes | Delete sessions not updated for N days; `0` = forever. See [Sessions](sessions.md). |
|
|
888
892
|
| `session.max_count` | `500` | yes | Keep the newest N; `0` = uncapped. |
|
|
889
893
|
| `session.keep_status` | none | yes | Comma list of statuses never pruned (a session a worker or `chi` has open is never pruned anyway). |
|
data/docs/internals/engine.md
CHANGED
|
@@ -294,7 +294,7 @@ Turn events (the main ones; `Events` in `events.rb` keeps the shared sets):
|
|
|
294
294
|
| `:generation_retrying` | the transport retries the request | `{iteration, ...}` (the retry's fields) |
|
|
295
295
|
| `:generation_completed` | a generation ended, or was cut | `{iteration, content_length, thinking_chars, served_model, requested_model, finish_reason, ...}`; a cut adds `stopped_by`, `stop_reason`; the Engine adds `speed`, `tokens` |
|
|
296
296
|
| `:generation_cancelled` | the turn was cancelled mid-loop | `{iteration, reason, stopped_by}` |
|
|
297
|
-
| `:empty_answer_retry` | an empty or
|
|
297
|
+
| `:empty_answer_retry` | an empty, cut or malformed answer is asked again | `{iteration, attempt, of, finish_reason, thinking_chars, stopped_by?, malformed?}` |
|
|
298
298
|
| `:steer_cut` | a steer cut the generation | `{iteration, source}` |
|
|
299
299
|
| `:pending_input_merged` | queued input joined the conversation | `{iteration, count, content, steers?, answer}` |
|
|
300
300
|
| `:tool_dispatch_started` / `:tool_dispatch_completed` | around a batch | `{iteration, call_count}` |
|
data/docs/sessions.md
CHANGED
|
@@ -181,7 +181,7 @@ chi send --wait 3fa2 # sends nothing: waits fo
|
|
|
181
181
|
|
|
182
182
|
The `delegate` tool hands a task to a **child session**: an ordinary chi session in a worker of its own, started in the parent's `working_directory` with the task, verbatim, as its first user message, on the parent's model unless the call names one (`model:` takes a name or an alias, resolved as `--model` is; an unknown host comes back as the tool's `Error:` line; an id the host's saved model list doesn't have starts the child with a `Warning: host 'h' doesn't list model 'x' (did you mean: …?)` line in the result). Children run in parallel and only their **final reply** comes back to the parent: the child's newest `output/<timestamp>.txt` file, which the worker writes at the end of each turn that produced visible text, cut head-and-tail like an `execute` result. Nothing of the child's trace enters the parent's context. Nothing is hidden: a child shows in `chi sessions list` (`↳ <parent>`), the web (a `↳` chip on its card, `delegated by` in its info bar, listed right after its parent in the all-sessions view) and the `list_sessions` tool (`child`; the parent shows as `parent` in the child's own list); the user can `chi --attach <child id>` and steer it mid-run, and `chi send` and `send_note` reach it like any session.
|
|
183
183
|
|
|
184
|
-
- `delegate(task, model:, session:, wait:, timeout:)`: starts a child and, by default, waits for its reply (`wait: false` returns at once, so several children can run side by side; `timeout`, default 600 s, leaves a slow child running; 0 checks once without waiting). With `session:` (a child's id or prefix) the task goes to that child as a follow-up instead, delivered like `chi send` (client id `delegate:<parent short id>`, shown as `delegate>` in the child's UIs); only this session's children take one.
|
|
184
|
+
- `delegate(task, model:, session:, wait:, timeout:)`: starts a child and, by default, waits for its reply (`wait: false` returns at once, so several children can run side by side, and the reply comes back by itself as a *delegate report*, below; `timeout`, default 600 s, leaves a slow child running; 0 checks once without waiting). With `session:` (a child's id or prefix) the task goes to that child as a follow-up instead, delivered like `chi send` (client id `delegate:<parent short id>`, shown as `delegate>` in the child's UIs); only this session's children take one.
|
|
185
185
|
- `delegate_result(session:, timeout:)`: waits for a child's next reply: the one named, or the newest running child (`timeout` as for `delegate`: 0 checks once and returns at once). The reply comes back once; a later call waits for the next one.
|
|
186
186
|
- What a wait returns, `session: <id>` then `status:` and the text, with the status words `chi send --wait --format json` uses: `answered` with the reply after a `---` line; `question` as soon as the child opens a question (the model's own, or a hook's), with the question's text, options and the `chi answer` command for it (nothing is answered for it: `chi answer`, attach or use the web, then wait again); `failed`, `canceled` or `no_answer` when the child's turn ended with nothing to say (its session shows what happened); `error` (its worker crashed) or `stopped` (`chi sessions stop`) at once; `worker_gone` when the child has had no worker for 15 seconds (it crashed without saying so, or idle-exited): `delegate` with `session:` starts it again; `running` on timeout, or when the parent's own turn is canceled (the child keeps running; the next turn can wait again). A reply saved before the child's worker idle-exited is returned at once.
|
|
187
187
|
- **A child's step-limit question goes back to the parent model**, not to its user: `delegate_result` returns `status: question` with the `chi answer` commands for Continue and Stop, and says the model may instead send a narrower follow-up (`delegate session:`, which drops the question) or report back. A parent's `chi answer` follows `turn.parent_continue` (see [chi as a sub-agent](sub-agent.md#the-step-limit)).
|
|
@@ -190,6 +190,13 @@ The `delegate` tool hands a task to a **child session**: an ordinary chi session
|
|
|
190
190
|
- How the child trusts it: the parent's POST that says "answered" (Bridge `POST /session/<child>/relay`) carries no answer. The child asks its parent session's live Bridge (`POST /session/<parent>/relay/status`), which reports only what the parent's question flow took from a UI (`RelayDesk`, in memory only), and takes it only for this child, the question pending now, answered. A parent agent's answer on the relay card (a piped or marked `chi --attach PARENT`, `chi answer`) is held to `guardrails.parent_approvals` by the parent and again by the child.
|
|
191
191
|
- A Stop of the parent's turn closes its card; the child's approval stays open there. A child's worker that goes closes the card (`… worker is gone`); a parent's worker that goes clears the child's "waiting in parent" line within 2 s. A later `delegate_result` relays it again.
|
|
192
192
|
- No relay where the parent can't host one (a `--no-shared` REPL, `--non-interactive`) or isn't waiting (`wait: false` until the next `delegate_result`): the approval waits on the child, with the web's bell, and the result's question block says to deny it or leave it for the user. The model's own questions and hooks' questions come back to the model as before. A grandchild's (a plugin's `ctx.sessions.fork` can make one) is relayed hop by hop: `delegate ab12 → cd34 asks:`.
|
|
193
|
+
- **Delegate reports: a child's reply comes back by itself.** When a child ends a turn that was its parent's (its task, a follow-up, a continue the parent answered, a reminder: not a turn its user typed into it directly), it rings the parent: a small file in the parent's `children/` dir, written after the child's turn is saved. So does a question the parent's model decides (the child's own `ask_user_question`, the step-limit continue; not an approval or a hook's question, which the user answers on the child's card), the turn after such a question whoever starts it, and the child's worker crashing. The parent reads the child as `delegate_result` would and hands its model a **report**: the same text (`session:`, `status:`, the reply), under its own header (`[Delegate report, delivered by chi …; not your user's message. Tell your user what it found …]`), as an input message with `source: "delegate_report"`.
|
|
194
|
+
- Parent mid-turn: the report joins the turn at its next iteration boundary (a "delegate report" bubble in the web, `delegate report> ab12cd34 answered: …` in the attached terminal).
|
|
195
|
+
- Parent idle: it runs a turn for the report (a *wake turn*, `last_turn.origin: "delegate_report"`) within the worker's 5 s tick, or wakes a worker for it when the parent's has idle-exited. Several reports that wait together are one turn and one message. A stopped, archived or scratch parent, or one open in a `--no-shared` REPL, is not woken: the rings wait on disk and join the next turn it runs (the user's message first). A worker just started (`chi send` to a stopped parent) runs no wake turn in its first 2 s, so the message it was started for goes first.
|
|
196
|
+
- The parent's cursor per child (the newest reply it was given and the child as it was then) is in its `delegates.json`, shared with `delegate_result` and `wait: true`: a reply the model already had is never reported again, also after the parent's worker restarts. A ring is deleted only when the turn that took its report is kept; a turn that fails leaves it for the next one (and the report isn't handed back to the composer).
|
|
197
|
+
- Limits: `session.max_wakes` (default `10`) wake turns in a row with no human input; past it the reports wait for the next message and the UIs show `N delegate reports waiting; they join your next message`. A wake turn that fails pauses wakes until the next human input. A continue offer waiting in the parent takes the reports into its continue turn instead.
|
|
198
|
+
- `session.delegate_reports`: `wake` (default), `queue` (no wake turns: reports join the parent's next turn) or `off` (no rings; only `delegate_result` asks, as before). Env `SAMAGOTCHI_SESSION_DELEGATE_REPORTS`, `SAMAGOTCHI_SESSION_MAX_WAKES`.
|
|
199
|
+
- The web's bell rings at a wake turn's end however short (nobody started it, so nobody watched it). The text a model writes about a report mid-turn, before its next tool call, stays visible after the report's bubble once the turn ends.
|
|
193
200
|
- Every child starts with the shipped `delegated` system memory preloaded (`preloaded_memory_names: ["system/delegated"]`; `--mute delegated` would hide it): put everything in the final reply, don't delegate further, don't write memories or change config, stay in the folder. Its system prompt also says `Delegated by session <parent id>`. The parent's own `--mute` list does not carry over.
|
|
194
201
|
- Limits: a child cannot `delegate` (depth 1; the tool refuses), and one session may have at most `session.max_children` (default `4`, env `SAMAGOTCHI_SESSION_MAX_CHILDREN`) children running at a time, counted from the session files (a finished or idle-exited child does not count; one whose worker is still starting does, for 15 s). A refused call creates no session. Nothing stops children with the parent: `chi sessions stop ID` does.
|
|
195
|
-
- The link is the session's `parent_id`, written before the worker starts, so a respawn keeps it; `GET /api/sessions` and `/api/sessions/:id` carry it, and `SessionManager.children_of` lists a parent's children.
|
|
202
|
+
- The link is the session's `parent_id`, written before the worker starts, so a respawn keeps it (and `delegate: true`, which a fork's `parent_id` doesn't come with; a child saved before that field is known by its `system/delegated` memory); `GET /api/sessions` and `/api/sessions/:id` carry it, and `SessionManager.children_of` lists a parent's children.
|
|
@@ -166,7 +166,7 @@ module Samagotchi
|
|
|
166
166
|
# this worker runs, where it showed live.
|
|
167
167
|
when :guardrail_warning then push({ type: :guardrail_warning, message: event[:message], label: event[:label] }
|
|
168
168
|
.compact.merge(in_turn: false, turns: @turns_done))
|
|
169
|
-
when :empty_answer_retry then add_turn_row(event.slice(:type, :attempt, :of, :stopped_by)) if @running
|
|
169
|
+
when :empty_answer_retry then add_turn_row(event.slice(:type, :attempt, :of, :stopped_by, :malformed)) if @running
|
|
170
170
|
when :steer_cut then add_turn_row(event.slice(:type, :source)) if @running
|
|
171
171
|
when :question_requested
|
|
172
172
|
add_turn_row({ type: :question, pending_question: Marshal.load(Marshal.dump(event[:pending_question])) }) if @running
|
|
@@ -11,7 +11,7 @@ module Samagotchi
|
|
|
11
11
|
module TurnNotice
|
|
12
12
|
FIELDS = {
|
|
13
13
|
hook_notice: %i[hook text level],
|
|
14
|
-
empty_answer_retry: %i[iteration attempt of stopped_by],
|
|
14
|
+
empty_answer_retry: %i[iteration attempt of stopped_by malformed],
|
|
15
15
|
steer_cut: %i[iteration source],
|
|
16
16
|
question_requested: %i[pending_question],
|
|
17
17
|
question_answered: %i[id answer],
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
Another chi session (the parent) gave you this task. It reads only your last message of each turn; the user may be watching, or not.
|
|
4
4
|
|
|
5
5
|
- Finish the task in this turn if you can. End with one reply the parent can act on without asking back: the result first, then evidence (paths, commands run and what they printed), then anything you could not verify or finish.
|
|
6
|
+
- Do only what the task asks. A question ("why does this spec fail?", "what calls X?", "check whether…") gets a report, not file changes: say what you found and what change you would make, and leave making it to the parent. Edit files only when the task asks you to change, fix, add or write something, and then only the files that takes.
|
|
6
7
|
- The parent may send follow-up messages later; each one is a new turn in this same session, with what you did so far.
|
|
7
8
|
- Work in the directory you were started in. Don't change config, install bundles, edit managed files or write memories unless the task asks for it.
|
|
8
9
|
- You can't delegate further. Don't start other chi sessions, and send notes only to the parent.
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: samagotchi-system
|
|
3
|
-
version: 0.
|
|
3
|
+
version: 0.31.0
|
|
4
4
|
scope: system
|
|
5
5
|
description: Default system memories — identity, self map, config modification protocol, memory guide and the delegated-session rules
|
|
6
6
|
files:
|
|
7
7
|
identity.md: sha256:29c2f7df9f6c5acbc5ec17000a4490741e55e426af15ca0d81c8706edd9bd027
|
|
8
|
-
self_map.md: sha256:
|
|
8
|
+
self_map.md: sha256:d8f93e2a4ce555316216ad63f9644c488e02e3f3dd19f6e8537a1ea1404314ff
|
|
9
9
|
config_modification_protocol.md: sha256:ec0b0c7023dbf03b93879e51cd5cdc38b22d43a2e5984ee11e8542ace077c3be
|
|
10
10
|
memory_guide.md: sha256:17126f5adb9ef7b33c40e920ef339388c8c8c045b38a7de825b08760604db4e4
|
|
11
|
-
delegated.md: sha256:
|
|
11
|
+
delegated.md: sha256:eaaf66830d3ab1346c75812a5fc8fcb9d3dbaa6a3f6bcf47c103155d96e415b5
|
|
@@ -24,7 +24,8 @@
|
|
|
24
24
|
`chi send --new --wait -m TEXT` starts a session the user can watch in the web and prints its
|
|
25
25
|
answer (exit 3: it waits for the user's answer); see "Starting a session".
|
|
26
26
|
- `delegate` hands a task to a child session that runs in parallel and returns only its final
|
|
27
|
-
reply (`delegate_result` waits for it; `
|
|
27
|
+
reply (`delegate_result` waits for it; with `wait: false` chi brings the reply to me by itself
|
|
28
|
+
as a delegate report when the child ends its turn; `session:` sends a follow-up to a child). A child is a
|
|
28
29
|
normal session: it shows in `chi sessions list` with `↳ <parent>`, and the user can steer it
|
|
29
30
|
with `chi --attach <id>`. While I wait for a child, its tool approvals go to my user as my own
|
|
30
31
|
approvals (I see only how each was answered); see `docs/sessions.md` "Delegating".
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "fileutils"
|
|
4
|
+
require_relative "session"
|
|
5
|
+
require_relative "session_inbox"
|
|
6
|
+
require_relative "reply_wait"
|
|
7
|
+
require_relative "steer"
|
|
8
|
+
require_relative "log"
|
|
9
|
+
require_relative "config"
|
|
10
|
+
require_relative "tools/delegate_wait"
|
|
11
|
+
require_relative "tools/delegate_cursor"
|
|
12
|
+
|
|
13
|
+
module Samagotchi
|
|
14
|
+
# Loaded on first use: session_manager requires terminal_ui, which
|
|
15
|
+
# requires KernelLoop and so the tools (a require cycle otherwise).
|
|
16
|
+
autoload :SessionManager, File.expand_path("session_manager", __dir__)
|
|
17
|
+
|
|
18
|
+
# A delegate child's news reaching its parent by itself (the delegate
|
|
19
|
+
# tool's wait: false). The child rings (ChildRing): a small file in the
|
|
20
|
+
# parent's children/ dir. The parent's worker reads the rings
|
|
21
|
+
# (ChildReports) and asks each child what happened, as delegate_result
|
|
22
|
+
# would, and hands the model a report: mid-turn at the next iteration
|
|
23
|
+
# boundary, or in a turn of its own when the parent was idle (Worker).
|
|
24
|
+
# The rings stay on disk until the turn that took them is kept, so a
|
|
25
|
+
# worker that exits, restarts or crashes meanwhile loses nothing.
|
|
26
|
+
module ChildRing
|
|
27
|
+
MODE_KEY = "session.delegate_reports"
|
|
28
|
+
MODES = %w[wake queue off].freeze
|
|
29
|
+
# Turns an idle parent runs for reports in a row with no human input.
|
|
30
|
+
MAX_WAKES_KEY = "session.max_wakes"
|
|
31
|
+
MAX_WAKES_DEFAULT = 10
|
|
32
|
+
|
|
33
|
+
module_function
|
|
34
|
+
|
|
35
|
+
# wake: an idle parent runs a turn for a report; queue: reports wait
|
|
36
|
+
# for its next turn; off: no rings at all.
|
|
37
|
+
def mode
|
|
38
|
+
value = Config.get(MODE_KEY).to_s
|
|
39
|
+
MODES.include?(value) ? value : "wake"
|
|
40
|
+
rescue StandardError
|
|
41
|
+
"wake"
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# Ring +child+'s parent; best effort (a failure is logged, never
|
|
45
|
+
# raised). A parent with no worker is woken on a thread
|
|
46
|
+
# (SessionManager.wake_for_report decides whether it may be).
|
|
47
|
+
# @param child [Session] a delegate child
|
|
48
|
+
# @param why [String] "turn_end", "question" or "crash"
|
|
49
|
+
# @param wake [#call, nil] (parent id) wakes a parent with no worker;
|
|
50
|
+
# nil: SessionManager.wake_for_report on a thread
|
|
51
|
+
# @return [String, nil] the ring file, nil when none was written
|
|
52
|
+
def ring(child, why:, state_dir:, wake: nil)
|
|
53
|
+
return nil if mode == "off"
|
|
54
|
+
return nil unless child&.delegate? && child.parent_id
|
|
55
|
+
# A deleted parent gets no orphan dir.
|
|
56
|
+
return nil unless Session.exist?(child.parent_id, state_dir: state_dir)
|
|
57
|
+
|
|
58
|
+
parent_dir = Session.session_dir(child.parent_id, state_dir: state_dir)
|
|
59
|
+
path = SessionInbox.write_ring(parent_dir, child_id: child.id, why: why)
|
|
60
|
+
Log.info(:worker, "delegate_rang", parent: child.parent_id[0, 8], why: why)
|
|
61
|
+
wake_parent(child.parent_id, state_dir: state_dir, wake: wake) if mode == "wake"
|
|
62
|
+
path
|
|
63
|
+
rescue StandardError => e
|
|
64
|
+
Log.warn(:worker, "delegate_ring_failed", parent: child&.parent_id.to_s[0, 8], error: e.class.name, msg: e.message)
|
|
65
|
+
nil
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# A live parent worker sees the ring on its next pass; one with no
|
|
69
|
+
# worker is woken.
|
|
70
|
+
def wake_parent(parent_id, state_dir:, wake:)
|
|
71
|
+
return if SessionManager.session_owner(parent_id, state_dir: state_dir)
|
|
72
|
+
|
|
73
|
+
if wake
|
|
74
|
+
wake.call(parent_id)
|
|
75
|
+
else
|
|
76
|
+
Thread.new do
|
|
77
|
+
SessionManager.wake_for_report(parent_id, state_dir: state_dir)
|
|
78
|
+
rescue StandardError => e
|
|
79
|
+
Log.warn(:worker, "delegate_wake_failed", parent: parent_id[0, 8], error: e.class.name, msg: e.message)
|
|
80
|
+
end
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
# The parent's side: reads the rings in its children/ dir and turns them
|
|
86
|
+
# into reports for its model. One per worker; the turn thread and the
|
|
87
|
+
# loop thread both call it (never at once: worker turns run on the loop
|
|
88
|
+
# thread, the drain inside them).
|
|
89
|
+
#
|
|
90
|
+
# A turn takes reports (#take, at each boundary) and the worker settles
|
|
91
|
+
# them as the turn ends: #commit when it is kept (completed or canceled):
|
|
92
|
+
# the cursors move on (DelegateCursors) and the rings go; #release when it
|
|
93
|
+
# failed: the next turn reads the same rings again.
|
|
94
|
+
class ChildReports
|
|
95
|
+
SOURCE = Steer::DELEGATE_REPORT
|
|
96
|
+
CLIENT_PREFIX = Steer::CHILD_CLIENT_PREFIX
|
|
97
|
+
|
|
98
|
+
# A report for the model: the text delegate_result would return
|
|
99
|
+
# (DelegateWait.finish, its report variant), the cursor it moves the child to,
|
|
100
|
+
# and the cursor it was read from (a commit after the model moved it
|
|
101
|
+
# itself, with delegate_result, leaves it alone).
|
|
102
|
+
Report = Data.define(:child_id, :text, :before, :after, :rings) do
|
|
103
|
+
def line(mark: nil) = Steer::Line.new(text: text, source: ChildReports::SOURCE, mark: mark)
|
|
104
|
+
def origin = { client_id: "#{ChildReports::CLIENT_PREFIX}#{child_id[0, 8]}" }
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def initialize(session_id:, state_dir:)
|
|
108
|
+
@session_id = session_id
|
|
109
|
+
@state_dir = state_dir
|
|
110
|
+
@session_dir = Session.session_dir(session_id, state_dir: state_dir)
|
|
111
|
+
# child id → the Report this turn took for it (rings merged).
|
|
112
|
+
@taken = {}
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
# Whether a ring this turn hasn't taken waits (the idle loop's check).
|
|
116
|
+
def waiting?
|
|
117
|
+
return false if ChildRing.mode == "off"
|
|
118
|
+
|
|
119
|
+
taken = taken_rings
|
|
120
|
+
SessionInbox.find_ring_files(@session_dir).any? { |file| !taken.include?(file) }
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
# The reports for rings this turn hasn't taken yet, recorded as taken.
|
|
124
|
+
# A ring that turns out to say nothing new (the reply was already given,
|
|
125
|
+
# an approval the user answers on the child's card) is deleted.
|
|
126
|
+
# @return [Array<Report>]
|
|
127
|
+
def take
|
|
128
|
+
return [] if ChildRing.mode == "off"
|
|
129
|
+
|
|
130
|
+
taken = taken_rings
|
|
131
|
+
fresh = SessionInbox.find_ring_files(@session_dir).reject { |file| taken.include?(file) }
|
|
132
|
+
by_child = fresh.group_by { |file| SessionInbox.read_ring(file)&.dig(:child_id) }
|
|
133
|
+
by_child.filter_map do |child_id, rings|
|
|
134
|
+
report = child_id && report_for(child_id, rings)
|
|
135
|
+
unless report
|
|
136
|
+
delete_rings(rings)
|
|
137
|
+
next
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
@taken[child_id] = merge_taken(@taken[child_id], report)
|
|
141
|
+
report
|
|
142
|
+
end
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
# The turn that took the reports was kept: move each child's cursor on
|
|
146
|
+
# (unless something else moved it since: the model's own
|
|
147
|
+
# delegate_result), then delete the rings.
|
|
148
|
+
def commit
|
|
149
|
+
@taken.each_value do |report|
|
|
150
|
+
Tools::DelegateCursors.update(@session_id, report.child_id, state_dir: @state_dir) do |cursor|
|
|
151
|
+
cursor == report.before ? report.after : cursor
|
|
152
|
+
end
|
|
153
|
+
delete_rings(report.rings)
|
|
154
|
+
rescue StandardError => e
|
|
155
|
+
Log.warn(:worker, "delegate_commit_failed", child: report.child_id[0, 8], error: e.class.name, msg: e.message)
|
|
156
|
+
end
|
|
157
|
+
@taken = {}
|
|
158
|
+
end
|
|
159
|
+
|
|
160
|
+
# The turn failed (rolled back): its rings stay for the next one.
|
|
161
|
+
def release
|
|
162
|
+
@taken = {}
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
def taken? = !@taken.empty?
|
|
166
|
+
|
|
167
|
+
private
|
|
168
|
+
|
|
169
|
+
def taken_rings
|
|
170
|
+
@taken.values.flat_map(&:rings)
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
# A second ring for a child taken earlier in this turn: read from where
|
|
174
|
+
# the first report left the cursor, keeping the first one's +before+.
|
|
175
|
+
def merge_taken(earlier, report)
|
|
176
|
+
return report unless earlier
|
|
177
|
+
|
|
178
|
+
report.with(before: earlier.before, rings: earlier.rings + report.rings)
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
# @return [Report, nil] nil when the child has nothing to report
|
|
182
|
+
def report_for(child_id, rings)
|
|
183
|
+
child = Session.load(child_id, state_dir: @state_dir)
|
|
184
|
+
return nil unless child.parent_id == @session_id
|
|
185
|
+
|
|
186
|
+
before = @taken[child_id]&.after || Tools::DelegateCursors.get(@session_id, child_id, state_dir: @state_dir)
|
|
187
|
+
wait = ReplyWait.call(child_id, state_dir: @state_dir, cursor: before.reply_file, baseline: before.baseline,
|
|
188
|
+
timeout: 0)
|
|
189
|
+
return nil unless reportable?(wait)
|
|
190
|
+
|
|
191
|
+
after = before.with(reply_file: wait.status == :done ? wait.file : before.reply_file)
|
|
192
|
+
.with_baseline(ReplyWait.baseline_of(wait.session))
|
|
193
|
+
text = Tools::DelegateWait.finish(wait, child_id, timeout: 0, report: true)
|
|
194
|
+
Report.new(child_id: child_id, text: text, before: before, after: after, rings: rings)
|
|
195
|
+
rescue ArgumentError
|
|
196
|
+
nil # the child is gone
|
|
197
|
+
end
|
|
198
|
+
|
|
199
|
+
# A reply, a turn that left none, a crash, or a question the parent's
|
|
200
|
+
# model decides (its own ask_user_question, the step-limit continue).
|
|
201
|
+
# Approvals and hooks' questions are the user's, on the child's card.
|
|
202
|
+
def reportable?(wait)
|
|
203
|
+
case wait.status
|
|
204
|
+
when :done, :no_reply, :error then true
|
|
205
|
+
when :waiting_for_answer then [nil, "", "continue"].include?(wait.question&.dig(:kind)&.to_s)
|
|
206
|
+
else false
|
|
207
|
+
end
|
|
208
|
+
end
|
|
209
|
+
|
|
210
|
+
def delete_rings(rings)
|
|
211
|
+
rings.each { |file| FileUtils.rm_f(file) }
|
|
212
|
+
end
|
|
213
|
+
end
|
|
214
|
+
end
|
data/lib/samagotchi/client.rb
CHANGED
|
@@ -250,11 +250,13 @@ module Samagotchi
|
|
|
250
250
|
# this request runs on. PromptWarmup's pin, and only while a warm-up
|
|
251
251
|
# still runs there: a request pinned to a slot the server has since
|
|
252
252
|
# cleared skips its prompt cache and prefills everything again
|
|
253
|
+
# @param cache_prompt [Boolean] llama.cpp only: false prefills the whole
|
|
254
|
+
# prompt, reusing nothing cached (a malformed generation's retry)
|
|
253
255
|
# @return [String] the generated text
|
|
254
256
|
def complete(prompt, stop: [], n_predict: nil, model: nil, on_chunk: nil, cancel_controller: nil, on_retry: nil,
|
|
255
|
-
images: [], sampling: {}, slot: nil)
|
|
257
|
+
images: [], sampling: {}, slot: nil, cache_prompt: true)
|
|
256
258
|
images = Array(images)
|
|
257
|
-
request = { stop: stop, n_predict: n_predict, model: model, sampling: sampling, slot: slot }
|
|
259
|
+
request = { stop: stop, n_predict: n_predict, model: model, sampling: sampling, slot: slot, cache_prompt: cache_prompt }
|
|
258
260
|
return stream_completion(LLM::Utf8Scrub.call(prompt), request, on_chunk, cancel_controller, on_retry) if images.empty?
|
|
259
261
|
|
|
260
262
|
# The media marker is random per server process: a restart between the
|
|
@@ -311,6 +313,33 @@ module Samagotchi
|
|
|
311
313
|
prompt_ms: timings["prompt_ms"]&.round)
|
|
312
314
|
end
|
|
313
315
|
|
|
316
|
+
# Whether llama.cpp's slot +slot+ is processing a request now (GET
|
|
317
|
+
# /slots, its is_processing): :busy or :idle. nil when unknown: no slot,
|
|
318
|
+
# another transport, the endpoint off (--no-slots), a slot it doesn't
|
|
319
|
+
# list, a bad body or a network error. One quick attempt (PromptWarmup:
|
|
320
|
+
# a warm-up pinned to a busy slot waits behind that request, and the
|
|
321
|
+
# next turn behind the warm-up).
|
|
322
|
+
# @return [Symbol, nil]
|
|
323
|
+
def slot_status(slot, model: nil)
|
|
324
|
+
return nil unless @transport.native? && slot.is_a?(Integer)
|
|
325
|
+
|
|
326
|
+
model = model.to_s.strip
|
|
327
|
+
query = model.empty? ? "" : "?#{URI.encode_www_form(model: model)}"
|
|
328
|
+
uri = URI("#{@scheme}://#{@host}:#{@port}/slots#{query}")
|
|
329
|
+
response = @http.fetch(uri, Net::HTTP::Get.new(uri), retries: false, check_status: false,
|
|
330
|
+
log_fields: { purpose: "slots" },
|
|
331
|
+
open_timeout: CONTEXT_WINDOW_PROBE_OPEN_TIMEOUT,
|
|
332
|
+
read_timeout: CONTEXT_WINDOW_PROBE_READ_TIMEOUT)
|
|
333
|
+
return nil unless response.code.to_s == "200"
|
|
334
|
+
|
|
335
|
+
entry = Array(JSON.parse(response.body.to_s)).find { |s| s.is_a?(Hash) && s["id"] == slot }
|
|
336
|
+
return nil unless entry&.key?("is_processing")
|
|
337
|
+
|
|
338
|
+
entry["is_processing"] == true ? :busy : :idle
|
|
339
|
+
rescue StandardError
|
|
340
|
+
nil
|
|
341
|
+
end
|
|
342
|
+
|
|
314
343
|
private def stream_completion(prompt, fields, on_chunk, cancel_controller, on_retry)
|
|
315
344
|
uri = completion_uri
|
|
316
345
|
request = Net::HTTP::Post.new(uri)
|
|
@@ -498,7 +527,7 @@ module Samagotchi
|
|
|
498
527
|
URI("#{@scheme}://#{@host}:#{@port}#{@transport.completion_path}")
|
|
499
528
|
end
|
|
500
529
|
|
|
501
|
-
def completion_payload(prompt, stop:, n_predict:, model:, sampling: {}, slot: nil)
|
|
530
|
+
def completion_payload(prompt, stop:, n_predict:, model:, sampling: {}, slot: nil, cache_prompt: true)
|
|
502
531
|
payload = { prompt: prompt, stop: stop, stream: true }
|
|
503
532
|
# A non-positive cap is dropped, which leaves the length to the server
|
|
504
533
|
# (unbounded): 0 here never means "only process the prompt". The
|
|
@@ -508,7 +537,9 @@ module Samagotchi
|
|
|
508
537
|
payload[:model] = model_name if model_name
|
|
509
538
|
if @transport.native?
|
|
510
539
|
# llama.cpp's default today, sent anyway: chi's next turn relies on it.
|
|
511
|
-
|
|
540
|
+
# false only for the retry of a malformed generation (KernelLoop),
|
|
541
|
+
# which a poisoned cache must not produce again.
|
|
542
|
+
payload[:cache_prompt] = cache_prompt != false
|
|
512
543
|
payload[:id_slot] = slot if slot.is_a?(Integer)
|
|
513
544
|
end
|
|
514
545
|
sendable_sampling(sampling).merge(payload)
|
data/lib/samagotchi/config.rb
CHANGED
|
@@ -95,6 +95,11 @@ module Samagotchi
|
|
|
95
95
|
Entry.new(key: "session.keep_empty", yaml_path: %w[session keep_empty], type: :bool, default: false, expose: %i[env config]),
|
|
96
96
|
# The most sessions one session may have delegated and still running (the delegate tool); a guard against a runaway model.
|
|
97
97
|
Entry.new(key: "session.max_children", yaml_path: %w[session max_children], type: :integer, default: 4, expose: %i[env config]),
|
|
98
|
+
# A delegate child's turn end reaches its parent (ChildReports): wake runs an idle parent's turn for it, queue waits
|
|
99
|
+
# for the parent's next turn, off is the old behaviour (only delegate_result asks).
|
|
100
|
+
Entry.new(key: "session.delegate_reports", yaml_path: %w[session delegate_reports], type: :enum, default: "wake", expose: %i[env config], enum_values: %w[wake queue off]),
|
|
101
|
+
# Turns a parent runs for delegate reports in a row with no human input; past it reports wait for the next turn.
|
|
102
|
+
Entry.new(key: "session.max_wakes", yaml_path: %w[session max_wakes], type: :integer, default: 10, expose: %i[env config]),
|
|
98
103
|
|
|
99
104
|
# Images sent to a model (ImageStore): the long side they are downscaled
|
|
100
105
|
# to, the most bytes one may take (bigger → re-encoded as jpeg), and how
|
data/lib/samagotchi/engine.rb
CHANGED
|
@@ -61,6 +61,7 @@ require_relative "thinking"
|
|
|
61
61
|
require_relative "answer_display"
|
|
62
62
|
require_relative "edit_preview"
|
|
63
63
|
require_relative "question_desk"
|
|
64
|
+
require_relative "child_reports"
|
|
64
65
|
require_relative "relay_desk"
|
|
65
66
|
require_relative "guardrail_wiring"
|
|
66
67
|
require_relative "plugin_tasks"
|
|
@@ -1190,6 +1191,7 @@ module Samagotchi
|
|
|
1190
1191
|
def record_last_turn(session, outcome, seconds, origin, event, limit: nil)
|
|
1191
1192
|
client_id = origin.is_a?(Hash) ? origin[:client_id].to_s : ""
|
|
1192
1193
|
source = if client_id.start_with?("#{Tools::Delegate::CLIENT_PREFIX}:") then "delegate"
|
|
1194
|
+
elsif client_id.start_with?(ChildReports::CLIENT_PREFIX) then "delegate_report"
|
|
1193
1195
|
elsif client_id == SessionManager::REMINDER_CLIENT_ID then "reminder"
|
|
1194
1196
|
else "client"
|
|
1195
1197
|
end
|
|
@@ -54,6 +54,7 @@ module Samagotchi
|
|
|
54
54
|
TOOL_OUTPUT_HEADER_RE = /\A\[([\w.:-]+)\]/
|
|
55
55
|
# The text the web sends for a turn of images alone (images.js turnText).
|
|
56
56
|
IMAGE_ONLY_RE = /\A\s*(?:\[image: [^\]\n]*\]\s*)+\z/
|
|
57
|
+
DELEGATE_REPORT_LINE = "(a delegate session's report, delivered by chi; not the user's words)"
|
|
57
58
|
|
|
58
59
|
module_function
|
|
59
60
|
|
|
@@ -71,6 +72,9 @@ module Samagotchi
|
|
|
71
72
|
# An image is a line naming it (refs only, never its bytes), unless
|
|
72
73
|
# the text is only the web's placeholder naming it already.
|
|
73
74
|
content = message["content"].to_s
|
|
75
|
+
# A delegate child's report (ChildReports) is chi's news, not
|
|
76
|
+
# what the user said.
|
|
77
|
+
content = "#{DELEGATE_REPORT_LINE}\n#{content}" if message["source"].to_s == Steer::DELEGATE_REPORT
|
|
74
78
|
images = IMAGE_ONLY_RE.match?(content) ? [] : image_lines(message["images"])
|
|
75
79
|
[content, *images].reject(&:empty?).join("\n")
|
|
76
80
|
when "model", "assistant"
|
|
@@ -97,6 +97,8 @@ module Samagotchi
|
|
|
97
97
|
|
|
98
98
|
DEFAULT_MAX_TOOL_OUTPUT_CHARS = 10_000
|
|
99
99
|
QWEN_INCOMPLETE_TOOL_CALL_RECOVERY_LIMIT = 2
|
|
100
|
+
# How much of a malformed generation's end the warning keeps.
|
|
101
|
+
MALFORMED_TAIL_CHARS = 400
|
|
100
102
|
QWEN_INCOMPLETE_TOOL_CALL_RECOVERY_PROMPT = "Continue the previous assistant message by finishing the open <tool_call> XML block. Output only the remaining XML needed to complete the tool call."
|
|
101
103
|
|
|
102
104
|
# @param tools [Tools::Registry, nil] the tools calls dispatch to (the
|
|
@@ -242,7 +244,8 @@ module Samagotchi
|
|
|
242
244
|
# the prefill, and what the caller gave.
|
|
243
245
|
Turn = Struct.new(:conversation, :context, :empty_retry, :tool_activity, :buffer, :streamed_thinking, :qwen_attempts,
|
|
244
246
|
:qwen_partial, :prefill, :pending_tool_calls, :model_name, :pending_input, :on_stream_event,
|
|
245
|
-
:cancel_controller, :cap, :emit, :iteration, :empty_steps, :ended_empty,
|
|
247
|
+
:cancel_controller, :cap, :emit, :iteration, :empty_steps, :ended_empty, :malformed_retried,
|
|
248
|
+
:uncached_next, keyword_init: true)
|
|
246
249
|
# One request: the prompt and its images as sent, the images' token
|
|
247
250
|
# estimate, and the window it was measured against.
|
|
248
251
|
Request = Struct.new(:prompt, :images, :image_tokens, :window, :prefill, keyword_init: true)
|
|
@@ -250,9 +253,10 @@ module Samagotchi
|
|
|
250
253
|
# generation's own server counts, the model the server named, the
|
|
251
254
|
# thinking it streamed, and why it stopped (the transport's finish
|
|
252
255
|
# reason, Client::Transport#finish_reason_from; nil when not named),
|
|
253
|
-
# and its prompt-cache counts (the largest seen: they come cumulative)
|
|
256
|
+
# and its prompt-cache counts (the largest seen: they come cumulative),
|
|
257
|
+
# and where its visible text starts in the turn's buffer.
|
|
254
258
|
Generation = Struct.new(:response, :cut, :usage, :served_model, :streamed_thinking, :finish_reason, :cache,
|
|
255
|
-
keyword_init: true)
|
|
259
|
+
:buffer_mark, keyword_init: true)
|
|
256
260
|
private_constant :Turn, :Request, :Generation
|
|
257
261
|
|
|
258
262
|
def start_turn(messages, on_stream_event:, cancel_controller:, pending_input:, cap:)
|
|
@@ -278,6 +282,9 @@ module Samagotchi
|
|
|
278
282
|
request = prepare_request(turn)
|
|
279
283
|
generation = generate(turn, request)
|
|
280
284
|
return after_cut(turn, generation) if generation.cut
|
|
285
|
+
if (reason = parser.malformed(generation.response))
|
|
286
|
+
return after_malformed(turn, generation, reason)
|
|
287
|
+
end
|
|
281
288
|
|
|
282
289
|
turn.conversation << { role: "model", content: request.prefill + generation.response.to_s }
|
|
283
290
|
# Profile-specific parse (incl. Qwen unterminated-block recovery); the
|
|
@@ -330,10 +337,13 @@ module Samagotchi
|
|
|
330
337
|
# thinking_chars=N content_length=…).
|
|
331
338
|
generation = Generation.new(usage: nil, served_model: nil, streamed_thinking: 0)
|
|
332
339
|
fire_hook(:before_generation, { type: :before_generation, iteration: turn.iteration }) if @hooks
|
|
333
|
-
buffer_mark = turn.buffer.length
|
|
340
|
+
buffer_mark = generation.buffer_mark = turn.buffer.length
|
|
334
341
|
# A cancel salvages only this generation's thinking: the earlier
|
|
335
342
|
# steps' is on their own messages already.
|
|
336
343
|
turn.streamed_thinking = +""
|
|
344
|
+
# Only the retry of a malformed generation skips the prompt cache.
|
|
345
|
+
uncached = turn.uncached_next
|
|
346
|
+
turn.uncached_next = false
|
|
337
347
|
generation.response = with_generation(turn.cancel_controller) do |generation_controller|
|
|
338
348
|
request_generation(
|
|
339
349
|
request.prompt,
|
|
@@ -341,6 +351,7 @@ module Samagotchi
|
|
|
341
351
|
cancel_controller: turn.cancel_controller,
|
|
342
352
|
model_name: turn.model_name,
|
|
343
353
|
sampling: turn.empty_retry.request_sampling(@turn_settings.sampling),
|
|
354
|
+
cache_prompt: !uncached,
|
|
344
355
|
on_chunk: ->(chunk) { stream_chunk(turn, generation, stream_splitter, chunk) },
|
|
345
356
|
on_retry: lambda { |retry_event|
|
|
346
357
|
# The retry streams from the start: its counts replace these.
|
|
@@ -462,6 +473,31 @@ module Samagotchi
|
|
|
462
473
|
cancelled_result(turn.conversation, tool_activity: turn.tool_activity, reason: :hook, partial_assistant_text: "")
|
|
463
474
|
end
|
|
464
475
|
|
|
476
|
+
# A corrupt generation (ToolCallParser#malformed) is no answer and runs
|
|
477
|
+
# no tool: it is logged and dropped, with its streamed text, and the
|
|
478
|
+
# same prompt goes once more without the server's prompt cache (a
|
|
479
|
+
# poisoned cache made it, ggml-org/llama.cpp#27148), shown as the loop
|
|
480
|
+
# asking again. A second one fails the turn. Returns :next.
|
|
481
|
+
def after_malformed(turn, generation, reason)
|
|
482
|
+
response = generation.response.to_s
|
|
483
|
+
Log.warn(:model, "generation_malformed", iteration: turn.iteration, reason: reason, id_slot: @last_slot,
|
|
484
|
+
cache_n: generation.cache&.dig(:cached), finish_reason: generation.finish_reason,
|
|
485
|
+
retried: turn.malformed_retried ? true : false,
|
|
486
|
+
payload: response[-MALFORMED_TAIL_CHARS..] || response)
|
|
487
|
+
turn.buffer.slice!(generation.buffer_mark..)
|
|
488
|
+
if turn.malformed_retried
|
|
489
|
+
label = @client.respond_to?(:host_name) && @client.host_name ? @client.host_name : "llama.cpp"
|
|
490
|
+
raise LLM::MalformedGeneration.new("#{label}: #{reason}, again without the prompt cache; nothing was saved",
|
|
491
|
+
host: label)
|
|
492
|
+
end
|
|
493
|
+
|
|
494
|
+
turn.malformed_retried = true
|
|
495
|
+
turn.uncached_next = true
|
|
496
|
+
emit(turn, type: :empty_answer_retry, iteration: turn.iteration, attempt: 1, of: 1,
|
|
497
|
+
finish_reason: generation.finish_reason, malformed: true)
|
|
498
|
+
:next
|
|
499
|
+
end
|
|
500
|
+
|
|
465
501
|
# A generation with no tool calls: a Qwen call left open is asked to be
|
|
466
502
|
# finished; an empty answer is asked again (EmptyAnswerRetry: not when
|
|
467
503
|
# a length stop came with the context full); queued input keeps the
|
|
@@ -612,8 +648,11 @@ module Samagotchi
|
|
|
612
648
|
end
|
|
613
649
|
|
|
614
650
|
# +sampling+: this request's (EmptyAnswerRetry#request_sampling).
|
|
615
|
-
|
|
651
|
+
# +cache_prompt+: false for a malformed generation's retry (only then sent).
|
|
652
|
+
def complete_kwargs(cancel_controller:, model_name: nil, on_chunk: nil, on_retry: nil, images: [], sampling: nil,
|
|
653
|
+
cache_prompt: true)
|
|
616
654
|
kwargs = {}
|
|
655
|
+
kwargs[:cache_prompt] = false unless cache_prompt
|
|
617
656
|
# Only a request with images names them: a text-only call is unchanged.
|
|
618
657
|
kwargs[:images] = images unless images.empty?
|
|
619
658
|
kwargs[:on_chunk] = on_chunk if on_chunk
|