samagotchi 0.24.0 → 0.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +54 -1
  3. data/docs/architecture.md +16 -9
  4. data/docs/cli.md +4 -3
  5. data/docs/configuration.md +21 -13
  6. data/docs/hooks.md +1 -1
  7. data/docs/internals/background-tasks.md +1 -0
  8. data/docs/internals/prompt-caching.md +94 -0
  9. data/docs/plugins.md +16 -2
  10. data/docs/testing.md +12 -0
  11. data/lib/samagotchi/bundles/loop-guard/manifest.yml +2 -2
  12. data/lib/samagotchi/bundles/loop-guard/plugin.rb +43 -7
  13. data/lib/samagotchi/bundles/system/manifest.yml +1 -1
  14. data/lib/samagotchi/command_steps.rb +233 -0
  15. data/lib/samagotchi/config.rb +5 -2
  16. data/lib/samagotchi/empty_answer_retry.rb +2 -2
  17. data/lib/samagotchi/engine.rb +23 -4
  18. data/lib/samagotchi/guardrails/shell_lex.rb +49 -5
  19. data/lib/samagotchi/idle_client.rb +14 -2
  20. data/lib/samagotchi/kernel_loop.rb +21 -3
  21. data/lib/samagotchi/llm/chat_loop.rb +17 -3
  22. data/lib/samagotchi/llm/errors.rb +26 -1
  23. data/lib/samagotchi/llm/openai_chat.rb +5 -4
  24. data/lib/samagotchi/llm/prompt_cache.rb +34 -9
  25. data/lib/samagotchi/llm/usage.rb +18 -3
  26. data/lib/samagotchi/log_subscriber.rb +2 -1
  27. data/lib/samagotchi/plugin/api.rb +3 -2
  28. data/lib/samagotchi/session_metrics.rb +38 -4
  29. data/lib/samagotchi/system_prompt.rb +30 -9
  30. data/lib/samagotchi/terminal_ui/attached_loop.rb +5 -2
  31. data/lib/samagotchi/terminal_ui/attached_view.rb +6 -2
  32. data/lib/samagotchi/terminal_ui/event_renderer.rb +14 -2
  33. data/lib/samagotchi/terminal_ui/formatting.rb +31 -9
  34. data/lib/samagotchi/thinking_tails.rb +103 -0
  35. data/lib/samagotchi/token_usage.rb +18 -6
  36. data/lib/samagotchi/tool_activity.rb +37 -3
  37. data/lib/samagotchi/tool_declarations.rb +14 -0
  38. data/lib/samagotchi/tool_view.rb +40 -4
  39. data/lib/samagotchi/tools/builtins.rb +4 -1
  40. data/lib/samagotchi/tools/execute.rb +2 -1
  41. data/lib/samagotchi/tools/registry.rb +11 -4
  42. data/lib/samagotchi/tools/shell.rb +32 -0
  43. data/lib/samagotchi/tools/task_runtime.rb +3 -3
  44. data/lib/samagotchi/turn_tally.rb +17 -6
  45. data/lib/samagotchi/version.rb +1 -1
  46. data/lib/samagotchi/web/public/command_view.js +71 -3
  47. data/lib/samagotchi/web/public/format.js +11 -1
  48. data/lib/samagotchi/web/public/index.html +25 -0
  49. data/lib/samagotchi/web/public/stage_model.js +5 -1
  50. data/lib/samagotchi/web/public/stage_view.js +1 -1
  51. data/lib/samagotchi/web/public/turn_view.js +3 -3
  52. metadata +5 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6d6348260d3f666ffbfc0b8c3f10a51a2c64c3a30ce9cbd5a5c1280fa0325306
4
- data.tar.gz: 0cad64155e27a000a49c82e89778d69804fc609526418c1e03846a32d2fb864c
3
+ metadata.gz: 6abdccc8a805a545ad1e27898b3caf43516d16ef1b7832444bac7b6e0a4b5e63
4
+ data.tar.gz: 8c0e0f354030f8ad954afdfd9dff96ef54d0b1fdd79d8df190cf494a8a32b18d
5
5
  SHA512:
6
- metadata.gz: b9a1f022f8a225c517e3160d0d512fbb5a97750c32a6df7769bbcb91ddff5534c5b12f540c4ef80871d8e4f2f80d460c71cd56107476435a69d54cef0ef78395
7
- data.tar.gz: 419a527ffca2330de67b42544f619db872c06388ded821d3bc5e7f17f07a488a7adf96ce56d55032a53c4bc21e6076d32af398c0b8aff15c48e651e6596e7873
6
+ metadata.gz: 36530955a235e3d73adaa4496545c12f6abea21daf1ff0a40eaba9477c22e1e7007dced1a500912878164c0a9f1a9ec76b45d4d8a34709473d8c496b1d53ce69
7
+ data.tar.gz: 125d6ab7dd048082ff6296d663803da9d6a8073c7ead520f2264ec4cc27c28d29bdb772addd47f2be39b9aa497308ef09236bd5e3e1bc43854341970fc8e6a6e
data/CHANGELOG.md CHANGED
@@ -8,6 +8,58 @@ and commands may change between minor versions. How releases are made:
8
8
 
9
9
  ## [Unreleased]
10
10
 
11
+ ## [0.25.0] - 2026-10-04
12
+
13
+ ### Added
14
+
15
+ - The web shows an `execute` command as the steps a person reads: `cd X &&` becomes an "in X" tag, `| head -20` a
16
+ chip, `echo "=== x ==="` a label, a heredoc a collapsed "EOF · N lines" chip; the collapsed row's title is the
17
+ first step `+N`. A `raw` toggle shows the whole command; anything the parser can't read shows as before.
18
+ - `execute` takes an optional `description` ("what the command does, in a few words"), written by the model; it
19
+ becomes the tool row's title in the web and replaces the cut command in the terminal's tool line. Turn it off
20
+ with `execute.description: false`.
21
+ - Every generation logs its prompt-cache counts (`prompt=`, `cached=`, `cache_write=`), and `/stats` shows cache
22
+ writes next to cached tokens.
23
+ - A generation cut mid-thinking (loop-guard, a plugin's stop, a steer) or ended at the provider's output cap keeps
24
+ the last 20k chars of its thinking in the session's folder, `thinking_tails.jsonl`, so an archived session holds
25
+ the loop it was archived for. The model never sees it.
26
+ - Turn records count `cuts` (generations a plugin cut) and `capped` (generations that hit the output cap), and
27
+ `/stats` shows them as "thinking cuts" and "output cap hits": `retries` counts network retries only.
28
+
29
+ ### Changed
30
+
31
+ - On macOS, `execute`, `task_create` and `!cmd` run commands in `/bin/zsh` emulating sh (with bash's `{1..3}`,
32
+ `[[ =~ ]]` groups and echo escapes kept) instead of `/bin/sh`, which is bash 3.2 there and failed on an apostrophe
33
+ in a heredoc inside `"$( )"`: `git commit -m "$(cat <<'EOF' … it's … EOF )"` now works. Linux keeps
34
+ `/bin/sh`; see [background tasks](docs/internals/background-tasks.md).
35
+ - loop-guard's cut notice (bundle 0.3.3) quotes the looping sentence: `thinking repeats itself ("Let me write the
36
+ tool call.", one sentence ×8, 33k chars, 63 s): cut`.
37
+ - The system prompt puts what every session shares first and the per-session lines (model, working directory,
38
+ session) last, and plugin tools come in a fixed order, so a new session reuses the server's prompt cache: on a
39
+ local llama.cpp or Splash server its first answer starts in about 0.5 s instead of 4–6 s, and on Claude via
40
+ OpenRouter a new session reads the shared part from the cache (about 7× cheaper to start).
41
+ - Chat hosts (`api: openai`) are no longer asked for `temperature: 0.0`: chi sends no temperature unless one is
42
+ configured, so each model runs at its provider's default (as native llama.cpp hosts already did). Greedy decoding
43
+ made DeepSeek v4.1-flash loop in its thinking. The idle recap and `/btw` follow suit. Set `sampling: {temperature:
44
+ …}` on a `hosts:` or `models:` entry to pin one; the empty-answer retry still runs at 0.6 unless one is set.
45
+
46
+ ### Fixed
47
+
48
+ - loop-guard (bundle 0.3.3) cuts thinking that loops in a cycle of 7 or 8 short sentences, or with a longer
49
+ sentence ("I'll write the spec file now.") in the cycle: 48 sentences in a row with 12 or fewer different ones are
50
+ a loop. New settings `thinking.window_sentences` (48) and `thinking.window_distinct` (12).
51
+ - A steer row names chi's senders in words: the web's row and trail flash and the terminal's nudged line say
52
+ `parent agent`, `chi send` and `plugin` instead of the raw `parent_agent` / `chi_send` / `plugin_send`.
53
+ - Thinking off on Gemma 4 no longer warns that off wasn't honoured: its empty thought (whitespace alone) is not
54
+ counted as thinking.
55
+ - An OpenRouter 402 whose `metadata.reason` is `weight_exceeds_budget` (the request alone is larger than the
56
+ key's credit budget) fails at once as `request too large for the credit budget on host <name>: …; lower
57
+ max_tokens (default.max_tokens) or raise the key's credit limit`, instead of being retried as credit held by
58
+ in-flight requests. Other 402s are unchanged.
59
+
60
+ Update with `chi update`: loop-guard moves to 0.3.3 (the window rule and the quoted cut notice). Restart `chi web`
61
+ and running sessions afterwards (`chi sessions restart ID`).
62
+
11
63
  ## [0.24.0] - 2026-10-04
12
64
 
13
65
  ### Added
@@ -1342,7 +1394,8 @@ and long-lived sessions.
1342
1394
  - A macOS desktop helper (`chi desktop install`): a "Send to chi" Service and a
1343
1395
  hotkey panel that send selected text or the clipboard to your sessions.
1344
1396
 
1345
- [Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.24.0...HEAD
1397
+ [Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.25.0...HEAD
1398
+ [0.25.0]: https://github.com/dm1try/samagotchi/compare/v0.24.0...v0.25.0
1346
1399
  [0.24.0]: https://github.com/dm1try/samagotchi/compare/v0.23.0...v0.24.0
1347
1400
  [0.23.0]: https://github.com/dm1try/samagotchi/compare/v0.22.0...v0.23.0
1348
1401
  [0.22.0]: https://github.com/dm1try/samagotchi/compare/v0.21.0...v0.22.0
data/docs/architecture.md CHANGED
@@ -184,12 +184,18 @@ engine.system_prompt # fully built system prompt string
184
184
  engine.session # current session (Engine owns create/resume)
185
185
  ```
186
186
 
187
- The system prompt (`SystemPrompt#build`) is the base prompt, then rg guidance,
188
- AGENT.md, the working directory, **the model** (`Model: this session runs on
189
- <ref> (host …; model key …)` and a line telling the model to answer "which model
190
- are you" from it, not from training), the session id and log, the memory indexes,
191
- the identity and the preloaded memories. It is built once per loop and rebuilt only
192
- by a model switch (or changed tools), so the model line costs no KV churn.
187
+ The system prompt (`SystemPrompt#build`) is the base prompt, then rg guidance, the
188
+ identity and the preloaded memories, AGENT.md, the memory indexes, and last **the
189
+ model** (`Model: this session runs on <ref> (host …; model key …)` and a line telling
190
+ the model to answer "which model are you" from it, not from training), the working
191
+ directory and the session id and log (Gemma's native tool declarations still follow).
192
+ The order is for the servers' prompt caches, which reuse only an exact token prefix:
193
+ what every session shares first, what changes least before what changes more, the
194
+ per-session lines last, so a new session reuses everything above them. It is built
195
+ once per loop and rebuilt only by a model switch (or changed tools), so the model
196
+ line costs no KV churn.
197
+ The full request layout, what breaks the cache and the rules for changes:
198
+ [internals/prompt-caching.md](internals/prompt-caching.md).
193
199
 
194
200
  #### Subscribing to the live stream (and the bridge)
195
201
 
@@ -275,11 +281,12 @@ The metrics in `/state`, `/snapshot`, `/stats` and the SSE `snapshot`/`reset` fr
275
281
  newest turn the worker finished and any it hasn't saved yet (at most 20 turns), with their tool
276
282
  calls and the running turn's finished ones. The full history is the session's `analytics.json`,
277
283
  which `SessionMetrics#persist` rewrites after each turn; the web server merges the two by id.
278
- The totals' `tokens` block adds the cached and reasoning sums, the cost, the decode time and tokens
284
+ The totals' `tokens` block adds the cached, cache-write and reasoning sums, the cost, the decode time and tokens
279
285
  behind the average speed (`avg_decode_tps`) and the last speeds (`last_decode_tps`,
280
286
  `last_prefill_tps`, `tps_source`: `server` or `estimate`). Each `generation_completed` event
281
- carries the generation's `speed` (`{decode_tps, source}`, null without one) and those running
282
- `tokens`: the Engine closes the generation in `SessionMetrics#finish_generation` before relaying
287
+ carries the generation's `speed` (`{decode_tps, source}`, null without one), its own prompt-cache
288
+ counts (`prompt_tokens`, `cached_tokens`, `cache_write_tokens`: null when the server reported no writes) and
289
+ those running `tokens`: the Engine closes the generation in `SessionMetrics#finish_generation` before relaying
283
290
  it, so the web's info bar updates per generation without parsing chunk payloads.
284
291
 
285
292
  The page's frequent reads of `GET /api/sessions/:id` stay light too:
data/docs/cli.md CHANGED
@@ -420,9 +420,10 @@ file and index line stay as they are; only this session doesn't see it.
420
420
  `/help` lists the commands this session takes, the installed bundles' too, each
421
421
  with a line on what it does. `/stats` shows the session's numbers: turns, tool
422
422
  calls (by tool, with errors), iterations, tokens in/out summed over every
423
- request (with the cached and reasoning tokens when the server reports them),
424
- the decode speed, generation latency, cancellations, retries, context used
425
- and window, the prompt profile and the model the server says it ran, and the
423
+ request (with the cached tokens, the cache writes and the reasoning tokens when the server reports them),
424
+ the decode speed, generation latency, cancellations, retries (the network's),
425
+ thinking cuts (generations a plugin such as loop-guard cut) and output cap hits
426
+ (generations that ended at the provider's cap), context used and window, the prompt profile and the model the server says it ran, and the
426
427
  cost when the provider reports one (OpenRouter's `usage.cost`; no prices are
427
428
  guessed). The speed line is the last generation's and the session's average
428
429
  (weighted by tokens): llama.cpp reports it exactly (`87 tok/s out, 1.9k tok/s
@@ -369,9 +369,9 @@ after an idle exit) gets that process's environment, so put a lasting choice in
369
369
 
370
370
  ## Sampling
371
371
 
372
- chi asks chat hosts (`api: openai`) for greedy decoding (`temperature: 0.0`) and sends native hosts no sampling
373
- fields, so their server's defaults apply (llama.cpp: temperature 0.8). `sampling:` on a `hosts:` entry or a `models:`
374
- entry sets request fields for the model's turns:
372
+ chi sends no sampling fields by default, to chat hosts (`api: openai`) or native ones, so each model runs at its
373
+ provider's defaults (llama.cpp: temperature 0.8; vLLM takes them from the model's `generation_config.json`).
374
+ `sampling:` on a `hosts:` entry or a `models:` entry sets request fields for the model's turns:
375
375
 
376
376
  ```yaml
377
377
  hosts:
@@ -394,15 +394,16 @@ models:
394
394
  - A `models:` entry's fields win over its host's, field by field (the host can set a penalty and the model move only
395
395
  the temperature). The entry is found the way `profile:` is (the name as typed, alias-resolved or without its host
396
396
  prefix).
397
- - `temperature: null` (or `~`) sends no temperature, so the provider's default applies (vLLM takes it from the
398
- model's `generation_config.json`).
397
+ - `temperature: null` (or `~`) in a model's entry takes back a host's temperature, so the provider's default applies;
398
+ it also keeps the empty-answer retry (below) at that default.
399
399
  - Fields chi sets itself are refused with a warning: `model`, `messages`, `prompt`, `stream`, `stream_options`,
400
400
  `tools`, `tool_choice`, `stop`, `n_predict`, `max_tokens`, `n`, `parallel_tool_calls`, `response_format`,
401
401
  `cache_prompt`. A `sampling:` that isn't a map warns and is skipped.
402
- - Greedy decoding can make a heavily quantized thinking model loop in its reasoning ("Let me write the reply…"
403
- for minutes) or end with an empty answer. Qwen's own advice for its thinking models is `temperature: 0.6,
404
- top_p: 0.95` (not greedy), with `presence_penalty` between 0 and 2 against endless repetition.
405
- - The idle recap and side questions keep their own short, deterministic settings.
402
+ - Greedy decoding (`temperature: 0`) can make a thinking model loop in its reasoning ("Let me write the reply…"
403
+ for minutes) or end with an empty answer; set it only for a model you have watched. Qwen's own advice for its
404
+ thinking models is `temperature: 0.6, top_p: 0.95`, with `presence_penalty` between 0 and 2 against endless
405
+ repetition.
406
+ - The idle recap and side questions don't use `sampling:`; they send their own short output cap and no temperature.
406
407
 
407
408
  `/model` shows what applies (`sampling: temperature=0.6 presence_penalty=1.5 (hosts.work)`), and each request's
408
409
  `stream` line in the debug log carries a `sampling=` field with what was sent. The fields are read every turn, so a
@@ -514,10 +515,14 @@ gateway that spreads requests over several providers can key on it to keep one c
514
515
  prompt caches hit and every turn is answered by the same model. Servers that don't know the header ignore it.
515
516
 
516
517
  Chat requests for a Claude model (an id containing `claude`, such as `anthropic/claude-sonnet-5.5`) on a remote
517
- OpenAI-compatible host get two prompt-cache breakpoints (`cache_control` on the system message and on the last
518
- message), so each step reads the earlier prompt from Anthropic's cache instead of paying for it again; the debug
518
+ OpenAI-compatible host get two prompt-cache breakpoints (`cache_control` on the system prompt and on the last
519
+ message), so each step reads the earlier prompt from Anthropic's cache instead of paying for it again. The system
520
+ prompt goes as two text parts, the breakpoint after the first: everything sessions share, then the model, working
521
+ directory and session lines, so a new session reads the shared part from the cache as well; the debug
519
522
  log's line shows `cache=on`. Nothing to configure. A provider behind a gateway may still not cache: check the
520
- cached tokens in `/stats`.
523
+ cached tokens in `/stats`. Each request's own counts are in the debug log's `generation_completed` line:
524
+ `prompt=` (its prompt tokens), `cached=` (read from the server's cache) and `cache_write=` (written to it, when the
525
+ server reports writes); recap and side requests log theirs as `recap request_usage`.
521
526
 
522
527
  ## Llama Model Routing
523
528
 
@@ -616,6 +621,8 @@ Transient network failures are retried automatically with exponential backoff.
616
621
  - An HTTP 402 about credit held by in-flight requests (OpenRouter reserves credit for each running request, so
617
622
  one session's long request can make another's fail) is retried like a 429, 20s apart unless `Retry-After`
618
623
  says otherwise: with the default `retry.max` that is about 100s. With `retry.max: 0` it fails at once.
624
+ Not when its `metadata.reason` is `weight_exceeds_budget`: that request alone is larger than the key's credit
625
+ budget, so it fails at once (lower `default.max_tokens` or raise the key's limit).
619
626
  - A refused connection (nothing listening) is not retried: the turn fails at once with `can't reach host <name> at
620
627
  <address> (connection refused) — is the server running?`.
621
628
  - A stream that has already produced output is never retried (the retry would repeat it); it fails the turn.
@@ -671,7 +678,7 @@ message (before, a failed llama.cpp `/completion` ended the turn as
671
678
  | connection | reset, timed out, dropped mid-stream; refused | yes (network retry), not mid-stream; refused: no |
672
679
  | rate limited | HTTP 429 | yes, honouring `Retry-After` |
673
680
  | credits held | HTTP 402 "… in-flight requests" | yes, after 20s (or `Retry-After`) |
674
- | credits | any other HTTP 402: out of credits | no |
681
+ | credits | any other HTTP 402: out of credits; one whose `metadata.reason` is `weight_exceeds_budget` (the request alone is over the key's budget: lower `default.max_tokens`) | no |
675
682
  | server | HTTP 5xx, llama.cpp's mid-stream `error:` event | 500/502/503/504/529 only |
676
683
  | auth | HTTP 401/403 | no |
677
684
  | bad request | other 4xx; a prompt larger than the context window, whatever the status | no |
@@ -912,6 +919,7 @@ described in their own sections.
912
919
  | `execute.preview_bytes` | `12288` | yes | |
913
920
  | `execute.telemetry_threshold_pct` | `80` | yes | |
914
921
  | `execute.timeout_sec` | `120` | yes | Seconds one `execute` command may run before it is stopped. |
922
+ | `execute.description` | `true` | | `execute` offers the model an optional `description` (a few words on what the command does), shown as the tool row's title in `chi web` and in place of the command in the TUI's tool line. `false`: the parameter isn't declared, and rows show the command's first step. |
915
923
  | `web.port` | `4567` | `--port` | `chi web`'s port. See [CLI](cli.md). |
916
924
  | `web.host` | `127.0.0.1` | yes | `127.0.0.1`, `::1` or `localhost`; `lan` (this machine's private IPv4 address) or one of its IPv4 addresses also opens `chi web` to the network, with an access token (`chi web --new-token` replaces it). Anything else binds `127.0.0.1` with a warning. See [CLI: chi web on your phone](cli.md#chi-web-on-your-phone). |
917
925
  | `web.markdown` | `false` | yes | Render answers as Markdown in `chi web`. |
data/docs/hooks.md CHANGED
@@ -399,7 +399,7 @@ stripped of surrounding whitespace, except file text, an edit's
399
399
 
400
400
  | Tool | `content:` from | `path:` from | Own fields |
401
401
  |------|-----------------|--------------|------------|
402
- | `execute` | `command` | — | `cwd` |
402
+ | `execute` | `command` | — | `description`, `cwd` |
403
403
  | `read` | `path` | — | `start_line`, `end_line` |
404
404
  | `write` | — | `path` | `content` |
405
405
  | `edit` | — | `path` | `old_text`, `new_text`, `start_line`, `end_line` |
@@ -17,6 +17,7 @@ Recommended workflow:
17
17
 
18
18
  Behavior:
19
19
 
20
+ - `execute`, `task_create` and the `!cmd` shell bang run the command in a non-login shell: `/bin/sh -c` on Linux; on macOS, when `/bin/zsh` is there, `/bin/zsh --emulate sh +o ignore_braces +o sh_glob -o bash_rematch +o bsd_echo -c` (macOS's `/bin/sh` is bash 3.2, which can't parse an apostrophe in a heredoc inside `"$( )"`, the usual commit message shape). That mode reads no rc files and keeps POSIX word splitting, unmatched globs left as is, `{1..3}`, `[[ … =~ (…) ]]` with `BASH_REMATCH`, arrays, `$'…'`, `&>` and echo expanding `\t`. Bash-only builtins and variables (`shopt`, `BASH_VERSION`, `${!prefix*}`, `read -p`) are not there.
20
21
  - Task metadata and output are persisted under `tmp/tasks/`.
21
22
  - Task listing is workspace-scoped (current project only).
22
23
  - `task_get` returns metadata and `output_path`; use `read` for output contents.
@@ -0,0 +1,94 @@
1
+ # Prompt caching: how a request is laid out, and how to keep it cache-friendly
2
+
3
+ A model server reuses work only for the **exact token prefix** it has already seen. Change one token and
4
+ everything after it is computed again (prefill). The local models we run (hybrid Qwen 3.6 / Ornith on llama.cpp
5
+ and Splash) can only resume from a **restore point near the end** of an earlier prompt, not from any shared
6
+ prefix. Anthropic (Claude via OpenRouter) caches only up to an explicit `cache_control` breakpoint. So the order
7
+ of what chi sends decides what a new turn, or a new session, has to pay for.
8
+
9
+ Measured with prompt-cache slice 1 (2026-10-04, fresh sessions, ~7.7k-token prompt):
10
+
11
+ | Server | Before | After |
12
+ |---|---|---|
13
+ | Ornith on llama.cpp | 0 tokens reused, 6.4 s prefill per session | 94% reused, 0.5 s |
14
+ | Splash Qwen 3.6 | 3.8 s | 97% reused from the 3rd session, 0.27 s |
15
+ | Sonnet via OpenRouter | full cache write, $0.024 per session | reads the stable part, $0.0035 |
16
+
17
+ ## The layers of a request
18
+
19
+ From the first token to the last. Every layer is a prefix of the next request's layers, unless something in the
20
+ right-hand column happens.
21
+
22
+ ```
23
+ shared by every session ────────────────────────────────────────────────── breaks it
24
+ ┌──────────────────────────────────────────────────────────────────────────┐
25
+ │ tools built-ins, then plugin tools sorted by (bundle, name) │ a tool added/changed,
26
+ │ (chat: the `tools` field; native: the model's template) │ /model, a profile change
27
+ ├──────────────────────────────────────────────────────────────────────────┤
28
+ │ SYSTEM, stable part SystemPrompt#system_prompt_with_index
29
+ │ [native thinking token] base prompt │
30
+ │ rg guidance │ /thinking level,
31
+ │ system identity (+ model overlay) │ /model, editing a
32
+ │ explicit (preloaded) memories │ memory / AGENT.md
33
+ │ AGENT.md │ (picked up by a new
34
+ │ project + system memory indexes │ process or a rebuild)
35
+ │ ◄── remote Claude: cache_control breakpoint #1 │
36
+ ├──────────────────────────────────────────────────────────────────────────┤
37
+ │ SYSTEM, volatile tail (per session, least → most changing) │
38
+ │ Model: this session runs on … │ another session
39
+ │ working directory (location) │ (expected: only
40
+ │ session id + log │ this tail differs)
41
+ │ [Gemma 4 native only: tool declarations after all of it] │
42
+ └──────────────────────────────────────────────────────────────────────────┘
43
+ shared by every turn of this session ─────────────────────────────────────
44
+ ┌──────────────────────────────────────────────────────────────────────────┐
45
+ │ history user / assistant / tool messages, APPEND-ONLY │ !rollback, a failed or
46
+ │ (images: the newest N, ImagePlan) │ cancelled turn's tail,
47
+ │ ◄── remote Claude: breakpoint #2 on the last │ the image window
48
+ │ non-system message │ sliding, history
49
+ ├──────────────────────────────────────────────────────────────────────────┤ sanitizing (thinking
50
+ │ tail notes reminders, turn notes, steers (system/user messages) │ stripped per spec)
51
+ │ new prompt this turn's user message │
52
+ └──────────────────────────────────────────────────────────────────────────┘
53
+ ```
54
+
55
+ - The system prompt is built **once per loop** and rebuilt only on `@prompt_builder.reset!`: a model switch
56
+ (`Engine#switch_model!`), a plugin's `tools_changed!`, or a profile change. A per-turn rebuild would
57
+ break everything below it.
58
+ - On remote Claude, `LLM::PromptCache.mark` sends the system message as two text parts. The stable part carries
59
+ breakpoint #1, so a **new session** reads it from the cache. The last non-system message carries breakpoint
60
+ #2, so the **next request in the session** reads everything up to it. Mid and tail system messages never take
61
+ a breakpoint. The split point comes from `SystemPrompt#stable_length` via the message's `cache_split:`, which
62
+ never goes on the wire.
63
+ - Side requests (idle recap, `/btw`) go through `IdleClient` with their own short system prompt. On a
64
+ single-slot local server they can push the session's prefix out of the cache. Fixing that is slice 2 of the
65
+ plan (R4 `id_slot`, R2 turn-end warm-up).
66
+
67
+ ## Rules for changes
68
+
69
+ 1. **Stable first, volatile last.** Anything that differs between sessions or turns (ids, paths, times, the
70
+ model line, counters) goes at the very end of the system prompt, ordered by how often it changes. Never put
71
+ a per-session value in the stable part. A guard spec checks that model, location and session are the last
72
+ sections (`spec/system_prompt_model_spec.rb`).
73
+ 2. **History is append-only.** Never rewrite an earlier message. Add a note at the tail instead. If a model's
74
+ spec forces a rewrite (e.g. stripping previous-turn thinking), accept the re-prefill and note it.
75
+ 3. **Deterministic order.** Tools, memory lists and anything else that's rendered from a hash or a set needs a
76
+ fixed sort. Plugin tools are sorted by (bundle, name) whatever order their plugins finished starting in
77
+ (R5a).
78
+ 4. **Don't rebuild the system prompt per turn.** Rebuild only when its content really changed (`reset!`).
79
+ 5. **Measure.** Every generation logs `prompt=`, `cached=` and, where the provider reports it, `cache_write=`
80
+ on its `generation_completed` log line. `/stats` shows "cached N (P%)" and "cache writes". A change that
81
+ drops `cached=` on the second fresh session is a regression. Update the prompt snapshot fixtures with
82
+ `UPDATE_PROMPTS=1` and read the diff: a cache-friendly change moves sections, it doesn't reorder per run.
83
+
84
+ ## Server-side settings worth knowing
85
+
86
+ - **Splash:** `--persistent-cache` (with `--max-cache-disk`) keeps restore points across restarts. Splash learns
87
+ a new branch point on the 2nd session, so the 3rd one gets the full reuse.
88
+ - **llama.cpp:** a single slot is shared by everything sent to it. `--parallel N` and `--cache-ram` give side
89
+ requests room without evicting the session. `--ctx-checkpoints` controls how many restore points a hybrid
90
+ model keeps.
91
+ - **OpenRouter / Anthropic:** the default cache TTL is 5 minutes. Provider routing (which upstream serves a
92
+ request) also decides whether a cache is there to hit.
93
+
94
+ Still to come: side requests on their own slot, a turn-end warm-up, image-window batches and longer remote TTLs.
data/docs/plugins.md CHANGED
@@ -847,14 +847,20 @@ stream](hooks.md#watching-the-stream)) and cuts it:
847
847
  thinking has short sentences too ("Hmm.", "Fine."), but spread out or all
848
848
  different (code lines); a longer sentence ends the run, and lines inside a
849
849
  ```` ``` ```` code block don't count.
850
+ - Any `window_sentences` sentences in a row, short or long, holding at most
851
+ `window_distinct` different ones are a loop too: a cycle of 7 or 8 short
852
+ sentences, or one where a longer sentence ("I'll write the spec file
853
+ now.") keeps ending the short run. Real thinking holds 40 or more
854
+ different sentences in any 48; DeepSeek's write loops hold 4 to 9.
850
855
  - Nothing triggers before `min_chars` of thinking. The watch sees a loop
851
856
  within one batch (2000 chars, or a second) of its third cycle.
852
857
 
853
858
  What happens (`action: retry`, the default):
854
859
 
855
860
  1. The first loop in a turn: the generation is cut and the model asked
856
- again, with a hidden note that it was cut off. The user sees
857
- `loop-guard> thinking repeats itself (3 sentences ×3, 4k chars, 8 s): cut`
861
+ again, with a hidden note that it was cut off. The user sees the loop's
862
+ first sentence quoted,
863
+ `loop-guard> thinking repeats itself ("Wait, the count of the letter r…", 3 sentences ×3, 4k chars, 8 s): cut`,
858
864
  and `↻ cut by loop-guard, asking again (1/1)`.
859
865
  2. If the retry loops too, the turn is stopped ("stopped the turn: …", "✕
860
866
  turn stopped by loop-guard") with a card that quotes the repeated
@@ -869,6 +875,12 @@ the first loop ends the turn as stopped by loop-guard, with the notice and
869
875
  no card. `action: stop` stops the turn at the first loop; `action: notify` only
870
876
  warns, once per generation.
871
877
 
878
+ The cut thinking never goes back to the model, but its last 20k chars are kept
879
+ in the session's folder, `thinking_tails.jsonl` (with a generation that ran to
880
+ the provider's output cap, or that a stop_turn ended): an archived session
881
+ keeps them. A turn's record counts its cuts and cap hits (`cuts`, `capped` in
882
+ `analytics.json`; `/stats` shows "thinking cuts" and "output cap hits").
883
+
872
884
  ```yaml
873
885
  bundles:
874
886
  loop-guard:
@@ -885,6 +897,8 @@ bundles:
885
897
  min_words: 5 # shorter sentences only count toward a short run
886
898
  short_run: 24 # this many short sentences in a row…
887
899
  short_distinct: 6 # …with at most this many different ones is a loop
900
+ window_sentences: 48 # any this many sentences in a row…
901
+ window_distinct: 12 # …with at most this many different ones is a loop
888
902
  forget_after: 10 # good steps in a row that forget a loop; 0: never
889
903
  ```
890
904
 
data/docs/testing.md CHANGED
@@ -15,6 +15,18 @@ bug) or a broken spec: fix it, or disable that line with the reason (`# rubocop:
15
15
  No spec reads your own setup. The suite points `XDG_CONFIG_HOME` and `XDG_STATE_HOME` at temp folders,
16
16
  clears your `SAMAGOTCHI_*` environment, and keeps every spec off the network (WebMock).
17
17
 
18
+ ## Worktrees
19
+
20
+ Each fix or feature gets its own worktree next to the main checkout:
21
+
22
+ ```sh
23
+ script/worktree NAME [-b BRANCH] # ../samagotchi-NAME off main, branch NAME unless -b; prints the path
24
+ ```
25
+
26
+ It runs `git worktree add ../samagotchi-NAME -b BRANCH main`, then `npm ci --prefer-offline` in it, so
27
+ `npm test` and `npm run e2e` work there right away (Ruby gems and Playwright's browsers are shared: nothing else
28
+ to install). It refuses when the folder or the branch already exists.
29
+
18
30
  ## Integration specs
19
31
 
20
32
  Examples tagged `:integration` (`spec/integration/`, the "with LLM access" part of `spec/hooks/integration_spec.rb`)
@@ -1,10 +1,10 @@
1
1
  ---
2
2
  name: loop-guard
3
- version: 0.3.2
3
+ version: 0.3.3
4
4
  scope: system
5
5
  description: "Breaks loops: the same tool call with the same result again and again in one turn is denied with advice (after a few denies the turn stops with a card), and thinking that repeats itself while it streams is cut and the model asked again"
6
6
  trust_level: reviewed
7
7
  plugin:
8
8
  file: plugin.rb
9
- sha256: sha256:27b135c462d8462313edaa1faa90f3104a1bff3c421d6965e9e71955f347d55b
9
+ sha256: sha256:8cf255cd1cc0846a759fcf70144af53bfc634c9d5c766981ce0ef5e476749e0a
10
10
  requires_chi: ">= 0.6.0"
@@ -56,12 +56,20 @@ require "digest"
56
56
  # too ("Hmm.", "Fine."), but spread out, or in runs of different ones
57
57
  # (code lines); a sentence of min_words or more ends the run, one with no
58
58
  # letters or digits neither counts nor ends it. Lines inside a ``` code
59
- # block are not short sentences.
59
+ # block are not short sentences;
60
+ # - a low-diversity window: among the last window_sentences sentences of any
61
+ # length, at most window_distinct different ones (exact, normalized). It
62
+ # sees a cycle too long for the short run (7-8 short sentences), or one
63
+ # where a longer sentence keeps ending the run ("Go. OK. I'll write the
64
+ # spec file now. Go."). Real thinking holds 40+ different sentences in
65
+ # any 48 (the lowest seen: 42); a loop holds under 10. Lines inside a ```
66
+ # code block don't count.
60
67
  # Nothing triggers before min_chars of thinking. O(new chars) per feed.
61
68
  class ThinkingWatch
62
69
  DEFAULTS = { "watch" => true, "action" => "retry", "min_chars" => 2000, "repeats" => 3, "max_period" => 6,
63
70
  "similarity" => 0.5, "min_span_sentences" => 6, "min_span_chars" => 600, "max_same" => 8,
64
- "min_words" => 5, "short_run" => 24, "short_distinct" => 6, "forget_after" => 10 }.freeze
71
+ "min_words" => 5, "short_run" => 24, "short_distinct" => 6, "window_sentences" => 48,
72
+ "window_distinct" => 12, "forget_after" => 10 }.freeze
65
73
  ACTIONS = %w[retry stop notify].freeze
66
74
  MAX_CARRY = 400
67
75
  # One sentence again and again needs this much more alike than a cycle.
@@ -81,7 +89,7 @@ class ThinkingWatch
81
89
  out["watch"] = !%w[false no off 0].include?(raw["watch"].to_s.downcase) if raw.key?("watch")
82
90
  out["action"] = raw["action"].to_s if ACTIONS.include?(raw["action"].to_s)
83
91
  %w[min_chars repeats max_period min_span_sentences min_span_chars max_same min_words short_run
84
- short_distinct].each do |key|
92
+ short_distinct window_sentences window_distinct].each do |key|
85
93
  number = Integer(raw[key].to_s, exception: false)
86
94
  out[key] = number if number&.positive?
87
95
  end
@@ -104,6 +112,8 @@ class ThinkingWatch
104
112
  @min_words = settings["min_words"]
105
113
  @short_run = settings["short_run"]
106
114
  @short_distinct = settings["short_distinct"]
115
+ @window_sentences = settings["window_sentences"]
116
+ @window_distinct = settings["window_distinct"]
107
117
  @same_similarity = [@similarity + SAME_MARGIN, 0.95].min
108
118
  @carry = +""
109
119
  @window = [] # the last max_period + 1 sentences
@@ -112,6 +122,8 @@ class ThinkingWatch
112
122
  @counts = Hash.new(0)
113
123
  @short_count = 0 # short sentences in a row
114
124
  @short_window = [] # the last short_run of them, [key, text]
125
+ @recent = [] # the last window_sentences sentences, [key, text]
126
+ @recent_counts = Hash.new(0) # key => times in @recent
115
127
  @in_code = false
116
128
  @thinking_chars = 0
117
129
  end
@@ -153,7 +165,8 @@ class ThinkingWatch
153
165
  end
154
166
  words = text.downcase.gsub(/[^[:alnum:]]+/, " ").split
155
167
  short = short_loop(text.strip, words) unless @in_code
156
- return (short if @thinking_chars >= @min_chars) if words.length < @min_words
168
+ window = window_loop(text.strip, words) unless @in_code
169
+ return (@thinking_chars >= @min_chars ? short || window : nil) if words.length < @min_words
157
170
 
158
171
  key = words.join(" ")
159
172
  sentence = Sentence.new(text: text.strip, words: words.map(&:hash).uniq,
@@ -164,7 +177,27 @@ class ThinkingWatch
164
177
  @window.shift if @window.length > @max_period + 1
165
178
  return nil if @thinking_chars < @min_chars
166
179
 
167
- cycle || (same >= @max_same ? Loop.new(period: 1, times: same, sentences: [sentence.text], chars: nil) : nil)
180
+ cycle || (same >= @max_same ? Loop.new(period: 1, times: same, sentences: [sentence.text], chars: nil) : nil) || window
181
+ end
182
+
183
+ # The last window_sentences sentences (any length) hold at most
184
+ # window_distinct different ones: a loop. Its sentences are the most
185
+ # frequent first.
186
+ def window_loop(text, words)
187
+ return nil if words.empty?
188
+
189
+ key = words.join(" ")
190
+ @recent << [key, text]
191
+ @recent_counts[key] += 1
192
+ if @recent.length > @window_sentences
193
+ old, = @recent.shift
194
+ @recent_counts.delete(old) if (@recent_counts[old] -= 1).zero?
195
+ end
196
+ return nil if @recent.length < @window_sentences || @recent_counts.size > @window_distinct
197
+
198
+ texts = @recent.to_h # key => its last text
199
+ top = @recent_counts.sort_by { |_key, n| -n }.first(3).map { |key, _n| texts[key] }
200
+ Loop.new(period: @recent_counts.size, times: @window_sentences / @recent_counts.size, sentences: top, chars: nil)
168
201
  end
169
202
 
170
203
  def short_loop(text, words)
@@ -276,14 +309,17 @@ class Plugin
276
309
  @watch_done = true
277
310
  what = "#{found.period == 1 ? "one sentence" : "#{found.period} sentences"} ×#{found.times}, " \
278
311
  "#{(@watch.thinking_chars / 1000.0).round}k chars, #{(event[:elapsed_ms].to_i / 1000.0).round} s"
312
+ # The notices quote the loop's sentences on one line (a short cycle
313
+ # whole), as the stop card does.
314
+ quoted = "\"#{cut(found.sentences.first(3).join(" ").gsub(/\s+/, " "))}\", #{what}"
279
315
  case @thinking["action"]
280
316
  when "notify"
281
- ctx.notify("thinking repeats itself (#{what})", level: :warn)
317
+ ctx.notify("thinking repeats itself (#{quoted})", level: :warn)
282
318
  when "retry"
283
319
  @thinking_loops += 1
284
320
  return stop_thinking(ctx, found, what) if @thinking_loops > 1
285
321
 
286
- ctx.notify("thinking repeats itself (#{what}): cut", level: :warn) if ctx.stop_generation("its thinking kept repeating itself")
322
+ ctx.notify("thinking repeats itself (#{quoted}): cut", level: :warn) if ctx.stop_generation("its thinking kept repeating itself")
287
323
  else
288
324
  stop_thinking(ctx, found, what)
289
325
  end
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: samagotchi-system
3
- version: 0.24.0
3
+ version: 0.25.0
4
4
  scope: system
5
5
  description: Default system memories — identity, self map, config modification protocol, memory guide and the delegated-session rules
6
6
  files: