samagotchi 0.24.0 → 0.25.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +54 -1
- data/docs/architecture.md +16 -9
- data/docs/cli.md +4 -3
- data/docs/configuration.md +21 -13
- data/docs/hooks.md +1 -1
- data/docs/internals/background-tasks.md +1 -0
- data/docs/internals/prompt-caching.md +94 -0
- data/docs/plugins.md +16 -2
- data/docs/testing.md +12 -0
- data/lib/samagotchi/bundles/loop-guard/manifest.yml +2 -2
- data/lib/samagotchi/bundles/loop-guard/plugin.rb +43 -7
- data/lib/samagotchi/bundles/system/manifest.yml +1 -1
- data/lib/samagotchi/command_steps.rb +233 -0
- data/lib/samagotchi/config.rb +5 -2
- data/lib/samagotchi/empty_answer_retry.rb +2 -2
- data/lib/samagotchi/engine.rb +23 -4
- data/lib/samagotchi/guardrails/shell_lex.rb +49 -5
- data/lib/samagotchi/idle_client.rb +14 -2
- data/lib/samagotchi/kernel_loop.rb +21 -3
- data/lib/samagotchi/llm/chat_loop.rb +17 -3
- data/lib/samagotchi/llm/errors.rb +26 -1
- data/lib/samagotchi/llm/openai_chat.rb +5 -4
- data/lib/samagotchi/llm/prompt_cache.rb +34 -9
- data/lib/samagotchi/llm/usage.rb +18 -3
- data/lib/samagotchi/log_subscriber.rb +2 -1
- data/lib/samagotchi/plugin/api.rb +3 -2
- data/lib/samagotchi/session_metrics.rb +38 -4
- data/lib/samagotchi/system_prompt.rb +30 -9
- data/lib/samagotchi/terminal_ui/attached_loop.rb +5 -2
- data/lib/samagotchi/terminal_ui/attached_view.rb +6 -2
- data/lib/samagotchi/terminal_ui/event_renderer.rb +14 -2
- data/lib/samagotchi/terminal_ui/formatting.rb +31 -9
- data/lib/samagotchi/thinking_tails.rb +103 -0
- data/lib/samagotchi/token_usage.rb +18 -6
- data/lib/samagotchi/tool_activity.rb +37 -3
- data/lib/samagotchi/tool_declarations.rb +14 -0
- data/lib/samagotchi/tool_view.rb +40 -4
- data/lib/samagotchi/tools/builtins.rb +4 -1
- data/lib/samagotchi/tools/execute.rb +2 -1
- data/lib/samagotchi/tools/registry.rb +11 -4
- data/lib/samagotchi/tools/shell.rb +32 -0
- data/lib/samagotchi/tools/task_runtime.rb +3 -3
- data/lib/samagotchi/turn_tally.rb +17 -6
- data/lib/samagotchi/version.rb +1 -1
- data/lib/samagotchi/web/public/command_view.js +71 -3
- data/lib/samagotchi/web/public/format.js +11 -1
- data/lib/samagotchi/web/public/index.html +25 -0
- data/lib/samagotchi/web/public/stage_model.js +5 -1
- data/lib/samagotchi/web/public/stage_view.js +1 -1
- data/lib/samagotchi/web/public/turn_view.js +3 -3
- metadata +5 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6abdccc8a805a545ad1e27898b3caf43516d16ef1b7832444bac7b6e0a4b5e63
|
|
4
|
+
data.tar.gz: 8c0e0f354030f8ad954afdfd9dff96ef54d0b1fdd79d8df190cf494a8a32b18d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 36530955a235e3d73adaa4496545c12f6abea21daf1ff0a40eaba9477c22e1e7007dced1a500912878164c0a9f1a9ec76b45d4d8a34709473d8c496b1d53ce69
|
|
7
|
+
data.tar.gz: 125d6ab7dd048082ff6296d663803da9d6a8073c7ead520f2264ec4cc27c28d29bdb772addd47f2be39b9aa497308ef09236bd5e3e1bc43854341970fc8e6a6e
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,58 @@ and commands may change between minor versions. How releases are made:
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [0.25.0] - 2026-10-04
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- The web shows an `execute` command as the steps a person reads: `cd X &&` becomes an "in X" tag, `| head -20` a
|
|
16
|
+
chip, `echo "=== x ==="` a label, a heredoc a collapsed "EOF · N lines" chip; the collapsed row's title is the
|
|
17
|
+
first step `+N`. A `raw` toggle shows the whole command; anything the parser can't read shows as before.
|
|
18
|
+
- `execute` takes an optional `description` ("what the command does, in a few words"), written by the model; it
|
|
19
|
+
becomes the tool row's title in the web and replaces the cut command in the terminal's tool line. Turn it off
|
|
20
|
+
with `execute.description: false`.
|
|
21
|
+
- Every generation logs its prompt-cache counts (`prompt=`, `cached=`, `cache_write=`), and `/stats` shows cache
|
|
22
|
+
writes next to cached tokens.
|
|
23
|
+
- A generation cut mid-thinking (loop-guard, a plugin's stop, a steer) or ended at the provider's output cap keeps
|
|
24
|
+
the last 20k chars of its thinking in the session's folder, `thinking_tails.jsonl`, so an archived session holds
|
|
25
|
+
the loop it was archived for. The model never sees it.
|
|
26
|
+
- Turn records count `cuts` (generations a plugin cut) and `capped` (generations that hit the output cap), and
|
|
27
|
+
`/stats` shows them as "thinking cuts" and "output cap hits": `retries` counts network retries only.
|
|
28
|
+
|
|
29
|
+
### Changed
|
|
30
|
+
|
|
31
|
+
- On macOS, `execute`, `task_create` and `!cmd` run commands in `/bin/zsh` emulating sh (with bash's `{1..3}`,
|
|
32
|
+
`[[ =~ ]]` groups and echo escapes kept) instead of `/bin/sh`, which is bash 3.2 there and failed on an apostrophe
|
|
33
|
+
in a heredoc inside `"$( )"`: `git commit -m "$(cat <<'EOF' … it's … EOF )"` now works. Linux keeps
|
|
34
|
+
`/bin/sh`; see [background tasks](docs/internals/background-tasks.md).
|
|
35
|
+
- loop-guard's cut notice (bundle 0.3.3) quotes the looping sentence: `thinking repeats itself ("Let me write the
|
|
36
|
+
tool call.", one sentence ×8, 33k chars, 63 s): cut`.
|
|
37
|
+
- The system prompt puts what every session shares first and the per-session lines (model, working directory,
|
|
38
|
+
session) last, and plugin tools come in a fixed order, so a new session reuses the server's prompt cache: on a
|
|
39
|
+
local llama.cpp or Splash server its first answer starts in about 0.5 s instead of 4–6 s, and on Claude via
|
|
40
|
+
OpenRouter a new session reads the shared part from the cache (about 7× cheaper to start).
|
|
41
|
+
- Chat hosts (`api: openai`) are no longer asked for `temperature: 0.0`: chi sends no temperature unless one is
|
|
42
|
+
configured, so each model runs at its provider's default (as native llama.cpp hosts already did). Greedy decoding
|
|
43
|
+
made DeepSeek v4.1-flash loop in its thinking. The idle recap and `/btw` follow suit. Set `sampling: {temperature:
|
|
44
|
+
…}` on a `hosts:` or `models:` entry to pin one; the empty-answer retry still runs at 0.6 unless one is set.
|
|
45
|
+
|
|
46
|
+
### Fixed
|
|
47
|
+
|
|
48
|
+
- loop-guard (bundle 0.3.3) cuts thinking that loops in a cycle of 7 or 8 short sentences, or with a longer
|
|
49
|
+
sentence ("I'll write the spec file now.") in the cycle: 48 sentences in a row with 12 or fewer different ones are
|
|
50
|
+
a loop. New settings `thinking.window_sentences` (48) and `thinking.window_distinct` (12).
|
|
51
|
+
- A steer row names chi's senders in words: the web's row and trail flash and the terminal's nudged line say
|
|
52
|
+
`parent agent`, `chi send` and `plugin` instead of the raw `parent_agent` / `chi_send` / `plugin_send`.
|
|
53
|
+
- Thinking off on Gemma 4 no longer warns that off wasn't honoured: its empty thought (whitespace alone) is not
|
|
54
|
+
counted as thinking.
|
|
55
|
+
- An OpenRouter 402 whose `metadata.reason` is `weight_exceeds_budget` (the request alone is larger than the
|
|
56
|
+
key's credit budget) fails at once as `request too large for the credit budget on host <name>: …; lower
|
|
57
|
+
max_tokens (default.max_tokens) or raise the key's credit limit`, instead of being retried as credit held by
|
|
58
|
+
in-flight requests. Other 402s are unchanged.
|
|
59
|
+
|
|
60
|
+
Update with `chi update`: loop-guard moves to 0.3.3 (the window rule and the quoted cut notice). Restart `chi web`
|
|
61
|
+
and running sessions afterwards (`chi sessions restart ID`).
|
|
62
|
+
|
|
11
63
|
## [0.24.0] - 2026-10-04
|
|
12
64
|
|
|
13
65
|
### Added
|
|
@@ -1342,7 +1394,8 @@ and long-lived sessions.
|
|
|
1342
1394
|
- A macOS desktop helper (`chi desktop install`): a "Send to chi" Service and a
|
|
1343
1395
|
hotkey panel that send selected text or the clipboard to your sessions.
|
|
1344
1396
|
|
|
1345
|
-
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.
|
|
1397
|
+
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.25.0...HEAD
|
|
1398
|
+
[0.25.0]: https://github.com/dm1try/samagotchi/compare/v0.24.0...v0.25.0
|
|
1346
1399
|
[0.24.0]: https://github.com/dm1try/samagotchi/compare/v0.23.0...v0.24.0
|
|
1347
1400
|
[0.23.0]: https://github.com/dm1try/samagotchi/compare/v0.22.0...v0.23.0
|
|
1348
1401
|
[0.22.0]: https://github.com/dm1try/samagotchi/compare/v0.21.0...v0.22.0
|
data/docs/architecture.md
CHANGED
|
@@ -184,12 +184,18 @@ engine.system_prompt # fully built system prompt string
|
|
|
184
184
|
engine.session # current session (Engine owns create/resume)
|
|
185
185
|
```
|
|
186
186
|
|
|
187
|
-
The system prompt (`SystemPrompt#build`) is the base prompt, then rg guidance,
|
|
188
|
-
AGENT.md, the
|
|
189
|
-
<ref> (host …; model key …)` and a line telling
|
|
190
|
-
are you" from it, not from training), the
|
|
191
|
-
|
|
192
|
-
|
|
187
|
+
The system prompt (`SystemPrompt#build`) is the base prompt, then rg guidance, the
|
|
188
|
+
identity and the preloaded memories, AGENT.md, the memory indexes, and last **the
|
|
189
|
+
model** (`Model: this session runs on <ref> (host …; model key …)` and a line telling
|
|
190
|
+
the model to answer "which model are you" from it, not from training), the working
|
|
191
|
+
directory and the session id and log (Gemma's native tool declarations still follow).
|
|
192
|
+
The order is for the servers' prompt caches, which reuse only an exact token prefix:
|
|
193
|
+
what every session shares first, what changes least before what changes more, the
|
|
194
|
+
per-session lines last, so a new session reuses everything above them. It is built
|
|
195
|
+
once per loop and rebuilt only by a model switch (or changed tools), so the model
|
|
196
|
+
line costs no KV churn.
|
|
197
|
+
The full request layout, what breaks the cache and the rules for changes:
|
|
198
|
+
[internals/prompt-caching.md](internals/prompt-caching.md).
|
|
193
199
|
|
|
194
200
|
#### Subscribing to the live stream (and the bridge)
|
|
195
201
|
|
|
@@ -275,11 +281,12 @@ The metrics in `/state`, `/snapshot`, `/stats` and the SSE `snapshot`/`reset` fr
|
|
|
275
281
|
newest turn the worker finished and any it hasn't saved yet (at most 20 turns), with their tool
|
|
276
282
|
calls and the running turn's finished ones. The full history is the session's `analytics.json`,
|
|
277
283
|
which `SessionMetrics#persist` rewrites after each turn; the web server merges the two by id.
|
|
278
|
-
The totals' `tokens` block adds the cached and reasoning sums, the cost, the decode time and tokens
|
|
284
|
+
The totals' `tokens` block adds the cached, cache-write and reasoning sums, the cost, the decode time and tokens
|
|
279
285
|
behind the average speed (`avg_decode_tps`) and the last speeds (`last_decode_tps`,
|
|
280
286
|
`last_prefill_tps`, `tps_source`: `server` or `estimate`). Each `generation_completed` event
|
|
281
|
-
carries the generation's `speed` (`{decode_tps, source}`, null without one)
|
|
282
|
-
`
|
|
287
|
+
carries the generation's `speed` (`{decode_tps, source}`, null without one), its own prompt-cache
|
|
288
|
+
counts (`prompt_tokens`, `cached_tokens`, `cache_write_tokens`: null when the server reported no writes) and
|
|
289
|
+
those running `tokens`: the Engine closes the generation in `SessionMetrics#finish_generation` before relaying
|
|
283
290
|
it, so the web's info bar updates per generation without parsing chunk payloads.
|
|
284
291
|
|
|
285
292
|
The page's frequent reads of `GET /api/sessions/:id` stay light too:
|
data/docs/cli.md
CHANGED
|
@@ -420,9 +420,10 @@ file and index line stay as they are; only this session doesn't see it.
|
|
|
420
420
|
`/help` lists the commands this session takes, the installed bundles' too, each
|
|
421
421
|
with a line on what it does. `/stats` shows the session's numbers: turns, tool
|
|
422
422
|
calls (by tool, with errors), iterations, tokens in/out summed over every
|
|
423
|
-
request (with the cached and reasoning tokens when the server reports them),
|
|
424
|
-
the decode speed, generation latency, cancellations, retries
|
|
425
|
-
|
|
423
|
+
request (with the cached tokens, the cache writes and the reasoning tokens when the server reports them),
|
|
424
|
+
the decode speed, generation latency, cancellations, retries (the network's),
|
|
425
|
+
thinking cuts (generations a plugin such as loop-guard cut) and output cap hits
|
|
426
|
+
(generations that ended at the provider's cap), context used and window, the prompt profile and the model the server says it ran, and the
|
|
426
427
|
cost when the provider reports one (OpenRouter's `usage.cost`; no prices are
|
|
427
428
|
guessed). The speed line is the last generation's and the session's average
|
|
428
429
|
(weighted by tokens): llama.cpp reports it exactly (`87 tok/s out, 1.9k tok/s
|
data/docs/configuration.md
CHANGED
|
@@ -369,9 +369,9 @@ after an idle exit) gets that process's environment, so put a lasting choice in
|
|
|
369
369
|
|
|
370
370
|
## Sampling
|
|
371
371
|
|
|
372
|
-
chi
|
|
373
|
-
|
|
374
|
-
entry sets request fields for the model's turns:
|
|
372
|
+
chi sends no sampling fields by default, to chat hosts (`api: openai`) or native ones, so each model runs at its
|
|
373
|
+
provider's defaults (llama.cpp: temperature 0.8; vLLM takes them from the model's `generation_config.json`).
|
|
374
|
+
`sampling:` on a `hosts:` entry or a `models:` entry sets request fields for the model's turns:
|
|
375
375
|
|
|
376
376
|
```yaml
|
|
377
377
|
hosts:
|
|
@@ -394,15 +394,16 @@ models:
|
|
|
394
394
|
- A `models:` entry's fields win over its host's, field by field (the host can set a penalty and the model move only
|
|
395
395
|
the temperature). The entry is found the way `profile:` is (the name as typed, alias-resolved or without its host
|
|
396
396
|
prefix).
|
|
397
|
-
- `temperature: null` (or `~`)
|
|
398
|
-
|
|
397
|
+
- `temperature: null` (or `~`) in a model's entry takes back a host's temperature, so the provider's default applies;
|
|
398
|
+
it also keeps the empty-answer retry (below) at that default.
|
|
399
399
|
- Fields chi sets itself are refused with a warning: `model`, `messages`, `prompt`, `stream`, `stream_options`,
|
|
400
400
|
`tools`, `tool_choice`, `stop`, `n_predict`, `max_tokens`, `n`, `parallel_tool_calls`, `response_format`,
|
|
401
401
|
`cache_prompt`. A `sampling:` that isn't a map warns and is skipped.
|
|
402
|
-
- Greedy decoding can make a
|
|
403
|
-
for minutes) or end with an empty answer. Qwen's own advice for its
|
|
404
|
-
top_p: 0.95
|
|
405
|
-
|
|
402
|
+
- Greedy decoding (`temperature: 0`) can make a thinking model loop in its reasoning ("Let me write the reply…"
|
|
403
|
+
for minutes) or end with an empty answer; set it only for a model you have watched. Qwen's own advice for its
|
|
404
|
+
thinking models is `temperature: 0.6, top_p: 0.95`, with `presence_penalty` between 0 and 2 against endless
|
|
405
|
+
repetition.
|
|
406
|
+
- The idle recap and side questions don't use `sampling:`; they send their own short output cap and no temperature.
|
|
406
407
|
|
|
407
408
|
`/model` shows what applies (`sampling: temperature=0.6 presence_penalty=1.5 (hosts.work)`), and each request's
|
|
408
409
|
`stream` line in the debug log carries a `sampling=` field with what was sent. The fields are read every turn, so a
|
|
@@ -514,10 +515,14 @@ gateway that spreads requests over several providers can key on it to keep one c
|
|
|
514
515
|
prompt caches hit and every turn is answered by the same model. Servers that don't know the header ignore it.
|
|
515
516
|
|
|
516
517
|
Chat requests for a Claude model (an id containing `claude`, such as `anthropic/claude-sonnet-5.5`) on a remote
|
|
517
|
-
OpenAI-compatible host get two prompt-cache breakpoints (`cache_control` on the system
|
|
518
|
-
message), so each step reads the earlier prompt from Anthropic's cache instead of paying for it again
|
|
518
|
+
OpenAI-compatible host get two prompt-cache breakpoints (`cache_control` on the system prompt and on the last
|
|
519
|
+
message), so each step reads the earlier prompt from Anthropic's cache instead of paying for it again. The system
|
|
520
|
+
prompt goes as two text parts, the breakpoint after the first: everything sessions share, then the model, working
|
|
521
|
+
directory and session lines, so a new session reads the shared part from the cache as well; the debug
|
|
519
522
|
log's line shows `cache=on`. Nothing to configure. A provider behind a gateway may still not cache: check the
|
|
520
|
-
cached tokens in `/stats`.
|
|
523
|
+
cached tokens in `/stats`. Each request's own counts are in the debug log's `generation_completed` line:
|
|
524
|
+
`prompt=` (its prompt tokens), `cached=` (read from the server's cache) and `cache_write=` (written to it, when the
|
|
525
|
+
server reports writes); recap and side requests log theirs as `recap request_usage`.
|
|
521
526
|
|
|
522
527
|
## Llama Model Routing
|
|
523
528
|
|
|
@@ -616,6 +621,8 @@ Transient network failures are retried automatically with exponential backoff.
|
|
|
616
621
|
- An HTTP 402 about credit held by in-flight requests (OpenRouter reserves credit for each running request, so
|
|
617
622
|
one session's long request can make another's fail) is retried like a 429, 20s apart unless `Retry-After`
|
|
618
623
|
says otherwise: with the default `retry.max` that is about 100s. With `retry.max: 0` it fails at once.
|
|
624
|
+
Not when its `metadata.reason` is `weight_exceeds_budget`: that request alone is larger than the key's credit
|
|
625
|
+
budget, so it fails at once (lower `default.max_tokens` or raise the key's limit).
|
|
619
626
|
- A refused connection (nothing listening) is not retried: the turn fails at once with `can't reach host <name> at
|
|
620
627
|
<address> (connection refused) — is the server running?`.
|
|
621
628
|
- A stream that has already produced output is never retried (the retry would repeat it); it fails the turn.
|
|
@@ -671,7 +678,7 @@ message (before, a failed llama.cpp `/completion` ended the turn as
|
|
|
671
678
|
| connection | reset, timed out, dropped mid-stream; refused | yes (network retry), not mid-stream; refused: no |
|
|
672
679
|
| rate limited | HTTP 429 | yes, honouring `Retry-After` |
|
|
673
680
|
| credits held | HTTP 402 "… in-flight requests" | yes, after 20s (or `Retry-After`) |
|
|
674
|
-
| credits | any other HTTP 402: out of credits | no |
|
|
681
|
+
| credits | any other HTTP 402: out of credits; one whose `metadata.reason` is `weight_exceeds_budget` (the request alone is over the key's budget: lower `default.max_tokens`) | no |
|
|
675
682
|
| server | HTTP 5xx, llama.cpp's mid-stream `error:` event | 500/502/503/504/529 only |
|
|
676
683
|
| auth | HTTP 401/403 | no |
|
|
677
684
|
| bad request | other 4xx; a prompt larger than the context window, whatever the status | no |
|
|
@@ -912,6 +919,7 @@ described in their own sections.
|
|
|
912
919
|
| `execute.preview_bytes` | `12288` | yes | |
|
|
913
920
|
| `execute.telemetry_threshold_pct` | `80` | yes | |
|
|
914
921
|
| `execute.timeout_sec` | `120` | yes | Seconds one `execute` command may run before it is stopped. |
|
|
922
|
+
| `execute.description` | `true` | | `execute` offers the model an optional `description` (a few words on what the command does), shown as the tool row's title in `chi web` and in place of the command in the TUI's tool line. `false`: the parameter isn't declared, and rows show the command's first step. |
|
|
915
923
|
| `web.port` | `4567` | `--port` | `chi web`'s port. See [CLI](cli.md). |
|
|
916
924
|
| `web.host` | `127.0.0.1` | yes | `127.0.0.1`, `::1` or `localhost`; `lan` (this machine's private IPv4 address) or one of its IPv4 addresses also opens `chi web` to the network, with an access token (`chi web --new-token` replaces it). Anything else binds `127.0.0.1` with a warning. See [CLI: chi web on your phone](cli.md#chi-web-on-your-phone). |
|
|
917
925
|
| `web.markdown` | `false` | yes | Render answers as Markdown in `chi web`. |
|
data/docs/hooks.md
CHANGED
|
@@ -399,7 +399,7 @@ stripped of surrounding whitespace, except file text, an edit's
|
|
|
399
399
|
|
|
400
400
|
| Tool | `content:` from | `path:` from | Own fields |
|
|
401
401
|
|------|-----------------|--------------|------------|
|
|
402
|
-
| `execute` | `command` | — | `cwd` |
|
|
402
|
+
| `execute` | `command` | — | `description`, `cwd` |
|
|
403
403
|
| `read` | `path` | — | `start_line`, `end_line` |
|
|
404
404
|
| `write` | — | `path` | `content` |
|
|
405
405
|
| `edit` | — | `path` | `old_text`, `new_text`, `start_line`, `end_line` |
|
|
@@ -17,6 +17,7 @@ Recommended workflow:
|
|
|
17
17
|
|
|
18
18
|
Behavior:
|
|
19
19
|
|
|
20
|
+
- `execute`, `task_create` and the `!cmd` shell bang run the command in a non-login shell: `/bin/sh -c` on Linux; on macOS, when `/bin/zsh` is there, `/bin/zsh --emulate sh +o ignore_braces +o sh_glob -o bash_rematch +o bsd_echo -c` (macOS's `/bin/sh` is bash 3.2, which can't parse an apostrophe in a heredoc inside `"$( )"`, the usual commit message shape). That mode reads no rc files and keeps POSIX word splitting, unmatched globs left as is, `{1..3}`, `[[ … =~ (…) ]]` with `BASH_REMATCH`, arrays, `$'…'`, `&>` and echo expanding `\t`. Bash-only builtins and variables (`shopt`, `BASH_VERSION`, `${!prefix*}`, `read -p`) are not there.
|
|
20
21
|
- Task metadata and output are persisted under `tmp/tasks/`.
|
|
21
22
|
- Task listing is workspace-scoped (current project only).
|
|
22
23
|
- `task_get` returns metadata and `output_path`; use `read` for output contents.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Prompt caching: how a request is laid out, and how to keep it cache-friendly
|
|
2
|
+
|
|
3
|
+
A model server reuses work only for the **exact token prefix** it has already seen. Change one token and
|
|
4
|
+
everything after it is computed again (prefill). The local models we run (hybrid Qwen 3.6 / Ornith on llama.cpp
|
|
5
|
+
and Splash) can only resume from a **restore point near the end** of an earlier prompt, not from any shared
|
|
6
|
+
prefix. Anthropic (Claude via OpenRouter) caches only up to an explicit `cache_control` breakpoint. So the order
|
|
7
|
+
of what chi sends decides what a new turn, or a new session, has to pay for.
|
|
8
|
+
|
|
9
|
+
Measured with prompt-cache slice 1 (2026-10-04, fresh sessions, ~7.7k-token prompt):
|
|
10
|
+
|
|
11
|
+
| Server | Before | After |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| Ornith on llama.cpp | 0 tokens reused, 6.4 s prefill per session | 94% reused, 0.5 s |
|
|
14
|
+
| Splash Qwen 3.6 | 3.8 s | 97% reused from the 3rd session, 0.27 s |
|
|
15
|
+
| Sonnet via OpenRouter | full cache write, $0.024 per session | reads the stable part, $0.0035 |
|
|
16
|
+
|
|
17
|
+
## The layers of a request
|
|
18
|
+
|
|
19
|
+
From the first token to the last. Every layer is a prefix of the next request's layers, unless something in the
|
|
20
|
+
right-hand column happens.
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
shared by every session ────────────────────────────────────────────────── breaks it
|
|
24
|
+
┌──────────────────────────────────────────────────────────────────────────┐
|
|
25
|
+
│ tools built-ins, then plugin tools sorted by (bundle, name) │ a tool added/changed,
|
|
26
|
+
│ (chat: the `tools` field; native: the model's template) │ /model, a profile change
|
|
27
|
+
├──────────────────────────────────────────────────────────────────────────┤
|
|
28
|
+
│ SYSTEM, stable part SystemPrompt#system_prompt_with_index
|
|
29
|
+
│ [native thinking token] base prompt │
|
|
30
|
+
│ rg guidance │ /thinking level,
|
|
31
|
+
│ system identity (+ model overlay) │ /model, editing a
|
|
32
|
+
│ explicit (preloaded) memories │ memory / AGENT.md
|
|
33
|
+
│ AGENT.md │ (picked up by a new
|
|
34
|
+
│ project + system memory indexes │ process or a rebuild)
|
|
35
|
+
│ ◄── remote Claude: cache_control breakpoint #1 │
|
|
36
|
+
├──────────────────────────────────────────────────────────────────────────┤
|
|
37
|
+
│ SYSTEM, volatile tail (per session, least → most changing) │
|
|
38
|
+
│ Model: this session runs on … │ another session
|
|
39
|
+
│ working directory (location) │ (expected: only
|
|
40
|
+
│ session id + log │ this tail differs)
|
|
41
|
+
│ [Gemma 4 native only: tool declarations after all of it] │
|
|
42
|
+
└──────────────────────────────────────────────────────────────────────────┘
|
|
43
|
+
shared by every turn of this session ─────────────────────────────────────
|
|
44
|
+
┌──────────────────────────────────────────────────────────────────────────┐
|
|
45
|
+
│ history user / assistant / tool messages, APPEND-ONLY │ !rollback, a failed or
|
|
46
|
+
│ (images: the newest N, ImagePlan) │ cancelled turn's tail,
|
|
47
|
+
│ ◄── remote Claude: breakpoint #2 on the last │ the image window
|
|
48
|
+
│ non-system message │ sliding, history
|
|
49
|
+
├──────────────────────────────────────────────────────────────────────────┤ sanitizing (thinking
|
|
50
|
+
│ tail notes reminders, turn notes, steers (system/user messages) │ stripped per spec)
|
|
51
|
+
│ new prompt this turn's user message │
|
|
52
|
+
└──────────────────────────────────────────────────────────────────────────┘
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
- The system prompt is built **once per loop** and rebuilt only on `@prompt_builder.reset!`: a model switch
|
|
56
|
+
(`Engine#switch_model!`), a plugin's `tools_changed!`, or a profile change. A per-turn rebuild would
|
|
57
|
+
break everything below it.
|
|
58
|
+
- On remote Claude, `LLM::PromptCache.mark` sends the system message as two text parts. The stable part carries
|
|
59
|
+
breakpoint #1, so a **new session** reads it from the cache. The last non-system message carries breakpoint
|
|
60
|
+
#2, so the **next request in the session** reads everything up to it. Mid and tail system messages never take
|
|
61
|
+
a breakpoint. The split point comes from `SystemPrompt#stable_length` via the message's `cache_split:`, which
|
|
62
|
+
never goes on the wire.
|
|
63
|
+
- Side requests (idle recap, `/btw`) go through `IdleClient` with their own short system prompt. On a
|
|
64
|
+
single-slot local server they can push the session's prefix out of the cache. Fixing that is slice 2 of the
|
|
65
|
+
plan (R4 `id_slot`, R2 turn-end warm-up).
|
|
66
|
+
|
|
67
|
+
## Rules for changes
|
|
68
|
+
|
|
69
|
+
1. **Stable first, volatile last.** Anything that differs between sessions or turns (ids, paths, times, the
|
|
70
|
+
model line, counters) goes at the very end of the system prompt, ordered by how often it changes. Never put
|
|
71
|
+
a per-session value in the stable part. A guard spec checks that model, location and session are the last
|
|
72
|
+
sections (`spec/system_prompt_model_spec.rb`).
|
|
73
|
+
2. **History is append-only.** Never rewrite an earlier message. Add a note at the tail instead. If a model's
|
|
74
|
+
spec forces a rewrite (e.g. stripping previous-turn thinking), accept the re-prefill and note it.
|
|
75
|
+
3. **Deterministic order.** Tools, memory lists and anything else that's rendered from a hash or a set needs a
|
|
76
|
+
fixed sort. Plugin tools are sorted by (bundle, name) whatever order their plugins finished starting in
|
|
77
|
+
(R5a).
|
|
78
|
+
4. **Don't rebuild the system prompt per turn.** Rebuild only when its content really changed (`reset!`).
|
|
79
|
+
5. **Measure.** Every generation logs `prompt=`, `cached=` and, where the provider reports it, `cache_write=`
|
|
80
|
+
on its `generation_completed` log line. `/stats` shows "cached N (P%)" and "cache writes". A change that
|
|
81
|
+
drops `cached=` on the second fresh session is a regression. Update the prompt snapshot fixtures with
|
|
82
|
+
`UPDATE_PROMPTS=1` and read the diff: a cache-friendly change moves sections, it doesn't reorder per run.
|
|
83
|
+
|
|
84
|
+
## Server-side settings worth knowing
|
|
85
|
+
|
|
86
|
+
- **Splash:** `--persistent-cache` (with `--max-cache-disk`) keeps restore points across restarts. Splash learns
|
|
87
|
+
a new branch point on the 2nd session, so the 3rd one gets the full reuse.
|
|
88
|
+
- **llama.cpp:** a single slot is shared by everything sent to it. `--parallel N` and `--cache-ram` give side
|
|
89
|
+
requests room without evicting the session. `--ctx-checkpoints` controls how many restore points a hybrid
|
|
90
|
+
model keeps.
|
|
91
|
+
- **OpenRouter / Anthropic:** the default cache TTL is 5 minutes. Provider routing (which upstream serves a
|
|
92
|
+
request) also decides whether a cache is there to hit.
|
|
93
|
+
|
|
94
|
+
Still to come: side requests on their own slot, a turn-end warm-up, image-window batches and longer remote TTLs.
|
data/docs/plugins.md
CHANGED
|
@@ -847,14 +847,20 @@ stream](hooks.md#watching-the-stream)) and cuts it:
|
|
|
847
847
|
thinking has short sentences too ("Hmm.", "Fine."), but spread out or all
|
|
848
848
|
different (code lines); a longer sentence ends the run, and lines inside a
|
|
849
849
|
```` ``` ```` code block don't count.
|
|
850
|
+
- Any `window_sentences` sentences in a row, short or long, holding at most
|
|
851
|
+
`window_distinct` different ones are a loop too: a cycle of 7 or 8 short
|
|
852
|
+
sentences, or one where a longer sentence ("I'll write the spec file
|
|
853
|
+
now.") keeps ending the short run. Real thinking holds 40 or more
|
|
854
|
+
different sentences in any 48; DeepSeek's write loops hold 4 to 9.
|
|
850
855
|
- Nothing triggers before `min_chars` of thinking. The watch sees a loop
|
|
851
856
|
within one batch (2000 chars, or a second) of its third cycle.
|
|
852
857
|
|
|
853
858
|
What happens (`action: retry`, the default):
|
|
854
859
|
|
|
855
860
|
1. The first loop in a turn: the generation is cut and the model asked
|
|
856
|
-
again, with a hidden note that it was cut off. The user sees
|
|
857
|
-
|
|
861
|
+
again, with a hidden note that it was cut off. The user sees the loop's
|
|
862
|
+
first sentence quoted,
|
|
863
|
+
`loop-guard> thinking repeats itself ("Wait, the count of the letter r…", 3 sentences ×3, 4k chars, 8 s): cut`,
|
|
858
864
|
and `↻ cut by loop-guard, asking again (1/1)`.
|
|
859
865
|
2. If the retry loops too, the turn is stopped ("stopped the turn: …", "✕
|
|
860
866
|
turn stopped by loop-guard") with a card that quotes the repeated
|
|
@@ -869,6 +875,12 @@ the first loop ends the turn as stopped by loop-guard, with the notice and
|
|
|
869
875
|
no card. `action: stop` stops the turn at the first loop; `action: notify` only
|
|
870
876
|
warns, once per generation.
|
|
871
877
|
|
|
878
|
+
The cut thinking never goes back to the model, but its last 20k chars are kept
|
|
879
|
+
in the session's folder, `thinking_tails.jsonl` (with a generation that ran to
|
|
880
|
+
the provider's output cap, or that a stop_turn ended): an archived session
|
|
881
|
+
keeps them. A turn's record counts its cuts and cap hits (`cuts`, `capped` in
|
|
882
|
+
`analytics.json`; `/stats` shows "thinking cuts" and "output cap hits").
|
|
883
|
+
|
|
872
884
|
```yaml
|
|
873
885
|
bundles:
|
|
874
886
|
loop-guard:
|
|
@@ -885,6 +897,8 @@ bundles:
|
|
|
885
897
|
min_words: 5 # shorter sentences only count toward a short run
|
|
886
898
|
short_run: 24 # this many short sentences in a row…
|
|
887
899
|
short_distinct: 6 # …with at most this many different ones is a loop
|
|
900
|
+
window_sentences: 48 # any this many sentences in a row…
|
|
901
|
+
window_distinct: 12 # …with at most this many different ones is a loop
|
|
888
902
|
forget_after: 10 # good steps in a row that forget a loop; 0: never
|
|
889
903
|
```
|
|
890
904
|
|
data/docs/testing.md
CHANGED
|
@@ -15,6 +15,18 @@ bug) or a broken spec: fix it, or disable that line with the reason (`# rubocop:
|
|
|
15
15
|
No spec reads your own setup. The suite points `XDG_CONFIG_HOME` and `XDG_STATE_HOME` at temp folders,
|
|
16
16
|
clears your `SAMAGOTCHI_*` environment, and keeps every spec off the network (WebMock).
|
|
17
17
|
|
|
18
|
+
## Worktrees
|
|
19
|
+
|
|
20
|
+
Each fix or feature gets its own worktree next to the main checkout:
|
|
21
|
+
|
|
22
|
+
```sh
|
|
23
|
+
script/worktree NAME [-b BRANCH] # ../samagotchi-NAME off main, branch NAME unless -b; prints the path
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
It runs `git worktree add ../samagotchi-NAME -b BRANCH main`, then `npm ci --prefer-offline` in it, so
|
|
27
|
+
`npm test` and `npm run e2e` work there right away (Ruby gems and Playwright's browsers are shared: nothing else
|
|
28
|
+
to install). It refuses when the folder or the branch already exists.
|
|
29
|
+
|
|
18
30
|
## Integration specs
|
|
19
31
|
|
|
20
32
|
Examples tagged `:integration` (`spec/integration/`, the "with LLM access" part of `spec/hooks/integration_spec.rb`)
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: loop-guard
|
|
3
|
-
version: 0.3.
|
|
3
|
+
version: 0.3.3
|
|
4
4
|
scope: system
|
|
5
5
|
description: "Breaks loops: the same tool call with the same result again and again in one turn is denied with advice (after a few denies the turn stops with a card), and thinking that repeats itself while it streams is cut and the model asked again"
|
|
6
6
|
trust_level: reviewed
|
|
7
7
|
plugin:
|
|
8
8
|
file: plugin.rb
|
|
9
|
-
sha256: sha256:
|
|
9
|
+
sha256: sha256:8cf255cd1cc0846a759fcf70144af53bfc634c9d5c766981ce0ef5e476749e0a
|
|
10
10
|
requires_chi: ">= 0.6.0"
|
|
@@ -56,12 +56,20 @@ require "digest"
|
|
|
56
56
|
# too ("Hmm.", "Fine."), but spread out, or in runs of different ones
|
|
57
57
|
# (code lines); a sentence of min_words or more ends the run, one with no
|
|
58
58
|
# letters or digits neither counts nor ends it. Lines inside a ``` code
|
|
59
|
-
# block are not short sentences
|
|
59
|
+
# block are not short sentences;
|
|
60
|
+
# - a low-diversity window: among the last window_sentences sentences of any
|
|
61
|
+
# length, at most window_distinct different ones (exact, normalized). It
|
|
62
|
+
# sees a cycle too long for the short run (7-8 short sentences), or one
|
|
63
|
+
# where a longer sentence keeps ending the run ("Go. OK. I'll write the
|
|
64
|
+
# spec file now. Go."). Real thinking holds 40+ different sentences in
|
|
65
|
+
# any 48 (the lowest seen: 42); a loop holds under 10. Lines inside a ```
|
|
66
|
+
# code block don't count.
|
|
60
67
|
# Nothing triggers before min_chars of thinking. O(new chars) per feed.
|
|
61
68
|
class ThinkingWatch
|
|
62
69
|
DEFAULTS = { "watch" => true, "action" => "retry", "min_chars" => 2000, "repeats" => 3, "max_period" => 6,
|
|
63
70
|
"similarity" => 0.5, "min_span_sentences" => 6, "min_span_chars" => 600, "max_same" => 8,
|
|
64
|
-
"min_words" => 5, "short_run" => 24, "short_distinct" => 6, "
|
|
71
|
+
"min_words" => 5, "short_run" => 24, "short_distinct" => 6, "window_sentences" => 48,
|
|
72
|
+
"window_distinct" => 12, "forget_after" => 10 }.freeze
|
|
65
73
|
ACTIONS = %w[retry stop notify].freeze
|
|
66
74
|
MAX_CARRY = 400
|
|
67
75
|
# One sentence again and again needs this much more alike than a cycle.
|
|
@@ -81,7 +89,7 @@ class ThinkingWatch
|
|
|
81
89
|
out["watch"] = !%w[false no off 0].include?(raw["watch"].to_s.downcase) if raw.key?("watch")
|
|
82
90
|
out["action"] = raw["action"].to_s if ACTIONS.include?(raw["action"].to_s)
|
|
83
91
|
%w[min_chars repeats max_period min_span_sentences min_span_chars max_same min_words short_run
|
|
84
|
-
short_distinct].each do |key|
|
|
92
|
+
short_distinct window_sentences window_distinct].each do |key|
|
|
85
93
|
number = Integer(raw[key].to_s, exception: false)
|
|
86
94
|
out[key] = number if number&.positive?
|
|
87
95
|
end
|
|
@@ -104,6 +112,8 @@ class ThinkingWatch
|
|
|
104
112
|
@min_words = settings["min_words"]
|
|
105
113
|
@short_run = settings["short_run"]
|
|
106
114
|
@short_distinct = settings["short_distinct"]
|
|
115
|
+
@window_sentences = settings["window_sentences"]
|
|
116
|
+
@window_distinct = settings["window_distinct"]
|
|
107
117
|
@same_similarity = [@similarity + SAME_MARGIN, 0.95].min
|
|
108
118
|
@carry = +""
|
|
109
119
|
@window = [] # the last max_period + 1 sentences
|
|
@@ -112,6 +122,8 @@ class ThinkingWatch
|
|
|
112
122
|
@counts = Hash.new(0)
|
|
113
123
|
@short_count = 0 # short sentences in a row
|
|
114
124
|
@short_window = [] # the last short_run of them, [key, text]
|
|
125
|
+
@recent = [] # the last window_sentences sentences, [key, text]
|
|
126
|
+
@recent_counts = Hash.new(0) # key => times in @recent
|
|
115
127
|
@in_code = false
|
|
116
128
|
@thinking_chars = 0
|
|
117
129
|
end
|
|
@@ -153,7 +165,8 @@ class ThinkingWatch
|
|
|
153
165
|
end
|
|
154
166
|
words = text.downcase.gsub(/[^[:alnum:]]+/, " ").split
|
|
155
167
|
short = short_loop(text.strip, words) unless @in_code
|
|
156
|
-
|
|
168
|
+
window = window_loop(text.strip, words) unless @in_code
|
|
169
|
+
return (@thinking_chars >= @min_chars ? short || window : nil) if words.length < @min_words
|
|
157
170
|
|
|
158
171
|
key = words.join(" ")
|
|
159
172
|
sentence = Sentence.new(text: text.strip, words: words.map(&:hash).uniq,
|
|
@@ -164,7 +177,27 @@ class ThinkingWatch
|
|
|
164
177
|
@window.shift if @window.length > @max_period + 1
|
|
165
178
|
return nil if @thinking_chars < @min_chars
|
|
166
179
|
|
|
167
|
-
cycle || (same >= @max_same ? Loop.new(period: 1, times: same, sentences: [sentence.text], chars: nil) : nil)
|
|
180
|
+
cycle || (same >= @max_same ? Loop.new(period: 1, times: same, sentences: [sentence.text], chars: nil) : nil) || window
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
# The last window_sentences sentences (any length) hold at most
|
|
184
|
+
# window_distinct different ones: a loop. Its sentences are the most
|
|
185
|
+
# frequent first.
|
|
186
|
+
def window_loop(text, words)
|
|
187
|
+
return nil if words.empty?
|
|
188
|
+
|
|
189
|
+
key = words.join(" ")
|
|
190
|
+
@recent << [key, text]
|
|
191
|
+
@recent_counts[key] += 1
|
|
192
|
+
if @recent.length > @window_sentences
|
|
193
|
+
old, = @recent.shift
|
|
194
|
+
@recent_counts.delete(old) if (@recent_counts[old] -= 1).zero?
|
|
195
|
+
end
|
|
196
|
+
return nil if @recent.length < @window_sentences || @recent_counts.size > @window_distinct
|
|
197
|
+
|
|
198
|
+
texts = @recent.to_h # key => its last text
|
|
199
|
+
top = @recent_counts.sort_by { |_key, n| -n }.first(3).map { |key, _n| texts[key] }
|
|
200
|
+
Loop.new(period: @recent_counts.size, times: @window_sentences / @recent_counts.size, sentences: top, chars: nil)
|
|
168
201
|
end
|
|
169
202
|
|
|
170
203
|
def short_loop(text, words)
|
|
@@ -276,14 +309,17 @@ class Plugin
|
|
|
276
309
|
@watch_done = true
|
|
277
310
|
what = "#{found.period == 1 ? "one sentence" : "#{found.period} sentences"} ×#{found.times}, " \
|
|
278
311
|
"#{(@watch.thinking_chars / 1000.0).round}k chars, #{(event[:elapsed_ms].to_i / 1000.0).round} s"
|
|
312
|
+
# The notices quote the loop's sentences on one line (a short cycle
|
|
313
|
+
# whole), as the stop card does.
|
|
314
|
+
quoted = "\"#{cut(found.sentences.first(3).join(" ").gsub(/\s+/, " "))}\", #{what}"
|
|
279
315
|
case @thinking["action"]
|
|
280
316
|
when "notify"
|
|
281
|
-
ctx.notify("thinking repeats itself (#{
|
|
317
|
+
ctx.notify("thinking repeats itself (#{quoted})", level: :warn)
|
|
282
318
|
when "retry"
|
|
283
319
|
@thinking_loops += 1
|
|
284
320
|
return stop_thinking(ctx, found, what) if @thinking_loops > 1
|
|
285
321
|
|
|
286
|
-
ctx.notify("thinking repeats itself (#{
|
|
322
|
+
ctx.notify("thinking repeats itself (#{quoted}): cut", level: :warn) if ctx.stop_generation("its thinking kept repeating itself")
|
|
287
323
|
else
|
|
288
324
|
stop_thinking(ctx, found, what)
|
|
289
325
|
end
|