samagotchi 0.99.0 → 0.100.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +20 -1
- data/README.md +2 -2
- data/docs/architecture.md +5 -2
- data/docs/bundle-authoring.md +1 -1
- data/docs/cli.md +1 -1
- data/docs/guardrails.md +2 -2
- data/docs/hooks.md +4 -1
- data/docs/memory.md +1 -1
- data/docs/plugins.md +67 -2
- data/docs/releasing.md +1 -1
- data/docs/sub-agent.md +7 -2
- data/lib/samagotchi/bundles/core/manifest.yml +4 -3
- data/lib/samagotchi/bundles/system/manifest.yml +1 -1
- data/lib/samagotchi/bundles/work-cap/manifest.yml +13 -0
- data/lib/samagotchi/bundles/work-cap/plugin.rb +391 -0
- data/lib/samagotchi/engine.rb +2 -1
- data/lib/samagotchi/hooks/loader.rb +17 -4
- data/lib/samagotchi/plugin/api.rb +2 -1
- data/lib/samagotchi/plugin/context.rb +10 -2
- data/lib/samagotchi/session_metrics/usage.rb +31 -0
- data/lib/samagotchi/session_metrics.rb +48 -5
- data/lib/samagotchi/testing/host.rb +30 -1
- data/lib/samagotchi/testing/recorder.rb +5 -1
- data/lib/samagotchi/version.rb +1 -1
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 10cafc8e1deb147528195213bd69b946f428918a8c344c57022b2c905cd2f4fb
|
|
4
|
+
data.tar.gz: 6b6764b06a4effc61ac2edd1d2221d7afd232b2a8ce3470f0b14280d3ded432d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a4da078e7bb452a96fbc8d38f9565946fa12a3e3b8185e35f26032fcc2a3ae76249ec5ad7712dd12b74fec0f52db5fac7c03890f5074fca747985944b50f8e26
|
|
7
|
+
data.tar.gz: 3cc980b577869624c31a8c577b69612198569875d5c3b1e2f9070a4aebed2b5d4e6d4d29d44e25e56e2144b2346f4017352dca2e7a01b2898bc6fa5046b4459e
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,24 @@ and commands may change between minor versions. How releases are made:
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [0.100.0] - 2026-10-22
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- Plugins can read the session's work so far with `ctx.usage`: turns, tool calls (by tool), steps and active time
|
|
16
|
+
(generations plus tool calls, never a wait on a question), across workers, the same numbers `/stats` shows. The
|
|
17
|
+
bundle test kit sets it with `bundle.set(usage: {...})`. See [docs/plugins.md](docs/plugins.md).
|
|
18
|
+
- The `work-cap` bundle (in the `core` profile; `chi update` installs it): an optional per-session cap on tool calls
|
|
19
|
+
and active minutes across turns, the bound a session whose step-limit Continue keeps being answered otherwise
|
|
20
|
+
lacks. At a cap the turn pauses on Continue / Stop, the question naming the session's totals; `/workcap` shows or
|
|
21
|
+
sets it, `chi send --new --set work-cap.tool_calls=400` caps a child. Inert until a cap is set. See
|
|
22
|
+
[docs/plugins.md](docs/plugins.md#the-work-cap-bundle).
|
|
23
|
+
|
|
24
|
+
### Fixed
|
|
25
|
+
|
|
26
|
+
- An empty `hooks:` key in config.yml no longer crashes chi, and a malformed `hooks:` section (or one event's
|
|
27
|
+
entry) warns instead of being dropped without a word.
|
|
28
|
+
|
|
11
29
|
## [0.99.0] - 2026-10-22
|
|
12
30
|
|
|
13
31
|
### Added
|
|
@@ -3774,7 +3792,8 @@ and long-lived sessions.
|
|
|
3774
3792
|
- A macOS desktop helper (`chi desktop install`): a "Send to chi" Service and a
|
|
3775
3793
|
hotkey panel that send selected text or the clipboard to your sessions.
|
|
3776
3794
|
|
|
3777
|
-
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.
|
|
3795
|
+
[Unreleased]: https://github.com/dm1try/samagotchi/compare/v0.100.0...HEAD
|
|
3796
|
+
[0.100.0]: https://github.com/dm1try/samagotchi/compare/v0.99.0...v0.100.0
|
|
3778
3797
|
[0.99.0]: https://github.com/dm1try/samagotchi/compare/v0.98.0...v0.99.0
|
|
3779
3798
|
[0.98.0]: https://github.com/dm1try/samagotchi/compare/v0.97.0...v0.98.0
|
|
3780
3799
|
[0.97.0]: https://github.com/dm1try/samagotchi/compare/v0.96.0...v0.97.0
|
data/README.md
CHANGED
|
@@ -78,8 +78,8 @@ Messages API, picks the
|
|
|
78
78
|
model (or asks, when there are several), sends one test request and writes
|
|
79
79
|
the config. With a config already there, it adds a `hosts:` entry and keeps
|
|
80
80
|
the rest of the file. Then it installs the `core` bundles (loop-guard,
|
|
81
|
-
check-in, guardrails, slow-commands, agents-md, cost-cap: the safety set for a
|
|
82
|
-
small local model, the cost
|
|
81
|
+
check-in, guardrails, slow-commands, agents-md, cost-cap, work-cap: the safety set for a
|
|
82
|
+
small local model, the cost and work caps inert until you set one) and, on a terminal, offers `dev` (known-names, mcp, btw, skills,
|
|
83
83
|
source-links, github-pr, coordinator, claude-peer, codex-peer, ruby, model-notes; see
|
|
84
84
|
[Bundle profiles](docs/memory.md#bundle-profiles-core-and-dev)). Edit the
|
|
85
85
|
file later as in [Configure](#configure); `chi bootstrap --help` has the
|
data/docs/architecture.md
CHANGED
|
@@ -411,7 +411,10 @@ stay the turns' only. `SessionMetrics#persist` is serialized (its own mutex, tak
|
|
|
411
411
|
threads' saves land in snapshot order.
|
|
412
412
|
A bundle's `ctx.cost` reads the same sums (`SessionMetrics#cost`, a frozen `SessionMetrics::Cost`): the `tokens`
|
|
413
413
|
block's `cost_sum` and `cost_estimate_sum` with the running turn's finished generations folded in, plus the `side`
|
|
414
|
-
block's, so it counts what earlier workers spent too ([plugins.md](plugins.md)).
|
|
414
|
+
block's, so it counts what earlier workers spent too ([plugins.md](plugins.md)). Its `ctx.usage` reads the work
|
|
415
|
+
(`SessionMetrics#usage`, a frozen `SessionMetrics::Usage`): the snapshot's `turns`, `tool_calls_total` /
|
|
416
|
+
`tool_calls_by_tool` and `iterations_total`, and `active_ms`, the generations' `gen_ms` plus the finished tool
|
|
417
|
+
records' `duration_ms` (tallied per tool as they are counted; `ask_user_question`'s, a wait on the user, left out).
|
|
415
418
|
|
|
416
419
|
The page's frequent reads of `GET /api/sessions/:id` stay light too:
|
|
417
420
|
- `?tail=1&recent=1` (each turn's end, a cancel, an answer display): `{tail, session: {id, status,
|
|
@@ -609,7 +612,7 @@ A map of the parts the overview above doesn't name (paths under `lib/samagotchi/
|
|
|
609
612
|
| Attached context | `context_sources.rb` (`ContextSources`), `context_poller.rb`, `context_absorber.rb`, `peer_sources.rb` (`PeerSources`: the sources peer bundles declare) | External sources a session keeps fresh, and the peer loop's wakes ([context.md](context.md)). |
|
|
610
613
|
| LLM context strategies | `llm_context_strategy.rb`, `llm_context_view.rb`, `llm_context_stale.rb`, `llm_context_forget.rb` | What a request sends of the history (none, stale, forget) ([internals/llm-context-forget.md](internals/llm-context-forget.md)). |
|
|
611
614
|
| Session chains | `session_continue.rb` (`SessionContinue`), `session_chain.rb` (`SessionChain`) | A session that continues an earlier one. `SessionChain.opener_for` gives the text web `continue →` prefills in the new link: the previous link's own first typed message when that link continues one itself, else nil (the 201's `opener`; not saved) ([sessions.md](sessions.md)). |
|
|
612
|
-
| Analytics | `analytics_file.rb` (`AnalyticsFile`), `session_metrics/totals.rb` (`SessionMetrics::Totals`), `session_metrics/model_segment.rb` (`SessionMetrics::ModelSegment`) | A session's `analytics.json` (`SessionMetrics#persist`); the sums over its turn records (`Totals`: tokens, cost, speeds, kept as running counters) and their split per model after a `/model` switch (`ModelSegment`: a run of turns on one model, the `model_segments` rows); `session_metrics/cost.rb` (`SessionMetrics::Cost`): the spend `ctx.cost` reads. `session_metrics/side_totals.rb` (`SessionMetrics::SideTotals`) and `side_usage.rb` (`SideUsage`): the side requests' sums, apart from the turns'. |
|
|
615
|
+
| Analytics | `analytics_file.rb` (`AnalyticsFile`), `session_metrics/totals.rb` (`SessionMetrics::Totals`), `session_metrics/model_segment.rb` (`SessionMetrics::ModelSegment`) | A session's `analytics.json` (`SessionMetrics#persist`); the sums over its turn records (`Totals`: tokens, cost, speeds, kept as running counters) and their split per model after a `/model` switch (`ModelSegment`: a run of turns on one model, the `model_segments` rows); `session_metrics/cost.rb` (`SessionMetrics::Cost`): the spend `ctx.cost` reads; `session_metrics/usage.rb` (`SessionMetrics::Usage`): the work `ctx.usage` reads. `session_metrics/side_totals.rb` (`SessionMetrics::SideTotals`) and `side_usage.rb` (`SideUsage`): the side requests' sums, apart from the turns'. |
|
|
613
616
|
| Images / vision | `image_store.rb` (`ImageStore`), `vision_support.rb`, `vision_context.rb` | Stored prompt images and whether a model takes them. |
|
|
614
617
|
| Toolkits | `toolkit.rb` (`Toolkit`), `toolkit_binding.rb` (`ToolkitBinding`, the Engine's seam), `toolkit_command.rb` (`/toolkit`), `tools/registry.rb` (`Registry#toolkit`, `#left_out_names`), `web/public/toolkit_chip.js` | Which registered tools a session's prompts declare: saved in the session, resolved at a turn's start (`Engine#apply_toolkit!`), named in the prompt's not-loaded line, a kit's prompt text after the appended files; `chi.toolkit` and `ctx.use_toolkit` for plugins ([configuration.md](configuration.md#toolkits)). |
|
|
615
618
|
| Tool index | `tools/tool_index.rb` (`Tools::ToolIndex`), `tools/deferred_tools.rb` | `find_tools` and `call_tool`: the tools a session knows but doesn't declare one by one (an mcp bundle's servers through `chi.tool_source`, the built-ins its toolkit leaves out), searched and called by name ([plugins.md](plugins.md)). On `api: anthropic` they also go out deferred, a hit loads its matches and the model calls them by name ([internals/anthropic.md](internals/anthropic.md)). |
|
data/docs/bundle-authoring.md
CHANGED
|
@@ -129,7 +129,7 @@ class, a hand-built `ctx`), that is a gap in the kit: say so to the user rather
|
|
|
129
129
|
of a checksum mismatch, and a line with no sha installs unchecked. `chi bundle check --fix` rewrites them; run it
|
|
130
130
|
after the last edit.
|
|
131
131
|
- **requires_chi.** Use the chi the user runs (`chi --version`). `check` warns when the plugin calls something
|
|
132
|
-
newer than the floor (`ctx.http` needs 0.80.0, `ctx.cost` 0.83.0, `ctx.sessions.children` 0.34.0). A keyword added
|
|
132
|
+
newer than the floor (`ctx.http` needs 0.80.0, `ctx.cost` 0.83.0, `ctx.usage` 0.100.0, `ctx.sessions.children` 0.34.0). A keyword added
|
|
133
133
|
later it can't see: `ctx.ask_model(model:)` needs 0.80.0, `ctx.sessions.send(delivery:)` 0.64.0.
|
|
134
134
|
- **trust_level.** Leave it out. `reviewed` / `trusted` are for someone who reviewed the bundle, not its author.
|
|
135
135
|
- **Restart.** A session loads bundles when it starts: after an install or a settings change, the user starts a
|
data/docs/cli.md
CHANGED
|
@@ -84,7 +84,7 @@ chi bootstrap # try localhost 8080, 11434, 1234, 8000
|
|
|
84
84
|
or with anchors gets the lines printed to paste instead. `--dry-run` shows
|
|
85
85
|
what it would write.
|
|
86
86
|
- **The bundles.** Unless writing the config failed, it installs the system
|
|
87
|
-
bundle and the `core` profile (loop-guard, check-in, guardrails, slow-commands, agents-md, cost-cap), one line
|
|
87
|
+
bundle and the `core` profile (loop-guard, check-in, guardrails, slow-commands, agents-md, cost-cap, work-cap), one line
|
|
88
88
|
each; on a terminal it then asks `Also install dev (known-names, mcp, btw,
|
|
89
89
|
skills, source-links, github-pr, coordinator, claude-peer, codex-peer, ruby, model-notes)? [y/N]`. Run again, it installs nothing new: a bundle
|
|
90
90
|
you uninstalled stays out ([Bundle profiles](memory.md#bundle-profiles-core-and-dev)).
|
data/docs/guardrails.md
CHANGED
|
@@ -125,7 +125,7 @@ answers it. Every question's options carry an **effect**, set by whoever asks:
|
|
|
125
125
|
- `deny`: Deny on an approval, Stop on the step-limit question or a plugin's
|
|
126
126
|
stop option;
|
|
127
127
|
- `spend`: more time and tokens, no new permission (Continue on the step-limit
|
|
128
|
-
question, cost-cap's Continue);
|
|
128
|
+
question, cost-cap's and work-cap's Continue);
|
|
129
129
|
- `permit`: one tool call (an approval's "Allow once");
|
|
130
130
|
- `permit_wide`: more than one call (this session, this repo or directory,
|
|
131
131
|
the rule).
|
|
@@ -140,7 +140,7 @@ guardrails:
|
|
|
140
140
|
- `hard`: `deny` and `answer` only. A parent can deny an approval
|
|
141
141
|
(`--option Deny --text "why"`, `--text` alone or `--dismiss`), Stop a
|
|
142
142
|
step-limit question and answer plain questions.
|
|
143
|
-
- `standard`: also `spend`: a parent may Continue a turn (or cost-cap's cap).
|
|
143
|
+
- `standard`: also `spend`: a parent may Continue a turn (or cost-cap's or work-cap's cap).
|
|
144
144
|
- `soft` (the default): also `permit`: `chi answer --option "Allow once"`
|
|
145
145
|
goes through, picked by the option's scope, not its label.
|
|
146
146
|
|
data/docs/hooks.md
CHANGED
|
@@ -26,6 +26,9 @@ hooks:
|
|
|
26
26
|
on_error: skip
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
+
Each event key holds a list of entries. A `hooks:` that isn't a mapping, or an event key that isn't a list
|
|
30
|
+
(`before_turn: guard.rb`), warns at each start and loads nothing for it; an empty key loads nothing, silently.
|
|
31
|
+
|
|
29
32
|
## Plugin Format
|
|
30
33
|
|
|
31
34
|
Each plugin is a `.rb` file in the hooks directory. The class name must match
|
|
@@ -672,7 +675,7 @@ Notes:
|
|
|
672
675
|
- Ordering: bundle hooks fire by `(priority, bundle_name, hook_name)` (lower priority first), then plain `config.yml` hooks in registration order.
|
|
673
676
|
- Settings: a hook class with `initialize(settings = {})` gets the bundle's section of `config.yml` `bundles:` (see [Settings](#settings)).
|
|
674
677
|
- A bundle can also ship a `plugin.rb` whose `chi.on(event)` blocks are bundle hooks too, next to commands and tools; see [Plugins](plugins.md). There `ctx.notify`, `ctx.steer` and the other `ctx` helpers act as this page's `event[:notify]`… do for the event the block runs for.
|
|
675
|
-
- Shipped bundles (each in a profile: `core` or `dev`, see [Bundle profiles](memory.md#bundle-profiles-core-and-dev)): `chi bundle install guardrails` (rules, see [Guardrails](guardrails.md#the-guardrails-bundle)) and `chi bundle install known-names` (a hook, see [Guardrails](guardrails.md#the-known-names-bundle)), `chi bundle install source-links` (a hook: announces source refs, see [The source-links bundle](#the-source-links-bundle)), `chi bundle install btw` (a plugin: `/btw`, see [Plugins](plugins.md#the-btw-bundle)), `chi bundle install mcp` (a plugin: tools from MCP servers, see [Plugins](plugins.md#the-mcp-bundle)) `chi bundle install loop-guard` (a plugin: breaks tool-call loops, see [Plugins](plugins.md#the-loop-guard-bundle)), `chi bundle install check-in` (a plugin: checks on a long turn, see [Plugins](plugins.md#the-check-in-bundle)), `chi bundle install slow-commands` (a plugin: commands that ran past the execute limit are run as tasks next time, see [Plugins](plugins.md#the-slow-commands-bundle)), `chi bundle install agents-md` (a plugin: a nested AGENTS.md is steered into the conversation on the first touch under its directory, see [Plugins](plugins.md#the-agents-md-bundle)), `chi bundle install cost-cap` (a plugin: a per-session $ cap, `/cap`, see [Plugins](plugins.md#the-cost-cap-bundle)), `chi bundle install ruby` (a plugin: an RSpec run with no examples at all, a hint, see [Plugins](plugins.md#the-ruby-bundle)) `chi bundle install skills` (a plugin: `/skill`, versions of skills, see [Plugins](plugins.md#the-skills-bundle)), `chi bundle install github-pr` (attached context: the branch's open PR, see [Plugins](plugins.md#the-github-pr-bundle)), `chi bundle install coordinator` (a plugin: parallel work in worktrees and child sessions, see [Plugins](plugins.md#the-coordinator-bundle)), `chi bundle install claude-peer` / `codex-peer` (a plugin: talk to a Claude Code or Codex session, see [Plugins](plugins.md#the-claude-peer-bundle)) and `chi bundle install model-notes` (memories: reviewed model notes, see [Plugins](plugins.md#the-model-notes-bundle)).
|
|
678
|
+
- Shipped bundles (each in a profile: `core` or `dev`, see [Bundle profiles](memory.md#bundle-profiles-core-and-dev)): `chi bundle install guardrails` (rules, see [Guardrails](guardrails.md#the-guardrails-bundle)) and `chi bundle install known-names` (a hook, see [Guardrails](guardrails.md#the-known-names-bundle)), `chi bundle install source-links` (a hook: announces source refs, see [The source-links bundle](#the-source-links-bundle)), `chi bundle install btw` (a plugin: `/btw`, see [Plugins](plugins.md#the-btw-bundle)), `chi bundle install mcp` (a plugin: tools from MCP servers, see [Plugins](plugins.md#the-mcp-bundle)) `chi bundle install loop-guard` (a plugin: breaks tool-call loops, see [Plugins](plugins.md#the-loop-guard-bundle)), `chi bundle install check-in` (a plugin: checks on a long turn, see [Plugins](plugins.md#the-check-in-bundle)), `chi bundle install slow-commands` (a plugin: commands that ran past the execute limit are run as tasks next time, see [Plugins](plugins.md#the-slow-commands-bundle)), `chi bundle install agents-md` (a plugin: a nested AGENTS.md is steered into the conversation on the first touch under its directory, see [Plugins](plugins.md#the-agents-md-bundle)), `chi bundle install cost-cap` (a plugin: a per-session $ cap, `/cap`, see [Plugins](plugins.md#the-cost-cap-bundle)), `chi bundle install work-cap` (a plugin: a per-session cap on tool calls and active minutes, `/workcap`, see [Plugins](plugins.md#the-work-cap-bundle)), `chi bundle install ruby` (a plugin: an RSpec run with no examples at all, a hint, see [Plugins](plugins.md#the-ruby-bundle)) `chi bundle install skills` (a plugin: `/skill`, versions of skills, see [Plugins](plugins.md#the-skills-bundle)), `chi bundle install github-pr` (attached context: the branch's open PR, see [Plugins](plugins.md#the-github-pr-bundle)), `chi bundle install coordinator` (a plugin: parallel work in worktrees and child sessions, see [Plugins](plugins.md#the-coordinator-bundle)), `chi bundle install claude-peer` / `codex-peer` (a plugin: talk to a Claude Code or Codex session, see [Plugins](plugins.md#the-claude-peer-bundle)) and `chi bundle install model-notes` (memories: reviewed model notes, see [Plugins](plugins.md#the-model-notes-bundle)).
|
|
676
679
|
- Installing a bundle executes its hook code at `Engine` startup. Only install bundles you trust, as you would a gem. Hooks are **not** executed at install time (copy-only); they are `module_eval`'d at `Engine.new` inside per-bundle `Samagotchi::Bundles::<name>` namespaces (no top-level `require` collisions). Keep hook files side-effect-free at load time; do work in `#call` — top-level side effects (require, IO, `at_exit`, global assignment) run once per `Engine.new` (class redefinition is idempotent).
|
|
677
680
|
|
|
678
681
|
Lifecycle:
|
data/docs/memory.md
CHANGED
|
@@ -387,7 +387,7 @@ profiles, meta bundles that install a set of them:
|
|
|
387
387
|
|
|
388
388
|
| profile | bundles |
|
|
389
389
|
|---|---|
|
|
390
|
-
| `core` | loop-guard, check-in, guardrails, slow-commands, agents-md, cost-cap: the recommended safety set |
|
|
390
|
+
| `core` | loop-guard, check-in, guardrails, slow-commands, agents-md, cost-cap, work-cap: the recommended safety set |
|
|
391
391
|
| `dev` | known-names, mcp, btw, skills, source-links, github-pr, coordinator, claude-peer, codex-peer, ruby, model-notes |
|
|
392
392
|
|
|
393
393
|
`chi bootstrap` installs `core` and, on a terminal, offers `dev`. An existing
|
data/docs/plugins.md
CHANGED
|
@@ -751,6 +751,7 @@ full label, `plugin.rb (bundle my-bundle)`.
|
|
|
751
751
|
| `ctx.model` | the model the session runs on now: its resolved ref (`host:id`), right after `/model` too |
|
|
752
752
|
| `ctx.model_key` | that model's memory overlay key (`<name>.<key>.md`, what `memory_write current_model_only` writes) |
|
|
753
753
|
| `ctx.cost` | the session's spend so far, read now, as a frozen value: `total` (USD, reported + estimated, side requests included; 0.0 before any), `reported` (by providers, `usage.cost`), `estimated` (from `hosts.<name>.models` prices; a generation is one or the other, never both), `side` (recap, `/btw`, `ask_model`: their share of `total`) and `known?` (false while generations or side requests ran but none was reported or priced, so a limit on it can't be enforced; a session with some priced generations reads known, the unpriced ones counting 0). It includes the running turn's finished generations and what earlier workers of the session spent. Use its methods, not its class. Needs chi 0.83.0 |
|
|
754
|
+
| `ctx.usage` | the session's work so far, read now, as a frozen value: `turns` (finished ones and the running one), `tool_calls` and `tool_calls_by_tool` (every call, running ones included; canonical names: an alias counts under its tool, a `call_tool` call under the inner tool), `steps` (the loop rounds that dispatched tools, `/stats`' iterations) and `active_ms` (`active_minutes` too): work time, the generations' time plus the finished tool calls' time, never the waits between them (a question, an approval, the user's answer to `ask_user_question`), with `tool_ms_by_tool` its tool part by tool, so a waiting tool's time (`task_wait`) can be left out. Its counts are the ones `/stats` and `chi sessions stats` show, earlier workers' included. Use its methods, not its class. Needs chi 0.100.0 (guard it with `ctx.respond_to?(:usage)`) |
|
|
754
755
|
| `ctx.settings` | the bundle's settings, frozen: config.yml's `bundles: <bundle>:` with the session's own `--set` values laid over it ([Per-session settings](#per-session-settings)); the config alone until the session is bound (in `register` and `chi.ctx`) |
|
|
755
756
|
| `ctx.data_dir` | `$XDG_STATE_HOME/samagotchi/plugins/<bundle>/`, created on first use. A session's own state goes in `sessions/<id>.json` or `sessions/<id>/` there: chi removes it when the session is deleted, discarded or pruned |
|
|
756
757
|
| `ctx.log` | `ctx.log.info(:event, key: value)`: debug-log records tagged `plugins`, with `bundle=<bundle>` |
|
|
@@ -1120,7 +1121,7 @@ CHANGELOG line and bumps the shipped bundles that use it. Anything else under
|
|
|
1120
1121
|
|
|
1121
1122
|
**Stable**
|
|
1122
1123
|
|
|
1123
|
-
1. `Samagotchi::Plugin::Api` / `chi.*` and `ctx` as documented above (`ctx.cost`'s
|
|
1124
|
+
1. `Samagotchi::Plugin::Api` / `chi.*` and `ctx` as documented above (`ctx.cost`'s and `ctx.usage`'s values by their methods), including
|
|
1124
1125
|
`Samagotchi::Plugin::ToolResult`, `Samagotchi::Plugin::ModelError` (and its
|
|
1125
1126
|
subclass `Samagotchi::Plugin::ModelCancelled`),
|
|
1126
1127
|
`Samagotchi::Plugin::Service::Stopped`, `Samagotchi::Plugin::Sessions::Error`
|
|
@@ -1187,6 +1188,7 @@ bundle.sessions.on(:children) { |all:| [] } # ctx.sessions, scripte
|
|
|
1187
1188
|
bundle.http.stub(:post, "http://127.0.0.1:9000/classify", json: { label: "safe" }) # ctx.http, never the network
|
|
1188
1189
|
bundle.set(running: false) # the session's values: cwd:, messages:, cancelled: …
|
|
1189
1190
|
bundle.set(cost: 2.5) # ctx.cost: reported USD, or { reported:, estimated:, side:, known: }
|
|
1191
|
+
bundle.set(usage: { tool_calls: 40, active_ms: 600_000 }) # ctx.usage: { turns:, tool_calls:, tool_calls_by_tool:, steps:, active_ms:, tool_ms_by_tool: }
|
|
1190
1192
|
bundle.set(session_settings: { "cap" => 1 }) # the session's --set values (Bundle.load takes session_settings: too)
|
|
1191
1193
|
bundle.tool_loads.last.names # ctx.load_tools calls; set(load_tools_staged: [...]) limits what they stage
|
|
1192
1194
|
bundle.shutdown
|
|
@@ -1222,6 +1224,10 @@ bundle.shutdown
|
|
|
1222
1224
|
- **`ctx.cost`** is zero and known unless the test sets it: `bundle.set(cost: 2.5)` (reported USD), a Hash of the
|
|
1223
1225
|
parts (`reported:`, `estimated:`, `side:`, `known:`; `total` is reported + estimated) or a callable giving either,
|
|
1224
1226
|
called at each read (a counting lambda shows whether the bundle read it).
|
|
1227
|
+
- **`ctx.usage`** is zero unless the test sets it: `bundle.set(usage: {...})`, a Hash of the parts (`turns:`,
|
|
1228
|
+
`tool_calls:`, `tool_calls_by_tool:`, `steps:`, `active_ms:`, `tool_ms_by_tool:`; zeros for the rest, an unknown
|
|
1229
|
+
part raises; `tool_calls` defaults to the sum of `tool_calls_by_tool`, `active_ms` to the sum of
|
|
1230
|
+
`tool_ms_by_tool`) or a callable giving one, called at each read.
|
|
1225
1231
|
- **`session_settings:`** (on `Bundle.load` and `bundle.set`) are the session's own `--set` values for the bundle,
|
|
1226
1232
|
keys its manifest lists in `session_settings:` (another key, or a value of another type than the manifest gives
|
|
1227
1233
|
it, raises `ArgumentError`, as chi refuses it). `ctx.settings`
|
|
@@ -1680,6 +1686,64 @@ its own card and doesn't wake its parent ([sessions.md](sessions.md#delegating))
|
|
|
1680
1686
|
The state is one file per session, `<data_dir>/sessions/<session id>.json` (`{"cap": 3.0, "step": 3.0}`), removed
|
|
1681
1687
|
with the session.
|
|
1682
1688
|
|
|
1689
|
+
Its sibling for work instead of money, tool calls and active minutes across the session's turns, is
|
|
1690
|
+
[the work-cap bundle](#the-work-cap-bundle): the cap a local model (no cost) can have.
|
|
1691
|
+
|
|
1692
|
+
## The work-cap bundle
|
|
1693
|
+
|
|
1694
|
+
`chi bundle install work-cap` installs the bundle shipped with chi (`chi bootstrap` installs it with the `core`
|
|
1695
|
+
profile; an existing install gets it with `chi update`). It caps the work one session does across its turns: tool
|
|
1696
|
+
calls and active minutes. The per-turn meters reset every turn (the step limit, check-in, loop-guard), and a
|
|
1697
|
+
Continue on the step-limit question starts the next 300 steps; a parent agent may answer that Continue itself
|
|
1698
|
+
(`chi answer`), so a child it keeps continuing has no overall bound but cost-cap's `$`, and on a local model none.
|
|
1699
|
+
work-cap is that bound, with cost-cap's shape: the harness watches, the model never sees any of it. It is inert
|
|
1700
|
+
without a cap, and there is none by default.
|
|
1701
|
+
|
|
1702
|
+
```yaml
|
|
1703
|
+
# config.yml
|
|
1704
|
+
bundles:
|
|
1705
|
+
work-cap:
|
|
1706
|
+
child: {tool_calls: 500, minutes: 90} # delegate children with no own value
|
|
1707
|
+
default: {} # every other session (and a delegate without child:); empty: no cap
|
|
1708
|
+
warn_at: 0.8 # tell the user (only) once at this share of a cap; 0/off: no warning
|
|
1709
|
+
ignore_tools: [task_wait, task_get, delegate, delegate_result] # not counted: waiting
|
|
1710
|
+
```
|
|
1711
|
+
|
|
1712
|
+
- **What counts** (`ctx.usage`): **tool calls**, the session's own across its turns and workers, less
|
|
1713
|
+
`ignore_tools` (the waiting tools by default, `delegate` among them since it blocks on its child, so a session
|
|
1714
|
+
waiting on a long task or a child isn't capped for waiting); and **active minutes**, work time: the generations' time plus the finished tool calls' time (less
|
|
1715
|
+
`ignore_tools`' time), never a wait (a question, its own included, an approval, the user's answer to
|
|
1716
|
+
`ask_user_question`), so idle time and a Continue answered hours later don't count.
|
|
1717
|
+
- **When it checks:** before each generation, like cost-cap: the overshoot is at most one generation's tool calls.
|
|
1718
|
+
When the step limit and a work cap fall in the same turn, the work cap asks first, and its Stop ends the turn.
|
|
1719
|
+
- **At `warn_at` of a cap** you get one notice per meter, `work-cap> 320 of 400 tool calls (80%)`; a raised cap
|
|
1720
|
+
warns again.
|
|
1721
|
+
- **At a cap** the turn pauses on a question (header "Work cap") that names the session's totals, so a parent
|
|
1722
|
+
answering it decides on the whole picture: `Work cap reached: 412 tool calls of 400 (6 turns, 47 min active,
|
|
1723
|
+
$1.20). Continue raises it to 800.` **Continue** raises the cap by another cap's worth (as many steps as it
|
|
1724
|
+
takes to pass the work, saved for the session), **Stop** stops the turn (`canceled` with `stopped_by: work-cap`)
|
|
1725
|
+
with a card showing the totals. Both meters at once: one question naming both, Continue raises each. With no one
|
|
1726
|
+
to ask (`--non-interactive`) it stops. Continue is tagged `:spend`, as cost-cap's: a parent agent may Continue
|
|
1727
|
+
under `guardrails.parent_answers: soft` or `standard`, only Stop under `hard` (see
|
|
1728
|
+
[Guardrails](guardrails.md#answers-from-a-parent-agent)).
|
|
1729
|
+
- **The caps of a session**, each meter on its own: the one `/workcap` or a Continue set, else its own value (`chi
|
|
1730
|
+
--set work-cap.tool_calls=400`, `work-cap.minutes=60`, `chi send --new --set …`; `off`, `0` or a negative number
|
|
1731
|
+
for none), else `child:`'s for a delegate child, else `default:`'s. Minutes in config.yml read `90`, `90m` or
|
|
1732
|
+
`1.5h`. A value that doesn't read (`tool_calls: many`, a `default:` that isn't a mapping) gets one notice,
|
|
1733
|
+
`work-cap: tool_calls: "many" isn't a number or off; using default.tool_calls: (400)`, and the next value is used,
|
|
1734
|
+
never a silent no cap.
|
|
1735
|
+
|
|
1736
|
+
**`/workcap`** (works while a turn runs; the next generation reads it):
|
|
1737
|
+
|
|
1738
|
+
- `/workcap`: `work-cap: 320 of 400 tool calls (6 turns, 47 min active, $1.20)`, or `work-cap: no cap (…)` with the
|
|
1739
|
+
totals.
|
|
1740
|
+
- `/workcap 400`: the tool-call cap (and what Continue adds) becomes 400 for this session; `/workcap 90m` or
|
|
1741
|
+
`/workcap 2h`: the active-minutes cap.
|
|
1742
|
+
- `/workcap off` (or `/workcap 0`): no work cap for this session, over the config.
|
|
1743
|
+
|
|
1744
|
+
The state is one file per session, `<data_dir>/sessions/<session id>.json` (`{"tool_calls": {"cap": 800, "step":
|
|
1745
|
+
400}}`, a meter's `"cap": null` for off).
|
|
1746
|
+
|
|
1683
1747
|
## The check-in bundle
|
|
1684
1748
|
|
|
1685
1749
|
`chi bundle install check-in` installs the bundle shipped with chi (`chi
|
|
@@ -1712,7 +1776,8 @@ what it has found. check-in counts the turn's tool calls, and when there are
|
|
|
1712
1776
|
- **`mode: notify`**: a notice line only.
|
|
1713
1777
|
|
|
1714
1778
|
The count is per turn: a new turn (a prompt, a continue, a reminder) starts
|
|
1715
|
-
from zero; steering merged into the running turn doesn't reset it
|
|
1779
|
+
from zero; steering merged into the running turn doesn't reset it (a cap
|
|
1780
|
+
across the session's turns is [the work-cap bundle](#the-work-cap-bundle)). The polling
|
|
1716
1781
|
tools are not counted. A nudge that arrives after the model's final answer is
|
|
1717
1782
|
dropped (logged), never restarting a turn that is done; the card's "Nudged the
|
|
1718
1783
|
model at N tool calls" then becomes "The answer came first; nudge not sent."
|
data/docs/releasing.md
CHANGED
|
@@ -13,7 +13,7 @@ can run the whole release; the user approves the notes before the tag and the
|
|
|
13
13
|
and `rake release:check` enforce it). The system bundle upgrades itself when
|
|
14
14
|
chi starts.
|
|
15
15
|
- **Every other shipped bundle** (btw, check-in, guardrails, known-names,
|
|
16
|
-
loop-guard, mcp, skills, slow-commands, agents-md, cost-cap, source-links, github-pr, coordinator, claude-peer, codex-peer, ruby, model-notes, and the profiles core and dev) has its own semver in its
|
|
16
|
+
loop-guard, mcp, skills, slow-commands, agents-md, cost-cap, work-cap, source-links, github-pr, coordinator, claude-peer, codex-peer, ruby, model-notes, and the profiles core and dev) has its own semver in its
|
|
17
17
|
`manifest.yml` and, when it needs a newer chi, a `requires_chi:` line. They never upgrade by themselves: users run
|
|
18
18
|
`chi update` (all of them, with the gem) or `chi bundle upgrade NAME`. So:
|
|
19
19
|
- a change to a bundle's files bumps that bundle's `version:`
|
data/docs/sub-agent.md
CHANGED
|
@@ -47,7 +47,9 @@ chi answer ID --question QID --dismiss [--timeout S] [--format json]
|
|
|
47
47
|
|
|
48
48
|
A child with a bundle setting of its own, such as a $ cap (the [cost-cap bundle](plugins.md#the-cost-cap-bundle)):
|
|
49
49
|
`chi send --new --wait --format json --set cost-cap.cap=3 -m "task"`. At the cap the wait exits 3 with the cost-cap
|
|
50
|
-
question; `chi answer ID --question QID --option Continue` (or `Stop`) answers it.
|
|
50
|
+
question; `chi answer ID --question QID --option Continue` (or `Stop`) answers it. A cap on the child's work across
|
|
51
|
+
its turns (the [work-cap bundle](plugins.md#the-work-cap-bundle)), the bound a parent that keeps answering the step
|
|
52
|
+
limit's Continue doesn't otherwise have: `--set work-cap.tool_calls=400` (or `work-cap.minutes=60`, active minutes).
|
|
51
53
|
|
|
52
54
|
`chi answer` waits after answering, the same way `chi send --wait` does, so the
|
|
53
55
|
next question comes back as exit 3 again. What it waits for is the answered
|
|
@@ -122,7 +124,10 @@ relayed to the parent's user the way approvals are.
|
|
|
122
124
|
|
|
123
125
|
A session at its cost cap (the [cost-cap bundle](plugins.md#the-cost-cap-bundle)) waits the same way, as a
|
|
124
126
|
question of kind `hook` with header "Cost cap" and options `Continue` and `Stop`: `--option Continue` raises the cap
|
|
125
|
-
by another cap's worth and the turn goes on, `--option Stop` ends it `canceled` with `stopped_by: cost-cap`.
|
|
127
|
+
by another cap's worth and the turn goes on, `--option Stop` ends it `canceled` with `stopped_by: cost-cap`. A session at its work cap (the
|
|
128
|
+
[work-cap bundle](plugins.md#the-work-cap-bundle)) asks the same, header "Work cap", the question naming the
|
|
129
|
+
session's totals (`Work cap reached: 412 tool calls of 400 (6 turns, 47 min active, $1.20). Continue raises it to
|
|
130
|
+
800.`); `stopped_by: work-cap` on Stop.
|
|
126
131
|
|
|
127
132
|
## Approvals
|
|
128
133
|
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: core
|
|
3
|
-
version: 0.
|
|
3
|
+
version: 0.4.0
|
|
4
4
|
scope: system
|
|
5
5
|
description: "Recommended safety set"
|
|
6
6
|
trust_level: reviewed
|
|
7
|
-
# Its members' highest floor (
|
|
8
|
-
requires_chi: ">= 0.
|
|
7
|
+
# Its members' highest floor (work-cap's: ctx.usage, raised at the release)
|
|
8
|
+
requires_chi: ">= 0.100.0"
|
|
9
9
|
includes:
|
|
10
10
|
- loop-guard
|
|
11
11
|
- check-in
|
|
@@ -13,3 +13,4 @@ includes:
|
|
|
13
13
|
- slow-commands
|
|
14
14
|
- agents-md
|
|
15
15
|
- cost-cap
|
|
16
|
+
- work-cap
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: work-cap
|
|
3
|
+
version: 0.1.0
|
|
4
|
+
scope: system
|
|
5
|
+
description: "A per-session work cap (tool calls, active minutes) across turns: at a cap the turn pauses on Continue (another cap's worth) / Stop with the session's totals, a warning at 80% goes to the user only; /workcap shows or sets it"
|
|
6
|
+
trust_level: reviewed
|
|
7
|
+
plugin:
|
|
8
|
+
file: plugin.rb
|
|
9
|
+
sha256: sha256:d4de4b835d1657223517f3d6a5d8a9b089b7c8fad08ed077f72748369323bd65
|
|
10
|
+
# ask_user's effects: are in chi 0.92.0; ctx.usage is new in the next chi
|
|
11
|
+
requires_chi: ">= 0.100.0"
|
|
12
|
+
# chi --set work-cap.tool_calls=400 / work-cap.minutes=60: the session's own caps, a number or off
|
|
13
|
+
session_settings: {tool_calls: number, minutes: number}
|
|
@@ -0,0 +1,391 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# The work-cap bundle (docs/plugins.md, The work-cap bundle): a per-session
|
|
4
|
+
# cap on the work a session does across its turns (ctx.usage: tool calls,
|
|
5
|
+
# active minutes), the session-wide bound the per-turn step limit and
|
|
6
|
+
# check-in don't give. Before each generation it compares the work with the
|
|
7
|
+
# caps: at warn_at of a cap the user (only) gets one notice, at a cap the
|
|
8
|
+
# turn pauses on a question, Continue (the cap goes up by a step) or Stop
|
|
9
|
+
# (the turn stops, with a card). With no one to ask it stops. The question
|
|
10
|
+
# names the session's totals, so a parent agent answering it decides on the
|
|
11
|
+
# whole picture. The model never sees any of it: no tool, no steer, no
|
|
12
|
+
# prompt text. cost-cap (the $ cap) is its sibling; the wordings follow it.
|
|
13
|
+
#
|
|
14
|
+
# Settings (config.yml, bundles: work-cap:), from ctx.settings, never at
|
|
15
|
+
# load: the caps are resolved once per session (see State below), so a
|
|
16
|
+
# config change reaches a running session through /workcap or a new
|
|
17
|
+
# session; warn_at and ignore_tools are read at each check.
|
|
18
|
+
# tool_calls: 400 this session's own tool-call cap (session_settings:,
|
|
19
|
+
# chi --set work-cap.tool_calls=400); 0/off: none
|
|
20
|
+
# minutes: 60 this session's own cap on active minutes, same rules
|
|
21
|
+
# child: {tool_calls: 500, minutes: 90} delegates (ctx.delegate?) with
|
|
22
|
+
# no own value
|
|
23
|
+
# default: {} every other session (and a delegate without child:)
|
|
24
|
+
# warn_at: 0.8 the share of a cap that warns; 0/off: no warning
|
|
25
|
+
# ignore_tools: [task_wait, task_get, delegate, delegate_result]
|
|
26
|
+
# tools whose calls and time aren't counted (waiting:
|
|
27
|
+
# delegate blocks on its child by default)
|
|
28
|
+
# Each meter resolves on its own: its own value, else child:'s for a
|
|
29
|
+
# delegate, else default:'s. A value that doesn't read (tool_calls: many)
|
|
30
|
+
# warns the user once and the next one is used, never a silent "no cap".
|
|
31
|
+
#
|
|
32
|
+
# What counts: ctx.usage's tool calls and active time (generations plus
|
|
33
|
+
# finished tool calls, never a question's wait), less ignore_tools'.
|
|
34
|
+
#
|
|
35
|
+
# State: <data_dir>/sessions/<session id>.json, {"tool_calls": {"cap": 800,
|
|
36
|
+
# "step": 400}, "minutes": {"cap": null}}, written by /workcap and by
|
|
37
|
+
# Continue; a meter's "cap": null is off for the session, over the config;
|
|
38
|
+
# a meter missing from the file comes from the settings. Resolved once per
|
|
39
|
+
# session id (ctx.delegate? parses the session file) and kept until
|
|
40
|
+
# /workcap or Continue writes a new one.
|
|
41
|
+
require "json"
|
|
42
|
+
require "fileutils"
|
|
43
|
+
|
|
44
|
+
class Plugin
|
|
45
|
+
USAGE = "usage: /workcap [N | Nm | Nh | off] (N tool calls, Nm or Nh active time; /workcap alone shows the totals)"
|
|
46
|
+
OPTIONS = %w[Continue Stop].freeze
|
|
47
|
+
# For a parent agent's answer (guardrails.parent_answers): Continue
|
|
48
|
+
# spends more, Stop denies; under hard a parent may only Stop.
|
|
49
|
+
EFFECTS = { "Continue" => :spend, "Stop" => :deny }.freeze
|
|
50
|
+
WARN_AT = 0.8
|
|
51
|
+
METERS = %w[tool_calls minutes].freeze
|
|
52
|
+
DEFAULT_IGNORE = %w[task_wait task_get delegate delegate_result].freeze
|
|
53
|
+
# A cap as text: 3, 2.5, .5, 1e1, -1 (a negative is none).
|
|
54
|
+
NUMBER = /\A-?(?:\d+(?:\.\d*)?|\.\d+)(?:e[-+]?\d+)?\z/i
|
|
55
|
+
# Minutes as text: a NUMBER, or one with m/min (minutes) or h (hours).
|
|
56
|
+
DURATION = /\A(?<number>.+?)\s*(?<unit>m|min|mins|minutes|h|hours?)?\z/i
|
|
57
|
+
|
|
58
|
+
def initialize(_settings)
|
|
59
|
+
@mutex = Mutex.new
|
|
60
|
+
@resolved = nil # {session:, meters: {meter => {cap:, step:, key:}}, unreadable: {name => value}}
|
|
61
|
+
@warned = {} # meter => [session id, cap] the share warning was given for
|
|
62
|
+
@unread_warned = nil # the session id the "isn't a number" notice was given for
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
def register(chi)
|
|
66
|
+
chi.on(:before_generation) { |_event, ctx| check(ctx) }
|
|
67
|
+
chi.command("/workcap", "see or set this session's work cap: /workcap, /workcap 400, /workcap 90m, /workcap off",
|
|
68
|
+
anytime: true) do |args, ctx|
|
|
69
|
+
command(args, ctx)
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
# A cap (a positive Float) or nil (none, 0, off, false, or a value that
|
|
74
|
+
# isn't a number: .readable? tells them apart). Minutes take a unit:
|
|
75
|
+
# 90m, 1.5h.
|
|
76
|
+
def self.amount(value, meter)
|
|
77
|
+
number = number(value, meter)
|
|
78
|
+
number&.positive? ? number : nil
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
# +value+ as a finite Float, else nil. "5." gets its 0 first
|
|
82
|
+
# (.float_text: Ruby 3.3's Float wants one).
|
|
83
|
+
def self.number(value, meter)
|
|
84
|
+
number = case value
|
|
85
|
+
when Numeric then value.to_f
|
|
86
|
+
when String then text_number(value.strip, meter)
|
|
87
|
+
end
|
|
88
|
+
number if number&.finite?
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
def self.text_number(text, meter)
|
|
92
|
+
return Float(float_text(text)) if text.match?(NUMBER)
|
|
93
|
+
return nil unless meter == "minutes" && (match = DURATION.match(text)) && match[:unit]
|
|
94
|
+
return nil unless match[:number].match?(NUMBER)
|
|
95
|
+
|
|
96
|
+
Float(float_text(match[:number])) * (match[:unit].downcase.start_with?("h") ? 60 : 1)
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# A NUMBER as Float reads it on every Ruby: a bare point ("5.", "5.e1")
|
|
100
|
+
# gets a 0 after it.
|
|
101
|
+
def self.float_text(text) = text.sub(/\.(?=e|\z)/i, ".0")
|
|
102
|
+
|
|
103
|
+
# Whether +value+ reads as a cap or as none: a number (text too), off,
|
|
104
|
+
# false or nil; not "many", true or a list.
|
|
105
|
+
def self.readable?(value, meter)
|
|
106
|
+
case value
|
|
107
|
+
when false, nil then true
|
|
108
|
+
when Numeric then value.finite?
|
|
109
|
+
when String then value.strip.casecmp?("off") || !number(value, meter).nil?
|
|
110
|
+
else false
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
# 400, 2.5: a whole number without its ".0".
|
|
115
|
+
def self.count(value)
|
|
116
|
+
rounded = value.round(1)
|
|
117
|
+
rounded == rounded.to_i ? rounded.to_i.to_s : rounded.to_s
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
private
|
|
121
|
+
|
|
122
|
+
def check(ctx)
|
|
123
|
+
resolved = resolve(ctx)
|
|
124
|
+
unreadable(ctx, resolved)
|
|
125
|
+
capped = resolved[:meters].select { |_meter, meter_cap| meter_cap[:cap] }
|
|
126
|
+
return if capped.empty?
|
|
127
|
+
|
|
128
|
+
usage = ctx.usage
|
|
129
|
+
used = used(ctx, usage)
|
|
130
|
+
reached = capped.select { |meter, meter_cap| used[meter] >= meter_cap[:cap] }
|
|
131
|
+
return at_cap(ctx, resolved, reached, usage, used) unless reached.empty?
|
|
132
|
+
|
|
133
|
+
share = warn_share(ctx)
|
|
134
|
+
return unless share
|
|
135
|
+
|
|
136
|
+
capped.each do |meter, meter_cap|
|
|
137
|
+
cap = meter_cap[:cap]
|
|
138
|
+
next unless used[meter] >= share * cap && first_warning?(meter, [resolved[:session], cap])
|
|
139
|
+
|
|
140
|
+
ctx.notify("#{count(used[meter])} of #{count(cap)} #{unit(meter)} (#{(used[meter] / cap * 100).floor}%)",
|
|
141
|
+
level: :warn)
|
|
142
|
+
end
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
# The counted work per meter: tool calls and active minutes, less the
|
|
146
|
+
# ignored tools' share.
|
|
147
|
+
def used(ctx, usage)
|
|
148
|
+
ignored = ignore_tools(ctx)
|
|
149
|
+
calls = usage.tool_calls - ignored.sum { |tool| usage.tool_calls_by_tool.fetch(tool, 0) }
|
|
150
|
+
active_ms = usage.active_ms - ignored.sum { |tool| usage.tool_ms_by_tool.fetch(tool, 0) }
|
|
151
|
+
{ "tool_calls" => [calls, 0].max.to_f, "minutes" => [active_ms, 0].max / 60_000.0 }
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# Once per session: the settings whose cap didn't read, and what is used.
|
|
155
|
+
def unreadable(ctx, resolved)
|
|
156
|
+
bad = resolved[:unreadable]
|
|
157
|
+
return if bad.empty? || !first_time?(:@unread_warned, resolved[:session])
|
|
158
|
+
|
|
159
|
+
named = bad.map { |key, value| "#{key}: #{value.inspect}" }.join(" and ")
|
|
160
|
+
used = resolved[:meters].values.filter_map do |meter_cap|
|
|
161
|
+
"#{meter_cap[:key]}: (#{count(meter_cap[:cap])})" if meter_cap[:cap]
|
|
162
|
+
end
|
|
163
|
+
using = used.empty? ? "no cap" : "using #{used.join(", ")}"
|
|
164
|
+
ctx.notify("work-cap: #{named} #{bad.size == 1 ? "isn't" : "aren't"} a number or off; #{using}", level: :warn)
|
|
165
|
+
end
|
|
166
|
+
|
|
167
|
+
def at_cap(ctx, resolved, reached, usage, used)
|
|
168
|
+
raised = reached.to_h do |meter, meter_cap|
|
|
169
|
+
[meter, raised_cap(meter_cap[:cap], meter_cap[:step] || meter_cap[:cap], used[meter])]
|
|
170
|
+
end
|
|
171
|
+
answer = ctx.ask_user(question: "Work cap reached: #{reached_text(reached, used)} (#{totals(ctx, usage, used, reached.keys)}). " \
|
|
172
|
+
"Continue raises #{raised_text(raised)}.",
|
|
173
|
+
header: "Work cap", options: OPTIONS, effects: EFFECTS)
|
|
174
|
+
if continue?(answer)
|
|
175
|
+
raised.each do |meter, cap|
|
|
176
|
+
save_meter(ctx, resolved[:session], meter, "cap" => cap, "step" => reached[meter][:step] || reached[meter][:cap])
|
|
177
|
+
end
|
|
178
|
+
ctx.notify("#{raised.size == 1 ? "cap" : "caps"} raised to " \
|
|
179
|
+
"#{raised.map { |meter, cap| "#{count(cap)} #{unit(meter)}" }.join(" and ")}")
|
|
180
|
+
else
|
|
181
|
+
stop(ctx, reached, usage, used)
|
|
182
|
+
end
|
|
183
|
+
end
|
|
184
|
+
|
|
185
|
+
# "412 tool calls of 400", and "61 active minutes of 60" for the other.
|
|
186
|
+
def reached_text(reached, used)
|
|
187
|
+
reached.map { |meter, meter_cap| "#{count(used[meter])} #{unit(meter)} of #{count(meter_cap[:cap])}" }.join(", ")
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
# "it to 800", or "them to 800 tool calls and 120 active minutes".
|
|
191
|
+
def raised_text(raised)
|
|
192
|
+
return "it to #{count(raised.values.first)}" if raised.size == 1
|
|
193
|
+
|
|
194
|
+
"them to #{raised.map { |meter, cap| "#{count(cap)} #{unit(meter)}" }.join(" and ")}"
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
# The session's totals a reader decides on, the reached meters left out
|
|
198
|
+
# (already named): turns, tool calls, active minutes and the spend.
|
|
199
|
+
def totals(ctx, usage, used, skip = [])
|
|
200
|
+
parts = ["#{usage.turns} #{usage.turns == 1 ? "turn" : "turns"}"]
|
|
201
|
+
parts << "#{count(used["tool_calls"])} tool calls" unless skip.include?("tool_calls")
|
|
202
|
+
parts << "#{active_text(used["minutes"])} active" unless skip.include?("minutes")
|
|
203
|
+
cost = ctx.cost
|
|
204
|
+
parts << format("$%.2f", cost.total) if cost.known? && cost.total.positive?
|
|
205
|
+
parts.join(", ")
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
# "47 min", or "<1 min" before a whole minute of work.
|
|
209
|
+
def active_text(minutes) = minutes < 1 ? "<1 min" : "#{count(minutes.floor)} min"
|
|
210
|
+
|
|
211
|
+
# Another step on the cap, as many steps as it takes to pass the work
|
|
212
|
+
# (a turn that overshot by more than a step asks once, not each step):
|
|
213
|
+
# Continue always leaves the cap past the work now.
|
|
214
|
+
def raised_cap(cap, step, used)
|
|
215
|
+
raised = cap + step
|
|
216
|
+
raised += step while raised <= used
|
|
217
|
+
raised.round(6)
|
|
218
|
+
end
|
|
219
|
+
|
|
220
|
+
def continue?(answer)
|
|
221
|
+
return false unless answer
|
|
222
|
+
|
|
223
|
+
Array(answer[:selected]).first == "Continue" || Array(answer[:selected_indices]).first&.zero?
|
|
224
|
+
end
|
|
225
|
+
|
|
226
|
+
def stop(ctx, reached, usage, used)
|
|
227
|
+
return unless ctx.stop_turn("work cap reached (#{reached_text(reached, used)})")
|
|
228
|
+
|
|
229
|
+
ctx.card(title: "work-cap stopped the turn",
|
|
230
|
+
body: "Used #{reached_text(reached, used)} (#{totals(ctx, usage, used, reached.keys)}).\n\n" \
|
|
231
|
+
"`/workcap N` raises the tool-call cap, `/workcap Nm` the active minutes, `/workcap off` removes both.",
|
|
232
|
+
level: :warn)
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
def command(args, ctx)
|
|
236
|
+
arg = args.to_s.strip.downcase
|
|
237
|
+
return show(ctx) if arg.empty?
|
|
238
|
+
|
|
239
|
+
session = ctx.session_id
|
|
240
|
+
return "work-cap: no session yet" unless session
|
|
241
|
+
|
|
242
|
+
if %w[off 0].include?(arg)
|
|
243
|
+
METERS.each { |meter| save_meter(ctx, session, meter, "cap" => nil) }
|
|
244
|
+
return "work-cap: no cap for this session"
|
|
245
|
+
end
|
|
246
|
+
meter = arg.match?(NUMBER) ? "tool_calls" : "minutes"
|
|
247
|
+
value = self.class.amount(arg, meter)
|
|
248
|
+
return USAGE unless value
|
|
249
|
+
|
|
250
|
+
save_meter(ctx, session, meter, "cap" => value, "step" => value)
|
|
251
|
+
"work-cap: #{unit(meter)} cap set to #{count(value)} (#{count(used(ctx, ctx.usage)[meter])} so far)"
|
|
252
|
+
rescue StandardError => e
|
|
253
|
+
ctx.log.warn(:workcap_command_failed, error: "#{e.class}: #{e.message}")
|
|
254
|
+
"work-cap: #{e.message}"
|
|
255
|
+
end
|
|
256
|
+
|
|
257
|
+
def show(ctx)
|
|
258
|
+
usage = ctx.usage
|
|
259
|
+
used = used(ctx, usage)
|
|
260
|
+
meters = resolve(ctx)[:meters]
|
|
261
|
+
return "work-cap: no cap (#{totals(ctx, usage, used)})" if meters.values.none? { |meter_cap| meter_cap[:cap] }
|
|
262
|
+
|
|
263
|
+
capped = meters.select { |_meter, meter_cap| meter_cap[:cap] }
|
|
264
|
+
caps = capped.map { |meter, meter_cap| "#{count(used[meter])} of #{count(meter_cap[:cap])} #{unit(meter)}" }
|
|
265
|
+
"work-cap: #{caps.join(", ")} (#{totals(ctx, usage, used, capped.keys)})"
|
|
266
|
+
end
|
|
267
|
+
|
|
268
|
+
def unit(meter) = meter == "tool_calls" ? "tool calls" : "active minutes"
|
|
269
|
+
|
|
270
|
+
# The session's caps now: per meter the state file's, else the session's
|
|
271
|
+
# own value (--set), else child:'s for a delegate, else default:'s.
|
|
272
|
+
# Cached per session id.
|
|
273
|
+
def resolve(ctx)
|
|
274
|
+
session = ctx.session_id
|
|
275
|
+
cached = @mutex.synchronize { @resolved }
|
|
276
|
+
return cached if cached && cached[:session] == session
|
|
277
|
+
|
|
278
|
+
saved = state(ctx, session)
|
|
279
|
+
configured = configured(ctx)
|
|
280
|
+
meters = METERS.to_h { |meter| [meter, saved[meter] || configured[:meters][meter]] }
|
|
281
|
+
resolved = { session: session, meters: meters, unreadable: configured[:unreadable] }
|
|
282
|
+
@mutex.synchronize { @resolved = resolved }
|
|
283
|
+
end
|
|
284
|
+
|
|
285
|
+
# The state file's meters ({meter => {cap:, step:}}), {} without one.
|
|
286
|
+
def state(ctx, session)
|
|
287
|
+
return {} unless session
|
|
288
|
+
|
|
289
|
+
saved = JSON.parse(File.read(state_path(ctx, session)))
|
|
290
|
+
return {} unless saved.is_a?(Hash)
|
|
291
|
+
|
|
292
|
+
METERS.each_with_object({}) do |meter, meters|
|
|
293
|
+
entry = saved[meter]
|
|
294
|
+
next unless entry.is_a?(Hash) && entry.key?("cap")
|
|
295
|
+
|
|
296
|
+
cap = self.class.amount(entry["cap"], meter)
|
|
297
|
+
meters[meter] = { cap: cap, step: self.class.amount(entry["step"], meter) || cap, key: "/workcap" }
|
|
298
|
+
end
|
|
299
|
+
rescue Errno::ENOENT, JSON::ParserError
|
|
300
|
+
{}
|
|
301
|
+
end
|
|
302
|
+
|
|
303
|
+
# Per meter the first of its own value, child:'s (a delegate's) and
|
|
304
|
+
# default:'s that is set and reads (.readable?); the ones set that don't
|
|
305
|
+
# read (and a child:/default: that isn't a mapping) are kept for the
|
|
306
|
+
# notice (#unreadable).
|
|
307
|
+
def configured(ctx)
|
|
308
|
+
settings = ctx.settings
|
|
309
|
+
unreadable = {}
|
|
310
|
+
groups = %w[child default].select { |group| settings.key?(group) }
|
|
311
|
+
groups.delete("child") if groups.include?("child") && !ctx.delegate?
|
|
312
|
+
groups = groups.select do |group|
|
|
313
|
+
next true if settings[group].is_a?(Hash)
|
|
314
|
+
|
|
315
|
+
unreadable[group] = settings[group] unless settings[group].nil?
|
|
316
|
+
false
|
|
317
|
+
end
|
|
318
|
+
meters = METERS.to_h do |meter|
|
|
319
|
+
sources = [[meter, settings, meter]] + groups.map { |group| ["#{group}.#{meter}", settings[group], meter] }
|
|
320
|
+
[meter, first_readable(sources, unreadable)]
|
|
321
|
+
end
|
|
322
|
+
{ meters: meters, unreadable: unreadable }
|
|
323
|
+
end
|
|
324
|
+
|
|
325
|
+
def first_readable(sources, unreadable)
|
|
326
|
+
sources.each do |name, values, meter|
|
|
327
|
+
next unless values.key?(meter)
|
|
328
|
+
|
|
329
|
+
value = values[meter]
|
|
330
|
+
unless self.class.readable?(value, meter)
|
|
331
|
+
unreadable[name] = value
|
|
332
|
+
next
|
|
333
|
+
end
|
|
334
|
+
cap = self.class.amount(value, meter)
|
|
335
|
+
return { cap: cap, step: cap, key: name }
|
|
336
|
+
end
|
|
337
|
+
{ cap: nil, step: nil, key: nil }
|
|
338
|
+
end
|
|
339
|
+
|
|
340
|
+
def ignore_tools(ctx)
|
|
341
|
+
settings = ctx.settings
|
|
342
|
+
settings.key?("ignore_tools") ? Array(settings["ignore_tools"]).map(&:to_s) : DEFAULT_IGNORE
|
|
343
|
+
end
|
|
344
|
+
|
|
345
|
+
def warn_share(ctx)
|
|
346
|
+
settings = ctx.settings
|
|
347
|
+
return WARN_AT unless settings.key?("warn_at")
|
|
348
|
+
|
|
349
|
+
share = settings["warn_at"]
|
|
350
|
+
share.is_a?(Numeric) && share.positive? ? share : nil
|
|
351
|
+
end
|
|
352
|
+
|
|
353
|
+
# Write one meter's cap into the session's state file (the other kept).
|
|
354
|
+
def save_meter(ctx, session, meter, data)
|
|
355
|
+
path = state_path(ctx, session)
|
|
356
|
+
FileUtils.mkdir_p(File.dirname(path))
|
|
357
|
+
saved = begin
|
|
358
|
+
JSON.parse(File.read(path))
|
|
359
|
+
rescue Errno::ENOENT, JSON::ParserError
|
|
360
|
+
{}
|
|
361
|
+
end
|
|
362
|
+
saved = {} unless saved.is_a?(Hash)
|
|
363
|
+
saved[meter] = data
|
|
364
|
+
Samagotchi::AtomicFile.write(path, "#{JSON.generate(saved)}\n")
|
|
365
|
+
# Resolved again at the next use, from the file now holding the meter.
|
|
366
|
+
@mutex.synchronize { @resolved = nil }
|
|
367
|
+
end
|
|
368
|
+
|
|
369
|
+
def state_path(ctx, session) = File.join(ctx.data_dir, "sessions", "#{session.gsub(/[^A-Za-z0-9_.-]/, "_")}.json")
|
|
370
|
+
|
|
371
|
+
def count(value) = self.class.count(value)
|
|
372
|
+
|
|
373
|
+
def first_warning?(meter, key)
|
|
374
|
+
@mutex.synchronize do
|
|
375
|
+
next false if @warned[meter] == key
|
|
376
|
+
|
|
377
|
+
@warned[meter] = key
|
|
378
|
+
true
|
|
379
|
+
end
|
|
380
|
+
end
|
|
381
|
+
|
|
382
|
+
# Whether +key+ is new for the once-only notice kept in +ivar+ (and keep it).
|
|
383
|
+
def first_time?(ivar, key)
|
|
384
|
+
@mutex.synchronize do
|
|
385
|
+
next false if instance_variable_get(ivar) == key
|
|
386
|
+
|
|
387
|
+
instance_variable_set(ivar, key)
|
|
388
|
+
true
|
|
389
|
+
end
|
|
390
|
+
end
|
|
391
|
+
end
|
data/lib/samagotchi/engine.rb
CHANGED
|
@@ -2680,7 +2680,8 @@ module Samagotchi
|
|
|
2680
2680
|
# returns [] itself: the || keeps a nil builder safe).
|
|
2681
2681
|
prompt_files: -> { @prompt_builder&.prompt_files || [] },
|
|
2682
2682
|
frontend: -> { interface },
|
|
2683
|
-
cost: -> { @metrics.cost }
|
|
2683
|
+
cost: -> { @metrics.cost },
|
|
2684
|
+
usage: -> { @metrics.usage }
|
|
2684
2685
|
)
|
|
2685
2686
|
end
|
|
2686
2687
|
|
|
@@ -52,15 +52,21 @@ module Samagotchi
|
|
|
52
52
|
|
|
53
53
|
# Load hooks from a config hash and return a Registry with registered plugins.
|
|
54
54
|
#
|
|
55
|
-
# @param config_hash [Hash, nil] the
|
|
55
|
+
# @param config_hash [Hash, nil] the parsed config.yml
|
|
56
56
|
# @param env [Hash] environment variables (default: ENV)
|
|
57
57
|
# @param failures [Guardrails::LoadFailures, nil] collects hooks that
|
|
58
58
|
# failed to load
|
|
59
59
|
# @return [Samagotchi::Hooks::Registry] the registry with all plugins registered
|
|
60
60
|
def load(config_hash, env: ENV, failures: nil)
|
|
61
|
-
|
|
61
|
+
hooks_config = config_hash["hooks"] if config_hash.is_a?(Hash)
|
|
62
|
+
# An empty `hooks:` key is nil in YAML: no hooks, like no key.
|
|
63
|
+
return Hooks::Registry.new if hooks_config.nil?
|
|
64
|
+
|
|
65
|
+
unless hooks_config.is_a?(Hash)
|
|
66
|
+
Log.warn(:hooks, "hooks_section_invalid", echo: "[samagotchi:hooks] config.yml hooks: must be a mapping of event name to hook entries; ignored")
|
|
67
|
+
return Hooks::Registry.new
|
|
68
|
+
end
|
|
62
69
|
|
|
63
|
-
hooks_config = config_hash["hooks"]
|
|
64
70
|
hooks_dir = hooks_dir(env, data: config_hash)
|
|
65
71
|
|
|
66
72
|
registry = Hooks::Registry.new
|
|
@@ -92,10 +98,17 @@ module Samagotchi
|
|
|
92
98
|
|
|
93
99
|
# Parse hook definitions from the config hooks section.
|
|
94
100
|
# Returns an array of { event_type:, path:, on_error: } hashes.
|
|
101
|
+
# An event key that holds something other than a list warns and is
|
|
102
|
+
# skipped; an empty one (nil) is skipped silently.
|
|
95
103
|
def self.parse_definitions(hooks_config)
|
|
96
104
|
definitions = []
|
|
97
105
|
hooks_config.each do |event_type, configs|
|
|
98
|
-
next
|
|
106
|
+
next if event_type.to_s == "hooks_dir" || configs.nil?
|
|
107
|
+
|
|
108
|
+
unless configs.is_a?(Array)
|
|
109
|
+
Log.warn(:hooks, "hooks_event_invalid", echo: "[samagotchi:hooks] config.yml hooks.#{event_type}: must be a list of hook entries; ignored", event: event_type.to_s)
|
|
110
|
+
next
|
|
111
|
+
end
|
|
99
112
|
|
|
100
113
|
configs.each do |cfg|
|
|
101
114
|
next unless cfg.is_a?(Hash) && cfg["path"]
|
|
@@ -40,7 +40,8 @@ module Samagotchi
|
|
|
40
40
|
"ctx.prompt_files" => "0.60.0", "chi.annotation" => "0.66.0", "chi.toolkit" => "0.66.0",
|
|
41
41
|
"ctx.use_toolkit" => "0.66.0", "ctx.toolkit" => "0.66.0", "ctx.declared_tools" => "0.66.0",
|
|
42
42
|
"chi.tool_source" => "0.67.0", "chi.peer_source" => "0.67.0", "ctx.http" => "0.80.0",
|
|
43
|
-
"ctx.cost" => "0.83.0", "ctx.sessions.children" => "0.34.0", "ctx.load_tools" => "0.99.0"
|
|
43
|
+
"ctx.cost" => "0.83.0", "ctx.sessions.children" => "0.34.0", "ctx.load_tools" => "0.99.0",
|
|
44
|
+
"ctx.usage" => "0.100.0"
|
|
44
45
|
}.freeze
|
|
45
46
|
|
|
46
47
|
# @param bundle [String]
|
|
@@ -4,6 +4,7 @@ require "fileutils"
|
|
|
4
4
|
require_relative "../log"
|
|
5
5
|
require_relative "../paths"
|
|
6
6
|
require_relative "../session_metrics/cost"
|
|
7
|
+
require_relative "../session_metrics/usage"
|
|
7
8
|
require_relative "../guardrails/context"
|
|
8
9
|
require_relative "../hook_question"
|
|
9
10
|
require_relative "../idle_client"
|
|
@@ -55,11 +56,12 @@ module Samagotchi
|
|
|
55
56
|
# ctx.http call without cancel: uses, nil for none: an init task's own,
|
|
56
57
|
# else the running turn's for work done for it (the turn's thread, or a
|
|
57
58
|
# plugin handler: +handler+) until the turn has ended.
|
|
58
|
-
# +cost+ returns the session's spend now (SessionMetrics::Cost)
|
|
59
|
+
# +cost+ returns the session's spend now (SessionMetrics::Cost),
|
|
60
|
+
# +usage+ its work now (SessionMetrics::Usage).
|
|
59
61
|
Host = Struct.new(:session_id, :cwd, :messages, :messages_partial, :notify, :ask_user, :cancelled, :card,
|
|
60
62
|
:ask_model, :model_name, :model_key, :state_dir, :scratch, :setup, :prompt_files,
|
|
61
63
|
:steer, :stop_turn, :stop_generation, :frontend, :toolkit, :declared_tools, :use_toolkit, :load_tools,
|
|
62
|
-
:cancel_controller, :cost, keyword_init: true)
|
|
64
|
+
:cancel_controller, :cost, :usage, keyword_init: true)
|
|
63
65
|
|
|
64
66
|
# ctx.ask_model failed: the model couldn't be reached, timed out, or
|
|
65
67
|
# sent nothing usable. The message says why, for the user.
|
|
@@ -167,6 +169,12 @@ module Samagotchi
|
|
|
167
169
|
# (docs/plugins.md); zero and known with no session
|
|
168
170
|
def cost = @host.cost&.call || SessionMetrics::Cost.zero
|
|
169
171
|
|
|
172
|
+
# @return [SessionMetrics::Usage] the session's work so far, read now:
|
|
173
|
+
# #turns, #tool_calls, #tool_calls_by_tool, #steps, #active_ms
|
|
174
|
+
# (#active_minutes) and #tool_ms_by_tool (docs/plugins.md); zero
|
|
175
|
+
# with no session
|
|
176
|
+
def usage = @host.usage&.call || SessionMetrics::Usage.zero
|
|
177
|
+
|
|
170
178
|
# @return [String, nil] the name of the session's toolkit ("full" for
|
|
171
179
|
# one from before toolkits; nil with no session)
|
|
172
180
|
def toolkit = @host.toolkit&.call
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Samagotchi
|
|
4
|
+
class SessionMetrics
|
|
5
|
+
# The session's work so far (SessionMetrics#usage, what a bundle's
|
|
6
|
+
# ctx.usage reads), across its turns and workers: the same numbers
|
|
7
|
+
# /stats and `chi sessions stats` show.
|
|
8
|
+
# +turns+: finished turns plus the running one. +tool_calls+ and
|
|
9
|
+
# +tool_calls_by_tool+ (canonical names: an alias counts under the tool
|
|
10
|
+
# it resolved to, a call_tool call under the inner tool): every call,
|
|
11
|
+
# running ones included. +steps+: the loop rounds that dispatched tools
|
|
12
|
+
# (the snapshot's iterations_total). +active_ms+: work time, the
|
|
13
|
+
# generations' time plus the finished tool calls' time (approval waits
|
|
14
|
+
# left out; ask_user_question's, a wait on the user, not counted),
|
|
15
|
+
# never a question's wait between them; +tool_ms_by_tool+ is the tool
|
|
16
|
+
# part by tool (the same names), so a reader can leave a waiting tool's
|
|
17
|
+
# time (task_wait, a blocking delegate) out.
|
|
18
|
+
# Bundles use its methods only, never this constant's name.
|
|
19
|
+
Usage = Data.define(:turns, :tool_calls, :tool_calls_by_tool, :steps, :active_ms, :tool_ms_by_tool) do
|
|
20
|
+
def self.zero = new(turns: 0, tool_calls: 0, tool_calls_by_tool: {}, steps: 0, active_ms: 0, tool_ms_by_tool: {})
|
|
21
|
+
|
|
22
|
+
# Data freezes itself, not its hashes: these are copied and frozen.
|
|
23
|
+
def initialize(tool_calls_by_tool:, tool_ms_by_tool:, **)
|
|
24
|
+
super(tool_calls_by_tool: tool_calls_by_tool.dup.freeze, tool_ms_by_tool: tool_ms_by_tool.dup.freeze, **)
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# @return [Float] active_ms in minutes
|
|
28
|
+
def active_minutes = active_ms / 60_000.0
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
end
|
|
@@ -14,6 +14,7 @@ require_relative "session"
|
|
|
14
14
|
require_relative "session_metrics/totals"
|
|
15
15
|
require_relative "session_metrics/model_segment"
|
|
16
16
|
require_relative "session_metrics/cost"
|
|
17
|
+
require_relative "session_metrics/usage"
|
|
17
18
|
require_relative "session_metrics/side_totals"
|
|
18
19
|
require_relative "side_usage"
|
|
19
20
|
|
|
@@ -53,6 +54,10 @@ module Samagotchi
|
|
|
53
54
|
SPEED_MIN_TOKENS = 8
|
|
54
55
|
SPEED_MIN_MS = 100
|
|
55
56
|
|
|
57
|
+
# The tools whose time is a wait on the user, not work (#usage).
|
|
58
|
+
USER_WAIT_TOOLS = %w[ask_user_question].freeze
|
|
59
|
+
private_constant :USER_WAIT_TOOLS
|
|
60
|
+
|
|
56
61
|
# The session's token totals as a saved analytics.json has them (zeros
|
|
57
62
|
# where an older file lacks a count), with how full the context was.
|
|
58
63
|
# A generation's decode speed (tokens per second) and where it came
|
|
@@ -377,6 +382,23 @@ module Samagotchi
|
|
|
377
382
|
@mutex.synchronize { Cost.of(@totals.with_running(@turn), @side) }
|
|
378
383
|
end
|
|
379
384
|
|
|
385
|
+
# The session's work now (ctx.usage): turns, tool calls (by tool), steps
|
|
386
|
+
# and active (work) time, earlier workers' loaded records and the
|
|
387
|
+
# running turn included. Safe from any thread.
|
|
388
|
+
# @return [Usage]
|
|
389
|
+
def usage
|
|
390
|
+
@mutex.synchronize do
|
|
391
|
+
turn = @turn
|
|
392
|
+
totals = @totals.with_running(turn)
|
|
393
|
+
calls, by_tool = tool_call_counts(turn)
|
|
394
|
+
tool_ms = @tool_tallies[:ms]
|
|
395
|
+
# Finished calls only: a running call's approval wait isn't known
|
|
396
|
+
# until it completes (waited_ms), so its time counts then.
|
|
397
|
+
Usage.new(turns: turn_count(turn), tool_calls: calls, tool_calls_by_tool: by_tool, steps: totals.iterations,
|
|
398
|
+
active_ms: (totals.gen_ms + tool_ms.values.sum).round, tool_ms_by_tool: tool_ms.transform_values(&:round))
|
|
399
|
+
end
|
|
400
|
+
end
|
|
401
|
+
|
|
380
402
|
# Event sink. Safe to call from any thread; errors are isolated by the
|
|
381
403
|
# caller (SessionObserver / TerminalUI) but we still guard internally.
|
|
382
404
|
# @param event [Hash]
|
|
@@ -542,10 +564,10 @@ module Samagotchi
|
|
|
542
564
|
def build_snapshot(records_mode)
|
|
543
565
|
turn = @turn
|
|
544
566
|
totals = @totals.with_running(turn)
|
|
545
|
-
|
|
567
|
+
calls, by_tool = tool_call_counts(turn)
|
|
546
568
|
{
|
|
547
569
|
session_id: @session_id,
|
|
548
|
-
turns:
|
|
570
|
+
turns: turn_count(turn),
|
|
549
571
|
cancellations: totals.cancellations,
|
|
550
572
|
tokens: totals.tokens_block,
|
|
551
573
|
# The tokens, cost and speeds per model, in order (ModelSegment.rows:
|
|
@@ -557,8 +579,8 @@ module Samagotchi
|
|
|
557
579
|
profile_source: @profile_source,
|
|
558
580
|
served_model: @served_model,
|
|
559
581
|
served_model_for: @served_model_for,
|
|
560
|
-
tool_calls_total:
|
|
561
|
-
tool_calls_by_tool:
|
|
582
|
+
tool_calls_total: calls,
|
|
583
|
+
tool_calls_by_tool: by_tool,
|
|
562
584
|
# The calls the model made under an alias's name (bash run as
|
|
563
585
|
# execute), by its own spelling; only when there are any.
|
|
564
586
|
**(@tool_tallies[:aliased].empty? ? {} : { tool_calls_aliased: @tool_tallies[:aliased].dup }),
|
|
@@ -582,6 +604,17 @@ module Samagotchi
|
|
|
582
604
|
}
|
|
583
605
|
end
|
|
584
606
|
|
|
607
|
+
# The finished turns and the running one. Caller holds the mutex.
|
|
608
|
+
def turn_count(turn) = @turn_records.size + (turn ? 1 : 0)
|
|
609
|
+
|
|
610
|
+
# Every tool call (loaded records, this process's, the running ones)
|
|
611
|
+
# and the calls by tool, a copy. Caller holds the mutex.
|
|
612
|
+
# @return [Array(Integer, Hash)]
|
|
613
|
+
def tool_call_counts(turn)
|
|
614
|
+
in_flight = turn ? turn.tool_calls_by_id.values : []
|
|
615
|
+
[@tool_records.size + in_flight.size, tally_into(@tool_tallies[:by_tool].dup, in_flight)]
|
|
616
|
+
end
|
|
617
|
+
|
|
585
618
|
# The model segments with the running turn in: the last one's when it
|
|
586
619
|
# runs on that model or names none yet, else a new one. Caller holds
|
|
587
620
|
# the mutex.
|
|
@@ -1199,7 +1232,8 @@ module Samagotchi
|
|
|
1199
1232
|
@totals = Totals.zero
|
|
1200
1233
|
@segments = []
|
|
1201
1234
|
@last_model_ref = nil
|
|
1202
|
-
|
|
1235
|
+
# ms: the finished calls' time by tool (#usage's active_ms).
|
|
1236
|
+
@tool_tallies = { by_tool: {}, aliased: {}, errors: 0, ms: {} }
|
|
1203
1237
|
end
|
|
1204
1238
|
|
|
1205
1239
|
def count_turn(record)
|
|
@@ -1219,6 +1253,15 @@ module Samagotchi
|
|
|
1219
1253
|
tally_into(@tool_tallies[:by_tool], [record])
|
|
1220
1254
|
called_as = record[:called_as].to_s
|
|
1221
1255
|
@tool_tallies[:aliased][called_as] = @tool_tallies[:aliased].fetch(called_as, 0) + 1 unless called_as.empty?
|
|
1256
|
+
count_tool_ms(record)
|
|
1257
|
+
end
|
|
1258
|
+
|
|
1259
|
+
def count_tool_ms(record)
|
|
1260
|
+
name = record[:tool].to_s
|
|
1261
|
+
ms = record[:duration_ms]
|
|
1262
|
+
return if name.empty? || !ms.is_a?(Numeric) || USER_WAIT_TOOLS.include?(name)
|
|
1263
|
+
|
|
1264
|
+
@tool_tallies[:ms][name] = @tool_tallies[:ms].fetch(name, 0) + ms
|
|
1222
1265
|
end
|
|
1223
1266
|
|
|
1224
1267
|
def tally_into(counts, tools)
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require_relative "../cancellation_controller"
|
|
4
4
|
require_relative "../plugin/context"
|
|
5
5
|
require_relative "../session_metrics/cost"
|
|
6
|
+
require_relative "../session_metrics/usage"
|
|
6
7
|
require_relative "recorder"
|
|
7
8
|
|
|
8
9
|
module Samagotchi
|
|
@@ -49,7 +50,8 @@ module Samagotchi
|
|
|
49
50
|
use_toolkit: ->(name) { recorder.use_toolkit(name, toolkits.call) },
|
|
50
51
|
load_tools: ->(names, bundle) { recorder.load_tools(names, bundle) },
|
|
51
52
|
cancel_controller: ->(handler: false) { recorder[:cancelled] ? stopped : nil },
|
|
52
|
-
cost: -> { cost(recorder[:cost]) }
|
|
53
|
+
cost: -> { cost(recorder[:cost]) },
|
|
54
|
+
usage: -> { usage(recorder[:usage]) }
|
|
53
55
|
)
|
|
54
56
|
end
|
|
55
57
|
|
|
@@ -75,6 +77,33 @@ module Samagotchi
|
|
|
75
77
|
else raise ArgumentError, "cost: a number, a Hash or a callable, not #{value.inspect}"
|
|
76
78
|
end
|
|
77
79
|
end
|
|
80
|
+
|
|
81
|
+
# The parts a test may give bundle.set(usage: {...}).
|
|
82
|
+
USAGE_PARTS = %i[turns tool_calls tool_calls_by_tool steps active_ms tool_ms_by_tool].freeze
|
|
83
|
+
|
|
84
|
+
# The SessionMetrics::Usage ctx.usage returns for the recorder's
|
|
85
|
+
# +usage+ value (Recorder::DEFAULTS): zeros for the parts not given;
|
|
86
|
+
# tool_calls defaults to the sum of tool_calls_by_tool, active_ms to
|
|
87
|
+
# the sum of tool_ms_by_tool.
|
|
88
|
+
# @return [SessionMetrics::Usage]
|
|
89
|
+
def usage(value)
|
|
90
|
+
value = value.call while value.respond_to?(:call)
|
|
91
|
+
case value
|
|
92
|
+
when nil then SessionMetrics::Usage.zero
|
|
93
|
+
when SessionMetrics::Usage then value
|
|
94
|
+
when Hash
|
|
95
|
+
parts = value.transform_keys(&:to_sym)
|
|
96
|
+
unknown = parts.keys - USAGE_PARTS
|
|
97
|
+
raise ArgumentError, "unknown usage parts: #{unknown.join(", ")}" unless unknown.empty?
|
|
98
|
+
|
|
99
|
+
by_tool = parts.fetch(:tool_calls_by_tool, {}).transform_keys(&:to_s)
|
|
100
|
+
ms_by_tool = parts.fetch(:tool_ms_by_tool, {}).transform_keys(&:to_s)
|
|
101
|
+
SessionMetrics::Usage.new(turns: parts.fetch(:turns, 0), tool_calls: parts.fetch(:tool_calls, by_tool.values.sum),
|
|
102
|
+
tool_calls_by_tool: by_tool, steps: parts.fetch(:steps, 0),
|
|
103
|
+
active_ms: parts.fetch(:active_ms, ms_by_tool.values.sum), tool_ms_by_tool: ms_by_tool)
|
|
104
|
+
else raise ArgumentError, "usage: a Hash or a callable, not #{value.inspect}"
|
|
105
|
+
end
|
|
106
|
+
end
|
|
78
107
|
end
|
|
79
108
|
|
|
80
109
|
# The hook runtime (Hooks::Runtime: event[:notify] and the other
|
|
@@ -37,7 +37,11 @@ module Samagotchi
|
|
|
37
37
|
prompt_files: [], cancelled: false, state_dir: nil, running: true, toolkit_staged: true, load_tools_staged: nil,
|
|
38
38
|
# ctx.cost: nil (zero), a Float (reported USD), a Hash of the parts
|
|
39
39
|
# (reported:, estimated:, side:, known:) or a callable giving one.
|
|
40
|
-
cost: nil
|
|
40
|
+
cost: nil,
|
|
41
|
+
# ctx.usage: nil (zero), a Hash of the parts (turns:, tool_calls:,
|
|
42
|
+
# tool_calls_by_tool:, steps:, active_ms:, tool_ms_by_tool:) or a
|
|
43
|
+
# callable giving one.
|
|
44
|
+
usage: nil
|
|
41
45
|
}.freeze
|
|
42
46
|
|
|
43
47
|
attr_reader :notices, :cards, :steers, :stops, :questions, :model_requests, :toolkit_uses, :tool_loads, :answers,
|
data/lib/samagotchi/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: samagotchi
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.100.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dmitry Dedov
|
|
@@ -266,6 +266,8 @@ files:
|
|
|
266
266
|
- lib/samagotchi/bundles/system/manifest.yml
|
|
267
267
|
- lib/samagotchi/bundles/system/memory_guide.md
|
|
268
268
|
- lib/samagotchi/bundles/system/self_map.md
|
|
269
|
+
- lib/samagotchi/bundles/work-cap/manifest.yml
|
|
270
|
+
- lib/samagotchi/bundles/work-cap/plugin.rb
|
|
269
271
|
- lib/samagotchi/cache_confirm.rb
|
|
270
272
|
- lib/samagotchi/cancellation_controller.rb
|
|
271
273
|
- lib/samagotchi/child_env.rb
|
|
@@ -523,6 +525,7 @@ files:
|
|
|
523
525
|
- lib/samagotchi/session_metrics/model_segment.rb
|
|
524
526
|
- lib/samagotchi/session_metrics/side_totals.rb
|
|
525
527
|
- lib/samagotchi/session_metrics/totals.rb
|
|
528
|
+
- lib/samagotchi/session_metrics/usage.rb
|
|
526
529
|
- lib/samagotchi/session_model_settings.rb
|
|
527
530
|
- lib/samagotchi/session_observer.rb
|
|
528
531
|
- lib/samagotchi/session_retention.rb
|