agent-bios 0.9.2 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/claude/agents/workhorse.md +2 -2
- package/claude/guides/claude-prompting.md +30 -12
- package/claude/guides/cli-multi-model-workflow.md +5 -5
- package/codex/guides/claude-prompting.md +30 -12
- package/codex/guides/cli-multi-model-workflow.md +5 -5
- package/config/agent-launch.toml +4 -4
- package/package.json +2 -2
- package/scripts/check-parity.sh +4 -4
- package/scripts/session-cost.py +39 -15
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: workhorse
|
|
3
3
|
description: WORKHORSE tier — bounded implementation, fixes, tests, and per-item judgment. Spawn decision-complete work with frozen scope/interfaces and a machine-checkable done-when; two or more independent items spawn in parallel.
|
|
4
|
-
model: claude-
|
|
5
|
-
effort:
|
|
4
|
+
model: claude-opus-5
|
|
5
|
+
effort: medium
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
Complete one bounded implementation, fix, or per-item judgment from a packet naming objective, frozen scope and inputs, allowed actions, output, done-when, and verification. Preserve out-of-scope behavior; batch independent reads; escalate missing decisions or authority instead of resolving them. Stage output for the main's acceptance — no external irreversible actions (push, install, credential, or remote mutation). Run the narrowest reliable changed-path check. Report: status, files_or_items_touched, evidence, verification or gap, risks_or_escalations.
|
|
@@ -17,7 +17,7 @@ core_rules:
|
|
|
17
17
|
- name the boundary explicitly — what to do without asking, and what to stop and ask about
|
|
18
18
|
targets:
|
|
19
19
|
- claude-fable-5
|
|
20
|
-
- claude-opus-
|
|
20
|
+
- claude-opus-5
|
|
21
21
|
- claude-sonnet-5
|
|
22
22
|
- claude-haiku-4-5
|
|
23
23
|
verification_focus:
|
|
@@ -55,15 +55,21 @@ pick the path.
|
|
|
55
55
|
## When to add blocks
|
|
56
56
|
|
|
57
57
|
- Long-horizon or autonomous work: give the full spec up front in one
|
|
58
|
-
well-specified turn and run at a high effort.
|
|
59
|
-
cadence
|
|
58
|
+
well-specified turn and run at a high effort. Do **not** add a self-check
|
|
59
|
+
cadence or a dedicated verifier subagent: the helm binding verifies its own
|
|
60
|
+
work unasked, so an instruction to verify buys over-verification instead.
|
|
61
|
+
Deleting inherited verification scaffolding costs no capability — this inverts
|
|
62
|
+
the usual self-check advice, so carve this tier out of a prompt library that
|
|
63
|
+
applies that advice uniformly.
|
|
60
64
|
- Review: state the evidence bar and the verdict shape. This tier follows
|
|
61
65
|
severity filters literally, so "only report high-severity" depresses measured
|
|
62
66
|
recall even as bug-finding improves — ask for every finding with confidence
|
|
63
67
|
and severity attached, and filter downstream.
|
|
64
|
-
- Delegation: say when to delegate
|
|
65
|
-
|
|
66
|
-
|
|
68
|
+
- Delegation: say when *not* to delegate, and cap the spawn count. The helm
|
|
69
|
+
binding reaches for subagents readily — the reverse of the binding it replaced
|
|
70
|
+
— and every spawn rebuilds context, reports back, and is then re-read, so
|
|
71
|
+
unbounded delegation multiplies cost and latency. File-based memory and custom
|
|
72
|
+
tools are the opposite case: they still need an explicit when-to-use trigger.
|
|
67
73
|
- Autonomous runs with no human watching: say so. Otherwise it asks permission
|
|
68
74
|
it does not need and blocks. Grant autonomy on minor choices (naming,
|
|
69
75
|
defaults, equivalent approaches) while keeping the ask for scope changes and
|
|
@@ -80,13 +86,17 @@ pick the path.
|
|
|
80
86
|
per subtask, and the orchestrator is not pinned to the slowest one.
|
|
81
87
|
- Effort ladder — `low`, `medium`, `high`, `xhigh`, `max` on the frontier, helm,
|
|
82
88
|
and workhorse bindings; `xhigh` is the best setting for most coding and
|
|
83
|
-
agentic work, `high` a sound default, `low` for cheap subagent scans.
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
89
|
+
agentic work, `high` a sound default, `low` for cheap subagent scans. Start
|
|
90
|
+
there and sweep *down* on a real eval set rather than reaching for the top: the
|
|
91
|
+
lower rungs are unusually strong on the helm binding, higher effort up front
|
|
92
|
+
often *reduces* turn count and total cost on agentic work, and some tasks land
|
|
93
|
+
equally well a level down. Effort does not control response length — see
|
|
94
|
+
Working rules.
|
|
87
95
|
- Per-model constraints differ across the `targets` bindings — thinking
|
|
88
96
|
configuration, sampling parameters, and effort support are not uniform, and
|
|
89
|
-
the sweep binding is the most restricted.
|
|
97
|
+
the sweep binding is the most restricted. On the helm binding, for instance,
|
|
98
|
+
thinking is on by default and turning it off is accepted only at `high` effort
|
|
99
|
+
or below. Confirm the constraint against the
|
|
90
100
|
live surface before relying on it in a dispatch; do not assume the frontier
|
|
91
101
|
binding's rules apply to the sweep one.
|
|
92
102
|
|
|
@@ -98,6 +108,13 @@ pick the path.
|
|
|
98
108
|
- Do not add "summarize every N tool calls" scaffolding — this tier narrates on
|
|
99
109
|
its own. If it narrates too much for a coding agent, set a silence default
|
|
100
110
|
instead: text only on a finding, a direction change, or a blocker.
|
|
111
|
+
- Length is a prompting lever, not an effort lever. This tier writes longer
|
|
112
|
+
answers and longer files than its predecessors, and lowering `effort` does not
|
|
113
|
+
reliably shorten visible output — only an explicit instruction does. Calibrate
|
|
114
|
+
the deliverable's length separately from the conversation's.
|
|
115
|
+
- Scope self-correction. Left alone this tier narrates its own earlier mistakes
|
|
116
|
+
at length, which reads as thrash. Ask it to correct only what would change the
|
|
117
|
+
reader's decisions, say it plainly, and carry on.
|
|
101
118
|
- Give it somewhere to write learnings, tell it to consult that place later, and
|
|
102
119
|
give the file a format. It performs notably better with a memory surface.
|
|
103
120
|
- Keep the deliverable readable: the final message is the reader's first look at
|
|
@@ -111,7 +128,8 @@ pick the path.
|
|
|
111
128
|
2. Name the boundaries — what to do freely, what to stop and ask about.
|
|
112
129
|
3. Give each tool a when-to-call description.
|
|
113
130
|
4. Say how progress claims must be grounded, and how the deliverable should read.
|
|
114
|
-
5. Delete the step-by-step scaffolding inherited from older models
|
|
131
|
+
5. Delete the step-by-step scaffolding inherited from older models — including
|
|
132
|
+
any verify-your-work step and any encouragement to delegate — then A/B the
|
|
115
133
|
prompt with it removed before keeping either version.
|
|
116
134
|
|
|
117
135
|
## Sources
|
|
@@ -156,13 +156,13 @@ Write for the next agent and re-verification, not narrative. Required content:
|
|
|
156
156
|
|
|
157
157
|
This is the human-readable projection of concrete models/tools; `config/agent-launch.toml` is the machine launch authority and parity checks keep them aligned. Re-probe when the binding is older than ~8 weeks or a newer observable model/tool changes the surface. `agent-bios install` overwrites deployed bindings, so edit the repo copy.
|
|
158
158
|
|
|
159
|
-
Binding (2026-07-
|
|
159
|
+
Binding (2026-07-25):
|
|
160
160
|
|
|
161
161
|
| Slot | Binding | Notes |
|
|
162
162
|
|---|---|---|
|
|
163
163
|
| FRONTIER | Claude Fable 5 · GPT-5.6 Sol (read-only; max default, task-fit effort including Ultra) | bounded hardest decisions and verdicts |
|
|
164
|
-
| HELM | Claude Opus
|
|
165
|
-
| WORKHORSE | Claude
|
|
164
|
+
| HELM | Claude Opus 5 (xhigh) · GPT-5.6 Sol (xhigh main; main Ultra requires explicit selection; bounded FRONTIER Ultra allowed) | standing main; Codex defaults bypass, explicit sandbox narrows |
|
|
165
|
+
| WORKHORSE | Claude Opus 5 (medium) · GPT-5.6 Terra (high) | implementation and per-item judgment |
|
|
166
166
|
| SWEEP | Claude Haiku 4.5 · GPT-5.6 Luna (low) | clear repeatable scans and mechanical work |
|
|
167
167
|
| VERIFIER-A | onto-mcp review | logic/authority kind; tier units per funnel |
|
|
168
168
|
| VERIFIER-B | `ultracode-for-codex` CLI v0.4.5 (Codex-backed; callable from Claude or Codex) | code/execution kind |
|
|
@@ -186,8 +186,8 @@ Dispatch packets:
|
|
|
186
186
|
| GPT-5.6 Sol FRONTIER | outcome, evidence, decision boundary, stop/verification; task-fit effort; read-only |
|
|
187
187
|
| GPT-5.6 Terra WORKHORSE | outcome, frozen scope/inputs, authority, done-when, evidence/report, escalation; high |
|
|
188
188
|
| GPT-5.6 Luna SWEEP | exact search space, rules, ambiguity behavior, stop, output; low; no architecture/debugging |
|
|
189
|
-
| Claude Opus
|
|
190
|
-
| Claude
|
|
189
|
+
| Claude Opus 5 HELM | xhigh for agentic work; high minimum for sensitive judgment; lower only when bounded/cost-led |
|
|
190
|
+
| Claude Opus 5 WORKHORSE | exact scope, apply-to-all rules, tools, verification, report; high default |
|
|
191
191
|
| Claude Haiku 4.5 SWEEP | closed, high-volume, latency-sensitive work with exact schema/examples |
|
|
192
192
|
|
|
193
193
|
Use only task-relevant tools; parallelize independent calls. Worker report: `status`, `files_or_items_touched`, `evidence`, `verification`, `risks_or_escalations`. Official basis: OpenAI [model](https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.6), [migration](https://developers.openai.com/api/docs/guides/upgrading-to-gpt-5p6-sol), [prompting](https://developers.openai.com/api/docs/guides/prompt-guidance-gpt-5p6), [Codex models](https://learn.chatgpt.com/docs/models); Anthropic [subagents](https://code.claude.com/docs/en/sub-agents), [model effort](https://code.claude.com/docs/en/model-config).
|
|
@@ -17,7 +17,7 @@ core_rules:
|
|
|
17
17
|
- name the boundary explicitly — what to do without asking, and what to stop and ask about
|
|
18
18
|
targets:
|
|
19
19
|
- claude-fable-5
|
|
20
|
-
- claude-opus-
|
|
20
|
+
- claude-opus-5
|
|
21
21
|
- claude-sonnet-5
|
|
22
22
|
- claude-haiku-4-5
|
|
23
23
|
verification_focus:
|
|
@@ -55,15 +55,21 @@ pick the path.
|
|
|
55
55
|
## When to add blocks
|
|
56
56
|
|
|
57
57
|
- Long-horizon or autonomous work: give the full spec up front in one
|
|
58
|
-
well-specified turn and run at a high effort.
|
|
59
|
-
cadence
|
|
58
|
+
well-specified turn and run at a high effort. Do **not** add a self-check
|
|
59
|
+
cadence or a dedicated verifier subagent: the helm binding verifies its own
|
|
60
|
+
work unasked, so an instruction to verify buys over-verification instead.
|
|
61
|
+
Deleting inherited verification scaffolding costs no capability — this inverts
|
|
62
|
+
the usual self-check advice, so carve this tier out of a prompt library that
|
|
63
|
+
applies that advice uniformly.
|
|
60
64
|
- Review: state the evidence bar and the verdict shape. This tier follows
|
|
61
65
|
severity filters literally, so "only report high-severity" depresses measured
|
|
62
66
|
recall even as bug-finding improves — ask for every finding with confidence
|
|
63
67
|
and severity attached, and filter downstream.
|
|
64
|
-
- Delegation: say when to delegate
|
|
65
|
-
|
|
66
|
-
|
|
68
|
+
- Delegation: say when *not* to delegate, and cap the spawn count. The helm
|
|
69
|
+
binding reaches for subagents readily — the reverse of the binding it replaced
|
|
70
|
+
— and every spawn rebuilds context, reports back, and is then re-read, so
|
|
71
|
+
unbounded delegation multiplies cost and latency. File-based memory and custom
|
|
72
|
+
tools are the opposite case: they still need an explicit when-to-use trigger.
|
|
67
73
|
- Autonomous runs with no human watching: say so. Otherwise it asks permission
|
|
68
74
|
it does not need and blocks. Grant autonomy on minor choices (naming,
|
|
69
75
|
defaults, equivalent approaches) while keeping the ask for scope changes and
|
|
@@ -80,13 +86,17 @@ pick the path.
|
|
|
80
86
|
per subtask, and the orchestrator is not pinned to the slowest one.
|
|
81
87
|
- Effort ladder — `low`, `medium`, `high`, `xhigh`, `max` on the frontier, helm,
|
|
82
88
|
and workhorse bindings; `xhigh` is the best setting for most coding and
|
|
83
|
-
agentic work, `high` a sound default, `low` for cheap subagent scans.
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
89
|
+
agentic work, `high` a sound default, `low` for cheap subagent scans. Start
|
|
90
|
+
there and sweep *down* on a real eval set rather than reaching for the top: the
|
|
91
|
+
lower rungs are unusually strong on the helm binding, higher effort up front
|
|
92
|
+
often *reduces* turn count and total cost on agentic work, and some tasks land
|
|
93
|
+
equally well a level down. Effort does not control response length — see
|
|
94
|
+
Working rules.
|
|
87
95
|
- Per-model constraints differ across the `targets` bindings — thinking
|
|
88
96
|
configuration, sampling parameters, and effort support are not uniform, and
|
|
89
|
-
the sweep binding is the most restricted.
|
|
97
|
+
the sweep binding is the most restricted. On the helm binding, for instance,
|
|
98
|
+
thinking is on by default and turning it off is accepted only at `high` effort
|
|
99
|
+
or below. Confirm the constraint against the
|
|
90
100
|
live surface before relying on it in a dispatch; do not assume the frontier
|
|
91
101
|
binding's rules apply to the sweep one.
|
|
92
102
|
|
|
@@ -98,6 +108,13 @@ pick the path.
|
|
|
98
108
|
- Do not add "summarize every N tool calls" scaffolding — this tier narrates on
|
|
99
109
|
its own. If it narrates too much for a coding agent, set a silence default
|
|
100
110
|
instead: text only on a finding, a direction change, or a blocker.
|
|
111
|
+
- Length is a prompting lever, not an effort lever. This tier writes longer
|
|
112
|
+
answers and longer files than its predecessors, and lowering `effort` does not
|
|
113
|
+
reliably shorten visible output — only an explicit instruction does. Calibrate
|
|
114
|
+
the deliverable's length separately from the conversation's.
|
|
115
|
+
- Scope self-correction. Left alone this tier narrates its own earlier mistakes
|
|
116
|
+
at length, which reads as thrash. Ask it to correct only what would change the
|
|
117
|
+
reader's decisions, say it plainly, and carry on.
|
|
101
118
|
- Give it somewhere to write learnings, tell it to consult that place later, and
|
|
102
119
|
give the file a format. It performs notably better with a memory surface.
|
|
103
120
|
- Keep the deliverable readable: the final message is the reader's first look at
|
|
@@ -111,7 +128,8 @@ pick the path.
|
|
|
111
128
|
2. Name the boundaries — what to do freely, what to stop and ask about.
|
|
112
129
|
3. Give each tool a when-to-call description.
|
|
113
130
|
4. Say how progress claims must be grounded, and how the deliverable should read.
|
|
114
|
-
5. Delete the step-by-step scaffolding inherited from older models
|
|
131
|
+
5. Delete the step-by-step scaffolding inherited from older models — including
|
|
132
|
+
any verify-your-work step and any encouragement to delegate — then A/B the
|
|
115
133
|
prompt with it removed before keeping either version.
|
|
116
134
|
|
|
117
135
|
## Sources
|
|
@@ -156,13 +156,13 @@ Write for the next agent and re-verification, not narrative. Required content:
|
|
|
156
156
|
|
|
157
157
|
This is the human-readable projection of concrete models/tools; `config/agent-launch.toml` is the machine launch authority and parity checks keep them aligned. Re-probe when the binding is older than ~8 weeks or a newer observable model/tool changes the surface. `agent-bios install` overwrites deployed bindings, so edit the repo copy.
|
|
158
158
|
|
|
159
|
-
Binding (2026-07-
|
|
159
|
+
Binding (2026-07-25):
|
|
160
160
|
|
|
161
161
|
| Slot | Binding | Notes |
|
|
162
162
|
|---|---|---|
|
|
163
163
|
| FRONTIER | Claude Fable 5 · GPT-5.6 Sol (read-only; max default, task-fit effort including Ultra) | bounded hardest decisions and verdicts |
|
|
164
|
-
| HELM | Claude Opus
|
|
165
|
-
| WORKHORSE | Claude
|
|
164
|
+
| HELM | Claude Opus 5 (xhigh) · GPT-5.6 Sol (xhigh main; main Ultra requires explicit selection; bounded FRONTIER Ultra allowed) | standing main; Codex defaults bypass, explicit sandbox narrows |
|
|
165
|
+
| WORKHORSE | Claude Opus 5 (medium) · GPT-5.6 Terra (high) | implementation and per-item judgment |
|
|
166
166
|
| SWEEP | Claude Haiku 4.5 · GPT-5.6 Luna (low) | clear repeatable scans and mechanical work |
|
|
167
167
|
| VERIFIER-A | onto-mcp review | logic/authority kind; tier units per funnel |
|
|
168
168
|
| VERIFIER-B | `ultracode-for-codex` CLI v0.4.5 (Codex-backed; callable from Claude or Codex) | code/execution kind |
|
|
@@ -186,8 +186,8 @@ Dispatch packets:
|
|
|
186
186
|
| GPT-5.6 Sol FRONTIER | outcome, evidence, decision boundary, stop/verification; task-fit effort; read-only |
|
|
187
187
|
| GPT-5.6 Terra WORKHORSE | outcome, frozen scope/inputs, authority, done-when, evidence/report, escalation; high |
|
|
188
188
|
| GPT-5.6 Luna SWEEP | exact search space, rules, ambiguity behavior, stop, output; low; no architecture/debugging |
|
|
189
|
-
| Claude Opus
|
|
190
|
-
| Claude
|
|
189
|
+
| Claude Opus 5 HELM | xhigh for agentic work; high minimum for sensitive judgment; lower only when bounded/cost-led |
|
|
190
|
+
| Claude Opus 5 WORKHORSE | exact scope, apply-to-all rules, tools, verification, report; high default |
|
|
191
191
|
| Claude Haiku 4.5 SWEEP | closed, high-volume, latency-sensitive work with exact schema/examples |
|
|
192
192
|
|
|
193
193
|
Use only task-relevant tools; parallelize independent calls. Worker report: `status`, `files_or_items_touched`, `evidence`, `verification`, `risks_or_escalations`. Official basis: OpenAI [model](https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.6), [migration](https://developers.openai.com/api/docs/guides/upgrading-to-gpt-5p6-sol), [prompting](https://developers.openai.com/api/docs/guides/prompt-guidance-gpt-5p6), [Codex models](https://learn.chatgpt.com/docs/models); Anthropic [subagents](https://code.claude.com/docs/en/sub-agents), [model effort](https://code.claude.com/docs/en/model-config).
|
package/config/agent-launch.toml
CHANGED
|
@@ -48,7 +48,7 @@ workhorse = "${CODEX_HOME}/agents/workhorse.toml"
|
|
|
48
48
|
sweep = "${CODEX_HOME}/agents/sweep.toml"
|
|
49
49
|
|
|
50
50
|
[hosts.claude]
|
|
51
|
-
models = ["claude-fable-5", "claude-opus-
|
|
51
|
+
models = ["claude-fable-5", "claude-opus-5", "claude-sonnet-5", "claude-haiku-4-5"]
|
|
52
52
|
# onto review seat used when a CROSS-family main (Codex) routes onto review to
|
|
53
53
|
# this (anthropic/claude) family; must be an onto review-role registered (provider, model).
|
|
54
54
|
onto_review = { provider = "anthropic", model = "claude-fable-5" }
|
|
@@ -58,12 +58,12 @@ model = "claude-fable-5"
|
|
|
58
58
|
effort = "max"
|
|
59
59
|
|
|
60
60
|
[hosts.claude.tiers.helm]
|
|
61
|
-
model = "claude-opus-
|
|
61
|
+
model = "claude-opus-5"
|
|
62
62
|
effort = "xhigh"
|
|
63
63
|
|
|
64
64
|
[hosts.claude.tiers.workhorse]
|
|
65
|
-
model = "claude-
|
|
66
|
-
effort = "
|
|
65
|
+
model = "claude-opus-5"
|
|
66
|
+
effort = "medium"
|
|
67
67
|
|
|
68
68
|
[hosts.claude.tiers.sweep]
|
|
69
69
|
model = "claude-haiku-4-5"
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-bios",
|
|
3
|
-
"version": "0.9.
|
|
4
|
-
"releaseDate": "2026-07-
|
|
3
|
+
"version": "0.9.4",
|
|
4
|
+
"releaseDate": "2026-07-25",
|
|
5
5
|
"description": "A thin, low-level instruction layer for LLM CLI agents: one set of principles and behavior whichever model you run. Deploys into $HOME by copy via an explicit `agent-bios install`.",
|
|
6
6
|
"bin": {
|
|
7
7
|
"agent-bios": "scripts/install.sh"
|
package/scripts/check-parity.sh
CHANGED
|
@@ -273,8 +273,8 @@ for tier, (model, effort) in expected_launch_tiers.items():
|
|
|
273
273
|
|
|
274
274
|
expected_claude_tiers = {
|
|
275
275
|
"frontier": ("claude-fable-5", "max"),
|
|
276
|
-
"helm": ("claude-opus-
|
|
277
|
-
"workhorse": ("claude-
|
|
276
|
+
"helm": ("claude-opus-5", "xhigh"),
|
|
277
|
+
"workhorse": ("claude-opus-5", "medium"),
|
|
278
278
|
"sweep": ("claude-haiku-4-5", "low"),
|
|
279
279
|
}
|
|
280
280
|
claude_launch_tiers = launch_profile.get("hosts", {}).get("claude", {}).get("tiers", {})
|
|
@@ -409,8 +409,8 @@ for guide_name in [
|
|
|
409
409
|
mark_fail(f"{guide} SWEEP must bind exact GPT-5.6 Luna")
|
|
410
410
|
for slot, claude_model in (
|
|
411
411
|
("FRONTIER", "Claude Fable 5"),
|
|
412
|
-
("HELM", "Claude Opus
|
|
413
|
-
("WORKHORSE", "Claude
|
|
412
|
+
("HELM", "Claude Opus 5"),
|
|
413
|
+
("WORKHORSE", "Claude Opus 5"),
|
|
414
414
|
("SWEEP", "Claude Haiku 4.5"),
|
|
415
415
|
):
|
|
416
416
|
if rows[slot] and claude_model not in rows[slot][1]:
|
package/scripts/session-cost.py
CHANGED
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
"""Aggregate token usage & cost for a Claude Code session (main + subagents).
|
|
3
3
|
|
|
4
4
|
Usage: session-cost.py <session>.jsonl [...]
|
|
5
|
-
Reads the session transcript
|
|
5
|
+
Reads the session transcript, splitting main-loop from subagent (sidechain)
|
|
6
|
+
usage, and also picks up <session-dir>/subagents/agent-*.jsonl when present.
|
|
6
7
|
Prints per-source, per-model token sums, modeled cost, and wall-clock span.
|
|
7
8
|
"""
|
|
8
9
|
import json, sys, glob, os
|
|
@@ -12,6 +13,7 @@ from datetime import datetime
|
|
|
12
13
|
PRICES = {
|
|
13
14
|
"claude-fable-5": (10.0, 50.0, 1.00, 12.50, 20.0),
|
|
14
15
|
"claude-mythos-5": (10.0, 50.0, 1.00, 12.50, 20.0),
|
|
16
|
+
"claude-opus-5": (5.0, 25.0, 0.50, 6.25, 10.0),
|
|
15
17
|
"claude-opus-4-8": (5.0, 25.0, 0.50, 6.25, 10.0),
|
|
16
18
|
"claude-opus-4-7": (5.0, 25.0, 0.50, 6.25, 10.0),
|
|
17
19
|
"claude-opus-4-6": (5.0, 25.0, 0.50, 6.25, 10.0),
|
|
@@ -27,9 +29,17 @@ def price_for(model):
|
|
|
27
29
|
return None
|
|
28
30
|
|
|
29
31
|
def scan(path):
|
|
30
|
-
"""-> {model: {in,out,cr,cw5,cw1,turns}}, (t_min, t_max)
|
|
31
|
-
|
|
32
|
-
|
|
32
|
+
"""-> {(scope, model): {in,out,cr,cw5,cw1,turns}}, (t_min, t_max), {scope: agent_ids}
|
|
33
|
+
|
|
34
|
+
A single API response is written to the transcript several times as it
|
|
35
|
+
streams, each line carrying output_tokens *so far* (e.g. 2, 2, 2, 540).
|
|
36
|
+
Keep the record with the LARGEST output_tokens per message id: keeping the
|
|
37
|
+
first one instead under-reports subagent output by ~95%, because sidechain
|
|
38
|
+
messages get snapshotted far more often than main-loop ones do.
|
|
39
|
+
"""
|
|
40
|
+
best, tmin, tmax = {}, None, None
|
|
41
|
+
agents = {"main": set(), "sub": set()}
|
|
42
|
+
for line in open(path, errors="replace"):
|
|
33
43
|
try:
|
|
34
44
|
d = json.loads(line)
|
|
35
45
|
except json.JSONDecodeError:
|
|
@@ -41,11 +51,17 @@ def scan(path):
|
|
|
41
51
|
u, model = m.get("usage"), m.get("model")
|
|
42
52
|
if not u or not model or model == "<synthetic>":
|
|
43
53
|
continue
|
|
44
|
-
|
|
45
|
-
if
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
54
|
+
scope = "sub" if d.get("isSidechain") else "main"
|
|
55
|
+
if d.get("agentId"):
|
|
56
|
+
agents[scope].add(d["agentId"])
|
|
57
|
+
key = (scope, m.get("id") or d.get("requestId"))
|
|
58
|
+
prev = best.get(key)
|
|
59
|
+
if prev is None or u.get("output_tokens", 0) > prev[1].get("output_tokens", 0):
|
|
60
|
+
best[key] = (model, u)
|
|
61
|
+
|
|
62
|
+
agg = {}
|
|
63
|
+
for (scope, _), (model, u) in best.items():
|
|
64
|
+
a = agg.setdefault((scope, model), dict(inp=0, out=0, cr=0, cw5=0, cw1=0, turns=0))
|
|
49
65
|
a["inp"] += u.get("input_tokens", 0)
|
|
50
66
|
a["out"] += u.get("output_tokens", 0)
|
|
51
67
|
a["cr"] += u.get("cache_read_input_tokens", 0)
|
|
@@ -56,7 +72,7 @@ def scan(path):
|
|
|
56
72
|
else:
|
|
57
73
|
a["cw5"] += u.get("cache_creation_input_tokens", 0)
|
|
58
74
|
a["turns"] += 1
|
|
59
|
-
return agg, (tmin, tmax)
|
|
75
|
+
return agg, (tmin, tmax), agents
|
|
60
76
|
|
|
61
77
|
def cost(model, a):
|
|
62
78
|
p = price_for(model)
|
|
@@ -70,19 +86,27 @@ def fmt(n):
|
|
|
70
86
|
|
|
71
87
|
def report(session_path):
|
|
72
88
|
base = session_path[:-6] # strip .jsonl
|
|
73
|
-
sources = [(
|
|
89
|
+
sources = [(None, session_path)]
|
|
74
90
|
sources += [(os.path.basename(f)[:-6], f)
|
|
75
91
|
for f in sorted(glob.glob(os.path.join(base, "subagents", "*.jsonl")))]
|
|
76
92
|
print(f"\n=== {os.path.basename(session_path)} ===")
|
|
77
93
|
grand, unpriced = 0.0, []
|
|
78
|
-
hdr = f"{'source':<38}{'model':<22}{'turns':>6}{'input':>9}{'output':>9}
|
|
94
|
+
hdr = (f"{'source':<38}{'model':<22}{'turns':>6}{'input':>9}{'output':>9}"
|
|
95
|
+
f"{'cache_rd':>10}{'cache_wr':>10}{'cost$':>9}")
|
|
79
96
|
print(hdr); print("-" * len(hdr))
|
|
80
97
|
span_min = span_max = None
|
|
81
|
-
for
|
|
82
|
-
agg, (tmin, tmax) = scan(path)
|
|
98
|
+
for label, path in sources:
|
|
99
|
+
agg, (tmin, tmax), agents = scan(path)
|
|
83
100
|
if tmin:
|
|
84
101
|
span_min = min(span_min or tmin, tmin); span_max = max(span_max or tmax, tmax)
|
|
85
|
-
for model, a in agg.items():
|
|
102
|
+
for (scope, model), a in sorted(agg.items()):
|
|
103
|
+
if label is not None:
|
|
104
|
+
name = label
|
|
105
|
+
elif scope == "sub":
|
|
106
|
+
n = len(agents["sub"])
|
|
107
|
+
name = f"subagents (n={n})" if n else "subagents"
|
|
108
|
+
else:
|
|
109
|
+
name = "main"
|
|
86
110
|
c = cost(model, a)
|
|
87
111
|
cs = f"{c:9.2f}" if c is not None else " ?"
|
|
88
112
|
if c is None:
|