sumo-agents 0.1.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +3 -2
- package/README.md +102 -611
- package/bin/sumo.mjs +3 -1
- package/guides/delegation.md +8 -7
- package/package.json +3 -2
- package/prompts/agent.md +3 -3
- package/prompts/dream.md +1 -0
- package/prompts/scribe.md +1 -0
- package/src/apply.mjs +79 -19
- package/src/card.mjs +11 -6
- package/src/chat.mjs +318 -54
- package/src/cli.mjs +120 -31
- package/src/cost.mjs +128 -0
- package/src/db.mjs +40 -1
- package/src/editor.mjs +110 -9
- package/src/export.mjs +1 -1
- package/src/herdr.mjs +5 -2
- package/src/hooks.mjs +13 -3
- package/src/jobs.mjs +606 -59
- package/src/jobtools.mjs +10 -7
- package/src/loop.mjs +358 -53
- package/src/mcp.mjs +44 -14
- package/src/memory.mjs +39 -4
- package/src/model.mjs +6 -5
- package/src/page.mjs +3 -1
- package/src/paths.mjs +9 -2
- package/src/prime.mjs +14 -5
- package/src/projects.mjs +18 -0
- package/src/render.mjs +1 -0
- package/src/route.mjs +110 -12
- package/src/sandbox.mjs +173 -0
- package/src/scribe.mjs +7 -5
- package/src/sessions.mjs +12 -8
- package/src/setup.mjs +76 -7
- package/src/sqlite.mjs +66 -0
- package/src/text.mjs +8 -3
- package/src/tools.mjs +562 -31
- package/src/tty.mjs +293 -43
- package/src/ui.mjs +507 -98
- package/src/verify.mjs +51 -8
- package/src/worktree.mjs +128 -0
package/README.md
CHANGED
|
@@ -1,626 +1,117 @@
|
|
|
1
1
|
# sumo-agents
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
tell it
|
|
5
|
-
jobs that run in its own loop against the Anthropic API. No harness: Sumo owns its prompt, its tools,
|
|
6
|
-
its context and its cost.
|
|
3
|
+
A coding agent you talk to from one place. It works across every project on your machine, remembers what
|
|
4
|
+
you tell it, and hands big work to background jobs that are checked before they can say they are done.
|
|
7
5
|
|
|
8
|
-
|
|
9
|
-
|
|
6
|
+
- **One chat for everything.** Mention a project by name; Sumo finds it on disk and works there.
|
|
7
|
+
- **Memory that lasts.** Your preferences, project rules and where you left off are kept locally and
|
|
8
|
+
loaded into every new session. You never explain the same thing twice.
|
|
9
|
+
- **Background jobs.** Big, parallel or reading-heavy work goes to scout, worker and reviewer jobs. A
|
|
10
|
+
local model picks the model and effort for each one, so the dearest model is never the default.
|
|
11
|
+
- **Checked results.** A worker cannot report done until the project's own tests and linters agree.
|
|
12
|
+
- **Its own loop.** Sumo talks to the Anthropic API directly and owns its prompt, tools, context and cost.
|
|
10
13
|
|
|
11
|
-
|
|
14
|
+
Everything it learns lives in `~/.sumo-agents/` on each machine, never in a repository, so a work laptop
|
|
15
|
+
and a personal one learn separately.
|
|
12
16
|
|
|
13
|
-
|
|
14
|
-
routes jobs and writes memory, and, for the chat and jobs, either `ANTHROPIC_API_KEY` or a `CLAUDE_CODE_OAUTH_TOKEN`
|
|
15
|
-
from `claude setup-token` in the environment.
|
|
17
|
+
## Install
|
|
16
18
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
sumo doctor # every line should say ok
|
|
21
|
-
sumo chat # talk
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
`npm update -g sumo-agents` moves to the newest release. To work on Sumo itself, clone the repo instead:
|
|
25
|
-
`git clone https://github.com/sumododda/sumo-agents.git && cd sumo-agents && npm ci && node bin/sumo.mjs setup`.
|
|
26
|
-
|
|
27
|
-
Setup asks one question, once: where to download the router model from. Enter keeps the default,
|
|
28
|
-
Hugging Face. Behind a corporate proxy, give the base URL of an Artifactory Hugging Face remote
|
|
29
|
-
instead (`https://<host>/artifactory/api/huggingfaceml/<repo-key>`); the same
|
|
30
|
-
`<repo>/resolve/main/<file>` path is fetched under it. The 2.5 GB file lands in
|
|
31
|
-
`~/.sumo-agents/models/` and is skipped when it is already there.
|
|
32
|
-
|
|
33
|
-
Sumo enables `NODE_USE_SYSTEM_CA=1` automatically, including for hooks and background processes.
|
|
34
|
-
It trusts the OS certificate store alongside Node's bundled CAs, so enterprise CAs installed and
|
|
35
|
-
trusted by IT work without a shell export. If IT supplies a separate PEM bundle, set
|
|
36
|
-
`NODE_EXTRA_CA_CERTS` to its path before starting Sumo; that bundle is preserved. Restart existing
|
|
37
|
-
Sumo processes after changing certificate trust.
|
|
38
|
-
|
|
39
|
-
```sh
|
|
40
|
-
sumo setup --model-source https://<host>/artifactory/api/huggingfaceml/hf-remote # non-interactive
|
|
41
|
-
sumo setup --no-model # skip the download
|
|
42
|
-
sumo config model.source <url> # change it later
|
|
43
|
-
sumo config chat.model opus # what `sumo chat` runs on; `auto` routes each turn
|
|
44
|
-
sumo config chat.effort high # effort when using a fixed chat model
|
|
45
|
-
sumo config chat.notify off # no bell when a long turn ends behind another window
|
|
46
|
-
sumo config guides.auto off # no guide riding in unasked, no turn held for an unchecked edit
|
|
47
|
-
sumo config jobs.advisor on # a job may ask the next model up for advice, mid-request (off)
|
|
48
|
-
sumo config projects.home ~/code # where a new project is made (your home, until you say)
|
|
49
|
-
sumo config scratch.days 30 # how long scratch keeps a chore nothing is written in (14)
|
|
50
|
-
sumo models # the API models: on or off, and why
|
|
51
|
-
sumo models discover # ask the API again which of them this credential can use
|
|
52
|
-
sumo models disable fable # keep one out of the chat, the router and the passes
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
Then just talk. Mention a project the way you normally would; the first time, the agent finds it on
|
|
56
|
-
disk, confirms the path with you, and registers it — and a name of yours it had to look up becomes an alias,
|
|
57
|
-
so it is looked up once. Work that belongs to no project is done in `~/.sumo-agents/scratch/`, a folder per
|
|
58
|
-
chore: with no project named the tools start there, wherever you started the chat; a job is handed it as the
|
|
59
|
-
project `scratch`; nothing is remembered or checked there, and a chore nothing is written in for 14 days is
|
|
60
|
-
deleted — move one out and register it to keep it. In the chat: `/fix`, `/feature`, `/review`
|
|
61
|
-
load the matching guide; `/dream` tidies the memory and says what changed; `/model` says the route
|
|
62
|
-
and sets it — `/model opus high`, `/model haiku`,
|
|
63
|
-
or `/model auto` to have the same local router that routes jobs pick a model and effort for each turn
|
|
64
|
-
from what you typed (the menu offers the choices as you type; a switch costs one uncached turn, and a
|
|
65
|
-
router that cannot answer refuses the turn); `/new` starts a fresh session, the memory block given to
|
|
66
|
-
the model again, at the top of the window — the old one scrolled up, not wiped; `/resume` picks a saved session up where it stopped, its last five turns drawn again so you see where that was — every
|
|
67
|
-
session is saved after each turn, whole but for its pictures and with secrets redacted, the menu lists
|
|
68
|
-
them newest first with the heavy ones marked, a session open in another terminal is refused, and
|
|
69
|
-
`sumo chat --resume [id]` opens one from the shell (the newest without an id); `/copy` puts the last reply on the clipboard as it was written — no indent, no line folded to the window — and `/copy code` its last code block; `/help` lists the commands and the keys; `/memory` opens the memory in your browser; `/mcp` lists the MCP servers — connected or not, their tools, what may run unasked; `/stop` stops a job at work in the background; `! <command>` runs a command yourself, where the agent would be refused; `/exit` (or `/quit`) ends it.
|
|
70
|
-
|
|
71
|
-
The chat is a screen, not a scroll of lines: a box to type in (`\`+Enter, Shift-Enter or Option-Enter
|
|
72
|
-
for a new line, Up for what you sent before — the last fifty, kept across sessions — `/` for the command menu;
|
|
73
|
-
Ctrl-V pastes an image from the clipboard, on macOS, and an image file dropped on the box is the image —
|
|
74
|
-
either stands in the box as `[Image #1]` and goes to the model with the message that names it; a paste too
|
|
75
|
-
long for the box is drawn by the ten rows around the cursor, all of it still there; `@` and a few letters
|
|
76
|
-
names a file of the project — the menu offers the ones they could mean, Enter or Tab puts the path in, Esc
|
|
77
|
-
shuts the menu and keeps your words; Ctrl-G opens the message in `$VISUAL` or `$EDITOR`, and what you save
|
|
78
|
-
there comes back to the box). What you said stands
|
|
79
|
-
at full weight with a bar down its side. Each tool call shows what ran and what came back: a command by its
|
|
80
|
-
first lines, and while it runs the last three lines it has printed and how long it has run; then the first
|
|
81
|
-
two and the last three lines of its output, because a run says how it went at the end; an edit as a diff —
|
|
82
|
-
the file from the project, the line it is on, `+1 −1`, the lines that changed numbered as the file numbers
|
|
83
|
-
them with a line of what stayed around them. Ctrl-O is everything in full — the whole conversation in your
|
|
84
|
-
pager (`$PAGER`, or `less`), every command whole with all its output, to scroll and to search; the screen
|
|
85
|
-
behind it is left as it was. The reply's markdown is drawn as it reads —
|
|
86
|
-
bold, italics, lists and task boxes, quotes, code blocks with their language and at full weight, tables as
|
|
87
|
-
tables — and a working line while the model is busy, which says so when the model has been quiet for twenty
|
|
88
|
-
seconds with nothing running. A delegated job stands under the call that started it while it works: how long,
|
|
89
|
-
how many calls, what it runs now; it ends in one line — done or failed, what it cost — over the top of its report.
|
|
90
|
-
|
|
91
|
-
A turn that made calls, or took ten seconds, ends on one line: how long, how many calls, the files it edited,
|
|
92
|
-
what it cost. If it took ten seconds and the terminal says its window is not the one in front, it also rings
|
|
93
|
-
— a bell and a notification (OSC 9), as does a question that waits for you; `sumo config chat.notify off`
|
|
94
|
-
turns that off. The window's title says `sumo · <project>`, with a mark while it works or waits. Under the
|
|
95
|
-
box: the model and effort, the size of the context, what the session has cost, and the project the tools run
|
|
96
|
-
in with its branch.
|
|
97
|
-
|
|
98
|
-
Esc stops the turn — first it closes an open command menu — and Ctrl-C first clears what is typed; Ctrl-C twice
|
|
99
|
-
on an empty box leaves. A message sent while the model works is read with its next request, in the same turn
|
|
100
|
-
(`next step:` above the box until it is); a command or a picture waits for the turn to end (`queued:`). Up on
|
|
101
|
-
an empty box takes back what is waiting, and stopping the turn brings all of it back to the box instead of
|
|
102
|
-
sending it. A resized window is redrawn to fit. The memory block goes to the model, not the screen; only a
|
|
103
|
-
warning in it is shown. Opening the chat scrolls the window clear rather than wiping it, so what the terminal
|
|
104
|
-
showed before is still in its scrollback; a resized window is drawn again from the top, and that does wipe it. While the
|
|
105
|
-
conversation is short the box stands at the foot of the window; until you send the first message a lion stands
|
|
106
|
-
faint in the room above it. The words on the working line are yours:
|
|
107
|
-
`spinner.txt`, one a line, a random one to start each turn and the next every five seconds; so is the lion,
|
|
108
|
-
`logo.txt`.
|
|
109
|
-
Piped (`echo … | sumo chat`), it prints plain lines, the memory block first.
|
|
110
|
-
|
|
111
|
-
## Update a machine
|
|
112
|
-
|
|
113
|
-
```sh
|
|
114
|
-
cd ~/sumo-agents && git pull
|
|
115
|
-
npm ci # refresh dependencies from the lockfile
|
|
116
|
-
sumo doctor # first line shows the commit now running — compare it with GitHub
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
`sumo` runs straight from the repo, so new code is live the moment it is pulled, and the
|
|
120
|
-
database upgrades itself the next time it is opened. Your memory is not touched — it lives in
|
|
121
|
-
`~/.sumo-agents/`, outside the repo. Run `sumo setup` again only if `sumo doctor` tells you to (you moved
|
|
122
|
-
the repo, or changed Node versions). Start a new `sumo chat` afterwards, so it loads the updated code
|
|
123
|
-
and `AGENTS.md`.
|
|
124
|
-
|
|
125
|
-
## What happens while you talk
|
|
126
|
-
|
|
127
|
-
Every session runs in Sumo's own loop: one request per turn to the Anthropic Messages API, three tools
|
|
128
|
-
(a shell and a file editor, both Anthropic-defined so no schema is sent, and `delegate`, which hands work
|
|
129
|
-
to a sub-agent) plus the tools of any MCP servers you configured, deferred behind a search tool so
|
|
130
|
-
none of their schemas sits in the context until it is needed (see *MCP servers*), a frozen system prompt that
|
|
131
|
-
caches across turns, and old tool results cleared server-side once the context passes 60k tokens.
|
|
132
|
-
History is never rewritten on the client. A request the API or the connection lets down — an overload, a
|
|
133
|
-
rate limit, a stream that breaks half-way — is sent again, up to six times with a wait that doubles from a
|
|
134
|
-
second to thirty (or what the API asked for), each try said on the screen and written to the ledger; a
|
|
135
|
-
request the API refuses as wrong is not. A turn the API pauses of its own accord goes on. The same call made
|
|
136
|
-
with the same answer three times in a row is said to the model; five times, and the turn is stopped. The turn
|
|
137
|
-
before the last of a run carries a warning, so a job that hits the turn limit still closes. Every model but
|
|
138
|
-
Haiku is asked to say what it is thinking in summary, and the end of it — its last three rows, folded to the
|
|
139
|
-
window — shows under the working line.
|
|
140
|
-
|
|
141
|
-
| When | What runs | Cost to the model you chat with |
|
|
142
|
-
|---|---|---|
|
|
143
|
-
| Session starts | The first user turn is the core block: your global preferences, workflows, projects, where you left off, open jobs, things to confirm. Default cap 800 tokens (`prime.budget`); overflow is reachable by search. | ≤ 800 tokens by default, once |
|
|
144
|
-
| You send a message | It is stored word for word (secrets redacted). First mention of a project adds its card — path, stack, commands, your rules for it, gotchas — as an operator message after the cached prefix. | 0, or ≤ 200 once per project |
|
|
145
|
-
| A shell command matches a destructive or secret-reading guard (`rm -rf ~` — by any path, escaped, behind `sudo`, `env`, `nice`, `timeout` or `xargs`, flags before or after the target — `git reset --hard`, `git clean -f`, `DROP TABLE`, `cat .env` …), or the editor tries to access a secret file | Refused in plain code before it runs, and the refusal says why. No memory is consulted. The way through is you: `! <command>` runs it yourself. A force-push is not on the list — it is an accepted way of cleaning up history here. | 0 |
|
|
146
|
-
| A command prints more than the model is shown (16k characters) | The two ends are shown, and the first and last 64 KB are written — without secrets — to the job's folder (or `~/.sumo-agents/logs/output/` for the chat), named in the cut, so the model reads a range of the file instead of running the command again. Kept a week. | 0 |
|
|
147
|
-
| An edit's `old_str` is not in the file exactly as written | It is matched again with the file's line endings, then line by line without trailing spaces, then without its indentation too — the replacement set in at the file's depth — and the result says how it matched. A looser match is still refused unless it is the only one. | 0 |
|
|
148
|
-
| The editor reaches for a path outside the project | Refused: the editor is jailed to the project (symlinks followed). Shell commands start in the project directory but can access other paths, and run with your environment — every token you exported, so a probe or a smoke test can use it — minus Sumo's own Anthropic credential. A command whose only job is to print the whole environment (`env`, `printenv`, bare `export` or `set`) is refused; a `.env` is loaded with `set -a; source .env; set +a`, never printed. | 0 |
|
|
149
|
-
| The agent is about to run a shell command a taught workflow gates (`gh pr create`, for a workflow taught with `--gate 'gh(-axi)? pr create'`) | The command is held back once and the agent is handed the workflow's steps. It follows them, then runs the command. The same steps ride in with your message when you ask for the thing yourself. | 0 until it fires |
|
|
150
|
-
| Your message asks for a fix, a feature or a review ("the export is broken", "add a --json flag", "review j12") | The matching guide rides in as an operator message, once per session — judged by the same word stems as a taught workflow, with no model asked. A taught workflow that matches the same message goes in instead. `/fix`, `/feature` and `/review` name one outright. The screen shows `guide: fix`. | 0, or ≤ 450 once per guide |
|
|
151
|
-
| The agent ends a turn after editing a file, and none of the project's declared checks (`npm test`, `make check` …) ran since | It is told the fact — which files, which check — and continued once, the way a question memory can answer is. A command that is not the check (`git status`) does not count; a project that declares no check is cleared by any command. The screen shows `held: 2 files edited, npm test not run`. | ≤ 60 tokens and one request, on such a turn |
|
|
152
|
-
| The agent ends a turn on a question | Memory is searched with the question's own words, in plain code. If something close is there, the agent is handed it once and carries on instead of waiting for you; if nothing is, the question reaches you untouched. | 0 unless memory answers |
|
|
153
|
-
| A turn ends | The **scribe** is scheduled when a pending turn contains a standing instruction, three turns are waiting, or its last run was at least five minutes ago (or it has never run). Session start and end also sweep up pending turns. A detached call to the local model proposes memories; code checks each against what you actually typed. | 0 |
|
|
154
|
-
| 3 finished sessions pile up | The **dream** pass reads them side by side: contradictions, repeated corrections, routines worth naming. | 0 |
|
|
155
|
-
| Every model response | One ledger row: tokens in, cached, written, out, cost, the job or session it served. `sumo scribe stats` sums them. | 0 |
|
|
156
|
-
|
|
157
|
-
What you state is saved silently. What the model could not tie to your exact words is held and put
|
|
158
|
-
to you as a one-line question at the next session start. Nothing from web pages, files or tool output
|
|
159
|
-
can become a memory.
|
|
160
|
-
|
|
161
|
-
Corrections that differ by one word or a case-sensitive name are kept as distinct statements;
|
|
162
|
-
similar memories can be flagged for you to settle. One message can also state separate rules for
|
|
163
|
-
several projects. Background passes serialize their runs and mark only the turns they actually
|
|
164
|
-
read as processed, so a message arriving during a pass remains pending. Dream waits for the writer
|
|
165
|
-
to finish filing a session before consolidating it.
|
|
166
|
-
|
|
167
|
-
In `sumo memory`, unsaved edits survive filtering, scope changes and a refused save within the page.
|
|
168
|
-
A successful save or Cancel clears that draft; reloading or closing the page loses it.
|
|
169
|
-
|
|
170
|
-
## See what it knows
|
|
171
|
-
|
|
172
|
-
```sh
|
|
173
|
-
sumo memory # the same, as a page in your browser: say yes or no to guesses, edit, forget
|
|
174
|
-
sumo export # everything, readable, grouped by project
|
|
175
|
-
sumo prime # exactly what a new session is shown
|
|
176
|
-
sumo project show <name> # a project's card
|
|
177
|
-
sumo search "<words>" [--project <name>]
|
|
178
|
-
sumo search "<words>" --turns # what you literally typed
|
|
179
|
-
sumo history <id> # what a memory replaced, and what replaced it
|
|
180
|
-
sumo gate <workflow-id> '<regex>' # which shell commands a workflow must come before (off to remove)
|
|
181
|
-
sumo scribe show # exactly what the cheap model would be sent right now
|
|
182
|
-
sumo scribe stats # what the background passes have cost
|
|
183
|
-
sumo forget <id> # stop it being true (history kept); --purge erases it for good
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
`sumo help` lists the rest. Ids look like `m12` (memories) and `j3` (jobs).
|
|
187
|
-
|
|
188
|
-
## Delegation
|
|
189
|
-
|
|
190
|
-
The main agent does small things itself. Big, parallel or reading-heavy work it hands off with its
|
|
191
|
-
`delegate` tool: a written brief becomes a job, and the job runs in the chat's own process on the route the
|
|
192
|
-
router chose. On the chat's screen it runs in the background: the call answers at once, the turn goes on or
|
|
193
|
-
ends, and you keep working — ask something else, start another job. The job stands in a row under the box
|
|
194
|
-
while it works (`⏺ j43 scout — Map the parser · 2m 10s · 14 tool calls · working`) — what it runs and thinks is
|
|
195
|
-
not said there. Over the jobs stands a row for the chat itself (`⏺ main · working`). Down from the box steps onto a
|
|
196
|
-
job and Enter opens it: the end of its record above the box — each call, the top of what came back, what it
|
|
197
|
-
said — as it grows; `@ message` then talks to that job, Ctrl-O shows all of the record, Esc closes it — or Down and
|
|
198
|
-
Enter on `main` comes back to the chat. When
|
|
199
|
-
it reports, how it went is drawn where the conversation has got to, and the model reads the report as an
|
|
200
|
-
automated event, never as your words: with its next request if it is at work, otherwise in a turn the
|
|
201
|
-
screen starts for it — so the main agent picks the work back up by itself. Esc stops the chat's turn, not
|
|
202
|
-
the job; `/stop` (or `/stop j43`)
|
|
203
|
-
stops a job, which says so with your next message and starts no turn; a question a job asks is answered or
|
|
204
|
-
refused (Esc) without stopping it; `/new` and `/resume` wait until no job is at work; leaving the chat stops
|
|
205
|
-
the jobs still running, to be picked up again later. Piped, and in `sumo job run`, nothing changes: the call
|
|
206
|
-
waits and the report is its result. Several `delegate` calls in one reply run at the same time. The job gets the brief as its only user turn — with the project's card and your global
|
|
207
|
-
rules in it — a short frozen system prompt that carries the Coding rules from `AGENTS.md`, and tools by role:
|
|
208
|
-
|
|
209
|
-
| Role | Model and effort | Tools | For |
|
|
210
|
-
|---|---|---|---|
|
|
211
|
-
| `scout` | chosen per job by the router | shell + web search and fetch + job tools | finding, tracing, auditing, summarizing — anything reading-heavy |
|
|
212
|
-
| `worker` | chosen per job by the router | shell + editor + `baseline`/`verify` + job tools | building and fixing |
|
|
213
|
-
| `reviewer` | chosen per job by the router | shell + job tools | judging a change it did not write |
|
|
214
|
-
|
|
215
|
-
The job tools are `note`, `ask`, `search_memory` and `finish`. Each runs the `sumo job` command a person
|
|
216
|
-
would, in a process of its own, so a worker's checks never stall the chat or the jobs beside it. A job that
|
|
217
|
-
stops without calling `finish` is told once to close itself. The Anthropic credential never enters a shell:
|
|
218
|
-
`sumo job run` and `sumo job new` typed into the chat's shell are refused with a pointer to `delegate`.
|
|
219
|
-
While jobs run the chat shows them: which job, on what model, each thing it runs (one line each; Ctrl-O for
|
|
220
|
-
the output) and what it says. `@ …` is said to the one job running, `@j<id> …` to a named one — read with
|
|
221
|
-
its next request; a job running elsewhere is reached on disk, as with `sumo job tell <id>`. Briefs, notes,
|
|
222
|
-
answers, reports and check results live in `~/.sumo-agents/jobs/<id>/`, so a job started today can be
|
|
223
|
-
picked up tomorrow: `delegate` with only its `job` id, or `sumo job run <id>` in a terminal. One worker per
|
|
224
|
-
project at a time — they share a working tree.
|
|
225
|
-
|
|
226
|
-
## The web
|
|
227
|
-
|
|
228
|
-
The chat and a `scout` can search the web and read a page: `web_search` and `web_fetch`, the API's own tools. They
|
|
229
|
-
run on Anthropic's side, inside the request — no key, no dependency, nothing run on this machine. A search shows as
|
|
230
|
-
`Search(…)` with the pages found under it, a page read as `Fetch(…)` with its size, and the pages a reply cites are
|
|
231
|
-
listed under it. Workers and reviewers do not have them: a page is somebody else's words, and it is kept out of
|
|
232
|
-
the conversations that change code or judge it. Nothing here enforces that a page's words stay information: the
|
|
233
|
-
chat that reads one has a shell with your environment, and only the model's judgment stands between them.
|
|
234
|
-
|
|
235
|
-
A search costs $0.01 on top of its tokens and is counted in the ledger; a fetch costs its tokens only. One search
|
|
236
|
-
and one page read is some 30,000 tokens of context, so a lookup across many pages belongs to a scout. A page is cut
|
|
237
|
-
at about 20,000 tokens; a PDF comes whole. There is no cap on how many searches a reply makes. Both tools are called
|
|
238
|
-
directly: the newer way, where the model filters results in code first, is refused on a Claude Code token.
|
|
239
|
-
|
|
240
|
-
## MCP servers
|
|
241
|
-
|
|
242
|
-
Sumo talks to the MCP servers you configure — Azure DevOps, Slack, whatever you use — and nothing is set up
|
|
243
|
-
by default. They live in `~/.sumo-agents/mcp.json`, in the shape Claude Code uses, so a config you already
|
|
244
|
-
have pastes in; `sumo mcp add` writes it for you:
|
|
245
|
-
|
|
246
|
-
```sh
|
|
247
|
-
sumo mcp add azure-devops -- npx -y @azure-devops/mcp <your-org> # run as a process (stdio); it signs in through `az login`
|
|
248
|
-
sumo mcp add slack --env 'SLACK_TOKEN=${SLACK_TOKEN}' -- npx -y <the slack server> # a token the server needs, named — never written
|
|
249
|
-
sumo mcp add remote --url https://host/mcp --header 'Authorization=Bearer ${TOKEN}' # reached over streamable HTTP
|
|
250
|
-
sumo mcp # every server: where it is, its tools or why it failed, what may run unasked (/mcp in the chat)
|
|
251
|
-
sumo mcp tools azure-devops # one server's tools
|
|
252
|
-
sumo mcp allow azure-devops wit_get_work_item # let it run without asking — `all` for the whole server; `revoke` undoes it
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
`${NAME}` in a value is filled from the shell when the server starts, so the file never holds a token, and a
|
|
256
|
-
name nothing fills refuses that server with the reason. A stdio server gets your environment without Sumo's
|
|
257
|
-
own credential, plus the entries you name — so a server that reads its token from a variable you exported just
|
|
258
|
-
works, and `env` is for one it wants under another name — and the node that runs Sumo first on its PATH, so
|
|
259
|
-
`npx` is found. Give commands and paths in full: a server starts where Sumo was.
|
|
260
|
-
|
|
261
|
-
The model sees the tools of every connected server, deferred behind Anthropic's tool search: none of their
|
|
262
|
-
schemas enters the context until the model searches for one by name or description, so forty Azure DevOps
|
|
263
|
-
tools cost nothing per turn until one is used, and the cached prefix is untouched. The system prompt names
|
|
264
|
-
the servers and how many tools each has. The chat and every job get them alike — a job started from a
|
|
265
|
-
terminal too.
|
|
266
|
-
|
|
267
|
-
Every call is held to an allow list. A tool not on it stops and asks on the chat's screen — `y` runs it
|
|
268
|
-
once, `a` always, `n` refuses it, Esc stops the turn — and `a` is written to the file. A job delegated from
|
|
269
|
-
the chat asks on the same screen, with its id on the question. `sumo job run` in a terminal, or the chat
|
|
270
|
-
piped, has nobody to ask, so the call is refused with the `sumo mcp allow` line that would allow it. A
|
|
271
|
-
refusal is final for that call: the model is told not to try it again.
|
|
272
|
-
|
|
273
|
-
What comes back passes through the same cap and redaction as any tool output; a picture or a file is
|
|
274
|
-
described, not sent. A server that dies is started again on the next call; one that fails to connect is
|
|
275
|
-
said on screen and simply absent for that session. Servers connect once per `sumo chat` process; a change
|
|
276
|
-
to the file is picked up by the next one.
|
|
277
|
-
|
|
278
|
-
Not in this version: the old HTTP+SSE transport, signing in to a remote server (give it a token in a
|
|
279
|
-
header), pictures to the model, MCP resources and prompts.
|
|
280
|
-
|
|
281
|
-
## Which model and effort a job gets
|
|
282
|
-
|
|
283
|
-
The model you chat with never decides this. Making the job does, in plain code; `sumo job new` prints the result:
|
|
284
|
-
|
|
285
|
-
```
|
|
286
|
-
created j27 [worker·sumo-agents·running] Session hygiene: context gauge …
|
|
287
|
-
route: opus/xhigh — router: a new feature across hooks and sessions with tests
|
|
288
|
-
run it: sumo job run 27 (append & to carry on without waiting for it)
|
|
289
|
-
```
|
|
290
|
-
|
|
291
|
-
The route is the router's answer, and nothing else's. On every `sumo job new` and every `sumo job retry`, a
|
|
292
|
-
4B open model (Qwen3-4B, through llama.cpp) reads the role, project, task text and the project's history,
|
|
293
|
-
describes the job — its kind, how much code it touches, whether it is risky — and then answers with a
|
|
294
|
-
model, an effort and one sentence. It runs locally, costs nothing, and takes about a second once the
|
|
295
|
-
model is loaded. A scout's job is put to it as reading, not as changing code: read as a change, an audit
|
|
296
|
-
is "novel" and so is a summary of an unfamiliar project, and both went to the dearest model; read as
|
|
297
|
-
reading, finding and describing are routine and only judging — an audit, a review, a bug hunt — is hard. There are no flags to override it, no project rules, no retry ladder and no floors.
|
|
298
|
-
|
|
299
|
-
The same model, through the same server, writes memory; see *The local model* below for what it is, how it
|
|
300
|
-
is run and how it was measured.
|
|
301
|
-
|
|
302
|
-
The history it reads is only the evidence against a route: a model/effort that failed here, or that a
|
|
303
|
-
review found Important issues in. A clean record is left out — it says nothing about what a cheaper
|
|
304
|
-
route would have done, and a router shown one keeps choosing it.
|
|
305
|
-
|
|
306
|
-
If the router is missing, fails, or answers something no job can run on (a model outside the list, or
|
|
307
|
-
no effort for a model that takes one), the command exits with the error and **no job is created**. The
|
|
308
|
-
answer schema pairs Haiku with no effort and every other model with one. The answer is taken as it is, for
|
|
309
|
-
every role; the router is offered only the models that are on, so `sumo models disable haiku` keeps every job off it.
|
|
310
|
-
`sumo doctor` fails while the router is missing.
|
|
311
|
-
|
|
312
|
-
Effort is real, not advisory: the route's effort goes on the request as `output_config.effort`, its model as the
|
|
313
|
-
model id, and the run's ledger rows say what ran — so what was chosen is what runs.
|
|
314
|
-
|
|
315
|
-
Models available to jobs: `haiku`, `sonnet`, `opus`, `fable`. Effort levels: `low`, `medium`,
|
|
316
|
-
`high`, `xhigh`, `max`; Haiku takes none.
|
|
317
|
-
|
|
318
|
-
Each of the four has a switch on this machine, and nothing assumes the API has all of them. `sumo setup`
|
|
319
|
-
asks the API once which of them this credential can use (one `GET /v1/models/<id>` per model) and turns
|
|
320
|
-
off the ones it does not have; `sumo models` lists them with the API id and the reason for each switch;
|
|
321
|
-
`sumo models discover` asks again and sets the switches from the answer, except that a model turned off by
|
|
322
|
-
hand stays off; `sumo models enable|disable <name>` sets one by hand. A check that fails — no credential, an error, or an answer with none of them —
|
|
323
|
-
changes nothing and says why. A model that is off is outside the router's grammar and prompt, refused by
|
|
324
|
-
`/model` and by `sumo config chat.model` / `scribe.model` / `dream.model`, never retried on by the passes
|
|
325
|
-
(the retry goes to the cheapest model that is on, or nowhere), and a job routed to it before it was
|
|
326
|
-
turned off is not run. `sumo doctor` says which are on, warns while nothing was checked, and fails when
|
|
327
|
-
none is on or a configured one is off.
|
|
328
|
-
|
|
329
|
-
**The label that grades a route is a review, not a green check.** A worker's DONE only proves the
|
|
330
|
-
checks passed; every Sonnet worker here passed its checks and the Opus reviews that followed found 1 to 7
|
|
331
|
-
Important issues each. So when a reviewer finishes, its Important count is stored on the job it judged,
|
|
332
|
-
and that is what the router's history and `sumo job stats` report:
|
|
333
|
-
|
|
334
|
-
```sh
|
|
335
|
-
sumo job stats --project <slug> # per role and model/effort: jobs, done, failed, reviewed, avg important — and what a job took: requests, time, money
|
|
336
|
-
sumo job retry <id> # a failed job, or one reviewed with 3+ Important: same brief,
|
|
337
|
-
# previous notes and report attached, routed afresh
|
|
338
|
-
```
|
|
339
|
-
|
|
340
|
-
The router itself is graded by a set of labelled tasks, each with the cheapest and the dearest route that
|
|
341
|
-
would be right for it. Run it after changing the router's prompt, its history or its model; it asks the
|
|
342
|
-
real router in a throwaway home and prints how many tasks were routed too high and too low:
|
|
19
|
+
You need Node 22.19+ or 24.6+, [llama.cpp](https://github.com/ggml-org/llama.cpp) (`brew install llama.cpp`)
|
|
20
|
+
for the local model, and `ANTHROPIC_API_KEY` or a `CLAUDE_CODE_OAUTH_TOKEN` (from `claude setup-token`) in
|
|
21
|
+
your environment.
|
|
343
22
|
|
|
344
23
|
```sh
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
24
|
+
npm install -g sumo-agents
|
|
25
|
+
sumo setup # creates ~/.sumo-agents and downloads the 2.5 GB router model
|
|
26
|
+
sumo doctor # every line should say ok
|
|
27
|
+
sumo chat
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
`npm update -g sumo-agents` moves to the newest release. Setup flags, corporate proxies and certificates
|
|
31
|
+
are covered in [Install](https://github.com/sumododda/sumo-agents/wiki/Install).
|
|
32
|
+
|
|
33
|
+
## Your first session
|
|
34
|
+
|
|
35
|
+
- Just talk. Name the project you mean; the first time, Sumo confirms its path and registers it. Started inside a
|
|
36
|
+
registered project, the chat works there from the start.
|
|
37
|
+
- Say how you like things done. "Always run the linter before you commit" is remembered in your words and
|
|
38
|
+
loaded next time.
|
|
39
|
+
- `/fix`, `/feature` and `/review` load a short written procedure for that kind of work. They also load on
|
|
40
|
+
their own when your message asks for one.
|
|
41
|
+
- Big work is delegated without being asked. A row for the job appears under the box, you keep talking,
|
|
42
|
+
and its report comes back into the conversation.
|
|
43
|
+
- `/new` starts a fresh session, and a new task deserves one. `/resume` picks an old one up. `/help` lists
|
|
44
|
+
the rest, and `?` on an empty box shows the keys you reach for most.
|
|
45
|
+
- Ctrl-R searches what you sent before. `/context` shows how full the context is and what the session has cost.
|
|
46
|
+
`/export` writes the conversation to a Markdown file.
|
|
47
|
+
|
|
48
|
+
## Good to know
|
|
49
|
+
|
|
50
|
+
- Destructive commands (`rm -rf ~`, `git reset --hard`, dropping a table) and commands that print secrets
|
|
51
|
+
are refused in code before they run. `! <command>` runs one yourself.
|
|
52
|
+
- On macOS the agent's shell runs in a sandbox: it writes only in the project, the temporary folders, Sumo's
|
|
53
|
+
home and the tool caches, never in git's hooks or config. A write it blocks is put to you once to run
|
|
54
|
+
outside it. `sumo config shell.network off` also keeps it off the network.
|
|
55
|
+
- The agent's shell gets your whole environment except the Anthropic credential. It never waits on a prompt,
|
|
56
|
+
stops a command silent for 2 minutes (`timeout 600 <cmd>` asks for longer), and runs a command ending in
|
|
57
|
+
`&` in the background with its own log.
|
|
58
|
+
- A worker's first DONE is answered with its task beside its change to check; the second runs your checks.
|
|
59
|
+
A failed first attempt is retried once on the next model up, and risky work is reviewed, without asking.
|
|
60
|
+
A second worker on a git project works in a worktree of its own.
|
|
61
|
+
- A job's run stops at $10 or 60 minutes (`jobs.budget`, `jobs.minutes`). `sumo cost` shows where the money went.
|
|
62
|
+
- `/undo` puts back the files the last chat turn changed with the editor.
|
|
63
|
+
- Only your words become memory. Web pages, files and tool output are never filed as yours.
|
|
64
|
+
- The local model, Qwen3-4B through llama.cpp, routes jobs and writes memory for $0 and sleeps when idle.
|
|
65
|
+
|
|
66
|
+
## Configuration
|
|
67
|
+
|
|
68
|
+
`sumo config <key> <value>` sets one; `sumo config` lists them. The ones most people touch:
|
|
69
|
+
|
|
70
|
+
| Key | Default | What it does |
|
|
363
71
|
|---|---|---|
|
|
364
|
-
|
|
|
365
|
-
|
|
|
366
|
-
|
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
`
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
sumo
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
… the worker works …
|
|
394
|
-
sumo job verify <id> runs the same checks and gives the verdict, without closing anything
|
|
395
|
-
sumo job finish <id> --status DONE
|
|
396
|
-
uses that verdict if the files have not moved since, otherwise runs the checks
|
|
397
|
-
again — and refuses DONE while anything is blocking
|
|
398
|
-
```
|
|
399
|
-
|
|
400
|
-
Which commands are "the project's checks": its `check` script or Makefile target when it has one (that
|
|
401
|
-
is the project's own gate), otherwise its `test`, `lint` and `typecheck` commands — the same ones shown
|
|
402
|
-
on the project card. Each gets nine minutes (`SUMO_AGENTS_CHECK_TIMEOUT_MS`); the end of its output is
|
|
403
|
-
kept in the job's folder.
|
|
404
|
-
|
|
405
|
-
| The verdict says | Meaning | Effect |
|
|
406
|
-
|---|---|---|
|
|
407
|
-
| a check `FAILED` that passed at the baseline | the job broke it | **blocks DONE** |
|
|
408
|
-
| a check `FAILED` and no baseline was taken | nothing shows it was already broken, so it counts as the job's | **blocks DONE** |
|
|
409
|
-
| a test that already existed was edited, renamed or deleted | tests judge the change; they are not part of it | **blocks DONE** — unless the job was created with `--tests-may-change` |
|
|
410
|
-
| a check declared at the start was removed, or its definition was changed | the original checks must still judge the work | **blocks DONE** — `--tests-may-change` allows definition changes, not removed commands |
|
|
411
|
-
| files changed while the checks ran | the verdict does not cover the resulting tree | **blocks DONE** — verify again after the files settle |
|
|
412
|
-
| a check `was already failing` | broken before the job began, and still is | allowed; both outputs are kept to compare |
|
|
413
|
-
| a key, token or private key was added (`ghp_…`, `sk-…`, `AKIA…`, a PEM block, a JWT), or a secret file (`.env`, `*.pem`, `id_rsa`) is in the change | a credential is about to be committed | **blocks DONE** |
|
|
414
|
-
| `look at:` added lines with `eslint-disable`, `@ts-ignore`, `# noqa`, `.skip(`, `t.Skip(` … | a checker was told to look away | allowed, and printed beside the STATUS line for a person to judge |
|
|
415
|
-
| `look at:` added lines with `password=`, `token=`, or a long random string | a credential, or a fixture that looks like one | allowed, and printed for a person to judge |
|
|
416
|
-
| `note:` not a git repository / no commands | it could not be known | said, never guessed |
|
|
417
|
-
|
|
418
|
-
A refused worker always has an honest way out, and the refusal names it: fix it, finish as `FAILED`, or
|
|
419
|
-
`sumo job ask`. When the checks cannot run on this machine, `--accept "<why>"` takes the work anyway and
|
|
420
|
-
marks it `DONE (UNVERIFIED …)` in the STATUS line and in the report. So a plain `STATUS: DONE` from a
|
|
421
|
-
worker means the checks agreed.
|
|
422
|
-
|
|
423
|
-
A baseline asked for after the first edit is refused: it would call the job's own breakage "already
|
|
424
|
-
there". New test files are always welcome — only tests that were there before the job are protected.
|
|
425
|
-
|
|
426
|
-
The worker's report has two sections for things that would otherwise go unsaid: **Concerns** (finished,
|
|
427
|
-
but not sure of) and **Decisions** (each call made on your behalf: what, why, what it costs if wrong).
|
|
428
|
-
A running worker settles small questions itself and lists them; it stops to ask only for something
|
|
429
|
-
destructive, security-sensitive, outside the project, or too unclear to do without guessing.
|
|
430
|
-
|
|
431
|
-
## Reviews
|
|
72
|
+
| `chat.model` | `opus` | What the chat runs on. `auto` lets the local router pick per turn. |
|
|
73
|
+
| `chat.effort` | `medium` | Effort when the chat model is fixed. |
|
|
74
|
+
| `chat.budget` | `5` | Dollars a chat session costs before the screen says so; `0` never. |
|
|
75
|
+
| `chat.motion` | `on` | `off` keeps the working line still: no colours flowing through it. |
|
|
76
|
+
| `projects.home` | `~` | Where new projects are created. |
|
|
77
|
+
| `jobs.budget` | `10` | Dollars one job may spend before it is stopped; `0` for no cap. |
|
|
78
|
+
| `jobs.floor` | `haiku` | The smallest model a job may run on: `sonnet` keeps every job off Haiku, `opus` puts them all on Opus or Fable. |
|
|
79
|
+
| `jobs.minutes` | `60` | Minutes one job may run before it is stopped; `0` for no limit. |
|
|
80
|
+
| `jobs.escalate` | `on` | A worker's failed first attempt is retried once on the next model up. |
|
|
81
|
+
| `jobs.review` | `risky` | Which verified work gets a reviewer without asking: `risky`, `all` or `off`. |
|
|
82
|
+
| `jobs.advisor` | `off` | `on` lets a job ask the next model up for advice. |
|
|
83
|
+
| `shell.sandbox` | `on` | The agent's shell in the macOS sandbox. Read when a chat or job starts. |
|
|
84
|
+
| `shell.network` | `on` | `off` keeps the sandboxed shell to this machine. |
|
|
85
|
+
|
|
86
|
+
The full list, the API model switches and the environment variables are in
|
|
87
|
+
[Install](https://github.com/sumododda/sumo-agents/wiki/Install#configuration).
|
|
88
|
+
|
|
89
|
+
## Learn more
|
|
90
|
+
|
|
91
|
+
The [wiki](https://github.com/sumododda/sumo-agents/wiki) has the detail:
|
|
92
|
+
|
|
93
|
+
- [The chat](https://github.com/sumododda/sumo-agents/wiki/The-chat): commands, keys, the screen, sessions
|
|
94
|
+
- [Projects and memory](https://github.com/sumododda/sumo-agents/wiki/Projects-and-memory): cards, scratch, how memory is written and managed
|
|
95
|
+
- [Jobs](https://github.com/sumododda/sumo-agents/wiki/Jobs): delegation, routing, coding guides, verifying DONE, reviews
|
|
96
|
+
- [Web search and MCP](https://github.com/sumododda/sumo-agents/wiki/Web-search-and-MCP)
|
|
97
|
+
- [Safety](https://github.com/sumododda/sumo-agents/wiki/Safety): what is enforced by code and what is only asked
|
|
98
|
+
- [How it works](https://github.com/sumododda/sumo-agents/wiki/How-it-works): a turn, the local model, costs
|
|
99
|
+
|
|
100
|
+
## Development
|
|
432
101
|
|
|
433
102
|
```sh
|
|
434
|
-
|
|
435
|
-
|
|
103
|
+
git clone https://github.com/sumododda/sumo-agents.git ~/sumo-agents && cd ~/sumo-agents
|
|
104
|
+
npm ci
|
|
105
|
+
node bin/sumo.mjs setup # points `sumo` at this clone instead of the package
|
|
106
|
+
npm test # offline suite, no real model calls
|
|
107
|
+
npm run build:bin # optional: a standalone binary in dist/sumo-agents/ (needs Bun); move the folder whole
|
|
436
108
|
```
|
|
437
109
|
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
file for any worker job), the original brief, and the author's report marked as *claims, not facts*.
|
|
441
|
-
It reports: asked vs built · findings, worst first · minor (listed, never a reason to reopen the work) ·
|
|
442
|
-
what the change alone could not show.
|
|
443
|
-
|
|
444
|
-
The reviewer runs in a fresh context, with its model and effort chosen by the local router for that
|
|
445
|
-
review. There is no guarantee that its model is stronger than the worker's. Findings are still
|
|
446
|
-
claims — `guides/review.md` ends with how to receive them.
|
|
447
|
-
|
|
448
|
-
## When to start a new session
|
|
449
|
-
|
|
450
|
-
Model quality falls with the absolute size of the context, not with the share of the window that is
|
|
451
|
-
full: degradation is measurable from tens of thousands of tokens, and a coding-agent study passed 8 of
|
|
452
|
-
10 runs clean and 3 of 10 with about 75k tokens of extra context, relevant or not. A new task gets a
|
|
453
|
-
new session. So:
|
|
454
|
-
|
|
455
|
-
- **A gauge, not a guess.** The exact context size comes back with every response and shows in the
|
|
456
|
-
chat's status line (or the plain-text prompt, `sumo 92k>`). At 80k tokens the agent is told once —
|
|
457
|
-
finish the piece in hand, then ask for `/new`. At 150k it is told to stop and note where it is.
|
|
458
|
-
- **A boundary nudge.** `sumo job finish` and `sumo job abandon` end with a `/new` line when the session
|
|
459
|
-
is already heavy, because a task boundary is the cheapest place to start over.
|
|
460
|
-
- **Nothing is lost.** The end of a turn records where you left off, and the next session's first
|
|
461
|
-
block brings it back. Open jobs resume from their briefs. The session itself is still there too:
|
|
462
|
-
`/resume` brings it back whole, for the price of one uncached turn, for thirty days.
|
|
463
|
-
- **Old tool output goes first.** Past 60k tokens the API clears the oldest tool results from what
|
|
464
|
-
the model reads, keeping the last five; the conversation itself stays intact.
|
|
465
|
-
- **A job that outgrows the context is summarized, not stopped.** At 150k tokens, at the end of a tool
|
|
466
|
-
round, the job's conversation so far is summarized server-side — told what to keep: the task, the
|
|
467
|
-
decisions, what was tried, what is done, the next step, the exact names and errors — and the job goes
|
|
468
|
-
on from the summary, with `sumo job brief <id>` named for whatever it left out. Not again within three
|
|
469
|
-
turns; a summary that cannot be had costs nothing but its request. The chat keeps its nudges: a new task
|
|
470
|
-
gets a new session.
|
|
471
|
-
- **A resumed session carries no thinking.** A thinking block is bound to the exact history it was made
|
|
472
|
-
in, and a saved session had its secrets redacted and its pictures taken out — so `/resume` sends the
|
|
473
|
-
words and the calls, not the thinking, and the first reply reasons afresh.
|
|
474
|
-
|
|
475
|
-
## What a project card tells you about hygiene
|
|
476
|
-
|
|
477
|
-
`sumo project add` scans what the project declares about itself. Besides stack and commands, the card
|
|
478
|
-
says what is absent — `not set up here: a lint command, a CI workflow` — so that "nothing failed" is
|
|
479
|
-
never mistaken for "it was checked". A `check` script or target counts as the project's gate and stands
|
|
480
|
-
in for test and lint. `sumo project rescan <slug>` after setting one up.
|
|
481
|
-
|
|
482
|
-
## What is enforced, and what is only asked
|
|
483
|
-
|
|
484
|
-
| Enforced by code | Asked in a prompt |
|
|
485
|
-
|---|---|
|
|
486
|
-
| a worker's DONE against the project's checks and a baseline | the steps in `guides/fix.md`, `feature.md`, `review.md` |
|
|
487
|
-
| existing tests untouched by a worker | the rules in `AGENTS.md` for work done in the conversation itself |
|
|
488
|
-
| reviewers and scouts receive no editor tool | reviewers and scouts use their shell only for reading |
|
|
489
|
-
| | a worker running `sumo job baseline` first — though skipping it only hurts the worker |
|
|
490
|
-
| a taught workflow's steps before a gated shell command | the nudge to start a new session |
|
|
491
|
-
| the end of a turn that edited a file and ran no declared check — held once, the check named | the guide that rode in being followed |
|
|
492
|
-
| no job, no run | |
|
|
493
|
-
| matched destructive and secret-reading shell commands are refused; editor paths stay inside the project and exclude secret files | shell commands stay inside the project |
|
|
494
|
-
| Sumo's own credential is withheld from every command; a command that prints the whole environment is refused; recognized secrets in output are redacted | |
|
|
495
|
-
| a key or token in a worker's change blocks DONE | |
|
|
496
|
-
| a job's model and effort, set on the request from its route | |
|
|
497
|
-
| an MCP tool not on the allow list waits for the user's key, and `always` is written to the file; with nobody to ask it is refused | |
|
|
498
|
-
|
|
499
|
-
The split is deliberate: a line in a prompt is a request, and the things that are cheap to check by
|
|
500
|
-
running something are checked by running something.
|
|
110
|
+
`node probes/coding-eval.mjs --route sonnet/medium` runs real jobs on small tasks and scores them with hidden
|
|
111
|
+
tests; `--hard` runs the harder set, `--no-guides` and `--advisor` change one thing at a time. It costs money.
|
|
501
112
|
|
|
502
|
-
|
|
113
|
+
A release is a commit on `main` that raises the version in `package.json`. The workflow runs the suite
|
|
114
|
+
and publishes it to npm. Probes, the coding eval and the repository layout are in
|
|
115
|
+
[Development](https://github.com/sumododda/sumo-agents/wiki/Development).
|
|
503
116
|
|
|
504
|
-
|
|
505
|
-
writes memory — the scribe after your turns, the dream pass over finished sessions. It is Qwen3-4B at
|
|
506
|
-
Q4_K_M (a 2.5 GB file), run by llama.cpp's `llama-server`, downloaded once by `sumo setup` from
|
|
507
|
-
`model.source`, `model.repo` and `model.file`. The router asks it without thinking and has an answer in
|
|
508
|
-
about a second; the passes ask it with thinking on, capped at 1,024 tokens of thought by the server's
|
|
509
|
-
reasoning budget, and have one in 8–15 s. Every answer is held to a JSON schema by the server's grammar,
|
|
510
|
-
then checked by code against what you actually typed before anything is filed.
|
|
511
|
-
|
|
512
|
-
It is one server, started by the first call that needs it and left running, detached, in router mode over
|
|
513
|
-
`~/.sumo-agents/models/` with a 32k-token context. It sleeps after ten idle minutes (under 0.5 GB resident)
|
|
514
|
-
and wakes in under a second; the first load after it starts takes up to about 15 s; it holds about 7 GB
|
|
515
|
-
while it answers. `sumo scribe status` shows its pid, port and whether the model is loaded. `sumo setup`
|
|
516
|
-
stops it, so a changed model file or binary takes effect. Its output goes to `~/.sumo-agents/logs/llama-server.log`.
|
|
517
|
-
|
|
518
|
-
```sh
|
|
519
|
-
sumo config model.repo <org>/<repo> # another GGUF repository on the model source
|
|
520
|
-
sumo config model.file <name>.gguf # the file within it — then: sumo setup
|
|
521
|
-
sumo config scribe.model haiku # send the writer to the API instead (also sonnet, or off); dream.model likewise
|
|
522
|
-
```
|
|
523
|
-
|
|
524
|
-
### How it was chosen
|
|
525
|
-
|
|
526
|
-
Measured on this machine (M5 Pro, 48 GB) on 2026-10-04 with the repo's own probes. The scope probe is
|
|
527
|
-
30 labelled turns over six conversations, 16 of them worth remembering: it asks whether the writer keeps
|
|
528
|
-
what it should, puts each memory in the right project, proves it with a quote from what was typed, and
|
|
529
|
-
leaves the 14 throwaway turns alone. Haiku 4.5 ran the same day for the baseline.
|
|
530
|
-
|
|
531
|
-
| Writer | Remembered | Right scope | Quoted | Noise filed | Per call | Resident |
|
|
532
|
-
|---|---|---|---|---|---|---|
|
|
533
|
-
| **Qwen3-4B, thinking capped at 1,024 — the default** | 16/16 | 16/16 | 16/16 | 0 | 8–15 s | ~7 GB awake |
|
|
534
|
-
| Haiku 4.5, through the API | 16/16 | 16/16 | 16/16 | 0 | 8–22 s, $0.088 for six | — |
|
|
535
|
-
| Qwen3.5-4B, thinking capped | 16/16 | 15/16 | 16/16 | 0 | 8–13 s | 3.7 GB |
|
|
536
|
-
| Qwen3.5-9B, thinking capped | 15/16 | 14/15 | 15/15 | 0 | 26–29 s | 6.4 GB |
|
|
537
|
-
| Qwen3.5-9B, thinking uncapped | every call hit the token limit with no answer | | | | 137 s | |
|
|
538
|
-
| Qwen3-4B, no thinking | 14/16 | 12/14 | 14/14 | 4 | 1–5 s | |
|
|
539
|
-
| Qwen3.5-4B, no thinking | 14/16 | 11/14 | 14/14 | 2 | 1–4 s | |
|
|
540
|
-
| Qwen3.5-9B, no thinking | 14/16 | 10/14 | 14/14 | 0 | 2–7 s | |
|
|
541
|
-
| Gemma 4 E4B, no thinking | 14/16 | 13/14 | 14/14 | 0 | 2–5 s | 5.3 GB |
|
|
542
|
-
|
|
543
|
-
Two things decided it. Thinking is what closes the gap to Haiku, and the cap is what makes thinking safe:
|
|
544
|
-
uncapped, the 9B thought until the token limit on every call, and the 4B did once on a trivial prompt. And
|
|
545
|
-
the model already here is the best of them: on the router's own grading set — 12 labelled tasks, each with
|
|
546
|
-
the cheapest and dearest acceptable route, over three project histories — Qwen3-4B routes 11 of 12 right on
|
|
547
|
-
each history, where Qwen3.5-4B fell to 9, 9 and 11. The larger candidates (Qwen3.6-35B-A3B, Nemotron 3 Nano
|
|
548
|
-
30B) are 22–25 GB files, too much to keep resident on a laptop that is also being worked on.
|
|
549
|
-
|
|
550
|
-
Caveats: the probe's bundles are about 750 tokens, while real scribe bundles are 4,800 tokens at the median
|
|
551
|
-
and 9,400 at the 90th percentile and carry the assistant's replies and related memories, so real accuracy
|
|
552
|
-
may be lower. Every number is one run at temperature 0 of a set written by the author.
|
|
553
|
-
|
|
554
|
-
```sh
|
|
555
|
-
node probes/scope-accuracy.mjs # the configured writer: local, $0, about 90 s
|
|
556
|
-
node probes/scope-accuracy.mjs haiku # the same turns on the API model, about 9 cents
|
|
557
|
-
node --disable-warning=ExperimentalWarning test/router-probes.mjs # the router's grading set
|
|
558
|
-
```
|
|
559
|
-
|
|
560
|
-
## What it costs
|
|
561
|
-
|
|
562
|
-
- A scribe or dream pass: $0, 8–15 s in the background, no credential needed. On this machine the 224 Haiku
|
|
563
|
-
scribe calls before the switch cost $1.86 in all, and dream's 18 cost $0.17.
|
|
564
|
-
- The router: one local call per job, about a second once the model is loaded, $0.
|
|
565
|
-
- The server: about 7 GB while it answers, under 0.5 GB asleep.
|
|
566
|
-
- A web search: $0.01 and the tokens of what it found; a page read: its tokens.
|
|
567
|
-
- The always-loaded prompt (`AGENTS.md`) is about 650 tokens, and a test keeps it there.
|
|
568
|
-
|
|
569
|
-
When a local call fails (server down, a bundle past the 32k context, an answer cut off) and a credential is in
|
|
570
|
-
the environment, the pass is retried once on Haiku; both calls are in the ledger, and `sumo scribe stats` sums
|
|
571
|
-
it per kind. `sumo config scribe.model haiku` (or `sonnet`) sends the writer straight to the API; `off` makes
|
|
572
|
-
the chat model save directly. `dream.model` is the same.
|
|
573
|
-
|
|
574
|
-
## Tests
|
|
575
|
-
|
|
576
|
-
```sh
|
|
577
|
-
npm test # offline suite, no real model calls (recorded answers)
|
|
578
|
-
node probes/scope-accuracy.mjs # live: the local writer on 30 labelled turns, $0 (`haiku` to compare, about 9 cents)
|
|
579
|
-
node --disable-warning=ExperimentalWarning probes/coding-eval.mjs # live: eight small jobs, scored by what they never saw
|
|
580
|
-
```
|
|
581
|
-
|
|
582
|
-
### The coding eval
|
|
583
|
-
|
|
584
|
-
Whether a change to the loop, the prompts, the tools or the router makes jobs come out right more often, or
|
|
585
|
-
cheaper, is not something the offline suite can say. `probes/coding-eval.mjs` runs eight small jobs for real
|
|
586
|
-
— six for a worker, two for a scout — each a project that fits on a page, written out fresh, handed over as
|
|
587
|
-
a brief, and scored by what the job never saw: hidden tests against the tree it left, the files it was told
|
|
588
|
-
not to touch, and for a scout the line its report was asked to end on. Each task has a way to be done wrong
|
|
589
|
-
that still looks done: `quantity || 1` passes the test given and counts a quantity of 0 as one; a bug whose
|
|
590
|
-
tests already pass can be left in place under a DONE; the test can be edited to agree with the code.
|
|
591
|
-
|
|
592
|
-
```sh
|
|
593
|
-
node --disable-warning=ExperimentalWarning probes/coding-eval.mjs --list
|
|
594
|
-
node --disable-warning=ExperimentalWarning probes/coding-eval.mjs # routed by the router
|
|
595
|
-
node --disable-warning=ExperimentalWarning probes/coding-eval.mjs --route sonnet/medium # one model for all: only the harness varies
|
|
596
|
-
node --disable-warning=ExperimentalWarning probes/coding-eval.mjs --task cart-quantity --runs 3 --json before.json
|
|
597
|
-
```
|
|
598
|
-
|
|
599
|
-
It prints a line per job — `PASS`, `FAIL`, or `FALSE DONE` for a job that said DONE and was wrong, which is
|
|
600
|
-
counted apart because nobody looks again at a DONE — with its route, requests, time and cost, and a line that
|
|
601
|
-
sums them. It needs a credential and the router model, runs in a home of its own, and reads the real one only
|
|
602
|
-
for which models are on. Run it before a change and after, on a pinned route, and more than once: one run of
|
|
603
|
-
eight tasks is a small sample.
|
|
604
|
-
|
|
605
|
-
The eval is itself under test (`test/coding-eval.test.mjs`): stand-in agents run the tasks through the real
|
|
606
|
-
job machinery, and the score must tell them apart — one that does each task right passes all eight, and the
|
|
607
|
-
sloppy fix, the idle DONE, the edited test and the report with one name too many do not.
|
|
608
|
-
|
|
609
|
-
## Layout
|
|
610
|
-
|
|
611
|
-
```
|
|
612
|
-
AGENTS.md the system prompt of the chat, and the only always-loaded instructions
|
|
613
|
-
prompts/ agent.md — the system prompt of a job; scribe.md, dream.md — the memory passes
|
|
614
|
-
guides/ fix · feature · review ride in when your words ask for them; memory · projects · workflows · delegation are read on demand
|
|
615
|
-
bin/sumo.mjs src/ the `sumo` CLI — Node, the Anthropic SDK, SQLite full-text search
|
|
616
|
-
src/loop.mjs the loop every conversation runs in; src/tools.mjs the shell and editor with their policy
|
|
617
|
-
src/web.mjs the web: the API's search and fetch tools, and how their blocks are read
|
|
618
|
-
src/mcp.mjs the MCP servers: their configuration, the two transports, the allow list, `sumo mcp`
|
|
619
|
-
src/chat.mjs the chat; src/hooks.mjs the session policy as events
|
|
620
|
-
src/ui.mjs the chat's screen (Ink, loaded only by `sumo chat`); src/editor.mjs the box you type in
|
|
621
|
-
spinner.txt the words the chat shows while it works — edit them; logo.txt the lion in the empty chat
|
|
622
|
-
src/route.mjs how a job's model and effort are chosen: the router, and the stats
|
|
623
|
-
src/local-server.mjs the one llama-server behind the router and the memory passes; src/model.mjs the calls to it and to the API
|
|
624
|
-
test/ probes/ the suite, and the live measurements: memory scope, routing, and the coding eval
|
|
625
|
-
docs/ADR-runtime.md the decision to own the runtime, and the plan it followed; docs/PLAN.md the original design
|
|
626
|
-
```
|
|
117
|
+
MIT licensed.
|