warden-agent 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +39 -4
- package/package.json +2 -2
- package/src/agent/catalog.mjs +18 -5
- package/src/agent/config.mjs +3 -0
- package/src/agent/images.mjs +165 -0
- package/src/agent/loop.mjs +73 -11
- package/src/agent/models.mjs +16 -3
- package/src/agent/prompt.mjs +5 -1
- package/src/agent/providers.mjs +105 -30
- package/src/agent/texttools.mjs +7 -1
- package/src/agent/tools.mjs +216 -26
- package/src/tui/app.mjs +729 -500
- package/src/tui/render.mjs +26 -4
- package/src/version.mjs +1 -1
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@ You chat with it in your project folder. It reads your code, edits files, runs c
|
|
|
6
6
|
|
|
7
7
|
It also includes the original Warden supervisor tools: record and review Claude Code sessions, and run the same task on several models to compare them.
|
|
8
8
|
|
|
9
|
-
> **Status: early (0.4.
|
|
9
|
+
> **Status: early (0.4.1).** Tested live with Gemini 2.5 Flash. Read [What is verified](#what-is-verified).
|
|
10
10
|
|
|
11
11
|
## Install
|
|
12
12
|
|
|
@@ -75,6 +75,8 @@ Then type what you want, e.g. `the tests in calc_test.py fail, fix them`. Warden
|
|
|
75
75
|
? Allow this change? [y] yes [a] always allow edit_file this session [n] no, or type what to do instead
|
|
76
76
|
```
|
|
77
77
|
|
|
78
|
+
**You can keep typing while it works.** Press Enter and the message is queued: the model reads it at its next step (after the tool it is running), so you can add details or change course without waiting. What you type shows in the status line.
|
|
79
|
+
|
|
78
80
|
Inside the app:
|
|
79
81
|
|
|
80
82
|
| | |
|
|
@@ -82,6 +84,8 @@ Inside the app:
|
|
|
82
84
|
| `/add` | add a model from any source (see above) |
|
|
83
85
|
| `/model` | pick from your saved models by number; `/model <name>` or `/model provider/model` switches directly (becomes your default) |
|
|
84
86
|
| `/models openrouter nemotron --free` | browse and search a source's models |
|
|
87
|
+
| `/image <description>` | make a picture with your image model, saved in `images/`; `--wide`, `--tall` or `--square` (see "Making images") |
|
|
88
|
+
| `/image-model` | pick the image model `/image` uses |
|
|
85
89
|
| `/providers` | which sources have a key |
|
|
86
90
|
| `/remove <name>` | remove a saved model; `/remove <source>` forgets that source's saved key |
|
|
87
91
|
| `/mode` | `ask` (approve edits and commands), `auto-edit` (edits in this folder run without asking), `yolo` (everything runs without asking). Shift+Tab cycles. |
|
|
@@ -89,8 +93,9 @@ Inside the app:
|
|
|
89
93
|
| `/resume` | continue an earlier conversation |
|
|
90
94
|
| `/init` | the model writes a `WARDEN.md` with notes about your project |
|
|
91
95
|
| `/usage` | tokens used |
|
|
96
|
+
| `/jobs` | servers and watchers the model started in the background; `/jobs stop <id>` or `/jobs stop all` |
|
|
92
97
|
| `!command` | run a shell command yourself; the model sees the output |
|
|
93
|
-
| Ctrl+C | interrupt the model; twice at an empty prompt quits |
|
|
98
|
+
| Esc or Ctrl+C | interrupt the model (the conversation is kept); Ctrl+C twice at an empty prompt quits |
|
|
94
99
|
|
|
95
100
|
Other ways to start:
|
|
96
101
|
|
|
@@ -103,9 +108,33 @@ warden -p "summarise this repo" # answer once and exit (scripts); --j
|
|
|
103
108
|
|
|
104
109
|
If a `WARDEN.md`, `AGENTS.md` or `CLAUDE.md` exists in the folder, its contents are given to the model as project instructions.
|
|
105
110
|
|
|
111
|
+
## Making images
|
|
112
|
+
|
|
113
|
+
Warden can make pictures for your project with a separate **image model**. Your coding model stays as it is.
|
|
114
|
+
|
|
115
|
+
1. Type `/add`, pick **OpenRouter**, and press Enter to keep your key.
|
|
116
|
+
2. At Search, type `recraft` and pick `recraft/recraft-v4.1-flash-v2:free`. Warden sees it is an image model, can make one small test image, and saves it as your image model.
|
|
117
|
+
3. Make images:
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
/image a black sport bike on a wet road at night, red light --wide
|
|
121
|
+
/image round logo with a red helmet on black --square
|
|
122
|
+
/image tall poster of a mountain road at sunrise --tall
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Each image is saved in `images/` in your project folder, named after the description. Nothing is overwritten. Then tell the model what to do with it, for example `use images/black-sport-bike-on-wet-road.webp as the hero image`. The extension is whatever the model sends: Recraft sends WebP (`.webp`), others may send `.png` or `.jpg`; every browser shows all of them. `/image-model` switches between saved image models. Esc cancels a running `/image`.
|
|
126
|
+
|
|
127
|
+
Good to know:
|
|
128
|
+
|
|
129
|
+
- Free image models are rare and come and go. `recraft-v4.1-flash-v2:free` is scheduled to go away on 28 November 2026. When a model disappears, `/image` says so; add another with `/add`.
|
|
130
|
+
- Google's image models (Nano Banana) work with `/image` too, but need billing turned on. A free Gemini key cannot make images.
|
|
131
|
+
- The coding model never makes images by itself. Only `/image` does, so your image quota is only used when you ask.
|
|
132
|
+
|
|
106
133
|
## Tools the model can use
|
|
107
134
|
|
|
108
|
-
`read_file`, `write_file
|
|
135
|
+
`read_file`, `write_file` (with `append` for writing big files in parts), `edit_file` (exact text replacement), `list_dir`, `glob`, `grep`, `bash`, and `job`. On Windows, `bash` uses Git Bash if installed, otherwise PowerShell.
|
|
136
|
+
|
|
137
|
+
Commands that keep running (dev servers, watchers) are started with `background=true`: Warden returns after their first output, the `job` tool reads their latest output or stops them, and Warden stops them all when it exits. A command that runs too long is stopped together with everything it started, so Warden never hangs on it.
|
|
109
138
|
|
|
110
139
|
## Safety
|
|
111
140
|
|
|
@@ -133,6 +162,9 @@ If a `WARDEN.md`, `AGENTS.md` or `CLAUDE.md` exists in the folder, its contents
|
|
|
133
162
|
"mode": "ask",
|
|
134
163
|
"maxSteps": 60,
|
|
135
164
|
"contextTokens": 120000,
|
|
165
|
+
"requestTimeoutSec": 180,
|
|
166
|
+
"idleTimeoutSec": 120,
|
|
167
|
+
"stallTimeoutSec": 300,
|
|
136
168
|
"providers": {
|
|
137
169
|
"deepseek": { "api": "openai", "baseUrl": "https://api.deepseek.com/v1", "keyEnv": ["DEEPSEEK_API_KEY"] }
|
|
138
170
|
}
|
|
@@ -142,6 +174,8 @@ If a `WARDEN.md`, `AGENTS.md` or `CLAUDE.md` exists in the folder, its contents
|
|
|
142
174
|
|
|
143
175
|
Any OpenAI-compatible server can be added under `providers` and used as `name/model`.
|
|
144
176
|
|
|
177
|
+
The timeouts stop a model request that hangs: no response at all after `requestTimeoutSec` (tried twice), a reply that goes silent for `idleTimeoutSec`, or a connection that sends only keep-alive pings for `stallTimeoutSec`. Your conversation is kept; send the message again.
|
|
178
|
+
|
|
145
179
|
## Supervisor tools (from Warden 0.1)
|
|
146
180
|
|
|
147
181
|
Record Claude Code sessions, review them with other models, and compare models on the same task:
|
|
@@ -156,7 +190,8 @@ See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for setup, hook configuration and
|
|
|
156
190
|
|
|
157
191
|
## What is verified
|
|
158
192
|
|
|
159
|
-
- `npm test` runs offline: a mock model server that streams tool calls, the real tools on a temp project, permissions, rate-limit retries, interrupts, saved sessions, the terminal app driven through a pipe, and the supervisor tests.
|
|
193
|
+
- `npm test` runs offline: a mock model server that streams tool calls, the real tools on a temp project, permissions, rate-limit retries, interrupts, timeouts (no response, silent stream, keep-alive only), reasoning streams, cut-off replies, saved sessions, background jobs, the terminal app driven through a pipe, and the supervisor tests.
|
|
194
|
+
- The interactive app is also tested on a simulated terminal with real keystrokes: chatting turn after turn, typing while the model works, Esc and Ctrl+C (also during an approval question), a model that stops responding, and continuing after each.
|
|
160
195
|
- **Live:** Gemini 2.5 Flash through Google's OpenAI-compatible endpoint. It found, read and fixed a bug, streamed its answer, asked for approval, and respected a denial. The `/add` tool test identified it as native tool calling. Its free daily quota (20 requests) ran out during testing, which is how the quota handling was found and fixed.
|
|
161
196
|
- **Live model lists:** OpenRouter (466 models, 16 free), Hugging Face (126, free monthly credits rather than free models) and NVIDIA (65 chat models) were listed through Warden's own code without keys.
|
|
162
197
|
- **Not tested live:** chatting through OpenRouter, Hugging Face, NVIDIA, Groq, Cerebras, Mistral, DeepSeek, Ollama, LM Studio, OpenAI and Anthropic (no keys or local servers were available). The text tool format was tested against a mock model only. They use the same code path as Gemini (except Anthropic, which has its own).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "warden-agent",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "A terminal coding agent for any model (Gemini, OpenRouter, Groq, Ollama, OpenAI, Anthropic), plus tools to review and compare coding agents.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai",
|
|
@@ -36,7 +36,7 @@
|
|
|
36
36
|
"scripts": {
|
|
37
37
|
"daemon": "node src/cli.mjs daemon",
|
|
38
38
|
"watch": "node src/cli.mjs watch",
|
|
39
|
-
"test": "node test/meta.mock.mjs && node test/agent.mock.mjs && node test/models.mock.mjs && node test/providers.mock.mjs && node test/arena.mock.mjs && node test/harnesses.mock.mjs",
|
|
39
|
+
"test": "node test/meta.mock.mjs && node test/agent.mock.mjs && node test/tui.mock.mjs && node test/images.mock.mjs && node test/models.mock.mjs && node test/providers.mock.mjs && node test/arena.mock.mjs && node test/harnesses.mock.mjs",
|
|
40
40
|
"build:standalone": "node scripts/build-standalone.mjs"
|
|
41
41
|
},
|
|
42
42
|
"engines": {
|
package/src/agent/catalog.mjs
CHANGED
|
@@ -6,16 +6,29 @@
|
|
|
6
6
|
|
|
7
7
|
const NON_CHAT = /embed|rerank|safety|guard|reward|whisper|tts|speech|transcri|moderation|ocr|parse|clip|retriever|detector|image-gen|flux|stable-diffusion/i;
|
|
8
8
|
|
|
9
|
+
// Ids that are image generators even when a source's list does not say so (used for sources without that metadata).
|
|
10
|
+
export const looksLikeImageModel = (id) => /(^|[/._:-])(image|images|imagen|nano-banana|flux|recraft|seedream|stable-diffusion|sdxl|dall-e|gpt-image|riverflow|krea)([/._:-]|\d|$)/i.test(String(id));
|
|
11
|
+
|
|
12
|
+
// OpenRouter: a model is free when nothing it charges for costs anything (image models also bill per image).
|
|
13
|
+
const orFree = (pr) => !!pr && Object.entries(pr).filter(([k]) => /^(prompt|completion|image_token|image_output)$/.test(k)).every(([, v]) => Number(v) === 0);
|
|
14
|
+
const orImage = (m) => (m.architecture?.output_modalities || []).includes("image");
|
|
15
|
+
|
|
9
16
|
export const CATALOG = {
|
|
10
17
|
openrouter: {
|
|
11
18
|
label: "OpenRouter", api: "openai", baseUrl: "https://openrouter.ai/api/v1", keyEnv: ["OPENROUTER_API_KEY"],
|
|
12
19
|
keyUrl: "https://openrouter.ai/keys", blurb: "hundreds of models, some free (Nemotron, Qwen, Llama…)", publicList: true,
|
|
13
20
|
async list(fetchJson) {
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
21
|
+
// the default list only has models that write text; image-only models (Recraft, Flux…) are asked for separately
|
|
22
|
+
const [j, img] = await Promise.all([
|
|
23
|
+
fetchJson("https://openrouter.ai/api/v1/models"),
|
|
24
|
+
fetchJson("https://openrouter.ai/api/v1/models?output_modalities=image").catch(() => ({ data: [] })),
|
|
25
|
+
]);
|
|
26
|
+
const seen = new Set();
|
|
27
|
+
return [...(j.data || []), ...(img.data || [])].filter((m) => m?.id && !seen.has(m.id) && seen.add(m.id)).map((m) => ({
|
|
28
|
+
id: m.id, ctx: orImage(m) ? null : m.context_length ?? null,
|
|
29
|
+
free: m.pricing ? orFree(m.pricing) : null,
|
|
30
|
+
tools: orImage(m) ? null : Array.isArray(m.supported_parameters) ? m.supported_parameters.includes("tools") : null,
|
|
31
|
+
image: orImage(m),
|
|
19
32
|
}));
|
|
20
33
|
},
|
|
21
34
|
},
|
package/src/agent/config.mjs
CHANGED
|
@@ -13,6 +13,9 @@ export const AGENT_DEFAULTS = {
|
|
|
13
13
|
maxOutputTokens: 8192,
|
|
14
14
|
maxSteps: 60, // model calls per user message before Warden stops and asks you
|
|
15
15
|
contextTokens: 120000, // rough budget; old tool output is trimmed above this
|
|
16
|
+
requestTimeoutSec: 180, // a model that has not started answering after this long is given up on (retried once)
|
|
17
|
+
idleTimeoutSec: 120, // a reply that goes completely silent this long is stopped
|
|
18
|
+
stallTimeoutSec: 300, // a connection kept open with no reply at all (only keep-alive pings) this long is stopped
|
|
16
19
|
providers: {}, // extra or overridden providers: { "name": { "api": "openai", "baseUrl": "...", "keyEnv": ["ENV"] } }
|
|
17
20
|
recentModels: [],
|
|
18
21
|
};
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
// Image generation for /image. A separate "image model" (saved with /add, switched with /image-model) makes the
|
|
2
|
+
// picture; your coding model is not involved and never makes images on its own.
|
|
3
|
+
//
|
|
4
|
+
// Wire formats (checked against the providers' docs on 2026-11-21):
|
|
5
|
+
// OpenRouter POST {baseUrl}/images { model, prompt, aspect_ratio, n } -> { data: [{ b64_json, media_type }] }
|
|
6
|
+
// Google POST .../v1beta/interactions { model, input, response_format } -> base64 image in an "image" content block
|
|
7
|
+
// (header Api-Revision: 2026-05-20). Google's image models are not on the free tier: they need billing.
|
|
8
|
+
// others POST {baseUrl}/images/generations (OpenAI style) { model, prompt, n, size } -> { data: [{ b64_json | url }] }
|
|
9
|
+
// The response reader below accepts every shape above (and a few older ones), so a provider tweaking its format does
|
|
10
|
+
// not break saving.
|
|
11
|
+
import fs from "node:fs";
|
|
12
|
+
import path from "node:path";
|
|
13
|
+
import { parseModelSpec, resolveProvider, keyFor, postWithRetry } from "./providers.mjs";
|
|
14
|
+
|
|
15
|
+
export const ASPECTS = { square: "1:1", wide: "16:9", tall: "9:16" };
|
|
16
|
+
const OPENAI_SIZES = { "1:1": "1024x1024", "16:9": "1536x1024", "9:16": "1024x1536" };
|
|
17
|
+
export const GOOGLE_API_REVISION = "2026-05-20";
|
|
18
|
+
|
|
19
|
+
export { looksLikeImageModel } from "./catalog.mjs";
|
|
20
|
+
|
|
21
|
+
export class ImageError extends Error {
|
|
22
|
+
constructor(message, { status, kind } = {}) { super(message); this.name = "ImageError"; this.status = status; this.kind = kind; }
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* Makes one image. Returns { bytes: Buffer, mime, ext }.
|
|
27
|
+
* spec: "openrouter/recraft/recraft-v4.1-flash-v2:free" or "gemini/gemini-nano-banana-2.1"
|
|
28
|
+
*/
|
|
29
|
+
export async function generateImage({ spec, prompt, aspect = "1:1", cfg = {}, signal, onRetry, env = process.env }) {
|
|
30
|
+
if (!String(prompt || "").trim()) throw new ImageError("Describe the image, e.g. /image a black sport bike at night");
|
|
31
|
+
const { provider, model } = parseModelSpec(spec);
|
|
32
|
+
const p = resolveProvider(provider, cfg);
|
|
33
|
+
const key = keyFor(provider, p, env);
|
|
34
|
+
const auth = key ? { authorization: `Bearer ${key}` } : {};
|
|
35
|
+
const timeoutMs = Math.max(60, Number(cfg.requestTimeoutSec) || 180) * 1000;
|
|
36
|
+
const opts = { signal, onRetry, retries: 2, timeoutMs, label: "The image model" };
|
|
37
|
+
|
|
38
|
+
let r;
|
|
39
|
+
try {
|
|
40
|
+
if (isGoogle(provider, p)) {
|
|
41
|
+
const url = p.imagesUrl || String(p.baseUrl).replace(/\/openai\/?$/, "") + "/interactions";
|
|
42
|
+
r = await postWithRetry(url, { "x-goog-api-key": key, "api-revision": GOOGLE_API_REVISION }, {
|
|
43
|
+
model, input: [{ type: "text", text: prompt }],
|
|
44
|
+
response_format: { type: "image", mime_type: "image/png", aspect_ratio: aspect },
|
|
45
|
+
}, opts);
|
|
46
|
+
} else if (provider === "openrouter" || /openrouter\.ai/.test(p.baseUrl)) {
|
|
47
|
+
r = await postWithRetry(p.imagesUrl || `${p.baseUrl}/images`, auth, { model, prompt, aspect_ratio: aspect, n: 1 }, opts);
|
|
48
|
+
} else {
|
|
49
|
+
r = await postWithRetry(p.imagesUrl || `${p.baseUrl}/images/generations`, auth, { model, prompt, n: 1, size: OPENAI_SIZES[aspect] || "1024x1024" }, opts);
|
|
50
|
+
}
|
|
51
|
+
} catch (e) {
|
|
52
|
+
if (signal?.aborted || e?.name === "AbortError") throw e;
|
|
53
|
+
throw explainImageError(e, provider);
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
const text = await r.text();
|
|
57
|
+
let j;
|
|
58
|
+
try { j = JSON.parse(text); } catch { throw new ImageError(`The image model sent something that is not an image response: ${text.replace(/\s+/g, " ").slice(0, 160)}`, { kind: "bad" }); }
|
|
59
|
+
const found = findImage(j);
|
|
60
|
+
if (!found) {
|
|
61
|
+
const said = typeof j?.output_text === "string" ? j.output_text : findText(j);
|
|
62
|
+
throw new ImageError(`The image model answered without an image${said ? `: "${said.replace(/\s+/g, " ").slice(0, 200)}"` : ""}. Try rewording the description.`, { kind: "noimage" });
|
|
63
|
+
}
|
|
64
|
+
let bytes;
|
|
65
|
+
if (found.url) {
|
|
66
|
+
const g = await fetch(found.url, { signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(60000)]) : AbortSignal.timeout(60000) });
|
|
67
|
+
if (!g.ok) throw new ImageError(`Could not download the image (${g.status}).`, { kind: "bad" });
|
|
68
|
+
bytes = Buffer.from(await g.arrayBuffer());
|
|
69
|
+
} else bytes = Buffer.from(found.b64.replace(/\s+/g, ""), "base64");
|
|
70
|
+
const kind = sniff(bytes);
|
|
71
|
+
if (!kind) throw new ImageError("The image model sent data that is not a valid image, so nothing was saved. Try again.", { kind: "bad" });
|
|
72
|
+
return { bytes, ...kind };
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
const isGoogle = (provider, p) => provider === "gemini" || /generativelanguage\.googleapis\.com/.test(p.baseUrl || "");
|
|
76
|
+
|
|
77
|
+
// Plain-language reasons for the usual failures.
|
|
78
|
+
export function explainImageError(e, provider) {
|
|
79
|
+
const msg = String(e?.message || e);
|
|
80
|
+
const st = e?.status;
|
|
81
|
+
const google = provider === "gemini";
|
|
82
|
+
if (e?.timeout) return new ImageError(`${msg}`, { status: st, kind: "timeout" });
|
|
83
|
+
// (a free key asking for a paid-only model gets "limit: 0" quota errors, or a billing / free-tier message)
|
|
84
|
+
if (google && /billing|free.?tier|limit:\s*0\b|FAILED_PRECONDITION|not available|paid/i.test(msg))
|
|
85
|
+
return new ImageError("Google's image models need billing turned on; a free Gemini key cannot make images. Turn on billing at https://aistudio.google.com, or switch to a free image model with /image-model.", { status: st, kind: "billing" });
|
|
86
|
+
if (st === 401 || st === 403) return new ImageError(`The provider rejected the key (${st}). Check it with /add, or pick another image model with /image-model.`, { status: st, kind: "auth" });
|
|
87
|
+
if (st === 402 || /insufficient credits|payment required/i.test(msg)) return new ImageError("This image model is not free and the account has no credits. Pick a free one with /image-model or /add.", { status: st, kind: "paid" });
|
|
88
|
+
if (st === 404 || /no endpoints|not found|deprecated|no longer available|removed/i.test(msg))
|
|
89
|
+
return new ImageError(`This image model is not available any more (${msg.slice(0, 120)}). Free models come and go: add another with /add, then pick it with /image-model.`, { status: st, kind: "gone" });
|
|
90
|
+
if (st === 429 || e?.quota) return new ImageError("The image model's free quota or rate limit is used up for now. Wait a bit and try again, or pick another image model with /image-model.", { status: st, kind: "quota" });
|
|
91
|
+
if (st === 400 && /safety|policy|blocked|moderat|content/i.test(msg)) return new ImageError(`The image model refused this description (${msg.slice(0, 160)}). Try describing it differently.`, { status: st, kind: "refused" });
|
|
92
|
+
return new ImageError(`The image model failed: ${msg.slice(0, 300)}`, { status: st, kind: "other" });
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
// Finds the first image in a response, whatever the provider's shape. Returns { b64 } or { url } or null.
|
|
96
|
+
export function findImage(j) {
|
|
97
|
+
const seen = new Set();
|
|
98
|
+
const stack = [j];
|
|
99
|
+
while (stack.length) {
|
|
100
|
+
const o = stack.shift();
|
|
101
|
+
if (!o || typeof o !== "object" || seen.has(o)) continue;
|
|
102
|
+
seen.add(o);
|
|
103
|
+
if (typeof o.b64_json === "string" && o.b64_json) return { b64: o.b64_json };
|
|
104
|
+
if (typeof o.bytesBase64Encoded === "string") return { b64: o.bytesBase64Encoded };
|
|
105
|
+
for (const k of ["inlineData", "inline_data", "output_image"]) if (typeof o[k]?.data === "string" && o[k].data) return { b64: o[k].data };
|
|
106
|
+
if (o.type === "image" && typeof o.data === "string" && o.data.length > 64) return fromMaybeDataUrl(o.data);
|
|
107
|
+
const iu = typeof o.image_url === "string" ? o.image_url : o.image_url?.url;
|
|
108
|
+
if (typeof iu === "string" && iu) return fromMaybeDataUrl(iu);
|
|
109
|
+
if (Array.isArray(j?.data) && j.data.includes(o) && typeof o.url === "string" && /^https?:\/\//.test(o.url)) return { url: o.url };
|
|
110
|
+
for (const [k, v] of Object.entries(o)) if (k !== "input" && v && typeof v === "object") stack.push(v);
|
|
111
|
+
}
|
|
112
|
+
return null;
|
|
113
|
+
}
|
|
114
|
+
function fromMaybeDataUrl(s) {
|
|
115
|
+
const m = /^data:[^;,]+;base64,(.*)$/s.exec(s);
|
|
116
|
+
if (m) return { b64: m[1] };
|
|
117
|
+
if (/^https?:\/\//.test(s)) return { url: s };
|
|
118
|
+
return { b64: s };
|
|
119
|
+
}
|
|
120
|
+
function findText(j) {
|
|
121
|
+
const stack = [j];
|
|
122
|
+
while (stack.length) {
|
|
123
|
+
const o = stack.shift();
|
|
124
|
+
if (!o || typeof o !== "object") continue;
|
|
125
|
+
if (o.type === "text" && typeof o.text === "string") return o.text;
|
|
126
|
+
if (typeof o.content === "string" && o.content) return o.content;
|
|
127
|
+
for (const [k, v] of Object.entries(o)) if (k !== "input" && v && typeof v === "object") stack.push(v);
|
|
128
|
+
}
|
|
129
|
+
return "";
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
// What kind of image the bytes are (by their first bytes, not by what the provider claims). null = not an image.
|
|
133
|
+
export function sniff(b) {
|
|
134
|
+
if (!b || b.length < 32) return null;
|
|
135
|
+
if (b[0] === 0x89 && b.toString("latin1", 1, 4) === "PNG") return { mime: "image/png", ext: "png" };
|
|
136
|
+
if (b[0] === 0xff && b[1] === 0xd8 && b[2] === 0xff) return { mime: "image/jpeg", ext: "jpg" };
|
|
137
|
+
if (b.toString("latin1", 0, 4) === "RIFF" && b.toString("latin1", 8, 12) === "WEBP") return { mime: "image/webp", ext: "webp" };
|
|
138
|
+
if (b.toString("latin1", 0, 4) === "GIF8") return { mime: "image/gif", ext: "gif" };
|
|
139
|
+
const head = b.toString("utf8", 0, Math.min(b.length, 600)).trimStart();
|
|
140
|
+
if (/^(<\?xml[^>]*>\s*)?(<!--[\s\S]*?-->\s*)*(<!DOCTYPE svg[^>]*>\s*)?<svg[\s>]/i.test(head)) return { mime: "image/svg+xml", ext: "svg" };
|
|
141
|
+
return null;
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
// "A black sport bike on a wet road!" -> "black-sport-bike-on-wet-road"
|
|
145
|
+
export function slugFor(prompt) {
|
|
146
|
+
const stop = new Set(["a", "an", "the", "of", "with", "and", "in", "at"]);
|
|
147
|
+
const words = String(prompt).toLowerCase().match(/[a-z0-9]+/g) || [];
|
|
148
|
+
const s = words.filter((w) => !stop.has(w)).slice(0, 6).join("-").slice(0, 50).replace(/-+$/, "");
|
|
149
|
+
return s || "image";
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
// Saves into <cwd>/images/<slug>.<ext>, adding -2, -3… instead of overwriting. Returns the path relative to cwd.
|
|
153
|
+
export function saveImage(cwd, prompt, { bytes, ext }) {
|
|
154
|
+
const dir = path.join(cwd, "images");
|
|
155
|
+
fs.mkdirSync(dir, { recursive: true });
|
|
156
|
+
const base = slugFor(prompt);
|
|
157
|
+
for (let i = 1; i < 1000; i++) {
|
|
158
|
+
const file = path.join(dir, `${base}${i > 1 ? "-" + i : ""}.${ext}`);
|
|
159
|
+
try { fs.writeFileSync(file, bytes, { flag: "wx" }); return path.relative(cwd, file).split(path.sep).join("/"); }
|
|
160
|
+
catch (e) { if (e.code !== "EEXIST") throw e; }
|
|
161
|
+
}
|
|
162
|
+
throw new ImageError("Too many images with this name in images/. Rename or delete some.");
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
export const fmtBytes = (n) => (n >= 1048576 ? `${(n / 1048576).toFixed(1)} MB` : `${Math.max(1, Math.round(n / 1024))} KB`);
|
package/src/agent/loop.mjs
CHANGED
|
@@ -1,9 +1,8 @@
|
|
|
1
|
-
// The agent loop: send the conversation to the model, run the tools it asks for (with
|
|
2
|
-
// it answers without tool calls. The UI is injected so the same loop runs in the terminal app and in headless mode.
|
|
1
|
+
// The agent loop: send the conversation to the model, run the tools it asks for (with approvals), repeat until it answers.
|
|
3
2
|
import { chat, toolsUnsupported } from "./providers.mjs";
|
|
4
|
-
import { findModel, toolModeFor, setToolMode } from "./models.mjs";
|
|
5
3
|
import { TOOLS, toolByName, needsApproval, describeCall } from "./tools.mjs";
|
|
6
4
|
import { systemPrompt } from "./prompt.mjs";
|
|
5
|
+
import { findModel, toolModeFor, setToolMode } from "./models.mjs";
|
|
7
6
|
|
|
8
7
|
const isAbort = (e) => e?.name === "AbortError" || /aborted/i.test(e?.message || "");
|
|
9
8
|
|
|
@@ -19,20 +18,56 @@ export function trimContext(messages, budgetTokens) {
|
|
|
19
18
|
return trimmed;
|
|
20
19
|
}
|
|
21
20
|
|
|
21
|
+
// File contents the model already wrote are on disk; keeping every full copy in the conversation makes each later
|
|
22
|
+
// request bigger and slower (and burns free-tier token limits). Older write_file / edit_file arguments are replaced
|
|
23
|
+
// by a short note; the model reads the file again when it needs it. The latest assistant message is left alone.
|
|
24
|
+
const BIG = 1500;
|
|
25
|
+
export function compactWrites(messages) {
|
|
26
|
+
let lastAssistant = -1;
|
|
27
|
+
for (let i = messages.length - 1; i >= 0; i--) if (messages[i].role === "assistant") { lastAssistant = i; break; }
|
|
28
|
+
let n = 0;
|
|
29
|
+
for (let i = 0; i < lastAssistant; i++) {
|
|
30
|
+
const m = messages[i];
|
|
31
|
+
if (m.role !== "assistant" || !m.toolCalls?.length) continue;
|
|
32
|
+
for (const c of m.toolCalls) {
|
|
33
|
+
const a = c.args;
|
|
34
|
+
if (!a || c.compacted || typeof a !== "object") continue;
|
|
35
|
+
if (c.name === "write_file" && String(a.content ?? "").length > BIG) {
|
|
36
|
+
const lines = String(a.content).split("\n").length;
|
|
37
|
+
c.args = { ...a, content: `[${lines} lines ${a.append ? "appended" : "written"} earlier; removed from the conversation to save space. Read ${a.path} to see the current contents.]` };
|
|
38
|
+
c.compacted = true; n++;
|
|
39
|
+
} else if (c.name === "edit_file" && String(a.old_string ?? "").length + String(a.new_string ?? "").length > BIG * 2) {
|
|
40
|
+
c.args = { ...a, old_string: `[${String(a.old_string).split("\n").length} lines, removed to save space]`, new_string: `[${String(a.new_string).split("\n").length} lines, removed to save space; read ${a.path} for the current contents]` };
|
|
41
|
+
c.compacted = true; n++;
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
return n;
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
const TRUNCATED_ARGS = (name, limit) =>
|
|
49
|
+
`Your reply hit the output limit (${limit} tokens) while you were still writing the arguments for ${name}, so the call was cut off and nothing was done.` +
|
|
50
|
+
(name === "write_file" ? " Write big files in parts: call write_file with the first part (under about 300 lines), then write_file with append=true for each further part." : " Send a shorter call, splitting the work into several smaller calls.");
|
|
51
|
+
|
|
22
52
|
/**
|
|
23
53
|
* Runs one user message to completion.
|
|
24
|
-
* ui: { onAssistantStart, onText, onAssistantEnd, onToolStart, onToolEnd, confirm, onRetry, onNotice
|
|
54
|
+
* ui: { onAssistantStart, onText, onReasoning, onAssistantEnd, onToolStart, onToolOutput, onToolEnd, confirm, onRetry, onNotice,
|
|
55
|
+
* takeQueued, onQueuedDelivered }
|
|
56
|
+
* takeQueued(): text the user typed while the model was working (or null). It is handed to the model between steps,
|
|
57
|
+
* so the user can steer without waiting for the whole task to finish.
|
|
25
58
|
* perm: { mode, allowed: Set<toolName> }
|
|
26
59
|
*/
|
|
27
60
|
export async function runTurn({ session, userText, cfg, ui, perm, signal, cwd = session.cwd }) {
|
|
28
61
|
session.push({ role: "user", content: userText });
|
|
29
62
|
const tools = TOOLS.map(({ name, description, parameters }) => ({ name, description, parameters }));
|
|
63
|
+
const limit = cfg.maxOutputTokens || 8192;
|
|
30
64
|
|
|
31
65
|
for (let step = 1; ; step++) {
|
|
32
66
|
if (step > cfg.maxSteps) {
|
|
33
67
|
ui.onNotice?.(`Stopped after ${cfg.maxSteps} model calls for one message. Say "continue" to keep going.`);
|
|
34
68
|
return { stopped: "max_steps" };
|
|
35
69
|
}
|
|
70
|
+
compactWrites(session.messages);
|
|
36
71
|
// a saved model may have a smaller context window than the global budget
|
|
37
72
|
const saved = findModel(cfg, session.model);
|
|
38
73
|
const budget = saved?.ctx ? Math.min(cfg.contextTokens, Math.floor(saved.ctx * 0.7)) : cfg.contextTokens;
|
|
@@ -46,12 +81,12 @@ export async function runTurn({ session, userText, cfg, ui, perm, signal, cwd =
|
|
|
46
81
|
try {
|
|
47
82
|
res = await chat({
|
|
48
83
|
spec: session.model, system: systemPrompt({ cwd, mode: perm.mode }), messages: session.messages, tools,
|
|
49
|
-
cfg, signal, maxOutputTokens:
|
|
50
|
-
onText: (d) => { partial += d; ui.onText?.(d); }, onRetry: ui.onRetry,
|
|
84
|
+
cfg, signal, maxOutputTokens: limit, toolMode, timeouts: cfg,
|
|
85
|
+
onText: (d) => { partial += d; ui.onText?.(d); }, onReasoning: (d) => ui.onReasoning?.(d), onToolDelta: (n, a) => ui.onToolDelta?.(n, a), onRetry: ui.onRetry,
|
|
51
86
|
});
|
|
52
87
|
} catch (e) {
|
|
53
88
|
ui.onAssistantEnd?.(partial);
|
|
54
|
-
if (isAbort(e)
|
|
89
|
+
if (signal?.aborted || (isAbort(e) && !e.timeout)) {
|
|
55
90
|
session.push({ role: "assistant", content: (partial ? partial + "\n" : "") + "[interrupted by the user]" });
|
|
56
91
|
return { interrupted: true };
|
|
57
92
|
}
|
|
@@ -63,22 +98,45 @@ export async function runTurn({ session, userText, cfg, ui, perm, signal, cwd =
|
|
|
63
98
|
step--;
|
|
64
99
|
continue;
|
|
65
100
|
}
|
|
66
|
-
// keep the conversation valid for the next message: the user turn stays
|
|
101
|
+
// keep the conversation valid for the next message: the user turn stays; what streamed so far is kept as context
|
|
102
|
+
if (partial) session.push({ role: "assistant", content: partial + "\n[reply cut off: " + String(e.message).slice(0, 120) + "]" });
|
|
67
103
|
throw e;
|
|
68
104
|
}
|
|
69
105
|
ui.onAssistantEnd?.(res.text);
|
|
70
106
|
session.usage.calls++;
|
|
71
107
|
if (res.usage) { session.usage.input += res.usage.input; session.usage.output += res.usage.output; }
|
|
108
|
+
const cutOff = /^(length|max_tokens|MAX_TOKENS)$/.test(res.stopReason || "");
|
|
72
109
|
session.push({ role: "assistant", content: res.text, ...(res.toolCalls.length ? { toolCalls: res.toolCalls } : {}) });
|
|
73
|
-
|
|
110
|
+
|
|
111
|
+
if (!res.toolCalls.length) {
|
|
112
|
+
const queued = ui.takeQueued?.();
|
|
113
|
+
if (queued) { // the user said something while this step ran: keep going with it instead of ending the turn
|
|
114
|
+
session.push({ role: "user", content: queued });
|
|
115
|
+
ui.onQueuedDelivered?.(queued);
|
|
116
|
+
continue;
|
|
117
|
+
}
|
|
118
|
+
if (!res.text.trim()) {
|
|
119
|
+
ui.onNotice?.(cutOff
|
|
120
|
+
? `The model used its whole output limit (${limit} tokens) without writing a reply, often by thinking too long. Ask again, ask for something smaller, or raise "maxOutputTokens" in settings.`
|
|
121
|
+
: `The model sent an empty reply${res.stopReason ? ` (finish reason: ${res.stopReason})` : ""}. Ask again or rephrase; if it keeps happening, try another model with /model.`);
|
|
122
|
+
return { done: true, text: "", empty: true, stopReason: res.stopReason };
|
|
123
|
+
}
|
|
124
|
+
if (cutOff) ui.onNotice?.(`The reply hit the output limit (${limit} tokens) and was cut off. Say "continue" for the rest.`);
|
|
125
|
+
return { done: true, text: res.text, stopReason: res.stopReason };
|
|
126
|
+
}
|
|
74
127
|
|
|
75
128
|
// Every tool call must get a result, even when interrupted, or the next request is invalid.
|
|
76
129
|
for (const call of res.toolCalls) {
|
|
77
130
|
const tool = toolByName(call.name);
|
|
78
131
|
const result = (content, isError = true) => session.push({ role: "tool", toolCallId: call.id, name: call.name, content, ...(isError ? { isError: true } : {}) });
|
|
79
132
|
if (signal?.aborted) { result("Interrupted by the user before this ran."); continue; }
|
|
133
|
+
if (call.args?.__invalid_json !== undefined) {
|
|
134
|
+
const truncated = cutOff || call.args.__truncated;
|
|
135
|
+
result(truncated ? TRUNCATED_ARGS(tool?.name || "the tool call", limit) : "Your tool arguments were not valid JSON. Send them again as a JSON object.");
|
|
136
|
+
ui.onToolEnd?.(call, tool, { error: truncated ? "cut off at the output limit; the model will retry in smaller parts" : "invalid arguments" });
|
|
137
|
+
continue;
|
|
138
|
+
}
|
|
80
139
|
if (!tool) { result(`Unknown tool "${call.name}". Available: ${TOOLS.map((t) => t.name).join(", ")}`); ui.onToolEnd?.(call, null, { error: "unknown tool" }); continue; }
|
|
81
|
-
if (call.args?.__invalid_json !== undefined) { result("Your tool arguments were not valid JSON. Send them again as a JSON object."); ui.onToolEnd?.(call, tool, { error: "invalid arguments" }); continue; }
|
|
82
140
|
|
|
83
141
|
ui.onToolStart?.(call, tool);
|
|
84
142
|
if (needsApproval(tool, call.args, { cwd, mode: perm.mode, allowed: perm.allowed })) {
|
|
@@ -90,9 +148,10 @@ export async function runTurn({ session, userText, cfg, ui, perm, signal, cwd =
|
|
|
90
148
|
ui.onToolEnd?.(call, tool, { denied: true });
|
|
91
149
|
continue;
|
|
92
150
|
}
|
|
151
|
+
ui.onToolApproved?.(call, tool);
|
|
93
152
|
}
|
|
94
153
|
try {
|
|
95
|
-
const out = await tool.run(call.args || {}, { cwd, signal });
|
|
154
|
+
const out = await tool.run(call.args || {}, { cwd, signal, onOutput: (s) => ui.onToolOutput?.(call, s) });
|
|
96
155
|
result(out.content, !!out.isError);
|
|
97
156
|
ui.onToolEnd?.(call, tool, out);
|
|
98
157
|
} catch (e) {
|
|
@@ -101,5 +160,8 @@ export async function runTurn({ session, userText, cfg, ui, perm, signal, cwd =
|
|
|
101
160
|
}
|
|
102
161
|
}
|
|
103
162
|
if (signal?.aborted) return { interrupted: true };
|
|
163
|
+
// anything the user typed meanwhile goes to the model now, before its next step
|
|
164
|
+
const queued = ui.takeQueued?.();
|
|
165
|
+
if (queued) { session.push({ role: "user", content: queued }); ui.onQueuedDelivered?.(queued); }
|
|
104
166
|
}
|
|
105
167
|
}
|
package/src/agent/models.mjs
CHANGED
|
@@ -1,10 +1,20 @@
|
|
|
1
1
|
// Models you added with /add, saved in ~/.warden/config.json under agent.models:
|
|
2
2
|
// [{ name: "nemotron-free", spec: "openrouter/nvidia/nemotron-3.5-lightning:free", ctx: 262144, tools: "native" | "text" | "none" | null }]
|
|
3
3
|
// tools = how this model gets to use Warden's tools: its own tool calling, the text fallback, or chat only.
|
|
4
|
+
// kind: "image" marks an image model (used only by /image; agent.imageModel holds the spec of the one in use).
|
|
4
5
|
import { saveAgentConfig } from "./config.mjs";
|
|
5
6
|
import { chat, toolsUnsupported } from "./providers.mjs";
|
|
6
7
|
|
|
7
8
|
export const savedModels = (cfg) => (Array.isArray(cfg.models) ? cfg.models : []);
|
|
9
|
+
export const chatModels = (cfg) => savedModels(cfg).filter((m) => m.kind !== "image");
|
|
10
|
+
export const imageModels = (cfg) => savedModels(cfg).filter((m) => m.kind === "image");
|
|
11
|
+
|
|
12
|
+
// The image model /image uses: { name, spec } (name may be null for one set by spec only), or null.
|
|
13
|
+
export function currentImageModel(cfg) {
|
|
14
|
+
if (!cfg.imageModel) return null;
|
|
15
|
+
return findModel(cfg, cfg.imageModel) || { name: null, spec: cfg.imageModel };
|
|
16
|
+
}
|
|
17
|
+
export function setImageModel(cfg, spec) { cfg.imageModel = spec || null; saveAgentConfig({ imageModel: cfg.imageModel }); }
|
|
8
18
|
|
|
9
19
|
export function findModel(cfg, nameOrSpec) {
|
|
10
20
|
return savedModels(cfg).find((m) => m.name === nameOrSpec) || savedModels(cfg).find((m) => m.spec === nameOrSpec) || null;
|
|
@@ -14,6 +24,7 @@ export function findModel(cfg, nameOrSpec) {
|
|
|
14
24
|
export function resolveModelInput(cfg, input) {
|
|
15
25
|
const s = String(input || "").trim();
|
|
16
26
|
const hit = savedModels(cfg).find((m) => m.name === s);
|
|
27
|
+
if (hit?.kind === "image") throw new Error(`"${s}" is an image model: it makes pictures with /image and cannot chat or code. Use it with /image-model ${s}`);
|
|
17
28
|
if (hit) return hit.spec;
|
|
18
29
|
if (s.includes("/")) return s;
|
|
19
30
|
throw new Error(`no saved model called "${s}". /model lists them; /add adds one; or use provider/model`);
|
|
@@ -29,9 +40,10 @@ export function addModel(cfg, entry) {
|
|
|
29
40
|
|
|
30
41
|
export function removeModel(cfg, name) {
|
|
31
42
|
const before = savedModels(cfg);
|
|
32
|
-
const
|
|
33
|
-
if (
|
|
34
|
-
persist(cfg,
|
|
43
|
+
const gone = before.find((m) => m.name === name);
|
|
44
|
+
if (!gone) return false;
|
|
45
|
+
persist(cfg, before.filter((m) => m !== gone));
|
|
46
|
+
if (gone.kind === "image" && cfg.imageModel === gone.spec) setImageModel(cfg, null);
|
|
35
47
|
return true;
|
|
36
48
|
}
|
|
37
49
|
|
|
@@ -39,6 +51,7 @@ export function removeModelsOf(cfg, provider) {
|
|
|
39
51
|
const after = savedModels(cfg).filter((m) => !m.spec.startsWith(provider + "/"));
|
|
40
52
|
const n = savedModels(cfg).length - after.length;
|
|
41
53
|
if (n) persist(cfg, after);
|
|
54
|
+
if (String(cfg.imageModel || "").startsWith(provider + "/")) setImageModel(cfg, null);
|
|
42
55
|
return n;
|
|
43
56
|
}
|
|
44
57
|
|
package/src/agent/prompt.mjs
CHANGED
|
@@ -34,9 +34,13 @@ export function systemPrompt({ cwd, mode }) {
|
|
|
34
34
|
How to work:
|
|
35
35
|
- Use the tools to look before you act. Read a file before editing it. Search with glob/grep instead of guessing paths.
|
|
36
36
|
- Make the smallest change that does what was asked. Do not refactor, rename or add features nobody asked for.
|
|
37
|
-
- Prefer edit_file for changes to existing files; use write_file only for new files or full rewrites.
|
|
37
|
+
- Prefer edit_file for changes to existing files; use write_file only for new files or full rewrites. When the user asks for a change to something you already made, read the file and make small edit_file changes; do not rewrite the whole file to change a few lines.
|
|
38
|
+
- Write large files (more than about 300 lines) in parts: write_file with the first part, then write_file with append=true for the rest. A single huge tool call can be cut off by the output limit.
|
|
39
|
+
- File contents you wrote earlier may be removed from the conversation to save space. Read a file again before editing it.
|
|
38
40
|
- After changing code, run the relevant tests or a quick check with bash when one exists, and say what you ran and what happened. If you did not verify something, say so plainly.
|
|
39
41
|
- Use bash for commands (tests, builds, git, package managers). Do not use it to read or write files.
|
|
42
|
+
- Never run anything that keeps running (dev servers, \`npm run dev\`, \`npx serve\`, \`python -m http.server\`, file watchers) without background=true: it would block until it times out. Start it with background=true, check it with the job tool, and tell the user the address. To show a static page, you can also just tell the user which file to open in a browser.
|
|
43
|
+
- The user may send new messages while you work. They arrive as user messages between your steps: read them and follow the latest instructions.
|
|
40
44
|
- Never claim you did something you did not do. If a tool fails, read the error and adjust; do not repeat the same failing call.
|
|
41
45
|
- If the user denies a tool call, do not retry it the same way; adapt or ask.
|
|
42
46
|
- Do not run destructive commands (deleting data, force-pushing, rewriting git history) unless the user explicitly asked.
|