s-app-scoder-agent 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +86 -20
- package/dist/agent.js +64 -5
- package/dist/build.js +3 -2
- package/dist/index.js +44 -0
- package/dist/server.js +256 -0
- package/dist/tools.js +36 -4
- package/dist/ui.js +76 -10
- package/dist/webfetch.js +251 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -80,9 +80,22 @@ You (task) → LLM plans → Tools execute code → Build runs → Success? →
|
|
|
80
80
|
| `edit_file` | Find-and-replace (unique match required) |
|
|
81
81
|
| `list_files` | Recursive directory listing |
|
|
82
82
|
| `run_command` | Execute any shell command (subject to permission policy) |
|
|
83
|
+
| `web_fetch` | Fetch a public HTTP(S) URL → readable text (HTML→text, JSON pretty-printed) |
|
|
83
84
|
|
|
84
85
|
File writes (`write_file` / `edit_file`) are symlink-safe: paths are resolved via `realpath` and cannot escape the working directory, even through a malicious symlink. `read_file` refuses files larger than 2MB with guidance to use `grep`/`head`/`tail`.
|
|
85
86
|
|
|
87
|
+
### Web access (`web_fetch`)
|
|
88
|
+
|
|
89
|
+
The agent can read the internet through the built-in `web_fetch` tool — it fetches exact URLs (it cannot search) and returns readable plain text: HTML is converted to text (scripts/styles stripped, entities decoded, title extracted), JSON is pretty-printed, binary content is refused. The system prompt instructs the agent *when to fetch* (external/API docs, URLs in error messages, current versions, changelogs) and when not to (answer available locally).
|
|
90
|
+
|
|
91
|
+
Safety defaults:
|
|
92
|
+
|
|
93
|
+
- **SSRF protection** — `localhost`, private and link-local addresses (10.x, 172.16/12, 192.168.x, 127.x, 169.254.x, IPv6 ULA/link-local) are blocked. Set `SCODER_WEB_ALLOW_PRIVATE=1` to allow local dev servers.
|
|
94
|
+
- Only `http`/`https`; up to 5 redirects — every hop is re-validated against the private-address block list.
|
|
95
|
+
- 30 s timeout, SIGINT-abortable; body capped at 512 KB; extracted text capped at 20k chars (configurable via `max_chars`).
|
|
96
|
+
- HTML is converted to plain text (no JS execution); images/archives/PDFs are refused.
|
|
97
|
+
- Read-only — the tool never writes locally, so it is allowed even in `readonly` permission mode.
|
|
98
|
+
|
|
86
99
|
## Configuration
|
|
87
100
|
|
|
88
101
|
### Environment variables
|
|
@@ -96,6 +109,9 @@ File writes (`write_file` / `edit_file`) are symlink-safe: paths are resolved vi
|
|
|
96
109
|
| `SCODER_MAX_STEPS` | `40` | Max loop iterations before giving up |
|
|
97
110
|
| `SCODER_PERMISSION` | `auto` | Permission mode: `auto` \| `ask` \| `readonly` |
|
|
98
111
|
| `SCODER_STREAM` | `1` | `0` to disable streaming LLM output |
|
|
112
|
+
| `SCODER_STREAM` | `1` | `0` to disable streaming LLM output |
|
|
113
|
+
| `SCODER_SERVER_TOKEN` | — | `scoder serve` only: require `Authorization: Bearer <token>` |
|
|
114
|
+
| `SCODER_WEB_ALLOW_PRIVATE` | — | Set to `1` to let `web_fetch` reach localhost/private addresses (local dev) |
|
|
99
115
|
| `SCODER_NO_APIKEY_WARNING` | — | Set to `1` to suppress the apiKey-in-rc warning |
|
|
100
116
|
|
|
101
117
|
### Per-project config file (.scoderrc.json)
|
|
@@ -225,7 +241,39 @@ Sessions older than 7 days are pruned automatically on the next run. Use `--no-s
|
|
|
225
241
|
|
|
226
242
|
## Streaming
|
|
227
243
|
|
|
228
|
-
|
|
244
|
+
scoder streams everything it can, live, as it happens:
|
|
245
|
+
|
|
246
|
+
- **LLM tokens** — assistant text renders character-by-character (OpenAI-compatible SSE and native Anthropic streaming). If a provider doesn't support streaming, scoder automatically falls back to non-streaming requests. Disable with `SCODER_STREAM=0`.
|
|
247
|
+
- **Tool output** — `run_command` and build output print live (dimmed block) while the command runs, instead of appearing only after it finishes. Live output is capped at 20k chars; the full output still goes to the LLM.
|
|
248
|
+
- **SIGINT-safe** — Ctrl+C kills the running command immediately; streamed rendering never blocks the event loop.
|
|
249
|
+
- Output honors `NO_COLOR` and degrades to plain text when piped (CI-friendly).
|
|
250
|
+
|
|
251
|
+
## HTTP Server Mode (`scoder serve`)
|
|
252
|
+
|
|
253
|
+
Run scoder as a local HTTP service — submit tasks programmatically and consume progress as Server-Sent Events:
|
|
254
|
+
|
|
255
|
+
```bash
|
|
256
|
+
scoder serve # http://127.0.0.1:0 (random port, printed at startup)
|
|
257
|
+
scoder serve --port 4114 # fixed port
|
|
258
|
+
SCODER_SERVER_TOKEN=secret scoder serve --host 127.0.0.1
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
| Endpoint | Method | Description |
|
|
262
|
+
|----------|--------|-------------|
|
|
263
|
+
| `/api/health` | GET | Liveness: `{ ok, running, runs }` |
|
|
264
|
+
| `/api/runs` | POST | Start a run: `{ "task": "...", "workdir?", "buildCmd?", "model?", "provider?", "maxSteps?", "permission?" }` → `202 { id, eventsUrl }` |
|
|
265
|
+
| `/api/runs` | GET | List runs with status/timing |
|
|
266
|
+
| `/api/runs/<id>` | GET | Run status + final result |
|
|
267
|
+
| `/api/runs/<id>/events` | GET | **SSE stream** — full event replay, then live, closes on finish |
|
|
268
|
+
| `/api/runs/<id>/abort` | POST | Abort a running run (equivalent to SIGINT) |
|
|
269
|
+
|
|
270
|
+
SSE event types: `run_start`, `step`, `llm_delta` (assistant text), `tool_call`, `tool_output` (live stdout/stderr chunks), `tool_result`, `llm_usage`, `build`, `finish`. The stream replays every event from the beginning, then stays live, and closes after `finish` — safe to (re)connect at any time.
|
|
271
|
+
|
|
272
|
+
Notes:
|
|
273
|
+
|
|
274
|
+
- Runs are always non-interactive; `permission: "ask"` is rejected — use `auto` or `readonly`.
|
|
275
|
+
- Binds `127.0.0.1` by default. Set `SCODER_SERVER_TOKEN` to require `Authorization: Bearer <token>` on every request.
|
|
276
|
+
- Max 4 concurrent runs (extra requests get `429`); per-run overrides: `workdir`, `buildCmd`, `model`, `provider`, `maxSteps`, `permission`.
|
|
229
277
|
|
|
230
278
|
## Logging
|
|
231
279
|
|
|
@@ -268,6 +316,7 @@ scoder detects the build command automatically:
|
|
|
268
316
|
```
|
|
269
317
|
scoder [options] "<task>" Run a task autonomously until the build passes
|
|
270
318
|
scoder Start interactive session
|
|
319
|
+
scoder serve [--port] [--host] HTTP server mode (see above)
|
|
271
320
|
scoder --help | --version Help and version
|
|
272
321
|
```
|
|
273
322
|
|
|
@@ -283,6 +332,9 @@ scoder --help | --version Help and version
|
|
|
283
332
|
| `-y` | Non-interactive mode (denies commands needing confirmation) |
|
|
284
333
|
| `--provider <name>` | `openai` \| `anthropic` (env SCODER_PROVIDER; auto-detected) |
|
|
285
334
|
| `--no-save` | Disable session auto-save to `.scoder/sessions/` |
|
|
335
|
+
| `--log-file` | Write a structured JSONL event log to `.scoder/logs/` |
|
|
336
|
+
| `--port <n>` | (`serve`) TCP port, `0` = random free port |
|
|
337
|
+
| `--host <addr>` | (`serve`) Bind address (default `127.0.0.1`) |
|
|
286
338
|
|
|
287
339
|
## Examples
|
|
288
340
|
|
|
@@ -305,6 +357,15 @@ scoder -m claude-sonnet-4-5 "refactor the parser module"
|
|
|
305
357
|
|
|
306
358
|
# CI: non-interactive, no session saving
|
|
307
359
|
scoder --no-save -y "run the full migration"
|
|
360
|
+
|
|
361
|
+
# Run as an HTTP service and consume progress via SSE
|
|
362
|
+
scoder serve --port 4114 &
|
|
363
|
+
curl -s -X POST localhost:4114/api/runs -H 'Content-Type: application/json' \
|
|
364
|
+
-d '{"task": "add input validation to the CLI"}'
|
|
365
|
+
curl -N http://127.0.0.1:4114/api/runs/<id>/events # live progress
|
|
366
|
+
|
|
367
|
+
# Fetch external docs during a task (agent decides when, via web_fetch)
|
|
368
|
+
scoder "upgrade express to v5 — check the official migration guide first"
|
|
308
369
|
```
|
|
309
370
|
|
|
310
371
|
## Architecture
|
|
@@ -312,22 +373,26 @@ scoder --no-save -y "run the full migration"
|
|
|
312
373
|
```
|
|
313
374
|
┌─────────────────────────────────────────────┐
|
|
314
375
|
│ CLI (src/index.ts) │
|
|
315
|
-
│ arg parsing, REPL, SIGINT
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
376
|
+
│ arg parsing, REPL, serve, SIGINT │
|
|
377
|
+
└───────────────┬─────────────────────────────┘
|
|
378
|
+
│
|
|
379
|
+
┌───────────────▼─────────────────────────────┐
|
|
319
380
|
│ Agent (src/agent.ts) │
|
|
320
|
-
│
|
|
321
|
-
│
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
│
|
|
327
|
-
│
|
|
328
|
-
|
|
329
|
-
│
|
|
330
|
-
|
|
381
|
+
│ Closed-loop orchestrator │
|
|
382
|
+
│ history trimming, error injection │
|
|
383
|
+
│ emits AgentEvent (UI / SSE consumers) │
|
|
384
|
+
└──────┬─────────────┬───────────────┬────────┘
|
|
385
|
+
│ │ │
|
|
386
|
+
┌──────▼───────┐ ┌──▼───────────┐ ┌▼──────────────────┐
|
|
387
|
+
│ LLM Client │ │ Tool Registry│ │ Server │
|
|
388
|
+
│ (src/llm.ts) │ │(src/tools.ts)│ │ (src/server.ts) │
|
|
389
|
+
│ OpenAI compat│ │ + web_fetch │ │ HTTP API + SSE │
|
|
390
|
+
│ (anthropic: │ │ (src/ │ │ runs in background│
|
|
391
|
+
│ provider- │ │ webfetch.ts)│ └───────────────────┘
|
|
392
|
+
│ anthropic) │ └──────────────┘
|
|
393
|
+
└──────────────┘
|
|
394
|
+
│
|
|
395
|
+
▼
|
|
331
396
|
┌─────────────────────────────────────────────┐
|
|
332
397
|
│ Build Checker (src/build.ts) │
|
|
333
398
|
│ run build → exit 0 = stop condition met │
|
|
@@ -337,7 +402,7 @@ scoder --no-save -y "run the full migration"
|
|
|
337
402
|
## Integration Testing
|
|
338
403
|
|
|
339
404
|
```bash
|
|
340
|
-
npm test # runs:
|
|
405
|
+
npm test # runs: build → unit tests (node --test) → e2e closed-loop → MCP smoke
|
|
341
406
|
npm run lint # eslint flat config (typescript-eslint + eqeqeq, prefer-const...)
|
|
342
407
|
npm run typecheck # tsc --noEmit
|
|
343
408
|
```
|
|
@@ -345,17 +410,18 @@ npm run typecheck # tsc --noEmit
|
|
|
345
410
|
Test suites (run with `npm test` — auto-discovers `test/*.test.mjs`):
|
|
346
411
|
- `test/permissions.test.mjs` — catastrophic patterns, mode classification, allowlist word-boundary, timeout-denies
|
|
347
412
|
- `test/permission-prompt.test.mjs` — interactive prompt timeout behavior
|
|
348
|
-
- `test/tools.test.mjs` — safePath traversal, symlink escape, file-size cap, write/edit/list
|
|
413
|
+
- `test/tools.test.mjs` — safePath traversal, symlink escape, file-size cap, write/edit/list, live `run_command` output streaming
|
|
414
|
+
- `test/webfetch.test.mjs` — HTML→text, JSON, redirects, SSRF guard, binary refusal, truncation
|
|
415
|
+
- `test/server.test.mjs` — HTTP API: run lifecycle via SSE, abort, auth, live `tool_output` events
|
|
349
416
|
- `test/core.test.mjs` — unified diff, retry backoff, context trimming
|
|
350
417
|
- `test/config.test.mjs` — env/rc/cli precedence, logFile mapping, permission lists
|
|
351
418
|
- `test/session.test.mjs` — session store, smart prune, resolve
|
|
419
|
+
- `test/cost.test.mjs` — token usage & cost aggregation
|
|
352
420
|
- `test/anthropic.test.mjs` — message translation + native SSE streaming
|
|
353
421
|
- `test/llm.test.mjs` — OpenAI-compatible chat + SSE streaming + redaction
|
|
354
422
|
- `test/run-e2e.mjs` — closed-loop integration (mock LLM server)
|
|
355
423
|
- `test/run-mcp-test.mjs` — MCP smoke (mock stdio server)
|
|
356
424
|
|
|
357
|
-
Coverage (lines): **90.3%** overall — see `npm run coverage`.
|
|
358
|
-
|
|
359
425
|
## Exit Codes
|
|
360
426
|
|
|
361
427
|
| Code | Meaning |
|
package/dist/agent.js
CHANGED
|
@@ -28,7 +28,13 @@ RULES:
|
|
|
28
28
|
6. Never invent requirements beyond the given task.
|
|
29
29
|
7. IMPORTANT: if a run_command is denied by permission policy, you will receive
|
|
30
30
|
an error message. Use this as a signal to find a safer alternative or to
|
|
31
|
-
communicate the limitation to the user
|
|
31
|
+
communicate the limitation to the user.
|
|
32
|
+
8. You have web access ONLY through web_fetch (it fetches exact URLs, it cannot
|
|
33
|
+
search). Use it when the task depends on external or fast-changing knowledge:
|
|
34
|
+
library/API documentation, a URL found in an error message or code comment,
|
|
35
|
+
current package versions, changelogs, release notes, standards. Skip it when
|
|
36
|
+
the answer is in the working directory or is stable general knowledge. After
|
|
37
|
+
fetching, ground your edits in what the page actually says.`;
|
|
32
38
|
export class Agent {
|
|
33
39
|
config;
|
|
34
40
|
client;
|
|
@@ -48,6 +54,10 @@ export class Agent {
|
|
|
48
54
|
mcp = null;
|
|
49
55
|
/** Aborted when abort() is called — cancels in-flight LLM requests (SIGINT). */
|
|
50
56
|
runSignal = new AbortController();
|
|
57
|
+
/** Optional structured-event sink (used by the HTTP/SSE server frontend). */
|
|
58
|
+
onEvent = null;
|
|
59
|
+
/** Set while a tool call is executing, for tool_output event correlation. */
|
|
60
|
+
currentToolCall = null;
|
|
51
61
|
constructor(config) {
|
|
52
62
|
this.config = config;
|
|
53
63
|
const commonOpts = {
|
|
@@ -72,6 +82,7 @@ export class Agent {
|
|
|
72
82
|
workdir: config.workdir,
|
|
73
83
|
permissions: this.permissions,
|
|
74
84
|
permissionMode: config.permissionMode,
|
|
85
|
+
onToolOutput: this.handleToolOutput,
|
|
75
86
|
};
|
|
76
87
|
this.session = new SessionStore(config.workdir, config.noSave !== true);
|
|
77
88
|
this.logger = JsonlLogger.forSession(config.workdir, this.session.id, config.logFile);
|
|
@@ -92,6 +103,27 @@ export class Agent {
|
|
|
92
103
|
// terminate its child process immediately (long builds must be interruptible).
|
|
93
104
|
this.ctx.signal = this.runSignal.signal;
|
|
94
105
|
}
|
|
106
|
+
/** Emit one structured run event to the sink (if attached). */
|
|
107
|
+
emit(ev) {
|
|
108
|
+
this.onEvent?.(ev);
|
|
109
|
+
}
|
|
110
|
+
/**
|
|
111
|
+
* Live tool-output handler: renders chunks in the terminal (dimmed block) and
|
|
112
|
+
* forwards them as `tool_output` events. Opens the stream block lazily on the
|
|
113
|
+
* first chunk; `endToolStream()` closes it after the tool completes.
|
|
114
|
+
*/
|
|
115
|
+
handleToolOutput = (name, stream, delta) => {
|
|
116
|
+
if (!ui.isToolStreamOpen()) {
|
|
117
|
+
ui.toolStreamStart();
|
|
118
|
+
}
|
|
119
|
+
ui.toolStreamChunk(delta);
|
|
120
|
+
this.emit({ type: "tool_output", name, stream, delta });
|
|
121
|
+
};
|
|
122
|
+
/** Close the live tool-output block (if open) before printing the next UI element. */
|
|
123
|
+
endToolStream() {
|
|
124
|
+
if (ui.isToolStreamOpen())
|
|
125
|
+
ui.toolStreamEnd();
|
|
126
|
+
}
|
|
95
127
|
/** Push one tool-result message back into the conversation. */
|
|
96
128
|
pushToolResult(tc, content) {
|
|
97
129
|
this.messages.push({
|
|
@@ -108,6 +140,7 @@ export class Agent {
|
|
|
108
140
|
*/
|
|
109
141
|
rejectToolCall(step, tc, output, logExtra) {
|
|
110
142
|
ui.toolResult(tc.name, false, output);
|
|
143
|
+
this.emit({ type: "tool_result", step, name: tc.name, ok: false, output });
|
|
111
144
|
this.logger.log({ type: "tool", step, tool: tc.name, ok: false, ...logExtra });
|
|
112
145
|
this.pushToolResult(tc, output);
|
|
113
146
|
}
|
|
@@ -126,6 +159,10 @@ export class Agent {
|
|
|
126
159
|
this.usageTotals.outputTokens += output;
|
|
127
160
|
this.usageTotals.cachedTokens += cached;
|
|
128
161
|
ui.info(`tokens: in=${usage.prompt_tokens ?? "?"} out=${usage.completion_tokens ?? "?"}`);
|
|
162
|
+
this.emit({
|
|
163
|
+
type: "llm_usage",
|
|
164
|
+
usage: { input_tokens: input, output_tokens: output, cached_tokens: cached },
|
|
165
|
+
});
|
|
129
166
|
this.logger.log({
|
|
130
167
|
type: "llm_usage",
|
|
131
168
|
model: this.client.model,
|
|
@@ -161,6 +198,7 @@ export class Agent {
|
|
|
161
198
|
started = true;
|
|
162
199
|
}
|
|
163
200
|
ui.llmStreamChunk(chunk.contentDelta);
|
|
201
|
+
this.emit({ type: "llm_delta", delta: chunk.contentDelta });
|
|
164
202
|
streamedText += chunk.contentDelta;
|
|
165
203
|
}
|
|
166
204
|
if (chunk.final)
|
|
@@ -217,9 +255,13 @@ export class Agent {
|
|
|
217
255
|
throw new Error("No API key configured. Set SCODER_API_KEY or OPENAI_API_KEY (or put apiKey in .scoderrc.json).");
|
|
218
256
|
}
|
|
219
257
|
const buildCmd = this.buildCmd;
|
|
220
|
-
ui.
|
|
221
|
-
|
|
222
|
-
|
|
258
|
+
ui.banner("scoder", [
|
|
259
|
+
["task", task],
|
|
260
|
+
["model", `${this.client.model} (${this.config.provider}${this.config.stream ? ", stream" : ""})`],
|
|
261
|
+
["workdir", this.config.workdir],
|
|
262
|
+
["build", buildCmd],
|
|
263
|
+
["steps", `max ${this.config.maxSteps} · permission ${this.config.permissionMode}`],
|
|
264
|
+
]);
|
|
223
265
|
this.currentTask = task;
|
|
224
266
|
this.resetRunSignal();
|
|
225
267
|
this.resetUsageTotals();
|
|
@@ -231,6 +273,14 @@ export class Agent {
|
|
|
231
273
|
},
|
|
232
274
|
{ role: "user", content: task },
|
|
233
275
|
];
|
|
276
|
+
this.emit({
|
|
277
|
+
type: "run_start",
|
|
278
|
+
task,
|
|
279
|
+
model: this.client.model,
|
|
280
|
+
provider: this.config.provider,
|
|
281
|
+
workdir: this.config.workdir,
|
|
282
|
+
buildCmd,
|
|
283
|
+
});
|
|
234
284
|
try {
|
|
235
285
|
await this.connectMcp();
|
|
236
286
|
return await this.runLoop(0, buildCmd, null);
|
|
@@ -304,6 +354,7 @@ export class Agent {
|
|
|
304
354
|
while (steps < this.config.maxSteps && !this.aborted) {
|
|
305
355
|
steps++;
|
|
306
356
|
ui.step(steps, this.config.maxSteps, "thinking...");
|
|
357
|
+
this.emit({ type: "step", step: steps, maxSteps: this.config.maxSteps });
|
|
307
358
|
const { message, usage } = await this.chatWithStream().catch((err) => {
|
|
308
359
|
// SIGINT during the LLM call — stop gracefully instead of crashing
|
|
309
360
|
if (this.aborted) {
|
|
@@ -363,6 +414,8 @@ export class Agent {
|
|
|
363
414
|
continue;
|
|
364
415
|
}
|
|
365
416
|
ui.toolCall(tc.name, args);
|
|
417
|
+
this.currentToolCall = { step: steps, id: tc.id, name: tc.name };
|
|
418
|
+
this.emit({ type: "tool_call", step: steps, id: tc.id, name: tc.name, args });
|
|
366
419
|
if (!tool && !(this.mcp?.hasTool(tc.name))) {
|
|
367
420
|
this.rejectToolCall(steps, tc, `ERROR: unknown tool "${tc.name}". Available: ${AGENT_TOOLS.map((t) => t.spec.function.name).join(", ")}${this.mcp ? " + MCP tools (mcp__<server>__<tool>)" : ""}`, { error: "unknown tool" });
|
|
368
421
|
continue;
|
|
@@ -426,7 +479,10 @@ export class Agent {
|
|
|
426
479
|
this.aborted = true;
|
|
427
480
|
ui.warn("edit confirmation aborted by user ([q]uit) — stopping the run");
|
|
428
481
|
}
|
|
482
|
+
this.endToolStream();
|
|
483
|
+
this.currentToolCall = null;
|
|
429
484
|
ui.toolResult(tc.name, result.ok, result.output);
|
|
485
|
+
this.emit({ type: "tool_result", step: steps, name: tc.name, ok: result.ok, output: result.output });
|
|
430
486
|
stepToolResults.set(tc.id, { ok: result.ok, output: result.output });
|
|
431
487
|
this.pushToolResult(tc, summariseOutput(result.output));
|
|
432
488
|
}
|
|
@@ -483,8 +539,10 @@ export class Agent {
|
|
|
483
539
|
}
|
|
484
540
|
/** Explicitly verify the build (independent of what the LLM claims). */
|
|
485
541
|
async checkBuild(buildCmd) {
|
|
486
|
-
const res = await runBuild(this.config.workdir, buildCmd);
|
|
542
|
+
const res = await runBuild(this.config.workdir, buildCmd, (stream, delta) => this.handleToolOutput("build", stream, delta));
|
|
543
|
+
this.endToolStream();
|
|
487
544
|
ui.buildStatus(res.success, res.output);
|
|
545
|
+
this.emit({ type: "build", success: res.success, command: buildCmd, output: res.output });
|
|
488
546
|
this.logger.log({ type: "build", success: res.success, command: buildCmd, output: res.output.slice(0, 10_000) });
|
|
489
547
|
return res;
|
|
490
548
|
}
|
|
@@ -495,6 +553,7 @@ export class Agent {
|
|
|
495
553
|
else {
|
|
496
554
|
ui.error(`Task NOT complete. ${summary}`);
|
|
497
555
|
}
|
|
556
|
+
this.emit({ type: "finish", success, steps, summary: summary.slice(0, 2000) });
|
|
498
557
|
this.logger.log({ type: "finish", success, steps });
|
|
499
558
|
return { success, steps, buildCmd, summary: summary.slice(0, 2000) };
|
|
500
559
|
}
|
package/dist/build.js
CHANGED
|
@@ -30,9 +30,10 @@ export function detectBuildCommand(workdir) {
|
|
|
30
30
|
/**
|
|
31
31
|
* Run the build command. This is the agent's stop condition:
|
|
32
32
|
* exit code 0 == task complete.
|
|
33
|
+
* `onOutput` (optional) receives stdout/stderr chunks live while the build runs.
|
|
33
34
|
*/
|
|
34
|
-
export async function runBuild(workdir, command) {
|
|
35
|
-
const res = await toolRunCommand(workdir, { command, timeout_secs: 600 });
|
|
35
|
+
export async function runBuild(workdir, command, onOutput) {
|
|
36
|
+
const res = await toolRunCommand(workdir, { command, timeout_secs: 600 }, undefined, onOutput);
|
|
36
37
|
return {
|
|
37
38
|
success: res.ok,
|
|
38
39
|
command,
|
package/dist/index.js
CHANGED
|
@@ -21,6 +21,9 @@ USAGE
|
|
|
21
21
|
scoder logs [id|last] [filter] Print a session's JSONL log (requires --log-file runs)
|
|
22
22
|
scoder cost [--session <id>] [--last <n>]
|
|
23
23
|
Token usage & estimated cost from JSONL logs
|
|
24
|
+
scoder serve [--port <n>] [--host <addr>]
|
|
25
|
+
Run as an HTTP server: POST tasks, stream
|
|
26
|
+
progress via Server-Sent Events (see below)
|
|
24
27
|
scoder --version | --help
|
|
25
28
|
|
|
26
29
|
OPTIONS
|
|
@@ -120,6 +123,17 @@ function parseArgs(argv) {
|
|
|
120
123
|
args.provider = p;
|
|
121
124
|
break;
|
|
122
125
|
}
|
|
126
|
+
case "--port": {
|
|
127
|
+
const n = Number(next());
|
|
128
|
+
if (!Number.isInteger(n) || n < 0 || n > 65535) {
|
|
129
|
+
throw new Error(`Invalid --port value (expected 0-65535): ${n}`);
|
|
130
|
+
}
|
|
131
|
+
args.servePort = n;
|
|
132
|
+
break;
|
|
133
|
+
}
|
|
134
|
+
case "--host":
|
|
135
|
+
args.serveHost = next();
|
|
136
|
+
break;
|
|
123
137
|
case "-h":
|
|
124
138
|
case "--help":
|
|
125
139
|
args.help = true;
|
|
@@ -155,6 +169,10 @@ function parseArgs(argv) {
|
|
|
155
169
|
args.subcommand = "cost";
|
|
156
170
|
args.task = null;
|
|
157
171
|
}
|
|
172
|
+
else if (rest[0] === "serve") {
|
|
173
|
+
args.subcommand = "serve";
|
|
174
|
+
args.task = null;
|
|
175
|
+
}
|
|
158
176
|
return args;
|
|
159
177
|
}
|
|
160
178
|
async function runOnce(cfg, task) {
|
|
@@ -302,6 +320,32 @@ async function main() {
|
|
|
302
320
|
console.log(formatCostReport(sessionCosts));
|
|
303
321
|
return;
|
|
304
322
|
}
|
|
323
|
+
if (args.subcommand === "serve") {
|
|
324
|
+
const { startServer } = await import("./server.js");
|
|
325
|
+
if (!cfg.apiKey) {
|
|
326
|
+
ui.warn("No API key configured — runs will fail until SCODER_API_KEY (or OPENAI_API_KEY) is set.");
|
|
327
|
+
}
|
|
328
|
+
const srv = await startServer(cfg, { host: args.serveHost, port: args.servePort });
|
|
329
|
+
const token = process.env.SCODER_SERVER_TOKEN ? " · token auth" : "";
|
|
330
|
+
ui.banner(`scoder serve @ http://${args.serveHost ?? "127.0.0.1"}:${srv.port}${token}`, [
|
|
331
|
+
["task", `POST /api/runs { "task": "..." } → 202 { id, eventsUrl }`],
|
|
332
|
+
["events", `GET /api/runs/<id>/events → SSE stream`],
|
|
333
|
+
["list", `GET /api/runs`],
|
|
334
|
+
["abort", `POST /api/runs/<id>/abort`],
|
|
335
|
+
["workdir", cfg.workdir],
|
|
336
|
+
["model", `${cfg.model} · permission ${cfg.permissionMode}`],
|
|
337
|
+
]);
|
|
338
|
+
const shutdown = async () => {
|
|
339
|
+
ui.info("shutting down server...");
|
|
340
|
+
await srv.close();
|
|
341
|
+
process.exit(0);
|
|
342
|
+
};
|
|
343
|
+
process.on("SIGINT", () => {
|
|
344
|
+
ui.warn("\nSIGINT — shutting down server, aborting in-flight runs...");
|
|
345
|
+
void shutdown();
|
|
346
|
+
});
|
|
347
|
+
return;
|
|
348
|
+
}
|
|
305
349
|
if (args.task) {
|
|
306
350
|
const code = await runOnce(cfg, args.task);
|
|
307
351
|
process.exit(code);
|
package/dist/server.js
ADDED
|
@@ -0,0 +1,256 @@
|
|
|
1
|
+
import { createServer } from "node:http";
|
|
2
|
+
import { randomUUID } from "node:crypto";
|
|
3
|
+
import { existsSync, statSync } from "node:fs";
|
|
4
|
+
import { resolve } from "node:path";
|
|
5
|
+
import process from "node:process";
|
|
6
|
+
import { Agent } from "./agent.js";
|
|
7
|
+
const MAX_BODY_BYTES = 1024 * 1024;
|
|
8
|
+
function readBody(req, maxBytes = MAX_BODY_BYTES) {
|
|
9
|
+
return new Promise((resolve, reject) => {
|
|
10
|
+
let body = "";
|
|
11
|
+
let size = 0;
|
|
12
|
+
req.on("data", (c) => {
|
|
13
|
+
size += c.byteLength;
|
|
14
|
+
if (size > maxBytes) {
|
|
15
|
+
reject(new Error(`request body too large (>${maxBytes} bytes)`));
|
|
16
|
+
req.destroy();
|
|
17
|
+
return;
|
|
18
|
+
}
|
|
19
|
+
body += c.toString("utf8");
|
|
20
|
+
});
|
|
21
|
+
req.on("end", () => resolve(body));
|
|
22
|
+
req.on("error", reject);
|
|
23
|
+
});
|
|
24
|
+
}
|
|
25
|
+
function sendJson(res, status, payload) {
|
|
26
|
+
res.writeHead(status, { "Content-Type": "application/json; charset=utf-8" });
|
|
27
|
+
res.end(JSON.stringify(payload));
|
|
28
|
+
}
|
|
29
|
+
function isAuthorized(req, token) {
|
|
30
|
+
if (!token)
|
|
31
|
+
return true;
|
|
32
|
+
const header = req.headers.authorization ?? "";
|
|
33
|
+
return header === `Bearer ${token}`;
|
|
34
|
+
}
|
|
35
|
+
export function startServer(baseConfig, opts = {}) {
|
|
36
|
+
const host = opts.host ?? "127.0.0.1";
|
|
37
|
+
const maxConcurrent = opts.maxConcurrent ?? 4;
|
|
38
|
+
const token = process.env.SCODER_SERVER_TOKEN || "";
|
|
39
|
+
/** id -> run record; Map preserves insertion order (= creation order). */
|
|
40
|
+
const runs = new Map();
|
|
41
|
+
const publicRun = (r) => ({
|
|
42
|
+
id: r.id,
|
|
43
|
+
task: r.task,
|
|
44
|
+
workdir: r.workdir,
|
|
45
|
+
status: r.status,
|
|
46
|
+
createdAt: r.createdAt,
|
|
47
|
+
finishedAt: r.finishedAt,
|
|
48
|
+
eventCount: r.events.length,
|
|
49
|
+
result: r.result,
|
|
50
|
+
error: r.error,
|
|
51
|
+
eventsUrl: `/api/runs/${r.id}/events`,
|
|
52
|
+
});
|
|
53
|
+
const runningCount = () => [...runs.values()].filter((r) => r.status === "running").length;
|
|
54
|
+
/** Launch an agent run in the background, wiring events into the record. */
|
|
55
|
+
const launchRun = (record, cfg, task) => {
|
|
56
|
+
const agent = new Agent(cfg);
|
|
57
|
+
record.agent = agent;
|
|
58
|
+
agent.onEvent = (ev) => {
|
|
59
|
+
record.events.push(ev);
|
|
60
|
+
if (ev.type === "finish") {
|
|
61
|
+
record.status = ev.success ? "success" : record.abortRequested ? "aborted" : "failed";
|
|
62
|
+
record.finishedAt = new Date().toISOString();
|
|
63
|
+
}
|
|
64
|
+
for (const sub of record.subscribers)
|
|
65
|
+
sub();
|
|
66
|
+
};
|
|
67
|
+
agent
|
|
68
|
+
.run(task)
|
|
69
|
+
.then((result) => {
|
|
70
|
+
record.result = result;
|
|
71
|
+
if (record.status === "running") {
|
|
72
|
+
// Should not happen (finish event always arrives first), but be safe.
|
|
73
|
+
record.status = result.success ? "success" : record.abortRequested ? "aborted" : "failed";
|
|
74
|
+
record.finishedAt = new Date().toISOString();
|
|
75
|
+
}
|
|
76
|
+
})
|
|
77
|
+
.catch((err) => {
|
|
78
|
+
record.error = err.message;
|
|
79
|
+
// Synthetic terminal events so SSE clients always see a stream end.
|
|
80
|
+
record.events.push({ type: "error", message: record.error });
|
|
81
|
+
record.events.push({ type: "finish", success: false, steps: 0, summary: `run failed: ${record.error}` });
|
|
82
|
+
record.status = "error";
|
|
83
|
+
record.finishedAt = new Date().toISOString();
|
|
84
|
+
for (const sub of record.subscribers)
|
|
85
|
+
sub();
|
|
86
|
+
});
|
|
87
|
+
};
|
|
88
|
+
const handleEventsSSE = (req, res, record) => {
|
|
89
|
+
res.writeHead(200, {
|
|
90
|
+
"Content-Type": "text/event-stream",
|
|
91
|
+
"Cache-Control": "no-cache",
|
|
92
|
+
Connection: "keep-alive",
|
|
93
|
+
});
|
|
94
|
+
let sent = 0;
|
|
95
|
+
const heartbeat = setInterval(() => {
|
|
96
|
+
if (!res.writableEnded)
|
|
97
|
+
res.write(": ping\n\n");
|
|
98
|
+
}, 15_000);
|
|
99
|
+
heartbeat.unref?.();
|
|
100
|
+
/** Replay buffered events + live ones; ends the stream on a terminal event. */
|
|
101
|
+
const flush = () => {
|
|
102
|
+
while (sent < record.events.length) {
|
|
103
|
+
const ev = record.events[sent++];
|
|
104
|
+
res.write(`data: ${JSON.stringify(ev)}\n\n`);
|
|
105
|
+
if (ev.type === "finish" || ev.type === "error") {
|
|
106
|
+
res.end();
|
|
107
|
+
record.subscribers.delete(flush);
|
|
108
|
+
return;
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
};
|
|
112
|
+
record.subscribers.add(flush);
|
|
113
|
+
flush(); // replays the buffer, then stays live via the subscription
|
|
114
|
+
req.on("close", () => {
|
|
115
|
+
record.subscribers.delete(flush);
|
|
116
|
+
clearInterval(heartbeat);
|
|
117
|
+
});
|
|
118
|
+
};
|
|
119
|
+
const handle = async (req, res) => {
|
|
120
|
+
const url = new URL(req.url ?? "/", `http://${req.headers.host ?? "localhost"}`);
|
|
121
|
+
const path = url.pathname;
|
|
122
|
+
if (!isAuthorized(req, token)) {
|
|
123
|
+
sendJson(res, 401, { error: "unauthorized (set Authorization: Bearer <SCODER_SERVER_TOKEN>)" });
|
|
124
|
+
return;
|
|
125
|
+
}
|
|
126
|
+
// ── GET /api/health ──
|
|
127
|
+
if (path === "/api/health" && req.method === "GET") {
|
|
128
|
+
sendJson(res, 200, { ok: true, running: runningCount(), runs: runs.size });
|
|
129
|
+
return;
|
|
130
|
+
}
|
|
131
|
+
// ── /api/runs ──
|
|
132
|
+
if (path === "/api/runs") {
|
|
133
|
+
if (req.method === "GET") {
|
|
134
|
+
sendJson(res, 200, { runs: [...runs.values()].map(publicRun) });
|
|
135
|
+
return;
|
|
136
|
+
}
|
|
137
|
+
if (req.method === "POST") {
|
|
138
|
+
if (runningCount() >= maxConcurrent) {
|
|
139
|
+
sendJson(res, 429, { error: `too many concurrent runs (max ${maxConcurrent}) — retry later` });
|
|
140
|
+
return;
|
|
141
|
+
}
|
|
142
|
+
let body;
|
|
143
|
+
try {
|
|
144
|
+
body = JSON.parse(await readBody(req));
|
|
145
|
+
}
|
|
146
|
+
catch (err) {
|
|
147
|
+
sendJson(res, 400, { error: err.message });
|
|
148
|
+
return;
|
|
149
|
+
}
|
|
150
|
+
if (typeof body.task !== "string" || body.task.trim().length === 0) {
|
|
151
|
+
sendJson(res, 400, { error: 'field "task" (non-empty string) is required' });
|
|
152
|
+
return;
|
|
153
|
+
}
|
|
154
|
+
const cfg = { ...baseConfig, yes: true };
|
|
155
|
+
if (body.workdir) {
|
|
156
|
+
const abs = resolve(body.workdir);
|
|
157
|
+
if (!existsSync(abs) || !statSync(abs).isDirectory()) {
|
|
158
|
+
sendJson(res, 400, { error: `workdir does not exist or is not a directory: ${body.workdir}` });
|
|
159
|
+
return;
|
|
160
|
+
}
|
|
161
|
+
cfg.workdir = abs;
|
|
162
|
+
}
|
|
163
|
+
const permission = body.permission ?? cfg.permissionMode;
|
|
164
|
+
if (permission === "ask") {
|
|
165
|
+
sendJson(res, 400, { error: 'permission "ask" is not supported by the server (it cannot prompt). Use "auto" or "readonly".' });
|
|
166
|
+
return;
|
|
167
|
+
}
|
|
168
|
+
cfg.permissionMode = permission;
|
|
169
|
+
if (body.buildCmd)
|
|
170
|
+
cfg.buildCmd = body.buildCmd;
|
|
171
|
+
if (body.model)
|
|
172
|
+
cfg.model = body.model;
|
|
173
|
+
if (body.provider)
|
|
174
|
+
cfg.provider = body.provider;
|
|
175
|
+
if (body.maxSteps)
|
|
176
|
+
cfg.maxSteps = Math.min(Math.max(body.maxSteps, 1), 500);
|
|
177
|
+
const record = {
|
|
178
|
+
id: randomUUID(),
|
|
179
|
+
task: body.task,
|
|
180
|
+
workdir: cfg.workdir,
|
|
181
|
+
status: "running",
|
|
182
|
+
createdAt: new Date().toISOString(),
|
|
183
|
+
finishedAt: null,
|
|
184
|
+
events: [],
|
|
185
|
+
subscribers: new Set(),
|
|
186
|
+
agent: null,
|
|
187
|
+
abortRequested: false,
|
|
188
|
+
result: null,
|
|
189
|
+
error: null,
|
|
190
|
+
};
|
|
191
|
+
runs.set(record.id, record);
|
|
192
|
+
launchRun(record, cfg, body.task);
|
|
193
|
+
sendJson(res, 202, publicRun(record));
|
|
194
|
+
return;
|
|
195
|
+
}
|
|
196
|
+
sendJson(res, 405, { error: "method not allowed" });
|
|
197
|
+
return;
|
|
198
|
+
}
|
|
199
|
+
// ── /api/runs/:id[...] ──
|
|
200
|
+
const runMatch = path.match(/^\/api\/runs\/([0-9a-f-]{36})(\/events|\/abort)?$/i);
|
|
201
|
+
if (runMatch) {
|
|
202
|
+
const record = runs.get(runMatch[1]);
|
|
203
|
+
if (!record) {
|
|
204
|
+
sendJson(res, 404, { error: `unknown run id: ${runMatch[1]}` });
|
|
205
|
+
return;
|
|
206
|
+
}
|
|
207
|
+
if (runMatch[2] === "/events" && req.method === "GET") {
|
|
208
|
+
handleEventsSSE(req, res, record);
|
|
209
|
+
return;
|
|
210
|
+
}
|
|
211
|
+
if (runMatch[2] === "/abort" && req.method === "POST") {
|
|
212
|
+
if (record.status !== "running" || !record.agent) {
|
|
213
|
+
sendJson(res, 409, { error: `run is not running (status: ${record.status})` });
|
|
214
|
+
return;
|
|
215
|
+
}
|
|
216
|
+
record.abortRequested = true;
|
|
217
|
+
record.agent.abort();
|
|
218
|
+
sendJson(res, 200, { ok: true, status: "running (aborting...)" });
|
|
219
|
+
return;
|
|
220
|
+
}
|
|
221
|
+
if (!runMatch[2] && req.method === "GET") {
|
|
222
|
+
sendJson(res, 200, publicRun(record));
|
|
223
|
+
return;
|
|
224
|
+
}
|
|
225
|
+
}
|
|
226
|
+
sendJson(res, 404, { error: `not found: ${req.method} ${path}` });
|
|
227
|
+
};
|
|
228
|
+
const server = createServer((req, res) => {
|
|
229
|
+
void handle(req, res).catch(() => {
|
|
230
|
+
if (!res.writableEnded)
|
|
231
|
+
sendJson(res, 500, { error: "internal server error" });
|
|
232
|
+
});
|
|
233
|
+
});
|
|
234
|
+
return new Promise((resolve, reject) => {
|
|
235
|
+
server.once("error", reject);
|
|
236
|
+
server.listen(opts.port ?? 0, host, () => {
|
|
237
|
+
const address = server.address();
|
|
238
|
+
const port = typeof address === "object" && address !== null ? address.port : 0;
|
|
239
|
+
resolve({
|
|
240
|
+
server,
|
|
241
|
+
port,
|
|
242
|
+
close: () => new Promise((res) => {
|
|
243
|
+
// Abort in-flight runs so background work does not keep the loop alive.
|
|
244
|
+
for (const r of runs.values()) {
|
|
245
|
+
if (r.status === "running") {
|
|
246
|
+
r.abortRequested = true;
|
|
247
|
+
r.agent?.abort();
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
server.close(() => res());
|
|
251
|
+
server.closeAllConnections?.();
|
|
252
|
+
}),
|
|
253
|
+
});
|
|
254
|
+
});
|
|
255
|
+
});
|
|
256
|
+
}
|
package/dist/tools.js
CHANGED
|
@@ -3,6 +3,7 @@ import { spawn } from "node:child_process";
|
|
|
3
3
|
import { resolve, dirname, basename, join, relative, isAbsolute } from "node:path";
|
|
4
4
|
import { unifiedDiff } from "./diff.js";
|
|
5
5
|
import * as ui from "./ui.js";
|
|
6
|
+
import { toolWebFetch } from "./webfetch.js";
|
|
6
7
|
const MAX_OUTPUT_CHARS = 20_000;
|
|
7
8
|
const MAX_FILE_SIZE_BYTES = 2 * 1024 * 1024; // refuse to read files > 2MB
|
|
8
9
|
function clamp(s, max = MAX_OUTPUT_CHARS) {
|
|
@@ -211,8 +212,10 @@ function toolListFiles(workdir, args) {
|
|
|
211
212
|
* Run a shell command asynchronously (async spawn keeps the event loop free,
|
|
212
213
|
* so streaming output renders and SIGINT is handled promptly).
|
|
213
214
|
* Kills the child on timeout or external abort (SIGINT via ctx.signal).
|
|
215
|
+
* When `onOutput` is provided, stdout/stderr chunks are forwarded live as they
|
|
216
|
+
* arrive (in addition to being collected into the final ToolResult).
|
|
214
217
|
*/
|
|
215
|
-
export async function toolRunCommand(workdir, args, signal) {
|
|
218
|
+
export async function toolRunCommand(workdir, args, signal, onOutput) {
|
|
216
219
|
const timeoutMs = Math.min(Math.max(args.timeout_secs ?? 120, 1), 600) * 1000;
|
|
217
220
|
return new Promise((resolve) => {
|
|
218
221
|
let settled = false;
|
|
@@ -252,8 +255,16 @@ export async function toolRunCommand(workdir, args, signal) {
|
|
|
252
255
|
onAbort();
|
|
253
256
|
else
|
|
254
257
|
signal?.addEventListener("abort", onAbort, { once: true });
|
|
255
|
-
child.stdout?.on("data", (c) => {
|
|
256
|
-
|
|
258
|
+
child.stdout?.on("data", (c) => {
|
|
259
|
+
const s = c.toString("utf8");
|
|
260
|
+
stdout += s;
|
|
261
|
+
onOutput?.("stdout", s);
|
|
262
|
+
});
|
|
263
|
+
child.stderr?.on("data", (c) => {
|
|
264
|
+
const s = c.toString("utf8");
|
|
265
|
+
stderr += s;
|
|
266
|
+
onOutput?.("stderr", s);
|
|
267
|
+
});
|
|
257
268
|
child.on("error", (err) => {
|
|
258
269
|
finish({ ok: false, output: clamp(`$ ${args.command}\n(spawn error) ${err.message}`) });
|
|
259
270
|
});
|
|
@@ -353,7 +364,28 @@ export const AGENT_TOOLS = [
|
|
|
353
364
|
}, ["command"]),
|
|
354
365
|
},
|
|
355
366
|
},
|
|
356
|
-
run: (ctx, args) => toolRunCommand(ctx.workdir, args, ctx.signal),
|
|
367
|
+
run: (ctx, args) => toolRunCommand(ctx.workdir, args, ctx.signal, ctx.onToolOutput ? (stream, delta) => ctx.onToolOutput?.("run_command", stream, delta) : undefined),
|
|
368
|
+
},
|
|
369
|
+
{
|
|
370
|
+
spec: {
|
|
371
|
+
type: "function",
|
|
372
|
+
function: {
|
|
373
|
+
name: "web_fetch",
|
|
374
|
+
description: "Fetch a public HTTP(S) URL and return its content as readable plain text " +
|
|
375
|
+
"(HTML converted to text, JSON pretty-printed). WHEN TO USE: the task depends on external, " +
|
|
376
|
+
"possibly outdated knowledge — library/API documentation, an error message or comment " +
|
|
377
|
+
"containing a URL, current package versions or changelogs, standards/specs. " +
|
|
378
|
+
"WHEN NOT TO: the answer is already in the working directory or in the user's request; " +
|
|
379
|
+
"you need a search engine (this tool fetches exact URLs only, it cannot search); " +
|
|
380
|
+
"the content is a binary file (images, archives, PDFs are refused). " +
|
|
381
|
+
"Private/internal addresses (localhost, 10.x, 192.168.x, 127.x) are blocked.",
|
|
382
|
+
parameters: obj({
|
|
383
|
+
url: str("Absolute http(s) URL to fetch"),
|
|
384
|
+
max_chars: str("Max characters of extracted content to return (default 20000, max 100000)"),
|
|
385
|
+
}, ["url"]),
|
|
386
|
+
},
|
|
387
|
+
},
|
|
388
|
+
run: (ctx, args) => toolWebFetch(ctx, args),
|
|
357
389
|
},
|
|
358
390
|
];
|
|
359
391
|
export function getTool(name) {
|
package/dist/ui.js
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
import process from "node:process";
|
|
2
|
-
|
|
3
|
-
const
|
|
4
|
-
const
|
|
5
|
-
const
|
|
6
|
-
const
|
|
7
|
-
const
|
|
8
|
-
const
|
|
9
|
-
const
|
|
2
|
+
// Honor NO_COLOR and non-TTY output: drop ANSI codes so logs/CI stay clean.
|
|
3
|
+
const COLOR = process.env.NO_COLOR === undefined && process.stdout.isTTY === true;
|
|
4
|
+
const RESET = COLOR ? "\x1b[0m" : "";
|
|
5
|
+
const BOLD = COLOR ? "\x1b[1m" : "";
|
|
6
|
+
const DIM = COLOR ? "\x1b[2m" : "";
|
|
7
|
+
const GREEN = COLOR ? "\x1b[32m" : "";
|
|
8
|
+
const YELLOW = COLOR ? "\x1b[33m" : "";
|
|
9
|
+
const RED = COLOR ? "\x1b[31m" : "";
|
|
10
|
+
const CYAN = COLOR ? "\x1b[36m" : "";
|
|
11
|
+
const BLUE = COLOR ? "\x1b[34m" : "";
|
|
10
12
|
function ts() {
|
|
11
13
|
return new Date().toISOString().slice(11, 23);
|
|
12
14
|
}
|
|
@@ -14,7 +16,7 @@ export function info(msg) {
|
|
|
14
16
|
console.log(`${DIM}[${ts()}]${RESET} ${msg}`);
|
|
15
17
|
}
|
|
16
18
|
export function step(stepNum, total, msg) {
|
|
17
|
-
console.log(`${BLUE}[${stepNum}/${total}]${RESET} ${BOLD}${msg}${RESET}`);
|
|
19
|
+
console.log(`${BLUE}[${String(stepNum).padStart(2)}/${total}]${RESET} ${BOLD}▸ ${msg}${RESET}`);
|
|
18
20
|
}
|
|
19
21
|
export function success(msg) {
|
|
20
22
|
console.log(`${GREEN}✓${RESET} ${GREEN}${msg}${RESET}`);
|
|
@@ -37,7 +39,7 @@ export function toolCall(name, args) {
|
|
|
37
39
|
export function toolResult(name, ok, output) {
|
|
38
40
|
const preview = output.split("\n")[0]?.slice(0, 120) ?? "";
|
|
39
41
|
const icon = ok ? `${GREEN}✓${RESET}` : `${RED}✗${RESET}`;
|
|
40
|
-
console.log(`${icon} ${CYAN}${name}${RESET}
|
|
42
|
+
console.log(`${icon} ${CYAN}${name}${RESET} ${DIM}→${RESET} ${preview}`);
|
|
41
43
|
}
|
|
42
44
|
export function buildStatus(success, output) {
|
|
43
45
|
const icon = success ? `${GREEN}✓ BUILD SUCCESS` : `${RED}✗ BUILD FAILED`;
|
|
@@ -46,6 +48,25 @@ export function buildStatus(success, output) {
|
|
|
46
48
|
export function header(msg) {
|
|
47
49
|
console.log(`\n${BOLD}${CYAN}── ${msg} ${"─".repeat(Math.max(2, 60 - msg.length))}${RESET}`);
|
|
48
50
|
}
|
|
51
|
+
/** Startup banner: a clean key/value box (plain-text fallback when piped). */
|
|
52
|
+
export function banner(title, rows) {
|
|
53
|
+
const keyWidth = Math.max(8, ...rows.map(([k]) => k.length));
|
|
54
|
+
const inner = Math.max(title.length + 4, ...rows.map(([, v]) => keyWidth + Math.max(...String(v).split("\n").map((l) => l.length)) + 1));
|
|
55
|
+
console.log(`${BOLD}${CYAN}╭─ ${title} ${"─".repeat(Math.max(1, inner - title.length - 3))}╮${RESET}`);
|
|
56
|
+
for (const [key, value] of rows) {
|
|
57
|
+
const lines = String(value).split("\n");
|
|
58
|
+
for (let i = 0; i < lines.length; i++) {
|
|
59
|
+
const k = (i === 0 ? key : "").padEnd(keyWidth);
|
|
60
|
+
let v = lines[i];
|
|
61
|
+
const maxV = inner - keyWidth - 1;
|
|
62
|
+
if (v.length > maxV)
|
|
63
|
+
v = v.slice(0, maxV - 1) + "…";
|
|
64
|
+
const pad = Math.max(0, inner - 1 - keyWidth - v.length);
|
|
65
|
+
console.log(`${BOLD}${CYAN}│${RESET} ${DIM}${k}${RESET}${v}${" ".repeat(pad)}${BOLD}${CYAN}│${RESET}`);
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
console.log(`${BOLD}${CYAN}╰${"─".repeat(inner)}╯${RESET}`);
|
|
69
|
+
}
|
|
49
70
|
// ── Streaming helpers ────────────────────────────────────────────────────────
|
|
50
71
|
let _streamLine = "";
|
|
51
72
|
let _streamColor = CYAN;
|
|
@@ -69,6 +90,51 @@ export function llmStreamEnd(suffix = "") {
|
|
|
69
90
|
process.stdout.write(`${suffix}\n`);
|
|
70
91
|
return line;
|
|
71
92
|
}
|
|
93
|
+
// ── Tool output streaming (live run_command output) ─────────────────────────
|
|
94
|
+
const TOOL_STREAM_MAX_CHARS = 20_000;
|
|
95
|
+
let _toolStreamOpen = false;
|
|
96
|
+
let _toolStreamLineOpen = false; // last chunk did not end with "\n"
|
|
97
|
+
let _toolStreamChars = 0;
|
|
98
|
+
/** Begin a live tool-output block. Output is printed dimmed, capped at 20k chars. */
|
|
99
|
+
export function toolStreamStart() {
|
|
100
|
+
if (_toolStreamOpen)
|
|
101
|
+
return;
|
|
102
|
+
_toolStreamOpen = true;
|
|
103
|
+
_toolStreamLineOpen = false;
|
|
104
|
+
_toolStreamChars = 0;
|
|
105
|
+
process.stdout.write(`${DIM}┌ output${RESET}`);
|
|
106
|
+
}
|
|
107
|
+
/** Append a tool-output delta to the live block. */
|
|
108
|
+
export function toolStreamChunk(delta) {
|
|
109
|
+
if (!_toolStreamOpen)
|
|
110
|
+
return;
|
|
111
|
+
if (_toolStreamChars >= TOOL_STREAM_MAX_CHARS)
|
|
112
|
+
return;
|
|
113
|
+
_toolStreamChars += delta.length;
|
|
114
|
+
if (_toolStreamChars >= TOOL_STREAM_MAX_CHARS) {
|
|
115
|
+
// Include the tail of this chunk up to the cap, then stop rendering live.
|
|
116
|
+
const keep = delta.slice(0, Math.max(0, delta.length - (_toolStreamChars - TOOL_STREAM_MAX_CHARS)));
|
|
117
|
+
if (keep.length > 0) {
|
|
118
|
+
process.stdout.write(`${DIM}${keep}${RESET}`);
|
|
119
|
+
_toolStreamLineOpen = !keep.endsWith("\n");
|
|
120
|
+
}
|
|
121
|
+
process.stdout.write(`\n${DIM}│ (live output truncated at ${TOOL_STREAM_MAX_CHARS} chars — full output in the result)${RESET}`);
|
|
122
|
+
return;
|
|
123
|
+
}
|
|
124
|
+
process.stdout.write(`\n${DIM}│ ${delta.replace(/\n/g, `\n${DIM}│ ${RESET}`)}${RESET}`);
|
|
125
|
+
_toolStreamLineOpen = !delta.endsWith("\n");
|
|
126
|
+
}
|
|
127
|
+
/** Finish the live tool-output block (closes the partial last line). */
|
|
128
|
+
export function toolStreamEnd() {
|
|
129
|
+
if (!_toolStreamOpen)
|
|
130
|
+
return;
|
|
131
|
+
_toolStreamOpen = false;
|
|
132
|
+
process.stdout.write(_toolStreamLineOpen ? `\n${DIM}└${RESET}\n` : `${DIM}└${RESET}\n`);
|
|
133
|
+
}
|
|
134
|
+
/** True while a live tool-output block is open. */
|
|
135
|
+
export function isToolStreamOpen() {
|
|
136
|
+
return _toolStreamOpen;
|
|
137
|
+
}
|
|
72
138
|
// ── Diff rendering ───────────────────────────────────────────────────────────
|
|
73
139
|
/** Print a unified diff with colors (+ green, - red, @@ cyan). */
|
|
74
140
|
export function printDiff(diffText) {
|
package/dist/webfetch.js
ADDED
|
@@ -0,0 +1,251 @@
|
|
|
1
|
+
import { isIP } from "node:net";
|
|
2
|
+
import { lookup } from "node:dns/promises";
|
|
3
|
+
/**
|
|
4
|
+
* web_fetch — read-only HTTP(S) fetch with SSRF protection.
|
|
5
|
+
*
|
|
6
|
+
* - Only http/https; follows up to 5 redirects, validating every hop.
|
|
7
|
+
* - Private/loopback/link-local addresses are blocked by default
|
|
8
|
+
* (env SCODER_WEB_ALLOW_PRIVATE=1 lifts the block for local dev/testing).
|
|
9
|
+
* - HTML is converted to plain text; JSON is pretty-printed; binary is refused.
|
|
10
|
+
* - Output is capped so a huge page cannot flood the LLM context.
|
|
11
|
+
*/
|
|
12
|
+
const FETCH_TIMEOUT_MS = 30_000;
|
|
13
|
+
const MAX_BODY_BYTES = 512 * 1024;
|
|
14
|
+
const MAX_REDIRECTS = 5;
|
|
15
|
+
const DEFAULT_MAX_CHARS = 20_000;
|
|
16
|
+
function allowPrivate() {
|
|
17
|
+
return process.env.SCODER_WEB_ALLOW_PRIVATE === "1";
|
|
18
|
+
}
|
|
19
|
+
/** True if the address is private/loopback/link-local (IPv4 + IPv6). */
|
|
20
|
+
function isPrivateAddress(ip) {
|
|
21
|
+
const v = isIP(ip);
|
|
22
|
+
if (v === 4) {
|
|
23
|
+
const parts = ip.split(".").map(Number);
|
|
24
|
+
const a = parts[0];
|
|
25
|
+
const b = parts[1];
|
|
26
|
+
if (a === 10 || a === 127 || a === 0)
|
|
27
|
+
return true;
|
|
28
|
+
if (a === 172 && b >= 16 && b <= 31)
|
|
29
|
+
return true;
|
|
30
|
+
if (a === 192 && b === 168)
|
|
31
|
+
return true;
|
|
32
|
+
if (a === 169 && b === 254)
|
|
33
|
+
return true; // link-local
|
|
34
|
+
if (a === 100 && b >= 64 && b <= 127)
|
|
35
|
+
return true; // CGNAT
|
|
36
|
+
return false;
|
|
37
|
+
}
|
|
38
|
+
if (v === 6) {
|
|
39
|
+
const lower = ip.toLowerCase();
|
|
40
|
+
if (lower === "::1" || lower === "::")
|
|
41
|
+
return true;
|
|
42
|
+
if (lower.startsWith("fe80"))
|
|
43
|
+
return true; // link-local
|
|
44
|
+
if (/^f[cd][0-9a-f]{2}:/.test(lower))
|
|
45
|
+
return true; // unique local fc00::/7
|
|
46
|
+
// IPv4-mapped ::ffff:10.0.0.1
|
|
47
|
+
const mapped = lower.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/);
|
|
48
|
+
if (mapped)
|
|
49
|
+
return isPrivateAddress(mapped[1]);
|
|
50
|
+
return false;
|
|
51
|
+
}
|
|
52
|
+
return true; // not parseable -> treat as private (fail closed)
|
|
53
|
+
}
|
|
54
|
+
/** Resolve hostname and refuse private addresses (unless explicitly allowed). */
|
|
55
|
+
async function assertPublicHost(hostname) {
|
|
56
|
+
if (allowPrivate())
|
|
57
|
+
return;
|
|
58
|
+
let addresses;
|
|
59
|
+
try {
|
|
60
|
+
addresses = await lookup(hostname, { all: true, verbatim: true });
|
|
61
|
+
}
|
|
62
|
+
catch (err) {
|
|
63
|
+
throw new Error(`DNS lookup failed for "${hostname}": ${err.message}`, { cause: err });
|
|
64
|
+
}
|
|
65
|
+
if (addresses.length === 0) {
|
|
66
|
+
throw new Error(`DNS lookup returned no addresses for "${hostname}"`);
|
|
67
|
+
}
|
|
68
|
+
for (const addr of addresses) {
|
|
69
|
+
if (isPrivateAddress(addr.address)) {
|
|
70
|
+
throw new Error(`Blocked: "${hostname}" resolves to a private/loopback address (${addr.address}). ` +
|
|
71
|
+
`web_fetch only reaches public hosts (set SCODER_WEB_ALLOW_PRIVATE=1 to allow local testing).`);
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
/** Decode the HTML entities that actually appear in practice. */
|
|
76
|
+
function decodeEntities(s) {
|
|
77
|
+
const named = {
|
|
78
|
+
amp: "&", lt: "<", gt: ">", quot: '"', apos: "'", nbsp: " ", ndash: "–", mdash: "—",
|
|
79
|
+
hellip: "…", rsquo: "’", lsquo: "‘", rdquo: "”", ldquo: "“", copy: "©", reg: "®",
|
|
80
|
+
};
|
|
81
|
+
return s.replace(/&(#x?[0-9a-fA-F]+|[a-zA-Z]+);/g, (whole, body) => {
|
|
82
|
+
if (body.startsWith("#x") || body.startsWith("#X")) {
|
|
83
|
+
const code = parseInt(body.slice(2), 16);
|
|
84
|
+
return Number.isFinite(code) ? String.fromCodePoint(code) : whole;
|
|
85
|
+
}
|
|
86
|
+
if (body.startsWith("#")) {
|
|
87
|
+
const code = parseInt(body.slice(1), 10);
|
|
88
|
+
return Number.isFinite(code) ? String.fromCodePoint(code) : whole;
|
|
89
|
+
}
|
|
90
|
+
return named[body.toLowerCase()] ?? whole;
|
|
91
|
+
});
|
|
92
|
+
}
|
|
93
|
+
/** Convert HTML to readable plain text (scripts/styles stripped, block tags → newlines). */
|
|
94
|
+
export function htmlToText(html) {
|
|
95
|
+
const out = html
|
|
96
|
+
.replace(/<script[\s\S]*?<\/script\s*>/gi, " ")
|
|
97
|
+
.replace(/<style[\s\S]*?<\/style\s*>/gi, " ")
|
|
98
|
+
.replace(/<noscript[\s\S]*?<\/noscript\s*>/gi, " ")
|
|
99
|
+
.replace(/<!--[\s\S]*?-->/g, " ")
|
|
100
|
+
.replace(/<(br|\/p|\/div|\/section|\/article|\/header|\/footer|\/h[1-6]|\/li|\/tr|\/pre|\/blockquote)[^>]*>/gi, "\n")
|
|
101
|
+
.replace(/<[^>]+>/g, " ")
|
|
102
|
+
.replace(/[ \t]+/g, " ")
|
|
103
|
+
.replace(/ *\n */g, "\n")
|
|
104
|
+
.replace(/\n{3,}/g, "\n\n")
|
|
105
|
+
.trim();
|
|
106
|
+
return decodeEntities(out);
|
|
107
|
+
}
|
|
108
|
+
/** Extract the <title> of an HTML document, if present. */
|
|
109
|
+
function extractTitle(html) {
|
|
110
|
+
const m = html.match(/<title[^>]*>([\s\S]*?)<\/title\s*>/i);
|
|
111
|
+
if (!m)
|
|
112
|
+
return null;
|
|
113
|
+
const t = decodeEntities(m[1]).replace(/\s+/g, " ").trim();
|
|
114
|
+
return t ? t.slice(0, 200) : null;
|
|
115
|
+
}
|
|
116
|
+
/** Read the response body as text, capped at MAX_BODY_BYTES, then cancel the stream. */
|
|
117
|
+
async function readBodyCapped(res) {
|
|
118
|
+
const reader = res.body?.getReader();
|
|
119
|
+
if (!reader)
|
|
120
|
+
return { text: "", truncated: false };
|
|
121
|
+
const decoder = new TextDecoder("utf-8");
|
|
122
|
+
let text = "";
|
|
123
|
+
let received = 0;
|
|
124
|
+
while (true) {
|
|
125
|
+
const { done, value } = await reader.read();
|
|
126
|
+
if (done)
|
|
127
|
+
break;
|
|
128
|
+
received += value.byteLength;
|
|
129
|
+
text += decoder.decode(value, { stream: true });
|
|
130
|
+
if (received >= MAX_BODY_BYTES) {
|
|
131
|
+
try {
|
|
132
|
+
await reader.cancel();
|
|
133
|
+
}
|
|
134
|
+
catch { /* already closed */ }
|
|
135
|
+
return { text: text + decoder.decode(), truncated: true };
|
|
136
|
+
}
|
|
137
|
+
}
|
|
138
|
+
return { text: text + decoder.decode(), truncated: false };
|
|
139
|
+
}
|
|
140
|
+
/** Implementation of the web_fetch tool. Also used by tests via the registry. */
|
|
141
|
+
export async function toolWebFetch(ctx, args) {
|
|
142
|
+
const maxChars = Math.min(Math.max(args.max_chars ?? DEFAULT_MAX_CHARS, 500), 100_000);
|
|
143
|
+
let url;
|
|
144
|
+
try {
|
|
145
|
+
url = new URL(args.url);
|
|
146
|
+
}
|
|
147
|
+
catch {
|
|
148
|
+
return { ok: false, output: `ERROR: invalid URL: ${args.url}` };
|
|
149
|
+
}
|
|
150
|
+
if (url.protocol !== "http:" && url.protocol !== "https:") {
|
|
151
|
+
return { ok: false, output: `ERROR: only http/https URLs are supported (got ${url.protocol})` };
|
|
152
|
+
}
|
|
153
|
+
// Combined timeout + external abort (SIGINT) — manual composition for Node 18 compat
|
|
154
|
+
const controller = new AbortController();
|
|
155
|
+
const timer = setTimeout(() => controller.abort(new Error(`timeout after ${FETCH_TIMEOUT_MS / 1000}s`)), FETCH_TIMEOUT_MS);
|
|
156
|
+
const onExternalAbort = () => controller.abort(new Error("interrupted by user"));
|
|
157
|
+
if (ctx.signal?.aborted)
|
|
158
|
+
onExternalAbort();
|
|
159
|
+
else
|
|
160
|
+
ctx.signal?.addEventListener("abort", onExternalAbort, { once: true });
|
|
161
|
+
const signal = controller.signal;
|
|
162
|
+
let current = url;
|
|
163
|
+
try {
|
|
164
|
+
// Redirect loop with per-hop SSRF validation
|
|
165
|
+
for (let hop = 0; hop <= MAX_REDIRECTS; hop++) {
|
|
166
|
+
if (signal.aborted) {
|
|
167
|
+
return { ok: false, output: `ERROR: fetch aborted (${signal.reason?.message ?? "aborted"})` };
|
|
168
|
+
}
|
|
169
|
+
await assertPublicHost(current.hostname);
|
|
170
|
+
const res = await fetch(current, {
|
|
171
|
+
method: "GET",
|
|
172
|
+
redirect: "manual",
|
|
173
|
+
headers: {
|
|
174
|
+
"User-Agent": "scoder-agent (+https://github.com/nbhson/app-scoder-agent)",
|
|
175
|
+
Accept: "text/html, application/json, text/plain;q=0.9, */*;q=0.1",
|
|
176
|
+
},
|
|
177
|
+
signal,
|
|
178
|
+
});
|
|
179
|
+
// Follow redirects manually so every hop is validated against SSRF
|
|
180
|
+
if (res.status >= 300 && res.status < 400) {
|
|
181
|
+
const loc = res.headers.get("location");
|
|
182
|
+
try {
|
|
183
|
+
await res.body?.cancel();
|
|
184
|
+
}
|
|
185
|
+
catch { /* ignore */ }
|
|
186
|
+
if (!loc) {
|
|
187
|
+
return { ok: false, output: `ERROR: ${res.status} redirect without Location header at ${current.href}` };
|
|
188
|
+
}
|
|
189
|
+
if (hop === MAX_REDIRECTS) {
|
|
190
|
+
return { ok: false, output: `ERROR: too many redirects (>${MAX_REDIRECTS}) fetching ${args.url}` };
|
|
191
|
+
}
|
|
192
|
+
current = new URL(loc, current);
|
|
193
|
+
continue;
|
|
194
|
+
}
|
|
195
|
+
if (!res.ok) {
|
|
196
|
+
try {
|
|
197
|
+
await res.body?.cancel();
|
|
198
|
+
}
|
|
199
|
+
catch { /* ignore */ }
|
|
200
|
+
return { ok: false, output: `ERROR: HTTP ${res.status} ${res.statusText} for ${current.href}` };
|
|
201
|
+
}
|
|
202
|
+
const contentType = res.headers.get("content-type") ?? "";
|
|
203
|
+
if (/^(image|audio|video|font|application\/(octet-stream|zip|gzip|pdf|wasm))/i.test(contentType)) {
|
|
204
|
+
try {
|
|
205
|
+
await res.body?.cancel();
|
|
206
|
+
}
|
|
207
|
+
catch { /* ignore */ }
|
|
208
|
+
return { ok: false, output: `ERROR: unsupported content-type "${contentType}" — web_fetch returns text only (html/json/plain).` };
|
|
209
|
+
}
|
|
210
|
+
const { text: body, truncated } = await readBodyCapped(res);
|
|
211
|
+
let text;
|
|
212
|
+
if (/json/i.test(contentType)) {
|
|
213
|
+
text = prettyJson(body) ?? body;
|
|
214
|
+
}
|
|
215
|
+
else if (/html/i.test(contentType) || /^\s*</.test(body)) {
|
|
216
|
+
text = htmlToText(body);
|
|
217
|
+
}
|
|
218
|
+
else {
|
|
219
|
+
text = body;
|
|
220
|
+
}
|
|
221
|
+
const title = /text\/html/i.test(contentType) ? extractTitle(body) : null;
|
|
222
|
+
const meta = [
|
|
223
|
+
`HTTP ${res.status} ${res.statusText || ""}`.trimEnd(),
|
|
224
|
+
contentType || "unknown content-type",
|
|
225
|
+
truncated ? `body truncated at ${MAX_BODY_BYTES / 1024}KB` : null,
|
|
226
|
+
].filter(Boolean).join(", ");
|
|
227
|
+
const out = `${text.slice(0, maxChars)}${text.length > maxChars ? `\n... [truncated, ${text.length - maxChars} more chars]` : ""}`;
|
|
228
|
+
return {
|
|
229
|
+
ok: true,
|
|
230
|
+
output: `${title ? `Title: ${title}\n` : ""}${current.href}\n(${meta})\n\n${out}`,
|
|
231
|
+
};
|
|
232
|
+
}
|
|
233
|
+
// unreachable
|
|
234
|
+
return { ok: false, output: "ERROR: redirect loop" };
|
|
235
|
+
}
|
|
236
|
+
catch (err) {
|
|
237
|
+
return { ok: false, output: `ERROR: fetch failed for ${current.href}: ${err.message}` };
|
|
238
|
+
}
|
|
239
|
+
finally {
|
|
240
|
+
clearTimeout(timer);
|
|
241
|
+
ctx.signal?.removeEventListener("abort", onExternalAbort);
|
|
242
|
+
}
|
|
243
|
+
}
|
|
244
|
+
function prettyJson(body) {
|
|
245
|
+
try {
|
|
246
|
+
return JSON.stringify(JSON.parse(body), null, 2);
|
|
247
|
+
}
|
|
248
|
+
catch {
|
|
249
|
+
return undefined;
|
|
250
|
+
}
|
|
251
|
+
}
|
package/package.json
CHANGED