s-app-scoder-agent 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -80,9 +80,22 @@ You (task) → LLM plans → Tools execute code → Build runs → Success? →
80
80
  | `edit_file` | Find-and-replace (unique match required) |
81
81
  | `list_files` | Recursive directory listing |
82
82
  | `run_command` | Execute any shell command (subject to permission policy) |
83
+ | `web_fetch` | Fetch a public HTTP(S) URL → readable text (HTML→text, JSON pretty-printed) |
83
84
 
84
85
  File writes (`write_file` / `edit_file`) are symlink-safe: paths are resolved via `realpath` and cannot escape the working directory, even through a malicious symlink. `read_file` refuses files larger than 2MB with guidance to use `grep`/`head`/`tail`.
85
86
 
87
+ ### Web access (`web_fetch`)
88
+
89
+ The agent can read the internet through the built-in `web_fetch` tool — it fetches exact URLs (it cannot search) and returns readable plain text: HTML is converted to text (scripts/styles stripped, entities decoded, title extracted), JSON is pretty-printed, binary content is refused. The system prompt instructs the agent *when to fetch* (external/API docs, URLs in error messages, current versions, changelogs) and when not to (answer available locally).
90
+
91
+ Safety defaults:
92
+
93
+ - **SSRF protection** — `localhost`, private and link-local addresses (10.x, 172.16/12, 192.168.x, 127.x, 169.254.x, IPv6 ULA/link-local) are blocked. Set `SCODER_WEB_ALLOW_PRIVATE=1` to allow local dev servers.
94
+ - Only `http`/`https`; up to 5 redirects — every hop is re-validated against the private-address block list.
95
+ - 30 s timeout, SIGINT-abortable; body capped at 512 KB; extracted text capped at 20k chars (configurable via `max_chars`).
96
+ - HTML is converted to plain text (no JS execution); images/archives/PDFs are refused.
97
+ - Read-only — the tool never writes locally, so it is allowed even in `readonly` permission mode.
98
+
86
99
  ## Configuration
87
100
 
88
101
  ### Environment variables
@@ -96,6 +109,9 @@ File writes (`write_file` / `edit_file`) are symlink-safe: paths are resolved vi
96
109
  | `SCODER_MAX_STEPS` | `40` | Max loop iterations before giving up |
97
110
  | `SCODER_PERMISSION` | `auto` | Permission mode: `auto` \| `ask` \| `readonly` |
98
111
  | `SCODER_STREAM` | `1` | `0` to disable streaming LLM output |
112
+ | `SCODER_STREAM` | `1` | `0` to disable streaming LLM output |
113
+ | `SCODER_SERVER_TOKEN` | — | `scoder serve` only: require `Authorization: Bearer <token>` |
114
+ | `SCODER_WEB_ALLOW_PRIVATE` | — | Set to `1` to let `web_fetch` reach localhost/private addresses (local dev) |
99
115
  | `SCODER_NO_APIKEY_WARNING` | — | Set to `1` to suppress the apiKey-in-rc warning |
100
116
 
101
117
  ### Per-project config file (.scoderrc.json)
@@ -225,7 +241,39 @@ Sessions older than 7 days are pruned automatically on the next run. Use `--no-s
225
241
 
226
242
  ## Streaming
227
243
 
228
- LLM output is streamed token-by-token by default (`SCODER_STREAM=0` to disable). If a provider doesn't support streaming, scoder automatically falls back to non-streaming requests.
244
+ scoder streams everything it can, live, as it happens:
245
+
246
+ - **LLM tokens** — assistant text renders character-by-character (OpenAI-compatible SSE and native Anthropic streaming). If a provider doesn't support streaming, scoder automatically falls back to non-streaming requests. Disable with `SCODER_STREAM=0`.
247
+ - **Tool output** — `run_command` and build output print live (dimmed block) while the command runs, instead of appearing only after it finishes. Live output is capped at 20k chars; the full output still goes to the LLM.
248
+ - **SIGINT-safe** — Ctrl+C kills the running command immediately; streamed rendering never blocks the event loop.
249
+ - Output honors `NO_COLOR` and degrades to plain text when piped (CI-friendly).
250
+
251
+ ## HTTP Server Mode (`scoder serve`)
252
+
253
+ Run scoder as a local HTTP service — submit tasks programmatically and consume progress as Server-Sent Events:
254
+
255
+ ```bash
256
+ scoder serve # http://127.0.0.1:0 (random port, printed at startup)
257
+ scoder serve --port 4114 # fixed port
258
+ SCODER_SERVER_TOKEN=secret scoder serve --host 127.0.0.1
259
+ ```
260
+
261
+ | Endpoint | Method | Description |
262
+ |----------|--------|-------------|
263
+ | `/api/health` | GET | Liveness: `{ ok, running, runs }` |
264
+ | `/api/runs` | POST | Start a run: `{ "task": "...", "workdir?", "buildCmd?", "model?", "provider?", "maxSteps?", "permission?" }` → `202 { id, eventsUrl }` |
265
+ | `/api/runs` | GET | List runs with status/timing |
266
+ | `/api/runs/<id>` | GET | Run status + final result |
267
+ | `/api/runs/<id>/events` | GET | **SSE stream** — full event replay, then live, closes on finish |
268
+ | `/api/runs/<id>/abort` | POST | Abort a running run (equivalent to SIGINT) |
269
+
270
+ SSE event types: `run_start`, `step`, `llm_delta` (assistant text), `tool_call`, `tool_output` (live stdout/stderr chunks), `tool_result`, `llm_usage`, `build`, `finish`. The stream replays every event from the beginning, then stays live, and closes after `finish` — safe to (re)connect at any time.
271
+
272
+ Notes:
273
+
274
+ - Runs are always non-interactive; `permission: "ask"` is rejected — use `auto` or `readonly`.
275
+ - Binds `127.0.0.1` by default. Set `SCODER_SERVER_TOKEN` to require `Authorization: Bearer <token>` on every request.
276
+ - Max 4 concurrent runs (extra requests get `429`); per-run overrides: `workdir`, `buildCmd`, `model`, `provider`, `maxSteps`, `permission`.
229
277
 
230
278
  ## Logging
231
279
 
@@ -268,6 +316,7 @@ scoder detects the build command automatically:
268
316
  ```
269
317
  scoder [options] "<task>" Run a task autonomously until the build passes
270
318
  scoder Start interactive session
319
+ scoder serve [--port] [--host] HTTP server mode (see above)
271
320
  scoder --help | --version Help and version
272
321
  ```
273
322
 
@@ -283,6 +332,9 @@ scoder --help | --version Help and version
283
332
  | `-y` | Non-interactive mode (denies commands needing confirmation) |
284
333
  | `--provider <name>` | `openai` \| `anthropic` (env SCODER_PROVIDER; auto-detected) |
285
334
  | `--no-save` | Disable session auto-save to `.scoder/sessions/` |
335
+ | `--log-file` | Write a structured JSONL event log to `.scoder/logs/` |
336
+ | `--port <n>` | (`serve`) TCP port, `0` = random free port |
337
+ | `--host <addr>` | (`serve`) Bind address (default `127.0.0.1`) |
286
338
 
287
339
  ## Examples
288
340
 
@@ -305,6 +357,15 @@ scoder -m claude-sonnet-4-5 "refactor the parser module"
305
357
 
306
358
  # CI: non-interactive, no session saving
307
359
  scoder --no-save -y "run the full migration"
360
+
361
+ # Run as an HTTP service and consume progress via SSE
362
+ scoder serve --port 4114 &
363
+ curl -s -X POST localhost:4114/api/runs -H 'Content-Type: application/json' \
364
+ -d '{"task": "add input validation to the CLI"}'
365
+ curl -N http://127.0.0.1:4114/api/runs/<id>/events # live progress
366
+
367
+ # Fetch external docs during a task (agent decides when, via web_fetch)
368
+ scoder "upgrade express to v5 — check the official migration guide first"
308
369
  ```
309
370
 
310
371
  ## Architecture
@@ -312,22 +373,26 @@ scoder --no-save -y "run the full migration"
312
373
  ```
313
374
  ┌─────────────────────────────────────────────┐
314
375
  │ CLI (src/index.ts) │
315
- │ arg parsing, REPL, SIGINT handling │
316
- └─────────────────┬───────────────────────────┘
317
- │
318
- ┌─────────────────▼───────────────────────────┐
376
+ │ arg parsing, REPL, serve, SIGINT │
377
+ └───────────────┬─────────────────────────────┘
378
+ │
379
+ ┌───────────────▼─────────────────────────────┐
319
380
  │ Agent (src/agent.ts) │
320
- │ Closed-loop orchestrator │
321
- │ history trimming, error injection │
322
- └───────┬───────────────────────┬─────────────┘
323
- │ │
324
- ┌───────▼───────┐ ┌─────────▼──────────┐
325
- │ LLM Client │ │ Tool Registry │
326
- │ (src/llm.ts) │ │ (src/tools.ts) │
327
- │ OpenAI compat │ │ read/write/run/list │
328
- └───────────────┘ └──────────────────────┘
329
- │
330
- ▼
381
+ │ Closed-loop orchestrator │
382
+ │ history trimming, error injection │
383
+ │ emits AgentEvent (UI / SSE consumers) │
384
+ └──────┬─────────────┬───────────────┬────────┘
385
+ │ │ │
386
+ ┌──────▼───────┐ ┌──▼───────────┐ ┌▼──────────────────┐
387
+ │ LLM Client │ │ Tool Registry│ │ Server │
388
+ │ (src/llm.ts) │ │(src/tools.ts)│ │ (src/server.ts) │
389
+ │ OpenAI compat│ │ + web_fetch │ │ HTTP API + SSE │
390
+ │ (anthropic: │ │ (src/ │ │ runs in background│
391
+ │ provider- │ │ webfetch.ts)│ └───────────────────┘
392
+ │ anthropic) │ └──────────────┘
393
+ └──────────────┘
394
+ │
395
+ ▼
331
396
  ┌─────────────────────────────────────────────┐
332
397
  │ Build Checker (src/build.ts) │
333
398
  │ run build → exit 0 = stop condition met │
@@ -337,7 +402,7 @@ scoder --no-save -y "run the full migration"
337
402
  ## Integration Testing
338
403
 
339
404
  ```bash
340
- npm test # runs: lint → typecheck → build → 116 unit tests + e2e + MCP smoke
405
+ npm test # runs: build → unit tests (node --test) → e2e closed-loop → MCP smoke
341
406
  npm run lint # eslint flat config (typescript-eslint + eqeqeq, prefer-const...)
342
407
  npm run typecheck # tsc --noEmit
343
408
  ```
@@ -345,17 +410,18 @@ npm run typecheck # tsc --noEmit
345
410
  Test suites (run with `npm test` — auto-discovers `test/*.test.mjs`):
346
411
  - `test/permissions.test.mjs` — catastrophic patterns, mode classification, allowlist word-boundary, timeout-denies
347
412
  - `test/permission-prompt.test.mjs` — interactive prompt timeout behavior
348
- - `test/tools.test.mjs` — safePath traversal, symlink escape, file-size cap, write/edit/list
413
+ - `test/tools.test.mjs` — safePath traversal, symlink escape, file-size cap, write/edit/list, live `run_command` output streaming
414
+ - `test/webfetch.test.mjs` — HTML→text, JSON, redirects, SSRF guard, binary refusal, truncation
415
+ - `test/server.test.mjs` — HTTP API: run lifecycle via SSE, abort, auth, live `tool_output` events
349
416
  - `test/core.test.mjs` — unified diff, retry backoff, context trimming
350
417
  - `test/config.test.mjs` — env/rc/cli precedence, logFile mapping, permission lists
351
418
  - `test/session.test.mjs` — session store, smart prune, resolve
419
+ - `test/cost.test.mjs` — token usage & cost aggregation
352
420
  - `test/anthropic.test.mjs` — message translation + native SSE streaming
353
421
  - `test/llm.test.mjs` — OpenAI-compatible chat + SSE streaming + redaction
354
422
  - `test/run-e2e.mjs` — closed-loop integration (mock LLM server)
355
423
  - `test/run-mcp-test.mjs` — MCP smoke (mock stdio server)
356
424
 
357
- Coverage (lines): **90.3%** overall — see `npm run coverage`.
358
-
359
425
  ## Exit Codes
360
426
 
361
427
  | Code | Meaning |
package/dist/agent.js CHANGED
@@ -28,7 +28,13 @@ RULES:
28
28
  6. Never invent requirements beyond the given task.
29
29
  7. IMPORTANT: if a run_command is denied by permission policy, you will receive
30
30
  an error message. Use this as a signal to find a safer alternative or to
31
- communicate the limitation to the user.`;
31
+ communicate the limitation to the user.
32
+ 8. You have web access ONLY through web_fetch (it fetches exact URLs, it cannot
33
+ search). Use it when the task depends on external or fast-changing knowledge:
34
+ library/API documentation, a URL found in an error message or code comment,
35
+ current package versions, changelogs, release notes, standards. Skip it when
36
+ the answer is in the working directory or is stable general knowledge. After
37
+ fetching, ground your edits in what the page actually says.`;
32
38
  export class Agent {
33
39
  config;
34
40
  client;
@@ -48,6 +54,10 @@ export class Agent {
48
54
  mcp = null;
49
55
  /** Aborted when abort() is called — cancels in-flight LLM requests (SIGINT). */
50
56
  runSignal = new AbortController();
57
+ /** Optional structured-event sink (used by the HTTP/SSE server frontend). */
58
+ onEvent = null;
59
+ /** Set while a tool call is executing, for tool_output event correlation. */
60
+ currentToolCall = null;
51
61
  constructor(config) {
52
62
  this.config = config;
53
63
  const commonOpts = {
@@ -72,6 +82,7 @@ export class Agent {
72
82
  workdir: config.workdir,
73
83
  permissions: this.permissions,
74
84
  permissionMode: config.permissionMode,
85
+ onToolOutput: this.handleToolOutput,
75
86
  };
76
87
  this.session = new SessionStore(config.workdir, config.noSave !== true);
77
88
  this.logger = JsonlLogger.forSession(config.workdir, this.session.id, config.logFile);
@@ -92,6 +103,27 @@ export class Agent {
92
103
  // terminate its child process immediately (long builds must be interruptible).
93
104
  this.ctx.signal = this.runSignal.signal;
94
105
  }
106
+ /** Emit one structured run event to the sink (if attached). */
107
+ emit(ev) {
108
+ this.onEvent?.(ev);
109
+ }
110
+ /**
111
+ * Live tool-output handler: renders chunks in the terminal (dimmed block) and
112
+ * forwards them as `tool_output` events. Opens the stream block lazily on the
113
+ * first chunk; `endToolStream()` closes it after the tool completes.
114
+ */
115
+ handleToolOutput = (name, stream, delta) => {
116
+ if (!ui.isToolStreamOpen()) {
117
+ ui.toolStreamStart();
118
+ }
119
+ ui.toolStreamChunk(delta);
120
+ this.emit({ type: "tool_output", name, stream, delta });
121
+ };
122
+ /** Close the live tool-output block (if open) before printing the next UI element. */
123
+ endToolStream() {
124
+ if (ui.isToolStreamOpen())
125
+ ui.toolStreamEnd();
126
+ }
95
127
  /** Push one tool-result message back into the conversation. */
96
128
  pushToolResult(tc, content) {
97
129
  this.messages.push({
@@ -108,6 +140,7 @@ export class Agent {
108
140
  */
109
141
  rejectToolCall(step, tc, output, logExtra) {
110
142
  ui.toolResult(tc.name, false, output);
143
+ this.emit({ type: "tool_result", step, name: tc.name, ok: false, output });
111
144
  this.logger.log({ type: "tool", step, tool: tc.name, ok: false, ...logExtra });
112
145
  this.pushToolResult(tc, output);
113
146
  }
@@ -126,6 +159,10 @@ export class Agent {
126
159
  this.usageTotals.outputTokens += output;
127
160
  this.usageTotals.cachedTokens += cached;
128
161
  ui.info(`tokens: in=${usage.prompt_tokens ?? "?"} out=${usage.completion_tokens ?? "?"}`);
162
+ this.emit({
163
+ type: "llm_usage",
164
+ usage: { input_tokens: input, output_tokens: output, cached_tokens: cached },
165
+ });
129
166
  this.logger.log({
130
167
  type: "llm_usage",
131
168
  model: this.client.model,
@@ -161,6 +198,7 @@ export class Agent {
161
198
  started = true;
162
199
  }
163
200
  ui.llmStreamChunk(chunk.contentDelta);
201
+ this.emit({ type: "llm_delta", delta: chunk.contentDelta });
164
202
  streamedText += chunk.contentDelta;
165
203
  }
166
204
  if (chunk.final)
@@ -217,9 +255,13 @@ export class Agent {
217
255
  throw new Error("No API key configured. Set SCODER_API_KEY or OPENAI_API_KEY (or put apiKey in .scoderrc.json).");
218
256
  }
219
257
  const buildCmd = this.buildCmd;
220
- ui.header("TASK");
221
- ui.info(task);
222
- ui.info(`model=${this.client.model} workdir=${this.config.workdir} build="${buildCmd}" maxSteps=${this.config.maxSteps}`);
258
+ ui.banner("scoder", [
259
+ ["task", task],
260
+ ["model", `${this.client.model} (${this.config.provider}${this.config.stream ? ", stream" : ""})`],
261
+ ["workdir", this.config.workdir],
262
+ ["build", buildCmd],
263
+ ["steps", `max ${this.config.maxSteps} · permission ${this.config.permissionMode}`],
264
+ ]);
223
265
  this.currentTask = task;
224
266
  this.resetRunSignal();
225
267
  this.resetUsageTotals();
@@ -231,6 +273,14 @@ export class Agent {
231
273
  },
232
274
  { role: "user", content: task },
233
275
  ];
276
+ this.emit({
277
+ type: "run_start",
278
+ task,
279
+ model: this.client.model,
280
+ provider: this.config.provider,
281
+ workdir: this.config.workdir,
282
+ buildCmd,
283
+ });
234
284
  try {
235
285
  await this.connectMcp();
236
286
  return await this.runLoop(0, buildCmd, null);
@@ -304,6 +354,7 @@ export class Agent {
304
354
  while (steps < this.config.maxSteps && !this.aborted) {
305
355
  steps++;
306
356
  ui.step(steps, this.config.maxSteps, "thinking...");
357
+ this.emit({ type: "step", step: steps, maxSteps: this.config.maxSteps });
307
358
  const { message, usage } = await this.chatWithStream().catch((err) => {
308
359
  // SIGINT during the LLM call — stop gracefully instead of crashing
309
360
  if (this.aborted) {
@@ -363,6 +414,8 @@ export class Agent {
363
414
  continue;
364
415
  }
365
416
  ui.toolCall(tc.name, args);
417
+ this.currentToolCall = { step: steps, id: tc.id, name: tc.name };
418
+ this.emit({ type: "tool_call", step: steps, id: tc.id, name: tc.name, args });
366
419
  if (!tool && !(this.mcp?.hasTool(tc.name))) {
367
420
  this.rejectToolCall(steps, tc, `ERROR: unknown tool "${tc.name}". Available: ${AGENT_TOOLS.map((t) => t.spec.function.name).join(", ")}${this.mcp ? " + MCP tools (mcp__<server>__<tool>)" : ""}`, { error: "unknown tool" });
368
421
  continue;
@@ -426,7 +479,10 @@ export class Agent {
426
479
  this.aborted = true;
427
480
  ui.warn("edit confirmation aborted by user ([q]uit) — stopping the run");
428
481
  }
482
+ this.endToolStream();
483
+ this.currentToolCall = null;
429
484
  ui.toolResult(tc.name, result.ok, result.output);
485
+ this.emit({ type: "tool_result", step: steps, name: tc.name, ok: result.ok, output: result.output });
430
486
  stepToolResults.set(tc.id, { ok: result.ok, output: result.output });
431
487
  this.pushToolResult(tc, summariseOutput(result.output));
432
488
  }
@@ -483,8 +539,10 @@ export class Agent {
483
539
  }
484
540
  /** Explicitly verify the build (independent of what the LLM claims). */
485
541
  async checkBuild(buildCmd) {
486
- const res = await runBuild(this.config.workdir, buildCmd);
542
+ const res = await runBuild(this.config.workdir, buildCmd, (stream, delta) => this.handleToolOutput("build", stream, delta));
543
+ this.endToolStream();
487
544
  ui.buildStatus(res.success, res.output);
545
+ this.emit({ type: "build", success: res.success, command: buildCmd, output: res.output });
488
546
  this.logger.log({ type: "build", success: res.success, command: buildCmd, output: res.output.slice(0, 10_000) });
489
547
  return res;
490
548
  }
@@ -495,6 +553,7 @@ export class Agent {
495
553
  else {
496
554
  ui.error(`Task NOT complete. ${summary}`);
497
555
  }
556
+ this.emit({ type: "finish", success, steps, summary: summary.slice(0, 2000) });
498
557
  this.logger.log({ type: "finish", success, steps });
499
558
  return { success, steps, buildCmd, summary: summary.slice(0, 2000) };
500
559
  }
package/dist/build.js CHANGED
@@ -30,9 +30,10 @@ export function detectBuildCommand(workdir) {
30
30
  /**
31
31
  * Run the build command. This is the agent's stop condition:
32
32
  * exit code 0 == task complete.
33
+ * `onOutput` (optional) receives stdout/stderr chunks live while the build runs.
33
34
  */
34
- export async function runBuild(workdir, command) {
35
- const res = await toolRunCommand(workdir, { command, timeout_secs: 600 });
35
+ export async function runBuild(workdir, command, onOutput) {
36
+ const res = await toolRunCommand(workdir, { command, timeout_secs: 600 }, undefined, onOutput);
36
37
  return {
37
38
  success: res.ok,
38
39
  command,
package/dist/index.js CHANGED
@@ -21,6 +21,9 @@ USAGE
21
21
  scoder logs [id|last] [filter] Print a session's JSONL log (requires --log-file runs)
22
22
  scoder cost [--session <id>] [--last <n>]
23
23
  Token usage & estimated cost from JSONL logs
24
+ scoder serve [--port <n>] [--host <addr>]
25
+ Run as an HTTP server: POST tasks, stream
26
+ progress via Server-Sent Events (see below)
24
27
  scoder --version | --help
25
28
 
26
29
  OPTIONS
@@ -120,6 +123,17 @@ function parseArgs(argv) {
120
123
  args.provider = p;
121
124
  break;
122
125
  }
126
+ case "--port": {
127
+ const n = Number(next());
128
+ if (!Number.isInteger(n) || n < 0 || n > 65535) {
129
+ throw new Error(`Invalid --port value (expected 0-65535): ${n}`);
130
+ }
131
+ args.servePort = n;
132
+ break;
133
+ }
134
+ case "--host":
135
+ args.serveHost = next();
136
+ break;
123
137
  case "-h":
124
138
  case "--help":
125
139
  args.help = true;
@@ -155,6 +169,10 @@ function parseArgs(argv) {
155
169
  args.subcommand = "cost";
156
170
  args.task = null;
157
171
  }
172
+ else if (rest[0] === "serve") {
173
+ args.subcommand = "serve";
174
+ args.task = null;
175
+ }
158
176
  return args;
159
177
  }
160
178
  async function runOnce(cfg, task) {
@@ -302,6 +320,32 @@ async function main() {
302
320
  console.log(formatCostReport(sessionCosts));
303
321
  return;
304
322
  }
323
+ if (args.subcommand === "serve") {
324
+ const { startServer } = await import("./server.js");
325
+ if (!cfg.apiKey) {
326
+ ui.warn("No API key configured — runs will fail until SCODER_API_KEY (or OPENAI_API_KEY) is set.");
327
+ }
328
+ const srv = await startServer(cfg, { host: args.serveHost, port: args.servePort });
329
+ const token = process.env.SCODER_SERVER_TOKEN ? " · token auth" : "";
330
+ ui.banner(`scoder serve @ http://${args.serveHost ?? "127.0.0.1"}:${srv.port}${token}`, [
331
+ ["task", `POST /api/runs { "task": "..." } → 202 { id, eventsUrl }`],
332
+ ["events", `GET /api/runs/<id>/events → SSE stream`],
333
+ ["list", `GET /api/runs`],
334
+ ["abort", `POST /api/runs/<id>/abort`],
335
+ ["workdir", cfg.workdir],
336
+ ["model", `${cfg.model} · permission ${cfg.permissionMode}`],
337
+ ]);
338
+ const shutdown = async () => {
339
+ ui.info("shutting down server...");
340
+ await srv.close();
341
+ process.exit(0);
342
+ };
343
+ process.on("SIGINT", () => {
344
+ ui.warn("\nSIGINT — shutting down server, aborting in-flight runs...");
345
+ void shutdown();
346
+ });
347
+ return;
348
+ }
305
349
  if (args.task) {
306
350
  const code = await runOnce(cfg, args.task);
307
351
  process.exit(code);
package/dist/server.js ADDED
@@ -0,0 +1,256 @@
1
+ import { createServer } from "node:http";
2
+ import { randomUUID } from "node:crypto";
3
+ import { existsSync, statSync } from "node:fs";
4
+ import { resolve } from "node:path";
5
+ import process from "node:process";
6
+ import { Agent } from "./agent.js";
7
+ const MAX_BODY_BYTES = 1024 * 1024;
8
+ function readBody(req, maxBytes = MAX_BODY_BYTES) {
9
+ return new Promise((resolve, reject) => {
10
+ let body = "";
11
+ let size = 0;
12
+ req.on("data", (c) => {
13
+ size += c.byteLength;
14
+ if (size > maxBytes) {
15
+ reject(new Error(`request body too large (>${maxBytes} bytes)`));
16
+ req.destroy();
17
+ return;
18
+ }
19
+ body += c.toString("utf8");
20
+ });
21
+ req.on("end", () => resolve(body));
22
+ req.on("error", reject);
23
+ });
24
+ }
25
+ function sendJson(res, status, payload) {
26
+ res.writeHead(status, { "Content-Type": "application/json; charset=utf-8" });
27
+ res.end(JSON.stringify(payload));
28
+ }
29
+ function isAuthorized(req, token) {
30
+ if (!token)
31
+ return true;
32
+ const header = req.headers.authorization ?? "";
33
+ return header === `Bearer ${token}`;
34
+ }
35
+ export function startServer(baseConfig, opts = {}) {
36
+ const host = opts.host ?? "127.0.0.1";
37
+ const maxConcurrent = opts.maxConcurrent ?? 4;
38
+ const token = process.env.SCODER_SERVER_TOKEN || "";
39
+ /** id -> run record; Map preserves insertion order (= creation order). */
40
+ const runs = new Map();
41
+ const publicRun = (r) => ({
42
+ id: r.id,
43
+ task: r.task,
44
+ workdir: r.workdir,
45
+ status: r.status,
46
+ createdAt: r.createdAt,
47
+ finishedAt: r.finishedAt,
48
+ eventCount: r.events.length,
49
+ result: r.result,
50
+ error: r.error,
51
+ eventsUrl: `/api/runs/${r.id}/events`,
52
+ });
53
+ const runningCount = () => [...runs.values()].filter((r) => r.status === "running").length;
54
+ /** Launch an agent run in the background, wiring events into the record. */
55
+ const launchRun = (record, cfg, task) => {
56
+ const agent = new Agent(cfg);
57
+ record.agent = agent;
58
+ agent.onEvent = (ev) => {
59
+ record.events.push(ev);
60
+ if (ev.type === "finish") {
61
+ record.status = ev.success ? "success" : record.abortRequested ? "aborted" : "failed";
62
+ record.finishedAt = new Date().toISOString();
63
+ }
64
+ for (const sub of record.subscribers)
65
+ sub();
66
+ };
67
+ agent
68
+ .run(task)
69
+ .then((result) => {
70
+ record.result = result;
71
+ if (record.status === "running") {
72
+ // Should not happen (finish event always arrives first), but be safe.
73
+ record.status = result.success ? "success" : record.abortRequested ? "aborted" : "failed";
74
+ record.finishedAt = new Date().toISOString();
75
+ }
76
+ })
77
+ .catch((err) => {
78
+ record.error = err.message;
79
+ // Synthetic terminal events so SSE clients always see a stream end.
80
+ record.events.push({ type: "error", message: record.error });
81
+ record.events.push({ type: "finish", success: false, steps: 0, summary: `run failed: ${record.error}` });
82
+ record.status = "error";
83
+ record.finishedAt = new Date().toISOString();
84
+ for (const sub of record.subscribers)
85
+ sub();
86
+ });
87
+ };
88
+ const handleEventsSSE = (req, res, record) => {
89
+ res.writeHead(200, {
90
+ "Content-Type": "text/event-stream",
91
+ "Cache-Control": "no-cache",
92
+ Connection: "keep-alive",
93
+ });
94
+ let sent = 0;
95
+ const heartbeat = setInterval(() => {
96
+ if (!res.writableEnded)
97
+ res.write(": ping\n\n");
98
+ }, 15_000);
99
+ heartbeat.unref?.();
100
+ /** Replay buffered events + live ones; ends the stream on a terminal event. */
101
+ const flush = () => {
102
+ while (sent < record.events.length) {
103
+ const ev = record.events[sent++];
104
+ res.write(`data: ${JSON.stringify(ev)}\n\n`);
105
+ if (ev.type === "finish" || ev.type === "error") {
106
+ res.end();
107
+ record.subscribers.delete(flush);
108
+ return;
109
+ }
110
+ }
111
+ };
112
+ record.subscribers.add(flush);
113
+ flush(); // replays the buffer, then stays live via the subscription
114
+ req.on("close", () => {
115
+ record.subscribers.delete(flush);
116
+ clearInterval(heartbeat);
117
+ });
118
+ };
119
+ const handle = async (req, res) => {
120
+ const url = new URL(req.url ?? "/", `http://${req.headers.host ?? "localhost"}`);
121
+ const path = url.pathname;
122
+ if (!isAuthorized(req, token)) {
123
+ sendJson(res, 401, { error: "unauthorized (set Authorization: Bearer <SCODER_SERVER_TOKEN>)" });
124
+ return;
125
+ }
126
+ // ── GET /api/health ──
127
+ if (path === "/api/health" && req.method === "GET") {
128
+ sendJson(res, 200, { ok: true, running: runningCount(), runs: runs.size });
129
+ return;
130
+ }
131
+ // ── /api/runs ──
132
+ if (path === "/api/runs") {
133
+ if (req.method === "GET") {
134
+ sendJson(res, 200, { runs: [...runs.values()].map(publicRun) });
135
+ return;
136
+ }
137
+ if (req.method === "POST") {
138
+ if (runningCount() >= maxConcurrent) {
139
+ sendJson(res, 429, { error: `too many concurrent runs (max ${maxConcurrent}) — retry later` });
140
+ return;
141
+ }
142
+ let body;
143
+ try {
144
+ body = JSON.parse(await readBody(req));
145
+ }
146
+ catch (err) {
147
+ sendJson(res, 400, { error: err.message });
148
+ return;
149
+ }
150
+ if (typeof body.task !== "string" || body.task.trim().length === 0) {
151
+ sendJson(res, 400, { error: 'field "task" (non-empty string) is required' });
152
+ return;
153
+ }
154
+ const cfg = { ...baseConfig, yes: true };
155
+ if (body.workdir) {
156
+ const abs = resolve(body.workdir);
157
+ if (!existsSync(abs) || !statSync(abs).isDirectory()) {
158
+ sendJson(res, 400, { error: `workdir does not exist or is not a directory: ${body.workdir}` });
159
+ return;
160
+ }
161
+ cfg.workdir = abs;
162
+ }
163
+ const permission = body.permission ?? cfg.permissionMode;
164
+ if (permission === "ask") {
165
+ sendJson(res, 400, { error: 'permission "ask" is not supported by the server (it cannot prompt). Use "auto" or "readonly".' });
166
+ return;
167
+ }
168
+ cfg.permissionMode = permission;
169
+ if (body.buildCmd)
170
+ cfg.buildCmd = body.buildCmd;
171
+ if (body.model)
172
+ cfg.model = body.model;
173
+ if (body.provider)
174
+ cfg.provider = body.provider;
175
+ if (body.maxSteps)
176
+ cfg.maxSteps = Math.min(Math.max(body.maxSteps, 1), 500);
177
+ const record = {
178
+ id: randomUUID(),
179
+ task: body.task,
180
+ workdir: cfg.workdir,
181
+ status: "running",
182
+ createdAt: new Date().toISOString(),
183
+ finishedAt: null,
184
+ events: [],
185
+ subscribers: new Set(),
186
+ agent: null,
187
+ abortRequested: false,
188
+ result: null,
189
+ error: null,
190
+ };
191
+ runs.set(record.id, record);
192
+ launchRun(record, cfg, body.task);
193
+ sendJson(res, 202, publicRun(record));
194
+ return;
195
+ }
196
+ sendJson(res, 405, { error: "method not allowed" });
197
+ return;
198
+ }
199
+ // ── /api/runs/:id[...] ──
200
+ const runMatch = path.match(/^\/api\/runs\/([0-9a-f-]{36})(\/events|\/abort)?$/i);
201
+ if (runMatch) {
202
+ const record = runs.get(runMatch[1]);
203
+ if (!record) {
204
+ sendJson(res, 404, { error: `unknown run id: ${runMatch[1]}` });
205
+ return;
206
+ }
207
+ if (runMatch[2] === "/events" && req.method === "GET") {
208
+ handleEventsSSE(req, res, record);
209
+ return;
210
+ }
211
+ if (runMatch[2] === "/abort" && req.method === "POST") {
212
+ if (record.status !== "running" || !record.agent) {
213
+ sendJson(res, 409, { error: `run is not running (status: ${record.status})` });
214
+ return;
215
+ }
216
+ record.abortRequested = true;
217
+ record.agent.abort();
218
+ sendJson(res, 200, { ok: true, status: "running (aborting...)" });
219
+ return;
220
+ }
221
+ if (!runMatch[2] && req.method === "GET") {
222
+ sendJson(res, 200, publicRun(record));
223
+ return;
224
+ }
225
+ }
226
+ sendJson(res, 404, { error: `not found: ${req.method} ${path}` });
227
+ };
228
+ const server = createServer((req, res) => {
229
+ void handle(req, res).catch(() => {
230
+ if (!res.writableEnded)
231
+ sendJson(res, 500, { error: "internal server error" });
232
+ });
233
+ });
234
+ return new Promise((resolve, reject) => {
235
+ server.once("error", reject);
236
+ server.listen(opts.port ?? 0, host, () => {
237
+ const address = server.address();
238
+ const port = typeof address === "object" && address !== null ? address.port : 0;
239
+ resolve({
240
+ server,
241
+ port,
242
+ close: () => new Promise((res) => {
243
+ // Abort in-flight runs so background work does not keep the loop alive.
244
+ for (const r of runs.values()) {
245
+ if (r.status === "running") {
246
+ r.abortRequested = true;
247
+ r.agent?.abort();
248
+ }
249
+ }
250
+ server.close(() => res());
251
+ server.closeAllConnections?.();
252
+ }),
253
+ });
254
+ });
255
+ });
256
+ }
package/dist/tools.js CHANGED
@@ -3,6 +3,7 @@ import { spawn } from "node:child_process";
3
3
  import { resolve, dirname, basename, join, relative, isAbsolute } from "node:path";
4
4
  import { unifiedDiff } from "./diff.js";
5
5
  import * as ui from "./ui.js";
6
+ import { toolWebFetch } from "./webfetch.js";
6
7
  const MAX_OUTPUT_CHARS = 20_000;
7
8
  const MAX_FILE_SIZE_BYTES = 2 * 1024 * 1024; // refuse to read files > 2MB
8
9
  function clamp(s, max = MAX_OUTPUT_CHARS) {
@@ -211,8 +212,10 @@ function toolListFiles(workdir, args) {
211
212
  * Run a shell command asynchronously (async spawn keeps the event loop free,
212
213
  * so streaming output renders and SIGINT is handled promptly).
213
214
  * Kills the child on timeout or external abort (SIGINT via ctx.signal).
215
+ * When `onOutput` is provided, stdout/stderr chunks are forwarded live as they
216
+ * arrive (in addition to being collected into the final ToolResult).
214
217
  */
215
- export async function toolRunCommand(workdir, args, signal) {
218
+ export async function toolRunCommand(workdir, args, signal, onOutput) {
216
219
  const timeoutMs = Math.min(Math.max(args.timeout_secs ?? 120, 1), 600) * 1000;
217
220
  return new Promise((resolve) => {
218
221
  let settled = false;
@@ -252,8 +255,16 @@ export async function toolRunCommand(workdir, args, signal) {
252
255
  onAbort();
253
256
  else
254
257
  signal?.addEventListener("abort", onAbort, { once: true });
255
- child.stdout?.on("data", (c) => { stdout += c.toString("utf8"); });
256
- child.stderr?.on("data", (c) => { stderr += c.toString("utf8"); });
258
+ child.stdout?.on("data", (c) => {
259
+ const s = c.toString("utf8");
260
+ stdout += s;
261
+ onOutput?.("stdout", s);
262
+ });
263
+ child.stderr?.on("data", (c) => {
264
+ const s = c.toString("utf8");
265
+ stderr += s;
266
+ onOutput?.("stderr", s);
267
+ });
257
268
  child.on("error", (err) => {
258
269
  finish({ ok: false, output: clamp(`$ ${args.command}\n(spawn error) ${err.message}`) });
259
270
  });
@@ -353,7 +364,28 @@ export const AGENT_TOOLS = [
353
364
  }, ["command"]),
354
365
  },
355
366
  },
356
- run: (ctx, args) => toolRunCommand(ctx.workdir, args, ctx.signal),
367
+ run: (ctx, args) => toolRunCommand(ctx.workdir, args, ctx.signal, ctx.onToolOutput ? (stream, delta) => ctx.onToolOutput?.("run_command", stream, delta) : undefined),
368
+ },
369
+ {
370
+ spec: {
371
+ type: "function",
372
+ function: {
373
+ name: "web_fetch",
374
+ description: "Fetch a public HTTP(S) URL and return its content as readable plain text " +
375
+ "(HTML converted to text, JSON pretty-printed). WHEN TO USE: the task depends on external, " +
376
+ "possibly outdated knowledge — library/API documentation, an error message or comment " +
377
+ "containing a URL, current package versions or changelogs, standards/specs. " +
378
+ "WHEN NOT TO: the answer is already in the working directory or in the user's request; " +
379
+ "you need a search engine (this tool fetches exact URLs only, it cannot search); " +
380
+ "the content is a binary file (images, archives, PDFs are refused). " +
381
+ "Private/internal addresses (localhost, 10.x, 192.168.x, 127.x) are blocked.",
382
+ parameters: obj({
383
+ url: str("Absolute http(s) URL to fetch"),
384
+ max_chars: str("Max characters of extracted content to return (default 20000, max 100000)"),
385
+ }, ["url"]),
386
+ },
387
+ },
388
+ run: (ctx, args) => toolWebFetch(ctx, args),
357
389
  },
358
390
  ];
359
391
  export function getTool(name) {
package/dist/ui.js CHANGED
@@ -1,12 +1,14 @@
1
1
  import process from "node:process";
2
- const RESET = "\x1b[0m";
3
- const BOLD = "\x1b[1m";
4
- const DIM = "\x1b[2m";
5
- const GREEN = "\x1b[32m";
6
- const YELLOW = "\x1b[33m";
7
- const RED = "\x1b[31m";
8
- const CYAN = "\x1b[36m";
9
- const BLUE = "\x1b[34m";
2
+ // Honor NO_COLOR and non-TTY output: drop ANSI codes so logs/CI stay clean.
3
+ const COLOR = process.env.NO_COLOR === undefined && process.stdout.isTTY === true;
4
+ const RESET = COLOR ? "\x1b[0m" : "";
5
+ const BOLD = COLOR ? "\x1b[1m" : "";
6
+ const DIM = COLOR ? "\x1b[2m" : "";
7
+ const GREEN = COLOR ? "\x1b[32m" : "";
8
+ const YELLOW = COLOR ? "\x1b[33m" : "";
9
+ const RED = COLOR ? "\x1b[31m" : "";
10
+ const CYAN = COLOR ? "\x1b[36m" : "";
11
+ const BLUE = COLOR ? "\x1b[34m" : "";
10
12
  function ts() {
11
13
  return new Date().toISOString().slice(11, 23);
12
14
  }
@@ -14,7 +16,7 @@ export function info(msg) {
14
16
  console.log(`${DIM}[${ts()}]${RESET} ${msg}`);
15
17
  }
16
18
  export function step(stepNum, total, msg) {
17
- console.log(`${BLUE}[${stepNum}/${total}]${RESET} ${BOLD}${msg}${RESET}`);
19
+ console.log(`${BLUE}[${String(stepNum).padStart(2)}/${total}]${RESET} ${BOLD}▸ ${msg}${RESET}`);
18
20
  }
19
21
  export function success(msg) {
20
22
  console.log(`${GREEN}✓${RESET} ${GREEN}${msg}${RESET}`);
@@ -37,7 +39,7 @@ export function toolCall(name, args) {
37
39
  export function toolResult(name, ok, output) {
38
40
  const preview = output.split("\n")[0]?.slice(0, 120) ?? "";
39
41
  const icon = ok ? `${GREEN}✓${RESET}` : `${RED}✗${RESET}`;
40
- console.log(`${icon} ${CYAN}${name}${RESET} → ${preview}`);
42
+ console.log(`${icon} ${CYAN}${name}${RESET} ${DIM}→${RESET} ${preview}`);
41
43
  }
42
44
  export function buildStatus(success, output) {
43
45
  const icon = success ? `${GREEN}✓ BUILD SUCCESS` : `${RED}✗ BUILD FAILED`;
@@ -46,6 +48,25 @@ export function buildStatus(success, output) {
46
48
  export function header(msg) {
47
49
  console.log(`\n${BOLD}${CYAN}── ${msg} ${"─".repeat(Math.max(2, 60 - msg.length))}${RESET}`);
48
50
  }
51
+ /** Startup banner: a clean key/value box (plain-text fallback when piped). */
52
+ export function banner(title, rows) {
53
+ const keyWidth = Math.max(8, ...rows.map(([k]) => k.length));
54
+ const inner = Math.max(title.length + 4, ...rows.map(([, v]) => keyWidth + Math.max(...String(v).split("\n").map((l) => l.length)) + 1));
55
+ console.log(`${BOLD}${CYAN}╭─ ${title} ${"─".repeat(Math.max(1, inner - title.length - 3))}╮${RESET}`);
56
+ for (const [key, value] of rows) {
57
+ const lines = String(value).split("\n");
58
+ for (let i = 0; i < lines.length; i++) {
59
+ const k = (i === 0 ? key : "").padEnd(keyWidth);
60
+ let v = lines[i];
61
+ const maxV = inner - keyWidth - 1;
62
+ if (v.length > maxV)
63
+ v = v.slice(0, maxV - 1) + "…";
64
+ const pad = Math.max(0, inner - 1 - keyWidth - v.length);
65
+ console.log(`${BOLD}${CYAN}│${RESET} ${DIM}${k}${RESET}${v}${" ".repeat(pad)}${BOLD}${CYAN}│${RESET}`);
66
+ }
67
+ }
68
+ console.log(`${BOLD}${CYAN}╰${"─".repeat(inner)}╯${RESET}`);
69
+ }
49
70
  // ── Streaming helpers ────────────────────────────────────────────────────────
50
71
  let _streamLine = "";
51
72
  let _streamColor = CYAN;
@@ -69,6 +90,51 @@ export function llmStreamEnd(suffix = "") {
69
90
  process.stdout.write(`${suffix}\n`);
70
91
  return line;
71
92
  }
93
+ // ── Tool output streaming (live run_command output) ─────────────────────────
94
+ const TOOL_STREAM_MAX_CHARS = 20_000;
95
+ let _toolStreamOpen = false;
96
+ let _toolStreamLineOpen = false; // last chunk did not end with "\n"
97
+ let _toolStreamChars = 0;
98
+ /** Begin a live tool-output block. Output is printed dimmed, capped at 20k chars. */
99
+ export function toolStreamStart() {
100
+ if (_toolStreamOpen)
101
+ return;
102
+ _toolStreamOpen = true;
103
+ _toolStreamLineOpen = false;
104
+ _toolStreamChars = 0;
105
+ process.stdout.write(`${DIM}┌ output${RESET}`);
106
+ }
107
+ /** Append a tool-output delta to the live block. */
108
+ export function toolStreamChunk(delta) {
109
+ if (!_toolStreamOpen)
110
+ return;
111
+ if (_toolStreamChars >= TOOL_STREAM_MAX_CHARS)
112
+ return;
113
+ _toolStreamChars += delta.length;
114
+ if (_toolStreamChars >= TOOL_STREAM_MAX_CHARS) {
115
+ // Include the tail of this chunk up to the cap, then stop rendering live.
116
+ const keep = delta.slice(0, Math.max(0, delta.length - (_toolStreamChars - TOOL_STREAM_MAX_CHARS)));
117
+ if (keep.length > 0) {
118
+ process.stdout.write(`${DIM}${keep}${RESET}`);
119
+ _toolStreamLineOpen = !keep.endsWith("\n");
120
+ }
121
+ process.stdout.write(`\n${DIM}│ (live output truncated at ${TOOL_STREAM_MAX_CHARS} chars — full output in the result)${RESET}`);
122
+ return;
123
+ }
124
+ process.stdout.write(`\n${DIM}│ ${delta.replace(/\n/g, `\n${DIM}│ ${RESET}`)}${RESET}`);
125
+ _toolStreamLineOpen = !delta.endsWith("\n");
126
+ }
127
+ /** Finish the live tool-output block (closes the partial last line). */
128
+ export function toolStreamEnd() {
129
+ if (!_toolStreamOpen)
130
+ return;
131
+ _toolStreamOpen = false;
132
+ process.stdout.write(_toolStreamLineOpen ? `\n${DIM}└${RESET}\n` : `${DIM}└${RESET}\n`);
133
+ }
134
+ /** True while a live tool-output block is open. */
135
+ export function isToolStreamOpen() {
136
+ return _toolStreamOpen;
137
+ }
72
138
  // ── Diff rendering ───────────────────────────────────────────────────────────
73
139
  /** Print a unified diff with colors (+ green, - red, @@ cyan). */
74
140
  export function printDiff(diffText) {
@@ -0,0 +1,251 @@
1
+ import { isIP } from "node:net";
2
+ import { lookup } from "node:dns/promises";
3
+ /**
4
+ * web_fetch — read-only HTTP(S) fetch with SSRF protection.
5
+ *
6
+ * - Only http/https; follows up to 5 redirects, validating every hop.
7
+ * - Private/loopback/link-local addresses are blocked by default
8
+ * (env SCODER_WEB_ALLOW_PRIVATE=1 lifts the block for local dev/testing).
9
+ * - HTML is converted to plain text; JSON is pretty-printed; binary is refused.
10
+ * - Output is capped so a huge page cannot flood the LLM context.
11
+ */
12
+ const FETCH_TIMEOUT_MS = 30_000;
13
+ const MAX_BODY_BYTES = 512 * 1024;
14
+ const MAX_REDIRECTS = 5;
15
+ const DEFAULT_MAX_CHARS = 20_000;
16
+ function allowPrivate() {
17
+ return process.env.SCODER_WEB_ALLOW_PRIVATE === "1";
18
+ }
19
+ /** True if the address is private/loopback/link-local (IPv4 + IPv6). */
20
+ function isPrivateAddress(ip) {
21
+ const v = isIP(ip);
22
+ if (v === 4) {
23
+ const parts = ip.split(".").map(Number);
24
+ const a = parts[0];
25
+ const b = parts[1];
26
+ if (a === 10 || a === 127 || a === 0)
27
+ return true;
28
+ if (a === 172 && b >= 16 && b <= 31)
29
+ return true;
30
+ if (a === 192 && b === 168)
31
+ return true;
32
+ if (a === 169 && b === 254)
33
+ return true; // link-local
34
+ if (a === 100 && b >= 64 && b <= 127)
35
+ return true; // CGNAT
36
+ return false;
37
+ }
38
+ if (v === 6) {
39
+ const lower = ip.toLowerCase();
40
+ if (lower === "::1" || lower === "::")
41
+ return true;
42
+ if (lower.startsWith("fe80"))
43
+ return true; // link-local
44
+ if (/^f[cd][0-9a-f]{2}:/.test(lower))
45
+ return true; // unique local fc00::/7
46
+ // IPv4-mapped ::ffff:10.0.0.1
47
+ const mapped = lower.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/);
48
+ if (mapped)
49
+ return isPrivateAddress(mapped[1]);
50
+ return false;
51
+ }
52
+ return true; // not parseable -> treat as private (fail closed)
53
+ }
54
+ /** Resolve hostname and refuse private addresses (unless explicitly allowed). */
55
+ async function assertPublicHost(hostname) {
56
+ if (allowPrivate())
57
+ return;
58
+ let addresses;
59
+ try {
60
+ addresses = await lookup(hostname, { all: true, verbatim: true });
61
+ }
62
+ catch (err) {
63
+ throw new Error(`DNS lookup failed for "${hostname}": ${err.message}`, { cause: err });
64
+ }
65
+ if (addresses.length === 0) {
66
+ throw new Error(`DNS lookup returned no addresses for "${hostname}"`);
67
+ }
68
+ for (const addr of addresses) {
69
+ if (isPrivateAddress(addr.address)) {
70
+ throw new Error(`Blocked: "${hostname}" resolves to a private/loopback address (${addr.address}). ` +
71
+ `web_fetch only reaches public hosts (set SCODER_WEB_ALLOW_PRIVATE=1 to allow local testing).`);
72
+ }
73
+ }
74
+ }
75
+ /** Decode the HTML entities that actually appear in practice. */
76
+ function decodeEntities(s) {
77
+ const named = {
78
+ amp: "&", lt: "<", gt: ">", quot: '"', apos: "'", nbsp: " ", ndash: "–", mdash: "—",
79
+ hellip: "…", rsquo: "’", lsquo: "‘", rdquo: "”", ldquo: "“", copy: "©", reg: "®",
80
+ };
81
+ return s.replace(/&(#x?[0-9a-fA-F]+|[a-zA-Z]+);/g, (whole, body) => {
82
+ if (body.startsWith("#x") || body.startsWith("#X")) {
83
+ const code = parseInt(body.slice(2), 16);
84
+ return Number.isFinite(code) ? String.fromCodePoint(code) : whole;
85
+ }
86
+ if (body.startsWith("#")) {
87
+ const code = parseInt(body.slice(1), 10);
88
+ return Number.isFinite(code) ? String.fromCodePoint(code) : whole;
89
+ }
90
+ return named[body.toLowerCase()] ?? whole;
91
+ });
92
+ }
93
+ /** Convert HTML to readable plain text (scripts/styles stripped, block tags → newlines). */
94
+ export function htmlToText(html) {
95
+ const out = html
96
+ .replace(/<script[\s\S]*?<\/script\s*>/gi, " ")
97
+ .replace(/<style[\s\S]*?<\/style\s*>/gi, " ")
98
+ .replace(/<noscript[\s\S]*?<\/noscript\s*>/gi, " ")
99
+ .replace(/<!--[\s\S]*?-->/g, " ")
100
+ .replace(/<(br|\/p|\/div|\/section|\/article|\/header|\/footer|\/h[1-6]|\/li|\/tr|\/pre|\/blockquote)[^>]*>/gi, "\n")
101
+ .replace(/<[^>]+>/g, " ")
102
+ .replace(/[ \t]+/g, " ")
103
+ .replace(/ *\n */g, "\n")
104
+ .replace(/\n{3,}/g, "\n\n")
105
+ .trim();
106
+ return decodeEntities(out);
107
+ }
108
+ /** Extract the <title> of an HTML document, if present. */
109
+ function extractTitle(html) {
110
+ const m = html.match(/<title[^>]*>([\s\S]*?)<\/title\s*>/i);
111
+ if (!m)
112
+ return null;
113
+ const t = decodeEntities(m[1]).replace(/\s+/g, " ").trim();
114
+ return t ? t.slice(0, 200) : null;
115
+ }
116
+ /** Read the response body as text, capped at MAX_BODY_BYTES, then cancel the stream. */
117
+ async function readBodyCapped(res) {
118
+ const reader = res.body?.getReader();
119
+ if (!reader)
120
+ return { text: "", truncated: false };
121
+ const decoder = new TextDecoder("utf-8");
122
+ let text = "";
123
+ let received = 0;
124
+ while (true) {
125
+ const { done, value } = await reader.read();
126
+ if (done)
127
+ break;
128
+ received += value.byteLength;
129
+ text += decoder.decode(value, { stream: true });
130
+ if (received >= MAX_BODY_BYTES) {
131
+ try {
132
+ await reader.cancel();
133
+ }
134
+ catch { /* already closed */ }
135
+ return { text: text + decoder.decode(), truncated: true };
136
+ }
137
+ }
138
+ return { text: text + decoder.decode(), truncated: false };
139
+ }
140
+ /** Implementation of the web_fetch tool. Also used by tests via the registry. */
141
+ export async function toolWebFetch(ctx, args) {
142
+ const maxChars = Math.min(Math.max(args.max_chars ?? DEFAULT_MAX_CHARS, 500), 100_000);
143
+ let url;
144
+ try {
145
+ url = new URL(args.url);
146
+ }
147
+ catch {
148
+ return { ok: false, output: `ERROR: invalid URL: ${args.url}` };
149
+ }
150
+ if (url.protocol !== "http:" && url.protocol !== "https:") {
151
+ return { ok: false, output: `ERROR: only http/https URLs are supported (got ${url.protocol})` };
152
+ }
153
+ // Combined timeout + external abort (SIGINT) — manual composition for Node 18 compat
154
+ const controller = new AbortController();
155
+ const timer = setTimeout(() => controller.abort(new Error(`timeout after ${FETCH_TIMEOUT_MS / 1000}s`)), FETCH_TIMEOUT_MS);
156
+ const onExternalAbort = () => controller.abort(new Error("interrupted by user"));
157
+ if (ctx.signal?.aborted)
158
+ onExternalAbort();
159
+ else
160
+ ctx.signal?.addEventListener("abort", onExternalAbort, { once: true });
161
+ const signal = controller.signal;
162
+ let current = url;
163
+ try {
164
+ // Redirect loop with per-hop SSRF validation
165
+ for (let hop = 0; hop <= MAX_REDIRECTS; hop++) {
166
+ if (signal.aborted) {
167
+ return { ok: false, output: `ERROR: fetch aborted (${signal.reason?.message ?? "aborted"})` };
168
+ }
169
+ await assertPublicHost(current.hostname);
170
+ const res = await fetch(current, {
171
+ method: "GET",
172
+ redirect: "manual",
173
+ headers: {
174
+ "User-Agent": "scoder-agent (+https://github.com/nbhson/app-scoder-agent)",
175
+ Accept: "text/html, application/json, text/plain;q=0.9, */*;q=0.1",
176
+ },
177
+ signal,
178
+ });
179
+ // Follow redirects manually so every hop is validated against SSRF
180
+ if (res.status >= 300 && res.status < 400) {
181
+ const loc = res.headers.get("location");
182
+ try {
183
+ await res.body?.cancel();
184
+ }
185
+ catch { /* ignore */ }
186
+ if (!loc) {
187
+ return { ok: false, output: `ERROR: ${res.status} redirect without Location header at ${current.href}` };
188
+ }
189
+ if (hop === MAX_REDIRECTS) {
190
+ return { ok: false, output: `ERROR: too many redirects (>${MAX_REDIRECTS}) fetching ${args.url}` };
191
+ }
192
+ current = new URL(loc, current);
193
+ continue;
194
+ }
195
+ if (!res.ok) {
196
+ try {
197
+ await res.body?.cancel();
198
+ }
199
+ catch { /* ignore */ }
200
+ return { ok: false, output: `ERROR: HTTP ${res.status} ${res.statusText} for ${current.href}` };
201
+ }
202
+ const contentType = res.headers.get("content-type") ?? "";
203
+ if (/^(image|audio|video|font|application\/(octet-stream|zip|gzip|pdf|wasm))/i.test(contentType)) {
204
+ try {
205
+ await res.body?.cancel();
206
+ }
207
+ catch { /* ignore */ }
208
+ return { ok: false, output: `ERROR: unsupported content-type "${contentType}" — web_fetch returns text only (html/json/plain).` };
209
+ }
210
+ const { text: body, truncated } = await readBodyCapped(res);
211
+ let text;
212
+ if (/json/i.test(contentType)) {
213
+ text = prettyJson(body) ?? body;
214
+ }
215
+ else if (/html/i.test(contentType) || /^\s*</.test(body)) {
216
+ text = htmlToText(body);
217
+ }
218
+ else {
219
+ text = body;
220
+ }
221
+ const title = /text\/html/i.test(contentType) ? extractTitle(body) : null;
222
+ const meta = [
223
+ `HTTP ${res.status} ${res.statusText || ""}`.trimEnd(),
224
+ contentType || "unknown content-type",
225
+ truncated ? `body truncated at ${MAX_BODY_BYTES / 1024}KB` : null,
226
+ ].filter(Boolean).join(", ");
227
+ const out = `${text.slice(0, maxChars)}${text.length > maxChars ? `\n... [truncated, ${text.length - maxChars} more chars]` : ""}`;
228
+ return {
229
+ ok: true,
230
+ output: `${title ? `Title: ${title}\n` : ""}${current.href}\n(${meta})\n\n${out}`,
231
+ };
232
+ }
233
+ // unreachable
234
+ return { ok: false, output: "ERROR: redirect loop" };
235
+ }
236
+ catch (err) {
237
+ return { ok: false, output: `ERROR: fetch failed for ${current.href}: ${err.message}` };
238
+ }
239
+ finally {
240
+ clearTimeout(timer);
241
+ ctx.signal?.removeEventListener("abort", onExternalAbort);
242
+ }
243
+ }
244
+ function prettyJson(body) {
245
+ try {
246
+ return JSON.stringify(JSON.parse(body), null, 2);
247
+ }
248
+ catch {
249
+ return undefined;
250
+ }
251
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "s-app-scoder-agent",
3
- "version": "0.4.0",
3
+ "version": "0.5.0",
4
4
  "description": "Autonomous coding agent CLI - similar to opencode/cline/claude. Runs tasks until build succeeds.",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",