@robzilla1738/agentswarm 0.3.0 → 0.6.0

package/README.md

@@ -1,8 +1,16 @@
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset=".github/assets/swarm-mark-light.png">
+    <img src=".github/assets/swarm-mark-dark.png" alt="agentswarm" width="120">
+  </picture>
+</p>
+
 # agentswarm
 
 [![npm](https://img.shields.io/npm/v/@robzilla1738/agentswarm)](https://www.npmjs.com/package/@robzilla1738/agentswarm)
 [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
 [![node](https://img.shields.io/badge/node-%E2%89%A520.10-brightgreen)](package.json)
+[![support](https://img.shields.io/badge/support-buy%20me%20a%20coffee-yellow)](https://buymeacoffee.com/robcourson)
 
 A local agent-swarm orchestrator with a terminal dashboard and a localhost web UI. Works with DeepSeek, OpenAI, Anthropic, xAI, MiniMax, OpenRouter, Ollama, LM Studio, or any OpenAI-compatible endpoint.
 
@@ -21,7 +29,7 @@ You give it a mission. A conductor model breaks the mission into tasks and hands
         │  T4 dep  │◀─────│ verify  │   adversarial verification
         └────┬─────┘      └─────────┘
         ┌────▼─────┐
-        │Synthesize│  → final-report.md + artifacts
+        │Synthesize│  → final report (.md + .html) + artifacts
         └──────────┘
 ```
 
@@ -87,23 +95,45 @@ swarm run "Research the best open-source vector DBs in 2026 and write a recommen
 | `swarm cancel <id>` | Stop a run. It still synthesizes a report from completed work. |
 | `swarm config [list\|get\|set …]` | Manage `~/.agentswarm/config.json`. |
 | `swarm models` | List models from the active provider. |
+| `swarm config unset <key>` | Remove a setting (e.g., `swarm config unset firecrawlApiKey`). |
 | `swarm demo` | Run a self-contained demo mission in an isolated workspace. |
 
 Run options (also on the UI launch form under Options): `--workers N` (parallelism), `--tasks N`, `--steps N` (tool steps per task), `--budget N` (token cap), `--model`, `--conductor`, `--verify off|normal|strict`, `--effort low|medium|high|max`, `--no-thinking`, `--sandbox host|docker|e2b|modal|vercel|auto` (shell runtime for this run), `--cwd <path>` (run against a real directory instead of an isolated workspace), `--fg` (foreground in this process).
 
 ## How it works
 
-The conductor is a model with three tools: `spawn_tasks`, `wait`, and `finish`. It reads the mission, spawns self-contained tasks (each with an objective, success criteria, a role, optional dependencies, and an optional `verify` flag), then reacts as reports come back.
+The conductor is a model with six tools: `spawn_tasks`, `set_phase`, `update_plan`, `read_report`, `wait`, and `finish`. It reads the mission, spawns self-contained tasks (each with an objective, success criteria, a role, optional dependencies, and an optional `verify` flag), then reacts as reports come back. On long missions it declares phases (`set_phase`) whose goals and exit criteria are pinned into every update — so the plan survives even when old history is trimmed and replaced by a mission ledger (settled tasks, decisions, current phase). On resume, the conductor is re-seeded with this ledger so it picks up where it left off without losing context.
+
+Each task becomes an autonomous agent with a tool budget. It works in small steps, posts durable findings to the blackboard (decisions are never trimmed from digests; `search_notes` now supports `kind` filters to find decisions, context, or source links without noise), journals progress checkpoints on long tasks, saves artifacts, and ends by reporting back with structured handoff fields (`key_facts`, `open_questions`, `files_touched`) plus any sources discovered. Sources flow through to the final report as numbered citations — every source is deduplicated, attributed, and linked inline (`[1]`) with a full bibliography at the end. Dependent tasks receive report excerpts plus those fields, and can pull full text with `read_report`.
+
+**Search & research.** Web search now includes engine rate-limit cooldowns (on a 429, the engine skips it for a while and re-plans); queries reformulate themselves down to keywords if they get zero results (lifting recall without noise); results are freshness-ranked so recent content bubbles up. For academic queries, agents can use `academic_search` to query arXiv and Crossref directly — no API key needed. Fetches from the web pull plain text via `fetch_url`, which extracts text from PDFs (zero runtime dependencies, zlib only), decodes non-UTF-8 charsets, and flags paywall shells so agents know when they hit a wall.
+
+**Scale.** A global AIMD limiter (`maxConcurrentCalls`) bounds concurrent model calls per endpoint — a 429 halves the ceiling, successes recover it, and conductor calls always jump the queue, so a 100-agent swarm degrades gracefully instead of melting down. Settles are debounced before waking the conductor; on big runs the task table collapses settled waves (failures stay itemized) and excess reports become one-liners the conductor can expand with `read_report`. Spawn specs take a `model` tier (`cheap` for scouts, `strong` for leads/verifiers via `cheapModel`/`strongModel` config) and `team:true` to run a task as a full sub-swarm — its own conductor decomposes it in parallel and reports one consolidated result, with all activity journaled under its `teamId`. Context windows are configurable per model via `contextWindows` config; the engine respects each model's actual limit and compacts agent context accordingly.
+
+**Worker tools.** The toolbelt gained `grep_files` for structured content search and `replace_in_file` with atomic multi-edit batches — both portable across sandboxes (Docker, E2B, Modal, Vercel).
 
-Each task becomes an autonomous agent with a tool budget. It works in small steps, posts durable findings to the blackboard, saves artifacts, and ends by reporting back. The report is the only thing the conductor sees, which keeps reports specific.
+**Verification & quality.** Tasks pass a mechanical format pre-check (JSON/CSV/HTML structure), then a blind LLM verifier with its own tools. Failed verifications retry with structured feedback (problem/evidence/fix). The verifier gets copies of all dependencies' reports for context. In `--verify strict` mode, the verifier must back verdicts with tool-gathered evidence (not just a pass statement), a completeness critic reviews the whole run for gaps before synthesis, and the final report is checked for faithfulness against the task reports.
+
+**Long horizon.** The conductor maintains a living `mission-plan.md` (`update_plan`) pinned into every update and restored on resume; every 25 settled tasks a progress snapshot lands in `artifacts/` so multi-day runs always have a partial deliverable; and real-directory runs leave a memory (`~/.agentswarm/memory/`) of missions, outcomes, and decisions that seeds the next swarm in the same workspace. When tasks fail, the cascade carries the root cause transitively — blocked tasks know why rather than just "dependency did not complete". Failed tasks surface their last failing tool call as diagnostics.
+
+**Planning & steering.** The UI now includes a Plan tab showing the living `mission-plan.md`, and the conductor can update it from an agent note (`swarm note <id> "update the plan: ..."`). The budget sparkline in the run dashboard shows at-a-glance how much token budget remains.
 
 The scheduler starts a task as soon as its dependencies are done, up to the parallelism cap. Tasks whose dependencies failed are blocked and surfaced to the conductor for re-planning.
 
-When the conductor finishes (or the budget forces it), a synthesizer composes `final-report.md` from every task report.
+When the conductor finishes (or the budget forces it), a synthesizer composes the final deliverable from every task report. Deliverables ship in the format the mission calls for — code, `.csv`/`.json` data, styled documents — alongside `final-report.md` and a self-contained `final-report.html` rendering (open it with `swarm report <id> --open`). The final report includes an inline-cited Sources section and all findings are preserved.
 
 The journal is the source of truth. Every run is an append-only `events.jsonl`; the terminal dashboard, the web UI, and `swarm ls` all reduce the same file. That's why runs survive crashes and can be resumed or replayed. Runs live under `~/.agentswarm/runs/<id>/`.
 
-If the engine process dies without writing a terminal status (kill -9, reboot), the hub notices the missing process and shows the run as interrupted instead of leaving it "running" forever.
+If the engine process dies without writing a terminal status (kill -9, reboot), the hub notices the missing process and shows the run as interrupted instead of leaving it "running" forever. `swarm resume <id>` continues it: settled tasks keep their results, and tasks that were mid-flight restart *warm* from their last journaled checkpoint instead of from scratch. SIGTERM flushes the journal synchronously and leaves the run resumable.
+
+## Troubleshooting
+
+- **"interrupted — the engine process is no longer running"** — the engine died without a terminal status (kill -9, reboot, crash). Check `~/.agentswarm/runs/<id>/exec.log` for the crash output, then `swarm resume <id>`.
+- **Run ended with "conductor unavailable"** — five consecutive conductor API calls failed (after backoff). Usually a provider outage or a bad model name; check the run's activity log for the underlying error, fix, and resume.
+- **"journal writes are failing"** — the engine could not append to `events.jsonl` (disk full, permissions). The run aborts deliberately rather than doing unrecorded work.
+- **A verified task keeps failing with "Claimed artifact(s) do not exist"** — the worker reported files it never wrote. That's the mechanical pre-verifier doing its job; the retry prompt tells the worker to actually create them.
+- **Docker sandbox fails to start** — confirm `docker info` works as your user, and that the configured `sandboxImage` can be pulled. `swarm sandbox test` checks the configured runtime end-to-end.
+- **Hung or wedged run** — `swarm cancel <id>` aborts in-flight agents within ~1s; sandbox teardown is bounded by a 15s timeout so it can't hang shutdown.
 
 ## Architecture
 
@@ -114,15 +144,24 @@ src/                         TypeScript engine (zero runtime deps)
   sandbox.ts    sandbox runtimes: host, docker, E2B, Modal, Vercel
   agent.ts      the agent loop: stream → tool calls → results → repeat, with compaction
   executor.ts   the orchestrator: conductor loop, parallel scheduler, verify, synth, budget
-  tools.ts      worker toolbelt (shell, files, web, blackboard, artifacts) + safety
-  webtools.ts   web search/fetch: SearchKit → TinyFish → DuckDuckGo fallback chain
+  tools.ts      worker toolbelt (shell, files, web, blackboard, artifacts) + safety + grep/replace
+  webtools.ts   web search/fetch: SearchKit → TinyFish → DuckDuckGo fallback chain, with cooldowns + reformulation
+  searchcore.ts search ranking (freshness boost, academic intent detection) + academic_search (arXiv/Crossref)
+  pdftext.ts    PDF text extraction (zero deps, zlib only)
+  crawltools.ts crawl backend resolver (firecrawl/context.dev/deepcrawl)
   journal.ts    append-only crash-safe event log (single source of truth)
-  state.ts      pure reducer: events → live run state
-  hub.ts        localhost HTTP API + SSE + static UI server
+  state.ts      pure reducer: events → live run state (with budgetSeries sampling)
+  hub.ts        localhost HTTP API + SSE + static UI server (CORS locked to localhost)
   terminal.ts   live TTY dashboard
   cli.ts        command-line interface
+  memory.ts     atomic runId-keyed cross-run memory + interim snapshots
 ui/             Next.js 15 + Tailwind 4 web app (static-exported, served by the hub)
+  components/SideRail Plan tab showing mission-plan.md
+  app/run/page.tsx Blackboard search with kind filters + budget sparkline
+  app/settings/page.tsx Test buttons for crawl/search backends, key management
 test/           end-to-end test with a scripted mock model (no API key needed)
+  e2e.js        21 phases covering the full pipeline, including citations + force + resume + budget + verify + teams
+  unit/*.test.js individual suites for tools, crawl, memory, pdftext, webtools, searchcore, citations
 ```
 
 ## Testing
@@ -135,11 +174,12 @@ Boots a mock model server and drives real missions through the engine, offline,
 
 ## Safety notes
 
-- Safe mode is on by default. It blocks obviously destructive shell commands and confines writes to the working directory. `--no-safe` turns it off for a run; only do that when you trust the mission.
+- Safe mode is on by default. It blocks obviously destructive shell commands and confines writes to the working directory, plus symlink escapes to parent directories. `--no-safe` turns it off for a run; only do that when you trust the mission.
+- The hub API (started by `swarm serve`) only accepts requests from localhost origins (`http://localhost:*` and `127.0.0.1:*`). The web UI runs in your browser locally and never phones home.
 - Runs default to an isolated per-run workspace on this machine. That's a private directory, not a container. Agents still execute with your user's permissions; the engine strips API keys and sandbox credentials from their environment, and safe mode constrains commands and writes. For untrusted or risky missions, use `--sandbox docker` or a cloud runtime.
 - Use `--cwd <path>` (or Workspace → "A directory on disk" in the UI) to let agents touch a real project. Those runs always execute on the host, since touching your real files is the point.
 - Costs are estimates based on list prices and the token counts the API reports. Models without pricing data show $0. Set a `--budget` either way.
-- Keys are stored in `~/.agentswarm/config.json` (chmod 600) and are only sent to the APIs you configured.
+- Keys are stored in `~/.agentswarm/config.json` (chmod 600) and are only sent to the APIs you configured. Use `swarm config unset <key>` to remove a key, or the Settings UI for test buttons on crawl/search backends.
 
 ## Author
 

package/dist/agent.js

@@ -2,6 +2,7 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.runAgent = runAgent;
 exports.estimateMessages = estimateMessages;
+const config_1 = require("./config");
 const deepseek_1 = require("./deepseek");
 const prompts_1 = require("./prompts");
 const types_1 = require("./types");
@@ -58,7 +59,20 @@ async function runAgent(p) {
         if (stopReason)
             break;
         steps++;
-        const res = await callModel();
+        let res;
+        try {
+            res = await callModel();
+        }
+        catch (e) {
+            // The chat client already retries 429/5xx; this catches the rest of the
+            // transient class (connection resets, DNS blips) once per step so a
+            // single network hiccup doesn't burn a whole task attempt.
+            if (p.signal.aborted)
+                throw e;
+            hooks.onLog?.("warn", `${p.agentId}: model call failed (${(0, util_1.errMsg)(e)}); retrying once`);
+            await new Promise((r) => setTimeout(r, 1500));
+            res = await callModel();
+        }
         hooks.onUsage?.(p.model, res.usage);
         usage = (0, types_1.addUsage)(usage, res.usage);
         if (res.toolCalls.length === 0) {
@@ -129,7 +143,7 @@ async function runAgent(p) {
             messages.push({ role: "tool", tool_call_id: call.id, content: result });
         }
         hooks.onTranscript?.(messages);
-        if (estimateMessages(messages) > cfg.contextTokenLimit) {
+        if (estimateMessages(messages) > (0, config_1.contextLimitFor)(cfg, p.model)) {
             messages = await compact(p, messages);
             hooks.onTranscript?.(messages);
             hooks.onLog?.("info", `${p.agentId}: context compacted`);
@@ -211,6 +225,8 @@ async function compact(p, messages) {
         });
         p.hooks.onUsage?.(p.model, res.usage);
         summary = res.content || "(compaction produced no summary)";
+        if (res.content)
+            p.hooks.onCheckpoint?.(res.content);
     }
     catch (e) {
         // Compaction is best-effort; fall back to hard truncation.

package/dist/cli.js

@@ -342,11 +342,24 @@ async function execForeground(cfg, meta, render, resume = false) {
     };
     process.on("uncaughtException", onFatal);
     process.on("unhandledRejection", onFatal);
+    // SIGTERM (kill, system shutdown): flush buffered journal lines synchronously
+    // and exit WITHOUT a terminal status — the run stays resumable, and viewers
+    // show it as interrupted once the pid disappears.
+    const onTerm = () => {
+        journal.append("log", { level: "warn", msg: "engine received SIGTERM — exiting; resume with: swarm resume " + meta.id });
+        journal.flushSync();
+        (0, run_1.clearPid)(meta.id);
+        if (renderer)
+            renderer.stop();
+        process.exit(143);
+    };
+    process.on("SIGTERM", onTerm);
     try {
         await executor.run();
     }
     finally {
         process.off("SIGINT", onSig);
+        process.off("SIGTERM", onTerm);
         process.off("uncaughtException", onFatal);
         process.off("unhandledRejection", onFatal);
         (0, run_1.clearPid)(meta.id);
@@ -476,8 +489,10 @@ function cmdReport(id, flags) {
         process.exit(1);
     }
     if (flags.open) {
-        openBrowser("file://" + file);
-        console.log(file);
+        const html = path.join((0, config_1.runDir)(id), "artifacts", "final-report.html");
+        const target = fs.existsSync(html) ? html : file;
+        openBrowser("file://" + target);
+        console.log(target);
         return;
     }
     process.stdout.write(fs.readFileSync(file, "utf8") + "\n");
@@ -509,15 +524,18 @@ async function cmdConfig(rest, flags) {
         const cfg = (0, config_1.loadConfig)();
         if (sub === "get" && rest[1]) {
             const key = rest[1];
-            const v = key === "apiKey" || key === "tinyfishApiKey" ? (0, config_1.maskKey)(String(cfg[key])) : cfg[key];
+            const v = /apikey|token|secret/i.test(key) ? (0, config_1.maskKey)(String(cfg[key] ?? "")) : cfg[key];
             console.log(typeof v === "object" ? JSON.stringify(v, null, 2) : String(v));
             return;
         }
         console.log(util_1.ansi.bold("config") + util_1.ansi.gray(`  (${(0, config_1.configPath)()})`));
         for (const k of config_1.SETTABLE_KEYS) {
             let v = cfg[k];
-            if (k === "apiKey" || k === "tinyfishApiKey")
-                v = v ? (0, config_1.maskKey)(String(v)) : util_1.ansi.red("(not set)");
+            // Every secret-bearing key prints masked — `config list` output ends up
+            // in terminal scrollback and pasted bug reports.
+            if (/apikey|token|secret/i.test(k)) {
+                v = v ? (0, config_1.maskKey)(String(v)) : k === "apiKey" ? util_1.ansi.red("(not set)") : "(not set)";
+            }
             console.log(`  ${k.padEnd(18)} ${util_1.ansi.gray(String(v))}`);
         }
         return;
@@ -544,11 +562,25 @@ async function cmdConfig(rest, flags) {
             console.log(util_1.ansi.gray("  verify it works: ") + "swarm models");
         return;
     }
+    if (sub === "unset") {
+        const key = rest[1];
+        if (!key)
+            throw new Error("usage: swarm config unset <key>");
+        // Only string-valued keys can sensibly clear to "" — numbers/enums keep
+        // their defaults via `set`.
+        const clearable = config_1.SETTABLE_KEYS.filter((k) => /apikey|token|secret|url|model/i.test(k));
+        if (!clearable.includes(key)) {
+            throw new Error(`not clearable. Clearable keys: ${clearable.join(", ")}`);
+        }
+        (0, config_1.saveConfig)({ [key]: "" });
+        console.log(util_1.ansi.green("✓ ") + `cleared ${key}`);
+        return;
+    }
     if (sub === "path") {
         console.log((0, config_1.configPath)());
         return;
     }
-    throw new Error("usage: swarm config [list|get <key>|set <key> <value>|path]");
+    throw new Error("usage: swarm config [list|get <key>|set <key> <value>|unset <key>|path]");
 }
 async function cmdModels() {
     const cfg = (0, config_1.loadConfig)();
@@ -614,7 +646,7 @@ function printFinalLine(id) {
     console.log("");
     if (fs.existsSync(reportFile)) {
         console.log(util_1.ansi.green("✓ final report: ") + reportFile);
-        console.log(util_1.ansi.gray("  view: ") + `swarm report ${id}`);
+        console.log(util_1.ansi.gray("  view: ") + `swarm report ${id}` + util_1.ansi.gray("  ·  open in browser: ") + `swarm report ${id} --open`);
     }
     else {
         console.log(util_1.ansi.gray(`run ${id} ended without a final report (see: swarm watch ${id})`));
@@ -661,7 +693,6 @@ ${b("RUN OPTIONS")}
 ${b("FIRST RUN")}
   swarm config set apiKey <key>             # key for the active provider (default: DeepSeek)
   swarm config set provider <id>            # deepseek | openai | anthropic | xai | minimax | openrouter | ollama | lmstudio | custom
-  pip install searchkit                     # optional: local, citable web search for agents
   swarm serve --open                        # open the web UI
 `);
 }

package/dist/config.js

@@ -33,7 +33,8 @@ var __importStar = (this && this.__importStar) || (function () {
     };
 })();
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.SETTABLE_KEYS = exports.SECRET_ENV_KEYS = exports.DEFAULTS = exports.DEFAULT_PRICING = void 0;
+exports.SETTABLE_KEYS = exports.SECRET_ENV_KEYS = exports.DEFAULTS = exports.DEFAULT_WINDOWS = exports.DEFAULT_PRICING = void 0;
+exports.contextLimitFor = contextLimitFor;
 exports.home = home;
 exports.runsDir = runsDir;
 exports.runDir = runDir;
@@ -63,6 +64,20 @@ exports.DEFAULT_PRICING = {
     "MiniMax-M2.1": { inMiss: 0.3, inHit: 0.03, out: 1.2 },
     "MiniMax-M2": { inMiss: 0.3, inHit: 0.03, out: 1.2 },
 };
+exports.DEFAULT_WINDOWS = {
+    // tokens (June 2026 published limits; conservative where ranges exist)
+    "deepseek-v4-flash": 128_000,
+    "deepseek-v4-pro": 128_000,
+    "deepseek-chat": 128_000,
+    "deepseek-reasoner": 128_000,
+    "gpt-5.1": 272_000,
+    "gpt-5.1-mini": 272_000,
+    "claude-opus-4-8": 200_000,
+    "claude-sonnet-4-6": 200_000,
+    "claude-haiku-4-5": 200_000,
+    "MiniMax-M2.1": 192_000,
+    "MiniMax-M2": 192_000,
+};
 exports.DEFAULTS = {
     provider: "deepseek",
     providers: {},
@@ -70,17 +85,24 @@ exports.DEFAULTS = {
     baseUrl: providers_1.PROVIDERS.deepseek.baseUrl,
     model: "deepseek-v4-flash",
     conductorModel: "deepseek-v4-flash",
+    cheapModel: "",
+    strongModel: "",
     maxWorkers: 6,
     maxStepsPerTask: 30,
-    maxTasks: 48,
+    maxTasks: 200,
     maxTokensPerRun: 12_000_000,
     verification: "normal",
+    verifyMaxAttempts: 2,
     thinking: true,
     reasoningEffort: "high",
     safeMode: true,
     tinyfishApiKey: "",
     searchBackend: "auto",
-    searchkitCmd: "searchkit",
+    firecrawlApiKey: "",
+    contextdevApiKey: "",
+    deepcrawlApiKey: "",
+    deepcrawlBaseUrl: "",
+    crawlBackend: "auto",
     sandboxRuntime: "host",
     sandboxImage: "node:22-bookworm",
     e2bApiKey: "",
@@ -90,14 +112,25 @@ exports.DEFAULTS = {
     vercelToken: "",
     vercelTeamId: "",
     vercelProjectId: "",
+    maxConcurrentCalls: 16,
     requestTimeoutMs: 900_000,
     idleTimeoutMs: 180_000,
     contextTokenLimit: 120_000,
-    maxToolResultChars: 12_000,
+    maxToolResultChars: 20_000,
     hubPort: 7777,
     uiPort: 7780,
     pricing: exports.DEFAULT_PRICING,
+    contextWindows: exports.DEFAULT_WINDOWS,
 };
+/**
+ * Effective compaction/trim threshold for a model: the configured limit,
+ * hard-capped by the model's known context window (15% headroom for output
+ * and estimation error). Models we don't know keep the configured limit.
+ */
+function contextLimitFor(cfg, model) {
+    const win = cfg.contextWindows[model];
+    return win ? Math.min(cfg.contextTokenLimit, Math.floor(win * 0.85)) : cfg.contextTokenLimit;
+}
 /**
  * Env vars that must never leak into agent shell commands when they execute
  * directly on the host: every provider key env plus the search/sandbox
@@ -109,6 +142,9 @@ exports.SECRET_ENV_KEYS = [
             .map((p) => p.keyEnv)
             .filter((k) => Boolean(k)),
         "TINYFISH_API_KEY",
+        "FIRECRAWL_API_KEY",
+        "CONTEXT_DEV_API_KEY",
+        "DEEPCRAWL_API_KEY",
         "E2B_API_KEY",
         "MODAL_TOKEN_ID",
         "MODAL_TOKEN_SECRET",
@@ -152,6 +188,7 @@ function loadConfig() {
         provider,
         providers: file.providers || {},
         pricing: { ...exports.DEFAULT_PRICING, ...(file.pricing || {}) },
+        contextWindows: { ...exports.DEFAULT_WINDOWS, ...(file.contextWindows || {}) },
         apiKey: cred.apiKey || "",
         baseUrl: cred.baseUrl || info.baseUrl,
     };
@@ -160,6 +197,14 @@ function loadConfig() {
         cfg.apiKey = process.env[info.keyEnv];
     if (process.env.TINYFISH_API_KEY)
         cfg.tinyfishApiKey = process.env.TINYFISH_API_KEY;
+    if (process.env.FIRECRAWL_API_KEY)
+        cfg.firecrawlApiKey = process.env.FIRECRAWL_API_KEY;
+    if (process.env.CONTEXT_DEV_API_KEY)
+        cfg.contextdevApiKey = process.env.CONTEXT_DEV_API_KEY;
+    if (process.env.DEEPCRAWL_API_KEY)
+        cfg.deepcrawlApiKey = process.env.DEEPCRAWL_API_KEY;
+    if (process.env.DEEPCRAWL_BASE_URL)
+        cfg.deepcrawlBaseUrl = process.env.DEEPCRAWL_BASE_URL;
     if (process.env.E2B_API_KEY)
         cfg.e2bApiKey = process.env.E2B_API_KEY;
     if (process.env.MODAL_TOKEN_ID)
@@ -218,17 +263,24 @@ exports.SETTABLE_KEYS = [
     "baseUrl",
     "model",
     "conductorModel",
+    "cheapModel",
+    "strongModel",
     "maxWorkers",
     "maxStepsPerTask",
     "maxTasks",
     "maxTokensPerRun",
     "verification",
+    "verifyMaxAttempts",
     "thinking",
     "reasoningEffort",
     "safeMode",
     "tinyfishApiKey",
     "searchBackend",
-    "searchkitCmd",
+    "firecrawlApiKey",
+    "contextdevApiKey",
+    "deepcrawlApiKey",
+    "deepcrawlBaseUrl",
+    "crawlBackend",
     "sandboxRuntime",
     "sandboxImage",
     "e2bApiKey",
@@ -238,15 +290,18 @@ exports.SETTABLE_KEYS = [
     "vercelToken",
     "vercelTeamId",
     "vercelProjectId",
+    "maxConcurrentCalls",
     "contextTokenLimit",
     "hubPort",
     "uiPort",
 ];
 /** Allowed ranges for numeric settings (values are clamped, not rejected). */
 const NUM_RANGES = {
-    maxWorkers: [1, 32],
+    maxWorkers: [1, 128],
+    maxConcurrentCalls: [1, 256],
     maxStepsPerTask: [3, 200],
     maxTasks: [1, 1000],
+    verifyMaxAttempts: [1, 5],
     maxTokensPerRun: [50_000, 2_000_000_000],
     contextTokenLimit: [8_000, 900_000],
     hubPort: [0, 65535],
@@ -256,6 +311,7 @@ const ENUMS = {
     verification: ["off", "normal", "strict"],
     reasoningEffort: ["low", "medium", "high", "max"],
     searchBackend: ["auto", "tinyfish", "ddg"],
+    crawlBackend: ["auto", "firecrawl", "contextdev", "deepcrawl", "off"],
     sandboxRuntime: ["auto", "host", "docker", "e2b", "modal", "vercel"],
     provider: Object.keys(providers_1.PROVIDERS),
 };