niceeval 0.1.0

package/README.md

@@ -0,0 +1,164 @@
+<div align="center">
+
+# NiceEval
+
+**Progressive, full-featured, excellent DX lightweight ai agent evals tool**
+
+[![typescript](https://img.shields.io/badge/typescript-5.6-blue?style=flat-square)](tsconfig.json)
+[![license](https://img.shields.io/badge/license-MIT-green?style=flat-square)](package.json)
+[![docs](https://img.shields.io/badge/docs-readable-111827?style=flat-square)](docs/README.md)
+
+[中文](README.zh.md)
+
+</div>
+
+NiceEval is a general-purpose agent eval tool inspired by [eve](https://eve.dev). It has an excellent DX design — anyone can get started and configured in about 10 minutes. It's also very versatile: it can eval plugins, Hooks, and Skills written for Claude Code/Codex coding agents, and can directly eval your own AI Agent framework (no matter if it's based on AI SDK, LangGraph, Pi, or any other interface, it's easy to integrate).
+
+After the eval completes, it generates readable reports and lets you view agent behavior details. Convenient for debugging and optimization.
+
+## Why NiceEval when DeepEval, LangFuse, and BrainTrust already exist
+
+NiceEval is an AI-native eval tool. In tools built around Dataset/golden-style Input vs. Expected Output, that shape doesn't fit real agent evaluation well. NiceEval is built for evaluating agents at a finer grain — multi-turn conversations, multi-agent setups, tool calls, skill loading, and more.
+
+It also coexists with LangFuse and BrainTrust: use them for tracing, or upload eval results to both (in progress).
+
+## Architecture
+
+NiceEval supports two integration modes, depending on whether the system under test needs an isolated sandbox filesystem.
+
+**Mode 1: Sandbox (Docker, E2B) — run coding agents like Codex and Claude Code that need a sandbox**
+
+```text
+   evals/*.eval.ts
+        │
+        ▼
+   ┌─────────────────────┐
+   │     NiceEval        │
+   └─────────────────────┘
+        │
+        │ Agent adapter (official)
+        ▼
+   ┌──────────────────────────────┐
+   │        Docker Sandbox         │
+   │   ┌────────────────────────┐  │
+   │   │ Codex / Claude Code /  │  │
+   │   │ apps needing isolation │  │
+   │   └────────────────────────┘  │
+   └──────────────────────────────┘
+```
+
+**Mode 2: Direct — connect straight to your own AI Agent**
+
+```text
+   evals/*.eval.ts
+        │
+        ▼
+   ┌─────────────────────┐
+   │     NiceEval        │
+   └─────────────────────┘
+        │
+        │ Agent adapter (official, or your own implementation)
+        ▼
+   ┌──────────────────────────────┐
+   │       your own Web Agent      │
+   │   (HTTP / AI SDK·LangGraph·   │
+   │    Pi and other frameworks —  │
+   │         no Docker needed)     │
+   └──────────────────────────────┘
+```
+
+- **NiceEval core** owns discovery, scheduling, scoring, reporting, and artifacts.
+- **Agent adapters** are the open boundary: you decide how to call the system under test.
+- Coding agents that need filesystem isolation run inside the **Docker Sandbox**; your own Web Agent can connect directly, without Docker.
+
+
+## Example
+
+Running an eval takes two files: the eval itself (what to check) and an experiment (which agent to run it against). The CLI won't run a bare eval id — the experiment in `niceeval exp <experiment> <eval prefix>` is what picks the system under test. Here's a real eval against a directly-connected web agent (full project in [`examples/zh/ai-sdk/`](examples/zh/ai-sdk/)), checking that the agent calls a tool for live weather questions and answers from the tool result instead of making it up:
+
+```ts
+// evals/eval-tool-call.eval.ts
+import { defineEval } from "niceeval";
+
+export default defineEval({
+  description: "Verify the agent calls the weather tool and answers from its result",
+
+  async test(t) {
+    const turn = await t.send("What's the weather in Beijing today?");
+    t.succeeded();
+
+    await t.group("calls get_weather with the right city", () => {
+      t.calledTool("get_weather", { input: { city: "Beijing" } });
+      t.messageIncludes(/°C|sunny|cloudy|rain/);
+    });
+
+    const second = await t.send("What about Shanghai tomorrow?");
+    second.messageIncludes("Shanghai");
+
+    t.judge.autoevals
+      .closedQA("Does the reply use the tool's weather data instead of making up a temperature?")
+      .atLeast(0.7);
+  },
+});
+```
+
+```ts
+// experiments/local.ts
+import { defineExperiment } from "niceeval";
+import { webAgent } from "./adapter"; // your agent adapter, pointed at the system under test
+
+export default defineExperiment({
+  agent: webAgent({ baseUrl: "http://127.0.0.1:5188" }),
+});
+```
+
+```sh
+npx niceeval exp local eval-tool-call  // run only eval-tool-call under the local experiment
+npx niceeval view
+```
+
+For coding agents that need an isolated workspace (Codex, Claude Code plugins/skills), see [`examples/zh/coding-agent-skill/`](examples/zh/coding-agent-skill/): evals there use `t.sandbox.uploadDirectory()` to seed the workspace, `t.fileChanged()` / `t.file()` to check what changed, and `t.sandbox.runCommand()` to run tests.
+
+## Quick Start
+
+```text
+READ https://raw.githubusercontent.com/CorrectRoadH/niceeval/refs/heads/main/INIT.md and install niceeval for this repo.
+```
+
+Start from the scenario that matches what you need to evaluate:
+
+- [Claude Code / Codex plugin eval](https://niceeval.com/docs/zh/example/claude-code-codex-plugin)
+- [Claude Code / Codex skill eval](https://niceeval.com/docs/zh/example/claude-code-codex-skill)
+- [AI Agent application eval](https://niceeval.com/docs/zh/example/ai-agent-application)
+
+
+## Roadmap
+Official Adapters
+- [ ] Agent Software
+  - [ ] Claude Code
+  - [ ] Codex
+  - [ ] Bub
+  - [ ] OpenClaw
+  - [ ] Hermess Agent
+  - [ ] Alma
+  - [ ] ...
+
+- [ ] Agent Frameworks
+  - [ ] AI SDK
+  - [ ] LangGraph
+  - [ ] Claude SDK
+  - [ ] Codex SDK
+  - [ ] vm0
+  - [ ] Cursor Agent SDK
+
+## Documentation
+
+- [Quickstart](https://niceeval.com/docs/quickstart)
+
+# Acknowledgements
+This project was inspired by — or had its code learned by AI from — the projects below:
+[eve](https://eve.dev)
+[agent eval](https://github.com/vercel-labs/agent-eval)
+[ponytail](https://github.com/DietrichGebert/ponytail)
+
+Thanks to the following communities

package/README.zh.md

@@ -0,0 +1,160 @@
+<div align="center">
+
+# NiceEval
+
+**渐进式、Agent Native、DX优秀的轻量 ai agent evals 工具**
+
+[![typescript](https://img.shields.io/badge/typescript-5.6-blue?style=flat-square)](tsconfig.json)
+[![license](https://img.shields.io/badge/license-MIT-green?style=flat-square)](package.json)
+[![docs](https://img.shields.io/badge/docs-readable-111827?style=flat-square)](docs/README.md)
+
+[English](README.md) 
+
+</div>
+
+NiceEval 是一个受[eve](https://eve.dev)启发的通用型 agent eval 工具。首先有非常优秀的 DX 设计，任何人可以在 10 分钟左右上手并配置。并且设计非常的通用。即可以用来 eval 给 Claude Code/Codex 写的 coding agent 的插件、Hook还有Skill。更可以直接 eval 自己的 AI Agent 框架(无论是基于 AI SDK、LangGraph、Pi都可以轻松接入)。
+
+在 eval 完成之后可以生成易读的报告与查看 Agent 的行为细节。方便 Debug 与理解 Agent 行为。
+
+## 为什么有了 DeepEval、LangFuse、BrainTrust 还需要 NiceEval
+NiceEval 是一个 AI Native 的 Eval 的评估工具。在这里工具里面像 Dataset 与 golden 构建 Input 与 Execpt Output 并不适合于真实的 Agent 评估。其次现在 Agent 需要对于多次用户对话、多 Agent、工具调用、Skill加载等等细粒进行评估时，NiceEval可以做的更好。
+
+同时与LangFuse、BrainTrust 也是可以共存的。可以使用前者来做 tracing。或者把评估结果上传到两者(实现中)
+
+## 架构
+
+NiceEval 支持两种接入方式，取决于被测系统是否需要隔离的沙箱文件系统。
+
+**模式一：Sandbox（Docker、E2B）—— 跑 Codex、Claude Code 等需要 sandbox 的 coding agent**
+
+```text
+   evals/*.eval.ts
+        │
+        ▼
+   ┌─────────────────────┐
+   │     NiceEval        │
+   └─────────────────────┘
+        │
+        │ Agent 适配器(官方)
+        ▼
+   ┌──────────────────────────────┐
+   │        Docker Sandbox        │
+   │   ┌────────────────────────┐ │
+   │   │ Codex / Claude Code |  │ │
+   │   │ 需要隔离文件系统的应用  │  | |
+   │   └────────────────────────┘ │
+   └──────────────────────────────┘
+```
+
+**模式二：直连 —— 直接连接你自己的 AI Agent**
+
+```text
+   evals/*.eval.ts
+        │
+        ▼
+   ┌─────────────────────┐
+   │     NiceEval        │
+   └─────────────────────┘
+        │
+        │ Agent 适配器(官方，或者自己实现)
+        ▼
+   ┌──────────────────────────────┐
+   │       你自己的 Web Agent      │
+   │   (HTTP / AI SDK·LangGraph   │
+   │    Pi 等自有框架，无需 Docker)  │
+   └──────────────────────────────┘
+```
+
+- **NiceEval 核心** 负责发现 eval、调度运行、打分、生成报告与 artifacts。
+- **Agent 适配器** 是开放的边界：你来决定如何调用被测系统。
+- 需要文件系统隔离的 coding agent 走 **Docker Sandbox**；自有的 Web Agent 可以直连，无需 Docker。
+
+
+## 示例
+
+跑一条 eval 需要两个文件：eval 本身（测什么）和 experiment（跑哪个 agent）。CLI 不接受裸 eval id——`niceeval exp <experiment> <eval 前缀>` 里的 experiment 才是决定「连哪个被测对象」的地方。下面是一个直连 Web Agent 的真实场景（完整项目见 [`examples/zh/ai-sdk/`](examples/zh/ai-sdk/)），验证 agent 遇到实时天气问题时会调用工具、并基于工具结果作答，而不是凭空编造：
+
+```ts
+// evals/eval-tool-call.eval.ts
+import { defineEval } from "niceeval";
+
+export default defineEval({
+  description: "测试 agent 在实时天气问题中正确调用工具并基于结果作答的能力",
+
+  async test(t) {
+    const turn = await t.send("北京今天天气怎么样？");
+    t.succeeded();
+
+    await t.group("调用 get_weather 且城市正确", () => {
+      t.calledTool("get_weather", { input: { city: "北京" } });
+      t.messageIncludes(/°C|气温|天气|晴|多云|雨/);
+    });
+
+    const second = await t.send("上海明天天气怎么样?");
+    second.messageIncludes("上海");
+
+    t.judge.autoevals
+      .closedQA("助手是否基于工具返回的天气数据作答，而不是凭空编造温度？")
+      .atLeast(0.7);
+  },
+});
+```
+
+```ts
+// experiments/local.ts
+import { defineExperiment } from "niceeval";
+import { webAgent } from "./adapter"; // 你自己写的 agent adapter，接被测 web agent
+
+export default defineExperiment({
+  agent: webAgent({ baseUrl: "http://127.0.0.1:5188" }),
+});
+```
+
+```sh
+pnpm exec niceeval exp local eval-tool-call  // 用 local experiment 只跑 eval-tool-call
+pnpm exec niceeval view // 查看评估结果
+```
+
+## 快速开始
+
+```text
+READ https://raw.githubusercontent.com/CorrectRoadH/niceeval/refs/heads/main/INIT.md and install niceeval for this repo.
+```
+
+从你的场景开始：
+
+- [如果你需要 eval 你的 Claude Code / Codex 插件](https://niceeval.com/docs/zh/example/claude-code-codex-plugin)
+- [如果你需要 eval 你的 Claude Code / Codex Skill](https://niceeval.com/docs/zh/example/claude-code-codex-skill)
+- [如果你需要 eval 你的 AI Agent 应用](https://niceeval.com/docs/zh/example/ai-agent-application)
+
+
+## Roadmap
+官方适配器
+- [ ] Agent 软件
+  - [ ] Claude Code
+  - [ ] Codex
+  - [ ] Bub
+  - [ ] OpenClaw
+  - [ ] Hermess Agent
+  - [ ] Alma
+  - [ ] ...
+
+- [ ] Agent 框架
+  - [ ] AI SDK
+  - [ ] LangGraph
+  - [ ] Claude SDK
+  - [ ] Codex SDK
+  - [ ] vm0
+  - [ ] Cursor Agent SDK
+
+## 文档
+
+- [快速开始](https://www.niceeval.com/docs/zh/quickstart)
+
+# 感谢
+该项目受下面项目所启发或者是由AI从下面项目中学习代码所写
+[eve](https://eve.dev)
+[agent eval](https://github.com/vercel-labs/agent-eval)
+[ponytail](https://github.com/DietrichGebert/ponytail)
+
+感谢下列社区

package/bin/niceeval.js

@@ -0,0 +1,11 @@
+#!/usr/bin/env node
+// niceeval 入口:注册 tsx 的 ESM loader(让我们能直接 import 用户的 .ts:
+// niceeval.config.ts、evals/*.eval.ts、agents/*.ts),再加载真正的 CLI。
+// 这样框架与被测项目都不需要编译步骤。
+import { fileURLToPath } from "node:url";
+import { register } from "tsx/esm/api";
+
+register();
+
+const cliUrl = new URL("../src/cli.ts", import.meta.url);
+await import(fileURLToPath(cliUrl));

package/package.json

@@ -0,0 +1,96 @@
+{
+  "name": "niceeval",
+  "version": "0.1.0",
+  "description": "Lightweight TypeScript agent eval tool — eval agents, services, functions, and coding-agent fixtures",
+  "type": "module",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/CorrectRoadH/niceeval"
+  },
+  "engines": {
+    "node": ">=18"
+  },
+  "bin": {
+    "niceeval": "bin/niceeval.js"
+  },
+  "types": "./src/index.ts",
+  "exports": {
+    ".": {
+      "types": "./src/index.ts",
+      "import": "./src/index.ts"
+    },
+    "./sandbox": {
+      "types": "./src/sandbox/index.ts",
+      "import": "./src/sandbox/index.ts"
+    },
+    "./adapter": {
+      "types": "./src/agents/index.ts",
+      "import": "./src/agents/index.ts"
+    },
+    "./expect": {
+      "types": "./src/expect/index.ts",
+      "import": "./src/expect/index.ts"
+    },
+    "./reporters": {
+      "types": "./src/runner/reporters/index.ts",
+      "import": "./src/runner/reporters/index.ts"
+    },
+    "./loaders": {
+      "types": "./src/loaders/index.ts",
+      "import": "./src/loaders/index.ts"
+    }
+  },
+  "files": [
+    "src",
+    "bin"
+  ],
+  "scripts": {
+    "typecheck": "tsc --noEmit",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "niceeval": "node bin/niceeval.js",
+    "view:build": "vite build --config src/view/app/vite.config.ts",
+    "prepare": "pnpm run view:build",
+    "site:dev": "vite site --host 127.0.0.1 --port 5174",
+    "site:build": "vite build site",
+    "docs:dev": "cd docs-site && npx --yes mint@latest dev",
+    "docs:validate": "cd docs-site && npx --yes mint@latest validate",
+    "docs:links": "cd docs-site && npx --yes mint@latest broken-links --check-anchors --check-redirects"
+  },
+  "dependencies": {
+    "autoevals": "0.0.132",
+    "effect": "^3.21.4",
+    "tar-stream": "^3.1.7",
+    "tsx": "^4.19.2"
+  },
+  "optionalDependencies": {
+    "@vercel/sandbox": "^2.2.1",
+    "dockerode": "^4.0.2",
+    "e2b": "^2.31.0"
+  },
+  "devDependencies": {
+    "@radix-ui/react-collapsible": "^1.1.14",
+    "@radix-ui/react-dialog": "^1.1.17",
+    "@radix-ui/react-tabs": "^1.1.15",
+    "@tailwindcss/vite": "^4.3.1",
+    "@types/dockerode": "^3.3.31",
+    "@types/node": "^22.0.0",
+    "@types/react": "^19.2.17",
+    "@types/react-dom": "^19.2.3",
+    "@types/tar-stream": "^3.1.3",
+    "@vitejs/plugin-react": "^6.0.3",
+    "class-variance-authority": "^0.7.1",
+    "clsx": "^2.1.1",
+    "lucide-react": "^1.21.0",
+    "prism-react-renderer": "^2.4.1",
+    "react": "^19.2.7",
+    "react-dom": "^19.2.7",
+    "react-grab": "^0.1.47",
+    "tailwind-merge": "^3.6.0",
+    "tailwindcss": "^4.3.1",
+    "typescript": "^5.6.0",
+    "vite": "^8.1.0",
+    "vitest": "^4.1.9"
+  }
+}

package/src/agents/bub.ts

@@ -0,0 +1,223 @@
+import { defineSandboxAgent } from "../define.ts";
+import { requireEnv, getEnv } from "../util.ts";
+import { shared } from "./shared.ts";
+import { createCheckpoint, restoreCheckpoint } from "../sandbox/checkpoint.ts";
+import type { Agent, Sandbox, StreamEvent } from "../types.ts";
+import { createHash } from "node:crypto";
+import { readFile, writeFile, mkdir } from "node:fs/promises";
+import { homedir } from "node:os";
+import { join, dirname } from "node:path";
+import { t } from "../i18n/index.ts";
+
+// ───────────────────────────────────────────────────────────────────────────
+// bub 的 agent adapter(沙箱型)。
+//
+// ⚠️ 现实校正:bub 是 PyPI 上的 `bub`(alpha,Python 3.12),不是 npm 包。
+//    · 安装:uv tool install bub(uv 自带 python 3.12,免 root)。
+//    · 调用:bub run "<prompt>" --session-id <id> --workspace <path>
+//    · 模型 + 代理:BUB_MODEL=openai:<model>、BUB_API_BASE、BUB_API_KEY。
+//    · 记忆:tape(总是开),落盘在 ~/.bub/tapes/<md5(ws)[:16]>__<md5(sess)[:16]>.jsonl。
+// ───────────────────────────────────────────────────────────────────────────
+
+export interface BubConfig {
+  /** OpenAI 兼容代理的 API key。省略时读 BUB_API_KEY env。 */
+  apiKey?: string;
+  /** OpenAI 兼容代理的 base URL。省略时读 BUB_API_BASE env。 */
+  apiBase?: string;
+  /**
+   * 额外装进 bub tool 环境的 Python 包(pip 名或 git URL)。
+   * 每个沙箱 setup 时作为 `uv tool install --with <pkg>` 追加到 bub 环境里。
+   * 示例:["bub-plugin-memory", "git+https://github.com/..."]
+   */
+  pythonPlugins?: string[];
+}
+
+const UV = "$HOME/.local/bin/uv";
+// bub 二进制:优先用镜像里(预制模板)烘焙在 PATH 上的 bub,否则用 uv 装到 $HOME/.local/bin 的那个。
+// 预制模板把 bub 装到 /usr/local/bin(见 sandbox/docker/Dockerfile),command -v 命中即用 → 跳过安装。
+const BUB = "$(command -v bub || echo $HOME/.local/bin/bub)";
+
+const BUB_OVERRIDE = "bub @ git+https://github.com/CorrectRoadH/bub.git@fix/streaming-usage-include-usage";
+const BUB_OVERRIDE_FILE = "/tmp/bub-override.txt";
+const OTEL_PLUGIN =
+  "git+https://github.com/CorrectRoadH/bub-contrib.git@fix/tapestore-otel-tape-entry-validation" +
+  "#subdirectory=packages/bub-tapestore-otel";
+
+const INSTALL_SPEC = `bub --override(${BUB_OVERRIDE}) --with ${OTEL_PLUGIN}`;
+const INSTALL_HASH = createHash("md5").update(INSTALL_SPEC).digest("hex").slice(0, 12);
+const DEFAULT_WORKSPACE = "/home/sandbox/workspace";
+
+function diskCachePath(home: string): string {
+  const homeKey = createHash("md5").update(home).digest("hex").slice(0, 8);
+  return join(homedir(), ".cache", "niceeval", `bub-checkpoint-${homeKey}-${INSTALL_HASH}.bin`);
+}
+
+// in-memory checkpoint + mutex keyed by sandbox $HOME,
+// so Docker(/home/node)と Vercel(/home/vercel-sandbox)でキャッシュが混ざらない。
+const memCheckpoints = new Map<string, Buffer>();
+const installsInProgress = new Map<string, Promise<void>>();
+
+async function ensureBub(sb: Sandbox, home: string): Promise<void> {
+  // 预制模板已把 bub 烘焙进镜像(PATH 上)→ 直接用,跳过 uv 安装 + checkpoint 全套。
+  if ((await sb.runShell("command -v bub >/dev/null 2>&1")).exitCode === 0) return;
+
+  const checkpointPaths = [`${home}/.local`, `${home}/.cache/uv`];
+  const cachePath = diskCachePath(home);
+
+  const mem = memCheckpoints.get(home);
+  if (mem) { await restoreCheckpoint(sb, mem); return; }
+
+  const disk = await readFile(cachePath).catch(() => undefined);
+  if (disk) {
+    try { await restoreCheckpoint(sb, disk); memCheckpoints.set(home, disk); return; } catch { /* 损坏,回退 */ }
+  }
+
+  const inflight = installsInProgress.get(home);
+  if (inflight) {
+    await inflight;
+    const after = memCheckpoints.get(home);
+    if (after) { await restoreCheckpoint(sb, after); return; }
+  }
+
+  let resolveInstall!: () => void;
+  let rejectInstall!: (e: unknown) => void;
+  const installPromise = new Promise<void>((res, rej) => { resolveInstall = res; rejectInstall = rej; });
+  installsInProgress.set(home, installPromise);
+
+  try {
+    await sb.runShell(`test -x ${UV} || (curl -LsSf https://astral.sh/uv/install.sh | sh)`);
+    await sb.runShell(`printf '%s\\n' '${BUB_OVERRIDE}' > ${BUB_OVERRIDE_FILE}`);
+    let last = { stdout: "", stderr: "" };
+    for (let attempt = 1; attempt <= 3; attempt++) {
+      const install = await sb.runShell(
+        `${UV} tool install --reinstall --python 3.12 --prerelease allow 'bub' --overrides ${BUB_OVERRIDE_FILE} --with '${OTEL_PLUGIN}'`,
+      );
+      if (install.exitCode === 0) break;
+      last = install;
+      if (attempt === 3) {
+        throw new Error(t("bub.installFailed", {
+          attempts: 3,
+          tail: (last.stdout + last.stderr).split("\n").slice(-15).join("\n"),
+        }));
+      }
+    }
+    const cp = await createCheckpoint(sb, checkpointPaths);
+    memCheckpoints.set(home, cp);
+    await mkdir(dirname(cachePath), { recursive: true }).catch(() => {});
+    await writeFile(cachePath, cp).catch(() => {});
+    resolveInstall();
+  } catch (e) {
+    rejectInstall(e);
+    installsInProgress.delete(home);
+    throw e;
+  }
+}
+
+function tapePath(workspace: string, sessionId: string, bubHome: string): string {
+  const w = createHash("md5").update(workspace).digest("hex").slice(0, 16);
+  const s = createHash("md5").update(sessionId).digest("hex").slice(0, 16);
+  return `${bubHome}/tapes/${w}__${s}.jsonl`;
+}
+
+function diagnose(
+  res: { exitCode: number; stdout: string; stderr: string },
+  events: StreamEvent[],
+  rawTape: string | undefined,
+): string {
+  const parts: string[] = [t("bub.diagnose.exitCode", { code: res.exitCode })];
+  if (rawTape === undefined) parts.push(t("bub.diagnose.noTape"));
+  else if (events.length === 0) parts.push(t("bub.diagnose.zeroEvents"));
+  const lastErr = [...events].reverse().find((e) => e.type === "error") as { type: "error"; message: string } | undefined;
+  if (lastErr) parts.push(t("bub.diagnose.lastError", { message: lastErr.message }));
+  const errTail = tail(res.stderr) || tail(res.stdout);
+  if (errTail) parts.push(t("bub.diagnose.outputTail", { tail: errTail }));
+  return parts.join(" · ");
+}
+
+function tail(s: string, n = 6): string {
+  return s.trim().split("\n").filter(Boolean).slice(-n).join(" ⏎ ").slice(0, 600);
+}
+
+export function bubAgent(config?: BubConfig): Agent {
+  const getApiKey = () => config?.apiKey ?? requireEnv("BUB_API_KEY");
+  const getApiBase = () => config?.apiBase ?? requireEnv("BUB_API_BASE");
+  // sandboxId → { home, workspace }; persists values detected in setup() so send() can use them.
+  const sessionInfo = new Map<string, { home: string; workspace: string }>();
+
+  return defineSandboxAgent({
+    name: "bub",
+    capabilities: { conversation: true, toolObservability: true, workspace: true, compactionObservability: true, tracing: true },
+
+    tracing: {
+      protocol: "http/protobuf",
+      env: (endpoint) => ({
+        BUB_TAPESTORE_OTEL_ENABLED: "true",
+        OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: endpoint,
+        OTEL_EXPORTER_OTLP_TRACES_PROTOCOL: "http/protobuf",
+      }),
+    },
+
+    async setup(sb) {
+      const home = (await sb.runShell("printf '%s' $HOME")).stdout.trim() || "/home/node";
+      const workspace = DEFAULT_WORKSPACE;
+      sessionInfo.set(sb.sandboxId, { home, workspace });
+      await ensureBub(sb, home);
+
+      if (config?.pythonPlugins?.length) {
+        const extraWith = config.pythonPlugins.map((p) => `--with '${p}'`).join(" ");
+        await sb.runShell(
+          `${UV} tool install --reinstall --python 3.12 --prerelease allow 'bub' --overrides ${BUB_OVERRIDE_FILE} --with '${OTEL_PLUGIN}' ${extraWith}`,
+        );
+      }
+
+      if (!(await sb.fileExists(`${workspace}/AGENTS.md`))) {
+        await shared.writeFile(
+          sb,
+          `${workspace}/AGENTS.md`,
+          [
+            `You are a coding agent working in a Next.js project at ${workspace}.`,
+            ``,
+            `Implement the requested feature by writing files directly to disk with the available tools:`,
+            `- fs_write(path, content): create or overwrite a file`,
+            `- fs_edit(path, old, new): edit an existing file`,
+            `- bash(cmd): run shell commands`,
+            ``,
+            `Do NOT respond with only a text explanation — write the actual code files.`,
+            `After writing, verify with bash("cd ${workspace} && npm run build").`,
+          ].join("\n"),
+        );
+      }
+    },
+
+    async send(input, ctx) {
+      const sb = ctx.sandbox;
+      const info = sessionInfo.get(sb.sandboxId) ?? { home: "/home/node", workspace: DEFAULT_WORKSPACE };
+      const { home, workspace } = info;
+      const bubHome = `${home}/.bub`;
+      const model = ctx.model ?? "gpt-5.4";
+      const sessionId = `fe-${sb.sandboxId}`;
+      ctx.session.id = sessionId;
+
+      const env = {
+        BUB_API_KEY: getApiKey(),
+        BUB_API_BASE: getApiBase(),
+        BUB_MODEL: `openai:${model}`,
+        BUB_HOME: bubHome,
+        ...ctx.telemetry?.env,
+      };
+      const escaped = input.text.replace(/'/g, "'\\''");
+      const res = await sb.runShell(
+        `${BUB} --workspace ${workspace} run '${escaped}' --session-id ${sessionId}`,
+        { env, stream: true },
+      );
+
+      const raw = await sb.readFile(tapePath(workspace, sessionId, bubHome)).catch(() => undefined);
+      const parsed = shared.parseBub(raw);
+      const events = [...parsed.events];
+      if (res.exitCode !== 0) events.push({ type: "error", message: diagnose(res, parsed.events, raw) });
+      return { events, usage: parsed.usage, status: res.exitCode === 0 ? "completed" : "failed" };
+    },
+  });
+}
+
+export default bubAgent();

package/src/agents/builtin.ts

@@ -0,0 +1,6 @@
+import claudeCodeDefault from "./claude-code.ts";
+import codexDefault from "./codex.ts";
+import bubDefault from "./bub.ts";
+import type { Agent } from "../types.ts";
+
+export const BUILTIN_AGENTS: readonly Agent[] = [claudeCodeDefault, codexDefault, bubDefault];