dsh-livebench-panel 0.1.1

package/README.md

@@ -0,0 +1,78 @@
+# dsh-livebench-panel
+
+DSH web 插件：在 **Trajectory 视图（轨迹视图）** 的「对话 / 轨迹」标签右侧新增一个 **LiveBench** 标签页。点进去即可用下拉框选择参数，直接对本机 LiveBench 发起评测并查看成绩。
+
+## ⚠️ 依赖要求（必读）
+
+本插件只是**控制台**，真正的评测由 [LiveBench](https://github.com/LiveBench/LiveBench) 项目在本机完成。仅安装插件时，面板底部会显示同样的安装指引。完整安装步骤（Windows，约 10–20 分钟）：
+
+```bat
+:: 1) 克隆 LiveBench（放到默认路径可免配置；装在其它目录请设环境变量 DSH_LIVEBENCH_HOME 指向它）
+git clone https://github.com/LiveBench/LiveBench V:\PythonProject\C_UtilizeSpace\LiveBench
+
+:: 2) 用 Python 3.11 建虚拟环境并安装（3.10 会因 litellm 报 NotRequired 错误）
+cd /d V:\PythonProject\C_UtilizeSpace\LiveBench
+py -3.11 -m venv .venv
+.venv\Scripts\python -m pip install -e .
+
+:: 3) 仅评测 coding 类需要：安装评分依赖（较大，含 TensorFlow）
+.venv\Scripts\python -m pip install -r livebench\code_runner\requirements_eval.txt
+
+:: 4) 下载题目数据（约 250MB）
+cd livebench
+..\.venv\Scripts\python download_questions.py
+
+:: 5) 重启 dsh web，回到面板点「刷新」
+```
+
+装在其它目录时：设置**系统环境变量** `DSH_LIVEBENCH_HOME=<你的LiveBench目录>`，重启 dsh web 生效。
+
+## 面板选项说明
+
+| 选项 | 含义 | 选择建议 |
+|---|---|---|
+| 模型 | harness 全部 provider 的全部模型（含内置 deepseek-official）；分组展示 | 首次验证选小任务 + 便宜模型 |
+| 推理强度 | 该模型的 reasoning effort（无配置则禁用）；编码进条目名，不同强度独立出分 | 日常 `medium`/`high`；对比测试固定同一档 |
+| 题集 release | LiveBench 题目发布批次 | 公开题目最全的是 `2024-11-25`（推荐） |
+| 分类 | 六大类：coding / math / reasoning / language / data_analysis / instruction_following | 首次验证选 `language` |
+| 任务 | 分类下的具体任务（如 language/typos 拼写纠错） | 首次验证选 `typos`（短平快） |
+| 题目序号范围 | 该任务题目的起止下标（0 起） | 冒烟测试填 0–1（只跑 2 题） |
+| max-tokens | 单次回答的 token 上限 | 推理模型给 8192+，否则思考被截断判 0 分 |
+
+## 功能
+
+- **标签页位置**：`conversation.view` 槽位 `id: "livebench"`、`order: 20`（对话 = 0，轨迹 = 10，LiveBench 排最右）。
+- **模型下拉框**：读取 `$DSH_HOME/settings.yaml` 的 `llm-pi-ai.providers` **并合并内置 `deepseek-official` 路由**（`@deepseek-ai/dsh-llm-deepseek` 注册的 DeepSeek-V4-Flash / V4-Pro / V4-Flash-Vision-Exp，默认 `https://api.deepseek.com` + `DEEPSEEK_API_KEY`，用户 `llm-deepseek` 设置节可覆盖），与 harness 模型选择同源，展示**全部 provider / 全部模型**；`openai-completions` 且配置了 `baseURL` 的 provider 会被标记为可直连 LiveBench（`--api-base` 路由）。
+- **推理强度下拉框**（模型右侧）：来自模型配置的 `reasoningEfforts` 映射（如 gpt-5.6 系 off/low/medium/high/xhigh/max，DeepSeek off/low/high/max）。选定后：
+  - 插件向 `livebench/model/model_configs/dsh_panel_generated.yaml` 写入一条模型配置，经 LiveBench 的 `api_kwargs.default.reasoning_effort` 透传给 API（`off` 表示不透传、由后端走默认）；
+  - 强度编码进 display-name（如 `code-gpt__gpt-5.6-sol@high`），**不同强度在成绩表中是独立条目**，可直接对比。
+- **参数下拉框**：题集 release（LiveBench 全部 releases）、分类（coding/math/reasoning/language/data_analysis/instruction_following）、任务（随分类联动）、题目序号范围、max-tokens。
+- **运行控制**：开始 / 停止 / 刷新；同一时间只允许一个评测；实时滚动日志（每 2.5s 轮询）。
+- **成绩表**：直接读取 `data/live_bench/**/model_judgment/ground_truth_judgment.jsonl` 计算 模型 × 任务 平均分（分数 = 正确率 ×100，附已判/总题数），无需等 LiveBench 自己出榜。
+
+## 依赖
+
+- 本机 LiveBench 检出：默认 `V:\PythonProject\C_UtilizeSpace\LiveBench`，可用环境变量 `DSH_LIVEBENCH_HOME` 覆盖。
+- 已配置好的评测 venv：`<root>\.venv\Scripts\python.exe`（Python 3.11，`pip install -e .` 完成）。
+- API Key：从 harness provider 的 `apiKeyEnv` 环境变量解析，经 `LIVEBENCH_API_KEY` 传给 LiveBench，**不会**出现在命令行或浏览器响应里。
+
+## API（同源）
+
+| 路由 | 方法 | 说明 |
+|---|---|---|
+| `/dsh-livebench-panel/api/config` | GET | 可用性、releases、分类→任务表、providers+models |
+| `/dsh-livebench-panel/api/start` | POST | 启动一次 `run_livebench.py` |
+| `/dsh-livebench-panel/api/status` | GET | 运行状态 + 日志尾部 |
+| `/dsh-livebench-panel/api/stop` | POST | 终止当前评测 |
+| `/dsh-livebench-panel/api/results` | GET | 汇总成绩行 |
+
+## 安装（本地开发，遵循 ~/.dsh/AGENTS.md）
+
+1. 插件源码位于 `~/.dsh/plugins/dsh-livebench-panel/`（开发/验证阶段标准位置）；
+2. 在 `~/.dsh/profiles/web/package.json` 的 `dependencies` 加入
+   `"dsh-livebench-panel": "link:../../plugins/dsh-livebench-panel"`，
+   并在 `dsh.profile.bundles` 数组加入 `"dsh-livebench-panel"`；
+3. 在 profile 目录执行 `pnpm install`；
+4. 重启 `dsh web`，打开任一会话的轨迹视图即可看到 **LiveBench** 标签。
+
+验证通过后可发布 npm，再以 `dsh plugin --profile web add <npm包名>` 正式安装。

package/cordis.patch.yml

@@ -0,0 +1,12 @@
+# dsh-livebench-panel bundle patch: mounts the plugin row into the profile
+# layer stack. The browser half registers a "LiveBench" tab into the
+# Trajectory view (slot conversation.view, id "livebench", order 20 — right
+# of 对话 at order 0 and 轨迹 at order 10); the node half registers the
+# /dsh-livebench-panel/api/* routes that spawn LiveBench runs.
+#
+# Install (local dev, per ~/.dsh/AGENTS.md):
+#   profile package.json: "dsh-livebench-panel": "link:../../plugins/dsh-livebench-panel"
+#   + add "dsh-livebench-panel" to dsh.profile.bundles, then restart dsh web.
+- insert:
+    - id: dsh-livebench-panel
+      name: dsh-livebench-panel

package/lib/client.js

@@ -0,0 +1,363 @@
+window.__ModuleLoader__.load({
+	id: "dsh-livebench-panel",
+	factory: (require) => {
+		var module = { exports: {} };
+		var exports = module.exports;
+		Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
+		let react = require("react");
+		const { createElement: h, useCallback, useEffect, useMemo, useRef, useState } = react;
+
+		//#region dsh-css:livebench-panel.css
+		const css = `.dlb_root{width:100%;max-width:1100px;margin:0 auto;padding:16px 20px 32px;color:var(--dsw-alias-label-primary);display:flex;flex-direction:column;gap:14px}
+.dlb_head{display:flex;align-items:baseline;gap:10px}
+.dlb_head h2{margin:0;font-size:16px;font-weight:600}
+.dlb_sub{color:var(--dsw-alias-label-tertiary);font-size:12px}
+.dlb_card{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-2);border-radius:10px;padding:14px;display:flex;flex-direction:column;gap:12px}
+.dlb_grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(180px,1fr));gap:10px}
+.dlb_field{display:flex;flex-direction:column;gap:4px;min-width:0}
+.dlb_label{color:var(--dsw-alias-label-tertiary);font-size:11px;text-transform:uppercase;letter-spacing:.02em}
+.dlb_select,.dlb_input{height:34px;border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-1);color:var(--dsw-alias-label-primary);font:inherit;font-size:13px;border-radius:8px;padding:0 8px;outline:none;min-width:0}
+.dlb_select:focus-visible,.dlb_input:focus-visible{border-color:var(--dsw-alias-state-business-primary)}
+.dlb_row{display:flex;align-items:center;gap:8px;flex-wrap:wrap}
+.dlb_btn{appearance:none;font:inherit;cursor:pointer;border-radius:8px;padding:6px 14px;font-size:13px;border:1px solid transparent;background:var(--dsw-alias-state-business-primary);color:var(--dsw-alias-label-primary)}
+.dlb_btn:disabled{cursor:default;opacity:.55}
+.dlb_btnGhost{background:transparent;border-color:var(--dsw-alias-border-l2);color:var(--dsw-alias-label-secondary)}
+.dlb_btnDanger{background:transparent;border-color:var(--dsw-alias-state-error-primary);color:var(--dsw-alias-state-error-primary)}
+.dlb_badge{display:inline-flex;align-items:center;gap:6px;border:1px solid var(--dsw-alias-border-l2);border-radius:999px;padding:2px 10px;font-size:11px;color:var(--dsw-alias-label-secondary)}
+.dlb_badge[data-ok="1"]{color:var(--dsw-alias-state-success-primary);border-color:color-mix(in srgb,var(--dsw-alias-state-success-primary) 40%,transparent)}
+.dlb_badge[data-ok="0"]{color:var(--dsw-alias-state-error-primary);border-color:color-mix(in srgb,var(--dsw-alias-state-error-primary) 40%,transparent)}
+.dlb_log{background:var(--dsw-alias-bg-layer-1);border:1px solid var(--dsw-alias-border-l2);border-radius:8px;padding:10px;font:12px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;max-height:260px;overflow:auto;white-space:pre-wrap;word-break:break-all;color:var(--dsw-alias-label-secondary)}
+.dlb_table{width:100%;border-collapse:collapse;font-size:12.5px}
+.dlb_table th,.dlb_table td{text-align:left;padding:6px 10px;border-bottom:1px solid var(--dsw-alias-border-l2)}
+.dlb_table th{color:var(--dsw-alias-label-tertiary);font-weight:500}
+.dlb_score{font-variant-numeric:tabular-nums;font-weight:600}
+.dlb_error{color:var(--dsw-alias-state-error-primary);font-size:12px;margin:0}
+.dlb_hint{color:var(--dsw-alias-label-tertiary);font-size:11.5px;margin:0;line-height:1.5}
+.dlb_notice{border-color:color-mix(in srgb,var(--dsw-alias-state-error-primary) 45%,transparent)}
+.dlb_noticeTitle{margin:0;font-size:13px;font-weight:600;color:var(--dsw-alias-state-error-primary)}
+.dlb_steps{margin:0;padding-left:18px;display:flex;flex-direction:column;gap:6px;font-size:12px;line-height:1.6;color:var(--dsw-alias-label-secondary)}
+.dlb_code{display:block;background:var(--dsw-alias-bg-layer-1);border:1px solid var(--dsw-alias-border-l2);border-radius:6px;color:var(--dsw-alias-label-primary);padding:4px 8px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:11px;word-break:break-all;user-select:all}`;
+		const tagId = "dsh-livebench-panel/panel.css";
+		if (typeof document !== "undefined" && document.querySelector("style[data-plugin-css=" + JSON.stringify(tagId) + "]") === null) {
+			const tag = document.createElement("style");
+			tag.dataset.plugin = "dsh-livebench-panel";
+			tag.dataset.pluginCss = tagId;
+			tag.textContent = css;
+			document.head.appendChild(tag);
+		}
+		const c = (n) => "dlb_" + n;
+		//#endregion
+
+		//#region lib/types/client/livebench.js
+		const NS = "livebenchPanel";
+		const inject = ["slots", "locale"];
+
+		const API = "/dsh-livebench-panel/api";
+
+		async function api(path, options) {
+			const response = await fetch(API + path, options);
+			const payload = await response.json().catch(() => ({ ok: false, error: `HTTP ${response.status}` }));
+			return payload;
+		}
+
+		/** Grouped <select> options: one optgroup per harness provider. */
+		function modelOptions(providers, selected) {
+			const out = [];
+			for (const p of providers) {
+				if (p.models.length === 0) continue;
+				out.push(h("optgroup", { key: p.id, label: p.name + (p.routable ? "" : " ·未接LiveBench") },
+					p.models.map((m) => h("option", { key: p.id + "::" + m.id, value: p.id + "::" + m.id }, `${m.name} — ${p.name}`))));
+			}
+			if (out.length === 0) {
+				out.push(h("option", { value: "" }, "（未发现 harness 模型）"));
+			}
+			void selected;
+			return out;
+		}
+
+		function LiveBenchView() {
+			const [config, setConfig] = useState(null);
+			const [configError, setConfigError] = useState(null);
+			const [sel, setSel] = useState({ provider: "", model: "", category: "", task: "", release: "2024-11-25", begin: "", end: "", maxTokens: "4096" });
+			const [busy, setBusy] = useState(false);
+			const [running, setRunning] = useState(false);
+			const [log, setLog] = useState("");
+			const [exitCode, setExitCode] = useState(null);
+			const [startError, setStartError] = useState(null);
+			const [results, setResults] = useState(null);
+			const logRef = useRef(null);
+
+			const loadConfig = useCallback(async () => {
+				const payload = await api("/config");
+				if (payload.ok) {
+					setConfig(payload);
+					setConfigError(null);
+					setSel((prev) => {
+						const next = { ...prev, release: payload.releases.includes(prev.release) ? prev.release : "2024-11-25" };
+						const provider = payload.providers.find((p) => p.id === prev.provider) ?? payload.providers.find((p) => p.models.length > 0) ?? null;
+						if (provider) {
+							next.provider = provider.id;
+							if (!provider.models.some((m) => m.id === prev.model)) {
+								next.model = provider.models[0]?.id ?? "";
+							}
+						}
+						if (!payload.releases.includes(next.release)) next.release = payload.releases[0] ?? "2024-11-25";
+						return next;
+					});
+				} else {
+					setConfigError(payload.error ?? "config unavailable");
+				}
+			}, []);
+
+			const loadResults = useCallback(async () => {
+				const payload = await api("/results");
+				if (payload.ok) setResults(payload);
+			}, []);
+
+			const refreshStatus = useCallback(async () => {
+				const payload = await api("/status");
+				if (payload.ok) {
+					setRunning(payload.running === true);
+					if (payload.hasRun) {
+						setLog(payload.log ?? "");
+						setExitCode(payload.exitCode);
+					}
+					return payload.running === true;
+				}
+				return false;
+			}, []);
+
+			useEffect(() => {
+				loadConfig();
+				loadResults();
+				refreshStatus();
+			}, [loadConfig, loadResults, refreshStatus]);
+
+			// poll while a run is active; refresh results when it ends
+			const wasRunning = useRef(false);
+			useEffect(() => {
+				const timer = setInterval(async () => {
+					const active = await refreshStatus();
+					if (wasRunning.current && !active) loadResults();
+					wasRunning.current = active;
+				}, 2500);
+				return () => clearInterval(timer);
+			}, [refreshStatus, loadResults]);
+
+			useEffect(() => {
+				if (logRef.current) logRef.current.scrollTop = logRef.current.scrollHeight;
+			}, [log]);
+
+			const categories = useMemo(() => (config ? Object.keys(config.tasks) : []), [config]);
+			const taskList = useMemo(() => {
+				if (!config || sel.category.length === 0) return [];
+				return config.tasks[sel.category] ?? [];
+			}, [config, sel.category]);
+			const selectedProvider = useMemo(
+				() => (config ? config.providers.find((p) => p.id === sel.provider) ?? null : null),
+				[config, sel.provider],
+			);
+			const selectedModel = useMemo(
+				() => selectedProvider?.models.find((m) => m.id === sel.model) ?? null,
+				[selectedProvider, sel.model],
+			);
+			const effortOptions = useMemo(() => {
+				const efforts = selectedModel?.efforts ?? [];
+				return efforts.map((e) => ({ value: e, label: e === "off" ? "off（关闭思考）" : e }));
+			}, [selectedModel]);
+
+			const onStart = async () => {
+				setStartError(null);
+				setBusy(true);
+				try {
+					const payload = await api("/start", {
+						method: "POST",
+						headers: { "content-type": "application/json" },
+						body: JSON.stringify({
+							provider: sel.provider,
+							model: sel.model,
+							reasoningEffort: sel.reasoning,
+							category: sel.category,
+							task: sel.task,
+							release: sel.release,
+							begin: sel.begin,
+							end: sel.end,
+							maxTokens: sel.maxTokens,
+						}),
+					});
+					if (!payload.ok) setStartError(payload.error ?? "启动失败");
+					else await refreshStatus();
+				} catch (error) {
+					setStartError(String(error.message ?? error));
+				} finally {
+					setBusy(false);
+				}
+			};
+
+			const onStop = async () => {
+				await api("/stop", { method: "POST" });
+				await refreshStatus();
+			};
+
+			const setField = (key) => (event) => {
+				const value = event.target.value;
+				setSel((prev) => {
+					const next = { ...prev, [key]: value };
+					if (key === "provider") {
+						const provider = config?.providers.find((p) => p.id === value) ?? null;
+						next.model = provider?.models[0]?.id ?? "";
+					}
+					if (key === "category") next.task = "";
+					return next;
+				});
+			};
+
+			// results table: pivot rows -> model x task
+			const resultTable = useMemo(() => {
+				if (!results || results.rows.length === 0) return null;
+				const models = [...new Set(results.rows.map((r) => r.model))].sort();
+				const tasks = [...new Set(results.rows.map((r) => r.task))].sort();
+				const cell = (model, task) => {
+					const row = results.rows.find((r) => r.model === model && r.task === task);
+					return row ? `${row.score.toFixed(1)} (${row.judged}/${row.total})` : "—";
+				};
+				return { models, tasks, cell };
+			}, [results]);
+
+			return h("div", { className: c("root") },
+				h("div", { className: c("head") },
+					h("h2", null, "LiveBench"),
+					h("span", { className: c("sub") }, "LiveBench 评测面板 · 基于 V:\\PythonProject\\C_UtilizeSpace\\LiveBench"),
+					h("span", { className: c("badge"), "data-ok": config?.available ? "1" : "0" },
+						config === null ? "检测中…" : config.available ? "LiveBench 就绪" : "未找到 LiveBench"),
+				),
+				configError !== null && h("p", { className: c("error") }, `加载配置失败：${configError}`),
+				h("div", { className: c("card") },
+					h("div", { className: c("grid") },
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "模型（harness 全部模型）"),
+							h("select", { className: c("select"), value: sel.provider + "::" + sel.model, onChange: (event) => {
+								const [providerId, ...rest] = event.target.value.split("::");
+								const modelId = rest.join("::");
+								setSel((prev) => ({ ...prev, provider: providerId, model: modelId, reasoning: "default" }));
+							} }, modelOptions(config?.providers ?? [])),
+						),
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "推理强度"),
+							h("select", { className: c("select"), value: sel.reasoning, onChange: setField("reasoning"), disabled: effortOptions.length === 0 },
+								h("option", { value: "default" }, effortOptions.length === 0 ? "（该模型不支持）" : "（模型默认）"),
+								effortOptions.map((e) => h("option", { key: e.value, value: e.value }, e.label))),
+						),
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "题集 release"),
+							h("select", { className: c("select"), value: sel.release, onChange: setField("release") },
+								(config?.releases ?? ["2024-11-25"]).map((r) => h("option", { key: r, value: r }, r))),
+						),
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "分类"),
+							h("select", { className: c("select"), value: sel.category, onChange: setField("category") },
+								h("option", { value: "" }, "全部分类"),
+								categories.map((cat) => h("option", { key: cat, value: cat }, cat))),
+						),
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "任务"),
+							h("select", { className: c("select"), value: sel.task, onChange: setField("task") },
+								h("option", { value: "" }, sel.category === "" ? "（先选分类）" : "全部任务"),
+								taskList.map((task) => h("option", { key: task, value: task }, task))),
+						),
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "题目序号范围（可选）"),
+							h("div", { className: c("row") },
+								h("input", { className: c("input"), type: "number", min: 0, placeholder: "起", value: sel.begin, onChange: setField("begin"), style: { width: "50%" } }),
+								h("input", { className: c("input"), type: "number", min: 0, placeholder: "止", value: sel.end, onChange: setField("end"), style: { width: "50%" } }),
+							),
+						),
+						h("div", { className: c("field") },
+							h("label", { className: c("label") }, "max-tokens"),
+							h("input", { className: c("input"), type: "number", min: 256, max: 32768, value: sel.maxTokens, onChange: setField("maxTokens") }),
+						),
+					),
+					h("div", { className: c("row") },
+						h("button", { className: c("btn"), onClick: onStart, disabled: busy || running || !config?.available || sel.model === "" },
+							running ? "评测运行中…" : busy ? "启动中…" : "开始评测"),
+						running && h("button", { className: c("btn") + " " + c("btnDanger"), onClick: onStop }, "停止"),
+						h("button", { className: c("btnGhost") + " " + c("btn"), onClick: () => { loadConfig(); loadResults(); } }, "刷新"),
+						selectedProvider && !selectedProvider.routable && h("span", { className: c("hint") },
+							"该 provider 未配置 openai-completions baseURL，LiveBench 将按模型名原生路由（未注册的模型名会失败）。"),
+					),
+					startError !== null && h("p", { className: c("error") }, startError),
+				),
+				(log.length > 0 || running) && h("div", { className: c("card") },
+					h("div", { className: c("row") },
+						h("span", { className: c("badge"), "data-ok": running ? "1" : "0" },
+							running ? "运行中" : exitCode === 0 ? "已完成" : exitCode === null ? "待运行" : `已退出 (${exitCode})`),
+					),
+					h("pre", { className: c("log"), ref: logRef }, log || "（暂无输出）"),
+				),
+				resultTable !== null && h("div", { className: c("card") },
+					h("div", { className: c("row") },
+						h("span", { className: c("label") }, "评测成绩（分数 = 平均分 ×100，括号内为 已判/总题数）"),
+					),
+					h("table", { className: c("table") },
+						h("thead", null, h("tr", null,
+							h("th", null, "model"),
+							resultTable.tasks.map((task) => h("th", { key: task }, task)))),
+						h("tbody", null,
+							resultTable.models.map((model) => h("tr", { key: model },
+								h("td", null, model),
+								resultTable.tasks.map((task) => h("td", { key: task, className: c("score") }, resultTable.cell(model, task)))))),
+					),
+				),
+				config !== null && config.available === false && h("div", { className: c("card") + " " + c("notice") },
+					h("p", { className: c("noticeTitle") }, "⚠ 本面板需要本地安装 LiveBench（当前未检测到）"),
+					h("p", { className: c("hint") },
+						"dsh-livebench-panel 只是控制台，真正的评测由 LiveBench 项目完成。面板会在以下位置查找它：环境变量 DSH_LIVEBENCH_HOME，或默认路径 V:\\PythonProject\\C_UtilizeSpace\\LiveBench（需含 .venv 与 livebench\\run_livebench.py）。按下面步骤安装："),
+					h("ol", { className: c("steps") },
+						h("li", null, "克隆 LiveBench 仓库（放到默认路径可免配置环境变量）：",
+							h("code", { className: c("code") }, "git clone https://github.com/LiveBench/LiveBench V:\\PythonProject\\C_UtilizeSpace\\LiveBench")),
+						h("li", null, "用 Python 3.11 建虚拟环境并安装（3.10 会因 litellm 报 NotRequired 错误）：",
+							h("code", { className: c("code") }, "cd /d V:\\PythonProject\\C_UtilizeSpace\\LiveBench && py -3.11 -m venv .venv"),
+							h("code", { className: c("code") }, ".venv\\Scripts\\python -m pip install -e .")),
+						h("li", null, "（评测 coding 类才需要）安装评分依赖：",
+							h("code", { className: c("code") }, ".venv\\Scripts\\python -m pip install -r livebench\\code_runner\\requirements_eval.txt")),
+						h("li", null, "下载题目数据：",
+							h("code", { className: c("code") }, "cd livebench && ..\\.venv\\Scripts\\python download_questions.py")),
+						h("li", null, "重启 dsh web，回到本页点「刷新」。",
+							h("br", null),
+							"若装在其它目录：设置系统环境变量 ", h("code", { className: c("code") }, "DSH_LIVEBENCH_HOME=你的LiveBench目录"), " 后再重启 dsh web。"),
+					),
+					h("p", { className: c("hint") }, "更完整的说明见插件目录内 README：~\\.dsh\\plugins\\dsh-livebench-panel\\README.md"),
+				),
+				h("p", { className: c("hint") },
+					"评分读取 LiveBench 的 ground_truth_judgment.jsonl；正式榜单可用 release 2024-11-25。多选题集请在「分类/任务」中缩小范围，避免长时间运行。"),
+			);
+		}
+
+		/**
+		 * Mount the LiveBench tab into the Trajectory view.
+		 * @param ctx - the browser plugin context.
+		 */
+		function apply(ctx) {
+			ctx.effect(() => ctx.locale.register(NS, {
+				zh: { tab: "LiveBench" },
+				en: { tab: "LiveBench" },
+			}), "livebench-panel: dictionaries");
+			const t = ctx.locale.bind(NS);
+			ctx.slots.inject("conversation.view", () => ctx.slots.register({
+				name: "conversation.view",
+				id: "livebench",
+				order: 20,
+				locale: NS,
+				label: () => t("tab"),
+				inject: () => ({}),
+			}, LiveBenchView));
+		}
+		//#endregion
+
+		exports.NS = NS;
+		exports.inject = inject;
+		exports.apply = apply;
+		exports.LiveBenchView = LiveBenchView;
+		return module.exports;
+	}
+});

package/lib/index.js

@@ -0,0 +1,594 @@
+/**
+ * dsh-livebench-panel — node half.
+ *
+ * Registers same-origin HTTP routes under /dsh-livebench-panel/api/* that
+ * bridge the web panel to a local LiveBench checkout:
+ *
+ *   GET  /config    availability, releases, category→task map, harness models
+ *   POST /start     spawn one `run_livebench.py` evaluation (single run at a time)
+ *   GET  /status    running flag + log tail + exit code of the last run
+ *   POST /stop      kill the running evaluation
+ *   GET  /results   per-model/per-task mean scores computed from the
+ *                   ground-truth judgment files LiveBench writes on disk
+ *
+ * The harness model list is read from $DSH_HOME/settings.yaml
+ * (llm-pi-ai.providers), which is the same source the model selection UI
+ * shows, so the dropdown always mirrors what the harness can serve. API keys
+ * are never returned to the browser: the node half resolves `apiKeyEnv` to a
+ * process env var and forwards it to LiveBench via LIVEBENCH_API_KEY.
+ *
+ * LiveBench checkout location (override with env DSH_LIVEBENCH_HOME):
+ *   V:\PythonProject\C_UtilizeSpace\LiveBench   (repo root)
+ *   <root>\.venv\Scripts\python.exe             (evaluation venv, Python 3.11)
+ *   <root>\livebench                            (cwd for run_livebench.py)
+ *
+ * @module dsh-livebench-panel
+ */
+import { spawn } from "node:child_process";
+import { readdirSync, readFileSync, existsSync, statSync, writeFileSync } from "node:fs";
+import { createRequire } from "node:module";
+import { homedir } from "node:os";
+import { join } from "node:path";
+import { fileURLToPath } from "node:url";
+
+/** Stable Cordis plugin name. */
+const name = "dsh-livebench-panel";
+/** Services required by this half: the webserver route registry. */
+const inject = ["webServer"];
+/** Route namespace. */
+const API = "/dsh-livebench-panel/api";
+/** Profile whose settings.yaml holds the harness provider config. */
+const PROFILE = "web";
+/** LiveBench checkout used unless DSH_LIVEBENCH_HOME overrides it. */
+const DEFAULT_LIVEBENCH_HOME = "V:\\PythonProject\\C_UtilizeSpace\\LiveBench";
+/** Question-set releases LiveBench accepts (mirrors livebench/common.py). */
+const RELEASES = [
+	"2024-06-24", "2024-07-26", "2024-08-31", "2024-11-25",
+	"2025-04-02", "2025-04-25", "2025-05-30", "2025-11-25",
+	"2025-12-23", "2026-01-08", "2026-06-25",
+];
+/** Sane cap so a runaway run cannot eat memory with its log. */
+const LOG_MAX_LINES = 600;
+/** One evaluation at a time — LiveBench grading is not cheap. */
+const MAX_CONCURRENT_RUNS = 1;
+
+/** Resolve the profile directory from the config-tree anchor (plugin-market pattern). */
+function resolveProfileDir(ctx) {
+	if (typeof ctx.baseUrl === "string" && ctx.baseUrl.startsWith("file:")) {
+		const path = fileURLToPath(ctx.baseUrl).replace(/[\\/]+$/, "");
+		if (existsSync(join(path, "package.json"))) return path;
+	}
+	const home = process.env.DSH_HOME ?? join(homedir(), ".dsh");
+	const fallback = join(home, "profiles", PROFILE);
+	if (existsSync(join(fallback, "package.json"))) return fallback;
+	throw new Error(`${name}: cannot locate the ${PROFILE} profile directory (ctx.baseUrl=${String(ctx.baseUrl)})`);
+}
+
+/** Load yaml parser through the profile's own dependency tree. */
+function loadYaml(profileDir) {
+	const requireFromProfile = createRequire(join(profileDir, "package.json"));
+	return requireFromProfile("yaml");
+}
+
+/**
+ * Read harness providers from settings.yaml, plus the built-in
+ * `deepseek-official` provider (registered by @deepseek-ai/dsh-llm-deepseek,
+ * which never appears under llm-pi-ai.providers).
+ * @returns {Array<{id: string, name: string, api: string|null, baseURL: string|null, keyEnv: string|null, models: Array<{id: string, name: string, efforts: string[]}>}>}
+ */
+function readProviders(profileDir) {
+	// profileDir = ~/.dsh/profiles/web  →  settings.yaml = ~/.dsh/settings.yaml
+	const settingsPath = join(profileDir.replace(/[\\/]+$/, ""), "..", "..", "settings.yaml");
+	const settings = existsSync(settingsPath) ? loadYaml(profileDir).parse(readFileSync(settingsPath, "utf8")) ?? {} : {};
+
+	const providers = [];
+	const piAi = settings?.["llm-pi-ai"]?.providers ?? {};
+	for (const [id, cfg] of Object.entries(piAi)) {
+		providers.push({
+			id,
+			name: typeof cfg?.displayName === "string" && cfg.displayName.length > 0 ? cfg.displayName : id,
+			api: typeof cfg?.api === "string" ? cfg.api : null,
+			baseURL: typeof cfg?.baseURL === "string" ? cfg.baseURL : null,
+			keyEnv: typeof cfg?.apiKeyEnv === "string" ? cfg.apiKeyEnv : null,
+			models: Array.isArray(cfg?.models)
+				? cfg.models
+					.filter((m) => m && typeof m.id === "string" && m.id.trim().length > 0)
+					.map((m) => ({ id: m.id.trim(), name: String(m.name ?? m.id).trim(), efforts: effortsOfModel(m) }))
+				: [],
+		});
+	}
+
+	// The harness's built-in DeepSeek route (dsh-llm-deepseek). Defaults mirror
+	// that package: DEEPSEEK_API_KEY, https://api.deepseek.com, and the
+	// off/low/high/max reasoning ladder; user settings may override any of it.
+	const dsCfg = settings?.["llm-deepseek"] ?? {};
+	const dsModels = Array.isArray(dsCfg.models) && dsCfg.models.length > 0
+		? dsCfg.models
+			.filter((m) => m && typeof m.id === "string" && m.id.trim().length > 0)
+			.map((m) => ({ id: m.id.trim(), name: String(m.name ?? m.id).trim(), efforts: [...DEEPSEEK_EFFORTS] }))
+		: DEFAULT_DEEPSEEK_MODELS.map((m) => ({ ...m, efforts: [...DEEPSEEK_EFFORTS] }));
+	providers.push({
+		id: "deepseek-official",
+		name: "DeepSeek 官方",
+		api: "openai-completions",
+		baseURL: typeof dsCfg.baseURL === "string" && dsCfg.baseURL.length > 0 ? dsCfg.baseURL : "https://api.deepseek.com",
+		keyEnv: typeof dsCfg.apiKeyEnv === "string" && dsCfg.apiKeyEnv.length > 0 ? dsCfg.apiKeyEnv : "DEEPSEEK_API_KEY",
+		models: dsModels,
+	});
+	return providers;
+}
+
+/** Reasoning efforts offered by the built-in DeepSeek route. */
+const DEEPSEEK_EFFORTS = ["off", "low", "high", "max"];
+/** Fallback catalog when settings.yaml has no llm-deepseek.models section. */
+const DEFAULT_DEEPSEEK_MODELS = [
+	{ id: "deepseek-v4-flash", name: "DeepSeek-V4-Flash" },
+	{ id: "deepseek-v4-pro", name: "DeepSeek-V4-Pro" },
+];
+
+/**
+ * Extract the reasoning-effort ladder from a settings.yaml model entry.
+ * `reasoningEfforts` maps effort slots to wire values, e.g.
+ * `{false: null, low: 'low', high: 'high', max: 'max'}` — a `false` slot whose
+ * value is null means "thinking can be turned off"; every non-null value is a
+ * selectable effort. Returns wire values, deduplicated, in insertion order.
+ */
+function effortsOfModel(model) {
+	const efforts = [];
+	const seen = new Set();
+	if (model && typeof model === "object" && model.reasoningEfforts && typeof model.reasoningEfforts === "object") {
+		for (const [slot, wire] of Object.entries(model.reasoningEfforts)) {
+			if ((slot === "false" || slot === "true") && (wire === null || wire === undefined)) {
+				if (!seen.has("off")) { seen.add("off"); efforts.push("off"); }
+				continue;
+			}
+			const value = String(wire ?? slot);
+			if (!seen.has(value)) { seen.add(value); efforts.push(value); }
+		}
+	}
+	return efforts;
+}
+
+/** LiveBench layout on disk. */
+function livebenchLayout() {
+	const root = process.env.DSH_LIVEBENCH_HOME ?? DEFAULT_LIVEBENCH_HOME;
+	const venvPython = join(root, ".venv", "Scripts", "python.exe");
+	return {
+		root,
+		livebenchDir: join(root, "livebench"),
+		dataDir: join(root, "livebench", "data", "live_bench"),
+		pythonExe: venvPython,
+		available: existsSync(venvPython) && existsSync(join(root, "livebench", "run_livebench.py")),
+	};
+}
+
+/** Scan data/live_bench/<category>/<task>/question.jsonl into a category→tasks map. */
+function scanCategoryTasks(dataDir) {
+	const tasks = {};
+	if (!existsSync(dataDir)) return tasks;
+	let categoryDirs;
+	try {
+		categoryDirs = readDirSafe(dataDir);
+	} catch {
+		return tasks;
+	}
+	for (const category of categoryDirs) {
+		const catDir = join(dataDir, category);
+		for (const task of readDirSafe(catDir)) {
+			if (existsSync(join(catDir, task, "question.jsonl"))) {
+				(tasks[category] ??= []).push(task);
+			}
+		}
+	}
+	return tasks;
+}
+
+function readDirSafe(dir) {
+	try {
+		return readdirNames(dir);
+	} catch {
+		return [];
+	}
+}
+
+function readdirNames(dir) {
+	try {
+		return readdirSync(dir).filter((entry) => {
+			try {
+				return statSync(join(dir, entry)).isDirectory();
+			} catch {
+				return false;
+			}
+		});
+	} catch {
+		return [];
+	}
+}
+
+/** Minimal JSON body reader with a hard size cap. */
+function readBody(req, cap = 16 * 1024) {
+	return new Promise((resolve, reject) => {
+		let size = 0;
+		const chunks = [];
+		req.on("data", (chunk) => {
+			size += chunk.length;
+			if (size > cap) {
+				reject(new Error("request body too large"));
+				req.destroy();
+				return;
+			}
+			chunks.push(chunk);
+		});
+		req.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")));
+		req.on("error", reject);
+	});
+}
+
+/** Same-origin guard (plugin-market pattern). */
+function originAllowed(req) {
+	const origin = req.headers.origin;
+	return origin === undefined || /^https?:\/\/(127\.0\.0\.1|localhost)(:\d+)?$/.test(origin);
+}
+
+function sendJson(res, status, payload) {
+	res.writeHead(status, { "content-type": "application/json" });
+	res.end(JSON.stringify(payload));
+}
+
+/** Filename-safe display name for a provider/model pair. */
+function displayModelName(providerId, modelId) {
+	return `${String(providerId).replace(/[^A-Za-z0-9._-]/g, "-")}__${String(modelId).replace(/[^A-Za-z0-9._-]/g, "-")}`;
+}
+
+/**
+ * Write a single-entry model config into LiveBench's model_configs directory so
+ * `get_model_config(displayName)` resolves it (instead of the bare custom-model
+ * fallback) and the api_kwargs — e.g. `reasoning_effort` — reach the API call.
+ * The file is regenerated on every start; secrets never go in here.
+ * @returns {string|null} error message, or null on success.
+ */
+function writeGeneratedModelConfig(layout, YAML, { displayName, modelId, reasoningEffort }) {
+	const configDir = join(layout.livebenchDir, "model", "model_configs");
+	const target = join(configDir, "dsh_panel_generated.yaml");
+	void YAML;
+	const doc = [
+		"# Generated by dsh-livebench-panel — regenerated on every evaluation start.",
+		"---",
+		`display_name: ${displayName}`,
+		"api_name:",
+		"  local: " + modelId,
+		"api_kwargs:",
+		"  default:",
+		`    reasoning_effort: ${reasoningEffort}`,
+		"",
+	].join("\n");
+	try {
+		if (!existsSync(configDir)) return `model_configs directory not found: ${configDir}`;
+		writeFileSync(target, doc, "utf8");
+		return null;
+	} catch (error) {
+		return `cannot write ${target}: ${error.message}`;
+	}
+}
+
+
+/** Read a JSONL file into objects, skipping broken lines. */
+function readJsonl(path) {
+	if (!existsSync(path)) return [];
+	try {
+		return readFileSync(path, "utf8")
+			.split(/\r?\n/)
+			.filter((line) => line.trim().length > 0)
+			.map((line) => {
+				try {
+					return JSON.parse(line);
+				} catch {
+					return null;
+				}
+			})
+			.filter((row) => row !== null);
+	} catch {
+		return [];
+	}
+}
+
+/**
+ * Compute mean scores per (model, category, task) from ground-truth judgment
+ * files, plus the judged/total question counts per task.
+ */
+function computeResults(dataDir) {
+	const judged = new Map(); // `${model}\u0000${task}` -> {model, category, task, sum, n}
+	const totals = new Map(); // task -> question count
+	if (!existsSync(dataDir)) return { rows: [], taskTotals: {} };
+	for (const category of readDirSafe(dataDir)) {
+		const catDir = join(dataDir, category);
+		for (const task of readDirSafe(catDir)) {
+			const taskDir = join(catDir, task);
+			const questions = readJsonl(join(taskDir, "question.jsonl"));
+			totals.set(task, questions.length);
+			const judgments = readJsonl(join(taskDir, "model_judgment", "ground_truth_judgment.jsonl"));
+			for (const row of judgments) {
+				const model = typeof row.model === "string" ? row.model : null;
+				const score = typeof row.score === "number" ? row.score : Number(row.score);
+				if (model === null || !Number.isFinite(score) || score < 0) continue;
+				const key = `${model}\u0000${task}`;
+				const entry = judged.get(key) ?? { model, category, task, sum: 0, n: 0 };
+				entry.sum += score;
+				entry.n += 1;
+				judged.set(key, entry);
+			}
+		}
+	}
+	const rows = [...judged.values()].map((entry) => ({
+		model: entry.model,
+		category: entry.category,
+		task: entry.task,
+		score: entry.n > 0 ? (entry.sum / entry.n) * 100 : 0,
+		judged: entry.n,
+		total: totals.get(entry.task) ?? 0,
+	}));
+	const taskTotals = Object.fromEntries(totals);
+	return { rows, taskTotals };
+}
+
+/** Clamp helper for query params. */
+function asInt(value, min, max, fallback) {
+	const n = Number(value);
+	if (!Number.isFinite(n)) return fallback;
+	return Math.min(max, Math.max(min, Math.trunc(n)));
+}
+
+function apply(ctx) {
+	/** The single run slot. */
+	let run = null;
+
+	const startRun = (body) => {
+		const layout = livebenchLayout();
+		if (!layout.available) {
+			return { status: 409, payload: { ok: false, error: `LiveBench venv not found under ${layout.root}` } };
+		}
+		if (run && run.exitCode === null) {
+			return { status: 409, payload: { ok: false, error: "another evaluation is already running" } };
+		}
+		const providerId = typeof body.provider === "string" ? body.provider : "";
+		const modelId = typeof body.model === "string" ? body.model.trim() : "";
+		const release = typeof body.release === "string" ? body.release : "2024-11-25";
+		const category = typeof body.category === "string" && body.category.length > 0 ? body.category : null;
+		const task = typeof body.task === "string" && body.task.length > 0 ? body.task : null;
+		const reasoningEffort = typeof body.reasoningEffort === "string" && body.reasoningEffort.length > 0 && body.reasoningEffort !== "default" ? body.reasoningEffort : null;
+		if (modelId.length === 0 || modelId.length > 120 || /[^\w.:\/_-]/.test(modelId)) {
+			return { status: 400, payload: { ok: false, error: "invalid model id" } };
+		}
+		if (reasoningEffort !== null && !/^[a-z0-9_-]{1,20}$/.test(reasoningEffort)) {
+			return { status: 400, payload: { ok: false, error: `invalid reasoning effort: ${reasoningEffort}` } };
+		}
+		if (!RELEASES.includes(release)) {
+			return { status: 400, payload: { ok: false, error: `unknown release: ${release}` } };
+		}
+
+		const profileDir = resolveProfileDir(ctx);
+		const providers = readProviders(profileDir);
+		const provider = providers.find((p) => p.id === providerId) ?? null;
+
+		// A selected reasoning effort is passed through LiveBench's model-config
+		// api_kwargs (chat_completion_openai merges them into the SDK call). The
+		// effort is encoded in the display name so different efforts land in
+		// separate result rows.
+		const hasEffortSuffix = reasoningEffort !== null && reasoningEffort !== "off";
+		const displayName = displayModelName(providerId || "direct", modelId) + (hasEffortSuffix ? "@" + reasoningEffort : "");
+		let cliModel = modelId;
+		if (hasEffortSuffix) {
+			const writeError = writeGeneratedModelConfig(layout, loadYaml(profileDir), { displayName, modelId, reasoningEffort });
+			if (writeError) return { status: 500, payload: { ok: false, error: writeError } };
+			cliModel = displayName;
+		}
+
+		const benchParts = ["live_bench", ...(category ? [category] : []), ...(task ? [task] : [])];
+		const args = [
+			"run_livebench.py",
+			"--model", cliModel,
+			"--model-display-name", displayName,
+			"--bench-name", benchParts.join("/"),
+			"--livebench-release-option", release,
+			"--max-tokens", String(asInt(body.maxTokens, 256, 32768, 4096)),
+			"--parallel-requests", String(asInt(body.parallel, 1, 8, 2)),
+			"--mode", "single",
+		];
+		if (body.begin !== undefined && body.begin !== null && `${body.begin}`.length > 0) {
+			args.push("--question-begin", String(asInt(body.begin, 0, 100000, 0)));
+		}
+		if (body.end !== undefined && body.end !== null && `${body.end}`.length > 0) {
+			args.push("--question-end", String(asInt(body.end, 0, 100000, 0)));
+		}
+		if (body.resume === true) args.push("--resume");
+
+		const env = {
+			...process.env,
+			HF_HOME: join(layout.root, ".hf_cache"),
+			HF_HUB_DISABLE_SYMLINKS_WARNING: "1",
+		};
+		// Only OpenAI-compatible providers can be routed with --api-base; for
+		// those, hand the key over via env (never the command line).
+		if (provider && provider.api === "openai-completions" && provider.baseURL) {
+			args.push("--api-base", provider.baseURL);
+			if (provider.keyEnv && process.env[provider.keyEnv]) {
+				env.LIVEBENCH_API_KEY = process.env[provider.keyEnv];
+			}
+		}
+
+		const child = spawn(layout.pythonExe, args, {
+			cwd: layout.livebenchDir,
+			env,
+			windowsHide: true,
+		});
+
+		const record = {
+			id: `${Date.now()}`,
+			proc: child,
+			exitCode: null,
+			startedAt: new Date().toISOString(),
+			command: [layout.pythonExe, ...args].join(" "),
+			log: [`$ ${[...args].join(" ")}`],
+		};
+		run = record;
+
+		child.stdout.on("data", (chunk) => appendLog(record, chunk));
+		child.stderr.on("data", (chunk) => appendLog(record, chunk));
+		child.on("error", (error) => {
+			appendLog(record, `[spawn error] ${error.message}`);
+			record.exitCode = -1;
+		});
+		child.on("close", (code) => {
+			record.exitCode = code === null ? -1 : code;
+			record.log.push(`[exit ${record.exitCode}]`);
+		});
+
+		return { status: 200, payload: { ok: true, runId: record.id, displayName, command: record.command } };
+	};
+
+	const appendLog = (record, chunk) => {
+		for (const line of String(chunk).split(/\r?\n/)) {
+			if (line.length === 0) continue;
+			record.log.push(line);
+		}
+		if (record.log.length > LOG_MAX_LINES) {
+			record.log.splice(0, record.log.length - LOG_MAX_LINES);
+		}
+	};
+
+	/**
+	 * Mount the routes.
+	 * @param ctx - plugin context carrying the webServer service.
+	 */
+	ctx.effect(() => ctx.webServer.register({
+		kind: "exact",
+		path: `${API}/config`,
+		handler: async (req, res) => {
+			if (req.method !== "GET") {
+				sendJson(res, 405, { ok: false, error: "method not allowed" });
+				return;
+			}
+			const layout = livebenchLayout();
+			let providers = [];
+			try {
+				providers = readProviders(resolveProfileDir(ctx));
+			} catch (error) {
+				providers = [];
+				appendLog({ log: [] }, `[config] settings.yaml unreadable: ${error.message}`);
+			}
+			sendJson(res, 200, {
+				ok: true,
+				available: layout.available,
+				root: layout.root,
+				releases: RELEASES,
+				tasks: scanCategoryTasks(layout.dataDir),
+				providers: providers.map(({ id, name: pname, models, api, baseURL }) => ({
+					id,
+					name: pname,
+					routable: api === "openai-completions" && typeof baseURL === "string" && baseURL.length > 0,
+					models,
+				})),
+			});
+		},
+	}), `${name}: config route`);
+
+	ctx.effect(() => ctx.webServer.register({
+		kind: "exact",
+		path: `${API}/start`,
+		handler: async (req, res) => {
+			if (req.method !== "POST") {
+				sendJson(res, 405, { ok: false, error: "method not allowed" });
+				return;
+			}
+			if (!originAllowed(req)) {
+				sendJson(res, 403, { ok: false, error: "forbidden origin" });
+				return;
+			}
+			let body;
+			try {
+				body = JSON.parse(await readBody(req));
+			} catch {
+				sendJson(res, 400, { ok: false, error: "invalid request body" });
+				return;
+			}
+			if (run !== null && run.exitCode === null && runsAlive() >= MAX_CONCURRENT_RUNS) {
+				sendJson(res, 409, { ok: false, error: "another evaluation is already running" });
+				return;
+			}
+			const result = startRun(body);
+			sendJson(res, result.status, result.payload);
+		},
+	}), `${name}: start route`);
+
+	ctx.effect(() => ctx.webServer.register({
+		kind: "exact",
+		path: `${API}/status`,
+		handler: async (req, res) => {
+			if (req.method !== "GET") {
+				sendJson(res, 405, { ok: false, error: "method not allowed" });
+				return;
+			}
+			if (run === null) {
+				sendJson(res, 200, { ok: true, running: false, hasRun: false });
+				return;
+			}
+			sendJson(res, 200, {
+				ok: true,
+				running: run.exitCode === null,
+				hasRun: true,
+				runId: run.id,
+				exitCode: run.exitCode,
+				startedAt: run.startedAt,
+				command: run.command,
+				log: run.log.slice(-120).join("\n"),
+			});
+		},
+	}), `${name}: status route`);
+
+	ctx.effect(() => ctx.webServer.register({
+		kind: "exact",
+		path: `${API}/stop`,
+		handler: async (req, res) => {
+			if (req.method !== "POST") {
+				sendJson(res, 405, { ok: false, error: "method not allowed" });
+				return;
+			}
+			if (!originAllowed(req)) {
+				sendJson(res, 403, { ok: false, error: "forbidden origin" });
+				return;
+			}
+			if (run === null || run.exitCode !== null) {
+				sendJson(res, 200, { ok: true, stopped: false });
+				return;
+			}
+			try {
+				run.proc.kill();
+			} catch (error) {
+				sendJson(res, 500, { ok: false, error: String(error.message ?? error) });
+				return;
+			}
+			sendJson(res, 200, { ok: true, stopped: true });
+		},
+	}), `${name}: stop route`);
+
+	ctx.effect(() => ctx.webServer.register({
+		kind: "exact",
+		path: `${API}/results`,
+		handler: async (req, res) => {
+			if (req.method !== "GET") {
+				sendJson(res, 405, { ok: false, error: "method not allowed" });
+				return;
+			}
+			const layout = livebenchLayout();
+			const { rows, taskTotals } = computeResults(layout.dataDir);
+			sendJson(res, 200, { ok: true, rows, taskTotals });
+		},
+	}), `${name}: results route`);
+}
+
+/** Count alive runs (at most one today, kept for future parallel lanes). */
+function runsAlive() {
+	return 1;
+}
+
+export { name, inject, apply, writeGeneratedModelConfig, readProviders };

package/package.json

@@ -0,0 +1,43 @@
+{
+  "name": "dsh-livebench-panel",
+  "version": "0.1.1",
+  "description": "DSH web plugin: a LiveBench tab in the Trajectory view (right of 对话/轨迹). Run LiveBench evaluations against every model configured in the DeepSeek Harness — pick provider/model, category, task, release and question range from dropdowns, watch progress, and read scores in place.",
+  "license": "MIT",
+  "type": "module",
+  "main": "lib/index.js",
+  "exports": {
+    ".": "./lib/index.js",
+    "./client": "./lib/client.js",
+    "./cordis.patch.yml": "./cordis.patch.yml",
+    "./package.json": "./package.json"
+  },
+  "engines": {
+    "node": ">=20"
+  },
+  "files": [
+    "lib",
+    "cordis.patch.yml",
+    "README.md"
+  ],
+  "keywords": [
+    "dsh-plugin",
+    "deepseek-harness",
+    "livebench",
+    "benchmark",
+    "evaluation"
+  ],
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    },
+    "client": {
+      "platform": "web",
+      "inject": [
+        "@deepseek-ai/dsh-client-runtime",
+        "@deepseek-ai/dsh-client-locale",
+        "@deepseek-ai/dsh-client-ui-slots",
+        "@deepseek-ai/dsh-client-ui-conversation"
+      ]
+    }
+  }
+}