mingdao-harness 0.2.3 → 0.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/ROADMAP-NEXT.md +13 -13
- package/package.json +1 -1
- package/skills/release-checklist/SKILL.md +16 -3
- package/src/agent.js +102 -30
- package/src/audit.js +2 -2
- package/src/batch.js +74 -16
- package/src/cachestats.js +57 -11
- package/src/cli.js +29 -679
- package/src/commands/desktop.js +1 -1
- package/src/commands/key.js +1 -1
- package/src/commands/repl.js +711 -0
- package/src/commands/schedule.js +7 -7
- package/src/commands/skill.js +18 -16
- package/src/commands/sync.js +5 -5
- package/src/commands/update.js +22 -9
- package/src/commands/workspace.js +2 -2
- package/src/compact.js +6 -6
- package/src/config.js +7 -4
- package/src/context.js +11 -11
- package/src/cost-guard.js +15 -5
- package/src/credentials.js +6 -0
- package/src/hooks.js +11 -11
- package/src/mcp-presets.js +5 -5
- package/src/mcp.js +19 -19
- package/src/memory.js +16 -16
- package/src/model-discovery.js +16 -15
- package/src/models.js +4 -4
- package/src/notify.js +2 -2
- package/src/permissions.js +7 -3
- package/src/pricing.js +35 -1
- package/src/prompts.js +1 -1
- package/src/providers/index.js +7 -7
- package/src/providers/openai-compatible.js +8 -8
- package/src/routing.js +5 -5
- package/src/schedule.js +28 -28
- package/src/session-index.js +5 -1
- package/src/session.js +11 -2
- package/src/skill-lib.js +62 -3
- package/src/skill-registry.js +10 -9
- package/src/skills.js +10 -10
- package/src/sync-server.js +86 -3
- package/src/sync.js +53 -20
- package/src/tasks/worker.js +126 -0
- package/src/tasks.js +13 -13
- package/src/titles.js +5 -5
- package/src/tokenizer.js +22 -4
- package/src/tools/bash.js +12 -12
- package/src/tools/fs-tools.js +66 -9
- package/src/tools/index.js +77 -36
- package/src/ui.js +83 -6
- package/src/update.js +8 -8
- package/src/web/app.js +43 -0
- package/src/web/attachments.js +1 -1
- package/src/web/index.html +12 -0
- package/src/web/routes/api.js +38 -786
- package/src/web/routes/domains/config.js +280 -0
- package/src/web/routes/domains/misc.js +128 -0
- package/src/web/routes/domains/schedule.js +127 -0
- package/src/web/routes/domains/sessions.js +105 -0
- package/src/web/routes/domains/skills.js +79 -0
- package/src/web/routes/domains/sync.js +101 -0
- package/src/web/routes/domains/workspace.js +113 -0
- package/src/web/server.js +29 -14
- package/src/web/web-io.js +54 -12
- package/src/workspace.js +14 -14
package/docs/ROADMAP-NEXT.md
CHANGED
|
@@ -26,19 +26,19 @@
|
|
|
26
26
|
9. cli.js 共享助手抽取(offpeak defer、mcpFacade 构建、run/run-worker 参数解析合一)
|
|
27
27
|
- 验收:全绿 + 新增 fs-browse 越界与限流回归断言。
|
|
28
28
|
|
|
29
|
-
### Phase B「省钱第二轮」v0.2.
|
|
30
|
-
1. **工具 Schema
|
|
31
|
-
2. **Batch 输入侧**:文本哈希去重(结果按 custom_id 回填全部位置)、单问超窗口预检报错、`--max-cost`
|
|
32
|
-
3. **费用二级分账 + 看板**:cache-stats 增 reasoning/
|
|
33
|
-
4. **护栏 downgrade**:`action:'downgrade'` —— 触顶后自动切 flash +
|
|
34
|
-
5. **评测基准扩充**:压缩质量集(5
|
|
35
|
-
-
|
|
36
|
-
|
|
37
|
-
### Phase C「结构渐进」v0.2.
|
|
38
|
-
1. routes/api.js
|
|
39
|
-
2. cli.js REPL → commands/repl.js
|
|
40
|
-
3. strict 棘轮削 300
|
|
41
|
-
- 验收:tsc full 错误数 ≤
|
|
29
|
+
### Phase B「省钱第二轮」v0.2.4(已完成 ✅)
|
|
30
|
+
1. ✅ **工具 Schema 按需挂载**:只读阶段只发 read/ls/glob/grep/skill/todo + 已用工具;模型表达写意图后下一轮注入全量(`cfg.schemaTier=false` 可关);已用工具深度瘦身(工具级 + 参数级描述清除,全用过降 48%)
|
|
31
|
+
2. ✅ **Batch 输入侧**:文本哈希去重(结果按 custom_id 回填全部位置)、单问超窗口预检报错、`--max-cost` 提交前估算拦截
|
|
32
|
+
3. ✅ **费用二级分账 + 看板**:cache-stats 增 reasoning / byTool 维度;WebUI 仪表盘加模型/工具 Top5 与近 14 天费用折线(零依赖 SVG);`mingdao cost report` 增按工具分账表
|
|
33
|
+
4. ✅ **护栏 downgrade**:`action:'downgrade'` —— 触顶后自动切 flash 继续执行 + 明确提示(warn/block 仍可选;已是 flash 则按 block 处理)
|
|
34
|
+
5. ✅ **评测基准扩充**:压缩质量集(5 组长会话样本 → 压缩 → 关键事实存活断言 + 尾部字节不变)、路由标注集扩至 100 条;bench 断言数 38 → 150+;覆盖率阈值 55% → 60%
|
|
35
|
+
- 版本说明:v0.2.3 被 0.2.2 桌面启动崩溃的紧急修复占用,Phase B 顺延为 v0.2.4。
|
|
36
|
+
|
|
37
|
+
### Phase C「结构渐进」v0.2.5(进行中)
|
|
38
|
+
1. ✅ routes/api.js 按域细分:`src/web/routes/domains/{config,sessions,skills,schedule,sync,workspace,misc}.js`(7 域,handle(ctx,deps,shared) 契约:命中返回 true)+ api.js 瘦身为编排器(访问控制/CSRF/静态壳 + 域循环);`test/api-contracts.js` 8 组契约测试入 run-all;顺带修复 icon 路由路径 bug(routes/icons → web/icons,契约测试捕获)
|
|
39
|
+
2. ✅ cli.js REPL → `src/commands/repl.js`(runRepl(ctx))、worker → `src/tasks/worker.js`(runWorkerTask);e2e-local 16 项全绿护栏
|
|
40
|
+
3. ⏳ strict 棘轮削 300 条(五安全关键模块 credentials/permissions/config/sync-server/session 全量注解 + 批量文件注解)
|
|
41
|
+
- 验收:tsc full 错误数 ≤812;结构清单在文档登记。
|
|
42
42
|
|
|
43
43
|
### Phase D「能力演进」v0.3.0(约 3–5 天,可按需裁剪)
|
|
44
44
|
1. **跨平台沙箱补位**:Windows Job Object + 降权 token(零依赖原生 API)、macOS seatbelt 规则;敏感命令平台无关拦截清单兜底
|
package/package.json
CHANGED
|
@@ -11,9 +11,22 @@ description: 准备发布新版本、打包交付前使用
|
|
|
11
11
|
2. **版本**:版本号统一(package.json / 配置 / 文档互相一致),遵循语义化版本。
|
|
12
12
|
3. **变更说明**:changelog 用用户语言描述新增/修复/破坏性变更。
|
|
13
13
|
4. **构建验证**:实际构建/打包一次,并在干净环境验证产物可安装可运行。
|
|
14
|
-
5.
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
5. **桌面版打包冒烟(0.2.2 事故护栏,发版必经)**:主进程模块求值必须真实跑通打包产物——
|
|
15
|
+
- CI 已内置:desktop.yml 在 Linux 腿用 xvfb 运行 `dist/linux-unpacked/mingdao --no-sandbox`(`MINGDAO_DESKTOP_SMOKE=1`),模块求值通过输出 `MINGDAO_DESKTOP_SMOKE_OK` 并退出 0,否则 workflow 失败;
|
|
16
|
+
- 本地有显示器时可加验:`cd desktop && MINGDAO_DESKTOP_SMOKE=1 dist/linux-unpacked/mingdao --no-sandbox`(AppImage 加 `--appimage-extract-and-run`);
|
|
17
|
+
- 教训:0.2.2 三平台启动即崩(main.js 顶层调用 createLogWriter 漏导入),打包期不报错、运行期才炸——静态护栏(test/smoke.js「桌面主进程静态护栏」组)与打包冒烟双保险。
|
|
18
|
+
6. **敏感信息**:密钥、token、内网地址、个人信息不得进入仓库与产物(`grep -rE 'sk-[A-Za-z0-9]{20,}' .`)。
|
|
19
|
+
7. **回滚方案**:明确如何回退到上一版本(旧包/旧 tag 可恢复)。
|
|
20
|
+
8. **发布后冒烟**:发布完成后立即做一次端到端验证。
|
|
21
|
+
|
|
22
|
+
## 桌面版发布流水线(官网分发 + 镜像 + 转运附件回收)
|
|
23
|
+
|
|
24
|
+
1. `npm version <v>`(preversion 自动跑 smoke/e2e/bench,version 钩子自动插 changelog,postversion 自动提交)→ 推 3 远端 + tag。
|
|
25
|
+
2. `npm publish` → GitHub desktop.yml CI 全绿(含打包冒烟)。
|
|
26
|
+
3. 服务器收割:`/tmp/harvest-<ver>.sh`(下载 7 安装包 + 2 yml,sha512 交叉校验 MATCH,再生成官网 3 份 feed)。
|
|
27
|
+
4. 官网 site/index.html 刷新版本/大小/sha256 校验行并部署;BBS 随 deploy.sh 重启。
|
|
28
|
+
5. gitee/gitcode 创建 Release(正文指向官网,无附件)。
|
|
29
|
+
6. 官网同步确认后:`MINGDAO_GITHUB_TOKEN=<PAT> node scripts/github-release-cleanup.mjs <版本号>` 删除 GitHub 转运附件(先 --dry-run)。
|
|
17
30
|
|
|
18
31
|
## 输出
|
|
19
32
|
|
package/src/agent.js
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
// → PostToolUse 钩子 → 结果回填 → 循环,直到模型给出纯文本回复。
|
|
3
3
|
// 附带:子代理(task 工具)、todo 清单状态、undo 备份仓、Ctrl+C 中断。
|
|
4
4
|
|
|
5
|
-
import { trimMessages, clampText, messageTokens } from './context.js';
|
|
5
|
+
import { trimMessages, clampText, messageTokens, approxTokens } from './context.js';
|
|
6
6
|
import { compactConversation } from './compact.js';
|
|
7
|
-
import {
|
|
7
|
+
import { buildToolSchemas, dispatch } from './tools/index.js';
|
|
8
8
|
import { modelPreset } from './models.js';
|
|
9
9
|
import { makeTokenCounter } from './tokenizer.js';
|
|
10
10
|
import { createHooks } from './hooks.js';
|
|
@@ -29,7 +29,7 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
29
29
|
const temperature = cfg.temperature ?? preset.temperature ?? 0.6;
|
|
30
30
|
const reasoningEffort = cfg.reasoningEffort ?? preset.reasoningEffort?.default ?? undefined;
|
|
31
31
|
const hooks = createHooks(cfg.hooks, workingDir);
|
|
32
|
-
const todos = [];
|
|
32
|
+
const todos = /** @type {any[]} */ ([]);
|
|
33
33
|
// 会话级共享:调用方传入则复用(/model 切换、子代理均共享,undo 不丢失)
|
|
34
34
|
const undo = undoStore || { backups: new Map() };
|
|
35
35
|
const stepLimit = maxSteps || MAX_STEPS;
|
|
@@ -39,14 +39,33 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
39
39
|
const count = makeTokenCounter(modelName);
|
|
40
40
|
// MCP 工具集(每次取,服务器晚就绪也能在后续轮次出现)
|
|
41
41
|
const mcpSchemas = () => (mcp ? mcp.toolSchemas() : []);
|
|
42
|
+
// 省钱 B1:本会话已调用过的工具名(内置名/MCP 前缀名)——其 schema 在后续轮次省略 description,
|
|
43
|
+
// 模型已在消息历史里见过用途;未用过的保留完整描述。省输入 token(工具 schema 按需瘦身)。
|
|
44
|
+
const usedToolNames = new Set();
|
|
45
|
+
// 省钱 B1(按需挂载):回合起始为「只读阶段」时只发只读工具(read/ls/glob/grep/skill/todo)
|
|
46
|
+
// + 已用过的工具;检测到写意图(用户消息或模型明说需要写/改/建)后注入全量工具。
|
|
47
|
+
const READONLY_TIER_SET = new Set(['read', 'ls', 'glob', 'grep', 'skill', 'todo']);
|
|
48
|
+
const WRITE_INTENT_RE = /写|建|创|改|修|删|装|加|添|增|补|换|移|部署|执行|运行|实现|重构|生成|迁移|安装|更新|升级|发布|调整|优化|修复|提交|推送|打包|编译|测试/;
|
|
49
|
+
const hasWriteIntent = (/** @type {any} */ text) => WRITE_INTENT_RE.test(String(text || ''));
|
|
50
|
+
const toolsFor = (/** @type {boolean} */ readOnlyPhase) => {
|
|
51
|
+
const schemas = buildToolSchemas(usedToolNames, mcpSchemas());
|
|
52
|
+
if (!readOnlyPhase) return schemas;
|
|
53
|
+
return schemas.filter((/** @type {any} */ t) => {
|
|
54
|
+
const n = t?.function?.name;
|
|
55
|
+
if (!n) return true;
|
|
56
|
+
if (READONLY_TIER_SET.has(n) || usedToolNames.has(n)) return true;
|
|
57
|
+
if (n.startsWith('mcp__')) return mcp ? mcp.isReadonly(n) : false;
|
|
58
|
+
return false;
|
|
59
|
+
});
|
|
60
|
+
};
|
|
42
61
|
|
|
43
62
|
// 子代理:全新上下文 + 同一 Provider/权限(提示带「子任务」标记),独立完成子任务后汇报
|
|
44
|
-
async function spawnTask(prompt, { description = '', readOnly = false } = {}) {
|
|
63
|
+
async function spawnTask(/** @type {any} */ prompt, { description = '', readOnly = false } = {}) {
|
|
45
64
|
const subIo = createIO({ quiet: true });
|
|
46
65
|
// 只读子代理(评估 A4:可并行):只读工具自动放行、写类直接拒绝,无交互询问
|
|
47
66
|
const subPermission = readOnly
|
|
48
|
-
? { mode: 'readonly-noask', check: (name) => READONLY_TOOLS_SET.has(name) }
|
|
49
|
-
: { check: (name, args) => permission.check(name, args, '(子任务)') };
|
|
67
|
+
? { mode: 'readonly-noask', check: (/** @type {any} */ name) => READONLY_TOOLS_SET.has(name) }
|
|
68
|
+
: { check: (/** @type {any} */ name, /** @type {any} */ args) => permission.check(name, args, '(子任务)') };
|
|
50
69
|
// 自动路由:子代理固定走 executor 模型(便宜的执行单元)
|
|
51
70
|
const subModel = subagentModel(cfg, modelName);
|
|
52
71
|
const subAgent = createAgent({
|
|
@@ -91,23 +110,47 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
91
110
|
budget,
|
|
92
111
|
todos,
|
|
93
112
|
undoStore: undo,
|
|
94
|
-
spawnTask: (prompt, opts) => spawnTask(prompt, opts),
|
|
113
|
+
spawnTask: (/** @type {any} */ prompt, /** @type {any} */ opts) => spawnTask(prompt, opts),
|
|
95
114
|
};
|
|
96
115
|
}
|
|
97
116
|
|
|
98
|
-
async function runTurn(messages) {
|
|
117
|
+
async function runTurn(/** @type {any} */ messages) {
|
|
99
118
|
let steps = 0;
|
|
100
119
|
let finish = null;
|
|
101
|
-
const usage = { prompt_tokens: 0, completion_tokens: 0 };
|
|
120
|
+
const usage = /** @type {{ prompt_tokens: number, completion_tokens: number, prompt_cache_hit_tokens?: number, prompt_cache_miss_tokens?: number }} */ ({ prompt_tokens: 0, completion_tokens: 0 });
|
|
102
121
|
const startedAt = Date.now();
|
|
103
122
|
// 回合性能指标(状态栏:LLM 时长 / 工具时长 / 首 token 延迟 / 步数)
|
|
104
123
|
let llmMsTotal = 0;
|
|
105
124
|
let toolMsTotal = 0;
|
|
106
|
-
let firstTokenAt = null;
|
|
107
|
-
|
|
125
|
+
let firstTokenAt = /** @type {any} */ (null);
|
|
126
|
+
// 省钱 B3(费用二级分账):推理 token 估算(按增量累计)与逐工具调用/耗时累加
|
|
127
|
+
let reasoningTokens = 0;
|
|
128
|
+
const toolStats = /** @type {Map<string, {calls: number, ms: number}>} */ (new Map());
|
|
129
|
+
const perf = () => ({
|
|
130
|
+
llmMs: llmMsTotal,
|
|
131
|
+
toolMs: toolMsTotal,
|
|
132
|
+
firstTokenMs: firstTokenAt == null ? null : firstTokenAt - startedAt,
|
|
133
|
+
steps,
|
|
134
|
+
reasoningTokens,
|
|
135
|
+
toolStats: [...toolStats.entries()].map(([tool, s]) => ({ tool, calls: s.calls, ms: s.ms })),
|
|
136
|
+
usedModel: activeModel, // 省钱 B4:本回合实际使用模型(降级后归属它)
|
|
137
|
+
});
|
|
108
138
|
let aborted = false;
|
|
109
139
|
let emptyRounds = 0; // 连续空/截断输出计数(防止无限续写)
|
|
110
|
-
let currentAc = null;
|
|
140
|
+
let currentAc = /** @type {any} */ (null);
|
|
141
|
+
// 省钱 B4(护栏降级):action='downgrade' 超限后本回合切换到便宜模型继续执行;
|
|
142
|
+
// activeModel 是本回合实际使用的模型(分账/记录归属它),downgraded 保证只提示一次。
|
|
143
|
+
let activeModel = modelName;
|
|
144
|
+
let downgraded = false;
|
|
145
|
+
// 省钱 B1:本回合只读阶段判定——最新用户消息无写意图则先只发只读工具,
|
|
146
|
+
// 模型明确表达写意图后(下一轮)注入全量。cfg.schemaTier=false 可关。
|
|
147
|
+
let readOnlyPhase = true;
|
|
148
|
+
if (cfg.schemaTier !== false) {
|
|
149
|
+
const lastUser = [...messages].reverse().find((m) => m?.role === 'user');
|
|
150
|
+
readOnlyPhase = !hasWriteIntent(lastUser?.content);
|
|
151
|
+
} else {
|
|
152
|
+
readOnlyPhase = false;
|
|
153
|
+
}
|
|
111
154
|
// 整个回合注册一次 SIGINT:思考、工具执行、权限询问期间都能中断
|
|
112
155
|
const offSigint = io.onSigint ? io.onSigint(() => { aborted = true; currentAc?.abort(); }) : () => {};
|
|
113
156
|
const ctx = makeCtx();
|
|
@@ -160,21 +203,22 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
160
203
|
for (const m of sanitized) {
|
|
161
204
|
const rc = m.reasoning_content;
|
|
162
205
|
if (typeof rc === 'string' && rc.length > 4000) {
|
|
163
|
-
sanitized = sanitized.map((x) => (x === m ? { ...x, reasoning_content: `[思考过程已省略(原 ${rc.length} 字)]` } : x));
|
|
206
|
+
sanitized = sanitized.map((/** @type {any} */ x) => (x === m ? { ...x, reasoning_content: `[思考过程已省略(原 ${rc.length} 字)]` } : x));
|
|
164
207
|
} else if (typeof rc === 'string' && rc.length > 1000) {
|
|
165
|
-
sanitized = sanitized.map((x) => (x === m ? { ...x, reasoning_content: rc.slice(-500) + ' …[思考过程已截断]' } : x));
|
|
208
|
+
sanitized = sanitized.map((/** @type {any} */ x) => (x === m ? { ...x, reasoning_content: rc.slice(-500) + ' …[思考过程已截断]' } : x));
|
|
166
209
|
}
|
|
167
210
|
}
|
|
168
211
|
|
|
169
212
|
// 护栏前置预估(Kimi P2-E):发送前按本轮最坏成本估算(trimmed prompt × 未命中输入价
|
|
170
213
|
// + maxOutput × 输出价,峰谷按当前时段);「今日已用 + 最坏成本」超上限时发送前拦截,
|
|
171
|
-
// 而不是等 200K 上下文的贵请求发出后才 block。
|
|
214
|
+
// 而不是等 200K 上下文的贵请求发出后才 block。action='downgrade'(省钱 B4)不在此拦截——
|
|
215
|
+
// 由降级流程接管:切到便宜模型后按 flash 价格重新估算,通常即可放行。
|
|
172
216
|
if (cfg.costGuard) {
|
|
173
217
|
const g = costGuardConfig();
|
|
174
|
-
if (g && Number(g.dailyLimitYuan) > 0) {
|
|
218
|
+
if (g && Number(g.dailyLimitYuan) > 0 && g.action !== 'downgrade') {
|
|
175
219
|
let promptTokens = 0;
|
|
176
220
|
for (const m of sanitized) promptTokens += messageTokens(m, count);
|
|
177
|
-
const worst = estimateCost(
|
|
221
|
+
const worst = estimateCost(activeModel, promptTokens, maxOutput, null, new Date());
|
|
178
222
|
const used = todayCost();
|
|
179
223
|
if (used + worst >= Number(g.dailyLimitYuan)) {
|
|
180
224
|
stripOrphanCalls();
|
|
@@ -187,7 +231,8 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
187
231
|
}
|
|
188
232
|
}
|
|
189
233
|
|
|
190
|
-
// 费用护栏(A2):每轮开始前按今日实际费用检查;block
|
|
234
|
+
// 费用护栏(A2/B4):每轮开始前按今日实际费用检查;block 暂停本轮;
|
|
235
|
+
// downgrade 自动切换便宜模型继续执行(每回合只切一次,切换即粘滞)
|
|
191
236
|
if (cfg.costGuard) {
|
|
192
237
|
const guard = checkCostGuard();
|
|
193
238
|
if (guard) {
|
|
@@ -206,7 +251,23 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
206
251
|
perf: perf(),
|
|
207
252
|
};
|
|
208
253
|
}
|
|
209
|
-
|
|
254
|
+
if (guard.downgrade && !downgraded) {
|
|
255
|
+
if (guard.downgradeModel !== activeModel) {
|
|
256
|
+
activeModel = guard.downgradeModel;
|
|
257
|
+
downgraded = true;
|
|
258
|
+
io.print(style(guard.message, C.yellow));
|
|
259
|
+
} else {
|
|
260
|
+
// 已经是降级目标模型:无法再降,按 block 处理
|
|
261
|
+
stripOrphanCalls();
|
|
262
|
+
return {
|
|
263
|
+
text: null, reasoning: '', usage, steps, finish, truncated: false, aborted: false,
|
|
264
|
+
note: `今日费用已达上限(实际 ¥${String(todayCost().toFixed(4))}),且已在最便宜模型上执行,已暂停——调整 config.costGuard 或明天自动恢复。`,
|
|
265
|
+
durationMs: Date.now() - startedAt, perf: perf(),
|
|
266
|
+
};
|
|
267
|
+
}
|
|
268
|
+
} else if (!guard.downgrade) {
|
|
269
|
+
io.print(style(guard.message, C.yellow));
|
|
270
|
+
}
|
|
210
271
|
}
|
|
211
272
|
}
|
|
212
273
|
|
|
@@ -221,18 +282,21 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
221
282
|
const llmT0 = Date.now();
|
|
222
283
|
try {
|
|
223
284
|
res = await provider.chat({
|
|
224
|
-
model:
|
|
285
|
+
model: activeModel,
|
|
225
286
|
messages: sanitized,
|
|
226
|
-
tools:
|
|
287
|
+
tools: toolsFor(readOnlyPhase),
|
|
227
288
|
temperature,
|
|
228
289
|
maxTokens: maxOutput,
|
|
229
290
|
reasoningEffort,
|
|
230
291
|
signal: ac.signal,
|
|
231
|
-
onDelta(d) {
|
|
292
|
+
onDelta(/** @type {any} */ d) {
|
|
232
293
|
io.stopSpinner();
|
|
233
294
|
if (firstTokenAt == null) firstTokenAt = Date.now(); // 首个增量即首 token
|
|
234
295
|
if (d.text) io.writeText(d.text);
|
|
235
|
-
if (d.reasoning)
|
|
296
|
+
if (d.reasoning) {
|
|
297
|
+
reasoningTokens += approxTokens(d.reasoning); // 省钱 B3:推理 token 估算(分账维度)
|
|
298
|
+
io.writeReasoning(d.reasoning);
|
|
299
|
+
}
|
|
236
300
|
},
|
|
237
301
|
});
|
|
238
302
|
llmMsTotal += Date.now() - llmT0;
|
|
@@ -248,6 +312,8 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
248
312
|
}
|
|
249
313
|
|
|
250
314
|
finish = res.finish ?? finish;
|
|
315
|
+
// 省钱 B1:只读阶段中模型文字明确表达写意图 → 下一轮注入全量工具(多一轮,几乎无感)
|
|
316
|
+
if (readOnlyPhase && hasWriteIntent(res.text)) readOnlyPhase = false;
|
|
251
317
|
if (res.usage) {
|
|
252
318
|
usage.prompt_tokens += res.usage.prompt_tokens || 0;
|
|
253
319
|
usage.completion_tokens += res.usage.completion_tokens || 0;
|
|
@@ -273,7 +339,7 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
273
339
|
|
|
274
340
|
// 预检:解析参数 → PreToolUse 钩子 → 权限检查;拒绝/失败只回填不执行(返回 null)
|
|
275
341
|
// task 工具标记 readOnly 时也可并行(评估 A4:只读子代理 Promise.all)
|
|
276
|
-
async function prepTool(tc) {
|
|
342
|
+
async function prepTool(/** @type {any} */ tc) {
|
|
277
343
|
const name = tc.function?.name || '';
|
|
278
344
|
// 审计(P3-5):参数与拒绝原因都记录(配置 audit:false 可关)
|
|
279
345
|
const auditOn = cfg.audit !== false;
|
|
@@ -287,7 +353,7 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
287
353
|
args: auditArgs(),
|
|
288
354
|
...extra,
|
|
289
355
|
});
|
|
290
|
-
let args = null;
|
|
356
|
+
let args = /** @type {any} */ (null);
|
|
291
357
|
try {
|
|
292
358
|
args = JSON.parse(tc.function?.arguments || '{}');
|
|
293
359
|
} catch {
|
|
@@ -337,8 +403,9 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
337
403
|
// 执行单个工具(渲染「执行中」→ dispatch → 捕获异常转错误结果)
|
|
338
404
|
// 审计 Hermes C4:同回合相同参数的只读工具(read/ls/glob/grep/skill)合并执行一次,
|
|
339
405
|
// 后续相同调用直接复用结果(仍逐个回填 tool 消息以保持 tool_call_id 配对)
|
|
340
|
-
async function runTool(prep) {
|
|
406
|
+
async function runTool(/** @type {any} */ prep) {
|
|
341
407
|
io.renderToolStart?.(prep.name, prep.args);
|
|
408
|
+
usedToolNames.add(prep.name); // 省钱 B1:执行过即标记,后续轮次省略其 description
|
|
342
409
|
const dedupKey = !prep.isMcp && READONLY_TOOLS_SET.has(prep.name) ? prep.name + ':' + JSON.stringify(prep.args || {}) : null;
|
|
343
410
|
if (dedupKey && turnToolCache.has(dedupKey)) {
|
|
344
411
|
prep.cached = true;
|
|
@@ -354,15 +421,20 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
354
421
|
}
|
|
355
422
|
if (dedupKey) turnToolCache.set(dedupKey, result);
|
|
356
423
|
return result;
|
|
357
|
-
} catch (err) {
|
|
424
|
+
} catch (/** @type {any} */ err) {
|
|
358
425
|
return JSON.stringify({ ok: false, error: String(err?.message || err) });
|
|
359
426
|
}
|
|
360
427
|
}
|
|
361
428
|
|
|
362
429
|
// 收尾:渲染结果 → todo 更新 → PostToolUse 钩子 → 回填消息(顺序与串行一致)
|
|
363
|
-
function finishTool(prep, result, t0) {
|
|
430
|
+
function finishTool(/** @type {any} */ prep, /** @type {any} */ result, /** @type {any} */ t0) {
|
|
364
431
|
const ms = Date.now() - t0;
|
|
365
432
|
toolMsTotal += ms;
|
|
433
|
+
// 省钱 B3:逐工具调用/耗时累加(费用二级分账的 byTool 维度)
|
|
434
|
+
const ts = toolStats.get(prep.name) || { calls: 0, ms: 0 };
|
|
435
|
+
ts.calls += 1;
|
|
436
|
+
ts.ms += ms;
|
|
437
|
+
toolStats.set(prep.name, ts);
|
|
366
438
|
io.renderTool(prep.name, prep.args, result, ms);
|
|
367
439
|
if (prep.name === 'todo' && result?.todos) io.renderTodo(result.todos);
|
|
368
440
|
hooks.post(prep.name, prep.args, typeof result === 'string' ? { output: result } : result).catch(() => {});
|
|
@@ -401,7 +473,7 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
401
473
|
const batch = []; // {prep, batchable}
|
|
402
474
|
while (i < res.toolCalls.length) {
|
|
403
475
|
const tc = res.toolCalls[i];
|
|
404
|
-
const prep = await prepTool(tc);
|
|
476
|
+
const prep = /** @type {any} */ (await prepTool(tc));
|
|
405
477
|
const name = tc.function?.name || '';
|
|
406
478
|
let batchable = canBatch && Boolean(prep) && !prep.isMcp && READONLY_BATCH.has(name);
|
|
407
479
|
if (!batchable && canBatch && Boolean(prep) && name === 'task' && prep.args?.readOnly === true) batchable = true;
|
|
@@ -529,7 +601,7 @@ export function createAgent({ provider, permission, io, modelName, workingDir, c
|
|
|
529
601
|
maxOutput,
|
|
530
602
|
temperature,
|
|
531
603
|
runTurn,
|
|
532
|
-
spawnTask: (prompt, opts) => spawnTask(prompt, opts),
|
|
604
|
+
spawnTask: (/** @type {any} */ prompt, /** @type {any} */ opts) => spawnTask(prompt, opts),
|
|
533
605
|
getTodos: () => todos.slice(),
|
|
534
606
|
};
|
|
535
607
|
}
|
package/src/audit.js
CHANGED
|
@@ -17,11 +17,11 @@ export function auditFile() {
|
|
|
17
17
|
}
|
|
18
18
|
|
|
19
19
|
// 轻量脱敏:sk- 系 API Key 掩码(审计日志可安全共享排查)
|
|
20
|
-
export function redactSecrets(text) {
|
|
20
|
+
export function redactSecrets(/** @type {any} */ text) {
|
|
21
21
|
return String(text ?? '').replace(/(sk-[A-Za-z0-9_-]{6,})/g, 'sk-***');
|
|
22
22
|
}
|
|
23
23
|
|
|
24
|
-
export function writeAudit(entry) {
|
|
24
|
+
export function writeAudit(/** @type {any} */ entry) {
|
|
25
25
|
try {
|
|
26
26
|
ensureHome();
|
|
27
27
|
const file = auditFile();
|
package/src/batch.js
CHANGED
|
@@ -8,15 +8,18 @@
|
|
|
8
8
|
|
|
9
9
|
import fs from 'node:fs';
|
|
10
10
|
import path from 'node:path';
|
|
11
|
+
import crypto from 'node:crypto';
|
|
11
12
|
import { resolveProviderConfig } from './providers/index.js';
|
|
12
13
|
import { estimateBatchCost, BATCH_DISCOUNT } from './pricing.js';
|
|
13
14
|
import { recordCacheStats } from './cachestats.js';
|
|
15
|
+
import { approxTokens } from './context.js';
|
|
16
|
+
import { modelPreset } from './models.js';
|
|
14
17
|
|
|
15
18
|
const DEFAULT_WINDOW = '24h';
|
|
16
19
|
const DEFAULT_ENDPOINT = '/v1/chat/completions';
|
|
17
20
|
|
|
18
21
|
// 批处理端点基址:config.batchBaseUrl 优先;否则取当前服务商 baseUrl 去掉 /v1 后缀
|
|
19
|
-
function batchBase(cfg, model) {
|
|
22
|
+
function batchBase(/** @type {any} */ cfg, /** @type {any} */ model) {
|
|
20
23
|
const explicit = String(cfg?.batchBaseUrl || '').trim().replace(/\/+$/, '');
|
|
21
24
|
if (explicit) return explicit;
|
|
22
25
|
const pc = resolveProviderConfig(cfg, model);
|
|
@@ -26,7 +29,7 @@ function batchBase(cfg, model) {
|
|
|
26
29
|
}
|
|
27
30
|
|
|
28
31
|
/** @returns {Promise<any>} */
|
|
29
|
-
async function api(base, apiKey, methodPath, payload, httpMethod = 'POST') {
|
|
32
|
+
async function api(/** @type {any} */ base, /** @type {any} */ apiKey, /** @type {any} */ methodPath, /** @type {any} */ payload, httpMethod = 'POST') {
|
|
30
33
|
const res = await fetch(base + methodPath, {
|
|
31
34
|
method: httpMethod,
|
|
32
35
|
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
|
|
@@ -41,7 +44,7 @@ async function api(base, apiKey, methodPath, payload, httpMethod = 'POST') {
|
|
|
41
44
|
return j;
|
|
42
45
|
}
|
|
43
46
|
|
|
44
|
-
async function uploadFile(base, apiKey, jsonl) {
|
|
47
|
+
async function uploadFile(/** @type {any} */ base, /** @type {any} */ apiKey, /** @type {any} */ jsonl) {
|
|
45
48
|
const form = new FormData();
|
|
46
49
|
form.append('file', new Blob([jsonl], { type: 'application/jsonl' }), 'mingdao-batch.jsonl');
|
|
47
50
|
form.append('purpose', 'batch');
|
|
@@ -59,7 +62,7 @@ async function uploadFile(base, apiKey, jsonl) {
|
|
|
59
62
|
return j.id;
|
|
60
63
|
}
|
|
61
64
|
|
|
62
|
-
async function downloadResults(base, apiKey, batch) {
|
|
65
|
+
async function downloadResults(/** @type {any} */ base, /** @type {any} */ apiKey, /** @type {any} */ batch) {
|
|
63
66
|
// DeepSeek 风格:直接取结果文件;回退 OpenAI 风格:按 output_file_id 取内容
|
|
64
67
|
const attempts = [
|
|
65
68
|
`/batches/${batch.id}/files/result`,
|
|
@@ -84,10 +87,12 @@ async function downloadResults(base, apiKey, batch) {
|
|
|
84
87
|
throw new Error('批处理结果文件不可用(端点不支持或文件已过期)');
|
|
85
88
|
}
|
|
86
89
|
|
|
87
|
-
/** 执行一次批处理。questions: string[]。返回 { ok, outputFile, results, usage, cost, batchId }
|
|
88
|
-
*
|
|
89
|
-
|
|
90
|
-
|
|
90
|
+
/** 执行一次批处理。questions: string[]。返回 { ok, outputFile, results, usage, cost, batchId, deduped }
|
|
91
|
+
* 省钱 B2:①文本哈希去重(重复问题只提交一次,结果回填全部位置);②单问超窗口预检(提交前报错,不烧钱);
|
|
92
|
+
* ③maxCost 预算上限(提交前按估算费用拦截,超出直接中止不提交)。
|
|
93
|
+
* @param {{ cfg: any, model: any, questions: any, workingDir?: string, maxTokens?: number, temperature?: any, signal?: any, onStatus?: any, maxCost?: number }} opts */
|
|
94
|
+
export async function runBatch({ cfg, model, questions, workingDir = process.cwd(), maxTokens = 4096, temperature, signal, onStatus, maxCost = 0 }) {
|
|
95
|
+
const list = (questions || []).map((/** @type {any} */ q) => String(q).trim()).filter(Boolean);
|
|
91
96
|
if (!list.length) return { error: '没有可批处理的问题(每行一个问题)' };
|
|
92
97
|
const pc = resolveProviderConfig(cfg, model);
|
|
93
98
|
if (!pc.apiKey) return { error: `模型 ${model} 没有可用 API Key(mingdao key set ${pc.name})` };
|
|
@@ -97,6 +102,50 @@ export async function runBatch({ cfg, model, questions, workingDir = process.cwd
|
|
|
97
102
|
// 不再携带完整系统提示(技能清单/用户记忆/AGENTS.md 对无工具批任务毫无意义,1000 问
|
|
98
103
|
// 可白烧 50-80 万 token);改用一行精简角色提示,剩余能力损失为零。
|
|
99
104
|
const systemPrompt = '你是 MingDao Harness 编程助手。直接针对每个问题给出准确、完整的答案,不要复述问题、不要解释过程。';
|
|
105
|
+
|
|
106
|
+
// —— 省钱 B2:文本哈希去重(trim 后逐字节一致视为重复)——
|
|
107
|
+
const positions = /** @type {Map<string, number[]>} */ (new Map()); // hash → [输入行下标...]
|
|
108
|
+
const unique = /** @type {string[]} */ ([]);
|
|
109
|
+
list.forEach((/** @type {string} */ q, /** @type {number} */ i) => {
|
|
110
|
+
const h = crypto.createHash('sha256').update(q).digest('hex');
|
|
111
|
+
if (!positions.has(h)) {
|
|
112
|
+
positions.set(h, [i]);
|
|
113
|
+
unique.push(q);
|
|
114
|
+
} else {
|
|
115
|
+
positions.get(h)?.push(i);
|
|
116
|
+
}
|
|
117
|
+
});
|
|
118
|
+
const posByUniqueIdx = unique.map((/** @type {string} */ q) => positions.get(crypto.createHash('sha256').update(q).digest('hex')) || []);
|
|
119
|
+
const deduped = list.length - unique.length;
|
|
120
|
+
if (deduped > 0) onStatus?.(`去重:${deduped} 条重复问题合并(${list.length} → ${unique.length} 条实际提交)`);
|
|
121
|
+
|
|
122
|
+
// —— 省钱 B2:单问超窗口预检(估算输入 + max_tokens 超过模型窗口 95% 即报错,绝不提交烧钱)——
|
|
123
|
+
const preset = modelPreset(model) || {};
|
|
124
|
+
const windowTokens = Number(preset.budgetTokens) || 128000;
|
|
125
|
+
const sysTokens = approxTokens(systemPrompt);
|
|
126
|
+
for (let i = 0; i < unique.length; i++) {
|
|
127
|
+
const est = sysTokens + approxTokens(unique[i]);
|
|
128
|
+
if (est + maxTokens > windowTokens * 0.95) {
|
|
129
|
+
return {
|
|
130
|
+
error: `第 ${posByUniqueIdx[i][0] + 1} 个问题超窗口:估算输入 ${est} + max_tokens ${maxTokens} > 模型窗口 ${windowTokens}(可 --max-tokens 调小或拆分问题)`,
|
|
131
|
+
};
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
// —— 省钱 B2:--max-cost 预算上限(提交前估算拦截)——
|
|
136
|
+
let estimatedCost = 0;
|
|
137
|
+
if (maxCost > 0) {
|
|
138
|
+
const estPrompt = unique.reduce((s, q) => s + sysTokens + approxTokens(q), 0);
|
|
139
|
+
const estCompletion = unique.length * Math.min(maxTokens, 2048); // 保守按平均 2K 输出估算
|
|
140
|
+
estimatedCost = estimateBatchCost(model, estPrompt, estCompletion);
|
|
141
|
+
if (estimatedCost > maxCost) {
|
|
142
|
+
return {
|
|
143
|
+
error: `预计费用 ≈¥${estimatedCost.toFixed(4)}(已按半价)超过 --max-cost ¥${maxCost},已中止提交。可缩小问题集或调高上限。`,
|
|
144
|
+
estimatedCost,
|
|
145
|
+
};
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
|
|
100
149
|
const bodyTemplate = {
|
|
101
150
|
model,
|
|
102
151
|
messages: null, // 逐行填充
|
|
@@ -105,7 +154,7 @@ export async function runBatch({ cfg, model, questions, workingDir = process.cwd
|
|
|
105
154
|
stream: false,
|
|
106
155
|
};
|
|
107
156
|
const jsonl =
|
|
108
|
-
|
|
157
|
+
unique
|
|
109
158
|
.map((q, i) =>
|
|
110
159
|
JSON.stringify({
|
|
111
160
|
custom_id: `md-${i}`,
|
|
@@ -143,7 +192,7 @@ export async function runBatch({ cfg, model, questions, workingDir = process.cwd
|
|
|
143
192
|
failures = 0;
|
|
144
193
|
} catch (err) {
|
|
145
194
|
failures += 1;
|
|
146
|
-
if (failures >= 10) return { error: `轮询失败:${err?.message || err}(任务仍在服务端,ID ${batch.id})`, batchId: batch.id };
|
|
195
|
+
if (failures >= 10) return { error: `轮询失败:${(/** @type {any} */ (err))?.message || err}(任务仍在服务端,ID ${batch.id})`, batchId: batch.id };
|
|
147
196
|
await new Promise((r) => setTimeout(r, Math.min(baseInterval * 1.5 ** failures, 30000)));
|
|
148
197
|
continue;
|
|
149
198
|
}
|
|
@@ -164,26 +213,35 @@ export async function runBatch({ cfg, model, questions, workingDir = process.cwd
|
|
|
164
213
|
}
|
|
165
214
|
onStatus?.('下载结果…');
|
|
166
215
|
const results = await downloadResults(base, apiKey, batch);
|
|
167
|
-
// 汇总 usage 与费用(batch
|
|
216
|
+
// 汇总 usage 与费用(batch 半价);结果按 custom_id 回填到全部重复位置(省钱 B2)
|
|
168
217
|
let prompt = 0;
|
|
169
218
|
let completion = 0;
|
|
170
|
-
const outputs =
|
|
219
|
+
const outputs = /** @type {Array<{id: string, content: string}>} */ (new Array(list.length));
|
|
171
220
|
for (const r of results) {
|
|
172
221
|
const body = r?.response?.body || {};
|
|
173
222
|
prompt += body.usage?.prompt_tokens || 0;
|
|
174
223
|
completion += body.usage?.completion_tokens || 0;
|
|
175
224
|
const content = body.choices?.[0]?.message?.content ?? body.choices?.[0]?.text ?? (r?.response?.status_code !== 200 ? `(错误 ${r?.response?.status_code})` : '');
|
|
176
|
-
|
|
225
|
+
const m = /^md-(\d+)$/.exec(String(r.custom_id || ''));
|
|
226
|
+
const ui = m ? Number(m[1]) : -1;
|
|
227
|
+
const idxs = ui >= 0 && ui < posByUniqueIdx.length ? posByUniqueIdx[ui] : [];
|
|
228
|
+
for (const pos of idxs) {
|
|
229
|
+
outputs[pos] = { id: String(pos), content: String(content || '').trim() };
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
// 防御:服务端漏回的 custom_id 用占位补全,保证输出行数与输入行数一一对应
|
|
233
|
+
for (let i = 0; i < outputs.length; i++) {
|
|
234
|
+
if (!outputs[i]) outputs[i] = { id: String(i), content: '' };
|
|
177
235
|
}
|
|
178
236
|
const usage = { prompt_tokens: prompt, completion_tokens: completion };
|
|
179
237
|
const cost = estimateBatchCost(model, prompt, completion);
|
|
180
238
|
const outFile = path.join(workingDir, `mingdao-batch-result-${Date.now()}.jsonl`);
|
|
181
239
|
fs.writeFileSync(outFile, outputs.map((o) => JSON.stringify(o)).join('\n') + '\n');
|
|
182
240
|
recordCacheStats({ model, prompt, completion, hit: null, miss: null, cost, saved: null, batch: true });
|
|
183
|
-
onStatus?.(`完成:${outputs.length}
|
|
184
|
-
return { ok: true, batchId: batch.id, outputFile: outFile, results: outputs, usage, cost, discount: BATCH_DISCOUNT };
|
|
241
|
+
onStatus?.(`完成:${outputs.length} 条结果${deduped ? `(去重合并 ${deduped} 条)` : ''}`);
|
|
242
|
+
return { ok: true, batchId: batch.id, outputFile: outFile, results: outputs, usage, cost, discount: BATCH_DISCOUNT, deduped, estimatedCost };
|
|
185
243
|
} catch (err) {
|
|
186
244
|
// 批处理端点不支持(404/405 等)→ 明确告知,不静默
|
|
187
|
-
return { error: `批处理不可用:${err?.message || err}(该服务商可能不支持 Batch API,可用 config.batchBaseUrl 指定支持的网关)` };
|
|
245
|
+
return { error: `批处理不可用:${(/** @type {any} */ (err))?.message || err}(该服务商可能不支持 Batch API,可用 config.batchBaseUrl 指定支持的网关)` };
|
|
188
246
|
}
|
|
189
247
|
}
|