@feng0531/dsh-code-review 0.84.0 → 0.85.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/DEVELOPMENT.md +16 -0
- package/README.md +8 -0
- package/RELEASE-NOTES.md +14 -8
- package/index.mjs +1 -1
- package/package.json +1 -1
- package/src/anchor-correction.mjs +6 -4
- package/src/dsh-model.mjs +117 -86
- package/src/model-contract.mjs +10 -5
- package/src/model-diagnostics.mjs +62 -0
- package/src/model-json.mjs +18 -10
- package/src/project-execution.mjs +40 -0
- package/src/project-review-service.mjs +12 -13
- package/src/response-correction.mjs +28 -0
- package/src/retrieval-loop.mjs +57 -12
- package/src/review.mjs +64 -34
- package/src/snapshot-reader.mjs +95 -48
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,20 @@
|
|
|
2
2
|
|
|
3
3
|
本文件介绍对使用者有意义的变化。安装与参数见 [README](README.md),本次发布正文见 [发布说明](RELEASE-NOTES.md)。历史提交和安装包保留在 [GitHub Releases](https://github.com/f-e-n-g-0531/dsh-code-review/releases);本文不把开发日志当作功能介绍。
|
|
4
4
|
|
|
5
|
+
## 0.85.1 — 模型失败诊断与定向纠正
|
|
6
|
+
|
|
7
|
+
- 修复宿主 error 终态详情读取位置,保留实际 `finish.reason.failure` 中的已知错误码、HTTP 状态和有效重试间隔,便于区分服务失败类别;不新增对鉴权、额度或取消的盲目重试。
|
|
8
|
+
- 非法 JSON 记录固定语法分类、可用的行列位置及响应指纹,按最近一次分类定向请求重新生成;不回放错误原文,不自动修补 JSON,最多两次纠正且继续遵循原调用预算。
|
|
9
|
+
- 新增诊断不保存模型响应片段或提供方原始 message,未知错误码和请求 ID 仅保留哈希;隔离进度回调对纠正诊断的修改,保持异常身份和 CR/LF/CRLF 定位一致。
|
|
10
|
+
- 当前宿主没有原生 JSON mode/response schema 接口,本版不声称保证模型始终输出合法 JSON,也不以离线验证替代真实模型失败率复测。
|
|
11
|
+
|
|
12
|
+
## 0.85.0 — 审查失败恢复与全项目提速
|
|
13
|
+
|
|
14
|
+
- 全项目独立批次默认 3 路并行,插件配置 projectConcurrency 支持 1~8;预览/报告显示并发数,结果仍按原批次顺序合并,保持共享调用上限和取消后等待全部清理。批内主审与复核不跳过,预算竞争时未完成文件仍待处理。
|
|
15
|
+
- 无效 JSON 最多两次显式纠正,记录所有尝试的脱敏诊断与实际调用次数;不修补响应,不重试取消、提供方失败或不完整输出。纠正结果不再插入可能破坏阶段 schema 的元数据。
|
|
16
|
+
- 主审身份、严重性、字段或行证据校验失败可请求一次新响应,仍严格验证,不自动改身份、映射严重性或伪造证据。全项目模式支持同文件当前侧唯一精确文本的行号修正,保留原候选并继续窗口校验。
|
|
17
|
+
- 向模型提供剩余检索预算;预检整批动态读取范围及预算,避免无效批次部分消耗。预算不足仅给一次受限最终答复,文件仍未完成,不把审查失败当缺陷或通过。
|
|
18
|
+
|
|
5
19
|
## 0.84.0 — 暂存区与未暂存审查范围
|
|
6
20
|
|
|
7
21
|
新增 Git `changeScope`:`staged` 只审查 HEAD 到索引的已暂存内容,工作区里尚未暂存的修改不作为变更正文参与或发送(上下文与规则文件仍按工作区读取);`unstaged` 只审查索引到工作区的未暂存改动,已暂存内容仅作旧侧基线。省略时仍是 HEAD 到工作区的净变化。两个范围只用于 Git 工作副本,不能与历史版本、SVN 参数或全项目模式混用;未跟踪文件仍默认排除。预览与报告会标明实际比较范围,执行前重新核验对应内容。
|
package/DEVELOPMENT.md
CHANGED
|
@@ -18,6 +18,22 @@
|
|
|
18
18
|
|
|
19
19
|
项目捕获最多10000清单节点、1000选择节点、16MiB,不设默认总模型调用次数或执行秒表,初次捕获/复核均遵循宿主与用户取消。SVN项目模式拒绝不支持节点属性,不隐式继承固定历史审查的网络权限。
|
|
20
20
|
|
|
21
|
+
## 全项目并发与离线速度对照
|
|
22
|
+
|
|
23
|
+
插件配置 `projectConcurrency` 默认为 3,范围 1~8;1 为串行回退。仅独立批准批次并行,各批仍有独立检索来源和账本,所有调用与 JSON 纠正共享原执行计数。该上限按单次执行生效,不是全宿主 RPM/TPM 限流器;提供方不支持并发或遇到 429 时应调低。取消/致命检查点错误必须等待全部在途模型清理后释放会话锁,后台 checkpoint 继续使用原串行持久化队列。
|
|
24
|
+
|
|
25
|
+
运行 `node scripts/review-speed-benchmark.mjs` 比较相同 12 个 mock 批次在串行与 3 路并发下的耗时;脚本模拟每次模型调用 40ms 延迟,不访问网络、不证明真实模型速度或质量。核对工作量、调用数、完成数和输出顺序一致,不把降低覆盖作为加速。
|
|
26
|
+
|
|
27
|
+
## 模型响应失败与诊断
|
|
28
|
+
|
|
29
|
+
当前核验的 DSH `llm.stream` 接口没有原生 JSON mode/response schema 参数;插件不发送会被忽略的 `responseFormat`,也不把工具参数 schema 当作结构化输出保证。先要求纯 JSON,再做本地解析和阶段 schema、证据校验。
|
|
30
|
+
|
|
31
|
+
正常 stop 后的非法 JSON 最多重新生成两次,共享原调用预算与取消信号;纠正请求只带固定语法分类和可用的行列位置,不回放错误响应、不自动修补 JSON、不删减发现来换取通过。语法位置相对于去掉外层空白及完整代码围栏后的 JSON 正文,偏移单位为 UTF-16 code unit;未知运行时错误文本回退到通用分类,不推测位置。
|
|
32
|
+
|
|
33
|
+
失败结果的 `protocolDiagnostic` 保留语法分类、行列位置(可用时)和响应 SHA-256 指纹,不存储响应片段或原生解析异常文本。宿主服务错误从 `finish.reason.failure` 读取,保留已知错误码、HTTP 状态、有效重试间隔和请求 ID 的哈希;未知码只存分类及哈希,不存提供方 message、原始 ID、URL、headers 或 body。已有工具链直接抛出的异常仍保持身份,不新增自动网络重试;AUTH、QUOTA、取消及 error 终态不能按格式错误重发。指纹只是对照线索,不是原文备份,也不构成跨模型确定性保证。
|
|
34
|
+
|
|
35
|
+
执行 `node --test test/model-diagnostics.test.mjs test/model-json-correction.test.mjs test/model-protocol-diagnostics.test.mjs test/dsh-model.test.mjs` 可验证本地纠正、失败归类和脱敏;宿主检查另覆盖真实 ToolRuntime 的纠正和服务失败事件,均为受控离线模型,不证明真实服务失败率下降。
|
|
36
|
+
|
|
21
37
|
## 审查预算
|
|
22
38
|
|
|
23
39
|
除 README 中的文件和输入上限外:
|
package/README.md
CHANGED
|
@@ -256,6 +256,8 @@ Git / SVN 历史审查也能使用这些参数;依赖和调用方按对应版
|
|
|
256
256
|
|
|
257
257
|
清单上限10000节点,单次选择最多1000节点/16MiB,12节点一批,单文件256KiB;超限明确拒绝,缩小选择后必须重新预览。单次采集共享8MiB元数据内存保护,不按命令条数、插件秒表或模型调用次数中断;耗时受宿主/模型提供方限制,用户可随时取消。SVN为本地已观察清单,稀疏/外部/链接及不支持属性显示限制;不证明仓库远端完整覆盖。
|
|
258
258
|
|
|
259
|
+
全项目审查默认最多 **3 个独立批次并行**,每批内主审、检索、复核仍按依赖顺序执行;预览和报告的 `execution.batchConcurrency` 显示实际配置。插件配置 `projectConcurrency` 可设为 1~8,设 1 回退串行;遇到模型提供方并发限制或 429,应降低并发。该设置不是 preview 参数,不扩大来源或共享模型调用预算,也不减少模型调用量或承诺 token 成本下降。仅一个批次时不会获得批间加速;实际提速受提供方吞吐、批次均衡及检查点写入影响。
|
|
260
|
+
|
|
259
261
|
对外只交付一份统一最终报告,给出总数、completed、excluded、blocked、pending、去重后的发现及未完成综合阶段;内部批次快照仅供证据追溯,不需要用户拼接报告。completed只表示所选捕获文件已处理,不代表代码正确;排除与阻断从不算完成。代码或清单变化会拒绝旧预览,运行后无法复核则标记outdated。
|
|
260
262
|
|
|
261
263
|
## 工具调用流程(给需要精确指定参数的人)
|
|
@@ -283,6 +285,12 @@ Git / SVN 历史审查也能使用这些参数;依赖和调用方按对应版
|
|
|
283
285
|
|
|
284
286
|
每个候选问题会保留位置、触发条件、影响和复核结果。复核可能支持、反驳、无法确定或未完成;引用原文匹配也不等于缺陷一定成立。**没有发现问题不证明没有缺陷,读取过上下文也不等于审查了整个项目。**
|
|
285
287
|
|
|
288
|
+
### 模型协议错误与读取预算恢复
|
|
289
|
+
|
|
290
|
+
已启用的模型桥接对完整结束但 JSON 无效的响应最多请求两次明确纠正(共最多三次调用),不会修补损坏 JSON;取消、提供方错误、不完整结束或输出超限不作此类重试。主审响应中的文件身份、严重性、必填字段或行证据校验失败时,最多请求一次完整响应纠正,仍按原快照、输入及调用预算严格验证。每次额外调用和纠正事件都会计数记录,恢复失败仍标为失败,不代表发现了缺陷。
|
|
291
|
+
|
|
292
|
+
全项目模式可将行号错位的证据重新定位到同文件当前侧的唯一精确文本;不跨文件、跨版本或模糊匹配,原候选保留供审计,窗口范围仍须通过校验。模型会收到共享检索预算余量;预算耗尽或请求超过余量时仅给一次不再检索的最终答复机会,不扩大读取权限。预算受限的文件仍列为未完成,已有有效候选保留,不会因返回空 findings 就算审查通过。
|
|
293
|
+
|
|
286
294
|
## 常见问题
|
|
287
295
|
|
|
288
296
|
| 遇到的问题 | 处理方法 |
|
package/RELEASE-NOTES.md
CHANGED
|
@@ -1,13 +1,19 @@
|
|
|
1
|
-
# v0.
|
|
1
|
+
# v0.85.1 发布说明
|
|
2
2
|
|
|
3
|
-
##
|
|
3
|
+
## 更可定位的模型失败
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
修复宿主服务错误详情读取位置:从实际的 `finish.reason.failure` 取得已知错误码、HTTP 状态及有效重试间隔,不再因读取错字段而只留下 `Model did not complete: error`。提供方没有返回的细节仍不可推断;本版不新增自动网络重试。
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
- `unstaged`:比较索引与工作区,只审查未暂存改动。已暂存内容只作为左侧基线,不代表由本次改动引入。
|
|
9
|
-
- 省略该参数时行为不变,仍是 HEAD 到工作区的净变化。
|
|
7
|
+
## 针对语法问题的有界纠正
|
|
10
8
|
|
|
11
|
-
|
|
9
|
+
正常 stop 后的非法 JSON 现在记录固定语法分类、可用的正文行列位置和 SHA-256 响应指纹,并根据最近一次分类给出定向重新生成提示。偏移使用 JSON 正文的 UTF-16 单位,支持 CR、LF、CRLF;运行时未提供位置时明确不填,不虚构错误位置。
|
|
12
10
|
|
|
13
|
-
|
|
11
|
+
仍最多重新生成两次,继续共享原调用预算和取消信号;不回放错误响应,不正则修补 JSON,不删除发现来换取格式通过,最终结果仍必须通过阶段 schema 与证据校验。进度回调拿到独立诊断副本,不能改变下一次纠正提示;异常访问器或 Proxy 不替换原始提供方异常。
|
|
12
|
+
|
|
13
|
+
## 安全与验证边界
|
|
14
|
+
|
|
15
|
+
新增诊断不保存模型响应片段、原生解析异常原文或提供方原始 message;未知错误码与请求 ID 仅保存哈希。已有直接抛出的异常仍保持身份,不把诊断扩展描述为全局日志脱敏。
|
|
16
|
+
|
|
17
|
+
已增加语法定位、定向纠正、真实宿主终态字段、脱敏、预算、取消及回调隔离回归,并由真实宿主 ToolRuntime 配合受控离线模型验证。当前宿主没有公开原生 JSON mode/response schema 能力,本版本不通过虚构参数绕过该限制;不保证模型永不出错,也没有把离线测试说成真实服务失败率已经下降。
|
|
18
|
+
|
|
19
|
+
用法见 [README](README.md),诊断字段与验证方式见 [DEVELOPMENT](DEVELOPMENT.md)。发布不自动安装、升级或重启 DSH;需要用户更新并重新加载插件。
|
package/index.mjs
CHANGED
|
@@ -13,7 +13,7 @@ const EXECUTE_PARAMETERS = { type: 'object', additionalProperties: false, proper
|
|
|
13
13
|
export function apply(ctx, config = {}) {
|
|
14
14
|
// This entry runs only when the host invokes the registered tool.
|
|
15
15
|
// Host tool policy owns permission; do not request a second approval here.
|
|
16
|
-
const service = createReviewService(ctx.llm, { allowModelSending: true, enableRiskPlanning: true, enableBusinessGrouping: true, enableAnchorCorrection: true, authorize: async () => true });
|
|
16
|
+
const service = createReviewService(ctx.llm, { allowModelSending: true, enableRiskPlanning: true, enableBusinessGrouping: true, enableAnchorCorrection: true, projectConcurrency: config.projectConcurrency ?? 3, authorize: async () => true });
|
|
17
17
|
ctx.tools.register({
|
|
18
18
|
name: 'code_review_preview', description: PREVIEW_DESCRIPTION, parameters: PREVIEW_PARAMETERS, output: reviewToolOutput,
|
|
19
19
|
async execute(args, exec) { const preview = await service.preview(args, exec); return { json: JSON.stringify(preview), markdown: '' }; },
|
package/package.json
CHANGED
|
@@ -6,13 +6,15 @@ export function correctAnchor(raw, file, changes, {signal}={}) {
|
|
|
6
6
|
const text=(a.side==='old'?file.left:file.right)?.text;if(typeof text!=='string'||Buffer.byteLength(text)>256*1024)return result('not-applicable','source-limit');
|
|
7
7
|
const source=text.replaceAll('\r\n','\n').split('\n'),snippet=a.snippet.replaceAll('\r\n','\n').split('\n');
|
|
8
8
|
if(source.slice(a.start-1,a.end).join('\n')===snippet.join('\n'))return result('unchanged','already-exact');
|
|
9
|
-
|
|
10
|
-
|
|
9
|
+
const currentSource = file.reviewMode === 'project' && a.side === 'new';
|
|
10
|
+
if(file.reviewMode === 'project' && !currentSource)return result('rejected','project-requires-new-side');
|
|
11
|
+
if(!currentSource&&(changes?.status!=='changed'||changes.precision!=='exact'||!Array.isArray(raw.attribution?.editIds)||!raw.attribution.editIds.length))return result('rejected','missing-exact-edit');
|
|
12
|
+
const edits=currentSource?[]:changes.edits.filter(e=>raw.attribution.editIds.includes(e.id));if(!currentSource&&edits.length!==raw.attribution.editIds.length)return result('rejected','unknown-or-duplicate-edit');
|
|
11
13
|
// Bound CPU even for a long multiline snippet; no silent partial search acceptance.
|
|
12
14
|
if(source.length*snippet.length>2000000)return result('rejected','search-work-limit');
|
|
13
15
|
let match;for(let i=0;i+snippet.length<=source.length;i++){signal?.throwIfAborted();if(!snippet.every((line,j)=>source[i+j]===line))continue;if(match!==undefined)return result('rejected','ambiguous-exact-text');match=i;}
|
|
14
16
|
if(match===undefined)return result('rejected','no-exact-text');
|
|
15
17
|
const start=match+1,end=match+snippet.length;
|
|
16
|
-
if(!edits.some(e=>e[a.side].count>0&&start<e[a.side].start+e[a.side].count&&end>=e[a.side].start))return result('rejected','does-not-overlap-referenced-edit');
|
|
17
|
-
const candidate=structuredClone(raw);candidate.anchor={...a,start,end};return {...result('corrected','unique-exact-text-overlaps-edit'),candidate};
|
|
18
|
+
if(!currentSource&&!edits.some(e=>e[a.side].count>0&&start<e[a.side].start+e[a.side].count&&end>=e[a.side].start))return result('rejected','does-not-overlap-referenced-edit');
|
|
19
|
+
const candidate=structuredClone(raw);candidate.anchor={...a,start,end};return {...result('corrected',currentSource?'unique-exact-current-source':'unique-exact-text-overlaps-edit'),candidate};
|
|
18
20
|
}
|
package/src/dsh-model.mjs
CHANGED
|
@@ -1,124 +1,155 @@
|
|
|
1
1
|
import { modelContract } from './model-contract.mjs';
|
|
2
2
|
import { modelJson } from './model-json.mjs';
|
|
3
|
+
import { jsonCorrectionHint, providerDiagnostic } from './model-diagnostics.mjs';
|
|
3
4
|
|
|
4
|
-
//
|
|
5
|
-
const JSON_CORRECTION_SUFFIX = ' 上次响应未能解析为完整JSON。本次是明确协议纠正:重新输出单个完整JSON对象,不使用Markdown,不省略引号或逗号;不要把这条指令当源码证据。严格使用当前阶段schema,不要把verdicts、risks或groups改写成findings
|
|
6
|
-
|
|
7
|
-
const CORRECTION_LIMITATION = '本响应由一次显式JSON协议纠正产生,原响应未被接受;额外模型调用已发生。';
|
|
5
|
+
// Fresh generation from the same approved input, never repair or replay malformed output.
|
|
6
|
+
const JSON_CORRECTION_SUFFIX = ' 上次响应未能解析为完整JSON。本次是明确协议纠正:重新输出单个完整JSON对象,不使用Markdown,不省略引号或逗号;不要把这条指令当源码证据。严格使用当前阶段schema,不要把verdicts、risks、requests或groups改写成findings;不能因格式纠正删除有证据的结论。所需字段与身份按输入原样核对,不以格式错误代替清洁结论。保持原证据与必需字段,精简非必需叙述以在输出预算内完成。';
|
|
7
|
+
const MAX_JSON_CORRECTIONS = 2;
|
|
8
8
|
|
|
9
9
|
const isToolCall = chunk =>
|
|
10
10
|
chunk.type === 'tool-call-delta' ||
|
|
11
11
|
(chunk.type === 'block-start' && chunk.blockType === 'tool-call') ||
|
|
12
12
|
(chunk.type === 'block-end' && chunk.block.type === 'tool-call');
|
|
13
13
|
|
|
14
|
-
/**
|
|
15
|
-
export function createDshModel(llm, route, {
|
|
14
|
+
/** Isolated host calls. Only completed, locally rejected JSON can trigger bounded correction. */
|
|
15
|
+
export function createDshModel(llm, route, {
|
|
16
|
+
maxTokens = 4096, maxOutputBytes = 128 * 1024, correctJson = false,
|
|
17
|
+
maxJsonCorrections = MAX_JSON_CORRECTIONS,
|
|
18
|
+
} = {}) {
|
|
16
19
|
if (!llm || typeof llm.stream !== 'function' || !route?.provider || !route?.model) {
|
|
17
20
|
throw new Error('DSH provider and model are required');
|
|
18
21
|
}
|
|
22
|
+
if (!Number.isSafeInteger(maxTokens) || maxTokens < 1 ||
|
|
23
|
+
!Number.isSafeInteger(maxOutputBytes) || maxOutputBytes < 1 ||
|
|
24
|
+
!Number.isSafeInteger(maxJsonCorrections) || maxJsonCorrections < 0 || maxJsonCorrections > MAX_JSON_CORRECTIONS) {
|
|
25
|
+
throw new Error('Invalid model output or JSON correction limit');
|
|
26
|
+
}
|
|
19
27
|
const identity = {
|
|
20
28
|
provider: route.provider,
|
|
21
29
|
model: route.model,
|
|
22
30
|
...(route.reasoningEffort ? { reasoningEffort: route.reasoningEffort } : {}),
|
|
23
31
|
};
|
|
32
|
+
// Provider failures with a coincidentally identical code are not format failures.
|
|
33
|
+
const invalidJsonErrors = new WeakSet();
|
|
34
|
+
const failureMetadata = new WeakMap();
|
|
24
35
|
|
|
25
|
-
async function once({ instructions, input, signal }) {
|
|
36
|
+
async function once({ instructions, input, signal }, contract) {
|
|
26
37
|
signal?.throwIfAborted();
|
|
27
38
|
let output = '';
|
|
28
39
|
let size = 0;
|
|
29
40
|
let finished = false;
|
|
30
41
|
let finishKind = 'missing';
|
|
31
|
-
|
|
32
42
|
const diagnostic = () => ({
|
|
33
|
-
visibleBytes: Buffer.byteLength(output),
|
|
34
|
-
|
|
35
|
-
finishKind,
|
|
36
|
-
inputBytes: Buffer.byteLength(input),
|
|
37
|
-
requestedMaxTokens: maxTokens,
|
|
38
|
-
});
|
|
39
|
-
|
|
40
|
-
const providerCode = reason =>
|
|
41
|
-
typeof reason.error?.code === 'string' ? reason.error.code
|
|
42
|
-
: typeof reason.code === 'string' ? reason.code
|
|
43
|
-
: undefined;
|
|
44
|
-
|
|
45
|
-
const stream = llm.stream({
|
|
46
|
-
...identity,
|
|
47
|
-
system: instructions + '\n' + modelContract(input, instructions),
|
|
48
|
-
messages: [{ role: 'user', content: [{ type: 'text', text: input }] }],
|
|
49
|
-
tools: [],
|
|
50
|
-
maxTokens,
|
|
51
|
-
signal,
|
|
43
|
+
visibleBytes: Buffer.byteLength(output), totalOutputBytes: size, finishKind,
|
|
44
|
+
inputBytes: Buffer.byteLength(input), requestedMaxTokens: maxTokens, maxOutputBytes,
|
|
52
45
|
});
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
if (
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
46
|
+
const failure = (message, code) => {
|
|
47
|
+
const error = Object.assign(new Error(message), { code, diagnostic: diagnostic() });
|
|
48
|
+
failureMetadata.set(error, { code, diagnostic: error.diagnostic });
|
|
49
|
+
return error;
|
|
50
|
+
};
|
|
51
|
+
try {
|
|
52
|
+
const stream = llm.stream({
|
|
53
|
+
...identity,
|
|
54
|
+
system: instructions + '\n' + contract,
|
|
55
|
+
messages: [{ role: 'user', content: [{ type: 'text', text: input }] }],
|
|
56
|
+
tools: [], maxTokens, signal,
|
|
57
|
+
});
|
|
58
|
+
for await (const chunk of stream) {
|
|
59
|
+
signal?.throwIfAborted();
|
|
60
|
+
if (finished) throw failure('Model emitted data after finish', 'MODEL_PROTOCOL_ERROR');
|
|
61
|
+
if (isToolCall(chunk)) throw failure('Review model attempted a tool call', 'MODEL_PROTOCOL_ERROR');
|
|
62
|
+
if (chunk.type === 'text-delta' || chunk.type === 'reasoning-delta') {
|
|
63
|
+
size += Buffer.byteLength(chunk.text);
|
|
64
|
+
if (size > maxOutputBytes) throw failure('Model output limit exceeded', 'MODEL_OUTPUT_LIMIT');
|
|
65
|
+
if (chunk.type === 'text-delta') output += chunk.text;
|
|
66
|
+
}
|
|
67
|
+
if (chunk.type === 'finish') {
|
|
68
|
+
finishKind = chunk.reason.kind;
|
|
69
|
+
if (finishKind !== 'stop') {
|
|
70
|
+
const error = failure('Model did not complete: ' + finishKind, 'MODEL_INCOMPLETE_OUTPUT');
|
|
71
|
+
Object.assign(error.diagnostic, providerDiagnostic(chunk.reason));
|
|
72
|
+
throw error;
|
|
73
|
+
}
|
|
74
|
+
finished = true;
|
|
72
75
|
}
|
|
73
|
-
finished = true;
|
|
74
76
|
}
|
|
75
|
-
}
|
|
76
|
-
|
|
77
|
-
if (!finished) {
|
|
78
|
-
throw Object.assign(new Error('Model stream ended without finish'), {
|
|
79
|
-
code: 'MODEL_INCOMPLETE_OUTPUT',
|
|
80
|
-
diagnostic: diagnostic(),
|
|
81
|
-
});
|
|
82
|
-
}
|
|
83
|
-
|
|
84
|
-
try {
|
|
85
|
-
return modelJson(output);
|
|
86
77
|
} catch (error) {
|
|
87
|
-
|
|
78
|
+
// Keep thrown provider/cancellation identity, including frozen and primitive values.
|
|
79
|
+
// Only add safe structured metadata when the error permits it.
|
|
80
|
+
if (error && (typeof error === 'object' || typeof error === 'function') && !failureMetadata.has(error)) {
|
|
81
|
+
const safeDiagnostic = { ...diagnostic(), ...providerDiagnostic(error) };
|
|
82
|
+
failureMetadata.set(error, { code: safeDiagnostic.providerCode, diagnostic: safeDiagnostic });
|
|
83
|
+
try {
|
|
84
|
+
if (Object.isExtensible(error) && !Object.hasOwn(error, 'diagnostic')) {
|
|
85
|
+
Object.defineProperty(error, 'diagnostic', { value: structuredClone(safeDiagnostic), configurable: true, enumerable: true, writable: true });
|
|
86
|
+
}
|
|
87
|
+
} catch { /* Diagnostic accessors/proxies must not replace the primary failure. */ }
|
|
88
|
+
}
|
|
88
89
|
throw error;
|
|
89
90
|
}
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
await request.onProtocolCorrection?.({
|
|
94
|
-
code: firstError.code,
|
|
95
|
-
diagnostic: firstError.diagnostic,
|
|
96
|
-
action: 'explicit-json-correction',
|
|
97
|
-
notice: 'Second model response requested; original output not repaired or accepted.',
|
|
98
|
-
});
|
|
99
|
-
let value;
|
|
91
|
+
// Cancellation during iterator cleanup/end must not become an invalid-JSON correction.
|
|
92
|
+
signal?.throwIfAborted();
|
|
93
|
+
if (!finished) throw failure('Model stream ended without finish', 'MODEL_INCOMPLETE_OUTPUT');
|
|
100
94
|
try {
|
|
101
|
-
|
|
102
|
-
} catch (
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
throw second;
|
|
108
|
-
}
|
|
109
|
-
if (value && typeof value === 'object' && !Array.isArray(value)) {
|
|
110
|
-
if (Array.isArray(value.limitations)) value.limitations.push(CORRECTION_LIMITATION);
|
|
95
|
+
return { text: modelJson(output), diagnostic: diagnostic() };
|
|
96
|
+
} catch (error) {
|
|
97
|
+
error.diagnostic = { ...diagnostic(), ...error.diagnostic };
|
|
98
|
+
invalidJsonErrors.add(error);
|
|
99
|
+
failureMetadata.set(error, { code: error.code, diagnostic: error.diagnostic });
|
|
100
|
+
throw error;
|
|
111
101
|
}
|
|
112
|
-
return JSON.stringify(value);
|
|
113
102
|
}
|
|
114
103
|
|
|
115
104
|
return async function invoke(request) {
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
105
|
+
// Snapshot these before an async budget/progress hook can mutate the request.
|
|
106
|
+
const original = { instructions: request.instructions, input: request.input, signal: request.signal };
|
|
107
|
+
const contract = modelContract(original.input, original.instructions);
|
|
108
|
+
const correctionLimit = correctJson ? maxJsonCorrections : 0;
|
|
109
|
+
const attempts = [];
|
|
110
|
+
for (let attempt = 1; ; attempt++) {
|
|
111
|
+
let result;
|
|
112
|
+
try {
|
|
113
|
+
result = await once({ ...original, instructions: original.instructions +
|
|
114
|
+
(attempt > 1 ? JSON_CORRECTION_SUFFIX + jsonCorrectionHint(attempts.at(-1)?.diagnostic) + ' 纠正次数:' + (attempt - 1) + '/' + correctionLimit + '。' : '') }, contract);
|
|
115
|
+
} catch (error) {
|
|
116
|
+
const metadata = failureMetadata.get(error) ?? {};
|
|
117
|
+
failureMetadata.delete(error);
|
|
118
|
+
attempts.push({ attempt, ...structuredClone(metadata) });
|
|
119
|
+
const invalidJson = invalidJsonErrors.delete(error);
|
|
120
|
+
if (!invalidJson || attempt > correctionLimit) {
|
|
121
|
+
try {
|
|
122
|
+
if (attempts.length > 1 && error && typeof error === 'object' && Object.isExtensible(error)) {
|
|
123
|
+
Object.defineProperty(error, 'attempts', { value: structuredClone(attempts), configurable: true, enumerable: true, writable: true });
|
|
124
|
+
if (invalidJson) {
|
|
125
|
+
error.message = 'Model response is not complete JSON after ' + (attempt - 1) +
|
|
126
|
+
' explicit JSON correction(s) (' + attempt + ' model calls); correction budget exhausted; no malformed output accepted';
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
} catch { /* Preserve provider identity even with hostile accessors/proxies. */ }
|
|
130
|
+
throw error;
|
|
131
|
+
}
|
|
132
|
+
original.signal?.throwIfAborted();
|
|
133
|
+
// Exactly one reservation per extra call; a thrown budget/cancellation veto is final.
|
|
134
|
+
await request.onProtocolCorrection?.({
|
|
135
|
+
...structuredClone(metadata), attempts: structuredClone(attempts),
|
|
136
|
+
action: 'explicit-json-correction', attempt: attempt + 1,
|
|
137
|
+
correction: attempt, maxJsonCorrections: correctionLimit,
|
|
138
|
+
notice: 'Additional model response requested; original output not repaired or accepted.',
|
|
139
|
+
});
|
|
140
|
+
original.signal?.throwIfAborted();
|
|
141
|
+
continue;
|
|
142
|
+
}
|
|
143
|
+
if (attempt > 1) {
|
|
144
|
+
// Out-of-band audit only: never inject metadata into stage-specific model schemas.
|
|
145
|
+
await request.onProtocolEvent?.({
|
|
146
|
+
action: 'json-correction-complete', attempt, corrections: attempt - 1,
|
|
147
|
+
attempts: structuredClone([...attempts, { attempt, diagnostic: result.diagnostic }]),
|
|
148
|
+
notice: 'Fresh complete JSON accepted; malformed outputs discarded; extra model calls occurred.',
|
|
149
|
+
});
|
|
150
|
+
original.signal?.throwIfAborted();
|
|
151
|
+
}
|
|
152
|
+
return result.text;
|
|
122
153
|
}
|
|
123
154
|
};
|
|
124
155
|
}
|
package/src/model-contract.mjs
CHANGED
|
@@ -3,12 +3,9 @@ const VERDICT_CONTRACT = '本阶段必须返回 {"verdicts":[{"candidateId":"c1"
|
|
|
3
3
|
const GROUPS_CONTRACT = '本阶段返回groups数组,每项含fileIds和reason,文件身份必须来自输入,不是findings响应。';
|
|
4
4
|
const RISK_CONTRACT = '本阶段返回 {"risks":[]} 或原指令要求的风险条目,不是findings。';
|
|
5
5
|
const INTERACTION_CONTRACT = '本阶段返回 {"findings":[],"limitations":[]};发现必须包含原指令全部字段和跨文件interaction.editRefs。无足够跨文件证据时说明limitations,不编造。';
|
|
6
|
-
const FINDINGS_CONTRACT = '本阶段最终返回 {"findings":[],"limitations":[]}。每项finding必须含fileId、severity(critical/high/medium/low)、title、evidence、trigger、impact、suggestion和anchor;fileId原样复制file.id
|
|
6
|
+
const FINDINGS_CONTRACT = '本阶段最终返回 {"findings":[],"limitations":[]}。每项finding必须含fileId、severity(critical/high/medium/low)、title、evidence、trigger、impact、suggestion和anchor;fileId原样复制file.id,不用文件名替代。行定位写成 anchor:{kind:"line",side:"new",start:1,end:1,snippet:"逐行原文"}(仅为形状示例,不是证据);start/end用原来源一基行号而非窗口内序号,精确匹配原行;old/new按当前阶段允许侧选择,示例不改变授权;仅当没有retrievalFinalization且仍有检索余量时,需要更多证据可按检索协议先返回requests。';
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
let p;
|
|
10
|
-
try { p = JSON.parse(input); } catch { return ''; }
|
|
11
|
-
if (!p || typeof p !== 'object' || Array.isArray(p)) return '';
|
|
8
|
+
function stageContract(p, instructions) {
|
|
12
9
|
if (Array.isArray(p.candidates)) return VERDICT_CONTRACT;
|
|
13
10
|
if (/返回\{groups:/.test(instructions)) return GROUPS_CONTRACT;
|
|
14
11
|
if (/仅制定审查计划/.test(instructions)) return RISK_CONTRACT;
|
|
@@ -16,3 +13,11 @@ export function modelContract(input, instructions = '') {
|
|
|
16
13
|
if (p.file) return FINDINGS_CONTRACT;
|
|
17
14
|
return '';
|
|
18
15
|
}
|
|
16
|
+
|
|
17
|
+
export function modelContract(input, instructions = '') {
|
|
18
|
+
let p;
|
|
19
|
+
try { p = JSON.parse(input); } catch { return ''; }
|
|
20
|
+
if (!p || typeof p !== 'object' || Array.isArray(p)) return '';
|
|
21
|
+
const contract = stageContract(p, instructions);
|
|
22
|
+
return contract + (p.retrievalFinalization ? ' 本次检索已受限:只返回当前阶段最终schema,禁止requests;不能将缺证据声称为无问题或完整覆盖。' : '');
|
|
23
|
+
}
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
|
|
3
|
+
// Never persist provider messages, response excerpts, headers, URLs or raw request IDs.
|
|
4
|
+
const digest = text => createHash('sha256').update(text).digest('hex');
|
|
5
|
+
const own = (value, key) => {
|
|
6
|
+
try { const d = value != null ? Object.getOwnPropertyDescriptor(value, key) : undefined; return d && 'value' in d ? d.value : undefined; }
|
|
7
|
+
catch { return undefined; }
|
|
8
|
+
};
|
|
9
|
+
const PROVIDER_CODES = new Set(['RATE_LIMIT','CONTEXT_WINDOW_EXCEEDED','QUOTA','ACCOUNT_QUOTA','AUTH','AUTHENTICATION','INVALID_CREDENTIAL','MISSING_CREDENTIAL','PERMISSION','TIMEOUT','NETWORK','SERVER_ERROR','SERVER','SERVICE_UNAVAILABLE','INVALID_ARGS','INVALID_REQUEST','BAD_REQUEST','EMPTY_RESPONSE','UNKNOWN','NO_ADAPTER','ABORTED','CANCELLED']);
|
|
10
|
+
export function providerDiagnostic(value) {
|
|
11
|
+
const carried = own(value, 'failure');
|
|
12
|
+
const code = own(carried, 'code') ?? own(value, 'code');
|
|
13
|
+
const status = own(carried, 'status') ?? own(value, 'status');
|
|
14
|
+
const retry = own(carried, 'providerRetryAfterMs') ?? own(value, 'providerRetryAfterMs');
|
|
15
|
+
const requestId = own(carried, 'requestId') ?? own(value, 'requestId');
|
|
16
|
+
return {
|
|
17
|
+
...(typeof code === 'string' && code.length > 0 && code.length <= 256 ? { providerCode: PROVIDER_CODES.has(code) ? code : 'UNRECOGNIZED', ...(PROVIDER_CODES.has(code) ? {} : { providerCodeHash: digest(code) }) } : {}),
|
|
18
|
+
...(Number.isInteger(status) && status >= 100 && status <= 599 ? { providerStatus: status } : {}),
|
|
19
|
+
...(Number.isFinite(retry) && retry > 0 && retry <= 86400000 ? { providerRetryAfterMs: retry } : {}),
|
|
20
|
+
...(typeof requestId === 'string' && requestId.length > 0 && requestId.length <= 512 ? { providerRequestIdHash: digest(requestId) } : {}),
|
|
21
|
+
};
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
const SYNTAX_HINTS = {
|
|
25
|
+
'property-name': '对象属性名必须使用双引号,不允许单引号、裸键或尾逗号。',
|
|
26
|
+
colon: '属性名与值之间必须有冒号。',
|
|
27
|
+
separator: '相邻属性或数组元素之间必须有逗号,结束处不可多加逗号。',
|
|
28
|
+
'invalid-escape': 'JSON 字符串中的反斜杠必须合法转义;换行、引号和路径须按 JSON 编码。',
|
|
29
|
+
'control-character': 'JSON 字符串中不能包含裸换行或控制字符,必须正确转义。',
|
|
30
|
+
'unterminated-string': '字符串必须闭合;不要用多行裸文本或截断字符串。',
|
|
31
|
+
'unexpected-end': '只输出一个语法完整且闭合的对象,不省略结尾。',
|
|
32
|
+
'trailing-content': 'JSON 对象结束后不得附加解释、第二个对象或 Markdown。',
|
|
33
|
+
syntax: '逐项检查双引号、转义、冒号、逗号和括号,输出完整严格 JSON。',
|
|
34
|
+
};
|
|
35
|
+
export function jsonSyntaxDiagnostic(body, error) {
|
|
36
|
+
const message = error instanceof SyntaxError ? error.message : '';
|
|
37
|
+
let category = 'syntax';
|
|
38
|
+
if (/^Expected (?:property name|double-quoted property name)/.test(message)) category = 'property-name';
|
|
39
|
+
else if (/^Expected ':'/.test(message)) category = 'colon';
|
|
40
|
+
else if (/^Expected ','/.test(message)) category = 'separator';
|
|
41
|
+
else if (/^Bad (?:escaped character|Unicode escape)/.test(message)) category = 'invalid-escape';
|
|
42
|
+
else if (/^Bad control character/.test(message)) category = 'control-character';
|
|
43
|
+
else if (/^Unterminated string/.test(message)) category = 'unterminated-string';
|
|
44
|
+
else if (/^Unexpected end of JSON input/.test(message)) category = 'unexpected-end';
|
|
45
|
+
else if (/^Unexpected non-whitespace character after JSON/.test(message)) category = 'trailing-content';
|
|
46
|
+
const match = / at position ([0-9]+)(?: \(line [0-9]+ column [0-9]+\))?$/.exec(message);
|
|
47
|
+
const offset = match ? Number(match[1]) : category === 'unexpected-end' ? body.length : undefined;
|
|
48
|
+
let location = {};
|
|
49
|
+
if (Number.isSafeInteger(offset) && offset >= 0 && offset <= body.length) {
|
|
50
|
+
const before = body.slice(0, offset);
|
|
51
|
+
const lines = before.split(/\r\n|\r|\n/);
|
|
52
|
+
location = { offset, line: lines.length, column: lines.at(-1).length + 1 };
|
|
53
|
+
}
|
|
54
|
+
return { syntaxCategory: category, ...location, offsetUnit: 'utf16-code-unit-in-json-body', responseHash: digest(body) };
|
|
55
|
+
}
|
|
56
|
+
export function jsonCorrectionHint(diagnostic) {
|
|
57
|
+
const category = diagnostic?.syntaxCategory;
|
|
58
|
+
if (typeof category !== 'string' || !Object.hasOwn(SYNTAX_HINTS, category)) return '';
|
|
59
|
+
const { line, column } = diagnostic;
|
|
60
|
+
const location = Number.isSafeInteger(line) && line > 0 && Number.isSafeInteger(column) && column > 0 ? ',JSON正文第' + line + '行第' + column + '列' : '';
|
|
61
|
+
return ' 本地解析诊断(不含原响应):' + category + location + '。' + SYNTAX_HINTS[category];
|
|
62
|
+
}
|
package/src/model-json.mjs
CHANGED
|
@@ -1,13 +1,21 @@
|
|
|
1
|
+
import { jsonSyntaxDiagnostic } from './model-diagnostics.mjs';
|
|
2
|
+
|
|
1
3
|
// Accept one complete JSON value, optionally inside one exact Markdown fence.
|
|
4
|
+
// This parser never repairs bytes or invokes a model; orchestration owns corrections.
|
|
2
5
|
export function modelJson(text) {
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
6
|
+
const bytes = Buffer.byteLength(text);
|
|
7
|
+
const trimmed = text.trim();
|
|
8
|
+
const fence = /^```(?:json)?\r?\n([\s\S]*)\r?\n```$/.exec(trimmed);
|
|
9
|
+
const body = fence ? fence[1] : trimmed;
|
|
10
|
+
try {
|
|
11
|
+
JSON.parse(body);
|
|
12
|
+
} catch (parseError) {
|
|
13
|
+
const envelope = fence ? 'json-fence' : 'plain';
|
|
14
|
+
const shape = !trimmed ? 'empty' : trimmed.startsWith('```') && !fence ? 'non-exact-fence' : /^[{\[]/.test(body) ? 'json-like-invalid' : 'non-json-leading';
|
|
15
|
+
const error = new Error('Model response is not complete JSON (bytes=' + bytes + ', envelope=' + envelope + ', shape=' + shape + '); no repair or retry performed by JSON parser');
|
|
16
|
+
error.code = 'MODEL_INVALID_JSON';
|
|
17
|
+
error.diagnostic = { jsonBytes: bytes, envelope, shape, ...jsonSyntaxDiagnostic(body, parseError) };
|
|
18
|
+
throw error;
|
|
19
|
+
}
|
|
20
|
+
return body;
|
|
13
21
|
}
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
import { reviewSnapshot } from './review.mjs';
|
|
2
|
+
|
|
3
|
+
export const DEFAULT_PROJECT_CONCURRENCY = 3;
|
|
4
|
+
export function projectConcurrency(value = DEFAULT_PROJECT_CONCURRENCY) {
|
|
5
|
+
if (!Number.isSafeInteger(value) || value < 1 || value > 8) throw new Error('Project concurrency must be an integer from 1 to 8');
|
|
6
|
+
return value;
|
|
7
|
+
}
|
|
8
|
+
|
|
9
|
+
// Parallelize independent approved snapshots, never dependent stages within a batch.
|
|
10
|
+
// Admission and the shared call reservation are synchronous; completion order is not report order.
|
|
11
|
+
export async function reviewProjectBatches(batches, model, { concurrency = DEFAULT_PROJECT_CONCURRENCY, remainingCalls, signal, onProgress } = {}) {
|
|
12
|
+
projectConcurrency(concurrency);
|
|
13
|
+
if (!Array.isArray(batches) || typeof remainingCalls !== 'function') throw new Error('Invalid project execution');
|
|
14
|
+
const controller = new AbortController();
|
|
15
|
+
const batchSignal = signal ? AbortSignal.any([signal, controller.signal]) : controller.signal;
|
|
16
|
+
const results = new Array(batches.length);
|
|
17
|
+
let next = 0, failed = false, failure;
|
|
18
|
+
const unstarted = batch => ({ snapshotId: batch.id, status: 'partial', files: batch.files.map(f => ({ fileId: f.id, path: f.path, status: f.eligibility === 'reviewable' ? 'pending' : f.eligibility, reason: 'Session project budget exhausted or cancelled' })), findings: [], limitations: [] });
|
|
19
|
+
const worker = async () => {
|
|
20
|
+
while (!failed) {
|
|
21
|
+
const index = next++;
|
|
22
|
+
if (index >= batches.length) return;
|
|
23
|
+
const batch = batches[index];
|
|
24
|
+
try {
|
|
25
|
+
if (batchSignal.aborted || remainingCalls() <= 0) { results[index] = unstarted(batch); continue; }
|
|
26
|
+
results[index] = await reviewSnapshot(batch, model, {
|
|
27
|
+
signal: batchSignal, maxCalls: remainingCalls(), remainingCalls, enableRetrieval: true, enableVerification: true,
|
|
28
|
+
enableAnchorCorrection: true, correctProtocol: true,
|
|
29
|
+
onProgress: onProgress ? event => onProgress({ ...event, snapshotId: batch.id, batchIndex: index }) : undefined,
|
|
30
|
+
});
|
|
31
|
+
} catch (error) {
|
|
32
|
+
if (!failed) { failed = true; failure = error; controller.abort(error); }
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
};
|
|
36
|
+
// Never release service ownership while a sibling model/cleanup is still running.
|
|
37
|
+
await Promise.all(Array.from({ length: Math.min(concurrency, batches.length) }, worker));
|
|
38
|
+
if (failed) throw failure;
|
|
39
|
+
return results;
|
|
40
|
+
}
|
|
@@ -5,10 +5,10 @@ import { hash } from './content.mjs';
|
|
|
5
5
|
import { resolveWorkspaceRepository } from './workspace-repository.mjs';
|
|
6
6
|
import { detectVcs } from './snapshot.mjs';
|
|
7
7
|
import { projectInventory, captureProjectBatch } from './project-snapshot.mjs';
|
|
8
|
-
import {
|
|
8
|
+
import { reviewProjectBatches, projectConcurrency as validateProjectConcurrency } from './project-execution.mjs';
|
|
9
9
|
import { createDshModel } from './dsh-model.mjs';
|
|
10
10
|
|
|
11
|
-
const PREVIEW_NOTICE_SUFFIX = ' Full current source audit, not regression-only diff. Shared session execution budget: no internal elapsed-time or model-call cutoff, 16MiB capture memory protection. Blocked/excluded/pending files are not reviewed successfully. No untracked/ignored/external bodies; no complete semantic call graph. Show scope and model before confirmed execute.';
|
|
11
|
+
const PREVIEW_NOTICE_SUFFIX = ' Full current source audit, not regression-only diff. Shared session execution budget: no internal elapsed-time or model-call cutoff, 16MiB capture memory protection. Blocked/excluded/pending files are not reviewed successfully. No untracked/ignored/external bodies; no complete semantic call graph. Independent approved batches run with bounded concurrency; per-batch evidence and global model-call budgets remain enforced. Show scope and model before confirmed execute.';
|
|
12
12
|
const REPORT_NOTICE_SUFFIX = ' Coverage counts processing of selected tracked files only, not proof of correctness. Excluded and blocked are never completed. No cross-batch whole-program semantic coverage is claimed.';
|
|
13
13
|
|
|
14
14
|
function owner(exec) {
|
|
@@ -66,10 +66,11 @@ async function captureProject(cwd, args, signal) {
|
|
|
66
66
|
return { ...result, id: hash(JSON.stringify(result)) };
|
|
67
67
|
}
|
|
68
68
|
|
|
69
|
-
export function createProjectReviewService(llm, { allowModelSending = false, authorize = async () => false, now = Date.now, ttlMs = 300000, maxPreviews = 8, projectMaxCalls = Number.MAX_SAFE_INTEGER } = {}) {
|
|
69
|
+
export function createProjectReviewService(llm, { allowModelSending = false, authorize = async () => false, now = Date.now, ttlMs = 300000, maxPreviews = 8, projectMaxCalls = Number.MAX_SAFE_INTEGER, projectConcurrency = 3 } = {}) {
|
|
70
70
|
for (const n of [ttlMs, maxPreviews]) if (!Number.isSafeInteger(n) || n < 1) throw new Error('Invalid project preview limit');
|
|
71
71
|
if (!Number.isSafeInteger(projectMaxCalls) || projectMaxCalls < 1) throw new Error('Invalid project model call limit');
|
|
72
72
|
|
|
73
|
+
validateProjectConcurrency(projectConcurrency);
|
|
73
74
|
const previews = new Map();
|
|
74
75
|
const lifetime = new AbortController();
|
|
75
76
|
let busy = false;
|
|
@@ -96,6 +97,7 @@ export function createProjectReviewService(llm, { allowModelSending = false, aut
|
|
|
96
97
|
model: current.route, modelSendingEnabled: allowModelSending,
|
|
97
98
|
totalTrackedNodes: s.totalTrackedNodes, selectedFiles: s.selectedFiles, selectionComplete: s.selectionComplete,
|
|
98
99
|
inventoryComplete: s.inventoryComplete, batches: s.batches.length,
|
|
100
|
+
execution: { batchConcurrency: projectConcurrency },
|
|
99
101
|
files: s.batches.flatMap(b => b.files.map(f => ({ path: f.path, eligibility: f.eligibility, reason: f.reason }))),
|
|
100
102
|
notice: s.notice + PREVIEW_NOTICE_SUFFIX,
|
|
101
103
|
};
|
|
@@ -127,23 +129,19 @@ export function createProjectReviewService(llm, { allowModelSending = false, aut
|
|
|
127
129
|
let calls = 0;
|
|
128
130
|
const adapter = createDshModel(llm, p.route, { correctJson: true });
|
|
129
131
|
const model = request => {
|
|
130
|
-
if (calls >= projectMaxCalls) throw new Error('Shared project model call limit exceeded');
|
|
132
|
+
if (calls >= projectMaxCalls) throw Object.assign(new Error('Shared project model call limit exceeded'), { code: 'PROJECT_MODEL_CALL_LIMIT' });
|
|
131
133
|
calls++;
|
|
132
134
|
return adapter({ ...request, onProtocolCorrection: async info => {
|
|
133
|
-
if (calls >= projectMaxCalls) throw new Error('Shared project model call limit exceeded');
|
|
135
|
+
if (calls >= projectMaxCalls) throw Object.assign(new Error('Shared project model call limit exceeded'), { code: 'PROJECT_MODEL_CALL_LIMIT' });
|
|
134
136
|
calls++;
|
|
135
137
|
await request.onProtocolCorrection?.(info);
|
|
136
138
|
}});
|
|
137
139
|
};
|
|
138
140
|
|
|
139
|
-
const results =
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
continue;
|
|
144
|
-
}
|
|
145
|
-
results.push(await reviewSnapshot(b, model, { signal, maxCalls: projectMaxCalls - calls, enableRetrieval: true, enableVerification: true, correctProtocol: true, onProgress: exec.reviewCheckpoint }));
|
|
146
|
-
}
|
|
141
|
+
const results = await reviewProjectBatches(s.batches, model, {
|
|
142
|
+
concurrency: projectConcurrency, signal, remainingCalls: () => projectMaxCalls - calls,
|
|
143
|
+
onProgress: exec.reviewCheckpoint,
|
|
144
|
+
});
|
|
147
145
|
|
|
148
146
|
const files = results.flatMap(r => r.files.map(f => ({ ...f, batchSnapshotId: r.snapshotId })));
|
|
149
147
|
const coverage = {
|
|
@@ -164,6 +162,7 @@ export function createProjectReviewService(llm, { allowModelSending = false, aut
|
|
|
164
162
|
repositoryRoot: s.root, vcs: s.vcs, model: p.route, modelCalls: calls, maxModelCalls: projectMaxCalls,
|
|
165
163
|
totalTrackedNodes: s.totalTrackedNodes, selectionComplete: s.selectionComplete, inventoryComplete: s.inventoryComplete,
|
|
166
164
|
coverage, files, findings, batches: results,
|
|
165
|
+
execution: { batchConcurrency: projectConcurrency },
|
|
167
166
|
notice: s.notice + REPORT_NOTICE_SUFFIX,
|
|
168
167
|
};
|
|
169
168
|
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
// Ask for one new response; never normalize identities, severities or evidence locally.
|
|
2
|
+
const GUIDANCE = {
|
|
3
|
+
MODEL_INVALID_REVIEW_RESPONSE: '返回含 findings 和 limitations 数组的当前阶段完整对象。',
|
|
4
|
+
MODEL_INVALID_FINDING_IDENTITY: 'fileId 必须原样复制本次主 file.id,不使用路径、候选编号或其他文件身份。',
|
|
5
|
+
MODEL_INVALID_FINDING_SEVERITY: 'severity 仅允许 critical、high、medium、low,不使用 P1/P2、数字或翻译标签。',
|
|
6
|
+
MODEL_MISSING_FINDING_FIELD: '每个发现必须包含非空 title/evidence/trigger/impact/suggestion。',
|
|
7
|
+
MODEL_INVALID_ANCHOR: 'anchor 必须使用当前阶段允许的 kind、side、整数 start/end 及精确 snippet。',
|
|
8
|
+
MODEL_ANCHOR_MISMATCH: 'snippet 必须逐行精确复制批准来源,start/end 是该来源的原始一基行号,不能使用展示窗口的相对行号。不得模糊匹配、改写源码或用 file anchor 隐藏定位失败。',
|
|
9
|
+
};
|
|
10
|
+
|
|
11
|
+
export async function correctedResponse(generate, validate, request, { enabled = false, canCorrect = () => true, onCorrection = async () => {} } = {}) {
|
|
12
|
+
const approved = { ...request };
|
|
13
|
+
const original = await generate({ ...approved });
|
|
14
|
+
try { return validate(original); } catch (error) {
|
|
15
|
+
approved.signal?.throwIfAborted();
|
|
16
|
+
if (!enabled || !Object.hasOwn(GUIDANCE, error?.code) || !canCorrect()) throw error;
|
|
17
|
+
await onCorrection({ action: 'review-response-correction', code: error.code, notice: 'Original response rejected; requesting one new response under the same approved scope and shared budgets.' });
|
|
18
|
+
approved.signal?.throwIfAborted();
|
|
19
|
+
const instructions = approved.instructions + '\n本次是一次明确的主审响应协议纠正:' + GUIDANCE[error.code] + ' 重新核对原批准源码并返回完整当前阶段 schema;不得把格式错误当成无缺陷,不得为通过校验编造证据或静默删除有证据的候选。原响应未被接受,纠正后仍需全部严格校验。';
|
|
20
|
+
// generate enforces the same input/call/reader limits and enclosing signal.
|
|
21
|
+
const response = await generate({ ...approved, instructions });
|
|
22
|
+
approved.signal?.throwIfAborted();
|
|
23
|
+
try { return validate(response); } catch (second) {
|
|
24
|
+
second.responseAttempts = [{ code: error.code }, { code: second.code ?? 'REVIEW_EXECUTION_FAILED' }];
|
|
25
|
+
throw second;
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
}
|
package/src/retrieval-loop.mjs
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { RETRIEVAL_INSTRUCTIONS } from './prompts.mjs';
|
|
2
|
-
import { executeRetrievalBatch } from './retrieval-protocol.mjs';
|
|
2
|
+
import { executeRetrievalBatch, parseRetrievalResponse } from './retrieval-protocol.mjs';
|
|
3
3
|
|
|
4
4
|
export { RETRIEVAL_INSTRUCTIONS };
|
|
5
5
|
|
|
@@ -18,7 +18,7 @@ function compactFollowup(input) {
|
|
|
18
18
|
}
|
|
19
19
|
|
|
20
20
|
export function retrievalRequest(request, scope, retrieved = []) {
|
|
21
|
-
return { ...request, instructions: request.instructions + '\n' + RETRIEVAL_INSTRUCTIONS, input: JSON.stringify({ ...JSON.parse(request.input), catalog: scope.catalog(), retrieved }) };
|
|
21
|
+
return { ...request, instructions: request.instructions + '\n' + RETRIEVAL_INSTRUCTIONS + BUDGET_SUFFIX, input: JSON.stringify({ ...JSON.parse(request.input), catalog: scope.catalog(), retrieved, ...(scope.budget ? { retrievalBudget: scope.budget() } : {}) }) };
|
|
22
22
|
}
|
|
23
23
|
|
|
24
24
|
function recordRetrieval(operation, result, onTruncated, onRetrieved) {
|
|
@@ -34,50 +34,95 @@ function recordRetrieval(operation, result, onTruncated, onRetrieved) {
|
|
|
34
34
|
|
|
35
35
|
const CORRECTION_SUFFIX = ' 本次是检索协议纠正:上次请求整批未执行。只返回requests数组,不带findings/limitations;read使用catalog的id和整数start/count,不用path/end。或直接返回当前阶段最终schema。不得扩大批准来源。';
|
|
36
36
|
|
|
37
|
-
|
|
37
|
+
const BUDGET_SUFFIX = ' retrievalBudget为宿主共享检索预算的当前余量,不是新授权。每项操作消耗一次call及实际序列化输出字节;不要请求超过余量的整批操作。预算将尽请及时按本阶段schema最终答复,只引用已提供证据,证据不足须保留限制或uncertain。';
|
|
38
|
+
const FINAL_SUFFIX = ' 检索预算最终答复:本次仅有一次最终schema答复机会,不得再返回requests。不得把缺证据变成无问题/已核查;只保留已提供且可严格验证的证据,其余说明限制或uncertain。不增加reader、模型调用、输入或超时预算。';
|
|
39
|
+
const budgetFailure = error => ['READER_CALL_BUDGET_EXCEEDED', 'READER_OUTPUT_BUDGET_EXCEEDED', 'RETRIEVAL_ROUND_BUDGET_EXCEEDED'].includes(error?.code);
|
|
40
|
+
|
|
41
|
+
function roundBudgetError() {
|
|
42
|
+
const error = new Error('Retrieval round budget exceeded');
|
|
43
|
+
error.code = 'RETRIEVAL_ROUND_BUDGET_EXCEEDED';
|
|
44
|
+
return error;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
export async function retrievalLoop(model, request, scope, { correctProtocol = false, maxRounds = Number.MAX_SAFE_INTEGER, maxInputBytes = 96 * 1024, beforeCall = () => {}, onTruncated = () => {}, onRetrieved = () => {}, onBudgetLimited = () => {} } = {}) {
|
|
38
48
|
if (!Number.isSafeInteger(maxRounds) || maxRounds < 0 || !Number.isSafeInteger(maxInputBytes) || maxInputBytes < 1) throw new Error('Invalid retrieval loop budget');
|
|
39
49
|
const retrieved = [];
|
|
40
|
-
let protocolCorrection;
|
|
50
|
+
let protocolCorrection, finalization;
|
|
51
|
+
const finalize = async error => {
|
|
52
|
+
finalization = { code: error.code, reason: error.message, ...(scope.budget ? { budget: scope.budget() } : {}) };
|
|
53
|
+
await onBudgetLimited(finalization);
|
|
54
|
+
await request.onProtocolEvent?.({ action: 'retrieval-budget-finalization', ...finalization });
|
|
55
|
+
};
|
|
56
|
+
const checkExhausted = async round => {
|
|
57
|
+
const budget = scope.budget?.();
|
|
58
|
+
if (!finalization && budget && (budget.remainingCalls === 0 || budget.remainingOutputBytes === 0)) {
|
|
59
|
+
const error = new Error(budget.remainingCalls === 0 ? 'Reader call budget exceeded' : 'Reader output budget exceeded');
|
|
60
|
+
error.code = budget.remainingCalls === 0 ? 'READER_CALL_BUDGET_EXCEEDED' : 'READER_OUTPUT_BUDGET_EXCEEDED';
|
|
61
|
+
await finalize(error);
|
|
62
|
+
} else if (!finalization && round >= maxRounds) await finalize(roundBudgetError());
|
|
63
|
+
};
|
|
64
|
+
const nextRequest = records => {
|
|
65
|
+
const next = retrievalRequest(request, scope, records);
|
|
66
|
+
if (finalization) next.input = JSON.stringify({ ...JSON.parse(next.input), retrievalFinalization: finalization });
|
|
67
|
+
next.instructions += (protocolCorrection ? CORRECTION_SUFFIX : '') + (finalization ? FINAL_SUFFIX : '');
|
|
68
|
+
return next;
|
|
69
|
+
};
|
|
41
70
|
for (let round = 0; ; round++) {
|
|
42
71
|
request.signal?.throwIfAborted();
|
|
43
|
-
|
|
44
|
-
|
|
72
|
+
await checkExhausted(round);
|
|
73
|
+
let { input, instructions } = nextRequest(retrieved);
|
|
45
74
|
if (correctProtocol && retrieved.length && Buffer.byteLength(input) + Buffer.byteLength(instructions) > maxInputBytes) input = compactFollowup(input);
|
|
46
75
|
if (Buffer.byteLength(input) + Buffer.byteLength(instructions) > maxInputBytes) throw new Error('Retrieval input budget exceeded');
|
|
76
|
+
request.signal?.throwIfAborted();
|
|
47
77
|
beforeCall();
|
|
78
|
+
request.signal?.throwIfAborted();
|
|
48
79
|
const response = await model({ ...request, instructions, input });
|
|
49
80
|
request.signal?.throwIfAborted();
|
|
50
81
|
const serialized = typeof response === 'string' ? response : JSON.stringify(response);
|
|
51
82
|
if (typeof serialized !== 'string' || Buffer.byteLength(serialized) > 128 * 1024) throw new Error('Retrieval response too large');
|
|
52
83
|
const parsed = JSON.parse(serialized);
|
|
53
84
|
if (!parsed || !Object.hasOwn(parsed, 'requests')) return parsed;
|
|
54
|
-
|
|
85
|
+
// Schema and authorization remain hard failures, including on the final turn.
|
|
86
|
+
// Preflight all dynamic ranges and aggregate budgets before any capability.
|
|
55
87
|
let batch;
|
|
56
88
|
try {
|
|
57
|
-
|
|
89
|
+
const operations = parseRetrievalResponse(parsed);
|
|
90
|
+
const allowed = new Set(scope.catalog().map(item => item.id));
|
|
91
|
+
if (operations.some(r => r.kind === 'read' ? !allowed.has(r.id) : r.sourceIds?.some(id => !allowed.has(id)))) throw new Error('Source not authorized');
|
|
92
|
+
if (finalization) {
|
|
93
|
+
const error = new Error(finalization.reason + '; final response required, no further retrieval permitted');
|
|
94
|
+
error.code = 'RETRIEVAL_FINAL_RESPONSE_REQUIRED';
|
|
95
|
+
throw error;
|
|
96
|
+
}
|
|
97
|
+
scope.preflight?.(operations);
|
|
98
|
+
const execute = () => executeRetrievalBatch(parsed, scope, request.signal);
|
|
99
|
+
batch = scope.transaction ? scope.transaction(execute) : execute();
|
|
58
100
|
} catch (error) {
|
|
59
101
|
request.signal?.throwIfAborted();
|
|
60
|
-
if (!
|
|
102
|
+
if (!finalization && budgetFailure(error)) { await finalize(error); continue; }
|
|
103
|
+
if (finalization || !correctProtocol || protocolCorrection || !(/Invalid retrieval batch|Invalid read request|Invalid search request|Invalid find request|Invalid line range|Line range outside source/.test(error.message))) throw error;
|
|
61
104
|
protocolCorrection = { code: error.code ?? 'MODEL_INVALID_RETRIEVAL' };
|
|
62
105
|
await request.onProtocolEvent?.({ action: 'retrieval-protocol-correction', code: protocolCorrection.code });
|
|
63
106
|
continue;
|
|
64
107
|
}
|
|
108
|
+
for (const item of batch) recordRetrieval(item.request, item.result, onTruncated, onRetrieved);
|
|
109
|
+
await checkExhausted(round + 1);
|
|
65
110
|
for (const item of batch) {
|
|
66
111
|
let candidate = [...retrieved, item];
|
|
67
|
-
let next =
|
|
112
|
+
let next = nextRequest(candidate);
|
|
68
113
|
if (correctProtocol) {
|
|
69
114
|
while (retrieved.length && Buffer.byteLength(compactFollowup(next.input)) + Buffer.byteLength(next.instructions) > maxInputBytes) {
|
|
70
115
|
retrieved.shift();
|
|
71
116
|
await request.onProtocolEvent?.({ action: 'prior-retrieval-deferred', notice: 'Prior returned fragment no longer in current prompt; reread approved source for evidence.' });
|
|
72
117
|
candidate = [...retrieved, item];
|
|
73
|
-
next =
|
|
118
|
+
next = nextRequest(candidate);
|
|
74
119
|
}
|
|
75
120
|
}
|
|
76
121
|
let fits = Buffer.byteLength(next.input) + Buffer.byteLength(next.instructions) <= maxInputBytes;
|
|
77
122
|
if (correctProtocol && !fits) fits = Buffer.byteLength(compactFollowup(next.input)) + Buffer.byteLength(next.instructions) <= maxInputBytes;
|
|
78
123
|
if (correctProtocol && !fits) {
|
|
79
124
|
const omitted = { request: item.request, result: { omitted: true, reason: 'Result omitted: request fewer lines/matches; no body or receipt provided.' } };
|
|
80
|
-
const minimal =
|
|
125
|
+
const minimal = nextRequest([...retrieved, omitted]);
|
|
81
126
|
if (Buffer.byteLength(compactFollowup(minimal.input)) + Buffer.byteLength(minimal.instructions) > maxInputBytes) throw new Error('Retrieval input metadata exceeds model capacity');
|
|
82
127
|
retrieved.push(omitted);
|
|
83
128
|
await request.onProtocolEvent?.({ action: 'oversized-retrieval-omitted', kind: item.request.kind });
|
package/src/review.mjs
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { REVIEW_INSTRUCTIONS, PROJECT_REVIEW_INSTRUCTIONS } from './prompts.mjs';
|
|
2
2
|
import { reviewRounds } from './review-rounds.mjs';
|
|
3
3
|
import { correctAnchor } from './anchor-correction.mjs';
|
|
4
|
+
import { correctedResponse } from './response-correction.mjs';
|
|
4
5
|
import { validateBusinessRequirement } from './business-requirement.mjs';
|
|
5
6
|
import { riskReadCoverage } from './risk-read-coverage.mjs';
|
|
6
7
|
import { BUSINESS_GROUP_INSTRUCTIONS, localBusinessGroups, prepareBusinessGroups, validateBusinessGroups } from './business-groups.mjs';
|
|
@@ -27,6 +28,7 @@ export { REVIEW_INSTRUCTIONS };
|
|
|
27
28
|
export { PROJECT_REVIEW_INSTRUCTIONS };
|
|
28
29
|
const nonempty = value => typeof value === 'string' && value.trim().length > 0 && value.length <= 8000;
|
|
29
30
|
const lines = text => text.replaceAll('\r\n', '\n').split('\n');
|
|
31
|
+
const protocolError = (code, message) => Object.assign(new Error(message), { code });
|
|
30
32
|
|
|
31
33
|
export function validateFindings(response, file) {
|
|
32
34
|
return validateResponse(response, file);
|
|
@@ -35,20 +37,21 @@ export function validateFindings(response, file) {
|
|
|
35
37
|
function validateResponse(response, file, knownChanges) {
|
|
36
38
|
if (Buffer.byteLength(typeof response === 'string' ? response : JSON.stringify(response) ?? '') > 128 * 1024) throw new Error('Review response too large');
|
|
37
39
|
if (typeof response === 'string') response = JSON.parse(response);
|
|
38
|
-
if (!response || !Array.isArray(response.findings) || response.findings.length > 50 || !Array.isArray(response.limitations) || response.limitations.length > 50 || response.limitations.some(s => !nonempty(s))) throw
|
|
40
|
+
if (!response || !Array.isArray(response.findings) || response.findings.length > 50 || !Array.isArray(response.limitations) || response.limitations.length > 50 || response.limitations.some(s => !nonempty(s))) throw protocolError('MODEL_INVALID_REVIEW_RESPONSE', 'Invalid review response');
|
|
39
41
|
const changes = knownChanges ?? changeMap(file.left.text, file.right.text);
|
|
40
42
|
const findings = response.findings.map(raw => {
|
|
41
|
-
if (!raw || raw.fileId !== file.id
|
|
42
|
-
|
|
43
|
+
if (!raw || raw.fileId !== file.id) throw protocolError('MODEL_INVALID_FINDING_IDENTITY', 'Invalid finding identity or severity');
|
|
44
|
+
if (!['critical', 'high', 'medium', 'low'].includes(raw.severity)) throw protocolError('MODEL_INVALID_FINDING_SEVERITY', 'Invalid finding identity or severity');
|
|
45
|
+
for (const key of ['title', 'evidence', 'trigger', 'impact', 'suggestion']) if (!nonempty(raw[key])) throw protocolError('MODEL_MISSING_FINDING_FIELD', 'Missing finding field: ' + key);
|
|
43
46
|
const anchor = raw.anchor;
|
|
44
47
|
if(file.reviewMode==='project'&&(anchor?.side==='old'||anchor?.kind==='property'))throw new Error('Project findings require current source evidence');
|
|
45
|
-
if (!anchor || !['line', 'property', 'file'].includes(anchor.kind)) throw
|
|
48
|
+
if (!anchor || !['line', 'property', 'file'].includes(anchor.kind)) throw protocolError('MODEL_INVALID_ANCHOR', 'Invalid anchor');
|
|
46
49
|
let validated;
|
|
47
50
|
if (anchor.kind === 'line') {
|
|
48
51
|
const side = anchor.side === 'old' ? file.left : anchor.side === 'new' ? file.right : null;
|
|
49
|
-
if (!side || !Number.isSafeInteger(anchor.start) || !Number.isSafeInteger(anchor.end) || anchor.start < 1 || anchor.end < anchor.start || !nonempty(anchor.snippet)) throw
|
|
52
|
+
if (!side || !Number.isSafeInteger(anchor.start) || !Number.isSafeInteger(anchor.end) || anchor.start < 1 || anchor.end < anchor.start || !nonempty(anchor.snippet)) throw protocolError('MODEL_INVALID_ANCHOR', 'Invalid line anchor');
|
|
50
53
|
const source = lines(side.text);
|
|
51
|
-
if (anchor.end > source.length || source.slice(anchor.start - 1, anchor.end).join('\n') !== anchor.snippet.replaceAll('\r\n', '\n')) throw
|
|
54
|
+
if (anchor.end > source.length || source.slice(anchor.start - 1, anchor.end).join('\n') !== anchor.snippet.replaceAll('\r\n', '\n')) throw protocolError('MODEL_ANCHOR_MISMATCH', 'Finding snippet does not match snapshot');
|
|
52
55
|
validated = { kind: 'line', side: anchor.side, start: anchor.start, end: anchor.end, snippet: anchor.snippet };
|
|
53
56
|
} else if (anchor.kind === 'property') {
|
|
54
57
|
if (!file.properties.some(p => p.name === anchor.name)) throw new Error('Unknown changed property');
|
|
@@ -128,7 +131,7 @@ export async function reviewSnapshot(snapshot, model, options = {}) {
|
|
|
128
131
|
await options.onProgress?.({stage:'file-started',snapshotId:input.id,fileId:file.id,path:file.path,status:state.status});
|
|
129
132
|
if (state.status !== 'pending') continue;
|
|
130
133
|
if (signal?.aborted) { state.status = 'cancelled'; state.reason = '用户取消'; continue; }
|
|
131
|
-
if (report.modelCalls >= maxCalls) { state.reason = '模型调用预算耗尽'; continue; }
|
|
134
|
+
if (report.modelCalls >= maxCalls || options.remainingCalls?.() <= 0) { state.reason = '模型调用预算耗尽'; continue; }
|
|
132
135
|
const prepared = preparePrimaryInput(input, file, { instructions, scope, grouping: report.grouping, maxInputBytes });
|
|
133
136
|
const { changes, payload } = prepared;
|
|
134
137
|
Object.assign(state, prepared.metadata);
|
|
@@ -141,13 +144,20 @@ export async function reviewSnapshot(snapshot, model, options = {}) {
|
|
|
141
144
|
if (state.groupFallback) report.limitations.push({ fileId: file.id, text: '关联组上下文超过输入预算,回退单文件审查' });
|
|
142
145
|
if (state.initialInputBytes > maxInputBytes) { state.status = 'blocked'; state.reason = '输入超过预算,未截断或提交模型'; continue; }
|
|
143
146
|
try {
|
|
144
|
-
const beforeCall = () => {
|
|
147
|
+
const beforeCall = ({ reserved = false } = {}) => {
|
|
148
|
+
if (!reserved && options.remainingCalls?.() <= 0) throw Object.assign(new Error('Shared project model call limit exceeded'), { code: 'PROJECT_MODEL_CALL_LIMIT' });
|
|
145
149
|
if (report.modelCalls >= maxCalls) throw new Error('Model call budget exceeded');
|
|
146
150
|
report.modelCalls++;
|
|
147
151
|
};
|
|
148
|
-
const
|
|
152
|
+
const onBudgetLimited = info => {
|
|
153
|
+
state.retrievalLimited = true;
|
|
154
|
+
(state.retrievalLimits ??= []).push(info);
|
|
155
|
+
report.limitations.push({ fileId: file.id, text: '检索预算受限,已保留现有结果但不能视为完整审查:' + info.reason });
|
|
156
|
+
};
|
|
157
|
+
const onProtocolEvent = async info => { await options.onProgress?.({ stage: 'protocol-correction', fileId: file.id, path: file.path, ...info }); };
|
|
158
|
+
const trackedModel=request=>model({...request,onProtocolEvent,onProtocolCorrection:async info=>{beforeCall({ reserved: true });await request.onProtocolCorrection?.(info);await options.onProgress?.({stage:'protocol-correction',fileId:file.id,path:file.path,...info});}});
|
|
149
159
|
const executor = scope
|
|
150
|
-
? request => retrievalLoop(trackedModel, request, scope, { correctProtocol:options.correctProtocol===true,maxInputBytes, beforeCall, onRetrieved: records => {
|
|
160
|
+
? (request, onLimited = () => {}) => retrievalLoop(trackedModel, { ...request, onProtocolEvent }, scope, { correctProtocol:options.correctProtocol===true,maxInputBytes, beforeCall, onBudgetLimited: info => { onBudgetLimited(info); onLimited(info); }, onRetrieved: records => {
|
|
151
161
|
(state.retrievalAudit ??= []).push(...records);
|
|
152
162
|
}, onTruncated: () => {
|
|
153
163
|
if (!state.retrievalTruncated) report.limitations.push({ fileId: file.id, text: '检索结果达到数量上限,部分匹配未提供给模型' });
|
|
@@ -164,7 +174,7 @@ export async function reviewSnapshot(snapshot, model, options = {}) {
|
|
|
164
174
|
const groupInput = prepareBusinessGroups(input.files);
|
|
165
175
|
if (Buffer.byteLength(groupInput) + Buffer.byteLength(BUSINESS_GROUP_INSTRUCTIONS) > maxInputBytes) throw new Error('Business grouping input budget exceeded');
|
|
166
176
|
beforeCall();
|
|
167
|
-
const response = await
|
|
177
|
+
const response = await trackedModel({ ...enclosingRequest, instructions: BUSINESS_GROUP_INSTRUCTIONS, input: groupInput });
|
|
168
178
|
enclosingRequest.signal.throwIfAborted();
|
|
169
179
|
grouped = validateBusinessGroups(response, input.files);
|
|
170
180
|
}
|
|
@@ -218,30 +228,48 @@ export async function reviewSnapshot(snapshot, model, options = {}) {
|
|
|
218
228
|
if(report.modelCalls>=maxCalls)throw new Error('Model call budget exceeded before next review pass');
|
|
219
229
|
if(rounds>1) state.reviewPasses[batchIndex].status='running';
|
|
220
230
|
const request = { ...enclosingRequest, input: batchPayload };
|
|
221
|
-
|
|
222
|
-
let
|
|
223
|
-
|
|
224
|
-
if (
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
231
|
+
let auditStart = state.retrievalAudit?.length ?? 0;
|
|
232
|
+
let finalResponseOnly = false;
|
|
233
|
+
const generate = async attempt => {
|
|
234
|
+
if (!initialInputBudget(attempt, scope, maxInputBytes).fits) throw new Error('Review correction input budget exceeded');
|
|
235
|
+
return executor(attempt, () => { finalResponseOnly = true; });
|
|
236
|
+
};
|
|
237
|
+
const validate = response => {
|
|
238
|
+
if (options.enableAnchorCorrection === true) {
|
|
239
|
+
if (Buffer.byteLength(typeof response === 'string' ? response : JSON.stringify(response) ?? '') > 128 * 1024) throw new Error('Review response too large');
|
|
240
|
+
const decoded = typeof response === 'string' ? JSON.parse(response) : structuredClone(response);
|
|
241
|
+
if (Array.isArray(decoded?.findings) && decoded.findings.length <= 50) {
|
|
242
|
+
decoded.findings = decoded.findings.map(raw => {
|
|
243
|
+
const target = batch.interaction ? input.files.find(f => f.id === raw?.fileId && batch.interaction.fileIds.includes(f.id) && f.eligibility === 'reviewable') : file;
|
|
244
|
+
if (!target) return raw;
|
|
245
|
+
const correction = correctAnchor(raw, target, target === file ? changes : changeMap(target.left.text,target.right.text), { signal: request.signal });
|
|
246
|
+
if (!['unchanged','not-applicable'].includes(correction.status)) (state.anchorCorrections ??= []).push(correction);
|
|
247
|
+
return correction.status === 'corrected' ? correction.candidate : raw;
|
|
248
|
+
});
|
|
249
|
+
response = decoded;
|
|
250
|
+
}
|
|
235
251
|
}
|
|
236
|
-
|
|
252
|
+
const generated = batch.interaction ? validateInteractionFindings(response, input, batch.interaction, (state.retrievalAudit ?? []).slice(auditStart), scope.catalog()) : validateResponse(response, file, changes);
|
|
253
|
+
if (batch.synthesis) validateSynthesisReads(generated.findings, changes, (state.retrievalAudit ?? []).slice(auditStart), scope.catalog(), file.id, input.id);
|
|
254
|
+
if (!batch.interaction && state.sourceMode === 'change-windows') validateWindowFindings(generated.findings, batch.synthesis ? prepared.synthesis.windows : JSON.parse(batch.payload).windows, batch.synthesis === true);
|
|
255
|
+
return generated;
|
|
256
|
+
};
|
|
257
|
+
const generated = await correctedResponse(generate, validate, request, {
|
|
258
|
+
enabled: options.correctProtocol === true,
|
|
259
|
+
canCorrect: () => !finalResponseOnly,
|
|
260
|
+
onCorrection: async info => {
|
|
261
|
+
// A fresh generation must establish its own synthesis/interaction read evidence.
|
|
262
|
+
auditStart = state.retrievalAudit?.length ?? 0;
|
|
263
|
+
(state.responseCorrections ??= []).push({ code: info.code, round: batch.round, phase });
|
|
264
|
+
report.limitations.push({ fileId: file.id, text: '主审响应校验失败,已请求一次显式纠正;原响应未接受,纠正不证明缺陷成立。' });
|
|
265
|
+
await options.onProgress?.({ stage: 'protocol-correction', fileId: file.id, path: file.path, ...info });
|
|
266
|
+
},
|
|
267
|
+
});
|
|
237
268
|
if (options.enableRiskPlanning === true && !batch.synthesis && batch.round === 1) {
|
|
238
269
|
const entry=state.riskPlans.at(-1);
|
|
239
270
|
entry.sourceReadCoverage=riskReadCoverage(entry.risks,(state.retrievalAudit??[]).slice(auditStart),scope?.catalog()??[],input.id);
|
|
240
271
|
if(entry.sourceReadCoverage.some(c=>c.unreadSourceIds.length)) report.limitations.push({fileId:file.id,text:'风险计划请求的部分来源未在本次生成阶段读取;不能视为已核查风险条件或排除问题'});
|
|
241
272
|
}
|
|
242
|
-
const generated = batch.interaction ? validateInteractionFindings(response, input, batch.interaction, (state.retrievalAudit ?? []).slice(auditStart), scope.catalog()) : validateResponse(response, file, changes);
|
|
243
|
-
if (batch.synthesis) validateSynthesisReads(generated.findings, changes, (state.retrievalAudit ?? []).slice(auditStart), scope.catalog(), file.id, input.id);
|
|
244
|
-
if (!batch.interaction && state.sourceMode === 'change-windows') validateWindowFindings(generated.findings, batch.synthesis ? prepared.synthesis.windows : JSON.parse(batch.payload).windows, batch.synthesis === true);
|
|
245
273
|
request.signal.throwIfAborted();
|
|
246
274
|
if(rounds>1)generated.findings.forEach(f=>{f.reviewRound=batch.round;});
|
|
247
275
|
report.findings.push(...generated.findings);
|
|
@@ -250,8 +278,8 @@ export async function reviewSnapshot(snapshot, model, options = {}) {
|
|
|
250
278
|
if (options.enableVerification === true && generated.findings.length) {
|
|
251
279
|
for (const finding of generated.findings) finding.verification = { status: 'pending', causality: 'unverified' };
|
|
252
280
|
try {
|
|
253
|
-
const verdicts = await verificationLoop(trackedModel, request, generated.findings, scope, {
|
|
254
|
-
correctProtocol:options.correctProtocol===true,maxInputBytes, beforeCall,
|
|
281
|
+
const verdicts = await verificationLoop(trackedModel, { ...request, onProtocolEvent }, generated.findings, scope, {
|
|
282
|
+
correctProtocol:options.correctProtocol===true,maxInputBytes, beforeCall, onBudgetLimited,
|
|
255
283
|
onRetrieved: records => { (state.retrievalAudit ??= []).push(...records); },
|
|
256
284
|
onTruncated: () => { report.limitations.push({ fileId: file.id, text: '复核检索结果达到数量上限' }); },
|
|
257
285
|
});
|
|
@@ -292,15 +320,17 @@ export async function reviewSnapshot(snapshot, model, options = {}) {
|
|
|
292
320
|
}, { instructions, input: payload }, timeoutMs, signal);
|
|
293
321
|
if (signal?.aborted) throw signal.reason;
|
|
294
322
|
|
|
295
|
-
state.status = 'completed';
|
|
323
|
+
state.status = state.retrievalLimited ? 'pending' : 'completed';
|
|
324
|
+
if (state.retrievalLimited) state.reason = '检索预算受限,未完成完整审查;已有发现保留,不表示通过';
|
|
296
325
|
|
|
297
326
|
} catch (error) {
|
|
298
|
-
state.status = signal?.aborted ? 'cancelled' : 'failed';
|
|
327
|
+
state.status = signal?.aborted ? 'cancelled' : error?.code === 'PROJECT_MODEL_CALL_LIMIT' ? 'pending' : 'failed';
|
|
299
328
|
state.reason = error?.message ?? String(error);
|
|
300
329
|
if(error.attempts)state.protocolAttempts=error.attempts;
|
|
330
|
+
if(error.responseAttempts)state.responseAttempts=error.responseAttempts;
|
|
301
331
|
if(error.diagnostic)state.protocolDiagnostic=error.diagnostic;
|
|
302
332
|
for(const pass of state.reviewPasses??[])if(pass.status==='running')pass.status=signal?.aborted?'cancelled':'failed';
|
|
303
|
-
state.failureCode = signal?.aborted ? 'CANCELLED' : ['MODEL_INVALID_JSON','MODEL_INCOMPLETE_OUTPUT','MODEL_TIMEOUT','MODEL_INVALID_READ_REQUEST'].includes(error?.code) ? error.code : 'REVIEW_EXECUTION_FAILED';
|
|
333
|
+
state.failureCode = signal?.aborted ? 'CANCELLED' : ['MODEL_INVALID_JSON','MODEL_INCOMPLETE_OUTPUT','MODEL_TIMEOUT','MODEL_INVALID_READ_REQUEST','MODEL_OUTPUT_LIMIT','MODEL_PROTOCOL_ERROR','MODEL_INVALID_REVIEW_RESPONSE','MODEL_INVALID_FINDING_IDENTITY','MODEL_INVALID_FINDING_SEVERITY','MODEL_MISSING_FINDING_FIELD','MODEL_INVALID_ANCHOR','MODEL_ANCHOR_MISMATCH','READER_CALL_BUDGET_EXCEEDED','READER_OUTPUT_BUDGET_EXCEEDED','RETRIEVAL_ROUND_BUDGET_EXCEEDED','RETRIEVAL_FINAL_RESPONSE_REQUIRED','PROJECT_MODEL_CALL_LIMIT'].includes(error?.code) ? error.code : 'REVIEW_EXECUTION_FAILED';
|
|
304
334
|
}
|
|
305
335
|
}
|
|
306
336
|
await options.onProgress?.({stage:'batch-complete',snapshotId:input.id,files:report.files,findings:report.findings,limitations:report.limitations});
|
package/src/snapshot-reader.mjs
CHANGED
|
@@ -14,6 +14,12 @@ function validQuery(query, limit) {
|
|
|
14
14
|
return typeof query === 'string' && query.length > 0 && query.length <= 256 && !/[\r\n]/.test(query) && Number.isSafeInteger(limit) && limit >= 1 && limit <= 100;
|
|
15
15
|
}
|
|
16
16
|
|
|
17
|
+
function budgetError(kind) {
|
|
18
|
+
const error = new Error('Reader ' + kind + ' budget exceeded');
|
|
19
|
+
error.code = kind === 'call' ? 'READER_CALL_BUDGET_EXCEEDED' : 'READER_OUTPUT_BUDGET_EXCEEDED';
|
|
20
|
+
return error;
|
|
21
|
+
}
|
|
22
|
+
|
|
17
23
|
// This capability reads only caller-approved text entries, never the filesystem.
|
|
18
24
|
export function createSnapshotReader(snapshotId, entries, { maxBytes = 4 * 1024 * 1024, maxCalls = 50, maxOutputBytes = 256 * 1024, signal } = {}) {
|
|
19
25
|
if (typeof snapshotId !== 'string' || !snapshotId || snapshotId.length > 200) throw new Error('Invalid snapshot identity');
|
|
@@ -24,16 +30,21 @@ export function createSnapshotReader(snapshotId, entries, { maxBytes = 4 * 1024
|
|
|
24
30
|
}
|
|
25
31
|
signal?.throwIfAborted();
|
|
26
32
|
|
|
27
|
-
let calls = 0, outputBytes = 0;
|
|
33
|
+
let calls = 0, outputBytes = 0, inTransaction = false;
|
|
28
34
|
const begin = () => {
|
|
29
35
|
signal?.throwIfAborted();
|
|
30
|
-
if (calls >= maxCalls) throw
|
|
36
|
+
if (calls >= maxCalls) throw budgetError('call');
|
|
31
37
|
calls++;
|
|
32
38
|
};
|
|
39
|
+
const resultSize = result => {
|
|
40
|
+
const size = Buffer.byteLength(JSON.stringify(result));
|
|
41
|
+
if (size > 64 * 1024) throw budgetError('output');
|
|
42
|
+
return size;
|
|
43
|
+
};
|
|
33
44
|
const finish = result => {
|
|
34
45
|
signal?.throwIfAborted();
|
|
35
|
-
const size =
|
|
36
|
-
if (
|
|
46
|
+
const size = resultSize(result);
|
|
47
|
+
if (outputBytes + size > maxOutputBytes) throw budgetError('output');
|
|
37
48
|
outputBytes += size;
|
|
38
49
|
return result;
|
|
39
50
|
};
|
|
@@ -47,55 +58,91 @@ export function createSnapshotReader(snapshotId, entries, { maxBytes = 4 * 1024
|
|
|
47
58
|
sources.set(entry.id, { path: entry.path, side: entry.side, text: entry.text, hash: createHash('sha256').update(entry.text).digest('hex') });
|
|
48
59
|
}
|
|
49
60
|
|
|
61
|
+
const find = ({ query, limit = 20 }) => {
|
|
62
|
+
if (!validQuery(query, limit)) throw new Error('Invalid find request');
|
|
63
|
+
const matches = [];
|
|
64
|
+
for (const [id, source] of sources) {
|
|
65
|
+
signal?.throwIfAborted();
|
|
66
|
+
if (typeof source.path !== 'string' || !source.path.includes(query)) continue;
|
|
67
|
+
if (matches.length === limit) return { snapshotId, matches, truncated: true };
|
|
68
|
+
matches.push({ sourceId: id, path: source.path, side: source.side, hash: source.hash });
|
|
69
|
+
}
|
|
70
|
+
return { snapshotId, matches, truncated: false };
|
|
71
|
+
};
|
|
72
|
+
const search = ({ query, limit = 20, sourceIds }) => {
|
|
73
|
+
if (!validQuery(query, limit)) throw new Error('Invalid search request');
|
|
74
|
+
if (sourceIds !== undefined && (!Array.isArray(sourceIds) || !sourceIds.length || sourceIds.length > 20 || new Set(sourceIds).size !== sourceIds.length || sourceIds.some(id => !sources.has(id)))) throw new Error('Invalid search source filter');
|
|
75
|
+
const selected = sourceIds === undefined ? null : new Set(sourceIds);
|
|
76
|
+
const matches = [];
|
|
77
|
+
for (const [id, source] of sources) {
|
|
78
|
+
if (selected && !selected.has(id)) continue;
|
|
79
|
+
signal?.throwIfAborted();
|
|
80
|
+
for (const range of lineRanges(source.text)) {
|
|
81
|
+
if (!source.text.slice(range.start, range.end).includes(query)) continue;
|
|
82
|
+
if (matches.length === limit) return { snapshotId, matches, truncated: true };
|
|
83
|
+
matches.push({ sourceId: id, hash: source.hash, line: range.line });
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
return { snapshotId, matches, truncated: false };
|
|
87
|
+
};
|
|
88
|
+
const read = ({ id, start, count }) => {
|
|
89
|
+
if (!sources.has(id)) throw new Error('Source not authorized');
|
|
90
|
+
if (!Number.isSafeInteger(start) || start < 1 || !Number.isSafeInteger(count) || count < 1 || count > 200) throw new Error('Invalid line range');
|
|
91
|
+
const source = sources.get(id);
|
|
92
|
+
let beginOffset = -1, endOffset = 0, selectedCount = 0;
|
|
93
|
+
for (const range of lineRanges(source.text)) {
|
|
94
|
+
if (range.line < start) continue;
|
|
95
|
+
if (beginOffset < 0) beginOffset = range.start;
|
|
96
|
+
endOffset = range.end;
|
|
97
|
+
selectedCount++;
|
|
98
|
+
if (selectedCount === count) break;
|
|
99
|
+
}
|
|
100
|
+
if (beginOffset < 0) throw new Error('Line range outside source');
|
|
101
|
+
const text = source.text.slice(beginOffset, endOffset);
|
|
102
|
+
if (Buffer.byteLength(text) > 64 * 1024) throw budgetError('output');
|
|
103
|
+
return { snapshotId, sourceId: id, hash: source.hash, start, count: selectedCount, text, endOfSource: endOffset === source.text.length };
|
|
104
|
+
};
|
|
105
|
+
|
|
50
106
|
return Object.freeze({
|
|
51
107
|
usage: () => ({ calls, outputBytes }),
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
108
|
+
budget: () => ({ maxCalls, maxOutputBytes, remainingCalls: maxCalls - calls, remainingOutputBytes: maxOutputBytes - outputBytes, maxResultBytes: 64 * 1024 }),
|
|
109
|
+
// A synchronous host-only batch commits its ledger only as a whole. This
|
|
110
|
+
// also covers decorated response bytes that preflight cannot predict.
|
|
111
|
+
// Callers must not publish results/audit records before this returns.
|
|
112
|
+
transaction(execute) {
|
|
113
|
+
signal?.throwIfAborted();
|
|
114
|
+
if (inTransaction || typeof execute !== 'function') throw new Error('Invalid reader transaction');
|
|
115
|
+
const previousCalls = calls, previousBytes = outputBytes;
|
|
116
|
+
inTransaction = true;
|
|
117
|
+
try {
|
|
118
|
+
const result = execute();
|
|
119
|
+
if (result && typeof result.then === 'function') throw new Error('Invalid asynchronous reader transaction');
|
|
57
120
|
signal?.throwIfAborted();
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
121
|
+
return result;
|
|
122
|
+
} catch (error) {
|
|
123
|
+
calls = previousCalls; outputBytes = previousBytes;
|
|
124
|
+
throw error;
|
|
125
|
+
} finally { inTransaction = false; }
|
|
63
126
|
},
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
if (
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
return finish({ snapshotId, matches, truncated: false });
|
|
127
|
+
// Host-only dry run. Validate ALL ranges and sources before charging anything.
|
|
128
|
+
// No evidence or receipts escape this preflight. Decorated reads are still
|
|
129
|
+
// measured by finish; this never reserves or increases the shared budget.
|
|
130
|
+
preflight(requests) {
|
|
131
|
+
signal?.throwIfAborted();
|
|
132
|
+
if (!Array.isArray(requests) || !requests.length || requests.length > 8) throw new Error('Invalid retrieval batch');
|
|
133
|
+
const results = requests.map(request => {
|
|
134
|
+
if (request?.kind === 'read') return read(request);
|
|
135
|
+
if (request?.kind === 'find') return find(request);
|
|
136
|
+
if (request?.kind === 'search') return search(request);
|
|
137
|
+
throw new Error('Unsupported retrieval operation');
|
|
138
|
+
});
|
|
139
|
+
if (calls + requests.length > maxCalls) throw budgetError('call');
|
|
140
|
+
const size = results.reduce((sum, result) => sum + resultSize(result), 0);
|
|
141
|
+
if (outputBytes + size > maxOutputBytes) throw budgetError('output');
|
|
80
142
|
},
|
|
143
|
+
find(request) { begin(); return finish(find(request)); },
|
|
144
|
+
search(request) { begin(); return finish(search(request)); },
|
|
81
145
|
// decorate is a host-only callback; model requests never supply it.
|
|
82
|
-
read(
|
|
83
|
-
begin();
|
|
84
|
-
if (!sources.has(id)) throw new Error('Source not authorized');
|
|
85
|
-
if (!Number.isSafeInteger(start) || start < 1 || !Number.isSafeInteger(count) || count < 1 || count > 200) throw new Error('Invalid line range');
|
|
86
|
-
const source = sources.get(id);
|
|
87
|
-
let beginOffset = -1, endOffset = 0, selectedCount = 0;
|
|
88
|
-
for (const range of lineRanges(source.text)) {
|
|
89
|
-
if (range.line < start) continue;
|
|
90
|
-
if (beginOffset < 0) beginOffset = range.start;
|
|
91
|
-
endOffset = range.end;
|
|
92
|
-
selectedCount++;
|
|
93
|
-
if (selectedCount === count) break;
|
|
94
|
-
}
|
|
95
|
-
if (beginOffset < 0) throw new Error('Line range outside source');
|
|
96
|
-
const text = source.text.slice(beginOffset, endOffset);
|
|
97
|
-
if (Buffer.byteLength(text) > 64 * 1024) throw new Error('Reader output budget exceeded');
|
|
98
|
-
return finish(decorate({ snapshotId, sourceId: id, hash: source.hash, start, count: selectedCount, text, endOfSource: endOffset === source.text.length }));
|
|
99
|
-
},
|
|
146
|
+
read(request, decorate = value => value) { begin(); return finish(decorate(read(request))); },
|
|
100
147
|
});
|
|
101
148
|
}
|