dsh-decision-layer 0.6.4 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +32 -18
- package/README.md +37 -19
- package/docs/en/DETAIL.md +20 -0
- package/docs/zh/DETAIL.md +20 -0
- package/lib/client.js +836 -358
- package/locale/en.json +2 -1
- package/locale/zh.json +2 -1
- package/package.json +6 -2
- package/service/config-store.mjs +60 -6
- package/service/danger-gate.mjs +16 -2
- package/service/dsh.mjs +33 -14
- package/service/log-analyze.mjs +6 -5
- package/service/manager-api.mjs +3 -3
- package/service/narrowing.mjs +29 -13
- package/service/self-check.mjs +16 -4
- package/service/session-state.mjs +5 -2
- package/service/task-completion.mjs +14 -4
package/README.en.md
CHANGED
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
|
|
5
5
|
`dsh-decision-layer` adds structured, explicitly scoped decisions to the DeepSeek Harness (DSH) agent loop, backed by a pluggable adjudication backend. Dangerous-call gating, output self-check, tool narrowing, and loop prevention hook into the agent loop as automatic interventions.
|
|
6
6
|
|
|
7
|
-
> **0.
|
|
7
|
+
> **0.7.1 is a preview release.** The plugin works, but how much real value the adjudication interventions add is still being explored — every threshold is an unvalidated heuristic default. Treat it as something to observe, not a proven guarantee of impact. See [ROADMAP.md](<ROADMAP.md>) for scope and progress.
|
|
8
8
|
|
|
9
9
|
## Installation
|
|
10
10
|
|
|
11
|
-
Published to npm and GitHub.
|
|
11
|
+
Published to npm and GitHub Release.
|
|
12
12
|
|
|
13
13
|
From npm (append `@latest`):
|
|
14
14
|
|
|
@@ -16,12 +16,14 @@ From npm (append `@latest`):
|
|
|
16
16
|
dsh plugin --profile web add dsh-decision-layer@latest
|
|
17
17
|
```
|
|
18
18
|
|
|
19
|
-
Or from GitHub
|
|
19
|
+
Or from a GitHub Release: install directly from the version's tgz asset URL.
|
|
20
20
|
|
|
21
21
|
```sh
|
|
22
|
-
dsh plugin --profile web add github
|
|
22
|
+
dsh plugin --profile web add https://github.com/xbzbing/dsh-decision-layer/releases/download/v<version>/dsh-decision-layer-<version>.tgz
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
+
> The repository no longer commits the built `lib/client.js` ([ADR-0012](<docs/adr/0012-distribution-via-github-actions.md>)), so installing `github:xbzbing/dsh-decision-layer` without a ref does **not** work — that tree has no `lib/`, and while the host half still loads, the panel never appears. Use the Release tarball on the GitHub side.
|
|
26
|
+
|
|
25
27
|
Or build locally and install the `file:` path:
|
|
26
28
|
|
|
27
29
|
```sh
|
|
@@ -34,20 +36,32 @@ Restart `dsh web` and refresh after installing or updating: the host and client
|
|
|
34
36
|
|
|
35
37
|
## Features
|
|
36
38
|
|
|
37
|
-
- **Manual verdicts** — `decision_evaluate` and the `decision-layer` skill submit `noul
|
|
38
|
-
- **
|
|
39
|
-
- **
|
|
40
|
-
- **
|
|
41
|
-
- **Task-completion review** — when the user spells out multiple delivery conditions
|
|
42
|
-
- **
|
|
43
|
-
- **
|
|
44
|
-
- **
|
|
45
|
-
- **
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
39
|
+
- **Manual verdicts** — `decision_evaluate` and the `decision-layer` skill submit `noul` / `score` / `choice` questions; `decision_check_connection` tests connectivity. Manual calls do not count as automatic decisions and do not change agent behavior.
|
|
40
|
+
- **Dangerous-call gate** — before a destructive tool call, a redacted summary asks the backend for a verdict; only a high-confidence deny blocks the call. Allow, ask, low confidence, backend failure, and invalid responses do not intervene and run the host's original path. Rules and the block confidence floor are configurable.
|
|
41
|
+
- **Output self-check and quality trend** — scores the final reply at turn end (0–2), observe-only by default; an opt-in low-score re-prompt fires at most once per turn. Several consecutive high-confidence low scores tint the dock trigger grey → amber → red as a reminder.
|
|
42
|
+
- **Tool narrowing** — at turn start, drops optional tools by task relevance, intersecting the host-allowed set and never expanding it; core tools and keep-prefix matches are always kept, over-cap falls back to observe-only, too-few-kept is treated as untrustworthy and skipped, and small-context backends are batched automatically. Enforced by default; observe-only is available.
|
|
43
|
+
- **Task-completion review** — when the user spells out multiple delivery conditions, each is judged satisfied / unsatisfied / insufficient evidence, with tool evidence primary. Observe-only by default, with an opt-in re-prompt.
|
|
44
|
+
- **Loop guard** — denies the same call repeated consecutively past a threshold; deterministic, local, and backend-free. The per-feature switches do not affect it, but the session master switch pauses it too.
|
|
45
|
+
- **Session panel and decision-analysis view** — the panel shows automatic-decision metrics, the session master switch, and the recent log; the "Decision analysis" tab shows a per-session process profile, trend backtest, and decision accuracy, with per-row ratings and a session intervention switch.
|
|
46
|
+
- **Configuration and persistence** — each decision point has its own switch and configurable thresholds in the plugin details, written to `~/.config/dsh-decision-layer/config.json` and applied live. Defaults and field names are in [ADR-0006](<docs/adr/0006-configurable-thresholds.md>). You can also edit that file by hand, but only with valid values: an out-of-range value, a misspelled field, or a wrong type makes the whole config fail validation, the config page cannot load it, and every decision point reverts to its own safe default (see [ADR-0005](<docs/adr/0005-config-wholesale-replace-and-live-reload.md>)). Decision logs and session metrics persist for 30 days for review and calibration.
|
|
47
|
+
- **Pluggable backend** — defaults to `https://api.typesafe.ai` and `jev-latest`, with per-field URL/key/model overrides under the same protocol; no request without a key. A custom `http://` endpoint requires confirming the plaintext-transport risk for that exact URL, and again whenever the URL changes.
|
|
48
|
+
|
|
49
|
+
## Privacy and backend
|
|
50
|
+
|
|
51
|
+
Once a key is configured, the plugin sends adjudication requests to the remote endpoint `https://api.typesafe.ai` (model `jev-latest`) by default. The payload carries real context:
|
|
52
|
+
|
|
53
|
+
- Gate: the tool name plus redacted command/path summaries (up to 2048 characters each).
|
|
54
|
+
- Self-check: the turn's final reply (up to about 24K characters).
|
|
55
|
+
- Narrowing: the text of this step's user messages (up to about 8K characters) plus candidate tool names/descriptions.
|
|
56
|
+
- Task-completion review: delivery conditions extracted from this turn's user messages (up to 20), tool names with their ok/error status (no arguments or output), and the first ~8K characters of the reply.
|
|
57
|
+
- Manual verdicts: `decision_evaluate` and the `decision-layer` skill send whatever the agent submits, unchanged.
|
|
58
|
+
|
|
59
|
+
User-message text may include context the host injects, such as workspace instructions (AGENTS.md), runtime details, and memory snippets. This content leaves your machine, and the remote provider may log, retain, or otherwise use it depending on its terms; redaction and truncation reduce risk but are not anonymization.
|
|
60
|
+
|
|
61
|
+
Two kinds of data are also stored locally for 30 days: decision logs in `~/.config/dsh-decision-layer/logs/` and session snapshots in `~/.config/dsh-decision-layer/state/`. They include redacted command/path summaries and short request subjects; delete them yourself if needed.
|
|
62
|
+
|
|
63
|
+
**If your session content is sensitive, prefer a locally deployed model** so adjudication data never leaves the machine. `tev1` is served natively by ollama at `/v1/systemone`; see [Running with a local tev1](<docs/en/deploy-tev1-with-plugin.md>). Any other local model works as long as it exposes a compatible `/v1/systemone` endpoint, pointed to from the config page.
|
|
50
64
|
|
|
51
65
|
## Safety boundary and development
|
|
52
66
|
|
|
53
|
-
Automatic intervention only subtracts: the gate only denies, narrowing only removes tools, and neither grants new permissions. The gate
|
|
67
|
+
Automatic intervention only subtracts: the gate only denies, narrowing only removes tools, and neither grants new permissions. The gate only blocks on a high-confidence deny; on backend failure, low confidence, or an invalid response it does not intervene and runs the host's original path. Self-check, narrowing, and loop-guard errors skip intervention. The redacted gate summary sent to the backend carries only tool name and command/path with secret redaction; the plugin never reads file content, but the command itself (for example a heredoc or text written with echo) is included as-is. Node.js `>=20.11`; `npm test` runs the tests, `npm run build:client` builds `lib/client.js` (this is also where strict `tsc` runs), and `npm run check` runs both. **Build output is not committed** ([ADR-0012](<docs/adr/0012-distribution-via-github-actions.md>)): `lib/` is gitignored and the source is the only source of truth. Releases run through `.github/workflows/release.yml`, which builds, gates, `npm pack`s and then asserts with `scripts/assert-pack.mjs` that the tarball really carries `lib/client.js` and leaks no sources — only then does it publish to npm or open a Release. It runs when a version bump lands on main and publishes through npm Trusted Publishing (OIDC), so no long-lived token is involved. Architecture decisions live in [docs/adr/](<docs/adr/README.md>): why each rule exists and what was rejected. 0.7.1 is a preview: the interventions work, but their real value and thresholds are still being explored — feedback on misjudgments and feel observed in the decision log is welcome.
|
package/README.md
CHANGED
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
|
|
5
5
|
DeepSeek Harness(DSH)的结构化裁决插件。它把有明确判断标准的问题交给同协议的裁决后端,供 agent 在需要时使用,并在 agent loop 上接入危险门控、产出自检、工具收窄和防循环等自动介入。
|
|
6
6
|
|
|
7
|
-
> **0.
|
|
7
|
+
> **0.7.1 是体验版。** 插件功能已可用,但裁决介入能带来多少实际价值仍在探索中——各决策点的阈值都是未经验证的启发式默认值,建议先以观察心态体验,别当成经过验证的效果保证。功能与后续进度以 [ROADMAP.md](<ROADMAP.md>) 为准。
|
|
8
8
|
|
|
9
9
|
## 安装
|
|
10
10
|
|
|
11
|
-
已发布到 npm 与 GitHub。
|
|
11
|
+
已发布到 npm 与 GitHub Release。
|
|
12
12
|
|
|
13
13
|
从 npm 安装(补 `@latest`):
|
|
14
14
|
|
|
@@ -16,12 +16,14 @@ DeepSeek Harness(DSH)的结构化裁决插件。它把有明确判断标准
|
|
|
16
16
|
dsh plugin --profile web add dsh-decision-layer@latest
|
|
17
17
|
```
|
|
18
18
|
|
|
19
|
-
或从 GitHub
|
|
19
|
+
或从 GitHub Release 安装:直接用对应版本 tgz 附件的 URL 安装。
|
|
20
20
|
|
|
21
21
|
```sh
|
|
22
|
-
dsh plugin --profile web add github
|
|
22
|
+
dsh plugin --profile web add https://github.com/xbzbing/dsh-decision-layer/releases/download/v<版本>/dsh-decision-layer-<版本>.tgz
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
+
> 仓库不再提交构建产物 `lib/client.js`([ADR-0012](<docs/adr/0012-distribution-via-github-actions.md>)),因此**不带 ref 的 `github:xbzbing/dsh-decision-layer` 安装不可用**——那样拿到的是没有 `lib/` 的源码树,host 半仍能加载,但面板不会出现。GitHub 侧请用上面的 Release 附件。
|
|
26
|
+
|
|
25
27
|
也可从源码本地构建后用 `file:` 安装:
|
|
26
28
|
|
|
27
29
|
```sh
|
|
@@ -34,24 +36,40 @@ dsh plugin --profile web add file:/绝对路径/dsh-decision-layer
|
|
|
34
36
|
|
|
35
37
|
## 功能
|
|
36
38
|
|
|
37
|
-
- **手动裁决**:`decision_evaluate` 工具与 `decision-layer` skill
|
|
38
|
-
-
|
|
39
|
-
-
|
|
40
|
-
-
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
39
|
+
- **手动裁决**:`decision_evaluate` 工具与 `decision-layer` skill 可显式提交 `noul` / `score` / `choice` 问题,`decision_check_connection` 做连通测试;手动调用不计入自动裁决指标,也不改变 agent 行为。
|
|
40
|
+
- **危险门控**:破坏性工具调用执行前用脱敏摘要请后端裁决,只有高置信「拒绝」才拦截;放行、询问、低置信、后端失败或响应不合法一律不介入,按宿主原路径执行。危险规则与拦截置信度门槛可配。
|
|
41
|
+
- **产出自检与质量趋势**:回合结束给最终回复打分(0–2),默认只记录;可开启低分补完,同一回合最多补一次。连续多次高置信低分时,dock 触发器图标会从灰变橙、变红作为提醒。
|
|
42
|
+
- **工具收窄**:回合开始时按任务相关性摘掉可选工具,只在宿主已允许的集合内取交集、不扩权;核心工具与保留前缀命中的工具始终保留,可选工具过多时自动降为仅观察、保留过少时视为判定不可信并跳过,对小上下文后端自动分批。默认真正生效,也可切到仅观察。
|
|
43
|
+
- **任务完成核对**:用户明确列出多个交付条件时,逐项判定「已满足 / 未满足 / 证据不足」,以工具证据为准;默认只观察,可配补完。
|
|
44
|
+
- **防循环**:同一调用连续重复超过阈值即拒绝,确定性本地逻辑,不依赖后端;不受各决策点开关影响,但会话总开关关闭时同样暂停。
|
|
45
|
+
- **会话面板与决策分析页**:面板展示自动裁决指标、会话总开关和最近日志;「决策分析」tab 按会话展示过程画像、趋势回测与判断准确度,并支持逐条标注和开关本会话的自动介入。
|
|
46
|
+
- **配置与持久化**:各决策点可独立开关,阈值在插件详情页调整,写入 `~/.config/dsh-decision-layer/config.json` 并热生效。默认值与字段说明见 [ADR-0006](<docs/adr/0006-configurable-thresholds.md>)。也可直接手改该文件,但只能写合法值:越界、拼错字段名或类型不符会让整份配置校验失败,配置页读不出来,各决策点退回各自的安全默认(见 [ADR-0005](<docs/adr/0005-config-wholesale-replace-and-live-reload.md>))。决策日志与会话指标落盘 30 天,供回看和校准。
|
|
47
|
+
- **可插拔后端**:默认 `https://api.typesafe.ai`、`jev-latest`,URL、Key、model 可逐字段覆盖;仅支持同协议端点,未配置 Key 时不发请求。自定义 `http://` 地址需针对该 URL 确认明文传输风险,地址变更须重新确认。
|
|
48
|
+
|
|
49
|
+
## 隐私与后端
|
|
50
|
+
|
|
51
|
+
配置 Key 后,插件默认把裁决请求发往远程端点 `https://api.typesafe.ai`(模型 `jev-latest`)。裁决会携带具体上下文:
|
|
52
|
+
|
|
53
|
+
- 门控:工具名与脱敏后的命令 / 路径摘要(各最多 2048 字符)。
|
|
54
|
+
- 产出自检:本轮最终回复(最多约 24K 字符)。
|
|
55
|
+
- 工具收窄:本步 user 消息的文本(最多约 8K 字符)与候选工具的名称 / 描述。
|
|
56
|
+
- 完成核对:从本回合 user 消息中提取的交付条件(最多 20 条)、工具名与成败状态(不含参数和输出)、回复前约 8K 字符。
|
|
57
|
+
- 手动裁决:`decision_evaluate` 与 `decision-layer` skill 会把 agent 提交的内容原样发送。
|
|
58
|
+
|
|
59
|
+
user 消息的文本里可能含有宿主注入的上下文,例如工作区指令(AGENTS.md)、运行时信息和记忆片段。这些内容会离开本机,远程服务提供方可能记录、留存或用于其他用途,具体取决于其服务条款;脱敏和截断只降低风险,不等于匿名化。
|
|
60
|
+
|
|
61
|
+
本地还会落盘两类数据,各保留 30 天:决策日志在 `~/.config/dsh-decision-layer/logs/`,会话快照在 `~/.config/dsh-decision-layer/state/`,其中包含脱敏后的命令 / 路径和请求主题摘要,可按需自行清理。
|
|
62
|
+
|
|
63
|
+
**会话内容敏感时,推荐改用本地部署的模型**,让裁决数据不出本机。`tev1` 可经 ollama 原生的 `/v1/systemone` 端点直接接入,部署步骤见 [用本地 tev1 配合插件(部署指南)](<docs/zh/deploy-tev1-with-plugin.md>);其他本地模型只要提供兼容 `/v1/systemone` 的端点,同样可以在配置页把服务地址指向本机。
|
|
50
64
|
|
|
51
65
|
## 安全边界
|
|
52
66
|
|
|
53
|
-
自动介入只减不增:门控只拒绝、收窄只删工具,都不给 agent 新权限。危险门控采用两档模型,只在模型高置信判定拒绝时拦截;后端失败、低置信或响应不合法时一律不介入、按原路径执行,不因裁决故障阻断,也不替宿主放行。自检、收窄与防循环异常则跳过介入。发送到后端的危险摘要只含工具名与命令 /
|
|
67
|
+
自动介入只减不增:门控只拒绝、收窄只删工具,都不给 agent 新权限。危险门控采用两档模型,只在模型高置信判定拒绝时拦截;后端失败、低置信或响应不合法时一律不介入、按原路径执行,不因裁决故障阻断,也不替宿主放行。自检、收窄与防循环异常则跳过介入。发送到后端的危险摘要只含工具名与命令 / 路径并对疑似密钥脱敏,插件不读取文件内容;但命令本身(如 heredoc、echo 写入的文本)会原样计入摘要。
|
|
54
68
|
|
|
55
69
|
## 开发
|
|
56
70
|
|
|
57
|
-
服务端使用 Node.js `>=20.11`(`.mjs`,无需构建);客户端用 TypeScript + esbuild 构建。`npm test` 跑本地测试,`npm run build:client` 生成 `lib/client.js
|
|
71
|
+
服务端使用 Node.js `>=20.11`(`.mjs`,无需构建);客户端用 TypeScript + esbuild 构建。`npm test` 跑本地测试,`npm run build:client` 生成 `lib/client.js`(`tsc` 严格模式就在这一步),`npm run check` 跑完两者。
|
|
72
|
+
|
|
73
|
+
**构建产物不入库**([ADR-0012](<docs/adr/0012-distribution-via-github-actions.md>)):`lib/` 已被 `.gitignore` 覆盖,源码是唯一事实来源。发布由 GitHub Actions 的 `release.yml` 完成——构建、跑门禁、`npm pack`,并用 `scripts/assert-pack.mjs` 断言产物里确实有 `lib/client.js` 且没有源码泄漏,之后才发 npm / 建 Release。该 workflow 在版本号变更推入 main 时自动执行;npm 侧使用 Trusted Publishing(OIDC),不依赖长期 token。
|
|
74
|
+
|
|
75
|
+
架构决策记录见 [docs/adr/](<docs/adr/README.md>):每条规则为什么存在、当时放弃了哪些备选。0.7.1 为体验版:介入能力已可用,但实际价值与阈值仍在探索,欢迎反馈决策日志观察到的误判与体感。
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# Decision Layer
|
|
2
|
+
|
|
3
|
+
A structured set of explicitly scoped decisions for the DeepSeek Harness agent loop: whether a destructive call should be blocked, whether the current tool set is enough, and whether this turn's output is complete are all judged by a pluggable adjudication model, while the host always owns the resulting behavior.
|
|
4
|
+
|
|
5
|
+
## Features
|
|
6
|
+
|
|
7
|
+
- **Dangerous-call gate** — before a destructive tool call, a redacted summary asks the backend for a verdict; only a high-confidence deny blocks the call, while backend failures, low confidence, and invalid responses fall back to the host's original path.
|
|
8
|
+
- **Output self-check and quality trend** — scores the final reply at turn end and can append one re-prompt when the score is low and the model is confident; several consecutive confident low scores raise an alert on the trigger button.
|
|
9
|
+
- **Task-completion review** — when the user lists multiple delivery conditions, each is judged satisfied / unsatisfied / insufficient evidence, with tool evidence primary and self-reported completion not counting.
|
|
10
|
+
- **Tool narrowing** — at turn start, judges each optional tool's relevance to the task and drops the ones not needed, intersecting the host-allowed set and never expanding it.
|
|
11
|
+
- **Loop guard** — denies the same call repeated consecutively past a threshold; deterministic and backend-free.
|
|
12
|
+
- **Decision panel and analysis view** — shows automatic-decision metrics and the decision log, and lets you rate each decision correct / wrong / unsure; the analysis view's header can enable or disable automatic interventions for the current session.
|
|
13
|
+
|
|
14
|
+
The backend, per-decision-point switches, and intervention thresholds are configurable; defaults remain uncalibrated heuristics.
|
|
15
|
+
|
|
16
|
+
## Repository
|
|
17
|
+
|
|
18
|
+
<https://github.com/xbzbing/dsh-decision-layer>
|
|
19
|
+
|
|
20
|
+
Installation and privacy notes live in the repository README; threshold defaults and field names are in `docs/adr/0006-configurable-thresholds.md`, and design trade-offs in `docs/adr/`.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# 决策层
|
|
2
|
+
|
|
3
|
+
为 DeepSeek Harness 的 agent 循环提供一组边界清晰的结构化裁决:危险调用要不要拦、当前工具够不够用、这轮产出有没有遗漏,都由可插拔的裁决模型给出建议,具体行为始终由宿主决定。
|
|
4
|
+
|
|
5
|
+
## 功能
|
|
6
|
+
|
|
7
|
+
- **危险调用门控**:破坏性工具调用执行前,用脱敏摘要请后端裁决;只有高置信「拒绝」才拦截,后端失败、低置信或响应不合法时一律按宿主原路径执行。
|
|
8
|
+
- **产出自检与质量趋势**:回合结束给最终回复打分,低分且模型有把握时可自动补一轮;连续多次高置信低分会在触发按钮上给出告警。
|
|
9
|
+
- **任务完成核对**:用户明确列出多个交付条件时,逐项判定「已满足 / 未满足 / 证据不足」,以工具证据为准,回复自述不算证据。
|
|
10
|
+
- **工具收窄**:回合开始时判断各可选工具与本轮任务的相关性,摘掉用不上的;只在宿主已允许的工具集内取交集,绝不扩权。
|
|
11
|
+
- **防循环**:同一调用连续重复超过阈值即拒绝,确定性本地逻辑,不依赖后端。
|
|
12
|
+
- **决策面板与分析页**:展示自动裁决指标与决策日志,并可逐条标注「判对 / 判错 / 说不准」;分析页顶部可直接开关本会话的自动介入。
|
|
13
|
+
|
|
14
|
+
裁决后端、各决策点开关与介入阈值均可配置;默认值仍是未标定的启发式。
|
|
15
|
+
|
|
16
|
+
## 仓库
|
|
17
|
+
|
|
18
|
+
<https://github.com/xbzbing/dsh-decision-layer>
|
|
19
|
+
|
|
20
|
+
安装与隐私说明见仓库内的 README;各阈值的默认值与字段说明见 `docs/adr/0006-configurable-thresholds.md`,设计取舍见 `docs/adr/`。
|