pi-verdict 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/README.zh-CN.md +3 -3
- package/extensions/pi-verdict.ts +112 -85
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -102,8 +102,8 @@ pi-verdict runs on both [pi](https://github.com/badlogic/pi-mono) and [oh-my-pi]
|
|
|
102
102
|
"toggleShortcut": "ctrl+shift+a",
|
|
103
103
|
"audit": false,
|
|
104
104
|
"notifyAllows": false,
|
|
105
|
+
"classifierMinConfidence": null,
|
|
105
106
|
"classifierFallbackModel": null,
|
|
106
|
-
"classifierFallbackConfidence": 50,
|
|
107
107
|
"classifierFallbackMode": "shadow"
|
|
108
108
|
}
|
|
109
109
|
```
|
|
@@ -115,7 +115,7 @@ pi-verdict runs on both [pi](https://github.com/badlogic/pi-mono) and [oh-my-pi]
|
|
|
115
115
|
- `classifierModel: "typesafe/jev-latest"` opts into the bundled **jev decisions adapter** — gray-zone verdicts via TypeSafe's jev (OpenRouter by default, or TypeSafe's official API directly with `PI_VERDICT_JEV_TRANSPORT=typesafe`); experimental, see [ADR-0003](docs/adr/0003-jev-decisions-adapter.md)
|
|
116
116
|
- `audit: true` records every **gray-zone adjudication** (the full transcript sent to the classifier, its raw response, the parsed verdict) as JSONL under `~/.pi/agent/verdicts/<sessionId>.jsonl` — one file per session, the 20 most recent kept. Interactive asks also record your answer (`userAnswer` ground truth, written after the confirm resolves), and protected-path asks are recorded too (#62); rule allow/deny stays unaudited. Local-only and full-fidelity (protected-path plaintext may appear — it never leaves your machine; [ADR-0002](docs/adr/0002-deny-paths-deterministic-ask.md) boundary note); the agent can neither read nor write the directory. `/automode` shows the audit state and path while on
|
|
117
117
|
- `notifyAllows: true` notifies on every **classifier allow** (reason + action line — e.g. jev's probability breakdown); default `false` keeps passes silent. Mechanical passes (your own allow rules, protected-path confirms) never notify; shadow-cache annotations stay debug-only; with both switches on the notification appears once
|
|
118
|
-
- `
|
|
118
|
+
- `classifierMinConfidence` (optional, [ADR-0004](docs/adr/0004-classifier-fallback-cascade.md)) sets the **confidence floor**: a jev verdict below it is demoted — cascaded to `classifierFallbackModel` if set (`shadow` = the second layer records its opinion and you are asked; `enforce` = the second layer adjudicates, except a demoted deny can never be auto-allowed), otherwise asked of you directly. At/above the floor the first layer is autonomous. A natural pairing: jev first + a haiku-class fallback
|
|
119
119
|
|
|
120
120
|
No built-in allowlist — every "always allow" claim is yours ([why](docs/configuration.md#why-no-built-in-allowlist)). Full reference: [docs/configuration.md](docs/configuration.md).
|
|
121
121
|
|
|
@@ -134,7 +134,7 @@ No built-in allowlist — every "always allow" claim is yours ([why](docs/config
|
|
|
134
134
|
- **Hosts**: pi only. On omp the setting warns and falls back to the session model; and it must never be selected as the session model (no text generation — selecting it warns)
|
|
135
135
|
- **Escape hatch**: `PI_VERDICT_JEV_URL` overrides the active transport's endpoint (OpenRouter's is an alpha API)
|
|
136
136
|
|
|
137
|
-
jev's calibrated confidence is exactly what the
|
|
137
|
+
jev's calibrated confidence is exactly what the confidence floor keys on — pair it with a second layer (`"classifierMinConfidence": 50, "classifierFallbackModel": "anthropic/claude-haiku-4-5"`) so its low-confidence calls go to a deeper model instead of standing ([ADR-0004](docs/adr/0004-classifier-fallback-cascade.md)).
|
|
138
138
|
|
|
139
139
|
### Self-protection (the gate guards itself — [ADR-0001](docs/adr/0001-self-protection-layer.md))
|
|
140
140
|
|
package/README.zh-CN.md
CHANGED
|
@@ -104,8 +104,8 @@ pi-verdict 同时支持 [pi](https://github.com/badlogic/pi-mono) 与 [oh-my-pi]
|
|
|
104
104
|
"toggleShortcut": "ctrl+shift+a",
|
|
105
105
|
"audit": false,
|
|
106
106
|
"notifyAllows": false,
|
|
107
|
+
"classifierMinConfidence": null,
|
|
107
108
|
"classifierFallbackModel": null,
|
|
108
|
-
"classifierFallbackConfidence": 50,
|
|
109
109
|
"classifierFallbackMode": "shadow"
|
|
110
110
|
}
|
|
111
111
|
```
|
|
@@ -117,7 +117,7 @@ pi-verdict 同时支持 [pi](https://github.com/badlogic/pi-mono) 与 [oh-my-pi]
|
|
|
117
117
|
- `classifierModel: "typesafe/jev-latest"` 启用随包的 **jev 决策适配器**——灰区裁决经 TypeSafe jev 完成(默认 OpenRouter,或 `PI_VERDICT_JEV_TRANSPORT=typesafe` 直连官方 API);实验性质,详见 [ADR-0003](docs/adr/0003-jev-decisions-adapter.md)
|
|
118
118
|
- `audit: true` 把每次**灰区裁决**(发给分类器的完整转录、其原始响应、解析出的裁决)以 JSONL 记录到 `~/.pi/agent/verdicts/<sessionId>.jsonl`——按会话一分文件,保留最近 20 个。交互式 ask 还会记录你的应答(`userAnswer` ground truth,确认结束后落盘),protected-path ask 也入审计(#62);规则 allow/deny 仍不入。仅存本机且全保真(受保护路径明文可能出现——永不出本机;[ADR-0002](docs/adr/0002-deny-paths-deterministic-ask.md) 边界注);agent 对该目录读写双拒。开启时 `/automode` 会显示审计状态与路径
|
|
119
119
|
- `notifyAllows: true` 对每次 **classifier 放行**发通知(reason + action 行——如 jev 的概率分解);默认 `false` 保持放行静默。机械放行(你自己的 allow 规则、protected-path 确认)永不通知;shadow 标注仍属 debug;两开关同开时通知只出现一次
|
|
120
|
-
- `
|
|
120
|
+
- `classifierMinConfidence`(可选,[ADR-0004](docs/adr/0004-classifier-fallback-cascade.md))设定**置信地板**:低于它的 jev 裁决被降级——配置了 `classifierFallbackModel` 则级联(`shadow` = 第二层只记录意见、由你裁决;`enforce` = 第二层全权裁决,但降级 deny 永不被自动翻成 allow),否则直接问你。不低于地板时第一层自主。天然搭配:jev 打头 + haiku 级兜底
|
|
121
121
|
|
|
122
122
|
没有内置白名单——每一条「永远放行」声明都归你([为什么](docs/configuration.md#why-no-built-in-allowlist))。完整参考:[docs/configuration.md](docs/configuration.md)。
|
|
123
123
|
|
|
@@ -136,7 +136,7 @@ pi-verdict 同时支持 [pi](https://github.com/badlogic/pi-mono) 与 [oh-my-pi]
|
|
|
136
136
|
- **宿主**:仅支持pi。omp 上该设置会警告并回退会话模型。也绝不能选作会话主模型(不生成文本,选中即警告)
|
|
137
137
|
- **逃生口**:`PI_VERDICT_JEV_URL` 可覆盖当前 transport 的端点(OpenRouter 侧为 alpha 接口)
|
|
138
138
|
|
|
139
|
-
jev 的校准 confidence
|
|
139
|
+
jev 的校准 confidence 正是置信地板的判定依据——搭配第二层使用(`"classifierMinConfidence": 50, "classifierFallbackModel": "anthropic/claude-haiku-4-5"`),让低置信调用交给更深的模型而非直接生效([ADR-0004](docs/adr/0004-classifier-fallback-cascade.md))。
|
|
140
140
|
|
|
141
141
|
### 自保护(门禁守护自身——[ADR-0001](docs/adr/0001-self-protection-layer.md))
|
|
142
142
|
|
package/extensions/pi-verdict.ts
CHANGED
|
@@ -264,15 +264,19 @@ interface UserRules {
|
|
|
264
264
|
audit: boolean;
|
|
265
265
|
/** Allow visibility (#60): info notification on classifier allows; mechanical passes stay silent. Default off. */
|
|
266
266
|
notifyAllows: boolean;
|
|
267
|
-
/** #
|
|
267
|
+
/** #67: autonomy floor for the first layer — a jev verdict with confidence strictly
|
|
268
|
+
* below this is demoted (cascaded to the fallback if configured, else asked of the
|
|
269
|
+
* user; non-interactive degrades to deny). null = floor off. */
|
|
270
|
+
classifierMinConfidence: number | null;
|
|
271
|
+
/** #63/#67: second-layer model spec (provider/id[:thinking]); consulted on demotion
|
|
272
|
+
* and fail-closed only. null = no second layer. */
|
|
268
273
|
classifierFallbackModel: string | null;
|
|
269
|
-
/** #
|
|
270
|
-
|
|
271
|
-
/** #63: "shadow" (default — observe-only, verdicts unchanged) | "enforce" (safety ratchet: the fallback may only escalate strictness, never relax) */
|
|
274
|
+
/** #67: does the second layer adjudicate cascaded calls ("enforce") or only record its
|
|
275
|
+
* opinion while the human decides ("shadow", default)? */
|
|
272
276
|
classifierFallbackMode: "shadow" | "enforce";
|
|
273
277
|
}
|
|
274
278
|
|
|
275
|
-
const EMPTY_RULES: UserRules = { allow: [], deny: [], denyPaths: [], builtinDenyFloor: true, classifierModel: null, toggleShortcut: DEFAULT_TOGGLE_SHORTCUT, audit: false, notifyAllows: false,
|
|
279
|
+
const EMPTY_RULES: UserRules = { allow: [], deny: [], denyPaths: [], builtinDenyFloor: true, classifierModel: null, toggleShortcut: DEFAULT_TOGGLE_SHORTCUT, audit: false, notifyAllows: false, classifierMinConfidence: null, classifierFallbackModel: null, classifierFallbackMode: "shadow" };
|
|
276
280
|
|
|
277
281
|
/** This module's own file location (import.meta.url resolved; null = unresolvable). */
|
|
278
282
|
const OWN_FILE_PATH: string | null = (() => {
|
|
@@ -337,8 +341,8 @@ const USER_CONFIG_TEMPLATE = `${JSON.stringify({
|
|
|
337
341
|
toggleShortcut: DEFAULT_TOGGLE_SHORTCUT,
|
|
338
342
|
audit: false,
|
|
339
343
|
notifyAllows: false,
|
|
344
|
+
classifierMinConfidence: null,
|
|
340
345
|
classifierFallbackModel: null,
|
|
341
|
-
classifierFallbackConfidence: 50,
|
|
342
346
|
classifierFallbackMode: "shadow",
|
|
343
347
|
}, null, 2)}\n`;
|
|
344
348
|
|
|
@@ -357,7 +361,7 @@ function loadUserRules(): { rules: UserRules; skipped: string[]; shortcutWarning
|
|
|
357
361
|
} catch { /* 只读环境静默跳过 */ }
|
|
358
362
|
return { rules: EMPTY_RULES, skipped: [], shortcutWarning: null };
|
|
359
363
|
}
|
|
360
|
-
let raw: { allow?: unknown; deny?: unknown; denyPaths?: unknown; builtinDenyFloor?: unknown; classifierModel?: unknown; toggleShortcut?: unknown; audit?: unknown; notifyAllows?: unknown; classifierFallbackModel?: unknown; classifierFallbackConfidence?: unknown; classifierFallbackMode?: unknown };
|
|
364
|
+
let raw: { allow?: unknown; deny?: unknown; denyPaths?: unknown; builtinDenyFloor?: unknown; classifierModel?: unknown; toggleShortcut?: unknown; audit?: unknown; notifyAllows?: unknown; classifierFallbackModel?: unknown; classifierFallbackConfidence?: unknown; classifierMinConfidence?: unknown; classifierFallbackMode?: unknown };
|
|
361
365
|
try {
|
|
362
366
|
raw = JSON.parse(fs.readFileSync(p, "utf8")) as typeof raw;
|
|
363
367
|
} catch (err) {
|
|
@@ -386,10 +390,11 @@ function loadUserRules(): { rules: UserRules; skipped: string[]; shortcutWarning
|
|
|
386
390
|
return [x.trim()];
|
|
387
391
|
});
|
|
388
392
|
const shortcut = resolveToggleShortcut(raw.toggleShortcut);
|
|
389
|
-
// #63:
|
|
390
|
-
|
|
391
|
-
const
|
|
392
|
-
|
|
393
|
+
// #63/#67: confidence-floor keys — invalid values skip into the one-shot warning channel
|
|
394
|
+
if (raw.classifierFallbackConfidence !== undefined) skipped.push("classifierFallbackConfidence: renamed to classifierMinConfidence (0.11.0) — key ignored");
|
|
395
|
+
const minConfRaw = raw.classifierMinConfidence;
|
|
396
|
+
const minConfOk = typeof minConfRaw === "number" && Number.isFinite(minConfRaw) && minConfRaw >= 0 && minConfRaw <= 100;
|
|
397
|
+
if (minConfRaw !== undefined && minConfRaw !== null && !minConfOk) skipped.push(`classifierMinConfidence: ${JSON.stringify(minConfRaw)}`);
|
|
393
398
|
const fbModeRaw = raw.classifierFallbackMode;
|
|
394
399
|
if (fbModeRaw !== undefined && fbModeRaw !== "shadow" && fbModeRaw !== "enforce") skipped.push(`classifierFallbackMode: ${JSON.stringify(fbModeRaw)}`);
|
|
395
400
|
return {
|
|
@@ -403,7 +408,7 @@ function loadUserRules(): { rules: UserRules; skipped: string[]; shortcutWarning
|
|
|
403
408
|
audit: raw.audit === true,
|
|
404
409
|
notifyAllows: raw.notifyAllows === true,
|
|
405
410
|
classifierFallbackModel: typeof raw.classifierFallbackModel === "string" && raw.classifierFallbackModel.trim() ? raw.classifierFallbackModel.trim() : null,
|
|
406
|
-
|
|
411
|
+
classifierMinConfidence: minConfOk ? minConfRaw : null,
|
|
407
412
|
classifierFallbackMode: fbModeRaw === "enforce" ? "enforce" : "shadow",
|
|
408
413
|
},
|
|
409
414
|
skipped,
|
|
@@ -1385,42 +1390,40 @@ function shadowTag(probe: ShadowProbe): string {
|
|
|
1385
1390
|
}
|
|
1386
1391
|
|
|
1387
1392
|
// ============================================================================
|
|
1388
|
-
//
|
|
1393
|
+
// Confidence cascade stats (#63/#67: observe-first, session-memory state; the #7 discipline)
|
|
1389
1394
|
// ============================================================================
|
|
1390
1395
|
|
|
1391
|
-
/** #63: ratchet strictness order — the fallback may only escalate, never relax */
|
|
1392
|
-
const STRICTNESS_RANK: Record<"allow" | "ask" | "deny", number> = { allow: 0, ask: 1, deny: 2 };
|
|
1393
|
-
|
|
1394
1396
|
interface FallbackStats {
|
|
1395
|
-
triggered: number; // the
|
|
1396
|
-
agreed: number; // fallback verdict
|
|
1397
|
-
|
|
1397
|
+
triggered: number; // the floor fired or the first layer fail-closed (with a fallback configured)
|
|
1398
|
+
agreed: number; // fallback verdict equals the first layer's (fail-closed defaults to deny)
|
|
1399
|
+
overruled: number; // fallback verdict differs (enforce applies it; shadow observes the would-be)
|
|
1398
1400
|
errored: number; // fallback unresolvable or its call failed
|
|
1399
1401
|
}
|
|
1400
1402
|
|
|
1401
1403
|
class FallbackCascade {
|
|
1402
|
-
readonly stats: FallbackStats = { triggered: 0, agreed: 0,
|
|
1404
|
+
readonly stats: FallbackStats = { triggered: 0, agreed: 0, overruled: 0, errored: 0 };
|
|
1403
1405
|
|
|
1404
1406
|
/** Session reset (#7 discipline: session-memory state) */
|
|
1405
1407
|
reset(): void {
|
|
1406
|
-
Object.assign(this.stats, { triggered: 0, agreed: 0,
|
|
1408
|
+
Object.assign(this.stats, { triggered: 0, agreed: 0, overruled: 0, errored: 0 });
|
|
1407
1409
|
}
|
|
1408
1410
|
|
|
1409
|
-
note(first: "allow" | "ask" | "deny", fb: "allow" | "ask" | "deny" | null): void {
|
|
1411
|
+
note(first: "allow" | "ask" | "deny" | null, fb: "allow" | "ask" | "deny" | null): void {
|
|
1410
1412
|
this.stats.triggered++;
|
|
1411
1413
|
if (fb === null) {
|
|
1412
1414
|
this.stats.errored++;
|
|
1413
1415
|
return;
|
|
1414
1416
|
}
|
|
1415
|
-
|
|
1417
|
+
// A fail-closed origin produced no first-layer verdict; its default outcome is deny
|
|
1418
|
+
if ((first ?? "deny") !== fb) this.stats.overruled++;
|
|
1416
1419
|
else this.stats.agreed++;
|
|
1417
1420
|
}
|
|
1418
1421
|
|
|
1419
1422
|
/** Summary line for /automode */
|
|
1420
1423
|
summary(mode: "shadow" | "enforce"): string {
|
|
1421
1424
|
const s = this.stats;
|
|
1422
|
-
if (s.triggered === 0) return "
|
|
1423
|
-
return `
|
|
1425
|
+
if (s.triggered === 0) return "confidence cascade: not triggered this session";
|
|
1426
|
+
return `confidence cascade (${mode}): triggered ${s.triggered} · agreed ${s.agreed} · ${mode === "enforce" ? "overruled" : "would-overrule"} ${s.overruled} · errored ${s.errored}`;
|
|
1424
1427
|
}
|
|
1425
1428
|
}
|
|
1426
1429
|
|
|
@@ -1431,21 +1434,21 @@ class FallbackCascade {
|
|
|
1431
1434
|
|
|
1432
1435
|
const AUDIT_KEEP_SESSIONS = 20;
|
|
1433
1436
|
|
|
1434
|
-
/** #63: second-layer
|
|
1435
|
-
*
|
|
1436
|
-
*
|
|
1437
|
+
/** #63/#67: second-layer outcome on a cascaded call. The record's top-level fields keep
|
|
1438
|
+
* first-layer semantics for corpus comparability; the verdict actually applied under
|
|
1439
|
+
* enforce lives in `effective` (failure rows carry the ask the human got). */
|
|
1437
1440
|
export interface FallbackAudit {
|
|
1438
1441
|
model: string;
|
|
1439
1442
|
mode: "shadow" | "enforce";
|
|
1440
|
-
triggeredBy: "
|
|
1441
|
-
/** jev confidence that fired the
|
|
1443
|
+
triggeredBy: "confidence" | "fail-closed";
|
|
1444
|
+
/** jev confidence that fired the floor; null unless triggeredBy = "confidence" */
|
|
1442
1445
|
confidence: number | null;
|
|
1443
1446
|
/** null = the fallback call itself failed (unresolvable model, timeout, parse) */
|
|
1444
1447
|
verdict: "allow" | "ask" | "deny" | null;
|
|
1445
1448
|
reason: string | null;
|
|
1446
1449
|
durationMs: number;
|
|
1447
1450
|
error: string | null;
|
|
1448
|
-
/** enforce mode only: the verdict applied
|
|
1451
|
+
/** enforce mode only: the verdict applied (pre headless-degradation) */
|
|
1449
1452
|
effective?: "allow" | "ask" | "deny";
|
|
1450
1453
|
}
|
|
1451
1454
|
|
|
@@ -1479,7 +1482,9 @@ export interface AuditRecord {
|
|
|
1479
1482
|
answeredAt?: string;
|
|
1480
1483
|
/** #62: protected-path records only — the matched path. */
|
|
1481
1484
|
detail?: string;
|
|
1482
|
-
/** #
|
|
1485
|
+
/** #67: the confidence floor fired — the first-layer verdict was demoted. */
|
|
1486
|
+
demoted?: true;
|
|
1487
|
+
/** #63/#67: second-layer outcome when the fallback was consulted. */
|
|
1483
1488
|
fallback?: FallbackAudit;
|
|
1484
1489
|
}
|
|
1485
1490
|
|
|
@@ -1627,66 +1632,78 @@ export interface AdjudicateEnv {
|
|
|
1627
1632
|
getFallbackModel?: () => { model: NonNullable<ExtensionContext["model"]>; thinking: ThinkingLevel } | null;
|
|
1628
1633
|
}
|
|
1629
1634
|
|
|
1630
|
-
/** #
|
|
1631
|
-
*
|
|
1632
|
-
*
|
|
1633
|
-
function
|
|
1634
|
-
if (
|
|
1635
|
-
if (outcome.source === "fail-closed") return { triggeredBy: "fail-closed", confidence: null };
|
|
1636
|
-
if (outcome.verdict === "ask") return { triggeredBy: "ask", confidence: null };
|
|
1635
|
+
/** #67: the confidence floor. Below it the first layer abstains and the call cascades —
|
|
1636
|
+
* to the fallback if configured, else to the human (headless degrades to deny). Numeric
|
|
1637
|
+
* confidence exists only on jev-formatted reasons; LLM first layers never demote. */
|
|
1638
|
+
function confidenceDemotion(outcome: ClassifierOutcome, rules: UserRules): { confidence: number } | null {
|
|
1639
|
+
if (rules.classifierMinConfidence === null || outcome.source === "fail-closed") return null;
|
|
1637
1640
|
const conf = parseJevConfidence(outcome.reason);
|
|
1638
|
-
if (conf !== null && conf < rules.
|
|
1641
|
+
if (conf !== null && conf < rules.classifierMinConfidence) return { confidence: conf };
|
|
1639
1642
|
return null;
|
|
1640
1643
|
}
|
|
1641
1644
|
|
|
1642
1645
|
interface CascadeResult {
|
|
1643
|
-
/**
|
|
1646
|
+
/** set whenever the confidence floor fired (with or without a fallback) */
|
|
1647
|
+
demoted?: true;
|
|
1648
|
+
/** audit material; present when the fallback was consulted */
|
|
1644
1649
|
fb?: FallbackAudit;
|
|
1645
|
-
/**
|
|
1650
|
+
/** the applied outcome when the cascade changes it (pre-degradation — the caller's
|
|
1651
|
+
* tail applies the usual headless ask → deny rule) */
|
|
1646
1652
|
effective?: { verdict: "allow" | "ask" | "deny"; reason: string; source: "classifier" | "fail-closed" };
|
|
1647
1653
|
}
|
|
1648
1654
|
|
|
1649
|
-
/** #
|
|
1650
|
-
*
|
|
1651
|
-
*
|
|
1652
|
-
*
|
|
1653
|
-
*
|
|
1654
|
-
|
|
1655
|
+
/** #67: run the cascade for one triggered call. `first` is the first-layer verdict, or
|
|
1656
|
+
* null when the first layer never produced one (fail-closed origin). Semantics:
|
|
1657
|
+
* - demotion with no fallback → ask the human
|
|
1658
|
+
* - shadow → the fallback records its opinion; a demotion still asks the human, a
|
|
1659
|
+
* fail-closed deny stands
|
|
1660
|
+
* - enforce → the fallback adjudicates de novo, with one carve-out: a demoted first-layer
|
|
1661
|
+
* deny may not be flipped to an automatic allow — the human decides
|
|
1662
|
+
* - fallback failure/unresolvable on a cascaded call → ask the human (the tier that was
|
|
1663
|
+
* to adjudicate is down); headless degrades downstream */
|
|
1664
|
+
async function runConfidenceCascade(
|
|
1655
1665
|
state: SessionState,
|
|
1656
1666
|
env: AdjudicateEnv,
|
|
1657
|
-
first: "allow" | "ask" | "deny",
|
|
1658
|
-
trigger: {
|
|
1667
|
+
first: { verdict: "allow" | "ask" | "deny"; reason: string } | null,
|
|
1668
|
+
trigger: { kind: "demotion"; confidence: number } | { kind: "fail-closed" },
|
|
1659
1669
|
denyPathsActive: boolean,
|
|
1660
1670
|
actionLine: string,
|
|
1661
1671
|
): Promise<CascadeResult> {
|
|
1662
1672
|
const rules = state.userRules;
|
|
1663
|
-
|
|
1673
|
+
const demotionAsk = (): CascadeResult["effective"] => ({
|
|
1674
|
+
verdict: "ask",
|
|
1675
|
+
reason: `${first!.reason} (confidence ${trigger.kind === "demotion" ? trigger.confidence : "?"}% is below your classifierMinConfidence of ${rules.classifierMinConfidence}%)`,
|
|
1676
|
+
source: "classifier",
|
|
1677
|
+
});
|
|
1678
|
+
const getFb = env.getFallbackModel;
|
|
1679
|
+
if (!rules.classifierFallbackModel || !getFb) {
|
|
1680
|
+
// A fail-closed without a fallback keeps its deny; a demotion asks the human
|
|
1681
|
+
return trigger.kind === "demotion" ? { demoted: true, effective: demotionAsk() } : {};
|
|
1682
|
+
}
|
|
1664
1683
|
const mode = rules.classifierFallbackMode;
|
|
1665
1684
|
const start = Date.now();
|
|
1666
|
-
const base = { mode, triggeredBy: trigger.
|
|
1667
|
-
|
|
1668
|
-
|
|
1669
|
-
// so failure rows read through the same sub-object as every other enforce row
|
|
1685
|
+
const base = { mode, triggeredBy: trigger.kind === "demotion" ? ("confidence" as const) : ("fail-closed" as const), confidence: trigger.kind === "demotion" ? trigger.confidence : null };
|
|
1686
|
+
const demotedMark = trigger.kind === "demotion" ? ({ demoted: true } as const) : {};
|
|
1687
|
+
const shadowApplied = trigger.kind === "demotion" ? { effective: demotionAsk() } : {};
|
|
1670
1688
|
const failed = (model: string, error: string): CascadeResult => {
|
|
1671
|
-
state.fallback.note(first, null);
|
|
1689
|
+
state.fallback.note(first?.verdict ?? null, null);
|
|
1672
1690
|
const fb: FallbackAudit = { ...base, model, verdict: null, reason: null, durationMs: Date.now() - start, error };
|
|
1673
|
-
|
|
1691
|
+
if (mode === "shadow") return { ...demotedMark, fb, ...shadowApplied };
|
|
1692
|
+
return { ...demotedMark, fb: { ...fb, effective: "ask" }, effective: { verdict: "ask", reason: "fallback classifier unavailable (first layer abstained) — your call", source: "fail-closed" } };
|
|
1674
1693
|
};
|
|
1675
|
-
const resolved =
|
|
1694
|
+
const resolved = getFb();
|
|
1676
1695
|
if (!resolved) return failed(rules.classifierFallbackModel, "fallback model unresolvable (not found or no configured auth)");
|
|
1677
1696
|
const outcome = await classifyWithModel(env.host, env.signal, env.complete, resolved.model, actionLine, resolved.thinking, denyPathsActive, FALLBACK_TIMEOUT_MS);
|
|
1678
|
-
const durationMs = Date.now() - start;
|
|
1679
1697
|
if (outcome.source !== "model") return failed(resolved.model.id, outcome.reason);
|
|
1680
|
-
state.fallback.note(first, outcome.verdict);
|
|
1681
|
-
const fb: FallbackAudit = { ...base, model: resolved.model.id, verdict: outcome.verdict, reason: outcome.reason, durationMs, error: null };
|
|
1682
|
-
if (mode === "
|
|
1683
|
-
|
|
1684
|
-
|
|
1685
|
-
|
|
1686
|
-
}
|
|
1687
|
-
return { fb: { ...fb, effective } };
|
|
1698
|
+
state.fallback.note(first?.verdict ?? null, outcome.verdict);
|
|
1699
|
+
const fb: FallbackAudit = { ...base, model: resolved.model.id, verdict: outcome.verdict, reason: outcome.reason, durationMs: Date.now() - start, error: null };
|
|
1700
|
+
if (mode === "shadow") return { ...demotedMark, fb, ...shadowApplied };
|
|
1701
|
+
// The one carve-out on second-layer authority: a demoted first-layer deny may not
|
|
1702
|
+
// become an automatic allow — the human decides (headless degrades to deny downstream)
|
|
1703
|
+
if (trigger.kind === "demotion" && first?.verdict === "deny" && outcome.verdict === "allow") {
|
|
1704
|
+
return { demoted: true, fb: { ...fb, effective: "ask" }, effective: { verdict: "ask", reason: `${outcome.reason} (first layer said deny at confidence ${trigger.confidence}%; second opinion allows — your call)`, source: "classifier" } };
|
|
1688
1705
|
}
|
|
1689
|
-
return { fb };
|
|
1706
|
+
return { ...demotedMark, fb: { ...fb, effective: outcome.verdict }, effective: { verdict: outcome.verdict, reason: outcome.reason, source: "classifier" } };
|
|
1690
1707
|
}
|
|
1691
1708
|
|
|
1692
1709
|
/**
|
|
@@ -1743,13 +1760,19 @@ export async function adjudicate(
|
|
|
1743
1760
|
const resolved = env.getModel();
|
|
1744
1761
|
if (!resolved) {
|
|
1745
1762
|
const reason = "no classifier model available (fail-closed)";
|
|
1746
|
-
// #
|
|
1747
|
-
//
|
|
1748
|
-
//
|
|
1749
|
-
const cascade = await
|
|
1763
|
+
// #67: a fail-closed origin cascades to the fallback if configured — under enforce
|
|
1764
|
+
// the fallback adjudicates de novo (superseding the 0.10.0 ratchet decision);
|
|
1765
|
+
// shadow records its opinion and the deny stands
|
|
1766
|
+
const cascade = await runConfidenceCascade(state, env, null, { kind: "fail-closed" }, state.userRules.denyPaths.length > 0, actionLine);
|
|
1767
|
+
const eff = cascade.effective;
|
|
1750
1768
|
const fcRecord = buildRecord({ verdict: "deny", reason, source: "fail-closed", degraded: false }, null, "-");
|
|
1751
1769
|
if (cascade.fb) fcRecord.fallback = cascade.fb;
|
|
1770
|
+
if (eff?.verdict === "ask" && env.hasUI) {
|
|
1771
|
+
return { verdict: "ask", reason: eff.reason, source: eff.source, degraded: false, ...(state.audit ? { pendingAudit: fcRecord } : {}) };
|
|
1772
|
+
}
|
|
1752
1773
|
state.audit?.append(fcRecord);
|
|
1774
|
+
if (eff?.verdict === "allow") return { verdict: "allow", reason: eff.reason, source: "classifier", degraded: false };
|
|
1775
|
+
if (eff) return { verdict: "deny", reason: eff.reason, source: eff.source, degraded: !env.hasUI };
|
|
1753
1776
|
return { verdict: "deny", reason, source: "fail-closed", degraded: false };
|
|
1754
1777
|
}
|
|
1755
1778
|
|
|
@@ -1769,22 +1792,26 @@ export async function adjudicate(
|
|
|
1769
1792
|
|
|
1770
1793
|
const shadow = shadowTag(probe);
|
|
1771
1794
|
|
|
1772
|
-
// #
|
|
1773
|
-
//
|
|
1774
|
-
const
|
|
1775
|
-
const cascade =
|
|
1795
|
+
// #67 cascade: a confidence-floor demotion, or a classifier fail-closed outcome
|
|
1796
|
+
// (the first layer produced no verdict)
|
|
1797
|
+
const demotion = confidenceDemotion(outcome, state.userRules);
|
|
1798
|
+
const cascade = demotion || outcome.source === "fail-closed"
|
|
1799
|
+
? await runConfidenceCascade(state, env, demotion ? { verdict: outcome.verdict, reason: outcome.reason } : null, demotion ? { kind: "demotion", confidence: demotion.confidence } : { kind: "fail-closed" }, state.userRules.denyPaths.length > 0, actionLine)
|
|
1800
|
+
: {};
|
|
1776
1801
|
const effVerdict = cascade.effective?.verdict ?? outcome.verdict;
|
|
1777
1802
|
const effReason = cascade.effective?.reason ?? outcome.reason;
|
|
1778
1803
|
const effSource = cascade.effective?.source ?? "classifier";
|
|
1779
1804
|
|
|
1780
|
-
// #62:
|
|
1781
|
-
//
|
|
1782
|
-
//
|
|
1783
|
-
|
|
1784
|
-
const
|
|
1805
|
+
// #62/#67: top-level keeps first-layer semantics (corpus comparability); the applied
|
|
1806
|
+
// verdict lives in fallback.effective (enforce rows). Non-interactive asks of any
|
|
1807
|
+
// origin — native, demoted, escalated — record as their effective deny, the
|
|
1808
|
+
// pre-existing ask-degradation convention.
|
|
1809
|
+
const appliedAskHeadless = !env.hasUI && effVerdict === "ask";
|
|
1810
|
+
const grayRecord = buildRecord({ verdict: appliedAskHeadless ? "deny" : outcome.verdict, reason: outcome.reason, source: outcome.source, degraded: appliedAskHeadless }, outcome.auditRaw ?? null, shadow);
|
|
1811
|
+
if (cascade.demoted) grayRecord.demoted = true;
|
|
1785
1812
|
if (cascade.fb) grayRecord.fallback = cascade.fb;
|
|
1786
1813
|
// #62: an interactive ask defers the append to the handler finalize (ground truth);
|
|
1787
|
-
//
|
|
1814
|
+
// every other outcome appends immediately as before
|
|
1788
1815
|
if (effVerdict === "ask" && env.hasUI) {
|
|
1789
1816
|
return { verdict: "ask", reason: effReason, source: effSource, degraded: false, shadow, ...(state.audit ? { pendingAudit: grayRecord } : {}) };
|
|
1790
1817
|
}
|
|
@@ -1792,7 +1819,7 @@ export async function adjudicate(
|
|
|
1792
1819
|
if (effVerdict === "allow") return { verdict: "allow", reason: effReason, source: effSource, degraded: false, shadow };
|
|
1793
1820
|
if (effVerdict === "deny") return { verdict: "deny", reason: effReason, source: effSource, degraded: false, shadow };
|
|
1794
1821
|
// ask:无 UI 降级为 deny(ask 降级,CONTEXT.md 词条)
|
|
1795
|
-
return { verdict: "deny", reason: effReason, source: effSource, degraded:
|
|
1822
|
+
return { verdict: "deny", reason: effReason, source: effSource, degraded: true, shadow };
|
|
1796
1823
|
}
|
|
1797
1824
|
|
|
1798
1825
|
// ============================================================================
|
|
@@ -1923,8 +1950,8 @@ export default function autoMode(pi: ExtensionAPI, deps: AutoModeDeps = {}) {
|
|
|
1923
1950
|
const denyPathsHint = () => (state.userRules.denyPaths.length > 0 ? `\ndenyPaths: ${state.userRules.denyPaths.length} active` : "");
|
|
1924
1951
|
/** Status line audit hint (#54): shown only while the sink is active */
|
|
1925
1952
|
const auditHint = () => (state.audit ? `\naudit: on → ${state.audit.dir}` : "");
|
|
1926
|
-
/** Status line
|
|
1927
|
-
const fallbackHint = () => (state.userRules.classifierFallbackModel ? `\n${state.fallback.summary(state.userRules.classifierFallbackMode)}` : "");
|
|
1953
|
+
/** Status line cascade hint (#63/#67): shown while the floor or the fallback is configured */
|
|
1954
|
+
const fallbackHint = () => (state.userRules.classifierMinConfidence !== null || state.userRules.classifierFallbackModel ? `\n${state.fallback.summary(state.userRules.classifierFallbackMode)}` : "");
|
|
1928
1955
|
|
|
1929
1956
|
pi.registerCommand("automode", {
|
|
1930
1957
|
description: "Show Auto Mode status and shadow-cache stats, or set it: /automode on|off",
|