@xaccefy/pi-casefile 0.9.0 → 0.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -6
- package/package.json +13 -4
- package/skills/casefile/SKILL.md +3 -3
- package/src/evidence.ts +501 -0
- package/src/harness-verify.ts +693 -0
- package/src/index.ts +202 -94
- package/src/ledger-worker-entry.ts +35 -0
- package/src/ledger-worker.ts +77 -0
- package/src/ledger.ts +491 -59
- package/src/pipeline-submit.ts +88 -33
- package/src/poc-runner.ts +67 -20
- package/src/safe-state.ts +108 -0
- package/src/scratchpad.ts +39 -24
- package/src/workflow.ts +25 -20
package/README.md
CHANGED
|
@@ -32,6 +32,9 @@ State is persisted next to the ledger as `xp-mode` (e.g. `.pi/xp-mode`).
|
|
|
32
32
|
|----------|---------|
|
|
33
33
|
| `PI_CASEFILE_PATH` | Absolute path to the SQLite ledger file |
|
|
34
34
|
| `CASEFILE_WORKSPACE_ROOT` / `PI_WORKSPACE_ROOT` | Override workspace root used to place `.pi/casefile.db` |
|
|
35
|
+
| `PI_POC_ALLOW_NETWORK=1` | Operator authorization for a networked PoC sandbox |
|
|
36
|
+
| `PI_POC_ALLOW_PRIVATE_REPLAY=1` | Operator authorization for harness replay to private/internal targets |
|
|
37
|
+
| `PI_POC_CONTROL_TARGETS` | Comma/newline-separated operator-approved control hosts/origins; agent-invented controls are rejected |
|
|
35
38
|
|
|
36
39
|
Default DB path: `<workspace>/.pi/casefile.db`
|
|
37
40
|
|
|
@@ -44,16 +47,16 @@ hypothesis → investigating → confirmed → reported
|
|
|
44
47
|
```
|
|
45
48
|
|
|
46
49
|
- **investigating** needs `evidence` + `confidence`
|
|
47
|
-
- **confirmed** only through the two-phase gate — `PromoteFinding`
|
|
48
|
-
- **Every promotion requires `control_path`** (the same bytes as the PoC — sha256 enforced) and a distinct `control_target
|
|
50
|
+
- **confirmed** only through the two-phase gate — `PromoteFinding` runs the PoC 2× target + 1× control, requires complete zero-exit runs and nonce-bound discriminating response-body evidence, then the harness performs a DNS-pinned identical replay and requires two conclusive responses with `target_only`. Reflection-capable requests may add a post-PoC harness-generated canary that must appear only on target. Status-only/trivial matchers and incomplete response capture are rejected. Exit zero is necessary but never proof. The main agent performs semantic review and calls `ConfirmFinding`, which captures a second fresh harness replay and binds it to the verdict; worker processes are rejected. Blind/OOB claims fail closed without a source-separated oracle.
|
|
51
|
+
- **Every promotion requires `control_path`** (the same bytes as the PoC — sha256 enforced) and a distinct `control_target` pre-approved by the operator in `PI_POC_CONTROL_TARGETS`; an agent cannot invent its own easy control. The control run is stored as `controlVerified`. Crashes, transport-inconclusive controls, status-only evidence, and missing evidence all block promotion.
|
|
49
52
|
- **New cases require `disproveIf`** — falsification conditions (what would disprove this hypothesis). A hypothesis that can't say what kills it isn't one yet.
|
|
50
|
-
- **A kill must be justified**: either an EvidenceAdd `refutation` item, or a kill-reason token (intended_behavior, duplicate, framework_protection, out_of_scope,
|
|
53
|
+
- **A kill must be justified**: either an EvidenceAdd `refutation` item, or a canonical kill-reason token (intended_behavior, duplicate, framework_protection, out_of_scope, insufficient_impact, no_attack_path, ...) in assumptions/nextStep. Bare `status: "killed"` is rejected.
|
|
51
54
|
- **reported** needs `CaseContext` first (records the report path; the report writer produces the final file)
|
|
52
55
|
- **killed** / **reported** are final (no more edits)
|
|
53
56
|
|
|
54
57
|
## Evidence items
|
|
55
58
|
|
|
56
|
-
`EvidenceAdd` records role-typed, artifact-backed evidence (observation / reproduction / impact / refutation / cleanup).
|
|
59
|
+
`EvidenceAdd` records role-typed, artifact-backed evidence (observation / reproduction / impact / refutation / cleanup). Artifact reads are restricted to regular, non-symlink files inside the workspace. Bytes are copied durably and stored as basename + SHA-256; the full source path is never persisted. The PoC gate auto-records the `reproduction` item at promotion.
|
|
57
60
|
|
|
58
61
|
## Tools
|
|
59
62
|
|
|
@@ -62,8 +65,8 @@ hypothesis → investigating → confirmed → reported
|
|
|
62
65
|
| `CaseAdd` | Open a case (`title` + `disproveIf` required; start as `hypothesis` or `investigating`) |
|
|
63
66
|
| `CaseUpdate` | Evidence, impact, severity, status (not direct confirm) |
|
|
64
67
|
| `EvidenceAdd` | Role-typed, hashed evidence item on a case (refutation justifies kills; cleanup tracks cleanup) |
|
|
65
|
-
| `PromoteFinding` | Phase 1:
|
|
66
|
-
| `ConfirmFinding` | Phase 2:
|
|
68
|
+
| `PromoteFinding` | Phase 1: PoC 2× target + 1× operator-approved control, then DNS-pinned harness-owned replay requiring conclusive `target_only`; optional reflection canary upgrades the recorded proof strength when observed only on target. `local:true` and private replay are operator-gated; blind/OOB proof fails closed without source separation |
|
|
69
|
+
| `ConfirmFinding` | Phase 2: main-agent-only semantic decision plus a fresh harness-owned target/control replay (CONFIRMED promotes; NOT_CONFIRMED keeps investigating; worker calls are rejected) |
|
|
67
70
|
| `CaseGet` / `CaseList` / `CaseSearch` | Read / filter / search |
|
|
68
71
|
| `CaseLink` / `CaseUnlink` | Bidirectional exploit chains |
|
|
69
72
|
| `ChainSuggest` | Scan cases for exploitable chain combinations (credential+endpoint→ATO, redirect+OAuth→token theft, XSS+state-change→CSRF, IDOR+user-data, SSTI→RCE, race+payment, info-disclosure+SSRF), ranked by confidence |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@xaccefy/pi-casefile",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.1",
|
|
4
4
|
"description": "Offensive security case tracker for Pi Agent — bug bounties, CTFs, security audits",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
@@ -15,12 +15,12 @@
|
|
|
15
15
|
"license": "MIT",
|
|
16
16
|
"repository": {
|
|
17
17
|
"type": "git",
|
|
18
|
-
"url": "git+https://github.com/
|
|
18
|
+
"url": "git+https://github.com/xaccefy/pi-xpi.git",
|
|
19
19
|
"directory": "packages/pi-casefile"
|
|
20
20
|
},
|
|
21
|
-
"homepage": "https://github.com/
|
|
21
|
+
"homepage": "https://github.com/xaccefy/pi-xpi/tree/main/packages/pi-casefile#readme",
|
|
22
22
|
"bugs": {
|
|
23
|
-
"url": "https://github.com/
|
|
23
|
+
"url": "https://github.com/xaccefy/pi-xpi/issues"
|
|
24
24
|
},
|
|
25
25
|
"scripts": {
|
|
26
26
|
"test": "bun test test",
|
|
@@ -32,6 +32,11 @@
|
|
|
32
32
|
},
|
|
33
33
|
"files": [
|
|
34
34
|
"src/index.ts",
|
|
35
|
+
"src/evidence.ts",
|
|
36
|
+
"src/harness-verify.ts",
|
|
37
|
+
"src/safe-state.ts",
|
|
38
|
+
"src/ledger-worker.ts",
|
|
39
|
+
"src/ledger-worker-entry.ts",
|
|
35
40
|
"src/ledger.ts",
|
|
36
41
|
"src/workflow.ts",
|
|
37
42
|
"src/poc-runner.ts",
|
|
@@ -43,6 +48,10 @@
|
|
|
43
48
|
"LICENSE"
|
|
44
49
|
],
|
|
45
50
|
"main": "src/index.ts",
|
|
51
|
+
"dependencies": {
|
|
52
|
+
"@xaccefy/pi-shared": "0.9.1",
|
|
53
|
+
"undici": "^8.9.0"
|
|
54
|
+
},
|
|
46
55
|
"pi": {
|
|
47
56
|
"extensions": [
|
|
48
57
|
"./src/index.ts"
|
package/skills/casefile/SKILL.md
CHANGED
|
@@ -13,7 +13,7 @@ Use Casefile to maintain durable security investigation state across agent turns
|
|
|
13
13
|
1. Check existing cases before opening a new one with CaseList or CaseSearch.
|
|
14
14
|
2. Open new leads with CaseAdd as `hypothesis` or `investigating`.
|
|
15
15
|
3. Promote cases with CaseUpdate only after materially new evidence, proof, impact, blockers, remediation, or status changes.
|
|
16
|
-
4. Mark `confirmed` only via the two-phase gate — `PromoteFinding` (
|
|
16
|
+
4. Mark `confirmed` only via the two-phase gate — `PromoteFinding` (PoC 2× target + 1× operator-approved control, nonce-bound body evidence, then a DNS-pinned conclusive `target_only` replay; use the post-PoC harness-generated canary for reflection-capable requests) → the main agent personally reviews → `ConfirmFinding`, which captures a fresh second harness replay before commit. Never delegate phase 2; worker calls are rejected. Exit zero is run integrity, never vulnerability proof. A predicate differential is evidence, not an automatic exploit verdict. Blind/OOB findings remain investigating without a source-separated oracle.
|
|
17
17
|
5. Use CaseLink and CaseUnlink for exploit chains. Do not edit linked case IDs directly.
|
|
18
18
|
6. Use CaseContext only for confirmed or already reported cases: it writes the full context bundle (complete record, verification logs, links, pipeline artifacts) and records the report path. Then have the report written (reporter agent in the full pipeline; yourself in lite mode) and CaseUpdate status=`reported`.
|
|
19
19
|
7. Use `killed` for disproven, duplicate, or dead-end leads, and include evidence, blockers, next step, or assumptions explaining why.
|
|
@@ -34,8 +34,8 @@ hypothesis → investigating → confirmed → reported
|
|
|
34
34
|
|
|
35
35
|
- `CaseAdd`: create a new case.
|
|
36
36
|
- `CaseUpdate`: update an existing case.
|
|
37
|
-
- `PromoteFinding`: phase 1
|
|
38
|
-
- `ConfirmFinding`: phase 2 —
|
|
37
|
+
- `PromoteFinding`: phase 1 — require a body predicate, bind the verify URL, pin DNS, and send the same request to target and an operator-approved `PI_POC_CONTROL_TARGETS` control. Both responses must be conclusive and only `target_only` passes. For reflection, place `{{PI_POC_CANARY}}` exactly once in the request and declare `verify.canary`; the harness creates the secret after the PoC exits and requires target-only reflection. Network/private access remains operator-gated; OOB fails closed without source separation.
|
|
38
|
+
- `ConfirmFinding`: phase 2 — main-agent-only review and fresh harness replay before commit (CONFIRMED promotes; NOT_CONFIRMED keeps investigating; worker/subagent calls are rejected). Record `canary_assessment=verified` when requested, otherwise `not_applicable` with a concrete reason.
|
|
39
39
|
- `CaseGet`: read one case by ID.
|
|
40
40
|
- `CaseList`: list cases with filters and pagination.
|
|
41
41
|
- `CaseSearch`: search all fields or a scoped field.
|
package/src/evidence.ts
ADDED
|
@@ -0,0 +1,501 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Evidence contract for PoC confirmation.
|
|
3
|
+
*
|
|
4
|
+
* A PoC must write `evidence.json` into $PI_POC_EVIDENCE_DIR: a nonce-bound,
|
|
5
|
+
* schema-validated record of what it claims and the request spec the harness
|
|
6
|
+
* executes against both target and control. The harness validates the file,
|
|
7
|
+
* binds it to the run via $PI_POC_NONCE, and acquires the responses itself.
|
|
8
|
+
* The main/coordinator agent remains the semantic reviewer after the machine differential.
|
|
9
|
+
*
|
|
10
|
+
* Exit zero is required run integrity, but never vulnerability proof.
|
|
11
|
+
*/
|
|
12
|
+
|
|
13
|
+
// ── PoC evidence (written by the PoC script) ────────────────────────
|
|
14
|
+
|
|
15
|
+
export type VerifyExpect = {
|
|
16
|
+
/** Acceptable HTTP statuses for the verify request. */
|
|
17
|
+
status?: number[];
|
|
18
|
+
/** Substrings the verify response must contain. */
|
|
19
|
+
body_contains?: string[];
|
|
20
|
+
/** Regexes the verify response must match. */
|
|
21
|
+
body_regex?: string[];
|
|
22
|
+
};
|
|
23
|
+
|
|
24
|
+
/** Replaced only inside the harness replay, after the PoC process has exited. */
|
|
25
|
+
export const POC_CANARY_PLACEHOLDER = "{{PI_POC_CANARY}}";
|
|
26
|
+
|
|
27
|
+
export type VerifyCanary = {
|
|
28
|
+
/** Reflection is machine-checked: target must return the fresh token and control must not. */
|
|
29
|
+
mode: "reflection";
|
|
30
|
+
/** Fixed literal; callers cannot choose or predict the harness-generated token. */
|
|
31
|
+
placeholder: typeof POC_CANARY_PLACEHOLDER;
|
|
32
|
+
};
|
|
33
|
+
|
|
34
|
+
export type PoCEvidence = {
|
|
35
|
+
/** Must equal the run's PI_POC_NONCE (harness-verified). */
|
|
36
|
+
nonce: string;
|
|
37
|
+
/** What the exploit asserts, e.g. "read /etc/passwd of target". */
|
|
38
|
+
claim: string;
|
|
39
|
+
/** Request spec the harness executes in phase 1 and again from the main-agent phase-2 call. */
|
|
40
|
+
verify: {
|
|
41
|
+
method: string;
|
|
42
|
+
url: string;
|
|
43
|
+
headers?: Record<string, string>;
|
|
44
|
+
body?: string;
|
|
45
|
+
expect: VerifyExpect;
|
|
46
|
+
/** Optional stronger causality dimension, independent of the authored predicate. */
|
|
47
|
+
canary?: VerifyCanary;
|
|
48
|
+
};
|
|
49
|
+
/** What the script itself saw — corroboration only, never proof. */
|
|
50
|
+
observations: string[];
|
|
51
|
+
/** Optional baseline request for the main agent's differential review. */
|
|
52
|
+
baseline?: {
|
|
53
|
+
method: string;
|
|
54
|
+
url: string;
|
|
55
|
+
headers?: Record<string, string>;
|
|
56
|
+
body?: string;
|
|
57
|
+
body_contains?: string[];
|
|
58
|
+
};
|
|
59
|
+
};
|
|
60
|
+
|
|
61
|
+
const HTTP_METHODS = ["GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS"];
|
|
62
|
+
const MAX_CLAIM_CHARS = 2_000;
|
|
63
|
+
const MAX_URL_CHARS = 4_096;
|
|
64
|
+
const MAX_REQUEST_BODY_CHARS = 128 * 1024;
|
|
65
|
+
const MAX_HEADERS = 64;
|
|
66
|
+
const MAX_HEADER_CHARS = 8 * 1024;
|
|
67
|
+
const MAX_EXPECT_VALUES = 16;
|
|
68
|
+
const MAX_EXPECT_CHARS = 1_024;
|
|
69
|
+
const MAX_REGEX_VALUES = 8;
|
|
70
|
+
const MAX_REGEX_CHARS = 512;
|
|
71
|
+
const MAX_OBSERVATIONS = 64;
|
|
72
|
+
const MAX_OBSERVATION_CHARS = 4_096;
|
|
73
|
+
const FORBIDDEN_REQUEST_HEADERS = new Set([
|
|
74
|
+
"connection",
|
|
75
|
+
"content-length",
|
|
76
|
+
"host",
|
|
77
|
+
"keep-alive",
|
|
78
|
+
"proxy-authorization",
|
|
79
|
+
"proxy-connection",
|
|
80
|
+
"te",
|
|
81
|
+
"trailer",
|
|
82
|
+
"transfer-encoding",
|
|
83
|
+
"upgrade",
|
|
84
|
+
]);
|
|
85
|
+
|
|
86
|
+
function isRecord(v: unknown): v is Record<string, unknown> {
|
|
87
|
+
return typeof v === "object" && v !== null && !Array.isArray(v);
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
function nonEmptyString(v: unknown): v is string {
|
|
91
|
+
return typeof v === "string" && v.trim().length > 0;
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
function boundedStringArray(v: unknown, maxItems: number, maxChars: number): v is string[] {
|
|
95
|
+
return (
|
|
96
|
+
Array.isArray(v) &&
|
|
97
|
+
v.length <= maxItems &&
|
|
98
|
+
v.every((x) => nonEmptyString(x) && x.length <= maxChars)
|
|
99
|
+
);
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
/** Bounded RFC-token header map with no CR/LF (injection/resource guard). */
|
|
103
|
+
function headerRecord(v: unknown): v is Record<string, string> {
|
|
104
|
+
if (!isRecord(v)) return false;
|
|
105
|
+
const entries = Object.entries(v);
|
|
106
|
+
return (
|
|
107
|
+
entries.length <= MAX_HEADERS &&
|
|
108
|
+
entries.every(
|
|
109
|
+
([key, value]) =>
|
|
110
|
+
/^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/.test(key) &&
|
|
111
|
+
!FORBIDDEN_REQUEST_HEADERS.has(key.toLowerCase()) &&
|
|
112
|
+
key.length <= 256 &&
|
|
113
|
+
typeof value === "string" &&
|
|
114
|
+
value.length <= MAX_HEADER_CHARS &&
|
|
115
|
+
!/[\r\n]/.test(value),
|
|
116
|
+
)
|
|
117
|
+
);
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
/** http(s) URL with a non-empty host — prefix-only matches would accept garbage. */
|
|
121
|
+
function httpUrl(v: unknown): v is string {
|
|
122
|
+
if (!nonEmptyString(v) || v.length > MAX_URL_CHARS) return false;
|
|
123
|
+
let parsed: URL;
|
|
124
|
+
try {
|
|
125
|
+
parsed = new URL(v);
|
|
126
|
+
} catch {
|
|
127
|
+
return false;
|
|
128
|
+
}
|
|
129
|
+
return (
|
|
130
|
+
(parsed.protocol === "http:" || parsed.protocol === "https:") && parsed.hostname.length > 0
|
|
131
|
+
);
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
/** body_regex entries must compile — evaluation is isolated and time-bounded by the harness. */
|
|
135
|
+
function validRegexArray(v: unknown): v is string[] {
|
|
136
|
+
return (
|
|
137
|
+
boundedStringArray(v, MAX_REGEX_VALUES, MAX_REGEX_CHARS) &&
|
|
138
|
+
v.every((re) => {
|
|
139
|
+
try {
|
|
140
|
+
new RegExp(re);
|
|
141
|
+
return true;
|
|
142
|
+
} catch {
|
|
143
|
+
return false;
|
|
144
|
+
}
|
|
145
|
+
})
|
|
146
|
+
);
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
function hasMeaningfulExpectation(expect: Record<string, unknown>): boolean {
|
|
150
|
+
return [expect.status, expect.body_contains, expect.body_regex].some(
|
|
151
|
+
(value) => Array.isArray(value) && value.length > 0,
|
|
152
|
+
);
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
function hasBodyExpectation(expect: Record<string, unknown>): boolean {
|
|
156
|
+
return [expect.body_contains, expect.body_regex].some(
|
|
157
|
+
(value) => Array.isArray(value) && value.length > 0,
|
|
158
|
+
);
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
/** Reject predicates that are structurally incapable of carrying useful proof. */
|
|
162
|
+
function hasDiscriminatingBodyExpectation(expect: Record<string, unknown>): boolean {
|
|
163
|
+
const contains = Array.isArray(expect.body_contains) ? expect.body_contains : [];
|
|
164
|
+
if (contains.some((value) => typeof value === "string" && value.trim().length >= 4)) {
|
|
165
|
+
return true;
|
|
166
|
+
}
|
|
167
|
+
const regexes = Array.isArray(expect.body_regex) ? expect.body_regex : [];
|
|
168
|
+
return regexes.some(
|
|
169
|
+
(pattern) =>
|
|
170
|
+
typeof pattern === "string" &&
|
|
171
|
+
// Require a literal alphabetic anchor, e.g. `root:\\S+` or `is_admin`.
|
|
172
|
+
// Patterns such as `.`, `.*`, or `\\d+` match broad response classes and
|
|
173
|
+
// cannot independently identify the claimed vulnerability effect.
|
|
174
|
+
/[A-Za-z]{4,}/.test(pattern.replace(/\\[dDsSwWbB]/g, "")),
|
|
175
|
+
);
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
function countOccurrences(value: string, needle: string): number {
|
|
179
|
+
return value.split(needle).length - 1;
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
/**
|
|
183
|
+
* Parse + validate a PoC's evidence.json. Returns the validated object or a
|
|
184
|
+
* field-level error. Deliberately strict: an invalid evidence file means the
|
|
185
|
+
* run failed the contract, which blocks promotion.
|
|
186
|
+
*/
|
|
187
|
+
export function parsePoCEvidence(
|
|
188
|
+
raw: unknown,
|
|
189
|
+
): { ok: true; evidence: PoCEvidence } | { ok: false; error: string } {
|
|
190
|
+
if (!isRecord(raw)) return { ok: false, error: "evidence.json must be a JSON object" };
|
|
191
|
+
if (!nonEmptyString(raw.nonce)) return { ok: false, error: "evidence.json missing nonce" };
|
|
192
|
+
if (raw.nonce.length > 256) return { ok: false, error: "evidence.json nonce is too long" };
|
|
193
|
+
if (!nonEmptyString(raw.claim)) return { ok: false, error: "evidence.json missing claim" };
|
|
194
|
+
if (raw.claim.length > MAX_CLAIM_CHARS) {
|
|
195
|
+
return { ok: false, error: `evidence.json claim exceeds ${MAX_CLAIM_CHARS} characters` };
|
|
196
|
+
}
|
|
197
|
+
const verify = raw.verify;
|
|
198
|
+
if (!isRecord(verify)) return { ok: false, error: "evidence.json verify must be an object" };
|
|
199
|
+
if (!nonEmptyString(verify.method) || !HTTP_METHODS.includes(verify.method.toUpperCase())) {
|
|
200
|
+
return {
|
|
201
|
+
ok: false,
|
|
202
|
+
error: `evidence.json verify.method must be one of ${HTTP_METHODS.join("|")}`,
|
|
203
|
+
};
|
|
204
|
+
}
|
|
205
|
+
if (!httpUrl(verify.url)) {
|
|
206
|
+
return { ok: false, error: "evidence.json verify.url must be an http(s) URL with a host" };
|
|
207
|
+
}
|
|
208
|
+
if (verify.headers !== undefined && !headerRecord(verify.headers)) {
|
|
209
|
+
return {
|
|
210
|
+
ok: false,
|
|
211
|
+
error:
|
|
212
|
+
"evidence.json verify.headers must be bounded valid end-to-end HTTP headers; authority, framing, proxy, and hop-by-hop headers are forbidden",
|
|
213
|
+
};
|
|
214
|
+
}
|
|
215
|
+
if (
|
|
216
|
+
verify.body !== undefined &&
|
|
217
|
+
(typeof verify.body !== "string" || verify.body.length > MAX_REQUEST_BODY_CHARS)
|
|
218
|
+
) {
|
|
219
|
+
return {
|
|
220
|
+
ok: false,
|
|
221
|
+
error: `evidence.json verify.body must be a string no longer than ${MAX_REQUEST_BODY_CHARS} characters`,
|
|
222
|
+
};
|
|
223
|
+
}
|
|
224
|
+
if (verify.canary !== undefined) {
|
|
225
|
+
if (
|
|
226
|
+
!isRecord(verify.canary) ||
|
|
227
|
+
verify.canary.mode !== "reflection" ||
|
|
228
|
+
verify.canary.placeholder !== POC_CANARY_PLACEHOLDER
|
|
229
|
+
) {
|
|
230
|
+
return {
|
|
231
|
+
ok: false,
|
|
232
|
+
error: `evidence.json verify.canary must be {"mode":"reflection","placeholder":"${POC_CANARY_PLACEHOLDER}"}`,
|
|
233
|
+
};
|
|
234
|
+
}
|
|
235
|
+
const canaryLocations = [
|
|
236
|
+
verify.url,
|
|
237
|
+
typeof verify.body === "string" ? verify.body : "",
|
|
238
|
+
...(isRecord(verify.headers)
|
|
239
|
+
? Object.values(verify.headers).filter(
|
|
240
|
+
(value): value is string => typeof value === "string",
|
|
241
|
+
)
|
|
242
|
+
: []),
|
|
243
|
+
];
|
|
244
|
+
const count = canaryLocations.reduce(
|
|
245
|
+
(total, value) => total + countOccurrences(value, POC_CANARY_PLACEHOLDER),
|
|
246
|
+
0,
|
|
247
|
+
);
|
|
248
|
+
if (count !== 1) {
|
|
249
|
+
return {
|
|
250
|
+
ok: false,
|
|
251
|
+
error: `evidence.json verify.canary requires exactly one ${POC_CANARY_PLACEHOLDER} placeholder across url, body, or header values (got ${count})`,
|
|
252
|
+
};
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
const expect = verify.expect;
|
|
256
|
+
if (!isRecord(expect))
|
|
257
|
+
return { ok: false, error: "evidence.json verify.expect must be an object" };
|
|
258
|
+
if (
|
|
259
|
+
expect.status !== undefined &&
|
|
260
|
+
(!Array.isArray(expect.status) ||
|
|
261
|
+
expect.status.length === 0 ||
|
|
262
|
+
expect.status.length > MAX_EXPECT_VALUES ||
|
|
263
|
+
!expect.status.every((s) => Number.isInteger(s) && s >= 100 && s <= 599))
|
|
264
|
+
) {
|
|
265
|
+
return {
|
|
266
|
+
ok: false,
|
|
267
|
+
error: "evidence.json verify.expect.status must be a non-empty array of HTTP status codes",
|
|
268
|
+
};
|
|
269
|
+
}
|
|
270
|
+
if (
|
|
271
|
+
expect.body_contains !== undefined &&
|
|
272
|
+
!boundedStringArray(expect.body_contains, MAX_EXPECT_VALUES, MAX_EXPECT_CHARS)
|
|
273
|
+
) {
|
|
274
|
+
return {
|
|
275
|
+
ok: false,
|
|
276
|
+
error: `evidence.json verify.expect.body_contains must have at most ${MAX_EXPECT_VALUES} non-empty strings of at most ${MAX_EXPECT_CHARS} characters`,
|
|
277
|
+
};
|
|
278
|
+
}
|
|
279
|
+
if (expect.body_regex !== undefined && !validRegexArray(expect.body_regex)) {
|
|
280
|
+
return {
|
|
281
|
+
ok: false,
|
|
282
|
+
error: `evidence.json verify.expect.body_regex must have at most ${MAX_REGEX_VALUES} compilable patterns of at most ${MAX_REGEX_CHARS} characters`,
|
|
283
|
+
};
|
|
284
|
+
}
|
|
285
|
+
if (!hasMeaningfulExpectation(expect)) {
|
|
286
|
+
return {
|
|
287
|
+
ok: false,
|
|
288
|
+
error:
|
|
289
|
+
"evidence.json verify.expect must contain at least one non-empty status, body_contains, or body_regex assertion",
|
|
290
|
+
};
|
|
291
|
+
}
|
|
292
|
+
if (!hasBodyExpectation(expect)) {
|
|
293
|
+
return {
|
|
294
|
+
ok: false,
|
|
295
|
+
error:
|
|
296
|
+
"evidence.json verify.expect needs a non-empty body_contains or body_regex assertion; status-only differences are not vulnerability proof",
|
|
297
|
+
};
|
|
298
|
+
}
|
|
299
|
+
if (!hasDiscriminatingBodyExpectation(expect)) {
|
|
300
|
+
return {
|
|
301
|
+
ok: false,
|
|
302
|
+
error:
|
|
303
|
+
"evidence.json verify.expect needs a discriminating body predicate: body_contains must include at least 4 characters or body_regex must include a literal alphabetic anchor; trivial matchers are not vulnerability proof",
|
|
304
|
+
};
|
|
305
|
+
}
|
|
306
|
+
if (
|
|
307
|
+
!Array.isArray(raw.observations) ||
|
|
308
|
+
raw.observations.length > MAX_OBSERVATIONS ||
|
|
309
|
+
!raw.observations.every((o) => typeof o === "string" && o.length <= MAX_OBSERVATION_CHARS)
|
|
310
|
+
) {
|
|
311
|
+
return { ok: false, error: "evidence.json observations exceed the bounded string-array limit" };
|
|
312
|
+
}
|
|
313
|
+
if (raw.baseline !== undefined) {
|
|
314
|
+
const b = raw.baseline;
|
|
315
|
+
if (!isRecord(b)) return { ok: false, error: "evidence.json baseline must be an object" };
|
|
316
|
+
if (
|
|
317
|
+
!nonEmptyString(b.method) ||
|
|
318
|
+
!HTTP_METHODS.includes(b.method.toUpperCase()) ||
|
|
319
|
+
!httpUrl(b.url)
|
|
320
|
+
) {
|
|
321
|
+
return {
|
|
322
|
+
ok: false,
|
|
323
|
+
error: "evidence.json baseline needs an http(s) url and a valid HTTP method",
|
|
324
|
+
};
|
|
325
|
+
}
|
|
326
|
+
if (b.headers !== undefined && !headerRecord(b.headers)) {
|
|
327
|
+
return {
|
|
328
|
+
ok: false,
|
|
329
|
+
error:
|
|
330
|
+
"evidence.json baseline.headers must be bounded valid end-to-end HTTP headers; authority, framing, proxy, and hop-by-hop headers are forbidden",
|
|
331
|
+
};
|
|
332
|
+
}
|
|
333
|
+
if (
|
|
334
|
+
b.body !== undefined &&
|
|
335
|
+
(typeof b.body !== "string" || b.body.length > MAX_REQUEST_BODY_CHARS)
|
|
336
|
+
) {
|
|
337
|
+
return {
|
|
338
|
+
ok: false,
|
|
339
|
+
error: `evidence.json baseline.body must be no longer than ${MAX_REQUEST_BODY_CHARS} characters`,
|
|
340
|
+
};
|
|
341
|
+
}
|
|
342
|
+
if (
|
|
343
|
+
b.body_contains !== undefined &&
|
|
344
|
+
!boundedStringArray(b.body_contains, MAX_EXPECT_VALUES, MAX_EXPECT_CHARS)
|
|
345
|
+
) {
|
|
346
|
+
return { ok: false, error: "evidence.json baseline.body_contains exceeds limits" };
|
|
347
|
+
}
|
|
348
|
+
}
|
|
349
|
+
return { ok: true, evidence: raw as unknown as PoCEvidence };
|
|
350
|
+
}
|
|
351
|
+
|
|
352
|
+
/** Bind evidence to its run: the nonce must equal the harness-generated one. */
|
|
353
|
+
export function evidenceNonceMatches(evidence: PoCEvidence, nonce: string): boolean {
|
|
354
|
+
return evidence.nonce === nonce;
|
|
355
|
+
}
|
|
356
|
+
|
|
357
|
+
/**
|
|
358
|
+
* The comparator for determinism + differential checks. Strips the nonce
|
|
359
|
+
* (per-run by design) and observations (free-form, run-dependent) — the
|
|
360
|
+
* load-bearing shape is claim + verify + baseline.
|
|
361
|
+
*/
|
|
362
|
+
export function normalizeEvidence(e: PoCEvidence): string {
|
|
363
|
+
return JSON.stringify({ claim: e.claim, verify: e.verify, baseline: e.baseline });
|
|
364
|
+
}
|
|
365
|
+
|
|
366
|
+
// ── Main-agent confirmation verdict ─────────────────────────────────
|
|
367
|
+
|
|
368
|
+
export const CONFIRM_VERDICT_VALUES = ["CONFIRMED", "NOT_CONFIRMED"] as const;
|
|
369
|
+
export type ConfirmVerdict = (typeof CONFIRM_VERDICT_VALUES)[number];
|
|
370
|
+
|
|
371
|
+
export const CONFIRM_DIFFERENTIAL_VALUES = [
|
|
372
|
+
"target_only",
|
|
373
|
+
"both",
|
|
374
|
+
"control_only",
|
|
375
|
+
"unclear",
|
|
376
|
+
] as const;
|
|
377
|
+
export type ConfirmDifferential = (typeof CONFIRM_DIFFERENTIAL_VALUES)[number];
|
|
378
|
+
|
|
379
|
+
export const SEVERITY_MATCH_VALUES = ["under", "over", "ok"] as const;
|
|
380
|
+
export const CANARY_ASSESSMENT_VALUES = ["verified", "not_applicable"] as const;
|
|
381
|
+
|
|
382
|
+
export type MainAgentVerdict = {
|
|
383
|
+
verdict: ConfirmVerdict;
|
|
384
|
+
reasoning: string;
|
|
385
|
+
/** Files/evidence the main agent actually reviewed. */
|
|
386
|
+
evidence_reviewed: string[];
|
|
387
|
+
/** What the main agent observed during its review and fresh harness replay. */
|
|
388
|
+
re_execution_note?: string;
|
|
389
|
+
/** Target vs control evidence comparison. CONFIRMED requires target_only. */
|
|
390
|
+
differential: ConfirmDifferential;
|
|
391
|
+
/** Claimed severity vs what the evidence shows. */
|
|
392
|
+
severity_match?: (typeof SEVERITY_MATCH_VALUES)[number];
|
|
393
|
+
/** The main agent's own failed attempt to disprove — becomes the case's disconfirmation. */
|
|
394
|
+
disconfirmation_attempt?: string;
|
|
395
|
+
/** Whether the machine replay carried a harness-generated causal canary. */
|
|
396
|
+
canary_assessment?: (typeof CANARY_ASSESSMENT_VALUES)[number];
|
|
397
|
+
/** Why no meaningful canary oracle exists for this exploit class. */
|
|
398
|
+
canary_reason?: string;
|
|
399
|
+
/** Which model judged (recorded for the accuracy ledger). */
|
|
400
|
+
model?: string;
|
|
401
|
+
};
|
|
402
|
+
|
|
403
|
+
/**
|
|
404
|
+
* Validate the main-agent verdict. CONFIRMED additionally requires a target-only
|
|
405
|
+
* differential, a concrete review note, and a disconfirmation attempt. The
|
|
406
|
+
* ledger separately requires a fresh harness-owned phase-2 replay; there is no
|
|
407
|
+
* caller-supplied `re_executed` checkbox.
|
|
408
|
+
*/
|
|
409
|
+
export function validateMainAgentVerdict(
|
|
410
|
+
raw: unknown,
|
|
411
|
+
): { ok: true; verdict: MainAgentVerdict } | { ok: false; error: string } {
|
|
412
|
+
if (!isRecord(raw)) return { ok: false, error: "verdict must be a JSON object" };
|
|
413
|
+
if (!nonEmptyString(raw.verdict) || !CONFIRM_VERDICT_VALUES.includes(raw.verdict as never)) {
|
|
414
|
+
return {
|
|
415
|
+
ok: false,
|
|
416
|
+
error: `verdict must be one of ${CONFIRM_VERDICT_VALUES.join(" | ")}`,
|
|
417
|
+
};
|
|
418
|
+
}
|
|
419
|
+
if (!nonEmptyString(raw.reasoning)) return { ok: false, error: "verdict reasoning required" };
|
|
420
|
+
if (!Array.isArray(raw.evidence_reviewed) || raw.evidence_reviewed.length === 0) {
|
|
421
|
+
return { ok: false, error: "verdict evidence_reviewed must be a non-empty array" };
|
|
422
|
+
}
|
|
423
|
+
if (!raw.evidence_reviewed.every((e) => typeof e === "string")) {
|
|
424
|
+
return { ok: false, error: "verdict evidence_reviewed entries must be strings" };
|
|
425
|
+
}
|
|
426
|
+
if (raw.re_execution_note !== undefined && !nonEmptyString(raw.re_execution_note)) {
|
|
427
|
+
return { ok: false, error: "verdict re_execution_note must be a non-empty string" };
|
|
428
|
+
}
|
|
429
|
+
if (
|
|
430
|
+
!nonEmptyString(raw.differential) ||
|
|
431
|
+
!CONFIRM_DIFFERENTIAL_VALUES.includes(raw.differential as never)
|
|
432
|
+
) {
|
|
433
|
+
return {
|
|
434
|
+
ok: false,
|
|
435
|
+
error: `verdict differential must be one of ${CONFIRM_DIFFERENTIAL_VALUES.join(" | ")}`,
|
|
436
|
+
};
|
|
437
|
+
}
|
|
438
|
+
if (
|
|
439
|
+
raw.canary_assessment !== undefined &&
|
|
440
|
+
!CANARY_ASSESSMENT_VALUES.includes(raw.canary_assessment as never)
|
|
441
|
+
) {
|
|
442
|
+
return {
|
|
443
|
+
ok: false,
|
|
444
|
+
error: `verdict canary_assessment must be one of ${CANARY_ASSESSMENT_VALUES.join(" | ")}`,
|
|
445
|
+
};
|
|
446
|
+
}
|
|
447
|
+
if (raw.canary_reason !== undefined && !nonEmptyString(raw.canary_reason)) {
|
|
448
|
+
return { ok: false, error: "verdict canary_reason must be a non-empty string" };
|
|
449
|
+
}
|
|
450
|
+
if (
|
|
451
|
+
raw.severity_match !== undefined &&
|
|
452
|
+
!SEVERITY_MATCH_VALUES.includes(raw.severity_match as never)
|
|
453
|
+
) {
|
|
454
|
+
return {
|
|
455
|
+
ok: false,
|
|
456
|
+
error: `verdict severity_match must be one of ${SEVERITY_MATCH_VALUES.join(" | ")}`,
|
|
457
|
+
};
|
|
458
|
+
}
|
|
459
|
+
if (raw.verdict === "CONFIRMED") {
|
|
460
|
+
if (raw.differential !== "target_only") {
|
|
461
|
+
return {
|
|
462
|
+
ok: false,
|
|
463
|
+
error:
|
|
464
|
+
'CONFIRMED requires differential "target_only" — the control run must not demonstrate the claimed impact',
|
|
465
|
+
};
|
|
466
|
+
}
|
|
467
|
+
if (!nonEmptyString(raw.re_execution_note)) {
|
|
468
|
+
return {
|
|
469
|
+
ok: false,
|
|
470
|
+
error:
|
|
471
|
+
"CONFIRMED requires re_execution_note — record what the main agent observed during review and the fresh harness replay",
|
|
472
|
+
};
|
|
473
|
+
}
|
|
474
|
+
if (!nonEmptyString(raw.disconfirmation_attempt)) {
|
|
475
|
+
return {
|
|
476
|
+
ok: false,
|
|
477
|
+
error:
|
|
478
|
+
"CONFIRMED requires disconfirmation_attempt — the main agent's own failed attempt to disprove",
|
|
479
|
+
};
|
|
480
|
+
}
|
|
481
|
+
if (!CANARY_ASSESSMENT_VALUES.includes(raw.canary_assessment as never)) {
|
|
482
|
+
return {
|
|
483
|
+
ok: false,
|
|
484
|
+
error: `CONFIRMED requires canary_assessment (${CANARY_ASSESSMENT_VALUES.join(" | ")})`,
|
|
485
|
+
};
|
|
486
|
+
}
|
|
487
|
+
if (raw.canary_assessment === "not_applicable" && !nonEmptyString(raw.canary_reason)) {
|
|
488
|
+
return {
|
|
489
|
+
ok: false,
|
|
490
|
+
error: "CONFIRMED with canary_assessment not_applicable requires canary_reason",
|
|
491
|
+
};
|
|
492
|
+
}
|
|
493
|
+
}
|
|
494
|
+
return { ok: true, verdict: raw as unknown as MainAgentVerdict };
|
|
495
|
+
}
|
|
496
|
+
|
|
497
|
+
/** @deprecated Compatibility alias for integrations built before phase 2 became main-agent-only. */
|
|
498
|
+
export type ConfirmerVerdict = MainAgentVerdict;
|
|
499
|
+
|
|
500
|
+
/** @deprecated Use validateMainAgentVerdict. */
|
|
501
|
+
export const validateConfirmerVerdict = validateMainAgentVerdict;
|