thoth-agents 0.9.8 → 0.9.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/agents/prompt-dialects.d.ts +1 -0
- package/dist/{chunk-EDGQVTV3.js → chunk-2ER6FKVC.js} +52 -5
- package/dist/{chunk-V5GDCWFR.js → chunk-RN2ERZLZ.js} +1 -1
- package/dist/{chunk-ARUBELNT.js → chunk-RVQ6IIBR.js} +10 -4
- package/dist/chunk-RWQPMWEM.js +624 -0
- package/dist/{chunk-QQV5S3TW.js → chunk-VHQMTZ5O.js} +5 -24
- package/dist/cli/index.js +4 -4
- package/dist/cli/pi-tool-config.d.ts +0 -1
- package/dist/cli/tui/index.js +3 -3
- package/dist/harness/gate-hook-script.d.ts +2 -0
- package/dist/harness/writers/claude-code-plugin-package.d.ts +4 -4
- package/dist/harness/writers/pi-agent.d.ts +0 -1
- package/dist/hooks/index.d.ts +1 -0
- package/dist/hooks/oracle-gate-verdict/hook.d.ts +11 -0
- package/dist/hooks/oracle-gate-verdict/index.d.ts +1 -0
- package/dist/index.js +42 -3
- package/dist/pi-gate-verdict.d.ts +4 -0
- package/dist/pi.d.ts +2 -1
- package/dist/pi.js +101 -2
- package/package.json +1 -1
- package/pi/.thoth-agents-assets.json +5 -5
- package/pi/agents/thoth-designer.md +0 -1
- package/pi/agents/thoth-explorer.md +0 -1
- package/pi/agents/thoth-librarian.md +0 -1
- package/pi/agents/thoth-oracle.md +0 -1
- package/pi/agents/thoth-worker.md +0 -1
- package/plugin/.claude-plugin/.thoth-agents-plugin-assets.json +6 -1
- package/plugin/.claude-plugin/plugin.json +3 -2
- package/plugin/.codex-plugin/.thoth-agents-plugin-assets.json +2 -2
- package/plugin/.codex-plugin/plugin.json +1 -1
- package/plugin/agents/orchestrator.md +5 -3
- package/plugin/claude-hooks/hooks.json +27 -0
- package/plugin/codex-hooks/hooks.json +14 -0
- package/plugin/codex-hooks/payloads/root.md +5 -3
- package/plugin/skills/plan-reviewer/SKILL.md +16 -5
- package/plugin/skills/thoth-sdd/SKILL.md +69 -23
- package/plugin/skills/thoth-sdd/references/phases/verify.md +6 -2
- package/plugin/skills/thoth-sdd/scripts/record-digest.mjs +15 -0
- package/plugin/skills/thoth-sdd/scripts/validate-result.d.mts +19 -0
- package/plugin/skills/thoth-sdd/scripts/validate-result.mjs +110 -41
- package/plugin/skills/thoth-sdd/scripts/validate.mjs +7 -16
- package/plugin/thoth-hooks/gate-verdict.mjs +80 -0
- package/skills/plan-reviewer/SKILL.md +16 -5
- package/skills/thoth-sdd/SKILL.md +69 -23
- package/skills/thoth-sdd/references/phases/verify.md +6 -2
- package/skills/thoth-sdd/scripts/record-digest.mjs +15 -0
- package/skills/thoth-sdd/scripts/validate-result.d.mts +19 -0
- package/skills/thoth-sdd/scripts/validate-result.mjs +110 -41
- package/skills/thoth-sdd/scripts/validate.mjs +7 -16
|
@@ -160,31 +160,74 @@ only; do not provision skills or run a network installer mid-workflow.
|
|
|
160
160
|
|
|
161
161
|
Only Oracle gate assignments (plan-review and final-verification) return a
|
|
162
162
|
structured envelope. Before recording plan-review `OKAY` or final-verification
|
|
163
|
-
`PASS`,
|
|
164
|
-
|
|
163
|
+
`PASS`, use the harness-delivered `thoth-gate-verdict` line for that result:
|
|
164
|
+
exactly one per delivered result; more than one, or one inside the envelope
|
|
165
|
+
fence, is invalid. No delivered verdict means the explicit fallback below,
|
|
166
|
+
not approval. Never re-emit the envelope to validate it when a verdict exists.
|
|
167
|
+
|
|
168
|
+
Require `valid:true` and `kind` matches the assignment. Plan review requires
|
|
169
|
+
`verdict:"OKAY"` and `record` null or the assigned record; ignore `mode` and
|
|
170
|
+
`recordDigest`. Final verification requires `verdict:"PASS"` plus:
|
|
171
|
+
|
|
172
|
+
- Record-bound final verification requires `record` = assigned record and
|
|
173
|
+
`mode:"record"`, with `recordDigest:"match"`: the bundled validator hashed exact
|
|
174
|
+
UTF-8 record bytes before `## Authorization`. A mismatch is no approval. If a
|
|
175
|
+
valid verdict has `recordDigest:"unavailable"`, independently calculate the
|
|
176
|
+
record-prefix SHA-256 and compare it with `recordSha256`; mismatch or inability
|
|
177
|
+
to calculate means no approval. This checks digest agreement only: do not rerun
|
|
178
|
+
the validator or re-emit the envelope.
|
|
179
|
+
- Recordless small-change final verification requires `mode:"no-record"`, null
|
|
180
|
+
`record` and `recordSha256`, and `recordDigest:"none"`.
|
|
181
|
+
|
|
182
|
+
Record equality, `mode` and `recordDigest` checks apply only to final verification.
|
|
183
|
+
See the [verify phase](references/phases/verify.md).
|
|
184
|
+
Copy the verdict and `recordSha256` into the record when there is one; final
|
|
185
|
+
verification uses that digest for `**Reviewed record SHA-256**`. Still judge
|
|
186
|
+
scope, evidence, checks, freshness, provenance, independence, and content.
|
|
187
|
+
Validation is not authorization.
|
|
188
|
+
|
|
189
|
+
Delivery differs by harness: Pi attaches verdicts to completion notifications,
|
|
190
|
+
`subagent_result`, and task-mode `subagent_run` (match `taskId`); OpenCode appends
|
|
191
|
+
them to Oracle `task` output. Claude Code verdicts arrive with Subagent hand-back
|
|
192
|
+
reports (foreground or background; `agent` is the agent id, match it to the Agent
|
|
193
|
+
result's `agentId`) and completed foreground `Agent` results whose content carries
|
|
194
|
+
the report. Other deliveries (e.g. background runs without hand-back) carry none;
|
|
195
|
+
use the explicit compact fallback below. Codex verdicts arrive only with
|
|
196
|
+
multi-agent v1 `wait_agent` results when plugin hooks are enabled and trusted;
|
|
197
|
+
`agent` is the status-map key, not proof of Oracle provenance. Multi-agent v2
|
|
198
|
+
delivers the Oracle's `FINAL_ANSWER` as an inter-agent message with no hook, so no
|
|
199
|
+
verdict arrives and root uses the explicit compact fallback below on the unchanged
|
|
200
|
+
result.
|
|
201
|
+
|
|
202
|
+
### Explicit compact fallback
|
|
203
|
+
|
|
204
|
+
Only when no verdict was delivered (hook unavailable/untrusted, a Codex multi-agent
|
|
205
|
+
v2 delivery, or a Claude background run without hand-back), resolve `<skill-dir>` from the installed
|
|
206
|
+
thoth-sdd SKILL.md directory, exactly as for `scripts/validate.mjs`, and run:
|
|
165
207
|
|
|
166
208
|
```text
|
|
167
209
|
node "<skill-dir>/scripts/validate-result.mjs" --review-kind <kind> [--record-sha256 <hex>] [--no-record] [--input <path>]
|
|
168
210
|
```
|
|
169
211
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
`final-verification`. For
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
- Exit **1** means no approval.
|
|
186
|
-
|
|
187
|
-
|
|
212
|
+
Supply the unchanged native result through stdin or an existing `--input <path>`;
|
|
213
|
+
never extract, rewrite, or wrap the envelope or create persisted evidence files.
|
|
214
|
+
The CLI prints only the compact summary, not the envelope. Set `--review-kind`
|
|
215
|
+
to the assigned `plan-review` or `final-verification`. For record-bound final
|
|
216
|
+
verification, independently calculate the expected record-prefix SHA-256 and
|
|
217
|
+
pass `--record-sha256 <hex>`. Apply the same kind-specific approval checks above;
|
|
218
|
+
the explicit expected digest supplies agreement if the CLI cannot read the
|
|
219
|
+
record itself. Copy the accepted summary and digest into
|
|
220
|
+
the record. For Oracle final verification of a small change without a record,
|
|
221
|
+
root passes `--no-record`; Oracle omits both `reviewedRecord` and
|
|
222
|
+
`reviewedRecordSha256`. Do not combine `--no-record` with `--record-sha256` or
|
|
223
|
+
plan-review. Without `--no-record`, final verification requires both fields.
|
|
224
|
+
|
|
225
|
+
- Exit **0** means structurally valid only, not approval: apply the assignment,
|
|
226
|
+
digest, content and provenance checks above.
|
|
227
|
+
- Exit **1** means no approval. Invalid delivered verdicts have the same rule.
|
|
228
|
+
Allow at most one format-only repair request per assignment, shared with any
|
|
229
|
+
runtime enforcement; it must not change findings or verdict. Exhaustion is a
|
|
230
|
+
contract failure, never converted into an Oracle `BLOCKED` judgment.
|
|
188
231
|
- Exit **2** or an unavailable skill/validator means validation cannot be completed:
|
|
189
232
|
report the error (missing assets are installation drift). This blocks only gate approvals;
|
|
190
233
|
ordinary delegations proceed. Do not provision replacements mid-workflow.
|
|
@@ -215,7 +258,8 @@ final-review and Archive placeholders. `closeout` additionally requires a fresh
|
|
|
215
258
|
independent Oracle PASS, every acceptance outcome with concrete PASS
|
|
216
259
|
check/evidence, exactly one matching record SHA-256 field, and Archive READY.
|
|
217
260
|
Order: `verify` → fresh final Oracle → root validates and pastes → `closeout` → archive.
|
|
218
|
-
|
|
261
|
+
Root validates via the delivery verdict (explicit compact fallback only if none
|
|
262
|
+
was delivered), records the verdict and digest, then pastes the Oracle's unchanged
|
|
219
263
|
final-verification envelope under `**Oracle envelope**:` in the record's
|
|
220
264
|
Verification section (post-PASS bookkeeping in Verification; no new review).
|
|
221
265
|
Plan-review envelopes are not embedded. Closeout fails without exactly one valid
|
|
@@ -230,8 +274,10 @@ The initial `PENDING` selection placeholder is allowed before `verify` only.
|
|
|
230
274
|
have exactly one `Source` entry in the same record:
|
|
231
275
|
`- Source: .thoth/specs/<capability>/spec.md | sha256:<digest>` when present, or
|
|
232
276
|
`- Source: .thoth/specs/<capability>/spec.md | absent` when absent. Missing or
|
|
233
|
-
stale coverage blocks `verify`, closeout, and archive.
|
|
234
|
-
|
|
277
|
+
stale coverage blocks `verify`, closeout, and archive. The record hash covers
|
|
278
|
+
exact UTF-8 bytes preceding the case-sensitive `## Authorization` heading; use
|
|
279
|
+
the delivery verdict's `recordSha256` when its digest matches. Independent hash
|
|
280
|
+
calculation is needed only for unavailable digest agreement or the CLI fallback.
|
|
235
281
|
After PASS, edits confined to Authorization, Verification, or Closeout need no
|
|
236
282
|
new review; any edit before `## Authorization` requires fresh independent review
|
|
237
283
|
and a new digest. This allowance never waives structural checks, truthful
|
|
@@ -2,13 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
Every change requires final verification against accepted outcomes and actual execution evidence. Small low-risk work uses focused independent root checks; substantial or materially risky work requires a fresh read-only Oracle independent from the writer. Judge the actual diff, behavior, completeness, correctness, coherence, checks, and residual risks.
|
|
4
4
|
|
|
5
|
-
For Oracle final verification of a small change without a record,
|
|
5
|
+
For Oracle final verification of a small change without a record, require a valid delivery verdict with `mode:"no-record"`, null `record` and `recordSha256`, and `recordDigest:"none"`. Oracle omits both `reviewedRecord` and `reviewedRecordSha256`. Only in the CLI fallback, root passes `--no-record`. Do not combine `--no-record` with `--record-sha256` or plan-review. Otherwise final verification still requires both binding fields.
|
|
6
6
|
|
|
7
7
|
Before requesting final Oracle for substantial work, run `node "<skill-dir>/scripts/validate.mjs" --change .thoth/changes/<id> --through verify --json`. This gate requires ready coverage, complete tasks, authorization disposition/provenance, and reviewed source digests/baselines; final-review fields, AC evidence rows, and Archive may remain placeholders. For each capability named by a durable delta, add exactly one `Source` entry for `.thoth/specs/<capability>/spec.md`: its reviewed SHA-256 when it exists, or `absent` only when no canonical file exists.
|
|
8
8
|
|
|
9
9
|
Assign Oracle a `final-verification` review of the substantial record. Oracle returns exactly one fenced `thoth-result` JSON envelope as the entire final message, following the Oracle gate contract with `role: "oracle"`, `review.kind: "final-verification"`, and `review.verdict: "PASS" | "FAIL" | "BLOCKED"`. Its review includes `reviewedRecord` naming the reviewed record path and `reviewedRecordSha256` containing the 64 lowercase hex SHA-256 of exact UTF-8 record bytes preceding the case-sensitive `## Authorization` heading. PASS requires `status: "complete"` and no blocking risks. FAIL reports findings; BLOCKED is an actual inability to complete review, never a substitute for a malformed result or exhausted format repair.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
Record equality, `mode` and `recordDigest` checks apply only to final verification. Plan review requires `valid:true`, `kind:"plan-review"`, `verdict:"OKAY"`, and `record` null or the assigned record; ignore `mode` and `recordDigest`.
|
|
12
|
+
|
|
13
|
+
Root uses the harness-delivered `thoth-gate-verdict` under thoth-sdd **Specialist result validation**: exactly one outside the envelope fence, `valid:true`, assigned `kind:"final-verification"`, `verdict:"PASS"`, assigned `record`, `mode:"record"`, and `recordDigest:"match"`. Never re-emit the envelope to validate it. If a valid verdict reports `recordDigest:"unavailable"`, independently hash exact UTF-8 record bytes before `## Authorization` and compare with `recordSha256`; mismatch or unavailable calculation means no approval, with no validator rerun. Use the explicit compact CLI fallback only when no verdict was delivered: unchanged result, `--review-kind final-verification --record-sha256 <hex>` with the independently calculated expected prefix digest. Claude Code verdicts arrive with Subagent hand-back reports (foreground or background; `agent` is the agent id, match it to the Agent result's `agentId`) and completed foreground `Agent` results whose content carries the report. Other deliveries (e.g. background runs without hand-back) carry none; use the explicit compact CLI fallback.
|
|
14
|
+
|
|
15
|
+
Root records `**Reviewer**: oracle`, `**Independent from implementer**: Yes`, `**Verdict**: PASS`, and `**Reviewed record SHA-256**` only from a validated PASS whose digest matches the current record, after judging content and confirming reviewer independence and provenance. Copy the verdict and `recordSha256` into the record (accepted CLI summary/digest in fallback); use `recordSha256` for the SHA field. Invalid results, FAIL, BLOCKED, and diagnosis/advice reviews do not authorize closeout or archive; route findings to convergence and report contract failures truthfully.
|
|
12
16
|
|
|
13
17
|
Substantial records must not contain other `thoth-result` fences or `**Oracle envelope**:` marker lines anywhere in the file; write examples inline instead.
|
|
14
18
|
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
|
|
3
|
+
/** Hash the exact UTF-8 prefix preceding the first case-sensitive heading. */
|
|
4
|
+
export function recordPrefixSha256(text) {
|
|
5
|
+
let offset = 0;
|
|
6
|
+
for (const raw of text.split(/(?<=\n)/)) {
|
|
7
|
+
let line = raw.replace(/\r?\n$/, '');
|
|
8
|
+
if (offset === 0) line = line.replace(/^\uFEFF/, '');
|
|
9
|
+
if (line === '## Authorization') break;
|
|
10
|
+
offset += raw.length;
|
|
11
|
+
}
|
|
12
|
+
return createHash('sha256')
|
|
13
|
+
.update(text.slice(0, offset), 'utf8')
|
|
14
|
+
.digest('hex');
|
|
15
|
+
}
|
|
@@ -44,4 +44,23 @@ export function validateResult(
|
|
|
44
44
|
message: string,
|
|
45
45
|
options?: ResultValidationOptions,
|
|
46
46
|
): ResultValidation;
|
|
47
|
+
export interface GateSummaryOptions {
|
|
48
|
+
cwd?: string;
|
|
49
|
+
}
|
|
50
|
+
export interface GateResultSummary {
|
|
51
|
+
valid: boolean;
|
|
52
|
+
role: string | null;
|
|
53
|
+
kind: string | null;
|
|
54
|
+
verdict: string | null;
|
|
55
|
+
mode: 'record' | 'no-record';
|
|
56
|
+
record: string | null;
|
|
57
|
+
recordSha256: string | null;
|
|
58
|
+
recordDigest: 'none' | 'match' | 'mismatch' | 'unavailable';
|
|
59
|
+
issues: Pick<ResultValidationIssue, 'code' | 'path'>[];
|
|
60
|
+
}
|
|
61
|
+
export function summarizeGateResult(
|
|
62
|
+
message: string,
|
|
63
|
+
options?: GateSummaryOptions,
|
|
64
|
+
): GateResultSummary;
|
|
65
|
+
export function formatGateVerdict(summary: GateResultSummary): string;
|
|
47
66
|
export const RESULT_VALIDATOR_USAGE: string;
|
|
@@ -1,10 +1,9 @@
|
|
|
1
|
-
import { readFileSync } from 'node:fs';
|
|
2
|
-
import { resolve } from 'node:path';
|
|
1
|
+
import { readFileSync, realpathSync } from 'node:fs';
|
|
2
|
+
import { isAbsolute, relative, resolve, sep } from 'node:path';
|
|
3
3
|
import { fileURLToPath } from 'node:url';
|
|
4
|
+
import { recordPrefixSha256 } from './record-digest.mjs';
|
|
5
|
+
import contract from './result-contract.json' with { type: 'json' };
|
|
4
6
|
|
|
5
|
-
const contract = JSON.parse(
|
|
6
|
-
readFileSync(new URL('./result-contract.json', import.meta.url), 'utf8'),
|
|
7
|
-
);
|
|
8
7
|
const childPath = (path, key) =>
|
|
9
8
|
`${path}/${String(key).replaceAll('~', '~0').replaceAll('/', '~1')}`;
|
|
10
9
|
|
|
@@ -123,32 +122,39 @@ function checkExpectations(options, issue) {
|
|
|
123
122
|
);
|
|
124
123
|
}
|
|
125
124
|
|
|
126
|
-
|
|
127
|
-
const issues = [];
|
|
128
|
-
const issue = (code, path, text) =>
|
|
129
|
-
issues.push({ code, path, message: text });
|
|
130
|
-
const invalid = () => ({ valid: false, issues });
|
|
131
|
-
checkExpectations(options, issue);
|
|
132
|
-
if (issues.length) return invalid();
|
|
125
|
+
function parseEnvelope(message) {
|
|
133
126
|
const trimmed = typeof message === 'string' ? message.trim() : '';
|
|
134
127
|
const block = new RegExp(
|
|
135
128
|
`^\`\`\`${contract.fence}\\r?\\n([\\s\\S]*?)\\r?\\n\`\`\`$`,
|
|
136
129
|
).exec(trimmed);
|
|
137
130
|
if (!block || (trimmed.match(/^```.*$/gm) ?? []).length !== 2) {
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
131
|
+
return {
|
|
132
|
+
issues: [
|
|
133
|
+
{
|
|
134
|
+
code: 'ENVELOPE_FORMAT',
|
|
135
|
+
path: '',
|
|
136
|
+
message:
|
|
137
|
+
'Expected exactly one thoth-result fence and no surrounding prose.',
|
|
138
|
+
},
|
|
139
|
+
],
|
|
140
|
+
};
|
|
144
141
|
}
|
|
145
|
-
let envelope;
|
|
146
142
|
try {
|
|
147
|
-
envelope
|
|
143
|
+
return { envelope: parseStrictJSON(block[1]), issues: [] };
|
|
148
144
|
} catch (error) {
|
|
149
|
-
issues
|
|
150
|
-
return invalid();
|
|
145
|
+
return { issues: [error] };
|
|
151
146
|
}
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
function validateEnvelope(parsed, options) {
|
|
150
|
+
const issues = [];
|
|
151
|
+
const issue = (code, path, text) =>
|
|
152
|
+
issues.push({ code, path, message: text });
|
|
153
|
+
const invalid = () => ({ valid: false, issues });
|
|
154
|
+
checkExpectations(options, issue);
|
|
155
|
+
if (issues.length) return invalid();
|
|
156
|
+
if (parsed.issues.length) return { valid: false, issues: parsed.issues };
|
|
157
|
+
const { envelope } = parsed;
|
|
152
158
|
function check(value, schema, path) {
|
|
153
159
|
const type =
|
|
154
160
|
value === null ? 'null' : Array.isArray(value) ? 'array' : typeof value;
|
|
@@ -314,9 +320,83 @@ export function validateResult(message, options = {}) {
|
|
|
314
320
|
return issues.length ? invalid() : { valid: true, envelope, issues };
|
|
315
321
|
}
|
|
316
322
|
|
|
323
|
+
export function validateResult(message, options = {}) {
|
|
324
|
+
return validateEnvelope(parseEnvelope(message), options);
|
|
325
|
+
}
|
|
326
|
+
|
|
327
|
+
const stringOrNull = (value) => (typeof value === 'string' ? value : null);
|
|
328
|
+
function gateMode(envelope) {
|
|
329
|
+
const review = envelope?.review;
|
|
330
|
+
return review?.kind === contract.rules.recordBindingReviewKind &&
|
|
331
|
+
contract.rules.recordBindingRequiredFields.every(
|
|
332
|
+
(key) => !Object.hasOwn(review, key),
|
|
333
|
+
)
|
|
334
|
+
? 'no-record'
|
|
335
|
+
: 'record';
|
|
336
|
+
}
|
|
337
|
+
|
|
338
|
+
function recordDigest(review, cwd) {
|
|
339
|
+
if (!review || !Object.hasOwn(review, 'reviewedRecordSha256')) return 'none';
|
|
340
|
+
if (!cwd || typeof review.reviewedRecord !== 'string') return 'unavailable';
|
|
341
|
+
const inside = (root, path) => {
|
|
342
|
+
const fromRoot = relative(root, path);
|
|
343
|
+
return (
|
|
344
|
+
fromRoot !== '..' &&
|
|
345
|
+
!fromRoot.startsWith(`..${sep}`) &&
|
|
346
|
+
!isAbsolute(fromRoot)
|
|
347
|
+
);
|
|
348
|
+
};
|
|
349
|
+
try {
|
|
350
|
+
const root = resolve(cwd);
|
|
351
|
+
const path = resolve(root, review.reviewedRecord);
|
|
352
|
+
if (!inside(root, path) || !inside(realpathSync(root), realpathSync(path)))
|
|
353
|
+
return 'unavailable';
|
|
354
|
+
return recordPrefixSha256(readFileSync(path, 'utf8')) ===
|
|
355
|
+
review.reviewedRecordSha256
|
|
356
|
+
? 'match'
|
|
357
|
+
: 'mismatch';
|
|
358
|
+
} catch {
|
|
359
|
+
return 'unavailable';
|
|
360
|
+
}
|
|
361
|
+
}
|
|
362
|
+
|
|
363
|
+
function summarizeValidation(envelope, result, cwd) {
|
|
364
|
+
const review = envelope?.review;
|
|
365
|
+
return {
|
|
366
|
+
valid: result.valid,
|
|
367
|
+
role: stringOrNull(envelope?.role),
|
|
368
|
+
kind: stringOrNull(review?.kind),
|
|
369
|
+
verdict: stringOrNull(review?.verdict),
|
|
370
|
+
mode: gateMode(envelope),
|
|
371
|
+
record: stringOrNull(review?.reviewedRecord),
|
|
372
|
+
recordSha256: stringOrNull(review?.reviewedRecordSha256),
|
|
373
|
+
recordDigest: recordDigest(review, cwd),
|
|
374
|
+
issues: result.issues.map(({ code, path }) => ({ code, path })),
|
|
375
|
+
};
|
|
376
|
+
}
|
|
377
|
+
|
|
378
|
+
export function summarizeGateResult(message, { cwd } = {}) {
|
|
379
|
+
const parsed = parseEnvelope(message);
|
|
380
|
+
return summarizeValidation(
|
|
381
|
+
parsed.envelope,
|
|
382
|
+
validateEnvelope(parsed, {
|
|
383
|
+
noRecord: gateMode(parsed.envelope) === 'no-record',
|
|
384
|
+
}),
|
|
385
|
+
cwd,
|
|
386
|
+
);
|
|
387
|
+
}
|
|
388
|
+
|
|
389
|
+
export function formatGateVerdict(summary) {
|
|
390
|
+
return `thoth-gate-verdict ${JSON.stringify(summary)}`;
|
|
391
|
+
}
|
|
392
|
+
|
|
317
393
|
export const RESULT_VALIDATOR_USAGE =
|
|
318
394
|
'node validate-result.mjs [--role oracle] [--review-kind plan-review|final-verification] [--record-sha256 <hex>] [--no-record] [--input <path>]';
|
|
319
395
|
function main(args) {
|
|
396
|
+
const print = (result, envelope) =>
|
|
397
|
+
console.log(
|
|
398
|
+
JSON.stringify(summarizeValidation(envelope, result, process.cwd())),
|
|
399
|
+
);
|
|
320
400
|
const options = {};
|
|
321
401
|
let input;
|
|
322
402
|
const names = {
|
|
@@ -335,14 +415,7 @@ function main(args) {
|
|
|
335
415
|
(flag !== '--no-record' &&
|
|
336
416
|
(!args[index + 1] || args[index + 1].startsWith('--')))
|
|
337
417
|
) {
|
|
338
|
-
|
|
339
|
-
JSON.stringify({
|
|
340
|
-
valid: false,
|
|
341
|
-
issues: [
|
|
342
|
-
{ code: 'CLI_USAGE', path: '', message: RESULT_VALIDATOR_USAGE },
|
|
343
|
-
],
|
|
344
|
-
}),
|
|
345
|
-
);
|
|
418
|
+
print({ valid: false, issues: [{ code: 'CLI_USAGE', path: '' }] });
|
|
346
419
|
return 2;
|
|
347
420
|
}
|
|
348
421
|
seen.add(flag);
|
|
@@ -359,31 +432,27 @@ function main(args) {
|
|
|
359
432
|
expectationIssues.push({ code, path, message });
|
|
360
433
|
});
|
|
361
434
|
if (expectationIssues.length) {
|
|
362
|
-
|
|
435
|
+
print({ valid: false, issues: expectationIssues });
|
|
363
436
|
return 2;
|
|
364
437
|
}
|
|
365
438
|
let message;
|
|
366
439
|
try {
|
|
367
440
|
message = readFileSync(input ?? 0, 'utf8');
|
|
368
441
|
} catch {
|
|
369
|
-
|
|
370
|
-
JSON.stringify({
|
|
371
|
-
valid: false,
|
|
372
|
-
issues: [
|
|
373
|
-
{ code: 'CLI_INPUT', path: '', message: 'Unable to read input.' },
|
|
374
|
-
],
|
|
375
|
-
}),
|
|
376
|
-
);
|
|
442
|
+
print({ valid: false, issues: [{ code: 'CLI_INPUT', path: '' }] });
|
|
377
443
|
return 2;
|
|
378
444
|
}
|
|
379
|
-
const
|
|
380
|
-
|
|
445
|
+
const parsed = parseEnvelope(message);
|
|
446
|
+
const result = validateEnvelope(parsed, options);
|
|
447
|
+
print(result, parsed.envelope);
|
|
381
448
|
if (result.valid) return 0;
|
|
382
449
|
return result.issues.some((issue) => issue.code === 'EXPECTATION_INVALID')
|
|
383
450
|
? 2
|
|
384
451
|
: 1;
|
|
385
452
|
}
|
|
453
|
+
// Bundled imports share the entry/chunk URL and must not invoke this CLI.
|
|
386
454
|
if (
|
|
455
|
+
import.meta.url.endsWith('/validate-result.mjs') &&
|
|
387
456
|
process.argv[1] &&
|
|
388
457
|
resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
|
389
458
|
)
|
|
@@ -17,6 +17,7 @@ import {
|
|
|
17
17
|
parseRequirementDelta,
|
|
18
18
|
preflightRequirementDeltas,
|
|
19
19
|
} from './durable-deltas.mjs';
|
|
20
|
+
import { recordPrefixSha256 } from './record-digest.mjs';
|
|
20
21
|
import { validateResult } from './validate-result.mjs';
|
|
21
22
|
|
|
22
23
|
const sha = (bytes) => createHash('sha256').update(bytes).digest('hex');
|
|
@@ -302,18 +303,12 @@ function coverageErrors(text, ids, errors, recordName) {
|
|
|
302
303
|
|
|
303
304
|
// Whole-file textual binding: Markdown containers never hide candidates or markers.
|
|
304
305
|
function scanOracleEnvelope(text) {
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
if (offset === 0) entry.line = entry.line.replace(/^\uFEFF/, '');
|
|
309
|
-
offset += raw.length;
|
|
310
|
-
return entry;
|
|
306
|
+
const lines = text.split(/(?<=\n)/).map((raw, index) => {
|
|
307
|
+
const line = raw.replace(/\r?\n$/, '');
|
|
308
|
+
return { raw, line: index === 0 ? line.replace(/^\uFEFF/, '') : line };
|
|
311
309
|
});
|
|
312
310
|
const markers = [];
|
|
313
311
|
const fences = [];
|
|
314
|
-
const authorization = lines.findIndex(
|
|
315
|
-
({ line }) => line === '## Authorization',
|
|
316
|
-
);
|
|
317
312
|
const verification = lines.findIndex(
|
|
318
313
|
({ line }) => line === '## Verification',
|
|
319
314
|
);
|
|
@@ -363,7 +358,6 @@ function scanOracleEnvelope(text) {
|
|
|
363
358
|
markers,
|
|
364
359
|
fences,
|
|
365
360
|
positioned,
|
|
366
|
-
authorizationOffset: lines[authorization]?.offset,
|
|
367
361
|
};
|
|
368
362
|
}
|
|
369
363
|
|
|
@@ -727,10 +721,7 @@ export function validate({ change, through }) {
|
|
|
727
721
|
: verificationText;
|
|
728
722
|
const verified = verificationFields.trim();
|
|
729
723
|
if (through === 'closeout') {
|
|
730
|
-
const
|
|
731
|
-
0,
|
|
732
|
-
extraction.authorizationOffset ?? text.length,
|
|
733
|
-
);
|
|
724
|
+
const recordSha256 = recordPrefixSha256(text);
|
|
734
725
|
if (
|
|
735
726
|
!/^\*\*Reviewer\*\*: oracle$/m.test(verified) ||
|
|
736
727
|
!/^\*\*Independent from implementer\*\*: Yes$/m.test(verified) ||
|
|
@@ -753,7 +744,7 @@ export function validate({ change, through }) {
|
|
|
753
744
|
digestLines[0],
|
|
754
745
|
)?.[1]
|
|
755
746
|
: undefined;
|
|
756
|
-
if (reviewedDigest !==
|
|
747
|
+
if (reviewedDigest !== recordSha256) {
|
|
757
748
|
errors.push(
|
|
758
749
|
issue(
|
|
759
750
|
'SDD-VERIFICATION-STALE',
|
|
@@ -765,7 +756,7 @@ export function validate({ change, through }) {
|
|
|
765
756
|
validateOracleEnvelope({
|
|
766
757
|
extraction,
|
|
767
758
|
verificationFields,
|
|
768
|
-
recordSha256
|
|
759
|
+
recordSha256,
|
|
769
760
|
reviewedDigest,
|
|
770
761
|
// Archived records retain the active path reviewed by the Oracle.
|
|
771
762
|
expectedRecord: `.thoth/changes/${id}/${id}.md`,
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
import { readFileSync } from 'node:fs';
|
|
2
|
+
|
|
3
|
+
try {
|
|
4
|
+
const input = JSON.parse(readFileSync(0, 'utf8'));
|
|
5
|
+
if (typeof input?.cwd === 'string') {
|
|
6
|
+
const messages = [];
|
|
7
|
+
let malformed = false;
|
|
8
|
+
if (input.hook_event_name === 'UserPromptSubmit' && typeof input.prompt === 'string') {
|
|
9
|
+
const lines = input.prompt.split(/\r?\n/);
|
|
10
|
+
const frameStart = /^<agent-message from="([^"<>\s]+)">$/;
|
|
11
|
+
// Frames quoted later in an ordinary prompt are not hand-back deliveries.
|
|
12
|
+
if (!frameStart.test(lines.find(line => line.trim() !== '') ?? '')) process.exit(0);
|
|
13
|
+
// Scan the whole delivery before validating: a closer must not approve a report prefix.
|
|
14
|
+
let frame;
|
|
15
|
+
for (const line of lines) {
|
|
16
|
+
const start = frameStart.exec(line);
|
|
17
|
+
if (line.startsWith('<agent-message')) {
|
|
18
|
+
if (frame || !start) malformed = true;
|
|
19
|
+
if (start) {
|
|
20
|
+
frame = { agent: start[1], text: '', raw: '', header: false, report: false };
|
|
21
|
+
messages.push(frame);
|
|
22
|
+
}
|
|
23
|
+
} else if (line === '</agent-message>') {
|
|
24
|
+
if (!frame?.report) malformed = true;
|
|
25
|
+
frame = undefined;
|
|
26
|
+
} else if (frame) {
|
|
27
|
+
frame.raw += line + '\n';
|
|
28
|
+
if (!frame.report) {
|
|
29
|
+
if (!frame.header) {
|
|
30
|
+
frame.header = line.startsWith('[Subagent hand-back]');
|
|
31
|
+
if (!frame.header) malformed = true;
|
|
32
|
+
}
|
|
33
|
+
if (/^[ \t]/.test(line)) malformed = true;
|
|
34
|
+
if (frame.header && line.endsWith('The report follows:')) frame.report = true;
|
|
35
|
+
} else {
|
|
36
|
+
if (line !== '' && !line.startsWith(' ')) malformed = true;
|
|
37
|
+
frame.text += line.slice(2) + '\n';
|
|
38
|
+
}
|
|
39
|
+
} else if (line.trim() !== '') {
|
|
40
|
+
malformed = true;
|
|
41
|
+
}
|
|
42
|
+
}
|
|
43
|
+
if (frame) malformed = true;
|
|
44
|
+
} else if (input.hook_event_name === 'PostToolUse' && input.tool_name === 'Agent' &&
|
|
45
|
+
input.tool_input?.subagent_type === "thoth-agents:oracle" &&
|
|
46
|
+
input.tool_response?.status === 'completed' &&
|
|
47
|
+
Array.isArray(input.tool_response.content)) {
|
|
48
|
+
const text = input.tool_response.content
|
|
49
|
+
.filter(block => block?.type === 'text' && typeof block.text === 'string')
|
|
50
|
+
.map(block => block.text).join('\n');
|
|
51
|
+
messages.push({ text, agent: input.tool_input.subagent_type });
|
|
52
|
+
} else if (input.hook_event_name === 'PostToolUse' &&
|
|
53
|
+
(input.tool_name === 'wait_agent' || input.tool_name === 'multi_agent_v1wait_agent') &&
|
|
54
|
+
typeof input.tool_response === 'string') {
|
|
55
|
+
// Codex serializes WaitAgentResult as JSON text; status keys are agent paths or thread IDs.
|
|
56
|
+
const response = JSON.parse(input.tool_response);
|
|
57
|
+
if (response?.status && typeof response.status === 'object' && !Array.isArray(response.status)) {
|
|
58
|
+
for (const [agent, status] of Object.entries(response.status)) {
|
|
59
|
+
if (typeof status?.completed === 'string') messages.push({ text: status.completed, agent });
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
const gates = messages.filter(({ text, raw }) =>
|
|
64
|
+
(malformed ? raw ?? text : text).includes('```thoth-result'));
|
|
65
|
+
if (gates.length) {
|
|
66
|
+
const { summarizeGateResult, formatGateVerdict } =
|
|
67
|
+
await import('../skills/thoth-sdd/scripts/validate-result.mjs');
|
|
68
|
+
// An empty envelope supplies the canonical ENVELOPE_FORMAT summary with no trusted fields.
|
|
69
|
+
const framingFailure = malformed ? summarizeGateResult('') : undefined;
|
|
70
|
+
const lines = gates.map(({ text, agent }) =>
|
|
71
|
+
formatGateVerdict({ ...(framingFailure ?? summarizeGateResult(text, { cwd: input.cwd })), agent }));
|
|
72
|
+
process.stdout.write(JSON.stringify({ hookSpecificOutput: {
|
|
73
|
+
hookEventName: input.hook_event_name,
|
|
74
|
+
additionalContext: lines.join('\n'),
|
|
75
|
+
} }));
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
} catch {
|
|
79
|
+
// Delivery is additive: malformed input or unavailable validation must never block the event.
|
|
80
|
+
}
|
|
@@ -38,11 +38,22 @@ message, following the Oracle gate contract: `role: "oracle"`,
|
|
|
38
38
|
`review.kind: "plan-review"`, and `review.verdict: "OKAY" | "REJECT"`.
|
|
39
39
|
Include concise evidence and at most three blocking risks for actual execution
|
|
40
40
|
blockers with smallest repairs; mark cautions nonblocking. An OKAY requires
|
|
41
|
-
`status: "complete"` and no blocking risks. Root
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
41
|
+
`status: "complete"` and no blocking risks. Root uses the harness-delivered
|
|
42
|
+
`thoth-gate-verdict` under the sibling thoth-sdd skill's **Specialist result
|
|
43
|
+
validation** procedure: exactly one outside the envelope fence, `valid:true`,
|
|
44
|
+
assigned `kind:"plan-review"`, `verdict:"OKAY"`, and `record` null or the assigned
|
|
45
|
+
record; ignore `mode` and `recordDigest` (final-verification checks only).
|
|
46
|
+
Never re-emit the envelope to validate it. Use the explicit
|
|
47
|
+
compact CLI fallback with `--review-kind plan-review` only when no verdict was delivered;
|
|
48
|
+
keep the native result unchanged. Claude Code verdicts arrive with Subagent hand-back
|
|
49
|
+
reports (foreground or background; `agent` is the agent id, match it to the Agent
|
|
50
|
+
result's `agentId`) and completed foreground `Agent` results whose content carries
|
|
51
|
+
the report. Other deliveries (e.g. background runs without hand-back) carry none;
|
|
52
|
+
use the explicit compact CLI fallback.
|
|
53
|
+
Copy the verdict and `recordSha256` into the record (accepted summary/digest
|
|
54
|
+
in fallback). Keep the shared one-request format-only repair budget. Root records
|
|
55
|
+
plan-review `OKAY` only after these checks and judging content and provenance;
|
|
56
|
+
a malformed or missing envelope is not approval.
|
|
46
57
|
Root repairs same-intent blockers in the same record, revalidates affected gates,
|
|
47
58
|
and uses a fresh Oracle for each round. A human-owned material blocker stops the
|
|
48
59
|
round. Root records both review disposition and selection provenance without
|