opencode-workflow-guard 1.18.1 → 1.19.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -4
- package/docs/installation.md +23 -0
- package/docs/policies.md +20 -0
- package/docs/testing.md +1 -0
- package/package.json +2 -2
- package/src/lib/audit.ts +52 -2
- package/src/lib/custom-tools.ts +154 -3
- package/src/lib/dogfood.ts +147 -0
- package/src/lib/evidence.ts +19 -1
- package/src/lib/guard-dispatcher.ts +30 -0
- package/src/lib/state.ts +221 -1
- package/src/lib/types.ts +64 -1
- package/src/lib/v2-plugin.ts +12 -1
- package/src/workflow-guard.ts +29 -2
- package/test/test.mts +626 -0
package/README.md
CHANGED
|
@@ -43,9 +43,9 @@ WORKFLOW_GUARD_LIVE_E2E=1 npm run test:install # Also run policy probes with Ope
|
|
|
43
43
|
|
|
44
44
|
## Core Guardrails Overview
|
|
45
45
|
|
|
46
|
-
The plugin enforces **
|
|
46
|
+
The plugin enforces **27 deterministic policy rules** across four main pillars:
|
|
47
47
|
|
|
48
|
-
* **Branch & History Protection:** Enforces feature-branch workflows and blocks edits and commits on `main`/`master`. Direct and forced pushes to protected branches are hard-blocked. The guard checks mergeability before PR creation, requires secondary review approval bound to the reviewed worktree content, and requires a PR changelog or changeset.
|
|
48
|
+
* **Branch & History Protection:** Enforces feature-branch workflows and blocks edits and commits on `main`/`master`. Direct and forced pushes to protected branches are hard-blocked. The guard checks mergeability before PR creation, requires secondary review approval bound to the reviewed worktree content, and requires a PR changelog or changeset. An opt-in dogfood gate can additionally require a recorded run of the change on real hardware before a PR opens.
|
|
49
49
|
* **Task & Verification Discipline:** Gates file mutations on active `todowrite` tasks, which subagents inherit. A task can be marked completed only after its own recorded work. Review and investigation tasks use an audited `[read-only]` marker instead. The guard also blocks concurrent direct edits to the same canonical file across sessions. It requires a same-session fresh read before `edit`/`write` can replace an existing file. Silent task deletion is blocked, and task completion requires fresh test verification evidence. Completion claims and sweeping repository claims are journaled against their evidence. Repeated failures force a live-documentation lookup instead of a guess from training data.
|
|
50
50
|
* **Secrets & Workspace Confinement:** Confines edit tools and recognized shell/git mutations to the workspace root, including symlink and `../` escape checks. Agent subshells have sensitive environment variables scrubbed, and `.env` reads return a safe schema mask. Arbitrary executables are not OS-sandboxed. Use containers or filesystem isolation when hard confinement is required.
|
|
51
51
|
* **Destructive CLI & Environment Safety:** Intercepts destructive cloud, database, and infrastructure commands. Stops script laundering and interpreter evasion, blocks TTY hangs such as `vim`, `nano`, and `sudo`, and guards against dangerous package manager flags.
|
|
@@ -59,7 +59,7 @@ For complete policy specifications and override rules, see [docs/policies.md](do
|
|
|
59
59
|
Enforcement is only half the surface. The plugin also registers first-class agent tools under the same names on OpenCode 1.x and 2.x, so disciplined workflow is the path of least resistance:
|
|
60
60
|
|
|
61
61
|
* **Task Planning:** `guard_next_tasks` surfaces durable repository planning context at session start or during planning. It reads `TODO.md`, `ROADMAP.md`, `PLAN.md`, `TASKS.md`, `BACKLOG.md`, their `docs/` counterparts, and plans under `docs/plans/`. `guard_status` reports active guardrails, branch protection, mutation counts, outstanding verification and review gates, a read-only git-hygiene snapshot, and the loaded `pluginVersion`.
|
|
62
|
-
* **Review & Verification:** before PR creation, a secondary reviewer subagent evaluates the diff against `guard_review_rubric`. The rubric covers five axes: test integrity, task completeness, cleanliness, security, and platform. The reviewer records its verdict with `record_review`, and approvals bind to the reviewed worktree content.
|
|
62
|
+
* **Review & Verification:** before PR creation, a secondary reviewer subagent evaluates the diff against `guard_review_rubric`. The rubric covers five axes: test integrity, task completeness, cleanliness, security, and platform. The reviewer records its verdict with `record_review`, and approvals bind to the reviewed worktree content. An opt-in dogfood gate adds a recorded end-to-end run on real hardware with `record_dogfood`.
|
|
63
63
|
* **Worktree Isolation:** `guard_worktree_create` and `guard_worktree_cleanup` give parallel subagents isolated linked worktrees, so concurrent mutations never collide.
|
|
64
64
|
* **Technical Debt:** `guard_review_followups` lists open P2/P3 durable review follow-ups. `guard_review_followup_resolve` closes one after the underlying issue is fixed and verified.
|
|
65
65
|
* **Policy Simulation:** `guard_why` returns the structured policy decision for a tool call or command without performing it. `guard_audit` shows recent audited policy entries.
|
|
@@ -74,7 +74,7 @@ The context modules are project memory, Socratic learning, and recovery checkpoi
|
|
|
74
74
|
|
|
75
75
|
| Guide | Description |
|
|
76
76
|
|---|---|
|
|
77
|
-
| [**Policy Reference**](docs/policies.md) | Specification of all
|
|
77
|
+
| [**Policy Reference**](docs/policies.md) | Specification of all 27 enforced policies, invariants, and the proactive tool surface (`guard_*`, `project_memory_*`, `learning_*`) |
|
|
78
78
|
| [**Installation & Configuration**](docs/installation.md) | Setup options, global and local install, worktree isolation, and project configuration |
|
|
79
79
|
| [**Static Policy Posture**](docs/policy-posture.md) | How the guard emits host-level policies for invariants a repository cannot lift. Covers both host envelopes (`experimental.policies` on 2.0.x, top-level `policies` on 2.1.x), precedence, and residual risk |
|
|
80
80
|
| [**Managed Deployment**](docs/managed-deployment.md) | Administrator-managed OpenCode policy, platform locations, and startup diagnostics |
|
package/docs/installation.md
CHANGED
|
@@ -245,6 +245,29 @@ For stricter environments, set `requireSubagentReview: true` in `.opencode/workf
|
|
|
245
245
|
|
|
246
246
|
---
|
|
247
247
|
|
|
248
|
+
## Pre-Deployment Dogfood Gate
|
|
249
|
+
|
|
250
|
+
Policy 27 gates PR creation on a recorded run of the change on real hardware. It is **off by default**, because a gate is only meaningful once a repository has a runner that can produce a real run. Enable it after that runner exists:
|
|
251
|
+
|
|
252
|
+
```json
|
|
253
|
+
{
|
|
254
|
+
"requireDogfood": true,
|
|
255
|
+
"dogfoodDeployMatrix": ["bluefin", "windows-ci"]
|
|
256
|
+
}
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
`WORKFLOW_GUARD_REQUIRE_DOGFOOD=1` enables the gate for any project, and `WORKFLOW_GUARD_REQUIRE_DOGFOOD=0` disables it even when project config enables it. `WORKFLOW_GUARD_DOGFOOD_DEPLOY_MATRIX` (comma-separated) overrides the matrix.
|
|
260
|
+
|
|
261
|
+
The guard supplies the gate, the evidence record, and the recursion refusal. It does not spawn the actor session; the plugin may constrain, record, and explain, but it does not delegate. A repository supplies the runner, the frozen scenario bank, and the taxonomy, then records the outcome with `record_dogfood`.
|
|
262
|
+
|
|
263
|
+
A recorded run binds to the commit hash and tracked-content worktree fingerprint of the directory it covered, exactly like review evidence. Commit every tracked change and run the scenario afterwards, so one pass satisfies both the review and dogfood gates. Only a `pass` verdict is durable, and only the newest record per workspace and host is kept. A `hold` stays diagnosable in memory with its taxonomy class and never becomes evidence. Recording a `hold` also discards that host's cached passes, so a process restart cannot re-open the gate on an older green run.
|
|
264
|
+
|
|
265
|
+
Records accumulate, so a multi-host deploy matrix keeps one pass per host. Only the newest record per host counts. A newer `hold` on a host therefore reopens the gate even when that host passed earlier, so a later regression cannot hide behind an older green run. Retention and hold eviction are scoped to the workspace, so one repository's runs never touch another's evidence.
|
|
266
|
+
|
|
267
|
+
The runner sets `WORKFLOW_GUARD_DOGFOOD_ACTOR=1` and `WORKFLOW_GUARD_DOGFOOD_DEPTH` on the session it spawns. That session runs the full plugin with every policy intact, and the marker only refuses `record_dogfood`, so a tested session cannot certify its own run.
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
248
271
|
## Live-Documentation Research Floor
|
|
249
272
|
|
|
250
273
|
Policy 25 is enabled by default. A session that accumulates failed tool calls with no live-documentation lookup since gets its next mutating or shell call blocked. The block holds until the agent consults a current source. Only non-research failures count, so a failed `websearch` never deepens the block. Reads, greps, and research tools are never gated.
|
package/docs/policies.md
CHANGED
|
@@ -142,6 +142,7 @@ This is a policy and enforcement layer, not an agent harness. Policies may const
|
|
|
142
142
|
- Active Git branch name and protected branch status.
|
|
143
143
|
- Test verification status (passed/failed, test command, and commit hash).
|
|
144
144
|
- Secondary review verdicts (reviewer name and approval status).
|
|
145
|
+
- Dogfood evidence state when Policy 27 is enabled, naming the deploy-matrix hosts still missing a passing run.
|
|
145
146
|
- Uncommitted mutation counts.
|
|
146
147
|
- Compacted project-memory context points back to `project_memory_search` for deeper historical retrieval instead of attempting to embed the full durable store.
|
|
147
148
|
- Preserves policy-relevant context across session compactions without assigning, prioritizing, or sequencing tasks for the model.
|
|
@@ -203,6 +204,17 @@ This is a policy and enforcement layer, not an agent harness. Policies may const
|
|
|
203
204
|
- Uses the `experimental.text.complete` hook to compare a sweeping claim against this session's read coverage (`sessionReadFileCount`: distinct files read or authored). A sweeping claim backed by fewer than 2 touched files is journaled to the audit trail (`Sweeping claim '<phrase>' with N file(s) read`) and logged at info level.
|
|
204
205
|
- **Observability, not gating**, matching Policy 24. The response is never blocked or rewritten, because a grep-heavy investigation can legitimately support "no other callers" while touching few files. Prior art covers the EPISTEMIC LEDGER "bare assertion" (arXiv 2602.07549) and the agent-control-tower false-completion study. Both find that more context does not fix overclaiming, and that the check belongs in the system, not in self-report.
|
|
205
206
|
|
|
207
|
+
### 27. Pre-Deployment Dogfood Evidence (opt-in)
|
|
208
|
+
- Gates PR creation on a recorded run of the change on real hardware. Off by default: a repository enables it with `.opencode/workflow-guard.json` (`requireDogfood: true`) or `WORKFLOW_GUARD_REQUIRE_DOGFOOD=1`, and disables it with `WORKFLOW_GUARD_REQUIRE_DOGFOOD=0`.
|
|
209
|
+
- The guard owns the gate, the evidence record, and the recursion refusal. It does not spawn the actor session, because the plugin's scope excludes planning and delegation. A repository supplies its own runner, scenario bank, and frozen taxonomy, then records the outcome with `record_dogfood`.
|
|
210
|
+
- **Evidence, not a receipt.** A run is recorded as `dogfood` evidence bound to the same subject as review evidence: workspace, commit hash, and tracked-content worktree fingerprint. Any tracked edit or new commit after the run makes the evidence stale, so the gate asks for a re-run rather than trusting a stale pass. No pre-commit layer is involved.
|
|
211
|
+
- **A HOLD is never durable.** A `hold` verdict stays in memory so its taxonomy class is diagnosable, and it is never written to the disk cache. Only a `pass` is cached, so a failed run can never later read as evidence. Recording a HOLD also drops that host's cached passes, because otherwise a process restart would drop the in-memory HOLD and serve the older green run. The cache keeps only the newest record per workspace and host, so a HOLD leaves no record behind for its key and a later PASS on another host cannot resurrect the evicted one.
|
|
212
|
+
- **A pass must be complete.** `record_dogfood` refuses a `pass` verdict when any check row is not `pass`, and refuses a record with no check rows at all. A skipped check is a HOLD by the design's own rule, so accepting it would let the gate pass by omission. An empty check list asserts nothing, so it is refused rather than read as a trivially complete pass. A HOLD must name a taxonomy class.
|
|
213
|
+
- **Deploy matrix.** `dogfoodDeployMatrix` (or `WORKFLOW_GUARD_DOGFOOD_DEPLOY_MATRIX`, comma-separated) lists the host labels that each need a passing record. Unset means the recorded host alone satisfies the gate. Records accumulate rather than replacing each other, so one host's pass never overwrites another's. Only the newest record per host counts, which means a newer HOLD on a host reopens the gate even if that host passed earlier. Retention and HOLD eviction are keyed on workspace plus host, never globally, so repeated runs on a busy host cannot evict another host's only pass and a run in one repository never touches another's evidence. A host outside the matrix cannot block the gate, so its HOLD is never reported as the blocking cause. The disk cache keeps only the newest record per workspace and host, so a HOLD leaves no durable record for its key and a later PASS on a different host cannot resurrect an evicted pass.
|
|
214
|
+
- **Recursion by refusal, not absence.** The runner sets `WORKFLOW_GUARD_DOGFOOD_ACTOR=1` and `WORKFLOW_GUARD_DOGFOOD_DEPTH` on the session it spawns. Every policy still applies in that session, and the plugin is present and enforcing. The marker only refuses `record_dogfood`, so the tested session cannot certify itself. An unparseable marker fails closed to a non-zero depth.
|
|
215
|
+
- Freshness is content-scoped like the review gate, so a nested run in a linked worktree can inherit the binding for identical tracked content. A record for unrelated content never satisfies the gate.
|
|
216
|
+
- `guard_status` reports `lastDogfood` and adds `dogfood` to `outstandingRequirements` when the gate is unsatisfied, naming the matrix hosts still missing a passing run.
|
|
217
|
+
|
|
206
218
|
### Tool Description Honesty & Cross-Model Adherence
|
|
207
219
|
- Via the `tool.definition` hook, the guard enriches tool descriptions to keep them honest and proactive across all model families. The families are OpenAI, Gemini, Claude, and open-weights models:
|
|
208
220
|
- `todowrite`: Explains its replacement-list lifecycle, finalization verification gate, and directs models to `guard_next_tasks` for roadmap discovery and `guard_status` for gate checks.
|
|
@@ -246,6 +258,14 @@ Besides enforcement hooks, the guard registers companion tools in OpenCode:
|
|
|
246
258
|
- **Durable Lower-Priority Follow-ups:** Accepted review summaries containing a P2 or P3 finding are persisted in the project's local Workflow Guard SQLite database. Open findings are included in compacted agent context. They remain open across sessions until explicitly resolved with `guard_review_followup_resolve`, and they are not silently discarded with the review session.
|
|
247
259
|
- **Privacy-Safe Local Telemetry:** Guard decisions remain in the local audit JSONL. Tool after-hook observations add `callID` and duration records for local diagnosis and performance analysis. The current hook API does not distinguish success from every failure path, so these records do not claim a success outcome. Command and patch bodies are never written to the journal. Only their byte count and SHA-256 fingerprint are retained for correlation.
|
|
248
260
|
|
|
261
|
+
### `record_dogfood`
|
|
262
|
+
- Records a pre-deployment dogfood run for Policy 27: `record_dogfood(scenarioId, scenarioSha256, taxonomySha256, verdict, checks, hostLabel, ...)`.
|
|
263
|
+
- `checks` is a JSON array of `{id, status, expected, observed, detail}` rows. `expected` sits beside `observed` so a HOLD is diagnosable from the record alone, without re-running the scenario.
|
|
264
|
+
- The verdict binds to the recording session's resolved directory, or to an explicit `directory`. It is refused, loudly and without recording, when no tracked-content fingerprint can be computed for that directory.
|
|
265
|
+
- A `pass` requires every check to be `pass`, and a `hold` must name a taxonomy class. A record with no check rows is refused: an empty list asserts nothing about the run. Every refusal is journaled under `record_dogfood.verdict` with `decision: "block"`.
|
|
266
|
+
- A session carrying `WORKFLOW_GUARD_DOGFOOD_ACTOR=1` or a non-zero `WORKFLOW_GUARD_DOGFOOD_DEPTH` is refused. The tested session cannot certify its own run.
|
|
267
|
+
- The recorded digest pair is part of the evidence source, so replaying a passing run onto a different scenario or taxonomy changes the evidence id.
|
|
268
|
+
|
|
249
269
|
---
|
|
250
270
|
|
|
251
271
|
## Overrides ("Unless Otherwise Specified")
|
package/docs/testing.md
CHANGED
|
@@ -57,4 +57,5 @@ npm run test:all
|
|
|
57
57
|
| **V2 Todo Capability** | Verifies the `todowrite` registry probe (`available` / `unavailable` / `unknown`), that an empty history gates only when planning is possible, the once-per-session fail-open warning and audit record, and `todoCapability` in `guard_status`. |
|
|
58
58
|
| **Per-Task Completion Evidence** | Verifies a task cannot move to `completed` without a mutation in its own window, that one mutation cannot vouch for several completions, that staggered windows reject a claim whose window saw no work, that mutations before a task became active never count, that `[read-only]` completes freely and is journaled, that the marker can be appended to an already-planned task without tripping the no-silent-deletion rule, that duplicate content cannot hide a new completion, and that `guard_why` reports the same verdict while moving no window. |
|
|
59
59
|
| **Static Policy Posture** | Verifies the emitted posture is deny-only, emits both host envelopes (`experimental.policies`/`permission` for 2.0.x, top-level `policies`/`tool.use` for 2.1.x), requires both to reach the same verdict for every probed resource, and matches the intended commands under a replica of the host's whole-string glob matcher, including the compound-command evasion it does not cover. |
|
|
60
|
+
| **Dogfood Gate (Policy 27)** | Verifies the gate stays off until a repository opts in, that PR creation is blocked without fresh evidence and allowed with it, that a tracked edit and a new commit each invalidate the record with a named cause, that a HOLD is never durable and removes that host's cached pass so a restart cannot re-satisfy the gate, that a same-millisecond HOLD and PASS resolve to the HOLD in the durable cache so a restart stays unsatisfied, that a later pass on another host cannot resurrect an evicted one, that a HOLD in a cold process or in another workspace leaves unrelated hosts' passes intact, that an out-of-matrix HOLD is not reported as the blocking cause, that a `pass` is refused when any check is skipped or the check list is empty, that a HOLD must name a taxonomy class, that an actor session cannot record its own verdict, that the deploy matrix needs one fresh pass per host, that a newer HOLD on a host reopens the gate, that the PR audit names every covered host, and that per-host retention cannot evict another host's pass. |
|
|
60
61
|
| **Modular Exports & Shape** | Verifies V1 `PluginModule` `{ id, server }` structure, modular policy imports, and `output.args` hook contract. |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "opencode-workflow-guard",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.19.0",
|
|
4
4
|
"description": "Deterministic workflow and safety policy enforcement for OpenCode",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -50,7 +50,7 @@
|
|
|
50
50
|
"prepublishOnly": "npm run typecheck"
|
|
51
51
|
},
|
|
52
52
|
"dependencies": {
|
|
53
|
-
"@opencode-ai/plugin": "1.19.
|
|
53
|
+
"@opencode-ai/plugin": "1.19.4",
|
|
54
54
|
"@opencode/ai": "2.2.0",
|
|
55
55
|
"@opencode/plugin": "2.2.0",
|
|
56
56
|
"@opentui/solid": "0.7.0",
|
package/src/lib/audit.ts
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
|
-
import { appendFileSync, chmodSync, existsSync, mkdirSync, openSync, readSync, closeSync, renameSync, statSync, readFileSync, writeFileSync } from "node:fs";
|
|
1
|
+
import { appendFileSync, chmodSync, existsSync, mkdirSync, openSync, readSync, closeSync, renameSync, statSync, readFileSync, unlinkSync, writeFileSync } from "node:fs";
|
|
2
2
|
import { createHash, randomUUID } from "node:crypto";
|
|
3
3
|
import { homedir } from "node:os";
|
|
4
4
|
import { join } from "node:path";
|
|
5
5
|
import { Database, type SqliteDatabase } from "./sqlite.ts";
|
|
6
|
-
import type { AuditEntry, PolicyDecision, VerifyResult, ReviewResult } from "./types.ts";
|
|
6
|
+
import type { AuditEntry, PolicyDecision, VerifyResult, ReviewResult, DogfoodResult } from "./types.ts";
|
|
7
7
|
import { asRecord } from "./utils.ts";
|
|
8
8
|
|
|
9
9
|
const AUDIT_DIR = join(
|
|
@@ -15,6 +15,7 @@ const AUDIT_FILE = join(AUDIT_DIR, "workflow-guard.jsonl");
|
|
|
15
15
|
const VERIFY_CACHE_FILE = join(AUDIT_DIR, "last-verify.json");
|
|
16
16
|
const VERIFY_HISTORY_FILE = join(AUDIT_DIR, "verify-history.jsonl");
|
|
17
17
|
const REVIEW_CACHE_FILE = join(AUDIT_DIR, "last-review.json");
|
|
18
|
+
const DOGFOOD_CACHE_FILE = join(AUDIT_DIR, "last-dogfood.json");
|
|
18
19
|
const MAX_AUDIT_BYTES = 4 * 1024 * 1024;
|
|
19
20
|
const RETAIN_AUDIT_BYTES = 2 * 1024 * 1024;
|
|
20
21
|
const MAX_VERIFY_HISTORY_BYTES = 1024 * 1024;
|
|
@@ -67,6 +68,10 @@ export function getReviewCacheFilePath(): string {
|
|
|
67
68
|
return REVIEW_CACHE_FILE;
|
|
68
69
|
}
|
|
69
70
|
|
|
71
|
+
export function getDogfoodCacheFilePath(): string {
|
|
72
|
+
return DOGFOOD_CACHE_FILE;
|
|
73
|
+
}
|
|
74
|
+
|
|
70
75
|
function appendBoundedJsonlUnlocked(path: string, value: unknown, maxBytes: number, retainBytes: number): void {
|
|
71
76
|
appendFileSync(path, JSON.stringify(value) + "\n", { encoding: "utf8", mode: 0o600 });
|
|
72
77
|
chmodSync(path, 0o600);
|
|
@@ -188,6 +193,51 @@ export function loadReviewCache(): ReviewResult | undefined {
|
|
|
188
193
|
return undefined;
|
|
189
194
|
}
|
|
190
195
|
|
|
196
|
+
/**
|
|
197
|
+
* Persists the passing dogfood records. Only passes are durable: a HOLD stays
|
|
198
|
+
* in memory so the taxonomy is diagnosable without a failed run later reading
|
|
199
|
+
* as evidence. Records are kept as an array, because a deploy matrix needs one
|
|
200
|
+
* pass per host.
|
|
201
|
+
*
|
|
202
|
+
* Never throws: a cache write must not break a tool call. A failure returns
|
|
203
|
+
* false so the caller can journal it, because a failed HOLD eviction would
|
|
204
|
+
* otherwise leave a stale pass that re-opens the gate after a restart.
|
|
205
|
+
*/
|
|
206
|
+
export function persistDogfoodCache(dogfoodData: DogfoodResult[]): boolean {
|
|
207
|
+
try {
|
|
208
|
+
// An empty list means no passing evidence remains, so the file is
|
|
209
|
+
// removed rather than left as an empty array. A hold on the only host
|
|
210
|
+
// must leave no durable evidence behind.
|
|
211
|
+
if (dogfoodData.length === 0) {
|
|
212
|
+
if (existsSync(DOGFOOD_CACHE_FILE)) unlinkSync(DOGFOOD_CACHE_FILE);
|
|
213
|
+
return true;
|
|
214
|
+
}
|
|
215
|
+
mkdirSync(AUDIT_DIR, { recursive: true });
|
|
216
|
+
writeFileSync(DOGFOOD_CACHE_FILE, JSON.stringify(dogfoodData, null, 2), { encoding: "utf8", mode: 0o600 });
|
|
217
|
+
chmodSync(DOGFOOD_CACHE_FILE, 0o600);
|
|
218
|
+
return true;
|
|
219
|
+
} catch {
|
|
220
|
+
return false;
|
|
221
|
+
}
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
/**
|
|
225
|
+
* Loads durable passing dogfood records from disk. A cache written by an older
|
|
226
|
+
* single-record format is read as a one-element list rather than discarded.
|
|
227
|
+
*/
|
|
228
|
+
export function loadDogfoodCache(): DogfoodResult[] {
|
|
229
|
+
try {
|
|
230
|
+
if (!existsSync(DOGFOOD_CACHE_FILE)) return [];
|
|
231
|
+
const raw = readFileSync(DOGFOOD_CACHE_FILE, "utf8");
|
|
232
|
+
const data = JSON.parse(raw);
|
|
233
|
+
const candidates = Array.isArray(data) ? data : [data];
|
|
234
|
+
return candidates.filter((entry): entry is DogfoodResult =>
|
|
235
|
+
Boolean(entry) && typeof entry.scenarioId === "string" && typeof entry.timestamp === "number" && entry.verdict === "pass",
|
|
236
|
+
);
|
|
237
|
+
} catch {}
|
|
238
|
+
return [];
|
|
239
|
+
}
|
|
240
|
+
|
|
191
241
|
/**
|
|
192
242
|
* Reads recent audit entries from the end of the JSONL audit log with a bounded
|
|
193
243
|
* buffer window to prevent memory spikes on long-lived installations.
|
package/src/lib/custom-tools.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { tool } from "@opencode-ai/plugin";
|
|
2
2
|
import { spawnSync } from "node:child_process";
|
|
3
3
|
import { resolve } from "node:path";
|
|
4
|
-
import type { AuditEntry, LearningEvidenceKind } from "./types.ts";
|
|
4
|
+
import type { AuditEntry, DogfoodCheck, DogfoodResult, LearningEvidenceKind } from "./types.ts";
|
|
5
5
|
import type { ProjectMemoryStore } from "./project-memory.ts";
|
|
6
6
|
import {
|
|
7
7
|
exportProjectKnowledge,
|
|
@@ -13,22 +13,26 @@ import {
|
|
|
13
13
|
searchProjectMemory,
|
|
14
14
|
} from "./project-memory.ts";
|
|
15
15
|
import { loadLearnerProfile, recordLearningEvidence, selectLearningOpportunity, updateLearnerProfile } from "./learning.ts";
|
|
16
|
-
import { isEvidenceFresh, reviewEvidence, verificationEvidence } from "./evidence.ts";
|
|
16
|
+
import { isEvidenceFresh, dogfoodEvidence, reviewEvidence, verificationEvidence } from "./evidence.ts";
|
|
17
17
|
import {
|
|
18
18
|
getLearningInterventionBudget,
|
|
19
19
|
getOperationProfile,
|
|
20
20
|
getProjectConfig,
|
|
21
21
|
getRalphMaxIterations,
|
|
22
|
+
getLastDogfoodResultForWorkspace,
|
|
23
|
+
getDogfoodDeployMatrix,
|
|
22
24
|
getLastReviewResultForWorkspace,
|
|
23
25
|
getLastVerifyResultForWorkspace,
|
|
24
26
|
getWorkspaceMutationCount,
|
|
25
27
|
getWorkspaceMutationTimestamp,
|
|
26
28
|
getWorkspaceRoot,
|
|
29
|
+
isDogfoodActor,
|
|
27
30
|
isDocumentationRequired,
|
|
28
31
|
isRecoveryCheckpointsEnabled,
|
|
29
32
|
isRalphModeEnabled,
|
|
30
33
|
isReviewRequired,
|
|
31
34
|
isSubagentReviewRequired,
|
|
35
|
+
recordDogfoodResult,
|
|
32
36
|
recordMutation,
|
|
33
37
|
recordReviewResult,
|
|
34
38
|
resolveEffectiveWorkspace,
|
|
@@ -39,6 +43,7 @@ import { getRalphOutcome } from "../policies/continuation.ts";
|
|
|
39
43
|
import { todoEvidenceReport } from "../policies/todo.ts";
|
|
40
44
|
import { findGitRoot, loadProjectConfig, projectRootKey } from "./project-config.ts";
|
|
41
45
|
import { detectVerifyCommand, getCurrentGitCommitHash, getGitWorktreeFingerprint, getTrackedWorktreeFingerprint } from "./verify.ts";
|
|
46
|
+
import { evaluateDogfoodGate } from "./dogfood.ts";
|
|
42
47
|
import { buildReviewRubric } from "./review.ts";
|
|
43
48
|
import { createGitWorktree, cleanupGitWorktree, validateHostWorktreeDirectory } from "./worktree.ts";
|
|
44
49
|
import type { HostWorktreePort } from "./host-worktree.ts";
|
|
@@ -73,6 +78,12 @@ export function buildWorkflowGuardSystemGuidance(options: {
|
|
|
73
78
|
projectMemoryEnabled?: boolean;
|
|
74
79
|
learningEnabled?: boolean;
|
|
75
80
|
recoveryCheckpointsEnabled?: boolean;
|
|
81
|
+
/**
|
|
82
|
+
* Whether this repository's PR preflight also requires dogfood evidence.
|
|
83
|
+
* Reported only when the gate is enabled, so an unopted repository never
|
|
84
|
+
* advertises a requirement it does not have.
|
|
85
|
+
*/
|
|
86
|
+
dogfoodRequired?: boolean;
|
|
76
87
|
isReadOnly?: boolean;
|
|
77
88
|
}): string {
|
|
78
89
|
const sections: string[] = [
|
|
@@ -106,6 +117,12 @@ export function buildWorkflowGuardSystemGuidance(options: {
|
|
|
106
117
|
);
|
|
107
118
|
}
|
|
108
119
|
|
|
120
|
+
if (options.dogfoodRequired && !options.isReadOnly) {
|
|
121
|
+
sections.push(
|
|
122
|
+
"- Dogfood Gate: This repository requires a recorded end-to-end run on real hardware before a PR opens. Run the repository's dogfood runner for the change, then record the observed checks with `record_dogfood`. Commit first: the evidence binds to the commit and tracked worktree content, so a later tracked edit or commit makes it stale.",
|
|
123
|
+
);
|
|
124
|
+
}
|
|
125
|
+
|
|
109
126
|
if (options.recoveryCheckpointsEnabled) {
|
|
110
127
|
sections.push(
|
|
111
128
|
"- Recovery: If workspace state needs rollback to the pre-run checkpoint, use `guard_recovery_restore`.",
|
|
@@ -272,9 +289,16 @@ export function createCustomTools(options: {
|
|
|
272
289
|
// while any tracked edit or new commit does.
|
|
273
290
|
const reviewFresh = Boolean(lastR && reviewEvidenceRecord && lastR.passed && isEvidenceFresh(reviewEvidenceRecord, { ...subject, worktreeFingerprint: getTrackedWorktreeFingerprint(root), sessionID: lastR.targetSessionID }, 0));
|
|
274
291
|
const documentationRequired = isDocumentationRequired(root);
|
|
292
|
+
// Dogfood is evaluated through the same helper the PR preflight
|
|
293
|
+
// uses, so the two can never disagree about the gate.
|
|
294
|
+
const dogfoodGate = evaluateDogfoodGate(root);
|
|
295
|
+
const lastD = dogfoodGate.matched ?? getLastDogfoodResultForWorkspace(root);
|
|
296
|
+
const lastDogfoodEvidence = lastD ? dogfoodEvidence(lastD) : undefined;
|
|
297
|
+
const dogfoodFresh = Boolean(dogfoodGate.required && dogfoodGate.missingHosts.length === 0);
|
|
275
298
|
const outstandingRequirements = [
|
|
276
299
|
...(verifyCommand && !verifyFresh ? ["verification"] : []),
|
|
277
300
|
...(reviewRequired && !reviewFresh ? ["review"] : []),
|
|
301
|
+
...(dogfoodGate.required && !dogfoodFresh ? ["dogfood"] : []),
|
|
278
302
|
...(documentationRequired && !branchHasDocumentationChange(root) ? ["documentation"] : []),
|
|
279
303
|
];
|
|
280
304
|
const recommendedActions: string[] = [];
|
|
@@ -284,6 +308,9 @@ export function createCustomTools(options: {
|
|
|
284
308
|
if (outstandingRequirements.includes("review")) {
|
|
285
309
|
recommendedActions.push("Call guard_review_rubric to retrieve the 5-axis review rubric, then spawn a secondary reviewer subagent via task to evaluate changes and record verdict with record_review.");
|
|
286
310
|
}
|
|
311
|
+
if (outstandingRequirements.includes("dogfood")) {
|
|
312
|
+
recommendedActions.push(`Run the repository's dogfood runner for this change, then record the result with record_dogfood.${dogfoodGate.missingHosts.length > 0 ? ` Hosts still missing a passing run: ${dogfoodGate.missingHosts.join(", ")}.` : ""}`);
|
|
313
|
+
}
|
|
287
314
|
if (outstandingRequirements.includes("documentation")) {
|
|
288
315
|
recommendedActions.push("Update documentation files (e.g. README.md, docs/) to satisfy the documentation gate before opening a PR.");
|
|
289
316
|
}
|
|
@@ -297,7 +324,7 @@ export function createCustomTools(options: {
|
|
|
297
324
|
}
|
|
298
325
|
const ralphOutcome = runWithRuntimeState(root, client, () => getRalphOutcome(toolContext.sessionID));
|
|
299
326
|
const taskEvidence = await runWithRuntimeState(root, client, () => todoEvidenceReport(toolContext.sessionID));
|
|
300
|
-
return JSON.stringify({ pluginVersion: loadedPluginVersion() ?? null, qualifiedHostVersion: qualifiedHostVersion() ?? null, hostVersion: hostVersion?.() ?? null, workspaceRoot: root, branch, onProtectedBranch: isProtected, todoCapability: todoCapability ? todoCapability() : "available", taskEvidence, outstandingRequirements, recommendedActions, lastMutationTimestamp: lastMut, mutationCount: getWorkspaceMutationCount(root), lastVerify: lastV && verifyEvidence ? { command: lastV.command, passed: lastV.passed, fresh: verifyFresh, evidenceId: verifyEvidence.id, commitHash: lastV.commitHash } : null, lastReview: lastR && reviewEvidenceRecord ? { reviewer: lastR.reviewer, passed: lastR.passed, summary: lastR.summary, fresh: reviewFresh, evidenceId: reviewEvidenceRecord.id } : null, ralph: { enabled: isRalphModeEnabled(root), maxIterations: getRalphMaxIterations(root), outcome: ralphOutcome ?? null }, projectConfig: { profile: getOperationProfile(root), protectedBranches: cfg.protectedBranches ?? ["main", "master"], verifyCommand: verifyCommand ?? null, requireReview: reviewRequired, requireDocumentation: documentationRequired, requireSubagentReview: isSubagentReviewRequired(root), recoveryCheckpoints: isRecoveryCheckpointsEnabled(root) }, gitHygiene: gitHygieneSnapshot(root) ?? null }, null, 2);
|
|
327
|
+
return JSON.stringify({ pluginVersion: loadedPluginVersion() ?? null, qualifiedHostVersion: qualifiedHostVersion() ?? null, hostVersion: hostVersion?.() ?? null, workspaceRoot: root, branch, onProtectedBranch: isProtected, todoCapability: todoCapability ? todoCapability() : "available", taskEvidence, outstandingRequirements, recommendedActions, lastMutationTimestamp: lastMut, mutationCount: getWorkspaceMutationCount(root), lastVerify: lastV && verifyEvidence ? { command: lastV.command, passed: lastV.passed, fresh: verifyFresh, evidenceId: verifyEvidence.id, commitHash: lastV.commitHash } : null, lastReview: lastR && reviewEvidenceRecord ? { reviewer: lastR.reviewer, passed: lastR.passed, summary: lastR.summary, fresh: reviewFresh, evidenceId: reviewEvidenceRecord.id } : null, lastDogfood: lastD && lastDogfoodEvidence ? { scenarioId: lastD.scenarioId, verdict: lastD.verdict, host: lastD.host.label, taxonomyClass: lastD.taxonomyClass ?? null, runCount: lastD.runCount, fresh: dogfoodGate.required ? dogfoodFresh : null, evidenceId: lastDogfoodEvidence.id, missingHosts: dogfoodGate.missingHosts, coveredHosts: (dogfoodGate.matchedRecords ?? []).map((record) => record.host.label) } : null, ralph: { enabled: isRalphModeEnabled(root), maxIterations: getRalphMaxIterations(root), outcome: ralphOutcome ?? null }, projectConfig: { profile: getOperationProfile(root), protectedBranches: cfg.protectedBranches ?? ["main", "master"], verifyCommand: verifyCommand ?? null, requireReview: reviewRequired, requireDocumentation: documentationRequired, requireSubagentReview: isSubagentReviewRequired(root), requireDogfood: dogfoodGate.required, dogfoodDeployMatrix: dogfoodGate.required ? getDogfoodDeployMatrix(root) : [], recoveryCheckpoints: isRecoveryCheckpointsEnabled(root) }, gitHygiene: gitHygieneSnapshot(root) ?? null }, null, 2);
|
|
301
328
|
},
|
|
302
329
|
}),
|
|
303
330
|
guard_audit: tool({ description: "View recent audit entries recorded by opencode-workflow-guard. Use to diagnose policy decisions, blocked commands, or outcome telemetry.", args: { limit: tool.schema.number().optional().describe("Maximum entries to return (default 10)") }, execute: async (args) => JSON.stringify(getRecentAuditEntries(typeof args?.limit === "number" ? Math.min(args.limit, 50) : 10), null, 2) }),
|
|
@@ -371,6 +398,130 @@ export function createCustomTools(options: {
|
|
|
371
398
|
: `[workflow-guard] Review recorded as CHANGES REQUESTED by ${args.reviewer}. ${boundTo} ${recorderNote}`;
|
|
372
399
|
},
|
|
373
400
|
}),
|
|
401
|
+
record_dogfood: tool({
|
|
402
|
+
description: "Record the verdict of a dogfood run: a change exercised end to end on real hardware before deployment. The runner produces the checks and the transcript; this tool binds the result to the repository content it ran against. A HOLD records the taxonomy class so the failure is diagnosable. Actor sessions spawned by the runner cannot record their own verdict.",
|
|
403
|
+
args: {
|
|
404
|
+
scenarioId: tool.schema.string().describe("Scenario identifier the actor ran, for example 'permission-rule-reorder'"),
|
|
405
|
+
scenarioSha256: tool.schema.string().describe("SHA-256 of the scenario file the actor ran, so the evidence names the frozen input"),
|
|
406
|
+
taxonomySha256: tool.schema.string().describe("SHA-256 of the frozen taxonomy file the judge classified against"),
|
|
407
|
+
verdict: tool.schema.string().describe("'pass' when every deterministic check passed, otherwise 'hold'"),
|
|
408
|
+
checks: tool.schema.string().describe("JSON array of check rows: [{id, status: 'pass'|'hold'|'skip', expected, observed, detail}]"),
|
|
409
|
+
hostLabel: tool.schema.string().describe("Stable host label the deploy matrix is written against, for example 'bluefin'"),
|
|
410
|
+
taxonomyClass: tool.schema.string().optional().describe("Taxonomy class naming why a HOLD happened; omit on a pass"),
|
|
411
|
+
runCount: tool.schema.number().optional().describe("Number of runs behind this verdict (default 1); single runs are flaky"),
|
|
412
|
+
hostOs: tool.schema.string().optional().describe("Host OS reported by the run"),
|
|
413
|
+
hostArch: tool.schema.string().optional().describe("Host architecture reported by the run"),
|
|
414
|
+
opencodeVersion: tool.schema.string().optional().describe("OpenCode version the actor session ran under"),
|
|
415
|
+
actorModel: tool.schema.string().optional().describe("Model pin used for the actor session"),
|
|
416
|
+
judgeModel: tool.schema.string().optional().describe("Model pin used for the judge session"),
|
|
417
|
+
stagedTreeHash: tool.schema.string().optional().describe("Hash of the staged harness tree the actor exercised"),
|
|
418
|
+
directory: tool.schema.string().optional().describe("Repository directory the run covered. Pass it explicitly when the recorder session has no workspace binding."),
|
|
419
|
+
},
|
|
420
|
+
execute: async (args, toolContext) => {
|
|
421
|
+
const auditVerdict = (verdict: "pass" | "hold" | "rejected", reason: string, binding?: NonNullable<AuditEntry["evidence"]>["binding"]) => audit({ ts: new Date().toISOString(), sessionID: toolContext.sessionID, tool: "record_dogfood.verdict", decision: verdict === "rejected" ? "block" : "allow", phase: "event", reason, evidence: { dogfoodVerdict: verdict, ...(binding ? { binding } : {}) } });
|
|
422
|
+
// The tested session cannot certify itself. The marker is set by
|
|
423
|
+
// the runner on the actor it spawns, so this is a refusal, not an
|
|
424
|
+
// absence: the plugin is present and enforcing in that session.
|
|
425
|
+
if (isDogfoodActor()) {
|
|
426
|
+
auditVerdict("rejected", "dogfood_actor_self_certification", { recorderSessionID: toolContext?.sessionID });
|
|
427
|
+
return `[workflow-guard] Dogfood record rejected: this session is a dogfood actor (WORKFLOW_GUARD_DOGFOOD_ACTOR/DEPTH is set), and the tested session cannot certify itself. The parent runner session records the verdict from the checks it observed.`;
|
|
428
|
+
}
|
|
429
|
+
const verdict = args.verdict.trim().toLowerCase();
|
|
430
|
+
if (verdict !== "pass" && verdict !== "hold") {
|
|
431
|
+
auditVerdict("rejected", "invalid_dogfood_verdict");
|
|
432
|
+
return "[workflow-guard] Dogfood record rejected: verdict must be 'pass' or 'hold'.";
|
|
433
|
+
}
|
|
434
|
+
// Empty identifiers would make the record unattributable: a scenario
|
|
435
|
+
// id names the frozen input, and a host label is what the deploy
|
|
436
|
+
// matrix is written against.
|
|
437
|
+
if (!args.scenarioId.trim() || !args.hostLabel.trim()) {
|
|
438
|
+
auditVerdict("rejected", "dogfood_missing_identity");
|
|
439
|
+
return "[workflow-guard] Dogfood record rejected: scenarioId and hostLabel must both be non-empty, because they identify the frozen input and the host the deploy matrix names.";
|
|
440
|
+
}
|
|
441
|
+
let checks: DogfoodCheck[];
|
|
442
|
+
try {
|
|
443
|
+
const parsed = JSON.parse(args.checks);
|
|
444
|
+
if (!Array.isArray(parsed)) throw new Error("not-array");
|
|
445
|
+
checks = parsed;
|
|
446
|
+
} catch {
|
|
447
|
+
auditVerdict("rejected", "invalid_dogfood_checks");
|
|
448
|
+
return "[workflow-guard] Dogfood record rejected: checks must be a JSON array of {id, status, expected, observed, detail} rows.";
|
|
449
|
+
}
|
|
450
|
+
const invalidCheck = checks.find((check) => !check || typeof check.id !== "string" || !["pass", "hold", "skip"].includes(check.status));
|
|
451
|
+
if (invalidCheck) {
|
|
452
|
+
auditVerdict("rejected", "invalid_dogfood_check_row");
|
|
453
|
+
return "[workflow-guard] Dogfood record rejected: every check row needs a string 'id' and a 'status' of pass, hold, or skip.";
|
|
454
|
+
}
|
|
455
|
+
// A record with no checks is a claim, not an observation. Without
|
|
456
|
+
// this, an empty array would slip past the per-row checks below
|
|
457
|
+
// because `[].some(...)` is false, and a pass would mean nothing.
|
|
458
|
+
if (checks.length === 0) {
|
|
459
|
+
auditVerdict("rejected", "dogfood_without_checks");
|
|
460
|
+
return "[workflow-guard] Dogfood record rejected: at least one check row is required. A verdict with no observed checks carries no information about the run.";
|
|
461
|
+
}
|
|
462
|
+
// A pass is only credible when every check actually ran. A skipped
|
|
463
|
+
// check is a HOLD by the design's own rule, so accepting it here
|
|
464
|
+
// would let the gate pass by omission.
|
|
465
|
+
if (verdict === "pass" && checks.some((check) => check.status !== "pass")) {
|
|
466
|
+
auditVerdict("rejected", "dogfood_pass_with_nonpassing_check");
|
|
467
|
+
return "[workflow-guard] Dogfood record rejected: a 'pass' verdict requires every check to be 'pass'. A skipped or held check is a HOLD.";
|
|
468
|
+
}
|
|
469
|
+
if (verdict === "hold" && !args.taxonomyClass) {
|
|
470
|
+
auditVerdict("rejected", "dogfood_hold_without_taxonomy_class");
|
|
471
|
+
return "[workflow-guard] Dogfood record rejected: a HOLD must name its taxonomy class so the failure is diagnosable.";
|
|
472
|
+
}
|
|
473
|
+
const hostProvidesWorkspace = Boolean(toolContext?.worktree);
|
|
474
|
+
const sessionBinding = await resolveSessionBoundWorkspace(toolContext?.sessionID);
|
|
475
|
+
if (!args.directory && !sessionBinding && !hostProvidesWorkspace) {
|
|
476
|
+
auditVerdict("rejected", "dogfood_binding_unresolved", { recorderSessionID: toolContext?.sessionID });
|
|
477
|
+
return "[workflow-guard] Dogfood record rejected: the run's workspace could not be resolved from this call (no explicit 'directory', no session workspace binding, no per-call worktree). Call record_dogfood with 'directory' set to the repository the run exercised.";
|
|
478
|
+
}
|
|
479
|
+
const dogfoodWorkspace = await resolveCallWorkspace(toolContext, args.directory);
|
|
480
|
+
const bindingFingerprint = getTrackedWorktreeFingerprint(dogfoodWorkspace);
|
|
481
|
+
if (!bindingFingerprint) {
|
|
482
|
+
auditVerdict("rejected", "dogfood_workspace_unbindingable", { workspace: dogfoodWorkspace, recorderSessionID: toolContext.sessionID });
|
|
483
|
+
return `[workflow-guard] Dogfood record rejected: the verdict cannot be bound because no tracked-content fingerprint is available for '${dogfoodWorkspace}' (not a git repository, or git is unavailable). The verdict was NOT recorded. Call record_dogfood with 'directory' set to the run's worktree, or run it from a session rooted there.`;
|
|
484
|
+
}
|
|
485
|
+
const parentSessionID = await runWithRuntimeState(effectiveRoot, client, () => fetchParentSessionID(toolContext.sessionID));
|
|
486
|
+
const bindingCommitHash = getCurrentGitCommitHash(dogfoodWorkspace);
|
|
487
|
+
const binding = {
|
|
488
|
+
workspace: dogfoodWorkspace,
|
|
489
|
+
commitHash: bindingCommitHash,
|
|
490
|
+
worktreeFingerprint: bindingFingerprint.slice(0, 16),
|
|
491
|
+
recorderSessionID: toolContext.sessionID,
|
|
492
|
+
recorderRole: parentSessionID ? ("subagent" as const) : ("root" as const),
|
|
493
|
+
targetSessionID: parentSessionID ?? toolContext.sessionID,
|
|
494
|
+
};
|
|
495
|
+
const recorded = recordDogfoodResult({
|
|
496
|
+
verdict: verdict as DogfoodResult["verdict"],
|
|
497
|
+
scenarioId: args.scenarioId,
|
|
498
|
+
scenarioSha256: args.scenarioSha256,
|
|
499
|
+
taxonomySha256: args.taxonomySha256,
|
|
500
|
+
taxonomyClass: args.taxonomyClass,
|
|
501
|
+
checks,
|
|
502
|
+
runCount: typeof args.runCount === "number" && Number.isFinite(args.runCount) && args.runCount >= 1 ? Math.floor(args.runCount) : 1,
|
|
503
|
+
host: {
|
|
504
|
+
label: args.hostLabel,
|
|
505
|
+
os: args.hostOs,
|
|
506
|
+
arch: args.hostArch,
|
|
507
|
+
opencodeVersion: args.opencodeVersion,
|
|
508
|
+
actorModel: args.actorModel,
|
|
509
|
+
judgeModel: args.judgeModel,
|
|
510
|
+
stagedTreeHash: args.stagedTreeHash,
|
|
511
|
+
},
|
|
512
|
+
workspace: dogfoodWorkspace,
|
|
513
|
+
}, parentSessionID ?? toolContext.sessionID);
|
|
514
|
+
auditVerdict(recorded.verdict, recorded.verdict === "pass" ? "dogfood_passed" : "dogfood_held", binding);
|
|
515
|
+
const boundTo = `Bound to ${dogfoodWorkspace} (commit ${bindingCommitHash?.slice(0, 12) ?? "unversioned"}, tracked fingerprint ${bindingFingerprint.slice(0, 12)}).`;
|
|
516
|
+
const recordedNote = parentSessionID
|
|
517
|
+
? `Recorded for parent session ${parentSessionID}.`
|
|
518
|
+
: `Recorded by root session ${toolContext.sessionID}.`;
|
|
519
|
+
const passedChecks = checks.filter((check) => check.status === "pass").length;
|
|
520
|
+
return recorded.verdict === "pass"
|
|
521
|
+
? `[workflow-guard] Dogfood recorded as PASS for scenario '${recorded.scenarioId}' on host '${recorded.host.label}' (${passedChecks}/${checks.length} checks passed, ${recorded.runCount} run(s)). ${boundTo} ${recordedNote}`
|
|
522
|
+
: `[workflow-guard] Dogfood recorded as HOLD for scenario '${recorded.scenarioId}' on host '${recorded.host.label}' with class '${recorded.taxonomyClass}'. A HOLD is not durable evidence: fix the finding and re-run. ${boundTo} ${recordedNote}`;
|
|
523
|
+
},
|
|
524
|
+
}),
|
|
374
525
|
guard_review_followups: tool({ description: "List durable local P2/P3 review follow-ups (technical debt) for this project. Proactively check during planning or before final verification to address open findings.", args: {}, execute: async () => JSON.stringify(followupStore ? listReviewFollowups(followupStore) : [], null, 2) }),
|
|
375
526
|
guard_review_followup_resolve: tool({ description: "Resolve a durable local review follow-up after the underlying issue has been fixed and verified.", args: { id: tool.schema.string() }, execute: async (args) => followupStore && resolveReviewFollowup(followupStore, args.id) ? `[workflow-guard] Review follow-up ${args.id} resolved.` : `[workflow-guard] Review follow-up ${args.id} was not open or was not found.` }),
|
|
376
527
|
guard_review_rubric: tool({
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
import { dogfoodEvidence } from "./evidence.ts";
|
|
2
|
+
import { getDogfoodDeployMatrix, getDogfoodRecordsForWorkspace, isDogfoodRequired } from "./state.ts";
|
|
3
|
+
import { getCurrentGitCommitHash, getTrackedWorktreeFingerprint } from "./verify.ts";
|
|
4
|
+
import type { DogfoodResult } from "./types.ts";
|
|
5
|
+
|
|
6
|
+
export interface DogfoodGateResult {
|
|
7
|
+
required: boolean;
|
|
8
|
+
/** Matrix hosts still missing a fresh passing record. */
|
|
9
|
+
missingHosts: string[];
|
|
10
|
+
/** The record that satisfied the gate, when one did. */
|
|
11
|
+
matched?: DogfoodResult;
|
|
12
|
+
/** Every record counted toward the gate, one per host. */
|
|
13
|
+
matchedRecords?: DogfoodResult[];
|
|
14
|
+
/** Why the gate is unsatisfied, cause-attributed. */
|
|
15
|
+
reason?: string;
|
|
16
|
+
/** Evidence id of the matched record, for audit correlation. */
|
|
17
|
+
evidenceId?: string;
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
/**
|
|
21
|
+
* Why one record cannot count for this worktree, or undefined when it can.
|
|
22
|
+
*
|
|
23
|
+
* Each cause is named separately because they have different remedies. A
|
|
24
|
+
* missing fingerprint is an environment problem, a fingerprint mismatch is a
|
|
25
|
+
* stale run, and a commit mismatch is an unrecorded commit. Collapsing them
|
|
26
|
+
* into one message sends the agent looking for the wrong fix.
|
|
27
|
+
*/
|
|
28
|
+
function stalenessReason(record: DogfoodResult, commitHash: string | undefined, fingerprint: string | undefined): string | undefined {
|
|
29
|
+
if (!fingerprint) {
|
|
30
|
+
return "Worktree fingerprint could not be computed (git unavailable or worktree unreadable), so dogfood evidence cannot be bound to this worktree; resolve worktree readability and re-run the dogfood scenario.";
|
|
31
|
+
}
|
|
32
|
+
if (!record.worktreeFingerprint) {
|
|
33
|
+
return `The recorded dogfood run ('${record.scenarioId}') has no worktree fingerprint binding (recorded against an unreadable worktree); re-run the dogfood scenario and record it with record_dogfood.`;
|
|
34
|
+
}
|
|
35
|
+
if (record.worktreeFingerprint !== fingerprint) {
|
|
36
|
+
return `The recorded dogfood run ('${record.scenarioId}' on '${record.host.label}') does not match the current worktree contents (tracked content changed after the run); re-run the dogfood scenario and record it with record_dogfood.`;
|
|
37
|
+
}
|
|
38
|
+
if (record.commitHash && commitHash && record.commitHash !== commitHash) {
|
|
39
|
+
return `The recorded dogfood run ('${record.scenarioId}' on '${record.host.label}') is bound to commit ${record.commitHash.slice(0, 12)} but this worktree is at commit ${commitHash.slice(0, 12)}; re-run the dogfood scenario at the current commit and record it with record_dogfood.`;
|
|
40
|
+
}
|
|
41
|
+
return undefined;
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
/**
|
|
45
|
+
* Judges the dogfood gate for one worktree.
|
|
46
|
+
*
|
|
47
|
+
* Shared by PR preflight and `guard_status` so the two can never disagree about
|
|
48
|
+
* whether the gate is satisfied. Freshness is content-scoped, matching review:
|
|
49
|
+
* a record covers the tracked tree it ran against, so untracked scratch
|
|
50
|
+
* cleanup cannot invalidate it while any tracked edit or new commit does.
|
|
51
|
+
*/
|
|
52
|
+
export function evaluateDogfoodGate(root: string): DogfoodGateResult {
|
|
53
|
+
if (!isDogfoodRequired(root)) return { required: false, missingHosts: [] };
|
|
54
|
+
const matrix = getDogfoodDeployMatrix(root);
|
|
55
|
+
const records = getDogfoodRecordsForWorkspace(root);
|
|
56
|
+
const commitHash = getCurrentGitCommitHash(root);
|
|
57
|
+
const fingerprint = getTrackedWorktreeFingerprint(root);
|
|
58
|
+
|
|
59
|
+
if (records.length === 0) {
|
|
60
|
+
return {
|
|
61
|
+
required: true,
|
|
62
|
+
missingHosts: matrix.length > 0 ? matrix : ["this host"],
|
|
63
|
+
reason: "Passing dogfood evidence is required (requireDogfood); run the dogfood runner and record the result with record_dogfood.",
|
|
64
|
+
};
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
// Only the newest record per host can count. An older pass is superseded by
|
|
68
|
+
// a newer run of the same host, including a newer HOLD, so a stale pass
|
|
69
|
+
// cannot hide a regression the same host later found. A same-millisecond
|
|
70
|
+
// tie resolves to the HOLD regardless of arrival order: re-running is cheap,
|
|
71
|
+
// and trusting a stale pass is the failure that matters.
|
|
72
|
+
const newestPerHost = new Map<string, DogfoodResult>();
|
|
73
|
+
for (const record of records) {
|
|
74
|
+
const seen = newestPerHost.get(record.host.label);
|
|
75
|
+
if (!seen || record.timestamp > seen.timestamp) {
|
|
76
|
+
newestPerHost.set(record.host.label, record);
|
|
77
|
+
} else if (record.timestamp === seen.timestamp && seen.verdict === "pass" && record.verdict === "hold") {
|
|
78
|
+
newestPerHost.set(record.host.label, record);
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
const unmatched: string[] = [];
|
|
83
|
+
const held: DogfoodResult[] = [];
|
|
84
|
+
const stale: Array<{ record: DogfoodResult; reason: string }> = [];
|
|
85
|
+
const matchedRecords: DogfoodResult[] = [];
|
|
86
|
+
for (const [host, record] of newestPerHost) {
|
|
87
|
+
// A host outside the deploy matrix cannot block the gate, so its HOLD or
|
|
88
|
+
// staleness must not be reported as the reason: that would name the wrong
|
|
89
|
+
// remedy. Only in-scope hosts collect their cause.
|
|
90
|
+
const inScope = matrix.length === 0 || matrix.includes(host);
|
|
91
|
+
if (record.verdict !== "pass") {
|
|
92
|
+
if (inScope) {
|
|
93
|
+
held.push(record);
|
|
94
|
+
unmatched.push(host);
|
|
95
|
+
}
|
|
96
|
+
continue;
|
|
97
|
+
}
|
|
98
|
+
const reason = stalenessReason(record, commitHash, fingerprint);
|
|
99
|
+
if (reason) {
|
|
100
|
+
if (inScope) {
|
|
101
|
+
stale.push({ record, reason });
|
|
102
|
+
unmatched.push(host);
|
|
103
|
+
}
|
|
104
|
+
continue;
|
|
105
|
+
}
|
|
106
|
+
if (!inScope) continue;
|
|
107
|
+
matchedRecords.push(record);
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
if (matrix.length > 0) {
|
|
111
|
+
for (const host of matrix) if (!newestPerHost.has(host)) unmatched.push(host);
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
if (unmatched.length > 0) {
|
|
115
|
+
const missingHosts = [...new Set(unmatched)];
|
|
116
|
+
// A HOLD is more actionable than a matrix gap, so name it first.
|
|
117
|
+
if (held.length > 0) {
|
|
118
|
+
const first = held[0];
|
|
119
|
+
return {
|
|
120
|
+
required: true,
|
|
121
|
+
missingHosts,
|
|
122
|
+
reason: `The most recent dogfood run for host '${first.host.label}' ('${first.scenarioId}') returned HOLD${first.taxonomyClass ? ` with class '${first.taxonomyClass}'` : ""}; fix the finding and re-run before opening a PR.`,
|
|
123
|
+
};
|
|
124
|
+
}
|
|
125
|
+
if (stale.length > 0) {
|
|
126
|
+
return { required: true, missingHosts, reason: stale[0].reason };
|
|
127
|
+
}
|
|
128
|
+
const outsideMatrix = [...newestPerHost.keys()].filter((host) => !matrix.includes(host));
|
|
129
|
+
if (matrix.length > 0 && matchedRecords.length === 0 && outsideMatrix.length > 0) {
|
|
130
|
+
return {
|
|
131
|
+
required: true,
|
|
132
|
+
missingHosts,
|
|
133
|
+
reason: `Dogfood evidence was recorded for host(s) ${outsideMatrix.join(", ")}, which are not in the deploy matrix (${matrix.join(", ")}); run the dogfood scenario on each matrix host and record each result.`,
|
|
134
|
+
};
|
|
135
|
+
}
|
|
136
|
+
return {
|
|
137
|
+
required: true,
|
|
138
|
+
missingHosts,
|
|
139
|
+
reason: `Dogfood evidence is missing for host(s) ${missingHosts.join(", ")} in the deploy matrix (${matrix.join(", ")}); run the dogfood scenario on each and record each result with record_dogfood.`,
|
|
140
|
+
};
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
// Satisfied. Report the newest contributing record, and keep the per-host
|
|
144
|
+
// set so the caller can journal every host that was actually covered.
|
|
145
|
+
const newest = matchedRecords.reduce<DogfoodResult | undefined>((best, record) => !best || record.timestamp > best.timestamp ? record : best, undefined);
|
|
146
|
+
return { required: true, missingHosts: [], matched: newest, matchedRecords, evidenceId: newest ? dogfoodEvidence(newest).id : undefined };
|
|
147
|
+
}
|