freeharness 0.4.2 → 0.4.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/NOTICE +2 -1
- package/README.md +20 -16
- package/canonical/HARVEST.yaml +2 -2
- package/canonical/PROVENANCE.yaml +2 -2
- package/canonical/agents/security-reviewer.md +8 -2
- package/canonical/hooks/freeharness-config-guard.mjs +644 -4567
- package/canonical/hooks/freeharness-cost-tracker.mjs +14 -27
- package/canonical/hooks/freeharness-precompact.mjs +12 -23
- package/canonical/hooks/freeharness-sdd-context.mjs +12 -23
- package/canonical/hooks/hooks.config-guard.claude-code.json +2 -3
- package/canonical/hooks/hooks.memory-guard.claude-code.json +2 -3
- package/canonical/skills/code-review/SKILL.md +2 -2
- package/canonical/skills/code-review/evals/cases.yaml +1 -1
- package/canonical/skills/subagent-driven-development/SKILL.md +4 -4
- package/canonical/skills/subagent-driven-development/evals/cases.yaml +149 -20
- package/canonical/skills/subagent-driven-development/reference/brief.md +15 -13
- package/canonical/skills/subagent-driven-development/reference/handoff.md +29 -15
- package/canonical/skills/subagent-driven-development/reference/loop.md +4 -5
- package/canonical/skills/subagent-driven-development/reference/workspace.md +2 -2
- package/canonical/skills/subagent-driven-development/scripts/validate-handoff.mjs +301 -1038
- package/dist/adapters/harness/claude-code.d.ts +7 -0
- package/dist/adapters/harness/claude-code.js +27 -21
- package/dist/adapters/harness/opencode.js +1 -1
- package/dist/adapters/harness/plugin.js +9 -9
- package/dist/adapters/memory/doctor.d.ts +11 -17
- package/dist/adapters/memory/doctor.js +55 -68
- package/dist/adapters/memory/paths.d.ts +3 -10
- package/dist/adapters/memory/paths.js +16 -54
- package/dist/adapters/memory/vendored.d.ts +2 -7
- package/dist/adapters/memory/vendored.js +12 -57
- package/dist/adapters/memory/worker-control.d.ts +9 -0
- package/dist/adapters/memory/worker-control.js +30 -0
- package/dist/cli/cleanup.d.ts +1 -1
- package/dist/cli/cleanup.js +29 -14
- package/dist/cli/cost.js +2 -2
- package/dist/cli/doctor.d.ts +9 -9
- package/dist/cli/doctor.js +21 -16
- package/dist/cli/hard-cleanup.js +1 -1
- package/dist/cli/install-facts.d.ts +4 -3
- package/dist/cli/install-facts.js +3 -1
- package/dist/cli/profile.d.ts +1 -1
- package/dist/cli/profile.js +10 -10
- package/dist/index.js +7 -5
- package/dist/infra/fsutil.d.ts +4 -5
- package/dist/infra/fsutil.js +5 -6
- package/node_modules/@freeharness/core/dist/registry.d.ts +1 -1
- package/node_modules/@freeharness/core/dist/registry.js +3 -3
- package/node_modules/@freeharness/core/package.json +1 -1
- package/node_modules/@freeharness/sdd/package.json +1 -1
- package/package.json +3 -3
- package/provenance/canonical.PROVENANCE.yaml +2 -2
- package/provenance/memory.PROVENANCE.yaml +18 -4
- package/canonical/openspec/config.base.yaml +0 -89
- package/canonical/rules/safety.md +0 -41
|
@@ -142,7 +142,7 @@ cases:
|
|
|
142
142
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
143
143
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
144
144
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
145
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
145
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
146
146
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
147
147
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
148
148
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -302,7 +302,7 @@ cases:
|
|
|
302
302
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
303
303
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
304
304
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
305
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
305
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
306
306
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
307
307
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
308
308
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -567,7 +567,7 @@ cases:
|
|
|
567
567
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
568
568
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
569
569
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
570
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
570
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
571
571
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
572
572
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
573
573
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -644,7 +644,7 @@ cases:
|
|
|
644
644
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
645
645
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
646
646
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
647
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
647
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
648
648
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
649
649
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
650
650
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -769,7 +769,7 @@ cases:
|
|
|
769
769
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
770
770
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
771
771
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
772
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
772
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
773
773
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
774
774
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
775
775
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -940,7 +940,7 @@ cases:
|
|
|
940
940
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
941
941
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
942
942
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
943
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
943
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
944
944
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
945
945
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
946
946
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -1137,7 +1137,7 @@ cases:
|
|
|
1137
1137
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
1138
1138
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
1139
1139
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
1140
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
1140
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
1141
1141
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
1142
1142
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
1143
1143
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -1318,7 +1318,7 @@ cases:
|
|
|
1318
1318
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
1319
1319
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
1320
1320
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
1321
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
1321
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
1322
1322
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
1323
1323
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
1324
1324
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -1463,7 +1463,7 @@ cases:
|
|
|
1463
1463
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
1464
1464
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
1465
1465
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
1466
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
1466
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
1467
1467
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
1468
1468
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
1469
1469
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -1712,7 +1712,7 @@ cases:
|
|
|
1712
1712
|
- type: regex
|
|
1713
1713
|
expected: "Effort: high"
|
|
1714
1714
|
- type: llm
|
|
1715
|
-
rubric: "Fail when the security brief runs on standard, at medium effort, or targets a separate high-effort agent; pass when it declares Class: security, Tier: strong and Effort: high, carries an allow-side probe
|
|
1715
|
+
rubric: "Fail when the security brief runs on standard, at medium effort, or targets a separate high-effort agent; pass when it declares Class: security, Tier: strong and Effort: high, carries an allow-side probe, and dispatches the single implementer agent with effort high passed as the dispatch effort."
|
|
1716
1716
|
ablation: true
|
|
1717
1717
|
|
|
1718
1718
|
- id: rule-route-code-standard
|
|
@@ -2003,7 +2003,7 @@ cases:
|
|
|
2003
2003
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
2004
2004
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
2005
2005
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
2006
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
2006
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
2007
2007
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
2008
2008
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
2009
2009
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -2558,7 +2558,7 @@ cases:
|
|
|
2558
2558
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
2559
2559
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
2560
2560
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
2561
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
2561
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
2562
2562
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
2563
2563
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
2564
2564
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -2699,7 +2699,7 @@ cases:
|
|
|
2699
2699
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
2700
2700
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
2701
2701
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
2702
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
2702
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
2703
2703
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
2704
2704
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
2705
2705
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -2860,7 +2860,7 @@ cases:
|
|
|
2860
2860
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
2861
2861
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
2862
2862
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
2863
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
2863
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
2864
2864
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
2865
2865
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
2866
2866
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -3067,7 +3067,7 @@ cases:
|
|
|
3067
3067
|
rubric: "Fail when the 1.4 brief carries either finding as a ruling, a design decision or a write-scope obligation; pass when the lead closes 1.3 by running openspec follow-up add once for each finding and the 1.4 digest and write scope hold neither."
|
|
3068
3068
|
ablation: true
|
|
3069
3069
|
|
|
3070
|
-
- id: rule-reviewer-shape-
|
|
3070
|
+
- id: rule-reviewer-shape-wontfix
|
|
3071
3071
|
source: rule
|
|
3072
3072
|
rule: evidence-blocking
|
|
3073
3073
|
input: |
|
|
@@ -3151,7 +3151,7 @@ cases:
|
|
|
3151
3151
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
3152
3152
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
3153
3153
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
3154
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
3154
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
3155
3155
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
3156
3156
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
3157
3157
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -3256,13 +3256,142 @@ cases:
|
|
|
3256
3256
|
test("listing the config directory is allowed", () => assert.equal(verdict("ls ~/.claude"), ""));
|
|
3257
3257
|
test("rm elsewhere is allowed", () => assert.equal(verdict("rm -rf build"), ""));
|
|
3258
3258
|
test("an empty command is allowed", () => assert.equal(verdict(""), ""));
|
|
3259
|
-
expected_regex: "
|
|
3259
|
+
expected_regex: "wontfix"
|
|
3260
3260
|
graders:
|
|
3261
3261
|
- type: regex
|
|
3262
3262
|
not_expected:
|
|
3263
3263
|
- "validate-handoff\\.mjs round-base"
|
|
3264
|
+
- "openspec follow-up add"
|
|
3265
|
+
- type: llm
|
|
3266
|
+
rubric: "Fail when the lead opens a fix round or records a follow-up for the reviewer-constructed quoting shape; pass when the lead closes it as wontfix citing the threat budget and accepts the task without a round."
|
|
3267
|
+
ablation: true
|
|
3268
|
+
|
|
3269
|
+
- id: rule-reviewer-shape-follow-up
|
|
3270
|
+
source: rule
|
|
3271
|
+
rule: evidence-blocking
|
|
3272
|
+
input: |
|
|
3273
|
+
Change copiloto-greenfield, task 2.5 (Class: security): every brief probe passes at HEAD,
|
|
3274
|
+
and the task reviewer reports one P1 on a secret shape it wrote itself, a fine-grained
|
|
3275
|
+
github_pat_ token that the memory redaction keeps, which no probe or spec scenario
|
|
3276
|
+
covers. State the lead's next step.
|
|
3277
|
+
workspace_files:
|
|
3278
|
+
- path: openspec/config.yaml
|
|
3279
|
+
content: |
|
|
3280
|
+
schema: spec-driven
|
|
3281
|
+
- path: openspec/changes/copiloto-greenfield/tasks.md
|
|
3282
|
+
content: |
|
|
3283
|
+
# Tasks: copiloto-greenfield
|
|
3284
|
+
|
|
3285
|
+
## 2. Memory
|
|
3286
|
+
|
|
3287
|
+
- [ ] 2.5 The memory sanitizer redacts a classic GitHub token before an observation is stored
|
|
3288
|
+
Verify: node --test test/redact.test.mjs
|
|
3289
|
+
Interfaces: src/redact.mjs exports redact(text: string) -> string
|
|
3290
|
+
Boundary: src/redact.mjs, test/redact.test.mjs
|
|
3291
|
+
Depends: none
|
|
3292
|
+
- path: src/redact.mjs
|
|
3293
|
+
content: |
|
|
3294
|
+
const GITHUB_TOKEN = /\bgh[pousr]_[A-Za-z0-9]{36}\b/g;
|
|
3295
|
+
|
|
3296
|
+
export function redact(text) {
|
|
3297
|
+
return String(text).replace(GITHUB_TOKEN, "[REDACTED]");
|
|
3298
|
+
}
|
|
3299
|
+
expected_regex: "openspec follow-up add"
|
|
3300
|
+
graders:
|
|
3301
|
+
- type: regex
|
|
3302
|
+
expected: "openspec follow-up add"
|
|
3303
|
+
- type: regex
|
|
3304
|
+
not_expected:
|
|
3305
|
+
- "validate-handoff\\.mjs round-base"
|
|
3306
|
+
- type: llm
|
|
3307
|
+
rubric: "Fail when the lead opens a fix round for the reviewer-constructed secret shape or closes it as wontfix; pass when the lead records it as a hardening follow-up carrying the reproducer and accepts the task without a round."
|
|
3308
|
+
ablation: true
|
|
3309
|
+
|
|
3310
|
+
- id: rule-security-review-no-trigger
|
|
3311
|
+
source: rule
|
|
3312
|
+
rule: security-review-trigger
|
|
3313
|
+
input: |
|
|
3314
|
+
Change cost-report-columns: task 1.1 is accepted, task 1.2 waits for its task review, and
|
|
3315
|
+
the branch diff touches only packages/cli/src/cli/cost.ts, packages/cli/test/cost.test.ts and
|
|
3316
|
+
Documents/ADVANCED.md. State the reviewers the lead dispatches for the task review of 1.2 and
|
|
3317
|
+
for the final branch review.
|
|
3318
|
+
workspace_files:
|
|
3319
|
+
- path: openspec/config.yaml
|
|
3320
|
+
content: |
|
|
3321
|
+
schema: spec-driven
|
|
3322
|
+
- path: openspec/changes/cost-report-columns/tasks.md
|
|
3323
|
+
content: |
|
|
3324
|
+
# Tasks: cost-report-columns
|
|
3325
|
+
|
|
3326
|
+
## 1. Cost report
|
|
3327
|
+
|
|
3328
|
+
- [x] 1.1 cost report prints a cache-read column
|
|
3329
|
+
Verify: node --test packages/cli/test/cost.test.ts
|
|
3330
|
+
Boundary: packages/cli/src/cli/cost.ts, packages/cli/test/cost.test.ts
|
|
3331
|
+
Depends: none
|
|
3332
|
+
- [ ] 1.2 Documents/ADVANCED.md describes the cache-read column
|
|
3333
|
+
Verify: node --test packages/cli/test/cost.test.ts
|
|
3334
|
+
Boundary: Documents/ADVANCED.md
|
|
3335
|
+
Depends: 1.1
|
|
3336
|
+
- path: packages/cli/src/cli/cost.ts
|
|
3337
|
+
content: |
|
|
3338
|
+
export function costRow(row: { model: string; input: number; cacheRead: number }): string {
|
|
3339
|
+
return `${row.model}\t${row.input}\t${row.cacheRead}`;
|
|
3340
|
+
}
|
|
3341
|
+
expected_regex: "trigger"
|
|
3342
|
+
graders:
|
|
3343
|
+
- type: regex
|
|
3344
|
+
not_expected:
|
|
3345
|
+
- "subagent_type\\W+security-reviewer"
|
|
3346
|
+
- "review-(final|\\d+_\\d+)-r\\d+-security"
|
|
3347
|
+
- type: llm
|
|
3348
|
+
rubric: "Fail when the lead dispatches security-reviewer in a task review or at the final branch review; pass when the lead plans the task review and the final branch review without security-reviewer because the diff touches no item of the trigger list."
|
|
3349
|
+
ablation: true
|
|
3350
|
+
|
|
3351
|
+
- id: rule-security-p0-blocks
|
|
3352
|
+
source: rule
|
|
3353
|
+
rule: evidence-blocking
|
|
3354
|
+
input: |
|
|
3355
|
+
Change memory-capture, final branch review: every brief probe passes at HEAD, and
|
|
3356
|
+
security-reviewer reports one P0 with a reproducer: the rules capture stores a classic
|
|
3357
|
+
ghp_ token unredacted in an observation title (src/capture.mjs:4; a Bash event whose command
|
|
3358
|
+
is git push https://ghp_<36 characters>@github.com/o/r is stored with that command as its
|
|
3359
|
+
title). State the lead's next step.
|
|
3360
|
+
workspace_files:
|
|
3361
|
+
- path: openspec/config.yaml
|
|
3362
|
+
content: |
|
|
3363
|
+
schema: spec-driven
|
|
3364
|
+
- path: openspec/changes/memory-capture/tasks.md
|
|
3365
|
+
content: |
|
|
3366
|
+
# Tasks: memory-capture
|
|
3367
|
+
|
|
3368
|
+
## 1. Capture
|
|
3369
|
+
|
|
3370
|
+
- [x] 1.1 The rules capture stores a Bash event as an observation titled by its command
|
|
3371
|
+
Verify: node --test test/capture.test.mjs
|
|
3372
|
+
Interfaces: src/capture.mjs exports observationOf(event) -> { title, narrative }
|
|
3373
|
+
Boundary: src/capture.mjs, test/capture.test.mjs
|
|
3374
|
+
Depends: none
|
|
3375
|
+
- path: src/capture.mjs
|
|
3376
|
+
content: |
|
|
3377
|
+
import { redact } from "./redact.mjs";
|
|
3378
|
+
|
|
3379
|
+
export function observationOf(event) {
|
|
3380
|
+
return { title: String(event.command).slice(0, 200), narrative: redact(String(event.output ?? "")) };
|
|
3381
|
+
}
|
|
3382
|
+
- path: src/redact.mjs
|
|
3383
|
+
content: |
|
|
3384
|
+
const GITHUB_TOKEN = /\bgh[pousr]_[A-Za-z0-9]{36}\b/g;
|
|
3385
|
+
|
|
3386
|
+
export function redact(text) {
|
|
3387
|
+
return String(text).replace(GITHUB_TOKEN, "[REDACTED]");
|
|
3388
|
+
}
|
|
3389
|
+
expected_regex: "round-base"
|
|
3390
|
+
graders:
|
|
3391
|
+
- type: regex
|
|
3392
|
+
expected: "validate-handoff\\.mjs round-base"
|
|
3264
3393
|
- type: llm
|
|
3265
|
-
rubric: "Fail when the lead
|
|
3394
|
+
rubric: "Fail when the lead closes the P0 as wontfix, records it only as a follow-up or accepts the branch with it open; pass when the lead rules it blocking and opens a fix round for it with round-base and a fix dispatch."
|
|
3266
3395
|
ablation: true
|
|
3267
3396
|
|
|
3268
3397
|
- id: rule-round-cap-remove-addition
|
|
@@ -3371,7 +3500,7 @@ cases:
|
|
|
3371
3500
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
3372
3501
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
3373
3502
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
3374
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
3503
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
3375
3504
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
3376
3505
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
3377
3506
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -3536,7 +3665,7 @@ cases:
|
|
|
3536
3665
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
3537
3666
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
3538
3667
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
3539
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
3668
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
3540
3669
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
3541
3670
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
3542
3671
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -23,15 +23,17 @@ the base removes, changes or inserts a line above the base file's last 5 lines,
|
|
|
23
23
|
replacing a closing line and appending entries pass. Briefs
|
|
24
24
|
dispatched together have disjoint scopes, and `scripts/validate-handoff.mjs briefs` refuses
|
|
25
25
|
the batch on overlap. The lead names the new brief files plus every brief file still in
|
|
26
|
-
flight, never the workspace directory: a directory argument skips the set checks
|
|
27
|
-
|
|
26
|
+
flight, never the workspace directory: a directory argument skips the set checks
|
|
27
|
+
(`tree-required`, resource overlap, manifest parallelism, the band and `tree-absent`),
|
|
28
|
+
since the workspace also holds landed briefs; `tree-path` and `setup-missing` stay
|
|
29
|
+
per-brief checks and run on each brief either way.
|
|
28
30
|
|
|
29
31
|
`Tier` and `Class` route the dispatch. `docs`, `extraction`, `exploration` and
|
|
30
32
|
`mechanical` (documentation, extraction, exploration, mechanical edits such as matrix
|
|
31
33
|
rows, renames, one-line values and transcription of complete plan text) run on `standard`
|
|
32
34
|
or `strong`; `summary` runs on `fast`, `standard` or `strong`, and `extraction` also on
|
|
33
|
-
`fast`; `security` (a
|
|
34
|
-
|
|
35
|
+
`fast`; `security` (a change to the memory sanitizer or redaction, the git guard deny list, the
|
|
36
|
+
config guard path judgment, or a datastore delete path; no task owes a hostile-input test) and `complex` (a parser or
|
|
35
37
|
state machine, concurrency, a cross-file contract, or a task that reached the round cap
|
|
36
38
|
before) run on `strong` with `Effort: high` only; `code` (a code change that is neither
|
|
37
39
|
`security` nor `complex`) runs on `strong`, or on `standard` only with a `Route:` header
|
|
@@ -52,9 +54,9 @@ scope; its entries count in the reading set. The size bounds are at most 3 write
|
|
|
52
54
|
files outside test and fixture directories and a reading set under 60,000 tokens at 4
|
|
53
55
|
bytes per token. `Ceiling` is optional, right after `Split` and `Read`: a positive
|
|
54
56
|
integer, the per-task cache-read token ceiling summed across every dispatch of the task
|
|
55
|
-
(impl, review and fix rounds alike), at or below the task's class ceiling:
|
|
56
|
-
`
|
|
57
|
-
|
|
57
|
+
(impl, review and fix rounds alike), at or below the task's class ceiling: 50,000,000 for
|
|
58
|
+
task `final` and 25,000,000 for any other task, `security` and `complex` alike, or the
|
|
59
|
+
owner's `Ceiling <class>:` value in the change's run authorization (reference/loop.md).
|
|
58
60
|
The header only lowers the class ceiling; `briefs` and `budget` refuse one above it.
|
|
59
61
|
|
|
60
62
|
`Route: C1-C7 - <reason>` is required on, and allowed only on, a brief with `Tier:
|
|
@@ -101,12 +103,12 @@ line without a reason), `changes-short <term>` (a term under 3 characters) and
|
|
|
101
103
|
check covers tests only, so a production file the plan missed is still found by reading
|
|
102
104
|
the design.
|
|
103
105
|
|
|
104
|
-
`Key-list` is optional,
|
|
106
|
+
`Key-list` is optional, placed after `Changes:` and its `Unaffected:` lines and before `Suite:`: `Key-list: <key>[, <key>...]`, one sibling
|
|
105
107
|
key per JSON object the task adds a key to. `briefs` also searches each `Changes` term
|
|
106
108
|
regex-escaped, and fails `changes-unscoped key-list:<key> <path>` for an uncovered test that
|
|
107
109
|
pins an exact key list naming the sibling (`Object.keys(` then a `deepEqual` array quoting it).
|
|
108
110
|
|
|
109
|
-
`Suite` is required,
|
|
111
|
+
`Suite` is required, after `Changes`, its `Unaffected:` lines and any `Key-list:`: the project's
|
|
110
112
|
full-suite command. The worker never runs it and runs only the `## Verify` commands;
|
|
111
113
|
`briefs` refuses a brief without it (`suite-missing`).
|
|
112
114
|
|
|
@@ -136,11 +138,11 @@ fails `scope`.
|
|
|
136
138
|
The `## Acceptance probes` section holds 1 to 12 lines `P<n>: <command, input or
|
|
137
139
|
observable> -> <expected verdict or outcome>`, written by the lead from the spec
|
|
138
140
|
scenarios and the design's edge cases before the dispatch. A `security` or `complex`
|
|
139
|
-
brief carries a probe whose expected outcome is an allow or a pass
|
|
140
|
-
a corpus, fixture replay or generator command over the changed behavior. A probe of a
|
|
141
|
+
brief carries a probe whose expected outcome is an allow or a pass. A probe of a
|
|
141
142
|
guard or hook verdict runs the hook through node on a payload file (the hook reading the
|
|
142
143
|
payload on stdin), never the probed command typed into Bash, so the session's own
|
|
143
|
-
permission layer never judges the probe: `node <hook>.mjs < payload.json -> <verdict>`.
|
|
144
|
+
permission layer never judges the probe: `node <hook>.mjs < payload.json -> <verdict>`. A guard carries one allow probe and one deny
|
|
145
|
+
probe, each run this way. The
|
|
144
146
|
`## Project rules` section holds 1 to 20 lines, since the workers run without CLAUDE.md:
|
|
145
147
|
the project CLAUDE.md invariants and protected boundaries that bear on the write scope,
|
|
146
148
|
plus the artifact rules every worker follows.
|
|
@@ -216,7 +218,7 @@ name test files, never a package's `npm test`, which the handoff guard reads as
|
|
|
216
218
|
- Never run the CLI, a hook or a worker against the real HOME; tests use a fake HOME.
|
|
217
219
|
- The reply is the JSON block plus the report path, under 8,000 characters.
|
|
218
220
|
- The reply and the report's first fenced json block follow this skeleton exactly: `{"task":"...","status":"completed|partial|blocked","files_changed":[...],"tests_run":[{"command":"...","result":"pass|fail|unavailable","summary":"..."}],"evidence":[...],"partial_output":null,"blocked_reason":null,"concerns":[]}`; the report file opens with it inside a ```json fence.
|
|
219
|
-
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name
|
|
221
|
+
- Cite every acceptance probe id (`P1:` ...) in `evidence` with its observed outcome; the first dispatch runs and cites every probe, and a fix round re-runs the probes its open findings name and cites each other probe as `P<n>: carried from round <n-1>`.
|
|
220
222
|
- An environment override (HOME, XDG_*) is set in the same shell command as the program it applies to; a separate export does not persist between tool calls.
|
|
221
223
|
- Edit files only with the Edit and Write tools, never through a Bash heredoc, `sed -i` or an interpreter script.
|
|
222
224
|
- Write a helper or probe script with the Write tool and run it as `node <file>`; a probe that makes a fake HOME write its `.claude/settings.json` or hooks runs from such a script, since the config guard refuses a Bash line naming those paths.
|
|
@@ -138,30 +138,44 @@ tree, while the brief, the report and the ledger stay under the main root's
|
|
|
138
138
|
- `tree-leak <path>`: the worker wrote the main tree instead of its own. Reported for a
|
|
139
139
|
write-scope path that differs in the main tree from its HEAD (an untracked file counts;
|
|
140
140
|
`--lead` paths are excepted), and with `--agent` for an Edit, MultiEdit, Write or
|
|
141
|
-
NotebookEdit path
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
as a round.
|
|
141
|
+
NotebookEdit path under the main root outside the tree and outside `<root>/.freeharness/`,
|
|
142
|
+
or in the report's change directory `<root>/.freeharness/sdd/<change>/` (only a tool write
|
|
143
|
+
to the report is spared), and for a git command that writes the main tree's index. It
|
|
144
|
+
returns to the worker as a round.
|
|
146
145
|
- `NOTE tree-cwd <command>`: with `--agent`, a Bash command that ran in a directory
|
|
147
146
|
outside the tree, typically one missing the `cd <Tree> &&` prefix. A note, not a
|
|
148
147
|
failure: the lead reads the command and judges whether it touched the main tree.
|
|
149
148
|
|
|
150
149
|
The change directory holds the ledger, the findings and the round copies the guard trusts,
|
|
151
|
-
so with `--agent` and without `--tree` a write there fails too. Only an Edit,
|
|
152
|
-
Write or NotebookEdit write to the report is spared, and only while the report is
|
|
153
|
-
file with one link at scan time: a symlink or hard link there would spare the file
|
|
154
|
-
|
|
155
|
-
naming the directory itself, or a Bash removal of the directory or a path in it fails as
|
|
156
|
-
`bash-write <path>`, and a tool path as
|
|
150
|
+
so with `--agent` and without `--tree` a tool write there fails too. Only an Edit,
|
|
151
|
+
MultiEdit, Write or NotebookEdit write to the report is spared, and only while the report is
|
|
152
|
+
a regular file with one link at scan time: a symlink or hard link there would spare the file
|
|
153
|
+
it shares. Any other tool path there fails as
|
|
157
154
|
- `workspace-write <path>`: an Edit, MultiEdit, Write or NotebookEdit path in the report's
|
|
158
155
|
change directory other than a spared report. The path prints from the main root; every other
|
|
159
156
|
`.freeharness/` path, another change's directory included, stays free.
|
|
160
157
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
158
|
+
The transcript check is literal: the diff guards and the lead's diff read are the controls,
|
|
159
|
+
and the scan only reports. It splits a Bash command on `;`, `&&`, `||` and `|` and each
|
|
160
|
+
part on whitespace, reads no quoting, no interpreter body and no file system, and fails only
|
|
161
|
+
- `index-write <command>`: a literal `git` command word (past assignments and wrapper words
|
|
162
|
+
such as `sudo -u me` or `env`) followed, after its global options, by a
|
|
163
|
+
subcommand that writes the index or HEAD (`add`, `rm`, `mv`, `commit`, `stash`, `reset`,
|
|
164
|
+
`restore --staged`, `checkout --`, `apply --index`, `switch`, `merge`, `rebase`,
|
|
165
|
+
`cherry-pick`, `pull`, `update-index`) whose directory is the tree; a temporary repository
|
|
166
|
+
stays free.
|
|
167
|
+
- `suite-run`: a Bash command that contains the brief's `Suite:` command.
|
|
168
|
+
|
|
169
|
+
Every other Bash finding is a note:
|
|
170
|
+
- `NOTE bash-path <path>`: a Bash command naming a path in the change directory or, with
|
|
171
|
+
`--tree`, under the main root outside the tree, its git admin dir and its `.freeharness/`,
|
|
172
|
+
or writing (a redirect, `tee`, `sed -i`, `cp`, `mv`, `install`, `rm`, `touch`, `ln`; an `mv`
|
|
173
|
+
source included, a `cp`, `install` or `ln` source only named) a path in the tree or its admin dir outside the write
|
|
174
|
+
scope and `.freeharness/`, or outside the tree. The lead reads it against the diff.
|
|
175
|
+
- `NOTE scan-capped <count>`: the lines past the fifty one kind prints.
|
|
176
|
+
- `NOTE transcript-unavailable <no-ledger|no-line|outside-projects|foreign-path|unreadable>`,
|
|
177
|
+
`NOTE transcript-truncated <agent_id>` and `NOTE transcript-tree-unresolved <error>`: the transcript
|
|
178
|
+
was not found, not read in full, or the tree not resolved by git.
|
|
165
179
|
|
|
166
180
|
`scripts/validate-handoff.mjs recover <transcript.jsonl> <report> [--force]` rebuilds the handoff
|
|
167
181
|
when a dispatch ends without a clean reply. Recovery reads only assistant-authored text:
|
|
@@ -7,23 +7,22 @@ Brief sizing and the budget, dispatch names, the handoff guards, fix rounds, the
|
|
|
7
7
|
- Sizing, before every brief: a brief covers one behavior (the scenarios of one requirement, or one cohesive part), with at most 3 write-scope files outside test and fixture directories and a reading set (write-scope files plus the `Read:` header's entries) under 60,000 tokens at 4 bytes per token. A file over 1,500 lines is named by range (`path:120-340`) or symbol (`path#symbolName`, counted as 200 lines) in the write scope or `Read:`. A task above a bound is split into sub-tasks `<task>.<n>` (`2.4.1`, `2.4.2`), each with its own brief, dispatch, review and acceptance; the first sub-task's review package carries a `## Split` note naming the sub-tasks and the behavior each owns, and the parent is ticked in `tasks.md` only when every sub-task is accepted. `Split: refused - <reason>` is written only when the work cannot be divided; `validate-handoff.mjs briefs` refuses an oversized brief without it.
|
|
8
8
|
- Pin grep: `briefs` searches each `Changes:` term plain and regex-escaped and, with `Key-list:`, finds exact key-list pins; the lead still greps by hand for a pinned count (a bare number) and puts each hit in the write scope or an `Unaffected:` line; a brief that changes a USAGE line or skill text runs content.test.ts in its Verify.
|
|
9
9
|
- Changed terms: `Changes:` names every identifier and literal value the task changes, and `validate-handoff.mjs briefs` refuses a brief whose write scope, Verify commands or `Unaffected:` lines miss a test that names one. The check covers tests only, so the lead still adds the production files a plan missed by reading the design.
|
|
10
|
-
- Class ceilings: `budget` stops a task at its class ceiling of cache-read tokens:
|
|
10
|
+
- Class ceilings: `budget` stops a task at its class ceiling of cache-read tokens: 50,000,000 for task `final` and 25,000,000 for any other task, `security` and `complex` alike. The class is `security` or `complex` when a brief declaring the task carries that `Class:` (the higher ceiling when both appear). It trips when the total plus a reserve, the median of the task's dispatches (the lower middle value for an even count), reaches the ceiling, so a typical next dispatch never crosses it. The output names the source of the ceiling: `class <class> default`, `class <class> authorized`, `brief <file>` or `flag`.
|
|
11
11
|
- Run authorization: the owner may set the ceilings and the round cap in one section of the change's `proposal.md` under the exact heading `## Run authorization`, approved with the proposal: `Ceiling <class>: <n>` (`<class>` one of `security`, `complex`, `final`, `other`; `<n>` a positive integer) and `Rounds: <1-9>`, each key at most once. Without it the class defaults and a cap of 3 rounds apply; a malformed section fails `budget`, `briefs` and `round-base` naming proposal.md and the line: `duplicate-section` (a second such heading), `near-miss-heading` (a `## Run authorization` heading in another case or with more words), `unknown-line` (any other non-blank line in the section), `duplicate-key` or `bad-value`. Only the owner edits it; the lead never raises a ceiling or the cap.
|
|
12
12
|
- A brief's `Ceiling:` header only lowers the class ceiling: the highest one among the task's briefs applies at or below it, and one above it fails `budget` and `briefs`. `--ceiling` on `budget` overrides every other source and is the owner's to pass.
|
|
13
13
|
|
|
14
14
|
## Dispatch names
|
|
15
15
|
|
|
16
|
-
- Write `_` for `.` because the
|
|
16
|
+
- Write `_` for `.` because the ledger and the `Dispatch:` line use the underscore form and `validate-handoff.mjs name` enforces it; the ledger stores the dotted id and reads role, task and round through the slug.
|
|
17
17
|
- The lens slug is lowercase letters, digits and hyphens (`review-final-r1-lenses-b`); any other character leaves role and task null in the ledger.
|
|
18
18
|
|
|
19
19
|
## Guards before the reviewer
|
|
20
20
|
|
|
21
21
|
- Run `validate-handoff.mjs handoff <report> --base <Base> --brief <brief.md> --agent <agent_id>`, where `<agent_id>` is the id the Agent tool result returns, adding `--in-flight <brief.md>` for every other brief still running so a peer's files are not charged to this worker, and `--lead <path>` for the lead's own paths (the change's `tasks.md`, any review package outside `.freeharness/`) so they are not charged either.
|
|
22
22
|
- A `claimed-unchanged`, `changed-unclaimed`, `scope`, `peer-scope` or `lead-path` failure, a `probe-missing`, or a `checkpoint-missing` or `checkpoint-oversized` failure returns to the worker as findings (a round), never straight to the reviewer; `peer-scope` means the worker claimed a file another dispatch owns, `lead-path` means it claimed one of the lead's own paths, neither ever worth passing on.
|
|
23
|
-
- With `--agent`, the command also scans the worker's transcript and fails `
|
|
23
|
+
- With `--agent`, the command also scans the worker's transcript literally and fails `index-write <command>` (a git command wrote the index or HEAD), `suite-run` (the worker ran the brief's `Suite:` command), `workspace-write <path>` (a tool write into the change directory in a shared-tree run) and the tree checks (`tree-path`, `tree-leak <path>`). It prints these NOTE lines: `NOTE bash-path <path>` (a Bash command named a path in the change directory or outside the tree, or wrote a path outside the write scope; the lead reads it against the diff), `NOTE tree-cwd <command>` (a Bash command ran in a directory outside the tree), `NOTE scan-capped <count>` (the lines past the fifty one kind prints), `NOTE transcript-unavailable <no-ledger|no-line|outside-projects|foreign-path|unreadable>` (a ledger path for the agent was not scanned; `foreign-path` names a file other than `agent-<agent_id>.jsonl` in a `subagents` directory, and every accepted path is still scanned), `NOTE transcript-truncated <agent_id>` (only part of the transcript was read) and `NOTE transcript-tree-unresolved <error>` (git named no working tree, so the current directory stands in for it).
|
|
24
24
|
- Each of these failures returns to the worker as a round, like the other guard failures. The lead may rule a false positive (for example a fixture generator run inside the tree on purpose): it reads the command in the transcript and writes a ruling line in the review package before the reviewer runs, never a silent pass.
|
|
25
|
-
- Threat
|
|
26
|
-
- Evidence test: on a `security` or `complex` task a P1 blocks only when it fails a brief probe, a spec scenario, or the corpus probe against the base (a deny the base prints that the task's code no longer prints); a shape the reviewer constructs is `hardening` with its reproducer, recorded as a follow-up.
|
|
25
|
+
- Threat budget: security work covers four threats: a secret or private-tagged text stored in memory and replayed, a history-destroying git command, a subagent writing live configuration or deleting a protected datastore, and a subagent writing outside its task scope. Constructed bypass shapes (quoting, tokenizer, wrapper, expansion, heredoc, alias), hostile multi-MiB input, symlink, FIFO and forged `.git` shapes, races with other local processes and proxy or environment edge cases are out of scope: a finding of such a shape closes as wontfix on sight, citing this budget, and is never a follow-up or a round. A shape a reviewer constructs inside the four threats that no probe covers is `hardening` with its reproducer, recorded as a follow-up, never a round. A security finding blocks only as a P0 of an in-scope threat or a P1 `defect` against a kept requirement with a reproducer. `security-reviewer` runs only when the diff touches the memory sanitizer or its secret redaction, the git guard deny list, the config guard path judgment, a settings-merge, uninstall or cleanup path that touches a datastore or a foreign hook entry, a new network call from a hook or the memory engine, or harvested third-party content entering `canonical/`; it runs at most once per change, at the final branch review, never in a task review. A `security` task, or a final-review fix loop opened by a security finding, asks the owner after 2 fix rounds with a blocking finding still open, whatever the round cap. No task owes a hostile-input test.
|
|
27
26
|
- Guard false positive: a worker that returns `blocked` with `needs-confirm` because the config or git guard denied a Verify or probe line that names no guarded path is answered by the lead rerunning that line itself in the task's tree, then re-dispatching the same round only to rerun Verify and write a completed report, with no code change, no new ruling id and no `round-base` call; the denied shape is recorded once as a follow-up. A line that names a guarded path keeps the stop: the owner opens the config-edit window or the task is re-scoped.
|
|
28
27
|
- `transcript-unavailable` with `no-ledger` or `no-line` happens when the dispatch has no ledger line (the other reasons are listed above): the lead dispatches from the repository root so the hooks find the project.
|
|
29
28
|
|
|
@@ -15,8 +15,8 @@ The run-start step and the validator inventory of the subagent-driven-developmen
|
|
|
15
15
|
The validator `scripts/validate-handoff.mjs` has fourteen subcommands:
|
|
16
16
|
|
|
17
17
|
- `brief-init <change> <task>`: writes `.freeharness/sdd/<change>/<task>.brief.md` from the task's line and annotation lines in `openspec/changes/<change>/tasks.md` and the template in reference/brief.md, with `Change:`, `Spec:`, `Task:`, `Base:` and `Report:` set, `## Run rules` and `## Handoff` copied, and one `<fill: ...>` line for every other header and section, then prints `OK brief-init <path>`; refuses `task-shape`, `tasks-missing`, `task-unknown`, `template-missing`, `workspace-escape` (the workspace directory resolves outside the project root), `brief-exists` and `brief-unwritable`, and `briefs` fails `placeholder-left <name>` while a placeholder remains.
|
|
18
|
-
- `briefs <dir|file...> [--band small|medium|large]`: headers, one task id, routing class and `Effort:`, size bounds, probes, `## Project rules`, digest budget, write-scope overlap; over the named brief files also the set checks `tree-required`, `tree-path`, `tree-absent`, `setup-missing`, `resource-overlap`, `manifest-parallel` and `band <count> over <cap>` (small when `--band` is absent); a directory argument skips
|
|
19
|
-
- `handoff <file> [--base <sha>] [--brief <brief.md>] [--in-flight <brief.md>]... [--lead <path>]... [--agent <agent_id>] [--tree <path>]`: schema plus the diff guards; `--base` adds `staged` and `base-not-ancestor`; `--lead` excludes the lead's own paths; `--brief` adds `probe-missing`, `checkpoint-missing` and `checkpoint-oversized`; `--agent`
|
|
18
|
+
- `briefs <dir|file...> [--band small|medium|large]`: headers, one task id, routing class and `Effort:`, size bounds, probes, `## Project rules`, digest budget, write-scope overlap; over the named brief files also the set checks `tree-required`, `tree-path`, `tree-absent`, `setup-missing`, `resource-overlap`, `manifest-parallel` and `band <count> over <cap>` (small when `--band` is absent); a directory argument skips `tree-required`, `resource-overlap`, `manifest-parallel`, `band` and `tree-absent` with `NOTE briefs-dir <count>`, while `tree-path` and `setup-missing` stay per-brief.
|
|
19
|
+
- `handoff <file> [--base <sha>] [--brief <brief.md>] [--in-flight <brief.md>]... [--lead <path>]... [--agent <agent_id>] [--tree <path>]`: schema plus the diff guards; `--base` adds `staged` and `base-not-ancestor`; `--lead` excludes the lead's own paths; `--brief` adds `probe-missing`, `checkpoint-missing` and `checkpoint-oversized`; `--agent` checks the worker's transcript literally for `workspace-write`, `index-write` and `suite-run`, noting `bash-path` and `NOTE tree-cwd` (a Bash command ran outside the tree); `--tree` reads the task's tree and adds `tree-leak`.
|
|
20
20
|
- `budget <ledger.jsonl> <task> [--ceiling <n>]`: the total, the median reserve and the class ceiling with its source (reference/loop.md).
|
|
21
21
|
- `ledger <change-dir>`.
|
|
22
22
|
- `recover <transcript.jsonl> <report> [--force]`.
|