harness-everything 0.42.0 → 0.42.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +18 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/README.md +68 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/green-final.log +162 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/green.log +158 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/live-installed-runtime-rejection.json +31 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/luna-action-family-cross-product.js +143 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/luna-action-family-cross-product.log +7 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/luna-codex-action-gate-prehook-smoke.js +78 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/luna-codex-action-gate-prehook-smoke.log +8 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/luna-review.md +151 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/original-rejection.json +31 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/red.log +158 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/review-descriptor-green.log +192 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/review-descriptor-red.log +192 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/review-family-green.log +206 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/review-family-red.log +206 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/review-hints-green.log +174 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/review-hints-red.log +174 -0
- package/benchmarks/results/codex-pr-metadata-2026-10-26/verification.md +31 -0
- package/benchmarks/results/doc-evidence-e1/README.md +68 -0
- package/benchmarks/results/doc-evidence-e1/evidence.json +360 -0
- package/benchmarks/results/doc-evidence-e1/pilot-report.json +175 -0
- package/benchmarks/results/h1-provider-independence-qualification/README.md +26 -3
- package/benchmarks/results/live-host/opencode-d1-d2-2026-10-25/README.md +30 -0
- package/benchmarks/results/live-host/opencode-d1-d2-2026-10-25/evidence.json +128 -0
- package/ci/action-gate.test.js +86 -0
- package/ci/lib/doc-reconciliation-prep-fixtures.js +5 -4
- package/ci/mechanism-2aa-codex-action-gate-policy.test.js +47 -0
- package/ci/mechanism-325-3b-s2b-wiring.test.js +11 -0
- package/ci/mechanism-task-git-authorization.test.js +12 -0
- package/docs/codex-action-gate.md +27 -1
- package/docs/doc-workflow-reconciliation.md +4 -3
- package/docs/document-lifecycle.md +3 -3
- package/docs/platform-capabilities.md +8 -2
- package/hooks/scripts/action-gate.js +35 -8
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/hooks/scripts/action-gate.js +35 -8
- package/plugins/harness-everything/plugin.json +1 -1
- package/sbom.cdx.json +7 -7
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
=== Issue #85 Phase 5 — Pre-action gate ===
|
|
2
|
+
PASS production action-gate rule table loads without fallback
|
|
3
|
+
PASS production rule table is non-empty and keeps the destructive baseline
|
|
4
|
+
PASS rule table contains git-force-push
|
|
5
|
+
PASS rule table contains git-reset-hard
|
|
6
|
+
PASS rule table contains git-clean-force-delete
|
|
7
|
+
PASS rule table contains recursive-delete
|
|
8
|
+
PASS rule table contains powershell-recursive-force-delete
|
|
9
|
+
PASS rule table contains sql-destructive-ddl
|
|
10
|
+
PASS rule table contains package-publish
|
|
11
|
+
PASS rule table contains github-release-create
|
|
12
|
+
PASS rule table contains production-deploy
|
|
13
|
+
PASS Claude force-push defers to the host permission flow without forcing a prompt
|
|
14
|
+
PASS deferred decision still names the matched rule
|
|
15
|
+
PASS matched Claude action is audited as deferred-to-host
|
|
16
|
+
PASS audit records matched rule
|
|
17
|
+
PASS audit hashes the exact command payload
|
|
18
|
+
PASS always-ask policy restores the forced Claude prompt
|
|
19
|
+
PASS ask decision uses Claude PreToolUse hookSpecificOutput
|
|
20
|
+
PASS ask reason names the matched rule
|
|
21
|
+
PASS always-ask action is audited as pending-approval
|
|
22
|
+
PASS git status passes untouched
|
|
23
|
+
PASS safe unmatched command creates no gate audit record
|
|
24
|
+
PASS permission_mode=default defers to the host without a forced prompt
|
|
25
|
+
PASS audit records permission_mode=default
|
|
26
|
+
PASS permission_mode=acceptEdits defers to the host without a forced prompt
|
|
27
|
+
PASS audit records permission_mode=acceptEdits
|
|
28
|
+
PASS permission_mode=plan defers to the host without a forced prompt
|
|
29
|
+
PASS audit records permission_mode=plan
|
|
30
|
+
PASS permission_mode=auto defers to the host without a forced prompt
|
|
31
|
+
PASS audit records permission_mode=auto
|
|
32
|
+
PASS permission_mode=dontAsk defers to the host without a forced prompt
|
|
33
|
+
PASS audit records permission_mode=dontAsk
|
|
34
|
+
PASS permission_mode=bypassPermissions defers to the host without a forced prompt
|
|
35
|
+
PASS audit records permission_mode=bypassPermissions
|
|
36
|
+
PASS permission_mode=missing defers to the host without a forced prompt
|
|
37
|
+
PASS audit records permission_mode=missing
|
|
38
|
+
PASS command classification gates destructive action even when router hint says not required
|
|
39
|
+
PASS router actionGate hint is recorded without controlling enforcement
|
|
40
|
+
PASS safe command is not gated solely because prompt router predicted risk
|
|
41
|
+
PASS structured MCP delete is gated before execution
|
|
42
|
+
PASS audit retains normalized structured action identity
|
|
43
|
+
PASS structured action audit binds to exact payload hash
|
|
44
|
+
PASS structured action audit does not persist raw secret arguments
|
|
45
|
+
PASS read-only MCP fixture passes without a false gate
|
|
46
|
+
PASS obvious structured mutation is classified from tool identity
|
|
47
|
+
PASS non-shell local-function destructive fixture uses the same structured gate
|
|
48
|
+
PASS unknown structured action fails closed when workflow requires a gate
|
|
49
|
+
PASS unknown structured action is not fabricated as destructive without a gate requirement
|
|
50
|
+
PASS changed structured payload cannot reuse a prior approval/audit decision
|
|
51
|
+
PASS PowerShell recursive force delete outside scratch is gated
|
|
52
|
+
PASS PowerShell recursive force delete inside session scratch is exempt
|
|
53
|
+
PASS rm -rf outside scratch is gated
|
|
54
|
+
PASS rm -rf inside OS temp is exempt
|
|
55
|
+
PASS git reset --hard HEAD matches git-reset-hard
|
|
56
|
+
PASS git clean -fd matches git-clean-force-delete
|
|
57
|
+
PASS psql -c "DROP TABLE users" matches sql-destructive-ddl
|
|
58
|
+
PASS npm publish matches package-publish
|
|
59
|
+
PASS gh release create v1.2.3 matches github-release-create
|
|
60
|
+
PASS npm run deploy matches production-deploy
|
|
61
|
+
PASS host without verified ask-style decision blocks with exit 2
|
|
62
|
+
PASS non-ask host block reason names the matched rule
|
|
63
|
+
PASS non-ask host block is audited as rejected
|
|
64
|
+
PASS internal action-gate error on Claude defers to the host instead of allowing or forcing a prompt
|
|
65
|
+
PASS internal error is shown to the user as a systemMessage without a permission decision
|
|
66
|
+
PASS internal error on Claude is audited as deferred-to-host
|
|
67
|
+
PASS internal action-gate error under always-ask fails closed to ask
|
|
68
|
+
PASS internal error on host without ask fails closed to block
|
|
69
|
+
PASS missing/invalid rule table visibly falls back to built-in defaults
|
|
70
|
+
PASS built-in fallback still classifies and audits destructive action
|
|
71
|
+
PASS fallback degradation is shown to the user as a systemMessage
|
|
72
|
+
PASS valid empty rule table does not silently activate built-in defaults
|
|
73
|
+
PASS empty-table mutation control disables detection, proving production non-empty assertion is meaningful
|
|
74
|
+
PASS null-prototype serialization preserves an own __proto__ key
|
|
75
|
+
PASS own __proto__ key changes the full-input digest
|
|
76
|
+
PASS PostToolUse closes a deferred payload as executed
|
|
77
|
+
PASS executed audit keeps exact approved payload hash
|
|
78
|
+
PASS PostToolUseFailure records attempted execution without pretending success
|
|
79
|
+
PASS post-tool payload mismatch is never accepted as the approved command
|
|
80
|
+
PASS Stop closes unexecuted deferred and pending payloads
|
|
81
|
+
PASS unexecuted deferred payload is audited as rejected
|
|
82
|
+
PASS unexecuted always-ask approval is audited as rejected
|
|
83
|
+
PASS hook process returns no decision for a matched command under the default policy
|
|
84
|
+
PASS hook process audits the deferred command with its permission mode
|
|
85
|
+
PASS HARNESS_ACTION_GATE_POLICY=always-ask makes the hook process emit ask
|
|
86
|
+
PASS Claude hook manifest wires action-gate to shell and MCP pre-action surfaces
|
|
87
|
+
PASS Claude hook manifest audits successful tool execution
|
|
88
|
+
PASS Claude hook manifest audits failed tool execution
|
|
89
|
+
PASS Claude hook manifest closes unresolved approvals on Stop
|
|
90
|
+
PASS OpenAI plugin preserves the attribution-aware Codex shell action-gate adapter
|
|
91
|
+
PASS OpenAI plugin packages an isolated structured MCP pre-action adapter without widening unrelated hooks
|
|
92
|
+
PASS OpenAI plugin packages an isolated structured MCP post-action audit adapter
|
|
93
|
+
PASS Codex plugin does not declare unsupported PostToolUseFailure lifecycle event
|
|
94
|
+
PASS Codex plugin infers action-gate outcome from supported PostToolUse
|
|
95
|
+
PASS Codex PostToolUse maps zero exit code to success
|
|
96
|
+
PASS Codex PostToolUse maps non-zero exit code to failure
|
|
97
|
+
PASS Codex PostToolUse conservatively recognizes failure stderr without exit code
|
|
98
|
+
PASS OpenAI plugin closes pending approvals at Stop
|
|
99
|
+
PASS canonical and plugin action-gate rule tables are byte-identical
|
|
100
|
+
PASS OpenAI plugin packages action-gate runtime
|
|
101
|
+
PASS update_pull_request body defers without a permission decision
|
|
102
|
+
PASS metadata remains a non-safe mutation
|
|
103
|
+
PASS update_pull_request title defers without a permission decision
|
|
104
|
+
PASS metadata remains a non-safe mutation
|
|
105
|
+
PASS update_pull_request title,body defers without a permission decision
|
|
106
|
+
PASS metadata remains a non-safe mutation
|
|
107
|
+
PASS mcp__github__update_pull_request body defers without a permission decision
|
|
108
|
+
PASS metadata remains a non-safe mutation
|
|
109
|
+
PASS mcp__github__update_pull_request title defers without a permission decision
|
|
110
|
+
PASS metadata remains a non-safe mutation
|
|
111
|
+
PASS mcp__github__update_pull_request title,body defers without a permission decision
|
|
112
|
+
PASS metadata remains a non-safe mutation
|
|
113
|
+
PASS mcp__codex_apps__github__update_pull_request body defers without a permission decision
|
|
114
|
+
PASS metadata remains a non-safe mutation
|
|
115
|
+
PASS mcp__codex_apps__github__update_pull_request title defers without a permission decision
|
|
116
|
+
PASS metadata remains a non-safe mutation
|
|
117
|
+
PASS mcp__codex_apps__github__update_pull_request title,body defers without a permission decision
|
|
118
|
+
PASS metadata remains a non-safe mutation
|
|
119
|
+
PASS mcp__codex_apps__github_update_pull_request body defers without a permission decision
|
|
120
|
+
PASS metadata remains a non-safe mutation
|
|
121
|
+
PASS mcp__codex_apps__github_update_pull_request title defers without a permission decision
|
|
122
|
+
PASS metadata remains a non-safe mutation
|
|
123
|
+
PASS mcp__codex_apps__github_update_pull_request title,body defers without a permission decision
|
|
124
|
+
PASS metadata remains a non-safe mutation
|
|
125
|
+
PASS provider-neutral selectors support metadata updates
|
|
126
|
+
PASS provider-neutral selectors support metadata updates
|
|
127
|
+
PASS own __proto__ field cannot escape the argument allowlist
|
|
128
|
+
PASS explicit higher-risk action cannot inherit metadata deferral
|
|
129
|
+
PASS metadata with {"state":"closed"} remains gated
|
|
130
|
+
PASS metadata with {"state":"open"} remains gated
|
|
131
|
+
PASS metadata with {"state":null} remains gated
|
|
132
|
+
PASS metadata with {"base_branch":"other"} remains gated
|
|
133
|
+
PASS metadata with {"maintainer_can_modify":false} remains gated
|
|
134
|
+
PASS metadata with {"maintainer_can_modify":null} remains gated
|
|
135
|
+
PASS metadata with {"unknown":true} remains gated
|
|
136
|
+
PASS metadata with {"title":"mixed","state":"closed"} remains gated
|
|
137
|
+
PASS metadata with {"draft":true} remains gated
|
|
138
|
+
PASS metadata with {"base":"other"} remains gated
|
|
139
|
+
PASS metadata with {"action_descriptor":{"action":"delete"}} remains gated
|
|
140
|
+
PASS missing or malformed metadata remains gated
|
|
141
|
+
PASS missing or malformed metadata remains gated
|
|
142
|
+
PASS missing or malformed metadata remains gated
|
|
143
|
+
PASS missing or malformed metadata remains gated
|
|
144
|
+
PASS missing or malformed metadata remains gated
|
|
145
|
+
PASS missing or malformed metadata remains gated
|
|
146
|
+
PASS missing or malformed metadata remains gated
|
|
147
|
+
PASS missing or malformed metadata remains gated
|
|
148
|
+
PASS mcp__demo__update_pull_requests receives no metadata exception
|
|
149
|
+
PASS mcp__demo__foo_update_pull_request receives no metadata exception
|
|
150
|
+
PASS mcp__demo__update_pull_request_and_delete receives no metadata exception
|
|
151
|
+
PASS mcp__codex_apps__github_update_pull_request_extra receives no metadata exception
|
|
152
|
+
PASS mcp__demo__merge_pull_request receives no metadata exception
|
|
153
|
+
PASS mcp__demo__close_pull_request receives no metadata exception
|
|
154
|
+
PASS claude retains existing mutation rule
|
|
155
|
+
PASS opencode retains existing mutation rule
|
|
156
|
+
PASS unknown retains existing mutation rule
|
|
157
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
158
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
159
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
160
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
161
|
+
PASS required hint envelope 0 wins over optional envelopes
|
|
162
|
+
PASS audit retains required hint instead of a conflicting optional hint
|
|
163
|
+
PASS PR creation still honors the documented explicit required gate
|
|
164
|
+
PASS required hint envelope 1 wins over optional envelopes
|
|
165
|
+
PASS audit retains required hint instead of a conflicting optional hint
|
|
166
|
+
PASS PR creation still honors the documented explicit required gate
|
|
167
|
+
PASS required hint envelope 2 wins over optional envelopes
|
|
168
|
+
PASS audit retains required hint instead of a conflicting optional hint
|
|
169
|
+
PASS PR creation still honors the documented explicit required gate
|
|
170
|
+
PASS required hint envelope 3 wins over optional envelopes
|
|
171
|
+
PASS audit retains required hint instead of a conflicting optional hint
|
|
172
|
+
PASS PR creation still honors the documented explicit required gate
|
|
173
|
+
|
|
174
|
+
PASS: action-gate phase 5 (0 failures)
|
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
=== Issue #85 Phase 5 — Pre-action gate ===
|
|
2
|
+
PASS production action-gate rule table loads without fallback
|
|
3
|
+
PASS production rule table is non-empty and keeps the destructive baseline
|
|
4
|
+
PASS rule table contains git-force-push
|
|
5
|
+
PASS rule table contains git-reset-hard
|
|
6
|
+
PASS rule table contains git-clean-force-delete
|
|
7
|
+
PASS rule table contains recursive-delete
|
|
8
|
+
PASS rule table contains powershell-recursive-force-delete
|
|
9
|
+
PASS rule table contains sql-destructive-ddl
|
|
10
|
+
PASS rule table contains package-publish
|
|
11
|
+
PASS rule table contains github-release-create
|
|
12
|
+
PASS rule table contains production-deploy
|
|
13
|
+
PASS Claude force-push defers to the host permission flow without forcing a prompt
|
|
14
|
+
PASS deferred decision still names the matched rule
|
|
15
|
+
PASS matched Claude action is audited as deferred-to-host
|
|
16
|
+
PASS audit records matched rule
|
|
17
|
+
PASS audit hashes the exact command payload
|
|
18
|
+
PASS always-ask policy restores the forced Claude prompt
|
|
19
|
+
PASS ask decision uses Claude PreToolUse hookSpecificOutput
|
|
20
|
+
PASS ask reason names the matched rule
|
|
21
|
+
PASS always-ask action is audited as pending-approval
|
|
22
|
+
PASS git status passes untouched
|
|
23
|
+
PASS safe unmatched command creates no gate audit record
|
|
24
|
+
PASS permission_mode=default defers to the host without a forced prompt
|
|
25
|
+
PASS audit records permission_mode=default
|
|
26
|
+
PASS permission_mode=acceptEdits defers to the host without a forced prompt
|
|
27
|
+
PASS audit records permission_mode=acceptEdits
|
|
28
|
+
PASS permission_mode=plan defers to the host without a forced prompt
|
|
29
|
+
PASS audit records permission_mode=plan
|
|
30
|
+
PASS permission_mode=auto defers to the host without a forced prompt
|
|
31
|
+
PASS audit records permission_mode=auto
|
|
32
|
+
PASS permission_mode=dontAsk defers to the host without a forced prompt
|
|
33
|
+
PASS audit records permission_mode=dontAsk
|
|
34
|
+
PASS permission_mode=bypassPermissions defers to the host without a forced prompt
|
|
35
|
+
PASS audit records permission_mode=bypassPermissions
|
|
36
|
+
PASS permission_mode=missing defers to the host without a forced prompt
|
|
37
|
+
PASS audit records permission_mode=missing
|
|
38
|
+
PASS command classification gates destructive action even when router hint says not required
|
|
39
|
+
PASS router actionGate hint is recorded without controlling enforcement
|
|
40
|
+
PASS safe command is not gated solely because prompt router predicted risk
|
|
41
|
+
PASS structured MCP delete is gated before execution
|
|
42
|
+
PASS audit retains normalized structured action identity
|
|
43
|
+
PASS structured action audit binds to exact payload hash
|
|
44
|
+
PASS structured action audit does not persist raw secret arguments
|
|
45
|
+
PASS read-only MCP fixture passes without a false gate
|
|
46
|
+
PASS obvious structured mutation is classified from tool identity
|
|
47
|
+
PASS non-shell local-function destructive fixture uses the same structured gate
|
|
48
|
+
PASS unknown structured action fails closed when workflow requires a gate
|
|
49
|
+
PASS unknown structured action is not fabricated as destructive without a gate requirement
|
|
50
|
+
PASS changed structured payload cannot reuse a prior approval/audit decision
|
|
51
|
+
PASS PowerShell recursive force delete outside scratch is gated
|
|
52
|
+
PASS PowerShell recursive force delete inside session scratch is exempt
|
|
53
|
+
PASS rm -rf outside scratch is gated
|
|
54
|
+
PASS rm -rf inside OS temp is exempt
|
|
55
|
+
PASS git reset --hard HEAD matches git-reset-hard
|
|
56
|
+
PASS git clean -fd matches git-clean-force-delete
|
|
57
|
+
PASS psql -c "DROP TABLE users" matches sql-destructive-ddl
|
|
58
|
+
PASS npm publish matches package-publish
|
|
59
|
+
PASS gh release create v1.2.3 matches github-release-create
|
|
60
|
+
PASS npm run deploy matches production-deploy
|
|
61
|
+
PASS host without verified ask-style decision blocks with exit 2
|
|
62
|
+
PASS non-ask host block reason names the matched rule
|
|
63
|
+
PASS non-ask host block is audited as rejected
|
|
64
|
+
PASS internal action-gate error on Claude defers to the host instead of allowing or forcing a prompt
|
|
65
|
+
PASS internal error is shown to the user as a systemMessage without a permission decision
|
|
66
|
+
PASS internal error on Claude is audited as deferred-to-host
|
|
67
|
+
PASS internal action-gate error under always-ask fails closed to ask
|
|
68
|
+
PASS internal error on host without ask fails closed to block
|
|
69
|
+
PASS missing/invalid rule table visibly falls back to built-in defaults
|
|
70
|
+
PASS built-in fallback still classifies and audits destructive action
|
|
71
|
+
PASS fallback degradation is shown to the user as a systemMessage
|
|
72
|
+
PASS valid empty rule table does not silently activate built-in defaults
|
|
73
|
+
PASS empty-table mutation control disables detection, proving production non-empty assertion is meaningful
|
|
74
|
+
PASS null-prototype serialization preserves an own __proto__ key
|
|
75
|
+
PASS own __proto__ key changes the full-input digest
|
|
76
|
+
PASS PostToolUse closes a deferred payload as executed
|
|
77
|
+
PASS executed audit keeps exact approved payload hash
|
|
78
|
+
PASS PostToolUseFailure records attempted execution without pretending success
|
|
79
|
+
PASS post-tool payload mismatch is never accepted as the approved command
|
|
80
|
+
PASS Stop closes unexecuted deferred and pending payloads
|
|
81
|
+
PASS unexecuted deferred payload is audited as rejected
|
|
82
|
+
PASS unexecuted always-ask approval is audited as rejected
|
|
83
|
+
PASS hook process returns no decision for a matched command under the default policy
|
|
84
|
+
PASS hook process audits the deferred command with its permission mode
|
|
85
|
+
PASS HARNESS_ACTION_GATE_POLICY=always-ask makes the hook process emit ask
|
|
86
|
+
PASS Claude hook manifest wires action-gate to shell and MCP pre-action surfaces
|
|
87
|
+
PASS Claude hook manifest audits successful tool execution
|
|
88
|
+
PASS Claude hook manifest audits failed tool execution
|
|
89
|
+
PASS Claude hook manifest closes unresolved approvals on Stop
|
|
90
|
+
PASS OpenAI plugin preserves the attribution-aware Codex shell action-gate adapter
|
|
91
|
+
PASS OpenAI plugin packages an isolated structured MCP pre-action adapter without widening unrelated hooks
|
|
92
|
+
PASS OpenAI plugin packages an isolated structured MCP post-action audit adapter
|
|
93
|
+
PASS Codex plugin does not declare unsupported PostToolUseFailure lifecycle event
|
|
94
|
+
PASS Codex plugin infers action-gate outcome from supported PostToolUse
|
|
95
|
+
PASS Codex PostToolUse maps zero exit code to success
|
|
96
|
+
PASS Codex PostToolUse maps non-zero exit code to failure
|
|
97
|
+
PASS Codex PostToolUse conservatively recognizes failure stderr without exit code
|
|
98
|
+
PASS OpenAI plugin closes pending approvals at Stop
|
|
99
|
+
PASS canonical and plugin action-gate rule tables are byte-identical
|
|
100
|
+
PASS OpenAI plugin packages action-gate runtime
|
|
101
|
+
PASS update_pull_request body defers without a permission decision
|
|
102
|
+
PASS metadata remains a non-safe mutation
|
|
103
|
+
PASS update_pull_request title defers without a permission decision
|
|
104
|
+
PASS metadata remains a non-safe mutation
|
|
105
|
+
PASS update_pull_request title,body defers without a permission decision
|
|
106
|
+
PASS metadata remains a non-safe mutation
|
|
107
|
+
PASS mcp__github__update_pull_request body defers without a permission decision
|
|
108
|
+
PASS metadata remains a non-safe mutation
|
|
109
|
+
PASS mcp__github__update_pull_request title defers without a permission decision
|
|
110
|
+
PASS metadata remains a non-safe mutation
|
|
111
|
+
PASS mcp__github__update_pull_request title,body defers without a permission decision
|
|
112
|
+
PASS metadata remains a non-safe mutation
|
|
113
|
+
PASS mcp__codex_apps__github__update_pull_request body defers without a permission decision
|
|
114
|
+
PASS metadata remains a non-safe mutation
|
|
115
|
+
PASS mcp__codex_apps__github__update_pull_request title defers without a permission decision
|
|
116
|
+
PASS metadata remains a non-safe mutation
|
|
117
|
+
PASS mcp__codex_apps__github__update_pull_request title,body defers without a permission decision
|
|
118
|
+
PASS metadata remains a non-safe mutation
|
|
119
|
+
PASS mcp__codex_apps__github_update_pull_request body defers without a permission decision
|
|
120
|
+
PASS metadata remains a non-safe mutation
|
|
121
|
+
PASS mcp__codex_apps__github_update_pull_request title defers without a permission decision
|
|
122
|
+
PASS metadata remains a non-safe mutation
|
|
123
|
+
PASS mcp__codex_apps__github_update_pull_request title,body defers without a permission decision
|
|
124
|
+
PASS metadata remains a non-safe mutation
|
|
125
|
+
PASS provider-neutral selectors support metadata updates
|
|
126
|
+
PASS provider-neutral selectors support metadata updates
|
|
127
|
+
PASS own __proto__ field cannot escape the argument allowlist
|
|
128
|
+
PASS explicit higher-risk action cannot inherit metadata deferral
|
|
129
|
+
PASS metadata with {"state":"closed"} remains gated
|
|
130
|
+
PASS metadata with {"state":"open"} remains gated
|
|
131
|
+
PASS metadata with {"state":null} remains gated
|
|
132
|
+
PASS metadata with {"base_branch":"other"} remains gated
|
|
133
|
+
PASS metadata with {"maintainer_can_modify":false} remains gated
|
|
134
|
+
PASS metadata with {"maintainer_can_modify":null} remains gated
|
|
135
|
+
PASS metadata with {"unknown":true} remains gated
|
|
136
|
+
PASS metadata with {"title":"mixed","state":"closed"} remains gated
|
|
137
|
+
PASS metadata with {"draft":true} remains gated
|
|
138
|
+
PASS metadata with {"base":"other"} remains gated
|
|
139
|
+
PASS metadata with {"action_descriptor":{"action":"delete"}} remains gated
|
|
140
|
+
PASS missing or malformed metadata remains gated
|
|
141
|
+
PASS missing or malformed metadata remains gated
|
|
142
|
+
PASS missing or malformed metadata remains gated
|
|
143
|
+
PASS missing or malformed metadata remains gated
|
|
144
|
+
PASS missing or malformed metadata remains gated
|
|
145
|
+
PASS missing or malformed metadata remains gated
|
|
146
|
+
PASS missing or malformed metadata remains gated
|
|
147
|
+
PASS missing or malformed metadata remains gated
|
|
148
|
+
PASS mcp__demo__update_pull_requests receives no metadata exception
|
|
149
|
+
PASS mcp__demo__foo_update_pull_request receives no metadata exception
|
|
150
|
+
PASS mcp__demo__update_pull_request_and_delete receives no metadata exception
|
|
151
|
+
PASS mcp__codex_apps__github_update_pull_request_extra receives no metadata exception
|
|
152
|
+
PASS mcp__demo__merge_pull_request receives no metadata exception
|
|
153
|
+
PASS mcp__demo__close_pull_request receives no metadata exception
|
|
154
|
+
PASS claude retains existing mutation rule
|
|
155
|
+
PASS opencode retains existing mutation rule
|
|
156
|
+
PASS unknown retains existing mutation rule
|
|
157
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
158
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
159
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
160
|
+
PASS explicit required metadata gate cannot be bypassed
|
|
161
|
+
PASS required hint envelope 0 wins over optional envelopes
|
|
162
|
+
PASS audit retains required hint instead of a conflicting optional hint
|
|
163
|
+
PASS PR creation still honors the documented explicit required gate
|
|
164
|
+
FAIL required hint envelope 1 wins over optional envelopes
|
|
165
|
+
FAIL audit retains required hint instead of a conflicting optional hint
|
|
166
|
+
FAIL PR creation still honors the documented explicit required gate
|
|
167
|
+
FAIL required hint envelope 2 wins over optional envelopes
|
|
168
|
+
FAIL audit retains required hint instead of a conflicting optional hint
|
|
169
|
+
FAIL PR creation still honors the documented explicit required gate
|
|
170
|
+
FAIL required hint envelope 3 wins over optional envelopes
|
|
171
|
+
FAIL audit retains required hint instead of a conflicting optional hint
|
|
172
|
+
FAIL PR creation still honors the documented explicit required gate
|
|
173
|
+
|
|
174
|
+
FAIL: action-gate phase 5 (9 failures)
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Final local regression
|
|
2
|
+
|
|
3
|
+
- Command: `PATH=/Users/daniel.chang/.nvm/versions/node/v24.22.1/bin:$PATH npm run verify:all`
|
|
4
|
+
- Process exit: `0`.
|
|
5
|
+
- Full local log: `/tmp/pr-metadata-verify-final.log`.
|
|
6
|
+
- Full log SHA-256: `630159501b19d1d0178683262f0bfe7c0ec9cacb29a336057bb95094481129f4`.
|
|
7
|
+
- Final core is pinned below; subsequent commits contain evidence only.
|
|
8
|
+
|
|
9
|
+
## Local gate run — paste-ready evidence
|
|
10
|
+
HEAD `c170a96cf856295200d2d1a62c15a552c3cb2b37` · node v24.22.1 · darwin · mode: full gate set · flags: (defaults)
|
|
11
|
+
|
|
12
|
+
| # | Gate | Result | Time |
|
|
13
|
+
| --- | --- | --- | --- |
|
|
14
|
+
| 1 | self-regression (hermetic + mechanism + routing contracts) | PASS | 386.3s |
|
|
15
|
+
| 2 | mutation checks | PASS | 38.5s |
|
|
16
|
+
| 3 | consistency (docs, manifests, repo contract, prompt contracts) | PASS | 0.5s |
|
|
17
|
+
| 4 | references resolve | PASS | 0.1s |
|
|
18
|
+
| 5 | release consistency | PASS | 3.0s |
|
|
19
|
+
| 6 | skill description collision | PASS | 0.1s |
|
|
20
|
+
| 7 | skill routing coverage | PASS | 1.2s |
|
|
21
|
+
| 8 | routing invariants | PASS | 4.9s |
|
|
22
|
+
| 9 | openai plugin package | PASS | 1.6s |
|
|
23
|
+
| 10 | openai public submission | PASS | 0.2s |
|
|
24
|
+
| 11 | platform compatibility matrix | PASS | 0.1s |
|
|
25
|
+
| 12 | test-source portability | PASS | 0.1s |
|
|
26
|
+
| 13 | node22 lane: self-regression execution smoke test | UNCHECKED (not requested; add --node22) | — |
|
|
27
|
+
| 14 | node22 lane: installer roundtrip execution smoke test | UNCHECKED (not requested; add --node22) | — |
|
|
28
|
+
| 15 | node22 lane: dependency install (npm ci under Node 22) | UNCHECKED (covered by the CI runtime-floor job; not replicated locally) | — |
|
|
29
|
+
| 16 | waza readiness (optional) | PASS | 7.0s |
|
|
30
|
+
|
|
31
|
+
Summary: 13 PASS, 0 FAIL, 3 UNCHECKED.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# #341 E1 live observation record — 2026-10-25
|
|
2
|
+
|
|
3
|
+
**Status: retained observations only.** The independent cold reviewer (Luna xhigh) adjudicated O1/O2 as non-qualifying, confirmed the withholding was correct, and found no false MUST PASS or false rejection. E1 remains OPEN/UNCHECKED; no formal GO/REVISE/NO-GO decision is made here.
|
|
4
|
+
|
|
5
|
+
## Task and correction
|
|
6
|
+
|
|
7
|
+
The first Claude Code session corrected the current status wording in [doc-workflow-reconciliation.md](../../../docs/doc-workflow-reconciliation.md) after comparing it with the retained [OpenCode D1/D2 record](../live-host/opencode-d1-d2-2026-10-25/README.md). A later host reviewer found three remaining current-status copies that conflicted with the correction. The owner authorized a narrow consistency edit to those copies in the same reference and [document-lifecycle.md](../../../docs/document-lifecycle.md). The edit preserved the original S2 entry condition: E1 observation begins only after a creditable S2.
|
|
8
|
+
|
|
9
|
+
After the host observations, a cold source review also found that the lifecycle heading labeled the whole table as a 2026-10-20 status even though the Phase 3b/3c rows had advanced. The heading now identifies the original checkpoint and the two updated rows. The Phase 3c row records **H1 preparation only** from #373; full H1–H4 and paired-task evaluation remain pending. These are source-traced publication corrections, not E1 runtime evidence.
|
|
10
|
+
|
|
11
|
+
No source or test code changed. The lifecycle contract is not in `.agents/harness-everything/project-docs.json`; it remains outside the current doc-check governed set.
|
|
12
|
+
|
|
13
|
+
## Expected and actual decisions
|
|
14
|
+
|
|
15
|
+
| Case | Expected | Actual | Interpretation |
|
|
16
|
+
|---|---|---|---|
|
|
17
|
+
| Initial single-path `doc-lint` | Pass (initial prediction) | Partial: no domains are declared, so bare `<DOMAIN>-NNN` references are unchecked | The initial prediction was wrong; later runs expected `partial`. This is a coverage limit, not a false rejection. |
|
|
18
|
+
| Single-path `link-check` | Pass | Pass: 11 links checked, 0 broken | Matches the initial expectation. |
|
|
19
|
+
| Final `doc-lint` with both changed paths | Partial | Partial: no domains are declared | No doc-lint credit. |
|
|
20
|
+
| Final `link-check` with both changed paths | Partial, because `document-lifecycle.md` is not governed | Partial: 11 links checked, 0 broken; lifecycle file skipped as not governed | The existing config does not establish link coverage for the second path. No config expansion was made. |
|
|
21
|
+
| Current E1 status review | Approve only if current status copies agree, D1/D2 remain non-qualifying, the S2 gate remains, and no GO/benefit is claimed | A read-only host reviewer approved the O2 observed candidate hashes. The cold reviewer found the later H1 row/date-heading inconsistencies and authorized their correction; the final publication hash is recorded separately. The runtime report retained reviewer/scope/artifact conflict diagnostics, so it accepted no review reference. | Content approval is recorded; runtime quality-review credit is not. |
|
|
22
|
+
| Verification step with partial coverage | Withhold PASS | Exit 2: `document verification pending: S1 grants no document credit`; `pilotCredit=false` | Expected fail-closed outcome. No false MUST PASS was observed. |
|
|
23
|
+
|
|
24
|
+
## Workflow and host evidence
|
|
25
|
+
|
|
26
|
+
- O1 was a real body-only edit and direct registered `doc-check` run, but the host routed it as Tier 1. No typed workflow or pilot report was observed, so O1 is exposure only.
|
|
27
|
+
- O2 used a Tier 2 `iterative-single` workflow on the pinned `25c4a8478d5617d1e803940951876bb3ae1ecb37` baseline. It remains `running`: `req-review` has a stale recorded digest and `req-verify` is pending.
|
|
28
|
+
- The runtime report in [pilot-report.json](pilot-report.json) is `reportOrigin: runtime`, `pilotCredit: false`, `pilotStatus: unresolved`, with no credited capabilities or accepted references. Diagnostics include partial coverage, missing result/path coverage, reviewer independence, scope, artifact, and event conflicts. This report records the observed diagnostics; it does not establish their root cause.
|
|
29
|
+
- Host: Claude Code 2.1.310, `claude-sonnet-5-5`, first-party provider, macOS 26.6.2 arm64. The staged Claude plugin was version 0.42.0 with 1,754 tracked files from the pinned baseline.
|
|
30
|
+
- The per-session hook wrapper recorded 8 raw hook payloads for O1 and 53 for O2, with no duplicate event/tool keys. The cold reviewer independently matched these counts and artifact hashes and found no credentials in retained public artifacts. Raw host transcripts, hook payloads, and state journals remain private in `/tmp/harness-e1-2026-10-25`; `evidence.json` records their hashes. The O2 reviewer saw `document-lifecycle.md` at `503ef6d2…`; the final publication bytes are `d66f5731…` after the separately source-traced Phase 3c update.
|
|
31
|
+
|
|
32
|
+
## Exact checks, cost, and next item
|
|
33
|
+
|
|
34
|
+
The checker commands captured in the host journals were:
|
|
35
|
+
|
|
36
|
+
```sh
|
|
37
|
+
node "/tmp/harness-e1-2026-10-25/plugin/harness-everything/scripts/doc-check.js" check --workspace "/Users/daniel.chang/Desktop/Skills/Harness-everything/.worktrees/341-e1-decision-evidence" --config .agents/harness-everything/project-docs.json --changed-path docs/doc-workflow-reconciliation.md
|
|
38
|
+
|
|
39
|
+
node "/tmp/harness-e1-2026-10-25/plugin/harness-everything/scripts/doc-check.js" check --workspace "/Users/daniel.chang/Desktop/Skills/Harness-everything/.worktrees/341-e1-decision-evidence" --config .agents/harness-everything/project-docs.json --changed-path docs/doc-workflow-reconciliation.md --changed-path docs/document-lifecycle.md
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
The plugin was staged with `node scripts/stage-claude-plugin.js /tmp/harness-e1-2026-10-25/plugin`. Claude was run with the pilot switch ON, per-run `HARNESS_STATE_HOME`, the staged plugin, per-session capture settings, and a reviewer restricted to `Read`/`Grep`. The raw stream and exact prompts are retained locally by hash; no global user settings or credentials were changed.
|
|
43
|
+
|
|
44
|
+
- O1 host result: 18.139 s, 6 turns, 1,815 output tokens, $0.126 reported cost, 5 unique journal tool-use IDs, 1 edit call.
|
|
45
|
+
- O2 resumed session: latest result 37.486 s; host reports 21,297 session output tokens and $1.058 cumulative session cost; journal has 26 unique tool-use IDs and 4 failed tool calls. These are host-reported values, not end-to-end wall time.
|
|
46
|
+
- Exact coordinator intervention count and total end-to-end wall time were not measured. Final correction scope is four E1 status rows, one separate Phase 3c row, and one historical-checkpoint heading clarification; no source code or tests were added.
|
|
47
|
+
- Initial `verify:all -- --fast` was interrupted after the missing-worktree-`node_modules` setup failure. `npm ci --ignore-scripts` then installed 106 packages without a lockfile change; npm also reported one moderate advisory. The coordinator owns final `verify:all`; this worker did not run it.
|
|
48
|
+
- Coordinator final `npm run verify:all` on Node 24.22.1 completed **13 PASS / 0 FAIL / 3 UNCHECKED**; optional Waza readiness passed. The three unchecked items are the Node 22 self-regression smoke, installer roundtrip, and `npm ci` lane (the CI runtime-floor job covers Node 22). Windows/remote CI was not run. Log: `LOCAL_ONLY:/tmp/harness-e1-final-verification-node24.log`, SHA-256 `cb2c783c0913a72a39c7b03aa179dc6317d90439ad49bdfa15a143ee96e9a530`.
|
|
49
|
+
|
|
50
|
+
**Smallest next item:** a future qualifying task needs complete `doc-lint` coverage, a changed-path set fully within truthful configured governance for `link-check`, and a fresh independent reviewer principal from the start. No domain or governance expansion is inferred here; the owner decides whether project-domain configuration should be added.
|
|
51
|
+
|
|
52
|
+
## Provider fallback context (non-qualifying)
|
|
53
|
+
|
|
54
|
+
The owner requested a disposition of the fallback paths. The read-only provider diagnostic reports:
|
|
55
|
+
|
|
56
|
+
- OpenCode's `glm-5.3-flash-free` primary had exhausted its free usage limit.
|
|
57
|
+
- OpenCode's selected `openai/gpt-6-luna#xhigh` route rejected the configured credential and returned no model response. The cause beyond that rejection is unknown; no key text is retained here and no credential was changed.
|
|
58
|
+
- A separate NVIDIA `glm-5.3` probe produced no output within a 58-second cap while another NVIDIA call was already active. The cause is unknown; this does not establish NVIDIA unavailability.
|
|
59
|
+
- The native Codex `gpt-5.6-luna` xhigh read-only worker probe returned `READY` (12,665 reported tokens). This establishes worker-path availability only; it is not the Claude subject or E1 evidence.
|
|
60
|
+
- Claude Code 2.1.310 was authenticated and used as the experimental subject for O1/O2. Those observations remain non-qualifying for the reasons above.
|
|
61
|
+
|
|
62
|
+
All fallback probes are non-qualifying and are kept as operational context, not behavior or E1 outcome evidence. The sanitized provider diagnostic hash is `b918e70720a2d2d3ec77ddc343fa411e332e328202da2ff3bd1d145c235d320d` (`LOCAL_ONLY:/tmp/harness-provider-diagnostic.md`).
|
|
63
|
+
|
|
64
|
+
## Qualification
|
|
65
|
+
|
|
66
|
+
`qualifyingE1Cycles` is recorded as **0**. O1 lacked a typed workflow and runtime report. O2 did not complete and received no runtime document credit. The cold reviewer marks the evidence layer as source-traced correspondence against the raw host records; the cycles remain non-qualifying. The #341 Phase 0D acceptance target—2–3 genuine development/review cycles with sufficient case mix—remains unsatisfied; this record closes no E1 acceptance criterion. It makes no E1 GO/REVISE/NO-GO or #320 promotion claim; #325 H1–H4 and paired-task evidence remain separate.
|
|
67
|
+
|
|
68
|
+
The first run's mistaken `doc-lint=pass` prediction, the later corrected `partial` expectation, both reviewer outcomes, and the verification refusal are retained in `evidence.json` and `pilot-report.json`. The current reviewer approval is not treated as a qualifying runtime review because the runtime report accepted no quality-review reference.
|