harness-everything 0.25.0 → 0.26.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +36 -0
- package/ci/mechanism-35-verification-contract.test.js +416 -0
- package/ci/mutation-check.js +2 -0
- package/ci/mutation-loader.js +4 -0
- package/docs/audit.md +2 -0
- package/harness-everything/SKILL.md +2 -2
- package/harness-everything/scripts/verify-gate.js +223 -75
- package/hooks/scripts/lib/workflow-isolation.js +3 -4
- package/hooks/scripts/state-persist.js +1 -1
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/hooks/scripts/lib/workflow-isolation.js +3 -4
- package/plugins/harness-everything/hooks/scripts/state-persist.js +1 -1
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/scripts/lib/verification-contract.js +357 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +2 -2
- package/plugins/harness-everything/skills/harness-everything/scripts/verify-gate.js +223 -75
- package/plugins/harness-everything/skills/using-git-worktrees/SKILL.md +14 -13
- package/plugins/harness-everything/skills/using-git-worktrees/references/submodules-in-worktrees.md +135 -0
- package/plugins/harness-everything/skills/verification-loop/SKILL.md +15 -23
- package/plugins/harness-everything/skills/verification-loop/references/project-verification-contract.md +112 -0
- package/plugins/harness-everything/skills/verification-loop/references/verification-phases.md +4 -3
- package/plugins/harness-everything/skills/verification-loop/references/verify-contract.schema.json +32 -0
- package/plugins/harness-everything/skills/verification-loop/references/verify-result.schema.json +65 -0
- package/sbom.cdx.json +7 -7
- package/scripts/lib/verification-contract.js +357 -0
- package/using-git-worktrees/SKILL.md +14 -13
- package/using-git-worktrees/references/submodules-in-worktrees.md +135 -0
- package/verification-loop/SKILL.md +15 -23
- package/verification-loop/references/project-verification-contract.md +112 -0
- package/verification-loop/references/verification-phases.md +4 -3
- package/verification-loop/references/verify-contract.schema.json +32 -0
- package/verification-loop/references/verify-result.schema.json +65 -0
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"name": "harness-everything",
|
|
11
11
|
"source": "./",
|
|
12
12
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
13
|
-
"version": "0.
|
|
13
|
+
"version": "0.26.0",
|
|
14
14
|
"author": {
|
|
15
15
|
"name": "Miya Daniel"
|
|
16
16
|
},
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "harness-everything",
|
|
3
3
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.26.0",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Miya Daniel"
|
|
7
7
|
},
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,42 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project are documented in this file.
|
|
4
4
|
|
|
5
|
+
## [0.26.0](https://github.com/dyphn1/Harness-everything/compare/v0.25.1...v0.26.0) (2026-09-24)
|
|
6
|
+
|
|
7
|
+
### Features
|
|
8
|
+
|
|
9
|
+
* **verify:** add shared verification contract model ([8e9548c](https://github.com/dyphn1/Harness-everything/commit/8e9548c278c4c13d4d29a07093a0d1dec67b7a76))
|
|
10
|
+
* **verify:** add structured multi-root verification gate ([e616978](https://github.com/dyphn1/Harness-everything/commit/e616978a2ae81dda9123f189b4639c6c6c9f97ed))
|
|
11
|
+
|
|
12
|
+
### Bug Fixes
|
|
13
|
+
|
|
14
|
+
* **verify:** align hook verification command recognition ([943659a](https://github.com/dyphn1/Harness-everything/commit/943659aa0639316beea7f90327c32b393d441931))
|
|
15
|
+
* **verify:** keep hook evidence fail closed ([ff19098](https://github.com/dyphn1/Harness-everything/commit/ff190980e2452b297624ae06be60a29bcfc74fb5))
|
|
16
|
+
* **verify:** preserve legacy evidence without broad word matching ([aa4ac71](https://github.com/dyphn1/Harness-everything/commit/aa4ac715b85321f29cd744c02221130e659de318))
|
|
17
|
+
* **verify:** record project-aware verification evidence ([cab04fc](https://github.com/dyphn1/Harness-everything/commit/cab04fce9691938dcf17da88591e4063a94b5339))
|
|
18
|
+
* **verify:** Require runnable commit hooks ([735790e](https://github.com/dyphn1/Harness-everything/commit/735790ea0fc39603f0e68f160f9bd20f49636095))
|
|
19
|
+
* **verify:** route shell evidence through shared classifier ([e205319](https://github.com/dyphn1/Harness-everything/commit/e205319011fb8cfef4e43310e550b2725d41bacb))
|
|
20
|
+
* **verify:** sync fail-closed verification classifier ([cfe1eb2](https://github.com/dyphn1/Harness-everything/commit/cfe1eb221686edac554f545637925e7c37002057))
|
|
21
|
+
|
|
22
|
+
## [Unreleased]
|
|
23
|
+
|
|
24
|
+
### Bug Fixes
|
|
25
|
+
|
|
26
|
+
* **verify:** discover and execute project verification contracts across npm and non-npm repositories, with multi-root structured evidence and aligned hook recognition (#242)
|
|
27
|
+
|
|
28
|
+
## [0.25.1](https://github.com/dyphn1/Harness-everything/compare/v0.25.0...v0.25.1) (2026-09-24)
|
|
29
|
+
|
|
30
|
+
### Bug Fixes
|
|
31
|
+
|
|
32
|
+
* **skills:** document worktree submodule reachability ([024185a](https://github.com/dyphn1/Harness-everything/commit/024185ab29363968bbe30d648cf2ef902751e9b9))
|
|
33
|
+
* **skills:** keep worktree contract within token budget ([c41903d](https://github.com/dyphn1/Harness-everything/commit/c41903d2c37e3af127b84b0ba9c5a82cd4fab3ee))
|
|
34
|
+
* **skills:** preserve isolation contract wording ([b14ea2a](https://github.com/dyphn1/Harness-everything/commit/b14ea2a6e6417f4103f16e5d2cf04e6fc994e583))
|
|
35
|
+
* **skills:** require submodule reachability reference ([5c61c2b](https://github.com/dyphn1/Harness-everything/commit/5c61c2bbae0f2e29818579bd2ebcd7d45a76c1b0))
|
|
36
|
+
* **skills:** sync compact worktree contract ([cd4e3a8](https://github.com/dyphn1/Harness-everything/commit/cd4e3a8db276083ac11346031f626058cd02bed3))
|
|
37
|
+
* **skills:** sync isolation contract wording ([7f59b1c](https://github.com/dyphn1/Harness-everything/commit/7f59b1c4e66d00ec84c225ad161e503a443ad1cd))
|
|
38
|
+
* **skills:** sync worktree submodule reference ([b3fdf5d](https://github.com/dyphn1/Harness-everything/commit/b3fdf5dff8b2c22ff02cc39510d23dc60c2489ae))
|
|
39
|
+
* **skills:** sync worktree submodule skill ([08dffec](https://github.com/dyphn1/Harness-everything/commit/08dffec0aa69602b61a1a90b6d10cbf97cea0dcb))
|
|
40
|
+
|
|
5
41
|
## [0.25.0](https://github.com/dyphn1/Harness-everything/compare/v0.24.0...v0.25.0) (2026-09-23)
|
|
6
42
|
|
|
7
43
|
### Features
|
|
@@ -0,0 +1,416 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
const assert = require('assert');
|
|
4
|
+
const fs = require('fs');
|
|
5
|
+
const os = require('os');
|
|
6
|
+
const path = require('path');
|
|
7
|
+
const { spawnSync } = require('child_process');
|
|
8
|
+
|
|
9
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
10
|
+
const VERIFY = path.join(ROOT, 'harness-everything', 'scripts', 'verify-gate.js');
|
|
11
|
+
const { isVerificationShell } = require('../hooks/scripts/lib/workflow-isolation');
|
|
12
|
+
const { hasCommitHook, hookIsRunnable } = require('../scripts/lib/verification-contract');
|
|
13
|
+
const { getWorkspaceKey } = require('../scripts/lib/workspace');
|
|
14
|
+
|
|
15
|
+
function run(command, args, cwd, options = {}) {
|
|
16
|
+
const result = spawnSync(command, args, {
|
|
17
|
+
cwd,
|
|
18
|
+
encoding: 'utf8',
|
|
19
|
+
env: { ...process.env, ...(options.env || {}) },
|
|
20
|
+
timeout: options.timeout || 30000,
|
|
21
|
+
});
|
|
22
|
+
if (!options.allowFailure && result.status !== 0) {
|
|
23
|
+
throw new Error([
|
|
24
|
+
`${command} ${args.join(' ')} failed with ${result.status}`,
|
|
25
|
+
result.stdout,
|
|
26
|
+
result.stderr,
|
|
27
|
+
].filter(Boolean).join('\n'));
|
|
28
|
+
}
|
|
29
|
+
return result;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
function git(cwd, args, options = {}) {
|
|
33
|
+
return run('git', args, cwd, options);
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
function initRepo(dir) {
|
|
37
|
+
fs.mkdirSync(dir, { recursive: true });
|
|
38
|
+
git(dir, ['init', '-b', 'main']);
|
|
39
|
+
git(dir, ['config', 'user.name', 'Harness Verify Test']);
|
|
40
|
+
git(dir, ['config', 'user.email', 'verify@example.invalid']);
|
|
41
|
+
fs.writeFileSync(path.join(dir, 'README.md'), 'fixture\n');
|
|
42
|
+
git(dir, ['add', 'README.md']);
|
|
43
|
+
git(dir, ['commit', '-m', 'seed']);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
function writeJson(file, value) {
|
|
47
|
+
fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
48
|
+
fs.writeFileSync(file, JSON.stringify(value, null, 2) + '\n');
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function invokeGate(cwd, options = {}) {
|
|
52
|
+
const args = options.json === false ? [] : ['--json'];
|
|
53
|
+
const result = run(process.execPath, [VERIFY, ...args], cwd, {
|
|
54
|
+
allowFailure: true,
|
|
55
|
+
env: options.env || {},
|
|
56
|
+
timeout: options.timeout || 30000,
|
|
57
|
+
});
|
|
58
|
+
if (options.json === false) return result;
|
|
59
|
+
let report;
|
|
60
|
+
try {
|
|
61
|
+
report = JSON.parse(result.stdout);
|
|
62
|
+
} catch (error) {
|
|
63
|
+
throw new Error(`verify-gate did not emit JSON (exit=${result.status})\nstdout:\n${result.stdout}\nstderr:\n${result.stderr}`);
|
|
64
|
+
}
|
|
65
|
+
return { ...result, report };
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
function packageScript(body) {
|
|
69
|
+
return {
|
|
70
|
+
name: 'verify-fixture',
|
|
71
|
+
version: '1.0.0',
|
|
72
|
+
private: true,
|
|
73
|
+
scripts: body,
|
|
74
|
+
};
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
const temp = fs.mkdtempSync(path.join(os.tmpdir(), 'harness-verify-contract-'));
|
|
78
|
+
|
|
79
|
+
assert.strictEqual(hookIsRunnable({ isFile: () => true, mode: 0o100755 }, 'linux'), true);
|
|
80
|
+
assert.strictEqual(hookIsRunnable({ isFile: () => true, mode: 0o100644 }, 'linux'), false);
|
|
81
|
+
assert.strictEqual(hookIsRunnable({ isFile: () => true, mode: 0o100644 }, 'win32'), true);
|
|
82
|
+
assert.strictEqual(hookIsRunnable({ isFile: () => false, mode: 0o100755 }, 'linux'), false);
|
|
83
|
+
|
|
84
|
+
try {
|
|
85
|
+
// Taxonomy: truly no checks is distinct from discovered-but-unrun.
|
|
86
|
+
const empty = path.join(temp, 'empty');
|
|
87
|
+
initRepo(empty);
|
|
88
|
+
let result = invokeGate(empty);
|
|
89
|
+
assert.strictEqual(result.status, 0);
|
|
90
|
+
assert.strictEqual(result.report.status, 'UNCHECKED_NO_CHECKS');
|
|
91
|
+
assert.deepStrictEqual(result.report.candidates, []);
|
|
92
|
+
const emptyHuman = invokeGate(empty, { json: false });
|
|
93
|
+
assert.match(emptyHuman.stdout + emptyHuman.stderr, /PROHIBITED: Do NOT cite this run as proof/i);
|
|
94
|
+
|
|
95
|
+
// npm compatibility: placeholder is no check; lint/test auto-run; failure blocks.
|
|
96
|
+
const npmRepo = path.join(temp, 'npm');
|
|
97
|
+
initRepo(npmRepo);
|
|
98
|
+
writeJson(path.join(npmRepo, 'package.json'), packageScript({
|
|
99
|
+
test: 'echo "Error: no test specified" && exit 1',
|
|
100
|
+
}));
|
|
101
|
+
result = invokeGate(npmRepo);
|
|
102
|
+
assert.strictEqual(result.report.status, 'UNCHECKED_NO_CHECKS');
|
|
103
|
+
|
|
104
|
+
writeJson(path.join(npmRepo, 'package.json'), packageScript({
|
|
105
|
+
lint: 'node -e "process.exit(0)"',
|
|
106
|
+
test: 'node -e "process.exit(0)"',
|
|
107
|
+
}));
|
|
108
|
+
result = invokeGate(npmRepo);
|
|
109
|
+
assert.strictEqual(result.status, 0);
|
|
110
|
+
assert.strictEqual(result.report.status, 'PASSED');
|
|
111
|
+
assert.strictEqual(result.report.checks.length, 2);
|
|
112
|
+
assert.ok(result.report.checks.every(check => check.exitCode === 0));
|
|
113
|
+
|
|
114
|
+
writeJson(path.join(npmRepo, 'package.json'), packageScript({
|
|
115
|
+
test: 'node -e "process.exit(9)"',
|
|
116
|
+
}));
|
|
117
|
+
result = invokeGate(npmRepo);
|
|
118
|
+
assert.strictEqual(result.status, 1);
|
|
119
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
120
|
+
assert.strictEqual(result.report.checks[0].exitCode, 9);
|
|
121
|
+
|
|
122
|
+
fs.writeFileSync(path.join(npmRepo, 'package.json'), '{ invalid json');
|
|
123
|
+
result = invokeGate(npmRepo);
|
|
124
|
+
assert.strictEqual(result.status, 1);
|
|
125
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
126
|
+
assert.match(result.report.reason, /package\.json/i);
|
|
127
|
+
|
|
128
|
+
fs.writeFileSync(path.join(npmRepo, '.verify-fail.tmp'), 'fail');
|
|
129
|
+
result = invokeGate(npmRepo);
|
|
130
|
+
assert.strictEqual(result.status, 1);
|
|
131
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
132
|
+
fs.rmSync(path.join(npmRepo, '.verify-fail.tmp'));
|
|
133
|
+
|
|
134
|
+
result = invokeGate(npmRepo, { env: { HARNESS_SKIP_PROJECT_CHECKS: '1' } });
|
|
135
|
+
assert.strictEqual(result.status, 0);
|
|
136
|
+
assert.strictEqual(result.report.status, 'SKIPPED');
|
|
137
|
+
|
|
138
|
+
// Read-only discovery: candidates are listed but never executed.
|
|
139
|
+
const detected = path.join(temp, 'detected');
|
|
140
|
+
initRepo(detected);
|
|
141
|
+
fs.writeFileSync(path.join(detected, 'sentinel.js'), 'require("fs").writeFileSync("RAN", "bad")\n');
|
|
142
|
+
fs.writeFileSync(path.join(detected, 'Makefile'), 'test:\n\tnode sentinel.js\ncheck:\n\t@echo check\n');
|
|
143
|
+
fs.writeFileSync(path.join(detected, 'justfile'), 'ci:\n node sentinel.js\n');
|
|
144
|
+
fs.writeFileSync(path.join(detected, '.pre-commit-config.yaml'), 'repos: []\n');
|
|
145
|
+
fs.writeFileSync(path.join(detected, 'pyproject.toml'), '[tool.pytest.ini_options]\naddopts = "-q"\n[tool.ruff]\nline-length = 100\n');
|
|
146
|
+
fs.writeFileSync(path.join(detected, 'tox.ini'), '[tox]\nenvlist = py\n');
|
|
147
|
+
fs.writeFileSync(path.join(detected, 'noxfile.py'), '# nox fixture\n');
|
|
148
|
+
fs.writeFileSync(path.join(detected, 'Cargo.toml'), '[package]\nname="x"\nversion="0.1.0"\n');
|
|
149
|
+
fs.writeFileSync(path.join(detected, 'go.mod'), 'module example.invalid/x\n\ngo 1.22\n');
|
|
150
|
+
result = invokeGate(detected);
|
|
151
|
+
assert.strictEqual(result.status, 0);
|
|
152
|
+
assert.strictEqual(result.report.status, 'UNCHECKED_DISCOVERED');
|
|
153
|
+
const candidateRuns = result.report.candidates.map(candidate => candidate.run);
|
|
154
|
+
for (const command of [
|
|
155
|
+
'pre-commit run --all-files',
|
|
156
|
+
'make test',
|
|
157
|
+
'make check',
|
|
158
|
+
'just ci',
|
|
159
|
+
'pytest',
|
|
160
|
+
'ruff check .',
|
|
161
|
+
'tox',
|
|
162
|
+
'nox',
|
|
163
|
+
'cargo test',
|
|
164
|
+
'go test ./...',
|
|
165
|
+
]) assert.ok(candidateRuns.includes(command), `missing candidate: ${command}\n${JSON.stringify(result.report.candidates)}`);
|
|
166
|
+
assert.ok(!fs.existsSync(path.join(detected, 'RAN')), 'detected commands must not run');
|
|
167
|
+
|
|
168
|
+
// package check/typecheck/verify are detected but not auto-run.
|
|
169
|
+
writeJson(path.join(detected, 'package.json'), packageScript({
|
|
170
|
+
test: 'node -e "process.exit(0)"',
|
|
171
|
+
check: 'node sentinel.js',
|
|
172
|
+
typecheck: 'node sentinel.js',
|
|
173
|
+
verify: 'node sentinel.js',
|
|
174
|
+
}));
|
|
175
|
+
result = invokeGate(detected);
|
|
176
|
+
assert.strictEqual(result.report.status, 'UNCHECKED_DISCOVERED');
|
|
177
|
+
assert.ok(result.report.checks.some(check => / run test$/.test(check.run)));
|
|
178
|
+
assert.ok(result.report.candidates.some(candidate => / run check$/.test(candidate.run)));
|
|
179
|
+
assert.ok(!fs.existsSync(path.join(detected, 'RAN')));
|
|
180
|
+
|
|
181
|
+
// Explicit contract is authoritative and executable.
|
|
182
|
+
const contracted = path.join(temp, 'contracted');
|
|
183
|
+
initRepo(contracted);
|
|
184
|
+
writeJson(path.join(contracted, 'package.json'), packageScript({
|
|
185
|
+
test: 'node -e "process.exit(13)"',
|
|
186
|
+
}));
|
|
187
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
188
|
+
version: 1,
|
|
189
|
+
roots: ['.'],
|
|
190
|
+
checks: [{
|
|
191
|
+
id: 'contract-pass',
|
|
192
|
+
run: 'node -e "if(process.env.HARNESS_SKIP_PROJECT_CHECKS!==\'1\') process.exit(7)"',
|
|
193
|
+
cwd: '.',
|
|
194
|
+
timeoutSec: 10,
|
|
195
|
+
}],
|
|
196
|
+
});
|
|
197
|
+
result = invokeGate(contracted);
|
|
198
|
+
assert.strictEqual(result.status, 0);
|
|
199
|
+
assert.strictEqual(result.report.status, 'PASSED');
|
|
200
|
+
assert.strictEqual(result.report.checks.length, 1, 'package.json must be ignored when contract exists');
|
|
201
|
+
assert.strictEqual(result.report.checks[0].id, 'contract-pass');
|
|
202
|
+
|
|
203
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
204
|
+
version: 1,
|
|
205
|
+
checks: [{ id: 'contract-fail', run: 'node -e "process.exit(4)"', cwd: '.' }],
|
|
206
|
+
});
|
|
207
|
+
result = invokeGate(contracted);
|
|
208
|
+
assert.strictEqual(result.status, 1);
|
|
209
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
210
|
+
assert.strictEqual(result.report.checks[0].id, 'contract-fail');
|
|
211
|
+
assert.strictEqual(result.report.checks[0].exitCode, 4);
|
|
212
|
+
|
|
213
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
214
|
+
version: 1,
|
|
215
|
+
checks: [{ id: 'timeout', run: 'node -e "setTimeout(()=>{}, 5000)"', timeoutSec: 1 }],
|
|
216
|
+
});
|
|
217
|
+
result = invokeGate(contracted, { timeout: 10000 });
|
|
218
|
+
assert.strictEqual(result.status, 1);
|
|
219
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
220
|
+
assert.match(result.report.reason, /timeout/i);
|
|
221
|
+
|
|
222
|
+
fs.writeFileSync(path.join(contracted, '.harness', 'verify.json'), '{ bad');
|
|
223
|
+
result = invokeGate(contracted);
|
|
224
|
+
assert.strictEqual(result.status, 1);
|
|
225
|
+
assert.match(result.report.reason, /verify\.json/i);
|
|
226
|
+
|
|
227
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
228
|
+
version: 1,
|
|
229
|
+
unknown: true,
|
|
230
|
+
checks: [{ id: 'x', run: 'node -e "process.exit(0)"' }],
|
|
231
|
+
});
|
|
232
|
+
result = invokeGate(contracted);
|
|
233
|
+
assert.strictEqual(result.status, 1);
|
|
234
|
+
assert.match(result.report.reason, /unknown/i);
|
|
235
|
+
|
|
236
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
237
|
+
version: 1,
|
|
238
|
+
roots: ['..'],
|
|
239
|
+
checks: [{ id: 'escape', run: 'node -e "process.exit(0)"' }],
|
|
240
|
+
});
|
|
241
|
+
result = invokeGate(contracted);
|
|
242
|
+
assert.strictEqual(result.status, 1);
|
|
243
|
+
assert.match(result.report.reason, /inside|escape|root/i);
|
|
244
|
+
|
|
245
|
+
// Multi-root contract reports every root and applies worst-root status.
|
|
246
|
+
fs.mkdirSync(path.join(contracted, 'sub', 'a'), { recursive: true });
|
|
247
|
+
fs.mkdirSync(path.join(contracted, 'sub', 'b'), { recursive: true });
|
|
248
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
249
|
+
version: 1,
|
|
250
|
+
roots: ['.', 'sub/a', 'sub/b'],
|
|
251
|
+
checks: [
|
|
252
|
+
{ id: 'root', run: 'node -e "process.exit(0)"', cwd: '.' },
|
|
253
|
+
{ id: 'a', run: 'node -e "process.exit(0)"', cwd: 'sub/a' },
|
|
254
|
+
{ id: 'b', run: 'node -e "process.exit(0)"', cwd: 'sub/b' },
|
|
255
|
+
],
|
|
256
|
+
});
|
|
257
|
+
result = invokeGate(contracted);
|
|
258
|
+
assert.strictEqual(result.report.status, 'PASSED');
|
|
259
|
+
assert.strictEqual(result.report.roots.length, 3);
|
|
260
|
+
assert.ok(result.report.roots.every(root => root.status === 'PASSED'));
|
|
261
|
+
|
|
262
|
+
writeJson(path.join(contracted, '.harness', 'verify.json'), {
|
|
263
|
+
version: 1,
|
|
264
|
+
roots: ['.', 'sub/a', 'sub/b'],
|
|
265
|
+
checks: [
|
|
266
|
+
{ id: 'root', run: 'node -e "process.exit(0)"', cwd: '.' },
|
|
267
|
+
{ id: 'a', run: 'node -e "process.exit(8)"', cwd: 'sub/a' },
|
|
268
|
+
{ id: 'b', run: 'node -e "process.exit(0)"', cwd: 'sub/b' },
|
|
269
|
+
],
|
|
270
|
+
});
|
|
271
|
+
result = invokeGate(contracted);
|
|
272
|
+
assert.strictEqual(result.status, 1);
|
|
273
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
274
|
+
assert.strictEqual(result.report.roots.find(root => /sub[\\/]a$/.test(root.path)).status, 'FAILED');
|
|
275
|
+
|
|
276
|
+
// No-contract multi-root discovery enumerates initialized submodules.
|
|
277
|
+
const childA = path.join(temp, 'child-a');
|
|
278
|
+
const childB = path.join(temp, 'child-b');
|
|
279
|
+
for (const child of [childA, childB]) {
|
|
280
|
+
initRepo(child);
|
|
281
|
+
writeJson(path.join(child, 'package.json'), packageScript({ test: 'node -e "process.exit(0)"' }));
|
|
282
|
+
git(child, ['add', 'package.json']);
|
|
283
|
+
git(child, ['commit', '-m', 'add passing test']);
|
|
284
|
+
}
|
|
285
|
+
const multi = path.join(temp, 'multi');
|
|
286
|
+
initRepo(multi);
|
|
287
|
+
writeJson(path.join(multi, 'package.json'), packageScript({ test: 'node -e "process.exit(0)"' }));
|
|
288
|
+
git(multi, ['add', 'package.json']);
|
|
289
|
+
git(multi, ['commit', '-m', 'parent test']);
|
|
290
|
+
git(multi, ['-c', 'protocol.file.allow=always', 'submodule', 'add', childA, 'sub/a']);
|
|
291
|
+
git(multi, ['-c', 'protocol.file.allow=always', 'submodule', 'add', childB, 'sub/b']);
|
|
292
|
+
git(multi, ['commit', '-am', 'add submodules']);
|
|
293
|
+
result = invokeGate(multi);
|
|
294
|
+
assert.strictEqual(result.report.status, 'PASSED');
|
|
295
|
+
assert.strictEqual(result.report.roots.length, 3);
|
|
296
|
+
|
|
297
|
+
writeJson(path.join(multi, 'sub', 'b', 'package.json'), packageScript({ test: 'node -e "process.exit(6)"' }));
|
|
298
|
+
result = invokeGate(multi);
|
|
299
|
+
assert.strictEqual(result.status, 1);
|
|
300
|
+
assert.strictEqual(result.report.status, 'FAILED');
|
|
301
|
+
assert.strictEqual(result.report.roots.find(root => /sub[\\/]b$/.test(root.path)).status, 'FAILED');
|
|
302
|
+
|
|
303
|
+
// Hook-side recognition uses the same project signals and contract.
|
|
304
|
+
const classifyRepo = path.join(temp, 'classify');
|
|
305
|
+
initRepo(classifyRepo);
|
|
306
|
+
fs.writeFileSync(path.join(classifyRepo, '.pre-commit-config.yaml'), 'repos: []\n');
|
|
307
|
+
fs.writeFileSync(path.join(classifyRepo, 'Makefile'), 'test:\n\t@echo ok\n');
|
|
308
|
+
fs.writeFileSync(path.join(classifyRepo, 'justfile'), 'ci:\n @echo ok\n');
|
|
309
|
+
fs.writeFileSync(path.join(classifyRepo, 'tox.ini'), '[tox]\nenvlist=py\n');
|
|
310
|
+
fs.writeFileSync(path.join(classifyRepo, 'Cargo.toml'), '[package]\nname="x"\nversion="0.1.0"\n');
|
|
311
|
+
fs.writeFileSync(path.join(classifyRepo, 'go.mod'), 'module example.invalid/x\n\ngo 1.22\n');
|
|
312
|
+
fs.writeFileSync(path.join(classifyRepo, 'pyproject.toml'), '[tool.pytest.ini_options]\naddopts="-q"\n');
|
|
313
|
+
writeJson(path.join(classifyRepo, '.harness', 'verify.json'), {
|
|
314
|
+
version: 1,
|
|
315
|
+
checks: [
|
|
316
|
+
{ id: 'precommit', run: 'pre-commit run --all-files' },
|
|
317
|
+
{ id: 'e2e', run: 'bash scripts/e2e.sh' },
|
|
318
|
+
],
|
|
319
|
+
});
|
|
320
|
+
fs.mkdirSync(path.join(classifyRepo, '.git', 'hooks'), { recursive: true });
|
|
321
|
+
const preCommitHook = path.join(classifyRepo, '.git', 'hooks', 'pre-commit');
|
|
322
|
+
fs.writeFileSync(preCommitHook, '#!/bin/sh\nexit 0\n');
|
|
323
|
+
assert.strictEqual(hasCommitHook(classifyRepo, 'linux'), false, 'a non-executable POSIX hook must not count as verification');
|
|
324
|
+
if (process.platform !== 'win32') fs.chmodSync(preCommitHook, 0o755);
|
|
325
|
+
|
|
326
|
+
assert.strictEqual(isVerificationShell('pre-commit run --all-files', { cwd: classifyRepo }), true);
|
|
327
|
+
assert.strictEqual(isVerificationShell('bash scripts/e2e.sh', { cwd: classifyRepo }), true);
|
|
328
|
+
assert.strictEqual(isVerificationShell('git commit -m x', { cwd: classifyRepo }), true);
|
|
329
|
+
assert.strictEqual(isVerificationShell('git commit --no-verify -m x', { cwd: classifyRepo }), false);
|
|
330
|
+
assert.strictEqual(isVerificationShell('git commit -n -m x', { cwd: classifyRepo }), false);
|
|
331
|
+
assert.strictEqual(isVerificationShell('git commit -an -m x', { cwd: classifyRepo }), false, 'combined -n must still bypass hooks');
|
|
332
|
+
if (process.platform !== 'win32') {
|
|
333
|
+
fs.chmodSync(preCommitHook, 0o644);
|
|
334
|
+
assert.strictEqual(isVerificationShell('git commit -m x', { cwd: classifyRepo }), false, 'a non-executable POSIX hook cannot verify a commit');
|
|
335
|
+
fs.chmodSync(preCommitHook, 0o755);
|
|
336
|
+
}
|
|
337
|
+
|
|
338
|
+
// An invalid authoritative contract must fail closed in the hook classifier too.
|
|
339
|
+
// The verify gate reports FAILED here, so legacy command matching must not
|
|
340
|
+
// silently manufacture lastVerifyAt evidence from the same workspace.
|
|
341
|
+
fs.writeFileSync(path.join(classifyRepo, '.harness', 'verify.json'), '{ broken');
|
|
342
|
+
assert.strictEqual(isVerificationShell('npm test', { cwd: classifyRepo }), false, 'invalid contract must suppress legacy verifier fallback');
|
|
343
|
+
assert.strictEqual(isVerificationShell('pre-commit run --all-files', { cwd: classifyRepo }), false, 'invalid contract must suppress detector-based evidence');
|
|
344
|
+
|
|
345
|
+
fs.rmSync(path.join(classifyRepo, '.harness', 'verify.json'));
|
|
346
|
+
assert.strictEqual(isVerificationShell('git commit -m x', { cwd: classifyRepo }), false, 'commit is not full verification without a declared hook contract');
|
|
347
|
+
assert.strictEqual(isVerificationShell('pre-commit run --all-files', { cwd: classifyRepo }), true);
|
|
348
|
+
assert.strictEqual(isVerificationShell('make test', { cwd: classifyRepo }), true);
|
|
349
|
+
assert.strictEqual(isVerificationShell('just ci', { cwd: classifyRepo }), true);
|
|
350
|
+
assert.strictEqual(isVerificationShell('tox', { cwd: classifyRepo }), true);
|
|
351
|
+
assert.strictEqual(isVerificationShell('cargo test', { cwd: classifyRepo }), true);
|
|
352
|
+
assert.strictEqual(isVerificationShell('go test ./...', { cwd: classifyRepo }), true);
|
|
353
|
+
assert.strictEqual(isVerificationShell('uv run pytest', { cwd: classifyRepo }), true);
|
|
354
|
+
assert.strictEqual(isVerificationShell('npm run build', { cwd: classifyRepo }), true, 'legacy build evidence remains recognized without a contract');
|
|
355
|
+
assert.strictEqual(isVerificationShell('npx tsc --noEmit', { cwd: classifyRepo }), true, 'legacy typecheck evidence remains recognized without a contract');
|
|
356
|
+
assert.strictEqual(isVerificationShell('echo test', { cwd: classifyRepo }), false, 'plain words must not become verification evidence');
|
|
357
|
+
|
|
358
|
+
// Hook evidence: successful declared pre-commit verification after an edit
|
|
359
|
+
// updates lastVerifyAt, so Stop does not emit a stale verification reminder.
|
|
360
|
+
writeJson(path.join(classifyRepo, '.harness', 'verify.json'), {
|
|
361
|
+
version: 1,
|
|
362
|
+
checks: [{ id: 'precommit', run: 'pre-commit run --all-files' }],
|
|
363
|
+
});
|
|
364
|
+
const stateHome = path.join(temp, 'state-home');
|
|
365
|
+
const session = 'verify-contract-hook';
|
|
366
|
+
const stateEnv = { HARNESS_STATE_HOME: stateHome, CLAUDE: '1' };
|
|
367
|
+
const statePersist = path.join(ROOT, 'hooks', 'scripts', 'state-persist.js');
|
|
368
|
+
const stopGate = path.join(ROOT, 'hooks', 'scripts', 'stop-gate.js');
|
|
369
|
+
const invokeHook = (script, payload) => spawnSync(process.execPath, [script], {
|
|
370
|
+
cwd: classifyRepo,
|
|
371
|
+
input: JSON.stringify(payload),
|
|
372
|
+
encoding: 'utf8',
|
|
373
|
+
env: { ...process.env, ...stateEnv },
|
|
374
|
+
});
|
|
375
|
+
|
|
376
|
+
fs.appendFileSync(path.join(classifyRepo, 'README.md'), 'dirty\n');
|
|
377
|
+
let hook = invokeHook(statePersist, {
|
|
378
|
+
session_id: session,
|
|
379
|
+
cwd: classifyRepo,
|
|
380
|
+
hook_event_name: 'PostToolUse',
|
|
381
|
+
tool_name: 'Edit',
|
|
382
|
+
tool_input: { file_path: path.join(classifyRepo, 'README.md') },
|
|
383
|
+
tool_response: { stdout: '', exitCode: 0 },
|
|
384
|
+
});
|
|
385
|
+
assert.strictEqual(hook.status, 0);
|
|
386
|
+
|
|
387
|
+
hook = invokeHook(statePersist, {
|
|
388
|
+
session_id: session,
|
|
389
|
+
cwd: classifyRepo,
|
|
390
|
+
hook_event_name: 'PostToolUse',
|
|
391
|
+
tool_name: 'Bash',
|
|
392
|
+
tool_input: { command: 'pre-commit run --all-files' },
|
|
393
|
+
tool_response: { stdout: 'ok', exitCode: 0 },
|
|
394
|
+
});
|
|
395
|
+
assert.strictEqual(hook.status, 0);
|
|
396
|
+
|
|
397
|
+
const stateFile = path.join(
|
|
398
|
+
stateHome,
|
|
399
|
+
'workspaces',
|
|
400
|
+
getWorkspaceKey(classifyRepo),
|
|
401
|
+
'state',
|
|
402
|
+
'sessions',
|
|
403
|
+
session,
|
|
404
|
+
'handoff-state.json',
|
|
405
|
+
);
|
|
406
|
+
const handoff = JSON.parse(fs.readFileSync(stateFile, 'utf8'));
|
|
407
|
+
assert.ok(handoff.lastVerifyAt >= handoff.lastEditAt, JSON.stringify(handoff));
|
|
408
|
+
|
|
409
|
+
const stop = invokeHook(stopGate, { session_id: session, cwd: classifyRepo });
|
|
410
|
+
assert.strictEqual(stop.status, 0);
|
|
411
|
+
assert.doesNotMatch(String(stop.stderr || ''), /Verification Reminder/i);
|
|
412
|
+
|
|
413
|
+
console.log('Issue #242 verification contract: taxonomy, discovery, execution safety, multi-root scope, hook recognition, and Stop evidence verified.');
|
|
414
|
+
} finally {
|
|
415
|
+
fs.rmSync(temp, { recursive: true, force: true });
|
|
416
|
+
}
|
package/ci/mutation-check.js
CHANGED
|
@@ -15,6 +15,7 @@ const checks = [
|
|
|
15
15
|
['transcript command qualifiers', 'behavioral-evals/transcript-parser.js', 'ci/mechanism-2t-transcript-parser.test.js'],
|
|
16
16
|
['fixture path boundaries', 'behavioral-evals/run.js', 'ci/mechanism-2j-behavioral-runner.test.js'],
|
|
17
17
|
['unbound workspace identity', 'scripts/lib/workspace.js', 'ci/mechanism-2u-issue-42-regressions.test.js'],
|
|
18
|
+
['verification contract precedence', 'harness-everything/scripts/verify-gate.js', 'ci/mechanism-35-verification-contract.test.js'],
|
|
18
19
|
];
|
|
19
20
|
|
|
20
21
|
let failures = 0;
|
|
@@ -32,6 +33,7 @@ for (const [label, target, testFile] of checks) {
|
|
|
32
33
|
'behavioral-evals/transcript-parser.js': 'transcript-command',
|
|
33
34
|
'behavioral-evals/run.js': 'runner-fixture',
|
|
34
35
|
'scripts/lib/workspace.js': 'workspace-identity',
|
|
36
|
+
'harness-everything/scripts/verify-gate.js': 'verify-contract-precedence',
|
|
35
37
|
}[target],
|
|
36
38
|
HARNESS_MUTATION_TARGET: path.join(ROOT, target),
|
|
37
39
|
NODE_OPTIONS: `--require=${loader}`,
|
package/ci/mutation-loader.js
CHANGED
|
@@ -40,6 +40,10 @@ const mutations = {
|
|
|
40
40
|
' if (!workspaceRoot) return UNBOUND_WORKSPACE_KEY;',
|
|
41
41
|
' if (!workspaceRoot) return path.basename(process.cwd()); // mutation: derive unbound identity from cwd'
|
|
42
42
|
),
|
|
43
|
+
'verify-contract-precedence': (source) => source.replace(
|
|
44
|
+
' finalize(contract ? contractPlan(contract) : discoveredPlan());',
|
|
45
|
+
' finalize(discoveredPlan()); // mutation: ignore the authoritative project contract'
|
|
46
|
+
),
|
|
43
47
|
};
|
|
44
48
|
|
|
45
49
|
if (target && mutation && mutations[mutation]) {
|
package/docs/audit.md
CHANGED
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
Maintainer-facing audit record. If you just want to know whether Harness works and how to verify it yourself, start at [VERIFICATION.md](../VERIFICATION.md) instead — this file records *how the scores below were obtained* and what changed between audit cycles.
|
|
4
4
|
|
|
5
|
+
> **Current verification-gate note (2026-09-24):** project verification now supports an authoritative `.harness/verify.json` contract, read-only ecosystem discovery, explicit `UNCHECKED_NO_CHECKS` versus `UNCHECKED_DISCOVERED`, and multi-root reporting. Hook-side verification evidence uses the same project-aware command model. Deterministic mechanism evidence is tracked under issue #242; this note does not claim new live-host evidence.
|
|
6
|
+
|
|
5
7
|
> **Historical snapshot boundary:** the scorecards below are dated audits and are intentionally preserved as evidence of what was measured on each date. Platform capabilities have changed since the 2026-09-01 snapshot (including the OpenAI/Codex local plugin work and the OpenCode plugin implementation). Do not treat any dated platform row as current-state documentation; use [platform-capabilities.md](platform-capabilities.md) for the current matrix.
|
|
6
8
|
|
|
7
9
|
**Last self-audited: 2026-09-18**, by running the deterministic test suite and gates on macOS (Node.js 24.14.1) — not by reading the code and assuming it works. The 2026-09-01 Windows 11 snapshot is preserved below. Live-host and behavioral evidence layers were not re-run in this cycle; retained artifacts are referenced, not re-measured.
|
|
@@ -4,7 +4,7 @@ description: "Route software/project work through Harness: classify tier/topolog
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.26.0
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Harness Everything
|
|
@@ -20,7 +20,7 @@ Kernel routing entry.
|
|
|
20
20
|
|
|
21
21
|
## Invariants
|
|
22
22
|
1. **Route (MUST)** — establish scope/tier and selected/deferred topology.
|
|
23
|
-
2. **Verify (MUST)** — completion claims need objective evidence.
|
|
23
|
+
2. **Verify (MUST)** — completion claims need objective evidence; `verify-gate.js --json` distinguishes passed, failed, discovered-but-unrun, and absent project checks.
|
|
24
24
|
3. **Zoom out (MUST)** — after 3 same-signature failures, use `zoom-out`.
|
|
25
25
|
4. **Resolve suggestions (MUST)** — read/evaluate each suggested `SKILL.md`; applicable core contracts MUST run, otherwise retain a flow-grounded reason.
|
|
26
26
|
5. **Surface status (MUST)** — non-trivial work renders `### 🚦 Harness Status` with bold `Current`, `Read / Evidence`, `Next`; optional `Risk / Blocked`.
|