harness-everything 0.12.2 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/.github/workflows/ci.yml +2 -0
- package/CHANGELOG.md +28 -0
- package/ci/mechanism-2ai-workspace-fingerprint-batch.test.js +216 -0
- package/ci/mechanism-2aj-waza-reference-adapter.test.js +223 -0
- package/docs/repository-contract.md +1 -1
- package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +1 -1
- package/fable-mode/SKILL.md +1 -1
- package/fable-mode/execution-guardrails/SKILL.md +1 -1
- package/fable-mode/fable-haiku/SKILL.md +1 -1
- package/fable-mode/fable-opus/SKILL.md +1 -1
- package/fable-mode/fable-sonnet/SKILL.md +1 -1
- package/fable-mode/references/model-matrix.md +2 -0
- package/harness-everything/SKILL.md +1 -1
- package/harness-everything/references/triage-and-tiers.md +2 -0
- package/hooks/scripts/lib/workflow-isolation.js +82 -25
- package/hooks/scripts/state-persist.js +4 -2
- package/hooks/scripts/workflow-gate.js +4 -2
- package/multi-agent-workspace/SKILL.md +2 -2
- package/multi-agent-workspace/references/orchestration.md +2 -0
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +4 -2
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/hooks/scripts/lib/workflow-isolation.js +82 -25
- package/plugins/harness-everything/hooks/scripts/state-persist.js +4 -2
- package/plugins/harness-everything/hooks/scripts/workflow-gate.js +4 -2
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/references/model-matrix.md +2 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +2 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +2 -2
- package/plugins/harness-everything/skills/multi-agent-workspace/references/orchestration.md +2 -0
- package/plugins/harness-everything/skills/skill-creator/SKILL.md +1 -1
- package/plugins/harness-everything/skills/skill-creator/references/quality-checklist.md +2 -0
- package/plugins/harness-everything/skills/tdd/SKILL.md +1 -1
- package/plugins/harness-everything/skills/tdd/references/core-discipline.md +1 -1
- package/plugins/harness-everything/skills/tdd/references/quality-model.md +2 -2
- package/scripts/waza-harness-check.js +361 -0
- package/skill-creator/SKILL.md +1 -1
- package/skill-creator/references/quality-checklist.md +2 -0
- package/tdd/SKILL.md +1 -1
- package/tdd/references/core-discipline.md +1 -1
- package/tdd/references/quality-model.md +2 -2
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"name": "harness-everything",
|
|
11
11
|
"source": "./",
|
|
12
12
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
13
|
-
"version": "0.
|
|
13
|
+
"version": "0.13.0",
|
|
14
14
|
"author": {
|
|
15
15
|
"name": "Miya Daniel"
|
|
16
16
|
},
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "harness-everything",
|
|
3
3
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.13.0",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Miya Daniel"
|
|
7
7
|
},
|
package/.github/workflows/ci.yml
CHANGED
|
@@ -82,6 +82,8 @@ jobs:
|
|
|
82
82
|
chmod +x "$HOME/bin/waza"
|
|
83
83
|
echo "$HOME/bin" >> "$GITHUB_PATH"
|
|
84
84
|
"$HOME/bin/waza" --version
|
|
85
|
+
- name: Run Harness Waza adapter against canonical skills
|
|
86
|
+
run: npm run waza:check
|
|
85
87
|
- name: Check every canonical SKILL.md passes waza readiness gates
|
|
86
88
|
run: |
|
|
87
89
|
fail=0
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,34 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project are documented in this file.
|
|
4
4
|
|
|
5
|
+
## [0.13.0](https://github.com/dyphn1/Harness-everything/compare/v0.12.3...v0.13.0) (2026-09-19)
|
|
6
|
+
|
|
7
|
+
### Features
|
|
8
|
+
|
|
9
|
+
* **waza:** expose non-link readiness failures ([f9c7d24](https://github.com/dyphn1/Harness-everything/commit/f9c7d242eece2af76fd68a6686eaa9521dcce443))
|
|
10
|
+
|
|
11
|
+
### Bug Fixes
|
|
12
|
+
|
|
13
|
+
* **waza:** add Harness reference adapter ([5223690](https://github.com/dyphn1/Harness-everything/commit/52236905be8a3660b89bf43396ea3d7bca984126))
|
|
14
|
+
* **waza:** align reference adapter with main ([87b634d](https://github.com/dyphn1/Harness-everything/commit/87b634df1d5d7de57b1c5ca45e8505423764e240))
|
|
15
|
+
* **waza:** normalize template and repo-scope links ([25558ca](https://github.com/dyphn1/Harness-everything/commit/25558caaa39b8bafb4a825c1299ff746ffbae91b))
|
|
16
|
+
* **waza:** preserve Harness placeholder delimiters ([f697ae8](https://github.com/dyphn1/Harness-everything/commit/f697ae83b6de54718b2e969041adfaa20b5e8efc))
|
|
17
|
+
* **waza:** scope checks to canonical skills ([de7b78b](https://github.com/dyphn1/Harness-everything/commit/de7b78b1702b412466337a1ea84ec524cf864e60))
|
|
18
|
+
|
|
19
|
+
## [0.12.3](https://github.com/dyphn1/Harness-everything/compare/v0.12.2...v0.12.3) (2026-09-19)
|
|
20
|
+
|
|
21
|
+
### Bug Fixes
|
|
22
|
+
|
|
23
|
+
* **workflow:** degrade oversized fingerprints conservatively ([ea90f30](https://github.com/dyphn1/Harness-everything/commit/ea90f30a4495e6144017373463c5279fae563252))
|
|
24
|
+
* **workflow:** degrade oversized fingerprints conservatively ([a618d22](https://github.com/dyphn1/Harness-everything/commit/a618d2216955ab3594ecb31b45635dea94e9b109))
|
|
25
|
+
* **workflow:** settle oversized fingerprint probes conservatively ([aafce50](https://github.com/dyphn1/Harness-everything/commit/aafce508551d6564afc884da17555ed71f989836))
|
|
26
|
+
* **workflow:** settle oversized fingerprint probes conservatively ([3092dd6](https://github.com/dyphn1/Harness-everything/commit/3092dd60ed19de97c096f57278c26813f77c3ff7))
|
|
27
|
+
|
|
28
|
+
### Performance Improvements
|
|
29
|
+
|
|
30
|
+
* **workflow:** batch workspace content hashing ([ad0fc18](https://github.com/dyphn1/Harness-everything/commit/ad0fc18260c5a716d7711cb4b83cf9f5059dd21d))
|
|
31
|
+
* **workflow:** batch workspace content hashing ([2cdd6e3](https://github.com/dyphn1/Harness-everything/commit/2cdd6e3dc568c9d6007ca10de76b06b47ff39880))
|
|
32
|
+
|
|
5
33
|
## [0.12.2](https://github.com/dyphn1/Harness-everything/compare/v0.12.1...v0.12.2) (2026-09-19)
|
|
6
34
|
|
|
7
35
|
### Bug Fixes
|
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
const assert = require('assert');
|
|
5
|
+
const crypto = require('crypto');
|
|
6
|
+
const fs = require('fs');
|
|
7
|
+
const os = require('os');
|
|
8
|
+
const path = require('path');
|
|
9
|
+
const { spawnSync } = require('child_process');
|
|
10
|
+
|
|
11
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
12
|
+
const isolation = require(path.join(ROOT, 'hooks/scripts/lib/workflow-isolation'));
|
|
13
|
+
const pluginIsolation = require(path.join(ROOT, 'plugins/harness-everything/hooks/scripts/lib/workflow-isolation'));
|
|
14
|
+
const state = require(path.join(ROOT, 'hooks/scripts/lib/harness-state'));
|
|
15
|
+
|
|
16
|
+
const temp = fs.mkdtempSync(path.join(os.tmpdir(), 'harness-workspace-fingerprint-'));
|
|
17
|
+
const repo = path.join(temp, 'repo');
|
|
18
|
+
fs.mkdirSync(repo);
|
|
19
|
+
const env = {
|
|
20
|
+
...process.env,
|
|
21
|
+
HARNESS_STATE_HOME: path.join(temp, 'state'),
|
|
22
|
+
HARNESS_WORKSPACE_ROOT: repo,
|
|
23
|
+
};
|
|
24
|
+
process.env.HARNESS_STATE_HOME = env.HARNESS_STATE_HOME;
|
|
25
|
+
process.env.HARNESS_WORKSPACE_ROOT = env.HARNESS_WORKSPACE_ROOT;
|
|
26
|
+
|
|
27
|
+
let passed = 0;
|
|
28
|
+
function check(value, message) {
|
|
29
|
+
assert.ok(value, message);
|
|
30
|
+
passed++;
|
|
31
|
+
console.log('PASS ' + message);
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
function run(command, args, options = {}) {
|
|
35
|
+
const result = spawnSync(command, args, {
|
|
36
|
+
cwd: options.cwd || repo,
|
|
37
|
+
env,
|
|
38
|
+
input: options.input,
|
|
39
|
+
encoding: 'utf8',
|
|
40
|
+
windowsHide: true,
|
|
41
|
+
maxBuffer: 64 * 1024 * 1024,
|
|
42
|
+
});
|
|
43
|
+
assert.strictEqual(result.status, 0, result.stderr || result.stdout);
|
|
44
|
+
return result.stdout;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
function git(args) {
|
|
48
|
+
return run('git', args);
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function node(script, payload, args = []) {
|
|
52
|
+
return spawnSync(process.execPath, [path.join(ROOT, script), ...args], {
|
|
53
|
+
cwd: repo,
|
|
54
|
+
env,
|
|
55
|
+
input: payload === undefined ? undefined : JSON.stringify(payload),
|
|
56
|
+
encoding: 'utf8',
|
|
57
|
+
windowsHide: true,
|
|
58
|
+
maxBuffer: 64 * 1024 * 1024,
|
|
59
|
+
});
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
function gitRaw(args) {
|
|
63
|
+
return run('git', args);
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
function legacyHashWorktreePath(root, relative) {
|
|
67
|
+
const target = path.join(root, relative);
|
|
68
|
+
let stat;
|
|
69
|
+
try {
|
|
70
|
+
stat = fs.lstatSync(target);
|
|
71
|
+
} catch (error) {
|
|
72
|
+
if (error.code === 'ENOENT') return null;
|
|
73
|
+
throw error;
|
|
74
|
+
}
|
|
75
|
+
if (stat.isDirectory()) {
|
|
76
|
+
const result = spawnSync('git', ['-C', target, 'rev-parse', 'HEAD'], {
|
|
77
|
+
encoding: 'utf8',
|
|
78
|
+
windowsHide: true,
|
|
79
|
+
});
|
|
80
|
+
return { mode: '160000', oid: result.status === 0 ? result.stdout.trim() : 'directory' };
|
|
81
|
+
}
|
|
82
|
+
const mode = stat.isSymbolicLink() ? '120000' : ((stat.mode & 0o111) ? '100755' : '100644');
|
|
83
|
+
const result = spawnSync('git', ['hash-object', '--path=' + relative, '--', relative], {
|
|
84
|
+
cwd: root,
|
|
85
|
+
encoding: 'utf8',
|
|
86
|
+
windowsHide: true,
|
|
87
|
+
});
|
|
88
|
+
if (result.status !== 0) throw new Error('legacy fingerprint failed for ' + relative);
|
|
89
|
+
return { mode, oid: result.stdout.trim() };
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
function legacyFingerprint(cwd) {
|
|
93
|
+
const root = isolation.canonical(gitRaw(['rev-parse', '--show-toplevel']).trim());
|
|
94
|
+
const entries = new Map();
|
|
95
|
+
for (const record of gitRaw(['ls-files', '-s', '-z']).split('\0')) {
|
|
96
|
+
if (!record) continue;
|
|
97
|
+
const tab = record.indexOf('\t');
|
|
98
|
+
assert.ok(tab >= 0, 'legacy index record has a tab');
|
|
99
|
+
const meta = record.slice(0, tab).trim().split(/\s+/);
|
|
100
|
+
const relative = record.slice(tab + 1);
|
|
101
|
+
entries.set(relative, { relative, mode: meta[0], oid: meta[1] });
|
|
102
|
+
}
|
|
103
|
+
for (const relative of gitRaw(['diff-files', '--name-only', '-z']).split('\0').filter(Boolean)) {
|
|
104
|
+
const value = legacyHashWorktreePath(root, relative);
|
|
105
|
+
if (value) entries.set(relative, { relative, mode: value.mode, oid: value.oid });
|
|
106
|
+
else entries.delete(relative);
|
|
107
|
+
}
|
|
108
|
+
for (const relative of gitRaw(['ls-files', '--others', '--exclude-standard', '-z']).split('\0').filter(Boolean)) {
|
|
109
|
+
const value = legacyHashWorktreePath(root, relative);
|
|
110
|
+
if (value) entries.set(relative, { relative, mode: value.mode, oid: value.oid });
|
|
111
|
+
}
|
|
112
|
+
const digest = crypto.createHash('sha256');
|
|
113
|
+
digest.update('harness-workspace-fingerprint-v1\0');
|
|
114
|
+
for (const entry of [...entries.values()].sort((a, b) => a.relative.localeCompare(b.relative))) {
|
|
115
|
+
digest.update(entry.relative); digest.update('\0');
|
|
116
|
+
digest.update(entry.mode); digest.update('\0');
|
|
117
|
+
digest.update(entry.oid); digest.update('\0');
|
|
118
|
+
}
|
|
119
|
+
return digest.digest('hex');
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
try {
|
|
123
|
+
git(['init']);
|
|
124
|
+
fs.writeFileSync(path.join(repo, 'tracked.txt'), 'tracked\n');
|
|
125
|
+
fs.writeFileSync(path.join(repo, 'delete-me.txt'), 'delete me\n');
|
|
126
|
+
git(['add', '.']);
|
|
127
|
+
git(['-c', 'user.name=Harness Test', '-c', 'user.email=harness@example.invalid', 'commit', '-m', 'fixture']);
|
|
128
|
+
|
|
129
|
+
const cleanLegacy = legacyFingerprint(repo);
|
|
130
|
+
check(isolation.workspaceFingerprint(repo) === cleanLegacy,
|
|
131
|
+
'#169 batched fingerprint matches the legacy clean-worktree fingerprint');
|
|
132
|
+
check(pluginIsolation.workspaceFingerprint(repo) === cleanLegacy,
|
|
133
|
+
'#169 packaged plugin fingerprint matches canonical/legacy on a clean worktree');
|
|
134
|
+
|
|
135
|
+
fs.writeFileSync(path.join(repo, 'tracked.txt'), 'tracked changed\n');
|
|
136
|
+
fs.writeFileSync(path.join(repo, 'untracked.txt'), 'untracked\n');
|
|
137
|
+
const dirtyLegacy = legacyFingerprint(repo);
|
|
138
|
+
const dirtyBatched = isolation.workspaceFingerprint(repo);
|
|
139
|
+
check(dirtyBatched === dirtyLegacy,
|
|
140
|
+
'#169 batched fingerprint matches legacy for dirty tracked + untracked content');
|
|
141
|
+
check(pluginIsolation.workspaceFingerprint(repo) === dirtyLegacy,
|
|
142
|
+
'#169 packaged plugin batched fingerprint matches legacy dirty semantics');
|
|
143
|
+
|
|
144
|
+
git(['add', 'tracked.txt', 'untracked.txt']);
|
|
145
|
+
check(isolation.workspaceFingerprint(repo) === dirtyBatched,
|
|
146
|
+
'#169 staging visible content does not change the workspace fingerprint');
|
|
147
|
+
|
|
148
|
+
fs.unlinkSync(path.join(repo, 'delete-me.txt'));
|
|
149
|
+
check(isolation.workspaceFingerprint(repo) === legacyFingerprint(repo),
|
|
150
|
+
'#169 tracked deletion keeps legacy fingerprint semantics');
|
|
151
|
+
|
|
152
|
+
git(['reset', '--hard', 'HEAD']);
|
|
153
|
+
git(['clean', '-fdx']);
|
|
154
|
+
const bulk = path.join(repo, 'bulk');
|
|
155
|
+
fs.mkdirSync(bulk);
|
|
156
|
+
for (let i = 0; i < 1000; i++) {
|
|
157
|
+
fs.writeFileSync(path.join(bulk, 'file-' + String(i).padStart(4, '0') + '.txt'), 'x\n');
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
const startedAt = process.hrtime.bigint();
|
|
161
|
+
const bulkFingerprint = isolation.workspaceFingerprint(repo);
|
|
162
|
+
const elapsedMs = Number(process.hrtime.bigint() - startedAt) / 1e6;
|
|
163
|
+
check(/^[a-f0-9]{64}$/.test(bulkFingerprint),
|
|
164
|
+
'#169 1000-untracked-file fingerprint returns a valid digest');
|
|
165
|
+
check(elapsedMs < 5000,
|
|
166
|
+
'#169 1000-untracked-file fingerprint stays below 5s regression ceiling (actual ' + elapsedMs.toFixed(1) + 'ms)');
|
|
167
|
+
check(pluginIsolation.workspaceFingerprint(repo) === bulkFingerprint,
|
|
168
|
+
'#169 packaged plugin fingerprint remains byte-identical for 1000 untracked files');
|
|
169
|
+
|
|
170
|
+
let limitError = null;
|
|
171
|
+
try {
|
|
172
|
+
isolation.workspaceFingerprint(repo, { maxPaths: 10 });
|
|
173
|
+
} catch (error) {
|
|
174
|
+
limitError = error;
|
|
175
|
+
}
|
|
176
|
+
check(limitError?.code === 'HARNESS_FINGERPRINT_LIMIT' &&
|
|
177
|
+
limitError.count === 1000 &&
|
|
178
|
+
limitError.maxPaths === 10,
|
|
179
|
+
'#169 explicit fingerprint cap reports a structured conservative-fallback signal');
|
|
180
|
+
|
|
181
|
+
for (let i = 1000; i < 5101; i++) {
|
|
182
|
+
fs.writeFileSync(path.join(bulk, 'file-' + String(i).padStart(4, '0') + '.txt'), 'x\n');
|
|
183
|
+
}
|
|
184
|
+
const sessionId = 'workspace-fingerprint-limit';
|
|
185
|
+
const payload = { session_id: sessionId, cwd: repo };
|
|
186
|
+
const routed = node('harness-everything/scripts/kernel-router.js', {
|
|
187
|
+
...payload,
|
|
188
|
+
prompt: 'Fix this checkout bug with a regression test',
|
|
189
|
+
});
|
|
190
|
+
check(routed.status === 0, '#169 large-workspace Tier-2 fixture routes normally');
|
|
191
|
+
const started = node('hooks/scripts/workflow-disposition.js', undefined, ['start', '--session-id', sessionId]);
|
|
192
|
+
check(started.status === 0, '#169 large-workspace Tier-2 fixture starts normally');
|
|
193
|
+
|
|
194
|
+
const admitted = node('hooks/scripts/workflow-gate.js', {
|
|
195
|
+
...payload,
|
|
196
|
+
tool_name: 'Bash',
|
|
197
|
+
tool_use_id: 'toolu_fingerprint_limit',
|
|
198
|
+
tool_input: { command: 'node large-fingerprint.js' },
|
|
199
|
+
});
|
|
200
|
+
check(admitted.status === 0,
|
|
201
|
+
'#169 >5000 observed paths degrade conservatively instead of blocking the shell call');
|
|
202
|
+
const workflowFile = path.join(state.getSessionDir(repo, sessionId), 'workflow-run.json');
|
|
203
|
+
const workflow = JSON.parse(fs.readFileSync(workflowFile, 'utf8'));
|
|
204
|
+
check(workflow.state === 'running' &&
|
|
205
|
+
workflow.budget?.counters?.iterations === 1 &&
|
|
206
|
+
workflow.lastMutationAt > 0 &&
|
|
207
|
+
workflow.budget?.events?.at(-1)?.evidence === 'shell:fingerprint-limit',
|
|
208
|
+
'#169 fingerprint-limit fallback records one auditable conservative mutation');
|
|
209
|
+
|
|
210
|
+
console.log('PASS: batched workspace fingerprint (' + passed + ' assertions, 1000-file ' + elapsedMs.toFixed(1) + 'ms)');
|
|
211
|
+
} finally {
|
|
212
|
+
if (!temp.startsWith(path.join(os.tmpdir(), 'harness-workspace-fingerprint-'))) {
|
|
213
|
+
throw new Error('unsafe fixture cleanup');
|
|
214
|
+
}
|
|
215
|
+
fs.rmSync(temp, { recursive: true, force: true });
|
|
216
|
+
}
|
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
const assert = require('assert');
|
|
5
|
+
const fs = require('fs');
|
|
6
|
+
const path = require('path');
|
|
7
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
8
|
+
const { discoverSkills, extractCandidates } = require('./reference-check');
|
|
9
|
+
const {
|
|
10
|
+
harnessReachableReferenceFiles,
|
|
11
|
+
classifyReportedLink,
|
|
12
|
+
normalizeSkillReport,
|
|
13
|
+
normalizeReport,
|
|
14
|
+
runCanonicalWazaChecks,
|
|
15
|
+
} = require('../scripts/waza-harness-check');
|
|
16
|
+
|
|
17
|
+
let passed = 0;
|
|
18
|
+
function check(value, message) {
|
|
19
|
+
assert.ok(value, message);
|
|
20
|
+
passed++;
|
|
21
|
+
console.log('PASS ' + message);
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
const skills = discoverSkills(ROOT);
|
|
25
|
+
check(skills.length >= 20, 'discovers the canonical Harness skill set');
|
|
26
|
+
|
|
27
|
+
const invoked = [];
|
|
28
|
+
const fakeSpawn = (command, args, options) => {
|
|
29
|
+
invoked.push({ command, args, cwd: options.cwd });
|
|
30
|
+
const skillPath = args[1];
|
|
31
|
+
const name = path.basename(skillPath);
|
|
32
|
+
return {
|
|
33
|
+
status: 0,
|
|
34
|
+
stdout: JSON.stringify({
|
|
35
|
+
timestamp: new Date(0).toISOString(),
|
|
36
|
+
skills: [{
|
|
37
|
+
name,
|
|
38
|
+
path: path.join(skillPath, 'SKILL.md'),
|
|
39
|
+
ready: true,
|
|
40
|
+
compliance: { level: 'High' },
|
|
41
|
+
tokenBudget: { exceeded: false },
|
|
42
|
+
specCompliance: [],
|
|
43
|
+
links: { passed: true },
|
|
44
|
+
}],
|
|
45
|
+
}),
|
|
46
|
+
stderr: '',
|
|
47
|
+
};
|
|
48
|
+
};
|
|
49
|
+
const scoped = runCanonicalWazaChecks(ROOT, 'waza', fakeSpawn);
|
|
50
|
+
check(scoped.skills.length === skills.length && invoked.length === skills.length,
|
|
51
|
+
'adapter invokes Waza exactly once per canonical top-level skill');
|
|
52
|
+
check(invoked.every((call, index) =>
|
|
53
|
+
call.command === 'waza' &&
|
|
54
|
+
call.args[0] === 'check' &&
|
|
55
|
+
call.args[1] === path.join(ROOT, skills[index]) &&
|
|
56
|
+
call.args[2] === '--format' &&
|
|
57
|
+
call.args[3] === 'json' &&
|
|
58
|
+
call.cwd === ROOT),
|
|
59
|
+
'Waza receives explicit canonical skill paths instead of recursive workspace discovery');
|
|
60
|
+
|
|
61
|
+
let skillsWithPlaceholderReferences = 0;
|
|
62
|
+
let totalResolved = 0;
|
|
63
|
+
for (const skill of skills) {
|
|
64
|
+
const skillFile = path.join(ROOT, skill, 'SKILL.md');
|
|
65
|
+
const text = fs.readFileSync(skillFile, 'utf8');
|
|
66
|
+
const directReferenceTargets = extractCandidates(text)
|
|
67
|
+
.filter(value => /^<this-skill-dir>\/references\//.test(value))
|
|
68
|
+
.map(value => value.replace(/^<this-skill-dir>\//, ''));
|
|
69
|
+
if (!directReferenceTargets.length) continue;
|
|
70
|
+
|
|
71
|
+
skillsWithPlaceholderReferences++;
|
|
72
|
+
const reachable = harnessReachableReferenceFiles(ROOT, skill);
|
|
73
|
+
for (const target of directReferenceTargets) {
|
|
74
|
+
check(reachable.has(target), skill + ' placeholder reference is reachable: ' + target);
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
const simulated = {
|
|
78
|
+
name: skill,
|
|
79
|
+
path: path.join(ROOT, skill),
|
|
80
|
+
ready: false,
|
|
81
|
+
compliance: { level: 'High' },
|
|
82
|
+
tokenBudget: { exceeded: false },
|
|
83
|
+
specCompliance: [{ name: 'spec-frontmatter', passed: true }],
|
|
84
|
+
schema: { valid: true },
|
|
85
|
+
links: {
|
|
86
|
+
passed: false,
|
|
87
|
+
orphanedFiles: [...new Set(directReferenceTargets)],
|
|
88
|
+
},
|
|
89
|
+
};
|
|
90
|
+
const normalized = normalizeSkillReport(ROOT, simulated);
|
|
91
|
+
check(normalized.harnessCompatibility.remainingOrphanedFiles.length === 0,
|
|
92
|
+
skill + ' removes Waza orphan advisories for Harness-reachable references');
|
|
93
|
+
check(normalized.harnessCompatibility.resolvedPlaceholderOrphans.length === new Set(directReferenceTargets).size,
|
|
94
|
+
skill + ' reports the suppressed advisories explicitly');
|
|
95
|
+
check(normalized.harnessCompatibility.harnessReady === true,
|
|
96
|
+
skill + ' recomputes readiness without false local-orphan failures');
|
|
97
|
+
totalResolved += normalized.harnessCompatibility.resolvedPlaceholderOrphans.length;
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
check(skillsWithPlaceholderReferences > 0, 'at least one canonical skill uses the Harness placeholder reference contract');
|
|
101
|
+
check(totalResolved > 0, 'adapter resolves canonical placeholder-orphan advisories');
|
|
102
|
+
|
|
103
|
+
const nestedReferenceContract = {
|
|
104
|
+
'fable-mode': ['references/execution-phases.md'],
|
|
105
|
+
'harness-everything': ['references/router-workflow-plan.md', 'references/skill-registry.md'],
|
|
106
|
+
'multi-agent-workspace': ['references/agency-agents.md'],
|
|
107
|
+
'skill-creator': [
|
|
108
|
+
'references/authoring-workflow.md',
|
|
109
|
+
'references/testing-workflow.md',
|
|
110
|
+
'references/quality-principles.md',
|
|
111
|
+
'references/dynamic-generation-contract.md',
|
|
112
|
+
],
|
|
113
|
+
tdd: ['references/quality-model.md', 'references/unit-testing.md', 'references/integration-testing.md'],
|
|
114
|
+
};
|
|
115
|
+
for (const [skill, expected] of Object.entries(nestedReferenceContract)) {
|
|
116
|
+
const reachable = harnessReachableReferenceFiles(ROOT, skill);
|
|
117
|
+
for (const reference of expected) {
|
|
118
|
+
check(reachable.has(reference), skill + ' nested reference remains reachable: ' + reference);
|
|
119
|
+
}
|
|
120
|
+
const normalized = normalizeSkillReport(ROOT, {
|
|
121
|
+
name: skill,
|
|
122
|
+
path: path.join(ROOT, skill),
|
|
123
|
+
ready: false,
|
|
124
|
+
compliance: { level: 'High' },
|
|
125
|
+
tokenBudget: { exceeded: false },
|
|
126
|
+
specCompliance: [],
|
|
127
|
+
links: { passed: false, orphanedFiles: expected },
|
|
128
|
+
});
|
|
129
|
+
check(normalized.harnessCompatibility.remainingOrphanedFiles.length === 0,
|
|
130
|
+
skill + ' nested reference advisories are resolved through the reachable reference graph');
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
const fableContract = classifyReportedLink(ROOT, 'fable-mode', {
|
|
134
|
+
source: 'SKILL.md',
|
|
135
|
+
target: '<skills-repo-root>/harness-everything/scripts/verify-gate.js',
|
|
136
|
+
reason: 'link escapes skill directory',
|
|
137
|
+
});
|
|
138
|
+
check(fableContract.category === 'resolved-contract',
|
|
139
|
+
'<skills-repo-root> existing target is a resolved Harness contract link');
|
|
140
|
+
|
|
141
|
+
const genericTemplate = classifyReportedLink(ROOT, 'repo-docs', {
|
|
142
|
+
source: 'SKILL.md',
|
|
143
|
+
target: '<workspace>/docs/generated.md',
|
|
144
|
+
reason: 'target does not exist',
|
|
145
|
+
});
|
|
146
|
+
check(genericTemplate.category === 'template',
|
|
147
|
+
'generic runtime placeholder is classified as a template advisory');
|
|
148
|
+
|
|
149
|
+
const badPlaceholder = classifyReportedLink(ROOT, 'repo-docs', {
|
|
150
|
+
source: 'SKILL.md',
|
|
151
|
+
target: '<this-folder>/references/missing.md',
|
|
152
|
+
reason: 'target does not exist',
|
|
153
|
+
});
|
|
154
|
+
check(badPlaceholder.category === 'local',
|
|
155
|
+
'unknown placeholder remains a real local-link problem');
|
|
156
|
+
|
|
157
|
+
const repoScopeLink = classifyReportedLink(ROOT, 'multi-agent-workspace', {
|
|
158
|
+
source: 'workflows/01-init.md',
|
|
159
|
+
target: '../../README.md#supported-ai-ides--tools',
|
|
160
|
+
reason: 'link escapes skill directory',
|
|
161
|
+
});
|
|
162
|
+
check(repoScopeLink.category === 'resolved-contract' && repoScopeLink.issue.resolved === 'README.md',
|
|
163
|
+
'existing repo-scope relative link is resolved by the Harness repository contract');
|
|
164
|
+
|
|
165
|
+
const missingRepoScopeLink = classifyReportedLink(ROOT, 'multi-agent-workspace', {
|
|
166
|
+
source: 'workflows/01-init.md',
|
|
167
|
+
target: '../../definitely-missing.md',
|
|
168
|
+
reason: 'link escapes skill directory',
|
|
169
|
+
});
|
|
170
|
+
check(missingRepoScopeLink.category === 'local',
|
|
171
|
+
'missing repo-scope relative link remains a hard local-link problem');
|
|
172
|
+
|
|
173
|
+
const templateLink = classifyReportedLink(ROOT, 'repo-docs', {
|
|
174
|
+
source: 'templates/multi-skills-readme-template.md',
|
|
175
|
+
target: './skills/category/{skill-name}/SKILL.md',
|
|
176
|
+
reason: 'target does not exist',
|
|
177
|
+
});
|
|
178
|
+
check(templateLink.category === 'template',
|
|
179
|
+
'links inside generated-document templates remain visible template advisories');
|
|
180
|
+
|
|
181
|
+
const synthetic = normalizeReport(ROOT, {
|
|
182
|
+
timestamp: new Date(0).toISOString(),
|
|
183
|
+
skills: [{
|
|
184
|
+
name: 'harness-everything',
|
|
185
|
+
path: path.join(ROOT, 'harness-everything'),
|
|
186
|
+
ready: false,
|
|
187
|
+
compliance: { level: 'High' },
|
|
188
|
+
tokenBudget: { exceeded: false },
|
|
189
|
+
specCompliance: [{ name: 'spec-frontmatter', passed: true }],
|
|
190
|
+
links: {
|
|
191
|
+
passed: false,
|
|
192
|
+
orphanedFiles: ['references/triage-and-tiers.md'],
|
|
193
|
+
deadURLs: [{ source: 'SKILL.md', target: 'https://example.invalid', reason: 'DNS lookup failed' }],
|
|
194
|
+
brokenLinks: [{ source: 'SKILL.md', target: '<workspace>/generated.md', reason: 'target does not exist' }],
|
|
195
|
+
},
|
|
196
|
+
}],
|
|
197
|
+
});
|
|
198
|
+
const compatibility = synthetic.skills[0].harnessCompatibility;
|
|
199
|
+
check(compatibility.localLinksPassed === true,
|
|
200
|
+
'network and generic-template findings do not masquerade as broken local links');
|
|
201
|
+
check(compatibility.networkAdvisories.length === 1 && compatibility.templateAdvisories.length === 1,
|
|
202
|
+
'network and template findings are retained in separate advisory categories');
|
|
203
|
+
check(compatibility.harnessReady === true,
|
|
204
|
+
'normalized readiness remains green when only false-orphan/network/template advisories remain');
|
|
205
|
+
|
|
206
|
+
const nonLinkFailure = normalizeSkillReport(ROOT, {
|
|
207
|
+
name: 'repo-docs',
|
|
208
|
+
path: path.join(ROOT, 'repo-docs'),
|
|
209
|
+
ready: false,
|
|
210
|
+
compliance: { level: 'Medium' },
|
|
211
|
+
tokenBudget: { exceeded: true },
|
|
212
|
+
specCompliance: [{ name: 'frontmatter', passed: false }],
|
|
213
|
+
schema: { valid: false },
|
|
214
|
+
links: { passed: true },
|
|
215
|
+
});
|
|
216
|
+
check(nonLinkFailure.harnessCompatibility.harnessReady === false &&
|
|
217
|
+
nonLinkFailure.harnessCompatibility.nonLinkFailures.includes('compliance:Medium') &&
|
|
218
|
+
nonLinkFailure.harnessCompatibility.nonLinkFailures.includes('token-budget-exceeded') &&
|
|
219
|
+
nonLinkFailure.harnessCompatibility.nonLinkFailures.includes('spec:frontmatter') &&
|
|
220
|
+
nonLinkFailure.harnessCompatibility.nonLinkFailures.includes('schema-invalid'),
|
|
221
|
+
'non-link readiness failures remain explicit machine-readable blockers');
|
|
222
|
+
|
|
223
|
+
console.log('PASS: Waza/Harness reference adapter (' + passed + ' assertions across ' + skills.length + ' skills)');
|
|
@@ -19,7 +19,7 @@ This page is generated from checked-in runtime and workflow sources. Do not edit
|
|
|
19
19
|
| Workflow | Node.js versions | Explicit triggers | npm quality gates |
|
|
20
20
|
| --- | --- | --- | --- |
|
|
21
21
|
| `behavioral-evals.yml` | 24 | schedule, workflow_dispatch | n/a |
|
|
22
|
-
| `ci.yml` | 22, 24 | workflow_call | `test`, `test:mutations`, `test:release:semantics`, `test:consistency`, `test:references`, `test:release`, `test:collision`, `test:plugin:openai`, `test:plugin:submission`, `test:platform:compatibility` |
|
|
22
|
+
| `ci.yml` | 22, 24 | workflow_call | `test`, `test:mutations`, `test:release:semantics`, `waza:check`, `test:consistency`, `test:references`, `test:release`, `test:collision`, `test:plugin:openai`, `test:plugin:submission`, `test:platform:compatibility` |
|
|
23
23
|
| `release.yml` | 24 | workflow_dispatch | n/a |
|
|
24
24
|
|
|
25
25
|
## Drift prevention
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-multi-agent-workspace-001
|
|
2
2
|
name: On-scope prompt triggers multi-agent-workspace
|
|
3
|
-
description: "Scaffold a verified multi-agent workspace and select bounded specialists from an external agency-agents catalog without vendoring the full roster."
|
|
3
|
+
description: "Scaffold a verified multi-agent workspace and select bounded specialists from an external agency-agents catalog without vendoring the full external roster."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
inputs:
|
package/fable-mode/SKILL.md
CHANGED
|
@@ -37,3 +37,5 @@ The orchestrator still owns scope lock, at most two full replans, stage
|
|
|
37
37
|
contracts, worker non-recursion, cold verification, and escalation. The native
|
|
38
38
|
host TODO tracker or a Markdown checklist records stage progress; no CLI TODO
|
|
39
39
|
state machine is part of this contract.
|
|
40
|
+
|
|
41
|
+
See [execution phases](execution-phases.md) for the staged run contract.
|