cyber-aced 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/.claude-plugin/plugin.json +18 -0
  2. package/.plugin/pins.json +1 -0
  3. package/.plugin/plugin.json +18 -0
  4. package/agents/aced-case-judge.md +165 -0
  5. package/agents/aced-impl-judge.md +55 -0
  6. package/agents/aced-scenario-writer.md +76 -0
  7. package/agents/aced-spec-validator.md +66 -0
  8. package/governances/README.md +52 -0
  9. package/governances/agent-tool-output.md +86 -0
  10. package/governances/cli-resolution.md +102 -0
  11. package/governances/skill-design.md +258 -0
  12. package/governances/skill-repo-structure.md +173 -0
  13. package/package.json +44 -0
  14. package/readme.md +124 -0
  15. package/skills/aced-architect-impl/README.md +9 -0
  16. package/skills/aced-architect-impl/SKILL.md +73 -0
  17. package/skills/aced-builder-impl/README.md +14 -0
  18. package/skills/aced-builder-impl/SKILL.md +46 -0
  19. package/skills/aced-builder-spec/README.md +15 -0
  20. package/skills/aced-builder-spec/SKILL.md +130 -0
  21. package/skills/aced-fit/README.md +21 -0
  22. package/skills/aced-fit/SKILL.md +105 -0
  23. package/skills/aced-impl-producer/SKILL.md +75 -0
  24. package/skills/add-scenario/README.md +25 -0
  25. package/skills/add-scenario/SKILL.md +75 -0
  26. package/skills/bench/README.md +64 -0
  27. package/skills/bench/SKILL.md +199 -0
  28. package/skills/bench/scripts/bench-claude-code.mts +187 -0
  29. package/skills/bench/scripts/bench-compare.mts +369 -0
  30. package/skills/bench/scripts/bench-record.mts +133 -0
  31. package/skills/bench/scripts/bench.mts +930 -0
  32. package/skills/check-freshness/README.md +57 -0
  33. package/skills/check-freshness/SKILL.md +76 -0
  34. package/skills/check-freshness/scripts/check-freshness.mts +399 -0
  35. package/skills/compare/README.md +25 -0
  36. package/skills/compare/SKILL.md +87 -0
  37. package/skills/contribute-skill/README.md +25 -0
  38. package/skills/contribute-skill/SKILL.md +241 -0
  39. package/skills/contribute-skill/scripts/source.mts +188 -0
  40. package/skills/define-agent/README.md +40 -0
  41. package/skills/define-agent/SKILL.md +279 -0
  42. package/skills/define-command/README.md +36 -0
  43. package/skills/define-command/SKILL.md +174 -0
  44. package/skills/define-governance/SKILL.md +151 -0
  45. package/skills/define-skill/README.md +37 -0
  46. package/skills/define-skill/SKILL.md +200 -0
  47. package/skills/extract-situation/README.md +32 -0
  48. package/skills/extract-situation/SKILL.md +54 -0
  49. package/skills/extract-situation/scripts/extract-situation.mts +287 -0
  50. package/skills/improve/README.md +25 -0
  51. package/skills/improve/SKILL.md +100 -0
  52. package/skills/improve-skill/README.md +38 -0
  53. package/skills/improve-skill/SKILL.md +169 -0
  54. package/skills/improve-skill/references/check-definitions.md +215 -0
  55. package/skills/improve-skill/scripts/validate.mts +1033 -0
  56. package/skills/init-aced/README.md +5 -0
  57. package/skills/init-aced/SKILL.md +76 -0
  58. package/skills/init-aced/scripts/ensure-results-ignored.mts +117 -0
  59. package/skills/list-skills/README.md +19 -0
  60. package/skills/list-skills/SKILL.md +55 -0
  61. package/skills/list-skills/scripts/list-skills.mts +210 -0
  62. package/skills/manage/README.md +35 -0
  63. package/skills/manage/SKILL.md +65 -0
  64. package/skills/manage-model-runners/README.md +25 -0
  65. package/skills/manage-model-runners/SKILL.md +113 -0
  66. package/skills/manage-skill-dirs/README.md +17 -0
  67. package/skills/manage-skill-dirs/SKILL.md +54 -0
  68. package/skills/manage-skill-dirs/scripts/manage-skill-dirs.mts +328 -0
  69. package/skills/repair-private-skills/README.md +17 -0
  70. package/skills/repair-private-skills/SKILL.md +59 -0
  71. package/skills/repair-private-skills/scripts/repair-private-skills.mts +279 -0
  72. package/skills/report/README.md +25 -0
  73. package/skills/report/SKILL.md +74 -0
  74. package/skills/run/README.md +25 -0
  75. package/skills/run/SKILL.md +220 -0
  76. package/skills/skillify/README.md +25 -0
  77. package/skills/skillify/SKILL.md +141 -0
@@ -0,0 +1,57 @@
1
+ # check-freshness
2
+
3
+ Internal, non-invokable ACED engine. It decides whether a recorded eval result still describes the
4
+ configuration on disk, and returns `current`, `stale`, `incomplete`, or `absent`.
5
+
6
+ ## What it does
7
+
8
+ `run` writes an eval result to `.agents/aced/results/<target>/<timestamp>.json` carrying an
9
+ `evaluated` set — every input the run reports consuming, each as a repository path plus a SHA-256
10
+ hash. This engine resolves a node's target from its `eval.md`, selects the newest **readable** result
11
+ whose own `target` field is that subject, re-hashes each recorded path, and compares.
12
+
13
+ Two rules do the work:
14
+
15
+ - A **file** entry hashes the bytes read, so a content change is detectable and a timestamp never
16
+ stands in for one.
17
+ - A **directory** entry hashes the **names** the listing returned, so a file *added* to a directory
18
+ the subject loads from is detectable without re-resolving what the subject depends on now.
19
+
20
+ A directory the run expanded carries both its own entry and a `file` entry per file the listing
21
+ yielded — the directory hash covers names only, so without the per-file entries an edit inside that
22
+ directory would read `current` forever.
23
+
24
+ ## Why the record and not the tree
25
+
26
+ The first attempt at this capability inferred the subject's dependencies from outside: it guessed
27
+ the file set from a slug, treated modification time as a content change, and read a judge's prose
28
+ with a regex. All three guesses trace to one missing upstream contract — the producer recorded
29
+ nothing about what it evaluated. Fixing the producer turns freshness into a comparison instead of
30
+ four stacked heuristics, which is why every decision here reads only what the record names.
31
+
32
+ ## Selection is by what the result records
33
+
34
+ The newest result is the one whose **recorded timestamp** is greatest, not the one whose filename
35
+ sorts last, and a result belongs to the target its own `target` field names, not the directory it
36
+ happens to sit in. An unparseable result is skipped and named rather than thrown on, so one corrupt
37
+ record cannot blind the check to the readable ones beside it.
38
+
39
+ ## Limits it states rather than hides
40
+
41
+ `current` means the run's account still holds — not that the run read everything it should have. An
42
+ under-reporting run records a shorter set whose entries all match, and no verdict here can find it,
43
+ because the only witness to what was read is the same self-report under test. See the trust boundary
44
+ in `SKILL.md`.
45
+
46
+ Nothing consults this engine yet: wiring `run` and `improve` to it is `#476`. Until that lands it is
47
+ specified, tested, and reachable only by a human running it directly — a recorded gap, not an
48
+ oversight.
49
+
50
+ ## Tests
51
+
52
+ `scripts/check-freshness.test.mts` — one test per frozen scenario, with fixtures chosen so a
53
+ guessing implementation cannot pass: the result is filed under a deliberately wrong directory name,
54
+ alphabetical filename order is the reverse of recorded timestamp order, and one fixture rewrites a
55
+ file's modification time while leaving its bytes identical.
56
+
57
+ Spec: `.agents/specs/aced/eval-run/check-freshness/`.
@@ -0,0 +1,76 @@
1
+ ---
2
+ name: check-freshness
3
+ description: "Partial Skill: invoke by name only — the ACED eval-result freshness check — decides whether a recorded result still describes the configuration on disk, not user-triggered."
4
+ user-invocable: false
5
+ metadata:
6
+ internal: true
7
+ ---
8
+
9
+ # check-freshness
10
+
11
+ The **freshness check** — the deterministic engine that reads the newest eval result `run` recorded
12
+ for a target, re-hashes the paths that result recorded, and returns one of four verdicts. It is
13
+ **read-only**: it never writes, repairs, or deletes a result.
14
+
15
+ It answers the question **from the record itself**. `run` records the inputs it reports consuming and
16
+ their hashes; this re-hashes those same paths in the working tree and compares. Nothing is inferred:
17
+ no modification times, no guessed file sets, no guessed directory names.
18
+
19
+ ## Invocation
20
+
21
+ ```
22
+ node "<skill>/scripts/check-freshness.mts" --node <node-dir>
23
+ ```
24
+
25
+ `<node-dir>` holds the node's `eval.md` (whose `subject:` names the target) and its frozen
26
+ `.feature`. Exit status is **zero only for `current`**; every other verdict, and every fail-closed
27
+ path, exits non-zero.
28
+
29
+ It also exposes the hashing routine, so `run` records entries with the same implementation that
30
+ reads them back — never a second one that merely agrees today:
31
+
32
+ ```
33
+ node "<skill>/scripts/check-freshness.mts" --hash-file <path>
34
+ node "<skill>/scripts/check-freshness.mts" --hash-dir <path>
35
+ ```
36
+
37
+ ## The four verdicts
38
+
39
+ | Verdict | Means |
40
+ |---|---|
41
+ | `current` | Every input the run recorded still hashes as recorded. |
42
+ | `stale` | At least one recorded **subject input** changed, is gone, or (a directory) now lists different entries — no individual score can be relied on without re-running. |
43
+ | `incomplete` | Every subject input matches, but the recorded **frozen suite** changed — the scores survive, the suite grew past them. |
44
+ | `absent` | There is nothing to compare: no results directory, no result for this target, none readable, no evaluated set, or an evaluated set that contradicts the record carrying it. |
45
+
46
+ A node with no `eval.md`, or an `eval.md` with no `subject:` key, **fails closed and emits no
47
+ verdict** — a guard that cannot decide must not manufacture one, and `absent` would read as
48
+ "checked, nothing recorded".
49
+
50
+ ## Two boundaries, both deliberate
51
+
52
+ **Closed world.** It compares the inputs the result recorded and nothing else; it never re-resolves
53
+ what the subject depends on *now*. So `current` means "every input this result recorded still hashes
54
+ as recorded", never "the subject has not grown". Growth is caught at the producer — `run` records a
55
+ hash over the entry names of every directory it listed. Growth reaching the subject by a path `run`
56
+ never consumed stays invisible by construction: it was not an input, so the result stays truthful
57
+ about what it measured.
58
+
59
+ **Trust boundary.** `evaluated` is what the run **reports** consuming, not a verified trace. An
60
+ under-reporting run records a shorter set whose entries all match, and this answers `current` with
61
+ full confidence — a silent failure pointing the unsafe way (`#475`). What is checkable is
62
+ **coherence**: a set omitting the `.feature` whose scenarios the record scores, or the configuration
63
+ its own `target` names, contradicts the record it accompanies and reads `absent`. That catches only
64
+ an *inconsistent* under-reporter; a uniform one escapes.
65
+
66
+ Reading `current` as "the run's account still holds" is sound. Reading it as "the run read everything
67
+ it should have" is not, and this engine never claims the second.
68
+
69
+ ## Boundaries
70
+
71
+ Scores nothing (`run`), compares no two versions (`compare`), rolls nothing up (`report`), and does
72
+ not judge whether a recorded *pass* was well-founded (a judge-protocol question, `#477`). It decides
73
+ a verdict and reports it; **what a caller does with that verdict is the caller's** — wiring `run` and
74
+ `improve` to consult it is `#476`.
75
+
76
+ Spec: `.agents/specs/aced/eval-run/check-freshness/`.
@@ -0,0 +1,399 @@
1
+ #!/usr/bin/env node
2
+ // check-freshness — is a recorded eval result still current?
3
+ //
4
+ // Reads the newest result `run` recorded for a node's target, re-hashes the paths that result
5
+ // recorded, and returns current / stale / incomplete / absent. Nothing is inferred: no modification
6
+ // times, no guessed file sets, no guessed directory names. The rejected first attempt at this
7
+ // capability guessed all three, which is why every decision here reads only what the record names.
8
+ //
9
+ // TWO BOUNDARIES, both deliberate, both stated in .agents/specs/aced/eval-run/check-freshness/:
10
+ //
11
+ // Closed world — this compares the inputs the result recorded and NOTHING else. It never
12
+ // re-resolves what the subject depends on now. So `current` means "every input this result
13
+ // recorded still hashes as recorded", never "the subject has not grown". Growth is caught at the
14
+ // producer: `run` records a hash over the entries of every directory it listed, so a file added
15
+ // to a directory the subject loads from changes a recorded entry. Growth reaching the subject by
16
+ // a path `run` never consumed stays invisible BY CONSTRUCTION — it was not an input.
17
+ //
18
+ // Trust boundary — `evaluated` is what the run REPORTS consuming, not a verified trace. An
19
+ // under-reporting run records a shorter set whose every entry then matches, and this returns
20
+ // `current` with full confidence. That failure is silent and points the unsafe way; see #475.
21
+ // What IS checkable is coherence: a set omitting the .feature whose scenarios the record scores,
22
+ // or the configuration its own `target` names, contradicts the record it accompanies and reads
23
+ // `absent`. That catches only an INCONSISTENT under-reporter — a uniform one escapes.
24
+ //
25
+ // This engine owns the hashing implementation for BOTH sides of the comparison. `run` computes its
26
+ // entries by invoking `--hash-file` / `--hash-dir` here rather than reimplementing the rule: two
27
+ // schemes that merely agree today would surface their divergence as a permanent `stale`, never as
28
+ // an error. The RULE is `run`'s contract (eval-run/run/); this is its single implementation.
29
+ //
30
+ // Pure functions are exported for node:test; running the file directly drives the CLI. No
31
+ // dependencies — plain node strips the types.
32
+
33
+ import { createHash } from 'node:crypto'
34
+ import { existsSync, readdirSync, readFileSync, realpathSync } from 'node:fs'
35
+ import { join, relative, resolve } from 'node:path'
36
+ import { pathToFileURL } from 'node:url'
37
+
38
+ // ─── types ────────────────────────────────────────────────────────────────────
39
+
40
+ export type Verdict = 'current' | 'stale' | 'incomplete' | 'absent'
41
+
42
+ // `kind` is what makes the two hash rules distinguishable at read time. Without it a directory
43
+ // entry is indistinguishable from a file entry and would be re-hashed as bytes, which fails for
44
+ // every directory and would read as a permanent `stale`.
45
+ export interface EvaluatedEntry {
46
+ path: string
47
+ sha256: string
48
+ kind: 'file' | 'directory'
49
+ }
50
+
51
+ export interface ResultRecord {
52
+ timestamp?: string
53
+ target?: string
54
+ scenarios?: unknown[]
55
+ evaluated?: EvaluatedEntry[]
56
+ }
57
+
58
+ export interface LoadedResult {
59
+ file: string
60
+ record: ResultRecord
61
+ }
62
+
63
+ export interface Decision {
64
+ verdict: Verdict
65
+ // Why, in one line, for the reader. Never a second machine channel — the exit code is that.
66
+ reason: string
67
+ // Recorded inputs that no longer match. Populated for stale/incomplete only: `absent` has
68
+ // nothing to compare and `current` has nothing that failed.
69
+ mismatched: string[]
70
+ skipped: string[]
71
+ }
72
+
73
+ // ─── hashing — the single routine both sides use ──────────────────────────────
74
+
75
+ /** A file entry hashes the bytes read. */
76
+ export function hashFile(abs: string): string {
77
+ return createHash('sha256').update(readFileSync(abs)).digest('hex')
78
+ }
79
+
80
+ /**
81
+ * A directory entry hashes the NAMES the listing returned, never their contents — that is what
82
+ * makes a file later ADDED to the directory detectable without re-resolving the subject.
83
+ *
84
+ * The canonical form is load-bearing: entry names (not paths), sorted, joined by \n, no trailing
85
+ * newline. `readdirSync` order is filesystem-dependent, so an unsorted join would hash differently
86
+ * on two machines holding identical trees and report `stale` for nothing.
87
+ */
88
+ export function hashDirListing(abs: string): string {
89
+ return createHash('sha256')
90
+ .update(canonicalListing(readdirSync(abs)))
91
+ .digest('hex')
92
+ }
93
+
94
+ /**
95
+ * The canonical form of a listing: sorted, `\n`-joined, no trailing newline.
96
+ *
97
+ * Split out as a pure function so the ordering guarantee is testable WITHOUT a filesystem. It is not
98
+ * otherwise falsifiable here: ext4 returns `readdirSync` already sorted, so deleting the sort keeps
99
+ * the whole suite green on Linux while handing every APFS and NTFS user a permanent spurious `stale`
100
+ * — the divergence would surface as a wrong verdict, never as an error.
101
+ */
102
+ export function canonicalListing(names: string[]): string {
103
+ return [...names].sort().join('\n')
104
+ }
105
+
106
+ // ─── eval.md — resolve the target ─────────────────────────────────────────────
107
+
108
+ /**
109
+ * Pulls `subject:` out of an eval.md's frontmatter. Deliberately not a YAML parser: the one key
110
+ * that matters is a scalar path, and a dependency-free engine that reads one key cannot be broken
111
+ * by an unrelated key's syntax.
112
+ */
113
+ export function readSubject(evalText: string): string | null {
114
+ const fm = /^---\r?\n([\s\S]*?)\r?\n---/.exec(evalText)
115
+ const body = fm ? fm[1] : evalText
116
+ for (const line of body.split(/\r?\n/)) {
117
+ const m = /^subject:\s*(.+?)\s*$/.exec(line)
118
+ if (!m) continue
119
+ const raw = m[1].replace(/^["']|["']$/g, '').trim()
120
+ return raw === '' ? null : raw
121
+ }
122
+ return null
123
+ }
124
+
125
+ /** The node's own frozen suite — one member of the evaluated set, handled apart from the rest. */
126
+ export function findFrozenFeature(nodeDir: string): string | null {
127
+ const hit = readdirSync(nodeDir).find((n) => n.endsWith('.feature'))
128
+ return hit ? join(nodeDir, hit) : null
129
+ }
130
+
131
+ // ─── result selection ─────────────────────────────────────────────────────────
132
+
133
+ /** Every `*.json` under the results tree, at any depth. */
134
+ export function collectResultFiles(resultsDir: string): string[] {
135
+ const out: string[] = []
136
+ const walk = (dir: string) => {
137
+ for (const e of readdirSync(dir, { withFileTypes: true })) {
138
+ const p = join(dir, e.name)
139
+ if (e.isDirectory()) walk(p)
140
+ else if (e.name.endsWith('.json')) out.push(p)
141
+ }
142
+ }
143
+ walk(resultsDir)
144
+ return out.sort()
145
+ }
146
+
147
+ export interface Scan {
148
+ forTarget: LoadedResult[]
149
+ unreadable: string[]
150
+ }
151
+
152
+ /**
153
+ * Partitions results by the target each one RECORDS — never by the directory it sits in. A result
154
+ * filed under a mis-slugged directory still belongs to the target its own `target` field names, and
155
+ * an implementation matching on directory name would silently miss it.
156
+ *
157
+ * An unparseable file is collected rather than thrown on: one corrupt record must not blind the
158
+ * check to the readable ones beside it.
159
+ */
160
+ export function scanResults(files: string[], target: string, read: (f: string) => string): Scan {
161
+ const forTarget: LoadedResult[] = []
162
+ const unreadable: string[] = []
163
+ for (const file of files) {
164
+ let record: ResultRecord
165
+ try {
166
+ record = JSON.parse(read(file)) as ResultRecord
167
+ } catch {
168
+ unreadable.push(file)
169
+ continue
170
+ }
171
+ if (record && record.target === target) forTarget.push({ file, record })
172
+ }
173
+ return { forTarget, unreadable }
174
+ }
175
+
176
+ /**
177
+ * Newest by RECORDED timestamp, not by filename. Filenames sort lexicographically and a timestamp
178
+ * is the fact that matters; where the two disagree, trusting the name picks the wrong record.
179
+ */
180
+ export function selectNewest(results: LoadedResult[]): LoadedResult | null {
181
+ let best: LoadedResult | null = null
182
+ for (const r of results) {
183
+ const t = r.record.timestamp ?? ''
184
+ if (!best || t > (best.record.timestamp ?? '')) best = r
185
+ }
186
+ return best
187
+ }
188
+
189
+ // ─── coherence — the conditional oracle ───────────────────────────────────────
190
+
191
+ /**
192
+ * Taking the record's OWN `scenarios` and `target` as given, does its evaluated set cover the
193
+ * inputs they imply were read? Scores cannot come from a `.feature` never read, and a configuration
194
+ * cannot be judged unopened.
195
+ *
196
+ * This is a CONDITIONAL relation, never a proof: `scenarios` and `target` are written by the same
197
+ * agent and self-reported exactly as `evaluated` is, so a fabricated-but-self-consistent record
198
+ * passes. It catches the inconsistent under-reporter only.
199
+ */
200
+ export function incoherence(record: ResultRecord, featureRel: string, targetRel: string): string | null {
201
+ const paths = new Set((record.evaluated ?? []).map((e) => e.path))
202
+ if ((record.scenarios?.length ?? 0) > 0 && !paths.has(featureRel)) {
203
+ return `the recorded provenance contradicts the result it accompanies: it scores ${record.scenarios?.length} scenarios but carries no entry for ${featureRel}`
204
+ }
205
+ if (record.target && !paths.has(targetRel)) {
206
+ return `the recorded provenance omits the configuration the result names: no entry for ${targetRel}`
207
+ }
208
+ return null
209
+ }
210
+
211
+ // ─── the verdict ──────────────────────────────────────────────────────────────
212
+
213
+ /**
214
+ * Split the evaluated set into the frozen suite and the subject inputs, then compare.
215
+ *
216
+ * The ORDER of the two questions is the whole `incomplete` / `stale` distinction. A moved subject
217
+ * input means no individual score survives, so it wins outright. Only when every subject input is
218
+ * unmoved does a changed suite mean something narrower: the scores stand, the suite grew past them.
219
+ * Collapsing the two would throw away that surviving fact.
220
+ */
221
+ export function decideVerdict(
222
+ record: ResultRecord,
223
+ repoRoot: string,
224
+ featureRel: string,
225
+ skipped: string[] = [],
226
+ ): Decision {
227
+ const entries = record.evaluated ?? []
228
+ const subject = entries.filter((e) => e.path !== featureRel)
229
+ const suite = entries.filter((e) => e.path === featureRel)
230
+
231
+ const moved = (e: EvaluatedEntry): string | null => {
232
+ const abs = join(repoRoot, e.path)
233
+ if (!existsSync(abs)) return `${e.path} (missing from the tree)`
234
+ try {
235
+ const now = e.kind === 'directory' ? hashDirListing(abs) : hashFile(abs)
236
+ return now === e.sha256 ? null : `${e.path} (content changed)`
237
+ } catch {
238
+ return `${e.path} (unreadable)`
239
+ }
240
+ }
241
+
242
+ const subjectMoved = subject.map(moved).filter((m): m is string => m !== null)
243
+ if (subjectMoved.length > 0) {
244
+ return {
245
+ verdict: 'stale',
246
+ reason: 'an input the scores rest on has moved; no individual score can be relied on without re-running',
247
+ mismatched: subjectMoved,
248
+ skipped,
249
+ }
250
+ }
251
+
252
+ const suiteMoved = suite.map(moved).filter((m): m is string => m !== null)
253
+ if (suiteMoved.length > 0) {
254
+ return {
255
+ verdict: 'incomplete',
256
+ reason:
257
+ 'every input the scores rest on is unmoved, but the suite changed past them, so the result no longer covers the whole suite',
258
+ mismatched: suiteMoved,
259
+ skipped,
260
+ }
261
+ }
262
+
263
+ return {
264
+ verdict: 'current',
265
+ reason: "every input the run recorded still hashes as recorded — the run's account still holds",
266
+ mismatched: [],
267
+ skipped,
268
+ }
269
+ }
270
+
271
+ // ─── repo root ────────────────────────────────────────────────────────────────
272
+
273
+ export function findRepoRoot(from: string): string | null {
274
+ let dir = resolve(from)
275
+ for (;;) {
276
+ if (existsSync(join(dir, 'pnpm-workspace.yaml')) || existsSync(join(dir, '.git'))) return dir
277
+ const up = resolve(dir, '..')
278
+ if (up === dir) return null
279
+ dir = up
280
+ }
281
+ }
282
+
283
+ export const RESULTS_DIR = join('.agents', 'aced', 'results')
284
+
285
+ // ─── CLI ──────────────────────────────────────────────────────────────────────
286
+
287
+ function report(d: Decision): void {
288
+ process.stdout.write(`verdict: ${d.verdict}\n`)
289
+ process.stdout.write(`${d.reason}\n`)
290
+ for (const s of d.skipped) process.stdout.write(`skipped (unreadable): ${s}\n`)
291
+ for (const m of d.mismatched) process.stdout.write(`no longer matching: ${m}\n`)
292
+ }
293
+
294
+ export function main(argv: string[]): number {
295
+ // Shared-hashing entry points. `run` calls these rather than reimplementing the rule.
296
+ const hashFileIdx = argv.indexOf('--hash-file')
297
+ if (hashFileIdx !== -1) {
298
+ const p = argv[hashFileIdx + 1]
299
+ if (!p) {
300
+ console.error('✗ --hash-file needs a path')
301
+ return 1
302
+ }
303
+ process.stdout.write(hashFile(resolve(p)) + '\n')
304
+ return 0
305
+ }
306
+ const hashDirIdx = argv.indexOf('--hash-dir')
307
+ if (hashDirIdx !== -1) {
308
+ const p = argv[hashDirIdx + 1]
309
+ if (!p) {
310
+ console.error('✗ --hash-dir needs a path')
311
+ return 1
312
+ }
313
+ process.stdout.write(hashDirListing(resolve(p)) + '\n')
314
+ return 0
315
+ }
316
+
317
+ const nodeIdx = argv.indexOf('--node')
318
+ const nodeArg = nodeIdx === -1 ? undefined : argv[nodeIdx + 1]
319
+ if (!nodeArg) {
320
+ console.error('✗ usage: check-freshness --node <node-dir>')
321
+ return 1
322
+ }
323
+ const nodeDir = resolve(nodeArg)
324
+
325
+ // Fail closed: no eval.md and no subject key each emit NO verdict. A guard that cannot decide
326
+ // must not manufacture one — `absent` would read as "checked, nothing recorded".
327
+ const evalPath = join(nodeDir, 'eval.md')
328
+ if (!existsSync(evalPath)) {
329
+ console.error(`✗ ${relative(process.cwd(), evalPath)}: no eval.md in the node directory; no verdict`)
330
+ return 1
331
+ }
332
+ const subject = readSubject(readFileSync(evalPath, 'utf8'))
333
+ if (!subject) {
334
+ console.error(`✗ ${relative(process.cwd(), evalPath)}: eval.md carries no subject key; no verdict`)
335
+ return 1
336
+ }
337
+
338
+ const repoRoot = findRepoRoot(nodeDir)
339
+ if (!repoRoot) {
340
+ console.error('✗ no repository root above the node directory; no verdict')
341
+ return 1
342
+ }
343
+
344
+ process.stdout.write(`target: ${subject}\n`)
345
+
346
+ const featureAbs = findFrozenFeature(nodeDir)
347
+ const featureRel = featureAbs ? relative(repoRoot, featureAbs) : ''
348
+
349
+ const resultsDir = join(repoRoot, RESULTS_DIR)
350
+ if (!existsSync(resultsDir)) {
351
+ report({ verdict: 'absent', reason: 'no result is recorded anywhere', mismatched: [], skipped: [] })
352
+ return 1
353
+ }
354
+
355
+ const scan = scanResults(collectResultFiles(resultsDir), subject, (f) => readFileSync(f, 'utf8'))
356
+ const skipped = scan.unreadable.map((f) => relative(repoRoot, f))
357
+
358
+ if (scan.forTarget.length === 0) {
359
+ report({
360
+ verdict: 'absent',
361
+ reason:
362
+ skipped.length > 0
363
+ ? 'every result recorded for this target is unreadable'
364
+ : 'no result is recorded for this target',
365
+ mismatched: [],
366
+ skipped,
367
+ })
368
+ return 1
369
+ }
370
+
371
+ const newest = selectNewest(scan.forTarget)!
372
+ if (!newest.record.evaluated) {
373
+ report({
374
+ verdict: 'absent',
375
+ reason: 'the result carries no recorded provenance',
376
+ mismatched: [],
377
+ skipped,
378
+ })
379
+ return 1
380
+ }
381
+
382
+ const bad = incoherence(newest.record, featureRel, subject)
383
+ if (bad) {
384
+ report({ verdict: 'absent', reason: bad, mismatched: [], skipped })
385
+ return 1
386
+ }
387
+
388
+ const decision = decideVerdict(newest.record, repoRoot, featureRel, skipped)
389
+ report(decision)
390
+ return decision.verdict === 'current' ? 0 : 1
391
+ }
392
+
393
+ // `import.meta.main` is Node >=24.2 and `undefined` on this repo's >=22 floor, where the CLI would
394
+ // never run — printing nothing and exiting 0, which a caller keying on the exit code would read as
395
+ // `current`. `pathToFileURL` rather than a `file://` concat: `import.meta.url` percent-encodes, so
396
+ // the naive form mismatches on any install path holding a space and reproduces the same bug.
397
+ if (process.argv[1] && import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href) {
398
+ process.exit(main(process.argv.slice(2)))
399
+ }
@@ -0,0 +1,25 @@
1
+ # compare
2
+
3
+ Run the frozen `.feature` suite against two versions of a target agent configuration and diff the results — the regression check before committing a change.
4
+
5
+ ## When to use
6
+
7
+ Use this skill when you've edited a skill, subagent, command, or AGENTS.md section and want to confirm the edit actually helped before committing it.
8
+
9
+ Good triggers include:
10
+
11
+ - "Did this change actually improve things?"
12
+ - "Check for regressions before I commit"
13
+ - Verifying an `improve`-proposed edit worked as intended
14
+
15
+ Defer to a sibling when you just want a single version's score (`run`), or you're diagnosing why a version fails rather than diffing two of them (`improve`).
16
+
17
+ ## What it does
18
+
19
+ Scores the frozen `.feature` suite against a "before" version (default: previous git revision) and an "after" version (default: current working tree), computes per-scenario and aggregate deltas, and reports improved / regressed / unchanged scenarios with an explicit regression warning if any scenario got worse.
20
+
21
+ ## Install
22
+
23
+ ```bash
24
+ npx skills add cyberuni/cyberplace --skill aced/compare
25
+ ```
@@ -0,0 +1,87 @@
1
+ ---
2
+ name: compare
3
+ description: Use this skill when comparing two versions of an agent configuration to detect regressions or confirm improvements before committing a change.
4
+ ---
5
+
6
+ # ACED Compare
7
+
8
+ Run the frozen `.feature` suite against two versions of a target agent configuration and diff the results.
9
+
10
+ ## Identify the two versions
11
+
12
+ Default: current working tree vs. previous git revision.
13
+
14
+ ```bash
15
+ git show HEAD:<relative-path-to-agent-config> > /tmp/aced-compare-before.md
16
+ ```
17
+
18
+ The user may also provide:
19
+ - Two explicit file paths
20
+ - A git ref (branch, commit SHA, tag) for the "before" version
21
+
22
+ Read both versions in full before proceeding. **If the "before" version cannot be read (an unresolvable git ref or a missing path), report that the version cannot be resolved and stop — score neither side.**
23
+
24
+ ## Run evals on both versions
25
+
26
+ For each version, run every scenario in the node's frozen `<node>.feature` in the project spec through `aced-case-judge` (same process as `run`) — passing the `.feature` path and the scenario name, never the scenario body. The judge blinds its own simulating context; handing it the `Then` or the rubric would defeat that.
27
+
28
+ Label results as **before** and **after**.
29
+
30
+ Do not write to `results/` during compare — this is a diff operation, not a recorded run. Only write if the user explicitly asks to record.
31
+
32
+ When you do write one, the record carries `scoring_model` on the same terms `run`'s does: the model
33
+ you dispatched `aced-case-judge` under, or `unknown` when you cannot name it. One value covers the
34
+ whole comparison — a diff scores both sides in a single invocation, so one model scored both — and
35
+ `unknown` is a recorded value, never an omitted field.
36
+
37
+ ## Compute the diff
38
+
39
+ For each scenario, record:
40
+ - Before and after **per dimension** (`score`/`max`), plus each side's total against the same maximum
41
+ - Before pass/fail, after pass/fail
42
+ - Change type: `improved` | `regressed` | `unchanged` | `now-passing` | `now-failing`
43
+
44
+ Both sides are scored against the **same frozen scenario**, so their maxima match and their totals
45
+ are comparable to each other. Totals are **not** comparable across different scenarios — maxima
46
+ differ per scenario, so never average raw totals into a headline number. Aggregate instead:
47
+ - Net pass rate delta (e.g., +3 scenarios passing)
48
+ - Count by change type
49
+ - The per-dimension deltas, which are what say *where* the change landed
50
+
51
+ A dimension delta is the useful signal: a total that holds steady while one dimension drops and
52
+ another rises is a real change the total hides.
53
+
54
+ ## Report
55
+
56
+ ```
57
+ ACED Compare — <name>
58
+ ──────────────────────────────
59
+ Before: 18/22 passing (82%)
60
+ After: 21/22 passing (95%)
61
+
62
+ Net change: +3 passing
63
+
64
+ IMPROVED (now passing, or a dimension gained):
65
+ ✓ no trigger for an audit request invoked no → yes (expected yes) now passing
66
+ ✓ stages only related files 3/5 → 4/5 correctness 2/3 → 3/3 now passing
67
+ ✓ red tests block the commit 3/5 → 5/5 correctness 1/3 → 3/3, completeness 2/2 → 2/2 now passing
68
+
69
+ REGRESSED (now failing, or a dimension lost):
70
+ none
71
+
72
+ UNCHANGED: 18 scenarios
73
+ ```
74
+
75
+ ## Regression gate
76
+
77
+ If any scenario regressed (a dimension dropped, the total dropped, or it flipped from pass to fail), warn explicitly:
78
+
79
+ ```
80
+ ⚠ REGRESSION DETECTED
81
+ ✗ trigger on skill creation invoked no → yes (expected no) (was passing, now failing)
82
+
83
+ Do not commit this change until the regression is resolved.
84
+ Run improve to address it, or revert and try a different edit.
85
+ ```
86
+
87
+ If no regressions and net improvement: confirm the change is safe to commit.
@@ -0,0 +1,25 @@
1
+ # contribute-skill
2
+
3
+ Contribute local improvements to an installed skill back to its source repository via pull request.
4
+
5
+ ## When to use
6
+
7
+ Use this skill when you improved a skill installed from another repo and want to send the change upstream.
8
+
9
+ Good triggers include:
10
+
11
+ - "Patch this skill upstream"
12
+ - "Contribute my skill fix back to the source repo"
13
+ - After editing a skill under `~/.agents/skills/` or `.agents/skills/`
14
+
15
+ Do not use for skills native to the current repo under `skills/<name>/` — that repo is already the source.
16
+
17
+ ## What it does
18
+
19
+ The skill maps local changes to `skills/<skill-name>/` in the upstream repo, opens a branch, and creates a PR. It never includes `SKILL.local.md` or paths outside the canonical skill folder.
20
+
21
+ ## Install
22
+
23
+ ```bash
24
+ npx skills add cyberuni/cyberplace --skill contribute-skill
25
+ ```