@miroshnik/elves 0.0.0 → 2026.1018.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +310 -0
- package/README.md +344 -0
- package/bin/ai-dev.ts +7 -0
- package/bin/elves.ts +2847 -0
- package/bin/shims/timeout +113 -0
- package/claude/CLAUDE.md +71 -0
- package/dist/github/scripts/github.js +3428 -0
- package/dist/spec/scripts/architecture.js +628 -0
- package/dist/spec/scripts/eslint-config.js +54 -0
- package/dist/spec/scripts/harness-qzb932c8.js +959 -0
- package/dist/spec/scripts/harness-v9s1v9w3.js +16045 -0
- package/dist/spec/scripts/harness.js +53 -0
- package/dist/spec/scripts/knip-hints.cjs +34 -0
- package/docs/ci-concurrency.md +82 -0
- package/docs/cloud-sessions.md +203 -0
- package/docs/judgment.md +81 -0
- package/docs/parallel-checkouts.md +220 -0
- package/docs/pr-checks.md +117 -0
- package/docs/testing.md +304 -0
- package/package.json +70 -1
- package/skills/ci-wait/SKILL.md +180 -0
- package/skills/ci-wait/scripts/wait-ci.sh +377 -0
- package/skills/dashboard/SKILL.md +212 -0
- package/skills/dashboard/scripts/dashboard.ts +912 -0
- package/skills/dashboard/scripts/sessions.ts +772 -0
- package/skills/est/SKILL.md +221 -0
- package/skills/est/reference.md +381 -0
- package/skills/est/scripts/est.ts +4854 -0
- package/skills/github/SKILL.md +206 -0
- package/skills/github/locales/en.json +449 -0
- package/skills/github/locales/ru.json +449 -0
- package/skills/github/reference.md +1073 -0
- package/skills/github/scripts/github.ts +4390 -0
- package/skills/github/scripts/locale.ts +495 -0
- package/skills/github/scripts/package.json +3 -0
- package/skills/slot/SKILL.md +145 -0
- package/skills/slot/scripts/package.json +3 -0
- package/skills/slot/scripts/slot.ts +279 -0
- package/skills/spec/SKILL.md +247 -0
- package/skills/spec/canon.md +219 -0
- package/skills/spec/checks.md +709 -0
- package/skills/spec/ci.md +378 -0
- package/skills/spec/migration.md +125 -0
- package/skills/spec/reference.md +889 -0
- package/skills/spec/scripts/architecture.ts +839 -0
- package/skills/spec/scripts/c4.ts +184 -0
- package/skills/spec/scripts/eslint-config.ts +82 -0
- package/skills/spec/scripts/harness.ts +1483 -0
- package/skills/spec/scripts/knip-hints.cjs +34 -0
- package/skills/spec/scripts/knip-hints.d.cts +17 -0
- package/skills/spec/scripts/package-run.ts +22 -0
- package/skills/spec/scripts/package.json +3 -0
- package/skills/spec/scripts/spec-break.ts +232 -0
- package/skills/spec/scripts/spec-claims.ts +198 -0
- package/skills/spec/scripts/spec-diff.ts +1111 -0
- package/skills/spec/scripts/spec-doc.ts +744 -0
- package/skills/spec/scripts/spec-exceptions.ts +233 -0
- package/skills/spec/scripts/spec-publish.ts +214 -0
- package/skills/spec/scripts/spec-run.ts +235 -0
- package/skills/spec/scripts/spec-shard.ts +267 -0
- package/skills/spec/scripts/speclib.ts +826 -0
- package/skills/spec/scripts/vendor/babel-parser.LICENSE +19 -0
- package/skills/spec/scripts/vendor/babel-parser.cjs +14615 -0
- package/skills/spec/scripts/vendor/babel-parser.d.cts +3 -0
- package/skills/task-env/SKILL.md +231 -0
- package/skills/task-env/proxy/compose.yaml +51 -0
- package/skills/task-env/reference.md +254 -0
- package/skills/task-env/scripts/task-env.ts +1017 -0
|
@@ -0,0 +1,279 @@
|
|
|
1
|
+
#!/usr/bin/env bun
|
|
2
|
+
/**
|
|
3
|
+
* slot — a queue for heavy runs on the machine (the slot skill): a command goes when one of SLOTS slots is free, the rest
|
|
4
|
+
* wait in arrival order and print whom they wait for. Parallel agent sessions ran the full check at once, each runner on
|
|
5
|
+
* all cores, and the runs hit timeouts (tests/capabilities/slot).
|
|
6
|
+
*
|
|
7
|
+
* A slot is a ticket file `<time>-<pid>` in `slots/` of the personal configuration directory: the name sets the order, the
|
|
8
|
+
* PID tells whether the owner is alive. A dead process, even one killed without cleanup, has its ticket ignored and removed
|
|
9
|
+
* by the first one to see it: there are no locks to remove by hand. The command gets ELVES_SLOT_CPUS: all slots together
|
|
10
|
+
* take half of the cores.
|
|
11
|
+
*
|
|
12
|
+
* An agent's sandbox doesn't let the configuration be written: the ticket goes to `elves/slots` of the temp directory
|
|
13
|
+
* (Claude Code's sandbox gives all sessions of the user one TMPDIR), and the queue is read from all places at once — runs
|
|
14
|
+
* inside and outside the sandbox wait for each other. No place takes the ticket — one warning line and the command runs
|
|
15
|
+
* without the queue.
|
|
16
|
+
*
|
|
17
|
+
* No queue in CI (the machine is given to the run) and inside a slot (ELVES_SLOT): a script that calls another script
|
|
18
|
+
* through the queue would otherwise wait for itself.
|
|
19
|
+
*
|
|
20
|
+
* The flow's former name (#545): a project whose copy `update` hasn't brought yet runs the slot of an earlier release —
|
|
21
|
+
* it queues in its own place of the temp directory (FORMER_TEMP), exports only the queue variables under the former name
|
|
22
|
+
* (FORMER_ENV), and its runner configs and `turbo.json` pass those. So the queue is read from the former temp place too,
|
|
23
|
+
* the queue variables go under both names, and either name marks a nested call.
|
|
24
|
+
*
|
|
25
|
+
* A package.json script takes its slot with `--for <shell PID>` (the project's shim `.agents/slot.sh` runs `elves slot
|
|
26
|
+
* --for $$`): the ticket carries the script's shell, not this process, and the queue variables are printed for the shell
|
|
27
|
+
* to export — the command runs as a plain command of the script, visible to tools that read package.json (knip).
|
|
28
|
+
*
|
|
29
|
+
* Runs under Bun (`bun slot.ts`), only node: APIs. A project without a copy of the flow runs it from its package as
|
|
30
|
+
* `elves slot` (bin/elves.ts imports `main` and `HELP`): the queue starts only when this file is the entry point.
|
|
31
|
+
*/
|
|
32
|
+
import { spawn } from "node:child_process";
|
|
33
|
+
import { existsSync, mkdirSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
|
34
|
+
import os from "node:os";
|
|
35
|
+
import path from "node:path";
|
|
36
|
+
|
|
37
|
+
/**
|
|
38
|
+
* `--help`, `-h` as the first argument — the rest go to the command (tests/standards/cli-help): synopsis, what, examples;
|
|
39
|
+
* `elves slot` prints it under its name.
|
|
40
|
+
*/
|
|
41
|
+
export const HELP = `slot '<command>' [arguments…]
|
|
42
|
+
slot --for <PID>
|
|
43
|
+
|
|
44
|
+
Runs the command through sh, the arguments appended to its end, as with npm run; waits while two heavy runs are going on
|
|
45
|
+
the machine, then runs with ELVES_SLOT_CPUS — a quarter of the cores. --for — takes a slot for the shell <PID> of a
|
|
46
|
+
package.json script, held until the shell exits, and prints the export of ELVES_SLOT, ELVES_SLOT_CPUS for it — the
|
|
47
|
+
project's shim runs it: ". .agents/slot.sh && <command>". No queue in CI and inside a slot. The exit code is the
|
|
48
|
+
command's; with --for — 0, nothing printed without the queue.
|
|
49
|
+
|
|
50
|
+
Examples:
|
|
51
|
+
slot 'vitest run'
|
|
52
|
+
slot 'bun test' tests/standards
|
|
53
|
+
`;
|
|
54
|
+
|
|
55
|
+
/** Heavy runs on the machine at once. */
|
|
56
|
+
const SLOTS = 2;
|
|
57
|
+
/** Runner workers per slot: all slots together get half of the cores, the other half is for the human, the app and the agents. */
|
|
58
|
+
const CPUS = Math.max(1, Math.floor(os.availableParallelism() / 2 / SLOTS));
|
|
59
|
+
/** How often a waiting process looks at the queue, ms: a directory of a few files is read in microseconds. */
|
|
60
|
+
const POLL = 250;
|
|
61
|
+
/**
|
|
62
|
+
* Pause before confirming "I am among the first SLOTS", ms: the ticket time is taken before the file is written, and a
|
|
63
|
+
* ticket with an earlier time may not have reached the disk yet when the queue was read.
|
|
64
|
+
*/
|
|
65
|
+
const SETTLE = 50;
|
|
66
|
+
/** A waiting process reminds whom it waits for once per this many ms, even if the queue hasn't moved. */
|
|
67
|
+
const REMIND = 60_000;
|
|
68
|
+
const SIGNALS = ["SIGINT", "SIGTERM", "SIGHUP"] as const;
|
|
69
|
+
/**
|
|
70
|
+
* The queue variables under the flow's former name, exported next to ours for one transition: runner configs and
|
|
71
|
+
* `turbo.json` of projects read and pass `AI_DEV_SLOT_CPUS` and `AI_DEV_SLOT` until they are renamed, and the slot of an
|
|
72
|
+
* earlier release marks a nested call with `AI_DEV_SLOT` only.
|
|
73
|
+
*/
|
|
74
|
+
const FORMER_ENV = { ELVES_SLOT: "AI_DEV_SLOT", ELVES_SLOT_CPUS: "AI_DEV_SLOT_CPUS" } as const;
|
|
75
|
+
/** The temp directory's place of the slot of an earlier release (FORMER_ENV): its tickets are read, ours aren't written there. */
|
|
76
|
+
const FORMER_TEMP = "ai-dev";
|
|
77
|
+
|
|
78
|
+
interface Ticket {
|
|
79
|
+
name: string;
|
|
80
|
+
cwd?: string;
|
|
81
|
+
cmd?: string;
|
|
82
|
+
since?: number;
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
/** An argument for sh: a plain word as is, anything else in single quotes. */
|
|
86
|
+
const quote = (s: string) => (/^[\w@%+=:,./-]+$/.test(s) ? s : `'${s.replaceAll("'", `'\\''`)}'`);
|
|
87
|
+
|
|
88
|
+
/** Whether the process is alive: signal 0 only checks; EPERM — alive, but someone else's. */
|
|
89
|
+
function alive(pid: number) {
|
|
90
|
+
try {
|
|
91
|
+
process.kill(pid, 0);
|
|
92
|
+
return true;
|
|
93
|
+
} catch (e) {
|
|
94
|
+
return (e as NodeJS.ErrnoException).code === "EPERM";
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
/**
|
|
99
|
+
* Where tickets live, in the order a ticket is written: the personal configuration, then the temp directory — an agent's
|
|
100
|
+
* sandbox doesn't let the configuration be written, while its TMPDIR is shared by the sandbox's sessions; Claude Code
|
|
101
|
+
* passes its sandbox's temp directory as CLAUDE_TMPDIR outside the sandbox too, so runs outside wait for runs inside.
|
|
102
|
+
* The personal configuration directory — the same expression in all scripts (tests/standards/config-dir).
|
|
103
|
+
*/
|
|
104
|
+
function places() {
|
|
105
|
+
const config = process.env.ELVES_CONFIG_DIR || process.env.AI_DEV_CONFIG_DIR || [path.join(os.homedir(), ".config", "elves"), path.join(os.homedir(), ".config", "ai-dev")].find((d) => existsSync(d)) || path.join(os.homedir(), ".config", "elves");
|
|
106
|
+
const temps = [os.tmpdir(), process.env.CLAUDE_TMPDIR].filter((d): d is string => !!d);
|
|
107
|
+
return [...new Set([config, ...temps.map((d) => path.join(d, "elves"))].map((d) => path.join(path.resolve(d), "slots")))];
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
/** Places the queue is read from: ours and the temp places of the slot of an earlier release (FORMER_TEMP). */
|
|
111
|
+
function readPlaces(dirs: string[]) {
|
|
112
|
+
const former = [os.tmpdir(), process.env.CLAUDE_TMPDIR].filter((d): d is string => !!d).map((d) => path.join(path.resolve(d), FORMER_TEMP, "slots"));
|
|
113
|
+
return [...new Set([...dirs, ...former])];
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
/** Live tickets of all places in arrival order; tickets of dead processes are removed where the directory can be written. */
|
|
117
|
+
function queue(dirs: string[]): Ticket[] {
|
|
118
|
+
const out: Ticket[] = [];
|
|
119
|
+
for (const dir of dirs) {
|
|
120
|
+
let names: string[];
|
|
121
|
+
try {
|
|
122
|
+
names = readdirSync(dir);
|
|
123
|
+
} catch {
|
|
124
|
+
continue; // no one has queued here yet
|
|
125
|
+
}
|
|
126
|
+
for (const name of names) {
|
|
127
|
+
const pid = Number(/^\d+-(\d+)$/.exec(name)?.[1]);
|
|
128
|
+
if (!pid) continue;
|
|
129
|
+
if (!alive(pid)) {
|
|
130
|
+
try {
|
|
131
|
+
rmSync(path.join(dir, name), { force: true });
|
|
132
|
+
} catch {
|
|
133
|
+
/* a sandbox can't write here: the ticket just doesn't count, a run outside the sandbox removes it */
|
|
134
|
+
}
|
|
135
|
+
continue;
|
|
136
|
+
}
|
|
137
|
+
let info = {};
|
|
138
|
+
try {
|
|
139
|
+
info = JSON.parse(readFileSync(path.join(dir, name), "utf8"));
|
|
140
|
+
} catch {
|
|
141
|
+
/* the ticket was just created and is still empty */
|
|
142
|
+
}
|
|
143
|
+
out.push({ ...info, name });
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
return out.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
const duration = (ms: number) => (ms < 60_000 ? `${Math.round(ms / 1000)} s` : `${Math.floor(ms / 60_000)} min ${Math.round((ms % 60_000) / 1000)} s`);
|
|
150
|
+
|
|
151
|
+
/** A path under the home directory — from `~`. */
|
|
152
|
+
function tilde(p: string) {
|
|
153
|
+
const home = os.homedir();
|
|
154
|
+
return p.startsWith(home + path.sep) ? `~${p.slice(home.length)}` : p;
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
/** Who holds the slot: the directory, the command and how long it has been running. */
|
|
158
|
+
function holder(t: Ticket) {
|
|
159
|
+
const cwd = t.cwd ? tilde(t.cwd) : "?";
|
|
160
|
+
const cmd = (t.cmd ?? "?").replace(/\s+/g, " ");
|
|
161
|
+
return `${cwd}: ${cmd.length > 80 ? cmd.slice(0, 79) + "…" : cmd}${t.since ? ` (${duration(Date.now() - t.since)})` : ""}`;
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
/**
|
|
165
|
+
* The command of the package.json script that sourced the shim, for whoever waits for its slot: npm and bun give the
|
|
166
|
+
* script's text in npm_lifecycle_script — without the shim's prefix (`. .agents/slot.sh &&`, a copy's slot.sh of an
|
|
167
|
+
* earlier install) it is the command; otherwise the script's name.
|
|
168
|
+
*/
|
|
169
|
+
function scriptCommand() {
|
|
170
|
+
const text = process.env.npm_lifecycle_script;
|
|
171
|
+
return text ? text.replace(/^\.\s+\S*slot\.sh\s*&&\s*/, "") : process.env.npm_lifecycle_event;
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
/** The command through sh with the environment on top of our own; signals go to it. Exit code — its code, killed by a signal — 128 + number. */
|
|
175
|
+
function run(command: string, env: Record<string, string>) {
|
|
176
|
+
return new Promise<number>((resolve) => {
|
|
177
|
+
const child = spawn(command, { shell: true, stdio: "inherit", env: { ...process.env, ...env } });
|
|
178
|
+
for (const s of SIGNALS) process.on(s, () => child.kill(s));
|
|
179
|
+
child.on("error", (e) => {
|
|
180
|
+
process.stderr.write(`slot: ${e.message}\n`);
|
|
181
|
+
resolve(127);
|
|
182
|
+
});
|
|
183
|
+
child.on("exit", (code, signal) => resolve(code ?? 128 + (signal ? os.constants.signals[signal] : 0)));
|
|
184
|
+
});
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
export async function main(argv: string[]): Promise<number> {
|
|
188
|
+
const [first, ...rest] = argv;
|
|
189
|
+
if (first === "-h" || first === "--help") return process.stdout.write(HELP), 0;
|
|
190
|
+
if (!first) return process.stderr.write(HELP), 2;
|
|
191
|
+
// --for: the slot is the shell's — it holds the ticket, gets the variables and runs the command itself
|
|
192
|
+
const shell = first === "--for" ? Number(rest[0]) : 0;
|
|
193
|
+
if (first === "--for" && !(rest.length === 1 && Number.isInteger(shell) && shell > 0 && alive(shell))) {
|
|
194
|
+
return process.stderr.write(`slot: --for takes the PID of a running shell, got ${rest.join(" ") || "nothing"}\n${HELP}`), 2;
|
|
195
|
+
}
|
|
196
|
+
const command = [first, ...rest.map(quote)].join(" ");
|
|
197
|
+
// without the queue the command goes at once; a script's shell has nothing to export
|
|
198
|
+
const free = () => (shell ? 0 : run(command, {}));
|
|
199
|
+
const ci = process.env.CI && !/^(0|false)$/i.test(process.env.CI);
|
|
200
|
+
if (ci || process.env.ELVES_SLOT || process.env[FORMER_ENV.ELVES_SLOT]) return free();
|
|
201
|
+
|
|
202
|
+
const dirs = places();
|
|
203
|
+
const read = readPlaces(dirs);
|
|
204
|
+
const name = `${String(Date.now()).padStart(15, "0")}-${shell || process.pid}`;
|
|
205
|
+
let file = "";
|
|
206
|
+
const enqueue = () => {
|
|
207
|
+
mkdirSync(path.dirname(file), { recursive: true });
|
|
208
|
+
writeFileSync(file, JSON.stringify({ cwd: process.cwd(), cmd: shell ? scriptCommand() : command, since: Date.now() }));
|
|
209
|
+
};
|
|
210
|
+
const refused: string[] = [];
|
|
211
|
+
for (const dir of dirs) {
|
|
212
|
+
file = path.join(dir, name);
|
|
213
|
+
try {
|
|
214
|
+
enqueue();
|
|
215
|
+
break;
|
|
216
|
+
} catch (e) {
|
|
217
|
+
refused.push(`${tilde(dir)} (${(e as NodeJS.ErrnoException).code ?? (e as Error).message})`);
|
|
218
|
+
file = "";
|
|
219
|
+
}
|
|
220
|
+
}
|
|
221
|
+
if (!file) {
|
|
222
|
+
process.stderr.write(`slot: running without the queue — no place takes the ticket: ${refused.join(", ")}\n`);
|
|
223
|
+
return free();
|
|
224
|
+
}
|
|
225
|
+
// the shell's ticket stays when we exit with its slot: the queue removes it when the shell is gone
|
|
226
|
+
let held = false;
|
|
227
|
+
const drop = () => {
|
|
228
|
+
if (!held) rmSync(file, { force: true });
|
|
229
|
+
};
|
|
230
|
+
process.on("exit", drop);
|
|
231
|
+
// while waiting, a signal removes the ticket and exits; in a slot, signals go to the command (run)
|
|
232
|
+
const quit = (s: NodeJS.Signals) => (drop(), process.exit(128 + os.constants.signals[s]));
|
|
233
|
+
for (const s of SIGNALS) process.on(s, quit);
|
|
234
|
+
|
|
235
|
+
const t0 = Date.now();
|
|
236
|
+
let confirmed = false;
|
|
237
|
+
let told = "";
|
|
238
|
+
let toldAt = 0;
|
|
239
|
+
for (;;) {
|
|
240
|
+
// the shell is gone while waiting (killed) — its ticket would be put back forever
|
|
241
|
+
if (shell && !alive(shell)) return 1;
|
|
242
|
+
let q = queue(read);
|
|
243
|
+
let i = q.findIndex((t) => t.name === name);
|
|
244
|
+
if (i < 0) {
|
|
245
|
+
enqueue(); // the ticket was removed from outside (the directory cleaned by hand) — the same place in the queue
|
|
246
|
+
q = queue(read);
|
|
247
|
+
i = q.findIndex((t) => t.name === name);
|
|
248
|
+
}
|
|
249
|
+
if (i >= 0 && i < SLOTS) {
|
|
250
|
+
if (confirmed) break;
|
|
251
|
+
confirmed = true;
|
|
252
|
+
await new Promise((r) => setTimeout(r, SETTLE));
|
|
253
|
+
continue;
|
|
254
|
+
}
|
|
255
|
+
confirmed = false;
|
|
256
|
+
const holders = q.slice(0, SLOTS);
|
|
257
|
+
const who = holders.map((t) => t.name).join(" ");
|
|
258
|
+
if (who !== told || Date.now() - toldAt >= REMIND) {
|
|
259
|
+
const ahead = i - SLOTS;
|
|
260
|
+
process.stderr.write(`slot: waiting for the machine queue — running ${holders.map(holder).join("; ")}${ahead ? `; ${ahead} more ahead` : ""}\n`);
|
|
261
|
+
[told, toldAt] = [who, Date.now()];
|
|
262
|
+
}
|
|
263
|
+
await new Promise((r) => setTimeout(r, POLL));
|
|
264
|
+
}
|
|
265
|
+
for (const s of SIGNALS) process.off(s, quit);
|
|
266
|
+
if (told) process.stderr.write(`slot: waited ${duration(Date.now() - t0)}\n`);
|
|
267
|
+
const own = { ELVES_SLOT: name, ELVES_SLOT_CPUS: String(CPUS) };
|
|
268
|
+
const env = { ...own, ...Object.fromEntries(Object.entries(own).map(([k, v]) => [FORMER_ENV[k as keyof typeof own], v])) };
|
|
269
|
+
if (shell) {
|
|
270
|
+
held = true;
|
|
271
|
+
process.stdout.write(`export ${Object.entries(env).map(([k, v]) => `${k}=${v}`).join(" ")}\n`);
|
|
272
|
+
return 0;
|
|
273
|
+
}
|
|
274
|
+
const code = await run(command, env);
|
|
275
|
+
drop();
|
|
276
|
+
return code;
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
if (import.meta.main) process.exit(await main(process.argv.slice(2)));
|
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec
|
|
3
|
+
description: Specification from tests — the spec canon (canon.md), deterministic scripts and a checks harness. spec-doc builds docs/spec from runner reports (Vitest/Jest, Playwright, bun JUnit) along tests/capabilities, tests/architecture and tests/standards, with prose from JSDoc; spec-diff prints removed, changed and added test names for the PR body; spec-publish publishes docs/spec to the spec branch after a merge, spec-run finds the green run for it; spec-shard splits CI shards by the last green run's durations; spec-claims checks entry points against the call log; spec-break proves a check can fail; spec-exceptions moves exceptions.ts into exceptions/; harness.ts — mechanical checks (registry + invariant). When — before creating a PR; in CI on a PR and after a merge; reading a project's spec; wiring spec into a project's CI; a convention must hold for every item, not an example; proving a test catches a breakage; asked what the system does or which requirements a PR removes — even if "specification" wasn't said.
|
|
4
|
+
allowed-tools: Bash(bun *skills/spec/scripts/spec-doc.ts *) Bash(bun *skills/spec/scripts/spec-diff.ts *) Bash(bun run spec:*) Bash(bun *skills/spec/scripts/spec-claims.ts *) Bash(bun *skills/spec/scripts/spec-break.ts *) Bash(bun *skills/spec/scripts/spec-exceptions.ts *) Bash(pnpm spec:*) Bash(bunx vitest run *) Bash(git status *) Bash(git diff *)
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# spec — documentation from test names, spec diff in the PR
|
|
8
|
+
|
|
9
|
+
The "Specification — decisions" rule is `canon.md` next to this file (the spec
|
|
10
|
+
canon moved out of the `AGENTS.md` core; read it when you write or migrate a
|
|
11
|
+
test, add a check, an exception, a standard): a decision exists while a
|
|
12
|
+
mechanism checks it — a test, a lint rule, a check over the architecture
|
|
13
|
+
model; each is wrapped in a runner test. So the documentation is the `tests/`
|
|
14
|
+
tree and the test names, and the diff of tests in a PR is the spec diff. The
|
|
15
|
+
scripts in `scripts/` do this deterministically, without a model and without
|
|
16
|
+
per-repository configuration:
|
|
17
|
+
|
|
18
|
+
- `spec-doc.ts` — runner reports and test JSDoc → `docs/spec/` (or a single
|
|
19
|
+
document to stdout);
|
|
20
|
+
- `spec-diff.ts` — test names on the base branch and in HEAD → three lists for
|
|
21
|
+
the PR body: removed, changed, added;
|
|
22
|
+
- `spec-publish.ts` — the built `docs/spec` → the `spec` branch (CI after a
|
|
23
|
+
merge: a run on `main` or the PR run's artifact); `main` has no
|
|
24
|
+
documentation;
|
|
25
|
+
- `spec-run.ts` — for publishing on a PR merge: the green run that checked
|
|
26
|
+
exactly the tree of the merge commit; none — publishing is skipped;
|
|
27
|
+
`--artifact` — the last green run's reports for `spec-shard`, one for all
|
|
28
|
+
shards of a run (green by its start);
|
|
29
|
+
- `spec-shard.ts` — a CI shard's test files by the last green run's
|
|
30
|
+
durations, not by the count of tests: greedily, the longest into the
|
|
31
|
+
lightest shard; the runner gets them as filters instead of `--shard`
|
|
32
|
+
(`ci.md`, "Shards by durations");
|
|
33
|
+
- `spec-claims.ts` — after a run: every entry point is called by a capability
|
|
34
|
+
test (the harness's `journal`, `resetJournal` at the start of the run),
|
|
35
|
+
registry ids don't repeat, journal ids come from the registry; the JUnit
|
|
36
|
+
report goes to `spec-doc`;
|
|
37
|
+
- `spec-break.ts` — check by breaking: edit the code, run the test, always
|
|
38
|
+
revert (even after an interruption); ✅ failed / ❌ didn't fail for each
|
|
39
|
+
breakage;
|
|
40
|
+
- `spec-exceptions.ts` — moves the decision folders' `exceptions.ts` into an
|
|
41
|
+
`exceptions/` directory, one file per exception (section "spec-exceptions");
|
|
42
|
+
- `harness.ts` — a library for the project's tests: a mechanical check
|
|
43
|
+
registers ordinary runner tests and gets into the spec (section "Checks").
|
|
44
|
+
|
|
45
|
+
Running: `bun --no-env-file <skill dir>/scripts/spec-doc.ts …` — TypeScript
|
|
46
|
+
without a build and without dependencies. Syntax, flags and exit codes — each
|
|
47
|
+
script's `--help`; what a script does in detail and why — `reference.md` next
|
|
48
|
+
to this file. The skill directory is the one holding this file
|
|
49
|
+
(`${CLAUDE_SKILL_DIR}` in Claude Code, `~/.claude/skills/spec`,
|
|
50
|
+
`~/.agents/skills/spec` or an Elves clone). The repository root is `git
|
|
51
|
+
toplevel` (or `--root DIR`); the tree is `tests/` at the root. In a repository wired up per the "Wiring into a
|
|
52
|
+
repository" section — its `spec:doc` and `spec:diff`, not the scripts from the
|
|
53
|
+
skill directory: the project's package version is pinned for CI.
|
|
54
|
+
|
|
55
|
+
## What to run when
|
|
56
|
+
|
|
57
|
+
| Moment | Action |
|
|
58
|
+
|---|---|
|
|
59
|
+
| Before a PR | 1. a test run with a JSON report (don't run e2e: for the names `playwright test --list --reporter=json > .spec-playwright.json` is enough); 2. `spec-doc --strict` with both reports — the tree and the main files are in order (`docs/spec` is in `.gitignore`, not committed); 3. `spec-diff --scenarios <task body>` → the `## Spec (tests)` section (in the project language) with the scenario check, into the PR body |
|
|
60
|
+
| In CI on a PR | shards — `spec-shard` (`ci.md`); run → `spec-doc --strict` (section "spec-doc"); `spec-diff` into the summary |
|
|
61
|
+
| In CI after a merge | a run on `main` → `spec-doc` → `spec-publish`; without CI on `main` — `docs/spec` from the PR run's artifact and `spec-publish --source <merge SHA>` (`ci.md`). The `spec` branch = the built `docs/spec`, the script verifies it after the push |
|
|
62
|
+
| Read a project's spec | `git fetch origin spec && git show origin/spec:README.md` (pages — `capabilities/<name>.md`…) or the tests themselves |
|
|
63
|
+
| Asked for documentation, "what does the system do" | `spec-doc <report> --stdout` — one document, files aren't touched |
|
|
64
|
+
| Asked which requirements a PR removed | `spec-diff` — the «Removed» list comes first |
|
|
65
|
+
| A project moves to a test spec | section "Wiring into a repository": installing the flow, scripts under Bun, workflow; migrating the old spec's requirements — `migration.md` (outcomes, removal categories, "the spec fell behind the code") |
|
|
66
|
+
| A convention for all items (mutations, routes, files) | `invariant` in `tests/standards/<name>/<name>.test.ts` — section "Checks" |
|
|
67
|
+
| Prove that a check can fail (migrating an old spec, a "covered" review) | `spec-break` — a breakage aimed at the check, one per run |
|
|
68
|
+
| After running all tests and shards | `spec-claims` — every entry point is called by a capability test; its report goes to `spec-doc` together with the runner reports |
|
|
69
|
+
| The harness failed «exceptions are one file per element in exceptions/, not exceptions.ts» | `spec-exceptions` — section "spec-exceptions" |
|
|
70
|
+
| spec-doc warns that `names.exceptions` «isn't read — names aren't exceptions any more, delete it» | delete that directory or file (#447) |
|
|
71
|
+
|
|
72
|
+
## Reports for spec-doc
|
|
73
|
+
|
|
74
|
+
The format is detected from the content; several reports can be passed at
|
|
75
|
+
once (Vitest + Playwright + bun test), identical tests are merged.
|
|
76
|
+
|
|
77
|
+
| Runner | Command |
|
|
78
|
+
|---|---|
|
|
79
|
+
| Vitest (and Jest — same format) | `vitest run --reporter=default --reporter=json --outputFile.json=.spec-report.json` |
|
|
80
|
+
| `bun test` | `bun test --reporter=junit --reporter-outfile=.spec-report.xml` — JUnit with nested `testsuite` per describe |
|
|
81
|
+
| Playwright | `PLAYWRIGHT_JSON_OUTPUT_NAME=.spec-playwright.json playwright test --reporter=json` |
|
|
82
|
+
| Playwright without a run (before a PR) | `playwright test --list --reporter=json > .spec-playwright.json` — the tree and names without browsers, tests show as skipped: for `spec-doc --strict` and `spec-diff`, not for publishing |
|
|
83
|
+
|
|
84
|
+
Reports are temporary files, in `.gitignore`; a report from CI with someone
|
|
85
|
+
else's absolute paths works. The runner itself merges shards (blob reports →
|
|
86
|
+
one JSON, section "Wiring into a repository"); don't pass shard reports
|
|
87
|
+
separately (`reference.md`, "Report formats").
|
|
88
|
+
|
|
89
|
+
## spec-doc
|
|
90
|
+
|
|
91
|
+
A page per decision folder is a story in three layers: the `<folder>.md`
|
|
92
|
+
description (**why**), `describe` sections with prose from their JSDoc
|
|
93
|
+
(**what it can do**), the section's tests in a collapsed block (**what checks
|
|
94
|
+
it**); plus the `README.md` index and `architecture.md` from the model
|
|
95
|
+
`tests/architecture/model.ts`. `--strict` — exit code 1 when:
|
|
96
|
+
|
|
97
|
+
- a test is outside `tests/capabilities/<name>`, `tests/architecture/<name>`
|
|
98
|
+
and `tests/standards/<name>` (in `src/`, in `tests/unit/`, a file directly in
|
|
99
|
+
`tests/capabilities/`) — «Outside the tree», a signal to move it;
|
|
100
|
+
- a folder has no main file `tests/<kind>/<name>/<name>.*` (in a monorepo also
|
|
101
|
+
`tests/<kind>/<name>/<package>/<name>.*`): its `describe` order is the order
|
|
102
|
+
of the story;
|
|
103
|
+
- a folder has no description `tests/<kind>/<name>/<name>.md`;
|
|
104
|
+
- a test file has a JSDoc header (its text belongs in `<folder>.md` or in
|
|
105
|
+
`//`), or a standard's main file is still `rule.*` (rename it to `<name>.*`);
|
|
106
|
+
- a `<!-- spec: c4-context -->` marker (`c4-container`, `c4-component`,
|
|
107
|
+
`c4-modules`) is unknown or has no model; the model doesn't load — exit
|
|
108
|
+
code 2.
|
|
109
|
+
|
|
110
|
+
Names of `describe` and `it` are statements in the project language
|
|
111
|
+
(`language` in `.agents/elves.json`): an identifier, a function or file name,
|
|
112
|
+
a path — a warning line, no exit code.
|
|
113
|
+
|
|
114
|
+
**Prose:** `<folder>.md` — why, the reason and the rejected alternatives (its
|
|
115
|
+
first paragraph is the index description); JSDoc `/** … */` before `describe`
|
|
116
|
+
or `it` — one to three sentences where the name isn't enough; `//` doesn't go
|
|
117
|
+
into the documentation. Each file, the order, names, an example —
|
|
118
|
+
`reference.md`, "spec-doc output" and `reference.md`, "Writing prose".
|
|
119
|
+
|
|
120
|
+
## spec-diff
|
|
121
|
+
|
|
122
|
+
The ready `## Spec (tests)` section for the PR body: test names compared
|
|
123
|
+
**statically** — the `tests/` files at two revisions via `git show`, without a
|
|
124
|
+
run. The removed come first: a removed test is a removed requirement. Then
|
|
125
|
+
«Changed» (renames), «Added» and, only if they changed, «Architecture model»,
|
|
126
|
+
«Exceptions», «Harness checks».
|
|
127
|
+
|
|
128
|
+
- The base is the merge-base of `--base` (default `origin/main`) and HEAD;
|
|
129
|
+
before running — `git fetch origin`, otherwise `origin/main` is stale.
|
|
130
|
+
`--worktree` — the working tree instead of HEAD, to look before a commit.
|
|
131
|
+
- `--scenarios` — the task body (`gh issue view N --json body --jq .body | …
|
|
132
|
+
--scenarios -`): its `## Scenarios` section against the added and changed
|
|
133
|
+
tests — «### Task scenarios»; a scenario that became a test is named the
|
|
134
|
+
same as the test, ❌ — a scenario without a test.
|
|
135
|
+
- Tests that the harness generates are only in the report: `--report <report>`
|
|
136
|
+
(repeatable) — the head from the PR run's report, the base from `tests.json`
|
|
137
|
+
of the `spec` branch.
|
|
138
|
+
- More tests than `--limit` (100) — a per-folder summary; `--full` — the full
|
|
139
|
+
list.
|
|
140
|
+
- The section is inserted into the PR body as a whole (after `Closes #N`); on
|
|
141
|
+
new commits with tests — regenerate it and replace it.
|
|
142
|
+
|
|
143
|
+
Matching, renames, a move into the tree, an example — `reference.md`,
|
|
144
|
+
"spec-diff output".
|
|
145
|
+
|
|
146
|
+
## spec-publish, spec-run — after a merge
|
|
147
|
+
|
|
148
|
+
In CI (`ci.md`). `spec-publish` puts the built `docs/spec` at the root of the
|
|
149
|
+
`spec` branch, `Source: <SHA>` in the commit, without touching the working
|
|
150
|
+
copy, and verifies the remote branch after the push; `--check` — comparison
|
|
151
|
+
only. `spec-run --tree <hash>` — without CI on `main`: the green run that
|
|
152
|
+
checked the merge commit's tree; stdout empty — publishing is skipped. Details
|
|
153
|
+
— `reference.md`, "spec-publish" and `reference.md`, "spec-run".
|
|
154
|
+
|
|
155
|
+
## spec-break
|
|
156
|
+
|
|
157
|
+
A decision's check must be able to fail. The agent picks the breakage for one
|
|
158
|
+
check (this is not mutation testing): `--find` occurs in the file exactly
|
|
159
|
+
once; a plan of several — each a separate run. A baseline run goes first: red
|
|
160
|
+
even without a breakage — exit code 2. After an interruption without a revert
|
|
161
|
+
— the next run or `--restore`. Details — `reference.md`, "spec-break".
|
|
162
|
+
|
|
163
|
+
## spec-exceptions
|
|
164
|
+
|
|
165
|
+
A decision's exceptions are the folder's `exceptions/` directory, one file per
|
|
166
|
+
exception (`<item>.json`: `{ item, rule?, issue, reason }`), not an array in
|
|
167
|
+
`exceptions.ts`. `spec-exceptions` moves each `exceptions.ts` of a decision
|
|
168
|
+
folder there and rewrites the imports (`exceptionsIn()`) and the paths in
|
|
169
|
+
`package.json` and workflows; a `!` line — fix by hand. A repeated run changes
|
|
170
|
+
nothing. Details — `reference.md`, "spec-exceptions".
|
|
171
|
+
|
|
172
|
+
## Checks — `harness.ts`, `architecture.ts`, `spec-claims.ts`
|
|
173
|
+
|
|
174
|
+
A mechanical check is runner tests (or a report in the runner's format): they
|
|
175
|
+
are in the report, and so in `docs/spec` and in `spec-diff`. Import — from the
|
|
176
|
+
project's package (`@miroshnik/elves/spec/harness`, `…/spec/architecture`,
|
|
177
|
+
`…/spec/eslint-config`), the runner doesn't matter — `it` passes the test in. Which check for what and how to wire
|
|
178
|
+
each one — `checks.md` next to this file:
|
|
179
|
+
|
|
180
|
+
- a convention on every item of a registry from the code — `invariant`
|
|
181
|
+
(`standards/<name>/`);
|
|
182
|
+
- a lint rule — `examples`, `eslintLinter`, the folder's `eslint.ts` fragment
|
|
183
|
+
collected by `collectEslint`;
|
|
184
|
+
- out of scope and exceptions — `fileOf`, marks `spec-outside(<decision>)` /
|
|
185
|
+
`spec-exception(<decision>) #N`, `exceptionsIn()`, `lintExceptions`;
|
|
186
|
+
- the architecture model (modules, external systems, containers, workspace
|
|
187
|
+
packages) — `architecture/model.ts`, `architecture`, `boundariesConfig`,
|
|
188
|
+
`networkGuard`, `cspConnectSrc`, `deployUnits`, `workspacePackages`;
|
|
189
|
+
- interaction order in a scenario — `trace`, `sequence`;
|
|
190
|
+
- every entry point is called by a capability test — `journal` +
|
|
191
|
+
`spec-claims` (`standards/entry-points/`);
|
|
192
|
+
- no code without a consumer — `deadCode` (knip);
|
|
193
|
+
- environment variables — `envVars`, `dotenvNames`, `configGet`.
|
|
194
|
+
|
|
195
|
+
## Wiring into a repository
|
|
196
|
+
|
|
197
|
+
The scripts and modules come from the project's package — `@miroshnik/elves`
|
|
198
|
+
in `devDependencies`, committed with its version: the scripts run as
|
|
199
|
+
`bun --no-env-file node_modules/.bin/elves spec-…`, a Node project included
|
|
200
|
+
(in CI — `oven-sh/setup-bun@v2`); on Yarn 2+, which runs the `.ts` bin by
|
|
201
|
+
Node and has no `node_modules` with PnP, —
|
|
202
|
+
`bunx @miroshnik/elves@"$(jq -r '.devDependencies["@miroshnik/elves"]' package.json)" spec-…`
|
|
203
|
+
(`install` writes this form there); the modules the project's tests and config import
|
|
204
|
+
(`@miroshnik/elves/spec/harness`, `…/spec/architecture`,
|
|
205
|
+
`…/spec/eslint-config`) load under the project's own runner on Node — from
|
|
206
|
+
the package's build. Each step with its details — `reference.md`,
|
|
207
|
+
"Wiring into a repository"; an old spec's requirements — `migration.md`.
|
|
208
|
+
|
|
209
|
+
1. **Install** at the project root, the result into a commit:
|
|
210
|
+
`bunx @miroshnik/elves install` — the package version, the manifest, the
|
|
211
|
+
shim and the `AGENTS.md` block; update — `update` in the current task's
|
|
212
|
+
branch, a few lines.
|
|
213
|
+
2. **Scripts** in `package.json` and CI — the package's `elves` under
|
|
214
|
+
`bun --no-env-file`: the bare bin would load the project's env files;
|
|
215
|
+
reports — those the project has:
|
|
216
|
+
```json
|
|
217
|
+
"spec:doc": "bun --no-env-file node_modules/.bin/elves spec-doc .spec-report.json .spec-playwright.json .spec-claims.xml --strict",
|
|
218
|
+
"spec:diff": "bun --no-env-file node_modules/.bin/elves spec-diff",
|
|
219
|
+
"spec:publish": "bun --no-env-file node_modules/.bin/elves spec-publish",
|
|
220
|
+
"spec:claims": "bun --no-env-file node_modules/.bin/elves spec-claims --entries tests/standards/entry-points/entries.ts"
|
|
221
|
+
```
|
|
222
|
+
Standards and architecture run once — the `test:spec` script, a fast CI job
|
|
223
|
+
of its own; `test` excludes them, e2e doesn't run them (`checks.md`):
|
|
224
|
+
```json
|
|
225
|
+
"test": "vitest run --exclude 'tests/standards/**' --exclude 'tests/architecture/**'",
|
|
226
|
+
"test:spec": "vitest run tests/standards tests/architecture"
|
|
227
|
+
```
|
|
228
|
+
In `.gitignore`: `docs/spec/`, `.spec-*.json`, `.spec-*.xml`,
|
|
229
|
+
`.spec-meta/`, `.spec-journal/`, `vitest-blob/`, `blob-report/`,
|
|
230
|
+
`playwright-blob/`.
|
|
231
|
+
3. **`docs/spec/` isn't committed to `main`** — CI publishes it to the `spec`
|
|
232
|
+
branch.
|
|
233
|
+
4. **CI** — a shard's files from `spec-shard` by the last green run's
|
|
234
|
+
durations, shards write blob reports, a `spec` job merges them and builds
|
|
235
|
+
`docs/spec` with `--strict`, only `spec-publish` has a write token,
|
|
236
|
+
`spec-diff` is a light job of its own: the workflow fragment — `ci.md`.
|
|
237
|
+
5. **Monorepo** — a decision's tests in package subfolders
|
|
238
|
+
`tests/<kind>/<name>/<package>/`, each package with its own runner project;
|
|
239
|
+
run from the repository root.
|
|
240
|
+
|
|
241
|
+
## Limitations to know about
|
|
242
|
+
|
|
243
|
+
A test's name is a string known without a run (a literal, a template without
|
|
244
|
+
substitutions): otherwise it isn't in the `spec-diff` lists — a line in
|
|
245
|
+
stderr. Vitest and `bun test` reports don't contain the skip reason — keep it
|
|
246
|
+
in the test name or in a comment next to it. On `main` everything must be ✅
|
|
247
|
+
or ⏭️ with a reason. The rest — `reference.md`, "Limitations to know about".
|