@kirill.konshin/lint 0.6.1 → 0.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -366,7 +366,7 @@ Where each runs:
|
|
|
366
366
|
| `spec-tools steward check` | `spec-tools gates`, every merge request: an error fails it | the edit hook on every edit Claude makes, and `spec-tools gates` before handing back |
|
|
367
367
|
| `spec-tools steward coverage`, `spec-tools steward evidence --json` | the coverage report, and the evidence `spec-tools scope` builds the spec-verify audit from | on demand |
|
|
368
368
|
| spec-steward's corpus audit, review rounds, `align` | never — it is heavy and its outcome is the owner's | on the owner's request: `spec-tools audit --audit spec-steward [--context <decisions file>]`, then the review rounds, where the accepted answers are applied |
|
|
369
|
-
| `spec-verify` | the reading
|
|
369
|
+
| `spec-verify` | one audit job — the reading, then the judging: its verdict gates merge requests and the release | `spec-tools audit`, and `/spec-verify changed` before handing back a spec edit |
|
|
370
370
|
| `spec-tools gates`, `evidence`, `diff` | their own jobs | before handing back |
|
|
371
371
|
|
|
372
372
|
Specifications change only by hand, or when the steward skill applies the owner's accepted review answers; `spec-tools steward check --fix` makes the only mechanical repairs. No audit edits code or specifications.
|
package/package.json
CHANGED
|
@@ -47,7 +47,7 @@ The author's pre-hand-back check: every requirement a hunk of the diff overlaps
|
|
|
47
47
|
|
|
48
48
|
### Run class — `spec-tools tier`
|
|
49
49
|
|
|
50
|
-
Decided once, printed as dotenv lines (`AUDIT_RUN`, `AUDIT_SCOPE`, `AUDIT_GATING`, `AUDIT_MODEL`, `AUDIT_EFFORT`, `AUDIT_MODEL_VERIFY`, `AUDIT_EFFORT_VERIFY`); every audit job reads the answer.
|
|
50
|
+
Decided once, printed as dotenv lines (`AUDIT_RUN`, `AUDIT_SCOPE`, `AUDIT_GATING`, `AUDIT_MODEL`, `AUDIT_EFFORT`, `AUDIT_MODEL_VERIFY`, `AUDIT_EFFORT_VERIFY`, `AUDIT_SLOTS`); every audit job reads the answer.
|
|
51
51
|
|
|
52
52
|
| run | decided by | scope | verifier | gating |
|
|
53
53
|
| -------------- | --------------------------------------- | -------- | --------- | -------- |
|
|
@@ -57,11 +57,11 @@ Decided once, printed as dotenv lines (`AUDIT_RUN`, `AUDIT_SCOPE`, `AUDIT_GATING
|
|
|
57
57
|
| merge-request | a merge request touching none | affected | cheap | gating |
|
|
58
58
|
| default-branch | anything else | affected | expensive | gating |
|
|
59
59
|
|
|
60
|
-
Every run reads on the cheap model. `AUDIT_MODEL_CHEAP`, `AUDIT_EFFORT_CHEAP`, `AUDIT_MODEL_EXPENSIVE` and `AUDIT_EFFORT_EXPENSIVE` override the defaults. A merge request whose open changes cannot be classified fails the command rather than reading as one that touches nothing.
|
|
60
|
+
Every run reads on the cheap model. `AUDIT_MODEL_CHEAP`, `AUDIT_EFFORT_CHEAP`, `AUDIT_MODEL_EXPENSIVE` and `AUDIT_EFFORT_EXPENSIVE` override the defaults. `AUDIT_SLOTS` is how many workers a reading job runs at once: as many as `AUDIT_JOB_MEMORY` (default 2 GiB) holds at 300 MB a worker, never more than `AUDIT_WORKERS` (default 2). A merge request whose open changes cannot be classified fails the command rather than reading as one that touches nothing.
|
|
61
61
|
|
|
62
62
|
### Scope — `spec-tools scope`
|
|
63
63
|
|
|
64
|
-
The corpus on the nightly, or wherever there is no merge-request target; otherwise the affected set — what `changed` names — judged with cross-capability checks against the corpus its evidence quotes. The evidence is assembled mechanically first: per capability, every requirement verbatim, its bound tests at their real lines, where its terms occur, its related requirements; citations are not evidence and are left out. The scope is cut into parts
|
|
64
|
+
The corpus on the nightly, or wherever there is no merge-request target; otherwise the affected set — what `changed` names — judged with cross-capability checks against the corpus its evidence quotes. The evidence is assembled mechanically first: per capability, every requirement verbatim, its bound tests at their real lines, where its terms occur, its related requirements; citations are not evidence and are left out. The scope is cut into `AUDIT_JOBS` × `AUDIT_SLOTS` parts — one round of workers across the pipeline's reading jobs (`AUDIT_JOBS`, its own count, default 1) — balanced by load, none larger than one cheap reader holds, a capability larger than that cut between its requirements; only a scope larger than a round holds opens more parts. The cut is a function of the evidence and those two numbers. Written to `audit-scope.json` and `audit-parts/`.
|
|
65
65
|
|
|
66
66
|
Cases:
|
|
67
67
|
|
|
@@ -70,7 +70,7 @@ Cases:
|
|
|
70
70
|
|
|
71
71
|
### Workers — `spec-tools workers [--complete|--verify]`
|
|
72
72
|
|
|
73
|
-
One headless `claude --print` worker per part loads the `spec-verify` skill and writes its own findings file.
|
|
73
|
+
One headless `claude --print` worker per part loads the `spec-verify` skill and writes its own findings file. In a parallel job (`CI_NODE_INDEX` of `CI_NODE_TOTAL`) it reads only its share: part p goes to job ((p − 1) mod total) + 1. `--complete` judges, as each part's next reader, exactly what the merge listed short; `--verify` puts every standing ERROR to one verifier that confirms it only when the divergence is real and critical. How many run at once follows the container's memory (300 MB a worker), never more than `AUDIT_WORKERS` (default 2), and each pass logs the job's memory peak against its limit; a worker past `AUDIT_WORKER_TIMEOUT` (20m) is killed and its part judged by the completion pass. A pass that changed any file outside the audit's outputs exits 1, naming the paths; a missing input exits 2.
|
|
74
74
|
|
|
75
75
|
### Merge — `spec-tools report [--gate] [--worktree]`
|
|
76
76
|
|
|
@@ -11,12 +11,30 @@ The jobs a GitLab repository runs around `spec-steward` and `spec-tools`. Names
|
|
|
11
11
|
- Report jobs depend on the audits with `dependencies:` and `when: always`, never `needs:` — a failed audit is exactly when its report matters.
|
|
12
12
|
- The default-branch pipeline publishes the reports on Pages, a failing one included; a report a run did not produce is omitted.
|
|
13
13
|
- The scheduled nightly sets `SPEC_AUDIT_NIGHTLY=true`, so `spec-tools tier` audits the corpus there.
|
|
14
|
+
- Reading jobs × workers is the pipeline's choice: `AUDIT_JOBS` reading jobs (default 1; a `parallel:` job sets the same number), each running `AUDIT_SLOTS` workers at once — as many as `AUDIT_JOB_MEMORY` holds, never more than `AUDIT_WORKERS` — and the scope is cut for one round across them. The reading and the judging hand off through files, so they run in one job (below) or as parallel reading jobs and a judge job that `needs:` them; `spec-tools workers` reads only its share in a parallel job. Set `AUDIT_JOB_MEMORY` once and give the AI jobs the same `KUBERNETES_MEMORY_LIMIT` (a YAML anchor keeps them one value; the runner must allow the overwrite); each pass logs the memory it used, which is what `AUDIT_WORKERS` is tuned from. Parallel reading jobs pay when one job cannot hold the round — a runner's default 2 GiB holds three workers; with the memory raised one job holds it, and the job that judges is the job that read: one pod, no artifact hop.
|
|
14
15
|
- spec-steward's corpus audit (`--audit spec-steward`) has no job: it is heavy, and its review file is the owner's to answer, locally.
|
|
15
16
|
- Declare what binds beyond spec-steward's defaults once, in `package.json`: `"spec-steward": { "binds": ["scripts/checks/**", "**/*.Dockerfile", ".gitlab-ci.yml"] }`.
|
|
16
17
|
|
|
17
18
|
## Getting the tools into an AI job
|
|
18
19
|
|
|
19
|
-
An AI job
|
|
20
|
+
An AI job needs Node and the Claude CLI, no workspace install. Pin the CLI rather than taking whatever an image carries: a model needs a CLI that knows it (its context window and its effort levels), and an older CLI fails a long worker with "Prompt is too long". Download Anthropic's release binary for the runner's platform, check it against the release manifest, and cache it per version:
|
|
21
|
+
|
|
22
|
+
```yaml
|
|
23
|
+
.claude-code: &claude-code
|
|
24
|
+
- |
|
|
25
|
+
set -e
|
|
26
|
+
platform="linux-$(uname -m | sed 's/x86_64/x64/; s/aarch64/arm64/')"
|
|
27
|
+
dir=".claude-code/$CLAUDE_CODE_VERSION/$platform"
|
|
28
|
+
if [ ! -x "$dir/claude" ]; then
|
|
29
|
+
base="https://downloads.claude.ai/claude-code-releases/$CLAUDE_CODE_VERSION"
|
|
30
|
+
sum=$(curl -fsSL "$base/manifest.json" | node -e "let s='';process.stdin.on('data',(d)=>(s+=d)).on('end',()=>console.log(JSON.parse(s).platforms[process.argv[1]].checksum))" "$platform")
|
|
31
|
+
mkdir -p "$dir" && curl -fsSL "$base/$platform/claude" -o "$dir/claude.download"
|
|
32
|
+
echo "$sum $dir/claude.download" | sha256sum -c - && mv "$dir/claude.download" "$dir/claude" && chmod +x "$dir/claude"
|
|
33
|
+
fi
|
|
34
|
+
export PATH="$CI_PROJECT_DIR/$dir:$PATH"
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Cache `.claude-code/` keyed on `CLAUDE_CODE_VERSION`, and gitignore it. The job that computes the scope runs with the workspace installed and hands the package's `skills/` on as an artifact: the self-contained CLI, `skills/spec-tools/scripts/cli.js`, and the skills its workers read. The AI jobs `needs:` that job and call the file with `node`.
|
|
20
38
|
|
|
21
39
|
`spec-tools scope` and `changed` load `typescript`, so they run in a job with the workspace installed.
|
|
22
40
|
|
|
@@ -65,24 +83,17 @@ spec-coverage:
|
|
|
65
83
|
node_modules/@kirill.konshin/lint/skills/,
|
|
66
84
|
]
|
|
67
85
|
|
|
68
|
-
# The
|
|
69
|
-
|
|
70
|
-
stage: review
|
|
71
|
-
image: <an image with claude and node>
|
|
72
|
-
needs: [spec-coverage]
|
|
73
|
-
parallel: 2 # 8 on the nightly
|
|
74
|
-
allow_failure: true
|
|
75
|
-
script:
|
|
76
|
-
- node node_modules/@kirill.konshin/lint/skills/spec-tools/scripts/cli.js workers
|
|
77
|
-
artifacts: { when: always, paths: [audit-parts/findings/] }
|
|
78
|
-
|
|
79
|
-
# The judge: merge, complete what the reading left short, verify every ERROR, gate.
|
|
86
|
+
# The audit, in one job: read (a failed reading leaves its parts to the completion pass), merge, complete what the
|
|
87
|
+
# reading left short, verify every ERROR, gate.
|
|
80
88
|
spec-verify:
|
|
81
89
|
stage: review
|
|
82
|
-
image: <an image with
|
|
83
|
-
needs: [spec-coverage
|
|
90
|
+
image: <an image with node>
|
|
91
|
+
needs: [spec-coverage]
|
|
92
|
+
variables: { KUBERNETES_MEMORY_LIMIT: 8Gi, KUBERNETES_CPU_LIMIT: '4' } # AUDIT_JOB_MEMORY says the same
|
|
84
93
|
allow_failure: { exit_codes: [3, 77] }
|
|
85
94
|
script:
|
|
95
|
+
- *claude-code
|
|
96
|
+
- node node_modules/@kirill.konshin/lint/skills/spec-tools/scripts/cli.js workers || echo "the completion pass reads what the reading left short"
|
|
86
97
|
- node node_modules/@kirill.konshin/lint/skills/spec-tools/scripts/cli.js report
|
|
87
98
|
- node node_modules/@kirill.konshin/lint/skills/spec-tools/scripts/cli.js workers --complete
|
|
88
99
|
- node node_modules/@kirill.konshin/lint/skills/spec-tools/scripts/cli.js report
|
|
@@ -94,10 +105,11 @@ spec-verify:
|
|
|
94
105
|
# Any other skill-driven review: headless, then its verdict.
|
|
95
106
|
some-review:
|
|
96
107
|
stage: review
|
|
97
|
-
image: <an image with
|
|
108
|
+
image: <an image with node>
|
|
98
109
|
needs: [spec-coverage]
|
|
99
110
|
allow_failure: { exit_codes: [3, 77] }
|
|
100
111
|
script:
|
|
112
|
+
- *claude-code
|
|
101
113
|
- node node_modules/@kirill.konshin/lint/skills/spec-tools/scripts/cli.js run some-review --verdict some-review.md --advisory
|
|
102
114
|
artifacts: { when: always, paths: [some-review.md, claude.jsonl, 'job-log-*.md'] }
|
|
103
115
|
|
|
@@ -3593,7 +3593,7 @@ function renderCapability(capability, ctx) {
|
|
|
3593
3593
|
for (const r of own) {
|
|
3594
3594
|
const trial = new Map(quoted);
|
|
3595
3595
|
const block = renderRequirement(r, ctx, false, trial);
|
|
3596
|
-
if (taken.length && Buffer.byteLength(textOf([...lines, ...block])) >
|
|
3596
|
+
if (taken.length && Buffer.byteLength(textOf([...lines, ...block])) > 368640) {
|
|
3597
3597
|
slices.push({
|
|
3598
3598
|
requirements: taken,
|
|
3599
3599
|
text: textOf(lines)
|
|
@@ -3666,7 +3666,7 @@ var init_auditParts = __esmMin((() => {
|
|
|
3666
3666
|
init_repo();
|
|
3667
3667
|
init_steward();
|
|
3668
3668
|
typescript = () => loaded ??= createRequire(import.meta.url)("typescript");
|
|
3669
|
-
PART_BYTES =
|
|
3669
|
+
PART_BYTES = 368640;
|
|
3670
3670
|
isMarker = (line) => /^\*\*⚠️ /.test(line.trim());
|
|
3671
3671
|
idOf = (r) => `${r.capability}#${r.slug}`;
|
|
3672
3672
|
isBoundary = (node) => typescript().isStatement(node) && !typescript().isBlock(node) || typescript().isClassElement(node) || typescript().isFunctionExpression(node) || typescript().isArrowFunction(node) || typescript().isPropertyAssignment(node);
|
|
@@ -3872,12 +3872,23 @@ var init_auditChanged = __esmMin((() => {
|
|
|
3872
3872
|
var tier_exports = /* @__PURE__ */ __exportAll({
|
|
3873
3873
|
CHEAP_EFFORT: () => CHEAP_EFFORT,
|
|
3874
3874
|
CHEAP_MODEL: () => CHEAP_MODEL,
|
|
3875
|
+
DEFAULT_JOB_MEMORY: () => DEFAULT_JOB_MEMORY,
|
|
3876
|
+
DEFAULT_WORKERS: () => 2,
|
|
3875
3877
|
EXPENSIVE_EFFORT: () => EXPENSIVE_EFFORT,
|
|
3876
3878
|
EXPENSIVE_MODEL: () => EXPENSIVE_MODEL,
|
|
3879
|
+
WORKER_BYTES: () => WORKER_BYTES,
|
|
3877
3880
|
dotenv: () => dotenv,
|
|
3881
|
+
memoryOf: () => memoryOf,
|
|
3878
3882
|
openChangeClass: () => openChangeClass,
|
|
3879
|
-
tier: () => tier
|
|
3883
|
+
tier: () => tier,
|
|
3884
|
+
workersFor: () => workersFor
|
|
3880
3885
|
});
|
|
3886
|
+
/** `2Gi`, `2G`, `1536Mi`, `512M` or bare bytes, in bytes; binary units either way, as a container limit is. */
|
|
3887
|
+
function memoryOf(text) {
|
|
3888
|
+
const m = /^(\d+(?:\.\d+)?)\s*([KMGT]?)i?B?$/i.exec(text.trim());
|
|
3889
|
+
if (!m) throw new Error(`bad memory size: ${text}`);
|
|
3890
|
+
return Math.round(Number(m[1]) * 1024 ** " KMGT".indexOf((m[2] || " ").toUpperCase()));
|
|
3891
|
+
}
|
|
3881
3892
|
/** The open changes this merge request touches, by workflow-evidence's classification against the gate base. */
|
|
3882
3893
|
function openChangeClass(env = process.env) {
|
|
3883
3894
|
const base = gateBase(env);
|
|
@@ -3890,8 +3901,8 @@ function openChangeClass(env = process.env) {
|
|
|
3890
3901
|
function tier(env = process.env, openChange = () => openChangeClass(env)) {
|
|
3891
3902
|
const mr = mergeRequest(env);
|
|
3892
3903
|
const run = isNightly(env) ? "nightly" : !mr ? "default-branch" : mr.draft ? "draft" : openChange() === "none" ? "merge-request" : "open-change";
|
|
3893
|
-
const cheap = [env.AUDIT_MODEL_CHEAP || "claude-haiku-
|
|
3894
|
-
const expensive = [env.AUDIT_MODEL_EXPENSIVE || "claude-sonnet-5", env.AUDIT_EFFORT_EXPENSIVE || "medium"];
|
|
3904
|
+
const cheap = [env.AUDIT_MODEL_CHEAP || "claude-haiku-5-5", env.AUDIT_EFFORT_CHEAP || "high"];
|
|
3905
|
+
const expensive = [env.AUDIT_MODEL_EXPENSIVE || "claude-sonnet-5-5", env.AUDIT_EFFORT_EXPENSIVE || "medium"];
|
|
3895
3906
|
const judge = run === "nightly" || run === "default-branch" ? expensive : cheap;
|
|
3896
3907
|
return {
|
|
3897
3908
|
AUDIT_RUN: run,
|
|
@@ -3900,18 +3911,22 @@ function tier(env = process.env, openChange = () => openChangeClass(env)) {
|
|
|
3900
3911
|
AUDIT_MODEL: cheap[0],
|
|
3901
3912
|
AUDIT_EFFORT: cheap[1],
|
|
3902
3913
|
AUDIT_MODEL_VERIFY: judge[0],
|
|
3903
|
-
AUDIT_EFFORT_VERIFY: judge[1]
|
|
3914
|
+
AUDIT_EFFORT_VERIFY: judge[1],
|
|
3915
|
+
AUDIT_SLOTS: workersFor(env, env.AUDIT_JOB_MEMORY ? memoryOf(env.AUDIT_JOB_MEMORY) : DEFAULT_JOB_MEMORY)
|
|
3904
3916
|
};
|
|
3905
3917
|
}
|
|
3906
|
-
var CHEAP_MODEL, CHEAP_EFFORT, EXPENSIVE_MODEL, EXPENSIVE_EFFORT, dotenv;
|
|
3918
|
+
var CHEAP_MODEL, CHEAP_EFFORT, EXPENSIVE_MODEL, EXPENSIVE_EFFORT, WORKER_BYTES, DEFAULT_JOB_MEMORY, workersFor, dotenv;
|
|
3907
3919
|
var init_tier = __esmMin((() => {
|
|
3908
3920
|
init_changeGates();
|
|
3909
3921
|
init_ci();
|
|
3910
3922
|
init_workflowEvidence();
|
|
3911
|
-
CHEAP_MODEL = "claude-haiku-
|
|
3923
|
+
CHEAP_MODEL = "claude-haiku-5-5";
|
|
3912
3924
|
CHEAP_EFFORT = "high";
|
|
3913
|
-
EXPENSIVE_MODEL = "claude-sonnet-5";
|
|
3925
|
+
EXPENSIVE_MODEL = "claude-sonnet-5-5";
|
|
3914
3926
|
EXPENSIVE_EFFORT = "medium";
|
|
3927
|
+
WORKER_BYTES = 300 * 1024 ** 2;
|
|
3928
|
+
DEFAULT_JOB_MEMORY = 2 * 1024 ** 3;
|
|
3929
|
+
workersFor = (env, memory) => Math.max(1, Math.min(Number(env.AUDIT_WORKERS) || 2, Math.floor(memory / WORKER_BYTES)));
|
|
3915
3930
|
dotenv = (t) => Object.entries(t).map(([key, value]) => `${key}=${value}`).join("\n") + "\n";
|
|
3916
3931
|
}));
|
|
3917
3932
|
//#endregion
|
|
@@ -4971,23 +4986,25 @@ function modeFor(env = process.env) {
|
|
|
4971
4986
|
return mergeRequest(env)?.target ? "affected" : "all";
|
|
4972
4987
|
}
|
|
4973
4988
|
/**
|
|
4974
|
-
* The partition the audit starts from — decided here so the orchestrator opens nothing before its
|
|
4975
|
-
*
|
|
4976
|
-
*
|
|
4977
|
-
*
|
|
4978
|
-
*
|
|
4979
|
-
*
|
|
4989
|
+
* The partition the audit starts from — decided here so the orchestrator opens nothing before its workers run.
|
|
4990
|
+
* `slots` parts are one round of workers: AUDIT_JOBS reading jobs (the pipeline's own count, one unless it says) of
|
|
4991
|
+
* AUDIT_SLOTS workers each (from `spec-tools tier`). Evidence files are
|
|
4992
|
+
* placed heaviest first, by evidence and requirements, each into the lightest part it fits in under `PART_BYTES`; a
|
|
4993
|
+
* file no part has room for opens another, so only a scope larger than a round holds takes a second round, and no part
|
|
4994
|
+
* is larger than one reader holds unless one file alone is. Empty parts are dropped: nothing in scope is none. A
|
|
4995
|
+
* function of the files alone, in whatever order they come: the parts are numbered heaviest first.
|
|
4980
4996
|
*/
|
|
4981
|
-
function partition(files, audit = "spec-verify") {
|
|
4997
|
+
function partition(files, audit = "spec-verify", slots = 1) {
|
|
4982
4998
|
const order = [...files].sort((x, y) => load(y) - load(x) || (x.file < y.file ? -1 : x.file > y.file ? 1 : 0));
|
|
4983
|
-
const bins = [];
|
|
4999
|
+
const bins = Array.from({ length: Math.max(1, slots) }, () => []);
|
|
4984
5000
|
const bytesOf = (bin) => bin.reduce((sum, f) => sum + f.bytes, 0);
|
|
5001
|
+
const loadOf = (bin) => bin.reduce((sum, f) => sum + load(f), 0);
|
|
4985
5002
|
for (const f of order) {
|
|
4986
|
-
const
|
|
4987
|
-
if (
|
|
5003
|
+
const room = bins.filter((b) => bytesOf(b) + f.bytes <= 368640 || !b.length);
|
|
5004
|
+
if (room.length) room.reduce((a, b) => loadOf(b) < loadOf(a) ? b : a).push(f);
|
|
4988
5005
|
else bins.push([f]);
|
|
4989
5006
|
}
|
|
4990
|
-
return bins.map((bin, index) => ({
|
|
5007
|
+
return bins.filter((bin) => bin.length).sort((a, b) => loadOf(b) - loadOf(a)).map((bin, index) => ({
|
|
4991
5008
|
part: index + 1,
|
|
4992
5009
|
capabilities: [...new Set(bin.map((f) => f.capability))],
|
|
4993
5010
|
files: bin.map((f) => f.file),
|
|
@@ -5012,7 +5029,7 @@ function renderScope(parts, kind = "all", detail) {
|
|
|
5012
5029
|
if (parts.length === 0) lines.push("Nothing in scope: the request touches no requirement, no delta, no bound test and no source where a requirement names a term — there is nothing to judge, and the verdict is PASS.");
|
|
5013
5030
|
else if (parts.length === 1) lines.push(`One part, ${parts[0].requirements} requirements (${kb(parts[0].bytes)}).`);
|
|
5014
5031
|
else {
|
|
5015
|
-
lines.push(`${parts.length} parts, 1 workers each,
|
|
5032
|
+
lines.push(`${parts.length} parts, 1 workers each, one round across the reading jobs:`, "");
|
|
5016
5033
|
parts.forEach((part) => lines.push(`- Part ${part.part} (${part.requirements} requirements, ${kb(part.bytes)}): ${part.files.map((f) => `\`${f.slice(PARTS_DIR.length + 1)}\``).join(", ")}`));
|
|
5017
5034
|
}
|
|
5018
5035
|
return lines.join("\n") + "\n";
|
|
@@ -5027,7 +5044,7 @@ function main$2(env = process.env, audit = "spec-verify") {
|
|
|
5027
5044
|
const all = requirementsOf(model);
|
|
5028
5045
|
const affected = kind === "affected" ? affectedScope(diffBase(env), all) : void 0;
|
|
5029
5046
|
const corpus = [...new Set(all.map((r) => r.capability))].sort();
|
|
5030
|
-
const capabilityParts = partition(writeParts(affected ? affected.capabilities : corpus, all), audit);
|
|
5047
|
+
const capabilityParts = partition(writeParts(affected ? affected.capabilities : corpus, all), audit, (Number(env.AUDIT_JOBS) || 1) * (Number(env.AUDIT_SLOTS) || 1));
|
|
5031
5048
|
const parts = audit === "spec-steward" ? [...capabilityParts, ...sweepUnits(model, capabilityParts.length)] : capabilityParts;
|
|
5032
5049
|
const report = renderScope(parts, kind, affected ? `The affected set against \`${affected.base}\`: ${affected.requirements.length} requirements in ${affected.capabilities.length} capabilities — the ones the diff changed or a touched delta names, the ones whose bound tests or own terms the changed files contain, the ones the changed code cites, and the requirements related to these — with cross-capability checks against the corpus the evidence quotes.` : void 0);
|
|
5033
5050
|
const json = {
|
|
@@ -5063,11 +5080,13 @@ var init_auditScope = __esmMin((() => {
|
|
|
5063
5080
|
* The specification audit's fan-out, deterministic: headless `claude --print` workers started from here — no
|
|
5064
5081
|
* orchestrating model decides when or whether the next worker starts. Each worker loads the `spec-verify` skill for its
|
|
5065
5082
|
* contract — the five checks, the grading, the findings file — and writes its own findings file; `spec-tools report`
|
|
5066
|
-
* merges after every pass. CI
|
|
5083
|
+
* merges after every pass. CI runs the passes in one job or the reading in parallel jobs and the rest in a judge job —
|
|
5084
|
+
* the hand-off is files either way — and `spec-tools audit` runs them locally, in this order:
|
|
5067
5085
|
*
|
|
5068
|
-
* 1. reading One worker per part of `audit-scope.json`, judging every check over every requirement of its part
|
|
5069
|
-
*
|
|
5070
|
-
*
|
|
5086
|
+
* 1. reading One worker per part of `audit-scope.json`, judging every check over every requirement of its part;
|
|
5087
|
+
* the scope cut one part per slot of every reading job, so the reading is one round. In a parallel
|
|
5088
|
+
* job (`ci.ts` `shard`) only the job's share: part p goes to job ((p - 1) mod total) + 1, so each job
|
|
5089
|
+
* works its share out from the scope alone. Then the merge.
|
|
5071
5090
|
* 2. complete One worker per part the merge left short — the `short` list of `spec-verify.json` — over exactly
|
|
5072
5091
|
* those items, writing the part's next reader file. Nothing already judged is judged twice.
|
|
5073
5092
|
* 3. verify One verifier per ERROR the merge left standing, over that finding alone, confirming it only when
|
|
@@ -5088,20 +5107,18 @@ var workers_exports = /* @__PURE__ */ __exportAll({
|
|
|
5088
5107
|
briefVerify: () => briefVerify,
|
|
5089
5108
|
concurrency: () => concurrency,
|
|
5090
5109
|
duration: () => duration,
|
|
5110
|
+
memoryLine: () => memoryLine,
|
|
5091
5111
|
shareOf: () => shareOf,
|
|
5092
5112
|
treeChanges: () => treeChanges,
|
|
5093
5113
|
treeState: () => treeState,
|
|
5094
5114
|
workers: () => workers
|
|
5095
5115
|
});
|
|
5096
|
-
/** How many workers run at once:
|
|
5116
|
+
/** How many workers run at once: in a container `tier`'s workersFor its memory; elsewhere AUDIT_WORKERS, else all. */
|
|
5097
5117
|
function concurrency(env, memoryLimit = cgroupLimit()) {
|
|
5098
|
-
|
|
5099
|
-
if (memoryLimit === null) return asked;
|
|
5100
|
-
const bound = Math.max(2, Math.floor(memoryLimit / PER_WORKER_BYTES));
|
|
5101
|
-
return asked > 0 ? Math.min(asked, bound) : bound;
|
|
5118
|
+
return memoryLimit === null ? Number(env.AUDIT_WORKERS) || 0 : workersFor(env, memoryLimit);
|
|
5102
5119
|
}
|
|
5103
|
-
function
|
|
5104
|
-
for (const file of
|
|
5120
|
+
function cgroupBytes(files) {
|
|
5121
|
+
for (const file of files) {
|
|
5105
5122
|
let value;
|
|
5106
5123
|
try {
|
|
5107
5124
|
value = fs$1.readFileSync(file, "utf8").trim();
|
|
@@ -5250,6 +5267,12 @@ async function workers(pass, { audit = "spec-verify", context, env = process.env
|
|
|
5250
5267
|
const effort = env.AUDIT_EFFORT || kind.effort;
|
|
5251
5268
|
const budget = duration(env.AUDIT_WORKER_TIMEOUT || "20m");
|
|
5252
5269
|
const limit = concurrency(env);
|
|
5270
|
+
/** One pass's workers, `limit` at a time, then the memory they used. */
|
|
5271
|
+
const runPass = async (tasks) => {
|
|
5272
|
+
await pool(tasks, limit);
|
|
5273
|
+
const memory = memoryLine();
|
|
5274
|
+
if (memory) log(`spec-tools workers: ${memory}, ${limit} at a time`);
|
|
5275
|
+
};
|
|
5253
5276
|
const findings = path$1.join(cwd, files.findings);
|
|
5254
5277
|
fs$1.mkdirSync(findings, { recursive: true });
|
|
5255
5278
|
const before = treeState(cwd);
|
|
@@ -5322,7 +5345,7 @@ async function workers(pass, { audit = "spec-verify", context, env = process.env
|
|
|
5322
5345
|
return 0;
|
|
5323
5346
|
}
|
|
5324
5347
|
log(`spec-tools workers: verification pass over ${targets.length} finding(s), model ${env.AUDIT_MODEL_VERIFY || model} effort ${env.AUDIT_EFFORT_VERIFY || effort}`);
|
|
5325
|
-
await
|
|
5348
|
+
await runPass(targets.map((t, i) => () => verify(t, i + 1)));
|
|
5326
5349
|
const code = unchanged();
|
|
5327
5350
|
if (code === 0) log("spec-tools workers: verification done");
|
|
5328
5351
|
return code;
|
|
@@ -5332,10 +5355,10 @@ async function workers(pass, { audit = "spec-verify", context, env = process.env
|
|
|
5332
5355
|
return 0;
|
|
5333
5356
|
}
|
|
5334
5357
|
log(`spec-tools workers: completion pass over ${report.short.length} short part(s), model ${model} effort ${effort}`);
|
|
5335
|
-
await
|
|
5358
|
+
await runPass(report.short.flatMap((s) => {
|
|
5336
5359
|
const p = partOf(s.part);
|
|
5337
5360
|
return p ? [() => work(p, s)] : [];
|
|
5338
|
-
})
|
|
5361
|
+
}));
|
|
5339
5362
|
const code = unchanged();
|
|
5340
5363
|
if (code === 0) log("spec-tools workers: completion done");
|
|
5341
5364
|
return code;
|
|
@@ -5354,14 +5377,14 @@ async function workers(pass, { audit = "spec-verify", context, env = process.env
|
|
|
5354
5377
|
return unchanged();
|
|
5355
5378
|
}
|
|
5356
5379
|
log(`spec-tools workers: ${share.length} parts${of}, model ${model} effort ${effort}, ${limit === 0 ? "all at once" : `${limit} at a time`}`);
|
|
5357
|
-
await
|
|
5380
|
+
await runPass(share.map((p) => () => work(p)));
|
|
5358
5381
|
const missing = share.map((p) => p.findings[0]).filter((f) => !fs$1.existsSync(path$1.join(cwd, f)));
|
|
5359
5382
|
if (missing.length) log(`spec-tools workers: findings files not written (the completion pass judges those parts):\n${missing.join("\n")}`);
|
|
5360
5383
|
const code = unchanged();
|
|
5361
5384
|
if (code === 0) log("spec-tools workers: done");
|
|
5362
5385
|
return code;
|
|
5363
5386
|
}
|
|
5364
|
-
var TOOLS, STEWARD_TOOLS,
|
|
5387
|
+
var TOOLS, STEWARD_TOOLS, OUTPUTS, cgroupLimit, cgroupPeak, gb, memoryLine, shareOf, TOOL_RULE, SKILL, loadSkill, brief, briefComplete, briefVerify, KINDS, clock, readJson;
|
|
5365
5388
|
var init_workers = __esmMin((() => {
|
|
5366
5389
|
init_ci();
|
|
5367
5390
|
init_files();
|
|
@@ -5371,8 +5394,11 @@ var init_workers = __esmMin((() => {
|
|
|
5371
5394
|
init_tier();
|
|
5372
5395
|
TOOLS = "Skill,Bash(git:git diff*|git log*|git show*),Read,Glob,Grep,Write,Edit";
|
|
5373
5396
|
STEWARD_TOOLS = "Skill,Bash(git:git diff*|git log*|git show*),Read,Glob,Grep,Write";
|
|
5374
|
-
PER_WORKER_BYTES = 629145600;
|
|
5375
5397
|
OUTPUTS = /^(audit-parts\/|spec-verify\.|job-log-)/;
|
|
5398
|
+
cgroupLimit = () => cgroupBytes(["/sys/fs/cgroup/memory.max", "/sys/fs/cgroup/memory/memory.limit_in_bytes"]);
|
|
5399
|
+
cgroupPeak = () => cgroupBytes(["/sys/fs/cgroup/memory.peak", "/sys/fs/cgroup/memory/memory.max_usage_in_bytes"]);
|
|
5400
|
+
gb = (bytes) => `${(bytes / 1024 ** 3).toFixed(2)} GB`;
|
|
5401
|
+
memoryLine = (limit = cgroupLimit(), peak = cgroupPeak()) => limit === null ? void 0 : `memory: peak ${peak === null ? "unknown" : gb(peak)} of a ${gb(limit)} limit`;
|
|
5376
5402
|
shareOf = (parts, { index, total }) => parts.filter((p) => (p.part - 1) % total === index - 1);
|
|
5377
5403
|
TOOL_RULE = "Tool rule: only Skill, Read, Glob, Grep, Write, Edit and the git commands — no ls/cat/find, no shell validation of your JSON; other commands are denied here and only cost a turn.";
|
|
5378
5404
|
SKILL = path$1.join(SKILLS_DIR, "spec-verify/SKILL.md");
|
|
@@ -5429,7 +5455,7 @@ Read, with the Read and Grep tools only: the requirement's block in the part's e
|
|
|
5429
5455
|
|
|
5430
5456
|
Confirm the ERROR only when ALL of these hold:
|
|
5431
5457
|
1. The quotes are genuine and say what the finding claims.
|
|
5432
|
-
2. The requirement and the code genuinely diverge — not a vague requirement read narrowly, not an ambiguity the surrounding code resolves, not behaviour implemented elsewhere that the finding did not look at.
|
|
5458
|
+
2. The requirement and the code genuinely diverge — not a vague requirement read narrowly, not an ambiguity the surrounding code resolves, not behaviour implemented elsewhere that the finding did not look at, and not a dependency's behaviour — a command-line flag, a library default — that the repository does not show.
|
|
5433
5459
|
3. The divergence is critical: an obvious code defect — the code does something different from what the requirement mandates in a way a caller, a test or an operator would observe — or a statement that would steer an agent wrongly (a prompt, a tool description, a workflow rule the agents follow).
|
|
5434
5460
|
|
|
5435
5461
|
Otherwise the finding is WARN. A citation or a comment is never evidence either way: judge the requirement and the code. Write exactly this JSON with the Write tool to ${file} and nothing else:
|
|
@@ -6087,7 +6113,7 @@ async function run(skill, log, env = process.env) {
|
|
|
6087
6113
|
"--print",
|
|
6088
6114
|
`/${skill}`,
|
|
6089
6115
|
"--model",
|
|
6090
|
-
env.AUDIT_MODEL || "claude-haiku-
|
|
6116
|
+
env.AUDIT_MODEL || "claude-haiku-5-5",
|
|
6091
6117
|
"--effort",
|
|
6092
6118
|
env.AUDIT_EFFORT || "high"
|
|
6093
6119
|
];
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: spec-verify
|
|
3
3
|
description: Audit whether the code does exactly what the OpenSpec specifications say, as written; its verdict gates CI. Checks what no tool can - cross-capability contradiction, EXACT code conformance, whether a bound test asserts its rule, and gaps nobody recorded. It audits code conformance only; corpus quality — placement, REQUIRED versus ⚠️ ADVISORY, evidence — is the spec-steward skill's audit. Workers only judge; each writes audit-parts/findings/part-<n>-<reader>.json and `spec-tools report` renders spec-verify.md at the repo root. Interactively, run `spec-tools audit`, then summarize spec-verify.md and ask the operator what to do. After changing a capability, `/spec-verify changed` judges what changed; the full audit runs when the operator asks.
|
|
4
|
-
model: claude-haiku-
|
|
4
|
+
model: claude-haiku-5-5
|
|
5
5
|
effort: high
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -9,7 +9,7 @@ effort: high
|
|
|
9
9
|
|
|
10
10
|
The capability specifications under `openspec/specs/` are the single source of truth. The tooling's contract — this audit's scope, partition, merge and verdict — is the `spec-tools` skill's `references/contract.md`; the grading and the finding kinds are below; how a specification is written is the `spec-steward` model (its `references/model.md`); how a change is made is the repository's `AGENTS.md`. A code citation is an optional pointer to a requirement, never evidence that code complies.
|
|
11
11
|
|
|
12
|
-
**Where it runs:** in CI as the gating audit (the reading
|
|
12
|
+
**Where it runs:** in CI as the gating audit (one job: the reading, then the judging), and locally as `spec-tools audit` and `/spec-verify changed`. spec-steward's corpus audit is the local, owner-requested counterpart; it never runs in CI.
|
|
13
13
|
|
|
14
14
|
**Invariant this skill enforces: CODE 100% follows the SPEC.** A requirement and its code must match exactly; a mismatch is always a finding (the owner decides which side is fixed — never leave them divergent).
|
|
15
15
|
|
|
@@ -52,7 +52,7 @@ A **requirement** is one `### Requirement:` heading and everything under it. Add
|
|
|
52
52
|
|
|
53
53
|
## The worker contract
|
|
54
54
|
|
|
55
|
-
One reader cannot hold the whole corpus and every symbol it governs; a run that tries samples, and a sample cannot support a `PASS`. So the evidence is assembled mechanically before the audit starts, and `spec-tools workers` starts one headless worker per part. The passes run in order
|
|
55
|
+
One reader cannot hold the whole corpus and every symbol it governs; a run that tries samples, and a sample cannot support a `PASS`. So the evidence is assembled mechanically before the audit starts, and `spec-tools workers` starts one headless worker per part. The passes run in order, in one job — the reading, the completion pass over what the reading left short, and the verification pass over every ERROR still standing. Each later brief names exactly what it judges, and the tool runs every pass, never a worker.
|
|
56
56
|
|
|
57
57
|
A worker's brief gives its part number, its reader number, the findings file it alone writes, its part's evidence files and its `requirementIds`. It judges all five checks over its evidence in one pass: the specification against itself (checks 2 and 3, from the requirement blocks and related lists the evidence carries) and the code against the specification (1, 4 and 5) — every requirement, every bound test and every listed related pair. It opens a source only where an excerpt is marked as a window, where a term hit must be confirmed, where the behaviour continues outside the excerpt, or where a related requirement's statement line is not enough to decide. Its tools are Skill, Read, Glob, Grep, Write, Edit (to repair its own findings file) and the git commands; anything else is denied and only costs a turn. It writes its findings file and returns one line: the path and its counts per tier.
|
|
58
58
|
|
|
@@ -72,7 +72,7 @@ Checks 2–5 are spec-steward's audit criteria 10, 9, 11 and 5 (its `references/
|
|
|
72
72
|
|
|
73
73
|
Every finding is in exactly one of three tiers:
|
|
74
74
|
|
|
75
|
-
- **ERROR** — a defect you demonstrated: you opened both sides and quote each at `file:line`. It fails the build once its verifier confirms it real and critical.
|
|
75
|
+
- **ERROR** — a defect you demonstrated: you opened both sides and quote each at `file:line`. It fails the build once its verifier confirms it real and critical. It rests on the repository alone: where what the code does turns on a dependency's behaviour — a command-line flag, a library default — that the repository does not show, it is not demonstrated.
|
|
76
76
|
- **WARN** — a defect of the same class, suspected but not demonstrated: suspected from a sample, or not read on both sides. Published, never a failure.
|
|
77
77
|
- **INFO** — inventory and state recorded for a reader. Never a failure.
|
|
78
78
|
|