@kirill.konshin/lint 0.6.3 → 0.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/commands/spec-author.md +2 -2
- package/package.json +1 -2
- package/skills/spec-steward/SKILL.md +14 -9
- package/skills/spec-steward/references/apply.md +14 -14
- package/skills/spec-steward/references/audit.md +15 -15
- package/skills/spec-steward/references/contract.md +6 -6
- package/skills/spec-steward/references/model.md +13 -1
- package/skills/spec-steward/references/review-format.md +70 -44
- package/skills/spec-steward/references/worker.md +51 -53
- package/skills/spec-tools/SKILL.md +14 -8
- package/skills/spec-tools/references/contract.md +19 -12
- package/skills/spec-tools/references/gitlab.md +22 -24
- package/skills/spec-tools/references/rules/common.md +46 -0
- package/skills/spec-tools/references/rules/spec-steward.md +71 -0
- package/skills/spec-tools/references/rules/spec-verify-judge.md +17 -0
- package/skills/spec-tools/references/rules/spec-verify.md +50 -0
- package/skills/spec-tools/scripts/cli.js +10990 -4528
- package/skills/spec-verify/SKILL.md +78 -128
- package/skills/spec-steward/references/criteria.md +0 -59
|
@@ -1,153 +1,103 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: spec-verify
|
|
3
|
-
description: Audit whether the code
|
|
3
|
+
description: Audit whether the code satisfies the intent of the OpenSpec specifications; its verdict gates CI. Checks what no tool can - cross-capability contradiction, code that does not satisfy a requirement's intent, whether a bound test asserts its scenario, and gaps nobody recorded. It audits code conformance only; corpus quality is the spec-steward skill's audit. The judging rules are in the spec-tools skill's references/rules/ (common.md, spec-verify.md). Readers only judge; each writes .spec-audit/parts/findings/part-<n>-<reader>.yaml and `spec-tools report` renders .spec-audit/spec-verify.md. Interactively, run `spec-tools audit`, then summarize the report and ask the operator what to do. After changing a capability, `/spec-verify changed` judges what changed.
|
|
4
4
|
model: claude-haiku-5-5
|
|
5
5
|
effort: high
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Verify spec adherence
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
This skill tells how the audit runs. What to judge is in the judging rules: [`common.md`](../spec-tools/references/rules/common.md), then [`spec-verify.md`](../spec-tools/references/rules/spec-verify.md). Read them before you judge. The tooling's contract (scope, partition, merge, verdict) is the `spec-tools` skill's `references/contract.md`.
|
|
11
11
|
|
|
12
|
-
|
|
12
|
+
There are three ways in:
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
- A reader: the brief names a part and a findings file. Follow _The reader_.
|
|
15
|
+
- `/spec-verify changed`: _Focused mode_.
|
|
16
|
+
- `/spec-verify` from an operator: _Interactive use_.
|
|
15
17
|
|
|
16
|
-
The audit
|
|
18
|
+
The audit is read-only. No reader changes code or specifications. A reader writes its own findings file and nothing else.
|
|
17
19
|
|
|
18
|
-
|
|
20
|
+
## What tools do, not this skill
|
|
19
21
|
|
|
20
|
-
|
|
22
|
+
Do not report what these report:
|
|
21
23
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
- `
|
|
25
|
-
- `spec-tools
|
|
26
|
-
- `spec-tools report` — **the merge.** It looks every ERROR's quotes up at their `file:line` in the audited commit, tiers by the kinds below, takes the weakest part's coverage, computes the verdict and renders `spec-verify.md`. No reader judges twice.
|
|
27
|
-
- `spec-tools steward coverage` — the coverage report: which requirements and scenarios a test binds, the ⚠️ Advisory requirements and the recorded known gaps. The audit does not read it.
|
|
28
|
-
- Corpus quality — whether a rule sits in the right place, whether it is REQUIRED or ⚠️ Advisory, whether its evidence is the right kind — is the `spec-steward` skill's audit.
|
|
29
|
-
|
|
30
|
-
**Why the split matters.** This skill is an LLM audit: one reader over the whole corpus samples, and two such runs over the same commit can reach different verdicts. Everything mechanically decidable was moved out of it and into code — including finding the evidence. What remains needs judgment — read in full, one worker per part, and graded so that only what you can demonstrate stops a pipeline.
|
|
31
|
-
|
|
32
|
-
## Scope
|
|
33
|
-
|
|
34
|
-
In CI, `spec-tools tier` chooses each pass's scope, model and effort — the run classes are the `spec-tools` contract's — and this frontmatter applies to an in-chat run only. The scope is partitioned before any worker starts, in `audit-scope.json`: its `parts` list the parts, one worker each, with the evidence files each part judges. Ignore build output (any `dist/`, `node_modules`, and whatever else the repository's `AGENTS.md` names as generated).
|
|
24
|
+
- `openspec validate` and `spec-tools gates`: the corpus gates (structure, citations, binding, markers, size, the scenario ratchet).
|
|
25
|
+
- `spec-tools scope`: the scope, the partition and the evidence of each part.
|
|
26
|
+
- `spec-tools report`: the merge. It re-proves each ERROR's quotes at their `file:line`, applies the tiers and the judge's verdicts, and computes the verdict.
|
|
27
|
+
- `spec-tools steward coverage`: which requirements and scenarios a test binds.
|
|
35
28
|
|
|
36
29
|
## Focused mode — `/spec-verify changed`
|
|
37
30
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
## The material
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
-
|
|
45
|
-
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
The
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
```json
|
|
92
|
-
{
|
|
93
|
-
"part": 1,
|
|
94
|
-
"reader": 1,
|
|
95
|
-
"capabilities": ["<capability>", "<another>"],
|
|
96
|
-
"findings": [
|
|
97
|
-
{
|
|
98
|
-
"kind": "code-mismatch",
|
|
99
|
-
"tier": "ERROR",
|
|
100
|
-
"where": "<capability>#requirement-<slug> / <package>/src/<file>.ts:6",
|
|
101
|
-
"quotes": [
|
|
102
|
-
{
|
|
103
|
-
"file": "openspec/specs/<capability>/spec.md",
|
|
104
|
-
"line": 101,
|
|
105
|
-
"text": "SHALL mark the in-flight assistant message as errored and render the failure text inside its bubble"
|
|
106
|
-
},
|
|
107
|
-
{
|
|
108
|
-
"file": "<package>/src/<file>.ts",
|
|
109
|
-
"line": 6,
|
|
110
|
-
"text": "if (event.type === 'RUN_ERROR') return;"
|
|
111
|
-
}
|
|
112
|
-
],
|
|
113
|
-
"detail": "The handler returns on RUN_ERROR, so the in-flight assistant message is never marked errored and no failure text renders in its bubble."
|
|
114
|
-
}
|
|
115
|
-
],
|
|
116
|
-
"coverage": {
|
|
117
|
-
"1": "full",
|
|
118
|
-
"2": "full",
|
|
119
|
-
"3": "full",
|
|
120
|
-
"4": "sampled: <capability>#requirement-<slug> — the excerpt was a window and the source was not opened",
|
|
121
|
-
"5": "full"
|
|
122
|
-
},
|
|
123
|
-
"judged": ["<capability>#requirement-<slug>", "<capability>#requirement-<other-slug>"],
|
|
124
|
-
"notes": [
|
|
125
|
-
"<capability>#requirement-<slug> and <capability>#requirement-<other-slug> overlap in prose on one rule; no shared term paired them"
|
|
126
|
-
]
|
|
127
|
-
}
|
|
31
|
+
`spec-tools changed` has already written `.spec-audit/parts/changed.yaml`: the requirements the working tree changed, each with its bound tests, where its terms occur, and its related requirements in full. You are the one reader. Read that file. Run checks 2, 3 and 4 over those requirements. Print the findings in the conversation. Do not start workers or write a file.
|
|
32
|
+
|
|
33
|
+
## The material
|
|
34
|
+
|
|
35
|
+
`spec-tools scope` writes two inputs. In CI they arrive as artifacts. Do not produce them.
|
|
36
|
+
|
|
37
|
+
- `.spec-audit/scope.yaml`: the parts. Each part has its number, the evidence `files` its reader reads, and the `requirementIds` it judges.
|
|
38
|
+
- `.spec-audit/parts/<capability>.yaml`: the evidence. Each requirement is there with its `id`, `location` and `block`, then its `boundTests` (each with `location`, `binds` and `code`), its `terms` and `occurrences`, and its `related` requirements. A bound test with a `window` is cut: read the file at those lines before you judge it. A bound test with `sameAs` is quoted under that other requirement.
|
|
39
|
+
|
|
40
|
+
Address a requirement as `<capability>#<slug>`.
|
|
41
|
+
|
|
42
|
+
## The reader
|
|
43
|
+
|
|
44
|
+
The tool starts one reader per part, then a completion reader for each part that is short, then one judge per ERROR. The judge's rules are [`spec-verify-judge.md`](../spec-tools/references/rules/spec-verify-judge.md); the judge loads no skill.
|
|
45
|
+
|
|
46
|
+
A reader judges checks 1–5 over every requirement of its part, writes its findings file, and returns one line: the path and its counts per tier. When the reader stops, the tool checks the file. If the file has errors, the tool sends them back, and the reader corrects the file.
|
|
47
|
+
|
|
48
|
+
## The findings file
|
|
49
|
+
|
|
50
|
+
Write it with the Write tool, at the path the brief names: `.spec-audit/parts/findings/part-<n>-<reader>.yaml`. Its shape:
|
|
51
|
+
|
|
52
|
+
```yaml
|
|
53
|
+
part: 1
|
|
54
|
+
reader: 1
|
|
55
|
+
capabilities:
|
|
56
|
+
- 'billing/refunds'
|
|
57
|
+
findings:
|
|
58
|
+
- kind: 'code-mismatch'
|
|
59
|
+
tier: 'ERROR'
|
|
60
|
+
where: 'billing/refunds#requirement-a-refund-is-applied-once / src/refunds.ts:6'
|
|
61
|
+
readerConfidence: 80
|
|
62
|
+
quotes:
|
|
63
|
+
- file: 'openspec/specs/billing/refunds/spec.md'
|
|
64
|
+
line: 12
|
|
65
|
+
text: |-
|
|
66
|
+
The service SHALL apply a refund at most once, however often it is retried.
|
|
67
|
+
- file: 'src/refunds.ts'
|
|
68
|
+
line: 6
|
|
69
|
+
text: |-
|
|
70
|
+
await ledger.credit(refund.amount);
|
|
71
|
+
detail: |
|
|
72
|
+
The handler credits the ledger on each retry. A retried refund pays twice.
|
|
73
|
+
coverage:
|
|
74
|
+
'1': 'full'
|
|
75
|
+
'2': 'full'
|
|
76
|
+
'3': 'full'
|
|
77
|
+
'4': 'full'
|
|
78
|
+
'5': 'full'
|
|
79
|
+
judged:
|
|
80
|
+
- 'billing/refunds#requirement-a-refund-is-applied-once'
|
|
81
|
+
notes:
|
|
82
|
+
- 'billing/refunds#requirement-x and billing/invoices#requirement-y overlap in their words; no shared term paired them'
|
|
128
83
|
```
|
|
129
84
|
|
|
130
|
-
|
|
131
|
-
- Keep the JSON plain: double quotes, no trailing commas, no comments; a `detail` or `note` is one string with no line breaks. A file that does not parse is a part that ran nothing.
|
|
132
|
-
- Write the file once and stop. Do not validate it with a shell — no `python3 -c`, `node -e`, `wc`: Bash is denied to workers, every attempt is a wasted turn, and the merge reads the file back.
|
|
133
|
-
- `quotes` carry the proof. An ERROR needs at least two — both sides, each from a specification or a source file, never from `audit-parts/` — each a **verbatim fragment of one line** (or of consecutive lines) copied from the evidence excerpt without its line-number prefix, at most ~200 characters. `spec-tools report` looks each up at its `file:line`; a quote that is not there turns the finding into WARN, so copy, never paraphrase. WARN and INFO findings carry no `quotes`: their `where` and `detail` are what the report shows.
|
|
134
|
-
- `coverage` has one string per check — `"1"` to `"5"`: `"full"`, or `"sampled: <what was skipped>"` / `"not-run: <why>"`. Strings, not objects — every bracket is a chance to break the file. The note after `sampled:` names the source you could not read; a judgment about a test — it covers the happy path only, it asserts less than the rule — is a finding of kind `untested`, and a check whose only shortfall is such a judgment is `"full"`.
|
|
135
|
-
- `judged` is the judged list: every requirement id in your evidence files, added as you finish judging it for checks 1–5, whether or not it yielded a finding. The merge compares this list with the part's requirements: an id missing from it was not judged, the check is sampled for the whole audit and the verdict is `INCOMPLETE`, whatever `coverage` says. So walk the requirements one by one and write each id down when it is done; a `"full"` with a short list is a contradiction the merge resolves against you. Before you write the file, set the list against your part's `requirementIds` in `audit-scope.json`: an id you judged but forgot goes in now; an id you did not judge stays out and is reported as such — never added to look complete.
|
|
136
|
-
- `notes` are prose-only overlaps the term lists could not pair — suspicions for the reader, published as such.
|
|
85
|
+
How to write each key:
|
|
137
86
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
87
|
+
- `kind`, `tier`, `where`, `file`: double-quoted strings. `tier` is `ERROR`, `WARN` or `INFO`.
|
|
88
|
+
- `readerConfidence`, `line`, `part`, `reader`: plain integers. Each finding has a `readerConfidence`.
|
|
89
|
+
- `detail`: a literal block (`|`), one or two sentences. The report shows its first sentence.
|
|
90
|
+
- `quotes[].text`: a stripped literal block (`|-`), copied from the source line without its line-number prefix. An ERROR has at least two quotes: one from the specification and one from the code. A quote from `.spec-audit/` or from installed dependency code (`node_modules/`) does not count: the audit judges the repository's own code. WARN and INFO findings have no quotes.
|
|
91
|
+
- `coverage`: one string per check, `"1"` to `"5"`: `"full"`. Write `"sampled: <the source>"` only when a source was not available after you tried to read it. The tool keeps it as a note; it does not change the verdict.
|
|
92
|
+
- `judged`: every requirement id of the part that you judged. A missing id leaves the run short until the completion pass judges it.
|
|
93
|
+
- Every path you name exists. Do not write a bare file name.
|
|
141
94
|
|
|
142
|
-
|
|
95
|
+
## Interactive use — `/spec-verify`
|
|
143
96
|
|
|
144
|
-
|
|
97
|
+
Run `spec-tools audit` (`--here` judges the uncommitted edits). Then summarize `.spec-audit/spec-verify.md` in ASD-STE100, one line for each finding, most severe first:
|
|
145
98
|
|
|
146
|
-
|
|
99
|
+
`⟨ERROR|WARN|INFO⟩ <capability>#requirement-<slug> <file>:<line> — <kind> — <one sentence>`
|
|
147
100
|
|
|
148
|
-
|
|
149
|
-
- **Fix the errors / fix errors and warnings** — you, never a worker, apply the whole batch at once. Per finding, propose the side — code or specification — with a one-line reason, show the list, and edit nothing until the operator has confirmed or changed each side. A specification fix is a direct edit of the standing specification unless the user asked for the delta flow. Before touching a requirement, read the requirements it names and that name it — they are in its evidence file — so the fix does not contradict a sibling; after the batch, run `spec-tools changed` and judge `audit-parts/changed.md` as _Focused mode_ describes, then the checks the repository's `AGENTS.md` lists. Report the diff of everything changed and stop; the operator decides whether to run the audit again.
|
|
150
|
-
- **Go one by one** — per finding, in the summary's order: _fix in code_ / _fix in spec_ / _skip_ / _instructions_. Show the exact edit before making it; apply the confirmed ones as one batch and finish as above.
|
|
151
|
-
- **Stop** — leave the findings files and say so.
|
|
101
|
+
End with the counts per tier. Without an operator (no AskUserQuestion tool), stop there. With an operator, ask once what to do next (rerun, fix the errors, fix errors and warnings, go one by one, stop), and do nothing until the operator answers. For each fix, propose the side to change (code or specification), and edit nothing until the operator confirms. After the fixes, run `spec-tools changed`, judge as in _Focused mode_, and run the checks the repository's `AGENTS.md` lists.
|
|
152
102
|
|
|
153
|
-
`spec-
|
|
103
|
+
`spec-tools report` writes `.spec-audit/spec-verify.md`. Its first line is the verdict CI reads. You never write it.
|
|
@@ -1,59 +0,0 @@
|
|
|
1
|
-
# Audit criteria
|
|
2
|
-
|
|
3
|
-
An audit judges whether the corpus holds the **right** rules with the **right** evidence — specification quality first, implementation compliance only where code reveals a problem with the rule. Read [model.md](model.md) first.
|
|
4
|
-
|
|
5
|
-
## What deserves an item
|
|
6
|
-
|
|
7
|
-
Every rule, requirement or scenario that matches one of these, numbered so findings can name them:
|
|
8
|
-
|
|
9
|
-
1. A REQUIRED rule that should be ⚠️ ADVISORY.
|
|
10
|
-
2. A rule that belongs in `AGENTS.md`, a scoped rule, a skill, a command, the OPSX config or a README rather than the specification (place it per `rules/agent.md`).
|
|
11
|
-
3. A rule better enforced by deterministic tooling than by an AI audit.
|
|
12
|
-
4. A rule that produces low-value or artificial tests.
|
|
13
|
-
5. A rule whose tests satisfy the wording while failing to protect the intent. (`spec-verify` check 5 judges it against the code.)
|
|
14
|
-
6. A rule that cannot reasonably be verified.
|
|
15
|
-
7. A rule whose wording allows materially different interpretations. An undefined evaluative term in a REQUIRED statement (_thin_, _safe_, _lean_, _at parity_, _needed_) is a finding: replace it with the criterion it stands for, drop the sentence where a precise rule already carries the obligation, or, when nothing checkable remains, make it ⚠️ ADVISORY.
|
|
16
|
-
8. A rule that describes implementation detail instead of an externally meaningful invariant.
|
|
17
|
-
9. A rule redundant with, or substantially overlapping, another. (`spec-verify` check 3.)
|
|
18
|
-
10. Contradictory rules. (`spec-verify` check 2.)
|
|
19
|
-
11. A rule that no longer matches the implementation or architecture. (`spec-verify` check 4 judges it against the code.)
|
|
20
|
-
12. Implementation suggesting the specification itself is wrong or outdated.
|
|
21
|
-
13. A missing high-value invariant the architecture clearly relies on.
|
|
22
|
-
14. A rule whose scope is unclear.
|
|
23
|
-
15. A rule that forces repository-wide reasoning where a subsystem scope would do.
|
|
24
|
-
16. A rule that encourages agents to modify specifications merely to make an implementation pass.
|
|
25
|
-
17. A rule whose verification relies on comments or superficial textual matching.
|
|
26
|
-
18. A tiny rule whose maintenance and enforcement cost is disproportionate to the failure it prevents.
|
|
27
|
-
19. Requirements or scenarios that repeat each other.
|
|
28
|
-
20. Scenario/When/Then structure that adds verbosity without precision.
|
|
29
|
-
|
|
30
|
-
Organisational criteria, for the mechanisms around the corpus:
|
|
31
|
-
|
|
32
|
-
21. The same concern held by two or more mechanisms (spec, design, `AGENTS.md`, scoped rule, skill, command, OPSX config, tooling, test, audit) — name the authoritative one and the copies to remove.
|
|
33
|
-
22. A rule shared by several repositories but worded differently.
|
|
34
|
-
23. A CI or audit configuration whose cost buys no signal, or whose classification of a run is wrong.
|
|
35
|
-
24. Citation machinery whose upkeep exceeds its value.
|
|
36
|
-
|
|
37
|
-
## How to judge
|
|
38
|
-
|
|
39
|
-
- **Ask what failure the verification would catch.** If the answer is trivial, artificial, coupled to implementation, or proves only textual or structural conformity, do not propose that verification.
|
|
40
|
-
- **Do not create compliance theatre.** Never propose a test merely because a rule lacks one.
|
|
41
|
-
- **Comments are never evidence and never a violation.** Judge executable behaviour and structure.
|
|
42
|
-
- **Two markers, nothing else.** A requirement is REQUIRED, or carries `**⚠️ Advisory:**`; absent behaviour or missing evidence is a `**⚠️ Known gap (<tracker>):**` line ([model.md](model.md#advisory-known-gap-or-neither), spellings in [contract.md](contract.md#markers)). A retired `**⚠️ Unenforced:**`, an untracked Known gap and an unmarked silence are findings, and a proposal never introduces one.
|
|
43
|
-
- **Ordinary bugs are out of scope** unless they show the rule is unrealistic, unclear, outdated, ambiguous, or contradicted by the tests' actual contract.
|
|
44
|
-
- **Do not decide intent silently.** Where the intent is ambiguous, say so and propose options; flag `needsHumanIntent`.
|
|
45
|
-
- **Do not flag style preference.** Every finding has a concrete reason.
|
|
46
|
-
- **Internal contracts are specification.** Do not move a rule out of the corpus because it is not user-facing.
|
|
47
|
-
- **Past owner decisions are stated.** A proposal that reverses one names it.
|
|
48
|
-
|
|
49
|
-
## Proposals name their target
|
|
50
|
-
|
|
51
|
-
Start every proposal with one of:
|
|
52
|
-
|
|
53
|
-
- `Keep → OpenSpec REQUIRED invariant` (optionally with a rewrite, or with a `**⚠️ Known gap (<tracker>):**` line when its behaviour or its evidence is missing)
|
|
54
|
-
- `Reclassify → OpenSpec ⚠️ ADVISORY` (the requirement gains an `**⚠️ Advisory:**` line; one that mixes a contract with guidance is split, the contract staying REQUIRED)
|
|
55
|
-
- `Move → <target>` — the design narrative (the repository's design documents or a package README), `<dir>/AGENTS.md`, `scoped rule (<glob>)`, `skill <name>` (referenced from `AGENTS.md` when it holds material moved out of it), `command <name>`, `openspec/config.yaml#<key>`, `README.md`
|
|
56
|
-
- `Replace → <tool>` — ESLint rule, dependency rule, type assertion, schema, script
|
|
57
|
-
- `Merge → <other rule>`
|
|
58
|
-
- `Rewrite → "<new text>"`
|
|
59
|
-
- `Delete → <reason>`
|