@kirill.konshin/lint 0.6.3 → 0.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/commands/spec-author.md +2 -2
- package/package.json +1 -2
- package/skills/spec-steward/SKILL.md +14 -9
- package/skills/spec-steward/references/apply.md +14 -14
- package/skills/spec-steward/references/audit.md +15 -15
- package/skills/spec-steward/references/contract.md +6 -6
- package/skills/spec-steward/references/model.md +13 -1
- package/skills/spec-steward/references/review-format.md +70 -44
- package/skills/spec-steward/references/worker.md +51 -53
- package/skills/spec-tools/SKILL.md +14 -8
- package/skills/spec-tools/references/contract.md +19 -12
- package/skills/spec-tools/references/gitlab.md +22 -24
- package/skills/spec-tools/references/rules/common.md +46 -0
- package/skills/spec-tools/references/rules/spec-steward.md +71 -0
- package/skills/spec-tools/references/rules/spec-verify-judge.md +17 -0
- package/skills/spec-tools/references/rules/spec-verify.md +50 -0
- package/skills/spec-tools/scripts/cli.js +10990 -4528
- package/skills/spec-verify/SKILL.md +78 -128
- package/skills/spec-steward/references/criteria.md +0 -59
package/README.md
CHANGED
|
@@ -356,7 +356,7 @@ Which to reach for:
|
|
|
356
356
|
| How do I write or change a rule? | `/spec-author` | The specification edited in place, with the code and tests that prove it |
|
|
357
357
|
| Do citations resolve, is every new scenario bound, did a rule weaken? | `spec-tools steward check` (the gate, the edit hook) | Fails the merge request on an error |
|
|
358
358
|
| Are these the right rules — placement, REQUIRED or ⚠️ ADVISORY, wording, evidence, duplicates, drift across repositories? | the `spec-steward` skill (audit on the `spec-tools` engine, review rounds, align) | A review file of proposals; the owner decides, steward applies; never a build failure |
|
|
359
|
-
| Does the code
|
|
359
|
+
| Does the code satisfy the intent of the rules, and do the bound tests assert them? | the `spec-verify` skill, run by `spec-tools` | Graded, quote-proved findings; PASS / FAIL / INCOMPLETE gates CI |
|
|
360
360
|
| Is the change finished, what did it change, how does CI run the audit? | `spec-tools` (gates, evidence, diff, tier, workers, verdict, comment, html) | Gates and reports for the pipeline |
|
|
361
361
|
|
|
362
362
|
Where each runs:
|
|
@@ -371,7 +371,7 @@ Where each runs:
|
|
|
371
371
|
|
|
372
372
|
Specifications change only by hand, or when the steward skill applies the owner's accepted review answers; `spec-tools steward check --fix` makes the only mechanical repairs. No audit edits code or specifications.
|
|
373
373
|
|
|
374
|
-
The audits share
|
|
374
|
+
The audits share one set of judging rules, `skills/spec-tools/references/rules/`: an ERROR is a demonstrated defect with a production effect (a broken flow, a contract or invariant violation, a sensitive-data leak, a performance problem), and it stands only when its judge confirms it above 70% confidence. `spec-verify`'s checks 2–5 are spec-steward's items 10, 9, 11 and 5, worded identically — steward asks whether the rule should change, `spec-verify` whether the code or the rule has diverged. Every file the tools write is under `.spec-audit/`, as YAML; Markdown only as the final report.
|
|
375
375
|
|
|
376
376
|
Setup and the GitLab pipeline are in the `spec-tools` skill.
|
|
377
377
|
|
package/commands/spec-author.md
CHANGED
|
@@ -18,7 +18,7 @@ How a specification is written is the spec-steward model, [`references/model.md`
|
|
|
18
18
|
|
|
19
19
|
4. **If the user chose a change folder**: `openspec new change <name>`, set `skip_specs: true` in its `.openspec.yaml`, write `proposal.md`, `design.md` and `tasks.md`. The behaviour still lands as a direct edit — the folder carries the plan, never deltas. Set `skip_specs` yourself when the change is plainly simple (a bugfix, a small feature); ask when in doubt. The folder is archived inside the merge or pull request, on the user's word.
|
|
20
20
|
|
|
21
|
-
5. **Write it.** Name the observable failure the rule prevents and the cheapest evidence that catches it; if there is none, mark the requirement `**⚠️ Advisory:** <why review is its evidence>`, and record what is knowingly missing as `**⚠️ Known gap (<tracker>):** <what is missing>`. Write a new or edited statement in EARS (`When <trigger>, the <system> SHALL <response>`, `While <state>, …`, `If <condition>, then …`, or `The <system> SHALL …`), keeping the RFC 2119 keyword in capitals, and each new or edited scenario in Given/When/Then, one case each, Given only where a precondition matters; leave untouched text as it is. Scenario names are unique within the file:
|
|
21
|
+
5. **Write it.** Name the observable failure the rule prevents and the cheapest evidence that catches it; if there is none, mark the requirement `**⚠️ Advisory:** <why review is its evidence>`, and record what is knowingly missing as `**⚠️ Known gap (<tracker>):** <what is missing>`. Write a new or edited statement in EARS (`When <trigger>, the <system> SHALL <response>`, `While <state>, …`, `If <condition>, then …`, or `The <system> SHALL …`), keeping the RFC 2119 keyword in capitals, and each new or edited scenario in Given/When/Then, one case each, Given only where a precondition matters; leave untouched text as it is. Write new and edited text in the model's light ASD-STE100 subset: one obligation per sentence, at most 25 words per sentence, one term for one concept, active voice, no vague word in a REQUIRED statement. Scenario names are unique within the file:
|
|
22
22
|
|
|
23
23
|
```markdown
|
|
24
24
|
### Requirement: <name>
|
|
@@ -36,4 +36,4 @@ How a specification is written is the spec-steward model, [`references/model.md`
|
|
|
36
36
|
|
|
37
37
|
6. **Verify**: run the checks the root `AGENTS.md` lists; where it lists none, `openspec validate --strict --all` and `spec-tools gates`.
|
|
38
38
|
|
|
39
|
-
7. **Hand back.** Say what changed and show the `spec-tools diff` first line. Do not commit, push, or archive; end with the decisions the user has to make.
|
|
39
|
+
7. **Hand back.** Say what changed, in ASD-STE100, and show the `spec-tools diff` first line. Do not commit, push, or archive; end with the decisions the user has to make.
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@kirill.konshin/lint",
|
|
3
3
|
"description": "ESLint + Prettier + lint-staged config and AI agent rules/skills for monorepos",
|
|
4
|
-
"version": "0.6.
|
|
4
|
+
"version": "0.6.5",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"skipGenerate": true,
|
|
7
7
|
"bin": {
|
|
@@ -84,7 +84,6 @@
|
|
|
84
84
|
"@types/eslint-plugin-jsx-a11y": "^6.10.1",
|
|
85
85
|
"@types/eslint-plugin-promise": "^7.3.0",
|
|
86
86
|
"github-markdown-css": "^5.9.0",
|
|
87
|
-
"jsonrepair": "^3.15.0",
|
|
88
87
|
"marked": "^18.0.5",
|
|
89
88
|
"tsdown": "^0.23.0",
|
|
90
89
|
"vitest": "^4.1.10"
|
|
@@ -1,16 +1,17 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: spec-steward
|
|
3
|
-
description: Guard, audit and repair an OpenSpec specification corpus — whether these are the right rules, in the right place, with the right evidence; it proposes spec changes and never fails a build. Use when auditing specs for corpus quality (the whole corpus, a change, or one capability), producing or processing a spec review file (review
|
|
3
|
+
description: Guard, audit and repair an OpenSpec specification corpus — whether these are the right rules, in the right place, with the right evidence; it proposes spec changes and never fails a build. Use when auditing specs for corpus quality (the whole corpus, a change, or one capability), producing or processing a spec review file (the YAML review `.spec-audit/spec-review.yaml`, where the owner answers each item's `decision` with ✅ / ❌ / a comment), aligning rules shared by several repositories, wiring the spec guard (edit hook, AGENTS.md routing, OPSX config) into a repository, or when the repository's spec gate, `spec-tools steward check` or the edit hook reports a finding. Deterministic scripts do the mechanical work; the skill supplies the procedure. It does not audit code conformance; the `spec-verify` skill does. Writing a rule is the `/spec-author` command.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Spec steward
|
|
7
7
|
|
|
8
|
-
Procedures for an OpenSpec corpus.
|
|
8
|
+
Procedures for an OpenSpec corpus. Five things live elsewhere and are not repeated here:
|
|
9
9
|
|
|
10
10
|
- the hard constraints for editing specs — the scoped rule `.agents/rules/openspec.md`
|
|
11
11
|
- where any guidance belongs — `.agents/rules/agent.md`
|
|
12
12
|
- the specification model (REQUIRED versus ⚠️ ADVISORY, Known gaps, the evidence ladder, what is not evidence, how requirements and scenarios are written) — [references/model.md](references/model.md)
|
|
13
|
-
- what the gate checks and emits (citations, binding, markers, size lines, the scenario ratchet, base resolution, exit codes, the coverage report, the evidence
|
|
13
|
+
- what the gate checks and emits (citations, binding, markers, size lines, the scenario ratchet, base resolution, exit codes, the coverage report, the evidence data) — [references/contract.md](references/contract.md)
|
|
14
|
+
- what the audit judges and how sure a finding must be — the judging rules, [`common.md`](../spec-tools/references/rules/common.md) and [`spec-steward.md`](../spec-tools/references/rules/spec-steward.md)
|
|
14
15
|
|
|
15
16
|
`steward` below means `spec-tools steward`, through the `spec-tools` bin `@kirill.konshin/lint` installs; where the bin is absent, `node <the spec-tools skill's folder>/scripts/cli.js steward`. The file is self-contained and needs Node ≥ 20 and git. Every command takes `--root NAME=path` (repeatable, for several repositories) and `--specs <dir>` (default `openspec/specs`).
|
|
16
17
|
|
|
@@ -49,19 +50,23 @@ The exact rule behind each kind is in [contract.md](references/contract.md#kinds
|
|
|
49
50
|
|
|
50
51
|
## Audit
|
|
51
52
|
|
|
52
|
-
Local only, on the owner's request — CI runs this skill's `check` (inside `spec-tools gates`), `coverage` and `evidence`, never the audit. Corpus-wide or affected-set, for corpus quality: placement, REQUIRED versus ⚠️ ADVISORY, evidence. It runs on the engine `spec-verify` runs on — `spec-tools audit --audit spec-steward` — with this skill's [worker contract](references/worker.md) and [sweeps](sweeps.json). Pass the owner's earlier decisions with `--context <file>`. The procedure around it is [references/audit.md](references/audit.md). The output is
|
|
53
|
+
Local only, on the owner's request — CI runs this skill's `check` (inside `spec-tools gates`), `coverage` and `evidence`, never the audit. Corpus-wide or affected-set, for corpus quality: placement, REQUIRED versus ⚠️ ADVISORY, evidence. It runs on the engine `spec-verify` runs on — `spec-tools audit --audit spec-steward` — with this skill's [worker contract](references/worker.md) and [sweeps](sweeps.json). Pass the owner's earlier decisions with `--context <file>`. The procedure around it is [references/audit.md](references/audit.md). The output is the YAML review file `.spec-audit/spec-review.yaml`, in the format of [references/review-format.md](references/review-format.md). Nothing in the specs or the code changes until the owner answers it.
|
|
53
54
|
|
|
54
55
|
## Apply
|
|
55
56
|
|
|
56
|
-
The owner answers each item in the review file
|
|
57
|
+
The owner answers each item in the review file, in its `decision`. [references/apply.md](references/apply.md) processes the answers round by round:
|
|
57
58
|
|
|
58
59
|
- apply ✅
|
|
59
60
|
- respect ❌
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
-
|
|
61
|
+
- examine a comment again
|
|
62
|
+
- audit again what changed
|
|
63
|
+
- add new items
|
|
63
64
|
|
|
64
|
-
`steward review status
|
|
65
|
+
`steward review status <review.yaml>` shows what is open.
|
|
66
|
+
|
|
67
|
+
## Language
|
|
68
|
+
|
|
69
|
+
Write review items, finding details, verdict reasons and messages to the owner in ASD-STE100 ([common rules](../spec-tools/references/rules/common.md#language)).
|
|
65
70
|
|
|
66
71
|
## Align
|
|
67
72
|
|
|
@@ -1,18 +1,18 @@
|
|
|
1
1
|
# Apply a review round
|
|
2
2
|
|
|
3
|
-
The owner has answered a review file ([review-format.md](review-format.md)). Process
|
|
3
|
+
The owner has answered a review file ([review-format.md](review-format.md)). Process each item. Do each one fully, and decide nothing for the owner.
|
|
4
4
|
|
|
5
|
-
1. **Read the
|
|
6
|
-
2. **Accepted (`✅`, `✅ <comment>`).** Apply the proposal,
|
|
7
|
-
- Before you
|
|
8
|
-
- A rule
|
|
9
|
-
- When a change moves guidance to a new place, put it
|
|
10
|
-
- Add `
|
|
11
|
-
3. **Rejected (`❌`, `❌ <comment>`).** Keep the rule as it is, or follow the owner's
|
|
12
|
-
4. **Comment only.**
|
|
13
|
-
5. **Theme with member
|
|
14
|
-
6. **
|
|
15
|
-
7. **Verify.** Run the repository's
|
|
16
|
-
8. **Stop** while
|
|
5
|
+
1. **Read the full file again.** Then run `steward review status <review.yaml>`. It lists the open items, the comments, and the accepted items that are not applied.
|
|
6
|
+
2. **Accepted (`✅`, `✅ <comment>`).** Apply the proposal, changed as the comment tells.
|
|
7
|
+
- Before you change a requirement, read the requirements that name it and that it names, and the code and tests involved. A fix must not break a sibling.
|
|
8
|
+
- A rule that several repositories share changes identically in each. `steward align` shows the drift.
|
|
9
|
+
- When a change moves guidance to a new place, put it where `rules/agent.md` tells, remove each copy, and reference it from its old place.
|
|
10
|
+
- Add `applied: "R<round> — <what changed, with paths>"` to the item. A second application makes `applied` a list.
|
|
11
|
+
3. **Rejected (`❌`, `❌ <comment>`).** Keep the rule as it is, or follow the owner's other instruction. Record what you did in `applied`.
|
|
12
|
+
4. **Comment only.** Examine the item again, and change the proposal or the explanation in place. Move the owner's comment into `previousDecision: "R<n>: <comment>"` and set `decision` to `""`. A comment is never acceptance.
|
|
13
|
+
5. **Theme with member decisions.** Apply the theme's decision to the members with no `decision` of their own. Do each member that has one as its decision tells, as above.
|
|
14
|
+
6. **Audit again what changed.** Run the audit ([audit.md](audit.md)) over the affected set: the capabilities you changed, the capabilities that name them, and the moved guidance. Then run the repository's spec gate (`AGENTS.md` → Checks), or `steward check --base auto` when it names none. Add the new findings with `steward review render <data> --out <review.yaml> --append --round <n>`.
|
|
15
|
+
7. **Verify.** Run the repository's checks, as its `AGENTS.md` lists them, plus `steward review lint` and `steward review verify`. Report each check that failed or that you did not run, with the reason.
|
|
16
|
+
8. **Stop** while an item is open or has an unresolved comment. Report the file, the counts and the decisions that are necessary. The loop ends only when each item is resolved, the accepted changes are in place, the repositories agree on shared rules, and a focused audit finds nothing important.
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
Do not commit unless the repository's instructions let you.
|
|
@@ -1,32 +1,32 @@
|
|
|
1
1
|
# Audit procedure
|
|
2
2
|
|
|
3
|
-
An audit asks
|
|
3
|
+
An audit asks if the corpus holds the correct rules with the correct evidence. Its judging rules are [`common.md`](../../spec-tools/references/rules/common.md) and [`spec-steward.md`](../../spec-tools/references/rules/spec-steward.md). It writes a review file ([review-format.md](review-format.md)). Nothing in the specifications or the code changes during an audit. If the code agrees with the specifications is the `spec-verify` audit's question.
|
|
4
4
|
|
|
5
5
|
## 1. Run the engine
|
|
6
6
|
|
|
7
|
-
Both audits run on one engine, `spec-tools` (the `spec-tools` skill), one repository per run
|
|
7
|
+
Both audits run on one engine, `spec-tools` (the `spec-tools` skill), one repository per run. Wording drift between repositories is [align](../SKILL.md#align). The full sequence, from a detached tree of `HEAD` (`--here` for uncommitted edits):
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
10
|
spec-tools audit --audit spec-steward --context <decisions file>
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
It runs locally, on the owner's request
|
|
13
|
+
It runs locally, on the owner's request. No pipeline runs it. `--context` passes a file of established facts and earlier owner decisions (earlier review answers, the decision log) into each worker's and judge's brief, so that no finding reverses a decision without saying so.
|
|
14
14
|
|
|
15
|
-
It runs these passes
|
|
15
|
+
It runs these passes. Each is also a command of its own:
|
|
16
16
|
|
|
17
|
-
1. `spec-tools scope --audit spec-steward
|
|
18
|
-
2. `spec-tools workers --audit spec-steward
|
|
19
|
-
3. `spec-tools report --audit spec-steward
|
|
20
|
-
4. `spec-tools workers --complete --audit spec-steward
|
|
21
|
-
5. `spec-tools workers --verify --audit spec-steward
|
|
17
|
+
1. `spec-tools scope --audit spec-steward`: the evidence, cut into parts that one reader holds, plus one unit per [sweep](../sweeps.json) (tooling and CI, tests citing the corpus, comments and citations, missing invariants, placement). It writes `.spec-audit/steward/scope.yaml`.
|
|
18
|
+
2. `spec-tools workers --audit spec-steward`: one headless worker per unit, under [worker.md](worker.md). Each writes `.spec-audit/steward/findings/part-<n>-<reader>.yaml`. The tool checks each file and sends its errors back to the worker.
|
|
19
|
+
3. `spec-tools report --audit spec-steward`: the merge. It looks up each quote at its line and drops a finding whose quote is not there. It lists the requirements that no worker judged as short.
|
|
20
|
+
4. `spec-tools workers --complete --audit spec-steward`: the short requirements, judged again. Then the merge again.
|
|
21
|
+
5. `spec-tools workers --verify --audit spec-steward`: one judge per finding, under the steward judge rules. It keeps, revises or drops the finding and gives `judgeConfidence`. Then the last merge joins duplicates (one rule, one problem), offers the findings that share a `themeKey` as a theme, and renders `.spec-audit/spec-review.yaml`.
|
|
22
22
|
|
|
23
|
-
The run's data
|
|
23
|
+
The run's data is `.spec-audit/steward/review-data.yaml`: the themes and findings, the dropped findings with the reason, the requirements judged sound, the short list and the workers' notes.
|
|
24
24
|
|
|
25
25
|
## 2. Consolidate and hand back
|
|
26
26
|
|
|
27
|
-
- **Read
|
|
28
|
-
- **Order the
|
|
29
|
-
- **
|
|
30
|
-
- **Check** with `steward review lint` and `steward review verify --root NAME=path …`.
|
|
27
|
+
- **Read each item yourself.** Make the wording clear and in ASD-STE100, correct what the readers got wrong, and decide each theme: keep it as one item when its members share one solution, split it when they do not.
|
|
28
|
+
- **Order the items** so that a decision that others depend on comes first.
|
|
29
|
+
- **List the requirements judged sound** at the end, in a YAML comment, so that the coverage is visible.
|
|
30
|
+
- **Check** with `steward review lint .spec-audit/spec-review.yaml` and `steward review verify .spec-audit/spec-review.yaml --root NAME=path …`.
|
|
31
31
|
|
|
32
|
-
|
|
32
|
+
Give the file to the owner and stop.
|
|
@@ -55,7 +55,7 @@ There are exactly two markers. Each stands on a line of its own: the Advisory li
|
|
|
55
55
|
|
|
56
56
|
| Marker | Meaning |
|
|
57
57
|
| --- | --- |
|
|
58
|
-
| `**⚠️ Advisory:** <why review is its evidence>` | The requirement is ADVISORY. It keeps its RFC 2119 keywords and is exempt from the scenario ratchet and from `absolute-unproven`. Coverage reports it as `advisory`, and the evidence
|
|
58
|
+
| `**⚠️ Advisory:** <why review is its evidence>` | The requirement is ADVISORY. It keeps its RFC 2119 keywords and is exempt from the scenario ratchet and from `absolute-unproven`. Coverage reports it as `advisory`, and the evidence model classes it `advisory`. Marking an existing requirement Advisory is reported as `weakened` ("newly marked advisory"). It qualifies the whole requirement, so a requirement that mixes a contract with guidance is split. |
|
|
59
59
|
| `**⚠️ Known gap (<tracker>):** <what is missing>` | Behaviour the requirement describes is knowingly absent or incomplete, or a REQUIRED rule lacks the evidence that would catch its failure. `<tracker>` is required text: a ticket key (`PROJ-123`), a tickets-page id, or an open `openspec/changes/<name>` folder. Steward checks only that it is present: any non-empty text inside the parentheses counts and is not validated, so a placeholder counts until it is replaced. In the statement, the line records the gap and exempts nothing, unless it names scenarios of its requirement by title after `exempts`, each in quotes (`exempts scenario 'A'`, `exempts scenarios 'A' and 'B'`; double quotes, curly quotes, `_…_` and `*…*` are read too). Exactly the named scenarios are then exempt from the scenario ratchet. Coverage lists every gap with its tracker. |
|
|
60
60
|
|
|
61
61
|
Retired spellings are `**⚠️ Unenforced:**` and `**⚠️ Known gap:**` without a tracker. Each is reported as `marker-hygiene` and grants no exemption. So is a marker inside a line rather than on a line of its own, and an Advisory or Known gap line inside a scenario. A misspelt marker (wrong case, missing emoji or bold) is reported as `marker-hygiene`, and `--fix` normalises it.
|
|
@@ -132,8 +132,8 @@ An explicit `--base <ref>` compares with the merge base of `<ref>` and `HEAD`. W
|
|
|
132
132
|
| Command | Purpose | Output |
|
|
133
133
|
| --- | --- | --- |
|
|
134
134
|
| `spec-tools steward check [--base auto\|<ref>] [--binds <glob>]… [--file <f>] [--fix] [--strict] [--json]` | The corpus gate, run locally, in CI and by the hook | Findings ([below](#findings)) |
|
|
135
|
-
| `spec-tools steward coverage [--out <file>] [--binds <glob>]…` | The [coverage report](#coverage-report) |
|
|
136
|
-
| `spec-tools steward evidence --json [--binds <glob>]…` | The audits' [evidence model](#evidence-
|
|
135
|
+
| `spec-tools steward coverage [--out <file>] [--binds <glob>]…` | The [coverage report](#coverage-report) | `.spec-audit/coverage.yaml`, and the Markdown at `--out` (default `.spec-audit/coverage.md`) |
|
|
136
|
+
| `spec-tools steward evidence --json [--binds <glob>]…` | The audits' [evidence model](#evidence-model), which `spec-tools scope` writes to `.spec-audit/evidence.yaml` and cuts into each audit's evidence files | JSON on stdout |
|
|
137
137
|
| `spec-tools steward review`, `align`, `wire`, `hook` | Review files, cross-repository drift, wiring, the edit hook | As each command documents |
|
|
138
138
|
|
|
139
139
|
`spec-tools steward` runs through the package's one `bin`, `spec-tools`. A repository calls it from its root scripts and never imports steward's modules: the package's `exports` map does not expose them, so the CLI and its JSON are the interface.
|
|
@@ -155,7 +155,7 @@ Exit 2 on an empty listing keeps a broken checkout from passing a scan that read
|
|
|
155
155
|
|
|
156
156
|
## Coverage report
|
|
157
157
|
|
|
158
|
-
`spec-tools steward coverage` writes a Markdown report that is MDX-safe: `<`, `>`, `{`, `}` and `|` are escaped. It gates nothing: it exits 0, or 2 on an environment error. Statuses come from [bindings](#binding) and [markers](#markers) only. A pointer never changes a status.
|
|
158
|
+
`spec-tools steward coverage` writes the coverage as data, `.spec-audit/coverage.yaml` (totals, each requirement and scenario with its status, bindings and pointers, the known gaps, the retired markers), and as a Markdown report for people that is MDX-safe: `<`, `>`, `{`, `}` and `|` are escaped. It gates nothing: it exits 0, or 2 on an environment error. Statuses come from [bindings](#binding) and [markers](#markers) only. A pointer never changes a status.
|
|
159
159
|
|
|
160
160
|
- **Headline.** Requirements and capabilities; Advisory requirements; Known gaps; REQUIRED requirements no test binds; REQUIRED scenarios no test binds; retired markers.
|
|
161
161
|
- **Per capability.** One table: `| Requirement / Scenario | Spec | Bound by | Pointers | Status |`.
|
|
@@ -164,9 +164,9 @@ Exit 2 on an empty listing keeps a broken checkout from passing a scan that read
|
|
|
164
164
|
- **Pointers.** The code and document citations of the requirement and its scenarios.
|
|
165
165
|
- **Closing sections.** "Known gaps" (tracker, requirement, text) and "Retired markers".
|
|
166
166
|
|
|
167
|
-
## Evidence
|
|
167
|
+
## Evidence model
|
|
168
168
|
|
|
169
|
-
`spec-tools steward evidence --json`
|
|
169
|
+
`spec-tools steward evidence --json` prints the evidence an audit reads, one entry per requirement. `spec-tools scope` writes the same model as `.spec-audit/evidence.yaml`:
|
|
170
170
|
|
|
171
171
|
```
|
|
172
172
|
{ version: 1, root, requirements: [{ id, capability, file, line, end, block, class: 'required'|'advisory',
|
|
@@ -21,7 +21,9 @@ If you cannot name a failure, the rule is not a REQUIRED contract.
|
|
|
21
21
|
| Kind | Meaning | Evidence |
|
|
22
22
|
| --- | --- | --- |
|
|
23
23
|
| **REQUIRED** | An invariant whose violation is a real defect — behavioural, API, compatibility, security, data. | The cheapest rung that catches the failure (ladder below). The requirement names that failure and that rung; a test is one rung, not the only one. |
|
|
24
|
-
| **⚠️ ADVISORY** |
|
|
24
|
+
| **⚠️ ADVISORY** | Guidance whose violation has no production effect, or that cannot reasonably be proven mechanically: design direction, implementation detail kept in the specification, development-only and local-only behaviour, demo agents, internal CI and developer tools, prompt wording. | Review judgement. Never a gate, never a reason to write a test. |
|
|
25
|
+
|
|
26
|
+
A REQUIRED requirement is one whose violation has a production effect, as the [common judging rules](../../spec-tools/references/rules/common.md#production-effect) define it: a broken flow, a contract or invariant violation, a sensitive-data leak, or a performance problem.
|
|
25
27
|
|
|
26
28
|
A REQUIRED requirement stating an absolute names its exceptions, or has none; one with neither is listed for the owner's review.
|
|
27
29
|
|
|
@@ -64,6 +66,16 @@ A scenario adds precision — one concrete case, preferably Given/When/Then —
|
|
|
64
66
|
|
|
65
67
|
New and edited text uses EARS for requirement statements — When <trigger>, the <system> SHALL <response>; While <state>, the <system> SHALL …; If <condition>, then the <system> SHALL …; or The <system> SHALL … — and Given/When/Then for scenarios, with Given only where a precondition matters. The RFC 2119 keyword stays in capitals: OpenSpec's strict validation fails a requirement whose body has no SHALL or MUST. Untouched text is not rewritten for syntax alone.
|
|
66
68
|
|
|
69
|
+
New and edited text is also written in a light subset of ASD-STE100:
|
|
70
|
+
|
|
71
|
+
- One obligation per sentence.
|
|
72
|
+
- At most 25 words per sentence.
|
|
73
|
+
- One term for one concept, and the same term each time.
|
|
74
|
+
- Active voice: the system or the component is the subject.
|
|
75
|
+
- No vague words in a REQUIRED statement (thin, safe, lean, at parity, needed): name the measurable condition.
|
|
76
|
+
|
|
77
|
+
Untouched text is not rewritten for this subset alone.
|
|
78
|
+
|
|
67
79
|
## Changing a rule
|
|
68
80
|
|
|
69
81
|
- Never weaken, narrow, delete or reclassify a rule to make code pass. A demonstrated divergence is decided by the owner: the code is wrong, or the rule is — and a wrong rule is changed openly, through the repository's change process.
|
|
@@ -1,61 +1,87 @@
|
|
|
1
1
|
# Review file format
|
|
2
2
|
|
|
3
|
-
An audit
|
|
3
|
+
An audit gives its findings to the owner as one YAML review file, `.spec-audit/spec-review.yaml`. The owner answers each item in place, and the file stays as the record across rounds. `steward review render` writes the items. `steward review lint` checks the shape. `steward review verify --root NAME=path` checks that each quote is at its location. `steward review status` counts the answers.
|
|
4
4
|
|
|
5
5
|
## Items
|
|
6
6
|
|
|
7
|
-
Each item
|
|
8
|
-
|
|
9
|
-
```
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
7
|
+
Each item is one top-level block, keyed by its id, `R-NNN`. An item about one rule has `rule`:
|
|
8
|
+
|
|
9
|
+
```yaml
|
|
10
|
+
R-001:
|
|
11
|
+
title: 'Implementation detail in a REQUIRED rule'
|
|
12
|
+
area: 'system-overview/logging'
|
|
13
|
+
layer: 'SPEC-ADVISORY'
|
|
14
|
+
criteria:
|
|
15
|
+
- 8
|
|
16
|
+
confidence: 85
|
|
17
|
+
rule:
|
|
18
|
+
text: |-
|
|
19
|
+
The server SHALL promote accountId, rcAccountId, rcRequestId and rcUserId…
|
|
20
|
+
location: 'SHOP openspec/specs/system-overview/logging/spec.md:47'
|
|
21
|
+
wrong: |
|
|
22
|
+
The rule names the exact log fields. A field change breaks the rule with no effect in production.
|
|
23
|
+
proposal: |
|
|
24
|
+
Reclassify → OpenSpec ⚠️ ADVISORY — mark the rule Advisory, or move the field list to the logger's scoped rule.
|
|
25
|
+
evidence:
|
|
26
|
+
- 'src/logger.ts:12'
|
|
27
|
+
decision: ''
|
|
17
28
|
```
|
|
18
29
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
30
|
+
An item about several rules with one problem and one solution is a theme. It has `theme` and a `rules` list:
|
|
31
|
+
|
|
32
|
+
```yaml
|
|
33
|
+
R-002:
|
|
34
|
+
theme: 'Development-only behaviour held as REQUIRED'
|
|
35
|
+
layer: 'SPEC-ADVISORY'
|
|
36
|
+
confidence: 75
|
|
37
|
+
rules:
|
|
38
|
+
- text: |-
|
|
39
|
+
WHEN the dev server starts THEN it seeds the demo account
|
|
40
|
+
location: 'SHOP openspec/specs/apps/runtime/spec.md:88'
|
|
41
|
+
- text: |-
|
|
42
|
+
The local database SHALL reset on each start
|
|
43
|
+
location: 'SHOP openspec/specs/apps/db/spec.md:12'
|
|
44
|
+
note: 'the reset is also in the README'
|
|
45
|
+
wrong: |
|
|
46
|
+
Both rules apply only on a developer machine. A violation has no effect in production.
|
|
47
|
+
proposal: |
|
|
48
|
+
Reclassify → OpenSpec ⚠️ ADVISORY — mark both rules Advisory.
|
|
49
|
+
decision: ''
|
|
50
|
+
```
|
|
27
51
|
|
|
28
|
-
|
|
29
|
-
### R-012 · DELETE · Theme: Scenarios that restate their requirement · 19, 20
|
|
52
|
+
The keys:
|
|
30
53
|
|
|
31
|
-
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
54
|
+
- `title` or `theme`: one line.
|
|
55
|
+
- `area`: the capability or the sweep that found the item.
|
|
56
|
+
- `layer`: the owning layer that the item proposes ([worker.md](worker.md)).
|
|
57
|
+
- `criteria`: the item numbers of the [steward rules](../../spec-tools/references/rules/spec-steward.md#items-to-report).
|
|
58
|
+
- `confidence`: an integer from 0 to 100, the judge's confidence, else the reader's. A theme takes the lowest confidence of its rules. Items at 70 or lower come after the comment `# Low confidence — 70% or lower`.
|
|
59
|
+
- `needsHumanIntent: true`: the intent is not clear, and the proposal gives options.
|
|
60
|
+
- `rule` / `rules[]`: `text` is the original text, so that the owner knows the rule without opening the file; `…` leaves text out. `location` is `NAME path:line`, where NAME is the repository and path is relative to it. `note` is the audit's remark about one member of a theme. `decision` on a member is the owner's answer for that member only.
|
|
61
|
+
- `wrong`: the problem, in ASD-STE100.
|
|
62
|
+
- `proposal`: starts with the target notation of the [steward rules](../../spec-tools/references/rules/spec-steward.md) (`Keep →`, `Reclassify →`, `Move →`, `Replace →`, `Merge →`, `Rewrite →`, `Delete →`).
|
|
63
|
+
- `evidence`, `crossRefs`, `reversesPastDecision`: present only when they have content.
|
|
64
|
+
- `decision`: the owner's answer. It starts empty.
|
|
39
65
|
|
|
40
|
-
|
|
66
|
+
A rule is in one item only. A second problem with the same rule is its own item, with a `crossRefs` entry to the first.
|
|
41
67
|
|
|
42
|
-
##
|
|
68
|
+
## Decisions
|
|
43
69
|
|
|
44
|
-
| The owner writes | Meaning
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
|
|
|
49
|
-
|
|
|
50
|
-
|
|
|
51
|
-
|
|
|
70
|
+
| The owner writes in `decision` | Meaning |
|
|
71
|
+
| ------------------------------ | -------------------------------------------------------------------------- |
|
|
72
|
+
| `""` or `"⬜"` | open: not answered |
|
|
73
|
+
| `"✅"` | accept the proposal |
|
|
74
|
+
| `"✅ <comment>"` | accept, with the instruction in the comment |
|
|
75
|
+
| `"<comment>"` | not accepted and not rejected: read the comment and examine the item again |
|
|
76
|
+
| `"❌"` | reject: keep the rule as it is |
|
|
77
|
+
| `"❌ <comment>"` | reject, and do what the comment tells |
|
|
52
78
|
|
|
53
|
-
|
|
79
|
+
The owner can answer one member of a theme with a `decision` on that member, in the same form. No answer is never approval. An empty `decision` is open.
|
|
54
80
|
|
|
55
81
|
## After a round
|
|
56
82
|
|
|
57
|
-
Processing a round ([apply.md](apply.md)) adds to the file and never
|
|
83
|
+
Processing a round ([apply.md](apply.md)) adds to the file and never removes from it:
|
|
58
84
|
|
|
59
|
-
- **An applied item** gets `
|
|
60
|
-
- **A comment-only item** is
|
|
61
|
-
- **New findings**
|
|
85
|
+
- **An applied item** gets `applied: "R<round> — <what changed, with paths>"`.
|
|
86
|
+
- **A comment-only item** is examined again. The comment moves into `previousDecision`, the proposal changes in place, and `decision` becomes `""` again.
|
|
87
|
+
- **New findings** come after the comment `# Round <n> — new`, numbered after the last item, so that the resolved items stay as the record.
|
|
@@ -1,67 +1,65 @@
|
|
|
1
1
|
# Audit worker
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
This file tells how one worker of the corpus-quality audit runs. `spec-tools workers --audit spec-steward` starts one worker per unit of the scope. Its brief names the unit, the files to read and the findings file to write. The procedure around the workers is [audit.md](audit.md).
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
What to judge is in the judging rules: [`common.md`](../../spec-tools/references/rules/common.md), then [`spec-steward.md`](../../spec-tools/references/rules/spec-steward.md). How a specification is written is [model.md](model.md). Where guidance belongs is the placement guide, `rules/agent.md` of this package. Read them before you judge.
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
- Specification quality first. An ordinary implementation bug is out of scope unless it reveals the rule is unrealistic, unclear, outdated, ambiguous, or contradicted by what the tests actually assert. Whether code conforms is the `spec-verify` audit's.
|
|
10
|
-
- Comments, descriptions, prompt text and citations are never evidence that code complies, and never a violation by themselves. Judge executable behaviour and structure.
|
|
11
|
-
- For every rule you would keep REQUIRED, ask what meaningful failure its verification catches. Never propose a test merely because a rule lacks one, and never a test of incidental text or structure.
|
|
12
|
-
- Internal component and API contracts are legitimate specification. The line is contract versus implementation or process, not product versus engineering.
|
|
13
|
-
- Do not decide intent silently: when the meaning is ambiguous, describe the readings, propose options and set `needsHumanIntent`.
|
|
14
|
-
- No style preferences: every finding states a concrete problem.
|
|
15
|
-
- A proposal that reverses an earlier owner decision says so, in `reversesPastDecision`.
|
|
16
|
-
- Quote the original text verbatim, from the line you name; open the file to confirm before writing it.
|
|
17
|
-
|
|
18
|
-
The audit is read-only: you write your findings file and nothing else. Your tools are Read, Glob, Grep, Write and the read-only git commands; anything else is denied and only costs a turn.
|
|
7
|
+
The audit is read-only. You write your findings file and nothing else. Your tools are Read, Glob, Grep, Write and the read-only git commands. Other commands are denied and only cost a turn.
|
|
19
8
|
|
|
20
9
|
## Your unit
|
|
21
10
|
|
|
22
|
-
- **A part** is a set of capabilities
|
|
23
|
-
- **A sweep** looks across the corpus from one angle
|
|
11
|
+
- **A part** is a set of capabilities. Each capability has its evidence file under `.spec-audit/parts/`: each requirement with its line, the tests bound to it, where its terms occur, and its related requirements. Read the evidence files and the specifications they name. Read code and tests where a verdict depends on them. Each requirement of the part is in a finding (on it or on one of its scenarios) or in `sound`. Each requirement you judged is in `judged`.
|
|
12
|
+
- **A sweep** looks across the corpus from one angle. Its focus names the items to report. Read the files the brief lists. `judged` and `sound` can stay empty.
|
|
24
13
|
|
|
25
14
|
## The findings file
|
|
26
15
|
|
|
27
|
-
|
|
16
|
+
Write one finding for each rule and problem. A rule can be a requirement, a scenario, a sentence, or, outside the specifications, a section or line of `AGENTS.md`, a skill, a configuration, a script or a CI job. Write the file with the Write tool, at the path the brief names:
|
|
28
17
|
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
"billing/refunds#requirement-a-refund-names-its-order"
|
|
56
|
-
],
|
|
57
|
-
"notes": []
|
|
58
|
-
}
|
|
18
|
+
```yaml
|
|
19
|
+
findings:
|
|
20
|
+
- file: 'openspec/specs/billing/refunds/spec.md'
|
|
21
|
+
line: 12
|
|
22
|
+
quote: |-
|
|
23
|
+
The service SHALL apply a refund at most once per request id
|
|
24
|
+
ruleName: 'Refunds are idempotent'
|
|
25
|
+
criteria:
|
|
26
|
+
- 6
|
|
27
|
+
- 7
|
|
28
|
+
layer: 'SPEC-ADVISORY'
|
|
29
|
+
whatsWrong: |
|
|
30
|
+
The rule names no failure that a test can catch. Its tests assert only the text of the log line.
|
|
31
|
+
proposed: |
|
|
32
|
+
Reclassify → OpenSpec ⚠️ ADVISORY — mark the requirement Advisory and remove the log-text test.
|
|
33
|
+
evidence:
|
|
34
|
+
- 'src/refunds.ts:40'
|
|
35
|
+
themeKey: 'unverifiable-absolute'
|
|
36
|
+
needsHumanIntent: false
|
|
37
|
+
readerConfidence: 75
|
|
38
|
+
sound:
|
|
39
|
+
- 'billing/refunds#requirement-a-refund-names-its-order'
|
|
40
|
+
judged:
|
|
41
|
+
- 'billing/refunds#requirement-refunds-are-idempotent'
|
|
42
|
+
- 'billing/refunds#requirement-a-refund-names-its-order'
|
|
43
|
+
notes: []
|
|
59
44
|
```
|
|
60
45
|
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
- `
|
|
64
|
-
- `
|
|
65
|
-
- `
|
|
46
|
+
How to write each key:
|
|
47
|
+
|
|
48
|
+
- `file`, `ruleName`, `layer`, `themeKey`, `reversesPastDecision`: double-quoted strings. `file` is relative to the repository.
|
|
49
|
+
- `line`, `criteria[]`, `readerConfidence`: plain integers. `line` is where the quoted text starts. `readerConfidence` is from 0 to 100 ([common rules](../../spec-tools/references/rules/common.md#confidence)).
|
|
50
|
+
- `quote`: a stripped literal block (`|-`), copied from that line, at most 300 characters. Use `…` where you leave text out. The merge looks for the quote at its line and drops a finding whose quote is not there.
|
|
51
|
+
- `whatsWrong`, `proposed`: literal blocks (`|`), in ASD-STE100.
|
|
52
|
+
- `evidence`, `crossRefs`, `sound`, `judged`, `notes`: block lists of double-quoted strings. Address a requirement as `capability#requirement-slug`.
|
|
53
|
+
- `needsHumanIntent`: `true` or `false`.
|
|
54
|
+
|
|
55
|
+
What each key holds:
|
|
56
|
+
|
|
57
|
+
- `criteria`: the item numbers of the [steward rules](../../spec-tools/references/rules/spec-steward.md#items-to-report).
|
|
58
|
+
- `proposed`: starts with the target notation (`Keep →`, `Reclassify →`, `Move →`, `Replace →`, `Merge →`, `Rewrite →`, `Delete →`) and tells exactly what to do. A rewrite gives the new text. A move names the target file.
|
|
59
|
+
- `layer`: the layer you propose as the owner: `SPEC-REQUIRED`, `SPEC-ADVISORY`, `DESIGN`, `AGENTS`, `RULE`, `SKILL`, `COMMAND`, `OPSX-CONFIG`, `README`, `TOOLING`, `TEST`, `CI`, `MERGE`, `REWRITE` or `DELETE`.
|
|
60
|
+
- `themeKey`: a kebab-case name for the problem under the finding. The merge groups the findings that share a key into a theme. Use a key that another finding uses for the same problem, for example `impl-detail-in-spec`, `advisory-not-required`, `restating-scenario`, `tooling-should-enforce`, `duplicate-rule` or `stale-rule`. When no key fits, make a new one.
|
|
61
|
+
- `reversesPastDecision`: the owner decision that the proposal reverses. Leave it out when there is none.
|
|
62
|
+
- `sound`: the requirements that you judged correct, in the correct place, and with sufficient evidence.
|
|
63
|
+
- `judged`: each requirement that you judged, with or without a finding. A requirement of your part that is not in `judged` goes to a completion worker.
|
|
66
64
|
|
|
67
|
-
|
|
65
|
+
When you stop, the tool checks the file. If the file has errors, the tool sends them back to you, and you correct the file. Then return one line: the path and your count of findings.
|