spec-controller 0.1.270 → 0.1.272
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -19
- package/dist/cli.js +2 -2
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
**One command, `balance`, that checks the behaviour your scenarios name against the evidence that actually ran.**
|
|
4
4
|
|
|
5
|
-
You give it your Gherkin scenarios and the reports your tools already write: a Vitest report, a Cucumber report, and your CI run's own record of what its steps did. It prints one report
|
|
5
|
+
You give it your Gherkin scenarios and the reports your tools already write: a Vitest report, a Cucumber report, and your CI run's own record of what its steps did. It prints one report. The report shows which scenarios have living evidence, which do not, and which evidence ran that no scenario asked for.
|
|
6
6
|
|
|
7
|
-
It runs nothing itself
|
|
7
|
+
It runs nothing itself. It only reads what your pipeline wrote.
|
|
8
8
|
|
|
9
|
-
Requires Node ≥ 22. ESM only.
|
|
9
|
+
Requires Node ≥ 22.12. ESM only.
|
|
10
10
|
|
|
11
11
|
## Install
|
|
12
12
|
|
|
@@ -22,15 +22,15 @@ npx spec-controller balance --help
|
|
|
22
22
|
|
|
23
23
|
## Your first run
|
|
24
24
|
|
|
25
|
-
Point it at your scenarios and
|
|
25
|
+
Point it at your scenarios and give it no test reports:
|
|
26
26
|
|
|
27
27
|
```
|
|
28
28
|
spec-controller balance --features ./features
|
|
29
29
|
```
|
|
30
30
|
|
|
31
|
-
Every scenario gets a Claim Reconciliation, whether or not any evidence exists for it. You have supplied no reports, so every scenario that names
|
|
31
|
+
Every scenario gets a Claim Reconciliation, whether or not any evidence exists for it. You have supplied no test reports, so every scenario that names a test reads **missing**, and the run is out-of-balance. (`balance` still reads your CI workflows in `.github/workflows` and your `package.json` by default, so a scenario a CI step answers can balance.) That is by design, not a fault in your repository. The tool starts from what you specified, not from what a tool happened to find. No input makes saying nothing the right answer.
|
|
32
32
|
|
|
33
|
-
**`balance` fails your build by default.** An out-of-balance run exits `1`, so the run above fails any pipeline you put it in. Two tools it sits beside do the opposite
|
|
33
|
+
**`balance` fails your build by default.** An out-of-balance run exits `1`, so the run above fails any pipeline you put it in. Two tools it sits beside do the opposite. Stryker's `thresholds.break` defaults to `null` (*"never let your build fail"*). The Sonar scanner exits `0` whatever its quality gate says, unless you set `sonar.qualitygate.wait`. Both make you opt in to gating. `balance` makes you opt out.
|
|
34
34
|
|
|
35
35
|
### Report first, gate later
|
|
36
36
|
|
|
@@ -40,12 +40,12 @@ To add it to an existing pipeline without failing that pipeline on day one, add
|
|
|
40
40
|
spec-controller balance --features ./features --vitest ./vitest-report.json --exit-zero
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
-
The report is the one a run without the flag prints: the same Claim Reconciliations, the same counts and the same run verdict. Only the exit status changes:
|
|
43
|
+
The report is the same one a run without the flag prints: the same Claim Reconciliations, the same counts and the same run verdict. Only the exit status changes:
|
|
44
44
|
|
|
45
45
|
- Out-of-balance (`1`) exits `0`.
|
|
46
|
-
- Usage (`2`), not reconciled (`3`) and an unexpected error (`4`) still fail. Each means the run could not be relied on, or reached no verdict
|
|
46
|
+
- Usage (`2`), not reconciled (`3`) and an unexpected error (`4`) still fail. Each one means the run could not be relied on, or reached no verdict. A flag that eases a verdict does not ease that.
|
|
47
47
|
|
|
48
|
-
Work through the out-of-balance rows, then delete `--exit-zero`. The gate is then on. Do not wrap the step in `continue-on-error` or `|| true` instead. Those swallow the codes that mean the run could not be trusted
|
|
48
|
+
Work through the out-of-balance rows, then delete `--exit-zero`. The gate is then on. Do not wrap the step in `continue-on-error` or `|| true` instead. Those swallow the codes that mean the run could not be trusted. They also leave nothing in the command to show that the gate is off.
|
|
49
49
|
|
|
50
50
|
### Give it the evidence
|
|
51
51
|
|
|
@@ -62,17 +62,17 @@ A scenario tagged `@unit` names an obligation: a unit test must exist and pass.
|
|
|
62
62
|
|
|
63
63
|
| Exit | What it says |
|
|
64
64
|
| --- | --- |
|
|
65
|
-
| `0` | balanced — every obligation has evidence, and
|
|
65
|
+
| `0` | balanced — every obligation has evidence, and that evidence holds |
|
|
66
66
|
| `1` | out-of-balance — the scenarios and the evidence disagree, and the report says where |
|
|
67
67
|
| `2` | usage — the invocation was wrong, or an input could not be read |
|
|
68
|
-
| `3` | not reconciled — the run could not be relied on
|
|
69
|
-
| `4` | unexpected error — the run stops on an error no refusal names, and gives no verdict
|
|
68
|
+
| `3` | not reconciled — the run could not be relied on. Its run verdict says why: `unsound`, when some evidence never ran; `claim-code-errors`, when a claim code is misfiled (the report lists it under `### Claim code errors`); or `nothing-to-reconcile`, when there is no Claim Reconciliation at all |
|
|
69
|
+
| `4` | unexpected error — the run stops on an error that no refusal names, and gives no verdict. One line on stderr names the error |
|
|
70
70
|
|
|
71
|
-
The whole report
|
|
71
|
+
The tool writes the whole report before it sets the exit code, so a failing run still tells you why.
|
|
72
72
|
|
|
73
|
-
**Design your pipeline for new codes.** The list is open at the top
|
|
73
|
+
**Design your pipeline for new codes.** The list is open at the top. A state that is none of the above gets a **new** code, not the nearest existing one. The highest is `4`. **Treat an unrecognised non-zero code as a failure you have not seen before**, not as a synonym for `1`. A new code does not affect a pipeline that matches `1` exactly. A pipeline that treats any non-zero code as a failed build is correct, only less precise about why.
|
|
74
74
|
|
|
75
|
-
**Node's default exit code is never one of these.** A run that cannot reconcile ends on one of the codes above
|
|
75
|
+
**Node's default exit code is never one of these.** A run that cannot reconcile ends on one of the codes above. It never ends by crashing out of the process. A crash that exits `1` would look, by its status, exactly like an honest disagreement.
|
|
76
76
|
|
|
77
77
|
**An input it cannot read is a usage error (`2`)**, never a verdict about your code. This covers:
|
|
78
78
|
|
|
@@ -81,15 +81,15 @@ The whole report is written before the exit code is set, so a failing run still
|
|
|
81
81
|
- an unparseable report, or a report that parses but is not in the shape its runner writes;
|
|
82
82
|
- a corpus the Gherkin parser rejects.
|
|
83
83
|
|
|
84
|
-
The refusal names the input, the path, and what was wrong with it. A well-formed report
|
|
84
|
+
The refusal names the input, the path, and what was wrong with it. A well-formed report with nothing in it, such as a Vitest run that matched no test file, is read, not refused.
|
|
85
85
|
|
|
86
|
-
**An output location it cannot write is also `2
|
|
86
|
+
**An output location it cannot write is also `2`.** This covers a `--format` path that is read-only, is a directory, is under a regular file, or is in a directory that does not exist. The tool refuses it before it prints any report, and names the location and the operating system's reason.
|
|
87
87
|
|
|
88
88
|
`--exit-zero` changes `1` to `0` and leaves every other code alone (see *Report first, gate later*).
|
|
89
89
|
|
|
90
90
|
### `allocate`
|
|
91
91
|
|
|
92
|
-
`spec-controller allocate` prints the next claim code for a feature code. It reaches no verdict
|
|
92
|
+
`spec-controller allocate` prints the next claim code for a feature code. It reaches no verdict. It exits `0` when it answers, and `2` for a usage error or an unknown feature code. A file or directory it cannot read, directly or through git, is a usage error. The tool refuses it, names the path and the reason, and gives no code.
|
|
93
93
|
|
|
94
94
|
## On CI
|
|
95
95
|
|
|
@@ -100,7 +100,7 @@ spec-controller balance --features ./features \
|
|
|
100
100
|
--format json:reconciliation.json
|
|
101
101
|
```
|
|
102
102
|
|
|
103
|
-
|
|
103
|
+
You can repeat `--format`, and it takes an optional `:<path>`. On GitHub Actions, writing Markdown to `$GITHUB_STEP_SUMMARY` puts the reconciliation on the run's summary page. That page is the channel GitHub provides for a table. Writing there needs no action, no plugin and no permission.
|
|
104
104
|
|
|
105
105
|
`spec-controller balance --help` lists every flag, one line each.
|
|
106
106
|
|
package/dist/cli.js
CHANGED
|
@@ -30531,7 +30531,7 @@ var init_refusal = __esm({
|
|
|
30531
30531
|
import { closeSync, constants, existsSync as existsSync2, fstatSync, lstatSync, openSync, readlinkSync, realpathSync, statSync as statSync2 } from "node:fs";
|
|
30532
30532
|
import { dirname, resolve, sep } from "node:path";
|
|
30533
30533
|
function unwritableRefusal(format, path, err) {
|
|
30534
|
-
const reason = err
|
|
30534
|
+
const reason = err.message;
|
|
30535
30535
|
return refusal("report-not-written" /* ReportNotWritten */, { format, path, reason });
|
|
30536
30536
|
}
|
|
30537
30537
|
function probeDestination(path) {
|
|
@@ -35456,7 +35456,7 @@ function corpusDirOf(args, defaultCorpus) {
|
|
|
35456
35456
|
try {
|
|
35457
35457
|
return join4(process.cwd(), defaultCorpus);
|
|
35458
35458
|
} catch (err) {
|
|
35459
|
-
return { path: defaultCorpus, reason: err
|
|
35459
|
+
return { path: defaultCorpus, reason: err.message };
|
|
35460
35460
|
}
|
|
35461
35461
|
}
|
|
35462
35462
|
var init_corpusDir = __esm({
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "spec-controller",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.272",
|
|
4
4
|
"description": "Reconcile the behaviour your scenarios name against the evidence that actually ran — the deterministic `balance` CLI.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"reconciliation",
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
"fast-xml-parser": "^5.11.2",
|
|
27
27
|
"ts-json-schema-generator": "^2.9.0",
|
|
28
28
|
"yaml": "^2.9.0",
|
|
29
|
-
"@3f-consulting/spec-controller-core": "0.1.
|
|
29
|
+
"@3f-consulting/spec-controller-core": "0.1.272"
|
|
30
30
|
},
|
|
31
31
|
"publishConfig": {
|
|
32
32
|
"access": "public"
|