spec-controller 0.1.269 → 0.1.271
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -19
- package/dist/cli.js +24 -24
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
**One command, `balance`, that checks the behaviour your scenarios name against the evidence that actually ran.**
|
|
4
4
|
|
|
5
|
-
You give it your Gherkin scenarios and the reports your tools already write: a Vitest report, a Cucumber report, and your CI run's own record of what its steps did. It prints one report
|
|
5
|
+
You give it your Gherkin scenarios and the reports your tools already write: a Vitest report, a Cucumber report, and your CI run's own record of what its steps did. It prints one report. The report shows which scenarios have living evidence, which do not, and which evidence ran that no scenario asked for.
|
|
6
6
|
|
|
7
|
-
It runs nothing itself
|
|
7
|
+
It runs nothing itself. It only reads what your pipeline wrote.
|
|
8
8
|
|
|
9
|
-
Requires Node ≥ 22. ESM only.
|
|
9
|
+
Requires Node ≥ 22.12. ESM only.
|
|
10
10
|
|
|
11
11
|
## Install
|
|
12
12
|
|
|
@@ -22,15 +22,15 @@ npx spec-controller balance --help
|
|
|
22
22
|
|
|
23
23
|
## Your first run
|
|
24
24
|
|
|
25
|
-
Point it at your scenarios and
|
|
25
|
+
Point it at your scenarios and give it no test reports:
|
|
26
26
|
|
|
27
27
|
```
|
|
28
28
|
spec-controller balance --features ./features
|
|
29
29
|
```
|
|
30
30
|
|
|
31
|
-
Every scenario gets a Claim Reconciliation, whether or not any evidence exists for it. You have supplied no reports, so every scenario that names
|
|
31
|
+
Every scenario gets a Claim Reconciliation, whether or not any evidence exists for it. You have supplied no test reports, so every scenario that names a test reads **missing**, and the run is out-of-balance. (`balance` still reads your CI workflows in `.github/workflows` and your `package.json` by default, so a scenario a CI step answers can balance.) That is by design, not a fault in your repository. The tool starts from what you specified, not from what a tool happened to find. No input makes saying nothing the right answer.
|
|
32
32
|
|
|
33
|
-
**`balance` fails your build by default.** An out-of-balance run exits `1`, so the run above fails any pipeline you put it in. Two tools it sits beside do the opposite
|
|
33
|
+
**`balance` fails your build by default.** An out-of-balance run exits `1`, so the run above fails any pipeline you put it in. Two tools it sits beside do the opposite. Stryker's `thresholds.break` defaults to `null` (*"never let your build fail"*). The Sonar scanner exits `0` whatever its quality gate says, unless you set `sonar.qualitygate.wait`. Both make you opt in to gating. `balance` makes you opt out.
|
|
34
34
|
|
|
35
35
|
### Report first, gate later
|
|
36
36
|
|
|
@@ -40,12 +40,12 @@ To add it to an existing pipeline without failing that pipeline on day one, add
|
|
|
40
40
|
spec-controller balance --features ./features --vitest ./vitest-report.json --exit-zero
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
-
The report is the one a run without the flag prints: the same Claim Reconciliations, the same counts and the same run verdict. Only the exit status changes:
|
|
43
|
+
The report is the same one a run without the flag prints: the same Claim Reconciliations, the same counts and the same run verdict. Only the exit status changes:
|
|
44
44
|
|
|
45
45
|
- Out-of-balance (`1`) exits `0`.
|
|
46
|
-
- Usage (`2`), not reconciled (`3`) and an unexpected error (`4`) still fail. Each means the run could not be relied on, or reached no verdict
|
|
46
|
+
- Usage (`2`), not reconciled (`3`) and an unexpected error (`4`) still fail. Each one means the run could not be relied on, or reached no verdict. A flag that eases a verdict does not ease that.
|
|
47
47
|
|
|
48
|
-
Work through the out-of-balance rows, then delete `--exit-zero`. The gate is then on. Do not wrap the step in `continue-on-error` or `|| true` instead. Those swallow the codes that mean the run could not be trusted
|
|
48
|
+
Work through the out-of-balance rows, then delete `--exit-zero`. The gate is then on. Do not wrap the step in `continue-on-error` or `|| true` instead. Those swallow the codes that mean the run could not be trusted. They also leave nothing in the command to show that the gate is off.
|
|
49
49
|
|
|
50
50
|
### Give it the evidence
|
|
51
51
|
|
|
@@ -62,17 +62,17 @@ A scenario tagged `@unit` names an obligation: a unit test must exist and pass.
|
|
|
62
62
|
|
|
63
63
|
| Exit | What it says |
|
|
64
64
|
| --- | --- |
|
|
65
|
-
| `0` | balanced — every obligation has evidence, and
|
|
65
|
+
| `0` | balanced — every obligation has evidence, and that evidence holds |
|
|
66
66
|
| `1` | out-of-balance — the scenarios and the evidence disagree, and the report says where |
|
|
67
67
|
| `2` | usage — the invocation was wrong, or an input could not be read |
|
|
68
|
-
| `3` | not reconciled — the run could not be relied on
|
|
69
|
-
| `4` | unexpected error — the run stops on an error no refusal names, and gives no verdict
|
|
68
|
+
| `3` | not reconciled — the run could not be relied on. Its run verdict says why: `unsound`, when some evidence never ran; `claim-code-errors`, when a claim code is misfiled (the report lists it under `### Claim code errors`); or `nothing-to-reconcile`, when there is no Claim Reconciliation at all |
|
|
69
|
+
| `4` | unexpected error — the run stops on an error that no refusal names, and gives no verdict. One line on stderr names the error |
|
|
70
70
|
|
|
71
|
-
The whole report
|
|
71
|
+
The tool writes the whole report before it sets the exit code, so a failing run still tells you why.
|
|
72
72
|
|
|
73
|
-
**Design your pipeline for new codes.** The list is open at the top
|
|
73
|
+
**Design your pipeline for new codes.** The list is open at the top. A state that is none of the above gets a **new** code, not the nearest existing one. The highest is `4`. **Treat an unrecognised non-zero code as a failure you have not seen before**, not as a synonym for `1`. A new code does not affect a pipeline that matches `1` exactly. A pipeline that treats any non-zero code as a failed build is correct, only less precise about why.
|
|
74
74
|
|
|
75
|
-
**Node's default exit code is never one of these.** A run that cannot reconcile ends on one of the codes above
|
|
75
|
+
**Node's default exit code is never one of these.** A run that cannot reconcile ends on one of the codes above. It never ends by crashing out of the process. A crash that exits `1` would look, by its status, exactly like an honest disagreement.
|
|
76
76
|
|
|
77
77
|
**An input it cannot read is a usage error (`2`)**, never a verdict about your code. This covers:
|
|
78
78
|
|
|
@@ -81,15 +81,15 @@ The whole report is written before the exit code is set, so a failing run still
|
|
|
81
81
|
- an unparseable report, or a report that parses but is not in the shape its runner writes;
|
|
82
82
|
- a corpus the Gherkin parser rejects.
|
|
83
83
|
|
|
84
|
-
The refusal names the input, the path, and what was wrong with it. A well-formed report
|
|
84
|
+
The refusal names the input, the path, and what was wrong with it. A well-formed report with nothing in it, such as a Vitest run that matched no test file, is read, not refused.
|
|
85
85
|
|
|
86
|
-
**An output location it cannot write is also `2
|
|
86
|
+
**An output location it cannot write is also `2`.** This covers a `--format` path that is read-only, is a directory, is under a regular file, or is in a directory that does not exist. The tool refuses it before it prints any report, and names the location and the operating system's reason.
|
|
87
87
|
|
|
88
88
|
`--exit-zero` changes `1` to `0` and leaves every other code alone (see *Report first, gate later*).
|
|
89
89
|
|
|
90
90
|
### `allocate`
|
|
91
91
|
|
|
92
|
-
`spec-controller allocate` prints the next claim code for a feature
|
|
92
|
+
`spec-controller allocate` prints the next claim code for a feature code. It reaches no verdict. It exits `0` when it answers, and `2` for a usage error or an unknown feature code. A file or directory it cannot read, directly or through git, is a usage error. The tool refuses it, names the path and the reason, and gives no code.
|
|
93
93
|
|
|
94
94
|
## On CI
|
|
95
95
|
|
|
@@ -100,7 +100,7 @@ spec-controller balance --features ./features \
|
|
|
100
100
|
--format json:reconciliation.json
|
|
101
101
|
```
|
|
102
102
|
|
|
103
|
-
|
|
103
|
+
You can repeat `--format`, and it takes an optional `:<path>`. On GitHub Actions, writing Markdown to `$GITHUB_STEP_SUMMARY` puts the reconciliation on the run's summary page. That page is the channel GitHub provides for a table. Writing there needs no action, no plugin and no permission.
|
|
104
104
|
|
|
105
105
|
`spec-controller balance --help` lists every flag, one line each.
|
|
106
106
|
|
package/dist/cli.js
CHANGED
|
@@ -102,10 +102,10 @@ var init_messages_en = __esm({
|
|
|
102
102
|
"more-than-one-to-stdout": "More than one output routes to stdout ({formats}); at most one may. Give one of them a :<path> to write it to a file instead (e.g. --format json:report.json).",
|
|
103
103
|
"outputs-share-file": "Two outputs name the same file: {first} and {second}. Give each output a file of its own.",
|
|
104
104
|
"output-overwrites-input": "An output would overwrite a file this run reads: {output} is {subject}, {path}. Give the report a file of its own.",
|
|
105
|
-
"unknown-
|
|
106
|
-
"no-
|
|
107
|
-
"
|
|
108
|
-
"not-a-feature-
|
|
105
|
+
"unknown-feature-code-refused": "unknown feature code \u2014 no feature file's header carries {featureCode}.\nThe feature codes this corpus's feature files carry: {featureCodes}.",
|
|
106
|
+
"no-feature-codes-carried": "none",
|
|
107
|
+
"feature-code-not-given": "no feature code was given: 'spec-controller allocate <FFF>' takes one, a feature file's header code.",
|
|
108
|
+
"not-a-feature-code": "''{featureCode}'' is not a feature code: two to five characters, each A-Z or 0-9.",
|
|
109
109
|
"ci-workflow": "a CI workflow",
|
|
110
110
|
"read-report": "a report",
|
|
111
111
|
"format-on-standard-output": "{fmt} on standard output",
|
|
@@ -174,17 +174,17 @@ var init_messages_en = __esm({
|
|
|
174
174
|
"format-flag": "Where to send output (md or json, optionally :<path> to write to a file). Repeatable.",
|
|
175
175
|
"strict-flag": "Read every scenario as if nothing were put off: a deferral tag is ignored, and a pending tag such as @integrationpending raises the Evidence Obligation it names.",
|
|
176
176
|
"exit-zero-flag": "Print the same report but exit 0 on an out-of-balance (1) verdict \u2014 the on-ramp before gating. Usage (2), not reconciled (3) and an unexpected error (4) still fail.",
|
|
177
|
-
"allocate-summary": "Print the next claim code for a feature
|
|
178
|
-
"feature-
|
|
179
|
-
"new-flag": "Start a
|
|
180
|
-
"shallow-history-risks-
|
|
177
|
+
"allocate-summary": "Print the next claim code for a feature code: one past the highest the repository has ever carried, the code alone on stdout.",
|
|
178
|
+
"feature-code-argument": "The feature code: a feature file's header code, two to five of A-Z and 0-9.",
|
|
179
|
+
"new-flag": "Start a feature code no feature file's header carries yet: its first code, or one past any number the repository already carries for it.",
|
|
180
|
+
"shallow-history-risks-feature-code": "history: shallow clone \u2014 history before {before} unread; a code of {featureCode} may have been used before",
|
|
181
181
|
"shallow-history-risks-code-above": "history: shallow clone \u2014 history before {before} unread; a code above {code} may have been used before",
|
|
182
|
-
"no-repository-risks-
|
|
182
|
+
"no-repository-risks-feature-code": "history: not a git repository \u2014 none read; a code of {featureCode} may have been used before",
|
|
183
183
|
"no-repository-risks-code-above": "history: not a git repository \u2014 none read; a code above {code} may have been used before",
|
|
184
184
|
"highest-seen-none": "highest seen: none",
|
|
185
185
|
"highest-seen-in-tree": "highest seen: {code}, {path}:{line}",
|
|
186
186
|
"highest-seen-in-commit": "highest seen: {code}, {path}:{line} in {commit}",
|
|
187
|
-
"
|
|
187
|
+
"feature-code-already-headed": "note: a feature file's header already carries {featureCode}, so --new was not needed"
|
|
188
188
|
};
|
|
189
189
|
}
|
|
190
190
|
});
|
|
@@ -35636,7 +35636,7 @@ var init_cliRegistry = __esm({
|
|
|
35636
35636
|
allocateCommand = {
|
|
35637
35637
|
name: "allocate",
|
|
35638
35638
|
summary: message("allocate-summary" /* AllocateSummary */),
|
|
35639
|
-
positional: { name: "<FFF>", description: message("feature-
|
|
35639
|
+
positional: { name: "<FFF>", description: message("feature-code-argument" /* FeatureCodeArgument */) },
|
|
35640
35640
|
flags: [
|
|
35641
35641
|
{
|
|
35642
35642
|
name: "--new",
|
|
@@ -35955,7 +35955,7 @@ var init_unreadable = __esm({
|
|
|
35955
35955
|
}
|
|
35956
35956
|
});
|
|
35957
35957
|
|
|
35958
|
-
// src/repository/
|
|
35958
|
+
// src/repository/readFeatureCodes.ts
|
|
35959
35959
|
import { readFileSync as readFileSync4 } from "node:fs";
|
|
35960
35960
|
function readFeatureCodes(featureFiles) {
|
|
35961
35961
|
const sources = featureFiles.map(({ path }) => ({
|
|
@@ -35966,8 +35966,8 @@ function readFeatureCodes(featureFiles) {
|
|
|
35966
35966
|
if (parse4.kind === "unparseable") throw new UnparseableFeatureFiles(parse4.faults);
|
|
35967
35967
|
return parse4.parsed.flatMap((feature) => featureCodeOfFile(feature) ?? []);
|
|
35968
35968
|
}
|
|
35969
|
-
var
|
|
35970
|
-
"src/repository/
|
|
35969
|
+
var init_readFeatureCodes = __esm({
|
|
35970
|
+
"src/repository/readFeatureCodes.ts"() {
|
|
35971
35971
|
"use strict";
|
|
35972
35972
|
init_dist3();
|
|
35973
35973
|
init_messages();
|
|
@@ -35979,10 +35979,10 @@ var init_readParts = __esm({
|
|
|
35979
35979
|
function readRequest(argv, registry) {
|
|
35980
35980
|
const why = argvRefusal(argv, registry);
|
|
35981
35981
|
if (why !== null) return { usageError: why };
|
|
35982
|
-
const [
|
|
35983
|
-
if (
|
|
35984
|
-
if (!isFeatureCode(
|
|
35985
|
-
return { featureCode
|
|
35982
|
+
const [featureCode] = positionalArguments(argv);
|
|
35983
|
+
if (featureCode === void 0) return { usageError: refusal("feature-code-not-given" /* FeatureCodeNotGiven */, {}) };
|
|
35984
|
+
if (!isFeatureCode(featureCode)) return { usageError: refusal("not-a-feature-code" /* NotAFeatureCode */, { featureCode }) };
|
|
35985
|
+
return { featureCode, isNew: argv.includes("--new") };
|
|
35986
35986
|
}
|
|
35987
35987
|
function argvRefusal(argv, registry) {
|
|
35988
35988
|
const args = parseArgs(argv);
|
|
@@ -36171,10 +36171,10 @@ function historyBoundaryLine(boundary, allocation) {
|
|
|
36171
36171
|
}
|
|
36172
36172
|
}
|
|
36173
36173
|
function shallowHistoryLine(before, allocation) {
|
|
36174
|
-
return allocation.highest === void 0 ? message("shallow-history-risks-
|
|
36174
|
+
return allocation.highest === void 0 ? message("shallow-history-risks-feature-code" /* ShallowHistoryRisksFeatureCode */, { before, featureCode: allocation.featureCode }) : message("shallow-history-risks-code-above" /* ShallowHistoryRisksCodeAbove */, { before, code: allocation.highest.code });
|
|
36175
36175
|
}
|
|
36176
36176
|
function noRepositoryLine(allocation) {
|
|
36177
|
-
return allocation.highest === void 0 ? message("no-repository-risks-
|
|
36177
|
+
return allocation.highest === void 0 ? message("no-repository-risks-feature-code" /* NoRepositoryRisksFeatureCode */, { featureCode: allocation.featureCode }) : message("no-repository-risks-code-above" /* NoRepositoryRisksCodeAbove */, { code: allocation.highest.code });
|
|
36178
36178
|
}
|
|
36179
36179
|
function highestSeenLine(allocation) {
|
|
36180
36180
|
const highest = allocation.highest;
|
|
@@ -36183,14 +36183,14 @@ function highestSeenLine(allocation) {
|
|
|
36183
36183
|
return commit === void 0 ? message("highest-seen-in-tree" /* HighestSeenInTree */, { code: highest.code, path, line }) : message("highest-seen-in-commit" /* HighestSeenInCommit */, { code: highest.code, path, line, commit });
|
|
36184
36184
|
}
|
|
36185
36185
|
function featureCodesListed({ featureCodes }) {
|
|
36186
|
-
return featureCodes.length === 0 ? message("no-
|
|
36186
|
+
return featureCodes.length === 0 ? message("no-feature-codes-carried" /* NoFeatureCodesCarried */) : featureCodes.join(", ");
|
|
36187
36187
|
}
|
|
36188
36188
|
function unknownFeatureCodeRefusal(unknown) {
|
|
36189
|
-
return refusal("unknown-
|
|
36189
|
+
return refusal("unknown-feature-code-refused" /* UnknownFeatureCodeRefused */, { featureCode: unknown.featureCode, featureCodes: featureCodesListed(unknown) });
|
|
36190
36190
|
}
|
|
36191
36191
|
function alreadyHeadedNote(allocation) {
|
|
36192
36192
|
if (allocation.alreadyHeaded !== true) return null;
|
|
36193
|
-
return message("
|
|
36193
|
+
return message("feature-code-already-headed" /* FeatureCodeAlreadyHeaded */, { featureCode: allocation.featureCode });
|
|
36194
36194
|
}
|
|
36195
36195
|
var init_allocateNotes = __esm({
|
|
36196
36196
|
"src/output/allocateNotes.ts"() {
|
|
@@ -36258,7 +36258,7 @@ var init_allocate = __esm({
|
|
|
36258
36258
|
init_dist3();
|
|
36259
36259
|
init_cliArgs();
|
|
36260
36260
|
init_cliRegistry();
|
|
36261
|
-
|
|
36261
|
+
init_readFeatureCodes();
|
|
36262
36262
|
init_request();
|
|
36263
36263
|
init_defaults();
|
|
36264
36264
|
init_unreadable();
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "spec-controller",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.271",
|
|
4
4
|
"description": "Reconcile the behaviour your scenarios name against the evidence that actually ran — the deterministic `balance` CLI.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"reconciliation",
|
|
@@ -26,7 +26,7 @@
|
|
|
26
26
|
"fast-xml-parser": "^5.11.2",
|
|
27
27
|
"ts-json-schema-generator": "^2.9.0",
|
|
28
28
|
"yaml": "^2.9.0",
|
|
29
|
-
"@3f-consulting/spec-controller-core": "0.1.
|
|
29
|
+
"@3f-consulting/spec-controller-core": "0.1.271"
|
|
30
30
|
},
|
|
31
31
|
"publishConfig": {
|
|
32
32
|
"access": "public"
|