specguard-ruby 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.github/workflows/ci.yml +49 -0
- data/.github/workflows/release.yml +100 -0
- data/LICENSE +21 -0
- data/README.md +1036 -0
- data/Rakefile +28 -0
- data/assets/built-with-yatfa.png +0 -0
- data/bin/specguard-ingest +40 -0
- data/bin/specguard-lint +34 -0
- data/lib/minitest/specguard_plugin.rb +25 -0
- data/lib/specguard/minitest/reporter.rb +244 -0
- data/lib/specguard/rspec/annotation_lookup.rb +363 -0
- data/lib/specguard/rspec/annotation_scanner.rb +190 -0
- data/lib/specguard/rspec/cli.rb +375 -0
- data/lib/specguard/rspec/configuration.rb +472 -0
- data/lib/specguard/rspec/file_selector.rb +276 -0
- data/lib/specguard/rspec/finding.rb +74 -0
- data/lib/specguard/rspec/formatter.rb +1110 -0
- data/lib/specguard/rspec/ingest_cli.rb +942 -0
- data/lib/specguard/rspec/ingest_reporter.rb +280 -0
- data/lib/specguard/rspec/json_reporter.rb +169 -0
- data/lib/specguard/rspec/linter.rb +156 -0
- data/lib/specguard/rspec/payload_normalizer.rb +143 -0
- data/lib/specguard/rspec/scanner.rb +235 -0
- data/lib/specguard/rspec/schemas/open-test-intent.v1.json +15 -0
- data/lib/specguard/rspec/transport.rb +449 -0
- data/lib/specguard/rspec/validator_backend.rb +1450 -0
- data/lib/specguard/rspec/version.rb +14 -0
- data/lib/specguard/rspec.rb +71 -0
- data/lib/specguard/version.rb +11 -0
- data/script/bump-version.sh +128 -0
- metadata +96 -0
|
@@ -0,0 +1,280 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module SpecGuard
|
|
6
|
+
module RSpec
|
|
7
|
+
# `specguard-ingest --json`: the machine-readable renderer over the same
|
|
8
|
+
# per-line facts the human report is built from.
|
|
9
|
+
#
|
|
10
|
+
# == Why a second renderer, on the one command that most needed it
|
|
11
|
+
#
|
|
12
|
+
# An HTTP 400 is the only *permanent* verdict in this command's contract
|
|
13
|
+
# ({IngestCLI::CONTENT_REFUSAL_CODES}) — a refused line is refused every
|
|
14
|
+
# time it is offered — so the only way to land the run is to learn which
|
|
15
|
+
# specs the platform objected to and fix the payload. The platform sends one
|
|
16
|
+
# error per offending spec, each naming it by index, file and line
|
|
17
|
+
# (`Ingest::Payload#label`, and `Api::BaseController#render_bad_request`
|
|
18
|
+
# puts every one of them on the wire). {Transport::Result} keeps the whole
|
|
19
|
+
# array, and then {Transport::Result#reason} renders three of them,
|
|
20
|
+
# truncated to 300 characters, because it is built for the **one stderr
|
|
21
|
+
# line an in-run CI warning is allowed**.
|
|
22
|
+
#
|
|
23
|
+
# That cap is right where it was set and is not touched here. It is a cap on
|
|
24
|
+
# a *line*, and this document is not a line: `specguard-ingest` already
|
|
25
|
+
# prints a row per line, a summary and its folding observations, out of band
|
|
26
|
+
# and nowhere near a CI log. So the refusal's own grounds do not reach a
|
|
27
|
+
# second channel, and this is that channel — the command whose entire job is
|
|
28
|
+
# to fix and re-send a refused run can now show you all of why it was
|
|
29
|
+
# refused.
|
|
30
|
+
#
|
|
31
|
+
# This is SPGD-305 done again for the gem's other executable, and it is a
|
|
32
|
+
# RENDERER: it reads the {IngestCLI::LineResult}s the delivery already
|
|
33
|
+
# produced and the {IngestCLI::ListedLine}s the listing already extracted,
|
|
34
|
+
# and decides nothing. The exit code, the statuses, the selection and the
|
|
35
|
+
# cap are all upstream of here and identical on both paths.
|
|
36
|
+
#
|
|
37
|
+
# == Why it is NOT `json_reporter.rb`
|
|
38
|
+
#
|
|
39
|
+
# {JSONReporter} is the *lint* document over `Linter::Result`, and it mirrors
|
|
40
|
+
# `validate-intent --json --source` key for key on purpose: the gem consumes
|
|
41
|
+
# that document, so the two must not need two parsers. This one is about
|
|
42
|
+
# **deliveries**, not annotations. It therefore does not claim that
|
|
43
|
+
# document's `"schema" => "open-test-intent.v1.json"` or its
|
|
44
|
+
# `"mode" => "source"` — naming a schema this document has nothing to do
|
|
45
|
+
# with would assert a conformance it cannot have. `"tool"` says what wrote
|
|
46
|
+
# it instead.
|
|
47
|
+
#
|
|
48
|
+
# == There is no `ok` and no `exit_code` in the document
|
|
49
|
+
#
|
|
50
|
+
# Deliberately, and this is the one place the two renderers differ in kind
|
|
51
|
+
# rather than in form. {JSONReporter} carries an `ok` because a linter's
|
|
52
|
+
# verdict is a boolean. This command's is not: 0, 1 and 2 mean "accepted",
|
|
53
|
+
# "the platform refused content" and "this tool could not do its job", and
|
|
54
|
+
# collapsing that to a boolean would have to pick which of 1 and 2 counts as
|
|
55
|
+
# false — the exact confusion {IngestCLI}'s class comment is arranged
|
|
56
|
+
# against. Restating the integer here would be a second copy of a fact the
|
|
57
|
+
# process already exits with, free to drift from it. So the exit status
|
|
58
|
+
# stays the one carrier of the verdict, unchanged by this flag, and the
|
|
59
|
+
# document carries the facts it is computed from.
|
|
60
|
+
#
|
|
61
|
+
# == Which runs emit a document
|
|
62
|
+
#
|
|
63
|
+
# By cause, not by exit code. {JSONReporter}'s rule — a run that checked
|
|
64
|
+
# nothing must not emit `{"findings": []}`, because that is how a gate that
|
|
65
|
+
# checked nothing gets mistaken for one that found nothing — transfers, but
|
|
66
|
+
# it does not transfer as "no exit-2 run emits a document": this command
|
|
67
|
+
# reaches 2 with a *real* report on a file whose lines were delivered and
|
|
68
|
+
# one of them never arrived.
|
|
69
|
+
#
|
|
70
|
+
# So the line is drawn where the file was: a run that got as far as reading
|
|
71
|
+
# <file> emits a document, whatever its exit code and even when the file
|
|
72
|
+
# held nothing to deliver (`"lines": []` next to a `summary` of zeroes is a
|
|
73
|
+
# true statement about an empty file, and the warning naming *why* it was
|
|
74
|
+
# empty is on stderr either way). A run that never got that far — a bad
|
|
75
|
+
# flag, `--from-line` with `--lines`, no endpoint or API key, a file that
|
|
76
|
+
# cannot be read — emits prose on stderr and nothing at all on stdout,
|
|
77
|
+
# because there is nothing yet to be a document about.
|
|
78
|
+
#
|
|
79
|
+
# == The counts are handed in, not recomputed
|
|
80
|
+
#
|
|
81
|
+
# `summary`'s status counts are the ones {IngestCLI} computes once for
|
|
82
|
+
# whichever renderer runs, for {JSONReporter}'s reason: two renderers of one
|
|
83
|
+
# result list that can disagree about how much of a file was delivered are
|
|
84
|
+
# worse than prose alone, because the disagreement is unfalsifiable from
|
|
85
|
+
# outside the process. That holds on the listing path too, where the only
|
|
86
|
+
# count that can be positive is `unparseable` and the text summary states no
|
|
87
|
+
# counterpart to disagree with — it is handed in anyway, so the discipline
|
|
88
|
+
# is structural at both entry points rather than true of one by luck. The
|
|
89
|
+
# folding observations are the same shape of thing — one grouping, rendered
|
|
90
|
+
# here as data and there as a sentence.
|
|
91
|
+
module IngestReporter
|
|
92
|
+
# What wrote the document. Not a schema id: see the class comment.
|
|
93
|
+
TOOL = "specguard-ingest"
|
|
94
|
+
|
|
95
|
+
# Whether lines were sent or only shown. The distinction a consumer most
|
|
96
|
+
# needs, because it decides whether the status counts below are verdicts
|
|
97
|
+
# or zeroes — see {#summary}'s `attempted`.
|
|
98
|
+
MODE_DELIVER = "deliver"
|
|
99
|
+
MODE_LIST = "list"
|
|
100
|
+
|
|
101
|
+
# A listed line that is a run. Its delivery-side counterparts are
|
|
102
|
+
# {IngestCLI::STATUS_LABELS}' keys, rendered by name (`undelivered`, not
|
|
103
|
+
# the prose renderer's `not delivered`) so a consumer branches on the
|
|
104
|
+
# tool's vocabulary rather than on its wording.
|
|
105
|
+
STATUS_LISTED = "listed"
|
|
106
|
+
STATUS_UNPARSEABLE = "unparseable"
|
|
107
|
+
|
|
108
|
+
# @param source [IngestCLI::Source] the file, its blanks and its skips
|
|
109
|
+
# @param results [Array<IngestCLI::LineResult>] one per line delivered,
|
|
110
|
+
# in the file's order
|
|
111
|
+
# @param counts [Hash{Symbol=>Integer}] status counts, computed once by
|
|
112
|
+
# {IngestCLI} for both renderers
|
|
113
|
+
# @param foldings [Array<IngestCLI::Folding>] the folding observations,
|
|
114
|
+
# grouped once for both renderers
|
|
115
|
+
# @return [String] one JSON document, without a trailing newline
|
|
116
|
+
def self.render_delivery(source:, results:, counts:, foldings:)
|
|
117
|
+
document(
|
|
118
|
+
mode: MODE_DELIVER,
|
|
119
|
+
source: source,
|
|
120
|
+
lines: results.map { |result| delivered(result) },
|
|
121
|
+
summary: summary(source, lines: results.length, counts: counts, attempted: attempted(counts)),
|
|
122
|
+
foldings: foldings.map { |folding| folded(folding) }
|
|
123
|
+
)
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
# @param source [IngestCLI::Source]
|
|
127
|
+
# @param lines [Array<IngestCLI::ListedLine>] the envelope facts the text
|
|
128
|
+
# listing prints, extracted once for both renderers
|
|
129
|
+
# @param counts [Hash{Symbol=>Integer}] status counts, computed once by
|
|
130
|
+
# {IngestCLI} for both renderers — `unparseable` is the only one a
|
|
131
|
+
# listing can carry
|
|
132
|
+
# @return [String] one JSON document, without a trailing newline
|
|
133
|
+
def self.render_listing(source:, lines:, counts:)
|
|
134
|
+
# Every delivery status is 0 and `attempted` is 0, which is this
|
|
135
|
+
# document's way of saying what the text listing says in words:
|
|
136
|
+
# nothing was delivered. `unparseable` is the one that can be positive,
|
|
137
|
+
# because a line that is not a run is knowable without sending it — and
|
|
138
|
+
# it is stated for the reason the text row states it, so a preview
|
|
139
|
+
# cannot under-report what the delivery would do.
|
|
140
|
+
document(
|
|
141
|
+
mode: MODE_LIST,
|
|
142
|
+
source: source,
|
|
143
|
+
lines: lines.map { |line| listed(line) },
|
|
144
|
+
summary: summary(source, lines: lines.length, attempted: 0, counts: counts),
|
|
145
|
+
foldings: []
|
|
146
|
+
)
|
|
147
|
+
end
|
|
148
|
+
|
|
149
|
+
def self.document(mode:, source:, lines:, summary:, foldings:)
|
|
150
|
+
JSON.pretty_generate(
|
|
151
|
+
"tool" => TOOL,
|
|
152
|
+
"mode" => mode,
|
|
153
|
+
"file" => source.path,
|
|
154
|
+
"summary" => summary,
|
|
155
|
+
"lines" => lines,
|
|
156
|
+
"foldings" => foldings
|
|
157
|
+
)
|
|
158
|
+
end
|
|
159
|
+
|
|
160
|
+
# `lines` is the rows in this document; `attempted` is how many of them
|
|
161
|
+
# were offered to the endpoint. The pair is what keeps the four status
|
|
162
|
+
# counts readable: three zeroes under `"mode": "list"` mean "nothing was
|
|
163
|
+
# sent", and the same three under a delivery of 40 lines would mean
|
|
164
|
+
# something very different.
|
|
165
|
+
#
|
|
166
|
+
# `blank` and `skipped` are the two ways a line of <file> is not a row
|
|
167
|
+
# here. They are stated always, and for the reason {IngestCLI::Source}
|
|
168
|
+
# counts rather than drops them: a summary that quietly narrows what it is
|
|
169
|
+
# summarising is the failure this command is arranged against.
|
|
170
|
+
def self.summary(source, lines:, counts:, attempted:)
|
|
171
|
+
{
|
|
172
|
+
"lines" => lines,
|
|
173
|
+
"attempted" => attempted,
|
|
174
|
+
"accepted" => counts.fetch(:accepted, 0),
|
|
175
|
+
"refused" => counts.fetch(:refused, 0),
|
|
176
|
+
"undelivered" => counts.fetch(:undelivered, 0),
|
|
177
|
+
"unparseable" => counts.fetch(:unparseable, 0),
|
|
178
|
+
"blank" => source.blank,
|
|
179
|
+
"skipped" => source.skipped,
|
|
180
|
+
"selector" => selector(source)
|
|
181
|
+
}
|
|
182
|
+
end
|
|
183
|
+
|
|
184
|
+
# Every line that reached the endpoint, which is every line except the
|
|
185
|
+
# ones that were never a run.
|
|
186
|
+
def self.attempted(counts)
|
|
187
|
+
counts.fetch(:accepted, 0) + counts.fetch(:refused, 0) + counts.fetch(:undelivered, 0)
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
# Which flag held lines back, named on exactly the terms the text summary
|
|
191
|
+
# names it: only when it demonstrably held something back. `--from-line`
|
|
192
|
+
# defaults to 1 when it was not given at all, so reporting it
|
|
193
|
+
# unconditionally would claim a selector the user never typed.
|
|
194
|
+
#
|
|
195
|
+
# @return [String, nil]
|
|
196
|
+
def self.selector(source)
|
|
197
|
+
return nil unless source.skipped.positive?
|
|
198
|
+
|
|
199
|
+
source.selector == :line_set ? "--lines" : "--from-line"
|
|
200
|
+
end
|
|
201
|
+
|
|
202
|
+
# One delivered line.
|
|
203
|
+
#
|
|
204
|
+
# `code` is the HTTP status, and `null` where there is not one: a line
|
|
205
|
+
# that was never a run, and a delivery that never got an answer at all
|
|
206
|
+
# (connection refused, DNS, TLS, a timeout). Together with `status` it is
|
|
207
|
+
# what tells the two apart from a refusal, which is why `reasons` can
|
|
208
|
+
# collapse all three into one list.
|
|
209
|
+
#
|
|
210
|
+
# `test_run_id` and `ci_run_id` are Strings or `null`, on the terms
|
|
211
|
+
# {Transport::Result#test_run_id} and {IngestCLI}'s `#scalar` already set:
|
|
212
|
+
# a non-scalar is not an id, and inventing structure the envelope does not
|
|
213
|
+
# have is not this tool's business.
|
|
214
|
+
def self.delivered(result)
|
|
215
|
+
{
|
|
216
|
+
"number" => result.number,
|
|
217
|
+
"status" => result.status.to_s,
|
|
218
|
+
"code" => result.code,
|
|
219
|
+
"reasons" => reasons(result.reasons),
|
|
220
|
+
"test_run_id" => result.test_run_id,
|
|
221
|
+
"ci_run_id" => result.ci_run_id
|
|
222
|
+
}
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
# One listed line: the envelope facts the text row prints, as values
|
|
226
|
+
# rather than as prose. `null` is the row's `no branch` / `no specs` /
|
|
227
|
+
# `no duration_seconds` — a fact the line does not carry, which is a
|
|
228
|
+
# different thing from one it carries as empty.
|
|
229
|
+
def self.listed(line)
|
|
230
|
+
{
|
|
231
|
+
"number" => line.number,
|
|
232
|
+
"status" => line.problem ? STATUS_UNPARSEABLE : STATUS_LISTED,
|
|
233
|
+
"reasons" => reasons(line.problem),
|
|
234
|
+
"branch" => line.branch,
|
|
235
|
+
"commit_sha" => line.commit_sha,
|
|
236
|
+
"ci_run_id" => line.ci_run_id,
|
|
237
|
+
"examples" => line.examples,
|
|
238
|
+
"duration_seconds" => line.duration_seconds
|
|
239
|
+
}
|
|
240
|
+
end
|
|
241
|
+
|
|
242
|
+
def self.folded(folding)
|
|
243
|
+
{
|
|
244
|
+
"ci_run_id" => folding.ci_run_id,
|
|
245
|
+
"test_run_id" => folding.test_run_id,
|
|
246
|
+
"lines" => folding.numbers
|
|
247
|
+
}
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
# ALWAYS a list of strings — `[]` where the line landed and where a
|
|
251
|
+
# refusal's body said nothing this gem could read, never `null` and never
|
|
252
|
+
# a bare string. {JSONReporter}'s `errors` makes the identical guarantee
|
|
253
|
+
# for the identical reason: `report.go:23-26` is explicit that a consumer
|
|
254
|
+
# must never have to branch on the type of the field that says why
|
|
255
|
+
# something failed, and one list is what lets a refusal's per-spec errors,
|
|
256
|
+
# a socket error and "this line is not a run" be read by one code path.
|
|
257
|
+
#
|
|
258
|
+
# The whole array, uncapped — that is the point of the document. It is
|
|
259
|
+
# what the platform sent, in its own words and in its own order, filtered
|
|
260
|
+
# only of what is not a String, because the source is a free-form JSON
|
|
261
|
+
# body and a Hash in there is not a reason.
|
|
262
|
+
#
|
|
263
|
+
# `scrub` for the same reason {Transport::Result#one_line} does it: a body
|
|
264
|
+
# from a proxy is bytes, not necessarily valid UTF-8, and `JSON.generate`
|
|
265
|
+
# raises on an invalid sequence. Raising here would cost the run its whole
|
|
266
|
+
# document — stdout empty, exit 2, an internal error on stderr — over a
|
|
267
|
+
# decoration. Note what is NOT done: the whitespace is left alone.
|
|
268
|
+
# Collapsing it is `#one_line`'s job because a CI log has one line to
|
|
269
|
+
# spend, and a JSON string has no such budget.
|
|
270
|
+
#
|
|
271
|
+
# @param values [Array, String, nil]
|
|
272
|
+
# @return [Array<String>]
|
|
273
|
+
def self.reasons(values)
|
|
274
|
+
Array(values).filter_map { |value| value.scrub("") if value.is_a?(String) }
|
|
275
|
+
end
|
|
276
|
+
|
|
277
|
+
private_class_method :document, :summary, :attempted, :selector, :delivered, :listed, :folded, :reasons
|
|
278
|
+
end
|
|
279
|
+
end
|
|
280
|
+
end
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module SpecGuard
|
|
6
|
+
module RSpec
|
|
7
|
+
# `specguard-lint --json`: the machine-readable renderer over the same
|
|
8
|
+
# `Array<Linter::Result>` the text report is built from.
|
|
9
|
+
#
|
|
10
|
+
# == Why this is a renderer and not a feature
|
|
11
|
+
#
|
|
12
|
+
# {CLI#check} has exactly one branch on which validator produced the
|
|
13
|
+
# verdicts, and it closes immediately: both arms return the same
|
|
14
|
+
# {Linter::Result} list. Everything downstream of it — the FAIL blocks, the
|
|
15
|
+
# summary line, the exit code — is shared code rather than two renderers
|
|
16
|
+
# that have to be kept in step. This hangs off that same point, so it
|
|
17
|
+
# inherits that guarantee: the `--json` document and the text report are the
|
|
18
|
+
# *same checks* rendered twice, not a second implementation that can drift.
|
|
19
|
+
#
|
|
20
|
+
# What the text renderer destroys at the print site is the structure. A
|
|
21
|
+
# {Linter::Result} carries file, line, kind, problem and reasons; the FAIL
|
|
22
|
+
# block flattens all five into prose, leaving a consumer the exit code and a
|
|
23
|
+
# regex. `kind` is the field that suffers most — {Finding}'s own comment
|
|
24
|
+
# says it is carried "because a failed extraction and an unparseable payload
|
|
25
|
+
# both land in `problem`, which makes the two indistinguishable downstream
|
|
26
|
+
# once flattened to prose", and then the CLI flattens it to prose.
|
|
27
|
+
#
|
|
28
|
+
# == The shape, and why it is the port's shape
|
|
29
|
+
#
|
|
30
|
+
# This mirrors `validate-intent --json --source`
|
|
31
|
+
# (open-test-intent, `cmd/validate-intent/report.go`) key for key:
|
|
32
|
+
#
|
|
33
|
+
# {"schema", "mode", "ok", "summary": {"files", "annotations", "failed"},
|
|
34
|
+
# "findings": [{"file", "line", "ok", "kind", "errors", "intent"}, ...]}
|
|
35
|
+
#
|
|
36
|
+
# The gem is already a *consumer* of exactly this document —
|
|
37
|
+
# {ValidatorBackend::Runner} runs `--source --json` and reconstructs
|
|
38
|
+
# {Linter::Result}s from it — so emitting a different shape would mean a
|
|
39
|
+
# consumer of both tools needs two parsers for one protocol. It does not
|
|
40
|
+
# try to be byte-identical to the binary's document, only key-, type- and
|
|
41
|
+
# value-identical, which is what a parser sees.
|
|
42
|
+
#
|
|
43
|
+
# Three properties are load-bearing for a consumer and are asserted rather
|
|
44
|
+
# than described (`spec/specguard/rspec/cli_spec.rb`):
|
|
45
|
+
#
|
|
46
|
+
# * `errors` is ALWAYS a list of strings — `reasons` when the schema
|
|
47
|
+
# rejected the annotation, `[problem]` when discovery could not produce
|
|
48
|
+
# one at all, `[]` when it passed. Never null, never a bare string:
|
|
49
|
+
# `report.go:23-26` is explicit that a consumer must never branch on its
|
|
50
|
+
# type, and this is the one place the gem's mutually-exclusive
|
|
51
|
+
# `problem`/`reasons` pair is normalised into the port's single list.
|
|
52
|
+
# * `line` is null exactly where the finding is not line-scoped, which is
|
|
53
|
+
# {Linter::Result#line_scoped?} — the same rule `#location` uses to
|
|
54
|
+
# print `file` rather than `file:0`. `:0` is not somewhere a reader can
|
|
55
|
+
# go, and emitting it here would hand every CI annotation and quickfix
|
|
56
|
+
# consumer a line that does not exist.
|
|
57
|
+
# * `kind` is null on a passing finding and one of `extraction`, `parse`,
|
|
58
|
+
# `read`, `schema` otherwise. The port has a fifth, `no-match`, which
|
|
59
|
+
# cannot appear here: {ValidatorBackend} maps it onto
|
|
60
|
+
# {Finding::KIND_READ} at the seam, because on this side of it a path
|
|
61
|
+
# that matched nothing *is* a named path that could not be opened.
|
|
62
|
+
#
|
|
63
|
+
# == The counts are handed in, not recomputed
|
|
64
|
+
#
|
|
65
|
+
# `summary.files` and `summary.annotations` are passed by {CLI}, which
|
|
66
|
+
# computes them once for both renderers. Recounting them here would let the
|
|
67
|
+
# text summary line and this document disagree about the same run, which is
|
|
68
|
+
# the failure mode the shared-downstream design exists to prevent — and a
|
|
69
|
+
# disagreement between two renderers of one result list is unfalsifiable
|
|
70
|
+
# from the outside.
|
|
71
|
+
#
|
|
72
|
+
# `ok` is likewise handed in, derived from the exit code the text path would
|
|
73
|
+
# also have produced rather than recomputed from the findings, following
|
|
74
|
+
# `report.go:84-89` for the same reason.
|
|
75
|
+
#
|
|
76
|
+
# `summary.failed`, by contrast, is counted here — from the findings this
|
|
77
|
+
# document actually emitted, so `failed` always equals the number of entries
|
|
78
|
+
# with `"ok": false`. Note that it is NOT the text summary's "M malformed":
|
|
79
|
+
# that clause counts malformed *annotations* and reports unreadable files in
|
|
80
|
+
# a separate clause, while `failed` counts every failing finding, read
|
|
81
|
+
# failures included, exactly as the port's `Emit` does. Two different
|
|
82
|
+
# questions, each answered the same way in both implementations.
|
|
83
|
+
#
|
|
84
|
+
# == What is deliberately NOT in the document
|
|
85
|
+
#
|
|
86
|
+
# The backend provenance line (SPGD-247 — one stderr line per run naming the
|
|
87
|
+
# implementation that produced the verdicts) is not duplicated in here.
|
|
88
|
+
# Two reasons, and the decision is recorded rather than defaulted into:
|
|
89
|
+
# it would be the first key by which this document differs from the port's,
|
|
90
|
+
# reintroducing the second parser this shape exists to avoid; and provenance
|
|
91
|
+
# would then have two homes that can disagree about one fact, which is the
|
|
92
|
+
# shape SPGD-247 was written to close, not to widen. `2>` the stderr stream
|
|
93
|
+
# and read the line; it is still exactly one line, on every run, on both
|
|
94
|
+
# arms.
|
|
95
|
+
module JSONReporter
|
|
96
|
+
# The port's `jsonSchemaID` (`report.go:19`), and the basename of the
|
|
97
|
+
# gem's own vendored schema. Pinned to each other by a spec: the document
|
|
98
|
+
# names the protocol it validated against, so it must not be able to name
|
|
99
|
+
# one the gem does not carry.
|
|
100
|
+
SCHEMA_ID = "open-test-intent.v1.json"
|
|
101
|
+
|
|
102
|
+
# The port's `--source` mode: annotations found in spec *sources*, which
|
|
103
|
+
# is the only thing `specguard-lint` does. It is not the *selection* mode
|
|
104
|
+
# (`--changed` vs explicit files) — that is this tool's own vocabulary and
|
|
105
|
+
# the port has no field for it. Emitting `"changed"` here would tell a
|
|
106
|
+
# consumer that already reads `validate-intent --json` that it is looking
|
|
107
|
+
# at a mode that does not exist.
|
|
108
|
+
MODE = "source"
|
|
109
|
+
|
|
110
|
+
# @param results [Array<Linter::Result>] every verdict, in discovery order
|
|
111
|
+
# @param files [Integer] spec files selected — the count the text path
|
|
112
|
+
# states in its leading `checked N spec file(s)` line
|
|
113
|
+
# @param annotations [Integer] annotation sites examined — the count the
|
|
114
|
+
# text path states in its trailing summary line
|
|
115
|
+
# @param ok [Boolean] whether the run passed, derived from the exit code
|
|
116
|
+
# @return [String] one JSON document, without a trailing newline
|
|
117
|
+
def self.render(results, files:, annotations:, ok:)
|
|
118
|
+
findings = results.map { |result| finding(result) }
|
|
119
|
+
|
|
120
|
+
JSON.pretty_generate(
|
|
121
|
+
"schema" => SCHEMA_ID,
|
|
122
|
+
"mode" => MODE,
|
|
123
|
+
"ok" => ok,
|
|
124
|
+
"summary" => {
|
|
125
|
+
"files" => files,
|
|
126
|
+
"annotations" => annotations,
|
|
127
|
+
"failed" => findings.count { |entry| !entry["ok"] }
|
|
128
|
+
},
|
|
129
|
+
"findings" => findings
|
|
130
|
+
)
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
# @param result [Linter::Result]
|
|
134
|
+
# @return [Hash] one `findings` entry
|
|
135
|
+
def self.finding(result)
|
|
136
|
+
{
|
|
137
|
+
"file" => result.file,
|
|
138
|
+
"line" => result.line_scoped? ? result.line : nil,
|
|
139
|
+
"ok" => result.ok?,
|
|
140
|
+
"kind" => result.kind&.to_s,
|
|
141
|
+
"errors" => errors(result),
|
|
142
|
+
# {Linter::Result#representable_intent}, not `intent`: a payload
|
|
143
|
+
# holding a lone low surrogate parses in Ruby and then cannot be
|
|
144
|
+
# GENERATED, so emitting it raw would raise here and take the whole
|
|
145
|
+
# document with it — a `--json` run that dies rendering a finding it
|
|
146
|
+
# had already decided about. Null is the same answer this document
|
|
147
|
+
# gives for every other payload it does not have.
|
|
148
|
+
"intent" => result.representable_intent
|
|
149
|
+
}
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
# `problem` and `reasons` are mutually exclusive by construction and mean
|
|
153
|
+
# the same thing to a consumer — why this finding failed. `kind` is what
|
|
154
|
+
# tells them apart, so both collapse into the one list here and nothing is
|
|
155
|
+
# lost. The port's `RunSourceJSON` performs the identical collapse, and
|
|
156
|
+
# {ValidatorBackend} performs its inverse when reading the port's output.
|
|
157
|
+
#
|
|
158
|
+
# @param result [Linter::Result]
|
|
159
|
+
# @return [Array<String>]
|
|
160
|
+
def self.errors(result)
|
|
161
|
+
return [result.problem] if result.problem
|
|
162
|
+
|
|
163
|
+
result.reasons
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
private_class_method :finding, :errors
|
|
167
|
+
end
|
|
168
|
+
end
|
|
169
|
+
end
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module SpecGuard
|
|
6
|
+
module RSpec
|
|
7
|
+
# The shared verdict shape both renderers and the exit code derive from.
|
|
8
|
+
#
|
|
9
|
+
# == Where this sits after SPGD-867 (the cutover)
|
|
10
|
+
#
|
|
11
|
+
# This used to be a class that APPLIED the schema to Findings the Ruby
|
|
12
|
+
# scanner had produced. That Ruby hand-rolled validation arm is gone:
|
|
13
|
+
# `specguard-lint` validates through the `validate-intent` binary and only
|
|
14
|
+
# the binary ({ValidatorBackend}), and the formatter's half
|
|
15
|
+
# ({AnnotationLookup}) asks the same binary. What survives is the one
|
|
16
|
+
# thing both sides of the seam still share — {Result}, the single shape
|
|
17
|
+
# the reporter and the exit code are both derived from, so "what was
|
|
18
|
+
# printed" and "what we exited with" cannot drift apart.
|
|
19
|
+
#
|
|
20
|
+
# == What counts as a failure
|
|
21
|
+
#
|
|
22
|
+
# Three things reach the same verdict from different directions:
|
|
23
|
+
#
|
|
24
|
+
# * the payload could not be captured off the line at all
|
|
25
|
+
# (`Finding::KIND_EXTRACTION`) — a typo'd annotation;
|
|
26
|
+
# * it was captured but is not JSON even after normalization
|
|
27
|
+
# (`Finding::KIND_PARSE`);
|
|
28
|
+
# * it parsed but violates the schema.
|
|
29
|
+
#
|
|
30
|
+
# All three are "an annotation is malformed", which the contract fixes at
|
|
31
|
+
# exit 1. A **missing** annotation is not among them — "lint, don't
|
|
32
|
+
# require" (SPGD-12 §1): a file with no `@intent:` at all is clean.
|
|
33
|
+
#
|
|
34
|
+
# `Finding::KIND_READ` — a spec file that could not be opened or is not
|
|
35
|
+
# valid UTF-8 — is also reported as a failure, and therefore also exits 1.
|
|
36
|
+
# That matches `validate-intent`, which classifies it separately (its own
|
|
37
|
+
# `read` kind) and still reports it as `FAIL` with exit 1.
|
|
38
|
+
module Linter
|
|
39
|
+
# One annotation's verdict. `problem` is set when discovery could not
|
|
40
|
+
# produce an intent at all; `reasons` when the schema rejected one.
|
|
41
|
+
# They are mutually exclusive by construction.
|
|
42
|
+
#
|
|
43
|
+
# `intent` is WHAT THE PAYLOAD PARSED TO, carried alongside the verdict
|
|
44
|
+
# rather than left for a second parser to re-derive. It is nil when there
|
|
45
|
+
# was no payload (any `problem`), and — like the port's `intent` key — it
|
|
46
|
+
# is populated even when `reasons` is not empty: a schema-rejected
|
|
47
|
+
# annotation did parse, and `ok?` already reports the verdict.
|
|
48
|
+
Result = Data.define(:file, :line, :kind, :problem, :reasons, :intent) do
|
|
49
|
+
def initialize(file:, line:, kind: nil, problem: nil, reasons: [], intent: nil)
|
|
50
|
+
super
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def ok?
|
|
54
|
+
problem.nil? && reasons.empty?
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def failed?
|
|
58
|
+
!ok?
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# A read failure is not line-scoped: nothing in the file was ever seen,
|
|
62
|
+
# and slice 1's `line` is a 0 sentinel rather than a location. The
|
|
63
|
+
# binary drops the line for exactly these findings (`JSONFinding` in
|
|
64
|
+
# open-test-intent's cmd/validate-intent/report.go — "`line` is null
|
|
65
|
+
# where a finding is not line-scoped, `kind` is null on a passing
|
|
66
|
+
# finding"), and `:0` is not somewhere a reader can go: anything
|
|
67
|
+
# parsing `file:line` — CI annotations, editor quickfix, review
|
|
68
|
+
# comments — would point at a line that does not exist.
|
|
69
|
+
#
|
|
70
|
+
# This is what makes `kind` load-bearing rather than decorative.
|
|
71
|
+
#
|
|
72
|
+
# The rule is a predicate rather than an inline comparison because it
|
|
73
|
+
# has two renderers: `CLI#report_failure` prints `file` instead of
|
|
74
|
+
# `file:0`, and `JSONReporter` emits `"line": null` for the same
|
|
75
|
+
# findings. Spelling `kind == Finding::KIND_READ` in both would let one
|
|
76
|
+
# of them keep emitting the sentinel after the rule changed here.
|
|
77
|
+
def line_scoped?
|
|
78
|
+
kind != Finding::KIND_READ
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def location
|
|
82
|
+
line_scoped? ? "#{file}:#{line}" : file
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
# The nesting budget a payload gets, well under `JSON.generate`'s
|
|
86
|
+
# default ceiling of 100.
|
|
87
|
+
#
|
|
88
|
+
# The margin is deliberate rather than tight. Both renderers WRAP the
|
|
89
|
+
# payload before generating it — {JSONReporter} in `findings[] ->
|
|
90
|
+
# finding -> intent` and {Formatter} in `specs[] -> spec -> intent`,
|
|
91
|
+
# three levels each — so a payload probed at the bare ceiling would be
|
|
92
|
+
# accepted here and still raise there. Budgeting 32 leaves either
|
|
93
|
+
# envelope room to grow without silently re-opening this hole.
|
|
94
|
+
#
|
|
95
|
+
# It costs nothing real: the schema admits four string values and
|
|
96
|
+
# nothing nested, so every payload a consumer could act on is depth 1.
|
|
97
|
+
# Anything deep enough to be refused here was already schema-invalid;
|
|
98
|
+
# this only decides whether its finding also carries the value.
|
|
99
|
+
MAX_INTENT_DEPTH = 32
|
|
100
|
+
|
|
101
|
+
# The intent, but only when it is a value this gem can actually hand on
|
|
102
|
+
# — nil otherwise.
|
|
103
|
+
#
|
|
104
|
+
# Parsing a payload is not the same as being able to REPRODUCE it, and
|
|
105
|
+
# Ruby draws that line in a place the validator does not. Three classes
|
|
106
|
+
# clear the binary's verdict AND then detonate at the point of USE:
|
|
107
|
+
#
|
|
108
|
+
# * a lone LOW surrogate (`"\udc00Or"`) — the report parser may accept
|
|
109
|
+
# it and hand back a String whose `valid_encoding?` is false, and
|
|
110
|
+
# `JSON.generate` refuses that same value with `source sequence is
|
|
111
|
+
# illegal/malformed utf-8`;
|
|
112
|
+
# * a non-finite Float — {ValidatorBackend}'s parse options set
|
|
113
|
+
# `allow_nan: true` on the way IN, while generate's default is
|
|
114
|
+
# `allow_nan: false` on the way OUT;
|
|
115
|
+
# * a container nested past the generator's `max_nesting` — the same
|
|
116
|
+
# asymmetry, since those parse options also set `max_nesting: false`.
|
|
117
|
+
#
|
|
118
|
+
# The last two are the parse options' own doing. Relaxing the parse side
|
|
119
|
+
# without relaxing the generate side does not remove the failure, it
|
|
120
|
+
# MOVES it — out of the parser, where it costs one annotation its
|
|
121
|
+
# payload, and into the generator, where it costs the whole batch its
|
|
122
|
+
# document and its exit code. Admitting a value we cannot then emit is
|
|
123
|
+
# strictly worse than never admitting it.
|
|
124
|
+
#
|
|
125
|
+
# So the question is ASKED rather than enumerated: hand the value to
|
|
126
|
+
# the generator and see whether it comes back. A predicate that lists the
|
|
127
|
+
# known-bad shapes is a list somebody has to keep complete, and the two
|
|
128
|
+
# cases above are exactly what that list missed while covering the
|
|
129
|
+
# first. `JSON.generate` is not an approximation of the oracle here, it
|
|
130
|
+
# IS the call both renderers go on to make.
|
|
131
|
+
#
|
|
132
|
+
# Answered here, once, because BOTH renderers need it and neither is the
|
|
133
|
+
# natural owner. Nothing is substituted — a repaired payload is one the
|
|
134
|
+
# author did not write, and shipping it is indistinguishable downstream
|
|
135
|
+
# from shipping the right one (KB SPGD-78). Unshippable means
|
|
136
|
+
# unannotated.
|
|
137
|
+
def representable_intent
|
|
138
|
+
intent unless unrepresentable?(intent)
|
|
139
|
+
end
|
|
140
|
+
|
|
141
|
+
private
|
|
142
|
+
|
|
143
|
+
# `NestingError` is rescued alongside `GeneratorError` rather than
|
|
144
|
+
# folded into it: despite being what the GENERATOR raises past
|
|
145
|
+
# max_nesting`, it descends from `JSON::ParserError`, so catching
|
|
146
|
+
# `GeneratorError` alone would let the over-deep case straight through.
|
|
147
|
+
def unrepresentable?(value)
|
|
148
|
+
JSON.generate(value, max_nesting: MAX_INTENT_DEPTH)
|
|
149
|
+
false
|
|
150
|
+
rescue JSON::GeneratorError, JSON::NestingError
|
|
151
|
+
true
|
|
152
|
+
end
|
|
153
|
+
end
|
|
154
|
+
end
|
|
155
|
+
end
|
|
156
|
+
end
|