specguard-ruby 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,1110 @@
1
+ # frozen_string_literal: true
2
+
3
+ # The formatter half of the client gem: it watches an RSpec run and records
4
+ # every example that finished — its name, where it lives, how long it took and
5
+ # how it ended — as one JSON object per run.
6
+ #
7
+ # == Why this file is not on `require "specguard/rspec"`'s chain
8
+ #
9
+ # `rspec` is a **development** dependency of this gem, not a runtime one
10
+ # (specguard-rspec.gemspec declares exactly one runtime dependency: `json`,
11
+ # pinned; since SPGD-867 the validator arrives as a resolved binary rather
12
+ # than as a gem). `bin/specguard-lint` loads `specguard/rspec`, and it has to
13
+ # keep working for someone who installed the gem to lint annotations on a
14
+ # machine with no RSpec at all. Putting `require "rspec/core"` on that chain
15
+ # would turn a missing test framework into a broken linter.
16
+ #
17
+ # So the formatter is opt-in, by its own path:
18
+ #
19
+ # # spec/spec_helper.rb
20
+ # require "specguard/rspec/formatter"
21
+ # RSpec.configure { |config| config.add_formatter(SpecGuard::RSpecFormatter) }
22
+ #
23
+ # # ...or, equivalently, in .rspec
24
+ # --require specguard/rspec/formatter
25
+ # --format SpecGuard::RSpecFormatter
26
+ #
27
+ # Neither form names a human formatter, and neither has to: see {#seed} for how
28
+ # the one RSpec would have installed is kept, why registering this class would
29
+ # otherwise take it away, and {#message} for the second listener that keeping it
30
+ # would otherwise leave on the stream.
31
+ #
32
+ # The `--require` is not optional in the `.rspec` form: RSpec resolves a
33
+ # `--format` argument by constant lookup and only then guesses a file name to
34
+ # require, and the name it guesses for this class is `spec_guard/r_spec_formatter`.
35
+ #
36
+ # spec/specguard/rspec/formatter_loading_spec.rb pins the separation in both
37
+ # directions.
38
+ require "rspec/core"
39
+ require "rspec/core/formatters/base_formatter"
40
+ require "json"
41
+ require "fileutils"
42
+
43
+ require_relative "configuration"
44
+ require_relative "transport"
45
+ # Brings the linter's discovery chain with it (Scanner, Finding, Schema). That
46
+ # direction is safe — `specguard/rspec` does not require `rspec/core`, so the
47
+ # linter stays loadable without RSpec; it is only the reverse that would break
48
+ # packaging.
49
+ require_relative "annotation_lookup"
50
+
51
+ module SpecGuard
52
+ # == A NOTE ON CONSTANT RESOLUTION — read this before editing
53
+ #
54
+ # `SpecGuard::RSpec` is *this gem's own namespace*. Inside a `module SpecGuard`
55
+ # body an unqualified `RSpec::Core::Formatters` therefore resolves to
56
+ # `SpecGuard::RSpec::Core::Formatters` and dies with
57
+ # `NameError: uninitialized constant SpecGuard::RSpec::Core` — on the very
58
+ # first line of the class definition, before anything else can go wrong.
59
+ #
60
+ # Every reference to the real RSpec below is consequently top-level-qualified
61
+ # with `::`. It is also why this class is `SpecGuard::RSpecFormatter`, a
62
+ # *sibling* of `SpecGuard::RSpec` rather than a member of it: the sibling name
63
+ # keeps the two namespaces from shadowing each other for readers as well as
64
+ # for the interpreter.
65
+ #
66
+ # == What it captures, and for whom
67
+ #
68
+ # Every example, annotated or not — but not because an unannotated test is
69
+ # anonymous. The row below carries `full_description` and both paths
70
+ # unconditionally, so a test nobody annotated still arrives named and
71
+ # located; its duration and its outcome ride along whenever the runner
72
+ # reported them, which is why both are written safe-navigated below. The
73
+ # protocol says the same thing in its preamble: annotated and unannotated
74
+ # tests are alike ingested — see `PROTOCOL.md` in open-test-intent, above
75
+ # "1. Annotation syntax".
76
+ #
77
+ # What an `@intent` adds is a *structured, authored* statement of what the
78
+ # test is for — fields a machine can group and compare, and a layer and
79
+ # preconditions that a description does not state — written deliberately
80
+ # rather than read off prose composed for a test runner's output. So the
81
+ # gap an annotation closes is one of structure, not of sight.
82
+ #
83
+ # That gap is still one you cannot report on over rows you never recorded,
84
+ # which is why this formatter captures the whole run: filtering to the
85
+ # annotated minority here would make the very first run of a new adopter
86
+ # look empty.
87
+ #
88
+ # id example.id — this example's identity within the run
89
+ # spec_file_path the spec file that *ran* the example, relative to the root
90
+ # file_path example.metadata[:file_path], relative to the project root
91
+ # line_number example.metadata[:line_number]
92
+ # name example.full_description — the composed describe/context/it
93
+ # duration example.execution_result.run_time, seconds, per example
94
+ # outcome example.execution_result.status — passed / failed / pending
95
+ # status "annotated" / "unannotated" — see {AnnotationLookup}
96
+ # intent the parsed annotation when annotated, null when not
97
+ #
98
+ # `status` is not decoration: `Ingest::Payload` validates it against a
99
+ # two-value enum for *every* spec and collects the failures globally, so a
100
+ # payload missing the key is not a payload with a gap — it is a 400 with one
101
+ # error per example.
102
+ #
103
+ # == Why `id` and `spec_file_path` exist alongside the coordinate
104
+ #
105
+ # `(file_path, line_number)` is the coordinate of the **code**, not of the
106
+ # **example**, and two entirely ordinary suite shapes put several examples on
107
+ # one coordinate:
108
+ #
109
+ # * a table-driven loop — `CASES.each { |c| it("...#{c}") { ... } }` writes
110
+ # the `it` once, so all N examples report the same line;
111
+ # * a shared example group — every including file reports the coordinate of
112
+ # `spec/support/shared.rb`, and the file that actually ran the example
113
+ # appears nowhere at all.
114
+ #
115
+ # Measured on a probe suite of a 3-case loop plus a 2-example shared group
116
+ # included by two files: 7 examples, 3 distinct coordinates. A key that folds
117
+ # three examples onto one row cannot carry a per-example duration, cannot
118
+ # follow one test's outcome across runs, and hands the duplicate-cluster
119
+ # surface rows that look identical because the *key* collapsed them — a
120
+ # manufactured duplicate in the product's headline answer.
121
+ #
122
+ # RSpec already ships the fix. `Example#id` is
123
+ # `"#{metadata[:rerun_file_path]}[#{metadata[:scoped_id]}]"`
124
+ # (rspec-core 3.13.6, `example.rb:117` → `metadata.rb:105`), it is rooted at
125
+ # the *including* file rather than the defining one, and it is also RSpec's
126
+ # own re-run argument — so a row that turns up slow or flaky is directly
127
+ # actionable: `rspec './spec/table_spec.rb[1:2]'`.
128
+ #
129
+ # **`id` is unique within a run, not stable across refactors.** `scoped_id` is
130
+ # positional, so reordering examples changes it — exactly as `line_number`
131
+ # changes when a line is inserted above it. It is the run-local primary key
132
+ # and nothing more; matching one test across runs remains `name` plus file.
133
+ #
134
+ # `spec_file_path` is `metadata[:rerun_file_path]`, which RSpec defaults to
135
+ # the defining file (`metadata.rb:160`) and overrides with the *including*
136
+ # file for a shared example. It is therefore equal to `file_path` for an
137
+ # ordinary example and differs only for a shared one — which is what makes
138
+ # duration-by-file aggregate to the file that ran the test rather than to a
139
+ # `spec/support/` helper.
140
+ #
141
+ # `file_path` and `line_number` keep their existing meaning — the definition
142
+ # site — and are deliberately not repurposed: the annotation lookup reads
143
+ # `@intent:` from exactly that line, so they are the coordinate the intent
144
+ # came from.
145
+ #
146
+ # == Where the run goes
147
+ #
148
+ # `api_key` set → one `POST <endpoint>/api/v1/ingest`.
149
+ # `api_key` unset → appended to `local_output_path`, one JSON object per
150
+ # line — a local development record, not a replay queue.
151
+ # delivery refused → appended to `output_path`, the replay queue.
152
+ #
153
+ # The credential is the switch on purpose: local development is then the
154
+ # default and needs no opt-out. See {#deliver} for what happens when the POST
155
+ # does not land.
156
+ #
157
+ # A malformed or schema-invalid annotation is recorded as `unannotated` with a
158
+ # null intent rather than shipped or shouted about; {AnnotationLookup}
159
+ # documents why, and the linter is the half of this gem that tells the author.
160
+ #
161
+ # == The never-block-CI contract, and why it lives *here*
162
+ #
163
+ # SpecGuard's non-negotiable is that telemetry never fails a build. It is
164
+ # tempting to read that as a property of the *transport* — a rescue around
165
+ # the POST — but RSpec does not sandbox formatters, so it is a property of
166
+ # the **capture layer** too. A raise in any of the three hooks below escapes
167
+ # `RSpec::Core::Runner.run` entirely; RSpec's own exit code is then never
168
+ # returned at all, and the process exits 1. Probed against rspec-core 3.13.6:
169
+ #
170
+ # hook=example_finished process_exit=1 rspec's exit status: never reached
171
+ # hook=stop process_exit=1 rspec's exit status: never reached
172
+ # hook=close process_exit=1 rspec's exit status: never reached
173
+ #
174
+ # In the `close` case the suite had already printed `2 examples, 0 failures`
175
+ # and the process still exited 1 — a green suite turned red by telemetry. A
176
+ # `nil` line number, an unwritable `log/`, a full disk: each of those is
177
+ # somebody's broken build, for tests that all passed.
178
+ #
179
+ # {#seed} was added after that probe and is guarded for the same reason, with
180
+ # the stakes slightly higher: it is dispatched from `Reporter#start`, before
181
+ # any example has run, so a raise there costs the whole suite rather than its
182
+ # exit code.
183
+ #
184
+ # So every hook body runs inside {#never_fail_the_run}, which swallows
185
+ # `StandardError` *and* `ScriptError` (an autoload blowing up is not a
186
+ # StandardError, and a bare rescue would miss it), warns once, and returns.
187
+ # `Interrupt`, `SignalException` and `SystemExit` are deliberately not caught:
188
+ # Ctrl-C must stay Ctrl-C.
189
+ #
190
+ # spec/specguard/rspec/formatter_spec.rb fails if any of those rescues is
191
+ # removed.
192
+ class RSpecFormatter < ::RSpec::Core::Formatters::BaseFormatter
193
+ # `:seed` and `:message` are not hooks this formatter has anything to say
194
+ # about; both are here for the *human* half of the output.
195
+ #
196
+ # `:seed` is the earliest notification of a run, and {#seed} uses it as the
197
+ # moment to repair RSpec's default-formatter suppression.
198
+ #
199
+ # `:message` is registered for the sake of the registration itself.
200
+ # `setup_default` installs a `FallbackMessageFormatter` unless some already
201
+ # registered formatter listens for `:message` (`formatters.rb:133-135`, via
202
+ # `existing_formatter_implements?` → `reporter.registered_listeners`), and
203
+ # that fallback would then print every message a *second* time alongside the
204
+ # formatter {#seed} restores. Listening here suppresses it and takes over its
205
+ # job — see {#message}, which is that same fallback under this class's roof.
206
+ ::RSpec::Core::Formatters.register self, :example_finished, :stop, :close, :seed, :message
207
+
208
+ # Prefixed so it is obvious in a CI log which tool is talking, and worded so
209
+ # a reader knows immediately that it is not their tests that broke.
210
+ WARNING_PREFIX = "SpecGuard: test telemetry failed and was skipped"
211
+
212
+ # The other half of that: the run was captured fine, the *delivery* did not
213
+ # land. Kept distinct from {WARNING_PREFIX} because the reader's situation
214
+ # is different — nothing was lost, the payload is sitting in a file, and the
215
+ # thing to fix is a key or a URL rather than a broken sink.
216
+ DELIVERY_WARNING_PREFIX = "SpecGuard: could not deliver test telemetry"
217
+
218
+ # The third shape, and the only one that reports a *deliberate* refusal
219
+ # rather than something going wrong. Nothing failed and nothing is
220
+ # recoverable from a file, because there was nothing worth keeping — see
221
+ # {#dry_run?}. It still gets said out loud: a user who wired the formatter
222
+ # up and then saw neither a POST nor a line would otherwise have to go
223
+ # looking for a bug that is not there.
224
+ DRY_RUN_WARNING_PREFIX = "SpecGuard: skipped test telemetry for a dry run"
225
+
226
+ # The refusal in full, as its own constant rather than a string built at the
227
+ # call site, because {#skip_dry_run} now needs it in two places: appended to
228
+ # with the coverage figures on the happy path, and emitted bare as the
229
+ # fallback if building those figures raises. The fallback is therefore
230
+ # *exactly* the line this formatter has always printed, which is what makes
231
+ # "the report failed" degrade to "the old behaviour" rather than to silence.
232
+ DRY_RUN_REFUSAL =
233
+ "#{DRY_RUN_WARNING_PREFIX} (rspec --dry-run executes no example bodies, so " \
234
+ "this run's durations and outcomes would not be measurements). " \
235
+ "Nothing was sent or written; the test run is unaffected."
236
+
237
+ # The report's indentation. Deliberately *not* `SpecGuard:`-prefixed: a run
238
+ # emits exactly one line carrying that prefix, so a reader scanning a CI log
239
+ # for "did SpecGuard have anything to say" gets one hit per run and not one
240
+ # hit per unannotated example. `formatter_run_spec.rb` pins that contract.
241
+ REPORT_INDENT = " "
242
+ WORKLIST_INDENT = " "
243
+
244
+ # `Ingest::Payload::STATUSES`, restated. The platform validates every spec
245
+ # against this pair, so they are the contract and not a local naming choice.
246
+ STATUS_ANNOTATED = "annotated"
247
+ STATUS_UNANNOTATED = "unannotated"
248
+
249
+ # The published formatter-protocol methods that mean "this formatter tells a
250
+ # human what the suite did", as opposed to RSpec's own auxiliaries, which
251
+ # speak only about deprecations and timings. {#reports_the_run?} is where
252
+ # this set is justified, and where the `respond_to?` approximation's limits
253
+ # are written down.
254
+ REPORTS_THE_RUN = %i[
255
+ example_started example_passed example_failed example_pending dump_summary
256
+ ].freeze
257
+
258
+ # @param output [IO] the stream RSpec hands every formatter. This class's
259
+ # product is a file, so nothing in the capture path writes here — but the
260
+ # stream is *not* unused, and a reader who is here because stdout went
261
+ # wrong is in the right place. {#message} writes to it, via
262
+ # {#relay_message} below: having taken over the `FallbackMessageFormatter`
263
+ # RSpec would otherwise have appointed, this formatter owes that
264
+ # formatter's duty, and printing a message no other registered formatter
265
+ # will print is precisely its job. That is the only write.
266
+ #
267
+ # The `nil` default has no caller. RSpec always supplies a stream, and the
268
+ # two direct constructions in the repo — both in `formatter_spec.rb` —
269
+ # pass one explicitly, so nothing exercises it. It survives because
270
+ # narrowing a public constructor's signature is not this slice's business,
271
+ # not because anything depends on it. It cannot reach
272
+ # `output.puts`: {#relay_message} returns early unless this object is in
273
+ # `RSpec.configuration.formatters`, which only RSpec puts it in, and doing
274
+ # so means RSpec built it. A `nil` that somehow got there would raise
275
+ # inside {#never_fail_the_run} and downgrade to a warning rather than
276
+ # taking the suite with it.
277
+ # @param error_stream [IO] where the one-shot warning goes. Injectable so a
278
+ # spec can read it back without reassigning `$stderr` globally.
279
+ # @param annotations [AnnotationLookup] resolves each example's `@intent:`.
280
+ # One per run: it is where the per-file scan is memoized, so sharing it
281
+ # across the run is what makes the cost O(files) rather than O(examples).
282
+ #
283
+ # Fully qualified for the same reason every `::RSpec` above is — and it is
284
+ # the same trap seen from the other side. This class is a *sibling* of
285
+ # `SpecGuard::RSpec`, not a member, so a bare `AnnotationLookup` here is
286
+ # looked up as `SpecGuard::RSpecFormatter::AnnotationLookup` and then
287
+ # `SpecGuard::AnnotationLookup`, neither of which exists — a NameError
288
+ # raised from a formatter's constructor, which RSpec reports as
289
+ # "No examples found".
290
+ def initialize(output = nil, error_stream: $stderr,
291
+ annotations: SpecGuard::RSpec::AnnotationLookup.new)
292
+ super(output)
293
+ @error_stream = error_stream
294
+ @annotations = annotations
295
+ @specs = []
296
+ @warned = false
297
+ # Stamped here rather than from a `start` hook on purpose. `:start` is not
298
+ # in this formatter's own registered set — it arrives only because
299
+ # BaseFormatter registered for it — and the wall clock a CI operator cares
300
+ # about includes loading the spec files, which happens before `start`
301
+ # fires. `duration_seconds` is therefore a superset of RSpec's own
302
+ # "Finished in N seconds", not a contradiction of it.
303
+ @started_at = monotonic_now
304
+ @duration_seconds = nil
305
+ end
306
+
307
+ # The first notification of the run — and the one hook here that exists for
308
+ # the *human* half of the output rather than the telemetry half.
309
+ #
310
+ # == The defect this repairs
311
+ #
312
+ # RSpec installs its own default formatter only if the user registered no
313
+ # formatter at all (rspec-core 3.13.6,
314
+ # `lib/rspec/core/formatters.rb:127`):
315
+ #
316
+ # add default_formatter, output_stream if @formatters.empty?
317
+ #
318
+ # Registering *this* formatter makes that list non-empty, so `progress` is
319
+ # never added — and since this formatter's product is a file, the run prints
320
+ # **nothing**. Measured on `formatter_run_spec.rb`'s `MIXED_SUITE`: ~900
321
+ # bytes of stdout without SpecGuard, **0** with it. No dots, no failure
322
+ # message, no diff, no `file:line`, no re-run command, and still exit 1. The
323
+ # telemetry was written perfectly (1 line, all 3 specs); the developer was
324
+ # left blind.
325
+ #
326
+ # (Two rules for the byte counts in this class's comments, both learned the
327
+ # hard way. **Name the suite** — they come from different ones and are not
328
+ # comparable across methods; this paragraph's and {#reports_the_run?}'s are
329
+ # `MIXED_SUITE`, {#message}'s are `NON_EXAMPLE_ERROR_SUITE`, both defined in
330
+ # `formatter_run_spec.rb`. And **prefer a delta or a count to a total**: a
331
+ # total carries the run's wall-clock digits, so it is not reproducible even
332
+ # on the same suite — `MIXED_SUITE`'s control measured 955 and 956 bytes on
333
+ # consecutive invocations, and `NON_EXAMPLE_ERROR_SUITE`'s 333 through 335.
334
+ # Hence the `~` above, and hence the figures that carry the argument
335
+ # elsewhere are deltas — 0, a 27-byte hole, one error block versus two.)
336
+ #
337
+ # It hit exactly the wrong person. The `.rspec` wiring escaped it only
338
+ # because the README spelled it with a second `--format progress` line, and
339
+ # a developer who explicitly chose `--format documentation` was unaffected —
340
+ # so the casualty was the reader who followed the README's first wiring
341
+ # block and customised nothing. (That `--format progress` line is gone from
342
+ # the README now: with this repair in place neither documented form needs
343
+ # it, which is what finally makes the two forms equivalent.)
344
+ #
345
+ # == Why here, and not in the user's `spec_helper`
346
+ #
347
+ # The obvious repair — `config.add_formatter(:progress) if
348
+ # config.formatters.empty?` inside `RSpec.configure` — is wrong, and
349
+ # silently so. `configuration_options.rb:21-25` applies `--require` (i.e.
350
+ # `spec_helper.rb`) *before* `--format`, so at that moment the list is `[]`
351
+ # no matter what the user asked for. Probed directly: with `.rspec` set to
352
+ # `--format documentation`, `config.formatters` still returns `[]` inside
353
+ # the helper. That repair gives the documentation user documentation *and*
354
+ # progress dots.
355
+ #
356
+ # `:seed` is the first notification `Reporter#start` sends
357
+ # (`reporter.rb:92`, ahead of `:start` on :93), and `Reporter#notify` runs
358
+ # `ensure_listeners_ready` — hence `setup_default` — before dispatching
359
+ # anything (`reporter.rb:207-208`). So by the time this runs, RSpec has
360
+ # finished deciding, and the decision can be read rather than predicted.
361
+ # Repairing here rather than in `start` is what keeps the output
362
+ # byte-identical: a formatter added during `:seed` still receives `:start`,
363
+ # and the only notification it misses *from `Reporter#start` onwards* is
364
+ # this one, which is forwarded to it by hand below. (Not "the only
365
+ # notification it misses" flatly — a `:message` can arrive before
366
+ # `Reporter#report` is ever entered, which is a real case and is {#message}'s
367
+ # to handle, not this method's.)
368
+ #
369
+ # The forwarding is not decoration, and it is worth knowing what goes if it
370
+ # goes: `:seed` is what prints `Randomized with seed N`, so without it a
371
+ # randomly-ordered run loses its *head* banner and keeps only the one RSpec
372
+ # re-sends at the end (`reporter.rb:189`). That is the line a developer
373
+ # needs to reproduce a flaky failure. Measured on `MIXED_SUITE` under
374
+ # `--order random`: banner twice both with and without this gem, and once if
375
+ # the forwarding loop is deleted (a 27-byte hole, the banner and its blank
376
+ # line). Under RSpec's *defined* ordering nothing prints a banner either
377
+ # way, which is why the parity example that pins this had to ask for random
378
+ # ordering explicitly.
379
+ #
380
+ # Arriving late is not the only way the restored formatter can diverge from
381
+ # one RSpec installed itself, and the second way cost a review round. By the
382
+ # time it is added, `setup_default` has already decided that nobody handles
383
+ # `:message` and installed a `FallbackMessageFormatter` to cover for it — so
384
+ # the newcomer, which does handle `:message`, became the *second* listener on
385
+ # that notification and every message printed twice. Measured on
386
+ # `NON_EXAMPLE_ERROR_SUITE`: one error block without SpecGuard, two with it
387
+ # (~335 bytes against ~546). {#message} is the fix, and it works
388
+ # by making `setup_default`'s premise true rather than by undoing its
389
+ # conclusion — there is no public way to withdraw a registered listener.
390
+ #
391
+ # == What counts as "the human formatter is missing"
392
+ #
393
+ # Not "the list is empty" — by now `setup_default` has appended RSpec's own
394
+ # `DeprecationFormatter` (and a `ProfileFormatter` under `--profile`). The
395
+ # question is whether any *other* registered formatter will give a human an
396
+ # account of the run, so that is what is asked: does it respond to any of
397
+ # `example_started`, `example_passed`, `example_failed`, `example_pending` or
398
+ # `dump_summary`? See {#reports_the_run?} for why that set, and for what the
399
+ # `respond_to?` approximation can and cannot see.
400
+ #
401
+ # Nothing here touches anything rspec-core marks `@private`: `formatters`,
402
+ # `add_formatter` and `default_formatter` are documented public API, and
403
+ # every method name in that set is part of the published formatter protocol
404
+ # (`formatters/protocol.rb`).
405
+ #
406
+ # `default_formatter` rather than a hard-coded `:progress`, for the same
407
+ # reason: a user who set `config.default_formatter = 'doc'` asked for that,
408
+ # and it is the value rspec-core's own line would have used.
409
+ def seed(notification)
410
+ never_fail_the_run { restore_suppressed_default_formatter(notification) }
411
+ end
412
+
413
+ # `RSpec::Core::Formatters::FallbackMessageFormatter`, moved inside this
414
+ # class — the other half of {#seed}'s repair, and the one that stops it
415
+ # printing everything twice.
416
+ #
417
+ # == Why this method has to exist at all
418
+ #
419
+ # `setup_default` ends with (`formatters.rb:133-135`):
420
+ #
421
+ # unless existing_formatter_implements?(:message)
422
+ # add FallbackMessageFormatter, output_stream
423
+ # end
424
+ #
425
+ # Somebody must print `reporter.message` output — a `--seed` banner, "No
426
+ # examples found.", and every non-example exception, since
427
+ # `notify_non_example_exception` routes through it (`reporter.rb:163-170`).
428
+ # `progress` and `documentation` do it via `BaseTextFormatter#message`; when
429
+ # no registered formatter listens for `:message`, RSpec appoints a fallback.
430
+ #
431
+ # Before this method, this class did not listen for `:message`, so the
432
+ # fallback was always appointed — and then {#seed} added `progress`, which
433
+ # listens for it too. Two listeners, one stream, every message twice. On
434
+ # `NON_EXAMPLE_ERROR_SUITE` the error block printed once without SpecGuard
435
+ # and twice with it, which is the claim that matters and the one the spec
436
+ # asserts; the streams were ~335 and ~546 bytes.
437
+ #
438
+ # The fallback cannot be withdrawn once appointed: `Reporter#register_listener`
439
+ # has no inverse, and `Configuration#formatters` hands back a `dup`
440
+ # (`configuration.rb:1024-1026`), so deleting from it changes nothing. The
441
+ # repair is therefore to stop it being appointed — this class listens for
442
+ # `:message`, `existing_formatter_implements?(:message)` is true, and the
443
+ # duty lands here instead.
444
+ #
445
+ # == When it actually prints
446
+ #
447
+ # Only when it is the last resort, which is exactly the fallback's own
448
+ # contract: if any *other* registered formatter handles `:message`, that one
449
+ # prints and this stays quiet. So the wirings behave identically to a run
450
+ # without this gem — `progress` prints it under the README's Ruby form once
451
+ # {#seed} has restored it, `documentation` prints it for a developer who
452
+ # asked for documentation, `json` swallows it into its hash exactly as it
453
+ # does on its own, and `--format html` (which has no `message`) gets it from
454
+ # here, just as it would have got it from the fallback.
455
+ #
456
+ # The check is made per message rather than cached, because a message can
457
+ # arrive **before** `:seed`: `Runner#setup` calls `world.announce_filters`,
458
+ # which reports "No examples found." through `reporter.message` before
459
+ # `Reporter#report` is ever entered. In that window this formatter really is
460
+ # the only listener, and it prints — which is what the fallback would have
461
+ # done.
462
+ #
463
+ # `respond_to?` is the public-API approximation of the question rspec-core
464
+ # answers with `registered_listeners(:message)`; {#reports_the_run?}
465
+ # documents how the two can differ. For `:message` specifically they agree
466
+ # across every formatter rspec-core ships, because the only classes that
467
+ # define `message` are the ones that register it.
468
+ def message(notification)
469
+ never_fail_the_run { relay_message(notification) }
470
+ end
471
+
472
+ # One example finished — passed, failed or pending. Called for every
473
+ # example in the run, in the order they complete.
474
+ def example_finished(notification)
475
+ never_fail_the_run { @specs << capture(notification.example) }
476
+ end
477
+
478
+ # The suite is over but the process is still alive. Sealing the duration
479
+ # here rather than in `close` keeps the number from absorbing the time the
480
+ # other formatters spend dumping their summaries.
481
+ def stop(_notification)
482
+ never_fail_the_run { @duration_seconds = elapsed_since_start }
483
+ end
484
+
485
+ # Last hook of the run: flush what we captured.
486
+ #
487
+ # `super` restores the output stream's sync setting, which BaseFormatter
488
+ # changed on our behalf at `start` (a notification it registers for, and so
489
+ # one this subclass receives too). Overriding `close` without calling it
490
+ # would leave stdout unbuffered for whatever runs next in the process.
491
+ def close(notification)
492
+ never_fail_the_run do
493
+ @duration_seconds ||= elapsed_since_start
494
+ deliver(payload)
495
+ end
496
+ never_fail_the_run { super(notification) }
497
+ end
498
+
499
+ # The run, as it will be written or POSTed. Public so a caller — or a spec
500
+ # — can inspect what was captured without going anywhere near the
501
+ # filesystem or the network.
502
+ #
503
+ # Key names match the platform's ingest contract
504
+ # (`Ingest::Payload`: commit_sha / branch / ci_run_id / duration_seconds /
505
+ # specs), which is what made adding transport a transport change rather
506
+ # than a reshaping of everything above it. {Transport} sends this Hash
507
+ # verbatim.
508
+ #
509
+ # `ci_run_id` is the field that keeps a sharded suite honest: every shard of
510
+ # one CI run emits the same one, and the platform folds them onto a single
511
+ # `TestRun` instead of recording one row per shard with a quarter of the
512
+ # denominator in it. `nil` here — a laptop run, an unrecognised
513
+ # provider — means "this run is its own run", which is the pre-existing
514
+ # behaviour and is left exactly alone.
515
+ #
516
+ # `shard_id` says *which* slice of that run this is, and it is what makes
517
+ # the fold idempotent. A CI run id survives a re-run by design (GitHub's
518
+ # `GITHUB_RUN_ID` is documented as unchanged across attempts), so without a
519
+ # per-shard key the platform could only add a re-delivered slice, never
520
+ # recognise it, and "re-run failed jobs" would report a suite bigger than
521
+ # the suite. With it, a retried shard replaces its own previous numbers.
522
+ # `nil` is allowed and still counts — see {Configuration::SHARD_ID_KEYS}.
523
+ #
524
+ # The settings are `run_id` / `shard_id` and the wire fields are
525
+ # `ci_run_id` / `shard_id`, deliberately.
526
+ # Configuration names are this gem's own (`SPECGUARD_RUN_ID`, next to
527
+ # `output_path` and `endpoint`); every key in *this* Hash is the platform's,
528
+ # spelled exactly as `TestRun` spells it. That rule is what lets a reader
529
+ # check the envelope against the schema without a translation table — and a
530
+ # run identity that two sides spell differently is one more way to split a
531
+ # run, which is the whole defect this field closes.
532
+ #
533
+ # @return [Hash]
534
+ def payload
535
+ configuration = SpecGuard::RSpec.configuration
536
+
537
+ {
538
+ "commit_sha" => configuration.commit_sha,
539
+ "branch" => configuration.branch,
540
+ "ci_run_id" => configuration.run_id,
541
+ "shard_id" => configuration.shard_id,
542
+ "duration_seconds" => @duration_seconds,
543
+ "specs" => @specs
544
+ }
545
+ end
546
+
547
+ private
548
+
549
+ # {#seed}'s body. Adds RSpec's default formatter when registering this one
550
+ # suppressed it, and hands the newcomer the notification it arrived too late
551
+ # to receive.
552
+ #
553
+ # The `equal?(self)` guard on the first line is not defensive padding: it is
554
+ # what keeps this method inert outside a real run. `spec/.../formatter_spec.rb`
555
+ # drives these hooks in-process against a formatter that RSpec never
556
+ # registered, and without the guard each of those examples would bolt a
557
+ # progress formatter onto the *parent* suite's live configuration. If this
558
+ # object is not in `RSpec.configuration.formatters`, nothing here is its
559
+ # business.
560
+ def restore_suppressed_default_formatter(notification)
561
+ configuration = rspec_configuration
562
+ registered = configuration.formatters
563
+ return unless registered.any? { |formatter| formatter.equal?(self) }
564
+ return if registered.any? { |formatter| reports_the_run?(formatter) }
565
+
566
+ configuration.add_formatter(configuration.default_formatter)
567
+ configuration.formatters.each do |formatter|
568
+ next if registered.any? { |already| already.equal?(formatter) }
569
+
570
+ formatter.seed(notification) if formatter.respond_to?(:seed)
571
+ end
572
+ end
573
+
574
+ # {#message}'s body: print, but only as the last resort.
575
+ #
576
+ # The `equal?(self)` guard is the same inertness guard
577
+ # {#restore_suppressed_default_formatter} carries, for the same reason — this
578
+ # object only owes anybody a message because RSpec registered it and skipped
579
+ # appointing its own fallback. A formatter RSpec never registered (the
580
+ # in-process examples in `spec/.../formatter_spec.rb`) owes nothing and
581
+ # writes nothing.
582
+ def relay_message(notification)
583
+ registered = rspec_configuration.formatters
584
+ return unless registered.any? { |formatter| formatter.equal?(self) }
585
+ return if registered.any? { |formatter| relays_messages?(formatter) }
586
+
587
+ output.puts notification.message
588
+ end
589
+
590
+ # The live RSpec configuration, behind a method so a spec can hand this
591
+ # object a stand-in. Stubbing `::RSpec.configuration` itself is not an
592
+ # option: rspec-core reads it throughout the example it is running, so the
593
+ # double gets asked things like `dry_run?` and the example dies inside
594
+ # RSpec rather than inside this class.
595
+ #
596
+ # The `::` is load-bearing and this is the only place it can be wrong.
597
+ # Inside `module SpecGuard` a bare `RSpec.configuration` reaches *this
598
+ # gem's* Configuration, which has no `dry_run?` — see {#dry_run?} for what
599
+ # that costs. Keeping every reader of the real configuration funnelled
600
+ # through this one method is what gives that hazard a single site to be
601
+ # pinned by a test, and formatter_spec.rb pins it with an `equal`
602
+ # assertion against `::RSpec.configuration`.
603
+ def rspec_configuration
604
+ ::RSpec.configuration
605
+ end
606
+
607
+ # Whether `formatter` is somebody else's account of the run — i.e. anything
608
+ # but this formatter that will tell a human what the suite did.
609
+ #
610
+ # == Why this set of methods
611
+ #
612
+ # It is the late-run translation of the condition rspec-core evaluates at
613
+ # `formatters.rb:127`, `@formatters.empty?`. That runs *before* RSpec appends
614
+ # its own auxiliaries, so "empty" there means "the user named no formatter";
615
+ # by the time {#seed} can look, the list also holds a `DeprecationFormatter`
616
+ # (and a `ProfileFormatter` under `--profile`). What separates those from
617
+ # every formatter a user can name is that they speak only about deprecations
618
+ # and timings — never about an example, never about the outcome of the run.
619
+ #
620
+ # So the question is asked in exactly those terms. Checked against the
621
+ # formatters rspec-core ships: `progress`, `documentation` and `html` answer
622
+ # on the `example_*` methods, `json` on `dump_summary`, `--format failures`
623
+ # (`FailureListFormatter`) on `example_failed`; `DeprecationFormatter`,
624
+ # `FallbackMessageFormatter` and `ProfileFormatter` answer on none of them.
625
+ #
626
+ # `dump_summary` alone is *not* enough, and shipping it alone was a bug. A
627
+ # suite run with `--format failures` prints one line per failure and no
628
+ # summary, so a `dump_summary`-only question read it as "nobody is reporting"
629
+ # and bolted a whole progress run onto an explicitly chosen formatter.
630
+ # Measured on `formatter_run_spec.rb`'s `MIXED_SUITE`: 61 bytes of failure
631
+ # list became ~963 bytes of dots, backtrace and summary. That
632
+ # is the additive promise broken in the same breath as it is kept.
633
+ #
634
+ # == What `respond_to?` cannot see
635
+ #
636
+ # rspec-core answers the structurally identical question with
637
+ # `@reporter.registered_listeners(notification).any?` (`formatters.rb:207-209`)
638
+ # — *registration*, not method presence. That is the more accurate question
639
+ # and it is deliberately not the one asked here: `registered_listeners` is
640
+ # `@private` (`reporter.rb:51`), and this repair is worth nothing if a minor
641
+ # rspec-core bump can silence a suite with it.
642
+ #
643
+ # The approximation is not free, and it is worth knowing which way it errs. A
644
+ # formatter that *inherits* one of these methods from `BaseTextFormatter`
645
+ # while registering only a subset of notifications reads as an account of the
646
+ # run and never receives one — so the default is not restored and the run is
647
+ # quiet, which is this ticket's own defect one level in. Nothing rspec-core
648
+ # ships is shaped that way (every class that defines one of these registers
649
+ # it), and a third-party formatter would have to inherit `BaseTextFormatter`
650
+ # specifically to avoid printing. The reverse error — a formatter that prints
651
+ # under some other method name, so the default is restored alongside it — is
652
+ # noisy rather than silent, and is the direction to prefer if this ever has
653
+ # to be re-tuned.
654
+ def reports_the_run?(formatter)
655
+ return false if formatter.equal?(self)
656
+
657
+ REPORTS_THE_RUN.any? { |method_name| formatter.respond_to?(method_name) }
658
+ end
659
+
660
+ # Whether `formatter` is somebody else's `:message` handling — the question
661
+ # {#message} asks before deciding it is the last resort. See {#message}.
662
+ def relays_messages?(formatter)
663
+ !formatter.equal?(self) && formatter.respond_to?(:message)
664
+ end
665
+
666
+ # Sink selection, and the fallback that keeps a failed delivery from being
667
+ # a silent one.
668
+ #
669
+ # Called from inside {#never_fail_the_run}, so nothing below has to guard
670
+ # itself against raising — including the fallback `append`, whose own
671
+ # failure modes (unwritable `log/`, full disk) are already covered there.
672
+ #
673
+ # == Why a failure writes the file rather than shrugging
674
+ #
675
+ # "Never block CI" is a promise about the *exit code*, not a licence to
676
+ # discard the run. A 401 with no file left behind loses the whole run's
677
+ # telemetry for a mistake — a rotated key, a typo'd URL — that is fixed in
678
+ # thirty seconds and cannot be re-run afterwards, because the suite is over.
679
+ # Writing the payload to the replay queue turns silent loss into recoverable
680
+ # loss, which is the same thing `output_path` already exists for ("supports
681
+ # a future replay-from-file ingestion path").
682
+ #
683
+ # == And the one case that goes to neither sink
684
+ #
685
+ # A dry run is refused outright — no POST, no line. It is the exception to
686
+ # the paragraph above because there is nothing to recover: the run measured
687
+ # nothing, so keeping it is not preserving telemetry, it is manufacturing
688
+ # it. {#dry_run?} has the argument in full.
689
+ def deliver(data)
690
+ return skip_dry_run if dry_run?
691
+
692
+ configuration = SpecGuard::RSpec.configuration
693
+ return append_local(data) if blank?(configuration.api_key)
694
+
695
+ obstacle = undeliverable_reason(configuration, data)
696
+ return fall_back(data, obstacle) if obstacle
697
+
698
+ result = transport_for(configuration).deliver(data)
699
+ return if result.success?
700
+
701
+ fall_back(data, result.reason)
702
+ end
703
+
704
+ # The checks worth doing *before* spending the run's wall clock on a request
705
+ # whose answer is already known.
706
+ #
707
+ # `commit_sha` is the sharp one: `Ingest::Payload#validate_commit_sha`
708
+ # refuses a blank one and the controller renders 400, so the run would be
709
+ # discarded whole — every example, not merely the empty field. Ten seconds
710
+ # of CI time to be told that is pure loss, and the operator learns more from
711
+ # being told *why* than from being shown the platform's rejection.
712
+ #
713
+ # @return [String, nil] nil when there is nothing standing in the way.
714
+ def undeliverable_reason(configuration, data)
715
+ if blank?(configuration.endpoint)
716
+ return "an API key is set but no endpoint is (set SPECGUARD_ENDPOINT)"
717
+ end
718
+
719
+ return unless blank?(data["commit_sha"])
720
+
721
+ "the commit could not be determined, and the endpoint rejects a run without one " \
722
+ "(set SPECGUARD_COMMIT_SHA)"
723
+ end
724
+
725
+ # Is RSpec executing example bodies at all?
726
+ #
727
+ # == Why a dry run must not be delivered
728
+ #
729
+ # `rspec --dry-run` walks the suite, builds every example and reports each
730
+ # one as `passed`, without running a single body. The two per-example fields
731
+ # this formatter exists to report are therefore both fabrications:
732
+ # `duration` is the cost of constructing an example (single-digit
733
+ # microseconds against the pinned rspec-core 3.13.6 — a `sleep 0.05`
734
+ # example understates its own runtime by three to four orders of magnitude)
735
+ # and `outcome` is `passed` for code that never executed.
736
+ #
737
+ # Nothing downstream can tell. The payload carries no dry-run marker,
738
+ # `Ingest::Payload` validates neither field, and `Repository#latest_test_run`
739
+ # is last-writer-wins — so one lint job that runs `rspec --dry-run` with an
740
+ # environment-level `SPECGUARD_API_KEY` in scope replaces the suite's
741
+ # headline with all-green zeroes that look exactly like a fast, healthy
742
+ # suite.
743
+ #
744
+ # The local sink gets the same refusal for the same reason: a `.jsonl` of
745
+ # zero-duration all-green runs is the identical corruption, deferred until
746
+ # something replays it.
747
+ #
748
+ # == The `::` is load-bearing — do not remove it
749
+ #
750
+ # This is the class-body hazard documented at the top of `module SpecGuard`,
751
+ # in its worst form. Inside this namespace a bare `RSpec.configuration`
752
+ # resolves to `SpecGuard::RSpec.configuration` — *this gem's* Configuration,
753
+ # whose accessors are `commit_sha`, `branch`, `run_id`, `shard_id`,
754
+ # `output_path`, `endpoint`, `api_key` and `timeout`, and which has no
755
+ # `dry_run?`. Note that four methods here — {#payload}, {#deliver},
756
+ # {#append} and {#warn_delivery_failure} — legitimately call
757
+ # `SpecGuard::RSpec.configuration` for this gem's own settings, so both
758
+ # spellings appear in this file and mean different things.
759
+ #
760
+ # The mis-spelling's consequence depends on the `respond_to?` below, and
761
+ # both halves were measured by mutating {#rspec_configuration} and running
762
+ # `bundle exec rspec` (601 examples on the tree this landed on):
763
+ #
764
+ # bare `RSpec`, respond_to? kept this gem's Configuration answers
765
+ # `respond_to?(:dry_run?) => false`, so
766
+ # every run is treated as ordinary and
767
+ # the defect this method closes is
768
+ # silently reinstated. 23 fail
769
+ # suite-wide; the one that names *this*
770
+ # cause rather than a downstream
771
+ # symptom is formatter_spec.rb's
772
+ # "reads the real RSpec's
773
+ # configuration, not this gem's
774
+ # same-named one". The other 22 are
775
+ # symptoms: 4 further dry-run ones, and
776
+ # 18 from the message relay, which
777
+ # reads this same method for
778
+ # `.formatters`.
779
+ # bare `RSpec`, respond_to? removed NoMethodError inside `close`'s
780
+ # {#never_fail_the_run}, swallowed — a
781
+ # formatter that delivers *nothing*, on
782
+ # every run, behind one warning line,
783
+ # while the suite still passes. 110
784
+ # fail suite-wide.
785
+ #
786
+ # Both totals are whole-suite figures and will move with any edit anywhere
787
+ # in the tree; the named example above is the durable half of the claim.
788
+ #
789
+ # So `respond_to?` is not only there for an RSpec build without the
790
+ # predicate: it is what keeps a mis-spelling from escalating into deleting
791
+ # all telemetry. Neither state is acceptable, and {#rspec_configuration}
792
+ # gives the `::` exactly one place to be wrong and one place to be pinned —
793
+ # formatter_spec.rb asserts it is `equal` to `::RSpec.configuration`, which
794
+ # is the only assertion that names the cause rather than a downstream
795
+ # symptom.
796
+ def dry_run?
797
+ configuration = rspec_configuration
798
+ return false unless configuration.respond_to?(:dry_run?)
799
+
800
+ configuration.dry_run?
801
+ end
802
+
803
+ # A dry run's one useful product, said out loud instead of thrown away.
804
+ #
805
+ # == Why there is anything to say at all
806
+ #
807
+ # {#capture} builds three fields side by side, and only two of them are
808
+ # fabricated by a dry run. `duration` and `outcome` come from
809
+ # `example.execution_result`, which under `--dry-run` describes a body that
810
+ # never ran — that is the whole reason delivery is refused. But `status`
811
+ # comes from {AnnotationLookup#intent_for}, which reads the *spec file* and
812
+ # scans its text; it is a fact about source, identical whether or not
813
+ # anything executed. A dry run builds every example, so it holds the exact
814
+ # numerator and denominator of the product's headline adoption metric, and
815
+ # until now discarded both.
816
+ #
817
+ # This adds a third destination the refusal never contemplated: the
818
+ # operator's own terminal. Nothing is published — the POST and the
819
+ # `log/test_results.jsonl` append stay refused, exactly as before.
820
+ #
821
+ # == "This working tree", never "your repository"
822
+ #
823
+ # The wording matters and is not decoration. The repository's headline
824
+ # figure is derived from the last *delivered* run; this one is derived from
825
+ # the files on disk right now. They legitimately differ the moment somebody
826
+ # edits a spec — which is precisely the gap this exists to close — so the
827
+ # line must not be confusable with the dashboard's number.
828
+ #
829
+ # == Why this does not go through {#emit_warning}
830
+ #
831
+ # The one-shot budget exists to stop a per-example failure printing
832
+ # thousands of identical lines. Neither half of it fits here:
833
+ #
834
+ # * *Reading* it would let an earlier warning swallow the report. That is
835
+ # worst in the case where the report matters most: if the annotation
836
+ # scanner blew up, every example is classified `unannotated` and the
837
+ # coverage figure reads 0%. An operator must see the warning *and* the
838
+ # figure to know the figure is explained by the failure rather than by
839
+ # their specs.
840
+ # * *Spending* it would silence a genuine later warning to buy a line that
841
+ # was never a warning in the first place. Nothing went wrong here.
842
+ #
843
+ # It cannot flood, either — it is emitted once, from {#deliver}, at `close`.
844
+ #
845
+ # Building the report is wrapped so that a failure degrades to the bare
846
+ # refusal this formatter has always printed, and the write is wrapped so a
847
+ # dead stream is still not worth failing a run over ({#emit_warning} makes
848
+ # the same trade). The never-block-CI contract is not suspended by a
849
+ # reporting feature.
850
+ #
851
+ # Both rescues are *behaviour*, not decoration, so both are pinned by an
852
+ # example that fails if they are removed or emptied:
853
+ #
854
+ # * the inner one degrades to the refusal rather than to silence — which is
855
+ # the whole reason {DRY_RUN_REFUSAL} is a constant — and "reports the
856
+ # refusal, and only the refusal, when the report cannot be built" asserts
857
+ # the emitted text *equals* it, so both emptying the fallback and
858
+ # swallowing it fail.
859
+ # * the outer one keeps a dead stream from spending the run's one warning
860
+ # through `close`'s {#never_fail_the_run}. Without it, a stream that
861
+ # failed here would consume the budget a *genuine* later failure needs,
862
+ # trading a line nobody could read for one somebody needed; "does not
863
+ # spend the run's one warning when the error stream is dead" pins it.
864
+ def skip_dry_run
865
+ lines =
866
+ begin
867
+ [dry_run_summary, *unannotated_worklist]
868
+ rescue ScriptError, StandardError
869
+ [DRY_RUN_REFUSAL]
870
+ end
871
+
872
+ lines.each { |line| @error_stream.puts(line) }
873
+ rescue ScriptError, StandardError
874
+ nil
875
+ end
876
+
877
+ # The single `SpecGuard:`-prefixed line of a dry run: the refusal, then the
878
+ # coverage it was able to compute anyway.
879
+ def dry_run_summary
880
+ total = @specs.length
881
+ unannotated = unannotated_specs.length
882
+ annotated = total - unannotated
883
+
884
+ "#{DRY_RUN_REFUSAL} #{coverage_sentence(total, annotated, unannotated)}"
885
+ end
886
+
887
+ def coverage_sentence(total, annotated, unannotated)
888
+ if total.zero?
889
+ return "No examples were built, so there is no annotation coverage " \
890
+ "to report for this working tree."
891
+ end
892
+
893
+ "Annotation coverage is a fact about source, not about execution, so it " \
894
+ "survives the refusal — this working tree: #{total} #{pluralize(total, 'example')}, " \
895
+ "#{annotated} annotated, #{unannotated} unannotated " \
896
+ "(#{annotated_percentage(annotated, total)}% annotated)."
897
+ end
898
+
899
+ # The same predicate the platform applies, spelled the same way round.
900
+ #
901
+ # `Ingest::Payload#annotated_specs` *rejects* `status == "unannotated"`
902
+ # rather than selecting `"annotated"`, and `Ingest::ObservationRecorder`
903
+ # writes that same string verbatim. Selecting on `STATUS_UNANNOTATED` here —
904
+ # rather than on `STATUS_ANNOTATED`, which would read more naturally — is
905
+ # what makes "local figure equals `total_specs - annotated_specs` for the
906
+ # same tree" true *by construction* instead of true until one side drifts.
907
+ def unannotated_specs
908
+ @specs.select { |spec| spec["status"] == STATUS_UNANNOTATED }
909
+ end
910
+
911
+ # The worklist: the same four fields `latest_run.unannotated_examples`
912
+ # serves server-side, so a local list and the dashboard's have one shape.
913
+ #
914
+ # Sorted by definition site rather than left in capture order, which under
915
+ # `--order random` is a different sequence every run. A worklist you can
916
+ # diff between two runs is worth more than one that mirrors execution.
917
+ def unannotated_worklist
918
+ specs = unannotated_specs
919
+ return [] if specs.empty?
920
+
921
+ sites = specs.map { |spec| ["#{spec['spec_file_path']}:#{spec['line_number']}", spec] }
922
+ .sort_by { |_site, spec| [spec["spec_file_path"].to_s, spec["line_number"].to_i] }
923
+ width = sites.map { |site, _| site.length }.max
924
+
925
+ heading = "#{REPORT_INDENT}the #{specs.length} unannotated " \
926
+ "#{pluralize(specs.length, 'example')}, by definition site:"
927
+
928
+ [heading] + sites.map do |site, spec|
929
+ "#{WORKLIST_INDENT}#{site.ljust(width)} #{spec['name']}"
930
+ end
931
+ end
932
+
933
+ # Rounded, but never rounded *across* either boundary: 999 of 1000 must not
934
+ # read `100% annotated`, and 1 of 1000 must not read `0%`. Both would say
935
+ # the suite had reached — or had never left — a state it had not, which is
936
+ # the one way a coverage figure actively misleads rather than merely
937
+ # approximates.
938
+ #
939
+ # There is deliberately no `total.zero?` guard here. {#coverage_sentence}
940
+ # already returns before reaching this, so a guard would be a branch no
941
+ # input could reach — and an unreachable branch is one nothing can falsify,
942
+ # which is exactly what the two clamps below are not. One zero-denominator
943
+ # answer, in one place, pinned by "says there is nothing to measure rather
944
+ # than dividing by zero".
945
+ def annotated_percentage(annotated, total)
946
+ percentage = (annotated * 100.0 / total).round
947
+ return 99 if percentage >= 100 && annotated < total
948
+ return 1 if percentage <= 0 && annotated.positive?
949
+
950
+ percentage
951
+ end
952
+
953
+ def pluralize(count, noun)
954
+ count == 1 ? noun : "#{noun}s"
955
+ end
956
+
957
+ def fall_back(data, reason)
958
+ # Warned before the write, not after: if the fallback write *also* fails,
959
+ # the outer guard's one allotted warning has already been spent on the
960
+ # more specific message, which is the one naming the status code.
961
+ warn_delivery_failure(reason)
962
+ append(data)
963
+ end
964
+
965
+ def transport_for(configuration)
966
+ SpecGuard::RSpec::Transport.new(
967
+ endpoint: configuration.endpoint,
968
+ api_key: configuration.api_key,
969
+ timeout: configuration.timeout
970
+ )
971
+ end
972
+
973
+ def blank?(value)
974
+ value.nil? || value.to_s.strip.empty?
975
+ end
976
+
977
+ def capture(example)
978
+ metadata = example.metadata || {}
979
+ result = example.execution_result
980
+ file_path = relative_path(metadata[:file_path])
981
+ line_number = metadata[:line_number]
982
+
983
+ # Its own envelope, inside the one `example_finished` already provides.
984
+ # The outer one drops the whole example; this one drops only its
985
+ # annotation, so an unreadable spec file or a broken schema costs the run
986
+ # the intent of the examples in it and not the examples themselves. It
987
+ # also routes through {#warn_once}, so the operator hears about it exactly
988
+ # once however many examples are affected.
989
+ intent = never_fail_the_run { @annotations.intent_for(file: file_path, line: line_number) }
990
+
991
+ {
992
+ # Not wrapped in `never_fail_the_run`, unlike the annotation lookup
993
+ # above. That guard is there because the lookup does file I/O; this is a
994
+ # memoized string interpolation over a Hash RSpec has already built, and
995
+ # a guard here could only ever add a way to emit an identity-less row
996
+ # silently. The outer guard at `#example_finished` still covers it.
997
+ "id" => example.id,
998
+ # `|| file_path` rather than a bare read: RSpec defaults
999
+ # `:rerun_file_path` for every example it builds, so the fallback is
1000
+ # unreachable in a real run — but if a producer ever hands us metadata
1001
+ # without it, the definition site is the best answer available and a
1002
+ # null would silently drop a whole file out of any by-file aggregate.
1003
+ "spec_file_path" => relative_path(metadata[:rerun_file_path]) || file_path,
1004
+ "file_path" => file_path,
1005
+ "line_number" => line_number,
1006
+ "name" => example.full_description,
1007
+ "duration" => result&.run_time,
1008
+ # A Symbol here would serialize fine but would compare unequal to the
1009
+ # string every consumer reads back out of JSON.
1010
+ "outcome" => result&.status&.to_s,
1011
+ "status" => intent.nil? ? STATUS_UNANNOTATED : STATUS_ANNOTATED,
1012
+ # Written explicitly rather than omitted. `Ingest::Payload#validate_intent`
1013
+ # accepts either for an unannotated spec (nil is nil whether the key was
1014
+ # absent or null), and a present null is the difference
1015
+ # between "we looked and there was no annotation" and "this producer
1016
+ # does not report annotations", which is precisely what slice 1's
1017
+ # payload could not say.
1018
+ "intent" => intent
1019
+ }
1020
+ end
1021
+
1022
+ # RSpec reports `./spec/orders_spec.rb` for a file under the working
1023
+ # directory and an absolute path for one outside it. Reuse RSpec's own
1024
+ # definition of "relative to the project root" rather than inventing a
1025
+ # second one, then drop the `./` so the value matches what a linter, a
1026
+ # `git diff --name-only` and the platform all use as a file's identity.
1027
+ def relative_path(path)
1028
+ return nil if path.nil?
1029
+
1030
+ string = path.to_s
1031
+ return nil if string.empty?
1032
+
1033
+ (::RSpec::Core::Metadata.relative_path(string) || string).sub(%r{\A\./}, "")
1034
+ end
1035
+
1036
+ # Appends one line. Opened in append mode and written with a single call so
1037
+ # that two suites sharing an output path (parallel CI shards, say) interleave
1038
+ # whole runs rather than halves of one.
1039
+ def append(data)
1040
+ path = SpecGuard::RSpec.configuration.output_path
1041
+ append_to(data, path)
1042
+ end
1043
+
1044
+ # The keyless branch's sink: `local_output_path`, a local development
1045
+ # record deliberately kept apart from {#append}'s replay queue so that
1046
+ # `specguard-ingest log/test_results.jsonl` re-delivers only genuine
1047
+ # failed deliveries. See {Configuration#local_output_path}.
1048
+ def append_local(data)
1049
+ append_to(data, SpecGuard::RSpec.configuration.local_output_path)
1050
+ end
1051
+
1052
+ def append_to(data, path)
1053
+ directory = File.dirname(path)
1054
+ FileUtils.mkdir_p(directory) unless directory == "." || File.directory?(directory)
1055
+
1056
+ File.open(path, "a") { |file| file.write("#{JSON.generate(data)}\n") }
1057
+ end
1058
+
1059
+ # `Errno::*` and `IOError` — the sink's failure modes — are both
1060
+ # `StandardError` descendants, so they need no separate clause; they are
1061
+ # named in the ticket because they are the *reason* this rescue is here, not
1062
+ # because Ruby files them somewhere unusual.
1063
+ def never_fail_the_run
1064
+ yield
1065
+ rescue ScriptError, StandardError => e
1066
+ warn_once(e)
1067
+ nil
1068
+ end
1069
+
1070
+ # Once per run. A formatter that warned per example would bury the failure
1071
+ # output of the suite it is supposed to be reporting on under thousands of
1072
+ # identical lines — which is its own way of breaking somebody's CI.
1073
+ def warn_once(error)
1074
+ emit_warning("#{WARNING_PREFIX} (#{error.class}: #{error.message}). The test run is unaffected.")
1075
+ end
1076
+
1077
+ # The delivery half, through the same one-shot budget so a run still emits
1078
+ # at most one line however many things went wrong. `reason` names the HTTP
1079
+ # status when there was one — a 400 and a 401 call for entirely different
1080
+ # actions, and a warning that only said "delivery failed" would leave the
1081
+ # reader unable to tell which.
1082
+ def warn_delivery_failure(reason)
1083
+ path = SpecGuard::RSpec.configuration.output_path
1084
+
1085
+ emit_warning("#{DELIVERY_WARNING_PREFIX} (#{reason}). " \
1086
+ "Falling back to #{path}; the test run is unaffected.")
1087
+ end
1088
+
1089
+ def emit_warning(line)
1090
+ return if @warned
1091
+
1092
+ @warned = true
1093
+ @error_stream.puts(line)
1094
+ rescue ScriptError, StandardError
1095
+ # The warning stream itself is gone. There is nothing left to say, and
1096
+ # saying it is not worth failing the run over.
1097
+ nil
1098
+ end
1099
+
1100
+ def elapsed_since_start
1101
+ (monotonic_now - @started_at).round(6)
1102
+ end
1103
+
1104
+ # Monotonic: immune to an NTP correction landing mid-suite, which on the
1105
+ # wall clock can produce a negative duration.
1106
+ def monotonic_now
1107
+ Process.clock_gettime(Process::CLOCK_MONOTONIC)
1108
+ end
1109
+ end
1110
+ end