constable-rails 1.4.1 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +120 -1
- data/README.md +2 -2
- data/lib/constable/cli.rb +40 -2
- data/lib/constable/cold_case/rspec.rb +76 -4
- data/lib/constable/importer/modernizer.rb +52 -1
- data/lib/constable/result.rb +1 -1
- data/lib/constable/runner.rb +6 -0
- data/lib/constable/version.rb +1 -1
- data/lib/constable/worker_databases.rb +20 -1
- data/lib/generators/constable/templates/case_helper.rb.tt +66 -0
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4ee7e9d9303ba3b3e4977f529ab515059faa946e96209dae4e80dbef42d2930c
|
|
4
|
+
data.tar.gz: feadad150285bf9d68b11cc671b7c1bc13188c39a0b6c76317aef1a7a6ca9c70
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 71adc2fab90c1ce4b1239159f7000c2b2200dc7f6becc3d365f6c935dc1702a9c012557b4602c909de880f2728433cfdfc9b4bd1524b8ec25581459c4a4f415a
|
|
7
|
+
data.tar.gz: 70fd3761fed31d90e365e1b24940025ddd14dc6f80f0761d004cdfe6c69145b4f1806f4fe49753a61af4137677cc2313c9a3f7cd0ce2f284e121ada1e75d0fdf
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,123 @@ All notable changes to this project are documented here. This project adheres to
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [2.0.0]
|
|
9
|
+
|
|
10
|
+
A tidy-up release. The API you write — `investigate`, `witness`, `briefing`, `docket`,
|
|
11
|
+
`attest` — has not changed, and no case file or config key needs editing. The major
|
|
12
|
+
version is for one removed accessor and for the behaviour change in 1.4.0, both below.
|
|
13
|
+
|
|
14
|
+
### `constable modernize --cold`
|
|
15
|
+
|
|
16
|
+
A conversion that comes back flagged is **not runnable**: the flagged constructs are left
|
|
17
|
+
verbatim, so `let!` stays `let!` and the class body raises the moment it loads. That is
|
|
18
|
+
deliberate — what a `let!` should become is a decision, not a rewrite — but it leaves a
|
|
19
|
+
file stranded in `spec/` when the goal is one tree.
|
|
20
|
+
|
|
21
|
+
```console
|
|
22
|
+
$ constable modernize spec/models/tag_spec.rb --cold
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
writes `spec/models/tag_case.rb` as a cold case: one line at the top, the body byte for
|
|
26
|
+
byte, run through real RSpec. Move it into `test/cases/` and the file is ported without
|
|
27
|
+
being converted. Verified end to end — a `let!`-using spec that cannot convert runs 9/9
|
|
28
|
+
from `test/` untouched.
|
|
29
|
+
|
|
30
|
+
So a port is now: `modernize` a file, convert it if it comes back clean, `--cold` it if it
|
|
31
|
+
does not, and revisit later. Nothing has to stay behind.
|
|
32
|
+
|
|
33
|
+
### A case file that will not load says why
|
|
34
|
+
|
|
35
|
+
A half-converted case still containing `let!` reported:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
no tests matched test/cases/tmp
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
A file that raises while loading registers no investigations, so the selection came back
|
|
42
|
+
empty and the empty-selection guard — added in 1.0.0 to catch mistyped paths — claimed
|
|
43
|
+
the path was wrong. For a file that was right there and broken. It now reports the real
|
|
44
|
+
error:
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
NoMethodError: undefined method `let!' for TagCase:Class
|
|
48
|
+
This file never ran. Nothing in it was tested.
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### `constable jail release --all`
|
|
52
|
+
|
|
53
|
+
Emptying the docket took one command per test, which is unusable at the scale a docket
|
|
54
|
+
reaches — before 1.4.0 a pass/fail flip jailed a test by itself, and one real suite put 29
|
|
55
|
+
on the docket from a single run.
|
|
56
|
+
|
|
57
|
+
Releasing is safe, and that is the point: a jailed test is *skipped*, so a docket full of
|
|
58
|
+
tests nobody nominated is silently reducing what the suite covers. A bare `release` with
|
|
59
|
+
no locator stays a usage error rather than an implicit "everything".
|
|
60
|
+
|
|
61
|
+
### Removed: `Result#coverage`
|
|
62
|
+
|
|
63
|
+
**Breaking, and why the major.** It was never written or read anywhere in the gem, and it
|
|
64
|
+
did not survive `to_h`/`from_h` — so a value set on it vanished crossing the pipe from a
|
|
65
|
+
parallel worker, silently, and only in parallel. An attribute that loses data is worse
|
|
66
|
+
than no attribute. Coverage has always travelled separately and still does.
|
|
67
|
+
|
|
68
|
+
### Also breaking, from 1.4.0
|
|
69
|
+
|
|
70
|
+
`jail_flakes` defaults to **false**: a test that passed last run and failed this one is no
|
|
71
|
+
longer jailed automatically. Jailing skips the test on every later run, and doing that to
|
|
72
|
+
a test nobody nominated is how a suite quietly stops testing things. `--jail` is
|
|
73
|
+
unaffected.
|
|
74
|
+
|
|
75
|
+
### Test coverage
|
|
76
|
+
|
|
77
|
+
The suite is now **1,025 runs**, up from 738 at 1.0.0. This release adds permanent tests
|
|
78
|
+
for the three areas that had none and carried the most risk:
|
|
79
|
+
|
|
80
|
+
- **`Result`** — the wire format between a worker and the parent. Every attribute is
|
|
81
|
+
asserted to survive a real `Marshal` round trip, because anything that does not is data
|
|
82
|
+
lost only in parallel runs, which is the worst way to find out. That is how the
|
|
83
|
+
`coverage` bug above was found.
|
|
84
|
+
- **Cold-case shapes** — 17 real-world files run end to end through both engines: pending
|
|
85
|
+
and `xit`, `before`/`around` hooks, tags, aggregate failures, a raising hook, shared
|
|
86
|
+
examples, `RSpec.configure` inside a spec, deep nesting, the same file twice, and seven
|
|
87
|
+
Minitest arrangements. Every serious bug this project has had lived in these adapters.
|
|
88
|
+
- **`Identity`** — stability under reformatting and comments, distinctness on real change,
|
|
89
|
+
unicode, cold-case keys, and the disambiguation added in 1.0.0.
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
## [1.4.2]
|
|
93
|
+
|
|
94
|
+
### Cold cases run inside a parallel worker
|
|
95
|
+
|
|
96
|
+
The defect recorded as known in 1.4.1, now understood and fixed. Two bugs, stacked, and
|
|
97
|
+
the second one hid the first.
|
|
98
|
+
|
|
99
|
+
**RSpec swallows errors in required files.** `Configuration#requires=` routes through
|
|
100
|
+
`load_file_handling_errors`, which rescues anything raised while loading, reports it
|
|
101
|
+
through a formatter, and sets `world.wants_to_quit = true`. Every later
|
|
102
|
+
`ExampleGroup.run` then returns immediately. Constable points RSpec's streams at a
|
|
103
|
+
throwaway `StringIO` — stdout belongs to the reporter — so that report went nowhere: the
|
|
104
|
+
file loaded, the examples registered, and none of them ran. A forked worker produced no
|
|
105
|
+
results, no error and no exception, and exited 0. Requires are now loaded directly, where
|
|
106
|
+
an exception carries its own message and backtrace, and the quit flag is checked and
|
|
107
|
+
surfaced if anything sets it anyway.
|
|
108
|
+
|
|
109
|
+
**Not every database can be sharded.** With the error visible, the real cause was one
|
|
110
|
+
line: worker setup renamed *every* database config to `<database>_<index>`, including
|
|
111
|
+
connections that are not per-worker test data at all. Caseflow talks to a legacy Oracle
|
|
112
|
+
system (VACOLS) alongside its own Postgres databases, and appending `_3` to an Oracle TNS
|
|
113
|
+
service name produces:
|
|
114
|
+
|
|
115
|
+
```
|
|
116
|
+
OCIError: ORA-12162: TNS:net service name is incorrectly specified
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
raised inside a `before(:suite)` hook — which is exactly what RSpec was swallowing. Only
|
|
120
|
+
adapters that per-worker copies make sense for are renamed now (postgresql, postgis,
|
|
121
|
+
mysql2, trilogy, sqlite3). An external system stays shared by every worker, which is what
|
|
122
|
+
it is for.
|
|
123
|
+
|
|
124
|
+
|
|
8
125
|
## [1.4.1]
|
|
9
126
|
|
|
10
127
|
### A parallel run that ran nothing is no longer a pass
|
|
@@ -712,7 +829,9 @@ Initial release.
|
|
|
712
829
|
- Diff-based coverage gate — only lines changed in the current diff are held to the
|
|
713
830
|
threshold. `constable beat` for the full picture, `--html` for a browsable report.
|
|
714
831
|
|
|
715
|
-
[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.4.
|
|
832
|
+
[Unreleased]: https://github.com/Ray-Hughes/constable/compare/v1.4.3...HEAD
|
|
833
|
+
[1.4.3]: https://github.com/Ray-Hughes/constable/compare/v1.4.2...v1.4.3
|
|
834
|
+
[1.4.2]: https://github.com/Ray-Hughes/constable/compare/v1.4.1...v1.4.2
|
|
716
835
|
[1.4.1]: https://github.com/Ray-Hughes/constable/compare/v1.4.0...v1.4.1
|
|
717
836
|
[1.4.0]: https://github.com/Ray-Hughes/constable/compare/v1.3.3...v1.4.0
|
|
718
837
|
[1.3.3]: https://github.com/Ray-Hughes/constable/compare/v1.3.2...v1.3.3
|
data/README.md
CHANGED
|
@@ -393,7 +393,7 @@ worse than one that resets.
|
|
|
393
393
|
| `constable test PATH[:LINE]` | One file, or one investigation at that line |
|
|
394
394
|
| `constable test --unsafe` | Cold cases only |
|
|
395
395
|
| `constable test --jail` | The full run, in jail mode |
|
|
396
|
-
| `constable jail [run\|parole\|release]` | The docket |
|
|
396
|
+
| `constable jail [run\|parole\|release]` | The docket. `release --all` empties it |
|
|
397
397
|
| `constable warrants [release]` | Outstanding warrants |
|
|
398
398
|
| `constable watchlist` | Everything under supervision right now |
|
|
399
399
|
| `constable status` | How the suite is doing over time |
|
|
@@ -402,7 +402,7 @@ worse than one that resets.
|
|
|
402
402
|
| `constable prepare [--workers N]` | Build the per-worker test databases `worker_databases: reuse` needs |
|
|
403
403
|
| `constable prune [--dry-run]` | Forget docket rows and warrants for tests that no longer exist |
|
|
404
404
|
| `constable import --from=rspec` | Adopt an existing suite as cold cases |
|
|
405
|
-
| `constable modernize PATH` | Opt-in AST rewrite into the native DSL |
|
|
405
|
+
| `constable modernize PATH [--cold]` | Opt-in AST rewrite into the native DSL. `--cold` moves it verbatim instead |
|
|
406
406
|
|
|
407
407
|
Flags: `--full --unsafe --jail --warrants --coverage --seed N --workers N --verbose --tier T\n--expanded --concise --output MODE --no-color`.
|
|
408
408
|
|
data/lib/constable/cli.rb
CHANGED
|
@@ -197,6 +197,8 @@ module Constable
|
|
|
197
197
|
DESC
|
|
198
198
|
option :alongside, type: :boolean, default: false, desc: "Write PATH_case.rb beside the original"
|
|
199
199
|
option :"in-place", type: :boolean, default: false, desc: "Overwrite the file"
|
|
200
|
+
option :cold, type: :boolean, default: false,
|
|
201
|
+
desc: "Move it verbatim as a cold case instead of converting"
|
|
200
202
|
option :"show-source", type: :boolean, default: false, desc: "Print the rewritten source"
|
|
201
203
|
def modernize(*paths)
|
|
202
204
|
if paths.empty?
|
|
@@ -205,6 +207,7 @@ module Constable
|
|
|
205
207
|
end
|
|
206
208
|
|
|
207
209
|
mode = if options[:"in-place"] then :in_place
|
|
210
|
+
elsif options[:cold] then :cold
|
|
208
211
|
elsif options[:alongside] then :alongside
|
|
209
212
|
else :none
|
|
210
213
|
end
|
|
@@ -312,13 +315,48 @@ module Constable
|
|
|
312
315
|
say "Paroled. It runs normally now; one failure sends it straight back."
|
|
313
316
|
end
|
|
314
317
|
|
|
315
|
-
desc "release PATH:LINE", "Fully release a test, no supervision"
|
|
316
|
-
|
|
318
|
+
desc "release [PATH:LINE]", "Fully release a test, no supervision"
|
|
319
|
+
long_desc <<~DESC
|
|
320
|
+
Takes one test off the docket. `--all` empties it.
|
|
321
|
+
|
|
322
|
+
Emptying is the right move after a docket that filled up on its own -- before
|
|
323
|
+
1.4.0 a pass/fail flip jailed a test automatically, and a suite with
|
|
324
|
+
order-dependent tests could put dozens on it that nobody chose. Releasing is safe:
|
|
325
|
+
anything genuinely broken fails again on the next run, in the open.
|
|
326
|
+
DESC
|
|
327
|
+
option :all, type: :boolean, default: false, desc: "Release every test on the docket"
|
|
328
|
+
def release(locator = nil)
|
|
329
|
+
return release_all if options[:all]
|
|
330
|
+
|
|
331
|
+
if locator.nil?
|
|
332
|
+
CLI.complain("release needs a PATH:LINE, or --all to empty the docket")
|
|
333
|
+
exit(EXIT_USAGE)
|
|
334
|
+
end
|
|
335
|
+
|
|
317
336
|
act(locator) { |jail, identity| jail.release(identity) }
|
|
318
337
|
say "Released."
|
|
319
338
|
end
|
|
320
339
|
|
|
321
340
|
no_commands do
|
|
341
|
+
# Emptying the docket in one go. Right after a docket that filled up on its own --
|
|
342
|
+
# before 1.4.0 a pass/fail flip jailed a test automatically, and a suite with
|
|
343
|
+
# order-dependent tests could put dozens on it that nobody chose. Releasing is
|
|
344
|
+
# safe: anything genuinely broken fails again on the next run, in the open.
|
|
345
|
+
def release_all
|
|
346
|
+
jail = Jail.new(config: Constable.config, storage: Constable.storage)
|
|
347
|
+
entries = jail.entries
|
|
348
|
+
|
|
349
|
+
if entries.empty?
|
|
350
|
+
say "The docket is already empty."
|
|
351
|
+
return 0
|
|
352
|
+
end
|
|
353
|
+
|
|
354
|
+
entries.each { |entry| jail.release(entry.identity) }
|
|
355
|
+
say "Released #{entries.size} #{entries.size == 1 ? "test" : "tests"}. The docket is empty."
|
|
356
|
+
say "Anything genuinely broken will fail on the next run, where you can see it."
|
|
357
|
+
0
|
|
358
|
+
end
|
|
359
|
+
|
|
322
360
|
def act(locator)
|
|
323
361
|
jail = Jail.new(config: Constable.config, storage: Constable.storage)
|
|
324
362
|
refuse_ambiguous(locator, jail.candidates(locator), "on the docket")
|
|
@@ -120,6 +120,7 @@ module Constable
|
|
|
120
120
|
with_engine do
|
|
121
121
|
collector = Collector.new
|
|
122
122
|
load_error = capture_load(path)
|
|
123
|
+
load_error ||= quit_flag_error(path)
|
|
123
124
|
unless load_error
|
|
124
125
|
# After the file has loaded -- that load is what registers them -- and
|
|
125
126
|
# before its examples run. See #run_pending_before_suite_hooks.
|
|
@@ -306,10 +307,22 @@ module Constable
|
|
|
306
307
|
requires = rspec_option_requires
|
|
307
308
|
return if requires.empty?
|
|
308
309
|
|
|
309
|
-
# `requires
|
|
310
|
-
#
|
|
311
|
-
#
|
|
312
|
-
|
|
310
|
+
# Deliberately NOT `configuration.requires =`, which is RSpec's own accessor.
|
|
311
|
+
# That routes through Configuration#load_file_handling_errors, which rescues
|
|
312
|
+
# anything raised while loading, reports it through a formatter, and sets
|
|
313
|
+
# `world.wants_to_quit`. Since stdout belongs to Constable's reporter, RSpec's
|
|
314
|
+
# streams are a throwaway StringIO, so that report goes nowhere -- and every
|
|
315
|
+
# later ExampleGroup.run returns immediately. The file loads, the examples
|
|
316
|
+
# register, and none of them run. A forked worker produced no results, no error
|
|
317
|
+
# and no exception, and exited 0.
|
|
318
|
+
#
|
|
319
|
+
# So the load path is set up the same way and the requires are done here, where
|
|
320
|
+
# an exception is an exception and carries its own message and backtrace.
|
|
321
|
+
add_spec_load_paths!(configuration)
|
|
322
|
+
requires.each { |path| require path }
|
|
323
|
+
configuration.instance_variable_set(:@requires, requires)
|
|
324
|
+
rescue Constable::Error
|
|
325
|
+
raise
|
|
313
326
|
rescue StandardError => e
|
|
314
327
|
# A helper that will not load is the suite's problem to fix, and it will say so
|
|
315
328
|
# loudly on the first file. Constable's job here is not to disappear.
|
|
@@ -317,6 +330,53 @@ module Constable
|
|
|
317
330
|
"Cold cases will run without it.", kind: :cold_case)
|
|
318
331
|
end
|
|
319
332
|
|
|
333
|
+
# What `configuration.requires=` does before requiring anything: puts `lib` and the
|
|
334
|
+
# default path (`spec`) on the load path, which is what makes a bare
|
|
335
|
+
# `require "rails_helper"` resolve at all.
|
|
336
|
+
def add_spec_load_paths!(configuration)
|
|
337
|
+
default_path = configuration.default_path if configuration.respond_to?(:default_path)
|
|
338
|
+
["lib", default_path].compact.uniq.each do |dir|
|
|
339
|
+
absolute = File.expand_path(dir, Constable.root)
|
|
340
|
+
$LOAD_PATH.unshift(absolute) if File.directory?(absolute) && !$LOAD_PATH.include?(absolute)
|
|
341
|
+
end
|
|
342
|
+
end
|
|
343
|
+
|
|
344
|
+
# RSpec does not let an error in a required file reach you.
|
|
345
|
+
# Configuration#load_file_handling_errors rescues anything raised while loading,
|
|
346
|
+
# reports it through `notify_non_example_exception`, and sets
|
|
347
|
+
# `world.wants_to_quit = true`. Every later ExampleGroup.run then returns
|
|
348
|
+
# immediately, so the file loads, the examples register, and none of them run.
|
|
349
|
+
#
|
|
350
|
+
# We point RSpec's output at a throwaway StringIO -- stdout belongs to the
|
|
351
|
+
# reporter -- so that report goes nowhere. The result was a forked worker that
|
|
352
|
+
# produced no results, no error and no exception, and exited 0. It took a probe
|
|
353
|
+
# inside the child to find that `wants_to_quit` was the difference.
|
|
354
|
+
#
|
|
355
|
+
# So: ask, and put the swallowed message back in front of the user.
|
|
356
|
+
def raise_if_loading_failed!(configuration, requires)
|
|
357
|
+
return unless ::RSpec.world.wants_to_quit
|
|
358
|
+
|
|
359
|
+
# Clear it, or every later file in this session inherits the flag and runs
|
|
360
|
+
# nothing either.
|
|
361
|
+
::RSpec.world.wants_to_quit = false
|
|
362
|
+
|
|
363
|
+
raise Constable::Error,
|
|
364
|
+
"RSpec could not load #{requires.join(", ")} (from .rspec). " \
|
|
365
|
+
"#{swallowed_output(configuration)}".strip
|
|
366
|
+
end
|
|
367
|
+
|
|
368
|
+
# Whatever RSpec wrote about it before we could ask. Its streams are ours, so this
|
|
369
|
+
# is the only copy in existence.
|
|
370
|
+
def swallowed_output(configuration)
|
|
371
|
+
[configuration.error_stream, configuration.output_stream, configuration.deprecation_stream]
|
|
372
|
+
.uniq
|
|
373
|
+
.filter_map { |io| io.string.to_s.strip if io.respond_to?(:string) }
|
|
374
|
+
.reject(&:empty?)
|
|
375
|
+
.first.to_s
|
|
376
|
+
rescue StandardError
|
|
377
|
+
""
|
|
378
|
+
end
|
|
379
|
+
|
|
320
380
|
# Parsed by RSpec itself, so `.rspec`, `~/.rspec`, `.rspec-local` and SPEC_OPTS are
|
|
321
381
|
# all read with its precedence rather than a guess at the format.
|
|
322
382
|
def rspec_option_requires
|
|
@@ -372,6 +432,18 @@ module Constable
|
|
|
372
432
|
configuration.start_time = ::RSpec::Core::Time.now if configuration.respond_to?(:start_time=)
|
|
373
433
|
end
|
|
374
434
|
|
|
435
|
+
# Same swallowing, one level down: a `require` inside the spec file that fails is
|
|
436
|
+
# rescued by RSpec, flagged, and never surfaced. Without this the file reports as
|
|
437
|
+
# a clean run of zero tests.
|
|
438
|
+
def quit_flag_error(path)
|
|
439
|
+
return nil unless ::RSpec.world.wants_to_quit
|
|
440
|
+
|
|
441
|
+
::RSpec.world.wants_to_quit = false
|
|
442
|
+
Constable::Error.new(
|
|
443
|
+
"RSpec stopped while loading #{path}. #{swallowed_output(::RSpec.configuration)}".strip
|
|
444
|
+
)
|
|
445
|
+
end
|
|
446
|
+
|
|
375
447
|
# A file that won't even parse is news, not a crash. Report it as one errored
|
|
376
448
|
# result so the run keeps going and the summary names the file.
|
|
377
449
|
def capture_load(path)
|
|
@@ -28,7 +28,7 @@ module Constable
|
|
|
28
28
|
# :none -- dry run. Report only. The default.
|
|
29
29
|
# :alongside -- write foo_spec.rb's conversion to foo_case.rb, never clobbering.
|
|
30
30
|
# :in_place -- overwrite the original file.
|
|
31
|
-
WRITE_MODES = %i[none alongside in_place].freeze
|
|
31
|
+
WRITE_MODES = %i[none alongside in_place cold].freeze
|
|
32
32
|
|
|
33
33
|
GROUP_METHODS = %i[describe context xdescribe xcontext fdescribe fcontext feature].freeze
|
|
34
34
|
EXAMPLE_METHODS = %i[it specify example scenario].freeze
|
|
@@ -175,9 +175,60 @@ module Constable
|
|
|
175
175
|
File.join(dir, "#{base}_case.rb")
|
|
176
176
|
end
|
|
177
177
|
|
|
178
|
+
# `--cold`: move the file into the native tree without converting a line of it.
|
|
179
|
+
#
|
|
180
|
+
# A conversion that comes back flagged is not runnable -- the flagged constructs
|
|
181
|
+
# are left verbatim, so `let!` stays `let!` and the class body raises the moment it
|
|
182
|
+
# loads. That is deliberate: what a `let!` should become is a decision, not a
|
|
183
|
+
# rewrite. But it leaves a file stuck in spec/ when the goal is one tree.
|
|
184
|
+
#
|
|
185
|
+
# A cold case is the answer that already exists: one line at the top, the body
|
|
186
|
+
# untouched, run through real RSpec, results folded into the same report. This
|
|
187
|
+
# writes exactly that, so a port can move every file and convert the ones worth
|
|
188
|
+
# converting on its own schedule.
|
|
189
|
+
def cold_wrap(result, root)
|
|
190
|
+
target = alongside_path(result.path)
|
|
191
|
+
if File.exist?(target)
|
|
192
|
+
result.error = "refusing to overwrite #{target.delete_prefix("#{root}/")}"
|
|
193
|
+
return
|
|
194
|
+
end
|
|
195
|
+
|
|
196
|
+
FileUtils.mkdir_p(File.dirname(target))
|
|
197
|
+
File.write(target, cold_source(result))
|
|
198
|
+
result.written_to = target.delete_prefix("#{root}/")
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
# The original bytes, between a header line and a final `end`. Nothing inside is
|
|
202
|
+
# parsed, reindented or touched -- that is the whole promise of a cold case.
|
|
203
|
+
def cold_source(result)
|
|
204
|
+
<<~RUBY
|
|
205
|
+
# frozen_string_literal: true
|
|
206
|
+
|
|
207
|
+
# Moved verbatim from #{result.relative_path}. Runs through real RSpec, with its
|
|
208
|
+
# results folded into Constable's reporting, flake history and CI gate.
|
|
209
|
+
#
|
|
210
|
+
# Nothing inside has been converted, so every RSpec feature still works --
|
|
211
|
+
# `let!`, `before(:all)`, shared examples, rspec-mocks. Convert it with
|
|
212
|
+
# `constable modernize` when it is worth doing; there is no deadline.
|
|
213
|
+
class #{cold_class_name(result)} < Constable::ColdCase::RSpec
|
|
214
|
+
#{indent(result.original.to_s.rstrip)}
|
|
215
|
+
end
|
|
216
|
+
RUBY
|
|
217
|
+
end
|
|
218
|
+
|
|
219
|
+
def cold_class_name(result)
|
|
220
|
+
base = File.basename(result.path, ".rb").sub(/_(?:spec|test)\z/, "")
|
|
221
|
+
"Legacy#{base.split(%r{[_/]}).map(&:capitalize).join}Spec"
|
|
222
|
+
end
|
|
223
|
+
|
|
224
|
+
def indent(source)
|
|
225
|
+
source.lines.map { |line| line.strip.empty? ? line : " #{line}" }.join
|
|
226
|
+
end
|
|
227
|
+
|
|
178
228
|
private
|
|
179
229
|
|
|
180
230
|
def persist(result, root, write)
|
|
231
|
+
return cold_wrap(result, root) if write == :cold
|
|
181
232
|
return unless result.ok? && result.changed?
|
|
182
233
|
|
|
183
234
|
case write
|
data/lib/constable/result.rb
CHANGED
|
@@ -18,7 +18,7 @@ module Constable
|
|
|
18
18
|
|
|
19
19
|
attr_reader :identity, :case_name, :description, :file, :line, :kind, :tier
|
|
20
20
|
attr_accessor :status, :duration, :failure, :warnings, :retries, :jail_reason,
|
|
21
|
-
:parole_day, :times_jailed, :seed
|
|
21
|
+
:parole_day, :times_jailed, :seed
|
|
22
22
|
|
|
23
23
|
def initialize(identity:, case_name:, description:, file:, line:, kind: :native, tier: nil,
|
|
24
24
|
status: :passed, duration: 0.0, failure: nil, warnings: [], retries: [])
|
data/lib/constable/runner.rb
CHANGED
|
@@ -231,6 +231,12 @@ module Constable
|
|
|
231
231
|
def refuse_empty_selection!(items)
|
|
232
232
|
return unless items.empty?
|
|
233
233
|
return unless @selection.explicit?
|
|
234
|
+
# A file that raised while loading registers no investigations, so the selection
|
|
235
|
+
# comes back empty and this guard would report "no tests matched" -- burying the
|
|
236
|
+
# actual error, which is already captured and about to be reported as a failure.
|
|
237
|
+
# "Your path is wrong" is the wrong thing to say about a file that is right there
|
|
238
|
+
# and broken.
|
|
239
|
+
return if load_errors.any?
|
|
234
240
|
|
|
235
241
|
raise Constable::Error, @selection.empty_selection_message
|
|
236
242
|
end
|
data/lib/constable/version.rb
CHANGED
|
@@ -236,13 +236,32 @@ module Constable
|
|
|
236
236
|
# Every database config for this environment, renamed to its per-worker sibling.
|
|
237
237
|
# `_database=` is exactly how Rails' own TestDatabases does the renaming, and this
|
|
238
238
|
# runs in a forked child, so the mutation dies with the worker.
|
|
239
|
+
# Adapters where "give each worker its own copy" is a thing that can be done at all.
|
|
240
|
+
#
|
|
241
|
+
# An app can hold connections Constable has no business renaming. Caseflow talks to a
|
|
242
|
+
# legacy Oracle system (VACOLS) alongside its own Postgres databases; appending `_3` to
|
|
243
|
+
# an Oracle TNS service name produces `ORA-12162: TNS:net service name is incorrectly
|
|
244
|
+
# specified`, and since that surfaces inside a `before(:suite)` hook, RSpec swallowed
|
|
245
|
+
# it and the worker ran nothing at all.
|
|
246
|
+
#
|
|
247
|
+
# An external system like that is shared by every worker on purpose: it is not
|
|
248
|
+
# per-worker test data, and there is no per-worker copy of it to make.
|
|
249
|
+
SHARDABLE_ADAPTERS = %w[postgresql postgis mysql2 trilogy sqlite3].freeze
|
|
250
|
+
|
|
251
|
+
def shardable_adapter?(db_config)
|
|
252
|
+
return false unless db_config.respond_to?(:adapter)
|
|
253
|
+
|
|
254
|
+
SHARDABLE_ADAPTERS.include?(db_config.adapter.to_s)
|
|
255
|
+
end
|
|
256
|
+
|
|
239
257
|
def each_worker_config(index)
|
|
240
258
|
configs = ::ActiveRecord::Base.configurations.configs_for(env_name: env_name,
|
|
241
259
|
include_hidden: true)
|
|
242
260
|
configs.each do |db_config|
|
|
243
|
-
db_config._database = "#{db_config.database}_#{index}"
|
|
244
261
|
next unless db_config.database_tasks?
|
|
262
|
+
next unless shardable_adapter?(db_config)
|
|
245
263
|
|
|
264
|
+
db_config._database = "#{db_config.database}_#{index}"
|
|
246
265
|
yield db_config
|
|
247
266
|
end
|
|
248
267
|
end
|
|
@@ -180,6 +180,72 @@ end
|
|
|
180
180
|
# -----------------------------------------------------------------------------
|
|
181
181
|
|
|
182
182
|
Constable.configure do |c|
|
|
183
|
+
# --- Settings. Uncomment what you want; each is shown at its default. --------
|
|
184
|
+
|
|
185
|
+
# Workers default to `auto` -- processor count minus a little headroom, so the
|
|
186
|
+
# machine stays usable while the suite runs. Pin it when a CI container lies
|
|
187
|
+
# about its core count, or to rule parallelism out while debugging.
|
|
188
|
+
#
|
|
189
|
+
# c.parallel_workers = 4
|
|
190
|
+
|
|
191
|
+
# How a parallel worker gets a database of its own. Sharing one is not a
|
|
192
|
+
# speed/safety trade, it is a correctness bug: on SQLite the run dissolves into
|
|
193
|
+
# "database is locked", and on a client/server database tests quietly see each
|
|
194
|
+
# other's rows, which is worse because it looks like it worked.
|
|
195
|
+
#
|
|
196
|
+
# :schema rebuild <database>_<index> from schema.rb every run. What Rails
|
|
197
|
+
# does for `rails test`, and correct by construction -- the
|
|
198
|
+
# databases cannot drift, because they are thrown away each time.
|
|
199
|
+
# The default, and the wrong choice only when it is slow or
|
|
200
|
+
# impossible.
|
|
201
|
+
# :reuse keep them between runs. Faster (a big schema is not replayed per
|
|
202
|
+
# worker), and the ONLY option that works when your schema cannot
|
|
203
|
+
# rebuild the database by itself -- true of any app with Postgres
|
|
204
|
+
# custom types, since CREATE TYPE has no schema.rb representation.
|
|
205
|
+
# Run `constable prepare` once to build them, and again after a
|
|
206
|
+
# migration; keeping them current is your job.
|
|
207
|
+
# :off do not shard, so do not fork. An explicit serial run: no attempt,
|
|
208
|
+
# no warning. Correct, and as slow as one core.
|
|
209
|
+
#
|
|
210
|
+
# c.worker_databases = :reuse
|
|
211
|
+
|
|
212
|
+
# The live stream while the suite runs. The summary is identical either way.
|
|
213
|
+
# :concise one glyph per test, grouped per case (default)
|
|
214
|
+
# :expanded a line per test: glyph, description, duration -- so you can see
|
|
215
|
+
# which test is hanging while it hangs, not afterwards
|
|
216
|
+
#
|
|
217
|
+
# c.output = :expanded
|
|
218
|
+
|
|
219
|
+
# Put a test on the docket by itself when it passed last run and failed this
|
|
220
|
+
# one. Off by default, and the reason is what jailing does: a jailed test is
|
|
221
|
+
# SKIPPED on every later run. On automatically, a suite with order-dependent
|
|
222
|
+
# tests quietly stops running things nobody chose to stop running.
|
|
223
|
+
#
|
|
224
|
+
# c.jail_flakes = true
|
|
225
|
+
|
|
226
|
+
# Rerun a failure in isolation before believing it. A test that then passes is
|
|
227
|
+
# flaky rather than broken: it gets a warrant instead of failing the build, and
|
|
228
|
+
# stays loudly visible until someone deals with it.
|
|
229
|
+
#
|
|
230
|
+
# c.warrants = true
|
|
231
|
+
|
|
232
|
+
# CI: fail the build when the warning count is not trending down. Warnings are
|
|
233
|
+
# cold-case files and `unsafe` blocks -- allowed, but visible.
|
|
234
|
+
#
|
|
235
|
+
# c.fail_on_warnings = true
|
|
236
|
+
|
|
237
|
+
# Diff-based coverage: only lines changed in the current diff are held to it.
|
|
238
|
+
#
|
|
239
|
+
# c.coverage = true
|
|
240
|
+
# c.coverage_threshold = 90
|
|
241
|
+
|
|
242
|
+
# Your existing RSpec/Minitest files, run verbatim as cold cases through their
|
|
243
|
+
# own engine. `constable import --from=rspec` writes this for you.
|
|
244
|
+
#
|
|
245
|
+
# c.cold_cases = ["spec/**/*_spec.rb"]
|
|
246
|
+
|
|
247
|
+
# --- Code. This can only be written here. -----------------------------------
|
|
248
|
+
|
|
183
249
|
# One-time global setup, run once per process before the whole suite. For
|
|
184
250
|
# configuring the world -- drivers, adapters, formats -- and deliberately not
|
|
185
251
|
# for creating records that tests then share. See the note at the bottom of
|