turbo_tests2 3.1.14 → 3.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +57 -1
- data/README.md +105 -4
- data/lib/turbo_tests/cli.rb +142 -5
- data/lib/turbo_tests/runner.rb +103 -10
- data/lib/turbo_tests/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +4 -4
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e208abc2c8f1bc3b6d9d8891911baf8874d3c8e5cfa097b108d3e71bd722ca13
|
|
4
|
+
data.tar.gz: b4f3f282c992f71ad38aaf07323fc635a3f980c37b6b062236e4eabd2eea4909
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9b2880a10a0eba9651b65d450864fa5e7cf5fa327dc90a6554f5e812a9f5566d4bc46c8b12dc7c43e58834b7c77db51d87d34062c58b9566909d400ccaa913b2
|
|
7
|
+
data.tar.gz: e8b80088da4717d0db1fa889b038f3d8684d0262b6270750d6801cf2227d1e4b1d64233b0feafc035e3a9c38831e7ea488cc87ab6ba5ff206ddf72814308f6fb
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -30,6 +30,60 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
30
30
|
|
|
31
31
|
### Security
|
|
32
32
|
|
|
33
|
+
## [3.2.0] - 2026-07-18
|
|
34
|
+
|
|
35
|
+
- TAG: [v3.2.0][3.2.0t]
|
|
36
|
+
- COVERAGE: 96.88% -- 869/897 lines in 10 files
|
|
37
|
+
- BRANCH COVERAGE: 84.44% -- 217/257 branches in 10 files
|
|
38
|
+
- 34.69% documented
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
|
|
42
|
+
- `--count` is now accepted as an alias for `-n`, matching the
|
|
43
|
+
`parallel_tests` process-count option while keeping the existing
|
|
44
|
+
`-w` / `--workers` aliases.
|
|
45
|
+
- Documented the supported `parallel_tests` option subset and clarified that
|
|
46
|
+
unsupported flags are not blindly passed through after `--`.
|
|
47
|
+
- Added `--worker-output MODE` and `TURBO_TESTS2_WORKER_OUTPUT` support for
|
|
48
|
+
selecting raw worker stdout/stderr handling: `warnings`, `stream`, `buffered`,
|
|
49
|
+
or `quiet`.
|
|
50
|
+
- `--group-by runtime|filesize|found` is now parsed explicitly and mapped to
|
|
51
|
+
`parallel_tests` grouping modes, including when supplied after the
|
|
52
|
+
compatibility `--` separator.
|
|
53
|
+
- `--allowed-missing PERCENT` and `--unknown-runtime SECONDS` are now parsed
|
|
54
|
+
explicitly and mapped to `parallel_tests` runtime grouping tuning options,
|
|
55
|
+
including when supplied after the compatibility `--` separator.
|
|
56
|
+
|
|
57
|
+
### Changed
|
|
58
|
+
|
|
59
|
+
- Pull request engine workflows now keep JRuby and TruffleRuby coverage
|
|
60
|
+
available via branch opt-in prefixes: `jruby/*` runs JRuby workflows,
|
|
61
|
+
`truffleruby/*` runs TruffleRuby workflows, and `engines/*` runs all engine
|
|
62
|
+
workflows. Other pull request branches continue to run MRI checks without the
|
|
63
|
+
alternate engine jobs.
|
|
64
|
+
- The delayed documentation formatter spec now lives under fixtures so the
|
|
65
|
+
normal suite does not pay its fixed load-time delay.
|
|
66
|
+
|
|
67
|
+
### Fixed
|
|
68
|
+
|
|
69
|
+
- Successful non-verbose runs now surface warning and deprecation lines emitted
|
|
70
|
+
as raw worker stdout/stderr without dumping all buffered worker output.
|
|
71
|
+
- `heads.yml` and `dep-heads.yml` now gate JRuby and TruffleRuby PR coverage
|
|
72
|
+
at the job level, so non-matching PR branches skip those engine jobs before a
|
|
73
|
+
runner is provisioned.
|
|
74
|
+
- Direct `turbo_tests2` runs now refresh the default
|
|
75
|
+
`tmp/turbo_rspec_runtime.log` timing log, and `--runtime-log` writes the
|
|
76
|
+
requested timing log even when the run groups by filesize.
|
|
77
|
+
- `--only-group GROUP_INDEX[,GROUP_INDEX]` is now parsed explicitly and runs
|
|
78
|
+
only the selected 1-based `parallel_tests` group index(es), using filesize
|
|
79
|
+
grouping by default for selected-group reruns.
|
|
80
|
+
- `--pattern PATTERN` is now parsed explicitly and mapped to `parallel_tests`
|
|
81
|
+
file discovery instead of being treated as a spec file when supplied after
|
|
82
|
+
`--`.
|
|
83
|
+
- `--exclude-pattern PATTERN` is now parsed explicitly and mapped to
|
|
84
|
+
`parallel_tests` file discovery instead of being treated as a spec file when
|
|
85
|
+
supplied after `--`.
|
|
86
|
+
|
|
33
87
|
## [3.1.14] - 2026-07-17
|
|
34
88
|
|
|
35
89
|
- TAG: [v3.1.14][3.1.14t]
|
|
@@ -302,7 +356,9 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
302
356
|
|
|
303
357
|
- Initial release
|
|
304
358
|
|
|
305
|
-
[Unreleased]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.
|
|
359
|
+
[Unreleased]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.0...HEAD
|
|
360
|
+
[3.2.0]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.1.14...v3.2.0
|
|
361
|
+
[3.2.0t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.2.0
|
|
306
362
|
[3.1.14]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.1.13...v3.1.14
|
|
307
363
|
[3.1.14t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.1.14
|
|
308
364
|
[3.1.13]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.1.12...v3.1.13
|
data/README.md
CHANGED
|
@@ -196,14 +196,23 @@ Usage: turbo_tests2 [options]
|
|
|
196
196
|
turbo_tests2 spec/bar spec/baz/xxx_spec.rb
|
|
197
197
|
|
|
198
198
|
Options:
|
|
199
|
-
-n,
|
|
199
|
+
-n, --count [PROCESSES] How many processes to use, default: available CPUs
|
|
200
|
+
-w, --workers [PROCESSES] Alias for -n, --count
|
|
200
201
|
-r, --require PATH Require a file.
|
|
201
202
|
-f, --format FORMATTER Choose a formatter. Available formatters: progress (p), documentation (d). Default: progress
|
|
202
203
|
-t, --tag TAG Run examples with the specified tag.
|
|
203
204
|
-o, --out FILE Write output to a file instead of $stdout
|
|
204
205
|
--runtime-log FILE Location of previously recorded test runtimes
|
|
205
206
|
--example-status-log FILE Use RSpec example status persistence timings for grouping
|
|
207
|
+
--pattern PATTERN Run spec files matching this regex pattern
|
|
208
|
+
--exclude-pattern PATTERN Exclude spec files matching this regex pattern
|
|
209
|
+
--only-group GROUP_INDEX[,GROUP_INDEX]
|
|
210
|
+
Run only the selected 1-based parallel_tests group index(es)
|
|
211
|
+
--group-by MODE Group files by runtime, filesize, or found order
|
|
212
|
+
--allowed-missing PERCENT Allowed percentage of missing runtimes for runtime grouping
|
|
213
|
+
--unknown-runtime SECONDS Runtime in seconds to assign files missing runtime history
|
|
206
214
|
-v, --verbose More output
|
|
215
|
+
--worker-output MODE Raw worker output mode: warnings, stream, buffered, or quiet; env: TURBO_TESTS2_WORKER_OUTPUT
|
|
207
216
|
--fail-fast=[N]
|
|
208
217
|
--seed SEED Seed for RSpec
|
|
209
218
|
--order ORDER RSpec example order: random (default) or defined
|
|
@@ -217,12 +226,104 @@ By default, `turbo_tests2` generates one random RSpec seed, prints it, and passe
|
|
|
217
226
|
that same seed to every worker process. Use `--seed SEED` to replay a run, or
|
|
218
227
|
`--order defined` / `--no-random` when you want ordered examples without a seed.
|
|
219
228
|
|
|
220
|
-
|
|
229
|
+
`turbo_tests2` supports selected `parallel_tests` options explicitly. It does
|
|
230
|
+
not invoke the `parallel_tests` CLI directly, and it does not blindly pass
|
|
231
|
+
unknown `parallel_tests` flags through after `--`. Options after `--` are parsed
|
|
232
|
+
only when listed as supported below.
|
|
233
|
+
|
|
234
|
+
To select or exclude files matching a regex:
|
|
235
|
+
|
|
236
|
+
```bash
|
|
237
|
+
bundle exec turbo_tests2 -n 4 --pattern spec/system
|
|
238
|
+
bundle exec turbo_tests2 -n 4 --exclude-pattern spec/system
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
The compatibility separator also accepts supported options:
|
|
221
242
|
|
|
222
243
|
```bash
|
|
223
|
-
bundle exec turbo_tests2 -n 4 -- --
|
|
244
|
+
bundle exec turbo_tests2 -n 4 -- --pattern spec/system
|
|
245
|
+
bundle exec turbo_tests2 -n 4 -- --exclude-pattern spec/system
|
|
224
246
|
```
|
|
225
247
|
|
|
248
|
+
To rerun one or more selected 1-based groups from a prior parallel split:
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
bundle exec turbo_tests2 -n 4 --only-group 2
|
|
252
|
+
bundle exec turbo_tests2 -n 4 -- --only-group 1,3
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
To choose a supported `parallel_tests` grouping mode explicitly:
|
|
256
|
+
|
|
257
|
+
```bash
|
|
258
|
+
bundle exec turbo_tests2 -n 4 --group-by runtime
|
|
259
|
+
bundle exec turbo_tests2 -n 4 --group-by filesize
|
|
260
|
+
bundle exec turbo_tests2 -n 4 --group-by found
|
|
261
|
+
bundle exec turbo_tests2 -n 4 -- --group-by runtime
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
`runtime` grouping uses the configured runtime log, which `turbo_tests2`
|
|
265
|
+
continues to record by default. `filesize` and `found` do not use runtime
|
|
266
|
+
history for the initial split, but worker runtime recording still runs so
|
|
267
|
+
future runtime-based runs can use the updated log.
|
|
268
|
+
|
|
269
|
+
Runtime grouping also supports the explicit `parallel_tests` tuning flags
|
|
270
|
+
`--allowed-missing PERCENT` and `--unknown-runtime SECONDS`:
|
|
271
|
+
|
|
272
|
+
```bash
|
|
273
|
+
bundle exec turbo_tests2 -n 4 --group-by runtime --allowed-missing 25
|
|
274
|
+
bundle exec turbo_tests2 -n 4 --group-by runtime --unknown-runtime 0.5
|
|
275
|
+
bundle exec turbo_tests2 -n 4 -- --allowed-missing 25 --unknown-runtime 0.5
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Selected group runs default to filesize grouping unless `--group-by` is
|
|
279
|
+
provided.
|
|
280
|
+
|
|
281
|
+
### Supported `parallel_tests` Options
|
|
282
|
+
|
|
283
|
+
| `parallel_tests` option | Status | `turbo_tests2` equivalent | Notes |
|
|
284
|
+
|-------------------------|--------|---------------------------|-------|
|
|
285
|
+
| `-n`, `--count PROCESSES` | Supported natively | `-n`, `--count`; `-w`, `--workers` aliases | Parsed as a `turbo_tests2` option before any `--` separator. |
|
|
286
|
+
| `-p`, `--pattern PATTERN` | Supported | `--pattern PATTERN` | May be supplied before or after `--`; mapped to `parallel_tests` file discovery. |
|
|
287
|
+
| `--exclude-pattern PATTERN` | Supported | `--exclude-pattern PATTERN` | May be supplied before or after `--`; mapped to `parallel_tests` file discovery. |
|
|
288
|
+
| `--only-group GROUP_INDEX[,GROUP_INDEX]` | Supported | `--only-group GROUP_INDEX[,GROUP_INDEX]` | May be supplied before or after `--`; uses 1-based `parallel_tests` group indexes. |
|
|
289
|
+
| `--group-by runtime` | Supported | `--group-by runtime` | May be supplied before or after `--`; uses runtime-log grouping. |
|
|
290
|
+
| `--group-by filesize` | Supported | `--group-by filesize` | May be supplied before or after `--`; groups by file size. |
|
|
291
|
+
| `--group-by found` | Supported | `--group-by found` | May be supplied before or after `--`; keeps discovered file order. |
|
|
292
|
+
| `--allowed-missing PERCENT` | Supported | `--allowed-missing PERCENT` | May be supplied before or after `--`; validates `0..100`. |
|
|
293
|
+
| `--unknown-runtime SECONDS` | Supported | `--unknown-runtime SECONDS` | May be supplied before or after `--`; validates finite non-negative seconds. |
|
|
294
|
+
| `--runtime-log PATH` | Supported natively | `--runtime-log FILE` | Parsed as a `turbo_tests2` option before any `--` separator. |
|
|
295
|
+
| `--nice` | Supported natively | `--nice` | Parsed as a `turbo_tests2` option before any `--` separator. |
|
|
296
|
+
| `--fail-fast` | Supported natively | `--fail-fast[=N]` | Uses `turbo_tests2` fail-fast handling and RSpec worker arguments. |
|
|
297
|
+
| `--verbose` | Supported natively | `-v`, `--verbose` | Enables `turbo_tests2` verbose output. |
|
|
298
|
+
| `--test-options`, `-o` | Not supported as pass-through | Use `turbo_tests2` RSpec-facing flags | `turbo_tests2` exposes selected RSpec options directly, such as `--seed`, `--order`, `--no-random`, `--tag`, `--format`, and `--out`. |
|
|
299
|
+
| `--group-by default`, `steps`, `scenarios` | Not supported | None | `turbo_tests2` targets RSpec file grouping, not every `parallel_tests` framework mode. |
|
|
300
|
+
| `--single`, `--isolate`, `--isolate-n` | Not supported | None | No first-class mapping yet. |
|
|
301
|
+
| `--specify-groups` | Not supported | None | No first-class mapping yet. |
|
|
302
|
+
| `-m`, `--multiply-processes` | Not supported | None | Use an explicit `-n`/`--count` value. |
|
|
303
|
+
| `-e`, `--exec`, `--exec-args`, `--non-parallel` | Not supported | None | `turbo_tests2` runs RSpec workers, not arbitrary command groups. |
|
|
304
|
+
| `--suffix`, `--type`, `--ignore-tags` | Not supported | None | `turbo_tests2` currently targets RSpec. |
|
|
305
|
+
| `--serialize-stdout`, `--prefix-output-with-test-env-number`, `--combine-stderr`, `--quiet`, `--verbose-command`, `--verbose-process-command`, `--verbose-rerun-command` | Not supported | None | `turbo_tests2` owns worker output collection and formatter reporting. |
|
|
306
|
+
| `--no-symlinks`, `--allow-duplicates`, `--first-is-1`, `--highest-exit-status`, `--failure-exit-code`, `--test-file-limit` | Not supported | None | These need explicit design and tests before they can be accepted. |
|
|
307
|
+
|
|
308
|
+
### Worker Output Modes
|
|
309
|
+
|
|
310
|
+
Raw worker stdout/stderr is separate from the parent process formatter output.
|
|
311
|
+
The progress dot parade is reconstructed from JSON formatter events and already
|
|
312
|
+
streams from the parent process as workers report examples. Use
|
|
313
|
+
`--worker-output MODE`, or `TURBO_TESTS2_WORKER_OUTPUT=MODE`, to choose how raw
|
|
314
|
+
worker stdout/stderr is handled. A CLI flag takes precedence over the
|
|
315
|
+
environment variable.
|
|
316
|
+
|
|
317
|
+
| Mode | Behavior |
|
|
318
|
+
|------|----------|
|
|
319
|
+
| `warnings` | Default. Buffer raw worker output, print warning/deprecation lines after successful runs, and print full buffered raw output on failures. |
|
|
320
|
+
| `stream` | Stream raw worker stdout/stderr immediately without enabling verbose command logging. Buffered output is not replayed at the end. |
|
|
321
|
+
| `buffered` | Buffer raw worker output and print all buffered output after the run completes. |
|
|
322
|
+
| `quiet` | Suppress raw worker output on successful runs. Failed runs still print buffered raw output. |
|
|
323
|
+
|
|
324
|
+
Known coverage formatter chatter remains collapsed into the concise coverage
|
|
325
|
+
summary.
|
|
326
|
+
|
|
226
327
|
`turbo_tests2` supports custom formatter such as Fuubar, but you might need to require it:
|
|
227
328
|
|
|
228
329
|
```bash
|
|
@@ -666,7 +767,7 @@ Thanks for RTFM. ☺️
|
|
|
666
767
|
[📌gitmoji]: https://gitmoji.dev
|
|
667
768
|
[📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
|
|
668
769
|
[🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
|
669
|
-
[🧮kloc-img]: https://img.shields.io/badge/KLOC-0.
|
|
770
|
+
[🧮kloc-img]: https://img.shields.io/badge/KLOC-0.897-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
|
|
670
771
|
[🔐security]: https://github.com/galtzo-floss/turbo_tests2/blob/main/SECURITY.md
|
|
671
772
|
[🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
|
|
672
773
|
[📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
|
data/lib/turbo_tests/cli.rb
CHANGED
|
@@ -25,6 +25,9 @@ module TurboTests
|
|
|
25
25
|
print_failed_group = false
|
|
26
26
|
create = false
|
|
27
27
|
nice = false
|
|
28
|
+
worker_output = nil
|
|
29
|
+
parallel_options = {}
|
|
30
|
+
cli_args, parallel_args = split_parallel_args(@argv)
|
|
28
31
|
|
|
29
32
|
OptionParser.new do |opts|
|
|
30
33
|
opts.banner = <<~BANNER
|
|
@@ -42,7 +45,16 @@ module TurboTests
|
|
|
42
45
|
Options:
|
|
43
46
|
BANNER
|
|
44
47
|
|
|
45
|
-
opts.on(
|
|
48
|
+
opts.on(
|
|
49
|
+
"-n [PROCESSES]",
|
|
50
|
+
"--count [PROCESSES]",
|
|
51
|
+
Integer,
|
|
52
|
+
"How many processes to use, default: available CPUs"
|
|
53
|
+
) do |n|
|
|
54
|
+
count = n
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
opts.on("-w [PROCESSES]", "--workers [PROCESSES]", Integer, "Alias for -n, --count") do |n|
|
|
46
58
|
count = n
|
|
47
59
|
end
|
|
48
60
|
|
|
@@ -83,10 +95,41 @@ module TurboTests
|
|
|
83
95
|
example_status_log = filename
|
|
84
96
|
end
|
|
85
97
|
|
|
98
|
+
opts.on("--pattern PATTERN", "Run spec files matching this regex pattern") do |pattern|
|
|
99
|
+
parallel_options[:pattern] = compile_pattern(pattern)
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
opts.on("--exclude-pattern PATTERN", "Exclude spec files matching this regex pattern") do |pattern|
|
|
103
|
+
parallel_options[:exclude_pattern] = compile_pattern(pattern)
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
opts.on("--only-group GROUP_INDEX[,GROUP_INDEX]", "Run only the selected 1-based parallel_tests group index(es)") do |groups|
|
|
107
|
+
parallel_options[:only_group] = parse_only_groups(groups)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
opts.on("--group-by MODE", "Group files by runtime, filesize, or found order") do |mode|
|
|
111
|
+
parallel_options[:group_by] = parse_group_by(mode)
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
opts.on("--allowed-missing PERCENT", Integer, "Allowed percentage of missing runtimes for runtime grouping") do |percent|
|
|
115
|
+
parallel_options[:allowed_missing_percent] = parse_allowed_missing(percent)
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
opts.on("--unknown-runtime SECONDS", Float, "Runtime in seconds to assign files missing runtime history") do |seconds|
|
|
119
|
+
parallel_options[:unknown_runtime] = parse_unknown_runtime(seconds)
|
|
120
|
+
end
|
|
121
|
+
|
|
86
122
|
opts.on("-v", "--verbose", "More output") do
|
|
87
123
|
verbose = true
|
|
88
124
|
end
|
|
89
125
|
|
|
126
|
+
opts.on(
|
|
127
|
+
"--worker-output MODE",
|
|
128
|
+
"Raw worker output mode: warnings, stream, buffered, or quiet; env: TURBO_TESTS2_WORKER_OUTPUT"
|
|
129
|
+
) do |mode|
|
|
130
|
+
worker_output = mode
|
|
131
|
+
end
|
|
132
|
+
|
|
90
133
|
opts.on("--fail-fast=[N]") do |n|
|
|
91
134
|
n = begin
|
|
92
135
|
Integer(n)
|
|
@@ -119,7 +162,9 @@ module TurboTests
|
|
|
119
162
|
opts.on("--nice", "execute test commands with low priority") do
|
|
120
163
|
nice = true
|
|
121
164
|
end
|
|
122
|
-
end.parse!(
|
|
165
|
+
end.parse!(cli_args)
|
|
166
|
+
|
|
167
|
+
parse_parallel_args(parallel_args, parallel_options) unless parallel_args.empty?
|
|
123
168
|
|
|
124
169
|
if create
|
|
125
170
|
return TurboTests::Runner.create(count)
|
|
@@ -142,8 +187,7 @@ module TurboTests
|
|
|
142
187
|
|
|
143
188
|
invoke_rake_hook("setup")
|
|
144
189
|
|
|
145
|
-
files =
|
|
146
|
-
parallel_options = {}
|
|
190
|
+
files = cli_args.empty? ? nil : cli_args
|
|
147
191
|
|
|
148
192
|
exitstatus = TurboTests::Runner.run(
|
|
149
193
|
formatters: formatters,
|
|
@@ -158,6 +202,7 @@ module TurboTests
|
|
|
158
202
|
order: order,
|
|
159
203
|
nice: nice,
|
|
160
204
|
print_failed_group: print_failed_group,
|
|
205
|
+
worker_output: worker_output,
|
|
161
206
|
parallel_options: parallel_options
|
|
162
207
|
)
|
|
163
208
|
|
|
@@ -177,12 +222,104 @@ module TurboTests
|
|
|
177
222
|
@argv.first == "fan"
|
|
178
223
|
end
|
|
179
224
|
|
|
225
|
+
def split_parallel_args(args)
|
|
226
|
+
separator_index = args.index("--")
|
|
227
|
+
return [args, []] unless separator_index
|
|
228
|
+
|
|
229
|
+
[
|
|
230
|
+
args[0...separator_index],
|
|
231
|
+
args[(separator_index + 1)..-1]
|
|
232
|
+
]
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
def parse_parallel_args(args, parallel_options)
|
|
236
|
+
OptionParser.new do |opts|
|
|
237
|
+
opts.on("--pattern PATTERN", "Run spec files matching this regex pattern") do |pattern|
|
|
238
|
+
parallel_options[:pattern] = compile_pattern(pattern)
|
|
239
|
+
end
|
|
240
|
+
|
|
241
|
+
opts.on("--exclude-pattern PATTERN", "Exclude spec files matching this regex pattern") do |pattern|
|
|
242
|
+
parallel_options[:exclude_pattern] = compile_pattern(pattern)
|
|
243
|
+
end
|
|
244
|
+
|
|
245
|
+
opts.on("--only-group GROUP_INDEX[,GROUP_INDEX]", "Run only the selected 1-based parallel_tests group index(es)") do |groups|
|
|
246
|
+
parallel_options[:only_group] = parse_only_groups(groups)
|
|
247
|
+
end
|
|
248
|
+
|
|
249
|
+
opts.on("--group-by MODE", "Group files by runtime, filesize, or found order") do |mode|
|
|
250
|
+
parallel_options[:group_by] = parse_group_by(mode)
|
|
251
|
+
end
|
|
252
|
+
|
|
253
|
+
opts.on("--allowed-missing PERCENT", Integer, "Allowed percentage of missing runtimes for runtime grouping") do |percent|
|
|
254
|
+
parallel_options[:allowed_missing_percent] = parse_allowed_missing(percent)
|
|
255
|
+
end
|
|
256
|
+
|
|
257
|
+
opts.on("--unknown-runtime SECONDS", Float, "Runtime in seconds to assign files missing runtime history") do |seconds|
|
|
258
|
+
parallel_options[:unknown_runtime] = parse_unknown_runtime(seconds)
|
|
259
|
+
end
|
|
260
|
+
end.parse!(args)
|
|
261
|
+
|
|
262
|
+
return if args.empty?
|
|
263
|
+
|
|
264
|
+
raise OptionParser::InvalidArgument, "unsupported parallel_tests argument(s): #{args.join(" ")}"
|
|
265
|
+
end
|
|
266
|
+
|
|
267
|
+
def compile_pattern(pattern)
|
|
268
|
+
/#{pattern}/
|
|
269
|
+
rescue RegexpError => error
|
|
270
|
+
raise OptionParser::InvalidArgument, "invalid regex pattern #{pattern.inspect}: #{error.message}"
|
|
271
|
+
end
|
|
272
|
+
|
|
273
|
+
def parse_only_groups(groups)
|
|
274
|
+
groups.to_s.split(",").map do |group|
|
|
275
|
+
group = group.strip
|
|
276
|
+
raise OptionParser::InvalidArgument, "invalid group index #{group.inspect}" unless group.match?(/\A[1-9]\d*\z/)
|
|
277
|
+
|
|
278
|
+
group.to_i
|
|
279
|
+
end
|
|
280
|
+
end
|
|
281
|
+
|
|
282
|
+
def parse_group_by(mode)
|
|
283
|
+
value = mode.to_s.strip
|
|
284
|
+
supported_modes = %w[runtime filesize found]
|
|
285
|
+
unless supported_modes.include?(value)
|
|
286
|
+
raise OptionParser::InvalidArgument, "invalid group-by mode #{value.inspect}; expected one of: #{supported_modes.join(", ")}"
|
|
287
|
+
end
|
|
288
|
+
|
|
289
|
+
value.to_sym
|
|
290
|
+
end
|
|
291
|
+
|
|
292
|
+
def parse_allowed_missing(percent)
|
|
293
|
+
unless (0..100).cover?(percent)
|
|
294
|
+
raise OptionParser::InvalidArgument, "invalid allowed missing percent #{percent.inspect}; expected 0 through 100"
|
|
295
|
+
end
|
|
296
|
+
|
|
297
|
+
percent
|
|
298
|
+
end
|
|
299
|
+
|
|
300
|
+
def parse_unknown_runtime(seconds)
|
|
301
|
+
unless seconds.finite? && !seconds.negative?
|
|
302
|
+
raise OptionParser::InvalidArgument, "invalid unknown runtime #{seconds.inspect}; expected a finite non-negative number"
|
|
303
|
+
end
|
|
304
|
+
|
|
305
|
+
seconds
|
|
306
|
+
end
|
|
307
|
+
|
|
180
308
|
def handle_fan_command
|
|
181
309
|
args = @argv.drop(1)
|
|
182
310
|
count = nil
|
|
183
311
|
parser = OptionParser.new do |opts|
|
|
184
312
|
opts.banner = "Usage: turbo_tests2 fan [options] COMMAND [ARGS]"
|
|
185
|
-
opts.on(
|
|
313
|
+
opts.on(
|
|
314
|
+
"-n [PROCESSES]",
|
|
315
|
+
"--count [PROCESSES]",
|
|
316
|
+
Integer,
|
|
317
|
+
"How many processes to use, default: available CPUs"
|
|
318
|
+
) do |n|
|
|
319
|
+
count = n
|
|
320
|
+
end
|
|
321
|
+
|
|
322
|
+
opts.on("-w [PROCESSES]", "--workers [PROCESSES]", Integer, "Alias for -n, --count") do |n|
|
|
186
323
|
count = n
|
|
187
324
|
end
|
|
188
325
|
end
|
data/lib/turbo_tests/runner.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "json"
|
|
4
|
+
require "fileutils"
|
|
4
5
|
require "parallel_tests/rspec/runner"
|
|
5
6
|
require "rspec/core"
|
|
6
7
|
require "shellwords"
|
|
@@ -11,6 +12,9 @@ require_relative "../utils/hash_extension"
|
|
|
11
12
|
module TurboTests
|
|
12
13
|
class Runner
|
|
13
14
|
using CoreExtensions
|
|
15
|
+
DEFAULT_RUNTIME_LOG = "tmp/turbo_rspec_runtime.log"
|
|
16
|
+
DEFAULT_WORKER_OUTPUT_MODE = :warnings
|
|
17
|
+
WORKER_OUTPUT_MODES = %i[warnings stream buffered quiet].freeze
|
|
14
18
|
|
|
15
19
|
class << self
|
|
16
20
|
def create(count)
|
|
@@ -33,7 +37,7 @@ module TurboTests
|
|
|
33
37
|
parallel_options = opts[:parallel_options] || {}
|
|
34
38
|
|
|
35
39
|
start_time = opts.fetch(:start_time) { RSpec::Core::Time.now }
|
|
36
|
-
runtime_log = opts.fetch(:runtime_log, nil)
|
|
40
|
+
runtime_log = opts.fetch(:runtime_log, nil) || DEFAULT_RUNTIME_LOG
|
|
37
41
|
example_status_log = opts.fetch(:example_status_log, nil)
|
|
38
42
|
verbose = opts.fetch(:verbose, false)
|
|
39
43
|
fail_fast = opts.fetch(:fail_fast, nil)
|
|
@@ -44,17 +48,22 @@ module TurboTests
|
|
|
44
48
|
seed = generate_seed if seed_used && seed.nil?
|
|
45
49
|
print_failed_group = opts.fetch(:print_failed_group, false)
|
|
46
50
|
nice = opts.fetch(:nice, false)
|
|
51
|
+
worker_output = normalize_worker_output_mode(
|
|
52
|
+
opts[:worker_output] || ENV["TURBO_TESTS2_WORKER_OUTPUT"]
|
|
53
|
+
)
|
|
47
54
|
|
|
48
55
|
use_runtime_info = default_file_discovery
|
|
56
|
+
parallel_options[:runtime_log] ||= runtime_log
|
|
49
57
|
|
|
50
58
|
if example_status_log
|
|
51
59
|
runtime_log = runtime_log_from_example_status(example_status_log)
|
|
52
60
|
parallel_options[:runtime_log] = runtime_log
|
|
53
61
|
elsif use_runtime_info
|
|
54
|
-
parallel_options[:runtime_log]
|
|
62
|
+
parallel_options[:runtime_log] ||= runtime_log
|
|
55
63
|
else
|
|
56
|
-
parallel_options[:group_by]
|
|
64
|
+
parallel_options[:group_by] ||= :filesize
|
|
57
65
|
end
|
|
66
|
+
parallel_options[:group_by] ||= :filesize if parallel_options[:only_group]
|
|
58
67
|
|
|
59
68
|
warn("VERBOSE") if verbose
|
|
60
69
|
|
|
@@ -78,10 +87,22 @@ module TurboTests
|
|
|
78
87
|
print_failed_group: print_failed_group,
|
|
79
88
|
use_runtime_info: use_runtime_info,
|
|
80
89
|
parallel_options: parallel_options,
|
|
81
|
-
nice: nice
|
|
90
|
+
nice: nice,
|
|
91
|
+
worker_output: worker_output
|
|
82
92
|
).run
|
|
83
93
|
end
|
|
84
94
|
|
|
95
|
+
def normalize_worker_output_mode(mode)
|
|
96
|
+
value = mode.to_s.strip.downcase.tr("-", "_")
|
|
97
|
+
return DEFAULT_WORKER_OUTPUT_MODE if value.empty?
|
|
98
|
+
|
|
99
|
+
normalized = value.to_sym
|
|
100
|
+
return normalized if WORKER_OUTPUT_MODES.include?(normalized)
|
|
101
|
+
|
|
102
|
+
raise ArgumentError,
|
|
103
|
+
"Unsupported worker output mode #{mode.inspect}; expected one of: #{WORKER_OUTPUT_MODES.join(", ")}"
|
|
104
|
+
end
|
|
105
|
+
|
|
85
106
|
def normalize_rspec_file_selection(files)
|
|
86
107
|
selectors = {}
|
|
87
108
|
files.each do |entry|
|
|
@@ -200,10 +221,10 @@ module TurboTests
|
|
|
200
221
|
|
|
201
222
|
# Supports runtime_log as a top level option,
|
|
202
223
|
# but also nested inside parallel_options
|
|
203
|
-
@runtime_log = opts[:runtime_log] ||
|
|
224
|
+
@runtime_log = opts[:runtime_log] || DEFAULT_RUNTIME_LOG
|
|
204
225
|
@parallel_options = opts.fetch(:parallel_options, {})
|
|
205
226
|
@parallel_options[:runtime_log] ||= @runtime_log
|
|
206
|
-
@record_runtime =
|
|
227
|
+
@record_runtime = true
|
|
207
228
|
|
|
208
229
|
@messages = Thread::Queue.new
|
|
209
230
|
@threads = []
|
|
@@ -214,10 +235,12 @@ module TurboTests
|
|
|
214
235
|
@deferred_run_options_messages = Hash.new { |hash, message| hash[message] = [] }
|
|
215
236
|
@error = false
|
|
216
237
|
@print_failed_group = opts[:print_failed_group]
|
|
238
|
+
@worker_output_mode = self.class.normalize_worker_output_mode(opts.fetch(:worker_output, DEFAULT_WORKER_OUTPUT_MODE))
|
|
217
239
|
end
|
|
218
240
|
|
|
219
241
|
def run
|
|
220
|
-
|
|
242
|
+
parallel_tests_options = @parallel_options.reject { |key, _value| key == :only_group }
|
|
243
|
+
tests_with_size = ParallelTests::RSpec::Runner.tests_with_size(@files, parallel_tests_options)
|
|
221
244
|
@num_processes = [
|
|
222
245
|
ParallelTests.determine_number_of_processes(@count),
|
|
223
246
|
tests_with_size.size
|
|
@@ -233,8 +256,9 @@ module TurboTests
|
|
|
233
256
|
ParallelTests::RSpec::Runner.tests_in_groups(
|
|
234
257
|
@files,
|
|
235
258
|
@num_processes,
|
|
236
|
-
|
|
259
|
+
**parallel_tests_options
|
|
237
260
|
)
|
|
261
|
+
tests_in_groups = selected_groups(tests_in_groups) if @parallel_options[:only_group]
|
|
238
262
|
@tests_in_groups = tests_in_groups
|
|
239
263
|
|
|
240
264
|
subprocess_opts = {
|
|
@@ -264,10 +288,11 @@ module TurboTests
|
|
|
264
288
|
statuses = @wait_threads.map(&:value)
|
|
265
289
|
|
|
266
290
|
if @reporter.failed_examples.empty? && statuses.all?(&:success?)
|
|
291
|
+
flush_successful_worker_output
|
|
267
292
|
report_coverage = true
|
|
268
293
|
exit_status = 0
|
|
269
294
|
else
|
|
270
|
-
flush_worker_output
|
|
295
|
+
flush_worker_output unless stream_worker_output?
|
|
271
296
|
# From https://github.com/galtzo-floss/turbo_tests2/pull/20/
|
|
272
297
|
exit_status = statuses.map(&:exitstatus).max
|
|
273
298
|
end
|
|
@@ -280,6 +305,18 @@ module TurboTests
|
|
|
280
305
|
|
|
281
306
|
private
|
|
282
307
|
|
|
308
|
+
def selected_groups(tests_in_groups)
|
|
309
|
+
requested_groups = @parallel_options[:only_group]
|
|
310
|
+
missing_groups = requested_groups.select { |index| index > tests_in_groups.size }
|
|
311
|
+
unless missing_groups.empty?
|
|
312
|
+
raise ArgumentError,
|
|
313
|
+
"Selected group index(es) out of range: #{missing_groups.join(", ")} " \
|
|
314
|
+
"(available groups: #{tests_in_groups.size})"
|
|
315
|
+
end
|
|
316
|
+
|
|
317
|
+
requested_groups.map { |index| tests_in_groups[index - 1] }
|
|
318
|
+
end
|
|
319
|
+
|
|
283
320
|
def handle_interrupt
|
|
284
321
|
if @interrupt_handled
|
|
285
322
|
Kernel.exit
|
|
@@ -337,6 +374,7 @@ module TurboTests
|
|
|
337
374
|
|
|
338
375
|
record_runtime_options =
|
|
339
376
|
if record_runtime
|
|
377
|
+
FileUtils.mkdir_p(File.dirname(@runtime_log))
|
|
340
378
|
[
|
|
341
379
|
"--format",
|
|
342
380
|
"ParallelTests::RSpec::RuntimeLogger",
|
|
@@ -483,7 +521,25 @@ module TurboTests
|
|
|
483
521
|
end
|
|
484
522
|
|
|
485
523
|
io = (stream == :stderr) ? $stderr : $stdout
|
|
486
|
-
io.write(msg) if
|
|
524
|
+
io.write(msg) if stream_worker_output?
|
|
525
|
+
end
|
|
526
|
+
|
|
527
|
+
def stream_worker_output?
|
|
528
|
+
@verbose || @worker_output_mode == :stream
|
|
529
|
+
end
|
|
530
|
+
|
|
531
|
+
def flush_successful_worker_output
|
|
532
|
+
case @worker_output_mode
|
|
533
|
+
when :warnings
|
|
534
|
+
flush_worker_warnings unless stream_worker_output?
|
|
535
|
+
when :buffered
|
|
536
|
+
flush_worker_output unless stream_worker_output?
|
|
537
|
+
when :quiet, :stream
|
|
538
|
+
nil
|
|
539
|
+
else
|
|
540
|
+
raise ArgumentError,
|
|
541
|
+
"Unsupported worker output mode #{@worker_output_mode.inspect}; expected one of: #{WORKER_OUTPUT_MODES.join(", ")}"
|
|
542
|
+
end
|
|
487
543
|
end
|
|
488
544
|
|
|
489
545
|
def flush_worker_output
|
|
@@ -504,6 +560,43 @@ module TurboTests
|
|
|
504
560
|
end
|
|
505
561
|
end
|
|
506
562
|
|
|
563
|
+
def flush_worker_warnings
|
|
564
|
+
output_by_process = @worker_output_mutex.synchronize do
|
|
565
|
+
@worker_output.transform_values(&:dup)
|
|
566
|
+
end
|
|
567
|
+
|
|
568
|
+
output_by_process.each do |process_id, streams|
|
|
569
|
+
streams.each do |stream, output|
|
|
570
|
+
warnings = warning_lines(output)
|
|
571
|
+
next if warnings.empty?
|
|
572
|
+
|
|
573
|
+
io = (stream == :stderr) ? $stderr : $stdout
|
|
574
|
+
io.puts
|
|
575
|
+
io.puts("TurboTests worker #{process_id} #{stream} warnings:")
|
|
576
|
+
warnings.each { |line| io.puts(line) }
|
|
577
|
+
end
|
|
578
|
+
end
|
|
579
|
+
end
|
|
580
|
+
|
|
581
|
+
def warning_lines(output)
|
|
582
|
+
output.each_line.each_with_object([]) do |line, warnings|
|
|
583
|
+
stripped = line.strip
|
|
584
|
+
next if stripped.empty?
|
|
585
|
+
next unless warning_line?(stripped)
|
|
586
|
+
next if coverage_output_line?(stripped)
|
|
587
|
+
|
|
588
|
+
warnings << stripped
|
|
589
|
+
end
|
|
590
|
+
end
|
|
591
|
+
|
|
592
|
+
def warning_line?(line)
|
|
593
|
+
line.match?(/warning:/i) || line.match?(/deprecat/i)
|
|
594
|
+
end
|
|
595
|
+
|
|
596
|
+
def coverage_output_line?(line)
|
|
597
|
+
line.start_with?("Coverage report generated for ", "JSON Coverage report generated for ", "Line coverage:", "Branch coverage:", "Line Coverage:", "Branch Coverage:")
|
|
598
|
+
end
|
|
599
|
+
|
|
507
600
|
def flush_coverage_summary
|
|
508
601
|
line_coverage = nil
|
|
509
602
|
branch_coverage = nil
|
data/lib/turbo_tests/version.rb
CHANGED
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: turbo_tests2
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.
|
|
4
|
+
version: 3.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Illia
|
|
@@ -312,10 +312,10 @@ licenses:
|
|
|
312
312
|
- MIT
|
|
313
313
|
metadata:
|
|
314
314
|
homepage_uri: https://turbo-tests2.galtzo.com
|
|
315
|
-
source_code_uri: https://github.com/galtzo-floss/turbo_tests2/tree/v3.
|
|
316
|
-
changelog_uri: https://github.com/galtzo-floss/turbo_tests2/blob/v3.
|
|
315
|
+
source_code_uri: https://github.com/galtzo-floss/turbo_tests2/tree/v3.2.0
|
|
316
|
+
changelog_uri: https://github.com/galtzo-floss/turbo_tests2/blob/v3.2.0/CHANGELOG.md
|
|
317
317
|
bug_tracker_uri: https://github.com/galtzo-floss/turbo_tests2/issues
|
|
318
|
-
documentation_uri: https://www.rubydoc.info/gems/turbo_tests2/3.
|
|
318
|
+
documentation_uri: https://www.rubydoc.info/gems/turbo_tests2/3.2.0
|
|
319
319
|
funding_uri: https://github.com/sponsors/pboling
|
|
320
320
|
wiki_uri: https://github.com/galtzo-floss/turbo_tests2/wiki
|
|
321
321
|
news_uri: https://www.railsbling.com/tags/turbo_tests2
|
metadata.gz.sig
CHANGED
|
Binary file
|