turbo_tests2 3.2.14 → 3.2.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +48 -1
- data/README.md +64 -16
- data/lib/turbo_tests/runner.rb +148 -14
- data/lib/turbo_tests/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +8 -8
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 0ba760f9553f5ac9180f943b7b6ba4292f8e3ae332c2ee8f93d704fb6df39fd8
|
|
4
|
+
data.tar.gz: 308a01984a1da21ef5a1005cd5f333404b8132b6b8b12717f9fcf1e6b213421d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d15f6b3dc4601510f2fc2785b722b966fe543352a2408f084160e9b0d064fcf020cb234dbcc22b7c03baca495d1a7c87a13d41e24e74084a4bc43b6fe4aec5ac
|
|
7
|
+
data.tar.gz: be13e868955aacabb64993bab93050538def66c7c341406b082878d9f06d8c3098c701052f40c1e1a7c36eed026de5bc1c6930c7db42ab2e109a8240944c5d48
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -30,6 +30,49 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
30
30
|
|
|
31
31
|
### Security
|
|
32
32
|
|
|
33
|
+
## [3.2.16] - 2026-10-14
|
|
34
|
+
|
|
35
|
+
- TAG: [v3.2.16][3.2.16t]
|
|
36
|
+
- COVERAGE: 95.94% -- 993/1035 lines in 11 files
|
|
37
|
+
- BRANCH COVERAGE: 83.86% -- 265/316 branches in 11 files
|
|
38
|
+
- 39.42% documented
|
|
39
|
+
|
|
40
|
+
### Fixed
|
|
41
|
+
|
|
42
|
+
- Watchdog kills now actually end the run: kill_remaining_workers takes pids from the live wait_threads (the parallel_tests pid file was unreliable — each track/untrack built a fresh Pids instance, so any normal worker exit wiped the file and the watchdog killed nothing), handle_messages stops at the watchdog sentinel instead of waiting for per-worker exits that can never arrive, and post-kill joins are bounded so a wedged worker can no longer hold the run past the watchdog (observed: CI job printed the stall report, then hung 14 more minutes until cancellation)
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
|
|
46
|
+
- [kc] kettle-jem/template: updated 32 project files:
|
|
47
|
+
- code and tests (1)
|
|
48
|
+
- dependencies (3)
|
|
49
|
+
- other (1)
|
|
50
|
+
- workflows (27)
|
|
51
|
+
|
|
52
|
+
## [3.2.15] - 2026-10-13
|
|
53
|
+
|
|
54
|
+
- TAG: [v3.2.15][3.2.15t]
|
|
55
|
+
- COVERAGE: 96.67% -- 988/1022 lines in 11 files
|
|
56
|
+
- BRANCH COVERAGE: 85.06% -- 262/308 branches in 11 files
|
|
57
|
+
- 39.42% documented
|
|
58
|
+
|
|
59
|
+
### Changed
|
|
60
|
+
|
|
61
|
+
- [kc] kettle-jem/prepare: updated 13 project files:
|
|
62
|
+
- configuration (1)
|
|
63
|
+
- dependencies (12)
|
|
64
|
+
- [kc] kettle-jem/template: updated 12 project files:
|
|
65
|
+
- code and tests (2)
|
|
66
|
+
- documentation (1)
|
|
67
|
+
- other (2)
|
|
68
|
+
- workflows (7)
|
|
69
|
+
|
|
70
|
+
### Fixed
|
|
71
|
+
|
|
72
|
+
- The “Runtime and Platform Support” table in the README listed TruffleRuby 22.3, 23.0 and TruffleRuby-head as covered by CI, and described `rspec-pending_for` spec exclusions for 22.3/23.0. No workflow runs those engines (CI covers TruffleRuby 23.1, 24.2, 25.0, 33.0 and current), `heads.yml` runs `ruby-head` and `jruby-head` only, and `rspec-pending_for` is neither a dependency nor referenced by any spec. The badge rows for 22.3/23.0 had already been removed by templating, leaving the prose table describing CI that no longer runs.
|
|
73
|
+
|
|
74
|
+
- Watchdog on the worker message queue: if no worker message arrives within TURBO_TESTS2_WATCHDOG_SECONDS (default 900), remaining workers are killed, unfinished groups reported, and the run fails — instead of blocking forever when a worker dies without an exit message (the parallel-worker CI hang class on 4-core runners)
|
|
75
|
+
|
|
33
76
|
## [3.2.14] - 2026-10-08
|
|
34
77
|
|
|
35
78
|
- TAG: [v3.2.14][3.2.14t]
|
|
@@ -718,7 +761,11 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
718
761
|
|
|
719
762
|
- Initial release
|
|
720
763
|
|
|
721
|
-
[Unreleased]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.
|
|
764
|
+
[Unreleased]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.16...HEAD
|
|
765
|
+
[3.2.16]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.15...v3.2.16
|
|
766
|
+
[3.2.16t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.2.16
|
|
767
|
+
[3.2.15]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.14...v3.2.15
|
|
768
|
+
[3.2.15t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.2.15
|
|
722
769
|
[3.2.14]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.13...v3.2.14
|
|
723
770
|
[3.2.14t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.2.14
|
|
724
771
|
[3.2.13]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.12...v3.2.13
|
data/README.md
CHANGED
|
@@ -48,18 +48,17 @@ work in an untested environment:
|
|
|
48
48
|
| MRI head | Parallel support | Unsupported | Unsupported | CI is allowed to fail while Ruby is in development. |
|
|
49
49
|
| JRuby 9.2-9.4 | Parallel support | Unsupported | Unsupported | Versioned JRuby workflows run on Linux. |
|
|
50
50
|
| JRuby 10.0/current/head | Parallel support | Unsupported | Unsupported | Experimental/current and head jobs are non-blocking in CI. |
|
|
51
|
-
| TruffleRuby 22.3 | Parallel support | Unsupported | Unsupported | Experimental. The CLI and multi-process spec groups are skipped because this release reports Ruby compatibility version 3.0. |
|
|
52
|
-
| TruffleRuby 23.0 | Single-worker support | Unsupported | Unsupported | Experimental and end-of-life; CI caps workers at one. The CLI and multi-process spec groups are also skipped. |
|
|
53
51
|
| TruffleRuby 23.1 | Parallel support | Unsupported | Unsupported | Experimental. |
|
|
54
52
|
| TruffleRuby 24.2/25.0/33.0/current | Parallel support | Unsupported | Unsupported | Linux workflows run the suite with parallel workers. |
|
|
55
|
-
| TruffleRuby head | Parallel support | Unsupported | Unsupported | CI is allowed to fail while TruffleRuby is in development. |
|
|
56
53
|
|
|
57
|
-
The
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
54
|
+
The Windows current-Ruby job runs in parallel, but runtime-log recording is
|
|
55
|
+
disabled there because the runtime logger needs file locking that Windows does
|
|
56
|
+
not provide reliably (`record_runtime = !Gem.win_platform?`); see below.
|
|
57
|
+
|
|
58
|
+
TruffleRuby 22.3, 23.0 and TruffleRuby-head are not listed because no workflow
|
|
59
|
+
covers them. `heads.yml` runs `ruby-head` and `jruby-head` only, and
|
|
60
|
+
`dep-heads.yml` runs the released `truffleruby` engine against head-of-branch
|
|
61
|
+
dependencies, which is not a head engine build.
|
|
63
62
|
|
|
64
63
|
### Why incremental output?
|
|
65
64
|
|
|
@@ -99,7 +98,7 @@ Finished in 2 minute 25.15 seconds (files took 0 seconds to load)
|
|
|
99
98
|
| Tokens to Remember | [![Gem name][⛳️name-img]][⛳️gem-name] [![Gem namespace][⛳️namespace-img]][⛳️gem-namespace] |
|
|
100
99
|
|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
101
100
|
| Works with JRuby | [![JRuby 9.2 Compat][💎jruby-9.2i]][🚎jruby-9.2-wf] [![JRuby 9.3 Compat][💎jruby-9.3i]][🚎jruby-9.3-wf] <br/> [![JRuby 9.4 Compat][💎jruby-9.4i]][🚎jruby-9.4-wf] [![JRuby 10.0 Compat][💎jruby-10.0i]][🚎jruby-10.0-wf] [![JRuby current Compat][💎jruby-c-i]][🚎10-j-wf] [![JRuby HEAD Compat][💎jruby-headi]][🚎3-hd-wf]|
|
|
102
|
-
| Works with Truffle Ruby | [![Truffle Ruby
|
|
101
|
+
| Works with Truffle Ruby | [![Truffle Ruby 23.1 Compat][💎truby-23.1i]][🚎truby-23.1-wf] <br/> [![Truffle Ruby 24.2 Compat][💎truby-24.2i]][🚎truby-24.2-wf] [![Truffle Ruby 25.0 Compat][💎truby-25.0i]][🚎truby-25.0-wf] [![Truffle Ruby 33.0 Compat][💎truby-33.0i]][🚎truby-33.0-wf] [![Truffle Ruby current Compat][💎truby-c-i]][🚎9-t-wf]|
|
|
103
102
|
| Works with MRI Ruby 4 | [![Ruby current Compat][💎ruby-c-i]][🚎11-c-wf] [![Ruby HEAD Compat][💎ruby-headi]][🚎3-hd-wf]|
|
|
104
103
|
| Works with MRI Ruby 3 | [![Ruby 3.0 Compat][💎ruby-3.0i]][🚎ruby-3.0-wf] [![Ruby 3.1 Compat][💎ruby-3.1i]][🚎ruby-3.1-wf] [![Ruby 3.2 Compat][💎ruby-3.2i]][🚎ruby-3.2-wf] [![Ruby 3.3 Compat][💎ruby-3.3i]][🚎ruby-3.3-wf] [![Ruby 3.4 Compat][💎ruby-3.4i]][🚎ruby-3.4-wf]|
|
|
105
104
|
| Works with MRI Ruby 2 | [![Ruby 2.4 Compat][💎ruby-2.4i]][🚎ruby-2.4-wf] [![Ruby 2.5 Compat][💎ruby-2.5i]][🚎ruby-2.5-wf] [![Ruby 2.6 Compat][💎ruby-2.6i]][🚎ruby-2.6-wf] [![Ruby 2.7 Compat][💎ruby-2.7i]][🚎ruby-2.7-wf]|
|
|
@@ -138,6 +137,23 @@ The _amazing_ test matrix is powered by the kettle-dev stack.
|
|
|
138
137
|
|
|
139
138
|
</details>
|
|
140
139
|
|
|
140
|
+
### Runtime and Platform Support
|
|
141
|
+
|
|
142
|
+
Support tiers describe what this project's CI actually exercises, not what may
|
|
143
|
+
work in an untested environment:
|
|
144
|
+
|
|
145
|
+
- **Parallel support**: CI runs the test suite with multiple workers available.
|
|
146
|
+
- **Unsupported**: the engine/platform combination is not covered by CI and is
|
|
147
|
+
not a project compatibility claim.
|
|
148
|
+
|
|
149
|
+
| Ruby engine/version | Linux | macOS | Windows | Notes |
|
|
150
|
+
|---------------------|-------|-------|---------|-------|
|
|
151
|
+
| MRI 3.1-3.4 | Parallel support | Unsupported | Unsupported | Versioned Ruby workflows run on Linux. |
|
|
152
|
+
| MRI current | Parallel support | Parallel support | Unsupported | Current Ruby workflow covers the declared OS platforms. |
|
|
153
|
+
| MRI head | Parallel support | Unsupported | Unsupported | CI is allowed to fail while Ruby is in development. |
|
|
154
|
+
| JRuby (all CI lanes) | Parallel support | Unsupported | Unsupported | Engine workflows run on Linux; macOS and Windows are not exercised by CI. |
|
|
155
|
+
| TruffleRuby (all CI lanes) | Parallel support | Unsupported | Unsupported | Never shipped a Windows build (zero Windows assets across 66 releases scanned 2026-10-10); setup-ruby hard-fails ([source](https://github.com/ruby/setup-ruby/blob/master/README.md)) |
|
|
156
|
+
|
|
141
157
|
### Federated DVCS
|
|
142
158
|
|
|
143
159
|
<details markdown="1">
|
|
@@ -368,6 +384,43 @@ count at the number of discovered test groups.
|
|
|
368
384
|
Known coverage formatter chatter remains collapsed into the concise coverage
|
|
369
385
|
summary.
|
|
370
386
|
|
|
387
|
+
### Stall watchdog
|
|
388
|
+
|
|
389
|
+
If the worker message queue goes silent — every worker alive but none
|
|
390
|
+
producing example/group messages — a run can otherwise hang until the CI
|
|
391
|
+
provider kills the job (GitHub Actions: 6 hours).
|
|
392
|
+
|
|
393
|
+
`TURBO_TESTS2_WATCHDOG_SECONDS=N` (integer, `0` = disabled) bounds that
|
|
394
|
+
silence. **Default is `0` (disabled) in 3.x.** In 4.0 the default becomes
|
|
395
|
+
`900` and the kill behavior is always on.
|
|
396
|
+
|
|
397
|
+
When the watchdog fires it kills the remaining worker processes, prints the
|
|
398
|
+
unfinished groups and worker pids, and fails the run.
|
|
399
|
+
|
|
400
|
+
Setting it:
|
|
401
|
+
|
|
402
|
+
```bash
|
|
403
|
+
TURBO_TESTS2_WATCHDOG_SECONDS=900 bundle exec turbo_tests2
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
Notes:
|
|
407
|
+
|
|
408
|
+
- The clock measures **silence across all workers**, not per-test duration.
|
|
409
|
+
In parallel lanes any other worker finishing an example keeps the run
|
|
410
|
+
alive; only serial lanes (`TURBO_TESTS2_MAX_PROCESSES=1`) can trip on a
|
|
411
|
+
single long test. Raise the value there if you have legitimately long,
|
|
412
|
+
silent tests.
|
|
413
|
+
- CI jobs that hang despite the watchdog will still hit the provider
|
|
414
|
+
limit; also set `timeout-minutes` on the CI job. Locally, an outer
|
|
415
|
+
supervisor (process watcher, container timeout) plays the same role.
|
|
416
|
+
- When the watchdog is disabled (`0`), a 900s silence prints a one-time
|
|
417
|
+
deprecation notice describing the 4.0 default change and both options
|
|
418
|
+
(adopt the kill via `TURBO_TESTS2_WATCHDOG_SECONDS`, or explicitly opt out
|
|
419
|
+
with `0`). The notice is provider-neutral: it warns that with the
|
|
420
|
+
watchdog off, whatever outer process watcher exists — a CI job timeout,
|
|
421
|
+
a container orchestrator, or a local supervisor — is what will eventually
|
|
422
|
+
end a hung run.
|
|
423
|
+
|
|
371
424
|
`turbo_tests2` supports custom formatter such as Fuubar, but you might need to require it:
|
|
372
425
|
|
|
373
426
|
```bash
|
|
@@ -734,8 +787,6 @@ Thanks for RTFM. ☺️
|
|
|
734
787
|
[🚎jruby-9.3-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/jruby-9.3.yml
|
|
735
788
|
[🚎jruby-9.4-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/jruby-9.4.yml
|
|
736
789
|
[🚎jruby-10.0-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/jruby-10.0.yml
|
|
737
|
-
[🚎truby-22.3-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/truffleruby-22.3.yml
|
|
738
|
-
[🚎truby-23.0-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/truffleruby-23.0.yml
|
|
739
790
|
[🚎truby-23.1-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/truffleruby-23.1.yml
|
|
740
791
|
[🚎truby-24.2-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/truffleruby-24.2.yml
|
|
741
792
|
[🚎truby-25.0-wf]: https://github.com/galtzo-floss/turbo_tests2/actions/workflows/truffleruby-25.0.yml
|
|
@@ -771,14 +822,11 @@ Thanks for RTFM. ☺️
|
|
|
771
822
|
[💎ruby-3.4i]: https://img.shields.io/badge/Ruby-3.4-CC342D?style=for-the-badge&logo=ruby&logoColor=white
|
|
772
823
|
[💎ruby-c-i]: https://img.shields.io/badge/Ruby-current-CC342D?style=for-the-badge&logo=ruby&logoColor=green
|
|
773
824
|
[💎ruby-headi]: https://img.shields.io/badge/Ruby-HEAD-CC342D?style=for-the-badge&logo=ruby&logoColor=blue
|
|
774
|
-
[💎truby-22.3i]: https://img.shields.io/badge/Truffle_Ruby-22.3-34BCB1?style=for-the-badge&logo=ruby&logoColor=pink
|
|
775
|
-
[💎truby-23.0i]: https://img.shields.io/badge/Truffle_Ruby-23.0-34BCB1?style=for-the-badge&logo=ruby&logoColor=pink
|
|
776
825
|
[💎truby-23.1i]: https://img.shields.io/badge/Truffle_Ruby-23.1-34BCB1?style=for-the-badge&logo=ruby&logoColor=pink
|
|
777
826
|
[💎truby-24.2i]: https://img.shields.io/badge/Truffle_Ruby-24.2-34BCB1?style=for-the-badge&logo=ruby&logoColor=pink
|
|
778
827
|
[💎truby-25.0i]: https://img.shields.io/badge/Truffle_Ruby-25.0-34BCB1?style=for-the-badge&logo=ruby&logoColor=pink
|
|
779
828
|
[💎truby-33.0i]: https://img.shields.io/badge/Truffle_Ruby-33.0-34BCB1?style=for-the-badge&logo=ruby&logoColor=pink
|
|
780
829
|
[💎truby-c-i]: https://img.shields.io/badge/Truffle_Ruby-current-34BCB1?style=for-the-badge&logo=ruby&logoColor=green
|
|
781
|
-
[💎truby-headi]: https://img.shields.io/badge/Truffle_Ruby-HEAD-34BCB1?style=for-the-badge&logo=ruby&logoColor=blue
|
|
782
830
|
[💎jruby-9.2i]: https://img.shields.io/badge/JRuby-9.2-FBE742?style=for-the-badge&logo=ruby&logoColor=red
|
|
783
831
|
[💎jruby-9.3i]: https://img.shields.io/badge/JRuby-9.3-FBE742?style=for-the-badge&logo=ruby&logoColor=red
|
|
784
832
|
[💎jruby-9.4i]: https://img.shields.io/badge/JRuby-9.4-FBE742?style=for-the-badge&logo=ruby&logoColor=red
|
|
@@ -811,7 +859,7 @@ Thanks for RTFM. ☺️
|
|
|
811
859
|
[📌gitmoji]: https://gitmoji.dev
|
|
812
860
|
[📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
|
|
813
861
|
[🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
|
814
|
-
[🧮kloc-img]: https://img.shields.io/badge/KLOC-
|
|
862
|
+
[🧮kloc-img]: https://img.shields.io/badge/KLOC-1.035-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
|
|
815
863
|
[🔐security]: https://github.com/galtzo-floss/turbo_tests2/blob/main/SECURITY.md
|
|
816
864
|
[🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
|
|
817
865
|
[📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
|
data/lib/turbo_tests/runner.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "json"
|
|
4
|
+
require "timeout"
|
|
4
5
|
require "fileutils"
|
|
5
6
|
require "parallel_tests/rspec/runner"
|
|
6
7
|
require "rspec/core"
|
|
@@ -248,6 +249,12 @@ module TurboTests
|
|
|
248
249
|
@load_time = 0
|
|
249
250
|
@load_count = 0
|
|
250
251
|
@failure_count = 0
|
|
252
|
+
# Deprecation-period default: disabled. In 4.0 the kill behavior
|
|
253
|
+
# becomes the default (TURBO_TESTS2_WATCHDOG_SECONDS=900) and this
|
|
254
|
+
# warning is removed. See README "Stall watchdog".
|
|
255
|
+
@watchdog_seconds = Float(ENV.fetch("TURBO_TESTS2_WATCHDOG_SECONDS", "0"))
|
|
256
|
+
@watchdog_warned = false
|
|
257
|
+
@last_message_monotonic = nil
|
|
251
258
|
|
|
252
259
|
# Supports runtime_log as a top level option,
|
|
253
260
|
# but also nested inside parallel_options
|
|
@@ -311,24 +318,41 @@ module TurboTests
|
|
|
311
318
|
|
|
312
319
|
handle_messages
|
|
313
320
|
|
|
314
|
-
@
|
|
321
|
+
if @watchdog_fired
|
|
322
|
+
# Workers were SIGKILLed by the watchdog; their reader threads end
|
|
323
|
+
# at EOF and waiters return as the OS reaps them — but if a kill
|
|
324
|
+
# failed (permissions, zombie, pid reuse) an unbounded join here
|
|
325
|
+
# would hang the run anyway, defeating the watchdog. Bound every
|
|
326
|
+
# join and treat any worker that still has not exited as failed.
|
|
327
|
+
@threads.each do |thread|
|
|
328
|
+
thread.join(FORCE_STOP_JOIN)
|
|
329
|
+
thread.kill if thread.alive?
|
|
330
|
+
end
|
|
331
|
+
@wait_threads.each do |wait_thr|
|
|
332
|
+
wait_thr.join(FORCE_STOP_JOIN)
|
|
333
|
+
end
|
|
334
|
+
flush_worker_output unless stream_worker_output?
|
|
335
|
+
exit_status = 1
|
|
336
|
+
else
|
|
337
|
+
@threads.each(&:join)
|
|
315
338
|
|
|
316
|
-
|
|
339
|
+
report_workers_without_results
|
|
317
340
|
|
|
318
|
-
|
|
341
|
+
report_failed_group(tests_in_groups) if @print_failed_group
|
|
319
342
|
|
|
320
|
-
|
|
343
|
+
Signal.trap(:INT, old_signal)
|
|
321
344
|
|
|
322
|
-
|
|
345
|
+
statuses = @wait_threads.map(&:value)
|
|
323
346
|
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
347
|
+
if @reporter.failed_examples.empty? && statuses.all?(&:success?)
|
|
348
|
+
flush_successful_worker_output
|
|
349
|
+
report_coverage = true
|
|
350
|
+
exit_status = 0
|
|
351
|
+
else
|
|
352
|
+
flush_worker_output unless stream_worker_output?
|
|
353
|
+
# From https://github.com/galtzo-floss/turbo_tests2/pull/20/
|
|
354
|
+
exit_status = statuses.map(&:exitstatus).max
|
|
355
|
+
end
|
|
332
356
|
end
|
|
333
357
|
end
|
|
334
358
|
|
|
@@ -725,12 +749,118 @@ module TurboTests
|
|
|
725
749
|
thread.kill.join(FORCE_STOP_JOIN)
|
|
726
750
|
end
|
|
727
751
|
|
|
752
|
+
# Blocks for the next worker message, enforcing a stall watchdog. A
|
|
753
|
+
# worker whose stdout reader and exit-ensuring thread both fail to
|
|
754
|
+
# deliver an exit message (process killed by the OOM killer, forked
|
|
755
|
+
# child wedged in a native call) used to leave handle_messages blocked
|
|
756
|
+
# on the queue forever — CI jobs hung for hours with no diagnostics.
|
|
757
|
+
# On a stall we kill the surviving worker pids, report which groups
|
|
758
|
+
# never finished, and fail the run instead.
|
|
759
|
+
def pop_message_with_watchdog
|
|
760
|
+
# 0 (the default) disables the watchdog: block indefinitely, exactly the
|
|
761
|
+
# pre-watchdog behavior.
|
|
762
|
+
return @messages.pop if @watchdog_seconds <= 0
|
|
763
|
+
|
|
764
|
+
loop do
|
|
765
|
+
timeout = [@watchdog_seconds - (Process.clock_gettime(Process::CLOCK_MONOTONIC) - @last_message_monotonic), 1.0].max
|
|
766
|
+
begin
|
|
767
|
+
message = Timeout.timeout(timeout) { @messages.pop }
|
|
768
|
+
@last_message_monotonic = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
769
|
+
return message
|
|
770
|
+
rescue Timeout::Error
|
|
771
|
+
stall_report = remaining_workers_report
|
|
772
|
+
warn("turbo_tests2: no worker messages for #{@watchdog_seconds}s; killing stalled workers and failing the run")
|
|
773
|
+
warn(stall_report)
|
|
774
|
+
kill_remaining_workers
|
|
775
|
+
@reporter.error_outside_of_examples(
|
|
776
|
+
"turbo_tests2: watchdog fired after #{@watchdog_seconds}s without worker messages; workers killed\n#{stall_report}"
|
|
777
|
+
)
|
|
778
|
+
@error = true
|
|
779
|
+
# The run is over: workers were just SIGKILLed. handle_messages must
|
|
780
|
+
# not keep waiting for per-worker exit messages that can never
|
|
781
|
+
# arrive, and the caller must not re-enter this watchdog (which would
|
|
782
|
+
# warn-and-kill in a 1-second spin).
|
|
783
|
+
@watchdog_fired = true
|
|
784
|
+
return {type: "exit", process_id: :watchdog}
|
|
785
|
+
end
|
|
786
|
+
end
|
|
787
|
+
end
|
|
788
|
+
|
|
789
|
+
def remaining_workers_report
|
|
790
|
+
pids = tracked_pids
|
|
791
|
+
live = []
|
|
792
|
+
pids.each do |pid|
|
|
793
|
+
begin
|
|
794
|
+
Process.kill(0, pid)
|
|
795
|
+
live << pid
|
|
796
|
+
rescue
|
|
797
|
+
nil
|
|
798
|
+
end
|
|
799
|
+
end
|
|
800
|
+
lines = ["Workers total: #{@num_processes}; tracked pids: #{pids.inspect}; live: #{live.inspect}"]
|
|
801
|
+
(@tests_in_groups || []).each_with_index do |group, idx|
|
|
802
|
+
lines << "group #{idx + 1}: #{Array(group).join(", ")}" unless group.nil?
|
|
803
|
+
end
|
|
804
|
+
lines.join("\n")
|
|
805
|
+
end
|
|
806
|
+
|
|
807
|
+
def tracked_pids
|
|
808
|
+
path = ParallelTests.pid_file_path
|
|
809
|
+
return [] unless path && File.file?(path)
|
|
810
|
+
|
|
811
|
+
File.read(path).lines.map { |l| l.strip.to_i }.reject(&:zero?)
|
|
812
|
+
rescue
|
|
813
|
+
[]
|
|
814
|
+
end
|
|
815
|
+
|
|
816
|
+
def kill_remaining_workers
|
|
817
|
+
# The live waiter objects are authoritative: the parallel_tests pid file
|
|
818
|
+
# loses entries because every track/untrack call builds a fresh Pids
|
|
819
|
+
# instance whose in-memory list starts empty, so a save overwrites (or,
|
|
820
|
+
# for untrack, empties) the file. After any normal worker exit the file
|
|
821
|
+
# is guaranteed useless — exactly when the watchdog needs it most.
|
|
822
|
+
pids = (@wait_threads || []).map do |wait_thr|
|
|
823
|
+
begin
|
|
824
|
+
wait_thr.pid
|
|
825
|
+
rescue
|
|
826
|
+
nil
|
|
827
|
+
end
|
|
828
|
+
end.compact
|
|
829
|
+
(pids | tracked_pids).each do |pid|
|
|
830
|
+
begin
|
|
831
|
+
Process.kill("KILL", pid)
|
|
832
|
+
rescue
|
|
833
|
+
nil
|
|
834
|
+
end
|
|
835
|
+
end
|
|
836
|
+
end
|
|
837
|
+
|
|
728
838
|
def handle_messages
|
|
729
839
|
exited_process_ids = {}
|
|
730
840
|
process_count = @tests_in_groups&.size || @num_processes
|
|
731
841
|
|
|
842
|
+
# Tests pre-seed @last_message_monotonic to simulate a stall; only
|
|
843
|
+
# initialize it here when unset.
|
|
844
|
+
@last_message_monotonic ||= Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
845
|
+
warned_silence = false
|
|
732
846
|
loop do
|
|
733
|
-
message =
|
|
847
|
+
message = pop_message_with_watchdog
|
|
848
|
+
unless warned_silence
|
|
849
|
+
silence = Process.clock_gettime(Process::CLOCK_MONOTONIC) - @last_message_monotonic
|
|
850
|
+
if silence >= 900
|
|
851
|
+
warn(
|
|
852
|
+
"turbo_tests2: workers have been silent for #{silence.to_i}s. " \
|
|
853
|
+
"In turbo_tests2 4.0 (and now when TURBO_TESTS2_WATCHDOG_SECONDS is set) a stall this long kills the " \
|
|
854
|
+
"remaining workers and fails the run. To adopt the kill now set TURBO_TESTS2_WATCHDOG_SECONDS=900 (or " \
|
|
855
|
+
"another value). To keep waiting indefinitely set TURBO_TESTS2_WATCHDOG_SECONDS=0; be aware that an " \
|
|
856
|
+
"outer process watcher (a CI job timeout, a container orchestrator, or a local supervisor) will then " \
|
|
857
|
+
"be what eventually ends the run, usually without a useful stall report. A single long test only " \
|
|
858
|
+
"trips this in serial lanes (TURBO_TESTS2_MAX_PROCESSES=1); in parallel lanes other workers keep " \
|
|
859
|
+
"messages flowing."
|
|
860
|
+
)
|
|
861
|
+
warned_silence = true
|
|
862
|
+
end
|
|
863
|
+
end
|
|
734
864
|
case message[:type]
|
|
735
865
|
when "example_passed"
|
|
736
866
|
example = FakeExample.from_obj(message[:example])
|
|
@@ -774,6 +904,10 @@ module TurboTests
|
|
|
774
904
|
nil
|
|
775
905
|
when "exit"
|
|
776
906
|
process_id = message[:process_id]
|
|
907
|
+
# Watchdog exit: workers were just SIGKILLed and no further per-worker
|
|
908
|
+
# exit messages can be relied upon — stop the loop immediately
|
|
909
|
+
# instead of waiting for process_count exits that never arrive.
|
|
910
|
+
break if process_id == :watchdog
|
|
777
911
|
next if exited_process_ids.key?(process_id)
|
|
778
912
|
|
|
779
913
|
exited_process_ids[process_id] = true
|
data/lib/turbo_tests/version.rb
CHANGED
|
@@ -4,7 +4,7 @@ module TurboTests
|
|
|
4
4
|
# Version namespace for this gem.
|
|
5
5
|
module Version
|
|
6
6
|
# Current gem version.
|
|
7
|
-
VERSION = "3.2.
|
|
7
|
+
VERSION = "3.2.16"
|
|
8
8
|
end
|
|
9
9
|
# Current gem version exposed at the traditional constant location.
|
|
10
10
|
VERSION = Version::VERSION # Traditional Constant Location
|
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: turbo_tests2
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.2.
|
|
4
|
+
version: 3.2.16
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Illia
|
|
@@ -111,7 +111,7 @@ dependencies:
|
|
|
111
111
|
version: '3.1'
|
|
112
112
|
- - ">="
|
|
113
113
|
- !ruby/object:Gem::Version
|
|
114
|
-
version: 3.1.
|
|
114
|
+
version: 3.1.12
|
|
115
115
|
type: :development
|
|
116
116
|
prerelease: false
|
|
117
117
|
version_requirements: !ruby/object:Gem::Requirement
|
|
@@ -121,7 +121,7 @@ dependencies:
|
|
|
121
121
|
version: '3.1'
|
|
122
122
|
- - ">="
|
|
123
123
|
- !ruby/object:Gem::Version
|
|
124
|
-
version: 3.1.
|
|
124
|
+
version: 3.1.12
|
|
125
125
|
- !ruby/object:Gem::Dependency
|
|
126
126
|
name: bundler-audit
|
|
127
127
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -219,7 +219,7 @@ dependencies:
|
|
|
219
219
|
version: '2.0'
|
|
220
220
|
- - ">="
|
|
221
221
|
- !ruby/object:Gem::Version
|
|
222
|
-
version: 2.0.
|
|
222
|
+
version: 2.0.25
|
|
223
223
|
type: :development
|
|
224
224
|
prerelease: false
|
|
225
225
|
version_requirements: !ruby/object:Gem::Requirement
|
|
@@ -229,7 +229,7 @@ dependencies:
|
|
|
229
229
|
version: '2.0'
|
|
230
230
|
- - ">="
|
|
231
231
|
- !ruby/object:Gem::Version
|
|
232
|
-
version: 2.0.
|
|
232
|
+
version: 2.0.25
|
|
233
233
|
- !ruby/object:Gem::Dependency
|
|
234
234
|
name: ruby-progressbar
|
|
235
235
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -316,10 +316,10 @@ licenses:
|
|
|
316
316
|
- MIT
|
|
317
317
|
metadata:
|
|
318
318
|
homepage_uri: https://turbo-tests2.galtzo.com
|
|
319
|
-
source_code_uri: https://github.com/galtzo-floss/turbo_tests2/tree/v3.2.
|
|
320
|
-
changelog_uri: https://github.com/galtzo-floss/turbo_tests2/blob/v3.2.
|
|
319
|
+
source_code_uri: https://github.com/galtzo-floss/turbo_tests2/tree/v3.2.16
|
|
320
|
+
changelog_uri: https://github.com/galtzo-floss/turbo_tests2/blob/v3.2.16/CHANGELOG.md
|
|
321
321
|
bug_tracker_uri: https://github.com/galtzo-floss/turbo_tests2/issues
|
|
322
|
-
documentation_uri: https://www.rubydoc.info/gems/turbo_tests2/3.2.
|
|
322
|
+
documentation_uri: https://www.rubydoc.info/gems/turbo_tests2/3.2.16
|
|
323
323
|
funding_uri: https://github.com/sponsors/pboling
|
|
324
324
|
wiki_uri: https://github.com/galtzo-floss/turbo_tests2/wiki
|
|
325
325
|
news_uri: https://www.railsbling.com/tags/turbo_tests2
|
metadata.gz.sig
CHANGED
|
Binary file
|