turbo_tests2 3.2.16 → 4.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0ba760f9553f5ac9180f943b7b6ba4292f8e3ae332c2ee8f93d704fb6df39fd8
-  data.tar.gz: 308a01984a1da21ef5a1005cd5f333404b8132b6b8b12717f9fcf1e6b213421d
+  metadata.gz: 7e14a9905e357ba16f5489987c5fc3638078587c35f232443698a02961cacd21
+  data.tar.gz: 9ea2481ca612f26d150e8514f6ccf12a758d49080a6ddd545a05f558a76dfe9a
 SHA512:
-  metadata.gz: d15f6b3dc4601510f2fc2785b722b966fe543352a2408f084160e9b0d064fcf020cb234dbcc22b7c03baca495d1a7c87a13d41e24e74084a4bc43b6fe4aec5ac
-  data.tar.gz: be13e868955aacabb64993bab93050538def66c7c341406b082878d9f06d8c3098c701052f40c1e1a7c36eed026de5bc1c6930c7db42ab2e109a8240944c5d48
+  metadata.gz: 86f70203dd5f5ea3824dbc3ef6acae6b4f469ec4b2c7798e4e6e0c847316a51fb7883a61885df45acc09b01d26373416c381b5632b82e9c27c1dd50d5be03bf5
+  data.tar.gz: 6ae7d9b0015581bda3e25ee998593becab94daffc2806039e52dc22b73bceb57d3117b95be595380ee9c9bd4db427321aa47cc8971600890256d63e0891fa6f0

data/CHANGELOG.md

@@ -30,6 +30,36 @@ Please file a bug if you notice a violation of semantic versioning.
 
 ### Security
 
+## [4.0.0] - 2026-10-15
+
+- TAG: [v4.0.0][4.0.0t]
+- COVERAGE: 96.01% -- 1034/1077 lines in 12 files
+- BRANCH COVERAGE: 82.42% -- 272/330 branches in 12 files
+- 37.39% documented
+
+### Changed
+
+- Breaking: replace the parallel_tests runtime dependency with runtime_groups (~> 0.1, >= 0.1.2) for process counts, pid bookkeeping, options-file discovery, and runtime-weighted grouping; turbo_tests2 now owns a TurboTests::RuntimeLogger RSpec formatter and reimplements the Rails db:create bootstrap directly on Process.spawn
+
+- Breaking: the stall watchdog is now ON by default (TURBO_TESTS2_WATCHDOG_SECONDS defaults to 900, killing silent workers and failing the run with a stall report); set TURBO_TESTS2_WATCHDOG_SECONDS=0 to restore the 3.x wait-forever behavior. The 3.x deprecation notice is replaced by a provider-neutral one-time warning when the watchdog is disabled and workers go silent 900s, naming the most common cause of total silence in a parallel lane (silently OOM-killed workers) and the fix (cap workers x per-worker memory)
+
+### Fixed
+
+- --allowed-missing now actually bounds the share of files allowed to miss the runtime log when grouping (parallel_tests silently ignored the option); runtime log is now written by the first-party TurboTests::RuntimeLogger through RuntimeGroups::RuntimeLog, and the watchdog's tracked-pid fallback reads the read-before-write pid ledger correctly
+
+## [3.2.17] - 2026-10-14
+
+- TAG: [v3.2.17][3.2.17t]
+- COVERAGE: 95.94% -- 993/1035 lines in 11 files
+- BRANCH COVERAGE: 83.86% -- 265/316 branches in 11 files
+- 39.42% documented
+
+### Changed
+
+- README: document the spawn-based (fork-free) worker architecture that makes parallel workers engine-portable to JRuby and TruffleRuby, and add a Resource-Constrained CI (JVM Engines) section with the canonical 2-worker + capped-heap + raised-watchdog setup, matching the kettle-jem CI templates
+
+- Stall diagnostics and in-code docs now explain silent worker death: the 900s silence warning and the watchdog stall path point at the OOM killer as the most common cause of total silence in a parallel lane (spawned workers, JVM engines defaulting to 1/4 RAM heap each), with the fix inline (TURBO_TESTS2_MAX_PROCESSES=2 + capped heap); start_subprocess and max_processes_from_environment document the spawn-based engine-portable architecture
+
 ## [3.2.16] - 2026-10-14
 
 - TAG: [v3.2.16][3.2.16t]
@@ -761,7 +791,11 @@ Please file a bug if you notice a violation of semantic versioning.
 
 - Initial release
 
-[Unreleased]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.16...HEAD
+[Unreleased]: https://github.com/galtzo-floss/turbo_tests2/compare/v4.0.0...HEAD
+[4.0.0]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.17...v4.0.0
+[4.0.0t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v4.0.0
+[3.2.17]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.16...v3.2.17
+[3.2.17t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.2.17
 [3.2.16]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.15...v3.2.16
 [3.2.16t]: https://github.com/galtzo-floss/turbo_tests2/releases/tag/v3.2.16
 [3.2.15]: https://github.com/galtzo-floss/turbo_tests2/compare/v3.2.14...v3.2.15

data/README.md

@@ -25,7 +25,7 @@ I've summarized my thoughts in [this blog post](https://dev.to/galtzo/hostile-ta
 |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | I ([@pboling](https://github.com/pboling)) discussed merging this project back into the original `turbo_tests` with [@ilyazub](https://github.com/ilyazub), and we might move in that direction, but it will take time. Since I need this now I'm releasing it as a separate gem/project. We'll be discussing potentially unifying the projects again soon. |
 
-`turbo_tests2` is an RSpec parallel test runner built on [grosser/parallel_tests](https://github.com/grosser/parallel_tests). It uses `parallel_tests` for process counts, file grouping, and runtime balancing, then replaces interleaved worker output with incremental summarized RSpec reporting. Source code of this gem is based on [Discourse](https://github.com/discourse/discourse/blob/6b9784cf8a18636bce281a7e4d18e65a0cbc6290/lib/turbo_tests.rb) and [RubyGems](https://github.com/rubygems/rubygems/tree/390335ceb351668cd433bd5bb9823dd021f82533/bundler/tool) work in this area.
+`turbo_tests2` is an RSpec parallel test runner. It uses [runtime_groups](https://github.com/galtzo-floss/runtime_groups) for process counts, pid bookkeeping, file grouping, and runtime balancing, then replaces interleaved worker output with incremental summarized RSpec reporting. Source code of this gem is based on [Discourse](https://github.com/discourse/discourse/blob/6b9784cf8a18636bce281a7e4d18e65a0cbc6290/lib/turbo_tests.rb) and [RubyGems](https://github.com/rubygems/rubygems/tree/390335ceb351668cd433bd5bb9823dd021f82533/bundler/tool) work in this area. As of v4.0.0 the `parallel_tests` dependency is gone; the CLI keeps compatibility with the `parallel_tests` grouping surface documented below.
 The project namespace remains `TurboTests`; the published gem and executable are `turbo_tests2`.
 
 Incremental summarized output doesn't [fit the vision](https://github.com/grosser/parallel_tests/issues/708) of the `parallel_tests` author, and [RSpec doesn't support built-in parallel testing yet](https://github.com/rspec/rspec-rails/issues/2104#issuecomment-658474900). This gem targets RSpec, not every framework supported by `parallel_tests`, and may not be useful once one of the issues above is implemented.
@@ -384,6 +384,60 @@ count at the number of discovered test groups.
 Known coverage formatter chatter remains collapsed into the concise coverage
 summary.
 
+### Resource-Constrained CI (JVM Engines)
+
+Worker parallelism is spawn-based, not fork-based: each worker is started as a
+real subprocess (`Process.spawn` / `Open3.popen3`), never `Kernel#fork`. That
+makes parallel workers engine-portable — MRI, JRuby, and TruffleRuby all run
+the same spawn-based pool, and the default worker count is the machine's core
+count (`Etc.nprocessors`) on every engine. There is no serial-only mode for
+JVM engines unless you ask for one with `TURBO_TESTS2_MAX_PROCESSES=1`.
+
+The cost of that portability on JVM engines is memory: each JRuby or
+TruffleRuby worker is a full JVM subprocess, and JRuby defaults its heap to
+one quarter of machine RAM. On a CI runner, `Etc.nprocessors` workers each
+grabbing a quarter of RAM is a recipe for the kernel's OOM killer, which kills
+workers **silently** — every worker stops messaging at once, and the stall
+watchdog then reports total silence. This is a resource problem, not a
+parallelism problem.
+
+The fix is to run fewer, smaller workers. This is the canonical
+resource-constrained setup, and the `kettle-jem` CI templates bake it into
+every JRuby workflow they generate:
+
+```bash
+TURBO_TESTS2_MAX_PROCESSES=2 \
+JRUBY_OPTS="-J-Xmx1280m" \
+TURBO_TESTS2_WATCHDOG_SECONDS=1500 \
+bundle exec turbo_tests2
+```
+
+Or in a GitHub Actions workflow:
+
+```yaml
+env:
+  # Fewer, smaller JVM workers on constrained runners.
+  TURBO_TESTS2_MAX_PROCESSES: 2
+  # Cap each worker JVM's heap (JRuby default: 1/4 of runner RAM).
+  JRUBY_OPTS: "-J-Xmx1280m"
+  # JVM lanes are slow; give the watchdog room without hitting the job timeout.
+  TURBO_TESTS2_WATCHDOG_SECONDS: 1500
+```
+
+Guidance:
+
+- **Cap workers** (`TURBO_TESTS2_MAX_PROCESSES=2`) when each worker is a heavy
+  process: JVM engines, large test suites, or shared/undersized runners. Two
+  JVM workers still roughly halve wall-clock time versus one.
+- **Cap the heap** (`JRUBY_OPTS="-J-Xmx1280m"`, or your engine's equivalent)
+  so total memory is bounded regardless of runner size.
+- **Raise the watchdog** (1500s) on slow engines: JIT warmup and heavy GC make
+  silence windows longer, and you want the watchdog report, not the provider
+  timeout, when something genuinely hangs.
+- The same knobs apply to any engine where a worker process is expensive —
+  the principle is: bound `workers × per-worker memory` under the runner's
+  capacity.
+
 ### Stall watchdog
 
 If the worker message queue goes silent — every worker alive but none
@@ -391,8 +445,9 @@ producing example/group messages — a run can otherwise hang until the CI
 provider kills the job (GitHub Actions: 6 hours).
 
 `TURBO_TESTS2_WATCHDOG_SECONDS=N` (integer, `0` = disabled) bounds that
-silence. **Default is `0` (disabled) in 3.x.** In 4.0 the default becomes
-`900` and the kill behavior is always on.
+silence. **Default is `900` as of 4.0.** (3.x defaulted to `0` — disabled —
+with a one-time deprecation notice; set `0` to keep the old wait-forever
+behavior.)
 
 When the watchdog fires it kills the remaining worker processes, prints the
 unfinished groups and worker pids, and fails the run.
@@ -414,12 +469,12 @@ Notes:
   limit; also set `timeout-minutes` on the CI job. Locally, an outer
   supervisor (process watcher, container timeout) plays the same role.
 - When the watchdog is disabled (`0`), a 900s silence prints a one-time
-  deprecation notice describing the 4.0 default change and both options
-  (adopt the kill via `TURBO_TESTS2_WATCHDOG_SECONDS`, or explicitly opt out
-  with `0`). The notice is provider-neutral: it warns that with the
-  watchdog off, whatever outer process watcher exists — a CI job timeout,
-  a container orchestrator, or a local supervisor — is what will eventually
-  end a hung run.
+  provider-neutral notice: with the watchdog off, whatever outer process
+  watcher exists — a CI job timeout, a container orchestrator, or a local
+  supervisor — is what will eventually end a hung run, usually without a
+  useful stall report. The notice also names the most common cause of total
+  silence in a parallel lane (silently OOM-killed workers) and the fix (cap
+  `workers x per-worker memory`).
 
 `turbo_tests2` supports custom formatter such as Fuubar, but you might need to require it:
 
@@ -859,7 +914,7 @@ Thanks for RTFM. ☺️
 [📌gitmoji]: https://gitmoji.dev
 [📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
 [🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
-[🧮kloc-img]: https://img.shields.io/badge/KLOC-1.035-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
+[🧮kloc-img]: https://img.shields.io/badge/KLOC-1.077-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
 [🔐security]: https://github.com/galtzo-floss/turbo_tests2/blob/main/SECURITY.md
 [🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
 [📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
@@ -885,7 +940,7 @@ Thanks for RTFM. ☺️
 | Field | Value |
 |---|---|
 | Package | turbo_tests2 |
-| Description | 🚀 turbo_tests2` is an RSpec parallel test runner built on `parallel_tests`. It uses `parallel_tests` for process counts, file grouping, and runtime balancing, then replaces interleaved worker output with incremental summarized RSpec reporting. Source code of `turbo_test2` gem is based on Discourse and Rubygems work in this area (see README.md). |
+| Description | 🚀 turbo_tests2` is an RSpec parallel test runner. It uses runtime_groups for process counts, pid bookkeeping, file grouping, and runtime balancing, and replaces interleaved worker output with incremental summarized RSpec reporting. Source code of `turbo_test2` gem is based on Discourse and Rubygems work in this area (see README.md). |
 | Homepage | https://github.com/galtzo-floss/turbo_tests2 |
 | Source | https://github.com/galtzo-floss/turbo_tests2 |
 | License | `MIT` |

data/lib/turbo_tests/cli.rb

@@ -243,6 +243,16 @@ module TurboTests
       ]
     end
 
+    # Inclusive ceiling on the worker pool when neither --count nor --workers
+    # is passed. Explicit CLI counts always win. Two main uses:
+    #
+    # 1. Outer orchestrators (kettle-family) budgeting a machine-wide process
+    #    pool across concurrent family jobs.
+    # 2. Resource-constrained CI, where each worker is an expensive process —
+    #    JVM engines (JRuby, TruffleRuby) spawn a full JVM per worker, so cap
+    #    the pool (typically 2) AND the per-worker heap (e.g. JRUBY_OPTS=
+    #    "-J-Xmx1280m") to stay under runner RAM. See README
+    #    "Resource-Constrained CI (JVM Engines)".
     def max_processes_from_environment
       value = ENV.fetch("TURBO_TESTS2_MAX_PROCESSES", "").strip
       return if value.empty?
@@ -353,20 +363,19 @@ module TurboTests
         exit(1)
       end
 
-      processes = ParallelTests.determine_number_of_processes(count)
-      pids = ParallelTests.with_pid_file do
-        (1..processes).map do |process_id|
+      processes = RuntimeGroups.process_count(count)
+      RuntimeGroups::PidFile.with do |pid_path|
+        pids = (1..processes).map do |process_id|
           env = {
             "TEST_ENV_NUMBER" => process_id.to_s,
             "PARALLEL_TEST_GROUPS" => processes.to_s,
-            "PARALLEL_PID_FILE" => ParallelTests.pid_file_path
+            "PARALLEL_PID_FILE" => pid_path
           }
           Process.spawn(env, *args)
         end
-      end
-      statuses = pids.map { |pid| Process.wait2(pid).last }
-
-      exit(statuses.all?(&:success?) ? 0 : 1)
+        statuses = pids.map { |pid| Process.wait2(pid).last }
+        statuses.all?(&:success?) ? 0 : 1
+      end.tap { |code| exit(code) }
     rescue OptionParser::ParseError => e
       warn(e.message)
       warn(parser)

data/lib/turbo_tests/reporter.rb

@@ -80,7 +80,7 @@ module TurboTests
     end
 
     def report_number_of_tests(groups)
-      name = ParallelTests::RSpec::Runner.test_file_name
+      name = "spec"
 
       num_processes = groups.size
       num_tests = groups.map(&:size).sum

data/lib/turbo_tests/runner.rb

@@ -3,7 +3,7 @@
 require "json"
 require "timeout"
 require "fileutils"
-require "parallel_tests/rspec/runner"
+require "runtime_groups"
 require "rspec/core"
 require "shellwords"
 require "tempfile"
@@ -30,14 +30,18 @@ module TurboTests
 
     class << self
       def create(count)
-        # We are unable to load parallel tests' tasks in the normal way (top of file)
-        # because it requires that the Rails.application instance already be configured
-        require "parallel_tests/tasks"
-
-        ENV["PARALLEL_TEST_FIRST_IS_1"] = "true"
-        command = ["bundle", "exec", "rake", "db:create", "RAILS_ENV=#{ParallelTests::Tasks.rails_env}"]
-        args = {count: count.to_s}
-        ParallelTests::Tasks.run_in_parallel(command, args)
+        # Rails bootstrap: prepare the database once per worker, in parallel.
+        # Implemented directly with Process.spawn so the runner does not
+        # depend on parallel_tests for anything.
+        count = RuntimeGroups.process_count(count)
+        pids = (1..count).map do |process_id|
+          Process.spawn(
+            {"TEST_ENV_NUMBER" => process_id.to_s, "PARALLEL_TEST_GROUPS" => count.to_s},
+            "bundle", "exec", "rake", "db:create", "RAILS_ENV=#{ENV.fetch("RAILS_ENV", "test")}"
+          )
+        end
+        statuses = pids.map { |pid| Process.wait2(pid).last }
+        statuses.all?(&:success?) ? 0 : 1
       end
 
       def run(opts = {})
@@ -249,10 +253,11 @@ module TurboTests
       @load_time = 0
       @load_count = 0
       @failure_count = 0
-      # Deprecation-period default: disabled. In 4.0 the kill behavior
-      # becomes the default (TURBO_TESTS2_WATCHDOG_SECONDS=900) and this
-      # warning is removed. See README "Stall watchdog".
-      @watchdog_seconds = Float(ENV.fetch("TURBO_TESTS2_WATCHDOG_SECONDS", "0"))
+      # 4.0 default: kill behavior on (900s of total worker silence). Set
+      # TURBO_TESTS2_WATCHDOG_SECONDS=0 to disable explicitly (a long-hang
+      # run then ends via whatever outer watcher exists). See README
+      # "Stall watchdog".
+      @watchdog_seconds = Float(ENV.fetch("TURBO_TESTS2_WATCHDOG_SECONDS", "900"))
       @watchdog_warned = false
       @last_message_monotonic = nil
 
@@ -279,10 +284,9 @@ module TurboTests
 
     def run
       parallel_tests_options = @parallel_options.reject { |key, _value| key == :only_group }
-      tests_with_size = ParallelTests::RSpec::Runner.tests_with_size(@files, parallel_tests_options.merge(quiet: true))
       @num_processes = [
-        ParallelTests.determine_number_of_processes(@count),
-        tests_with_size.size
+        RuntimeGroups.process_count(@count),
+        @files.size
       ].min
 
       if @num_processes.zero?
@@ -292,10 +296,13 @@ module TurboTests
       end
 
       tests_in_groups =
-        ParallelTests::RSpec::Runner.tests_in_groups(
+        RuntimeGroups::Grouping.in_groups(
           @files,
           @num_processes,
-          **parallel_tests_options
+          group_by: parallel_tests_options[:group_by] || :runtime,
+          runtime_log: parallel_tests_options[:runtime_log],
+          allowed_missing: (parallel_tests_options[:allowed_missing_percent] || 100).to_f / 100,
+          unknown_runtime: parallel_tests_options[:unknown_runtime]
         )
       tests_in_groups = selected_groups(tests_in_groups) if @parallel_options[:only_group]
       @tests_in_groups = tests_in_groups
@@ -304,60 +311,72 @@ module TurboTests
         record_runtime: @record_runtime
       }
 
-      ParallelTests.with_pid_file do
-        exit_status = nil
-        report_coverage = false
-
-        @reporter.report(tests_in_groups) do |_reporter|
-          old_signal = Signal.trap(:INT) { handle_interrupt }
-
-          @wait_threads = tests_in_groups.map.with_index do |tests, process_id|
-            start_regular_subprocess(tests, process_id + 1, **subprocess_opts)
-          end.compact
-          @interrupt_handled = false
-
-          handle_messages
-
-          if @watchdog_fired
-            # Workers were SIGKILLed by the watchdog; their reader threads end
-            # at EOF and waiters return as the OS reaps them — but if a kill
-            # failed (permissions, zombie, pid reuse) an unbounded join here
-            # would hang the run anyway, defeating the watchdog. Bound every
-            # join and treat any worker that still has not exited as failed.
-            @threads.each do |thread|
-              thread.join(FORCE_STOP_JOIN)
-              thread.kill if thread.alive?
-            end
-            @wait_threads.each do |wait_thr|
-              wait_thr.join(FORCE_STOP_JOIN)
-            end
-            flush_worker_output unless stream_worker_output?
-            exit_status = 1
-          else
-            @threads.each(&:join)
+      pid_file_was_set = ENV.key?("PARALLEL_PID_FILE")
+      pid_file_before = ENV["PARALLEL_PID_FILE"]
+      begin
+        RuntimeGroups::PidFile.with do |pid_path|
+          ENV["PARALLEL_PID_FILE"] = pid_path
+
+          exit_status = nil
+          report_coverage = false
+
+          @reporter.report(tests_in_groups) do |_reporter|
+            old_signal = Signal.trap(:INT) { handle_interrupt }
+
+            @wait_threads = tests_in_groups.map.with_index do |tests, process_id|
+              start_regular_subprocess(tests, process_id + 1, **subprocess_opts)
+            end.compact
+            @interrupt_handled = false
+
+            handle_messages
+
+            if @watchdog_fired
+              # Workers were SIGKILLed by the watchdog; their reader threads end
+              # at EOF and waiters return as the OS reaps them — but if a kill
+              # failed (permissions, zombie, pid reuse) an unbounded join here
+              # would hang the run anyway, defeating the watchdog. Bound every
+              # join and treat any worker that still has not exited as failed.
+              @threads.each do |thread|
+                thread.join(FORCE_STOP_JOIN)
+                thread.kill if thread.alive?
+              end
+              @wait_threads.each do |wait_thr|
+                wait_thr.join(FORCE_STOP_JOIN)
+              end
+              flush_worker_output unless stream_worker_output?
+              exit_status = 1
+            else
+              @threads.each(&:join)
 
-            report_workers_without_results
+              report_workers_without_results
 
-            report_failed_group(tests_in_groups) if @print_failed_group
+              report_failed_group(tests_in_groups) if @print_failed_group
 
-            Signal.trap(:INT, old_signal)
+              Signal.trap(:INT, old_signal)
 
-            statuses = @wait_threads.map(&:value)
+              statuses = @wait_threads.map(&:value)
 
-            if @reporter.failed_examples.empty? && statuses.all?(&:success?)
-              flush_successful_worker_output
-              report_coverage = true
-              exit_status = 0
-            else
-              flush_worker_output unless stream_worker_output?
-              # From https://github.com/galtzo-floss/turbo_tests2/pull/20/
-              exit_status = statuses.map(&:exitstatus).max
+              if @reporter.failed_examples.empty? && statuses.all?(&:success?)
+                flush_successful_worker_output
+                report_coverage = true
+                exit_status = 0
+              else
+                flush_worker_output unless stream_worker_output?
+                # From https://github.com/galtzo-floss/turbo_tests2/pull/20/
+                exit_status = statuses.map(&:exitstatus).max
+              end
             end
           end
-        end
 
-        flush_coverage_summary if report_coverage
-        exit_status
+          flush_coverage_summary if report_coverage
+          exit_status
+        end
+      ensure
+        if pid_file_was_set
+          ENV["PARALLEL_PID_FILE"] = pid_file_before
+        else
+          ENV.delete("PARALLEL_PID_FILE")
+        end
       end
     end
 
@@ -408,6 +427,14 @@ module TurboTests
       )
     end
 
+    # Spawns one worker as a real OS subprocess via Open3.popen3 — never
+    # Kernel#fork. This is what makes the worker pool engine-portable: MRI,
+    # JRuby, and TruffleRuby all spawn subprocesses the same way, and the
+    # default pool size is Etc.nprocessors on every engine. Remember each
+    # worker is a full interpreter/VM process: on JVM engines that is a whole
+    # JVM with its own heap, so bound the pool (TURBO_TESTS2_MAX_PROCESSES)
+    # and the per-worker heap (e.g. JRUBY_OPTS) on memory-constrained
+    # runners — see README "Resource-Constrained CI (JVM Engines)".
     def start_subprocess(env, extra_args, tests, process_id, record_runtime:)
       if tests.empty?
         @messages << {
@@ -439,7 +466,7 @@ module TurboTests
             FileUtils.mkdir_p(File.dirname(@runtime_log))
             [
               "--format",
-              "ParallelTests::RSpec::RuntimeLogger",
+              "TurboTests::RuntimeLogger",
               "--out",
               @runtime_log
             ]
@@ -459,7 +486,7 @@ module TurboTests
           []
         end
 
-        spec_opts = self.class.worker_spec_opts(ParallelTests::RSpec::Runner.send(:spec_opts))
+        spec_opts = self.class.worker_spec_opts(RuntimeGroups::Options.spec_opts)
 
         command = [
           *command_name,
@@ -548,11 +575,11 @@ module TurboTests
     end
 
     def track_parallel_pid(pid, pid_file_path = parallel_pid_file_path)
-      ParallelTests::Pids.new(pid_file_path).add(pid) if pid && pid_file_path
+      RuntimeGroups::Pids.new(pid_file_path).add(pid) if pid && pid_file_path
     end
 
     def untrack_parallel_pid(pid, pid_file_path = parallel_pid_file_path)
-      ParallelTests::Pids.new(pid_file_path).delete(pid) if pid && pid_file_path
+      RuntimeGroups::Pids.new(pid_file_path).delete(pid) if pid && pid_file_path
     end
 
     def start_copy_thread(src, process_id, stream)
@@ -756,6 +783,14 @@ module TurboTests
     # on the queue forever — CI jobs hung for hours with no diagnostics.
     # On a stall we kill the surviving worker pids, report which groups
     # never finished, and fail the run instead.
+    #
+    # Note the OOM killer is the most common silent worker death on CI: the
+    # kernel kills the process with no stderr, no exit message, and no
+    # partial output. Workers are spawn-based real processes (never forked),
+    # so on JVM engines (JRuby, TruffleRuby) each one is a full JVM whose
+    # default heap can be 1/4 of machine RAM — a core-count pool of those
+    # exhausts a CI runner. Bound `workers x per-worker memory` on
+    # constrained runners (see README "Resource-Constrained CI").
     def pop_message_with_watchdog
       # 0 (the default) disables the watchdog: block indefinitely, exactly the
       # pre-watchdog behavior.
@@ -805,10 +840,10 @@ module TurboTests
     end
 
     def tracked_pids
-      path = ParallelTests.pid_file_path
+      path = parallel_pid_file_path
       return [] unless path && File.file?(path)
 
-      File.read(path).lines.map { |l| l.strip.to_i }.reject(&:zero?)
+      RuntimeGroups::Pids.new(path).to_a
     rescue
       []
     end
@@ -850,13 +885,13 @@ module TurboTests
           if silence >= 900
             warn(
               "turbo_tests2: workers have been silent for #{silence.to_i}s. " \
-              "In turbo_tests2 4.0 (and now when TURBO_TESTS2_WATCHDOG_SECONDS is set) a stall this long kills the " \
-              "remaining workers and fails the run. To adopt the kill now set TURBO_TESTS2_WATCHDOG_SECONDS=900 (or " \
-              "another value). To keep waiting indefinitely set TURBO_TESTS2_WATCHDOG_SECONDS=0; be aware that an " \
-              "outer process watcher (a CI job timeout, a container orchestrator, or a local supervisor) will then " \
-              "be what eventually ends the run, usually without a useful stall report. A single long test only " \
-              "trips this in serial lanes (TURBO_TESTS2_MAX_PROCESSES=1); in parallel lanes other workers keep " \
-              "messages flowing."
+              "The stall watchdog is disabled (TURBO_TESTS2_WATCHDOG_SECONDS=0), so this run will keep waiting " \
+              "until some outer watcher ends it — a CI job timeout, a container orchestrator, or a local " \
+              "supervisor — usually without a useful stall report. Total silence in a PARALLEL lane usually means " \
+              "workers died rather than slowed down — the most common cause is the kernel OOM killer (each worker " \
+              "is a real spawned process; on JVM engines each is a full JVM whose default heap is 1/4 of RAM). " \
+              "Cap workers x per-worker memory: TURBO_TESTS2_MAX_PROCESSES=2 with a capped heap (e.g. " \
+              "JRUBY_OPTS=\"-J-Xmx1280m\"), or re-enable the watchdog to convert hangs into fast failures."
             )
             warned_silence = true
           end

data/lib/turbo_tests/runtime_logger.rb

@@ -0,0 +1,77 @@
+# frozen_string_literal: true
+
+require "rspec/core"
+require "runtime_groups"
+
+# RSpec formatter that records per-file runtimes into a runtime log
+# (TurboGroups path:seconds format, read back by RuntimeGroups::Grouping).
+# Semantics retained from parallel_tests' RuntimeLogger:
+# - only records when running in parallel (TEST_ENV_NUMBER is set)
+# - accumulates time across nested example groups per top-level file
+# - appends at dump time, sorted slowest-first
+# Min-merging across re-runs happens in RuntimeGroups::RuntimeLog.read via
+# last-write-wins per line, and worker groups are disjoint, so concurrent
+# workers never write the same file.
+class TurboTests::RuntimeLogger
+  RSpec::Core::Formatters.register self, :example_group_started, :example_group_finished, :start_dump
+
+  def initialize(output)
+    @output = output
+    @example_times = Hash.new(0)
+    @group_nesting = 0
+    @time = 0.0
+  end
+
+  def example_group_started(_notification)
+    @group_nesting += 1
+    @time = monotonic if @group_nesting == 1
+  end
+
+  def example_group_finished(notification)
+    @group_nesting -= 1
+    if @group_nesting.zero?
+      @example_times[notification.group.file_path] += monotonic - @time
+    end
+  end
+
+  def seed(*)
+  end
+
+  def dump_summary(*)
+  end
+
+  def dump_failures(*)
+  end
+
+  def dump_pending(*)
+  end
+
+  def message(*)
+  end
+
+  def close(*)
+  end
+
+  def start_dump(*)
+    return unless ENV["TEST_ENV_NUMBER"] # only record when running in parallel
+
+    log_path = output_path
+    return unless log_path
+
+    log = RuntimeGroups::RuntimeLog.new(log_path)
+    @example_times.sort_by(&:last).reverse_each do |file, time|
+      relative_path = file.sub(%r{^#{Regexp.escape(Dir.pwd)}/}, "").sub(%r{^\./}, "")
+      log.log(relative_path, [time, 0].max)
+    end
+  end
+
+  private
+
+  def monotonic
+    Process.clock_gettime(Process::CLOCK_MONOTONIC)
+  end
+
+  def output_path
+    @output.respond_to?(:path) ? @output.path : @output.to_s
+  end
+end

data/lib/turbo_tests/version.rb

@@ -4,7 +4,7 @@ module TurboTests
   # Version namespace for this gem.
   module Version
     # Current gem version.
-    VERSION = "3.2.16"
+    VERSION = "4.0.0"
   end
   # Current gem version exposed at the traditional constant location.
   VERSION = Version::VERSION # Traditional Constant Location

data/lib/turbo_tests.rb

@@ -7,10 +7,10 @@ require "json"
 
 require "rspec"
 
-require "parallel_tests"
-require "parallel_tests/rspec/runner"
+require "runtime_groups"
 
 require "turbo_tests/reporter"
+require "turbo_tests/runtime_logger"
 require "turbo_tests/runner"
 require "turbo_tests/json_rows_formatter"
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: turbo_tests2
 version: !ruby/object:Gem::Version
-  version: 3.2.16
+  version: 4.0.0
 platform: ruby
 authors:
 - Illia
@@ -49,39 +49,39 @@ cert_chain:
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
-  name: parallel_tests
+  name: rspec
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 3.3.0
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '6'
+        version: '3.10'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 3.3.0
-    - - "<"
-      - !ruby/object:Gem::Version
-        version: '6'
+        version: '3.10'
 - !ruby/object:Gem::Dependency
-  name: rspec
+  name: runtime_groups
   requirement: !ruby/object:Gem::Requirement
     requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
     - - ">="
       - !ruby/object:Gem::Version
-        version: '3.10'
+        version: 0.1.3
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
     - - ">="
       - !ruby/object:Gem::Version
-        version: '3.10'
+        version: 0.1.3
 - !ruby/object:Gem::Dependency
   name: version_gem
   requirement: !ruby/object:Gem::Requirement
@@ -284,11 +284,10 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 2.0.15
-description: "\U0001F680 turbo_tests2` is an RSpec parallel test runner built on `parallel_tests`.
-  It uses `parallel_tests` for process counts, file grouping, and runtime balancing,
-  then replaces interleaved worker output with incremental summarized RSpec reporting.
-  Source code of `turbo_test2` gem is based on Discourse and Rubygems work in this
-  area (see README.md)."
+description: "\U0001F680 turbo_tests2` is an RSpec parallel test runner. It uses runtime_groups
+  for process counts, pid bookkeeping, file grouping, and runtime balancing, and replaces
+  interleaved worker output with incremental summarized RSpec reporting. Source code
+  of `turbo_test2` gem is based on Discourse and Rubygems work in this area (see README.md)."
 email:
 - floss@galtzo.com
 executables:
@@ -305,6 +304,7 @@ files:
 - lib/turbo_tests/json_rows_formatter.rb
 - lib/turbo_tests/reporter.rb
 - lib/turbo_tests/runner.rb
+- lib/turbo_tests/runtime_logger.rb
 - lib/turbo_tests/shim.rb
 - lib/turbo_tests/utils/paths.rb
 - lib/turbo_tests/version.rb
@@ -316,10 +316,10 @@ licenses:
 - MIT
 metadata:
   homepage_uri: https://turbo-tests2.galtzo.com
-  source_code_uri: https://github.com/galtzo-floss/turbo_tests2/tree/v3.2.16
-  changelog_uri: https://github.com/galtzo-floss/turbo_tests2/blob/v3.2.16/CHANGELOG.md
+  source_code_uri: https://github.com/galtzo-floss/turbo_tests2/tree/v4.0.0
+  changelog_uri: https://github.com/galtzo-floss/turbo_tests2/blob/v4.0.0/CHANGELOG.md
   bug_tracker_uri: https://github.com/galtzo-floss/turbo_tests2/issues
-  documentation_uri: https://www.rubydoc.info/gems/turbo_tests2/3.2.16
+  documentation_uri: https://www.rubydoc.info/gems/turbo_tests2/4.0.0
   funding_uri: https://github.com/sponsors/pboling
   wiki_uri: https://github.com/galtzo-floss/turbo_tests2/wiki
   news_uri: https://www.railsbling.com/tags/turbo_tests2
@@ -328,8 +328,8 @@ metadata:
   rubygems_mfa_required: 'true'
 rdoc_options:
 - "--title"
-- "turbo_tests2 - \U0001F680 RSpec parallel test runner built on `parallel_tests`,
-  with incremental summarized output"
+- "turbo_tests2 - \U0001F680 RSpec parallel test runner with incremental summarized
+  output"
 - "--main"
 - README.md
 - "--exclude"
@@ -352,6 +352,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.20
 specification_version: 4
-summary: "\U0001F680 RSpec parallel test runner built on `parallel_tests`, with incremental
-  summarized output"
+summary: "\U0001F680 RSpec parallel test runner with incremental summarized output"
 test_files: []