polyrun 2.1.2 → 2.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 756cf8e1b8e2176c4097520752dfdd4b856d03b134a999462641f90a9f217e71
4
- data.tar.gz: 284e536d886b4bfc913415b0e3301a31ca98ef3446da1b24163bcc6af485e642
3
+ metadata.gz: 4a5cd4e5af5195cb25ca3185b3e828556746f21a6911f8a60f1b43b5b6a316d0
4
+ data.tar.gz: 67362c68f463339cfd8a66e665b9fb6a5f75cf8ed6276e1a846c68147a27e848
5
5
  SHA512:
6
- metadata.gz: 445f64e0e62be50ab6c63026f27ad8f5f093521bbeed4a05b0ce7adc6a13bcb0496f1489d8243b403f859826e599194c3399a761a9210cbbf5d8027f3fdba52e
7
- data.tar.gz: 554a292c2c7ac40e3df11d886ca7a71c10b81f1fa144af00c5c1d20b950ff6d999306be1e3fa75a9982450b2d202bc2bee34d5201ab39dbb386b1b73b42f1403
6
+ metadata.gz: b7f7ff8592774bfb1bd173753f5c3cca77d99a392a9dbabdd0bf11c85b510a3dc4cf0c2a66912cf5d48384445e74039a2cfa296d6aeb40b5d008aa575700ef0b
7
+ data.tar.gz: 05aad7cb17b505112548d072895d56c88f885f07b31e3052ab12e5b4ab0979a555bdf9872380107a60211a8bd54dbd176b98a8970eaba7756fa2e87f8a475a15
data/CHANGELOG.md CHANGED
@@ -2,6 +2,13 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ ## 2.1.3 (2026-07-10)
6
+
7
+ - Add `Polyrun::WorkerOutput` for per-shard worker stdout/stderr routing (`POLYRUN_WORKER_OUTPUT_ROUTING=1` or `POLYRUN_WORKER_LOG_DIR`); optional prefixed TTY echo (`POLYRUN_WORKER_OUTPUT_PREFIX=0` for log-only).
8
+ - Add `partition.paths_build.exclude_prefixes` to omit path prefixes from `build-paths` / `start` while keeping files runnable outside polyrun.
9
+ - Add `Polyrun::RSpec.install_example_debug!`, `install_example_timeout!`, `install_example_rails_logging!`, `install_example_prosopite!` (`POLYRUN_EXAMPLE_DEBUG=1` and related env vars).
10
+ - Add `Polyrun::RSpec.install_sharded_formatter_compat!` for progress formatters (e.g. Fuubar) under `POLYRUN_SHARD_*`.
11
+
5
12
  ## 2.1.2 (2026-07-05)
6
13
 
7
14
  - Fix per-example timing JSON when RSpec supplies `absolute_file_path` metadata; fix formatter registration when `install_example_timing!` uses a custom output path.
data/CONTRIBUTING.md CHANGED
@@ -27,6 +27,7 @@ Run the suite under alternate Ruby constraints (see `Appraisals` and `gemfiles/`
27
27
  ```bash
28
28
  bundle exec appraisal ruby32 rspec
29
29
  bundle exec appraisal ruby34 rspec
30
+ bundle exec appraisal ruby40 rspec
30
31
  ```
31
32
 
32
33
  ## Linting
@@ -44,11 +44,40 @@ Rule: anything expensive (compile, `yarn`, Playwright download) belongs in prepa
44
44
  |--------|---------|
45
45
  | Plain glob | `partition.paths_build.all_glob: spec/**/*_spec.rb` and empty or minimal `stages` |
46
46
  | Ordered stages | `partition.paths_build.stages`: regex (e.g. slow integration first) or `sort_by_substring_order` for stable ordering |
47
+ | Exclude prefixes | `partition.paths_build.exclude_prefixes`: omit paths (e.g. `spec/benchmark/`) from `paths_file` while keeping them runnable outside polyrun |
47
48
 
48
49
  `paths_build` controls **membership** in `partition.paths_file`, not shard assignment order. Default `round_robin` sorts paths alphabetically before mod assignment. Use `strategy: preserve_order_round_robin` to honor paths-file line order. Set `partition.timing_file` without `strategy` to auto-select `cost_binpack`; use `lazy_robin` for round-robin assignment with timing diagnostics.
49
50
 
50
51
  Refresh list: `polyrun -c polyrun.yml build-paths` (also runs automatically before `plan` / `run-shards` when configured).
51
52
 
53
+ ## 5b. Parallel worker logs (opt-in)
54
+
55
+ When `run-shards` fans out multiple workers, stdout/stderr interleave on the parent TTY by default.
56
+
57
+ | Env | Effect |
58
+ |--------|--------|
59
+ | `POLYRUN_WORKER_OUTPUT_ROUTING=1` | Route each worker through per-shard log files (default dir `tmp/polyrun/workers`) |
60
+ | `POLYRUN_WORKER_LOG_DIR` | Custom log directory (also enables routing when set) |
61
+ | `POLYRUN_WORKER_OUTPUT_ROUTING=0` | Force inherited stdio even when log dir is set |
62
+ | `POLYRUN_WORKER_OUTPUT_PREFIX=0` | Write logs only; no live prefixed TTY echo |
63
+
64
+ Carriage-return progress chunks (Fuubar-style) pass through unprefixed on the TTY and in logs.
65
+
66
+ ## 5c. RSpec example debug (local investigation)
67
+
68
+ Orchestration tracing stays on `DEBUG=1` / `POLYRUN_DEBUG=1`. Per-example tooling is separate:
69
+
70
+ | Env | Effect |
71
+ |--------|--------|
72
+ | `POLYRUN_EXAMPLE_DEBUG=1` | Enable example debug installers |
73
+ | `POLYRUN_DEBUG_SQL=1` or `DEBUG_SQL=1` | Log mutating SQL |
74
+ | `POLYRUN_DEBUG_TRACE=1` or `DEBUG_TRACE=1` | TracePoint under app root |
75
+ | `DEBUG_PROSOPITE=1` | Prosopite N+1 scan when gem is loaded |
76
+ | `DEBUG_LOG_LEVEL` | Rails / ActiveRecord logger severity: `debug`, `info`, `warn`, `error`, `fatal`, or Ruby Logger integers (`0` debug … `4` fatal); default `debug` |
77
+ | `RSPEC_EXAMPLE_TIMEOUT_SEC` | Per-example timeout (disabled while example debug is on) |
78
+
79
+ In `spec_helper` / `spec/support`: `require "polyrun/rspec"` then `Polyrun::RSpec.install_example_debug!`, `install_example_rails_logging!`, `install_example_timeout!`, and `install_sharded_formatter_compat!` when using Fuubar under `POLYRUN_SHARD_*`. Pair with `install_worker_ping!` for idle timeouts.
80
+
52
81
  ## 6. Coverage and CI reports
53
82
 
54
83
  | Field | Options |
@@ -23,6 +23,9 @@ module Polyrun
23
23
  Parallel workers: POLYRUN_WORKERS default 5, max 10 (run-shards / parallel-rspec / start). CI local processes per job: POLYRUN_SHARD_PROCESSES or ci-shard --shard-processes (not POLYRUN_WORKERS)
24
24
  Per-worker wall timeout: --worker-timeout SEC or POLYRUN_WORKER_TIMEOUT_SEC. Exit 124; parent stops remaining workers.
25
25
  Per-worker idle timeout: --worker-idle-timeout SEC or POLYRUN_WORKER_IDLE_TIMEOUT_SEC after a progress ping (POLYRUN_WORKER_PING_FILE). Enable pings in test setup. Exit 125. Optional periodic pings: POLYRUN_WORKER_PING_THREAD=1 (POLYRUN_WORKER_PING_INTERVAL_SEC).
26
+ Worker output routing (opt-in): POLYRUN_WORKER_OUTPUT_ROUTING=1 or POLYRUN_WORKER_LOG_DIR; per-shard logs under tmp/polyrun/workers (POLYRUN_WORKER_OUTPUT_PREFIX=0 for log-only)
27
+ Example debug (RSpec, opt-in): POLYRUN_EXAMPLE_DEBUG=1; POLYRUN_DEBUG_SQL / POLYRUN_DEBUG_TRACE
28
+ Sharded formatter compat: progress formatters (e.g. Fuubar) under POLYRUN_SHARD_* (see docs/SETUP_PROFILE.md)
26
29
  Orchestration warnings on process stderr: POLYRUN_ORCHESTRATION_STDERR=1
27
30
  Spec quality (opt-in): POLYRUN_SPEC_QUALITY=1; run-shards --merge-spec-quality; merge-spec-quality / report-spec-quality
28
31
  Partition timing granularity (default file): POLYRUN_TIMING_GRANULARITY=file|example (experimental; see partition.timing_granularity)
@@ -3,6 +3,7 @@ require "optparse"
3
3
  require "shellwords"
4
4
 
5
5
  require_relative "../queue/duration"
6
+ require_relative "../queue/worker_loop"
6
7
 
7
8
  module Polyrun
8
9
  class CLI
@@ -83,44 +84,16 @@ module Polyrun
83
84
  workers.times do |i|
84
85
  wid = "worker-#{i}"
85
86
  pid = Process.fork do
86
- run_queue_worker_loop(store: store, worker_id: wid, batch: batch, cmd: cmd, on_failure: on_failure)
87
+ result = Polyrun::Queue::WorkerLoop.run(
88
+ store: store, worker_id: wid, batch: batch, cmd: cmd, on_failure: on_failure
89
+ )
90
+ exit result[:exit_code]
87
91
  end
88
92
  pids << {pid: pid, worker_id: wid}
89
93
  end
90
94
  pids
91
95
  end
92
96
 
93
- def run_queue_worker_loop(store:, worker_id:, batch:, cmd:, on_failure:)
94
- batches_ok = 0
95
- batches_fail = 0
96
- loop do
97
- claim = store.claim!(worker_id: worker_id, batch_size: batch)
98
- paths = claim["paths"] || []
99
- break if paths.empty?
100
-
101
- code = run_queue_run_batch(cmd, paths)
102
- if code == 0
103
- store.ack!(lease_id: claim["lease_id"], worker_id: worker_id)
104
- batches_ok += 1
105
- elsif on_failure.to_s == "requeue"
106
- store.reclaim_lease!(claim["lease_id"])
107
- batches_fail += 1
108
- exit 1
109
- else
110
- batches_fail += 1
111
- exit code.zero? ? 1 : code
112
- end
113
- end
114
- exit 0
115
- rescue Polyrun::Error => e
116
- Polyrun::Log.warn "polyrun run-queue worker #{worker_id}: #{e.message}"
117
- exit 2
118
- end
119
-
120
- def run_queue_run_batch(cmd, paths)
121
- system(*cmd, *paths) ? 0 : ($?.exitstatus || 1)
122
- end
123
-
124
97
  def run_queue_wait_workers(pids, store:, on_failure:)
125
98
  ok = 0
126
99
  fail = 0
@@ -1,5 +1,6 @@
1
1
  require "fileutils"
2
2
 
3
+ require_relative "../worker_output"
3
4
  require_relative "run_shards_parallel_wait"
4
5
 
5
6
  module Polyrun
@@ -22,6 +23,7 @@ module Polyrun
22
23
  mt = ctx[:matrix_shard_total]
23
24
 
24
25
  pids = []
26
+ Polyrun::WorkerOutput.prepare_log_dir! if Polyrun::WorkerOutput.routing_enabled? && parallel
25
27
  workers.times do |shard|
26
28
  paths = plan.shard(shard)
27
29
  if paths.empty?
@@ -60,6 +62,9 @@ module Polyrun
60
62
  pids << {pid: pid, shard: shard, spawned_at: spawned_at, ping_path: ping_path}
61
63
  Polyrun::Debug.log("[parent pid=#{$$}] run-shards: Process.spawn shard=#{shard} child_pid=#{pid} spec_files=#{paths.size}")
62
64
  Polyrun::Log.warn "polyrun run-shards: started shard #{shard} pid=#{pid} (#{paths.size} file(s))" if parallel
65
+ if Polyrun::WorkerOutput.routing_enabled? && parallel
66
+ Polyrun::Log.warn "polyrun run-shards: shard #{shard} log → #{Polyrun::WorkerOutput.log_path_for(shard)}"
67
+ end
63
68
  end
64
69
  [pids, nil]
65
70
  end
@@ -81,6 +86,8 @@ module Polyrun
81
86
  end
82
87
 
83
88
  def run_shards_spawn_one_worker(child_env, cmd, paths, hook_cfg)
89
+ return Polyrun::WorkerOutput.spawn_worker(child_env, cmd, paths, hook_cfg) if Polyrun::WorkerOutput.routing_enabled?
90
+
84
91
  if hook_cfg.worker_hooks? && !Polyrun::Hooks.disabled?
85
92
  Process.spawn(child_env, "sh", "-c", hook_cfg.build_worker_shell_script(cmd, paths))
86
93
  else
@@ -111,6 +111,7 @@ module Polyrun
111
111
  end
112
112
 
113
113
  def run_shards_finalize_reaped_worker!(h, hook_cfg, workers, st, shard_results, after_hook_err)
114
+ Polyrun::WorkerOutput.finish_worker(h[:pid])
114
115
  exitstatus = st.exitstatus
115
116
  ok = st.success?
116
117
  Polyrun::Debug.log("[parent pid=#{$$}] run-shards: Process.wait child_pid=#{h[:pid]} shard=#{h[:shard]} exit=#{exitstatus} success=#{ok}")
@@ -218,6 +219,7 @@ module Polyrun
218
219
  _t, loc = run_shards_read_worker_ping_payload(h[:ping_path])
219
220
  ping_suffix = (loc && !loc.to_s.strip.empty?) ? "; last ping #{loc.to_s.strip}" : ""
220
221
  Polyrun::Log.orchestration_warn "polyrun run-shards: WORKER IDLE TIMEOUT after #{idle_sec}s since last per-example progress ping — shard #{h[:shard]} pid #{h[:pid]}#{ping_suffix}."
222
+ Polyrun::WorkerOutput.warn_shard_log(h[:shard])
221
223
  Polyrun::Log.warn "polyrun run-shards: idle shard file sample: #{sample}#{suffix}"
222
224
  Polyrun::Log.warn "polyrun run-shards: enable per-example worker progress pings in your test setup so idle timeouts reflect real work; exit #{WORKER_IDLE_TIMEOUT_EXIT_STATUS}."
223
225
  end
@@ -262,6 +264,7 @@ module Polyrun
262
264
  ""
263
265
  end
264
266
  Polyrun::Log.orchestration_warn "polyrun run-shards: WORKER TIMEOUT after #{timeout_sec}s (wall time since worker spawn) — shard #{h[:shard]} pid #{h[:pid]}."
267
+ Polyrun::WorkerOutput.warn_shard_log(h[:shard])
265
268
  Polyrun::Log.warn "polyrun run-shards: timeout shard includes: #{sample}#{suffix}"
266
269
  Polyrun::Log.warn "polyrun run-shards: override with --worker-timeout SEC or POLYRUN_WORKER_TIMEOUT_SEC; recorded exit #{WORKER_TIMEOUT_EXIT_STATUS} for this worker."
267
270
  end
@@ -124,6 +124,7 @@ module Polyrun
124
124
 
125
125
  def run_shards_warn_interleaved(parallel, pid_count)
126
126
  return unless parallel && pid_count > 1
127
+ return if Polyrun::WorkerOutput.routing_enabled?
127
128
 
128
129
  Polyrun::Log.warn "polyrun run-shards: #{pid_count} children running; RSpec output below may be interleaved."
129
130
  Polyrun::Log.warn "polyrun run-shards: each worker prints its own summary line; the last \"N examples\" line is not a total across shards."
@@ -1,3 +1,5 @@
1
+ require_relative "../worker_output"
2
+
1
3
  module Polyrun
2
4
  class CLI
3
5
  # SIGINT/SIGTERM handling and non-blocking reap for parallel worker PIDs (used by run-shards / ci-shard fan-out).
@@ -7,12 +9,17 @@ module Polyrun
7
9
  def run_shards_log_interrupt_workers(pids, _ctx)
8
10
  parts = pids.map { |h| "shard=#{h[:shard]} pid=#{h[:pid]}" }
9
11
  Polyrun::Log.orchestration_warn "polyrun run-shards: SIGINT/SIGTERM while waiting on workers — stopping: #{parts.join(", ")}"
10
- Polyrun::Log.warn "polyrun run-shards: search this log for each shard's started … pid= line and RSpec output; repeat SIGINT during cleanup escalates to SIGKILL"
12
+ if Polyrun::WorkerOutput.routing_enabled?
13
+ Polyrun::Log.warn "polyrun run-shards: per-shard worker logs under #{Polyrun::WorkerOutput.worker_log_directory_label}"
14
+ else
15
+ Polyrun::Log.warn "polyrun run-shards: search this log for each shard's started … pid= line and RSpec output; repeat SIGINT during cleanup escalates to SIGKILL"
16
+ end
11
17
  end
12
18
 
13
19
  # Best-effort worker teardown then exit. Does not return.
14
20
  def run_shards_shutdown_on_signal!(pids, code)
15
21
  run_shards_log_interrupt_workers(pids, nil)
22
+ Polyrun::WorkerOutput.shutdown_all!
16
23
  run_shards_terminate_children!(pids)
17
24
  exit(code)
18
25
  rescue Interrupt
@@ -23,6 +30,7 @@ module Polyrun
23
30
 
24
31
  # Send SIGTERM to each worker PID and wait so Ctrl+C / SIGTERM does not leave orphans.
25
32
  def run_shards_terminate_children!(pids)
33
+ Polyrun::WorkerOutput.shutdown_all!
26
34
  run_shards_signal_workers_term(pids)
27
35
  run_shards_reap_worker_pids_interruptible(pids.map { |h| h[:pid] })
28
36
  end
@@ -40,12 +40,20 @@ module Polyrun
40
40
 
41
41
  pool = glob_under_cwd(all_glob, cwd)
42
42
  pool.uniq!
43
+ pool = reject_exclude_prefixes(pool, pb)
43
44
  stages = Array(pb["stages"])
44
45
  return sort_paths(pool) if stages.empty?
45
46
 
46
47
  apply_stages_to_pool(stages, pool, cwd)
47
48
  end
48
49
 
50
+ def reject_exclude_prefixes(pool, pb)
51
+ prefixes = Array(pb["exclude_prefixes"]).map(&:to_s).reject(&:empty?)
52
+ return pool if prefixes.empty?
53
+
54
+ pool.reject { |path| prefixes.any? { |prefix| path.start_with?(prefix) } }
55
+ end
56
+
49
57
  def apply_stages_to_pool(stages, pool, cwd)
50
58
  remaining = Set.new(pool)
51
59
  out = []
@@ -0,0 +1,39 @@
1
+ module Polyrun
2
+ module Queue
3
+ # Claim / run / ack loop for +polyrun run-queue+ workers (forked or direct).
4
+ module WorkerLoop
5
+ module_function
6
+
7
+ def run(store:, worker_id:, batch:, cmd:, on_failure:)
8
+ batches_ok = 0
9
+ batches_fail = 0
10
+ loop do
11
+ claim = store.claim!(worker_id: worker_id, batch_size: batch)
12
+ paths = claim["paths"] || []
13
+ break if paths.empty?
14
+
15
+ code = run_batch(cmd, paths)
16
+ if code == 0
17
+ store.ack!(lease_id: claim["lease_id"], worker_id: worker_id)
18
+ batches_ok += 1
19
+ elsif on_failure.to_s == "requeue"
20
+ store.reclaim_lease!(claim["lease_id"])
21
+ batches_fail += 1
22
+ return {ok: batches_ok, fail: batches_fail, exit_code: 1}
23
+ else
24
+ batches_fail += 1
25
+ return {ok: batches_ok, fail: batches_fail, exit_code: code.zero? ? 1 : code}
26
+ end
27
+ end
28
+ {ok: batches_ok, fail: batches_fail, exit_code: 0}
29
+ rescue Polyrun::Error => e
30
+ Polyrun::Log.warn "polyrun run-queue worker #{worker_id}: #{e.message}"
31
+ {ok: batches_ok, fail: batches_fail, exit_code: 2}
32
+ end
33
+
34
+ def run_batch(cmd, paths)
35
+ system(*cmd, *paths) ? 0 : ($?.exitstatus || 1)
36
+ end
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,158 @@
1
+ require "logger"
2
+ require "timeout"
3
+ require_relative "example_debug_instrumentation"
4
+
5
+ module Polyrun
6
+ module RSpec
7
+ # Per-example RSpec debugging for local investigation (SQL, TracePoint, timeouts).
8
+ #
9
+ # +POLYRUN_EXAMPLE_DEBUG=1+ enables installers; +DEBUG=1+ / +POLYRUN_DEBUG=1+ trace orchestration only.
10
+ # +POLYRUN_DEBUG_SQL=1+ or legacy +DEBUG_SQL=1+ logs mutating SQL.
11
+ # +POLYRUN_DEBUG_TRACE=1+ or legacy +DEBUG_TRACE=1+ traces :call/:raise under the app root.
12
+ # +DEBUG_LOG_LEVEL+ accepts Ruby Logger severities as integers (0 debug … 4 fatal) or names.
13
+ module ExampleDebug
14
+ LOG_LEVEL_BY_NAME = {
15
+ "debug" => Logger::DEBUG,
16
+ "info" => Logger::INFO,
17
+ "warn" => Logger::WARN,
18
+ "error" => Logger::ERROR,
19
+ "fatal" => Logger::FATAL
20
+ }.freeze
21
+
22
+ module_function
23
+
24
+ def enabled?
25
+ truthy?(ENV["POLYRUN_EXAMPLE_DEBUG"])
26
+ end
27
+
28
+ def sql_enabled?
29
+ enabled? && (truthy?(ENV["POLYRUN_DEBUG_SQL"]) || truthy?(ENV["DEBUG_SQL"]))
30
+ end
31
+
32
+ def trace_enabled?
33
+ enabled? && (truthy?(ENV["POLYRUN_DEBUG_TRACE"]) || truthy?(ENV["DEBUG_TRACE"]))
34
+ end
35
+
36
+ def prosopite_enabled?
37
+ enabled? && truthy?(ENV["DEBUG_PROSOPITE"])
38
+ end
39
+
40
+ def print_spec_enabled?
41
+ enabled? && truthy?(ENV["DEBUG_PRINT_SPEC"])
42
+ end
43
+
44
+ def example_timeout_disabled?
45
+ enabled?
46
+ end
47
+
48
+ def log_level
49
+ parse_log_level(ENV.fetch("DEBUG_LOG_LEVEL", "debug"))
50
+ end
51
+
52
+ def rails_log_level
53
+ case log_level
54
+ when Logger::DEBUG then :debug
55
+ when Logger::INFO then :info
56
+ when Logger::WARN then :warn
57
+ when Logger::ERROR then :error
58
+ else :fatal
59
+ end
60
+ end
61
+
62
+ def parse_log_level(raw)
63
+ normalized = raw.to_s.strip.downcase
64
+ return LOG_LEVEL_BY_NAME.fetch(normalized) if LOG_LEVEL_BY_NAME.key?(normalized)
65
+
66
+ Integer(normalized)
67
+ rescue ArgumentError
68
+ Logger::DEBUG
69
+ end
70
+
71
+ def install!(rspec_config: fetch_rspec_configuration!)
72
+ install_spec_path_helpers!(rspec_config)
73
+ install_sql_debug!(rspec_config) if sql_enabled?
74
+ install_trace_debug!(rspec_config) if trace_enabled?
75
+ end
76
+
77
+ def install_rails_logging!(rspec_config: fetch_rspec_configuration!)
78
+ return unless enabled?
79
+ return unless defined?(Rails)
80
+
81
+ rspec_config.before do |example|
82
+ group = example.metadata[:example_group]
83
+ Polyrun::Log.puts "\n\nRunning #{group[:file_path]}:#{group[:line_number]}"
84
+
85
+ level = log_level
86
+ Rails.logger.level = level
87
+ if defined?(ActiveRecord::Base)
88
+ ar_logger = Logger.new(Polyrun::Log.stdout)
89
+ ar_logger.level = level
90
+ ActiveRecord::Base.logger = ar_logger
91
+ end
92
+ end
93
+ end
94
+
95
+ def install_example_timeout!(
96
+ rspec_config,
97
+ seconds: ENV.fetch("RSPEC_EXAMPLE_TIMEOUT_SEC", "30").to_f
98
+ )
99
+ return if seconds <= 0
100
+ return if example_timeout_disabled?
101
+
102
+ rspec_config.around(:each) do |example|
103
+ if example.metadata[:benchmark] || example.metadata[:slow]
104
+ example.run
105
+ else
106
+ Timeout.timeout(seconds) { example.run }
107
+ end
108
+ rescue Timeout::Error
109
+ raise "Example timed out after #{seconds}s (#{example.location})"
110
+ end
111
+ end
112
+
113
+ def install_prosopite!(rspec_config: fetch_rspec_configuration!)
114
+ return unless prosopite_enabled?
115
+ return unless defined?(Prosopite) && defined?(Rails)
116
+
117
+ log_path = Rails.root.join("tmp", "prosopite_#{Time.current.strftime("%Y%m%d_%H%M%S")}.log")
118
+ Prosopite.custom_logger = Logger.new(log_path)
119
+
120
+ rspec_config.before { Prosopite.scan }
121
+ rspec_config.after { Prosopite.finish }
122
+ end
123
+
124
+ def install_spec_path_helpers!(rspec_config)
125
+ rspec_config.before do |example|
126
+ group = example.metadata[:example_group]
127
+ spec_file_path = group[:file_path]
128
+
129
+ define_singleton_method(:spec_dirname) { File.dirname(spec_file_path) }
130
+ define_singleton_method(:spec_basename) { File.basename(spec_file_path) }
131
+
132
+ if ExampleDebug.print_spec_enabled?
133
+ Polyrun::Log.puts "\nRunning #{spec_file_path}:#{group[:line_number]}\n"
134
+ end
135
+ end
136
+
137
+ rspec_config.after do |example|
138
+ next unless ExampleDebug.print_spec_enabled?
139
+
140
+ group = example.metadata[:example_group]
141
+ Polyrun::Log.puts "\nFinished #{group[:file_path]}:#{group[:line_number]}\n"
142
+ end
143
+ end
144
+
145
+ def fetch_rspec_configuration!
146
+ require "rspec/core"
147
+ ::RSpec.configuration
148
+ end
149
+
150
+ def truthy?(value)
151
+ return false if value.nil?
152
+
153
+ %w[1 true yes on].include?(value.to_s.strip.downcase)
154
+ end
155
+ private_class_method :truthy?, :parse_log_level, :fetch_rspec_configuration!
156
+ end
157
+ end
158
+ end
@@ -0,0 +1,73 @@
1
+ module Polyrun
2
+ module RSpec
3
+ module ExampleDebug
4
+ SKIPPED_SQL_PREFIX = /\A(?:SELECT|SET|SHOW|BEGIN|COMMIT|ROLLBACK|RELEASE|SAVEPOINT)/
5
+
6
+ module_function
7
+
8
+ def install_sql_debug!(rspec_config, io: Polyrun::Log.stdout)
9
+ return unless defined?(ActiveSupport::Notifications)
10
+
11
+ rspec_config.around do |example|
12
+ subscriber = ActiveSupport::Notifications.subscribe("sql.active_record") do |event|
13
+ payload = event.payload[:sql]
14
+ next unless loggable_sql?(payload)
15
+
16
+ line = sql_with_interpolated_binds(payload, event.payload[:type_casted_binds])
17
+ io.puts "+ #{line}"
18
+ end
19
+
20
+ example.run
21
+ ensure
22
+ ActiveSupport::Notifications.unsubscribe(subscriber) if subscriber
23
+ end
24
+ end
25
+
26
+ def install_trace_debug!(rspec_config, root: trace_root, io: Polyrun::Log.stdout)
27
+ require "pp"
28
+ root_path = File.expand_path(root.to_s)
29
+ trace = TracePoint.new do |trace_point|
30
+ if trace_point.event == :call && trace_point.path.to_s.start_with?(root_path)
31
+ io.puts PP.pp({event: :call, path: "#{trace_point.path}:#{trace_point.lineno}"}, +"")
32
+ elsif trace_point.event == :raise
33
+ io.puts PP.pp(
34
+ {
35
+ event: :raise,
36
+ raised_exception: trace_point.raised_exception,
37
+ path: "#{trace_point.path}:#{trace_point.lineno}",
38
+ method_id: trace_point.method_id
39
+ },
40
+ +""
41
+ )
42
+ end
43
+ end
44
+
45
+ rspec_config.around do |example|
46
+ trace.enable
47
+ example.run
48
+ ensure
49
+ trace.disable
50
+ end
51
+ end
52
+
53
+ def loggable_sql?(sql)
54
+ !sql.to_s.match?(SKIPPED_SQL_PREFIX)
55
+ end
56
+
57
+ def sql_with_interpolated_binds(sql, binds)
58
+ output = sql.to_s.dup
59
+ Array(binds).each_with_index do |bind, index|
60
+ output = output.gsub("$#{index + 1}", "'#{bind}'")
61
+ end
62
+ output
63
+ end
64
+
65
+ def trace_root
66
+ return Rails.root.to_s if defined?(Rails)
67
+
68
+ Dir.pwd
69
+ end
70
+ private_class_method :trace_root
71
+ end
72
+ end
73
+ end
@@ -0,0 +1,34 @@
1
+ module Polyrun
2
+ module RSpec
3
+ # Formatter tweaks for progress bars (e.g. Fuubar) under +POLYRUN_SHARD_*+ workers.
4
+ module ShardedFormatterCompat
5
+ module_function
6
+
7
+ def install!(rspec_config: fetch_rspec_configuration!)
8
+ return unless sharded_worker?
9
+
10
+ rspec_config.silence_filter_announcements = true
11
+ noop_fuubar_seed! if defined?(Fuubar)
12
+ end
13
+
14
+ def sharded_worker?
15
+ !ENV["POLYRUN_SHARD_TOTAL"].to_s.strip.empty?
16
+ end
17
+
18
+ def noop_fuubar_seed!
19
+ return unless defined?(Fuubar)
20
+
21
+ Fuubar.class_eval do
22
+ def seed(_notification)
23
+ end
24
+ end
25
+ end
26
+
27
+ def fetch_rspec_configuration!
28
+ require "rspec/core"
29
+ ::RSpec.configuration
30
+ end
31
+ private_class_method :fetch_rspec_configuration!
32
+ end
33
+ end
34
+ end
data/lib/polyrun/rspec.rb CHANGED
@@ -72,5 +72,39 @@ module Polyrun
72
72
 
73
73
  Polyrun::WorkerPing.ensure_interval_ping_thread!
74
74
  end
75
+
76
+ def install_example_debug!(rspec_config: nil)
77
+ require_relative "rspec/example_debug"
78
+ Polyrun::RSpec::ExampleDebug.install!(rspec_config: rspec_config || fetch_rspec_configuration!)
79
+ end
80
+
81
+ def install_example_timeout!(rspec_config: nil, seconds: ENV.fetch("RSPEC_EXAMPLE_TIMEOUT_SEC", "30").to_f)
82
+ require_relative "rspec/example_debug"
83
+ Polyrun::RSpec::ExampleDebug.install_example_timeout!(
84
+ rspec_config || fetch_rspec_configuration!,
85
+ seconds: seconds
86
+ )
87
+ end
88
+
89
+ def install_example_rails_logging!(rspec_config: nil)
90
+ require_relative "rspec/example_debug"
91
+ Polyrun::RSpec::ExampleDebug.install_rails_logging!(rspec_config: rspec_config || fetch_rspec_configuration!)
92
+ end
93
+
94
+ def install_example_prosopite!(rspec_config: nil)
95
+ require_relative "rspec/example_debug"
96
+ Polyrun::RSpec::ExampleDebug.install_prosopite!(rspec_config: rspec_config || fetch_rspec_configuration!)
97
+ end
98
+
99
+ def install_sharded_formatter_compat!(rspec_config: nil)
100
+ require_relative "rspec/sharded_formatter_compat"
101
+ Polyrun::RSpec::ShardedFormatterCompat.install!(rspec_config: rspec_config || fetch_rspec_configuration!)
102
+ end
103
+
104
+ def fetch_rspec_configuration!
105
+ require "rspec/core"
106
+ ::RSpec.configuration
107
+ end
108
+ private_class_method :fetch_rspec_configuration!
75
109
  end
76
110
  end
@@ -1,3 +1,3 @@
1
1
  module Polyrun
2
- VERSION = "2.1.2"
2
+ VERSION = "2.1.3"
3
3
  end
@@ -0,0 +1,159 @@
1
+ require "fileutils"
2
+ require_relative "worker_output_forwarders"
3
+
4
+ module Polyrun
5
+ # Routes parallel worker stdout/stderr through per-shard log files with optional prefixed TTY echo.
6
+ #
7
+ # Opt-in: +POLYRUN_WORKER_OUTPUT_ROUTING=1+ (or set +POLYRUN_WORKER_LOG_DIR+).
8
+ # +POLYRUN_WORKER_OUTPUT_ROUTING=0+ keeps inherited stdio (default polyrun behavior).
9
+ # +POLYRUN_WORKER_OUTPUT_PREFIX=0+ writes logs only (no live prefixed echo).
10
+ module WorkerOutput
11
+ module_function
12
+
13
+ def routing_enabled?
14
+ return false if disabled_by_env?
15
+
16
+ truthy?(ENV["POLYRUN_WORKER_OUTPUT_ROUTING"]) || !log_directory.nil?
17
+ end
18
+
19
+ def log_directory
20
+ value = ENV["POLYRUN_WORKER_LOG_DIR"]
21
+ return nil if value.nil?
22
+
23
+ stripped = value.to_s.strip
24
+ stripped.empty? ? nil : stripped
25
+ end
26
+
27
+ def prefix_live?
28
+ return true if ENV["POLYRUN_WORKER_OUTPUT_PREFIX"].nil?
29
+
30
+ truthy?(ENV["POLYRUN_WORKER_OUTPUT_PREFIX"])
31
+ end
32
+
33
+ def log_path_for(shard)
34
+ File.expand_path("shard-#{shard}.log", log_directory || default_log_directory)
35
+ end
36
+
37
+ def worker_log_directory_label
38
+ log_directory || default_log_directory
39
+ end
40
+
41
+ def prepare_log_dir!
42
+ directory = log_directory || default_log_directory
43
+ FileUtils.mkdir_p(directory)
44
+ Dir.glob(File.join(directory, "shard-*.log")).each { |path| File.delete(path) }
45
+ end
46
+
47
+ def warn_shard_log(shard)
48
+ return unless routing_enabled?
49
+
50
+ Polyrun::Log.warn "polyrun run-shards: shard #{shard} worker log → #{log_path_for(shard)}"
51
+ end
52
+
53
+ def spawn_worker(child_env, cmd, paths, hook_cfg)
54
+ shard = child_env.fetch("POLYRUN_SHARD_INDEX", "?")
55
+ out_read, out_write = IO.pipe
56
+ err_read, err_write = IO.pipe
57
+
58
+ pid =
59
+ if hook_cfg.worker_hooks? && !Polyrun::Hooks.disabled?
60
+ Process.spawn(
61
+ child_env,
62
+ "sh",
63
+ "-c",
64
+ hook_cfg.build_worker_shell_script(cmd, paths),
65
+ out: out_write,
66
+ err: err_write
67
+ )
68
+ else
69
+ Process.spawn(child_env, *cmd, *paths, out: out_write, err: err_write)
70
+ end
71
+
72
+ out_write.close
73
+ err_write.close
74
+
75
+ start_forwarders(
76
+ pid: pid,
77
+ shard: shard,
78
+ stdout_io: out_read,
79
+ stderr_io: err_read,
80
+ log_path: log_path_for(shard)
81
+ )
82
+ pid
83
+ end
84
+
85
+ def start_forwarders(pid:, shard:, stdout_io:, stderr_io:, log_path:)
86
+ forwarder = WorkerForwarder.new(
87
+ shard: shard,
88
+ pid: pid,
89
+ log_path: log_path,
90
+ prefix_live: prefix_live?
91
+ )
92
+ threads = [
93
+ forwarder_thread(stdout_io, forwarder, :stdout),
94
+ forwarder_thread(stderr_io, forwarder, :stderr)
95
+ ]
96
+ registry[pid] = {threads: threads, ios: [stdout_io, stderr_io], forwarder: forwarder}
97
+ end
98
+
99
+ def finish_worker(pid)
100
+ entry = registry.delete(pid)
101
+ return unless entry
102
+
103
+ drain_forwarders(entry)
104
+ end
105
+
106
+ def shutdown_all!
107
+ registry.each_value { |entry| drain_forwarders(entry) }
108
+ registry.clear
109
+ end
110
+
111
+ def default_log_directory
112
+ "tmp/polyrun/workers"
113
+ end
114
+
115
+ def disabled_by_env?
116
+ %w[0 false no off].include?(ENV["POLYRUN_WORKER_OUTPUT_ROUTING"].to_s.strip.downcase)
117
+ end
118
+
119
+ def truthy?(value)
120
+ return false if value.nil?
121
+
122
+ %w[1 true yes on].include?(value.to_s.strip.downcase)
123
+ end
124
+ private_class_method :truthy?
125
+
126
+ # rubocop:disable ThreadSafety/ClassInstanceVariable -- per-parent-process worker forwarder registry
127
+ def registry
128
+ @registry ||= {}
129
+ end
130
+ private_class_method :registry
131
+ # rubocop:enable ThreadSafety/ClassInstanceVariable
132
+
133
+ def forwarder_thread(io, forwarder, stream)
134
+ # rubocop:disable ThreadSafety/NewThread -- one reader thread per worker pipe
135
+ Thread.new do
136
+ loop do
137
+ forwarder.consume(stream, io.readpartial(4096))
138
+ end
139
+ rescue IOError, Errno::EPIPE
140
+ # worker closed the pipe
141
+ ensure
142
+ io.close unless io.closed?
143
+ end
144
+ # rubocop:enable ThreadSafety/NewThread
145
+ end
146
+ private_class_method :forwarder_thread
147
+
148
+ def drain_forwarders(entry)
149
+ entry[:ios].each do |io|
150
+ io.close unless io.closed?
151
+ rescue IOError
152
+ nil
153
+ end
154
+ entry[:threads].each(&:join)
155
+ entry[:forwarder].close
156
+ end
157
+ private_class_method :drain_forwarders
158
+ end
159
+ end
@@ -0,0 +1,90 @@
1
+ require "time"
2
+
3
+ module Polyrun
4
+ module WorkerOutput
5
+ class LineForwarder
6
+ def initialize(shard:, pid:, log_io:, prefix_live:, tty_io:)
7
+ @shard = shard
8
+ @pid = pid
9
+ @log_io = log_io
10
+ @prefix_live = prefix_live
11
+ @tty_io = tty_io
12
+ @buffer = +""
13
+ @mutex = Mutex.new
14
+ end
15
+
16
+ def consume(chunk)
17
+ return if chunk.nil? || chunk.empty?
18
+
19
+ @mutex.synchronize do
20
+ if progress_chunk?(chunk)
21
+ write_tty(chunk)
22
+ @log_io&.write(chunk)
23
+ return
24
+ end
25
+
26
+ @buffer << chunk
27
+ while (newline_index = @buffer.index("\n"))
28
+ emit_line(@buffer.slice!(0, newline_index + 1))
29
+ end
30
+ end
31
+ end
32
+
33
+ def flush
34
+ @mutex.synchronize do
35
+ next if @buffer.empty?
36
+
37
+ emit_line(@buffer)
38
+ @buffer.clear
39
+ end
40
+ end
41
+
42
+ private
43
+
44
+ def progress_chunk?(chunk)
45
+ chunk.include?("\r") && !chunk.include?("\n")
46
+ end
47
+
48
+ def emit_line(line)
49
+ prefixed = "#{prefix}#{line}"
50
+ @log_io&.write(prefixed)
51
+ write_tty(prefixed) if @prefix_live
52
+ end
53
+
54
+ def prefix
55
+ "[#{Time.now.strftime("%Y-%m-%dT%H:%M:%S")} shard=#{@shard} pid=#{@pid}] "
56
+ end
57
+
58
+ def write_tty(text)
59
+ @tty_io.write(text)
60
+ @tty_io.flush
61
+ end
62
+ end
63
+
64
+ class WorkerForwarder
65
+ def initialize(shard:, pid:, log_path:, prefix_live:)
66
+ @shard = shard
67
+ @pid = pid
68
+ @log_io = File.open(log_path, "wb")
69
+ @stdout = LineForwarder.new(shard: shard, pid: pid, log_io: @log_io, prefix_live: prefix_live, tty_io: Polyrun::Log.stdout)
70
+ @stderr = LineForwarder.new(shard: shard, pid: pid, log_io: @log_io, prefix_live: prefix_live, tty_io: Polyrun::Log.stderr)
71
+ @write_mutex = Mutex.new
72
+ end
73
+
74
+ def consume(stream, chunk)
75
+ @write_mutex.synchronize do
76
+ case stream
77
+ when :stdout then @stdout.consume(chunk)
78
+ when :stderr then @stderr.consume(chunk)
79
+ end
80
+ end
81
+ end
82
+
83
+ def close
84
+ @stdout.flush
85
+ @stderr.flush
86
+ @log_io.close
87
+ end
88
+ end
89
+ end
90
+ end
@@ -7,5 +7,15 @@ module Polyrun
7
7
  def self.install_failure_fragments!: (?only_if: untyped?) -> void
8
8
 
9
9
  def self.install_worker_ping!: () -> void
10
+
11
+ def self.install_example_debug!: (?rspec_config: untyped?) -> void
12
+
13
+ def self.install_example_timeout!: (?rspec_config: untyped?, ?seconds: Float) -> void
14
+
15
+ def self.install_example_rails_logging!: (?rspec_config: untyped?) -> void
16
+
17
+ def self.install_example_prosopite!: (?rspec_config: untyped?) -> void
18
+
19
+ def self.install_sharded_formatter_compat!: (?rspec_config: untyped?) -> void
10
20
  end
11
21
  end
@@ -0,0 +1,11 @@
1
+ module Polyrun
2
+ module WorkerOutput
3
+ def self.routing_enabled?: () -> bool
4
+ def self.log_path_for: (Integer | String shard) -> String
5
+ def self.prepare_log_dir!: () -> void
6
+ def self.warn_shard_log: (Integer | String shard) -> void
7
+ def self.spawn_worker: (Hash[Symbol, untyped] child_env, Array[String] cmd, Array[String] paths, untyped hook_cfg) -> Integer
8
+ def self.finish_worker: (Integer pid) -> void
9
+ def self.shutdown_all!: () -> void
10
+ end
11
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: polyrun
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.1.2
4
+ version: 2.1.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrei Makarov
@@ -266,6 +266,7 @@ files:
266
266
  - lib/polyrun/queue/duration.rb
267
267
  - lib/polyrun/queue/file_store.rb
268
268
  - lib/polyrun/queue/file_store_pending.rb
269
+ - lib/polyrun/queue/worker_loop.rb
269
270
  - lib/polyrun/quick.rb
270
271
  - lib/polyrun/quick/assertions.rb
271
272
  - lib/polyrun/quick/errors.rb
@@ -281,6 +282,9 @@ files:
281
282
  - lib/polyrun/reporting/rspec_failure_fragment_formatter.rb
282
283
  - lib/polyrun/reporting/rspec_junit.rb
283
284
  - lib/polyrun/rspec.rb
285
+ - lib/polyrun/rspec/example_debug.rb
286
+ - lib/polyrun/rspec/example_debug_instrumentation.rb
287
+ - lib/polyrun/rspec/sharded_formatter_compat.rb
284
288
  - lib/polyrun/spec_quality.rb
285
289
  - lib/polyrun/spec_quality/config.rb
286
290
  - lib/polyrun/spec_quality/fragment.rb
@@ -303,6 +307,8 @@ files:
303
307
  - lib/polyrun/timing/summary.rb
304
308
  - lib/polyrun/timing/variance_report.rb
305
309
  - lib/polyrun/version.rb
310
+ - lib/polyrun/worker_output.rb
311
+ - lib/polyrun/worker_output_forwarders.rb
306
312
  - lib/polyrun/worker_ping.rb
307
313
  - polyrun.gemspec
308
314
  - sig/polyrun.rbs
@@ -313,6 +319,7 @@ files:
313
319
  - sig/polyrun/minitest.rbs
314
320
  - sig/polyrun/quick.rbs
315
321
  - sig/polyrun/rspec.rbs
322
+ - sig/polyrun/worker_output.rbs
316
323
  - sig/polyrun/worker_ping.rbs
317
324
  homepage: https://github.com/amkisko/polyrun.rb
318
325
  licenses: