polyrun 1.3.0 → 1.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e3162ed760c231d4fa78ff396f55f708af13bda62808ba340f3c1494cf2dd97e
-  data.tar.gz: f401bd075462bafa14905c08a996fa46370025be9872f6a4d7aefbfdde251d2d
+  metadata.gz: 7666af9186562083f29dc56e6c867e48b877acdff6ad28ff8c351e8d3c308582
+  data.tar.gz: 503f5435deb22112044f7841a82728e6782a770eb656859419e8412d623dcff0
 SHA512:
-  metadata.gz: 3d0bf0e8ac88d3d0a007f53ce340c324bcd65c6397b4b67905d428d104ebff3078bc41220873b0b7d741e6cc65be06d5d103f8b8abd64e18ce0dc2d8f3e55bc1
-  data.tar.gz: 42906169eeeae14b3531d870359ce8fe9bd59261770553a6e7e6872e48e47b7b1013709eedea5c176a94ccee5c1e498551e8d6285ffc68b37c51d05169b184b1
+  metadata.gz: d0d5d248f1e072c446049bafff111db6e29d784a4a0992528214a6e29cca7b156b67144e6b9c22fc71fa9143f110ac443cda2953219aa816c5562d3247b5e02b
+  data.tar.gz: d0be776ce5d4a7a5acacfe237a4d07a47c078562aee6a054fb5b63a31a78e199bc2b5ee276b0a8e77a0f0cacd3e4f03cb72d8fd5037baf782d0efb8a1cdf1b04

data/CHANGELOG.md

@@ -1,5 +1,21 @@
 # CHANGELOG
 
+## 1.4.1 (2026-04-16)
+
+- Add `polyrun merge-failures` and `run-shards --merge-failures` / `--merge-failures-output` / `--merge-failures-format`; merge per-worker JSONL under `tmp/polyrun_failures/polyrun-failure-fragment-*.jsonl` (or RSpec JSON via `-i`). Run merge after all workers exit, including when a shard failed (`--merge-coverage` still runs only after all shards succeed).
+- Add `Polyrun::Reporting::FailureMerge`, `Polyrun::RSpec.install_failure_fragments!`, and `Polyrun::Reporting::RspecFailureFragmentFormatter`; parent sets `POLYRUN_FAILURE_FRAGMENTS=1` on workers when merge-failures is enabled.
+- Add optional `reporting:` in `polyrun.yml` and `Polyrun::Config#reporting` for merge-failures toggles and paths; honor `POLYRUN_MERGE_FAILURES`, `POLYRUN_MERGED_FAILURES_OUT`, `POLYRUN_MERGED_FAILURES_FORMAT`; set `POLYRUN_MERGED_FAILURES_PATH` for `after_suite` when merge wrote a file.
+- On bad JSONL or non-RSpec JSON, raise `Polyrun::Error` with path and line where applicable; `merge-failures` exits 1 without a full stack trace; a failed merge after successful workers forces `run-shards` exit 1.
+- Fix `run_shards_plan_ready_log` to take `cfg` so debug logging for merge-failures does not raise `NameError`.
+
+## 1.4.0 (2026-04-16)
+
+- Add `hooks:` in `polyrun.yml` — shell commands for `before_suite` / `after_suite`, `before_shard` / `after_shard`, `before_worker` / `after_worker` (RSpec-style YAML keys `before(:suite)`, `before(:all)`, `before(:each)` accepted). Wire hooks into `run-shards`, `parallel-rspec`, and `ci-shard-*`.
+- Add `hooks.ruby` / `hooks.ruby_file` and `Polyrun::Hooks::Dsl` (`before(:suite)` … `after(:each)` blocks); worker Ruby hooks run in the child via `ruby -e` + `POLYRUN_HOOKS_RUBY_FILE`.
+- Add `polyrun hook run <phase>` (`--shard` / `--total` optional). Set `POLYRUN_HOOKS_DISABLE=1` to skip hooks during orchestration only; `hook run` still executes.
+- On `ci-shard-run` / `ci-shard-rspec`, skip automatic `before_suite` / `after_suite` when `POLYRUN_SHARD_TOTAL` > 1 (matrix); run suite hooks once via `polyrun hook run` or set `POLYRUN_HOOKS_SUITE_PER_MATRIX_JOB=1` to run them on every matrix job.
+- Document hook phases, matrix vs suite, and `after_shard` ordering in `README.md`; list `Polyrun::Hooks` and `Polyrun::Hooks::Dsl` in the library section.
+
 ## 1.3.0 (2026-04-15)
 
 - Add safe parsing for `ci-shard-run` / `ci-shard-rspec` `--shard-processes` and `--workers` (warn + exit 2 on missing or non-integer values).

data/README.md

@@ -25,6 +25,61 @@ Capybara and Playwright stay in your application; Polyrun does not replace brows
 4. Run workers with `bin/polyrun run-shards --workers N -- bundle exec rspec`: N separate OS processes, each running RSpec with its own file list from `partition.paths_file`, or `spec/spec_paths.txt`, or else `spec/**/*_spec.rb`. Stderr shows where paths came from; after a successful multi-worker run it reminds you to run merge-coverage unless you use `parallel-rspec` or `run-shards --merge-coverage`.
 5. Merge artifacts with `bin/polyrun merge-coverage` on `coverage/polyrun-fragment-*.json` (one fragment per `POLYRUN_SHARD_INDEX` when coverage is on), or use `bin/polyrun parallel-rspec` or `run-shards --merge-coverage` so Polyrun runs merge for you. Optional: `merge-timing`, `report-timing`, `report-junit`.
 
+### Hooks (`hooks:` in `polyrun.yml`)
+
+Optional **shell** commands and/or a **Ruby DSL** file for instrumentation (telemetry, Slack, logging, manual debugging). Names mirror RSpec’s API (`before(:suite)`, `before(:all)`, `before(:each)`), but **Polyrun hooks are about process orchestration**, not RSpec example groups. Below, **suite / shard / worker** mean Polyrun’s model unless stated otherwise.
+
+#### What “suite”, “shard”, and “worker” mean
+
+| Term | Process | Meaning |
+|------|---------|---------|
+| **Suite** | **Parent** only | One **orchestration run** on a single machine: a single `polyrun run-shards` / `parallel-rspec` / `ci-shard-run` (with **one** global shard, see below). `before_suite` runs once before any worker is started; `after_suite` runs once after all workers have exited and (when used) merge-coverage has finished. This is **not** the same as “the whole RSpec suite in one process”—with `--workers N`, RSpec runs in **N separate processes**, each with its own examples. |
+| **Shard** | **Parent** only | One **partition** of the path list for this run, identified by `POLYRUN_SHARD_INDEX` / `POLYRUN_SHARD_TOTAL` **for that parallel layout** (0 … N−1 for `run-shards` with N workers). `before_shard` runs in the parent **immediately before** `Process.spawn` for that index; `after_shard` runs **after** that child has exited. The parent **waits workers in shard index order** (0, then 1, …), so `after_shard` runs in that order—not in “who finished first” order if workers overlap in time. Empty partitions are skipped (no spawn, no hooks). |
+| **Worker** | **Child** OS process | The process that runs your command after `--` (e.g. `bundle exec rspec …`). `before_worker` / `after_worker` run **in that child**, directly before and after the test command. **Individual examples run only after `before_worker` completes** (inside the same process as RSpec/Minitest). |
+
+**CI matrix** (`POLYRUN_SHARD_TOTAL` > 1, one job per index): each job is a **global shard**, not a full “suite” in the pipeline sense. **`ci-shard-run` / `ci-shard-rspec` with one process per job do not run `before_suite` / `after_suite` automatically**—otherwise they would run once per matrix cell. Put pipeline-wide setup/teardown in a **separate CI step** (e.g. `bin/polyrun hook run before_suite` and `hook run after_suite` once), or set `POLYRUN_HOOKS_SUITE_PER_MATRIX_JOB=1` to restore the old behaviour (suite hooks on every matrix job). **`before_shard` / `after_shard` / worker hooks still run** per job, with `POLYRUN_SHARD_INDEX` / `POLYRUN_SHARD_TOTAL` set from the matrix. A **single** non-matrix `ci-shard-run` (`POLYRUN_SHARD_TOTAL` is 1) still runs suite hooks like `run-shards --workers 1`. Fan-out on one host (`--shard-processes` > 1) still runs **`before_suite` / `after_suite` once** for that job, around the local workers.
+
+#### Lifecycle (typical `run-shards` with multiple workers)
+
+```text
+[Parent]  before_suite
+[Parent]  for each shard index i that has paths:
+            before_shard(i) → spawn worker i
+[Child i] before_worker →  test runner starts → examples run → runner exits
+[Parent]  after_shard(i)   (parent waits children in shard index order 0…N−1, not global finish order)
+[Parent]  merge-coverage (if requested)
+[Parent]  after_suite
+```
+
+`after_worker` runs in the child after the test command exits, before the parent’s `after_shard`.
+
+#### Order and priority within one phase
+
+1. **Ruby DSL, then shell (YAML)** — For the same phase (e.g. `before_suite`), all **Ruby** blocks from `hooks.ruby` run first, then every **shell** command from YAML for that phase.
+2. **Multiple Ruby blocks** — In one DSL file, registrations run in **source order** (e.g. two `before(:suite)` blocks run top to bottom).
+3. **Multiple shell commands** — Use a **YAML list**; entries run in list order:
+   ```yaml
+   before_suite:
+     - echo first
+     - echo second
+   ```
+4. **Duplicate YAML keys** — Do **not** repeat the same key (e.g. two `before_suite:` lines). Parsers may keep only one value; behaviour is undefined. Prefer a **list** under a single key.
+5. **Failure** — If any step in a phase fails (non-zero exit from shell; uncaught error in Ruby), orchestration stops or marks failure per existing `run-shards` rules; `after_worker` shell steps use `|| true` so a failing teardown does not mask the test exit code (Ruby `after_worker` is wrapped similarly in the worker script).
+
+Environment includes `POLYRUN_HOOK_PHASE`, `POLYRUN_HOOK=1`, `POLYRUN_HOOK_ORCHESTRATOR` (`1` in parent, `0` in workers), `POLYRUN_SHARD_*`, and `POLYRUN_SUITE_EXIT_STATUS` on `after_suite`. Worker children get `POLYRUN_HOOKS_RUBY_FILE` when using the Ruby DSL. Set `POLYRUN_HOOKS_DISABLE=1` to skip hooks during `run-shards` / `parallel-rspec` / `ci-shard-*` (orchestration only); `polyrun hook run` still executes hooks. **`POLYRUN_HOOKS_SUITE_PER_MATRIX_JOB=1`** — when set, run `before_suite` / `after_suite` on every CI matrix job (not recommended for expensive global setup).
+
+**YAML keys (shell) and RSpec-style names in YAML:**
+
+| YAML key | DSL in `hooks.ruby` | When |
+|----------|----------------------|------|
+| `before_suite` / `after_suite` | `before(:suite)` / `after(:suite)` | Parent: once per orchestration on one host; **skipped** for `ci-shard-run` when `POLYRUN_SHARD_TOTAL` > 1 unless `POLYRUN_HOOKS_SUITE_PER_MATRIX_JOB=1` (see matrix paragraph above) |
+| `before_shard` / `after_shard` | `before(:all)` / `after(:all)` | Parent: per shard index (spawn / after exit) |
+| `before_worker` / `after_worker` | `before(:each)` / `after(:each)` | Child: around the test command |
+
+**Ruby DSL (`hooks.ruby` or `hooks.ruby_file`):** path to a `.rb` file (relative to the project root). Blocks receive a `Hash` with **string keys** (same env as shell hooks). Worker-phase Ruby hooks run in the child via `ruby -e 'require "polyrun"; …'`.
+
+Run one phase by hand: `bin/polyrun hook run before_suite` (optional `--shard N --total M`). YAML may use quoted keys such as `"before(:suite)"` instead of `before_suite`.
+
 Quick CLI samples:
 
 If the current directory already has `polyrun.yml` or `config/polyrun.yml`, you can omit `-c` (same as `Config.load` default discovery). Pass `-c PATH` or set `POLYRUN_CONFIG` when the file lives elsewhere or uses another name.
@@ -41,6 +96,7 @@ bin/polyrun env --shard 0 --total 4   # print DATABASE_URL exports from polyrun.
 bin/polyrun init --list
 bin/polyrun init --profile gem -o polyrun.yml   # starter YAML; see docs/SETUP_PROFILE.md
 bin/polyrun quick   # Polyrun::Quick examples under spec/polyrun_quick/ or test/polyrun_quick/
+bin/polyrun hook run before_suite   # run hooks.before_suite from polyrun.yml (manual / CI)
 ```
 
 ### Matrix shards and timing
@@ -87,6 +143,8 @@ That single require loads the CLI and core library **without** loading RSpec or
 | `Polyrun::Prepare::Assets` | Digest trees, marker file, `assets:precompile`. |
 | `Polyrun::Database::Shard` | Shard env map, `%{shard}` DB names, URL path suffix for `postgres://`, `mysql2://`, `mongodb://`, etc. |
 | `Polyrun::Database::UrlBuilder` | URLs from `polyrun.yml` `databases:` — nested blocks or `adapter:` for common Rails stacks (`postgresql`, `mysql`/`mysql2`, `trilogy`, `sqlserver`/`mssql`, `sqlite3`/`sqlite`, `mongodb`/`mongo`). |
+| `Polyrun::Hooks` | Load from `Config#hooks`; `run_phase` / `run_phase_if_enabled`; `build_worker_shell_script` wraps the worker command. |
+| `Polyrun::Hooks::Dsl` | Ruby hook file (`hooks.ruby`); `before(:suite)` / `after(:each)` etc. in `config/polyrun_hooks.rb` (see README). |
 
 ## Development
 

data/lib/polyrun/cli/ci_shard_hooks.rb

@@ -0,0 +1,121 @@
+module Polyrun
+  class CLI
+    # Suite / shard / worker shell hooks for +ci-shard-run+ / +ci-shard-rspec+.
+    module CiShardHooks
+      private
+
+      # rubocop:disable Metrics/AbcSize -- suite hooks + spawn/wait + failure paths
+      def ci_shard_run_fanout!(ctx)
+        hook_cfg = Polyrun::Hooks.from_config(ctx[:cfg])
+        suite_started = false
+        exit_code = 1
+
+        begin
+          env_suite = ENV.to_h.merge(
+            "POLYRUN_HOOK_ORCHESTRATOR" => "1",
+            "POLYRUN_SHARD_TOTAL" => ctx[:workers].to_s
+          )
+          code = hook_cfg.run_phase_if_enabled(:before_suite, env_suite)
+          return code if code != 0
+
+          suite_started = true
+
+          pids, spawn_err = run_shards_spawn_workers(ctx, hook_cfg)
+          if spawn_err
+            exit_code = spawn_err
+            return spawn_err
+          end
+          return 1 if pids.empty?
+
+          run_shards_warn_interleaved(ctx[:parallel], pids.size)
+          shard_results, wait_hook_err = run_shards_wait_all_children(pids, hook_cfg, ctx)
+          failed = shard_results.reject { |r| r[:success] }.map { |r| r[:shard] }
+
+          if failed.any?
+            Polyrun::Log.warn "polyrun ci-shard: finished #{pids.size} worker(s) (some failed)"
+            run_shards_log_failed_reruns(failed, shard_results, ctx[:plan], ctx[:parallel], ctx[:workers], ctx[:cmd])
+            exit_code = 1
+            exit_code = 1 if wait_hook_err != 0
+            return exit_code
+          end
+
+          exit_code = (wait_hook_err == 0) ? 0 : 1
+          Polyrun::Log.warn "polyrun ci-shard: finished #{pids.size} worker(s) (exit 0)" if exit_code == 0
+          exit_code
+        ensure
+          if suite_started
+            env_after = ENV.to_h.merge(
+              "POLYRUN_HOOK_ORCHESTRATOR" => "1",
+              "POLYRUN_SHARD_TOTAL" => ctx[:workers].to_s,
+              "POLYRUN_SUITE_EXIT_STATUS" => exit_code.to_s
+            )
+            hook_cfg.run_phase_if_enabled(:after_suite, env_after)
+          end
+        end
+      end
+      # rubocop:enable Metrics/AbcSize
+
+      # One matrix shard, one OS process: same hook phases as +run-shards+ with +--workers 1+ (no +exec+ when hooks exist).
+      # rubocop:disable Metrics/AbcSize -- suite / shard / worker lifecycle
+      def ci_shard_run_single!(cmd, paths, cfg, pc, _config_path)
+        hook_cfg = Polyrun::Hooks.from_config(cfg)
+        if hook_cfg.empty? || Polyrun::Hooks.disabled?
+          exec(*cmd, *paths)
+        end
+
+        si = Polyrun::Config::Resolver.resolve_shard_index(pc)
+        st = Polyrun::Config::Resolver.resolve_shard_total(pc)
+        suite_started = false
+        exit_code = 1
+        # Distributed CI matrix (N > 1 global shards): each job is one shard; suite hooks are pipeline-wide.
+        # Run them once via +polyrun hook run before_suite+ / +after_suite+ (e.g. dedicated job), or set
+        # +POLYRUN_HOOKS_SUITE_PER_MATRIX_JOB=1+ to run suite hooks on every matrix job.
+        matrix_shards = st > 1
+        run_suite_hooks = !matrix_shards || Polyrun::Hooks.suite_per_matrix_job?
+
+        begin
+          env_orch = ENV.to_h.merge(
+            "POLYRUN_HOOK_ORCHESTRATOR" => "1",
+            "POLYRUN_SHARD_INDEX" => si.to_s,
+            "POLYRUN_SHARD_TOTAL" => st.to_s
+          )
+          if run_suite_hooks
+            code = hook_cfg.run_phase_if_enabled(:before_suite, env_orch)
+            return code if code != 0
+
+            suite_started = true
+          end
+
+          code = hook_cfg.run_phase_if_enabled(:before_shard, env_orch)
+          return code if code != 0
+
+          mx, mt = ci_shard_matrix_context(pc, 1)
+          child_env = shard_child_env(cfg: cfg, workers: 1, shard: 0, matrix_index: mx, matrix_total: mt)
+          child_env = child_env.merge("POLYRUN_HOOK_ORCHESTRATOR" => "0")
+          child_env = hook_cfg.merge_worker_ruby_env(child_env)
+
+          if hook_cfg.worker_hooks? && !Polyrun::Hooks.disabled?
+            system(child_env, "sh", "-c", hook_cfg.build_worker_shell_script(cmd, paths))
+          else
+            system(child_env, *cmd, *paths)
+          end
+          exit_code = $?.exitstatus
+
+          rc = hook_cfg.run_phase_if_enabled(:after_shard, env_orch.merge(
+            "POLYRUN_WORKER_EXIT_STATUS" => exit_code.to_s
+          ))
+          exit_code = rc if rc != 0
+
+          exit_code
+        ensure
+          if suite_started
+            hook_cfg.run_phase_if_enabled(:after_suite, env_orch.merge(
+              "POLYRUN_SUITE_EXIT_STATUS" => exit_code.to_s
+            ))
+          end
+        end
+      end
+      # rubocop:enable Metrics/AbcSize
+    end
+  end
+end

data/lib/polyrun/cli/ci_shard_run_command.rb

@@ -1,5 +1,7 @@
 require "shellwords"
 
+require_relative "ci_shard_hooks"
+
 module Polyrun
   class CLI
     # One CI matrix job = one global shard (POLYRUN_SHARD_INDEX / POLYRUN_SHARD_TOTAL), not +run-shards+
@@ -14,6 +16,8 @@ module Polyrun
     # After +--+, prefer **multiple argv tokens** (+bundle+, +exec+, +rspec+, …). A single token that
     # contains spaces is split with +Shellwords+ (not a full shell); exotic quoting differs from +sh -c+.
     module CiShardRunCommand
+      include CiShardHooks
+
       private
 
       # @return [Array(Array<String>, Integer)] [paths, 0] on success, or [nil, exit_code] on failure
@@ -47,24 +51,6 @@ module Polyrun
         [resolve_shard_index(pc), n]
       end
 
-      def ci_shard_run_fanout!(ctx)
-        pids = run_shards_spawn_workers(ctx)
-        return 1 if pids.empty?
-
-        run_shards_warn_interleaved(ctx[:parallel], pids.size)
-        shard_results = run_shards_wait_all_children(pids)
-        failed = shard_results.reject { |r| r[:success] }.map { |r| r[:shard] }
-
-        if failed.any?
-          Polyrun::Log.warn "polyrun ci-shard: finished #{pids.size} worker(s) (some failed)"
-          run_shards_log_failed_reruns(failed, shard_results, ctx[:plan], ctx[:parallel], ctx[:workers], ctx[:cmd])
-          return 1
-        end
-
-        Polyrun::Log.warn "polyrun ci-shard: finished #{pids.size} worker(s) (exit 0)"
-        0
-      end
-
       def ci_shard_fanout_context(cfg:, pc:, paths:, shard_processes:, cmd:, config_path:)
         plan = ci_shard_local_plan!(paths, shard_processes)
         mx, mt = ci_shard_matrix_context(pc, shard_processes)
@@ -113,8 +99,7 @@ module Polyrun
         return code if code != 0
 
         if shard_processes <= 1
-          exec(*cmd, *paths)
-          return 0
+          return ci_shard_run_single!(cmd, paths, cfg, pc, config_path)
         end
 
         ctx = ci_shard_fanout_context(
@@ -145,8 +130,7 @@ module Polyrun
         cmd = ["bundle", "exec", "rspec", *rspec_argv]
 
         if shard_processes <= 1
-          exec(*cmd, *paths)
-          return 0
+          return ci_shard_run_single!(cmd, paths, cfg, pc, config_path)
         end
 
         ctx = ci_shard_fanout_context(

data/lib/polyrun/cli/failure_commands.rb

@@ -0,0 +1,92 @@
+require "json"
+require "fileutils"
+require "optparse"
+
+require_relative "../reporting/failure_merge"
+
+module Polyrun
+  class CLI
+    module FailureCommands
+      private
+
+      def cmd_merge_failures(argv, _config_path)
+        inputs, output, format = merge_failures_parse_argv(argv)
+        if inputs.empty?
+          Polyrun::Log.warn "merge-failures: need at least one existing -i FILE (after glob expansion)"
+          return 2
+        end
+        Polyrun::Log.warn "merge-failures: merging #{inputs.size} fragment(s)" if @verbose
+        n = merge_failures_merge_files_or_warn!(inputs, output: output, format: format)
+        return 1 if n.nil?
+
+        Polyrun::Log.puts File.expand_path(output)
+        Polyrun::Log.warn "merge-failures: #{n} failure row(s)" if @verbose
+        0
+      end
+
+      def merge_failures_merge_files_or_warn!(inputs, output:, format:)
+        Polyrun::Reporting::FailureMerge.merge_files!(inputs, output: output, format: format)
+      rescue Polyrun::Error => e
+        Polyrun::Log.warn e.message.to_s
+        nil
+      end
+
+      def merge_failures_parse_argv(argv)
+        inputs = []
+        output = File.join("tmp", "polyrun_failures", "merged.jsonl")
+        format = "jsonl"
+        OptionParser.new do |opts|
+          opts.banner = "usage: polyrun merge-failures -i FILE [-i FILE] [-o PATH] [--format jsonl|json]"
+          opts.on("-i", "--input FILE", "JSONL fragment or RSpec JSON (repeatable; globs ok)") do |f|
+            expand_merge_input_pattern(f).each { |x| inputs << x }
+          end
+          opts.on("-o", "--output PATH", String) { |v| output = v }
+          opts.on("--format VAL", "jsonl (default) or json") { |v| format = v }
+        end.parse!(argv)
+        inputs.uniq!
+        inputs.select! { |p| File.file?(p) }
+        [inputs, output, format]
+      end
+
+      # After run-shards workers exit: merge polyrun failure fragments when requested.
+      # Runs even when shards failed (unlike --merge-coverage).
+      # @return [String, nil] absolute path to merged file, or nil when skipped / nothing written
+      def merge_failures_after_shards(ctx)
+        return nil unless ctx[:merge_failures]
+
+        pattern = Polyrun::Reporting::FailureMerge.default_fragment_glob
+        files = Dir.glob(pattern).sort
+        if files.empty?
+          Polyrun::Log.warn "polyrun run-shards: --merge-failures: no #{Polyrun::Reporting::FailureMerge::FRAGMENT_GLOB} under fragment dir (enable Polyrun::RSpec.install_failure_fragments! in spec_helper?)"
+          return nil
+        end
+
+        fmt = merge_failures_resolved_format(ctx)
+        out = merge_failures_resolved_output_path(ctx, fmt)
+        Polyrun::Log.warn "polyrun run-shards: merging #{files.size} failure fragment(s) → #{out} (#{fmt})"
+        Polyrun::Debug.log_kv(merge_failures: "start", output: out, inputs: files, format: fmt)
+        n = Polyrun::Reporting::FailureMerge.merge_files!(files, output: out, format: fmt)
+        Polyrun::Debug.log_kv(merge_failures: "done", rows: n, output: File.expand_path(out))
+        File.expand_path(out)
+      end
+
+      def merge_failures_resolved_format(ctx)
+        f = ctx[:merge_failures_format].to_s.strip.downcase
+        return "jsonl" if f.empty?
+        return "jsonl" if f == "jsonl"
+        return "json" if f == "json"
+
+        Polyrun::Log.warn "polyrun run-shards: unknown merge_failures_format=#{ctx[:merge_failures_format].inspect}; using jsonl"
+        "jsonl"
+      end
+
+      def merge_failures_resolved_output_path(ctx, fmt)
+        raw = ctx[:merge_failures_output]
+        return File.expand_path(raw) if raw && !raw.to_s.strip.empty?
+
+        ext = (fmt == "json") ? "json" : "jsonl"
+        File.expand_path(File.join("tmp", "polyrun_failures", "merged.#{ext}"))
+      end
+    end
+  end
+end

data/lib/polyrun/cli/help.rb

@@ -21,6 +21,7 @@ module Polyrun
           Skip start auto-prepare / auto DB provision: POLYRUN_START_SKIP_PREPARE=1, POLYRUN_START_SKIP_DATABASES=1
           Skip writing paths_file from partition.paths_build: POLYRUN_SKIP_PATHS_BUILD=1
           Warn if merge-coverage wall time exceeds N seconds (default 10): POLYRUN_MERGE_SLOW_WARN_SECONDS (0 disables)
+          Failure fragments (run-shards --merge-failures): POLYRUN_MERGE_FAILURES=1; parent sets POLYRUN_FAILURE_FRAGMENTS=1 in workers; POLYRUN_FAILURE_FRAGMENT_DIR, POLYRUN_MERGED_FAILURES_OUT, POLYRUN_MERGED_FAILURES_FORMAT; after_suite sets POLYRUN_MERGED_FAILURES_PATH when merge ran
           Parallel RSpec workers: POLYRUN_WORKERS default 5, max 10 (run-shards / parallel-rspec / start); distinct from POLYRUN_SHARD_PROCESSES / ci-shard --shard-processes (local processes per CI matrix job)
           Partition timing granularity (default file): POLYRUN_TIMING_GRANULARITY=file|example (experimental per-example; see partition.timing_granularity)
 
@@ -29,7 +30,8 @@ module Polyrun
             plan                 emit partition manifest JSON
             prepare              run prepare recipe: default | assets (optional prepare.command overrides bin/rails assets:precompile) | shell (prepare.command required)
             merge-coverage       merge SimpleCov JSON fragments (json/lcov/cobertura/console)
-            run-shards           fan out N parallel OS processes (POLYRUN_SHARD_*; not Ruby threads); optional --merge-coverage
+            merge-failures       merge per-shard failure JSONL fragments or RSpec JSON files (jsonl/json)
+            run-shards           fan out N parallel OS processes (POLYRUN_SHARD_*; not Ruby threads); optional --merge-coverage / --merge-failures
             parallel-rspec       run-shards + merge-coverage (defaults to: bundle exec rspec after --)
             start                parallel-rspec; auto-runs prepare (shell/assets) and db:setup-* when polyrun.yml configures them; legacy script/build_spec_paths.rb if paths_build absent
             ci-shard-run         CI matrix: build-paths + plan for POLYRUN_SHARD_INDEX / POLYRUN_SHARD_TOTAL (or config), then run your command with that shard's paths after --; optional --shard-processes M or --workers M (POLYRUN_SHARD_PROCESSES; not POLYRUN_WORKERS) for N×M jobs × processes on this host
@@ -38,6 +40,7 @@ module Polyrun
             init                 write a starter polyrun.yml or POLYRUN.md from built-in templates (see docs/SETUP_PROFILE.md)
             queue                file-backed batch queue: init (optional --shard/--total etc. as plan, then claim/ack); M workers share one dir; no duplicate paths across claims
             quick                run Polyrun::Quick (describe/it, before/after, let, expect…to, assert_*; optional capybara!)
+            hook run <phase>     run one shell hook from polyrun.yml hooks: (e.g. before_suite); optional --shard/--total
             report-coverage      write all coverage formats from one JSON file
             report-junit         RSpec JSON or Polyrun testcase JSON → JUnit XML (CI)
             report-timing        print slow-file summary from merged timing JSON

data/lib/polyrun/cli/hooks_command.rb

@@ -0,0 +1,97 @@
+module Polyrun
+  class CLI
+    # +polyrun hook run <phase>+ — run one lifecycle phase from +polyrun.yml+ +hooks:+ (manual debugging / CI).
+    module HooksCommand
+      private
+
+      def cmd_hook(argv, config_path)
+        sub = argv.shift
+        case sub
+        when "run"
+          cmd_hook_run(argv, config_path)
+        when nil, "help", "-h", "--help"
+          print_hook_help
+          0
+        else
+          Polyrun::Log.warn "polyrun hook: unknown subcommand #{sub.inspect} (try: polyrun hook run <phase>)"
+          print_hook_help
+          2
+        end
+      end
+
+      def cmd_hook_run(argv, config_path)
+        phase = argv.shift
+        if phase.nil? || phase == "-h" || phase == "--help"
+          print_hook_help
+          return 2
+        end
+
+        shard, total = hook_run_parse_shard_flags!(argv)
+
+        unless argv.empty?
+          Polyrun::Log.warn "polyrun hook run: unexpected arguments: #{argv.inspect}"
+          return 2
+        end
+
+        phase_sym = Polyrun::Hooks.parse_phase(phase)
+        unless phase_sym && Polyrun::Hooks::PHASES.include?(phase_sym)
+          Polyrun::Log.warn "polyrun hook run: unknown phase #{phase.inspect} (expected: #{Polyrun::Hooks::PHASES.join(", ")})"
+          return 2
+        end
+
+        cfg = Polyrun::Config.load(path: config_path || ENV["POLYRUN_CONFIG"])
+        hook_cfg = Polyrun::Hooks.from_config(cfg)
+        env = hook_run_env(shard, total)
+
+        hook_cfg.run_phase(phase_sym, env)
+      rescue ArgumentError => e
+        Polyrun::Log.warn "polyrun hook run: #{e.message}"
+        2
+      end
+
+      def hook_run_parse_shard_flags!(argv)
+        shard = nil
+        total = nil
+        while (a = argv.first)
+          case a
+          when "--shard"
+            argv.shift
+            shard = Integer(argv.shift || (raise ArgumentError, "--shard needs a value"))
+          when "--total"
+            argv.shift
+            total = Integer(argv.shift || (raise ArgumentError, "--total needs a value"))
+          else
+            break
+          end
+        end
+        [shard, total]
+      end
+
+      def hook_run_env(shard, total)
+        env = ENV.to_h.merge(
+          "POLYRUN_HOOK_ORCHESTRATOR" => "1",
+          "POLYRUN_HOOK_CLI" => "1"
+        )
+        env["POLYRUN_SHARD_INDEX"] = shard.to_s unless shard.nil?
+        env["POLYRUN_SHARD_TOTAL"] = total.to_s unless total.nil?
+        env
+      end
+
+      def print_hook_help
+        Polyrun::Log.puts <<~HELP
+          usage: polyrun hook run <phase> [--shard N] [--total M]
+
+          Runs hook(s) from polyrun.yml: Ruby DSL (+hooks.ruby+) then shell strings for <phase> (same names as RSpec lifecycle:
+          before_suite / after_suite as before(:suite) / after(:suite); before_shard / after_shard as
+          before(:all) / after(:all); before_worker / after_worker as before(:each) / after(:each)).
+
+          Phases: #{Polyrun::Hooks::PHASES.join(", ")}
+
+          Optional --shard / --total set POLYRUN_SHARD_INDEX / POLYRUN_SHARD_TOTAL for the hook process.
+          POLYRUN_HOOKS_DISABLE=1 skips hooks during run-shards / ci-shard only; polyrun hook run still executes.
+          For CI matrix (POLYRUN_SHARD_TOTAL > 1), ci-shard-run skips before_suite / after_suite unless POLYRUN_HOOKS_SUITE_PER_MATRIX_JOB=1; run those phases here or in a dedicated CI job.
+        HELP
+      end
+    end
+  end
+end

data/lib/polyrun/cli/run_shards_command.rb

@@ -2,12 +2,14 @@ require "optparse"
 require "rbconfig"
 
 require_relative "start_bootstrap"
+require_relative "failure_commands"
 require_relative "run_shards_run"
 
 module Polyrun
   class CLI
     module RunShardsCommand
       include StartBootstrap
+      include FailureCommands
       include RunShardsRun
 
       private
@@ -114,10 +116,11 @@ module Polyrun
       # ENV for a worker process: POLYRUN_SHARD_* plus per-shard database URLs from polyrun.yml or DATABASE_URL.
       # When +matrix_total+ > 1 with multiple local workers, sets +POLYRUN_SHARD_MATRIX_INDEX+ / +POLYRUN_SHARD_MATRIX_TOTAL+
       # so {Coverage::Collector} can name fragments uniquely across CI matrix jobs (NxM sharding).
-      def shard_child_env(cfg:, workers:, shard:, matrix_index: nil, matrix_total: nil)
+      def shard_child_env(cfg:, workers:, shard:, matrix_index: nil, matrix_total: nil, failure_fragments: false)
         child_env = ENV.to_h.merge(
           Polyrun::Database::Shard.env_map(shard_index: shard, shard_total: workers)
         )
+        child_env["POLYRUN_FAILURE_FRAGMENTS"] = "1" if failure_fragments
         mt = matrix_total.nil? ? 0 : Integer(matrix_total)
         if mt > 1
           if matrix_index.nil?

data/lib/polyrun/cli/run_shards_parallel_children.rb

@@ -0,0 +1,99 @@
+module Polyrun
+  class CLI
+    # Spawns and waits on worker processes for +run-shards+ / +ci-shard-*+ fan-out.
+    module RunShardsParallelChildren
+      private
+
+      # @return [Array(Array, Integer, nil)] +[pids, spawn_error_code]+; +spawn_error_code+ is +nil+ when all spawns succeeded
+      # rubocop:disable Metrics/AbcSize -- shard loop: spawn + shard hooks + env
+      def run_shards_spawn_workers(ctx, hook_cfg)
+        workers = ctx[:workers]
+        cmd = ctx[:cmd]
+        cfg = ctx[:cfg]
+        plan = ctx[:plan]
+        parallel = ctx[:parallel]
+        mx = ctx[:matrix_shard_index]
+        mt = ctx[:matrix_shard_total]
+
+        pids = []
+        workers.times do |shard|
+          paths = plan.shard(shard)
+          if paths.empty?
+            Polyrun::Log.warn "polyrun run-shards: shard #{shard} skipped (no paths)" if @verbose || parallel
+            next
+          end
+
+          env_shard = ENV.to_h.merge(
+            "POLYRUN_HOOK_ORCHESTRATOR" => "1",
+            "POLYRUN_SHARD_INDEX" => shard.to_s,
+            "POLYRUN_SHARD_TOTAL" => workers.to_s
+          )
+          code = hook_cfg.run_phase_if_enabled(:before_shard, env_shard)
+          if code != 0
+            run_shards_terminate_children!(pids)
+            return [pids, code]
+          end
+
+          child_env = shard_child_env(
+            cfg: cfg,
+            workers: workers,
+            shard: shard,
+            matrix_index: mx,
+            matrix_total: mt,
+            failure_fragments: ctx[:merge_failures]
+          )
+          child_env = child_env.merge("POLYRUN_HOOK_ORCHESTRATOR" => "0")
+          child_env = hook_cfg.merge_worker_ruby_env(child_env)
+
+          Polyrun::Log.warn "polyrun run-shards: shard #{shard} → #{paths.size} file(s)" if @verbose
+          pid = run_shards_spawn_one_worker(child_env, cmd, paths, hook_cfg)
+          pids << {pid: pid, shard: shard}
+          Polyrun::Debug.log("[parent pid=#{$$}] run-shards: Process.spawn shard=#{shard} child_pid=#{pid} spec_files=#{paths.size}")
+          Polyrun::Log.warn "polyrun run-shards: started shard #{shard} pid=#{pid} (#{paths.size} file(s))" if parallel
+        end
+        [pids, nil]
+      end
+      # rubocop:enable Metrics/AbcSize
+
+      def run_shards_spawn_one_worker(child_env, cmd, paths, hook_cfg)
+        if hook_cfg.worker_hooks? && !Polyrun::Hooks.disabled?
+          Process.spawn(child_env, "sh", "-c", hook_cfg.build_worker_shell_script(cmd, paths))
+        else
+          Process.spawn(child_env, *cmd, *paths)
+        end
+      end
+
+      # @return [Array(Array, Integer)] +[shard_results, after_shard_hook_error_code]+ (0 when all +after_shard+ hooks passed)
+      def run_shards_wait_all_children(pids, hook_cfg, ctx)
+        workers = ctx[:workers]
+        shard_results = []
+        after_hook_err = 0
+        Polyrun::Debug.time("Process.wait (#{pids.size} worker process(es))") do
+          pids.each do |h|
+            Process.wait(h[:pid])
+            exitstatus = $?.exitstatus
+            ok = $?.success?
+            Polyrun::Debug.log("[parent pid=#{$$}] run-shards: Process.wait child_pid=#{h[:pid]} shard=#{h[:shard]} exit=#{exitstatus} success=#{ok}")
+            env_after = ENV.to_h.merge(
+              "POLYRUN_HOOK_ORCHESTRATOR" => "1",
+              "POLYRUN_SHARD_INDEX" => h[:shard].to_s,
+              "POLYRUN_SHARD_TOTAL" => workers.to_s,
+              "POLYRUN_WORKER_EXIT_STATUS" => exitstatus.to_s
+            )
+            rc = hook_cfg.run_phase_if_enabled(:after_shard, env_after)
+            after_hook_err = rc if rc != 0 && after_hook_err == 0
+            shard_results << {shard: h[:shard], exitstatus: exitstatus, success: ok}
+          end
+        rescue Interrupt
+          # Do not trap SIGINT: Process.wait raises Interrupt; a trap races and prints Interrupt + SystemExit traces.
+          run_shards_shutdown_on_signal!(pids, 130)
+        rescue SignalException => e
+          raise unless e.signm == "SIGTERM"
+
+          run_shards_shutdown_on_signal!(pids, 143)
+        end
+        [shard_results, after_hook_err]
+      end
+    end
+  end
+end