polyrun 1.4.2 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f37969f7a73313a98aa0fd343ae1ed871a32b6dc504da56dd5cb51f42ef2969a
-  data.tar.gz: 3e17734a4419ea15b243ad01261701106d99e3470e4408d2b545deb5876ddcf3
+  metadata.gz: 4ab4d8c8553db3e8e62e83967c9090fc7e788c994d450febb0dc64d39d946f3b
+  data.tar.gz: 3edbc34c324880653c8fd6d7675a31193adfebb0f0a4439d33f74e4ab9d1c327
 SHA512:
-  metadata.gz: 68b10554a23d62042db0d6d5307933ffdbb0fb58cc2452b78e1fca2e1a9b42095bc25e7e98b6b104f30a8be6cb0c4aad09212a57b1f4bbdbd2e2efad3501e3fc
-  data.tar.gz: 70594cb4f7cd5fb32be05aa0da60058e0fc9b50c9ca3ae4223e88a39293a5199e078eda52a2cb461f6782011aa22f05e3ca5efdc7b52cacc184f67a7326c13db
+  metadata.gz: 1e372f737bf9d8cf76d39979f87dec6acd33e325ef1b5355acdbdb588d9542bc94d0fa0236eecb6632152c0b148560e489a9e26132bede885ddbb73628fa3830
+  data.tar.gz: c61bedba0ea3f6399ffee3d217d719182de6c3e945fbbb14aea9c811f0b75e7e968b7e30a30162fb944635a949629e0102976d0f01e2851411b7694649ac8c5f

data/CHANGELOG.md

@@ -2,6 +2,44 @@
 
 ## Unreleased
 
+## 2.1.0 (2026-07-03)
+
+- Add experimental per-example spec quality (`Polyrun::SpecQuality`): `POLYRUN_SPEC_QUALITY=1`, worker JSONL fragments, `merge-spec-quality`, `report-spec-quality`, and `run-shards --merge-spec-quality`.
+- Add `Polyrun::RSpec.install_spec_quality!`, `Polyrun::Minitest.install_spec_quality!`, and Quick example hooks; `polyrun init --profile spec-quality`.
+- Add `Polyrun::Coverage::ExampleDiff` for per-example line hit deltas from `Coverage.peek_result`; record per-example factory counts via `Data::FactoryCounts`.
+- Report zero-hit examples, line churn, hot lines, shard attribution, optional partition hints (`report-spec-quality --plan`), optional CPU/GC/IO profiling (stdlib), and CI gate thresholds in `config/polyrun_spec_quality.yml`.
+- Document spec quality in `docs/SPEC_QUALITY.md`.
+
+## 2.0.0 (2026-07-03)
+
+- Add `polyrun run-queue` to init a file-backed queue and run N workers that claim batches until drained; `--on-failure exit|requeue` reclaims open leases when a worker exits before ack.
+- Add `polyrun queue reclaim` with `--older-than DURATION` (`10m`, `1h`, …) and optional `--worker ID`; add `Queue::FileStore#reclaim!` and `#reclaim_lease!`.
+- Add partition imbalance and dominant-file reports after `plan` and `run-shards` (`Partition::Reports`); warn on stale or missing timing coverage before cost-based partition (`Partition::TimingDiagnostics`).
+- Add partition strategies `lazy_robin` (sorted round-robin with timing diagnostics), `preserve_order_round_robin` (paths-file order), `stable_cost_binpack` (stable assignment with LPT fallback), and `weighted_hrw` (rendezvous with `shard_weights`).
+- Add `partition.stable_assignment_file` and `partition.shard_weights` for stable binpack and weighted HRW; wire into `plan` and `run-shards`.
+- Add `Polyrun::Timing::Stats` for rich timing entries (`last_seconds`, `mean`, `p95`, `runs`, `failures`, `timeouts`); `merge-timing` merges via `Stats.merge_entries` and emits `Timing::VarianceReport` warnings.
+- Add `POLYRUN_HRW_FAST_SCORE` for faster deterministic HRW scoring.
+- Auto-select `cost_binpack` when `partition.timing_file` or `--timing` is set and strategy is not explicit; load timing for `lazy_robin` and diagnostics even when strategy stays round-robin.
+- Fix `paths_build` glob stages to match against the remaining pool (`File.fnmatch?`) instead of re-globbing the cwd.
+- Warn when a database URL has no database segment (shard suffix skipped); name `shard_index` in `CloneShards` errors.
+- Refactor LPT forced-item handling into a single pass before free-item balance.
+- Add Makefile, Prayfile, and root `polyrun.yml`; run repo specs via polyrun in CI.
+- BREAKING: `partition.timing_file` without an explicit `strategy` now implies `cost_binpack` instead of `round_robin`.
+- BREAKING: `merge-timing` output entries are objects with timing stats, not bare scalar seconds; tools that read merged JSON must accept objects or use `Stats.binpack_weight`.
+- BREAKING: `paths_build` glob stages filter the staged pool only; membership and order can differ from prior re-glob behavior.
+
+## 1.5.0 (2026-05-04)
+
+- Add `run-shards --worker-timeout SEC` and `POLYRUN_WORKER_TIMEOUT_SEC` (wall time per worker since spawn); stop stuck workers; record exit 124 for that shard.
+- Add `run-shards --worker-idle-timeout SEC` and `POLYRUN_WORKER_IDLE_TIMEOUT_SEC`; parent reads monotonic timestamps from `POLYRUN_WORKER_PING_FILE`; record exit 125 when the last ping is stale. Idle applies only after a valid positive ping (use wall timeout until the first ping).
+- Add `Polyrun::WorkerPing` (`ping!`, `ensure_interval_ping_thread!` when `POLYRUN_WORKER_PING_THREAD`). Add `Polyrun::RSpec.install_worker_ping!` and `Polyrun::Minitest.install_worker_ping!`; Polyrun Quick calls `WorkerPing.ping!` around each example. Parent creates ping paths under `tmp/polyrun/` and unlinks files after workers exit.
+- Poll every live shard worker together when timeouts are enabled so idle and wall limits apply to all children, not only the first waiter.
+- Split parallel worker teardown into `RunShardsParallelWait` and `RunShardsWorkerInterrupt`; keep spawn logic in `RunShardsParallelChildren`.
+- Add `Polyrun::Log.orchestration_warn`; when `POLYRUN_ORCHESTRATION_STDERR=1`, copy one line to process `$stderr` if `Log.stderr` is not the same object (custom/null sinks).
+- Wire `env_worker_timeout_sec` / `env_worker_idle_timeout_sec` into `ci-shard-run` plan context. Rescue `Interrupt` around `after_suite` in `run-shards` and `ci-shard` orchestration where suite hooks run.
+- In `Polyrun::Hooks#run_phase`, rescue `Interrupt` for Ruby DSL and shell hook phases (return 130).
+- Document worker timeout, idle ping, and `POLYRUN_ORCHESTRATION_STDERR` in `polyrun help`. Add `sig/polyrun/worker_ping.rbs` and extend `Polyrun::RSpec` / `Polyrun::Minitest` installer signatures.
+
 ## 1.4.2 (2026-04-24)
 
 - Add richer HTML coverage reports: summary cards, group coverage, sortable file tables, project-relative paths, and per-file source detail.

data/README.md

@@ -23,7 +23,7 @@ Capybara and Playwright stay in your application; Polyrun does not replace brows
 2. Add a `polyrun.yml` beside the app, or pass `-c` to point at one. Configure `partition` (paths, shard index and total, strategy), and optionally `databases` (Postgres template and `shard_db_pattern`), `prepare`, and `coverage`. If you use `partition.paths_build`, Polyrun can write `partition.paths_file` (for example `spec/spec_paths.txt`) from globs and ordered stages—substring priorities for integration specs, or a regex stage for “Rails-heavy files first”—without a per-project Ruby script. That step runs before `plan` and `run-shards`. Use `bin/polyrun build-paths` to refresh the paths file only.
 3. Run prepare once before fan-out—for example `script/ci_prepare` for Vite or webpack builds, and `Polyrun::Prepare::Assets` digest markers. See `examples/TESTING_REQUIREMENTS.md`.
 4. Run workers with `bin/polyrun run-shards --workers N -- bundle exec rspec`: N separate OS processes, each running RSpec with its own file list from `partition.paths_file`, or `spec/spec_paths.txt`, or else `spec/**/*_spec.rb`. Stderr shows where paths came from; after a successful multi-worker run it reminds you to run merge-coverage unless you use `parallel-rspec` or `run-shards --merge-coverage`.
-5. Merge artifacts with `bin/polyrun merge-coverage` on `coverage/polyrun-fragment-*.json` (one fragment per `POLYRUN_SHARD_INDEX` when coverage is on), or use `bin/polyrun parallel-rspec` or `run-shards --merge-coverage` so Polyrun runs merge for you. Optional: `merge-timing`, `report-timing`, `report-junit`.
+5. Merge artifacts with `bin/polyrun merge-coverage` on `coverage/polyrun-fragment-*.json` (one fragment per `POLYRUN_SHARD_INDEX` when coverage is on), or use `bin/polyrun parallel-rspec` or `run-shards --merge-coverage` so Polyrun runs merge for you. Optional: `merge-timing`, `report-timing`, `report-junit`, `merge-spec-quality`, `report-spec-quality` (experimental per-example spec quality; see [docs/SPEC_QUALITY.md](docs/SPEC_QUALITY.md)).
 
 ### Hooks (`hooks:` in `polyrun.yml`)
 
@@ -189,7 +189,7 @@ bin/polyrun quick spec/polyrun_quick/smoke.rb
 
 Shard index and total in CI (`Polyrun::Env::Ci`): when set, `POLYRUN_SHARD_INDEX` and `POLYRUN_SHARD_TOTAL` take precedence. When `CI` is truthy, `CI_NODE_INDEX` / `CI_NODE_TOTAL` and other parallel-job environment variables are read if present. If your runner does not export those, set `POLYRUN_SHARD_*` from the job matrix.
 
-File queue (`polyrun queue …`): batches live on disk under a lock file; paths move from `pending` to `leases` on claim and to `done` on ack. There is no lease TTL: if a worker dies after claiming, paths remain in `leases` until you recover them (manually or with a future reclaim command).
+File queue (`polyrun queue …`): batches live on disk under a lock file; paths move from `pending` to `leases` on claim and to `done` on ack. Stale leases can be returned to `pending` with `polyrun queue reclaim --older-than 10m` (or `--worker ID`). Use `polyrun queue status --json` for lease ages. `polyrun run-queue` reclaims a worker's open lease when that worker exits before ack.
 
 ## Examples
 

data/docs/SETUP_PROFILE.md

@@ -45,6 +45,8 @@ Rule: anything expensive (compile, `yarn`, Playwright download) belongs in prepa
 | Plain glob | `partition.paths_build.all_glob: spec/**/*_spec.rb` and empty or minimal `stages` |
 | Ordered stages | `partition.paths_build.stages`: regex (e.g. slow integration first) or `sort_by_substring_order` for stable ordering |
 
+`paths_build` controls **membership** in `partition.paths_file`, not shard assignment order. Default `round_robin` sorts paths alphabetically before mod assignment. Use `strategy: preserve_order_round_robin` to honor paths-file line order. Set `partition.timing_file` without `strategy` to auto-select `cost_binpack`; use `lazy_robin` for round-robin assignment with timing diagnostics.
+
 Refresh list: `polyrun -c polyrun.yml build-paths` (also runs automatically before `plan` / `run-shards` when configured).
 
 ## 6. Coverage and CI reports

data/lib/polyrun/cli/ci_shard_hooks.rb

@@ -49,7 +49,11 @@ module Polyrun
               "POLYRUN_SHARD_TOTAL" => ctx[:workers].to_s,
               "POLYRUN_SUITE_EXIT_STATUS" => exit_code.to_s
             )
-            hook_cfg.run_phase_if_enabled(:after_suite, env_after)
+            begin
+              hook_cfg.run_phase_if_enabled(:after_suite, env_after)
+            rescue Interrupt
+              Polyrun::Log.warn "polyrun ci-shard: after_suite hook interrupted"
+            end
           end
         end
       end
@@ -109,9 +113,13 @@ module Polyrun
           exit_code
         ensure
           if suite_started
-            hook_cfg.run_phase_if_enabled(:after_suite, env_orch.merge(
-              "POLYRUN_SUITE_EXIT_STATUS" => exit_code.to_s
-            ))
+            begin
+              hook_cfg.run_phase_if_enabled(:after_suite, env_orch.merge(
+                "POLYRUN_SUITE_EXIT_STATUS" => exit_code.to_s
+              ))
+            rescue Interrupt
+              Polyrun::Log.warn "polyrun ci-shard: after_suite hook interrupted"
+            end
           end
         end
       end

data/lib/polyrun/cli/ci_shard_run_command.rb

@@ -66,7 +66,9 @@ module Polyrun
           merge_format: nil,
           config_path: config_path,
           matrix_shard_index: mx,
-          matrix_shard_total: mt
+          matrix_shard_total: mt,
+          worker_timeout_sec: env_worker_timeout_sec,
+          worker_idle_timeout_sec: env_worker_idle_timeout_sec
         }
       end
 

data/lib/polyrun/cli/help.rb

@@ -23,7 +23,12 @@ module Polyrun
           Warn if merge-coverage wall time exceeds N seconds (default 10): POLYRUN_MERGE_SLOW_WARN_SECONDS (0 disables)
           Failure fragments (run-shards --merge-failures): POLYRUN_MERGE_FAILURES=1; parent sets POLYRUN_FAILURE_FRAGMENTS=1 in workers; POLYRUN_FAILURE_FRAGMENT_DIR, POLYRUN_MERGED_FAILURES_OUT, POLYRUN_MERGED_FAILURES_FORMAT; after_suite sets POLYRUN_MERGED_FAILURES_PATH when merge ran
           Parallel RSpec workers: POLYRUN_WORKERS default 5, max 10 (run-shards / parallel-rspec / start); distinct from POLYRUN_SHARD_PROCESSES / ci-shard --shard-processes (local processes per CI matrix job)
+          Per-worker wall timeout: run-shards --worker-timeout SEC or POLYRUN_WORKER_TIMEOUT_SEC (max time since each worker spawn). Parent polls all live workers together. Exit 124; remaining workers stopped.
+          Per-worker idle timeout: --worker-idle-timeout SEC or POLYRUN_WORKER_IDLE_TIMEOUT_SEC counts only after a successful ping timestamp (positive float in POLYRUN_WORKER_PING_FILE); empty or unreadable pings do not satisfy idle enforcement—use wall timeout until the first ping. RSpec/Minitest/Quick installers call Polyrun::WorkerPing.ping! per example/suite. Ping files live under tmp/polyrun/ (gitignored via tmp/); parent unlinks each after its worker exits. Exit 125. Optional outer cap: --worker-timeout (exit 124). Optional periodic pings: POLYRUN_WORKER_PING_THREAD=1 (POLYRUN_WORKER_PING_INTERVAL_SEC); WorkerPing.ensure_interval_ping_thread! (installers invoke it—call yourself if wiring workers without install_worker_ping!).
+          If Polyrun::Log.stderr is null or redirected away, set POLYRUN_ORCHESTRATION_STDERR=1 to also print timeout/SIGINT summary lines to process stderr.
+          Spec quality (opt-in): POLYRUN_SPEC_QUALITY=1; per-example JSONL fragments; POLYRUN_SPEC_QUALITY_SAMPLE=0.0-1.0 (default 1.0); POLYRUN_SPEC_QUALITY_STRICT=1; run-shards --merge-spec-quality sets POLYRUN_SPEC_QUALITY_FRAGMENTS=1 in workers; merge-spec-quality / report-spec-quality
           Partition timing granularity (default file): POLYRUN_TIMING_GRANULARITY=file|example (experimental per-example; see partition.timing_granularity)
+          Partition strategies: round_robin (default, sorted), preserve_order_round_robin (paths-file order), lazy_robin (sorted RR + timing diagnostics), cost_binpack (LPT), hrw. partition.timing_file without strategy implies cost_binpack.
 
           commands:
             version              print version
@@ -31,20 +36,23 @@ module Polyrun
             prepare              run prepare recipe: default | assets (optional prepare.command overrides bin/rails assets:precompile) | shell (prepare.command required)
             merge-coverage       merge SimpleCov JSON fragments (json/lcov/cobertura/console)
             merge-failures       merge per-shard failure JSONL fragments or RSpec JSON files (jsonl/json)
-            run-shards           fan out N parallel OS processes (POLYRUN_SHARD_*; not Ruby threads); optional --merge-coverage / --merge-failures
+            run-shards           fan out N parallel OS processes (POLYRUN_SHARD_*; not Ruby threads); optional --merge-coverage / --merge-failures / --merge-spec-quality
             parallel-rspec       run-shards + merge-coverage (defaults to: bundle exec rspec after --)
             start                parallel-rspec; auto-runs prepare (shell/assets) and db:setup-* when polyrun.yml configures them; legacy script/build_spec_paths.rb if paths_build absent
             ci-shard-run         CI matrix: build-paths + plan for POLYRUN_SHARD_INDEX / POLYRUN_SHARD_TOTAL (or config), then run your command with that shard's paths after --; optional --shard-processes M or --workers M (POLYRUN_SHARD_PROCESSES; not POLYRUN_WORKERS) for N×M jobs × processes on this host
             ci-shard-rspec       same as ci-shard-run -- bundle exec rspec; optional --shard-processes / --workers / -- [rspec-only flags]
             build-paths          write partition.paths_file from partition.paths_build (same as auto step before plan/run-shards)
             init                 write a starter polyrun.yml or POLYRUN.md from built-in templates (see docs/SETUP_PROFILE.md)
-            queue                file-backed batch queue: init (optional --shard/--total etc. as plan, then claim/ack); M workers share one dir; no duplicate paths across claims
+            queue                file-backed batch queue: init (optional --shard/--total etc. as plan, then claim/ack/reclaim/status --json)
+            run-queue            init queue and run N workers that claim batches until drained
             quick                run Polyrun::Quick (describe/it, before/after, let, expect…to, assert_*; optional capybara!)
             hook run <phase>     run one shell hook from polyrun.yml hooks: (e.g. before_suite); optional --shard/--total
             report-coverage      write all coverage formats from one JSON file
             report-junit         RSpec JSON or Polyrun testcase JSON → JUnit XML (CI)
             report-timing        print slow-file summary from merged timing JSON
             merge-timing         merge polyrun_timing_*.json shards
+            merge-spec-quality   merge polyrun-spec-quality-fragment-*.jsonl shards
+            report-spec-quality  spec quality report from merged JSON (zero-hit, hot lines, churn)
             config               print effective config by dotted path (see Polyrun::Config::Effective; same tree as YAML plus merged prepare.env, resolved partition shard fields, workers)
             env                  print shard + database env (see polyrun.yml databases)
             db:setup-template    migrate template DB (PostgreSQL)

data/lib/polyrun/cli/helpers.rb

@@ -11,6 +11,28 @@ module Polyrun
         Polyrun::Config::Resolver.env_int(name, fallback)
       end
 
+      # Per-worker wall clock (from spawn) for run-shards / ci-shard fan-out; unset or invalid means no limit.
+      def env_worker_timeout_sec
+        s = ENV["POLYRUN_WORKER_TIMEOUT_SEC"].to_s.strip
+        return nil if s.empty?
+
+        f = Float(s, exception: false)
+        return nil if f.nil? || f <= 0
+
+        f
+      end
+
+      # Max seconds without a new monotonic timestamp ping in the worker (see +polyrun/worker_ping+).
+      def env_worker_idle_timeout_sec
+        s = ENV["POLYRUN_WORKER_IDLE_TIMEOUT_SEC"].to_s.strip
+        return nil if s.empty?
+
+        f = Float(s, exception: false)
+        return nil if f.nil? || f <= 0
+
+        f
+      end
+
       def resolve_shard_index(pc)
         Polyrun::Config::Resolver.resolve_shard_index(pc)
       end
@@ -97,6 +119,22 @@ module Polyrun
       def resolve_partition_timing_granularity(pc, cli_val)
         Polyrun::Config::Resolver.resolve_partition_timing_granularity(pc, cli_val)
       end
+
+      def load_stable_assignment(pc)
+        path = pc["stable_assignment_file"] || pc[:stable_assignment_file]
+        return nil unless path
+
+        abs = File.expand_path(path.to_s, Dir.pwd)
+        return nil unless File.file?(abs)
+
+        data = JSON.parse(File.read(abs))
+        return data if data.is_a?(Hash)
+
+        nil
+      rescue JSON::ParserError
+        Polyrun::Log.warn "polyrun: invalid stable_assignment_file JSON: #{abs}"
+        nil
+      end
     end
   end
 end

data/lib/polyrun/cli/init_command.rb

@@ -1,4 +1,5 @@
 require "optparse"
+require "fileutils"
 
 module Polyrun
   class CLI
@@ -7,6 +8,7 @@ module Polyrun
         "gem" => "minimal_gem.polyrun.yml",
         "rails" => "rails_prepare.polyrun.yml",
         "ci-matrix" => "ci_matrix.polyrun.yml",
+        "spec-quality" => "polyrun_spec_quality.yml",
         "doc" => "POLYRUN.md"
       }.freeze
 
@@ -34,6 +36,7 @@ module Polyrun
         path = File.expand_path(dest)
         return init_refuses_overwrite(path) if File.file?(path) && !force
 
+        FileUtils.mkdir_p(File.dirname(path))
         File.write(path, body)
         Polyrun::Log.warn "polyrun init: wrote #{path}"
         0
@@ -92,7 +95,11 @@ module Polyrun
       end
 
       def default_init_output(profile)
-        (profile == "doc") ? "POLYRUN.md" : "polyrun.yml"
+        case profile
+        when "doc" then "POLYRUN.md"
+        when "spec-quality" then "config/polyrun_spec_quality.yml"
+        else "polyrun.yml"
+        end
       end
     end
   end

data/lib/polyrun/cli/partition_diagnostics.rb

@@ -0,0 +1,22 @@
+module Polyrun
+  class CLI
+    # Shared stderr diagnostics after {Partition::Plan} is built.
+    module PartitionDiagnostics
+      private
+
+      def partition_emit_diagnostics!(plan:, items:, costs:, timing_path:, granularity: :file)
+        return unless timing_path && costs && !costs.empty?
+
+        analysis = Polyrun::Partition::TimingDiagnostics.analyze(
+          items: items,
+          costs: costs,
+          timing_path: timing_path,
+          root: plan.root,
+          granularity: granularity
+        )
+        Polyrun::Partition::TimingDiagnostics.emit_warnings!(analysis)
+        Polyrun::Partition::Reports.emit_all!(plan)
+      end
+    end
+  end
+end

data/lib/polyrun/cli/plan_command.rb

@@ -1,3 +1,4 @@
+# rubocop:disable Polyrun/FileLength -- plan argv + partition emit
 require "json"
 require "optparse"
 
@@ -38,8 +39,9 @@ module Polyrun
 
         items, costs, strategy = bundle
         constraints = load_partition_constraints(pc, ctx[:constraints_path])
+        stable = load_stable_assignment(pc)
 
-        manifest = plan_command_build_manifest(
+        manifest, plan = plan_command_build_manifest(
           items: items,
           total: ctx[:total],
           strategy: strategy,
@@ -47,7 +49,16 @@ module Polyrun
           costs: costs,
           constraints: constraints,
           shard: ctx[:shard],
-          timing_granularity: ctx[:timing_granularity]
+          timing_granularity: ctx[:timing_granularity],
+          stable_assignment: stable,
+          shard_weights: pc["shard_weights"] || pc[:shard_weights]
+        )
+        partition_emit_diagnostics!(
+          plan: plan,
+          items: items,
+          costs: costs,
+          timing_path: timing_path,
+          granularity: ctx[:timing_granularity]
         )
         [manifest, 0]
       end
@@ -56,7 +67,12 @@ module Polyrun
         items = plan_plan_items(paths_file, argv)
         return nil if items.nil?
 
-        loaded = plan_load_costs_and_strategy(timing_path, ctx[:strategy], ctx[:timing_granularity])
+        loaded = plan_load_costs_and_strategy(
+          timing_path,
+          ctx[:strategy],
+          ctx[:timing_granularity],
+          strategy_explicit: ctx[:strategy_explicit]
+        )
         return nil if loaded.nil?
 
         costs, strategy = loaded
@@ -68,6 +84,7 @@ module Polyrun
           shard: resolve_shard_index(pc),
           total: resolve_shard_total(pc),
           strategy: (pc["strategy"] || pc[:strategy] || "round_robin").to_s,
+          strategy_explicit: !!(pc["strategy"] || pc[:strategy]),
           seed: pc["seed"] || pc[:seed],
           paths_file: nil,
           timing_path: nil,
@@ -80,10 +97,13 @@ module Polyrun
       def plan_command_register_partition_options!(opts, ctx)
         opts.on("--shard INDEX", Integer) { |v| ctx[:shard] = v }
         opts.on("--total N", Integer) { |v| ctx[:total] = v }
-        opts.on("--strategy NAME", String) { |v| ctx[:strategy] = v }
+        opts.on("--strategy NAME", String) do |v|
+          ctx[:strategy] = v
+          ctx[:strategy_explicit] = true
+        end
         opts.on("--seed VAL") { |v| ctx[:seed] = v }
         opts.on("--constraints PATH", "YAML: pin / serial_glob (see spec_queue.md)") { |v| ctx[:constraints_path] = v }
-        opts.on("--timing PATH", "path => seconds JSON; implies cost_binpack unless strategy is cost-based or hrw") do |v|
+        opts.on("--timing PATH", "path => seconds JSON; implies cost_binpack unless strategy is explicit or timing-aware") do |v|
           ctx[:timing_path] = v
         end
         opts.on("--timing-granularity VAL", "file (default) or example (experimental: path:line items)") do |v|
@@ -104,7 +124,7 @@ module Polyrun
         end.parse!(argv)
       end
 
-      def plan_command_build_manifest(items:, total:, strategy:, seed:, costs:, constraints:, shard:, timing_granularity: :file)
+      def plan_command_build_manifest(items:, total:, strategy:, seed:, costs:, constraints:, shard:, timing_granularity: :file, stable_assignment: nil, shard_weights: nil)
         plan = Polyrun::Debug.time("Partition::Plan.new (plan command)") do
           Polyrun::Partition::Plan.new(
             items: items,
@@ -114,7 +134,9 @@ module Polyrun
             costs: costs,
             constraints: constraints,
             root: Dir.pwd,
-            timing_granularity: timing_granularity
+            timing_granularity: timing_granularity,
+            stable_assignment: stable_assignment,
+            shard_weights: shard_weights
           )
         end
         Polyrun::Debug.log_kv(
@@ -124,16 +146,13 @@ module Polyrun
           strategy: strategy,
           path_count: items.size
         )
-        plan.manifest(shard)
+        [plan.manifest(shard), plan]
       end
 
-      def plan_resolve_timing_path(pc, timing_path, strategy)
+      def plan_resolve_timing_path(pc, timing_path, _strategy = nil)
         return timing_path if timing_path
 
-        tf = pc["timing_file"] || pc[:timing_file]
-        return tf if tf && (Polyrun::Partition::Plan.cost_strategy?(strategy) || Polyrun::Partition::Plan.hrw_strategy?(strategy))
-
-        nil
+        pc["timing_file"] || pc[:timing_file]
       end
 
       def plan_plan_items(paths_file, argv)
@@ -147,8 +166,13 @@ module Polyrun
         end
       end
 
-      def plan_load_costs_and_strategy(timing_path, strategy, timing_granularity)
+      def plan_load_costs_and_strategy(timing_path, strategy, timing_granularity, strategy_explicit: false)
+        strategy = strategy.to_s
         if timing_path
+          if strategy_explicit && strategy == "round_robin"
+            return [nil, strategy]
+          end
+
           costs = Polyrun::Partition::Plan.load_timing_costs(
             File.expand_path(timing_path.to_s, Dir.pwd),
             granularity: timing_granularity
@@ -157,12 +181,16 @@ module Polyrun
             Polyrun::Log.warn "polyrun plan: timing file missing or has no entries: #{timing_path}"
             return nil
           end
-          unless Polyrun::Partition::Plan.cost_strategy?(strategy) || Polyrun::Partition::Plan.hrw_strategy?(strategy)
+          if Polyrun::Partition::Plan.timing_load_strategy?(strategy)
+            return [costs, strategy]
+          end
+          unless strategy_explicit
             Polyrun::Log.warn "polyrun plan: using cost_binpack (timing data present)" if @verbose
-            strategy = "cost_binpack"
+            return [costs, "cost_binpack"]
           end
-          [costs, strategy]
-        elsif Polyrun::Partition::Plan.cost_strategy?(strategy)
+
+          [nil, strategy]
+        elsif Polyrun::Partition::Plan.cost_strategy?(strategy) || Polyrun::Partition::Plan.lazy_robin_strategy?(strategy)
           Polyrun::Log.warn "polyrun plan: --timing or partition.timing_file required for strategy #{strategy}"
           nil
         else
@@ -172,3 +200,4 @@ module Polyrun
     end
   end
 end
+# rubocop:enable Polyrun/FileLength

data/lib/polyrun/cli/queue_command.rb

@@ -34,8 +34,10 @@ module Polyrun
           queue_cmd_ack(argv, dir, lease_id, worker)
         when "status"
           queue_cmd_status(argv, dir)
+        when "reclaim"
+          queue_cmd_reclaim(argv, dir)
         else
-          Polyrun::Log.warn "usage: polyrun queue {init|claim|ack|status} [options]"
+          Polyrun::Log.warn "usage: polyrun queue {init|claim|ack|status|reclaim} [options]"
           2
         end
       end
@@ -126,13 +128,34 @@ module Polyrun
       end
 
       def queue_cmd_status(argv, dir)
+        json_detail = false
         OptionParser.new do |opts|
+          opts.banner = "usage: polyrun queue status [--dir DIR] [--json]"
           opts.on("--dir PATH") { |v| dir = v }
+          opts.on("--json", "Include lease details") { json_detail = true }
         end.parse!(argv)
-        s = Polyrun::Queue::FileStore.new(dir).status
+        s = Polyrun::Queue::FileStore.new(dir).status(detailed: json_detail)
         Polyrun::Log.puts JSON.generate(s)
         0
       end
+
+      def queue_cmd_reclaim(argv, dir)
+        older_than = nil
+        worker = nil
+        OptionParser.new do |opts|
+          opts.banner = "usage: polyrun queue reclaim [--dir DIR] [--older-than DURATION] [--worker ID]"
+          opts.on("--dir PATH") { |v| dir = v }
+          opts.on("--older-than DUR", "e.g. 10m, 1h, 600s") { |v| older_than = Polyrun::Queue::Duration.parse_seconds(v) }
+          opts.on("--worker ID") { |v| worker = v }
+        end.parse!(argv)
+        unless older_than || worker
+          Polyrun::Log.warn "queue reclaim: need --older-than or --worker"
+          return 2
+        end
+        n = Polyrun::Queue::FileStore.new(dir).reclaim!(older_than: older_than, worker_id: worker)
+        Polyrun::Log.puts JSON.generate({"reclaimed_paths" => n})
+        0
+      end
     end
   end
 end

data/lib/polyrun/cli/run_queue_command.rb

@@ -0,0 +1,145 @@
+require "json"
+require "optparse"
+require "shellwords"
+
+require_relative "../queue/duration"
+
+module Polyrun
+  class CLI
+    module RunQueueCommand
+      private
+
+      # rubocop:disable Metrics/AbcSize -- queue worker argv + spawn loop
+      def cmd_run_queue(argv, config_path)
+        dir = ".polyrun-queue"
+        batch = 5
+        on_failure = "exit"
+        paths_file = nil
+        workers = nil
+
+        sep = argv.index("--")
+        unless sep
+          Polyrun::Log.warn "polyrun run-queue: need -- before the command"
+          return 2
+        end
+
+        head = argv[0...sep]
+        cmd = argv[(sep + 1)..].map(&:to_s)
+        return 2 if cmd.empty?
+
+        cfg = Polyrun::Config.load(path: config_path || ENV["POLYRUN_CONFIG"])
+        pc = cfg.partition
+        ctx = plan_command_initial_context(pc)
+
+        OptionParser.new do |opts|
+          opts.banner = "usage: polyrun run-queue [--workers N] [--batch N] [--dir DIR] [--on-failure exit|requeue] [partition options] -- <command>"
+          opts.on("--workers N", Integer) { |v| workers = v }
+          opts.on("--batch N", Integer) { |v| batch = v }
+          opts.on("--dir PATH") { |v| dir = v }
+          opts.on("--paths-file PATH", String) { |v| paths_file = v }
+          opts.on("--on-failure MODE", "exit (default) or requeue") { |v| on_failure = v }
+          plan_command_register_partition_options!(opts, ctx)
+        end.parse!(head)
+
+        workers ||= env_int("POLYRUN_WORKERS", Polyrun::Config::DEFAULT_PARALLEL_WORKERS)
+        paths_file ||= pc["paths_file"] || pc[:paths_file]
+        unless paths_file
+          Polyrun::Log.warn "polyrun run-queue: need --paths-file or partition.paths_file"
+          return 2
+        end
+
+        code = Polyrun::Partition::PathsBuild.apply!(partition: pc, cwd: Dir.pwd)
+        return code if code != 0
+
+        store = Polyrun::Queue::FileStore.new(dir)
+        if File.file?(File.join(File.expand_path(dir), "queue.json"))
+          Polyrun::Log.warn "polyrun run-queue: queue already exists at #{dir}; remove it or use --dir"
+          return 2
+        end
+
+        ordered, code = queue_partition_manifest_and_ordered_paths(cfg, pc, ctx, paths_file)
+        return code if code != 0
+
+        store.init!(ordered)
+        Polyrun::Log.warn "polyrun run-queue: #{ordered.size} path(s), #{workers} worker(s), batch=#{batch}" if @verbose
+
+        run_t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        pids = run_queue_spawn_workers(store: store, workers: workers, batch: batch, cmd: cmd, on_failure: on_failure)
+        results = run_queue_wait_workers(pids, store: store, on_failure: on_failure)
+        wall = Process.clock_gettime(Process::CLOCK_MONOTONIC) - run_t0
+
+        stat = store.status(detailed: true)
+        Polyrun::Log.warn format(
+          "polyrun run-queue: done pending=%d done=%d leases=%d wall=%.1fs batches_ok=%d batches_fail=%d",
+          stat["pending"], stat["done"], stat["leases"], wall, results[:ok], results[:fail]
+        )
+
+        results[:fail].positive? ? 1 : 0
+      end
+      # rubocop:enable Metrics/AbcSize
+
+      def run_queue_spawn_workers(store:, workers:, batch:, cmd:, on_failure:)
+        pids = []
+        workers.times do |i|
+          wid = "worker-#{i}"
+          pid = Process.fork do
+            run_queue_worker_loop(store: store, worker_id: wid, batch: batch, cmd: cmd, on_failure: on_failure)
+          end
+          pids << {pid: pid, worker_id: wid}
+        end
+        pids
+      end
+
+      def run_queue_worker_loop(store:, worker_id:, batch:, cmd:, on_failure:)
+        batches_ok = 0
+        batches_fail = 0
+        loop do
+          claim = store.claim!(worker_id: worker_id, batch_size: batch)
+          paths = claim["paths"] || []
+          break if paths.empty?
+
+          code = run_queue_run_batch(cmd, paths)
+          if code == 0
+            store.ack!(lease_id: claim["lease_id"], worker_id: worker_id)
+            batches_ok += 1
+          elsif on_failure.to_s == "requeue"
+            store.reclaim_lease!(claim["lease_id"])
+            batches_fail += 1
+            exit 1
+          else
+            batches_fail += 1
+            exit code.zero? ? 1 : code
+          end
+        end
+        exit 0
+      rescue Polyrun::Error => e
+        Polyrun::Log.warn "polyrun run-queue worker #{worker_id}: #{e.message}"
+        exit 2
+      end
+
+      def run_queue_run_batch(cmd, paths)
+        system(*cmd, *paths) ? 0 : ($?.exitstatus || 1)
+      end
+
+      def run_queue_wait_workers(pids, store:, on_failure:)
+        ok = 0
+        fail = 0
+        pid_to_worker = pids.each_with_object({}) { |entry, h| h[entry[:pid]] = entry[:worker_id] }
+        while pid_to_worker.any?
+          pid, st = Process.wait2(-1)
+          worker_id = pid_to_worker.delete(pid)
+          next unless worker_id
+
+          if st.success?
+            ok += 1
+          else
+            fail += 1
+            reclaimed = store.reclaim!(worker_id: worker_id)
+            Polyrun::Log.warn "polyrun run-queue: reclaimed #{reclaimed} path(s) from #{worker_id}" if reclaimed.positive?
+          end
+        end
+        {ok: ok, fail: fail}
+      end
+    end
+  end
+end