wurk 1.10.0 → 1.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ce85d34d0d8e1f5aa0837c76310a94c2ca3e584ff0e9ca4d9e977c0347510f89
-  data.tar.gz: 113f78f30121c728a4d7f3db5dc6d1472f13ab1b6b7b50d6ee73223d681585f4
+  metadata.gz: d9594aab6027f299c217e81215d9d10d5764a5f5533b3e40cde2cd1feaae8a86
+  data.tar.gz: 26161ff0d0ea58e60acbabd550e212ca354824f31434f0d085fabbc9401b8490
 SHA512:
-  metadata.gz: ca00027cf185252fd95e78c3283a90d4a0c21acbc3cd6623bc1a4a1e2add8b6625d8436f60fc80b934a2fb35325d013903791dcad3606605b580b7633958fa9f
-  data.tar.gz: bcae6990d734afd7dbc30e951aead9b7d35b77d060c1198a11d4d34d17ba866e6e81ba1ab168a417b115c1a6a690e8cc4c9926951fec58615d7c2e40a9619fda
+  metadata.gz: 3f2c0b2115f2c7132ab9104e7850f85cf34ac694d4d67b89f615cac765112582506197ae139753489de05f7e6217abb7770129aefbd3125e6dd2be17c04a50ac
+  data.tar.gz: 0f08e99addbeeebefc503916d3a033aa88dd9ccc3ad1f557ce4aabfd43e7809b07392589e0137deff11b358304b435ed798766d89f3d3f205c3a105484a65875

data/README.md

@@ -22,11 +22,11 @@
 
 Wurk is wire-compatible with Sidekiq — same Redis keys, same job JSON, same Ruby DSL. Swap one line in your `Gemfile` and your existing jobs, batches, limiters, cron entries, and live Redis data keep working untouched. The Pro and Enterprise feature sets ship in the same free gem, with no license check and no tiers.
 
-**In production:** Wurk runs the background work at [developerz.ai](https://developerz.ai) and at partner deployments, across many servers, on the fork-based swarm described below. It is not a preview. Throughput numbers from those deployments are not published yet; what has been measured is in [docs/benchmarks.md](docs/benchmarks.md).
+**In production:** Wurk runs background work at partner deployments, across many servers, on the fork-based swarm described below. It is not a preview. Throughput numbers from those deployments are not published yet; what has been measured is in [docs/benchmarks.md](docs/benchmarks.md).
 
 **At scale:** Wurk is built for fleets, not just for one box. Kubernetes `/live` + `/ready` probes are a config line, not a sidecar; a bearer-scoped [HTTP API](docs/api-http.md) lets non-Ruby services enqueue and inspect; [OpenTelemetry](docs/telemetry.md) traces propagate client → server; per-queue [global concurrency caps](docs/rate-limiting.md) hold cluster-wide limits; and monitoring is the dashboard you already mount — live SSE, charts, no separate stack to run. See [Wurk extras](#wurk-extras).
 
-**AI-managed:** Wurk is a 100% AI-managed project. Every author's issues and pull requests, org members' included, and dependency updates, are triaged, reviewed, merged or closed by the [developerz.ai](https://developerz.ai) maintainer agent, which asks a human only for security reports and license questions, and cuts a release when a shipped fix has waited for one. New features are still built by us, humans and AI together — see [CONTRIBUTING.md](CONTRIBUTING.md) and [AI-maintained](#ai-maintained). The agent always identifies itself as a bot: it works publicly as `developerz-ai[bot]` and marks every comment it posts as bot-made. Its work and its rules are public too — browse the [repository](https://github.com/developerz-ai/wurk) and read the committed policy in [.dz/maintainer/maintainer.yml](.dz/maintainer/maintainer.yml).
+**AI-managed:** Wurk is a 100% AI-managed project. Every author's issues and pull requests, org members' included, and dependency updates, are triaged, reviewed, merged or closed by the [developerz.ai](https://developerz.ai) maintainer agent, which asks a human only for security reports, license questions and breaking changes, and judges weekly whether a release is due. New features are still built by us, humans and AI together — see [CONTRIBUTING.md](CONTRIBUTING.md) and [AI-maintained](#ai-maintained). The agent always identifies itself as a bot: it works publicly as `developerz-ai[bot]` and marks every comment it posts as bot-made. Its work and its rules are public too — browse the [repository](https://github.com/developerz-ai/wurk) and read the committed policy in [.dz/maintainer/maintainer.yml](.dz/maintainer/maintainer.yml).
 
 **On Sidekiq:** Wurk implements Sidekiq's API because it is a genuinely good API. Sidekiq is human-maintained and funds that work through its paid tiers; Wurk is AI-maintained, which is what lets the same surface be free software. Wurk is independent and not affiliated with or endorsed by Sidekiq or its maintainers — see [Why Wurk exists](#why-wurk-exists).
 
@@ -151,7 +151,7 @@ Method, per-invocation records, workload definitions, and the separate `rake ben
 - **[Batches](https://github.com/developerz-ai/wurk/blob/main/docs/batches.md)** · **[Rate limiting](https://github.com/developerz-ai/wurk/blob/main/docs/rate-limiting.md)** · **[Periodic (cron) jobs](https://github.com/developerz-ai/wurk/blob/main/docs/periodic-jobs.md)** · **[Unique jobs](https://github.com/developerz-ai/wurk/blob/main/docs/unique-jobs.md)** — the Pro/Ent features, free.
 - **[Iterable jobs](https://github.com/developerz-ai/wurk/blob/main/docs/iterable-jobs.md)** · **[Middleware](https://github.com/developerz-ai/wurk/blob/main/docs/middleware.md)** · **[Encryption](https://github.com/developerz-ai/wurk/blob/main/docs/encryption.md)** · **[Profiling](https://github.com/developerz-ai/wurk/blob/main/docs/profiling.md)**
 - **[Data API](https://github.com/developerz-ai/wurk/blob/main/docs/api.md)** · **[Metrics](https://github.com/developerz-ai/wurk/blob/main/docs/metrics.md)** — inspect queues and jobs from Ruby; job metrics, Statsd/DogStatsD, custom history.
-- **[Sentry](https://github.com/developerz-ai/wurk/blob/main/docs/sentry.md)** — built-in error reporting (`sentry-sidekiq` can't be installed alongside Wurk); terminal-failure-only reports, per-job scope, no job args.
+- **[Sentry](https://github.com/developerz-ai/wurk/blob/main/docs/sentry.md)** — built-in error reporting (`sentry-sidekiq` needs the `sidekiq` shim gem; prefer this); terminal-failure-only reports, per-job scope, no job args.
 - **API reference (parity specs):** [Sidekiq OSS](https://github.com/developerz-ai/wurk/blob/main/docs/target/sidekiq-free.md) · [Pro](https://github.com/developerz-ai/wurk/blob/main/docs/target/sidekiq-pro.md) · [Enterprise](https://github.com/developerz-ai/wurk/blob/main/docs/target/sidekiq-ent.md) — the authoritative surface Wurk matches exactly.
 - **[Authentication & authorization](https://github.com/developerz-ai/wurk/blob/main/docs/authentication.md)** — gate the dashboard behind Devise/Warden, Sorcery, Basic auth, or a token; role-based read/write; CSRF.
 - **[Securing the dashboard](https://github.com/developerz-ai/wurk/blob/main/docs/dashboard.md)** · **[Metrics history](https://github.com/developerz-ai/wurk/blob/main/docs/metrics-history.md)**
@@ -167,7 +167,7 @@ Method, per-invocation records, workload definitions, and the separate `rake ben
 
 The full test suite also runs as a non-required `compat` workflow job across Ruby 3.2.x and the newest Ruby on the current Rails series, the oldest supported Rails series on the newest Ruby, and Valkey 8 as a drop-in store alongside Redis 7.x.
 
-JRuby, TruffleRuby, and Windows fall back to threads-only mode (no fork) — behaviorally equivalent to stock Sidekiq.
+Without `fork` (JRuby, TruffleRuby, Windows; untested in CI) only the single-process `wurk` runner and `config.wurk.embed_in_web` apply: `wurkswarm` and the railtie's swarm have no fallback.
 
 ## Running the workers
 
@@ -284,13 +284,13 @@ Infrastructure this basic should be free software. A Rails app shouldn't need a
 What has made that hard is maintenance: someone has to be paid to do it. Sidekiq funds a decade of *human* maintenance through its paid tiers, which is an honest trade. Wurk makes a different one — it is maintained **by AI**: every author's issues and pull requests, the team's own included, and dependency updates, are handled end to end by a maintainer agent, and the implementation, parity suite, docs, and benchmarks are written and kept current by AI agents working with the team, which still builds the features and cuts the releases. A fix, a doc update, or a version bump is no longer somebody's week, which is what makes it practical to:
 
 - ship the entire Pro + Enterprise surface with no tier, no flag gate, and no license check;
-- keep parity honest mechanically rather than by hand — an independently written parity oracle suite, pinned to a documented Sidekiq revision, plus sidekiq-cron's upstream suite running against Wurk on every push ([docs/idea/14-ecosystem-compat.md](https://github.com/developerz-ai/wurk/blob/main/docs/idea/14-ecosystem-compat.md) tracks the rest);
+- keep parity honest mechanically rather than by hand — an independently written parity oracle suite, pinned to a documented Sidekiq revision, plus the upstream suites of sidekiq-cron, sidekiq-unique-jobs, sidekiq-failures and sentry-sidekiq running against Wurk on every push ([docs/idea/14-ecosystem-compat.md](https://github.com/developerz-ai/wurk/blob/main/docs/idea/14-ecosystem-compat.md) tracks the rest);
 - keep adding surface Sidekiq doesn't have — the [Wurk extras](#wurk-extras) above landed as one release;
 - hold ourselves to published numbers instead of adjectives — the measured comparison against stock Sidekiq is published [as measured](docs/benchmarks.md), including the unflattering ones, and `rake bench:vs_sidekiq` reproduces it. It is a published measurement, not a per-release CI run.
 
 DHH makes the broader version of this argument in [Let the agents democratize open source](https://world.hey.com/dhh/let-the-agents-democratize-open-source-9fd630a9): open source fought for everyone's right to change the software they run, and refusing agent-written code re-erects the gate it spent decades tearing down — "all programmers are equal, but some programmers are more equal than others." His subject is contributions; ours is maintenance, which is the same economics from the other end. The reason a licence key guards batches and cron is not that the code is precious, it's that somebody had to be paid to keep it working. Drop that cost and the tier stops being necessary.
 
-What makes it work in practice is that the agents run inside machinery built to check them. The parity oracle above is a merge-blocking gate, not decoration; sidekiq-cron's upstream suite runs against Wurk in CI and reports on PRs, but it is not a required check and never gates a merge; the `rake bench` PR regression check flags deltas past a noise-adjusted threshold as an advisory comment; and the published benchmark numbers are one-off historical measurements, reproduced on demand rather than enforced in CI. The release gate has never once let an unverified gem reach RubyGems. Agents supply the pace, the gates supply the certainty — and when a gate does catch something, the fix is to make that class of mistake structurally impossible rather than to slow the agents down. The release lane derives its own tag from `Wurk::VERSION` so the two can't drift apart; [RELEASE.md](https://github.com/developerz-ai/wurk/blob/main/RELEASE.md) walks through it.
+What makes it work in practice is that the agents run inside machinery built to check them. The parity oracle above is a merge-blocking gate, not decoration; four gems' upstream suites run against Wurk in CI and report on PRs, but they are not required checks and never gate a merge; the `rake bench` PR regression check flags deltas past a noise-adjusted threshold as an advisory comment; and the published benchmark numbers are one-off historical measurements, reproduced on demand rather than enforced in CI. The release gate has never once let an unverified gem reach RubyGems. Agents supply the pace, the gates supply the certainty — and when a gate does catch something, the fix is to make that class of mistake structurally impossible rather than to slow the agents down. The release lane derives its own tag from `Wurk::VERSION` so the two can't drift apart; [RELEASE.md](https://github.com/developerz-ai/wurk/blob/main/RELEASE.md) walks through it.
 
 Wurk is MIT and stays that way. If what you need is a commercial support contract and a human on the other end of an email, buying that is a perfectly good answer.
 
@@ -303,9 +303,9 @@ Wurk is MIT and stays that way. If what you need is a commercial support contrac
 + gem "wurk"
 ```
 
-`bundle install && restart`. Wurk reads and writes the same Redis schema, so there is no data migration: Wurk picks up the queues, schedules, retries, batches and periodic loops Sidekiq left behind. Running Sidekiq and Wurk workers against one Redis *at the same time* is not tested yet, so a live production move is a **drain cutover** (stop Sidekiq's workers, confirm nothing is in flight, start Wurk), with a matching rollback; the [production cutover](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md#9-production-cutover) section has every step and command. sidekiq-cron's upstream suite runs against Wurk in the [`ecosystem` CI job](https://github.com/developerz-ai/wurk/blob/main/.github/workflows/ecosystem.yml) (see [`test/ecosystem/`](https://github.com/developerz-ai/wurk/tree/main/test/ecosystem)). sidekiq-unique-jobs, sidekiq-scheduler, sidekiq-status, sidekiq-failures and sidekiq-throttled are target additions tracked in [docs/idea/14-ecosystem-compat.md](https://github.com/developerz-ai/wurk/blob/main/docs/idea/14-ecosystem-compat.md).
+`bundle install && restart`. Wurk reads and writes the same Redis schema, so there is no data migration: Wurk picks up the queues, schedules, retries, batches and periodic loops Sidekiq left behind. Running Sidekiq and Wurk workers against one Redis *at the same time* is not tested yet, so a live production move is a **drain cutover** (stop Sidekiq's workers, confirm nothing is in flight, start Wurk), with a matching rollback; the [production cutover](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md#9-production-cutover) section has every step and command. The upstream suites of sidekiq-cron, sidekiq-unique-jobs, sidekiq-failures and sentry-sidekiq run against Wurk in the [`ecosystem` CI job](https://github.com/developerz-ai/wurk/blob/main/.github/workflows/ecosystem.yml) (see [`test/ecosystem/`](https://github.com/developerz-ai/wurk/tree/main/test/ecosystem)). sidekiq-scheduler, sidekiq-status and sidekiq-throttled are target additions tracked in [docs/idea/14-ecosystem-compat.md](https://github.com/developerz-ai/wurk/blob/main/docs/idea/14-ecosystem-compat.md).
 
-**Keeping a `sidekiq-*` add-on gem?** Those gems declare `add_dependency "sidekiq"`, which would reinstall real Sidekiq next to Wurk. Satisfy it with Wurk's git-only [`sidekiq` shim gem](https://github.com/developerz-ai/wurk/tree/main/ecosystem/sidekiq-shim) instead: `gem "sidekiq", github: "developerz-ai/wurk", glob: "ecosystem/sidekiq-shim/*.gemspec"`. Only sidekiq-cron is exercised that way on every PR; treat the others as untested.
+**Keeping a `sidekiq-*` add-on gem?** Those gems declare `add_dependency "sidekiq"`, which would reinstall real Sidekiq next to Wurk. Satisfy it with Wurk's git-only [`sidekiq` shim gem](https://github.com/developerz-ai/wurk/tree/main/ecosystem/sidekiq-shim) instead: `gem "sidekiq", github: "developerz-ai/wurk", glob: "ecosystem/sidekiq-shim/*.gemspec"`. Only sidekiq-cron, sidekiq-unique-jobs, sidekiq-failures and sentry-sidekiq are exercised that way, by the `ecosystem` workflow; treat the others as untested.
 
 Full walkthrough — config side-by-side, the Redis key/`sidekiq_options` mapping, known incompatibilities, moving sidekiq-cron schedules over (`rake wurk:import:cron`), and the production cutover and rollback playbook: **[docs/migrate-from-sidekiq.md](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md)**. On the night, keep the **[incident runbook](https://github.com/developerz-ai/wurk/blob/main/docs/runbook.md)** open.
 
@@ -332,7 +332,7 @@ TypeScript end to end, MIT, and as measured about its own claims as this file is
 | Dependabot | driven to green and merged |
 | an issue or pull request by anyone with write access (org members, collaborators) | handled like anyone's: triaged, reviewed, driven to green, merged |
 | an outside change to CI, release or publish automation | declined: members maintain the pipeline that publishes the gem |
-| a breaking change | the bot's call, not a human's; a change that breaks the drop-in Sidekiq contract is declined, since a revert cannot take that promise back |
+| a breaking change | a human decides; a change that breaks the drop-in Sidekiq contract is declined, since a revert cannot take that promise back |
 
 Releases follow the work, not a calendar. A weekday checkup asks whether a shipping fix has merged and nobody has released it. When one has, the bot files a release request, its coding fleet opens the `Wurk::VERSION` bump PR, and merging that PR publishes the gem ([RELEASE.md](RELEASE.md)). The bot's policy is committed in [`.dz/maintainer/maintainer.yml`](.dz/maintainer/maintainer.yml).
 

data/app/controllers/wurk/dashboard_controller.rb

@@ -18,14 +18,18 @@ module Wurk
   # Vite dev server (default :5173) so contributors get HMR without
   # rebuilding the bundle on every change.
   class DashboardController < ApplicationController
+    # bin/dev exports VITE_PORT and vite.config.ts binds it, with strictPort, so this
+    # has to read the same variable or an overridden port answers every page with a 500.
+    def self.vite_dev_url
+      "http://localhost:#{ENV.fetch('VITE_PORT', '5173')}#{VITE_ASSET_BASE}"
+    end
+
     # Vite serves the dev shell under its `base` (vite.config.ts), which is the
     # same path the engine mounts assets at — so derive these from
     # AssetMount::PREFIX to keep the Ruby side in lockstep. Fetching the server
     # root instead returns Vite's "did you mean /wurk-assets/" hint page, not the
     # shell (issue #181).
-    VITE_DEV_HOST = 'http://localhost:5173'
     VITE_ASSET_BASE = "#{::Wurk::Engine::AssetMount::PREFIX}/".freeze # "/wurk-assets/"
-    VITE_DEV_URL = "#{VITE_DEV_HOST}#{VITE_ASSET_BASE}".freeze
     INDEX_REL_PATH = ['vendor', 'assets', 'dashboard', 'index.html'].freeze
     # The SPA mount point, matched without its closing `>` so the locale hint
     # appends to whatever attributes the shell already carries.
@@ -115,11 +119,12 @@ module Wurk
     end
 
     def fetch_vite_dev_shell
-      uri = ::URI.parse(VITE_DEV_URL)
+      uri = ::URI.parse(self.class.vite_dev_url)
       rewrite_dev_asset_urls(::Net::HTTP.get(uri))
     rescue ::StandardError => e
-      raise "Wurk dashboard: cannot reach Vite dev server at #{VITE_DEV_URL} " \
-            "(#{e.class}: #{e.message}). Run `bin/rake frontend:dev` from the gem root."
+      raise "Wurk dashboard: cannot reach Vite dev server at #{self.class.vite_dev_url} " \
+            "(#{e.class}: #{e.message}). Run `bin/rake frontend:dev` from the gem root, or `bin/dev`, " \
+            'with the same VITE_PORT.'
     end
 
     # The shell's entry URLs (`src="/wurk-assets/@vite/client"`,
@@ -130,7 +135,7 @@ module Wurk
     # `server.origin` covers runtime-generated asset URLs but not these entry
     # tags, so rewrite them here.
     def rewrite_dev_asset_urls(html)
-      html.gsub(/(["'])#{::Regexp.escape(VITE_ASSET_BASE)}/, "\\1#{VITE_DEV_URL}")
+      html.gsub(/(["'])#{::Regexp.escape(VITE_ASSET_BASE)}/, "\\1#{self.class.vite_dev_url}")
     end
 
     def read_built_index

data/exe/wurk

@@ -4,12 +4,18 @@
 # Standalone runner. Does NOT load the Rails engine.
 # Usage: `exe/wurk -C config/wurk.yml`     — worker: one process, thread pool
 #        `exe/wurk api --port 7433`        — machine HTTP API, runs no jobs
+#        `exe/wurk preflight` / `verify`   — read-only cutover checks, run no jobs
 
 $stdout.sync = true
 $LOAD_PATH.unshift(File.expand_path('../lib', __dir__))
 
 require 'wurk'
 
+if %w[preflight verify].include?(ARGV.first)
+  require 'wurk/preflight/command'
+  exit Wurk::Preflight::Command.run(ARGV.dup)
+end
+
 begin
   cli = Wurk::CLI.instance
   cli.parse
@@ -20,7 +26,6 @@ begin
 rescue StandardError => e
   raise e if $DEBUG
 
-  warn "wurk: #{e.message}"
-  warn e.backtrace.join("\n")
+  Wurk::CLI.report_failure('wurk', e)
   exit 1
 end

data/exe/wurkswarm

@@ -17,7 +17,6 @@ begin
 rescue StandardError => e
   raise e if $DEBUG
 
-  warn "wurkswarm: #{e.message}"
-  warn e.backtrace.join("\n")
+  Wurk::CLI.report_failure('wurkswarm', e)
   exit 1
 end

data/lib/generators/wurk/install/install_generator.rb

@@ -17,6 +17,18 @@ module Wurk
       def insert_mount_line
         route(%(# mount Wurk::Engine => "/wurk"  # uncomment to expose the Wurk dashboard))
       end
+
+      def show_next_steps
+        say <<~STEPS
+
+          Next steps:
+            1. Set the queue adapter if you use Active Job: config.active_job.queue_adapter = :wurk
+            2. Start workers: bundle exec wurkswarm (or bundle exec wurk for one process)
+            3. To use the dashboard, uncomment the mount line in config/routes.rb.
+               It has no login of its own, so put it behind authentication first:
+               docs/authentication.md in the wurk gem's repository
+        STEPS
+      end
     end
   end
 end

data/lib/sidekiq/web.rb

@@ -2,3 +2,8 @@
 
 # Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
 require 'wurk'
+
+# Extension renderer dependencies, deferred on the worker path (web/extension.rb).
+# Loaded here for a helper called outside a request, as unique-jobs' specs do.
+require 'cgi'
+require 'erb'

data/lib/wurk/boot_checks.rb

@@ -11,6 +11,8 @@ module Wurk
     # Oldest version each non-Redis backend has been run against.
     TESTED_FLOORS = { 'dragonfly' => '1.34.0', 'valkey' => '8.0.0' }.freeze
     MAX_LISTED = 20
+    SUBCLASS = ::Module.instance_method(:<)
+    NAME = ::Module.instance_method(:name)
 
     module_function
 
@@ -71,11 +73,15 @@ module Wurk
       end
     end
 
+    # A walk over every class meets classes that redefine these with other arguments
+    # (Faker::Travel::Airport.name takes required keywords), and one such class must
+    # not cost the app this check.
     def locked_job_classes
       ::ObjectSpace.each_object(::Class).filter_map do |klass|
-        next unless klass.name && klass < Wurk::Worker
+        next unless SUBCLASS.bind_call(klass, Wurk::Worker)
 
-        klass.name if klass.get_sidekiq_options.key?('lock')
+        name = NAME.bind_call(klass)
+        name if name && klass.get_sidekiq_options.key?('lock')
       end.sort
     end
 

data/lib/wurk/cli.rb

@@ -6,6 +6,7 @@
 # a Rails app that enqueues and processes jobs pays that on every boot for a
 # command-line parser it never runs. `require` is idempotent, so the lazy call
 # is a $LOADED_FEATURES hash lookup on every invocation after the first.
+require 'rbconfig'
 require_relative 'version'
 require_relative 'configuration'
 require_relative 'component'
@@ -32,6 +33,14 @@ module Wurk
     # only one the swarm binaries accept — so a subcommand is never anything
     # but the first argument, and every argument after it is that command's.
     COMMANDS = %w[api health].freeze
+    # Set only across the one re-exec of #load_rails_framework, and removed on arrival.
+    PRELOAD_FIRST_ENV = 'WURK_BOOT_PRELOAD_FIRST'
+    # `bundle exec` loads the binstub in-process, so argv0 is Bundler's executable, and
+    # `ruby -e "load Gem.bin_path(...)"` has `-e` there: the running program can't name it.
+    SWARM_EXECUTABLE = ::File.expand_path('../../exe/wurkswarm', __dir__)
+
+    # A refusal this file raises itself, with nothing a trace would add.
+    class StartupError < ArgumentError; end
 
     # `wurk api` defaults. 0.0.0.0 because the point of the machine API is for
     # another service to reach it; it is bearer-gated, and answers 404 to
@@ -108,7 +117,15 @@ module Wurk
       ['-s', '--strict', :health_strict, 'Fail when Redis cannot be read, rather than passing']
     ].freeze
 
-    HELP_COMMANDS = "\nCommands:\n    health    Exit 0 if this host's workers are heartbeating\n\nOptions:"
+    HELP_COMMANDS = "\nCommands:\n    " \
+                    "health    Exit 0 if this host's workers are heartbeating\n    " \
+                    "preflight Read a live Redis: what will carry over to Wurk (read-only)\n    " \
+                    "verify    Post-cutover check of a live fleet (read-only)\n\nOptions:"
+    HELP_API_NOTE = "\nThe worker flags -c, -g, -q and -t are accepted here and have no effect.\n\nOptions:"
+    HELP_SWARM_NOTE = "\nForks WURK_COUNT worker processes from a preloaded parent and supervises them.\n\nOptions:"
+    HELP_FOOTER = "\nEnvironment:\n    REDIS_URL         Redis to use (default redis://localhost:6379/0)\n    " \
+                  "WURK_COUNT        Worker processes for wurkswarm\n\n" \
+                  'Docs: https://github.com/developerz-ai/wurk#readme'
 
     attr_accessor :launcher, :environment, :config
 
@@ -146,6 +163,32 @@ module Wurk
       self
     end
 
+    # What the binaries print for a failed start. Only errors this file fully
+    # understands drop the trace. Anything else may be a bug in the host app, whose
+    # file and line are the point, and asking for -v means changing a deployed command.
+    def self.report_failure(program, error, argv: ARGV)
+      warn "#{program}: #{error.message}"
+      hint = failure_hint(error)
+      warn "#{program}: #{hint}" if hint
+      return unless error.backtrace
+      return unless failure_trace_wanted?(error, hint) || argv.intersect?(%w[-v --verbose])
+
+      warn error.backtrace.join("\n")
+    end
+
+    def self.failure_trace_wanted?(error, hint) = hint.nil? && !error.is_a?(StartupError)
+    private_class_method :failure_trace_wanted?
+
+    def self.failure_hint(error)
+      case error
+      when ::RedisClient::ConnectionError
+        'could not reach Redis; check REDIS_URL'
+      when ->(e) { defined?(::OptionParser::ParseError) && e.is_a?(::OptionParser::ParseError) }
+        'see --help'
+      end
+    end
+    private_class_method :failure_hint
+
     # `boot_app:` / `warmup:` are test seams. Production always passes true.
     def run(boot_app: true, warmup: true) # rubocop:disable Metrics/AbcSize
       exit(run_health) if @command == 'health'
@@ -159,6 +202,7 @@ module Wurk
       # false`).
       enter_server_mode
       boot_application if boot_app
+      Swarm::TestModeGuard.enforce!(@config.logger)
       self_read, self_write = ::IO.pipe
       begin
         trap_signals(self_write)
@@ -192,7 +236,7 @@ module Wurk
     # `SIDEKIQ_PRELOAD_APP` whole-app eager-load) and boots `Process.warmup`
     # before the fork so children share warmed pages (copy-on-write). Spec §7.
     def run_swarm(boot_app: true, warmup: true) # rubocop:disable Metrics/AbcSize
-      raise ArgumentError, "the swarm runner takes no subcommand; run `wurk #{@command}` instead" if @command
+      raise StartupError, "the swarm runner takes no subcommand; run `wurk #{@command}` instead" if @command
 
       # Server mode before the app loads — see #run. The flag rides through the
       # fork into every child (the config object is copied), so configure_server
@@ -254,7 +298,7 @@ module Wurk
       # Load the app here, not on the first request: a broken load should fail
       # the boot an operator is watching, not the first client to call.
       require_relative 'api/app'
-      raise ArgumentError, NO_API_TOKEN_MESSAGE unless @config.api_enabled?
+      raise StartupError, NO_API_TOKEN_MESSAGE unless @config.api_enabled?
 
       handler ||= api_handler(@config[:api_server])
       logger.info { "Wurk API listening on http://#{api_bind}:#{api_port}#{Wurk::API::VERSION_PREFIX}" }
@@ -301,8 +345,8 @@ module Wurk
       registry = handler_registry
       name ? registry.get(name) : registry.default
     rescue ::LoadError, ::NameError, ::ArgumentError => e
-      raise ArgumentError, "wurk api could not start a web server (#{e.message}). Add one to your Gemfile " \
-                           '— puma, falcon, or rackup + webrick — or name an installed one with --server.'
+      raise StartupError, "wurk api could not start a web server (#{e.message}). Add one to your Gemfile " \
+                          '— puma, falcon, or rackup + webrick — or name an installed one with --server.'
     end
 
     def handler_registry
@@ -354,7 +398,8 @@ module Wurk
     end
 
     def signal_names
-      %w[INT TERM TSTP TTIN INFO USR2]
+      # INFO exists on BSD and macOS only; asking for it on Linux just warned at every boot.
+      %w[INT TERM TSTP TTIN INFO USR2].select { |name| ::Signal.list.key?(name) }
     end
 
     def reopen_logs
@@ -389,7 +434,7 @@ module Wurk
 
     def validate_pool_sizes!
       @config.capsules.each_pair do |name, cap|
-        raise ArgumentError, "Pool size too small for #{name}" if cap.redis_pool.size < cap.concurrency
+        raise StartupError, "Pool size too small for #{name}" if cap.redis_pool.size < cap.concurrency
       end
     end
 
@@ -455,7 +500,7 @@ module Wurk
     end
 
     def parse_config(path)
-      raise ArgumentError, "No such file #{path}" unless ::File.exist?(path)
+      raise StartupError, "No such file #{path}" unless ::File.exist?(path)
 
       require 'erb'
       require 'yaml'
@@ -490,6 +535,7 @@ module Wurk
       parser = option_parser(opts)
       @parser = parser
       parser.parse!(argv)
+      warn "wurk: ignoring #{argv.join(' ')}: not a command or a flag (see --help)" unless argv.empty?
       opts
     end
 
@@ -497,17 +543,37 @@ module Wurk
       require 'optparse'
 
       ::OptionParser.new do |o|
-        o.banner = @command ? "wurk #{@command} [options]" : "wurk [options]\nwurk COMMAND [options]"
-        o.separator HELP_COMMANDS unless @command
+        o.banner = help_banner
+        o.separator help_intro if help_intro
         define_value_flags(o, opts, @command == 'health' ? HEALTH_OPTION_FLAGS : OPTION_FLAGS)
         define_value_flags(o, opts, API_OPTION_FLAGS) if @command == 'api'
         define_meta_flags(o, opts)
       end
     end
 
+    def swarm_binary? = ::File.basename($PROGRAM_NAME).include?('swarm')
+
+    def help_banner
+      return "wurk #{@command} [options]" if @command
+      return "#{::File.basename($PROGRAM_NAME)} [options]" if swarm_binary?
+
+      "wurk [options]\nwurk COMMAND [options]"
+    end
+
+    def help_intro
+      return HELP_API_NOTE if @command == 'api'
+      return if @command
+
+      swarm_binary? ? HELP_SWARM_NOTE : HELP_COMMANDS
+    end
+
     def define_value_flags(parser, opts, flags)
       flags.each do |short, long, key, desc, transform|
-        parser.on(short, long, desc) { |arg| assign_flag(opts, key, arg, transform) }
+        parser.on(short, long, desc) do |arg|
+          assign_flag(opts, key, arg, transform)
+        rescue ArgumentError
+          raise StartupError, "#{long.split.first}: #{arg.inspect} is not a number"
+        end
       end
     end
 
@@ -525,26 +591,62 @@ module Wurk
         exit(0)
       end
       parser.on_tail('-h', '--help', 'Show help') do
-        puts parser
+        puts parser, HELP_FOOTER
         exit(0)
       end
     end
 
     # --- application bootstrap --------------------------------------------
 
-    # A Rails app expects Rails to be loaded before its gems: `config/application.rb`
-    # requires the framework and only then calls `Bundler.require`, and gems read
-    # `defined?(Rails)` or reference ActiveModel while loading. Requiring the groups
-    # first broke those gems in this boot path only. So for a Rails target, run
-    # the app's own first half before the group preload.
+    # A Rails target has two shapes that need opposite orders. A gem may need Rails
+    # loaded while it requires: `config/application.rb` requires the framework and
+    # only then calls `Bundler.require`, so running the app's first half before the
+    # group preload suits it. An app may instead use a gem it expects the swarm to have
+    # preloaded, as stock sidekiqswarm's order (spec §7.3) gives it, and then that
+    # file raises NameError here. Only the file can tell us which it is, and by then
+    # its class body may have half-run, registering initializers and middleware that a
+    # second load in this process would register again. So the retry is a fresh process:
+    # same pid, same environment as before the attempt, preload first. Once only, so a
+    # genuine bug surfaces with its own message.
     def load_rails_framework
       root = rails_app_root(@config[:require])
       return unless root
 
+      restart = restart_state
+      first_error = ENV.delete(PRELOAD_FIRST_ENV)
       ENV['RACK_ENV'] = ENV['RAILS_ENV'] = @environment
       require 'rails'
       define_active_job_adapter
-      require ::File.join(root, 'config/application.rb')
+      return note_preload_first(first_error) if first_error
+
+      begin
+        require ::File.join(root, 'config/application.rb')
+      rescue ::NameError => e
+        restart_with_preload_first(restart, e)
+      end
+    end
+
+    def restart_state
+      { env: ::ENV.to_h, argv: ::ARGV.dup, dir: ::Dir.pwd }
+    end
+
+    def note_preload_first(first_error)
+      logger.warn do
+        "config/application.rb raised before the preload (#{first_error}), so this boot preloads first. " \
+          'If it fails too, that first error may be the one to fix.'
+      end
+    end
+
+    # Bundler may have arrived only inside the failed attempt (`config/boot.rb`), and the
+    # snapshot predates it, so the fresh process is told about it explicitly. If the
+    # exec itself fails, the app's error is still the one worth reporting.
+    def restart_with_preload_first(state, error)
+      env = state[:env].merge(PRELOAD_FIRST_ENV => error.message.lines.first.to_s.strip[0, 300])
+      env['BUNDLE_GEMFILE'] ||= ::Bundler.default_gemfile.to_s if defined?(::Bundler)
+      ruby = [::RbConfig.ruby, *('-rbundler/setup' if defined?(::Bundler))]
+      ::Process.exec(env, *ruby, SWARM_EXECUTABLE, *state[:argv], unsetenv_others: true, chdir: state[:dir])
+    rescue ::SystemCallError
+      raise error
     end
 
     # The app root for a `-r` that is the app directory or its environment file.
@@ -559,7 +661,8 @@ module Wurk
     # copy-on-write into every child. Comma-separated group list; the native
     # `WURK_PRELOAD` wins over the `SIDEKIQ_PRELOAD` drop-in alias. Defaults to
     # the `default` group; an explicit empty value (`WURK_PRELOAD=`) disables
-    # the preload. Runs before `boot_application` so groups load before the app.
+    # the preload. Runs before `boot_application`, so before the initializers; see
+    # #load_rails_framework for where `config/application.rb` falls.
     def preload_bundler_groups
       return unless defined?(::Bundler)
 
@@ -641,23 +744,29 @@ module Wurk
     end
 
     def validate!
-      require_path = @config[:require]
-      bad_path = !::File.exist?(require_path.to_s)
-      bad_rails = ::File.directory?(require_path.to_s) &&
-                  !::File.exist?(::File.join(require_path, 'config/application.rb'))
-      return print_help_and_die if bad_path || bad_rails
+      reason = require_path_problem(@config[:require].to_s)
+      return print_help_and_die(reason) if reason
 
       %i[concurrency timeout].each do |opt|
-        raise ArgumentError, "#{opt}: #{@config[opt]} is not a valid value" if @config[opt].to_i <= 0
+        raise StartupError, "#{opt}: #{@config[opt]} is not a valid value" if @config[opt].to_i <= 0
       end
     end
 
-    def print_help_and_die
-      logger.info '=================================================================='
-      logger.info '  Please point Wurk to a Rails application or a Ruby file'
-      logger.info '  to load your job classes with -r [DIR|FILE].'
-      logger.info '=================================================================='
-      logger.info @parser
+    def require_path_problem(path)
+      return 'no such file or directory' unless ::File.exist?(path)
+
+      return unless ::File.directory?(path) && !::File.exist?(::File.join(path, 'config/application.rb'))
+
+      'it has no config/application.rb'
+    end
+
+    def print_help_and_die(reason)
+      logger.error '=================================================================='
+      logger.error "  Wurk cannot load job classes from #{@config[:require]}: #{reason}."
+      logger.error '  Please point Wurk to a Rails application or a Ruby file'
+      logger.error '  to load your job classes with -r [DIR|FILE].'
+      logger.error '=================================================================='
+      logger.error @parser.to_s
       exit(1)
     end
   end

data/lib/wurk/client.rb

@@ -57,6 +57,9 @@ module Wurk
     # costs one lookup per write phase.
     WRITE_STATE_KEY = :wurk_client_write_state
 
+    NO_ARGS = [].freeze
+    private_constant :NO_ARGS
+
     attr_accessor :redis_pool
 
     def initialize(pool: nil, config: nil, chain: nil)
@@ -241,12 +244,20 @@ module Wurk
       jids
     end
 
+    # Only `args` and `jid` differ between the jobs of a slice, so normalizing
+    # runs once and each job is a merge over the result. A per-job `at` keeps
+    # the full walk: the expiry and deadline stamps are measured from it.
     def build_bulk_payloads(slice, base, ats)
-      slice.each_with_index.map do |job_args, idx|
-        item = base.merge('args' => job_args)
-        item['at'] = ats[idx] if ats
-        normed = normalize_item(item)
-        invoke_chain_verified(normed)
+      if ats
+        return slice.each_with_index.map do |job_args, idx|
+          invoke_chain_verified(normalize_item(base.merge('args' => job_args, 'at' => ats[idx])))
+        end
+      end
+
+      template = normalize_item(base.merge('args' => NO_ARGS))
+      explicit_jid = base['jid']
+      slice.map do |job_args|
+        invoke_chain_verified(template.merge('args' => job_args, 'jid' => explicit_jid || SecureRandom.hex(12)))
       end
     end
 

data/lib/wurk/compat.rb

@@ -119,6 +119,8 @@ module Sidekiq
   private_constant :RailsNamespace
 
   BasicFetch       = Wurk::Fetcher::Reliable
+  WebHelpers       = Wurk::Web::Extension::Helpers
+  Paginator        = Wurk::Web::Paginator
   Batch            = Wurk::Batch
   BatchSet         = Wurk::BatchSet
   Capsule          = Wurk::Capsule
@@ -235,6 +237,8 @@ module Sidekiq
       name == :Rails ? RailsNamespace : super
     end
 
+    def gem_version = Gem::Version.new(VERSION)
+    def freeze! = Wurk.default_configuration.freeze!
     def strict_args!(mode = :raise) = Wurk.strict_args!(mode)
     def transactional_push! = Wurk.transactional_push!
     def testing!(mode = :fake, &) = Wurk.testing!(mode, &)