wurk 1.7.6 → 1.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d232f2e0c78233c2bbd71051a07df6e791d1fb1c0d701c9e3ff0820035207077
-  data.tar.gz: 06d15b628db2d8ab75ab8440b48f372ea0943108a0c19533db0c2cf82a1b74fe
+  metadata.gz: 3d5734527cfa0b903c07b143f7eb0b821fca970cd98a8773f16cbdc12dcbb43c
+  data.tar.gz: e3e610d19a29b71e071896a45710a807928123bb0c530348fb809dafb101cfe3
 SHA512:
-  metadata.gz: 522ee07752a14d58635f770411bf7d74d53fca26141f04f73e0f6596e27374ea580b11d405cdcaf297c53aeaca3e08f06e1409b876159fea34cd6c91bbbcbf89
-  data.tar.gz: 1ffc5b66a6ce06ee39353e3593296f22ac99f8851000fa1b78fcbfb30ff7617a742804fdf6584a07b9a4b039477b736b26c644e8844c8f420766fb21670fc287
+  metadata.gz: c08daf7f94145029bcfd74e49c4426c8881effdce3aa65329453c5e7c2930ed87d93ac6ea85423ea8ee6a9df3465152837ebac692abfff3376623e69198209bc
+  data.tar.gz: a8374380a15e20e599374bdd7afe6e911a3364b6f1acb532fca2ece894d31d8a113baf4375773a7b94ae2fb0a4a1da98767ca16667b705a53c8f58fdde9ad9d0

data/README.md

@@ -13,7 +13,7 @@
 [![Live Demo](https://img.shields.io/badge/live%20demo-wurk.demo.developerz.ai-22c55e?logo=googlechrome&logoColor=white)](https://wurk.demo.developerz.ai/wurk)
 [![Gem Version](https://img.shields.io/gem/v/wurk.svg)](https://rubygems.org/gems/wurk)
 [![CI](https://github.com/developerz-ai/wurk/actions/workflows/test.yml/badge.svg)](https://github.com/developerz-ai/wurk/actions/workflows/test.yml)
-[![Coverage gate](https://img.shields.io/badge/coverage%20gate-line%20%E2%89%A590%25-brightgreen.svg)](https://github.com/developerz-ai/wurk/actions/workflows/test.yml)
+[![Coverage gate](https://img.shields.io/badge/coverage%20gate-line%20%2B%20branch%20%E2%89%A590%25-brightgreen.svg)](https://github.com/developerz-ai/wurk/actions/workflows/test.yml)
 [![Ruby](https://img.shields.io/badge/ruby-%E2%89%A5%203.2-CC342D.svg)](https://www.ruby-lang.org)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
@@ -21,7 +21,7 @@
 
 Wurk is wire-compatible with Sidekiq — same Redis keys, same job JSON, same Ruby DSL. Swap one line in your `Gemfile` and your existing jobs, batches, limiters, cron entries, and live Redis data keep working untouched. The Pro and Enterprise feature sets ship in the same free gem, with no license check and no tiers.
 
-**In production:** Wurk runs the background work at [developerz.ai](https://developerz.ai) and at partner deployments — millions of jobs an hour, across many servers, on the fork-based swarm described below. It is not a preview.
+**In production:** Wurk runs the background work at [developerz.ai](https://developerz.ai) and at partner deployments, across many servers, on the fork-based swarm described below. It is not a preview. Throughput numbers from those deployments are not published yet; what has been measured is in [docs/benchmarks.md](docs/benchmarks.md).
 
 **At scale:** Wurk is built for fleets, not just for one box. Kubernetes `/live` + `/ready` probes are a config line, not a sidecar; a bearer-scoped [HTTP API](docs/api-http.md) lets non-Ruby services enqueue and inspect; [OpenTelemetry](docs/telemetry.md) traces propagate client → server; per-queue [global concurrency caps](docs/rate-limiting.md) hold cluster-wide limits; and monitoring is the dashboard you already mount — live SSE, charts, per-job progress, no separate stack to run. See [Wurk extras](#wurk-extras).
 
@@ -136,6 +136,7 @@ Method, per-invocation records, workload definitions, and the separate `rake ben
 - **[Starting the worker](https://github.com/developerz-ai/wurk/blob/main/docs/running.md)** — Rails auto-start, the `wurk`/`wurkswarm` runners, and running standalone without Rails.
 - **[Configuration reference](https://github.com/developerz-ai/wurk/blob/main/docs/configuration.md)** — every option, env var, YAML key, and CLI flag, with precedence and pool sizing.
 - **[Deploying](https://github.com/developerz-ai/wurk/blob/main/docs/deployment.md)** — systemd, Capistrano, Heroku, Docker, Kubernetes, rolling restarts, memory limits.
+- **[Incident runbook](https://github.com/developerz-ai/wurk/blob/main/docs/runbook.md)** — symptom → check → fix for stuck queues, orphaned private lists, latency spikes, a stuck cron leader, poison-job storms, Redis failover, memory growth, stuck batches, dead-set growth, and rollback to Sidekiq.
 - **[Secrets & credentials](https://github.com/developerz-ai/wurk/blob/main/docs/secrets.md)** — which values are secret vs config, how to supply them (ENV, Rails credentials, an init file), precedence, and what to never commit.
 - **[Active Job adapter](https://github.com/developerz-ai/wurk/blob/main/docs/active-job.md)** — run `ActiveJob`/`deliver_later` on Wurk with `queue_adapter = :wurk`.
 - **[Testing jobs](https://github.com/developerz-ai/wurk/blob/main/docs/testing.md)** — fake/inline modes, the jobs array, Minitest and RSpec setup.
@@ -297,9 +298,11 @@ Wurk is MIT and stays that way. If what you need is a commercial support contrac
 + gem "wurk"
 ```
 
-`bundle install && restart`. Wurk reads and writes the same Redis schema, so a rolling deploy can run Sidekiq and Wurk against the same Redis during the cutover. sidekiq-cron's upstream suite runs against Wurk in the [`ecosystem` CI job](https://github.com/developerz-ai/wurk/blob/main/.github/workflows/ecosystem.yml) (see [`test/ecosystem/`](https://github.com/developerz-ai/wurk/tree/main/test/ecosystem)). sidekiq-unique-jobs, sidekiq-scheduler, sidekiq-status, sidekiq-failures and sidekiq-throttled are target additions tracked in [docs/idea/14-ecosystem-compat.md](https://github.com/developerz-ai/wurk/blob/main/docs/idea/14-ecosystem-compat.md).
+`bundle install && restart`. Wurk reads and writes the same Redis schema, so there is no data migration: Wurk picks up the queues, schedules, retries, batches and periodic loops Sidekiq left behind. Running Sidekiq and Wurk workers against one Redis *at the same time* is not tested yet, so a live production move is a **drain cutover** (stop Sidekiq's workers, confirm nothing is in flight, start Wurk), with a matching rollback; the [production cutover](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md#9-production-cutover) section has every step and command. sidekiq-cron's upstream suite runs against Wurk in the [`ecosystem` CI job](https://github.com/developerz-ai/wurk/blob/main/.github/workflows/ecosystem.yml) (see [`test/ecosystem/`](https://github.com/developerz-ai/wurk/tree/main/test/ecosystem)). sidekiq-unique-jobs, sidekiq-scheduler, sidekiq-status, sidekiq-failures and sidekiq-throttled are target additions tracked in [docs/idea/14-ecosystem-compat.md](https://github.com/developerz-ai/wurk/blob/main/docs/idea/14-ecosystem-compat.md).
 
-Full walkthrough — config side-by-side, the Redis key/`sidekiq_options` mapping, known incompatibilities, and a one-page cutover checklist: **[docs/migrate-from-sidekiq.md](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md)**.
+**Keeping a `sidekiq-*` add-on gem?** Those gems declare `add_dependency "sidekiq"`, which would reinstall real Sidekiq next to Wurk. Satisfy it with Wurk's git-only [`sidekiq` shim gem](https://github.com/developerz-ai/wurk/tree/main/ecosystem/sidekiq-shim) instead: `gem "sidekiq", github: "developerz-ai/wurk", glob: "ecosystem/sidekiq-shim/*.gemspec"`. Only sidekiq-cron is exercised that way on every PR; treat the others as untested.
+
+Full walkthrough — config side-by-side, the Redis key/`sidekiq_options` mapping, known incompatibilities, moving sidekiq-cron schedules over (`rake wurk:import:cron`), and the production cutover and rollback playbook: **[docs/migrate-from-sidekiq.md](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md)**. On the night, keep the **[incident runbook](https://github.com/developerz-ai/wurk/blob/main/docs/runbook.md)** open.
 
 ## Also from developerz.ai: Ultimate
 

data/app/controllers/concerns/wurk/sse_streaming.rb

@@ -30,7 +30,8 @@ module Wurk
       # Headers are already flushed by the time a tick hits Redis, so the
       # 503 ApplicationController's rescue_from renders for a plain request
       # can't happen here — emit a structured SSE event instead and close;
-      # the SPA's EventSource reconnects and gets a fresh chance at Redis.
+      # the closed stream is reopened by EventSource after the `retry:`
+      # interval and gets a fresh chance at Redis.
       logger.warn("wurk web: stream redis unavailable (#{e.class}: #{e.message})")
       sse.write({ error: 'redis_unavailable' }, event: 'error')
     ensure

data/app/controllers/concerns/wurk/stream_concurrency_guard.rb

@@ -2,35 +2,48 @@
 
 module Wurk
   # Per-process cap on concurrent SSE streams. ActionController::Live pins one
-  # Puma thread for every open `/api/stream`; without a bound, a burst of stale
-  # browser tabs could hold every thread and starve the JSON API. Past the cap
-  # we 503 with Retry-After — the SPA's EventSource reconnects (and its polling
-  # fallback honors Retry-After) once a slot frees. Per-process is the right
-  # scope: it's this process's own thread pool we're protecting.
+  # server thread for every open `/api/stream`; the cap is half the server's
+  # request threads (at least one) so streams can never take the threads the
+  # host app's own requests need. A fixed cap of 10 sat above Rails' default of
+  # 3 Puma threads, so three dashboard tabs hung the host app for the stream's
+  # whole lifetime. Past the cap we 503 (Retry-After is set for non-browser
+  # clients; EventSource can't read it, so the SPA backs off on its own from 3s,
+  # doubling to 30s).
+  # Per-process is the right scope: it's this process's own thread pool we're
+  # protecting. `config.web.max_streams` overrides the derived cap.
   #
   # Slots are held as thread references rather than tallied in a counter so the
   # cap can heal itself. A stream whose thread is killed mid-flight never
   # reaches the `ensure` below (Puma hard-reaps worker threads past
   # `force_shutdown_after`, and a thread killed inside an uninterruptible read
   # can skip its ensure), which a counter would record as a slot held by nobody
-  # — ten of those and `/api/stream` 503s for the life of the process. A dead
-  # holder is instead evicted by the next acquire.
+  # — a cap's worth of those and `/api/stream` 503s for the life of the
+  # process. A dead holder is instead evicted by the next acquire.
   module StreamConcurrencyGuard
     extend ActiveSupport::Concern
 
-    MAX_CONCURRENT_STREAMS = 10
     RETRY_AFTER_SECONDS = 3
+    # Rails' generated puma.rb default (`RAILS_MAX_THREADS || 3`), used when
+    # neither Puma nor the environment says how many threads serve requests.
+    DEFAULT_SERVER_THREADS = 3
 
     @holders = []
     @lock = Mutex.new
 
     class << self
+      def max_streams
+        configured = ::Wurk::Web.config.max_streams
+        return Integer(configured) if configured
+
+        [server_threads / 2, 1].max
+      end
+
       # Reserve a stream slot for the calling thread; false when the cap is
       # already reached by threads that are still alive.
       def acquire
         @lock.synchronize do
           @holders.keep_if(&:alive?)
-          return false if @holders.size >= MAX_CONCURRENT_STREAMS
+          return false if @holders.size >= max_streams
 
           @holders << Thread.current
           true
@@ -47,6 +60,17 @@ module Wurk
           @holders.delete_at(index) if index
         end
       end
+
+      private
+
+      # Puma's own max_threads when it booted through its launcher (`puma`,
+      # `rails s`), else RAILS_MAX_THREADS — what the generated puma.rb reads.
+      def server_threads
+        puma = ::Puma.cli_config&.options&.[](:max_threads) if defined?(::Puma) && ::Puma.respond_to?(:cli_config)
+        Integer(puma || ENV.fetch('RAILS_MAX_THREADS', DEFAULT_SERVER_THREADS))
+      rescue ::ArgumentError, ::TypeError
+        DEFAULT_SERVER_THREADS
+      end
     end
 
     private

data/app/controllers/wurk/api/pagination.rb

@@ -34,6 +34,18 @@ module Wurk
         }
       end
 
+      # The paging fields every listing answers with. `page` is the page
+      # actually served (an out-of-range ?page= is clamped, so the SPA must
+      # adopt it) and `max_page` the deepest page the server will walk to.
+      # A filtered slice adds `filtered_total` — matching rows found — and
+      # `filtered_total_exact`; when false the count is a lower bound.
+      def meta(page)
+        fields = { page: page[:page], count: page[:count], max_page: MAX_PAGE }
+        return fields unless page.key?(:matched)
+
+        fields.merge(filtered_total: page[:matched], filtered_total_exact: page[:exhausted])
+      end
+
       def clamp_int(value, min, max, default)
         Integer(value, 10).clamp(min, max)
       rescue ::ArgumentError, ::TypeError
@@ -85,26 +97,32 @@ module Wurk
 
       # Filtered path: the offset counts MATCHING rows, so page boundaries
       # stay stable (raw-index offsets re-emit or skip matches across pages).
+      # Records the matches seen in `page[:matched]` and whether the set was
+      # walked to its end in `page[:exhausted]`: stopping once the page fills
+      # is what keeps a filtered request cheap, so the count is only exact
+      # when nothing was left unread.
       def slice_filtered(enumerable, offset, page)
         results = []
         matched = 0
         examined = 0
         enumerable.each do |member|
-          examined += 1
-          break if examined > FILTER_SCAN_LIMIT
+          break if (examined += 1) > FILTER_SCAN_LIMIT
 
           payload = yield(member)
           next unless payload && match?(payload, page[:substr])
 
-          if matched >= offset
-            results << payload
-            break if results.size >= page[:count]
-          end
-          matched += 1
+          results << payload if (matched += 1) > offset
+          break if results.size >= page[:count]
         end
+        record_match_count(page, matched, examined, results.size)
         results
       end
 
+      def record_match_count(page, matched, examined, served)
+        page[:matched] = matched
+        page[:exhausted] = examined <= FILTER_SCAN_LIMIT && served < page[:count]
+      end
+
       def match?(payload, substr)
         return true if substr.nil? || substr.empty?
 

data/app/controllers/wurk/api_controller.rb

@@ -26,7 +26,9 @@ module Wurk
     include SseStreaming
 
     STREAM_TICK_SECONDS = 2.0
-    STREAM_MAX_DURATION = 120.0
+    # Short so a capped slot rotates: a tab parked on the dashboard gives its
+    # thread back every half-minute and has to win a slot again to reopen.
+    STREAM_MAX_DURATION = 30.0
     # Positive floor for `?tick=`: a zero/negative tick would `sleep 0` the SSE
     # loop into a tight Redis-read/write spin (Live runs it in a spawned thread)
     # for up to STREAM_MAX_DURATION. Clamp before it reaches drive_stream.
@@ -69,7 +71,7 @@ module Wurk
       jobs = ::Wurk::Api::Pagination.slice(q, page) { |rec| ::Wurk::Api::Serializers.job_record(rec) }
       render json: {
         name: q.name, size: q.size, latency: q.latency, paused: q.paused?,
-        page: page[:page], count: page[:count], jobs: jobs
+        **::Wurk::Api::Pagination.meta(page), jobs: jobs
       }
     end
 
@@ -83,7 +85,7 @@ module Wurk
     # (Fast::QueueExt#delete_job) instead of a paged find_job walk.
     def delete_queue_job
       removed = ::Wurk::Queue.new(params[:name].to_s).delete_job(params[:jid].to_s)
-      return render(json: { error: 'unknown job' }, status: :not_found) if removed.zero?
+      return render(json: { error: 'unknown job' }, status: :not_found) if removed.nil?
 
       render json: { ok: true, deleted: true }
     end
@@ -125,12 +127,11 @@ module Wurk
 
     def scheduled_all
       set = ::Wurk::ScheduledSet.new
-      count = case params[:cmd].to_s
-              when 'delete'       then clear_set(set)
-              when 'add_to_queue' then drain_set(set, :add_to_queue)
-              else return render(json: { error: 'unknown action' }, status: :bad_request)
-              end
-      render json: { ok: true, count: count }
+      case params[:cmd].to_s
+      when 'delete'       then render json: { ok: true, count: clear_set(set) }
+      when 'add_to_queue' then drain_set(set, :add_to_queue)
+      else render(json: { error: 'unknown action' }, status: :bad_request)
+      end
     end
 
     # --- Dead set mutations --------------------------------------------------
@@ -170,7 +171,7 @@ module Wurk
       set = ::Wurk::BatchSet.new
       page = ::Wurk::Api::Pagination.window(params)
       rows = ::Wurk::Api::Pagination.slice(set, page) { |status| status.data.transform_keys(&:to_sym) }
-      render json: { total: set.size, page: page[:page], count: page[:count], batches: rows }
+      render json: { total: set.size, **::Wurk::Api::Pagination.meta(page), batches: rows }
     end
 
     def batch
@@ -186,7 +187,7 @@ module Wurk
       set = ::Wurk::FlowSet.new
       page = ::Wurk::Api::Pagination.window(params)
       rows = ::Wurk::Api::Pagination.slice(set, page) { |status| ::Wurk::Api::Serializers.flow_row(status) }
-      render json: { total: set.size, page: page[:page], count: page[:count], flows: rows }
+      render json: { total: set.size, **::Wurk::Api::Pagination.meta(page), flows: rows }
     end
 
     def flow
@@ -210,7 +211,7 @@ module Wurk
     def limiters
       names = ::Wurk::Web::Enterprise::Limits.list(filter: params[:substr])
       page = ::Wurk::Api::Pagination.window(params)
-      render json: { total: names.size, page: page[:page], count: page[:count], limiters: limiter_rows(names, page) }
+      render json: { total: names.size, **::Wurk::Api::Pagination.meta(page), limiters: limiter_rows(names, page) }
     end
 
     def reset_limiter
@@ -312,9 +313,12 @@ module Wurk
 
     # SSE: one `event: stats` per tick with a fresh Stats snapshot. Caps at
     # `STREAM_MAX_DURATION` so a stale browser tab can't tie a Rails worker
-    # forever — the client reconnects automatically when the stream closes.
-    # Bounded per process by StreamConcurrencyGuard (503 + Retry-After past the
-    # cap) so a burst of tabs can't pin every Puma thread.
+    # forever; a stream that closes normally is reopened by EventSource after
+    # the `retry:` interval sent below. Bounded per process by
+    # StreamConcurrencyGuard so a burst of tabs can't pin every Puma thread.
+    # Past the cap the answer is a 503, which EventSource treats as fatal and
+    # whose Retry-After it cannot read: the SPA reopens it itself, backing off
+    # from 3s and doubling to 30s (frontend/src/hooks/useSSE.ts).
     #
     # `?max_duration=` and `?tick=` are test/debug knobs; the SPA never sets
     # them. `?max_duration=0` emits one tick and closes.
@@ -373,27 +377,48 @@ module Wurk
       entries = entries_for(set, [params[:key]])
       return render(json: { error: 'unknown job' }, status: :not_found) if entries.empty?
 
-      entries.each { |entry| entry.public_send(method) }
-      render json: { ok: true, count: entries.size }
+      render_applied(entries.size, apply_each(entries, method))
     end
 
     # Bulk variant: `keys[]` + a single `cmd` applied to every resolved entry.
-    def bulk_entry_action(set, actions) # rubocop:disable Metrics/AbcSize
+    def bulk_entry_action(set, actions)
       method = actions[params[:cmd].to_s]
       return render(json: { error: 'unknown action' }, status: :bad_request) unless method
 
-      count = 0
       keys = Array(params[:keys]).map(&:to_s).uniq
       # Each key costs a ZRANGEBYSCORE plus per-entry mutations; an uncapped
       # list lets one POST issue tens of thousands of round-trips. The UI
       # selects at most a page (200).
       return render(json: { error: 'too many keys (max 1000)' }, status: :bad_request) if keys.size > 1000
 
-      entries_for(set, keys).each do |entry|
+      entries = entries_for(set, keys)
+      render_applied(entries.size, apply_each(entries, method))
+    end
+
+    # One entry the client rejects (a payload failing validation, a raising
+    # middleware) must not 500 the request with every later entry unapplied.
+    # SortedEntry has already put a failed entry back in its set, so the
+    # failure is reported per key and the loop moves on. A Redis error is not
+    # per-entry — it propagates to the 503 handler instead of failing every
+    # remaining key one by one.
+    def apply_each(entries, method)
+      entries.each_with_object([]) do |entry, failures|
         entry.public_send(method)
-        count += 1
+      rescue *::Wurk::Configuration::REDIS_ERROR_CLASSES
+        raise
+      rescue StandardError => e
+        ::Wurk.configuration.handle_exception(e, context: "Wurk::ApiController##{method}", jid: entry.jid)
+        failures << { key: entry.id, error: "#{e.class}: #{e.message}" }
       end
-      render json: { ok: true, count: count }
+    end
+
+    # 422 when any entry failed, so the SPA's error toast fires; `count` is
+    # what was applied either way.
+    def render_applied(total, failures)
+      body = { ok: failures.empty?, count: total - failures.size }
+      return render(json: body) if failures.empty?
+
+      render json: body.merge(failed: failures), status: 422
     end
 
     # Sends `method` (:quiet! / :stop!) to one process by identity, or to every
@@ -442,15 +467,14 @@ module Wurk
     # Entries removed concurrently no-op their apply.
     def drain_set(set, method)
       entries = set.to_a
-      entries.each { |entry| entry.public_send(method) }
-      entries.size
+      render_applied(entries.size, apply_each(entries, method))
     end
 
     def render_sorted_set(set)
       page = ::Wurk::Api::Pagination.window(params)
       total = set.size
       entries = ::Wurk::Api::Pagination.slice(set, page) { |entry| ::Wurk::Api::Serializers.sorted_entry(entry) }
-      render json: { total: total, page: page[:page], count: page[:count], entries: entries }
+      render json: { total: total, **::Wurk::Api::Pagination.meta(page), entries: entries }
     end
 
     # substr is already applied by Limits.list (matches on name), so slice the

data/app/controllers/wurk/extensions_controller.rb

@@ -37,11 +37,17 @@ module Wurk
     private
 
     def respond_with((status, headers, body))
-      return redirect_to(headers['Location'], allow_other_host: false) if status == 302
+      return redirect_to(headers['Location'], allow_other_host: false) if status == 302 && headers['Location']
 
+      content_type = 'text/html; charset=utf-8'
+      headers.each do |key, value|
+        next content_type = value if key.casecmp?('content-type')
+
+        response.headers[key] = value
+      end
       # Extension output is host-registered server code, same trust model as
       # Sidekiq::Web rendering its extensions — not user input.
-      render html: body.html_safe, layout: false, status: status
+      render body: body, content_type: content_type, status: status
     end
   end
 end

data/app/controllers/wurk/profiles_controller.rb

@@ -30,37 +30,73 @@ module Wurk
       # Firefox profiler store is a side effect — read-only deploys (e.g. the
       # public demo) must not let any visitor exfiltrate profiling data.
       return head(:forbidden) if ::Wurk::Web.config.read_only?
+      return head(:forbidden) unless navigated_from_dashboard?
 
-      blob = profile_blob(params[:key])
-      return head(:not_found) unless blob
-
-      hash = upload_to_profiler(blob)
-      return head(:bad_gateway) unless hash
+      sid = profile_sid(params[:key])
+      return head(:not_found) if sid == :missing
+      return head(:bad_gateway) unless sid
 
-      redirect_to(format(::Wurk::Web.config.profile_view_url, hash), allow_other_host: true)
+      redirect_to(format(::Wurk::Web.config.profile_view_url, sid), allow_other_host: true)
     end
 
     private
 
+    # The upload is a side effect behind a GET, so a cross-site `<img src>`
+    # could trigger it with the operator's session. Allow the dashboard's own
+    # link (same-origin) and a typed URL or bookmark (`none`). A browser too
+    # old to send Sec-Fetch-Site still sends a Referer, which must then be
+    # this host.
+    def navigated_from_dashboard?
+      site = request.headers['Sec-Fetch-Site']
+      return %w[same-origin none].include?(site) if site
+
+      referer = URI.parse(request.referer.to_s)
+      referer.host == request.host && referer.port == request.port
+    rescue URI::InvalidURIError
+      false
+    end
+
     def profile_blob(key)
       ::Wurk::ProfileRecord.data_for(key)
     end
 
-    # POSTs the gzipped profile to the Firefox profiler's compressed-store.
-    # Returns the public hash (used to build the view URL) or nil on failure.
+    # The profile-store id, cached in the HASH's `sid` field after the first
+    # upload exactly as Sidekiq's Web UI does, so either UI reuses the other's
+    # upload. :missing when the profile is gone, nil when the upload failed.
+    def profile_sid(key)
+      sid = ::Wurk.redis { |c| c.call('HGET', key, 'sid') }
+      return sid if sid
+
+      blob = profile_blob(key)
+      return :missing unless blob
+
+      sid = upload_to_profiler(blob)
+      ::Wurk.redis { |c| c.call('HSET', key, 'sid', sid) } if sid
+      sid
+    end
+
+    # POSTs the gzipped profile to the Firefox profiler's compressed-store,
+    # which answers with a JWT whose payload carries the `profileToken` the
+    # view URL needs. Returns that token, or nil on failure.
     def upload_to_profiler(gzipped)
       uri = URI.parse(::Wurk::Web.config.profile_store_url)
       res = post_gzip(uri, gzipped)
-      res.is_a?(Net::HTTPSuccess) ? res.body.to_s.strip : nil
+      res.is_a?(Net::HTTPSuccess) ? profile_token(res.body.to_s) : nil
     rescue StandardError => e
       Wurk.configuration.handle_exception(e, context: 'Wurk::ProfilesController#upload')
       nil
     end
 
+    def profile_token(jwt)
+      payload = jwt.strip.split('.')[1].to_s.tr('-_', '+/')
+      ::JSON.parse(payload.unpack1('m'))['profileToken']
+    end
+
     def post_gzip(uri, body)
       req = Net::HTTP::Post.new(uri)
       req['Content-Encoding'] = 'gzip'
       req['Content-Type'] = 'application/json'
+      req['Accept'] = 'application/vnd.firefox-profiler+json;version=1.0'
       req.body = body
       # Explicit timeouts so a slow/unreachable profiler can't tie up the Rails
       # request thread for Ruby's ~60s defaults.

data/config/routes.rb

@@ -25,6 +25,17 @@ Wurk::Engine.routes.draw do
     # against the per-set whitelist and returns a JSON 400 for an unknown action,
     # matching the single/bulk contract. A route-level constraint would turn that
     # into a 404 route miss instead.
+    #
+    # Responses: 200 `{ ok: true, count }`; 400 `{ error }` for an unknown cmd
+    # or too many keys; 404 `{ error: 'unknown job' }` when a single key is
+    # gone. The single (`<set>/:key`), bulk (`<set>`) and
+    # `scheduled/all/add_to_queue` paths apply entry by entry and answer 422
+    # `{ ok: false, count, failed: [{ key, error }] }` when any entry raised —
+    # `count` is what WAS applied, so the client must refetch the list.
+    # Listings answer `{ total, page, count, max_page, entries }`; `page` is
+    # the page actually served (clamped to `max_page`), and a `?substr=`
+    # listing adds `filtered_total` + `filtered_total_exact` (false: lower
+    # bound).
     get  'retries',          to: 'api#retries'
     post 'retries',          to: 'api#retries_bulk',   as: :api_retries_bulk
     post 'retries/all/:cmd', to: 'api#retries_all',    as: :api_retries_all
@@ -51,17 +62,20 @@ Wurk::Engine.routes.draw do
     # read-only mode 403s it via the Authorization middleware, no guard needed
     # here.
     get  'flows',            to: 'api#flows'
-    get  'flows/:fid',       to: 'api#flow', as: :api_flow
-    post 'flows/:fid/abandon', to: 'api#abandon_flow', as: :api_abandon_flow
+    get  'flows/:fid',       to: 'api#flow', as: :api_flow, constraints: { fid: %r{[^/]+} }
+    post 'flows/:fid/abandon', to: 'api#abandon_flow', as: :api_abandon_flow, constraints: { fid: %r{[^/]+} }
     get  'batches',          to: 'api#batches'
-    get  'batches/:bid',     to: 'api#batch', as: :api_batch
+    get  'batches/:bid',     to: 'api#batch', as: :api_batch, constraints: { bid: %r{[^/]+} }
     get  'limiters',         to: 'api#limiters'
-    post 'limiters/:name/reset', to: 'api#reset_limiter', as: :api_reset_limiter
+    # Limiter names, loop ids, flow and batch ids are user-chosen and may carry
+    # dots; without the segment constraint Rails reads `.b` as a format and the
+    # route 404s.
+    post 'limiters/:name/reset', to: 'api#reset_limiter', as: :api_reset_limiter, constraints: { name: %r{[^/]+} }
     get  'cron',             to: 'api#cron'
-    post 'cron/:lid/pause',  to: 'api#pause_cron', as: :api_pause_cron
-    post 'cron/:lid/unpause', to: 'api#unpause_cron', as: :api_unpause_cron
-    post 'cron/:lid/enqueue', to: 'api#enqueue_cron', as: :api_enqueue_cron
-    get  'cron/:lid/history', to: 'api#cron_history', as: :api_cron_history
+    post 'cron/:lid/pause',  to: 'api#pause_cron', as: :api_pause_cron, constraints: { lid: %r{[^/]+} }
+    post 'cron/:lid/unpause', to: 'api#unpause_cron', as: :api_unpause_cron, constraints: { lid: %r{[^/]+} }
+    post 'cron/:lid/enqueue', to: 'api#enqueue_cron', as: :api_enqueue_cron, constraints: { lid: %r{[^/]+} }
+    get  'cron/:lid/history', to: 'api#cron_history', as: :api_cron_history, constraints: { lid: %r{[^/]+} }
     get  'metrics',          to: 'api#metrics'
     get  'metrics/:klass',   to: 'api#metrics_for_job', as: :api_metrics_for_job, constraints: { klass: %r{[^/]+} }
     get  'history/snapshots', to: 'api#history_snapshots', as: :api_history_snapshots
@@ -109,6 +123,13 @@ Wurk::Engine.routes.draw do
   get 'ext-assets/:name/*file', to: 'extensions#asset', as: :extension_asset,
                                 format: false, constraints: { name: %r{[^/]+} }
 
-  # SPA catch-all — let the SolidJS router handle the rest.
-  get '*path', to: 'dashboard#index', constraints: ->(req) { req.format == :html }
+  # SPA catch-all — let the SolidJS router handle the rest. `format: false`
+  # because client routes carry user-chosen dots (`/queues/emails.critical`,
+  # `/retries/1700000000.5-jid`) that Rails would otherwise read as a format
+  # and 404 before the SPA loads; the HTML check then runs on Accept alone. A
+  # path ending in a static-file extension still 404s, so a stale asset or
+  # source-map URL doesn't come back as a 200 HTML page.
+  static_file = /\.(?:js|mjs|css|map|json|png|jpe?g|gif|svg|ico|webp|woff2?|ttf|txt)\z/i
+  get '*path', to: 'dashboard#index', format: false,
+               constraints: ->(req) { req.format == :html && !req.path.match?(static_file) }
 end

data/lib/generators/wurk/install/templates/wurk.rb

@@ -16,7 +16,7 @@ Wurk.configure_server do |config|
   # Seconds to let in-flight jobs finish on shutdown (Sidekiq-compatible key):
   # config[:timeout] = 25
 
-  # By default Wurk forks one worker process per CPU core (set WURK_COUNT, or
+  # By default Wurk forks one worker process per usable CPU (cgroup quota aware) (set WURK_COUNT, or
   # SIDEKIQ_COUNT, to override). For full control declare a topology — `flat`
   # spawns N identical forks; use `slot`s for dedicated queues. Total in-flight
   # jobs = forks × concurrency. See docs/idea/03-process-model.md.

data/lib/sidekiq/capsule.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/component.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/config.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/deploy.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/embedded.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/fetch.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/iterable_job.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/job/interrupt_handler.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/job/iterable/active_record_enumerator.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/job/iterable/csv_enumerator.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/job/iterable.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/job_logger.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/job_util.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'

data/lib/sidekiq/loader.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+# Drop-in require path (see lib/sidekiq.rb). Wurk loads this surface whole.
+require 'wurk'