wurk 1.7.2 → 1.7.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +14 -0
- data/lib/wurk/api/validation.rb +1 -2
- data/lib/wurk/cli.rb +4 -0
- data/lib/wurk/rails_boot.rb +6 -0
- data/lib/wurk/stats.rb +3 -3
- data/lib/wurk/swarm.rb +24 -0
- data/lib/wurk/version.rb +1 -1
- data/lib/wurk.rb +25 -0
- data/vendor/assets/dashboard/wurk-manifest.json +2 -2
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 33b583b405b2f2cf26f62fea6c6551d0f6900c3d72879ec6968dd7c82c976f26
|
|
4
|
+
data.tar.gz: fe626c58ce7bd322765cd0df1f9655097647ab2fdcd5b9aa4200012d9dc09825
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 177b520c3e822141886b62d740dfb88e16fc984bd5c21e949c039f5ecab578121d1e311e01dc917197257a419bb82ee1de23375a15c953b721de74ff22a3fc93
|
|
7
|
+
data.tar.gz: a14fb212831e08062df9211bdb2f4e78c6750d7b0380362c735985a662de11da8a7ea8cde08cb7df30c55c44279e959081a39b4e1e330b5129b07d6536a5b4ae
|
data/README.md
CHANGED
|
@@ -29,6 +29,8 @@ Wurk is wire-compatible with Sidekiq — same Redis keys, same job JSON, same Ru
|
|
|
29
29
|
|
|
30
30
|
**On speed:** Wurk is not currently faster than stock Sidekiq — it runs at roughly 0.87×–1.02× depending on workload shape, with parity on CPU and I/O but still behind on framework overhead (noop) and boot time. Numbers, method, and the reproduction command are in [docs/benchmarks.md](docs/benchmarks.md); run them yourself with `rake bench:vs_sidekiq`.
|
|
31
31
|
|
|
32
|
+
**Also from us:** [Ultimate](https://github.com/developerz-ai/ultimate) — the AI-first full-stack framework, for a weekend pet project or the system a company runs on. What a whole stack looks like when the one writing the code is an agent, not a human: [`bunx create-ultimate myapp`](#also-from-developerzai-ultimate).
|
|
33
|
+
|
|
32
34
|
## Install
|
|
33
35
|
|
|
34
36
|
```ruby
|
|
@@ -299,6 +301,18 @@ Wurk is MIT and stays that way. If what you need is a commercial support contrac
|
|
|
299
301
|
|
|
300
302
|
Full walkthrough — config side-by-side, the Redis key/`sidekiq_options` mapping, known incompatibilities, and a one-page cutover checklist: **[docs/migrate-from-sidekiq.md](https://github.com/developerz-ai/wurk/blob/main/docs/migrate-from-sidekiq.md)**.
|
|
301
303
|
|
|
304
|
+
## Also from developerz.ai: Ultimate
|
|
305
|
+
|
|
306
|
+
**[Ultimate](https://github.com/developerz-ai/ultimate) — the AI-first full-stack framework.** Rails' opinions, Bun's speed, one command that means shippable. Wurk is what one AI-maintained *library* looks like; Ultimate is the whole stack built on the same bet — the best framework you can hand a coding agent, whether what you are building is a weekend pet project or the system a company runs on.
|
|
307
|
+
|
|
308
|
+
```bash
|
|
309
|
+
bunx create-ultimate myapp && cd myapp && x dev
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
Every framework of the last fifteen years optimised for a human typing the code. Ultimate assumes an agent writes it, and that rewrites the defaults: **one way to do each thing**, so there is nothing to choose between; conventions that are **build errors**, so the compiler corrects the agent instead of a reviewer; errors carrying a stable code, a cause, and the exact command that fixes it, so a failure costs one round-trip; docs shipped in `node_modules` and `--json` on every command, so nothing reaches for a wiki. `x verify` is the whole contract — green means shippable.
|
|
313
|
+
|
|
314
|
+
TypeScript end to end, MIT, and as measured about its own claims as this file is: what is proven, what is not, and what it refuses to claim yet are all in [its README](https://github.com/developerz-ai/ultimate#readme).
|
|
315
|
+
|
|
302
316
|
## Contributing
|
|
303
317
|
|
|
304
318
|
Issues and pull requests are welcome — see **[CONTRIBUTING.md](https://github.com/developerz-ai/wurk/blob/main/CONTRIBUTING.md)** for the dev setup, test layers, and conventions, and **[SECURITY.md](https://github.com/developerz-ai/wurk/blob/main/SECURITY.md)** to report a vulnerability.
|
data/lib/wurk/api/validation.rb
CHANGED
|
@@ -94,13 +94,12 @@ module Wurk
|
|
|
94
94
|
# one without reopening this file. Baked into the literal rather than
|
|
95
95
|
# appended at load, so the parallel test runner never observes a
|
|
96
96
|
# half-built list.
|
|
97
|
-
# rubocop:disable Style/MutableConstant
|
|
97
|
+
# rubocop:disable-next Style/MutableConstant
|
|
98
98
|
JOB_KEYS = %w[
|
|
99
99
|
class args queue at retry jid backtrace tags dead
|
|
100
100
|
retry_for retry_queue expires_in track timeout deadline
|
|
101
101
|
encrypt unique_for unique_until collapse log_level locale wrapped
|
|
102
102
|
]
|
|
103
|
-
# rubocop:enable Style/MutableConstant
|
|
104
103
|
|
|
105
104
|
# `push_bulk`'s envelope keys, on top of the job keys it shares.
|
|
106
105
|
BULK_KEYS = %w[batch_size spread_interval].freeze
|
data/lib/wurk/cli.rb
CHANGED
|
@@ -240,7 +240,11 @@ module Wurk
|
|
|
240
240
|
|
|
241
241
|
# --- run helpers ----------------------------------------------------
|
|
242
242
|
|
|
243
|
+
# Claim the worker boot before the host app loads, so the railtie's
|
|
244
|
+
# after_initialize hook knows this process already has a worker and does
|
|
245
|
+
# not fork a second one on top of it (#worker_boot_claimed?).
|
|
243
246
|
def enter_server_mode
|
|
247
|
+
Wurk.claim_worker_boot!
|
|
244
248
|
Wurk.enter_server_mode(@config)
|
|
245
249
|
end
|
|
246
250
|
|
data/lib/wurk/rails_boot.rb
CHANGED
|
@@ -42,8 +42,14 @@ module Wurk
|
|
|
42
42
|
# A process that won't run workers isn't a server: skip both server mode
|
|
43
43
|
# and the swarm boot. Console mode is detected reliably here — the console
|
|
44
44
|
# command file defines ::Rails::Console before initializers run.
|
|
45
|
+
#
|
|
46
|
+
# `Wurk.worker_boot_claimed?` covers the CLI: `bundle exec wurk` boots the
|
|
47
|
+
# host app itself and then runs a Launcher in this very process, so forking
|
|
48
|
+
# a swarm here as well would put two independent workers on one queue and
|
|
49
|
+
# run every job — every cron tick included — twice.
|
|
45
50
|
def skip_boot?
|
|
46
51
|
ENV['WURK_DISABLED'] == '1' ||
|
|
52
|
+
Wurk.worker_boot_claimed? ||
|
|
47
53
|
building? ||
|
|
48
54
|
defined?(::Rails::Console) ||
|
|
49
55
|
::Rails.env.test?
|
data/lib/wurk/stats.rb
CHANGED
|
@@ -116,9 +116,9 @@ module Wurk
|
|
|
116
116
|
# Single pipeline for the cheap counters. Eagerly invoked at initialize
|
|
117
117
|
# so callers can read many fields without paying per-method round trips.
|
|
118
118
|
FAST_QUERIES = [
|
|
119
|
-
['GET',
|
|
120
|
-
['GET',
|
|
121
|
-
['GET',
|
|
119
|
+
['GET', 'stat:processed'],
|
|
120
|
+
['GET', 'stat:failed'],
|
|
121
|
+
['GET', Keys::STAT_EXPIRED],
|
|
122
122
|
['ZCARD', Keys::SCHEDULE],
|
|
123
123
|
['ZCARD', Keys::RETRY],
|
|
124
124
|
['ZCARD', Keys::DEAD],
|
data/lib/wurk/swarm.rb
CHANGED
|
@@ -65,6 +65,25 @@ module Wurk
|
|
|
65
65
|
# them mid-tail, so its own wait always extends past theirs by this much.
|
|
66
66
|
SHUTDOWN_GRACE = 5
|
|
67
67
|
|
|
68
|
+
# How often the drain re-relays TERM to whatever is still in the child
|
|
69
|
+
# table. A child forked in the instant before the drain can miss the first
|
|
70
|
+
# one outright: from `Process.fork` returning in the child until ChildBoot
|
|
71
|
+
# resets the inherited traps, the PARENT's handler is what a TERM finds
|
|
72
|
+
# there, and it writes the signal into the parent's self-pipe (or, once
|
|
73
|
+
# `fork_child` has dropped that pipe, into nothing) instead of stopping the
|
|
74
|
+
# child. The child then boots clean, drains nothing, and `wait_for_children`
|
|
75
|
+
# burns its whole budget before `hard_kill_stragglers` SIGKILLs it — taking
|
|
76
|
+
# its in-flight jobs with it. The window is the `_fork` hook chain
|
|
77
|
+
# (ActiveSupport's ForkTracker and friends all run child-side before
|
|
78
|
+
# `Process.fork` returns), so it widens with load: measured 10.5s drains
|
|
79
|
+
# against a 2.6s baseline once that chain is slowed. No child-side fix can
|
|
80
|
+
# close it — the window precedes any code the child controls — so the
|
|
81
|
+
# supervisor repeats itself instead. Re-sending is safe: by the second pass
|
|
82
|
+
# the child owns its handler, TERM to a child already draining is a no-op,
|
|
83
|
+
# and the pids come from the child table, where an exited child is a zombie
|
|
84
|
+
# until reaped and so can never be a recycled pid.
|
|
85
|
+
RETERM_INTERVAL = 1.0
|
|
86
|
+
|
|
68
87
|
# USR2 is relayed (log reopen) — without a trap, a logrotate config that
|
|
69
88
|
# signals the master pid would hit USR2's default disposition and kill the
|
|
70
89
|
# whole swarm.
|
|
@@ -503,8 +522,13 @@ module Wurk
|
|
|
503
522
|
|
|
504
523
|
def wait_for_children(timeout)
|
|
505
524
|
deadline = monotonic + timeout
|
|
525
|
+
reterm_at = monotonic + RETERM_INTERVAL
|
|
506
526
|
while monotonic < deadline && any_children?
|
|
507
527
|
reap_children
|
|
528
|
+
if monotonic >= reterm_at
|
|
529
|
+
relay_signal('TERM')
|
|
530
|
+
reterm_at = monotonic + RETERM_INTERVAL
|
|
531
|
+
end
|
|
508
532
|
sleep 0.1
|
|
509
533
|
end
|
|
510
534
|
end
|
data/lib/wurk/version.rb
CHANGED
data/lib/wurk.rb
CHANGED
|
@@ -225,6 +225,31 @@ module Wurk
|
|
|
225
225
|
config[:server] = true
|
|
226
226
|
end
|
|
227
227
|
|
|
228
|
+
# True once something in THIS process has taken responsibility for running
|
|
229
|
+
# the workers. The CLI claims it before it boots the host application; the
|
|
230
|
+
# railtie's `after_initialize` hook reads it and stands down.
|
|
231
|
+
#
|
|
232
|
+
# Without the claim, `bundle exec wurk` inside a Rails app runs two
|
|
233
|
+
# independent workers: the CLI boots the app, the railtie forks a swarm
|
|
234
|
+
# from after_initialize, and then the CLI starts its own Launcher in the
|
|
235
|
+
# parent. Both drain the same queue, so every job runs twice — including
|
|
236
|
+
# every cron tick. See #claim_worker_boot!.
|
|
237
|
+
def worker_boot_claimed?
|
|
238
|
+
!!@worker_boot_claimed
|
|
239
|
+
end
|
|
240
|
+
|
|
241
|
+
# Claimed by the wurk/wurkswarm CLI. Not part of `enter_server_mode`: the
|
|
242
|
+
# railtie enters server mode too, and claiming there would make the railtie
|
|
243
|
+
# skip its own boot and run no workers at all.
|
|
244
|
+
def claim_worker_boot!
|
|
245
|
+
@worker_boot_claimed = true
|
|
246
|
+
end
|
|
247
|
+
|
|
248
|
+
# Test seam, matching `attr_writer :server` above: the claim is
|
|
249
|
+
# process-global, so a test that runs the CLI has to hand it back or it
|
|
250
|
+
# leaks into whatever test class shares the worker process next.
|
|
251
|
+
attr_writer :worker_boot_claimed
|
|
252
|
+
|
|
228
253
|
# Wurk ships Pro+Ent features in the free gem; these flags exist solely
|
|
229
254
|
# for third-party gems that branch on Sidekiq.pro? / Sidekiq.ent?.
|
|
230
255
|
def pro?
|