worker_killer 0.1.1.39838 → 1.0.1.75538
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +53 -25
- data/lib/worker_killer/delayed_job_plugin.rb +16 -0
- data/lib/worker_killer/middleware.rb +3 -3
- data/lib/worker_killer/version.rb +1 -1
- data/spec/support/logger.rb +2 -2
- metadata +9 -9
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2c6211b790da17ea0b47bad73ee9689a2b5b77e1a5154f9463d9edce4e0afc55
|
|
4
|
+
data.tar.gz: 2934e7d52ee80eb1ccf87478666bc57b91c869d042f593f63763287bf1a88c7f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4f64d4a287ec283d6f5adb1411b7bffc6272765c27e7120030597e056a825766bcbbe37746a5c9bd4d4c3e6b0392a6d72ab1984a56f6b6c9b0fbf35683dd2430
|
|
7
|
+
data.tar.gz: 8ca469adb753f2e49936eecc8d6b893369921d644af6e04c14a4fdcf908b04ba515c2860f2b8ab10c4e3aae5c003e3fd75a3c5586b7e15a797b885da0fd4ff4f
|
data/README.md
CHANGED
|
@@ -4,69 +4,97 @@
|
|
|
4
4
|
[](https://rubygems.org/gems/worker_killer/versions)
|
|
5
5
|
[](http://www.rubydoc.info/gems/worker_killer)
|
|
6
6
|
|
|
7
|
-
[](https://lysander.rnds.pro/api/v1/badges/wkiller_coverage.html)
|
|
8
|
+
[](https://lysander.rnds.pro/api/v1/badges/wkiller_quality.html)
|
|
9
|
+
[](https://lysander.rnds.pro/api/v1/badges/wkiller_outdated.html)
|
|
10
|
+
[](https://lysander.rnds.pro/api/v1/badges/wkiller_vulnerable.html)
|
|
11
11
|
|
|
12
12
|
Kill any workers by memory and/or request counts or take custom reaction. Inspired by [unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer).
|
|
13
13
|
|
|
14
|
-
`worker-killer` gem provides automatic restart of Web-server based on 1) max number of requests, and 2) process memory size (RSS). This will greatly improves site's stability by avoiding unexpected memory exhaustion at the application nodes.
|
|
14
|
+
`worker-killer` gem provides automatic restart of Web-server and/or background job processor based on 1) max number of requests, and 2) process memory size (RSS). This will greatly improves site's stability by avoiding unexpected memory exhaustion at the application nodes.
|
|
15
15
|
|
|
16
16
|
Features:
|
|
17
17
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
Planned:
|
|
23
|
-
|
|
24
|
-
* DelayedJob support
|
|
18
|
+
- generic middleware implementation
|
|
19
|
+
- Phusion Passenger support(through `passenger-config detach-process <PID>`)
|
|
20
|
+
- DelayedJob support
|
|
21
|
+
- custom reaction hook
|
|
25
22
|
|
|
26
23
|
# Install
|
|
27
24
|
|
|
28
25
|
No external process like `god` is required. Just install one gem: `worker-killer`.
|
|
26
|
+
|
|
29
27
|
```ruby
|
|
30
28
|
gem 'worker-killer'
|
|
31
29
|
```
|
|
32
30
|
|
|
33
31
|
# Usage
|
|
34
32
|
|
|
35
|
-
|
|
33
|
+
## Rack-based Web-server
|
|
34
|
+
|
|
35
|
+
Add these lines to your `config.ru` or `application.rb`. (These lines should be added above the `require ::File.expand_path('../config/environment', __FILE__)` line.
|
|
36
36
|
|
|
37
37
|
```ruby
|
|
38
38
|
# self-process killer
|
|
39
39
|
require 'worker_killer/middleware'
|
|
40
|
-
|
|
40
|
+
|
|
41
|
+
killer = WorkerKiller::Killer::Passenger.new
|
|
42
|
+
|
|
41
43
|
# Max requests per worker
|
|
42
|
-
|
|
43
|
-
|
|
44
|
+
middleware.insert_before(
|
|
45
|
+
Rack::Sendfile,
|
|
46
|
+
WorkerKiller::Middleware::RequestsLimiter, killer: killer, min: 3072, max: 4096
|
|
47
|
+
)
|
|
48
|
+
|
|
44
49
|
# Max memory size (RSS) per worker
|
|
45
|
-
|
|
50
|
+
middleware.insert_before(
|
|
51
|
+
Rack::Sendfile,
|
|
52
|
+
WorkerKiller::Middleware::OOMLimiter, killer: killer, min: 500 * (1024**2), max: 600 * (1024**2)
|
|
53
|
+
)
|
|
46
54
|
```
|
|
47
55
|
|
|
48
|
-
|
|
56
|
+
## DelayedJob background processor
|
|
49
57
|
|
|
50
|
-
|
|
58
|
+
Add these lines to your `initializers/delayed_job.rb` or `application.rb`.
|
|
51
59
|
|
|
52
|
-
|
|
60
|
+
```ruby
|
|
61
|
+
# self-process killer
|
|
62
|
+
require 'worker_killer/delayed_job_plugin'
|
|
63
|
+
|
|
64
|
+
Delayed::Worker.plugins.tap do |plugins|
|
|
65
|
+
killer = WorkerKiller::Killer::DelayedJob.new
|
|
66
|
+
|
|
67
|
+
plugins << WorkerKiller::DelayedJobPlugin::JobsLimiter.new(
|
|
68
|
+
killer: killer, min: 200, max: 300
|
|
69
|
+
)
|
|
70
|
+
|
|
71
|
+
plugins << WorkerKiller::DelayedJobPlugin::OOMLimiter.new(
|
|
72
|
+
killer: killer, min: 500 * (1024**2), max: 600 * (1024**2)
|
|
73
|
+
)
|
|
74
|
+
end
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
This gem provides two modules: WorkerKiller::CountLimiter and WorkerKiller::MemoryLimiter, some Rack integration and DelayedJob plugin.
|
|
78
|
+
|
|
79
|
+
### WorkerKiller::Middleware::RequestsLimiter and WorkerKiller::DelayedJobPlugin::JobsLimiter
|
|
80
|
+
|
|
81
|
+
This module automatically restarts/kills the workers, based on the number of requests/jobs which worker processed.
|
|
53
82
|
|
|
54
83
|
`min` and `max` specify the min and max of maximum requests per worker. The actual limit is decided by rand() between `min` and `max` per worker, to prevent all workers to be dead at the same time. Once the number exceeds the limit, that worker is automatically restarted.
|
|
55
84
|
|
|
56
|
-
If `verbose` is set to true, then after every request, your log will show the requests left before restart.
|
|
85
|
+
If `verbose` is set to true, then after every request, your log will show the requests left before restart. This logging is done at the `info` level.
|
|
57
86
|
|
|
58
|
-
### WorkerKiller::Middleware::OOMLimiter
|
|
87
|
+
### WorkerKiller::Middleware::OOMLimiter and WorkerKiller::DelayedJobPlugin::OOMLimiter
|
|
59
88
|
|
|
60
89
|
This module automatically restarts/kills the workers, based on its memory size.
|
|
61
90
|
|
|
62
|
-
`min` and `max` specify the min and max of maximum memory in bytes per worker. The actual limit is decided by rand() between `min` and `max` per worker, to prevent all workers to be dead at the same time.
|
|
91
|
+
`min` and `max` specify the min and max of maximum memory in bytes per worker. The actual limit is decided by rand() between `min` and `max` per worker, to prevent all workers to be dead at the same time. Once the memory size exceeds `memory_size`, that worker is automatically restarted.
|
|
63
92
|
|
|
64
93
|
The memory size check is done in every `check_cycle` requests.
|
|
65
94
|
|
|
66
|
-
If `verbose` is set to true, then every memory size check will be shown in your logs.
|
|
95
|
+
If `verbose` is set to true, then every memory size check will be shown in your logs. This logging is done at the `info` level.
|
|
67
96
|
|
|
68
97
|
# Special Thanks
|
|
69
98
|
|
|
70
99
|
- [@hotchpotch](http://github.com/hotchpotch/) for the [original idea](https://gist.github.com/hotchpotch/1258681)
|
|
71
100
|
- [@kzk](http://github.com/kzk/) for the [unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer)
|
|
72
|
-
|
|
@@ -26,6 +26,22 @@ module WorkerKiller
|
|
|
26
26
|
end
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
+
class JobsLimiter < ::WorkerKiller::DelayedJobPlugin
|
|
30
|
+
|
|
31
|
+
def initialize(**opts)
|
|
32
|
+
super(klass: ::WorkerKiller::CountLimiter, **opts)
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
class OOMLimiter < ::WorkerKiller::DelayedJobPlugin
|
|
38
|
+
|
|
39
|
+
def initialize(**opts)
|
|
40
|
+
super(klass: ::WorkerKiller::MemoryLimiter, **opts)
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
end
|
|
44
|
+
|
|
29
45
|
end
|
|
30
46
|
end
|
|
31
47
|
|
|
@@ -14,13 +14,13 @@ module WorkerKiller
|
|
|
14
14
|
k.kill(l.started_at)
|
|
15
15
|
end
|
|
16
16
|
|
|
17
|
-
@limiter = klass.new(opts)
|
|
17
|
+
@limiter = klass.new(**opts)
|
|
18
18
|
end
|
|
19
19
|
|
|
20
20
|
def call(env)
|
|
21
|
-
|
|
21
|
+
@app.call(env)
|
|
22
|
+
ensure
|
|
22
23
|
reaction.call(limiter, killer) if limiter.check
|
|
23
|
-
response
|
|
24
24
|
end
|
|
25
25
|
|
|
26
26
|
class RequestsLimiter < ::WorkerKiller::Middleware
|
data/spec/support/logger.rb
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
RSpec.configure do |config|
|
|
2
2
|
config.before(:suite) do
|
|
3
3
|
$logger = Logger.new(STDOUT).tap do |logger|
|
|
4
|
-
logger.progname =
|
|
5
|
-
logger.level =
|
|
4
|
+
logger.progname = 'WorkerKiller'
|
|
5
|
+
logger.level = 'ERROR'
|
|
6
6
|
end
|
|
7
7
|
WorkerKiller.configure do |cfg|
|
|
8
8
|
cfg.logger = $logger
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: worker_killer
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 1.0.1.75538
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Samoilenko Yuri
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2022-02-07 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: get_process_mem
|
|
@@ -169,13 +169,13 @@ signing_key:
|
|
|
169
169
|
specification_version: 4
|
|
170
170
|
summary: Kill any workers by memory and request counts or take custom reaction
|
|
171
171
|
test_files:
|
|
172
|
-
- spec/middleware_spec.rb
|
|
173
|
-
- spec/killer_spec.rb
|
|
174
|
-
- spec/spec_helper.rb
|
|
175
|
-
- spec/killer/passenger_spec.rb
|
|
176
172
|
- spec/killer/signal_spec.rb
|
|
177
173
|
- spec/killer/delayed_job_spec.rb
|
|
178
|
-
- spec/
|
|
179
|
-
- spec/
|
|
180
|
-
- spec/worker_killer_spec.rb
|
|
174
|
+
- spec/killer/passenger_spec.rb
|
|
175
|
+
- spec/killer_spec.rb
|
|
181
176
|
- spec/count_limiter_spec.rb
|
|
177
|
+
- spec/worker_killer_spec.rb
|
|
178
|
+
- spec/support/logger.rb
|
|
179
|
+
- spec/middleware_spec.rb
|
|
180
|
+
- spec/spec_helper.rb
|
|
181
|
+
- spec/memory_limiter_spec.rb
|