worker_killer 0.1.0.19842 → 1.0.0.39839

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 5ea06851fd7e2eabda1ebc603b59c497faf8996d146a2274f110d3faabfc0cf8
4
- data.tar.gz: 44ab1e7b8b070e002e72f7ecad5d7f94b538315360690a538d34e9e814a364c8
3
+ metadata.gz: 8c2cabf961e621ef921c3f5d8ce24ef87c49248c0c0c93fa7141431df5e208ce
4
+ data.tar.gz: 7022f38e793bd749a3ca88d9b3b641c70a5009a00921652322a71e41f0be5df5
5
5
  SHA512:
6
- metadata.gz: ada3a973c748b46830e9ea9efa91dd9bb26ddbffd09eed8ed7eede5d7a913a3b336bad37b7cea2f0d12cbf108344e0e43eca722e74f31aa8ce2e8ee2bbe575d9
7
- data.tar.gz: a1415f2f642552b9732ccf33dfce35f71e90b873e9db6bd1109d172a55ac20c6c1b3b5034202c3ab9cbff43be20d2e6a12d3d120a503ce4b2b01cc3972665635
6
+ metadata.gz: 2e8ff2b3c4dcf2a926160dcee34c4c051565ac9084c8ec8317a84172d0555739121650bda21e2ce4b09b82038bd6cb37f799cd872be7c37d52fab62e3aefb03f
7
+ data.tar.gz: a443e0c89295e33f85d8563fb3f9fd3fe8e692dd4fc2cfa4ae9d9c378edf7aaefda77b0458c95fb6221b810edd4386c1700af167c0f747a006a33981920a95c0
data/README.md CHANGED
@@ -1,62 +1,101 @@
1
1
  # worker-killer
2
2
 
3
- Kill any workers by memory and request counts or take custom reaction. Inspired by [unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer).
3
+ [![Gem Version](https://badge.fury.io/rb/worker_killer.svg)](https://rubygems.org/gems/worker_killer)
4
+ [![Gem](https://img.shields.io/gem/dt/worker_killer.svg)](https://rubygems.org/gems/worker_killer/versions)
5
+ [![YARD](https://badgen.net/badge/YARD/doc/blue)](http://www.rubydoc.info/gems/worker_killer)
4
6
 
5
- `worker-killer` gem provides automatic restart of Web-server based on 1) max number of requests, and 2) process memory size (RSS). This will greatly improves site's stability by avoiding unexpected memory exhaustion at the application nodes.
7
+ [![Coverage](https://lysander.rnds.pro/api/v1/badges/wkiller_coverage.svg)](https://lysander.rnds.pro/api/v1/badges/wkiller_coverage.html)
8
+ [![Quality](https://lysander.rnds.pro/api/v1/badges/wkiller_quality.svg)](https://lysander.rnds.pro/api/v1/badges/wkiller_quality.html)
9
+ [![Outdated](https://lysander.rnds.pro/api/v1/badges/wkiller_outdated.svg)](https://lysander.rnds.pro/api/v1/badges/wkiller_outdated.html)
10
+ [![Vulnerabilities](https://lysander.rnds.pro/api/v1/badges/wkiller_vulnerable.svg)](https://lysander.rnds.pro/api/v1/badges/wkiller_vulnerable.html)
6
11
 
7
- Features:
12
+ Kill any workers by memory and/or request counts or take custom reaction. Inspired by [unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer).
8
13
 
9
- * generic middleware implementation
10
- * custom reaction hook
14
+ `worker-killer` gem provides automatic restart of Web-server and/or background job processor based on 1) max number of requests, and 2) process memory size (RSS). This will greatly improves site's stability by avoiding unexpected memory exhaustion at the application nodes.
11
15
 
12
- Planned:
16
+ Features:
13
17
 
14
- * DelayedJob support
18
+ - generic middleware implementation
19
+ - Phusion Passenger support(through `passenger-config detach-process <PID>`)
20
+ - DelayedJob support
21
+ - custom reaction hook
15
22
 
16
23
  # Install
17
24
 
18
25
  No external process like `god` is required. Just install one gem: `worker-killer`.
26
+
19
27
  ```ruby
20
28
  gem 'worker-killer'
21
29
  ```
22
30
 
23
31
  # Usage
24
32
 
25
- Add these lines to your `config.ru` or `application.rb`. (These lines should be added above the `require ::File.expand_path('../config/environment', __FILE__)` line.
33
+ ## Rack-based Web-server
34
+
35
+ Add these lines to your `config.ru` or `application.rb`. (These lines should be added above the `require ::File.expand_path('../config/environment', __FILE__)` line.
26
36
 
27
37
  ```ruby
28
38
  # self-process killer
29
39
  require 'worker_killer/middleware'
30
-
40
+
41
+ killer = WorkerKiller::Killer::Passenger.new
42
+
31
43
  # Max requests per worker
32
- config.middleware.insert_before(Rack::Sendfile, WorkerKiller::Middleware::RequestsLimiter, min: 4096, max: 5120)
33
-
44
+ middleware.insert_before(
45
+ Rack::Sendfile,
46
+ WorkerKiller::Middleware::RequestsLimiter, killer: killer, min: 3072, max: 4096
47
+ )
48
+
34
49
  # Max memory size (RSS) per worker
35
- config.middleware.insert_before(Rack::Sendfile, WorkerKiller::Middleware::OOMLimiter, min: 300 * (1024**2), max: 400 * (1024**2))
50
+ middleware.insert_before(
51
+ Rack::Sendfile,
52
+ WorkerKiller::Middleware::OOMLimiter, killer: killer, min: 500 * (1024**2), max: 600 * (1024**2)
53
+ )
36
54
  ```
37
55
 
38
- This gem provides two modules: WorkerKiller::CountLimiter and WorkerKiller::MemoryLimiter and some Rack integration.
56
+ ## DelayedJob background processor
39
57
 
40
- ### WorkerKiller::Middleware::RequestsLimiter
58
+ Add these lines to your `initializers/delayed_job.rb` or `application.rb`.
41
59
 
42
- This module automatically restarts the kill workers, based on the number of requests which worker processed.
60
+ ```ruby
61
+ # self-process killer
62
+ require 'worker_killer/delayed_job_plugin'
63
+
64
+ Delayed::Worker.plugins.tap do |plugins|
65
+ plugins << Gorynich::Head::DelayedJob
66
+ killer = WorkerKiller::Killer::DelayedJob.new
67
+
68
+ plugins << WorkerKiller::DelayedJobPlugin::JobsLimiter.new(
69
+ killer: killer, min: 200, max: 300
70
+ )
71
+
72
+ plugins << WorkerKiller::DelayedJobPlugin::OOMLimiter.new(
73
+ killer: killer, min: 500 * (1024**2), max: 600 * (1024**2)
74
+ )
75
+ end
76
+ ```
77
+
78
+ This gem provides two modules: WorkerKiller::CountLimiter and WorkerKiller::MemoryLimiter, some Rack integration and DelayedJob plugin.
79
+
80
+ ### WorkerKiller::Middleware::RequestsLimiter and WorkerKiller::DelayedJobPlugin::JobsLimiter
81
+
82
+ This module automatically restarts/kills the workers, based on the number of requests/jobs which worker processed.
43
83
 
44
84
  `min` and `max` specify the min and max of maximum requests per worker. The actual limit is decided by rand() between `min` and `max` per worker, to prevent all workers to be dead at the same time. Once the number exceeds the limit, that worker is automatically restarted.
45
85
 
46
- If `verbose` is set to true, then after every request, your log will show the requests left before restart. This logging is done at the `info` level.
86
+ If `verbose` is set to true, then after every request, your log will show the requests left before restart. This logging is done at the `info` level.
47
87
 
48
- ### WorkerKiller::Middleware::OOMLimiter
88
+ ### WorkerKiller::Middleware::OOMLimiter and WorkerKiller::DelayedJobPlugin::OOMLimiter
49
89
 
50
- This module automatically restarts the Unicorn workers, based on its memory size.
90
+ This module automatically restarts/kills the workers, based on its memory size.
51
91
 
52
- `min` and `max` specify the min and max of maximum memory in bytes per worker. The actual limit is decided by rand() between `min` and `max` per worker, to prevent all workers to be dead at the same time. Once the memory size exceeds `memory_size`, that worker is automatically restarted.
92
+ `min` and `max` specify the min and max of maximum memory in bytes per worker. The actual limit is decided by rand() between `min` and `max` per worker, to prevent all workers to be dead at the same time. Once the memory size exceeds `memory_size`, that worker is automatically restarted.
53
93
 
54
94
  The memory size check is done in every `check_cycle` requests.
55
95
 
56
- If `verbose` is set to true, then every memory size check will be shown in your logs. This logging is done at the `info` level.
96
+ If `verbose` is set to true, then every memory size check will be shown in your logs. This logging is done at the `info` level.
57
97
 
58
98
  # Special Thanks
59
99
 
60
100
  - [@hotchpotch](http://github.com/hotchpotch/) for the [original idea](https://gist.github.com/hotchpotch/1258681)
61
101
  - [@kzk](http://github.com/kzk/) for the [unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer)
62
-
@@ -3,6 +3,7 @@ require 'worker_killer/configuration'
3
3
  require 'worker_killer/count_limiter'
4
4
  require 'worker_killer/memory_limiter'
5
5
  require 'worker_killer/middleware'
6
+ require 'worker_killer/killer'
6
7
 
7
8
  module WorkerKiller
8
9
 
@@ -27,25 +28,45 @@ module WorkerKiller
27
28
  # the process isn't killed after `configuration.quit_attempts` QUIT signals,
28
29
  # send TERM signals until `configuration.kill_attempts`. Finally, send a KILL
29
30
  # signal. A single signal is sent per request.
30
- def self.kill_self(logger, start_time)
31
- alive_sec = (Time.now - start_time).round
32
- self_pid = Process.pid
33
-
34
- @kill_attempts ||= 0
35
- @kill_attempts += 1
36
-
37
- if configuration.use_quit
38
- sig = :QUIT
39
- sig = :TERM if @kill_attempts > configuration.quit_attempts
40
- sig = :KILL if @kill_attempts > (configuration.quit_attempts + configuration.kill_attempts)
41
- else
42
- sig = :TERM
43
- sig = :KILL if @kill_attempts > configuration.kill_attempts
44
- end
45
-
46
- logger.warn "#{self} send SIG#{sig} (pid: #{self_pid}) alive: #{alive_sec} sec (trial #{@kill_attempts})"
47
- Process.kill sig, self_pid
48
- end
31
+ # def self.kill_self(logger, start_time)
32
+ # alive_sec = (Time.now - start_time).round
33
+
34
+ # @kill_attempts ||= 0
35
+ # @kill_attempts += 1
36
+
37
+ # if configuration.use_quit
38
+ # sig = :QUIT
39
+ # sig = :TERM if @kill_attempts > configuration.quit_attempts
40
+ # sig = :KILL if @kill_attempts > (configuration.quit_attempts + configuration.term_attempts)
41
+ # else
42
+ # sig = :TERM
43
+ # sig = :KILL if @kill_attempts > configuration.term_attempts
44
+ # end
45
+
46
+ # if sig == :QUIT && configuration.passenger?
47
+ # kill_by_passenger(logger, alive_sec, configuration.passenger_config, Process.pid)
48
+ # else
49
+ # kill_by_signal(logger, alive_sec, sig, Process.pid)
50
+ # end
51
+ # end
52
+
53
+ # def self.kill_by_signal(logger, alive_sec, signal, pid)
54
+ # logger.warn "#{self} send SIG#{signal} (pid: #{pid}) alive: #{alive_sec} sec (trial #{@kill_attempts})"
55
+ # Process.kill signal, pid
56
+ # end
57
+
58
+ # def self.kill_by_passenger(logger, alive_sec, passenger, pid)
59
+ # return if @already_detached
60
+ # @already_detached = true
61
+
62
+ # cmd = "#{passenger} detach-process #{pid}"
63
+ # logger.warn "#{self} run #{cmd.inspect} (pid: #{pid}) alive: #{alive_sec} sec (trial #{@kill_attempts})"
64
+ # Thread.new(cmd) do |command|
65
+ # unless Kernel.system(command)
66
+ # logger.warn "#{self} run #{cmd.inspect} failed: #{$?.inspect}"
67
+ # end
68
+ # end
69
+ # end
49
70
 
50
71
  end
51
72
 
@@ -4,13 +4,12 @@ module WorkerKiller
4
4
  # Methods for configuring WorkerKiller
5
5
  class Configuration
6
6
 
7
- attr_accessor :logger, :quit_attempts, :kill_attempts, :use_quit
7
+ attr_accessor :logger, :quit_attempts, :term_attempts
8
8
 
9
9
  # Override defaults for configuration
10
- def initialize(quit_attempts: 5, kill_attempts: 10, use_quit: true)
10
+ def initialize(quit_attempts: 10, term_attempts: 50)
11
11
  @quit_attempts = quit_attempts
12
- @kill_attempts = kill_attempts
13
- @use_quit = use_quit
12
+ @term_attempts = term_attempts
14
13
  @logger = Logger.new(STDOUT, level: Logger::INFO, progname: 'WorkerKiller')
15
14
  end
16
15
 
@@ -2,15 +2,13 @@ module WorkerKiller
2
2
  class CountLimiter
3
3
 
4
4
  attr_reader :min, :max, :left, :limit, :started_at
5
- attr_accessor :reaction
6
5
 
7
- def initialize(min: 3072, max: 4096, verbose: false, &block)
6
+ def initialize(min: 3072, max: 4096, verbose: false)
8
7
  @min = min
9
8
  @max = max
10
9
  @limit = @min + WorkerKiller.randomize(@max - @min + 1)
11
10
  @left = @limit
12
11
  @verbose = verbose
13
- @reaction = block
14
12
  end
15
13
 
16
14
  def check
@@ -25,7 +23,6 @@ module WorkerKiller
25
23
  return false if (@left -= 1) > 0
26
24
 
27
25
  logger.warn "#{self}: worker (pid: #{Process.pid}) exceeds max number of requests (limit: #{@limit})"
28
- @reaction.call(self)
29
26
 
30
27
  true
31
28
  end
@@ -0,0 +1,47 @@
1
+ require 'worker_killer/memory_limiter'
2
+ require 'worker_killer/count_limiter'
3
+
4
+ module WorkerKiller
5
+ class DelayedJobPlugin
6
+
7
+ attr_reader :limiter, :killer, :reaction
8
+
9
+ def initialize(klass:, killer:, reaction: nil, **opts)
10
+ @killer = killer
11
+
12
+ @reaction = reaction || proc do |l, k, dj|
13
+ k.kill(l.started_at, dj: dj)
14
+ end
15
+
16
+ @limiter = klass.new(opts)
17
+ end
18
+
19
+ def new(*_args)
20
+ configure_lifecycle(Delayed::Worker.lifecycle)
21
+ end
22
+
23
+ def configure_lifecycle(lifecycle)
24
+ lifecycle.after(:perform) do |worker, *_args|
25
+ reaction.call(limiter, killer, worker) if limiter.check
26
+ end
27
+ end
28
+
29
+ class JobsLimiter < ::WorkerKiller::DelayedJobPlugin
30
+
31
+ def initialize(**opts)
32
+ super(klass: ::WorkerKiller::CountLimiter, **opts)
33
+ end
34
+
35
+ end
36
+
37
+ class OOMLimiter < ::WorkerKiller::DelayedJobPlugin
38
+
39
+ def initialize(**opts)
40
+ super(klass: ::WorkerKiller::MemoryLimiter, **opts)
41
+ end
42
+
43
+ end
44
+
45
+ end
46
+ end
47
+
@@ -0,0 +1,38 @@
1
+ module WorkerKiller
2
+ module Killer
3
+ class Base
4
+
5
+ attr_accessor :config, :kill_attempts, :logger
6
+
7
+ def initialize(logger: WorkerKiller.configuration.logger, **_kwargs)
8
+ @logger = logger
9
+ @config = WorkerKiller.configuration
10
+ @kill_attempts = 0
11
+ end
12
+
13
+ def kill(start_time, **params)
14
+ alive_sec = (Time.now - start_time).round
15
+
16
+ @kill_attempts += 1
17
+
18
+ sig = :QUIT
19
+ sig = :TERM if kill_attempts > config.quit_attempts
20
+ sig = :KILL if kill_attempts > (config.quit_attempts + config.term_attempts)
21
+
22
+ do_kill(sig, Process.pid, alive_sec, **params)
23
+ end
24
+
25
+ # :nocov:
26
+ def do_kill(*_args)
27
+ raise 'Not Implemented'
28
+ end
29
+ # :nocov:
30
+
31
+ end
32
+ end
33
+ end
34
+
35
+ require_relative 'killer/signal'
36
+ require_relative 'killer/passenger'
37
+ require_relative 'killer/delayed_job'
38
+
@@ -0,0 +1,27 @@
1
+ module WorkerKiller
2
+ module Killer
3
+ class DelayedJob < ::WorkerKiller::Killer::Base
4
+
5
+ def do_kill(sig, pid, alive_sec, dj:, **_params)
6
+ if sig == :KILL
7
+ logger.error "#{self} force to #{sig} self (pid: #{pid}) alive: #{alive_sec} sec (trial #{kill_attempts})"
8
+ Process.kill sig, pid
9
+ return
10
+ end
11
+
12
+ dj.stop
13
+
14
+ return if sig != :TERM
15
+
16
+ if @termination
17
+ logger.warn "#{self} force to #{sig} self (pid: #{pid}) alive: #{alive_sec} sec (trial #{kill_attempts})"
18
+ Process.kill sig, pid
19
+ else
20
+ @termination = true
21
+ end
22
+ end
23
+
24
+ end
25
+ end
26
+ end
27
+
@@ -0,0 +1,55 @@
1
+ module WorkerKiller
2
+ module Killer
3
+ class Passenger < ::WorkerKiller::Killer::Base
4
+
5
+ attr_reader :passenger_config
6
+
7
+ def initialize path: nil, **kwrags
8
+ super
9
+ @passenger_config = if path.nil? || path.empty?
10
+ self.class.check_passenger_config(`which passenger-config`)
11
+ else
12
+ self.class.check_passenger_config!(path)
13
+ end
14
+ end
15
+
16
+ def do_kill(sig, pid, alive_sec, **params)
17
+ cmd = "#{passenger_config} detach-process #{pid}"
18
+ if sig == :KILL
19
+ logger.error "#{self} force to kill self (pid: #{pid}) alive: #{alive_sec} sec (trial #{kill_attempts})"
20
+ Process.kill sig, pid
21
+ return
22
+ end
23
+
24
+ return if @already_detached
25
+
26
+ logger.warn "#{self} run #{cmd.inspect} (pid: #{pid}) alive: #{alive_sec} sec (trial #{kill_attempts})"
27
+ @already_detached = true
28
+
29
+ Thread.new(cmd) do |command|
30
+ unless Kernel.system(command)
31
+ logger.warn "#{self} run #{command.inspect} failed: #{$?.inspect}"
32
+ end
33
+ end
34
+ end
35
+
36
+ def self.check_passenger_config path
37
+ path.strip!
38
+ help = `#{path} detach-process --help 2> /dev/null`
39
+ return path if help['Remove an application process from the Phusion Passenger process pool']
40
+ rescue StandardError => e
41
+ nil
42
+ end
43
+
44
+ def self.check_passenger_config! path
45
+ if path = check_passenger_config(path)
46
+ path
47
+ else
48
+ raise "Can't find passenger config at #{path.inspect}"
49
+ end
50
+ end
51
+
52
+ end
53
+ end
54
+ end
55
+
@@ -0,0 +1,16 @@
1
+ module WorkerKiller
2
+ module Killer
3
+ class Signal < ::WorkerKiller::Killer::Base
4
+
5
+ def do_kill(sig, pid, alive_sec, **params)
6
+ return if sig == @last_signal
7
+
8
+ @last_signal = sig
9
+ logger.warn "#{self} send SIG#{sig} (pid: #{pid}) alive: #{alive_sec} sec (trial #{kill_attempts})"
10
+ Process.kill sig, pid
11
+ end
12
+
13
+ end
14
+ end
15
+ end
16
+
@@ -4,16 +4,14 @@ module WorkerKiller
4
4
  class MemoryLimiter
5
5
 
6
6
  attr_reader :min, :max, :limit, :started_at, :check_cycle
7
- attr_accessor :reaction
8
7
 
9
- def initialize(min: (1024**3), max: (2 * (1024**3)), check_cycle: 16, verbose: false, &block)
8
+ def initialize(min: (1024**3), max: (2 * (1024**3)), check_cycle: 16, verbose: false)
10
9
  @min = min
11
10
  @max = max
12
11
  @limit = @min + WorkerKiller.randomize(@max - @min + 1)
13
12
  @check_cycle = check_cycle
14
13
  @check_count = 0
15
14
  @verbose = verbose
16
- @reaction = block
17
15
  end
18
16
 
19
17
  def check
@@ -33,7 +31,6 @@ module WorkerKiller
33
31
  return false if rss <= @limit
34
32
 
35
33
  logger.warn "#{self}: worker (pid: #{Process.pid}) exceeds memory limit (#{rss} bytes > #{@limit} bytes)"
36
- @reaction.call(self)
37
34
 
38
35
  true
39
36
  end
@@ -4,22 +4,23 @@ require 'worker_killer/count_limiter'
4
4
  module WorkerKiller
5
5
  class Middleware
6
6
 
7
- attr_reader :limiter
7
+ attr_reader :limiter, :killer, :reaction
8
8
 
9
- def initialize(app, klass:, reaction: nil, **opts)
9
+ def initialize(app, killer:, klass:, reaction: nil, **opts)
10
10
  @app = app
11
+ @killer = killer
11
12
 
12
- reaction ||= proc do |limiter|
13
- WorkerKiller.kill_self(limiter.logger, limiter.started_at)
13
+ @reaction = reaction || proc do |l, k|
14
+ k.kill(l.started_at)
14
15
  end
15
16
 
16
- @limiter = klass.new(opts, &reaction)
17
+ @limiter = klass.new(opts)
17
18
  end
18
19
 
19
20
  def call(env)
20
- response = @app.call(env)
21
- @limiter.check
22
- response
21
+ @app.call(env)
22
+ ensure
23
+ reaction.call(limiter, killer) if limiter.check
23
24
  end
24
25
 
25
26
  class RequestsLimiter < ::WorkerKiller::Middleware
@@ -2,7 +2,7 @@
2
2
 
3
3
  module WorkerKiller
4
4
 
5
- VERSION = '0.1.0'
5
+ VERSION = '1.0.0'
6
6
 
7
7
  end
8
8
 
@@ -1,20 +1,15 @@
1
1
  RSpec.describe WorkerKiller::CountLimiter do
2
- let(:logger){ Logger.new(nil) }
3
-
4
2
  subject{ described_class.new(options) }
5
3
  let(:min){ rand(50..100) }
6
4
  let(:max){ min + rand(100) }
7
- let(:options){ { min: min, max: max } }
5
+ let(:options){ { min: min, max: max, verbose: true } }
8
6
 
9
7
  it { is_expected.to have_attributes(min: min, max: max, limit: a_value_between(min, max), left: subject.limit) }
10
8
 
11
9
  it 'expect not to react while less than limit' do
12
- expect do |b|
13
- subject.reaction = b.to_proc
14
- (subject.limit - 1).times do
15
- expect(subject.check).to be_falsey
16
- end
17
- end.not_to yield_control
10
+ (subject.limit - 1).times do
11
+ expect(subject.check).to be_falsey
12
+ end
18
13
  end
19
14
 
20
15
  it 'expect call reaction when check succeded' do
@@ -22,11 +17,8 @@ RSpec.describe WorkerKiller::CountLimiter do
22
17
  expect(subject.check).to be_falsey
23
18
  end
24
19
 
25
- expect do |b|
26
- subject.reaction = b.to_proc
27
- expect(subject.check).to be_truthy
28
- expect(subject.check).to be_truthy
29
- end.to yield_control.exactly(2).times
20
+ expect(subject.check).to be_truthy
21
+ expect(subject.check).to be_truthy
30
22
  end
31
23
  end
32
24
 
@@ -0,0 +1,34 @@
1
+ RSpec.describe WorkerKiller::Killer::DelayedJob do
2
+ let(:config) do
3
+ WorkerKiller::Configuration.new.tap do |c|
4
+ c.quit_attempts = 2
5
+ c.term_attempts = 2
6
+ end
7
+ end
8
+
9
+ let(:killer){ described_class.new() }
10
+ let(:dj){ double }
11
+
12
+ describe '#kill' do
13
+ context 'with use_quit TRUE' do
14
+ around do |example|
15
+ prev = WorkerKiller.configuration
16
+ WorkerKiller.configuration = config
17
+ example.run
18
+ ensure
19
+ WorkerKiller.configuration = prev
20
+ end
21
+
22
+ it 'expect right signal order' do
23
+ expect(dj).to receive(:stop).exactly(4)
24
+ expect(Process).to receive(:kill).with(:TERM, anything).exactly(1).times
25
+ expect(Process).to receive(:kill).with(:KILL, anything).exactly(5).times
26
+
27
+ 2.times { killer.kill(Time.now, dj: dj) } # 1 QUIT
28
+ 2.times { killer.kill(Time.now, dj: dj) } # 1 TERM
29
+ 5.times { killer.kill(Time.now, dj: dj) } # 5 KILL
30
+ end
31
+ end
32
+ end
33
+ end
34
+
@@ -0,0 +1,43 @@
1
+ RSpec.describe WorkerKiller::Killer::Passenger do
2
+ let(:config) do
3
+ WorkerKiller::Configuration.new.tap do |c|
4
+ c.quit_attempts = 2
5
+ c.term_attempts = 2
6
+ end
7
+ end
8
+
9
+ let(:killer){ described_class.new() }
10
+
11
+ describe '#kill' do
12
+ before do
13
+ allow(described_class).to receive(:check_passenger_config).and_return('custompath')
14
+ end
15
+
16
+ around do |example|
17
+ prev = WorkerKiller.configuration
18
+ WorkerKiller.configuration = config
19
+ example.run
20
+ ensure
21
+ WorkerKiller.configuration = prev
22
+ end
23
+
24
+ it 'expect right signal order' do
25
+ expect(Kernel).to receive(:system).with("custompath detach-process #{Process.pid}").and_return(true)
26
+ expect(Process).to receive(:kill).with(:KILL, anything).exactly(5).times
27
+
28
+ thread = killer.kill(Time.now)
29
+ thread.join
30
+
31
+ 1.times { killer.kill(Time.now) } # 1 QUIT
32
+ 2.times { killer.kill(Time.now) } # 1 TERM
33
+ 5.times { killer.kill(Time.now) } # 5 KILL
34
+ end
35
+ end
36
+
37
+ describe '#check_passenger_config!' do
38
+ it do
39
+ expect{ described_class.check_passenger_config!('nonenone') }.to raise_error(/Can't find passenger/)
40
+ end
41
+ end
42
+ end
43
+
@@ -0,0 +1,33 @@
1
+ RSpec.describe WorkerKiller::Killer::Signal do
2
+ let(:config) do
3
+ WorkerKiller::Configuration.new.tap do |c|
4
+ c.quit_attempts = 2
5
+ c.term_attempts = 2
6
+ end
7
+ end
8
+
9
+ let(:killer){ described_class.new() }
10
+
11
+ describe '#kill' do
12
+ context 'with use_quit TRUE' do
13
+ around do |example|
14
+ prev = WorkerKiller.configuration
15
+ WorkerKiller.configuration = config
16
+ example.run
17
+ ensure
18
+ WorkerKiller.configuration = prev
19
+ end
20
+
21
+ it 'expect right signal order' do
22
+ expect(Process).to receive(:kill).with(:QUIT, anything).exactly(1).times
23
+ expect(Process).to receive(:kill).with(:TERM, anything).exactly(1).times
24
+ expect(Process).to receive(:kill).with(:KILL, anything).exactly(1).times
25
+
26
+ 2.times { killer.kill(Time.now) } # 1 QUIT
27
+ 2.times { killer.kill(Time.now) } # 1 TERM
28
+ 5.times { killer.kill(Time.now) } # 1 KILL
29
+ end
30
+ end
31
+ end
32
+ end
33
+
@@ -0,0 +1,34 @@
1
+ RSpec.describe WorkerKiller::Killer::Base do
2
+ let(:config) do
3
+ WorkerKiller::Configuration.new.tap do |c|
4
+ c.quit_attempts = 2
5
+ c.term_attempts = 2
6
+ end
7
+ end
8
+
9
+ let(:killer){ described_class.new() }
10
+
11
+ describe '#kill' do
12
+ context 'with use_quit TRUE' do
13
+ around do |example|
14
+ prev = WorkerKiller.configuration
15
+ WorkerKiller.configuration = config
16
+ example.run
17
+ ensure
18
+ WorkerKiller.configuration = prev
19
+ end
20
+
21
+ it 'expect right signal order' do
22
+ expect(killer).to receive(:do_kill).with(:QUIT, anything, anything, anything).exactly(2).times
23
+ expect(killer).to receive(:do_kill).with(:TERM, anything, anything, anything).exactly(2).times
24
+ expect(killer).to receive(:do_kill).with(:KILL, anything, anything, anything).exactly(5).times
25
+
26
+ 2.times { killer.kill(Time.now) } # 2 QUIT
27
+ 2.times { killer.kill(Time.now) } # 2 TERM
28
+ 5.times { killer.kill(Time.now) } # other - KILL
29
+ end
30
+ end
31
+
32
+ end
33
+ end
34
+
@@ -6,7 +6,7 @@ RSpec.describe WorkerKiller::MemoryLimiter do
6
6
  let(:min){ rand(50..100) * mb }
7
7
  let(:max){ min + rand(100) * mb }
8
8
  let(:check_cycle){ 5 }
9
- let(:options){ { min: min, max: max, check_cycle: check_cycle } }
9
+ let(:options){ { min: min, max: max, check_cycle: check_cycle, verbose: true} }
10
10
 
11
11
  it { is_expected.to have_attributes(min: min, max: max, limit: a_value_between(min, max)) }
12
12
 
@@ -19,10 +19,7 @@ RSpec.describe WorkerKiller::MemoryLimiter do
19
19
  it 'expect to skip check while less than cycle count' do
20
20
  expect(GetProcessMem).not_to receive(:new)
21
21
 
22
- expect do |b|
23
- subject.reaction = b.to_proc
24
- skip_cycles(subject, check_cycle)
25
- end.not_to yield_control
22
+ skip_cycles(subject, check_cycle)
26
23
  end
27
24
 
28
25
  it 'expect to skip check after cycle count reached' do
@@ -30,11 +27,8 @@ RSpec.describe WorkerKiller::MemoryLimiter do
30
27
  expect(memory).to receive(:bytes).and_return(min - 1)
31
28
  expect(GetProcessMem).to receive(:new).and_return(memory)
32
29
 
33
- expect do |b|
34
- subject.reaction = b.to_proc
35
- skip_cycles(subject, check_cycle)
36
- expect(subject.check).to be_falsey
37
- end.not_to yield_control
30
+ skip_cycles(subject, check_cycle)
31
+ expect(subject.check).to be_falsey
38
32
  end
39
33
 
40
34
  it 'expect call reaction when check succeded' do
@@ -42,11 +36,8 @@ RSpec.describe WorkerKiller::MemoryLimiter do
42
36
  expect(memory).to receive(:bytes).and_return(subject.limit + 1)
43
37
  expect(GetProcessMem).to receive(:new).and_return(memory)
44
38
 
45
- expect do |b|
46
- subject.reaction = b.to_proc
47
- skip_cycles(subject, check_cycle)
48
- expect(subject.check).to be_truthy
49
- end.to yield_control.exactly(1).times
39
+ skip_cycles(subject, check_cycle)
40
+ expect(subject.check).to be_truthy
50
41
  end
51
42
  end
52
43
 
@@ -1,44 +1,79 @@
1
1
  require 'securerandom'
2
2
 
3
3
  RSpec.describe WorkerKiller::Middleware do
4
- let(:logger){ Logger.new(nil) }
5
-
6
- let(:app){ double }
7
- let(:reaction){ ->{} }
4
+ let(:app){ double(call: {}) }
5
+ let(:killer) { double }
6
+ let(:reaction){ double }
8
7
  let(:anykey){ SecureRandom.hex(8) }
9
8
 
10
9
  describe 'Custom class' do
11
10
  let(:klass){ double }
12
- let(:options){ { klass: klass, reaction: reaction, anykey: anykey } }
11
+ let(:options){ { killer: killer, klass: klass, reaction: reaction, anykey: anykey } }
13
12
  subject{ described_class.new(app, options) }
14
13
 
15
14
  it 'is expected to be initialized' do
16
15
  expect(klass).to receive(:new).with(anykey: anykey).and_return(99)
17
16
  expect(subject.limiter).to eq(99)
17
+ expect(subject.killer).to eq(killer)
18
18
  end
19
19
  end
20
20
 
21
21
  describe WorkerKiller::Middleware::RequestsLimiter do
22
- let(:options){ { reaction: reaction, min: 1111 } }
22
+ let(:options){ { killer: killer, min: 3, max: 3 } }
23
23
  subject{ described_class.new(app, options) }
24
24
 
25
+ it 'is expected to be initialized without reaction' do
26
+ expect(WorkerKiller::CountLimiter).to receive(:new).with(min: 3, max: 3).and_call_original
27
+ expect(subject.limiter).to be_an(WorkerKiller::CountLimiter)
28
+ expect(subject.limiter.min).to eq(3)
29
+ expect(subject.limiter.max).to eq(3)
30
+ expect(killer).to receive(:kill).with(Time).twice
31
+
32
+ 4.times do
33
+ subject.call({})
34
+ end
35
+ end
36
+
25
37
  it 'is expected to be initialized with reaction' do
26
- expect(WorkerKiller::CountLimiter).to receive(:new).with(min: 1111).and_call_original
38
+ options[:reaction] = reaction
39
+
40
+ expect(WorkerKiller::CountLimiter).to receive(:new).with(min: 3, max: 3).and_call_original
27
41
  expect(subject.limiter).to be_an(WorkerKiller::CountLimiter)
28
- expect(subject.limiter.min).to eq(1111)
29
- expect(subject.limiter.reaction).to eq(reaction)
42
+ expect(subject.limiter.min).to eq(3)
43
+ expect(subject.limiter.max).to eq(3)
44
+ expect(reaction).to receive(:call).with(subject.limiter, killer).twice
45
+
46
+ 4.times do
47
+ subject.call({})
48
+ end
30
49
  end
31
50
  end
32
51
 
33
52
  describe WorkerKiller::Middleware::OOMLimiter do
34
- let(:options){ { reaction: reaction, min: 2222 } }
53
+ let(:options){ { killer: killer, min: 2222, max: 2223 } }
35
54
  subject{ described_class.new(app, options) }
36
55
 
56
+ it 'is expected to be initialized without reaction' do
57
+ expect(WorkerKiller::MemoryLimiter).to receive(:new).with(min: 2222, max: 2223).and_call_original
58
+ expect(subject.limiter).to be_an(WorkerKiller::MemoryLimiter)
59
+ expect(subject.limiter.min).to eq(2222)
60
+ expect(subject.limiter.max).to eq(2223)
61
+ expect(killer).to receive(:kill).with(subject.limiter.started_at).once
62
+ expect(subject.limiter).to receive(:check).and_return(true)
63
+
64
+ subject.call({})
65
+ end
66
+
37
67
  it 'is expected to be initialized with reaction' do
38
- expect(WorkerKiller::MemoryLimiter).to receive(:new).with(min: 2222).and_call_original
68
+ options[:reaction] = reaction
69
+ expect(WorkerKiller::MemoryLimiter).to receive(:new).with(min: 2222, max: 2223).and_call_original
39
70
  expect(subject.limiter).to be_an(WorkerKiller::MemoryLimiter)
40
71
  expect(subject.limiter.min).to eq(2222)
41
- expect(subject.limiter.reaction).to eq(reaction)
72
+ expect(subject.limiter.max).to eq(2223)
73
+ expect(reaction).to receive(:call).with(subject.limiter, killer)
74
+ expect(subject.limiter).to receive(:check).and_return(true)
75
+
76
+ subject.call({})
42
77
  end
43
78
  end
44
79
  end
@@ -29,6 +29,7 @@ SimpleCov.formatter = SimpleCov::Formatter::MultiFormatter.new([
29
29
  SimpleCov.start
30
30
 
31
31
  require 'worker_killer'
32
+ require 'worker_killer/delayed_job_plugin'
32
33
 
33
34
  $root = File.join(File.dirname(__dir__), 'spec')
34
35
  Dir[File.join(__dir__, 'support', '**', '*.rb')].sort.each {|f| require f }
@@ -0,0 +1,12 @@
1
+ RSpec.configure do |config|
2
+ config.before(:suite) do
3
+ $logger = Logger.new(STDOUT).tap do |logger|
4
+ logger.progname = 'WorkerKiller'
5
+ logger.level = 'ERROR'
6
+ end
7
+ WorkerKiller.configure do |cfg|
8
+ cfg.logger = $logger
9
+ end
10
+ end
11
+ end
12
+
@@ -1,15 +1,5 @@
1
1
  RSpec.describe WorkerKiller do
2
2
  let(:logger){ Logger.new(nil) }
3
- let(:config) do
4
- WorkerKiller::Configuration.new.tap do |c|
5
- c.quit_attempts = 2
6
- c.kill_attempts = 2
7
- end
8
- end
9
-
10
- before do
11
- WorkerKiller.instance_variable_set('@kill_attempts', nil)
12
- end
13
3
 
14
4
  describe '#randomize' do
15
5
  [1, 5, 25, 125, 5000, 10_000].each do |max|
@@ -22,45 +12,5 @@ RSpec.describe WorkerKiller do
22
12
  end
23
13
  end
24
14
  end
25
-
26
- describe '#kill_self' do
27
- context 'with use_quit TRUE' do
28
- around do |example|
29
- prev = WorkerKiller.configuration
30
- config.use_quit = true
31
- WorkerKiller.configuration = config
32
- example.run
33
- ensure
34
- WorkerKiller.configuration = prev
35
- end
36
-
37
- it 'expect right signal order' do
38
- expect(Process).to receive(:kill).with(:QUIT, anything).exactly(2).times
39
- expect(Process).to receive(:kill).with(:TERM, anything).exactly(2).times
40
- expect(Process).to receive(:kill).with(:KILL, anything).exactly(5).times
41
- 2.times { WorkerKiller.kill_self(logger, Time.now) } # 2 QUIT
42
- 2.times { WorkerKiller.kill_self(logger, Time.now) } # 2 TERM
43
- 5.times { WorkerKiller.kill_self(logger, Time.now) } # other - KILL
44
- end
45
- end
46
-
47
- context 'with use_quit FALSE' do
48
- around do |example|
49
- prev = WorkerKiller.configuration
50
- config.use_quit = false
51
- WorkerKiller.configuration = config
52
- example.run
53
- ensure
54
- WorkerKiller.configuration = prev
55
- end
56
-
57
- it 'expect right signal order' do
58
- expect(Process).to receive(:kill).with(:TERM, anything).exactly(2).times
59
- expect(Process).to receive(:kill).with(:KILL, anything).exactly(5).times
60
- 2.times { WorkerKiller.kill_self(logger, Time.now) } # 2 TERM
61
- 5.times { WorkerKiller.kill_self(logger, Time.now) } # other - KILL
62
- end
63
- end
64
- end
65
15
  end
66
16
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: worker_killer
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0.19842
4
+ version: 1.0.0.39839
5
5
  platform: ruby
6
6
  authors:
7
7
  - Samoilenko Yuri
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-11-28 00:00:00.000000000 Z
11
+ date: 2020-10-28 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: get_process_mem
@@ -127,13 +127,23 @@ files:
127
127
  - lib/worker_killer.rb
128
128
  - lib/worker_killer/configuration.rb
129
129
  - lib/worker_killer/count_limiter.rb
130
+ - lib/worker_killer/delayed_job_plugin.rb
131
+ - lib/worker_killer/killer.rb
132
+ - lib/worker_killer/killer/delayed_job.rb
133
+ - lib/worker_killer/killer/passenger.rb
134
+ - lib/worker_killer/killer/signal.rb
130
135
  - lib/worker_killer/memory_limiter.rb
131
136
  - lib/worker_killer/middleware.rb
132
137
  - lib/worker_killer/version.rb
133
138
  - spec/count_limiter_spec.rb
139
+ - spec/killer/delayed_job_spec.rb
140
+ - spec/killer/passenger_spec.rb
141
+ - spec/killer/signal_spec.rb
142
+ - spec/killer_spec.rb
134
143
  - spec/memory_limiter_spec.rb
135
144
  - spec/middleware_spec.rb
136
145
  - spec/spec_helper.rb
146
+ - spec/support/logger.rb
137
147
  - spec/worker_killer_spec.rb
138
148
  homepage: https://github.com/RnD-Soft/worker_killer
139
149
  licenses:
@@ -159,8 +169,13 @@ signing_key:
159
169
  specification_version: 4
160
170
  summary: Kill any workers by memory and request counts or take custom reaction
161
171
  test_files:
172
+ - spec/support/logger.rb
173
+ - spec/count_limiter_spec.rb
174
+ - spec/worker_killer_spec.rb
175
+ - spec/killer/passenger_spec.rb
176
+ - spec/killer/delayed_job_spec.rb
177
+ - spec/killer/signal_spec.rb
162
178
  - spec/middleware_spec.rb
163
179
  - spec/spec_helper.rb
164
180
  - spec/memory_limiter_spec.rb
165
- - spec/worker_killer_spec.rb
166
- - spec/count_limiter_spec.rb
181
+ - spec/killer_spec.rb