unicorn-worker-killer 0.3.6 → 0.4.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 472f5a54002c4f7d424ef88e613bce3b8d0b6a12
-  data.tar.gz: 50cf272d41b6cd9bbbd9d3e2218fe189c2aac5d6
+SHA256:
+  metadata.gz: 3b19012954bb3a259a52418a03a945bd4e1d0fbe682109a5c6a074f98daeeed2
+  data.tar.gz: d4882e751bae724d4d24a9194a3b76fe519d45427be55b52685a55309a82bca7
 SHA512:
-  metadata.gz: 7a5a124e1b518ebb417abc8a14f6a93202a816973a6b69073b050072193c082d071a6f6ca418b04b15b211f80327e643cccf3534452a33286df916a2aa6050bd
-  data.tar.gz: 65bb35218c266c2264d90d71c641ea872eba204a6420b98173256f67454cb301ad9170099e54e4c340c66adc2e53f54fa74d98f2bc77f334339ffb57cc59fafa
+  metadata.gz: 642bf83fc8c7ee86a2d7bfa36910418627b53701e4c53a6b61d2f2df5ce12563ff689f80189a647343677bab1bd05765bb6a05d65fcdd50a1fe9078ade066fe1
+  data.tar.gz: 36100a92f0d917fff98a7ff73bdc64bb0b4adb16b17389d7109807f65a78842d64b4dec0fd370b9571b92780002c94d91e8a2e368ac5d6b958f063bca0b15d34

data/AUTHORS

@@ -1,2 +1,3 @@
 Kazuki Ohta <kazuki.ohta _at_ gmail.com>
 Sadayuki FURUHASHI <frsyuki _at_ gmail.com>
+Jonathan Clem <jonathan _at_ jclem.net>

data/ChangeLog

@@ -1,3 +1,22 @@
+Release 0.4.5 - 2021/04/13
+  * update for Unicorn 6
+
+Release 0.4.4 - 2015/11/13
+  * update for Unicorn 5
+
+Release 0.4.3 - 2013/12/23
+  * fix for occasional 'invalid argument' error
+
+Release 0.4.2 - 2013/09/23
+  * `verbose` options provides verbose logging
+
+Release 0.4.1 - 2013/03/12
+  * Kill attempts happen once per request, not in a loop
+
+Release 0.4.0 - 2013/03/12
+  * Send signals from inside a Thread (required for QUIT signal)
+  * Fix broken if/else in kill_self
+
 Release 0.3.6 - 2013/03/09
   * do not require configuration
   * fix missing randomize() in MaxRequests killer

data/README.md

@@ -1,6 +1,6 @@
 # unicorn-worker-killer
 
-[Unicorn](http://unicorn.bogomips.org/) is widely used HTTP-server for Rack applications. One thing we thought Unicorn misssed, is killing the Unicorn workers based on the number of requests and consumed memories.
+[Unicorn](http://unicorn.bogomips.org/) is widely used HTTP-server for Rack applications. One thing we thought Unicorn missed, is killing the Unicorn workers based on the number of requests and consumed memories.
 
 `unicorn-worker-killer` gem provides automatic restart of Unicorn workers based on 1) max number of requests, and 2) process memory size (RSS), without affecting any requests. This will greatly improves site's stability by avoiding unexpected memory exhaustion at the application nodes.
 
@@ -12,7 +12,7 @@ No external process like `god` is required. Just install one gem: `unicorn-worke
 
 # Usage
 
-Add these lines to your `config.ru`.
+Add these lines to your `config.ru`. (These lines should be added above the `require ::File.expand_path('../config/environment',  __FILE__)` line.
 
     # Unicorn self-process killer
     require 'unicorn/worker_killer'
@@ -25,19 +25,25 @@ Add these lines to your `config.ru`.
 
 This gem provides two modules.
 
-### Unicorn::WorkerKiller::MaxRequests(max_requests_min=3072, max_requests_max=4096)
+### `Unicorn::WorkerKiller::MaxRequests(max_requests_min=3072, max_requests_max=4096, verbose=false)`
 
 This module automatically restarts the Unicorn workers, based on the number of requests which worker processed.
 
 `max_requests_min` and `max_requests_max` specify the min and max of maximum requests per worker. The actual limit is decided by rand() between `max_requests_min` and `max_requests_max` per worker, to prevent all workers to be dead at the same time. Once the number exceeds the limit, that worker is automatically restarted.
 
-### Unicorn::WorkerKiller::Oom(memory_limit_min=(1024**3), memory_limit_max=(2*(1024**3)), check_cycle = 16)
+If `verbose` is set to true, then after every request, your log will show the requests left before restart.  This logging is done at the `info` level.
+
+### `Unicorn::WorkerKiller::Oom(memory_limit_min=(1024\*\*3), memory_limit_max=(2\*(1024\*\*3)), check_cycle = 16, verbose = false)`
 
 This module automatically restarts the Unicorn workers, based on its memory size.
 
-`memory_limit_min` and `memory_limit_max` specify the min and max of maximum memory per worker. The actual limit is decided by rand() between `memory_limit_min` and `memory_limit_max` per worker, to prevent all workers to be dead at the same time.  Once the memory size exceeds `memory_size`, that worker is automatically restarted.
+`memory_limit_min` and `memory_limit_max` specify the min and max of maximum memory in bytes per worker. The actual limit is decided by rand() between `memory_limit_min` and `memory_limit_max` per worker, to prevent all workers to be dead at the same time.  Once the memory size exceeds `memory_size`, that worker is automatically restarted.
 
 The memory size check is done in every `check_cycle` requests.
 
-# TODO
-- Get RSS (Resident Set Size) without forking the child process at Mac OS and Windows
+If `verbose` is set to true, then every memory size check will be shown in your logs.   This logging is done at the `info` level.
+
+# Special Thanks
+
+- [@hotchpotch](http://github.com/hotchpotch/) for the [original idea](https://gist.github.com/hotchpotch/1258681)
+

data/VERSION

@@ -1 +1 @@
-0.3.6
+0.4.5

data/lib/unicorn/worker_killer.rb

@@ -1,32 +1,30 @@
-require 'unicorn/configuration'
+require 'unicorn'
+require 'unicorn/worker_killer/configuration'
+require 'get_process_mem'
 
 module Unicorn::WorkerKiller
   class << self
     attr_accessor :configuration
   end
 
-  # Self-destruction by sending the signals to myself. The process sometimes
-  # doesn't terminate by SIGQUIT, so this tries to send SIGTERM and SIGKILL
-  # if it doesn't finish immediately.
+  # Kill the current process by telling it to send signals to itself. If
+  # the process isn't killed after `configuration.max_quit` QUIT signals,
+  # send TERM signals until `configuration.max_term`. Finally, send a KILL
+  # signal. A single signal is sent per request.
   # @see http://unicorn.bogomips.org/SIGNALS.html
   def self.kill_self(logger, start_time)
-    alive_sec = (Time.now - start_time).to_i
-
-    i = 0
-    while true
-      i += 1
-      sig = :QUIT
-      if i > configuration.max_quit
-        sig = :TERM
-      elsif i > configuration.max_term
-        sig = :KILL
-      end
+    alive_sec = (Time.now - start_time).round
+    worker_pid = Process.pid
 
-      logger.warn "#{self} send SIG#{sig} (pid: #{Process.pid}) alive: #{alive_sec} sec (trial #{i})"
-      Process.kill sig, Process.pid
+    @@kill_attempts ||= 0
+    @@kill_attempts += 1
 
-      sleep configuration.sleep_interval
-    end
+    sig = :QUIT
+    sig = :TERM if @@kill_attempts > configuration.max_quit
+    sig = :KILL if @@kill_attempts > configuration.max_term
+
+    logger.warn "#{self} send SIG#{sig} (pid: #{worker_pid}) alive: #{alive_sec} sec (trial #{@@kill_attempts})"
+    Process.kill sig, worker_pid
   end
 
   module Oom
@@ -35,19 +33,20 @@ module Unicorn::WorkerKiller
     # affect the request.
     #
     # @see https://github.com/defunkt/unicorn/blob/master/lib/unicorn/oob_gc.rb#L40
-    def self.new(app, memory_limit_min = (1024**3), memory_limit_max = (2*(1024**3)), check_cycle = 16)
+    def self.new(app, memory_limit_min = (1024**3), memory_limit_max = (2*(1024**3)), check_cycle = 16, verbose = false)
       ObjectSpace.each_object(Unicorn::HttpServer) do |s|
         s.extend(self)
         s.instance_variable_set(:@_worker_memory_limit_min, memory_limit_min)
         s.instance_variable_set(:@_worker_memory_limit_max, memory_limit_max)
         s.instance_variable_set(:@_worker_check_cycle, check_cycle)
         s.instance_variable_set(:@_worker_check_count, 0)
+        s.instance_variable_set(:@_verbose, verbose)
       end
       app # pretend to be Rack middleware since it was in the past
     end
 
     def randomize(integer)
-      RUBY_VERSION > "1.9" ? Random.rand(integer) : rand(integer)
+      RUBY_VERSION > "1.9" ? Random.rand(integer.abs) : rand(integer)
     end
 
     def process_client(client)
@@ -58,7 +57,8 @@ module Unicorn::WorkerKiller
       @_worker_memory_limit ||= @_worker_memory_limit_min + randomize(@_worker_memory_limit_max - @_worker_memory_limit_min + 1)
       @_worker_check_count += 1
       if @_worker_check_count % @_worker_check_cycle == 0
-        rss = _worker_rss()
+        rss = GetProcessMem.new.bytes
+        logger.info "#{self}: worker (pid: #{Process.pid}) using #{rss} bytes." if @_verbose
         if rss > @_worker_memory_limit
           logger.warn "#{self}: worker (pid: #{Process.pid}) exceeds memory limit (#{rss} bytes > #{@_worker_memory_limit} bytes)"
           Unicorn::WorkerKiller.kill_self(logger, @_worker_process_start)
@@ -66,38 +66,6 @@ module Unicorn::WorkerKiller
         @_worker_check_count = 0
       end
     end
-
-    private
-
-    def _worker_rss
-      proc_status = "/proc/#{Process.pid}/status"
-      if File.exists? proc_status
-        open(proc_status).each_line { |l|
-          if l.include? 'VmRSS'
-            ls = l.split
-            if ls.length == 3
-              value = ls[1].to_i
-              unit = ls[2]
-              case unit.downcase
-              when 'kb'
-                return value*(1024**1)
-              when 'mb'
-                return value*(1024**2)
-              when 'gb'
-                return value*(1024**3)
-              end
-            end
-          end
-        }
-      end
-
-      # Forking the child process sometimes fails under low memory condition.
-      # It would be ideal for not forking the process to get RSS. For Linux,
-      # this module reads '/proc/<pid>/status' to get RSS, but not for other
-      # environments (e.g. MacOS and Windows).
-      kb = `ps -o rss= -p #{Process.pid}`.to_i
-      return kb * 1024
-    end
   end
 
   module MaxRequests
@@ -106,17 +74,19 @@ module Unicorn::WorkerKiller
     # affect the request.
     #
     # @see https://github.com/defunkt/unicorn/blob/master/lib/unicorn/oob_gc.rb#L40
-    def self.new(app, max_requests_min = 3072, max_requests_max = 4096)
+    def self.new(app, max_requests_min = 3072, max_requests_max = 4096, verbose = false)
       ObjectSpace.each_object(Unicorn::HttpServer) do |s|
         s.extend(self)
         s.instance_variable_set(:@_worker_max_requests_min, max_requests_min)
         s.instance_variable_set(:@_worker_max_requests_max, max_requests_max)
+        s.instance_variable_set(:@_verbose, verbose)
       end
+
       app # pretend to be Rack middleware since it was in the past
     end
 
     def randomize(integer)
-      RUBY_VERSION > "1.9" ? Random.rand(integer) : rand(integer)
+      RUBY_VERSION > "1.9" ? Random.rand(integer.abs) : rand(integer)
     end
 
     def process_client(client)
@@ -126,6 +96,8 @@ module Unicorn::WorkerKiller
       @_worker_process_start ||= Time.now
       @_worker_cur_requests ||= @_worker_max_requests_min + randomize(@_worker_max_requests_max - @_worker_max_requests_min + 1)
       @_worker_max_requests ||= @_worker_cur_requests
+      logger.info "#{self}: worker (pid: #{Process.pid}) has #{@_worker_cur_requests} left before being killed" if @_verbose
+
       if (@_worker_cur_requests -= 1) <= 0
         logger.warn "#{self}: worker (pid: #{Process.pid}) exceeds max number of requests (limit: #{@_worker_max_requests})"
         Unicorn::WorkerKiller.kill_self(logger, @_worker_process_start)

data/unicorn-worker-killer.gemspec

@@ -7,14 +7,16 @@ Gem::Specification.new do |gem|
   gem.homepage    = "https://github.com/kzk/unicorn-worker-killer"
   gem.summary     = gem.description
   gem.version     = File.read("VERSION").strip
-  gem.authors     = ["Kazuki Ohta", "Sadayuki Furuhashi"]
-  gem.email       = ["kazuki.ohta@gmail.com", "frsyuki@gmail.com"]
+  gem.authors     = ["Kazuki Ohta", "Sadayuki Furuhashi", "Jonathan Clem"]
+  gem.email       = ["kazuki.ohta@gmail.com", "frsyuki@gmail.com", "jonathan@jclem.net"]
   gem.has_rdoc    = false
   #gem.platform    = Gem::Platform::RUBY
   gem.files       = `git ls-files`.split("\n")
   gem.test_files  = `git ls-files -- {test,spec,features}/*`.split("\n")
   gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  gem.licenses    = ["GPLv2+", "Ruby 1.8"]
   gem.require_paths = ['lib']
-  gem.add_dependency "unicorn", "~> 4"
+  gem.add_dependency "unicorn",         [">= 4", "< 7"]
+  gem.add_dependency "get_process_mem", "~> 0"
   gem.add_development_dependency "rake", ">= 0.9.2"
 end

metadata

@@ -1,48 +1,70 @@
 --- !ruby/object:Gem::Specification
 name: unicorn-worker-killer
 version: !ruby/object:Gem::Version
-  version: 0.3.6
+  version: 0.4.5
 platform: ruby
 authors:
 - Kazuki Ohta
 - Sadayuki Furuhashi
+- Jonathan Clem
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-03-09 00:00:00.000000000 Z
+date: 2021-04-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: unicorn
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - ">="
       - !ruby/object:Gem::Version
         version: '4'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '7'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - ">="
       - !ruby/object:Gem::Version
         version: '4'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '7'
+- !ruby/object:Gem::Dependency
+  name: get_process_mem
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: 0.9.2
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: 0.9.2
 description: Kill unicorn workers by memory and request counts
 email:
 - kazuki.ohta@gmail.com
 - frsyuki@gmail.com
+- jonathan@jclem.net
 executables: []
 extensions: []
 extra_rdoc_files: []
@@ -54,11 +76,13 @@ files:
 - README.md
 - Rakefile
 - VERSION
-- lib/unicorn/configuration.rb
 - lib/unicorn/worker_killer.rb
+- lib/unicorn/worker_killer/configuration.rb
 - unicorn-worker-killer.gemspec
 homepage: https://github.com/kzk/unicorn-worker-killer
-licenses: []
+licenses:
+- GPLv2+
+- Ruby 1.8
 metadata: {}
 post_install_message: 
 rdoc_options: []
@@ -66,17 +90,16 @@ require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.0.0
+rubygems_version: 3.0.8
 signing_key: 
 specification_version: 4
 summary: Kill unicorn workers by memory and request counts