serverengine 1.5.11 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 9d0f894b59ef81b2998f4d6ab09a3c84f8c4e7c1
-  data.tar.gz: 56a730fe75ba58f16c783c61834342f6074e9cbc
+  metadata.gz: 104467c419789df193725e70a8d3de43e6351c73
+  data.tar.gz: 0d10ddbca870bb40408d8bf56e9b2d52c024636d
 SHA512:
-  metadata.gz: cd028d040dd71a518538363ccaa641ce96e9a83a800067ae268e34a48aad7327dbc19301294b89d6d9d0e9fb17acf3b905aaa5d3c0c41c6d3d89a959accc3515
-  data.tar.gz: 44e2e9ee06c2168d7758507385f4e2ad04caf91248814f8297e05be53ac4c3c710a64390a83a89379422de33190435925f94aa7ac8fe355700107ba26fcac4a1
+  metadata.gz: 8d33c621c6bc34a2029f100bfd61cf1a5f56c5bf81446702e1e28e2e814515ca312e5267816384b4e000da7a48a2977178297275018847fa4ef356cf650fe361
+  data.tar.gz: 334a1f7311e02f4d37b660ff5cfe832d17fd2f755120c04c509f2997b77186b28581614c5ba13193b0affb1afe7a0da6b9cd4bde2dde4984e8d6daf1839eb94c

data/.travis.yml

@@ -1,9 +1,6 @@
 language: ruby
 
 rvm:
-  - 1.9.3
-  - 2.0
-  - 2.1
   - 2.2
 
 branches:
@@ -11,3 +8,6 @@ branches:
     - master
 
 script: bundle exec rake spec
+
+before_install:
+ - gem update bundler

data/Changelog

@@ -1,3 +1,12 @@
+2015-01-07 version 1.6.0:
+
+* Added SocketManager, a utility class for multiprocess servers to listen on
+  the same TCP or UDP port dynamically.
+* Added a new attr_reader accessor at Daemon#server and Supervisor#server
+* Added ServerEngine.windows? method to check Windows platform
+* ProcessManager now considers Windows platform
+
+
 2015-09-28 version 1.5.11:
 
 * Fix unexpected logger option handling [#22]

data/README.md

@@ -18,16 +18,27 @@ ServerEngine is a framework to implement robust multiprocess servers like Unicor
                                                   ---+
 ```
 
-**Other features:**
-
-- logging and log rotation
-- signal handlers
-- stacktrace and heap dump on signal
-- chuser, chgroup and chumask
-- changing process names
-
-
-## API
+ServerEngine also provides useful options and utilities such as logging, signal handlers, changing process names shown by `ps` command, chuser, stacktrace and heap dump on signal.
+
+* [Examples](#examples)
+  * [Simplest server](#simplest-server)
+  * [Multiprocess server](#multiprocess-server)
+  * [Multiprocess TCP server](#multiprocess-tcp-server)
+  * [Multiprocess server on Windows and JRuby platforms](#multiprocess-server-on-windows-and-jruby-platforms)
+* [Logging](#logging)
+* [Supervisor auto restart](#supervisor-auto-restart)
+* [Live restart](#live-restart)
+* [Dynamic config reloading](#dynamic-config-reloading)
+* [Signals](#signals)
+* [Utilities](#utilities)
+  * [BlockingFlag](#blockingflag)
+  * [SocketManager](#socketmanager)
+* [Module API](#module-api)
+  * [Worker module](#worker-module)
+  * [Server module](#server-module)
+* [List of all configurations](#list-of-all-configurations)
+
+## Examples
 
 ### Simplest server
 
@@ -62,7 +73,7 @@ Send `TERM` signal to kill the daemon. See also **Signals** section bellow for d
 
 ### Multiprocess server
 
-Simply set **worker_type=process** or **worker_type=thread** parameter, and set number of workers to `workers` parameter.
+Simply set **worker_type: "process"** or **worker_type: "thread"** parameter, and set number of workers to `workers` parameter.
 
 ```ruby
 se = ServerEngine.create(nil, MyWorker, {
@@ -123,12 +134,12 @@ se.run
 ```
 
 
-### Multiprocess server on Windows and JRuby platform
+### Multiprocess server on Windows and JRuby platforms
 
-Above **worker_type=process** depends on `fork` system call, which doesn't work on Windows or JRuby platform.
-ServerEngine provides **worker_type=spawn** for those platforms (This is still EXPERIMENTAL). However, unfortunately, you need to implement different worker module because `worker_type=spawn` is not compatible with **worker_type=process** in terms of API.
+Above **worker_type: "process"** depends on `fork` system call, which doesn't work on Windows or JRuby platform.
+ServerEngine provides **worker_type: "spawn"** for those platforms (This is still EXPERIMENTAL). However, unfortunately, you need to implement different worker module because `worker_type: "spawn"` is not compatible with **worker_type: "process"** in terms of API.
 
-What you need to implement at least to use worker_type=spawn is `spawn(process_manager)` method. You will call `process_manager.spawn` at the method, where `spawn` is same with `Process.spawn` excepting return value.
+What you need to implement at least to use worker_type: "spawn" is `def spawn(process_manager)` method. You will call `process_manager.spawn` at the method, where `spawn` is same with `Process.spawn` excepting return value.
 
 ```ruby
 module MyWorker
@@ -164,7 +175,7 @@ se.run
 ```
 
 
-### Logging
+## Logging
 
 ServerEngine logger rotates logs by 1MB and keeps 5 generations by default.
 
@@ -187,25 +198,25 @@ ServerEngine's default logger extends from Ruby's standard Logger library to:
 See also **Configuration** section bellow.
 
 
-### Enabling supervisor process
+## Supervisor auto restart
 
 Server programs running 24x7 hours need to survive even if a process stalled because of unexpected memory swapping or network errors.
 
-Supervisor process runs as the parent process of the server process and monitor it to restart automatically.
+Supervisor process runs as the parent process of the server process and monitor it to restart automatically. You can enable supervisor process by setting `supervisor: true` parameter:
 
 ```ruby
 se = ServerEngine.create(nil, MyWorker, {
   daemonize: true,
   pid_path: 'myserver.pid',
-  supervisor: true,  # enable supervisor process
+  supervisor: true,  # enables supervisor process
 })
 se.run
 ```
 
 
-### Live restart
+## Live restart
 
-You can restart a server process without waiting for completion of all workers using `INT` signal (`supervisor=true` and `enable_detach=true` parameters must be enabled).
+You can restart a server process without waiting for completion of all workers using `INT` signal (`supervisor: true` and `enable_detach: true` parameters must be enabled).
 This feature allows you to minimize downtime where workers take long time to complete a task.
 
 ```
@@ -238,10 +249,10 @@ This feature allows you to minimize downtime where workers take long time to com
                   +----------+    +-----------+
 ```
 
-Note that network servers (which listen sockets) shouldn't use live restart because it causes "Address already in use" error at the server process. Instead, simply use `worker_type=process` configuration and send `USR1` to restart workers instead of the server. It restarts a worker without waiting for shutdown of the other workers. This way doesn't cause downtime because server process doesn't close listening sockets and keeps accepting new clients (See also `restart_server_process` parameter if necessary).
+Note that network servers (which listen sockets) shouldn't use live restart because it causes "Address already in use" error at the server process. Instead, simply use `worker_type: "process"` configuration and send `USR1` to restart workers instead of the server. It restarts a worker without waiting for shutdown of the other workers. This way doesn't cause downtime because server process doesn't close listening sockets and keeps accepting new clients (See also `restart_server_process` parameter if necessary).
 
 
-### Dynamic configuration reloading
+## Dynamic config reloading
 
 Robust servers should not restart only to update configuration parameters.
 
@@ -280,6 +291,20 @@ se.run
 Send `USR2` signal to reload configuration file.
 
 
+## Signals
+
+- **TERM:** graceful shutdown
+- **QUIT:** immediate shutdown (available only when `worker_type` is "process")
+- **USR1:** graceful restart
+- **HUP:** immediate restart (available only when `worker_type` is "process")
+- **USR2:** reload config file and reopen log file
+- **INT:** detach process for live restarting (available only when `supervisor` and `enable_detach` parameters are true. otherwise graceful shutdown)
+- **CONT:** dump stacktrace and memory information to /tmp/sigdump-<pid>.log file
+
+Immediate shutdown and restart send SIGQUIT signal to worker processes which kills the processes.
+Graceful shutdown and restart call `Worker#stop` method and wait for completion of `Worker#run` method.
+
+
 ## Utilities
 
 ### BlockingFlag
@@ -313,11 +338,67 @@ se.run
 ```
 
 
-## Module methods
+### SocketManager
 
-### Worker module
+`ServerEngine::SocketManager` is a powerful library to listen on the same port across multiple worker processes dynamically.
+
+```ruby
+module MyServer
+  def before_run
+    @socket_manager_path = ServerEngine::SocketManager.generate_path
+    @socket_manager_server = ServerEngine::SocketManager::Server.open(@socket_manager_path)
+  end
+
+  def after_run
+    @socket_manager_server.close
+  end
+
+  attr_reader :socket_manager_path
+end
+
+module MyWorker
+  def initialize
+    @stop_flag = ServerEngine::BlockingFlag.new
+    @socket_manager = ServerEngine::SocketManager::Client.new(server.socket_manager_path)
+  end
+
+  def run
+    lsock = @socket_manager.listen_tcp('0.0.0.0', 12345)
+    until @stop
+      c = lsock.accept
+      c.write "Awesome work!"
+      c.close
+    end
+  end
 
-Available methods are different depending on `worker_type`.
+  def stop
+    @stop = true
+  end
+end
+
+se = ServerEngine.create(MyServer, MyWorker, {
+  daemonize: true,
+  log: 'myserver.log',
+  pid_path: 'myserver.pid',
+  worker_type: 'process',
+  workers: 4,
+  bind: '0.0.0.0',
+  port: 9071,
+})
+se.run
+```
+
+
+## Module API
+
+Available methods are different depending on `worker_type`. ServerEngine supports 3 worker types:
+
+- **embedded**: uses a thread to run worker module (default). This type doesn't support immediate shutdown or immediate restart.
+- **thread**: uses threads to run worker modules. This type doesn't support immediate shutdown or immediate restart.
+- **process**: uses processes to run worker modules. This type doesn't work on Windows or JRuby platform.
+- **spawn**: uses processes to run worker modules. This type works on Windows and JRuby platform but available interface of worker module is limited (See also Worker module section).
+
+### Worker module
 
 - interface
   - `initialize` is called in the parent process (or thread) in contrast to the other methods
@@ -350,31 +431,7 @@ Available methods are different depending on `worker_type`.
   - `logger` logger
 
 
-## Worker types
-
-ServerEngine supports 3 worker types:
-
-- **embedded**: uses a thread to run worker module (default). This type doesn't support immediate shutdown or immediate restart.
-- **thread**: uses threads to run worker modules. This type doesn't support immediate shutdown or immediate restart.
-- **process**: uses processes to run worker modules. This type doesn't work on Windows or JRuby platform.
-- **spawn**: uses processes to run worker modules. This type works on Windows and JRuby platform but available interface of worker module is limited (See also Worker module section).
-
-
-## Signals
-
-- **TERM:** graceful shutdown
-- **QUIT:** immediate shutdown (available only when `worker_type` is "process")
-- **USR1:** graceful restart
-- **HUP:** immediate restart (available only when `worker_type` is "process")
-- **USR2:** reload config file and reopen log file
-- **INT:** detach process for live restarting (available only when `supervisor` and `enable_detach` parameters are true. otherwise graceful shutdown)
-- **CONT:** dump stacktrace and memory information to /tmp/sigdump-<pid>.log file
-
-Immediate shutdown and restart send SIGQUIT signal to worker processes which kills the processes.
-Graceful shutdown and restart call `Worker#stop` method and wait for completion of `Worker#run` method.
-
-
-## Configuration
+## List of all configurations
 
 - Daemon
   - **daemonize** enables daemonize (default: false)

data/appveyor.yml

@@ -0,0 +1,18 @@
+---
+install:
+  - SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
+  - ruby --version
+  - gem --version
+  - bundle install
+build: off
+test_script:
+  - bundle exec rake -rdevkit
+
+environment:
+  matrix:
+    - ruby_version: "200"
+    - ruby_version: "200-x64"
+    - ruby_version: "21"
+    - ruby_version: "21-x64"
+    - ruby_version: "22"
+    - ruby_version: "22-x64"

data/lib/serverengine/config_loader.rb

@@ -58,10 +58,9 @@ module ServerEngine
     def create_logger
       if logger = @config[:logger]
         @logger = logger
-        return
+      else
+        @logger = @logger_class.new(logdev_from_config(@config), @config)
       end
-
-      @logger = @logger_class.new(logdev_from_config(@config), @config)
     end
 
     def logdev_from_config(config)

data/lib/serverengine/daemon.rb

@@ -49,6 +49,9 @@ module ServerEngine
       @chumask = @config[:chumask]
     end
 
+    # server is available when run() is called. It is a Supervisor instance if supervisor is set to true. Otherwise a Server instance.
+    attr_reader :server
+
     module Signals
       GRACEFUL_STOP = :TERM
       IMMEDIATE_STOP = :QUIT
@@ -157,7 +160,7 @@ module ServerEngine
     private
 
     def create_server(logger)
-      @create_server_proc.call(@load_config_proc, logger)
+      @server = @create_server_proc.call(@load_config_proc, logger)
     end
   end
 end

data/lib/serverengine/multi_spawn_server.rb

@@ -19,12 +19,21 @@ module ServerEngine
 
   class MultiSpawnServer < MultiWorkerServer
     def initialize(worker_module, load_config_proc={}, &block)
-      @pm = ProcessManager.new(
-        auto_tick: false,
-        graceful_kill_signal: Daemon::Signals::GRACEFUL_STOP,
-        immediate_kill_signal: Daemon::Signals::IMMEDIATE_STOP,
-        enable_heartbeat: false,
-      )
+      if ServerEngine.windows?
+        @pm = ProcessManager.new(
+          auto_tick: false,
+          graceful_kill_signal: Daemon::Signals::GRACEFUL_STOP,
+          immediate_kill_signal: false,
+          enable_heartbeat: false,
+        )
+      else
+        @pm = ProcessManager.new(
+          auto_tick: false,
+          graceful_kill_signal: Daemon::Signals::GRACEFUL_STOP,
+          immediate_kill_signal: Daemon::Signals::IMMEDIATE_STOP,
+          enable_heartbeat: false,
+        )
+      end
 
       super(worker_module, load_config_proc, &block)
 

data/lib/serverengine/process_manager.rb

@@ -34,6 +34,10 @@ module ServerEngine
       @auto_tick_interval = config[:auto_tick_interval] || 1
 
       @enable_heartbeat = !!config[:enable_heartbeat]
+      if ServerEngine.windows?
+        # heartbeat is not supported on Windows platform. See also spawn method.
+        @enable_heartbeat = false
+      end
       @auto_heartbeat = !!config.fetch(:auto_heartbeat, true)
 
       case op = config[:on_heartbeat_error]
@@ -93,6 +97,10 @@ module ServerEngine
     end
 
     def fork(&block)
+      if ServerEngine.windows?
+        raise NotImplementedError, "fork is not available on this platform. Please use spawn (worker_type: 'spawn')."
+      end
+
       rpipe, wpipe = new_pipe_pair
 
       begin
@@ -143,12 +151,14 @@ module ServerEngine
 
       # pipe is necessary even if @enable_heartbeat == false because
       # parent process detects shutdown of a child process using it
-      rpipe, wpipe = new_pipe_pair
-
       begin
-        options[[wpipe.fileno]] = wpipe
-        if @enable_heartbeat
-          env['SERVERENGINE_HEARTBEAT_PIPE'] = wpipe.fileno.to_s
+        unless ServerEngine.windows?
+          # heartbeat is not supported on Windows platform
+          rpipe, wpipe = new_pipe_pair
+          options[[wpipe.fileno]] = wpipe
+          if @enable_heartbeat
+            env['SERVERENGINE_HEARTBEAT_PIPE'] = wpipe.fileno.to_s
+          end
         end
 
         pid = Process.spawn(env, *args, options)
@@ -156,13 +166,16 @@ module ServerEngine
         m = Monitor.new(self, pid)
 
         @monitors << m
-        @rpipes[rpipe] = m
-        rpipe = nil
+
+        unless ServerEngine.windows?
+          @rpipes[rpipe] = m
+          rpipe = nil
+        end
 
         return m
 
       ensure
-        wpipe.close
+        wpipe.close if wpipe
         rpipe.close if rpipe
       end
     end
@@ -199,30 +212,34 @@ module ServerEngine
         raise AlreadyClosedError.new
       end
 
-      if @rpipes.empty?
-        sleep blocking_timeout if blocking_timeout > 0
-        return nil
-      end
+      time ||= Time.now
 
-      ready_pipes, _, _ = IO.select(@rpipes.keys, nil, nil, blocking_timeout)
+      unless ServerEngine.windows?
+        # heartbeat is not supported on Windows platform.
 
-      time ||= Time.now
+        if @rpipes.empty?
+          sleep blocking_timeout if blocking_timeout > 0
+          return nil
+        end
 
-      if ready_pipes
-        ready_pipes.each do |r|
-          begin
-            r.read_nonblock(1024, @read_buffer)
-          rescue Errno::EAGAIN, Errno::EINTR
-            next
-          rescue #EOFError
-            m = @rpipes.delete(r)
-            m.start_immediate_stop!
-            r.close rescue nil
-            next
-          end
+        ready_pipes, _, _ = IO.select(@rpipes.keys, nil, nil, blocking_timeout)
+
+        if ready_pipes
+          ready_pipes.each do |r|
+            begin
+              r.read_nonblock(1024, @read_buffer)
+            rescue Errno::EAGAIN, Errno::EINTR
+              next
+            rescue #EOFError
+              m = @rpipes.delete(r)
+              m.start_immediate_stop!
+              r.close rescue nil
+              next
+            end
 
-          if m = @rpipes[r]
-            m.last_heartbeat_time = time
+            if m = @rpipes[r]
+              m.last_heartbeat_time = time
+            end
           end
         end
       end
@@ -275,6 +292,7 @@ module ServerEngine
       end
 
       attr_accessor :last_heartbeat_time
+      attr_reader :pid
 
       def heartbeat_delay
         now = Time.now
@@ -389,7 +407,11 @@ module ServerEngine
         end
 
         begin
-          Process.kill(signal, pid)
+          if ServerEngine.windows? && (signal == :KILL || signal == :SIGKILL)
+            system("taskkill /f /pid #{pid}")
+          else
+            Process.kill(signal, pid)
+          end
         rescue #Errno::ECHILD, Errno::ESRCH, Errno::EPERM
           # assume that any errors mean the child process is dead
           @pid = nil