kinetic 0.0.3 → 0.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: f720debb00d4ab62ce3b3e8d7e4f7eaf422be407
4
- data.tar.gz: a01b254919820ca788f0aa9ee8d42c06a7a7ca06
3
+ metadata.gz: 0fdcf06cf8d1fc5fff905da13c55c97c5c30f6bc
4
+ data.tar.gz: ff4a1483557bb7abd8863a87000795b3648ad0db
5
5
  SHA512:
6
- metadata.gz: 6243b0b4ae9062a16aeda4bf06626088fd0cfaf80ed2de1b4a385f827ff491c063a69e168ef90cd46d35ba8f6b0ca0a1909a261c99670dd38634b730ede822d2
7
- data.tar.gz: be6f7059f4fb70d65e6d1cab62357d9f082f783a249e3e7e40694d93de2a7cf71b182795d26d184d26588ab524d93d2aab6b0c4535723013a095cfcf4fbae731
6
+ metadata.gz: 84695516ec05676070029529c68f77aaafd9a72204f329121cbf39eca176a6fe8546074b8b3072d867ce4820ed8bb6817360355e57eba23090e3a790c6906ea5
7
+ data.tar.gz: 40ad8453c43d7e53a00f0552dfb703329a946e41085211694cf1432a8f94b349d8545734ede32f73b1b9757af7b023169c3054c773b480617b965b89d57a81f7
data/.gitignore CHANGED
@@ -19,3 +19,4 @@ tmp
19
19
  *.iml
20
20
  doc/**
21
21
  *.pid
22
+ *.log
data/.ruby-version ADDED
@@ -0,0 +1 @@
1
+ 2.0.0-p353
data/README.md CHANGED
@@ -1,8 +1,7 @@
1
1
  # Kinetic
2
2
 
3
3
  [<img src="https://secure.travis-ci.org/fugufish/kinetic.png">](http://travis-ci.org/fugufish/kinetic)
4
- [![Code Climate](https://codeclimate.com/repos/52b06fa413d637197401ebb7/badges/c71d7f47797b4caec5ae/gpa.png)](https://codeclimate.com/repos/52b06fa413d637197401ebb7/feed)
5
-
4
+ [<img src="https://codeclimate.com/repos/52b06fa413d637197401ebb7/badges/c71d7f47797b4caec5ae/gpa.png">](https://codeclimate.com/repos/52b06fa413d637197401ebb7/feed)
6
5
 
7
6
  Kinetic is an AMQP worker framework designed in vein of microframeworks such as sinatra. Its goal is to provide an
8
7
  easy way to configure and run AMQP consumers in order to reduce developer overhead when working with AMQP. Kinetic
@@ -32,34 +31,72 @@ the routing key 'message'. By default the message is expected to be in JSON form
32
31
  serializers may be available.
33
32
 
34
33
  ## Configuration
35
- Kinetic configuration follows a similar DSL to that of sinatra. Configuration can be set by calling the `set` method,
36
- and can be retrieved by referencing the `config` variable or the `get` or `get!` methods.
34
+ Kinetic configuration follows a similar DSL to that of sinatra in that it allows the use of the {Kinetic::DSL#set set}
35
+ method to set configuration. It also allows a configuration file to be loaded by using
36
+ {Kinetic::DSL#config_file config_file}.
37
+
38
+ ### Using {Kinetic::DSL#set set}
37
39
 
38
- ### set
39
- To set a configuration value use set:
40
+ Simply pass in the key and value to the {Kinetic::DSL#set set} method.
40
41
 
41
42
  ```ruby
42
- set :host, 'localhost'
43
+ require 'kinetic'
43
44
 
44
- on 'message' do |message|
45
- puts "Received #{message}"
46
- end
45
+ set host: 'localhost'
46
+ set port: 5672
47
+ ```
48
+
49
+ ### Using {Kinetic::DSL#config_file config_file}
50
+
51
+ You can add support to your Kinetic app for file based configuration by defining a Yaml based config file. Note that
52
+ the {Kinetic::DSL#config_file config_file} directive accepts any path. Relative paths will be relative to the
53
+ application file's root directory.
54
+
55
+ ```ruby
56
+ require 'kinetic'
57
+
58
+ config_file 'config/config.yml'
47
59
  ```
48
60
 
49
61
  ### Defaults
50
- [Kinetic::Configuration](http://rdoc.info/github/fugufish/kinetic/master/Kinetic/Configuration)
51
62
 
63
+ ```ruby
64
+ {
65
+ name: File.basename($0), # name of the application. This also translates to the name of the AMQP exchange
66
+ root: File.dirname($0), # the application root directory
67
+ app_file: $0, # the application file
68
+ workers: 1, # the number of worker processes to run
69
+ log_file: 'kinetic.log', # the path of the log file to be used
70
+ host: 'localhost', # the AMQP host
71
+ port: 5672, # the AMQP port
72
+ serializer: :JSON, # the message serializer to use
73
+ timeout: 1, # the time (in seconds) to wait for various operations
74
+ pid: "#{File.basename($0)}.pid" # the process pid file
75
+ }
76
+ ```
77
+
78
+ ### Accessing
52
79
 
80
+ Accessing configuration values by their method accessors will raise {Kinetic::Errors::MissingConfigurationValue}
81
+ error. This can be used to ensure that specific values are set at runtime. To access a value without raising an
82
+ exception use the hash key instead.
83
+
84
+ ```ruby
85
+ config.host # raises an error if the config host value is not set
86
+ config[:host] # returns nil if the host value is not set
87
+ ```
53
88
 
54
89
  ## Changelog
55
90
 
91
+ ### 0.0.4
92
+ * removed `get` and `get!` methods as they are no longer needed.
93
+ * added ability to correctly daemonize process
94
+
56
95
  ### 0.0.3
57
96
  * Fix bug where process shut down does not kill children
58
97
  * Add ability to set process name for master and workers
59
98
  * Add after fork block capability allowing code to be written to run after the worker forks
60
99
 
61
- ### 0.0.1
62
- * Initial release
63
100
 
64
101
 
65
102
 
data/lib/kinetic/cli.rb CHANGED
@@ -1,4 +1,4 @@
1
- require 'kinetic/base'
1
+ require 'optparse'
2
2
  module Kinetic
3
3
  module CLI
4
4
 
@@ -8,24 +8,20 @@ module Kinetic
8
8
  opts.banner = "Usage #{File.basename($0)} (#{COMMANDS.join('|')}) [options]"
9
9
  end
10
10
 
11
- def self.parse
12
- command = nil
13
- if ARGV.empty? || (ARGV[0] && COMMANDS.include?(ARGV[0]))
14
- command = ARGV.shift
15
- else
16
- puts PARSER.banner
17
- exit! 1
18
- end
19
- command || 'run'
20
- end
21
-
22
11
  end
23
12
 
24
- class Application < Base
25
- end
13
+ if ARGV.empty? || (ARGV[0] && CLI::COMMANDS.include?(ARGV[0]))
14
+ require 'kinetic/version'
15
+ require 'kinetic/base'
26
16
 
17
+ class Application < Base
18
+ end
27
19
 
28
- at_exit { Master.new(Application.new).send(CLI.parse) if $!.nil? }
20
+ at_exit { Master.new(Application.new).send(ARGV[0] || 'run') if $!.nil? }
21
+ else
22
+ puts CLI::PARSER.banner
23
+ exit(1)
24
+ end
29
25
 
30
26
  end
31
27
 
@@ -1,16 +1,35 @@
1
1
  module Kinetic
2
+
3
+ # Configuration can be set either by passing the {Kinetic::DSL#set set} method and or by using the
4
+ # {Kinetic::DSL#config_file config_file} method.
5
+ #
6
+ # @example
7
+ # require 'kinetic'
8
+ #
9
+ # config_file 'config.yml' # loads the values in config_file into configuration
10
+ # set :host, 'localhost' # sets config.host to 'localhost'
11
+ #
12
+ # Accessing configuration values by their method accessors will raise {Kinetic::Errors::MissingConfigurationValue}
13
+ # error. This can be used to ensure that specific values are set at runtime. To access a value without raising an
14
+ # exception use the hash key instead.
15
+ #
16
+ # @example
17
+ # config.host # raises an error if the config host value is not set
18
+ # config[:host] # returns nil if the host value is not set
2
19
  class Configuration < Confstruct::Configuration
3
20
 
21
+ # Default configuration values.
4
22
  DEFAULTS = {
5
- name: File.basename($0),
6
- root: File.dirname($0),
7
- app_file: $0,
8
- workers: 1,
9
- log_file: STDOUT,
10
- host: 'localhost',
11
- port: 5672,
12
- serializer: :JSON,
13
- timeout: 1
23
+ name: File.basename($0), # name of the application. This also translates to the name of the AMQP exchange
24
+ root: File.dirname($0), # the application root directory
25
+ app_file: $0, # the application file
26
+ workers: 1, # the number of worker processes to run
27
+ log_file: 'kinetic.log', # the path of the log file to be used
28
+ host: 'localhost', # the AMQP host
29
+ port: 5672, # the AMQP port
30
+ serializer: :JSON, # the message serializer to use
31
+ timeout: 1, # the time (in seconds) to wait for various operations
32
+ pid: "#{File.basename($0)}.pid" # the process pid file
14
33
  }
15
34
 
16
35
  def initialize
@@ -18,5 +37,14 @@ module Kinetic
18
37
  self.deep_merge!(DEFAULTS)
19
38
  end
20
39
 
40
+
41
+ private
42
+
43
+ def method_missing(meth, *args, &block)
44
+ value = super
45
+ raise Kinetic::Errors::MissingConfigurationValue.new("Configuration is missing value for #{meth}") unless value
46
+ value
47
+ end
48
+
21
49
  end
22
50
  end
data/lib/kinetic/dsl.rb CHANGED
@@ -1,52 +1,16 @@
1
+ require_relative 'dsl/configuration'
1
2
  module Kinetic
2
3
 
3
4
  module DSL
4
5
 
6
+ include Configuration
7
+
5
8
  def after_fork(&block)
6
9
  raise Kinetic::Errors::BlockMissing unless block_given?
7
10
  after_fork_procs << block
8
11
  end
9
12
 
10
- # Sets a configuration value
11
- #
12
- # @param [Symbol] key the configuration key to set
13
- # @param [Symbol] value the configuration value
14
- #
15
- # @return [Boolean] returns true
16
- def set(key, value)
17
- logger.debug "Setting '#{key}' to '#{value}'"
18
- config[key.to_sym] = value
19
- true
20
- end
21
13
 
22
- # Set configuration file.
23
- #
24
- # @param [String] path to configuration file
25
- def config_file(file)
26
- set :config_file, File.expand_path(file)
27
- config.configure(YAML.load(File.new(get!(:config_file), 'r')))
28
- end
29
-
30
- # Gets a configuration value
31
- #
32
- # @param [Symbol] key the key of the value to get
33
- #
34
- # @return [Object, nil] returns the requested configuration value or nil if the value does not exist
35
- def get(key)
36
- config[key.to_sym]
37
- end
38
-
39
- # Gets a configuration value and returns an exception if the value is not present
40
- #
41
- # @param [Symbol] key the key of the value to get
42
- #
43
- # @return [Object]
44
- #
45
- # @raise [Kinetic::Errors::MissingConfigurationValue] if the value does not exist
46
- def get!(key)
47
- raise Kinetic::Errors::MissingConfigurationValue.new(key) unless (value = get(key))
48
- value
49
- end
50
14
 
51
15
  # Defines a direct queue subscription. Direct queues do not allow fuzzy matching so all messages sent to this queue
52
16
  # must exactly match the key.
@@ -68,17 +32,13 @@ module Kinetic
68
32
 
69
33
  # @return [Logger] returns the application logger instance
70
34
  def logger
71
- @logger ||= reopen_logger
72
- end
73
-
74
- def config
75
- @config ||= Kinetic::Configuration.new
35
+ @logger ||= config[:logging] ? reopen_logger : Logger.new('/dev/null')
76
36
  end
77
37
 
78
38
  protected
79
39
 
80
40
  def reopen_logger
81
- @logger = Logger.new(config.log_file)
41
+ @logger = Logger.new(config[:daemonize] ? config.log_file : STDOUT)
82
42
  end
83
43
 
84
44
 
@@ -0,0 +1,36 @@
1
+ require 'yaml'
2
+ module Kinetic
3
+ module DSL
4
+ module Configuration
5
+
6
+ # Sets a configuration value
7
+ #
8
+ # @param [Symbol] key the configuration key to set
9
+ # @param [Symbol] value the configuration value
10
+ #
11
+ # @return [Boolean] returns true
12
+ def set(key, value)
13
+ logger.debug "Setting '#{key}' to '#{value}'"
14
+ config[key.to_sym] = value
15
+ true
16
+ end
17
+
18
+ # Set configuration file.
19
+ #
20
+ # @param [String] path to configuration file
21
+ #
22
+ # @return [Boolean] returns true
23
+ def config_file(file)
24
+ set :config_file, File.expand_path(file)
25
+ config.configure(YAML.load(File.new(config.config_file, 'r')))
26
+ true
27
+ end
28
+
29
+ def config
30
+ @config ||= Kinetic::Configuration.new
31
+ end
32
+
33
+
34
+ end
35
+ end
36
+ end
@@ -7,6 +7,15 @@ module Kinetic
7
7
  class NoSubscriberBlock < BlockMissing; end
8
8
  class KeyMustBeString < KineticError; end
9
9
 
10
+ # Raised when a configuration value is accessed via accessor when it is not set.
11
+ #
12
+ # @example
13
+ # require 'kinetic'
14
+ #
15
+ # set host: 'localhost'
16
+ # set port: 5674
17
+ #
18
+ # puts config.unset_configuration_value # raises MissingConfigurationValue
10
19
  class MissingConfigurationValue < KineticError
11
20
 
12
21
  def initialize(key)
@@ -7,8 +7,7 @@ module Kinetic
7
7
  WORKERS = {}
8
8
  SIG_QUEUE = []
9
9
 
10
- delegate :logger, :get, :get!, :set, :after_fork_procs, to: :app
11
-
10
+ delegate :logger, :config, :set, :after_fork_procs, to: :app
12
11
 
13
12
  attr_reader :app, :reexecpid
14
13
 
@@ -16,16 +15,25 @@ module Kinetic
16
15
  @app = app
17
16
  end
18
17
 
18
+ # Starts and daemonizes the master process
19
+ def start
20
+ if File.exists?(config.pid)
21
+ puts "Unable to start master process, a process with the pidfile #{config.pid} already exists." +
22
+ 'If you wish to run another instance of this application please pass --pid to set a new pid file.'
23
+ exit(1)
24
+ end
25
+ config.daemonize = true
26
+ run
27
+ end
28
+
19
29
  # Starts and initializes the master process.
20
- #
21
- # @return [Kinetic::Master] the master process
22
30
  def run
23
31
  begin
24
- logger.info "Starting Kinetic #{Kinetic::VERSION} with PID #{Process.pid}"
32
+ daemonize if config[:daemonize]
33
+ enable_logging!
25
34
  logger.debug 'Configuration:'
26
35
  logger.ap app.class.send(:config).to_hash
27
36
  proc_name 'master'
28
- write_pidfile!
29
37
  initialize_self_pipe!
30
38
  initialize_signal_traps!
31
39
  spawn_missing_workers
@@ -39,8 +47,29 @@ module Kinetic
39
47
  end
40
48
  end
41
49
 
50
+ def stop
51
+ unless File.exists?(File.join(config.root, config.pid))
52
+ puts "Unable to stop process, a process with the pidfile #{config.pid} could not be found" +
53
+ 'If you have started this app with another pid file location pass --pid to set the pid file.'
54
+ exit(1)
55
+ end
56
+ pid = File.open(File.join(config.root, config.pid), 'r') { |f| f.read }.to_i
57
+ Process.kill(:QUIT, pid)
58
+ end
59
+
42
60
  private
43
61
 
62
+ def enable_logging!
63
+ config.logging = true
64
+ app.class.instance_variable_set(:@logger, nil)
65
+ end
66
+
67
+ def daemonize
68
+ logger.info 'Daemonizing process'
69
+ Process.daemon true, false
70
+ write_pidfile!
71
+ end
72
+
44
73
  def master_sleep(time)
45
74
  IO.select([ SELF_PIPE[0] ], nil, nil, time) or return
46
75
  SELF_PIPE[0].kgio_tryread(11)
@@ -50,9 +79,10 @@ module Kinetic
50
79
  logger.debug 'Joining thread'
51
80
  case SIG_QUEUE.shift
52
81
  when :QUIT
82
+ stop_all(true)
53
83
  break
54
84
  when :TERM, :INT
55
- stop(false)
85
+ stop_all(false)
56
86
  break
57
87
  else
58
88
  master_sleep(0.5)
@@ -76,20 +106,20 @@ module Kinetic
76
106
  end
77
107
 
78
108
  def write_pidfile!
79
- set(:pid, File.join(get!(:root), "#{get!(:name)}.pid"))
80
- logger.info "Writing PID file to #{get!(:pid)}"
81
- File.open(get!(:pid), 'w') { |f| f.write(Process.pid) }
109
+ set(:pid, File.join(config.root, "#{config.name}.pid"))
110
+ logger.info "Writing PID file to #{config.pid}"
111
+ File.open(config.pid, 'w') { |f| f.write(Process.pid) }
82
112
  end
83
113
 
84
114
  def call_on_exit_callbacks
85
115
  ensure
86
- FileUtils.rm(get(:pid)) if get(:pid)
116
+ FileUtils.rm(config[:pid]) if config[:pid] && config[:daemonize]
87
117
  end
88
118
 
89
119
  def spawn_missing_workers
90
120
  logger.debug 'Spawning missing workers'
91
121
  worker_nr = -1
92
- until (worker_nr += 1) == get!(:workers)
122
+ until (worker_nr += 1) == config.workers
93
123
  WORKERS.value?(worker_nr) and next
94
124
  #noinspection RubyArgCount
95
125
  worker = Worker.new(worker_nr, @app)
@@ -136,9 +166,9 @@ module Kinetic
136
166
  end
137
167
 
138
168
  # Terminates all workers, but does not exit master process
139
- def stop(graceful = true)
169
+ def stop_all(graceful = true)
140
170
  logger.warn graceful ? 'Process is shutting down gracefully' : 'Process is shutting down immediately!'
141
- limit = Time.now + get!(:timeout)
171
+ limit = Time.now + config.timeout
142
172
  until WORKERS.empty? || Time.now > limit
143
173
  if graceful
144
174
  soft_kill_each_worker(:QUIT)
@@ -163,6 +193,11 @@ module Kinetic
163
193
  worker = WORKERS.delete(wpid) and worker.close rescue nil
164
194
  end
165
195
 
196
+ def soft_kill_each_worker(signal)
197
+ WORKERS.each_value { |worker| worker.soft_kill(signal) }
198
+ end
199
+
200
+
166
201
  # reaps all unreaped workers
167
202
  def reap_all_workers
168
203
 
@@ -185,7 +220,7 @@ module Kinetic
185
220
  end
186
221
 
187
222
  def proc_name(tag)
188
- $0 = "#{get!(:name)} [#{tag}]"
223
+ $0 = "#{config.name} [#{tag}]"
189
224
  end
190
225
 
191
226
  end
@@ -1,3 +1,3 @@
1
1
  module Kinetic
2
- VERSION = '0.0.3'
2
+ VERSION = '0.0.4'
3
3
  end
@@ -1,7 +1,7 @@
1
1
  module Kinetic
2
2
  class Worker
3
3
 
4
- delegate :logger, :get, :get!, :set, :deserialize, to: :app
4
+ delegate :logger, :config, :set, :deserialize, to: :app
5
5
 
6
6
  attr_reader :id, :app, :channel, :exchanges
7
7
 
@@ -23,8 +23,8 @@ module Kinetic
23
23
  end
24
24
 
25
25
  def run
26
- logger.debug "Establishing connection host: '#{get!(:host)}', port: '#{get!(:port)}'"
27
- AMQP.start(host: get!(:host), port: get!(:port)) do |connection|
26
+ logger.debug "Establishing connection host: '#{config.host}', port: '#{config.port}'"
27
+ AMQP.start(host: config.host, port: config.port) do |connection|
28
28
  logger.debug "AMQP started with conneciton #{connection}"
29
29
  initialize_channel!(connection)
30
30
  initialize_exchanges!
@@ -32,6 +32,22 @@ module Kinetic
32
32
  end
33
33
  end
34
34
 
35
+ # master sends fake signals to children
36
+ def soft_kill(sig) # :nodoc:
37
+ case sig
38
+ when Integer
39
+ signum = sig
40
+ else
41
+ signum = Signal.list[sig.to_s] or
42
+ raise ArgumentError, "BUG: bad signal: #{sig.inspect}"
43
+ end
44
+ # writing and reading 4 bytes on a pipe is atomic on all POSIX platforms
45
+ # Do not care in the odd case the buffer is full, here.
46
+ @master.kgio_trywrite([signum].pack('l'))
47
+ rescue Errno::EPIPE
48
+ # worker will be reaped soon
49
+ end
50
+
35
51
 
36
52
  private
37
53
 
@@ -43,7 +59,7 @@ module Kinetic
43
59
  def initialize_exchanges!
44
60
  logger.debug 'Initializing exchanges'
45
61
  @exchanges = {}
46
- prefix = get!(:name)
62
+ prefix = config.name
47
63
  app.exchanges.each_key do |name|
48
64
  logger.debug " Initializing #{name} exchange"
49
65
  @exchanges[name] = channel.send(name, "#{prefix}.#{name}")
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kinetic
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.3
4
+ version: 0.0.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - Jarod Reid
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-01-16 00:00:00.000000000 Z
11
+ date: 2014-01-24 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: confstruct
@@ -172,6 +172,7 @@ extensions: []
172
172
  extra_rdoc_files: []
173
173
  files:
174
174
  - .gitignore
175
+ - .ruby-version
175
176
  - .travis.yml
176
177
  - Gemfile
177
178
  - LICENSE.txt
@@ -184,6 +185,7 @@ files:
184
185
  - lib/kinetic/cli.rb
185
186
  - lib/kinetic/configuration.rb
186
187
  - lib/kinetic/dsl.rb
188
+ - lib/kinetic/dsl/configuration.rb
187
189
  - lib/kinetic/errors.rb
188
190
  - lib/kinetic/master.rb
189
191
  - lib/kinetic/publisher.rb
@@ -221,4 +223,3 @@ test_files:
221
223
  - spec/base_spec.rb
222
224
  - spec/dsl_spec.rb
223
225
  - spec/spec_helper.rb
224
- has_rdoc: