racecar 0.3.1 → 0.3.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 712a502336151d29108a93f1f00d2eb3b8b0b072
-  data.tar.gz: 9cd9f1fe7b345f1dc9aaf8ceba6751f7e848df98
+  metadata.gz: 88312452dcd53ac7b8097149e4b2f07231fdcd57
+  data.tar.gz: 9206d8f5584cd93d366f9d41f15314a9a8e89abf
 SHA512:
-  metadata.gz: c468794299f01ab66e9727bca985cc334cbcf0c200ec89260f6e3eca55a6325823ab3741c92d0d43f119abea99f3111a534b081da6dd7ae0aaea483b7c56723f
-  data.tar.gz: aaff7d20a5da77ea2b447fa4ed554c3a1230eafca78db0fecfee03c9cd11e37d7a0f0cb6bbf566f3343bfe31c07c643c9b248f0943b3eb2829d34ba36a2bb3ec
+  metadata.gz: 3bc5f769117603f47314259f3d6b639a370fac11b17b66883f4fe9e30087cc9d14be1c63e48603e9c69ad5a51059dcb846a666499b447915c81f454d5e0b197d
+  data.tar.gz: 5fcdd199a4f3c047341aa96ccf458ff7ba9ff211e61513453cd5ed1a57dc7b183e9bfa5c933deeb4956b823b1045f9732fd1e7b4b74145a62738c48134602a75

data/CHANGELOG.md

@@ -0,0 +1,8 @@
+# Changelog
+
+## Unreleased
+
+* Use KingKonf for defining configuration variables.
+* Allow setting configuration variables through the CLI.
+* Make all configuration variables available over the ENV.
+* Allow configuring Datadog monitoring.

data/README.md

@@ -18,7 +18,8 @@ Using [ruby-kafka](https://github.com/zendesk/ruby-kafka) directly can be a chal
     8. [Operations](#operations)
 3. [Development](#development)
 4. [Contributing](#contributing)
-5. [Copyright and license](#copyright-and-license)
+5. [Support and Discussion](#support-and-discussion)
+6. [Copyright and license](#copyright-and-license)
 
 
 ## Installation
@@ -173,6 +174,7 @@ Racecar is first and foremost an executable _consumer runner_. The `racecar` exe
 
 The first time you execute `racecar` with a consumer class a _consumer group_ will be created with a group id derived from the class name (this can be configured). If you start `racecar` with the same consumer class argument multiple times, the processes will join the existing group – even if you start them on other nodes. You will typically want to have at least two consumers in each of your groups – preferably on separate nodes – in order to deal with failures.
 
+
 ### Configuration
 
 Racecar provides a flexible way to configure your consumer in a way that feels at home in a Rails application. If you haven't already, run `bundle exec rails generate racecar:install` in order to generate a config file. You'll get a separate section for each Rails environment, with the common configuration values in a shared `common` section.
@@ -219,7 +221,7 @@ Racecar has support for using SASL to authenticate clients using either the GSSA
 
 If using GSSAPI:
 
-* `sasl_gssapi_principal` – The GSSAPI principal
+* `sasl_gssapi_principal` – The GSSAPI principal.
 * `sasl_gssapi_keytab` – Optional GSSAPI keytab.
 
 If using PLAIN:
@@ -280,6 +282,23 @@ If you've ever used Heroku you'll recognize the format – indeed, deploying to
 With Foreman, you can easily run these processes locally by executing `foreman run`; in production you'll want to _export_ to another process management format such as Upstart or Runit. [capistrano-foreman](https://github.com/hyperoslo/capistrano-foreman) allows you to do this with Capistrano.
 
 
+#### Running consumers in the background
+
+While it is recommended that you use a process supervisor to manage the Racecar consumer processes, it is possible to _daemonize_ the Racecar processes themselves if that is more to your liking. Note that this support is currently in alpha, as it hasn't been tested extensively in production settings.
+
+In order to daemonize Racecar, simply pass in `--daemonize` when executing the command:
+
+    $ bundle exec racecar --daemonize ResizeImagesConsumer
+
+This will start the consumer process in the background. A file containing the process id (the "pidfile") will be created, with the file name being constructed from the consumer class name. If you want to specify the name of the pidfile yourself, pass in `--pidfile=some-file.pid`.
+
+Since the process is daemonized, you need to know the process id (PID) in order to be able to stop it. Use the `racecarctl` command to do this:
+
+    $ bundle exec racecarctl stop --pidfile=some-file.pid
+
+Again, the recommended approach is to manage the processes using process managers. Only do this if you have to.
+
+
 ### Handling errors
 
 When processing messages from a Kafka topic, your code may encounter an error and raise an exception. The cause is typically on of two things:
@@ -334,13 +353,20 @@ In order to introspect the configuration of a consumer process, send it the `SIG
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rspec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/zendesk/racecar.
+Bug reports and pull requests are welcome on [GitHub](https://github.com/zendesk/racecar). Feel free to [join our Slack team](https://ruby-kafka-slack.herokuapp.com/) and ask how best to contribute!
+
+
+## Support and Discussion
+
+If you've discovered a bug, please file a [Github issue](https://github.com/zendesk/racecar/issues/new), and make sure to include all the relevant information, including the version of Racecar, ruby-kafka, and Kafka that you're using.
+
+If you have other questions, or would like to discuss best practises, how to contribute to the project, or any other ruby-kafka related topic, [join our Slack team](https://ruby-kafka-slack.herokuapp.com/)!
+
 
 ## Copyright and license
 

data/exe/racecar

@@ -5,4 +5,10 @@ require "racecar/cli"
 
 $LOAD_PATH.unshift(Dir.pwd)
 
-Racecar::Cli.main(ARGV)
+begin
+  Racecar::Cli.main(ARGV)
+rescue
+  exit(1)
+else
+  exit(0)
+end

data/exe/racecarctl

@@ -7,6 +7,5 @@ begin
   Racecar::Ctl.main(ARGV)
 rescue Racecar::Error => e
   $stderr.puts "Error: #{e.message}"
-  $stderr.puts "Run `racecarctl produce -h` to see valid options"
   exit 1
 end

data/lib/racecar/cli.rb

@@ -1,32 +1,26 @@
 require "optparse"
 require "logger"
+require "fileutils"
 require "racecar/rails_config_file_loader"
+require "racecar/daemon"
 
 module Racecar
-  module Cli
+  class Cli
     def self.main(args)
-      parser = OptionParser.new do |opts|
-        opts.banner = "Usage: racecar MyConsumer [options]"
-
-        opts.on("-r", "--require LIBRARY", "Require the LIBRARY before starting the consumer") do |lib|
-          require lib
-        end
-
-        opts.on("-l", "--log LOGFILE", "Log to the specified file") do |logfile|
-          Racecar.config.logfile = logfile
-        end
-
-        opts.on_tail("--version", "Show Racecar version") do
-          require "racecar/version"
-          $stderr.puts "Racecar #{Racecar::VERSION}"
-          exit
-        end
-      end
+      new(args).run
+    end
 
-      parser.parse!(args)
+    def initialize(args)
+      @parser = build_parser
+      @parser.parse!(args)
+      @consumer_name = args.first or raise Racecar::Error, "no consumer specified"
+    end
 
-      consumer_name = args.first or raise Racecar::Error, "no consumer specified"
+    def config
+      Racecar.config
+    end
 
+    def run
       $stderr.puts "=> Starting Racecar consumer #{consumer_name}..."
 
       RailsConfigFileLoader.load!
@@ -35,23 +29,120 @@ module Racecar
       consumer_class = Kernel.const_get(consumer_name)
 
       # Load config defined by the consumer class itself.
-      Racecar.config.load_consumer_class(consumer_class)
+      config.load_consumer_class(consumer_class)
 
-      Racecar.config.validate!
+      config.validate!
+
+      if config.logfile
+        $stderr.puts "=> Logging to #{config.logfile}"
+        Racecar.logger = Logger.new(config.logfile)
+      end
 
-      if Racecar.config.logfile
-        $stderr.puts "=> Logging to #{Racecar.config.logfile}"
-        Racecar.logger = Logger.new(Racecar.config.logfile)
+      if config.log_level
+        Racecar.logger.level = config.log_level
+      end
+
+      if config.datadog_enabled
+        configure_datadog
       end
 
       $stderr.puts "=> Wrooooom!"
-      $stderr.puts "=> Ctrl-C to shutdown consumer"
+
+      if config.daemonize
+        daemonize!
+      else
+        $stderr.puts "=> Ctrl-C to shutdown consumer"
+      end
 
       processor = consumer_class.new
 
-      Racecar.run(processor)
+      begin
+        Racecar.run(processor)
+      rescue => e
+        $stderr.puts "=> Crashed: #{e}"
+
+        raise
+      end
+    end
+
+    private
+
+    attr_reader :consumer_name
+
+    def daemonize!
+      daemon = Daemon.new(File.expand_path(config.pidfile))
+
+      daemon.check_pid
+
+      $stderr.puts "=> Starting background process"
+      $stderr.puts "=> Writing PID to #{daemon.pidfile}"
+
+      daemon.suppress_input
+
+      if config.logfile.nil?
+        daemon.suppress_output
+      else
+        daemon.redirect_output(config.logfile)
+      end
+
+      daemon.daemonize!
+      daemon.write_pid
+    end
+
+    def build_parser
+      OptionParser.new do |opts|
+        opts.banner = "Usage: racecar MyConsumer [options]"
+
+        opts.on("-r", "--require STRING", "Require a library before starting the consumer") do |lib|
+          require lib
+        end
+
+        opts.on("-l", "--log STRING", "Log to the specified file") do |logfile|
+          config.logfile = logfile
+        end
+
+        Racecar::Config.variables.each do |variable|
+          opt_name = "--" << variable.name.to_s.gsub("_", "-")
+          opt_name << " #{variable.type.upcase}" unless variable.boolean?
+
+          desc = variable.description || "N/A"
+
+          if variable.default
+            desc << " (default: #{variable.default.inspect})"
+          end
+
+          opts.on(opt_name, desc) do |value|
+            if variable.boolean?
+              # Boolean switches are automatically mapped to true/false.
+              config.set(variable.name, value)
+            else
+              # Other CLI params need to be decoded into values of the correct type.
+              config.decode(variable.name, value)
+            end
+          end
+        end
+
+        opts.on_tail("--version", "Show Racecar version") do
+          require "racecar/version"
+          $stderr.puts "Racecar #{Racecar::VERSION}"
+          exit
+        end
+
+        opts.on_tail("-h", "--help", "Show this message") do
+          puts opts
+          exit
+        end
+      end
+    end
+
+    def configure_datadog
+      require "kafka/datadog"
 
-      $stderr.puts "=> Shut down"
+      datadog = Kafka::Datadog
+      datadog.host = config.datadog_host if config.datadog_host.present?
+      datadog.port = config.datadog_port if config.datadog_port.present?
+      datadog.namespace = config.datadog_namespace if config.datadog_namespace.present?
+      datadog.tags = config.datadog_tags if config.datadog_tags.present?
     end
   end
 end

data/lib/racecar/config.rb

@@ -1,100 +1,120 @@
-require "erb"
-require "yaml"
-require "racecar/env_loader"
+require "king_konf"
 
 module Racecar
-  class Config
-    ALLOWED_KEYS = %w(
-      brokers
-      client_id
+  class Config < KingKonf::Config
+    prefix :racecar
 
-      offset_commit_interval
-      offset_commit_threshold
+    desc "A list of Kafka brokers in the cluster that you're consuming from"
+    list :brokers, default: ["localhost:9092"]
 
-      session_timeout
-      heartbeat_interval
-      pause_timeout
-      connect_timeout
-      socket_timeout
-      group_id_prefix
-      group_id
-      subscriptions
-      max_wait_time
+    desc "A string used to identify the client in logs and metrics"
+    string :client_id, default: "racecar"
 
-      error_handler
-      logfile
+    desc "How frequently to commit offset positions"
+    integer :offset_commit_interval, default: 10
 
-      ssl_ca_cert
-      ssl_ca_cert_file_path
-      ssl_client_cert
-      ssl_client_cert_key
+    desc "How many messages to process before forcing a checkpoint"
+    integer :offset_commit_threshold, default: 0
 
-      sasl_gssapi_principal
-      sasl_gssapi_keytab
-      sasl_plain_authzid
-      sasl_plain_username
-      sasl_plain_password
-    )
+    desc "How often to send a heartbeat message to Kafka"
+    integer :heartbeat_interval, default: 10
 
-    REQUIRED_KEYS = %w(
-      brokers
-      client_id
-    )
+    desc "How long to pause a partition for if the consumer raises an exception while processing a message"
+    integer :pause_timeout, default: 10
 
-    DEFAULT_CONFIG = {
-      brokers: ["localhost:9092"],
-      client_id: "racecar",
+    desc "The idle timeout after which a consumer is kicked out of the group"
+    integer :session_timeout, default: 30
 
-      subscriptions: [],
+    desc "How long to wait when trying to connect to a Kafka broker"
+    integer :connect_timeout, default: 10
 
-      # Default is to commit offsets every 10 seconds.
-      offset_commit_interval: 10,
+    desc "How long to wait when trying to communicate with a Kafka broker"
+    integer :socket_timeout, default: 30
 
-      # Default is no buffer threshold trigger.
-      offset_commit_threshold: 0,
+    desc "How long to allow the Kafka brokers to wait before returning messages"
+    integer :max_wait_time, default: 5
 
-      # Default is to send a heartbeat every 10 seconds.
-      heartbeat_interval: 10,
+    desc "A prefix used when generating consumer group names"
+    string :group_id_prefix
 
-      # Default is to pause partitions for 10 seconds on processing errors.
-      pause_timeout: 10,
+    desc "The group id to use for a given group of consumers"
+    string :group_id
 
-      # Default is to kick consumers out of a group after 30 seconds without activity.
-      session_timeout: 30,
+    desc "A filename that log messages should be written to"
+    string :logfile
 
-      # Default is to allow at most 10 seconds when connecting to a broker.
-      connect_timeout: 10,
+    desc "The log level for the Racecar logs"
+    string :log_level
 
-      # Default is to allow at most 30 seconds when reading or writing to
-      # a broker socket.
-      socket_timeout: 30,
+    desc "A valid SSL certificate authority"
+    string :ssl_ca_cert
 
-      # Default is to allow the brokers up to 5 seconds before returning
-      # messages.
-      max_wait_time: 5,
+    desc "The path to a valid SSL certificate authority file"
+    string :ssl_ca_cert_file_path
 
-      # Default is to do nothing on exceptions.
-      error_handler: proc {},
-    }
+    desc "A valid SSL client certificate"
+    string :ssl_client_cert
 
-    attr_accessor(*ALLOWED_KEYS)
+    desc "A valid SSL client certificate key"
+    string :ssl_client_cert_key
 
-    def initialize
-      load(DEFAULT_CONFIG)
-      load_env!
+    desc "The GSSAPI principal"
+    string :sasl_gssapi_principal
+
+    desc "Optional GSSAPI keytab"
+    string :sasl_gssapi_keytab
+
+    desc "The authorization identity to use"
+    string :sasl_plain_authzid
+
+    desc "The username used to authenticate"
+    string :sasl_plain_username
+
+    desc "The password used to authenticate"
+    string :sasl_plain_password
+
+    desc "The file in which to store the Racecar process' PID when daemonized"
+    string :pidfile
+
+    desc "Run the Racecar process in the background as a daemon"
+    boolean :daemonize, default: false
+
+    desc "Enable Datadog metrics"
+    boolean :datadog_enabled, default: false
+
+    desc "The host running the Datadog agent"
+    string :datadog_host
+
+    desc "The port of the Datadog agent"
+    integer :datadog_port
+
+    desc "The namespace to use for Datadog metrics"
+    string :datadog_namespace
+
+    desc "Tags that should always be set on Datadog metrics"
+    list :datadog_tags
+
+    # The error handler must be set directly on the object.
+    attr_reader :error_handler
+
+    attr_accessor :subscriptions
+
+    def initialize(env: ENV)
+      super(env: env)
+      @error_handler = proc {}
+      @subscriptions = []
     end
 
     def inspect
-      ALLOWED_KEYS
+      self.class.variables
+        .map(&:name)
         .map {|key| [key, get(key).inspect].join(" = ") }
         .join("\n")
     end
 
     def validate!
-      REQUIRED_KEYS.each do |key|
-        if send(key).nil?
-          raise ConfigError, "required configuration key `#{key}` not defined"
-        end
+      if brokers.empty?
+        raise ConfigError, "`brokers` must not be empty"
       end
 
       if socket_timeout <= max_wait_time
@@ -106,41 +126,10 @@ module Racecar
       end
     end
 
-    def load_file(path, environment)
-      # First, load the ERB template from disk.
-      template = ERB.new(File.new(path).read)
-
-      # The last argument to `safe_load` allows us to use aliasing to share
-      # configuration between environments.
-      processed = YAML.safe_load(template.result(binding), [], [], true)
-
-      data = processed.fetch(environment)
-
-      load(data)
-    end
-
-    def get(key)
-      public_send(key)
-    end
-
-    def set(key, value)
-      unless ALLOWED_KEYS.include?(key.to_s)
-        raise ConfigError, "unknown configuration key `#{key}`"
-      end
-
-      instance_variable_set("@#{key}", value)
-    end
-
-    def load(data)
-      data.each do |key, value|
-        set(key, value)
-      end
-    end
-
     def load_consumer_class(consumer_class)
-      @group_id = consumer_class.group_id || @group_id
+      self.group_id = consumer_class.group_id || self.group_id
 
-      @group_id ||= [
+      self.group_id ||= [
         # Configurable and optional prefix:
         group_id_prefix,
 
@@ -148,32 +137,13 @@ module Racecar
         consumer_class.name.gsub(/[a-z][A-Z]/) {|str| str[0] << "-" << str[1] }.downcase,
       ].compact.join("")
 
-      @subscriptions = consumer_class.subscriptions
-      @max_wait_time = consumer_class.max_wait_time || @max_wait_time
+      self.subscriptions = consumer_class.subscriptions
+      self.max_wait_time = consumer_class.max_wait_time || self.max_wait_time
+      self.pidfile ||= "#{group_id}.pid"
     end
 
     def on_error(&handler)
       @error_handler = handler
     end
-
-    private
-
-    def load_env!
-      loader = EnvLoader.new(ENV, self)
-
-      loader.string_list(:brokers)
-      loader.string(:client_id)
-      loader.string(:group_id_prefix)
-      loader.string(:group_id)
-      loader.integer(:offset_commit_interval)
-      loader.integer(:offset_commit_threshold)
-      loader.integer(:heartbeat_interval)
-      loader.integer(:pause_timeout)
-      loader.integer(:connect_timeout)
-      loader.integer(:socket_timeout)
-      loader.integer(:max_wait_time)
-
-      loader.validate!
-    end
   end
 end

data/lib/racecar/ctl.rb

@@ -1,19 +1,62 @@
 require "optparse"
 require "racecar/rails_config_file_loader"
+require "racecar/daemon"
 
 module Racecar
   class Ctl
     ProduceMessage = Struct.new(:value, :key, :topic)
 
     def self.main(args)
-      command = args.shift or raise Racecar::Error, "no command specified"
+      command = args.shift
 
-      ctl = new
+      if command.nil?
+        puts "No command specified. Commands:"
+        puts " - status"
+        puts " - stop"
+        puts " - produce"
+      else
+        ctl = new(command)
+
+        if ctl.respond_to?(command)
+          ctl.send(command, args)
+        else
+          raise Racecar::Error, "invalid command: #{command}"
+        end
+      end
+    end
+
+    def initialize(command)
+      @command = command
+    end
 
-      if ctl.respond_to?(command)
-        ctl.send(command, args)
+    def status(args)
+      parse_options!(args)
+
+      pidfile = Racecar.config.pidfile
+      daemon = Daemon.new(pidfile)
+
+      if daemon.running?
+        puts "running (PID = #{daemon.pid})"
       else
-        raise Racecar::Error, "invalid command: #{command}"
+        puts daemon.pid_status
+      end
+    end
+
+    def stop(args)
+      parse_options!(args)
+
+      pidfile = Racecar.config.pidfile
+      daemon = Daemon.new(pidfile)
+
+      if daemon.running?
+        daemon.stop!
+        while daemon.running?
+          puts "Waiting for Racecar process to stop..."
+          sleep 5
+        end
+        puts "Racecar stopped"
+      else
+        puts "Racecar is not currently running"
       end
     end
 
@@ -62,5 +105,19 @@ module Racecar
 
       $stderr.puts "=> Delivered message to Kafka cluster"
     end
+
+    private
+
+    def parse_options!(args)
+      parser = OptionParser.new do |opts|
+        opts.banner = "Usage: racecarctl #{@command} [options]"
+
+        opts.on("--pidfile PATH", "Use the PID stored in the specified file") do |path|
+          Racecar.config.pidfile = File.expand_path(path)
+        end
+      end
+
+      parser.parse!(args)
+    end
   end
 end

data/lib/racecar/daemon.rb

@@ -0,0 +1,97 @@
+module Racecar
+  class Daemon
+    attr_reader :pidfile
+
+    def initialize(pidfile)
+      raise Racecar::Error, "No pidfile specified" if pidfile.nil?
+
+      @pidfile = pidfile
+    end
+
+    def pidfile?
+      !pidfile.nil?
+    end
+
+    def daemonize!
+      exit if fork
+      Process.setsid
+      exit if fork
+      Dir.chdir "/"
+    end
+
+    def redirect_output(logfile)
+      FileUtils.mkdir_p(File.dirname(logfile), mode: 0755)
+      FileUtils.touch(logfile)
+
+      File.chmod(0644, logfile)
+
+      $stderr.reopen(logfile, 'a')
+      $stdout.reopen($stderr)
+      $stdout.sync = $stderr.sync = true
+    end
+
+    def suppress_input
+      $stdin.reopen('/dev/null')
+    end
+
+    def suppress_output
+      $stderr.reopen('/dev/null', 'a')
+      $stdout.reopen($stderr)
+    end
+
+    def check_pid
+      if pidfile?
+        case pid_status
+        when :running, :not_owned
+          $stderr.puts "=> Racecar is already running with that PID file (#{pidfile})"
+          exit(1)
+        when :dead
+          File.delete(pidfile)
+        end
+      end
+    end
+
+    def pid
+      if File.exists?(pidfile)
+        File.read(pidfile).to_i
+      else
+        nil
+      end
+    end
+
+    def running?
+      pid_status == :running || pid_status == :not_owned
+    end
+
+    def stop!
+      Process.kill("TERM", pid)
+    end
+
+    def pid_status
+      return :exited if pid.nil?
+      return :dead if pid == 0
+
+      # This will raise Errno::ESRCH if the process doesn't exist.
+      Process.kill(0, pid)
+
+      :running
+    rescue Errno::ESRCH
+      :dead
+    rescue Errno::EPERM
+      :not_owned
+    end
+
+    def write_pid
+      File.open(pidfile, ::File::CREAT | ::File::EXCL | ::File::WRONLY) do |f|
+        f.write(Process.pid.to_s)
+      end
+
+      at_exit do
+        File.delete(pidfile) if File.exists?(pidfile)
+      end
+    rescue Errno::EEXIST
+      check_pid
+      retry
+    end
+  end
+end

data/lib/racecar/rails_config_file_loader.rb

@@ -5,12 +5,18 @@ module Racecar
 
       begin
         require "rails"
+      rescue LoadError
+        # Not a Rails application.
+      end
 
+      if defined?(Rails)
         $stderr.puts "=> Detected Rails, booting application..."
 
         require "./config/environment"
 
-        Racecar.config.load_file(config_file, Rails.env)
+        if (Rails.root + config_file).readable?
+          Racecar.config.load_file(config_file, Rails.env)
+        end
 
         # In development, write Rails logs to STDOUT. This mirrors what e.g.
         # Unicorn does.
@@ -20,8 +26,6 @@ module Racecar
           console.level = Rails.logger.level
           Rails.logger.extend(ActiveSupport::Logger.broadcast(console))
         end
-      rescue LoadError
-        # Not a Rails application.
       end
     end
   end

data/lib/racecar/runner.rb

@@ -80,14 +80,18 @@ module Racecar
 
         # Restart the consumer loop.
         retry
+      rescue Kafka::InvalidSessionTimeout
+        raise ConfigError, "`session_timeout` is set either too high or too low"
       rescue Kafka::Error => e
         error = "#{e.class}: #{e.message}\n" + e.backtrace.join("\n")
         @logger.error "Consumer thread crashed: #{error}"
 
         config.error_handler.call(e)
-      end
 
-      @logger.info "Gracefully shutting down"
+        raise
+      else
+        @logger.info "Gracefully shutting down"
+      end
     end
   end
 end

data/lib/racecar/version.rb

@@ -1,3 +1,3 @@
 module Racecar
-  VERSION = "0.3.1"
+  VERSION = "0.3.2"
 end

data/racecar.gemspec

@@ -20,6 +20,7 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
+  spec.add_runtime_dependency "king_konf", "~> 0.1.6"
   spec.add_runtime_dependency "ruby-kafka", "~> 0.4"
 
   spec.add_development_dependency "bundler", "~> 1.13"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: racecar
 version: !ruby/object:Gem::Version
-  version: 0.3.1
+  version: 0.3.2
 platform: ruby
 authors:
 - Daniel Schierbeck
@@ -9,8 +9,22 @@ authors:
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-08-22 00:00:00.000000000 Z
+date: 2017-10-16 00:00:00.000000000 Z
 dependencies:
+- !ruby/object:Gem::Dependency
+  name: king_konf
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.1.6
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.1.6
 - !ruby/object:Gem::Dependency
   name: ruby-kafka
   requirement: !ruby/object:Gem::Requirement
@@ -79,6 +93,7 @@ extra_rdoc_files: []
 files:
 - ".gitignore"
 - ".rspec"
+- CHANGELOG.md
 - Gemfile
 - LICENSE.txt
 - Procfile
@@ -99,7 +114,7 @@ files:
 - lib/racecar/config.rb
 - lib/racecar/consumer.rb
 - lib/racecar/ctl.rb
-- lib/racecar/env_loader.rb
+- lib/racecar/daemon.rb
 - lib/racecar/rails_config_file_loader.rb
 - lib/racecar/runner.rb
 - lib/racecar/version.rb
@@ -124,7 +139,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.4.5.1
+rubygems_version: 2.6.11
 signing_key: 
 specification_version: 4
 summary: A framework for running Kafka consumers

data/lib/racecar/env_loader.rb

@@ -1,49 +0,0 @@
-module Racecar
-  class EnvLoader
-    def initialize(env, config)
-      @env = env
-      @config = config
-      @loaded_keys = []
-    end
-
-    def string(name)
-      set(name) {|value| value }
-    end
-
-    def integer(name)
-      set(name) do |value|
-        begin
-          Integer(value)
-        rescue ArgumentError
-          raise ConfigError, "#{value.inspect} is not an integer"
-        end
-      end
-    end
-
-    def string_list(name)
-      set(name) {|value| value.split(",") }
-    end
-
-    def validate!
-      # Make sure the user hasn't made a typo and added a key we don't know
-      # about.
-      @env.keys.grep(/^RACECAR_/).each do |key|
-        unless @loaded_keys.include?(key)
-          raise ConfigError, "unknown config variable #{key}"
-        end
-      end
-    end
-
-    private
-
-    def set(name)
-      key = "RACECAR_#{name.upcase}"
-
-      if @env.key?(key)
-        value = yield @env.fetch(key)
-        @config.set(name, value)
-        @loaded_keys << key
-      end
-    end
-  end
-end