krump 0.3.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: f8ef5b85176e1456ef7da988aebb76654794be6c
+  data.tar.gz: 1249ca52f23eb3fa715838c563e053423ff88654
+SHA512:
+  metadata.gz: f51b81533a0946b478a2f4977444f6737d54c26fa05fc1a1f99d1788e6a5d76e49254b1629ff5f9148bf602bcd0499adde582bf56b555d1965fb5210621dd405
+  data.tar.gz: af0993ef4d7219585e66a462a1caa4eecaee941d241c26866967b85a16b61d070623ba8c2772e92f118755de9203be8e18ccb49cb807ec0d2ecdb6bfee633dfd

data/.gitignore

@@ -0,0 +1,10 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+*.gem

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.ruby-version

@@ -0,0 +1 @@
+2.2.2

data/.travis.yml

@@ -0,0 +1,3 @@
+language: ruby
+rvm:
+  - 2.2.2

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in krump.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,12 @@
+Copyright (c) 2016, Funding Circle Ltd. 
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,117 @@
+# Krump
+
+A Kafka consumer focused on convenience.
+
+This application was written because of a need for a `tail`-like way to consume Kafka messages. `kafka-console-consumer.sh`, which is distributed with Kafka, allows you to either read _all_ messages in a topic, or any _new_ message. However, most often I want to to see the most recent few messages from a topic.
+
+`krump` provides that as well as a number of other conveniences.
+
+## Feature Overview
+
+1. See the most recent `n` messages in a topic
+2. See a specific range of messages
+3. See messages from specific partitions
+4. Get a count of messages for a particular topic/partition
+5. Automatically set up SSH tunnels so it can be run locally
+
+## Installation
+
+    $ gem install krump
+
+## Usage
+
+Here are some examples on how to use the application. In these examples no broker information is given so it just connects to `localhost:9092`.
+
+**See the most recent 2 messages from the topic `sometopic`:**
+
+```bash
+$ krump --topic sometopic --offset -2
+===== Topic: sometopic = Partition: 0 =======
+{"id":"111"}
+{"id":"222"}
+===== Topic: sometopic = Partition: 1 =======
+{"id":"333"}
+{"id":"444"}
+===== Topic: sometopic = Partition: 2 =======
+{"id":"555"}
+{"id":"666"}
+===== Topic: sometopic = Partition: 3 =======
+{"id":"777"}
+{"id":"888"}
+```
+
+**See 3 messages starting at offset 100 on partitions 1 & 2:**
+
+```bash
+$ krump --topic sometopic --partitions 1 2 --offset 100 --read-count 3
+===== Topic: sometopic = Partition: 0 =======
+{"id":"123"}
+{"id":"456"}
+{"id":"789"}
+===== Topic: sometopic = Partition: 1 =======
+{"id":"abc"}
+{"id":"def"}
+{"id":"fff"}
+```
+
+**Show how many messages are in each partition:**
+
+```bash
+$ krump --topic sometopic --count-messages
+sometopic | Partition 0 | 29043 messages
+sometopic | Partition 1 | 29776 messages
+sometopic | Partition 2 | 29118 messages
+sometopic | Partition 3 | 27406 messages
+```
+
+Use `krump --help` to see all options.
+
+### Config File
+
+You can set configurations for different environments in the config file (`~/.krump` by default).
+
+This is especially useful if your cluster is behind a gateway server (e.g. an AWS VPC).
+
+Example config file:
+
+    # Directly access a Kafka cluster on the Internet
+    dev.kafka_broker=51.120.33.24:9092
+
+    # Or connect to a Kafka cluster behind a gateway server
+    staging.gateway_hostname=51.150.99.142
+    staging.gateway_user=ec2-user
+    staging.gateway_identityfile=~/.ssh/mykey.pem
+    staging.kafka_broker=10.0.100.1:9092
+    staging.kafka_broker=10.0.100.2:9092
+    staging.kafka_broker=10.0.100.3:9092
+
+    # You can also use a host alias and it will get the connection info from
+    # /etc/ssh/config or ~/.ssh/config
+    uat.gateway_host=uat-bastion
+    uat.kafka_broker=10.0.105.1:9092
+    uat.kafka_broker=10.0.105.2:9092
+    uat.kafka_broker=10.0.105.3:9092
+
+Use the `--environment` flag to use the connection settings for that environment, for example:
+
+```bash
+$ krump --environment staging --topic sometopic --latest-offset
+```
+
+If an environment's settings include gateway particulars, `krump` will handle setting up temporary SSH tunnels.
+
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release` to create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+
+## Contributing
+
+1. Fork it ( https://github.com/[my-github-username]/krump/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/Rakefile

@@ -0,0 +1,5 @@
+require 'rspec/core/rake_task'
+
+
+task :default => [:spec]
+spec = RSpec::Core::RakeTask.new(:spec)

data/bin/console

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "krump"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+require "pry"
+Pry.start

data/bin/krump

@@ -0,0 +1,246 @@
+#!/usr/bin/env ruby
+require 'net/ssh'
+require 'trollop'
+require 'krump/config_parser'
+require 'krump/kafka_consumer'
+require 'krump/local_open_port'
+require 'krump/ssh_tunnels'
+
+
+def main(config, retries_left = 10)
+  if config[:gateway_hostname]
+    ssh_tunnels = Krump::SshTunnels.new(
+      config[:gateway_hostname],
+      config[:gateway_user],
+      config[:gateway_identityfile],
+      config[:ssh_tunnel_info]
+    )
+    ssh_tunnels.open { run_task(config) }
+  else
+    run_task(config)
+  end
+
+# Between the time a local port is assigned for an SSH tunnel and that tunnel is actually
+# opened, it's possible for a different application to use that port.
+rescue Errno::EADDRINUSE => e
+  raise e if retries_left == 0
+  set_new_port_for_port_in_use!(config, e)
+  retries_left =- 1
+  retry
+
+rescue Interrupt
+end
+
+
+def set_new_port_for_port_in_use!(config, e)
+  port_in_use = e.message[/port \d{5}/].split.last.to_i
+  new_port = Krump::LocalOpenPort.find
+
+  config[:brokers] = config[:brokers].map do |broker|
+    broker.split(':').last.to_i == port_in_use ? "localhost:#{new_port}" : broker
+  end
+
+  config[:ssh_tunnel_info] = config[:ssh_tunnel_info].map do |info|
+    info.local_port = new_port if info.local_port == port_in_use
+    info
+  end
+end
+
+
+def init_config
+  cmd_line_opts = parse_opts
+  config = Krump::ConfigParser.new(cmd_line_opts[:config_file],
+                                     cmd_line_opts[:environment]).parse
+
+  # Command line options will supercede those in the config file
+  cmd_line_opts.keys.each do |key|
+    config[key] = cmd_line_opts[key] unless cmd_line_opts[key].nil?
+  end
+
+  if config[:gateway_host]
+    host_config = Net::SSH::Config.for(config[:gateway_host])
+    config[:gateway_hostname] = host_config[:host_name]
+    config[:gateway_user] = host_config[:user]
+    config[:gateway_identityfile] = host_config[:keys].first
+  end
+
+  set_defaults!(config)
+  config
+
+rescue Interrupt
+end
+
+
+def parse_opts
+  opts = Trollop::options do
+    opt :environment, 'Which environment to use from the config file', :type => :string
+    opt :brokers, 'List (space separated) of Kafka brokers', :type => :strings
+    opt :topic, 'Kafka topic', :type => :string, :required => true
+    opt :partitions, 'List (space separated) of partions to consider', :type => :integers,
+        :default => []
+    opt :offset, 'Print messages starting from this offset (use a negative offset for the ' +
+        'most recent n messages)', :default => -10
+    opt :earliest_offset, 'Print messages starting from the earliest offset'
+    opt :latest_offset, 'Print messages starting from most recent offset'
+    opt :read_count, 'Max number of messages to read (exits after reading currently ' +
+        'available messages even if this number is not reached)', :type => :integer
+    opt :print_offset, 'Include offset number in output'
+    opt :dump, 'Exit after printing the available messages'
+    opt :count_messages, 'Display the number of messages on a topic'
+    opt :min_max_offset, 'Display the minimum and maximum offsets on a topic'
+    opt :config_file, 'File containing environment settings', :default => '~/.krump'
+    opt :gateway_host, 'Host alias (from SSH config file) for gateway server in front of the ' +
+        'Kafka cluster', :type => :string
+    opt :gateway_hostname, 'Hostname for gateway server in front of the Kafka cluster',
+        :type => :string
+    opt :gateway_user, 'User for gateway server in front of the Kafka cluster',
+        :type => :string
+    opt :gateway_identityfile, 'Path to key pair for gateway server in front of the Kafka ' +
+        'cluster', :type => :string
+    opt :skip_header, "Don't print header showing the partition and offset for a given group " +
+        'of messages'
+    conflicts :offset, :earliest_offset, :latest_offset, :count_messages
+    conflicts :dump, :read_count, :count_messages
+    conflicts :gateway_host, :gateway_hostname
+    conflicts :gateway_host, :gateway_user
+    conflicts :gateway_host, :gateway_identityfile
+  end
+
+  if opts[:min_max_offset] && !opts[:count_messages]
+    Trollop::die :min_max_offset, "cannot be set unless --count-messages is set"
+  end
+
+  opts[:offset] = :earliest_offset if opts[:earliest_offset] || opts[:count_messages]
+  opts[:offset] = :latest_offset if opts[:latest_offset]
+  opts
+end
+
+
+# Normally you'd set defaults in the Trollop::options block. However, we want
+# the command line options to supercede those in the config file, but not if
+# the command line option is just the default (not explicitly set).
+def set_defaults!(config)
+  config[:brokers] ||= ['localhost:9092']
+  config[:partition] ||= 0
+end
+
+
+def run_task(config)
+  consumers = init_consumers(config)
+
+  if config[:count_messages]
+    print_message_count(consumers, config)
+  else
+    print_messages(consumers, config)
+  end
+end
+
+
+def init_consumers(config)
+  if config[:partitions].empty?
+    init_consumers_for_all_partitions(config)
+
+  else
+    config[:partitions].map do |partition|
+      Krump::KafkaConsumer.new(
+        config[:brokers],
+        config[:topic],
+        partition,
+        config[:offset]
+      )
+    end
+  end
+end
+
+
+# Normally you'd need to know the Zookeeper particulars to get a list of
+# available partitions. To keep the configuration simple, this simply loops
+# until it exceeds the max partition.
+def init_consumers_for_all_partitions(config)
+  consumers = []
+  partition = 0
+
+  loop do
+    consumers << Krump::KafkaConsumer.new(
+      config[:brokers],
+      config[:topic],
+      partition,
+      config[:offset]
+    )
+    partition += 1
+  end
+rescue Poseidon::Errors::UnknownTopicOrPartition
+  consumers
+end
+
+
+def print_message_count(consumers, config)
+  config[:offset] = :latest_offset
+
+  earliest_offset_consumers = consumers
+  latest_offset_consumers = init_consumers(config)
+
+  earliest_offset_consumers.zip(latest_offset_consumers).each do |earliest, latest|
+    msg_count = latest.consumer.next_offset - earliest.consumer.next_offset
+    output = "#{config[:topic]} | Partition #{latest.partition} | #{msg_count} messages"
+    if config[:min_max_offset]
+      output += " (#{earliest.consumer.next_offset}, #{latest.consumer.next_offset})"
+    end
+    puts output
+  end
+end
+
+
+def print_messages(consumers, config)
+  loop do
+    consumers.each do |consumer|
+      messages = consumer.fetch
+
+      if consumer.last_fetch_size > 0
+        print_header(config, consumer) unless config[:skip_header]
+
+        messages.each do |msg|
+          output = format_message(config, msg)
+          puts output if config[:read_count].nil? || consumer.messages_read < config[:read_count]
+          consumer.messages_read += 1
+        end
+      end
+    end
+
+    break if finished_consuming?(config, consumers)
+    sleep 1 if nothing_fetched?(consumers)
+  end
+end
+
+
+def print_header(config, consumer)
+  puts "===== Topic: #{config[:topic]} = Partition: #{consumer.partition} ======="
+end
+
+
+def format_message(config, msg)
+  if config[:print_offset]
+    "#{msg.offset} | #{msg.value}"
+  else
+    msg.value
+  end
+end
+
+
+def finished_consuming?(config, consumers)
+  (config[:read_count] && all_consumers_exceeded_read_count?(config[:read_count], consumers)) ||
+  (config[:dump] && nothing_fetched?(consumers))
+end
+
+
+def all_consumers_exceeded_read_count?(read_count, consumers)
+  consumers.all? { |consumer| consumer.messages_read >= read_count }
+end
+
+
+def nothing_fetched?(consumers)
+  consumers.all? { |consumer| consumer.last_fetch_size == 0 }
+end
+
+
+main(init_config)

data/bin/setup

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/krump.gemspec

@@ -0,0 +1,33 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'krump/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = 'krump'
+  spec.version       = Krump::VERSION
+  spec.authors       = ['Dean Morin']
+  spec.email         = ['dean.morin@fundingcircle.com']
+
+  spec.summary       = %q{A Kafka consumer focused on convenience.}
+  spec.description   = %q{This application was written because of a need for a tail-like way to consume Kafka messages. kafka-console-consumer.sh, which is distributed with Kafka, allows you to either read all messages in a topic, or any new message. However, most often I want to to see the most recent few messages from a topic. Krump provides that as well as a number of other conveniences.
+}
+  spec.homepage      = 'https://github.com/fundingcircle/krump'
+  spec.license       = 'BSD 3-Clause'
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = 'bin'
+  spec.executables   = 'krump'
+  spec.require_paths = ['lib']
+
+  spec.add_dependency 'net-ssh-gateway', '~> 1.2'
+  spec.add_dependency 'poseidon', '0.0.5'
+  spec.add_dependency 'trollop', '~> 2.1'
+
+  spec.add_development_dependency 'bundler', '~> 1.9'
+  spec.add_development_dependency 'pry', '>= 0.10.1'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.3'
+
+  spec.required_ruby_version = '>= 2.0.0'
+end

data/lib/krump.rb

@@ -0,0 +1,2 @@
+require 'krump/kafka_consumer'
+require 'krump/ssh_tunnels'

data/lib/krump/config_parser.rb

@@ -0,0 +1,104 @@
+require 'krump/local_open_port'
+require 'krump/ssh_tunnel_info'
+
+
+module Krump
+  class ConfigParser
+
+    def initialize(filename, environment)
+      @filename =
+        if !filename.nil? && filename.start_with?('~/')
+          "#{Dir.home}/#{filename.split('/', 2).last}"
+        else
+          filename
+        end
+      @environment = environment
+    end
+
+    def parse
+      if @filename && @environment
+        File.open(@filename, 'r') { |fh| parse_config(fh.readlines) }
+      else
+        {}
+      end
+    rescue Errno::ENOENT => e
+      # Ignore file-not-found error if it's the default filename
+      raise unless @filename == default_config
+      {}
+    rescue StandardError => e
+      STDERR.puts 'There is an error in your config file'
+      raise
+    end
+
+    private
+
+    def parse_config(lines)
+      lines.keep_if { |line| line.start_with?(@environment) }
+
+      if lines.empty?
+        fail "No configuration for environment '#{@environment}'"
+      end
+
+      config = {}
+      config[:brokers] = []
+      config[:ssh_tunnel_info] = []
+
+      lines.each do |line|
+        key = line.split("#{@environment}.").last.split('=').first
+        value = line.split('=').last.chomp
+
+        case key
+        when 'gateway_host'         then config[:gateway_host] = value
+        when 'gateway_hostname'     then config[:gateway_hostname] = value
+        when 'gateway_user'         then config[:gateway_user] = value
+        when 'gateway_identityfile' then config[:gateway_identityfile] = value
+        when 'kafka_broker'         then add_broker_to_config!(config, value)
+        else
+          fail "#{key} is not a supported config option"
+        end
+      end
+
+      fail_if_invalid_config(config)
+      config
+    end
+
+    def add_broker_to_config!(config, value)
+      host = value.split(':').first
+      port = value.split(':').last
+
+      if broker_behind_gateway?(config)
+        local_port = LocalOpenPort.find
+        config[:ssh_tunnel_info] << SshTunnelInfo.new(host, port, local_port)
+        config[:brokers] << "localhost:#{local_port}"
+      else
+        config[:brokers] << "#{host}:#{port}"
+      end
+    end
+
+    def broker_behind_gateway?(config)
+      config[:gateway_host] || config[:gateway_hostname]
+    end
+
+    def fail_if_invalid_config(config)
+      fail_if_incompatible_settings(config[:gateway_host], config[:gateway_hostname])
+      fail_if_incompatible_settings(config[:gateway_host], config[:gateway_user])
+      fail_if_incompatible_settings(config[:gateway_host], config[:gateway_identityfile])
+
+      gateway_credential_keys = [:gateway_hostname, :gateway_user, :gateway_identityfile]
+
+      if gateway_credential_keys.any? { |key| config[key] }
+        unless gateway_credential_keys.all? { |key| config[key] }
+          fail "If any of (#{gateway_credential_keys.join(',')}) are set then all need to be set"
+        end
+      end
+    end
+
+    def fail_if_incompatible_settings(setting1, setting2)
+      fail "Both #{setting1} and #{setting2} cannot be set" if setting1 && setting2
+    end
+
+    def default_config
+      "#{Dir.home}/.krump"
+    end
+  end
+end

data/lib/krump/kafka_consumer.rb

@@ -0,0 +1,75 @@
+require 'poseidon'
+
+
+module Krump
+  class KafkaConsumer
+    attr_accessor :messages_read
+    attr_reader :broker, :consumer, :last_fetch_size, :offset, :partition, :topic
+
+    def initialize(brokers, topic, partition, offset)
+      @topic = topic
+      @partition = partition
+      @offset = offset
+      @broker = find_broker_for_partition_or_fail(brokers.clone)
+      @consumer = init_consumer
+      @messages_read = 0
+      @last_fetch_size = 0
+    end
+
+    def fetch
+      messages = @consumer.fetch
+      @last_fetch_size = messages.size
+      messages
+    end
+
+    private
+
+    # Poseiden expects that you know which broker to find a particular
+    # partition on. This method simply tries them all. Errors are silently ignored,
+    # unless all brokers are checked and the requested topic/partition is still
+    # not found.
+    #
+    # It uses a negative offset because a positive one won't fail on consumer.next_offset.
+    #
+    def find_broker_for_partition_or_fail(brokers)
+      broker = brokers.shift
+      host = broker.split(':').first
+      port = broker.split(':').last
+
+      consumer = Poseidon::PartitionConsumer.new(
+        "krump-kafka-test-consumer_#{@topic}_#{@partition}_#{DateTime.now}",
+        host,
+        port,
+        @topic,
+        @partition,
+        -1
+      )
+      consumer.next_offset
+      consumer.close
+
+      broker
+
+    rescue Poseidon::Errors::NotLeaderForPartition => e
+      retry if brokers.size > 0
+      raise e
+    rescue Poseidon::Errors::UnknownTopicOrPartition => e
+      retry if brokers.size > 0
+      raise e
+    end
+
+    def init_consumer
+      host = @broker.split(':').first
+      port = @broker.split(':').last
+
+      consumer = Poseidon::PartitionConsumer.new(
+        "krump-kafka-consumer_#{@topic}_#{@partition}_#{DateTime.now}",
+        host,
+        port,
+        @topic,
+        @partition,
+        @offset
+      )
+      consumer
+    end
+  end
+end

data/lib/krump/local_open_port.rb

@@ -0,0 +1,13 @@
+require 'socket'
+
+module Krump
+  class LocalOpenPort
+
+    # Return an open port from the ephemeral port range
+    def self.find
+      socket = Socket.new(:INET, :STREAM, 0)
+      socket.bind(Addrinfo.tcp("127.0.0.1", 0))
+      socket.local_address.ip_port
+    end
+  end
+end

data/lib/krump/ssh_tunnel_info.rb

@@ -0,0 +1,11 @@
+module Krump
+  class SshTunnelInfo
+    attr_accessor :host, :port, :local_port
+
+    def initialize(host, port, local_port)
+      @host = host
+      @port = port.to_i
+      @local_port = local_port.to_i
+    end
+  end
+end

data/lib/krump/ssh_tunnels.rb

@@ -0,0 +1,41 @@
+require 'net/ssh/gateway'
+
+
+module Krump
+  class SshTunnels
+
+    def initialize(gateway_hostname, gateway_user, gateway_identityfile, ssh_tunnel_info)
+      @gateway_hostname = gateway_hostname
+      @gateway_user = gateway_user
+      @gateway_identityfile = gateway_identityfile
+      @ssh_tunnel_info = Array(ssh_tunnel_info)
+    end
+
+    def open(&block)
+      block_given? ? open_with_block(&block) : open_without_block
+    end
+
+    private
+
+    def open_with_block(&block)
+      gateway = open_without_block
+      yield gateway
+    ensure
+      gateway.shutdown! unless gateway.nil?
+    end
+
+    def open_without_block
+      gateway = Net::SSH::Gateway.new(
+        @gateway_hostname,
+        @gateway_user,
+        :keys => [@gateway_identityfile]
+      )
+
+      @ssh_tunnel_info.each do |info|
+        gateway.open(info.host, info.port, info.local_port)
+      end
+
+      gateway
+    end
+  end
+end

data/lib/krump/version.rb

@@ -0,0 +1,3 @@
+module Krump
+  VERSION = '0.3.1'
+end

metadata

@@ -0,0 +1,163 @@
+--- !ruby/object:Gem::Specification
+name: krump
+version: !ruby/object:Gem::Version
+  version: 0.3.1
+platform: ruby
+authors:
+- Dean Morin
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-04-01 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: net-ssh-gateway
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+- !ruby/object:Gem::Dependency
+  name: poseidon
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.0.5
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.0.5
+- !ruby/object:Gem::Dependency
+  name: trollop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.1'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.9'
+- !ruby/object:Gem::Dependency
+  name: pry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.10.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.10.1
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.3'
+description: |
+  This application was written because of a need for a tail-like way to consume Kafka messages. kafka-console-consumer.sh, which is distributed with Kafka, allows you to either read all messages in a topic, or any new message. However, most often I want to to see the most recent few messages from a topic. Krump provides that as well as a number of other conveniences.
+email:
+- dean.morin@fundingcircle.com
+executables:
+- krump
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".ruby-version"
+- ".travis.yml"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/krump
+- bin/setup
+- krump.gemspec
+- lib/krump.rb
+- lib/krump/config_parser.rb
+- lib/krump/kafka_consumer.rb
+- lib/krump/local_open_port.rb
+- lib/krump/ssh_tunnel_info.rb
+- lib/krump/ssh_tunnels.rb
+- lib/krump/version.rb
+homepage: https://github.com/fundingcircle/krump
+licenses:
+- BSD 3-Clause
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.0.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.5
+signing_key: 
+specification_version: 4
+summary: A Kafka consumer focused on convenience.
+test_files: []