nsq-krakow 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 27cdf9c39570b9a623614f40d8337722d76e3f5e
+  data.tar.gz: 96b61e498c662d817dddd4c4e560127b40724b27
+SHA512:
+  metadata.gz: 30f924e9a9521464dab1afffa017bcf8e5cfc2661a66f056e18954b699f436b7222d0ee52ee603d35080d9cb29ffd429df207b61886585ea4b8e2c96a73cef10
+  data.tar.gz: 82e1d149c54b245df61b36babe1c94c318cc98c2573104900c41a7cdd7cffcee7ede68e15ee1511173a647df0bc6312c6986eaf3af5356a50687c4be8b152ea9

data/CHANGELOG.md

@@ -0,0 +1,84 @@
+## v0.4.2
+* Update command subclass loading to be explicit (remove glob loading)
+
+## v0.4.0
+* Prevent duplicate connection consumption loops (#23) (thanks @phopkins)
+* Refactor connection implementation to prevent crashing blocks (#22) (thanks @phopkins)
+* Properly calculate message sizes using byte length, not string length (#24) (thanks @phopkins)
+* Clear responses prior to message transmission (#20) (thanks @i2amsam)
+* Rebuild testing specs (not fully covered, but a good start)
+* Consumer and Producer provide better connection failure recovery
+* Fix in-flight issues within distribution on connection failures
+
+_NOTE: Large portions of `Krakow::Connection` has been refactored_
+
+## v0.3.12
+* Update Consumer#confirm and Consumer#touch to rescue out lookups and abort
+
+## v0.3.10
+* Remove exclusive from Connection#init!
+
+## v0.3.8
+* Remove locks and move logic to connection access
+* Check for result within response prior to access (prevent slaying actor)
+
+## v0.3.6
+* Allow `:options` key within `Producer` to set low level connection settings
+* Make snappy an optional dependency
+* Add initial support for authentication
+* Update allowed types for optional notifier
+
+## v0.3.4
+* Explicitly require version file (#11 and #12)
+
+## v0.3.2
+* Fix return value from Connection#wait_time_for (#9) (thanks @AlphaB and @davidpelaez)
+
+## v0.3.0
+* Include jitter to discovery interval lookups
+* Typecast to String on PUB and MPUB
+* Update exception types used for not implemented methods
+* Add #confirm, #requeue, and #touch helpers to FrameType::Message instances
+* Update Utils::Lazy implementation to be faster and clearer
+* Add #safe_socket method on Connection to add stability
+* Rebuild connections on error to prevent consumer teardown
+* Reference connections without requirement of connection instance being alive
+* Use #read over #recv on underlying socket to ensure proper number of bytes (thanks @thomas-holmes)
+* Expand spec testing
+
+A big thanks to @bschwartz for a large contribution in this changeset
+including expanded spec coverage, message proxy helper methods, and
+isolation of instability around Connection interactions.
+
+## v0.2.2
+* Fix `nsqlookupd` attribute in `Consumer` and `Discovery`
+
+## v0.2.0
+* Fix the rest of the namespacing issues
+* Start adding some tests
+* Use better exception types (NotImplementedError instead of NoMethodError)
+* Be smart about responses within connections
+* Add snappy support
+* Add deflate support
+* Add TLS support
+* Prevent division by zero in distribution
+* Add query methods to lazy helper (`attribute_name`?)
+
+## v0.1.2
+* Include backoff support
+* Remove `method_missing` magic
+* Force message redistribution when connection removed
+* Make discovery interval configurable
+* Add support for HTTP producer
+* Include namespace for custom exceptions #1 (thanks @copiousfreetime)
+* Fix timeout method access in req command #1 (thanks @copiousfreetime)
+
+## v0.1.0
+* Add logging support
+* Include valid responses within commands
+* Segregate responses from messages
+* Manage connections in consumer (closed/reconnect)
+* Add message distribution support
+
+## v0.0.1
+* Initial release

data/CONTRIBUTING.md

@@ -0,0 +1,25 @@
+# Contributing
+
+## Branches
+
+### `master` branch
+
+The master branch is the current stable released version.
+
+### `develop` branch
+
+The develop branch is the current edge of development.
+
+## Pull requests
+
+* https://github.com/chrisroberts/krakow/pulls
+
+Please base all pull requests of the `develop` branch. Merges to
+`master` only occur through the `develop` branch. Pull requests
+based on `master` will likely be cherry picked.
+
+## Issues
+
+Need to report an issue? Use the github issues:
+
+* https://github.com/chrisroberts/krakow/issues

data/LICENSE

@@ -0,0 +1,13 @@
+   Copyright 2014 Chris Roberts
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

data/README.md

@@ -0,0 +1,249 @@
+# Krakow
+
+"KRAKOW! KRAKOW! Two direct hits!"
+
+## Spiff
+
+```ruby
+require 'krakow'
+
+producer = Krakow::Producer.new(
+  :host => 'HOST',
+  :port => 'PORT',
+  :topic => 'target'
+)
+producer.write('KRAKOW!', 'KRAKOW!')
+```
+
+## Zargons
+
+```ruby
+require 'krakow'
+
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship'
+)
+
+consumer.queue.size # => 2
+2.times do
+  msg = consumer.queue.pop
+  puts "Received: #{msg}"
+  consumer.confirm(msg.message_id)
+end
+```
+
+## What is this?
+
+It's a Ruby library for [NSQ][1] using [Celluloid][2] under the hood.
+
+## Information and FAQ that I totally made up
+
+### Max in flight for consumers is 1, regardless of number of producers
+
+Yep, that's right. Just one lowly message at a time. And that's probably not what
+you want, so adjust it when you create your consumer instance.
+
+```ruby
+require 'krakow'
+
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :max_in_flight => 30
+)
+```
+
+### Clean up after yourself
+
+Since [Celluloid][2] is in use under the hood, and the main interaction points are
+Actors (`Consumer` and `Producer`) you'll need to be sure you clean up. This simply
+means terminating the instance (since falling out of scope will not cause it to be
+garbage collected).
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :max_in_flight => 30
+)
+
+# do stuff
+
+consumer.terminate
+```
+
+### Please make it shutup!
+
+Sure:
+
+```ruby
+Krakow::Utils::Logging.level = :warn # :debug / :info / :warn / :error / :fatal
+```
+
+### Why is it forcing something called an "unready state"?
+
+Because forcing starvation is mean. We don't want to be mean, so we'll ensure we
+are consuming from all registered connections.
+
+### I just want to connect to a producer, not a lookup service
+
+Fine!
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :host => 'HOST',
+  :port => 'PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :max_in_flight => 30
+)
+```
+Great for testing, but you really should use the lookup service in the "real world"
+
+### Backoff support
+
+NSQ has this backoff notion. It's pretty swell. Basically, if messages from a specific
+producer get re-queued (fail), then message consumption from that producer is halted,
+and slowly ramped back up. It gives time for downstream issues to work themselves out,
+if possible, instead of just keeping the firehose of gasoline on. Neat.
+
+By default backoff support is disabled. It can be enabled by setting the `:backoff_interval`
+when constructing the `Consumer`. The interval is in seconds (and yes, floats are allowed
+for sub-second intervals):
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :max_in_flight => 30,
+  :backoff_interval => 1
+)
+```
+
+### I need TLS!
+
+OK!
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :connection_options => {
+    :features => {
+      :tls_v1 => true
+    }
+  }
+)
+```
+
+### I need Snappy compression!
+
+OK!
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :connection_options => {
+    :features => {
+      :snappy => true
+    }
+  }
+)
+```
+
+*NOTE*: snappy support requires the snappy
+gem and is not provided by default, so you
+will need to ensure it is installed either
+on the system, or within the bundle.
+
+### I need Deflate compression!
+
+OK!
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :connection_options => {
+    :features => {
+      :deflate => true
+    }
+  }
+)
+```
+
+### I want to use TLS based authentication!
+
+OK!
+
+```ruby
+consumer = Krakow::Consumer.new(
+  :nsqlookupd => 'http://HOST:PORT',
+  :topic => 'target',
+  :channel => 'ship',
+  :connection_options => {
+    :features => {
+      :tls_v1 => true
+    },
+    :config => {
+      :ssl_context => {
+        :certificate => '/path/to/cert',
+        :key => '/path/to/key'
+      }
+    }
+  }
+)
+```
+
+### Running the tests
+
+Run them all!
+
+```
+bundle exec ruby test/run.rb
+```
+
+Or, run part of them:
+
+```
+bundle exec ruby test/specs/consumer_spec.rb
+```
+
+*NOTE*: the specs expect that `nsqd` and `nsqlookupd` are available in `$PATH`
+
+### It doesn't work
+
+Create an issue on the github repository
+
+* https://github.com/chrisroberts/krakow/issues
+
+### It doesn't do `x`
+
+Create an issue, or even better, send a PR.
+
+* https://github.com/chrisroberts/krakow/pulls
+
+# Info
+* Repo: https://github.com/chrisroberts/krakow
+* Docs: http://code.chrisroberts.org/krakow
+* IRC: Freenode @ spox
+
+[1]: http://bitly.github.io/nsq/ "NSQ: a realtime distributed messaging platform"
+[2]: http://celluloid.io "Celluloid: Actor-based concurrent object framework for Ruby"
+
+# Contributors
+
+* Pete Hopkins (@phopkins)
+* Sam Phillips (@i2amsam)
+* Brendan Schwartz (@bschwartz)
+* Thomas Holmes (@thomas-holmes)
+* Jeremy Hinegardner (@copiousfreetime)

data/krakow.gemspec

@@ -0,0 +1,22 @@
+$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__)) + '/lib/'
+require 'krakow/version'
+Gem::Specification.new do |s|
+  s.name = 'nsq-krakow'
+  s.version = Krakow::VERSION.version
+  s.summary = 'NSQ library'
+  s.author = 'Chris Roberts'
+  s.email = 'code@chrisroberts.org'
+  s.homepage = 'http://github.com/chrisroberts/krakow'
+  s.description = 'NSQ ruby library'
+  s.license = 'Apache 2.0'
+  s.require_path = 'lib'
+  s.add_runtime_dependency 'celluloid', '~> 0.16.0'
+  s.add_runtime_dependency 'http'
+  s.add_runtime_dependency 'multi_json'
+  s.add_runtime_dependency 'digest-crc'
+  s.add_development_dependency 'childprocess'
+  s.add_development_dependency 'snappy'
+  s.add_development_dependency 'minitest'
+  s.files = Dir['lib/**/*'] + %w(krakow.gemspec README.md CHANGELOG.md CONTRIBUTING.md LICENSE)
+  s.extra_rdoc_files = %w(CHANGELOG.md CONTRIBUTING.md LICENSE)
+end

data/lib/krakow.rb

@@ -0,0 +1,25 @@
+require 'celluloid'
+
+if(ENV['DEBUG'])
+  Celluloid.task_class = Celluloid::TaskThread
+end
+
+require 'celluloid/autostart'
+require 'multi_json'
+
+# NSQ client and producer library
+module Krakow
+  autoload :Command, 'krakow/command'
+  autoload :Connection, 'krakow/connection'
+  autoload :ConnectionFeatures, 'krakow/connection_features'
+  autoload :Consumer, 'krakow/consumer'
+  autoload :Discovery, 'krakow/discovery'
+  autoload :Distribution, 'krakow/distribution'
+  autoload :Error, 'krakow/exceptions'
+  autoload :FrameType, 'krakow/frame_type'
+  autoload :Ksocket, 'krakow/ksocket'
+  autoload :Producer, 'krakow/producer'
+  autoload :Utils, 'krakow/utils'
+end
+
+require 'krakow/version'

data/lib/krakow/command.rb

@@ -0,0 +1,89 @@
+require 'krakow'
+
+module Krakow
+  # Messages for sending to remote server
+  class Command
+
+    include Utils::Lazy
+    # @!parse include Utils::Lazy::InstanceMethods
+    # @!parse extend Utils::Lazy::ClassMethods
+
+    autoload :Auth, 'krakow/command/auth'
+    autoload :Cls, 'krakow/command/cls'
+    autoload :Fin, 'krakow/command/fin'
+    autoload :Identify, 'krakow/command/identify'
+    autoload :Mpub, 'krakow/command/mpub'
+    autoload :Nop, 'krakow/command/nop'
+    autoload :Pub, 'krakow/command/pub'
+    autoload :Rdy, 'krakow/command/rdy'
+    autoload :Req, 'krakow/command/req'
+    autoload :Sub, 'krakow/command/sub'
+    autoload :Touch, 'krakow/command/touch'
+
+    class << self
+
+      # Allowed OK return values
+      #
+      # @return [Array<String>]
+      def ok
+        []
+      end
+
+      # Allowed ERROR return values
+      #
+      # @return [Array<String>]
+      def error
+        []
+      end
+
+      # Response type expected
+      #
+      # @param message [Krakow::Message] message to check
+      # @return [Symbol] response expected (:none, :error_only, :required)
+      def response_for(message)
+        if(message.class.ok.empty?)
+          if(message.class.error.empty?)
+            :none
+          else
+            :error_only
+          end
+        else
+          :required
+        end
+      end
+
+    end
+
+    # @return [Krakow::FrameType] response to command
+    attr_accessor :response
+
+    # @return [String] name of command
+    def name
+      self.class.name.split('::').last.upcase
+    end
+
+    # Convert to line output
+    #
+    # @return [String] socket ready string
+    def to_line(*args)
+      raise NotImplementedError.new 'No line conversion method defined!'
+    end
+
+    # Is response OK
+    #
+    # @return [TrueClass, FalseClass]
+    def ok?(response)
+      response = response.content if response.is_a?(FrameType)
+      self.class.ok.include?(response)
+    end
+
+    # Is response ERROR
+    #
+    # @return [TrueClass, FalseClass]
+    def error?(response)
+      response = response.content if response.is_a?(FrameType)
+      self.class.error.include?(response)
+    end
+
+  end
+end