circuitry 3.1.3 → 3.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1f3c99a64adc915007123a1956871036f326888b
-  data.tar.gz: 8e87c4341121bbee53f2eab1de36a3ac7c946f00
+  metadata.gz: 99b3ddd979a6bea1d3a1dfaba8222705d54d0444
+  data.tar.gz: e6a6a65560048bb702b66f6c5ace6d04449470e3
 SHA512:
-  metadata.gz: 1109edbb48c25e003d4b1ecd0254d955cd5dd47109414064cedc2bc4fc8af45b11b0f962a04de15dbb8960df8368efb6f379058d053cfa6d3801418cc31da4a4
-  data.tar.gz: 81416baf0e0e6090230081ae0577670bb36ec1740e85ed1ba752475532b88fe329f8045a36c58d2ba1ad9e28c9dd0dc6202cc5770a5bab17fd83dd4250477fa2
+  metadata.gz: c7771d7687a28ea226bcff384cfa30baebabbd780983e4bdb593abc377f128e09a2d72d6dacfa3f43615e5b48153f2433d97559cfd8d4a54175dd6993a96a9c4
+  data.tar.gz: eee566c00b1239c2e87d80dff6dbb04573b5cc9bf0c43f58c6b937410548763c8d82745dc4ae4a9c0d2957e378c2b6cd57c7791d006e1ecba6e39e3c20e0b127

data/CHANGELOG.md

@@ -1,3 +1,11 @@
+## Circuitry 3.1.4 (Apr 21, 2016)
+
+* Fixed issue with `circuitry help` missing dependency. *Matt Huggins*
+* Fixed issue with `circuitry:setup` rake task when no topics are defined. *Matt Huggins*
+* Fixed issues with `circuitry:setup` rake task in vanilla Ruby projects. *Matt Huggins*
+* Removed subscriber flush middleware in favor of an inline flush. *Matt Huggins*
+* Added SNS internal failure as a retriable error. *Matt Huggins*
+
 ## Circuitry 3.1.3 (Mar 23, 2016)
 
 * Added retries for message publishing if there's an SNS connection failure. *Matt Huggins*

data/README.md

@@ -7,9 +7,26 @@ Decouple ruby applications using [SNS](http://aws.amazon.com/sns/) fanout with [
 
 A Circuitry publisher application can broadcast events which can be fanned out to any number of SQS queues. This technique is a [common approach](http://docs.aws.amazon.com/sns/latest/dg/SNS_Scenarios.html) to implementing an enterprise message bus. For example, applications which care about billing or new user onboarding can react when a user signs up, without the origin web application being concerned with those domains. In this way, new capabilities can be connected to an enterprise system without change proliferation.
 
-## How is Circuitry different from Shoryuken?
+## Features
 
-[Shoryuken](https://github.com/phstc/shoryuken) is a way to leverage SQS to execute workloads later within the same application. Circuitry is a way to execute any number of workloads in different applications after an event has taken place.
+What circuitry provides:
+
+* *Decoupling:* apps can send and receive messages to each other without explicitly coding destinations into your app.
+* *Fan-out:* multiple queues (i.e.: multiple apps) can receive the same message by publishing it a single time.
+* *Reliability:* if your app goes down (intentionally or otherwise), messages will be waiting in the queue whenever it starts up again.
+* *Speed:* because it's built on AWS, message delivery and receipt is *fast*.
+* *Duplication:* although SQS messages can be delivered multiple times, circuitry safeguards to ensure they're only received by your app once.
+* *Retries:* if a received message fails to be processed, it will be retried (unless otherwise configured).
+* *Customization:* configure your publisher and subscriber to behave the way each app independently expects.
+
+What circuitry does not provide:
+
+* *Ordering:* messages may not arrive in the order they were sent.
+* *Scheduling:* messages are processed as they're received.
+
+## Example
+
+A [circuitry-example](https://github.com/kapost/circuitry-example) app is available to quickly try the gem on your own AWS account.
 
 ## Installation
 
@@ -154,6 +171,8 @@ two methods: the circuitry CLI or the `rake circuitry:setup` task. The rake task
 subscriber queue and publishing topics that are configured within your application.
 
 ```ruby
+require 'circuitry/tasks'
+
 Circuitry.subscriber_config do |c|
   c.queue_name = 'myapp-production-events'
   c.topic_names = ['theirapp-production-stuff-created', 'theirapp-production-stuff-deleted']
@@ -555,16 +574,6 @@ your middleware:
 * `#clear`: Removes all middleware classes from the chain.
   * `middleware.clear`
 
-### Default Subscriber Middleware:
-
-* `Circuitry::Middleware::Entries::Flush`: ensures any pending async publishes are run after a
-  message is received.  Useful for publishes that happen as a result of processing received
-  messages.
-
-### Default Publisher Middleware:
-
-None.
-
 ## Testing
 
 Circuitry provides a simple option for testing publishing and subscribing without actually hitting

data/circuitry.gemspec

@@ -22,6 +22,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'aws-sdk', '~> 2'
   spec.add_dependency 'retries', '~> 0.0.5'
   spec.add_dependency 'virtus', '~> 1.0'
+  spec.add_dependency 'thor'
 
   spec.add_development_dependency 'bundler', '~> 1.8'
   spec.add_development_dependency 'codeclimate-test-reporter'
@@ -34,5 +35,4 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'redis'
   spec.add_development_dependency 'rspec', '~> 3.2'
   spec.add_development_dependency 'rspec-its', '~> 1.2'
-  spec.add_development_dependency 'thor'
 end

data/lib/circuitry.rb

@@ -18,23 +18,23 @@ require 'circuitry/version'
 module Circuitry
   class << self
     def subscriber_config
-      @_sub_config ||= Config::SubscriberSettings.new
-      yield @_sub_config if block_given?
-      @_sub_config
+      @sub_config ||= Config::SubscriberSettings.new
+      yield @sub_config if block_given?
+      @sub_config
     end
 
     def subscriber_config=(options)
-      @_sub_config = Config::SubscriberSettings.new(options)
+      @sub_config = Config::SubscriberSettings.new(options)
     end
 
     def publisher_config
-      @_pub_config ||= Config::PublisherSettings.new
-      yield @_pub_config if block_given?
-      @_pub_config
+      @pub_config ||= Config::PublisherSettings.new
+      yield @pub_config if block_given?
+      @pub_config
     end
 
     def publisher_config=(options)
-      @_pub_config = Config::PublisherSettings.new(options)
+      @pub_config = Config::PublisherSettings.new(options)
     end
 
     def publish(topic_name, object, options = {})

data/lib/circuitry/config/publisher_settings.rb

@@ -11,12 +11,6 @@ module Circuitry
         validate_setting(value, Publisher.async_strategies)
         super
       end
-
-      def middleware
-        @middleware ||= Middleware::Chain.new
-        yield @middleware if block_given?
-        @middleware
-      end
     end
   end
 end

data/lib/circuitry/config/shared_settings.rb

@@ -16,6 +16,12 @@ module Circuitry
         base.attribute :async_strategy, Symbol, default: ->(_page, _att) { :fork }
       end
 
+      def middleware
+        @middleware ||= Middleware::Chain.new
+        yield @middleware if block_given?
+        @middleware
+      end
+
       def aws_options
         {
           access_key_id:     access_key,

data/lib/circuitry/config/subscriber_settings.rb

@@ -1,6 +1,5 @@
 require 'virtus'
 require 'circuitry/config/shared_settings'
-require 'circuitry/middleware/entries/flush'
 
 module Circuitry
   module Config
@@ -28,16 +27,6 @@ module Circuitry
           raise ConfigError, "invalid lock strategy \"#{value.inspect}\""
         end
       end
-
-      def middleware
-        @middleware ||= Middleware::Chain.new do |middleware|
-          middleware.add Middleware::Entries::Flush
-        end
-
-        yield @middleware if block_given?
-
-        @middleware
-      end
     end
   end
 end

data/lib/circuitry/provisioning/provisioner.rb

@@ -1,3 +1,4 @@
+require 'circuitry'
 require 'circuitry/provisioning/queue_creator'
 require 'circuitry/provisioning/topic_creator'
 require 'circuitry/provisioning/subscription_creator'

data/lib/circuitry/provisioning/queue_creator.rb

@@ -25,11 +25,11 @@ module Circuitry
       end
 
       def create_queue
-        @_queue ||= Queue.new(create_primary_queue_internal)
+        @queue ||= Queue.new(create_primary_queue_internal)
       end
 
       def create_dead_letter_queue(name, max_receive_count)
-        @_dl_queue ||= Queue.new(create_dl_queue_internal(name, max_receive_count))
+        @dl_queue ||= Queue.new(create_dl_queue_internal(name, max_receive_count))
       end
 
       private

data/lib/circuitry/provisioning/subscription_creator.rb

@@ -26,10 +26,10 @@ module Circuitry
         topics.each do |topic|
           sns.subscribe(topic_arn: topic.arn, endpoint: queue.arn, protocol: 'sqs')
         end
-        sqs.set_queue_attributes(
-          queue_url: queue.url,
-          attributes: build_policy
-        )
+
+        if topics.any?
+          sqs.set_queue_attributes(queue_url: queue.url, attributes: build_policy)
+        end
       end
 
       private

data/lib/circuitry/provisioning/topic_creator.rb

@@ -17,10 +17,10 @@ module Circuitry
       end
 
       def topic
-        return @_topic if defined?(@_topic)
+        return @topic if defined?(@topic)
 
         response = sns.create_topic(name: topic_name)
-        @_topic = Topic.new(response.topic_arn)
+        @topic = Topic.new(response.topic_arn)
       end
 
       private

data/lib/circuitry/publisher.rb

@@ -16,7 +16,8 @@ module Circuitry
     }.freeze
 
     CONNECTION_ERRORS = [
-      Seahorse::Client::NetworkingError
+      ::Seahorse::Client::NetworkingError,
+      ::Aws::SNS::Errors::InternalFailure
     ].freeze
 
     attr_reader :timeout
@@ -59,7 +60,7 @@ module Circuitry
             logger.warn("Error publishing attempt ##{attempt_number}: #{error.class} (#{error.message}); retrying...")
           end
 
-          with_retries(max_tries: 3, handler: handler, rescue: CONNECTION_ERRORS, base_sleep_seconds: 0, max_sleep_seconds: 0) do
+          with_retries(max_tries: 3, handler: handler, rescue: CONNECTION_ERRORS, base_sleep_seconds: 0.05, max_sleep_seconds: 0.25) do
             topic = Topic.find(topic_name)
             sns.publish(topic_arn: topic.arn, message: message)
           end

data/lib/circuitry/subscriber.rb

@@ -108,6 +108,7 @@ module Circuitry
 
       poller.poll(max_number_of_messages: batch_size, wait_time_seconds: wait_time, skip_delete: true) do |messages|
         process_messages(Array(messages), &block)
+        Circuitry.flush
       end
     end
 

data/lib/circuitry/tasks.rb

@@ -1,6 +1,7 @@
 namespace :circuitry do
   desc 'Create subscriber queues and subscribe queue to topics'
-  task setup: :environment do
+  task :setup do
+    require 'logger'
     require 'circuitry/provisioning'
 
     logger = Logger.new(STDOUT)

data/lib/circuitry/version.rb

@@ -1,3 +1,3 @@
 module Circuitry
-  VERSION = '3.1.3'
+  VERSION = '3.1.4'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: circuitry
 version: !ruby/object:Gem::Version
-  version: 3.1.3
+  version: 3.1.4
 platform: ruby
 authors:
 - Matt Huggins
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-03-23 00:00:00.000000000 Z
+date: 2016-04-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: aws-sdk
@@ -53,6 +53,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -207,20 +221,6 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.2'
-- !ruby/object:Gem::Dependency
-  name: thor
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 description: A Circuitry publisher application can broadcast events which can be processed
   independently by Circuitry subscriber applications.
 email:
@@ -258,7 +258,6 @@ files:
 - lib/circuitry/locks/redis.rb
 - lib/circuitry/message.rb
 - lib/circuitry/middleware/chain.rb
-- lib/circuitry/middleware/entries/flush.rb
 - lib/circuitry/middleware/entry.rb
 - lib/circuitry/processor.rb
 - lib/circuitry/processors/batcher.rb

data/lib/circuitry/middleware/entries/flush.rb

@@ -1,16 +0,0 @@
-module Circuitry
-  module Middleware
-    module Entries
-      class Flush
-        def initialize(_options = {})
-        end
-
-        def call(_topic, _message)
-          yield
-        ensure
-          Circuitry.flush
-        end
-      end
-    end
-  end
-end