toot 0.1.3 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 61eb3541be9eb9395b6186880b157f86a11ad5a6
-  data.tar.gz: c1ae6a89295589372da37e5c0c96e67d4b8074b6
+  metadata.gz: 083d9404092e22b97c3cc28414d2a7544f00e4af
+  data.tar.gz: a384f7ab9ebc684a9374b2d6b24f4f1844717322
 SHA512:
-  metadata.gz: 2f7a6502c9ab7f892ac5870e766800b35c5c678e7018c070d1141c15824bfeaf07ebae97e8e902e7c96e4ce21a44c6990bff634d28ee58753055ce857b1734b6
-  data.tar.gz: f3da1d5b7acf29f6f3cca1fe3740da34014a7e5791e5691d9fef3270d2abd6da91748b2066189cb49f53efdf0a84747d4d1eab5f44dc3842b20e60a3d04de240
+  metadata.gz: fc56a2166877aad3957fa83a7a24342bac4191ab05df5b3d73fb409bb6e92740ff0592606cfa66a23741cba03bcc2f8163bddd5c4ec43bb1158a3fd454606201
+  data.tar.gz: 945bc0a83d6b9e0fc831346b835a2157fc21fbff6c0413a1df648853009bfff7cfb46fcb3245eddf9f68814759aae51fd63b16663ced58fd999064b81008ef45

data/README.md

@@ -1,8 +1,170 @@
-# Toot
+# Toot :postal_horn: :dash:
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/toot`. To experiment with that code, run `bin/console` for an interactive prompt.
+Toot is a library for basic PubSub over HTTP. Toot fires off event
+payloads, and provides lightweight Rack apps for handling event callbacks and
+subscription management that you can mount up in any Rack compatible
+app. All of these actions happen through Sidekiq providing asynchronous,
+persistent, and fault-tolerant events.
 
-TODO: Delete this and the text above, and describe your gem
+## Usage
+
+Toot is composed of several standalone components. An app can play the
+rolls of an event publisher, subscriber, or both. It just depends on how
+you configure it. Let's take a look at an example using two apps:
+"publisher" and "subscriber".
+
+### Overview
+
+Here is how you could publish an event on the channel "users.updated.v1"
+from the "publisher" app and then subscribe to that channel on the
+"subscriber" app.
+
+```ruby
+# Publish an event
+Toot.publish "users.updated.v1", id: 123
+
+# Subscribe to an event
+Toot.subscribe :publisher, "users.updated.v1", -> (event) {
+  puts "User with id #{event["id"]} was updated."
+}
+```
+
+The "subscriber" app has a named `source` called "publisher" that is
+configured in the config block. Let's take a look at what configuration
+looks like on both apps. First let's take a look at the publisher:
+
+```ruby
+# Configuration on the publisher
+Toot.config do |c|
+  c.channel_prefix = "org.example.publisher."
+end
+
+# Mount up the subscriptions service (e.g. For Rails in routes.rb)
+match "/subscriptions", to: Toot::SubscriptionsService, via: :post
+```
+
+We set the `channel_prefix` which will prepend all events we publish
+with this string. This ensures that we don't have name collisions with
+other publishers we may introduce.
+
+The `Toot::SubscriptionsService` is a lightweight Rack app that can be
+mounted up in any Rack compatible application. You can mount it at
+whatever path you like. We'll need the full URL when we configure our
+subscriber.
+
+Let's take a look at the subscriber configuration now:
+
+```ruby
+# Configuration on the subscriber
+Toot.config do |c|
+  c.callback_url = "https://subscriber.example.org/callbacks"
+  c.source :publisher,
+    subscription_url: "https://publisher.example.org/subscriptions",
+    channel_prefix: "org.example.publisher."
+end
+
+# Mount up the handler service (e.g. For Rails in routes.rb)
+match "/callbacks", to: Toot::HandlerService, via: :post
+```
+
+The `source` option defines a new event source called "publisher". We
+reference this name in any calls to `Toot.subscribe`. We define the
+`subscription_url` to match the URL that we mounted up the
+`Toot::SubscriptionsService` Rack app on the publisher, and the
+`channel_prefix` should match the publisher's.
+
+The `Toot::HandlerService` is the Rack app that handles the event
+callback from the publisher. The `callback_url` configuration option
+should match whatever URL you mount this Rack app to on your
+application. This is used in the subscibe process to let the publisher
+know how to notify us about a new event.
+
+### Event Lifecycle
+
+If you're having trouble picturing how this is actually working let's
+trace the journey of a newly published event from the "publisher" to the
+"subscriber".
+
+1. Event is published using the `Event.publish` method.
+2. This enqueues a background job which checks the Database (Redis) for
+   subscribers to this channel and enqueues another job for each
+   subscriber it finds.
+3. The subscriber's callback URL is called with the event's data payload
+4. The subscriber enqueues a background job if it has a handler for this
+   event.
+5. This background job runs the actual event handler code.
+
+### Subscribing
+
+Subscribing is handled by a callable class called
+`Toot::RegistersSusbcriptions`. There is also a rake task available that
+does the same thing called `toot:register_subscriptions`. If you are
+using Rails this rake task will be available to you automatically.
+
+### Configuration
+
+We've looked at an example, but let's look at the conifguration options
+in greater detail.
+
+* `channel_prefix`: This option sets a string that will be prepended to
+  all events published from this app. In all of our examples we've been
+  using the [reverse DNS notation][1], but you can use whatever
+  convention you'd like.
+* `callback_url`: The event handler URL that this application will
+  register to remote publishers. It should match the full URL to
+  wherever you have mounted the `Toot::HandlerService` for this app.
+* `http_connection`: An instance of Faraday::Connection which is used
+  for any external HTTP calls. You can use this to add custom middleware
+  for authorization, logging, or debugging.
+* `redis_connection`: If you'd like to use a custom Redis connection you
+  can configure a callable here that yields a redis connection. By
+  default the Sidekiq Redis connection pool is used.
+* `source`: This is a method call that defines a named source. It should
+  include a `subscription_url` option and a `channel_prefix` option.
+  * The first argument is the name of this source
+  * The `subscription_url` option is the URL to this source's
+    `Toot::SubscriptionsService` app.
+  * The `channel_prefix` should match this app's configured
+    channel_prefix. This ensures that event channel names will be built
+    correctly on the publisher side and on the subscriber side.
+* `subscribe`: This is a method call that adds a subscription to a given
+  source's channel. It takes three arguments:
+  * The source name. This must match a previously defined source
+  * The channel suffix. This will be joined with the `channel_prefix`
+    configured on the source to form the full channel name.
+  * A callable event handler. The event object will be passed in as the
+    only argument.
+
+### The Event Object
+
+The schema of the publisher's payload is a contract between publisher
+and subscriber and is outside of the scope of this library. However,
+Toot does provide some extra information to aid in debugging and general
+happiness:
+
+```ruby
+Toot.subscribe :publisher, "awesome.channel", -> (event) {
+  event.id                # => "c6c4af6b-a227-4b0e-a7b5-5259d31cf98b"
+  event.timestamp         # => 2015-10-23 11:50:09 -0500
+  event.channel           # => "org.example.publisher.awesome.channel"
+  event.payload           # => { "publisher_data" => 123 }
+  event["publisher_data"] # => 123
+}
+```
+
+### Authentication
+
+It is important to note that there is no authentication baked into
+Toot's services. If you aren't running in a trusted environment (almost
+everybody) you will need to wrap these Rack apps in some kind of
+authentication middleware. In Rails, you can use the [constraints
+routing config][2], or more generically, you can just decorate the
+service with another callable object that does the authentication
+checks.
+
+**TODO:** Currently we don't have the ability to add credentials to
+outgoing requests. This will need to be added before this problem is
+100% solved.
 
 ## Installation
 
@@ -20,17 +182,26 @@ Or install it yourself as:
 
     $ gem install toot
 
-## Usage
+### Requirements
 
-TODO: Write usage instructions here
+* Sidekiq (and therefore Redis)
+* Ruby 2.1+ (or at least that's what's tested)
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake rspec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies.
+Then, run `rake rspec` to run the tests. You can also run `bin/console`
+for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake
+install`. To release a new version, update the version number in
+`version.rb`, and then run `bundle exec rake release`, which will create
+a git tag for the version, push git commits and tags, and push the
+`.gem` file to [rubygems.org](https://rubygems.org).
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/toot.
+Bug reports and pull requests are welcome on GitHub at https://github.com/watermarkchurch/toot.
 
+[1]: https://en.wikipedia.org/wiki/Reverse_domain_name_notation
+[2]: http://guides.rubyonrails.org/routing.html#advanced-constraints

data/Rakefile

@@ -4,3 +4,10 @@ require "rspec/core/rake_task"
 RSpec::Core::RakeTask.new(:spec)
 
 task :default => :spec
+
+task :environment do
+  $LOAD_PATH.unshift File.expand_path('../lib', __FILE__)
+  require 'toot'
+end
+
+load './lib/tasks/toot.rake'

data/bin/rspec

@@ -0,0 +1,16 @@
+#!/usr/bin/env ruby
+#
+# This file was generated by Bundler.
+#
+# The application 'rspec' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require 'pathname'
+ENV['BUNDLE_GEMFILE'] ||= File.expand_path("../../Gemfile",
+  Pathname.new(__FILE__).realpath)
+
+require 'rubygems'
+require 'bundler/setup'
+
+load Gem.bin_path('rspec-core', 'rspec')

data/circle.yml

@@ -0,0 +1,6 @@
+machine:
+  ruby:
+    version: 2.2.0
+dependencies:
+  pre:
+    - gem install bundler --version='~>1.10'

data/lib/tasks/toot.rake

@@ -0,0 +1,9 @@
+task :environment
+
+namespace :toot do
+  desc 'Register subscriptions with remote sources'
+  task :register_subscriptions => :environment do
+    Toot::RegistersSubscriptions.call
+  end
+end
+

data/lib/toot.rb

@@ -1,5 +1,54 @@
-require "toot/version"
+require 'toot/version'
+
+require 'faraday'
+require 'sidekiq'
+
+require 'toot/config'
+require 'toot/event'
+require 'toot/source'
+require 'toot/subscription'
+
+require 'toot/calls_event_callback'
+require 'toot/calls_handlers'
+require 'toot/handler_service'
+require 'toot/publishes_event'
+require 'toot/registers_subscriptions'
+require 'toot/subscriptions_service'
 
 module Toot
-  # Your code goes here...
+
+  class Error < StandardError; end
+  class ConfigError < Error; end
+  class CallbackFailure < Error; end
+  class RegisterSubscriptionFailure < Error; end
+
+  def self.config
+    if block_given?
+      yield config
+    else
+      @config ||= Config.new
+    end
+  end
+
+  def self.reset_config
+    @config = Config.new
+  end
+
+  def self.publish(channel, payload, prefix: config.channel_prefix)
+    Event.new(
+      channel: [prefix, channel].join,
+      payload: payload
+    ).publish
+  end
+
+  def self.subscribe(*args)
+    config.subscribe(*args)
+  end
+
+  def self.redis(connection=config.redis_connection, &block)
+    connection.call(&block)
+  end
+
 end
+
+require 'toot/rails' if defined?(Rails)

data/lib/toot/calls_event_callback.rb

@@ -0,0 +1,27 @@
+require 'net/http'
+
+module Toot
+  class CallsEventCallback
+    include Sidekiq::Worker
+
+    def perform(callback_url, event_data)
+      uri = URI(callback_url)
+
+      response = Toot.config.http_connection.post do |request|
+        request.url uri
+        request.body = event_data.to_json
+        request.headers["Content-Type"] = "application/json"
+      end
+
+      if response.success?
+        if response.headers["X-Toot-Unsubscribe"]
+          Toot.redis do |r|
+            r.srem event_data["channel"], callback_url
+          end
+        end
+      else
+        raise CallbackFailure, "Response code: #{response.status}"
+      end
+    end
+  end
+end

data/lib/toot/calls_handlers.rb

@@ -0,0 +1,13 @@
+module Toot
+  class CallsHandlers
+    include Sidekiq::Worker
+
+    def perform(event_data)
+      event = Event.new(event_data)
+      Toot.config
+        .subscriptions_for_channel(event.channel)
+        .each { |s| s.handler.call(event) }
+    end
+
+  end
+end

data/lib/toot/config.rb

@@ -0,0 +1,58 @@
+module Toot
+  CONFIG_ATTRS = %i[
+    channel_prefix
+    http_connection
+    redis_connection
+    callback_url
+  ]
+
+  class Config < Struct.new(*CONFIG_ATTRS)
+
+    def source(name, subscription_url:, channel_prefix:)
+      Source.new(
+        name: name,
+        subscription_url: subscription_url,
+        channel_prefix: channel_prefix
+      ).tap do |source|
+        sources << source
+      end
+    end
+
+    def subscribe(source_name, channel_suffix, handler)
+      source = find_source_by_name(source_name) or
+        fail(ConfigError, "You cannot subscribe to an undefined source: #{source_name}")
+      Subscription.new(
+        source: source,
+        channel: [source.channel_prefix, channel_suffix].join,
+        handler: handler
+      ).tap do |subscription|
+        subscriptions << subscription
+      end
+    end
+
+    def sources
+      @sources ||= []
+    end
+
+    def subscriptions
+      @subscriptions ||= []
+    end
+
+    def find_source_by_name(name)
+      sources.find { |source| source.name == name }
+    end
+
+    def subscriptions_for_channel(channel)
+      subscriptions.select { |s| s.channel == channel }
+    end
+
+    def http_connection
+      self[:http_connection] ||= Faraday::Connection.new
+    end
+
+    def redis_connection
+      self[:redis_connection] ||= Sidekiq.method(:redis)
+    end
+
+  end
+end

data/lib/toot/event.rb

@@ -0,0 +1,46 @@
+require 'securerandom'
+
+module Toot
+  class Event
+
+    attr_accessor :id, :timestamp, :payload, :channel
+
+    DEFAULTS = -> {
+      {
+        id: SecureRandom.uuid,
+        timestamp: Time.now,
+        payload: {},
+      }
+    }
+
+    def initialize(args={})
+      args = DEFAULTS.().merge(args.symbolize_keys)
+      @id = args[:id]
+      @timestamp = args[:timestamp]
+      @payload = args[:payload]
+      @channel = args[:channel]
+    end
+
+    def publish
+      PublishesEvent.perform_async(to_h)
+      self
+    end
+
+    def to_h
+      {
+        id: id,
+        timestamp: timestamp,
+        payload: payload,
+        channel: channel,
+      }
+    end
+
+    def ==(other)
+      self.id == other.id
+    end
+
+    def [](key)
+      payload[key]
+    end
+  end
+end

data/lib/toot/handler_service.rb

@@ -0,0 +1,25 @@
+require 'rack'
+
+module Toot
+  class HandlerService
+
+    def call(env)
+      request = Rack::Request.new(env)
+      response = Rack::Response.new
+      event_data = JSON.parse(request.body.read)
+
+      if Toot.config.subscriptions_for_channel(event_data["channel"]).any?
+        CallsHandlers.perform_async(event_data)
+      else
+        response["X-Toot-Unsubscribe"] = "True"
+      end
+
+      response.finish
+    end
+
+    def self.call(env)
+      new.call(env)
+    end
+
+  end
+end

data/lib/toot/publishes_event.rb

@@ -0,0 +1,19 @@
+module Toot
+  class PublishesEvent
+    include Sidekiq::Worker
+
+    def perform(event_data)
+      event = Event.new(event_data)
+      channel_callback_urls(event.channel)
+        .map { |callback| CallsEventCallback.perform_async(callback, event_data) }
+    end
+
+    private
+
+    def channel_callback_urls(channel)
+      Toot.redis do |r|
+        r.smembers channel
+      end
+    end
+  end
+end

data/lib/toot/rails.rb

@@ -0,0 +1,9 @@
+module Toot
+  class Rails < Rails::Railtie
+    rake_tasks do
+      Dir.glob(File.expand_path("../../tasks/*.rake", __FILE__)).each do |rake_file|
+        load rake_file
+      end
+    end
+  end if defined?(Rails::Railtie)
+end

data/lib/toot/registers_subscriptions.rb

@@ -0,0 +1,36 @@
+module Toot
+  class RegistersSubscriptions
+
+    def call
+      subscriptions = Toot.config
+        .subscriptions
+        .each_with_object({}) { |s, hash| hash[[s.source, s.channel]] = s }
+        .values
+
+      subscriptions.each do |subscription|
+        register(subscription)
+      end
+    end
+
+    def self.call(*args)
+      new.call(*args)
+    end
+
+    private def register(subscription)
+      uri = URI(subscription.source.subscription_url)
+
+      response = Toot.config.http_connection.post uri do |request|
+        request.body = {
+          callback_url: Toot.config.callback_url,
+          channel: subscription.channel,
+        }.to_json
+        request.headers["Content-Type"] = "application/json"
+      end
+
+      unless response.success?
+        raise RegisterSubscriptionFailure, "Response code: #{response.status}"
+      end
+    end
+
+  end
+end

data/lib/toot/source.rb

@@ -0,0 +1,13 @@
+module Toot
+  class Source
+
+    attr_accessor :name, :subscription_url, :channel_prefix
+
+    def initialize(name:, subscription_url:, channel_prefix:)
+      @name = name
+      @subscription_url = subscription_url
+      @channel_prefix = channel_prefix
+    end
+
+  end
+end

data/lib/toot/subscription.rb

@@ -0,0 +1,13 @@
+module Toot
+  class Subscription
+
+    attr_accessor :source, :channel, :handler
+
+    def initialize(source:, channel:, handler:)
+      @source = source
+      @channel = channel
+      @handler = handler
+    end
+
+  end
+end

data/lib/toot/subscriptions_service.rb

@@ -0,0 +1,31 @@
+module Toot
+  class SubscriptionsService
+
+    def call(env)
+      request = Rack::Request.new(env)
+      response = Rack::Response.new
+      json = parse_body_json(request)
+
+      if json["channel"] && json["callback_url"]
+        Toot.redis do |r|
+          r.sadd json["channel"], json["callback_url"]
+        end
+      else
+        response.status = 422
+      end
+
+      response.finish
+    end
+
+    def self.call(env)
+      new.call(env)
+    end
+
+    private def parse_body_json(request)
+      JSON.parse(request.body.read)
+    rescue JSON::ParserError
+      {}
+    end
+
+  end
+end

data/lib/toot/version.rb

@@ -1,3 +1,3 @@
 module Toot
-  VERSION = "0.1.3"
+  VERSION = "0.2.0"
 end

data/toot.gemspec

@@ -9,8 +9,8 @@ Gem::Specification.new do |spec|
   spec.authors       = ["Travis Petticrew"]
   spec.email         = ["travis@petticrew.net"]
 
-  spec.summary       = %q{WIP: A tool for sounding off events to remote services}
-  spec.description   = %q{WIP: A tool for sounding off events to remote services}
+  spec.summary       = %q{Send and receive events from remote services over HTTP.}
+  spec.description   = %q{Send and receive events from remote services over HTTP.}
   spec.homepage      = "https://github.com/watermarkchurch/toot"
 
   spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
@@ -18,7 +18,12 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
+  spec.add_dependency "sidekiq", ">=2"
+  spec.add_dependency "rack", ">=1"
+  spec.add_dependency "faraday", "<1"
+
   spec.add_development_dependency "bundler", "~> 1.10"
   spec.add_development_dependency "rake", "~> 10.0"
   spec.add_development_dependency "rspec"
+  spec.add_development_dependency "webmock"
 end

metadata

@@ -1,15 +1,57 @@
 --- !ruby/object:Gem::Specification
 name: toot
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.2.0
 platform: ruby
 authors:
 - Travis Petticrew
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2015-10-16 00:00:00.000000000 Z
+date: 2015-11-09 00:00:00.000000000 Z
 dependencies:
+- !ruby/object:Gem::Dependency
+  name: sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2'
+- !ruby/object:Gem::Dependency
+  name: rack
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1'
+- !ruby/object:Gem::Dependency
+  name: faraday
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '1'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -52,7 +94,21 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: 'WIP: A tool for sounding off events to remote services'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Send and receive events from remote services over HTTP.
 email:
 - travis@petticrew.net
 executables: []
@@ -66,8 +122,22 @@ files:
 - README.md
 - Rakefile
 - bin/console
+- bin/rspec
 - bin/setup
+- circle.yml
+- lib/tasks/toot.rake
 - lib/toot.rb
+- lib/toot/calls_event_callback.rb
+- lib/toot/calls_handlers.rb
+- lib/toot/config.rb
+- lib/toot/event.rb
+- lib/toot/handler_service.rb
+- lib/toot/publishes_event.rb
+- lib/toot/rails.rb
+- lib/toot/registers_subscriptions.rb
+- lib/toot/source.rb
+- lib/toot/subscription.rb
+- lib/toot/subscriptions_service.rb
 - lib/toot/version.rb
 - toot.gemspec
 homepage: https://github.com/watermarkchurch/toot
@@ -92,5 +162,5 @@ rubyforge_project:
 rubygems_version: 2.4.2
 signing_key: 
 specification_version: 4
-summary: 'WIP: A tool for sounding off events to remote services'
+summary: Send and receive events from remote services over HTTP.
 test_files: []