nervion 0.0.1 → 0.0.2

data/.gitignore

@@ -1,4 +1,6 @@
-TODOs
 .DS_Store
-examples
+.rbx
+.yardoc
 coverage
+doc
+examples

data/.travis.yml

@@ -0,0 +1,9 @@
+language: ruby
+matrix:
+  allow_failures:
+    - rvm: ruby-head
+rvm:
+  - rbx-19mode
+  - 1.9.2
+  - 1.9.3
+  - ruby-head

data/CHANGELOG.md

@@ -0,0 +1,9 @@
+# version 0.0.2
+
+- Support for firehose endpoint
+- Validate client setup before start streaming
+
+# version 0.0.1
+
+- Support for sample and filter endpoints
+- OAuth authentication

data/Gemfile.lock

@@ -22,6 +22,7 @@ GEM
     http_parser.rb (0.5.3)
     json (1.7.3)
     multi_json (1.3.6)
+    rake (0.9.2.2)
     rspec (2.9.0)
       rspec-core (~> 2.9.0)
       rspec-expectations (~> 2.9.0)
@@ -35,6 +36,7 @@ GEM
       simplecov-html (~> 0.5.3)
     simplecov-html (0.5.3)
     yajl-ruby (1.1.0)
+    yard (0.8.2.1)
 
 PLATFORMS
   ruby
@@ -42,5 +44,7 @@ PLATFORMS
 DEPENDENCIES
   cucumber
   nervion!
+  rake
   rspec
   simplecov
+  yard

data/README.md

@@ -1,24 +1,16 @@
-# Nervion
+# Nervion [![Build Status](https://secure.travis-ci.org/jacegu/nervion.png?branch=master)][travis] [![Dependency Status](https://gemnasium.com/jacegu/nervion.png?travis)][gemnasium]
 
-**A minimalistic Twitter Stream API Ruby client**.
+**A minimalistic Ruby client for the
+[Public Streams](https://dev.twitter.com/docs/streaming-apis/streams/public)
+of Twitter Streaming API**.
 
-
-## Motivation
-
-In our current project we had the need to consume the stream provided by twitter.
-Although there are a few gems available we had to suffer the pain of poor
-documentation and error swallowing, which made us lose a lot of time.
-
-At that point I decided to build one on my own, and that's why you are reading
-this.
+[travis]: http://travis-ci.org/jacegu/nervion
+[gemnasium]: https://gemnasium.com/jacegu/nervion
 
 
 
 ## Installation
 
-**WARNING: This project hasn't been released as a Gem yet**. This is here for
-future reference.
-
 Add this line to your application's Gemfile:
 
     gem 'nervion'
@@ -33,36 +25,29 @@ Or install it yourself as:
 
 
 
-## Usage
+## Overview
 
 Nervion mimics the endpoints provided by the
-[Twitter Stream API](https://dev.twitter.com/docs/streaming-apis).
-Currently it supports the
-[Public Streams](https://dev.twitter.com/docs/streaming-apis/streams/public).
-In the future we will add support for the
-[User Streams](https://dev.twitter.com/docs/streaming-apis/streams/user)
-and the
-[Site Streams](Use://dev.twitter.com/docs/streaming-apis/streams/site).
-
-Specifically the two calls that are that are available to the broad audience:
+[Twitter Stream API](https://dev.twitter.com/docs/streaming-apis)
+through the following methods:
 
 - [`follow`](https://dev.twitter.com/docs/api/1/post/statuses/filter)
 - [`sample`](https://dev.twitter.com/docs/api/1/get/statuses/sample)
-
-[`firehose`](https://dev.twitter.com/docs/api/1/get/statuses/firehose)
-is not supported yet since requires a special access level.
-
-Checkout the docs of both endpoints to know what tweets you can query the
-Streaming API for.
-
-You can specify any of the parameters supported by the endpoints by passing them
+- [`firehose`](https://dev.twitter.com/docs/api/1/get/statuses/firehose)
+*notice that the firehose support hasn't been tested against the actual API
+since it requires a level of access I don't have. If you were able to verify
+that it works, please, let me know*
+
+Checkout the docs of the endpoints to know what tweets you can query the
+Streaming API for and what parameters you have to provide to do so. You can
+specify any of the parameters supported by the endpoints by passing them
 as named parameters to the provided methods:
 
 ```ruby
-require 'nervion'
-
-Nervion.filter(delimited: 1953, track: 'ruby', stall_warnings: true) do |parsed_status|
-  #do something with the parsed status
+# This is tracking every tweet that includes the string "madrid" OR any tweet
+# that is geo-located in Madrid.
+Nervion.filter(track: 'madrid', locations: '40.364,-3.760,40.365,-3.609') do |message|
+  # do something with the message
 end
 ```
 
@@ -72,7 +57,7 @@ Twitter.
 
 
 
-###Authentication
+## Authentication
 
 Since Twitter plans to remove support for basic auth eventually, **Nervion only
 supports OAuth authentication**.
@@ -81,15 +66,16 @@ You can provide the tokens and secrets in a configuration flavour:
 
 ```ruby
 Nervion.configure do |config|
-  config.consumer_key = the_consumer_key
-  config.consumer_secret = the_consumer_secret
-  config.access_token = the_access_token
-  config.access_token_secret = the_access_token_secret
+  config.consumer_key        = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
+  config.consumer_secret     = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
+  config.access_token        = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
+  config.access_token_secret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
 end
 ```
 
 
-###Parsing JSON
+
+## JSON Parsing
 
 **Nervion will parse the JSON returned by twitter for you**. It uses
 [Yajl](https://github.com/brianmario/yajl-ruby) as JSON parser for its out of
@@ -100,37 +86,42 @@ to use symbols to fetch data in the callbacks.
 
 
 
-###Callbacks
+## Callbacks
 
-Nowdays Nervion only has one callback that acts upon the received statuses. It
-**will support callbacks on specific types of tweets and errors** in future
-versions.
+Nervion provides three callbacks:
 
-The callbacks will receive only one parameter: the hash with the symbolized keys
-resultant of the JSON parsing. You get to choose what to do with the hash:
-[mash](https://github.com/intridea/hashie) it before working with it or even
-wrap it in some object that specializes on querying the information that is
-relevant to you.
-
-To know what keys to expect you should browse the
-[*Platform Objects Documentation*](https://dev.twitter.com/docs/platform-objects/tweets).
+- **Message callback**: called when message is received and parsed
+- **HTTP error callback**: called when Twitter responds with a status above 200
+- **Network error callback**: called when the connection to the stream is lost
 
 
-####Status Callback
+### Message Callback
 
-You can setup a callback that **acts on all the received statuses** by simply
+You must setup a callback that **acts on all the received messages** by simply
 passing in a block to the API call you are making:
 
 ```ruby
-Nervion.sample { |status| puts status[:text] if status.has_key? :text }
+Nervion.sample { |message| puts message[:text] if message.has_key? :text }
 ```
 
-Be aware that **the callback will be called with any type of timeline update**
-(or even with warnings if the `stall_warnings` parameter is set to `true`. Keep
-this in mind when querying the hash.
+Be aware that
+**[every message type](https://dev.twitter.com/docs/streaming-apis/messages)
+will trigger this callback**. Keep this in mind when querying the hash.
+
+The callback receives only one parameter: the hash with the symbolized keys
+resultant of the JSON parsing. You get to choose what to do with the hash:
+[mash](https://github.com/intridea/hashie) it before working with it or even
+wrap it in some object that specializes on querying the information that is
+relevant to you.
+
+To know what keys to expect you should browse the
+[*Platform Objects Documentation*](https://dev.twitter.com/docs/platform-objects/tweets)
+and know the different
+[message types](https://dev.twitter.com/docs/streaming-apis/messages)
+.
 
 
-#### HTTP Error Callback
+### HTTP Error Callback
 
 This callback will be executed when the Streaming API sends a response with a
 status code above 200. After the callback has been executed a retry will be
@@ -142,17 +133,18 @@ You can setup the callback like this:
 
 ```ruby
 Nervion.on_http_error do |status, body|
-  puts "Response status was: #{status}"
-  puts "Response body was: #{body}"
+  puts "the status of the response was: #{status}"
+  puts "the body of the response body was: #{body}"
 end
 ```
 
-If no callback is set, Nervion's default behaviour will be to output the an
-error message to `STDERR` that contains both the status and the body of Twitter
-Streaming API's response.
+Given that most of the HTTP errors are due to client configuration, if no
+callback is set, Nervion's default behaviour will be to output an error message
+to `STDERR` that contains both the status and the body of Twitter Streaming
+API's response.
 
 
-#### Network Error callback
+### Network Error callback
 
 This callback will be executed when the connection with the Twitter Stream API
 is unexpectedly closed.
@@ -163,10 +155,26 @@ Nervion.on_network_error do
 end
 ```
 
-**Nervion will do nothing by default when network errors occurr** because it is
+Nervion will do nothing by default when network errors occur because it is
 unlikely that they are provoked by the client itself.
 
 
+### Callback chaining
+
+Callback setup can be chained like this:
+
+```ruby
+Nervion.on_network_error do
+  #do something about the error
+end.on_http_error do |status, body|
+  #do something about the error
+end.sample do |status|
+  #do something with the status
+end
+```
+
+
+
 ## EventMachine Integration
 
 Nervion runs on the top of EventMachine.
@@ -174,6 +182,8 @@ Nervion runs on the top of EventMachine.
 In the near future this `README` will provide a guideline to take advantage of
 the benefits that EventMachine can provide when used correctly.
 
+
+
 ## Roadmap
 
 There are some features that are needed and that will be developed before the first
@@ -184,14 +194,12 @@ release of the gem:
   - <del>Adhere to the
   [Twitter Connection guidelines](https://dev.twitter.com/docs/streaming-api/concepts#connecting)</del>
   *done!*
-  - Take advantage of EventMachine deferrables on callbacks
-  - Rewrite and improve the DSL provided to setup Nervion
+  - <del>Improve the DSL provided to setup Nervion to validate the client
+  setup</del> *done!*
 
-Once those basic features are provided there are a few more that will be very
-interesting to have:
+Future features will be:
 
   - Use a gzip compressed stream
-  - Add callbacks to act on specific types of tweets: i.e. `on_retweet`,
-  `on_deleted_status`
-
-If people start using the client more features will be added.
+  - Be able to configure the client to skip parsing and yield bare Strings with
+  the JSON of the streamed messages. The objective is to improve performance by
+  parsing in other process.

data/Rakefile

@@ -1,10 +1,11 @@
-#!/usr/bin/env rake
-
-require "bundler/gem_tasks"
+require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
 require 'cucumber/rake/task'
+require 'yard'
 
 task :default => [:test]
+
+desc 'Run all the features and specs'
 task :test    => [:rspec, :cucumber]
 
 RSpec::Core::RakeTask.new(:rspec) do |t|
@@ -15,7 +16,10 @@ Cucumber::Rake::Task.new(:cucumber) do |t|
   t.cucumber_opts = '--format progress'
 end
 
-task :cov do
+YARD::Rake::YardocTask.new
+
+desc 'Generate test coverage report'
+task :coverage do
   ENV['COVERAGE'] = 'true'
   Rake::Task[:rspec].execute
   Rake::Task[:cucumber].execute

data/features/{streaming.feature → callbacks.feature}

@@ -1,7 +1,8 @@
 Feature: Callbacks
 
   Background:
-    Given Nervion is connected to Twitter Streaming API
+    Given Nervion has been configured
+      And Nervion is connected to Twitter Streaming API
 
   Scenario: Calling the status callback
      When a status update is sent by Twitter

data/features/client_validation.feature

@@ -0,0 +1,17 @@
+Feature: Client setup validation
+
+  Scenario: Missing authentication
+    Given I haven't configured Nervion
+     When I try to start streaming
+     Then I get an error pointing me to the readme file
+
+  Scenario Outline: Calling filter without a message callback
+    Given Nervion has been configured
+     When I try to start streaming the <endpoint_name> endpoint
+     Then I get an error pointing me to the readme file
+
+    Examples:
+      | endpoint_name |
+      | filter        |
+      | firehose      |
+      | sample        |

data/features/step_definitions/client_validation_steps.rb

@@ -0,0 +1,26 @@
+AUTHENTICATION_README_URL = 'https://github.com/jacegu/nervion'
+
+def capture_errors_in
+  begin
+    yield
+  rescue Exception => error
+    @error = error
+  end
+end
+
+Given /^I haven't configured Nervion$/ do
+  Nervion::Configuration.instance_variable_set '@configured', nil
+end
+
+When /^I try to start streaming$/ do
+  capture_errors_in { Nervion.sample { |status| puts status } }
+end
+
+When /^I try to start streaming the (.*?) endpoint$/ do |endpoint_name|
+  params = { stall_warnings: true }
+  capture_errors_in { Nervion.send(endpoint_name.to_sym, params) }
+end
+
+Then /^I get an error pointing me to the readme file$/ do
+  @error.message.should match /#{AUTHENTICATION_README_URL}/
+end

data/features/step_definitions/streaming_steps.rb

@@ -1,3 +1,4 @@
+KNOWN_STATUS_COUNT = 100
 TEST_HOST = '0.0.0.0'
 TEST_PORT = '9000'
 
@@ -12,7 +13,16 @@ def test_client_with(server_version)
   EM.run do
     EM.start_server(TEST_HOST, TEST_PORT, server_version)
     EM.add_timer(0)   { Nervion.sample { |status| @statuses << status } }
-    EM.add_timer(0.1) { EM.stop }
+    EM.add_timer(0.3) { EM.stop }
+  end
+end
+
+Given /^Nervion has been configured$/ do
+  Nervion.configure do |config|
+    config.consumer_key        = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
+    config.consumer_secret     = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
+    config.access_token        = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
+    config.access_token_secret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
   end
 end
 
@@ -42,7 +52,7 @@ When /^a network error occurs$/ do
 end
 
 Then /^Nervion calls the status callback with it$/ do
-  @statuses.count.should eq 100
+  @statuses.count.should eq KNOWN_STATUS_COUNT
 end
 
 Then /^Nervion calls the HTTP error callback$/ do

data/lib/nervion.rb

@@ -1,4 +1,5 @@
 $: << File.join(File.dirname(__FILE__), '..')
+$: << File.join(File.dirname(__FILE__), '..', 'lib')
 
 require 'nervion/facade'
 require 'nervion/version'

data/lib/nervion/client.rb

@@ -1,5 +1,3 @@
-require 'nervion/configuration'
-require 'nervion/request'
 require 'nervion/stream'
 require 'nervion/stream_handler'
 
@@ -11,12 +9,12 @@ module Nervion
     end
 
     def stream(request, callbacks)
-      @stream_handler = StreamHandler.new(callbacks)
-      EM.run { EM.connect @host, @port, Stream, request, @stream_handler }
+      handler = StreamHandler.new(callbacks)
+      EM.run { @stream = EM.connect @host, @port, Stream, request, handler }
     end
 
     def stop
-      @stream_handler.close_stream
+      @stream.close
       EM.stop
     end
   end

data/lib/nervion/configuration.rb

@@ -1,12 +1,19 @@
 module Nervion
 
+  # Allows to configure Nervion.
+  #
+  # @yieldparam [Configuration] config the configuration object.
   def self.configure
+    Configuration.configured!
     yield Configuration
   end
 
   class Configuration
     UNCONFIGURED_SETTING = ''
 
+    # Configures the consumer key
+    #
+    # @param [String] consumer_key the consumer key
     def self.consumer_key=(consumer_key)
       @consumer_key = consumer_key
     end
@@ -15,6 +22,9 @@ module Nervion
       @consumer_key || UNCONFIGURED_SETTING
     end
 
+    # Configures the consumer secret
+    #
+    # @param [String] consumer_secret the consumer secret
     def self.consumer_secret=(consumer_secret)
       @consumer_secret = consumer_secret
     end
@@ -23,6 +33,9 @@ module Nervion
       @consumer_secret || UNCONFIGURED_SETTING
     end
 
+    # Configures the access token
+    #
+    # @param [String] access_token the access token
     def self.access_token=(access_token)
       @access_token = access_token
     end
@@ -31,6 +44,9 @@ module Nervion
       @access_token || UNCONFIGURED_SETTING
     end
 
+    # Configures the access token secret
+    #
+    # @param [String] access_token_secret the access token secret
     def self.access_token_secret=(access_token_secret)
       @access_token_secret = access_token_secret
     end
@@ -46,5 +62,13 @@ module Nervion
     def self.fetch(setting)
       send setting.to_sym
     end
+
+    def self.configured?
+      @configured
+    end
+
+    def self.configured!
+      @configured = true
+    end
   end
 end

data/lib/nervion/facade.rb

@@ -4,31 +4,91 @@ require 'nervion/request'
 require 'nervion/configuration'
 
 module Nervion
-  STREAM_API_HOST = 'stream.twitter.com'
-  STREAM_API_PORT = 443
-  SAMPLE_ENDPOINT = "https://#{STREAM_API_HOST}/1/statuses/sample.json"
-  FILTER_ENDPOINT = "https://#{STREAM_API_HOST}/1/statuses/filter.json"
-
+  # Sets up the callback to be called upon HTTP errors (when the response from
+  # Twitter's Streaming API has a status above 200).
+  #
+  # @param [Proc] callback the callback
+  # @return [self] to allow callback setup chaining
   def self.on_http_error(&callback)
     callback_table[:http_error] = callback
     self
   end
 
+  # Sets up the callback to be called upon network errors or unexpected
+  # disconnection.
+  #
+  # @param [Proc] callback the callback
+  # @return [self] to allow callback setup chaining
   def self.on_network_error(&callback)
     callback_table[:network_error] = callback
     self
   end
 
+  # Sets up the message callback and starts streaming the sample endpoint.
+  #
+  # @see https://dev.twitter.com/docs/api/1/get/statuses/sample
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#delimited
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#stall_warnings
+  #
+  # @param [hash] params the parameters submitted to the sample endpoint
+  # @option params [Boolean] :delimited specifies whether messages should be
+  #   length-delimited.
+  # @option params [Boolean] :stall_warnings specifies whether stall warnings
+  #   should be delivered.
+  # @param [Proc] callback the callback
   def self.sample(params = {}, &callback)
-    callback_table[:status] = callback
-    new_client.tap { |c| c.stream sample_endpoint(params), callback_table }
+    stream sample_endpoint(params), callback
   end
 
+  # Sets up the message callback and starts streaming the filter endpoint.
+  #
+  # @note At least one predicate parameter (follow, locations, or track) must be
+  #   specified.
+  #
+  # @see https://dev.twitter.com/docs/api/1/get/statuses/filter
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#follow
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#track
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#locations
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#delimited
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#stall_warnings
+  #
+  # @param [hash] params the parameters submitted to the sample endpoint
+  # @option params [String] :follow a comma separated list of user IDs,
+  #   indicating the users to return statuses for in the stream.
+  # @option params [String] :track keywords to track. Phrases of keywords are
+  #   specified by a comma-separated list.
+  # @option params [String] :locations Specifies a set of bounding boxes to track.
+  # @option params [Boolean] :delimited specifies whether messages should be
+  #   length-delimited.
+  # @option params [Boolean] :stall_warnings specifies whether stall warnings
+  #   should be delivered.
+  # @param [Proc] callback the callback
   def self.filter(params, &callback)
-    callback_table[:status] = callback
-    new_client.tap { |c| c.stream filter_endpoint(params), callback_table }
+    stream filter_endpoint(params), callback
+  end
+
+  # Sets up the message callback and starts streaming the firehose endpoint.
+  #
+  # @note This endpoint requires a special access level.
+  # @since 0.0.2
+  #
+  # @see https://dev.twitter.com/docs/api/1/get/statuses/firehose
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#count
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#delimited
+  # @see https://dev.twitter.com/docs/streaming-apis/parameters#stall_warnings
+  #
+  # @param [hash] params the parameters submitted to the sample endpoint
+  # @option params [Integer] :count the number of messages to backfill.
+  # @option params [Boolean] :delimited specifies whether messages should be
+  #   length-delimited.
+  # @option params [Boolean] :stall_warnings specifies whether stall warnings
+  #   should be delivered.
+  # @param [Proc] callback the callback
+  def self.firehose(params = {}, &callback)
+    stream firehose_endpoint(params), callback
   end
 
+  # Stops streaming
   def self.stop
     raise 'Nervion is not running' if @client.nil?
     @client.stop
@@ -40,15 +100,44 @@ module Nervion
     @callback_table ||= CallbackTable.new
   end
 
+  def self.stream(endpoint, callback)
+    raise_not_configured_error unless Configuration.configured?
+    raise_no_message_callback_error if callback.nil?
+    callback_table[:message] = callback
+    new_client.tap { |c| c.stream endpoint, callback_table }
+  end
+
   def self.new_client
     @client = Client.new(STREAM_API_HOST, STREAM_API_PORT)
   end
 
   def self.sample_endpoint(params)
-     get SAMPLE_ENDPOINT, params, Configuration
+    get SAMPLE_ENDPOINT, params, Configuration
   end
 
   def self.filter_endpoint(params)
     post FILTER_ENDPOINT, params, Configuration
   end
+
+  def self.firehose_endpoint(params)
+    get FIREHOSE_ENDPOINT, params, Configuration
+  end
+
+  def self.raise_not_configured_error
+    raise "You need to setup the authentication information for Nervion to work. Please, check out #{AUTHENTICATION_README_URL}"
+  end
+
+  def self.raise_no_message_callback_error
+    raise "You have to setup a message callback. Please, check out #{MSG_CALLBACK_README_URL}"
+  end
+
+  STREAM_API_HOST   = 'stream.twitter.com'
+  STREAM_API_PORT   = 443
+  SAMPLE_ENDPOINT   = "https://#{STREAM_API_HOST}/1/statuses/sample.json"
+  FILTER_ENDPOINT   = "https://#{STREAM_API_HOST}/1/statuses/filter.json"
+  FIREHOSE_ENDPOINT = "https://#{STREAM_API_HOST}/1/statuses/firehose.json"
+
+  AUTHENTICATION_README_URL = 'https://github.com/jacegu/nervion#authentication'
+  MSG_CALLBACK_README_URL   = 'https://github.com/jacegu/nervion#message-callback'
+
 end

data/lib/nervion/request.rb

@@ -1,14 +1,7 @@
 require_relative 'oauth_header'
+require_relative 'percent_encoder'
 
 module Nervion
-  def self.get(uri, params = {}, oauth_params)
-    Get.new uri, params, oauth_params
-  end
-
-  def self.post(uri, params = {}, oauth_params)
-    Post.new uri, params, oauth_params
-  end
-
   module Request
     attr_reader :params, :oauth_params
 
@@ -73,7 +66,6 @@ module Nervion
 
   class Post
     include Request
-    include PercentEncoder
 
     def to_s
       "#{request_line}\r\n#{headers.join("\r\n")}\r\n\r\n#{body}\r\n"
@@ -96,4 +88,15 @@ module Nervion
       percent_encode params
     end
   end
+
+  private
+
+  def self.get(uri, params = {}, oauth_params)
+    Get.new uri, params, oauth_params
+  end
+
+  def self.post(uri, params = {}, oauth_params)
+    Post.new uri, params, oauth_params
+  end
+
 end

data/lib/nervion/stream.rb

@@ -1,5 +1,5 @@
 require 'eventmachine'
-require 'nervion/http_parser'
+require 'nervion/stream_parser'
 require 'nervion/reconnection_scheduler'
 
 module Nervion
@@ -33,29 +33,37 @@ module Nervion
     end
 
     def unbind
-      handle_closed_stream unless @handler.stream_close_requested?
+      handle_closed_stream unless close_requested?
     end
 
     def http_error_occurred?
       not http_error.nil?
     end
 
+    def close_requested?
+      @close_stream ||= false
+    end
+
+    def close
+      @close_stream = true
+    end
+
     private
 
     def handle_closed_stream
       if http_error_occurred?
-        handle_http_error_and_reopen_stream
+        handle_http_error_and_retry
       else
-        handle_network_error_and_reopen_stream
+        handle_network_error_and_retry
       end
     end
 
-    def handle_http_error_and_reopen_stream
+    def handle_http_error_and_retry
       @handler.handle_http_error http_error
       @scheduler.reconnect_after_http_error_in self
     end
 
-    def handle_network_error_and_reopen_stream
+    def handle_network_error_and_retry
       @handler.handle_network_error
       @scheduler.reconnect_after_network_error_in self
     end

data/lib/nervion/stream_handler.rb

@@ -1,42 +1,24 @@
-require 'yajl'
-require 'nervion/http_parser'
+require 'nervion/stream_parser'
 
 module Nervion
   class StreamHandler
-
-    def initialize(callbacks)
-      @callbacks = callbacks
-      @http_parser = HttpParser.new(setup_json_parser)
+    def initialize(callbacks, stream_parser = StreamParser.new)
+      @callbacks, @stream_parser = callbacks, stream_parser
+      @stream_parser.on_json_parsed = @callbacks[:message]
     end
 
     def <<(data)
-      @http_parser << data
+      @stream_parser << data
     end
 
     def handle_http_error(error)
-      @http_parser.reset!
+      @stream_parser.reset!
       @callbacks[:http_error].call(error.status, error.body)
     end
 
     def handle_network_error
-      @http_parser.reset!
+      @stream_parser.reset!
       @callbacks[:network_error].call
     end
-
-    def stream_close_requested?
-      @close_stream ||= false
-    end
-
-    def close_stream
-      @close_stream = true
-    end
-
-    private
-
-    def setup_json_parser
-      Yajl::Parser.new(symbolize_keys: true).tap do |json_parser|
-        json_parser.on_parse_complete = @callbacks[:status]
-      end
-    end
   end
 end

data/lib/nervion/{http_parser.rb → stream_parser.rb}

@@ -1,12 +1,18 @@
+require 'yajl'
 require 'http/parser'
 
 module Nervion
-  class HttpParser
+  class StreamParser
     attr_reader :json_parser, :http_parser
 
-    def initialize(json_parser)
-      @json_parser = json_parser
-      @http_parser = setup_http_parser
+    def initialize(parsers = {})
+      @http_parser = parsers[:http_parser] || Http::Parser.new
+      @json_parser = parsers[:json_parser] || Yajl::Parser.new(symbolize_keys: true)
+      @http_parser.on_body = method(:process)
+    end
+
+    def on_json_parsed=(callback)
+      @json_parser.on_parse_complete = callback
     end
 
     def <<(http_stream)
@@ -19,10 +25,6 @@ module Nervion
 
     private
 
-    def setup_http_parser
-      Http::Parser.new.tap { |parser| parser.on_body = method(:process) }
-    end
-
     def process(chunk)
       if request_successful?
         parse_json_from chunk

data/lib/nervion/version.rb

@@ -1,3 +1,3 @@
 module Nervion
-  VERSION = "0.0.1"
+  VERSION = "0.0.2"
 end

data/nervion.gemspec

@@ -18,7 +18,9 @@ Gem::Specification.new do |gem|
   gem.add_runtime_dependency 'eventmachine', '~> 1.0.0.beta.4'
   gem.add_runtime_dependency 'http_parser.rb', '~> 0.5.3'
   gem.add_runtime_dependency 'yajl-ruby', '~> 1.1.0'
-  gem.add_development_dependency 'rspec'
   gem.add_development_dependency 'cucumber'
+  gem.add_development_dependency 'rake'
+  gem.add_development_dependency 'rspec'
   gem.add_development_dependency 'simplecov'
+  gem.add_development_dependency 'yard'
 end

data/spec/nervion/client_spec.rb

@@ -5,6 +5,7 @@ describe Nervion::Client do
   subject              { described_class.new('http://twitter.com', 443) }
   let(:request)        { stub :request }
   let(:callbacks)      { stub :callbacks }
+  let(:stream)         { mock :stream }
   let(:stream_handler) { stub :stream_handler }
 
   before(:all) do
@@ -43,7 +44,8 @@ describe Nervion::Client do
   end
 
   it 'stops streaming' do
-    stream_handler.should_receive(:close_stream).ordered
+    EM.stub(:connect).and_return(stream)
+    stream.should_receive(:close).ordered
     EM.should_receive(:stop).ordered
     subject.stream(request, callbacks)
     subject.stop

data/spec/nervion/configuration_spec.rb

@@ -2,11 +2,25 @@ require 'nervion/configuration'
 
 describe Nervion::Configuration do
 
+  before do
+    %w{
+      consumer_key consumer_secret access_token access_token_secret configured
+    }.each do |variable|
+      described_class.instance_variable_set "@#{variable}", nil
+    end
+  end
+
   it 'allows configuration from the top level' do
     Nervion::Configuration.should_receive(:access_key=).with 'access_key'
     Nervion.configure { |config| config.access_key = 'access_key' }
   end
 
+  it 'knows if Nervion has been configured' do
+    Nervion::Configuration.should_not be_configured
+    Nervion.configure { |config| }
+    Nervion::Configuration.should be_configured
+  end
+
   context 'when it has not been configured' do
     it 'has an empty string as consumer_key' do
       described_class.consumer_key.should eq ''

data/spec/nervion/facade_spec.rb

@@ -1,8 +1,10 @@
 require 'nervion/facade'
 
+class Callable; def call; end; end;
+
 describe "Facade that exposes Nervion's API" do
   let(:callback_table)   { mock(:callback_table).as_null_object }
-  let(:status_callback)  { lambda { :status_callback } }
+  let(:message_callback)  { lambda { :message_callback } }
   let(:http_callback)    { lambda { :http_error_callback } }
   let(:network_callback) { lambda { :network_error_callback } }
 
@@ -40,43 +42,65 @@ describe "Facade that exposes Nervion's API" do
     let(:request) { stub :request }
 
     before do
+      Nervion.configure { |config| }
       Nervion::Client.stub(:new).
         with(Nervion::STREAM_API_HOST, Nervion::STREAM_API_PORT).
         and_return(client)
     end
 
-    context 'sample endpoint' do
-      it 'sets up the status callback' do
-        callback_table.should_receive(:[]=).with(:status, status_callback)
-        Nervion.sample(&status_callback)
+    shared_examples_for 'an endpoint' do
+      it 'sets up the message callback' do
+        callback_table.should_receive(:[]=).with(:message, message_callback)
+        Nervion.send(method_name, params, &message_callback)
       end
 
       it 'starts the streaming to the sample endpoint' do
-        Nervion.stub(:get).with(Nervion::SAMPLE_ENDPOINT, params, config).
+        Nervion.stub(http_method).with(endpoint, params, config).
           and_return(request)
         client.should_receive(:stream).with(request, callback_table)
-        Nervion.sample(params, &status_callback)
+        Nervion.send(method_name, params, &message_callback)
       end
-    end
 
-    context 'filter endpoint' do
-      it 'sets up the status callback' do
-        callback_table.should_receive(:[]=).with(:status, status_callback)
-        Nervion.filter(params, &status_callback)
+      it 'raises an error if Nervion was not configured' do
+        expect do
+          Nervion::Configuration.instance_variable_set(:@configured, nil)
+          Nervion.send(method_name, params, &message_callback)
+        end.to raise_error
       end
 
-      it 'starts the streaming to the filter endpoint' do
-        Nervion.stub(:post).with(Nervion::FILTER_ENDPOINT, params, config).
-          and_return(request)
-        client.should_receive(:stream).with(request, callback_table)
-        Nervion.filter(params, &status_callback)
+      it 'raises an error if no message callback was provided' do
+        expect { Nervion.send(method_name, params) }.to raise_error
       end
     end
 
+    context 'sample endpoint' do
+      let(:http_method) { :get }
+      let(:endpoint)    { Nervion::SAMPLE_ENDPOINT }
+      let(:method_name) { :sample }
+
+      it_behaves_like 'an endpoint'
+    end
+
+    context 'filter endpoint' do
+      let(:http_method) { :post }
+      let(:endpoint)    { Nervion::FILTER_ENDPOINT }
+      let(:method_name) { :filter }
+
+      it_behaves_like 'an endpoint'
+    end
+
+    context 'firehose endpoint' do
+      let(:http_method) { :get }
+      let(:endpoint)    { Nervion::FIREHOSE_ENDPOINT }
+      let(:method_name) { :firehose }
+
+      it_behaves_like 'an endpoint'
+    end
+
     context 'stoping' do
       it 'can stop the streaming' do
         client.should_receive(:stop)
-        Nervion.sample(->{})
+        Nervion.sample{}
         Nervion.stop
       end
 

data/spec/nervion/request_spec.rb

@@ -59,7 +59,6 @@ describe Nervion::Request do
       subject.path.should eq '/endpoint?p1=param%20value&p2=%24%26'
     end
 
-
     it 'has an string representation' do
       Nervion::OAuthHeader.stub(:for).with(subject).and_return 'OAuth xxx'
       subject.to_s.should eq EXPECTED_GET_REQUEST

data/spec/nervion/stream_handler_spec.rb

@@ -3,27 +3,25 @@ require 'nervion/stream_handler'
 describe Nervion::StreamHandler do
   subject           { described_class.new callbacks }
   let(:http_parser) { mock(:http_parser).as_null_object }
-  let(:json_parser) { mock(:json_parser).as_null_object }
 
   let(:callbacks) do
     {
-      status:        status_callback,
+      message:       message_callback,
       http_error:    http_error_callback,
       network_error: network_error_callback
     }
   end
 
-  let(:status_callback)        { mock :status_callback }
+  let(:message_callback)       { mock :message_callback }
   let(:http_error_callback)    { mock(:http_error_callback).as_null_object }
   let(:network_error_callback) { mock(:network_error_callback).as_null_object }
 
   before do
-    Yajl::Parser.stub(:new).with(symbolize_keys: true).and_return(json_parser)
-    Nervion::HttpParser.stub(:new).with(json_parser).and_return(http_parser)
+    Nervion::StreamParser.stub(:new).and_return(http_parser)
   end
 
-  it 'sets up the status received callback' do
-    json_parser.should_receive(:on_parse_complete=).with(status_callback)
+  it 'sets up the message received callback' do
+    http_parser.should_receive(:on_json_parsed=).with(message_callback)
     described_class.new callbacks
   end
 
@@ -33,12 +31,6 @@ describe Nervion::StreamHandler do
     subject << data
   end
 
-  it 'can be told to close the stream' do
-    subject.stream_close_requested?.should be_false
-    subject.close_stream
-    subject.stream_close_requested?.should be_true
-  end
-
   context 'handling HTTP errors' do
     let(:http_error) { stub(:http_error, status: 401, body: 'Unauthorized') }
 

data/spec/nervion/stream_parser_spec.rb

@@ -0,0 +1,44 @@
+require 'nervion/stream_parser'
+require 'fixtures/responses'
+
+describe Nervion::StreamParser do
+  subject           { described_class.new(json_parser: json_parser) }
+  let(:json_parser) { stub(:json_parser).as_null_object }
+
+  it 'takes a JSON parser' do
+    subject.json_parser.should be json_parser
+  end
+
+  it 'uses Http::Parser as the default HTTP parser' do
+    http_parser = stub(:http_parser).as_null_object
+    Http::Parser.stub(:new).and_return(http_parser)
+    described_class.new.http_parser.should be http_parser
+  end
+
+  it 'uses Yajl with symbolized keys as the default JSON parser' do
+    yajl_parser = stub(:yajl_parser)
+    Yajl::Parser.stub(:new).with(symbolize_keys: true).and_return(yajl_parser)
+    described_class.new.json_parser.should be yajl_parser
+  end
+
+  it 'can be set up with a JSON parse complete callback' do
+    callback = lambda {}
+    json_parser.should_receive(:on_parse_complete=).with callback
+    subject.on_json_parsed = callback
+  end
+
+  it 'can be reset' do
+    subject << RESPONSE_200
+    subject.reset!
+    expect { subject << RESPONSE_200 }.not_to raise_error Http::Parser::Error
+  end
+
+  it 'parses response body if the response status is 200' do
+    json_parser.should_receive(:<<).with(BODY_200)
+    subject << RESPONSE_200
+  end
+
+  it 'raises an error if the response status is above 200' do
+    expect { subject << RESPONSE_401 }.to raise_error Nervion::HttpError
+  end
+end

data/spec/nervion/stream_spec.rb

@@ -46,10 +46,15 @@ describe Nervion::Stream do
     subject.retry
   end
 
+  it 'can be closed' do
+    subject.close_requested?.should be_false
+    subject.close
+    subject.close_requested?.should be_true
+  end
+
   context 'on unbound connections' do
     context 'due to HTTP errors' do
       before do
-        handler.stub(:stream_close_requested?).and_return(false)
         subject.stub(:http_error).and_return(http_error)
       end
 
@@ -67,8 +72,6 @@ describe Nervion::Stream do
     end
 
     context 'due to network errors' do
-      before { handler.stub(:stream_close_requested?).and_return(false) }
-
       it 'notifies the error' do
         handler.should_receive(:handle_network_error)
         scheduler.stub(:reconnect_after_network_error_in)
@@ -84,11 +87,11 @@ describe Nervion::Stream do
 
     context 'due to a request to close the stream' do
       it 'lets the stream to be closed' do
-        handler.stub(:stream_close_requested?).and_return(true)
         handler.should_not_receive(:handle_http_error)
         handler.should_not_receive(:handle_network_error)
         scheduler.should_not_receive(:reconnect_after_http_error_in)
         scheduler.should_not_receive(:reconnect_after_network_error_in)
+        subject.close
         subject.unbind
       end
     end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nervion
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.2
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-06-19 00:00:00.000000000 Z
+date: 2012-06-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: eventmachine
@@ -60,7 +60,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: 1.1.0
 - !ruby/object:Gem::Dependency
-  name: rspec
+  name: cucumber
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
@@ -76,7 +76,23 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: cucumber
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
@@ -107,6 +123,22 @@ dependencies:
     - - ! '>='
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: A minimalistic Twitter Stream API Ruby client
 email:
 - j4cegu@gmail.com
@@ -117,13 +149,17 @@ files:
 - .gitignore
 - .rspec
 - .rvmrc
+- .travis.yml
+- CHANGELOG.md
 - Gemfile
 - Gemfile.lock
 - LICENSE
 - README.md
 - Rakefile
+- features/callbacks.feature
+- features/client_validation.feature
+- features/step_definitions/client_validation_steps.rb
 - features/step_definitions/streaming_steps.rb
-- features/streaming.feature
 - features/support/env.rb
 - features/support/streaming_api_double.rb
 - fixtures/responses.rb
@@ -133,7 +169,6 @@ files:
 - lib/nervion/client.rb
 - lib/nervion/configuration.rb
 - lib/nervion/facade.rb
-- lib/nervion/http_parser.rb
 - lib/nervion/oauth_header.rb
 - lib/nervion/oauth_signature.rb
 - lib/nervion/percent_encoder.rb
@@ -141,19 +176,20 @@ files:
 - lib/nervion/request.rb
 - lib/nervion/stream.rb
 - lib/nervion/stream_handler.rb
+- lib/nervion/stream_parser.rb
 - lib/nervion/version.rb
 - nervion.gemspec
 - spec/nervion/callback_table_spec.rb
 - spec/nervion/client_spec.rb
 - spec/nervion/configuration_spec.rb
 - spec/nervion/facade_spec.rb
-- spec/nervion/http_parser_spec.rb
 - spec/nervion/oauth_header_spec.rb
 - spec/nervion/oauth_signature_spec.rb
 - spec/nervion/percent_encoder_spec.rb
 - spec/nervion/reconnection_scheduler_spec.rb
 - spec/nervion/request_spec.rb
 - spec/nervion/stream_handler_spec.rb
+- spec/nervion/stream_parser_spec.rb
 - spec/nervion/stream_spec.rb
 - spec/spec_helper.rb
 homepage: https://github.com/jacegu/nervion
@@ -181,20 +217,23 @@ signing_key:
 specification_version: 3
 summary: ''
 test_files:
+- features/callbacks.feature
+- features/client_validation.feature
+- features/step_definitions/client_validation_steps.rb
 - features/step_definitions/streaming_steps.rb
-- features/streaming.feature
 - features/support/env.rb
 - features/support/streaming_api_double.rb
 - spec/nervion/callback_table_spec.rb
 - spec/nervion/client_spec.rb
 - spec/nervion/configuration_spec.rb
 - spec/nervion/facade_spec.rb
-- spec/nervion/http_parser_spec.rb
 - spec/nervion/oauth_header_spec.rb
 - spec/nervion/oauth_signature_spec.rb
 - spec/nervion/percent_encoder_spec.rb
 - spec/nervion/reconnection_scheduler_spec.rb
 - spec/nervion/request_spec.rb
 - spec/nervion/stream_handler_spec.rb
+- spec/nervion/stream_parser_spec.rb
 - spec/nervion/stream_spec.rb
 - spec/spec_helper.rb
+has_rdoc: 

data/spec/nervion/http_parser_spec.rb

@@ -1,26 +0,0 @@
-require 'nervion/http_parser'
-require 'fixtures/responses'
-
-describe Nervion::HttpParser do
-  subject           { described_class.new(json_parser) }
-  let(:json_parser) { stub(:json_parser).as_null_object }
-
-  it 'takes a JSON parser' do
-    subject.json_parser.should be json_parser
-  end
-
-  it 'can be reset' do
-    subject << RESPONSE_200
-    subject.reset!
-    expect { subject << RESPONSE_200 }.not_to raise_error Http::Parser::Error
-  end
-
-  it 'parses response body if the response status is 200' do
-    json_parser.should_receive(:<<).with(BODY_200)
-    subject << RESPONSE_200
-  end
-
-  it 'raises an error if the response status is above 200' do
-    expect { subject << RESPONSE_401 }.to raise_error Nervion::HttpError
-  end
-end