nervion 0.0.1

data/.gitignore

@@ -0,0 +1,4 @@
+TODOs
+.DS_Store
+examples
+coverage

data/.rspec

@@ -0,0 +1,2 @@
+-I .
+-I lib/

data/.rvmrc

@@ -0,0 +1 @@
+rvm use 1.9.3@nervion --create

data/Gemfile

@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+
+gemspec

data/Gemfile.lock

@@ -0,0 +1,46 @@
+PATH
+  remote: .
+  specs:
+    nervion (0.0.1)
+      eventmachine (~> 1.0.0.beta.4)
+      http_parser.rb (~> 0.5.3)
+      yajl-ruby (~> 1.1.0)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    builder (3.0.0)
+    cucumber (1.2.1)
+      builder (>= 2.1.2)
+      diff-lcs (>= 1.1.3)
+      gherkin (~> 2.11.0)
+      json (>= 1.4.6)
+    diff-lcs (1.1.3)
+    eventmachine (1.0.0.beta.4)
+    gherkin (2.11.0)
+      json (>= 1.4.6)
+    http_parser.rb (0.5.3)
+    json (1.7.3)
+    multi_json (1.3.6)
+    rspec (2.9.0)
+      rspec-core (~> 2.9.0)
+      rspec-expectations (~> 2.9.0)
+      rspec-mocks (~> 2.9.0)
+    rspec-core (2.9.0)
+    rspec-expectations (2.9.1)
+      diff-lcs (~> 1.1.3)
+    rspec-mocks (2.9.0)
+    simplecov (0.6.4)
+      multi_json (~> 1.0)
+      simplecov-html (~> 0.5.3)
+    simplecov-html (0.5.3)
+    yajl-ruby (1.1.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  cucumber
+  nervion!
+  rspec
+  simplecov

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2012 Javier Acero
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,197 @@
+# Nervion
+
+**A minimalistic Twitter Stream API Ruby client**.
+
+
+## Motivation
+
+In our current project we had the need to consume the stream provided by twitter.
+Although there are a few gems available we had to suffer the pain of poor
+documentation and error swallowing, which made us lose a lot of time.
+
+At that point I decided to build one on my own, and that's why you are reading
+this.
+
+
+
+## Installation
+
+**WARNING: This project hasn't been released as a Gem yet**. This is here for
+future reference.
+
+Add this line to your application's Gemfile:
+
+    gem 'nervion'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install nervion
+
+
+
+## Usage
+
+Nervion mimics the endpoints provided by the
+[Twitter Stream API](https://dev.twitter.com/docs/streaming-apis).
+Currently it supports the
+[Public Streams](https://dev.twitter.com/docs/streaming-apis/streams/public).
+In the future we will add support for the
+[User Streams](https://dev.twitter.com/docs/streaming-apis/streams/user)
+and the
+[Site Streams](Use://dev.twitter.com/docs/streaming-apis/streams/site).
+
+Specifically the two calls that are that are available to the broad audience:
+
+- [`follow`](https://dev.twitter.com/docs/api/1/post/statuses/filter)
+- [`sample`](https://dev.twitter.com/docs/api/1/get/statuses/sample)
+
+[`firehose`](https://dev.twitter.com/docs/api/1/get/statuses/firehose)
+is not supported yet since requires a special access level.
+
+Checkout the docs of both endpoints to know what tweets you can query the
+Streaming API for.
+
+You can specify any of the parameters supported by the endpoints by passing them
+as named parameters to the provided methods:
+
+```ruby
+require 'nervion'
+
+Nervion.filter(delimited: 1953, track: 'ruby', stall_warnings: true) do |parsed_status|
+  #do something with the parsed status
+end
+```
+
+If the API adds support for more parameters in the future they will be supported
+straight away since Nervion does no work on them: they are just submitted to
+Twitter.
+
+
+
+###Authentication
+
+Since Twitter plans to remove support for basic auth eventually, **Nervion only
+supports OAuth authentication**.
+
+You can provide the tokens and secrets in a configuration flavour:
+
+```ruby
+Nervion.configure do |config|
+  config.consumer_key = the_consumer_key
+  config.consumer_secret = the_consumer_secret
+  config.access_token = the_access_token
+  config.access_token_secret = the_access_token_secret
+end
+```
+
+
+###Parsing JSON
+
+**Nervion will parse the JSON returned by twitter for you**. It uses
+[Yajl](https://github.com/brianmario/yajl-ruby) as JSON parser for its out of
+the box support for JSON streams.
+
+**The hash keys are symbolized in the process of parsing**. You will always have
+to use symbols to fetch data in the callbacks.
+
+
+
+###Callbacks
+
+Nowdays Nervion only has one callback that acts upon the received statuses. It
+**will support callbacks on specific types of tweets and errors** in future
+versions.
+
+The callbacks will receive only one parameter: the hash with the symbolized keys
+resultant of the JSON parsing. You get to choose what to do with the hash:
+[mash](https://github.com/intridea/hashie) it before working with it or even
+wrap it in some object that specializes on querying the information that is
+relevant to you.
+
+To know what keys to expect you should browse the
+[*Platform Objects Documentation*](https://dev.twitter.com/docs/platform-objects/tweets).
+
+
+####Status Callback
+
+You can setup a callback that **acts on all the received statuses** by simply
+passing in a block to the API call you are making:
+
+```ruby
+Nervion.sample { |status| puts status[:text] if status.has_key? :text }
+```
+
+Be aware that **the callback will be called with any type of timeline update**
+(or even with warnings if the `stall_warnings` parameter is set to `true`. Keep
+this in mind when querying the hash.
+
+
+#### HTTP Error Callback
+
+This callback will be executed when the Streaming API sends a response with a
+status code above 200. After the callback has been executed a retry will be
+scheduled adhering to the
+[connection Guidelines](https://dev.twitter.com/docs/streaming-api/concepts#connecting)
+provided by twitter.
+
+You can setup the callback like this:
+
+```ruby
+Nervion.on_http_error do |status, body|
+  puts "Response status was: #{status}"
+  puts "Response body was: #{body}"
+end
+```
+
+If no callback is set, Nervion's default behaviour will be to output the an
+error message to `STDERR` that contains both the status and the body of Twitter
+Streaming API's response.
+
+
+#### Network Error callback
+
+This callback will be executed when the connection with the Twitter Stream API
+is unexpectedly closed.
+
+```ruby
+Nervion.on_network_error do
+  puts 'There was a connection error but Nervion will automatically reconnect'
+end
+```
+
+**Nervion will do nothing by default when network errors occurr** because it is
+unlikely that they are provoked by the client itself.
+
+
+## EventMachine Integration
+
+Nervion runs on the top of EventMachine.
+
+In the near future this `README` will provide a guideline to take advantage of
+the benefits that EventMachine can provide when used correctly.
+
+## Roadmap
+
+There are some features that are needed and that will be developed before the first
+release of the gem:
+
+  - <del>Provide an HTTP error callback</del> *done!*
+  - <del>Provide a network error callback</del> *done!*
+  - <del>Adhere to the
+  [Twitter Connection guidelines](https://dev.twitter.com/docs/streaming-api/concepts#connecting)</del>
+  *done!*
+  - Take advantage of EventMachine deferrables on callbacks
+  - Rewrite and improve the DSL provided to setup Nervion
+
+Once those basic features are provided there are a few more that will be very
+interesting to have:
+
+  - Use a gzip compressed stream
+  - Add callbacks to act on specific types of tweets: i.e. `on_retweet`,
+  `on_deleted_status`
+
+If people start using the client more features will be added.

data/Rakefile

@@ -0,0 +1,22 @@
+#!/usr/bin/env rake
+
+require "bundler/gem_tasks"
+require 'rspec/core/rake_task'
+require 'cucumber/rake/task'
+
+task :default => [:test]
+task :test    => [:rspec, :cucumber]
+
+RSpec::Core::RakeTask.new(:rspec) do |t|
+  t.rspec_opts = %w{ --color --format=progress --require spec/spec_helper.rb }
+end
+
+Cucumber::Rake::Task.new(:cucumber) do |t|
+  t.cucumber_opts = '--format progress'
+end
+
+task :cov do
+  ENV['COVERAGE'] = 'true'
+  Rake::Task[:rspec].execute
+  Rake::Task[:cucumber].execute
+end

data/features/step_definitions/streaming_steps.rb

@@ -0,0 +1,55 @@
+TEST_HOST = '0.0.0.0'
+TEST_PORT = '9000'
+
+def point_client_to_fake_server
+  Nervion.send :remove_const, 'STREAM_API_HOST'
+  Nervion.send :remove_const, 'STREAM_API_PORT'
+  Nervion.const_set 'STREAM_API_HOST', TEST_HOST
+  Nervion.const_set 'STREAM_API_PORT', TEST_PORT
+end
+
+def test_client_with(server_version)
+  EM.run do
+    EM.start_server(TEST_HOST, TEST_PORT, server_version)
+    EM.add_timer(0)   { Nervion.sample { |status| @statuses << status } }
+    EM.add_timer(0.1) { EM.stop }
+  end
+end
+
+Given /^Nervion is connected to Twitter Streaming API$/ do
+  point_client_to_fake_server
+end
+
+When /^a status update is sent by Twitter$/ do
+  @statuses = []
+  test_client_with WorkingStreamingApiDouble
+end
+
+When /^an HTTP error occurs$/ do
+  Nervion.on_http_error do |status, body|
+    @status, @body = status, body
+    Nervion.stop
+  end
+  test_client_with HttpErrorStreamingApiDouble
+end
+
+When /^a network error occurs$/ do
+  Nervion.on_network_error do
+    @network_error_detected = true
+    Nervion.stop
+  end
+  test_client_with NetworkErrorStreamingApiDouble
+end
+
+Then /^Nervion calls the status callback with it$/ do
+  @statuses.count.should eq 100
+end
+
+Then /^Nervion calls the HTTP error callback$/ do
+  @status.should eq 401
+  @body.should match /Unauthorized/
+end
+
+Then /^Nervion calls the network error callback$/ do
+  @network_error_detected.should be_true
+end

data/features/streaming.feature

@@ -0,0 +1,16 @@
+Feature: Callbacks
+
+  Background:
+    Given Nervion is connected to Twitter Streaming API
+
+  Scenario: Calling the status callback
+     When a status update is sent by Twitter
+     Then Nervion calls the status callback with it
+
+  Scenario: Calling the http error callback
+     When an HTTP error occurs
+     Then Nervion calls the HTTP error callback
+
+  Scenario: Calling the network error callback
+     When a network error occurs
+     Then Nervion calls the network error callback

data/features/support/env.rb

@@ -0,0 +1,9 @@
+$: << File.join(File.dirname(__FILE__), '..', '..', 'lib')
+
+require 'nervion'
+require 'eventmachine'
+
+if ENV['COVERAGE']
+  require 'simplecov'
+  SimpleCov.start
+end

data/features/support/streaming_api_double.rb

@@ -0,0 +1,45 @@
+require 'fixtures/responses'
+
+STREAM_FILE_PATH = 'fixtures/stream.txt'
+
+class WorkingStreamingApiDouble < EM::Connection
+  def post_init
+    start_tls
+  end
+
+  def receive_data(data)
+    send_response_ok
+    stream_sample
+  end
+
+  def send_response_ok
+    send_data RESPONSE_200_HEADERS
+  end
+
+  def stream_sample
+    EM::FileStreamer.new(self, STREAM_FILE_PATH, http_chunks: true).callback do
+      close_connection_after_writing
+    end
+  end
+end
+
+class HttpErrorStreamingApiDouble < EM::Connection
+  def post_init
+    start_tls
+  end
+
+  def receive_data(data)
+    send_data RESPONSE_401
+    close_connection_after_writing
+  end
+end
+
+class NetworkErrorStreamingApiDouble < EM::Connection
+  def post_init
+    start_tls
+  end
+
+  def receive_data(data)
+    close_connection
+  end
+end

data/fixtures/responses.rb

@@ -0,0 +1,53 @@
+RESPONSE_200_HEADERS = <<RESPONSE_200_HEADERS
+HTTP/1.1 200 OK\r
+Content-Type: application/json\r
+Transfer-Encoding: chunked\r\n\r
+RESPONSE_200_HEADERS
+
+BODY_200 = <<BODY_200
+{"delete":{"status":{"user_id_str":"482755917","id":182856546533908480,"user_id":482755917,"id_str":"182856546533908480"}}}\r
+BODY_200
+
+RESPONSE_200 = <<RESPONSE_200
+HTTP/1.1 200 OK\r
+Content-Type: application/json\r
+Transfer-Encoding: chunked\r
+\r
+7d\r
+{"delete":{"status":{"user_id_str":"482755917","id":182856546533908480,"user_id":482755917,"id_str":"182856546533908480"}}}\r
+RESPONSE_200
+
+BODY_401 = <<BODY_401
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
+<title>Error 401 Unauthorized</title>
+</head>
+<body>
+<h2>HTTP ERROR: 401</h2>
+<p>Problem accessing '/1/statuses/sample.json'. Reason:
+<pre>Unauthorized</pre>
+</body>
+</html>\r
+BODY_401
+
+RESPONSE_401 = <<RESPONSE_401
+HTTP/1.1 401 Unauthorized\r
+Content-Type: text/html\r
+WWW-Authenticate: Basic realm="Firehose"\r
+Cache-Control: must-revalidate,no-cache,no-store\r
+Content-Length: 1241\r
+Connection: close\r
+\r
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
+<title>Error 401 Unauthorized</title>
+</head>
+<body>
+<h2>HTTP ERROR: 401</h2>
+<p>Problem accessing '/1/statuses/sample.json'. Reason:
+<pre>Unauthorized</pre>
+</body>
+</html>\r
+RESPONSE_401