xh5-tweetstream 1.1.0

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2009 Intridea, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,166 @@
+= TweetStream
+
+TweetStream provides simple Ruby access to Twitter's Streaming API
+(http://apiwiki.twitter.com/Streaming-API-Documentation).
+
+This fork has been modified to support the site_streams API currently in Beta. 
+
+We have also added support for location-based queries to the streaming API. 
+
+== Installation
+    
+To install from Gemcutter:
+
+    gem install tweetstream
+
+== Usage
+
+Using TweetStream is quite simple:
+
+    require 'rubygems'
+    require 'tweetstream'
+    
+    # This will pull a sample of all tweets based on
+    # your Twitter account's Streaming API role.
+    TweetStream::Client.new('username','password').sample do |status|
+      # The status object is a special Hash with 
+      # method access to its keys.
+      puts "#{status.text}"
+    end
+    
+You can also use it to track keywords or follow a given set of
+user ids:
+
+  # Use 'track' to track a list of single-word keywords
+  TweetStream::Client.new('username','password').track('term1', 'term2') do |status|
+    puts "#{status.text}"
+  end
+  
+  # Use 'follow' to follow a group of user ids (integers, not screen names)
+  TweetStream::Client.new('username','password').follow(14252, 53235) do |status|
+    puts "#{status.text}"
+  end
+  
+The methods available to TweetStream::Client will be kept in parity
+with the methods available on the Streaming API wiki page.
+
+== Swappable JSON Parsing
+
+As of version 1.0, TweetStream supports swappable JSON backends for
+parsing the Tweets. These are specified when you initialize the
+client or daemon by passing it in as the last argument:
+
+    # Parse tweets using Yajl-Ruby
+    TweetStream::Client.new('abc','def',:yajl) # ...
+    
+Available options are <tt>:yajl</tt>, <tt>:json_gem</tt> (default),
+<tt>:json_pure</tt>, and <tt>:active_support</tt>.
+
+== Handling Deletes and Rate Limitations
+
+Sometimes the Streaming API will send messages other than statuses.
+Specifically, it does so when a status is deleted or rate limitations
+have caused some tweets not to appear in the stream. To handle these,
+you can use the on_delete and on_limit methods. Example:
+
+    @client = TweetStream::Client.new('user','pass')
+    
+    @client.on_delete do |status_id, user_id|
+      Tweet.delete(status_id)
+    end
+    
+    @client.on_limit do |skip_count|
+      # do something
+    end  
+    
+    @client.track('intridea')
+    
+The on_delete and on_limit methods can also be chained, like so:
+
+    TweetStream::Client.new('user','pass').on_delete{ |status_id, user_id|
+      Tweet.delete(status_id)
+    }.on_limit { |skip_count|
+      # do something
+    }.track('intridea') do |status|
+      # do something with the status like normal
+    end
+    
+You can also provide <tt>:delete</tt> and/or <tt>:limit</tt> 
+options when you make your method call:
+
+    TweetStream::Client.new('user','pass').track('intridea',
+      :delete => Proc.new{ |status_id, user_id| # do something },
+      :limit => Proc.new{ |skip_count| # do something }
+    ) do |status|
+      # do something with the status like normal
+    end
+    
+Twitter recommends honoring deletions as quickly as possible, and
+you would likely be wise to integrate this functionality into your
+application.
+
+== Errors and Reconnecting
+
+TweetStream uses EventMachine to connect to the Twitter Streaming
+API, and attempts to honor Twitter's guidelines in terms of automatic
+reconnection. When Twitter becomes unavailable, the block specified
+by you in <tt>on_error</tt> will be called. Note that this does not
+indicate something is actually wrong, just that Twitter is momentarily
+down. It could be for routine maintenance, etc.
+
+    TweetStream::Client.new('abc','def').on_error do |message|
+      # Log your error message somewhere
+    end.track('term') do |status|
+      # Do things when nothing's wrong
+    end
+
+However, if the maximum number of reconnect attempts has been reached,
+TweetStream will raise a <tt>TweetStream::ReconnectError</tt> with 
+information about the timeout and number of retries attempted.
+
+== Terminating a TweetStream
+
+It is often the case that you will need to change the parameters of your
+track or follow tweet streams. In the case that you need to terminate
+a stream, you may add a second argument to your block that will yield
+the client itself:
+
+    # Stop after collecting 10 statuses
+    @statuses = []
+    TweetStream::Client.new('username','password').sample do |status, client|
+      @statuses << status
+      client.stop if @statuses.size >= 10
+    end
+    
+When <tt>stop</tt> is called, TweetStream will return from the block
+the last successfully yielded status, allowing you to make note of
+it in your application as necessary.
+
+== Daemonizing
+
+It is also possible to create a daemonized script quite easily
+using the TweetStream library:
+
+  # The third argument is an optional process name
+  TweetStream::Daemon.new('username','password', 'tracker').track('term1', 'term2') do |status|
+    # do something in the background
+  end
+  
+If you put the above into a script and run the script with <tt>ruby scriptname.rb</tt>, you will see a list of daemonization commands such
+as start, stop, and run.
+
+== Note on Patches/Pull Requests
+ 
+* Fork the project.
+* Make your feature addition or bug fix.
+* Add tests for it. This is important so I don't break it in a future version unintentionally.
+* Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
+* Send me a pull request. Bonus points for topic branches.
+
+== Contributors
+
+* Michael Bleigh (initial gem)
+
+== Copyright
+
+Copyright (c) 2009 Intridea, Inc. (http://www.intridea.com/). See LICENSE for details.

data/RELEASE_NOTES.rdoc

@@ -0,0 +1,5 @@
+== Version 1.0.0
+
+* Swappable JSON backend support
+* Switches to use EventMachine instead of Yajl for the HTTP Stream
+* Support reconnect and on_error

data/Rakefile

@@ -0,0 +1,60 @@
+require 'rubygems'
+require 'rake'
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "xh5-tweetstream"
+    gem.summary = %Q{TweetStream is a simple wrapper for consuming the Twitter Streaming API.}
+    gem.description = %Q{TweetStream allows you to easily consume the Twitter Streaming API utilizing the YAJL Ruby gem.}
+    gem.email = "eric@xhfive.com"
+    gem.homepage = "http://github.com/erichurst/tweetstream"
+    gem.authors = ["Michael Bleigh"]
+    gem.files = FileList["[A-Z]*", "{lib,spec,examples}/**/*"] - FileList["**/*.log"]
+    gem.add_development_dependency "rspec"
+    gem.add_dependency 'xh5-twitter-stream'
+    gem.add_dependency 'daemons'
+    # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
+  end
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+  puts "Jeweler (or a dependency) not available. Install it with: sudo gem install jeweler"
+end
+
+namespace :release do
+  %w(patch minor major).each do |level|
+    desc "Tag a #{level} version and push it to Gemcutter."
+    task level.to_sym => %w(version:bump:patch release gemcutter:release)
+  end
+end
+
+require 'spec/rake/spectask'
+Spec::Rake::SpecTask.new(:spec) do |spec|
+  spec.libs << 'lib' << 'spec'
+  spec.spec_files = FileList['spec/**/*_spec.rb']
+end
+
+Spec::Rake::SpecTask.new(:rcov) do |spec|
+  spec.libs << 'lib' << 'spec'
+  spec.pattern = 'spec/**/*_spec.rb'
+  spec.rcov = true
+  spec.rcov_opts = %w{--exclude "spec\/*,gems\/*"}
+end
+
+task :spec => :check_dependencies
+
+task :default => :spec
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  if File.exist?('VERSION')
+    version = File.read('VERSION')
+  else
+    version = ""
+  end
+
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "tweetstream #{version}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end

data/VERSION

@@ -0,0 +1 @@
+1.1.0

data/examples/growl_daemon.rb

@@ -0,0 +1,14 @@
+require 'rubygems'
+require 'tweetstream'
+require 'ruby-growl'
+
+if args_start = ARGV.index('--')
+  username, password = ARGV[args_start + 1].split(':')
+  tracks = ARGV[args_start + 2 .. -1] 
+  puts "Starting a GrowlTweet to track: #{tracks.inspect}"
+end
+
+TweetStream::Daemon.new(username,password).track(*tracks) do |status|
+  g = Growl.new 'localhost', 'growltweet', ['tweet']
+  g.notify 'tweet', status.user.screen_name, status.text
+end

data/lib/tweetstream.rb

@@ -0,0 +1,21 @@
+require 'tweetstream/client'
+require 'tweetstream/hash'
+require 'tweetstream/status'
+require 'tweetstream/user'
+require 'tweetstream/daemon'
+
+module TweetStream
+  class Terminated < ::StandardError; end
+  class Error < ::StandardError; end
+  class ConnectionError < TweetStream::Error; end
+  # A ReconnectError is raised when the maximum number of retries has
+  # failed to re-establish a connection.
+  class ReconnectError < StandardError
+    attr_accessor :timeout, :retries
+    def initialize(timeout, retries)
+      self.timeout = timeout
+      self.retries = retries
+      super("Failed to reconnect after #{retries} tries.")
+    end
+  end
+end

data/lib/tweetstream/client.rb

@@ -0,0 +1,318 @@
+require 'uri'
+require 'cgi'
+require 'eventmachine'
+require 'twitter_stream/json_stream'
+require 'json'
+
+module TweetStream
+  # Provides simple access to the Twitter Streaming API (http://apiwiki.twitter.com/Streaming-API-Documentation)
+  # for Ruby scripts that need to create a long connection to
+  # Twitter for tracking and other purposes.
+  #
+  # Basic usage of the library is to call one of the provided
+  # methods and provide a block that will perform actions on
+  # a yielded TweetStream::Status. For example:
+  #
+  #     TweetStream::Client.new('user','pass').track('fail') do |status|
+  #       puts "[#{status.user.screen_name}] #{status.text}"
+  #     end
+  #
+  # For information about a daemonized TweetStream client,
+  # view the TweetStream::Daemon class.
+  class Client
+    attr_accessor :consumer_key, :consumer_secret, :access_key, :access_secret
+    attr_reader :parser
+
+    # Set the JSON Parser for this client. Acceptable options are:
+    #
+    # <tt>:json_gem</tt>:: Parse using the JSON gem.
+    # <tt>:json_pure</tt>:: Parse using the pure-ruby implementation of the JSON gem.
+    # <tt>:active_support</tt>:: Parse using ActiveSupport::JSON.decode
+    # <tt>:yajl</tt>:: Parse using <tt>yajl-ruby</tt>.
+    #
+    # You may also pass a class that will return a hash with symbolized
+    # keys when <tt>YourClass.parse</tt> is called with a JSON string.
+    def parser=(parser)
+      @parser = parser_from(parser)
+    end
+    
+    # Create a new client with the Twitter credentials
+    # of the account you want to be using its API quota.
+    # You may also set the JSON parsing library as specified
+    # in the #parser= setter.
+    def initialize(consumer_key, consumer_secret, access_key, access_secret, parser = :json_gem)
+      self.consumer_key = consumer_key
+      self.consumer_secret = consumer_secret
+      self.access_key = access_key
+      self.access_secret = access_secret
+      self.parser = parser
+    end
+   
+    # Returns all public statuses. The Firehose is not a generally
+    # available resource. Few applications require this level of access. 
+    # Creative use of a combination of other resources and various access 
+    # levels can satisfy nearly every application use case. 
+    def firehose(query_parameters = {}, &block)
+      start('statuses/firehose', query_parameters, &block)
+    end
+    
+    # Returns all retweets. The retweet stream is not a generally available 
+    # resource. Few applications require this level of access. Creative
+    # use of a combination of other resources and various access levels
+    # can satisfy nearly every application use case. As of 9/11/2009,
+    # the site-wide retweet feature has not yet launched,
+    # so there are currently few, if any, retweets on this stream.
+    def retweet(query_parameters = {}, &block)
+      start('statuses/retweet', query_parameters, &block)
+    end
+    
+    # Returns a random sample of all public statuses. The default access level 
+    # provides a small proportion of the Firehose. The "Gardenhose" access
+    # level provides a proportion more suitable for data mining and
+    # research applications that desire a larger proportion to be statistically
+    # significant sample.
+    def sample(query_parameters = {}, &block)
+      start('statuses/sample', query_parameters, &block)
+    end
+
+    # Specify keywords to track. Queries are subject to Track Limitations, 
+    # described in Track Limiting and subject to access roles, described in 
+    # the statuses/filter method. Track keywords are case-insensitive logical 
+    # ORs. Terms are exact-matched, and also exact-matched ignoring
+    # punctuation. Phrases, keywords with spaces, are not supported. 
+    # Keywords containing punctuation will only exact match tokens.
+    # Query parameters may be passed as the last argument.
+    def track(*keywords, &block)
+      query_params = keywords.pop if keywords.last.is_a?(::Hash)
+      query_params ||= {}
+      filter(query_params.merge(:track => keywords), &block)
+    end
+
+    # Returns public statuses from or in reply to a set of users. Mentions 
+    # ("Hello @user!") and implicit replies ("@user Hello!" created without 
+    # pressing the reply "swoosh") are not matched. Requires integer user
+    # IDs, not screen names. Query parameters may be passed as the last argument.
+    def follow(*user_ids, &block)
+      query_params = user_ids.pop if user_ids.last.is_a?(::Hash)
+      query_params ||= {}
+      filter(query_params.merge(:follow => user_ids), &block)
+    end
+
+    def locations(coords, &block)
+      filter(query_params.merge(:locations => coords), &block)
+    end
+    
+    # Make a call to the statuses/filter method of the Streaming API,
+    # you may provide <tt>:follow</tt>, <tt>:track</tt> or both as options
+    # to follow the tweets of specified users or track keywords. This
+    # method is provided separately for cases when it would conserve the
+    # number of HTTP connections to combine track and follow.
+    def filter(query_params = {}, &block)
+      [:follow, :track, :locations].each do |param|
+        if query_params[param].is_a?(Array)
+          query_params[param] = query_params[param].collect{|q| q.to_s}.join(',')
+        elsif query_params[param]
+          query_params[param] = query_params[param].to_s
+        end
+      end
+      start('statuses/filter', query_params.merge(:method => :post), &block)
+    end
+    
+    def site_follow(*user_ids, &block)
+      query_params ||= {:follow => user_ids}
+      
+      if query_params[:follow].is_a?(Array)
+        query_params[:follow] = query_params[:follow].collect{|q| q.to_s}.join(',')
+      elsif query_params[:follow]
+        query_params[:follow] = query_params[:follow].to_s
+      end
+      query_params[:site_streams] = query_params[:follow]
+      query_params[:track] = ["foo"]
+      
+      start('site', query_params.merge(:method => :post, :host => 'betastream.twitter.com', :version => '2b'), &block)
+    end
+    
+    # Set a Proc to be run when a deletion notice is received
+    # from the Twitter stream. For example:
+    #
+    #     @client = TweetStream::Client.new('user','pass')
+    #     @client.on_delete do |status_id, user_id|
+    #       Tweet.delete(status_id)
+    #     end
+    #
+    # Block must take two arguments: the status id and the user id.
+    # If no block is given, it will return the currently set 
+    # deletion proc. When a block is given, the TweetStream::Client
+    # object is returned to allow for chaining.
+    def on_delete(&block)
+      if block_given?
+        @on_delete = block
+        self
+      else
+        @on_delete
+      end
+    end
+    
+    # Set a Proc to be run when a rate limit notice is received
+    # from the Twitter stream. For example:
+    #
+    #     @client = TweetStream::Client.new('user','pass')
+    #     @client.on_limit do |discarded_count|
+    #       # Make note of discarded count
+    #     end
+    #
+    # Block must take one argument: the number of discarded tweets.
+    # If no block is given, it will return the currently set 
+    # limit proc. When a block is given, the TweetStream::Client
+    # object is returned to allow for chaining.
+    def on_limit(&block)
+      if block_given?
+        @on_limit = block
+        self
+      else
+        @on_limit
+      end
+    end
+    
+    # Set a Proc to be run when an HTTP error is encountered in the
+    # processing of the stream. Note that TweetStream will automatically
+    # try to reconnect, this is for reference only. Don't panic!
+    #
+    #     @client = TweetStream::Client.new('user','pass')
+    #     @client.on_error do |message|
+    #       # Make note of error message
+    #     end
+    #
+    # Block must take one argument: the error message.
+    # If no block is given, it will return the currently set 
+    # error proc. When a block is given, the TweetStream::Client
+    # object is returned to allow for chaining.
+    def on_error(&block)
+      if block_given?
+        @on_error = block
+        self
+      else
+        @on_error
+      end
+    end
+    
+    def start(path, query_parameters = {}, &block) #:nodoc:
+
+      host = query_parameters.delete(:host) || 'stream.twitter.com'
+      version = query_parameters.delete(:version) || '1'
+
+      method = query_parameters.delete(:method) || :get
+      delete_proc = query_parameters.delete(:delete) || self.on_delete
+      limit_proc = query_parameters.delete(:limit) || self.on_limit
+      error_proc = query_parameters.delete(:error) || self.on_error
+      
+      uri = method == :get ? build_uri(path, version, query_parameters) : build_uri(path, version)
+      
+      oauth = {
+        :consumer_key => self.consumer_key,
+        :consumer_secret => self.consumer_secret,
+        :access_key => self.access_key,
+        :access_secret => self.access_secret
+      }
+      
+      EventMachine::run {
+        @stream = TwitterStream::JSONStream.connect(
+          :path => uri,
+          :host => host,
+          :oauth => oauth,
+          :filters => query_parameters[:track],
+          :follow => query_parameters[:follow],
+          :locations => query_parameters[:locations],
+          :site_streams => query_parameters[:site_streams],
+          :method => method.to_s.upcase,
+          :content => (method == :post ? build_post_body(query_parameters) : ''),
+          :user_agent => 'TweetStream'
+        )
+        
+        @stream.each_item do |item|
+          raw_hash = @parser.decode(item)
+          
+          unless raw_hash.is_a?(::Hash)
+            error_proc.call("Unexpected JSON object in stream: #{item}")
+            next
+          end
+          
+          hash = TweetStream::Hash.new(raw_hash) # @parser.parse(item)
+          
+          if hash[:delete] && hash[:delete][:status]
+            delete_proc.call(hash[:delete][:status][:id], hash[:delete][:status][:user_id]) if delete_proc.is_a?(Proc)
+          elsif hash[:limit] && hash[:limit][:track]
+            limit_proc.call(hash[:limit][:track]) if limit_proc.is_a?(Proc)
+          elsif hash[:text] && hash[:user]
+            @last_status = TweetStream::Status.new(hash)
+            
+            # Give the block the option to receive either one
+            # or two arguments, depending on its arity.
+            case block.arity
+              when 1
+                yield @last_status
+              when 2
+                yield @last_status, self
+            end
+          elsif hash[:for_user]
+            
+            @last_status = TweetStream::Status.new(hash[:messages] || hash[:message])
+            
+            # Give the block the option to receive either one
+            # or two arguments, depending on its arity.
+            case block.arity
+              when 2
+                yield @last_status, hash[:for_user]
+              when 3
+                yield @last_status, hash[:for_user], self
+            end
+          end
+        end
+        
+        @stream.on_error do |message|
+          error_proc.call(message) if error_proc.is_a?(Proc)
+        end
+        
+        @stream.on_max_reconnects do |timeout, retries|
+          raise TweetStream::ReconnectError.new(timeout, retries)
+        end
+      }
+    end
+    
+    # Terminate the currently running TweetStream.
+    def stop
+      EventMachine.stop_event_loop
+      @last_status
+    end
+ 
+    protected
+
+    def parser_from(parser)
+      case parser
+        when Class
+          parser
+        when Symbol
+          require "tweetstream/parsers/#{parser.to_s}"
+          eval("TweetStream::Parsers::#{parser.to_s.split('_').map{|s| s.capitalize}.join('')}")
+      end
+    end
+    
+    def build_uri(path, version = '1', query_parameters = {}) #:nodoc:
+      URI.parse("/#{version}/#{path}.json#{build_query_parameters(query_parameters)}")
+    end
+
+    def build_query_parameters(query)
+      query.size > 0 ? "?#{build_post_body(query)}" : ''
+    end
+    
+    def build_post_body(query) #:nodoc:
+      return '' unless query && query.is_a?(::Hash) && query.size > 0
+      pairs = []
+      
+      query.each_pair do |k,v|
+        pairs << "#{k.to_s}=#{CGI.escape(v.to_s)}"
+      end
+      pairs.join('&')
+    end
+  end
+end

data/lib/tweetstream/daemon.rb

@@ -0,0 +1,39 @@
+require 'daemons'
+
+# A daemonized TweetStream client that will allow you to
+# create backgroundable scripts for application specific
+# processes. For instance, if you create a script called
+# <tt>tracker.rb</tt> and fill it with this:
+#
+#     require 'rubygems'
+#     require 'tweetstream'
+#
+#     TweetStream::Daemon.new('user','pass', 'tracker').track('intridea') do |status|
+#       # do something here
+#     end
+#
+# And then you call this from the shell:
+#
+#     ruby tracker.rb start
+#
+# A daemon process will spawn that will automatically
+# run the code in the passed block whenever a new tweet
+# matching your search term ('intridea' in this case)
+# is posted.
+#
+class TweetStream::Daemon < TweetStream::Client
+  # Initialize a Daemon with the credentials of the
+  # Twitter account you wish to use. The daemon has
+  # an optional process name for use when querying
+  # running processes.
+  def initialize(consumer_key, consumer_secret, access_key, access_secret, app_name=nil, parser=:json_gem)
+    @app_name = app_name
+    super(consumer_key, consumer_secret, access_key, access_secret, parser)
+  end
+  
+  def start(path, query_parameters = {}, &block) #:nodoc:
+    Daemons.run_proc(@app_name || 'tweetstream', :multiple => true) do
+      super(path, query_parameters, &block)
+    end
+  end
+end

data/lib/tweetstream/hash.rb

@@ -0,0 +1,17 @@
+class TweetStream::Hash < ::Hash #:nodoc: all
+  def initialize(other_hash = {})
+    other_hash.keys.each do |key|
+      value = other_hash[key]
+      value = TweetStream::Hash.new(value) if value.is_a?(::Hash)
+      self[key.to_sym] = value
+    end
+  end
+    
+  def method_missing(method_name, *args)
+    if key?(method_name.to_sym)
+      self[method_name.to_sym]
+    else
+      super
+    end
+  end
+end

data/lib/tweetstream/parsers/active_support.rb

@@ -0,0 +1,11 @@
+require 'active_support/json' unless defined?(::ActiveSupport::JSON)
+
+module TweetStream
+  module Parsers
+    class ActiveSupport
+      def self.decode(string)
+        ::ActiveSupport::JSON.decode(string)
+      end
+    end
+  end
+end

data/lib/tweetstream/parsers/json_gem.rb

@@ -0,0 +1,11 @@
+require 'json' unless defined?(JSON)
+
+module TweetStream
+  module Parsers
+    class JsonGem
+      def self.decode(string)
+        ::JSON.parse(string)
+      end
+    end
+  end
+end

data/lib/tweetstream/parsers/json_pure.rb

@@ -0,0 +1,11 @@
+require 'json/pure' unless defined?(::JSON)
+
+module TweetStream
+  module Parsers
+    class JsonPure
+      def self.decode(string)
+        ::JSON.parse(string)
+      end
+    end
+  end
+end

data/lib/tweetstream/parsers/yajl.rb

@@ -0,0 +1,11 @@
+require 'yajl' unless defined?(Yajl)
+
+module TweetStream
+  module Parsers
+    class Yajl
+      def self.decode(string)
+        ::Yajl::Parser.new(:symbolize_keys => true).parse(string)
+      end
+    end
+  end
+end

data/lib/tweetstream/status.rb

@@ -0,0 +1,12 @@
+# A simple Hash wrapper that gives you method-based
+# access to the properties of a Twitter status.
+class TweetStream::Status < TweetStream::Hash
+  def initialize(hash)
+    super
+    self[:user] = TweetStream::User.new(self[:user]) if self[:user]
+  end
+  
+  def id
+    self[:id] || super
+  end
+end

data/lib/tweetstream/user.rb

@@ -0,0 +1,7 @@
+# A simple Hash wrapper that gives you method-based
+# access to user properties returned by the streamer.
+class TweetStream::User < TweetStream::Hash
+  def id
+    self[:id] || super
+  end
+end

data/spec/data/statuses.json

@@ -0,0 +1 @@
+{"favorited":false,"text":"listening to Where U Headed by Universal Playaz. http://iLike.com/s/9zpOZ #musicmonday something for the ladies","in_reply_to_user_id":null,"in_reply_to_screen_name":null,"source":"<a href=\"http://www.iLike.com\" rel=\"nofollow\">iLike</a>","truncated":false,"created_at":"Tue Sep 22 01:29:13 +0000 2009","user":{"statuses_count":378,"favourites_count":1,"profile_text_color":"666666","location":"Atlanta, Ga","profile_background_image_url":"http://a3.twimg.com/profile_background_images/36516125/Universal_Playaz.jpg","profile_link_color":"2FC2EF","description":"Paper Chaser","following":null,"verified":false,"notifications":null,"profile_sidebar_fill_color":"252429","profile_image_url":"http://a1.twimg.com/profile_images/413331530/DIESELSTATScopy_normal.jpg","url":"http://www.myspace.com/DieselDtheg","profile_sidebar_border_color":"181A1E","screen_name":"DieselD2143","profile_background_tile":true,"followers_count":75,"protected":false,"time_zone":"Eastern Time (US & Canada)","created_at":"Thu Jun 18 15:56:32 +0000 2009","name":"Diesel D","friends_count":119,"profile_background_color":"1A1B1F","id":48392351,"utc_offset":-18000},"in_reply_to_status_id":null,"id":4161231023} {"favorited":false,"text":"David Bowie and Nine Inch Nails perform \"Hurt\" http://bit.ly/AOaWG #musicmonday #nineinchnails #nin","in_reply_to_user_id":null,"in_reply_to_screen_name":null,"source":"web","truncated":false,"created_at":"Tue Sep 22 01:29:16 +0000 2009","user":{"statuses_count":668,"favourites_count":25,"profile_text_color":"445d85","location":"S\u00e3o Paulo, Brazil","profile_background_image_url":"http://a3.twimg.com/profile_background_images/38174991/GeorgeRomero-oil-400.jpg","profile_link_color":"555757","description":"You think I ain't worth a dollar, but I feel like a millionaire","following":null,"verified":false,"notifications":null,"profile_sidebar_fill_color":"a3a7ad","profile_image_url":"http://a1.twimg.com/profile_images/96034368/n1076431955_30001395_7912_normal.jpg","url":null,"profile_sidebar_border_color":"c7d1ed","screen_name":"RenatonMiranda","profile_background_tile":true,"followers_count":111,"protected":false,"time_zone":"Santiago","created_at":"Sat Mar 14 15:03:59 +0000 2009","name":"Renato Miranda","friends_count":143,"profile_background_color":"287356","id":24379310,"utc_offset":-14400},"in_reply_to_status_id":null,"id":4161232008} {"favorited":false,"text":"#musicmonday ,time to download some songs today!! :)","in_reply_to_user_id":null,"in_reply_to_screen_name":null,"source":"web","truncated":false,"created_at":"Tue Sep 22 01:29:19 +0000 2009","user":{"statuses_count":188,"favourites_count":0,"profile_text_color":"3D1957","location":"under the water","profile_background_image_url":"http://s.twimg.com/a/1253562286/images/themes/theme10/bg.gif","profile_link_color":"FF0000","description":"ask me ","following":null,"verified":false,"notifications":null,"profile_sidebar_fill_color":"7AC3EE","profile_image_url":"http://a1.twimg.com/profile_images/421281292/twit_pic_normal.jpg","url":"http://www.exploretalent.com/contest_video.php?talentnum=2053105&cm_id=3398","profile_sidebar_border_color":"65B0DA","screen_name":"julieanne11343","profile_background_tile":true,"followers_count":9,"protected":false,"time_zone":"Pacific Time (US & Canada)","created_at":"Mon Jul 20 21:08:22 +0000 2009","name":"Julieanne","friends_count":17,"profile_background_color":"642D8B","id":58591151,"utc_offset":-28800},"in_reply_to_status_id":null,"id":4161233120} {"text":"#Musicmonday \"Dont be tardy f0r the party\"","truncated":false,"source":"<a href=\"http://twitterhelp.blogspot.com/2008/05/twitter-via-mobile-web-mtwittercom.html\" rel=\"nofollow\">mobile web</a>","in_reply_to_status_id":null,"favorited":false,"created_at":"Tue Sep 22 01:29:19 +0000 2009","user":{"verified":false,"notifications":null,"profile_sidebar_fill_color":"e0ff92","location":"Dope Girl Island","profile_sidebar_border_color":"87bc44","description":"","following":null,"profile_background_tile":false,"followers_count":29,"profile_image_url":"http://a3.twimg.com/profile_images/217487577/badbad_normal.jpg","time_zone":"Eastern Time (US & Canada)","url":null,"friends_count":65,"profile_background_color":"9ae4e8","screen_name":"SwagGirlOnDeck","protected":false,"statuses_count":847,"favourites_count":0,"created_at":"Fri May 01 16:59:15 +0000 2009","profile_text_color":"000000","name":"Mariah Reta","id":36987168,"profile_background_image_url":"http://s.twimg.com/a/1253301564/images/themes/theme1/bg.png","utc_offset":-18000,"profile_link_color":"0000ff"},"in_reply_to_user_id":null,"id":4161233317,"in_reply_to_screen_name":null} 

data/spec/spec.opts

@@ -0,0 +1,2 @@
+--colour
+--format progress

data/spec/spec_helper.rb

@@ -0,0 +1,25 @@
+$LOAD_PATH.unshift(File.dirname(__FILE__))
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+
+require 'rubygems'
+require 'tweetstream'
+require 'spec'
+require 'spec/autorun'
+require 'yajl'
+require 'json'
+
+def sample_tweets
+  if @tweets
+    @tweets
+  else
+    @tweets = []
+    Yajl::Parser.parse(File.open(File.dirname(__FILE__) + '/data/statuses.json', 'r'), :symbolize_keys => true) do |hash|
+      @tweets << hash
+    end
+    @tweets
+  end
+end
+
+Spec::Runner.configure do |config|
+  
+end

data/spec/tweetstream/client_spec.rb

@@ -0,0 +1,233 @@
+require File.dirname(__FILE__) + '/../spec_helper'
+
+describe TweetStream::Client do
+  #it 'should set the username and password from the initializers' do
+    #@client = TweetStream::Client.new('abc','def','hij','klm')
+    #@client.username.should == 'abc'
+    #@client.password.should == 'def'
+  #end
+
+  describe '#build_uri' do
+    before do
+      @client = TweetStream::Client.new('abc','def','hij','klm')
+    end
+
+    it 'should return a URI' do
+      @client.send(:build_uri, '').is_a?(URI).should be_true
+    end
+
+    it 'should have the specified path with the version prefix and a json extension' do
+      @client.send(:build_uri, 'awesome').path.should == '/1/awesome.json'
+    end
+
+    it 'should add on a query string if such parameters are specified' do
+      @client.send(:build_uri, 'awesome', 1, :q => 'abc').query.should == 'q=abc'
+    end
+  end
+
+  describe '#build_post_body' do
+    before do
+      @client = TweetStream::Client.new('abc','def','hij','klm')
+    end
+  
+    it 'should return a blank string if passed a nil value' do
+      @client.send(:build_post_body, nil).should == ''
+    end
+
+    it 'should return a blank string if passed an empty hash' do
+      @client.send(:build_post_body, {}).should == ''
+    end
+
+    it 'should add a query parameter for a key' do
+      @client.send(:build_post_body, {:query => 'abc'}).should == 'query=abc'
+    end
+
+    it 'should escape characters in the value' do
+      @client.send(:build_post_body, {:query => 'awesome guy'}).should == 'query=awesome+guy'
+    end
+
+    it 'should join multiple pairs together' do
+      ['a=b&c=d','c=d&a=b'].include?(@client.send(:build_post_body, {:a => 'b', :c => 'd'})).should be_true
+    end
+  end
+
+  describe '#start' do
+    before do
+      @stream = stub("TwitterStream::JSONStream", 
+        :connect => true,
+        :unbind => true, 
+        :each_item => true, 
+        :on_error => true,
+        :on_max_reconnects => true,
+        :connection_completed => true
+      )
+      EM.stub!(:run).and_yield
+      TwitterStream::JSONStream.stub!(:connect).and_return(@stream)
+      @client = TweetStream::Client.new('abc','def','hij','klm')
+    end
+    
+    it 'should try to connect via a JSON stream' do
+      #TwitterStream::JSONStream.should_receive(:connect).with(
+      #  :oauth => {:access_key=>"hij", :access_secret=>"klm", :consumer_key=>"abc", :consumer_secret=>"def"},
+      #  :content => 'track=monday',
+      #  :path => URI.parse('/1/statuses/filter.json'),
+      #  :method => 'POST',
+      #  :user_agent => 'TweetStream'
+      #).and_return(@stream)
+      
+      @client.track('monday')
+    end
+    
+    describe '#each_item' do
+      it 'should call the appropriate parser' do
+        @client = TweetStream::Client.new('abc','def','hij','klm',:active_support)
+        TweetStream::Parsers::ActiveSupport.should_receive(:decode).and_return({})
+        @stream.should_receive(:each_item).and_yield(sample_tweets[0].to_json)
+        @client.track('abc','def')
+      end
+      
+      it 'should yield a TweetStream::Status' do
+        @stream.should_receive(:each_item).and_yield(sample_tweets[0].to_json)
+        @client.track('abc'){|s| s.should be_kind_of(TweetStream::Status)}
+      end
+      
+      it 'should also yield the client if a block with arity 2 is given' do
+        @stream.should_receive(:each_item).and_yield(sample_tweets[0].to_json)
+        @client.track('abc'){|s,c| c.should == @client}
+      end
+      
+      it 'should include the proper values' do
+        tweet = sample_tweets[0]
+        tweet[:id] = 123
+        tweet[:user][:screen_name] = 'monkey'
+        tweet[:text] = "Oo oo aa aa"
+        @stream.should_receive(:each_item).and_yield(tweet.to_json)
+        @client.track('abc') do |s| 
+          s[:id].should == 123
+          s.user.screen_name.should == 'monkey'
+          s.text.should == 'Oo oo aa aa'
+        end
+      end
+      
+      it 'should call the on_delete if specified' do
+        delete = '{ "delete": { "status": { "id": 1234, "user_id": 3 } } }'
+        @stream.should_receive(:each_item).and_yield(delete)
+        @client.on_delete do |id, user_id| 
+          id.should == 1234
+          user_id.should == 3
+        end.track('abc')
+      end
+      
+      it 'should call the on_limit if specified' do
+        limit = '{ "limit": { "track": 1234 } }'
+        @stream.should_receive(:each_item).and_yield(limit)
+        @client.on_limit do |track|
+          track.should == 1234
+        end.track('abc')
+      end
+      
+      it 'should call on_error if a non-hash response is received' do
+        @stream.should_receive(:each_item).and_yield('["favorited"]')
+        @client.on_error do |message|
+          message.should == 'Unexpected JSON object in stream: ["favorited"]'
+        end.track('abc')
+      end
+    end
+    
+    describe '#on_error' do
+      it 'should pass the message on to the error block' do
+        @stream.should_receive(:on_error).and_yield('Uh oh')
+        @client.on_error do |m|
+          m.should == 'Uh oh'
+        end.track('abc')
+      end
+    end
+    
+    describe '#on_max_reconnects' do
+      it 'should raise a ReconnectError' do
+        @stream.should_receive(:on_max_reconnects).and_yield(30, 20)
+        lambda{@client.track('abc')}.should raise_error(TweetStream::ReconnectError) do |e|
+          e.timeout.should == 30
+          e.retries.should == 20
+        end
+      end
+    end
+  end
+  
+  describe ' API methods' do
+    before do
+      @client = TweetStream::Client.new('abc','def','hij','klm')
+    end
+    
+    %w(firehose retweet sample).each do |method|
+      it "##{method} should make a call to start with \"statuses/#{method}\"" do
+        @client.should_receive(:start).once.with('statuses/' + method, {})
+        @client.send(method)
+      end
+    end
+    
+    it '#track should make a call to start with "statuses/filter" and a track query parameter' do
+      @client.should_receive(:start).once.with('statuses/filter', :track => 'test', :method => :post)
+      @client.track('test')
+    end
+    
+    it '#track should comma-join multiple arguments' do
+      @client.should_receive(:start).once.with('statuses/filter', :track => 'foo,bar,baz', :method => :post)
+      @client.track('foo', 'bar', 'baz')
+    end
+    
+    it '#follow should make a call to start with "statuses/filter" and a follow query parameter' do
+      @client.should_receive(:start).once.with('statuses/filter', :follow => '123', :method => :post)
+      @client.follow(123)
+    end
+    
+    it '#follow should comma-join multiple arguments' do
+      @client.should_receive(:start).once.with('statuses/filter', :follow => '123,456', :method => :post)
+      @client.follow(123, 456)
+    end
+    
+    it '#filter should make a call to "statuses/filter" with the query params provided' do
+      @client.should_receive(:start).once.with('statuses/filter', :follow => '123', :method => :post)
+      @client.filter(:follow => 123)
+    end
+  end
+  
+  %w(on_delete on_limit).each do |proc_setter|
+    describe "##{proc_setter}" do
+      before do
+        @client = TweetStream::Client.new('abc','def','hij','klm')
+      end
+      
+      it 'should set when a block is given' do
+        proc = Proc.new{|a,b| puts a }
+        @client.send(proc_setter, &proc)
+        @client.send(proc_setter).should == proc
+      end
+    end
+  end
+
+  describe '#track' do
+    before do
+      @client = TweetStream::Client.new('abc','def','hij','klm')
+    end
+
+    it 'should call #start with "statuses/filter" and the provided queries' do
+      @client.should_receive(:start).once.with('statuses/filter', :track => 'rock', :method => :post)
+      @client.track('rock')
+    end
+  end
+
+  describe 'instance .stop' do
+    it 'should call EventMachine::stop_event_loop' do
+      EventMachine.should_receive :stop_event_loop
+      TweetStream::Client.new('abc','def','hij','klm').stop.should be_nil
+    end
+    
+    it 'should return the last status yielded' do
+      EventMachine.should_receive :stop_event_loop
+      client = TweetStream::Client.new('abc','def','hij','klm')
+      client.send(:instance_variable_set, :@last_status, {})
+      client.stop.should == {}
+    end
+  end
+end

data/spec/tweetstream/hash_spec.rb

@@ -0,0 +1,19 @@
+require File.dirname(__FILE__) + '/../spec_helper'
+
+describe TweetStream::Hash do
+  it 'should be initialized by passing in an existing hash' do
+    TweetStream::Hash.new(:abc => 123)[:abc].should == 123
+  end
+  
+  it 'should symbolize incoming keys' do
+    TweetStream::Hash.new('abc' => 123)[:abc].should == 123
+  end
+  
+  it 'should allow access via method calls' do
+    TweetStream::Hash.new(:abc => 123).abc.should == 123
+  end
+  
+  it 'should still throw NoMethod for non-existent keys' do
+    lambda{TweetStream::Hash.new({}).akabi}.should raise_error(NoMethodError)
+  end
+end

data/spec/tweetstream/parser_spec.rb

@@ -0,0 +1,39 @@
+require File.dirname(__FILE__) + '/../spec_helper'
+
+describe 'TweetStream JSON Parsers' do
+  it 'should default to the JSON Gem' do
+    TweetStream::Client.new('abc','def','hij','klm').parser.should == TweetStream::Parsers::JsonGem
+  end
+  
+  [:json_gem, :yajl, :active_support, :json_pure].each do |engine|
+    describe "#{engine} parsing" do
+      before do
+        @client = TweetStream::Client.new('abc','def','hij','klm',engine)
+        @class_name = "TweetStream::Parsers::#{engine.to_s.split('_').map{|s| s.capitalize}.join('')}"
+      end
+      
+      it 'should set the parser to the appropriate class' do
+        @client.parser.to_s == @class_name
+      end
+      
+      it 'should be settable via client.parser=' do
+        @client.parser = nil
+        @client.parser.should be_nil
+        @client.parser = engine
+        @client.parser.to_s.should == @class_name
+      end
+    end
+  end
+  
+  class FakeParser
+    def self.decode(text)
+      {}
+    end
+  end
+  
+  it 'should be settable to a class' do
+    @client = TweetStream::Client.new('abc','def','hij','klm')
+    @client.parser = FakeParser
+    @client.parser.should == FakeParser
+  end
+end

data/spec/tweetstream/status_spec.rb

@@ -0,0 +1,15 @@
+require File.dirname(__FILE__) + '/../spec_helper'
+
+describe TweetStream::Status do
+  it 'should modify the :user key into a TweetStream::User object' do
+    @status = TweetStream::Status.new(:user => {:screen_name => 'bob'})
+    @status.user.is_a?(TweetStream::User).should be_true
+    @status.user.screen_name.should == 'bob'
+  end
+  
+  it 'should override the #id method for itself and the user' do
+    @status = TweetStream::Status.new(:id => 123, :user => {:id => 345})
+    @status.id.should == 123
+    @status.user.id.should == 345
+  end
+end

data/spec/tweetstream_spec.rb

@@ -0,0 +1,5 @@
+require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
+
+describe TweetStream do
+
+end

metadata

@@ -0,0 +1,127 @@
+--- !ruby/object:Gem::Specification 
+name: xh5-tweetstream
+version: !ruby/object:Gem::Version 
+  prerelease: false
+  segments: 
+  - 1
+  - 1
+  - 0
+  version: 1.1.0
+platform: ruby
+authors: 
+- Michael Bleigh
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2010-10-12 00:00:00 -05:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: rspec
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :development
+  version_requirements: *id001
+- !ruby/object:Gem::Dependency 
+  name: xh5-twitter-stream
+  prerelease: false
+  requirement: &id002 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id002
+- !ruby/object:Gem::Dependency 
+  name: daemons
+  prerelease: false
+  requirement: &id003 !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id003
+description: TweetStream allows you to easily consume the Twitter Streaming API utilizing the YAJL Ruby gem.
+email: eric@xhfive.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: 
+- LICENSE
+- README.rdoc
+files: 
+- LICENSE
+- README.rdoc
+- RELEASE_NOTES.rdoc
+- Rakefile
+- VERSION
+- examples/growl_daemon.rb
+- lib/tweetstream.rb
+- lib/tweetstream/client.rb
+- lib/tweetstream/daemon.rb
+- lib/tweetstream/hash.rb
+- lib/tweetstream/parsers/active_support.rb
+- lib/tweetstream/parsers/json_gem.rb
+- lib/tweetstream/parsers/json_pure.rb
+- lib/tweetstream/parsers/yajl.rb
+- lib/tweetstream/status.rb
+- lib/tweetstream/user.rb
+- spec/data/statuses.json
+- spec/spec.opts
+- spec/spec_helper.rb
+- spec/tweetstream/client_spec.rb
+- spec/tweetstream/hash_spec.rb
+- spec/tweetstream/parser_spec.rb
+- spec/tweetstream/status_spec.rb
+- spec/tweetstream_spec.rb
+has_rdoc: true
+homepage: http://github.com/erichurst/tweetstream
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.6
+signing_key: 
+specification_version: 3
+summary: TweetStream is a simple wrapper for consuming the Twitter Streaming API.
+test_files: 
+- spec/spec_helper.rb
+- spec/tweetstream/client_spec.rb
+- spec/tweetstream/hash_spec.rb
+- spec/tweetstream/parser_spec.rb
+- spec/tweetstream/status_spec.rb
+- spec/tweetstream_spec.rb
+- examples/growl_daemon.rb