jls-tweetstream 2.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,6 @@
1
+ --markup markdown
2
+ -
3
+ CHANGELOG.md
4
+ CONTRIBUTING.md
5
+ LICENSE.md
6
+ README.md
@@ -0,0 +1,76 @@
1
+ Version 2.4.0
2
+ =============
3
+ * Revert "use extract_options! from the Twitter gem"
4
+ * Reorganize development and test dependencies
5
+
6
+ Version 2.3.0
7
+ =============
8
+ * Added support for Site Stream friends list
9
+ * Update paths for API 1.1
10
+ * Added stall warning handling
11
+
12
+ Version 2.2.0
13
+ =============
14
+ * Change method to request_method
15
+
16
+ Version 2.1.0
17
+ =============
18
+ * Disable identity map to reduce memory usage
19
+ * Added options support to UserStreams
20
+
21
+ Version 2.0.1
22
+ =============
23
+ * Fixed Twitter gem objects
24
+ * Added on_unauthorized callback method (koenpunt)
25
+
26
+ Version 2.0.0
27
+ =============
28
+ * Added Site Stream support
29
+ * Switched to [em-twitter](https://github.com/spagalloco/em-twitter) for underlying streaming lib
30
+ * Switched to Twitter gem objects instead of custom hashes, see [47e5cd3d21a9562b3d959bc231009af460b37567](https://github.com/intridea/tweetstream/commit/47e5cd3d21a9562b3d959bc231009af460b37567) for details (sferik)
31
+ * Made OAuth the default authentication method
32
+ * Removed on_interval callback method
33
+ * Removed parser configuration option
34
+
35
+ Version 1.1.5
36
+ =============
37
+ * Added support for the scrub_geo response (augustj)
38
+ * Update multi_json and twitter-stream version dependencies
39
+
40
+ Version 1.1.4
41
+ =============
42
+ * Added Client#connect to start streaming inside an EM reactor (pelle)
43
+ * Added shutdown_stream to cleanly stop the stream (lud)
44
+ * Loosened multi_json dependency for Rails 3.2 compatibiltiy
45
+
46
+ Version 1.1.3
47
+ =============
48
+ * Added on_reconnect callback method
49
+
50
+ Version 1.1.2
51
+ =============
52
+ * Added support for statuses/links
53
+ * Client now checks that specified json_parser can be loaded during initialization
54
+
55
+ Version 1.1.1
56
+ =============
57
+ * Fix for 1.8.6 compatibility
58
+
59
+ Version 1.1.0
60
+ =============
61
+ * OAuth authentication
62
+ * User Stream support
63
+ * Removed swappable JSON backend support for MultiJson
64
+ * Added EventMachine epoll and kqueue support
65
+ * Added on_interval callback
66
+ * Added on_inited callback
67
+
68
+ Version 1.0.5
69
+ =============
70
+ * Force SSL to comply with
71
+
72
+ Version 1.0.0
73
+ =============
74
+ * Swappable JSON backend support
75
+ * Switches to use EventMachine instead of Yajl for the HTTP Stream
76
+ * Support reconnect and on_error
@@ -0,0 +1,9 @@
1
+ ## Contributing
2
+ * Fork the project.
3
+ * Make your feature addition or bug fix.
4
+ * Add tests for it. This is important so I don't break it in a future version
5
+ unintentionally.
6
+ * Commit, do not mess with rakefile, version, or history. (if you want to have
7
+ your own version, that is fine but bump version in a commit by itself I can
8
+ ignore when I pull)
9
+ * Send me a pull request. Bonus points for topic branches.
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2012-2013 Intridea, Inc.
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,392 @@
1
+ # TweetStream
2
+
3
+ [![Gem Version](https://badge.fury.io/rb/tweetstream.png)][gem]
4
+ [![Build Status](https://secure.travis-ci.org/intridea/tweetstream.png?branch=master)][travis]
5
+ [![Dependency Status](https://gemnasium.com/intridea/tweetstream.png?travis)][gemnasium]
6
+ [![Code Climate](https://codeclimate.com/github/intridea/tweetstream.png)][codeclimate]
7
+ [![Coverage Status](https://coveralls.io/repos/intridea/tweetstream/badge.png?branch=master)][coveralls]
8
+
9
+ [gem]: https://rubygems.org/gems/tweetstream
10
+ [travis]: http://travis-ci.org/intridea/tweetstream
11
+ [gemnasium]: https://gemnasium.com/intridea/tweetstream
12
+ [codeclimate]: https://codeclimate.com/github/intridea/tweetstream
13
+ [coveralls]: https://coveralls.io/r/intridea/tweetstream
14
+
15
+ TweetStream provides simple Ruby access to [Twitter's Streaming API](https://dev.twitter.com/docs/streaming-api).
16
+
17
+ **Note:** TweetStream does not currently work with Ruby 2.0, this is a [known issue](https://github.com/intridea/tweetstream/issues/117).
18
+
19
+ ## Installation
20
+
21
+ gem install tweetstream
22
+
23
+ ## Usage
24
+
25
+ Using TweetStream is quite simple:
26
+
27
+ ```ruby
28
+ require 'tweetstream'
29
+
30
+ TweetStream.configure do |config|
31
+ config.consumer_key = 'abcdefghijklmnopqrstuvwxyz'
32
+ config.consumer_secret = '0123456789'
33
+ config.oauth_token = 'abcdefghijklmnopqrstuvwxyz'
34
+ config.oauth_token_secret = '0123456789'
35
+ config.auth_method = :oauth
36
+ end
37
+
38
+ # This will pull a sample of all tweets based on
39
+ # your Twitter account's Streaming API role.
40
+ TweetStream::Client.new.sample do |status|
41
+ # The status object is a special Hash with
42
+ # method access to its keys.
43
+ puts "#{status.text}"
44
+ end
45
+ ```
46
+
47
+ You can also use it to track keywords or follow a given set of
48
+ user ids:
49
+
50
+ ```ruby
51
+ # Use 'track' to track a list of single-word keywords
52
+ TweetStream::Client.new.track('term1', 'term2') do |status|
53
+ puts "#{status.text}"
54
+ end
55
+
56
+ # Use 'follow' to follow a group of user ids (integers, not screen names)
57
+ TweetStream::Client.new.follow(14252, 53235) do |status|
58
+ puts "#{status.text}"
59
+ end
60
+ ```
61
+
62
+ The methods available to TweetStream::Client are kept in parity
63
+ with the methods available on the Streaming API wiki page.
64
+
65
+ ## Changes in 2.0
66
+
67
+ TweetStream 2.0 introduces a number of requested features and bug fixes. For
68
+ the complete list refer to the [changelog](CHANGELOG.md#version-200). Notable
69
+ additions in 2.0 include:
70
+
71
+ ### OAuth
72
+
73
+ OAuth is now the default authentication method. Both userstreams and Site
74
+ Streams exclusively work with OAuth. TweetStream still supports Basic Auth,
75
+ however it is no longer the default. If you are still using Basic Auth, you
76
+ should plan to move to OAuth as soon as possible.
77
+
78
+ ### Site Stream Support
79
+
80
+ Site Streams are now fully supported, including the connection management functionality.
81
+
82
+ ### Compatability with the Twitter gem
83
+
84
+ TweetStream now emits objects from the [Twitter gem](https://github.com/sferik/twitter) instead of custom hashes. These objects are already defined in the `twitter` gem and are superior to the custom objects in the following ways:
85
+
86
+ 1. Object equivalence (`#==` returns true if `#id`s are the same).
87
+ 2. The `#created_at` method returns a `Date` instead of a `String`.
88
+ 3. Allows boolean methods to be called with a question mark (e.g.
89
+ `User#protected?`)
90
+
91
+ Additionally, any new features that are added to objects in the
92
+ `twitter` gem (e.g. identity map) will be automatically inherited by TweetStream.
93
+
94
+ ### em-twitter
95
+
96
+ We've replaced the underlying gem that connects to the streaming API. [twitter-stream](https://github.com/voloko/twitter-stream) has been replaced with [em-twitter](https://github.com/spagalloco/em-twitter).
97
+ It offers functionality parity with twitter-stream while also supporting several new features.
98
+
99
+ ### Removal of on_interval callback
100
+
101
+ We have removed the `on_interval` callback. If you require interval-based timers, it is possible to run
102
+ TweetStream inside an already running EventMachine reactor in which you can define `EM::Timer` or `EM::PeriodicTimer`
103
+ for time-based operations:
104
+
105
+ ```ruby
106
+ EM.run do
107
+ client = TweetStream::Client.new
108
+
109
+ EM::PeriodicTimer.new(10) do
110
+ # do something on an interval
111
+ end
112
+ end
113
+ ```
114
+
115
+ ### Additional Notes
116
+
117
+ The parser configuration method has been removed as MultiJson automatically detects existing parsers.
118
+
119
+ ## Using the Twitter Userstream
120
+
121
+ Using the Twitter userstream works similarly to regular streaming, except you use the `userstream` method.
122
+
123
+ ```ruby
124
+ # Use 'userstream' to get message from your stream
125
+ client = TweetStream::Client.new
126
+
127
+ client.userstream do |status|
128
+ puts status.text
129
+ end
130
+ ```
131
+
132
+ ## Using Twitter Site Streams
133
+
134
+ ```ruby
135
+ client = TweetStream::Client.new
136
+
137
+ client.sitestream(['115192457'], :followings => true) do |status|
138
+ puts status.inspect
139
+ end
140
+ ```
141
+
142
+ Once connected, you can [control the Site Stream connection](https://dev.twitter.com/docs/streaming-apis/streams/site/control):
143
+
144
+ ```ruby
145
+ # add users to the stream
146
+ client.control.add_user('2039761')
147
+
148
+ # remove users from the stream
149
+ client.control.remove_user('115192457')
150
+
151
+ # obtain a list of followings of users in the stream
152
+ client.control.friends_ids('115192457') do |friends|
153
+ # do something
154
+ end
155
+
156
+ # obtain the current state of the stream
157
+ client.control.info do |info|
158
+ # do something
159
+ end
160
+ ```
161
+
162
+ Note that per Twitter's documentation, connection management features are not
163
+ immediately available when connected
164
+
165
+ You also can use method hooks for both regular timeline statuses and direct messages.
166
+
167
+ ```ruby
168
+ client = TweetStream::Client.new
169
+
170
+ client.on_direct_message do |direct_message|
171
+ puts "direct message"
172
+ puts direct_message.text
173
+ end
174
+
175
+ client.on_timeline_status do |status|
176
+ puts "timeline status"
177
+ puts status.text
178
+ end
179
+
180
+ client.userstream
181
+ ```
182
+
183
+ ## Authentication
184
+
185
+ TweetStream supports OAuth and Basic Auth. `TweetStream::Client` now accepts
186
+ a hash:
187
+
188
+ ```ruby
189
+ TweetStream::Client.new(:username => 'you', :password => 'pass')
190
+ ```
191
+
192
+ Alternatively, you can configure TweetStream via the configure method:
193
+
194
+ ```ruby
195
+ TweetStream.configure do |config|
196
+ config.consumer_key = 'cVcIw5zoLFE2a4BdDsmmA'
197
+ config.consumer_secret = 'yYgVgvTT9uCFAi2IuscbYTCqwJZ1sdQxzISvLhNWUA'
198
+ config.oauth_token = '4618-H3gU7mjDQ7MtFkAwHhCqD91Cp4RqDTp1AKwGzpHGL3I'
199
+ config.oauth_token_secret = 'xmc9kFgOXpMdQ590Tho2gV7fE71v5OmBrX8qPGh7Y'
200
+ config.auth_method = :oauth
201
+ end
202
+ ```
203
+
204
+ If you are using Basic Auth:
205
+
206
+ ```ruby
207
+ TweetStream.configure do |config|
208
+ config.username = 'username'
209
+ config.password = 'password'
210
+ config.auth_method = :basic
211
+ end
212
+ ```
213
+
214
+ TweetStream assumes OAuth by default. If you are using Basic Auth, it is recommended
215
+ that you update your code to use OAuth as Twitter is likely to phase out Basic Auth
216
+ support. Basic Auth is only available for public streams as User Stream and Site Stream
217
+ functionality [only support OAuth](https://dev.twitter.com/docs/streaming-apis/connecting#Authentication).
218
+
219
+ ## Parsing JSON
220
+
221
+ TweetStream supports swappable JSON backends via MultiJson. Simply require your preferred
222
+ JSON parser and it will be used to parse responses.
223
+
224
+ ## Handling Deletes and Rate Limitations
225
+
226
+ Sometimes the Streaming API will send messages other than statuses.
227
+ Specifically, it does so when a status is deleted or rate limitations
228
+ have caused some tweets not to appear in the stream. To handle these,
229
+ you can use the on_delete, on_limit and on_enhance_your_calm methods. Example:
230
+
231
+ ```ruby
232
+ @client = TweetStream::Client.new
233
+
234
+ @client.on_delete do |status_id, user_id|
235
+ Tweet.delete(status_id)
236
+ end
237
+
238
+ @client.on_limit do |skip_count|
239
+ # do something
240
+ end
241
+
242
+ @client.on_enhance_your_calm do
243
+ # do something
244
+ end
245
+
246
+ @client.track('intridea')
247
+ ```
248
+
249
+ The on_delete and on_limit methods can also be chained:
250
+
251
+ ```ruby
252
+ TweetStream::Client.new.on_delete{ |status_id, user_id|
253
+ Tweet.delete(status_id)
254
+ }.on_limit { |skip_count|
255
+ # do something
256
+ }.track('intridea') do |status|
257
+ # do something with the status like normal
258
+ end
259
+ ```
260
+
261
+ You can also provide `:delete` and/or `:limit`
262
+ options when you make your method call:
263
+
264
+ ```ruby
265
+ TweetStream::Client.new.track('intridea',
266
+ :delete => Proc.new{ |status_id, user_id| # do something },
267
+ :limit => Proc.new{ |skip_count| # do something }
268
+ ) do |status|
269
+ # do something with the status like normal
270
+ end
271
+ ```
272
+
273
+ Twitter recommends honoring deletions as quickly as possible, and
274
+ you would likely be wise to integrate this functionality into your
275
+ application.
276
+
277
+ ## Errors and Reconnecting
278
+
279
+ TweetStream uses EventMachine to connect to the Twitter Streaming
280
+ API, and attempts to honor Twitter's guidelines in terms of automatic
281
+ reconnection. When Twitter becomes unavailable, the block specified
282
+ by you in `on_error` will be called. Note that this does not
283
+ indicate something is actually wrong, just that Twitter is momentarily
284
+ down. It could be for routine maintenance, etc.
285
+
286
+ ```ruby
287
+ TweetStream::Client.new.on_error do |message|
288
+ # Log your error message somewhere
289
+ end.track('term') do |status|
290
+ # Do things when nothing's wrong
291
+ end
292
+ ```
293
+
294
+ However, if the maximum number of reconnect attempts has been reached,
295
+ TweetStream will raise a `TweetStream::ReconnectError` with
296
+ information about the timeout and number of retries attempted.
297
+
298
+ On reconnect, the block specified by you in `on_reconnect` will be called:
299
+
300
+ ```ruby
301
+ TweetStream::Client.new.on_reconnect do |timeout, retries|
302
+ # Do something with the reconnect
303
+ end.track('term') do |status|
304
+ # Do things when nothing's wrong
305
+ end
306
+ ```
307
+
308
+ ## Terminating a TweetStream
309
+
310
+ It is often the case that you will need to change the parameters of your
311
+ track or follow tweet streams. In the case that you need to terminate
312
+ a stream, you may add a second argument to your block that will yield
313
+ the client itself:
314
+
315
+ ```ruby
316
+ # Stop after collecting 10 statuses
317
+ @statuses = []
318
+ TweetStream::Client.new.sample do |status, client|
319
+ @statuses << status
320
+ client.stop if @statuses.size >= 10
321
+ end
322
+ ```
323
+
324
+ When `stop` is called, TweetStream will return from the block
325
+ the last successfully yielded status, allowing you to make note of
326
+ it in your application as necessary.
327
+
328
+ ## Daemonizing
329
+
330
+ It is also possible to create a daemonized script quite easily
331
+ using the TweetStream library:
332
+
333
+ ```ruby
334
+ # The third argument is an optional process name
335
+ TweetStream::Daemon.new('tracker').track('term1', 'term2') do |status|
336
+ # do something in the background
337
+ end
338
+ ```
339
+
340
+ If you put the above into a script and run the script with `ruby scriptname.rb`,
341
+ you will see a list of daemonization commands such as start, stop, and run.
342
+
343
+ A frequent use case is to use TweetStream along with ActiveRecord to insert new
344
+ statuses to a database. The library TweetStream uses the `daemons` gem for
345
+ daemonization which forks a new process when the daemon is created. After forking,
346
+ you'll need to reconnect to the database:
347
+
348
+ ```ruby
349
+ ENV["RAILS_ENV"] ||= "production"
350
+
351
+ root = File.expand_path(File.join(File.dirname(__FILE__), '..'))
352
+ require File.join(root, "config", "environment")
353
+
354
+ daemon = TweetStream::Daemon.new('tracker', :log_output => true)
355
+ daemon.on_inited do
356
+ ActiveRecord::Base.connection.reconnect!
357
+ ActiveRecord::Base.logger = Logger.new(File.open('log/stream.log', 'w+'))
358
+ end
359
+ daemon.track('term1') do |tweet|
360
+ Status.create_from_tweet(tweet)
361
+ end
362
+ ```
363
+
364
+ ## Proxy Support
365
+
366
+ TweetStream supports a configurable proxy:
367
+
368
+ ```ruby
369
+ TweetStream.configure do |config|
370
+ config.proxy = { :uri => 'http://myproxy:8081' }
371
+ end
372
+ ```
373
+
374
+ Your proxy will now be used for all connections.
375
+
376
+ ## REST
377
+
378
+ To access the Twitter REST API, we recommend the [Twitter][] gem.
379
+
380
+ [twitter]: https://github.com/sferik/twitter
381
+
382
+ ## Contributors
383
+
384
+ * [Michael Bleigh](https://github.com/mbleigh) (initial gem)
385
+ * [Steve Agalloco](https://github.com/spagalloco) (current maintainer)
386
+ * [Erik Michaels-Ober](https://github.com/sferik) (current maintainer)
387
+ * [Countless others](https://github.com/intridea/tweetstream/graphs/contributors)
388
+
389
+ ## Copyright
390
+
391
+ Copyright (c) 2012-2013 Intridea, Inc. (http://www.intridea.com/). See
392
+ [LICENSE](LICENSE.md) for details.