weeter 0.9.2 → 0.10.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.travis.yml +5 -0
- data/Gemfile +1 -0
- data/README.md +13 -6
- data/lib/weeter.rb +12 -11
- data/lib/weeter/configuration.rb +13 -5
- data/lib/weeter/configuration/client_app_config.rb +1 -1
- data/lib/weeter/configuration/limiter_config.rb +16 -0
- data/lib/weeter/configuration/twitter_config.rb +1 -1
- data/lib/weeter/limitator.rb +76 -0
- data/lib/weeter/plugins/subscription/redis.rb +5 -5
- data/lib/weeter/runner.rb +13 -2
- data/lib/weeter/twitter/tweet_consumer.rb +14 -2
- data/lib/weeter/twitter/tweet_item.rb +12 -7
- data/lib/weeter/version.rb +1 -1
- data/spec/spec_helper.rb +1 -1
- data/spec/weeter/limitator_spec.rb +97 -0
- data/spec/weeter/twitter/tweet_consumer_spec.rb +11 -6
- data/weeter.conf.example +11 -5
- metadata +107 -97
data/.travis.yml
ADDED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -6,20 +6,22 @@ Status
|
|
6
6
|
======
|
7
7
|
Alpha. A previous version of this code has been in production for some time. It has been substantially refactored and will be battle-tested soon.
|
8
8
|
|
9
|
+
[![Build Status](https://secure.travis-ci.org/lukemelia/weeter.png?branch=master)](http://travis-ci.org/lukemelia/weeter)
|
10
|
+
|
9
11
|
Getting set up
|
10
12
|
==============
|
11
13
|
|
12
14
|
$ bundle install
|
13
15
|
|
14
16
|
Make a copy of the weeter.conf.example file named weeter.conf. Twitter configuration, client app configuration and weeter configuration are defined in separate
|
15
|
-
blocks. To configure how you connect to Twitter (basic auth or oauth), modify the twitter section of the configuration.
|
17
|
+
blocks. To configure how you connect to Twitter (basic auth or oauth), modify the twitter section of the configuration.
|
16
18
|
|
17
19
|
To configure how weeter connects to your client app, modify the client app configuration section:
|
18
20
|
|
19
21
|
Notifications
|
20
22
|
-------------
|
21
23
|
|
22
|
-
* *notification_plugin*: A symbol matching the underscorized name of the NotificationPlugin subclass to use. Current options are :http and :resque
|
24
|
+
* *notification_plugin*: A symbol matching the underscorized name of the NotificationPlugin subclass to use. Current options are :http and :resque
|
23
25
|
|
24
26
|
For option :http, also provide the following:
|
25
27
|
|
@@ -39,26 +41,31 @@ For option :resque, provide the following:
|
|
39
41
|
Subscriptions
|
40
42
|
-------------
|
41
43
|
|
42
|
-
* *subscription_plugin*: A symbol matching the underscorized name of the SubscrptionsPlugin subclass to use. Current options are :http and :redis
|
44
|
+
* *subscription_plugin*: A symbol matching the underscorized name of the SubscrptionsPlugin subclass to use. Current options are :http and :redis
|
43
45
|
|
44
46
|
For option :http, also provide the following:
|
45
47
|
|
46
48
|
* *oauth*: See the conf file for an example
|
47
49
|
|
48
50
|
* *subscriptions_url*: The URL at which to find JSON describing the Twitter users to follow (maximum 5,000 at the default API access level) and the terms to track (maximum 400 at the default API access level). Example content:
|
49
|
-
`{"follow":"19466709", "759251"
|
51
|
+
`{"follow":["19466709", "759251"],"track":["#lolcats","#bieber"]}`
|
50
52
|
|
51
53
|
* *subscription_updates_port*: The port Weeter should listen on for HTTP connections. If you have changes to your subscriptions data, POST the full JSON to the weeter's root URL. This will trigger weeter to reconnect to Twitter with the updated filters in place.
|
52
54
|
|
53
55
|
For option :redis, also provide the following:
|
54
56
|
|
55
57
|
* *subscriptions_key*: The Redis key at which the Weeter can find JSON describing the Twitter users to follow (maximum 5,000 at the default API access level) and the terms to track (maximum 400 at the default API access level). Example content:
|
56
|
-
`{"follow":"19466709", "759251"
|
58
|
+
`{"follow":["19466709", "759251"],"track":["#lolcats","#bieber"]}`
|
57
59
|
|
58
60
|
* *subscriptions_changed_channel*: The Redis publish/subscribe channel to subscribe to in order to be notified that the subscriptions have changed. When your app has an updated set of subscriptions, it should update the _subscriptions_key_ and publish a "CHANGED" message to this channel. Weeter will then retrieve an updated set of subscriptions from Redis and reconnect to twitter.
|
59
61
|
|
60
62
|
* *redis_uri*: Redis connection string
|
61
63
|
|
64
|
+
Rate Limiting
|
65
|
+
-------------
|
66
|
+
|
67
|
+
If you track high-volume hashtags, it's easy to bite off more than your infrastructure can chew. To help with this situation, Weeter has basic rate-limiting built-in. You can enable it and configure a max and time period in your conf file (see weeter.conf.example for syntax). If the max is reached in a given period, additional tweets with that hashtag that come in during the time period will be logged and discarded. New tweets will begin being allowed through at the beginning of the subsequent period.
|
68
|
+
|
62
69
|
Running weeter
|
63
70
|
==============
|
64
71
|
|
@@ -91,7 +98,7 @@ TODO
|
|
91
98
|
|
92
99
|
Credits
|
93
100
|
======
|
94
|
-
Thanks to Weplay for initial development and open sourcing Weeter. In particular, credit goes to Noah Davis and Joey Aghion. Further development by Luke Melia at Yapp.
|
101
|
+
Thanks to Weplay for initial development and open sourcing Weeter. In particular, credit goes to Noah Davis and Joey Aghion. Further development by Luke Melia and Stefan Penner at Yapp.
|
95
102
|
|
96
103
|
License
|
97
104
|
=======
|
data/lib/weeter.rb
CHANGED
@@ -2,20 +2,21 @@ require 'eventmachine'
|
|
2
2
|
require 'json'
|
3
3
|
require 'logger'
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
require 'weeter/plugins'
|
8
|
-
require 'weeter/runner'
|
9
|
-
require 'weeter/twitter'
|
5
|
+
module Weeter
|
6
|
+
extend self
|
10
7
|
|
8
|
+
autoload 'Cli', 'weeter/cli'
|
9
|
+
autoload 'Plugins', 'weeter/plugins'
|
10
|
+
autoload 'Runner', 'weeter/runner'
|
11
|
+
autoload 'Twitter', 'weeter/twitter'
|
12
|
+
autoload 'Limitator', 'weeter/limitator'
|
13
|
+
autoload 'Configuration', 'weeter/configuration'
|
11
14
|
|
12
|
-
|
13
|
-
|
14
|
-
def self.configure
|
15
|
+
def configure
|
15
16
|
yield Configuration.instance
|
16
17
|
end
|
17
|
-
|
18
|
-
def
|
18
|
+
|
19
|
+
def logger
|
19
20
|
@logger ||= begin
|
20
21
|
if Configuration.instance.log_path == false
|
21
22
|
nil
|
@@ -26,4 +27,4 @@ module Weeter
|
|
26
27
|
end
|
27
28
|
end
|
28
29
|
end
|
29
|
-
end
|
30
|
+
end
|
data/lib/weeter/configuration.rb
CHANGED
@@ -1,22 +1,30 @@
|
|
1
1
|
require "singleton"
|
2
|
-
|
3
|
-
require "weeter/configuration/twitter_config"
|
4
|
-
|
2
|
+
|
5
3
|
module Weeter
|
6
4
|
|
7
5
|
class Configuration
|
8
6
|
include Singleton
|
9
7
|
attr_accessor :log_path
|
10
8
|
|
9
|
+
autoload :ClientAppConfig, 'weeter/configuration/client_app_config'
|
10
|
+
autoload :TwitterConfig, 'weeter/configuration/twitter_config'
|
11
|
+
autoload :LimiterConfig, 'weeter/configuration/limiter_config'
|
12
|
+
|
13
|
+
|
11
14
|
def twitter
|
12
15
|
yield Configuration::TwitterConfig.instance if block_given?
|
13
16
|
Configuration::TwitterConfig.instance
|
14
17
|
end
|
15
|
-
|
18
|
+
|
19
|
+
def limiter
|
20
|
+
yield Configuration::LimiterConfig.instance if block_given?
|
21
|
+
Configuration::LimiterConfig.instance
|
22
|
+
end
|
23
|
+
|
16
24
|
def client_app
|
17
25
|
@client_app_config ||= Configuration::ClientAppConfig.new
|
18
26
|
yield @client_app_config if block_given?
|
19
27
|
@client_app_config
|
20
28
|
end
|
21
29
|
end
|
22
|
-
end
|
30
|
+
end
|
@@ -0,0 +1,16 @@
|
|
1
|
+
require 'singleton'
|
2
|
+
require 'active_support/core_ext/numeric'
|
3
|
+
|
4
|
+
module Weeter
|
5
|
+
class Configuration
|
6
|
+
class LimiterConfig
|
7
|
+
include Singleton
|
8
|
+
attr_writer :enabled
|
9
|
+
attr_accessor :max, :duration
|
10
|
+
|
11
|
+
def enabled
|
12
|
+
@enabled || false
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
@@ -0,0 +1,76 @@
|
|
1
|
+
require 'active_support/core_ext/numeric/time'
|
2
|
+
|
3
|
+
module Weeter
|
4
|
+
class Limitator
|
5
|
+
|
6
|
+
module UNLIMITED
|
7
|
+
def self.limit?(*args)
|
8
|
+
false
|
9
|
+
end
|
10
|
+
end
|
11
|
+
|
12
|
+
class TimeWindow
|
13
|
+
def initialize(options = {})
|
14
|
+
@start = options.fetch(:start)
|
15
|
+
@duration = options.fetch(:duration)
|
16
|
+
end
|
17
|
+
|
18
|
+
def over?(time)
|
19
|
+
time - @start > @duration
|
20
|
+
end
|
21
|
+
|
22
|
+
def begin_new_window(at)
|
23
|
+
@start = at
|
24
|
+
end
|
25
|
+
end
|
26
|
+
|
27
|
+
attr_accessor :lookup, :window, :max
|
28
|
+
|
29
|
+
def initialize(options = {})
|
30
|
+
self.window= TimeWindow.new({
|
31
|
+
start: self.now,
|
32
|
+
duration: options.fetch(:duration)
|
33
|
+
})
|
34
|
+
|
35
|
+
self.max = options.fetch(:max)
|
36
|
+
|
37
|
+
flush
|
38
|
+
end
|
39
|
+
|
40
|
+
def limit?(*keys)
|
41
|
+
ensure_correct_window
|
42
|
+
|
43
|
+
keys.each do |key|
|
44
|
+
increment(key)
|
45
|
+
end
|
46
|
+
|
47
|
+
keys.any? { |key| exceeds_max?(key) }
|
48
|
+
end
|
49
|
+
|
50
|
+
protected
|
51
|
+
|
52
|
+
def now
|
53
|
+
Time.now
|
54
|
+
end
|
55
|
+
|
56
|
+
def increment(key)
|
57
|
+
lookup[key] += 1
|
58
|
+
end
|
59
|
+
|
60
|
+
def exceeds_max?(key)
|
61
|
+
lookup[key] > max
|
62
|
+
end
|
63
|
+
|
64
|
+
def ensure_correct_window
|
65
|
+
return unless window.over?(now)
|
66
|
+
|
67
|
+
flush
|
68
|
+
|
69
|
+
window.begin_new_window(now)
|
70
|
+
end
|
71
|
+
|
72
|
+
def flush
|
73
|
+
self.lookup = Hash.new(0)
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
@@ -5,7 +5,7 @@ module Weeter
|
|
5
5
|
module Subscription
|
6
6
|
class Redis
|
7
7
|
include Weeter::Plugins::Net::Redis
|
8
|
-
|
8
|
+
|
9
9
|
def initialize(client_app_config)
|
10
10
|
@config = client_app_config
|
11
11
|
end
|
@@ -29,9 +29,9 @@ module Weeter
|
|
29
29
|
end
|
30
30
|
end
|
31
31
|
end
|
32
|
-
|
32
|
+
|
33
33
|
protected
|
34
|
-
|
34
|
+
|
35
35
|
def redis
|
36
36
|
@redis ||= create_redis_client
|
37
37
|
end
|
@@ -39,8 +39,8 @@ module Weeter
|
|
39
39
|
def pub_sub_redis
|
40
40
|
@pub_sub_redis ||= create_redis_client
|
41
41
|
end
|
42
|
-
|
42
|
+
|
43
43
|
end
|
44
44
|
end
|
45
45
|
end
|
46
|
-
end
|
46
|
+
end
|
data/lib/weeter/runner.rb
CHANGED
@@ -24,7 +24,18 @@ module Weeter
|
|
24
24
|
end
|
25
25
|
|
26
26
|
protected
|
27
|
-
|
27
|
+
|
28
|
+
def limiter
|
29
|
+
@limiter ||= if @config.limiter.enabled
|
30
|
+
Weeter::Limitator.new({
|
31
|
+
max: @config.limiter.max,
|
32
|
+
duration: @config.limiter.duration
|
33
|
+
})
|
34
|
+
else
|
35
|
+
Weeter::Limitator::UNLIMITED
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
28
39
|
def notification_plugin
|
29
40
|
@notification_plugin ||= Weeter::Plugins::NotificationPlugin.new(@config.client_app)
|
30
41
|
end
|
@@ -34,7 +45,7 @@ module Weeter
|
|
34
45
|
end
|
35
46
|
|
36
47
|
def tweet_consumer
|
37
|
-
@tweet_consumer ||= Weeter::Twitter::TweetConsumer.new(@config.twitter, notification_plugin)
|
48
|
+
@tweet_consumer ||= Weeter::Twitter::TweetConsumer.new(@config.twitter, notification_plugin, limiter)
|
38
49
|
end
|
39
50
|
end
|
40
51
|
end
|
@@ -5,9 +5,12 @@ module Weeter
|
|
5
5
|
module Twitter
|
6
6
|
class TweetConsumer
|
7
7
|
|
8
|
-
|
8
|
+
attr_reader :limiter
|
9
|
+
|
10
|
+
def initialize(twitter_config, notifier, limiter)
|
9
11
|
@config = twitter_config
|
10
12
|
@notifier = notifier
|
13
|
+
@limiter = limiter
|
11
14
|
end
|
12
15
|
|
13
16
|
def connect(filter_params)
|
@@ -19,7 +22,9 @@ module Weeter
|
|
19
22
|
begin
|
20
23
|
tweet_item = TweetItem.new(MultiJson.decode(item))
|
21
24
|
|
22
|
-
if tweet_item.
|
25
|
+
if limiter.limit?(*tweet_item.limiting_facets)
|
26
|
+
rate_limit_tweet(tweet_item)
|
27
|
+
elsif tweet_item.deletion?
|
23
28
|
@notifier.delete_tweet(tweet_item)
|
24
29
|
elsif tweet_item.publishable?
|
25
30
|
@notifier.publish_tweet(tweet_item)
|
@@ -63,6 +68,13 @@ module Weeter
|
|
63
68
|
Weeter.logger.info("Ignoring tweet #{id} from user #{user_id}: #{text}")
|
64
69
|
end
|
65
70
|
|
71
|
+
def rate_limit_tweet(tweet_item)
|
72
|
+
id = tweet_item['id_str']
|
73
|
+
text = tweet_item['text']
|
74
|
+
user_id = tweet_item['user']['id_str']
|
75
|
+
|
76
|
+
Weeter.logger.info("Rate Limiting tweet #{id} from user #{user_id}: #{text}")
|
77
|
+
end
|
66
78
|
end
|
67
79
|
end
|
68
80
|
end
|
@@ -10,26 +10,31 @@ module Weeter
|
|
10
10
|
def deletion?
|
11
11
|
!@tweet_hash['delete'].nil?
|
12
12
|
end
|
13
|
-
|
13
|
+
|
14
14
|
def retweeted?
|
15
15
|
!@tweet_hash['retweeted_status'].nil? || @tweet_hash['text'] =~ /^RT @/i
|
16
16
|
end
|
17
|
-
|
17
|
+
|
18
18
|
def reply?
|
19
19
|
!@tweet_hash['in_reply_to_user_id_str'].nil? || @tweet_hash['text'] =~ /^@/
|
20
20
|
end
|
21
|
-
|
21
|
+
|
22
22
|
def publishable?
|
23
23
|
!retweeted? && !reply?
|
24
24
|
end
|
25
|
-
|
25
|
+
|
26
26
|
def [](val)
|
27
27
|
@tweet_hash[val]
|
28
28
|
end
|
29
|
-
|
29
|
+
|
30
30
|
def to_json
|
31
31
|
MultiJson.encode(@tweet_hash)
|
32
32
|
end
|
33
|
-
end
|
34
33
|
|
35
|
-
|
34
|
+
def limiting_facets
|
35
|
+
self['entities']['hashtags'].map do |tag|
|
36
|
+
tag['text'].downcase.chomp
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
data/lib/weeter/version.rb
CHANGED
data/spec/spec_helper.rb
CHANGED
@@ -0,0 +1,97 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Weeter::Limitator do
|
4
|
+
let(:limitator) do
|
5
|
+
Weeter::Limitator.new({
|
6
|
+
max: max,
|
7
|
+
duration: duration
|
8
|
+
})
|
9
|
+
end
|
10
|
+
|
11
|
+
let(:duration) { 10.minutes }
|
12
|
+
let(:max) { 10 }
|
13
|
+
|
14
|
+
let(:keys) { ['key'] }
|
15
|
+
|
16
|
+
describe '.new' do
|
17
|
+
it { limitator.should be }
|
18
|
+
end
|
19
|
+
|
20
|
+
describe '#limit?' do
|
21
|
+
|
22
|
+
subject do
|
23
|
+
limitator.limit?(*keys)
|
24
|
+
end
|
25
|
+
|
26
|
+
context 'max: 0' do
|
27
|
+
let(:max) { 0 }
|
28
|
+
it { should be_true }
|
29
|
+
|
30
|
+
context 'no keys' do
|
31
|
+
let(:keys) { [] }
|
32
|
+
it { should be_false }
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
context 'max: 1' do
|
37
|
+
let(:max) { 1 }
|
38
|
+
|
39
|
+
it { should be_false }
|
40
|
+
|
41
|
+
context 'two keys within max' do
|
42
|
+
let(:keys) { ['key', 'key2'] }
|
43
|
+
|
44
|
+
it { should be_false }
|
45
|
+
end
|
46
|
+
|
47
|
+
context 'no keys' do
|
48
|
+
let(:keys) { [] }
|
49
|
+
it { should be_false }
|
50
|
+
end
|
51
|
+
|
52
|
+
context 'one key outside max' do
|
53
|
+
before do
|
54
|
+
max.times do
|
55
|
+
limitator.limit?(*keys)
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
it { should be_true }
|
60
|
+
|
61
|
+
context 'outside duration' do
|
62
|
+
let(:some_time_after_duration) do
|
63
|
+
Time.now + duration
|
64
|
+
end
|
65
|
+
|
66
|
+
before do
|
67
|
+
limitator.stub(:now).and_return(some_time_after_duration)
|
68
|
+
end
|
69
|
+
|
70
|
+
it { should be_false }
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
context 'two keys outside' do
|
75
|
+
let(:keys) { ['key', 'key2'] }
|
76
|
+
|
77
|
+
before do
|
78
|
+
limitator.limit?(*keys)
|
79
|
+
end
|
80
|
+
|
81
|
+
it { should be_true }
|
82
|
+
end
|
83
|
+
|
84
|
+
context 'one key outside max: 1, one key within max: 1' do
|
85
|
+
let(:max) { 1 }
|
86
|
+
let(:keys) { ['key', 'key2'] }
|
87
|
+
|
88
|
+
before do
|
89
|
+
limitator.limit?(*[keys.first])
|
90
|
+
end
|
91
|
+
|
92
|
+
it { should be_true }
|
93
|
+
end
|
94
|
+
end
|
95
|
+
end
|
96
|
+
end
|
97
|
+
|
@@ -1,7 +1,12 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
|
3
3
|
describe Weeter::Twitter::TweetConsumer do
|
4
|
-
|
4
|
+
let(:limiter) do
|
5
|
+
Weeter::Limitator.new({
|
6
|
+
max: 1,
|
7
|
+
duration: 10.minutes
|
8
|
+
})
|
9
|
+
end
|
5
10
|
|
6
11
|
describe "auth" do
|
7
12
|
it 'should use connect to JSON stream with auth options for the configuration' do
|
@@ -9,7 +14,7 @@ describe Weeter::Twitter::TweetConsumer do
|
|
9
14
|
Twitter::JSONStream.stub!(:connect).and_return(@mock_stream)
|
10
15
|
|
11
16
|
Weeter::Configuration::TwitterConfig.instance.stub!(:auth_options).and_return(:foo => :bar)
|
12
|
-
consumer = Weeter::Twitter::TweetConsumer.new(Weeter::Configuration::TwitterConfig.instance, mock('NotificationPlugin'))
|
17
|
+
consumer = Weeter::Twitter::TweetConsumer.new(Weeter::Configuration::TwitterConfig.instance, mock('NotificationPlugin'), limiter)
|
13
18
|
Twitter::JSONStream.should_receive(:connect).with(hash_including(:foo => :bar))
|
14
19
|
consumer.connect({'follow' => ['1','2']})
|
15
20
|
end
|
@@ -25,7 +30,7 @@ describe Weeter::Twitter::TweetConsumer do
|
|
25
30
|
@mock_stream.stub!(:each_item).and_yield(MultiJson.encode(@tweet_values))
|
26
31
|
Twitter::JSONStream.stub!(:connect).and_return(@mock_stream)
|
27
32
|
@client_proxy = mock('NotificationPlugin', :publish_tweet => nil)
|
28
|
-
@consumer = Weeter::Twitter::TweetConsumer.new(Weeter::Configuration::TwitterConfig.instance, @client_proxy)
|
33
|
+
@consumer = Weeter::Twitter::TweetConsumer.new(Weeter::Configuration::TwitterConfig.instance, @client_proxy, limiter)
|
29
34
|
end
|
30
35
|
|
31
36
|
after(:each) do
|
@@ -43,19 +48,19 @@ describe Weeter::Twitter::TweetConsumer do
|
|
43
48
|
end
|
44
49
|
|
45
50
|
it "should publish new tweet if publishable" do
|
46
|
-
mock_tweet = mock('tweet', :deletion? => false, :publishable? => true)
|
51
|
+
mock_tweet = mock('tweet', :deletion? => false, :publishable? => true, :limiting_facets => [])
|
47
52
|
tweet_item = Weeter::TweetItem.stub!(:new).and_return mock_tweet
|
48
53
|
@client_proxy.should_receive(:publish_tweet).with(mock_tweet)
|
49
54
|
end
|
50
55
|
|
51
56
|
it "should not publish unpublishable tweets" do
|
52
|
-
mock_tweet = mock('tweet', :deletion? => false, :publishable? => false, :[] => '')
|
57
|
+
mock_tweet = mock('tweet', :deletion? => false, :publishable? => false, :[] => '', :limiting_facets => [])
|
53
58
|
tweet_item = Weeter::TweetItem.stub!(:new).and_return mock_tweet
|
54
59
|
@client_proxy.should_not_receive(:publish_tweet).with(mock_tweet)
|
55
60
|
end
|
56
61
|
|
57
62
|
it "should delete deletion tweets" do
|
58
|
-
mock_tweet = mock('tweet', :deletion? => true, :publishable? => false)
|
63
|
+
mock_tweet = mock('tweet', :deletion? => true, :publishable? => false, :limiting_facets => [])
|
59
64
|
tweet_item = Weeter::TweetItem.stub!(:new).and_return mock_tweet
|
60
65
|
@client_proxy.should_receive(:delete_tweet).with(mock_tweet)
|
61
66
|
end
|
data/weeter.conf.example
CHANGED
@@ -12,7 +12,7 @@ Weeter.configure do |conf|
|
|
12
12
|
conf.twitter do |twitter|
|
13
13
|
# For basic auth
|
14
14
|
# twitter.basic_auth = {:username => 'johnny', :password => 'secret'}
|
15
|
-
|
15
|
+
|
16
16
|
# Or, for oauth
|
17
17
|
twitter.oauth = {:consumer_key => ENV['WEETER_TWITTER_CONSUMER_KEY'],
|
18
18
|
:consumer_secret => ENV['WEETER_TWITTER_CONSUMER_SECRET'],
|
@@ -21,6 +21,12 @@ Weeter.configure do |conf|
|
|
21
21
|
}
|
22
22
|
end
|
23
23
|
|
24
|
+
conf.limiter do |limit|
|
25
|
+
limit.enabled = true
|
26
|
+
limit.max = 100
|
27
|
+
limit.duration = 10.minutes
|
28
|
+
end
|
29
|
+
|
24
30
|
conf.client_app do |client_app|
|
25
31
|
client_app.notification_plugin = :http
|
26
32
|
client_app.oauth = {
|
@@ -31,21 +37,21 @@ Weeter.configure do |conf|
|
|
31
37
|
}
|
32
38
|
client_app.publish_url = ENV['WEETER_CLIENT_PUBLISH_URL']
|
33
39
|
client_app.delete_url = ENV['WEETER_CLIENT_DELETE_URL']
|
34
|
-
|
40
|
+
|
35
41
|
client_app.subscription_plugin = :http
|
36
42
|
client_app.subscriptions_url = ENV['WEETER_CLIENT_SUBSCRIPTIONS_URL']
|
37
43
|
client_app.subscription_updates_port = 7337 # only effective if running as a daemon
|
38
44
|
end
|
39
|
-
|
45
|
+
|
40
46
|
# Alternately...
|
41
47
|
|
42
48
|
# conf.client_app do |client_app|
|
43
49
|
# client_app.notification_plugin = :resque
|
44
50
|
# client_app.queue = 'weeter'
|
45
51
|
# client_app.redis_uri = 'redis://redistogo:abcdef0123456789abcdef0123456789@somehost.redistogo.com:9052/'
|
46
|
-
#
|
52
|
+
#
|
47
53
|
# client_app.subscription_plugin = :redis
|
48
54
|
# client_app.subscriptions_key = 'weeter:subscriptions'
|
49
55
|
# client_app.subscriptions_changed_channel = 'weeter:subscriptions_changed'
|
50
56
|
# end
|
51
|
-
end
|
57
|
+
end
|
metadata
CHANGED
@@ -1,150 +1,153 @@
|
|
1
|
-
--- !ruby/object:Gem::Specification
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
2
|
name: weeter
|
3
|
-
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.9.2
|
3
|
+
version: !ruby/object:Gem::Version
|
5
4
|
prerelease:
|
5
|
+
version: 0.10.0
|
6
6
|
platform: ruby
|
7
|
-
authors:
|
7
|
+
authors:
|
8
8
|
- Luke Melia
|
9
9
|
- Noah Davis
|
10
10
|
- Joey Aghion
|
11
11
|
autorequire:
|
12
12
|
bindir: bin
|
13
13
|
cert_chain: []
|
14
|
-
|
15
|
-
|
16
|
-
|
14
|
+
|
15
|
+
date: 2012-10-24 00:00:00 Z
|
16
|
+
dependencies:
|
17
|
+
- !ruby/object:Gem::Dependency
|
17
18
|
name: eventmachine
|
18
|
-
|
19
|
+
prerelease: false
|
20
|
+
requirement: &id001 !ruby/object:Gem::Requirement
|
19
21
|
none: false
|
20
|
-
requirements:
|
21
|
-
- -
|
22
|
-
- !ruby/object:Gem::Version
|
23
|
-
version:
|
22
|
+
requirements:
|
23
|
+
- - ">="
|
24
|
+
- !ruby/object:Gem::Version
|
25
|
+
version: "0"
|
24
26
|
type: :runtime
|
25
|
-
|
26
|
-
|
27
|
-
- !ruby/object:Gem::Dependency
|
27
|
+
version_requirements: *id001
|
28
|
+
- !ruby/object:Gem::Dependency
|
28
29
|
name: eventmachine_httpserver
|
29
|
-
|
30
|
+
prerelease: false
|
31
|
+
requirement: &id002 !ruby/object:Gem::Requirement
|
30
32
|
none: false
|
31
|
-
requirements:
|
32
|
-
- -
|
33
|
-
- !ruby/object:Gem::Version
|
33
|
+
requirements:
|
34
|
+
- - ">="
|
35
|
+
- !ruby/object:Gem::Version
|
34
36
|
version: 0.2.1
|
35
37
|
type: :runtime
|
36
|
-
|
37
|
-
|
38
|
-
- !ruby/object:Gem::Dependency
|
38
|
+
version_requirements: *id002
|
39
|
+
- !ruby/object:Gem::Dependency
|
39
40
|
name: em-hiredis
|
40
|
-
|
41
|
+
prerelease: false
|
42
|
+
requirement: &id003 !ruby/object:Gem::Requirement
|
41
43
|
none: false
|
42
|
-
requirements:
|
43
|
-
- -
|
44
|
-
- !ruby/object:Gem::Version
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
45
47
|
version: 0.1.0
|
46
48
|
type: :runtime
|
47
|
-
|
48
|
-
|
49
|
-
- !ruby/object:Gem::Dependency
|
49
|
+
version_requirements: *id003
|
50
|
+
- !ruby/object:Gem::Dependency
|
50
51
|
name: multi_json
|
51
|
-
|
52
|
+
prerelease: false
|
53
|
+
requirement: &id004 !ruby/object:Gem::Requirement
|
52
54
|
none: false
|
53
|
-
requirements:
|
54
|
-
- -
|
55
|
-
- !ruby/object:Gem::Version
|
55
|
+
requirements:
|
56
|
+
- - ">="
|
57
|
+
- !ruby/object:Gem::Version
|
56
58
|
version: 1.0.2
|
57
59
|
type: :runtime
|
58
|
-
|
59
|
-
|
60
|
-
- !ruby/object:Gem::Dependency
|
60
|
+
version_requirements: *id004
|
61
|
+
- !ruby/object:Gem::Dependency
|
61
62
|
name: hashie
|
62
|
-
|
63
|
+
prerelease: false
|
64
|
+
requirement: &id005 !ruby/object:Gem::Requirement
|
63
65
|
none: false
|
64
|
-
requirements:
|
65
|
-
- -
|
66
|
-
- !ruby/object:Gem::Version
|
66
|
+
requirements:
|
67
|
+
- - ">="
|
68
|
+
- !ruby/object:Gem::Version
|
67
69
|
version: 1.1.0
|
68
70
|
type: :runtime
|
69
|
-
|
70
|
-
|
71
|
-
- !ruby/object:Gem::Dependency
|
71
|
+
version_requirements: *id005
|
72
|
+
- !ruby/object:Gem::Dependency
|
72
73
|
name: em-http-request
|
73
|
-
|
74
|
+
prerelease: false
|
75
|
+
requirement: &id006 !ruby/object:Gem::Requirement
|
74
76
|
none: false
|
75
|
-
requirements:
|
76
|
-
- -
|
77
|
-
- !ruby/object:Gem::Version
|
77
|
+
requirements:
|
78
|
+
- - ">="
|
79
|
+
- !ruby/object:Gem::Version
|
78
80
|
version: 1.0.0
|
79
81
|
type: :runtime
|
80
|
-
|
81
|
-
|
82
|
-
- !ruby/object:Gem::Dependency
|
82
|
+
version_requirements: *id006
|
83
|
+
- !ruby/object:Gem::Dependency
|
83
84
|
name: i18n
|
84
|
-
|
85
|
+
prerelease: false
|
86
|
+
requirement: &id007 !ruby/object:Gem::Requirement
|
85
87
|
none: false
|
86
|
-
requirements:
|
88
|
+
requirements:
|
87
89
|
- - ~>
|
88
|
-
- !ruby/object:Gem::Version
|
90
|
+
- !ruby/object:Gem::Version
|
89
91
|
version: 0.6.0
|
90
92
|
type: :runtime
|
91
|
-
|
92
|
-
|
93
|
-
- !ruby/object:Gem::Dependency
|
93
|
+
version_requirements: *id007
|
94
|
+
- !ruby/object:Gem::Dependency
|
94
95
|
name: activesupport
|
95
|
-
|
96
|
+
prerelease: false
|
97
|
+
requirement: &id008 !ruby/object:Gem::Requirement
|
96
98
|
none: false
|
97
|
-
requirements:
|
98
|
-
- -
|
99
|
-
- !ruby/object:Gem::Version
|
99
|
+
requirements:
|
100
|
+
- - ">="
|
101
|
+
- !ruby/object:Gem::Version
|
100
102
|
version: 3.1.1
|
101
103
|
type: :runtime
|
102
|
-
|
103
|
-
|
104
|
-
- !ruby/object:Gem::Dependency
|
104
|
+
version_requirements: *id008
|
105
|
+
- !ruby/object:Gem::Dependency
|
105
106
|
name: simple_oauth
|
106
|
-
|
107
|
+
prerelease: false
|
108
|
+
requirement: &id009 !ruby/object:Gem::Requirement
|
107
109
|
none: false
|
108
|
-
requirements:
|
110
|
+
requirements:
|
109
111
|
- - ~>
|
110
|
-
- !ruby/object:Gem::Version
|
112
|
+
- !ruby/object:Gem::Version
|
111
113
|
version: 0.1.5
|
112
114
|
type: :runtime
|
113
|
-
|
114
|
-
|
115
|
-
- !ruby/object:Gem::Dependency
|
115
|
+
version_requirements: *id009
|
116
|
+
- !ruby/object:Gem::Dependency
|
116
117
|
name: lukemelia-twitter-stream
|
117
|
-
|
118
|
+
prerelease: false
|
119
|
+
requirement: &id010 !ruby/object:Gem::Requirement
|
118
120
|
none: false
|
119
|
-
requirements:
|
121
|
+
requirements:
|
120
122
|
- - ~>
|
121
|
-
- !ruby/object:Gem::Version
|
123
|
+
- !ruby/object:Gem::Version
|
122
124
|
version: 0.1.15
|
123
125
|
type: :runtime
|
124
|
-
|
125
|
-
|
126
|
-
- !ruby/object:Gem::Dependency
|
126
|
+
version_requirements: *id010
|
127
|
+
- !ruby/object:Gem::Dependency
|
127
128
|
name: rspec
|
128
|
-
|
129
|
+
prerelease: false
|
130
|
+
requirement: &id011 !ruby/object:Gem::Requirement
|
129
131
|
none: false
|
130
|
-
requirements:
|
132
|
+
requirements:
|
131
133
|
- - ~>
|
132
|
-
- !ruby/object:Gem::Version
|
134
|
+
- !ruby/object:Gem::Version
|
133
135
|
version: 2.6.0
|
134
136
|
type: :development
|
135
|
-
|
136
|
-
|
137
|
-
|
138
|
-
streaming API, and notifies your app with each new tweet.
|
139
|
-
email:
|
137
|
+
version_requirements: *id011
|
138
|
+
description: Weeter subscribes to a set of twitter users or search terms using Twitter's streaming API, and notifies your app with each new tweet.
|
139
|
+
email:
|
140
140
|
- luke@lukemelia.com
|
141
|
-
executables:
|
141
|
+
executables:
|
142
142
|
- weeter
|
143
143
|
- weeter_control
|
144
144
|
extensions: []
|
145
|
+
|
145
146
|
extra_rdoc_files: []
|
146
|
-
|
147
|
+
|
148
|
+
files:
|
147
149
|
- .gitignore
|
150
|
+
- .travis.yml
|
148
151
|
- Gemfile
|
149
152
|
- LICENSE
|
150
153
|
- README.md
|
@@ -156,7 +159,9 @@ files:
|
|
156
159
|
- lib/weeter/cli.rb
|
157
160
|
- lib/weeter/configuration.rb
|
158
161
|
- lib/weeter/configuration/client_app_config.rb
|
162
|
+
- lib/weeter/configuration/limiter_config.rb
|
159
163
|
- lib/weeter/configuration/twitter_config.rb
|
164
|
+
- lib/weeter/limitator.rb
|
160
165
|
- lib/weeter/plugins.rb
|
161
166
|
- lib/weeter/plugins/lib/oauth_http.rb
|
162
167
|
- lib/weeter/plugins/lib/redis.rb
|
@@ -178,6 +183,7 @@ files:
|
|
178
183
|
- spec/weeter/configuration/client_app_config_spec.rb
|
179
184
|
- spec/weeter/configuration/twitter_config_spec.rb
|
180
185
|
- spec/weeter/configuration_spec.rb
|
186
|
+
- spec/weeter/limitator_spec.rb
|
181
187
|
- spec/weeter/plugins/notification_plugin_spec.rb
|
182
188
|
- spec/weeter/plugins/subscription/update_server_spec.rb
|
183
189
|
- spec/weeter/plugins/subscription_plugin_spec.rb
|
@@ -188,33 +194,37 @@ files:
|
|
188
194
|
- weeter.gemspec
|
189
195
|
homepage: http://github.com/lukemelia/weeter
|
190
196
|
licenses: []
|
197
|
+
|
191
198
|
post_install_message:
|
192
199
|
rdoc_options: []
|
193
|
-
|
200
|
+
|
201
|
+
require_paths:
|
194
202
|
- lib
|
195
|
-
required_ruby_version: !ruby/object:Gem::Requirement
|
203
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
196
204
|
none: false
|
197
|
-
requirements:
|
198
|
-
- -
|
199
|
-
- !ruby/object:Gem::Version
|
200
|
-
version:
|
201
|
-
required_rubygems_version: !ruby/object:Gem::Requirement
|
205
|
+
requirements:
|
206
|
+
- - ">="
|
207
|
+
- !ruby/object:Gem::Version
|
208
|
+
version: "0"
|
209
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
202
210
|
none: false
|
203
|
-
requirements:
|
204
|
-
- -
|
205
|
-
- !ruby/object:Gem::Version
|
206
|
-
version:
|
211
|
+
requirements:
|
212
|
+
- - ">="
|
213
|
+
- !ruby/object:Gem::Version
|
214
|
+
version: "0"
|
207
215
|
requirements: []
|
216
|
+
|
208
217
|
rubyforge_project: weeter
|
209
218
|
rubygems_version: 1.8.10
|
210
219
|
signing_key:
|
211
220
|
specification_version: 3
|
212
221
|
summary: Consume the Twitter stream and notify your app
|
213
|
-
test_files:
|
222
|
+
test_files:
|
214
223
|
- spec/spec_helper.rb
|
215
224
|
- spec/weeter/configuration/client_app_config_spec.rb
|
216
225
|
- spec/weeter/configuration/twitter_config_spec.rb
|
217
226
|
- spec/weeter/configuration_spec.rb
|
227
|
+
- spec/weeter/limitator_spec.rb
|
218
228
|
- spec/weeter/plugins/notification_plugin_spec.rb
|
219
229
|
- spec/weeter/plugins/subscription/update_server_spec.rb
|
220
230
|
- spec/weeter/plugins/subscription_plugin_spec.rb
|