ryespy 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.ruby-version +1 -1
- data/.travis.yml +2 -0
- data/CHANGELOG.md +9 -2
- data/README.md +149 -60
- data/bin/ryespy +23 -2
- data/lib/ryespy/app.rb +5 -2
- data/lib/ryespy/listener/amzn_s3.rb +1 -0
- data/lib/ryespy/listener/ftp.rb +2 -0
- data/lib/ryespy/listener/goog_cs.rb +1 -0
- data/lib/ryespy/listener/goog_drv.rb +80 -0
- data/lib/ryespy/listener/imap.rb +2 -0
- data/lib/ryespy/listener/rax_cf.rb +1 -0
- data/lib/ryespy/version.rb +1 -1
- data/ryespy.gemspec +4 -3
- data/test/ryespy/listener/goog_drv_test.rb +172 -0
- metadata +23 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a22c1520fc9edcc6b58d6f6225f2ea5d260f84bd
|
4
|
+
data.tar.gz: dba9c58d80ac2f2b6b1550e48fbf7b12b51cb20e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6b95c386e358f1647086668a23bb53c9af760440f8d2f2da36d931e3de6ac6a1cd0a31e8355dfcef3b38ae43559211114fcb76b0d7aa31d46006a8f8a3c34967
|
7
|
+
data.tar.gz: e6be1a6bcf162dd9896191c57509c4c329344232538ab047fe7d2fe185180f3742679ae5f83d4425cf28fc2cf761e3a6a66b8a01f03c682ba05bf75410384193
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
|
1
|
+
2.1.2
|
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -2,8 +2,15 @@
|
|
2
2
|
|
3
3
|
This changelog documents the main changes between released versions.
|
4
4
|
For a full list of changes, consult the commit history.
|
5
|
-
|
6
|
-
|
5
|
+
|
6
|
+
|
7
|
+
## 1.1.0
|
8
|
+
|
9
|
+
- start of support for Ruby 2.1.1
|
10
|
+
- start of support for Ruby 2.1.2
|
11
|
+
- improved `README` with workers examples
|
12
|
+
- [#1] new Google Drive listener (`--listener goog-drv`) (thank you, @Lewis-Clayton)
|
13
|
+
- extension of [#1] with changes detection and tests
|
7
14
|
|
8
15
|
|
9
16
|
## 1.0.0
|
data/README.md
CHANGED
@@ -4,109 +4,199 @@
|
|
4
4
|
[](https://travis-ci.org/tiredpixel/ryespy)
|
5
5
|
[](https://codeclimate.com/github/tiredpixel/ryespy)
|
6
6
|
|
7
|
-
[
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
FTP folders,
|
13
|
-
Amazon S3 buckets,
|
14
|
-
|
15
|
-
|
16
|
-
keeps track of what it's seen using [Redis](http://redis.io), and writes
|
17
|
-
Sidekiq/Resque-compatible payloads.
|
7
|
+
[Redis](http://redis.io) [Sidekiq](https://github.com/mperham/sidekiq)/
|
8
|
+
[Resque](https://github.com/resque/resque)
|
9
|
+
IMAP, FTP, Amazon S3, Google Cloud Storage, Google Drive, Rackspace Cloud Files
|
10
|
+
listener.
|
11
|
+
|
12
|
+
Ryespy provides an executable for listening to IMAP mailboxes, FTP folders,
|
13
|
+
Amazon S3 buckets, Google Cloud Storage buckets, Google Drive accounts, or
|
14
|
+
Rackspace Cloud Files containers, keeps track of what it's seen using
|
15
|
+
Redis, and writes Sidekiq/Resque-compatible payloads.
|
18
16
|
|
19
17
|
Ryespy was inspired by [Redimap](https://github.com/tiredpixel/redimap).
|
20
|
-
Yes, it's sometimes possible to inspire oneself.
|
21
|
-
Ryespy with my little eye.
|
18
|
+
Yes, it's sometimes possible to inspire oneself. Ryespy with my little eye.
|
22
19
|
|
23
20
|
More sleep lost by [tiredpixel](http://www.tiredpixel.com).
|
24
21
|
|
25
22
|
|
23
|
+
## Externals
|
24
|
+
|
25
|
+
- [Redis](http://redis.io)
|
26
|
+
|
27
|
+
|
26
28
|
## Installation
|
27
29
|
|
28
30
|
Install using:
|
29
31
|
|
30
|
-
|
32
|
+
gem install ryespy
|
31
33
|
|
32
|
-
|
34
|
+
Listener dependencies are required dynamically. That means that it may be
|
35
|
+
necessary to manually install the indicated gems if you are using that listener.
|
36
|
+
If you are not using that listener, the dependencies need not be installed.
|
33
37
|
|
34
|
-
|
38
|
+
### Amazon S3 Listener
|
39
|
+
|
40
|
+
gem install fog -v '~> 1.19'
|
35
41
|
|
36
|
-
|
42
|
+
### Google Cloud Storage Listener
|
37
43
|
|
38
|
-
|
39
|
-
|
40
|
-
$ gem install fog -v '~> 1.19'
|
44
|
+
gem install fog -v '~> 1.19'
|
41
45
|
|
42
|
-
|
43
|
-
|
44
|
-
$ gem install fog -v '~> 1.19'
|
46
|
+
### Google Drive Listener
|
45
47
|
|
46
|
-
|
47
|
-
|
48
|
-
|
48
|
+
gem install google_drive -v '~> 0.3'
|
49
|
+
|
50
|
+
### Rackspace Cloud Files Listener
|
51
|
+
|
52
|
+
gem install fog -v '~> 1.19'
|
53
|
+
|
54
|
+
The default Ruby version supported is defined in `.ruby-version`.
|
55
|
+
Any other versions supported are defined in `.travis.yml`.
|
49
56
|
|
50
57
|
|
51
58
|
## Usage
|
52
59
|
|
53
60
|
View the available options:
|
54
61
|
|
55
|
-
|
56
|
-
|
57
|
-
It is necessary to specify a listener and at least one notifier. Currently, the only notifier is `--notifier-sidekiq` (this must be specified).
|
62
|
+
ryespy --help
|
58
63
|
|
59
|
-
|
64
|
+
It is necessary to specify a listener and at least one notifier.
|
65
|
+
Currently, the only notifier is `--notifier-sidekiq`.
|
60
66
|
|
61
|
-
|
67
|
+
To run eternally, use `--eternal` (no need for Cron, but you can if you prefer).
|
62
68
|
|
63
69
|
|
64
70
|
### IMAP Listener
|
65
71
|
|
66
|
-
Check IMAP, queue new email UIDs, and quit
|
72
|
+
Check IMAP, queue new email UIDs, and quit:
|
73
|
+
|
74
|
+
ryespy --listener imap --imap-host mail.example.com --imap-username a@example.com --imap-password helpimacarrot --notifier-sidekiq
|
75
|
+
|
76
|
+
For non-SSL, use `--no-imap-ssl`.
|
77
|
+
For non-INBOX or multiple mailboxes, use `--imap-mailboxes INBOX,Sent`.
|
67
78
|
|
68
|
-
|
79
|
+
#### Example Worker
|
69
80
|
|
70
|
-
|
81
|
+
class RyespyIMAPJob
|
82
|
+
include Sidekiq::Worker
|
83
|
+
|
84
|
+
sidekiq_options :queue => :ryespy
|
85
|
+
|
86
|
+
def perform(mailbox, uid)
|
87
|
+
end
|
88
|
+
end
|
71
89
|
|
72
90
|
### FTP Listener
|
73
91
|
|
74
|
-
Check FTP, queue new file paths, and quit
|
92
|
+
Check FTP, queue new file paths, and quit:
|
93
|
+
|
94
|
+
ryespy --listener ftp --ftp-host ftp.example.com --ftp-username b@example.com --ftp-password helpimacucumber --notifier-sidekiq
|
75
95
|
|
76
|
-
|
96
|
+
For PASSIVE mode, use `--ftp-passive`.
|
97
|
+
For non-root or multiple directories, use `--ftp-dirs /DIR1,/DIR2`.
|
77
98
|
|
78
|
-
|
99
|
+
#### Example Worker
|
100
|
+
|
101
|
+
class RyespyFTPJob
|
102
|
+
include Sidekiq::Worker
|
103
|
+
|
104
|
+
sidekiq_options :queue => :ryespy
|
105
|
+
|
106
|
+
def perform(dir, filename)
|
107
|
+
end
|
108
|
+
end
|
79
109
|
|
80
110
|
### Amazon S3 Listener
|
81
111
|
|
82
|
-
Check Amazon S3, queue new file keys, and quit
|
112
|
+
Check Amazon S3, queue new file keys, and quit:
|
113
|
+
|
114
|
+
ryespy --listener amzn-s3 --amzn-s3-access-key c/example/com --amzn-s3-secret-key helpimabroccoli --amzn-s3-bucket vegetable-box --notifier-sidekiq
|
83
115
|
|
84
|
-
|
116
|
+
For non-* or multiple key prefix filters,
|
117
|
+
use `--amzn-s3-prefixes virtual-dir1/,virtual-dir`.
|
85
118
|
|
86
|
-
|
119
|
+
#### Example Worker
|
120
|
+
|
121
|
+
class RyespyAmznS3Job
|
122
|
+
include Sidekiq::Worker
|
123
|
+
|
124
|
+
sidekiq_options :queue => :ryespy
|
125
|
+
|
126
|
+
def perform(filename)
|
127
|
+
end
|
128
|
+
end
|
87
129
|
|
88
130
|
### Google Cloud Storage Listener
|
89
131
|
|
90
|
-
Check Google Cloud Storage, queue new file keys, and quit
|
132
|
+
Check Google Cloud Storage, queue new file keys, and quit:
|
133
|
+
|
134
|
+
ryespy --listener goog-cs --goog-cs-access-key d/example/com --goog-cs-secret-key helpimanasparagus --goog-cs-bucket vegetable-box --notifier-sidekiq
|
135
|
+
|
136
|
+
For non-* or multiple key prefix filters,
|
137
|
+
use `--goog-cs-prefixes virtual-dir1/,virtual-dir`.
|
138
|
+
|
139
|
+
#### Example Worker
|
140
|
+
|
141
|
+
class RyespyGoogCSJob
|
142
|
+
include Sidekiq::Worker
|
143
|
+
|
144
|
+
sidekiq_options :queue => :ryespy
|
145
|
+
|
146
|
+
def perform(filename)
|
147
|
+
end
|
148
|
+
end
|
149
|
+
|
150
|
+
### Google Drive Listener
|
151
|
+
|
152
|
+
Check Google Drive, queue new file resource ids, and quit:
|
153
|
+
|
154
|
+
ryespy --listener goog-drv --goog-drv-username vegetable-box@gmail.com --goog-drv-password helpimgarlic --goog-drv-filters vegetable-box --notifier-sidekiq
|
91
155
|
|
92
|
-
|
156
|
+
#### Example Worker
|
93
157
|
|
94
|
-
|
158
|
+
class RyespyGoogDrvJob
|
159
|
+
include Sidekiq::Worker
|
160
|
+
|
161
|
+
sidekiq_options :queue => :ryespy
|
162
|
+
|
163
|
+
def perform(resource_id)
|
164
|
+
end
|
165
|
+
end
|
95
166
|
|
96
167
|
### Rackspace Cloud Files Listener
|
97
168
|
|
98
|
-
Check Rackspace Cloud Files, queue new file keys, and quit
|
169
|
+
Check Rackspace Cloud Files, queue new file keys, and quit:
|
170
|
+
|
171
|
+
ryespy --listener rax-cf --rax-cf-username vegetable --rax-cf-api-key helpimacelery --rax-cf-container vegetable-box --notifier-sidekiq
|
99
172
|
|
100
|
-
|
173
|
+
For non-DFW region, use `--rax-cf-region lon`.
|
174
|
+
For non-US auth endpoint, use `--rax-cf-endpoint uk`.
|
175
|
+
Is your Rackspace account in London? Fret not; combine these and
|
176
|
+
use `--rax-cf-endpoint uk --rax-cf-region lon`.
|
177
|
+
For non-* or multiple key prefix filters,
|
178
|
+
use `--rax-cf-prefixes virtual-dir1/,virtual-dir`.
|
101
179
|
|
102
|
-
|
180
|
+
#### Example Worker
|
181
|
+
|
182
|
+
class RyespyRaxCFJob
|
183
|
+
include Sidekiq::Worker
|
184
|
+
|
185
|
+
sidekiq_options :queue => :ryespy
|
186
|
+
|
187
|
+
def perform(filename)
|
188
|
+
end
|
189
|
+
end
|
103
190
|
|
104
191
|
|
105
192
|
## Advanced Usage
|
106
193
|
|
107
|
-
If you want to do something rather more magical
|
194
|
+
If you want to do something rather more magical such as checking multiple
|
195
|
+
accounts for a listener or even multiple listeners, then you may wish to use the
|
196
|
+
Ryespy library directly instead of the `ryespy` executable.
|
108
197
|
|
109
|
-
Depend upon the `ryespy` gem in a `Gemfile`, remembering to add any manual
|
198
|
+
Depend upon the `ryespy` gem in a `Gemfile`, remembering to add any manual
|
199
|
+
dependencies for listeners as detailed in [Installation](#installation):
|
110
200
|
|
111
201
|
# Gemfile
|
112
202
|
|
@@ -134,7 +224,10 @@ Create the notifiers:
|
|
134
224
|
:namespace => 'resque'
|
135
225
|
)
|
136
226
|
|
137
|
-
For each listener, configure like in `ryespy --help` but without the
|
227
|
+
For each listener, configure like in `ryespy --help` but without the prefix and
|
228
|
+
with `-` changed to `_` (e.g. `--amzn-s3-access-key` => `:access_key`). Pass in
|
229
|
+
an array of notifiers. Note that the `check()` argument varies per listener,
|
230
|
+
meaning IMAP mailbox, FTP directory, Google Drive filter, or storage key prefix.
|
138
231
|
|
139
232
|
require 'ryespy/listener/amzn_s3'
|
140
233
|
|
@@ -162,15 +255,6 @@ You can also become a [watcher](https://github.com/tiredpixel/ryespy/watchers)
|
|
162
255
|
on GitHub. And don't forget you can become a [stargazer](https://github.com/tiredpixel/ryespy/stargazers) if you are so minded. :D
|
163
256
|
|
164
257
|
|
165
|
-
## Growing Like Flowers
|
166
|
-
|
167
|
-
Dear Me, Here is a vague wishlist:
|
168
|
-
|
169
|
-
- Additional notifiers (e.g. URL ?)
|
170
|
-
|
171
|
-
Also take a look at the [issue tracker](https://github.com/tiredpixel/ryespy/issues).
|
172
|
-
|
173
|
-
|
174
258
|
## Contributions
|
175
259
|
|
176
260
|
Contributions are embraced with much love and affection!
|
@@ -187,7 +271,11 @@ which is included by default in Ruby 1.9 onwards. To run all tests:
|
|
187
271
|
|
188
272
|
rake test
|
189
273
|
|
190
|
-
When using the `ryespy` executable in development, you'll probably want to set
|
274
|
+
When using the `ryespy` executable in development, you'll probably want to set
|
275
|
+
`--debug` mode so debug-level messages are logged and stack traces raised.
|
276
|
+
|
277
|
+
We render grateful thanks unto our
|
278
|
+
[contributors](https://github.com/tiredpixel/ryespy/graphs/contributors).
|
191
279
|
|
192
280
|
|
193
281
|
## Blessing
|
@@ -197,6 +285,7 @@ May you find peace, and help others to do likewise.
|
|
197
285
|
|
198
286
|
## Licence
|
199
287
|
|
200
|
-
© [tiredpixel](http://www.tiredpixel.com)
|
288
|
+
© [tiredpixel](http://www.tiredpixel.com) 2014 and
|
289
|
+
[others](https://github.com/tiredpixel/ryespy/graphs/contributors).
|
201
290
|
It is free software, released under the MIT License, and may be redistributed
|
202
291
|
under the terms specified in `LICENSE`.
|
data/bin/ryespy
CHANGED
@@ -31,8 +31,9 @@ OptionParser.new do |opts|
|
|
31
31
|
'ftp',
|
32
32
|
'amzn-s3',
|
33
33
|
'goog-cs',
|
34
|
+
'goog-drv',
|
34
35
|
'rax-cf',
|
35
|
-
], "Listener (imap|ftp|amzn-s3|goog-cs|rax-cf)") do |o|
|
36
|
+
], "Listener (imap|ftp|amzn-s3|goog-cs|goog-drv|rax-cf)") do |o|
|
36
37
|
options[:listener] = o.tr('-', '_').to_sym
|
37
38
|
end
|
38
39
|
|
@@ -149,7 +150,22 @@ OptionParser.new do |opts|
|
|
149
150
|
opts.on("--goog-cs-prefixes [P1,P2]", Array, "Prefixes P1,P2 (default: *)") do |o|
|
150
151
|
options[:goog_cs_filters] = o
|
151
152
|
end
|
152
|
-
|
153
|
+
|
154
|
+
opts.separator ""
|
155
|
+
opts.separator "--listener goog-drv (Google Drive) :"
|
156
|
+
|
157
|
+
opts.on("--goog-drv-username USERNAME", "USERNAME") do |o|
|
158
|
+
options[:goog_drv_username] = o
|
159
|
+
end
|
160
|
+
|
161
|
+
opts.on("--goog-drv-password PASSWORD", "PASSWORD") do |o|
|
162
|
+
options[:goog_drv_password] = o
|
163
|
+
end
|
164
|
+
|
165
|
+
opts.on("--goog-drv-filters [F1,F2]", Array, "Filters F1,F2 (default: *)") do |o|
|
166
|
+
options[:goog_drv_filters] = o
|
167
|
+
end
|
168
|
+
|
153
169
|
opts.separator ""
|
154
170
|
opts.separator "--listener rax-cf (Rackspace Cloud Files) :"
|
155
171
|
|
@@ -270,6 +286,11 @@ end
|
|
270
286
|
:bucket,
|
271
287
|
:filters, # prefixes
|
272
288
|
],
|
289
|
+
:goog_drv => [
|
290
|
+
:username,
|
291
|
+
:password,
|
292
|
+
:filters,
|
293
|
+
],
|
273
294
|
:rax_cf => [
|
274
295
|
:endpoint,
|
275
296
|
:region,
|
data/lib/ryespy/app.rb
CHANGED
@@ -31,6 +31,9 @@ module Ryespy
|
|
31
31
|
:goog_cs => {
|
32
32
|
:filters => [''], # prefixes
|
33
33
|
},
|
34
|
+
:goog_drv => {
|
35
|
+
:filters => [''],
|
36
|
+
},
|
34
37
|
:rax_cf => {
|
35
38
|
:endpoint => :us,
|
36
39
|
:region => :dfw,
|
@@ -68,7 +71,6 @@ module Ryespy
|
|
68
71
|
def notifiers
|
69
72
|
unless @notifiers
|
70
73
|
@notifiers = []
|
71
|
-
|
72
74
|
@config.notifiers[:sidekiq].each do |notifier_url|
|
73
75
|
@notifiers << Notifier::Sidekiq.new(
|
74
76
|
:url => notifier_url,
|
@@ -118,7 +120,7 @@ module Ryespy
|
|
118
120
|
check_all
|
119
121
|
rescue StandardError => e
|
120
122
|
@logger.error { e.to_s }
|
121
|
-
|
123
|
+
|
122
124
|
raise if @config.log_level == :DEBUG
|
123
125
|
end
|
124
126
|
|
@@ -140,6 +142,7 @@ module Ryespy
|
|
140
142
|
:ftp => :FTP,
|
141
143
|
:amzn_s3 => :AmznS3,
|
142
144
|
:goog_cs => :GoogCS,
|
145
|
+
:goog_drv => :GoogDrv,
|
143
146
|
:rax_cf => :RaxCF,
|
144
147
|
}
|
145
148
|
|
data/lib/ryespy/listener/ftp.rb
CHANGED
@@ -0,0 +1,80 @@
|
|
1
|
+
require "google_drive"
|
2
|
+
|
3
|
+
require_relative 'base'
|
4
|
+
|
5
|
+
|
6
|
+
module Ryespy
|
7
|
+
module Listener
|
8
|
+
class GoogDrv < Base
|
9
|
+
|
10
|
+
REDIS_KEY_PREFIX = 'goog_drv'.freeze
|
11
|
+
SIDEKIQ_JOB_CLASS = 'RyespyGoogDrvJob'.freeze
|
12
|
+
|
13
|
+
def initialize(opts = {})
|
14
|
+
@config = {
|
15
|
+
:username => opts[:username],
|
16
|
+
:password => opts[:password],
|
17
|
+
}
|
18
|
+
|
19
|
+
super(opts)
|
20
|
+
end
|
21
|
+
|
22
|
+
def check(filter)
|
23
|
+
@logger.debug { "filter: #{filter}" }
|
24
|
+
|
25
|
+
@logger.debug { "redis_key: #{redis_key}" }
|
26
|
+
|
27
|
+
seen_files = @redis.hgetall(redis_key)
|
28
|
+
|
29
|
+
unseen_files = get_unseen_files(filter, seen_files)
|
30
|
+
|
31
|
+
@logger.debug { "unseen_files: #{unseen_files}" }
|
32
|
+
|
33
|
+
unseen_files.each do |key, val|
|
34
|
+
@redis.hset(redis_key, key, val)
|
35
|
+
|
36
|
+
@notifiers.each { |n| n.notify(SIDEKIQ_JOB_CLASS, [key]) }
|
37
|
+
end
|
38
|
+
|
39
|
+
@logger.info { "#{filter} has #{unseen_files.count} new files" }
|
40
|
+
end
|
41
|
+
|
42
|
+
private
|
43
|
+
|
44
|
+
def connect_service
|
45
|
+
@google_drive = GoogleDrive.login(@config[:username], @config[:password])
|
46
|
+
end
|
47
|
+
|
48
|
+
def redis_key
|
49
|
+
[
|
50
|
+
REDIS_KEY_PREFIX,
|
51
|
+
@config[:username]
|
52
|
+
].join(':')
|
53
|
+
end
|
54
|
+
|
55
|
+
def get_unseen_files(filter, seen_files)
|
56
|
+
files = {}
|
57
|
+
|
58
|
+
@google_drive.files.each do |file|
|
59
|
+
next unless file.title =~ /#{filter}/ && file.resource_id && file.resource_type != 'folder'
|
60
|
+
|
61
|
+
# updated should be present for all resource_type , but there is often
|
62
|
+
# a delay until it is set.
|
63
|
+
updated = file.document_feed_entry.css('updated').first.text
|
64
|
+
|
65
|
+
# etag is present for most resource_type , but is nil for spreadsheet.
|
66
|
+
etag = file.document_feed_entry.attribute('etag')
|
67
|
+
|
68
|
+
checksum = "#{updated},#{etag}"
|
69
|
+
|
70
|
+
if seen_files[file.resource_id] != checksum
|
71
|
+
files[file.resource_id] = checksum
|
72
|
+
end
|
73
|
+
end
|
74
|
+
|
75
|
+
files
|
76
|
+
end
|
77
|
+
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
data/lib/ryespy/listener/imap.rb
CHANGED
data/lib/ryespy/version.rb
CHANGED
data/ryespy.gemspec
CHANGED
@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
|
|
8
8
|
spec.version = Ryespy::VERSION
|
9
9
|
spec.authors = ["tiredpixel"]
|
10
10
|
spec.email = ["tp@tiredpixel.com"]
|
11
|
-
spec.
|
12
|
-
spec.
|
11
|
+
spec.summary = %q{Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage, Google Drive, Rackspace Cloud Files listener.}
|
12
|
+
spec.description = %q{}
|
13
13
|
spec.homepage = "https://github.com/tiredpixel/ryespy"
|
14
14
|
spec.license = "MIT"
|
15
15
|
|
@@ -25,5 +25,6 @@ Gem::Specification.new do |spec|
|
|
25
25
|
spec.add_development_dependency "rake"
|
26
26
|
spec.add_development_dependency "fog", "~> 1.19" # conditional dependency
|
27
27
|
spec.add_development_dependency "mocha", "~> 0.14"
|
28
|
-
spec.add_development_dependency "sidekiq-spy", "
|
28
|
+
spec.add_development_dependency "sidekiq-spy", ">= 0.3.2"
|
29
|
+
spec.add_development_dependency "google_drive", '~> 0.3'
|
29
30
|
end
|
@@ -0,0 +1,172 @@
|
|
1
|
+
require_relative '../../helper'
|
2
|
+
|
3
|
+
require_relative '../../../lib/ryespy/listener/goog_drv'
|
4
|
+
|
5
|
+
|
6
|
+
describe Ryespy::Listener::GoogDrv do
|
7
|
+
|
8
|
+
before do
|
9
|
+
updated = '1897-05-26 00:00:00 UTC'
|
10
|
+
|
11
|
+
dfe = stub
|
12
|
+
dfe.stubs(:css).with('updated').returns([stub(:text => updated)])
|
13
|
+
dfe.stubs(:attribute).with('etag').returns('E13l2izwZbgWKfS5')
|
14
|
+
|
15
|
+
dfe2 = stub
|
16
|
+
dfe2.stubs(:css).with('updated').returns([stub(:text => updated)])
|
17
|
+
dfe2.stubs(:attribute).with('etag').returns(nil)
|
18
|
+
|
19
|
+
@files = [
|
20
|
+
stub(
|
21
|
+
:resource_id => 'folder:FFTSKJtqo5v2TRrIADAmVOlsqe8h9TqM',
|
22
|
+
:resource_type => 'folder',
|
23
|
+
:title => 'Flies/',
|
24
|
+
:document_feed_entry => dfe,
|
25
|
+
),
|
26
|
+
stub(
|
27
|
+
:resource_id => 'presentation:SuOQSdIwu3E58fq7HsbtfjG2aeuTDtOJ',
|
28
|
+
:resource_type => 'presentation',
|
29
|
+
:title => 'Flies Presentation',
|
30
|
+
:document_feed_entry => dfe,
|
31
|
+
),
|
32
|
+
stub(
|
33
|
+
:resource_id => 'spreadsheet:GfVKovEtmvWtMgWDTv94kcYZLbje1O3q',
|
34
|
+
:resource_type => 'spreadsheet',
|
35
|
+
:title => 'Flies Spreadsheet',
|
36
|
+
:document_feed_entry => dfe2,
|
37
|
+
),
|
38
|
+
stub(
|
39
|
+
:resource_id => 'drawing:ceKiP0rwNufPU3qguhleVIxmtXqMjaWe',
|
40
|
+
:resource_type => 'drawing',
|
41
|
+
:title => 'FliesBuzz Drawing',
|
42
|
+
:document_feed_entry => dfe,
|
43
|
+
),
|
44
|
+
stub(
|
45
|
+
:resource_id => 'form:OCBeiWQf51qvmPdWVT8olFRqTM8pl5sC',
|
46
|
+
:resource_type => 'form',
|
47
|
+
:title => 'Spiders and Flies Form',
|
48
|
+
:document_feed_entry => dfe,
|
49
|
+
),
|
50
|
+
stub(
|
51
|
+
:resource_id => 'document:gvZHG5YA5rkcgIXGUnDXhMOI0qE7WKnm',
|
52
|
+
:resource_type => 'document',
|
53
|
+
:title => 'Spiders Document',
|
54
|
+
:document_feed_entry => dfe,
|
55
|
+
),
|
56
|
+
stub(
|
57
|
+
:resource_id => 'file:i7iqdRt3CxbJEWuo8kXIFBe9WeTnQRL3',
|
58
|
+
:resource_type => 'file',
|
59
|
+
:title => 'Spiders File',
|
60
|
+
:document_feed_entry => dfe,
|
61
|
+
),
|
62
|
+
]
|
63
|
+
|
64
|
+
@files_no_dirs = @files.select { |f| f.resource_type != 'folder' }
|
65
|
+
|
66
|
+
@google_drive = stub(
|
67
|
+
:files => @files
|
68
|
+
)
|
69
|
+
|
70
|
+
GoogleDrive.stubs(:login).with('r.m.renfield', 'master').returns(@google_drive)
|
71
|
+
end
|
72
|
+
|
73
|
+
describe "#check" do
|
74
|
+
before do
|
75
|
+
Ryespy::Test::Redis::setup
|
76
|
+
|
77
|
+
@notifier = mock()
|
78
|
+
|
79
|
+
@goog_drv = Ryespy::Listener::GoogDrv.new(
|
80
|
+
:username => 'r.m.renfield',
|
81
|
+
:password => 'master',
|
82
|
+
:notifiers => [@notifier],
|
83
|
+
)
|
84
|
+
|
85
|
+
@redis = @goog_drv.instance_variable_get(:@redis)
|
86
|
+
end
|
87
|
+
|
88
|
+
after do
|
89
|
+
@goog_drv.close
|
90
|
+
|
91
|
+
Ryespy::Test::Redis::flush_namespace(@redis)
|
92
|
+
end
|
93
|
+
|
94
|
+
it "notifies when new files filter *" do
|
95
|
+
@files_no_dirs.each do |file|
|
96
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', [file.resource_id]).once
|
97
|
+
end
|
98
|
+
|
99
|
+
@goog_drv.check('')
|
100
|
+
end
|
101
|
+
|
102
|
+
it "notifies when new files filter ^Flies\\b" do
|
103
|
+
@files_no_dirs.select { |f| f.title =~ /^Flies\b/ }.each do |file|
|
104
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', [file.resource_id]).once
|
105
|
+
end
|
106
|
+
|
107
|
+
@goog_drv.check('^Flies\b')
|
108
|
+
end
|
109
|
+
|
110
|
+
it "notifies when new files filter Flies" do
|
111
|
+
@files_no_dirs.select { |f| f.title =~ /Flies/ }.each do |file|
|
112
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', [file.resource_id]).once
|
113
|
+
end
|
114
|
+
|
115
|
+
@goog_drv.check('Flies')
|
116
|
+
end
|
117
|
+
|
118
|
+
it "doesn't notify when no new files" do
|
119
|
+
@notifier.expects(:notify).times(3)
|
120
|
+
|
121
|
+
@goog_drv.check('Spiders')
|
122
|
+
@goog_drv.check('Spiders')
|
123
|
+
end
|
124
|
+
|
125
|
+
it "doesn't notify when no new files filter subset" do
|
126
|
+
@notifier.expects(:notify).times(3)
|
127
|
+
|
128
|
+
@goog_drv.check('Spiders')
|
129
|
+
@goog_drv.check('Spiders\ and\ Flies')
|
130
|
+
end
|
131
|
+
|
132
|
+
it "notifies when new files filter distinct" do
|
133
|
+
@notifier.expects(:notify).times(4)
|
134
|
+
|
135
|
+
@goog_drv.check('Spiders\ and\ Flies')
|
136
|
+
@goog_drv.check('Flies')
|
137
|
+
end
|
138
|
+
|
139
|
+
it "notifies when changed updated" do
|
140
|
+
dfe = stub
|
141
|
+
dfe.stubs(:css).with('updated').returns([stub(:text => '1899-03-23 01:00:00 UTC')])
|
142
|
+
dfe.stubs(:attribute).with('etag').returns('E13l2izwZbgWKfS5')
|
143
|
+
|
144
|
+
@notifier.expects(:notify).times(4)
|
145
|
+
|
146
|
+
@goog_drv.check('Flies')
|
147
|
+
|
148
|
+
@files[1].stubs(:document_feed_entry).returns(dfe)
|
149
|
+
|
150
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', ['presentation:SuOQSdIwu3E58fq7HsbtfjG2aeuTDtOJ']).once
|
151
|
+
|
152
|
+
@goog_drv.check('Flies')
|
153
|
+
end
|
154
|
+
|
155
|
+
it "notifies when changed etag" do
|
156
|
+
dfe = stub
|
157
|
+
dfe.stubs(:css).with('updated').returns([stub(:text => '1897-05-26 00:00:00 UTC')])
|
158
|
+
dfe.stubs(:attribute).with('etag').returns('qGE4UWIcITePyidC')
|
159
|
+
|
160
|
+
@notifier.expects(:notify).times(4)
|
161
|
+
|
162
|
+
@goog_drv.check('Flies')
|
163
|
+
|
164
|
+
@files[3].stubs(:document_feed_entry).returns(dfe)
|
165
|
+
|
166
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', ['drawing:ceKiP0rwNufPU3qguhleVIxmtXqMjaWe']).once
|
167
|
+
|
168
|
+
@goog_drv.check('Flies')
|
169
|
+
end
|
170
|
+
end
|
171
|
+
|
172
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: ryespy
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- tiredpixel
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-06-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: redis
|
@@ -102,6 +102,20 @@ dependencies:
|
|
102
102
|
version: '0.14'
|
103
103
|
- !ruby/object:Gem::Dependency
|
104
104
|
name: sidekiq-spy
|
105
|
+
requirement: !ruby/object:Gem::Requirement
|
106
|
+
requirements:
|
107
|
+
- - ">="
|
108
|
+
- !ruby/object:Gem::Version
|
109
|
+
version: 0.3.2
|
110
|
+
type: :development
|
111
|
+
prerelease: false
|
112
|
+
version_requirements: !ruby/object:Gem::Requirement
|
113
|
+
requirements:
|
114
|
+
- - ">="
|
115
|
+
- !ruby/object:Gem::Version
|
116
|
+
version: 0.3.2
|
117
|
+
- !ruby/object:Gem::Dependency
|
118
|
+
name: google_drive
|
105
119
|
requirement: !ruby/object:Gem::Requirement
|
106
120
|
requirements:
|
107
121
|
- - "~>"
|
@@ -114,8 +128,7 @@ dependencies:
|
|
114
128
|
- - "~>"
|
115
129
|
- !ruby/object:Gem::Version
|
116
130
|
version: '0.3'
|
117
|
-
description:
|
118
|
-
Cloud Files listener.
|
131
|
+
description: ''
|
119
132
|
email:
|
120
133
|
- tp@tiredpixel.com
|
121
134
|
executables:
|
@@ -140,6 +153,7 @@ files:
|
|
140
153
|
- lib/ryespy/listener/fogable.rb
|
141
154
|
- lib/ryespy/listener/ftp.rb
|
142
155
|
- lib/ryespy/listener/goog_cs.rb
|
156
|
+
- lib/ryespy/listener/goog_drv.rb
|
143
157
|
- lib/ryespy/listener/imap.rb
|
144
158
|
- lib/ryespy/listener/rax_cf.rb
|
145
159
|
- lib/ryespy/notifier/sidekiq.rb
|
@@ -150,6 +164,7 @@ files:
|
|
150
164
|
- test/ryespy/listener/amzn_s3_test.rb
|
151
165
|
- test/ryespy/listener/ftp_test.rb
|
152
166
|
- test/ryespy/listener/goog_cs_test.rb
|
167
|
+
- test/ryespy/listener/goog_drv_test.rb
|
153
168
|
- test/ryespy/listener/imap_test.rb
|
154
169
|
- test/ryespy/listener/rax_cf_test.rb
|
155
170
|
- test/ryespy/notifier/sidekiq_test.rb
|
@@ -174,17 +189,18 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
174
189
|
version: '0'
|
175
190
|
requirements: []
|
176
191
|
rubyforge_project:
|
177
|
-
rubygems_version: 2.2.
|
192
|
+
rubygems_version: 2.2.2
|
178
193
|
signing_key:
|
179
194
|
specification_version: 4
|
180
|
-
summary: Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage,
|
181
|
-
Cloud Files listener.
|
195
|
+
summary: Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage, Google Drive,
|
196
|
+
Rackspace Cloud Files listener.
|
182
197
|
test_files:
|
183
198
|
- test/helper.rb
|
184
199
|
- test/ryespy/app_test.rb
|
185
200
|
- test/ryespy/listener/amzn_s3_test.rb
|
186
201
|
- test/ryespy/listener/ftp_test.rb
|
187
202
|
- test/ryespy/listener/goog_cs_test.rb
|
203
|
+
- test/ryespy/listener/goog_drv_test.rb
|
188
204
|
- test/ryespy/listener/imap_test.rb
|
189
205
|
- test/ryespy/listener/rax_cf_test.rb
|
190
206
|
- test/ryespy/notifier/sidekiq_test.rb
|