ryespy 1.0.0 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.ruby-version +1 -1
- data/.travis.yml +2 -0
- data/CHANGELOG.md +9 -2
- data/README.md +149 -60
- data/bin/ryespy +23 -2
- data/lib/ryespy/app.rb +5 -2
- data/lib/ryespy/listener/amzn_s3.rb +1 -0
- data/lib/ryespy/listener/ftp.rb +2 -0
- data/lib/ryespy/listener/goog_cs.rb +1 -0
- data/lib/ryespy/listener/goog_drv.rb +80 -0
- data/lib/ryespy/listener/imap.rb +2 -0
- data/lib/ryespy/listener/rax_cf.rb +1 -0
- data/lib/ryespy/version.rb +1 -1
- data/ryespy.gemspec +4 -3
- data/test/ryespy/listener/goog_drv_test.rb +172 -0
- metadata +23 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a22c1520fc9edcc6b58d6f6225f2ea5d260f84bd
|
4
|
+
data.tar.gz: dba9c58d80ac2f2b6b1550e48fbf7b12b51cb20e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6b95c386e358f1647086668a23bb53c9af760440f8d2f2da36d931e3de6ac6a1cd0a31e8355dfcef3b38ae43559211114fcb76b0d7aa31d46006a8f8a3c34967
|
7
|
+
data.tar.gz: e6be1a6bcf162dd9896191c57509c4c329344232538ab047fe7d2fe185180f3742679ae5f83d4425cf28fc2cf761e3a6a66b8a01f03c682ba05bf75410384193
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
|
1
|
+
2.1.2
|
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -2,8 +2,15 @@
|
|
2
2
|
|
3
3
|
This changelog documents the main changes between released versions.
|
4
4
|
For a full list of changes, consult the commit history.
|
5
|
-
|
6
|
-
|
5
|
+
|
6
|
+
|
7
|
+
## 1.1.0
|
8
|
+
|
9
|
+
- start of support for Ruby 2.1.1
|
10
|
+
- start of support for Ruby 2.1.2
|
11
|
+
- improved `README` with workers examples
|
12
|
+
- [#1] new Google Drive listener (`--listener goog-drv`) (thank you, @Lewis-Clayton)
|
13
|
+
- extension of [#1] with changes detection and tests
|
7
14
|
|
8
15
|
|
9
16
|
## 1.0.0
|
data/README.md
CHANGED
@@ -4,109 +4,199 @@
|
|
4
4
|
[![Build Status](https://travis-ci.org/tiredpixel/ryespy.png?branch=master,stable)](https://travis-ci.org/tiredpixel/ryespy)
|
5
5
|
[![Code Climate](https://codeclimate.com/github/tiredpixel/ryespy.png)](https://codeclimate.com/github/tiredpixel/ryespy)
|
6
6
|
|
7
|
-
[
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
FTP folders,
|
13
|
-
Amazon S3 buckets,
|
14
|
-
|
15
|
-
|
16
|
-
keeps track of what it's seen using [Redis](http://redis.io), and writes
|
17
|
-
Sidekiq/Resque-compatible payloads.
|
7
|
+
[Redis](http://redis.io) [Sidekiq](https://github.com/mperham/sidekiq)/
|
8
|
+
[Resque](https://github.com/resque/resque)
|
9
|
+
IMAP, FTP, Amazon S3, Google Cloud Storage, Google Drive, Rackspace Cloud Files
|
10
|
+
listener.
|
11
|
+
|
12
|
+
Ryespy provides an executable for listening to IMAP mailboxes, FTP folders,
|
13
|
+
Amazon S3 buckets, Google Cloud Storage buckets, Google Drive accounts, or
|
14
|
+
Rackspace Cloud Files containers, keeps track of what it's seen using
|
15
|
+
Redis, and writes Sidekiq/Resque-compatible payloads.
|
18
16
|
|
19
17
|
Ryespy was inspired by [Redimap](https://github.com/tiredpixel/redimap).
|
20
|
-
Yes, it's sometimes possible to inspire oneself.
|
21
|
-
Ryespy with my little eye.
|
18
|
+
Yes, it's sometimes possible to inspire oneself. Ryespy with my little eye.
|
22
19
|
|
23
20
|
More sleep lost by [tiredpixel](http://www.tiredpixel.com).
|
24
21
|
|
25
22
|
|
23
|
+
## Externals
|
24
|
+
|
25
|
+
- [Redis](http://redis.io)
|
26
|
+
|
27
|
+
|
26
28
|
## Installation
|
27
29
|
|
28
30
|
Install using:
|
29
31
|
|
30
|
-
|
32
|
+
gem install ryespy
|
31
33
|
|
32
|
-
|
34
|
+
Listener dependencies are required dynamically. That means that it may be
|
35
|
+
necessary to manually install the indicated gems if you are using that listener.
|
36
|
+
If you are not using that listener, the dependencies need not be installed.
|
33
37
|
|
34
|
-
|
38
|
+
### Amazon S3 Listener
|
39
|
+
|
40
|
+
gem install fog -v '~> 1.19'
|
35
41
|
|
36
|
-
|
42
|
+
### Google Cloud Storage Listener
|
37
43
|
|
38
|
-
|
39
|
-
|
40
|
-
$ gem install fog -v '~> 1.19'
|
44
|
+
gem install fog -v '~> 1.19'
|
41
45
|
|
42
|
-
|
43
|
-
|
44
|
-
$ gem install fog -v '~> 1.19'
|
46
|
+
### Google Drive Listener
|
45
47
|
|
46
|
-
|
47
|
-
|
48
|
-
|
48
|
+
gem install google_drive -v '~> 0.3'
|
49
|
+
|
50
|
+
### Rackspace Cloud Files Listener
|
51
|
+
|
52
|
+
gem install fog -v '~> 1.19'
|
53
|
+
|
54
|
+
The default Ruby version supported is defined in `.ruby-version`.
|
55
|
+
Any other versions supported are defined in `.travis.yml`.
|
49
56
|
|
50
57
|
|
51
58
|
## Usage
|
52
59
|
|
53
60
|
View the available options:
|
54
61
|
|
55
|
-
|
56
|
-
|
57
|
-
It is necessary to specify a listener and at least one notifier. Currently, the only notifier is `--notifier-sidekiq` (this must be specified).
|
62
|
+
ryespy --help
|
58
63
|
|
59
|
-
|
64
|
+
It is necessary to specify a listener and at least one notifier.
|
65
|
+
Currently, the only notifier is `--notifier-sidekiq`.
|
60
66
|
|
61
|
-
|
67
|
+
To run eternally, use `--eternal` (no need for Cron, but you can if you prefer).
|
62
68
|
|
63
69
|
|
64
70
|
### IMAP Listener
|
65
71
|
|
66
|
-
Check IMAP, queue new email UIDs, and quit
|
72
|
+
Check IMAP, queue new email UIDs, and quit:
|
73
|
+
|
74
|
+
ryespy --listener imap --imap-host mail.example.com --imap-username a@example.com --imap-password helpimacarrot --notifier-sidekiq
|
75
|
+
|
76
|
+
For non-SSL, use `--no-imap-ssl`.
|
77
|
+
For non-INBOX or multiple mailboxes, use `--imap-mailboxes INBOX,Sent`.
|
67
78
|
|
68
|
-
|
79
|
+
#### Example Worker
|
69
80
|
|
70
|
-
|
81
|
+
class RyespyIMAPJob
|
82
|
+
include Sidekiq::Worker
|
83
|
+
|
84
|
+
sidekiq_options :queue => :ryespy
|
85
|
+
|
86
|
+
def perform(mailbox, uid)
|
87
|
+
end
|
88
|
+
end
|
71
89
|
|
72
90
|
### FTP Listener
|
73
91
|
|
74
|
-
Check FTP, queue new file paths, and quit
|
92
|
+
Check FTP, queue new file paths, and quit:
|
93
|
+
|
94
|
+
ryespy --listener ftp --ftp-host ftp.example.com --ftp-username b@example.com --ftp-password helpimacucumber --notifier-sidekiq
|
75
95
|
|
76
|
-
|
96
|
+
For PASSIVE mode, use `--ftp-passive`.
|
97
|
+
For non-root or multiple directories, use `--ftp-dirs /DIR1,/DIR2`.
|
77
98
|
|
78
|
-
|
99
|
+
#### Example Worker
|
100
|
+
|
101
|
+
class RyespyFTPJob
|
102
|
+
include Sidekiq::Worker
|
103
|
+
|
104
|
+
sidekiq_options :queue => :ryespy
|
105
|
+
|
106
|
+
def perform(dir, filename)
|
107
|
+
end
|
108
|
+
end
|
79
109
|
|
80
110
|
### Amazon S3 Listener
|
81
111
|
|
82
|
-
Check Amazon S3, queue new file keys, and quit
|
112
|
+
Check Amazon S3, queue new file keys, and quit:
|
113
|
+
|
114
|
+
ryespy --listener amzn-s3 --amzn-s3-access-key c/example/com --amzn-s3-secret-key helpimabroccoli --amzn-s3-bucket vegetable-box --notifier-sidekiq
|
83
115
|
|
84
|
-
|
116
|
+
For non-* or multiple key prefix filters,
|
117
|
+
use `--amzn-s3-prefixes virtual-dir1/,virtual-dir`.
|
85
118
|
|
86
|
-
|
119
|
+
#### Example Worker
|
120
|
+
|
121
|
+
class RyespyAmznS3Job
|
122
|
+
include Sidekiq::Worker
|
123
|
+
|
124
|
+
sidekiq_options :queue => :ryespy
|
125
|
+
|
126
|
+
def perform(filename)
|
127
|
+
end
|
128
|
+
end
|
87
129
|
|
88
130
|
### Google Cloud Storage Listener
|
89
131
|
|
90
|
-
Check Google Cloud Storage, queue new file keys, and quit
|
132
|
+
Check Google Cloud Storage, queue new file keys, and quit:
|
133
|
+
|
134
|
+
ryespy --listener goog-cs --goog-cs-access-key d/example/com --goog-cs-secret-key helpimanasparagus --goog-cs-bucket vegetable-box --notifier-sidekiq
|
135
|
+
|
136
|
+
For non-* or multiple key prefix filters,
|
137
|
+
use `--goog-cs-prefixes virtual-dir1/,virtual-dir`.
|
138
|
+
|
139
|
+
#### Example Worker
|
140
|
+
|
141
|
+
class RyespyGoogCSJob
|
142
|
+
include Sidekiq::Worker
|
143
|
+
|
144
|
+
sidekiq_options :queue => :ryespy
|
145
|
+
|
146
|
+
def perform(filename)
|
147
|
+
end
|
148
|
+
end
|
149
|
+
|
150
|
+
### Google Drive Listener
|
151
|
+
|
152
|
+
Check Google Drive, queue new file resource ids, and quit:
|
153
|
+
|
154
|
+
ryespy --listener goog-drv --goog-drv-username vegetable-box@gmail.com --goog-drv-password helpimgarlic --goog-drv-filters vegetable-box --notifier-sidekiq
|
91
155
|
|
92
|
-
|
156
|
+
#### Example Worker
|
93
157
|
|
94
|
-
|
158
|
+
class RyespyGoogDrvJob
|
159
|
+
include Sidekiq::Worker
|
160
|
+
|
161
|
+
sidekiq_options :queue => :ryespy
|
162
|
+
|
163
|
+
def perform(resource_id)
|
164
|
+
end
|
165
|
+
end
|
95
166
|
|
96
167
|
### Rackspace Cloud Files Listener
|
97
168
|
|
98
|
-
Check Rackspace Cloud Files, queue new file keys, and quit
|
169
|
+
Check Rackspace Cloud Files, queue new file keys, and quit:
|
170
|
+
|
171
|
+
ryespy --listener rax-cf --rax-cf-username vegetable --rax-cf-api-key helpimacelery --rax-cf-container vegetable-box --notifier-sidekiq
|
99
172
|
|
100
|
-
|
173
|
+
For non-DFW region, use `--rax-cf-region lon`.
|
174
|
+
For non-US auth endpoint, use `--rax-cf-endpoint uk`.
|
175
|
+
Is your Rackspace account in London? Fret not; combine these and
|
176
|
+
use `--rax-cf-endpoint uk --rax-cf-region lon`.
|
177
|
+
For non-* or multiple key prefix filters,
|
178
|
+
use `--rax-cf-prefixes virtual-dir1/,virtual-dir`.
|
101
179
|
|
102
|
-
|
180
|
+
#### Example Worker
|
181
|
+
|
182
|
+
class RyespyRaxCFJob
|
183
|
+
include Sidekiq::Worker
|
184
|
+
|
185
|
+
sidekiq_options :queue => :ryespy
|
186
|
+
|
187
|
+
def perform(filename)
|
188
|
+
end
|
189
|
+
end
|
103
190
|
|
104
191
|
|
105
192
|
## Advanced Usage
|
106
193
|
|
107
|
-
If you want to do something rather more magical
|
194
|
+
If you want to do something rather more magical such as checking multiple
|
195
|
+
accounts for a listener or even multiple listeners, then you may wish to use the
|
196
|
+
Ryespy library directly instead of the `ryespy` executable.
|
108
197
|
|
109
|
-
Depend upon the `ryespy` gem in a `Gemfile`, remembering to add any manual
|
198
|
+
Depend upon the `ryespy` gem in a `Gemfile`, remembering to add any manual
|
199
|
+
dependencies for listeners as detailed in [Installation](#installation):
|
110
200
|
|
111
201
|
# Gemfile
|
112
202
|
|
@@ -134,7 +224,10 @@ Create the notifiers:
|
|
134
224
|
:namespace => 'resque'
|
135
225
|
)
|
136
226
|
|
137
|
-
For each listener, configure like in `ryespy --help` but without the
|
227
|
+
For each listener, configure like in `ryespy --help` but without the prefix and
|
228
|
+
with `-` changed to `_` (e.g. `--amzn-s3-access-key` => `:access_key`). Pass in
|
229
|
+
an array of notifiers. Note that the `check()` argument varies per listener,
|
230
|
+
meaning IMAP mailbox, FTP directory, Google Drive filter, or storage key prefix.
|
138
231
|
|
139
232
|
require 'ryespy/listener/amzn_s3'
|
140
233
|
|
@@ -162,15 +255,6 @@ You can also become a [watcher](https://github.com/tiredpixel/ryespy/watchers)
|
|
162
255
|
on GitHub. And don't forget you can become a [stargazer](https://github.com/tiredpixel/ryespy/stargazers) if you are so minded. :D
|
163
256
|
|
164
257
|
|
165
|
-
## Growing Like Flowers
|
166
|
-
|
167
|
-
Dear Me, Here is a vague wishlist:
|
168
|
-
|
169
|
-
- Additional notifiers (e.g. URL ?)
|
170
|
-
|
171
|
-
Also take a look at the [issue tracker](https://github.com/tiredpixel/ryespy/issues).
|
172
|
-
|
173
|
-
|
174
258
|
## Contributions
|
175
259
|
|
176
260
|
Contributions are embraced with much love and affection!
|
@@ -187,7 +271,11 @@ which is included by default in Ruby 1.9 onwards. To run all tests:
|
|
187
271
|
|
188
272
|
rake test
|
189
273
|
|
190
|
-
When using the `ryespy` executable in development, you'll probably want to set
|
274
|
+
When using the `ryespy` executable in development, you'll probably want to set
|
275
|
+
`--debug` mode so debug-level messages are logged and stack traces raised.
|
276
|
+
|
277
|
+
We render grateful thanks unto our
|
278
|
+
[contributors](https://github.com/tiredpixel/ryespy/graphs/contributors).
|
191
279
|
|
192
280
|
|
193
281
|
## Blessing
|
@@ -197,6 +285,7 @@ May you find peace, and help others to do likewise.
|
|
197
285
|
|
198
286
|
## Licence
|
199
287
|
|
200
|
-
© [tiredpixel](http://www.tiredpixel.com)
|
288
|
+
© [tiredpixel](http://www.tiredpixel.com) 2014 and
|
289
|
+
[others](https://github.com/tiredpixel/ryespy/graphs/contributors).
|
201
290
|
It is free software, released under the MIT License, and may be redistributed
|
202
291
|
under the terms specified in `LICENSE`.
|
data/bin/ryespy
CHANGED
@@ -31,8 +31,9 @@ OptionParser.new do |opts|
|
|
31
31
|
'ftp',
|
32
32
|
'amzn-s3',
|
33
33
|
'goog-cs',
|
34
|
+
'goog-drv',
|
34
35
|
'rax-cf',
|
35
|
-
], "Listener (imap|ftp|amzn-s3|goog-cs|rax-cf)") do |o|
|
36
|
+
], "Listener (imap|ftp|amzn-s3|goog-cs|goog-drv|rax-cf)") do |o|
|
36
37
|
options[:listener] = o.tr('-', '_').to_sym
|
37
38
|
end
|
38
39
|
|
@@ -149,7 +150,22 @@ OptionParser.new do |opts|
|
|
149
150
|
opts.on("--goog-cs-prefixes [P1,P2]", Array, "Prefixes P1,P2 (default: *)") do |o|
|
150
151
|
options[:goog_cs_filters] = o
|
151
152
|
end
|
152
|
-
|
153
|
+
|
154
|
+
opts.separator ""
|
155
|
+
opts.separator "--listener goog-drv (Google Drive) :"
|
156
|
+
|
157
|
+
opts.on("--goog-drv-username USERNAME", "USERNAME") do |o|
|
158
|
+
options[:goog_drv_username] = o
|
159
|
+
end
|
160
|
+
|
161
|
+
opts.on("--goog-drv-password PASSWORD", "PASSWORD") do |o|
|
162
|
+
options[:goog_drv_password] = o
|
163
|
+
end
|
164
|
+
|
165
|
+
opts.on("--goog-drv-filters [F1,F2]", Array, "Filters F1,F2 (default: *)") do |o|
|
166
|
+
options[:goog_drv_filters] = o
|
167
|
+
end
|
168
|
+
|
153
169
|
opts.separator ""
|
154
170
|
opts.separator "--listener rax-cf (Rackspace Cloud Files) :"
|
155
171
|
|
@@ -270,6 +286,11 @@ end
|
|
270
286
|
:bucket,
|
271
287
|
:filters, # prefixes
|
272
288
|
],
|
289
|
+
:goog_drv => [
|
290
|
+
:username,
|
291
|
+
:password,
|
292
|
+
:filters,
|
293
|
+
],
|
273
294
|
:rax_cf => [
|
274
295
|
:endpoint,
|
275
296
|
:region,
|
data/lib/ryespy/app.rb
CHANGED
@@ -31,6 +31,9 @@ module Ryespy
|
|
31
31
|
:goog_cs => {
|
32
32
|
:filters => [''], # prefixes
|
33
33
|
},
|
34
|
+
:goog_drv => {
|
35
|
+
:filters => [''],
|
36
|
+
},
|
34
37
|
:rax_cf => {
|
35
38
|
:endpoint => :us,
|
36
39
|
:region => :dfw,
|
@@ -68,7 +71,6 @@ module Ryespy
|
|
68
71
|
def notifiers
|
69
72
|
unless @notifiers
|
70
73
|
@notifiers = []
|
71
|
-
|
72
74
|
@config.notifiers[:sidekiq].each do |notifier_url|
|
73
75
|
@notifiers << Notifier::Sidekiq.new(
|
74
76
|
:url => notifier_url,
|
@@ -118,7 +120,7 @@ module Ryespy
|
|
118
120
|
check_all
|
119
121
|
rescue StandardError => e
|
120
122
|
@logger.error { e.to_s }
|
121
|
-
|
123
|
+
|
122
124
|
raise if @config.log_level == :DEBUG
|
123
125
|
end
|
124
126
|
|
@@ -140,6 +142,7 @@ module Ryespy
|
|
140
142
|
:ftp => :FTP,
|
141
143
|
:amzn_s3 => :AmznS3,
|
142
144
|
:goog_cs => :GoogCS,
|
145
|
+
:goog_drv => :GoogDrv,
|
143
146
|
:rax_cf => :RaxCF,
|
144
147
|
}
|
145
148
|
|
data/lib/ryespy/listener/ftp.rb
CHANGED
@@ -0,0 +1,80 @@
|
|
1
|
+
require "google_drive"
|
2
|
+
|
3
|
+
require_relative 'base'
|
4
|
+
|
5
|
+
|
6
|
+
module Ryespy
|
7
|
+
module Listener
|
8
|
+
class GoogDrv < Base
|
9
|
+
|
10
|
+
REDIS_KEY_PREFIX = 'goog_drv'.freeze
|
11
|
+
SIDEKIQ_JOB_CLASS = 'RyespyGoogDrvJob'.freeze
|
12
|
+
|
13
|
+
def initialize(opts = {})
|
14
|
+
@config = {
|
15
|
+
:username => opts[:username],
|
16
|
+
:password => opts[:password],
|
17
|
+
}
|
18
|
+
|
19
|
+
super(opts)
|
20
|
+
end
|
21
|
+
|
22
|
+
def check(filter)
|
23
|
+
@logger.debug { "filter: #{filter}" }
|
24
|
+
|
25
|
+
@logger.debug { "redis_key: #{redis_key}" }
|
26
|
+
|
27
|
+
seen_files = @redis.hgetall(redis_key)
|
28
|
+
|
29
|
+
unseen_files = get_unseen_files(filter, seen_files)
|
30
|
+
|
31
|
+
@logger.debug { "unseen_files: #{unseen_files}" }
|
32
|
+
|
33
|
+
unseen_files.each do |key, val|
|
34
|
+
@redis.hset(redis_key, key, val)
|
35
|
+
|
36
|
+
@notifiers.each { |n| n.notify(SIDEKIQ_JOB_CLASS, [key]) }
|
37
|
+
end
|
38
|
+
|
39
|
+
@logger.info { "#{filter} has #{unseen_files.count} new files" }
|
40
|
+
end
|
41
|
+
|
42
|
+
private
|
43
|
+
|
44
|
+
def connect_service
|
45
|
+
@google_drive = GoogleDrive.login(@config[:username], @config[:password])
|
46
|
+
end
|
47
|
+
|
48
|
+
def redis_key
|
49
|
+
[
|
50
|
+
REDIS_KEY_PREFIX,
|
51
|
+
@config[:username]
|
52
|
+
].join(':')
|
53
|
+
end
|
54
|
+
|
55
|
+
def get_unseen_files(filter, seen_files)
|
56
|
+
files = {}
|
57
|
+
|
58
|
+
@google_drive.files.each do |file|
|
59
|
+
next unless file.title =~ /#{filter}/ && file.resource_id && file.resource_type != 'folder'
|
60
|
+
|
61
|
+
# updated should be present for all resource_type , but there is often
|
62
|
+
# a delay until it is set.
|
63
|
+
updated = file.document_feed_entry.css('updated').first.text
|
64
|
+
|
65
|
+
# etag is present for most resource_type , but is nil for spreadsheet.
|
66
|
+
etag = file.document_feed_entry.attribute('etag')
|
67
|
+
|
68
|
+
checksum = "#{updated},#{etag}"
|
69
|
+
|
70
|
+
if seen_files[file.resource_id] != checksum
|
71
|
+
files[file.resource_id] = checksum
|
72
|
+
end
|
73
|
+
end
|
74
|
+
|
75
|
+
files
|
76
|
+
end
|
77
|
+
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
data/lib/ryespy/listener/imap.rb
CHANGED
data/lib/ryespy/version.rb
CHANGED
data/ryespy.gemspec
CHANGED
@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
|
|
8
8
|
spec.version = Ryespy::VERSION
|
9
9
|
spec.authors = ["tiredpixel"]
|
10
10
|
spec.email = ["tp@tiredpixel.com"]
|
11
|
-
spec.
|
12
|
-
spec.
|
11
|
+
spec.summary = %q{Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage, Google Drive, Rackspace Cloud Files listener.}
|
12
|
+
spec.description = %q{}
|
13
13
|
spec.homepage = "https://github.com/tiredpixel/ryespy"
|
14
14
|
spec.license = "MIT"
|
15
15
|
|
@@ -25,5 +25,6 @@ Gem::Specification.new do |spec|
|
|
25
25
|
spec.add_development_dependency "rake"
|
26
26
|
spec.add_development_dependency "fog", "~> 1.19" # conditional dependency
|
27
27
|
spec.add_development_dependency "mocha", "~> 0.14"
|
28
|
-
spec.add_development_dependency "sidekiq-spy", "
|
28
|
+
spec.add_development_dependency "sidekiq-spy", ">= 0.3.2"
|
29
|
+
spec.add_development_dependency "google_drive", '~> 0.3'
|
29
30
|
end
|
@@ -0,0 +1,172 @@
|
|
1
|
+
require_relative '../../helper'
|
2
|
+
|
3
|
+
require_relative '../../../lib/ryespy/listener/goog_drv'
|
4
|
+
|
5
|
+
|
6
|
+
describe Ryespy::Listener::GoogDrv do
|
7
|
+
|
8
|
+
before do
|
9
|
+
updated = '1897-05-26 00:00:00 UTC'
|
10
|
+
|
11
|
+
dfe = stub
|
12
|
+
dfe.stubs(:css).with('updated').returns([stub(:text => updated)])
|
13
|
+
dfe.stubs(:attribute).with('etag').returns('E13l2izwZbgWKfS5')
|
14
|
+
|
15
|
+
dfe2 = stub
|
16
|
+
dfe2.stubs(:css).with('updated').returns([stub(:text => updated)])
|
17
|
+
dfe2.stubs(:attribute).with('etag').returns(nil)
|
18
|
+
|
19
|
+
@files = [
|
20
|
+
stub(
|
21
|
+
:resource_id => 'folder:FFTSKJtqo5v2TRrIADAmVOlsqe8h9TqM',
|
22
|
+
:resource_type => 'folder',
|
23
|
+
:title => 'Flies/',
|
24
|
+
:document_feed_entry => dfe,
|
25
|
+
),
|
26
|
+
stub(
|
27
|
+
:resource_id => 'presentation:SuOQSdIwu3E58fq7HsbtfjG2aeuTDtOJ',
|
28
|
+
:resource_type => 'presentation',
|
29
|
+
:title => 'Flies Presentation',
|
30
|
+
:document_feed_entry => dfe,
|
31
|
+
),
|
32
|
+
stub(
|
33
|
+
:resource_id => 'spreadsheet:GfVKovEtmvWtMgWDTv94kcYZLbje1O3q',
|
34
|
+
:resource_type => 'spreadsheet',
|
35
|
+
:title => 'Flies Spreadsheet',
|
36
|
+
:document_feed_entry => dfe2,
|
37
|
+
),
|
38
|
+
stub(
|
39
|
+
:resource_id => 'drawing:ceKiP0rwNufPU3qguhleVIxmtXqMjaWe',
|
40
|
+
:resource_type => 'drawing',
|
41
|
+
:title => 'FliesBuzz Drawing',
|
42
|
+
:document_feed_entry => dfe,
|
43
|
+
),
|
44
|
+
stub(
|
45
|
+
:resource_id => 'form:OCBeiWQf51qvmPdWVT8olFRqTM8pl5sC',
|
46
|
+
:resource_type => 'form',
|
47
|
+
:title => 'Spiders and Flies Form',
|
48
|
+
:document_feed_entry => dfe,
|
49
|
+
),
|
50
|
+
stub(
|
51
|
+
:resource_id => 'document:gvZHG5YA5rkcgIXGUnDXhMOI0qE7WKnm',
|
52
|
+
:resource_type => 'document',
|
53
|
+
:title => 'Spiders Document',
|
54
|
+
:document_feed_entry => dfe,
|
55
|
+
),
|
56
|
+
stub(
|
57
|
+
:resource_id => 'file:i7iqdRt3CxbJEWuo8kXIFBe9WeTnQRL3',
|
58
|
+
:resource_type => 'file',
|
59
|
+
:title => 'Spiders File',
|
60
|
+
:document_feed_entry => dfe,
|
61
|
+
),
|
62
|
+
]
|
63
|
+
|
64
|
+
@files_no_dirs = @files.select { |f| f.resource_type != 'folder' }
|
65
|
+
|
66
|
+
@google_drive = stub(
|
67
|
+
:files => @files
|
68
|
+
)
|
69
|
+
|
70
|
+
GoogleDrive.stubs(:login).with('r.m.renfield', 'master').returns(@google_drive)
|
71
|
+
end
|
72
|
+
|
73
|
+
describe "#check" do
|
74
|
+
before do
|
75
|
+
Ryespy::Test::Redis::setup
|
76
|
+
|
77
|
+
@notifier = mock()
|
78
|
+
|
79
|
+
@goog_drv = Ryespy::Listener::GoogDrv.new(
|
80
|
+
:username => 'r.m.renfield',
|
81
|
+
:password => 'master',
|
82
|
+
:notifiers => [@notifier],
|
83
|
+
)
|
84
|
+
|
85
|
+
@redis = @goog_drv.instance_variable_get(:@redis)
|
86
|
+
end
|
87
|
+
|
88
|
+
after do
|
89
|
+
@goog_drv.close
|
90
|
+
|
91
|
+
Ryespy::Test::Redis::flush_namespace(@redis)
|
92
|
+
end
|
93
|
+
|
94
|
+
it "notifies when new files filter *" do
|
95
|
+
@files_no_dirs.each do |file|
|
96
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', [file.resource_id]).once
|
97
|
+
end
|
98
|
+
|
99
|
+
@goog_drv.check('')
|
100
|
+
end
|
101
|
+
|
102
|
+
it "notifies when new files filter ^Flies\\b" do
|
103
|
+
@files_no_dirs.select { |f| f.title =~ /^Flies\b/ }.each do |file|
|
104
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', [file.resource_id]).once
|
105
|
+
end
|
106
|
+
|
107
|
+
@goog_drv.check('^Flies\b')
|
108
|
+
end
|
109
|
+
|
110
|
+
it "notifies when new files filter Flies" do
|
111
|
+
@files_no_dirs.select { |f| f.title =~ /Flies/ }.each do |file|
|
112
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', [file.resource_id]).once
|
113
|
+
end
|
114
|
+
|
115
|
+
@goog_drv.check('Flies')
|
116
|
+
end
|
117
|
+
|
118
|
+
it "doesn't notify when no new files" do
|
119
|
+
@notifier.expects(:notify).times(3)
|
120
|
+
|
121
|
+
@goog_drv.check('Spiders')
|
122
|
+
@goog_drv.check('Spiders')
|
123
|
+
end
|
124
|
+
|
125
|
+
it "doesn't notify when no new files filter subset" do
|
126
|
+
@notifier.expects(:notify).times(3)
|
127
|
+
|
128
|
+
@goog_drv.check('Spiders')
|
129
|
+
@goog_drv.check('Spiders\ and\ Flies')
|
130
|
+
end
|
131
|
+
|
132
|
+
it "notifies when new files filter distinct" do
|
133
|
+
@notifier.expects(:notify).times(4)
|
134
|
+
|
135
|
+
@goog_drv.check('Spiders\ and\ Flies')
|
136
|
+
@goog_drv.check('Flies')
|
137
|
+
end
|
138
|
+
|
139
|
+
it "notifies when changed updated" do
|
140
|
+
dfe = stub
|
141
|
+
dfe.stubs(:css).with('updated').returns([stub(:text => '1899-03-23 01:00:00 UTC')])
|
142
|
+
dfe.stubs(:attribute).with('etag').returns('E13l2izwZbgWKfS5')
|
143
|
+
|
144
|
+
@notifier.expects(:notify).times(4)
|
145
|
+
|
146
|
+
@goog_drv.check('Flies')
|
147
|
+
|
148
|
+
@files[1].stubs(:document_feed_entry).returns(dfe)
|
149
|
+
|
150
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', ['presentation:SuOQSdIwu3E58fq7HsbtfjG2aeuTDtOJ']).once
|
151
|
+
|
152
|
+
@goog_drv.check('Flies')
|
153
|
+
end
|
154
|
+
|
155
|
+
it "notifies when changed etag" do
|
156
|
+
dfe = stub
|
157
|
+
dfe.stubs(:css).with('updated').returns([stub(:text => '1897-05-26 00:00:00 UTC')])
|
158
|
+
dfe.stubs(:attribute).with('etag').returns('qGE4UWIcITePyidC')
|
159
|
+
|
160
|
+
@notifier.expects(:notify).times(4)
|
161
|
+
|
162
|
+
@goog_drv.check('Flies')
|
163
|
+
|
164
|
+
@files[3].stubs(:document_feed_entry).returns(dfe)
|
165
|
+
|
166
|
+
@notifier.expects(:notify).with('RyespyGoogDrvJob', ['drawing:ceKiP0rwNufPU3qguhleVIxmtXqMjaWe']).once
|
167
|
+
|
168
|
+
@goog_drv.check('Flies')
|
169
|
+
end
|
170
|
+
end
|
171
|
+
|
172
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: ryespy
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- tiredpixel
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-06-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: redis
|
@@ -102,6 +102,20 @@ dependencies:
|
|
102
102
|
version: '0.14'
|
103
103
|
- !ruby/object:Gem::Dependency
|
104
104
|
name: sidekiq-spy
|
105
|
+
requirement: !ruby/object:Gem::Requirement
|
106
|
+
requirements:
|
107
|
+
- - ">="
|
108
|
+
- !ruby/object:Gem::Version
|
109
|
+
version: 0.3.2
|
110
|
+
type: :development
|
111
|
+
prerelease: false
|
112
|
+
version_requirements: !ruby/object:Gem::Requirement
|
113
|
+
requirements:
|
114
|
+
- - ">="
|
115
|
+
- !ruby/object:Gem::Version
|
116
|
+
version: 0.3.2
|
117
|
+
- !ruby/object:Gem::Dependency
|
118
|
+
name: google_drive
|
105
119
|
requirement: !ruby/object:Gem::Requirement
|
106
120
|
requirements:
|
107
121
|
- - "~>"
|
@@ -114,8 +128,7 @@ dependencies:
|
|
114
128
|
- - "~>"
|
115
129
|
- !ruby/object:Gem::Version
|
116
130
|
version: '0.3'
|
117
|
-
description:
|
118
|
-
Cloud Files listener.
|
131
|
+
description: ''
|
119
132
|
email:
|
120
133
|
- tp@tiredpixel.com
|
121
134
|
executables:
|
@@ -140,6 +153,7 @@ files:
|
|
140
153
|
- lib/ryespy/listener/fogable.rb
|
141
154
|
- lib/ryespy/listener/ftp.rb
|
142
155
|
- lib/ryespy/listener/goog_cs.rb
|
156
|
+
- lib/ryespy/listener/goog_drv.rb
|
143
157
|
- lib/ryespy/listener/imap.rb
|
144
158
|
- lib/ryespy/listener/rax_cf.rb
|
145
159
|
- lib/ryespy/notifier/sidekiq.rb
|
@@ -150,6 +164,7 @@ files:
|
|
150
164
|
- test/ryespy/listener/amzn_s3_test.rb
|
151
165
|
- test/ryespy/listener/ftp_test.rb
|
152
166
|
- test/ryespy/listener/goog_cs_test.rb
|
167
|
+
- test/ryespy/listener/goog_drv_test.rb
|
153
168
|
- test/ryespy/listener/imap_test.rb
|
154
169
|
- test/ryespy/listener/rax_cf_test.rb
|
155
170
|
- test/ryespy/notifier/sidekiq_test.rb
|
@@ -174,17 +189,18 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
174
189
|
version: '0'
|
175
190
|
requirements: []
|
176
191
|
rubyforge_project:
|
177
|
-
rubygems_version: 2.2.
|
192
|
+
rubygems_version: 2.2.2
|
178
193
|
signing_key:
|
179
194
|
specification_version: 4
|
180
|
-
summary: Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage,
|
181
|
-
Cloud Files listener.
|
195
|
+
summary: Redis Sidekiq/Resque IMAP, FTP, Amazon S3, Google Cloud Storage, Google Drive,
|
196
|
+
Rackspace Cloud Files listener.
|
182
197
|
test_files:
|
183
198
|
- test/helper.rb
|
184
199
|
- test/ryespy/app_test.rb
|
185
200
|
- test/ryespy/listener/amzn_s3_test.rb
|
186
201
|
- test/ryespy/listener/ftp_test.rb
|
187
202
|
- test/ryespy/listener/goog_cs_test.rb
|
203
|
+
- test/ryespy/listener/goog_drv_test.rb
|
188
204
|
- test/ryespy/listener/imap_test.rb
|
189
205
|
- test/ryespy/listener/rax_cf_test.rb
|
190
206
|
- test/ryespy/notifier/sidekiq_test.rb
|