pushpop 0.1.1 → 0.1.2
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +1 -0
- data/Gemfile +1 -1
- data/README.md +73 -165
- data/Rakefile +0 -34
- data/bin/pushpop +8 -0
- data/lib/pushpop.rb +1 -0
- data/lib/pushpop/cli.rb +77 -0
- data/lib/pushpop/job.rb +1 -1
- data/lib/pushpop/version.rb +1 -1
- data/pushpop.gemspec +4 -2
- data/spec/pushpop/cli_spec.rb +35 -0
- data/spec/pushpop/job_spec.rb +35 -35
- data/spec/pushpop/step_spec.rb +25 -25
- data/spec/pushpop_spec.rb +9 -7
- data/spec/simple_job_spec.rb +4 -4
- metadata +44 -9
- data/Gemfile.lock +0 -36
- data/Procfile +0 -1
- data/jobs/example_job.rb +0 -20
data/.gitignore
CHANGED
data/Gemfile
CHANGED
data/README.md
CHANGED
@@ -4,7 +4,7 @@
|
|
4
4
|
|
5
5
|
[![Build Status](https://travis-ci.org/pushpop-project/pushpop.svg)](https://travis-ci.org/pushpop-project/pushpop)
|
6
6
|
|
7
|
-
### A framework for scheduled integrations
|
7
|
+
### A framework for scheduled integrations between popular services
|
8
8
|
|
9
9
|
<hr>
|
10
10
|
<img src="http://f.cl.ly/items/1I421w263a10340a0u2q/Screen%20Shot%202014-04-16%20at%204.35.47%20PM.png" width="45%" alt="Pingpong Daily Response Time Report">
|
@@ -14,13 +14,15 @@
|
|
14
14
|
|
15
15
|
## Overview
|
16
16
|
|
17
|
-
Pushpop is a
|
18
|
-
This can be used to do
|
17
|
+
Pushpop is a powerful framework for taking actions and integrating services at regular intervals.
|
18
|
+
This can be used to do anything from scheduled data collection to alerting based on patterns in data.
|
19
19
|
|
20
|
-
Pushpop
|
20
|
+
Pushpop began as a way to send notifications and reports based on events captured with [Keen IO](https://keen.io).
|
21
21
|
See plugins for more services on the [Pushpop organization](https://github.com/pushpop-project) home page.
|
22
22
|
|
23
|
-
|
23
|
+
Pushpop is packaged as a Ruby gem. It can be added to existing Ruby projects or used in new ones.
|
24
|
+
|
25
|
+
#### Ideas for using Pushpop
|
24
26
|
|
25
27
|
##### Alerts
|
26
28
|
|
@@ -31,10 +33,15 @@ See plugins for more services on the [Pushpop organization](https://github.com/p
|
|
31
33
|
|
32
34
|
+ Send a sales report to your inbox every day at noon
|
33
35
|
+ Send analytics reports to your customers every week
|
36
|
+
|
37
|
+
##### Monitoring
|
38
|
+
|
39
|
+
+ Track the performance of web services and APIs
|
40
|
+
+ Read sensor values for analysis and alerting
|
34
41
|
|
35
|
-
####
|
42
|
+
#### Example Pushpop job
|
36
43
|
|
37
|
-
|
44
|
+
Pushpop organizes work into jobs. Here's a Pushpop job that uses the `keen` and `sendgrid` plugins to send a nightly email. The email contains the day's number of pageviews:
|
38
45
|
|
39
46
|
``` ruby
|
40
47
|
require 'pushpop-keen'
|
@@ -60,162 +67,92 @@ job do
|
|
60
67
|
end
|
61
68
|
```
|
62
69
|
|
63
|
-
The email is sent by [Sendgrid](https://sendgrid.com)
|
64
|
-
|
65
|
-
Pushpop syntax is short and sweet, but because Pushpop is just Ruby it's also quite powerful.
|
66
|
-
|
67
|
-
### Get Started
|
68
|
-
|
69
|
-
Excited to try out Pushpop with your Keen IO projects? Here's a few options to choose from:
|
70
|
-
|
71
|
-
#### The Quickstart
|
72
|
-
|
73
|
-
Setup Pushpop locally and run your first job in minutes.
|
74
|
-
|
75
|
-
**[Go to the Quickstart](#quickstart)**
|
76
|
-
|
77
|
-
#### Deploy a Pushpop Instance
|
78
|
-
|
79
|
-
Ready to deploy a local Pushpop job? Detailed instructions for Heroku are provided as well as a basic guide for other platforms.
|
70
|
+
Keen IO provides the analytics data behind the report. The email is sent by [Sendgrid](https://sendgrid.com) via the [sendgrid](https://github.com/pushpop-project/pushpop-sendgrid) Pushpop plugin.
|
80
71
|
|
81
|
-
|
82
|
-
|
83
|
-
#### Need help?
|
84
|
-
|
85
|
-
Don't have a hacker on hand? The friendly folks at Keen IO can set a Pushpop up for you.
|
86
|
-
|
87
|
-
**Email [team@keen.io](mailto:team@keen.io?subject=I want a Pushpop!)** with the subject "I want a Pushpop!". Include information about what queries you'd like to run (and when) and how you'd like the results communicated.
|
72
|
+
Pushpop syntax is short and sweet, but because Pushpop is pure Ruby it's also quite powerful.
|
88
73
|
|
89
74
|
## Quickstart
|
90
75
|
|
91
|
-
|
92
|
-
|
93
|
-
#### Prerequisites
|
94
|
-
|
95
|
-
+ A working [Ruby installation](https://www.ruby-lang.org/en/installation/) (1.9+)
|
76
|
+
Install Pushpop as a Ruby gem:
|
96
77
|
|
97
|
-
#### Steps
|
98
|
-
|
99
|
-
##### Clone the Pushpop starter project
|
100
|
-
|
101
|
-
``` shell
|
102
|
-
$ git clone git@github.com:pushpop-project/pushpop-starter.git
|
103
78
|
```
|
104
|
-
|
105
|
-
Enter the `pushpop-starter` directory and install dependencies.
|
106
|
-
|
107
|
-
``` shell
|
108
|
-
$ cd pushpop-starter
|
109
|
-
$ gem install bundler
|
110
|
-
$ bundle install
|
79
|
+
$ gem install pushpop
|
111
80
|
```
|
112
81
|
|
113
|
-
|
114
|
-
|
115
|
-
There is an example job in `jobs/example_job.rb` of the pushpop-starter repository. It simply prints output to the console. Run this job via a rake task to make sure your configuration is setup properly.
|
82
|
+
You should now have a `pushpop` command available in your shell. Try it out with no arguments to see a list of possible commands:
|
116
83
|
|
117
|
-
``` shell
|
118
|
-
$ bundle exec rake jobs:run_once[jobs/example_job.rb]
|
119
84
|
```
|
120
|
-
|
121
|
-
|
122
|
-
|
123
|
-
|
124
|
-
|
125
|
-
|
85
|
+
$ pushpop
|
86
|
+
Commands:
|
87
|
+
pushpop help [COMMAND] # Describe available commands or one specific command
|
88
|
+
pushpop jobs:describe # Describe jobs
|
89
|
+
pushpop jobs:run # Run jobs ongoing
|
90
|
+
pushpop jobs:run_once # Run jobs once
|
91
|
+
pushpop version # Print the Pushpop version
|
126
92
|
```
|
127
93
|
|
128
|
-
|
94
|
+
##### Create a simple job
|
129
95
|
|
130
|
-
|
131
|
-
|
132
|
-
+ Write and test more jobs. See the [Pushpop API Documentation](#pushpop-api-documentation) below for more examples of what you can do.
|
133
|
-
+ See [pushpop-recipes](https://github.com/pushpop-project/pushpop-recipes) for reusable code and inspiration.
|
134
|
-
+ Continue on to the [Deploy Guide](#deploy-guide) to deploy the job you just created.
|
96
|
+
Copy the code below into a file called `example_job.rb`. This job simply prints output to the console:
|
135
97
|
|
136
|
-
|
137
|
-
|
138
|
-
#### Heroku
|
139
|
-
|
140
|
-
These instructions are for Heroku, but should be relevant to most environments.
|
141
|
-
|
142
|
-
##### Prerequisites
|
143
|
-
|
144
|
-
You'll need a [Heroku](https://heroku.com/) account, and the [Heroku toolbelt](https://toolbelt.heroku.com/) installed.
|
145
|
-
|
146
|
-
##### Create a new Heroku app
|
147
|
-
|
148
|
-
Make sure you're inside a Pushpop project directory (e.g. pushpop-starter), than create a new Heroku app.
|
149
|
-
|
150
|
-
``` shell
|
151
|
-
$ heroku create
|
152
|
-
```
|
98
|
+
``` ruby
|
99
|
+
require 'pushpop'
|
153
100
|
|
154
|
-
|
101
|
+
job do
|
155
102
|
|
156
|
-
|
103
|
+
every 1.seconds
|
157
104
|
|
158
|
-
|
105
|
+
step do
|
106
|
+
puts 'Hello World!'
|
107
|
+
end
|
159
108
|
|
160
|
-
|
161
|
-
$ git commit -am 'Created my first Pushpop job'
|
109
|
+
end
|
162
110
|
```
|
163
111
|
|
164
|
-
|
165
|
-
|
166
|
-
Now that your code is commited and config variables pushed we can begin a deploy. We'll also need to scale the number of worker processes to 1.
|
112
|
+
Run this job once via using the `jobs:run_once` command:
|
167
113
|
|
168
114
|
``` shell
|
169
|
-
$
|
170
|
-
$ heroku scale worker=1
|
115
|
+
$ pushpop jobs:run_once --file example_job.rb
|
171
116
|
```
|
172
117
|
|
173
|
-
|
174
|
-
|
175
|
-
To see that jobs are running and that there are no errors, tail the logs on Heroku.
|
118
|
+
You should see the following output:
|
176
119
|
|
177
|
-
```
|
178
|
-
|
120
|
+
``` html
|
121
|
+
Hello World!
|
179
122
|
```
|
180
123
|
|
181
|
-
|
182
|
-
|
183
|
-
Another note - by default this will run all jobs in the `jobs` folder. You might want to delete the `example_job.rb` file in a separate commit once you've got the hang of things. You can change this behavior by editing the Procfile.
|
184
|
-
|
185
|
-
#### Other environments
|
186
|
-
|
187
|
-
Pushpop is deployed as one long-running Ruby process. Anywhere you can run this process you can run Pushpop. Here's the command:
|
124
|
+
That's all there is to it. To run the job repeatedly at the times specified by `every` just change `run_once` to `run`:
|
188
125
|
|
189
126
|
``` shell
|
190
|
-
$
|
127
|
+
$ pushpop jobs:run --file example_job.rb
|
191
128
|
```
|
192
129
|
|
193
|
-
|
130
|
+
Make sure to leave the process running in your terminal, or [send it to the background](http://stackoverflow.com/questions/625409/how-do-i-put-an-already-running-process-under-nohup), so that it keeps running.
|
194
131
|
|
195
|
-
|
196
|
-
$ bundle exec rake jobs:run
|
197
|
-
```
|
132
|
+
##### Next steps
|
198
133
|
|
199
|
-
|
134
|
+
+ Write and test more jobs. See the [Pushpop API Documentation](#pushpop-api-documentation) below for more examples of what you can do.
|
135
|
+
+ See the [pushpop-project](https://github.com/pushpop-project) Github organization to find plugins and reusable code.
|
200
136
|
|
201
|
-
##
|
137
|
+
## Pushpop Commands
|
202
138
|
|
203
|
-
Pushpop comes with
|
139
|
+
Pushpop comes with commands to describe and run jobs.
|
204
140
|
|
205
|
-
All `jobs:*`
|
141
|
+
All `jobs:*` commands optionally take a filename or directory as a `--file` or `-f` parameter. The file/directory is meant to contain one or more Pushpop jobs.
|
206
142
|
|
207
143
|
Specifying a specific file looks like this:
|
208
144
|
|
209
145
|
``` shell
|
210
|
-
$
|
146
|
+
$ pushpop jobs:run --file jobs/just_this_job.rb
|
211
147
|
```
|
212
148
|
|
213
|
-
Here's a list of the available
|
149
|
+
Here's a list of the available commands:
|
214
150
|
|
215
151
|
+ `jobs:describe` - Print out the names of jobs in the jobs folder.
|
216
152
|
+ `jobs:run_once` - Run each job once, right now.
|
217
|
-
+ `jobs:run` - Run jobs as scheduled in a long-running process. This is the
|
218
|
-
|
153
|
+
+ `jobs:run` - Run jobs as scheduled in a long-running process. This is the command you should use for a deployed Pushpop.
|
154
|
+
|
155
|
+
All `jobs:*` tasks also use [dotenv](https://github.com/bkeepers/dotenv) to load environment variables, often contained in a `.env` file.
|
219
156
|
|
220
157
|
## Pushpop API Documentation
|
221
158
|
|
@@ -226,8 +163,8 @@ Steps and jobs are the heart of the Pushpop workflow. Job files are written in p
|
|
226
163
|
Jobs have the following attributes:
|
227
164
|
|
228
165
|
+ `name`: (optional) something that describe the job, useful in logs
|
229
|
-
+ `period`: how frequently to run the job
|
230
|
-
+ `every_options` (optional): options related to when the job runs
|
166
|
+
+ `period`: how frequently to run the job; the first param to `every`
|
167
|
+
+ `every_options` (optional): options related to when the job runs; the second param to `every`
|
231
168
|
+ `steps`: an ordered list of steps to run
|
232
169
|
|
233
170
|
These attributes are easily specified using the DSL's block syntax. Here's an example:
|
@@ -244,9 +181,9 @@ end
|
|
244
181
|
The name of this job is 'print job'. It runs every 5 minutes and it has 1 step.
|
245
182
|
|
246
183
|
Inside of a `job` configuration block, steps are added by using the `step` method. They can also be
|
247
|
-
added by using a method registered by a plugin, like `keen` or `twilio`. For more
|
184
|
+
added by using a method registered by a plugin, like `keen` or `twilio`. For more informatio on plugins see [Plugins](#plugins).
|
248
185
|
|
249
|
-
The period of the job's execution is set via the `every` method. This is
|
186
|
+
The period of the job's execution is set via the `every` method. This is a passthrough to the [Clockwork](https://github.com/tomykaira/clockwork) long-running process scheduler. Clockwork gives you a great deal of flexibility when it comes specifiying when jobs should run. Here are some examples:
|
250
187
|
|
251
188
|
``` ruby
|
252
189
|
every 5.seconds
|
@@ -258,7 +195,7 @@ every 5.seconds, at: '10:**'
|
|
258
195
|
every 1.week, at: 'Monday 12:30'
|
259
196
|
```
|
260
197
|
|
261
|
-
See the full
|
198
|
+
See the full range of possibilities on the [Clockwork README](https://github.com/tomykaira/clockwork#event-parameters).
|
262
199
|
|
263
200
|
##### Job workflow
|
264
201
|
|
@@ -269,6 +206,8 @@ The map is keyed by step name, which defaults to a plugin name if a plugin was u
|
|
269
206
|
Here's an example that shows how the response chain works:
|
270
207
|
|
271
208
|
``` ruby
|
209
|
+
require 'pushpop'
|
210
|
+
|
272
211
|
job do
|
273
212
|
every 5.minutes
|
274
213
|
step 'one' do
|
@@ -278,7 +217,7 @@ job do
|
|
278
217
|
5 + response
|
279
218
|
end
|
280
219
|
step 'add previous steps' do |response, step_responses|
|
281
|
-
puts response # prints
|
220
|
+
puts response # prints 6
|
282
221
|
puts step_responses['one'] + step_responses['two'] # prints 6
|
283
222
|
end
|
284
223
|
end
|
@@ -287,6 +226,8 @@ end
|
|
287
226
|
If a `step` returns false, subsequent steps **are not run**. Here's a simple example that illustrates this:
|
288
227
|
|
289
228
|
``` ruby
|
229
|
+
require 'pushpop'
|
230
|
+
|
290
231
|
job 'lame job' do
|
291
232
|
every 5.minutes
|
292
233
|
step 'one' do
|
@@ -302,6 +243,8 @@ This behavior is designed to make *conditional* alerting easy. Here's an example
|
|
302
243
|
for certain query responses:
|
303
244
|
|
304
245
|
``` ruby
|
246
|
+
require 'pushpop'
|
247
|
+
|
305
248
|
job do
|
306
249
|
|
307
250
|
every 1.minute
|
@@ -365,13 +308,15 @@ for doing common things with Pushpop. Check it out for some inspiration!
|
|
365
308
|
|
366
309
|
## Plugins
|
367
310
|
|
368
|
-
Plugins are packaged as gems. See the [
|
311
|
+
Plugins are packaged as gems. See the [pushpop-project](https://github.com/pushpop-project) github organization for a sampling of popular plugins.
|
369
312
|
|
370
|
-
|
313
|
+
### Creating plugins
|
371
314
|
|
372
315
|
Plugins are just subclasses of `Pushpop::Step`. Plugins should implement a `run` method and register themselves. Here's a simple plugin that stops job execution if the input into the step is 0:
|
373
316
|
|
374
317
|
``` ruby
|
318
|
+
require 'pushpop'
|
319
|
+
|
375
320
|
module Pushpop
|
376
321
|
class StopIfZero < Step
|
377
322
|
PLUGIN_NAME = 'stop_if_zero'
|
@@ -392,41 +337,11 @@ job do
|
|
392
337
|
end
|
393
338
|
```
|
394
339
|
|
395
|
-
|
396
|
-
|
397
|
-
Pushpop can also be embedded in existing Ruby projects as a Ruby gem. Here's some steps on how to do that.
|
398
|
-
|
399
|
-
##### Install the gem
|
400
|
-
|
401
|
-
``` ruby
|
402
|
-
# bundler
|
403
|
-
gem 'pushpop'
|
404
|
-
|
405
|
-
# not bundler
|
406
|
-
gem install 'pushpop'
|
407
|
-
```
|
408
|
-
|
409
|
-
##### Require job files and run
|
410
|
-
|
411
|
-
Once the gem is available you can load or require Pushpop job files. Once each file loads the jobs it contains are ready to be run or scheduled. Here's that sequence:
|
412
|
-
|
413
|
-
``` ruby
|
414
|
-
load 'some_job.rb'
|
415
|
-
|
416
|
-
# you could run the jobs once
|
417
|
-
Pushpop.run
|
418
|
-
|
419
|
-
# or schedule and run the jobs with clockwork
|
420
|
-
Pushpop.schedule
|
421
|
-
Clockwork.manager.run
|
422
|
-
```
|
423
|
-
|
424
|
-
The `pushpop` gem does not declare dependencies other than `clockwork` and `keen`. If you're using
|
425
|
-
Pushpop plugins like Sendgrid or Twilio you'll need to bundle and require those dependencies separately.
|
340
|
+
See [pushpop-plugin](https://github.com/pushpop-project/pushpop-plugin) for a repository that you can clone to make creating and packaging plugins easier.
|
426
341
|
|
427
342
|
## Contributing
|
428
343
|
|
429
|
-
Issues and pull requests are very welcome!
|
344
|
+
Issues and pull requests are very welcome. One of the goals of the pushpop-project is to get as many unique contributors as possible. Beginners welcome too!
|
430
345
|
|
431
346
|
##### Wishlist
|
432
347
|
|
@@ -442,10 +357,3 @@ Please make sure the specs pass before you submit your pull request. Pushpop has
|
|
442
357
|
``` shell
|
443
358
|
$ bundle exec rake spec
|
444
359
|
```
|
445
|
-
|
446
|
-
## Inspirations
|
447
|
-
|
448
|
-
> "Technology shouldn't require all of our attention, just some of it, and only when necessary."
|
449
|
-
> [calmtechnology.com](http://calmtechnology.com/)
|
450
|
-
|
451
|
-
Dashboards and reports are human presentation vehicles. They require our attention in order to gain meaning. That's great when we're actively seeking answers and want to explore. But as a means to become aware of interesting, timely events it's neither effective nor efficient. A tool like Pushpop works better in those cases. It's a calmer technology.
|
data/Rakefile
CHANGED
@@ -12,37 +12,3 @@ begin
|
|
12
12
|
task default: :spec
|
13
13
|
rescue LoadError
|
14
14
|
end
|
15
|
-
|
16
|
-
def require_jobfiles(args)
|
17
|
-
require 'pushpop'
|
18
|
-
if jobfile = args[:jobfile]
|
19
|
-
load "#{File.dirname(__FILE__)}/#{jobfile}"
|
20
|
-
else
|
21
|
-
Dir.glob("#{File.dirname(__FILE__)}/jobs/**/*.rb").each { |file|
|
22
|
-
require file
|
23
|
-
}
|
24
|
-
end
|
25
|
-
end
|
26
|
-
|
27
|
-
namespace :jobs do
|
28
|
-
desc 'Describe jobs'
|
29
|
-
task :describe, :jobfile do |_, args|
|
30
|
-
require_jobfiles(args)
|
31
|
-
Pushpop.jobs.each do |job|
|
32
|
-
puts job.name
|
33
|
-
end
|
34
|
-
end
|
35
|
-
|
36
|
-
desc 'Run each job once'
|
37
|
-
task :run_once, :jobfile do |_, args|
|
38
|
-
require_jobfiles(args)
|
39
|
-
Pushpop.run
|
40
|
-
end
|
41
|
-
|
42
|
-
desc 'Run jobs ongoing'
|
43
|
-
task :run, :jobfile do |_, args|
|
44
|
-
require_jobfiles(args)
|
45
|
-
Pushpop.schedule
|
46
|
-
Clockwork.manager.run
|
47
|
-
end
|
48
|
-
end
|
data/bin/pushpop
ADDED
data/lib/pushpop.rb
CHANGED
data/lib/pushpop/cli.rb
ADDED
@@ -0,0 +1,77 @@
|
|
1
|
+
require 'thor'
|
2
|
+
require 'dotenv'
|
3
|
+
require 'pushpop'
|
4
|
+
|
5
|
+
module Pushpop
|
6
|
+
class CLI < Thor
|
7
|
+
|
8
|
+
def self.file_options
|
9
|
+
option :file, :aliases => '-f'
|
10
|
+
end
|
11
|
+
|
12
|
+
desc 'version', 'Print the Pushpop version'
|
13
|
+
map %w(-v --version) => :version
|
14
|
+
|
15
|
+
def version
|
16
|
+
"Pushpop version #{Pushpop::VERSION}".tap do |s|
|
17
|
+
puts s
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
desc 'jobs:describe', 'Describe jobs'
|
22
|
+
map 'jobs:describe' => 'describe_jobs'
|
23
|
+
file_options
|
24
|
+
|
25
|
+
def describe_jobs
|
26
|
+
Dotenv.load
|
27
|
+
require_file(options[:file])
|
28
|
+
Pushpop.jobs.tap do |jobs|
|
29
|
+
jobs.each do |job|
|
30
|
+
puts job.name
|
31
|
+
end
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
desc 'jobs:run_once', 'Run jobs once'
|
36
|
+
map 'jobs:run_once' => 'run_jobs_once'
|
37
|
+
file_options
|
38
|
+
|
39
|
+
def run_jobs_once
|
40
|
+
Dotenv.load
|
41
|
+
require_file(options[:file])
|
42
|
+
Pushpop.run
|
43
|
+
end
|
44
|
+
|
45
|
+
desc 'jobs:run', 'Run jobs ongoing'
|
46
|
+
map 'jobs:run' => 'run_jobs'
|
47
|
+
file_options
|
48
|
+
|
49
|
+
def run_jobs
|
50
|
+
Dotenv.load
|
51
|
+
require_file(options[:file])
|
52
|
+
Pushpop.schedule
|
53
|
+
Clockwork.manager.run
|
54
|
+
end
|
55
|
+
|
56
|
+
private
|
57
|
+
|
58
|
+
def require_file(file)
|
59
|
+
if file
|
60
|
+
if File.directory?(file)
|
61
|
+
Dir.glob("#{file}/**/*.rb").each { |file|
|
62
|
+
load "#{Dir.pwd}/#{file}"
|
63
|
+
}
|
64
|
+
else
|
65
|
+
load file
|
66
|
+
end
|
67
|
+
else
|
68
|
+
Dir.glob("#{Dir.pwd}/jobs/**/*.rb").each { |file|
|
69
|
+
load file
|
70
|
+
}
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
|
data/lib/pushpop/job.rb
CHANGED
data/lib/pushpop/version.rb
CHANGED
data/pushpop.gemspec
CHANGED
@@ -9,10 +9,12 @@ Gem::Specification.new do |s|
|
|
9
9
|
s.authors = ["Josh Dzielak"]
|
10
10
|
s.email = "josh@keen.io"
|
11
11
|
s.homepage = "https://github.com/pushpop-project/pushpop"
|
12
|
-
s.summary = "
|
13
|
-
s.description = "Pushpop is a
|
12
|
+
s.summary = "A framework for scheduled integrations between popular services"
|
13
|
+
s.description = "Pushpop is a powerful framework for taking actions and integrating services at regular intervals."
|
14
14
|
|
15
15
|
s.add_dependency "clockwork"
|
16
|
+
s.add_dependency "thor"
|
17
|
+
s.add_dependency "dotenv"
|
16
18
|
|
17
19
|
s.files = `git ls-files`.split("\n")
|
18
20
|
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
@@ -0,0 +1,35 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Pushpop::CLI do
|
4
|
+
|
5
|
+
def start(str=nil)
|
6
|
+
Pushpop::CLI.start(str ? str.split(" ") : [])
|
7
|
+
end
|
8
|
+
|
9
|
+
it 'prints help by default' do
|
10
|
+
_, options = start
|
11
|
+
expect(_).to be_empty
|
12
|
+
end
|
13
|
+
|
14
|
+
describe 'with -v' do
|
15
|
+
it 'prints the version' do
|
16
|
+
_, options = start('-v')
|
17
|
+
expect(_).to match('Pushpop version')
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
describe 'jobs:describe' do
|
22
|
+
it 'prints job information' do
|
23
|
+
_, options = start('jobs:describe --file spec/jobs')
|
24
|
+
expect(_.name).to eq('Simple Math')
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
describe 'jobs:run_once' do
|
29
|
+
it 'runs jobs once' do
|
30
|
+
_, options = start('jobs:run_once --file spec/jobs')
|
31
|
+
expect(_.first).to equal(30)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
end
|
data/spec/pushpop/job_spec.rb
CHANGED
@@ -6,44 +6,44 @@ describe Pushpop::Job do
|
|
6
6
|
let (:empty_step) { Pushpop::Step.new('bar') do end }
|
7
7
|
|
8
8
|
describe '#register_plugins' do
|
9
|
-
it '
|
9
|
+
it 'registers a plugin' do
|
10
10
|
Pushpop::Job.register_plugin('blaz', Class)
|
11
|
-
Pushpop::Job.plugins['blaz'].
|
11
|
+
expect(Pushpop::Job.plugins['blaz']).to eq(Class)
|
12
12
|
end
|
13
13
|
end
|
14
14
|
|
15
15
|
describe '#initialize' do
|
16
|
-
it '
|
16
|
+
it 'sets a name and evaluate a block' do
|
17
17
|
block_ran = false
|
18
18
|
job = Pushpop::Job.new('foo') do block_ran = true end
|
19
|
-
job.name.
|
20
|
-
job.period.
|
21
|
-
job.every_options.
|
22
|
-
block_ran.
|
19
|
+
expect(job.name).to eq('foo')
|
20
|
+
expect(job.period).to be_nil
|
21
|
+
expect(job.every_options).to eq({})
|
22
|
+
expect(block_ran).to be_truthy
|
23
23
|
end
|
24
24
|
|
25
|
-
it '
|
25
|
+
it 'auto-generates a name' do
|
26
26
|
job = Pushpop::Job.new do end
|
27
|
-
job.name.
|
27
|
+
expect(job.name).not_to be_nil
|
28
28
|
end
|
29
29
|
end
|
30
30
|
|
31
31
|
describe '#every' do
|
32
|
-
it '
|
32
|
+
it 'sets period and options' do
|
33
33
|
job = empty_job
|
34
34
|
job.every(10.seconds, at: '01:02')
|
35
|
-
job.period.
|
36
|
-
job.every_options.
|
35
|
+
expect(job.period).to eq(10)
|
36
|
+
expect(job.every_options).to eq({ at: '01:02' })
|
37
37
|
end
|
38
38
|
end
|
39
39
|
|
40
40
|
describe '#step' do
|
41
|
-
it '
|
41
|
+
it 'adds the step to the internal list of steps' do
|
42
42
|
empty_proc = Proc.new {}
|
43
43
|
job = empty_job
|
44
44
|
job.step('blah', &empty_proc)
|
45
|
-
job.steps.first.name.
|
46
|
-
job.steps.first.block.
|
45
|
+
expect(job.steps.first.name).to eq('blah')
|
46
|
+
expect(job.steps.first.block).to eq(empty_proc)
|
47
47
|
end
|
48
48
|
|
49
49
|
context 'plugin specified' do
|
@@ -54,17 +54,17 @@ describe Pushpop::Job do
|
|
54
54
|
Pushpop::Job.register_plugin('blaz', FakeStep)
|
55
55
|
end
|
56
56
|
|
57
|
-
it '
|
57
|
+
it 'uses the registered plugin to instantiate the class' do
|
58
58
|
empty_proc = Proc.new {}
|
59
59
|
job = empty_job
|
60
60
|
job.step('blah', 'blaz', &empty_proc)
|
61
|
-
job.steps.first.name.
|
62
|
-
job.steps.first.plugin.
|
63
|
-
job.steps.first.class.
|
64
|
-
job.steps.first.block.
|
61
|
+
expect(job.steps.first.name).to eq('blah')
|
62
|
+
expect(job.steps.first.plugin).to eq('blaz')
|
63
|
+
expect(job.steps.first.class).to eq(FakeStep)
|
64
|
+
expect(job.steps.first.block).to eq(empty_proc)
|
65
65
|
end
|
66
66
|
|
67
|
-
it '
|
67
|
+
it 'throws an exception for an unregistered plugin' do
|
68
68
|
empty_proc = Proc.new {}
|
69
69
|
job = empty_job
|
70
70
|
expect {
|
@@ -75,7 +75,7 @@ describe Pushpop::Job do
|
|
75
75
|
end
|
76
76
|
|
77
77
|
describe '#run' do
|
78
|
-
it '
|
78
|
+
it 'calls each step with the response to the previous' do
|
79
79
|
job = Pushpop::Job.new('foo') do
|
80
80
|
step 'one' do
|
81
81
|
10
|
@@ -85,12 +85,12 @@ describe Pushpop::Job do
|
|
85
85
|
response + 20
|
86
86
|
end
|
87
87
|
end
|
88
|
-
job.run.
|
88
|
+
expect(job.run).to eq([30, { 'one' => 10, 'two' => 30 }])
|
89
89
|
end
|
90
90
|
end
|
91
91
|
|
92
92
|
describe '#schedule' do
|
93
|
-
it '
|
93
|
+
it 'adds the job to clockwork' do
|
94
94
|
period = 1.seconds
|
95
95
|
simple_job = Pushpop::Job.new('foo') do
|
96
96
|
every period
|
@@ -103,12 +103,12 @@ describe Pushpop::Job do
|
|
103
103
|
simple_job.schedule
|
104
104
|
|
105
105
|
Clockwork.manager.tick(Time.now)
|
106
|
-
simple_job.run.first.
|
106
|
+
expect(simple_job.run.first).to eq(2)
|
107
107
|
Clockwork.manager.tick(Time.now + period)
|
108
|
-
simple_job.run.first.
|
108
|
+
expect(simple_job.run.first).to eq(4)
|
109
109
|
end
|
110
110
|
|
111
|
-
it '
|
111
|
+
it 'fails if the period was not specified' do
|
112
112
|
simple_job = Pushpop::Job.new('foo') do end
|
113
113
|
expect {
|
114
114
|
simple_job.schedule
|
@@ -123,26 +123,26 @@ describe Pushpop::Job do
|
|
123
123
|
before do
|
124
124
|
end
|
125
125
|
|
126
|
-
it '
|
126
|
+
it 'assumes its a registered plugin name and try to create a step' do
|
127
127
|
Pushpop::Job.register_plugin('blaz', FakeStep)
|
128
128
|
simple_job = job do
|
129
129
|
blaz 'hi' do end
|
130
130
|
end
|
131
|
-
simple_job.steps.first.name.
|
132
|
-
simple_job.steps.first.class.
|
131
|
+
expect(simple_job.steps.first.name).to eq('hi')
|
132
|
+
expect(simple_job.steps.first.class).to eq(FakeStep)
|
133
133
|
end
|
134
134
|
|
135
|
-
it '
|
135
|
+
it 'does not assume a name' do
|
136
136
|
Pushpop::Job.register_plugin('blaz', FakeStep)
|
137
137
|
simple_job = job do
|
138
138
|
blaz do end
|
139
139
|
end
|
140
|
-
simple_job.steps.first.name.
|
141
|
-
simple_job.steps.first.plugin.
|
142
|
-
simple_job.steps.first.class.
|
140
|
+
expect(simple_job.steps.first.name).not_to be_nil
|
141
|
+
expect(simple_job.steps.first.plugin).to eq('blaz')
|
142
|
+
expect(simple_job.steps.first.class).to eq(FakeStep)
|
143
143
|
end
|
144
144
|
|
145
|
-
it '
|
145
|
+
it 'raises an exception if there is no registered plugin' do
|
146
146
|
expect {
|
147
147
|
job do
|
148
148
|
blaze do end
|
data/spec/pushpop/step_spec.rb
CHANGED
@@ -6,72 +6,72 @@ describe Pushpop::Step do
|
|
6
6
|
|
7
7
|
describe 'initialize' do
|
8
8
|
|
9
|
-
it '
|
9
|
+
it 'sets a name, a plugin, and a block' do
|
10
10
|
empty_proc = Proc.new {}
|
11
11
|
step = Pushpop::Step.new('foo', 'foopie', &empty_proc)
|
12
|
-
step.name.
|
13
|
-
step.plugin.
|
14
|
-
step.block.
|
12
|
+
expect(step.name).to eq('foo')
|
13
|
+
expect(step.plugin).to eq('foopie')
|
14
|
+
expect(step.block).to eq(empty_proc)
|
15
15
|
end
|
16
16
|
|
17
|
-
it '
|
17
|
+
it 'auto-generates a name if not given and plugin not given' do
|
18
18
|
empty_proc = Proc.new {}
|
19
19
|
step = Pushpop::Step.new(&empty_proc)
|
20
|
-
step.name.
|
21
|
-
step.plugin.
|
22
|
-
step.block.
|
20
|
+
expect(step.name).not_to be_nil
|
21
|
+
expect(step.plugin).to be_nil
|
22
|
+
expect(step.block).to eq(empty_proc)
|
23
23
|
end
|
24
24
|
|
25
|
-
it '
|
25
|
+
it 'sets name to plugin name if not given' do
|
26
26
|
empty_proc = Proc.new {}
|
27
27
|
step = Pushpop::Step.new(nil, 'whee', &empty_proc)
|
28
|
-
step.name.
|
29
|
-
step.plugin.
|
30
|
-
step.block.
|
28
|
+
expect(step.name).to eq('whee')
|
29
|
+
expect(step.plugin).to eq('whee')
|
30
|
+
expect(step.block).to eq(empty_proc)
|
31
31
|
end
|
32
32
|
|
33
|
-
it '
|
33
|
+
it 'does not require a plugin' do
|
34
34
|
empty_proc = Proc.new {}
|
35
35
|
step = Pushpop::Step.new('foo', &empty_proc)
|
36
|
-
step.name.
|
37
|
-
step.block.
|
36
|
+
expect(step.name).to eq('foo')
|
37
|
+
expect(step.block).to eq(empty_proc)
|
38
38
|
end
|
39
39
|
|
40
40
|
end
|
41
41
|
|
42
42
|
describe 'run' do
|
43
43
|
|
44
|
-
it '
|
44
|
+
it 'calls the block with the same args' do
|
45
45
|
arg1, arg2 = nil
|
46
46
|
times_run = 0
|
47
47
|
empty_proc = Proc.new { |a1, a2| arg1 = a1; arg2 = a2; times_run += 1 }
|
48
48
|
step = Pushpop::Step.new('foo', &empty_proc)
|
49
49
|
step.run('foo', 'bar')
|
50
|
-
arg1.
|
51
|
-
arg2.
|
52
|
-
times_run.
|
50
|
+
expect(arg1).to eq('foo')
|
51
|
+
expect(arg2).to eq('bar')
|
52
|
+
expect(times_run).to eq(1)
|
53
53
|
end
|
54
54
|
|
55
|
-
it '
|
55
|
+
it 'executes the block bound to the step' do
|
56
56
|
_self = nil
|
57
57
|
step = Pushpop::Step.new(nil, nil) do
|
58
58
|
_self = self
|
59
59
|
end
|
60
60
|
step.run
|
61
|
-
_self.
|
61
|
+
expect(_self).to eq(step)
|
62
62
|
end
|
63
63
|
|
64
64
|
end
|
65
65
|
|
66
66
|
describe 'template' do
|
67
|
-
it '
|
67
|
+
it 'renders the named template with the response binding' do
|
68
68
|
step = Pushpop::Step.new
|
69
|
-
step.template('spec.html.erb', 500, {}, SPEC_TEMPLATES_DIRECTORY).strip.
|
69
|
+
expect(step.template('spec.html.erb', 500, {}, SPEC_TEMPLATES_DIRECTORY).strip).to eq('<pre>500</pre>')
|
70
70
|
end
|
71
71
|
|
72
|
-
it '
|
72
|
+
it 'renders the named template with the step_response binding' do
|
73
73
|
step = Pushpop::Step.new
|
74
|
-
step.template('spec.html.erb', nil, { test: 600 }, SPEC_TEMPLATES_DIRECTORY).strip.
|
74
|
+
expect(step.template('spec.html.erb', nil, { test: 600 }, SPEC_TEMPLATES_DIRECTORY).strip).to eq('<pre>600</pre>')
|
75
75
|
end
|
76
76
|
end
|
77
77
|
|
data/spec/pushpop_spec.rb
CHANGED
@@ -1,24 +1,26 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
|
3
3
|
describe 'job' do
|
4
|
-
|
5
|
-
|
4
|
+
it 'has a name' do
|
5
|
+
job 'foo-main' do end
|
6
|
+
expect(Pushpop.jobs.first.name).to eq('foo-main')
|
7
|
+
end
|
6
8
|
end
|
7
9
|
|
8
10
|
describe Pushpop do
|
9
11
|
|
10
12
|
describe 'add_job' do
|
11
|
-
it '
|
13
|
+
it 'adds a job to the list' do
|
12
14
|
empty_proc = Proc.new {}
|
13
15
|
Pushpop.add_job('foo', &empty_proc)
|
14
|
-
Pushpop.jobs.first.name.
|
16
|
+
expect(Pushpop.jobs.first.name).to eq('foo')
|
15
17
|
end
|
16
18
|
end
|
17
19
|
|
18
20
|
describe 'random_name' do
|
19
|
-
it '
|
20
|
-
Pushpop.random_name.
|
21
|
+
it 'is 8 characters and alphanumeric' do
|
22
|
+
expect(Pushpop.random_name).to match(/^\w{8}$/)
|
21
23
|
end
|
22
24
|
end
|
23
25
|
|
24
|
-
end
|
26
|
+
end
|
data/spec/simple_job_spec.rb
CHANGED
@@ -1,9 +1,9 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
|
3
3
|
describe 'run a job end to end' do
|
4
|
-
it '
|
4
|
+
it 'runs and return template contents' do
|
5
5
|
require File.expand_path('../jobs/simple_job', __FILE__)
|
6
|
-
Pushpop.jobs.length.
|
7
|
-
Pushpop.jobs.first.run.
|
6
|
+
expect(Pushpop.jobs.length).to eq(1)
|
7
|
+
expect(Pushpop.jobs.first.run).to eq([30, { "return 10" => 10, "increase by 20" => 30}])
|
8
8
|
end
|
9
|
-
end
|
9
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pushpop
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.2
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-
|
12
|
+
date: 2014-07-17 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: clockwork
|
@@ -27,28 +27,61 @@ dependencies:
|
|
27
27
|
- - ! '>='
|
28
28
|
- !ruby/object:Gem::Version
|
29
29
|
version: '0'
|
30
|
-
|
31
|
-
|
30
|
+
- !ruby/object:Gem::Dependency
|
31
|
+
name: thor
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
33
|
+
none: false
|
34
|
+
requirements:
|
35
|
+
- - ! '>='
|
36
|
+
- !ruby/object:Gem::Version
|
37
|
+
version: '0'
|
38
|
+
type: :runtime
|
39
|
+
prerelease: false
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
41
|
+
none: false
|
42
|
+
requirements:
|
43
|
+
- - ! '>='
|
44
|
+
- !ruby/object:Gem::Version
|
45
|
+
version: '0'
|
46
|
+
- !ruby/object:Gem::Dependency
|
47
|
+
name: dotenv
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
49
|
+
none: false
|
50
|
+
requirements:
|
51
|
+
- - ! '>='
|
52
|
+
- !ruby/object:Gem::Version
|
53
|
+
version: '0'
|
54
|
+
type: :runtime
|
55
|
+
prerelease: false
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
57
|
+
none: false
|
58
|
+
requirements:
|
59
|
+
- - ! '>='
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
description: Pushpop is a powerful framework for taking actions and integrating services
|
63
|
+
at regular intervals.
|
32
64
|
email: josh@keen.io
|
33
|
-
executables:
|
65
|
+
executables:
|
66
|
+
- pushpop
|
34
67
|
extensions: []
|
35
68
|
extra_rdoc_files: []
|
36
69
|
files:
|
37
70
|
- .gitignore
|
38
71
|
- .travis.yml
|
39
72
|
- Gemfile
|
40
|
-
- Gemfile.lock
|
41
73
|
- LICENSE
|
42
|
-
- Procfile
|
43
74
|
- README.md
|
44
75
|
- Rakefile
|
45
|
-
-
|
76
|
+
- bin/pushpop
|
46
77
|
- lib/pushpop.rb
|
78
|
+
- lib/pushpop/cli.rb
|
47
79
|
- lib/pushpop/job.rb
|
48
80
|
- lib/pushpop/step.rb
|
49
81
|
- lib/pushpop/version.rb
|
50
82
|
- pushpop.gemspec
|
51
83
|
- spec/jobs/simple_job.rb
|
84
|
+
- spec/pushpop/cli_spec.rb
|
52
85
|
- spec/pushpop/job_spec.rb
|
53
86
|
- spec/pushpop/step_spec.rb
|
54
87
|
- spec/pushpop_spec.rb
|
@@ -79,12 +112,14 @@ rubyforge_project:
|
|
79
112
|
rubygems_version: 1.8.23
|
80
113
|
signing_key:
|
81
114
|
specification_version: 3
|
82
|
-
summary:
|
115
|
+
summary: A framework for scheduled integrations between popular services
|
83
116
|
test_files:
|
84
117
|
- spec/jobs/simple_job.rb
|
118
|
+
- spec/pushpop/cli_spec.rb
|
85
119
|
- spec/pushpop/job_spec.rb
|
86
120
|
- spec/pushpop/step_spec.rb
|
87
121
|
- spec/pushpop_spec.rb
|
88
122
|
- spec/simple_job_spec.rb
|
89
123
|
- spec/spec_helper.rb
|
90
124
|
- spec/templates/spec.html.erb
|
125
|
+
has_rdoc:
|
data/Gemfile.lock
DELETED
@@ -1,36 +0,0 @@
|
|
1
|
-
GEM
|
2
|
-
remote: https://rubygems.org/
|
3
|
-
specs:
|
4
|
-
activesupport (4.1.0)
|
5
|
-
i18n (~> 0.6, >= 0.6.9)
|
6
|
-
json (~> 1.7, >= 1.7.7)
|
7
|
-
minitest (~> 5.1)
|
8
|
-
thread_safe (~> 0.1)
|
9
|
-
tzinfo (~> 1.1)
|
10
|
-
clockwork (0.7.4)
|
11
|
-
activesupport
|
12
|
-
tzinfo
|
13
|
-
diff-lcs (1.2.5)
|
14
|
-
i18n (0.6.9)
|
15
|
-
json (1.8.1)
|
16
|
-
minitest (5.3.3)
|
17
|
-
rake (10.3.1)
|
18
|
-
rspec (2.14.1)
|
19
|
-
rspec-core (~> 2.14.0)
|
20
|
-
rspec-expectations (~> 2.14.0)
|
21
|
-
rspec-mocks (~> 2.14.0)
|
22
|
-
rspec-core (2.14.8)
|
23
|
-
rspec-expectations (2.14.5)
|
24
|
-
diff-lcs (>= 1.1.3, < 2.0)
|
25
|
-
rspec-mocks (2.14.6)
|
26
|
-
thread_safe (0.3.3)
|
27
|
-
tzinfo (1.1.0)
|
28
|
-
thread_safe (~> 0.1)
|
29
|
-
|
30
|
-
PLATFORMS
|
31
|
-
ruby
|
32
|
-
|
33
|
-
DEPENDENCIES
|
34
|
-
clockwork
|
35
|
-
rake
|
36
|
-
rspec
|
data/Procfile
DELETED
@@ -1 +0,0 @@
|
|
1
|
-
worker: bundle exec rake jobs:run
|
data/jobs/example_job.rb
DELETED
@@ -1,20 +0,0 @@
|
|
1
|
-
require 'pushpop'
|
2
|
-
|
3
|
-
job 'Simple Math' do
|
4
|
-
|
5
|
-
every 1.minutes
|
6
|
-
|
7
|
-
step 'return 10' do 10 end
|
8
|
-
|
9
|
-
step 'increase by 20' do |response|
|
10
|
-
20 + response
|
11
|
-
end
|
12
|
-
|
13
|
-
step 'print out via template' do |response|
|
14
|
-
html = template 'first_template.html.erb', response
|
15
|
-
puts 'Hey Pushpop, let\'s do a math!'
|
16
|
-
puts html
|
17
|
-
html
|
18
|
-
end
|
19
|
-
|
20
|
-
end
|