pushpop 0.1.0 → 0.1.1

data/Gemfile

@@ -1,13 +1,8 @@
 source 'https://rubygems.org'
 
 gem 'clockwork'
-gem 'keen'
-gem 'twilio-ruby'
-gem 'mail'
-gem 'foreman'
 
 group :development, :test do
   gem 'rake'
   gem 'rspec'
-  gem 'webmock'
 end

data/Gemfile.lock

@@ -7,32 +7,13 @@ GEM
       minitest (~> 5.1)
       thread_safe (~> 0.1)
       tzinfo (~> 1.1)
-    addressable (2.3.6)
-    builder (3.2.2)
     clockwork (0.7.4)
       activesupport
       tzinfo
-    crack (0.4.2)
-      safe_yaml (~> 1.0.0)
     diff-lcs (1.2.5)
-    dotenv (0.7.0)
-    foreman (0.67.0)
-      dotenv (~> 0.7.0)
-      thor (~> 0.17.0)
     i18n (0.6.9)
     json (1.8.1)
-    jwt (0.1.11)
-      multi_json (>= 1.5)
-    keen (0.8.1)
-      addressable (~> 2.3.5)
-      multi_json (~> 1.3)
-    mail (2.5.4)
-      mime-types (~> 1.16)
-      treetop (~> 1.4.8)
-    mime-types (1.25.1)
     minitest (5.3.3)
-    multi_json (1.9.2)
-    polyglot (0.3.4)
     rake (10.3.1)
     rspec (2.14.1)
       rspec-core (~> 2.14.0)
@@ -42,31 +23,14 @@ GEM
     rspec-expectations (2.14.5)
       diff-lcs (>= 1.1.3, < 2.0)
     rspec-mocks (2.14.6)
-    safe_yaml (1.0.2)
-    thor (0.17.0)
     thread_safe (0.3.3)
-    treetop (1.4.15)
-      polyglot
-      polyglot (>= 0.3.1)
-    twilio-ruby (3.11.5)
-      builder (>= 2.1.2)
-      jwt (>= 0.1.2)
-      multi_json (>= 1.3.0)
     tzinfo (1.1.0)
       thread_safe (~> 0.1)
-    webmock (1.17.4)
-      addressable (>= 2.2.7)
-      crack (>= 0.3.2)
 
 PLATFORMS
   ruby
 
 DEPENDENCIES
   clockwork
-  foreman
-  keen
-  mail
   rake
   rspec
-  twilio-ruby
-  webmock

data/README.md

@@ -1,7 +1,10 @@
-# Pushpop
-[![Build Status](https://travis-ci.org/keenlabs/pushpop.svg)](https://travis-ci.org/keenlabs/pushpop)
+<p align="center">
+<img align="center" src="http://f.cl.ly/items/0z182V2i3X0Q0Y3I2A2G/pushpop-logo.png"> 
+</p>
 
-### Automatic delivery of regular reports and alerts 
+[![Build Status](https://travis-ci.org/pushpop-project/pushpop.svg)](https://travis-ci.org/pushpop-project/pushpop)
+
+### A framework for scheduled integrations
 
 <hr>
 <img src="http://f.cl.ly/items/1I421w263a10340a0u2q/Screen%20Shot%202014-04-16%20at%204.35.47%20PM.png" width="45%" alt="Pingpong Daily Response Time Report">
@@ -11,101 +14,108 @@
 
 ## Overview
 
-Pushpop is a simple but powerful Ruby app that sends notifications based on events captured with Keen IO.
+Pushpop is a simple but powerful Ruby gem that lets you share data between services at regular intervals.
+This can be used to do things like regular data collection as well as triggering alerts based on patterns in data.
+
+Pushpop includes support for sending notifications and reports based on events captured with [Keen IO](https://keen.io).
+See plugins for more services on the [Pushpop organization](https://github.com/pushpop-project) home page.
 
 #### Ways to use Pushpop
 
-**Regular reports**
+##### Alerts
 
-+ Send a sales report to your inbox every day at noon
-+ Send analytics reports to your customers every week
++ Send an email when your site has been busier than usual in the last hour
++ Send an SMS if the performance of your signup funnel dramatically changes
 
-**Alerts**
+##### Recurring reports
 
-+ Send an SMS if the performance of your signup funnel dramatically changes
-+ Send an email when your site has been busier than usual in the last hour
++ Send a sales report to your inbox every day at noon
++ Send analytics reports to your customers every week
 
 #### An example Pushpop job
 
-Here's a simple Pushpop job that uses [Twilio](https://twilio.com/) to send an SMS containing the number of daily pageviews each night at midnight:
+This Pushpop job uses the `keen` and `sendgrid` plugins to send a nightly email containing the day's number of pageviews:
 
 ``` ruby
-require 'pushpop'
+require 'pushpop-keen'
+require 'pushpop-sendgrid'
 
 job do
 
   every 24.hours, at: '00:00'
 
   keen do
-    event_collection 'pageviews'
-    analysis_type 'count'
-    timeframe 'last_24_hours'
+    event_collection  'pageviews'
+    analysis_type     'count'
+    timeframe         'last_24_hours'
   end
 
-  twilio do |response|
-    to '+18005555555'
-    body "There were #{response} pageviews today!"
+  sendgrid do |response, _|
+    to        'josh+pushpop@keen.io'
+    from      'josh+pushpop@keen.io'
+    subject   'Pushpop Daily Pageviews Report'
+    body      "There were #{response} pageviews today!"
   end
 
 end
 ```
 
-Pushpop syntax is short and sweet, but because anything Ruby can be used it's also quite powerful.
+The email is sent by [Sendgrid](https://sendgrid.com), made possible by the [sendgrid](https://github.com/pushpop-project/pushpop-sendgrid) Pushpop plugin.
 
-### Where to next?
+Pushpop syntax is short and sweet, but because Pushpop is just Ruby it's also quite powerful.
 
-Excited to try out Pushpop with your data? Here's a few options to choose from:
+### Get Started
 
-#### Quickstart
+Excited to try out Pushpop with your Keen IO projects? Here's a few options to choose from:
 
-Setup Pushpop locally. It takes 10 minutes to get that first shiny report in your inbox, and even less if you already have a Keen IO, Sendgrid or Twilio account.
+#### The Quickstart
+
+Setup Pushpop locally and run your first job in minutes.
 
 **[Go to the Quickstart](#quickstart)**
 
 #### Deploy a Pushpop Instance
 
-Ready to deploy the Pushpop job you wrote locally and start getting regular reports? Detailed instructions for Heroku are provided, as well as the basics for other platforms.
+Ready to deploy a local Pushpop job? Detailed instructions for Heroku are provided as well as a basic guide for other platforms.
 
 **[Go to the Deploy Guide](#deploy-guide)**
 
 #### Need help?
 
-Don't have a hacker at hand? The friendly folks at Keen IO are happy to help you get a Pushpop instance running.
+Don't have a hacker on hand? The friendly folks at Keen IO can set a Pushpop up for you.
 
-**Email [team@keen.io](mailto:team@keen.io?subject=I want a Pushpop!)** with the subject "I want a Pushpop!"
+**Email [team@keen.io](mailto:team@keen.io?subject=I want a Pushpop!)** with the subject "I want a Pushpop!". Include information about what queries you'd like to run (and when) and how you'd like the results communicated.
 
 ## Quickstart
 
-The goal of the Quickstart is to get a Pushpop instance running locally. This should take less than 10 minutes.
+The goal of the Quickstart is to get a Pushpop instance running locally and write your first job. This should take less than 10 minutes.
 
 #### Prerequisites
 
-+ A working Ruby installation (1.9+)
-+ A [Keen IO](https://keen.io) account and project and associated API keys
-+ A [Sendgrid](https://sendgrid.com) and/or [Twilio](https://twilio.com) account and associated API keys
++ A working [Ruby installation](https://www.ruby-lang.org/en/installation/) (1.9+)
 
 #### Steps
 
-**Clone this repository**
+##### Clone the Pushpop starter project
 
 ``` shell
-$ git clone git@github.com:keenlabs/pushpop.git
+$ git clone git@github.com:pushpop-project/pushpop-starter.git
 ```
 
-Enter the pushpop directory and install dependencies.
+Enter the `pushpop-starter` directory and install dependencies.
 
 ``` shell
-$ cd pushpop
+$ cd pushpop-starter
 $ gem install bundler
 $ bundle install
 ```
 
-**Test an example job**
+##### Test the included job
 
-There is an example job in [jobs/example_job.rb](jobs/example_job.rb). All it does is print some output to the console. Run this job via a rake task to make sure your configuration is setup properly.
+There is an example job in `jobs/example_job.rb` of the pushpop-starter repository. It simply prints output to the console. Run this job via a rake task to make sure your configuration is setup properly.
 
 ``` shell
-$ foreman run rake jobs:run_once[jobs/example_job.rb]
+$ bundle exec rake jobs:run_once[jobs/example_job.rb]
 ```
 
 You should see the following output (followed by a logging statement):
@@ -115,92 +125,35 @@ Hey Pushpop, let's do a math!
 <pre>The number 30!</pre>
 ```
 
-**Specify your API credentials**
-
-Now it's time to write a job that connects to APIs and does something real. For that we'll need to specify API keys. We'll use [foreman](https://github.com/ddollar/foreman) to tell Pushpop about these API keys. When you use foreman to run a process, it adds variables from a local `.env` file to the process environment. It's very handy for keeping secure API keys out of your code (`.env` files are gitignored by Pushpop).
-
-Create a `.env` file in the project directory and add the API configuration properties and keys that you have. Here's what an example file looks like with settings from all three services:
-
-```
-KEEN_PROJECT_ID=*********
-KEEN_READ_KEY=*********
-SENDGRID_DOMAIN=*********
-SENDGRID_PASSWORD=*********
-SENDGRID_USERNAME=*********
-TWILIO_AUTH_TOKEN=*********
-TWILIO_FROM=*********
-TWILIO_SID=*********
-```
-
-**Write your first job**
-
-Let's write a job that performs a count of one of your Keen IO collections and sends an email (or SMS) with the result. We'll set it to run every 24 hours.
-
-Create a file in the `jobs` folder called `first_job.rb` and paste in the following example:
-
-``` ruby
-job do
-
-  # how frequently do we want this job to run?
-  every 24.hours
-
-  # what keen io query should be performed?
-  keen do
-    event_collection '<my-keen-collection-name>'
-    analysis_type 'count'
-    timeframe 'last_24_hours'
-  end
-
-  # use this block to send an email
-  sendgrid do |_, step_responses|
-    to '<my-to-email-address>'
-    from '<my-from-email-address>'
-    subject "There were #{step_responses['keen']} events in the last 24 hours!"
-    body 'Blowing up!'
-  end
-  
-  # use this block to send an sms
-  twilio do |_, step_responses|
-    to '<to-phone-number>'
-    body "There were #{step_responses['keen']} events in the last 24 hours!"
-  end
-end
-```
-
-Now modify the example to use your specific information. You'll want to specify a `to` and a `from` address if you're using Sendgrid, and a `to` phone number if you're using Twilio. Everything you need to change is marked with `<>`. You'll also want to remove either Sendgrid or Twilio blocks you're not using them.
+That's it. You're ready to add your own jobs to the jobs folder.
 
-Save the file and test this job using the same `jobs:run_once` rake task that we used before.
-
-``` shell
-$ foreman run rake jobs:run_once[jobs/first_job.rb]
-```
-
-The output of each step will be logged to the console, and if everything worked you'll receive an email or a text message within a few seconds!
-
-**Next steps**
+##### Next steps
 
 + Write and test more jobs. See the [Pushpop API Documentation](#pushpop-api-documentation) below for more examples of what you can do.
-+ Continue on to the deploy guide to deploy a Pushpop instance and start getting regular reports and alerts.
++ See [pushpop-recipes](https://github.com/pushpop-project/pushpop-recipes) for reusable code and inspiration.
++ Continue on to the [Deploy Guide](#deploy-guide) to deploy the job you just created.
 
 ## Deploy Guide
 
-##### Heroku
+#### Heroku
 
 These instructions are for Heroku, but should be relevant to most environments.
 
-**Prerequisites**
+##### Prerequisites
 
-You'll need a Heroku account, and the [Heroku toolbelt](https://toolbelt.heroku.com/) installed.
+You'll need a [Heroku](https://heroku.com/) account, and the [Heroku toolbelt](https://toolbelt.heroku.com/) installed.
 
-**Create a new Heroku app**
+##### Create a new Heroku app
 
-Make sure you're inside the Pushpop directory.
+Make sure you're inside a Pushpop project directory (e.g. pushpop-starter), than create a new Heroku app.
 
 ``` shell
 $ heroku create
 ```
 
-**Commit changes**
+This will create a Heroku app and add a new git remote destination called `heroku` to your git configuration.
+
+##### Commit changes
 
 If you created a new job from the Quickstart guide, you'll want to commit that code before deploying.
 
@@ -208,24 +161,16 @@ If you created a new job from the Quickstart guide, you'll want to commit that c
 $ git commit -am 'Created my first Pushpop job'
 ```
 
-**Set Heroku config variables**
-
-The easiest way to do this is with the [heroku-config](https://github.com/ddollar/heroku-config) plugin. This step assumes you have created a `.env` file containing your keys as demonstrated in the Quickstart guide.
-
-``` shell
-$ heroku plugins:install git://github.com/ddollar/heroku-config.git
-$ heroku config:push
-```
-
-**Deploy code to Heroku**
+##### Deploy code to Heroku
 
-Now that your code is commited and config variables pushed we can begin a deploy.
+Now that your code is commited and config variables pushed we can begin a deploy. We'll also need to scale the number of worker processes to 1.
 
 ``` shell
 $ git push heroku master
+$ heroku scale worker=1
 ```
 
-**Tail logs to confirm it's working**
+##### Tail logs to confirm it's working
 
 To see that jobs are running and that there are no errors, tail the logs on Heroku.
 
@@ -235,12 +180,11 @@ $ heroku logs --tail
 
 Note that if you have jobs that are set to run at specific times of day you might not see output for a while.
 
-Also note - by default this will run all jobs in the `jobs` folder. You might want to delete the `example_job.rb` file in
-a separate commit once you've got the hang of things.
+Another note - by default this will run all jobs in the `jobs` folder. You might want to delete the `example_job.rb` file in a separate commit once you've got the hang of things. You can change this behavior by editing the Procfile.
 
-##### Other environments
+#### Other environments
 
-Pushpop is run entirely by one long-running Ruby process. Anywhere you can run this process in a monitored fashion you can run a Pushpop instance. Here's the command:
+Pushpop is deployed as one long-running Ruby process. Anywhere you can run this process you can run Pushpop. Here's the command:
 
 ``` shell
 $ foreman run rake jobs:run
@@ -252,14 +196,18 @@ If you don't want to use foreman and prefer to set the environment variables you
 $ bundle exec rake jobs:run
 ```
 
+Note: You probably want to monitor the process via something like [supervisord](http://supervisord.org/).
+
 ## Rake Tasks
 
-All `jobs:*` rake tasks optionally take a single filename as a parameter. The file is meant to contain one or more Pushpop jobs. If no filename is specified, all jobs in the jobs folder are considered.
+Pushpop comes with some rake tasks to make command line interaction and deployment easier.
+
+All `jobs:*` rake tasks optionally take a single filename as a parameter. The file is meant to contain one or more Pushpop jobs. If no filename is specified, all jobs in the jobs folder are required.
 
 Specifying a specific file looks like this:
 
 ``` shell
-$ foreman run rake jobs:run[jobs/just_this_job.rb]
+$ bundle exec rake jobs:run[jobs/just_this_job.rb]
 ```
 
 Here's a list of the available rake tasks:
@@ -271,16 +219,16 @@ Here's a list of the available rake tasks:
 
 ## Pushpop API Documentation
 
-Steps and jobs are the heart of the Pushpop workflow. Ruby jobs files contain one or more jobs, and each job consists of one or more steps.
+Steps and jobs are the heart of the Pushpop workflow. Job files are written in pure Ruby and contain one or more jobs. Each job consists of one or more steps.
 
 #### Jobs
 
 Jobs have the following attributes:
 
 + `name`: (optional) something that describe the job, useful in logs
-+ `every_duration`: the frequency at which to run the job
-+ `every_options` (optional): options related to when the job runs
-+ `steps`: the ordered list of steps to run
++ `period`: how frequently to run the job, first param to `every`
++ `every_options` (optional): options related to when the job runs, second param to `every`
++ `steps`: an ordered list of steps to run
 
 These attributes are easily specified using the DSL's block syntax. Here's an example:
 
@@ -293,11 +241,12 @@ job 'print job' do
 end
 ```
 
+The name of this job is 'print job'. It runs every 5 minutes and it has 1 step.
+
 Inside of a `job` configuration block, steps are added by using the `step` method. They can also be
-added by using a method registered by a plugin, like `keen` or `twilio`. For more information, see [Plugin Documentation](#plugin-documentation).
+added by using a method registered by a plugin, like `keen` or `twilio`. For more information on plugins see [Plugin Documentation](#plugin-documentation).
 
-The frequency of the job is set via the `every` method. This is basically a passthrough to Clockwork.
-Here are some cool things you can do with regard to scheduling:
+The period of the job's execution is set via the `every` method. This is basically a passthrough to the [Clockwork](https://github.com/tomykaira/clockwork) long-running process scheduler. Here are some cool things you can do with regard to setting times and days:
 
 ``` ruby
 every 5.seconds
@@ -380,11 +329,11 @@ In this example, the `twilio` step will only be ran if the `keen` step returned
 
 Steps have the following attributes:
 
-+ `name`: (optional) something that describes the step. Useful in logs, and is the key in the `step_responses` hash. Defaults to plugin name if a plugin is used
++ `name`: (optional) something that describes the step. Useful in logs, and is the key in the `step_responses` hash. Defaults to plugin name if a plugin is used. If you use the same plugin more than twice, you'll need to give steps individual names.
 + `plugin`: (optional) if the step is backed by a plugin, it's the name of the plugin
 + `block`: A block that runs to configure the step (when a plugin is used) or run it
 
-Steps can be pure Ruby code or leverage a plugin DSL. Plugins are just fancy abstractions for creating steps.
+Steps can be pure Ruby code or use a DSL provided by a plugin. Plugins are just fancy abstractions for creating steps.
 
 Steps have built-in support for ERB templating. This is useful for generating more complex emails and reports.
 
@@ -392,20 +341,17 @@ Here's an example that uses a template:
 
 ``` ruby
 sendgrid do |response, step_responses|
-  to 'josh+pushpop@keen.io'
-  from 'pushpopapp+123@keen.io'
-  subject 'Pingpong Daily Response Time Report'
+  to            'josh+pushpop@keen.io'
+  from          'pushpopapp+123@keen.io'
+  subject       'Pingpong Daily Response Time Report'
   body template 'pingpong_report.html.erb', response, step_responses
-  preview false
 end
 ```
 
-`template` is a function that renders a template in context of the step responses and returns a string.
-The first argument is a template file name, located in the `templates` directory by default.
-The second and third arguments are the `response` and `step_responses` respectively.
-An optional fourth parameter can be used to change the path templates are looked up in.
+`template` is a function that renders a template in context of the `response` and `step_responses` and returns a string.
+The first argument is a template file name, located in the `templates` directory by default. The second and third arguments are the `response` and `step_responses` respectively. An optional fourth parameter can be used to change the path templates are looked up in.
 
-Here's a very simple template:
+Here's a very simple template that uses the `response` variable in context:
 
 ``` erb
 <h1>Daily Report</h1>
@@ -414,126 +360,43 @@ Here's a very simple template:
 
 ## Recipes
 
-The community-driven [pushpop-recipes](https://github.com/keenlabs/pushpop-recipes) repository contains jobs and templates
+The community-driven [pushpop-recipes](https://github.com/pushpop-project/pushpop-recipes) repository contains jobs and templates
 for doing common things with Pushpop. Check it out for some inspiration!
 
-## Plugin Documentation
-
-Plugins are located at `lib/plugins`. They are loaded automatically.
-
-##### Keen
-
-The `keen` plugin gives you a DSL to specify Keen query parameters. When it runs, it
-passes those parameters to the [keen gem](https://github.com/keenlabs/keen-gem), which
-in turn runs the query against the Keen IO API.
+## Plugins
 
-Here's an example that shows most of the options you can specify:
-
-``` ruby
-job 'daily average response time by check for successful requests in april' do
-
-  keen do
-    event_collection  'checks'
-    analysis_type     'average'
-    target_property   'request.duration'
-    group_by          'check.name'
-    interval          'daily'
-    timeframe         ({ start: '2014-04-01T00:00Z' })
-    filters           [{ property_name: "response.successful",
-                         operator: "eq",
-                         property_value: true }]
-  end
-
-end
-```
-
-The `keen` plugin requires that the following environment variables are set: `KEEN_PROJECT_ID` and `KEEN_READ_KEY`.
-
-A `steps` method is also supported for [funnels](https://keen.io/docs/data-analysis/funnels/),
-as well as `analyses` for doing a [multi-analysis](https://keen.io/docs/data-analysis/multi-analysis/).
-
-##### Sendgrid
-
-The `sendgrid` plugin gives you a DSL to specify email parameters like subject and body.
-
-Here's an example:
-
-``` ruby
-job 'send an email' do
-
-  sendgrid do
-    to 'josh+pushpop@keen.io'
-    from 'pushpopapp+123@keen.io'
-    subject 'Is your inbox lonely?'
-    body 'This email was intentionally left blank.'
-    preview false
-  end
-
-end
-```
-
-The `sendgrid` plugin requires that the following environment variables are set: `SENDGRID_DOMAIN`, `SENDGRID_USERNAME`, and `SENDGRID_PASSWORD`.
-
-The `preview` directive is optional and defaults to false. If you set it to true, the email contents will print out
-to the console, but the email will not be sent.
-
-The `body` method can take a string, or it can take the same parameters as `template`,
-in which case it will render a template to create the body. For example:
-
-``` ruby
-body 'pingpong_report.html.erb', response, step_responses
-```
-
-##### Twilio
-
-The `twilio` plugin provides DSL to specify SMS recipient information as well as the text itself.
-
-Here's an example:
-
-``` ruby
-job 'send a text' do
-
-  twilio do
-    to '18005555555'
-    body 'Breathe in through the nose, out through the mouth.'
-  end
-
-end
-```
-
-The `twilio` plugin requires that the following environment variables are set: `TWILIO_AUTH_TOKEN`, `TWILIO_SID`, and `TWILIO_FROM`.
+Plugins are packaged as gems. See the [Pushpop organization](https://github.com/pushpop-project) page for a sampling of popular plugins.
 
 ## Creating plugins
 
-Plugins are just subclasses of `Pushpop::Step`. Plugins should implement a run method, and
-register themselves. Here's a simple plugin that stops job execution if the input into the step is 0:
+Plugins are just subclasses of `Pushpop::Step`. Plugins should implement a `run` method and register themselves. Here's a simple plugin that stops job execution if the input into the step is 0:
 
 ``` ruby
 module Pushpop
-  class BreakIfZero < Step
-    PLUGIN_NAME = 'break_if_zero'
+  class StopIfZero < Step
+    PLUGIN_NAME = 'stop_if_zero'
     def run(last_response=nil, step_responses=nil)
       last_response == 0
     end
   end
 
-  Pushpop::Job.register_plugin(BreakIfZero::PLUGIN_NAME, BreakIfZero)
+  Pushpop::Job.register_plugin(StopIfZero::PLUGIN_NAME, StopIfZero)
 end
 
-# now in your job you can use the break_if_zero step
+# now in your job you can use the stop_if_zero step
 
 job do
   step do [0, 1].shuffle.first end
-  break_if_zero
+  stop_if_zero
   step do puts 'made it through!' end
 end
 ```
 
 ## Usage as a Ruby gem
 
-Pushpop is also a Ruby gem, so you can add it to an existing Ruby project. Here's how to do that:
+Pushpop can also be embedded in existing Ruby projects as a Ruby gem. Here's some steps on how to do that.
 
-**Install the gem**
+##### Install the gem
 
 ``` ruby
 # bundler
@@ -543,15 +406,14 @@ gem 'pushpop'
 gem install 'pushpop'
 ```
 
-**Require job files and run**
+##### Require job files and run
 
-Once the gem is available, you should be able to load in Pushpop job files. Once a file loads,
-all of its jobs are ready to be run or scheduled.
+Once the gem is available you can load or require Pushpop job files. Once each file loads the jobs it contains are ready to be run or scheduled. Here's that sequence:
 
 ``` ruby
 load 'some_job.rb'
 
-# run the jobs once
+# you could run the jobs once
 Pushpop.run
 
 # or schedule and run the jobs with clockwork
@@ -559,24 +421,31 @@ Pushpop.schedule
 Clockwork.manager.run
 ```
 
-Note that `pushpop` does not declare dependencies other than `clockwork` and `keen`. If you're using
-Pushpop plugins like Sendgrid or Twilio you'll need to install and require those dependencies explicitly.
+The `pushpop` gem does not declare dependencies other than `clockwork` and `keen`. If you're using
+Pushpop plugins like Sendgrid or Twilio you'll need to bundle and require those dependencies separately.
 
 ## Contributing
 
-Issues and pull requests are welcome!
+Issues and pull requests are very welcome!
 
-**Wishlist**
+##### Wishlist
 
-+ Add plugins for more data collection and notification services
-+ Add a web interface that shows the job names, job results, and a countdown to the next run
-+ Add a web interface that lets you preview emails in the browser
-+ Beautiful email templates with support for typical Keen IO query responses (groups, series, etc)
++ Add plugins for more data collection and email/SMS/notification services
++ Add a web interface that shows job names, previous job results, and countdowns to the next run
++ Add a web interface for previewing emails in the browser
++ Add beautiful email templates with support for typical Keen IO query responses (groups, series, etc)
 
-**Testing**
+##### Testing
 
-Pushpop has a full set of specs (including plugins). Run them like this:
+Please make sure the specs pass before you submit your pull request. Pushpop has a full set of specs (including plugins). Run them like this:
 
 ``` shell
 $ bundle exec rake spec
 ```
+
+## Inspirations
+
+> "Technology shouldn't require all of our attention, just some of it, and only when necessary."
+> [calmtechnology.com](http://calmtechnology.com/)
+
+Dashboards and reports are human presentation vehicles. They require our attention in order to gain meaning. That's great when we're actively seeking answers and want to explore. But as a means to become aware of interesting, timely events it's neither effective nor efficient. A tool like Pushpop works better in those cases. It's a calmer technology.

data/lib/pushpop.rb

@@ -4,11 +4,6 @@ require 'pushpop/version'
 require 'pushpop/job'
 require 'pushpop/step'
 
-# require all plugins
-Dir["#{File.expand_path('../plugins/*', __FILE__)}.rb"].each { |file|
-  require file
-}
-
 module Pushpop
   class << self
     cattr_accessor :logger
@@ -33,6 +28,10 @@ module Pushpop
     def schedule
       self.jobs.map &:schedule
     end
+
+    def load_plugin(name)
+      load "#{File.expand_path("../plugins/#{name}", __FILE__)}.rb"
+    end
   end
 end
 

data/lib/pushpop/job.rb

@@ -15,7 +15,7 @@ module Pushpop
     end
 
     attr_accessor :name
-    attr_accessor :every_duration
+    attr_accessor :period
     attr_accessor :every_options
     attr_accessor :steps
 
@@ -26,8 +26,8 @@ module Pushpop
       self.instance_eval(&block)
     end
 
-    def every(duration, options={})
-      self.every_duration = duration
+    def every(period, options={})
+      self.period = period
       self.every_options = options
     end
 
@@ -48,7 +48,8 @@ module Pushpop
     end
 
     def schedule
-      Clockwork.manager.every(every_duration, name, every_options) do
+      raise 'Set job period via "every"' unless self.period
+      Clockwork.manager.every(period, name, every_options) do
         run
       end
     end
@@ -79,6 +80,16 @@ module Pushpop
     def method_missing(method, *args, &block)
       plugin_class = self.class.plugins[method.to_s]
 
+      unless plugin_class
+        begin
+          Pushpop.load_plugin method.to_s
+          plugin_class = self.class.plugins[method.to_s]
+          raise "Plugin not loaded: #{method.to_s}" if plugin_class.nil?
+        rescue LoadError
+          Pushpop.logger.warn("Could not find plugin #{method.to_s}")
+        end
+      end
+
       name = args[0]
       plugin = method.to_s
 

data/lib/pushpop/version.rb

@@ -1,3 +1,3 @@
 module Pushpop
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
 end

data/pushpop.gemspec

@@ -8,12 +8,11 @@ Gem::Specification.new do |s|
   s.version     = Pushpop::VERSION
   s.authors     = ["Josh Dzielak"]
   s.email       = "josh@keen.io"
-  s.homepage    = "https://github.com/keenlabs/pushpop"
-  s.summary     = "Automatic delivery of reports and alerts based on Keen IO events"
-  s.description = "Pushpop is a simple but powerful Ruby app that sends notifications based on events captured with Keen IO."
+  s.homepage    = "https://github.com/pushpop-project/pushpop"
+  s.summary     = "Share data between services at regular intervals"
+  s.description = "Pushpop is a simple but powerful Ruby app that sends notifications about events captured with Keen IO."
 
   s.add_dependency "clockwork"
-  s.add_dependency "keen"
 
   s.files         = `git ls-files`.split("\n")
   s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")

data/spec/pushpop/job_spec.rb

@@ -17,7 +17,7 @@ describe Pushpop::Job do
       block_ran = false
       job = Pushpop::Job.new('foo') do block_ran = true end
       job.name.should == 'foo'
-      job.every_duration.should be_nil
+      job.period.should be_nil
       job.every_options.should == {}
       block_ran.should be_true
     end
@@ -29,10 +29,10 @@ describe Pushpop::Job do
   end
 
   describe '#every' do
-    it 'should set duration and options' do
+    it 'should set period and options' do
       job = empty_job
       job.every(10.seconds, at: '01:02')
-      job.every_duration.should == 10
+      job.period.should == 10
       job.every_options.should == { at: '01:02' }
     end
   end
@@ -91,9 +91,9 @@ describe Pushpop::Job do
 
   describe '#schedule' do
     it 'should add the job to clockwork' do
-      frequency = 1.seconds
+      period = 1.seconds
       simple_job = Pushpop::Job.new('foo') do
-        every frequency
+        every period
         step 'track_times_run' do
           @times_run ||= 0
           @times_run += 1
@@ -104,9 +104,16 @@ describe Pushpop::Job do
 
       Clockwork.manager.tick(Time.now)
       simple_job.run.first.should == 2
-      Clockwork.manager.tick(Time.now + frequency)
+      Clockwork.manager.tick(Time.now + period)
       simple_job.run.first.should == 4
     end
+
+     it 'should fail if the period was not specified' do
+       simple_job = Pushpop::Job.new('foo') do end
+       expect {
+         simple_job.schedule
+       }.to raise_error
+     end
   end
 
   describe '#method_missing' do

data/spec/pushpop/step_spec.rb

@@ -24,9 +24,9 @@ describe Pushpop::Step do
 
     it 'should set name to plugin name if not given' do
       empty_proc = Proc.new {}
-      step = Pushpop::Step.new(nil, 'keen', &empty_proc)
-      step.name.should == 'keen'
-      step.plugin.should == 'keen'
+      step = Pushpop::Step.new(nil, 'whee', &empty_proc)
+      step.name.should == 'whee'
+      step.plugin.should == 'whee'
       step.block.should == empty_proc
     end
 

data/spec/spec_helper.rb

@@ -1,5 +1,3 @@
-require 'webmock/rspec'
-
 $: << File.join(File.dirname(__FILE__), '../lib')
 
 require 'pushpop'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pushpop
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-11 00:00:00.000000000 Z
+date: 2014-06-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: clockwork
@@ -27,24 +27,8 @@ dependencies:
     - - ! '>='
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: keen
-  requirement: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    none: false
-    requirements:
-    - - ! '>='
-      - !ruby/object:Gem::Version
-        version: '0'
-description: Pushpop is a simple but powerful Ruby app that sends notifications based
-  on events captured with Keen IO.
+description: Pushpop is a simple but powerful Ruby app that sends notifications about
+  events captured with Keen IO.
 email: josh@keen.io
 executables: []
 extensions: []
@@ -59,17 +43,12 @@ files:
 - README.md
 - Rakefile
 - jobs/example_job.rb
-- lib/plugins/keen.rb
-- lib/plugins/sendgrid.rb
-- lib/plugins/twilio.rb
 - lib/pushpop.rb
 - lib/pushpop/job.rb
 - lib/pushpop/step.rb
 - lib/pushpop/version.rb
 - pushpop.gemspec
 - spec/jobs/simple_job.rb
-- spec/plugins/keen_spec.rb
-- spec/plugins/sendgrid_spec.rb
 - spec/pushpop/job_spec.rb
 - spec/pushpop/step_spec.rb
 - spec/pushpop_spec.rb
@@ -77,7 +56,7 @@ files:
 - spec/spec_helper.rb
 - spec/templates/spec.html.erb
 - templates/first_template.html.erb
-homepage: https://github.com/keenlabs/pushpop
+homepage: https://github.com/pushpop-project/pushpop
 licenses: []
 post_install_message: 
 rdoc_options: []
@@ -100,11 +79,9 @@ rubyforge_project:
 rubygems_version: 1.8.23
 signing_key: 
 specification_version: 3
-summary: Automatic delivery of reports and alerts based on Keen IO events
+summary: Share data between services at regular intervals
 test_files:
 - spec/jobs/simple_job.rb
-- spec/plugins/keen_spec.rb
-- spec/plugins/sendgrid_spec.rb
 - spec/pushpop/job_spec.rb
 - spec/pushpop/step_spec.rb
 - spec/pushpop_spec.rb

data/lib/plugins/keen.rb

@@ -1,78 +0,0 @@
-require 'keen'
-
-module Pushpop
-
-  class Keen < Step
-
-    PLUGIN_NAME = 'keen'
-
-    attr_accessor :_event_collection
-    attr_accessor :_analysis_type
-    attr_accessor :_timeframe
-    attr_accessor :_target_property
-    attr_accessor :_group_by
-    attr_accessor :_interval
-    attr_accessor :_filters
-    attr_accessor :_steps
-    attr_accessor :_analyses
-
-    def run(last_response=nil, step_responses=nil)
-      self.configure(last_response, step_responses)
-      ::Keen.send(self._analysis_type, self._event_collection, self.to_analysis_options)
-    end
-
-    def configure(last_response=nil, step_responses=nil)
-      self.instance_exec(last_response, step_responses, &block)
-    end
-
-    def to_analysis_options
-      { timeframe: self._timeframe,
-        target_property: self._target_property,
-        group_by: self._group_by,
-        interval: self._interval,
-        filters: self._filters,
-        analyses: self._analyses,
-        steps: self._steps
-      }.delete_if { |_, v| v.nil? }
-    end
-
-    def event_collection(event_collection)
-      self._event_collection = event_collection
-    end
-
-    def analysis_type(analysis_type)
-      self._analysis_type = analysis_type
-    end
-
-    def timeframe(timeframe)
-      self._timeframe = timeframe
-    end
-
-    def target_property(target_property)
-      self._target_property = target_property
-    end
-
-    def group_by(group_by)
-      self._group_by = group_by
-    end
-
-    def interval(interval)
-      self._interval = interval
-    end
-
-    def filters(filters)
-      self._filters = filters
-    end
-
-    def steps(steps)
-      self._steps = steps
-    end
-
-    def analyses(analyses)
-      self._analyses = analyses
-    end
-
-  end
-
-  Pushpop::Job.register_plugin(Keen::PLUGIN_NAME, Keen)
-end

data/lib/plugins/sendgrid.rb

@@ -1,94 +0,0 @@
-require 'mail'
-
-Mail.defaults do
-  delivery_method :smtp, { address:   'smtp.sendgrid.net',
-                           port:      587,
-                           domain:    ENV['SENDGRID_DOMAIN'],
-                           user_name: ENV['SENDGRID_USERNAME'],
-                           password:  ENV['SENDGRID_PASSWORD'],
-                           authentication: 'plain',
-                           enable_starttls_auto: true }
-end
-
-module Pushpop
-
-  class Sendgrid < Step
-
-    PLUGIN_NAME = 'sendgrid'
-
-    attr_accessor :_from
-    attr_accessor :_to
-    attr_accessor :_subject
-    attr_accessor :_body
-    attr_accessor :_preview
-
-    def run(last_response=nil, step_responses=nil)
-
-      self.configure(last_response, step_responses)
-
-      # print the message if its just a preview
-      return print_preview if self._preview
-
-      _to = self._to
-      _from = self._from
-      _subject = self._subject
-      _body = self._body
-
-      Mail.deliver do
-        to _to
-        from _from
-        subject _subject
-        text_part do
-          body _body
-        end
-        html_part do
-          content_type 'text/html; charset=UTF-8'
-          body _body
-        end
-      end
-    end
-
-    def configure(last_response=nil, step_responses=nil)
-      self.instance_exec(last_response, step_responses, &block)
-    end
-
-    def from(from)
-      self._from = from
-    end
-
-    def to(to)
-      self._to = to
-    end
-
-    def subject(subject)
-      self._subject = subject
-    end
-
-    def preview(preview)
-      self._preview = preview
-    end
-
-    def body(*args)
-      if args.length == 1
-        self._body = args.first
-      else
-        self._body = template *args
-      end
-    end
-
-    private
-
-    def print_preview
-      puts <<MESSAGE
-To: #{self._to}
-From: #{self._from}
-Subject: #{self._subject}
-
-      #{self._body}
-MESSAGE
-    end
-
-  end
-
-  Pushpop::Job.register_plugin(Sendgrid::PLUGIN_NAME, Sendgrid)
-end

data/lib/plugins/twilio.rb

@@ -1,52 +0,0 @@
-require 'twilio-ruby'
-
-TWILIO_SID = ENV['TWILIO_SID']
-TWILIO_AUTH_TOKEN = ENV['TWILIO_AUTH_TOKEN']
-TWILIO_FROM = ENV['TWILIO_FROM']
-
-module Pushpop
-
-  class Twilio < Step
-
-    PLUGIN_NAME = 'twilio'
-
-    attr_accessor :_from
-    attr_accessor :_to
-    attr_accessor :_body
-
-    def run(last_response=nil, step_responses=nil)
-
-      self.configure(last_response, step_responses)
-
-      _to = self._to
-      _from = self._from || TWILIO_FROM
-      _body = self._body
-
-      client = ::Twilio::REST::Client.new(TWILIO_SID, TWILIO_AUTH_TOKEN)
-
-      client.account.messages.create(
-          from: _from,
-          to: _to,
-          body: _body )
-    end
-
-    def configure(last_response=nil, step_responses=nil)
-      self.instance_exec(last_response, step_responses, &block)
-    end
-
-    def from(from)
-      self._from = from
-    end
-
-    def to(to)
-      self._to = to
-    end
-
-    def body(body)
-      self._body = body
-    end
-
-  end
-
-  Pushpop::Job.register_plugin(Twilio::PLUGIN_NAME, Twilio)
-end

data/spec/plugins/keen_spec.rb

@@ -1,88 +0,0 @@
-require 'spec_helper'
-
-describe Pushpop::Keen do
-
-  describe '#configure' do
-
-    it 'should set various params' do
-
-      step = Pushpop::Keen.new do
-        event_collection 'pageviews'
-        analysis_type 'count'
-        timeframe 'last_3_days'
-        target_property 'trinkets'
-        group_by 'referer'
-        interval 'hourly'
-        filters [{ property_value: 'referer',
-                   operator: 'ne',
-                   property_value: 'yahoo.com' }]
-        steps [{ event_collection: 'pageviews',
-                 actor_property: 'user.id' }]
-        analyses [{ analysis_type: 'count' }]
-      end
-
-      step.configure
-
-      step._event_collection.should == 'pageviews'
-      step._analysis_type.should == 'count'
-      step._timeframe.should == 'last_3_days'
-      step._group_by.should == 'referer'
-      step._interval.should == 'hourly'
-      step._steps.should == [{
-         event_collection: 'pageviews',
-         actor_property: 'user.id'
-        }]
-      step._analyses.should == [{ analysis_type: 'count' }]
-    end
-
-  end
-
-  describe '#run' do
-    it 'should run the query based on the analysis type' do
-      Keen.stub(:count).with('pageviews', {
-          timeframe: 'last_3_days'
-      }).and_return(365)
-
-      step = Pushpop::Keen.new('one') do
-        event_collection 'pageviews'
-        analysis_type 'count'
-        timeframe 'last_3_days'
-      end
-      response = step.run
-      response.should == 365
-    end
-  end
-
-  describe '#to_analysis_options' do
-    it 'should include various options' do
-      step = Pushpop::Keen.new('one') do end
-      step._timeframe = 'last_4_days'
-      step._group_by = 'referer'
-      step._target_property = 'trinkets'
-      step._interval = 'hourly'
-      step._filters = [{ property_value: 'referer',
-                         operator: 'ne',
-                         property_value: 'yahoo.com' }]
-      step._steps = [{ event_collection: 'pageviews',
-                       actor_property: 'user.id' }]
-      step._analyses = [{ analysis_type: 'count' }]
-      step.to_analysis_options.should == {
-          timeframe: 'last_4_days',
-          target_property: 'trinkets',
-          group_by: 'referer',
-          interval: 'hourly',
-          filters: [{ property_value: 'referer',
-                         operator: 'ne',
-                         property_value: 'yahoo.com' }],
-          steps: [{ event_collection: 'pageviews',
-                       actor_property: 'user.id' }],
-          analyses: [{ analysis_type: 'count' }]
-      }
-    end
-
-    it 'should not include nils' do
-      step = Pushpop::Keen.new('one') do end
-      step.to_analysis_options.should == {}
-    end
-  end
-end

data/spec/plugins/sendgrid_spec.rb

@@ -1,66 +0,0 @@
-require 'spec_helper'
-
-SPEC_TEMPLATES_DIRECTORY ||= File.expand_path('../../templates', __FILE__)
-
-describe Pushpop::Sendgrid do
-
-  describe '#configure' do
-
-    it 'should set various params' do
-
-      step = Pushpop::Sendgrid.new do
-        to 'josh@keen.io'
-        from 'depths@hell.com'
-        subject 'time is up'
-        body 'use code 3:16 for high leniency'
-        preview true
-      end
-
-      step.configure
-
-      step._to.should == 'josh@keen.io'
-      step._from.should == 'depths@hell.com'
-      step._subject.should == 'time is up'
-      step._body.should == 'use code 3:16 for high leniency'
-      step._preview.should be_true
-
-    end
-
-  end
-
-  describe '#run' do
-
-    it 'should send some email' do
-
-      Mail.stub(:deliver)
-
-      step = Pushpop::Sendgrid.new do |response|
-        to 'josh@keen.io'
-        from 'alerts+pushpop@keen.io'
-        subject "There were #{response} Pageviews Today!"
-        body 'hey wats up'
-      end
-
-      step.run(365)
-
-    end
-
-  end
-
-  describe '#body' do
-
-    it 'should use a string if given 1 arg' do
-      step = Pushpop::Sendgrid.new
-      step.body 'hello world'
-      step._body.should == 'hello world'
-    end
-
-    it 'should use a template if more than 1 arg is passed' do
-      step = Pushpop::Sendgrid.new
-      step.body('spec.html.erb', 500, {}, SPEC_TEMPLATES_DIRECTORY)
-      step._body.strip.should == '<pre>500</pre>'
-    end
-
-  end
-
-end