minicron 0.1.0

data/README.md

@@ -0,0 +1,187 @@
+minicron
+=======
+
+[![Build Status](http://img.shields.io/travis/jamesrwhite/minicron.svg)](http://travis-ci.org/jamesrwhite/minicron) [![Coverage Status](http://img.shields.io/coveralls/jamesrwhite/minicron.svg)](https://coveralls.io/r/jamesrwhite/minicron?branch=master) [![Code Climate](http://img.shields.io/codeclimate/github/jamesrwhite/minicron.svg)](https://codeclimate.com/github/jamesrwhite/minicron) [![Dependency Status](http://img.shields.io/gemnasium/jamesrwhite/minicron.svg)](https://gemnasium.com/jamesrwhite/minicron) [![Inline docs](http://inch-pages.github.io/github/jamesrwhite/minicron.png)](http://inch-pages.github.io/github/jamesrwhite/minicron)
+
+minicron aims to complement ````cron```` by making it easier to manage and monitor cron jobs, it can largely be thought of as two components that interact together, the CLI and the Hub. The CLI is what is installed on your server(s) and executes your cron command and reports the status back to the Hub. The Hub is the central point where data from one or many instances of the CLI are is recieved and stored in a database. The Hub also provides a web interface to the data and makes it easy to manage your cron jobs.
+
+- [Background](https://github.com/jamesrwhite/minicron/blob/master/README.md#background)
+- [Goals](https://github.com/jamesrwhite/minicron/blob/master/README.md#features)
+- [Features](https://github.com/jamesrwhite/minicron/blob/master/README.md#goals)
+- [Installation](https://github.com/jamesrwhite/minicron/blob/master/README.md#installation)
+- [Requirements](https://github.com/jamesrwhite/minicron/blob/master/README.md#requirements)
+- [Usage](https://github.com/jamesrwhite/minicron/blob/master/README.md#usage)
+- [Documentation](https://github.com/jamesrwhite/minicron/blob/master/README.md#documentation)
+- [Versioning](https://github.com/jamesrwhite/minicron/blob/master/README.md#versioning)
+- [Contributing](https://github.com/jamesrwhite/minicron/blob/master/README.md#contributing)
+- [Support](https://github.com/jamesrwhite/minicron/blob/master/README.md#support)
+- [License](https://github.com/jamesrwhite/minicron/blob/master/README.md#license)
+
+Background
+-----------
+
+I'm developing minicron as part of my dissertation at university which is due in May but I plan to continue development after that. My inspiration for developing minicron comes from time spent working at [Miniclip](http://www.miniclip.com) where the management and monitoring of cron jobs at times proved to be tricky!
+
+Goals
+------
+
+Some rough goals that minicron is trying to achieve.
+
+- Remove the need to understand cron syntax
+- Increase visiblity into cron failures and missed executions
+- Have minimal external dependencies
+- Fault tolerance
+
+Features
+---------
+
+- Web UI
+  - CRUD for cron jobs using ssh
+  - GUI for cron schedule create/read/update
+  - View realtime output/status as jobs run
+  - Historical data for all job executions
+- Alerts when jobs executions are missed or fail via:
+  - Email
+  - SMS ([using Twilio](https://www.twilio.com))
+  - [PagerDuty](www.pagerduty.com)
+
+Lots more is planned for the feature, see issues tagged [feature](https://github.com/jamesrwhite/minicron/issues?labels=feature&milestone=&page=1&state=open).
+
+Installation
+-------------
+
+minicron is currently under heavy development and as such I would not recommend that you use this in production yet but I encourage you to give it a try in a non critical environment and help me improve it.
+
+#### Development Install
+
+If you wish to test the latest from GitHub version you can clone this repo ````bundle install```` and ````rake install````.
+
+#### Released Install
+
+To install the latest release (currently 0.1) you can ````gem install minicron````.
+
+#### Database Setup
+
+Set your database configuration options in ````/etc/minicron.toml```` and you can then ````minicron db setup````
+
+> **WARNING** this will drop any existing tables in the configured database and create new ones
+
+or set it up manually using the [schema dump provided](https://github.com/jamesrwhite/minicron/blob/master/lib/minicron/hub/db/schema.sql).
+
+Requirements
+-------------
+
+#### Ruby
+- MRI
+  - 1.9.3 and above (tested on 1.9.3, 2.0.0, 2.1.0)
+- <del>Rubinius</del>
+  - <del>Travis builds are run on the latest release</del> [*awaiting bug fix*](https://github.com/rubinius/rubinius/issues/2944)
+- JRuby
+  - As yet untested
+
+
+#### Database
+- MySQL
+- Support for PostgreSQL and SQlite is planned in the future
+
+#### Browser
+- I have been testing the web interface in the latest versions of Chrome, Firefox and Safari. I'm currently unsure of how it functions in the various of Internet Explorer but in theory it should support IE9+
+
+#### OS
+- Should run on any linux/bsd based OS that the above ruby versions run on.
+- No windows support.
+
+Usage
+-----
+
+minicron is packaged as a gem and can be interacted with from the command line once you have installed it. The current commands are as follows:
+
+#### Run a command
+
+````
+minicron run 'command --options'
+````
+
+You can alter the way in which the command alters it's output by passing the --mode option to the run argument with the value of either 'line' or 'char'. Most of the time you'll want to use the default value of line but for some commands e.g a script that outputs a progress bar minicron printing the output each character at a time can be useful.
+
+````
+minicron run 'mysqldump db > backup.sql'
+````
+
+The global ````--verbose```` option can also be passed to the ````run```` argument like so ````minicron run --verbose ls````, for further information see ````minicron help run````.
+
+#### Get help
+
+Running ````minicron```` with no arguments is an alias to running ````minicron help````, ````minicron -h```` or ````minicron --help````. You can also use the help argument to get information on any command as shown above in the run a command section or alternatively you can pass the ````-h```` or ````--help```` options like so ````minicron run -h````.
+
+#### Server
+
+To launch the server (aka the Hub) run ````minicron server```` - by default it will bind to port 9292 on the host 127.0.0.1 but this can be configured by the command line arguments ````--host```` ````--port```` and ````--path```` or in the config file.
+
+See [sample.nginx.conf](https://github.com/jamesrwhite/minicron/blob/master/sample.nginx.conf) for an example of how to run minicron behind a reverse proxy. It should be possible to run minicron behind any web server/reverse proxy as long as it supports WebSockets.
+
+#### Version
+
+Like many command line programs minicron will show it's version number when the global options ````-v```` or ````--version```` are passed to the CLI.
+
+#### Configuration
+
+Some configuration options can be passed in manually but the recommend way to configure minicron is through the use of a config file. You can specify the path to the file using the ````--config```` global option. The file is expected to be in the [toml](https://github.com/mojombo/toml "toml") format. The default options are specified in the [default.config.toml](https://github.com/jamesrwhite/minicron/blob/master/default.config.toml "default.config.toml") file and minicron will parse a config located in ````/etc/minicron.toml```` if it exists. Options specified via the command line will take precedence over those taken from a config file.
+
+Documentation
+-------------
+
+minicron uses [Yard](http://yardoc.org/ "Yard") for it's code documentation, you can either generate it and view it locally using the following commands:
+
+````
+yard doc
+yard server
+````
+
+or view the most up to date version online at [RubyDoc.info](http://rdoc.info/github/jamesrwhite/minicron/master/frames "RubyDoc.info").
+
+Versioning
+-----------
+
+Where possible all releases will follow the [semantic versioning](http://semver.org/) guidelines.
+
+Releases will be numbered with the following format:
+
+`<major>.<minor>.<patch>`
+
+Based on the following guidelines:
+
+* A new *major* release indicates a large change where backwards compatibility is broken.
+* A new *minor* release indicates a normal change that maintains backwards compatibility.
+* A new *patch* release indicates a bugfix or small change which does not affect compatibility.
+
+Contributing
+------------
+
+Feedback and pull requests are welcome, please see [CONTRIBUTING.md](https://github.com/jamesrwhite/minicron/blob/master/CONTRIBUTING.md "CONTRIBUTING.md") for more info.
+
+Areas that I would love some help with:
+
+- Any of the unassigned [issues here](https://github.com/jamesrwhite/minicron/issues?state=open).
+- General testing of the system, let me know what you think and create issues for any bugs you find!
+- Tests!!
+- Validation and error handling improvements
+- Documentation improvements. Find something confusing or unexpected, let me know and I'll add or improve documentation for it!
+- Look for '[TODO:](https://github.com/jamesrwhite/minicron/search?q=TODO%3A)' notices littered around the code, I'm trying to convert them all to issues but there are a lot..
+- Code refactoring, I had to have 0.1 ready by a certain deadline so some parts are far from perfect
+- UI improvements
+
+Support
+--------
+
+Where possible I will try and provide support for minicron, you can get in touch with me via:
+
+- Twitter [@jamesrwhite](https://twitter.com/jamesrwhite)
+- Email [dev.jameswhite+minicron@gmail.com](mailto:dev.jameswhite+minicron@gmail.com)
+
+Or feel free to open an issue and I'll do my best to help.
+
+License
+--------
+
+minicron is licensed under the GPL v3, [see here for the full license](https://github.com/jamesrwhite/minicron/blob/master/LICENSE "see here")

data/Rakefile

@@ -0,0 +1,17 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+require 'sinatra/activerecord/rake'
+require 'minicron'
+require 'minicron/hub/app'
+
+# rspec tests
+desc 'Run specs'
+RSpec::Core::RakeTask.new do |t|
+  t.verbose = true
+  t.rspec_opts = '--color --order random'
+end
+
+# Show a list of tasks by default
+task :default do
+  puts `rake --tasks`
+end

data/bin/minicron

@@ -0,0 +1,26 @@
+#!/usr/bin/env ruby
+require 'minicron'
+
+begin
+  # Capture any output from STDERR so we can handle it how we want to
+  Minicron.capture_output(:type => :stderr) do
+    # Run the CLI and pass it all arguments passed to us
+    Minicron::CLI.new.run(ARGV) do |output|
+      print output
+      STDOUT.flush
+    end
+  end
+# Handle ctrl-c and other nasty interruptions
+rescue Interrupt, SignalException
+  puts "\nExiting.."
+# Accept failure
+rescue Exception => e
+  $stderr.write("#{e.message}\n")
+
+  if Minicron.config['cli']['trace']
+    puts
+    fail e
+  end
+
+  exit(1)
+end

data/lib/minicron.rb

@@ -0,0 +1,179 @@
+require 'stringio'
+require 'toml'
+require 'sshkey'
+require 'minicron/cli'
+require 'minicron/constants'
+
+# @author James White <dev.jameswhite+minicron@gmail.com>
+module Minicron
+  # Default configuration, this can be overriden
+  @config = {
+    'global' => {
+      'verbose' => false,
+      'trace' => false
+    },
+    'client' => {
+      'scheme' => 'http',
+      'host' => '127.0.0.1',
+      'port' => 9292,
+      'path' => '/',
+      'connect_timeout' => 5,
+      'inactivity_timeout' => 5
+    },
+    'server' => {
+      'host' => '127.0.0.1',
+      'port' => 9292,
+      'path' => '/'
+    },
+    'database' => {
+      'type' => 'mysql',
+      'host' => '127.0.0.1',
+      'database' => 'minicron',
+      'username' => 'minicron',
+      'password' => 'password'
+    },
+    'cli' => {
+      'mode' => 'line',
+      'dry_run' => false
+    },
+    'alerts' => {
+      'email' => {
+        'enabled' => false,
+        'smtp' => {
+          'address' => 'localhost',
+          'port' => 25
+        }
+      },
+      'sms' => {
+        'enabled' => false
+      },
+      'pagerduty' => {
+        'enabled' => false
+      }
+    }
+  }
+
+  class << self
+    attr_accessor :config
+  end
+
+  # Parse the given config file and update the config hash
+  #
+  # @param file_path [String]
+  def self.parse_file_config(file_path)
+    file_path ||= Minicron::DEFAULT_CONFIG_FILE
+
+    begin
+      @config = TOML.load_file(file_path)
+    rescue Errno::ENOENT
+      # Fail if the file doesn't exist unless it's the default config file
+      if file_path != DEFAULT_CONFIG_FILE
+        fail Exception, "Unable to the load the file '#{file_path}', are you sure it exists?"
+      end
+    rescue Errno::EACCES
+      fail Exception, "Unable to the readthe file '#{file_path}', check it has the right permissions."
+    rescue TOML::ParseError
+      fail Exception, "An error occured parsing the config file '#{file_path}', please check it uses valid TOML syntax."
+    end
+  end
+
+  # Parses the config options from the cli
+  # @option options [Hash] global global options
+  # @option options [Hash] server server options
+  # @option options [Hash] cli cli options
+  def self.parse_cli_config(options = {})
+    options.each do |type, _|
+      options[type].each do |key, value|
+        if value
+          @config[type][key] = value
+        end
+      end
+    end
+  end
+
+  # Helper function to capture STDOUT and/or STDERR
+  # adapted from http://stackoverflow.com/a/11349621/483271
+  #
+  # @option options [Symbol] type (:both) what to capture: :stdout, :stderr or :both
+  # @return [StringIO] if the type was set to :stdout or :stderr
+  # @return [Hash] containg both the StringIO instances if the type was set to :both
+  def self.capture_output(options = {}, &block)
+    # Default options
+    options[:type] ||= :both
+
+    # Make copies of the origin STDOUT/STDERR
+    original_stdout = $stdout
+    original_stderr = $stderr
+
+    # Which are we handling?
+    case options[:type]
+    when :stdout
+      $stdout = stdout = StringIO.new
+    when :stderr
+      $stderr = stderr = StringIO.new
+    when :both
+      $stderr = $stdout = stdout = stderr = StringIO.new
+    else
+      fail ArgumentError, 'The type must be one of [stdout, stderr, both]'
+    end
+
+    # Yield to the code block to do whatever it has to do
+    begin
+      yield
+    # Whatever happens make sure we reset STDOUT/STDERR
+    ensure
+      $stdout = original_stdout
+      $stderr = original_stderr
+    end
+
+    # What are we going to return?
+    case options[:type]
+    when :stdout
+      stdout
+    when :stderr
+      stderr
+    else
+      { :stdout => stdout, :stderr => stderr }
+    end
+  end
+
+  # Sanitize a filename - taken from http://guides.rubyonrails.org/security.html
+  #
+  # @param filename [String]
+  # @return [String]
+  def self.sanitize_filename(filename)
+    filename.strip.tap do |name|
+      name.sub!(/\A.*(\\|\/)/, '')
+      # Finally, replace all non alphanumeric, underscore
+      # or periods with underscore
+      name.gsub!(/[^\w\.\-]/, '_')
+    end
+
+    filename
+  end
+
+
+  # Used to generate SSH keys for hosts but is completely generic
+  #
+  # @param type [String] the thing that is using the key, this is just here
+  # so this could be used for something other than hosts if needed
+  # @param id [Integer]
+  # @param name [String]
+  def self.generate_ssh_key(type, id, name)
+    key = SSHKey.generate(:comment => "minicron public key for #{name}")
+
+    # Set the locations to save the public key private key pair
+    private_key_path = sanitize_filename(File.expand_path("~/.ssh/minicron_#{type}_#{id}_rsa"))
+    public_key_path = sanitize_filename(File.expand_path("~/.ssh/minicron_#{type}_#{id}_rsa.pub"))
+
+    # Save the public key private key pair
+    File.write(private_key_path, key.private_key)
+    File.write(public_key_path, key.ssh_public_key)
+
+    # Set the correct permissions on the files
+    File.chmod(0600, private_key_path)
+    File.chmod(0644, public_key_path)
+
+    key
+  end
+end

data/lib/minicron/alert.rb

@@ -0,0 +1,115 @@
+require 'minicron/alert/email'
+require 'minicron/alert/sms'
+require 'minicron/alert/pagerduty'
+require 'minicron/hub/models/alert'
+require 'minicron/hub/models/job'
+
+module Minicron
+  class Alert
+    # Send an alert using all enabled mediums
+    #
+    # @option options [String] kind 'fail' or 'miss'
+    # @option options [Integer, nil] schedule_id only applies to 'miss' alerts
+    # @option options [Integer, nil] execution_id only used by 'fail' alerts
+    # @option options [Integer] job_id used by the #send method
+    # @option options [Time] expected_at only applies to 'miss' alerts
+    def send_all(options = {})
+      Minicron.config['alerts'].each do |medium, value|
+        # Check if the medium is enabled and alert hasn't already been sent
+        if value['enabled'] && !sent?(options)
+          send(
+            :kind => options[:kind],
+            :schedule_id => options[:schedule_id],
+            :execution_id => options[:execution_id],
+            :job_id => options[:job_id],
+            :expected_at => options[:expected_at],
+            :medium => medium
+          )
+        end
+      end
+    end
+
+    # Send an individual alert
+    #
+    # @option options [String] kind 'fail' or 'miss'
+    # @option options [Integer, nil] schedule_id only applies to 'miss' alerts
+    # @option options [Integer, nil] execution_id only used by 'fail' alerts
+    # @option options [Integer] job_id used to look up the job name for the alert message
+    # @option options [Time] expected_at when the schedule was expected to execute
+    # @option options [String] medium the medium to send the alert via
+    def send(options = {})
+      # Look up the job for this schedule
+      options[:job] = Minicron::Hub::Job.find(options[:job_id])
+
+      # Switch the medium that the alert will be sent via
+      case options[:medium]
+      when 'email'
+        send_email(options)
+      when 'sms'
+        send_sms(options)
+      when 'pagerduty'
+        send_pagerduty(options)
+      else
+        raise Exception, "The medium '#{options[:medium]}' is not supported!"
+      end
+
+      # Store that we sent the alert
+      Minicron::Hub::Alert.create(
+        :schedule_id => options[:schedule_id],
+        :execution_id => options[:execution_id],
+        :kind => options[:kind],
+        :expected_at => options[:expected_at],
+        :medium => options[:medium],
+        :sent_at => Time.now.utc
+      )
+    end
+
+    # Send an email alert, this has the same options as #send
+    def send_email(options = {})
+      email = Minicron::Email.new
+      email.send(
+        Minicron.config['alerts']['email']['from'],
+        Minicron.config['alerts']['email']['to'],
+        "minicron alert for job '#{options[:job].name}'!",
+        email.get_message(options)
+      )
+    end
+
+    # Send an sms alert, this has the same options as #send
+    def send_sms(options = {})
+      sms = Minicron::SMS.new
+      sms.send(
+        Minicron.config['alerts']['sms']['from'],
+        Minicron.config['alerts']['sms']['to'],
+        sms.get_message(options)
+      )
+    end
+
+    # Send a pagerduty alert, this has the same options as #send
+    def send_pagerduty(options = {})
+      pagerduty = Minicron::PagerDuty.new
+      pagerduty.send(
+        options[:kind] == 'fail' ? 'Job failed!' : 'Job missed!',
+        pagerduty.get_message(options)
+      )
+    end
+
+    # Queries the database to determine if an alert for this kind has already
+    # been sent
+    #
+    # @option options [String] kind 'fail' or 'miss'
+    # @option options [Integer, nil] schedule_id only applies to 'miss' alerts
+    # @option options [Integer, nil] execution_id only used by 'fail' alerts
+    # @option options [Time] expected_at when the schedule was expected to execute
+    # @option options [String] medium the medium to send the alert via
+    def sent?(options = {})
+      Minicron::Hub::Alert.exists?(
+        :kind => options[:kind],
+        :schedule_id => options[:schedule_id],
+        :execution_id => options[:execution_id],
+        :expected_at => options[:expected_at],
+        :medium => options[:medium]
+      )
+    end
+  end
+end