RubyGems - puppet-twitch - Versions diffs - 0.1.0 - Mend

puppet-twitch 0.1.0

Files changed (17) hide show

checksums.yaml +15 -0
data/Gemfile +12 -0
data/LICENSE.txt +21 -0
data/README.md +102 -0
data/bin/puppet-twitch +4 -0
data/lib/puppet_twitch/basic_http_server.rb +85 -0
data/lib/puppet_twitch/controller.rb +52 -0
data/lib/puppet_twitch/logger.rb +43 -0
data/lib/puppet_twitch/param_parser.rb +27 -0
data/lib/puppet_twitch/puppet.rb +30 -0
data/lib/puppet_twitch/server.rb +16 -0
data/lib/puppet_twitch/version.rb +3 -0
data/puppet-twitch.gemspec +24 -0
data/spec/puppet_twitch/basic_http_server_spec.rb +25 -0
data/spec/puppet_twitch/param_parser_spec.rb +44 -0
data/spec/spec_helper.rb +5 -0
metadata +76 -0

checksums.yaml ADDED

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    MmY4YzJlMTY4ZTZkYjAwOTRmNDdiMzdhNGE2Yjc5NGNiZDNhYWY2Zg==
+  data.tar.gz: !binary |-
+    YzQ3NDFlYThmYzNkMmMwZTNlMDE2ZjhiNjVlY2M4NzUyM2Y5NGQ5YQ==
+SHA512:
+  metadata.gz: !binary |-
+    ODQzNzQ0ODEzNGU5NWZhNzYxNGM1NjAwZTE2ZmUzYjNiNGM0MDA0OWIwNTEz
+    N2Q2YjgzMTY2ZWMwZDQ4NDM4ZjBkMjM5N2E2ZWE2MWVjZmM2YjY2MjdhNDJh
+    Mzk5ZTFkYWU2YWQ4MGNhYTkyZDk5MmM2MjhhNjA2ZWU4ZTRjOTg=
+  data.tar.gz: !binary |-
+    Mzc5MDVjMmYwOWYxNmUwMjMwY2Q5NTAyOWM3ODZmMzRmYTI1MzVkNThiODlj
+    NGQyMjYwZjAzM2M4ZWVmMWJkY2E3MWZjNGZmZWNhMzkwMDdlNTQ4Yzk0NDdl
+    YWEwOGRkMDE1YzhkODkwOGRiMjczMWRlYzg1MzJjNDlmOTMyMzk=

data/Gemfile ADDED

@@ -0,0 +1,12 @@
+source 'https://rubygems.org/'
+gemspec
+group :development do
+  gem 'puppet'
+  gem 'rake'
+end
+group :test do
+  gem 'rspec', '~> 3.2.0'
+end

data/LICENSE.txt ADDED

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+Copyright (c) 2015 TomPoulton
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md ADDED

@@ -0,0 +1,102 @@
+# Puppet Twitch
+Puppet Twitch provides a lightweight http interface for remotely triggering puppet runs. That's it! The plan was to not use more resources than it had to, and also not open up any security holes.
+Currently this project is definitely in the beta phase, there are probably a few kinks to work out, and it's worth highlighting that at the moment **puppet twitch doesn't use a "proper" http server**, it's just a TCP socket with an http shaped regex. This makes it lightweight, but there might be a whole bunch of compatibility/security issues with it, so I'm looking to swap that out (see ToDo section)
+### Triggering puppet
+To trigger a puppet run on a node running puppet-twitch, simply hit the `/puppet/twitch` endpoint over http. For example, to trigger puppet on `node01.example.com` running puppet-twitch on the default port:
+`$ curl http://node01.example.com:2023/puppet/twitch`
+Puppet-twitch will respond with one of the following (`<status code>, <body>`):
+ - `202, Triggered puppet run` - The puppet run has been triggered asynchronously
+ - `409, Puppet already running` - Either the server is already processing a request, or puppet's lock file exists indicating the agent is already running
+ - `500, <error message>` - Something went wrong!
+##### Async
+You can use the `async` parameter (default `true`) to make the request wait for the puppet run to finish before returning. Note that this can be a few minutes, so timeouts etc will need to be taken into account:
+`$ curl http://node01.example.com:2023/puppet/twitch?async=false`
+In this case puppet-twitch will respond with `200, Puppet run complete`
+### Puppet Install
+There is a puppet module called `twitch` in the source that can be used to setup and install puppet-twitch. The module isn't in the forge yet so copy and paste for now.
+To use the defaults on Linux, simply `include twitch`. This will:
+ - Create a user and group called `twitch`
+ - Add specific puppet commands to `/etc/sudoers.d/twitch` (see `init.pp` in the `twitch` module for the list of commands)
+ - Create the `/var/run/puppet-twitch` directory that is writable by the `twitch` user
+ - Install the gem
+ - Start the server as the `twitch` user on the default port (2023)
+The `twitch` class is parameterized, so user, port, run directory, etc can be configured if necessary
+##### Prerequisites
+ - **ruby 1.9.3** or above as `require_relative` isn't supported by lower versions
+ - Open the port to incoming connections, the `twitch` module will not change any network configuration.
+ - Add `/etc/sudoers.d` to the `sudoers` file. This is a common configuration and is done by default on some VMs (e.g.Vagrant boxes, EC2 instances) but the `twitch` module assumes this is already setup.
+### Manual Install:
+`$ gem install puppet-twitch` (run as root so that it's available to all users)
+Use the `puppet-twitch` command to start/stop the server
+`$ puppet-twitch {start|stop|restart|status} dir=</path/to/run/dir> [-- [bind=<binding>] [port=<port>]]`
+For example to start the server on a custom port
+`$ puppet-twitch start dir=/var/run/puppet-twitch -- port=8090`
+And then stop it
+`$ puppet-twitch stop dir=/var/run/puppet-twitch`
+##### Prerequisites
+ - All prerequisites from the puppet install apply
+ - Create a user to run the server (one that doesn't have root permissions)
+ - Allow user to run the necessary puppet commands with `sudo` and without password (see `init.pp` for commands)
+ - Create a run directory for pids and logs (writable by the user)
+##### Args:
+ - `dir`: Absolute path for directory that will store the pids and logs. **The directory must be manually created and must be writable by the user that is running the server**. It must be provided for every action (`start`/`stop`/`status` etc), there is a ToDo to fix this
+ - `bind`: The IP address to bind to, defaults to `0.0.0.0`
+ - `port`: The port to listen on, defaults to `2023`
+### Running from source (for development/testing)
+You can use `rake install` to build and install the gem locally (I recommend using `rvm` gemsets to isolate gem installs), but you can also run the code direct from source in two ways:
+ - `ruby lib/puppet-twitch/controller.rb [bind=<binding>] [port=<port>]`
+ - `ruby lib/puppet-twitch/server.rb dir=</path/to/run/dir> [-- [bind=<binding>] [port=<port>]]`
+The difference is that running `controller.rb` will run the server in the foreground and output will be logged to `STDOUT`. Running `server.rb` will start the server as a daemon and output will be logged to a logfile in the run directory
+If you want to run the server in single threaded mode (i.e. the server doesn't fork threads for incoming connections) then set `async = false` in `controller.rb`, this is useful for debugging issues that aren't related to threading.
+##### Vagrant
+To test in a more isolated environment you can use vagrant, just spin up the boxes (see `nodes.json`), ssh into a node, then run puppet. After that, puppet-twitch should be running on the node and you can test by hitting the endpoint:
+```
+[user@localhost] $ vagrant up
+[user@localhost] $ vagrant ssh node01
+[vagrant@node01] $ sudo puppet agent -t
+[vagrant@node01] $ curl http://localhost:2023/puppet/twitch
+```
+The puppet class that's applied to the nodes (`twitch::test::vagrant`) inherits from the normal `twitch` class but also:
+ - Gives the `twitch` user a password (set to `twitch`) and a shell (`/bin/bash`) so you can switch to the `twitch` user for debugging
+ - Installs the gem from the project's `pkg` directory that is mounted by vagrant, rather than downloading the gem from rubygems
+In order to get local changes running on the vagrant machines, use the `rake latest` command which builds the gem as normal, but then also creates a symlink called `puppet-twitch-latest.gem` to the built gem. This is the file that the `twitch::test::vagrant` class looks for so that you don't have to specify a version
+### ToDo:
+ - Proper http server - I tried using Sinatra but it wasn't playing nice with Daemons and logging
+ - Auth key/password - At the moment anyone with network access to the port can trigger a run, not a massive deal as all they can do is trigger a puppet run, but it might be nice to add a configurable key/phrase/password
+ - Add support for Windows - It should be easy(-ish) to configure, I tried to write everything in an agnostic manner, but need to test and fix
+ - Cache run directory - It's annoying having to provide the run directory for each action

data/bin/puppet-twitch ADDED

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+require 'rubygems'
+require 'puppet_twitch/server'

data/lib/puppet_twitch/basic_http_server.rb ADDED

@@ -0,0 +1,85 @@
+require 'socket'
+require_relative 'logger'
+require_relative 'param_parser'
+module PuppetTwitch
+  class BasicHttpServer
+    attr_accessor :logger
+    def initialize(bind, port, multiple_threads = true)
+      @bind = bind
+      @port = port
+      @multiple_threads = multiple_threads
+      @logger = PuppetTwitch::Logger.new()
+      @logger.level = PuppetTwitch::Logger::DEBUG
+      @actions = {}
+    end
+    def start()
+      @logger.info 'Starting server ...'
+      server = TCPServer.new(@bind, @port)
+      @logger.info "Listening on #{@bind}:#{@port}"
+      server_loop = proc { |socket|
+        begin
+          process_request socket
+        rescue => e
+          @logger.error e.to_s
+          socket.puts form_response(500, 'Unexpected error')
+        ensure
+          socket.close
+        end
+      }
+      loop do
+        if @multiple_threads
+          Thread.fork(server.accept, &server_loop)
+        else
+          server_loop.call server.accept
+        end
+      end
+    end
+    def process_request(socket)
+      client_ip       = socket.peeraddr[3]
+      client_hostname = socket.peeraddr[2]
+      request         = socket.gets
+      endpoint, params = get_endpoint_and_params(request)
+      @logger.info "Connection from: #{client_hostname} (#{client_ip}) | Endpoint: #{endpoint} | Params: #{params}"
+      response = @actions.has_key?(endpoint) ? @actions[endpoint].call(params) : [400, "Unrecognised endpoint: #{endpoint}"]
+      @logger.debug "#{response[0]}: #{response[1]}"
+      socket.puts form_response(response[0], response[1])
+      @logger.debug "Closing connection from #{client_ip}"
+    end
+    def endpoint(endpoint, &block)
+      @actions[endpoint] = block
+    end
+    def get_endpoint_and_params(request)
+      match = request.match(/^\s*(?<verb>GET|POST|PUT|DELETE) (?<endpoint>[\w\-\/]*)(\?(?<params>\S*)\s*|\s*)HTTP.*/)
+      return match['endpoint'], parse_params(match['params'])
+    end
+    def parse_params(param_string)
+      PuppetTwitch::ParamParser.parse(param_string.to_s.strip.split('&'))
+    end
+    def form_response(status, body)
+<<-EOF
+HTTP/1.1 #{status}
+Content-Type: text/plain
+Content-Length: #{body.bytesize}
+Connection: close
+#{body}
+EOF
+    end
+  end
+end

data/lib/puppet_twitch/controller.rb ADDED

@@ -0,0 +1,52 @@
+#!/usr/bin/env ruby
+require_relative 'basic_http_server'
+require_relative 'param_parser'
+require_relative 'puppet'
+require_relative 'version'
+module PuppetTwitch
+  args = ParamParser.parse(ARGV)
+  bind  = args[:bind] || '0.0.0.0'
+  port  = args[:port] || 2023
+  threads = true
+  @processing_request = Mutex.new
+  http_server = PuppetTwitch::BasicHttpServer.new(bind, port, threads)
+  http_server.endpoint '/puppet/twitch' do |params|
+    response = []
+    if @processing_request.try_lock
+      begin
+        if PuppetTwitch::Puppet.is_running?
+          response = [409, 'Puppet already running']
+        else
+          async = (params[:async].to_s != 'false')
+          PuppetTwitch::Puppet.run_puppet(async)
+          response = async ? [202, 'Triggered puppet run'] : [200, 'Puppet run complete']
+        end
+      rescue => e
+        http_server.logger.error e.to_s
+        response = [500, 'Error running puppet']
+      ensure
+        @processing_request.unlock
+      end
+    else
+      response = [409, 'Puppet already running']
+    end
+    response
+  end
+  http_server.endpoint '/version' do
+    [200, PuppetTwitch::VERSION]
+  end
+  http_server.endpoint '/info' do
+    [200, 'Puppet Twitch - A simple endpoint for triggering puppet']
+  end
+  http_server.start
+end

data/lib/puppet_twitch/logger.rb ADDED

@@ -0,0 +1,43 @@
+module PuppetTwitch
+  # Logger just prints to STDOUT.
+  # When the server is running in the foreground (not Daemonized),
+  # all output is visible in the terminal.
+  # When the server is running as a Daemon,
+  # STDOUT is redirected to a logfile,
+  # so this simple logger works for both scenarios
+  class Logger
+    DEBUG = 0
+    INFO  = 1
+    WARN  = 2
+    ERROR = 3
+    attr_accessor :level
+    def initialize(level = INFO)
+      @level = level
+    end
+    def debug(message)
+      log DEBUG, "Log [DEBUG]: #{message}"
+    end
+    def info(message)
+      log INFO, "Log [INFO]: #{message}"
+    end
+    def warn(message)
+      log WARN, "Log [WARN]: #{message}"
+    end
+    def error(message)
+      log ERROR, "Log [ERROR]: #{message}"
+    end
+    private
+    def log(level, message)
+      puts message if level >= @level
+    end
+  end
+end

data/lib/puppet_twitch/param_parser.rb ADDED

@@ -0,0 +1,27 @@
+module PuppetTwitch
+  class ParamParser
+    # Takes an array of parameter strings and
+    # returns a hash of the parameters and their values where
+    # the keys are the parameter names as symbols.
+    #
+    # Parameter strings can have the following formats:
+    #   'name=value'    -->    { :name => 'value' }
+    #   'name'          -->    { :name => true }
+    #
+    def self.parse(param_strings)
+      params = {}
+      param_strings.each { |pair|
+        key_value = pair.split('=')
+        if key_value.size > 2 || pair[-1] == '='
+          raise StandardError, "Invalid parameter format: #{pair}"
+        else
+          params[key_value[0].to_sym] = (key_value.size == 1) ? true : key_value[1]
+        end
+      }
+      params
+    end
+  end
+end

data/lib/puppet_twitch/puppet.rb ADDED

@@ -0,0 +1,30 @@
+module PuppetTwitch
+  class Puppet
+    class << self
+      def is_running?
+        lock_file_path_cmd = 'puppet config print agent_catalog_run_lockfile'
+        lock_file_path = `#{sudo_if_required lock_file_path_cmd}`.strip
+        if $?.to_i != 0
+          raise StandardError, "Failed to find puppet lock file path: #{lock_file_path}"
+        end
+        File.exists?(lock_file_path)
+      end
+      def run_puppet(async = true)
+        command = async ? 'puppet agent --onetime' : 'puppet agent --onetime --no-daemonize'
+        output = `#{sudo_if_required command}`
+        exit_code = $?
+        unless [0, 2].include?(exit_code.to_i) # 2 indicates a puppet change, 3 is a failure
+          raise StandardError, "Error running puppet: #{exit_code}: #{output}"
+        end
+      end
+      def sudo_if_required(command)
+        "sudo #{command}" unless Gem.win_platform?
+      end
+    end
+  end
+end

data/lib/puppet_twitch/server.rb ADDED

@@ -0,0 +1,16 @@
+require 'daemons'
+require_relative 'param_parser'
+args = PuppetTwitch::ParamParser.parse(ARGV.reject {|arg| arg == '--'})
+run_dir = args[:dir]
+abort 'Please provide a run directory' unless run_dir
+# Pids and log files will be written to run_dir directory
+options = {
+  :dir_mode   => :normal,
+  :dir        => run_dir,
+  :log_output => true,
+  :monitor    => true,
+}
+Daemons.run(File.expand_path('../controller.rb', __FILE__), options)

data/lib/puppet_twitch/version.rb ADDED

@@ -0,0 +1,3 @@
+module PuppetTwitch
+  VERSION = '0.1.0'
+end

data/puppet-twitch.gemspec ADDED

@@ -0,0 +1,24 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'puppet_twitch/version'
+excude = ['.gitignore', 'Rakefile', 'puppet/.*', 'nodes.json', 'Vagrantfile']
+Gem::Specification.new do |gem|
+  gem.name          = 'puppet-twitch'
+  gem.version       = PuppetTwitch::VERSION
+  gem.description   = 'Trigger a puppet run remotely'
+  gem.summary       = 'http trigger for puppet'
+  gem.author        = 'Tom Poulton'
+  gem.license       = 'MIT'
+  gem.homepage      = 'http://github.com/Accuity/puppet-twitch'
+  gem.files         = `git ls-files`.split($/).reject { |file| file =~ /^(#{excude.join('|')})$/ }
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ['lib']
+  gem.add_dependency('daemons', '~> 1.2.2')
+end

data/spec/puppet_twitch/basic_http_server_spec.rb ADDED

@@ -0,0 +1,25 @@
+require 'spec_helper'
+require_relative '../../lib/puppet_twitch/basic_http_server'
+module PuppetTwitch
+  describe BasicHttpServer do
+    describe '#get_endpoint_and_params' do
+      let(:basic_http_server) { BasicHttpServer.new('0.0.0.0', 1234) }
+      it 'returns endpoint' do
+        endpoint, params = basic_http_server.get_endpoint_and_params('GET /foo HTTP')
+        expect(endpoint).to eq '/foo'
+      end
+      it 'returns endpoint containing punctuation' do
+        endpoint, params = basic_http_server.get_endpoint_and_params('GET /foo_bar/baz-baa HTTP')
+        expect(endpoint).to eq '/foo_bar/baz-baa'
+      end
+    end
+  end
+end

data/spec/puppet_twitch/param_parser_spec.rb ADDED

@@ -0,0 +1,44 @@
+require 'spec_helper'
+require_relative '../../lib/puppet_twitch/param_parser'
+module PuppetTwitch
+  describe ParamParser do
+    describe '.parse' do
+      it 'converts param keys to symbols' do
+        params = ParamParser.parse ['foo=bar']
+        expect(params).to have_key :foo
+      end
+      it 'parses multiple key value params' do
+        params = ParamParser.parse ['foo=bar', 'baz=baa']
+        expect(params).to eq( {:foo => 'bar', :baz => 'baa'} )
+      end
+      it 'empty params are set to true' do
+        params = ParamParser.parse ['foo', 'bar']
+        expect(params).to eq( {:foo => true, :bar => true} )
+      end
+      it 'returns empty hash for empty input array' do
+        params = ParamParser.parse []
+        expect(params).to eq( {} )
+      end
+      it 'raises error when param is malformed' do
+        expect {
+          ParamParser.parse ['foo=bar=baz']
+        }.to raise_error StandardError, /Invalid parameter format/
+      end
+      it 'raises error when param value is missing' do
+        expect {
+          ParamParser.parse ['foo=']
+        }.to raise_error StandardError, /Invalid parameter format/
+      end
+    end
+  end
+end

data/spec/spec_helper.rb ADDED

@@ -0,0 +1,5 @@
+require 'rspec'
+RSpec.configure do |c|
+  c.color = true
+end

metadata ADDED

@@ -0,0 +1,76 @@
+--- !ruby/object:Gem::Specification
+name: puppet-twitch
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Tom Poulton
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2015-04-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: daemons
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.2.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.2.2
+description: Trigger a puppet run remotely
+email:
+executables:
+- puppet-twitch
+extensions: []
+extra_rdoc_files: []
+files:
+- Gemfile
+- LICENSE.txt
+- README.md
+- bin/puppet-twitch
+- lib/puppet_twitch/basic_http_server.rb
+- lib/puppet_twitch/controller.rb
+- lib/puppet_twitch/logger.rb
+- lib/puppet_twitch/param_parser.rb
+- lib/puppet_twitch/puppet.rb
+- lib/puppet_twitch/server.rb
+- lib/puppet_twitch/version.rb
+- puppet-twitch.gemspec
+- spec/puppet_twitch/basic_http_server_spec.rb
+- spec/puppet_twitch/param_parser_spec.rb
+- spec/spec_helper.rb
+homepage: http://github.com/Accuity/puppet-twitch
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project:
+rubygems_version: 2.4.5
+signing_key:
+specification_version: 4
+summary: http trigger for puppet
+test_files:
+- spec/puppet_twitch/basic_http_server_spec.rb
+- spec/puppet_twitch/param_parser_spec.rb
+- spec/spec_helper.rb