RubyGems - puppet-twitch - Versions diffs - 0.1.0 - Mend

puppet-twitch 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (17) hide show

checksums.yaml +15 -0
data/Gemfile +12 -0
data/LICENSE.txt +21 -0
data/README.md +102 -0
data/bin/puppet-twitch +4 -0
data/lib/puppet_twitch/basic_http_server.rb +85 -0
data/lib/puppet_twitch/controller.rb +52 -0
data/lib/puppet_twitch/logger.rb +43 -0
data/lib/puppet_twitch/param_parser.rb +27 -0
data/lib/puppet_twitch/puppet.rb +30 -0
data/lib/puppet_twitch/server.rb +16 -0
data/lib/puppet_twitch/version.rb +3 -0
data/puppet-twitch.gemspec +24 -0
data/spec/puppet_twitch/basic_http_server_spec.rb +25 -0
data/spec/puppet_twitch/param_parser_spec.rb +44 -0
data/spec/spec_helper.rb +5 -0
metadata +76 -0

checksums.yaml ADDED

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    MmY4YzJlMTY4ZTZkYjAwOTRmNDdiMzdhNGE2Yjc5NGNiZDNhYWY2Zg==
+  data.tar.gz: !binary |-
+    YzQ3NDFlYThmYzNkMmMwZTNlMDE2ZjhiNjVlY2M4NzUyM2Y5NGQ5YQ==
+SHA512:
+  metadata.gz: !binary |-
+    ODQzNzQ0ODEzNGU5NWZhNzYxNGM1NjAwZTE2ZmUzYjNiNGM0MDA0OWIwNTEz
+    N2Q2YjgzMTY2ZWMwZDQ4NDM4ZjBkMjM5N2E2ZWE2MWVjZmM2YjY2MjdhNDJh
+    Mzk5ZTFkYWU2YWQ4MGNhYTkyZDk5MmM2MjhhNjA2ZWU4ZTRjOTg=
+  data.tar.gz: !binary |-
+    Mzc5MDVjMmYwOWYxNmUwMjMwY2Q5NTAyOWM3ODZmMzRmYTI1MzVkNThiODlj
+    NGQyMjYwZjAzM2M4ZWVmMWJkY2E3MWZjNGZmZWNhMzkwMDdlNTQ4Yzk0NDdl
+    YWEwOGRkMDE1YzhkODkwOGRiMjczMWRlYzg1MzJjNDlmOTMyMzk=

data/Gemfile ADDED

@@ -0,0 +1,12 @@
+source 'https://rubygems.org/'
+gemspec
+group :development do
+  gem 'puppet'
+  gem 'rake'
+end
+group :test do
+  gem 'rspec', '~> 3.2.0'
+end

data/LICENSE.txt ADDED

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+Copyright (c) 2015 TomPoulton
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md ADDED

@@ -0,0 +1,102 @@
+# Puppet Twitch
+Puppet Twitch provides a lightweight http interface for remotely triggering puppet runs. That's it! The plan was to not use more resources than it had to, and also not open up any security holes.
+Currently this project is definitely in the beta phase, there are probably a few kinks to work out, and it's worth highlighting that at the moment **puppet twitch doesn't use a "proper" http server**, it's just a TCP socket with an http shaped regex. This makes it lightweight, but there might be a whole bunch of compatibility/security issues with it, so I'm looking to swap that out (see ToDo section)
+### Triggering puppet
+To trigger a puppet run on a node running puppet-twitch, simply hit the `/puppet/twitch` endpoint over http. For example, to trigger puppet on `node01.example.com` running puppet-twitch on the default port:
+`$ curl http://node01.example.com:2023/puppet/twitch`
+Puppet-twitch will respond with one of the following (`<status code>, <body>`):
+ - `202, Triggered puppet run` - The puppet run has been triggered asynchronously
+ - `409, Puppet already running` - Either the server is already processing a request, or puppet's lock file exists indicating the agent is already running
+ - `500, <error message>` - Something went wrong!
+##### Async
+You can use the `async` parameter (default `true`) to make the request wait for the puppet run to finish before returning. Note that this can be a few minutes, so timeouts etc will need to be taken into account:
+`$ curl http://node01.example.com:2023/puppet/twitch?async=false`
+In this case puppet-twitch will respond with `200, Puppet run complete`
+### Puppet Install
+There is a puppet module called `twitch` in the source that can be used to setup and install puppet-twitch. The module isn't in the forge yet so copy and paste for now.
+To use the defaults on Linux, simply `include twitch`. This will:
+ - Create a user and group called `twitch`
+ - Add specific puppet commands to `/etc/sudoers.d/twitch` (see `init.pp` in the `twitch` module for the list of commands)
+ - Create the `/var/run/puppet-twitch` directory that is writable by the `twitch` user
+ - Install the gem
+ - Start the server as the `twitch` user on the default port (2023)
+The `twitch` class is parameterized, so user, port, run directory, etc can be configured if necessary
+##### Prerequisites
+ - **ruby 1.9.3** or above as `require_relative` isn't supported by lower versions
+ - Open the port to incoming connections, the `twitch` module will not change any network configuration.
+ - Add `/etc/sudoers.d` to the `sudoers` file. This is a common configuration and is done by default on some VMs (e.g.Vagrant boxes, EC2 instances) but the `twitch` module assumes this is already setup.
+### Manual Install:
+`$ gem install puppet-twitch` (run as root so that it's available to all users)
+Use the `puppet-twitch` command to start/stop the server
+`$ puppet-twitch {start|stop|restart|status} dir=</path/to/run/dir> [-- [bind=<binding>] [port=<port>]]`
+For example to start the server on a custom port
+`$ puppet-twitch start dir=/var/run/puppet-twitch -- port=8090`
+And then stop it
+`$ puppet-twitch stop dir=/var/run/puppet-twitch`
+##### Prerequisites
+ - All prerequisites from the puppet install apply
+ - Create a user to run the server (one that doesn't have root permissions)
+ - Allow user to run the necessary puppet commands with `sudo` and without password (see `init.pp` for commands)
+ - Create a run directory for pids and logs (writable by the user)
+##### Args:
+ - `dir`: Absolute path for directory that will store the pids and logs. **The directory must be manually created and must be writable by the user that is running the server**. It must be provided for every action (`start`/`stop`/`status` etc), there is a ToDo to fix this
+ - `bind`: The IP address to bind to, defaults to `0.0.0.0`
+ - `port`: The port to listen on, defaults to `2023`
+### Running from source (for development/testing)
+You can use `rake install` to build and install the gem locally (I recommend using `rvm` gemsets to isolate gem installs), but you can also run the code direct from source in two ways:
+ - `ruby lib/puppet-twitch/controller.rb [bind=<binding>] [port=<port>]`
+ - `ruby lib/puppet-twitch/server.rb dir=</path/to/run/dir> [-- [bind=<binding>] [port=<port>]]`
+The difference is that running `controller.rb` will run the server in the foreground and output will be logged to `STDOUT`. Running `server.rb` will start the server as a daemon and output will be logged to a logfile in the run directory
+If you want to run the server in single threaded mode (i.e. the server doesn't fork threads for incoming connections) then set `async = false` in `controller.rb`, this is useful for debugging issues that aren't related to threading.
+##### Vagrant
+To test in a more isolated environment you can use vagrant, just spin up the boxes (see `nodes.json`), ssh into a node, then run puppet. After that, puppet-twitch should be running on the node and you can test by hitting the endpoint:
+```
+[user@localhost] $ vagrant up
+[user@localhost] $ vagrant ssh node01
+[vagrant@node01] $ sudo puppet agent -t
+[vagrant@node01] $ curl http://localhost:2023/puppet/twitch
+```
+The puppet class that's applied to the nodes (`twitch::test::vagrant`) inherits from the normal `twitch` class but also:
+ - Gives the `twitch` user a password (set to `twitch`) and a shell (`/bin/bash`) so you can switch to the `twitch` user for debugging
+ - Installs the gem from the project's `pkg` directory that is mounted by vagrant, rather than downloading the gem from rubygems
+In order to get local changes running on the vagrant machines, use the `rake latest` command which builds the gem as normal, but then also creates a symlink called `puppet-twitch-latest.gem` to the built gem. This is the file that the `twitch::test::vagrant` class looks for so that you don't have to specify a version
+### ToDo:
+ - Proper http server - I tried using Sinatra but it wasn't playing nice with Daemons and logging
+ - Auth key/password - At the moment anyone with network access to the port can trigger a run, not a massive deal as all they can do is trigger a puppet run, but it might be nice to add a configurable key/phrase/password
+ - Add support for Windows - It should be easy(-ish) to configure, I tried to write everything in an agnostic manner, but need to test and fix
+ - Cache run directory - It's annoying having to provide the run directory for each action

data/bin/puppet-twitch ADDED

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+require 'rubygems'
+require 'puppet_twitch/server'

data/lib/puppet_twitch/basic_http_server.rb ADDED

@@ -0,0 +1,85 @@
+require 'socket'
+require_relative 'logger'
+require_relative 'param_parser'
+module PuppetTwitch
+  class BasicHttpServer
+    attr_accessor :logger
+    def initialize(bind, port, multiple_threads = true)
+      @bind = bind
+      @port = port
+      @multiple_threads = multiple_threads
+      @logger = PuppetTwitch::Logger.new()
+      @logger.level = PuppetTwitch::Logger::DEBUG
+      @actions = {}
+    end
+    def start()
+      @logger.info 'Starting server ...'
+      server = TCPServer.new(@bind, @port)
+      @logger.info "Listening on #{@bind}:#{@port}"
+      server_loop = proc { |socket|
+        begin
+          process_request socket
+        rescue => e
+          @logger.error e.to_s
+          socket.puts form_response(500, 'Unexpected error')
+        ensure
+          socket.close
+        end
+      }
+      loop do
+        if @multiple_threads
+          Thread.fork(server.accept, &server_loop)
+        else
+          server_loop.call server.accept
+        end
+      end
+    end
+    def process_request(socket)
+      client_ip       = socket.peeraddr[3]
+      client_hostname = socket.peeraddr[2]
+      request         = socket.gets
+      endpoint, params = get_endpoint_and_params(request)
+      @logger.info "Connection from: #{client_hostname} (#{client_ip}) | Endpoint: #{endpoint} | Params: #{params}"
+      response = @actions.has_key?(endpoint) ? @actions[endpoint].call(params) : [400, "Unrecognised endpoint: #{endpoint}"]
+      @logger.debug "#{response[0]}: #{response[1]}"
+      socket.puts form_response(response[0], response[1])
+      @logger.debug "Closing connection from #{client_ip}"
+    end
+    def endpoint(endpoint, &block)
+      @actions[endpoint] = block
+    end
+    def get_endpoint_and_params(request)
+      match = request.match(/^\s*(?<verb>GET|POST|PUT|DELETE) (?<endpoint>[\w\-\/]*)(\?(?<params>\S*)\s*|\s*)HTTP.*/)
+      return match['endpoint'], parse_params(match['params'])
+    end
+    def parse_params(param_string)
+      PuppetTwitch::ParamParser.parse(param_string.to_s.strip.split('&'))
+    end
+    def form_response(status, body)
+<<-EOF
+HTTP/1.1 #{status}
+Content-Type: text/plain
+Content-Length: #{body.bytesize}
+Connection: close
+#{body}
+EOF
+    end
+  end
+end

data/lib/puppet_twitch/controller.rb ADDED

@@ -0,0 +1,52 @@
+#!/usr/bin/env ruby
+require_relative 'basic_http_server'
+require_relative 'param_parser'
+require_relative 'puppet'
+require_relative 'version'
+module PuppetTwitch
+  args = ParamParser.parse(ARGV)
+  bind  = args[:bind] || '0.0.0.0'
+  port  = args[:port] || 2023
+  threads = true
+  @processing_request = Mutex.new
+  http_server = PuppetTwitch::BasicHttpServer.new(bind, port, threads)
+  http_server.endpoint '/puppet/twitch' do |params|
+    response = []
+    if @processing_request.try_lock
+      begin
+        if PuppetTwitch::Puppet.is_running?
+          response = [409, 'Puppet already running']
+        else
+          async = (params[:async].to_s != 'false')
+          PuppetTwitch::Puppet.run_puppet(async)
+          response = async ? [202, 'Triggered puppet run'] : [200, 'Puppet run complete']
+        end
+      rescue => e
+        http_server.logger.error e.to_s
+        response = [500, 'Error running puppet']
+      ensure
+        @processing_request.unlock
+      end
+    else
+      response = [409, 'Puppet already running']
+    end
+    response
+  end
+  http_server.endpoint '/version' do
+    [200, PuppetTwitch::VERSION]
+  end
+  http_server.endpoint '/info' do
+    [200, 'Puppet Twitch - A simple endpoint for triggering puppet']
+  end
+  http_server.start
+end

data/lib/puppet_twitch/logger.rb ADDED

@@ -0,0 +1,43 @@
+module PuppetTwitch
+  # Logger just prints to STDOUT.
+  # When the server is running in the foreground (not Daemonized),
+  # all output is visible in the terminal.
+  # When the server is running as a Daemon,
+  # STDOUT is redirected to a logfile,
+  # so this simple logger works for both scenarios
+  class Logger
+    DEBUG = 0
+    INFO  = 1
+    WARN  = 2
+    ERROR = 3
+    attr_accessor :level
+    def initialize(level = INFO)
+      @level = level
+    end
+    def debug(message)
+      log DEBUG, "Log [DEBUG]: #{message}"
+    end
+    def info(message)
+      log INFO, "Log [INFO]: #{message}"
+    end
+    def warn(message)
+      log WARN, "Log [WARN]: #{message}"
+    end
+    def error(message)
+      log ERROR, "Log [ERROR]: #{message}"
+    end
+    private
+    def log(level, message)
+      puts message if level >= @level
+    end
+  end
+end

data/lib/puppet_twitch/param_parser.rb ADDED

@@ -0,0 +1,27 @@
+module PuppetTwitch
+  class ParamParser
+    # Takes an array of parameter strings and
+    # returns a hash of the parameters and their values where
+    # the keys are the parameter names as symbols.
+    #
+    # Parameter strings can have the following formats:
+    #   'name=value'    -->    { :name => 'value' }
+    #   'name'          -->    { :name => true }
+    #
+    def self.parse(param_strings)
+      params = {}
+      param_strings.each { |pair|
+        key_value = pair.split('=')
+        if key_value.size > 2 || pair[-1] == '='
+          raise StandardError, "Invalid parameter format: #{pair}"
+        else
+          params[key_value[0].to_sym] = (key_value.size == 1) ? true : key_value[1]
+        end
+      }
+      params
+    end
+  end
+end

data/lib/puppet_twitch/puppet.rb ADDED

@@ -0,0 +1,30 @@
+module PuppetTwitch
+  class Puppet
+    class << self
+      def is_running?
+        lock_file_path_cmd = 'puppet config print agent_catalog_run_lockfile'
+        lock_file_path = `#{sudo_if_required lock_file_path_cmd}`.strip
+        if $?.to_i != 0
+          raise StandardError, "Failed to find puppet lock file path: #{lock_file_path}"
+        end
+        File.exists?(lock_file_path)
+      end
+      def run_puppet(async = true)
+        command = async ? 'puppet agent --onetime' : 'puppet agent --onetime --no-daemonize'
+        output = `#{sudo_if_required command}`
+        exit_code = $?
+        unless [0, 2].include?(exit_code.to_i) # 2 indicates a puppet change, 3 is a failure
+          raise StandardError, "Error running puppet: #{exit_code}: #{output}"
+        end
+      end
+      def sudo_if_required(command)
+        "sudo #{command}" unless Gem.win_platform?
+      end
+    end
+  end
+end

data/lib/puppet_twitch/server.rb ADDED

@@ -0,0 +1,16 @@
+require 'daemons'
+require_relative 'param_parser'
+args = PuppetTwitch::ParamParser.parse(ARGV.reject {|arg| arg == '--'})
+run_dir = args[:dir]
+abort 'Please provide a run directory' unless run_dir
+# Pids and log files will be written to run_dir directory
+options = {
+  :dir_mode   => :normal,
+  :dir        => run_dir,
+  :log_output => true,
+  :monitor    => true,
+}
+Daemons.run(File.expand_path('../controller.rb', __FILE__), options)

data/lib/puppet_twitch/version.rb ADDED

@@ -0,0 +1,3 @@
+module PuppetTwitch
+  VERSION = '0.1.0'
+end

data/puppet-twitch.gemspec ADDED

@@ -0,0 +1,24 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'puppet_twitch/version'
+excude = ['.gitignore', 'Rakefile', 'puppet/.*', 'nodes.json', 'Vagrantfile']
+Gem::Specification.new do |gem|
+  gem.name          = 'puppet-twitch'
+  gem.version       = PuppetTwitch::VERSION
+  gem.description   = 'Trigger a puppet run remotely'
+  gem.summary       = 'http trigger for puppet'
+  gem.author        = 'Tom Poulton'
+  gem.license       = 'MIT'
+  gem.homepage      = 'http://github.com/Accuity/puppet-twitch'
+  gem.files         = `git ls-files`.split($/).reject { |file| file =~ /^(#{excude.join('|')})$/ }
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ['lib']
+  gem.add_dependency('daemons', '~> 1.2.2')
+end

data/spec/puppet_twitch/basic_http_server_spec.rb ADDED

@@ -0,0 +1,25 @@
+require 'spec_helper'
+require_relative '../../lib/puppet_twitch/basic_http_server'
+module PuppetTwitch
+  describe BasicHttpServer do
+    describe '#get_endpoint_and_params' do
+      let(:basic_http_server) { BasicHttpServer.new('0.0.0.0', 1234) }
+      it 'returns endpoint' do
+        endpoint, params = basic_http_server.get_endpoint_and_params('GET /foo HTTP')
+        expect(endpoint).to eq '/foo'
+      end
+      it 'returns endpoint containing punctuation' do
+        endpoint, params = basic_http_server.get_endpoint_and_params('GET /foo_bar/baz-baa HTTP')
+        expect(endpoint).to eq '/foo_bar/baz-baa'
+      end
+    end
+  end
+end

data/spec/puppet_twitch/param_parser_spec.rb ADDED

@@ -0,0 +1,44 @@
+require 'spec_helper'
+require_relative '../../lib/puppet_twitch/param_parser'
+module PuppetTwitch
+  describe ParamParser do
+    describe '.parse' do
+      it 'converts param keys to symbols' do
+        params = ParamParser.parse ['foo=bar']
+        expect(params).to have_key :foo
+      end
+      it 'parses multiple key value params' do
+        params = ParamParser.parse ['foo=bar', 'baz=baa']
+        expect(params).to eq( {:foo => 'bar', :baz => 'baa'} )
+      end
+      it 'empty params are set to true' do
+        params = ParamParser.parse ['foo', 'bar']
+        expect(params).to eq( {:foo => true, :bar => true} )
+      end
+      it 'returns empty hash for empty input array' do
+        params = ParamParser.parse []
+        expect(params).to eq( {} )
+      end
+      it 'raises error when param is malformed' do
+        expect {
+          ParamParser.parse ['foo=bar=baz']
+        }.to raise_error StandardError, /Invalid parameter format/
+      end
+      it 'raises error when param value is missing' do
+        expect {
+          ParamParser.parse ['foo=']
+        }.to raise_error StandardError, /Invalid parameter format/
+      end
+    end
+  end
+end

data/spec/spec_helper.rb ADDED

@@ -0,0 +1,5 @@
+require 'rspec'
+RSpec.configure do |c|
+  c.color = true
+end

metadata ADDED

@@ -0,0 +1,76 @@
+--- !ruby/object:Gem::Specification
+name: puppet-twitch
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Tom Poulton
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2015-04-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: daemons
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.2.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.2.2
+description: Trigger a puppet run remotely
+email:
+executables:
+- puppet-twitch
+extensions: []
+extra_rdoc_files: []
+files:
+- Gemfile
+- LICENSE.txt
+- README.md
+- bin/puppet-twitch
+- lib/puppet_twitch/basic_http_server.rb
+- lib/puppet_twitch/controller.rb
+- lib/puppet_twitch/logger.rb
+- lib/puppet_twitch/param_parser.rb
+- lib/puppet_twitch/puppet.rb
+- lib/puppet_twitch/server.rb
+- lib/puppet_twitch/version.rb
+- puppet-twitch.gemspec
+- spec/puppet_twitch/basic_http_server_spec.rb
+- spec/puppet_twitch/param_parser_spec.rb
+- spec/spec_helper.rb
+homepage: http://github.com/Accuity/puppet-twitch
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project:
+rubygems_version: 2.4.5
+signing_key:
+specification_version: 4
+summary: http trigger for puppet
+test_files:
+- spec/puppet_twitch/basic_http_server_spec.rb
+- spec/puppet_twitch/param_parser_spec.rb
+- spec/spec_helper.rb