foreman_dlm 0.1.0

data/README.md

@@ -0,0 +1,84 @@
+# Foreman Distributed Lock Manager
+
+This is a plugin for Foreman that allows Foreman to act as a distributed lock manager.
+Updates are key to security, but updates of an operating system are hard to apply and existing tools are hard to manage at scale. This might lead to a large drift between important security updates becoming available and all your hosts being successfully patched. Security experts recommend to install updates as soon as they come available. The ability to easily update software is the most effective way to improve server security. Automation is key to ensure this goal is reached.
+This plug-in aims to provide painless updates of the operating system. This keeps your company more secure and frees up resources of your operations team for more important tasks.
+
+With this plugin servers can acquire a lock in Foreman to ensure only one server in a cluster installs updates at the same time. This can successfully prevent service disruptions.
+
+## Compatibility
+
+| Foreman Version | Plugin Version |
+| --------------- | -------------- |
+| >= 1.15         | any            |
+
+## Installation
+
+See [Plugins install instructions](https://theforeman.org/plugins/)
+for how to install Foreman plugins.
+On Enterprise Linux, you need to install the package `tfm-rubygem-foreman_dlm`.
+
+## Usage
+
+Servers are identified by their puppet certificates. Support for using subscription-manager certificates is planned, but currently not implemented.
+Here you find a usage example using `curl` that shows how to acquire the `test` lock. The lock will be created if it does not exist.
+
+```
+curl -D - -H 'Content-Type: application/json' --key $(puppet config print hostprivkey) --cert $(puppet config print hostcert) -X PUT https://foreman.example.com/api/dlmlocks/test/lock
+```
+
+Use the HTTP method `GET` to show a lock, `PUT` to acquire a lock and `DELETE` to release a lock.
+Foreman will respond with the HTTP status code `200 OK` if the action was successful and `412 Precondition Failed` if the lock could not be acquired or release. This may happen, if the lock is taken by another host.
+
+To process the HTTP status code in a bash script, you can do something like this:
+```
+curl --write-out %{http_code} -H 'Content-Type: application/json' -sS -o /dev/null -X PUT --key $(puppet config print hostprivkey) --cert $(puppet config print hostcert) https://foreman.example.com/api/dlmlocks/test/lock
+```
+
+## Note about curl on macOS
+
+macOS uses curl with a different ssl library which gets problematic testing the cert-signed requests.
+The Error:
+`WARNING: SSL: CURLOPT_SSLKEY is ignored by Secure Transport. The private key must be in the Keychain.`
+
+A fix is described here:
+https://github.com/curl/curl/issues/283
+
+```
+$ brew install curl --with-openssl
+$ brew link curl --force
+```
+
+After that `curl --version` changes from
+```
+$ curl --version
+curl 7.54.0 (x86_64-apple-darwin16.0) libcurl/7.54.0 SecureTransport zlib/1.2.8
+```
+to
+```
+$ curl --version
+curl 7.56.1 (x86_64-apple-darwin16.7.0) libcurl/7.56.1 OpenSSL/1.0.2m zlib/1.2.8
+```
+
+
+## Contributing
+
+Fork and send a Pull Request. Thanks!
+
+## Copyright
+
+Copyright (c) 2018 dm-drogerie markt GmbH & Co. KG, https://dm.de
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+

data/Rakefile

@@ -0,0 +1,47 @@
+#!/usr/bin/env rake
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+begin
+  require 'rdoc/task'
+rescue LoadError
+  require 'rdoc/rdoc'
+  require 'rake/rdoctask'
+  RDoc::Task = Rake::RDocTask
+end
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'ForemanDlm'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+APP_RAKEFILE = File.expand_path('../test/dummy/Rakefile', __FILE__)
+
+Bundler::GemHelper.install_tasks
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+task default: :test
+
+begin
+  require 'rubocop/rake_task'
+  RuboCop::RakeTask.new
+rescue => _
+  puts 'Rubocop not loaded.'
+end
+
+task :default do
+  Rake::Task['rubocop'].execute
+end

data/app/controllers/api/v2/dlmlocks_controller.rb

@@ -0,0 +1,157 @@
+module Api
+  module V2
+    class DlmlocksController < V2::BaseController
+      include Api::Version2
+      include Foreman::Controller::Parameters::Dlmlocks
+      include ::ForemanDlm::FindHostByClientCert
+
+      wrap_parameters Dlmlock, :include => dlmlocks_params_filter.accessible_attributes(parameter_filter_context)
+
+      authorize_host_by_client_cert [:show, :release, :acquire]
+
+      before_action :find_resource, :only => [:show, :update, :destroy]
+      before_action :find_resource_or_create, :only => [:release, :acquire]
+      before_action :find_host, :only => [:release, :acquire]
+      before_action :setup_search_options, :only => [:index]
+
+      def_param_group :dlmlock do
+        param :dlmlock, Hash, :required => true, :action_aware => true do
+          param :name, String, :required => true, :desc => N_('Name')
+          param :type, ['Dlmlock:Update'], :required => true, :desc => N_('Type, e.g. Dlmlock:Update')
+          param :enabled, :bool, :desc => N_('Enable the lock')
+        end
+      end
+
+      api :GET, '/dlmlocks/', N_('List all DLM locks')
+      param_group :search_and_pagination, ::Api::V2::BaseController
+
+      def index
+        @dlmlocks = resource_scope_for_index
+        @total = resource_scope_for_index.count
+      end
+
+      api :GET, '/dlmlocks/:id/', N_('Show a DLM lock')
+      api :GET, '/dlmlocks/:id/lock', N_('Show a DLM lock')
+      error 404, 'Lock could not be found.'
+      param :id, String, :required => true, :desc => N_('Id or name of the DLM lock')
+
+      def show; end
+
+      api :POST, '/dlmlocks', N_('Create a DLM lock')
+      param_group :dlmlock, :as => :create
+
+      def create
+        @dlmlock = Dlmlock.new(dlmlocks_params)
+        process_response @dlmlock.save
+      end
+
+      api :PUT, "/dlmlocks/:id/", N_("Update a DLM lock")
+      param :id, String, :required => true, :desc => N_('Id or name of the DLM lock')
+      param_group :dlmlock
+
+      def update
+        process_response @dlmlock.update_attributes(dlmlocks_params)
+      end
+
+      api :DELETE, '/dlmlocks/:id/', N_('Delete a DLM lock')
+      param :id, String, :required => true, :desc => N_('Id or name of the DLM lock')
+
+      def destroy
+        process_response @dlmlock.destroy
+      end
+
+      api :PUT, '/dlmlocks/:id/lock', N_('Acquire a DLM lock')
+      param :id, String, :required => true, :desc => N_('Id or name of the DLM lock')
+      error 200, 'Lock acquired successfully.'
+      error 412, 'Lock could not be acquired.'
+      description <<-EOS
+        == Acquire a lock
+        This action acquires a lock.
+        It fails, if the lock is currently taken by another host.
+
+        == Authentication & Host Identification
+        The host is authenticated via a client certificate and identified via the CN of that certificate.
+        EOS
+
+      def acquire
+        process_lock_response @dlmlock.acquire!(@host)
+      end
+
+      api :DELETE, '/dlmlocks/:id/lock', N_('Release a DLM lock')
+      param :id, String, :required => true, :desc => N_('Id or name of the DLM lock')
+      error 200, 'Lock released successfully.'
+      error 412, 'Lock could not be released.'
+
+      description <<-EOS
+        == Release a lock
+         This action releases a lock.
+         It fails, if the lock is currently taken by another host.
+
+        == Authentication & Host Identification
+         The host is authenticated via a client certificate and identified via the CN of that certificate.
+        EOS
+
+      def release
+        process_lock_response @dlmlock.release!(@host)
+      end
+
+      private
+
+      def resource_finder(scope, id)
+        super
+      rescue ActiveRecord::RecordNotFound
+        result = scope.find_by(:name => id)
+        raise ActiveRecord::RecordNotFound unless result
+        result
+      end
+
+      def find_resource_or_create
+        find_resource
+      rescue ActiveRecord::RecordNotFound
+        @dlmlock = Dlmlock.create(:name => params[:id], :type => 'Dlmlock::Update')
+      end
+
+      def action_permission
+        case params[:action]
+          when 'release', 'acquire'
+            :edit
+          else
+            super
+        end
+      end
+
+      def find_host
+        @host = detected_host
+        unless @host
+          logger.info 'Denying access because no host could be detected.'
+          if User.current
+            render_error 'access_denied', :status => :forbidden, :locals => { :details => 'You need to authenticate with a valid client cert. The DN has to match a known host.' }
+          else
+            render_error 'unauthorized', :status => :unauthorized, :locals => { :user_login => get_client_cert_hostname }
+          end
+        end
+        true
+      end
+
+      def process_lock_response(condition, response = nil)
+        if condition
+          process_success response
+        else
+          process_lock_resource_error
+        end
+      end
+
+      def process_lock_resource_error(options = {})
+        resource = options[:resource] || get_resource(options[:message])
+
+        if resource.respond_to?(:permission_failed?) && resource.permission_failed?
+          deny_access
+        else
+          render_error 'precondition_failed', :status => :precondition_failed, :locals => {
+            :message => 'Precondition failed. Lock is in invalid state for this operation.',
+          }
+        end
+      end
+    end
+  end
+end

data/app/controllers/concerns/foreman/controller/parameters/dlmlocks.rb

@@ -0,0 +1,15 @@
+module Foreman::Controller::Parameters::Dlmlocks
+  extend ActiveSupport::Concern
+
+  class_methods do
+    def dlmlocks_params_filter
+      Foreman::ParameterFilter.new(::SshKey).tap do |filter|
+        filter.permit :name, :type, :host_id, :enabled
+      end
+    end
+  end
+
+  def dlmlocks_params
+    self.class.dlmlocks_params_filter.filter_params(params, parameter_filter_context)
+  end
+end

data/app/controllers/concerns/foreman_dlm/find_host_by_client_cert.rb

@@ -0,0 +1,63 @@
+module ForemanDlm
+  module FindHostByClientCert
+    extend ActiveSupport::Concern
+
+    module ClassMethods
+      def authorize_host_by_client_cert(actions, options = {})
+        skip_before_action :require_login, :only => actions, :raise => false
+        skip_before_action :authorize, :only => actions
+        skip_before_action :verify_authenticity_token, :only => actions
+        skip_before_action :set_taxonomy, :only => actions, :raise => false
+        skip_before_action :session_expiry, :update_activity_time, :only => actions
+        before_action(:only => actions) { require_client_cert_or_login }
+        attr_reader :detected_host
+      end
+    end
+
+    private
+
+    # Permits Hosts authorized by their client cert
+    # or a user with permission
+    def require_client_cert_or_login
+      @detected_host = find_host_by_client_cert
+
+      if detected_host
+        set_admin_user
+        return true
+      end
+
+      require_login
+      unless User.current
+        render_error 'unauthorized', :status => :unauthorized unless performed? && api_request?
+        return false
+      end
+      authorize
+    end
+
+    def find_host_by_client_cert
+      hostname = get_client_cert_hostname
+
+      return unless hostname
+
+      host ||= Host::Base.find_by_certname(hostname) ||
+      Host::Base.find_by_name(hostname)
+      logger.info { "Found Host #{host} by client cert #{hostname}" } if host
+      host
+    end
+
+    def get_client_cert_hostname
+      verify = request.env[Setting[:ssl_client_verify_env]]
+      unless verify == 'SUCCESS'
+        logger.info { "Client certificate is invalid: #{verify}" }
+        return
+      end
+
+      dn = request.env[Setting[:ssl_client_dn_env]]
+      return unless (dn && dn =~ /CN=([^\s\/,]+)/i)
+
+      hostname = $1.downcase
+      logger.debug "Extracted hostname '#{hostname}' from client certificate."
+      hostname
+    end
+  end
+end

data/app/controllers/concerns/foreman_dlm/find_host_by_ip.rb

@@ -0,0 +1,54 @@
+module ForemanDlm
+  module FindHostByIp
+    extend ActiveSupport::Concern
+
+    module ClassMethods
+      def authorize_host_by_ip(actions, options = {})
+        skip_before_action :require_login, :only => actions, :raise => false
+        skip_before_action :authorize, :only => actions
+        skip_before_action :verify_authenticity_token, :only => actions
+        skip_before_action :set_taxonomy, :only => actions, :raise => false
+        skip_before_action :session_expiry, :update_activity_time, :only => actions
+        before_action(:only => actions) { require_ip_auth_or_login }
+        attr_reader :detected_host
+      end
+    end
+
+    private
+
+    # Permits Hosts with an IP or a user with permission
+    def require_ip_auth_or_login
+      @detected_host = find_host_by_ip
+
+      if detected_host
+        set_admin_user
+        return true
+      end
+
+      require_login
+      unless User.current
+        render_error 'access_denied', :status => :forbidden unless performed? && api_request?
+        return false
+      end
+      authorize
+    end
+
+    def find_host_by_ip
+      # try to find host based on our client ip address
+      ip = ip_from_request_env
+
+      # in case we got back multiple ips (see #1619)
+      ip = ip.split(',').first
+
+      # host is readonly because of association so we reload it if we find it
+      host = Host.joins(:provision_interface).where(:nics => { :ip => ip }).first
+      host = host ? Host.find(host.id) : nil
+      logger.info { "Found Host #{host} by request IP #{ip}" } if host
+      host
+    end
+
+    def ip_from_request_env
+      request.env['REMOTE_ADDR']
+    end
+  end
+end

data/app/controllers/dlmlocks_controller.rb

@@ -0,0 +1,13 @@
+class DlmlocksController < ::ApplicationController
+  include Foreman::Controller::AutoCompleteSearch
+
+  before_action :setup_search_options, :only => :index
+  before_action :find_resource, :only => [:show]
+
+  def index
+    @dlmlocks = resource_base_search_and_page(:host)
+  end
+
+  def show
+  end
+end

data/app/helpers/foreman_dlm/dlmlock_helper.rb

@@ -0,0 +1,15 @@
+module ForemanDlm
+  module DlmlockHelper
+    def dlmlock_status_icon_class(lock)
+      return 'ban' if lock.disabled?
+      return 'lock' if lock.taken?
+      'unlock'
+    end
+
+    def dlmlock_status_icon_color_class(lock)
+      return 'text-danger' if lock.disabled?
+      return 'text-success' if lock.taken?
+      'text-info'
+    end
+  end
+end

data/app/models/concerns/foreman_dlm/host_extensions.rb

@@ -0,0 +1,14 @@
+module ForemanDlm
+  module HostExtensions
+    extend ActiveSupport::Concern
+
+    included do
+      has_many :dlmlocks,
+        foreign_key: 'host_id',
+        dependent: :nullify
+
+      define_model_callbacks :lock, :only => :after
+      define_model_callbacks :unlock, :only => :after
+    end
+  end
+end

data/app/models/concerns/foreman_dlm/host_monitoring_extensions.rb

@@ -0,0 +1,36 @@
+module ForemanDlm
+  module HostMonitoringExtensions
+    extend ActiveSupport::Concern
+
+    included do
+      after_lock :add_lock_monitoring_downtime
+      after_unlock :remove_lock_monitoring_downtime
+    end
+
+    def add_lock_monitoring_downtime
+      return unless monitored?
+      logger.info "Setting Monitoring downtime for #{self}"
+      monitoring.set_downtime_host(self, lock_monitoring_downtime_options)
+      true
+    rescue ProxyAPI::ProxyException => e
+      Foreman::Logging.exception("Unable to set monitoring downtime for #{e}", e)
+    end
+
+    def remove_lock_monitoring_downtime
+      return unless monitored?
+      logger.info "Deleting Monitoring downtime for #{self}"
+      monitoring.del_downtime_host(self, lock_monitoring_downtime_options)
+      true
+    rescue ProxyAPI::ProxyException => e
+      Foreman::Logging.exception("Unable to remove monitoring downtime for #{e}", e)
+    end
+
+    def lock_monitoring_downtime_options
+      {
+        comment: _('Host acquired lock.'),
+        start_time: Time.current.to_i,
+        end_time: Time.current.advance(:minutes => 180).to_i
+      }
+    end
+  end
+end

data/app/models/dlmlock/update.rb

@@ -0,0 +1,5 @@
+class Dlmlock::Update < Dlmlock
+  def humanized_type
+    _('Update Lock')
+  end
+end

data/app/models/dlmlock.rb

@@ -0,0 +1,79 @@
+class Dlmlock < ActiveRecord::Base
+  include Authorizable
+
+  def self.humanize_class_name
+    N_('Distributed Lock')
+  end
+
+  belongs_to_host
+  audited
+
+  validates :name, presence: true, uniqueness: true
+
+  scoped_search :on => :name, :complete_value => true, :default_order => true
+  scoped_search :relation => :host, :on => :name, :complete_value => true, :rename => :host
+  scoped_search :on => :type, :complete_value => true, :default_order => true
+  scoped_search :on => :enabled, :complete_value => { :true => true, :false => false }, :only_explicit => true
+
+  attr_accessor :old
+
+  def acquire!(host)
+    atomic_update(nil, host)
+  end
+
+  def release!(host)
+    atomic_update(host, nil)
+  end
+
+  def locked_by?(host)
+    self.host == host
+  end
+  alias_method :acquired_by?, :locked_by?
+
+  def disabled?
+    !enabled?
+  end
+
+  def locked?
+    host.present?
+  end
+  alias_method :taken?, :locked?
+
+  def humanized_type
+    _('Generic Lock')
+  end
+
+  private
+
+  def atomic_update(old_host, new_host)
+    changes = {
+      host_id: new_host.try(:id)
+    }
+    self.old = dup
+    num_updated = self.class.where(
+      id: id,
+      host_id: [new_host.try(:id), old_host.try(:id)],
+      enabled: true
+    ).update_all(changes.merge(updated_at: DateTime.now))
+    if num_updated > 0
+      reload
+      process_host_change(old_host, new_host, changes)
+      return self
+    end
+    false
+  end
+
+  def process_host_change(old_host, new_host, changes)
+    return if host.try(:id) == old.host.try(:id)
+    write_audit(action: 'update', audited_changes: changes)
+    run_callback(old_host, :unlock) if old.host
+    run_callback(new_host, :lock) if host
+  end
+
+  def run_callback(h, callback)
+    h.run_callbacks callback do
+      logger.debug { "custom hook after_#{callback} on #{h} will be executed if defined." }
+      true
+    end
+  end
+end

data/app/views/api/v2/dlmlocks/acquire.json.rabl

@@ -0,0 +1,3 @@
+object @dlmlock
+
+extends 'api/v2/dlmlocks/main'

data/app/views/api/v2/dlmlocks/base.json.rabl

@@ -0,0 +1,3 @@
+object @dlmlock
+
+attributes :id, :host_id, :name, :enabled, :type

data/app/views/api/v2/dlmlocks/create.json.rabl

@@ -0,0 +1,3 @@
+object @dlmlock
+
+extends 'api/v2/dlmlocks/show'

data/app/views/api/v2/dlmlocks/index.json.rabl

@@ -0,0 +1,3 @@
+collection @dlmlocks
+
+extends 'api/v2/dlmlocks/main'

data/app/views/api/v2/dlmlocks/main.json.rabl

@@ -0,0 +1,7 @@
+object @dlmlock
+
+extends 'api/v2/dlmlocks/base'
+
+attributes :created_at, :updated_at
+
+child(:host) { attributes :name }

data/app/views/api/v2/dlmlocks/release.json.rabl

@@ -0,0 +1,3 @@
+object @dlmlock
+
+extends 'api/v2/dlmlocks/main'

data/app/views/api/v2/dlmlocks/show.json.rabl

@@ -0,0 +1,8 @@
+object @dlmlock
+
+extends 'api/v2/dlmlocks/main'
+
+child :host do
+  node(:name) { |host| host.name }
+  node(:self) { |host| host == @detected_host } if @detected_host
+end

data/app/views/api/v2/dlmlocks/update.json.rabl

@@ -0,0 +1,3 @@
+object @dlmlock
+
+extends 'api/v2/dlmlocks/show'

data/app/views/api/v2/errors/precondition_failed.json.rabl

@@ -0,0 +1,5 @@
+object resource = controller.get_resource
+
+attributes :id
+
+node(:message) { locals[:message] || 'Precondition failed.' }

data/app/views/dlmlocks/_details.html.erb

@@ -0,0 +1,35 @@
+<table class="<%= table_css_classes('table-fixed') %>">
+  <thead>
+    <tr>
+      <th><%= _("Owner") %></th>
+      <th><%= _("Initiator") %></th>
+      <th><%= _("Timestamp") %></th>
+      <th><%= _("Enabled") %></th>
+    </tr>
+  </thead>
+  <tbody>
+    <% @dlmlock.audits.includes(:user).reorder(created_at: :desc).each do |audit| %>
+      <% revision = audit.revision %>
+      <tr>
+        <td>
+          <% if revision.host.present? %>
+            <%= link_to_if_authorized(revision.host.name, hash_for_host_path(:id => revision.host)) %>
+          <% end %>
+        </td>
+        <td>
+          <% if audit.user.hidden? %>
+            <em><%= audit.user.name %></em>
+          <% else %>
+            <%= link_to_if_authorized(audit.user.name, hash_for_edit_user_path(audit.user)) %>
+          <% end %>
+        </td>
+        <td>
+          <%= audit.created_at %>
+        </td>
+        <td>
+          <%= revision.enabled? %>
+        </td>
+      </tr>
+    <% end %>
+  </tbody>
+</table>