foreman_vault 0.0.1 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2ae077f554e08226360e85ff0f76687f9c54e312b78bedc70efef11bc7e8f334
-  data.tar.gz: dc35f0f802d0596c0f826d4220eb86006b3f58dec734d9cfd95871f26d002b6f
+  metadata.gz: 6107c566ac8bf65dd8b0c11f2c6534d26303e781407fef11f29612d5ca77325e
+  data.tar.gz: ae555560613c5b4dbe17281c05c7db8df0d244f82ffd27c56785e805c7ac04da
 SHA512:
-  metadata.gz: fcd8996dc26ffd1f5eb6f66b34aedd7859cfb866a8dc3a84b5043293fafc78475845e92b19e0e2742f0bdabd8dd6b0af6086521e35f82734fedbdb8c12f14583
-  data.tar.gz: e99e99aa6338c7d2438f1f4e9a49039161bdb40e9034e53b4d60a8e59a6971f8f001b812796dfd928e3f038ddb08e0d0434679eee8513cb44a3bd2e1bc03484f
+  metadata.gz: 3a86e4e2f4fc926c0c8ade16116830d311be2b54b00cca629465d1af4c4ecfaa84cd42614ebb8090c17b50eb92ce1f732d15969f1bbfea972e9fe489b73d12f3
+  data.tar.gz: 30afd42d83d3d91ce4d72fc4419cd2b0f360f495462d6e916b209db7619a63ada8a365169ec8903508aa738e9686a627bb9577b4648851bacffa8bf60b8592f5

data/README.md

@@ -2,26 +2,93 @@
 
 [<img src="https://opensourcelogos.aws.dmtech.cloud/dmTECH_opensource_logo.svg" height="21" width="130">](https://www.dmtech.de/)
 
-This is a plugin for Foreman that adds support for using credentials from Hashicorp Vault.
+**Foreman Vault** is a plugin for Foreman that integrates with Hashicorp Vault for different things. Currently, it offers two distinct features.
 
-## Installation
+## 1. Vault secrets in Foreman templates
 
-See [Plugins install instructions](https://theforeman.org/plugins/) for how to install Foreman plugins.
+This adds two new macros which can be used in Foreman templates:
+
+- `vault_secret` - Retreive secrets at a given Vault path
+- `vault_issue_certificate` - Issues new certificates
+
+## 2. Management of Vault Policies and AuthMethods
+
+Vault [policies](https://www.vaultproject.io/docs/concepts/policies) and [auth methods](https://www.vaultproject.io/docs/concepts/auth) (of type _cert_) can be created automatically as part of the **host orchestration**.
+Auth methods also get deleted after the host is removed from Foreman.
+
+This allows Foreman to create everything needed to access Hashicorp Vault directly from a VM using it's Puppet certificate (e.g. for _Deferred functions_ in Puppet or other CLI tools).
 
-## Usage
+## Compatibility
 
-Setup Vault "Dev" mode:
+| Foreman Version | Plugin Version |
+| --------------- | -------------- |
+| >= 2.3          | ~> 1.0         |
+| >= 1.23         | ~> 0.3, ~> 0.4 |
+| >= 1.20         | ~> 0.2         |
+
+## Requirements
+
+- Foreman >= 1.20
+- Working Vault instance
+  - with _cert_ auth enabled
+  - with _approle_ auth enabled
+  - with _kv_ secret store enabled
+- valid Vault Token
+
+**Dev Vault Instance**
+
+To run a local Vault dev environment on MacOS use:
 
 ```
 $ brew install vault
 $ vault server -dev
 $ export VAULT_ADDR='http://127.0.0.1:8200'
 $ vault secrets enable kv
+$ vault auth enable cert
+
+$ vault token create -period=60m
+[...]
 ```
 
-To set up a connection between Foreman and Vault first navigate to the "Infrastructure" > "Vault Connections" menu and then hit the button labeled "Create Vault Connection". Now you should see a form. You have to fill in name, url and token (you can receive a token with the `$ vault token create -period=60m` command) and hit the "Submit" button.
+To interact with Vault you can use Vault UI, which is available at `http://127.0.0.1:8200/ui`.
 
-You can now use `vault_secret(vault_connection_name, secret_path)` macro in your templates to fetch secrets from Vault (you can write secrets with the `$ vault write kv/my_secret foo=bar` command), e.g.
+- The AppRole auth method
+
+```
+$ vault auth enable approle
+$ vault write auth/approle/role/my-role policies="default"
+Success! Data written to: auth/approle/role/my-role
+$ vault read auth/approle/role/my-role/role-id
+Key        Value
+---        -----
+role_id    8403910c-e563-d2f2-1c77-6e26319be8b5
+$ vault write -f auth/approle/role/my-role/secret-id
+Key                   Value
+---                   -----
+secret_id             1058434b-b4aa-bf5a-b376-a15d9efb1059
+secret_id_accessor    9cc19ed7-201f-7438-782e-561edd12b2a8
+```
+
+See also [Vault CLI testing AppRole](https://gist.github.com/kamils-iRonin/d099908eaf0500de8ad9c2cea5658d01)
+
+## Installation
+
+See [Plugins install instructions](https://theforeman.org/plugins/) for how to install Foreman plugins.
+
+## Basic configuration
+
+To create a new Vault connection navigate to `Infrastructure -> Vault Connections` and hit the `Create Vault Connection` button. There you can enter a name, the Vault URL and a secret token.
+
+## Vault secrets in templates
+
+At this point you can utilize two new macros in your templates:
+
+- vault_secret(vault_connection_name, secret_path)
+- vault_issue_certificate(vault_connection_name, pki_role_path, options...)
+
+### vault_secret(vault_connection_name, secret_path)
+
+To fetch secrets from Vault (you can write secrets with the `vault write kv/my_secret foo=bar` command), e.g.
 
 ```
 <%= vault_secret('MyVault', 'kv/my_secret') %>
@@ -33,13 +100,51 @@ As result you should get secret data, e.g.
 {:foo=>"bar"}
 ```
 
+### vault_issue_certificate(vault_connection_name, pki_role_path, options...)
+
+Issueing certificates is just as easy. Be sure to have a correctly set-up PKI, meaning, configure it so you can generate certificates from within the Vault UI. This means that you'll have to set-up a CA or Intermediate CA. Once done, you can generate a certificate like this:
+
+```
+<%= vault_issue_certificate('MyVault', 'pkiEngine/issue/testRole', common_name: 'test.mydomain.com', ttl: '10s') %>
+```
+
+The _common_name_ and _ttl_ are optional, but there are [more options to configure](https://www.vaultproject.io/api/secret/pki/index.html#generate-certificate)
+
+## Vault policies and auth methods
+
+### Policies
+
+The policy is based on a new template kind `VaultPolicy` which is basically a [Vault Policy](https://www.vaultproject.io/docs/concepts/policies#policy-syntax) extended with ERB.
+
+The name of the policy is extracted from a _Magic comment_ within the template. Therefore you can use ERB to influence the name:
+
+```
+# NAME: <%= @host.owner %>-<%= @host.environment %>
+
+path "secret/foo" {
+  capabilities = ["read"]
+}
+```
+
+You can create multiple `VaultPolicy` templates and configure the default policy used in host orchestration by setting the Foreman Setting `vault_policy_template` to the desired one.
+
+**Note: If the policy renders empty (yes, you can use conditions within ERB), the orchestration is skipped!**
+
+### Auth methods
+
+[Auth methods of type `cert`](https://www.vaultproject.io/docs/auth/cert) are created with three attributes:
+
+- **certificate**: content of the Foreman setting `ssl_ca_file`
+- **allowed_common_names**: FQDN of the host which triggered the orchestration
+- **token_policies**: This is automatically linked to the policy from above
+
 ## Contributing
 
 Fork and send a Pull Request. Thanks!
 
 ## Copyright
 
-Copyright (c) 2018 dmTECH GmbH, [dmtech.de](https://www.dmtech.de/)
+Copyright (c) 2018-2020 dmTECH GmbH, [dmtech.de](https://www.dmtech.de/)
 
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
@@ -48,8 +153,8 @@ the Free Software Foundation, either version 3 of the License, or
 
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 GNU General Public License for more details.
 
 You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
+along with this program. If not, see <http://www.gnu.org/licenses/>.

data/app/controllers/concerns/foreman_vault/controller/parameters/vault_connection.rb

@@ -9,7 +9,7 @@ module ForemanVault
         class_methods do
           def vault_connection_params_filter
             Foreman::ParameterFilter.new(::VaultConnection).tap do |filter|
-              filter.permit :name, :url, :token
+              filter.permit :name, :url, :token, :role_id, :secret_id
             end
           end
         end

data/app/lib/foreman_vault/macros.rb

@@ -4,12 +4,21 @@ module ForemanVault
   module Macros
     def vault_secret(vault_connection_name, secret_path)
       vault = VaultConnection.find_by!(name: vault_connection_name)
-      raise VaultError.new(N_('Invalid token for %s'), vault.name) unless vault.token_valid?
+      raise VaultError.new(N_('Invalid token for %s'), vault.name) if vault.with_token? && !vault.token_valid?
+
       vault.fetch_secret(secret_path)
     rescue ActiveRecord::RecordNotFound => e
       raise VaultError, e.message
     end
 
+    def vault_issue_certificate(vault_connection_name, secret_path, *options)
+      vault = VaultConnection.find_by!(name: vault_connection_name)
+      raise VaultError.new(N_('Invalid token for %s'), vault.name) if vault.with_token? && !vault.token_valid?
+      vault.issue_certificate(secret_path, *options)
+    rescue ActiveRecord::RecordNotFound => e
+      raise VaultError, e.message
+    end
+
     class VaultError < Foreman::Exception; end
   end
 end

data/app/models/concerns/foreman_vault/host_extensions.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module ForemanVault
+  module HostExtensions
+    extend ActiveSupport::Concern
+
+    included do
+      include ForemanVault::Orchestration::VaultPolicy
+    end
+
+    def vault_policy
+      VaultPolicy.new(self)
+    end
+
+    def vault_auth_method
+      VaultAuthMethod.new(self)
+    end
+
+    def vault_connection
+      return unless vault_connection_name
+
+      ::VaultConnection.find_by(name: vault_connection_name)
+    end
+
+    private
+
+    def vault_connection_name
+      params['vault_connection'] || Setting['vault_connection']
+    end
+  end
+end

data/app/models/concerns/foreman_vault/orchestration/vault_policy.rb

@@ -0,0 +1,70 @@
+# frozen_string_literal: true
+
+module ForemanVault
+  module Orchestration
+    module VaultPolicy
+      extend ActiveSupport::Concern
+
+      MAGIC_COMMENT_PREFIX = '# NAME: '
+
+      included do
+        after_validation :queue_vault_push
+        before_destroy :queue_vault_destroy
+      end
+
+      protected
+
+      def queue_vault_push
+        return if !managed? || errors.any?
+        return unless orchestration_enabled?
+        return unless vault_policy.valid?
+        return unless vault_auth_method.valid?
+
+        queue.create(name: _('Push %s data to Vault') % self, priority: 100,
+                     action: [self, :set_vault])
+      end
+
+      def queue_vault_destroy
+        return if !managed? || errors.any?
+        return unless orchestration_enabled?
+        return unless vault_auth_method.valid?
+
+        queue.create(name: _('Clear %s Vault data') % self, priority: 60,
+                     action: [self, :del_vault])
+      end
+
+      # rubocop:disable Metrics/AbcSize
+      def set_vault
+        logger.info "Pushing #{name} data to Vault"
+
+        vault_policy.save if vault_policy.new?
+
+        if vault_auth_method.name != old&.vault_auth_method&.name
+          old&.vault_auth_method&.delete
+          vault_auth_method.save
+        end
+        true
+      rescue StandardError => e
+        Foreman::Logging.exception("Failed to push #{name} data to Vault.", e)
+        failure format(_('Failed to push %{name} data to Vault: %{message}\n '), name: name, message: e.message), e
+      end
+      # rubocop:enable Metrics/AbcSize
+
+      def del_vault
+        logger.info "Clearing #{name} Vault data"
+
+        vault_auth_method&.delete
+      rescue StandardError => e
+        Foreman::Logging.exception("Failed to clear #{name} Vault data", e)
+        failure format(_("Failed to clear %{name} Vault data: %{message}\n "), name: name, message: e.message), e
+      end
+
+      def orchestration_enabled?
+        return false unless Setting[:vault_orchestration_enabled]
+        return false if vault_connection.nil?
+
+        true
+      end
+    end
+  end
+end

data/app/models/concerns/foreman_vault/provisioning_template_extensions.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module ForemanVault
+  module ProvisioningTemplateExtensions
+    extend ActiveSupport::Concern
+
+    # rubocop:disable Metrics/ParameterLists
+    def render(renderer: Foreman::Renderer, host: nil, params: {}, variables: {}, mode: Foreman::Renderer::REAL_MODE, template_input_values: {}, source_klass: nil)
+      source_klass = Foreman::Renderer::Source::Database if template_kind == TemplateKind.find_by(name: 'VaultPolicy')
+
+      super
+    end
+    # rubocop:enable Metrics/ParameterLists
+  end
+end

data/app/models/setting/vault.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+class Setting
+  class Vault < ::Setting
+    BLANK_ATTRS << 'vault_connection'
+    BLANK_ATTRS << 'vault_policy_template'
+
+    def self.default_settings
+      [set_vault_connection, set_vault_policy_template, set_vault_orchestration_enabled]
+    end
+
+    def self.load_defaults
+      # Check the table exists
+      return unless super
+
+      transaction do
+        default_settings.each { |s| create! s.update(category: 'Setting::Vault') }
+      end
+
+      true
+    end
+
+    def self.humanized_category
+      N_('Vault')
+    end
+
+    class << self
+      private
+
+      def set_vault_connection
+        set(
+          'vault_connection',
+          N_('Default Vault Connection that can be override using parameters'),
+          default_vault_connection,
+          N_('Default Vault Connection'),
+          nil,
+          collection: vault_connections_collection,
+          include_blank: _('Select Vault Connection')
+        )
+      end
+
+      def default_vault_connection
+        return nil unless VaultConnection.table_exists?
+        return unless VaultConnection.unscoped.count == 1
+
+        VaultConnection.unscoped.first.name
+      end
+
+      def vault_connections_collection
+        return [] unless VaultConnection.table_exists?
+
+        proc { Hash[VaultConnection.unscoped.all.map { |vc| [vc.name, vc.name] }] }
+      end
+
+      def set_vault_policy_template
+        set(
+          'vault_policy_template',
+          N_('The name of the ProvisioningTemplate that will be used for Vault Policy'),
+          default_vault_policy_template,
+          N_('Vault Policy template name'),
+          nil,
+          collection: vault_policy_templates_collection,
+          include_blank: _('Select Template')
+        )
+      end
+
+      def default_vault_policy_template
+        ProvisioningTemplate.unscoped.of_kind(:VaultPolicy).find_by(name: 'Default Vault Policy')&.name
+      end
+
+      def vault_policy_templates_collection
+        proc { Hash[ProvisioningTemplate.unscoped.of_kind(:VaultPolicy).map { |tmpl| [tmpl.name, tmpl.name] }] }
+      end
+
+      def set_vault_orchestration_enabled
+        set(
+          'vault_orchestration_enabled',
+          N_('Enable or disable the Vault orchestration step for managing policies and auth methods'),
+          false,
+          N_('Vault Orchestration enabled')
+        )
+      end
+    end
+  end
+end

data/app/models/vault_connection.rb

@@ -5,18 +5,42 @@ class VaultConnection < ApplicationRecord
 
   validates_lengths_from_database
   validates :name, presence: true, uniqueness: true
-  validates :url, :token, presence: true
+  validates :url, presence: true
   validates :url, format: URI.regexp(['http', 'https'])
 
-  before_create :set_expire_time
-  before_update :update_expire_time
+  validates :token, presence: true, if: -> { role_id.nil? || secret_id.nil? }
+  validates :token, inclusion: { in: [nil], message: _('AppRole or token must be blank') }, unless: -> { role_id.nil? || secret_id.nil? }
+  validates :role_id, presence: true, if: -> { token.nil? }
+  validates :role_id, inclusion: { in: [nil], message: _('AppRole or token must be blank') }, unless: -> { token.nil? }
+  validates :secret_id, presence: true, if: -> { token.nil? }
+  validates :secret_id, inclusion: { in: [nil], message: _('AppRole or token must be blank') }, unless: -> { token.nil? }
 
-  scope :with_valid_token, -> { where(vault_error: nil).where('expire_time > ?', Time.zone.now) }
+  before_validation :normalize_blank_values
+  before_create :set_expire_time, unless: -> { token.nil? }
+  before_update :update_expire_time, unless: -> { token.nil? }
 
-  delegate :fetch_expire_time, :fetch_secret, to: :client
+  scope :with_approle, -> { where.not(role_id: nil).where.not(secret_id: nil) }
+  scope :with_token, -> { where.not(token: nil) }
+  scope :with_valid_token, -> { with_token.where(vault_error: nil).where('expire_time > ?', Time.zone.now) }
+
+  delegate :fetch_expire_time, :fetch_secret, :issue_certificate,
+           :policy, :policies, :put_policy, :delete_policy,
+           :set_certificate, :certificates, :delete_certificate, to: :client
+
+  def with_token?
+    token.present?
+  end
+
+  def with_approle?
+    role_id.present? && secret_id.present?
+  end
 
   def token_valid?
-    vault_error.nil? && expire_time && expire_time > Time.zone.now
+    return false unless with_token?
+    return false unless vault_error.nil?
+    return true unless expire_time
+
+    expire_time > Time.zone.now
   end
 
   def renew_token!
@@ -38,6 +62,7 @@ class VaultConnection < ApplicationRecord
     self.expire_time = fetch_expire_time
   rescue StandardError => e
     errors.add(:base, e.message)
+    Foreman::Logging.exception('Failed to set vault expiry time', e)
     throw(:abort)
   end
 
@@ -49,7 +74,13 @@ class VaultConnection < ApplicationRecord
     self.vault_error = e.message
   end
 
+  def normalize_blank_values
+    attributes.each do |column, _value|
+      self[column].present? || self[column] = nil
+    end
+  end
+
   def client
-    @client ||= ForemanVault::VaultClient.new(url, token)
+    @client ||= ForemanVault::VaultClient.new(url, token, role_id, secret_id)
   end
 end