foreman_vault 0.0.1 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +115 -10
  3. data/app/controllers/concerns/foreman_vault/controller/parameters/vault_connection.rb +1 -1
  4. data/app/lib/foreman_vault/macros.rb +10 -1
  5. data/app/models/concerns/foreman_vault/host_extensions.rb +31 -0
  6. data/app/models/concerns/foreman_vault/orchestration/vault_policy.rb +70 -0
  7. data/app/models/concerns/foreman_vault/provisioning_template_extensions.rb +15 -0
  8. data/app/models/setting/vault.rb +85 -0
  9. data/app/models/vault_connection.rb +38 -7
  10. data/app/services/foreman_vault/vault_auth_method.rb +58 -0
  11. data/app/services/foreman_vault/vault_client.rb +24 -6
  12. data/app/services/foreman_vault/vault_policy.rb +65 -0
  13. data/app/views/unattended/provisioning_templates/VaultPolicy/default.erb +6 -0
  14. data/app/views/vault_connections/_form.html.erb +20 -4
  15. data/app/views/vault_connections/index.html.erb +21 -7
  16. data/app/views/vault_connections/welcome.html.erb +12 -0
  17. data/db/migrate/20201203220058_add_approle_to_vault_connection.rb +8 -0
  18. data/db/seeds.d/103-provisioning_templates.rb +25 -0
  19. data/lib/foreman_vault/engine.rb +15 -3
  20. data/lib/foreman_vault/version.rb +1 -1
  21. data/test/factories/{foreman_vault_factories.rb → vault_connection.rb} +3 -3
  22. data/test/factories/vault_policy_template.rb +11 -0
  23. data/test/factories/vault_setting.rb +10 -0
  24. data/test/fixtures/ca.crt +21 -0
  25. data/test/functional/api/v2/vault_connections_controller_test.rb +1 -1
  26. data/test/models/foreman_vault/orchestration/vault_policy_test.rb +167 -0
  27. data/test/models/vault_policy_template_test.rb +28 -0
  28. data/test/test_plugin_helper.rb +0 -1
  29. data/test/unit/services/foreman_vault/vault_auth_method_test.rb +130 -0
  30. data/test/unit/services/foreman_vault/vault_client_test.rb +67 -8
  31. data/test/unit/services/foreman_vault/vault_policy_test.rb +171 -0
  32. metadata +33 -10
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2ae077f554e08226360e85ff0f76687f9c54e312b78bedc70efef11bc7e8f334
4
- data.tar.gz: dc35f0f802d0596c0f826d4220eb86006b3f58dec734d9cfd95871f26d002b6f
3
+ metadata.gz: 6107c566ac8bf65dd8b0c11f2c6534d26303e781407fef11f29612d5ca77325e
4
+ data.tar.gz: ae555560613c5b4dbe17281c05c7db8df0d244f82ffd27c56785e805c7ac04da
5
5
  SHA512:
6
- metadata.gz: fcd8996dc26ffd1f5eb6f66b34aedd7859cfb866a8dc3a84b5043293fafc78475845e92b19e0e2742f0bdabd8dd6b0af6086521e35f82734fedbdb8c12f14583
7
- data.tar.gz: e99e99aa6338c7d2438f1f4e9a49039161bdb40e9034e53b4d60a8e59a6971f8f001b812796dfd928e3f038ddb08e0d0434679eee8513cb44a3bd2e1bc03484f
6
+ metadata.gz: 3a86e4e2f4fc926c0c8ade16116830d311be2b54b00cca629465d1af4c4ecfaa84cd42614ebb8090c17b50eb92ce1f732d15969f1bbfea972e9fe489b73d12f3
7
+ data.tar.gz: 30afd42d83d3d91ce4d72fc4419cd2b0f360f495462d6e916b209db7619a63ada8a365169ec8903508aa738e9686a627bb9577b4648851bacffa8bf60b8592f5
data/README.md CHANGED
@@ -2,26 +2,93 @@
2
2
 
3
3
  [<img src="https://opensourcelogos.aws.dmtech.cloud/dmTECH_opensource_logo.svg" height="21" width="130">](https://www.dmtech.de/)
4
4
 
5
- This is a plugin for Foreman that adds support for using credentials from Hashicorp Vault.
5
+ **Foreman Vault** is a plugin for Foreman that integrates with Hashicorp Vault for different things. Currently, it offers two distinct features.
6
6
 
7
- ## Installation
7
+ ## 1. Vault secrets in Foreman templates
8
8
 
9
- See [Plugins install instructions](https://theforeman.org/plugins/) for how to install Foreman plugins.
9
+ This adds two new macros which can be used in Foreman templates:
10
+
11
+ - `vault_secret` - Retreive secrets at a given Vault path
12
+ - `vault_issue_certificate` - Issues new certificates
13
+
14
+ ## 2. Management of Vault Policies and AuthMethods
15
+
16
+ Vault [policies](https://www.vaultproject.io/docs/concepts/policies) and [auth methods](https://www.vaultproject.io/docs/concepts/auth) (of type _cert_) can be created automatically as part of the **host orchestration**.
17
+ Auth methods also get deleted after the host is removed from Foreman.
18
+
19
+ This allows Foreman to create everything needed to access Hashicorp Vault directly from a VM using it's Puppet certificate (e.g. for _Deferred functions_ in Puppet or other CLI tools).
10
20
 
11
- ## Usage
21
+ ## Compatibility
12
22
 
13
- Setup Vault "Dev" mode:
23
+ | Foreman Version | Plugin Version |
24
+ | --------------- | -------------- |
25
+ | >= 2.3 | ~> 1.0 |
26
+ | >= 1.23 | ~> 0.3, ~> 0.4 |
27
+ | >= 1.20 | ~> 0.2 |
28
+
29
+ ## Requirements
30
+
31
+ - Foreman >= 1.20
32
+ - Working Vault instance
33
+ - with _cert_ auth enabled
34
+ - with _approle_ auth enabled
35
+ - with _kv_ secret store enabled
36
+ - valid Vault Token
37
+
38
+ **Dev Vault Instance**
39
+
40
+ To run a local Vault dev environment on MacOS use:
14
41
 
15
42
  ```
16
43
  $ brew install vault
17
44
  $ vault server -dev
18
45
  $ export VAULT_ADDR='http://127.0.0.1:8200'
19
46
  $ vault secrets enable kv
47
+ $ vault auth enable cert
48
+
49
+ $ vault token create -period=60m
50
+ [...]
20
51
  ```
21
52
 
22
- To set up a connection between Foreman and Vault first navigate to the "Infrastructure" > "Vault Connections" menu and then hit the button labeled "Create Vault Connection". Now you should see a form. You have to fill in name, url and token (you can receive a token with the `$ vault token create -period=60m` command) and hit the "Submit" button.
53
+ To interact with Vault you can use Vault UI, which is available at `http://127.0.0.1:8200/ui`.
23
54
 
24
- You can now use `vault_secret(vault_connection_name, secret_path)` macro in your templates to fetch secrets from Vault (you can write secrets with the `$ vault write kv/my_secret foo=bar` command), e.g.
55
+ - The AppRole auth method
56
+
57
+ ```
58
+ $ vault auth enable approle
59
+ $ vault write auth/approle/role/my-role policies="default"
60
+ Success! Data written to: auth/approle/role/my-role
61
+ $ vault read auth/approle/role/my-role/role-id
62
+ Key Value
63
+ --- -----
64
+ role_id 8403910c-e563-d2f2-1c77-6e26319be8b5
65
+ $ vault write -f auth/approle/role/my-role/secret-id
66
+ Key Value
67
+ --- -----
68
+ secret_id 1058434b-b4aa-bf5a-b376-a15d9efb1059
69
+ secret_id_accessor 9cc19ed7-201f-7438-782e-561edd12b2a8
70
+ ```
71
+
72
+ See also [Vault CLI testing AppRole](https://gist.github.com/kamils-iRonin/d099908eaf0500de8ad9c2cea5658d01)
73
+
74
+ ## Installation
75
+
76
+ See [Plugins install instructions](https://theforeman.org/plugins/) for how to install Foreman plugins.
77
+
78
+ ## Basic configuration
79
+
80
+ To create a new Vault connection navigate to `Infrastructure -> Vault Connections` and hit the `Create Vault Connection` button. There you can enter a name, the Vault URL and a secret token.
81
+
82
+ ## Vault secrets in templates
83
+
84
+ At this point you can utilize two new macros in your templates:
85
+
86
+ - vault_secret(vault_connection_name, secret_path)
87
+ - vault_issue_certificate(vault_connection_name, pki_role_path, options...)
88
+
89
+ ### vault_secret(vault_connection_name, secret_path)
90
+
91
+ To fetch secrets from Vault (you can write secrets with the `vault write kv/my_secret foo=bar` command), e.g.
25
92
 
26
93
  ```
27
94
  <%= vault_secret('MyVault', 'kv/my_secret') %>
@@ -33,13 +100,51 @@ As result you should get secret data, e.g.
33
100
  {:foo=>"bar"}
34
101
  ```
35
102
 
103
+ ### vault_issue_certificate(vault_connection_name, pki_role_path, options...)
104
+
105
+ Issueing certificates is just as easy. Be sure to have a correctly set-up PKI, meaning, configure it so you can generate certificates from within the Vault UI. This means that you'll have to set-up a CA or Intermediate CA. Once done, you can generate a certificate like this:
106
+
107
+ ```
108
+ <%= vault_issue_certificate('MyVault', 'pkiEngine/issue/testRole', common_name: 'test.mydomain.com', ttl: '10s') %>
109
+ ```
110
+
111
+ The _common_name_ and _ttl_ are optional, but there are [more options to configure](https://www.vaultproject.io/api/secret/pki/index.html#generate-certificate)
112
+
113
+ ## Vault policies and auth methods
114
+
115
+ ### Policies
116
+
117
+ The policy is based on a new template kind `VaultPolicy` which is basically a [Vault Policy](https://www.vaultproject.io/docs/concepts/policies#policy-syntax) extended with ERB.
118
+
119
+ The name of the policy is extracted from a _Magic comment_ within the template. Therefore you can use ERB to influence the name:
120
+
121
+ ```
122
+ # NAME: <%= @host.owner %>-<%= @host.environment %>
123
+
124
+ path "secret/foo" {
125
+ capabilities = ["read"]
126
+ }
127
+ ```
128
+
129
+ You can create multiple `VaultPolicy` templates and configure the default policy used in host orchestration by setting the Foreman Setting `vault_policy_template` to the desired one.
130
+
131
+ **Note: If the policy renders empty (yes, you can use conditions within ERB), the orchestration is skipped!**
132
+
133
+ ### Auth methods
134
+
135
+ [Auth methods of type `cert`](https://www.vaultproject.io/docs/auth/cert) are created with three attributes:
136
+
137
+ - **certificate**: content of the Foreman setting `ssl_ca_file`
138
+ - **allowed_common_names**: FQDN of the host which triggered the orchestration
139
+ - **token_policies**: This is automatically linked to the policy from above
140
+
36
141
  ## Contributing
37
142
 
38
143
  Fork and send a Pull Request. Thanks!
39
144
 
40
145
  ## Copyright
41
146
 
42
- Copyright (c) 2018 dmTECH GmbH, [dmtech.de](https://www.dmtech.de/)
147
+ Copyright (c) 2018-2020 dmTECH GmbH, [dmtech.de](https://www.dmtech.de/)
43
148
 
44
149
  This program is free software: you can redistribute it and/or modify
45
150
  it under the terms of the GNU General Public License as published by
@@ -48,8 +153,8 @@ the Free Software Foundation, either version 3 of the License, or
48
153
 
49
154
  This program is distributed in the hope that it will be useful,
50
155
  but WITHOUT ANY WARRANTY; without even the implied warranty of
51
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
156
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
52
157
  GNU General Public License for more details.
53
158
 
54
159
  You should have received a copy of the GNU General Public License
55
- along with this program. If not, see <http://www.gnu.org/licenses/>.
160
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
@@ -9,7 +9,7 @@ module ForemanVault
9
9
  class_methods do
10
10
  def vault_connection_params_filter
11
11
  Foreman::ParameterFilter.new(::VaultConnection).tap do |filter|
12
- filter.permit :name, :url, :token
12
+ filter.permit :name, :url, :token, :role_id, :secret_id
13
13
  end
14
14
  end
15
15
  end
@@ -4,12 +4,21 @@ module ForemanVault
4
4
  module Macros
5
5
  def vault_secret(vault_connection_name, secret_path)
6
6
  vault = VaultConnection.find_by!(name: vault_connection_name)
7
- raise VaultError.new(N_('Invalid token for %s'), vault.name) unless vault.token_valid?
7
+ raise VaultError.new(N_('Invalid token for %s'), vault.name) if vault.with_token? && !vault.token_valid?
8
+
8
9
  vault.fetch_secret(secret_path)
9
10
  rescue ActiveRecord::RecordNotFound => e
10
11
  raise VaultError, e.message
11
12
  end
12
13
 
14
+ def vault_issue_certificate(vault_connection_name, secret_path, *options)
15
+ vault = VaultConnection.find_by!(name: vault_connection_name)
16
+ raise VaultError.new(N_('Invalid token for %s'), vault.name) if vault.with_token? && !vault.token_valid?
17
+ vault.issue_certificate(secret_path, *options)
18
+ rescue ActiveRecord::RecordNotFound => e
19
+ raise VaultError, e.message
20
+ end
21
+
13
22
  class VaultError < Foreman::Exception; end
14
23
  end
15
24
  end
@@ -0,0 +1,31 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ForemanVault
4
+ module HostExtensions
5
+ extend ActiveSupport::Concern
6
+
7
+ included do
8
+ include ForemanVault::Orchestration::VaultPolicy
9
+ end
10
+
11
+ def vault_policy
12
+ VaultPolicy.new(self)
13
+ end
14
+
15
+ def vault_auth_method
16
+ VaultAuthMethod.new(self)
17
+ end
18
+
19
+ def vault_connection
20
+ return unless vault_connection_name
21
+
22
+ ::VaultConnection.find_by(name: vault_connection_name)
23
+ end
24
+
25
+ private
26
+
27
+ def vault_connection_name
28
+ params['vault_connection'] || Setting['vault_connection']
29
+ end
30
+ end
31
+ end
@@ -0,0 +1,70 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ForemanVault
4
+ module Orchestration
5
+ module VaultPolicy
6
+ extend ActiveSupport::Concern
7
+
8
+ MAGIC_COMMENT_PREFIX = '# NAME: '
9
+
10
+ included do
11
+ after_validation :queue_vault_push
12
+ before_destroy :queue_vault_destroy
13
+ end
14
+
15
+ protected
16
+
17
+ def queue_vault_push
18
+ return if !managed? || errors.any?
19
+ return unless orchestration_enabled?
20
+ return unless vault_policy.valid?
21
+ return unless vault_auth_method.valid?
22
+
23
+ queue.create(name: _('Push %s data to Vault') % self, priority: 100,
24
+ action: [self, :set_vault])
25
+ end
26
+
27
+ def queue_vault_destroy
28
+ return if !managed? || errors.any?
29
+ return unless orchestration_enabled?
30
+ return unless vault_auth_method.valid?
31
+
32
+ queue.create(name: _('Clear %s Vault data') % self, priority: 60,
33
+ action: [self, :del_vault])
34
+ end
35
+
36
+ # rubocop:disable Metrics/AbcSize
37
+ def set_vault
38
+ logger.info "Pushing #{name} data to Vault"
39
+
40
+ vault_policy.save if vault_policy.new?
41
+
42
+ if vault_auth_method.name != old&.vault_auth_method&.name
43
+ old&.vault_auth_method&.delete
44
+ vault_auth_method.save
45
+ end
46
+ true
47
+ rescue StandardError => e
48
+ Foreman::Logging.exception("Failed to push #{name} data to Vault.", e)
49
+ failure format(_('Failed to push %{name} data to Vault: %{message}\n '), name: name, message: e.message), e
50
+ end
51
+ # rubocop:enable Metrics/AbcSize
52
+
53
+ def del_vault
54
+ logger.info "Clearing #{name} Vault data"
55
+
56
+ vault_auth_method&.delete
57
+ rescue StandardError => e
58
+ Foreman::Logging.exception("Failed to clear #{name} Vault data", e)
59
+ failure format(_("Failed to clear %{name} Vault data: %{message}\n "), name: name, message: e.message), e
60
+ end
61
+
62
+ def orchestration_enabled?
63
+ return false unless Setting[:vault_orchestration_enabled]
64
+ return false if vault_connection.nil?
65
+
66
+ true
67
+ end
68
+ end
69
+ end
70
+ end
@@ -0,0 +1,15 @@
1
+ # frozen_string_literal: true
2
+
3
+ module ForemanVault
4
+ module ProvisioningTemplateExtensions
5
+ extend ActiveSupport::Concern
6
+
7
+ # rubocop:disable Metrics/ParameterLists
8
+ def render(renderer: Foreman::Renderer, host: nil, params: {}, variables: {}, mode: Foreman::Renderer::REAL_MODE, template_input_values: {}, source_klass: nil)
9
+ source_klass = Foreman::Renderer::Source::Database if template_kind == TemplateKind.find_by(name: 'VaultPolicy')
10
+
11
+ super
12
+ end
13
+ # rubocop:enable Metrics/ParameterLists
14
+ end
15
+ end
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ class Setting
4
+ class Vault < ::Setting
5
+ BLANK_ATTRS << 'vault_connection'
6
+ BLANK_ATTRS << 'vault_policy_template'
7
+
8
+ def self.default_settings
9
+ [set_vault_connection, set_vault_policy_template, set_vault_orchestration_enabled]
10
+ end
11
+
12
+ def self.load_defaults
13
+ # Check the table exists
14
+ return unless super
15
+
16
+ transaction do
17
+ default_settings.each { |s| create! s.update(category: 'Setting::Vault') }
18
+ end
19
+
20
+ true
21
+ end
22
+
23
+ def self.humanized_category
24
+ N_('Vault')
25
+ end
26
+
27
+ class << self
28
+ private
29
+
30
+ def set_vault_connection
31
+ set(
32
+ 'vault_connection',
33
+ N_('Default Vault Connection that can be override using parameters'),
34
+ default_vault_connection,
35
+ N_('Default Vault Connection'),
36
+ nil,
37
+ collection: vault_connections_collection,
38
+ include_blank: _('Select Vault Connection')
39
+ )
40
+ end
41
+
42
+ def default_vault_connection
43
+ return nil unless VaultConnection.table_exists?
44
+ return unless VaultConnection.unscoped.count == 1
45
+
46
+ VaultConnection.unscoped.first.name
47
+ end
48
+
49
+ def vault_connections_collection
50
+ return [] unless VaultConnection.table_exists?
51
+
52
+ proc { Hash[VaultConnection.unscoped.all.map { |vc| [vc.name, vc.name] }] }
53
+ end
54
+
55
+ def set_vault_policy_template
56
+ set(
57
+ 'vault_policy_template',
58
+ N_('The name of the ProvisioningTemplate that will be used for Vault Policy'),
59
+ default_vault_policy_template,
60
+ N_('Vault Policy template name'),
61
+ nil,
62
+ collection: vault_policy_templates_collection,
63
+ include_blank: _('Select Template')
64
+ )
65
+ end
66
+
67
+ def default_vault_policy_template
68
+ ProvisioningTemplate.unscoped.of_kind(:VaultPolicy).find_by(name: 'Default Vault Policy')&.name
69
+ end
70
+
71
+ def vault_policy_templates_collection
72
+ proc { Hash[ProvisioningTemplate.unscoped.of_kind(:VaultPolicy).map { |tmpl| [tmpl.name, tmpl.name] }] }
73
+ end
74
+
75
+ def set_vault_orchestration_enabled
76
+ set(
77
+ 'vault_orchestration_enabled',
78
+ N_('Enable or disable the Vault orchestration step for managing policies and auth methods'),
79
+ false,
80
+ N_('Vault Orchestration enabled')
81
+ )
82
+ end
83
+ end
84
+ end
85
+ end
@@ -5,18 +5,42 @@ class VaultConnection < ApplicationRecord
5
5
 
6
6
  validates_lengths_from_database
7
7
  validates :name, presence: true, uniqueness: true
8
- validates :url, :token, presence: true
8
+ validates :url, presence: true
9
9
  validates :url, format: URI.regexp(['http', 'https'])
10
10
 
11
- before_create :set_expire_time
12
- before_update :update_expire_time
11
+ validates :token, presence: true, if: -> { role_id.nil? || secret_id.nil? }
12
+ validates :token, inclusion: { in: [nil], message: _('AppRole or token must be blank') }, unless: -> { role_id.nil? || secret_id.nil? }
13
+ validates :role_id, presence: true, if: -> { token.nil? }
14
+ validates :role_id, inclusion: { in: [nil], message: _('AppRole or token must be blank') }, unless: -> { token.nil? }
15
+ validates :secret_id, presence: true, if: -> { token.nil? }
16
+ validates :secret_id, inclusion: { in: [nil], message: _('AppRole or token must be blank') }, unless: -> { token.nil? }
13
17
 
14
- scope :with_valid_token, -> { where(vault_error: nil).where('expire_time > ?', Time.zone.now) }
18
+ before_validation :normalize_blank_values
19
+ before_create :set_expire_time, unless: -> { token.nil? }
20
+ before_update :update_expire_time, unless: -> { token.nil? }
15
21
 
16
- delegate :fetch_expire_time, :fetch_secret, to: :client
22
+ scope :with_approle, -> { where.not(role_id: nil).where.not(secret_id: nil) }
23
+ scope :with_token, -> { where.not(token: nil) }
24
+ scope :with_valid_token, -> { with_token.where(vault_error: nil).where('expire_time > ?', Time.zone.now) }
25
+
26
+ delegate :fetch_expire_time, :fetch_secret, :issue_certificate,
27
+ :policy, :policies, :put_policy, :delete_policy,
28
+ :set_certificate, :certificates, :delete_certificate, to: :client
29
+
30
+ def with_token?
31
+ token.present?
32
+ end
33
+
34
+ def with_approle?
35
+ role_id.present? && secret_id.present?
36
+ end
17
37
 
18
38
  def token_valid?
19
- vault_error.nil? && expire_time && expire_time > Time.zone.now
39
+ return false unless with_token?
40
+ return false unless vault_error.nil?
41
+ return true unless expire_time
42
+
43
+ expire_time > Time.zone.now
20
44
  end
21
45
 
22
46
  def renew_token!
@@ -38,6 +62,7 @@ class VaultConnection < ApplicationRecord
38
62
  self.expire_time = fetch_expire_time
39
63
  rescue StandardError => e
40
64
  errors.add(:base, e.message)
65
+ Foreman::Logging.exception('Failed to set vault expiry time', e)
41
66
  throw(:abort)
42
67
  end
43
68
 
@@ -49,7 +74,13 @@ class VaultConnection < ApplicationRecord
49
74
  self.vault_error = e.message
50
75
  end
51
76
 
77
+ def normalize_blank_values
78
+ attributes.each do |column, _value|
79
+ self[column].present? || self[column] = nil
80
+ end
81
+ end
82
+
52
83
  def client
53
- @client ||= ForemanVault::VaultClient.new(url, token)
84
+ @client ||= ForemanVault::VaultClient.new(url, token, role_id, secret_id)
54
85
  end
55
86
  end