abide-data-processor 1.1.0 → 1.1.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0fb3fd8f80e90db869d7f0230ecead72fe3fa500c0c6046aad010b6fea953cbd
4
- data.tar.gz: d44ec5bc9e68724c0f0cae4a2acfb4d8fd39fd59118774bceb3ac47f8e36b625
3
+ metadata.gz: 443d1b0c6a1fe99a2e5e151c088dc7a0516027842859e9eb21218d4a0012d9dd
4
+ data.tar.gz: fde649c454ab6756a475706f7d4a63c07d8c9e7d0be77d2f523c684dca0c2600
5
5
  SHA512:
6
- metadata.gz: c7ef4cbcc7e1862f2a6ecf3e9fb33df5b972de5b2fcbe1ee3f2297478510fdaa7666995a3065d139bbb087c3beb167a26ffb92e8d0db9572341eb10753c391ce
7
- data.tar.gz: 411bc8d6084ebac6345cc4625160d689e3d277d3ff6c892b048bf787c264f2665c53e8b8b8761b739b9cfde8e2eb2c06ed11d5fdfc34869e295d96dfe83e85d7
6
+ metadata.gz: fc98abe49e53a067989f355cdc2d3f17111ef801f21ae96edb2dc2a86dbabe38c43eb0fb3ffd7c34bce2eb34fa2628a84b78055f5c5ae26710f99b4a6997fe6c
7
+ data.tar.gz: 14038054669a6e55bb9344ffe4dc5c801427944fb3267b93e26ed6e24264ec93c58a8407ec4fe839c524645410636a21359212b6279ee2eb90e68a5de31245ab
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- abide-data-processor (1.1.0)
4
+ abide-data-processor (1.1.1)
5
5
  deep_merge (~> 1.2)
6
6
  rgl (~> 0.5)
7
7
 
data/README.md CHANGED
@@ -2,7 +2,37 @@
2
2
 
3
3
  This gem provides the functionality to process data parsed from a Hiera file for Puppetlabs CEM modules.
4
4
 
5
- ## Installation
5
+ - [Abide-Data-Processor](#abide-data-processor)
6
+ - [About the Gem](#about-the-gem)
7
+ - [Installation](#installation)
8
+ - [Usage](#usage)
9
+ - [Development](#development)
10
+ - [CEM Resource Data Specification](#cem-resource-data-specification)
11
+ - [1: Meta Information](#1-meta-information)
12
+ - [2: Common.yaml](#2-commonyaml)
13
+ - [3: Top-level key](#3-top-level-key)
14
+ - [4: Resources](#4-resources)
15
+ - [4.1: Tile Key](#41-tile-key)
16
+ - [Example 4.1.1 - Basic plain-text title key](#example-411---basic-plain-text-title-key)
17
+ - [Example 4.1.2 - File path title key](#example-412---file-path-title-key)
18
+ - [4.2: Type](#42-type)
19
+ - [Example 4.2.1 - Basic type property](#example-421---basic-type-property)
20
+ - [Example 4.2.2 - Complex type property](#example-422---complex-type-property)
21
+ - [5: Controls](#5-controls)
22
+ - [Example 5.1 - Basic controls property](#example-51---basic-controls-property)
23
+ - [Example 5.2 - Multiple controls map to the same parameter and value](#example-52---multiple-controls-map-to-the-same-parameter-and-value)
24
+ - [Example 5.3 - Multiple controls map to the same parameter with different values](#example-53---multiple-controls-map-to-the-same-parameter-with-different-values)
25
+ - [Example 5.4 - One control maps to multiple different parameters](#example-54---one-control-maps-to-multiple-different-parameters)
26
+ - [Example 5.5 - A control does not supply parameters, but maps to the whole resource](#example-55---a-control-does-not-supply-parameters-but-maps-to-the-whole-resource)
27
+ - [6: Metaparameters](#6-metaparameters)
28
+ - [Example 6.0.1 - Declaring metaparameters](#example-601---declaring-metaparameters)
29
+ - [Example 6.0.1.1 - Declaring a metaparameter at the resource level](#example-6011---declaring-a-metaparameter-at-the-resource-level)
30
+ - [Example 6.0.1.2 - Declaring a metaparameter at the control level](#example-6012---declaring-a-metaparameter-at-the-control-level)
31
+ - [6.1: Dependent](#61-dependent)
32
+
33
+ ## About the Gem
34
+
35
+ ### Installation
6
36
 
7
37
  Add this line to your application's Gemfile:
8
38
 
@@ -18,20 +48,261 @@ Or install it yourself as:
18
48
 
19
49
  $ gem install abide-data-processor
20
50
 
21
- ## Usage
51
+ ### Usage
22
52
 
23
53
  Since this gem is more of a library, there is no executable to run it. This gem is designed to be use by Puppetlabs CEM modules to process the data that those modules generate.
24
54
 
25
- ## Development
55
+ ### Development
26
56
 
27
57
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
28
58
 
29
59
  To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
30
60
 
31
- ## Contributing
61
+ ## CEM Resource Data Specification
62
+
63
+ CEM resource data is the basis for how the CEM modules know what code to enforce, and how, for each of our supported compliance frameworks. Resource data is implemented as module-level Hiera data in each of the CEM modules. The goal of resource data is to provide detailed information about how specific Puppet resources and their parameters enforce compliance standards, which is then used in the modules to include the correct Puppet resources and values in the catalog. This is included in this repo because the data processor is the parser for CEM resource data.
64
+
65
+ ### 1: Meta Information
66
+
67
+ - File Type: [YAML version 1.2](https://yaml.org/spec/1.2.2)
68
+ - File Extension: `.yaml`
69
+ - Base Directory: `data/`
70
+ - Hiera hierarchy levels:
71
+ - `%{facts.os.family}/%{facts.os.name}/%{facts.os.release.major}.yaml`
72
+ - `%{facts.os.family}/%{facts.os.name}.yaml`
73
+ - `%{facts.os.family}/%{facts.os.release.major}.yaml`
74
+ - `%{facts.os.family}.yaml`
75
+ - `common.yaml`
76
+
77
+ ### 2: Common.yaml
78
+
79
+ `common.yaml` must declare lookup options for the key `<module name>::resources:` as follows:
80
+
81
+ ```yaml
82
+ lookup_options:
83
+ <module name>::resources:
84
+ merge:
85
+ strategy: deep
86
+ merge_hash_arrays: true
87
+ ```
88
+
89
+ ### 3: Top-level key
90
+
91
+ There must only be one top-level key in each resource data file. This key serves as a universal lookup key for the entire data structure. The top-level key in each resource data file must follow the convention: `<module name>::resources:`.
92
+
93
+ ### 4: Resources
94
+
95
+ Under the top-level key, all sub-hashes constitute a singular Puppet resource, whether that resource is a `class`, a `defined type`, etc. Each resource consists of a [title-key](#41-title-key), a [type](#42-type), various [controls](#5-controls), and optional [metaparameters](#6-metaparameters).
96
+
97
+ #### 4.1: Tile Key
98
+
99
+ Title keys exist beneath the top-level key. There may be any amount of title keys. Title keys must consist of resource titles. A resource title is the unique string assigned to a Puppet resource after it's type declaration:
100
+
101
+ ```ruby
102
+ exec { 'This is the resource title':
103
+ ...
104
+ }
105
+ ```
106
+
107
+ Title keys must be unique once the entire data structure is resolved via Hiera lookup. This is partially because Puppet resource names must be unique in Puppet manifests, but also because keys at the same level in Ruby hashes must be unique. This requirement is less flexible than resource naming in Puppet itself, because types with multiple name variables that are combined to make the actual resource name are not considered.
108
+
109
+ ##### Example 4.1.1 - Basic plain-text title key
110
+
111
+ ```yaml
112
+ <module name>::resources:
113
+ 'This is the resource title':
114
+ ...
115
+ ```
116
+
117
+ ##### Example 4.1.2 - File path title key
118
+
119
+ ```yaml
120
+ <module name>::resources:
121
+ '/opt/puppetlabs/cem/test_file.txt':
122
+ ```
123
+
124
+ In this example, we specify a file path that would be used by a Puppet resource such as `file`. However, we cannot use the file path `/opt/puppetlabs/cem/test_file.txt` again as a title key.
125
+
126
+ #### 4.2: Type
127
+
128
+ Type is a property of a resource defined by a title key. Type must be a Puppet resource type. When Puppet resources are created during catalog compilation, this is the type of resource that will be created.
129
+
130
+ ```ruby
131
+ this_is_the_resource_type { 'This is the resource title':
132
+ ...
133
+ }
134
+ ```
135
+
136
+ Type is defined as a single key-value pair under the title key.
137
+
138
+ #### Example 4.2.1 - Basic type property
139
+
140
+ ```yaml
141
+ <module name>::resources:
142
+ 'This is the resource title':
143
+ type: exec
144
+ ```
145
+
146
+ #### Example 4.2.2 - Complex type property
147
+
148
+ ```yaml
149
+ <module name>::resources:
150
+ 'This is the resource title':
151
+ type: 'cem_linux::utils::bootloader::grub2::password'
152
+ ```
153
+
154
+ ### 5: Controls
155
+
156
+ Controls is a property of a resource that maps compliance framework controls to the various Puppet resource parameters that those controls are concerned with. Each mapping in controls consists of one or more keys that use an compliance control identifier.
157
+
158
+ #### Example 5.1 - Basic controls property
159
+
160
+ We have a compliance framework that defines a control 'Ensure you run the ls command'. The Puppet code to enforce this control looks like this:
161
+
162
+ ```ruby
163
+ # Enforces 'Ensure you run the ls command
164
+ exec { 'This is the resource title':
165
+ command => 'ls',
166
+ }
167
+ ```
168
+
169
+ The above Puppet code translates to the following resource data:
170
+
171
+ ```yaml
172
+ <module name>::resources:
173
+ 'This is the resource title':
174
+ type: exec
175
+ controls:
176
+ 'Ensure you run the ls command':
177
+ command: 'ls'
178
+ ```
179
+
180
+ #### Example 5.2 - Multiple controls map to the same parameter and value
181
+
182
+ We have two compliance frameworks. Both frameworks define a control that is equal in all but name. Framework 1 defines the control 'Ensure you run the ls command' and Framework 2 defines the control 'The ls command must be ran'. The Puppet code to enforce this control looks like this:
183
+
184
+ ```ruby
185
+ # Framework 1 - Enforces 'Ensure you run the ls command
186
+ # Framework 2 - The ls command must be ran
187
+ exec { 'This is the resource title':
188
+ command => 'ls',
189
+ }
190
+ ```
191
+
192
+ The above Puppet code translates to the following resource data:
193
+
194
+ ```yaml
195
+ <module name>::resources:
196
+ 'This is the resource title':
197
+ type: exec
198
+ controls:
199
+ ? - 'Ensure you run the ls command'
200
+ - 'The ls command must be ran'
201
+ : command: 'ls'
202
+ ```
203
+
204
+ When two or more controls map to the same parameter and the same value, [YAML complex mapping keys](https://yaml.org/spec/1.2.2/#22-structures) must be used.
205
+
206
+ #### Example 5.3 - Multiple controls map to the same parameter with different values
207
+
208
+ ```yaml
209
+ <module name>::resources:
210
+ 'This is the resource title':
211
+ type: exec
212
+ controls:
213
+ 'Ensure you run the ls command':
214
+ command: 'ls'
215
+ 'The ls -l command must be ran':
216
+ command: 'ls -l'
217
+ ```
218
+
219
+ #### Example 5.4 - One control maps to multiple different parameters
220
+
221
+ ```yaml
222
+ <module name>::resources:
223
+ 'This is the resource title':
224
+ type: exec
225
+ controls:
226
+ 'Ensure you run the ls command only if test.txt exists':
227
+ command: 'ls'
228
+ onlyif: 'test -f test.txt'
229
+ ```
230
+
231
+ When the Puppet resources are created from resource data during catalog compilation, the resources `cem_linux::utils::packages::absenter { 'avahi': }` and `cem_linux::utils::packages::absenter { 'avahi-autoipd': }` will be created before the resource `cem_linux::utils::disable_service { 'avahi-daemon': }`. You do not need any other resource data to specify `cem_linux::utils::packages::absenter { 'avahi': }` and `cem_linux::utils::packages::absenter { 'avahi-autoipd': }`.
232
+
233
+ #### Example 5.5 - A control does not supply parameters, but maps to the whole resource
234
+
235
+ Sometimes, controls will not map to specific resource parameters but to the whole resource itself. This is often the case with resources that represent defined types with very specific functionality. In this case, the `controls` key must be an `Array`, not a `Hash`. Each item of that `Array` must be a control name that maps to the declared resource.
236
+
237
+ Below is an example of Puppet code where the control would not manage a parameter:
238
+
239
+ ```ruby
240
+ cem_linux::utils::disable_service { 'avahi-daemon': }
241
+ ```
242
+
243
+ With this particular defined type, the resource title can also serve as the one and only parameter. In resource data, we declares a resource with `control` as an `Array`.
244
+
245
+ ```yaml
246
+ cem_linux::resources:
247
+ 'avahi-daemon':
248
+ type: 'cem_linux::utils::disable_service'
249
+ controls:
250
+ - 'Ensure Avahi server is not installed'
251
+ ```
252
+
253
+ ### 6: Metaparameters
254
+
255
+ Both resources and controls support five metaparameters: [before](#61-before), [require](#62-require), [notify](#63-notify), [subscribe](#64-subscribe), and [dependent](#65-dependent). Metaparameters are used for ordering resources, and are used just as they are in Puppet code. Dependent is a special metaparameter with no Puppet code equivalent. Metaparameters are applied to a resource either directly, by declaring a metaparameter key at the same level as the `type` key or the `controls` key, or at the control level just as you would declare control parameters. Regardless of if a metaparameter is declared at the resource or control level, the metaparameter applies to the whole resource.
256
+
257
+ #### Example 6.0.1 - Declaring metaparameters
258
+
259
+ Below is an example of Puppet code that uses the Puppet metaparameter `before`:
260
+
261
+ ```ruby
262
+ cem_linux::utils::disable_service { 'avahi-daemon':
263
+ before => [
264
+ Cem_linux::Utils::Packages::Absenter['avahi'],
265
+ Cem_linux::Utils::Packages::Absenter['avahi-autoipd'],
266
+ ],
267
+ }
268
+ ```
269
+
270
+ This Puppet code infers, through the use of Puppet resource references, that the Puppet resources `cem_linux::utils::packages::absenter { 'avahi': }` and `cem_linux::utils::packages::absenter { 'avahi-autoipd': }` are declared somewhere in the manifest. In resource data, we declare the dependent resources in the metaparameter itself.
271
+
272
+ There is no need for resources declared in metaparameters to exist already in resource data. Additionally, if a resource declared in resource data is also declared in a metaparameter, only one copy of that resource will be created.
273
+
274
+ ##### Example 6.0.1.1 - Declaring a metaparameter at the resource level
275
+
276
+ ```yaml
277
+ cem_linux::resources:
278
+ 'avahi-daemon':
279
+ type: 'cem_linux::utils::disable_service'
280
+ before:
281
+ 'avahi-autoipd':
282
+ type: 'cem_linux::utils::packages::absenter'
283
+ 'avahi':
284
+ type: 'cem_linux::utils::packages::absenter'
285
+ controls:
286
+ - 'Ensure Avahi server is not installed'
287
+ ```
288
+
289
+ ##### Example 6.0.1.2 - Declaring a metaparameter at the control level
290
+
291
+ ```yaml
292
+ cem_linux::resources:
293
+ 'avahi-daemon':
294
+ type: 'cem_linux::utils::disable_service'
295
+ controls:
296
+ 'Ensure Avahi server is not installed':
297
+ before:
298
+ 'avahi-autoipd':
299
+ type: 'cem_linux::utils::packages::absenter'
300
+ 'avahi':
301
+ type: 'cem_linux::utils::packages::absenter'
302
+ ```
32
303
 
33
- Bug reports and pull requests are welcome on GitHub at https://github.com/puppetlabs/abide-data-processor.
304
+ When the Puppet resources are created from resource data during catalog compilation, the resources `cem_linux::utils::packages::absenter { 'avahi': }` and `cem_linux::utils::packages::absenter { 'avahi-autoipd': }` will be created before the resource `cem_linux::utils::disable_service { 'avahi-daemon': }`.
34
305
 
35
- ## Code of Conduct
306
+ #### 6.1: Dependent
36
307
 
37
- Everyone interacting in the Abide-Data-Processor project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/puppetlabs/abide-data-processor/blob/main/CODE_OF_CONDUCT.md).
308
+ Dependent is a special metaparameter used in resource data used to ensure mutual inclusion / exclusion of resources that depend on each other to function. When Resource A declares Resource B in the `dependent` metaparameter, Resource A will only be enforced if Resource B is included in the catalog as well. Dependent ensures that controls residing in the `ignore` list will not disrupt the entire catalog.
@@ -681,7 +681,11 @@ module AbideDataProcessor
681
681
  cached = cache_get(cache_key)
682
682
  return cached unless cached.nil?
683
683
 
684
- new_resource = Resource.new(resource_name, resource_data, control_maps)
684
+ begin
685
+ new_resource = Resource.new(resource_name, resource_data, control_maps)
686
+ rescue ArgumentError => e
687
+ raise ArgumentError, "Failed to create resource #{resource_name}: #{e.message}"
688
+ end
685
689
  cache_add(cache_key, new_resource)
686
690
  new_resource
687
691
  end
@@ -697,7 +701,11 @@ module AbideDataProcessor
697
701
  cached = cache_get(cache_key)
698
702
  return cached unless cached.nil?
699
703
 
700
- new_control = Control.new(control_name, control_data, control_maps)
704
+ begin
705
+ new_control = Control.new(control_name, control_data, control_maps)
706
+ rescue ArgumentError => e
707
+ raise ArgumentError, "Failed to create control #{control_name}: #{e.message}"
708
+ end
701
709
  cache_add(cache_key, new_control)
702
710
  new_control
703
711
  end
@@ -1,3 +1,3 @@
1
1
  module AbideDataProcessor
2
- VERSION = "1.1.0"
2
+ VERSION = "1.1.1"
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: abide-data-processor
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - abide-team
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2022-01-26 00:00:00.000000000 Z
11
+ date: 2022-01-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: deep_merge