chef_sous_vide 0.1.0 → 0.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 4dbf479c7c26026bf1c478fed4b9268743c018cc
4
- data.tar.gz: f903092d21a62916b09d1b03c5b0a6e2e00be2c5
3
+ metadata.gz: b7625a1692ecb152693bc6dc151c671801001d59
4
+ data.tar.gz: 38a2599b444790b518ccec7c70d668b4bcc7b859
5
5
  SHA512:
6
- metadata.gz: 6cf82abbceeb1d85371f5cda965cf3c27dbbd61bdc220253c9e5ded22149ce6ab6f658f2473cd2a15a050c127d9c525536ee13d5ba62cd7e496e488211a8b1ab
7
- data.tar.gz: c80e255d8f134869e2e72448c2b30ebdec32044568271175138933129dd5ab643eaa370b5a5af5729f06b33019b208c9afc652bf16622eb1446408af5fb16960
6
+ metadata.gz: 4a4a9001104866457fb7591800de2707122db72db4e5fa1f13c4444c8c6a13c82665823125afe5719b2b8486869d17e8111baace07afb6ea21bd9e7d578ac413
7
+ data.tar.gz: 19f0b7ad71404d464a687d4cbfabfc58f047734675b6f03038c8497d10c03d77a249594ece19a8806bd4f9f4be12e3d59f56ad799bb359ea154fde63acea3ce8
data/.gitignore CHANGED
@@ -12,3 +12,4 @@ vendor
12
12
  .rspec_status
13
13
  .kitchen/
14
14
  .kitchen.local.yml
15
+ *.gem
data/Gemfile CHANGED
@@ -1,3 +1,5 @@
1
1
  source "https://rubygems.org"
2
2
 
3
3
  gemspec
4
+
5
+ gem "yard"
@@ -1,8 +1,8 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- sous_vide (0.1.0)
5
- chef (~> 12.17.44)
4
+ chef_sous_vide (0.2.0)
5
+ chef
6
6
 
7
7
  GEM
8
8
  remote: https://rubygems.org/
@@ -278,6 +278,7 @@ GEM
278
278
  rubyzip (~> 1.1)
279
279
  winrm (~> 2.0)
280
280
  wmi-lite (1.0.2)
281
+ yard (0.9.19)
281
282
 
282
283
  PLATFORMS
283
284
  ruby
@@ -285,13 +286,14 @@ PLATFORMS
285
286
  DEPENDENCIES
286
287
  berkshelf
287
288
  bundler (~> 1.17)
289
+ chef_sous_vide!
288
290
  cucumber (~> 3.1.2)
289
291
  kitchen-docker
290
292
  kitchen-transport-rsync
291
293
  pry
292
294
  rake (~> 10.0)
293
- sous_vide!
294
295
  test-kitchen
296
+ yard
295
297
 
296
298
  BUNDLED WITH
297
299
  1.17.1
data/README.md CHANGED
@@ -2,6 +2,9 @@
2
2
 
3
3
  **=======> SousVide is not ready to use. <=======**
4
4
 
5
+
6
+ ![SousVide example dashboard](media/kibana-dashboard.png?raw=true)
7
+
5
8
  SousVide is a Chef Handler you can use to collect & visualize `chef-client` converge process. It receives event data from `chef-client` and keeps track of the converge process. It's essentially a stream parser hooked into `Chef::EventDispatch`.
6
9
 
7
10
  At the end you will have an extensive report about events occured during the run.
@@ -24,7 +27,7 @@ Here's what SousVide will tell you:
24
27
  - simple counters for each type
25
28
  - notification type & notifying resource when available
26
29
 
27
- All this and more will be available in a simple JSON friendly data structure ready to serve anywhere you like.
30
+ All this and more will be available in a flat JSON data structure.
28
31
 
29
32
  Feed it to Kibana, save to file or print at the end of chef-client run. `SousVide` comes with common outputs (see `SousVide::Outputs`) but you can write your own or even pass it a Proc.
30
33
 
@@ -33,12 +36,17 @@ Feed it to Kibana, save to file or print at the end of chef-client run. `SousVid
33
36
  Add to your recipe:
34
37
 
35
38
  ```ruby
36
- chef_gem "sous_vide" do
39
+ chef_gem "chef_sous_vide" do
37
40
  action :install
38
41
  end
39
42
 
40
- require "sous_vide"
41
- SousVide::Handler.register(node.run_context)
43
+ ruby_block "register sous handler" do
44
+ block do
45
+ require "sous_vide"
46
+ SousVide.register(node.run_context)
47
+ end
48
+ action :nothing
49
+ end.run_action(:run)
42
50
  ```
43
51
 
44
52
  You should add these lines as early as possible. SousVide will not detect compile-time executions before it's registration, but otherwise it will work just fine (or just as one would expect).
@@ -81,19 +89,26 @@ Once SousVide is registered, it's for the most part up to you to consume the out
81
89
  "chef_run_started_at": "2019-04-01 11:46:18",
82
90
  "chef_run_completed_at": "2019-04-01 11:46:24",
83
91
  "chef_run_success": true
84
- },
85
-
86
- #...
92
+ }
87
93
  ]
88
94
  ```
89
95
 
90
- You can configure SousVide in the recipe:
96
+ Example configuration for JsonHTTP:
91
97
 
92
98
  ```ruby
93
- require "sous_vide"
94
- json_http = SousVide::Outputs::JsonHTTP.new(url: "http://elasticsearch:3000")
95
- SousVide::Handler.instance.sous_output = json_http
96
- SousVide::Handler.register(node.run_context)
99
+ chef_gem "chef_sous_vide" do
100
+ action :install
101
+ end
102
+
103
+ ruby_block "register sous handler" do
104
+ block do
105
+ require "sous_vide"
106
+ json_http = SousVide::Outputs::JsonHTTP.new(url: "http://elasticsearch:3000")
107
+ SousVide.sous_output = json_http
108
+ SousVide.register(node.run_context)
109
+ end
110
+ action :nothing
111
+ end.run_action(:run)
97
112
  ```
98
113
 
99
114
  Open `cookbooks/sous_vide/recipes/install.rb` to see how to configure other outputs or use more than one.
@@ -116,6 +131,8 @@ Once `chef-client` finishes converging you can access a Kibana dashboard and see
116
131
 
117
132
  There are more example kitchen configurations you can converge and see the runs in Kibana. You can change the default recipe, converge again and see it in Kibana.
118
133
 
134
+ [![asciicast](https://asciinema.org/a/RerbmOQ5FzZisOM312zarxcYX.svg)](https://asciinema.org/a/RerbmOQ5FzZisOM312zarxcYX)
135
+
119
136
  ## Contributing
120
137
 
121
138
  Bug reports, suggestions and pull requests are welcome on GitHub.
@@ -41,6 +41,8 @@ suites:
41
41
  forward:
42
42
  - "5601:5601" # kibana
43
43
  - "9200:9200" # es
44
+ cap_add:
45
+ - IPC_LOCK
44
46
  run_list:
45
47
  - sous_vide::install
46
48
  - sous_vide::default
@@ -70,6 +72,9 @@ suites:
70
72
  java:
71
73
  jdk_version: 8
72
74
  - name: nginx
75
+ driver:
76
+ links:
77
+ - elasticsearch
73
78
  includes:
74
79
  - ubuntu-16.04
75
80
  run_list:
@@ -1,10 +1,45 @@
1
- require "chef/handler"
2
- require "chef/http"
3
1
  require "sous_vide/version"
4
-
5
- require "sous_vide/tracked_resource"
6
2
  require "sous_vide/handler"
3
+ require "chef"
7
4
 
5
+ # Interface to use SousVide in Chef.
6
+ #
7
+ # Provides a shortcut methods to configure and enable SousVide.
8
+ #
9
+ # @example Enable SousVide with JSON HTTP output and custom run name
10
+ #
11
+ # ruby_block "enable SousVide" do
12
+ # block do
13
+ # json_http_output = SousVide::Outputs::JsonHTTP.new(url: "http://localhost:3000")
14
+ # SousVide.run_name = "custom run name"
15
+ # SousVide.sous_output = json_http_output
16
+ # SousVide.register(node.run_context)
17
+ # end
18
+ # action :nothing
19
+ # end.run_action(:run)
8
20
  module SousVide
9
- class Error < StandardError; end
21
+ # (see SousVide::Handler.register)
22
+ def self.register(chef_run_context)
23
+ SousVide::Handler.register(chef_run_context)
24
+ end
25
+
26
+ # (see SousVide::Handler#sous_output)
27
+ def self.sous_output=(output)
28
+ SousVide::Handler.instance.sous_output = output
29
+ end
30
+
31
+ # (see SousVide::Handler#run_name)
32
+ def self.run_name=(text)
33
+ SousVide::Handler.instance.run_name = text
34
+ end
35
+
36
+ # (see SousVide::Handler#run_id)
37
+ def self.run_id=(text)
38
+ SousVide::Handler.instance.run_id = text
39
+ end
40
+
41
+ # (see SousVide::Handler#logger)
42
+ def self.logger=(logger)
43
+ SousVide::Handler.instance.logger = logger
44
+ end
10
45
  end
@@ -7,13 +7,15 @@ module SousVide
7
7
  # 2. resource_* events (possibly multiple)
8
8
  # 3. resource_action_complete
9
9
  #
10
- # The code is intentionally procedural and explicit. If :resource_action_start assigned
11
- # @processing_now only then other events will work. :resource_action_completed unassigns
12
- # @processing_now so all events will be ignored until :resource_action_start is called
13
- # again.
10
+ # If :resource_action_start assigned @processing_now only then other methods will perform any
11
+ # processing.
12
+ #
13
+ # :resource_action_completed unassigns @processing_now so all events will be ignored until
14
+ # :resource_action_start is called again.
15
+ #
16
+ # @see https://www.rubydoc.info/gems/chef/Chef/EventDispatch/Base
14
17
  module EventMethods
15
- # This hook will always fire whenever chef is about to converge a resource, including why_run
16
- # mode, notifications or skipped resources.
18
+ # Called before action is executed on a resource.
17
19
  def resource_action_start(new_resource, action, notification_type, notifying_resource)
18
20
  if nested?(new_resource) # ignore nested resources
19
21
  new_r_name = "#{new_resource.resource_name}[#{new_resource.name}]##{action}"
@@ -53,6 +55,7 @@ module SousVide
53
55
  true
54
56
  end
55
57
 
58
+ # Called after a resource has been completely converged, but only if modifications were made.
56
59
  def resource_updated(new_resource, _action)
57
60
  return false if @processing_now.nil? || nested?(new_resource) # ignore nested resources
58
61
 
@@ -61,8 +64,7 @@ module SousVide
61
64
  true
62
65
  end
63
66
 
64
- # Resource is skipped when a guard instruction stops the converge process or when
65
- # `action :nothing` is used (it's a guard too).
67
+ # Called when a resource action has been skipped b/c of a conditional.
66
68
  def resource_skipped(new_resource, _action, conditional)
67
69
  return false if @processing_now.nil? || nested?(new_resource) # ignore nested resources
68
70
 
@@ -72,6 +74,7 @@ module SousVide
72
74
  true
73
75
  end
74
76
 
77
+ # Called when a resource has no converge actions, e.g., it was already correct.
75
78
  def resource_up_to_date(new_resource, _action)
76
79
  return false if @processing_now.nil? || nested?(new_resource) # ignore nested resources
77
80
 
@@ -80,6 +83,7 @@ module SousVide
80
83
  true
81
84
  end
82
85
 
86
+ # Called when a resource action has been completed.
83
87
  def resource_completed(new_resource)
84
88
  return false if @processing_now.nil? || nested?(new_resource) # ignore nested resources
85
89
 
@@ -110,7 +114,7 @@ module SousVide
110
114
  # why-run mode and notifications are not in natural order and must not move the cursor.
111
115
  #
112
116
  # Having it pointing ahead is relevant because current resource has just been converged
113
- # (technically 'failed') and it is not 'unprocessed'.
117
+ # (maybe 'failed') and it is not 'unprocessed'.
114
118
  if !@processing_now.notifying_resource && # not notified
115
119
  !::Chef::Config[:why_run] && # not why-run
116
120
  @run_phase == "converge" # only converge phase
@@ -124,6 +128,7 @@ module SousVide
124
128
  true
125
129
  end
126
130
 
131
+ # Called when a resource fails and will not be retried.
127
132
  def resource_failed(new_resource, _action, exception)
128
133
  return false if @processing_now.nil? || nested?(new_resource) # ignore nested resources
129
134
 
@@ -134,6 +139,8 @@ module SousVide
134
139
  true
135
140
  end
136
141
 
142
+ # Called when a resource fails, but will retry.
143
+ #
137
144
  # Resources with retries can succeed on subsequent attempts or ignore_failure option may be
138
145
  # set and it's the only place we can capture intermittent errors.
139
146
  #
@@ -148,6 +155,7 @@ module SousVide
148
155
  true
149
156
  end
150
157
 
158
+ # Called before convergence starts
151
159
  def converge_start
152
160
  debug("Received :converge_start")
153
161
  debug("Changed run phase to 'converge'.")
@@ -155,6 +163,7 @@ module SousVide
155
163
  @run_name ||= [@run_started_at, @chef_node_role, @chef_node_ipv4, @run_id].join(" ")
156
164
  end
157
165
 
166
+ # Called when the converge phase is finished (success)
158
167
  def converge_complete
159
168
  debug("Received :converge_completed")
160
169
  @run_success = true
@@ -162,6 +171,8 @@ module SousVide
162
171
  send_to_output!
163
172
  end
164
173
 
174
+
175
+ # Called if the converge phase fails
165
176
  def converge_failed(_exception)
166
177
  debug("Received :converge_failed")
167
178
  @run_success = false
@@ -1,55 +1,152 @@
1
+ require "sous_vide/tracked_resource"
1
2
  require "sous_vide/event_methods"
2
- require "sous_vide/outputs/json_file"
3
- require "sous_vide/outputs/json_http"
4
- require "sous_vide/outputs/logger"
5
- require "sous_vide/outputs/multi"
3
+ require "sous_vide/outputs"
6
4
 
7
5
  require "securerandom"
8
6
  require "singleton"
9
7
 
10
8
  module SousVide
11
- # == SousVide::Handler
9
+ # SousVide exposes minimal API as it's driven by events passed from chef-client.
12
10
  #
13
- # The Handler receives event data from chef-client and keeps track of the converge process. It's
14
- # essentially a stream parser hooked into Chef::EventDispatch.
11
+ # It's sufficient to enable it and with default configuration it will print summary to
12
+ # chef-client logs.
15
13
  #
16
- # Event methods are all in SousVide::EventMethods module. This file contains logic that does not
17
- # deal with events directly.
14
+ # Collected data can be sent to different outputs, ie. HTTP endpoint.
15
+ #
16
+ # :run_name and :run_id can be customized, it will be passed to the configured output along
17
+ # all data.
18
+ #
19
+ # @note You should interact with SousVide via top level methods, i.e. SousVide.run_id = '1123'.
20
+ #
21
+ # @example Enable SousVide with JSON HTTP output and custom run name
22
+ #
23
+ # ruby_block "enable SousVide" do
24
+ # block do
25
+ # json_http_output = SousVide::Outputs::JsonHTTP.new(url: "http://localhost:3000")
26
+ # SousVide::Handler.instance.run_name = "custom run name"
27
+ # SousVide::Handler.instance.sous_output = json_http_output
28
+ # SousVide::Handler.register(node.run_context)
29
+ # end
30
+ # action :nothing
31
+ # end.run_action(:run)
32
+ #
33
+ # @see SousVide::Handler.register
34
+ # @see SousVide::Handler#sous_output
35
+ # @see https://www.rubydoc.info/gems/chef/Chef/EventDispatch/Base
18
36
  class Handler
19
37
  include Singleton
20
38
  include EventMethods
21
39
 
22
- attr_accessor :chef_run_context,
23
- :logger,
24
- :sous_output,
25
- :processing_now,
26
- :processed,
27
- :run_phase,
28
- :run_id,
29
- :run_name
40
+ # Chef-client run phase. One of:
41
+ #
42
+ # * compile
43
+ # * converge
44
+ # * delayed
45
+ # * post-converge
46
+ #
47
+ # Compile & converge phases reflect standard Chef two-pass model. Delayed & post-converge
48
+ # are custom phases added by SousVide.
49
+ #
50
+ # SousVide enters delayed phase when chef-client begins processing delayed notifications, that
51
+ # is after all resources in the run list have bggen converged.
52
+ #
53
+ # Post-converge phase happens when chef-client run fails and aborts the converge process. It
54
+ # will include resources that should, but were not converged.
55
+ #
56
+ # @api private
57
+ #
58
+ # @return [String]
59
+ attr_reader :run_phase
30
60
 
31
- # Enables the handler. Call it anywhere in the recipe, ideally as early as possible.
61
+ # Chef::RunContext object from chef-client.
62
+ # @see https://www.rubydoc.info/gems/chef/Chef/RunContext
63
+ # @api private
64
+ attr_writer :chef_run_context
65
+
66
+ # Logger object SousVide will use (default Chef::Log)
32
67
  #
33
- # SousVide::Handler.register(node.run_context)
68
+ # Event methods emit debug level messages and it can be helpful to configure a dedicated logger
69
+ # for SousVide to avoid excessive messages from chef-client.
70
+ #
71
+ # @example Configure custom debug logger
72
+ #
73
+ # logger = Logger.new(STDOUT)
74
+ # logger.level = Logger::DEBUG
75
+ # @sous_vide.logger = logger
76
+ #
77
+ # @see https://www.rubydoc.info/gems/chef/Chef/Log
78
+ attr_accessor :logger
79
+
80
+ # An object SousVide will pass all collected data to at the end of the chef-client run.
81
+ #
82
+ # The output must respond to #call (like Proc).
83
+ #
84
+ # Defaults to SousVide::Outputs::Logger.
85
+ #
86
+ # @example Configure JsonHTTP output
87
+ #
88
+ # json_http_output = SousVide::Outputs::JsonHTTP.new(url: "http://localhost:3000")
89
+ # @sous_vide.sous_output = json_http_output
90
+ #
91
+ # @example Configure multiple outputs
92
+ #
93
+ # json_file_output = SousVide::Outputs::JsonFile.new
94
+ # logger_output = SousVide::Outputs::Logger.new
95
+ # multi_output = SousVide::Outputs::Multi.new(json_file_output, logger_output)
96
+ # @sous_vide.sous_output = multi_output
97
+ #
98
+ #
99
+ # @see SousVide::Outputs::JsonFile
100
+ # @see SousVide::Outputs::JsonHTTP
101
+ # @see SousVide::Outputs::Logger
102
+ # @see SousVide::Outputs::Multi
103
+ attr_accessor :sous_output
104
+
105
+
106
+ # @return [SousVide::TrackedResource] a resource SousVide is currently processing.
107
+ # @api private
108
+ attr_reader :processing_now
109
+
110
+ # @return [Array<SousVide::TrackedResource>] a list of processed resources.
111
+ # @api private
112
+ attr_reader :processed
113
+
114
+ # SousVide custom run name. It will be included in the output, not used otherwise.
115
+ #
116
+ # @return [String]
117
+ attr_accessor :run_name
118
+
119
+ # SousVide custom run ID. It will be included in the output, not used otherwise.
120
+ #
121
+ # It is not chef client run id.
122
+ #
123
+ # @return [String]
124
+ attr_accessor :run_id
125
+
126
+ # Enables SousVide. It can be called anywhere in the recipe, ideally as early as possible.
34
127
  #
35
128
  # All converge-time resources will be included in the report regardless at what point
36
129
  # registration happens.
37
130
  #
38
131
  # Compile-time resources defined before registration will not be included.
39
- # TODO: see client.rb start_handlers as an option.
40
132
  #
41
- # `chef_handler` resource does not support subscribing to :events so we have give up DSL and
42
- # use Chef API.
133
+ # @note chef_handler resource does not support subscribing to :events. Chef.event_handler DSL
134
+ # could be used but dealing with returns and exceptions in blocks is tricky. Using Chef API
135
+ # and Ruby feels the most reliable.
43
136
  #
44
- # The `Chef.event_handler` DSL could be used but dealing with returns and exceptions in procs
45
- # is a pain.
46
- def self.register(run_context)
137
+ # @todo see client.rb start_handlers as an option.
138
+ #
139
+ # @example
140
+ # SousVide::Handler.register(node.run_context)
141
+ #
142
+ # @return (void)
143
+ def self.register(chef_run_context)
47
144
  ::Chef::Log.info "Registering SousVide"
48
145
 
49
- instance.chef_run_context = run_context
50
- instance.post_initialize
146
+ instance.chef_run_context = chef_run_context
147
+ instance.populate_node_data
51
148
 
52
- run_context.events.register(instance)
149
+ chef_run_context.events.register(instance)
53
150
  end
54
151
 
55
152
  def initialize
@@ -61,23 +158,17 @@ module SousVide
61
158
 
62
159
  @run_started_at = Time.now.strftime("%F %T")
63
160
 
64
- # Not related to RunStatus#run_id, it's our internal run id.
161
+ # Not related to RunStatus#run_id, it's SousVide internal run ID.
65
162
  @run_id = SecureRandom.uuid.split("-").first # => 596e9d00
66
163
  @run_phase = "compile"
67
164
 
68
- # Default to Chef logger, but can be changed to anything that responds to #call
69
165
  @logger = ::Chef::Log
70
166
  @sous_output = Outputs::Logger.new(logger: @logger)
71
167
  end
72
168
 
73
- # This is called in #register, as soon as @chef_run_context is available.
74
- def post_initialize
75
- @chef_node_ipv4 = @chef_run_context.node["ipaddress"] || "<no ip>"
76
- @chef_node_role = @chef_run_context.node["roles"].first || "<no role>"
77
- @chef_node_instance_id = @chef_run_context.node.name
78
- end
79
-
80
-
169
+ # Chef-client run related attributes.
170
+ #
171
+ # @return (Hash)
81
172
  def run_data
82
173
  {
83
174
  chef_run_id: @run_id,
@@ -88,6 +179,9 @@ module SousVide
88
179
  }
89
180
  end
90
181
 
182
+ # Node related attributes
183
+ #
184
+ # @return (Hash)
91
185
  def node_data
92
186
  {
93
187
  chef_node_ipv4: @chef_node_ipv4,
@@ -96,6 +190,31 @@ module SousVide
96
190
  }
97
191
  end
98
192
 
193
+ # Populates node attributes from Chef run context. These attributes will be passed to the
194
+ # configured output as :node_data.
195
+ #
196
+ # @api private
197
+ def populate_node_data
198
+ @chef_node_ipv4 = @chef_run_context.node["ipaddress"] || "<no ip>"
199
+ @chef_node_role = @chef_run_context.node["roles"].first || "<no role>"
200
+ @chef_node_instance_id = @chef_run_context.node.name
201
+ end
202
+
203
+ private
204
+
205
+ # Sends all collected data to configured output.
206
+ #
207
+ # It is called at the end of the converge process, both failure and success.
208
+ def send_to_output!
209
+ @sous_output.call(run_data: run_data, node_data: node_data, resources_data: @processed)
210
+ end
211
+
212
+ # Creates SousVide::TrackedResource from Chef resource and action.
213
+ #
214
+ # @param chef_resource [Chef::Resource]
215
+ # @param action [Symbol] an action chef-client executes on the resource
216
+ #
217
+ # @return (SousVide::TrackedResource)
99
218
  def create(chef_resource:, action:)
100
219
  tracked = TrackedResource.new(action: action,
101
220
  name: chef_resource.name,
@@ -108,30 +227,35 @@ module SousVide
108
227
  tracked
109
228
  end
110
229
 
111
- # We will ignore nested resources. Once a top level resource triggered :resource_action
112
- # started any events not related to it will be ignored.
230
+
231
+ # Tells if an event is related to the current SousVide resource. If not it's nested.
232
+ #
233
+ # Once a top level resource triggered :resource_action_start any events not related to it will
234
+ # be ignored by SousVide.
235
+ #
236
+ # @return (Boolean)
113
237
  def nested?(chef_resource)
114
238
  @processing_now &&
115
239
  @processing_now.chef_resource_handle != chef_resource
116
240
  end
117
241
 
118
- # When chef-client fails we haven't seen all resources and need to backfill the handler.
242
+ # Backfills unprocessed resources. This is called only if chef-client failed (:converge_failed
243
+ # event). SousVide is now in "post-converge" run phase.
244
+ #
245
+ # Unprocessed resources as a result of notifications are not included.
119
246
  def consume_unprocessed_resources!
120
247
  all_known_resources = expand_chef_resources!
121
248
 
122
249
  # No unprocessed resources left. Failure likely occured on last resource or in a delayed
123
250
  # notification.
124
- # TODO: check delayed notification failure
251
+ # TODO: check delayed notification failure. We may also want to backfeed unprocessed delayed
252
+ # notifications too? Maybe.
125
253
  return if @resource_collection_cursor >= all_known_resources.size
126
254
 
127
255
  unprocessed = all_known_resources[@resource_collection_cursor..-1]
128
256
 
129
- # We will pass unprocessed resources via :resource_action_start and :resource_completed so
130
- # they will end up in @processed array, but with status set to 'unprocessed' and execution
131
- # phase 'post-converge'.
132
- #
133
- # TODO: consider placing these resources before delayed notification, currently they are
134
- # always at the very end. It _maybe_ makes sense.
257
+ # Pass unprocessed resources via :resource_action_start and :resource_completed so
258
+ # they will end up in @processed collection, but with status set to "unprocessed".
135
259
  unprocessed.each do |tracked|
136
260
  resource_action_start(
137
261
  tracked.chef_resource_handle, # new_resource
@@ -146,34 +270,30 @@ module SousVide
146
270
  end
147
271
  end
148
272
 
149
- # Resources with multiple actions must be expanded, ie. given resource:
273
+ # Expands chef-client run list. Resources with multiple actions must be expanded,
274
+ # ie. given resource:
150
275
  #
151
- # service 'nginx' do
276
+ # service "nginx" do
152
277
  # action [:enable, :start]
153
278
  # end
154
279
  #
155
- # After expansion we should have 2 resources:
280
+ # After expansion we will have 2 resources:
156
281
  #
157
282
  # * service[nginx] with action :enable
158
283
  # * service[nginx] with action :start
159
284
  #
160
- # We keep track of the progress and on failure we will pick up unprocessed resources from here
161
- # to feed the handler in post-converge stage.
285
+ # On failure SousVide will pick up unprocessed resources from here to feed the handler in
286
+ # post-converge stage.
162
287
  #
163
- # On a successful chef-client run @processed will contain all resources and this method won't
164
- # be called.
288
+ # On a successful chef-client run this method won't be called.
165
289
  def expand_chef_resources!
166
- chef_run_context.resource_collection.flat_map do |chef_resource|
290
+ @chef_run_context.resource_collection.flat_map do |chef_resource|
167
291
  Array(chef_resource.action).map do |action|
168
292
  create(chef_resource: chef_resource, action: action)
169
293
  end
170
294
  end
171
295
  end
172
296
 
173
- def send_to_output!
174
- @sous_output.call(run_data: run_data, node_data: node_data, resources_data: @processed)
175
- end
176
-
177
297
  def debug(*args)
178
298
  message = args.compact.join(" ")
179
299
  logger.debug(message)
@@ -0,0 +1,13 @@
1
+ require "sous_vide/outputs/json_file"
2
+ require "sous_vide/outputs/json_http"
3
+ require "sous_vide/outputs/logger"
4
+ require "sous_vide/outputs/multi"
5
+
6
+ module SousVide
7
+ # @see SousVide::Outputs::JsonFile
8
+ # @see SousVide::Outputs::JsonHTTP
9
+ # @see SousVide::Outputs::Logger
10
+ # @see SousVide::Outputs::Multi
11
+ module Outputs
12
+ end
13
+ end
@@ -1,19 +1,19 @@
1
- require "chef/config"
2
- require "chef/json_compat"
3
- require "chef/log"
4
-
5
1
  module SousVide
6
2
  module Outputs
7
- # Saves the report to a JSON file on a node. The file will be saved to chef cache directory.
3
+ # Saves the report to a JSON file. The file will be saved to chef cache directory.
4
+ #
5
+ # Default file name is "sous-vide-report.json".
8
6
  #
9
- # Outputs::JsonFile.new
7
+ # @example
10
8
  #
11
- # By the report will be saved to "<chef-cache-path>/sous-vide-report.json".
9
+ # SousVide::Outputs::JsonFile.new
12
10
  class JsonFile
13
- def initialize(logger: logger)
11
+ def initialize(logger: nil, file_name: "sous-vide-report.json")
14
12
  @logger = logger
13
+ @file_name = file_name
15
14
  end
16
15
 
16
+ # Saves report to file.
17
17
  def call(run_data:, node_data:, resources_data:)
18
18
  log "=============== #{self.class.name} ==============="
19
19
  log ""
@@ -23,13 +23,14 @@ module SousVide
23
23
  tracked.to_h.merge(node_data).merge(run_data)
24
24
  end
25
25
 
26
- ::Chef::FileCache.store("sous-vide-report.json",
27
- ::Chef::JSONCompat.to_json_pretty(json_data))
26
+ ::Chef::FileCache.store(@file_name, ::Chef::JSONCompat.to_json_pretty(json_data))
28
27
 
29
- log "The report is in #{Chef::Config[:file_cache_path]}/sous-vide-report.json file."
28
+ log "The report is in #{Chef::Config[:file_cache_path]}/#{@file_name} file."
30
29
  log ""
31
30
  end
32
31
 
32
+ private
33
+
33
34
  def log(*args)
34
35
  message = args.compact.join(" ")
35
36
  logger.info(message)
@@ -6,16 +6,27 @@ module SousVide
6
6
  module Outputs
7
7
  # Makes a POST request to a configured endpoint. Logstash & Elasticsearch friendly format.
8
8
  #
9
+ # It uses Net::HTTP to perform requests, it can be customized via :http_client accessor.
10
+ #
11
+ # @example
12
+ #
9
13
  # JsonHTTP.new(url: "http://localhost:9200/endpoint", max_retries: 10)
10
14
  class JsonHTTP
11
- def initialize(url:, max_retries: 0, logger: nil)
15
+ # Provides access to Net::HTTP client object. Use it to enable SSL or pass your own client.
16
+ # @return [Net::HTTP]
17
+ attr_accessor :http_client
18
+
19
+ # @param max_retries [Fixnum] number retries across all requests made.
20
+ def initialize(url:, max_retries: 2, logger: nil)
12
21
  @endpoint = URI(url)
13
22
  @logger = logger
14
-
15
- @max_retries = max_retries || 2
23
+ @retry = 0
24
+ @max_retries = max_retries
16
25
  @http_client = Net::HTTP.new(@endpoint.host, @endpoint.port)
17
26
  end
18
27
 
28
+ # Sends a POST request with a JSON payload using @http_client object per resource.
29
+ # @return (void)
19
30
  def call(run_data:, node_data:, resources_data:)
20
31
  log "=============== #{self.class.name} ==============="
21
32
  log ""
@@ -38,24 +49,26 @@ module SousVide
38
49
  log ""
39
50
  end
40
51
 
52
+ private
53
+
41
54
  def call_with_retries(nethttp_request)
42
- _retry = 0
43
55
  begin
44
56
  @http_client.request(nethttp_request)
45
57
  rescue
46
- if _retry < @max_retries
47
- _retry += 1
58
+ if @retry < @max_retries
59
+ logger.warn("Request failed, retry #{@retry} of #{@max_retries}.")
60
+ @retry += 1
48
61
  sleep 2
49
62
  retry
50
63
  else
64
+ logger.error("Request failed, retry #{@retry} of #{@max_retries}. Abort.")
51
65
  raise
52
66
  end
53
67
  end
54
68
  end
55
69
 
56
70
  def log(*args)
57
- message = args.compact.join(" ")
58
- logger.info(message)
71
+ logger.info(args.compact.join(" "))
59
72
  end
60
73
 
61
74
  def logger
@@ -1,13 +1,15 @@
1
1
  module SousVide
2
2
  module Outputs
3
- # Prints the report to the logger (usually Chef logger).
3
+ # Prints the report to logger.
4
4
  #
5
- # Outputs::Logger.new
5
+ # @example
6
+ # SousVide::Outputs::Logger.new
6
7
  class Logger
7
8
  def initialize(logger: nil)
8
9
  @logger = logger
9
10
  end
10
11
 
12
+ # Prints the report to logger.
11
13
  def call(run_data:, node_data:, resources_data:)
12
14
  log "=============== #{self.class.name} ==============="
13
15
  log ""
@@ -35,6 +37,8 @@ module SousVide
35
37
  log ""
36
38
  end
37
39
 
40
+ private
41
+
38
42
  def log(*args)
39
43
  message = args.compact.join(' ')
40
44
  logger.info(message)
@@ -2,14 +2,18 @@ module SousVide
2
2
  module Outputs
3
3
  # Combines multiple outputs
4
4
  #
5
- # es = Outputs::ES.new ...
6
- # log = Outputs::Logger.new ...
7
- # multi = Outputs::Multi.new(es, log)
5
+ # @example
6
+ # http = JsonHTTP.new(url: "http://localhost:9200/endpoint")
7
+ # logger = SousVide::Outputs::Logger.new
8
+ # file = SousVide::Outputs::JsonFile.new
9
+ #
10
+ # SousVide::Outputs::Multi.new(logger, file, http)
8
11
  class Multi
9
12
  def initialize(*outputs)
10
13
  @outputs = outputs
11
14
  end
12
15
 
16
+ # Calls all configured outputs in order.
13
17
  def call(run_data:, node_data:, resources_data:)
14
18
  @outputs.each do |output|
15
19
  output.call(run_data: run_data, node_data: node_data,
@@ -1,31 +1,67 @@
1
1
  module SousVide
2
- # == SousVide::TrackedResource
2
+ # It is a very simple data structure SousVide uses to capture information about resource
3
+ # execution.
3
4
  #
4
- # This is a very simple data structure SousVide uses to capture interesting
5
- # information.
5
+ # @see https://www.rubydoc.info/gems/chef/Chef/Resource
6
6
  class TrackedResource
7
- attr_accessor :type,
8
- :name,
9
- :action,
10
- :status,
11
- :duration_ms,
12
- :guard_description,
13
- :execution_phase,
14
- :execution_order,
15
- :notifying_resource,
16
- :notification_type,
17
- :before_notifications,
18
- :immediate_notifications,
19
- :delayed_notifications,
20
- :retries,
21
- :error_output,
22
- :error_source,
23
- :cookbook_name,
24
- :cookbook_recipe,
25
- :source_line,
26
- :started_at,
27
- :completed_at
28
7
 
8
+ attr_accessor :type
9
+
10
+ attr_accessor :name
11
+
12
+ # Action executed on this resource.
13
+ attr_accessor :action
14
+
15
+ # Result of the action, ie. "updated", "skipped", ...
16
+ attr_accessor :status
17
+
18
+ attr_accessor :duration_ms
19
+
20
+ attr_accessor :guard_description
21
+
22
+ # SousVide run-phase when this resource was executed.
23
+ attr_accessor :execution_phase
24
+
25
+ # Resource execution order. Takes into account notifications and does not reflect resource.
26
+ # order on the run list.
27
+ attr_accessor :execution_order
28
+
29
+ # Resource that triggered this execution. Delayed notifications do not have this.
30
+ attr_accessor :notifying_resource
31
+
32
+ attr_accessor :notification_type
33
+
34
+ # Number of before notifications attached to this resource.
35
+ attr_accessor :before_notifications
36
+
37
+ # Number of immediate notifications attached to this resource.
38
+ attr_accessor :immediate_notifications
39
+
40
+ # Number of delayed notifications attached to this resource.
41
+ attr_accessor :delayed_notifications
42
+
43
+ # Number of retries.
44
+ attr_accessor :retries
45
+
46
+ # Last error output if available. This can be populated when resource failed and when resource
47
+ # succeed after a retry.
48
+ attr_accessor :error_output
49
+
50
+ # Resource definition that triggered the error.
51
+ attr_accessor :error_source
52
+
53
+ attr_accessor :cookbook_name
54
+
55
+ attr_accessor :cookbook_recipe
56
+
57
+ attr_accessor :source_line
58
+
59
+ attr_accessor :started_at
60
+
61
+ attr_accessor :completed_at
62
+
63
+ # Chef API resource. It is used for comparsion only.
64
+ # @api private
29
65
  attr_accessor :chef_resource_handle
30
66
 
31
67
  def initialize(name:, action:, type:)
@@ -42,10 +78,14 @@ module SousVide
42
78
  @error_source = nil
43
79
  end
44
80
 
81
+ # String and human friendly represtnation of the resource
82
+ # @return [String]
45
83
  def to_s
46
84
  "#{@type}[#{@name}]##{@action}"
47
85
  end
48
86
 
87
+ # Serializes the resource to hash
88
+ # @return [Hash]
49
89
  def to_h
50
90
  {
51
91
  chef_resource: "#{@type}[#{@name}]##{@action}",
@@ -1,3 +1,3 @@
1
1
  module SousVide
2
- VERSION = "0.1.0".freeze
2
+ VERSION = "0.2.0".freeze
3
3
  end
@@ -15,14 +15,14 @@ Gem::Specification.new do |spec|
15
15
 
16
16
  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
17
17
  spec.files = Dir.chdir(File.expand_path(__dir__)) do
18
- `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(cookbooks|kitchen|features)/}) }
18
+ `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(cookbooks|kitchen|features|media)/}) }
19
19
  end
20
20
 
21
21
  spec.bindir = "exe"
22
22
  spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
23
23
  spec.require_paths = ["lib"]
24
24
 
25
- spec.add_dependency "chef", "~> 12.17.44"
25
+ spec.add_dependency "chef"
26
26
 
27
27
  spec.add_development_dependency "bundler", "~> 1.17"
28
28
  spec.add_development_dependency "rake", "~> 10.0"
metadata CHANGED
@@ -1,29 +1,29 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: chef_sous_vide
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - robuye
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2019-04-01 00:00:00.000000000 Z
11
+ date: 2019-04-25 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: chef
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
- - - "~>"
17
+ - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 12.17.44
19
+ version: '0'
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
- - - "~>"
24
+ - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 12.17.44
26
+ version: '0'
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: bundler
29
29
  requirement: !ruby/object:Gem::Requirement
@@ -159,6 +159,7 @@ files:
159
159
  - lib/sous_vide.rb
160
160
  - lib/sous_vide/event_methods.rb
161
161
  - lib/sous_vide/handler.rb
162
+ - lib/sous_vide/outputs.rb
162
163
  - lib/sous_vide/outputs/json_file.rb
163
164
  - lib/sous_vide/outputs/json_http.rb
164
165
  - lib/sous_vide/outputs/logger.rb