config_skeleton 0.0.0.1.ENOTAG

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,56 @@
1
+ The `ConfigSkeleton` provides a framework for creating a common class of
2
+ service which periodically (or in response to external stimulii) rewrites
3
+ a file on disk and (optionally) causes a stimulus to be sent, in turn, to
4
+ another process.
5
+
6
+
7
+ # Installation
8
+
9
+ It's a gem:
10
+
11
+ gem install config_skeleton
12
+
13
+ There's also the wonders of [the Gemfile](http://bundler.io):
14
+
15
+ gem 'config_skeleton'
16
+
17
+ If you're the sturdy type that likes to run from git:
18
+
19
+ rake install
20
+
21
+ Or, if you've eschewed the convenience of Rubygems entirely, then you
22
+ presumably know what to do already.
23
+
24
+
25
+ # Usage
26
+
27
+ All of the documentation is provided in the [ConfigSkeleton class](https://rubydoc.info/gems/config_skeleton/ConfigSkeleton).
28
+
29
+
30
+ # Contributing
31
+
32
+ Patches can be sent as [a Github pull
33
+ request](https://github.com/discourse/config_skeleton). This project is
34
+ intended to be a safe, welcoming space for collaboration, and contributors
35
+ are expected to adhere to the [Contributor Covenant code of
36
+ conduct](CODE_OF_CONDUCT.md).
37
+
38
+
39
+ # Licence
40
+
41
+ Unless otherwise stated, everything in this repo is covered by the following
42
+ copyright notice:
43
+
44
+ Copyright (C) 2020 Civilized Discourse Construction Kit, Inc.
45
+
46
+ This program is free software: you can redistribute it and/or modify it
47
+ under the terms of the GNU General Public License version 3, as
48
+ published by the Free Software Foundation.
49
+
50
+ This program is distributed in the hope that it will be useful,
51
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
52
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
53
+ GNU General Public License for more details.
54
+
55
+ You should have received a copy of the GNU General Public License
56
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
@@ -0,0 +1,40 @@
1
+ begin
2
+ require 'git-version-bump'
3
+ rescue LoadError
4
+ nil
5
+ end
6
+
7
+ Gem::Specification.new do |s|
8
+ s.name = "config_skeleton"
9
+
10
+ s.version = GVB.version rescue "0.0.0.1.NOGVB"
11
+ s.date = GVB.date rescue Time.now.strftime("%Y-%m-%d")
12
+
13
+ s.platform = Gem::Platform::RUBY
14
+
15
+ s.summary = "Dynamically generate configs and reload servers"
16
+
17
+ s.authors = ["Matt Palmer"]
18
+ s.email = ["matt.palmer@discourse.org"]
19
+ s.homepage = "https://github.com/discourse/config_skeleton"
20
+
21
+ s.files = `git ls-files -z`.split("\0").reject { |f| f =~ /^(G|spec|Rakefile)/ }
22
+
23
+ s.required_ruby_version = ">= 2.3.0"
24
+
25
+ s.add_runtime_dependency 'diffy', '~> 3.0'
26
+ s.add_runtime_dependency 'frankenstein', '~> 1.0'
27
+ s.add_runtime_dependency 'rb-inotify', '~> 0.9'
28
+ s.add_runtime_dependency 'service_skeleton', '> 0.a'
29
+
30
+ s.add_development_dependency 'bundler'
31
+ s.add_development_dependency 'github-release'
32
+ s.add_development_dependency 'git-version-bump'
33
+ s.add_development_dependency 'rake', "~> 12.0"
34
+ s.add_development_dependency 'redcarpet'
35
+ s.add_development_dependency 'rubocop'
36
+ s.add_development_dependency 'yard'
37
+ s.add_development_dependency 'rspec'
38
+ s.add_development_dependency 'pry'
39
+ s.add_development_dependency 'pry-byebug'
40
+ end
@@ -0,0 +1,549 @@
1
+ require 'diffy'
2
+ require 'fileutils'
3
+ require 'frankenstein'
4
+ require 'logger'
5
+ require 'rb-inotify'
6
+ require 'service_skeleton'
7
+ require 'tempfile'
8
+
9
+ # Framework for creating config generation systems.
10
+ #
11
+ # There are many systems which require some sort of configuration file to
12
+ # operate, and need that configuration to by dynamic over time. The intent
13
+ # of this class is to provide a common pattern for config generators, with
14
+ # solutions for common problems like monitoring, environment variables, and
15
+ # signal handling.
16
+ #
17
+ # To use this class for your own config generator, you need to:
18
+ #
19
+ # 1. Subclass this class.
20
+ #
21
+ # 1. Declare all the environment variables you care about, with the
22
+ # ServiceSkeleton declaration methods `string`, `integer`, etc.
23
+ #
24
+ # 1. Implement service-specific config generation and reloading code, by
25
+ # overriding the private methods #config_file, #config_data, and #reload_server
26
+ # (and also potentially #config_ok? and #sleep_duration).
27
+ # See the documentation for those methods for what they need to do.
28
+ #
29
+ # 1. Setup any file watchers you want with .watch and #watch.
30
+ #
31
+ # 1. Instantiate your new class, passing in an environment hash, and then call
32
+ # #start. Something like this should do the trick:
33
+ #
34
+ # class MyConfigGenerator < ConfigSkeleton
35
+ # # Implement all the necessary methods
36
+ # end
37
+ #
38
+ # MyConfigGenerator.new(ENV).start if __FILE__ == $0
39
+ #
40
+ # 1. Sit back and relax.
41
+ #
42
+ #
43
+ # # Environment Variables
44
+ #
45
+ # In keeping with the principles of the [12 factor app](https://12factor.net),
46
+ # all configuration of the config generator should generally be done via the
47
+ # process environment. To make this easier, ConfigSkeleton leverages
48
+ # [ServiceSkeleton's configuration
49
+ # system](https://github.com/discourse/service_skeleton#configuration) to allow
50
+ # you to declare environment variables of various types, provide defaults, and
51
+ # access the configuration values via the `config` method. See the
52
+ # [ServiceSkeleton
53
+ # documentation](https://github.com/discourse/service_skeleton#configuration)
54
+ # for more details on how all this is accomplished.
55
+ #
56
+ #
57
+ # # Signal Handling
58
+ #
59
+ # Config generators automatically hook several signals when they are created:
60
+ #
61
+ # * **`SIGHUP`**: Trigger a regeneration of the config file and force a reload
62
+ # of the associated server.
63
+ #
64
+ # * **`SIGINT`/`SIGTERM`**: Immediately terminate the process.
65
+ #
66
+ # * **`SIGUSR1`/`SIGUSR2`**: Increase (`USR1`) or decrease (`USR2`) the verbosity
67
+ # of log messages output.
68
+ #
69
+ #
70
+ # # Exported Metrics
71
+ #
72
+ # No modern system is complete without Prometheus metrics. These can be scraped
73
+ # by making a HTTP request to the `/metrics` path on the port specified by the
74
+ # `<SERVICEPREFIX>_METRICS_PORT` environment variable (if no port is specified,
75
+ # the metrics server is turned off, for security). The metrics server will provide
76
+ # the config generator-specific metrics by default:
77
+ #
78
+ # * **`<prefix>_generation_requests_total: The number of times the config generator
79
+ # has tried to generate a new config. This includes any attempts that failed
80
+ # due to exception.
81
+ #
82
+ # * **`<prefix>_generation_request_duration_seconds{,_sum,_count}`**: A
83
+ # histogram of the amount of time taken for the `config_data` method to
84
+ # generate a new config.
85
+ #
86
+ # * **`<prefix>_generation_exceptions_total`**: A set of counters which record
87
+ # the number of times the `config_data` method raised an exception, labelled
88
+ # with the `class` of the exception that occurred. The backtrace and error
89
+ # message should also be present in the logs.
90
+ #
91
+ # * ** `<prefix>_generation_in_progress_count`**: A gauge that should be either
92
+ # `1` or `0`, depending on whether the `config_data` method is currently being
93
+ # called.
94
+ #
95
+ # * **`<prefix>_last_generation_timestamp`**: A floating-point number of seconds
96
+ # since the Unix epoch indicating when a config was last successfully generated.
97
+ # This timestamp is updated every time the config generator checks to see if
98
+ # the config has changed, whether or not a new config is written.
99
+ #
100
+ # * **`<prefix>_last_change_timestamp`**: A floating-point number of seconds since
101
+ # the Unix epoch indicating when the config was last changed (that is, a new
102
+ # config file written and the server reloaded).
103
+ #
104
+ # * **`<prefix>_reload_total`**: A set of counters indicating the number of
105
+ # times the server has been asked to reload, usually as the result of a changed
106
+ # config file, but potentially also due to receiving a SIGHUP. The counters are
107
+ # labelled by the `status` of the reload: `"success"` (all is well), `"failure"`
108
+ # (the attempt to reload failed, indicating a problem in the `reload_server`
109
+ # method), `"bad-config"` (the server reload succeeded, but the `config_ok?`
110
+ # check subsequently failed), or `"everything-is-awful"` (the `config_ok?`
111
+ # check failed both before *and* after the reload, indicating something is
112
+ # *very* wrong with the underlying server).
113
+ #
114
+ # * **`<prefix>_signals_total`**: A set of counters indicating how many of each
115
+ # signal have been received, labelled by `signal`.
116
+ #
117
+ # * **`<prefix>_config_ok`**: A gauge that should be either `1` or `0`, depending on
118
+ # whether the last generated config was loaded successfully by the server.
119
+ # If the #config_ok? method has not been overridden, this will always be `1`.
120
+ #
121
+ # Note that all of the above metrics have a `<prefix>` at the beginning; the
122
+ # value of this is derived from the class name, by snake-casing.
123
+ #
124
+ #
125
+ # # Watching files
126
+ #
127
+ # Sometimes your config, or the server, relies on other files on the filesystem
128
+ # managed by other systems (typically a configuration management system), and
129
+ # when those change, the config needs to change, or the server needs to be
130
+ # reloaded. To accommodate this requirement, you can declare a "file watch"
131
+ # in your config generator, and any time the file or directory being watched
132
+ # changes, a config regeneration and server reload will be forced.
133
+ #
134
+ # To declare a file watch, just call the .watch class method, or #watch instance
135
+ # method, passing one or more strings containing the full path to files or
136
+ # directories to watch.
137
+ #
138
+ class ConfigSkeleton < ServiceSkeleton
139
+ # All ConfigSkeleton-related errors will be subclasses of this.
140
+ class Error < StandardError; end
141
+
142
+ # If you get this, someone didn't read the documentation.
143
+ class NotImplementedError < Error; end
144
+
145
+ # It is useful for consumers to manually request a config regen. An instance
146
+ # of this class is made via the regen_notifier method.
147
+ class ConfigRegenNotifier
148
+ def initialize(io_write)
149
+ @io_write = io_write
150
+ end
151
+
152
+ def trigger_regen
153
+ @io_write << "."
154
+ end
155
+ end
156
+
157
+ # Declare a file watch on all instances of the config generator.
158
+ #
159
+ # When you're looking to watch a file whose path is well-known and never-changing, you
160
+ # can declare the watch in the class.
161
+ #
162
+ # @param f [String] one or more file paths to watch.
163
+ #
164
+ # @return [void]
165
+ #
166
+ # @example reload every time a logfile is written to
167
+ # class MyConfig
168
+ # watch "/var/log/syslog"
169
+ # end
170
+ #
171
+ # @see #watch for more details on how file and directory watches work.
172
+ #
173
+ def self.watch(*f)
174
+ @watches ||= []
175
+ @watches += f
176
+ end
177
+
178
+ # Retrieve the list of class-level file watches.
179
+ #
180
+ # Not interesting for most users.
181
+ #
182
+ # @return [Array<String>]
183
+ #
184
+ def self.watches
185
+ @watches || []
186
+ end
187
+
188
+ # Create a new config generator.
189
+ #
190
+ # @param env [Hash<String, String>] the environment in which this config
191
+ # generator runs. Typically you'll just pass `ENV` in here, but you can
192
+ # pass in any hash you like, for testing purposes.
193
+ #
194
+ def initialize(env)
195
+ super
196
+
197
+ hook_signal(:HUP) do
198
+ logger.info("SIGHUP") { "received SIGHUP, triggering config regeneration" }
199
+ regenerate_config(force_reload: true)
200
+ end
201
+
202
+ initialize_config_skeleton_metrics
203
+ @trigger_regen_r, @trigger_regen_w = IO.pipe
204
+ end
205
+
206
+ # Expose the write pipe which can be written to to trigger a config
207
+ # regeneration with a forced reload; a similar mechanism is used for
208
+ # shutdown but in that case writes are managed internally.
209
+ #
210
+ # Usage: config.regen_notifier.trigger_regen
211
+ #
212
+ # @return [ConfigRegenNotifier]
213
+ def regen_notifier
214
+ @regen_notifier ||= ConfigRegenNotifier.new(@trigger_regen_w)
215
+ end
216
+
217
+ # Set the config generator running.
218
+ #
219
+ # Does the needful to generate configs and reload the server. Typically
220
+ # never returns, unless you send the process a `SIGTERM`/`SIGINT`.
221
+ #
222
+ # @return [void]
223
+ #
224
+ def run
225
+ logger.info(logloc) { "Commencing config management" }
226
+
227
+ write_initial_config
228
+
229
+ watch(*self.class.watches)
230
+
231
+ logger.debug(logloc) { "notifier fd is #{notifier.to_io.inspect}" }
232
+
233
+ @terminate_r, @terminate_w = IO.pipe
234
+
235
+ loop do
236
+ if ios = IO.select(
237
+ [notifier.to_io, @terminate_r, @trigger_regen_r],
238
+ [], [],
239
+ sleep_duration.tap { |d| logger.debug(logloc) { "Sleeping for #{d} seconds" } }
240
+ )
241
+ if ios.first.include?(notifier.to_io)
242
+ logger.debug(logloc) { "inotify triggered" }
243
+ notifier.process
244
+ regenerate_config(force_reload: true)
245
+ elsif ios.first.include?(@terminate_r)
246
+ logger.debug(logloc) { "triggered by termination pipe" }
247
+ break
248
+ elsif ios.first.include?(@trigger_regen_r)
249
+ # we want to wait until everything in the backlog is read
250
+ # before proceeding so we don't run out of buffer memory
251
+ # for the pipe
252
+ while @trigger_regen_r.read_nonblock(20, nil, exception: false) != :wait_readable; end
253
+
254
+ logger.debug(logloc) { "triggered by regen pipe" }
255
+ regenerate_config(force_reload: true)
256
+ else
257
+ logger.error(logloc) { "Mysterious return from select: #{ios.inspect}" }
258
+ end
259
+ else
260
+ logger.debug(logloc) { "triggered by timeout" }
261
+ regenerate_config
262
+ end
263
+ end
264
+ end
265
+
266
+ # Trigger the run loop to stop running.
267
+ #
268
+ def shutdown
269
+ @terminate_w.write(".")
270
+ end
271
+
272
+ # Setup a file watch.
273
+ #
274
+ # If the files you want to watch could be in different places on different
275
+ # systems (for instance, if your config generator's working directory can be
276
+ # configured via environment), then you'll need to call this in your
277
+ # class' initialize method to setup the watch.
278
+ #
279
+ # Watching a file, for our purposes, simply means that whenever it is modified,
280
+ # the config is regenerated and the server process reloaded.
281
+ #
282
+ # Watches come in two flavours: *file* watches, and *directory* watches.
283
+ # A file watch is straightforward: if the contents of the file are
284
+ # modified, off we go. For a directory, if a file is created in the
285
+ # directory, or deleted from the directory, or *if any file in the
286
+ # directory is modified*, the regen/reload process is triggered. Note
287
+ # that directory watches are recursive; all files and subdirectories under
288
+ # the directory specified will be watched.
289
+ #
290
+ # @param files [Array<String>] the paths to watch for changes.
291
+ #
292
+ # @return [void]
293
+ #
294
+ # @see .watch for watching files and directories whose path never changes.
295
+ #
296
+ def watch(*files)
297
+ files.each do |f|
298
+ if File.directory?(f)
299
+ notifier.watch(f, :recursive, :create, :modify, :delete, :move) { |ev| logger.info("#{logloc} watcher") { "detected #{ev.flags.join(", ")} on #{ev.watcher.path}/#{ev.name}; regenerating config" } }
300
+ else
301
+ notifier.watch(f, :close_write) { |ev| logger.info("#{logloc} watcher") { "detected #{ev.flags.join(", ")} on #{ev.watcher.path}; regenerating config" } }
302
+ end
303
+ end
304
+ end
305
+
306
+ private
307
+
308
+ # Register metrics in the ServiceSkeleton metrics registry
309
+ #
310
+ # @return [void]
311
+ #
312
+ def initialize_config_skeleton_metrics
313
+ @config_generation = Frankenstein::Request.new("#{service_name}_generation", outgoing: false, description: "config generation", registry: metrics)
314
+
315
+ metrics.gauge(:"#{service_name}_last_generation_timestamp", "When the last config generation run was made")
316
+ metrics.gauge(:"#{service_name}_last_change_timestamp", "When the config file was last written to")
317
+ metrics.counter(:"#{service_name}_reload_total", "How many times we've asked the server to reload")
318
+ metrics.counter(:"#{service_name}_signals_total", "How many signals have been received (and handled)")
319
+ metrics.gauge(:"#{service_name}_config_ok", "Whether the last config change was accepted by the server")
320
+
321
+ metrics.last_generation_timestamp.set({}, 0)
322
+ metrics.last_change_timestamp.set({}, 0)
323
+ metrics.config_ok.set({}, 0)
324
+ end
325
+
326
+ # Write out a config file if one doesn't exist, or do an initial regen run
327
+ # to make sure everything's up-to-date.
328
+ #
329
+ # @return [void]
330
+ #
331
+ def write_initial_config
332
+ if File.exists?(config_file)
333
+ logger.info(logloc) { "Triggering a config regen on startup to ensure config is up-to-date" }
334
+ regenerate_config
335
+ else
336
+ logger.info(logloc) { "No existing config file #{config_file} found; writing one" }
337
+ File.write(config_file, instrumented_config_data)
338
+ metrics.last_change_timestamp.set({}, Time.now.to_f)
339
+ end
340
+ end
341
+
342
+ # The file in which the config should be written.
343
+ #
344
+ # @note this *must* be implemented by subclasses.
345
+ #
346
+ # @return [String] the absolute path to the config file to write.
347
+ #
348
+ def config_file
349
+ raise NotImplementedError, "config_file must be implemented in subclass."
350
+ end
351
+
352
+ # Generate a configuration data string.
353
+ #
354
+ # @note this *must* be implemented by subclasses.
355
+ #
356
+ # This should return the desired contents of the configuration file as at
357
+ # the moment it is called. It will be compared against the current contents
358
+ # of the config file to determine whether the server needs to be reloaded.
359
+ #
360
+ # @return [String] the desired contents of the configuration file.
361
+ #
362
+ def config_data
363
+ raise NotImplementedError, "config_data must be implemented in subclass."
364
+ end
365
+
366
+ # Verify that the currently running config is acceptable.
367
+ #
368
+ # In the event that a generated config is "bad", it may be possible to detect
369
+ # that the server hasn't accepted the new config, and if so, the config can
370
+ # be rolled back to a known-good state and the `<prefix>_config_ok` metric
371
+ # set to `0` to indicate a problem. Not all servers are able to be
372
+ # interrogated for correctness, so by default the config_ok? check is a no-op,
373
+ # but where possible it should be used, as it is a useful safety net and
374
+ # monitoring point.
375
+ #
376
+ def config_ok?
377
+ true
378
+ end
379
+
380
+ # Perform a reload of the server that consumes this config.
381
+ #
382
+ # The vast majority of services out there require an explicit "kick" to
383
+ # read a new configuration, whether that's being sent a SIGHUP, or a request
384
+ # to a special URL, or even a hard restart. That's what this method needs
385
+ # to do.
386
+ #
387
+ # If possible, this method should not return until the reload is complete,
388
+ # because the next steps after reloading the server assume that the server
389
+ # is available and the new config has been loaded.
390
+ #
391
+ # @raise [StandardError] this method can raise any exception, and it will
392
+ # be caught and logged by the caller, and the reload considered "failed".
393
+ #
394
+ def reload_server
395
+ raise NotImplementedError, "reload_server must be implemented in subclass."
396
+ end
397
+
398
+ # Internal method for calling the subclass' #config_data method, with exception
399
+ # handling and stats capture.
400
+ #
401
+ # @return [String]
402
+ #
403
+ def instrumented_config_data
404
+ begin
405
+ @config_generation.measure { config_data.tap { metrics.last_generation_timestamp.set({}, Time.now.to_f) } }
406
+ rescue => ex
407
+ log_exception(ex, logloc) { "Call to config_data raised exception" }
408
+ nil
409
+ end
410
+ end
411
+
412
+ # Determine how long to sleep between attempts to proactively regenerate the config.
413
+ #
414
+ # Whilst signals and file watching are great for deciding when the config
415
+ # needs to be rewritten, by far the most common reason for checking whether
416
+ # things are changed is "because it's time to". Thus, this method exists to
417
+ # allow subclasses to define when that is. The default, a hard-coded `60`,
418
+ # just means "wake up every minute". Some systems can get away with a much
419
+ # longer interval, others need a shorter one, and if you're really lucky,
420
+ # you can calculate how long to sleep based on a cache TTL or similar.
421
+ #
422
+ # @return [Integer] the number of seconds to sleep for. This *must not* be
423
+ # negative, lest you create a tear in the space-time continuum.
424
+ #
425
+ def sleep_duration
426
+ 60
427
+ end
428
+
429
+ # The instance of INotify::Notifier that is holding our file watches.
430
+ #
431
+ # @return [INotify::Notifier]
432
+ #
433
+ def notifier
434
+ @notifier ||= INotify::Notifier.new
435
+ end
436
+
437
+ # Do the hard yards of actually regenerating the config and performing the reload.
438
+ #
439
+ # @param force_reload [Boolean] normally, whether or not to tell the server
440
+ # to reload is conditional on the new config file being different from the
441
+ # old one. If you want to make it happen anyway (as occurs if a `SIGHUP` is
442
+ # received, for instance), set `force_reload: true` and we'll be really insistent.
443
+ #
444
+ # @return [void]
445
+ #
446
+ def regenerate_config(force_reload: false)
447
+ logger.debug(logloc) { "force? #{force_reload.inspect}" }
448
+ tmpfile = Tempfile.new(service_name, File.dirname(config_file))
449
+ logger.debug(logloc) { "Tempfile is #{tmpfile.path}" }
450
+ unless (new_config = instrumented_config_data).nil?
451
+ File.write(tmpfile.path, new_config)
452
+ tmpfile.close
453
+ logger.debug(logloc) { require 'digest/md5'; "Existing config hash: #{Digest::MD5.hexdigest(File.read(config_file))}, new config hash: #{Digest::MD5.hexdigest(File.read(tmpfile.path))}" }
454
+
455
+ match_perms(config_file, tmpfile.path)
456
+
457
+ diff = Diffy::Diff.new(config_file, tmpfile.path, source: 'files', context: 3, include_diff_info: true)
458
+ if diff.to_s != ""
459
+ logger.info(logloc) { "Config has changed. Diff:\n#{diff.to_s}" }
460
+ end
461
+
462
+ if force_reload
463
+ logger.debug(logloc) { "Forcing config reload because force_reload == true" }
464
+ end
465
+
466
+ if force_reload || diff.to_s != ""
467
+ cycle_config(tmpfile.path)
468
+ end
469
+ end
470
+ ensure
471
+ metrics.last_change_timestamp.set({}, File.stat(config_file).mtime.to_f)
472
+ tmpfile.close rescue nil
473
+ tmpfile.unlink rescue nil
474
+ end
475
+
476
+ # Ensure the target file's ownership and permission bits match that of the source
477
+ #
478
+ # When writing a new config file, you typically want to ensure that it has the same
479
+ # permissions as the existing one. It's just simple politeness. In the absence
480
+ # of an easy-to-find method in FileUtils to do this straightforward task, we have
481
+ # this one, instead.
482
+ #
483
+ # @param source [String] the path to the file whose permissions we wish to duplicate.
484
+ # @param target [String] the path to the file whose permissions we want to change.
485
+ # @return [void]
486
+ #
487
+ def match_perms(source, target)
488
+ stat = File.stat(source)
489
+
490
+ File.chmod(stat.mode, target)
491
+ File.chown(stat.uid, stat.gid, target)
492
+ end
493
+
494
+ # Shuffle files around and reload the server
495
+ #
496
+ # @return [void]
497
+ #
498
+ def cycle_config(new_config_file)
499
+ logger.debug(logloc) { "Cycling #{new_config_file} into operation" }
500
+
501
+ # If the daemon isn't currently working correctly, there's no downside to
502
+ # leaving a new, also-broken configuration in place, and it can help during
503
+ # bootstrapping (where the daemon can't be reloaded because it isn't
504
+ # *actually* running yet). So, let's remember if it was working before we
505
+ # started fiddling, and only rollback if we broke it.
506
+ config_was_ok = config_ok?
507
+ logger.debug(logloc) { config_was_ok ? "Current config is OK" : "Current config is a dumpster fire" }
508
+
509
+ old_copy = "#{new_config_file}.old"
510
+ FileUtils.copy(config_file, old_copy)
511
+ File.rename(new_config_file, config_file)
512
+ begin
513
+ logger.debug(logloc) { "Reloading the server..." }
514
+ reload_server
515
+ rescue => ex
516
+ log_exception(ex, logloc) { "Server reload failed" }
517
+ if config_was_ok
518
+ logger.debug(logloc) { "Restored previous config file" }
519
+ File.rename(old_copy, config_file)
520
+ end
521
+ metrics.reload_total.increment(status: "failure")
522
+
523
+ return
524
+ end
525
+
526
+ logger.debug(logloc) { "Server reloaded successfully" }
527
+
528
+ if config_ok?
529
+ metrics.config_ok.set({}, 1)
530
+ logger.debug(logloc) { "Configuration successfully updated." }
531
+ metrics.reload_total.increment(status: "success")
532
+ metrics.last_change_timestamp.set({}, Time.now.to_f)
533
+ else
534
+ metrics.config_ok.set({}, 0)
535
+ if config_was_ok
536
+ logger.warn(logloc) { "New config file failed config_ok? test; rolling back to previous known-good config" }
537
+ File.rename(old_copy, config_file)
538
+ reload_server
539
+ metrics.reload_total.increment(status: "bad-config")
540
+ else
541
+ logger.warn(logloc) { "New config file failed config_ok? test; leaving new config in place because old config is broken too" }
542
+ metrics.reload_total.increment(status: "everything-is-awful")
543
+ metrics.last_change_timestamp.set({}, Time.now.to_f)
544
+ end
545
+ end
546
+ ensure
547
+ File.unlink(old_copy) rescue nil
548
+ end
549
+ end