service_skeleton 0.0.0.1.ENOTAG → 0.0.0.2.g46c1e0e

Sign up to get free protection for your applications and to get access to all the features.
Files changed (69) hide show
  1. checksums.yaml +5 -5
  2. data/.gitignore +0 -2
  3. data/.rubocop.yml +114 -9
  4. data/.travis.yml +11 -0
  5. data/README.md +153 -279
  6. data/lib/service_skeleton/background_worker.rb +80 -0
  7. data/lib/service_skeleton/config.rb +18 -78
  8. data/lib/service_skeleton/config_variable.rb +8 -29
  9. data/lib/service_skeleton/config_variables.rb +68 -54
  10. data/lib/service_skeleton/error.rb +3 -5
  11. data/lib/service_skeleton/filtering_logger.rb +0 -2
  12. data/lib/service_skeleton/logging_helpers.rb +3 -10
  13. data/lib/service_skeleton/metrics_methods.rb +13 -28
  14. data/lib/service_skeleton/signal_handler.rb +183 -0
  15. data/lib/service_skeleton.rb +145 -22
  16. data/service_skeleton.gemspec +9 -10
  17. metadata +19 -102
  18. data/.editorconfig +0 -7
  19. data/.git-blame-ignore-revs +0 -2
  20. data/.github/workflows/ci.yml +0 -50
  21. data/lib/service_skeleton/config_class.rb +0 -16
  22. data/lib/service_skeleton/config_variable/boolean.rb +0 -21
  23. data/lib/service_skeleton/config_variable/enum.rb +0 -27
  24. data/lib/service_skeleton/config_variable/float.rb +0 -25
  25. data/lib/service_skeleton/config_variable/integer.rb +0 -25
  26. data/lib/service_skeleton/config_variable/kv_list.rb +0 -26
  27. data/lib/service_skeleton/config_variable/path_list.rb +0 -13
  28. data/lib/service_skeleton/config_variable/string.rb +0 -18
  29. data/lib/service_skeleton/config_variable/url.rb +0 -36
  30. data/lib/service_skeleton/config_variable/yaml_file.rb +0 -42
  31. data/lib/service_skeleton/generator.rb +0 -165
  32. data/lib/service_skeleton/metric_method_name.rb +0 -9
  33. data/lib/service_skeleton/runner.rb +0 -46
  34. data/lib/service_skeleton/service_name.rb +0 -20
  35. data/lib/service_skeleton/signal_manager.rb +0 -202
  36. data/lib/service_skeleton/signals_methods.rb +0 -15
  37. data/lib/service_skeleton/ultravisor_children.rb +0 -20
  38. data/lib/service_skeleton/ultravisor_loggerstash.rb +0 -11
  39. data/ultravisor/.yardopts +0 -1
  40. data/ultravisor/Guardfile +0 -9
  41. data/ultravisor/README.md +0 -404
  42. data/ultravisor/lib/ultravisor/child/call.rb +0 -21
  43. data/ultravisor/lib/ultravisor/child/call_receiver.rb +0 -14
  44. data/ultravisor/lib/ultravisor/child/cast.rb +0 -16
  45. data/ultravisor/lib/ultravisor/child/cast_receiver.rb +0 -11
  46. data/ultravisor/lib/ultravisor/child/process_cast_call.rb +0 -39
  47. data/ultravisor/lib/ultravisor/child.rb +0 -481
  48. data/ultravisor/lib/ultravisor/error.rb +0 -25
  49. data/ultravisor/lib/ultravisor/logging_helpers.rb +0 -32
  50. data/ultravisor/lib/ultravisor.rb +0 -216
  51. data/ultravisor/spec/example_group_methods.rb +0 -19
  52. data/ultravisor/spec/example_methods.rb +0 -8
  53. data/ultravisor/spec/spec_helper.rb +0 -52
  54. data/ultravisor/spec/ultravisor/add_child_spec.rb +0 -79
  55. data/ultravisor/spec/ultravisor/child/call_spec.rb +0 -121
  56. data/ultravisor/spec/ultravisor/child/cast_spec.rb +0 -111
  57. data/ultravisor/spec/ultravisor/child/id_spec.rb +0 -21
  58. data/ultravisor/spec/ultravisor/child/new_spec.rb +0 -152
  59. data/ultravisor/spec/ultravisor/child/restart_delay_spec.rb +0 -40
  60. data/ultravisor/spec/ultravisor/child/restart_spec.rb +0 -70
  61. data/ultravisor/spec/ultravisor/child/run_spec.rb +0 -95
  62. data/ultravisor/spec/ultravisor/child/shutdown_spec.rb +0 -124
  63. data/ultravisor/spec/ultravisor/child/spawn_spec.rb +0 -107
  64. data/ultravisor/spec/ultravisor/child/unsafe_instance_spec.rb +0 -55
  65. data/ultravisor/spec/ultravisor/child/wait_spec.rb +0 -32
  66. data/ultravisor/spec/ultravisor/new_spec.rb +0 -71
  67. data/ultravisor/spec/ultravisor/remove_child_spec.rb +0 -49
  68. data/ultravisor/spec/ultravisor/run_spec.rb +0 -334
  69. data/ultravisor/spec/ultravisor/shutdown_spec.rb +0 -106
data/README.md CHANGED
@@ -6,11 +6,10 @@ other parts of a larger system. It provides:
6
6
  * Prometheus-based metrics registry;
7
7
  * Signal handling;
8
8
  * Configuration extraction from the process environment;
9
- * Supervision and automated restarting of your service code;
10
9
  * and more.
11
10
 
12
11
  The general philosophy of `ServiceSkeleton` is to provide features which have
13
- been found to be almost universally necessary in modern deployment
12
+ been found to be almost universally necessary, in modern deployment
14
13
  configurations, to prefer convenience over configuration, and to always be
15
14
  secure by default.
16
15
 
@@ -41,9 +40,7 @@ like this:
41
40
 
42
41
  require "service_skeleton"
43
42
 
44
- class HelloService
45
- include ServiceSkeleton
46
-
43
+ class HelloService < ServiceSkeleton
47
44
  def run
48
45
  loop do
49
46
  puts "Hello, Service!"
@@ -52,132 +49,69 @@ like this:
52
49
  end
53
50
  end
54
51
 
55
- ServiceSkeleton::Runner.new(HelloService, ENV).run if __FILE__ == $0
52
+ HelloService.new(ENV).start if __FILE__ == $0
56
53
 
57
54
  First, we require the `"service_skeleton"` library, which is a pre-requisite
58
- for the `ServiceSkeleton` module to be available. Your code is placed in
59
- its own class in the `run` method, where you put your service's logic. The
60
- `ServiceSkeleton` module provides helper methods and initializers, which will
61
- be introduced as we go along.
62
-
63
- The `run` method is typically an infinite loop, because services are long-running,
64
- persistent processes. If you `run` method exits, or raises an unhandled exception,
65
- the supervisor will restart it.
55
+ for the `ServiceSkeleton` base class to be available, which is subclassed by
56
+ `HelloService`. The `run` method is where you put your service's work,
57
+ typically in an infinite loop, because services are long-running, persistent
58
+ processes.
66
59
 
67
- Finally, the last line uses the `ServiceSkeleton::Runner` class to actually run
68
- your service. This ensures that all of the scaffolding services, like the
69
- signal handler and metrics server, are up and running alongside your service
70
- code.
60
+ Finally, the last line instantiates an instance of our service (passing the
61
+ process environment in, for configuration), and then calls `.run` on that
62
+ object -- but only if we were called as a standalone program (rather than
63
+ being `require`'d ourselves).
71
64
 
72
65
 
73
66
  ## The `#run` loop
74
67
 
75
68
  The core of a service is usually some sort of infinite loop, which waits for a
76
- reason to do something, and then does it. A lot of services are network
77
- accessible, and so the "reason to do something" is "because someone made a
78
- connection to a port on which I'm listening". Other times it could be because
79
- of a periodic timer firing, a filesystem event, or anything else that takes
80
- your fancy.
69
+ reason to do something, and then does it. A lot of services are
70
+ network-accessible, and so the "reason to do something" is "because someone
71
+ made a connection to a port I'm listening". Other times it could be because of
72
+ a periodic timer, a filesystem event, or anything else that takes your fancy.
81
73
 
82
74
  Whatever it is, `ServiceSkeleton` doesn't discriminate. All you have to do is
83
- write it in your service class' `#run` method, and we'll take care of the rest.
75
+ write it in your subclass' `#run` method, and `ServiceSkeleton` will take care
76
+ of the rest.
84
77
 
85
78
 
86
79
  ### STAHP!
87
80
 
88
- When your service needs to be stopped for one reason or another, `ServiceSkeleton`
89
- needs to be able to tell your code to stop. By default, the thread that is
90
- running your service will just be killed, which might be fine if your service
91
- holds no state or persistent resources, but often that isn't the case.
92
-
93
- If your code needs to stop gracefully, you should define a (thread-safe)
94
- instance method, `#shutdown`, which does whatever is required to signal to
95
- your service worker code that it is time to return from the `#run` method.
96
- What that does, exactly, is up to you.
97
-
98
- ```
99
- class CustomShutdownService
100
- include ServiceSkeleton
101
-
102
- def run
103
- until @shutdown do
104
- puts "Hello, Service!"
105
- sleep 1
106
- end
107
-
108
- puts "Shutting down gracefully..."
109
- end
110
-
111
- def shutdown
112
- @shutdown = true
113
- end
114
- end
115
- ```
116
-
117
- To avoid the unpleasantness of a hung service, there is a limit on the amount
118
- of time that `ServiceSkeleton` will wait for your service code to terminate.
119
- This is, by default, five seconds, but you can modify that by defining a
120
- `#shutdown_timeout` method, which returns a `Numeric`, to specify the number of
121
- seconds that `ServiceSkeleton` should wait for termination.
122
-
123
- ```
124
- class SlowShutdownService
125
- include ServiceSkeleton
126
-
127
- def run
128
- until @shutdown do
129
- puts "Hello, Service!"
130
- sleep 60
131
- end
132
- end
133
-
134
- def shutdown
135
- @shutdown = true
136
- end
137
-
138
- def shutdown_timeout
139
- # We need an unusually long shutdown timeout for this service because
140
- # the shutdown flag is only checked once a minute, which is much longer
141
- # than the default shutdown period.
142
- 90
143
- end
144
- end
145
- ```
146
-
147
- If your service code does not terminate before the timeout, the thread will be,
148
- once again, unceremoniously killed.
81
+ Because the range of what a service can do is so broad, we can't provide a
82
+ generic way to stop your service. You'll want to provide a (thread-safe)
83
+ `#shutdown` method, which will gracefully cause your `#run` method to clean up
84
+ its resources and return. If you don't provide such a method, then the default
85
+ behaviour of `#shutdown` is to send an exception to the running thread, which
86
+ is somewhat brutal and prone to unpleasantness. If your service wants to
87
+ cleanly exit of its own accord, it can also just return from the `#run` method,
88
+ and the service will terminate without any fuss.
149
89
 
150
90
 
151
91
  ### Exceptional Behaviour
152
92
 
153
93
  If your `#run` loop happens to raise an unhandled exception, it will be caught,
154
- logged, and your service will be restarted. This involves instantiating a new
155
- instance of your service class, and calling `#run` again.
156
-
157
- In the event that the problem that caused the exception isn't transient, and
158
- your service code keeps exiting (either by raising an exception, or the `#run`
159
- method returning), the supervisor will, after a couple of retries, terminate
160
- the whole process.
161
-
162
- This allows for a *really* clean slate restart, by starting a whole new
163
- process. Your process manager should handle automatically restarting the
164
- process in a sensible manner.
94
+ logged, and the service will terminate. `ServiceSkeleton` believes in the
95
+ value of "fail fast", and if an exception makes it all way out, then there's no
96
+ telling what has gone wrong or if it can be reasonably recovered. The safest
97
+ option is to terminate the service process entirely, and have your service
98
+ supervisor start everything up again from scratch.
165
99
 
166
100
 
167
101
  ## The Service Name
168
102
 
169
103
  Several aspects of a `ServiceSkeleton` service, including environment variable
170
- and metric names, can incorporate the service's name, usually as a prefix. The
171
- service name is derived from the name of the class that you provide to
172
- `ServiceSkeleton::Runner.new`, by converting the `CamelCase` class name into a
173
- `snake_case` service name. If the class name is in a namespace, that is
174
- included also, with the `::` turned into `_`.
104
+ and metric names, expect the service's name as a prefix by default. The
105
+ service name is derived from the name of the class that subclasses
106
+ `ServiceSkeleton`, by converting the `CamelCase` class name into a `snake_case`
107
+ service name. If the class name is in a namespace, that is included also, with
108
+ the `::` turned into `_`.
175
109
 
176
110
 
177
111
  ## Configuration
178
112
 
179
113
  Almost every service has a need for some amount of configuration. In keeping
180
- with the general principles of the [12 factor app](https://12factor.net),
114
+ with the general principles of the [12 factor app](https://12factor.net), the
181
115
  `ServiceSkeleton` takes configuration from the environment. However, we try to
182
116
  minimise the amount of manual effort you need to expend to make that happen,
183
117
  and provide configuration management as a first-class operation.
@@ -185,17 +119,15 @@ and provide configuration management as a first-class operation.
185
119
 
186
120
  ### Basic Configuration
187
121
 
188
- The `ServiceSkeleton` module defines an instance method, called `#config`, which
189
- returns an instance of {ServiceSkeleton::Config} (or some other class you
122
+ Every `ServiceSkeleton` has an instance method defined, called `config`, which
123
+ returns an instance of `ServiceSkeleton::Config` (or some other class you
190
124
  specify; more on that below), which provides access to the environment that was
191
125
  passed into the service object at instantiation time (ie the `ENV` in
192
- `ServiceSkeleton.new(MyService, ENV)`) via the `#[]` method. So, in a very simple
126
+ `MyService.new(ENV)`) via the `#[]` method. So, in a very simple
193
127
  application where you want to get the name of the thing to say hello to, it
194
128
  might look like this:
195
129
 
196
- class GenericHelloService
197
- include ServiceSkeleton
198
-
130
+ class GenericHelloService < ServiceSkeleton
199
131
  def run
200
132
  loop do
201
133
  puts "Hello, #{config["RECIPIENT"]}!"
@@ -204,21 +136,17 @@ might look like this:
204
136
  end
205
137
  end
206
138
 
207
- ServiceSkeleton::Runner.new(GenericHelloService, "RECIPIENT" => "Bob").start
208
139
 
209
- This will print "Hello, Bob!" every second.
210
-
211
-
212
- ### Declaring Configuration Variables
140
+ ### Variable Declaration
213
141
 
214
142
  If your application has very minimal needs, it's possible that directly
215
- accessing the environment will be sufficient. However, you can (and usually
143
+ accessing the environment will be sufficient. However, you can (and probably
216
144
  should) declare your configuration variables in your service class, because
217
- that way you can get coerced values (numbers, booleans, lists, etc, rather than
218
- just plain strings), range and format checking (say "the number must be an
219
- integer between one and ten", or "the string must match this regex"), default
220
- values, and error reporting. You also get direct access to the configuration
221
- value as a method call on the `config` object.
145
+ that way you can get coerced values (numbers and booleans, rather than strings
146
+ everywhere), range and format checking (say "the number must be between one and
147
+ ten", or "the string must match this regex"), default values, and error
148
+ reporting. You also get direct access to the configuration value as a method
149
+ call on the `@config` object.
222
150
 
223
151
  To declare configuration variables, simply call one of the "config declaration
224
152
  methods" (as listed in the `ServiceSkeleton::ConfigVariables` module) in your
@@ -226,25 +154,21 @@ class definition, and pass it an environment variable name (as a string or
226
154
  symbol) and any relevant configuration parameters (like a default, or a
227
155
  validity range, or whatever).
228
156
 
229
- When you run your service (via {ServiceSkeleton::Runner#new}), the environment
230
- you pass in will be examined and the configuration initialised. If any values
231
- are invalid (number out of range, etc) or missing (for any configuration
232
- variable that doesn't have a default), then a
233
- {ServiceSkeleton::InvalidEnvironmentError} exception will be raised and the
234
- service will not start.
157
+ When your service object is instantiated, the environment will be examined and
158
+ the configuration setup. If any values are invalid (number out of range, etc)
159
+ or missing (for any configuration variable that doesn't have a default), then
160
+ an error will be logged and the service will not start.
235
161
 
236
162
  During your service's execution, any time you need to access a configuration
237
163
  value, just call the matching method name (the all-lowercase version of the
238
- environment variable name, without the service name prefix) on `config`, and
239
- you'll get the value in your lap.
164
+ environment variable name) on `config`, and you'll get the value in your
165
+ lap.
240
166
 
241
167
  Here's a version of our generic greeter service, using declared configuration
242
168
  variables:
243
169
 
244
- class GenericHelloService
245
- include ServiceSkeleton
246
-
247
- string :RECIPIENT, match: /\A\w+\z/
170
+ class GenericHelloService < ServiceSkeleton
171
+ string :RECIPIENT, matches: /\A\w+\z/
248
172
 
249
173
  def run
250
174
  loop do
@@ -254,52 +178,28 @@ variables:
254
178
  end
255
179
  end
256
180
 
257
- begin
258
- ServiceSkeleton::Runner.new(GenericHelloService, ENV).run
259
- rescue ServiceSkeleton::InvalidEnvironmentError => ex
260
- $stderr.puts "Configuration error found: #{ex.message}"
261
- exit 1
262
- end
263
-
264
- This service, if run without a `RECIPIENT` environment variable being available,
265
- will exit with an error. If that isn't what you want, you can declare a
266
- default for a config variable, like so:
267
-
268
- class GenericHelloService
269
- include ServiceSkeleton
270
-
271
- string :RECIPIENT, match: /\a\w+\z/, default: "Anonymous Coward"
272
-
273
- # ...
274
-
275
- *This* version will print "Hello, Anonymous Coward!" if no `RECIPIENT`
276
- environment variable is available.
277
-
278
181
 
279
182
  ### Environment Variable Prefixes
280
183
 
281
184
  It's common for all (or almost all) of your environment variables to have a
282
- common prefix, usually named for your service, to distinguish your service's
283
- configuration from any other environment variables lying around. However, to
284
- save on typing, you don't want to have to use that prefix when accessing your
285
- `config` methods.
185
+ common prefix, usually named for your service, so it's easy to identify your
186
+ service's configuration from anything else that's lying around. However, you
187
+ don't want to have to use that prefix when accessing your `config` methods.
286
188
 
287
189
  Enter: the service name prefix. Any of your environment variables whose name
288
190
  starts with [your service's name](#the-service-name) (matched
289
191
  case-insensitively) followed by an underscore will have that part of the
290
192
  environment variable name removed to determine the method name on `config`.
291
193
  The *original* environment variable name is still matched to a variable
292
- declaration, so, you need to declare the variable *with* the prefix, it is only
293
- the method name on the `config` object that won't have the prefix.
194
+ declaration, so, you need to declare the variable *with* the prefix, but the
195
+ method name won't have it.
294
196
 
295
197
  Using this environment variable prefix support, the `GenericHelloService` would
296
198
  have a (case-insensitive) prefix of `generic_hello_service_`. In that case,
297
199
  extending the above example a little more, you could do something like this:
298
200
 
299
- class GenericHelloService
300
- include ServiceSkeleton
301
-
302
- string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
201
+ class GenericHelloService < ServiceSkeleton
202
+ string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
303
203
 
304
204
  def run
305
205
  loop do
@@ -319,14 +219,12 @@ Sometimes your service will take configuration data that really, *really*
319
219
  shouldn't be available to subprocesses or anyone who manages to catch a
320
220
  sneak-peek at your service's environment. In that case, you can declare an
321
221
  environment variable as "sensitive", and after the configuration is parsed,
322
- that environment variable will be redacted from the environment.
222
+ that environment variable will be wiped from the environment.
323
223
 
324
224
  To declare an environment variable as "sensitive", simply pass the `sensitive`
325
225
  parameter, with a trueish value, to the variable declaration in your class:
326
226
 
327
- class DatabaseManager
328
- include ServiceSkeleton
329
-
227
+ class DatabaseManager < ServiceSkeleton
330
228
  string :DB_PASSWORD, sensitive: true
331
229
 
332
230
  ...
@@ -334,18 +232,17 @@ parameter, with a trueish value, to the variable declaration in your class:
334
232
 
335
233
  > **NOTE**: The process environment can only be modified if you pass the real,
336
234
  > honest-to-goodness `ENV` object into `MyServiceClass.new(ENV)`. If you
337
- > provide a copy of `ENV`, or some other hash entirely, that'll work if you
338
- > don't have any sensitive variables declared, but the moment you declare a
339
- > sensitive variable, passing in any hash other than `ENV` will cause the
340
- > service to log an error and refuse to start. This avoids the problems of
341
- > accidentally modifying global state if that would be potentially bad (we
342
- > assume you copied `ENV` for a reason) without leaving a gaping security hole
343
- > (sensitive data blindly passed into subprocesses that you didn't expect).
235
+ > provide a copy, or some other hash, that'll work *normally*, but if you have
236
+ > sensitive variables, the service will log an error and refuse to start. This
237
+ > avoids the problems of accidentally modifying global state if that would be
238
+ > potentially bad (we assume you copied `ENV` for a reason) without leaving a
239
+ > gaping security hole (sensitive data blindly passed into subprocesses that
240
+ > you didn't expect).
344
241
 
345
242
 
346
- ### Using a Custom Configuration Class
243
+ ### Custom Configuration Class
347
244
 
348
- Whilst we hope that {ServiceSkeleton::Config} will be useful in most
245
+ Whilst we hope that `ServiceSkeleton::Config` class will be useful in most
349
246
  situations, there are undoubtedly cases where the config management we provide
350
247
  won't be enough. In that case, you are encouraged to subclass
351
248
  `ServiceSkeleton::Config` and augment the standard interface with your own
@@ -361,16 +258,14 @@ class method in your service's class definition, like this:
361
258
  end
362
259
  end
363
260
 
364
- class MyService
365
- include ServiceSkeleton
366
-
261
+ class MyService < ServiceSkeleton
367
262
  config_class MyServiceConfig
368
263
 
369
264
  def run
370
265
  loop do
371
- puts config.something_funny
372
- sleep 1
373
- end
266
+ puts config.something_funny
267
+ sleep 1
268
+ end
374
269
  end
375
270
  end
376
271
 
@@ -384,11 +279,11 @@ for you to use.
384
279
 
385
280
  ### What You Get
386
281
 
387
- Every instance of your service class has a method named, uncreatively,
388
- `logger`. It is a (more-or-less) straight-up instance of the Ruby stdlib
389
- `Logger`, on which you can call all the usual methods (`#debug`, `#info`,
390
- `#warn`, `#error`, etc). By default, it sends all log messages to standard
391
- error.
282
+ Every instance of your service class (as subclassed from `ServiceSkeleton`)
283
+ has a method named, uncreatively, `logger`. It is a (more-or-less) straight-up
284
+ instance of the Ruby stdlib `Logger`, on which you can call all the usual
285
+ methods (`#debug`, `#info`, `#warn`, `#error`, etc). By default, it sends all
286
+ log messages to standard error.
392
287
 
393
288
  When calling the logger, you really, *really* want to use the
394
289
  "progname+message-in-a-block" style of recording log messages, which looks like
@@ -404,22 +299,18 @@ wish to actively debug, based on log messages that are tagged with a specified
404
299
  progname. No more grovelling through thousands of lines of debug logging to
405
300
  find the One Useful Message.
406
301
 
407
- You also get, as part of this package, built-in dynamic log level adjustment;
302
+ The `ServiceSkeleton` also provides built-in dynamic log level adjustment;
408
303
  using Unix signals or the admin HTTP interface (if enabled), you can tell the
409
304
  logger to increase or decrease logging verbosity *without interrupting
410
305
  service*. We are truly living in the future.
411
306
 
412
- Finally, if you're a devotee of the ELK stack, the logger can automagically
413
- send log entries straight into logstash, rather than you having to do it in
414
- some more roundabout fashion.
415
-
416
307
 
417
308
  ### Logging Configuration
418
309
 
419
310
  The logger automatically sets its configuration from, you guessed it, the
420
311
  environment. The following environment variables are recognised by the logger.
421
- All environment variable names are all-uppercase, and the `<SERVICENAME>_`
422
- portion is the all-uppercase [service name](#the-service-name).
312
+ All are all-uppercase, and the `<SERVICENAME>_` portion is the all-uppercase
313
+ [service name](#the-service-name).
423
314
 
424
315
  * **`<SERVICENAME>_LOG_LEVEL`** (default: `"INFO"`) -- the minimum severity of
425
316
  log messages which will be emitted by the logger.
@@ -430,36 +321,27 @@ portion is the all-uppercase [service name](#the-service-name).
430
321
 
431
322
  If you wish to change the severity level for a single progname, you can
432
323
  override the default log level for messages with a specific progname, by
433
- specifying one or more "progname/severity" pairs, separated by commas. A
434
- progname/severity pair looks like this:
435
-
324
+ specifying one or more "progname severities" separated by commas. A progname
325
+ severity looks like this:
326
+
436
327
  <progname>=<severity>
437
328
 
438
329
  To make things even more fun, if `<progname>` looks like a regular expression
439
330
  (starts with `/` or `%r{`, and ends with `/` or `}` plus optional flag
440
331
  characters), then all log messages with prognames *matching* the specified
441
- regex will have that severity applied. First match wins. The default is
332
+ regexp will have that severity applied. First match wins. The default is
442
333
  still specified as a bare severity name, and the default can only be set
443
334
  once.
444
335
 
445
- That's a lot to take in, so here's an example which sets the default to
446
- `INFO`, debugs the `buggy` progname, and only emits errors for messages with
447
- the (case-insensitive) string `noisy` in their progname:
336
+ That's a lot to take in, so here's an example which sets the default to `INFO`,
337
+ debugs the `buggy` progname, and only emits errors for messages with the
338
+ (case-insensitive) string `noisy` in their progname:
448
339
 
449
340
  INFO,buggy=DEBUG,/noisy/i=ERROR
450
341
 
451
342
  Logging levels can be changed at runtime, via [signals](#default-signals) or
452
343
  [the HTTP admin interface](#http-admin-interface).
453
344
 
454
- * **`<SERVICENAME>_LOGSTASH_SERVER`** (string; default `""`) -- if set to a
455
- non-empty string, we will engage the services of the [loggerstash
456
- gem](https://github.com/discourse/loggerstash) on your behalf to send all log
457
- entries to the logstash server you specify (as [an `address:port`,
458
- `hostname:port`, or SRV
459
- record](https://github.com/discourse/logstash_writer#usage). Just be sure
460
- and [configure logstash
461
- appropriately](https://github.com/discourse/loggerstash#logstash-configuration).
462
-
463
345
  * **`<SERVICENAME>_LOG_ENABLE_TIMESTAMPS`** (boolean; default: `"no"`) -- if
464
346
  set to a true-ish value (`yes`/`y`/`on`/`true`/`1`), then the log entries
465
347
  emitted by the logger will have the current time (to the nearest nanosecond)
@@ -470,8 +352,8 @@ portion is the all-uppercase [service name](#the-service-name).
470
352
  where log messages aren't automatically timestamped, then you can use this to
471
353
  get them back.
472
354
 
473
- * **`<SERVICENAME>_LOG_FILE`** (string; default: `"/dev/stderr"`) -- the file
474
- to which log messages are written. The default, to send messages to standard
355
+ * **`<SERVICENAME>_LOG_FILE`** (string; default: `"/dev/stderr`) -- the file to
356
+ which log messages are written. The default, to send messages to standard
475
357
  error, is a good choice if you are using a supervisor system which captures
476
358
  service output to its own logging system, however if you are stuck without
477
359
  such niceties, you can specify a file on disk to log to instead.
@@ -506,7 +388,7 @@ portion is the all-uppercase [service name](#the-service-name).
506
388
  ## Metrics
507
389
 
508
390
  Running a service without metrics is like trying to fly a fighter jet whilst
509
- blindfolded: everything seems to be going OK until you slam into the side of a
391
+ blindfolded. Everything seems to be going OK until you slam into the side of a
510
392
  mountain you never saw coming. For that reason, `ServiceSkeleton` provides a
511
393
  Prometheus-based metrics registry, a bunch of default process-level metrics, an
512
394
  optional HTTP metrics server, and simple integration with [the Prometheus ruby
@@ -518,43 +400,45 @@ easy as possible to instrument the heck out of your service.
518
400
  ### Defining and Using Metrics
519
401
 
520
402
  All the metrics you want to use within your service need to be registered
521
- before use. This is done via class methods, similar to declaring environment
522
- variables.
403
+ before use. This is typically done in the `#run` method, before entering the
404
+ infinite loop.
523
405
 
524
406
  To register a metric, use one of the standard metric registration methods from
525
407
  [Prometheus::Client::Registry](https://www.rubydoc.info/gems/prometheus-client/0.8.0/Prometheus/Client/Registry)
526
- (`counter`, `gauge`, `histogram`, `summary`) or `metric` (equivalent
527
- to the `register` method of `Prometheus::Client::Registry) in your class
528
- definition to register the metric for use.
408
+ (`#counter`, `gauge`, `histogram`, `summary`, or `register`) on the `metrics`
409
+ object to create or register the metric.
529
410
 
530
411
  In our generic greeter service we've been using as an example so far, you might
531
412
  like to define a metric to count how many greetings have been sent. You'd define
532
413
  such a metric like this:
533
414
 
534
- class GenericHelloService
535
- include ServiceSkeleton
536
-
537
- string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
538
-
539
- counter :greetings_total, docstring: "How many greetings we have sent", labels: %i{recipient}
415
+ class GenericHelloService < ServiceSkeleton
416
+ string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
540
417
 
541
- # ...
542
-
543
- When it comes time to actually *use* the metrics you have created, you access
544
- them as methods on the `metrics` method in your service worker instance. Thus,
545
- to increment our greeting counter, you simply do:
418
+ def run
419
+ metrics.counter(:greetings_total, "How many greetings we have sent")
546
420
 
547
- class GenericHelloService
548
- include ServiceSkeleton
421
+ loop do
422
+ puts "Hello, #{config.recipient}!"
423
+ sleep 1
424
+ end
425
+ end
426
+ end
549
427
 
550
- string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
428
+ When it comes time to actually *use* the metrics you have created, it's typical
429
+ to keep a copy of the metric object laying around, or call `metrics.get`. However,
430
+ we make it easier to access your metrics, by defining a method named for the metric
431
+ on `metrics`. Thus, to increment our greeting counter, you can simply do:
551
432
 
552
- counter :greetings_total, docstring: "How many greetings we have sent", labels: %i{recipient}
433
+ class GenericHelloService < ServiceSkeleton
434
+ string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
553
435
 
554
436
  def run
437
+ metrics.counter(:greetings_total, "How many greetings we have sent")
438
+
555
439
  loop do
556
440
  puts "Hello, #{config.recipient}!"
557
- metrics.greetings_total.increment(labels: { recipient: config.recipient })
441
+ metrics.greetings_total.increment(recipient: config.recipient)
558
442
  sleep 1
559
443
  end
560
444
  end
@@ -565,17 +449,15 @@ any metrics you define which have the [service name](#the-service-name) as a
565
449
  prefix will have that prefix (and the immediately-subsequent underscore) removed
566
450
  before defining the metric accessor method, which keeps typing to a minimum:
567
451
 
568
- class GenericHelloService
569
- include ServiceSkeleton
570
-
571
- string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
572
-
573
- counter :generic_hello_service_greetings_total, docstring: "How many greetings we have sent", labels: %i{recipient}
452
+ class GenericHelloService < ServiceSkeleton
453
+ string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
574
454
 
575
455
  def run
456
+ metrics.counter(:generic_hello_service_greetings_total, "How many greetings we have sent")
457
+
576
458
  loop do
577
459
  puts "Hello, #{config.recipient}!"
578
- metrics.greetings_total.increment(labels: { recipient: config.recipient })
460
+ metrics.greetings_total.increment(recipient: config.recipient)
579
461
  sleep 1
580
462
  end
581
463
  end
@@ -594,6 +476,9 @@ metrics](https://www.rubydoc.info/gems/frankenstein/Frankenstein/RubyGCMetrics),
594
476
  and [Ruby VM
595
477
  metrics](https://www.rubydoc.info/gems/frankenstein/Frankenstein/RubyVMMetrics).
596
478
 
479
+ If, for some reason, you *don't* want any default metrics in your bundle of
480
+ metrics, call `no_default_metrics` in your service class definition.
481
+
597
482
 
598
483
  ### Metrics Server Configuration
599
484
 
@@ -604,7 +489,7 @@ all-uppercase, and the `<SERVICENAME>_` portion is the all-uppercase version
604
489
  of [the service name](#the-service-name).
605
490
 
606
491
  * **`<SERVICENAME>_METRICS_PORT`** (integer; range 1..65535; default: `""`) --
607
- if set to an integer which is a valid port number (`1` to `65535`,
492
+ if set to a non-empty integer which is a valid port number (`1` to `65535`,
608
493
  inclusive), an HTTP server will be started which will respond to a request to
609
494
  `/metrics` with a Prometheus-compatible dump of time series data.
610
495
 
@@ -622,15 +507,14 @@ behaviours for common signals.
622
507
 
623
508
  ### Default Signals
624
509
 
625
- When the `#run` method on a `ServiceSkeleton::Runner` instance is called, the
626
- following signals will be hooked, and will perform the described action when
627
- that signal is received:
510
+ Unless told otherwise, by calling the `no_default_signals` class method in your
511
+ service class' definition, the following signals will be hooked with the following
512
+ behaviour:
628
513
 
629
514
  * **`SIGUSR1`** -- increase the default minimum severity for messages which
630
- will be emitted by the logger (`FATAL` -> `ERROR` -> `WARN` -> `INFO` ->
631
- `DEBUG`). The default severity only applies to log messages whose progname
632
- does not match a "progname/severity" pair (see [Logging
633
- Configuration](#logging-configuration)).
515
+ will be emitted by the logger. The default severity only applies to log
516
+ messages whose progname does not match a "progname specifier" (see "[Logging
517
+ Configuration](#logging-configuration)").
634
518
 
635
519
  * **`SIGUSR2`** -- decrease the default minimum severity for messages which
636
520
  will be emitted by the logger.
@@ -638,7 +522,7 @@ that signal is received:
638
522
  * **`SIGHUP`** -- close and reopen the log file, if logging to a file on disk.
639
523
  Because of the `ServiceSkeleton`'s default log rotation policy, this shouldn't
640
524
  ordinarily be required, but if you've turned off the default log rotation,
641
- you may need this.
525
+ you may need tis.
642
526
 
643
527
  * **`SIGQUIT`** -- dump a *whooooooole* lot of debugging information to
644
528
  standard error, including memory allocation summaries and stack traces of all
@@ -648,43 +532,38 @@ that signal is received:
648
532
 
649
533
  * **`SIGINT`** / **`SIGTERM`** -- ask the service to gracefully stop running.
650
534
  It will call your service's `#shutdown` method to ask it to stop what it's
651
- doing and exit. If the signal is sent a second time, the service will be
652
- summarily terminated as soon as practical, without being given the
653
- opportunity to gracefully release resources. As usual, if a service process
654
- needs to be whacked completely and utterly *right now*, `SIGKILL` is what you
655
- want to use.
535
+ doing and exit. If the signal is sent twice, your run method will be
536
+ summarily terminated and everything will be terminated quickly. As usual, if
537
+ a service process needs to be whacked completely and utterly *right now*,
538
+ `SIGKILL` is what you want to use.
656
539
 
657
540
 
658
541
  ### Hooking Signals
659
542
 
660
543
  In addition to the above default signal dispositions, you can also hook signals
661
544
  yourself for whatever purpose you desire. This is typically done in your
662
- `#run` method, before entering the main service loop.
545
+ `#run` method, before entering the infinite loop.
663
546
 
664
547
  To hook a signal, just call `hook_signal` with a signal specification and a
665
- block of code to execute when the signal fires in your class definition. You
666
- can even hook the same signal more than once, because the signal handlers that
667
- `ServiceSkeleton` uses chain to other signal handlers. As an example, if you
668
- want to print "oof!" every time the `SIGCONT` signal is received, you'd do
669
- something like this:
670
-
671
- class MyService
672
- include ServiceSkeleton
673
-
674
- hook_signal("CONT") { puts "oof!" }
548
+ block of code to execute when the signal fires. You can even hook the same
549
+ signal more than once, because the signal handlers that `SkeletonService` uses
550
+ chain to other signal handlers. As an example, if you want to print "oof!"
551
+ every time the `SIGCONT` signal is received, you'd do something like this:
675
552
 
553
+ class MyService < ServiceSkeleton
676
554
  def run
555
+ hook_signal("CONT") { puts "oof!" }
556
+
677
557
  loop { sleep }
678
558
  end
679
559
  end
680
560
 
681
- The code in the block will be executed in the context of the service worker
682
- instance that is running at the time the signal is received. You are
683
- responsible for ensuring that whatever your handler does is concurrency-safe.
684
-
685
561
  When the service is shutdown, all signal handlers will be automatically
686
562
  unhooked, which saves you having to do it yourself.
687
563
 
564
+ > **NOTE**: You can define a maximum of 256 signal hooks in a single service,
565
+ > including the default signal hooks.
566
+
688
567
 
689
568
  ## HTTP Admin Interface
690
569
 
@@ -696,9 +575,9 @@ different?
696
575
  ### HTTP Admin Configuration
697
576
 
698
577
  In the spirit of "secure by default", you must explicitly enable the HTTP admin
699
- interface, and configure an authentication method. To do that, use the
700
- following environment variables, where `<SERVICENAME>_` is the all-uppercase
701
- version of [the service name](#the-service-name).
578
+ interface, and it requires authentication. To do that, use the following
579
+ environment variables, where `<SERVICENAME>_` is the all-uppercase version of
580
+ [the service name](#the-service-name).
702
581
 
703
582
  * **`<SERVICENAME>_HTTP_ADMIN_PORT`** (integer; range 1..65535; default: `""`)
704
583
  -- if set to a valid port number (`1` to `65535` inclusive), the HTTP admin
@@ -722,11 +601,7 @@ version of [the service name](#the-service-name).
722
601
  interface to be enabled.
723
602
 
724
603
 
725
- ### HTTP Admin Usage
726
-
727
- The HTTP admin interface provides both an interactive, browser-based mode,
728
- as well as a RESTful interface, which should, in general, provide equivalent
729
- functionality.
604
+ ### HTTP Admin Features
730
605
 
731
606
  * Visiting the service's `IP address:port` in a web browser will bring up an HTML
732
607
  interface showing all the features that are available. Usage should
@@ -751,8 +626,7 @@ conduct](CODE_OF_CONDUCT.md).
751
626
  Unless otherwise stated, everything in this repo is covered by the following
752
627
  copyright notice:
753
628
 
754
- Copyright (C) 2018, 2019 Civilized Discourse Construction Kit, Inc.
755
- Copyright (C) 2019, 2020 Matt Palmer
629
+ Copyright (C) 2018 Civilized Discourse Construction Kit, Inc.
756
630
 
757
631
  This program is free software: you can redistribute it and/or modify it
758
632
  under the terms of the GNU General Public License version 3, as