service_skeleton 0.0.0.1.ENOTAG → 0.0.0.2.g46c1e0e
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.gitignore +0 -2
- data/.rubocop.yml +114 -9
- data/.travis.yml +11 -0
- data/README.md +153 -279
- data/lib/service_skeleton/background_worker.rb +80 -0
- data/lib/service_skeleton/config.rb +18 -78
- data/lib/service_skeleton/config_variable.rb +8 -29
- data/lib/service_skeleton/config_variables.rb +68 -54
- data/lib/service_skeleton/error.rb +3 -5
- data/lib/service_skeleton/filtering_logger.rb +0 -2
- data/lib/service_skeleton/logging_helpers.rb +3 -10
- data/lib/service_skeleton/metrics_methods.rb +13 -28
- data/lib/service_skeleton/signal_handler.rb +183 -0
- data/lib/service_skeleton.rb +145 -22
- data/service_skeleton.gemspec +9 -10
- metadata +19 -102
- data/.editorconfig +0 -7
- data/.git-blame-ignore-revs +0 -2
- data/.github/workflows/ci.yml +0 -50
- data/lib/service_skeleton/config_class.rb +0 -16
- data/lib/service_skeleton/config_variable/boolean.rb +0 -21
- data/lib/service_skeleton/config_variable/enum.rb +0 -27
- data/lib/service_skeleton/config_variable/float.rb +0 -25
- data/lib/service_skeleton/config_variable/integer.rb +0 -25
- data/lib/service_skeleton/config_variable/kv_list.rb +0 -26
- data/lib/service_skeleton/config_variable/path_list.rb +0 -13
- data/lib/service_skeleton/config_variable/string.rb +0 -18
- data/lib/service_skeleton/config_variable/url.rb +0 -36
- data/lib/service_skeleton/config_variable/yaml_file.rb +0 -42
- data/lib/service_skeleton/generator.rb +0 -165
- data/lib/service_skeleton/metric_method_name.rb +0 -9
- data/lib/service_skeleton/runner.rb +0 -46
- data/lib/service_skeleton/service_name.rb +0 -20
- data/lib/service_skeleton/signal_manager.rb +0 -202
- data/lib/service_skeleton/signals_methods.rb +0 -15
- data/lib/service_skeleton/ultravisor_children.rb +0 -20
- data/lib/service_skeleton/ultravisor_loggerstash.rb +0 -11
- data/ultravisor/.yardopts +0 -1
- data/ultravisor/Guardfile +0 -9
- data/ultravisor/README.md +0 -404
- data/ultravisor/lib/ultravisor/child/call.rb +0 -21
- data/ultravisor/lib/ultravisor/child/call_receiver.rb +0 -14
- data/ultravisor/lib/ultravisor/child/cast.rb +0 -16
- data/ultravisor/lib/ultravisor/child/cast_receiver.rb +0 -11
- data/ultravisor/lib/ultravisor/child/process_cast_call.rb +0 -39
- data/ultravisor/lib/ultravisor/child.rb +0 -481
- data/ultravisor/lib/ultravisor/error.rb +0 -25
- data/ultravisor/lib/ultravisor/logging_helpers.rb +0 -32
- data/ultravisor/lib/ultravisor.rb +0 -216
- data/ultravisor/spec/example_group_methods.rb +0 -19
- data/ultravisor/spec/example_methods.rb +0 -8
- data/ultravisor/spec/spec_helper.rb +0 -52
- data/ultravisor/spec/ultravisor/add_child_spec.rb +0 -79
- data/ultravisor/spec/ultravisor/child/call_spec.rb +0 -121
- data/ultravisor/spec/ultravisor/child/cast_spec.rb +0 -111
- data/ultravisor/spec/ultravisor/child/id_spec.rb +0 -21
- data/ultravisor/spec/ultravisor/child/new_spec.rb +0 -152
- data/ultravisor/spec/ultravisor/child/restart_delay_spec.rb +0 -40
- data/ultravisor/spec/ultravisor/child/restart_spec.rb +0 -70
- data/ultravisor/spec/ultravisor/child/run_spec.rb +0 -95
- data/ultravisor/spec/ultravisor/child/shutdown_spec.rb +0 -124
- data/ultravisor/spec/ultravisor/child/spawn_spec.rb +0 -107
- data/ultravisor/spec/ultravisor/child/unsafe_instance_spec.rb +0 -55
- data/ultravisor/spec/ultravisor/child/wait_spec.rb +0 -32
- data/ultravisor/spec/ultravisor/new_spec.rb +0 -71
- data/ultravisor/spec/ultravisor/remove_child_spec.rb +0 -49
- data/ultravisor/spec/ultravisor/run_spec.rb +0 -334
- data/ultravisor/spec/ultravisor/shutdown_spec.rb +0 -106
data/README.md
CHANGED
@@ -6,11 +6,10 @@ other parts of a larger system. It provides:
|
|
6
6
|
* Prometheus-based metrics registry;
|
7
7
|
* Signal handling;
|
8
8
|
* Configuration extraction from the process environment;
|
9
|
-
* Supervision and automated restarting of your service code;
|
10
9
|
* and more.
|
11
10
|
|
12
11
|
The general philosophy of `ServiceSkeleton` is to provide features which have
|
13
|
-
been found to be almost universally necessary in modern deployment
|
12
|
+
been found to be almost universally necessary, in modern deployment
|
14
13
|
configurations, to prefer convenience over configuration, and to always be
|
15
14
|
secure by default.
|
16
15
|
|
@@ -41,9 +40,7 @@ like this:
|
|
41
40
|
|
42
41
|
require "service_skeleton"
|
43
42
|
|
44
|
-
class HelloService
|
45
|
-
include ServiceSkeleton
|
46
|
-
|
43
|
+
class HelloService < ServiceSkeleton
|
47
44
|
def run
|
48
45
|
loop do
|
49
46
|
puts "Hello, Service!"
|
@@ -52,132 +49,69 @@ like this:
|
|
52
49
|
end
|
53
50
|
end
|
54
51
|
|
55
|
-
|
52
|
+
HelloService.new(ENV).start if __FILE__ == $0
|
56
53
|
|
57
54
|
First, we require the `"service_skeleton"` library, which is a pre-requisite
|
58
|
-
for the `ServiceSkeleton`
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
The `run` method is typically an infinite loop, because services are long-running,
|
64
|
-
persistent processes. If you `run` method exits, or raises an unhandled exception,
|
65
|
-
the supervisor will restart it.
|
55
|
+
for the `ServiceSkeleton` base class to be available, which is subclassed by
|
56
|
+
`HelloService`. The `run` method is where you put your service's work,
|
57
|
+
typically in an infinite loop, because services are long-running, persistent
|
58
|
+
processes.
|
66
59
|
|
67
|
-
Finally, the last line
|
68
|
-
|
69
|
-
|
70
|
-
|
60
|
+
Finally, the last line instantiates an instance of our service (passing the
|
61
|
+
process environment in, for configuration), and then calls `.run` on that
|
62
|
+
object -- but only if we were called as a standalone program (rather than
|
63
|
+
being `require`'d ourselves).
|
71
64
|
|
72
65
|
|
73
66
|
## The `#run` loop
|
74
67
|
|
75
68
|
The core of a service is usually some sort of infinite loop, which waits for a
|
76
|
-
reason to do something, and then does it. A lot of services are
|
77
|
-
accessible, and so the "reason to do something" is "because someone
|
78
|
-
connection to a port
|
79
|
-
|
80
|
-
your fancy.
|
69
|
+
reason to do something, and then does it. A lot of services are
|
70
|
+
network-accessible, and so the "reason to do something" is "because someone
|
71
|
+
made a connection to a port I'm listening". Other times it could be because of
|
72
|
+
a periodic timer, a filesystem event, or anything else that takes your fancy.
|
81
73
|
|
82
74
|
Whatever it is, `ServiceSkeleton` doesn't discriminate. All you have to do is
|
83
|
-
write it in your
|
75
|
+
write it in your subclass' `#run` method, and `ServiceSkeleton` will take care
|
76
|
+
of the rest.
|
84
77
|
|
85
78
|
|
86
79
|
### STAHP!
|
87
80
|
|
88
|
-
|
89
|
-
|
90
|
-
|
91
|
-
|
92
|
-
|
93
|
-
|
94
|
-
|
95
|
-
|
96
|
-
What that does, exactly, is up to you.
|
97
|
-
|
98
|
-
```
|
99
|
-
class CustomShutdownService
|
100
|
-
include ServiceSkeleton
|
101
|
-
|
102
|
-
def run
|
103
|
-
until @shutdown do
|
104
|
-
puts "Hello, Service!"
|
105
|
-
sleep 1
|
106
|
-
end
|
107
|
-
|
108
|
-
puts "Shutting down gracefully..."
|
109
|
-
end
|
110
|
-
|
111
|
-
def shutdown
|
112
|
-
@shutdown = true
|
113
|
-
end
|
114
|
-
end
|
115
|
-
```
|
116
|
-
|
117
|
-
To avoid the unpleasantness of a hung service, there is a limit on the amount
|
118
|
-
of time that `ServiceSkeleton` will wait for your service code to terminate.
|
119
|
-
This is, by default, five seconds, but you can modify that by defining a
|
120
|
-
`#shutdown_timeout` method, which returns a `Numeric`, to specify the number of
|
121
|
-
seconds that `ServiceSkeleton` should wait for termination.
|
122
|
-
|
123
|
-
```
|
124
|
-
class SlowShutdownService
|
125
|
-
include ServiceSkeleton
|
126
|
-
|
127
|
-
def run
|
128
|
-
until @shutdown do
|
129
|
-
puts "Hello, Service!"
|
130
|
-
sleep 60
|
131
|
-
end
|
132
|
-
end
|
133
|
-
|
134
|
-
def shutdown
|
135
|
-
@shutdown = true
|
136
|
-
end
|
137
|
-
|
138
|
-
def shutdown_timeout
|
139
|
-
# We need an unusually long shutdown timeout for this service because
|
140
|
-
# the shutdown flag is only checked once a minute, which is much longer
|
141
|
-
# than the default shutdown period.
|
142
|
-
90
|
143
|
-
end
|
144
|
-
end
|
145
|
-
```
|
146
|
-
|
147
|
-
If your service code does not terminate before the timeout, the thread will be,
|
148
|
-
once again, unceremoniously killed.
|
81
|
+
Because the range of what a service can do is so broad, we can't provide a
|
82
|
+
generic way to stop your service. You'll want to provide a (thread-safe)
|
83
|
+
`#shutdown` method, which will gracefully cause your `#run` method to clean up
|
84
|
+
its resources and return. If you don't provide such a method, then the default
|
85
|
+
behaviour of `#shutdown` is to send an exception to the running thread, which
|
86
|
+
is somewhat brutal and prone to unpleasantness. If your service wants to
|
87
|
+
cleanly exit of its own accord, it can also just return from the `#run` method,
|
88
|
+
and the service will terminate without any fuss.
|
149
89
|
|
150
90
|
|
151
91
|
### Exceptional Behaviour
|
152
92
|
|
153
93
|
If your `#run` loop happens to raise an unhandled exception, it will be caught,
|
154
|
-
logged, and
|
155
|
-
|
156
|
-
|
157
|
-
|
158
|
-
|
159
|
-
method returning), the supervisor will, after a couple of retries, terminate
|
160
|
-
the whole process.
|
161
|
-
|
162
|
-
This allows for a *really* clean slate restart, by starting a whole new
|
163
|
-
process. Your process manager should handle automatically restarting the
|
164
|
-
process in a sensible manner.
|
94
|
+
logged, and the service will terminate. `ServiceSkeleton` believes in the
|
95
|
+
value of "fail fast", and if an exception makes it all way out, then there's no
|
96
|
+
telling what has gone wrong or if it can be reasonably recovered. The safest
|
97
|
+
option is to terminate the service process entirely, and have your service
|
98
|
+
supervisor start everything up again from scratch.
|
165
99
|
|
166
100
|
|
167
101
|
## The Service Name
|
168
102
|
|
169
103
|
Several aspects of a `ServiceSkeleton` service, including environment variable
|
170
|
-
and metric names,
|
171
|
-
service name is derived from the name of the class that
|
172
|
-
`ServiceSkeleton
|
173
|
-
|
174
|
-
|
104
|
+
and metric names, expect the service's name as a prefix by default. The
|
105
|
+
service name is derived from the name of the class that subclasses
|
106
|
+
`ServiceSkeleton`, by converting the `CamelCase` class name into a `snake_case`
|
107
|
+
service name. If the class name is in a namespace, that is included also, with
|
108
|
+
the `::` turned into `_`.
|
175
109
|
|
176
110
|
|
177
111
|
## Configuration
|
178
112
|
|
179
113
|
Almost every service has a need for some amount of configuration. In keeping
|
180
|
-
with the general principles of the [12 factor app](https://12factor.net),
|
114
|
+
with the general principles of the [12 factor app](https://12factor.net), the
|
181
115
|
`ServiceSkeleton` takes configuration from the environment. However, we try to
|
182
116
|
minimise the amount of manual effort you need to expend to make that happen,
|
183
117
|
and provide configuration management as a first-class operation.
|
@@ -185,17 +119,15 @@ and provide configuration management as a first-class operation.
|
|
185
119
|
|
186
120
|
### Basic Configuration
|
187
121
|
|
188
|
-
|
189
|
-
returns an instance of
|
122
|
+
Every `ServiceSkeleton` has an instance method defined, called `config`, which
|
123
|
+
returns an instance of `ServiceSkeleton::Config` (or some other class you
|
190
124
|
specify; more on that below), which provides access to the environment that was
|
191
125
|
passed into the service object at instantiation time (ie the `ENV` in
|
192
|
-
`
|
126
|
+
`MyService.new(ENV)`) via the `#[]` method. So, in a very simple
|
193
127
|
application where you want to get the name of the thing to say hello to, it
|
194
128
|
might look like this:
|
195
129
|
|
196
|
-
class GenericHelloService
|
197
|
-
include ServiceSkeleton
|
198
|
-
|
130
|
+
class GenericHelloService < ServiceSkeleton
|
199
131
|
def run
|
200
132
|
loop do
|
201
133
|
puts "Hello, #{config["RECIPIENT"]}!"
|
@@ -204,21 +136,17 @@ might look like this:
|
|
204
136
|
end
|
205
137
|
end
|
206
138
|
|
207
|
-
ServiceSkeleton::Runner.new(GenericHelloService, "RECIPIENT" => "Bob").start
|
208
139
|
|
209
|
-
|
210
|
-
|
211
|
-
|
212
|
-
### Declaring Configuration Variables
|
140
|
+
### Variable Declaration
|
213
141
|
|
214
142
|
If your application has very minimal needs, it's possible that directly
|
215
|
-
accessing the environment will be sufficient. However, you can (and
|
143
|
+
accessing the environment will be sufficient. However, you can (and probably
|
216
144
|
should) declare your configuration variables in your service class, because
|
217
|
-
that way you can get coerced values (numbers
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
|
145
|
+
that way you can get coerced values (numbers and booleans, rather than strings
|
146
|
+
everywhere), range and format checking (say "the number must be between one and
|
147
|
+
ten", or "the string must match this regex"), default values, and error
|
148
|
+
reporting. You also get direct access to the configuration value as a method
|
149
|
+
call on the `@config` object.
|
222
150
|
|
223
151
|
To declare configuration variables, simply call one of the "config declaration
|
224
152
|
methods" (as listed in the `ServiceSkeleton::ConfigVariables` module) in your
|
@@ -226,25 +154,21 @@ class definition, and pass it an environment variable name (as a string or
|
|
226
154
|
symbol) and any relevant configuration parameters (like a default, or a
|
227
155
|
validity range, or whatever).
|
228
156
|
|
229
|
-
When
|
230
|
-
|
231
|
-
|
232
|
-
|
233
|
-
{ServiceSkeleton::InvalidEnvironmentError} exception will be raised and the
|
234
|
-
service will not start.
|
157
|
+
When your service object is instantiated, the environment will be examined and
|
158
|
+
the configuration setup. If any values are invalid (number out of range, etc)
|
159
|
+
or missing (for any configuration variable that doesn't have a default), then
|
160
|
+
an error will be logged and the service will not start.
|
235
161
|
|
236
162
|
During your service's execution, any time you need to access a configuration
|
237
163
|
value, just call the matching method name (the all-lowercase version of the
|
238
|
-
environment variable name
|
239
|
-
|
164
|
+
environment variable name) on `config`, and you'll get the value in your
|
165
|
+
lap.
|
240
166
|
|
241
167
|
Here's a version of our generic greeter service, using declared configuration
|
242
168
|
variables:
|
243
169
|
|
244
|
-
class GenericHelloService
|
245
|
-
|
246
|
-
|
247
|
-
string :RECIPIENT, match: /\A\w+\z/
|
170
|
+
class GenericHelloService < ServiceSkeleton
|
171
|
+
string :RECIPIENT, matches: /\A\w+\z/
|
248
172
|
|
249
173
|
def run
|
250
174
|
loop do
|
@@ -254,52 +178,28 @@ variables:
|
|
254
178
|
end
|
255
179
|
end
|
256
180
|
|
257
|
-
begin
|
258
|
-
ServiceSkeleton::Runner.new(GenericHelloService, ENV).run
|
259
|
-
rescue ServiceSkeleton::InvalidEnvironmentError => ex
|
260
|
-
$stderr.puts "Configuration error found: #{ex.message}"
|
261
|
-
exit 1
|
262
|
-
end
|
263
|
-
|
264
|
-
This service, if run without a `RECIPIENT` environment variable being available,
|
265
|
-
will exit with an error. If that isn't what you want, you can declare a
|
266
|
-
default for a config variable, like so:
|
267
|
-
|
268
|
-
class GenericHelloService
|
269
|
-
include ServiceSkeleton
|
270
|
-
|
271
|
-
string :RECIPIENT, match: /\a\w+\z/, default: "Anonymous Coward"
|
272
|
-
|
273
|
-
# ...
|
274
|
-
|
275
|
-
*This* version will print "Hello, Anonymous Coward!" if no `RECIPIENT`
|
276
|
-
environment variable is available.
|
277
|
-
|
278
181
|
|
279
182
|
### Environment Variable Prefixes
|
280
183
|
|
281
184
|
It's common for all (or almost all) of your environment variables to have a
|
282
|
-
common prefix, usually named for your service, to
|
283
|
-
configuration from
|
284
|
-
|
285
|
-
`config` methods.
|
185
|
+
common prefix, usually named for your service, so it's easy to identify your
|
186
|
+
service's configuration from anything else that's lying around. However, you
|
187
|
+
don't want to have to use that prefix when accessing your `config` methods.
|
286
188
|
|
287
189
|
Enter: the service name prefix. Any of your environment variables whose name
|
288
190
|
starts with [your service's name](#the-service-name) (matched
|
289
191
|
case-insensitively) followed by an underscore will have that part of the
|
290
192
|
environment variable name removed to determine the method name on `config`.
|
291
193
|
The *original* environment variable name is still matched to a variable
|
292
|
-
declaration, so, you need to declare the variable *with* the prefix,
|
293
|
-
|
194
|
+
declaration, so, you need to declare the variable *with* the prefix, but the
|
195
|
+
method name won't have it.
|
294
196
|
|
295
197
|
Using this environment variable prefix support, the `GenericHelloService` would
|
296
198
|
have a (case-insensitive) prefix of `generic_hello_service_`. In that case,
|
297
199
|
extending the above example a little more, you could do something like this:
|
298
200
|
|
299
|
-
class GenericHelloService
|
300
|
-
|
301
|
-
|
302
|
-
string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
|
201
|
+
class GenericHelloService < ServiceSkeleton
|
202
|
+
string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
|
303
203
|
|
304
204
|
def run
|
305
205
|
loop do
|
@@ -319,14 +219,12 @@ Sometimes your service will take configuration data that really, *really*
|
|
319
219
|
shouldn't be available to subprocesses or anyone who manages to catch a
|
320
220
|
sneak-peek at your service's environment. In that case, you can declare an
|
321
221
|
environment variable as "sensitive", and after the configuration is parsed,
|
322
|
-
that environment variable will be
|
222
|
+
that environment variable will be wiped from the environment.
|
323
223
|
|
324
224
|
To declare an environment variable as "sensitive", simply pass the `sensitive`
|
325
225
|
parameter, with a trueish value, to the variable declaration in your class:
|
326
226
|
|
327
|
-
class DatabaseManager
|
328
|
-
include ServiceSkeleton
|
329
|
-
|
227
|
+
class DatabaseManager < ServiceSkeleton
|
330
228
|
string :DB_PASSWORD, sensitive: true
|
331
229
|
|
332
230
|
...
|
@@ -334,18 +232,17 @@ parameter, with a trueish value, to the variable declaration in your class:
|
|
334
232
|
|
335
233
|
> **NOTE**: The process environment can only be modified if you pass the real,
|
336
234
|
> honest-to-goodness `ENV` object into `MyServiceClass.new(ENV)`. If you
|
337
|
-
> provide a copy
|
338
|
-
>
|
339
|
-
>
|
340
|
-
>
|
341
|
-
>
|
342
|
-
>
|
343
|
-
> (sensitive data blindly passed into subprocesses that you didn't expect).
|
235
|
+
> provide a copy, or some other hash, that'll work *normally*, but if you have
|
236
|
+
> sensitive variables, the service will log an error and refuse to start. This
|
237
|
+
> avoids the problems of accidentally modifying global state if that would be
|
238
|
+
> potentially bad (we assume you copied `ENV` for a reason) without leaving a
|
239
|
+
> gaping security hole (sensitive data blindly passed into subprocesses that
|
240
|
+
> you didn't expect).
|
344
241
|
|
345
242
|
|
346
|
-
###
|
243
|
+
### Custom Configuration Class
|
347
244
|
|
348
|
-
Whilst we hope that
|
245
|
+
Whilst we hope that `ServiceSkeleton::Config` class will be useful in most
|
349
246
|
situations, there are undoubtedly cases where the config management we provide
|
350
247
|
won't be enough. In that case, you are encouraged to subclass
|
351
248
|
`ServiceSkeleton::Config` and augment the standard interface with your own
|
@@ -361,16 +258,14 @@ class method in your service's class definition, like this:
|
|
361
258
|
end
|
362
259
|
end
|
363
260
|
|
364
|
-
class MyService
|
365
|
-
include ServiceSkeleton
|
366
|
-
|
261
|
+
class MyService < ServiceSkeleton
|
367
262
|
config_class MyServiceConfig
|
368
263
|
|
369
264
|
def run
|
370
265
|
loop do
|
371
|
-
|
372
|
-
|
373
|
-
|
266
|
+
puts config.something_funny
|
267
|
+
sleep 1
|
268
|
+
end
|
374
269
|
end
|
375
270
|
end
|
376
271
|
|
@@ -384,11 +279,11 @@ for you to use.
|
|
384
279
|
|
385
280
|
### What You Get
|
386
281
|
|
387
|
-
Every instance of your service class
|
388
|
-
`logger`. It is a (more-or-less) straight-up
|
389
|
-
`Logger`, on which you can call all the usual
|
390
|
-
`#warn`, `#error`, etc). By default, it sends all
|
391
|
-
error.
|
282
|
+
Every instance of your service class (as subclassed from `ServiceSkeleton`)
|
283
|
+
has a method named, uncreatively, `logger`. It is a (more-or-less) straight-up
|
284
|
+
instance of the Ruby stdlib `Logger`, on which you can call all the usual
|
285
|
+
methods (`#debug`, `#info`, `#warn`, `#error`, etc). By default, it sends all
|
286
|
+
log messages to standard error.
|
392
287
|
|
393
288
|
When calling the logger, you really, *really* want to use the
|
394
289
|
"progname+message-in-a-block" style of recording log messages, which looks like
|
@@ -404,22 +299,18 @@ wish to actively debug, based on log messages that are tagged with a specified
|
|
404
299
|
progname. No more grovelling through thousands of lines of debug logging to
|
405
300
|
find the One Useful Message.
|
406
301
|
|
407
|
-
|
302
|
+
The `ServiceSkeleton` also provides built-in dynamic log level adjustment;
|
408
303
|
using Unix signals or the admin HTTP interface (if enabled), you can tell the
|
409
304
|
logger to increase or decrease logging verbosity *without interrupting
|
410
305
|
service*. We are truly living in the future.
|
411
306
|
|
412
|
-
Finally, if you're a devotee of the ELK stack, the logger can automagically
|
413
|
-
send log entries straight into logstash, rather than you having to do it in
|
414
|
-
some more roundabout fashion.
|
415
|
-
|
416
307
|
|
417
308
|
### Logging Configuration
|
418
309
|
|
419
310
|
The logger automatically sets its configuration from, you guessed it, the
|
420
311
|
environment. The following environment variables are recognised by the logger.
|
421
|
-
All
|
422
|
-
|
312
|
+
All are all-uppercase, and the `<SERVICENAME>_` portion is the all-uppercase
|
313
|
+
[service name](#the-service-name).
|
423
314
|
|
424
315
|
* **`<SERVICENAME>_LOG_LEVEL`** (default: `"INFO"`) -- the minimum severity of
|
425
316
|
log messages which will be emitted by the logger.
|
@@ -430,36 +321,27 @@ portion is the all-uppercase [service name](#the-service-name).
|
|
430
321
|
|
431
322
|
If you wish to change the severity level for a single progname, you can
|
432
323
|
override the default log level for messages with a specific progname, by
|
433
|
-
specifying one or more "progname
|
434
|
-
|
435
|
-
|
324
|
+
specifying one or more "progname severities" separated by commas. A progname
|
325
|
+
severity looks like this:
|
326
|
+
|
436
327
|
<progname>=<severity>
|
437
328
|
|
438
329
|
To make things even more fun, if `<progname>` looks like a regular expression
|
439
330
|
(starts with `/` or `%r{`, and ends with `/` or `}` plus optional flag
|
440
331
|
characters), then all log messages with prognames *matching* the specified
|
441
|
-
|
332
|
+
regexp will have that severity applied. First match wins. The default is
|
442
333
|
still specified as a bare severity name, and the default can only be set
|
443
334
|
once.
|
444
335
|
|
445
|
-
That's a lot to take in, so here's an example which sets the default to
|
446
|
-
|
447
|
-
|
336
|
+
That's a lot to take in, so here's an example which sets the default to `INFO`,
|
337
|
+
debugs the `buggy` progname, and only emits errors for messages with the
|
338
|
+
(case-insensitive) string `noisy` in their progname:
|
448
339
|
|
449
340
|
INFO,buggy=DEBUG,/noisy/i=ERROR
|
450
341
|
|
451
342
|
Logging levels can be changed at runtime, via [signals](#default-signals) or
|
452
343
|
[the HTTP admin interface](#http-admin-interface).
|
453
344
|
|
454
|
-
* **`<SERVICENAME>_LOGSTASH_SERVER`** (string; default `""`) -- if set to a
|
455
|
-
non-empty string, we will engage the services of the [loggerstash
|
456
|
-
gem](https://github.com/discourse/loggerstash) on your behalf to send all log
|
457
|
-
entries to the logstash server you specify (as [an `address:port`,
|
458
|
-
`hostname:port`, or SRV
|
459
|
-
record](https://github.com/discourse/logstash_writer#usage). Just be sure
|
460
|
-
and [configure logstash
|
461
|
-
appropriately](https://github.com/discourse/loggerstash#logstash-configuration).
|
462
|
-
|
463
345
|
* **`<SERVICENAME>_LOG_ENABLE_TIMESTAMPS`** (boolean; default: `"no"`) -- if
|
464
346
|
set to a true-ish value (`yes`/`y`/`on`/`true`/`1`), then the log entries
|
465
347
|
emitted by the logger will have the current time (to the nearest nanosecond)
|
@@ -470,8 +352,8 @@ portion is the all-uppercase [service name](#the-service-name).
|
|
470
352
|
where log messages aren't automatically timestamped, then you can use this to
|
471
353
|
get them back.
|
472
354
|
|
473
|
-
* **`<SERVICENAME>_LOG_FILE`** (string; default: `"/dev/stderr
|
474
|
-
|
355
|
+
* **`<SERVICENAME>_LOG_FILE`** (string; default: `"/dev/stderr`) -- the file to
|
356
|
+
which log messages are written. The default, to send messages to standard
|
475
357
|
error, is a good choice if you are using a supervisor system which captures
|
476
358
|
service output to its own logging system, however if you are stuck without
|
477
359
|
such niceties, you can specify a file on disk to log to instead.
|
@@ -506,7 +388,7 @@ portion is the all-uppercase [service name](#the-service-name).
|
|
506
388
|
## Metrics
|
507
389
|
|
508
390
|
Running a service without metrics is like trying to fly a fighter jet whilst
|
509
|
-
blindfolded
|
391
|
+
blindfolded. Everything seems to be going OK until you slam into the side of a
|
510
392
|
mountain you never saw coming. For that reason, `ServiceSkeleton` provides a
|
511
393
|
Prometheus-based metrics registry, a bunch of default process-level metrics, an
|
512
394
|
optional HTTP metrics server, and simple integration with [the Prometheus ruby
|
@@ -518,43 +400,45 @@ easy as possible to instrument the heck out of your service.
|
|
518
400
|
### Defining and Using Metrics
|
519
401
|
|
520
402
|
All the metrics you want to use within your service need to be registered
|
521
|
-
before use. This is done
|
522
|
-
|
403
|
+
before use. This is typically done in the `#run` method, before entering the
|
404
|
+
infinite loop.
|
523
405
|
|
524
406
|
To register a metric, use one of the standard metric registration methods from
|
525
407
|
[Prometheus::Client::Registry](https://www.rubydoc.info/gems/prometheus-client/0.8.0/Prometheus/Client/Registry)
|
526
|
-
(
|
527
|
-
to
|
528
|
-
definition to register the metric for use.
|
408
|
+
(`#counter`, `gauge`, `histogram`, `summary`, or `register`) on the `metrics`
|
409
|
+
object to create or register the metric.
|
529
410
|
|
530
411
|
In our generic greeter service we've been using as an example so far, you might
|
531
412
|
like to define a metric to count how many greetings have been sent. You'd define
|
532
413
|
such a metric like this:
|
533
414
|
|
534
|
-
class GenericHelloService
|
535
|
-
|
536
|
-
|
537
|
-
string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
|
538
|
-
|
539
|
-
counter :greetings_total, docstring: "How many greetings we have sent", labels: %i{recipient}
|
415
|
+
class GenericHelloService < ServiceSkeleton
|
416
|
+
string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
|
540
417
|
|
541
|
-
|
542
|
-
|
543
|
-
When it comes time to actually *use* the metrics you have created, you access
|
544
|
-
them as methods on the `metrics` method in your service worker instance. Thus,
|
545
|
-
to increment our greeting counter, you simply do:
|
418
|
+
def run
|
419
|
+
metrics.counter(:greetings_total, "How many greetings we have sent")
|
546
420
|
|
547
|
-
|
548
|
-
|
421
|
+
loop do
|
422
|
+
puts "Hello, #{config.recipient}!"
|
423
|
+
sleep 1
|
424
|
+
end
|
425
|
+
end
|
426
|
+
end
|
549
427
|
|
550
|
-
|
428
|
+
When it comes time to actually *use* the metrics you have created, it's typical
|
429
|
+
to keep a copy of the metric object laying around, or call `metrics.get`. However,
|
430
|
+
we make it easier to access your metrics, by defining a method named for the metric
|
431
|
+
on `metrics`. Thus, to increment our greeting counter, you can simply do:
|
551
432
|
|
552
|
-
|
433
|
+
class GenericHelloService < ServiceSkeleton
|
434
|
+
string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
|
553
435
|
|
554
436
|
def run
|
437
|
+
metrics.counter(:greetings_total, "How many greetings we have sent")
|
438
|
+
|
555
439
|
loop do
|
556
440
|
puts "Hello, #{config.recipient}!"
|
557
|
-
|
441
|
+
metrics.greetings_total.increment(recipient: config.recipient)
|
558
442
|
sleep 1
|
559
443
|
end
|
560
444
|
end
|
@@ -565,17 +449,15 @@ any metrics you define which have the [service name](#the-service-name) as a
|
|
565
449
|
prefix will have that prefix (and the immediately-subsequent underscore) removed
|
566
450
|
before defining the metric accessor method, which keeps typing to a minimum:
|
567
451
|
|
568
|
-
class GenericHelloService
|
569
|
-
|
570
|
-
|
571
|
-
string :GENERIC_HELLO_SERVICE_RECIPIENT, match: /\A\w+\z/
|
572
|
-
|
573
|
-
counter :generic_hello_service_greetings_total, docstring: "How many greetings we have sent", labels: %i{recipient}
|
452
|
+
class GenericHelloService < ServiceSkeleton
|
453
|
+
string :GENERIC_HELLO_SERVICE_RECIPIENT, matches: /\A\w+\z/
|
574
454
|
|
575
455
|
def run
|
456
|
+
metrics.counter(:generic_hello_service_greetings_total, "How many greetings we have sent")
|
457
|
+
|
576
458
|
loop do
|
577
459
|
puts "Hello, #{config.recipient}!"
|
578
|
-
|
460
|
+
metrics.greetings_total.increment(recipient: config.recipient)
|
579
461
|
sleep 1
|
580
462
|
end
|
581
463
|
end
|
@@ -594,6 +476,9 @@ metrics](https://www.rubydoc.info/gems/frankenstein/Frankenstein/RubyGCMetrics),
|
|
594
476
|
and [Ruby VM
|
595
477
|
metrics](https://www.rubydoc.info/gems/frankenstein/Frankenstein/RubyVMMetrics).
|
596
478
|
|
479
|
+
If, for some reason, you *don't* want any default metrics in your bundle of
|
480
|
+
metrics, call `no_default_metrics` in your service class definition.
|
481
|
+
|
597
482
|
|
598
483
|
### Metrics Server Configuration
|
599
484
|
|
@@ -604,7 +489,7 @@ all-uppercase, and the `<SERVICENAME>_` portion is the all-uppercase version
|
|
604
489
|
of [the service name](#the-service-name).
|
605
490
|
|
606
491
|
* **`<SERVICENAME>_METRICS_PORT`** (integer; range 1..65535; default: `""`) --
|
607
|
-
if set to
|
492
|
+
if set to a non-empty integer which is a valid port number (`1` to `65535`,
|
608
493
|
inclusive), an HTTP server will be started which will respond to a request to
|
609
494
|
`/metrics` with a Prometheus-compatible dump of time series data.
|
610
495
|
|
@@ -622,15 +507,14 @@ behaviours for common signals.
|
|
622
507
|
|
623
508
|
### Default Signals
|
624
509
|
|
625
|
-
|
626
|
-
following signals will be hooked
|
627
|
-
|
510
|
+
Unless told otherwise, by calling the `no_default_signals` class method in your
|
511
|
+
service class' definition, the following signals will be hooked with the following
|
512
|
+
behaviour:
|
628
513
|
|
629
514
|
* **`SIGUSR1`** -- increase the default minimum severity for messages which
|
630
|
-
will be emitted by the logger
|
631
|
-
|
632
|
-
|
633
|
-
Configuration](#logging-configuration)).
|
515
|
+
will be emitted by the logger. The default severity only applies to log
|
516
|
+
messages whose progname does not match a "progname specifier" (see "[Logging
|
517
|
+
Configuration](#logging-configuration)").
|
634
518
|
|
635
519
|
* **`SIGUSR2`** -- decrease the default minimum severity for messages which
|
636
520
|
will be emitted by the logger.
|
@@ -638,7 +522,7 @@ that signal is received:
|
|
638
522
|
* **`SIGHUP`** -- close and reopen the log file, if logging to a file on disk.
|
639
523
|
Because of the `ServiceSkeleton`'s default log rotation policy, this shouldn't
|
640
524
|
ordinarily be required, but if you've turned off the default log rotation,
|
641
|
-
you may need
|
525
|
+
you may need tis.
|
642
526
|
|
643
527
|
* **`SIGQUIT`** -- dump a *whooooooole* lot of debugging information to
|
644
528
|
standard error, including memory allocation summaries and stack traces of all
|
@@ -648,43 +532,38 @@ that signal is received:
|
|
648
532
|
|
649
533
|
* **`SIGINT`** / **`SIGTERM`** -- ask the service to gracefully stop running.
|
650
534
|
It will call your service's `#shutdown` method to ask it to stop what it's
|
651
|
-
doing and exit. If the signal is sent
|
652
|
-
summarily terminated
|
653
|
-
|
654
|
-
|
655
|
-
want to use.
|
535
|
+
doing and exit. If the signal is sent twice, your run method will be
|
536
|
+
summarily terminated and everything will be terminated quickly. As usual, if
|
537
|
+
a service process needs to be whacked completely and utterly *right now*,
|
538
|
+
`SIGKILL` is what you want to use.
|
656
539
|
|
657
540
|
|
658
541
|
### Hooking Signals
|
659
542
|
|
660
543
|
In addition to the above default signal dispositions, you can also hook signals
|
661
544
|
yourself for whatever purpose you desire. This is typically done in your
|
662
|
-
`#run` method, before entering the
|
545
|
+
`#run` method, before entering the infinite loop.
|
663
546
|
|
664
547
|
To hook a signal, just call `hook_signal` with a signal specification and a
|
665
|
-
block of code to execute when the signal fires
|
666
|
-
|
667
|
-
|
668
|
-
|
669
|
-
something like this:
|
670
|
-
|
671
|
-
class MyService
|
672
|
-
include ServiceSkeleton
|
673
|
-
|
674
|
-
hook_signal("CONT") { puts "oof!" }
|
548
|
+
block of code to execute when the signal fires. You can even hook the same
|
549
|
+
signal more than once, because the signal handlers that `SkeletonService` uses
|
550
|
+
chain to other signal handlers. As an example, if you want to print "oof!"
|
551
|
+
every time the `SIGCONT` signal is received, you'd do something like this:
|
675
552
|
|
553
|
+
class MyService < ServiceSkeleton
|
676
554
|
def run
|
555
|
+
hook_signal("CONT") { puts "oof!" }
|
556
|
+
|
677
557
|
loop { sleep }
|
678
558
|
end
|
679
559
|
end
|
680
560
|
|
681
|
-
The code in the block will be executed in the context of the service worker
|
682
|
-
instance that is running at the time the signal is received. You are
|
683
|
-
responsible for ensuring that whatever your handler does is concurrency-safe.
|
684
|
-
|
685
561
|
When the service is shutdown, all signal handlers will be automatically
|
686
562
|
unhooked, which saves you having to do it yourself.
|
687
563
|
|
564
|
+
> **NOTE**: You can define a maximum of 256 signal hooks in a single service,
|
565
|
+
> including the default signal hooks.
|
566
|
+
|
688
567
|
|
689
568
|
## HTTP Admin Interface
|
690
569
|
|
@@ -696,9 +575,9 @@ different?
|
|
696
575
|
### HTTP Admin Configuration
|
697
576
|
|
698
577
|
In the spirit of "secure by default", you must explicitly enable the HTTP admin
|
699
|
-
interface, and
|
700
|
-
|
701
|
-
|
578
|
+
interface, and it requires authentication. To do that, use the following
|
579
|
+
environment variables, where `<SERVICENAME>_` is the all-uppercase version of
|
580
|
+
[the service name](#the-service-name).
|
702
581
|
|
703
582
|
* **`<SERVICENAME>_HTTP_ADMIN_PORT`** (integer; range 1..65535; default: `""`)
|
704
583
|
-- if set to a valid port number (`1` to `65535` inclusive), the HTTP admin
|
@@ -722,11 +601,7 @@ version of [the service name](#the-service-name).
|
|
722
601
|
interface to be enabled.
|
723
602
|
|
724
603
|
|
725
|
-
### HTTP Admin
|
726
|
-
|
727
|
-
The HTTP admin interface provides both an interactive, browser-based mode,
|
728
|
-
as well as a RESTful interface, which should, in general, provide equivalent
|
729
|
-
functionality.
|
604
|
+
### HTTP Admin Features
|
730
605
|
|
731
606
|
* Visiting the service's `IP address:port` in a web browser will bring up an HTML
|
732
607
|
interface showing all the features that are available. Usage should
|
@@ -751,8 +626,7 @@ conduct](CODE_OF_CONDUCT.md).
|
|
751
626
|
Unless otherwise stated, everything in this repo is covered by the following
|
752
627
|
copyright notice:
|
753
628
|
|
754
|
-
Copyright (C) 2018
|
755
|
-
Copyright (C) 2019, 2020 Matt Palmer
|
629
|
+
Copyright (C) 2018 Civilized Discourse Construction Kit, Inc.
|
756
630
|
|
757
631
|
This program is free software: you can redistribute it and/or modify it
|
758
632
|
under the terms of the GNU General Public License version 3, as
|