semantic_logger 5.0.0 → 5.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +9 -0
  3. data/Rakefile +29 -0
  4. data/docs/api.md +480 -0
  5. data/docs/appenders.md +990 -0
  6. data/docs/config.md +605 -0
  7. data/docs/index.md +197 -0
  8. data/docs/log.md +108 -0
  9. data/docs/metrics.md +168 -0
  10. data/docs/operations.md +400 -0
  11. data/docs/rails.md +816 -0
  12. data/docs/security.md +119 -0
  13. data/docs/testing.md +285 -0
  14. data/docs/upgrading.md +409 -0
  15. data/lib/semantic_logger/appender/cloudwatch_logs.rb +4 -2
  16. data/lib/semantic_logger/appender/elasticsearch.rb +2 -5
  17. data/lib/semantic_logger/appender/elasticsearch_base.rb +17 -17
  18. data/lib/semantic_logger/appender/elasticsearch_http.rb +14 -7
  19. data/lib/semantic_logger/appender/file.rb +5 -0
  20. data/lib/semantic_logger/appender/new_relic.rb +6 -0
  21. data/lib/semantic_logger/appender/new_relic_logs.rb +2 -1
  22. data/lib/semantic_logger/appender/open_telemetry.rb +2 -1
  23. data/lib/semantic_logger/appender/opensearch.rb +2 -1
  24. data/lib/semantic_logger/appender/rabbitmq.rb +4 -3
  25. data/lib/semantic_logger/appender/sentry.rb +5 -0
  26. data/lib/semantic_logger/appender/syslog.rb +6 -3
  27. data/lib/semantic_logger/appender/tcp.rb +4 -3
  28. data/lib/semantic_logger/appender/wrapper.rb +2 -1
  29. data/lib/semantic_logger/appender.rb +6 -3
  30. data/lib/semantic_logger/appenders.rb +2 -1
  31. data/lib/semantic_logger/base.rb +0 -2
  32. data/lib/semantic_logger/formatters/color.rb +5 -2
  33. data/lib/semantic_logger/formatters/default.rb +7 -2
  34. data/lib/semantic_logger/formatters/logfmt.rb +4 -2
  35. data/lib/semantic_logger/formatters/pattern.rb +3 -1
  36. data/lib/semantic_logger/formatters/syslog.rb +2 -1
  37. data/lib/semantic_logger/formatters/syslog_cee.rb +2 -1
  38. data/lib/semantic_logger/metric/signalfx.rb +4 -2
  39. data/lib/semantic_logger/queue_processor.rb +5 -2
  40. data/lib/semantic_logger/reporters/minitest.rb +4 -4
  41. data/lib/semantic_logger/test/capture_log_events.rb +1 -1
  42. data/lib/semantic_logger/version.rb +1 -1
  43. metadata +14 -3
data/docs/rails.md ADDED
@@ -0,0 +1,816 @@
1
+ ---
2
+ layout: default
3
+ ---
4
+
5
+ ## Rails
6
+ {:.no_toc}
7
+
8
+ **Contents**
9
+
10
+ * TOC
11
+ {:toc}
12
+
13
+ [rails_semantic_logger](https://github.com/reidmorrison/rails_semantic_logger) is a companion gem
14
+ that wires Semantic Logger into Rails for you. Once installed it:
15
+
16
+ * Replaces the default Rails logger with Semantic Logger, so Rails, your application code, and many
17
+ common gems all log through it.
18
+ * Collapses the several lines Rails normally logs per request into a single, structured "Completed"
19
+ line, while keeping the individual fields (controller, action, status, durations, and so on)
20
+ searchable.
21
+ * Lets you send logs anywhere Semantic Logger supports: the standard Rails log file, standard out as
22
+ JSON for a container platform, a centralized log service, or several of these at once.
23
+
24
+ This page is a step-by-step guide that assumes no prior knowledge of either gem. Work through it
25
+ top to bottom: each section builds on the previous one. For the underlying logging API (logging
26
+ methods, tags, metrics, and so on) see the [Programmer's Guide](api.html), and for the full catalog
27
+ of log destinations see [Appenders](appenders.html).
28
+
29
+ > **Upgrading from v4?** The way appenders (log destinations) are configured changed in v5. Jump to
30
+ > [Migrating from v4 to v5](#migrating-from-v4-to-v5), then come back here.
31
+
32
+ ### Requirements
33
+
34
+ Rails Semantic Logger v5 requires **Ruby 3.2 or later** and **Rails 7.2 or later**. For the exact
35
+ list of tested Ruby and Rails versions, see the
36
+ [CI workflow](https://github.com/reidmorrison/rails_semantic_logger/blob/main/.github/workflows/ci.yml).
37
+
38
+ ### Installation
39
+
40
+ Add the following lines to your `Gemfile`:
41
+
42
+ ~~~ruby
43
+ gem "rails_semantic_logger"
44
+ gem "amazing_print" # optional
45
+ ~~~
46
+
47
+ `amazing_print` is optional but recommended: it produces colorized, readable output of the
48
+ structured data (the Hash payload) in development.
49
+
50
+ Install with bundler:
51
+
52
+ bundle install
53
+
54
+ That is all that is required. Rails Semantic Logger automatically replaces the standard Rails logger
55
+ with Semantic Logger and writes to the usual Rails log file.
56
+
57
+ #### Remove conflicting gems
58
+
59
+ Remove the following gems if present. They conflict with or duplicate what Rails Semantic Logger
60
+ already does:
61
+
62
+ * `lograge`
63
+ * `rails_stdout_logging`
64
+ * `rails_12factor`
65
+
66
+ ### Out of the box
67
+
68
+ With no configuration at all, Rails Semantic Logger:
69
+
70
+ * Writes to `log/<environment>.log` (for example `log/development.log`), the same file Rails uses.
71
+ * Colorizes that output when Rails colorized logging is enabled (the default in development).
72
+ * Logs to **standard out** when you run `rails server`, so you see requests in your terminal.
73
+ * Logs to **standard error** when you run `rails console`, so log lines do not get mixed up with
74
+ the return values of the commands you type.
75
+ * Replaces the multi-line Rails request log with a single structured "Completed" line.
76
+
77
+ Standard Rails log output for a single page request:
78
+
79
+ ![Rails Default](images/rails_default.png)
80
+
81
+ The same request after adding the `rails_semantic_logger` gem:
82
+
83
+ ![Rails Single Line](images/rails_single_line.png)
84
+
85
+ The rest of this page shows how to change **where** logs go and **how** they are formatted (the
86
+ appenders block), and then how to fine-tune **what** Rails logs.
87
+
88
+ Configuration goes in `config/application.rb` (for all environments) or in an environment file under
89
+ `config/environments/` (for one environment).
90
+
91
+ Use those two places, not `config/initializers/*`. Rails builds the logger **before** it loads
92
+ initializers, so the appenders block and the `semantic`, `replace_sidekiq_logger`, and
93
+ `replace_solid_queue_logger` options have no effect from an initializer; Rails Semantic Logger
94
+ prints a warning when it detects this. The output-tuning options consumed later in boot (`started`,
95
+ `processing`, `rendered`, `quiet_assets`, `action_message_format`) do still work from an
96
+ initializer, but not once the application has finished booting.
97
+
98
+ ---
99
+
100
+ ## Configuring where logs go: the appenders block
101
+
102
+ An **appender** is a destination for log output: a file, standard out, a centralized log service,
103
+ and so on. You declare the appenders you want inside a single block:
104
+
105
+ ~~~ruby
106
+ config.rails_semantic_logger.appenders do |appenders|
107
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :color)
108
+ end
109
+ ~~~
110
+
111
+ There are three ways to declare an appender. **The method name says _when_ the appender is created;
112
+ the arguments say _where_ it writes and _how_ it is formatted.**
113
+
114
+ | Method | Created when… | Default destination |
115
+ |--------|---------------|---------------------|
116
+ | `add` | Always, during Rails initialization | (you must specify one) |
117
+ | `add_server` | Only when serving requests: `rails server` (see the note under [Step 3](#step-3-log-to-the-screen-only-while-serving)) and Sidekiq in server mode | `$stdout` |
118
+ | `add_console` | Only inside a `rails console` session | `$stderr` |
119
+
120
+ The arguments to all three are exactly the arguments to `SemanticLogger.add_appender` (covered in
121
+ detail in [the next section](#appender-options-and-destinations)), so anything Semantic Logger can
122
+ log to, any of these can declare. One default of note: since `add_server` and `add_console` are
123
+ screen appenders, their `formatter:` defaults to `:color` when not specified; `add` uses the
124
+ Semantic Logger default of plain text.
125
+
126
+ > **Important:** As soon as you declare **any** appender in this block, Rails Semantic Logger stops
127
+ > adding **all** of its automatic appenders: the default `log/<env>.log` file, the standard-out
128
+ > logger it normally adds under `rails server`, and the standard-error logger it normally adds in
129
+ > `rails console`. The block becomes the single source of truth for every destination. So declare
130
+ > what you want: `add` for an always-on destination (such as the file log), `add_server` for screen
131
+ > output while serving, and `add_console` for the Rails console.
132
+
133
+ ### Step 1: a single log file
134
+
135
+ ~~~ruby
136
+ config.rails_semantic_logger.appenders do |appenders|
137
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :color)
138
+ end
139
+ ~~~
140
+
141
+ ### Step 2: add a JSON file for a log aggregator
142
+
143
+ Keep the human-readable color log and *also* write a JSON file for ingestion by Elasticsearch,
144
+ Splunk, Datadog, and the like:
145
+
146
+ ~~~ruby
147
+ config.rails_semantic_logger.appenders do |appenders|
148
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :color)
149
+ appenders.add(file_name: "log/#{Rails.env}.json", formatter: :json)
150
+ end
151
+ ~~~
152
+
153
+ You can declare as many appenders as you like; every log entry is sent to all of them.
154
+
155
+ ### Step 3: log to the screen only while serving
156
+
157
+ `add_server` declares an appender that is created **only** when the application is actually serving
158
+ requests (under `rails server` or Sidekiq in server mode), and never during rake tasks, runners, or
159
+ generators. It defaults to `$stdout`:
160
+
161
+ ~~~ruby
162
+ config.rails_semantic_logger.appenders do |appenders|
163
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :json)
164
+ appenders.add_server(formatter: :color) # → $stdout, only when serving
165
+ end
166
+ ~~~
167
+
168
+ > **Note:** Under `rails server`, the appender is created when Rails itself would log to standard
169
+ > out: in development, when not daemonized. To get it in another environment, pass
170
+ > `--log-to-stdout` to `rails server`. App servers started directly (bare `puma`, `rackup`, and so
171
+ > on) need a one-line boot hook; see
172
+ > [Other app servers](#other-app-servers-puma-rackup-passenger-unicorn).
173
+
174
+ ### Step 4: a dedicated console logger
175
+
176
+ `add_console` declares an appender created **only** inside a `rails console` session. It defaults to
177
+ `$stderr` so log output does not interleave with the results of the expressions you type:
178
+
179
+ ~~~ruby
180
+ config.rails_semantic_logger.appenders do |appenders|
181
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :json)
182
+ appenders.add_server(formatter: :color) # $stdout while serving
183
+ appenders.add_console(formatter: :color) # $stderr inside `rails console`
184
+ end
185
+ ~~~
186
+
187
+ ### Several appenders in one context
188
+
189
+ Because each call simply appends to its context, a context can have more than one appender. For
190
+ example, write a color stream *and* a JSON file, but only while serving:
191
+
192
+ ~~~ruby
193
+ config.rails_semantic_logger.appenders do |appenders|
194
+ appenders.add_server(io: $stdout, formatter: :color)
195
+ appenders.add_server(file_name: "log/#{Rails.env}.log", formatter: :json)
196
+ end
197
+ ~~~
198
+
199
+ ---
200
+
201
+ ## Appender options and destinations
202
+
203
+ Every `add`, `add_server`, and `add_console` call accepts the same arguments as
204
+ `SemanticLogger.add_appender`. This section is the one-stop reference for those arguments, written as
205
+ Rails examples. For the complete list of destinations and their service-specific options, see
206
+ [Appenders](appenders.html).
207
+
208
+ ### Common options
209
+
210
+ In addition to a destination, most appenders accept these options:
211
+
212
+ | Option | Description |
213
+ |--------|-------------|
214
+ | `level` | Only write entries at this level or higher to this appender. Defaults to `SemanticLogger.default_level` (which Rails Semantic Logger sets from `config.log_level`). |
215
+ | `formatter` | How to format the output: `:default`, `:color`, `:json`, `:logfmt`, `:one_line`, or a custom formatter (see [Output formats](#output-formats)). |
216
+ | `filter` | A `Regexp` or `Proc` selecting which entries this appender accepts. See [Filtering](config.html#filtering). |
217
+ | `application`, `environment`, `host` | Override the global values for this appender only. |
218
+
219
+ For example, send only warnings and above to a separate JSON file:
220
+
221
+ ~~~ruby
222
+ config.rails_semantic_logger.appenders do |appenders|
223
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :color)
224
+ appenders.add(file_name: "log/#{Rails.env}_errors.json", formatter: :json, level: :warn)
225
+ end
226
+ ~~~
227
+
228
+ ### Destinations
229
+
230
+ The destination is chosen by the argument you pass:
231
+
232
+ | Destination | Argument | Notes |
233
+ |-------------|----------|-------|
234
+ | Text or JSON file | `file_name:` | A path under `log/`. |
235
+ | IO stream | `io:` | `$stdout`, `$stderr`, or any `IO`. |
236
+ | Built-in appender | `appender: :name` | Selects a packaged appender by name (syslog, elasticsearch, http, bugsnag, and many more). |
237
+ | Existing Ruby/Rails logger | `logger:` | Wrap another logger instance. |
238
+ | Metrics destination | `metric:` | See [Metrics](metrics.html). |
239
+
240
+ A few common examples, all inside the appenders block:
241
+
242
+ ~~~ruby
243
+ config.rails_semantic_logger.appenders do |appenders|
244
+ # Local file
245
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :color)
246
+
247
+ # Local Syslog
248
+ appenders.add(appender: :syslog)
249
+
250
+ # Remote syslog such as syslog-ng over TCP
251
+ appenders.add(appender: :syslog, url: "tcp://myloghost:514")
252
+
253
+ # Elasticsearch
254
+ appenders.add(appender: :elasticsearch, url: "http://localhost:9200")
255
+
256
+ # A generic HTTP(S) endpoint
257
+ appenders.add(appender: :http, url: "https://logs.example.com/ingest")
258
+
259
+ # Bugsnag (errors and above)
260
+ appenders.add(appender: :bugsnag, level: :error)
261
+ end
262
+ ~~~
263
+
264
+ Appenders for third-party services require their backing gem to be installed. See
265
+ [Appenders](appenders.html) for the full list of destinations, their gems, and their options.
266
+
267
+ ### Output formats
268
+
269
+ The `formatter:` option controls how each appender renders a log entry. Because it is per appender,
270
+ you can write color to the screen and JSON to a file at the same time.
271
+
272
+ | Formatter | Output |
273
+ |-----------|--------|
274
+ | `:default` | Plain text, no color. |
275
+ | `:color` | Plain text with color (uses Amazing Print for the payload when installed). |
276
+ | `:json` | One JSON object per entry. |
277
+ | `:logfmt` | `key=value` logfmt output. |
278
+ | `:one_line` | Each entry reduced to a single line. |
279
+ | A class instance | Any instance of a class derived from `SemanticLogger::Formatters::Base`. |
280
+ | A `Proc` | Called with the `log` entry; returns the formatted output. |
281
+
282
+ JSON example:
283
+
284
+ ~~~ruby
285
+ config.rails_semantic_logger.appenders do |appenders|
286
+ appenders.add(file_name: "log/#{Rails.env}.json", formatter: :json)
287
+ end
288
+ ~~~
289
+
290
+ Custom formatter. Create `app/lib/my_formatter.rb`:
291
+
292
+ ~~~ruby
293
+ # A custom colorized formatter
294
+ class MyFormatter < SemanticLogger::Formatters::Color
295
+ # Return the complete log level name in uppercase
296
+ def level
297
+ "#{color}#{log.level.upcase}#{color_map.clear}"
298
+ end
299
+ end
300
+ ~~~
301
+
302
+ Then use it:
303
+
304
+ ~~~ruby
305
+ config.rails_semantic_logger.appenders do |appenders|
306
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: MyFormatter.new)
307
+ end
308
+ ~~~
309
+
310
+ See [SemanticLogger::Formatters::Color](https://github.com/reidmorrison/semantic_logger/blob/main/lib/semantic_logger/formatters/color.rb)
311
+ for the methods you can override, and [Custom formatters](config.html#custom-formatters) for more on formatters.
312
+
313
+ #### Amazing Print options for the color formatter
314
+
315
+ The color formatter renders the payload Hash with Amazing Print. To pass options to it, give the
316
+ `:color` formatter a Hash:
317
+
318
+ ~~~ruby
319
+ config.rails_semantic_logger.appenders do |appenders|
320
+ appenders.add(
321
+ file_name: "log/#{Rails.env}.log",
322
+ formatter: {color: {ap: {multiline: false}}}
323
+ )
324
+ end
325
+ ~~~
326
+
327
+ See the [Amazing Print documentation](https://github.com/amazing-print/amazing_print) for the
328
+ available options (or set defaults in a `~/.aprc` file). This has no effect if Amazing Print is not
329
+ installed.
330
+
331
+ ---
332
+
333
+ ## Common recipes
334
+
335
+ ### Development
336
+
337
+ Color to both the log file and the screen:
338
+
339
+ ~~~ruby
340
+ # config/environments/development.rb
341
+ config.rails_semantic_logger.appenders do |appenders|
342
+ appenders.add(file_name: "log/development.log", formatter: :color)
343
+ end
344
+ ~~~
345
+
346
+ (The default already does this, so in development you often need no configuration at all.)
347
+
348
+ ### Production on a container platform (Docker, Kubernetes, Heroku)
349
+
350
+ On a container platform the convention is to log JSON to standard out and let the platform collect
351
+ it. Use `add` (not `add_server`) so that rake tasks and one-off processes also log to stdout:
352
+
353
+ ~~~ruby
354
+ # config/environments/production.rb
355
+ config.rails_semantic_logger.appenders do |appenders|
356
+ appenders.add(io: $stdout, formatter: :json)
357
+ end
358
+ ~~~
359
+
360
+ Because the block disables the automatic appenders, this JSON-to-stdout appender is the *only*
361
+ destination: there is no default file log and no separate color logger under `rails server`, which
362
+ is exactly what a container platform wants.
363
+
364
+ To keep one configuration that works both locally and in-cluster, gate the JSON appender on an
365
+ environment variable that is only set in the container platform. For example, in
366
+ `config/application.rb` (so it applies to every environment), switch to JSON on standard out only
367
+ when running inside Kubernetes:
368
+
369
+ ~~~ruby
370
+ # config/application.rb
371
+ config.semantic_logger.application = "my_application"
372
+ config.semantic_logger.environment = ENV["STACK_NAME"] || Rails.env
373
+ config.log_level = ENV["LOG_LEVEL"] || :info
374
+
375
+ if ENV["LOG_TO_CONSOLE"] || ENV["KUBERNETES_SERVICE_HOST"]
376
+ config.rails_semantic_logger.appenders do |appenders|
377
+ appenders.add(io: $stdout, formatter: :json)
378
+ end
379
+ end
380
+ ~~~
381
+
382
+ Outside the cluster the block is skipped, so the default `log/<env>.log` file and the usual screen
383
+ loggers apply; inside the cluster JSON to stdout becomes the only destination.
384
+
385
+ On Heroku, also allow the log level to be set from the environment:
386
+
387
+ ~~~ruby
388
+ config.log_level = ENV["LOG_LEVEL"].presence&.downcase&.to_sym || :info
389
+ ~~~
390
+
391
+ `heroku config:set LOG_LEVEL=debug`
392
+
393
+ ### Production writing to files plus an error service
394
+
395
+ ~~~ruby
396
+ # config/environments/production.rb
397
+ config.rails_semantic_logger.appenders do |appenders|
398
+ appenders.add(file_name: "log/production.json", formatter: :json)
399
+ appenders.add(appender: :bugsnag, level: :error)
400
+ end
401
+ ~~~
402
+
403
+ ### Other app servers: puma, rackup, Passenger, Unicorn
404
+
405
+ `add_server` appenders are created automatically under `rails server` (following Rails' own
406
+ log-to-stdout rule: development and not daemonized, or the `--log-to-stdout` flag) and under Sidekiq
407
+ in server mode, because those have a definitive startup hook. App servers started **directly** (bare `puma`,
408
+ `rackup`, Passenger, Unicorn) have no such first-party hook, and Rails Semantic Logger deliberately
409
+ does **not** guess (a detection that only sometimes works is worse than none).
410
+
411
+ If you start your app with one of those servers and want your `add_server` appenders created, call
412
+ the helper from that server's own boot hook. For example, in `config/puma.rb`:
413
+
414
+ ~~~ruby
415
+ on_booted { RailsSemanticLogger.add_server_appenders }
416
+ ~~~
417
+
418
+ Alternatively, if you simply want a destination created in every context, declare it with `add`
419
+ instead of `add_server`.
420
+
421
+ ### Sidekiq
422
+
423
+ Sidekiq in server mode is treated as a serving context, so `add_server` appenders are created
424
+ automatically. No extra configuration is required.
425
+
426
+ This wiring is part of the Sidekiq logger integration, so setting
427
+ `config.rails_semantic_logger.replace_sidekiq_logger = false` also turns off the automatic creation.
428
+ In that case, call `RailsSemanticLogger.add_server_appenders` yourself from a Sidekiq startup hook:
429
+
430
+ ~~~ruby
431
+ # config/initializers/sidekiq.rb
432
+ Sidekiq.configure_server do |config|
433
+ config.on(:startup) { RailsSemanticLogger.add_server_appenders }
434
+ end
435
+ ~~~
436
+
437
+ ---
438
+
439
+ ## Tuning what Rails logs
440
+
441
+ The options below adjust Rails' own log output. They are independent of the appenders block and can
442
+ be combined with it.
443
+
444
+ ### Log level
445
+
446
+ ~~~ruby
447
+ # One of :trace, :debug, :info, :warn, :error, :fatal
448
+ config.log_level = :debug
449
+ ~~~
450
+
451
+ To change the level inside a running `rails console`:
452
+
453
+ ~~~ruby
454
+ SemanticLogger.default_level = :debug
455
+ ~~~
456
+
457
+ ### Re-enable Started, Processing, and Rendered messages
458
+
459
+ By default these messages are logged at `:debug` so they do not appear in production. To show them:
460
+
461
+ ~~~ruby
462
+ config.rails_semantic_logger.started = true # Rack "Started" line
463
+ config.rails_semantic_logger.processing = true # Controller "Processing" line
464
+ config.rails_semantic_logger.rendered = true # Action View render lines
465
+ ~~~
466
+
467
+ ![Semantic](images/rails_semantic.png)
468
+
469
+ ### Keep Rails' original wording
470
+
471
+ By default Action Controller and Active Record messages are converted to structured data:
472
+
473
+ ~~~
474
+ Rack -- Started -- { :ip => "127.0.0.1", :method => "GET", :path => "/users" }
475
+ UserController -- Completed #index -- { :action => "index", :db_runtime => 54.64, :format => "HTML", :method => "GET", :path => "/users", :status => 200, :status_message => "OK", :view_runtime => 709.88 }
476
+ ~~~
477
+
478
+ To keep Rails' original text messages (with Semantic Logger formatting) instead:
479
+
480
+ ~~~ruby
481
+ config.rails_semantic_logger.semantic = false
482
+ config.rails_semantic_logger.started = true
483
+ config.rails_semantic_logger.processing = true
484
+ config.rails_semantic_logger.rendered = true
485
+ ~~~
486
+
487
+ ![Semantic Disabled](images/rails_semantic_false.png)
488
+
489
+ ### Quiet asset logging
490
+
491
+ Rails logs asset requests at the debug level, which can clutter development logs:
492
+
493
+ ~~~
494
+ Rack -- Started -- {:ip => "127.0.0.1", :method => "GET", :path => "/assets/application.css"}
495
+ ~~~
496
+
497
+ To silence them:
498
+
499
+ ~~~ruby
500
+ config.rails_semantic_logger.quiet_assets = true
501
+ ~~~
502
+
503
+ ### Color output
504
+
505
+ Color is chosen per appender with the `formatter:` option: use `:color` for colorized output and
506
+ `:default` for plain text (see [Output formats](#output-formats)). For example, color on screen and
507
+ plain text in a file:
508
+
509
+ ~~~ruby
510
+ config.rails_semantic_logger.appenders do |appenders|
511
+ appenders.add(file_name: "log/#{Rails.env}.log", formatter: :default)
512
+ appenders.add_server(formatter: :color)
513
+ end
514
+ ~~~
515
+
516
+ The Rails `config.colorize_logging` setting does **not** affect appenders declared in the block; it
517
+ only influences the deprecated default file appender (see
518
+ [Deprecated configuration options](#deprecated-configuration-options)).
519
+
520
+ ### Named tags
521
+
522
+ Add tags to every log entry on a per-request basis by setting `config.log_tags` to a Hash:
523
+
524
+ ~~~ruby
525
+ config.log_tags = {
526
+ request_id: :request_id,
527
+ ip: :remote_ip,
528
+ user: ->(request) { request.cookie_jar["login"] }
529
+ }
530
+ ~~~
531
+
532
+ Notes:
533
+
534
+ * If a value returns `nil`, that key is omitted for that request.
535
+ * To turn named tags off in development, set `config.log_tags = nil` in
536
+ `config/environments/development.rb`.
537
+
538
+ ### Source file name and line number
539
+
540
+ To include the file and line number where each message originated:
541
+
542
+ ~~~ruby
543
+ config.semantic_logger.backtrace_level = :info
544
+ ~~~
545
+
546
+ ![with_source](images/rails_with_backtrace.png)
547
+
548
+ **Warning:** capturing a backtrace for every log entry allocates many objects. In production set this
549
+ to `nil` (disabled) or to a high level such as `:error`. By default backtraces are only captured for
550
+ `:error` and `:fatal`. This feature is best used in development.
551
+
552
+ ### Add custom data to the Completed message
553
+
554
+ Add an `append_info_to_payload` method to a controller to include extra fields in its Completed
555
+ message:
556
+
557
+ ~~~ruby
558
+ class ThingController < ApplicationController
559
+ private
560
+
561
+ def append_info_to_payload(payload)
562
+ super
563
+ payload[:user_id] = current_user&.id
564
+ end
565
+ end
566
+ ~~~
567
+
568
+ ### Customize the Completed message text
569
+
570
+ Provide a Proc to build the Action Controller message from the message and payload:
571
+
572
+ ~~~ruby
573
+ config.rails_semantic_logger.action_message_format = ->(message, payload) do
574
+ "#{message} - #{payload[:controller]}##{payload[:action]}"
575
+ end
576
+ ~~~
577
+
578
+ ### Background job loggers
579
+
580
+ By default Rails Semantic Logger replaces the Sidekiq and SolidQueue loggers. To leave them alone:
581
+
582
+ ~~~ruby
583
+ config.rails_semantic_logger.replace_sidekiq_logger = false
584
+ config.rails_semantic_logger.replace_solid_queue_logger = false
585
+ ~~~
586
+
587
+ Sidekiq v7 and v8 are supported. Earlier Sidekiq versions were supported up to
588
+ Rails Semantic Logger v5.0.
589
+
590
+ #### Sidekiq job lifecycle messages
591
+
592
+ For every Sidekiq job, Rails Semantic Logger emits a `Start #perform` and a `Completed #perform`
593
+ entry (with the `sidekiq.queue.latency` and `sidekiq.job.perform` metrics). On very high job volumes
594
+ these can add noise and cost in log aggregation tools. To turn them off (the job still runs and any
595
+ exceptions are still logged):
596
+
597
+ ~~~ruby
598
+ # config/initializers/sidekiq.rb
599
+ RailsSemanticLogger::Sidekiq::JobLogger.perform_messages = false
600
+ ~~~
601
+
602
+ This defaults to `true`, so the messages are emitted unless you opt out.
603
+
604
+ On Sidekiq 8, the standard Sidekiq setting has the same effect and is also honored:
605
+
606
+ ~~~ruby
607
+ Sidekiq.configure_server do |config|
608
+ config[:skip_default_job_logging] = true
609
+ end
610
+ ~~~
611
+
612
+ #### Sidekiq job logging context
613
+
614
+ Every log entry emitted while a job runs is tagged with the job's `jid`, class, and queue, plus its
615
+ `bid` (batch id) and `tags` when present. On Sidekiq 8, the standard `logged_job_attributes` setting
616
+ is honored, so additional job attributes can be added to the logging context:
617
+
618
+ ~~~ruby
619
+ Sidekiq.configure_server do |config|
620
+ config[:logged_job_attributes] = %w[bid tags priority]
621
+ end
622
+ ~~~
623
+
624
+ ### Custom controller base class
625
+
626
+ If your application uses a controller base class other than `ActionController::Base` or
627
+ `ActionController::API`, Rails Semantic Logger falls back to the `ActionController::Base` logger, so
628
+ those entries are named `ActionController::Base`. To give them the correct class name, include the
629
+ mixin in your base class:
630
+
631
+ ~~~ruby
632
+ include SemanticLogger::Loggable
633
+ ~~~
634
+
635
+ If the base class lives in a third-party gem, do it from an initializer:
636
+
637
+ ~~~ruby
638
+ CustomControllerBase.include(SemanticLogger::Loggable)
639
+ ~~~
640
+
641
+ ---
642
+
643
+ ## Metrics (prototype)
644
+
645
+ > **Prototype:** metrics support and the metric names below are an early prototype and **subject to
646
+ > change** in a future release. Do not hard-code these names into long-lived dashboards or alerts yet.
647
+
648
+ In addition to the structured log entry, Rails Semantic Logger attaches a Semantic Logger
649
+ [metric](metrics.html) to each entry that is logged at `:info`, `:warn`, or `:error`. When the entry
650
+ carries a duration (a request, query, render, job run, ...) the metric records that timing; otherwise
651
+ it acts as an event counter. Wire up a metrics appender (StatsD, Prometheus, ...) and these flow
652
+ through automatically alongside the logs.
653
+
654
+ Metric names follow `rails.<component>.<event>`, where `<component>` is the Rails component with its
655
+ `action_`/`active_` prefix dropped:
656
+
657
+ | Component | Metrics |
658
+ | --- | --- |
659
+ | Action Controller | `rails.controller.process_action`, `rails.controller.send_file`, `rails.controller.send_data`, `rails.controller.redirect_to`, `rails.controller.halted_callback`, `rails.controller.rescue_from_callback`, and the fragment-cache events (`rails.controller.write_fragment`, `read_fragment`, `exist_fragment`, `expire_fragment`, `expire_page`, `write_page`) |
660
+ | Action View | `rails.view.render.template`, `rails.view.render.partial`, `rails.view.render.layout`, `rails.view.render.collection` |
661
+ | Active Job | `rails.job.<event>` for every job event (`enqueue`, `enqueue_at`, `enqueue_all`, `perform_start`, `perform`, `enqueue_retry`, `retry_stopped`, `discard`, and the Rails 8.1 Continuation events `interrupt`, `resume`, `step_skipped`, `step_started`, `step`) |
662
+ | Action Mailer | `rails.mailer.deliver` |
663
+ | Solid Queue | `rails.solid_queue.<event>` for every Solid Queue event logged at info/warn/error (e.g. `rails.solid_queue.start_process`, `rails.solid_queue.thread_error`) |
664
+
665
+ Entries logged at `:debug` carry **no** metric. Because ActiveRecord logs SQL at `:debug`, there is
666
+ currently no `rails.db`/`rails.record` SQL metric, and Action View render metrics are only emitted
667
+ when rendered events are raised to `:info` (see
668
+ [Re-enable Started, Processing, and Rendered messages](#re-enable-started-processing-and-rendered-messages)).
669
+
670
+ ---
671
+
672
+ ## Operational notes
673
+
674
+ ### Process forking
675
+
676
+ If you use a forking server (Puma, Unicorn) or fork worker processes, see
677
+ [Process Forking](operations.html#process-forking). With Semantic Logger v5 appenders are reopened automatically after
678
+ a fork, so the manual `after_fork { SemanticLogger.reopen }` hook is usually no longer needed.
679
+
680
+ ### Log rotation
681
+
682
+ Because the log file is held open between writes, rotate it with a **copy-truncate** strategy rather
683
+ than deleting and recreating the file. Example `logrotate` configuration for Linux:
684
+
685
+ ~~~
686
+ /var/www/my_app/shared/log/*.log {
687
+ daily
688
+ missingok
689
+ copytruncate
690
+ rotate 14
691
+ compress
692
+ delaycompress
693
+ notifempty
694
+ }
695
+ ~~~
696
+
697
+ ### Loggers that are replaced automatically
698
+
699
+ After they initialize, Rails Semantic Logger replaces the loggers of these libraries when present:
700
+
701
+ - Action Cable
702
+ - Bugsnag
703
+ - Delayed Job
704
+ - IOStreams
705
+ - Mongo
706
+ - Mongoid
707
+ - Moped
708
+ - Resque
709
+ - Sidekiq (unless `replace_sidekiq_logger = false`)
710
+ - Sidetiq
711
+ - Solid Queue (unless `replace_solid_queue_logger = false`)
712
+
713
+ ---
714
+
715
+ ## Migrating from v4 to v5
716
+
717
+ ### Ruby and Rails minimums
718
+
719
+ v5 requires Ruby 3.2+ and Rails 7.2+. It also depends on Semantic Logger v5; review the
720
+ [Semantic Logger upgrade notes](upgrading.html) for changes there (the most relevant for Rails apps
721
+ is that appenders are now reopened automatically after fork, so you can remove manual reopen hooks).
722
+
723
+ ### Appender configuration is the main change
724
+
725
+ In v4 the log file, its format, and any extra destinations were configured through several separate
726
+ options (`format`, `add_file_appender`, `ap_options`, `filter`, `console_logger`) plus direct
727
+ `config.semantic_logger.add_appender(...)` calls. In v5 all of that lives in one place, the
728
+ [appenders block](#configuring-where-logs-go-the-appenders-block).
729
+
730
+ These v4 options still work in v5 but emit a deprecation warning and will be **removed in v6**.
731
+ Migrate them as follows:
732
+
733
+ | v4 | v5 |
734
+ |----|----|
735
+ | `config.rails_semantic_logger.format = :json` | `appenders.add(file_name: "log/#{Rails.env}.log", formatter: :json)` |
736
+ | `config.rails_semantic_logger.add_file_appender = false` then `config.semantic_logger.add_appender(...)` | Declare your destinations with `appenders.add(...)` (declaring any appender already replaces the default file appender) |
737
+ | `config.rails_semantic_logger.ap_options = {multiline: false}` | `appenders.add(..., formatter: {color: {ap: {multiline: false}}})` |
738
+ | `config.rails_semantic_logger.filter = /MyClass/` | `appenders.add(..., filter: /MyClass/)` |
739
+ | `config.rails_semantic_logger.console_logger = false` | Omit `add_console` (declare a console appender only if you want one) |
740
+
741
+ A v4 Heroku / standard-out configuration like:
742
+
743
+ ~~~ruby
744
+ # v4
745
+ if ENV["RAILS_LOG_TO_STDOUT"].present?
746
+ $stdout.sync = true
747
+ config.rails_semantic_logger.add_file_appender = false
748
+ config.semantic_logger.add_appender(io: $stdout, formatter: config.rails_semantic_logger.format)
749
+ end
750
+ ~~~
751
+
752
+ becomes, in v5:
753
+
754
+ ~~~ruby
755
+ # v5
756
+ config.rails_semantic_logger.appenders do |appenders|
757
+ appenders.add(io: $stdout, formatter: :json)
758
+ end
759
+ ~~~
760
+
761
+ ### Server standard-out behavior
762
+
763
+ In v4, running `rails server` always added a standard-out logger, which you suppressed with
764
+ `bin/rails s --daemon` or Puma's `--quiet`. In v5 this is the job of `add_server`: when you use the
765
+ appenders block, declare an `add_server` appender to log to the screen while serving, or omit it to
766
+ stay silent. If you do not use the appenders block at all, the v4 behavior is preserved.
767
+
768
+ ---
769
+
770
+ ## Migrating from earlier versions
771
+
772
+ These notes apply to upgrades between older releases and are retained for reference.
773
+
774
+ ### v4.16: Sidekiq metrics support
775
+
776
+ Rails Semantic Logger added support for Sidekiq metrics, available when the JSON logging format is
777
+ used:
778
+
779
+ * `sidekiq.job.perform` &mdash; the duration of each Sidekiq job; `duration` contains the time in
780
+ milliseconds that the job took to run.
781
+ * `sidekiq.queue.latency` &mdash; the time between when a Sidekiq job was enqueued and when it was
782
+ started; `metric_amount` contains the time in milliseconds that the job was waiting in the queue.
783
+
784
+ ### v4.15 and v4.16: Sidekiq support
785
+
786
+ Rails Semantic Logger introduced direct support for Sidekiq v4, v5, v6, and v7. Remove any previous
787
+ custom patches or configurations used to make Sidekiq work with Semantic Logger. To see the complete
788
+ list of patches and to contribute your own, see
789
+ [Sidekiq Patches](https://github.com/reidmorrison/rails_semantic_logger/blob/v5.0.0/lib/rails_semantic_logger/extensions/sidekiq/sidekiq.rb).
790
+ Support for Sidekiq v4, v5, and v6 ended after Rails Semantic Logger v5.0; Sidekiq v7 and v8 are
791
+ supported and tested.
792
+
793
+ ### v4.4
794
+
795
+ With some forking frameworks it was necessary to call `reopen` after the fork. As of v4.4 the
796
+ workaround for Ruby 2.5 crashes is no longer needed. Remove the following line if it is called
797
+ anywhere:
798
+
799
+ ~~~ruby
800
+ SemanticLogger::Processor.instance.instance_variable_set(:@queue, Queue.new)
801
+ ~~~
802
+
803
+ ---
804
+
805
+ ## Deprecated configuration options
806
+
807
+ The following options still function in v5 for backward compatibility but emit deprecation warnings
808
+ and will be **removed in v6**. Each is replaced by the [appenders block](#configuring-where-logs-go-the-appenders-block).
809
+
810
+ | Deprecated option | Replacement |
811
+ |-------------------|-------------|
812
+ | `config.rails_semantic_logger.format` | `formatter:` on each appender, e.g. `appenders.add(file_name: ..., formatter: :json)` |
813
+ | `config.rails_semantic_logger.ap_options` | `formatter: {color: {ap: {...}}}` on the appender |
814
+ | `config.rails_semantic_logger.filter` | `filter:` on the appender |
815
+ | `config.rails_semantic_logger.console_logger` | Declare (or omit) an `add_console` appender |
816
+ | `config.rails_semantic_logger.add_file_appender` | Declare appenders in the block (doing so already replaces the default file appender) |