semantic_logger 5.2.0 → 5.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/docs/api.md +160 -9
- data/docs/appenders.md +461 -34
- data/docs/config.md +543 -36
- data/docs/log.md +38 -7
- data/docs/metrics.md +139 -28
- data/docs/operations.md +293 -24
- data/docs/rails.md +319 -5
- data/docs/security.md +135 -107
- data/docs/testing.md +23 -0
- data/docs/upgrading.md +644 -1
- data/lib/semantic_logger/appender/async.rb +29 -11
- data/lib/semantic_logger/appender/bugsnag.rb +7 -2
- data/lib/semantic_logger/appender/cloudwatch_logs.rb +46 -6
- data/lib/semantic_logger/appender/elasticsearch_base.rb +197 -29
- data/lib/semantic_logger/appender/elasticsearch_http.rb +22 -5
- data/lib/semantic_logger/appender/file.rb +29 -14
- data/lib/semantic_logger/appender/graylog.rb +81 -7
- data/lib/semantic_logger/appender/honeybadger.rb +7 -1
- data/lib/semantic_logger/appender/honeybadger_insights.rb +8 -2
- data/lib/semantic_logger/appender/http.rb +244 -41
- data/lib/semantic_logger/appender/io.rb +11 -6
- data/lib/semantic_logger/appender/kafka.rb +192 -46
- data/lib/semantic_logger/appender/loki.rb +42 -6
- data/lib/semantic_logger/appender/mongodb.rb +94 -9
- data/lib/semantic_logger/appender/new_relic.rb +2 -3
- data/lib/semantic_logger/appender/new_relic_logs.rb +30 -8
- data/lib/semantic_logger/appender/open_telemetry.rb +37 -35
- data/lib/semantic_logger/appender/rabbitmq.rb +241 -17
- data/lib/semantic_logger/appender/sentry.rb +2 -3
- data/lib/semantic_logger/appender/sentry_ruby.rb +82 -15
- data/lib/semantic_logger/appender/splunk.rb +4 -4
- data/lib/semantic_logger/appender/splunk_http.rb +92 -1
- data/lib/semantic_logger/appender/syslog.rb +89 -10
- data/lib/semantic_logger/appender/tcp.rb +68 -98
- data/lib/semantic_logger/appender/text_output.rb +65 -0
- data/lib/semantic_logger/appender/udp.rb +130 -18
- data/lib/semantic_logger/appender/wrapper.rb +10 -7
- data/lib/semantic_logger/appender.rb +115 -10
- data/lib/semantic_logger/appenders.rb +136 -4
- data/lib/semantic_logger/backtrace_cleaning.rb +77 -0
- data/lib/semantic_logger/base.rb +53 -71
- data/lib/semantic_logger/call_site.rb +235 -0
- data/lib/semantic_logger/context.rb +222 -0
- data/lib/semantic_logger/core_ext/process.rb +20 -5
- data/lib/semantic_logger/core_ext/warning.rb +70 -0
- data/lib/semantic_logger/deprecation.rb +139 -0
- data/lib/semantic_logger/exception_details.rb +4 -3
- data/lib/semantic_logger/formatters/base.rb +139 -19
- data/lib/semantic_logger/formatters/color.rb +3 -4
- data/lib/semantic_logger/formatters/default.rb +35 -7
- data/lib/semantic_logger/formatters/ecs.rb +9 -4
- data/lib/semantic_logger/formatters/json.rb +9 -0
- data/lib/semantic_logger/formatters/logfmt.rb +9 -7
- data/lib/semantic_logger/formatters/loki/labels.rb +282 -0
- data/lib/semantic_logger/formatters/loki.rb +412 -24
- data/lib/semantic_logger/formatters/new_relic_logs.rb +15 -13
- data/lib/semantic_logger/formatters/one_line.rb +44 -10
- data/lib/semantic_logger/formatters/open_telemetry.rb +186 -29
- data/lib/semantic_logger/formatters/pattern.rb +60 -8
- data/lib/semantic_logger/formatters/raw.rb +38 -8
- data/lib/semantic_logger/formatters/signalfx.rb +55 -15
- data/lib/semantic_logger/formatters/syslog.rb +4 -2
- data/lib/semantic_logger/formatters/syslog_cee.rb +10 -4
- data/lib/semantic_logger/formatters/syslog_header.rb +49 -0
- data/lib/semantic_logger/formatters.rb +1 -0
- data/lib/semantic_logger/log.rb +236 -80
- data/lib/semantic_logger/loggable.rb +55 -95
- data/lib/semantic_logger/logger.rb +31 -5
- data/lib/semantic_logger/metric/new_relic.rb +32 -17
- data/lib/semantic_logger/metric/signalfx.rb +21 -12
- data/lib/semantic_logger/metric/statsd.rb +26 -8
- data/lib/semantic_logger/processor.rb +194 -5
- data/lib/semantic_logger/queue_processor/backoff.rb +140 -0
- data/lib/semantic_logger/queue_processor/batch_failure.rb +69 -0
- data/lib/semantic_logger/queue_processor/held_write.rb +62 -0
- data/lib/semantic_logger/queue_processor/inbox.rb +97 -0
- data/lib/semantic_logger/queue_processor/reopener.rb +56 -0
- data/lib/semantic_logger/queue_processor/reporter.rb +113 -0
- data/lib/semantic_logger/queue_processor/request.rb +48 -10
- data/lib/semantic_logger/queue_processor/throttled_counter.rb +60 -17
- data/lib/semantic_logger/queue_processor.rb +426 -176
- data/lib/semantic_logger/ractor_copy.rb +120 -0
- data/lib/semantic_logger/ractor_processor.rb +82 -0
- data/lib/semantic_logger/ractor_receiver.rb +282 -0
- data/lib/semantic_logger/ractor_support.rb +28 -0
- data/lib/semantic_logger/redactor.rb +310 -0
- data/lib/semantic_logger/semantic_logger.rb +313 -79
- data/lib/semantic_logger/subscriber.rb +169 -15
- data/lib/semantic_logger/sync_processor.rb +226 -26
- data/lib/semantic_logger/test/capture_log_events.rb +20 -2
- data/lib/semantic_logger/test/minitest.rb +2 -1
- data/lib/semantic_logger/test/rspec.rb +1 -1
- data/lib/semantic_logger/trap_safe.rb +113 -0
- data/lib/semantic_logger/utils.rb +266 -79
- data/lib/semantic_logger/version.rb +1 -1
- data/lib/semantic_logger.rb +35 -2
- metadata +22 -2
data/docs/rails.md
CHANGED
|
@@ -76,6 +76,11 @@ With no configuration at all, Rails Semantic Logger:
|
|
|
76
76
|
* Logs to **standard error** when you run `rails console`, so log lines do not get mixed up with
|
|
77
77
|
the return values of the commands you type.
|
|
78
78
|
* Replaces the multi-line Rails request log with a single structured "Completed" line.
|
|
79
|
+
* Redacts the keys in `config.filter_parameters`, such as `password` and `token`, from every log
|
|
80
|
+
entry, not only from the request parameters. See [Redacting sensitive data](#redacting-sensitive-data).
|
|
81
|
+
* In development, records the file, line number, and method of the code behind each entry, such as
|
|
82
|
+
`orders_controller.rb:12 OrdersController#create`. See
|
|
83
|
+
[Source file name, line number, and method](#source-file-name-line-number-and-method).
|
|
79
84
|
|
|
80
85
|
Standard Rails log output for a single page request:
|
|
81
86
|
|
|
@@ -95,8 +100,8 @@ Use those two places, not `config/initializers/*`. Rails builds the logger **bef
|
|
|
95
100
|
initializers, so the appenders block and the `semantic`, `replace_sidekiq_logger`, and
|
|
96
101
|
`replace_solid_queue_logger` options have no effect from an initializer; Rails Semantic Logger
|
|
97
102
|
prints a warning when it detects this. The output-tuning options consumed later in boot (`started`,
|
|
98
|
-
`processing`, `rendered`, `quiet_assets`, `action_message_format`) do
|
|
99
|
-
initializer, but not once the application has finished booting.
|
|
103
|
+
`processing`, `rendered`, `quiet_assets`, `action_message_format`, `redact_filter_parameters`) do
|
|
104
|
+
still work from an initializer, but not once the application has finished booting.
|
|
100
105
|
|
|
101
106
|
---
|
|
102
107
|
|
|
@@ -214,7 +219,7 @@ In addition to a destination, most appenders accept these options:
|
|
|
214
219
|
|
|
215
220
|
| Option | Description |
|
|
216
221
|
|--------|-------------|
|
|
217
|
-
| `level` | Only write entries at this level or higher to this appender
|
|
222
|
+
| `level` | Only write entries at this level or higher to this appender; for what it defaults to, and how it works with each logger's own level, which Rails Semantic Logger sets from `config.log_level`, see [Per-appender settings](config.html#per-appender-settings). |
|
|
218
223
|
| `formatter` | How to format the output: `:default`, `:color`, `:json`, `:logfmt`, `:one_line`, or a custom formatter (see [Output formats](#output-formats)). |
|
|
219
224
|
| `filter` | A `Regexp` or `Proc` selecting which entries this appender accepts. See [Filtering](config.html#filtering). |
|
|
220
225
|
| `application`, `environment`, `host` | Override the global values for this appender only. |
|
|
@@ -537,10 +542,97 @@ Notes:
|
|
|
537
542
|
* If a value returns `nil`, that key is omitted for that request.
|
|
538
543
|
* To turn named tags off in development, set `config.log_tags = nil` in
|
|
539
544
|
`config/environments/development.rb`.
|
|
545
|
+
* The tags are computed when the request enters the middleware stack, before the session is loaded
|
|
546
|
+
and before authentication middleware such as Warden (used by Devise) runs. A value can come from
|
|
547
|
+
the request's headers or cookies, but not from the signed-in user.
|
|
540
548
|
|
|
541
|
-
|
|
549
|
+
#### Tag a request with the signed-in user
|
|
542
550
|
|
|
543
|
-
To
|
|
551
|
+
To tag every entry logged while a controller handles a request with values that are only known
|
|
552
|
+
inside the controller, such as the signed-in user, wrap the action in an `around_action`:
|
|
553
|
+
|
|
554
|
+
~~~ruby
|
|
555
|
+
class ApplicationController < ActionController::Base
|
|
556
|
+
around_action :tag_logs
|
|
557
|
+
|
|
558
|
+
private
|
|
559
|
+
|
|
560
|
+
def tag_logs(&)
|
|
561
|
+
SemanticLogger.tagged({user_id: current_user&.id}.compact, &)
|
|
562
|
+
end
|
|
563
|
+
end
|
|
564
|
+
~~~
|
|
565
|
+
|
|
566
|
+
The tags apply until the action finishes, and are removed even when it raises an exception. The
|
|
567
|
+
`compact` leaves the tag out for a request without a signed-in user. The tags are not on the
|
|
568
|
+
`Started`, `Processing`, and `Completed` entries, which are logged outside the action's callbacks. To
|
|
569
|
+
add the user to the `Completed` entry as well, see
|
|
570
|
+
[Add custom data to the Completed message](#add-custom-data-to-the-completed-message).
|
|
571
|
+
|
|
572
|
+
Use the block form, as above, rather than `SemanticLogger.push_named_tags` in a `before_action`.
|
|
573
|
+
Tags added with `push_tags` or `push_named_tags` stay on the thread until they are popped, and web
|
|
574
|
+
servers such as Puma serve many requests on each thread, so tags pushed during one request would be
|
|
575
|
+
logged for every later request on that thread.
|
|
576
|
+
|
|
577
|
+
When a nested block sets a named tag that is already set, its value replaces the earlier one. A
|
|
578
|
+
Hash value, such as `user: {id: 1}`, is replaced as a whole rather than merged with the earlier
|
|
579
|
+
Hash, so prefer flat keys such as `user_id:`.
|
|
580
|
+
|
|
581
|
+
### Source file name, line number, and method
|
|
582
|
+
|
|
583
|
+
In development, Rails Semantic Logger records the file, line number, and method of the code behind
|
|
584
|
+
each entry, at every level. In the Rails console it records them for each entry at the console's log
|
|
585
|
+
level, in every environment.
|
|
586
|
+
|
|
587
|
+
~~~
|
|
588
|
+
2026-11-15 13:58:05.906754 I [8462:puma srv tp 001 orders_controller.rb:12 OrdersController#create] OrdersController -- Creating order
|
|
589
|
+
~~~
|
|
590
|
+
|
|
591
|
+
To record them in another environment, or only at and above a level, set that level:
|
|
592
|
+
|
|
593
|
+
~~~ruby
|
|
594
|
+
config.semantic_logger.call_site_level = :info
|
|
595
|
+
~~~
|
|
596
|
+
|
|
597
|
+
To stop recording them in development, set it to `nil` in `config/application.rb`,
|
|
598
|
+
`config/environments/development.rb`, or an initializer:
|
|
599
|
+
|
|
600
|
+
~~~ruby
|
|
601
|
+
config.semantic_logger.call_site_level = nil
|
|
602
|
+
~~~
|
|
603
|
+
|
|
604
|
+
This reads only the stack frames up to the code that made the log call, so it is cheap enough to
|
|
605
|
+
leave on in production. See
|
|
606
|
+
[Recording where each entry was logged](config.html#recording-where-each-entry-was-logged) for its
|
|
607
|
+
cost, and for which formatters write it.
|
|
608
|
+
|
|
609
|
+
Rails logs the entries for its own events from Active Support's notifications, so the code that makes
|
|
610
|
+
their log call is Active Support's. Rails Semantic Logger records the application code behind each
|
|
611
|
+
one instead:
|
|
612
|
+
|
|
613
|
+
| Entry | Call site |
|
|
614
|
+
|-------|-----------|
|
|
615
|
+
| SQL query, strict loading violation | The code that ran the query |
|
|
616
|
+
| `Redirected to`, `Sent data`, `Sent file`, unpermitted parameters, fragment caching | The line in the action |
|
|
617
|
+
| `Processing`, `Completed`, CSRF warnings | The controller's action |
|
|
618
|
+
| `rescue_from` | Where the exception was raised |
|
|
619
|
+
| `Rendering`, `Rendered` | The code that called `render`, or the template that rendered a partial |
|
|
620
|
+
| Active Job | The code that enqueued or performed the job, or else the job's `#perform`, as for a job that a worker, or the inline or test adapter, performs from its queue |
|
|
621
|
+
| Action Mailer | The code that delivered the mail, or else the mailer's action |
|
|
622
|
+
| An exception that a request raised | Where your code raised it |
|
|
623
|
+
|
|
624
|
+
Entries with no application code behind them record no call site: `Started`, a halted filter chain,
|
|
625
|
+
a template that Rails renders once the action has returned, and the entries of Sidekiq's and Solid
|
|
626
|
+
Queue's own processes. Code that only wraps what Rails runs, such as an `around_action` or a
|
|
627
|
+
worker's middleware, is not taken for the code behind an entry that Rails logs once it has taken over
|
|
628
|
+
from your code, as it does to render an action's template, or to perform a job from its queue. The
|
|
629
|
+
application code is the first frame that the subscriber's `backtrace_cleaner` keeps,
|
|
630
|
+
`Rails.backtrace_cleaner` unless you assign another, such as
|
|
631
|
+
`RailsSemanticLogger::ActiveRecord::LogSubscriber.backtrace_cleaner`. It is also the `source` that
|
|
632
|
+
`config.active_record.verbose_query_logs` and `config.active_job.verbose_enqueue_logs` add to the
|
|
633
|
+
payload.
|
|
634
|
+
|
|
635
|
+
To capture the whole backtrace of each entry instead, use `backtrace_level`:
|
|
544
636
|
|
|
545
637
|
~~~ruby
|
|
546
638
|
config.semantic_logger.backtrace_level = :info
|
|
@@ -552,6 +644,95 @@ config.semantic_logger.backtrace_level = :info
|
|
|
552
644
|
to `nil` (disabled) or to a high level such as `:error`. By default backtraces are only captured for
|
|
553
645
|
`:error` and `:fatal`. This feature is best used in development.
|
|
554
646
|
|
|
647
|
+
### Cleaning exception backtraces
|
|
648
|
+
|
|
649
|
+
Rails Semantic Logger cleans the backtraces of logged exceptions with `Rails.backtrace_cleaner`, as
|
|
650
|
+
Rails' own log does: only your application's frames are logged, relative to `Rails.root`. The
|
|
651
|
+
silencers and filters added in `config/initializers/backtrace_silencers.rb` apply, since they change
|
|
652
|
+
the same cleaner. Set the `BACKTRACE` environment variable to log full backtraces again, as with the
|
|
653
|
+
rest of Rails.
|
|
654
|
+
|
|
655
|
+
The text and JSON formatters use it, including the JSON sent by the HTTP, Elasticsearch HTTP, TCP,
|
|
656
|
+
UDP, Kafka, RabbitMQ, and CloudWatch Logs appenders. Error services, such as Sentry, still receive
|
|
657
|
+
the full backtrace. When none of an exception's frames is your application's, such as one raised
|
|
658
|
+
entirely within a gem, the full backtrace is logged.
|
|
659
|
+
|
|
660
|
+
A request for a missing route, and the other exceptions that Rails logs without a backtrace when
|
|
661
|
+
none of its frames is your application's (those in `ActionDispatch::ExceptionWrapper.silent_exceptions`),
|
|
662
|
+
is logged on one line, after the response's status, as in Rails' own log:
|
|
663
|
+
|
|
664
|
+
~~~
|
|
665
|
+
Rails -- 404 Not Found -- Exception: ActionController::RoutingError: No route matches [GET] "/apple-touch-icon.png"
|
|
666
|
+
~~~
|
|
667
|
+
|
|
668
|
+
That entry has the exception's class and message, and names the exceptions that caused it, which
|
|
669
|
+
Rails lists as "Caused by", but not the exception itself (`log_exception: :partial`): the `:json`
|
|
670
|
+
formatter writes them in its `message`, and Semantic Logger's own error service appenders, such as
|
|
671
|
+
`sentry_ruby`, receive a message rather than an exception. Error services hooked into Rails still receive the exception. Set `BACKTRACE` to log
|
|
672
|
+
these with their backtrace, or set `config.action_dispatch.log_rescued_responses = false` to leave
|
|
673
|
+
them out of the log.
|
|
674
|
+
|
|
675
|
+
To use another cleaner, or none, assign it in `config/application.rb`. This does not change how the
|
|
676
|
+
exceptions above are logged:
|
|
677
|
+
|
|
678
|
+
~~~ruby
|
|
679
|
+
config.semantic_logger.backtrace_cleaner = nil
|
|
680
|
+
~~~
|
|
681
|
+
|
|
682
|
+
See [Cleaning backtraces](config.html#cleaning-backtraces) for which formatters use it, and how to
|
|
683
|
+
choose for one appender.
|
|
684
|
+
|
|
685
|
+
### Redacting sensitive data
|
|
686
|
+
|
|
687
|
+
Rails filters `config.filter_parameters` only from the request parameters that it logs. Rails Semantic
|
|
688
|
+
Logger also has Semantic Logger redact them from every log entry: the value of each key that they
|
|
689
|
+
match is written as `"[FILTERED]"`, at any depth, in the payload, named tags, and context, by every
|
|
690
|
+
appender. That covers your own calls, such as `logger.info("Charged", card: card_params)`, and the
|
|
691
|
+
entries of other gems, as well as Rails' own. The filters are read once the application has
|
|
692
|
+
initialized, so those in `config/initializers/filter_parameter_logging.rb` apply.
|
|
693
|
+
|
|
694
|
+
Keys are matched by name, both as written and in snake_case, so `:_key` also matches an `X-Api-Key`
|
|
695
|
+
header, as `x_api_key`. A filter matches any key that contains it, so the default list of a new Rails
|
|
696
|
+
application (`:passw, :email, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn, :cvv,
|
|
697
|
+
:cvc`) also redacts keys that are not secret, such as `max_tokens`, `token_count`, `cache_key`, and
|
|
698
|
+
`api_key_id`, the `concurrency_key` of Solid Queue's entries, and every `email`.
|
|
699
|
+
|
|
700
|
+
That list does not match the `Authorization` or `Cookie` header. Add them:
|
|
701
|
+
|
|
702
|
+
~~~ruby
|
|
703
|
+
# config/initializers/filter_parameter_logging.rb
|
|
704
|
+
Rails.application.config.filter_parameters += %i[authorization cookie]
|
|
705
|
+
~~~
|
|
706
|
+
|
|
707
|
+
What is not redacted:
|
|
708
|
+
|
|
709
|
+
* Active Job `arguments` and Action Mailer `args`, which are logged as JSON text. For a job that takes
|
|
710
|
+
a secret, leave its arguments out of the log with `self.log_arguments = false`.
|
|
711
|
+
* A key that only a Proc filter matches: a Proc changes a value rather than naming a key, so it is
|
|
712
|
+
left out.
|
|
713
|
+
* A key nested under the one a dotted filter names. A filter such as `"credit_card.code"` only
|
|
714
|
+
matches a key with that whole name, not `code` inside `credit_card`. That includes the
|
|
715
|
+
`"model.attribute"` filters that Rails adds for Active Record encrypted attributes.
|
|
716
|
+
* The message, plain tags, and exception messages: only keyed data is redacted.
|
|
717
|
+
* Entries logged while the application starts, before the list is complete.
|
|
718
|
+
|
|
719
|
+
To use other keys instead, assign them in `config/application.rb`, and Rails Semantic Logger keeps
|
|
720
|
+
them:
|
|
721
|
+
|
|
722
|
+
~~~ruby
|
|
723
|
+
config.semantic_logger.sensitive_keys = %i[passw secret token _key authorization cookie]
|
|
724
|
+
~~~
|
|
725
|
+
|
|
726
|
+
To turn redaction off:
|
|
727
|
+
|
|
728
|
+
~~~ruby
|
|
729
|
+
config.rails_semantic_logger.redact_filter_parameters = false
|
|
730
|
+
~~~
|
|
731
|
+
|
|
732
|
+
Assigning `config.semantic_logger.sensitive_keys = nil` does not turn it off, since that cannot be
|
|
733
|
+
told apart from never assigning it. See [Redacting sensitive data](security.html#redacting-sensitive-data)
|
|
734
|
+
for what `sensitive_keys` accepts, and when redaction runs.
|
|
735
|
+
|
|
555
736
|
### Add custom data to the Completed message
|
|
556
737
|
|
|
557
738
|
Add an `append_info_to_payload` method to a controller to include extra fields in its Completed
|
|
@@ -624,6 +805,69 @@ Sidekiq.configure_server do |config|
|
|
|
624
805
|
end
|
|
625
806
|
~~~
|
|
626
807
|
|
|
808
|
+
#### Testing Sidekiq jobs and middleware
|
|
809
|
+
|
|
810
|
+
Use the [test helpers](testing.html) (`capture_semantic_logger_events` in RSpec,
|
|
811
|
+
`semantic_logger_events` in Minitest) to capture the log events from jobs and server middleware. This
|
|
812
|
+
relies on `replace_sidekiq_logger` being on (the default): with it off, jobs and middleware log
|
|
813
|
+
through a plain `Sidekiq::Logger`, which the helpers do not capture.
|
|
814
|
+
|
|
815
|
+
Two things differ from a running Sidekiq server:
|
|
816
|
+
|
|
817
|
+
- Sidekiq's testing modes (`inline!`, and `fake!` with `drain` or `perform_one`) run only the
|
|
818
|
+
middleware added to `Sidekiq::Testing.server_middleware`, not the chain from
|
|
819
|
+
`Sidekiq.configure_server`. Add the middleware under test there.
|
|
820
|
+
- The job logger does not run, so there are no `Start #perform` or `Completed #perform` entries, and
|
|
821
|
+
no `jid`, class, or queue named tags. Named tags added by your own middleware are still present.
|
|
822
|
+
|
|
823
|
+
Capture every event rather than one class's. Middleware that includes `Sidekiq::ServerMiddleware`
|
|
824
|
+
logs through Sidekiq's configured logger, which a capture scoped to a class, such as
|
|
825
|
+
`capture_semantic_logger_events(Sidekiq)`, does not reach.
|
|
826
|
+
|
|
827
|
+
The example below turns on Sidekiq's testing modes with `Sidekiq.testing!(:fake)`, available from
|
|
828
|
+
Sidekiq 8.1.1. On earlier versions use `require "sidekiq/testing"` instead.
|
|
829
|
+
|
|
830
|
+
~~~ruby
|
|
831
|
+
# spec/rails_helper.rb
|
|
832
|
+
Sidekiq.testing!(:fake)
|
|
833
|
+
|
|
834
|
+
# app/sidekiq/tenant_logging_middleware.rb
|
|
835
|
+
class TenantLoggingMiddleware
|
|
836
|
+
include Sidekiq::ServerMiddleware
|
|
837
|
+
|
|
838
|
+
def call(_job_instance, job, _queue, &block)
|
|
839
|
+
SemanticLogger.tagged(tenant: job["tenant"], &block)
|
|
840
|
+
end
|
|
841
|
+
end
|
|
842
|
+
|
|
843
|
+
# app/sidekiq/report_job.rb
|
|
844
|
+
class ReportJob
|
|
845
|
+
include Sidekiq::Job
|
|
846
|
+
|
|
847
|
+
def perform
|
|
848
|
+
logger.info("Building report")
|
|
849
|
+
end
|
|
850
|
+
end
|
|
851
|
+
|
|
852
|
+
# spec/sidekiq/tenant_logging_middleware_spec.rb
|
|
853
|
+
RSpec.describe TenantLoggingMiddleware do
|
|
854
|
+
around { |example| Sidekiq::Testing.inline! { example.run } }
|
|
855
|
+
before { Sidekiq::Testing.server_middleware { |chain| chain.add TenantLoggingMiddleware } }
|
|
856
|
+
after { Sidekiq::Testing.server_middleware { |chain| chain.remove TenantLoggingMiddleware } }
|
|
857
|
+
|
|
858
|
+
it "tags the job's log entries with the tenant" do
|
|
859
|
+
events = capture_semantic_logger_events { ReportJob.set(tenant: "acme").perform_async }
|
|
860
|
+
event = events.find { |e| e.name == "ReportJob" }
|
|
861
|
+
|
|
862
|
+
expect(event).to be_a_semantic_logger_event(message: "Building report")
|
|
863
|
+
expect(event.named_tags).to include(tenant: "acme")
|
|
864
|
+
end
|
|
865
|
+
end
|
|
866
|
+
~~~
|
|
867
|
+
|
|
868
|
+
The last line checks only the `tenant` tag, so the test still passes when other named tags are
|
|
869
|
+
active.
|
|
870
|
+
|
|
627
871
|
### Custom controller base class
|
|
628
872
|
|
|
629
873
|
If your application uses a controller base class other than `ActionController::Base` or
|
|
@@ -730,6 +974,76 @@ After they initialize, Rails Semantic Logger replaces the loggers of these libra
|
|
|
730
974
|
|
|
731
975
|
---
|
|
732
976
|
|
|
977
|
+
## Upgrading to v5.3
|
|
978
|
+
|
|
979
|
+
v5.3 requires Semantic Logger v5.3; see [Upgrading to Semantic Logger v5.3](upgrading.html#upgrading-to-semantic-logger-v53)
|
|
980
|
+
for the changes there. It needs no configuration changes, but several log entries change. Review
|
|
981
|
+
these before you upgrade if a log query, dashboard, or alert depends on them.
|
|
982
|
+
|
|
983
|
+
### Exception backtraces are cleaned
|
|
984
|
+
|
|
985
|
+
The backtraces of logged exceptions are now cleaned with `Rails.backtrace_cleaner`, as in Rails' own
|
|
986
|
+
log: only your application's frames are written, relative to `Rails.root`. Error services, such as
|
|
987
|
+
Sentry, still receive the full backtrace. To log full backtraces again, set the `BACKTRACE`
|
|
988
|
+
environment variable, or turn cleaning off in `config/application.rb`:
|
|
989
|
+
|
|
990
|
+
~~~ruby
|
|
991
|
+
config.semantic_logger.backtrace_cleaner = nil
|
|
992
|
+
~~~
|
|
993
|
+
|
|
994
|
+
A request for a missing route, and the other exceptions in
|
|
995
|
+
`ActionDispatch::ExceptionWrapper.silent_exceptions`, is now logged on one line, without the exception
|
|
996
|
+
itself, when none of its frames is your application's. Semantic Logger's own error service appenders,
|
|
997
|
+
such as `sentry_ruby`, therefore receive a message for it rather than an exception. See
|
|
998
|
+
[Cleaning exception backtraces](#cleaning-exception-backtraces).
|
|
999
|
+
|
|
1000
|
+
### `config.filter_parameters` are redacted from every entry
|
|
1001
|
+
|
|
1002
|
+
The keys in `config.filter_parameters` are now redacted from the payload, named tags, and context of
|
|
1003
|
+
every log entry, not only from the request parameters. A filter matches any key that contains it, so
|
|
1004
|
+
Rails' default filters also redact keys such as `email`, `max_tokens`, `cache_key`, and
|
|
1005
|
+
`api_key_id`. Rails' defaults do not match the `Authorization` or `Cookie` headers, so consider adding
|
|
1006
|
+
them. To turn redaction off:
|
|
1007
|
+
|
|
1008
|
+
~~~ruby
|
|
1009
|
+
config.rails_semantic_logger.redact_filter_parameters = false
|
|
1010
|
+
~~~
|
|
1011
|
+
|
|
1012
|
+
See [Redacting sensitive data](#redacting-sensitive-data).
|
|
1013
|
+
|
|
1014
|
+
### Call sites are recorded in development
|
|
1015
|
+
|
|
1016
|
+
In development, each entry now records the file, line, and method of the code behind it. The text
|
|
1017
|
+
formatters write it after the thread name, and the `:json` formatter as `file`, `line`, and `method`.
|
|
1018
|
+
In the Rails console it replaces capturing a backtrace for every entry. To turn it off:
|
|
1019
|
+
|
|
1020
|
+
~~~ruby
|
|
1021
|
+
config.semantic_logger.call_site_level = nil
|
|
1022
|
+
~~~
|
|
1023
|
+
|
|
1024
|
+
See [Source file name, line number, and method](#source-file-name-line-number-and-method).
|
|
1025
|
+
|
|
1026
|
+
### Other log output changes
|
|
1027
|
+
|
|
1028
|
+
* Active Record: SQL bind values that have no column name are keyed by position (`$1`, `$2`, ...),
|
|
1029
|
+
as in Rails 8.2's own log, instead of all under a `nil` key. Update anything that matches
|
|
1030
|
+
`payload.binds.nil`.
|
|
1031
|
+
* Active Job: the message of an enqueue that failed now closes its parentheses:
|
|
1032
|
+
`Failed enqueuing MyJob (Job ID: 1a2b) to Sidekiq(default): RuntimeError (boom)`.
|
|
1033
|
+
* Action Controller: the `source` of a `Redirected to` entry, written when
|
|
1034
|
+
`ActionDispatch.verbose_redirect_logs` is on, is now relative to `Rails.root` instead of an absolute
|
|
1035
|
+
path.
|
|
1036
|
+
* `config.active_record.verbose_query_logs` and `config.active_job.verbose_enqueue_logs`, which new
|
|
1037
|
+
Rails applications turn on in development, are now honored: query and enqueue entries gain a
|
|
1038
|
+
`source` in their payload.
|
|
1039
|
+
* Rails 8.1+: open redirects and path relative redirects reported with `:notify` are now logged at
|
|
1040
|
+
`:warn`. On Rails 8.2, CSRF failures are logged at `:warn` again.
|
|
1041
|
+
|
|
1042
|
+
See the [CHANGELOG](https://github.com/reidmorrison/rails_semantic_logger/blob/main/CHANGELOG.md)
|
|
1043
|
+
for the full list, including Rails 8.2 support.
|
|
1044
|
+
|
|
1045
|
+
---
|
|
1046
|
+
|
|
733
1047
|
## Upgrading to v5.1
|
|
734
1048
|
|
|
735
1049
|
v5.1 requires Semantic Logger v5.1 and needs no configuration changes. Two behaviors differ.
|