lumberjack_sidekiq 1.0.1 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/continuous_integration.yml +4 -2
- data/CHANGE_LOG.md +20 -0
- data/README.md +20 -20
- data/VERSION +1 -1
- data/lib/lumberjack/sidekiq/attribute_passthrough_middleware.rb +83 -0
- data/lib/lumberjack/sidekiq/job_logger.rb +104 -40
- data/lib/lumberjack/sidekiq/message_formatter.rb +9 -3
- data/lib/lumberjack/sidekiq.rb +4 -1
- data/lumberjack_sidekiq.gemspec +2 -2
- metadata +7 -11
- data/lib/lumberjack/sidekiq/tag_passthrough_middleware.rb +0 -54
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f48f7d5f12c23e6df229a12e1e6906936cf2609f6327e006d1a20613583f3a3c
|
4
|
+
data.tar.gz: af324ef8cd1db551edafa69837e83f596d7aff736747890ee288be7345afce0d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 7caf943f23350cc8357fe3eb685c732adbbe8b794b868533ce8bd91054c5fd27c39fde7e3e135b2a6251b47ab6e9ecf54616b7db4d8c69ff504be0c047a7150b
|
7
|
+
data.tar.gz: ff07e2de83965ec9992e30bd624b07cf0607d7595ebf17f726673bcc0d3f2198aecefbb2e6b6d779fad18909be908ee9464bee2f5f36b8c852dfb7af2101733e
|
@@ -5,8 +5,6 @@ on:
|
|
5
5
|
branches:
|
6
6
|
- main
|
7
7
|
- actions-*
|
8
|
-
tags:
|
9
|
-
- v*
|
10
8
|
pull_request:
|
11
9
|
branches-ignore:
|
12
10
|
- actions-*
|
@@ -27,6 +25,7 @@ jobs:
|
|
27
25
|
include:
|
28
26
|
- ruby: "ruby"
|
29
27
|
standardrb: true
|
28
|
+
yard: true
|
30
29
|
- ruby: "3.2"
|
31
30
|
appraisal: "sidekiq_8"
|
32
31
|
- ruby: "2.7"
|
@@ -50,3 +49,6 @@ jobs:
|
|
50
49
|
- name: standardrb
|
51
50
|
if: matrix.standardrb
|
52
51
|
run: bundle exec standardrb
|
52
|
+
- name: yard
|
53
|
+
if: matrix.yard
|
54
|
+
run: bundle exec yardoc --fail-on-warning
|
data/CHANGE_LOG.md
CHANGED
@@ -4,6 +4,26 @@ All notable changes to this project will be documented in this file.
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
5
5
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
6
6
|
|
7
|
+
## 2.0.0
|
8
|
+
|
9
|
+
### Added
|
10
|
+
|
11
|
+
- Lumberjack 2 support.
|
12
|
+
- Added Lumberjack global context around the job logger.
|
13
|
+
|
14
|
+
### Changed
|
15
|
+
|
16
|
+
- Attributes are now passed through the logger's attribute formatter before serializing them to JSON for inclusion in job payloads.
|
17
|
+
- **Breaking Change** Updated terminology to match Lumberjack 2 change to refer to "attributes" rather than "tags".
|
18
|
+
- Changed config option `log_tag_prefix` to `log_attributes_prefix`
|
19
|
+
- Renamed `Lumberjack::Sidekiq::TagPassthroughMiddleware` to `Lumberjack::Sidekiq::AttributePassthroughMiddleware`.
|
20
|
+
|
21
|
+
## 1.0.1
|
22
|
+
|
23
|
+
### Changed
|
24
|
+
|
25
|
+
- Switched to using the standard `with_level` method for temporarily changing log levels.
|
26
|
+
|
7
27
|
## 1.0.0
|
8
28
|
|
9
29
|
### Added
|
data/README.md
CHANGED
@@ -8,9 +8,9 @@ This gem provides an enhanced logging setup for [Sidekiq](https://github.com/mpe
|
|
8
8
|
|
9
9
|
**Key Features:**
|
10
10
|
|
11
|
-
- **Structured Job Logging**: Automatically adds structured
|
12
|
-
- **Context Propagation**: Pass log
|
13
|
-
- **Flexible Configuration**: Control logging behavior per job with options for log levels, argument filtering, and custom
|
11
|
+
- **Structured Job Logging**: Automatically adds structured attributes for job metadata (class, job ID, queue, duration, etc.)
|
12
|
+
- **Context Propagation**: Pass log attributes from client to server to maintain request context across job execution
|
13
|
+
- **Flexible Configuration**: Control logging behavior per job with options for log levels, argument filtering, and custom attributes
|
14
14
|
- **Performance Tracking**: Automatic timing of job execution and queue wait times
|
15
15
|
|
16
16
|
## Usage
|
@@ -24,7 +24,7 @@ To use it, configure Sidekiq to use the Lumberjack job logger:
|
|
24
24
|
```ruby
|
25
25
|
require 'lumberjack_sidekiq'
|
26
26
|
|
27
|
-
#
|
27
|
+
# First you'll need a Lumberjack logger instance
|
28
28
|
logger = Lumberjack::Logger.new(STDOUT)
|
29
29
|
|
30
30
|
# Configure Sidekiq to use Lumberjack
|
@@ -34,7 +34,7 @@ Sidekiq.configure_server do |config|
|
|
34
34
|
end
|
35
35
|
```
|
36
36
|
|
37
|
-
The job logger automatically adds structured
|
37
|
+
The job logger automatically adds structured attributes to your log entries:
|
38
38
|
|
39
39
|
- `class` - The worker class name
|
40
40
|
- `jid` - The job ID
|
@@ -43,32 +43,32 @@ The job logger automatically adds structured tags to your log entries:
|
|
43
43
|
- `duration` - Job execution time in seconds
|
44
44
|
- `enqueued_ms` - Time the job was queued before execution
|
45
45
|
- `retry_count` - Number of retries (if > 0)
|
46
|
-
- `
|
46
|
+
- `attributes` - Any custom Sidekiq attributes
|
47
47
|
|
48
|
-
You can add an optional prefix to all
|
48
|
+
You can add an optional prefix to all attributes:
|
49
49
|
|
50
50
|
```ruby
|
51
51
|
Sidekiq.configure_server do |config|
|
52
|
-
config[:
|
52
|
+
config[:log_attribute_prefix] = "sidekiq."
|
53
53
|
end
|
54
54
|
```
|
55
55
|
|
56
|
-
###
|
56
|
+
### Attribute Passthrough Middleware
|
57
57
|
|
58
|
-
The `Lumberjack::Sidekiq::
|
58
|
+
The `Lumberjack::Sidekiq::AttributePassthroughMiddleware` allows you to pass log attributes from the client (where jobs are enqueued) to the server (where jobs are executed). This is useful for maintaining context like user IDs or request IDs across the job execution.
|
59
59
|
|
60
60
|
Configure the middleware on the client side:
|
61
61
|
|
62
62
|
```ruby
|
63
63
|
Sidekiq.configure_client do |config|
|
64
64
|
config.client_middleware do |chain|
|
65
|
-
# Pass through :user_id and :request_id
|
66
|
-
chain.add(Lumberjack::Sidekiq::
|
65
|
+
# Pass through :user_id and :request_id attributes to the job logger
|
66
|
+
chain.add(Lumberjack::Sidekiq::AttributePassthroughMiddleware, :user_id, :request_id)
|
67
67
|
end
|
68
68
|
end
|
69
69
|
```
|
70
70
|
|
71
|
-
Now when you enqueue a job with those
|
71
|
+
Now when you enqueue a job with those attributes in the current logging context, they will be propagated to the logs when the job runs.
|
72
72
|
|
73
73
|
```ruby
|
74
74
|
logger.tag(user_id: 123, request_id: "abc-def") do
|
@@ -78,9 +78,9 @@ end
|
|
78
78
|
|
79
79
|
### Adding Additional Metadata
|
80
80
|
|
81
|
-
You can add additional metadata to your job logs by adding your own server middleware. Job logging sets up
|
81
|
+
You can add additional metadata to your job logs by adding your own server middleware. Job logging sets up an attribute context so any attributes you add in your middleware will be included in the job log when it finishes.
|
82
82
|
|
83
|
-
|
83
|
+
Attributes added before the `yield` in your middleware will be included in all logs for the job processing. Attributes added after the `yield` will only be included in the final job lifecycle event log.
|
84
84
|
|
85
85
|
```ruby
|
86
86
|
class MyLogTaggingMiddleware
|
@@ -92,7 +92,7 @@ class MyLogTaggingMiddleware
|
|
92
92
|
|
93
93
|
yield
|
94
94
|
|
95
|
-
# Add tag_2
|
95
|
+
# Add tag_2 to the final job log only.
|
96
96
|
Sidekiq.logger.tag(tag_2: job["value_2"]) if Sidekiq.logger.is_a?(Lumberjack::Logger)
|
97
97
|
end
|
98
98
|
end
|
@@ -117,7 +117,7 @@ class MyWorker
|
|
117
117
|
skip: false, # Skip logging lifecycle events for this job
|
118
118
|
skip_start: true, # Skip the "Start job" lifecycle log message
|
119
119
|
args: ["param1"], # Only log specific arguments by name; can specify false to omit all args
|
120
|
-
|
120
|
+
attributes: {custom: "value"} # Add custom attributes to job logs
|
121
121
|
}
|
122
122
|
|
123
123
|
def perform(param1, param2)
|
@@ -136,11 +136,11 @@ Sidekiq.configure_server do |config|
|
|
136
136
|
end
|
137
137
|
```
|
138
138
|
|
139
|
-
You can add a prefix to all automatically generated log
|
139
|
+
You can add a prefix to all automatically generated log attributes by setting `:log_attribute_prefix`.
|
140
140
|
|
141
141
|
```ruby
|
142
142
|
Sidekiq.configure_server do |config|
|
143
|
-
config[:
|
143
|
+
config[:log_attribute_prefix] = "sidekiq."
|
144
144
|
end
|
145
145
|
```
|
146
146
|
|
@@ -160,7 +160,7 @@ Sidekiq.configure_server do |config|
|
|
160
160
|
end
|
161
161
|
```
|
162
162
|
|
163
|
-
You can customize the message format by implementing your own `Lumberjack::Sidekiq::MessageFormatter` and setting it in the configuration. You can use this if
|
163
|
+
You can customize the message format by implementing your own `Lumberjack::Sidekiq::MessageFormatter` and setting it in the configuration. You can use this if your existing log processing pipeline is expecting specific message formats.
|
164
164
|
|
165
165
|
```ruby
|
166
166
|
Sidekiq.configure_server do |config|
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
|
1
|
+
2.0.0
|
@@ -0,0 +1,83 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "json"
|
4
|
+
|
5
|
+
# Sidekiq client middleware that can pass through log attributes from the current Lumberjack
|
6
|
+
# logger to job logger when the job is executed on the Sidekiq server. This can be
|
7
|
+
# useful to maintain context in logs when a job is executed.
|
8
|
+
#
|
9
|
+
# @example Adding middleware to pass through attributes
|
10
|
+
# Sidekiq.configure_client do |config|
|
11
|
+
# config.client_middleware do |chain|
|
12
|
+
# # Pass through :user_id and :request_id attributes to the job logger.
|
13
|
+
# chain.add(Lumberjack::Sidekiq::AttributePassthroughMiddleware, :user_id, :request_id)
|
14
|
+
# end
|
15
|
+
# end
|
16
|
+
class Lumberjack::Sidekiq::AttributePassthroughMiddleware
|
17
|
+
include ::Sidekiq::ClientMiddleware
|
18
|
+
|
19
|
+
# Types that can be safely serialized to JSON without losing information
|
20
|
+
JSON_SAFE_TYPES = [String, Integer, Float, TrueClass, FalseClass].freeze
|
21
|
+
|
22
|
+
# @param pass_through_attributes [Array<String, Symbol>] Log attributes to pass through to the job logger when the job is executed.
|
23
|
+
def initialize(*pass_through_attributes)
|
24
|
+
@pass_through_attributes = pass_through_attributes.flatten.map(&:to_s)
|
25
|
+
end
|
26
|
+
|
27
|
+
# Sidekiq client middleware hook that adds configured log attributes to the job data
|
28
|
+
# so they can be used by the job logger when the job is executed.
|
29
|
+
#
|
30
|
+
# @param job_class_or_string [String, Class] The worker class or class name
|
31
|
+
# @param job [Hash] The job hash containing job data
|
32
|
+
# @param queue [String] The queue name
|
33
|
+
# @param redis_pool [ConnectionPool] The Redis connection pool
|
34
|
+
# @yield The next middleware in the chain
|
35
|
+
# @return [void]
|
36
|
+
def call(job_class_or_string, job, queue, redis_pool)
|
37
|
+
return yield unless Sidekiq.logger.is_a?(Lumberjack::Logger)
|
38
|
+
|
39
|
+
job["logging"] ||= {}
|
40
|
+
attributes = job["logging"]["attributes"] || {}
|
41
|
+
|
42
|
+
unless @pass_through_attributes.empty?
|
43
|
+
logger_attributes = logger_attributes_helper
|
44
|
+
@pass_through_attributes.each do |attribute|
|
45
|
+
value = json_value(logger_attributes[attribute])
|
46
|
+
attributes[attribute] = value unless value.nil?
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
job["logging"]["attributes"] = attributes unless attributes.empty?
|
51
|
+
|
52
|
+
yield
|
53
|
+
end
|
54
|
+
|
55
|
+
private
|
56
|
+
|
57
|
+
# Gets logger attributes helper for extracting formatted attributes.
|
58
|
+
#
|
59
|
+
# @return [Lumberjack::AttributesHelper] Helper for accessing formatted attributes
|
60
|
+
def logger_attributes_helper
|
61
|
+
attributes = Sidekiq.logger.attributes
|
62
|
+
formatter = Sidekiq.logger.attribute_formatter
|
63
|
+
if formatter
|
64
|
+
attributes = formatter.format(attributes)
|
65
|
+
end
|
66
|
+
Lumberjack::AttributesHelper.new(attributes)
|
67
|
+
end
|
68
|
+
|
69
|
+
# Converts a value to a JSON-safe format.
|
70
|
+
#
|
71
|
+
# @param value [Object] The value to convert
|
72
|
+
# @return [Object, nil] JSON-safe value or nil if conversion fails
|
73
|
+
def json_value(value)
|
74
|
+
return nil if value.nil?
|
75
|
+
return value if JSON_SAFE_TYPES.include?(value.class)
|
76
|
+
|
77
|
+
begin
|
78
|
+
JSON.parse(JSON.generate(value))
|
79
|
+
rescue JSON::JSONError
|
80
|
+
nil
|
81
|
+
end
|
82
|
+
end
|
83
|
+
end
|
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
# This is a replacement for Sidekiq's built in JobLogger. Like the built in JobLogger, it
|
4
4
|
# will log job lifecycle events (start, end, failure) with timing information and job metadata.
|
5
|
-
# It the standard metadata for jobs:
|
5
|
+
# It will include the standard metadata for jobs:
|
6
6
|
# - Job class name
|
7
7
|
# - Job ID
|
8
8
|
# - Duration of job execution
|
@@ -13,7 +13,7 @@
|
|
13
13
|
# - Retry count
|
14
14
|
# - Enqueued time in milliseconds (if available)
|
15
15
|
#
|
16
|
-
# Log messages will also include more information to be human readable
|
16
|
+
# Log messages will also include more information to be human readable including the job arguments:
|
17
17
|
#
|
18
18
|
# Finished Sidekiq job MyWorker.perform("foo", 12)`
|
19
19
|
#
|
@@ -21,18 +21,29 @@
|
|
21
21
|
#
|
22
22
|
# sidekiq_options logging: {args: [:arg1]} # only `arg1` will appear in the logs
|
23
23
|
#
|
24
|
-
# @example
|
25
|
-
#
|
26
|
-
#
|
27
|
-
#
|
24
|
+
# @example Setting up the job logger
|
25
|
+
# Sidekiq.configure_server do |config|
|
26
|
+
# config.logger = Lumberjack::Logger.new(STDOUT)
|
27
|
+
# config[:job_logger] = Lumberjack::Sidekiq::JobLogger
|
28
|
+
# end
|
28
29
|
class Lumberjack::Sidekiq::JobLogger
|
30
|
+
# Creates a new JobLogger instance.
|
31
|
+
#
|
32
|
+
# @param config [Sidekiq::Config] The Sidekiq configuration object
|
29
33
|
def initialize(config)
|
30
34
|
@config = config
|
31
35
|
@logger = @config.logger
|
32
|
-
@prefix = @config[:
|
36
|
+
@prefix = @config[:log_attribute_prefix] || ""
|
33
37
|
@message_formatter = @config[:job_logger_message_formatter] || Lumberjack::Sidekiq::MessageFormatter.new(@config)
|
34
38
|
end
|
35
39
|
|
40
|
+
# Sidekiq server middleware hook that logs job lifecycle events (start, completion, failure)
|
41
|
+
# with timing information and job metadata.
|
42
|
+
#
|
43
|
+
# @param job [Hash] The job hash containing job data
|
44
|
+
# @param _queue [String] The queue name (unused)
|
45
|
+
# @yield The job execution block
|
46
|
+
# @return [void]
|
36
47
|
def call(job, _queue)
|
37
48
|
enqueued_time = enqueued_time_ms(job) unless skip_enqueued_time_logging?
|
38
49
|
begin
|
@@ -49,7 +60,10 @@ class Lumberjack::Sidekiq::JobLogger
|
|
49
60
|
end
|
50
61
|
end
|
51
62
|
|
52
|
-
#
|
63
|
+
# Determines if start job logging should be skipped for the given job.
|
64
|
+
#
|
65
|
+
# @param job [Hash] The job hash containing job data
|
66
|
+
# @return [Boolean] true if start logging should be skipped
|
53
67
|
def skip_start_job_logging?(job)
|
54
68
|
return true if @config[:skip_start_job_logging]
|
55
69
|
return true if skip_logging?(job)
|
@@ -60,6 +74,10 @@ class Lumberjack::Sidekiq::JobLogger
|
|
60
74
|
!!logging_options["skip_start"]
|
61
75
|
end
|
62
76
|
|
77
|
+
# Determines if logging should be skipped entirely for the given job.
|
78
|
+
#
|
79
|
+
# @param job [Hash] The job hash containing job data
|
80
|
+
# @return [Boolean] true if logging should be skipped
|
63
81
|
def skip_logging?(job)
|
64
82
|
logging_options = job["logging"]
|
65
83
|
return false unless logging_options.is_a?(Hash)
|
@@ -67,73 +85,107 @@ class Lumberjack::Sidekiq::JobLogger
|
|
67
85
|
!!logging_options["skip"]
|
68
86
|
end
|
69
87
|
|
88
|
+
# Determines if enqueued time logging should be skipped globally.
|
89
|
+
#
|
90
|
+
# @return [Boolean] true if enqueued time logging should be skipped
|
70
91
|
def skip_enqueued_time_logging?
|
71
92
|
@config[:skip_enqueued_time_logging] || false
|
72
93
|
end
|
73
94
|
|
95
|
+
# Prepares the logging context for a job by setting up Lumberjack attributes and
|
96
|
+
# executing the block within that context. This includes job metadata like class name,
|
97
|
+
# job ID, and any attributes passed through from the client.
|
98
|
+
#
|
99
|
+
# @param job [Hash] The job hash containing job data
|
100
|
+
# @yield The block to execute within the logging context
|
101
|
+
# @return [void]
|
74
102
|
def prepare(job, &block)
|
75
103
|
return yield unless @logger.is_a?(Lumberjack::Logger)
|
76
104
|
|
77
|
-
|
105
|
+
attributes = {
|
78
106
|
"#{@prefix}class" => worker_class(job),
|
79
107
|
"#{@prefix}jid" => job["jid"]
|
80
108
|
}
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
|
88
|
-
|
89
|
-
|
90
|
-
|
91
|
-
|
92
|
-
|
109
|
+
attributes["#{@prefix}bid"] = job["bid"] if job.include?("bid")
|
110
|
+
attributes["#{@prefix}attributes"] = job["attributes"] if job.include?("attributes")
|
111
|
+
|
112
|
+
persisted_attributes = passthrough_attributes(job)
|
113
|
+
attributes.merge!(persisted_attributes) if persisted_attributes.is_a?(Hash)
|
114
|
+
|
115
|
+
Lumberjack.context do
|
116
|
+
@logger.tag(attributes) do
|
117
|
+
level = job.dig("logging", "level") || job["log_level"]
|
118
|
+
if level
|
119
|
+
@logger.with_level(level, &block)
|
120
|
+
else
|
121
|
+
yield
|
122
|
+
end
|
93
123
|
end
|
94
124
|
end
|
95
125
|
end
|
96
126
|
|
97
127
|
private
|
98
128
|
|
129
|
+
# Logs the start of a job.
|
130
|
+
#
|
131
|
+
# @param job [Hash] The job hash containing job data
|
99
132
|
def log_start_job(job)
|
100
133
|
message = @message_formatter.start_job(job)
|
101
134
|
if @logger.is_a?(Lumberjack::Logger)
|
102
|
-
|
103
|
-
@logger.info(message,
|
135
|
+
attributes = job_attributes(job)
|
136
|
+
@logger.info(message, attributes)
|
104
137
|
else
|
105
138
|
@logger.info(message)
|
106
139
|
end
|
107
140
|
end
|
108
141
|
|
142
|
+
# Logs the successful completion of a job.
|
143
|
+
#
|
144
|
+
# @param job [Hash] The job hash containing job data
|
145
|
+
# @param start [Float] The start time from Process.clock_gettime
|
146
|
+
# @param enqueued_time [Integer, nil] The enqueued time in milliseconds
|
109
147
|
def log_end_job(job, start, enqueued_time)
|
110
148
|
message = @message_formatter.end_job(job, elapsed_time(start))
|
111
149
|
if @logger.is_a?(Lumberjack::Logger)
|
112
|
-
|
113
|
-
|
114
|
-
|
115
|
-
@logger.info(message,
|
150
|
+
attributes = job_attributes(job)
|
151
|
+
attributes["#{@prefix}duration"] = elapsed_time(start)
|
152
|
+
attributes["#{@prefix}enqueued_ms"] = enqueued_time if enqueued_time
|
153
|
+
@logger.info(message, attributes)
|
116
154
|
else
|
117
155
|
@logger.info(message)
|
118
156
|
end
|
119
157
|
end
|
120
158
|
|
159
|
+
# Logs the failure of a job.
|
160
|
+
#
|
161
|
+
# @param job [Hash] The job hash containing job data
|
162
|
+
# @param err [Exception] The exception that caused the failure
|
163
|
+
# @param start [Float] The start time from Process.clock_gettime
|
164
|
+
# @param enqueued_time [Integer, nil] The enqueued time in milliseconds
|
121
165
|
def log_failed_job(job, err, start, enqueued_time)
|
122
166
|
message = @message_formatter.failed_job(job, err, elapsed_time(start))
|
123
167
|
if @logger.is_a?(Lumberjack::Logger)
|
124
|
-
|
125
|
-
|
126
|
-
|
127
|
-
@logger.error(message,
|
168
|
+
attributes = job_attributes(job)
|
169
|
+
attributes["#{@prefix}duration"] = elapsed_time(start)
|
170
|
+
attributes["#{@prefix}enqueued_ms"] = enqueued_time if enqueued_time
|
171
|
+
@logger.error(message, attributes)
|
128
172
|
else
|
129
173
|
@logger.error(message)
|
130
174
|
end
|
131
175
|
end
|
132
176
|
|
177
|
+
# Calculates the elapsed time since start.
|
178
|
+
#
|
179
|
+
# @param start [Float] The start time from Process.clock_gettime
|
180
|
+
# @return [Float] The elapsed time in seconds
|
133
181
|
def elapsed_time(start)
|
134
182
|
(::Process.clock_gettime(::Process::CLOCK_MONOTONIC) - start).round(6)
|
135
183
|
end
|
136
184
|
|
185
|
+
# Calculates the enqueued time in milliseconds.
|
186
|
+
#
|
187
|
+
# @param job [Hash] The job hash containing job data
|
188
|
+
# @return [Integer, nil] The enqueued time in milliseconds or nil if not available
|
137
189
|
def enqueued_time_ms(job)
|
138
190
|
enqueued_at = job["enqueued_at"]
|
139
191
|
return nil unless enqueued_at.is_a?(Numeric)
|
@@ -146,26 +198,38 @@ class Lumberjack::Sidekiq::JobLogger
|
|
146
198
|
enqueued_ms
|
147
199
|
end
|
148
200
|
|
149
|
-
|
150
|
-
|
201
|
+
# Builds job attributes hash for logging.
|
202
|
+
#
|
203
|
+
# @param job [Hash] The job hash containing job data
|
204
|
+
# @return [Hash] Hash of attributes to add to the log entry
|
205
|
+
def job_attributes(job)
|
206
|
+
attributes = {}
|
151
207
|
|
152
208
|
retry_count = job["retry_count"]
|
153
|
-
|
209
|
+
attributes["#{@prefix}retry_count"] = retry_count if retry_count && retry_count > 0
|
154
210
|
|
155
|
-
|
211
|
+
attributes["#{@prefix}queue"] = job["queue"] if job["queue"]
|
156
212
|
|
157
|
-
::Sidekiq::Context.current&.each do |
|
158
|
-
|
213
|
+
::Sidekiq::Context.current&.each do |attribute, value|
|
214
|
+
attributes["#{@prefix}#{attribute}"] = value
|
159
215
|
end
|
160
216
|
|
161
|
-
|
217
|
+
attributes
|
162
218
|
end
|
163
219
|
|
220
|
+
# Extracts the worker class name from job data.
|
221
|
+
#
|
222
|
+
# @param job [Hash] The job hash containing job data
|
223
|
+
# @return [String] The worker class name
|
164
224
|
def worker_class(job)
|
165
225
|
job["display_class"] || job["wrapped"] || job["class"]
|
166
226
|
end
|
167
227
|
|
168
|
-
|
169
|
-
|
228
|
+
# Extracts passthrough attributes from job logging configuration.
|
229
|
+
#
|
230
|
+
# @param job [Hash] The job hash containing job data
|
231
|
+
# @return [Hash, nil] The passthrough attributes or nil if none
|
232
|
+
def passthrough_attributes(job)
|
233
|
+
job.dig("logging", "attributes")
|
170
234
|
end
|
171
235
|
end
|
@@ -19,7 +19,7 @@ module Lumberjack::Sidekiq
|
|
19
19
|
# `end_job`, and `failed_job` methods and set it in your Sidekiq configuration:
|
20
20
|
#
|
21
21
|
# Sidekiq.configure_server do |config|
|
22
|
-
# config
|
22
|
+
# config[:job_logger_message_formatter] = MyCustomFormatter.new(config)
|
23
23
|
# end
|
24
24
|
class MessageFormatter
|
25
25
|
# @param config [::Sidekiq::Config] The Sidekiq configuration.
|
@@ -89,7 +89,7 @@ module Lumberjack::Sidekiq
|
|
89
89
|
end
|
90
90
|
end
|
91
91
|
|
92
|
-
# Returns true
|
92
|
+
# Returns true if job arguments should never be logged.
|
93
93
|
#
|
94
94
|
# @return [Boolean] True if job arguments should not be logged.
|
95
95
|
def skip_logging_job_arguments?
|
@@ -98,7 +98,7 @@ module Lumberjack::Sidekiq
|
|
98
98
|
|
99
99
|
# Helper method to get the job worker class name. If the job has a `display_class` or `wrapped` key,
|
100
100
|
# it will return that value for logging purposes.
|
101
|
-
#
|
101
|
+
#
|
102
102
|
# @param job [Hash] The job data.
|
103
103
|
# @return [String] The worker class name.
|
104
104
|
def worker_class(job)
|
@@ -107,6 +107,12 @@ module Lumberjack::Sidekiq
|
|
107
107
|
|
108
108
|
private
|
109
109
|
|
110
|
+
# Filters job arguments based on the args filter configuration.
|
111
|
+
#
|
112
|
+
# @param job [Hash] The job data
|
113
|
+
# @param args [Array] The job arguments
|
114
|
+
# @param args_filter [Array] The list of argument names to include
|
115
|
+
# @return [Array<String>] The filtered arguments for display
|
110
116
|
def filtered_args(job, args, args_filter)
|
111
117
|
class_name = job["wrapped"] || job["class"]
|
112
118
|
klass = Object.const_get(class_name) if class_name && Object.const_defined?(class_name)
|
data/lib/lumberjack/sidekiq.rb
CHANGED
@@ -3,9 +3,12 @@
|
|
3
3
|
require "lumberjack"
|
4
4
|
require "sidekiq"
|
5
5
|
|
6
|
+
# Lumberjack Sidekiq integration module that provides enhanced logging capabilities
|
7
|
+
# for Sidekiq jobs with support for structured logging and attribute passthrough.
|
6
8
|
module Lumberjack::Sidekiq
|
9
|
+
VERSION = File.read(File.expand_path("../../../VERSION", __FILE__)).strip.freeze
|
7
10
|
end
|
8
11
|
|
9
12
|
require_relative "sidekiq/job_logger"
|
10
13
|
require_relative "sidekiq/message_formatter"
|
11
|
-
require_relative "sidekiq/
|
14
|
+
require_relative "sidekiq/attribute_passthrough_middleware"
|
data/lumberjack_sidekiq.gemspec
CHANGED
@@ -4,7 +4,7 @@ Gem::Specification.new do |spec|
|
|
4
4
|
spec.authors = ["Brian Durand"]
|
5
5
|
spec.email = ["bbdurand@gmail.com"]
|
6
6
|
|
7
|
-
spec.summary = "Structured logging for Sidekiq jobs using the Lumberjack framework with automatic
|
7
|
+
spec.summary = "Structured logging for Sidekiq jobs using the Lumberjack framework with automatic attributeging, timing, and context propagation."
|
8
8
|
spec.homepage = "https://github.com/bdurand/lumberjack_sidekiq"
|
9
9
|
spec.license = "MIT"
|
10
10
|
|
@@ -28,6 +28,6 @@ Gem::Specification.new do |spec|
|
|
28
28
|
|
29
29
|
spec.required_ruby_version = ">= 2.7"
|
30
30
|
|
31
|
-
spec.add_dependency "lumberjack", ">=
|
31
|
+
spec.add_dependency "lumberjack", ">=2.0"
|
32
32
|
spec.add_dependency "sidekiq", ">=7.0"
|
33
33
|
end
|
metadata
CHANGED
@@ -1,14 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lumberjack_sidekiq
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 2.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Brian Durand
|
8
|
-
autorequire:
|
9
8
|
bindir: bin
|
10
9
|
cert_chain: []
|
11
|
-
date:
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
12
11
|
dependencies:
|
13
12
|
- !ruby/object:Gem::Dependency
|
14
13
|
name: lumberjack
|
@@ -16,14 +15,14 @@ dependencies:
|
|
16
15
|
requirements:
|
17
16
|
- - ">="
|
18
17
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
18
|
+
version: '2.0'
|
20
19
|
type: :runtime
|
21
20
|
prerelease: false
|
22
21
|
version_requirements: !ruby/object:Gem::Requirement
|
23
22
|
requirements:
|
24
23
|
- - ">="
|
25
24
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
25
|
+
version: '2.0'
|
27
26
|
- !ruby/object:Gem::Dependency
|
28
27
|
name: sidekiq
|
29
28
|
requirement: !ruby/object:Gem::Requirement
|
@@ -38,7 +37,6 @@ dependencies:
|
|
38
37
|
- - ">="
|
39
38
|
- !ruby/object:Gem::Version
|
40
39
|
version: '7.0'
|
41
|
-
description:
|
42
40
|
email:
|
43
41
|
- bbdurand@gmail.com
|
44
42
|
executables: []
|
@@ -53,16 +51,15 @@ files:
|
|
53
51
|
- README.md
|
54
52
|
- VERSION
|
55
53
|
- lib/lumberjack/sidekiq.rb
|
54
|
+
- lib/lumberjack/sidekiq/attribute_passthrough_middleware.rb
|
56
55
|
- lib/lumberjack/sidekiq/job_logger.rb
|
57
56
|
- lib/lumberjack/sidekiq/message_formatter.rb
|
58
|
-
- lib/lumberjack/sidekiq/tag_passthrough_middleware.rb
|
59
57
|
- lib/lumberjack_sidekiq.rb
|
60
58
|
- lumberjack_sidekiq.gemspec
|
61
59
|
homepage: https://github.com/bdurand/lumberjack_sidekiq
|
62
60
|
licenses:
|
63
61
|
- MIT
|
64
62
|
metadata: {}
|
65
|
-
post_install_message:
|
66
63
|
rdoc_options: []
|
67
64
|
require_paths:
|
68
65
|
- lib
|
@@ -77,9 +74,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
77
74
|
- !ruby/object:Gem::Version
|
78
75
|
version: '0'
|
79
76
|
requirements: []
|
80
|
-
rubygems_version: 3.
|
81
|
-
signing_key:
|
77
|
+
rubygems_version: 3.6.9
|
82
78
|
specification_version: 4
|
83
79
|
summary: Structured logging for Sidekiq jobs using the Lumberjack framework with automatic
|
84
|
-
|
80
|
+
attributeging, timing, and context propagation.
|
85
81
|
test_files: []
|
@@ -1,54 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
require "json"
|
4
|
-
|
5
|
-
# Sidekiq client middleware that can pass through log tags from the current Lumberjack
|
6
|
-
# logger to job logger when the job is executed on the Sidekiq server. This can be
|
7
|
-
# useful to maintain context in logs when a job is executed.
|
8
|
-
#
|
9
|
-
# @example
|
10
|
-
# Sidekiq.configure_client do |config|
|
11
|
-
# config.client_middleware do |chain|
|
12
|
-
# # Pass through :user_id and :request_id tags to the job logger.
|
13
|
-
# chain.add(Lumberjack::Sidekiq::TagPassthroughMiddleware, :user_id, :request_id)
|
14
|
-
# end
|
15
|
-
# end
|
16
|
-
class Lumberjack::Sidekiq::TagPassthroughMiddleware
|
17
|
-
include ::Sidekiq::ClientMiddleware
|
18
|
-
|
19
|
-
JSON_SAFE_TYPES = [String, Integer, Float, TrueClass, FalseClass].freeze
|
20
|
-
|
21
|
-
# @param pass_through_tags [Array<String, Symbol>] Log tags to pass through to the job logger when the job is executed.
|
22
|
-
def initialize(*pass_through_tags)
|
23
|
-
@pass_through_tags = pass_through_tags.flatten.map(&:to_s)
|
24
|
-
end
|
25
|
-
|
26
|
-
def call(job_class_or_string, job, queue, redis_pool)
|
27
|
-
return yield unless Sidekiq.logger.is_a?(Lumberjack::Logger)
|
28
|
-
|
29
|
-
job["logging"] ||= {}
|
30
|
-
tags = job["logging"]["tags"] || {}
|
31
|
-
|
32
|
-
@pass_through_tags.each do |tag|
|
33
|
-
value = json_value(Sidekiq.logger.tag_value(tag))
|
34
|
-
tags[tag] = value unless value.nil?
|
35
|
-
end
|
36
|
-
|
37
|
-
job["logging"]["tags"] = tags unless tags.empty?
|
38
|
-
|
39
|
-
yield
|
40
|
-
end
|
41
|
-
|
42
|
-
private
|
43
|
-
|
44
|
-
def json_value(value)
|
45
|
-
return nil if value.nil?
|
46
|
-
return value if JSON_SAFE_TYPES.include?(value.class)
|
47
|
-
|
48
|
-
begin
|
49
|
-
JSON.parse(JSON.generate(value))
|
50
|
-
rescue JSON::JSONError
|
51
|
-
nil
|
52
|
-
end
|
53
|
-
end
|
54
|
-
end
|