activesupport-json_logging 1.3.1 → 1.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -3
- data/README.md +30 -16
- data/lib/json_logging/execution_state.rb +65 -0
- data/lib/json_logging/helpers.rb +17 -1
- data/lib/json_logging/json_logger.rb +1 -1
- data/lib/json_logging/json_logger_extension.rb +11 -18
- data/lib/json_logging/line_encoder.rb +27 -16
- data/lib/json_logging/message_parser.rb +2 -0
- data/lib/json_logging/payload_builder.rb +29 -15
- data/lib/json_logging/sanitizer.rb +27 -3
- data/lib/json_logging/structured_hash_json_encoder.rb +6 -2
- data/lib/json_logging/structured_hash_sanitizer.rb +2 -0
- data/lib/json_logging/version.rb +1 -1
- data/lib/json_logging.rb +20 -5
- data/sig/json_logging.rbs +4 -0
- metadata +6 -19
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 34ff048fcdd516e4f0f8e2b5d6d9911e44a6a486ee166ee3287b5be217140385
|
|
4
|
+
data.tar.gz: af1f7dac1075da66a5e2fa4135d143cc58521f3cc848f9d40aeb54254f8cf907
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f3811c1fbf10caf6931e5eca77c645360def106697f651d7d6ba313415a5630f07517f0be570689bf25d0f8704c586614f23d71f75488211158b4d1783f47091
|
|
7
|
+
data.tar.gz: 5982806cd91b314b756e7d25dd9c2e6af92e58cfb41bef9413eec0621f18b76faa6c78d739329794a4bfdfaee0333f06e255b542233c9936fc686a30d634ca67
|
data/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,15 @@
|
|
|
1
1
|
# CHANGELOG
|
|
2
2
|
|
|
3
|
-
##
|
|
3
|
+
## 1.3.2 (2026-11-07)
|
|
4
|
+
|
|
5
|
+
- Prefer `JSON.fast_generate` on generic `to_json_line`, structured-hash encoder assembly, and `LineEncoder` fast-path lines
|
|
6
|
+
- Keep hot-path optimizations always on: timestamp cache per UTC second, sanitizer and context fast paths, tag-list cache, single `Severity.name_for` per line
|
|
7
|
+
- Experimental: add `JsonLogging.isolation_level = :fiber` so context, tags, and hot-path caches isolate per Fiber (`:thread` remains default)
|
|
8
|
+
- Emit valid JSON when a large structured hash body is empty after field overrides
|
|
9
|
+
- Resolve the home-directory prefix without ActiveSupport `blank?`
|
|
10
|
+
- Stop release when confirmation input closes before an answer
|
|
11
|
+
- Require a matching `CHANGELOG.md` version heading before publish
|
|
12
|
+
- Run quality, test, and coverage jobs on pull requests to `main`
|
|
4
13
|
|
|
5
14
|
## 1.3.1 (2026-09-04)
|
|
6
15
|
|
|
@@ -17,7 +26,7 @@
|
|
|
17
26
|
|
|
18
27
|
## 1.3.0 (2026-07-25)
|
|
19
28
|
|
|
20
|
-
- BREAKING: Require Ruby 3.2+ and Rails 7+ (drop Ruby 2.7
|
|
29
|
+
- BREAKING: Require Ruby 3.2+ and Rails 7+ (drop Ruby 2.7-3.1 and Rails 6 support)
|
|
21
30
|
- Add opt-in `JsonLogging::EventSubscriber` for Rails 8.1 `ActiveSupport::EventReporter` / `Rails.event` structured events
|
|
22
31
|
|
|
23
32
|
## 1.2.3 (2026-07-13)
|
|
@@ -51,7 +60,7 @@
|
|
|
51
60
|
## 1.2.0 (2025-11-07)
|
|
52
61
|
|
|
53
62
|
- Add support for service-specific tagged loggers: create loggers with permanent tags using `logger.tagged("service")` without a block
|
|
54
|
-
- Improve BroadcastLogger compatibility: service-specific loggers work
|
|
63
|
+
- Improve BroadcastLogger compatibility: service-specific loggers work with `ActiveSupport::BroadcastLogger`
|
|
55
64
|
- Fix LocalTagStorage implementation to match Rails' TaggedLogging behavior: use `tag_stack` attribute accessor pattern for proper tag isolation
|
|
56
65
|
- Add comprehensive examples in README for service-specific loggers and BroadcastLogger integration
|
|
57
66
|
|
data/README.md
CHANGED
|
@@ -3,8 +3,8 @@
|
|
|
3
3
|
[](https://badge.fury.io/rb/activesupport-json_logging) [](https://github.com/amkisko/activesupport-json_logging.rb/actions/workflows/test.yml) [](https://app.codecov.io/github/amkisko/activesupport-json_logging.rb) [](https://sonarcloud.io/project/overview?id=amkisko_activesupport-json_logging.rb)
|
|
4
4
|
|
|
5
5
|
Structured JSON logging for Rails and ActiveSupport with a safe, single-line formatter.
|
|
6
|
-
No dependencies beyond Rails and
|
|
7
|
-
Supports Ruby 3.2+ and Rails 7
|
|
6
|
+
No dependencies beyond Rails and ActiveSupport.
|
|
7
|
+
Supports Ruby 3.2+ and Rails 7-8.
|
|
8
8
|
|
|
9
9
|
## Installation
|
|
10
10
|
|
|
@@ -35,10 +35,10 @@ end
|
|
|
35
35
|
- `JsonLogging.logger(*args)` - Convenience method that creates an `ActiveSupport::Logger` and wraps it
|
|
36
36
|
- Safe JSON serialization that never raises from the formatter
|
|
37
37
|
- `JsonLogging.with_context` to attach contextual fields per-thread
|
|
38
|
-
-
|
|
38
|
+
- Message parsing for hashes, JSON strings, plain strings, and Exception objects
|
|
39
39
|
- Native `tagged` method support - use it just like Rails' tagged logger
|
|
40
40
|
- Service-specific tagged loggers - create loggers with permanent tags using `logger.tagged("service")` without a block
|
|
41
|
-
-
|
|
41
|
+
- Compatible with `ActiveSupport::BroadcastLogger` (Rails 7.1+)
|
|
42
42
|
- Opt-in `JsonLogging::EventSubscriber` for Rails 8.1 `ActiveSupport::EventReporter` / `Rails.event`
|
|
43
43
|
- Automatic Rails integration via Railtie (auto-requires the gem in Rails apps)
|
|
44
44
|
|
|
@@ -93,8 +93,21 @@ This gem does **not** automatically configure your Rails app. You set it up manu
|
|
|
93
93
|
|
|
94
94
|
**Note:**
|
|
95
95
|
- In Rails apps, the gem is automatically required via Railtie, so you typically don't need to manually `require "json_logging"` in initializers (though it's harmless if you do).
|
|
96
|
-
- In Rails 7.1+, Rails
|
|
97
|
-
- In Rails 7.1+, tag storage uses `ActiveSupport::IsolatedExecutionState`
|
|
96
|
+
- In Rails 7.1+, Rails wraps the logger in `ActiveSupport::BroadcastLogger` so one logger can write to several destinations (for example STDOUT and a file). `JsonLogging` stays compatible: the JSON logger is wrapped and method calls delegate as usual.
|
|
97
|
+
- In Rails 7.1+, default tag/context storage uses `ActiveSupport::IsolatedExecutionState` (thread isolation unless the app sets Rails fiber isolation).
|
|
98
|
+
|
|
99
|
+
### Experimental Fiber isolation
|
|
100
|
+
|
|
101
|
+
For Fiber-based servers or code that interleaves many Fibers on one Thread, enable JsonLogging's experimental fiber storage:
|
|
102
|
+
|
|
103
|
+
```ruby
|
|
104
|
+
# config/initializers/json_logging.rb
|
|
105
|
+
JsonLogging.isolation_level = :fiber
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
This isolates `with_context`, logger tags, and JsonLogging hot-path caches per Fiber. It does **not** flip Rails' own `config.active_support.isolation_level`; Falcon-style apps usually set both.
|
|
109
|
+
|
|
110
|
+
Default remains `:thread`.
|
|
98
111
|
|
|
99
112
|
### Service-specific loggers with tags
|
|
100
113
|
|
|
@@ -159,7 +172,7 @@ end
|
|
|
159
172
|
|
|
160
173
|
### BroadcastLogger integration
|
|
161
174
|
|
|
162
|
-
`ActiveSupport::BroadcastLogger` (Rails 7.1+)
|
|
175
|
+
`ActiveSupport::BroadcastLogger` (Rails 7.1+) writes the same log stream to multiple destinations. `JsonLogging` works with `BroadcastLogger`:
|
|
163
176
|
|
|
164
177
|
```ruby
|
|
165
178
|
# Create JSON loggers for different destinations
|
|
@@ -203,7 +216,7 @@ end
|
|
|
203
216
|
- Tagged logging (`tagged`) works correctly through `BroadcastLogger`
|
|
204
217
|
- Service-specific tagged loggers work with `BroadcastLogger`
|
|
205
218
|
- Each destination receives properly formatted JSON logs
|
|
206
|
-
-
|
|
219
|
+
- Wrap your `JsonLogging` logger in `BroadcastLogger` when you need multiple destinations
|
|
207
220
|
|
|
208
221
|
### Basic setup
|
|
209
222
|
|
|
@@ -443,13 +456,13 @@ logger.tagged("BCX").tagged("Jason").info("Stuff")
|
|
|
443
456
|
**Wrapping Compatibility:** You can wrap loggers that have already been wrapped with `ActiveSupport::TaggedLogging`:
|
|
444
457
|
|
|
445
458
|
```ruby
|
|
446
|
-
# Wrap a TaggedLogging logger
|
|
459
|
+
# Wrap a TaggedLogging logger
|
|
447
460
|
tagged_logger = ActiveSupport::TaggedLogging.new(Logger.new(STDOUT))
|
|
448
461
|
json_logger = JsonLogging.new(tagged_logger)
|
|
449
462
|
json_logger.tagged("TEST") { json_logger.info("message") } # Tags appear at root level in JSON
|
|
450
463
|
```
|
|
451
464
|
|
|
452
|
-
**Note:** If you wrap a `JsonLogging` logger with `ActiveSupport::TaggedLogging`, the TaggedLogging's text-based tags will appear as part of the message string in the JSON output, not as structured tags at the root level.
|
|
465
|
+
**Note:** If you wrap a `JsonLogging` logger with `ActiveSupport::TaggedLogging`, the TaggedLogging's text-based tags will appear as part of the message string in the JSON output, not as structured tags at the root level. Wrap loggers with `JsonLogging` last.
|
|
453
466
|
|
|
454
467
|
### JsonLogging::JsonLogger
|
|
455
468
|
|
|
@@ -551,7 +564,7 @@ end
|
|
|
551
564
|
|
|
552
565
|
The transformer receives the current thread-local context hash and should return a hash. If the transformer raises an error, the base context will be returned to avoid breaking logging.
|
|
553
566
|
|
|
554
|
-
**Note:** The transformer is called every time a log entry is created, so keep it
|
|
567
|
+
**Note:** The transformer is called every time a log entry is created, so keep it cheap to avoid slowing each line.
|
|
555
568
|
|
|
556
569
|
### Inherited Rails Logger Features
|
|
557
570
|
|
|
@@ -607,10 +620,11 @@ logger.reopen # Reopen the logger (if supported by logdev)
|
|
|
607
620
|
## Features
|
|
608
621
|
|
|
609
622
|
- **Native tagged logging**: Use `logger.tagged("TAG")` just like Rails' tagged logger
|
|
610
|
-
- **
|
|
623
|
+
- **Message parsing**: Handles hashes, JSON strings, plain strings, and Exception objects (with class, message, and backtrace)
|
|
611
624
|
- **Thread-safe context**: `JsonLogging.with_context` works across threads
|
|
612
|
-
- **Rails 7.1+ thread
|
|
613
|
-
- **
|
|
625
|
+
- **Rails 7.1+ isolation**: Default `:thread` storage uses `ActiveSupport::IsolatedExecutionState` when available
|
|
626
|
+
- **Experimental Fiber isolation**: `JsonLogging.isolation_level = :fiber` isolates context, tags, and hot-path caches per Fiber (for Fiber servers / interleaved Fibers on one Thread). Does not change Rails `config.active_support.isolation_level`; set that separately for CurrentAttributes and the rest of Rails.
|
|
627
|
+
- **Safe on encode errors**: Formatter and logger methods fall back to safe entries instead of raising
|
|
614
628
|
- **Single-line JSON**: Each log entry is a single line, safe for log aggregation tools
|
|
615
629
|
|
|
616
630
|
## Security & privacy
|
|
@@ -622,9 +636,9 @@ logger.reopen # Reopen the logger (if supported by logdev)
|
|
|
622
636
|
- Maximum nesting depth: 10 levels (deeper structures return `{"error" => "max_depth_exceeded"}`)
|
|
623
637
|
- Maximum backtrace lines: 20 lines per exception
|
|
624
638
|
- **Single-line JSON**: Emits single-line JSON to avoid log injection via newlines
|
|
625
|
-
- **
|
|
639
|
+
- **Safe on encode errors**: Formatter and logger never raise; fallback to safe entries on serialization errors
|
|
626
640
|
- **Sensitive key detection**: Falls back to pattern matching when Rails ParameterFilter isn't available
|
|
627
|
-
- **
|
|
641
|
+
- **Caller-owned context**: You decide what goes into context via `JsonLogging.with_context`; avoid sensitive data
|
|
628
642
|
|
|
629
643
|
### Configuring sensitive parameter filtering
|
|
630
644
|
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module JsonLogging
|
|
4
|
+
# Storage for request-scoped logging state (context, tags, hot-path caches).
|
|
5
|
+
#
|
|
6
|
+
# Default +:thread+ follows ActiveSupport::IsolatedExecutionState when present
|
|
7
|
+
# (Rails 7.1+), otherwise Thread.current.
|
|
8
|
+
#
|
|
9
|
+
# Experimental +:fiber+ stores JsonLogging keys on the current Fiber so concurrent
|
|
10
|
+
# Fibers on one Thread do not share context or tags. Does not change Rails'
|
|
11
|
+
# ActiveSupport::IsolatedExecutionState.isolation_level; Falcon-style apps should
|
|
12
|
+
# still set config.active_support.isolation_level = :fiber for the rest of Rails.
|
|
13
|
+
module ExecutionState
|
|
14
|
+
PREPARED_TAGS_CACHE_KEY = :__json_logging_prepared_tags
|
|
15
|
+
|
|
16
|
+
module_function
|
|
17
|
+
|
|
18
|
+
def [](key)
|
|
19
|
+
backend[key]
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def []=(key, value)
|
|
23
|
+
backend[key] = value
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def delete(key)
|
|
27
|
+
backend.delete(key)
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def key?(key)
|
|
31
|
+
backend.key?(key)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def prepared_tags_cache
|
|
35
|
+
self[PREPARED_TAGS_CACHE_KEY] ||= {}
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def backend
|
|
39
|
+
if JsonLogging.isolation_level == :fiber
|
|
40
|
+
fiber_backend
|
|
41
|
+
else
|
|
42
|
+
thread_backend
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def fiber_backend
|
|
47
|
+
state = Fiber.current.json_logging_execution_state
|
|
48
|
+
unless state
|
|
49
|
+
state = {}
|
|
50
|
+
Fiber.current.json_logging_execution_state = state
|
|
51
|
+
end
|
|
52
|
+
state
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def thread_backend
|
|
56
|
+
if defined?(ActiveSupport::IsolatedExecutionState)
|
|
57
|
+
ActiveSupport::IsolatedExecutionState
|
|
58
|
+
else
|
|
59
|
+
Thread.current
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
Fiber.attr_accessor :json_logging_execution_state unless Fiber.method_defined?(:json_logging_execution_state)
|
data/lib/json_logging/helpers.rb
CHANGED
|
@@ -2,6 +2,9 @@ module JsonLogging
|
|
|
2
2
|
module Helpers
|
|
3
3
|
ISO8601_MICROSECOND_UTC_PATTERN = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{6}Z\z/
|
|
4
4
|
|
|
5
|
+
# Microsecond bundle exec / hot add path: avoid fresh iso8601(6) every line when still in the same UTC second.
|
|
6
|
+
TIMESTAMP_CACHE_KEY = :__json_logging_ts_cache
|
|
7
|
+
|
|
5
8
|
module_function
|
|
6
9
|
|
|
7
10
|
# Normalize timestamp to ISO8601 with microseconds
|
|
@@ -12,7 +15,20 @@ module JsonLogging
|
|
|
12
15
|
end
|
|
13
16
|
|
|
14
17
|
def current_timestamp
|
|
15
|
-
|
|
18
|
+
cached_log_timestamp
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def cached_log_timestamp
|
|
22
|
+
now = current_time
|
|
23
|
+
sec_key = now.to_i
|
|
24
|
+
cache = ExecutionState[TIMESTAMP_CACHE_KEY]
|
|
25
|
+
if cache.is_a?(Array) && cache[0] == sec_key && cache[1]
|
|
26
|
+
return cache[1]
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
formatted = format_timestamp(now)
|
|
30
|
+
ExecutionState[TIMESTAMP_CACHE_KEY] = [sec_key, formatted]
|
|
31
|
+
formatted
|
|
16
32
|
end
|
|
17
33
|
|
|
18
34
|
def format_timestamp(time)
|
|
@@ -12,7 +12,7 @@ module JsonLogging
|
|
|
12
12
|
shift_age = args[1] || 0
|
|
13
13
|
shift_size = args[2]
|
|
14
14
|
|
|
15
|
-
# Handle both positional and keyword arguments for Rails 7
|
|
15
|
+
# Handle both positional and keyword arguments for Rails 7-8 compatibility
|
|
16
16
|
if kwargs.empty? && shift_size
|
|
17
17
|
super(logdev, shift_age, shift_size)
|
|
18
18
|
elsif kwargs.empty?
|
|
@@ -22,10 +22,11 @@ module JsonLogging
|
|
|
22
22
|
|
|
23
23
|
msg = nil
|
|
24
24
|
msg = extract_log_message(message, progname, &block)
|
|
25
|
+
timestamp = Helpers.cached_log_timestamp
|
|
25
26
|
line = LineEncoder.build_line(
|
|
26
27
|
msg: msg,
|
|
27
28
|
severity: severity,
|
|
28
|
-
timestamp:
|
|
29
|
+
timestamp: timestamp,
|
|
29
30
|
tags: formatter.current_tags,
|
|
30
31
|
additional_context: JsonLogging.additional_context_for_payload,
|
|
31
32
|
additional_context_sanitized: true,
|
|
@@ -77,8 +78,9 @@ module JsonLogging
|
|
|
77
78
|
|
|
78
79
|
def write_add_fallback(severity, msg, error)
|
|
79
80
|
msg ||= "<uninitialized>"
|
|
81
|
+
ts = Helpers.cached_log_timestamp
|
|
80
82
|
fallback = {
|
|
81
|
-
timestamp:
|
|
83
|
+
timestamp: ts,
|
|
82
84
|
severity: Severity.name_for(severity),
|
|
83
85
|
message: Sanitizer.sanitize_string(Helpers.safe_string(msg)),
|
|
84
86
|
logger_error: {
|
|
@@ -94,27 +96,18 @@ module JsonLogging
|
|
|
94
96
|
end
|
|
95
97
|
|
|
96
98
|
def current_tags
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
else
|
|
102
|
-
Thread.current[tags_key] ||= []
|
|
103
|
-
end
|
|
99
|
+
tags = ExecutionState[tags_key]
|
|
100
|
+
return tags if tags
|
|
101
|
+
|
|
102
|
+
ExecutionState[tags_key] = []
|
|
104
103
|
end
|
|
105
104
|
|
|
106
105
|
def set_tags(new_tags)
|
|
107
|
-
|
|
108
|
-
# Falls back to Thread.current for Rails 7.0
|
|
109
|
-
if defined?(ActiveSupport::IsolatedExecutionState)
|
|
110
|
-
ActiveSupport::IsolatedExecutionState[tags_key] = new_tags
|
|
111
|
-
else
|
|
112
|
-
Thread.current[tags_key] = new_tags
|
|
113
|
-
end
|
|
106
|
+
ExecutionState[tags_key] = new_tags
|
|
114
107
|
end
|
|
115
108
|
|
|
116
109
|
def push_tags(tags)
|
|
117
|
-
flat = Sanitizer.prepare_tags(tags)
|
|
110
|
+
flat = Sanitizer.prepare_tags(tags, tag_source: :logger_extension_push)
|
|
118
111
|
return if flat.empty?
|
|
119
112
|
|
|
120
113
|
set_tags(current_tags + flat)
|
|
@@ -154,7 +147,7 @@ module JsonLogging
|
|
|
154
147
|
end
|
|
155
148
|
|
|
156
149
|
def push_tags(tags)
|
|
157
|
-
flat = Sanitizer.prepare_tags(tags)
|
|
150
|
+
flat = Sanitizer.prepare_tags(tags, tag_source: :local_tag_stack)
|
|
158
151
|
return [] if flat.empty?
|
|
159
152
|
|
|
160
153
|
@tags.concat(flat)
|
|
@@ -10,10 +10,12 @@ module JsonLogging
|
|
|
10
10
|
module_function
|
|
11
11
|
|
|
12
12
|
def build_line(msg:, severity:, timestamp:, tags:, additional_context:, additional_context_sanitized: false, sanitize_tags: true)
|
|
13
|
+
severity_string = Severity.name_for(severity)
|
|
14
|
+
|
|
13
15
|
if simple_string_line?(msg, tags, additional_context)
|
|
14
16
|
return simple_string_json_line(
|
|
15
17
|
message: msg,
|
|
16
|
-
severity:
|
|
18
|
+
severity: severity_string,
|
|
17
19
|
timestamp: timestamp
|
|
18
20
|
)
|
|
19
21
|
end
|
|
@@ -21,7 +23,7 @@ module JsonLogging
|
|
|
21
23
|
if tagged_simple_string_line?(msg, tags, additional_context)
|
|
22
24
|
return tagged_simple_string_json_line(
|
|
23
25
|
message: msg,
|
|
24
|
-
severity:
|
|
26
|
+
severity: severity_string,
|
|
25
27
|
timestamp: timestamp,
|
|
26
28
|
tags: tags,
|
|
27
29
|
sanitize_tags: sanitize_tags
|
|
@@ -31,7 +33,7 @@ module JsonLogging
|
|
|
31
33
|
if contextual_simple_string_line?(msg, tags, additional_context, additional_context_sanitized: additional_context_sanitized)
|
|
32
34
|
return contextual_simple_string_json_line(
|
|
33
35
|
message: msg,
|
|
34
|
-
severity:
|
|
36
|
+
severity: severity_string,
|
|
35
37
|
timestamp: timestamp,
|
|
36
38
|
additional_context: additional_context
|
|
37
39
|
)
|
|
@@ -40,7 +42,7 @@ module JsonLogging
|
|
|
40
42
|
if standalone_hash_line?(msg, tags, additional_context)
|
|
41
43
|
return standalone_hash_json_line(
|
|
42
44
|
message: msg,
|
|
43
|
-
severity:
|
|
45
|
+
severity: severity_string,
|
|
44
46
|
timestamp: timestamp
|
|
45
47
|
)
|
|
46
48
|
end
|
|
@@ -48,7 +50,7 @@ module JsonLogging
|
|
|
48
50
|
if tagged_hash_line?(msg, tags, additional_context)
|
|
49
51
|
return tagged_hash_json_line(
|
|
50
52
|
message: msg,
|
|
51
|
-
severity:
|
|
53
|
+
severity: severity_string,
|
|
52
54
|
timestamp: timestamp,
|
|
53
55
|
tags: tags,
|
|
54
56
|
sanitize_tags: sanitize_tags
|
|
@@ -58,13 +60,13 @@ module JsonLogging
|
|
|
58
60
|
if contextual_hash_line?(msg, tags, additional_context, additional_context_sanitized: additional_context_sanitized)
|
|
59
61
|
return contextual_hash_json_line(
|
|
60
62
|
message: msg,
|
|
61
|
-
severity:
|
|
63
|
+
severity: severity_string,
|
|
62
64
|
timestamp: timestamp,
|
|
63
65
|
additional_context: additional_context
|
|
64
66
|
)
|
|
65
67
|
end
|
|
66
68
|
|
|
67
|
-
sev =
|
|
69
|
+
sev = severity_string
|
|
68
70
|
payload = PayloadBuilder.build_base_payload(msg, severity: sev, timestamp: timestamp)
|
|
69
71
|
unless tags.empty? && PayloadBuilder.empty_additional_context?(additional_context)
|
|
70
72
|
payload = PayloadBuilder.merge_context(
|
|
@@ -81,7 +83,7 @@ module JsonLogging
|
|
|
81
83
|
def to_json_line(payload_hash)
|
|
82
84
|
compacted = payload_hash.compact
|
|
83
85
|
json_payload = string_keyed_structure?(compacted) ? compacted : compacted.deep_stringify_keys
|
|
84
|
-
"#{json_payload
|
|
86
|
+
"#{JSON.fast_generate(json_payload)}\n"
|
|
85
87
|
end
|
|
86
88
|
|
|
87
89
|
def string_keyed_structure?(object)
|
|
@@ -104,7 +106,7 @@ module JsonLogging
|
|
|
104
106
|
|
|
105
107
|
def simple_string_json_line(message:, severity:, timestamp:)
|
|
106
108
|
sanitized_message = Sanitizer.sanitize_string(message)
|
|
107
|
-
"#{JSON.
|
|
109
|
+
"#{JSON.fast_generate("message" => sanitized_message, "severity" => severity, "timestamp" => timestamp)}\n"
|
|
108
110
|
end
|
|
109
111
|
|
|
110
112
|
def tagged_simple_string_line?(message, tags, additional_context)
|
|
@@ -116,8 +118,13 @@ module JsonLogging
|
|
|
116
118
|
|
|
117
119
|
def tagged_simple_string_json_line(message:, severity:, timestamp:, tags:, sanitize_tags:)
|
|
118
120
|
sanitized_message = Sanitizer.sanitize_string(message)
|
|
119
|
-
prepared_tags = sanitize_tags
|
|
120
|
-
|
|
121
|
+
prepared_tags = if sanitize_tags
|
|
122
|
+
t = Sanitizer.prepare_tags(tags, tag_source: :line_encoder_tagged_simple)
|
|
123
|
+
t.uniq
|
|
124
|
+
else
|
|
125
|
+
tags
|
|
126
|
+
end
|
|
127
|
+
"#{JSON.fast_generate("message" => sanitized_message, "severity" => severity, "timestamp" => timestamp, "tags" => prepared_tags)}\n"
|
|
121
128
|
end
|
|
122
129
|
|
|
123
130
|
def contextual_simple_string_line?(message, tags, additional_context, additional_context_sanitized:)
|
|
@@ -137,7 +144,7 @@ module JsonLogging
|
|
|
137
144
|
}
|
|
138
145
|
context = PayloadBuilder.context_payload_portion(additional_context)
|
|
139
146
|
payload["context"] = context unless context.empty?
|
|
140
|
-
"#{JSON.
|
|
147
|
+
"#{JSON.fast_generate(payload)}\n"
|
|
141
148
|
end
|
|
142
149
|
|
|
143
150
|
def standalone_hash_line?(message, tags, additional_context)
|
|
@@ -158,7 +165,7 @@ module JsonLogging
|
|
|
158
165
|
payload = Sanitizer.sanitize_hash(hash)
|
|
159
166
|
payload["severity"] = severity
|
|
160
167
|
payload["timestamp"] = timestamp
|
|
161
|
-
"#{JSON.
|
|
168
|
+
"#{JSON.fast_generate(payload)}\n"
|
|
162
169
|
end
|
|
163
170
|
|
|
164
171
|
def tagged_hash_line?(message, tags, additional_context)
|
|
@@ -169,7 +176,11 @@ module JsonLogging
|
|
|
169
176
|
|
|
170
177
|
def tagged_hash_json_line(message:, severity:, timestamp:, tags:, sanitize_tags:)
|
|
171
178
|
hash = message.is_a?(Hash) ? message : message.to_hash
|
|
172
|
-
prepared_tags = sanitize_tags
|
|
179
|
+
prepared_tags = if sanitize_tags
|
|
180
|
+
Sanitizer.prepare_tags(tags, tag_source: :line_encoder_tagged_hash)
|
|
181
|
+
else
|
|
182
|
+
tags
|
|
183
|
+
end
|
|
173
184
|
merged_tags = (Array(hash[:tags] || hash["tags"]) + prepared_tags).uniq
|
|
174
185
|
field_overrides = {"tags" => merged_tags}
|
|
175
186
|
|
|
@@ -185,7 +196,7 @@ module JsonLogging
|
|
|
185
196
|
payload["severity"] = severity
|
|
186
197
|
payload["timestamp"] = timestamp
|
|
187
198
|
payload["tags"] = merged_tags
|
|
188
|
-
"#{JSON.
|
|
199
|
+
"#{JSON.fast_generate(payload)}\n"
|
|
189
200
|
end
|
|
190
201
|
|
|
191
202
|
def contextual_hash_line?(message, tags, additional_context, additional_context_sanitized:)
|
|
@@ -220,7 +231,7 @@ module JsonLogging
|
|
|
220
231
|
existing_context = payload["context"].is_a?(Hash) ? payload["context"] : {}
|
|
221
232
|
payload["context"] = existing_context.merge(merged_context)
|
|
222
233
|
end
|
|
223
|
-
"#{JSON.
|
|
234
|
+
"#{JSON.fast_generate(payload)}\n"
|
|
224
235
|
end
|
|
225
236
|
|
|
226
237
|
def deep_stringify_structure(obj)
|
|
@@ -25,6 +25,8 @@ module JsonLogging
|
|
|
25
25
|
end
|
|
26
26
|
|
|
27
27
|
def json_string?(str)
|
|
28
|
+
# Short strings cannot be JSON object/array literals with both delimiters.
|
|
29
|
+
return false if str.length < 2
|
|
28
30
|
return false if str.length > Sanitizer::MAX_STRING_LENGTH
|
|
29
31
|
|
|
30
32
|
(str.start_with?("{") && str.end_with?("}")) ||
|
|
@@ -25,30 +25,42 @@ module JsonLogging
|
|
|
25
25
|
return merge_tags_only(payload, tags: tags, sanitize_tags: sanitize_tags) if tags_only_merge?(payload, additional_context: additional_context, tags: tags)
|
|
26
26
|
|
|
27
27
|
existing_context = payload["context"].is_a?(Hash) ? payload["context"] : {}
|
|
28
|
+
user_context = filtered_user_context(additional_context, sanitized: additional_context_sanitized)
|
|
29
|
+
deduped = user_context.reject { |key, _| payload.key?(key) || payload.key?(key.to_s) }
|
|
30
|
+
merged_context = existing_context.merge(deduped)
|
|
28
31
|
|
|
29
|
-
|
|
32
|
+
assign_tags!(payload, tags: tags, sanitize_tags: sanitize_tags) unless tags.empty?
|
|
33
|
+
assign_context!(payload, merged_context) unless merged_context.empty?
|
|
34
|
+
payload
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def filtered_user_context(additional_context, sanitized:)
|
|
38
|
+
context = resolve_additional_context(additional_context, sanitized: sanitized)
|
|
39
|
+
context.except(*SYSTEM_CONTROLLED_KEYS)
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def resolve_additional_context(additional_context, sanitized:)
|
|
43
|
+
if sanitized
|
|
30
44
|
additional_context.is_a?(Hash) ? additional_context : {}
|
|
31
45
|
elsif additional_context.is_a?(Hash) && !additional_context.empty?
|
|
32
46
|
Sanitizer.sanitize_hash(additional_context)
|
|
33
47
|
else
|
|
34
48
|
additional_context || {}
|
|
35
49
|
end
|
|
50
|
+
end
|
|
36
51
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
unless tags.empty?
|
|
43
|
-
existing_tags = Array(payload["tags"])
|
|
44
|
-
prepared_tags = sanitize_tags ? tags.map { |tag| Sanitizer.sanitize_string(tag.to_s) } : tags
|
|
45
|
-
payload["tags"] = (existing_tags + prepared_tags).uniq
|
|
46
|
-
end
|
|
52
|
+
def assign_tags!(payload, tags:, sanitize_tags:)
|
|
53
|
+
existing_tags = Array(payload["tags"])
|
|
54
|
+
prepared_tags = sanitize_tags ? tags.map { |tag| Sanitizer.sanitize_string(tag.to_s) } : tags
|
|
55
|
+
payload["tags"] = (existing_tags + prepared_tags).uniq
|
|
56
|
+
end
|
|
47
57
|
|
|
48
|
-
|
|
49
|
-
|
|
58
|
+
def assign_context!(payload, merged_context)
|
|
59
|
+
payload["context"] = if merged_context.keys.all? { |key| key.is_a?(String) }
|
|
60
|
+
merged_context
|
|
61
|
+
else
|
|
62
|
+
merged_context.transform_keys(&:to_s)
|
|
50
63
|
end
|
|
51
|
-
payload
|
|
52
64
|
end
|
|
53
65
|
|
|
54
66
|
def empty_additional_context?(additional_context)
|
|
@@ -73,7 +85,8 @@ module JsonLogging
|
|
|
73
85
|
end
|
|
74
86
|
|
|
75
87
|
def merge_tags_only(payload, tags:, sanitize_tags:)
|
|
76
|
-
|
|
88
|
+
raw = payload["tags"]
|
|
89
|
+
existing_tags = raw ? Array(raw) : []
|
|
77
90
|
prepared_tags = sanitize_tags ? tags.map { |tag| Sanitizer.sanitize_string(tag.to_s) } : tags
|
|
78
91
|
payload["tags"] = (existing_tags + prepared_tags).uniq
|
|
79
92
|
payload
|
|
@@ -81,6 +94,7 @@ module JsonLogging
|
|
|
81
94
|
|
|
82
95
|
def context_payload_portion(additional_context)
|
|
83
96
|
return {} unless additional_context.is_a?(Hash)
|
|
97
|
+
return additional_context if additional_context.empty?
|
|
84
98
|
|
|
85
99
|
additional_context.reject do |key, _|
|
|
86
100
|
SYSTEM_CONTROLLED_KEYS.include?(key) || SYSTEM_CONTROLLED_KEYS.include?(key.to_s)
|
|
@@ -2,6 +2,7 @@ require_relative "structured_hash_sanitizer"
|
|
|
2
2
|
require_relative "sanitizer_hash_limit"
|
|
3
3
|
|
|
4
4
|
module JsonLogging
|
|
5
|
+
# rubocop:disable Metrics/ModuleLength -- sanitizer owns string/hash/tag policy in one place
|
|
5
6
|
module Sanitizer
|
|
6
7
|
# Control characters that should be escaped or removed from log messages
|
|
7
8
|
CONTROL_CHARS = /[\x00-\x1F\x7F]/
|
|
@@ -70,8 +71,21 @@ module JsonLogging
|
|
|
70
71
|
@parameter_filter_requires_full_tree_walk = false
|
|
71
72
|
end
|
|
72
73
|
|
|
73
|
-
def prepare_tags(tags)
|
|
74
|
-
tags.flatten.compact.map(&:to_s).reject(&:empty?)
|
|
74
|
+
def prepare_tags(tags, tag_source: nil)
|
|
75
|
+
flat = tags.flatten.compact.map(&:to_s).reject(&:empty?)
|
|
76
|
+
return [] if flat.empty?
|
|
77
|
+
|
|
78
|
+
if tag_source && flat.none? { |t| t.match?(CONTROL_CHARS) } && flat.all? { |t| t.length <= MAX_STRING_LENGTH }
|
|
79
|
+
cache = ExecutionState.prepared_tags_cache
|
|
80
|
+
key = [tag_source, flat]
|
|
81
|
+
return cache[key] if cache.key?(key)
|
|
82
|
+
|
|
83
|
+
prepared = flat.map { |tag| sanitize_string(tag) }
|
|
84
|
+
cache[key] = prepared.freeze
|
|
85
|
+
return prepared
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
flat.map { |tag| sanitize_string(tag) }
|
|
75
89
|
end
|
|
76
90
|
|
|
77
91
|
# Sanitize a string by removing/escaping control characters and truncating
|
|
@@ -97,6 +111,7 @@ module JsonLogging
|
|
|
97
111
|
def sanitize_hash(hash, depth: 0)
|
|
98
112
|
return hash unless hash.is_a?(Hash)
|
|
99
113
|
return {"error" => "max_depth_exceeded"} if depth > MAX_DEPTH
|
|
114
|
+
return hash if hash.empty?
|
|
100
115
|
|
|
101
116
|
limited_hash = limited_hash_for_sanitization(hash)
|
|
102
117
|
fast_path_result = fast_path_sanitized_hash(limited_hash, depth: depth)
|
|
@@ -108,6 +123,10 @@ module JsonLogging
|
|
|
108
123
|
end
|
|
109
124
|
|
|
110
125
|
def limited_hash_for_sanitization(hash)
|
|
126
|
+
if !rails_parameter_filter && hash.is_a?(Hash) && hash.size <= MAX_CONTEXT_SIZE
|
|
127
|
+
return hash
|
|
128
|
+
end
|
|
129
|
+
|
|
111
130
|
SanitizerHashLimit.apply(hash, filter_config: rails_parameter_filter && @parameter_filter_config)
|
|
112
131
|
end
|
|
113
132
|
|
|
@@ -146,6 +165,8 @@ module JsonLogging
|
|
|
146
165
|
when Hash
|
|
147
166
|
sanitize_hash(value, depth: depth)
|
|
148
167
|
when Array
|
|
168
|
+
return [] if value.empty?
|
|
169
|
+
|
|
149
170
|
sanitized = value.first(MAX_CONTEXT_SIZE).map { |v| sanitize_value(v, depth: depth + 1) }
|
|
150
171
|
sanitized << "[truncated]" if value.size > MAX_CONTEXT_SIZE
|
|
151
172
|
sanitized
|
|
@@ -232,7 +253,9 @@ module JsonLogging
|
|
|
232
253
|
|
|
233
254
|
def home_directory_prefix
|
|
234
255
|
prefix = ENV["HOME"]
|
|
235
|
-
|
|
256
|
+
# rubocop:disable Rails/Blank -- string path must not require ActiveSupport blank?
|
|
257
|
+
prefix = Dir.home if prefix.nil? || prefix.empty?
|
|
258
|
+
# rubocop:enable Rails/Blank
|
|
236
259
|
prefix = prefix.to_s.chomp("/").chomp("\\")
|
|
237
260
|
return if prefix.empty?
|
|
238
261
|
|
|
@@ -247,4 +270,5 @@ module JsonLogging
|
|
|
247
270
|
str.gsub(/#{Regexp.escape(prefix)}(?=\/|\\|\z)/, "~")
|
|
248
271
|
end
|
|
249
272
|
end
|
|
273
|
+
# rubocop:enable Metrics/ModuleLength
|
|
250
274
|
end
|
|
@@ -63,9 +63,13 @@ module JsonLogging
|
|
|
63
63
|
end
|
|
64
64
|
|
|
65
65
|
def append_json_line(tree, severity:, timestamp:, field_overrides:)
|
|
66
|
-
body = JSON.generate(tree)
|
|
67
66
|
extras = field_overrides.merge("severity" => severity, "timestamp" => timestamp)
|
|
68
|
-
|
|
67
|
+
# rubocop:disable Rails/Blank -- encoder path must not require ActiveSupport blank?
|
|
68
|
+
return "#{JSON.fast_generate(extras)}\n" if tree.nil? || tree.empty?
|
|
69
|
+
# rubocop:enable Rails/Blank
|
|
70
|
+
|
|
71
|
+
body = JSON.fast_generate(tree)
|
|
72
|
+
extra_json = extras.map { |key, value| "#{JSON.fast_generate(key.to_s)}:#{JSON.fast_generate(value)}" }.join(",")
|
|
69
73
|
"#{body.chop},#{extra_json}}\n"
|
|
70
74
|
end
|
|
71
75
|
|
data/lib/json_logging/version.rb
CHANGED
data/lib/json_logging.rb
CHANGED
|
@@ -3,6 +3,7 @@ require "json"
|
|
|
3
3
|
require "time"
|
|
4
4
|
|
|
5
5
|
require_relative "json_logging/version"
|
|
6
|
+
require_relative "json_logging/execution_state"
|
|
6
7
|
require_relative "json_logging/helpers"
|
|
7
8
|
require_relative "json_logging/sanitizer"
|
|
8
9
|
require_relative "json_logging/message_parser"
|
|
@@ -17,15 +18,29 @@ require_relative "json_logging/event_subscriber"
|
|
|
17
18
|
module JsonLogging
|
|
18
19
|
THREAD_CONTEXT_KEY = :__json_logging_context
|
|
19
20
|
SANITIZED_CONTEXT_CACHE_KEY = :__json_logging_sanitized_context
|
|
21
|
+
ISOLATION_LEVELS = [:thread, :fiber].freeze
|
|
22
|
+
|
|
20
23
|
@additional_context_warned = Set.new
|
|
21
24
|
@additional_context_transformer = nil
|
|
25
|
+
@isolation_level = :thread
|
|
22
26
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
27
|
+
# Experimental: +:fiber+ isolates JsonLogging context, tags, and caches per Fiber.
|
|
28
|
+
# Default +:thread+ keeps Rails IsolatedExecutionState / Thread.current behavior.
|
|
29
|
+
def self.isolation_level
|
|
30
|
+
@isolation_level
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def self.isolation_level=(level)
|
|
34
|
+
normalized = level.to_sym
|
|
35
|
+
unless ISOLATION_LEVELS.include?(normalized)
|
|
36
|
+
raise ArgumentError, "isolation_level must be :thread or :fiber, got: #{level.inspect}"
|
|
28
37
|
end
|
|
38
|
+
|
|
39
|
+
@isolation_level = normalized
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def self.context_storage
|
|
43
|
+
ExecutionState
|
|
29
44
|
end
|
|
30
45
|
private_class_method :context_storage
|
|
31
46
|
|
data/sig/json_logging.rbs
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: activesupport-json_logging
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.3.
|
|
4
|
+
version: 1.3.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Andrei Makarov
|
|
@@ -83,14 +83,14 @@ dependencies:
|
|
|
83
83
|
requirements:
|
|
84
84
|
- - ">="
|
|
85
85
|
- !ruby/object:Gem::Version
|
|
86
|
-
version: 2.2.
|
|
86
|
+
version: 2.2.9
|
|
87
87
|
type: :development
|
|
88
88
|
prerelease: false
|
|
89
89
|
version_requirements: !ruby/object:Gem::Requirement
|
|
90
90
|
requirements:
|
|
91
91
|
- - ">="
|
|
92
92
|
- !ruby/object:Gem::Version
|
|
93
|
-
version: 2.2.
|
|
93
|
+
version: 2.2.9
|
|
94
94
|
- !ruby/object:Gem::Dependency
|
|
95
95
|
name: rake
|
|
96
96
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -105,20 +105,6 @@ dependencies:
|
|
|
105
105
|
- - "~>"
|
|
106
106
|
- !ruby/object:Gem::Version
|
|
107
107
|
version: '13'
|
|
108
|
-
- !ruby/object:Gem::Dependency
|
|
109
|
-
name: rspec_junit_formatter
|
|
110
|
-
requirement: !ruby/object:Gem::Requirement
|
|
111
|
-
requirements:
|
|
112
|
-
- - "~>"
|
|
113
|
-
- !ruby/object:Gem::Version
|
|
114
|
-
version: '0.6'
|
|
115
|
-
type: :development
|
|
116
|
-
prerelease: false
|
|
117
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
118
|
-
requirements:
|
|
119
|
-
- - "~>"
|
|
120
|
-
- !ruby/object:Gem::Version
|
|
121
|
-
version: '0.6'
|
|
122
108
|
- !ruby/object:Gem::Dependency
|
|
123
109
|
name: standard
|
|
124
110
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -315,8 +301,8 @@ dependencies:
|
|
|
315
301
|
- - "~>"
|
|
316
302
|
- !ruby/object:Gem::Version
|
|
317
303
|
version: '0.2'
|
|
318
|
-
description:
|
|
319
|
-
|
|
304
|
+
description: JSON logger and formatter for Rails/ActiveSupport. No extra deps beyond
|
|
305
|
+
Rails/ActiveSupport. Compatible with Rails 7-8.
|
|
320
306
|
email:
|
|
321
307
|
- contact@kiskolabs.com
|
|
322
308
|
executables: []
|
|
@@ -330,6 +316,7 @@ files:
|
|
|
330
316
|
- lib/activesupport/json_logging/railtie.rb
|
|
331
317
|
- lib/json_logging.rb
|
|
332
318
|
- lib/json_logging/event_subscriber.rb
|
|
319
|
+
- lib/json_logging/execution_state.rb
|
|
333
320
|
- lib/json_logging/formatter.rb
|
|
334
321
|
- lib/json_logging/formatter_with_tags.rb
|
|
335
322
|
- lib/json_logging/helpers.rb
|