logsy 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +18 -0
- data/README.md +71 -10
- data/lib/logsy/configuration.rb +19 -2
- data/lib/logsy/controller_hooks.rb +12 -12
- data/lib/logsy/job_hooks.rb +70 -0
- data/lib/logsy/json_formatter.rb +62 -9
- data/lib/logsy/rack_middleware.rb +34 -0
- data/lib/logsy/railtie.rb +25 -0
- data/lib/logsy/sidekiq_middleware/server.rb +5 -2
- data/lib/logsy/version.rb +1 -1
- data/lib/logsy.rb +3 -0
- metadata +18 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: be9d1c01334a5549c684e6e0807eb2f0b1b00f7d481b0862a0d0d005de31b81f
|
|
4
|
+
data.tar.gz: 993aadd7127a75825114c9cadb6db4ef211fcd1e582320661b73ba483653aed9
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4d48f08e5b3c2f13c14ac9f8d3983c46bbb7361785343912e9840b8652cef8f6d24e3e98726dcf97fa00b1473aa776f9c0c4418add83b5888d60d398f0d081a5
|
|
7
|
+
data.tar.gz: e420eec6c13c97386d41bfbd32d91f54511c137808ce486ce00518a00df09226e6055526e8957baad210903ad71011dfc0f99fd013393cdac747b2f5558b1471
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.3.0] - 2026-06-21
|
|
4
|
+
|
|
5
|
+
- **`Logsy::JobHooks`**: new ActiveJob concern — the background-job counterpart of `ControllerHooks`. Emits one wide event (`event: "job"`) when each job finishes, carrying `job_class`, `queue`, `active_job_id`, `executions`, `duration_ms`, `status` (`"ok"`/`"error"`), `error`, and every Logsy tag set during the run. Emitted even when the job raises (then re-raised). Override `logsy_job_summary_extras` to add fields. Job arguments are deliberately omitted — ActiveJob already logs them at enqueue time, correlatable by `active_job_id`.
|
|
6
|
+
- **Auto-wiring**: the Railtie now auto-includes `Logsy::JobHooks` into `ActiveJob::Base` (via `ActiveSupport.on_load(:active_job)`), so jobs emit summaries with zero per-app setup. Opt out with `config.auto_include_job_hooks = false` and include it manually. (Non-Rails ActiveJob users include it themselves.)
|
|
7
|
+
- `SidekiqMiddleware::Server`: now also sets `Logsy[:job_class]` (from `job['wrapped']`, the ActiveJob class name, falling back to `job['class']` for plain Sidekiq workers), so every job log line — including ActiveJob's own "Performing ..." — is tagged with the class, not just the `job_id`/jid.
|
|
8
|
+
- `Configuration`: new `job_summary_event_name` (default `"job"`), mirroring `request_summary_event_name`; new `auto_include_job_hooks` (default `true`).
|
|
9
|
+
- **Default `ignored_caller_paths`** now also skips `app/middleware` / `app/middlewares` frames — Rack middlewares sit in every request's stack and would otherwise be picked as the "app frame" for lines emitted below them. (Previously each app had to add this itself.)
|
|
10
|
+
|
|
11
|
+
## [0.2.0] - 2026-06-07
|
|
12
|
+
|
|
13
|
+
- **`Logsy::RackMiddleware` + Railtie**: request_id is now captured in Rack middleware (auto-inserted after `ActionDispatch::RequestId`) instead of a controller `before_action`. Every log line of a request now carries `request_id` — including "Started GET ..." and "Processing by ..." which a controller callback fires too late to tag. Falls back to `X-Request-Id`, then a generated UUID, so the tag is always present.
|
|
14
|
+
- **Breaking-ish**: `Logsy::ControllerHooks` no longer registers the request-id `before_action` (the middleware supersedes it). It still emits the wide event.
|
|
15
|
+
- `JsonFormatter`: `file` and `line` merged into a single `file` field (`"app/models/order.rb:42"`).
|
|
16
|
+
- `JsonFormatter`: caller-location now skips gem/stdlib frames and attributes the line to the nearest *app* frame (under `Rails.root`, excluding `vendor/bundle`) — an ActiveRecord SQL line points at your controller/model code, not `active_record/log_subscriber.rb`. Omitted when no app frame exists.
|
|
17
|
+
- `JsonFormatter`: String messages are cleaned — ANSI color codes stripped (ActiveRecord SQL highlighting) and surrounding whitespace removed.
|
|
18
|
+
- `JsonFormatter`: caller-location lookup walks the stack in batches of 16 frames and gives up after `caller_location_max_depth` frames (default 64, configurable) instead of capturing the full 100+-frame Rails stack on every line. Typical app log line: ~7µs; framework-only line: bounded by the cap.
|
|
19
|
+
- `ControllerHooks`: the wide event no longer reports a wrong `status` when the action raised (the final status is decided by `rescue_from` after the hook unwinds); `status` is omitted on error, with `error` carrying the class/message.
|
|
20
|
+
|
|
3
21
|
## [0.1.0] - Unreleased
|
|
4
22
|
|
|
5
23
|
- Initial release.
|
data/README.md
CHANGED
|
@@ -25,7 +25,11 @@ config.logger = ActiveSupport::Logger.new($stdout)
|
|
|
25
25
|
.tap { |l| l.formatter = Logsy::JsonFormatter.new }
|
|
26
26
|
```
|
|
27
27
|
|
|
28
|
-
### 2.
|
|
28
|
+
### 2. Request id — automatic
|
|
29
|
+
|
|
30
|
+
In a Rails app there is nothing to do: a Railtie inserts `Logsy::RackMiddleware` right after `ActionDispatch::RequestId`, so `Logsy[:request_id]` is set before *any* request log line is emitted — including Rails' own "Started GET ..." and "Processing by ..." lines. It uses `X-Request-Id` when the caller (or your proxy) sends one, otherwise the UUID Rails generated, otherwise generates one itself — the tag is always present.
|
|
31
|
+
|
|
32
|
+
### 3. Include the controller hooks (wide event)
|
|
29
33
|
|
|
30
34
|
```ruby
|
|
31
35
|
# app/controllers/application_controller.rb
|
|
@@ -34,11 +38,9 @@ class ApplicationController < ActionController::API
|
|
|
34
38
|
end
|
|
35
39
|
```
|
|
36
40
|
|
|
37
|
-
This
|
|
38
|
-
- `Logsy[:request_id]` automatically populated from `request.request_id` (or `X-Request-Id` header) on every request
|
|
39
|
-
- One "wide event" log line at end of each request with `event: "request"`, method, path, controller, action, status, duration_ms, and any error class/message — plus every tag you set during the request
|
|
41
|
+
This emits one "wide event" log line at end of each request with `event: "request"`, method, path, controller, action, status, duration_ms, and any error class/message — plus every tag you set during the request. (On an unhandled error, `status` is omitted — the final status is decided by `rescue_from` after the hook unwinds — and `error` carries the class/message.)
|
|
40
42
|
|
|
41
|
-
###
|
|
43
|
+
### 4. Set tags wherever they become known
|
|
42
44
|
|
|
43
45
|
```ruby
|
|
44
46
|
# In a controller, model, service, anywhere:
|
|
@@ -51,7 +53,7 @@ end
|
|
|
51
53
|
|
|
52
54
|
That's it. No `Current` model to define, no attribute declarations — just key/value.
|
|
53
55
|
|
|
54
|
-
###
|
|
56
|
+
### 5. (Optional) Background job propagation
|
|
55
57
|
|
|
56
58
|
If you use Sidekiq:
|
|
57
59
|
|
|
@@ -80,10 +82,33 @@ The bundled middleware:
|
|
|
80
82
|
- **Client side**: when a job is enqueued, copies the configured tags from `Logsy[]` into the job payload
|
|
81
83
|
- **Server side**: when the job runs, sets those tags back in `Logsy[]` AND sets `Logsy[:job_id] = job['jid']` automatically
|
|
82
84
|
|
|
83
|
-
Anything the job code writes via `Logsy[:foo] = bar` while running shows up on every log line emitted during the job. Tags reset automatically after each job (no leak between jobs sharing a worker thread).
|
|
85
|
+
Anything the job code writes via `Logsy[:foo] = bar` while running shows up on every log line emitted during the job. Tags reset automatically after each job (no leak between jobs sharing a worker thread). The server middleware also sets `Logsy[:job_class]` (the wrapped ActiveJob class name, e.g. `"GenerateEodJob"`, falling back to the worker class for plain Sidekiq jobs).
|
|
84
86
|
|
|
85
87
|
For other job systems (Resque, GoodJob, custom), write your own middleware using the same pattern — Logsy's `[]=` / `[]` / `tags` / `reset` API is generic.
|
|
86
88
|
|
|
89
|
+
### 6. Job wide event — automatic
|
|
90
|
+
|
|
91
|
+
`Logsy::JobHooks` is the background-job counterpart of `ControllerHooks`. In a Rails app there's **nothing to do**: the Railtie auto-includes it into `ActiveJob::Base`, so every job emits one wide event with `event: "job"` when it finishes, carrying `job_class`, `queue`, `active_job_id`, `executions` (attempt count), `duration_ms`, `status` (`"ok"`/`"error"`), any `error` class/message — plus every tag set during the run (`job_id`/jid, propagated `request_id`, and anything you wrote via `Logsy[:key] = value`). It's emitted even when the job raises (then re-raised). Job arguments are deliberately omitted — ActiveJob already logs them at enqueue time, correlatable by `active_job_id`.
|
|
92
|
+
|
|
93
|
+
Override `logsy_job_summary_extras` in a job to add fields:
|
|
94
|
+
|
|
95
|
+
```ruby
|
|
96
|
+
class SomeJob < ApplicationJob
|
|
97
|
+
private
|
|
98
|
+
|
|
99
|
+
def logsy_job_summary_extras
|
|
100
|
+
{ tenant: account_id }
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
To opt out of the auto-include and wire it yourself:
|
|
106
|
+
|
|
107
|
+
```ruby
|
|
108
|
+
Logsy.configure { |c| c.auto_include_job_hooks = false }
|
|
109
|
+
# then `include Logsy::JobHooks` in the jobs you want
|
|
110
|
+
```
|
|
111
|
+
|
|
87
112
|
## What it looks like in your logs
|
|
88
113
|
|
|
89
114
|
A regular log line:
|
|
@@ -92,9 +117,8 @@ A regular log line:
|
|
|
92
117
|
{
|
|
93
118
|
"ts":"2026-05-02T10:00:00.123Z",
|
|
94
119
|
"level":"INFO",
|
|
95
|
-
"msg":"Calling
|
|
96
|
-
"file":"app/
|
|
97
|
-
"line":437,
|
|
120
|
+
"msg":"Calling payment gateway",
|
|
121
|
+
"file":"app/services/billing_service.rb:437",
|
|
98
122
|
"request_id":"abc-123",
|
|
99
123
|
"user_id":"u-1",
|
|
100
124
|
"message_id":"m-42",
|
|
@@ -122,8 +146,32 @@ The wide event at end of request:
|
|
|
122
146
|
}
|
|
123
147
|
```
|
|
124
148
|
|
|
149
|
+
The wide event at end of a job (with `Logsy::JobHooks`):
|
|
150
|
+
|
|
151
|
+
```json
|
|
152
|
+
{
|
|
153
|
+
"ts":"2026-05-02T10:00:00.250Z",
|
|
154
|
+
"level":"INFO",
|
|
155
|
+
"event":"job",
|
|
156
|
+
"job_class":"SpgReverseTransactionJob",
|
|
157
|
+
"queue":"default",
|
|
158
|
+
"active_job_id":"09fe3546-211b-45cf-9434-fb2689c580c5",
|
|
159
|
+
"executions":1,
|
|
160
|
+
"duration_ms":842.10,
|
|
161
|
+
"status":"ok",
|
|
162
|
+
"job_id":"402797cb0cfc9324cc10cc7f",
|
|
163
|
+
"transaction_id":"019c0440-a797-7479-8d3a-4260ba896681",
|
|
164
|
+
"rrn":"123456789012"
|
|
165
|
+
}
|
|
166
|
+
```
|
|
167
|
+
|
|
125
168
|
Search your log store by `message_id` to find every request from that message. Pivot from a request's `request_id` to see every breadcrumb log line emitted while it ran.
|
|
126
169
|
|
|
170
|
+
Notes on the fields:
|
|
171
|
+
|
|
172
|
+
- **`file`** points at the nearest *app* frame — gem, stdlib, and logging-machinery frames are skipped, so an ActiveRecord SQL line is attributed to the controller/model code that ran the query, not to `active_record/log_subscriber.rb`. Framework-only lines with no app frame (e.g. "Started GET ...") omit `file` rather than show a useless gem path.
|
|
173
|
+
- **`msg`** is cleaned: ANSI color codes (ActiveRecord's SQL highlighting) are stripped and surrounding whitespace removed, so the text is plain and searchable.
|
|
174
|
+
|
|
127
175
|
## API reference
|
|
128
176
|
|
|
129
177
|
```ruby
|
|
@@ -167,12 +215,25 @@ Logsy.configure do |c|
|
|
|
167
215
|
# Disable if you measure overhead. Default: true
|
|
168
216
|
c.include_caller_location = true
|
|
169
217
|
|
|
218
|
+
# How many stack frames to inspect before giving up on attributing a log
|
|
219
|
+
# line to app code. Bounds the cost of framework-only lines that have no
|
|
220
|
+
# app frame anywhere in the stack. Default: 64
|
|
221
|
+
c.caller_location_max_depth = 64
|
|
222
|
+
|
|
170
223
|
# Regex patterns for caller frames to skip. Defaults already cover Logger,
|
|
171
224
|
# ActiveSupport, lograge, sprockets, quiet_assets, and Logsy itself.
|
|
172
225
|
c.ignored_caller_paths += [%r{/my_internal_logger/}]
|
|
173
226
|
|
|
174
227
|
# The "event" name on the request summary. Default: "request"
|
|
175
228
|
c.request_summary_event_name = 'http_request'
|
|
229
|
+
|
|
230
|
+
# The "event" name on the job summary (Logsy::JobHooks). Default: "job"
|
|
231
|
+
c.job_summary_event_name = 'background_job'
|
|
232
|
+
|
|
233
|
+
# Auto-include Logsy::JobHooks into ActiveJob::Base (the Railtie does this
|
|
234
|
+
# so jobs emit summaries with no setup). Set false to wire it yourself.
|
|
235
|
+
# Default: true
|
|
236
|
+
c.auto_include_job_hooks = true
|
|
176
237
|
end
|
|
177
238
|
```
|
|
178
239
|
|
data/lib/logsy/configuration.rb
CHANGED
|
@@ -8,7 +8,12 @@ module Logsy
|
|
|
8
8
|
%r{/lograge/},
|
|
9
9
|
%r{/sprockets/},
|
|
10
10
|
%r{/quiet_assets/},
|
|
11
|
-
%r{/lib/logsy/}
|
|
11
|
+
%r{/lib/logsy/},
|
|
12
|
+
# Rack middlewares sit in every request's call stack, so without this
|
|
13
|
+
# they'd be picked as the "app frame" for lines emitted below them
|
|
14
|
+
# (e.g. "Started GET ..." would be attributed to a health_check
|
|
15
|
+
# middleware). Matches app/middleware and app/middlewares.
|
|
16
|
+
%r{/app/middlewares?/}
|
|
12
17
|
].freeze
|
|
13
18
|
|
|
14
19
|
# Tag keys that should be carried across job boundaries (e.g. when a
|
|
@@ -16,13 +21,25 @@ module Logsy
|
|
|
16
21
|
# these). The middleware that does the actual carrying is job-runner
|
|
17
22
|
# specific (Logsy ships one for Sidekiq).
|
|
18
23
|
attr_accessor :job_propagated_keys, :ignored_caller_paths,
|
|
19
|
-
:include_caller_location, :request_summary_event_name
|
|
24
|
+
:include_caller_location, :request_summary_event_name,
|
|
25
|
+
:job_summary_event_name, :caller_location_max_depth,
|
|
26
|
+
:auto_include_job_hooks
|
|
20
27
|
|
|
21
28
|
def initialize
|
|
22
29
|
@job_propagated_keys = [:request_id]
|
|
23
30
|
@ignored_caller_paths = DEFAULT_IGNORED_CALLER_PATHS.dup
|
|
24
31
|
@include_caller_location = true
|
|
25
32
|
@request_summary_event_name = 'request'
|
|
33
|
+
@job_summary_event_name = 'job'
|
|
34
|
+
# When true (the default), the Railtie auto-includes Logsy::JobHooks
|
|
35
|
+
# into ActiveJob::Base, so every job emits a summary wide event with
|
|
36
|
+
# no per-app setup. Set to false to opt out and include it manually.
|
|
37
|
+
@auto_include_job_hooks = true
|
|
38
|
+
# How many stack frames to inspect before giving up on attributing
|
|
39
|
+
# the log line to app code. App frames sit within a few dozen frames
|
|
40
|
+
# of the logger; framework-only lines (no app frame at all) would
|
|
41
|
+
# otherwise pay a full walk of a 100+ deep Rails stack on every line.
|
|
42
|
+
@caller_location_max_depth = 64
|
|
26
43
|
end
|
|
27
44
|
end
|
|
28
45
|
end
|
|
@@ -3,13 +3,14 @@
|
|
|
3
3
|
require 'active_support/concern'
|
|
4
4
|
|
|
5
5
|
module Logsy
|
|
6
|
-
# Mix into your ApplicationController (or specific controllers) to
|
|
6
|
+
# Mix into your ApplicationController (or specific controllers) to emit
|
|
7
|
+
# a single "wide event" log line at end of request with the method,
|
|
8
|
+
# path, status, duration_ms, controller, action, error, and every Logsy
|
|
9
|
+
# tag that was set during the request.
|
|
7
10
|
#
|
|
8
|
-
#
|
|
9
|
-
#
|
|
10
|
-
#
|
|
11
|
-
# method, path, status, duration_ms, controller, action, error, and
|
|
12
|
-
# every Current.* attribute that was set during the request.
|
|
11
|
+
# (Request-id capture lives in Logsy::RackMiddleware, inserted
|
|
12
|
+
# automatically by the Railtie — not here. A controller callback fires
|
|
13
|
+
# too late to tag the "Started GET" / "Processing by" lines.)
|
|
13
14
|
#
|
|
14
15
|
# Usage:
|
|
15
16
|
#
|
|
@@ -27,16 +28,11 @@ module Logsy
|
|
|
27
28
|
extend ActiveSupport::Concern
|
|
28
29
|
|
|
29
30
|
included do
|
|
30
|
-
before_action :_logsy_capture_request_id
|
|
31
31
|
around_action :_logsy_emit_request_summary
|
|
32
32
|
end
|
|
33
33
|
|
|
34
34
|
private
|
|
35
35
|
|
|
36
|
-
def _logsy_capture_request_id
|
|
37
|
-
Logsy[:request_id] = request.request_id || request.headers['X-Request-Id']
|
|
38
|
-
end
|
|
39
|
-
|
|
40
36
|
def _logsy_emit_request_summary
|
|
41
37
|
started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
42
38
|
error = nil
|
|
@@ -52,7 +48,11 @@ module Logsy
|
|
|
52
48
|
path: request.path,
|
|
53
49
|
controller: controller_name,
|
|
54
50
|
action: action_name,
|
|
55
|
-
status
|
|
51
|
+
# On an unhandled error the final status is decided by rescue_from /
|
|
52
|
+
# the exception app *after* this around_action unwinds, so
|
|
53
|
+
# response.status still holds the pre-error default — omit it
|
|
54
|
+
# rather than report it wrong.
|
|
55
|
+
status: error ? nil : response&.status,
|
|
56
56
|
duration_ms:,
|
|
57
57
|
error: error && "#{error.class}: #{error.message}"
|
|
58
58
|
}.merge(logsy_request_summary_extras).compact
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'active_support/concern'
|
|
4
|
+
|
|
5
|
+
module Logsy
|
|
6
|
+
# Include into your ApplicationJob (or specific jobs) to emit a single
|
|
7
|
+
# "wide event" log line when the job finishes — carrying the job class,
|
|
8
|
+
# queue, ActiveJob id, attempt count, duration_ms, status/error, and
|
|
9
|
+
# every Logsy tag set during the run (job_id/jid, propagated request_id,
|
|
10
|
+
# and anything the job wrote via `Logsy[:key] = value`, e.g. an RRN).
|
|
11
|
+
#
|
|
12
|
+
# This is the background-job counterpart of Logsy::ControllerHooks.
|
|
13
|
+
# Per-job context capture (job_id, job_class, propagated tags) lives in
|
|
14
|
+
# Logsy::SidekiqMiddleware::Server — not here — so even ActiveJob's own
|
|
15
|
+
# "Performing ..." line is already tagged before this hook runs.
|
|
16
|
+
#
|
|
17
|
+
# Usage:
|
|
18
|
+
#
|
|
19
|
+
# class ApplicationJob < ActiveJob::Base
|
|
20
|
+
# include Logsy::JobHooks
|
|
21
|
+
# end
|
|
22
|
+
#
|
|
23
|
+
# Override `logsy_job_summary_extras` to add custom fields to the event:
|
|
24
|
+
#
|
|
25
|
+
# def logsy_job_summary_extras
|
|
26
|
+
# { tenant: account_id }
|
|
27
|
+
# end
|
|
28
|
+
#
|
|
29
|
+
# Job arguments are intentionally omitted — ActiveJob already logs them
|
|
30
|
+
# once at enqueue time ("Enqueued ... with arguments: ..."), correlatable
|
|
31
|
+
# by active_job_id, so repeating them here would only duplicate (and risk
|
|
32
|
+
# leaking) payload data.
|
|
33
|
+
module JobHooks
|
|
34
|
+
extend ActiveSupport::Concern
|
|
35
|
+
|
|
36
|
+
included do
|
|
37
|
+
around_perform :_logsy_emit_job_summary
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
def _logsy_emit_job_summary
|
|
43
|
+
started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
44
|
+
error = nil
|
|
45
|
+
yield
|
|
46
|
+
rescue StandardError => e
|
|
47
|
+
error = e
|
|
48
|
+
raise
|
|
49
|
+
ensure
|
|
50
|
+
duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at) * 1000).round(2)
|
|
51
|
+
payload = {
|
|
52
|
+
event: Logsy.configuration.job_summary_event_name,
|
|
53
|
+
job_class: self.class.name,
|
|
54
|
+
queue: queue_name,
|
|
55
|
+
active_job_id: job_id,
|
|
56
|
+
executions:,
|
|
57
|
+
duration_ms:,
|
|
58
|
+
status: error ? 'error' : 'ok',
|
|
59
|
+
error: error && "#{error.class}: #{error.message}"
|
|
60
|
+
}.merge(logsy_job_summary_extras).compact
|
|
61
|
+
|
|
62
|
+
Rails.logger.info(payload)
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# Override in subclasses to add fields to the job summary wide event.
|
|
66
|
+
def logsy_job_summary_extras
|
|
67
|
+
{}
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
data/lib/logsy/json_formatter.rb
CHANGED
|
@@ -11,8 +11,18 @@ module Logsy
|
|
|
11
11
|
# - ts: UTC ISO-8601 timestamp with millis
|
|
12
12
|
# - level: severity ('INFO', 'ERROR', ...)
|
|
13
13
|
# - msg: the log message (when message is a String or Exception)
|
|
14
|
-
# - file
|
|
15
|
-
#
|
|
14
|
+
# - file: "path:line" of the nearest app frame (when
|
|
15
|
+
# include_caller_location). Gem, stdlib, and ignored frames
|
|
16
|
+
# are skipped, so an ActiveRecord SQL line points at the
|
|
17
|
+
# controller/model code that ran the query — not at
|
|
18
|
+
# active_record/log_subscriber.rb. Omitted entirely for
|
|
19
|
+
# framework-only lines (e.g. "Started GET ..."), which have
|
|
20
|
+
# no app frame.
|
|
21
|
+
# - any non-nil tags from Logsy[]
|
|
22
|
+
#
|
|
23
|
+
# String messages are cleaned before emission: ANSI color codes (e.g.
|
|
24
|
+
# ActiveRecord's SQL highlighting) are stripped and surrounding
|
|
25
|
+
# whitespace removed, so log stores get plain searchable text.
|
|
16
26
|
#
|
|
17
27
|
# When the message is a Hash, its keys are merged directly into the
|
|
18
28
|
# top-level payload (useful for "wide event" emissions like a request
|
|
@@ -20,9 +30,12 @@ module Logsy
|
|
|
20
30
|
#
|
|
21
31
|
# Example output:
|
|
22
32
|
# {"ts":"2026-05-02T10:00:00.123Z","level":"INFO","msg":"hello",
|
|
23
|
-
# "file":"app/controllers/orders_controller.rb
|
|
33
|
+
# "file":"app/controllers/orders_controller.rb:34",
|
|
24
34
|
# "request_id":"abc-123","user_id":"u-1"}
|
|
25
35
|
class JsonFormatter < ::Logger::Formatter
|
|
36
|
+
ANSI_ESCAPE = /\e\[[0-9;]*m/
|
|
37
|
+
FRAME_BATCH_SIZE = 16
|
|
38
|
+
|
|
26
39
|
def call(severity, time, _progname, message)
|
|
27
40
|
payload = {
|
|
28
41
|
ts: time.utc.iso8601(3),
|
|
@@ -41,19 +54,59 @@ module Logsy
|
|
|
41
54
|
private
|
|
42
55
|
|
|
43
56
|
def add_caller_location!(payload)
|
|
44
|
-
location =
|
|
57
|
+
location = find_user_frame
|
|
45
58
|
return unless location
|
|
46
59
|
|
|
47
|
-
payload[:file] = relative_path(location.path)
|
|
48
|
-
|
|
60
|
+
payload[:file] = "#{relative_path(location.path)}:#{location.lineno}"
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
# Walk the stack in small batches instead of capturing it all at once:
|
|
64
|
+
# the frame we want is usually a handful of frames above the logger
|
|
65
|
+
# machinery, while Rails request stacks run 100+ frames deep —
|
|
66
|
+
# capturing everything costs ~3x more per log line. The walk is capped
|
|
67
|
+
# at caller_location_max_depth so framework-only lines (no app frame
|
|
68
|
+
# anywhere) don't pay for a full-stack scan either.
|
|
69
|
+
def find_user_frame
|
|
70
|
+
start = 2 # skip find_user_frame and add_caller_location!
|
|
71
|
+
limit = start + Logsy.configuration.caller_location_max_depth
|
|
72
|
+
while start < limit
|
|
73
|
+
batch = caller_locations(start, [FRAME_BATCH_SIZE, limit - start].min)
|
|
74
|
+
return nil if batch.nil? || batch.empty?
|
|
75
|
+
|
|
76
|
+
batch.each { |loc| return loc if user_frame?(loc.path) }
|
|
77
|
+
return nil if batch.size < FRAME_BATCH_SIZE
|
|
78
|
+
|
|
79
|
+
start += FRAME_BATCH_SIZE
|
|
80
|
+
end
|
|
81
|
+
nil
|
|
49
82
|
end
|
|
50
83
|
|
|
84
|
+
# A frame worth attributing the log line to: app code, not gems,
|
|
85
|
+
# stdlib, or logging machinery. With an app root (Rails), only frames
|
|
86
|
+
# under it count (vendored gems under root excluded); without one,
|
|
87
|
+
# any non-gem frame counts.
|
|
51
88
|
def user_frame?(path)
|
|
52
|
-
Logsy.configuration.ignored_caller_paths.
|
|
89
|
+
return false if Logsy.configuration.ignored_caller_paths.any? { |pattern| path.match?(pattern) }
|
|
90
|
+
|
|
91
|
+
if (root = app_root)
|
|
92
|
+
path.start_with?(root) && !path.include?('/vendor/bundle/')
|
|
93
|
+
else
|
|
94
|
+
!gem_frame?(path)
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
def gem_frame?(path)
|
|
99
|
+
path.include?('/gems/') ||
|
|
100
|
+
path.include?('/rubygems/') ||
|
|
101
|
+
path.start_with?('<internal:', RbConfig::CONFIG['rubylibdir'])
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def app_root
|
|
105
|
+
defined?(Rails) && Rails.respond_to?(:root) && Rails.root ? "#{Rails.root}/" : nil
|
|
53
106
|
end
|
|
54
107
|
|
|
55
108
|
def relative_path(path)
|
|
56
|
-
root =
|
|
109
|
+
root = app_root
|
|
57
110
|
return path unless root && path.start_with?(root)
|
|
58
111
|
|
|
59
112
|
path[root.length..]
|
|
@@ -70,7 +123,7 @@ module Logsy
|
|
|
70
123
|
def merge_message!(payload, message)
|
|
71
124
|
case message
|
|
72
125
|
when ::String
|
|
73
|
-
payload[:msg] = message
|
|
126
|
+
payload[:msg] = message.gsub(ANSI_ESCAPE, '').strip
|
|
74
127
|
when ::Hash
|
|
75
128
|
message.each { |k, v| payload[k.to_sym] = v }
|
|
76
129
|
when ::Exception
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'securerandom'
|
|
4
|
+
|
|
5
|
+
module Logsy
|
|
6
|
+
# Rack middleware that captures the request id into Logsy[:request_id]
|
|
7
|
+
# at the earliest useful point in the stack — before Rails::Rack::Logger
|
|
8
|
+
# emits "Started GET ..." and before ActionController's start_processing
|
|
9
|
+
# event fires — so *every* log line of the request carries it, not just
|
|
10
|
+
# the ones emitted after controller callbacks run.
|
|
11
|
+
#
|
|
12
|
+
# The Railtie inserts this automatically right after
|
|
13
|
+
# ActionDispatch::RequestId (which populates `action_dispatch.request_id`
|
|
14
|
+
# from X-Request-Id or generates a UUID). Outside Rails, falls back to
|
|
15
|
+
# the X-Request-Id header, then to a generated UUID, so the tag is
|
|
16
|
+
# always present.
|
|
17
|
+
#
|
|
18
|
+
# Tag cleanup between requests is handled by the Rails executor
|
|
19
|
+
# (ActionDispatch::Executor wraps this middleware and resets
|
|
20
|
+
# CurrentAttributes); plain Rack apps should call Logsy.reset themselves.
|
|
21
|
+
class RackMiddleware
|
|
22
|
+
def initialize(app)
|
|
23
|
+
@app = app
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def call(env)
|
|
27
|
+
Logsy[:request_id] = env['action_dispatch.request_id'] ||
|
|
28
|
+
env['HTTP_X_REQUEST_ID'] ||
|
|
29
|
+
SecureRandom.uuid
|
|
30
|
+
|
|
31
|
+
@app.call(env)
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Logsy
|
|
4
|
+
# Wires Logsy into Rails with zero per-app setup:
|
|
5
|
+
#
|
|
6
|
+
# - Inserts the request-id capturing middleware right after
|
|
7
|
+
# ActionDispatch::RequestId, before Rails::Rack::Logger — early enough
|
|
8
|
+
# that the "Started GET ..." line is already tagged.
|
|
9
|
+
# - Includes Logsy::JobHooks into ActiveJob::Base so every job emits a
|
|
10
|
+
# summary wide event (disable via config.auto_include_job_hooks = false).
|
|
11
|
+
class Railtie < ::Rails::Railtie
|
|
12
|
+
initializer 'logsy.insert_rack_middleware' do |app|
|
|
13
|
+
app.middleware.insert_after(ActionDispatch::RequestId, Logsy::RackMiddleware)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
initializer 'logsy.include_job_hooks' do
|
|
17
|
+
# Fires when ActiveJob::Base loads (eager load in production, first
|
|
18
|
+
# reference in development) — after config/initializers have run, so
|
|
19
|
+
# an app's auto_include_job_hooks = false is respected.
|
|
20
|
+
ActiveSupport.on_load(:active_job) do
|
|
21
|
+
include(Logsy::JobHooks) if Logsy.configuration.auto_include_job_hooks
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
@@ -4,14 +4,17 @@ module Logsy
|
|
|
4
4
|
module SidekiqMiddleware
|
|
5
5
|
# Server-side middleware: reads propagated tags back from the job
|
|
6
6
|
# payload, sets them on the per-job Logsy store, sets job_id from
|
|
7
|
-
# Sidekiq's `jid
|
|
8
|
-
#
|
|
7
|
+
# Sidekiq's `jid` and job_class from the worker (the wrapped ActiveJob
|
|
8
|
+
# class name when run through ActiveJob), and resets the store after the
|
|
9
|
+
# job runs (even on error) so tags don't leak between jobs sharing a
|
|
10
|
+
# worker thread.
|
|
9
11
|
#
|
|
10
12
|
# Anything the job code writes via `Logsy[:foo] = bar` while running
|
|
11
13
|
# will appear on every log line emitted during the job.
|
|
12
14
|
class Server
|
|
13
15
|
def call(_worker_instance, job, _queue)
|
|
14
16
|
Logsy[:job_id] = job['jid']
|
|
17
|
+
Logsy[:job_class] = job['wrapped'] || job['class']
|
|
15
18
|
Logsy.configuration.job_propagated_keys.each do |key|
|
|
16
19
|
value = job[key.to_s]
|
|
17
20
|
Logsy[key] = value if value
|
data/lib/logsy/version.rb
CHANGED
data/lib/logsy.rb
CHANGED
|
@@ -5,6 +5,9 @@ require 'logsy/store'
|
|
|
5
5
|
require 'logsy/configuration'
|
|
6
6
|
require 'logsy/json_formatter'
|
|
7
7
|
require 'logsy/controller_hooks'
|
|
8
|
+
require 'logsy/job_hooks'
|
|
9
|
+
require 'logsy/rack_middleware'
|
|
10
|
+
require 'logsy/railtie' if defined?(Rails::Railtie)
|
|
8
11
|
|
|
9
12
|
module Logsy
|
|
10
13
|
class Error < StandardError; end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: logsy
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ruby_is_love
|
|
@@ -37,6 +37,20 @@ dependencies:
|
|
|
37
37
|
- - ">="
|
|
38
38
|
- !ruby/object:Gem::Version
|
|
39
39
|
version: '7.0'
|
|
40
|
+
- !ruby/object:Gem::Dependency
|
|
41
|
+
name: activejob
|
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
|
43
|
+
requirements:
|
|
44
|
+
- - ">="
|
|
45
|
+
- !ruby/object:Gem::Version
|
|
46
|
+
version: '7.0'
|
|
47
|
+
type: :development
|
|
48
|
+
prerelease: false
|
|
49
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
50
|
+
requirements:
|
|
51
|
+
- - ">="
|
|
52
|
+
- !ruby/object:Gem::Version
|
|
53
|
+
version: '7.0'
|
|
40
54
|
- !ruby/object:Gem::Dependency
|
|
41
55
|
name: rake
|
|
42
56
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -94,7 +108,10 @@ files:
|
|
|
94
108
|
- lib/logsy.rb
|
|
95
109
|
- lib/logsy/configuration.rb
|
|
96
110
|
- lib/logsy/controller_hooks.rb
|
|
111
|
+
- lib/logsy/job_hooks.rb
|
|
97
112
|
- lib/logsy/json_formatter.rb
|
|
113
|
+
- lib/logsy/rack_middleware.rb
|
|
114
|
+
- lib/logsy/railtie.rb
|
|
98
115
|
- lib/logsy/sidekiq_middleware.rb
|
|
99
116
|
- lib/logsy/sidekiq_middleware/client.rb
|
|
100
117
|
- lib/logsy/sidekiq_middleware/server.rb
|