visionary-sdk 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +56 -4
- data/lib/visionary/agent.rb +7 -2
- data/lib/visionary/config.rb +8 -1
- data/lib/visionary/context.rb +33 -3
- data/lib/visionary/logs/logger.rb +274 -0
- data/lib/visionary/logs/pipeline.rb +193 -0
- data/lib/visionary/logs.rb +307 -0
- data/lib/visionary/middleware.rb +5 -1
- data/lib/visionary/schema/mysql/logs.sql +31 -0
- data/lib/visionary/schema/postgres/logs.sql +31 -0
- data/lib/visionary/schema/sqlite/logs.sql +31 -0
- data/lib/visionary/schema.rb +10 -2
- data/lib/visionary/version.rb +1 -1
- data/lib/visionary/writer.rb +10 -2
- data/lib/visionary.rb +35 -2
- metadata +8 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 65f72d9800110a073d55857e5143ebe26c42d2659e11113ae0f18a1751e86f96
|
|
4
|
+
data.tar.gz: 16afcc488a1077bbad0e2b57de401e45bcf636cf394a6098b04ce89cf275caa0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0adf2d6a605e74af5cf91843d7d289369b1969a02eafde24fa97aa2102bd6fc59d43c1c938167fe49682ba624c7ff92b53414272d67929fdf0a9e94fa22de8c1
|
|
7
|
+
data.tar.gz: 69057563d811c46baefd734d7b480013c21d563c33f616695dbc1ec6873a3f19c66a4c40a13f6dc8808ea1c9d5ce02945bc84863e62e1331a8211a8b22981c75
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
- Application logs in `app_logs`: `Visionary.logger(inner = nil)` (a Logger
|
|
6
|
+
that forwards every call to `inner` unchanged and also captures) and
|
|
7
|
+
`Visionary.capture_rails_logger` (broadcasts `Rails.logger`). Lines link to
|
|
8
|
+
their request (`request_id`, `user_id`, `trace_id` from `traceparent`) and
|
|
9
|
+
record where they were logged. What is kept follows Settings → Logs: level,
|
|
10
|
+
overrides by route or user, drop rules and a per-minute cap; lower lines are
|
|
11
|
+
held in the request and kept when it fails or is slow. Secrets are scrubbed
|
|
12
|
+
from messages and attrs. Written by a writer thread of its own.
|
|
13
|
+
`VISIONARY_LOGS`, `VISIONARY_LOG_LEVEL`.
|
|
14
|
+
|
|
3
15
|
## 0.1.0
|
|
4
16
|
|
|
5
17
|
First release of the Ruby SDK (`visionary-sdk`, `require "visionary"`).
|
data/README.md
CHANGED
|
@@ -3,8 +3,8 @@
|
|
|
3
3
|
Add one gem to a Rails or Rack app and every request shows up in the
|
|
4
4
|
[Visionary](../../Readme.md) dashboard: route, status, time, user, redacted
|
|
5
5
|
bodies, the database and HTTP calls it made, and any exception. You also get
|
|
6
|
-
|
|
7
|
-
dashboard.
|
|
6
|
+
the app's own log lines linked to their request, host and process metrics,
|
|
7
|
+
and CPU profiles when you turn them on in the dashboard.
|
|
8
8
|
|
|
9
9
|
- [Quick start](#quick-start)
|
|
10
10
|
- [Integration recipes](#integration-recipes)
|
|
@@ -288,6 +288,49 @@ your own code (not gems, the standard library or this SDK), as `method@path`
|
|
|
288
288
|
relative to the working directory, so the group stays the same across deploys
|
|
289
289
|
and line changes. At most 20 rows per fingerprint per minute are written.
|
|
290
290
|
|
|
291
|
+
### Logs
|
|
292
|
+
|
|
293
|
+
**When:** you want the app's log lines in the dashboard, next to the request
|
|
294
|
+
that logged them. Route your logs through the SDK; what your app prints does
|
|
295
|
+
not change.
|
|
296
|
+
|
|
297
|
+
```ruby
|
|
298
|
+
# Rails: config/initializers/visionary.rb
|
|
299
|
+
Visionary.capture_rails_logger
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
```ruby
|
|
303
|
+
# Any app: wrap the logger you already use
|
|
304
|
+
logger = Visionary.logger(Logger.new($stdout))
|
|
305
|
+
logger.info("charged card") # printed as before, and captured
|
|
306
|
+
logger.debug { "cart: #{cart.inspect}" } # kept when the request fails
|
|
307
|
+
logger.warn(message: "slow upstream", ms: 912) # a Hash: message plus attrs
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
`capture_rails_logger` broadcasts `Rails.logger` to a capturing logger (Rails
|
|
311
|
+
7.1+ `broadcast_to`, older `ActiveSupport::Logger.broadcast`); tags from
|
|
312
|
+
`Rails.logger.tagged` become the `tags` attribute. `Visionary.logger(inner)`
|
|
313
|
+
forwards every call to `inner` unchanged and evaluates a block once. With no
|
|
314
|
+
`inner` it only captures. Lines are captured once the agent has started (on
|
|
315
|
+
the first request, or at boot with `Visionary.start`).
|
|
316
|
+
|
|
317
|
+
What is kept follows **Settings → Logs** in the dashboard, read every 30 s:
|
|
318
|
+
the lowest level (`VISIONARY_LOG_LEVEL` until then, default `info`), overrides
|
|
319
|
+
for a route or a user, drop rules, and a cap of lines per minute. Lines below
|
|
320
|
+
the level logged during a request are held until it ends, and kept when it
|
|
321
|
+
failed (5xx, an exception, `capture_exception`) or was slow, so a failure
|
|
322
|
+
comes with its debug lines. Because of that the capturing logger reports
|
|
323
|
+
`debug?` as true while debug lines may be kept; with Rails 7.1+ that makes
|
|
324
|
+
`Rails.logger.level` read as debug, though Rails still prints at its own level.
|
|
325
|
+
|
|
326
|
+
**Records:** an `app_logs` row per kept line: level, message (secrets such as
|
|
327
|
+
passwords, bearer tokens, JWTs and AWS keys replaced), attrs (nested keys
|
|
328
|
+
dotted, redacted fields `[REDACTED]`), the request's `request_id`, `user_id`
|
|
329
|
+
and W3C `trace_id` (from `traceparent`), and where it was logged
|
|
330
|
+
(`code_file`, `code_line`, `code_func`). Exceptions are logged as
|
|
331
|
+
`Class: message` and the backtrace. The SDK's own `visionary: ` lines are
|
|
332
|
+
never captured. `VISIONARY_LOGS=false` turns capture off.
|
|
333
|
+
|
|
291
334
|
### Error codes from JSON responses
|
|
292
335
|
|
|
293
336
|
**When:** your API answers errors with a JSON body like
|
|
@@ -412,7 +455,7 @@ Notes:
|
|
|
412
455
|
errors.
|
|
413
456
|
- For D1, `D1_API_BASE` replaces `https://api.cloudflare.com/client/v4`, for
|
|
414
457
|
a proxy.
|
|
415
|
-
- The tables (`api_logs`, `system_metrics`, `spans`, `errors`, `releases`)
|
|
458
|
+
- The tables (`api_logs`, `system_metrics`, `spans`, `errors`, `releases`, `app_logs`)
|
|
416
459
|
are created on first use, and tables from older versions gain the newer
|
|
417
460
|
columns. `profiles` and `visionary_settings` are created by the Visionary
|
|
418
461
|
server.
|
|
@@ -442,7 +485,7 @@ options in code, plus the hooks below. Set them before the first request
|
|
|
442
485
|
| `VISIONARY_LOG_REDACT_REQUEST_FIELDS` | `redact_request_fields` | empty | Comma list of keys to redact in request bodies only |
|
|
443
486
|
| `VISIONARY_LOG_ERROR_CODE_FIELD` | `error_code_field` | `error.code` | Dotted path to the error code in JSON error bodies |
|
|
444
487
|
| `VISIONARY_LOG_ERROR_KEY_FIELD` | `error_key_field` | `error.key` | Dotted path to the error key |
|
|
445
|
-
| `VISIONARY_LOG_RETENTION_DAYS` | `log_retention_days` | `0` (off) | Above 0, the SDK also deletes older `api_logs`, `spans`, `errors` and `
|
|
488
|
+
| `VISIONARY_LOG_RETENTION_DAYS` | `log_retention_days` | `0` (off) | Above 0, the SDK also deletes older `api_logs`, `spans`, `errors`, `profiles` and `app_logs` rows |
|
|
446
489
|
| `VISIONARY_SYSTEM_METRICS` | `system_metrics` | `true` | Write `system_metrics` samples |
|
|
447
490
|
| `VISIONARY_SYSTEM_INTERVAL` | `system_interval` (seconds) | `15s` | Sample interval: `15`, `15s`, `500ms`, `1m`, `1h` |
|
|
448
491
|
| `VISIONARY_SYSTEM_RETENTION_DAYS` | `system_retention_days` | `0` (off) | Above 0, the SDK also deletes older `system_metrics` rows |
|
|
@@ -451,6 +494,8 @@ options in code, plus the hooks below. Set them before the first request
|
|
|
451
494
|
| `VISIONARY_SPAN_MAX` | `span_max` | `100` | Span rows stored per request |
|
|
452
495
|
| `VISIONARY_ERRORS` | `errors` | `true` | Record exceptions |
|
|
453
496
|
| `VISIONARY_PROFILE` | `profile` | `true` | Follow the dashboard's Profiling setting; `false` never profiles |
|
|
497
|
+
| `VISIONARY_LOGS` | `logs` | `true` | Capture log lines the app routes to the SDK (see [Logs](#logs)) |
|
|
498
|
+
| `VISIONARY_LOG_LEVEL` | `log_level` | `info` | Lowest level kept until Settings → Logs has been read |
|
|
454
499
|
|
|
455
500
|
If `VISIONARY_RELEASE` is unset, the release is read from the first of
|
|
456
501
|
`RAILWAY_GIT_COMMIT_SHA`, `RENDER_GIT_COMMIT`, `VERCEL_GIT_COMMIT_SHA`,
|
|
@@ -472,6 +517,7 @@ Options only available in code:
|
|
|
472
517
|
| `db_stats` | the ActiveRecord pool | `-> { { total:, idle:, in_use:, max: } }`, pool metrics |
|
|
473
518
|
| `disk_path` | working directory | Filesystem whose usage is reported |
|
|
474
519
|
| `queue_size` | `5000` | Rows queued before new ones are dropped |
|
|
520
|
+
| `log_queue_size` | `10000` | Log lines queued (on their own writer) before new ones are dropped |
|
|
475
521
|
| `batch_size` | `50` | Rows per write (1 to 50) |
|
|
476
522
|
| `flush_interval` | `3.0` | Seconds between writes when the batch isn't full |
|
|
477
523
|
|
|
@@ -499,6 +545,8 @@ Always redacted: the headers `Authorization`, `Proxy-Authorization`,
|
|
|
499
545
|
| `visionary: queue full; N entries dropped` | Writes can't keep up with traffic (common on SQLite and D1) | Use PostgreSQL or MySQL, or log fewer paths or no bodies |
|
|
500
546
|
| Rows lost on shutdown, or `visionary: log flush timed out; N pending` | The process was killed before the queue was flushed | Let the process exit normally (`at_exit` calls `Visionary.close`). In Puma, add `on_worker_shutdown { Visionary.close }`. In scripts, call `Visionary.close` before exiting |
|
|
501
547
|
| Nothing from Puma or Unicorn workers | The agent started in the master before `fork`, and the restart failed | Look for `visionary: restart after fork failed`. Add `on_worker_boot { Visionary.after_fork }` |
|
|
548
|
+
| No `app_logs` rows | The app's logger isn't routed to the SDK, the agent hasn't started yet, or Settings → Logs keeps a higher level | Call `Visionary.capture_rails_logger` or log through `Visionary.logger(...)`; send a request or call `Visionary.start`; check the level in Settings → Logs |
|
|
549
|
+
| `visionary: N log lines dropped over the cap of R a minute (Settings → Logs)` | The app logged more than the per-minute cap | Raise the rate in Settings → Logs, or add a drop rule for the noisy lines |
|
|
502
550
|
| A request body is empty in the log | Bodies are captured as the app reads them; the app never read it | Expected. The size is still stored, from `Content-Length` |
|
|
503
551
|
|
|
504
552
|
## How it works
|
|
@@ -521,6 +569,8 @@ Visionary SDK follows the same contract,
|
|
|
521
569
|
- `releases`: one row per process start, when the release is known.
|
|
522
570
|
- `system_metrics`: a sample every `VISIONARY_SYSTEM_INTERVAL`.
|
|
523
571
|
- `profiles`: CPU profiles, only while they are on in the dashboard.
|
|
572
|
+
- `app_logs`: the log lines the app routes to the SDK, written by a writer
|
|
573
|
+
thread of their own so a flood of lines never delays request rows.
|
|
524
574
|
|
|
525
575
|
Request bodies are captured as the app reads them. Multipart, image, video,
|
|
526
576
|
audio, octet-stream, zip and PDF bodies are not captured.
|
|
@@ -598,6 +648,8 @@ Visionary.login_attempt(params[:email]) # the account a sign-in request
|
|
|
598
648
|
Visionary.span(name, kind: "custom", target: nil, detail: nil) { ... } # a manual span
|
|
599
649
|
Visionary.background(name) { ... } # a thread that stays with the request
|
|
600
650
|
Visionary.capture_exception(e) # a handled error (handled = 1)
|
|
651
|
+
Visionary.logger(inner = nil) # a Logger that also writes app_logs
|
|
652
|
+
Visionary.capture_rails_logger # copy Rails.logger to app_logs
|
|
601
653
|
Visionary.configure { |c| ... } # options and hooks, before the first request
|
|
602
654
|
Visionary.start # start now instead of on the first request
|
|
603
655
|
Visionary.after_fork # restart threads in a forked worker
|
data/lib/visionary/agent.rb
CHANGED
|
@@ -27,7 +27,7 @@ module Visionary
|
|
|
27
27
|
# no database configured it logs one warning and every call is a no-op; it
|
|
28
28
|
# never raises into the app once built.
|
|
29
29
|
class Agent
|
|
30
|
-
attr_reader :config, :http, :telemetry, :logger, :instance
|
|
30
|
+
attr_reader :config, :http, :telemetry, :logger, :instance, :logs
|
|
31
31
|
|
|
32
32
|
def initialize(cfg)
|
|
33
33
|
@config = cfg
|
|
@@ -46,6 +46,7 @@ module Visionary
|
|
|
46
46
|
end
|
|
47
47
|
@profiler = Profiler.new(cfg, @instance, @store, @writer, @logger) if cfg.profile && @writer
|
|
48
48
|
@telemetry = Telemetry::Recorder.new(cfg, @instance, @writer) if cfg.spans || cfg.errors
|
|
49
|
+
@logs = Logs::Pipeline.new(cfg, @instance, @store, @schemas, @logger) if cfg.logs
|
|
49
50
|
if cfg.system_metrics && cfg.system_interval.to_f.positive?
|
|
50
51
|
@system = System::Recorder.new(cfg, @instance, @store, @schemas, @http, @logger)
|
|
51
52
|
end
|
|
@@ -57,10 +58,11 @@ module Visionary
|
|
|
57
58
|
end
|
|
58
59
|
tables["system_metrics"] = cfg.system_retention_days if @system
|
|
59
60
|
tables["profiles"] = cfg.log_retention_days if @profiler
|
|
61
|
+
tables["app_logs"] = cfg.log_retention_days if @logs
|
|
60
62
|
@retention = Retention.new(@store, @logger, tables)
|
|
61
63
|
end
|
|
62
64
|
|
|
63
|
-
def enabled? = !@store.nil? && !@closed && (!@writer.nil? || !@system.nil?)
|
|
65
|
+
def enabled? = !@store.nil? && !@closed && (!@writer.nil? || !@system.nil? || !@logs.nil?)
|
|
64
66
|
|
|
65
67
|
def log_path?(path) = @config.log_requests && !@writer.nil? && path.start_with?(@config.path_prefix.to_s)
|
|
66
68
|
|
|
@@ -94,6 +96,7 @@ module Visionary
|
|
|
94
96
|
@system.start
|
|
95
97
|
@logger.info("visionary: recording system metrics every #{@config.system_interval}s as #{@instance}")
|
|
96
98
|
end
|
|
99
|
+
@logs&.start
|
|
97
100
|
@retention.start
|
|
98
101
|
@profiler&.start
|
|
99
102
|
install_hooks
|
|
@@ -112,6 +115,7 @@ module Visionary
|
|
|
112
115
|
@retention&.close
|
|
113
116
|
@profiler&.close
|
|
114
117
|
idle = @writer ? @writer.close(timeout) : true
|
|
118
|
+
idle = @logs.close([deadline - mono, 0].max) && idle if @logs
|
|
115
119
|
idle = @system.close([deadline - mono, 0].max) && idle if @system
|
|
116
120
|
@store.close if idle && @store
|
|
117
121
|
rescue StandardError => e
|
|
@@ -130,6 +134,7 @@ module Visionary
|
|
|
130
134
|
@system&.http = @http
|
|
131
135
|
@system&.after_fork
|
|
132
136
|
@writer&.after_fork
|
|
137
|
+
@logs&.after_fork
|
|
133
138
|
@retention&.after_fork if @retention&.needed?
|
|
134
139
|
@profiler&.after_fork
|
|
135
140
|
@logger.info("visionary: restarted in forked process #{@pid}")
|
data/lib/visionary/config.rb
CHANGED
|
@@ -22,7 +22,7 @@ module Visionary
|
|
|
22
22
|
:error_code_field, :error_key_field,
|
|
23
23
|
:system_metrics, :system_interval, :system_retention_days, :disk_path,
|
|
24
24
|
:service, :version, :spans, :span_threshold_ms, :span_max, :errors,
|
|
25
|
-
:profile
|
|
25
|
+
:profile, :logs, :log_level, :log_queue_size
|
|
26
26
|
|
|
27
27
|
# Runs on the writer thread, once per distinct user per batch, for users
|
|
28
28
|
# missing a handle or role: ->(id) { [handle, role] } or nil.
|
|
@@ -66,6 +66,11 @@ module Visionary
|
|
|
66
66
|
@errors = true
|
|
67
67
|
# false: never profile, whatever the dashboard says.
|
|
68
68
|
@profile = true
|
|
69
|
+
# false: never capture application logs (VISIONARY_LOGS).
|
|
70
|
+
@logs = true
|
|
71
|
+
# The lowest level kept until Settings → Logs has been read.
|
|
72
|
+
@log_level = "info"
|
|
73
|
+
@log_queue_size = 10_000
|
|
69
74
|
end
|
|
70
75
|
|
|
71
76
|
def self.from_env(env = ENV)
|
|
@@ -97,6 +102,8 @@ module Visionary
|
|
|
97
102
|
c.span_max = parse_int(s.("VISIONARY_SPAN_MAX"), 100)
|
|
98
103
|
c.errors = parse_bool(s.("VISIONARY_ERRORS"), true)
|
|
99
104
|
c.profile = parse_bool(s.("VISIONARY_PROFILE"), true)
|
|
105
|
+
c.logs = parse_bool(s.("VISIONARY_LOGS"), true)
|
|
106
|
+
c.log_level = s.("VISIONARY_LOG_LEVEL").then { |v| Logs.parse_level(v).positive? ? v.downcase : "info" }
|
|
100
107
|
c
|
|
101
108
|
end
|
|
102
109
|
|
data/lib/visionary/context.rb
CHANGED
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
module Visionary
|
|
4
4
|
# What the request being handled has said about itself: the user, the route
|
|
5
|
-
# pattern,
|
|
5
|
+
# pattern, the spans and errors recorded while it ran, and the log lines
|
|
6
|
+
# held back until it ends.
|
|
6
7
|
#
|
|
7
8
|
# The middleware puts one object in fiber storage (Ruby 3.2+, inherited by
|
|
8
9
|
# child fibers and threads, so it works under Falcon and in Async tasks) or a
|
|
@@ -11,10 +12,11 @@ module Visionary
|
|
|
11
12
|
class RequestContext
|
|
12
13
|
KEY = :visionary_request
|
|
13
14
|
|
|
14
|
-
attr_reader :request_id, :method, :path, :started_at, :started_mono, :errors, :db_visible
|
|
15
|
+
attr_reader :request_id, :method, :path, :started_at, :started_mono, :errors, :db_visible, :trace_id
|
|
15
16
|
|
|
16
|
-
def initialize(request_id:, method:, path:, started_at:, started_mono:, span_max:, db_visible:)
|
|
17
|
+
def initialize(request_id:, method:, path:, started_at:, started_mono:, span_max:, db_visible:, trace_id: nil)
|
|
17
18
|
@request_id = request_id
|
|
19
|
+
@trace_id = trace_id
|
|
18
20
|
@method = method
|
|
19
21
|
@path = path
|
|
20
22
|
@started_at = started_at
|
|
@@ -31,6 +33,8 @@ module Visionary
|
|
|
31
33
|
@db_ms = @http_ms = 0.0
|
|
32
34
|
@logged = false
|
|
33
35
|
@late = 0
|
|
36
|
+
@failed = false
|
|
37
|
+
@held = nil
|
|
34
38
|
end
|
|
35
39
|
|
|
36
40
|
def set_user(id, handle, role)
|
|
@@ -91,6 +95,32 @@ module Visionary
|
|
|
91
95
|
@mutex.synchronize { @errors << row }
|
|
92
96
|
end
|
|
93
97
|
|
|
98
|
+
# Marks the request failed, so its held log lines are kept down to logs.buffer_level.
|
|
99
|
+
def failed!
|
|
100
|
+
@mutex.synchronize { @failed = true }
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def failed? = @mutex.synchronize { @failed || !@errors.empty? }
|
|
104
|
+
|
|
105
|
+
# Holds at most max lines, dropping the oldest. False once the request has been logged.
|
|
106
|
+
def hold_log(record, max)
|
|
107
|
+
@mutex.synchronize do
|
|
108
|
+
return false if @logged
|
|
109
|
+
@held ||= []
|
|
110
|
+
@held.shift if @held.size >= max
|
|
111
|
+
@held << record
|
|
112
|
+
true
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def take_held_logs
|
|
117
|
+
@mutex.synchronize do
|
|
118
|
+
out = @held || []
|
|
119
|
+
@held = nil
|
|
120
|
+
out
|
|
121
|
+
end
|
|
122
|
+
end
|
|
123
|
+
|
|
94
124
|
# [db_count, db_ms, http_count, http_ms]; database totals are NULL when no
|
|
95
125
|
# database hook could see this request's calls.
|
|
96
126
|
def totals
|
|
@@ -0,0 +1,274 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Visionary
|
|
4
|
+
module Logs
|
|
5
|
+
# A ::Logger that copies every line to app_logs and forwards every call to inner unchanged.
|
|
6
|
+
# Visionary also sees levels inner doesn't print, so failed requests keep their debug lines.
|
|
7
|
+
# In a Hash message, "message"/"msg" is the text and the other keys become attrs.
|
|
8
|
+
class Logger
|
|
9
|
+
# ::Logger's severities, without requiring the logger gem.
|
|
10
|
+
SEVERITIES = { debug: 0, info: 1, warn: 2, error: 3, fatal: 4, unknown: 5 }.freeze
|
|
11
|
+
TO_SEV = [DEBUG, INFO, WARN, ERROR, FATAL, FATAL].freeze
|
|
12
|
+
GUARD = :visionary_capturing
|
|
13
|
+
SDK_DIR = File.expand_path("..", __dir__).tr("\\", "/")
|
|
14
|
+
# Frames of the logging machinery between the app and this logger.
|
|
15
|
+
MACHINERY = %r{/logger\.rb\z|/logger/[^/]+\.rb\z|/active_support/(?:broadcast_logger|tagged_logging|logger\w*)\.rb\z|/semantic_logger/}
|
|
16
|
+
|
|
17
|
+
attr_reader :inner
|
|
18
|
+
attr_accessor :progname, :formatter, :datetime_format
|
|
19
|
+
|
|
20
|
+
# tags_from: a logger whose ActiveSupport::TaggedLogging tags are
|
|
21
|
+
# stored as the "tags" attribute (inner when not given).
|
|
22
|
+
def initialize(inner = nil, progname: nil, tags_from: nil)
|
|
23
|
+
@inner = inner
|
|
24
|
+
@progname = progname
|
|
25
|
+
@tags_from = tags_from || inner
|
|
26
|
+
@level = nil
|
|
27
|
+
@local_key = :"visionary_log_level_#{object_id}"
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def add(severity, message = nil, progname = nil, &block)
|
|
31
|
+
block = once(block)
|
|
32
|
+
result = @inner ? @inner.add(severity, message, progname, &block) : true
|
|
33
|
+
capture(severity, message, progname, block)
|
|
34
|
+
result
|
|
35
|
+
end
|
|
36
|
+
alias log add
|
|
37
|
+
|
|
38
|
+
SEVERITIES.each do |name, n|
|
|
39
|
+
define_method(name) do |progname = nil, &block|
|
|
40
|
+
block = once(block)
|
|
41
|
+
result = @inner ? @inner.public_send(name, progname, &block) : true
|
|
42
|
+
capture(n, nil, progname, block)
|
|
43
|
+
result
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
define_method(:"#{name}?") do
|
|
47
|
+
(@inner&.public_send(:"#{name}?") || false) || n >= capture_level
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
define_method(:"#{name}!") do
|
|
51
|
+
@inner&.public_send(:"#{name}!")
|
|
52
|
+
@level = n unless @inner
|
|
53
|
+
self
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def <<(msg)
|
|
58
|
+
result = @inner ? (@inner << msg) : self
|
|
59
|
+
text = Visionary.utf8(msg).chomp
|
|
60
|
+
sev = Logs.text_level(text)
|
|
61
|
+
sev = INFO if sev.zero?
|
|
62
|
+
capture_record(sev, Logs.strip_level_word(text, sev), nil, {})
|
|
63
|
+
result
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Alone (broadcast from Rails.logger), the lowest level Visionary keeps, so those lines reach it.
|
|
67
|
+
def level
|
|
68
|
+
return @inner.level if @inner
|
|
69
|
+
capture_level
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Alone it changes nothing: what is kept follows Settings → Logs.
|
|
73
|
+
def level=(v)
|
|
74
|
+
@inner ? (@inner.level = v) : (@level = v)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# Set per thread or fiber by ActiveSupport's LoggerSilence (Rails.logger.silence).
|
|
78
|
+
def local_level
|
|
79
|
+
Thread.current[@local_key]
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def local_level=(v)
|
|
83
|
+
@inner.local_level = v if @inner.respond_to?(:local_level=)
|
|
84
|
+
Thread.current[@local_key] = v.nil? ? nil : Logger.severity(v)
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
def silence(severity = SEVERITIES[:error], &block)
|
|
88
|
+
prev = Thread.current[@local_key]
|
|
89
|
+
Thread.current[@local_key] = Logger.severity(severity)
|
|
90
|
+
@inner.respond_to?(:silence) ? @inner.silence(severity, &block) : yield(self)
|
|
91
|
+
ensure
|
|
92
|
+
Thread.current[@local_key] = prev
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def close = @inner&.close
|
|
96
|
+
def reopen(*args) = @inner&.reopen(*args)
|
|
97
|
+
|
|
98
|
+
def method_missing(name, *args, **kwargs, &block)
|
|
99
|
+
return super unless @inner.respond_to?(name)
|
|
100
|
+
@inner.public_send(name, *args, **kwargs, &block)
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def respond_to_missing?(name, include_private = false)
|
|
104
|
+
@inner.respond_to?(name) || super
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def self.severity(v)
|
|
108
|
+
return v if v.is_a?(Integer)
|
|
109
|
+
SEVERITIES.fetch(v.to_s.downcase.to_sym, SEVERITIES[:unknown])
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
private
|
|
113
|
+
|
|
114
|
+
def pipeline
|
|
115
|
+
agent = Visionary.running_agent
|
|
116
|
+
agent.respond_to?(:logs) ? agent.logs : nil
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
def capture_level
|
|
120
|
+
lowest = pipeline&.lowest
|
|
121
|
+
return SEVERITIES[:unknown] unless lowest
|
|
122
|
+
TO_SEV.index(lowest) || 0
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
# A block that runs at most once, so inner and the capture share its value.
|
|
126
|
+
def once(block)
|
|
127
|
+
return nil unless block
|
|
128
|
+
done = false
|
|
129
|
+
value = nil
|
|
130
|
+
proc do |*args|
|
|
131
|
+
unless done
|
|
132
|
+
value = block.call(*args)
|
|
133
|
+
done = true
|
|
134
|
+
end
|
|
135
|
+
value
|
|
136
|
+
end
|
|
137
|
+
end
|
|
138
|
+
|
|
139
|
+
# Like ::Logger#add: with no message, the block or else progname is the message.
|
|
140
|
+
def capture(severity, message, progname, block)
|
|
141
|
+
pipe = pipeline
|
|
142
|
+
return unless pipe
|
|
143
|
+
n = severity.nil? ? SEVERITIES[:unknown] : Logger.severity(severity)
|
|
144
|
+
sev = TO_SEV[n.clamp(0, 5)]
|
|
145
|
+
return if skip?(n) || !pipe.wants?(sev)
|
|
146
|
+
|
|
147
|
+
if message.nil?
|
|
148
|
+
if block
|
|
149
|
+
message = block.call
|
|
150
|
+
else
|
|
151
|
+
message = progname
|
|
152
|
+
progname = nil
|
|
153
|
+
end
|
|
154
|
+
end
|
|
155
|
+
progname ||= @progname
|
|
156
|
+
text, attrs = Logger.split(message)
|
|
157
|
+
capture_record(sev, text, progname, attrs, pipe)
|
|
158
|
+
rescue StandardError
|
|
159
|
+
nil
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
def capture_record(sev, text, progname, attrs, pipe = pipeline)
|
|
163
|
+
return if pipe.nil? || Thread.current[GUARD] || Telemetry.internal?
|
|
164
|
+
return if text.empty? && attrs.empty?
|
|
165
|
+
|
|
166
|
+
Thread.current[GUARD] = true
|
|
167
|
+
begin
|
|
168
|
+
rec = Record.new(Time.now, sev, progname.nil? ? nil : progname.to_s, text)
|
|
169
|
+
tags = Logger.tags(@tags_from)
|
|
170
|
+
attrs["tags"] = tags.join(", ") unless tags.empty?
|
|
171
|
+
COLUMN_ATTRS.each do |k|
|
|
172
|
+
v = attrs[k]
|
|
173
|
+
next unless v.is_a?(String) || v.is_a?(Integer)
|
|
174
|
+
rec[k.to_sym] = v.to_s
|
|
175
|
+
attrs.delete(k)
|
|
176
|
+
end
|
|
177
|
+
rec.attrs = attrs unless attrs.empty?
|
|
178
|
+
if (loc = Logger.location)
|
|
179
|
+
rec.file = Logs.frames.file(loc.absolute_path || loc.path)
|
|
180
|
+
rec.line = loc.lineno
|
|
181
|
+
rec.func = loc.label
|
|
182
|
+
end
|
|
183
|
+
otel(rec) if rec.trace_id.nil?
|
|
184
|
+
pipe.capture(rec)
|
|
185
|
+
ensure
|
|
186
|
+
Thread.current[GUARD] = nil
|
|
187
|
+
end
|
|
188
|
+
rescue StandardError
|
|
189
|
+
nil
|
|
190
|
+
end
|
|
191
|
+
|
|
192
|
+
def skip?(n)
|
|
193
|
+
return true if Thread.current[GUARD] || Telemetry.internal?
|
|
194
|
+
local = Thread.current[@local_key]
|
|
195
|
+
!local.nil? && n < local
|
|
196
|
+
end
|
|
197
|
+
|
|
198
|
+
# Falls back to the active OpenTelemetry span when the request brought no traceparent.
|
|
199
|
+
def otel(rec)
|
|
200
|
+
return unless defined?(::OpenTelemetry::Trace)
|
|
201
|
+
ctx = RequestContext.current
|
|
202
|
+
return if ctx&.trace_id
|
|
203
|
+
sc = ::OpenTelemetry::Trace.current_span.context
|
|
204
|
+
return unless sc.valid?
|
|
205
|
+
rec.trace_id = sc.hex_trace_id
|
|
206
|
+
rec.span_id = sc.hex_span_id
|
|
207
|
+
rescue StandardError
|
|
208
|
+
nil
|
|
209
|
+
end
|
|
210
|
+
|
|
211
|
+
class << self
|
|
212
|
+
def split(message)
|
|
213
|
+
case message
|
|
214
|
+
when String then [Visionary.utf8(message), {}]
|
|
215
|
+
when Exception
|
|
216
|
+
text = "#{message.class}: #{message.message}"
|
|
217
|
+
bt = message.backtrace
|
|
218
|
+
text = "#{text}\n#{bt.join("\n")}" if bt && !bt.empty?
|
|
219
|
+
[Visionary.utf8(text), {}]
|
|
220
|
+
when Hash
|
|
221
|
+
attrs = Logs.flatten(message)
|
|
222
|
+
text = (attrs.delete("message") || attrs.delete("msg")).to_s
|
|
223
|
+
# With no message the line reads like logfmt, as it is printed.
|
|
224
|
+
text = attrs.map { |k, v| "#{k}=#{v}" }.join(" ") if text.empty?
|
|
225
|
+
[text, attrs]
|
|
226
|
+
else [Visionary.utf8(message.inspect), {}]
|
|
227
|
+
end
|
|
228
|
+
end
|
|
229
|
+
|
|
230
|
+
# TaggedLogging tags, including through a Rails 7.1+ BroadcastLogger.
|
|
231
|
+
def tags(logger)
|
|
232
|
+
return [] unless logger
|
|
233
|
+
loggers = logger.respond_to?(:broadcasts) ? [logger, *logger.broadcasts] : [logger]
|
|
234
|
+
loggers.each do |l|
|
|
235
|
+
f = l.respond_to?(:formatter) ? l.formatter : nil
|
|
236
|
+
next unless f.respond_to?(:current_tags)
|
|
237
|
+
tags = f.current_tags
|
|
238
|
+
return tags.map(&:to_s) unless tags.nil? || tags.empty?
|
|
239
|
+
end
|
|
240
|
+
[]
|
|
241
|
+
rescue StandardError
|
|
242
|
+
[]
|
|
243
|
+
end
|
|
244
|
+
|
|
245
|
+
# The first frame outside this SDK and the logger machinery.
|
|
246
|
+
def location
|
|
247
|
+
i = 1
|
|
248
|
+
while i < 40
|
|
249
|
+
loc = caller_locations(i, 1)&.first
|
|
250
|
+
return nil unless loc
|
|
251
|
+
return loc unless machinery?(loc.absolute_path || loc.path)
|
|
252
|
+
i += 1
|
|
253
|
+
end
|
|
254
|
+
nil
|
|
255
|
+
end
|
|
256
|
+
|
|
257
|
+
def machinery?(path)
|
|
258
|
+
return true if path.nil? || path.start_with?("<internal:")
|
|
259
|
+
p = path.tr("\\", "/")
|
|
260
|
+
sdk = "#{SDK_DIR}/"
|
|
261
|
+
if Gem.win_platform?
|
|
262
|
+
p = p.downcase
|
|
263
|
+
sdk = sdk.downcase
|
|
264
|
+
end
|
|
265
|
+
p.start_with?(sdk) || MACHINERY.match?(p)
|
|
266
|
+
end
|
|
267
|
+
end
|
|
268
|
+
end
|
|
269
|
+
|
|
270
|
+
def self.frames
|
|
271
|
+
@frames ||= Profiler::Frames.new
|
|
272
|
+
end
|
|
273
|
+
end
|
|
274
|
+
end
|