visionary-sdk 1.1.1 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +17 -0
- data/LICENSE +87 -69
- data/README.md +73 -7
- data/lib/visionary/agent.rb +13 -2
- data/lib/visionary/config.rb +9 -2
- data/lib/visionary/context.rb +9 -4
- data/lib/visionary/logs/logger.rb +2 -3
- data/lib/visionary/logs/pipeline.rb +5 -2
- data/lib/visionary/metrics.rb +392 -0
- data/lib/visionary/middleware.rb +10 -5
- data/lib/visionary/queue.rb +4 -0
- data/lib/visionary/schema/mysql/metrics.sql +20 -0
- data/lib/visionary/schema/postgres/metrics.sql +20 -0
- data/lib/visionary/schema/sqlite/metrics.sql +20 -0
- data/lib/visionary/schema.rb +27 -7
- data/lib/visionary/schemas.rb +1 -0
- data/lib/visionary/store/active_record.rb +1 -0
- data/lib/visionary/store/sql.rb +94 -24
- data/lib/visionary/telemetry/hooks.rb +18 -5
- data/lib/visionary/telemetry.rb +9 -4
- data/lib/visionary/trace_context.rb +111 -0
- data/lib/visionary/version.rb +1 -1
- data/lib/visionary/writer.rb +89 -17
- data/lib/visionary.rb +30 -7
- metadata +8 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: '06876950e4e7e007c2e59fe232268e03b981383062f47c4a1ef2e7c8fe803df2'
|
|
4
|
+
data.tar.gz: bb09cbeffec3774e6bec59a5114ecdc5aaa80ebef88c77ff11c3c9f674445dcb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c220abde41d7f0f77a15385ee8f4ecab2fabea986ff25c9e8a64e61880d65f7370b97561f3983137a8ff77c1165825054ac9141469352ad71c890c120f3bd3e2
|
|
7
|
+
data.tar.gz: '0091d167f78f3b35379febe0adcfbfdc8da41ea7c4ca561f62545fad9d1a5dcf49426b7d0faff9691336f7daeeb4d58df6007bf408aee18bfa731ddb16107cf6'
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,23 @@
|
|
|
2
2
|
|
|
3
3
|
## Unreleased
|
|
4
4
|
|
|
5
|
+
- Faster writes: SQL backends get each table's rows as multi-row INSERTs
|
|
6
|
+
(500 rows a batch, `batch_size` up to 2000), with up to four batches in
|
|
7
|
+
flight on PostgreSQL and MySQL. SQLite opens in WAL mode with a 5 s busy
|
|
8
|
+
timeout. D1 is unchanged. `Visionary.writer_stats` reports rows queued,
|
|
9
|
+
written and dropped.
|
|
10
|
+
|
|
11
|
+
- Custom metrics in `metrics`: `Visionary.counter(name, **labels).add(n)`,
|
|
12
|
+
`Visionary.gauge(...).set(v)`, `Visionary.histogram(...).observe(v)` and
|
|
13
|
+
`Visionary.describe_metric(name, unit:, buckets:)`. Added up in memory and
|
|
14
|
+
written one row per series every interval by a writer thread of their own;
|
|
15
|
+
at most 1000 label sets per name. `VISIONARY_METRICS`,
|
|
16
|
+
`VISIONARY_METRICS_INTERVAL`, `VISIONARY_METRICS_MAX_SERIES`.
|
|
17
|
+
- Distributed tracing: requests continue an incoming W3C `traceparent` or
|
|
18
|
+
start a trace, `Net::HTTP` sends `traceparent` (and `tracestate`) on, and
|
|
19
|
+
`api_logs` and `spans` get `trace_id`, `span_id` and `parent_span_id`
|
|
20
|
+
(added to existing tables at startup). Calls inside `Visionary.span` and
|
|
21
|
+
`Visionary.background` nest under them; log lines carry the current span.
|
|
5
22
|
- Application logs in `app_logs`: `Visionary.logger(inner = nil)` (a Logger
|
|
6
23
|
that forwards every call to `inner` unchanged and also captures) and
|
|
7
24
|
`Visionary.capture_rails_logger` (broadcasts `Rails.logger`). Lines link to
|
data/LICENSE
CHANGED
|
@@ -1,94 +1,108 @@
|
|
|
1
|
-
Visionary
|
|
1
|
+
Visionary Source-Available License (VSAL) v2.0
|
|
2
2
|
|
|
3
3
|
Copyright (c) 2026 Saurav Sajeev. All rights reserved, except as expressly
|
|
4
4
|
permitted below.
|
|
5
5
|
|
|
6
|
+
In short: you may use, run, copy and modify Visionary for any purpose,
|
|
7
|
+
including in a business and in production, to monitor software of your own,
|
|
8
|
+
even software that makes money. You may not sell Visionary, charge for access
|
|
9
|
+
to it, or offer it to others as a product or service. This summary is for
|
|
10
|
+
convenience; the terms below are what apply.
|
|
11
|
+
|
|
6
12
|
|
|
7
13
|
1. Definitions
|
|
8
14
|
|
|
9
15
|
"Software" means Visionary, including its source code, object code, server,
|
|
10
|
-
host agent, SDKs, libraries, documentation, and associated
|
|
11
|
-
distributed under this License.
|
|
16
|
+
host agent, SDKs, libraries, command-line tools, documentation, and associated
|
|
17
|
+
materials distributed under this License.
|
|
12
18
|
|
|
13
19
|
"Author" means Saurav Sajeev, the original author and copyright holder of the
|
|
14
20
|
Software, to the extent of their ownership of the relevant material.
|
|
15
21
|
|
|
16
|
-
"
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
22
|
+
"SDKs" means the client libraries distributed as part of the Software for
|
|
23
|
+
embedding in other software, including the server, browser, mobile and test
|
|
24
|
+
reporter libraries.
|
|
25
|
+
|
|
26
|
+
"Your Applications" means software, websites, mobile apps, services and
|
|
27
|
+
infrastructure that you or your organization build, own or operate, other than
|
|
28
|
+
the Software itself.
|
|
29
|
+
|
|
30
|
+
"Sell or Monetize" means any of the following, whether or not the Software is
|
|
31
|
+
modified:
|
|
32
|
+
|
|
33
|
+
a. Selling, licensing, sublicensing, renting or leasing the Software or
|
|
34
|
+
copies of it.
|
|
35
|
+
|
|
36
|
+
b. Charging any fee, subscription or other consideration for access to the
|
|
37
|
+
Software or to its functionality.
|
|
38
|
+
|
|
39
|
+
c. Offering the Software to third parties as a hosted, managed,
|
|
40
|
+
white-labelled or software-as-a-service product, whether paid or offered
|
|
41
|
+
to win or keep paying customers.
|
|
42
|
+
|
|
43
|
+
d. Including the Software, or a substantial portion of it, in a product or
|
|
44
|
+
service that you sell or license to third parties, where the Software's
|
|
45
|
+
functionality is a substantial part of what the customer receives.
|
|
21
46
|
|
|
22
47
|
"Redistribution" means making the Software, or a modified or derivative version
|
|
23
48
|
of it, available to another person or entity by distribution, publication,
|
|
24
49
|
hosting for download, sublicensing, or other transfer.
|
|
25
50
|
|
|
26
|
-
"Internal Use" means use by an individual or an organization solely for its
|
|
27
|
-
own internal operations, without providing the Software or its functionality
|
|
28
|
-
as a product or service to external customers or third parties.
|
|
29
|
-
|
|
30
51
|
|
|
31
52
|
2. Grant of Permission
|
|
32
53
|
|
|
33
|
-
Subject to the terms of this License, the Author grants you a
|
|
34
|
-
|
|
35
|
-
applicable law, and non-transferable license to:
|
|
54
|
+
Subject to the terms of this License, the Author grants you a worldwide,
|
|
55
|
+
non-exclusive, royalty-free, non-transferable license to:
|
|
36
56
|
|
|
37
57
|
a. Access, view, and study the Software's source code.
|
|
38
58
|
|
|
39
|
-
b. Download
|
|
59
|
+
b. Download, install, run and use the Software for any purpose, personal or
|
|
60
|
+
commercial, in development, testing, staging and production, within an
|
|
61
|
+
organization of any size.
|
|
62
|
+
|
|
63
|
+
c. Use the Software to monitor, observe, analyse, test and alert on Your
|
|
64
|
+
Applications, including applications you sell, applications that earn
|
|
65
|
+
revenue, and applications you build or operate for your clients.
|
|
40
66
|
|
|
41
|
-
|
|
42
|
-
production environments for Internal Use.
|
|
67
|
+
d. Modify the Software and use your modified version under the same terms.
|
|
43
68
|
|
|
44
|
-
|
|
45
|
-
commercial organizations, provided such modifications are not
|
|
46
|
-
redistributed or commercially exploited.
|
|
69
|
+
e. Make copies needed for the uses above, including backups.
|
|
47
70
|
|
|
48
|
-
|
|
71
|
+
f. Have contractors, consultants and service providers exercise these rights
|
|
72
|
+
on your behalf and for your benefit, provided they comply with this
|
|
73
|
+
License.
|
|
49
74
|
|
|
50
75
|
These permissions do not transfer ownership of the Software or grant any rights
|
|
51
76
|
beyond those expressly stated in this License.
|
|
52
77
|
|
|
53
78
|
|
|
54
|
-
3.
|
|
79
|
+
3. SDKs Inside Your Applications
|
|
55
80
|
|
|
56
|
-
You
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
the Software.
|
|
60
|
-
|
|
61
|
-
b. Offer the Software or a modified version as a paid product, subscription,
|
|
62
|
-
managed service, or hosted monitoring service.
|
|
81
|
+
You may include the SDKs, modified or not, in Your Applications and distribute
|
|
82
|
+
them as part of Your Applications, for example in a website's scripts, a mobile
|
|
83
|
+
app, a server application or a container image, provided that:
|
|
63
84
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
offered to third parties.
|
|
85
|
+
a. the SDKs are distributed only as part of Your Applications, not as a
|
|
86
|
+
standalone library, package or product; and
|
|
67
87
|
|
|
68
|
-
|
|
69
|
-
|
|
88
|
+
b. the copyright notice and this License (or a link to it) are kept with
|
|
89
|
+
the SDK code where your build process keeps such notices.
|
|
70
90
|
|
|
71
|
-
|
|
72
|
-
|
|
91
|
+
Installing the SDKs from the package registries where the Author publishes them
|
|
92
|
+
is permitted.
|
|
73
93
|
|
|
74
|
-
Internal production use by a commercial organization is permitted under
|
|
75
|
-
Section 2, provided that the organization does not commercially exploit or
|
|
76
|
-
redistribute the Software in a manner prohibited by this License.
|
|
77
94
|
|
|
95
|
+
4. Restrictions
|
|
78
96
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
Without the Author's prior written permission, you must not:
|
|
97
|
+
You must not, without the Author's prior written permission:
|
|
82
98
|
|
|
83
|
-
a.
|
|
84
|
-
form.
|
|
99
|
+
a. Sell or Monetize the Software.
|
|
85
100
|
|
|
86
|
-
b.
|
|
87
|
-
or
|
|
101
|
+
b. Redistribute the Software, a modified version, fork or derivative work,
|
|
102
|
+
in source or binary form, except as Section 3 allows.
|
|
88
103
|
|
|
89
|
-
c.
|
|
90
|
-
|
|
91
|
-
platforms for redistribution.
|
|
104
|
+
c. Publish the Software or modified versions to public package registries,
|
|
105
|
+
marketplaces, application stores or download platforms.
|
|
92
106
|
|
|
93
107
|
d. Sublicense, transfer, or grant others rights to use the Software beyond
|
|
94
108
|
the permissions expressly provided in this License.
|
|
@@ -96,22 +110,24 @@ Without the Author's prior written permission, you must not:
|
|
|
96
110
|
e. Remove or misrepresent the Software's authorship, copyright notices, or
|
|
97
111
|
license information.
|
|
98
112
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
infrastructure.
|
|
113
|
+
f. Use the Author's name, the Visionary name or its logos to imply
|
|
114
|
+
endorsement of, or to market, your own products or services.
|
|
102
115
|
|
|
116
|
+
For clarity, the following are permitted and are not Selling or Monetizing:
|
|
117
|
+
running the Software inside a business; using it for applications that make
|
|
118
|
+
money; charging your clients for work on their own applications while you use
|
|
119
|
+
the Software to monitor them; and installing or configuring the Software on a
|
|
120
|
+
client's own infrastructure for that client's own use, provided you do not
|
|
121
|
+
charge for the Software itself or for access to it.
|
|
103
122
|
|
|
104
|
-
5. Commercial Licensing
|
|
105
|
-
|
|
106
|
-
The Author may separately grant commercial licenses, redistribution rights, or
|
|
107
|
-
other permissions under different terms.
|
|
108
123
|
|
|
109
|
-
|
|
110
|
-
exploit the Software beyond the permissions granted here must obtain prior
|
|
111
|
-
written authorization from the Author.
|
|
124
|
+
5. Commercial Licensing
|
|
112
125
|
|
|
113
|
-
|
|
114
|
-
|
|
126
|
+
The Author may separately grant licenses to Sell or Monetize, redistribute,
|
|
127
|
+
sublicense or otherwise use the Software beyond the permissions granted here.
|
|
128
|
+
Such a license requires prior written authorization from the Author and may set
|
|
129
|
+
fees, attribution requirements, support terms, distribution rights and other
|
|
130
|
+
conditions.
|
|
115
131
|
|
|
116
132
|
|
|
117
133
|
6. Ownership and Intellectual Property
|
|
@@ -124,8 +140,8 @@ identify the unmodified Software truthfully.
|
|
|
124
140
|
|
|
125
141
|
You retain ownership of your independently created contributions to the extent
|
|
126
142
|
provided by applicable law, but this does not grant you permission to
|
|
127
|
-
|
|
128
|
-
|
|
143
|
+
redistribute, Sell or Monetize the Software or derivative works in violation
|
|
144
|
+
of this License.
|
|
129
145
|
|
|
130
146
|
|
|
131
147
|
7. Third-Party Components
|
|
@@ -147,8 +163,7 @@ Your permissions under this License terminate automatically if you materially
|
|
|
147
163
|
violate its terms.
|
|
148
164
|
|
|
149
165
|
Upon termination, you must cease activities that are no longer authorized by
|
|
150
|
-
this License,
|
|
151
|
-
subject to any rights or remedies provided by applicable law.
|
|
166
|
+
this License, subject to any rights or remedies provided by applicable law.
|
|
152
167
|
|
|
153
168
|
Permissions may be reinstated by the Author in writing.
|
|
154
169
|
|
|
@@ -195,14 +210,17 @@ determined by a separate written agreement between the relevant parties or, in
|
|
|
195
210
|
the absence of such an agreement, by applicable law and conflict-of-law rules.
|
|
196
211
|
|
|
197
212
|
|
|
198
|
-
13. Entire Agreement
|
|
213
|
+
13. Entire Agreement and Versions
|
|
199
214
|
|
|
200
215
|
This License governs the permissions granted by the Author for the Software
|
|
201
216
|
covered by it. If any provision is held unenforceable, the remaining provisions
|
|
202
217
|
shall continue to apply to the extent permitted by law.
|
|
203
218
|
|
|
219
|
+
This version applies to copies of the Software that carry it. It replaces the
|
|
220
|
+
Visionary Non-Commercial License (VNCL) v1.0 for those copies.
|
|
221
|
+
|
|
204
222
|
|
|
205
|
-
License: Visionary
|
|
223
|
+
License: Visionary Source-Available License (VSAL) v2.0
|
|
206
224
|
Copyright holder: Saurav Sajeev
|
|
207
225
|
Project: Visionary
|
|
208
|
-
|
|
226
|
+
Selling or monetizing: Requires separate written authorization.
|
data/README.md
CHANGED
|
@@ -3,8 +3,9 @@
|
|
|
3
3
|
Add one gem to a Rails or Rack app and every request shows up in the
|
|
4
4
|
[Visionary](../../Readme.md) dashboard: route, status, time, user, redacted
|
|
5
5
|
bodies, the database and HTTP calls it made, and any exception. You also get
|
|
6
|
-
the app's own log lines linked to their request,
|
|
7
|
-
|
|
6
|
+
the app's own log lines linked to their request, traces that follow a request
|
|
7
|
+
across services, your own counters, gauges and histograms, host and process
|
|
8
|
+
metrics, and CPU profiles when you turn them on in the dashboard.
|
|
8
9
|
|
|
9
10
|
- [Quick start](#quick-start)
|
|
10
11
|
- [Integration recipes](#integration-recipes)
|
|
@@ -70,6 +71,11 @@ visionary: recording system metrics every 15.0s as web-1
|
|
|
70
71
|
Tables are created on first use. At exit the SDK flushes what is queued (it
|
|
71
72
|
registers an `at_exit` hook that calls `Visionary.close`).
|
|
72
73
|
|
|
74
|
+
Rows are written as multi-row INSERTs, 500 rows a batch, with up to four
|
|
75
|
+
batches in flight on PostgreSQL and MySQL (one at a time on SQLite and D1).
|
|
76
|
+
`Visionary.writer_stats` returns rows `queued`, `written` and `dropped`, for
|
|
77
|
+
load tests and health checks.
|
|
78
|
+
|
|
73
79
|
**5. Check the dashboard.** Open the Visionary dashboard. The request appears
|
|
74
80
|
on the API overview within a few seconds, with its route, status, time and
|
|
75
81
|
any error. Open it from Performance → Slowest requests to see its database and
|
|
@@ -153,7 +159,7 @@ Call it from any auth code (a `before_action`, a Warden callback, a Rack
|
|
|
153
159
|
middleware) once the user is known. Only `id` is required. To fill in handle
|
|
154
160
|
and role from the id, set `lookup_user` as in the Rails recipe. It runs on the
|
|
155
161
|
writer thread, once per distinct user per batch, only for users missing a
|
|
156
|
-
handle or role.
|
|
162
|
+
handle or role; on PostgreSQL and MySQL from up to four batches at once.
|
|
157
163
|
|
|
158
164
|
**Records:** `user_id`, `user_handle`, `user_role` on the request row and on
|
|
159
165
|
its errors.
|
|
@@ -215,6 +221,27 @@ Clients that don't use `Net::HTTP` (Faraday with another adapter, `httpx`,
|
|
|
215
221
|
`typhoeus`) are not seen. Wrap their calls in a custom span with
|
|
216
222
|
`kind: "http"`.
|
|
217
223
|
|
|
224
|
+
### Traces across services
|
|
225
|
+
|
|
226
|
+
**When:** one service calls another and you want to follow a request through
|
|
227
|
+
both. Nothing to add beyond the middleware in each service, each with its own
|
|
228
|
+
`VISIONARY_SERVICE`.
|
|
229
|
+
|
|
230
|
+
The middleware continues the trace in an incoming W3C `traceparent` header,
|
|
231
|
+
or starts one, and every `Net::HTTP` request made while it runs (or inside a
|
|
232
|
+
`span` or `background` block it started) sends `traceparent` on, naming its
|
|
233
|
+
`http` span as the parent, plus the `tracestate` that came in. A
|
|
234
|
+
`traceparent` the app set itself is left alone, the request object is left
|
|
235
|
+
as it was, and the header is sent even with `VISIONARY_SPANS=false` or on a
|
|
236
|
+
path that is not logged. When the app also runs OpenTelemetry, the request
|
|
237
|
+
joins the active OpenTelemetry span's trace.
|
|
238
|
+
|
|
239
|
+
**Records:** `trace_id`, `span_id` and `parent_span_id` on the request's
|
|
240
|
+
`api_logs` row and on every span: calls made inside `Visionary.span` or
|
|
241
|
+
`Visionary.background` are children of that span. Log lines get the trace
|
|
242
|
+
and span they were logged in. The dashboard's Traces page and service map
|
|
243
|
+
read them, and OpenTelemetry services join the same traces.
|
|
244
|
+
|
|
218
245
|
### Custom spans (Redis, queues, CPU work)
|
|
219
246
|
|
|
220
247
|
**When:** time inside a request shows as **Untracked** in its breakdown. That
|
|
@@ -262,6 +289,8 @@ It runs the block in a new thread and returns the `Thread`.
|
|
|
262
289
|
totals.
|
|
263
290
|
- Outside a request the span has no `request_id` and is stored only when it
|
|
264
291
|
takes at least `VISIONARY_SPAN_THRESHOLD_MS`.
|
|
292
|
+
- It stays in the request's trace: calls inside it are children of the
|
|
293
|
+
`background` span.
|
|
265
294
|
|
|
266
295
|
On Ruby 3.2+ any thread or fiber started during a request inherits it, so a
|
|
267
296
|
plain `Thread.new` is attributed too. `background` also carries the request
|
|
@@ -326,11 +355,39 @@ comes with its debug lines. Because of that the capturing logger reports
|
|
|
326
355
|
**Records:** an `app_logs` row per kept line: level, message (secrets such as
|
|
327
356
|
passwords, bearer tokens, JWTs and AWS keys replaced), attrs (nested keys
|
|
328
357
|
dotted, redacted fields `[REDACTED]`), the request's `request_id`, `user_id`
|
|
329
|
-
and W3C `trace_id`
|
|
358
|
+
and the W3C `trace_id` and `span_id` of the request or span it was logged
|
|
359
|
+
in, and where it was logged
|
|
330
360
|
(`code_file`, `code_line`, `code_func`). Exceptions are logged as
|
|
331
361
|
`Class: message` and the backtrace. The SDK's own `visionary: ` lines are
|
|
332
362
|
never captured. `VISIONARY_LOGS=false` turns capture off.
|
|
333
363
|
|
|
364
|
+
### Custom metrics
|
|
365
|
+
|
|
366
|
+
**When:** you want numbers of your own on the Metrics page: orders placed,
|
|
367
|
+
queue depth, checkout time.
|
|
368
|
+
|
|
369
|
+
```ruby
|
|
370
|
+
Visionary.counter("orders_placed", region: "eu").add(1) # things that happen
|
|
371
|
+
Visionary.gauge("queue_depth", queue: "emails").set(jobs.size) # a level
|
|
372
|
+
took = Visionary.histogram("checkout_ms", method: "card") # values such as durations
|
|
373
|
+
took.observe(elapsed_ms)
|
|
374
|
+
|
|
375
|
+
Visionary.describe_metric("checkout_ms", unit: "ms", buckets: [50, 100, 250, 500, 1000])
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
Labels are keywords or a Hash; each distinct set is one series. Keep the
|
|
379
|
+
handle for hot paths. `counter#add` (or `inc`) takes positive numbers;
|
|
380
|
+
`gauge#set` and `gauge#add` any finite number; `histogram#observe` any
|
|
381
|
+
finite number. Recording works before the agent starts and never waits for
|
|
382
|
+
the database.
|
|
383
|
+
|
|
384
|
+
**Records:** one `metrics` row per series every `VISIONARY_METRICS_INTERVAL`
|
|
385
|
+
(15 s): a counter's increase, a gauge's value (every interval once set), a
|
|
386
|
+
histogram's count, sum, min, max and bucket counts. The table is created the
|
|
387
|
+
first time there is something to write. A metric keeps at most
|
|
388
|
+
`VISIONARY_METRICS_MAX_SERIES` (1000) label sets; new ones past that are
|
|
389
|
+
dropped and a warning is logged, so never put user or request ids in a label.
|
|
390
|
+
|
|
334
391
|
### Error codes from JSON responses
|
|
335
392
|
|
|
336
393
|
**When:** your API answers errors with a JSON body like
|
|
@@ -496,6 +553,9 @@ options in code, plus the hooks below. Set them before the first request
|
|
|
496
553
|
| `VISIONARY_PROFILE` | `profile` | `true` | Follow the dashboard's Profiling setting; `false` never profiles |
|
|
497
554
|
| `VISIONARY_LOGS` | `logs` | `true` | Capture log lines the app routes to the SDK (see [Logs](#logs)) |
|
|
498
555
|
| `VISIONARY_LOG_LEVEL` | `log_level` | `info` | Lowest level kept until Settings → Logs has been read |
|
|
556
|
+
| `VISIONARY_METRICS` | `metrics` | `true` | Record custom metrics; `false` makes `counter`, `gauge` and `histogram` no-ops |
|
|
557
|
+
| `VISIONARY_METRICS_INTERVAL` | `metrics_interval` (seconds) | `15s` | How often each series is written |
|
|
558
|
+
| `VISIONARY_METRICS_MAX_SERIES` | `metrics_max_series` | `1000` | Label sets kept per metric name |
|
|
499
559
|
|
|
500
560
|
If `VISIONARY_RELEASE` is unset, the release is read from the first of
|
|
501
561
|
`RAILWAY_GIT_COMMIT_SHA`, `RENDER_GIT_COMMIT`, `VERCEL_GIT_COMMIT_SHA`,
|
|
@@ -518,7 +578,7 @@ Options only available in code:
|
|
|
518
578
|
| `disk_path` | working directory | Filesystem whose usage is reported |
|
|
519
579
|
| `queue_size` | `5000` | Rows queued before new ones are dropped |
|
|
520
580
|
| `log_queue_size` | `10000` | Log lines queued (on their own writer) before new ones are dropped |
|
|
521
|
-
| `batch_size` | `50` | Rows per write (
|
|
581
|
+
| `batch_size` | `500` on SQL, `50` on D1 | Rows per write (up to 2000; D1 at most 50) |
|
|
522
582
|
| `flush_interval` | `3.0` | Seconds between writes when the batch isn't full |
|
|
523
583
|
|
|
524
584
|
Always redacted: the headers `Authorization`, `Proxy-Authorization`,
|
|
@@ -559,8 +619,8 @@ Visionary SDK follows the same contract,
|
|
|
559
619
|
### What is written
|
|
560
620
|
|
|
561
621
|
- `api_logs`: one row per request: method, route pattern, status, timing,
|
|
562
|
-
user, redacted headers and bodies, error codes,
|
|
563
|
-
`http_count`, `http_ms`.
|
|
622
|
+
user, redacted headers and bodies, error codes, `db_count`, `db_ms`,
|
|
623
|
+
`http_count`, `http_ms`, and `trace_id`, `span_id`, `parent_span_id`.
|
|
564
624
|
- `spans`: database, outbound HTTP, custom and background spans. Rows are
|
|
565
625
|
stored for every request, at most `VISIONARY_SPAN_MAX` per request. Spans
|
|
566
626
|
outside requests are stored when they take at least
|
|
@@ -571,6 +631,8 @@ Visionary SDK follows the same contract,
|
|
|
571
631
|
- `profiles`: CPU profiles, only while they are on in the dashboard.
|
|
572
632
|
- `app_logs`: the log lines the app routes to the SDK, written by a writer
|
|
573
633
|
thread of their own so a flood of lines never delays request rows.
|
|
634
|
+
- `metrics`: custom counters, gauges and histograms, one row per series per
|
|
635
|
+
`VISIONARY_METRICS_INTERVAL`, on a writer thread of their own.
|
|
574
636
|
|
|
575
637
|
Request bodies are captured as the app reads them. Multipart, image, video,
|
|
576
638
|
audio, octet-stream, zip and PDF bodies are not captured.
|
|
@@ -647,6 +709,10 @@ Visionary.set_route("/users/:id") # route pattern, for routers Vi
|
|
|
647
709
|
Visionary.login_attempt(params[:email]) # the account a sign-in request tried
|
|
648
710
|
Visionary.span(name, kind: "custom", target: nil, detail: nil) { ... } # a manual span
|
|
649
711
|
Visionary.background(name) { ... } # a thread that stays with the request
|
|
712
|
+
Visionary.counter("orders", region: "eu").add(1) # custom metrics: counter, gauge, histogram
|
|
713
|
+
Visionary.gauge("queue_depth").set(n)
|
|
714
|
+
Visionary.histogram("checkout_ms").observe(ms)
|
|
715
|
+
Visionary.describe_metric(name, unit: "ms", buckets: [...]) # unit and histogram buckets
|
|
650
716
|
Visionary.capture_exception(e) # a handled error (handled = 1)
|
|
651
717
|
Visionary.logger(inner = nil) # a Logger that also writes app_logs
|
|
652
718
|
Visionary.capture_rails_logger # copy Rails.logger to app_logs
|
data/lib/visionary/agent.rb
CHANGED
|
@@ -47,6 +47,7 @@ module Visionary
|
|
|
47
47
|
@profiler = Profiler.new(cfg, @instance, @store, @writer, @logger) if cfg.profile && @writer
|
|
48
48
|
@telemetry = Telemetry::Recorder.new(cfg, @instance, @writer) if cfg.spans || cfg.errors
|
|
49
49
|
@logs = Logs::Pipeline.new(cfg, @instance, @store, @schemas, @logger) if cfg.logs
|
|
50
|
+
@metrics = Metrics::Recorder.new(cfg, @instance, @store, @schemas, @logger) if cfg.metrics
|
|
50
51
|
if cfg.system_metrics && cfg.system_interval.to_f.positive?
|
|
51
52
|
@system = System::Recorder.new(cfg, @instance, @store, @schemas, @http, @logger)
|
|
52
53
|
end
|
|
@@ -59,10 +60,11 @@ module Visionary
|
|
|
59
60
|
tables["system_metrics"] = cfg.system_retention_days if @system
|
|
60
61
|
tables["profiles"] = cfg.log_retention_days if @profiler
|
|
61
62
|
tables["app_logs"] = cfg.log_retention_days if @logs
|
|
63
|
+
tables["metrics"] = cfg.log_retention_days if @metrics
|
|
62
64
|
@retention = Retention.new(@store, @logger, tables)
|
|
63
65
|
end
|
|
64
66
|
|
|
65
|
-
def enabled? = !@store.nil? && !@closed && (!@writer.nil? || !@system.nil? || !@logs.nil?)
|
|
67
|
+
def enabled? = !@store.nil? && !@closed && (!@writer.nil? || !@system.nil? || !@logs.nil? || !@metrics.nil?)
|
|
66
68
|
|
|
67
69
|
def log_path?(path) = @config.log_requests && !@writer.nil? && path.start_with?(@config.path_prefix.to_s)
|
|
68
70
|
|
|
@@ -71,6 +73,11 @@ module Visionary
|
|
|
71
73
|
@writer&.enqueue(item) unless @closed
|
|
72
74
|
end
|
|
73
75
|
|
|
76
|
+
def writer_stats
|
|
77
|
+
zero = WriterStats.new(queued: 0, written: 0, dropped: 0)
|
|
78
|
+
[@writer, @logs&.writer, @metrics&.writer].compact.sum(zero, &:stats)
|
|
79
|
+
end
|
|
80
|
+
|
|
74
81
|
# The route pattern for a finished request: the app's own resolver,
|
|
75
82
|
# then what the framework left in the env.
|
|
76
83
|
def route_for(env)
|
|
@@ -97,6 +104,7 @@ module Visionary
|
|
|
97
104
|
@logger.info("visionary: recording system metrics every #{@config.system_interval}s as #{@instance}")
|
|
98
105
|
end
|
|
99
106
|
@logs&.start
|
|
107
|
+
@metrics&.start
|
|
100
108
|
@retention.start
|
|
101
109
|
@profiler&.start
|
|
102
110
|
install_hooks
|
|
@@ -116,6 +124,7 @@ module Visionary
|
|
|
116
124
|
@profiler&.close
|
|
117
125
|
idle = @writer ? @writer.close(timeout) : true
|
|
118
126
|
idle = @logs.close([deadline - mono, 0].max) && idle if @logs
|
|
127
|
+
idle = @metrics.close([deadline - mono, 0].max) && idle if @metrics
|
|
119
128
|
idle = @system.close([deadline - mono, 0].max) && idle if @system
|
|
120
129
|
@store.close if idle && @store
|
|
121
130
|
rescue StandardError => e
|
|
@@ -135,6 +144,7 @@ module Visionary
|
|
|
135
144
|
@system&.after_fork
|
|
136
145
|
@writer&.after_fork
|
|
137
146
|
@logs&.after_fork
|
|
147
|
+
@metrics&.after_fork
|
|
138
148
|
@retention&.after_fork if @retention&.needed?
|
|
139
149
|
@profiler&.after_fork
|
|
140
150
|
@logger.info("visionary: restarted in forked process #{@pid}")
|
|
@@ -153,9 +163,10 @@ module Visionary
|
|
|
153
163
|
nil
|
|
154
164
|
end
|
|
155
165
|
|
|
166
|
+
# Net::HTTP carries the trace on even when spans are off
|
|
156
167
|
def install_hooks
|
|
157
|
-
return unless @telemetry&.spans_on
|
|
158
168
|
Telemetry::NetHTTPHook.install
|
|
169
|
+
return unless @telemetry&.spans_on
|
|
159
170
|
Telemetry::ActiveRecordHook.install if defined?(::ActiveRecord::Base) && defined?(::ActiveSupport::Notifications)
|
|
160
171
|
rescue StandardError => e
|
|
161
172
|
@logger.warn("visionary: span hooks not installed: #{e.message}")
|
data/lib/visionary/config.rb
CHANGED
|
@@ -22,7 +22,8 @@ module Visionary
|
|
|
22
22
|
:error_code_field, :error_key_field,
|
|
23
23
|
:system_metrics, :system_interval, :system_retention_days, :disk_path,
|
|
24
24
|
:service, :version, :spans, :span_threshold_ms, :span_max, :errors,
|
|
25
|
-
:profile, :logs, :log_level, :log_queue_size
|
|
25
|
+
:profile, :logs, :log_level, :log_queue_size,
|
|
26
|
+
:metrics, :metrics_interval, :metrics_max_series
|
|
26
27
|
|
|
27
28
|
# Runs on the writer thread, once per distinct user per batch, for users
|
|
28
29
|
# missing a handle or role: ->(id) { [handle, role] } or nil.
|
|
@@ -46,7 +47,7 @@ module Visionary
|
|
|
46
47
|
@capture_bodies = true
|
|
47
48
|
@max_body_bytes = 8192
|
|
48
49
|
@queue_size = 5000
|
|
49
|
-
@batch_size =
|
|
50
|
+
@batch_size = 0
|
|
50
51
|
@flush_interval = 3.0
|
|
51
52
|
@log_retention_days = 0 # 0: off, the Visionary server applies Settings → Retention
|
|
52
53
|
@redact_headers = []
|
|
@@ -71,6 +72,9 @@ module Visionary
|
|
|
71
72
|
# The lowest level kept until Settings → Logs has been read.
|
|
72
73
|
@log_level = "info"
|
|
73
74
|
@log_queue_size = 10_000
|
|
75
|
+
@metrics = true
|
|
76
|
+
@metrics_interval = 15.0
|
|
77
|
+
@metrics_max_series = 1000
|
|
74
78
|
end
|
|
75
79
|
|
|
76
80
|
def self.from_env(env = ENV)
|
|
@@ -104,6 +108,9 @@ module Visionary
|
|
|
104
108
|
c.profile = parse_bool(s.("VISIONARY_PROFILE"), true)
|
|
105
109
|
c.logs = parse_bool(s.("VISIONARY_LOGS"), true)
|
|
106
110
|
c.log_level = s.("VISIONARY_LOG_LEVEL").then { |v| Logs.parse_level(v).positive? ? v.downcase : "info" }
|
|
111
|
+
c.metrics = parse_bool(s.("VISIONARY_METRICS"), true)
|
|
112
|
+
c.metrics_interval = parse_duration(s.("VISIONARY_METRICS_INTERVAL"), 15.0)
|
|
113
|
+
c.metrics_max_series = parse_int(s.("VISIONARY_METRICS_MAX_SERIES"), 1000)
|
|
107
114
|
c
|
|
108
115
|
end
|
|
109
116
|
|
data/lib/visionary/context.rb
CHANGED
|
@@ -12,11 +12,15 @@ module Visionary
|
|
|
12
12
|
class RequestContext
|
|
13
13
|
KEY = :visionary_request
|
|
14
14
|
|
|
15
|
-
attr_reader :request_id, :method, :path, :started_at, :started_mono, :errors, :db_visible, :trace_id
|
|
15
|
+
attr_reader :request_id, :method, :path, :started_at, :started_mono, :errors, :db_visible, :trace_id,
|
|
16
|
+
:span_id, :parent_span_id
|
|
16
17
|
|
|
17
|
-
def initialize(request_id:, method:, path:, started_at:, started_mono:, span_max:, db_visible:, trace_id: nil
|
|
18
|
+
def initialize(request_id:, method:, path:, started_at:, started_mono:, span_max:, db_visible:, trace_id: nil,
|
|
19
|
+
span: nil, parent_span_id: nil)
|
|
18
20
|
@request_id = request_id
|
|
19
|
-
@trace_id = trace_id
|
|
21
|
+
@trace_id = span ? span.trace_id : trace_id
|
|
22
|
+
@span_id = span&.span_id
|
|
23
|
+
@parent_span_id = parent_span_id
|
|
20
24
|
@method = method
|
|
21
25
|
@path = path
|
|
22
26
|
@started_at = started_at
|
|
@@ -148,5 +152,6 @@ module Visionary
|
|
|
148
152
|
|
|
149
153
|
# One database, HTTP, cache, queue, custom, graphql or background call.
|
|
150
154
|
Span = Struct.new(:kind, :name, :target, :detail, :started_at, :offset_ms, :duration_ms,
|
|
151
|
-
:status, :error, :row_count, :request_id,
|
|
155
|
+
:status, :error, :row_count, :request_id, :trace_id, :span_id, :parent_span_id,
|
|
156
|
+
keyword_init: true)
|
|
152
157
|
end
|
|
@@ -195,11 +195,10 @@ module Visionary
|
|
|
195
195
|
!local.nil? && n < local
|
|
196
196
|
end
|
|
197
197
|
|
|
198
|
-
# Falls back to the active OpenTelemetry span
|
|
198
|
+
# Falls back to the active OpenTelemetry span outside any Visionary span
|
|
199
199
|
def otel(rec)
|
|
200
200
|
return unless defined?(::OpenTelemetry::Trace)
|
|
201
|
-
|
|
202
|
-
return if ctx&.trace_id
|
|
201
|
+
return if TraceContext.current
|
|
203
202
|
sc = ::OpenTelemetry::Trace.current_span.context
|
|
204
203
|
return unless sc.valid?
|
|
205
204
|
rec.trace_id = sc.hex_trace_id
|
|
@@ -75,12 +75,15 @@ module Visionary
|
|
|
75
75
|
return if !s.enabled || rec.message.start_with?("visionary: ")
|
|
76
76
|
return if s.dropped?(@service, rec.logger, rec.message, rec.sev)
|
|
77
77
|
|
|
78
|
+
if rec.trace_id.nil? && (sc = TraceContext.current)
|
|
79
|
+
rec.trace_id = sc.trace_id
|
|
80
|
+
rec.span_id ||= sc.span_id
|
|
81
|
+
end
|
|
78
82
|
unless ctx
|
|
79
83
|
emit(rec) if rec.sev >= s.floor(@service, "", "", rec.at)
|
|
80
84
|
return
|
|
81
85
|
end
|
|
82
86
|
rec.request_id ||= ctx.request_id
|
|
83
|
-
rec.trace_id ||= ctx.trace_id
|
|
84
87
|
rec.user_id ||= blank(ctx.user[0])
|
|
85
88
|
return emit(rec) if rec.sev >= s.hold_below(@service, rec.at)
|
|
86
89
|
return if ctx.hold_log(rec, MAX_HELD)
|
|
@@ -186,7 +189,7 @@ module Visionary
|
|
|
186
189
|
end
|
|
187
190
|
|
|
188
191
|
def statements(_kind, rec, _users)
|
|
189
|
-
[
|
|
192
|
+
[fit(:log, @pipeline.row(rec))]
|
|
190
193
|
end
|
|
191
194
|
end
|
|
192
195
|
end
|