visionary-sdk 1.1.1 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 297ea0df8d34fbccf535d4b427c10d6b7179d9589539c47c019a5a1904550464
4
- data.tar.gz: 10ee4085748676113fb78a9f275818f2a367487731803ecea13bcd9b94eb7956
3
+ metadata.gz: '06876950e4e7e007c2e59fe232268e03b981383062f47c4a1ef2e7c8fe803df2'
4
+ data.tar.gz: bb09cbeffec3774e6bec59a5114ecdc5aaa80ebef88c77ff11c3c9f674445dcb
5
5
  SHA512:
6
- metadata.gz: 2a44b0e16061e33a041dbba94d9ab599db87934bfbb4214e3cf2b3140958f78cf34bb41e902b58a90723d64e3dc1ff26f0fefd5256c47d8612c48dbd71e42061
7
- data.tar.gz: a57381c08c1b70261bae0348955725df2594017d8fb3ce8c8f815f55869f0053c0c1350a606737ef85e08105e9d7afc3c3ceaab4830326941d1ea038e68b324f
6
+ metadata.gz: c220abde41d7f0f77a15385ee8f4ecab2fabea986ff25c9e8a64e61880d65f7370b97561f3983137a8ff77c1165825054ac9141469352ad71c890c120f3bd3e2
7
+ data.tar.gz: '0091d167f78f3b35379febe0adcfbfdc8da41ea7c4ca561f62545fad9d1a5dcf49426b7d0faff9691336f7daeeb4d58df6007bf408aee18bfa731ddb16107cf6'
data/CHANGELOG.md CHANGED
@@ -2,6 +2,23 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ - Faster writes: SQL backends get each table's rows as multi-row INSERTs
6
+ (500 rows a batch, `batch_size` up to 2000), with up to four batches in
7
+ flight on PostgreSQL and MySQL. SQLite opens in WAL mode with a 5 s busy
8
+ timeout. D1 is unchanged. `Visionary.writer_stats` reports rows queued,
9
+ written and dropped.
10
+
11
+ - Custom metrics in `metrics`: `Visionary.counter(name, **labels).add(n)`,
12
+ `Visionary.gauge(...).set(v)`, `Visionary.histogram(...).observe(v)` and
13
+ `Visionary.describe_metric(name, unit:, buckets:)`. Added up in memory and
14
+ written one row per series every interval by a writer thread of their own;
15
+ at most 1000 label sets per name. `VISIONARY_METRICS`,
16
+ `VISIONARY_METRICS_INTERVAL`, `VISIONARY_METRICS_MAX_SERIES`.
17
+ - Distributed tracing: requests continue an incoming W3C `traceparent` or
18
+ start a trace, `Net::HTTP` sends `traceparent` (and `tracestate`) on, and
19
+ `api_logs` and `spans` get `trace_id`, `span_id` and `parent_span_id`
20
+ (added to existing tables at startup). Calls inside `Visionary.span` and
21
+ `Visionary.background` nest under them; log lines carry the current span.
5
22
  - Application logs in `app_logs`: `Visionary.logger(inner = nil)` (a Logger
6
23
  that forwards every call to `inner` unchanged and also captures) and
7
24
  `Visionary.capture_rails_logger` (broadcasts `Rails.logger`). Lines link to
data/LICENSE CHANGED
@@ -1,94 +1,108 @@
1
- Visionary Non-Commercial License (VNCL) v1.0
1
+ Visionary Source-Available License (VSAL) v2.0
2
2
 
3
3
  Copyright (c) 2026 Saurav Sajeev. All rights reserved, except as expressly
4
4
  permitted below.
5
5
 
6
+ In short: you may use, run, copy and modify Visionary for any purpose,
7
+ including in a business and in production, to monitor software of your own,
8
+ even software that makes money. You may not sell Visionary, charge for access
9
+ to it, or offer it to others as a product or service. This summary is for
10
+ convenience; the terms below are what apply.
11
+
6
12
 
7
13
  1. Definitions
8
14
 
9
15
  "Software" means Visionary, including its source code, object code, server,
10
- host agent, SDKs, libraries, documentation, and associated materials
11
- distributed under this License.
16
+ host agent, SDKs, libraries, command-line tools, documentation, and associated
17
+ materials distributed under this License.
12
18
 
13
19
  "Author" means Saurav Sajeev, the original author and copyright holder of the
14
20
  Software, to the extent of their ownership of the relevant material.
15
21
 
16
- "Commercial Use" means using the Software, in whole or in part, to sell,
17
- license, sublicense, rent, lease, monetize, commercially distribute, or provide
18
- a paid product or service. It also includes offering the Software as a hosted
19
- service to third parties for compensation, or incorporating it into a product
20
- or service distributed commercially.
22
+ "SDKs" means the client libraries distributed as part of the Software for
23
+ embedding in other software, including the server, browser, mobile and test
24
+ reporter libraries.
25
+
26
+ "Your Applications" means software, websites, mobile apps, services and
27
+ infrastructure that you or your organization build, own or operate, other than
28
+ the Software itself.
29
+
30
+ "Sell or Monetize" means any of the following, whether or not the Software is
31
+ modified:
32
+
33
+ a. Selling, licensing, sublicensing, renting or leasing the Software or
34
+ copies of it.
35
+
36
+ b. Charging any fee, subscription or other consideration for access to the
37
+ Software or to its functionality.
38
+
39
+ c. Offering the Software to third parties as a hosted, managed,
40
+ white-labelled or software-as-a-service product, whether paid or offered
41
+ to win or keep paying customers.
42
+
43
+ d. Including the Software, or a substantial portion of it, in a product or
44
+ service that you sell or license to third parties, where the Software's
45
+ functionality is a substantial part of what the customer receives.
21
46
 
22
47
  "Redistribution" means making the Software, or a modified or derivative version
23
48
  of it, available to another person or entity by distribution, publication,
24
49
  hosting for download, sublicensing, or other transfer.
25
50
 
26
- "Internal Use" means use by an individual or an organization solely for its
27
- own internal operations, without providing the Software or its functionality
28
- as a product or service to external customers or third parties.
29
-
30
51
 
31
52
  2. Grant of Permission
32
53
 
33
- Subject to the terms of this License, the Author grants you a limited,
34
- worldwide, non-exclusive, royalty-free, revocable only as permitted by
35
- applicable law, and non-transferable license to:
54
+ Subject to the terms of this License, the Author grants you a worldwide,
55
+ non-exclusive, royalty-free, non-transferable license to:
36
56
 
37
57
  a. Access, view, and study the Software's source code.
38
58
 
39
- b. Download and run the Software for personal, non-commercial purposes.
59
+ b. Download, install, run and use the Software for any purpose, personal or
60
+ commercial, in development, testing, staging and production, within an
61
+ organization of any size.
62
+
63
+ c. Use the Software to monitor, observe, analyse, test and alert on Your
64
+ Applications, including applications you sell, applications that earn
65
+ revenue, and applications you build or operate for your clients.
40
66
 
41
- c. Install and deploy the Software in development, testing, staging, and
42
- production environments for Internal Use.
67
+ d. Modify the Software and use your modified version under the same terms.
43
68
 
44
- d. Modify the Software for personal use or Internal Use, including within
45
- commercial organizations, provided such modifications are not
46
- redistributed or commercially exploited.
69
+ e. Make copies needed for the uses above, including backups.
47
70
 
48
- e. Make reasonable backup copies for permitted use.
71
+ f. Have contractors, consultants and service providers exercise these rights
72
+ on your behalf and for your benefit, provided they comply with this
73
+ License.
49
74
 
50
75
  These permissions do not transfer ownership of the Software or grant any rights
51
76
  beyond those expressly stated in this License.
52
77
 
53
78
 
54
- 3. Non-Commercial Restriction
79
+ 3. SDKs Inside Your Applications
55
80
 
56
- You must not, without the Author's prior written permission:
57
-
58
- a. Sell, license, sublicense, rent, lease, or otherwise commercially exploit
59
- the Software.
60
-
61
- b. Offer the Software or a modified version as a paid product, subscription,
62
- managed service, or hosted monitoring service.
81
+ You may include the SDKs, modified or not, in Your Applications and distribute
82
+ them as part of Your Applications, for example in a website's scripts, a mobile
83
+ app, a server application or a container image, provided that:
63
84
 
64
- c. Incorporate the Software or a substantial portion of it into a product or
65
- service that is sold, licensed, distributed, or otherwise commercially
66
- offered to third parties.
85
+ a. the SDKs are distributed only as part of Your Applications, not as a
86
+ standalone library, package or product; and
67
87
 
68
- d. Use the Software to provide monitoring, observability, analytics, or other
69
- functionality to external customers as a commercial service.
88
+ b. the copyright notice and this License (or a link to it) are kept with
89
+ the SDK code where your build process keeps such notices.
70
90
 
71
- e. Monetize access to the Software or charge third parties for access to its
72
- functionality.
91
+ Installing the SDKs from the package registries where the Author publishes them
92
+ is permitted.
73
93
 
74
- Internal production use by a commercial organization is permitted under
75
- Section 2, provided that the organization does not commercially exploit or
76
- redistribute the Software in a manner prohibited by this License.
77
94
 
95
+ 4. Restrictions
78
96
 
79
- 4. Redistribution and Publication
80
-
81
- Without the Author's prior written permission, you must not:
97
+ You must not, without the Author's prior written permission:
82
98
 
83
- a. Redistribute or republish the original Software, in source or binary
84
- form.
99
+ a. Sell or Monetize the Software.
85
100
 
86
- b. Publish, distribute, or make publicly available a modified version, fork,
87
- or derivative work of the Software.
101
+ b. Redistribute the Software, a modified version, fork or derivative work,
102
+ in source or binary form, except as Section 3 allows.
88
103
 
89
- c. Upload the Software or modified versions to public package registries,
90
- software marketplaces, application stores, repositories, or download
91
- platforms for redistribution.
104
+ c. Publish the Software or modified versions to public package registries,
105
+ marketplaces, application stores or download platforms.
92
106
 
93
107
  d. Sublicense, transfer, or grant others rights to use the Software beyond
94
108
  the permissions expressly provided in this License.
@@ -96,22 +110,24 @@ Without the Author's prior written permission, you must not:
96
110
  e. Remove or misrepresent the Software's authorship, copyright notices, or
97
111
  license information.
98
112
 
99
- Private modifications and internal deployments permitted by Section 2 are not
100
- prohibited merely because they exist within a commercial organization's
101
- infrastructure.
113
+ f. Use the Author's name, the Visionary name or its logos to imply
114
+ endorsement of, or to market, your own products or services.
102
115
 
116
+ For clarity, the following are permitted and are not Selling or Monetizing:
117
+ running the Software inside a business; using it for applications that make
118
+ money; charging your clients for work on their own applications while you use
119
+ the Software to monitor them; and installing or configuring the Software on a
120
+ client's own infrastructure for that client's own use, provided you do not
121
+ charge for the Software itself or for access to it.
103
122
 
104
- 5. Commercial Licensing
105
-
106
- The Author may separately grant commercial licenses, redistribution rights, or
107
- other permissions under different terms.
108
123
 
109
- Anyone wishing to sell, redistribute, sublicense, incorporate, or commercially
110
- exploit the Software beyond the permissions granted here must obtain prior
111
- written authorization from the Author.
124
+ 5. Commercial Licensing
112
125
 
113
- A separate commercial agreement may specify fees, attribution requirements,
114
- support terms, distribution rights, and other conditions.
126
+ The Author may separately grant licenses to Sell or Monetize, redistribute,
127
+ sublicense or otherwise use the Software beyond the permissions granted here.
128
+ Such a license requires prior written authorization from the Author and may set
129
+ fees, attribution requirements, support terms, distribution rights and other
130
+ conditions.
115
131
 
116
132
 
117
133
  6. Ownership and Intellectual Property
@@ -124,8 +140,8 @@ identify the unmodified Software truthfully.
124
140
 
125
141
  You retain ownership of your independently created contributions to the extent
126
142
  provided by applicable law, but this does not grant you permission to
127
- distribute or commercially exploit the Software or derivative works in
128
- violation of this License.
143
+ redistribute, Sell or Monetize the Software or derivative works in violation
144
+ of this License.
129
145
 
130
146
 
131
147
  7. Third-Party Components
@@ -147,8 +163,7 @@ Your permissions under this License terminate automatically if you materially
147
163
  violate its terms.
148
164
 
149
165
  Upon termination, you must cease activities that are no longer authorized by
150
- this License, including prohibited redistribution or commercial exploitation,
151
- subject to any rights or remedies provided by applicable law.
166
+ this License, subject to any rights or remedies provided by applicable law.
152
167
 
153
168
  Permissions may be reinstated by the Author in writing.
154
169
 
@@ -195,14 +210,17 @@ determined by a separate written agreement between the relevant parties or, in
195
210
  the absence of such an agreement, by applicable law and conflict-of-law rules.
196
211
 
197
212
 
198
- 13. Entire Agreement
213
+ 13. Entire Agreement and Versions
199
214
 
200
215
  This License governs the permissions granted by the Author for the Software
201
216
  covered by it. If any provision is held unenforceable, the remaining provisions
202
217
  shall continue to apply to the extent permitted by law.
203
218
 
219
+ This version applies to copies of the Software that carry it. It replaces the
220
+ Visionary Non-Commercial License (VNCL) v1.0 for those copies.
221
+
204
222
 
205
- License: Visionary Non-Commercial License (VNCL) v1.0
223
+ License: Visionary Source-Available License (VSAL) v2.0
206
224
  Copyright holder: Saurav Sajeev
207
225
  Project: Visionary
208
- Commercial licensing: Requires separate written authorization.
226
+ Selling or monetizing: Requires separate written authorization.
data/README.md CHANGED
@@ -3,8 +3,9 @@
3
3
  Add one gem to a Rails or Rack app and every request shows up in the
4
4
  [Visionary](../../Readme.md) dashboard: route, status, time, user, redacted
5
5
  bodies, the database and HTTP calls it made, and any exception. You also get
6
- the app's own log lines linked to their request, host and process metrics,
7
- and CPU profiles when you turn them on in the dashboard.
6
+ the app's own log lines linked to their request, traces that follow a request
7
+ across services, your own counters, gauges and histograms, host and process
8
+ metrics, and CPU profiles when you turn them on in the dashboard.
8
9
 
9
10
  - [Quick start](#quick-start)
10
11
  - [Integration recipes](#integration-recipes)
@@ -70,6 +71,11 @@ visionary: recording system metrics every 15.0s as web-1
70
71
  Tables are created on first use. At exit the SDK flushes what is queued (it
71
72
  registers an `at_exit` hook that calls `Visionary.close`).
72
73
 
74
+ Rows are written as multi-row INSERTs, 500 rows a batch, with up to four
75
+ batches in flight on PostgreSQL and MySQL (one at a time on SQLite and D1).
76
+ `Visionary.writer_stats` returns rows `queued`, `written` and `dropped`, for
77
+ load tests and health checks.
78
+
73
79
  **5. Check the dashboard.** Open the Visionary dashboard. The request appears
74
80
  on the API overview within a few seconds, with its route, status, time and
75
81
  any error. Open it from Performance → Slowest requests to see its database and
@@ -153,7 +159,7 @@ Call it from any auth code (a `before_action`, a Warden callback, a Rack
153
159
  middleware) once the user is known. Only `id` is required. To fill in handle
154
160
  and role from the id, set `lookup_user` as in the Rails recipe. It runs on the
155
161
  writer thread, once per distinct user per batch, only for users missing a
156
- handle or role.
162
+ handle or role; on PostgreSQL and MySQL from up to four batches at once.
157
163
 
158
164
  **Records:** `user_id`, `user_handle`, `user_role` on the request row and on
159
165
  its errors.
@@ -215,6 +221,27 @@ Clients that don't use `Net::HTTP` (Faraday with another adapter, `httpx`,
215
221
  `typhoeus`) are not seen. Wrap their calls in a custom span with
216
222
  `kind: "http"`.
217
223
 
224
+ ### Traces across services
225
+
226
+ **When:** one service calls another and you want to follow a request through
227
+ both. Nothing to add beyond the middleware in each service, each with its own
228
+ `VISIONARY_SERVICE`.
229
+
230
+ The middleware continues the trace in an incoming W3C `traceparent` header,
231
+ or starts one, and every `Net::HTTP` request made while it runs (or inside a
232
+ `span` or `background` block it started) sends `traceparent` on, naming its
233
+ `http` span as the parent, plus the `tracestate` that came in. A
234
+ `traceparent` the app set itself is left alone, the request object is left
235
+ as it was, and the header is sent even with `VISIONARY_SPANS=false` or on a
236
+ path that is not logged. When the app also runs OpenTelemetry, the request
237
+ joins the active OpenTelemetry span's trace.
238
+
239
+ **Records:** `trace_id`, `span_id` and `parent_span_id` on the request's
240
+ `api_logs` row and on every span: calls made inside `Visionary.span` or
241
+ `Visionary.background` are children of that span. Log lines get the trace
242
+ and span they were logged in. The dashboard's Traces page and service map
243
+ read them, and OpenTelemetry services join the same traces.
244
+
218
245
  ### Custom spans (Redis, queues, CPU work)
219
246
 
220
247
  **When:** time inside a request shows as **Untracked** in its breakdown. That
@@ -262,6 +289,8 @@ It runs the block in a new thread and returns the `Thread`.
262
289
  totals.
263
290
  - Outside a request the span has no `request_id` and is stored only when it
264
291
  takes at least `VISIONARY_SPAN_THRESHOLD_MS`.
292
+ - It stays in the request's trace: calls inside it are children of the
293
+ `background` span.
265
294
 
266
295
  On Ruby 3.2+ any thread or fiber started during a request inherits it, so a
267
296
  plain `Thread.new` is attributed too. `background` also carries the request
@@ -326,11 +355,39 @@ comes with its debug lines. Because of that the capturing logger reports
326
355
  **Records:** an `app_logs` row per kept line: level, message (secrets such as
327
356
  passwords, bearer tokens, JWTs and AWS keys replaced), attrs (nested keys
328
357
  dotted, redacted fields `[REDACTED]`), the request's `request_id`, `user_id`
329
- and W3C `trace_id` (from `traceparent`), and where it was logged
358
+ and the W3C `trace_id` and `span_id` of the request or span it was logged
359
+ in, and where it was logged
330
360
  (`code_file`, `code_line`, `code_func`). Exceptions are logged as
331
361
  `Class: message` and the backtrace. The SDK's own `visionary: ` lines are
332
362
  never captured. `VISIONARY_LOGS=false` turns capture off.
333
363
 
364
+ ### Custom metrics
365
+
366
+ **When:** you want numbers of your own on the Metrics page: orders placed,
367
+ queue depth, checkout time.
368
+
369
+ ```ruby
370
+ Visionary.counter("orders_placed", region: "eu").add(1) # things that happen
371
+ Visionary.gauge("queue_depth", queue: "emails").set(jobs.size) # a level
372
+ took = Visionary.histogram("checkout_ms", method: "card") # values such as durations
373
+ took.observe(elapsed_ms)
374
+
375
+ Visionary.describe_metric("checkout_ms", unit: "ms", buckets: [50, 100, 250, 500, 1000])
376
+ ```
377
+
378
+ Labels are keywords or a Hash; each distinct set is one series. Keep the
379
+ handle for hot paths. `counter#add` (or `inc`) takes positive numbers;
380
+ `gauge#set` and `gauge#add` any finite number; `histogram#observe` any
381
+ finite number. Recording works before the agent starts and never waits for
382
+ the database.
383
+
384
+ **Records:** one `metrics` row per series every `VISIONARY_METRICS_INTERVAL`
385
+ (15 s): a counter's increase, a gauge's value (every interval once set), a
386
+ histogram's count, sum, min, max and bucket counts. The table is created the
387
+ first time there is something to write. A metric keeps at most
388
+ `VISIONARY_METRICS_MAX_SERIES` (1000) label sets; new ones past that are
389
+ dropped and a warning is logged, so never put user or request ids in a label.
390
+
334
391
  ### Error codes from JSON responses
335
392
 
336
393
  **When:** your API answers errors with a JSON body like
@@ -496,6 +553,9 @@ options in code, plus the hooks below. Set them before the first request
496
553
  | `VISIONARY_PROFILE` | `profile` | `true` | Follow the dashboard's Profiling setting; `false` never profiles |
497
554
  | `VISIONARY_LOGS` | `logs` | `true` | Capture log lines the app routes to the SDK (see [Logs](#logs)) |
498
555
  | `VISIONARY_LOG_LEVEL` | `log_level` | `info` | Lowest level kept until Settings → Logs has been read |
556
+ | `VISIONARY_METRICS` | `metrics` | `true` | Record custom metrics; `false` makes `counter`, `gauge` and `histogram` no-ops |
557
+ | `VISIONARY_METRICS_INTERVAL` | `metrics_interval` (seconds) | `15s` | How often each series is written |
558
+ | `VISIONARY_METRICS_MAX_SERIES` | `metrics_max_series` | `1000` | Label sets kept per metric name |
499
559
 
500
560
  If `VISIONARY_RELEASE` is unset, the release is read from the first of
501
561
  `RAILWAY_GIT_COMMIT_SHA`, `RENDER_GIT_COMMIT`, `VERCEL_GIT_COMMIT_SHA`,
@@ -518,7 +578,7 @@ Options only available in code:
518
578
  | `disk_path` | working directory | Filesystem whose usage is reported |
519
579
  | `queue_size` | `5000` | Rows queued before new ones are dropped |
520
580
  | `log_queue_size` | `10000` | Log lines queued (on their own writer) before new ones are dropped |
521
- | `batch_size` | `50` | Rows per write (1 to 50) |
581
+ | `batch_size` | `500` on SQL, `50` on D1 | Rows per write (up to 2000; D1 at most 50) |
522
582
  | `flush_interval` | `3.0` | Seconds between writes when the batch isn't full |
523
583
 
524
584
  Always redacted: the headers `Authorization`, `Proxy-Authorization`,
@@ -559,8 +619,8 @@ Visionary SDK follows the same contract,
559
619
  ### What is written
560
620
 
561
621
  - `api_logs`: one row per request: method, route pattern, status, timing,
562
- user, redacted headers and bodies, error codes, and `db_count`, `db_ms`,
563
- `http_count`, `http_ms`.
622
+ user, redacted headers and bodies, error codes, `db_count`, `db_ms`,
623
+ `http_count`, `http_ms`, and `trace_id`, `span_id`, `parent_span_id`.
564
624
  - `spans`: database, outbound HTTP, custom and background spans. Rows are
565
625
  stored for every request, at most `VISIONARY_SPAN_MAX` per request. Spans
566
626
  outside requests are stored when they take at least
@@ -571,6 +631,8 @@ Visionary SDK follows the same contract,
571
631
  - `profiles`: CPU profiles, only while they are on in the dashboard.
572
632
  - `app_logs`: the log lines the app routes to the SDK, written by a writer
573
633
  thread of their own so a flood of lines never delays request rows.
634
+ - `metrics`: custom counters, gauges and histograms, one row per series per
635
+ `VISIONARY_METRICS_INTERVAL`, on a writer thread of their own.
574
636
 
575
637
  Request bodies are captured as the app reads them. Multipart, image, video,
576
638
  audio, octet-stream, zip and PDF bodies are not captured.
@@ -647,6 +709,10 @@ Visionary.set_route("/users/:id") # route pattern, for routers Vi
647
709
  Visionary.login_attempt(params[:email]) # the account a sign-in request tried
648
710
  Visionary.span(name, kind: "custom", target: nil, detail: nil) { ... } # a manual span
649
711
  Visionary.background(name) { ... } # a thread that stays with the request
712
+ Visionary.counter("orders", region: "eu").add(1) # custom metrics: counter, gauge, histogram
713
+ Visionary.gauge("queue_depth").set(n)
714
+ Visionary.histogram("checkout_ms").observe(ms)
715
+ Visionary.describe_metric(name, unit: "ms", buckets: [...]) # unit and histogram buckets
650
716
  Visionary.capture_exception(e) # a handled error (handled = 1)
651
717
  Visionary.logger(inner = nil) # a Logger that also writes app_logs
652
718
  Visionary.capture_rails_logger # copy Rails.logger to app_logs
@@ -47,6 +47,7 @@ module Visionary
47
47
  @profiler = Profiler.new(cfg, @instance, @store, @writer, @logger) if cfg.profile && @writer
48
48
  @telemetry = Telemetry::Recorder.new(cfg, @instance, @writer) if cfg.spans || cfg.errors
49
49
  @logs = Logs::Pipeline.new(cfg, @instance, @store, @schemas, @logger) if cfg.logs
50
+ @metrics = Metrics::Recorder.new(cfg, @instance, @store, @schemas, @logger) if cfg.metrics
50
51
  if cfg.system_metrics && cfg.system_interval.to_f.positive?
51
52
  @system = System::Recorder.new(cfg, @instance, @store, @schemas, @http, @logger)
52
53
  end
@@ -59,10 +60,11 @@ module Visionary
59
60
  tables["system_metrics"] = cfg.system_retention_days if @system
60
61
  tables["profiles"] = cfg.log_retention_days if @profiler
61
62
  tables["app_logs"] = cfg.log_retention_days if @logs
63
+ tables["metrics"] = cfg.log_retention_days if @metrics
62
64
  @retention = Retention.new(@store, @logger, tables)
63
65
  end
64
66
 
65
- def enabled? = !@store.nil? && !@closed && (!@writer.nil? || !@system.nil? || !@logs.nil?)
67
+ def enabled? = !@store.nil? && !@closed && (!@writer.nil? || !@system.nil? || !@logs.nil? || !@metrics.nil?)
66
68
 
67
69
  def log_path?(path) = @config.log_requests && !@writer.nil? && path.start_with?(@config.path_prefix.to_s)
68
70
 
@@ -71,6 +73,11 @@ module Visionary
71
73
  @writer&.enqueue(item) unless @closed
72
74
  end
73
75
 
76
+ def writer_stats
77
+ zero = WriterStats.new(queued: 0, written: 0, dropped: 0)
78
+ [@writer, @logs&.writer, @metrics&.writer].compact.sum(zero, &:stats)
79
+ end
80
+
74
81
  # The route pattern for a finished request: the app's own resolver,
75
82
  # then what the framework left in the env.
76
83
  def route_for(env)
@@ -97,6 +104,7 @@ module Visionary
97
104
  @logger.info("visionary: recording system metrics every #{@config.system_interval}s as #{@instance}")
98
105
  end
99
106
  @logs&.start
107
+ @metrics&.start
100
108
  @retention.start
101
109
  @profiler&.start
102
110
  install_hooks
@@ -116,6 +124,7 @@ module Visionary
116
124
  @profiler&.close
117
125
  idle = @writer ? @writer.close(timeout) : true
118
126
  idle = @logs.close([deadline - mono, 0].max) && idle if @logs
127
+ idle = @metrics.close([deadline - mono, 0].max) && idle if @metrics
119
128
  idle = @system.close([deadline - mono, 0].max) && idle if @system
120
129
  @store.close if idle && @store
121
130
  rescue StandardError => e
@@ -135,6 +144,7 @@ module Visionary
135
144
  @system&.after_fork
136
145
  @writer&.after_fork
137
146
  @logs&.after_fork
147
+ @metrics&.after_fork
138
148
  @retention&.after_fork if @retention&.needed?
139
149
  @profiler&.after_fork
140
150
  @logger.info("visionary: restarted in forked process #{@pid}")
@@ -153,9 +163,10 @@ module Visionary
153
163
  nil
154
164
  end
155
165
 
166
+ # Net::HTTP carries the trace on even when spans are off
156
167
  def install_hooks
157
- return unless @telemetry&.spans_on
158
168
  Telemetry::NetHTTPHook.install
169
+ return unless @telemetry&.spans_on
159
170
  Telemetry::ActiveRecordHook.install if defined?(::ActiveRecord::Base) && defined?(::ActiveSupport::Notifications)
160
171
  rescue StandardError => e
161
172
  @logger.warn("visionary: span hooks not installed: #{e.message}")
@@ -22,7 +22,8 @@ module Visionary
22
22
  :error_code_field, :error_key_field,
23
23
  :system_metrics, :system_interval, :system_retention_days, :disk_path,
24
24
  :service, :version, :spans, :span_threshold_ms, :span_max, :errors,
25
- :profile, :logs, :log_level, :log_queue_size
25
+ :profile, :logs, :log_level, :log_queue_size,
26
+ :metrics, :metrics_interval, :metrics_max_series
26
27
 
27
28
  # Runs on the writer thread, once per distinct user per batch, for users
28
29
  # missing a handle or role: ->(id) { [handle, role] } or nil.
@@ -46,7 +47,7 @@ module Visionary
46
47
  @capture_bodies = true
47
48
  @max_body_bytes = 8192
48
49
  @queue_size = 5000
49
- @batch_size = 50
50
+ @batch_size = 0
50
51
  @flush_interval = 3.0
51
52
  @log_retention_days = 0 # 0: off, the Visionary server applies Settings → Retention
52
53
  @redact_headers = []
@@ -71,6 +72,9 @@ module Visionary
71
72
  # The lowest level kept until Settings → Logs has been read.
72
73
  @log_level = "info"
73
74
  @log_queue_size = 10_000
75
+ @metrics = true
76
+ @metrics_interval = 15.0
77
+ @metrics_max_series = 1000
74
78
  end
75
79
 
76
80
  def self.from_env(env = ENV)
@@ -104,6 +108,9 @@ module Visionary
104
108
  c.profile = parse_bool(s.("VISIONARY_PROFILE"), true)
105
109
  c.logs = parse_bool(s.("VISIONARY_LOGS"), true)
106
110
  c.log_level = s.("VISIONARY_LOG_LEVEL").then { |v| Logs.parse_level(v).positive? ? v.downcase : "info" }
111
+ c.metrics = parse_bool(s.("VISIONARY_METRICS"), true)
112
+ c.metrics_interval = parse_duration(s.("VISIONARY_METRICS_INTERVAL"), 15.0)
113
+ c.metrics_max_series = parse_int(s.("VISIONARY_METRICS_MAX_SERIES"), 1000)
107
114
  c
108
115
  end
109
116
 
@@ -12,11 +12,15 @@ module Visionary
12
12
  class RequestContext
13
13
  KEY = :visionary_request
14
14
 
15
- attr_reader :request_id, :method, :path, :started_at, :started_mono, :errors, :db_visible, :trace_id
15
+ attr_reader :request_id, :method, :path, :started_at, :started_mono, :errors, :db_visible, :trace_id,
16
+ :span_id, :parent_span_id
16
17
 
17
- def initialize(request_id:, method:, path:, started_at:, started_mono:, span_max:, db_visible:, trace_id: nil)
18
+ def initialize(request_id:, method:, path:, started_at:, started_mono:, span_max:, db_visible:, trace_id: nil,
19
+ span: nil, parent_span_id: nil)
18
20
  @request_id = request_id
19
- @trace_id = trace_id
21
+ @trace_id = span ? span.trace_id : trace_id
22
+ @span_id = span&.span_id
23
+ @parent_span_id = parent_span_id
20
24
  @method = method
21
25
  @path = path
22
26
  @started_at = started_at
@@ -148,5 +152,6 @@ module Visionary
148
152
 
149
153
  # One database, HTTP, cache, queue, custom, graphql or background call.
150
154
  Span = Struct.new(:kind, :name, :target, :detail, :started_at, :offset_ms, :duration_ms,
151
- :status, :error, :row_count, :request_id, keyword_init: true)
155
+ :status, :error, :row_count, :request_id, :trace_id, :span_id, :parent_span_id,
156
+ keyword_init: true)
152
157
  end
@@ -195,11 +195,10 @@ module Visionary
195
195
  !local.nil? && n < local
196
196
  end
197
197
 
198
- # Falls back to the active OpenTelemetry span when the request brought no traceparent.
198
+ # Falls back to the active OpenTelemetry span outside any Visionary span
199
199
  def otel(rec)
200
200
  return unless defined?(::OpenTelemetry::Trace)
201
- ctx = RequestContext.current
202
- return if ctx&.trace_id
201
+ return if TraceContext.current
203
202
  sc = ::OpenTelemetry::Trace.current_span.context
204
203
  return unless sc.valid?
205
204
  rec.trace_id = sc.hex_trace_id
@@ -75,12 +75,15 @@ module Visionary
75
75
  return if !s.enabled || rec.message.start_with?("visionary: ")
76
76
  return if s.dropped?(@service, rec.logger, rec.message, rec.sev)
77
77
 
78
+ if rec.trace_id.nil? && (sc = TraceContext.current)
79
+ rec.trace_id = sc.trace_id
80
+ rec.span_id ||= sc.span_id
81
+ end
78
82
  unless ctx
79
83
  emit(rec) if rec.sev >= s.floor(@service, "", "", rec.at)
80
84
  return
81
85
  end
82
86
  rec.request_id ||= ctx.request_id
83
- rec.trace_id ||= ctx.trace_id
84
87
  rec.user_id ||= blank(ctx.user[0])
85
88
  return emit(rec) if rec.sev >= s.hold_below(@service, rec.at)
86
89
  return if ctx.hold_log(rec, MAX_HELD)
@@ -186,7 +189,7 @@ module Visionary
186
189
  end
187
190
 
188
191
  def statements(_kind, rec, _users)
189
- [stmt(:log, @pipeline.row(rec))]
192
+ [fit(:log, @pipeline.row(rec))]
190
193
  end
191
194
  end
192
195
  end