closeyourit-ruby 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +16 -1
- data/lib/closeyourit/background_worker.rb +14 -14
- data/lib/closeyourit/breadcrumb.rb +2 -2
- data/lib/closeyourit/breadcrumb_buffer.rb +2 -2
- data/lib/closeyourit/client.rb +24 -24
- data/lib/closeyourit/configuration.rb +104 -103
- data/lib/closeyourit/event.rb +6 -6
- data/lib/closeyourit/events/error_event.rb +16 -16
- data/lib/closeyourit/events/job_metric_event.rb +11 -8
- data/lib/closeyourit/events/log_event.rb +17 -17
- data/lib/closeyourit/events/message_event.rb +4 -4
- data/lib/closeyourit/events/performance_issue_event.rb +4 -4
- data/lib/closeyourit/events/slow_method_event.rb +6 -6
- data/lib/closeyourit/events/slow_query_event.rb +4 -4
- data/lib/closeyourit/instrumenter.rb +6 -6
- data/lib/closeyourit/job_context.rb +61 -0
- data/lib/closeyourit/line_cache.rb +4 -4
- data/lib/closeyourit/log_buffer.rb +10 -10
- data/lib/closeyourit/log_device.rb +18 -18
- data/lib/closeyourit/monitor.rb +2 -2
- data/lib/closeyourit/performance/request_profile.rb +5 -5
- data/lib/closeyourit/performance/rollup.rb +3 -3
- data/lib/closeyourit/rails/active_job_extension.rb +36 -31
- data/lib/closeyourit/rails/capture_exceptions.rb +3 -3
- data/lib/closeyourit/rails/error_subscriber.rb +4 -4
- data/lib/closeyourit/rails/log_broadcast.rb +12 -12
- data/lib/closeyourit/rails/net_http_patch.rb +22 -22
- data/lib/closeyourit/rails/query_source.rb +3 -3
- data/lib/closeyourit/rails/railtie.rb +29 -65
- data/lib/closeyourit/rails/request_body.rb +7 -7
- data/lib/closeyourit/rails/request_context.rb +27 -27
- data/lib/closeyourit/scope.rb +29 -29
- data/lib/closeyourit/scrubber.rb +50 -50
- data/lib/closeyourit/sidekiq/error_handler.rb +2 -2
- data/lib/closeyourit/sidekiq/job_metrics_middleware.rb +39 -17
- data/lib/closeyourit/stats.rb +8 -8
- data/lib/closeyourit/subscribers/job_performance.rb +102 -24
- data/lib/closeyourit/subscribers/request_performance.rb +4 -4
- data/lib/closeyourit/subscribers/slow_query.rb +42 -20
- data/lib/closeyourit/trace_context.rb +22 -22
- data/lib/closeyourit/transport.rb +22 -22
- data/lib/closeyourit/usage_registry.rb +17 -16
- data/lib/closeyourit/version.rb +1 -1
- data/lib/closeyourit-ruby.rb +125 -125
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d77ef65ad83c3890320e2c3c3a002a6708ef52d223204da4b553ac75ec58c4b9
|
|
4
|
+
data.tar.gz: 4bc0274665e54f19fe33d6e379270f31802a71317075ceae65fecdd4b8a6d66c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cc89f3b0f20d8781b5a963b9fda04935b735500612db5883b29f86abc92bc560a452abc133db66859b919cdfcd8994ff6c4891fdcafe39a6c7878a960c74b627
|
|
7
|
+
data.tar.gz: 5b55acc5e39e78e9fc875e6cbfd32dfce56bbb9e27d600ec30f86a6d27d12660c1c61195a28c32f32ac22b418c78ac9d41e83af8bbdbd26f96781d740bd6f1cd
|
data/README.md
CHANGED
|
@@ -109,6 +109,7 @@ end
|
|
|
109
109
|
| `slow_request_threshold_ms` | `1000` | Durata totale della richiesta (ms) oltre cui = `slow_request` |
|
|
110
110
|
| `slow_external_threshold_ms` | `1000` | Durata di una singola HTTP esterna (ms) oltre cui = `slow_external_http` |
|
|
111
111
|
| `capture_external_http` | `true` | Strumenta `Net::HTTP` per rilevare le HTTP esterne (effettivo solo con `detect_performance_issues`) |
|
|
112
|
+
| `job_lifecycle_logs` | `false` | Log informativo per esito di tentativo osservato, opt-in. |
|
|
112
113
|
| `monitor_jobs` | `true` | Misura durata e attesa in coda dei background job (ActiveJob + Sidekiq) → metriche `slow_job`/`job_queue_latency` (vedi [Metriche dei background job](#metriche-dei-background-job)) |
|
|
113
114
|
| `slow_job_threshold_ms` | `5000` | Durata del job (ms) oltre cui = `slow_job` |
|
|
114
115
|
| `job_queue_latency_threshold_ms` | `60000` | Attesa in coda (enqueue→esecuzione, ms) oltre cui = `job_queue_latency` |
|
|
@@ -359,7 +360,7 @@ sulla pipeline metriche (`/api/v1/projects/:id/metrics`). Funziona con **ActiveJ
|
|
|
359
360
|
Due `subtype`:
|
|
360
361
|
|
|
361
362
|
- **`slow_job`** — la durata del `perform` supera `slow_job_threshold_ms` (default 5000 ms).
|
|
362
|
-
- **`job_queue_latency`** — l'attesa
|
|
363
|
+
- **`job_queue_latency`** — l'attesa dopo la scadenza effettiva (`max(enqueued_at, scheduled_at)`) supera
|
|
363
364
|
`job_queue_latency_threshold_ms` (default 60000 ms): coda intasata o worker insufficienti.
|
|
364
365
|
|
|
365
366
|
Ogni metrica porta la **label** (nome della classe del job), la **queue**, l'**adapter**
|
|
@@ -380,6 +381,20 @@ CloseYourIt.init do |c|
|
|
|
380
381
|
end
|
|
381
382
|
```
|
|
382
383
|
|
|
384
|
+
Il contesto additivo `contexts.job` usa il contratto `schema_version: 1`: separa durata monotona,
|
|
385
|
+
attesa in coda e ritardo programmato. Un orologio mancante resta `null`; un'attesa negativa viene
|
|
386
|
+
azzerata con `clock_skew: true`. Sidekiq 8 usa timestamp in millisecondi, le versioni precedenti
|
|
387
|
+
secondi: la conversione segue la versione del framework, senza euristiche sulla grandezza.
|
|
388
|
+
|
|
389
|
+
Per osservare anche gli esiti dei tentativi veloci, abilita `c.job_lifecycle_logs = true` (default
|
|
390
|
+
`false`). Emette un log `info` con messaggio fisso `Background job attempt finished` e
|
|
391
|
+
`attributes.contexts.job`, senza argomenti, risultati o header. `retry` non è terminale;
|
|
392
|
+
`discarded` è esplicito; un errore con retry dell'adapter non noto ha `terminal: null`.
|
|
393
|
+
Un `rescue_from` applicativo non identificabile resta `unknown`, non diventa successo.
|
|
394
|
+
ActiveJob dentro Sidekiq è osservato una sola volta quando il subscriber ActiveJob è installato.
|
|
395
|
+
Sampling, disabilitazione o perdita del transport impediscono di considerare questi log un censimento.
|
|
396
|
+
Il raggruppamento backend v1 separa framework, nome, coda e misura; i gruppi legacy restano intatti.
|
|
397
|
+
|
|
383
398
|
## Privacy & PII
|
|
384
399
|
|
|
385
400
|
Privacy-by-default (`send_pii = false`). In sintesi:
|
|
@@ -3,9 +3,9 @@
|
|
|
3
3
|
require "concurrent"
|
|
4
4
|
|
|
5
5
|
module CloseYourIt
|
|
6
|
-
#
|
|
7
|
-
#
|
|
8
|
-
# (
|
|
6
|
+
# Runs the send fire-and-forget. With `threads == 0` it runs synchronously (test/dev);
|
|
7
|
+
# otherwise it uses a thread pool with a bounded queue and `fallback_policy: :discard`
|
|
8
|
+
# (when the queue is full the event is lost, never backpressure on the request).
|
|
9
9
|
class BackgroundWorker
|
|
10
10
|
attr_reader :executor
|
|
11
11
|
|
|
@@ -13,29 +13,29 @@ module CloseYourIt
|
|
|
13
13
|
@executor = build_executor(threads.to_i, max_queue)
|
|
14
14
|
end
|
|
15
15
|
|
|
16
|
-
#
|
|
17
|
-
#
|
|
16
|
+
# Returns true if the event was accepted (or run synchronously), false if dropped
|
|
17
|
+
# because the queue was full (`fallback_policy: :discard`). Never backpressure on the request.
|
|
18
18
|
def perform(&block)
|
|
19
19
|
accepted = @executor.post do
|
|
20
20
|
block.call
|
|
21
21
|
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
22
|
-
#
|
|
22
|
+
# Never propagate: telemetry must not be able to crash the host app.
|
|
23
23
|
CloseYourIt.internal_logger.error("CloseYourIt background worker: #{e.class}: #{e.message}")
|
|
24
24
|
end
|
|
25
25
|
|
|
26
26
|
unless accepted
|
|
27
27
|
CloseYourIt.stats.increment(:dropped)
|
|
28
|
-
CloseYourIt.internal_logger.warn("CloseYourIt background worker:
|
|
28
|
+
CloseYourIt.internal_logger.warn("CloseYourIt background worker: queue full, event dropped")
|
|
29
29
|
CloseYourIt.notify_diagnostic(:drop, reason: :queue_full)
|
|
30
30
|
end
|
|
31
31
|
|
|
32
32
|
accepted
|
|
33
33
|
end
|
|
34
34
|
|
|
35
|
-
#
|
|
36
|
-
# (Transport::MAX_REQUEST_SECONDS):
|
|
37
|
-
#
|
|
38
|
-
#
|
|
35
|
+
# Drains the queue waiting up to `timeout` seconds. The default is the duration cap of ONE POST
|
|
36
|
+
# (Transport::MAX_REQUEST_SECONDS): waiting less abandons still healthy sends, which is what
|
|
37
|
+
# happened with the old fixed one second (CYRB-24). If the drain expires, the tasks left in the
|
|
38
|
+
# queue are lost: count them as `dropped`, otherwise the end-of-life snapshot reports zero losses.
|
|
39
39
|
def shutdown(timeout = Transport::MAX_REQUEST_SECONDS)
|
|
40
40
|
return unless @executor.respond_to?(:shutdown)
|
|
41
41
|
|
|
@@ -47,12 +47,12 @@ module CloseYourIt
|
|
|
47
47
|
|
|
48
48
|
private
|
|
49
49
|
|
|
50
|
-
#
|
|
51
|
-
#
|
|
50
|
+
# Tasks queued and never run when the drain expired. `queue_length` does not exist on the
|
|
51
|
+
# ImmediateExecutor (synchronous, no queue) → zero left over.
|
|
52
52
|
def report_abandoned(timeout)
|
|
53
53
|
abandoned = @executor.respond_to?(:queue_length) ? @executor.queue_length.to_i : 0
|
|
54
54
|
CloseYourIt.internal_logger.warn(
|
|
55
|
-
"CloseYourIt background worker: drain
|
|
55
|
+
"CloseYourIt background worker: drain expired after #{timeout}s, #{abandoned} events abandoned"
|
|
56
56
|
)
|
|
57
57
|
abandoned.times do
|
|
58
58
|
CloseYourIt.stats.increment(:dropped)
|
|
@@ -3,8 +3,8 @@
|
|
|
3
3
|
require "time"
|
|
4
4
|
|
|
5
5
|
module CloseYourIt
|
|
6
|
-
#
|
|
7
|
-
#
|
|
6
|
+
# A single context breadcrumb (query, navigation, custom event) preceding an error.
|
|
7
|
+
# Sentry event shape (`breadcrumbs.values[]`). `data` is already scrubbed upstream (module API).
|
|
8
8
|
class Breadcrumb
|
|
9
9
|
def initialize(message: nil, category: nil, type: "default", level: "info", data: {}, timestamp: nil)
|
|
10
10
|
@timestamp = timestamp || Time.now.utc.iso8601
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
module CloseYourIt
|
|
4
|
-
#
|
|
5
|
-
#
|
|
4
|
+
# Bounded breadcrumb ring buffer. It lives in the Scope (one buffer per execution context),
|
|
5
|
+
# written only by the owning thread → no mutex. Beyond `max_size` it drops the oldest.
|
|
6
6
|
class BreadcrumbBuffer
|
|
7
7
|
def initialize(max_size)
|
|
8
8
|
@max_size = max_size.to_i
|
data/lib/closeyourit/client.rb
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
module CloseYourIt
|
|
4
|
-
#
|
|
5
|
-
#
|
|
4
|
+
# Combines Transport + BackgroundWorker: applies `before_send` and dispatches
|
|
5
|
+
# the send fire-and-forget.
|
|
6
6
|
class Client
|
|
7
|
-
#
|
|
8
|
-
# (413 R413-LOG-002)
|
|
9
|
-
#
|
|
7
|
+
# Log cap per single request to /logs. The backend rejects a batch beyond this limit
|
|
8
|
+
# (413 R413-LOG-002) dropping the WHOLE request — and the buffer has already been drained → lost
|
|
9
|
+
# logs. Must stay ≤ the server limit (backend LOGS_MAX_BATCH = 1000). See #flush_logs.
|
|
10
10
|
LOGS_MAX_BATCH = 1000
|
|
11
11
|
|
|
12
12
|
def initialize(configuration)
|
|
@@ -22,7 +22,7 @@ module CloseYourIt
|
|
|
22
22
|
payload = event.to_h
|
|
23
23
|
payload = @configuration.before_send.call(payload) if @configuration.before_send
|
|
24
24
|
if payload.nil?
|
|
25
|
-
# before_send
|
|
25
|
+
# before_send dropped the event (returned nil): an intentional drop, made visible (CYRB-12).
|
|
26
26
|
CloseYourIt.stats.increment(:dropped)
|
|
27
27
|
CloseYourIt.notify_diagnostic(:drop, reason: :before_send)
|
|
28
28
|
return nil
|
|
@@ -36,21 +36,21 @@ module CloseYourIt
|
|
|
36
36
|
end
|
|
37
37
|
payload
|
|
38
38
|
rescue StandardError => e
|
|
39
|
-
#
|
|
40
|
-
#
|
|
41
|
-
#
|
|
42
|
-
#
|
|
39
|
+
# Telemetry must NEVER propagate into the host app's path: capture_event is called by the
|
|
40
|
+
# sql.active_record subscriber, which runs on the query's thread. to_h (scrubber on a malformed
|
|
41
|
+
# bind) and before_send are evaluated here synchronously → if they raise, we absorb, log and
|
|
42
|
+
# drop the event instead of disturbing the host query. See CYRB-2.
|
|
43
43
|
CloseYourIt.internal_logger.error("CloseYourIt client: #{e.class}: #{e.message}")
|
|
44
44
|
CloseYourIt.stats.increment(:dropped)
|
|
45
45
|
CloseYourIt.notify_diagnostic(:drop, reason: :error, error: e.class.name)
|
|
46
46
|
nil
|
|
47
47
|
end
|
|
48
48
|
|
|
49
|
-
#
|
|
50
|
-
#
|
|
51
|
-
# LOGS_MAX_BATCH
|
|
52
|
-
#
|
|
53
|
-
#
|
|
49
|
+
# Sends a batch of logs as an ARRAY to /logs (the endpoint accepts a single item or an array).
|
|
50
|
+
# before_send is applied to each payload; dropped ones (nil) are not sent. Payloads beyond
|
|
51
|
+
# LOGS_MAX_BATCH are split into several sequential POSTs (one chunk = one POST), so a large flush
|
|
52
|
+
# is not rejected as a whole by the backend and lost — see R3 / LOGS_MAX_BATCH. A flush within
|
|
53
|
+
# the limit stays a single POST.
|
|
54
54
|
def flush_logs(events)
|
|
55
55
|
return nil if events.nil? || events.empty?
|
|
56
56
|
|
|
@@ -68,8 +68,8 @@ module CloseYourIt
|
|
|
68
68
|
payloads
|
|
69
69
|
end
|
|
70
70
|
|
|
71
|
-
# CYSK-29 —
|
|
72
|
-
#
|
|
71
|
+
# CYSK-29 — the usage telemetry flush: ONE POST per window, a payload free of user data by
|
|
72
|
+
# construction (route = Controller#action). Fire-and-forget via the worker, like everything else.
|
|
73
73
|
def flush_usage(symbols:, truncated:, window_started_at:, window_ended_at:)
|
|
74
74
|
payload = {
|
|
75
75
|
environment: @configuration.environment.to_s,
|
|
@@ -90,22 +90,22 @@ module CloseYourIt
|
|
|
90
90
|
|
|
91
91
|
private
|
|
92
92
|
|
|
93
|
-
#
|
|
94
|
-
#
|
|
95
|
-
#
|
|
96
|
-
#
|
|
93
|
+
# Builds the payload of ONE log entry. `to_h` (scrubber on a non-UTF-8 byte) and `before_send`
|
|
94
|
+
# may raise: here the buffer has already been drained, so a single broken entry evaluated together
|
|
95
|
+
# with the others would take the whole batch of healthy entries with it. Each entry is isolated
|
|
96
|
+
# and dropped on its own, like the JS client does (CYRB-24). `nil` = entry not to send, already counted.
|
|
97
97
|
def build_log_payload(event)
|
|
98
98
|
payload = event.to_h
|
|
99
99
|
payload = @configuration.before_send.call(payload) if @configuration.before_send
|
|
100
100
|
return payload unless payload.nil?
|
|
101
101
|
|
|
102
|
-
#
|
|
103
|
-
#
|
|
102
|
+
# Intentional before_send drop: counted as in #capture_event, otherwise it would silently
|
|
103
|
+
# disappear from the counters (CYRB-12).
|
|
104
104
|
CloseYourIt.stats.increment(:dropped)
|
|
105
105
|
CloseYourIt.notify_diagnostic(:drop, reason: :before_send)
|
|
106
106
|
nil
|
|
107
107
|
rescue StandardError => e
|
|
108
|
-
CloseYourIt.internal_logger.error("CloseYourIt client: log
|
|
108
|
+
CloseYourIt.internal_logger.error("CloseYourIt client: log dropped — #{e.class}: #{e.message}")
|
|
109
109
|
CloseYourIt.stats.increment(:dropped)
|
|
110
110
|
CloseYourIt.notify_diagnostic(:drop, reason: :error, error: e.class.name)
|
|
111
111
|
nil
|