logister-ruby 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +94 -24
- data/lib/generators/logister/templates/logister.rb +5 -0
- data/lib/logister/client.rb +204 -12
- data/lib/logister/configuration.rb +11 -1
- data/lib/logister/railtie.rb +5 -0
- data/lib/logister/reporter.rb +8 -0
- data/lib/logister/reporting_scope.rb +39 -0
- data/lib/logister/request_subscriber.rb +3 -0
- data/lib/logister/sql_subscriber.rb +2 -0
- data/lib/logister/version.rb +1 -1
- data/lib/logister.rb +9 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ff89324842289ec0925e84131ee9442d4dfed5589b86deb1bb56498e55eb5977
|
|
4
|
+
data.tar.gz: f262d7ab245a8c5703fb28873e65d4ee11194be7146833f656fd74d85434f2ff
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 28a1b3621688f111349b37c9996d530855d07081c17ce4525e33426b2ff30f1c714f8871555925dd7c11ec78ff09ad0566295552c54d05d141c28b944fe2f383
|
|
7
|
+
data.tar.gz: c262d8b152f55680dc7ad02d19733089b59415f4a853cf33d5e8ffcade2851a69d4ada2963a7e05e4af8ca758e08340288213ea5e90da68d68c1b6baaf768108
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v0.4.0 - 2026-08-08
|
|
4
|
+
|
|
5
|
+
- Added stable UUID assignment before synchronous or asynchronous delivery so retries preserve one logical event identity.
|
|
6
|
+
- Added configurable gzip/NDJSON batching for the Logister batch-ingest endpoint, including deterministic batch IDs, bounded batch size and interval, and automatic fallback to stable single-event delivery for older servers.
|
|
7
|
+
- Added recursive `413` batch splitting while continuing every split or fallback attempt, preventing one failed subset from short-circuiting the remainder.
|
|
8
|
+
- Added transient-only retries with connect, read, and write timeouts, numeric or HTTP-date `Retry-After` support, capped exponential backoff, and bounded jitter.
|
|
9
|
+
- Added batch and retry settings to the Rails generator and Railtie configuration bridge, and made `Logister.flush` wait for in-flight batch delivery rather than only an empty queue.
|
|
10
|
+
|
|
11
|
+
## v0.3.1 - 2026-07-26
|
|
12
|
+
|
|
13
|
+
- Added `Logister.suppress_reporting` and `Logister.reporting_suppressed?` for recursion-safe telemetry processing, including early exits in automatic SQL and request subscribers.
|
|
14
|
+
|
|
3
15
|
## v0.3.0 - 2026-07-25
|
|
4
16
|
|
|
5
17
|
- Raised the supported runtime floor to Ruby 3.3 and refreshed the development runtime to Ruby 4.0.6, which includes current interpreter security fixes.
|
data/README.md
CHANGED
|
@@ -1,11 +1,47 @@
|
|
|
1
1
|
# logister-ruby
|
|
2
2
|
|
|
3
|
-
`logister-ruby`
|
|
3
|
+
`logister-ruby` sends Ruby and Rails errors, logs, metrics, transactions, spans, and scheduled-job check-ins to Logister. Rails apps also get automatic reporting for unhandled requests and failed Active Job executions.
|
|
4
4
|
|
|
5
5
|
Install it from RubyGems as `logister-ruby`.
|
|
6
6
|
|
|
7
|
+
Requires Ruby 3.3 or newer and Active Support 8.x.
|
|
8
|
+
|
|
9
|
+
## Quick start
|
|
10
|
+
|
|
11
|
+
Before you start, create a project in Logister and generate a project API key under **Project settings → API keys**.
|
|
12
|
+
|
|
13
|
+
Add the gem and generate a Rails initializer:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
bundle add logister-ruby
|
|
17
|
+
bin/rails generate logister:install
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Store configuration in environment variables:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
export LOGISTER_API_KEY="<project-api-key>"
|
|
24
|
+
export LOGISTER_ENDPOINT="https://logister.example.com/api/v1/ingest_events"
|
|
25
|
+
export LOGISTER_SERVICE="checkout-web"
|
|
26
|
+
export LOGISTER_RELEASE="$(git rev-parse --short HEAD)"
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Start Rails, then send a safe test event from `bin/rails console`:
|
|
30
|
+
|
|
31
|
+
```ruby
|
|
32
|
+
Logister.report_error(
|
|
33
|
+
RuntimeError.new("README test error"),
|
|
34
|
+
context: { component: "checkout" },
|
|
35
|
+
fingerprint: "readme-test-error"
|
|
36
|
+
)
|
|
37
|
+
Logister.flush
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Open the project inbox and confirm that **README test error** appears. A `401` response usually means the API key or endpoint is wrong; see the [Ruby integration guide](https://logister.org/docs/integrations/ruby/) for troubleshooting.
|
|
41
|
+
|
|
7
42
|
## Table Of Contents
|
|
8
43
|
|
|
44
|
+
- [Quick start](#quick-start)
|
|
9
45
|
- [What this gem is for](#what-this-gem-is-for)
|
|
10
46
|
- [Package Links](#package-links)
|
|
11
47
|
- [Self-hosted backend](#self-hosted-backend)
|
|
@@ -18,17 +54,19 @@ Install it from RubyGems as `logister-ruby`.
|
|
|
18
54
|
- [Breadcrumbs and dependencies](#breadcrumbs-and-dependencies)
|
|
19
55
|
- [ActiveJob error context](#activejob-error-context)
|
|
20
56
|
- [Manual reporting](#manual-reporting)
|
|
21
|
-
- [Using project Insights
|
|
57
|
+
- [Using project Insights](#using-project-insights)
|
|
58
|
+
- [GitHub source context and deployments](#github-source-context-and-deployments)
|
|
22
59
|
- [Documentation](#documentation)
|
|
60
|
+
- [Development](#development)
|
|
23
61
|
- [Release](#release)
|
|
24
62
|
|
|
25
63
|
## What this gem is for
|
|
26
64
|
|
|
27
|
-
Use this gem when
|
|
65
|
+
Use this gem when a Ruby process should send telemetry to a hosted or self-hosted Logister server. It is an ingest client, not the Logister server itself.
|
|
28
66
|
|
|
29
67
|
- Main Logister app: https://github.com/taimoorq/logister
|
|
30
|
-
- Ruby integration docs: https://
|
|
31
|
-
- Product docs: https://
|
|
68
|
+
- Ruby integration docs: https://logister.org/docs/integrations/ruby/
|
|
69
|
+
- Product docs: https://logister.org/docs/
|
|
32
70
|
- RubyGems package: https://rubygems.org/gems/logister-ruby
|
|
33
71
|
|
|
34
72
|
## Package Links
|
|
@@ -36,7 +74,7 @@ Use this gem when you want a Ruby or Rails app to send telemetry into the Logist
|
|
|
36
74
|
- RubyGems package: https://rubygems.org/gems/logister-ruby
|
|
37
75
|
- GitHub releases: https://github.com/taimoorq/logister-ruby/releases
|
|
38
76
|
- Source repository: https://github.com/taimoorq/logister-ruby
|
|
39
|
-
- Integration docs: https://
|
|
77
|
+
- Integration docs: https://logister.org/docs/integrations/ruby/
|
|
40
78
|
|
|
41
79
|
## Self-hosted backend
|
|
42
80
|
|
|
@@ -100,17 +138,34 @@ end
|
|
|
100
138
|
|
|
101
139
|
If you are using a self-hosted Logister install, point `config.endpoint` at your own Logister host instead of `logister.org`.
|
|
102
140
|
|
|
141
|
+
Keep `LOGISTER_API_KEY` in your deployment secret store. Project API keys are write-only ingest credentials, but exposing one still lets another party submit unwanted telemetry to the project.
|
|
142
|
+
|
|
103
143
|
## Reliability options
|
|
104
144
|
|
|
105
145
|
```ruby
|
|
106
146
|
Logister.configure do |config|
|
|
107
147
|
config.async = true
|
|
108
148
|
config.queue_size = 1000
|
|
149
|
+
config.batch_size = 50
|
|
150
|
+
config.batch_interval = 0.05
|
|
151
|
+
config.batch_compression = true
|
|
109
152
|
config.max_retries = 3
|
|
110
153
|
config.retry_base_interval = 0.5
|
|
154
|
+
config.max_retry_delay = 30.0
|
|
155
|
+
config.retry_jitter = 0.2
|
|
111
156
|
end
|
|
112
157
|
```
|
|
113
158
|
|
|
159
|
+
Asynchronous delivery assigns a UUID before enqueueing, combines queued events into
|
|
160
|
+
gzip/NDJSON batches, and retries the same identifiers. This makes a whole-batch retry
|
|
161
|
+
safe against a Logister server that supports the batch endpoint. Older servers are
|
|
162
|
+
detected automatically and receive the same stable events through the single-event
|
|
163
|
+
endpoint. Call `Logister.flush` before a short-lived process exits.
|
|
164
|
+
|
|
165
|
+
Every HTTP attempt applies `timeout_seconds` to connect, read, and write operations.
|
|
166
|
+
Retryable responses honor `Retry-After` when present, cap any individual wait at
|
|
167
|
+
`max_retry_delay`, and add bounded positive jitter controlled by `retry_jitter`.
|
|
168
|
+
|
|
114
169
|
## Filtering and redaction
|
|
115
170
|
|
|
116
171
|
```ruby
|
|
@@ -126,12 +181,21 @@ Logister.configure do |config|
|
|
|
126
181
|
end
|
|
127
182
|
```
|
|
128
183
|
|
|
184
|
+
Use scoped suppression around work that handles or forwards telemetry. Every manual reporter and automatic Rails subscriber returns without publishing while the scope is active, and nested scopes restore the previous state even when the block raises:
|
|
185
|
+
|
|
186
|
+
```ruby
|
|
187
|
+
Logister.suppress_reporting do
|
|
188
|
+
TelemetryMirror.call(event)
|
|
189
|
+
end
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
This is especially important when a Rails application reports into a Logister project hosted by that same application. Keep suppression narrow so failures outside the telemetry-processing boundary remain observable.
|
|
193
|
+
|
|
129
194
|
## Rails auto-reporting
|
|
130
195
|
|
|
131
|
-
If Rails is present, the gem installs middleware that reports unhandled exceptions automatically.
|
|
132
|
-
|
|
133
|
-
Set `config.capture_request_spans = true` to emit root `server` spans for request
|
|
134
|
-
Manual `Logister.report_error` calls use the same shared enrichment path, so Ruby apps get runtime, deployment, breadcrumb, dependency, user, and nested exception cause context even when an error is reported outside the Rails middleware.
|
|
196
|
+
If Rails is present, the gem installs middleware that reports unhandled exceptions automatically. It attaches trace IDs, route and response data, performance context, breadcrumbs, dependency calls, and user metadata when available.
|
|
197
|
+
|
|
198
|
+
Set `config.capture_request_spans = true` to emit root `server` spans for request-load waterfall charts while keeping the existing transaction events. Manual `Logister.report_error` calls use the same enrichment path, including runtime, deployment, breadcrumb, dependency, user, and nested-exception context.
|
|
135
199
|
|
|
136
200
|
## Database load metrics (ActiveRecord)
|
|
137
201
|
|
|
@@ -225,7 +289,7 @@ Logister.report_check_in(
|
|
|
225
289
|
)
|
|
226
290
|
```
|
|
227
291
|
|
|
228
|
-
## Using project Insights
|
|
292
|
+
## Using project Insights
|
|
229
293
|
|
|
230
294
|
The Logister project Insights tab combines Inbox, Activity, and Performance signals into live dashboard views. Ruby apps get the most useful Insights experience when every event carries stable deployment context plus a few low-cardinality custom attributes.
|
|
231
295
|
|
|
@@ -322,28 +386,33 @@ Logister.record_deployment(
|
|
|
322
386
|
|
|
323
387
|
## Documentation
|
|
324
388
|
|
|
325
|
-
- Ruby integration docs: https://
|
|
326
|
-
- Insights
|
|
327
|
-
- Main Logister docs: https://
|
|
389
|
+
- Ruby integration docs: https://logister.org/docs/integrations/ruby/
|
|
390
|
+
- Insights guide: https://logister.org/docs/product/#insights
|
|
391
|
+
- Main Logister docs: https://logister.org/docs/
|
|
328
392
|
- [Contributing](CONTRIBUTING.md)
|
|
329
393
|
- [Code of Conduct](CODE_OF_CONDUCT.md)
|
|
330
394
|
- [Security Policy](SECURITY.md)
|
|
331
395
|
- [Pull Request Template](.github/PULL_REQUEST_TEMPLATE.md)
|
|
332
396
|
|
|
397
|
+
## Development
|
|
398
|
+
|
|
399
|
+
```bash
|
|
400
|
+
bundle install
|
|
401
|
+
bundle-audit check --update
|
|
402
|
+
bundle exec rake test
|
|
403
|
+
bundle exec rake build
|
|
404
|
+
```
|
|
405
|
+
|
|
333
406
|
## Release
|
|
334
407
|
|
|
335
|
-
|
|
408
|
+
`lib/logister/version.rb` is the package version source of truth. Update it and `CHANGELOG.md` together. After CI passes on `main`, the release-from-main workflow creates a matching `vX.Y.Z` tag and dispatches the release workflow.
|
|
336
409
|
|
|
337
410
|
```bash
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
# 3) commit changes
|
|
341
|
-
# 4) merge to main, or push a matching tag manually:
|
|
342
|
-
git tag -a v0.2.8 -m "Release logister-ruby v0.2.8"
|
|
343
|
-
git push origin main v0.2.8
|
|
411
|
+
git tag -a vX.Y.Z -m "Release logister-ruby vX.Y.Z"
|
|
412
|
+
git push origin vX.Y.Z
|
|
344
413
|
```
|
|
345
414
|
|
|
346
|
-
The
|
|
415
|
+
The release workflow verifies tag/version parity, audits and tests the package, builds the gem, publishes to RubyGems with trusted publishing, and only then creates the GitHub Release. RubyGems versions are immutable; corrections need a new patch version.
|
|
347
416
|
|
|
348
417
|
Before tag releases can publish the gem, configure a RubyGems trusted publisher for:
|
|
349
418
|
|
|
@@ -352,8 +421,9 @@ Before tag releases can publish the gem, configure a RubyGems trusted publisher
|
|
|
352
421
|
- Workflow file: `.github/workflows/release.yml`
|
|
353
422
|
- Environment: leave blank unless you also add a GitHub release environment to the workflow
|
|
354
423
|
|
|
355
|
-
|
|
424
|
+
Verify both release surfaces before calling a release complete:
|
|
356
425
|
|
|
357
426
|
```bash
|
|
358
|
-
|
|
427
|
+
curl -fsSL https://rubygems.org/api/v2/rubygems/logister-ruby/versions/X.Y.Z.json | jq '{number,ruby_version,sha}'
|
|
428
|
+
gh release view vX.Y.Z
|
|
359
429
|
```
|
|
@@ -10,8 +10,13 @@ Logister.configure do |config|
|
|
|
10
10
|
|
|
11
11
|
config.async = true
|
|
12
12
|
config.queue_size = 1000
|
|
13
|
+
config.batch_size = 50
|
|
14
|
+
config.batch_interval = 0.05
|
|
15
|
+
config.batch_compression = true
|
|
13
16
|
config.max_retries = 3
|
|
14
17
|
config.retry_base_interval = 0.5
|
|
18
|
+
config.max_retry_delay = 30.0
|
|
19
|
+
config.retry_jitter = 0.2
|
|
15
20
|
|
|
16
21
|
config.ignore_environments = []
|
|
17
22
|
config.ignore_exceptions = []
|
data/lib/logister/client.rb
CHANGED
|
@@ -2,11 +2,29 @@
|
|
|
2
2
|
|
|
3
3
|
require 'json'
|
|
4
4
|
require 'net/http'
|
|
5
|
+
require 'digest'
|
|
6
|
+
require 'securerandom'
|
|
7
|
+
require 'time'
|
|
5
8
|
require 'uri'
|
|
9
|
+
require 'zlib'
|
|
6
10
|
|
|
7
11
|
module Logister
|
|
8
12
|
class Client
|
|
9
13
|
CONTENT_TYPE = 'application/json'
|
|
14
|
+
BATCH_CONTENT_TYPE = 'application/x-ndjson'
|
|
15
|
+
UNSUPPORTED_BATCH_STATUSES = %w[404 405 415 501].freeze
|
|
16
|
+
|
|
17
|
+
class UnsupportedBatchEndpoint < StandardError; end
|
|
18
|
+
|
|
19
|
+
class RequestError < StandardError
|
|
20
|
+
attr_reader :status, :retry_after
|
|
21
|
+
|
|
22
|
+
def initialize(status, retry_after: nil)
|
|
23
|
+
@status = status.to_i
|
|
24
|
+
@retry_after = retry_after
|
|
25
|
+
super("HTTP #{status}")
|
|
26
|
+
end
|
|
27
|
+
end
|
|
10
28
|
|
|
11
29
|
def initialize(configuration)
|
|
12
30
|
@configuration = configuration
|
|
@@ -14,12 +32,17 @@ module Logister
|
|
|
14
32
|
@queue = SizedQueue.new(@configuration.queue_size)
|
|
15
33
|
@worker = nil
|
|
16
34
|
@running = false
|
|
35
|
+
@pending_mutex = Mutex.new
|
|
36
|
+
@pending_condition = ConditionVariable.new
|
|
37
|
+
@pending_count = 0
|
|
17
38
|
|
|
18
39
|
# Cache values that are static for the lifetime of this client so we
|
|
19
40
|
# don't allocate on every send_request call.
|
|
20
41
|
@uri = URI.parse(@configuration.endpoint).freeze
|
|
42
|
+
@batch_uri = URI.parse(@configuration.batch_endpoint).freeze
|
|
21
43
|
@deployment_uri = URI.parse(@configuration.deployment_endpoint).freeze
|
|
22
44
|
@use_ssl = @uri.scheme == 'https'
|
|
45
|
+
@batch_use_ssl = @batch_uri.scheme == 'https'
|
|
23
46
|
@deployment_use_ssl = @deployment_uri.scheme == 'https'
|
|
24
47
|
@auth_header = "Bearer #{@configuration.api_key}".freeze
|
|
25
48
|
end
|
|
@@ -27,6 +50,8 @@ module Logister
|
|
|
27
50
|
def publish(payload)
|
|
28
51
|
return false unless ready?
|
|
29
52
|
|
|
53
|
+
payload = with_stable_uuid(payload)
|
|
54
|
+
|
|
30
55
|
return publish_sync(payload) unless @configuration.async
|
|
31
56
|
|
|
32
57
|
ensure_worker_started
|
|
@@ -43,10 +68,13 @@ module Logister
|
|
|
43
68
|
return true unless @configuration.async
|
|
44
69
|
|
|
45
70
|
deadline = monotonic_now + timeout
|
|
46
|
-
|
|
47
|
-
|
|
71
|
+
@pending_mutex.synchronize do
|
|
72
|
+
while @pending_count.positive?
|
|
73
|
+
remaining = deadline - monotonic_now
|
|
74
|
+
return false unless remaining.positive?
|
|
48
75
|
|
|
49
|
-
|
|
76
|
+
@pending_condition.wait(@pending_mutex, [remaining, 0.05].min)
|
|
77
|
+
end
|
|
50
78
|
end
|
|
51
79
|
|
|
52
80
|
true
|
|
@@ -69,9 +97,11 @@ module Logister
|
|
|
69
97
|
private
|
|
70
98
|
|
|
71
99
|
def enqueue(payload)
|
|
100
|
+
increment_pending
|
|
72
101
|
@queue.push(payload, true)
|
|
73
102
|
true
|
|
74
103
|
rescue ThreadError
|
|
104
|
+
complete_pending(1)
|
|
75
105
|
@configuration.logger.warn('logister queue full; dropping event')
|
|
76
106
|
false
|
|
77
107
|
end
|
|
@@ -90,11 +120,31 @@ module Logister
|
|
|
90
120
|
end
|
|
91
121
|
|
|
92
122
|
def run_worker
|
|
123
|
+
stop_after_batch = false
|
|
93
124
|
loop do
|
|
94
125
|
payload = @queue.pop
|
|
95
126
|
break if payload.nil?
|
|
96
127
|
|
|
97
|
-
|
|
128
|
+
batch = [payload]
|
|
129
|
+
deadline = monotonic_now + batch_interval
|
|
130
|
+
|
|
131
|
+
while batch.length < batch_size && monotonic_now < deadline
|
|
132
|
+
begin
|
|
133
|
+
queued = @queue.pop(true)
|
|
134
|
+
if queued.nil?
|
|
135
|
+
stop_after_batch = true
|
|
136
|
+
break
|
|
137
|
+
end
|
|
138
|
+
batch << queued
|
|
139
|
+
rescue ThreadError
|
|
140
|
+
remaining = deadline - monotonic_now
|
|
141
|
+
sleep([remaining, 0.005].min) if remaining.positive?
|
|
142
|
+
end
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
publish_batch_sync(batch)
|
|
146
|
+
complete_pending(batch.length)
|
|
147
|
+
break if stop_after_batch
|
|
98
148
|
end
|
|
99
149
|
rescue StandardError => e
|
|
100
150
|
@configuration.logger.warn("logister worker crashed: #{e.class} #{e.message}")
|
|
@@ -110,8 +160,8 @@ module Logister
|
|
|
110
160
|
attempts += 1
|
|
111
161
|
send_request(payload)
|
|
112
162
|
rescue StandardError => e
|
|
113
|
-
if attempts <= @configuration.max_retries
|
|
114
|
-
sleep(
|
|
163
|
+
if attempts <= @configuration.max_retries && retryable_error?(e)
|
|
164
|
+
sleep(retry_delay(e, attempts))
|
|
115
165
|
retry
|
|
116
166
|
end
|
|
117
167
|
|
|
@@ -120,14 +170,46 @@ module Logister
|
|
|
120
170
|
end
|
|
121
171
|
end
|
|
122
172
|
|
|
173
|
+
def publish_batch_sync(payloads)
|
|
174
|
+
attempts = 0
|
|
175
|
+
begin
|
|
176
|
+
attempts += 1
|
|
177
|
+
send_batch_request(payloads)
|
|
178
|
+
rescue UnsupportedBatchEndpoint
|
|
179
|
+
payloads.map { |payload| publish_sync(payload) }.all?
|
|
180
|
+
rescue RequestError => e
|
|
181
|
+
if e.status == 413 && payloads.length > 1
|
|
182
|
+
middle = (payloads.length / 2.0).ceil
|
|
183
|
+
first_half_delivered = publish_batch_sync(payloads.first(middle))
|
|
184
|
+
second_half_delivered = publish_batch_sync(payloads.drop(middle))
|
|
185
|
+
return first_half_delivered && second_half_delivered
|
|
186
|
+
end
|
|
187
|
+
if attempts <= @configuration.max_retries && retryable_error?(e)
|
|
188
|
+
sleep(retry_delay(e, attempts))
|
|
189
|
+
retry
|
|
190
|
+
end
|
|
191
|
+
|
|
192
|
+
@configuration.logger.warn("logister batch publish failed: #{e.class} #{e.message}")
|
|
193
|
+
false
|
|
194
|
+
rescue StandardError => e
|
|
195
|
+
if attempts <= @configuration.max_retries && retryable_error?(e)
|
|
196
|
+
sleep(retry_delay(e, attempts))
|
|
197
|
+
retry
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
@configuration.logger.warn("logister batch publish failed: #{e.class} #{e.message}")
|
|
201
|
+
false
|
|
202
|
+
end
|
|
203
|
+
end
|
|
204
|
+
|
|
123
205
|
def publish_deployment_sync(payload)
|
|
124
206
|
attempts = 0
|
|
125
207
|
begin
|
|
126
208
|
attempts += 1
|
|
127
209
|
send_deployment_request(payload)
|
|
128
210
|
rescue StandardError => e
|
|
129
|
-
if attempts <= @configuration.max_retries
|
|
130
|
-
sleep(
|
|
211
|
+
if attempts <= @configuration.max_retries && retryable_error?(e)
|
|
212
|
+
sleep(retry_delay(e, attempts))
|
|
131
213
|
retry
|
|
132
214
|
end
|
|
133
215
|
|
|
@@ -147,12 +229,41 @@ module Logister
|
|
|
147
229
|
@uri.port,
|
|
148
230
|
use_ssl: @use_ssl,
|
|
149
231
|
open_timeout: @configuration.timeout_seconds,
|
|
150
|
-
read_timeout: @configuration.timeout_seconds
|
|
232
|
+
read_timeout: @configuration.timeout_seconds,
|
|
233
|
+
write_timeout: @configuration.timeout_seconds
|
|
234
|
+
) { |http| http.request(request) }
|
|
235
|
+
|
|
236
|
+
return true if response.is_a?(Net::HTTPSuccess)
|
|
237
|
+
|
|
238
|
+
raise request_error(response)
|
|
239
|
+
end
|
|
240
|
+
|
|
241
|
+
def send_batch_request(payloads)
|
|
242
|
+
request = Net::HTTP::Post.new(@batch_uri)
|
|
243
|
+
request['Content-Type'] = BATCH_CONTENT_TYPE
|
|
244
|
+
request['Authorization'] = @auth_header
|
|
245
|
+
request['X-Logister-Batch-Id'] = batch_id(payloads)
|
|
246
|
+
|
|
247
|
+
body = payloads.map { |payload| { event: payload }.to_json }.join("\n") << "\n"
|
|
248
|
+
if @configuration.batch_compression
|
|
249
|
+
request['Content-Encoding'] = 'gzip'
|
|
250
|
+
body = Zlib.gzip(body)
|
|
251
|
+
end
|
|
252
|
+
request.body = body
|
|
253
|
+
|
|
254
|
+
response = Net::HTTP.start(
|
|
255
|
+
@batch_uri.host,
|
|
256
|
+
@batch_uri.port,
|
|
257
|
+
use_ssl: @batch_use_ssl,
|
|
258
|
+
open_timeout: @configuration.timeout_seconds,
|
|
259
|
+
read_timeout: @configuration.timeout_seconds,
|
|
260
|
+
write_timeout: @configuration.timeout_seconds
|
|
151
261
|
) { |http| http.request(request) }
|
|
152
262
|
|
|
153
263
|
return true if response.is_a?(Net::HTTPSuccess)
|
|
264
|
+
raise UnsupportedBatchEndpoint, "HTTP #{response.code}" if UNSUPPORTED_BATCH_STATUSES.include?(response.code)
|
|
154
265
|
|
|
155
|
-
raise
|
|
266
|
+
raise request_error(response)
|
|
156
267
|
end
|
|
157
268
|
|
|
158
269
|
def send_deployment_request(payload)
|
|
@@ -166,18 +277,99 @@ module Logister
|
|
|
166
277
|
@deployment_uri.port,
|
|
167
278
|
use_ssl: @deployment_use_ssl,
|
|
168
279
|
open_timeout: @configuration.timeout_seconds,
|
|
169
|
-
read_timeout: @configuration.timeout_seconds
|
|
280
|
+
read_timeout: @configuration.timeout_seconds,
|
|
281
|
+
write_timeout: @configuration.timeout_seconds
|
|
170
282
|
) { |http| http.request(request) }
|
|
171
283
|
|
|
172
284
|
return true if response.is_a?(Net::HTTPSuccess)
|
|
173
285
|
|
|
174
|
-
raise
|
|
286
|
+
raise request_error(response)
|
|
175
287
|
end
|
|
176
288
|
|
|
177
289
|
def monotonic_now
|
|
178
290
|
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
179
291
|
end
|
|
180
292
|
|
|
293
|
+
def with_stable_uuid(payload)
|
|
294
|
+
attributes = payload.to_h.dup
|
|
295
|
+
key = attributes.keys.any? { |candidate| candidate.is_a?(String) } ? 'uuid' : :uuid
|
|
296
|
+
uuid = [attributes[:uuid], attributes['uuid']].find { |value| !blank_identifier?(value) }
|
|
297
|
+
event_id = [attributes[:event_id], attributes['event_id']].find { |value| !blank_identifier?(value) }
|
|
298
|
+
|
|
299
|
+
attributes.delete(:uuid)
|
|
300
|
+
attributes.delete('uuid')
|
|
301
|
+
attributes.delete(:event_id) if blank_identifier?(attributes[:event_id])
|
|
302
|
+
attributes.delete('event_id') if blank_identifier?(attributes['event_id'])
|
|
303
|
+
attributes[key] = uuid || event_id || SecureRandom.uuid
|
|
304
|
+
attributes
|
|
305
|
+
end
|
|
306
|
+
|
|
307
|
+
def blank_identifier?(value)
|
|
308
|
+
value.nil? || value.to_s.strip.empty?
|
|
309
|
+
end
|
|
310
|
+
|
|
311
|
+
def batch_id(payloads)
|
|
312
|
+
identifiers = payloads.map do |payload|
|
|
313
|
+
payload[:uuid] || payload['uuid'] || payload[:event_id] || payload['event_id']
|
|
314
|
+
end
|
|
315
|
+
Digest::SHA256.hexdigest(identifiers.join("\n"))
|
|
316
|
+
end
|
|
317
|
+
|
|
318
|
+
def batch_size
|
|
319
|
+
[@configuration.batch_size.to_i, 1].max
|
|
320
|
+
end
|
|
321
|
+
|
|
322
|
+
def batch_interval
|
|
323
|
+
[@configuration.batch_interval.to_f, 0.0].max
|
|
324
|
+
end
|
|
325
|
+
|
|
326
|
+
def increment_pending
|
|
327
|
+
@pending_mutex.synchronize { @pending_count += 1 }
|
|
328
|
+
end
|
|
329
|
+
|
|
330
|
+
def complete_pending(count)
|
|
331
|
+
@pending_mutex.synchronize do
|
|
332
|
+
@pending_count = [@pending_count - count, 0].max
|
|
333
|
+
@pending_condition.broadcast if @pending_count.zero?
|
|
334
|
+
end
|
|
335
|
+
end
|
|
336
|
+
|
|
337
|
+
def retryable_error?(error)
|
|
338
|
+
return true unless error.is_a?(RequestError)
|
|
339
|
+
|
|
340
|
+
[408, 425, 429].include?(error.status) || error.status >= 500
|
|
341
|
+
end
|
|
342
|
+
|
|
343
|
+
def request_error(response)
|
|
344
|
+
RequestError.new(response.code, retry_after: retry_after_seconds(response['Retry-After']))
|
|
345
|
+
end
|
|
346
|
+
|
|
347
|
+
def retry_after_seconds(value)
|
|
348
|
+
header = value.to_s.strip
|
|
349
|
+
return nil if header.empty?
|
|
350
|
+
|
|
351
|
+
seconds = Float(header, exception: false)
|
|
352
|
+
return [seconds, 0.0].max if seconds&.finite?
|
|
353
|
+
|
|
354
|
+
[Time.httpdate(header) - Time.now, 0.0].max
|
|
355
|
+
rescue ArgumentError
|
|
356
|
+
nil
|
|
357
|
+
end
|
|
358
|
+
|
|
359
|
+
def retry_delay(error, attempt)
|
|
360
|
+
configured_cap = [@configuration.max_retry_delay.to_f, 0.0].max
|
|
361
|
+
base_delay = if error.is_a?(RequestError) && !error.retry_after.nil?
|
|
362
|
+
error.retry_after.to_f
|
|
363
|
+
else
|
|
364
|
+
@configuration.retry_base_interval.to_f * (2**(attempt - 1))
|
|
365
|
+
end
|
|
366
|
+
bounded_delay = [[base_delay, 0.0].max, configured_cap].min
|
|
367
|
+
jitter_ratio = @configuration.retry_jitter.to_f.clamp(0.0, 1.0)
|
|
368
|
+
jitter = bounded_delay * jitter_ratio * rand
|
|
369
|
+
|
|
370
|
+
[bounded_delay + jitter, configured_cap].min
|
|
371
|
+
end
|
|
372
|
+
|
|
181
373
|
def ready?
|
|
182
374
|
@configuration.enabled && !@configuration.api_key.to_s.empty?
|
|
183
375
|
end
|
|
@@ -6,11 +6,12 @@ module Logister
|
|
|
6
6
|
:repository, :commit_sha, :branch, :enabled, :timeout_seconds, :logger,
|
|
7
7
|
:ignore_exceptions, :ignore_environments, :ignore_paths, :before_notify,
|
|
8
8
|
:async, :queue_size, :max_retries, :retry_base_interval,
|
|
9
|
+
:max_retry_delay, :retry_jitter, :batch_size, :batch_interval, :batch_compression,
|
|
9
10
|
:capture_db_metrics, :db_metric_min_duration_ms, :db_metric_sample_rate,
|
|
10
11
|
:feature_flags_resolver, :dependency_resolver, :anonymize_ip,
|
|
11
12
|
:max_breadcrumbs, :max_dependencies, :capture_request_spans,
|
|
12
13
|
:capture_sql_breadcrumbs, :sql_breadcrumb_min_duration_ms
|
|
13
|
-
attr_writer :deployment_endpoint
|
|
14
|
+
attr_writer :deployment_endpoint, :batch_endpoint
|
|
14
15
|
|
|
15
16
|
def initialize
|
|
16
17
|
@api_key = ENV['LOGISTER_API_KEY']
|
|
@@ -36,6 +37,11 @@ module Logister
|
|
|
36
37
|
@queue_size = 1000
|
|
37
38
|
@max_retries = 3
|
|
38
39
|
@retry_base_interval = 0.5
|
|
40
|
+
@max_retry_delay = 30.0
|
|
41
|
+
@retry_jitter = 0.2
|
|
42
|
+
@batch_size = 50
|
|
43
|
+
@batch_interval = 0.05
|
|
44
|
+
@batch_compression = true
|
|
39
45
|
|
|
40
46
|
@capture_db_metrics = false
|
|
41
47
|
@db_metric_min_duration_ms = 0.0
|
|
@@ -55,6 +61,10 @@ module Logister
|
|
|
55
61
|
@deployment_endpoint || endpoint.to_s.sub(%r{/ingest_events\z}, '/deployments')
|
|
56
62
|
end
|
|
57
63
|
|
|
64
|
+
def batch_endpoint
|
|
65
|
+
@batch_endpoint || endpoint.to_s.sub(%r{/ingest_events\z}, '/ingest_events/batch')
|
|
66
|
+
end
|
|
67
|
+
|
|
58
68
|
private
|
|
59
69
|
|
|
60
70
|
def env_value(name)
|
data/lib/logister/railtie.rb
CHANGED
|
@@ -20,8 +20,13 @@ module Logister
|
|
|
20
20
|
copy_setting(app, config, :before_notify)
|
|
21
21
|
copy_setting(app, config, :async)
|
|
22
22
|
copy_setting(app, config, :queue_size)
|
|
23
|
+
copy_setting(app, config, :batch_size)
|
|
24
|
+
copy_setting(app, config, :batch_interval)
|
|
25
|
+
copy_setting(app, config, :batch_compression)
|
|
23
26
|
copy_setting(app, config, :max_retries)
|
|
24
27
|
copy_setting(app, config, :retry_base_interval)
|
|
28
|
+
copy_setting(app, config, :max_retry_delay)
|
|
29
|
+
copy_setting(app, config, :retry_jitter)
|
|
25
30
|
copy_setting(app, config, :capture_db_metrics)
|
|
26
31
|
copy_setting(app, config, :db_metric_min_duration_ms)
|
|
27
32
|
copy_setting(app, config, :db_metric_sample_rate)
|
data/lib/logister/reporter.rb
CHANGED
|
@@ -44,6 +44,7 @@ module Logister
|
|
|
44
44
|
end
|
|
45
45
|
|
|
46
46
|
def report_error(exception, context: {}, tags: {}, level: 'error', fingerprint: nil)
|
|
47
|
+
return false if Logister.reporting_suppressed?
|
|
47
48
|
return false if ignored_exception?(exception)
|
|
48
49
|
return false if ignored_path?(context)
|
|
49
50
|
|
|
@@ -69,6 +70,7 @@ module Logister
|
|
|
69
70
|
end
|
|
70
71
|
|
|
71
72
|
def report_metric(message:, value: nil, unit: nil, level: 'info', context: {}, tags: {}, fingerprint: nil)
|
|
73
|
+
return false if Logister.reporting_suppressed?
|
|
72
74
|
return false if ignored_environment?
|
|
73
75
|
return false if ignored_path?(context)
|
|
74
76
|
|
|
@@ -100,6 +102,7 @@ module Logister
|
|
|
100
102
|
end
|
|
101
103
|
|
|
102
104
|
def report_transaction(name:, duration_ms:, level: 'info', context: {}, tags: {}, fingerprint: nil, status: nil)
|
|
105
|
+
return false if Logister.reporting_suppressed?
|
|
103
106
|
return false if ignored_environment?
|
|
104
107
|
return false if ignored_path?(context)
|
|
105
108
|
|
|
@@ -137,6 +140,7 @@ module Logister
|
|
|
137
140
|
tags: {},
|
|
138
141
|
fingerprint: nil
|
|
139
142
|
)
|
|
143
|
+
return false if Logister.reporting_suppressed?
|
|
140
144
|
return false if ignored_environment?
|
|
141
145
|
return false if ignored_path?(context)
|
|
142
146
|
|
|
@@ -182,6 +186,7 @@ module Logister
|
|
|
182
186
|
end
|
|
183
187
|
|
|
184
188
|
def report_log(message:, level: 'info', context: {}, tags: {}, fingerprint: nil)
|
|
189
|
+
return false if Logister.reporting_suppressed?
|
|
185
190
|
return false if ignored_environment?
|
|
186
191
|
return false if ignored_path?(context)
|
|
187
192
|
|
|
@@ -212,6 +217,7 @@ module Logister
|
|
|
212
217
|
trace_id: nil,
|
|
213
218
|
request_id: nil
|
|
214
219
|
)
|
|
220
|
+
return false if Logister.reporting_suppressed?
|
|
215
221
|
return false if ignored_environment?
|
|
216
222
|
|
|
217
223
|
payload = build_payload(
|
|
@@ -253,6 +259,8 @@ module Logister
|
|
|
253
259
|
workflow_run_url: nil,
|
|
254
260
|
deployment_url: nil
|
|
255
261
|
)
|
|
262
|
+
return false if Logister.reporting_suppressed?
|
|
263
|
+
|
|
256
264
|
payload = {
|
|
257
265
|
release: release || @configuration.release,
|
|
258
266
|
environment: environment || @configuration.environment,
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Logister
|
|
4
|
+
module ReportingScope
|
|
5
|
+
STATE_KEY = :__logister_reporting_suppressed
|
|
6
|
+
|
|
7
|
+
module_function
|
|
8
|
+
|
|
9
|
+
def suppress
|
|
10
|
+
previous = state
|
|
11
|
+
self.state = true
|
|
12
|
+
yield
|
|
13
|
+
ensure
|
|
14
|
+
self.state = previous
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def suppressed?
|
|
18
|
+
state == true
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def state
|
|
22
|
+
if defined?(ActiveSupport::IsolatedExecutionState)
|
|
23
|
+
ActiveSupport::IsolatedExecutionState[STATE_KEY]
|
|
24
|
+
else
|
|
25
|
+
Thread.current[STATE_KEY]
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
private_class_method :state
|
|
29
|
+
|
|
30
|
+
def state=(value)
|
|
31
|
+
if defined?(ActiveSupport::IsolatedExecutionState)
|
|
32
|
+
ActiveSupport::IsolatedExecutionState[STATE_KEY] = value
|
|
33
|
+
else
|
|
34
|
+
Thread.current[STATE_KEY] = value
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
private_class_method :state=
|
|
38
|
+
end
|
|
39
|
+
end
|
|
@@ -22,6 +22,7 @@ module Logister
|
|
|
22
22
|
private
|
|
23
23
|
|
|
24
24
|
def handle_process_action(payload)
|
|
25
|
+
return if Logister.reporting_suppressed?
|
|
25
26
|
return unless payload.is_a?(Hash)
|
|
26
27
|
|
|
27
28
|
request_id = payload[:request_id].to_s.presence
|
|
@@ -89,6 +90,8 @@ module Logister
|
|
|
89
90
|
end
|
|
90
91
|
|
|
91
92
|
def handle_sql_breadcrumb(started, finished, payload)
|
|
93
|
+
return if Logister.reporting_suppressed?
|
|
94
|
+
|
|
92
95
|
config = configuration
|
|
93
96
|
return unless config&.capture_sql_breadcrumbs
|
|
94
97
|
return unless payload.is_a?(Hash)
|
data/lib/logister/version.rb
CHANGED
data/lib/logister.rb
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
require_relative 'logister/version'
|
|
2
2
|
require_relative 'logister/configuration'
|
|
3
|
+
require_relative 'logister/reporting_scope'
|
|
3
4
|
require_relative 'logister/client'
|
|
4
5
|
require_relative 'logister/reporter'
|
|
5
6
|
require_relative 'logister/context_helpers'
|
|
@@ -23,6 +24,14 @@ module Logister
|
|
|
23
24
|
@reporter ||= Reporter.new(configuration)
|
|
24
25
|
end
|
|
25
26
|
|
|
27
|
+
def suppress_reporting(&block)
|
|
28
|
+
ReportingScope.suppress(&block)
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def reporting_suppressed?
|
|
32
|
+
ReportingScope.suppressed?
|
|
33
|
+
end
|
|
34
|
+
|
|
26
35
|
def report_error(exception, **kwargs)
|
|
27
36
|
reporter.report_error(exception, **kwargs)
|
|
28
37
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: logister-ruby
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Logister
|
|
@@ -86,6 +86,7 @@ files:
|
|
|
86
86
|
- lib/logister/middleware.rb
|
|
87
87
|
- lib/logister/railtie.rb
|
|
88
88
|
- lib/logister/reporter.rb
|
|
89
|
+
- lib/logister/reporting_scope.rb
|
|
89
90
|
- lib/logister/request_subscriber.rb
|
|
90
91
|
- lib/logister/sql_subscriber.rb
|
|
91
92
|
- lib/logister/version.rb
|