zharp 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 58fd6ce97a15cf4ba775b28da313a421cc5809e145ab7ba7f744034b9589e052
4
+ data.tar.gz: fd36fde29b5fe7681a8233493f037b117aa1d7e08af5beca0b6ed6531d669905
5
+ SHA512:
6
+ metadata.gz: f1871fd7081b1815a909bf83699df5a9ba6576f687cdfb0a4c752f3d751d8fce303999a0d2ab1b2fab238f58c775cbf701d4ca8b0b4daeac4dc4b71d30795587
7
+ data.tar.gz: 860e2dce601c5de97655001888854f3bfb676d00198cbc4becc8bdbd0c58a4e34e1620d5daa57456e53973ee3944099f19c80535c249a8b3468e2e63e8b21374
data/CHANGELOG.md ADDED
@@ -0,0 +1,33 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 (unreleased)
4
+
5
+ The first release: one `zharp` gem for Rails, Rack apps, Sidekiq and plain Ruby, built on the
6
+ official OpenTelemetry SDK (`opentelemetry-sdk` 1.13) and its instrumentations for Rack, Rails,
7
+ Sinatra, Grape, Net::HTTP, Faraday, HTTP.rb, HTTPX, pg, mysql2, trilogy, Redis, Sidekiq, Resque and
8
+ Delayed Job.
9
+ Ruby 3.3+, Rails 7.1+.
10
+
11
+ - Rails starts it by itself (a Railtie); `bin/rails generate zharp:install` writes
12
+ `config/initializers/zharp.rb` and can store the key in the credentials.
13
+ - Rack apps: `use Zharp::Rack`. Plain Ruby: `Zharp.init`. Sidekiq outside Rails: `require "zharp/sidekiq"` in the file Sidekiq loads.
14
+ - Settings from `ZHARP_KEY`, `ZHARP_SERVICE`, `ZHARP_ENVIRONMENT`, `ZHARP_RELEASE`,
15
+ `ZHARP_ENDPOINT`, `ZHARP_SAMPLE_RATE`, `ZHARP_LOGS`, `ZHARP_LOG_CORRELATION`,
16
+ `ZHARP_CAPTURE_HEADERS`, `ZHARP_CAPTURE_QUERY_ERRORS`, `ZHARP_CAPTURE_SOURCE_CONTEXT`,
17
+ `ZHARP_TRACING`, `ZHARP_DISABLED`; in Rails also `credentials.zharp.key`.
18
+ - Errors counted once (Action Pack, `Rails.error`, Rack, Sidekiq, Active Job, `at_exit`), with the
19
+ code around the app's frames as `zharp.source_context`.
20
+ - All request and response headers on the server span, secrets as `[REDACTED]`.
21
+ - Failed queries (pg, mysql2, trilogy, Active Record) reported once with system, database,
22
+ obfuscated statement and error code; Postgres's `DETAIL:` line dropped.
23
+ - Sidekiq and Active Job jobs in the trace of the request that queued them.
24
+ - ` trace_id=… span_id=…` at the end of each `Logger` and semantic_logger line in a request;
25
+ `ZHARP_LOGS=true` sends the lines from the SDK.
26
+ - `capture_bodies` (off by default): request and response bodies on Rack and Net::HTTP spans,
27
+ secrets filtered, files and binary bodies described, 32 KB cap. Secret query values are masked
28
+ in every span's URL.
29
+ - Database error messages carry no values: Postgres's `DETAIL:` line, MySQL's `Duplicate entry
30
+ '…'`, invalid-input values.
31
+ - Fork safety for Puma cluster mode and Unicorn: each worker's exporters get their own connection.
32
+ - `Zharp.capture_error`, `set_user`, `clear_user`, `set_tag`, `log`, `start_span`, `flush`,
33
+ `shutdown`.
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Zharp
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,170 @@
1
+ # Zharp for Ruby
2
+
3
+ Application Monitoring for Ruby apps: traces, errors and logs, sent to [Zharp](https://zharp.io).
4
+ One gem for Rails, any Rack app (Sinatra, Hanami, Roda, Grape), Sidekiq and plain Ruby, built on
5
+ the official OpenTelemetry SDK.
6
+
7
+ - Requests named by route, continuing the caller's `traceparent`; database queries (pg, mysql2,
8
+ trilogy, Active Record), outgoing HTTP (Net::HTTP, Faraday, HTTP.rb, HTTPX), Redis, jobs
9
+ (Sidekiq, Active Job, Resque, Delayed Job), Action View and Action Mailer, each switched on when
10
+ its library is loaded. Anything else you instrument with OpenTelemetry is sent too.
11
+ - **Errors, each counted once**, with type, message, the stack and the code that ran (the lines
12
+ around your app's innermost five frames, read on your server; never gems or Ruby's library).
13
+ - **Request and response headers** on the request's span, secrets (authorization, cookies, API
14
+ keys, tokens, sessions) as `[REDACTED]`.
15
+ - **Failed queries** reported once, with the statement (obfuscated), the database and the error
16
+ code; Postgres's `DETAIL:` line (the row's values) is dropped.
17
+ - **Jobs in the request's trace**: Sidekiq and Active Job (Solid Queue, GoodJob, Delayed Job…) run
18
+ as children of the request that queued them.
19
+ - **Logs**: each line written through `Logger` (so `Rails.logger` and lograge) or semantic_logger
20
+ during a request ends with ` trace_id=… span_id=…` (two fields on a JSON line), so the Zharp
21
+ agent links it to its request. `ZHARP_LOGS=true` sends the lines from the SDK instead, where
22
+ there is no agent.
23
+ - **Forking servers**: Puma in cluster mode (`preload_app!`) and Unicorn need nothing.
24
+
25
+ Ruby 3.3+ (3.3, 3.4, 4.0), Rails 7.1+.
26
+
27
+ ## Rails
28
+
29
+ ```sh
30
+ bundle add zharp
31
+ ```
32
+
33
+ Set your key, from **Settings → Applications** in Zharp:
34
+
35
+ ```sh
36
+ ZHARP_KEY=zpk_… bin/rails server
37
+ ```
38
+
39
+ or keep it in the credentials (`bin/rails credentials:edit`):
40
+
41
+ ```yaml
42
+ zharp:
43
+ key: zpk_…
44
+ ```
45
+
46
+ That's all: the gem's Railtie starts Zharp when Rails boots, in the web server, `bin/rails runner`,
47
+ Sidekiq and Solid Queue alike. To change a default, generate the initializer, which lists every
48
+ option commented out (with `--key` it also stores the key in the credentials):
49
+
50
+ ```sh
51
+ bin/rails generate zharp:install --key=zpk_…
52
+ ```
53
+
54
+ Puma in cluster mode with `preload_app!` prints "Detected 2 Thread(s) started in app boot": those
55
+ are OpenTelemetry's export threads in the master process. Each worker starts its own after the fork
56
+ and sends over a connection of its own; nothing needs changing.
57
+
58
+ ## Rack apps: Sinatra, Hanami, Roda, Grape, plain Rack
59
+
60
+ ```sh
61
+ bundle add zharp
62
+ ```
63
+
64
+ In `config.ru`:
65
+
66
+ ```ruby
67
+ require "zharp"
68
+ use Zharp::Rack
69
+
70
+ run App
71
+ ```
72
+
73
+ and `ZHARP_KEY` in the environment. Sinatra and Grape spans are named by their routes; others by
74
+ the path with ids replaced (`GET /orders/:id`). Call `Zharp.init(service: "shop")` before `use`
75
+ to set options in code.
76
+
77
+ ## Plain Ruby: scripts, workers, CLIs
78
+
79
+ ```ruby
80
+ require "zharp"
81
+ Zharp.init(service: "billing-worker")
82
+ ```
83
+
84
+ Call it after your gems are required, so their instrumentation is switched on. An exception that
85
+ ends the script is reported and sent before it exits.
86
+
87
+ ### Sidekiq outside Rails
88
+
89
+ At the top of the file Sidekiq loads (`bundle exec sidekiq -r ./worker.rb`), with `ZHARP_KEY` in
90
+ the environment:
91
+
92
+ ```ruby
93
+ require "zharp/sidekiq"
94
+ ```
95
+
96
+ Sidekiq keeps only one `-r`, so this goes in that file rather than on the command line. Jobs queued
97
+ from a traced request or span run in its trace. (In a Rails app Sidekiq needs nothing.)
98
+
99
+ ## Settings
100
+
101
+ Every setting can be passed to `Zharp.init` or set in the environment.
102
+
103
+ | `Zharp.init` | Environment | Default |
104
+ |---|---|---|
105
+ | `key:` | `ZHARP_KEY` (Rails: then `credentials.zharp.key`) | — nothing is sent without one |
106
+ | `service:` | `ZHARP_SERVICE`, `OTEL_SERVICE_NAME` | the Rails app's name, else the project folder |
107
+ | `environment:` | `ZHARP_ENVIRONMENT` | `Rails.env` |
108
+ | `release:` | `ZHARP_RELEASE` | — |
109
+ | `endpoint:` | `ZHARP_ENDPOINT` | `https://api.zharp.io/api/v1/otlp` |
110
+ | `sample_rate:` | `ZHARP_SAMPLE_RATE` | `1.0` (errors are always sent) |
111
+ | `logs:` | `ZHARP_LOGS` | `false` |
112
+ | `log_correlation:` | `ZHARP_LOG_CORRELATION` | `true` |
113
+ | `capture_headers:` | `ZHARP_CAPTURE_HEADERS` | on; `{ redact: ["x-tenant"] }` filters more names |
114
+ | `capture_query_errors:` | `ZHARP_CAPTURE_QUERY_ERRORS` | `true` |
115
+ | `capture_source_context:` | `ZHARP_CAPTURE_SOURCE_CONTEXT` | `true` |
116
+ | `capture_bodies:` | `ZHARP_CAPTURE_BODIES` (`errors`, `all`, `off`) | off |
117
+ | `tracing:` | `ZHARP_TRACING` | `true`; `false` sends errors and logs only |
118
+ | `instrumentation:` | — | options per OpenTelemetry instrumentation, by class name |
119
+ | — | `ZHARP_DISABLED=true` | turns everything off |
120
+
121
+ ### Request and response bodies
122
+
123
+ Off by default. `capture_bodies: "errors"` (or `true`) keeps the bodies of failed requests (status
124
+ 400 and above, or no response), `"all"` of every request: incoming requests (any Rack app, Rails
125
+ included) and outgoing Net::HTTP calls (Faraday's default adapter too).
126
+
127
+ ```ruby
128
+ Zharp.init(capture_bodies: { mode: "errors", max_size: 32_768, redact_keys: ["date_of_birth"], urls: ["api.example.com"] })
129
+ ```
130
+
131
+ Text is capped at 32 KB (`max_size`, at most 128 KB). Files in uploads, inline images and binary
132
+ bodies are described (name, type, size), never sent. Field names are kept; values under
133
+ secret-looking keys (`password`, `token`, `authorization`, `card`…) and `redact_keys` become
134
+ `[Filtered]` before anything is sent. `urls` limits it to some requests (a host, a URL prefix or a
135
+ Regexp). Secret-looking query values in URLs are masked on every span either way.
136
+
137
+ ## API
138
+
139
+ ```ruby
140
+ Zharp.capture_error(error, order_id: 42) # an exception you rescued, as a handled error
141
+ Zharp.set_user(id: user.id, email: user.email) # on the request's spans, logs and errors
142
+ Zharp.set_tag("tenant", "acme") # on every span and log from now on
143
+ Zharp.log.info("order paid", order_id: 42) # a log line sent from the SDK, whatever ZHARP_LOGS says
144
+ Zharp.start_span("render invoice") { render } # a span around some work
145
+ Zharp.flush # send what's queued now (end of a script or job)
146
+ ```
147
+
148
+ Rails' own `Rails.error.report` and `Rails.error.handle` reach Zharp too.
149
+
150
+ A browser or mobile app using a Zharp SDK sends its user and session with each request (W3C
151
+ `baggage`); they are put on the request's spans, logs and errors without any code.
152
+
153
+ ## Development
154
+
155
+ No Ruby needed on the host: everything runs in Docker.
156
+
157
+ ```sh
158
+ docker run --rm -v "$PWD:/src" -w /src ruby:3.4 sh -c "gem install bundler && bundle install && bundle exec rake"
159
+ e2e/unit.sh # unit tests: Ruby 3.3 / 3.4 / 4.0 × no Rails, Rails 7.1 / 7.2 / 8.1
160
+ e2e/generator.sh 3.4 8.1.0 # rails new, bundle add zharp, the generator, boot
161
+ # End-to-end against a Zharp backend: a throwaway app per case, its docker compose stack, what
162
+ # Zharp stored saved in e2e/out/<case>.json. Cases: rails81 rails71 sinatra rack hanami script semantic.
163
+ ZHARP_API=http://localhost:3000/api/v1 ZHARP_TOKEN=<user JWT> ZHARP_WORKSPACE=<id> node e2e/run.mjs rails81
164
+ ```
165
+
166
+ `e2e/bundler-check.sh` times `bundle add zharp` in fresh Rails apps with Bundler 2.6, 2.7 and 4.
167
+
168
+ ## License
169
+
170
+ MIT
@@ -0,0 +1,39 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "rails/generators"
4
+
5
+ module Zharp
6
+ module Generators
7
+ # bin/rails generate zharp:install [--key=zpk_…]
8
+ #
9
+ # Writes config/initializers/zharp.rb (every option there, commented out) and, given a key,
10
+ # stores it in the credentials as zharp.key. Without one it says how to set ZHARP_KEY.
11
+ class InstallGenerator < ::Rails::Generators::Base
12
+ source_root File.expand_path("templates", __dir__)
13
+
14
+ class_option :key, type: :string, desc: "Your Zharp ingest key (zpk_…), stored in the credentials as zharp.key"
15
+
16
+ def create_initializer
17
+ template "zharp.rb.tt", "config/initializers/zharp.rb"
18
+ end
19
+
20
+ def store_key
21
+ key = options[:key].to_s.strip
22
+ if key.empty?
23
+ say "Set your key: ZHARP_KEY=zpk_… in the environment, or bin/rails credentials:edit and add\n zharp:\n key: zpk_…", :yellow
24
+ return
25
+ end
26
+ credentials = ::Rails.application.credentials
27
+ content = credentials.read.to_s
28
+ if content.match?(/^zharp:\s*$/)
29
+ say "credentials already have a zharp: section; left as it is", :yellow
30
+ return
31
+ end
32
+ credentials.write("#{content.chomp}\n\nzharp:\n key: #{key}\n".lstrip)
33
+ say_status :credentials, "zharp.key added (#{credentials.content_path.basename})"
34
+ rescue StandardError => e
35
+ say "Could not write the credentials (#{e.message}). Set ZHARP_KEY=#{key} in the environment instead.", :red
36
+ end
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,15 @@
1
+ # Zharp Application Monitoring. Rails starts it without this file: what's here only changes the
2
+ # defaults. The key comes from ZHARP_KEY, then credentials (zharp.key).
3
+ Zharp.init(
4
+ <%= %( # service: "#{Rails.application.class.name.to_s.split("::").first.to_s.gsub(/([a-z\d])([A-Z])/, '\1-\2').downcase}",).ljust(37) %># ZHARP_SERVICE
5
+ # environment: Rails.env, # ZHARP_ENVIRONMENT
6
+ # release: ENV["GIT_SHA"], # ZHARP_RELEASE: the version you deploy, for "new in this release"
7
+ # sample_rate: 1.0, # ZHARP_SAMPLE_RATE: share of traces kept; errors are always sent
8
+ # logs: false, # ZHARP_LOGS: send log lines from here (off: the Zharp agent reads them)
9
+ # log_correlation: true, # ZHARP_LOG_CORRELATION: " trace_id=… span_id=…" at the end of each line
10
+ # capture_headers: { redact: ["x-tenant"] }, # more header names to filter; false turns headers off
11
+ # capture_query_errors: true, # a failed query is reported once, with its statement
12
+ # capture_source_context: true, # the code around each error's lines is sent with it
13
+ # capture_bodies: "errors", # ZHARP_CAPTURE_BODIES: bodies of failed requests ("all": every one)
14
+ # instrumentation: { "OpenTelemetry::Instrumentation::Sidekiq" => { span_naming: :job_class } },
15
+ )
@@ -0,0 +1,386 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+ require "uri"
5
+ require "bigdecimal"
6
+
7
+ module Zharp
8
+ # Request and response bodies on HTTP spans (capture_bodies), and secrets in URLs.
9
+ #
10
+ # A port of the rules every Zharp SDK shares (@zharp/internal/bodies, zharp-python's _bodies.py),
11
+ # so a payload looks the same whichever SDK sent it:
12
+ # - field names stay; values under keys that look secret (password, token, authorization, card…)
13
+ # become [Filtered], and an object under such a key keeps its keys with every value filtered;
14
+ # - an inline image or other long base64 value becomes [image/png · 24 KB], a file in a form
15
+ # upload [file photo.png · image/png · 240 KB], and a body that isn't text at all
16
+ # [binary · application/pdf · 1.2 MB] or [event stream];
17
+ # - text is capped (32 KB by default), with http.*.body.truncated set when it was cut.
18
+ module Bodies
19
+ DEFAULT_MAX_BODY = 32 * 1024
20
+ MAX_BODY = 128 * 1024
21
+ FILTERED = "[Filtered]"
22
+
23
+ REQUEST_BODY = "http.request.body"
24
+ REQUEST_TRUNCATED = "http.request.body.truncated"
25
+ REQUEST_TYPE = "http.request.header.content-type"
26
+ RESPONSE_BODY = "http.response.body"
27
+ RESPONSE_TRUNCATED = "http.response.body.truncated"
28
+ RESPONSE_TYPE = "http.response.header.content-type"
29
+ ATTRS = {
30
+ "request" => [REQUEST_BODY, REQUEST_TRUNCATED, REQUEST_TYPE],
31
+ "response" => [RESPONSE_BODY, RESPONSE_TRUNCATED, RESPONSE_TYPE]
32
+ }.freeze
33
+
34
+ # mode: "errors" (status 400 and above, or no response) or "all"; max_size: characters kept;
35
+ # redact_keys: more secret keys (a String matches ignoring case, - and _; a Regexp is searched
36
+ # in the key); urls: which requests (a bare host, a URL prefix, or a Regexp), nil for all.
37
+ Settings = Struct.new(:mode, :max_size, :redact_keys, :urls, keyword_init: true)
38
+
39
+ module_function
40
+
41
+ # capture_bodies: true, "errors", "all", false/"off", or a Hash with mode, max_size,
42
+ # redact_keys, urls. ZHARP_CAPTURE_BODIES (errors, all, off) when nothing is passed. Off by default.
43
+ def settings(option, env_value)
44
+ value = option
45
+ if value.nil? && env_value
46
+ text = env_value.to_s.strip.downcase
47
+ value = if %w[errors all].include?(text) then text
48
+ elsif Config::TRUE.match?(text) then true
49
+ end
50
+ end
51
+ return nil if value.nil? || value == false
52
+
53
+ value = { mode: value } if value.is_a?(String) || value.is_a?(Symbol)
54
+ value = {} if value == true
55
+ return nil unless value.is_a?(Hash)
56
+
57
+ value = value.transform_keys(&:to_sym)
58
+ mode = value[:mode].to_s.strip.downcase
59
+ return nil if ["", "off", "false", "no", "none", "0"].include?(mode) && value.key?(:mode)
60
+
61
+ size = value[:max_size]
62
+ size = size.is_a?(Numeric) && size.positive? ? [size.floor, MAX_BODY].min : DEFAULT_MAX_BODY
63
+ Settings.new(
64
+ mode: mode == "all" ? "all" : "errors",
65
+ max_size: size,
66
+ redact_keys: Array(value[:redact_keys]),
67
+ urls: value[:urls].nil? ? nil : Array(value[:urls])
68
+ )
69
+ end
70
+
71
+ # Raw text read before shortening and capping: inline images shrink to a few words.
72
+ def raw_limit(settings) = [64 * 1024, settings.max_size * 2].max
73
+
74
+ # Whether a request with this outcome has its bodies kept (no status: no response).
75
+ def wanted?(settings, status) = settings.mode == "all" || status.nil? || status.to_i.zero? || status.to_i >= 400
76
+
77
+ def allowed?(settings, url) = settings.urls.nil? || url_matches?(url.to_s, settings.urls)
78
+
79
+ # ─── content types ───────────────────────────────────────────────────────────
80
+
81
+ def mime(content_type) = content_type.to_s.split(";").first.to_s.strip.downcase
82
+ def event_stream?(content_type) = mime(content_type) == "text/event-stream"
83
+ def multipart?(content_type) = mime(content_type) == "multipart/form-data"
84
+ def form?(content_type) = mime(content_type) == "application/x-www-form-urlencoded"
85
+
86
+ TEXT_APPLICATION = %r{\Aapplication/(json|xml|x-www-form-urlencoded|graphql|javascript|x-ndjson|ld\+json)\z}
87
+
88
+ # Whether a body of this type is text worth showing. An unknown type gets the benefit of the doubt.
89
+ def textual?(content_type)
90
+ kind = mime(content_type)
91
+ return true if kind.empty?
92
+ return false if kind == "text/event-stream"
93
+
94
+ kind.start_with?("text/") || kind.end_with?("+json", "+xml") || TEXT_APPLICATION.match?(kind)
95
+ end
96
+
97
+ # 240 KB, 1.4 MB.
98
+ def format_size(size)
99
+ return "#{size} B" if size < 1024
100
+ return "#{(size / 1024.0 + 0.5).floor} KB" if size < 1024 * 1024
101
+
102
+ "#{format('%.1f', ((size / (1024.0 * 1024)) * 10 + 0.5).floor / 10.0)} MB"
103
+ end
104
+
105
+ # [file photo.png · image/png · 240 KB]
106
+ def file_label(name, content_type, size, partial = false)
107
+ parts = ["file #{name.nil? || name.to_s.empty? ? 'blob' : name}"]
108
+ parts << content_type if content_type && !content_type.to_s.empty?
109
+ parts << "#{format_size(size)}#{partial ? '+' : ''}" unless size.nil?
110
+ "[#{parts.join(' · ')}]"
111
+ end
112
+
113
+ # [binary · image/png · 24 KB], [event stream]
114
+ def describe_body(content_type, size = nil, partial = false)
115
+ return "[event stream]" if event_stream?(content_type)
116
+
117
+ parts = ["binary"]
118
+ kind = mime(content_type)
119
+ parts << kind unless kind.empty?
120
+ parts << "#{format_size(size)}#{partial ? '+' : ''}" if size&.positive?
121
+ "[#{parts.join(' · ')}]"
122
+ end
123
+
124
+ # ─── redaction ───────────────────────────────────────────────────────────────
125
+
126
+ # Built-in keys whose values never leave the process, compared without case, - or _.
127
+ CONTAINS = %w[
128
+ password passwd passphrase secret token apikey accesskey privatekey authorization cookie
129
+ creditcard cardnumber cvv cvc ssn iban sessionid signature credential
130
+ ].freeze
131
+ EXACT = %w[auth pin pwd otp card pan jwt sid codeverifier].freeze
132
+ # Also secret in a query string, where they are API keys and signed-URL signatures.
133
+ URL_EXACT = %w[key sig code state session].freeze
134
+
135
+ def normalize(key) = key.to_s.downcase.gsub(/[^a-z0-9]/, "")
136
+
137
+ def secret_key?(key, extra = [])
138
+ k = normalize(key)
139
+ return false if k.empty?
140
+ return true if EXACT.include?(k) || CONTAINS.any? { |c| k.include?(c) }
141
+
142
+ extra.any? { |pattern| pattern.is_a?(Regexp) ? pattern.match?(key.to_s) : normalize(pattern) == k }
143
+ end
144
+
145
+ DATA_URI = %r{\Adata:([\w.+-]+/[\w.+-]+)?(?:;[\w=.+-]+)*;base64,([A-Za-z0-9+/=\s]*)\z}
146
+ BASE64 = %r{\A[A-Za-z0-9+/_-]{256,}={0,2}\z}
147
+
148
+ # A long base64 value as a short description; other strings as they are.
149
+ def shorten(value)
150
+ return value if value.length < 256
151
+
152
+ if (uri = DATA_URI.match(value))
153
+ return "[#{uri[1] || 'data'} · #{format_size(uri[2].length * 3 / 4)}]"
154
+ end
155
+ return "[base64 · #{format_size(value.length * 3 / 4)}]" if BASE64.match?(value) && value.match?(%r{[0-9+/]})
156
+
157
+ value
158
+ end
159
+
160
+ def filter_all(value, depth)
161
+ return FILTERED if depth > 20
162
+ return value if value.nil? || value == ""
163
+ return value.map { |v| filter_all(v, depth + 1) } if value.is_a?(Array)
164
+ return value.transform_values { |v| filter_all(v, depth + 1) } if value.is_a?(Hash)
165
+
166
+ FILTERED
167
+ end
168
+
169
+ def redact_value(value, extra, depth = 0)
170
+ return shorten(value) if value.is_a?(String)
171
+ return value if depth > 20 || !(value.is_a?(Hash) || value.is_a?(Array))
172
+ return value.map { |v| redact_value(v, extra, depth + 1) } if value.is_a?(Array)
173
+
174
+ value.to_h { |k, v| [k, secret_key?(k, extra) ? filter_all(v, depth + 1) : redact_value(v, extra, depth + 1)] }
175
+ end
176
+
177
+ RUNS = %r{"((?:data:[\w.+-]+/[\w.+-]+(?:;[\w=.+-]+)*;base64,)?[A-Za-z0-9+/_=-]{256,})("?)}
178
+ PAIRS = /("([^"\\]{1,100})"\s*:\s*)("(?:[^"\\]|\\.)*"?|-?\d[\d.eE+-]*|true|false)/
179
+
180
+ # Text that didn't parse: secret "key": value pairs filtered, long base64 runs shortened.
181
+ def redact_loose(text, extra)
182
+ text = text.gsub(RUNS) do
183
+ m = Regexp.last_match
184
+ short = shorten(m[1])
185
+ if short == m[1] then m[0]
186
+ elsif m[2].empty? then %("#{short[0..-2]}+]")
187
+ else %("#{short}")
188
+ end
189
+ end
190
+ text.gsub(PAIRS) { (m = Regexp.last_match) && secret_key?(m[2], extra) ? %(#{m[1]}"#{FILTERED}") : m[0] }
191
+ end
192
+
193
+ def decode_component(text) = URI.decode_www_form_component(text)
194
+
195
+ def redact_form(text, extra)
196
+ text.split("&", -1).map do |pair|
197
+ eq = pair.index("=")
198
+ next pair unless eq
199
+
200
+ key = begin
201
+ decode_component(pair[0...eq])
202
+ rescue ArgumentError
203
+ pair[0...eq]
204
+ end
205
+ next "#{pair[0...eq]}=#{FILTERED}" if secret_key?(key, extra)
206
+
207
+ value = pair[(eq + 1)..]
208
+ value.length >= 256 ? "#{pair[0...eq]}=#{shorten_encoded(value)}" : pair
209
+ end.join("&")
210
+ end
211
+
212
+ def shorten_encoded(value)
213
+ decoded = decode_component(value)
214
+ short = shorten(decoded)
215
+ short == decoded ? value : short
216
+ rescue ArgumentError
217
+ value
218
+ end
219
+
220
+ # ─── JSON as JavaScript writes it ────────────────────────────────────────────
221
+
222
+ # JSON.stringify(value): no spaces, numbers as JavaScript writes them (1.0 is 1, 1e21 is 1e+21).
223
+ def stringify(value)
224
+ case value
225
+ when nil then "null"
226
+ when true then "true"
227
+ when false then "false"
228
+ when Integer then value.to_s
229
+ when Float then js_number(value)
230
+ when String then JSON.generate(value)
231
+ when Hash then "{#{value.map { |k, v| "#{JSON.generate(k.to_s)}:#{stringify(v)}" }.join(",")}}"
232
+ when Array then "[#{value.map { |v| stringify(v) }.join(",")}]"
233
+ else JSON.generate(value.to_s)
234
+ end
235
+ end
236
+
237
+ def js_number(value)
238
+ return "null" if value.nan? || value.infinite?
239
+ return "0" if value.zero?
240
+ return value.to_i.to_s if value == value.floor && value.abs < 1e21
241
+
242
+ text = value.to_s
243
+ return text unless text.include?("e")
244
+
245
+ mantissa, exponent = text.split("e")
246
+ exp = exponent.to_i
247
+ return BigDecimal(text).to_s("F").sub(/\.0\z/, "") if exp > -7 && exp < 21
248
+
249
+ "#{mantissa.sub(/\.0\z/, '')}e#{exp.positive? ? '+' : '-'}#{exp.abs}"
250
+ end
251
+
252
+ # ─── the attribute value ─────────────────────────────────────────────────────
253
+
254
+ # A body as it is sent: redacted, then cut to max_size. cut says the text was already cut
255
+ # short where it was read. nil for an empty body. Returns [value, truncated].
256
+ def format_body(text, content_type, settings, cut = false)
257
+ return nil if text.nil?
258
+
259
+ body = text.to_s.dup.force_encoding(Encoding::UTF_8).scrub("\uFFFD")
260
+ trimmed = body.strip
261
+ return nil if trimmed.empty?
262
+
263
+ if form?(content_type)
264
+ body = redact_form(body, settings.redact_keys)
265
+ else
266
+ parsed = nil
267
+ ok = false
268
+ if !cut && "{[".include?(trimmed[0])
269
+ begin
270
+ parsed = JSON.parse(trimmed)
271
+ ok = true
272
+ rescue JSON::ParserError, SystemStackError, EncodingError
273
+ ok = false
274
+ end
275
+ end
276
+ body = ok ? stringify(redact_value(parsed, settings.redact_keys)) : redact_loose(body, settings.redact_keys)
277
+ end
278
+ length = utf16_length(body)
279
+ truncated = cut || length > settings.max_size
280
+ [length > settings.max_size ? utf16_slice(body, settings.max_size) : body, truncated]
281
+ end
282
+
283
+ def utf16_length(text) = text.length + text.each_char.count { |c| c.ord > 0xFFFF }
284
+
285
+ def utf16_slice(text, limit)
286
+ count = 0
287
+ text.each_char.with_index do |c, i|
288
+ count += c.ord > 0xFFFF ? 2 : 1
289
+ return text[0, i] if count > limit
290
+ end
291
+ text
292
+ end
293
+
294
+ # The span attributes for one side of a request ("request" or "response").
295
+ def body_attributes(side, text, content_type, settings, cut = false)
296
+ return {} unless textual?(content_type) || multipart?(content_type)
297
+
298
+ formatted = format_body(text, content_type, settings, cut)
299
+ return {} unless formatted
300
+
301
+ body_key, truncated_key, type_key = ATTRS.fetch(side)
302
+ out = { body_key => formatted[0] }
303
+ out[truncated_key] = true if formatted[1]
304
+ out[type_key] = content_type if content_type && !content_type.to_s.empty?
305
+ out
306
+ end
307
+
308
+ # The attributes for a body that isn't text: what it was, not what it said.
309
+ def described_attributes(side, content_type, size = nil, partial = false)
310
+ body_key, _, type_key = ATTRS.fetch(side)
311
+ out = { body_key => describe_body(content_type, size, partial) }
312
+ out[type_key] = content_type if content_type && !content_type.to_s.empty?
313
+ out
314
+ end
315
+
316
+ # A body never read (a streamed response): text is [stream · application/json · 60 B].
317
+ def unread_attributes(side, content_type, size)
318
+ return described_attributes(side, content_type, size) unless textual?(content_type)
319
+
320
+ parts = ["stream"]
321
+ kind = mime(content_type)
322
+ parts << kind unless kind.empty?
323
+ parts << format_size(size) if size&.positive?
324
+ body_key, _, type_key = ATTRS.fetch(side)
325
+ out = { body_key => "[#{parts.join(' · ')}]" }
326
+ out[type_key] = content_type if content_type && !content_type.to_s.empty?
327
+ out
328
+ end
329
+
330
+ # ─── URLs ────────────────────────────────────────────────────────────────────
331
+
332
+ # A URL as it is recorded: secret-looking query values become [Filtered], user:password@ dropped.
333
+ def redact_url(url, extra = [])
334
+ return url if url.nil? || url.empty?
335
+
336
+ out = url.sub(%r{\A([a-z][a-z0-9+.-]*://)[^/?#@]*@}i, '\1')
337
+ q = out.index("?")
338
+ return out unless q
339
+
340
+ hash_at = out.index("#", q)
341
+ query = hash_at ? out[(q + 1)...hash_at] : out[(q + 1)..]
342
+ redacted = redact_query(query, extra)
343
+ redacted == query ? out : "#{out[0..q]}#{redacted}#{hash_at ? out[hash_at..] : ''}"
344
+ end
345
+
346
+ # A query string (without ?) with secret-looking values filtered.
347
+ def redact_query(query, extra = [])
348
+ return query if query.nil? || query.empty?
349
+
350
+ query.split("&", -1).map do |pair|
351
+ eq = pair.index("=")
352
+ next pair unless eq
353
+
354
+ key = begin
355
+ decode_component(pair[0...eq])
356
+ rescue ArgumentError
357
+ pair[0...eq]
358
+ end
359
+ secret = secret_key?(key, extra) || URL_EXACT.include?(normalize(key))
360
+ secret && eq < pair.length - 1 ? "#{pair[0...eq]}=#{FILTERED}" : pair
361
+ end.join("&")
362
+ end
363
+
364
+ DEFAULT_PORTS = { "http" => 80, "https" => 443, "ws" => 80, "wss" => 443, "ftp" => 21 }.freeze
365
+
366
+ # A bare host (with or without a port) matches that host; a string with a scheme matches as a
367
+ # prefix; a Regexp is searched in the URL.
368
+ def url_matches?(url, patterns)
369
+ host = hostname = ""
370
+ begin
371
+ uri = URI.parse(url)
372
+ hostname = uri.host.to_s.downcase
373
+ host = uri.port.nil? || DEFAULT_PORTS[uri.scheme.to_s.downcase] == uri.port ? hostname : "#{hostname}:#{uri.port}"
374
+ rescue URI::Error
375
+ nil
376
+ end
377
+ patterns.any? do |pattern|
378
+ if pattern.is_a?(Regexp) then pattern.match?(url)
379
+ elsif pattern.to_s.empty? then false
380
+ elsif pattern.include?("://") then url.start_with?(pattern)
381
+ else [host, hostname].include?(pattern)
382
+ end
383
+ end
384
+ end
385
+ end
386
+ end