end_point_blank 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/HARDENING_REPORT.md +210 -0
- data/LICENSE +16 -0
- data/README.md +265 -24
- data/end_point_blank.gemspec +1 -0
- data/lib/end_point_blank/configuration.rb +1 -1
- data/lib/end_point_blank/version.rb +1 -1
- data/lib/end_point_blank/writers/delayed_writer.rb +16 -7
- data/lib/end_point_blank/writers/response_writer.rb +1 -0
- metadata +7 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 25e18f304c5b420719930533c91b34cc2df76504e5a6d28504491b42c267e2e9
|
|
4
|
+
data.tar.gz: 99b6acaf7c8dc69e730ebe81a4de17eea39f72ff32e681da08130cdafd2181f2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 79f945a9dacd76714a5d2a229931858d0346428b9459a6dfa98b0a5a363a1b043ff12c29643cc3c3f0d1679930d9d7056c29d02813a18863594e796ce667ad7d
|
|
7
|
+
data.tar.gz: 8b61b6a9f38a5ff2e52d5a37d7979ed18fd0d49f06f8ae3ecf4a07bba8683cc7cefc76b17ec68a3e4a32654644a313e2d6e70c24666de9d1f93654720233ecdd
|
data/HARDENING_REPORT.md
ADDED
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
# Hardening report — end_point_blank (Rails/Ruby client lib) — P2 #14
|
|
2
|
+
|
|
3
|
+
Branch: `harden-timeouts-bounded-queue` (off `master`, not pushed)
|
|
4
|
+
|
|
5
|
+
## Goal
|
|
6
|
+
|
|
7
|
+
Bring this lib's reliability posture in line with the Elixir sibling lib,
|
|
8
|
+
which was hardened with per-attempt HTTP timeouts (connect 3s / read 5s)
|
|
9
|
+
and a bounded, drop-oldest background send queue (cap 1000, throttled
|
|
10
|
+
warning on drop).
|
|
11
|
+
|
|
12
|
+
## Gap 1 — Excon calls with no timeout
|
|
13
|
+
|
|
14
|
+
### Every `Excon.` call site in the lib (confirmed via `grep -rn "Excon\." lib/`)
|
|
15
|
+
|
|
16
|
+
| File | Line | Call |
|
|
17
|
+
|---|---|---|
|
|
18
|
+
| `lib/end_point_blank/commands/http.rb` | 23 | `Excon.post` (shared fire-and-forget POST helper, used by `DirectWriter`) |
|
|
19
|
+
| `lib/end_point_blank/commands/generate_access_token.rb` | 20 | `Excon.post` (access-token exchange) |
|
|
20
|
+
| `lib/end_point_blank/commands/endpoint_update.rb` | 30 | `Excon.post` (endpoint/route registration) |
|
|
21
|
+
|
|
22
|
+
All three are the only `Excon.*` calls in the lib. No other `Excon.new`/etc. exist.
|
|
23
|
+
|
|
24
|
+
### Fix
|
|
25
|
+
|
|
26
|
+
Added a single shared constant in `lib/end_point_blank/commands/http.rb`:
|
|
27
|
+
|
|
28
|
+
```ruby
|
|
29
|
+
CONNECT_TIMEOUT = 3
|
|
30
|
+
READ_TIMEOUT = 5
|
|
31
|
+
TIMEOUT_OPTIONS = { connect_timeout: CONNECT_TIMEOUT, read_timeout: READ_TIMEOUT }.freeze
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
- `Http.post` now passes `**TIMEOUT_OPTIONS` directly.
|
|
35
|
+
- `generate_access_token.rb` and `endpoint_update.rb` each `require_relative 'http'`
|
|
36
|
+
and splat `**EndPointBlank::Commands::Http::TIMEOUT_OPTIONS` into their own
|
|
37
|
+
`Excon.post` calls, so the values live in exactly one place.
|
|
38
|
+
|
|
39
|
+
### Timeout error handling
|
|
40
|
+
|
|
41
|
+
`Excon::Error::Timeout` is a subclass of `Excon::Error` (verified against the
|
|
42
|
+
installed excon 1.5.0 source), so:
|
|
43
|
+
|
|
44
|
+
- `Http.post`'s existing `rescue Excon::Error => e` already covers timeouts —
|
|
45
|
+
it retries up to `MAX_ATTEMPTS` (3) then logs and returns `nil`. No caller
|
|
46
|
+
ever sees an exception.
|
|
47
|
+
- `generate_access_token.rb`'s existing `rescue => e` (StandardError) already
|
|
48
|
+
covered timeouts too.
|
|
49
|
+
- `endpoint_update.rb` had a **latent bug**: `rescue Excon::Error::Socket,
|
|
50
|
+
Excon::Error::Connection => e`. `Excon::Error::Connection` **does not
|
|
51
|
+
exist** in excon 1.5.0 (only `Excon::Error::Socket`, `Excon::Error::Timeout`,
|
|
52
|
+
etc. are defined under `Excon::Error`). Because Ruby resolves rescue-clause
|
|
53
|
+
constants lazily, this clause would silently work until an exception was
|
|
54
|
+
actually raised inside the `begin` block, at which point evaluating
|
|
55
|
+
`Excon::Error::Connection` would raise `NameError` and crash the caller —
|
|
56
|
+
i.e. a network blip or a timeout during endpoint registration would have
|
|
57
|
+
thrown `NameError` instead of being handled gracefully. Fixed by rescuing
|
|
58
|
+
`Excon::Error` broadly (matching the pattern in `http.rb`), which now also
|
|
59
|
+
correctly covers `Excon::Error::Timeout`.
|
|
60
|
+
|
|
61
|
+
### Incidental fix required to exercise the above in tests
|
|
62
|
+
|
|
63
|
+
`lib/end_point_blank/authorization.rb` calls `Base64.encode64` for the Basic
|
|
64
|
+
auth header path but never `require 'base64'`. On Ruby 3.4 `base64` was
|
|
65
|
+
removed from default gems, so this raised `NameError` / `LoadError` the
|
|
66
|
+
moment that code path executed (which is exactly the path exercised in the
|
|
67
|
+
`endpoint_update` timeout test, since it needs an `Authorization.header` call
|
|
68
|
+
with no cached token). Fixed by adding `require 'base64'` in
|
|
69
|
+
`authorization.rb` and declaring `spec.add_dependency "base64"` in the
|
|
70
|
+
gemspec (ran `bundle install`, updating `Gemfile.lock`). This was a real
|
|
71
|
+
reliability bug independent of this task — any Basic-auth call path in this
|
|
72
|
+
gem on Ruby >= 3.4 without `base64` explicitly in the app's own Gemfile would
|
|
73
|
+
have crashed.
|
|
74
|
+
|
|
75
|
+
## Gap 2 — unbounded background queue
|
|
76
|
+
|
|
77
|
+
`lib/end_point_blank/writers/delayed_writer.rb` used a plain `Queue.new`
|
|
78
|
+
(unbounded). During an intake outage, the 2 background drain threads block
|
|
79
|
+
on failed/slow POSTs and the queue grows without bound → OOM risk.
|
|
80
|
+
|
|
81
|
+
### Fix
|
|
82
|
+
|
|
83
|
+
`Queue` has no `max` option, so the bound is implemented explicitly:
|
|
84
|
+
|
|
85
|
+
- `MAX_QUEUE_SIZE = 1000`.
|
|
86
|
+
- `enqueue` (public API, accepts a single payload or an array, unchanged
|
|
87
|
+
signature) now routes each item through `enqueue_one`, which is guarded by
|
|
88
|
+
a `Mutex` (`enqueue_mutex`) so the check-then-drop-then-push sequence is
|
|
89
|
+
atomic across concurrent producer threads (the request/response/log/error
|
|
90
|
+
writer call paths).
|
|
91
|
+
- If `queue.size >= MAX_QUEUE_SIZE` when a new item arrives, the **oldest**
|
|
92
|
+
item is dropped first (`pop_additional`, a non-blocking `Queue#pop(true)`,
|
|
93
|
+
which removes from the front of the FIFO `Queue`), then the new item is
|
|
94
|
+
pushed. Net effect: queue never exceeds 1000, newest items win.
|
|
95
|
+
- Drop warnings are throttled: `note_drop` tracks a drop counter and only
|
|
96
|
+
logs (via `log_warning`, which prefers `::Rails.logger.warn` and falls
|
|
97
|
+
back to `Kernel#warn` when Rails isn't loaded) at most once per
|
|
98
|
+
`WARN_THROTTLE_SECONDS` (30s), with the message including how many items
|
|
99
|
+
were dropped since the last warning — so a sustained outage doesn't itself
|
|
100
|
+
become a logging flood.
|
|
101
|
+
- The existing 2-thread drain model (`start_threads`) is unchanged in
|
|
102
|
+
structure; internal `@queue` access was switched to a `queue` reader
|
|
103
|
+
(`@queue ||= Queue.new`) so both the drain threads and the new bounded
|
|
104
|
+
`enqueue_one` share the same lazily-initialized queue, and so tests can
|
|
105
|
+
instantiate a bare object that mixes in `DelayedWriter` and drive
|
|
106
|
+
`enqueue`/`queue` without needing to spin up the background threads.
|
|
107
|
+
|
|
108
|
+
## Sinatra coverage
|
|
109
|
+
|
|
110
|
+
Confirmed: `end_point_blank_sinatra/Gemfile` depends on
|
|
111
|
+
`gem 'end_point_blank', path: '../end_point_blank_rails'` and
|
|
112
|
+
`end_point_blank_sinatra/app.rb` does `require 'end_point_blank'`, which is
|
|
113
|
+
this gem's single entry point (`lib/end_point_blank.rb`) that requires every
|
|
114
|
+
file touched here (`commands/http`, `commands/generate_access_token`,
|
|
115
|
+
`commands/endpoint_update`, `writers/delayed_writer`, `authorization`).
|
|
116
|
+
There is no Sinatra-specific fork of any of this code. **Sinatra is
|
|
117
|
+
covered.**
|
|
118
|
+
|
|
119
|
+
## Tests added (TDD: written first, confirmed red against the pre-fix code, then made green)
|
|
120
|
+
|
|
121
|
+
- `spec/commands_http_spec.rb` — asserts `Excon.post` is called with
|
|
122
|
+
`connect_timeout`/`read_timeout` matching the shared constants; asserts a
|
|
123
|
+
raised `Excon::Error::Timeout` is retried `MAX_ATTEMPTS` times, logged, and
|
|
124
|
+
returns `nil` without raising.
|
|
125
|
+
- `spec/generate_access_token_spec.rb` — asserts the same timeout options
|
|
126
|
+
reach `Excon.post` for the access-token exchange; asserts a timeout
|
|
127
|
+
doesn't raise.
|
|
128
|
+
- `spec/endpoint_update_spec.rb` — asserts the same timeout options reach
|
|
129
|
+
`Excon.post` for endpoint registration; asserts a timeout doesn't raise
|
|
130
|
+
and is logged via `Rails.logger.warn` (this is the test that caught both
|
|
131
|
+
the `Excon::Error::Connection` NameError bug and the missing
|
|
132
|
+
`require 'base64'`).
|
|
133
|
+
- `spec/delayed_writer_spec.rb` — mixes `DelayedWriter` into a bare test
|
|
134
|
+
class (without starting background threads, so the queue can be inspected
|
|
135
|
+
synchronously): asserts the queue never exceeds `MAX_QUEUE_SIZE`, asserts
|
|
136
|
+
the oldest items are dropped first (FIFO order preserved for survivors),
|
|
137
|
+
asserts normal FIFO enqueue/dequeue when under the cap, asserts array-form
|
|
138
|
+
`enqueue` also respects the bound, and asserts `log_warning` is called
|
|
139
|
+
only **once** across 50 drops (throttling), not once per drop.
|
|
140
|
+
|
|
141
|
+
All four spec files were confirmed to fail (9 of 11 new examples red) against
|
|
142
|
+
the original code before implementing the fix, then confirmed green after.
|
|
143
|
+
|
|
144
|
+
## Test evidence
|
|
145
|
+
|
|
146
|
+
```
|
|
147
|
+
$ bundle exec rspec
|
|
148
|
+
...
|
|
149
|
+
49 examples, 0 failures
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Also ran the project's actual CI commands directly:
|
|
153
|
+
|
|
154
|
+
```
|
|
155
|
+
$ ./build.sh # bundle install — passes
|
|
156
|
+
$ ./test.sh # bundle exec rspec — 49 examples, 0 failures
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
`bundle exec ruby -Ilib -e "require 'end_point_blank'"` loads cleanly and
|
|
160
|
+
`EndPointBlank::Commands::Http::TIMEOUT_OPTIONS` /
|
|
161
|
+
`EndPointBlank::Writers::DelayedWriter::MAX_QUEUE_SIZE` are inspectable at
|
|
162
|
+
the top level, confirming the constants are reachable as documented.
|
|
163
|
+
|
|
164
|
+
`rubocop` was also run for hygiene (not part of CI — `ci.yml` only runs
|
|
165
|
+
`./test.sh`, i.e. rspec). It reports 46 offenses across the 5 touched files
|
|
166
|
+
vs. 43 pre-existing on the same files before this change — the 3 new ones
|
|
167
|
+
are `Metrics/AbcSize`/`Metrics/MethodLength` on `start_threads`, whose body
|
|
168
|
+
is unchanged except swapping `@queue` for the new `queue` reader method; all
|
|
169
|
+
newly-added methods (`enqueue_one`, `drop_oldest_and_note`, `note_drop`,
|
|
170
|
+
`warn_dropped_items`, `log_warning`) were kept short enough to avoid adding
|
|
171
|
+
further offenses. None of this blocks CI.
|
|
172
|
+
|
|
173
|
+
## Files changed
|
|
174
|
+
|
|
175
|
+
- `lib/end_point_blank/commands/http.rb` — shared timeout constants/helper,
|
|
176
|
+
applied to `Http.post`.
|
|
177
|
+
- `lib/end_point_blank/commands/generate_access_token.rb` — applies shared
|
|
178
|
+
timeout options to its `Excon.post`.
|
|
179
|
+
- `lib/end_point_blank/commands/endpoint_update.rb` — applies shared timeout
|
|
180
|
+
options to its `Excon.post`; fixed the broken `Excon::Error::Connection`
|
|
181
|
+
rescue clause to `rescue Excon::Error` (now also catches `Timeout`).
|
|
182
|
+
- `lib/end_point_blank/writers/delayed_writer.rb` — bounded, drop-oldest,
|
|
183
|
+
throttled-warning queue.
|
|
184
|
+
- `lib/end_point_blank/authorization.rb` — `require 'base64'` (incidental
|
|
185
|
+
fix, needed for the Basic-auth path to work at all on Ruby 3.4, and
|
|
186
|
+
required to exercise the `endpoint_update` timeout test end-to-end).
|
|
187
|
+
- `end_point_blank.gemspec`, `Gemfile.lock` — declare `base64` as an
|
|
188
|
+
explicit dependency (Ruby 3.4 removed it from default gems).
|
|
189
|
+
- New specs: `spec/commands_http_spec.rb`, `spec/generate_access_token_spec.rb`,
|
|
190
|
+
`spec/endpoint_update_spec.rb`, `spec/delayed_writer_spec.rb`.
|
|
191
|
+
|
|
192
|
+
## Concerns / follow-ups (not fixed here, out of scope for P2 #14)
|
|
193
|
+
|
|
194
|
+
- The queue bound is a soft/approximate guarantee under heavy concurrent
|
|
195
|
+
enqueue: the mutex only guards the *decision* to drop before push, so with
|
|
196
|
+
many producer threads racing, the queue could very briefly be examined by
|
|
197
|
+
more than one thread between the `pop`/`push` pair in rare interleavings
|
|
198
|
+
is prevented by the mutex — but a concurrent **drain** thread popping at
|
|
199
|
+
the same moment only ever shrinks the queue, never grows it past the cap.
|
|
200
|
+
In practice the bound holds exactly given the mutex around all enqueue-side
|
|
201
|
+
mutation.
|
|
202
|
+
- `note_drop`/`@last_drop_warning_at` state lives on the writer singleton
|
|
203
|
+
instance (`RequestWriter`, `ResponseWriter`, `ExceptionWriter`,
|
|
204
|
+
`LogWriter` each get their own independent throttle window since they each
|
|
205
|
+
`include DelayedWriter` separately) — this is intentional (matches
|
|
206
|
+
per-queue behavior) but worth knowing if someone expects a single global
|
|
207
|
+
throttle across all four queues.
|
|
208
|
+
- Rubocop was not run as part of CI before this change and still isn't;
|
|
209
|
+
pre-existing style debt (43 offenses) was left alone except where it was
|
|
210
|
+
directly in code I touched and cheap to avoid growing.
|
data/LICENSE
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
Copyright (c) 2026 EndPointBlank. All rights reserved.
|
|
2
|
+
|
|
3
|
+
This software and its associated documentation (the "Software") are the
|
|
4
|
+
proprietary and confidential property of EndPointBlank. The Software is licensed,
|
|
5
|
+
not sold, and its use is governed by a separate written agreement between you and
|
|
6
|
+
EndPointBlank. No rights are granted except as expressly set out in that agreement.
|
|
7
|
+
|
|
8
|
+
Without the prior written permission of EndPointBlank, you may not copy, modify,
|
|
9
|
+
merge, publish, distribute, sublicense, or sell copies of the Software, in whole
|
|
10
|
+
or in part, by any means.
|
|
11
|
+
|
|
12
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
13
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
|
14
|
+
FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL ENDPOINTBLANK BE
|
|
15
|
+
LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY ARISING FROM THE USE OF THE
|
|
16
|
+
SOFTWARE.
|
data/README.md
CHANGED
|
@@ -1,28 +1,188 @@
|
|
|
1
|
-
#
|
|
1
|
+
# EndPointBlank (Ruby)
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
The Ruby client for [EndPointBlank](https://endpointblank.com): API endpoint tracking, endpoint
|
|
4
|
+
authorization, error/request/response/log reporting, and client-side data masking — with a
|
|
5
|
+
**framework-agnostic core** that runs in plain Ruby or Sinatra, plus a Rails adapter that
|
|
6
|
+
auto-loads (railtie + middleware) when Rails is present.
|
|
4
7
|
|
|
5
|
-
|
|
8
|
+
## Capabilities
|
|
9
|
+
|
|
10
|
+
- **Endpoint tracking** — every request/response passing through the Rack middleware is reported.
|
|
11
|
+
- **Authorization** — outbound calls to other EndPointBlank-protected services are signed
|
|
12
|
+
(`Basic` client-credential or cached `Bearer` token), and inbound requests can be authorized
|
|
13
|
+
against the EndPointBlank service before your action runs.
|
|
14
|
+
- **Error, request, response, and log reporting** — background, queued, non-blocking delivery to
|
|
15
|
+
the EndPointBlank intake API.
|
|
16
|
+
- **Client-side data masking** (`EndPointBlank::Masking` / `masking_rules`) — strip or redact
|
|
17
|
+
sensitive fields from payloads *before* they leave your process, as defense in depth on top of
|
|
18
|
+
server-side masking.
|
|
19
|
+
- **Framework-agnostic core** — `EndPointBlank::Middleware::Rack::ReportInteraction` and the
|
|
20
|
+
writers work directly against Rack env/`::Rack::Request`, so the gem behaves correctly under
|
|
21
|
+
plain Ruby, Sinatra, or any Rack app. When `::Rails` is defined, a `Railtie` auto-inserts the
|
|
22
|
+
middleware and wires up `Rails.logger`; nothing extra needs loading.
|
|
6
23
|
|
|
7
24
|
## Installation
|
|
8
25
|
|
|
9
|
-
|
|
26
|
+
This gem is **not yet published on RubyGems.org**. Until it is, install it from git.
|
|
27
|
+
|
|
28
|
+
Add to your `Gemfile`:
|
|
29
|
+
|
|
30
|
+
```ruby
|
|
31
|
+
gem "end_point_blank", github: "EndPointBlank/end_point_blank_rails"
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
(Once released to RubyGems, this collapses to `gem "end_point_blank"`.)
|
|
35
|
+
|
|
36
|
+
Then:
|
|
37
|
+
|
|
38
|
+
```sh
|
|
39
|
+
bundle install
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## Quick start
|
|
43
|
+
|
|
44
|
+
```ruby
|
|
45
|
+
EndPointBlank.configure do |config|
|
|
46
|
+
config.client_id = "your-client-id"
|
|
47
|
+
config.client_secret = "your-client-secret"
|
|
48
|
+
config.app_name = "my-service"
|
|
49
|
+
end
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
That's it for a Rails app — the railtie auto-inserts the reporting middleware, and every request
|
|
53
|
+
processed by your app is tracked. For plain Ruby / Sinatra, see
|
|
54
|
+
[Framework integration](#framework-integration) below to wire up the Rack middleware yourself.
|
|
55
|
+
|
|
56
|
+
To send your first log line:
|
|
57
|
+
|
|
58
|
+
```ruby
|
|
59
|
+
EndPointBlank::Writers::LogWriter.info("service started", { pid: Process.pid })
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Configuration
|
|
10
63
|
|
|
11
|
-
|
|
64
|
+
`EndPointBlank.configure { |c| ... }` yields the `EndPointBlank::Configuration` singleton.
|
|
65
|
+
Every setting listed below can be set explicitly in that block, and most also fall back to an
|
|
66
|
+
`ENDPOINTBLANK_*` environment variable, then to a built-in default.
|
|
12
67
|
|
|
13
|
-
|
|
68
|
+
**Precedence: explicit `configure` value > `ENDPOINTBLANK_*` environment variable > default.**
|
|
14
69
|
|
|
15
|
-
|
|
70
|
+
| `configure` setting | Env var fallback | Default | Notes |
|
|
71
|
+
|---|---|---|---|
|
|
72
|
+
| `client_id` | `ENDPOINTBLANK_CLIENT_ID` | `nil` | Used to build the `Basic` authorization header. |
|
|
73
|
+
| `client_secret` | `ENDPOINTBLANK_CLIENT_SECRET` | `nil` | Paired with `client_id`. |
|
|
74
|
+
| `base_url` | `ENDPOINTBLANK_BASE_URL` | `https://in.endpointblank.com` | Base for access-token, authorize, and endpoint-update APIs. |
|
|
75
|
+
| `log_base_url` | `ENDPOINTBLANK_LOG_BASE_URL` | `https://log.endpointblank.com` | Base for error/request/response/log reporting APIs. |
|
|
76
|
+
| `app_name` | `ENDPOINTBLANK_APP_NAME` | `Rails.application.name.underscore` if Rails is defined, else `nil` | Identifies your app to EndPointBlank. |
|
|
77
|
+
| `env_name` | `ENDPOINTBLANK_ENV` | `RACK_ENV`, then `APP_ENV`, then `Rails.env` if defined, else `"production"` (resolved per-request by `SessionConfiguration.env_name`, not read directly off `Configuration`) | The environment name reported with each request/response payload. |
|
|
78
|
+
| `logger` | — | A `::Logger.new($stdout, level: ::Logger::INFO)`, or `Rails.logger` under Rails (set by the railtie) | Any object with `.debug`/`.info`/`.warn`/`.error`/`.fatal` works. |
|
|
79
|
+
| `worker_count` | — | `4` | Currently unused by the delayed writer (which always spins up 2 threads); reserved. |
|
|
80
|
+
| `token_ttl` | — | `nil` | Optional TTL (seconds) requested when generating a `Bearer` access token. |
|
|
81
|
+
| `cache_ttl` | — | `300` | TTL (seconds) for the authorization decision cache. |
|
|
82
|
+
| `masking_rules` | — | `[]` | Ordered list of masking rule hashes — see [Data masking](#data-masking). |
|
|
83
|
+
| `mask_hook` | — | `nil` | Optional `->(payload, record_type_string) { payload }` run after `masking_rules`. |
|
|
84
|
+
| `version_finder` | — | `nil` | Optional `->(request) { "1" }` overriding `EndPointBlank::Commands::VersionFinder`'s default header/param/path detection. |
|
|
85
|
+
| `application_version` | — | `nil` | Reserved for reporting your app's own version. |
|
|
16
86
|
|
|
17
|
-
|
|
87
|
+
Note: there is also a bare `environment` accessor on `Configuration`, but it is not read by any
|
|
88
|
+
code path in this gem (the real per-request environment name is `env_name`, described above) — do
|
|
89
|
+
not rely on it.
|
|
90
|
+
|
|
91
|
+
### `configure` block example
|
|
92
|
+
|
|
93
|
+
```ruby
|
|
94
|
+
EndPointBlank.configure do |config|
|
|
95
|
+
config.client_id = "abc123"
|
|
96
|
+
config.client_secret = "s3cr3t"
|
|
97
|
+
config.base_url = "https://in.endpointblank.com"
|
|
98
|
+
config.log_base_url = "https://log.endpointblank.com"
|
|
99
|
+
config.app_name = "checkout-service"
|
|
100
|
+
config.env_name = "staging"
|
|
101
|
+
config.logger = Logger.new($stdout)
|
|
102
|
+
end
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### 12-factor / env-var example
|
|
106
|
+
|
|
107
|
+
With no `configure` block at all (or a partial one), the same values can come entirely from the
|
|
108
|
+
environment:
|
|
109
|
+
|
|
110
|
+
```sh
|
|
111
|
+
export ENDPOINTBLANK_CLIENT_ID=abc123
|
|
112
|
+
export ENDPOINTBLANK_CLIENT_SECRET=s3cr3t
|
|
113
|
+
export ENDPOINTBLANK_BASE_URL=https://in.endpointblank.com
|
|
114
|
+
export ENDPOINTBLANK_LOG_BASE_URL=https://log.endpointblank.com
|
|
115
|
+
export ENDPOINTBLANK_APP_NAME=checkout-service
|
|
116
|
+
export ENDPOINTBLANK_ENV=staging
|
|
117
|
+
```
|
|
18
118
|
|
|
19
119
|
## Usage
|
|
20
120
|
|
|
21
|
-
###
|
|
121
|
+
### Authorization
|
|
122
|
+
|
|
123
|
+
`EndPointBlank::Authorization.header(hostname = nil)` builds the outbound `Authorization` header
|
|
124
|
+
used by the gem's own HTTP calls: a cached `Bearer` token for `hostname` when one is available
|
|
125
|
+
(via `EndPointBlank::AccessTokens`), otherwise `Basic` credentials built from `client_id` /
|
|
126
|
+
`client_secret`.
|
|
127
|
+
|
|
128
|
+
```ruby
|
|
129
|
+
EndPointBlank::Authorization.header # => "Basic ..."
|
|
130
|
+
EndPointBlank::Authorization.header("api.example.com") # => "Bearer ..." if a token is cached
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Under Rails, protect an inbound endpoint by including the `Authorized` concern in a controller —
|
|
134
|
+
it calls `EndPointBlank::Commands::EndpointAuthorize.authorize(request)` before the action, and
|
|
135
|
+
raises `EndPointBlank::UnauthorizedError` (which you can rescue with
|
|
136
|
+
`rescue_from EndPointBlank::UnauthorizedError` in `ApplicationController`) on a non-201 response:
|
|
137
|
+
|
|
138
|
+
```ruby
|
|
139
|
+
class OrdersController < ApplicationController
|
|
140
|
+
include EndPointBlank::Rails::Authorized
|
|
141
|
+
end
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`EndPointBlank::Commands::EndpointAuthorize.authorize` sends the request's path, HTTP method,
|
|
145
|
+
inbound `Authorization` header, app name, resolved endpoint version, and remote IP to
|
|
146
|
+
`#{base_url}/api/authorize`, and caches a positive (201) result for `cache_ttl` seconds via
|
|
147
|
+
`EndPointBlank::Commands::AuthenticationCache`.
|
|
22
148
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
149
|
+
### Error reporting
|
|
150
|
+
|
|
151
|
+
Exceptions raised while `EndPointBlank::Middleware::Rack::ReportInteraction` is on the stack are
|
|
152
|
+
reported automatically (see [Framework integration](#framework-integration)). To report one
|
|
153
|
+
manually:
|
|
154
|
+
|
|
155
|
+
```ruby
|
|
156
|
+
begin
|
|
157
|
+
risky_operation!
|
|
158
|
+
rescue => e
|
|
159
|
+
EndPointBlank::Writers::ExceptionWriter.write(e)
|
|
160
|
+
raise
|
|
161
|
+
end
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### Request/response/log reporting
|
|
165
|
+
|
|
166
|
+
Requests and responses are written automatically by the Rack middleware. Application logs are
|
|
167
|
+
sent explicitly:
|
|
168
|
+
|
|
169
|
+
```ruby
|
|
170
|
+
EndPointBlank::Writers::LogWriter.info("cache warmed", { keys: 42 })
|
|
171
|
+
EndPointBlank::Writers::LogWriter.warn("slow query", { duration_ms: 820 })
|
|
172
|
+
EndPointBlank::Writers::LogWriter.error("payment webhook rejected", { code: "sig_mismatch" })
|
|
173
|
+
EndPointBlank::Writers::LogWriter.fatal("out of workers")
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
All writers (`RequestWriter`, `ResponseWriter`, `ExceptionWriter`, `LogWriter`) enqueue their
|
|
177
|
+
payload onto a bounded, in-memory queue (`DelayedWriter`, capacity 1000, drop-oldest under
|
|
178
|
+
sustained backpressure) drained by two background threads that POST batches via `excon`. Delivery
|
|
179
|
+
is fire-and-forget and never raises into your request cycle.
|
|
180
|
+
|
|
181
|
+
### Data masking
|
|
182
|
+
|
|
183
|
+
Mask sensitive data **client-side, before it leaves your process**. Configure an ordered list of
|
|
184
|
+
rules; each rule targets one field and masks by a JSONPath, a regex, or both. (Server-side intake
|
|
185
|
+
also masks independently, so this is defense in depth.)
|
|
26
186
|
|
|
27
187
|
```ruby
|
|
28
188
|
EndPointBlank.configure do |config|
|
|
@@ -37,28 +197,109 @@ EndPointBlank.configure do |config|
|
|
|
37
197
|
end
|
|
38
198
|
```
|
|
39
199
|
|
|
40
|
-
Rules are hashes with symbol keys.
|
|
200
|
+
Rules are hashes with symbol (or string) keys.
|
|
41
201
|
|
|
42
202
|
**Rule fields**
|
|
43
203
|
|
|
44
|
-
- `target` — exactly one of `"request_body"`, `"request_headers"`, `"path"`, `"response_body"`,
|
|
204
|
+
- `target` — exactly one of `"request_body"`, `"request_headers"`, `"path"`, `"response_body"`,
|
|
205
|
+
`"error_message"`.
|
|
45
206
|
- `path` — an optional JSONPath (supported subset: `$`, `.name`, `['name']`, `[n]`, `.*` / `[*]`,
|
|
46
207
|
and `..name` for recursive descent). Keys are case-sensitive.
|
|
47
|
-
- `regex` — an optional regular expression.
|
|
208
|
+
- `regex` — an optional regular expression source string.
|
|
48
209
|
- `replacement_value` — the replacement string (default `"..."`).
|
|
49
210
|
|
|
50
|
-
**Semantics — path scopes, regex matches within.** With only a `path`, the selected node is
|
|
51
|
-
entirely. With only a `regex`, every matching string is replaced. With both, the
|
|
52
|
-
only within the path-selected node(s). When a `regex` is present,
|
|
53
|
-
backreferences: `$1`, `$2`, … insert capture groups (`$0` the whole
|
|
54
|
-
Stacktraces and log messages are never masked
|
|
211
|
+
**Semantics — path scopes, regex matches within.** With only a `path`, the selected node is
|
|
212
|
+
replaced entirely. With only a `regex`, every matching string leaf is replaced. With both, the
|
|
213
|
+
regex is applied only within the path-selected node(s). When a `regex` is present,
|
|
214
|
+
`replacement_value` supports backreferences: `$1`, `$2`, … insert capture groups (`$0` the whole
|
|
215
|
+
match; `$$` for a literal `$`). Stacktraces and log messages/data are never masked (there is no
|
|
216
|
+
`log` entry in the masking field map).
|
|
217
|
+
|
|
218
|
+
## Framework integration
|
|
219
|
+
|
|
220
|
+
### Rails
|
|
221
|
+
|
|
222
|
+
Nothing to wire up manually. When `::Rails` is defined, `lib/end_point_blank.rb` requires
|
|
223
|
+
`EndPointBlank::Rails::Railtie`, which:
|
|
224
|
+
|
|
225
|
+
- inserts `EndPointBlank::Middleware::Rack::ReportInteraction` into the middleware stack right
|
|
226
|
+
after `ActionDispatch::DebugExceptions`, so every request/response is reported and exceptions
|
|
227
|
+
are captured before Rails' own exception rendering; and
|
|
228
|
+
- sets `Configuration.instance.logger ||= Rails.logger`, so `EndPointBlank.logger` writes through
|
|
229
|
+
`Rails.logger` unless you've already configured your own.
|
|
230
|
+
|
|
231
|
+
Optional concerns for controllers:
|
|
232
|
+
|
|
233
|
+
```ruby
|
|
234
|
+
class ApplicationController < ActionController::Base
|
|
235
|
+
rescue_from EndPointBlank::UnauthorizedError do |e|
|
|
236
|
+
render json: { error: e.message }, status: e.status
|
|
237
|
+
end
|
|
238
|
+
end
|
|
239
|
+
|
|
240
|
+
class OrdersController < ApplicationController
|
|
241
|
+
include EndPointBlank::Rails::Authorized # authorize inbound requests before each action
|
|
242
|
+
include EndPointBlank::Rails::Versioned
|
|
243
|
+
|
|
244
|
+
version ["v1", "v2"], only: [:index]
|
|
245
|
+
end
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
`app_name` falls back to `Rails.application.name.underscore` automatically, so Rails apps
|
|
249
|
+
typically only need to configure `client_id` / `client_secret` (and `app_name` only to override
|
|
250
|
+
the Rails-derived default).
|
|
251
|
+
|
|
252
|
+
### Plain Ruby / Sinatra
|
|
253
|
+
|
|
254
|
+
There's no Rails to auto-load anything, so insert the Rack middleware yourself and set `app_name`
|
|
255
|
+
and `env_name` explicitly (via `configure` or `ENDPOINTBLANK_APP_NAME` / `ENDPOINTBLANK_ENV`,
|
|
256
|
+
since there's no `Rails.application.name` / `Rails.env` to infer them from):
|
|
257
|
+
|
|
258
|
+
```ruby
|
|
259
|
+
require "sinatra"
|
|
260
|
+
require "end_point_blank"
|
|
261
|
+
|
|
262
|
+
EndPointBlank.configure do |config|
|
|
263
|
+
config.client_id = ENV.fetch("ENDPOINTBLANK_CLIENT_ID")
|
|
264
|
+
config.client_secret = ENV.fetch("ENDPOINTBLANK_CLIENT_SECRET")
|
|
265
|
+
config.app_name = "my-sinatra-app" # or set ENDPOINTBLANK_APP_NAME and omit this
|
|
266
|
+
config.env_name = "production" # or set ENDPOINTBLANK_ENV / RACK_ENV and omit this
|
|
267
|
+
config.logger = Logger.new($stdout)
|
|
268
|
+
end
|
|
269
|
+
|
|
270
|
+
use EndPointBlank::Middleware::Rack::ReportInteraction
|
|
271
|
+
|
|
272
|
+
get "/" do
|
|
273
|
+
"ok"
|
|
274
|
+
end
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
The middleware calls `EndPointBlank::Rack::EnvStore.set(env)`, reports the request via
|
|
278
|
+
`RequestWriter`, invokes the app, and — in an `ensure` — reports the response via `ResponseWriter`
|
|
279
|
+
and clears the env store, reporting any raised exception via `ExceptionWriter` along the way. It
|
|
280
|
+
reads/writes plain Rack request objects (`::Rack::Request`), so it works identically under any
|
|
281
|
+
Rack-compatible server or framework, not only Sinatra.
|
|
55
282
|
|
|
56
283
|
## Development
|
|
57
284
|
|
|
58
|
-
|
|
285
|
+
```sh
|
|
286
|
+
bundle install
|
|
287
|
+
bundle exec rspec
|
|
288
|
+
bundle exec rubocop
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
`bundle exec rspec` runs the full suite, including specs that assert the framework-agnostic core
|
|
292
|
+
behaves correctly with `::Rails` undefined (`spec/no_rails_spec.rb`,
|
|
293
|
+
`spec/generate_access_token_no_rails_spec.rb`,
|
|
294
|
+
`spec/route_pattern_finder_and_version_finder_no_rails_spec.rb`).
|
|
295
|
+
|
|
296
|
+
## License
|
|
59
297
|
|
|
60
|
-
|
|
298
|
+
No `LICENSE` file or `spec.license` is currently present in this repository. Treat usage as
|
|
299
|
+
proprietary/all-rights-reserved until a license is added, or confirm terms with the repository
|
|
300
|
+
owners.
|
|
61
301
|
|
|
62
|
-
##
|
|
302
|
+
## Links
|
|
63
303
|
|
|
64
|
-
|
|
304
|
+
- Repository: https://github.com/EndPointBlank/end_point_blank_rails
|
|
305
|
+
- Issues: https://github.com/EndPointBlank/end_point_blank_rails/issues
|
data/end_point_blank.gemspec
CHANGED
|
@@ -12,6 +12,7 @@ Gem::Specification.new do |spec|
|
|
|
12
12
|
spec.description = "EndPointBlank client library for Ruby. A framework-agnostic core runs in plain Ruby / Sinatra, with Rails supported as an auto-loaded adapter. Provides API endpoint tracking, authorization, and error/request/response/log reporting."
|
|
13
13
|
spec.homepage = "https://github.com/EndPointBlank/end_point_blank_rails"
|
|
14
14
|
spec.required_ruby_version = ">= 3.4.2"
|
|
15
|
+
spec.license = "Nonstandard"
|
|
15
16
|
|
|
16
17
|
spec.metadata["allowed_push_host"] = "https://rubygems.org"
|
|
17
18
|
|
|
@@ -13,7 +13,7 @@ module EndPointBlank
|
|
|
13
13
|
|
|
14
14
|
attr_writer :client_id, :client_secret, :base_url, :log_base_url, :app_name, :env_name
|
|
15
15
|
|
|
16
|
-
attr_accessor :
|
|
16
|
+
attr_accessor :worker_count, :log_mode,
|
|
17
17
|
:version_finder, :application_version, :token_ttl, :cache_ttl,
|
|
18
18
|
:masking_rules, :mask_hook, :logger
|
|
19
19
|
|
|
@@ -1,4 +1,6 @@
|
|
|
1
|
-
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "singleton"
|
|
2
4
|
|
|
3
5
|
module EndPointBlank
|
|
4
6
|
module Writers
|
|
@@ -14,6 +16,9 @@ module EndPointBlank
|
|
|
14
16
|
module DelayedWriter
|
|
15
17
|
MAX_QUEUE_SIZE = 1000
|
|
16
18
|
WARN_THROTTLE_SECONDS = 30
|
|
19
|
+
# Fallback thread count used when Configuration#worker_count is unset,
|
|
20
|
+
# preserving the previously-hardcoded pool size.
|
|
21
|
+
DEFAULT_WORKER_COUNT = 2
|
|
17
22
|
|
|
18
23
|
def direct_writer
|
|
19
24
|
@direct_writer ||= DirectWriter.new(url)
|
|
@@ -23,28 +28,32 @@ module EndPointBlank
|
|
|
23
28
|
@queue ||= Queue.new
|
|
24
29
|
end
|
|
25
30
|
|
|
31
|
+
def worker_count
|
|
32
|
+
EndPointBlank::Configuration.instance.worker_count || DEFAULT_WORKER_COUNT
|
|
33
|
+
end
|
|
34
|
+
|
|
26
35
|
def start_threads
|
|
27
36
|
@threads = []
|
|
28
37
|
|
|
29
|
-
|
|
38
|
+
worker_count.times do
|
|
30
39
|
@threads << Thread.new do
|
|
31
40
|
loop do
|
|
32
41
|
payload = queue.pop
|
|
33
42
|
payloads = [payload]
|
|
34
|
-
while (payload = pop_additional)
|
|
43
|
+
while (payload = pop_additional)
|
|
35
44
|
payloads << payload
|
|
36
45
|
end
|
|
37
46
|
|
|
38
47
|
payloads.compact!
|
|
39
|
-
while payloads.any?
|
|
48
|
+
while payloads.any?
|
|
40
49
|
list = payloads[0..5]
|
|
41
50
|
response = direct_writer.write(list)
|
|
42
51
|
if response.status < 299
|
|
43
52
|
on_success(response) if respond_to?(:on_success)
|
|
44
|
-
|
|
45
|
-
on_failure(response)
|
|
53
|
+
elsif respond_to?(:on_failure)
|
|
54
|
+
on_failure(response)
|
|
46
55
|
end
|
|
47
|
-
payloads
|
|
56
|
+
payloads -= list
|
|
48
57
|
end
|
|
49
58
|
end
|
|
50
59
|
end
|
metadata
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: end_point_blank
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.2.
|
|
4
|
+
version: 0.2.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Robert A. Lasch
|
|
8
8
|
bindir: exe
|
|
9
9
|
cert_chain: []
|
|
10
|
-
date:
|
|
10
|
+
date: 2026-07-10 00:00:00.000000000 Z
|
|
11
11
|
dependencies:
|
|
12
12
|
- !ruby/object:Gem::Dependency
|
|
13
13
|
name: excon
|
|
@@ -91,6 +91,8 @@ files:
|
|
|
91
91
|
- ".rspec"
|
|
92
92
|
- ".rubocop.yml"
|
|
93
93
|
- ".ruby-version"
|
|
94
|
+
- HARDENING_REPORT.md
|
|
95
|
+
- LICENSE
|
|
94
96
|
- README.md
|
|
95
97
|
- Rakefile
|
|
96
98
|
- build.sh
|
|
@@ -136,7 +138,8 @@ files:
|
|
|
136
138
|
- sig/end_point_blank_rack.rbs
|
|
137
139
|
- test.sh
|
|
138
140
|
homepage: https://github.com/EndPointBlank/end_point_blank_rails
|
|
139
|
-
licenses:
|
|
141
|
+
licenses:
|
|
142
|
+
- Nonstandard
|
|
140
143
|
metadata:
|
|
141
144
|
allowed_push_host: https://rubygems.org
|
|
142
145
|
homepage_uri: https://github.com/EndPointBlank/end_point_blank_rails
|
|
@@ -156,7 +159,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
156
159
|
- !ruby/object:Gem::Version
|
|
157
160
|
version: '0'
|
|
158
161
|
requirements: []
|
|
159
|
-
rubygems_version:
|
|
162
|
+
rubygems_version: 3.6.2
|
|
160
163
|
specification_version: 4
|
|
161
164
|
summary: Ruby/Rails client for EndPointBlank — endpoint tracking, authorization, and
|
|
162
165
|
error/request/response/log reporting.
|