barkibu-kb 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -1
- data/Gemfile.lock +8 -3
- data/README.md +77 -5
- data/barkibu-kb.gemspec +3 -0
- data/lib/barkibu-kb.rb +11 -0
- data/lib/kb/client.rb +18 -29
- data/lib/kb/connections.rb +70 -0
- data/lib/kb/instrumentation/datadog.rb +12 -0
- data/lib/kb/models/concerns/listable.rb +0 -2
- data/lib/kb/retry_policy.rb +92 -0
- data/lib/kb/version.rb +1 -1
- metadata +45 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 5edd7ddce6b2d27e72f6cafcfb8efba18c35c001efe54ce96eb7a51a83d1d8ae
|
|
4
|
+
data.tar.gz: 7d1be392c9297f09f8420258adf07ad3b69b7f57d7a6a07210ef9f250aeb193e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5427db71ffb7d349463fa04291f24106676b1761ea9e55c5cc0c0440e424dd8c5da57a70dbc17a7528f3e79a37dbe9f0ac57e80f6d735661c984eaf31b3b21a2
|
|
7
|
+
data.tar.gz: 2c92ff67fe128d0d5d9ae10e575e2df5d95d73945dbe10534aba6a6f1e493829fe09f8f0cb0613300f8b7318c73a46828139ea6a9098f6b15aa113b0f963d136
|
data/CHANGELOG.md
CHANGED
|
@@ -6,7 +6,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
8
|
## [unreleased]
|
|
9
|
-
- See diff: https://github.com/barkibu/kb-ruby/compare/v1.
|
|
9
|
+
- See diff: https://github.com/barkibu/kb-ruby/compare/v1.4.0...HEAD
|
|
10
|
+
|
|
11
|
+
## [1.4.0]
|
|
12
|
+
- Reuse connections to KB (keep-alive) with the stock faraday-net_http_persistent adapter. New runtime dependencies: `faraday-net_http_persistent ~> 1.2`, `net-http-persistent ~> 4.0` (4.0.8 accepts `connection_pool` 2.2.4 up to < 4). On by default; `KB.config.request.keep_alive = false` restores one connection per call. New `KB.config.request.idle_timeout` (default 30s, below the Heroku router's ~55s idle close).
|
|
13
|
+
- Every `KB::Client` now shares one Faraday connection (`KB::Connections`), so all model clients use one connection pool per process. Clients send full URLs and their own `x-api-key` header per request; URLs, headers and cache keys are unchanged. Settings are read when that connection is built, on the process's first KB call; `KB::Connections.reset!` applies later changes.
|
|
14
|
+
- A call with its own `read_timeout:` goes through a separate plain net_http connection, so its override never reaches the shared pool's other calls.
|
|
15
|
+
- With keep-alive, a connect timeout is `Faraday::TimeoutError` (was `Faraday::ConnectionFailed`) wrapping `Net::OpenTimeout`, and a refused connection is `Faraday::ConnectionFailed` wrapping `Net::HTTP::Persistent::Error`. Both still become `KB::Error` in model calls and are still retried for every verb: `KB::RetryPolicy.root_cause` unwraps `Net::HTTP::Persistent::Error` to the underlying `Errno`. Datadog `error.type` on connect-timeout spans changes accordingly.
|
|
16
|
+
|
|
17
|
+
## [1.3.0]
|
|
18
|
+
- Retry transport failures once (`faraday-retry`, already in the Faraday 1.10 bundle, now an explicit dependency). `KB::RetryPolicy` decides by the underlying error, not the Faraday class: failures where the request never left (`Net::OpenTimeout`, `ECONNREFUSED`, `EHOSTUNREACH`, `ENETUNREACH`, `EADDRNOTAVAIL`, `SocketError`) retry for every verb; any other transport failure (read/write timeout, reset, EOF, TLS) retries for GET/HEAD only (and not when the call raised its own `read_timeout:`, so a 30s read isn't doubled); HTTP error responses never retry. New settings `KB.config.request.retries` (default 1, 0 disables) and `retry_interval` (default 0.1s, randomized up to 2x). Worst-case latency is now two attempts' worth of phase budgets.
|
|
19
|
+
- `request.kb_client` payload gains `retries` and `retry_errors` on retried calls; the Datadog subscriber tags them as `kb.retries` / `kb.retry_errors`. The event and span cover all attempts, so a call that succeeded on retry is not an error.
|
|
20
|
+
- `KB::Listable.all` (and so `PetParent.all`, `Pet.all`, `Breed.all`, `Product.all`, `Plan.all`, `Assessment.all`) now wraps `Faraday::ConnectionFailed` in `KB::Error` like every other model call, instead of re-raising it raw. Code rescuing `Faraday::ConnectionFailed` around `.all` must rescue `KB::Error` instead; none of Funnel, Global Admin or connected_health does.
|
|
10
21
|
|
|
11
22
|
## [1.2.0]
|
|
12
23
|
- `KB::Client` emits one `request.kb_client` `ActiveSupport::Notifications` event per KB call (`KB::Client::REQUEST_EVENT`), wrapping cache lookup, connect, TLS, write, read and parsing. Payload: `verb`, `path`, `base_url`, `cache_hit` (GET only), `status`, plus ActiveSupport's `exception`/`exception_object` when the call raised. Every public method now goes through one private `perform` seam; no behaviour change (same cache keys, params and error classes).
|
data/Gemfile.lock
CHANGED
|
@@ -1,17 +1,20 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
barkibu-kb (1.
|
|
4
|
+
barkibu-kb (1.4.0)
|
|
5
5
|
activemodel (>= 4.0.2)
|
|
6
6
|
activerecord
|
|
7
7
|
activesupport (>= 3.0.0)
|
|
8
8
|
dry-configurable (~> 0.9)
|
|
9
9
|
faraday
|
|
10
10
|
faraday-net_http (~> 1.0)
|
|
11
|
+
faraday-net_http_persistent (~> 1.2)
|
|
12
|
+
faraday-retry (~> 1.0)
|
|
11
13
|
faraday_middleware
|
|
12
14
|
i18n
|
|
13
|
-
|
|
14
|
-
|
|
15
|
+
net-http-persistent (~> 4.0)
|
|
16
|
+
barkibu-kb-fake (1.4.0)
|
|
17
|
+
barkibu-kb (= 1.4.0)
|
|
15
18
|
countries
|
|
16
19
|
sinatra
|
|
17
20
|
webmock
|
|
@@ -109,6 +112,8 @@ GEM
|
|
|
109
112
|
multipart-post (2.3.0)
|
|
110
113
|
mustermann (3.0.0)
|
|
111
114
|
ruby2_keywords (~> 0.0.1)
|
|
115
|
+
net-http-persistent (4.0.8)
|
|
116
|
+
connection_pool (>= 2.2.4, < 4)
|
|
112
117
|
parallel (1.24.0)
|
|
113
118
|
parser (3.3.0.5)
|
|
114
119
|
ast (~> 2.4.1)
|
data/README.md
CHANGED
|
@@ -74,14 +74,84 @@ and write budgets stay global:
|
|
|
74
74
|
KB::Pet.kb_client.request('birthdays', filters: { month: 9, day: 22, size: 1000 }, read_timeout: 30)
|
|
75
75
|
```
|
|
76
76
|
|
|
77
|
+
#### Retries
|
|
78
|
+
|
|
79
|
+
The client retries a failed call once when the failure is in the transport, never
|
|
80
|
+
when KB answered with an HTTP error. Which calls retry depends on whether the
|
|
81
|
+
request can have reached KB (`KB::RetryPolicy`):
|
|
82
|
+
|
|
83
|
+
| Failure | Retried for |
|
|
84
|
+
|---|---|
|
|
85
|
+
| Never sent: connect/TLS timeout (`Net::OpenTimeout`), connection refused, host/network unreachable, DNS failure | every verb, POST included |
|
|
86
|
+
| Maybe sent: read/write timeout, connection reset, EOF, TLS error mid-stream | GET and HEAD only |
|
|
87
|
+
| HTTP 4xx/5xx | never |
|
|
88
|
+
|
|
89
|
+
PUT and DELETE are not retried on "maybe sent" failures: `upsert` and
|
|
90
|
+
`PetParent#merge!` are PUTs whose second run is not a no-op on KB's side, and a
|
|
91
|
+
repeated DELETE would turn a success into a 404. A call that raised its own read
|
|
92
|
+
budget (`read_timeout:` on `KB::Client#request`) is not retried on "maybe sent"
|
|
93
|
+
failures either, so a 30s birthdays read can't become 60s; failures that never
|
|
94
|
+
reached KB are still retried.
|
|
95
|
+
|
|
96
|
+
```ruby
|
|
97
|
+
# config/initializers/kb_ruby.rb
|
|
98
|
+
KB.config.request.retries = 1 # default; 0 disables retries
|
|
99
|
+
KB.config.request.retry_interval = 0.1 # default, seconds; each wait is 1x-2x this
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Like the timeouts, these are read when the process's shared KB connection is
|
|
103
|
+
built, i.e. on its first KB call, so set them in an initializer (or call
|
|
104
|
+
`KB::Connections.reset!` after changing them at runtime).
|
|
105
|
+
|
|
106
|
+
Worst case, a call now takes two attempts' worth of phase budgets plus the
|
|
107
|
+
interval, e.g. a GET that read-times-out twice takes about 2 x (1 + 3 + 5)s with
|
|
108
|
+
the default timeouts.
|
|
109
|
+
|
|
110
|
+
#### Keep-alive connections
|
|
111
|
+
|
|
112
|
+
KB calls reuse TCP/TLS connections instead of opening a new one per call, through
|
|
113
|
+
the stock `net_http_persistent` Faraday adapter (net-http-persistent). Every
|
|
114
|
+
model's client shares one Faraday connection (`KB::Connections`), so they all
|
|
115
|
+
draw on one connection pool per process; each
|
|
116
|
+
thread checks a connection out per call. A pooled connection idle for longer
|
|
117
|
+
than `idle_timeout` is closed and reopened on the next call.
|
|
118
|
+
|
|
119
|
+
```ruby
|
|
120
|
+
# config/initializers/kb_ruby.rb
|
|
121
|
+
KB.config.request.keep_alive = true # default; false opens a connection per call (net_http)
|
|
122
|
+
KB.config.request.idle_timeout = 30 # default, seconds
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Keep `idle_timeout` below the Heroku router's own idle close: it drops an idle
|
|
126
|
+
client connection after about 55 seconds (measured against KB staging,
|
|
127
|
+
2026-09-23). A connection the server already closed is noticed before the next
|
|
128
|
+
request is written, and reopened. A close that races the request in the same
|
|
129
|
+
instant can't be noticed: a GET is retried on a fresh connection (see Retries),
|
|
130
|
+
a POST fails with `Faraday::ConnectionFailed` wrapping `EOFError`.
|
|
131
|
+
|
|
132
|
+
Net::HTTP's own retry of idempotent requests stays off (`max_retries = 0`), so a
|
|
133
|
+
PUT is never resent behind `KB::RetryPolicy`'s back.
|
|
134
|
+
|
|
135
|
+
A call with its own `read_timeout:` goes through a separate plain connection,
|
|
136
|
+
opened for that call: the pooled adapter keeps one timeout setting for the whole
|
|
137
|
+
pool, where one call's override would reach other threads' calls.
|
|
138
|
+
|
|
139
|
+
Differences from `keep_alive = false`: a connect timeout surfaces as
|
|
140
|
+
`Faraday::TimeoutError` wrapping `Net::OpenTimeout` (retries classify it by that
|
|
141
|
+
underlying error, so it is still retried for every verb), and a refused
|
|
142
|
+
connection as `Faraday::ConnectionFailed` wrapping `Net::HTTP::Persistent::Error`.
|
|
143
|
+
|
|
77
144
|
#### Instrumentation
|
|
78
145
|
|
|
79
146
|
Every KB call emits one `request.kb_client` event through
|
|
80
147
|
`ActiveSupport::Notifications`, wrapping the whole call: cache lookup, TCP
|
|
81
148
|
connect, TLS, write, read and JSON parsing. The payload carries `verb`, `path`,
|
|
82
|
-
`base_url`, `cache_hit` (GET calls only), `status` (when a response arrived)
|
|
83
|
-
|
|
84
|
-
|
|
149
|
+
`base_url`, `cache_hit` (GET calls only), `status` (when a response arrived),
|
|
150
|
+
`retries` and `retry_errors` (only when the call was retried: the count, and the
|
|
151
|
+
underlying error class of each failed attempt, e.g. `["Net::OpenTimeout"]`) and
|
|
152
|
+
ActiveSupport's `exception` / `exception_object` when the call raised. The event
|
|
153
|
+
covers the whole call including retries, so `exception` is set only when every
|
|
154
|
+
attempt failed. Subscribe to it for logging, metrics or anything else:
|
|
85
155
|
|
|
86
156
|
```ruby
|
|
87
157
|
ActiveSupport::Notifications.subscribe(KB::Client::REQUEST_EVENT) do |event|
|
|
@@ -110,8 +180,10 @@ tracer's own Net::HTTP spans nest under it. It inherits the app's service
|
|
|
110
180
|
knowledge-base service either: it measures the client's whole call, not a KB
|
|
111
181
|
operation. Resources are low-cardinality (`GET /v1/pets/birthdays`,
|
|
112
182
|
`GET /v1/pets/?/contracts`). Tags: `peer.hostname` (the KB host used),
|
|
113
|
-
`kb.method`, `kb.cache_hit` (GET calls only), `http.status_code`,
|
|
114
|
-
|
|
183
|
+
`kb.method`, `kb.cache_hit` (GET calls only), `http.status_code`, `kb.retries`
|
|
184
|
+
and `kb.retry_errors` (retried calls only), plus the standard
|
|
185
|
+
`error.type`/`error.message` when the call raises. A span with `kb.retries` and
|
|
186
|
+
no error is a failure the retry absorbed.
|
|
115
187
|
|
|
116
188
|
Why not rely on the Net::HTTP tracer alone: faraday-net_http opens the socket
|
|
117
189
|
before `Net::HTTP#request`, the method that tracer patches, so a connect timeout
|
data/barkibu-kb.gemspec
CHANGED
|
@@ -53,6 +53,9 @@ Gem::Specification.new do |spec|
|
|
|
53
53
|
spec.add_runtime_dependency 'activesupport', '>= 3.0.0'
|
|
54
54
|
spec.add_runtime_dependency 'faraday'
|
|
55
55
|
spec.add_runtime_dependency 'faraday-net_http', '~> 1.0'
|
|
56
|
+
spec.add_runtime_dependency 'faraday-net_http_persistent', '~> 1.2'
|
|
56
57
|
spec.add_runtime_dependency 'faraday_middleware'
|
|
58
|
+
spec.add_runtime_dependency 'faraday-retry', '~> 1.0'
|
|
57
59
|
spec.add_runtime_dependency 'i18n'
|
|
60
|
+
spec.add_runtime_dependency 'net-http-persistent', '~> 4.0'
|
|
58
61
|
end
|
data/lib/barkibu-kb.rb
CHANGED
|
@@ -21,6 +21,15 @@ module KB
|
|
|
21
21
|
setting :connect_timeout, default: 1
|
|
22
22
|
setting :write_timeout, default: 3
|
|
23
23
|
setting :read_timeout, default: 5
|
|
24
|
+
# Retries after a transport failure, per KB::RetryPolicy. 0 disables retries.
|
|
25
|
+
setting :retries, default: 1
|
|
26
|
+
setting :retry_interval, default: 0.1
|
|
27
|
+
# Reuse connections to KB across calls (faraday-net_http_persistent, one pool
|
|
28
|
+
# per process, see KB::Connections). false opens a connection per call.
|
|
29
|
+
setting :keep_alive, default: true
|
|
30
|
+
# Seconds a pooled connection may sit idle before it is closed and reopened.
|
|
31
|
+
# Keep it below the Heroku router's idle close (about 55s, see README).
|
|
32
|
+
setting :idle_timeout, default: 30
|
|
24
33
|
end
|
|
25
34
|
end
|
|
26
35
|
|
|
@@ -29,6 +38,8 @@ require 'kb/inflections'
|
|
|
29
38
|
require 'kb/cache'
|
|
30
39
|
require 'kb/client_resolver'
|
|
31
40
|
require 'kb/errors'
|
|
41
|
+
require 'kb/retry_policy'
|
|
42
|
+
require 'kb/connections'
|
|
32
43
|
require 'kb/client'
|
|
33
44
|
|
|
34
45
|
require 'kb/concerns'
|
data/lib/kb/client.rb
CHANGED
|
@@ -2,6 +2,7 @@ module KB
|
|
|
2
2
|
class Client
|
|
3
3
|
# Emitted once per KB call, wrapping cache lookup and the HTTP request.
|
|
4
4
|
# Payload: verb, path, base_url, cache_hit (GET only), status (when a response arrived),
|
|
5
|
+
# retries / retry_errors (only when the call was retried, see KB::RetryPolicy),
|
|
5
6
|
# plus ActiveSupport's exception/exception_object when the call raised.
|
|
6
7
|
REQUEST_EVENT = 'request.kb_client'.freeze
|
|
7
8
|
|
|
@@ -74,7 +75,7 @@ module KB
|
|
|
74
75
|
end
|
|
75
76
|
|
|
76
77
|
def http(event, payload, read_timeout)
|
|
77
|
-
response =
|
|
78
|
+
response = send_request(event, payload, read_timeout)
|
|
78
79
|
event[:status] = response.status
|
|
79
80
|
response.body
|
|
80
81
|
rescue Faraday::ClientError, Faraday::ServerError => e
|
|
@@ -82,11 +83,12 @@ module KB
|
|
|
82
83
|
raise
|
|
83
84
|
end
|
|
84
85
|
|
|
85
|
-
def
|
|
86
|
-
|
|
87
|
-
'
|
|
88
|
-
|
|
89
|
-
|
|
86
|
+
def send_request(event, payload, read_timeout)
|
|
87
|
+
connection(read_timeout).public_send(event[:verb], url_for(event[:path]), payload) do |req|
|
|
88
|
+
req.headers[:'x-api-key'] = api_key # a symbol, so Faraday names it X-api-key, as the log filter expects
|
|
89
|
+
req.options.read_timeout = read_timeout if read_timeout
|
|
90
|
+
RetryPolicy.track(req, event, read_timeout)
|
|
91
|
+
end
|
|
90
92
|
end
|
|
91
93
|
|
|
92
94
|
def attributes_case_transform(attributes)
|
|
@@ -99,31 +101,18 @@ module KB
|
|
|
99
101
|
attributes_case_transform(attributes).to_json
|
|
100
102
|
end
|
|
101
103
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
if KB.config.log_level == :debugger
|
|
107
|
-
conn.response :logger do |logger|
|
|
108
|
-
logger.filter(/(X-api-key:\s)("\w+")/, '\1[API_KEY_SCRUBBED]')
|
|
109
|
-
end
|
|
110
|
-
end
|
|
111
|
-
conn.adapter :net_http
|
|
112
|
-
end
|
|
113
|
-
end
|
|
114
|
-
|
|
115
|
-
def request_options(read_timeout)
|
|
116
|
-
return nil if read_timeout.nil?
|
|
117
|
-
|
|
118
|
-
->(req) { req.options.read_timeout = read_timeout }
|
|
104
|
+
# The process-wide connection every client shares (see KB::Connections):
|
|
105
|
+
# keep-alive unless disabled, or unless this one call sets its own read_timeout.
|
|
106
|
+
def connection(read_timeout = nil)
|
|
107
|
+
Connections.fetch(keep_alive: KB.config.request.keep_alive && read_timeout.nil?)
|
|
119
108
|
end
|
|
120
109
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
110
|
+
# The URL a Faraday connection on base_url would build for `path`, by Faraday's
|
|
111
|
+
# own joining rules ("" is base_url itself, anything else is relative to
|
|
112
|
+
# base_url + "/"), so sharing one connection across clients changes no URL.
|
|
113
|
+
def url_for(path)
|
|
114
|
+
@url_builder ||= Faraday::Connection.new(url: base_url)
|
|
115
|
+
@url_builder.build_exclusive_url(path).to_s
|
|
127
116
|
end
|
|
128
117
|
end
|
|
129
118
|
end
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
require 'faraday/net_http_persistent'
|
|
2
|
+
|
|
3
|
+
module KB
|
|
4
|
+
# The Faraday connections behind every KB::Client, shared per process.
|
|
5
|
+
#
|
|
6
|
+
# Each model class has its own KB::Client (PetParent, Pet, Breed...). Sharing
|
|
7
|
+
# one Faraday connection between them means one adapter instance, so with
|
|
8
|
+
# keep-alive (the stock faraday-net_http_persistent adapter) one connection
|
|
9
|
+
# pool serves every client. Clients send full URLs and their own API key per
|
|
10
|
+
# request, so nothing on the connection is specific to one client; the pool
|
|
11
|
+
# keeps its sockets per host and port.
|
|
12
|
+
#
|
|
13
|
+
# Two connections:
|
|
14
|
+
# - keep-alive, for every call on the global timeouts;
|
|
15
|
+
# - plain net_http (a new connection per call), for calls that set their own
|
|
16
|
+
# `read_timeout:`. The persistent adapter writes each call's timeouts onto the
|
|
17
|
+
# shared pool object, where any thread's next checkout reads them, so a
|
|
18
|
+
# per-call override there would leak into other threads' calls.
|
|
19
|
+
#
|
|
20
|
+
# Built on first use from KB.config; `reset!` applies later config changes.
|
|
21
|
+
module Connections
|
|
22
|
+
MUTEX = Mutex.new
|
|
23
|
+
@connections = {}
|
|
24
|
+
|
|
25
|
+
class << self
|
|
26
|
+
def fetch(keep_alive:)
|
|
27
|
+
@connections[keep_alive] || MUTEX.synchronize { @connections[keep_alive] ||= build(keep_alive) }
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def reset!
|
|
31
|
+
MUTEX.synchronize { @connections = {} }
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
private
|
|
35
|
+
|
|
36
|
+
def build(keep_alive)
|
|
37
|
+
Faraday.new(headers: { 'Content-Type': 'application/json' }, request: timeouts) do |conn|
|
|
38
|
+
conn.request :retry, RetryPolicy.middleware_options
|
|
39
|
+
conn.response :json
|
|
40
|
+
conn.response :raise_error
|
|
41
|
+
log(conn) if KB.config.log_level == :debugger
|
|
42
|
+
adapter(conn, keep_alive)
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# The block runs on every request, on the shared Net::HTTP::Persistent.
|
|
47
|
+
# faraday-net_http's configure_request also sets max_retries = 0 there, so
|
|
48
|
+
# Net::HTTP never retries on its own: retries belong to KB::RetryPolicy.
|
|
49
|
+
def adapter(conn, keep_alive)
|
|
50
|
+
return conn.adapter(:net_http) unless keep_alive
|
|
51
|
+
|
|
52
|
+
conn.adapter(:net_http_persistent) { |http| http.idle_timeout = KB.config.request.idle_timeout }
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def log(conn)
|
|
56
|
+
conn.response :logger do |logger|
|
|
57
|
+
logger.filter(/(X-api-key:\s)("\w+")/, '\1[API_KEY_SCRUBBED]')
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def timeouts
|
|
62
|
+
{
|
|
63
|
+
open_timeout: KB.config.request.connect_timeout,
|
|
64
|
+
write_timeout: KB.config.request.write_timeout,
|
|
65
|
+
read_timeout: KB.config.request.read_timeout
|
|
66
|
+
}
|
|
67
|
+
end
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
|
@@ -72,9 +72,21 @@ module KB
|
|
|
72
72
|
|
|
73
73
|
span.set_tag('kb.cache_hit', payload[:cache_hit].to_s) if payload.key?(:cache_hit)
|
|
74
74
|
span.set_tag('http.status_code', payload[:status].to_s) if payload[:status]
|
|
75
|
+
tag_retries(span, payload)
|
|
75
76
|
span.set_error(payload[:exception_object]) if payload[:exception_object]
|
|
76
77
|
span.finish
|
|
77
78
|
end
|
|
79
|
+
|
|
80
|
+
private
|
|
81
|
+
|
|
82
|
+
# Only on retried calls: `kb.retries` (numeric) and the distinct underlying
|
|
83
|
+
# errors that triggered them, e.g. `Net::OpenTimeout`.
|
|
84
|
+
def tag_retries(span, payload)
|
|
85
|
+
return unless payload[:retries]
|
|
86
|
+
|
|
87
|
+
span.set_tag('kb.retries', payload[:retries])
|
|
88
|
+
span.set_tag('kb.retry_errors', payload[:retry_errors].uniq.join(','))
|
|
89
|
+
end
|
|
78
90
|
end
|
|
79
91
|
end
|
|
80
92
|
end
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
require 'socket'
|
|
2
|
+
require 'net/http'
|
|
3
|
+
require 'net/http/persistent'
|
|
4
|
+
require 'faraday/retry'
|
|
5
|
+
|
|
6
|
+
module KB
|
|
7
|
+
# Decides which failed KB calls the client retries.
|
|
8
|
+
#
|
|
9
|
+
# Two classes of transport failure, told apart by the underlying Ruby error
|
|
10
|
+
# rather than the Faraday class (adapters disagree on the Faraday class: the
|
|
11
|
+
# net_http adapter wraps Net::OpenTimeout as ConnectionFailed, the persistent
|
|
12
|
+
# one as TimeoutError):
|
|
13
|
+
#
|
|
14
|
+
# - never sent: TCP connect / TLS handshake did not complete, so KB cannot have
|
|
15
|
+
# seen the request. Safe to retry for every verb, POST included.
|
|
16
|
+
# - maybe sent: anything else the transport raises (read/write timeout, reset,
|
|
17
|
+
# EOF, TLS error mid-stream). KB may have processed it, so only GET/HEAD are
|
|
18
|
+
# retried. PUT/DELETE are left out on purpose: `upsert` and `merge!` are PUTs
|
|
19
|
+
# whose second run is not a no-op on KB's side, and a repeated DELETE would
|
|
20
|
+
# turn a success into a 404.
|
|
21
|
+
# A call that raised its own read budget (`read_timeout:`, e.g. 30s for
|
|
22
|
+
# birthdays) is not retried on these either, so its worst case isn't doubled.
|
|
23
|
+
#
|
|
24
|
+
# HTTP responses (4xx/5xx) are never retried: KB answered.
|
|
25
|
+
module RetryPolicy
|
|
26
|
+
NOT_SENT_ERRORS = [
|
|
27
|
+
Net::OpenTimeout,
|
|
28
|
+
Errno::ECONNREFUSED,
|
|
29
|
+
Errno::EHOSTUNREACH,
|
|
30
|
+
Errno::ENETUNREACH,
|
|
31
|
+
Errno::EADDRNOTAVAIL,
|
|
32
|
+
SocketError # DNS resolution
|
|
33
|
+
].freeze
|
|
34
|
+
TRANSPORT_ERRORS = [Faraday::ConnectionFailed, Faraday::TimeoutError, Faraday::SSLError].freeze
|
|
35
|
+
MAYBE_SENT_VERBS = %i[get head].freeze
|
|
36
|
+
|
|
37
|
+
module_function
|
|
38
|
+
|
|
39
|
+
def retry?(verb, error, own_read_budget: false)
|
|
40
|
+
return false unless TRANSPORT_ERRORS.any? { |klass| error.is_a?(klass) }
|
|
41
|
+
|
|
42
|
+
not_sent?(error) || (MAYBE_SENT_VERBS.include?(verb) && !own_read_budget)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def not_sent?(error)
|
|
46
|
+
cause = root_cause(error)
|
|
47
|
+
NOT_SENT_ERRORS.any? { |klass| cause.is_a?(klass) }
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
# The Ruby error behind a Faraday error. `wrapped_exception` is Faraday's own
|
|
51
|
+
# explicit link to it (Ruby's `cause` is only whatever was being rescued at
|
|
52
|
+
# the raise, usually the same object). Faraday's adapters wrap the Ruby error
|
|
53
|
+
# one level deep, so one level is enough, except that net-http-persistent
|
|
54
|
+
# re-raises a refused connection as its own Net::HTTP::Persistent::Error with
|
|
55
|
+
# the Errno as its `cause`.
|
|
56
|
+
def root_cause(error)
|
|
57
|
+
cause = (error.respond_to?(:wrapped_exception) && error.wrapped_exception) || error.cause || error
|
|
58
|
+
cause.is_a?(Net::HTTP::Persistent::Error) && cause.cause ? cause.cause : cause
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# Options for faraday-retry's middleware, read from KB.config.request.
|
|
62
|
+
def middleware_options
|
|
63
|
+
{
|
|
64
|
+
max: KB.config.request.retries,
|
|
65
|
+
interval: KB.config.request.retry_interval,
|
|
66
|
+
interval_randomness: 1, # 1x-2x the interval, so a burst of callers doesn't retry in lockstep
|
|
67
|
+
exceptions: TRANSPORT_ERRORS,
|
|
68
|
+
methods: [], # always ask retry_if
|
|
69
|
+
retry_if: lambda do |env, error|
|
|
70
|
+
retry?(env[:method], error, own_read_budget: env[:request].context&.dig(:kb_own_read_budget))
|
|
71
|
+
end,
|
|
72
|
+
retry_block: ->(env, _options, _retries_left, error) { record(env, error) }
|
|
73
|
+
}
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# Hands the request.kb_client event payload (and whether the call set its own
|
|
77
|
+
# read budget) to the middleware through the request context, so each retry
|
|
78
|
+
# is reported on the call's own event.
|
|
79
|
+
def track(request, event, read_timeout = nil)
|
|
80
|
+
tracking = { kb_event: event, kb_own_read_budget: !read_timeout.nil? }
|
|
81
|
+
request.options.context = (request.options.context || {}).merge(tracking)
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def record(env, error)
|
|
85
|
+
event = env[:request].context&.dig(:kb_event)
|
|
86
|
+
return unless event
|
|
87
|
+
|
|
88
|
+
event[:retries] = event.fetch(:retries, 0) + 1
|
|
89
|
+
(event[:retry_errors] ||= []) << root_cause(error).class.name
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
end
|
data/lib/kb/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: barkibu-kb
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Léo Figea
|
|
@@ -247,6 +247,20 @@ dependencies:
|
|
|
247
247
|
- - "~>"
|
|
248
248
|
- !ruby/object:Gem::Version
|
|
249
249
|
version: '1.0'
|
|
250
|
+
- !ruby/object:Gem::Dependency
|
|
251
|
+
name: faraday-net_http_persistent
|
|
252
|
+
requirement: !ruby/object:Gem::Requirement
|
|
253
|
+
requirements:
|
|
254
|
+
- - "~>"
|
|
255
|
+
- !ruby/object:Gem::Version
|
|
256
|
+
version: '1.2'
|
|
257
|
+
type: :runtime
|
|
258
|
+
prerelease: false
|
|
259
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
260
|
+
requirements:
|
|
261
|
+
- - "~>"
|
|
262
|
+
- !ruby/object:Gem::Version
|
|
263
|
+
version: '1.2'
|
|
250
264
|
- !ruby/object:Gem::Dependency
|
|
251
265
|
name: faraday_middleware
|
|
252
266
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -261,6 +275,20 @@ dependencies:
|
|
|
261
275
|
- - ">="
|
|
262
276
|
- !ruby/object:Gem::Version
|
|
263
277
|
version: '0'
|
|
278
|
+
- !ruby/object:Gem::Dependency
|
|
279
|
+
name: faraday-retry
|
|
280
|
+
requirement: !ruby/object:Gem::Requirement
|
|
281
|
+
requirements:
|
|
282
|
+
- - "~>"
|
|
283
|
+
- !ruby/object:Gem::Version
|
|
284
|
+
version: '1.0'
|
|
285
|
+
type: :runtime
|
|
286
|
+
prerelease: false
|
|
287
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
288
|
+
requirements:
|
|
289
|
+
- - "~>"
|
|
290
|
+
- !ruby/object:Gem::Version
|
|
291
|
+
version: '1.0'
|
|
264
292
|
- !ruby/object:Gem::Dependency
|
|
265
293
|
name: i18n
|
|
266
294
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -275,6 +303,20 @@ dependencies:
|
|
|
275
303
|
- - ">="
|
|
276
304
|
- !ruby/object:Gem::Version
|
|
277
305
|
version: '0'
|
|
306
|
+
- !ruby/object:Gem::Dependency
|
|
307
|
+
name: net-http-persistent
|
|
308
|
+
requirement: !ruby/object:Gem::Requirement
|
|
309
|
+
requirements:
|
|
310
|
+
- - "~>"
|
|
311
|
+
- !ruby/object:Gem::Version
|
|
312
|
+
version: '4.0'
|
|
313
|
+
type: :runtime
|
|
314
|
+
prerelease: false
|
|
315
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
316
|
+
requirements:
|
|
317
|
+
- - "~>"
|
|
318
|
+
- !ruby/object:Gem::Version
|
|
319
|
+
version: '4.0'
|
|
278
320
|
description: A wrapper of Barkibu's Knowledge Base Endpoint to make those entities
|
|
279
321
|
and their respective CRUD operations available to a ruby app
|
|
280
322
|
email:
|
|
@@ -311,6 +353,7 @@ files:
|
|
|
311
353
|
- lib/kb/client_resolver.rb
|
|
312
354
|
- lib/kb/concerns.rb
|
|
313
355
|
- lib/kb/concerns/as_kb_wrapper.rb
|
|
356
|
+
- lib/kb/connections.rb
|
|
314
357
|
- lib/kb/errors.rb
|
|
315
358
|
- lib/kb/errors/client_error.rb
|
|
316
359
|
- lib/kb/errors/conflict_error.rb
|
|
@@ -352,6 +395,7 @@ files:
|
|
|
352
395
|
- lib/kb/models/referral.rb
|
|
353
396
|
- lib/kb/models/search_result.rb
|
|
354
397
|
- lib/kb/models/symptom.rb
|
|
398
|
+
- lib/kb/retry_policy.rb
|
|
355
399
|
- lib/kb/type/array_of_conditions_type.rb
|
|
356
400
|
- lib/kb/type/array_of_strings_type.rb
|
|
357
401
|
- lib/kb/type/array_of_symptoms_type.rb
|