standard_circuit 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ac355e3881c42900c272ba14d433216f4d8e6019e003e4b5bd52f84545c11c79
4
- data.tar.gz: ffdb732b389d5b752deef02d6cdc82fd148af38e3e861d046876cce329e50d36
3
+ metadata.gz: 60bd3d9dbd17cad5bdf8062be096c8f7b1d8baa00f167e25f1ea96581a078cca
4
+ data.tar.gz: 5abdcf7eb5bf00b65bc0b724d3ced093f71f6ee1cb59a8d594c6a8d3944998e1
5
5
  SHA512:
6
- metadata.gz: e2ffa97cfc68660ca8e32bd606df2251700f619f728969ce0e27ed485b6887eb6d8ecd6f1b641a0cea9d22c4842c3a5041a25b00bac97a84c37ea00ca78adf48
7
- data.tar.gz: 8f245331cafaeb355b33e0ddd515bb23824713826d908544533959cd1f2dcb6c21d0160887370dadeef566a495d0474f66603095c0e1d93d1275f67c35f9f27d
6
+ metadata.gz: f88dc067ff12e4651a0f6f5317d55f22f5a9b78507d0bc31bf7fc737fa2dc3a7efa09853b64bb89f3fb608660eb4da79b126921d90a87e7ad9396bcb81cb317b
7
+ data.tar.gz: 1ed24fba0bdd0e373dfbe4cc4a18cc9e22675db527c109b151395ba2681d403ea64dbf0a12672b53250edfd8886aa66be1cedd42a36099833c23562a828b89c0
data/CHANGELOG.md CHANGED
@@ -2,8 +2,50 @@
2
2
 
3
3
  All notable changes to this project will be documented in this file.
4
4
 
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
5
8
  ## [Unreleased]
6
9
 
10
+ ## [0.4.0] - 2026-09-24
11
+
12
+ A developer-experience release that moves code copy-pasted across the five consumer apps into the gem. Everything is additive: behaviour is unchanged unless you opt in, and deprecated APIs keep working until 0.5.
13
+
14
+ ### Added
15
+ - **`ErrorTaxonomies::Postmark` / `AdapterErrors::Postmark`.** `server_errors` is `[Postmark::HttpServerError, Postmark::TimeoutError]` and `caller_errors` is `[Postmark::ApiInputError, Postmark::InvalidApiKeyError]`. Both return `[]` when the postmark gem isn't loaded. `ErrorTaxonomies.default_skipped_for` now also covers Postmark, because `ApiInputError` and `InvalidApiKeyError` subclass the tracked `HttpServerError`. A circuit that tracks `HttpServerError` and has no `skipped_errors:` now skips them by default, the same way AWS caller errors are skipped since 0.3.1. None of the consumers is affected: all three Postmark circuits pass `skipped_errors:` explicitly.
16
+ - **`Config#register_preset(:postmark | :s3, name:, **overrides)`.** It replaces the identical `:postmark` registrations in jumpdrive-web, fundbright-web and luminality-web, and the identical `register_prefix(:s3, ...)` in luminality-web, sidekick-web and nutripod-web. Each preset requires its SDK first (`postmark`, `aws-sdk-s3`) and raises `ArgumentError` if the SDK is missing. This matters when `aws-sdk-s3` is declared `require: false`: in that case `ErrorTaxonomies::Aws.tracked` evaluated at configure time contains only network errors. There is deliberately no `:stripe` preset, because the apps' Stripe registrations differ.
17
+ - **Opt-in `config.mailer_retry`** (`true`, or `{ wait:, attempts:, jitter: }` over the defaults `90 / 5 / 0.15`; off by default). It installs `retry_on StandardCircuit::Mailer::CircuitOpenError` on `ActionMailer::MailDeliveryJob`. The install is idempotent and reload-safe, and it stands down if a `CircuitOpenError` handler is already present. On exhaustion it writes an error log line, sends a Sentry `:error` event and emits a new `standard_circuit.mailer.retries_exhausted` event, all carrying recipient domains only. It replaces `config/initializers/mail_delivery_retry.rb` in jumpdrive-web, sidekick-web and nutripod-web (their provider-specific retries and discards stay in the apps), and the `CircuitOpenError` `retry_on` in fundbright-web's `ApplicationMailDeliveryJob`. The Sentry fingerprint is new, so the first exhaustion after switching opens a new issue.
18
+ - `StandardCircuit.deprecator`, registered as `Rails.application.deprecators[:standard_circuit]`.
19
+ - `Config#add_notifier(notifier, key:)`: an optional `key:` for registering several instances of one notifier class (see Changed).
20
+ - README sections: presets, mail delivery and `mailer_retry`, calling `configure` more than once, supported test API (`force_open`, `force_closed`, `reset_force!`, `reset!`, `standard_circuit/rspec`), and deprecations. Each feature has a "replace your host code with" snippet.
21
+
22
+ ### Changed
23
+ - **`StandardCircuit::HealthController` is autoloaded by the engine.** It moved to `app/controllers/standard_circuit/health_controller.rb`, so drawing `get "/health", to: "standard_circuit/health#show"` is all a host needs. The boot probe spec now covers autoload, eager load and both legacy `require` placements. The install generator's `--with-health-endpoint` no longer writes `config/initializers/standard_circuit_health.rb`, which only ever contained the require; it prints the route hint.
24
+ - **Repeated `configure` calls no longer stack notifiers.** Every consumer calls `configure` from `to_prepare`, which re-runs on every code reload. `add_notifier` now replaces an existing entry with the same identity in place: the same explicit `key:`, otherwise the same class name, or for procs and methods the same source location. The newest instance wins. `extra_notifiers` is deliberately not reset on each `configure`, because apps split configuration across several calls and a reset would silently drop notifiers added earlier.
25
+ - **gemspec declares `actionmailer` and `actionpack` (`>= 8.0`).** `require "standard_circuit"` has always loaded both, but only `railties` was declared, and railties doesn't pull in actionmailer. `activestorage` stays undeclared because the S3 service is only ever loaded by ActiveStorage's own configurator. The SDKs stay optional. The gem package now includes `app/**`.
26
+
27
+ ### Deprecated
28
+ Each item below still works and warns through `StandardCircuit.deprecator`. All are removed in 0.5.
29
+ - `require "standard_circuit/health_controller"`: remove it, the controller is autoloaded. Carried in `config/routes.rb` by jumpdrive-web, fundbright-web and nutripod-web.
30
+ - `StandardCircuit.health_snapshot` and `StandardCircuit.health_overall`: use `StandardCircuit.health_report[:circuits]` / `[:status]`. No consumer calls them.
31
+ - `StandardCircuit::AdapterErrors::Faraday.caller_errors`: `Faraday::ClientError` is never tracked, so skipping it is a no-op. No consumer calls it. (`Aws.caller_errors` is kept because `default_skipped_for` uses it.)
32
+ - `StandardCircuit::ActiveStorage::S3Service`: use `ActiveStorage::Service::StandardCircuitS3Service`, or `service: StandardCircuitS3` in storage.yml. No consumer references it.
33
+
34
+ ## [0.3.1] - 2026-09-24
35
+
36
+ Two behaviour changes — both are bug fixes, but both change numbers or breaker behaviour you may be alerting on.
37
+
38
+ ### Fixed
39
+ - **Circuit-open requests were counted twice in `<metric_prefix>.request`.** `Runner` already emits `<metric_prefix>.request{status: circuit_open}` when a call is rejected by a tripped or forced-open circuit; `ControllerSupport#handle_circuit_open` then emitted the same metric again when the `Stoplight::Error::RedLight` reached a controller. The controller-side emission (and the private `emit_circuit_open_metric` helper) is removed; fallback dispatch is unchanged. **Behaviour change:** `circuit_open` request counts for rejections that reach a controller drop by half, back to the true value — dashboards and alert thresholds tuned against the doubled number should be revisited.
40
+ - **S3 caller errors tripped the S3 breaker.** `ErrorTaxonomies::Aws.tracked` includes `Aws::Errors::ServiceError`, which is the superclass of `Aws::S3::Errors::AccessDenied` and `NoSuchKey` as well as of the dynamically generated 5xx errors, so a burst of missing-key lookups or permission errors opened the circuit. New `ErrorTaxonomies.default_skipped_for(tracked)` returns the AWS caller errors whenever the tracked list covers them, and `register` / `register_prefix` now use it as the default `skipped_errors`. **Behaviour change:** AWS circuits registered without `skipped_errors:` no longer count `AccessDenied` / `NoSuchKey` toward the threshold; 5xx (`ServiceUnavailable` etc.) and `Seahorse::Client::NetworkingError` still trip. An explicit `skipped_errors:` — including `[]` — always wins, and circuits for other adapters keep defaulting to `[]`.
41
+
42
+ ### Added
43
+ - Specs for `NetworkErrors` and the Stripe / AWS / Faraday / SMTP `AdapterErrors` modules.
44
+ - README section on error taxonomies and the AWS `skipped_errors` default; the initializer template notes it too.
45
+
46
+ ### Documentation
47
+ - **Consumer list corrected in `CLAUDE.md`: this gem has five consumers, not four.** `sidekick-web` was missing. The same entry also claimed the `workspace-os` → `jumpdrive-web` directory rename "was deferred" and pointed at `~/Workspace/rarebit-one/workspace-os`; that rename completed 2026-07-14 and the old husk is gone, so an agent following the note was looking in a directory that no longer exists. Verified against the canonical matrix in the workspace's `rollout-gem/SKILL.md`, which the new advisory `check-gem-family-drift.sh` now diffs this list against on every sweep.
48
+
7
49
  ## [0.3.0] - 2026-07-30
8
50
 
9
51
  ### Added
data/README.md CHANGED
@@ -5,10 +5,10 @@ Circuit breaker primitives for Rails apps, built on [stoplight](https://github.c
5
5
  Wraps the upstream `stoplight` gem with:
6
6
 
7
7
  - Opinionated default error taxonomy (network errors track; caller/config errors do not)
8
- - SDK-specific adapter error bundles (Stripe, AWS, Faraday, SMTP)
8
+ - SDK-specific adapter error bundles (Stripe, AWS, Faraday, SMTP, Postmark) and circuit presets (`:postmark`, `:s3`)
9
9
  - Rails event emission (`standard_circuit.circuit.{opened,closed,degraded,fallback_invoked,registered}`) with built-in Logger, Sentry, and Sentry::Metrics subscribers
10
10
  - ActiveStorage S3 adapter with per-bucket circuit keying
11
- - Generic ActionMailer delivery-method wrapper (supports both instance and symbol `underlying:` forms)
11
+ - Generic ActionMailer delivery-method wrapper (supports both instance and symbol `underlying:` forms) with opt-in `mailer_retry` for `deliver_later` jobs
12
12
  - Controller concern for standardized 503 responses on `Stoplight::Error::RedLight`
13
13
  - Test helpers (`force_open`, `force_closed`, `reset_force!`) with RSpec auto-cleanup
14
14
 
@@ -27,10 +27,8 @@ bundle add standard_circuit
27
27
  rails g standard_circuit:install
28
28
  ```
29
29
 
30
- Pass `--with-health-endpoint` to also generate
31
- `config/initializers/standard_circuit_health.rb` (which requires the opt-in
32
- health controller); the generator prints the matching route line for you to
33
- add to `config/routes.rb`.
30
+ Pass `--with-health-endpoint` to also print the route line for the health
31
+ endpoint, for you to add to `config/routes.rb`.
34
32
 
35
33
  The generator is idempotent — re-running skips an existing initializer
36
34
  unless you pass `--force`.
@@ -58,6 +56,58 @@ StandardCircuit.run(:stripe) do
58
56
  end
59
57
  ```
60
58
 
59
+ ## Error taxonomies
60
+
61
+ `tracked_errors` decide what counts toward tripping a circuit; `skipped_errors` are re-raised without counting (and win over `tracked_errors` when a class matches both). `StandardCircuit::ErrorTaxonomies::<Adapter>.tracked` combines `NetworkErrors.defaults` with the adapter's server-side errors for `Stripe`, `Smtp`, `Aws`, `Faraday`, and `Postmark`; `StandardCircuit::AdapterErrors::<Adapter>.caller_errors` lists the adapter's caller-side (4xx-style) errors.
62
+
63
+ When `skipped_errors:` is omitted it defaults to `[]` — except when the tracked list covers the AWS or Postmark caller errors. (Postmark's `ApiInputError` and `InvalidApiKeyError` subclass `Postmark::HttpServerError`, so a circuit tracking `ErrorTaxonomies::Postmark.tracked` without `skipped_errors:` defaults to skipping them.) AWS 5xx responses are dynamically generated `Aws::Errors::ServiceError` subclasses, so `ErrorTaxonomies::Aws.tracked` has to track `ServiceError` itself, which is also the superclass of `Aws::S3::Errors::AccessDenied` and `NoSuchKey`. So for such circuits `skipped_errors` defaults to `AdapterErrors::Aws.caller_errors` (via `ErrorTaxonomies.default_skipped_for`), and a burst of missing-key lookups or permission errors no longer trips the S3 breaker:
64
+
65
+ ```ruby
66
+ StandardCircuit.configure do |c|
67
+ # skipped_errors defaults to [Aws::S3::Errors::NoSuchKey, Aws::S3::Errors::AccessDenied]
68
+ c.register_prefix(:s3, tracked_errors: StandardCircuit::ErrorTaxonomies::Aws.tracked)
69
+
70
+ # An explicit skipped_errors — even [] — always wins over the default.
71
+ # c.register(:s3_strict, tracked_errors: StandardCircuit::ErrorTaxonomies::Aws.tracked, skipped_errors: [])
72
+ end
73
+ ```
74
+
75
+ ## Presets
76
+
77
+ Some integrations were registered identically, line for line, in several apps. `c.register_preset` registers them with the shared settings; any `register` option you pass alongside wins, and `name:` renames the circuit.
78
+
79
+ | Preset | Registers | Settings |
80
+ |--------|-----------|----------|
81
+ | `:postmark` | `register(:postmark, ...)` | threshold 3, cool-off 60s, `:standard`, `ErrorTaxonomies::Postmark.tracked`, skips `AdapterErrors::Postmark.caller_errors` |
82
+ | `:s3` | `register_prefix(:s3, ...)` — matches the `s3_<bucket>` circuits `StandardCircuitS3` opens | threshold 3, cool-off 30s, `:standard`, `ErrorTaxonomies::Aws.tracked`, skips `NoSuchKey` / `AccessDenied` |
83
+
84
+ Each preset requires its SDK (`postmark`, `aws-sdk-s3`) before building the error lists and raises `ArgumentError` if the gem is missing. That matters for `gem "aws-sdk-s3", require: false`: `ErrorTaxonomies::Aws.tracked` returns only network errors when the SDK hasn't been loaded yet at configure time, which quietly leaves S3 5xx responses untracked.
85
+
86
+ Replace your host code with:
87
+
88
+ ```ruby
89
+ # Before — config/initializers/standard_circuit.rb
90
+ c.register(:postmark,
91
+ threshold: 3,
92
+ cool_off_time: 60,
93
+ criticality: :standard,
94
+ tracked_errors: StandardCircuit::NetworkErrors.defaults +
95
+ [ Postmark::HttpServerError, Postmark::TimeoutError ],
96
+ skipped_errors: [ Postmark::ApiInputError, Postmark::InvalidApiKeyError ])
97
+
98
+ c.register_prefix(:s3,
99
+ threshold: 3,
100
+ cool_off_time: 30,
101
+ criticality: :standard,
102
+ tracked_errors: StandardCircuit::ErrorTaxonomies::Aws.tracked)
103
+
104
+ # After
105
+ c.register_preset(:postmark)
106
+ c.register_preset(:s3)
107
+ ```
108
+
109
+ There is deliberately no `:stripe` preset — the apps that wrap Stripe disagree on threshold, criticality, and skips, so `ErrorTaxonomies::Stripe.tracked` stays the shared piece.
110
+
61
111
  ## Circuit state storage (`data_store`)
62
112
 
63
113
  Circuit state (failure counts, colors, locks) lives in a Stoplight data store. StandardCircuit defaults to `Stoplight::DataStore::Memory.new`, which is **per-process**: each Puma worker, Sidekiq/SolidQueue worker, and console gets its own independent view of every circuit.
@@ -140,6 +190,85 @@ StandardCircuit.configure do |c|
140
190
  end
141
191
  ```
142
192
 
193
+ ## Mail delivery
194
+
195
+ The `:standard_circuit` delivery method wraps any registered ActionMailer delivery method in a circuit and raises `StandardCircuit::Mailer::CircuitOpenError` (instead of attempting the send) while that circuit is open:
196
+
197
+ ```ruby
198
+ # config/environments/production.rb
199
+ config.action_mailer.delivery_method = :standard_circuit
200
+ config.action_mailer.standard_circuit_settings = {
201
+ underlying: :postmark,
202
+ underlying_settings: { api_token: ENV.fetch("POSTMARK_API_TOKEN") },
203
+ circuit: :postmark
204
+ }
205
+ ```
206
+
207
+ ### Retrying `deliver_later` while the circuit is open (`mailer_retry`)
208
+
209
+ Without a retry, a `deliver_later` job that runs during an outage fails once and the email is lost. Opt in and the gem installs `retry_on CircuitOpenError` on `ActionMailer::MailDeliveryJob`:
210
+
211
+ ```ruby
212
+ StandardCircuit.configure do |c|
213
+ c.mailer_retry = true # wait: 90, attempts: 5, jitter: 0.15
214
+ # c.mailer_retry = { wait: 120, attempts: 8 } # partial overrides; wait: takes anything retry_on does
215
+ end
216
+ ```
217
+
218
+ - **Off by default.** Nothing is installed unless you set it.
219
+ - **Keep `wait` longer than the mail circuit's `cool_off_time`** (the `:postmark` preset uses 60s), or retries land on a circuit that is still open.
220
+ - **Reload-safe and idempotent.** Calling `configure` from `to_prepare` on every reload installs the handler once. If a `retry_on CircuitOpenError` is already on the job (for example your old initializer, during migration), the gem leaves it alone. The first install wins, so changing the options later needs a restart.
221
+ - **Subclasses inherit it.** A custom `self.delivery_job = MyJob < ActionMailer::MailDeliveryJob` gets the handler, and a `retry_on CircuitOpenError` declared on the subclass takes precedence.
222
+ - **On exhaustion** it writes one `error` log line, sends a Sentry `:error` event (when `sentry_enabled` and Sentry is initialized, fingerprinted by mailer and action), and emits `standard_circuit.mailer.retries_exhausted`. Every payload holds only `mailer_class`, `mail_action`, `recipient_domains`, `job_id` and `executions`: recipient **domains**, never addresses or subjects.
223
+ - Only `CircuitOpenError` is retried. Errors from the provider while the circuit is closed, such as a SendGrid 429 or a Postmark 422, keep their existing behaviour. Add your own `retry_on` / `discard_on` for those.
224
+
225
+ Replace your host code with:
226
+
227
+ ```ruby
228
+ # Before — config/initializers/mail_delivery_retry.rb (~40–60 lines)
229
+ Rails.application.config.to_prepare do
230
+ next if ActionMailer::MailDeliveryJob.rescue_handlers.any? { |h| h.first == StandardCircuit::Mailer::CircuitOpenError.name }
231
+
232
+ ActionMailer::MailDeliveryJob.retry_on(StandardCircuit::Mailer::CircuitOpenError,
233
+ wait: 90.seconds, attempts: 5, jitter: 0.15) do |job, error|
234
+ # ...recipient-domain extraction, Rails.logger.error, Sentry.capture_message...
235
+ end
236
+ end
237
+
238
+ # After — inside your existing StandardCircuit.configure block
239
+ c.mailer_retry = true
240
+ # or, keeping ENV tunables:
241
+ c.mailer_retry = {
242
+ wait: ENV.fetch("SENDGRID_CIRCUIT_RETRY_WAIT", 90).to_i,
243
+ attempts: ENV.fetch("SENDGRID_CIRCUIT_RETRY_ATTEMPTS", 5).to_i
244
+ }
245
+ ```
246
+
247
+ The Sentry fingerprint changes to `["standard_circuit-mailer-retries-exhausted", mailer, action]`, so the first exhaustion after switching opens a new Sentry issue rather than regrouping into your old one.
248
+
249
+ ### Calling `configure` more than once
250
+
251
+ `configure` is safe to call repeatedly. Apps register circuits from `to_prepare` because it's the first point where autoloaded error classes are available, and `to_prepare` re-runs on every code reload. Each call re-applies the config and rebuilds the subscribers. `add_notifier` is idempotent: adding "the same" notifier again replaces the earlier entry in place and does not stack a duplicate. Two notifiers are the same when they share an explicit `key:`, or else the same class name (a reloaded class is a new object with the same name), or for lambdas and methods the same source location. The newest instance wins. To register two instances of one class, give them distinct keys:
252
+
253
+ ```ruby
254
+ c.add_notifier(WebhookNotifier.new(ops_url), key: :ops)
255
+ c.add_notifier(WebhookNotifier.new(audit_url), key: :audit)
256
+ ```
257
+
258
+ Replace your host code with:
259
+
260
+ ```ruby
261
+ # Before — hand-rolled guard against stacking a notifier on every reload
262
+ unless c.extra_notifiers.any? { |n| n.class.name == "CircuitAuditNotifier" }
263
+ c.add_notifier(CircuitAuditNotifier.new)
264
+ end
265
+
266
+ # After
267
+ c.add_notifier(CircuitAuditNotifier.new)
268
+ ```
269
+
270
+ Circuit registrations (`register` / `register_prefix` / `register_preset`) were already idempotent: the same name overwrites.
271
+
143
272
  ## Streaming and non-controller contexts
144
273
 
145
274
  `ControllerSupport.circuit_open_fallback` only works for non-streaming responses — once a `Live` controller has flushed any output, Rails can't render an error template over the wire. For a streaming controller, catch `Stoplight::Error::RedLight` *inside* the streaming proc and write a degraded payload before the stream closes:
@@ -174,19 +303,28 @@ Same pattern applies in background jobs (where `circuit_open_fallback` doesn't h
174
303
 
175
304
  ## Health endpoint
176
305
 
177
- StandardCircuit ships an opt-in controller that renders `StandardCircuit.health_report` as JSON. It returns 503 when the rolled-up status is `:critical` (so orchestrators pull the instance out of rotation) and 200 otherwise.
306
+ StandardCircuit ships a controller that renders `StandardCircuit.health_report` as JSON. It returns 503 when the rolled-up status is `:critical` (so orchestrators pull the instance out of rotation) and 200 otherwise.
178
307
 
179
- It's opt-in not auto-required so apps that don't want a health route don't pay for it.
308
+ The controller lives in the engine's `app/controllers`, so it is autoloaded. The route is the only opt-in, and apps that don't draw it never load the controller.
180
309
 
181
310
  ```ruby
182
311
  # config/routes.rb
183
- require "standard_circuit/health_controller"
184
-
185
312
  Rails.application.routes.draw do
186
313
  get "/health", to: "standard_circuit/health#show"
187
314
  end
188
315
  ```
189
316
 
317
+ Replace your host code with:
318
+
319
+ ```ruby
320
+ # Before — top of config/routes.rb (or config/initializers/standard_circuit_health.rb)
321
+ require "standard_circuit/health_controller"
322
+
323
+ # After — delete the line (and the initializer, if that's all it contained).
324
+ ```
325
+
326
+ The old `require` still works in 0.4 but emits a deprecation through `Rails.application.deprecators[:standard_circuit]`. It will be removed in 0.5.
327
+
190
328
  The controller inherits from `ActionController::API` to sidestep app-level filters (authentication, bootstrap redirects, etc.) so probes can call it anonymously.
191
329
 
192
330
  **If your app also mounts `StandardHealth::Engine` at `/health`, draw the aggregate route first:**
@@ -198,6 +336,32 @@ mount StandardHealth::Engine => "/health", as: :standard_health
198
336
 
199
337
  `StandardHealth::Engine` registers sub-paths only (`/alive`, `/ready`, `/diagnostics/env`) — it never serves the aggregate tier itself. An app that mounts the engine and assumes `/health` is covered silently has no aggregate tier at all, with no boot error and no failing route spec to reveal it. The ordering is load-bearing; draw the aggregate route explicitly, first.
200
338
 
339
+ ## Test API
340
+
341
+ These are supported public API for host test suites, not internals:
342
+
343
+ | Method | What it does |
344
+ |--------|--------------|
345
+ | `StandardCircuit.force_open(name) { ... }` | Treat `name` as open for the block (or until `reset_force!` without a block): `run` raises `Stoplight::Error::RedLight`, or returns the fallback. Emits the same `run.completed` / `fallback_invoked` events as a real open circuit. |
346
+ | `StandardCircuit.force_closed(name) { ... }` | Bypass the circuit for the block — `run` just yields. No events. |
347
+ | `StandardCircuit.reset_force!` | Clear every forced state. |
348
+ | `StandardCircuit.reset!` | Clear the light cache and forced states, and swap in a fresh `Memory` data store (left alone when the store is Redis). |
349
+ | `require "standard_circuit/rspec"` | Adds a `before(:each)` that runs `reset!` and tears down subscribers. Circuit registrations are kept. |
350
+
351
+ All of these are **process-local**. They are test and console tools, not an operational kill switch (see `data_store` above).
352
+
353
+ ## Deprecations
354
+
355
+ 0.4 deprecates the following. Each still works and warns through `Rails.application.deprecators[:standard_circuit]`, so your app's `config.active_support.deprecation` setting applies. They will be removed in 0.5.
356
+
357
+ | Deprecated | Use instead |
358
+ |------------|-------------|
359
+ | `require "standard_circuit/health_controller"` | nothing — the controller is autoloaded |
360
+ | `StandardCircuit.health_snapshot` | `StandardCircuit.health_report[:circuits]` |
361
+ | `StandardCircuit.health_overall` | `StandardCircuit.health_report[:status]` |
362
+ | `StandardCircuit::AdapterErrors::Faraday.caller_errors` | drop it — `Faraday::ClientError` is never tracked, so skipping it is a no-op |
363
+ | `StandardCircuit::ActiveStorage::S3Service` | `ActiveStorage::Service::StandardCircuitS3Service` / `service: StandardCircuitS3` |
364
+
201
365
  ## License
202
366
 
203
367
  MIT
@@ -0,0 +1,28 @@
1
+ module StandardCircuit
2
+ # Health-check controller. Renders +StandardCircuit.health_report+ as JSON
3
+ # and returns 503 when the rolled-up status is +:critical+ so upstream
4
+ # orchestrators pull the instance out of rotation. :degraded and :ok both
5
+ # return 200 — the app can still serve most traffic.
6
+ #
7
+ # Autoloaded by the engine (it lives under app/controllers), so a host only
8
+ # draws the route — no `require` needed:
9
+ #
10
+ # # config/routes.rb
11
+ # Rails.application.routes.draw do
12
+ # get "/health", to: "standard_circuit/health#show"
13
+ # end
14
+ #
15
+ # The pre-0.4 `require "standard_circuit/health_controller"` still works but
16
+ # is deprecated (see lib/standard_circuit/health_controller.rb).
17
+ #
18
+ # Inherits from +ActionController::API+ to sidestep any ApplicationController
19
+ # filters (authentication, bootstrap redirects, etc.) — health probes must be
20
+ # callable anonymously from load balancers and uptime monitors.
21
+ class HealthController < ::ActionController::API
22
+ def show
23
+ report = StandardCircuit.health_report
24
+ http_status = report[:status] == :critical ? :service_unavailable : :ok
25
+ render json: report, status: http_status
26
+ end
27
+ end
28
+ end
@@ -7,10 +7,10 @@ module StandardCircuit
7
7
  # By default, writes config/initializers/standard_circuit.rb with
8
8
  # commented-out examples covering the public Config DSL.
9
9
  #
10
- # When +--with-health-endpoint+ is passed, also writes
11
- # config/initializers/standard_circuit_health.rb (which requires the
12
- # opt-in HealthController) and prints the route line the host should
13
- # add to config/routes.rb to expose the endpoint.
10
+ # When +--with-health-endpoint+ is passed, also prints the route line the
11
+ # host should add to config/routes.rb to expose the endpoint. (Before 0.4
12
+ # it also wrote config/initializers/standard_circuit_health.rb to require
13
+ # the controller; the engine now autoloads it, so no file is needed.)
14
14
  #
15
15
  # Idempotent: re-running on an existing initializer logs and skips. Pass
16
16
  # +--force+ to overwrite.
@@ -23,17 +23,16 @@ module StandardCircuit
23
23
  covering circuit registration, prefix registration, and notifier
24
24
  wiring.
25
25
 
26
- Pass --with-health-endpoint to also write
27
- config/initializers/standard_circuit_health.rb (which requires the
28
- opt-in HealthController) and print the route line to add to
29
- config/routes.rb.
26
+ Pass --with-health-endpoint to also print the route line to add to
27
+ config/routes.rb. The health controller is autoloaded by the engine,
28
+ so the route is all you need.
30
29
 
31
30
  The generator is idempotent — already-installed initializers are
32
31
  skipped with a clear message. Pass --force to overwrite.
33
32
  DESC
34
33
 
35
34
  class_option :with_health_endpoint, type: :boolean, default: false,
36
- desc: "Also create config/initializers/standard_circuit_health.rb and print the route hint"
35
+ desc: "Also print the config/routes.rb line for the health endpoint"
37
36
 
38
37
  def create_initializer_file
39
38
  path = "config/initializers/standard_circuit.rb"
@@ -45,25 +44,15 @@ module StandardCircuit
45
44
  template "initializer.rb.tt", path
46
45
  end
47
46
 
48
- def create_health_initializer_file
49
- return unless options[:with_health_endpoint]
50
-
51
- path = "config/initializers/standard_circuit_health.rb"
52
- if File.exist?(File.join(destination_root, path)) && !options[:force]
53
- say_status("skip", "#{path} already present, skipping (use --force to overwrite)", :yellow)
54
- else
55
- template "health_initializer.rb.tt", path
56
- end
57
- end
58
-
59
47
  def print_health_route_hint
60
48
  return unless options[:with_health_endpoint]
61
49
 
62
50
  say ""
63
51
  say "=" * 79
64
- say "StandardCircuit health endpoint installed."
52
+ say "StandardCircuit health endpoint"
65
53
  say ""
66
- say "Add the following to config/routes.rb to expose the endpoint:"
54
+ say "Add the following to config/routes.rb to expose the endpoint (the"
55
+ say "controller is autoloaded by the engine — no require needed):"
67
56
  say ""
68
57
  say ' get "/health", to: "standard_circuit/health#show"'
69
58
  say ""
@@ -37,6 +37,14 @@ StandardCircuit.configure do |config|
37
37
  # Default: nil
38
38
  # config.logger = Rails.logger
39
39
 
40
+ # Retry ActionMailer::MailDeliveryJob on
41
+ # StandardCircuit::Mailer::CircuitOpenError while the mail circuit is open.
42
+ # Keep `wait` longer than the mail circuit's cool_off_time. On exhaustion:
43
+ # error log + Sentry :error (recipient domains only).
44
+ # true -> { wait: 90, attempts: 5, jitter: 0.15 }
45
+ # Default: nil (off)
46
+ # config.mailer_retry = true
47
+
40
48
  # Stoplight data store. Default: in-memory (per-process). Use a shared
41
49
  # store (e.g. Stoplight::DataStore::Redis) when you want circuit state
42
50
  # shared across processes/hosts.
@@ -54,7 +62,10 @@ StandardCircuit.configure do |config|
54
62
  # tracked_errors — error classes that count toward the threshold
55
63
  # (default: StandardCircuit::NetworkErrors.defaults)
56
64
  # skipped_errors — error classes that bypass the circuit (re-raised
57
- # without counting — typically caller/validation errors)
65
+ # without counting — typically caller/validation errors;
66
+ # default: [], or the AWS / Postmark caller errors
67
+ # when tracked_errors covers them — see
68
+ # StandardCircuit::ErrorTaxonomies.default_skipped_for)
58
69
  # criticality — :critical | :standard | :optional
59
70
  # (affects health rollup; default: :standard)
60
71
 
@@ -69,12 +80,20 @@ StandardCircuit.configure do |config|
69
80
  # tracked_errors: StandardCircuit::ErrorTaxonomies::Smtp.tracked,
70
81
  # criticality: :standard)
71
82
 
83
+ # Presets for integrations with shared settings (any option passed wins):
84
+ # :postmark — register(:postmark, ...) with the Postmark taxonomy + skips
85
+ # :s3 — register_prefix(:s3, ...) for `service: StandardCircuitS3`
86
+ # config.register_preset(:postmark)
87
+ # config.register_preset(:s3)
88
+
72
89
  # ---------------------------------------------------------------------------
73
90
  # Prefix registration
74
91
  # ---------------------------------------------------------------------------
75
92
  # Use `register_prefix` to dynamically name circuits matching a pattern.
76
93
  # The first matching prefix wins; the circuit name is "<prefix>_<suffix>".
77
94
 
95
+ # AWS caller errors (NoSuchKey, AccessDenied) are skipped by default here,
96
+ # so missing keys and permission errors don't trip the S3 breaker.
78
97
  # config.register_prefix(:s3,
79
98
  # threshold: 10,
80
99
  # tracked_errors: StandardCircuit::ErrorTaxonomies::Aws.tracked,
@@ -91,11 +110,14 @@ StandardCircuit.configure do |config|
91
110
  # contract as the built-in subscribers.
92
111
 
93
112
  # config.add_notifier(->(name, payload) { MyTracer.record(name, payload) })
113
+ # Idempotent across repeated configure calls (e.g. from to_prepare):
114
+ # re-adding the same notifier replaces it rather than stacking a duplicate.
94
115
 
95
116
  # ---------------------------------------------------------------------------
96
117
  # Health endpoint (see --with-health-endpoint)
97
118
  # ---------------------------------------------------------------------------
98
119
  # The aggregate route is `get "/health", to: "standard_circuit/health#show"`.
120
+ # The controller is autoloaded by the engine — no require needed.
99
121
  # If your app also mounts StandardHealth::Engine at "/health", draw the
100
122
  # aggregate route BEFORE the mount. That engine registers sub-paths only
101
123
  # ("/alive", "/ready", "/diagnostics/env") and never serves the aggregate tier
@@ -35,7 +35,15 @@ module ActiveStorage
35
35
  end
36
36
 
37
37
  module StandardCircuit
38
+ # @deprecated The StandardCircuit::ActiveStorage::S3Service alias is unused —
39
+ # storage.yml resolves `service: StandardCircuitS3` to
40
+ # ActiveStorage::Service::StandardCircuitS3Service — and is removed in 0.5.
38
41
  module ActiveStorage
39
- S3Service = ::ActiveStorage::Service::StandardCircuitS3Service
42
+ include ::ActiveSupport::Deprecation::DeprecatedConstantAccessor
43
+
44
+ deprecate_constant "S3Service", "::ActiveStorage::Service::StandardCircuitS3Service",
45
+ deprecator: StandardCircuit.deprecator,
46
+ message: "StandardCircuit::ActiveStorage::S3Service is deprecated and will be removed in 0.5; " \
47
+ "use ActiveStorage::Service::StandardCircuitS3Service (or `service: StandardCircuitS3` in storage.yml)."
40
48
  end
41
49
  end
@@ -10,7 +10,15 @@ module StandardCircuit
10
10
  errors.select { |klass| klass.is_a?(Class) }
11
11
  end
12
12
 
13
+ # @deprecated Removed in 0.5. Faraday::ClientError (4xx) is not in
14
+ # +server_errors+, so it never counts toward a Faraday circuit —
15
+ # listing it in +skipped_errors+ is a no-op. Unused by every consumer.
13
16
  def caller_errors
17
+ StandardCircuit.deprecator.warn(
18
+ "StandardCircuit::AdapterErrors::Faraday.caller_errors is deprecated and will be removed in 0.5: " \
19
+ "Faraday::ClientError is never tracked by ErrorTaxonomies::Faraday.tracked, so skipping it is a no-op. " \
20
+ "Drop it from skipped_errors."
21
+ )
14
22
  return [] unless defined?(::Faraday::ClientError)
15
23
 
16
24
  [ ::Faraday::ClientError ]
@@ -0,0 +1,31 @@
1
+ module StandardCircuit
2
+ module AdapterErrors
3
+ # Postmark (the `postmark` gem, usually via `postmark-rails`).
4
+ #
5
+ # Guarded like the other adapters: every method returns `[]` when the
6
+ # postmark gem isn't loaded, so this file is safe to require in apps that
7
+ # don't send mail through Postmark.
8
+ #
9
+ # Postmark's error hierarchy is the reason `caller_errors` matters here:
10
+ # `ApiInputError` (422 — invalid payload, inactive recipient, 429-style
11
+ # throttling) and `InvalidApiKeyError` (401 — rotated / revoked token) both
12
+ # subclass `HttpServerError`, and Stoplight matches tracked errors with
13
+ # `is_a?`. Tracking `HttpServerError` without skipping them would let a
14
+ # burst of bad payloads or a config bug trip the breaker.
15
+ module Postmark
16
+ class << self
17
+ def server_errors
18
+ return [] unless defined?(::Postmark::HttpServerError)
19
+
20
+ [ ::Postmark::HttpServerError, ::Postmark::TimeoutError ]
21
+ end
22
+
23
+ def caller_errors
24
+ return [] unless defined?(::Postmark::HttpServerError)
25
+
26
+ [ ::Postmark::ApiInputError, ::Postmark::InvalidApiKeyError ]
27
+ end
28
+ end
29
+ end
30
+ end
31
+ end
@@ -5,6 +5,10 @@ module StandardCircuit
5
5
  DEFAULT_WINDOW = 60
6
6
  DEFAULT_CRITICALITY = :standard
7
7
  CRITICALITIES = [ :critical, :standard, :optional ].freeze
8
+ # 90s > the :postmark preset's 60s cool-off, so a retry lands after the
9
+ # breaker has had a chance to half-open. 5 attempts ≈ 7.5 minutes; 15%
10
+ # jitter so a backlog queued during an outage doesn't retry in lockstep.
11
+ MAILER_RETRY_DEFAULTS = { wait: 90, attempts: 5, jitter: 0.15 }.freeze
8
12
 
9
13
  CircuitSpec = Struct.new(
10
14
  :threshold,
@@ -22,19 +26,24 @@ module StandardCircuit
22
26
  "invalid criticality #{criticality.inspect}; must be one of #{CRITICALITIES.inspect}"
23
27
  end
24
28
 
29
+ tracked_errors = opts.fetch(:tracked_errors, NetworkErrors.defaults)
30
+
25
31
  new(
26
32
  threshold: opts.fetch(:threshold, DEFAULT_THRESHOLD),
27
33
  cool_off_time: opts.fetch(:cool_off_time, DEFAULT_COOL_OFF),
28
34
  window_size: opts.fetch(:window_size, DEFAULT_WINDOW),
29
- tracked_errors: opts.fetch(:tracked_errors, NetworkErrors.defaults),
30
- skipped_errors: opts.fetch(:skipped_errors, []),
35
+ tracked_errors: tracked_errors,
36
+ # An explicit `skipped_errors:` (including `[]`) always wins; the
37
+ # default only kicks in when the key is absent. See
38
+ # ErrorTaxonomies.default_skipped_for for why AWS needs one.
39
+ skipped_errors: opts.fetch(:skipped_errors) { ErrorTaxonomies.default_skipped_for(tracked_errors) },
31
40
  criticality: criticality
32
41
  )
33
42
  end
34
43
  end
35
44
 
36
45
  attr_accessor :sentry_enabled, :metric_prefix, :data_store, :logger
37
- attr_reader :circuits, :prefixes, :extra_notifiers, :sentry_criticality_levels
46
+ attr_reader :circuits, :prefixes, :extra_notifiers, :sentry_criticality_levels, :mailer_retry
38
47
 
39
48
  def initialize
40
49
  @sentry_enabled = true
@@ -45,6 +54,22 @@ module StandardCircuit
45
54
  @circuits = {}
46
55
  @prefixes = {}
47
56
  @extra_notifiers = []
57
+ @extra_notifier_keys = []
58
+ @mailer_retry = nil
59
+ end
60
+
61
+ # Opt in to retrying ActionMailer::MailDeliveryJob on
62
+ # StandardCircuit::Mailer::CircuitOpenError (see Mailer::Retry). Accepts:
63
+ #
64
+ # nil / false — (default) off; the gem installs nothing.
65
+ # true — MAILER_RETRY_DEFAULTS (wait: 90, attempts: 5, jitter: 0.15)
66
+ # Hash — MAILER_RETRY_DEFAULTS merged with :wait / :attempts /
67
+ # :jitter. `wait:` takes anything `retry_on` does (seconds,
68
+ # a Duration, :polynomially_longer, a Proc).
69
+ #
70
+ # The reader returns nil or a frozen, complete Hash.
71
+ def mailer_retry=(value)
72
+ @mailer_retry = normalize_mailer_retry(value)
48
73
  end
49
74
 
50
75
  # Opt in to criticality-aware Sentry reporting for the built-in Sentry
@@ -67,6 +92,7 @@ module StandardCircuit
67
92
  @circuits.clear
68
93
  @prefixes.clear
69
94
  @extra_notifiers.clear
95
+ @extra_notifier_keys.clear
70
96
  end
71
97
 
72
98
  def register(name, **opts)
@@ -89,16 +115,53 @@ module StandardCircuit
89
115
  spec
90
116
  end
91
117
 
118
+ # Register a circuit from a named preset (see StandardCircuit::Presets):
119
+ #
120
+ # c.register_preset(:postmark) # c.register(:postmark, ...)
121
+ # c.register_preset(:s3) # c.register_prefix(:s3, ...)
122
+ # c.register_preset(:postmark, name: :mail, threshold: 5)
123
+ #
124
+ # +name:+ overrides the circuit name (or prefix); any other keyword is a
125
+ # `register` option that wins over the preset's value.
126
+ def register_preset(preset, name: preset, **overrides)
127
+ scope, opts = Presets.resolve(preset, **overrides)
128
+ scope == :prefix ? register_prefix(name, **opts) : register(name, **opts)
129
+ end
130
+
92
131
  # Register a host-supplied subscriber. Subscribers must respond to
93
132
  # `call(event_name, payload)` — Stoplight-shaped 4-arg notifiers from the
94
133
  # 0.1.x API are no longer accepted as extras (Logger / Sentry / Metrics
95
134
  # demonstrate the new shape).
96
- def add_notifier(notifier)
135
+ #
136
+ # Idempotent: hosts call `configure` from `to_prepare`, which re-runs on
137
+ # every code reload, so re-adding "the same" notifier replaces the earlier
138
+ # entry in place instead of stacking a duplicate. "The same" means the same
139
+ # +key:+ when one is given, otherwise:
140
+ #
141
+ # * a Proc / Method — same source location (a lambda re-created on reload)
142
+ # * a Class / Module used directly — same name
143
+ # * any other object — same class name (a reloaded class is a new class
144
+ # object with the same name, so compare names, not classes)
145
+ # * an instance of an anonymous class — never deduped
146
+ #
147
+ # The newest instance wins, so a reloaded notifier class takes effect.
148
+ # Pass distinct +key:+ values to register two instances of one class
149
+ # (e.g. two webhook notifiers pointed at different URLs).
150
+ def add_notifier(notifier, key: nil)
97
151
  unless notifier.respond_to?(:call)
98
152
  raise ArgumentError,
99
153
  "extra notifiers must respond to `call(event_name, payload)`; got #{notifier.class}"
100
154
  end
101
- @extra_notifiers << notifier
155
+
156
+ identity = key.nil? ? notifier_identity(notifier) : [ :key, key ]
157
+ index = identity && @extra_notifier_keys.index(identity)
158
+ if index
159
+ @extra_notifiers[index] = notifier
160
+ else
161
+ @extra_notifiers << notifier
162
+ @extra_notifier_keys << identity
163
+ end
164
+ notifier
102
165
  end
103
166
 
104
167
  def spec_for(name)
@@ -119,6 +182,33 @@ module StandardCircuit
119
182
  end
120
183
  end
121
184
 
185
+ def notifier_identity(notifier)
186
+ if notifier.respond_to?(:source_location) && notifier.source_location
187
+ [ :source, notifier.source_location ]
188
+ elsif notifier.is_a?(Module)
189
+ notifier.name && [ :module, notifier.name ]
190
+ else
191
+ notifier.class.name && [ :class, notifier.class.name ]
192
+ end
193
+ end
194
+
195
+ def normalize_mailer_retry(value)
196
+ case value
197
+ when nil, false then nil
198
+ when true then MAILER_RETRY_DEFAULTS
199
+ when Hash
200
+ options = value.transform_keys(&:to_sym)
201
+ unknown = options.keys - MAILER_RETRY_DEFAULTS.keys
202
+ unless unknown.empty?
203
+ raise ArgumentError,
204
+ "unknown mailer_retry option(s) #{unknown.inspect}; allowed: #{MAILER_RETRY_DEFAULTS.keys.inspect}"
205
+ end
206
+ MAILER_RETRY_DEFAULTS.merge(options).freeze
207
+ else
208
+ raise ArgumentError, "mailer_retry must be nil, true, false, or a Hash; got #{value.class}"
209
+ end
210
+ end
211
+
122
212
  def merge_sentry_criticality_levels(map)
123
213
  normalized = map.to_h do |criticality, level|
124
214
  # Check symbolizability before coercing: a key like 42 would otherwise
@@ -47,9 +47,10 @@ module StandardCircuit
47
47
 
48
48
  private
49
49
 
50
- def handle_circuit_open(error)
50
+ def handle_circuit_open(_error)
51
+ # No metric here: Runner already counted this rejection as
52
+ # `<metric_prefix>.request{status: circuit_open}` before re-raising.
51
53
  fallback = self.class._circuit_open_fallback || {}
52
- emit_circuit_open_metric(error)
53
54
 
54
55
  return instance_exec(&fallback[:json]) if request.format.json? && fallback[:json]
55
56
  return instance_exec(response, &fallback[:stream]) if streaming_controller? && fallback[:stream]
@@ -61,14 +62,5 @@ module StandardCircuit
61
62
  def streaming_controller?
62
63
  defined?(::ActionController::Live) && self.class.include?(::ActionController::Live)
63
64
  end
64
-
65
- def emit_circuit_open_metric(error)
66
- prefix = StandardCircuit.config.metric_prefix
67
- ::Sentry::Metrics.count(
68
- "#{prefix}.request",
69
- value: 1,
70
- attributes: { service: error.light_name, status: "circuit_open" }
71
- )
72
- end
73
65
  end
74
66
  end
@@ -0,0 +1,12 @@
1
+ require "active_support"
2
+ require "active_support/deprecation"
3
+
4
+ module StandardCircuit
5
+ # The gem's ActiveSupport::Deprecation instance. Registered with the host as
6
+ # `Rails.application.deprecators[:standard_circuit]` by the engine, so the
7
+ # host's `config.active_support.deprecation` behaviour (log / raise /
8
+ # silence / report) applies to StandardCircuit deprecations too.
9
+ def self.deprecator
10
+ @deprecator ||= ActiveSupport::Deprecation.new("0.5", "StandardCircuit")
11
+ end
12
+ end
@@ -23,12 +23,19 @@ module StandardCircuit
23
23
  # spec/integration/health_route_boot_spec.rb, which fails if this ever
24
24
  # regresses).
25
25
  #
26
+ # `StandardCircuit::HealthController` lives in the engine's app/controllers
27
+ # and is autoloaded (and eager-loaded in production) by the host.
28
+ #
26
29
  # One real consequence: `StandardCircuit::HealthController` now picks up the
27
30
  # engine's (empty) url_helpers instead of the application's, so app path
28
31
  # helpers inside it — or inside a host subclass of it — need a `main_app.`
29
32
  # prefix. The controller only renders JSON, so nothing in-gem is affected.
30
33
  isolate_namespace StandardCircuit
31
34
 
35
+ initializer "standard_circuit.deprecator" do |app|
36
+ app.deprecators[:standard_circuit] = StandardCircuit.deprecator if app.respond_to?(:deprecators)
37
+ end
38
+
32
39
  initializer "standard_circuit.subscribers", after: :load_config_initializers do
33
40
  StandardCircuit.subscribers.setup!
34
41
  end
@@ -16,6 +16,34 @@ module StandardCircuit
16
16
  # `AdapterErrors::*` because the right `skipped_errors` set is usually
17
17
  # app-specific and a shared taxonomy would over-skip.
18
18
  module ErrorTaxonomies
19
+ # Default `skipped_errors` for a circuit registered without an explicit
20
+ # `skipped_errors:`. Most adapters default to `[]`, but two can't, because
21
+ # their server-side error class is also the superclass of caller errors:
22
+ #
23
+ # * AWS — 5xx errors are dynamically generated `Aws::Errors::ServiceError`
24
+ # subclasses (e.g. `Aws::S3::Errors::ServiceUnavailable`), so `Aws.tracked`
25
+ # has to track `ServiceError` itself — which is also the superclass of
26
+ # `AccessDenied` and `NoSuchKey`. Without a skip list, a burst of
27
+ # missing-key lookups or permission errors would trip the S3 breaker.
28
+ # * Postmark — `ApiInputError` and `InvalidApiKeyError` subclass
29
+ # `Postmark::HttpServerError`, which `Postmark.tracked` tracks.
30
+ #
31
+ # Returns the caller errors that some entry of +tracked+ covers (is the
32
+ # same class or an ancestor of), or `[]` when none are covered. Returns a
33
+ # fresh array each call.
34
+ def self.default_skipped_for(tracked)
35
+ trackable = Array(tracked).grep(Module)
36
+ covered_caller_errors.select do |caller_error|
37
+ trackable.any? { |klass| caller_error <= klass }
38
+ end
39
+ end
40
+
41
+ # @api private
42
+ def self.covered_caller_errors
43
+ AdapterErrors::Aws.caller_errors + AdapterErrors::Postmark.caller_errors
44
+ end
45
+ private_class_method :covered_caller_errors
46
+
19
47
  module Stripe
20
48
  def self.tracked
21
49
  NetworkErrors.defaults + AdapterErrors::Stripe.server_errors
@@ -39,5 +67,13 @@ module StandardCircuit
39
67
  NetworkErrors.defaults + AdapterErrors::Faraday.server_errors
40
68
  end
41
69
  end
70
+
71
+ # Pair with `skipped_errors: AdapterErrors::Postmark.caller_errors` — or
72
+ # omit `skipped_errors:` and let `default_skipped_for` supply it.
73
+ module Postmark
74
+ def self.tracked
75
+ NetworkErrors.defaults + AdapterErrors::Postmark.server_errors
76
+ end
77
+ end
42
78
  end
43
79
  end
@@ -3,10 +3,10 @@ module StandardCircuit
3
3
  # and the Config's registered circuits / prefixes and return structured
4
4
  # snapshots plus an overall health status.
5
5
  #
6
- # Intended for mounting in a Rails HealthController. Prefer +health_report+
7
- # over calling +health_snapshot+ and +health_overall+ separately the
8
- # combined call takes a single atomic snapshot, so the rendered status and
9
- # circuits always describe the same moment:
6
+ # Intended for mounting in a Rails HealthController. Use +health_report+
7
+ # it takes a single atomic snapshot, so the rendered status and circuits
8
+ # always describe the same moment (the separate module-level
9
+ # +health_snapshot+ / +health_overall+ readers are deprecated):
10
10
  #
11
11
  # report = StandardCircuit.health_report
12
12
  # render json: report, status: (report[:status] == :critical ? 503 : 200)
@@ -1,28 +1,23 @@
1
+ # Deprecated load path, kept so `require "standard_circuit/health_controller"`
2
+ # at the top of a host's config/routes.rb (the pre-0.4 opt-in) keeps working.
3
+ #
4
+ # The controller now lives at app/controllers/standard_circuit/health_controller.rb
5
+ # and the engine autoloads it — delete the require and keep the route.
1
6
  require "action_controller"
7
+ require "standard_circuit"
2
8
 
3
- module StandardCircuit
4
- # Opt-in health-check controller. Renders +StandardCircuit.health_report+ as
5
- # JSON and returns 503 when the rolled-up status is +:critical+ so upstream
6
- # orchestrators pull the instance out of rotation. :degraded and :ok both
7
- # return 200 — the app can still serve most traffic.
8
- #
9
- # This controller is not auto-loaded. Consumers opt in:
10
- #
11
- # # config/routes.rb
12
- # require "standard_circuit/health_controller"
13
- #
14
- # Rails.application.routes.draw do
15
- # get "/health", to: "standard_circuit/health#show"
16
- # end
17
- #
18
- # Inherits from +ActionController::API+ to sidestep any ApplicationController
19
- # filters (authentication, bootstrap redirects, etc.) — health probes must be
20
- # callable anonymously from load balancers and uptime monitors.
21
- class HealthController < ::ActionController::API
22
- def show
23
- report = StandardCircuit.health_report
24
- http_status = report[:status] == :critical ? :service_unavailable : :ok
25
- render json: report, status: http_status
26
- end
27
- end
9
+ StandardCircuit.deprecator.warn(
10
+ 'require "standard_circuit/health_controller" is no longer needed: the engine ' \
11
+ "autoloads StandardCircuit::HealthController. Remove the require and keep the " \
12
+ '`get "/health", to: "standard_circuit/health#show"` route.'
13
+ )
14
+
15
+ # Booted Rails app (e.g. required from config/routes.rb): the engine's
16
+ # autoloader already has an autoload set for the constant, so there is nothing
17
+ # to do — loading the file here would race Zeitwerk for the same path.
18
+ # Otherwise (required from an initializer before autoloaders are set up, or
19
+ # outside Rails entirely) load it directly; Zeitwerk later sees the constant
20
+ # is defined and leaves it alone.
21
+ unless StandardCircuit.autoload?(:HealthController) || StandardCircuit.const_defined?(:HealthController, false)
22
+ require_relative "../../app/controllers/standard_circuit/health_controller"
28
23
  end
@@ -0,0 +1,121 @@
1
+ require_relative "circuit_open_error"
2
+
3
+ module StandardCircuit
4
+ module Mailer
5
+ # Opt-in `retry_on CircuitOpenError` for ActionMailer's delivery job, so a
6
+ # `deliver_later` that runs while the mail circuit is open is retried after
7
+ # the breaker has had a chance to half-open, instead of failing once and
8
+ # losing the email. Enabled with:
9
+ #
10
+ # StandardCircuit.configure do |c|
11
+ # c.mailer_retry = true # 90s wait, 5 attempts, 15% jitter
12
+ # c.mailer_retry = { wait: 120, attempts: 8 } # partial overrides
13
+ # end
14
+ #
15
+ # Keep `wait` longer than the mail circuit's `cool_off_time`, or each retry
16
+ # lands on a still-open circuit.
17
+ #
18
+ # `ActionMailer::MailDeliveryJob` inherits from `ActiveJob::Base`, not the
19
+ # host's ApplicationJob, which is why every app was patching it from an
20
+ # initializer. Installation is idempotent — `configure` runs on every
21
+ # `to_prepare` (so on every code reload in development), and `retry_on`
22
+ # appends a rescue handler each time it's called — and it also stands down
23
+ # when a handler for CircuitOpenError is already present, so a host that
24
+ # still carries its own initializer doesn't end up with two.
25
+ #
26
+ # Subclasses of MailDeliveryJob (`self.delivery_job = MyJob`) inherit the
27
+ # handler; a `retry_on CircuitOpenError` declared on the subclass wins.
28
+ #
29
+ # On exhaustion: one error log line, a Sentry `:error` event (when
30
+ # `sentry_enabled` and Sentry is initialized), and a
31
+ # `standard_circuit.mailer.retries_exhausted` event. PII: recipient
32
+ # DOMAINS only — never addresses or subjects.
33
+ module Retry
34
+ EXHAUSTED_EVENT = "standard_circuit.mailer.retries_exhausted".freeze
35
+ EXHAUSTED_MESSAGE = "Email delivery failed: mail circuit breaker retries exhausted".freeze
36
+ FINGERPRINT = "standard_circuit-mailer-retries-exhausted".freeze
37
+
38
+ class << self
39
+ # Defers installation until ActiveJob::Base has loaded (immediately if
40
+ # it already has), so enabling this from an early initializer doesn't
41
+ # force ActiveJob to load before its own configuration is applied. The
42
+ # hook reads the live config when it runs; it is registered once.
43
+ def install_on_load
44
+ return if @on_load_registered
45
+
46
+ @on_load_registered = true
47
+ ::ActiveSupport.on_load(:active_job) do
48
+ options = StandardCircuit.config.mailer_retry
49
+ StandardCircuit::Mailer::Retry.install(::ActionMailer::MailDeliveryJob, options) if options
50
+ end
51
+ end
52
+
53
+ # Adds the retry_on to +job_class+ unless a CircuitOpenError handler is
54
+ # already there. Returns true when it installed one.
55
+ def install(job_class, options)
56
+ return false if installed?(job_class)
57
+
58
+ job_class.retry_on(
59
+ CircuitOpenError,
60
+ wait: options.fetch(:wait),
61
+ attempts: options.fetch(:attempts),
62
+ jitter: options.fetch(:jitter)
63
+ ) do |job, error|
64
+ StandardCircuit::Mailer::Retry.report_exhausted(job, error)
65
+ end
66
+ true
67
+ end
68
+
69
+ def installed?(job_class)
70
+ job_class.rescue_handlers.any? { |(error_name, _handler)| error_name == CircuitOpenError.name }
71
+ end
72
+
73
+ def report_exhausted(job, error)
74
+ mailer_class, mail_action = job.arguments
75
+ details = {
76
+ mailer_class: mailer_class,
77
+ mail_action: mail_action,
78
+ recipient_domains: recipient_domains(error.recipients),
79
+ job_id: job.job_id,
80
+ executions: job.executions
81
+ }
82
+
83
+ logger&.error("[standard_circuit] #{EXHAUSTED_MESSAGE} #{details.to_json}")
84
+ capture_sentry(details)
85
+ EventEmitter.emit(EXHAUSTED_EVENT, details)
86
+ details
87
+ end
88
+
89
+ def recipient_domains(recipients)
90
+ Array(recipients)
91
+ .filter_map { |address| address.to_s.split("@", 2)[1]&.strip&.downcase }
92
+ .reject(&:empty?)
93
+ .uniq
94
+ end
95
+
96
+ # @api private — test isolation.
97
+ def reset_on_load_registration!
98
+ @on_load_registered = false
99
+ end
100
+
101
+ private
102
+
103
+ def logger
104
+ StandardCircuit.config.logger || (::Rails.logger if defined?(::Rails) && ::Rails.respond_to?(:logger))
105
+ end
106
+
107
+ def capture_sentry(details)
108
+ return unless StandardCircuit.config.sentry_enabled
109
+ return unless defined?(::Sentry) && ::Sentry.respond_to?(:initialized?) && ::Sentry.initialized?
110
+
111
+ ::Sentry.capture_message(
112
+ EXHAUSTED_MESSAGE,
113
+ level: :error,
114
+ fingerprint: [ FINGERPRINT, details[:mailer_class], details[:mail_action] ].map(&:to_s),
115
+ extra: details
116
+ )
117
+ end
118
+ end
119
+ end
120
+ end
121
+ end
@@ -0,0 +1,90 @@
1
+ module StandardCircuit
2
+ # Named circuit registrations for integrations whose settings were being
3
+ # copy-pasted, line for line, across host apps. Use via
4
+ # `Config#register_preset`:
5
+ #
6
+ # StandardCircuit.configure do |c|
7
+ # c.register_preset(:postmark) # named circuit :postmark
8
+ # c.register_preset(:s3) # prefix "s3" (s3_<bucket>)
9
+ # c.register_preset(:postmark, criticality: :critical) # override any option
10
+ # end
11
+ #
12
+ # A preset only fills in defaults — every `register` option passed alongside
13
+ # it wins. Each preset also requires its SDK before computing the error
14
+ # lists, so the taxonomy can't silently degrade to network errors only when
15
+ # the SDK is declared `require: false` and hasn't been loaded yet at
16
+ # configure time.
17
+ module Presets
18
+ Preset = Struct.new(:scope, :sdk_require, :sdk_constant, :options, keyword_init: true)
19
+
20
+ REGISTRY = {
21
+ # Postmark API delivery behind the `:standard_circuit` mailer delivery
22
+ # method. 60s cool-off pairs with `mailer_retry`'s 90s default wait, so a
23
+ # retried delivery lands after the breaker has had a chance to half-open.
24
+ # :standard — an email outage is user-visible but non-blocking.
25
+ postmark: Preset.new(
26
+ scope: :name,
27
+ sdk_require: "postmark",
28
+ sdk_constant: "Postmark::HttpServerError",
29
+ options: -> {
30
+ {
31
+ threshold: 3,
32
+ cool_off_time: 60,
33
+ criticality: :standard,
34
+ tracked_errors: ErrorTaxonomies::Postmark.tracked,
35
+ skipped_errors: AdapterErrors::Postmark.caller_errors
36
+ }
37
+ }
38
+ ),
39
+ # Per-bucket S3 circuits for `service: StandardCircuitS3` in storage.yml,
40
+ # which keys its circuit as `s3_<bucket>` — hence a prefix registration.
41
+ # `skipped_errors` is left to `ErrorTaxonomies.default_skipped_for`, which
42
+ # skips AccessDenied / NoSuchKey.
43
+ s3: Preset.new(
44
+ scope: :prefix,
45
+ sdk_require: "aws-sdk-s3",
46
+ sdk_constant: "Aws::S3::Errors::NoSuchKey",
47
+ options: -> {
48
+ {
49
+ threshold: 3,
50
+ cool_off_time: 30,
51
+ criticality: :standard,
52
+ tracked_errors: ErrorTaxonomies::Aws.tracked
53
+ }
54
+ }
55
+ )
56
+ }.freeze
57
+
58
+ class << self
59
+ def names
60
+ REGISTRY.keys
61
+ end
62
+
63
+ # Returns `[scope, options]` for +preset+ with +overrides+ merged on top.
64
+ # Raises ArgumentError for an unknown preset or a missing SDK.
65
+ def resolve(preset, **overrides)
66
+ entry = REGISTRY.fetch(preset.to_sym) do
67
+ raise ArgumentError, "unknown preset #{preset.inspect}; available: #{names.inspect}"
68
+ end
69
+ load_sdk!(preset, entry)
70
+ [ entry.scope, entry.options.call.merge(overrides) ]
71
+ end
72
+
73
+ private
74
+
75
+ def load_sdk!(preset, entry)
76
+ return if Object.const_defined?(entry.sdk_constant)
77
+
78
+ begin
79
+ require entry.sdk_require
80
+ rescue LoadError
81
+ nil
82
+ end
83
+ return if Object.const_defined?(entry.sdk_constant)
84
+
85
+ raise ArgumentError,
86
+ "preset #{preset.inspect} needs the #{entry.sdk_require} gem (#{entry.sdk_constant} is not defined)"
87
+ end
88
+ end
89
+ end
90
+ end
@@ -1,3 +1,3 @@
1
1
  module StandardCircuit
2
- VERSION = "0.3.0"
2
+ VERSION = "0.4.0"
3
3
  end
@@ -3,12 +3,15 @@ require "concurrent"
3
3
  require "sentry-ruby"
4
4
 
5
5
  require "standard_circuit/version"
6
+ require "standard_circuit/deprecator"
6
7
  require "standard_circuit/network_errors"
7
8
  require "standard_circuit/adapter_errors/stripe"
8
9
  require "standard_circuit/adapter_errors/aws"
9
10
  require "standard_circuit/adapter_errors/faraday"
10
11
  require "standard_circuit/adapter_errors/smtp"
12
+ require "standard_circuit/adapter_errors/postmark"
11
13
  require "standard_circuit/error_taxonomies"
14
+ require "standard_circuit/presets"
12
15
  require "standard_circuit/event_emitter"
13
16
  require "standard_circuit/notifier_bridge"
14
17
  require "standard_circuit/notifiers/logger"
@@ -20,6 +23,7 @@ require "standard_circuit/health"
20
23
  require "standard_circuit/runner"
21
24
  require "standard_circuit/mailer/circuit_open_error"
22
25
  require "standard_circuit/mailer/delivery_method"
26
+ require "standard_circuit/mailer/retry"
23
27
  require "standard_circuit/controller_support"
24
28
  require "standard_circuit/engine" if defined?(::Rails::Engine)
25
29
 
@@ -32,6 +36,7 @@ module StandardCircuit
32
36
  yield config
33
37
  runner.apply_config!(config)
34
38
  subscribers.setup!
39
+ Mailer::Retry.install_on_load if config.mailer_retry
35
40
  config
36
41
  end
37
42
 
@@ -67,11 +72,21 @@ module StandardCircuit
67
72
  runner.reset!
68
73
  end
69
74
 
75
+ # @deprecated Use +health_report[:circuits]+. Removed in 0.5.
70
76
  def health_snapshot
77
+ deprecator.warn(
78
+ "StandardCircuit.health_snapshot is deprecated and will be removed in 0.5; " \
79
+ "use StandardCircuit.health_report[:circuits] (one atomic read of status + circuits)."
80
+ )
71
81
  runner.health_snapshot
72
82
  end
73
83
 
84
+ # @deprecated Use +health_report[:status]+. Removed in 0.5.
74
85
  def health_overall(snapshot = nil)
86
+ deprecator.warn(
87
+ "StandardCircuit.health_overall is deprecated and will be removed in 0.5; " \
88
+ "use StandardCircuit.health_report[:status] (one atomic read of status + circuits)."
89
+ )
75
90
  runner.health_overall(snapshot)
76
91
  end
77
92
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: standard_circuit
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.0
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Jaryl Sim
@@ -65,6 +65,34 @@ dependencies:
65
65
  - - ">="
66
66
  - !ruby/object:Gem::Version
67
67
  version: '8.0'
68
+ - !ruby/object:Gem::Dependency
69
+ name: actionmailer
70
+ requirement: !ruby/object:Gem::Requirement
71
+ requirements:
72
+ - - ">="
73
+ - !ruby/object:Gem::Version
74
+ version: '8.0'
75
+ type: :runtime
76
+ prerelease: false
77
+ version_requirements: !ruby/object:Gem::Requirement
78
+ requirements:
79
+ - - ">="
80
+ - !ruby/object:Gem::Version
81
+ version: '8.0'
82
+ - !ruby/object:Gem::Dependency
83
+ name: actionpack
84
+ requirement: !ruby/object:Gem::Requirement
85
+ requirements:
86
+ - - ">="
87
+ - !ruby/object:Gem::Version
88
+ version: '8.0'
89
+ type: :runtime
90
+ prerelease: false
91
+ version_requirements: !ruby/object:Gem::Requirement
92
+ requirements:
93
+ - - ">="
94
+ - !ruby/object:Gem::Version
95
+ version: '8.0'
68
96
  - !ruby/object:Gem::Dependency
69
97
  name: brakeman
70
98
  requirement: !ruby/object:Gem::Requirement
@@ -120,18 +148,20 @@ files:
120
148
  - LICENSE
121
149
  - README.md
122
150
  - Rakefile
151
+ - app/controllers/standard_circuit/health_controller.rb
123
152
  - lib/active_storage/service/standard_circuit_s3_service.rb
124
153
  - lib/generators/standard_circuit/install/install_generator.rb
125
- - lib/generators/standard_circuit/install/templates/health_initializer.rb.tt
126
154
  - lib/generators/standard_circuit/install/templates/initializer.rb.tt
127
155
  - lib/standard_circuit.rb
128
156
  - lib/standard_circuit/active_storage/s3_service.rb
129
157
  - lib/standard_circuit/adapter_errors/aws.rb
130
158
  - lib/standard_circuit/adapter_errors/faraday.rb
159
+ - lib/standard_circuit/adapter_errors/postmark.rb
131
160
  - lib/standard_circuit/adapter_errors/smtp.rb
132
161
  - lib/standard_circuit/adapter_errors/stripe.rb
133
162
  - lib/standard_circuit/config.rb
134
163
  - lib/standard_circuit/controller_support.rb
164
+ - lib/standard_circuit/deprecator.rb
135
165
  - lib/standard_circuit/engine.rb
136
166
  - lib/standard_circuit/error_taxonomies.rb
137
167
  - lib/standard_circuit/event_emitter.rb
@@ -139,11 +169,13 @@ files:
139
169
  - lib/standard_circuit/health_controller.rb
140
170
  - lib/standard_circuit/mailer/circuit_open_error.rb
141
171
  - lib/standard_circuit/mailer/delivery_method.rb
172
+ - lib/standard_circuit/mailer/retry.rb
142
173
  - lib/standard_circuit/network_errors.rb
143
174
  - lib/standard_circuit/notifier_bridge.rb
144
175
  - lib/standard_circuit/notifiers/logger.rb
145
176
  - lib/standard_circuit/notifiers/metrics.rb
146
177
  - lib/standard_circuit/notifiers/sentry.rb
178
+ - lib/standard_circuit/presets.rb
147
179
  - lib/standard_circuit/rspec.rb
148
180
  - lib/standard_circuit/runner.rb
149
181
  - lib/standard_circuit/subscribers.rb
@@ -1,25 +0,0 @@
1
- # StandardCircuit health-check controller
2
- # Generated by: rails g standard_circuit:install --with-health-endpoint
3
- #
4
- # The controller is opt-in: this require pulls it onto the autoload path so
5
- # the route below resolves. Add the matching route to config/routes.rb:
6
- #
7
- # get "/health", to: "standard_circuit/health#show"
8
- #
9
- # ORDERING IS LOAD-BEARING: if your app also mounts StandardHealth::Engine at
10
- # "/health", draw this aggregate route BEFORE the mount —
11
- #
12
- # get "/health", to: "standard_circuit/health#show" # aggregate — FIRST
13
- # mount StandardHealth::Engine => "/health", as: :standard_health
14
- #
15
- # StandardHealth::Engine registers sub-paths only ("/alive", "/ready",
16
- # "/diagnostics/env") — it never serves the aggregate tier itself. So an app
17
- # that mounts the engine and assumes "/health" is covered has no aggregate tier
18
- # at all, and nothing says so: no boot error, and no failing route spec unless
19
- # one already exists. Draw this route explicitly, first.
20
- #
21
- # The endpoint renders StandardCircuit.health_report as JSON and returns
22
- # 503 when the rolled-up status is :critical (so orchestrators pull the
23
- # instance out of rotation) and 200 otherwise.
24
-
25
- require "standard_circuit/health_controller"