redeye-breaker 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -104,7 +104,7 @@ Every process pointed at the same Redis instance (and using the same operation n
104
104
 
105
105
  ### Bring your own store
106
106
 
107
- `RedisStore` implements a `Store` interface with two required methods and four *optional* ones:
107
+ `RedisStore` implements a `Store` interface with two required methods and seven *optional* ones:
108
108
 
109
109
  ```ts
110
110
  export interface Store {
@@ -115,19 +115,62 @@ export interface Store {
115
115
  // Optional — implement these for the reliability guarantees below.
116
116
  // Without them, redeye falls back to best-effort semantics and logs a
117
117
  // one-time warning telling you exactly what's degraded.
118
- recordFailureAtomic?(key: string, opts: { ttlSeconds: number; failureThreshold: number; now: number }): Promise<CircuitBreakerState>;
119
- recordOutcomeAtomic?(key: string, opts: { success: boolean; decay: number; minimumCalls: number; errorRateThreshold: number; ttlSeconds: number; now: number }): Promise<CircuitBreakerState & { openedNow: boolean }>;
118
+ recordFailureAtomic?(key: string, opts: { ttlSeconds: number; failureThreshold: number; now: number; operation: string }): Promise<CircuitBreakerState>;
119
+ recordOutcomeAtomic?(key: string, opts: { success: boolean; decay: number; minimumCalls: number; errorRateThreshold: number; ttlSeconds: number; now: number; operation: string }): Promise<CircuitBreakerState & { openedNow: boolean }>;
120
120
  claimTrial?(key: string, ttlSeconds: number): Promise<string | null>;
121
121
  releaseTrial?(key: string, token: string): Promise<void>;
122
+
123
+ // Optional — only needed for the closed-state local cache (`localCache`, see below).
124
+ closeAtomic?(key: string, eventsKey: string, opts: { ttlSeconds: number; operation: string }): Promise<CircuitBreakerState>;
125
+ reopenTrialFailureAtomic?(key: string, eventsKey: string, opts: { ttlSeconds: number; failureThreshold: number; now: number; operation: string }): Promise<CircuitBreakerState>;
126
+ subscribeTransitions?(eventsKey: string, handler: (event: TransitionEvent | null) => void): Promise<() => void>;
122
127
  }
123
128
  ```
124
129
 
125
- - `recordFailureAtomic` (`strategy: 'consecutive'`) should increment the failure count and decide `isOpen` in one atomic round trip (a Lua script in Redis, a conditional update in DynamoDB, etc.).
126
- - `recordOutcomeAtomic` (`strategy: 'errorRate'`) should fold one call's outcome into the EWMA rate and decide `isOpen` in one atomic round trip — called on every closed-phase call, not just failures.
130
+ - `recordFailureAtomic` (`strategy: 'consecutive'`) should increment the failure count and decide `isOpen` in one atomic round trip (a Lua script in Redis, a conditional update in DynamoDB, etc.). Should also bump a monotonic `version` counter and publish a transition event to `eventsKey` (see below) when — and only when — this call flips `isOpen`; a plain sub-threshold increment isn't a transition and shouldn't publish one.
131
+ - `recordOutcomeAtomic` (`strategy: 'errorRate'`) should fold one call's outcome into the EWMA rate and decide `isOpen` in one atomic round trip — called on every closed-phase call, not just failures. Same `version`/publish rule as above, keyed off an `isOpen` flip.
127
132
  - `claimTrial` should be a conditional "create if absent" write (`SET key token NX EX ttl` in Redis, with `token` a fresh unique value per call, e.g. a UUID) — it's what makes half-open recovery exclusive to one caller instead of a free-for-all. Return the token you wrote if this call won the claim, or `null` if the key was already taken.
128
133
  - `releaseTrial` should be a compare-and-delete (`if GET key == token then DEL key` in Redis, a conditional delete elsewhere), not an unconditional delete. This is what stops a trial that outran its TTL (see [item 11](#whats-still-a-fundamental-tradeoff-not-a-bug)) from deleting a *different* instance's newer claim on the same key when it finally gets around to releasing its own now-stale one. If omitted (or if a call has no confirmed ownership token to release at all — e.g. `claimTrial` itself errored), redeye logs a one-time warning and leaves the claim for its TTL to expire naturally, rather than risk an unconditional delete that could destroy a claim it never confirmed was its own.
134
+ - `closeAtomic` should close `key`, bumping `version` and publishing a transition, **but only if there was anything to clear** (it was open, or had accumulated sub-threshold failures) — a true no-op, no write at all, when the state was already clean. This is what lets the healthy-path write-skip (see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug)) happen atomically server-side instead of relying on the caller's own possibly-stale view.
135
+ - `reopenTrialFailureAtomic` should reopen `key` after a failed half-open trial — `isOpen: true`, `openCount + 1`, `failures` raised to at least `failureThreshold`, `lastFailure: now`, `version + 1` — and always publish a transition (a failed trial is always a real transition, unlike `closeAtomic`).
136
+ - `subscribeTransitions` should subscribe to a stream of transition events at `eventsKey` — **one shared stream per store/prefix, not one per operation** — and invoke `handler` with each decoded event, or with `null` whenever the underlying subscription drops or errors (callers must then distrust every locally cached entry until it's re-verified by a real read; implementations should retry their own subscription with backoff, not expect the caller to re-subscribe). Returns an unsubscribe function.
137
+
138
+ Implement the two required methods against Memcached, DynamoDB, your own cache wrapper, etc., and you have a working (best-effort) distributed breaker. Add `recordFailureAtomic`/`recordOutcomeAtomic`/`claimTrial`/`releaseTrial` when that store supports a real atomic increment and a real conditional write, and you get the full reliability guarantees. Add `closeAtomic`/`reopenTrialFailureAtomic`/`subscribeTransitions` on top of that if you also want the [closed-state local cache](#local-caching-hybrid-mode) — every capability is independently optional, so skip any subset that doesn't fit your store.
139
+
140
+ ## Local caching (hybrid mode)
141
+
142
+ Everything up to this point reads the store on every gate check (a genuine read every closed-path call; a cached, 0-read rejection once open — see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug)). `localCache` goes one step further, opt-in, only for `RedisStore` (or any store implementing `subscribeTransitions`): it caches the **closed** state locally too, invalidated push-style instead of polling, so a healthy call can skip the store read entirely, not just the write.
143
+
144
+ ```ts
145
+ const breaker = new CircuitBreaker({
146
+ failureThreshold: 5,
147
+ resetTimeout: 60_000,
148
+ store,
149
+ localCache: { staleToleranceMs: 100 }, // opt-in; default staleToleranceMs is 100 once the object is present
150
+ });
151
+ ```
129
152
 
130
- Implement the two required methods against Memcached, DynamoDB, your own cache wrapper, etc., and you have a working (best-effort) distributed breaker. Add the optional methods relevant to the strategy you use when that store supports a real atomic increment and a real conditional write, and you get the full reliability guarantees.
153
+ **What's cached, and how it's invalidated.** Per `operation`, redeye keeps `{ state, version, fetchedAt, trusted }` in memory. A gate check allows immediately, with zero store reads, when the entry is `trusted`, closed (`!state.isOpen`), and fresher than `staleToleranceMs`. Three independent mechanisms keep this from going stale silently:
154
+
155
+ - **Push invalidation.** `subscribeTransitions` delivers a decoded event the instant any instance's write flips `isOpen` or clears real accumulated state (never on a plain sub-threshold counter update — those aren't transitions and don't invalidate the cache, by design, since they can't affect the *closed* fast path this cache serves). A higher `version` than what's cached overwrites the entry immediately.
156
+ - **Fallback poll.** The existing local-mode monitor interval doubles as a missed-message safety net in distributed mode: any cached entry older than 5s gets one authoritative `get` and is reconciled by `version`. In steady state (no missed messages) this is the *only* background traffic the cache generates — one `GET` per stale operation per sweep, not per call.
157
+ - **Your own writes are free.** The state `recordFailureAtomic`/`recordOutcomeAtomic`/`closeAtomic`/`reopenTrialFailureAtomic` hand back already updates this instance's own cache — the instance that causes a transition never has a stale window on it, regardless of stream or poll timing.
158
+
159
+ **The fail-open window, as a formula, not an adjective.** This is the one place in redeye that's biased toward availability over freshness on the *closed* side (everywhere else — the open-state cache, `failOpenOnStoreError: false`, etc. — is fail-closed or symmetric by default). If the breaker trips elsewhere in the fleet, a call landing on this instance's still-fresh cached-closed entry can pass through before that instance learns about it. The bound is exactly:
160
+
161
+ ```
162
+ worst-case fail-open window = staleToleranceMs + transition-delivery latency
163
+ ```
164
+
165
+ — and it's a hard ceiling, not a typical case: the cache stops trusting itself once `staleToleranceMs` elapses regardless of whether the transition event ever arrives (that's what makes the fallback poll a correctness backstop and not just a nice-to-have), and delivery latency in the common case is one Redis stream round trip, not a poll cycle. Set `staleToleranceMs` to whatever window you're comfortable a downstream outage could go un-noticed on this one path — `100`ms (the default) is a reasonable starting point for most services; push it lower if even a hundred milliseconds of continued traffic against a freshly-tripped dependency is unacceptable, at the cost of the cache being useful for a shorter fraction of the healthy path.
166
+
167
+ **Compare this to the open-state cache deliberately.** `openCacheRefreshMs` never risks an extra call through — staleness there can only make a rejection *late*, not wrong, because it always falls through to a real read right at the real eligibility instant (see limitation item 1). `localCache`'s closed-side cache is the opposite shape: staleness there can make an *allow* wrong for a bounded window, in exchange for the healthy path's read too. That's the real tradeoff you're opting into, not a bug to route around — the same one Envoy's outlier detection makes by default (cache state locally with a bounded TTL, accept staleness, skip the round trip), just off by default here and explicit about the bound when you turn it on.
168
+
169
+ **The version/absence rule, and mixed-fleet rolling deploys.** `CircuitBreakerState.version` is a monotonic counter that increments only on a real transition — its *absence* (not `0`) means "written by a library version that predates `localCache`, provenance unknown." A cache entry built from a version-less read is marked `trusted: false` and never used for the fast path; the very next call for that operation performs a real read instead, forever, until a version-bearing write appears. This is what makes a rolling deploy safe: instances running an old version keep working exactly as before (they don't publish transitions, and their writes lack `version`), and new instances simply never trust a versionless observation rather than guessing. Nothing about this requires deploying in any particular order.
170
+
171
+ **Strategy asymmetry, stated plainly.** `consecutive` is the strategy this cache is built for: combined with the healthy-path write skip, a run of successes against a warm cache is a genuine 0-store-op healthy path. `errorRate` cannot get the same benefit — `recordOutcomeAtomic` must fold *every* outcome (success or failure) into the EWMA to keep the rate meaningful, so it floors at 1 write per call no matter what's cached. `localCache` still saves the *read* for `errorRate` the same way it does for `consecutive`; it just can't save the write, structurally, not as an oversight.
172
+
173
+ **Requires `subscribeTransitions`.** Without it (a custom `Store` that doesn't implement the capability), `localCache` is a no-op: one-time warning, and behavior is identical to not setting it at all. `RedisStore` implements it via a dedicated (`duplicate()`d) blocking connection running `XREAD BLOCK` against the shared `circuit_breaker:events` stream, reconnecting with backoff on its own if that connection drops — see the `Store` interface docs above for the reconnect contract.
131
174
 
132
175
  ## API
133
176
 
@@ -167,6 +210,7 @@ redeye can't stop you from doing this, but it can flag it: `execute`/`recordFail
167
210
  | `logger` | no-op | `{ warn(msg), log(msg) }` — plug in your own logger |
168
211
  | `maxOperations` | none | Logs a one-time warning once the breaker has seen more than this many distinct `operation` names — see the cardinality note above. Unset: no limit, no warning. |
169
212
  | `openCacheRefreshMs` | `2000` | Distributed mode only: while `operation` is known-open, reject locally without a store read, re-verifying against the store at most once per this many ms (to catch an early close by another instance's trial, or a manual `reset()`). Never delays trial eligibility — see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug). `0` disables the cache (a store read on every call, the pre-cache behavior). |
213
+ | `localCache` | none (disabled) | Distributed mode, opt-in, requires a store implementing `subscribeTransitions`: `{ staleToleranceMs?: number }` (default `100` once the object is present) — caches CLOSED state locally, invalidated push-style, so a healthy call can skip the store read too, not just the write. See [Local caching (hybrid mode)](#local-caching-hybrid-mode) for the fail-open window this trades for that. |
170
214
 
171
215
  ### Store unavailability: fail-open vs fail-closed
172
216
 
@@ -204,7 +248,7 @@ If your store *doesn't* implement the atomic method your chosen strategy needs (
204
248
 
205
249
  ### Split-brain: what's prevented, and what isn't
206
250
 
207
- There's no split-brain among your app instances in the classic sense, because instances never hold independent authoritative state to disagree over — every instance reads the store fresh before every single decision (`gateDistributed`) instead of trusting a locally cached opinion. The store is the only "brain"; instances are just readers/writers of it. Two mechanisms enforce agreement on the common path:
251
+ There's no split-brain among your app instances in the classic sense, because instances never hold independent authoritative state *to write from* — every state-changing decision goes through the store, and the two local caches (`openCacheRefreshMs`, `localCache`) exist only to skip *reading* it before an already-known-safe decision, never to substitute for it. The store is the only "brain"; instances are just readers/writers of it, some of the time from a recent, bounded-staleness memory of what they last read instead of asking again. Two mechanisms enforce agreement on the common path:
208
252
 
209
253
  - **Atomic writes.** `recordFailureAtomic`/`recordOutcomeAtomic` run as one Lua script — GET, decode, increment, decide, SET — inside Redis's single-threaded execution, so two instances failing at the same instant can't both read the same count and both write the same increment.
210
254
  - **Exclusive trial claim.** `claimTrial` is a `SET ... NX`, which Redis resolves atomically, so exactly one instance ever runs the half-open recovery probe — never two instances simultaneously deciding they're the one testing recovery.
@@ -221,7 +265,8 @@ So the design doesn't eliminate split-brain so much as route around it: it pushe
221
265
 
222
266
  These are true of any distributed circuit breaker, rate limiter, or lock built on a shared store — not gaps specific to redeye:
223
267
 
224
- 1. **Distributed mode costs Redis round trips per guarded call, not zeroexcept while the breaker is actually open, which is exactly when that matters most.** On the closed, healthy path, every `execute()` call does a gate read (`store.get`) before deciding whether to proceed; what happens after differs by strategy — `errorRate` writes (`recordOutcomeAtomic`) on every closed-phase call, success or failure, since the rate needs both to be meaningful, while `consecutive` only writes when the gate's own read observed accumulated state to clear (an open breaker, or `failures > 0`), making a run of healthy successes 1 read and 0 writes per call, not 1-and-1 (a measured claim about the steady state, not a guarantee — a dependency flapping near the failure threshold still writes on most calls). **Once the breaker is open, rejections stop costing a read at all**: the gate caches the open state locally (`lastFailure`, `openCount`, and the same jittered `expiresAt` the eligibility check itself uses) and rejects straight from that cache, re-verifying against the store only once per `openCacheRefreshMs` (default 2000ms) — to catch an early close by another instance's trial, or a manual `reset()` — and always falling through to a real read right at `expiresAt`, so the cache can delay a rejection's freshness but can never delay a trial's eligibility. Net effect: the request-volume-scaling cost this item used to describe applies to the closed, healthy path the load *during an incident*, when the store or its host infrastructure may itself be degraded, drops instead of scaling with traffic. A resolving half-open trial still adds a round trip to claim it and another to release its claim — that path is unaffected, and unconditional, regardless of strategy. On a latency-sensitive path the closed-path round trip is still added tail latency — the opposite tradeoff from something like Envoy's outlier detection, which caches state locally with a short TTL and accepts bounded staleness instead of paying a store round trip per call. redeye chose per-call freshness over that on the closed path; it's a reasonable choice for many services, but measure it before putting this in front of your highest-QPS call. Local mode has none of this cost — see [Local mode vs. distributed mode](#local-mode-vs-distributed-mode-which-do-you-need) if you're not sure you need shared state at all.
268
+ 1. **Distributed mode's per-call store cost is opt-in-able down to a bound you choose, not fixedbut "zero" always means "zero, at a stated staleness price," never "zero, for free."** By default, every `execute()` call does a gate read (`store.get`) before deciding whether to proceed; what happens after differs by strategy — `errorRate` writes (`recordOutcomeAtomic`) on every closed-phase call, success or failure, since the rate needs both to be meaningful (this never goes away, at any cache setting — see below), while `consecutive` only writes when the gate's own read observed accumulated state to clear (an open breaker, or `failures > 0`), making a run of healthy successes 1 read and 0 writes per call by default, not 1-and-1. **Once the breaker is open, rejections stop costing a read at all**, unconditionally, no opt-in required: the gate caches the open state locally (`lastFailure`, `openCount`, and the same jittered `expiresAt` the eligibility check itself uses) and rejects straight from that cache, re-verifying against the store only once per `openCacheRefreshMs` (default 2000ms), and always falling through to a real read right at `expiresAt` so this cache can delay a rejection's freshness but can never delay a trial's eligibility, and never risks letting an extra call through. That's the fail-*closed*-biased half of the story: staleness there can only make things more conservative.
269
+ The closed, healthy path is the other half, and it's where a real tradeoff lives: **`localCache` (opt-in) is redeye's answer to exactly the caching-with-bounded-staleness tradeoff Envoy's outlier detection makes by default** — cache state locally, accept a bounded staleness window, skip the round trip. Off by default (redeye chooses per-call freshness there until you say otherwise), and explicit about the bound when you turn it on: the worst case is `staleToleranceMs + transition-delivery latency`, a formula, not an adjective — see [Local caching (hybrid mode)](#local-caching-hybrid-mode) for exactly what that means and what it doesn't cover (`errorRate`'s write floor, in particular, which `localCache` cannot remove — only `consecutive` reaches a genuine 0-store-op healthy path). A resolving half-open trial always adds a round trip to claim it and another to release its claim, at any cache setting, on either strategy — that path is never cacheable, and rightly so. Local mode has none of this cost at all — see [Local mode vs. distributed mode](#local-mode-vs-distributed-mode-which-do-you-need) if you're not sure you need shared state to begin with.
225
270
  2. **Correctness depends on trusting your store.** If Redis evicts a breaker key early under memory pressure (e.g. `maxmemory-policy allkeys-lru` with `circuit_breaker:*` competing for space with everything else), the breaker can reset earlier than intended. Mitigate by giving circuit-breaker keys their own keyspace/DB with a policy that doesn't evict them, or by monitoring `onStoreError` and Redis memory pressure directly. This is not fixable in-library — it's an operational configuration matter.
226
271
  3. **Distributed timing is sensitive to clock skew.** Backoff and jitter windows are computed by comparing each instance's local `Date.now()` against a `lastFailure` timestamp written by whichever instance recorded it. With reasonably synchronized clocks (standard NTP) this is a non-issue; on a fleet with significant clock drift, instances can disagree by that drift amount about exactly when a trial becomes eligible. This affects *timing* only — the half-open claim mechanism (`claimTrial`) still guarantees exactly one trial runs regardless of skew, so skew cannot cause a thundering herd, only a slightly early or late trial attempt.
227
272
  4. **`recordFailure`/`recordSuccess`/`canExecuteAsync` don't participate in trial claiming.** Only `execute()` claims and releases the half-open trial slot. If you build your own call flow around the manual API instead of `execute()`, you lose the single-trial guarantee and get the old "everyone retries once elapsed" behavior for that flow. Prefer `execute()`.
@@ -252,6 +297,8 @@ docker compose down
252
297
 
253
298
  It connects to `REDIS_URL` (default `redis://localhost:6379`).
254
299
 
300
+ `npm run bench` runs `bench/index.js` against a real Redis, comparing total store round trips and p50/p99 latency across three configurations: no caching, today's defaults (`openCacheRefreshMs`), and `localCache` on top. Same `REDIS_URL`; start Redis first the same way as the integration suite. See [BENCHMARK.md](BENCHMARK.md) for a full run's results and how to read them.
301
+
255
302
  ## Contributing
256
303
 
257
304
  See [CONTRIBUTING.md](CONTRIBUTING.md). Changes are tracked in [CHANGELOG.md](CHANGELOG.md).
@@ -1,11 +1,11 @@
1
1
  import { Store } from './store';
2
- import { CircuitBreakerState, CircuitMetrics } from './types';
2
+ import { CircuitBreakerState, CircuitMetrics, TransitionEvent } from './types';
3
3
  export type CircuitState = 'closed' | 'open' | 'half-open';
4
4
  export interface BreakerLogger {
5
5
  warn(message: string): void;
6
6
  log(message: string): void;
7
7
  }
8
- export type { CircuitBreakerState, CircuitMetrics };
8
+ export type { CircuitBreakerState, CircuitMetrics, TransitionEvent };
9
9
  /**
10
10
  * Thrown in distributed mode when the backing store is unreachable and
11
11
  * `failOpenOnStoreError` is `false`. Distinguishable from a downstream
@@ -122,6 +122,32 @@ export interface CircuitBreakerOptions {
122
122
  * Default: 2000.
123
123
  */
124
124
  openCacheRefreshMs?: number;
125
+ /**
126
+ * Distributed mode, opt-in. Serves gate decisions for CLOSED state from a
127
+ * local cache while it's fresher than `staleToleranceMs`, invalidated
128
+ * push-style via the store's transition stream (`Store.subscribeTransitions`)
129
+ * instead of polling — so a healthy call can skip the store read entirely,
130
+ * not just the write Release 1 already skips.
131
+ *
132
+ * This only ever short-circuits an *allow*: a cache entry observed open
133
+ * always defers to the open-state cache (`openCacheRefreshMs`) and a real
134
+ * read, which remains authoritative for all rejection handling. Requires
135
+ * a store implementing `subscribeTransitions`; without one, this option
136
+ * is a no-op (one-time warning) and behavior is unchanged from not
137
+ * setting it at all.
138
+ *
139
+ * The worst case is a bounded fail-open window: if the breaker trips
140
+ * elsewhere in the fleet, a calls still landing on a cached entry can be
141
+ * let through for up to `staleToleranceMs` plus however long the
142
+ * transition event takes to arrive — never longer, since the cache entry
143
+ * expires against `staleToleranceMs` on its own regardless of whether the
144
+ * event ever arrives. See the README's local caching section for the
145
+ * full fail-open/fail-closed asymmetry this creates versus the
146
+ * (fail-closed-biased) open-state cache.
147
+ */
148
+ localCache?: {
149
+ staleToleranceMs?: number;
150
+ };
125
151
  }
126
152
  /**
127
153
  * A circuit breaker with an optional distributed mode.
@@ -159,6 +185,22 @@ export declare class CircuitBreaker {
159
185
  * an early close by another instance or a manual `reset()`.
160
186
  */
161
187
  private readonly cachedOpen;
188
+ /** Fixed, unprefixed key for the shared transition-event stream — one per store/prefix, not one per operation. `RedisStore` applies its own `keyPrefix` the same way it does for every other key this class hands it. */
189
+ private readonly eventsKey;
190
+ /**
191
+ * Distributed mode only, `localCache` opt-in: a local, push-invalidated
192
+ * record of the last known state for `operation`, used to serve gate
193
+ * decisions for CLOSED state with zero store reads while fresh and
194
+ * trusted. `trusted: false` means the last write we saw for this entry
195
+ * had no `version` (unknown provenance — an older library version wrote
196
+ * it) or the transition subscription itself dropped; either way, the
197
+ * gate must not use it and instead performs a real read next time,
198
+ * which re-establishes trust.
199
+ */
200
+ private readonly closedCache;
201
+ /** True once `subscribeTransitions` has actually succeeded; false (the closed cache is never consulted) if `localCache` wasn't set, the store lacks the capability, or the initial subscribe attempt failed. */
202
+ private localCacheActive;
203
+ private unsubscribeTransitions?;
162
204
  private readonly failures;
163
205
  private readonly lastFailureTime;
164
206
  private readonly openOperations;
@@ -169,7 +211,33 @@ export declare class CircuitBreaker {
169
211
  private readonly sampleCounts;
170
212
  private readonly monitorHandle;
171
213
  constructor(options?: CircuitBreakerOptions);
172
- /** Stops the local-mode monitor interval. Call this when the breaker is no longer needed. */
214
+ /**
215
+ * Attempts to establish the closed-state cache's push invalidation via
216
+ * `Store.subscribeTransitions`. Best-effort and one-shot: if the store
217
+ * doesn't implement it, or the initial subscribe attempt itself throws,
218
+ * `localCache` stays disabled for this breaker's lifetime (a one-time
219
+ * warning either way) — reconnects *after* a successful subscription are
220
+ * the store implementation's own responsibility (see the interface doc).
221
+ */
222
+ private setupLocalCache;
223
+ /**
224
+ * `null` means the subscription's connection dropped or errored: every
225
+ * cached entry may now be missing events, so all of them are marked
226
+ * untrusted until their next authoritative (real-read) refresh — the
227
+ * fallback poll in `monitor()` is what eventually forces that refresh if
228
+ * no call happens to trigger one first.
229
+ */
230
+ private handleTransitionEvent;
231
+ /**
232
+ * Refreshes the closed-state cache entry for `operation` with a fresh
233
+ * authoritative read, whether the write came from this instance (already
234
+ * has the resulting state in hand) or a real gate read. A no-op if
235
+ * `localCache` isn't active. Never regresses a cache entry that a newer
236
+ * transition event has already superseded, since a real read can race a
237
+ * push notification and land after it.
238
+ */
239
+ private updateClosedCache;
240
+ /** Stops the local-mode monitor interval and any transition-event subscription. Call this when the breaker is no longer needed. */
173
241
  destroy(): void;
174
242
  execute<T>(operation: string, fn: () => Promise<T>): Promise<T>;
175
243
  /**
@@ -240,6 +308,15 @@ export declare class CircuitBreaker {
240
308
  private key;
241
309
  private trialKey;
242
310
  private stateTtlSeconds;
311
+ /**
312
+ * Same generous, non-load-bearing safety-net TTL as `stateTtlSeconds`, for
313
+ * the one call site (`reopenTrialFailureAtomic`) that must pick a TTL
314
+ * *before* knowing the resulting `openCount` the atomic script itself
315
+ * computes server-side. Sized off `maxResetTimeout` — the ceiling
316
+ * `backoffCapped` can ever reach at any `openCount` — so it's always at
317
+ * least as generous as the true per-`openCount` value would be.
318
+ */
319
+ private maxStateTtlSeconds;
243
320
  private trialTtlSeconds;
244
321
  private reportStoreError;
245
322
  private warnMissingCapability;
@@ -302,6 +379,7 @@ export declare class CircuitBreaker {
302
379
  * breaker open with no way to ever attempt another trial.
303
380
  */
304
381
  private monitor;
382
+ private reconcileClosedCache;
305
383
  private executeWithTimeout;
306
384
  }
307
385
  //# sourceMappingURL=circuit-breaker.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAChC,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAA8B,MAAM,SAAS,CAAC;AAE1F,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAE3D,MAAM,WAAW,aAAa;IAC5B,IAAI,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,GAAG,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CAC5B;AAED,YAAY,EAAE,mBAAmB,EAAE,cAAc,EAAE,CAAC;AAEpD;;;;;GAKG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC;gBAEZ,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO;CAK9C;AAED,MAAM,WAAW,qBAAqB;IACpC;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,EAAE,aAAa,GAAG,WAAW,CAAC;IACvC,wGAAwG;IACxG,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;OAIG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,mGAAmG;IACnG,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,2FAA2F;IAC3F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qIAAqI;IACrI,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,oFAAoF;IACpF,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;OAIG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IACjE;;;;;OAKG;IACH,KAAK,CAAC,EAAE,KAAK,CAAC;IACd;;;;;;;;;;;;;;OAcG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3D,MAAM,CAAC,EAAE,aAAa,CAAC;IACvB;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;;;;;OAQG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;CAC7B;AA0BD;;;;;;;;;;;;;;GAcG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAqBtB;IAEF,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAU;IACtC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAqB;IACxD,OAAO,CAAC,iCAAiC,CAAS;IAClD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,0BAA0B,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAqC;IAC7D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgF;IACpH;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyG;IAGpI,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA6B;IACtD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA6B;IAC7D,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAqB;IACpD,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAA6B;IAC5D,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA6B;IAE1D,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAiC;gBAEnD,OAAO,GAAE,qBAA0B;IA8B/C,6FAA6F;IAC7F,OAAO,IAAI,IAAI;IAIT,OAAO,CAAC,CAAC,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;IAKrE;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IAsB1B;;;;;;;;OAQG;IACH,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAiBtC;;;;;OAKG;IACG,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAM1D,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAetC,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAStC;;;;;;;;;OASG;YACW,yBAAyB;IAQjC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAYzD,KAAK,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAgB7C,wPAAwP;IACxP,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,cAAc;IAK7C,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC;IAM/C,OAAO,CAAC,UAAU;IAWlB,OAAO,CAAC,aAAa;IAKrB;;;;;;;;;OASG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,cAAc;IAatB,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,QAAQ;IAIhB,OAAO,CAAC,eAAe;IAUvB,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,qBAAqB;YAQf,OAAO;YASP,OAAO;IAQrB,qFAAqF;YACvE,WAAW;IAQzB,4NAA4N;YAC9M,YAAY;IAQ1B;;;;;;;;;;;;;;;;OAgBG;YACW,gBAAgB;YAahB,gBAAgB;IAM9B;;;;;OAKG;YACW,eAAe;YA0Df,kBAAkB;YAkDlB,gCAAgC;IA+C9C;;;;OAIG;YACW,wBAAwB;YAoDxB,iCAAiC;YAmBjC,YAAY;IAkC1B,gFAAgF;IAChF,OAAO,CAAC,aAAa;IAcrB,oFAAoF;IACpF,OAAO,CAAC,cAAc;IAuBtB,OAAO,CAAC,0BAA0B;IAclC,kIAAkI;IAClI,OAAO,CAAC,sBAAsB;IAkB9B,OAAO,CAAC,2BAA2B;IAUnC,OAAO,CAAC,UAAU;IAiBlB;;;;;;OAMG;IACH,OAAO,CAAC,OAAO;IAgBf,OAAO,CAAC,kBAAkB;CAiB3B"}
1
+ {"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAChC,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAA8B,eAAe,EAAE,MAAM,SAAS,CAAC;AAE3G,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAE3D,MAAM,WAAW,aAAa;IAC5B,IAAI,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,GAAG,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CAC5B;AAED,YAAY,EAAE,mBAAmB,EAAE,cAAc,EAAE,eAAe,EAAE,CAAC;AAErE;;;;;GAKG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC;gBAEZ,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO;CAK9C;AAED,MAAM,WAAW,qBAAqB;IACpC;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,EAAE,aAAa,GAAG,WAAW,CAAC;IACvC,wGAAwG;IACxG,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;OAIG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,mGAAmG;IACnG,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,2FAA2F;IAC3F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qIAAqI;IACrI,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,oFAAoF;IACpF,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;OAIG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IACjE;;;;;OAKG;IACH,KAAK,CAAC,EAAE,KAAK,CAAC;IACd;;;;;;;;;;;;;;OAcG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3D,MAAM,CAAC,EAAE,aAAa,CAAC;IACvB;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;;;;;OAQG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;;;;;;;;;;;;;;;;;;;OAsBG;IACH,UAAU,CAAC,EAAE;QAAE,gBAAgB,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;CAC5C;AAqCD;;;;;;;;;;;;;;GAcG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAuBtB;IAEF,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAU;IACtC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAqB;IACxD,OAAO,CAAC,iCAAiC,CAAS;IAClD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,0BAA0B,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAqC;IAC7D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgF;IACpH;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyG;IAEpI,yNAAyN;IACzN,OAAO,CAAC,QAAQ,CAAC,SAAS,CAA4B;IACtD;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,WAAW,CAA2G;IACvI,gNAAgN;IAChN,OAAO,CAAC,gBAAgB,CAAS;IACjC,OAAO,CAAC,sBAAsB,CAAC,CAAa;IAG5C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA6B;IACtD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA6B;IAC7D,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAqB;IACpD,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAA6B;IAC5D,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA6B;IAE1D,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAiC;gBAEnD,OAAO,GAAE,qBAA0B;IAmC/C;;;;;;;OAOG;YACW,eAAe;IAkB7B;;;;;;OAMG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,iBAAiB;IAiBzB,mIAAmI;IACnI,OAAO,IAAI,IAAI;IAKT,OAAO,CAAC,CAAC,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;IAKrE;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IAsB1B;;;;;;;;OAQG;IACH,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAiBtC;;;;;OAKG;IACG,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAM1D,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAetC,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAStC;;;;;;;;;OASG;YACW,yBAAyB;IAQjC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAYzD,KAAK,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAiB7C,wPAAwP;IACxP,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,cAAc;IAK7C,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC;IAM/C,OAAO,CAAC,UAAU;IAWlB,OAAO,CAAC,aAAa;IAKrB;;;;;;;;;OASG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,cAAc;IAatB,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,QAAQ;IAIhB,OAAO,CAAC,eAAe;IAUvB;;;;;;;OAOG;IACH,OAAO,CAAC,kBAAkB;IAI1B,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,qBAAqB;YAUf,OAAO;YASP,OAAO;IAQrB,qFAAqF;YACvE,WAAW;IAQzB,4NAA4N;YAC9M,YAAY;IAQ1B;;;;;;;;;;;;;;;;OAgBG;YACW,gBAAgB;YAahB,gBAAgB;IAyB9B;;;;;OAKG;YACW,eAAe;YAsEf,kBAAkB;YAsDlB,gCAAgC;IAkD9C;;;;OAIG;YACW,wBAAwB;YAuDxB,iCAAiC;YAyCjC,YAAY;IAkC1B,gFAAgF;IAChF,OAAO,CAAC,aAAa;IAcrB,oFAAoF;IACpF,OAAO,CAAC,cAAc;IAuBtB,OAAO,CAAC,0BAA0B;IAclC,kIAAkI;IAClI,OAAO,CAAC,sBAAsB;IAkB9B,OAAO,CAAC,2BAA2B;IAUnC,OAAO,CAAC,UAAU;IAiBlB;;;;;;OAMG;IACH,OAAO,CAAC,OAAO;YA6BD,oBAAoB;IAOlC,OAAO,CAAC,kBAAkB;CAiB3B"}
@@ -52,6 +52,21 @@ class CircuitBreaker {
52
52
  * an early close by another instance or a manual `reset()`.
53
53
  */
54
54
  this.cachedOpen = new Map();
55
+ /** Fixed, unprefixed key for the shared transition-event stream — one per store/prefix, not one per operation. `RedisStore` applies its own `keyPrefix` the same way it does for every other key this class hands it. */
56
+ this.eventsKey = 'circuit_breaker:events';
57
+ /**
58
+ * Distributed mode only, `localCache` opt-in: a local, push-invalidated
59
+ * record of the last known state for `operation`, used to serve gate
60
+ * decisions for CLOSED state with zero store reads while fresh and
61
+ * trusted. `trusted: false` means the last write we saw for this entry
62
+ * had no `version` (unknown provenance — an older library version wrote
63
+ * it) or the transition subscription itself dropped; either way, the
64
+ * gate must not use it and instead performs a real read next time,
65
+ * which re-establishes trust.
66
+ */
67
+ this.closedCache = new Map();
68
+ /** True once `subscribeTransitions` has actually succeeded; false (the closed cache is never consulted) if `localCache` wasn't set, the store lacks the capability, or the initial subscribe attempt failed. */
69
+ this.localCacheActive = false;
55
70
  // local mode state
56
71
  this.failures = new Map();
57
72
  this.lastFailureTime = new Map();
@@ -82,14 +97,90 @@ class CircuitBreaker {
82
97
  logger: options.logger ?? noopLogger,
83
98
  maxOperations: options.maxOperations,
84
99
  openCacheRefreshMs: options.openCacheRefreshMs ?? 2000,
100
+ localCache: options.localCache ? { staleToleranceMs: options.localCache.staleToleranceMs ?? 100 } : undefined,
85
101
  };
86
102
  this.distributed = !!options.store;
87
103
  this.monitorHandle = setInterval(() => this.monitor(), this.options.monitorInterval);
88
104
  this.monitorHandle.unref?.();
105
+ if (this.distributed && this.options.localCache) {
106
+ void this.setupLocalCache();
107
+ }
89
108
  }
90
- /** Stops the local-mode monitor interval. Call this when the breaker is no longer needed. */
109
+ /**
110
+ * Attempts to establish the closed-state cache's push invalidation via
111
+ * `Store.subscribeTransitions`. Best-effort and one-shot: if the store
112
+ * doesn't implement it, or the initial subscribe attempt itself throws,
113
+ * `localCache` stays disabled for this breaker's lifetime (a one-time
114
+ * warning either way) — reconnects *after* a successful subscription are
115
+ * the store implementation's own responsibility (see the interface doc).
116
+ */
117
+ async setupLocalCache() {
118
+ const store = this.options.store;
119
+ if (!store.subscribeTransitions) {
120
+ this.options.logger.warn('localCache is set but the store does not implement subscribeTransitions(); the closed-state cache stays disabled (falling back to open-cache-only behavior). See README.');
121
+ return;
122
+ }
123
+ try {
124
+ this.unsubscribeTransitions = await store.subscribeTransitions(this.eventsKey, (event) => this.handleTransitionEvent(event));
125
+ this.localCacheActive = true;
126
+ }
127
+ catch (error) {
128
+ this.options.logger.warn(`localCache is set but subscribing to transition events failed; the closed-state cache stays disabled: ${error?.message ?? error}`);
129
+ }
130
+ }
131
+ /**
132
+ * `null` means the subscription's connection dropped or errored: every
133
+ * cached entry may now be missing events, so all of them are marked
134
+ * untrusted until their next authoritative (real-read) refresh — the
135
+ * fallback poll in `monitor()` is what eventually forces that refresh if
136
+ * no call happens to trigger one first.
137
+ */
138
+ handleTransitionEvent(event) {
139
+ if (event === null) {
140
+ for (const entry of this.closedCache.values())
141
+ entry.trusted = false;
142
+ return;
143
+ }
144
+ const existing = this.closedCache.get(event.operation);
145
+ if (existing && event.version <= existing.version)
146
+ return; // stale/duplicate delivery -- ignore
147
+ this.closedCache.set(event.operation, {
148
+ state: { ...types_1.CLOSED_STATE, isOpen: event.isOpen, lastFailure: event.lastFailure, openCount: event.openCount, version: event.version },
149
+ version: event.version,
150
+ fetchedAt: Date.now(),
151
+ trusted: true,
152
+ });
153
+ }
154
+ /**
155
+ * Refreshes the closed-state cache entry for `operation` with a fresh
156
+ * authoritative read, whether the write came from this instance (already
157
+ * has the resulting state in hand) or a real gate read. A no-op if
158
+ * `localCache` isn't active. Never regresses a cache entry that a newer
159
+ * transition event has already superseded, since a real read can race a
160
+ * push notification and land after it.
161
+ */
162
+ updateClosedCache(operation, state) {
163
+ if (!this.localCacheActive)
164
+ return;
165
+ // null (never written) is unambiguously clean, same as CLOSED_STATE's
166
+ // own version: 0 -- distrust is for a *stored* object that predates the
167
+ // version field, not for "nothing has ever been written here."
168
+ const trusted = state === null || state.version !== undefined;
169
+ const version = state?.version ?? 0;
170
+ const existing = this.closedCache.get(operation);
171
+ if (trusted && existing?.trusted && version < existing.version)
172
+ return; // don't regress past a newer stream event
173
+ this.closedCache.set(operation, {
174
+ state: state ?? { ...types_1.CLOSED_STATE },
175
+ version,
176
+ fetchedAt: Date.now(),
177
+ trusted,
178
+ });
179
+ }
180
+ /** Stops the local-mode monitor interval and any transition-event subscription. Call this when the breaker is no longer needed. */
91
181
  destroy() {
92
182
  clearInterval(this.monitorHandle);
183
+ this.unsubscribeTransitions?.();
93
184
  }
94
185
  async execute(operation, fn) {
95
186
  this.checkOperationName(operation);
@@ -212,6 +303,7 @@ class CircuitBreaker {
212
303
  await this.safeDelTrial(operation);
213
304
  this.cachedResetTimeouts.delete(operation);
214
305
  this.cachedOpen.delete(operation);
306
+ this.closedCache.delete(operation);
215
307
  if (prior?.isOpen) {
216
308
  this.options.onStateChange?.('closed', operation);
217
309
  }
@@ -303,6 +395,17 @@ class CircuitBreaker {
303
395
  // to vary a generous cleanup TTL.
304
396
  return Math.max(1, Math.ceil((this.backoffCapped(openCount) * 3) / 1000));
305
397
  }
398
+ /**
399
+ * Same generous, non-load-bearing safety-net TTL as `stateTtlSeconds`, for
400
+ * the one call site (`reopenTrialFailureAtomic`) that must pick a TTL
401
+ * *before* knowing the resulting `openCount` the atomic script itself
402
+ * computes server-side. Sized off `maxResetTimeout` — the ceiling
403
+ * `backoffCapped` can ever reach at any `openCount` — so it's always at
404
+ * least as generous as the true per-`openCount` value would be.
405
+ */
406
+ maxStateTtlSeconds() {
407
+ return Math.max(1, Math.ceil((this.options.maxResetTimeout * 3) / 1000));
408
+ }
306
409
  trialTtlSeconds() {
307
410
  return Math.max(1, Math.ceil(this.options.trialTimeout / 1000));
308
411
  }
@@ -384,9 +487,29 @@ class CircuitBreaker {
384
487
  }
385
488
  }
386
489
  async closeDistributed(operation) {
490
+ if (this.options.store.closeAtomic) {
491
+ try {
492
+ const next = await this.options.store.closeAtomic(this.key(operation), this.eventsKey, {
493
+ ttlSeconds: this.stateTtlSeconds(0),
494
+ operation,
495
+ });
496
+ this.cachedResetTimeouts.delete(operation);
497
+ this.cachedOpen.delete(operation);
498
+ this.updateClosedCache(operation, next);
499
+ return;
500
+ }
501
+ catch (error) {
502
+ this.reportStoreError(error, operation);
503
+ // fall through to non-atomic fallback below
504
+ }
505
+ }
506
+ else {
507
+ this.warnMissingCapability('closeAtomic');
508
+ }
387
509
  await this.safeSet(operation, { ...types_1.CLOSED_STATE }, this.stateTtlSeconds(0));
388
510
  this.cachedResetTimeouts.delete(operation);
389
511
  this.cachedOpen.delete(operation);
512
+ this.updateClosedCache(operation, { ...types_1.CLOSED_STATE });
390
513
  }
391
514
  /**
392
515
  * Determines whether a call is allowed through in distributed mode.
@@ -395,6 +518,15 @@ class CircuitBreaker {
395
518
  * `canExecuteAsync`) is read-only and never claims.
396
519
  */
397
520
  async gateDistributed(operation, claim) {
521
+ if (this.localCacheActive) {
522
+ const entry = this.closedCache.get(operation);
523
+ if (entry && entry.trusted && !entry.state.isOpen && Date.now() - entry.fetchedAt < this.options.localCache.staleToleranceMs) {
524
+ return { allowed: true, isTrial: false, assumeClean: true };
525
+ }
526
+ // entry.state.isOpen, stale, untrusted, or absent -- defer entirely to
527
+ // the open-state cache / a real read below, which remain
528
+ // authoritative for all open/rejection handling.
529
+ }
398
530
  const cached = this.cachedOpen.get(operation);
399
531
  if (cached && this.options.openCacheRefreshMs > 0) {
400
532
  const now = Date.now();
@@ -421,6 +553,7 @@ class CircuitBreaker {
421
553
  return { allowed: true, isTrial: false };
422
554
  throw new StoreUnavailableError(operation, error);
423
555
  }
556
+ this.updateClosedCache(operation, state); // every real read refreshes the closed cache too, whatever it finds
424
557
  if (!state?.isOpen) {
425
558
  this.cachedOpen.delete(operation);
426
559
  return { allowed: true, isTrial: false, observedState: state };
@@ -474,15 +607,19 @@ class CircuitBreaker {
474
607
  else {
475
608
  // consecutive strategy, non-trial success: only write if the gate
476
609
  // positively observed dirty state. If the read errored
477
- // (observedState undefined), write anyway — we can't know.
610
+ // (observedState undefined), write anyway — we can't know. A
611
+ // closed-cache fast-pathed gate (`assumeClean`) skips the write
612
+ // too, accepting the same bounded staleness the 0-read cache hit
613
+ // itself already accepted -- this is what gets `consecutive` to a
614
+ // genuine 0-round-trip healthy path.
478
615
  //
479
616
  // Race analysis: if a concurrent failure lands between our gate
480
- // read and this skipped write, not writing preserves that failure
481
- // count — strictly safer than the old unconditional
617
+ // read (or cache hit) and this skipped write, not writing preserves
618
+ // that failure count — strictly safer than the old unconditional
482
619
  // `closeDistributed`, which could erase a legitimate concurrent
483
620
  // increment by overwriting it with CLOSED_STATE.
484
621
  const s = gate.observedState;
485
- const observedClean = s !== undefined && (s === null || (!s.isOpen && s.failures === 0));
622
+ const observedClean = gate.assumeClean || (s !== undefined && (s === null || (!s.isOpen && s.failures === 0)));
486
623
  if (!observedClean) {
487
624
  await this.closeDistributed(operation);
488
625
  }
@@ -511,10 +648,12 @@ class CircuitBreaker {
511
648
  ttlSeconds: this.stateTtlSeconds(0),
512
649
  failureThreshold: this.options.failureThreshold,
513
650
  now: Date.now(),
651
+ operation,
514
652
  });
515
653
  this.options.logger.warn(`Recorded failure for operation: ${operation}, total failures: ${next.failures}`);
516
654
  if (next.isOpen)
517
655
  this.cacheOpenState(operation, next);
656
+ this.updateClosedCache(operation, next);
518
657
  if (next.isOpen && next.failures === this.options.failureThreshold) {
519
658
  this.options.logger.warn(`Circuit breaker opened for operation: ${operation}`);
520
659
  this.options.onStateChange?.('open', operation);
@@ -546,6 +685,7 @@ class CircuitBreaker {
546
685
  await this.safeSet(operation, next, this.stateTtlSeconds(0));
547
686
  if (next.isOpen)
548
687
  this.cacheOpenState(operation, next);
688
+ this.updateClosedCache(operation, next);
549
689
  this.options.logger.warn(`Recorded failure for operation: ${operation}, total failures: ${next.failures}`);
550
690
  if (next.isOpen && !current.isOpen) {
551
691
  this.options.logger.warn(`Circuit breaker opened for operation: ${operation}`);
@@ -567,9 +707,11 @@ class CircuitBreaker {
567
707
  errorRateThreshold: this.options.errorRateThreshold,
568
708
  ttlSeconds: this.stateTtlSeconds(0),
569
709
  now: Date.now(),
710
+ operation,
570
711
  });
571
712
  if (next.isOpen)
572
713
  this.cacheOpenState(operation, next);
714
+ this.updateClosedCache(operation, next);
573
715
  if (next.openedNow) {
574
716
  this.options.logger.warn(`Circuit breaker opened for operation: ${operation} (error rate ${(next.errorRate * 100).toFixed(1)}% over ${next.sampleCount} calls)`);
575
717
  this.options.onStateChange?.('open', operation);
@@ -600,12 +742,35 @@ class CircuitBreaker {
600
742
  await this.safeSet(operation, next, this.stateTtlSeconds(0));
601
743
  if (isOpen)
602
744
  this.cacheOpenState(operation, next);
745
+ this.updateClosedCache(operation, next);
603
746
  if (isOpen && !current.isOpen) {
604
747
  this.options.logger.warn(`Circuit breaker opened for operation: ${operation} (error rate ${(errorRate * 100).toFixed(1)}% over ${sampleCount} calls)`);
605
748
  this.options.onStateChange?.('open', operation);
606
749
  }
607
750
  }
608
751
  async reopenAfterFailedTrialDistributed(operation) {
752
+ if (this.options.store.reopenTrialFailureAtomic) {
753
+ try {
754
+ const next = await this.options.store.reopenTrialFailureAtomic(this.key(operation), this.eventsKey, {
755
+ ttlSeconds: this.maxStateTtlSeconds(),
756
+ failureThreshold: this.options.failureThreshold,
757
+ now: Date.now(),
758
+ operation,
759
+ });
760
+ this.cacheOpenState(operation, next);
761
+ this.updateClosedCache(operation, next);
762
+ this.options.logger.warn(`Half-open trial failed for operation: ${operation}; backing off (attempt ${next.openCount + 1})`);
763
+ this.options.onStateChange?.('open', operation);
764
+ return;
765
+ }
766
+ catch (error) {
767
+ this.reportStoreError(error, operation);
768
+ // fall through to non-atomic fallback below
769
+ }
770
+ }
771
+ else {
772
+ this.warnMissingCapability('reopenTrialFailureAtomic');
773
+ }
609
774
  const current = (await this.safeGet(operation)) ?? { ...types_1.CLOSED_STATE, isOpen: true };
610
775
  const openCount = (current.openCount ?? 0) + 1;
611
776
  const next = {
@@ -618,6 +783,7 @@ class CircuitBreaker {
618
783
  };
619
784
  await this.safeSet(operation, next, this.stateTtlSeconds(openCount));
620
785
  this.cacheOpenState(operation, next); // this instance authored the open state; cache it
786
+ this.updateClosedCache(operation, next);
621
787
  this.options.logger.warn(`Half-open trial failed for operation: ${operation}; backing off (attempt ${openCount + 1})`);
622
788
  this.options.onStateChange?.('open', operation);
623
789
  }
@@ -760,6 +926,24 @@ class CircuitBreaker {
760
926
  this.options.logger.warn(`Circuit breaker trial for operation: ${operation} appears stuck (wrapped function never settled); force-releasing after ${staleBound}ms. Consider configuring a timeout.`);
761
927
  }
762
928
  }
929
+ // Missed-message safety net for the closed-state cache: transitions are
930
+ // normally learned about push-style via subscribeTransitions, but a
931
+ // dropped stream message (or a subscription gap) would otherwise leave
932
+ // a cache entry stale forever. In steady state this is the only
933
+ // background traffic the closed-state cache generates -- one GET per
934
+ // stale operation per monitor tick, not per call.
935
+ if (this.distributed && this.localCacheActive) {
936
+ const pollAgeMs = 5000;
937
+ for (const [operation, entry] of this.closedCache.entries()) {
938
+ if (now - entry.fetchedAt > pollAgeMs) {
939
+ void this.reconcileClosedCache(operation);
940
+ }
941
+ }
942
+ }
943
+ }
944
+ async reconcileClosedCache(operation) {
945
+ const state = await this.safeGet(operation);
946
+ this.updateClosedCache(operation, state);
763
947
  }
764
948
  // ---- shared -------------------------------------------------------------
765
949
  executeWithTimeout(fn) {