redeye-breaker 0.3.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -104,7 +104,7 @@ Every process pointed at the same Redis instance (and using the same operation n
104
104
 
105
105
  ### Bring your own store
106
106
 
107
- `RedisStore` implements a `Store` interface with two required methods and four *optional* ones:
107
+ `RedisStore` implements a `Store` interface with two required methods and seven *optional* ones:
108
108
 
109
109
  ```ts
110
110
  export interface Store {
@@ -115,24 +115,67 @@ export interface Store {
115
115
  // Optional — implement these for the reliability guarantees below.
116
116
  // Without them, redeye falls back to best-effort semantics and logs a
117
117
  // one-time warning telling you exactly what's degraded.
118
- recordFailureAtomic?(key: string, opts: { ttlSeconds: number; failureThreshold: number; now: number }): Promise<CircuitBreakerState>;
119
- recordOutcomeAtomic?(key: string, opts: { success: boolean; decay: number; minimumCalls: number; errorRateThreshold: number; ttlSeconds: number; now: number }): Promise<CircuitBreakerState & { openedNow: boolean }>;
118
+ recordFailureAtomic?(key: string, opts: { ttlSeconds: number; failureThreshold: number; now: number; operation: string }): Promise<CircuitBreakerState>;
119
+ recordOutcomeAtomic?(key: string, opts: { success: boolean; decay: number; minimumCalls: number; errorRateThreshold: number; ttlSeconds: number; now: number; operation: string }): Promise<CircuitBreakerState & { openedNow: boolean }>;
120
120
  claimTrial?(key: string, ttlSeconds: number): Promise<string | null>;
121
121
  releaseTrial?(key: string, token: string): Promise<void>;
122
+
123
+ // Optional — only needed for the closed-state local cache (`localCache`, see below).
124
+ closeAtomic?(key: string, eventsKey: string, opts: { ttlSeconds: number; operation: string }): Promise<CircuitBreakerState>;
125
+ reopenTrialFailureAtomic?(key: string, eventsKey: string, opts: { ttlSeconds: number; failureThreshold: number; now: number; operation: string }): Promise<CircuitBreakerState>;
126
+ subscribeTransitions?(eventsKey: string, handler: (event: TransitionEvent | null) => void): Promise<() => void>;
122
127
  }
123
128
  ```
124
129
 
125
- - `recordFailureAtomic` (`strategy: 'consecutive'`) should increment the failure count and decide `isOpen` in one atomic round trip (a Lua script in Redis, a conditional update in DynamoDB, etc.).
126
- - `recordOutcomeAtomic` (`strategy: 'errorRate'`) should fold one call's outcome into the EWMA rate and decide `isOpen` in one atomic round trip — called on every closed-phase call, not just failures.
130
+ - `recordFailureAtomic` (`strategy: 'consecutive'`) should increment the failure count and decide `isOpen` in one atomic round trip (a Lua script in Redis, a conditional update in DynamoDB, etc.). Should also bump a monotonic `version` counter and publish a transition event to `eventsKey` (see below) when — and only when — this call flips `isOpen`; a plain sub-threshold increment isn't a transition and shouldn't publish one.
131
+ - `recordOutcomeAtomic` (`strategy: 'errorRate'`) should fold one call's outcome into the EWMA rate and decide `isOpen` in one atomic round trip — called on every closed-phase call, not just failures. Same `version`/publish rule as above, keyed off an `isOpen` flip.
127
132
  - `claimTrial` should be a conditional "create if absent" write (`SET key token NX EX ttl` in Redis, with `token` a fresh unique value per call, e.g. a UUID) — it's what makes half-open recovery exclusive to one caller instead of a free-for-all. Return the token you wrote if this call won the claim, or `null` if the key was already taken.
128
133
  - `releaseTrial` should be a compare-and-delete (`if GET key == token then DEL key` in Redis, a conditional delete elsewhere), not an unconditional delete. This is what stops a trial that outran its TTL (see [item 11](#whats-still-a-fundamental-tradeoff-not-a-bug)) from deleting a *different* instance's newer claim on the same key when it finally gets around to releasing its own now-stale one. If omitted (or if a call has no confirmed ownership token to release at all — e.g. `claimTrial` itself errored), redeye logs a one-time warning and leaves the claim for its TTL to expire naturally, rather than risk an unconditional delete that could destroy a claim it never confirmed was its own.
134
+ - `closeAtomic` should close `key`, bumping `version` and publishing a transition, **but only if there was anything to clear** (it was open, or had accumulated sub-threshold failures) — a true no-op, no write at all, when the state was already clean. This is what lets the healthy-path write-skip (see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug)) happen atomically server-side instead of relying on the caller's own possibly-stale view.
135
+ - `reopenTrialFailureAtomic` should reopen `key` after a failed half-open trial — `isOpen: true`, `openCount + 1`, `failures` raised to at least `failureThreshold`, `lastFailure: now`, `version + 1` — and always publish a transition (a failed trial is always a real transition, unlike `closeAtomic`).
136
+ - `subscribeTransitions` should subscribe to a stream of transition events at `eventsKey` — **one shared stream per store/prefix, not one per operation** — and invoke `handler` with each decoded event, or with `null` whenever the underlying subscription drops or errors (callers must then distrust every locally cached entry until it's re-verified by a real read; implementations should retry their own subscription with backoff, not expect the caller to re-subscribe). Returns an unsubscribe function.
137
+
138
+ Implement the two required methods against Memcached, DynamoDB, your own cache wrapper, etc., and you have a working (best-effort) distributed breaker. Add `recordFailureAtomic`/`recordOutcomeAtomic`/`claimTrial`/`releaseTrial` when that store supports a real atomic increment and a real conditional write, and you get the full reliability guarantees. Add `closeAtomic`/`reopenTrialFailureAtomic`/`subscribeTransitions` on top of that if you also want the [closed-state local cache](#local-caching-hybrid-mode) — every capability is independently optional, so skip any subset that doesn't fit your store.
139
+
140
+ ## Local caching (hybrid mode)
141
+
142
+ Everything up to this point reads the store on every gate check (a genuine read every closed-path call; a cached, 0-read rejection once open — see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug)). `localCache` goes one step further, opt-in, only for `RedisStore` (or any store implementing `subscribeTransitions`): it caches the **closed** state locally too, invalidated push-style instead of polling, so a healthy call can skip the store read entirely, not just the write.
143
+
144
+ ```ts
145
+ const breaker = new CircuitBreaker({
146
+ failureThreshold: 5,
147
+ resetTimeout: 60_000,
148
+ store,
149
+ localCache: { staleToleranceMs: 100 }, // opt-in; default staleToleranceMs is 100 once the object is present
150
+ });
151
+ ```
129
152
 
130
- Implement the two required methods against Memcached, DynamoDB, your own cache wrapper, etc., and you have a working (best-effort) distributed breaker. Add the optional methods relevant to the strategy you use when that store supports a real atomic increment and a real conditional write, and you get the full reliability guarantees.
153
+ **What's cached, and how it's invalidated.** Per `operation`, redeye keeps `{ state, version, fetchedAt, trusted }` in memory. A gate check allows immediately, with zero store reads, when the entry is `trusted`, closed (`!state.isOpen`), and fresher than `staleToleranceMs`. Three independent mechanisms keep this from going stale silently:
154
+
155
+ - **Push invalidation.** `subscribeTransitions` delivers a decoded event the instant any instance's write flips `isOpen` or clears real accumulated state (never on a plain sub-threshold counter update — those aren't transitions and don't invalidate the cache, by design, since they can't affect the *closed* fast path this cache serves). A higher `version` than what's cached overwrites the entry immediately.
156
+ - **Fallback poll.** The existing local-mode monitor interval doubles as a missed-message safety net in distributed mode: any cached entry older than 5s gets one authoritative `get` and is reconciled by `version`. In steady state (no missed messages) this is the *only* background traffic the cache generates — one `GET` per stale operation per sweep, not per call.
157
+ - **Your own writes are free.** The state `recordFailureAtomic`/`recordOutcomeAtomic`/`closeAtomic`/`reopenTrialFailureAtomic` hand back already updates this instance's own cache — the instance that causes a transition never has a stale window on it, regardless of stream or poll timing.
158
+
159
+ **The fail-open window, as a formula, not an adjective.** This is the one place in redeye that's biased toward availability over freshness on the *closed* side (everywhere else — the open-state cache, `failOpenOnStoreError: false`, etc. — is fail-closed or symmetric by default). If the breaker trips elsewhere in the fleet, a call landing on this instance's still-fresh cached-closed entry can pass through before that instance learns about it. The bound is exactly:
160
+
161
+ ```
162
+ worst-case fail-open window = staleToleranceMs + transition-delivery latency
163
+ ```
164
+
165
+ — and it's a hard ceiling, not a typical case: the cache stops trusting itself once `staleToleranceMs` elapses regardless of whether the transition event ever arrives (that's what makes the fallback poll a correctness backstop and not just a nice-to-have), and delivery latency in the common case is one Redis stream round trip, not a poll cycle. Set `staleToleranceMs` to whatever window you're comfortable a downstream outage could go un-noticed on this one path — `100`ms (the default) is a reasonable starting point for most services; push it lower if even a hundred milliseconds of continued traffic against a freshly-tripped dependency is unacceptable, at the cost of the cache being useful for a shorter fraction of the healthy path.
166
+
167
+ **Compare this to the open-state cache deliberately.** `openCacheRefreshMs` never risks an extra call through — staleness there can only make a rejection *late*, not wrong, because it always falls through to a real read right at the real eligibility instant (see limitation item 1). `localCache`'s closed-side cache is the opposite shape: staleness there can make an *allow* wrong for a bounded window, in exchange for the healthy path's read too. That's the real tradeoff you're opting into, not a bug to route around — the same one Envoy's outlier detection makes by default (cache state locally with a bounded TTL, accept staleness, skip the round trip), just off by default here and explicit about the bound when you turn it on.
168
+
169
+ **The version/absence rule, and mixed-fleet rolling deploys.** `CircuitBreakerState.version` is a monotonic counter that increments only on a real transition — its *absence* (not `0`) means "written by a library version that predates `localCache`, provenance unknown." A cache entry built from a version-less read is marked `trusted: false` and never used for the fast path; the very next call for that operation performs a real read instead, forever, until a version-bearing write appears. This is what makes a rolling deploy safe: instances running an old version keep working exactly as before (they don't publish transitions, and their writes lack `version`), and new instances simply never trust a versionless observation rather than guessing. Nothing about this requires deploying in any particular order.
170
+
171
+ **Strategy asymmetry, stated plainly.** `consecutive` is the strategy this cache is built for: combined with the healthy-path write skip, a run of successes against a warm cache is a genuine 0-store-op healthy path. `errorRate` cannot get the same benefit — `recordOutcomeAtomic` must fold *every* outcome (success or failure) into the EWMA to keep the rate meaningful, so it floors at 1 write per call no matter what's cached. `localCache` still saves the *read* for `errorRate` the same way it does for `consecutive`; it just can't save the write, structurally, not as an oversight.
172
+
173
+ **Requires `subscribeTransitions`.** Without it (a custom `Store` that doesn't implement the capability), `localCache` is a no-op: one-time warning, and behavior is identical to not setting it at all. `RedisStore` implements it via a dedicated (`duplicate()`d) blocking connection running `XREAD BLOCK` against the shared `circuit_breaker:events` stream, reconnecting with backoff on its own if that connection drops — see the `Store` interface docs above for the reconnect contract.
131
174
 
132
175
  ## API
133
176
 
134
177
  - `execute(operation, fn)` — runs `fn` if the breaker allows it (closed, or this call won the half-open trial); throws immediately without calling `fn` otherwise. Records the outcome automatically.
135
- - `canExecute(operation)` — synchronous, read-only, best-effort check. **Always returns `true` in distributed mode** (it can't await the store) — use `canExecuteAsync` there. Never claims a trial slot. Logs a one-time warning the first time it's called in distributed mode, since a `true` there doesn't reflect real breaker state.
178
+ - `canExecute(operation)` — synchronous, read-only, best-effort check. In distributed mode it can't await the store, so it can only consult the local open-state cache (see `openCacheRefreshMs` below): a `false` is a real, cached-open result, but a `true` is still only advisory it means either nothing is cached open, or nothing has been cached yet, not a confirmed closed state — use `canExecuteAsync` for that. Never claims a trial slot. Logs a one-time warning the first time it's called in distributed mode, explaining that asymmetry.
136
179
  - `canExecuteAsync(operation)` — async, store-aware, read-only check. Honors `failOpenOnStoreError`. Never claims a trial slot — see the TOCTOU note below.
137
180
  - `recordFailure(operation)` / `recordSuccess(operation)` — manually record an outcome without routing the call through `execute`. Note: these bypass the half-open trial-claim mechanism entirely (see [limitations](#reliability-model--limitations)) — prefer `execute`.
138
181
  - `getState(operation)` — returns `{ failures, lastFailure, isOpen, openCount, errorRate, sampleCount }`. `openCount` is how many consecutive half-open trials have failed since the breaker last fully closed (drives backoff). `errorRate`/`sampleCount` are only meaningful under `strategy: 'errorRate'`.
@@ -166,6 +209,8 @@ redeye can't stop you from doing this, but it can flag it: `execute`/`recordFail
166
209
  | `onStoreError` | none | `(error: unknown, operation: string) => void` — fired whenever a store read/write fails, in addition to being logged |
167
210
  | `logger` | no-op | `{ warn(msg), log(msg) }` — plug in your own logger |
168
211
  | `maxOperations` | none | Logs a one-time warning once the breaker has seen more than this many distinct `operation` names — see the cardinality note above. Unset: no limit, no warning. |
212
+ | `openCacheRefreshMs` | `2000` | Distributed mode only: while `operation` is known-open, reject locally without a store read, re-verifying against the store at most once per this many ms (to catch an early close by another instance's trial, or a manual `reset()`). Never delays trial eligibility — see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug). `0` disables the cache (a store read on every call, the pre-cache behavior). |
213
+ | `localCache` | none (disabled) | Distributed mode, opt-in, requires a store implementing `subscribeTransitions`: `{ staleToleranceMs?: number }` (default `100` once the object is present) — caches CLOSED state locally, invalidated push-style, so a healthy call can skip the store read too, not just the write. See [Local caching (hybrid mode)](#local-caching-hybrid-mode) for the fail-open window this trades for that. |
169
214
 
170
215
  ### Store unavailability: fail-open vs fail-closed
171
216
 
@@ -203,7 +248,7 @@ If your store *doesn't* implement the atomic method your chosen strategy needs (
203
248
 
204
249
  ### Split-brain: what's prevented, and what isn't
205
250
 
206
- There's no split-brain among your app instances in the classic sense, because instances never hold independent authoritative state to disagree over — every instance reads the store fresh before every single decision (`gateDistributed`) instead of trusting a locally cached opinion. The store is the only "brain"; instances are just readers/writers of it. Two mechanisms enforce agreement on the common path:
251
+ There's no split-brain among your app instances in the classic sense, because instances never hold independent authoritative state *to write from* — every state-changing decision goes through the store, and the two local caches (`openCacheRefreshMs`, `localCache`) exist only to skip *reading* it before an already-known-safe decision, never to substitute for it. The store is the only "brain"; instances are just readers/writers of it, some of the time from a recent, bounded-staleness memory of what they last read instead of asking again. Two mechanisms enforce agreement on the common path:
207
252
 
208
253
  - **Atomic writes.** `recordFailureAtomic`/`recordOutcomeAtomic` run as one Lua script — GET, decode, increment, decide, SET — inside Redis's single-threaded execution, so two instances failing at the same instant can't both read the same count and both write the same increment.
209
254
  - **Exclusive trial claim.** `claimTrial` is a `SET ... NX`, which Redis resolves atomically, so exactly one instance ever runs the half-open recovery probe — never two instances simultaneously deciding they're the one testing recovery.
@@ -220,7 +265,8 @@ So the design doesn't eliminate split-brain so much as route around it: it pushe
220
265
 
221
266
  These are true of any distributed circuit breaker, rate limiter, or lock built on a shared store — not gaps specific to redeye:
222
267
 
223
- 1. **Distributed mode costs Redis round trips per guarded call, not zero.** Every `execute()` call does a gate read (`store.get`) before deciding whether to proceed. What happens after that differs by strategy: `errorRate` writes (`recordOutcomeAtomic`) on every closed-phase call, success or failure, since the rate needs both to be meaningful. `consecutive` is cheaper on the healthy patha success only writes (`closeDistributed`) when the gate's own read observed accumulated state to clear (an open breaker, or `failures > 0`); against a clean key, a run of successes is 1 read and 0 writes per call, not 1-and-1. This is a measured claim about the steady state, not a blanket guarantee: a dependency flapping near the failure threshold, or one recovering through a half-open trial (which always writes its close, unconditionally, regardless of strategy see below), still writes on most or every call. A resolving half-open trial adds a third round trip to release its claim. On a latency-sensitive path this is added tail latency, and Redis load that scales with request volume — the opposite tradeoff from something like Envoy's outlier detection, which caches state locally with a short TTL and accepts bounded staleness instead of paying a store round trip per call. redeye chose per-call freshness over that; it's a reasonable choice for many services, but measure it before putting this in front of your highest-QPS call. Local mode has none of this cost see [Local mode vs. distributed mode](#local-mode-vs-distributed-mode-which-do-you-need) if you're not sure you need shared state at all.
268
+ 1. **Distributed mode's per-call store cost is opt-in-able down to a bound you choose, not fixed — but "zero" always means "zero, at a stated staleness price," never "zero, for free."** By default, every `execute()` call does a gate read (`store.get`) before deciding whether to proceed; what happens after differs by strategy `errorRate` writes (`recordOutcomeAtomic`) on every closed-phase call, success or failure, since the rate needs both to be meaningful (this never goes away, at any cache setting see below), while `consecutive` only writes when the gate's own read observed accumulated state to clear (an open breaker, or `failures > 0`), making a run of healthy successes 1 read and 0 writes per call by default, not 1-and-1. **Once the breaker is open, rejections stop costing a read at all**, unconditionally, no opt-in required: the gate caches the open state locally (`lastFailure`, `openCount`, and the same jittered `expiresAt` the eligibility check itself uses) and rejects straight from that cache, re-verifying against the store only once per `openCacheRefreshMs` (default 2000ms), and always falling through to a real read right at `expiresAt` so this cache can delay a rejection's freshness but can never delay a trial's eligibility, and never risks letting an extra call through. That's the fail-*closed*-biased half of the story: staleness there can only make things more conservative.
269
+ The closed, healthy path is the other half, and it's where a real tradeoff lives: **`localCache` (opt-in) is redeye's answer to exactly the caching-with-bounded-staleness tradeoff Envoy's outlier detection makes by default** — cache state locally, accept a bounded staleness window, skip the round trip. Off by default (redeye chooses per-call freshness there until you say otherwise), and explicit about the bound when you turn it on: the worst case is `staleToleranceMs + transition-delivery latency`, a formula, not an adjective — see [Local caching (hybrid mode)](#local-caching-hybrid-mode) for exactly what that means and what it doesn't cover (`errorRate`'s write floor, in particular, which `localCache` cannot remove — only `consecutive` reaches a genuine 0-store-op healthy path). A resolving half-open trial always adds a round trip to claim it and another to release its claim, at any cache setting, on either strategy — that path is never cacheable, and rightly so. Local mode has none of this cost at all — see [Local mode vs. distributed mode](#local-mode-vs-distributed-mode-which-do-you-need) if you're not sure you need shared state to begin with.
224
270
  2. **Correctness depends on trusting your store.** If Redis evicts a breaker key early under memory pressure (e.g. `maxmemory-policy allkeys-lru` with `circuit_breaker:*` competing for space with everything else), the breaker can reset earlier than intended. Mitigate by giving circuit-breaker keys their own keyspace/DB with a policy that doesn't evict them, or by monitoring `onStoreError` and Redis memory pressure directly. This is not fixable in-library — it's an operational configuration matter.
225
271
  3. **Distributed timing is sensitive to clock skew.** Backoff and jitter windows are computed by comparing each instance's local `Date.now()` against a `lastFailure` timestamp written by whichever instance recorded it. With reasonably synchronized clocks (standard NTP) this is a non-issue; on a fleet with significant clock drift, instances can disagree by that drift amount about exactly when a trial becomes eligible. This affects *timing* only — the half-open claim mechanism (`claimTrial`) still guarantees exactly one trial runs regardless of skew, so skew cannot cause a thundering herd, only a slightly early or late trial attempt.
226
272
  4. **`recordFailure`/`recordSuccess`/`canExecuteAsync` don't participate in trial claiming.** Only `execute()` claims and releases the half-open trial slot. If you build your own call flow around the manual API instead of `execute()`, you lose the single-trial guarantee and get the old "everyone retries once elapsed" behavior for that flow. Prefer `execute()`.
@@ -251,6 +297,8 @@ docker compose down
251
297
 
252
298
  It connects to `REDIS_URL` (default `redis://localhost:6379`).
253
299
 
300
+ `npm run bench` runs `bench/index.js` against a real Redis, comparing total store round trips and p50/p99 latency across three configurations: no caching, today's defaults (`openCacheRefreshMs`), and `localCache` on top. Same `REDIS_URL`; start Redis first the same way as the integration suite. See [BENCHMARK.md](BENCHMARK.md) for a full run's results and how to read them.
301
+
254
302
  ## Contributing
255
303
 
256
304
  See [CONTRIBUTING.md](CONTRIBUTING.md). Changes are tracked in [CHANGELOG.md](CHANGELOG.md).
@@ -1,11 +1,11 @@
1
1
  import { Store } from './store';
2
- import { CircuitBreakerState, CircuitMetrics } from './types';
2
+ import { CircuitBreakerState, CircuitMetrics, TransitionEvent } from './types';
3
3
  export type CircuitState = 'closed' | 'open' | 'half-open';
4
4
  export interface BreakerLogger {
5
5
  warn(message: string): void;
6
6
  log(message: string): void;
7
7
  }
8
- export type { CircuitBreakerState, CircuitMetrics };
8
+ export type { CircuitBreakerState, CircuitMetrics, TransitionEvent };
9
9
  /**
10
10
  * Thrown in distributed mode when the backing store is unreachable and
11
11
  * `failOpenOnStoreError` is `false`. Distinguishable from a downstream
@@ -112,6 +112,42 @@ export interface CircuitBreakerOptions {
112
112
  * indefinitely. Unset (default): no limit, no warning. See README.
113
113
  */
114
114
  maxOperations?: number;
115
+ /**
116
+ * Distributed mode only: while the breaker is known-open, reject locally
117
+ * without a store read, re-verifying against the store at most once per
118
+ * this many ms (to catch an early close by another instance's trial or a
119
+ * manual `reset()`). Staleness here is strictly fail-closed — a stale
120
+ * cache entry can only cause an extra rejection, never an extra call
121
+ * through. `0` disables the cache entirely (a store read on every call).
122
+ * Default: 2000.
123
+ */
124
+ openCacheRefreshMs?: number;
125
+ /**
126
+ * Distributed mode, opt-in. Serves gate decisions for CLOSED state from a
127
+ * local cache while it's fresher than `staleToleranceMs`, invalidated
128
+ * push-style via the store's transition stream (`Store.subscribeTransitions`)
129
+ * instead of polling — so a healthy call can skip the store read entirely,
130
+ * not just the write Release 1 already skips.
131
+ *
132
+ * This only ever short-circuits an *allow*: a cache entry observed open
133
+ * always defers to the open-state cache (`openCacheRefreshMs`) and a real
134
+ * read, which remains authoritative for all rejection handling. Requires
135
+ * a store implementing `subscribeTransitions`; without one, this option
136
+ * is a no-op (one-time warning) and behavior is unchanged from not
137
+ * setting it at all.
138
+ *
139
+ * The worst case is a bounded fail-open window: if the breaker trips
140
+ * elsewhere in the fleet, a calls still landing on a cached entry can be
141
+ * let through for up to `staleToleranceMs` plus however long the
142
+ * transition event takes to arrive — never longer, since the cache entry
143
+ * expires against `staleToleranceMs` on its own regardless of whether the
144
+ * event ever arrives. See the README's local caching section for the
145
+ * full fail-open/fail-closed asymmetry this creates versus the
146
+ * (fail-closed-biased) open-state cache.
147
+ */
148
+ localCache?: {
149
+ staleToleranceMs?: number;
150
+ };
115
151
  }
116
152
  /**
117
153
  * A circuit breaker with an optional distributed mode.
@@ -138,6 +174,33 @@ export declare class CircuitBreaker {
138
174
  private warnedDynamicOperationName;
139
175
  private readonly metrics;
140
176
  private readonly cachedResetTimeouts;
177
+ /**
178
+ * Distributed mode only: a local, best-effort record that `operation` was
179
+ * last observed open, so a burst of calls during an incident can reject
180
+ * without a store round trip each time. `expiresAt` is the same instant
181
+ * the gate's own eligibility check uses (`lastFailure + effectiveResetTimeout`)
182
+ * — this cache can only make a rejection cheaper, never delay eligibility
183
+ * for a trial past that instant. `lastRefetch` paces how often a real read
184
+ * re-verifies the cache against the store (`openCacheRefreshMs`), to catch
185
+ * an early close by another instance or a manual `reset()`.
186
+ */
187
+ private readonly cachedOpen;
188
+ /** Fixed, unprefixed key for the shared transition-event stream — one per store/prefix, not one per operation. `RedisStore` applies its own `keyPrefix` the same way it does for every other key this class hands it. */
189
+ private readonly eventsKey;
190
+ /**
191
+ * Distributed mode only, `localCache` opt-in: a local, push-invalidated
192
+ * record of the last known state for `operation`, used to serve gate
193
+ * decisions for CLOSED state with zero store reads while fresh and
194
+ * trusted. `trusted: false` means the last write we saw for this entry
195
+ * had no `version` (unknown provenance — an older library version wrote
196
+ * it) or the transition subscription itself dropped; either way, the
197
+ * gate must not use it and instead performs a real read next time,
198
+ * which re-establishes trust.
199
+ */
200
+ private readonly closedCache;
201
+ /** True once `subscribeTransitions` has actually succeeded; false (the closed cache is never consulted) if `localCache` wasn't set, the store lacks the capability, or the initial subscribe attempt failed. */
202
+ private localCacheActive;
203
+ private unsubscribeTransitions?;
141
204
  private readonly failures;
142
205
  private readonly lastFailureTime;
143
206
  private readonly openOperations;
@@ -148,7 +211,33 @@ export declare class CircuitBreaker {
148
211
  private readonly sampleCounts;
149
212
  private readonly monitorHandle;
150
213
  constructor(options?: CircuitBreakerOptions);
151
- /** Stops the local-mode monitor interval. Call this when the breaker is no longer needed. */
214
+ /**
215
+ * Attempts to establish the closed-state cache's push invalidation via
216
+ * `Store.subscribeTransitions`. Best-effort and one-shot: if the store
217
+ * doesn't implement it, or the initial subscribe attempt itself throws,
218
+ * `localCache` stays disabled for this breaker's lifetime (a one-time
219
+ * warning either way) — reconnects *after* a successful subscription are
220
+ * the store implementation's own responsibility (see the interface doc).
221
+ */
222
+ private setupLocalCache;
223
+ /**
224
+ * `null` means the subscription's connection dropped or errored: every
225
+ * cached entry may now be missing events, so all of them are marked
226
+ * untrusted until their next authoritative (real-read) refresh — the
227
+ * fallback poll in `monitor()` is what eventually forces that refresh if
228
+ * no call happens to trigger one first.
229
+ */
230
+ private handleTransitionEvent;
231
+ /**
232
+ * Refreshes the closed-state cache entry for `operation` with a fresh
233
+ * authoritative read, whether the write came from this instance (already
234
+ * has the resulting state in hand) or a real gate read. A no-op if
235
+ * `localCache` isn't active. Never regresses a cache entry that a newer
236
+ * transition event has already superseded, since a real read can race a
237
+ * push notification and land after it.
238
+ */
239
+ private updateClosedCache;
240
+ /** Stops the local-mode monitor interval and any transition-event subscription. Call this when the breaker is no longer needed. */
152
241
  destroy(): void;
153
242
  execute<T>(operation: string, fn: () => Promise<T>): Promise<T>;
154
243
  /**
@@ -161,8 +250,12 @@ export declare class CircuitBreaker {
161
250
  private checkOperationName;
162
251
  /**
163
252
  * Synchronous, best-effort, read-only check (never claims a trial slot).
164
- * In distributed mode this cannot await the store, so it always returns
165
- * `true` use `canExecuteAsync` there.
253
+ * In distributed mode this cannot await the store, so it can only ever
254
+ * consult the local open-state cache (see `openCacheRefreshMs`): a `false`
255
+ * is a real, cached-open result, but a `true` is still only advisory —
256
+ * it means either the operation isn't cached open, or nothing has been
257
+ * cached yet — not a confirmed closed state. Use `canExecuteAsync` for a
258
+ * store-verified check.
166
259
  */
167
260
  canExecute(operation: string): boolean;
168
261
  /**
@@ -203,9 +296,27 @@ export declare class CircuitBreaker {
203
296
  * back on consecutive checks milliseconds apart.
204
297
  */
205
298
  private effectiveResetTimeout;
299
+ /**
300
+ * Records that `operation` was just observed open (or authored as open,
301
+ * e.g. by this instance's own failed trial), so the next calls can reject
302
+ * from this cache instead of reading the store. Reuses
303
+ * `effectiveResetTimeout`'s per-episode jitter cache — same `expiresAt` the
304
+ * gate's real eligibility check uses, so the cache can never make a trial
305
+ * eligible later than it already would be.
306
+ */
307
+ private cacheOpenState;
206
308
  private key;
207
309
  private trialKey;
208
310
  private stateTtlSeconds;
311
+ /**
312
+ * Same generous, non-load-bearing safety-net TTL as `stateTtlSeconds`, for
313
+ * the one call site (`reopenTrialFailureAtomic`) that must pick a TTL
314
+ * *before* knowing the resulting `openCount` the atomic script itself
315
+ * computes server-side. Sized off `maxResetTimeout` — the ceiling
316
+ * `backoffCapped` can ever reach at any `openCount` — so it's always at
317
+ * least as generous as the true per-`openCount` value would be.
318
+ */
319
+ private maxStateTtlSeconds;
209
320
  private trialTtlSeconds;
210
321
  private reportStoreError;
211
322
  private warnMissingCapability;
@@ -268,6 +379,7 @@ export declare class CircuitBreaker {
268
379
  * breaker open with no way to ever attempt another trial.
269
380
  */
270
381
  private monitor;
382
+ private reconcileClosedCache;
271
383
  private executeWithTimeout;
272
384
  }
273
385
  //# sourceMappingURL=circuit-breaker.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAChC,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAA8B,MAAM,SAAS,CAAC;AAE1F,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAE3D,MAAM,WAAW,aAAa;IAC5B,IAAI,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,GAAG,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CAC5B;AAED,YAAY,EAAE,mBAAmB,EAAE,cAAc,EAAE,CAAC;AAEpD;;;;;GAKG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC;gBAEZ,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO;CAK9C;AAED,MAAM,WAAW,qBAAqB;IACpC;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,EAAE,aAAa,GAAG,WAAW,CAAC;IACvC,wGAAwG;IACxG,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;OAIG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,mGAAmG;IACnG,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,2FAA2F;IAC3F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qIAAqI;IACrI,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,oFAAoF;IACpF,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;OAIG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IACjE;;;;;OAKG;IACH,KAAK,CAAC,EAAE,KAAK,CAAC;IACd;;;;;;;;;;;;;;OAcG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3D,MAAM,CAAC,EAAE,aAAa,CAAC;IACvB;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;CACxB;AA0BD;;;;;;;;;;;;;;GAcG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAqBtB;IAEF,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAU;IACtC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAqB;IACxD,OAAO,CAAC,iCAAiC,CAAS;IAClD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,0BAA0B,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAqC;IAC7D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgF;IAGpH,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA6B;IACtD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA6B;IAC7D,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAqB;IACpD,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAA6B;IAC5D,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA6B;IAE1D,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAiC;gBAEnD,OAAO,GAAE,qBAA0B;IA6B/C,6FAA6F;IAC7F,OAAO,IAAI,IAAI;IAIT,OAAO,CAAC,CAAC,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;IAKrE;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IAsB1B;;;;OAIG;IACH,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAatC;;;;;OAKG;IACG,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAM1D,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAetC,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAStC;;;;;;;;;OASG;YACW,yBAAyB;IAQjC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAYzD,KAAK,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAe7C,wPAAwP;IACxP,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,cAAc;IAK7C,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC;IAM/C,OAAO,CAAC,UAAU;IAWlB,OAAO,CAAC,aAAa;IAKrB;;;;;;;;;OASG;IACH,OAAO,CAAC,qBAAqB;IAiB7B,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,QAAQ;IAIhB,OAAO,CAAC,eAAe;IAUvB,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,qBAAqB;YAQf,OAAO;YASP,OAAO;IAQrB,qFAAqF;YACvE,WAAW;IAQzB,4NAA4N;YAC9M,YAAY;IAQ1B;;;;;;;;;;;;;;;;OAgBG;YACW,gBAAgB;YAahB,gBAAgB;IAK9B;;;;;OAKG;YACW,eAAe;YAsCf,kBAAkB;YAkDlB,gCAAgC;IA6C9C;;;;OAIG;YACW,wBAAwB;YAkDxB,iCAAiC;YAkBjC,YAAY;IAkC1B,gFAAgF;IAChF,OAAO,CAAC,aAAa;IAcrB,oFAAoF;IACpF,OAAO,CAAC,cAAc;IAuBtB,OAAO,CAAC,0BAA0B;IAclC,kIAAkI;IAClI,OAAO,CAAC,sBAAsB;IAkB9B,OAAO,CAAC,2BAA2B;IAUnC,OAAO,CAAC,UAAU;IAiBlB;;;;;;OAMG;IACH,OAAO,CAAC,OAAO;IAgBf,OAAO,CAAC,kBAAkB;CAiB3B"}
1
+ {"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAChC,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAA8B,eAAe,EAAE,MAAM,SAAS,CAAC;AAE3G,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAE3D,MAAM,WAAW,aAAa;IAC5B,IAAI,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,GAAG,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CAC5B;AAED,YAAY,EAAE,mBAAmB,EAAE,cAAc,EAAE,eAAe,EAAE,CAAC;AAErE;;;;;GAKG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC;gBAEZ,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO;CAK9C;AAED,MAAM,WAAW,qBAAqB;IACpC;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,EAAE,aAAa,GAAG,WAAW,CAAC;IACvC,wGAAwG;IACxG,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;OAIG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,mGAAmG;IACnG,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,2FAA2F;IAC3F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qIAAqI;IACrI,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,oFAAoF;IACpF,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;OAIG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IACjE;;;;;OAKG;IACH,KAAK,CAAC,EAAE,KAAK,CAAC;IACd;;;;;;;;;;;;;;OAcG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3D,MAAM,CAAC,EAAE,aAAa,CAAC;IACvB;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;;;;;OAQG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;;;;;;;;;;;;;;;;;;;OAsBG;IACH,UAAU,CAAC,EAAE;QAAE,gBAAgB,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;CAC5C;AAqCD;;;;;;;;;;;;;;GAcG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAuBtB;IAEF,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAU;IACtC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAqB;IACxD,OAAO,CAAC,iCAAiC,CAAS;IAClD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,0BAA0B,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAqC;IAC7D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgF;IACpH;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyG;IAEpI,yNAAyN;IACzN,OAAO,CAAC,QAAQ,CAAC,SAAS,CAA4B;IACtD;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,WAAW,CAA2G;IACvI,gNAAgN;IAChN,OAAO,CAAC,gBAAgB,CAAS;IACjC,OAAO,CAAC,sBAAsB,CAAC,CAAa;IAG5C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA6B;IACtD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA6B;IAC7D,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAqB;IACpD,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAA6B;IAC5D,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA6B;IAE1D,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAiC;gBAEnD,OAAO,GAAE,qBAA0B;IAmC/C;;;;;;;OAOG;YACW,eAAe;IAkB7B;;;;;;OAMG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,iBAAiB;IAiBzB,mIAAmI;IACnI,OAAO,IAAI,IAAI;IAKT,OAAO,CAAC,CAAC,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;IAKrE;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IAsB1B;;;;;;;;OAQG;IACH,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAiBtC;;;;;OAKG;IACG,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAM1D,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAetC,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAStC;;;;;;;;;OASG;YACW,yBAAyB;IAQjC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAYzD,KAAK,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAiB7C,wPAAwP;IACxP,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,cAAc;IAK7C,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC;IAM/C,OAAO,CAAC,UAAU;IAWlB,OAAO,CAAC,aAAa;IAKrB;;;;;;;;;OASG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,cAAc;IAatB,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,QAAQ;IAIhB,OAAO,CAAC,eAAe;IAUvB;;;;;;;OAOG;IACH,OAAO,CAAC,kBAAkB;IAI1B,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,qBAAqB;YAUf,OAAO;YASP,OAAO;IAQrB,qFAAqF;YACvE,WAAW;IAQzB,4NAA4N;YAC9M,YAAY;IAQ1B;;;;;;;;;;;;;;;;OAgBG;YACW,gBAAgB;YAahB,gBAAgB;IAyB9B;;;;;OAKG;YACW,eAAe;YAsEf,kBAAkB;YAsDlB,gCAAgC;IAkD9C;;;;OAIG;YACW,wBAAwB;YAuDxB,iCAAiC;YAyCjC,YAAY;IAkC1B,gFAAgF;IAChF,OAAO,CAAC,aAAa;IAcrB,oFAAoF;IACpF,OAAO,CAAC,cAAc;IAuBtB,OAAO,CAAC,0BAA0B;IAclC,kIAAkI;IAClI,OAAO,CAAC,sBAAsB;IAkB9B,OAAO,CAAC,2BAA2B;IAUnC,OAAO,CAAC,UAAU;IAiBlB;;;;;;OAMG;IACH,OAAO,CAAC,OAAO;YA6BD,oBAAoB;IAOlC,OAAO,CAAC,kBAAkB;CAiB3B"}