redeye-breaker 0.4.0 → 0.5.1

package/README.md

@@ -104,7 +104,7 @@ Every process pointed at the same Redis instance (and using the same operation n
 
 ### Bring your own store
 
-`RedisStore` implements a `Store` interface with two required methods and four *optional* ones:
+`RedisStore` implements a `Store` interface with two required methods and seven *optional* ones:
 
 ```ts
 export interface Store {
@@ -115,19 +115,65 @@ export interface Store {
   // Optional — implement these for the reliability guarantees below.
   // Without them, redeye falls back to best-effort semantics and logs a
   // one-time warning telling you exactly what's degraded.
-  recordFailureAtomic?(key: string, opts: { ttlSeconds: number; failureThreshold: number; now: number }): Promise<CircuitBreakerState>;
-  recordOutcomeAtomic?(key: string, opts: { success: boolean; decay: number; minimumCalls: number; errorRateThreshold: number; ttlSeconds: number; now: number }): Promise<CircuitBreakerState & { openedNow: boolean }>;
+  recordFailureAtomic?(key: string, opts: { ttlSeconds: number; failureThreshold: number; now: number; operation: string }): Promise<CircuitBreakerState>;
+  recordOutcomeAtomic?(key: string, opts: { success: boolean; decay: number; minimumCalls: number; errorRateThreshold: number; ttlSeconds: number; now: number; operation: string }): Promise<CircuitBreakerState & { openedNow: boolean }>;
   claimTrial?(key: string, ttlSeconds: number): Promise<string | null>;
   releaseTrial?(key: string, token: string): Promise<void>;
+
+  // Optional — only needed for the closed-state local cache (`localCache`, see below).
+  closeAtomic?(key: string, eventsKey: string, opts: { ttlSeconds: number; operation: string }): Promise<CircuitBreakerState>;
+  reopenTrialFailureAtomic?(key: string, eventsKey: string, opts: { ttlSeconds: number; failureThreshold: number; now: number; operation: string }): Promise<CircuitBreakerState>;
+  subscribeTransitions?(eventsKey: string, handler: (event: TransitionEvent | null) => void): Promise<() => void>;
 }
 ```
 
-- `recordFailureAtomic` (`strategy: 'consecutive'`) should increment the failure count and decide `isOpen` in one atomic round trip (a Lua script in Redis, a conditional update in DynamoDB, etc.).
-- `recordOutcomeAtomic` (`strategy: 'errorRate'`) should fold one call's outcome into the EWMA rate and decide `isOpen` in one atomic round trip — called on every closed-phase call, not just failures.
+- `recordFailureAtomic` (`strategy: 'consecutive'`) should increment the failure count and decide `isOpen` in one atomic round trip (a Lua script in Redis, a conditional update in DynamoDB, etc.). Should also bump a monotonic `version` counter and publish a transition event to `eventsKey` (see below) when — and only when — this call flips `isOpen`; a plain sub-threshold increment isn't a transition and shouldn't publish one.
+- `recordOutcomeAtomic` (`strategy: 'errorRate'`) should fold one call's outcome into the EWMA rate and decide `isOpen` in one atomic round trip — called on every closed-phase call, not just failures. Same `version`/publish rule as above, keyed off an `isOpen` flip.
 - `claimTrial` should be a conditional "create if absent" write (`SET key token NX EX ttl` in Redis, with `token` a fresh unique value per call, e.g. a UUID) — it's what makes half-open recovery exclusive to one caller instead of a free-for-all. Return the token you wrote if this call won the claim, or `null` if the key was already taken.
 - `releaseTrial` should be a compare-and-delete (`if GET key == token then DEL key` in Redis, a conditional delete elsewhere), not an unconditional delete. This is what stops a trial that outran its TTL (see [item 11](#whats-still-a-fundamental-tradeoff-not-a-bug)) from deleting a *different* instance's newer claim on the same key when it finally gets around to releasing its own now-stale one. If omitted (or if a call has no confirmed ownership token to release at all — e.g. `claimTrial` itself errored), redeye logs a one-time warning and leaves the claim for its TTL to expire naturally, rather than risk an unconditional delete that could destroy a claim it never confirmed was its own.
+- `closeAtomic` should close `key`, bumping `version` and publishing a transition, **but only if there was anything to clear** (it was open, or had accumulated sub-threshold failures) — a true no-op, no write at all, when the state was already clean. This is what lets the healthy-path write-skip (see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug)) happen atomically server-side instead of relying on the caller's own possibly-stale view.
+- `reopenTrialFailureAtomic` should reopen `key` after a failed half-open trial — `isOpen: true`, `openCount + 1`, `failures` raised to at least `failureThreshold`, `lastFailure: now`, `version + 1` — and always publish a transition (a failed trial is always a real transition, unlike `closeAtomic`).
+- `subscribeTransitions` should subscribe to a stream of transition events at `eventsKey` — **one shared stream per store/prefix, not one per operation** — and invoke `handler` with each decoded event, or with `null` whenever the underlying subscription drops or errors (callers must then distrust every locally cached entry until it's re-verified by a real read; implementations should retry their own subscription with backoff, not expect the caller to re-subscribe). Returns an unsubscribe function.
+
+Implement the two required methods against Memcached, DynamoDB, your own cache wrapper, etc., and you have a working (best-effort) distributed breaker. Add `recordFailureAtomic`/`recordOutcomeAtomic`/`claimTrial`/`releaseTrial` when that store supports a real atomic increment and a real conditional write, and you get the full reliability guarantees. Add `closeAtomic`/`reopenTrialFailureAtomic`/`subscribeTransitions` on top of that if you also want the [closed-state local cache](#local-caching-hybrid-mode) — every capability is independently optional, so skip any subset that doesn't fit your store.
+
+## Local caching (hybrid mode)
+
+Everything up to this point reads the store on every gate check (a genuine read every closed-path call; a cached, 0-read rejection once open — see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug)). `localCache` goes one step further, opt-in, only for `RedisStore` (or any store implementing `subscribeTransitions`): it caches the **closed** state locally too, invalidated push-style instead of polling, so a healthy call can skip the store read entirely, not just the write.
+
+```ts
+const breaker = new CircuitBreaker({
+  failureThreshold: 5,
+  resetTimeout: 60_000,
+  store,
+  localCache: { staleToleranceMs: 100 }, // opt-in; default staleToleranceMs is 100 once the object is present
+});
+```
+
+**What's cached, and how it's invalidated.** Per `operation`, redeye keeps `{ state, version, fetchedAt, trusted }` in memory. A gate check allows immediately, with zero store reads, when the entry is `trusted`, closed (`!state.isOpen`), and fresher than `staleToleranceMs`. Three independent mechanisms keep this from going stale silently:
 
-Implement the two required methods against Memcached, DynamoDB, your own cache wrapper, etc., and you have a working (best-effort) distributed breaker. Add the optional methods relevant to the strategy you use when that store supports a real atomic increment and a real conditional write, and you get the full reliability guarantees.
+- **Push invalidation.** `subscribeTransitions` delivers a decoded event the instant any instance's write flips `isOpen` or clears real accumulated state (never on a plain sub-threshold counter update — those aren't transitions and don't invalidate the cache, by design, since they can't affect the *closed* fast path this cache serves). A higher `version` than what's cached overwrites the entry immediately.
+- **Fallback poll.** The existing local-mode monitor interval doubles as a missed-message safety net in distributed mode: any cached entry older than 5s gets one authoritative `get` and is reconciled by `version`. In steady state (no missed messages) this is the *only* background traffic the cache generates — one `GET` per stale operation per sweep, not per call.
+- **Your own writes are free.** The state `recordFailureAtomic`/`recordOutcomeAtomic`/`closeAtomic`/`reopenTrialFailureAtomic` hand back already updates this instance's own cache — the instance that causes a transition never has a stale window on it, regardless of stream or poll timing.
+
+**The fail-open window, as a formula, not an adjective.** This is the one place in redeye that's biased toward availability over freshness on the *closed* side (everywhere else — the open-state cache, `failOpenOnStoreError: false`, etc. — is fail-closed or symmetric by default). If the breaker trips elsewhere in the fleet, a call landing on this instance's still-fresh cached-closed entry can pass through before that instance learns about it. The bound is exactly:
+
+```
+worst case:    staleToleranceMs
+typical case:  one stream delivery (usually much shorter than staleToleranceMs)
+```
+
+`staleToleranceMs` is the hard ceiling, full stop, not one term in a larger sum — the cache stops trusting itself the instant it elapses, regardless of whether the transition event ever arrives (that's what makes the fallback poll a correctness backstop and not just a nice-to-have: a dropped or delayed event can never push the window past this bound, only fail to shorten it). Delivery latency only ever makes the *actual* window tighter than the ceiling, never looser — in the common case it's one Redis stream round trip, resolving the window well before `staleToleranceMs` would have on its own. Set `staleToleranceMs` to whatever window you're comfortable a downstream outage could go un-noticed on this one path — `100`ms (the default) is a reasonable starting point for most services; push it lower if even a hundred milliseconds of continued traffic against a freshly-tripped dependency is unacceptable, at the cost of the cache being useful for a shorter fraction of the healthy path.
+
+**Compare this to the open-state cache deliberately.** `openCacheRefreshMs` never risks an extra call through — staleness there can only make a rejection *late*, not wrong, because it always falls through to a real read right at the real eligibility instant (see limitation item 1). `localCache`'s closed-side cache is the opposite shape: staleness there can make an *allow* wrong for a bounded window, in exchange for the healthy path's read too. That's the real tradeoff you're opting into, not a bug to route around — the same one Envoy's outlier detection makes by default (cache state locally with a bounded TTL, accept staleness, skip the round trip), just off by default here and explicit about the bound when you turn it on.
+
+**The version/absence rule, and mixed-fleet rolling deploys.** `CircuitBreakerState.version` is a monotonic counter that increments only on a real transition — its *absence* (not `0`) means "written by a library version that predates `localCache`, provenance unknown." A cache entry built from a version-less read is marked `trusted: false` and never used for the fast path; the very next call for that operation performs a real read instead, forever, until a version-bearing write appears. This is what makes a rolling deploy safe: instances running an old version keep working exactly as before (they don't publish transitions, and their writes lack `version`), and new instances simply never trust a versionless observation rather than guessing. Nothing about this requires deploying in any particular order.
+
+**Strategy asymmetry, stated plainly.** `consecutive` is the strategy this cache is built for: combined with the healthy-path write skip, a run of successes against a warm cache is a genuine 0-store-op healthy path. `errorRate` cannot get the same benefit — `recordOutcomeAtomic` must fold *every* outcome (success or failure) into the EWMA to keep the rate meaningful, so it floors at 1 write per call no matter what's cached. `localCache` still saves the *read* for `errorRate` the same way it does for `consecutive`; it just can't save the write, structurally, not as an oversight.
+
+Read "0-store-op" as *decoupled from request volume*, not as a literal constant: under sustained load it's at most one read per `staleToleranceMs` window (a real read resets that window's clock), plus one poll per idle operation roughly every 5s if nothing else refreshes it sooner. A short benchmark run's literal op count is partly an artifact of the run fitting inside a handful of those windows — see [BENCHMARK.md](BENCHMARK.md#reading-the-numbers).
+
+**Requires `subscribeTransitions`.** Without it (a custom `Store` that doesn't implement the capability), `localCache` is a no-op: one-time warning, and behavior is identical to not setting it at all. `RedisStore` implements it via a dedicated (`duplicate()`d) blocking connection running `XREAD BLOCK` against the shared `circuit_breaker:events` stream, reconnecting with backoff on its own if that connection drops — see the `Store` interface docs above for the reconnect contract.
 
 ## API
 
@@ -167,6 +213,7 @@ redeye can't stop you from doing this, but it can flag it: `execute`/`recordFail
 | `logger` | no-op | `{ warn(msg), log(msg) }` — plug in your own logger |
 | `maxOperations` | none | Logs a one-time warning once the breaker has seen more than this many distinct `operation` names — see the cardinality note above. Unset: no limit, no warning. |
 | `openCacheRefreshMs` | `2000` | Distributed mode only: while `operation` is known-open, reject locally without a store read, re-verifying against the store at most once per this many ms (to catch an early close by another instance's trial, or a manual `reset()`). Never delays trial eligibility — see [limitation item 1](#whats-still-a-fundamental-tradeoff-not-a-bug). `0` disables the cache (a store read on every call, the pre-cache behavior). |
+| `localCache` | none (disabled) | Distributed mode, opt-in, requires a store implementing `subscribeTransitions`: `{ staleToleranceMs?: number }` (default `100` once the object is present) — caches CLOSED state locally, invalidated push-style, so a healthy call can skip the store read too, not just the write. See [Local caching (hybrid mode)](#local-caching-hybrid-mode) for the fail-open window this trades for that. |
 
 ### Store unavailability: fail-open vs fail-closed
 
@@ -204,7 +251,7 @@ If your store *doesn't* implement the atomic method your chosen strategy needs (
 
 ### Split-brain: what's prevented, and what isn't
 
-There's no split-brain among your app instances in the classic sense, because instances never hold independent authoritative state to disagree over — every instance reads the store fresh before every single decision (`gateDistributed`) instead of trusting a locally cached opinion. The store is the only "brain"; instances are just readers/writers of it. Two mechanisms enforce agreement on the common path:
+There's no split-brain among your app instances in the classic sense, because instances never hold independent authoritative state *to write from* — every state-changing decision goes through the store, and the two local caches (`openCacheRefreshMs`, `localCache`) exist only to skip *reading* it before an already-known-safe decision, never to substitute for it. The store is the only "brain"; instances are just readers/writers of it, some of the time from a recent, bounded-staleness memory of what they last read instead of asking again. Two mechanisms enforce agreement on the common path:
 
 - **Atomic writes.** `recordFailureAtomic`/`recordOutcomeAtomic` run as one Lua script — GET, decode, increment, decide, SET — inside Redis's single-threaded execution, so two instances failing at the same instant can't both read the same count and both write the same increment.
 - **Exclusive trial claim.** `claimTrial` is a `SET ... NX`, which Redis resolves atomically, so exactly one instance ever runs the half-open recovery probe — never two instances simultaneously deciding they're the one testing recovery.
@@ -221,7 +268,8 @@ So the design doesn't eliminate split-brain so much as route around it: it pushe
 
 These are true of any distributed circuit breaker, rate limiter, or lock built on a shared store — not gaps specific to redeye:
 
-1. **Distributed mode costs Redis round trips per guarded call, not zero — except while the breaker is actually open, which is exactly when that matters most.** On the closed, healthy path, every `execute()` call does a gate read (`store.get`) before deciding whether to proceed; what happens after differs by strategy — `errorRate` writes (`recordOutcomeAtomic`) on every closed-phase call, success or failure, since the rate needs both to be meaningful, while `consecutive` only writes when the gate's own read observed accumulated state to clear (an open breaker, or `failures > 0`), making a run of healthy successes 1 read and 0 writes per call, not 1-and-1 (a measured claim about the steady state, not a guarantee — a dependency flapping near the failure threshold still writes on most calls). **Once the breaker is open, rejections stop costing a read at all**: the gate caches the open state locally (`lastFailure`, `openCount`, and the same jittered `expiresAt` the eligibility check itself uses) and rejects straight from that cache, re-verifying against the store only once per `openCacheRefreshMs` (default 2000ms) — to catch an early close by another instance's trial, or a manual `reset()` — and always falling through to a real read right at `expiresAt`, so the cache can delay a rejection's freshness but can never delay a trial's eligibility. Net effect: the request-volume-scaling cost this item used to describe applies to the closed, healthy path — the load *during an incident*, when the store or its host infrastructure may itself be degraded, drops instead of scaling with traffic. A resolving half-open trial still adds a round trip to claim it and another to release its claim — that path is unaffected, and unconditional, regardless of strategy. On a latency-sensitive path the closed-path round trip is still added tail latency — the opposite tradeoff from something like Envoy's outlier detection, which caches state locally with a short TTL and accepts bounded staleness instead of paying a store round trip per call. redeye chose per-call freshness over that on the closed path; it's a reasonable choice for many services, but measure it before putting this in front of your highest-QPS call. Local mode has none of this cost — see [Local mode vs. distributed mode](#local-mode-vs-distributed-mode-which-do-you-need) if you're not sure you need shared state at all.
+1. **Distributed mode's per-call store cost is opt-in-able down to a bound you choose, not fixed — but "zero" always means "zero, at a stated staleness price," never "zero, for free."** By default, every `execute()` call does a gate read (`store.get`) before deciding whether to proceed; what happens after differs by strategy — `errorRate` writes (`recordOutcomeAtomic`) on every closed-phase call, success or failure, since the rate needs both to be meaningful (this never goes away, at any cache setting — see below), while `consecutive` only writes when the gate's own read observed accumulated state to clear (an open breaker, or `failures > 0`), making a run of healthy successes 1 read and 0 writes per call by default, not 1-and-1. **Once the breaker is open, rejections stop costing a read at all**, unconditionally, no opt-in required: the gate caches the open state locally (`lastFailure`, `openCount`, and the same jittered `expiresAt` the eligibility check itself uses) and rejects straight from that cache, re-verifying against the store only once per `openCacheRefreshMs` (default 2000ms), and always falling through to a real read right at `expiresAt` — so this cache can delay a rejection's freshness but can never delay a trial's eligibility, and never risks letting an extra call through. That's the fail-*closed*-biased half of the story: staleness there can only make things more conservative.
+   The closed, healthy path is the other half, and it's where a real tradeoff lives: **`localCache` (opt-in) is redeye's answer to exactly the caching-with-bounded-staleness tradeoff Envoy's outlier detection makes by default** — cache state locally, accept a bounded staleness window, skip the round trip. Off by default (redeye chooses per-call freshness there until you say otherwise), and explicit about the bound when you turn it on: worst case `staleToleranceMs`, full stop — the cache stops trusting itself at that bound regardless of whether a transition event ever arrives; typical case one stream delivery, usually well under it — a formula, not an adjective — see [Local caching (hybrid mode)](#local-caching-hybrid-mode) for exactly what that means and what it doesn't cover (`errorRate`'s write floor, in particular, which `localCache` cannot remove — only `consecutive` reaches a genuine 0-store-op healthy path). A resolving half-open trial always adds a round trip to claim it and another to release its claim, at any cache setting, on either strategy — that path is never cacheable, and rightly so. Local mode has none of this cost at all — see [Local mode vs. distributed mode](#local-mode-vs-distributed-mode-which-do-you-need) if you're not sure you need shared state to begin with.
 2. **Correctness depends on trusting your store.** If Redis evicts a breaker key early under memory pressure (e.g. `maxmemory-policy allkeys-lru` with `circuit_breaker:*` competing for space with everything else), the breaker can reset earlier than intended. Mitigate by giving circuit-breaker keys their own keyspace/DB with a policy that doesn't evict them, or by monitoring `onStoreError` and Redis memory pressure directly. This is not fixable in-library — it's an operational configuration matter.
 3. **Distributed timing is sensitive to clock skew.** Backoff and jitter windows are computed by comparing each instance's local `Date.now()` against a `lastFailure` timestamp written by whichever instance recorded it. With reasonably synchronized clocks (standard NTP) this is a non-issue; on a fleet with significant clock drift, instances can disagree by that drift amount about exactly when a trial becomes eligible. This affects *timing* only — the half-open claim mechanism (`claimTrial`) still guarantees exactly one trial runs regardless of skew, so skew cannot cause a thundering herd, only a slightly early or late trial attempt.
 4. **`recordFailure`/`recordSuccess`/`canExecuteAsync` don't participate in trial claiming.** Only `execute()` claims and releases the half-open trial slot. If you build your own call flow around the manual API instead of `execute()`, you lose the single-trial guarantee and get the old "everyone retries once elapsed" behavior for that flow. Prefer `execute()`.
@@ -252,6 +300,8 @@ docker compose down
 
 It connects to `REDIS_URL` (default `redis://localhost:6379`).
 
+`npm run bench` runs `bench/index.js` against a real Redis, comparing total store round trips and p50/p99 latency across three configurations: no caching, today's defaults (`openCacheRefreshMs`), and `localCache` on top. Same `REDIS_URL`; start Redis first the same way as the integration suite. See [BENCHMARK.md](BENCHMARK.md) for a full run's results and how to read them.
+
 ## Contributing
 
 See [CONTRIBUTING.md](CONTRIBUTING.md). Changes are tracked in [CHANGELOG.md](CHANGELOG.md).

package/dist/circuit-breaker.d.ts

@@ -1,11 +1,11 @@
 import { Store } from './store';
-import { CircuitBreakerState, CircuitMetrics } from './types';
+import { CircuitBreakerState, CircuitMetrics, TransitionEvent } from './types';
 export type CircuitState = 'closed' | 'open' | 'half-open';
 export interface BreakerLogger {
     warn(message: string): void;
     log(message: string): void;
 }
-export type { CircuitBreakerState, CircuitMetrics };
+export type { CircuitBreakerState, CircuitMetrics, TransitionEvent };
 /**
  * Thrown in distributed mode when the backing store is unreachable and
  * `failOpenOnStoreError` is `false`. Distinguishable from a downstream
@@ -122,6 +122,34 @@ export interface CircuitBreakerOptions {
      * Default: 2000.
      */
     openCacheRefreshMs?: number;
+    /**
+     * Distributed mode, opt-in. Serves gate decisions for CLOSED state from a
+     * local cache while it's fresher than `staleToleranceMs`, invalidated
+     * push-style via the store's transition stream (`Store.subscribeTransitions`)
+     * instead of polling — so a healthy call can skip the store read entirely,
+     * not just the write Release 1 already skips.
+     *
+     * This only ever short-circuits an *allow*: a cache entry observed open
+     * always defers to the open-state cache (`openCacheRefreshMs`) and a real
+     * read, which remains authoritative for all rejection handling. Requires
+     * a store implementing `subscribeTransitions`; without one, this option
+     * is a no-op (one-time warning) and behavior is unchanged from not
+     * setting it at all.
+     *
+     * The worst case is a bounded fail-open window of exactly
+     * `staleToleranceMs`, full stop: if the breaker trips elsewhere in the
+     * fleet, a call landing on a cached entry can be let through until that
+     * entry ages out, since it stops trusting itself at `staleToleranceMs`
+     * regardless of whether the transition event ever arrives. Delivery
+     * latency only ever makes the *actual* window tighter than that ceiling
+     * (typically one stream round trip, well under `staleToleranceMs`), never
+     * looser. See the README's local caching section for the full
+     * fail-open/fail-closed asymmetry this creates versus the
+     * (fail-closed-biased) open-state cache.
+     */
+    localCache?: {
+        staleToleranceMs?: number;
+    };
 }
 /**
  * A circuit breaker with an optional distributed mode.
@@ -159,6 +187,22 @@ export declare class CircuitBreaker {
      * an early close by another instance or a manual `reset()`.
      */
     private readonly cachedOpen;
+    /** Fixed, unprefixed key for the shared transition-event stream — one per store/prefix, not one per operation. `RedisStore` applies its own `keyPrefix` the same way it does for every other key this class hands it. */
+    private readonly eventsKey;
+    /**
+     * Distributed mode only, `localCache` opt-in: a local, push-invalidated
+     * record of the last known state for `operation`, used to serve gate
+     * decisions for CLOSED state with zero store reads while fresh and
+     * trusted. `trusted: false` means the last write we saw for this entry
+     * had no `version` (unknown provenance — an older library version wrote
+     * it) or the transition subscription itself dropped; either way, the
+     * gate must not use it and instead performs a real read next time,
+     * which re-establishes trust.
+     */
+    private readonly closedCache;
+    /** True once `subscribeTransitions` has actually succeeded; false (the closed cache is never consulted) if `localCache` wasn't set, the store lacks the capability, or the initial subscribe attempt failed. */
+    private localCacheActive;
+    private unsubscribeTransitions?;
     private readonly failures;
     private readonly lastFailureTime;
     private readonly openOperations;
@@ -169,7 +213,63 @@ export declare class CircuitBreaker {
     private readonly sampleCounts;
     private readonly monitorHandle;
     constructor(options?: CircuitBreakerOptions);
-    /** Stops the local-mode monitor interval. Call this when the breaker is no longer needed. */
+    /**
+     * Attempts to establish the closed-state cache's push invalidation via
+     * `Store.subscribeTransitions`. Best-effort and one-shot: if the store
+     * doesn't implement it, or the initial subscribe attempt itself throws,
+     * `localCache` stays disabled for this breaker's lifetime (a one-time
+     * warning either way) — reconnects *after* a successful subscription are
+     * the store implementation's own responsibility (see the interface doc).
+     */
+    private setupLocalCache;
+    /**
+     * `null` means the subscription's connection dropped or errored: every
+     * cached entry may now be missing events, so all of them are marked
+     * untrusted until their next authoritative (real-read) refresh — the
+     * fallback poll in `monitor()` is what eventually forces that refresh if
+     * no call happens to trigger one first.
+     *
+     * A version at or below what's cached is deliberately *not* dropped as a
+     * plain stale/duplicate delivery: a key's version sequence is only
+     * monotonic within that key's lifetime, and keys legitimately die (the
+     * state TTL expiring after a long idle period, or another instance's
+     * `reset()`, which DELs with no published event at all). A reborn key
+     * restarts at a low version, which is indistinguishable from a stale
+     * delivery by version alone — so instead of guessing, this triggers a
+     * real read to resolve it. Reads always win (see `updateClosedCache`),
+     * so this self-corrects regardless of which case it actually was.
+     */
+    private handleTransitionEvent;
+    /**
+     * Refreshes the closed-state cache entry for `operation` with a fresh
+     * authoritative read, whether the write came from this instance (already
+     * has the resulting state in hand) or a real gate read. A no-op if
+     * `localCache` isn't active. Always overwrites *by version* — comparing a
+     * read's version against a cached one can't distinguish "this read is
+     * stale" from "the key was reborn with a fresh, lower version," so it
+     * doesn't try to (see `handleTransitionEvent`) — but an authoritative
+     * read can still be stale *in time*: it can have been in flight when a
+     * newer push event already installed a fresher entry, and resolve after.
+     * `readStartedAt` (captured by the caller before issuing whatever async
+     * call produced `state`) guards exactly that: if the cached entry is
+     * strictly newer than that, something demonstrably landed while this
+     * read was in flight, and this write is dropped instead of regressing
+     * the cache in time. An entry marked untrusted by a dropped subscription
+     * keeps its old `fetchedAt`, so a read started after the drop still
+     * overwrites it and re-establishes trust — this guard only ever blocks a
+     * write that's genuinely older than what's already cached.
+     *
+     * Strictly-greater-than, not greater-or-equal: `Date.now()` is
+     * millisecond-resolution, and a reconcile triggered synchronously off
+     * the back of another update to the same operation (e.g. a resolved
+     * event's reconcile read, or two updates from the same fast test/request
+     * path) can legitimately share a millisecond with `existing.fetchedAt`
+     * without either being stale relative to the other — treating a tie as
+     * "blocked" would reject correct, newer writes as often as it catches
+     * the race it's meant to guard against.
+     */
+    private updateClosedCache;
+    /** Stops the local-mode monitor interval and any transition-event subscription. Call this when the breaker is no longer needed. */
     destroy(): void;
     execute<T>(operation: string, fn: () => Promise<T>): Promise<T>;
     /**
@@ -240,6 +340,15 @@ export declare class CircuitBreaker {
     private key;
     private trialKey;
     private stateTtlSeconds;
+    /**
+     * Same generous, non-load-bearing safety-net TTL as `stateTtlSeconds`, for
+     * the one call site (`reopenTrialFailureAtomic`) that must pick a TTL
+     * *before* knowing the resulting `openCount` the atomic script itself
+     * computes server-side. Sized off `maxResetTimeout` — the ceiling
+     * `backoffCapped` can ever reach at any `openCount` — so it's always at
+     * least as generous as the true per-`openCount` value would be.
+     */
+    private maxStateTtlSeconds;
     private trialTtlSeconds;
     private reportStoreError;
     private warnMissingCapability;
@@ -302,6 +411,20 @@ export declare class CircuitBreaker {
      * breaker open with no way to ever attempt another trial.
      */
     private monitor;
+    /**
+     * Deliberately doesn't use `safeGet`: that collapses a store error into
+     * the same `null` a genuinely-absent key produces, and `updateClosedCache`
+     * treats `null` as "never written, unambiguously clean" -- so a store
+     * error reaching it would install a trusted, closed entry from a read
+     * that never actually happened. Under `failOpenOnStoreError: false` that
+     * would be a real contract violation: the gate would start serving
+     * allows from this cache with zero store contact instead of throwing
+     * `StoreUnavailableError` as the caller explicitly asked for. A failed
+     * read here instead marks any existing entry untrusted (never installs
+     * one) and leaves its `fetchedAt` alone, so the next monitor tick retries
+     * the reconcile -- the behavior wanted during an outage.
+     */
+    private reconcileClosedCache;
     private executeWithTimeout;
 }
 //# sourceMappingURL=circuit-breaker.d.ts.map

package/dist/circuit-breaker.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAChC,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAA8B,MAAM,SAAS,CAAC;AAE1F,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAE3D,MAAM,WAAW,aAAa;IAC5B,IAAI,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,GAAG,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CAC5B;AAED,YAAY,EAAE,mBAAmB,EAAE,cAAc,EAAE,CAAC;AAEpD;;;;;GAKG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC;gBAEZ,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO;CAK9C;AAED,MAAM,WAAW,qBAAqB;IACpC;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,EAAE,aAAa,GAAG,WAAW,CAAC;IACvC,wGAAwG;IACxG,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;OAIG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,mGAAmG;IACnG,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,2FAA2F;IAC3F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qIAAqI;IACrI,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,oFAAoF;IACpF,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;OAIG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IACjE;;;;;OAKG;IACH,KAAK,CAAC,EAAE,KAAK,CAAC;IACd;;;;;;;;;;;;;;OAcG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3D,MAAM,CAAC,EAAE,aAAa,CAAC;IACvB;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;;;;;OAQG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;CAC7B;AA0BD;;;;;;;;;;;;;;GAcG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAqBtB;IAEF,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAU;IACtC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAqB;IACxD,OAAO,CAAC,iCAAiC,CAAS;IAClD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,0BAA0B,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAqC;IAC7D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgF;IACpH;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyG;IAGpI,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA6B;IACtD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA6B;IAC7D,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAqB;IACpD,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAA6B;IAC5D,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA6B;IAE1D,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAiC;gBAEnD,OAAO,GAAE,qBAA0B;IA8B/C,6FAA6F;IAC7F,OAAO,IAAI,IAAI;IAIT,OAAO,CAAC,CAAC,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;IAKrE;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IAsB1B;;;;;;;;OAQG;IACH,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAiBtC;;;;;OAKG;IACG,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAM1D,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAetC,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAStC;;;;;;;;;OASG;YACW,yBAAyB;IAQjC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAYzD,KAAK,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAgB7C,wPAAwP;IACxP,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,cAAc;IAK7C,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC;IAM/C,OAAO,CAAC,UAAU;IAWlB,OAAO,CAAC,aAAa;IAKrB;;;;;;;;;OASG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,cAAc;IAatB,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,QAAQ;IAIhB,OAAO,CAAC,eAAe;IAUvB,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,qBAAqB;YAQf,OAAO;YASP,OAAO;IAQrB,qFAAqF;YACvE,WAAW;IAQzB,4NAA4N;YAC9M,YAAY;IAQ1B;;;;;;;;;;;;;;;;OAgBG;YACW,gBAAgB;YAahB,gBAAgB;IAM9B;;;;;OAKG;YACW,eAAe;YA0Df,kBAAkB;YAkDlB,gCAAgC;IA+C9C;;;;OAIG;YACW,wBAAwB;YAoDxB,iCAAiC;YAmBjC,YAAY;IAkC1B,gFAAgF;IAChF,OAAO,CAAC,aAAa;IAcrB,oFAAoF;IACpF,OAAO,CAAC,cAAc;IAuBtB,OAAO,CAAC,0BAA0B;IAclC,kIAAkI;IAClI,OAAO,CAAC,sBAAsB;IAkB9B,OAAO,CAAC,2BAA2B;IAUnC,OAAO,CAAC,UAAU;IAiBlB;;;;;;OAMG;IACH,OAAO,CAAC,OAAO;IAgBf,OAAO,CAAC,kBAAkB;CAiB3B"}
+{"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAChC,OAAO,EAAE,mBAAmB,EAAE,cAAc,EAA8B,eAAe,EAAE,MAAM,SAAS,CAAC;AAE3G,MAAM,MAAM,YAAY,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAE3D,MAAM,WAAW,aAAa;IAC5B,IAAI,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,GAAG,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;CAC5B;AAED,YAAY,EAAE,mBAAmB,EAAE,cAAc,EAAE,eAAe,EAAE,CAAC;AAErE;;;;;GAKG;AACH,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC;gBAEZ,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,OAAO;CAK9C;AAED,MAAM,WAAW,qBAAqB;IACpC;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,EAAE,aAAa,GAAG,WAAW,CAAC;IACvC,wGAAwG;IACxG,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;OAIG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,mGAAmG;IACnG,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,2FAA2F;IAC3F,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,qIAAqI;IACrI,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB;;;;;;OAMG;IACH,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,oFAAoF;IACpF,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB;;;;OAIG;IACH,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IACjE;;;;;OAKG;IACH,KAAK,CAAC,EAAE,KAAK,CAAC;IACd;;;;;;;;;;;;;;OAcG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAC/B,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,KAAK,IAAI,CAAC;IAC3D,MAAM,CAAC,EAAE,aAAa,CAAC;IACvB;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;;;;;OAQG;IACH,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B;;;;;;;;;;;;;;;;;;;;;;;;OAwBG;IACH,UAAU,CAAC,EAAE;QAAE,gBAAgB,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;CAC5C;AAqCD;;;;;;;;;;;;;;GAcG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,QAAQ,CAAC,OAAO,CAuBtB;IAEF,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAU;IACtC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAqB;IACxD,OAAO,CAAC,iCAAiC,CAAS;IAClD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,mBAAmB,CAAS;IACpC,OAAO,CAAC,0BAA0B,CAAS;IAC3C,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAqC;IAC7D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgF;IACpH;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAyG;IAEpI,yNAAyN;IACzN,OAAO,CAAC,QAAQ,CAAC,SAAS,CAA4B;IACtD;;;;;;;;;OASG;IACH,OAAO,CAAC,QAAQ,CAAC,WAAW,CAA2G;IACvI,gNAAgN;IAChN,OAAO,CAAC,gBAAgB,CAAS;IACjC,OAAO,CAAC,sBAAsB,CAAC,CAAa;IAG5C,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAA6B;IACtD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA6B;IAC7D,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAqB;IACpD,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAqB;IACrD,OAAO,CAAC,QAAQ,CAAC,cAAc,CAA6B;IAC5D,OAAO,CAAC,QAAQ,CAAC,UAAU,CAA6B;IACxD,OAAO,CAAC,QAAQ,CAAC,YAAY,CAA6B;IAE1D,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAiC;gBAEnD,OAAO,GAAE,qBAA0B;IAmC/C;;;;;;;OAOG;YACW,eAAe;IAkB7B;;;;;;;;;;;;;;;;OAgBG;IACH,OAAO,CAAC,qBAAqB;IAkD7B;;;;;;;;;;;;;;;;;;;;;;;;;;;OA2BG;IACH,OAAO,CAAC,iBAAiB;IAgBzB,mIAAmI;IACnI,OAAO,IAAI,IAAI;IAKT,OAAO,CAAC,CAAC,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;IAKrE;;;;;;OAMG;IACH,OAAO,CAAC,kBAAkB;IAsB1B;;;;;;;;OAQG;IACH,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO;IAiBtC;;;;;OAKG;IACG,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAM1D,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAetC,aAAa,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI;IAStC;;;;;;;;;OASG;YACW,yBAAyB;IAQjC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,mBAAmB,CAAC;IAYzD,KAAK,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAiB7C,wPAAwP;IACxP,UAAU,CAAC,SAAS,EAAE,MAAM,GAAG,cAAc;IAK7C,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC;IAM/C,OAAO,CAAC,UAAU;IAWlB,OAAO,CAAC,aAAa;IAKrB;;;;;;;;;OASG;IACH,OAAO,CAAC,qBAAqB;IAe7B;;;;;;;OAOG;IACH,OAAO,CAAC,cAAc;IAatB,OAAO,CAAC,GAAG;IAIX,OAAO,CAAC,QAAQ;IAIhB,OAAO,CAAC,eAAe;IAUvB;;;;;;;OAOG;IACH,OAAO,CAAC,kBAAkB;IAI1B,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,gBAAgB;IAMxB,OAAO,CAAC,qBAAqB;YAUf,OAAO;YASP,OAAO;IAQrB,qFAAqF;YACvE,WAAW;IAQzB,4NAA4N;YAC9M,YAAY;IAQ1B;;;;;;;;;;;;;;;;OAgBG;YACW,gBAAgB;YAahB,gBAAgB;IAkC9B;;;;;OAKG;YACW,eAAe;YAuEf,kBAAkB;YAsDlB,gCAAgC;IAoD9C;;;;OAIG;YACW,wBAAwB;YAyDxB,iCAAiC;YA2CjC,YAAY;IAkC1B,gFAAgF;IAChF,OAAO,CAAC,aAAa;IAcrB,oFAAoF;IACpF,OAAO,CAAC,cAAc;IAuBtB,OAAO,CAAC,0BAA0B;IAclC,kIAAkI;IAClI,OAAO,CAAC,sBAAsB;IAkB9B,OAAO,CAAC,2BAA2B;IAUnC,OAAO,CAAC,UAAU;IAiBlB;;;;;;OAMG;IACH,OAAO,CAAC,OAAO;IA6Bf;;;;;;;;;;;;OAYG;YACW,oBAAoB;IAgBlC,OAAO,CAAC,kBAAkB;CAiB3B"}