@zakkster/lite-pick 0.9.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +194 -0
- package/GUIDE.md +105 -0
- package/Pick.d.ts +85 -28
- package/Pick.js +364 -62
- package/Pool.d.ts +63 -17
- package/Pool.js +275 -62
- package/README.md +73 -36
- package/RECIPES.md +132 -36
- package/llms.txt +166 -75
- package/package.json +22 -5
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **
|
|
3
|
+
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v1.0.0 ships the complete ten-strategy roster -- `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, the latency-aware `PeakEwmaBalancer`, the sticky/affinity `ConsistentHashBalancer` (a Maglev lookup table), the hotspot-protecting `BoundedLoadBalancer` (consistent hashing with bounded loads -- sticky routing + an occupancy cap that overflows a hot backend to its neighbours), and `WeightedRandomBalancer` (O(1) Vose alias-table sampling with rejection-sampling eligibility)** -- on the substrate seams (`VERSION`, `PICK_NONE`, a deterministic `Prng`, and `BalancerBase`'s shared read-only eligibility view), plus a **`@zakkster/lite-pick/pool`** subpath: the async dispatch/settle counter layer with distinct-endpoint failover and a duck-typed query-cache fetcher. Not sure which strategy? See **[GUIDE.md](./GUIDE.md)**.
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/@zakkster/lite-pick)
|
|
6
6
|
[](https://github.com/sponsors/PeshoVurtoleta)
|
|
@@ -21,7 +21,7 @@ The npm landscape has old algorithm libraries (`load-balancers`, `loadbalance`,
|
|
|
21
21
|
- **Two pieces of evidence, both shipped.** A **0 B/op** witness on the pick path (no object, closure, string, or array created per pick), and a measured **balance-quality anchor** -- peak-to-average load within the strategy's theoretical ceiling (for P2C, the Azar-Broder-Karlin-Upfal `ln ln n / ln 2` bound) and strictly better than a random foil.
|
|
22
22
|
- **A pure selector, not a proxy.** It **consumes** health and circuit state; it never owns them. Health is a shared read-only bitmap written by [`@zakkster/lite-di-health`](https://www.npmjs.com/package/@zakkster/lite-di-health); circuit state comes from [`@zakkster/lite-statechart`](https://www.npmjs.com/package/@zakkster/lite-statechart); load counters are caller-owned typed arrays. `pick()` only reads.
|
|
23
23
|
|
|
24
|
-
> **Status:
|
|
24
|
+
> **Status: v1.0.1 -- the audit bug-fix release (see CHANGELOG and decisions/0013) on top of M10 (v1.0.0), the roster-complete release.** Ships the substrate seams **plus all ten strategies: `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, the latency-aware `PeakEwmaBalancer`, the sticky/affinity `ConsistentHashBalancer` (a Maglev table), the hotspot-protecting `BoundedLoadBalancer` (consistent hashing with bounded loads), and `WeightedRandomBalancer` (O(1) Vose alias-table sampling with rejection-sampling eligibility)**, the **`@zakkster/lite-pick/pool`** request layer (with opt-in latency-feedback, occupancy-feedback, and keyed-routing hooks), the **benchmark suite** -- the balance anchor + GC blast-radius headlines, a seeded/version-stamped `results.json`, a `bench:verify` drift check with teeth, and the vs-AWS positioning (see *Evidence* below) -- and the **[GUIDE.md](./GUIDE.md)** strategy-selection capstone. Every strategy is gated: `pick()` **allocates 0 B/op** (PerfGate scavenge counting) and **retains 0 B/op** (torture), RoundRobin **perfectly fair** with **zero dead picks** vs the naive `i++ % n` foil, SmoothWRR **exactly weighted** and **smooth**, **P2C proves the `ln ln n` balance ceiling** (peak-to-mean gap ~2 vs a random foil's ~21 at n=1024), **LeastConn is greedy-perfect** (max-minus-min load <= 1), **SED tracks weight within 1%**, **PeakEWMA steers around a 10x-slow node** (it takes <= 25% of P2C's share for it and cuts service p99), **ConsistentHash remaps only ~1.6% of keys on a scale event** (vs ~98% for naive modulo), **BoundedLoad tames a hotspot plain consistent hashing can't** (under a skewed key stream ConsistentHash spikes a hot backend to ~13x the mean occupancy while BoundedLoad's `(1+eps)` cap holds it near the mean by overflowing to neighbours), and **WeightedRandom holds every node's share within 2% of its weight** while its O(1) alias sample beats an O(n) cumsum foil by >=3x ops/ms at n=4096 -- all held under a **seeded invariant fuzzer** (`test/fuzz.mjs`) that checks state-synchronisation after *every* op. Roster complete **for now, not closed** (AZ-aware routing, hedging, subsetting are post-1.0). See [ROADMAP.md](./ROADMAP.md), and [decisions/](./decisions) for the ownership boundary (ADR 0001), anti-flapping (ADR 0002), the RoundRobin (0003), SmoothWRR (0004), P2C (0005), LeastConn-family (0006), pool-adapter (0007), benchmark-suite (0008), PeakEWMA (0009), ConsistentHash (0010), BoundedLoad/CHBL (0011), and WeightedRandom (0012) design forks.
|
|
25
25
|
|
|
26
26
|
```bash
|
|
27
27
|
npm install @zakkster/lite-pick
|
|
@@ -32,11 +32,14 @@ npm install @zakkster/lite-pick
|
|
|
32
32
|
```js
|
|
33
33
|
import { RoundRobinBalancer, PICK_NONE } from '@zakkster/lite-pick';
|
|
34
34
|
|
|
35
|
-
// A pool of 4 endpoints. The eligibility view is
|
|
36
|
-
// lite-di-health
|
|
35
|
+
// A pool of 4 endpoints. The eligibility view is read-only to pick(); at runtime it is
|
|
36
|
+
// flipped only through setEligible (lite-di-health / a breaker drives that call).
|
|
37
37
|
const eligible = Uint8Array.from([1, 1, 0, 1]); // endpoint 2 is down
|
|
38
38
|
|
|
39
39
|
const rr = new RoundRobinBalancer(4, eligible);
|
|
40
|
+
// After construction, flip nodes via rr.setEligible(i, up) -- the only supported writer
|
|
41
|
+
// (a direct eligible[i] = 0 at runtime desyncs the cached live count). Each balancer
|
|
42
|
+
// gets its own eligibility array.
|
|
40
43
|
|
|
41
44
|
rr.pick(); // -> 0
|
|
42
45
|
rr.pick(); // -> 1
|
|
@@ -90,7 +93,7 @@ const p2c = new P2cBalancer(4, eligible, inflight);
|
|
|
90
93
|
|
|
91
94
|
const i = p2c.pick(); // the lower-loaded of two random eligibles
|
|
92
95
|
inflight[i]++; // you increment on dispatch...
|
|
93
|
-
// ...and inflight[i]-- when the request settles (
|
|
96
|
+
// ...and inflight[i]-- when the request settles (or use @zakkster/lite-pick/pool, which does this)
|
|
94
97
|
```
|
|
95
98
|
|
|
96
99
|
The proof (from `test/balance.mjs`, the library's analytical anchor):
|
|
@@ -148,7 +151,7 @@ import { PeakEwmaBalancer } from '@zakkster/lite-pick';
|
|
|
148
151
|
|
|
149
152
|
const eligible = Uint8Array.from([1, 1, 1, 1]);
|
|
150
153
|
const inflight = new Uint32Array(4); // YOU own this; read live by pick()
|
|
151
|
-
const TAU_NS = 30e6; // EWMA
|
|
154
|
+
const TAU_NS = 30e6; // EWMA TIME CONSTANT: 30ms of latency memory (half-life = tau x ln2)
|
|
152
155
|
|
|
153
156
|
// `now` and rtt samples are CALLER-supplied nanoseconds -- deterministic, testable, zero-GC.
|
|
154
157
|
const pe = new PeakEwmaBalancer(4, eligible, inflight, TAU_NS);
|
|
@@ -161,10 +164,11 @@ inflight[i]--; // settle
|
|
|
161
164
|
pe.recordRtt(i, perfNs() - now, perfNs()); // feed the observed rtt back (the warm path)
|
|
162
165
|
```
|
|
163
166
|
|
|
164
|
-
- **Decay-on-read.** `pick()` *never writes* -- it applies exponential decay when it reads (`ewmaAt(i, now) = _ewma[i] x exp(-(now - _stamp[i]) / tau)`), so the hot path is a pure read and allocates
|
|
165
|
-
- **The peak rule.** `recordRtt` *snaps the cost up* to a larger sample instantly (a spike is felt on the next pick) and *decays it down* over `~tau`.
|
|
166
|
-
- **
|
|
167
|
-
- **`now` must be finite.** `now` (for `pick(now)` / `recordRtt`) and `sampleNs` must be finite numbers. `recordRtt` throws on a non-finite argument; `pick(now)` never throws (the fail-closed contract), so a non-finite `now` yields P2C-random selection rather than an error.
|
|
167
|
+
- **Decay-on-read.** `pick()` *never writes* -- it applies exponential decay when it reads (`ewmaAt(i, now) = _ewma[i] x exp(-max(now - _stamp[i], 0) / tau)`, `dt` clamped `>= 0` so a non-monotonic clock never inflates the estimate), so the hot path is a pure read and allocates 0 B/op.
|
|
168
|
+
- **The peak rule.** `recordRtt` *snaps the cost up* to a larger sample instantly (a spike is felt on the next pick) and *decays it down* over `~tau`. `tau` is the EWMA **time constant** (half-life = `tau x ln2`); it **is** the anti-flap smoothing -- no extra dwell ([ADR 0002](./decisions/0002-anti-flapping.md)).
|
|
169
|
+
- **Cold-start and hung-node pricing (1.0.1).** `inflight` is your live-read `Uint32Array`; the EWMA state (`_ewma` / `_stamp`) is **balancer-owned** and written *only* by `recordRtt`. Cost per candidate is three cases: an unsampled node costs **0 while idle** (graceful least-connections; it holds one probe request at a time until its first sample) and the pool's **lifetime mean sampled rtt once busy** (so a cold-but-busy node is not mistaken for a 1.0 ns node); a sampled node costs `(inflight + 1) x max(decayedEWMA, dt-while-busy)`, so a hung node (`dt` grows, no completion) gets **more** expensive over time, not less. Never `NaN`. A node that never records a sample (fails fast, caller records nothing) keeps winning while idle -- record failures too (the `/pool` layer does, see below). CAVEAT: an idle-then-busy node is priced by time-since-last-response until that response completes (an exact busy-since stamp is a 1.1.0 item).
|
|
170
|
+
- **`now` must be finite.** `now` (for `pick(now)` / `recordRtt`) and `sampleNs` must be finite numbers. `recordRtt` throws on a non-finite argument (a non-integer or out-of-range index throws `RangeError`); `pick(now)` never throws (the fail-closed contract), so a non-finite `now` yields P2C-random selection rather than an error.
|
|
171
|
+
- **KNOWN LIMITATION (1.0.1; buffer-based API planned for 1.1.0).** `pick(now)` / `recordRtt(..., now)` take `now` as a plain number argument. When the call is not inlined, V8 boxes a non-small-integer value into a ~16 B transient `HeapNumber` -- so a realistic nanosecond clock makes these ~16 B/op (transient, does not retain, does not force a major GC). Integer arguments within V8's small-integer range are 0 B/op; that range is build-dependent (below 2^31 on stock 64-bit Node, below 2^30 on pointer-compressed builds such as Chrome/Electron), and a value produced by `%` or division can box even when its value is a small integer. Buffer-based variants that keep `now` in a `Float64Array` slot are planned for 1.1.0.
|
|
168
172
|
|
|
169
173
|
The proof (from `test/balance.mjs`, a closed-loop queue sim with one node at 10x service time):
|
|
170
174
|
|
|
@@ -206,7 +210,7 @@ function fnv1a(s) { let h = 0x811c9dc5; for (let k = 0; k < s.length; k++) { h ^
|
|
|
206
210
|
|
|
207
211
|
## BoundedLoad -- consistent hashing with bounded loads (v0.9.0)
|
|
208
212
|
|
|
209
|
-
Plain consistent hashing is sticky and minimally-disruptive, but it has one failure mode: a **hot key**. If a handful of keys carry most of the traffic, consistent hashing pins each one's *entire* load on its one hashed backend -- an unbounded **hotspot**. `BoundedLoadBalancer` is [`ConsistentHashBalancer`](#consistenthash--sticky--cache-affinity-routing-v080) (the Maglev table) **plus a per-backend occupancy cap** `cap = (1 + eps) x
|
|
213
|
+
Plain consistent hashing is sticky and minimally-disruptive, but it has one failure mode: a **hot key**. If a handful of keys carry most of the traffic, consistent hashing pins each one's *entire* load on its one hashed backend -- an unbounded **hotspot**. `BoundedLoadBalancer` is [`ConsistentHashBalancer`](#consistenthash--sticky--cache-affinity-routing-v080) (the Maglev table) **plus a per-backend occupancy cap** `cap = ceil((1 + eps) x (total + 1) / live)` -- the load-bearing part is the `+ 1` that counts the **incoming** request (the Mirrokni-Thorup-Zadimoghaddam per-bin capacity), so the cap is always `>= 1`: a key sticks to its hashed home **unless** that backend is over cap, in which case the request **overflows** along the same bounded probe to the next eligible, under-cap backend (Mirrokni et al. *Consistent Hashing with Bounded Loads*, Google Research; Vimeo's `eps = 0.25` -- [ADR 0011](./decisions/0011-boundedload.md)). You keep stickiness + minimal disruption **and** gain the hotspot protection consistent hashing lacks. (Note: at small `eps` a second concurrent request for the same key still overflows the home until `(1+eps)(total+1)/live > 1`; a larger `eps` buys more low-load stickiness. HAProxy's `hash-balance-factor` shares the `+1` but splits one global slot budget by weight, which is stricter -- not the same definition.)
|
|
210
214
|
|
|
211
215
|
```js
|
|
212
216
|
import { BoundedLoadBalancer } from '@zakkster/lite-pick';
|
|
@@ -225,7 +229,7 @@ inflight[i]--; bl.note(i, -1); // settle: net-zero on both
|
|
|
225
229
|
```
|
|
226
230
|
|
|
227
231
|
- **Sticky + overflow.** `pick(keyHash)` maps the integer key to its Maglev home; if that backend is under cap it wins (the common, sticky path). If it is over cap, the request overflows along the bounded probe to the first eligible, under-cap backend. If nothing in the window is under cap, it falls back to the first eligible seen -- **sticky wins; the cap is a soft preference, never a dead pick**. When `_total === 0` the cap is skipped entirely, so it behaves as pure `ConsistentHashBalancer`.
|
|
228
|
-
- **`note()`
|
|
232
|
+
- **`note()` maintains the running sum.** BoundedLoad **owns** a running occupancy sum `_total` and keeps it O(1)-current through `note(i, +1)` on dispatch / `note(i, -1)` on settle -- so the cap's mean never needs a scan; `inflight` is your live-read per-backend occupancy. **Contract:** update `inflight[i]` **and** call `note(i, +/-1)` in **lockstep** (or drive it through the `/pool` adapter, which does both). `note` maintains `_total`; it does **not** write `inflight`. A direct `inflight` write without the matching `note` desyncs `_total` -- UB, the same asymmetry `SmoothWRRBalancer` has for its weights. `note()` clamps `_total` at 0, validates its index (`RangeError` on a non-integer / out-of-range), and `totalInflight` exposes the sum.
|
|
229
233
|
- **Inherits the Maglev table.** It extends `ConsistentHashBalancer`, so `setWeight(i, w)` / `rebuild()` / `tableSize` and the whole weighted-Maglev build + bounded-probe walk are reused verbatim; the `weights` / `m` / `seed` constructor args are the same. `pick()` and `note()` are both **0 B/op**, `O(1)`. `PICK_NONE` only when no eligible backend is reachable in the probe window -- never merely because backends are over cap.
|
|
230
234
|
- **Why not "P2C with a cap"?** A note on the design (the honest one): power-of-two-choices over in-flight *plus* a `(1+eps) x mean` cap is **byte-identical to plain P2C** -- an under-cap draw always has lower in-flight than an over-cap one, so "prefer under-cap" and "lower-of-two" pick the same node. The cap is a no-op there. It is only *load-bearing* when the primary choice is fixed by something other than load -- a **hash**. That is CHBL, and it is why BoundedLoad is built on consistent hashing ([ADR 0011](./decisions/0011-boundedload.md)).
|
|
231
235
|
|
|
@@ -233,26 +237,58 @@ The proof (from `test/balance.mjs`, a Zipfian-skewed key stream over 64 backends
|
|
|
233
237
|
|
|
234
238
|
| lane | mean occupancy | max backend occupancy |
|
|
235
239
|
|---|---|---|
|
|
236
|
-
| **BoundedLoad (CHBL)** | 10 | **13** (cap =
|
|
240
|
+
| **BoundedLoad (CHBL)** | 10 | **13** (cap = ceil((1+eps) x (total+1)/live) ~= 13 -- overflow holds it near the mean) |
|
|
237
241
|
| ConsistentHash (no cap) | 10 | **129** (~13x -- the hotspot) |
|
|
238
242
|
|
|
239
243
|
BoundedLoad caps the hot backend near `(1 + eps) x mean` while plain consistent hashing lets it run away, and both reroute only **~1.6%** of keys on a scale event (`test/balance.mjs`). See [`ConsistentHashBalancer`](#consistenthash--sticky--cache-affinity-routing-v080) above for the integer-key contract and the FNV-1a helper.
|
|
240
244
|
|
|
245
|
+
## WeightedRandom -- O(1) alias-table weighted selection (v1.0.0)
|
|
246
|
+
|
|
247
|
+
The weighted strategy for **very large pools**. Where `SmoothWRRBalancer` is deterministic and smooth but scans O(cap) per pick and owns per-endpoint accumulator state, `WeightedRandomBalancer` is a **stateless O(1) sample**: one draw from a precomputed **Vose/Walker alias table** (one column draw + one probability compare) returns an endpoint proportional to its weight. It converges to the weight ratios by the law of large numbers -- trading SmoothWRR's low-variance smoothness for sampling variance.
|
|
248
|
+
|
|
249
|
+
```js
|
|
250
|
+
import { WeightedRandomBalancer, PICK_NONE } from '@zakkster/lite-pick';
|
|
251
|
+
|
|
252
|
+
const eligible = Uint8Array.from([1, 1, 1, 1]);
|
|
253
|
+
const weights = Uint32Array.from([1, 2, 3, 10]); // YOU own this; endpoint 3 gets ~10/16 of traffic
|
|
254
|
+
const wr = new WeightedRandomBalancer(4, eligible, weights);
|
|
255
|
+
|
|
256
|
+
wr.pick(); // -> a weighted-random eligible index (mostly 3, sometimes 0/1/2)
|
|
257
|
+
|
|
258
|
+
// Reweight is COLD (rebuilds the alias table); the balancer is the sole writer of its table.
|
|
259
|
+
wr.setWeight(3, 1); // now roughly uniform
|
|
260
|
+
wr.setEligible(1, false); // an eligibility flap is FREE -- it never rebuilds the table (anti-flap)
|
|
261
|
+
wr.pick(); // never returns endpoint 1 (down) or a weight-0 node
|
|
262
|
+
|
|
263
|
+
// Whole pool down, or every eligible node weight 0 -> fail closed.
|
|
264
|
+
for (let i = 0; i < 4; i++) wr.setEligible(i, false);
|
|
265
|
+
wr.pick() === PICK_NONE; // -> true
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
- **O(1), 0 B/op, never throws.** One alias-column draw + one compare. The table is built **cold** in the constructor (and on `setWeight` / `rebuild`) with the standard Vose small/large worklist -- reusing scratch buffers, so a rebuild allocates nothing and `pick()` allocates nothing.
|
|
269
|
+
- **Eligibility by rejection sampling** (the same discipline as P2C, [ADR 0005](./decisions/0005-p2c-draw.md)): the table is built over the **eligible-independent** weights, so a **weight-0 node is never a column** (never returned). If a drawn candidate is ineligible, `pick()` redraws up to a bounded 64, then falls back to a 0-B/op rotated linear eligible scan. Because every candidate is a positive-weight node, rejecting the ineligible ones **renormalizes** the weight distribution over the surviving eligible mass -- each eligible node's share converges to `weight[i] / sum(eligible weights)`.
|
|
270
|
+
- **Sole writer of its table.** `weights` is your `Uint32Array` (the SmoothWRR/SED seam); the balancer owns the derived alias table and is its only writer via cold `setWeight` / `rebuild`. Mutate `weights` directly and the table desyncs (UB). An eligibility flap **never** rebuilds. `PICK_NONE` only when `live === 0` or no eligible node has a positive weight.
|
|
271
|
+
- **Not `@zakkster/lite-random`.** That is a *game RNG* (loot tables, particles) whose `weighted(items, weights)` returns an **item** one-shot and is not eligibility-aware. WeightedRandom returns an endpoint **index**, honours the shared eligibility bitmap, and owns a persistent table -- different domain (see [GUIDE.md](./GUIDE.md) / [ADR 0012](./decisions/0012-weightedrandom.md)).
|
|
272
|
+
|
|
273
|
+
The proof (from `test/balance.mjs`, n=64, skewed weights 1..16, 8e6 seeded draws): every node's observed share is within **2%** of `weight[i]/sum` (measured worst ~0.84%), a cumsum-linear O(n) foil matches the *same* fairness, and the O(1) alias sample beats that foil by **~107x ops/ms** at n=4096. Under half the pool down: **0 ineligible / 0 weight-0** returns and survivor shares within **3%** of the renormalized target.
|
|
274
|
+
|
|
275
|
+
> **Which weighted strategy?** Small-to-medium pools or when smoothness matters -> **SmoothWRR**; very large pools where the O(cap) scan hurts -> **WeightedRandom**. Full decision tree in **[GUIDE.md](./GUIDE.md)**.
|
|
276
|
+
|
|
241
277
|
## Evidence -- the two headlines (v0.6.0 benchmark suite)
|
|
242
278
|
|
|
243
|
-
> **Framing: parity on speed, superiority on the contract + balance + tail.** A trivial `i++ % n` round-robin -- or `wrr` -- *matches* P2C on raw ops/sec, so `lite-pick` does **not** claim "N times faster." Throughput is claimed at **parity**; the wins are **zero-GC**, **balance quality**, **tail latency** (GC blast-radius), and **never a dead pick**. Every number below is **seeded** and regenerated by `npm run bench:report
|
|
279
|
+
> **Framing: parity on speed, superiority on the contract + balance + tail.** A trivial `i++ % n` round-robin -- or `wrr` -- *matches* P2C on raw ops/sec, so `lite-pick` does **not** claim "N times faster." Throughput is claimed at **parity**; the wins are **zero-GC**, **balance quality**, **tail latency** (GC blast-radius), and **never a dead pick**. Every number below is **seeded** and regenerated by `npm run bench:report`. `npm run bench:verify` fails CI if a README number drifts from source: the balance and disruption tables are RE-MEASURED fresh each run and must match exactly; the GC and competitor throughput numbers are compared against the stored `benchmark/results.json` (exact for the 0-major / 0 B/op lanes, timing within +/-15%) and are NOT re-timed. `results.json` was recorded on one machine (see its env stamp); re-run `npm run bench:report` to refresh it.
|
|
244
280
|
|
|
245
281
|
### Throughput parity vs the incumbents (ops/ms, same pool)
|
|
246
282
|
|
|
247
|
-
The real pinned npm incumbents (`load-balancers`, `loadbalance`, `wrr`) run through the **same** harness on the **same** `n=1024` pool as the matching `lite-pick` strategy of the **same complexity class** -- ops/ms side by side, not a winner. On the same-work P2C row `lite-pick` holds parity
|
|
283
|
+
The real pinned npm incumbents (`load-balancers`, `loadbalance`, `wrr`) run through the **same** harness on the **same** `n=1024` pool as the matching `lite-pick` strategy of the **same complexity class** -- ops/ms side by side, not a winner (exact ops/ms are in the table below and stamped in `results.json`). On the same-work P2C row `lite-pick` holds parity. On the RoundRobin row `lite-pick` is a little slower, and that gap is owned, not hidden: `loadbalance@1.0.0` is a bare `i++ % n` with no liveness, while `lite-pick`'s `RoundRobinBalancer` forward-scans the eligibility bitmap to skip down nodes -- so it never returns a dead pick. That scan is the constant-factor cost of a guarantee none of these incumbents offer. The weighted-random row now races the **shipped** `WeightedRandomBalancer` (M10, alias table) against `wrr@1.0.0`, and here `lite-pick` is the slower one -- about **2.3x** below `wrr` on this row. That is owned too: `wrr@1.0.0` is a bare weight-expansion cursor with no eligibility filter and no PRNG, while `WeightedRandomBalancer` draws from a Vose alias table AND applies rejection-sampling eligibility over the shared bitmap on every pick (so it never returns a down or weight-0 node) -- the eligibility contract and the O(1)-at-any-pool-size sample are what you pay for. `lite-pick` claims parity only where the work is equal; where it is slower, it is slower for the liveness contract, and the balance + tail wins above are the reason to pay it.
|
|
248
284
|
|
|
249
285
|
<!-- bench:competitors -->
|
|
250
286
|
|
|
251
287
|
| family | lite-pick | lite-pick ops/ms | incumbent (npm) | incumbent ops/ms |
|
|
252
288
|
| --- | --- | --- | --- | --- |
|
|
253
|
-
| P2C (power-of-two-choices) | P2cBalancer |
|
|
254
|
-
| RoundRobin | RoundRobinBalancer |
|
|
255
|
-
| Weighted-random |
|
|
289
|
+
| P2C (power-of-two-choices) | P2cBalancer | 57019 | load-balancers@1.3.52 | 59778 |
|
|
290
|
+
| RoundRobin | RoundRobinBalancer | 229911 | loadbalance@1.0.0 | 261301 |
|
|
291
|
+
| Weighted-random | WeightedRandomBalancer | 73673 | wrr@1.0.0 | 172137 |
|
|
256
292
|
|
|
257
293
|
<!-- /bench:competitors -->
|
|
258
294
|
|
|
@@ -278,10 +314,10 @@ The point of zero-GC is **not** the pick's own latency -- a major GC pause freez
|
|
|
278
314
|
|
|
279
315
|
<!-- bench:gc -->
|
|
280
316
|
|
|
281
|
-
| lane | major GC | pick B/op | max GC pause (ms) |
|
|
317
|
+
| lane | major GC | pick retained B/op | max GC pause (ms) |
|
|
282
318
|
| --- | --- | --- | --- |
|
|
283
|
-
| lite-pick | 0 | 0 | 0.
|
|
284
|
-
| allocating foil | 13 | allocates |
|
|
319
|
+
| lite-pick | 0 | 0 | 0.2 |
|
|
320
|
+
| allocating foil | 13 | allocates | 1.8 |
|
|
285
321
|
|
|
286
322
|
<!-- /bench:gc -->
|
|
287
323
|
|
|
@@ -306,11 +342,11 @@ On a scale event (add / remove a node), what fraction of keys keep their node? T
|
|
|
306
342
|
|
|
307
343
|
| operation | when | allocates |
|
|
308
344
|
| --- | --- | --- |
|
|
309
|
-
| `new <Strategy>Balancer(...)` | construction, once | the balancer object + its owned
|
|
310
|
-
| `new ConsistentHashBalancer(...)` / `rebuild()` / `setWeight()` | cold, on build / membership / reweight | the Maglev lookup table: **`M x 4` bytes** (`~256KB` at the `65537` default `M`), a
|
|
345
|
+
| `new <Strategy>Balancer(...)` | construction, once | the balancer object + its owned state (SmoothWRR's Float64 `current`; PeakEWMA's `_ewma`/`_stamp`/lifetime-mean Float64 arrays -- **balancer-owned**, not caller-owned). The eligibility / inflight views stay caller-owned and are read live. **ConsistentHash / BoundedLoad COPY the weights** into a balancer-owned array |
|
|
346
|
+
| `new ConsistentHashBalancer(...)` / `rebuild()` / `setWeight()` (ConsistentHash / BoundedLoad) | cold, on build / membership / reweight | the Maglev lookup table: **`M x 4` bytes** (`~256KB` at the `65537` default `M`), a `Uint32Array` allocation + an `O(M x N)` populate. Each `setWeight` / `rebuild` also allocates the populate SCRATCH (a few `Int32Array(N)` + a `Uint8Array(M)`), so a rebuild leaves cold garbage -- a **cold-path** cost, never on `pick()`. `M` is **configurable down** for small pools. A health flap does **not** rebuild -- the bounded probe absorbs it |
|
|
311
347
|
| `setEligible(i, up)` | cold, on a health flip | **0** -- one byte write + an O(1) live-count adjust |
|
|
312
|
-
| `setWeight(i, w)` (SmoothWRR) | cold, on reweight | **0** -- one array write + an O(1) eligible-total adjust |
|
|
313
|
-
| `pick()` | **HOT**, per request | **0 B/op**
|
|
348
|
+
| `setWeight(i, w)` (SmoothWRR) | cold, on reweight | **0** -- one array write + an O(1) eligible-total adjust (also resets the node's smoothing credit) |
|
|
349
|
+
| `pick()` | **HOT**, per request | **allocates 0 B/op** (PerfGate scavenge counting), **retains 0 B/op** (torture). CAVEAT: `pick(now)` / `pick(keyHash)` box a non-small-integer number argument into a ~16 B transient `HeapNumber` when not inlined -- see the PeakEWMA / ConsistentHash sections; integer args in V8's small-integer range are 0 B/op |
|
|
314
350
|
| `Pool.run(fn)` (`/pool`) | per request | a promise + one small `held` array -- an async wrapper, **not** the kernel path ([ADR 0007](./decisions/0007-pool-adapter.md)) |
|
|
315
351
|
|
|
316
352
|
### Complementary to AWS NLB / ALB (not a competitor)
|
|
@@ -332,8 +368,8 @@ The composition: inbound traffic still enters through your **ALB/NLB -> service*
|
|
|
332
368
|
```js
|
|
333
369
|
import { BalancerBase, Prng, PICK_NONE, VERSION } from '@zakkster/lite-pick';
|
|
334
370
|
|
|
335
|
-
// A pool of 4 endpoints. The eligibility view is
|
|
336
|
-
// lite-di-health
|
|
371
|
+
// A pool of 4 endpoints. The eligibility view is read-only to pick(); at runtime it is
|
|
372
|
+
// flipped only through setEligible (lite-di-health / a breaker drives that call).
|
|
337
373
|
const eligible = Uint8Array.from([1, 1, 0, 1]); // endpoint 2 is down
|
|
338
374
|
|
|
339
375
|
const base = new BalancerBase(4, eligible);
|
|
@@ -352,10 +388,10 @@ rng.nextBelow(4); // -> a uint32 in [0, 4)
|
|
|
352
388
|
rng.reset(); // replays the exact stream
|
|
353
389
|
|
|
354
390
|
PICK_NONE; // -> -1 (fail-closed sentinel: no endpoint, never a dead pick)
|
|
355
|
-
VERSION; // -> '0.
|
|
391
|
+
VERSION; // -> '1.0.1'
|
|
356
392
|
```
|
|
357
393
|
|
|
358
|
-
`BalancerBase.pick()` is **abstract** -- it throws, so an unfinished strategy fails loudly rather than returning a dead index. Every shipped strategy (`RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`, `ConsistentHashBalancer`, `BoundedLoadBalancer`) extends it and reads the same
|
|
394
|
+
`BalancerBase.pick()` is **abstract** -- it throws, so an unfinished strategy fails loudly rather than returning a dead index. Every shipped strategy (`RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`, `ConsistentHashBalancer`, `BoundedLoadBalancer`, `WeightedRandomBalancer`) extends it and reads the same eligibility view; you subclass it the same way to add your own. Flip eligibility only through `setEligible` (it keeps `live` exact) and give each balancer its own eligibility array (a shared `Eligibility` object is a 2.0 item).
|
|
359
395
|
|
|
360
396
|
## Wiring it up -- `@zakkster/lite-pick/pool` (v0.5.0)
|
|
361
397
|
|
|
@@ -384,7 +420,7 @@ const fetcher = liteQueryFetcher(pool,
|
|
|
384
420
|
// query(qc, { key: ['users'], fetcher });
|
|
385
421
|
```
|
|
386
422
|
|
|
387
|
-
**Two layers, no overlap** ([ADR 0007](./decisions/0007-pool-adapter.md)): the pool owns **spatial** failover (try a different endpoint *now*); your query cache owns **temporal** retry (backoff, staleness). `run()` is a normal async wrapper -- it adds O(1) counter ops per attempt, **it is not held to the kernel's 0 B/op bar** (that's `pick()`). See it end-to-end -- least-conn fan-out over a flaky pool with a node killed mid-run, proving 0 dead picks and 0 leaked in-flight:
|
|
423
|
+
**Two layers, no overlap** ([ADR 0007](./decisions/0007-pool-adapter.md)): the pool owns **spatial** failover (try a genuinely different endpoint *now* -- it tracks the endpoints tried this run and stops when no untried eligible endpoint remains, so it never re-dispatches to a node that already failed); your query cache owns **temporal** retry (backoff, staleness). `run()` is a normal async wrapper -- it adds O(1) counter ops per attempt, **it is not held to the kernel's 0 B/op bar** (that's `pick()`). See it end-to-end -- least-conn fan-out over a flaky pool with a node killed mid-run, proving 0 dead picks and 0 leaked in-flight (from a clone of the repo; `demo/` is not in the published tarball):
|
|
388
424
|
|
|
389
425
|
```bash
|
|
390
426
|
npm run demo
|
|
@@ -396,9 +432,10 @@ lite-pick owns **no mutable state it can avoid owning** ([ADR 0001](./decisions/
|
|
|
396
432
|
|
|
397
433
|
| Concern | Owner | lite-pick's role |
|
|
398
434
|
| --- | --- | --- |
|
|
399
|
-
| Liveness / eligibility | `@zakkster/lite-di-health`
|
|
435
|
+
| Liveness / eligibility | `@zakkster/lite-di-health` owns a `Uint8Array`, flips it via `setEligible` | **reads** it; `setEligible` is the only supported writer (a direct byte write desyncs the cached `live`) |
|
|
400
436
|
| Circuit state | `@zakkster/lite-statechart` (consumed) | never built in; sees only the bit |
|
|
401
|
-
| In-flight
|
|
437
|
+
| In-flight counters | caller-owned `Uint32Array` | **reads** them; pure `pick()` |
|
|
438
|
+
| Latency / rtt (EWMA) state | **balancer-owned** `Float64Array` (`PeakEwmaBalancer`), written only by `recordRtt` | maintains it on the warm path; `pick()` decays on read |
|
|
402
439
|
| Whole pool down | -- | fail-closed: returns `PICK_NONE` (-1) |
|
|
403
440
|
| Routing flap | the layer that writes the shared view | hysteresis/dwell ([ADR 0002](./decisions/0002-anti-flapping.md)); `pick()` stays greedy |
|
|
404
441
|
|
|
@@ -406,7 +443,7 @@ lite-pick owns **no mutable state it can avoid owning** ([ADR 0001](./decisions/
|
|
|
406
443
|
|
|
407
444
|
## Composes with
|
|
408
445
|
|
|
409
|
-
The moat is not the algorithms -- it is that lite-pick wires already-proven zero-GC parts of the suite: [`lite-di-health`](https://www.npmjs.com/package/@zakkster/lite-di-health) (liveness), [`lite-o1`](https://www.npmjs.com/package/@zakkster/lite-o1) (`RandomSet` / `AliasTable` / `RingLog` substrate), [`lite-logn`](https://www.npmjs.com/package/@zakkster/lite-logn) (exact least-conn heap / Fenwick weights), [`lite-lru`](https://www.npmjs.com/package/@zakkster/lite-lru) (sticky affinity), [`lite-statechart`](https://www.npmjs.com/package/@zakkster/lite-statechart) (breaker), [`lite-query`](https://www.npmjs.com/package/@zakkster/lite-query) (the fetcher adapter), and [`lite-await`](https://www.npmjs.com/package/@zakkster/lite-await) (hedging). **None is a
|
|
446
|
+
The moat is not the algorithms -- it is that lite-pick wires already-proven zero-GC parts of the suite: [`lite-di-health`](https://www.npmjs.com/package/@zakkster/lite-di-health) (liveness), [`lite-o1`](https://www.npmjs.com/package/@zakkster/lite-o1) (`RandomSet` / `AliasTable` / `RingLog` substrate), [`lite-logn`](https://www.npmjs.com/package/@zakkster/lite-logn) (exact least-conn heap / Fenwick weights), [`lite-lru`](https://www.npmjs.com/package/@zakkster/lite-lru) (sticky affinity), [`lite-statechart`](https://www.npmjs.com/package/@zakkster/lite-statechart) (breaker), [`lite-query`](https://www.npmjs.com/package/@zakkster/lite-query) (the fetcher adapter), and [`lite-await`](https://www.npmjs.com/package/@zakkster/lite-await) (hedging). **None is a dependency of any kind** -- `dependencies`, `peerDependencies` and `peerDependenciesMeta` are all `{}`. Every seam is duck-typed over a shared TypedArray, so the kernel runs with nothing else installed; a peer would be declared only if a shipped code path imported one (none does today).
|
|
410
447
|
|
|
411
448
|
## Gates
|
|
412
449
|
|
|
@@ -415,15 +452,15 @@ Every strategy session must pass, no exceptions:
|
|
|
415
452
|
```bash
|
|
416
453
|
npm test # node:test boundary suite
|
|
417
454
|
npm run test:types # tsc type-surface check (Pick.d.ts vs runtime)
|
|
418
|
-
npm run torture # lite-leak retention + lite-gc-profiler 0 B/op (needs --expose-gc)
|
|
419
|
-
npm run test:perf # lite-perf-gate
|
|
455
|
+
npm run torture # lite-leak retention + lite-gc-profiler: pick RETAINS 0 B/op (needs --expose-gc)
|
|
456
|
+
npm run test:perf # lite-perf-gate: pick ALLOCATES 0 B/op (scavenge counting) + a mustFail teeth-check
|
|
420
457
|
npm run witness # pick throughput flatness across a pool-size sweep
|
|
421
458
|
npm run balance # peak-to-average vs the strategy ceiling + random foil (the anchor)
|
|
422
459
|
npm run fuzz # seeded invariant fuzzer: state-sync invariants after every op
|
|
423
460
|
npm run verify # all of the above
|
|
424
461
|
```
|
|
425
462
|
|
|
426
|
-
The zero-GC proof is two complementary tools kept separate (the suite's torture-harness discipline): a soak
|
|
463
|
+
The zero-GC proof is two complementary tools kept separate (the suite's torture-harness discipline): a **retention** soak (`torture.mjs`, [`@zakkster/lite-leak`](https://www.npmjs.com/package/@zakkster/lite-leak) + [`@zakkster/lite-gc-profiler`](https://www.npmjs.com/package/@zakkster/lite-gc-profiler)) that proves `pick()` **retains** 0 B/op (it reports the retained B/op), and a node:test-native **allocation** gate (`test/perf/PerfGate.test.mjs`, [`@zakkster/lite-perf-gate`](https://www.npmjs.com/package/@zakkster/lite-perf-gate)) that proves `pick()` **allocates** 0 B/op by scavenge counting at a pinned 1 MB semi-space, and includes a `mustFail` scenario proving the instrument has teeth.
|
|
427
464
|
|
|
428
465
|
## License
|
|
429
466
|
|
package/RECIPES.md
CHANGED
|
@@ -59,25 +59,32 @@ send(endpoints[i]);
|
|
|
59
59
|
|
|
60
60
|
---
|
|
61
61
|
|
|
62
|
-
## 3. Wire health -> eligibility (the
|
|
62
|
+
## 3. Wire health -> eligibility (the bitmap)
|
|
63
63
|
|
|
64
|
-
Eligibility is a
|
|
65
|
-
|
|
66
|
-
|
|
64
|
+
Eligibility is a `Uint8Array` (1 = pickable, 0 = down). A health checker, a circuit
|
|
65
|
+
breaker, or `@zakkster/lite-di-health` decides who is up; the balancer reads that state
|
|
66
|
+
through `pick()`. Flip a node **only** through `setEligible`:
|
|
67
67
|
|
|
68
68
|
```js
|
|
69
|
-
|
|
70
|
-
eligible[2] = 0; // endpoint c is down
|
|
71
|
-
|
|
72
|
-
// (b) go through the balancer so its O(1) `live` count stays exact (recommended):
|
|
73
|
-
lb.setEligible(2, false); // COLD path; idempotent; keeps `live` correct
|
|
69
|
+
lb.setEligible(2, false); // COLD path; idempotent; keeps `live` exact
|
|
74
70
|
lb.setEligible(2, true); // back up
|
|
75
71
|
|
|
76
|
-
lb.isEligible(2); // -> boolean, HOT, out-of-range is false
|
|
72
|
+
lb.isEligible(2); // -> boolean, HOT, out-of-range (or non-integer) is false, never throws
|
|
77
73
|
```
|
|
78
74
|
|
|
79
|
-
|
|
80
|
-
|
|
75
|
+
- **`setEligible()` is the only supported writer.** It flips the byte AND keeps the
|
|
76
|
+
balancer's cached `live` count (and SmoothWRR's eligible-weight total) exact in
|
|
77
|
+
lockstep. A DIRECT write to the array (`eligible[2] = 0`) desyncs that cache: `pick()`
|
|
78
|
+
then reads a stale `live`, which can fail closed on a pool that is actually up, or
|
|
79
|
+
destroy weight ratios and funnel 100% of traffic to one node. Have your health source
|
|
80
|
+
call `setEligible` rather than write the byte.
|
|
81
|
+
- **Each balancer needs its OWN eligibility array.** Do not share one `Uint8Array`
|
|
82
|
+
across two balancers -- each caches its own `live`, so a `setEligible` on one leaves
|
|
83
|
+
the other's count stale. Give each balancer its own array (an `Eligibility` value
|
|
84
|
+
object that multiple balancers can share is deferred to 2.0; ADR 0001, amended 1.0.1).
|
|
85
|
+
|
|
86
|
+
Health flapping is the writer's problem: apply hysteresis/dwell in the health layer --
|
|
87
|
+
`pick()` stays greedy and stateless.
|
|
81
88
|
|
|
82
89
|
---
|
|
83
90
|
|
|
@@ -96,7 +103,7 @@ lb.setWeight(0, 5); // a is 5x
|
|
|
96
103
|
lb.setWeight(1, 1);
|
|
97
104
|
lb.setWeight(2, 1);
|
|
98
105
|
lb.setWeight(3, 1);
|
|
99
|
-
// pick() interleaves smoothly (nginx smooth WRR): a a b a c a a
|
|
106
|
+
// pick() interleaves smoothly (nginx smooth WRR): a a b a c a d a ... not a a a a a b c d
|
|
100
107
|
```
|
|
101
108
|
|
|
102
109
|
Use SmoothWRR when weights are known/config-driven and change rarely.
|
|
@@ -164,20 +171,25 @@ const body = await pool.run((i, signal) => fetchFrom(endpoints[i], { signal }));
|
|
|
164
171
|
## 7. Failover -- try a different endpoint on error
|
|
165
172
|
|
|
166
173
|
Set `tries > 1`. On a thrown error, Pool keeps the failed node's in-flight count
|
|
167
|
-
elevated and
|
|
168
|
-
|
|
174
|
+
elevated and fails over to a genuinely DIFFERENT endpoint: it re-picks while the
|
|
175
|
+
strategy repeats an already-tried endpoint (bounded), then scans for an eligible untried
|
|
176
|
+
one (spread across keys for a keyed run, cursor-rotated otherwise). It stops -- surfacing
|
|
177
|
+
the last error -- as soon as no untried eligible endpoint remains, so a 1-node pool with
|
|
178
|
+
`tries: 3` makes exactly **one** attempt (Pool owns spatial failover, never temporal
|
|
179
|
+
retry against the same node).
|
|
169
180
|
|
|
170
181
|
```js
|
|
171
182
|
const body = await pool.run(
|
|
172
183
|
(i, signal) => fetchFrom(endpoints[i], { signal }),
|
|
173
|
-
{ tries: 3, signal: req.signal } // up to 3
|
|
184
|
+
{ tries: 3, signal: req.signal } // up to 3 DISTINCT endpoints
|
|
174
185
|
);
|
|
175
186
|
```
|
|
176
187
|
|
|
177
188
|
Boundary: Pool owns **spatial** failover (move across the pool, once each, in-process).
|
|
178
189
|
The caller or your query cache owns **temporal** retry (backoff, staleness, dedup).
|
|
179
|
-
Don't double-own them.
|
|
180
|
-
|
|
190
|
+
Don't double-own them. `signal` is checked before EVERY attempt: an already-aborted
|
|
191
|
+
signal dispatches nothing and rejects with the signal's `reason` (or a `LITE_PICK_ABORTED`
|
|
192
|
+
-coded error); an abort after a failure stops failover and the abort propagates.
|
|
181
193
|
|
|
182
194
|
---
|
|
183
195
|
|
|
@@ -194,7 +206,7 @@ Manual loop:
|
|
|
194
206
|
```js
|
|
195
207
|
import { PeakEwmaBalancer } from '@zakkster/lite-pick';
|
|
196
208
|
|
|
197
|
-
const TAU_NS = 30_000_000; // 30ms half-life
|
|
209
|
+
const TAU_NS = 30_000_000; // 30ms EWMA TIME CONSTANT (half-life = tau x ln2 ~= 21ms)
|
|
198
210
|
const inflight = new Uint32Array(CAP);
|
|
199
211
|
const lb = new PeakEwmaBalancer(CAP, eligible, inflight, TAU_NS);
|
|
200
212
|
const nowNs = () => Number(process.hrtime.bigint());
|
|
@@ -214,23 +226,70 @@ async function handle(req) {
|
|
|
214
226
|
}
|
|
215
227
|
```
|
|
216
228
|
|
|
217
|
-
Or let Pool do the feedback for you
|
|
218
|
-
`
|
|
229
|
+
Or let Pool do the feedback for you. A latency balancer **requires** `opts.clock`
|
|
230
|
+
(otherwise `run` rejects with a `LITE_PICK_CLOCK_REQUIRED`-coded error). Pool reads the
|
|
231
|
+
clock before each dispatch, drives `pick(now)`, records the settled rtt on success, and
|
|
232
|
+
-- new in 1.0.1 -- feeds a PENALTY on a thrown attempt so a fast-failing endpoint stops
|
|
233
|
+
looking cheap:
|
|
219
234
|
|
|
220
235
|
```js
|
|
221
236
|
import { Pool } from '@zakkster/lite-pick/pool';
|
|
222
237
|
const pool = new Pool(lb, inflight);
|
|
238
|
+
const clock = () => Number(process.hrtime.bigint());
|
|
239
|
+
|
|
223
240
|
const body = await pool.run(
|
|
224
241
|
(i, signal) => fetchFrom(endpoints[i], { signal }),
|
|
225
|
-
{
|
|
242
|
+
{
|
|
243
|
+
clock, // REQUIRED for PeakEWMA; validated finite each read
|
|
244
|
+
tries: 2,
|
|
245
|
+
failurePenaltyNs: 1_000_000_000, // a throw records max(elapsed, this) as the rtt (default 1s)
|
|
246
|
+
}
|
|
226
247
|
);
|
|
227
248
|
```
|
|
228
249
|
|
|
250
|
+
Without the penalty a node that fails instantly would be sampled at ~0 rtt and become
|
|
251
|
+
the most attractive pick (a black hole). With it, a failing node is priced expensive and
|
|
252
|
+
only RE-PROBED roughly every `tauNs x ln(failurePenaltyNs / healthyRttNs)`, so it
|
|
253
|
+
recovers when it heals but never dominates while broken.
|
|
254
|
+
|
|
255
|
+
**Feedback never loses a result and never re-runs `fn`.** `fn` runs exactly once per
|
|
256
|
+
attempt. If settle-time feedback after a SUCCESS fails (a clock that throws or returns
|
|
257
|
+
non-finite, or a throwing `recordRtt`), `run` rejects with a `LITE_PICK_FEEDBACK`-coded
|
|
258
|
+
error carrying `.cause` (the feedback error) and `.result` (fn's resolved value), so the
|
|
259
|
+
caller can still use the result:
|
|
260
|
+
|
|
261
|
+
```js
|
|
262
|
+
try {
|
|
263
|
+
return await pool.run(work, { clock, tries: 2 });
|
|
264
|
+
} catch (e) {
|
|
265
|
+
if (e.code === 'LITE_PICK_FEEDBACK') return e.result; // fn succeeded; only the rtt bookkeeping failed
|
|
266
|
+
throw e;
|
|
267
|
+
}
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
(A feedback failure after a FAILED attempt instead re-throws `fn`'s own error unchanged,
|
|
271
|
+
with the feedback error attached as a non-enumerable `liteFeedbackError` -- identity is
|
|
272
|
+
preserved. A backwards-stepping but finite clock records no sample and resolves normally.)
|
|
273
|
+
|
|
229
274
|
Notes:
|
|
230
|
-
- **Cold start
|
|
231
|
-
|
|
275
|
+
- **Cold start.** An unsampled node costs 0 while idle (so it takes one probe request at
|
|
276
|
+
a time) and the pool's lifetime mean sampled rtt once it is busy -- graceful, never NaN,
|
|
277
|
+
never the old 1.0 ns black hole. A node that never records a sample (e.g. one that fails
|
|
278
|
+
fast so the caller records nothing) keeps winning while idle: record failures too (the
|
|
279
|
+
Pool `failurePenaltyNs` above does this for you).
|
|
280
|
+
- `now` must be a FINITE number. A non-finite `now` degrades `pick` to P2C-random (no
|
|
281
|
+
throw); `recordRtt` throws on a non-finite argument.
|
|
232
282
|
- Pick `tauNs` around your p50-p90 rtt: smaller = reacts faster to a slowdown, larger =
|
|
233
|
-
steadier.
|
|
283
|
+
steadier. `tauNs` is the EWMA TIME CONSTANT (half-life = `tauNs x ln2`); it IS the
|
|
284
|
+
anti-flap smoothing, no extra dwell needed.
|
|
285
|
+
- **KNOWN LIMITATION (1.0.1, buffer-based API planned for 1.1.0).** `pick(now)` /
|
|
286
|
+
`recordRtt(..., now)` take a nanosecond `now` as a plain number argument. When the call
|
|
287
|
+
is not inlined, V8 boxes a non-small-integer number into a ~16 B HeapNumber -- so with a
|
|
288
|
+
realistic nanosecond clock these calls allocate ~16 B/op (transient, dies young; it does
|
|
289
|
+
not RETAIN and does not force a major GC). Small-integer arguments are 0 B/op; the
|
|
290
|
+
small-integer range is build-dependent (below 2^31 on stock 64-bit Node, below 2^30 on
|
|
291
|
+
pointer-compressed builds such as Chrome/Electron). A buffer-based `recordRttFrom`/clock
|
|
292
|
+
API that keeps `now` in a `Float64Array` slot is planned for 1.1.0.
|
|
234
293
|
- Measured effect: with one node at 10x latency, PeakEWMA sends it a tiny fraction of
|
|
235
294
|
the traffic P2C-over-inflight would, and cuts service p99 sharply.
|
|
236
295
|
|
|
@@ -277,6 +336,24 @@ Notes:
|
|
|
277
336
|
allocation. Turn `M` down for a small pool (any prime `>= N`).
|
|
278
337
|
- `pick(keyHash)` coerces `keyHash >>> 0` and never throws; it returns `PICK_NONE` only when the
|
|
279
338
|
pool is down or no eligible backend is reachable within the probe bound.
|
|
339
|
+
- **Through `/pool`, a keyed balancer REQUIRES `opts.key`** (an integer), or `pool.run`
|
|
340
|
+
rejects with a `LITE_PICK_KEY_REQUIRED`-coded error -- the key never routes silently to
|
|
341
|
+
backend 0. The key drives the keyed `pick(key)` only; it is never passed to a latency
|
|
342
|
+
balancer as `now`:
|
|
343
|
+
```js
|
|
344
|
+
import { Pool } from '@zakkster/lite-pick/pool';
|
|
345
|
+
const pool = new Pool(lb, new Uint32Array(CAP)); // CH does not read inflight; any view works
|
|
346
|
+
const body = await pool.run(
|
|
347
|
+
(endpoint, signal) => fetchFrom(endpoints[endpoint], { signal }),
|
|
348
|
+
{ key: fnv1a(sessionId), tries: 2 } // same key -> same backend; failover overflows to a neighbour
|
|
349
|
+
);
|
|
350
|
+
```
|
|
351
|
+
- **KNOWN LIMITATION (1.0.1).** `pick(keyHash)` takes the key as a plain number argument.
|
|
352
|
+
A key `>= 2^31` (about half of a 32-bit FNV-1a output) boxes into a ~16 B HeapNumber
|
|
353
|
+
when the call is not inlined (transient, does not retain; keys below the build's
|
|
354
|
+
small-integer range -- 2^31 on stock 64-bit Node, 2^30 on pointer-compressed builds --
|
|
355
|
+
are 0 B/op). A value produced by `%` or division can box even when it is a small
|
|
356
|
+
integer. A buffer-based key API is planned for 1.1.0.
|
|
280
357
|
|
|
281
358
|
---
|
|
282
359
|
|
|
@@ -311,10 +388,13 @@ const fetcher = liteQueryFetcher(
|
|
|
311
388
|
| SED | in-flight / weight | O(cap) | weighted least-conn |
|
|
312
389
|
| NQ | idle-first else SED | O(cap)/O(1) | worker pools -- never queue while a worker is free |
|
|
313
390
|
| PeakEWMA | in-flight x ewma(rtt)| O(1) | heterogeneous / flaky backends; steer around slow nodes |
|
|
391
|
+
| WeightedRandom| static weight (O(1))| O(1) | weighted at very large pools where SmoothWRR's O(cap) scan hurts; accepts sampling variance |
|
|
314
392
|
| ConsistentHash| key hash (sticky) | O(1) | affinity/sticky: same key -> same backend, minimal disruption on scale |
|
|
393
|
+
| BoundedLoad | key hash + occupancy cap | O(1) | sticky routing AND a few hot keys would otherwise overload one backend |
|
|
315
394
|
|
|
316
395
|
The load-aware strategies read the SAME `inflight` array live, so you can swap among them
|
|
317
|
-
without rewiring; ConsistentHash instead
|
|
396
|
+
without rewiring; ConsistentHash and BoundedLoad instead take an integer key per pick
|
|
397
|
+
(recipe 9). See [GUIDE.md](./GUIDE.md) for the full decision tree.
|
|
318
398
|
|
|
319
399
|
---
|
|
320
400
|
|
|
@@ -333,8 +413,9 @@ bounded-load / AZ / occupancy machinery. Feed rtt from your `fetch` timings via
|
|
|
333
413
|
lite-pick declares ZERO hard dependencies and an EMPTY `peerDependencies`. Each seam is
|
|
334
414
|
a shared TypedArray or a duck-typed shape, so you wire in a sibling only if you use it:
|
|
335
415
|
|
|
336
|
-
- `@zakkster/lite-di-health` --
|
|
337
|
-
|
|
416
|
+
- `@zakkster/lite-di-health` -- drives `setEligible` from health checks (the supported
|
|
417
|
+
writer; a direct byte write desyncs the cached `live`, recipe 3).
|
|
418
|
+
- `@zakkster/lite-statechart` -- a circuit breaker that flips eligibility via `setEligible`.
|
|
338
419
|
- `@zakkster/lite-query` -- the cache behind `liteQueryFetcher` (recipe 10).
|
|
339
420
|
- `@zakkster/lite-sketch` -- `DDSketch` for a p99-aware PeakEWMA variant (deferred).
|
|
340
421
|
- `@zakkster/lite-filter` -- a hot-key / known-key oracle at the ConsistentHash key-routing
|
|
@@ -350,9 +431,15 @@ None is required; the kernel runs over raw TypedArrays with nothing installed.
|
|
|
350
431
|
- Allocate `eligible` / `inflight` / `weights` ONCE at startup and reuse them. Never
|
|
351
432
|
build arrays per pick.
|
|
352
433
|
- The counters are YOURS -- mutate them in place (`inflight[i]++/--`), don't replace them.
|
|
353
|
-
- `pick()` /
|
|
354
|
-
|
|
355
|
-
|
|
434
|
+
- `pick()` allocates 0 B/op (proven by PerfGate scavenge counting) and retains 0 B/op
|
|
435
|
+
(proven by torture). CAVEAT: `pick(now)` / `recordRtt(..., now)` and
|
|
436
|
+
`pick(keyHash)` take a number argument; V8 boxes a non-small-integer value into a
|
|
437
|
+
~16 B transient HeapNumber when the call is not inlined -- so a realistic nanosecond
|
|
438
|
+
clock or a key `>= 2^31` allocates ~16 B/op (transient, does not retain, does not force
|
|
439
|
+
a major GC). Small-integer arguments are 0 B/op; buffer-based variants are planned for
|
|
440
|
+
1.1.0. See section 8/9.
|
|
441
|
+
- The only async allocation is the promise your own `fn` already creates (disclosed;
|
|
442
|
+
`Pool.run` adds O(1) integer ops plus one small per-run array).
|
|
356
443
|
- Fixed capacity: the pool size is set at construction and the backing arrays never
|
|
357
444
|
reallocate.
|
|
358
445
|
|
|
@@ -362,14 +449,23 @@ None is required; the kernel runs over raw TypedArrays with nothing installed.
|
|
|
362
449
|
|
|
363
450
|
- **PICK_NONE (-1)** is always possible -- handle it before indexing (recipe 2).
|
|
364
451
|
- **SmoothWRR weights** must go through `setWeight`; direct array mutation is UB.
|
|
365
|
-
- **
|
|
366
|
-
|
|
452
|
+
- **BoundedLoad occupancy** -- update `inflight[i]` AND call `note(i, +/-1)` in LOCKSTEP
|
|
453
|
+
(or drive it through `/pool`, which does both). `note` maintains the balancer's `_total`;
|
|
454
|
+
it does NOT write `inflight`. A direct `inflight` write without the matching `note`
|
|
455
|
+
desyncs `_total` and the cap goes wrong (UB).
|
|
456
|
+
- **ConsistentHash / BoundedLoad take an INTEGER key** -- hash strings yourself, cold
|
|
457
|
+
(recipe 9). Never `pick(someString)` on the hot path; `M` must be a prime `>= capacity`.
|
|
458
|
+
Through `/pool` a keyed balancer requires `opts.key` (`LITE_PICK_KEY_REQUIRED` otherwise).
|
|
367
459
|
- **Load-aware strategies need the dispatch/settle loop** -- forget the `inflight--` in
|
|
368
460
|
a `finally` and load leaks upward forever. Use `/pool` (recipe 6) to avoid it.
|
|
369
461
|
- **PeakEWMA needs a finite `now`** and rtt feedback -- without `recordRtt` it behaves
|
|
370
|
-
like LeastConn (cold-start baseline).
|
|
371
|
-
|
|
372
|
-
|
|
462
|
+
like LeastConn (cold-start baseline). Through `/pool` it requires `opts.clock`
|
|
463
|
+
(`LITE_PICK_CLOCK_REQUIRED` otherwise); pass `failurePenaltyNs` so failures are priced.
|
|
464
|
+
- **Eligibility flips go through `setEligible`** -- it is the only supported writer and
|
|
465
|
+
keeps `live` exact; a direct byte write desyncs the cached count (recipe 3). Each
|
|
466
|
+
balancer needs its own eligibility array.
|
|
467
|
+
- **Tie order is unspecified** -- LeastConn/SED/NQ break an exact tie deterministically
|
|
468
|
+
but on no promised index (1.0.1; a rotating tie-break is planned for 1.1.0).
|
|
373
469
|
- **lite-pick is not a proxy** -- it returns an index; you own transport, retries/backoff
|
|
374
470
|
(temporal), health checking, and the socket.
|
|
375
471
|
|