@zakkster/lite-pick 0.7.2 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +99 -0
- package/Pick.d.ts +87 -3
- package/Pick.js +385 -6
- package/Pool.d.ts +10 -1
- package/Pool.js +31 -4
- package/README.md +76 -17
- package/RECIPES.md +58 -8
- package/llms.txt +104 -11
- package/package.json +4 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,105 @@ All notable changes to `@zakkster/lite-pick` are documented here. The format fol
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.9.0] - 2026-09-23
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **`BoundedLoadBalancer` (M9)** -- **Consistent Hashing with Bounded Loads** (CHBL: Mirrokni et al.,
|
|
12
|
+
Google Research; Vimeo's `eps = 0.25`). It **extends `ConsistentHashBalancer`** (the Maglev table) and
|
|
13
|
+
adds a per-backend occupancy cap `cap = (1 + eps) x _total / live`: `pick(keyHash)` sticks a key to its
|
|
14
|
+
hashed home **unless** that backend is over cap, in which case the request **overflows** along the same
|
|
15
|
+
bounded forward-probe to the next eligible, under-cap backend. If none in the probe window is under cap
|
|
16
|
+
it falls back to the first eligible seen (**sticky wins; the cap is a soft preference, never a dead
|
|
17
|
+
pick**); `_total === 0` skips the cap entirely, behaving as pure consistent hashing. This keeps
|
|
18
|
+
consistent hashing's stickiness + minimal disruption **and** adds the **hotspot protection** plain
|
|
19
|
+
consistent hashing lacks. The Maglev build, probe walk, and `setWeight` / `rebuild` / `tableSize` are
|
|
20
|
+
reused verbatim. The running occupancy sum `_total` is **balancer-owned** (starts at 0) and written
|
|
21
|
+
**solely** via the warm `note(i, delta)` seam (dispatch `+1` / settle `-1`), so the cap's mean stays
|
|
22
|
+
O(1)-current without a scan; `inflight` is the caller's `Uint32Array`, read **live** as the per-backend
|
|
23
|
+
occupancy. `pick()` and `note()` are both **O(1)** / **0 B/op**. `eps` is validated typeof-first
|
|
24
|
+
(TypeError non-number, RangeError non-finite / `<= 0`) before the table is allocated; `note()` validates
|
|
25
|
+
`i` in range and `delta` as an integer, and clamps `_total` at 0. `totalInflight` exposes `_total`.
|
|
26
|
+
`PICK_NONE` only when no eligible backend is reachable within the probe window -- **never** for
|
|
27
|
+
over-cap (fail open on overload). **Contract:** the mirrored inflight counter is mutated **only** through
|
|
28
|
+
`note()` / the `/pool` adapter -- direct mutation desyncs `_total` (UB, the SmoothWRR-weights asymmetry).
|
|
29
|
+
([ADR 0011](./decisions/0011-boundedload.md)).
|
|
30
|
+
- **The pivot** (ADR 0011): the first M9 draft built the "overload" reading -- P2C-over-inflight with a
|
|
31
|
+
`(1 + eps) x mean` cap -- and it was proven **byte-identical to plain P2C** (an under-cap draw always
|
|
32
|
+
has lower inflight than an over-cap one, so "prefer under-cap" and "lower-of-two" pick the same node).
|
|
33
|
+
The cap is only *load-bearing* when the primary choice is a **hash**, so M9 is CHBL -- the algorithm
|
|
34
|
+
the roadmap cited. The P2C-with-cap reading is withdrawn as non-distinct.
|
|
35
|
+
- **Hotspot anchor** (`test/balance.mjs`) -- 64 backends, a Zipfian-skewed key stream (6 hot keys, 85% of
|
|
36
|
+
traffic), a fixed concurrency window: plain ConsistentHash pins a hot key on one backend -- measured
|
|
37
|
+
**max occupancy 129** vs a mean of **10** (a ~13x hotspot) -- while CHBL's cap holds **max occupancy 13**
|
|
38
|
+
(`cap = (1 + eps) x mean = 12.5`) by overflowing to neighbours, materially below ConsistentHash's max
|
|
39
|
+
(the foil FAILS the bounded-occupancy band). Both keep **~1.55%** minimal disruption on a scale event
|
|
40
|
+
(`<= 2/N`). Thresholds are measured from a correct run with a small margin and noted -- the impl is
|
|
41
|
+
never bent to a number.
|
|
42
|
+
- Gates extended for the new strategy: `test/BoundedLoad.test.js` boundary suite (ctor + eps validation,
|
|
43
|
+
sticky same-key routing, overflow when a home is over cap, fail-open, PICK_NONE only pool-down,
|
|
44
|
+
pure-ConsistentHash when `_total === 0`, `note()` validation + clamp, `totalInflight` tracking, minimal
|
|
45
|
+
disruption, a Pool `opts.key` net-zero round-trip); `test/fuzz.mjs` keyed note-driven subject +
|
|
46
|
+
`checkBoundedLoad` (`totalInflight === sum(inflight)` after every op) + the ConsistentHash structural
|
|
47
|
+
invariant; `test/torture.mjs` retention (small-instance CHBL loop) + `pick(keyHash)` and `note()` 0 B/op
|
|
48
|
+
phases; `test/perf/PerfGate.test.mjs` `boundedLoadPick` + `boundedLoadNote` zero-alloc scenarios + a
|
|
49
|
+
`mustFail` alloc tooth; `test/witness.mjs` O(1) const flat-work keyed subject; `benchmark/Matrix.mjs`
|
|
50
|
+
subject (dims throughput/balance/gc over the skewed-cost workload); `Pick.d.ts` +
|
|
51
|
+
`test/types/pick.test-d.ts` typed surface.
|
|
52
|
+
|
|
53
|
+
### Changed
|
|
54
|
+
|
|
55
|
+
- `Pool.run` gains an **inert-unless-duck-typed `note` hook** (mirror each dispatch as `note(i, +1)` and
|
|
56
|
+
each settle as `note(i, -1)`, net-zero per run) paralleling the PeakEWMA `recordRtt` wiring, and an
|
|
57
|
+
**`opts.key`** option -- when supplied, Pool drives `pick(key)` (keyed / CHBL routing); failover
|
|
58
|
+
re-picks with the same key, and because the failed backend's occupancy stays elevated a CHBL re-pick
|
|
59
|
+
naturally overflows to the next backend. All hooks are inert when not applicable -- Pool stays generic,
|
|
60
|
+
in-flight stays net-zero, abort/failover unchanged.
|
|
61
|
+
- `Pick.js`: STRATEGY-APPEND only -- the other eight strategies are **byte-identical**; the sole changes
|
|
62
|
+
are the header roster/count (eight -> nine), the `VERSION` bump, and the appended `BoundedLoadBalancer`
|
|
63
|
+
(which **extends `ConsistentHashBalancer`**, reusing its Maglev build + probe verbatim; ConsistentHash
|
|
64
|
+
itself is unchanged).
|
|
65
|
+
- `VERSION` bumped 0.8.0 -> **0.9.0** across the three sync sites (package.json, `Pick.js`, llms.txt);
|
|
66
|
+
package `description` roster updated (keywords already carried `bounded-load`). `peerDependencies`
|
|
67
|
+
stays `{}` (CHBL reuses M8 -- imports nothing new).
|
|
68
|
+
|
|
69
|
+
## [0.8.0] - 2026-09-23
|
|
70
|
+
|
|
71
|
+
### Added
|
|
72
|
+
|
|
73
|
+
- **`ConsistentHashBalancer` (M8)** -- sticky / cache-affinity routing via a prebuilt **Maglev
|
|
74
|
+
lookup table** (the in-kernel/production choice: Linux IPVS `mh`, Meta Katran, Cilium).
|
|
75
|
+
`pick(keyHash)` maps a caller-supplied **integer** key to a backend (`slot = keyHash % M`, a table
|
|
76
|
+
read, and a bounded forward-probe past down slots) -- **O(1)**, **0 B/op**. The key is coerced
|
|
77
|
+
`>>> 0` (NaN -> 0) and `pick` never throws (fail-closed). Per-pick *string* hashing is the one
|
|
78
|
+
zero-GC hazard, so callers hash string keys themselves (cold); `lite-pick` adds **no hashing
|
|
79
|
+
dependency**. The balancer owns the lookup `Uint32Array` (`M x 4` bytes -- ~256KB at the `65537`
|
|
80
|
+
default `M`, a disclosed **cold** one-time allocation; `M` is configurable down for small pools)
|
|
81
|
+
and an internal weights array; `setWeight(i, w)` / `rebuild()` rebuild the table cold, while a
|
|
82
|
+
**health flap never rebuilds** -- the bounded probe (<= 64 slots) absorbs it. Weighted Maglev
|
|
83
|
+
populate gives each backend a per-backend slot quota proportional to its weight (unweighted = equal).
|
|
84
|
+
Fails closed (`PICK_NONE`) when the pool is down or no eligible backend is reachable within the bound.
|
|
85
|
+
Exports `CH_DEFAULT_M` (65537) and `CH_PROBE_LIMIT` (64) alongside the class. ([ADR 0010](./decisions/0010-consistenthash.md)).
|
|
86
|
+
- **Minimal-disruption anchor** -- removing 1 of 64 backends remaps only **~1.6%** of keys (`test/balance.mjs`,
|
|
87
|
+
`benchmark/Disruption.mjs`), versus the naive-modulo foil's **~98%**. `benchmark/Disruption.mjs`
|
|
88
|
+
replaces its M6 explicit ConsistentHash **SKIP** row with a real measured Maglev row (vs the modulo
|
|
89
|
+
foil and the `1/n` ideal); `benchmark/results.json` + the README fences regenerated (bench:verify green).
|
|
90
|
+
- Gates extended for the new strategy: `test/ConsistentHash.test.js` boundary suite; `test/fuzz.mjs`
|
|
91
|
+
keyed subject + `checkConsistentHash` / `reachableWithinBound` invariants; `test/torture.mjs` retention
|
|
92
|
+
+ a `pick(keyHash)` 0 B/op phase (build excluded, cold); `test/perf/PerfGate.test.mjs`
|
|
93
|
+
`consistentHashPick` scenario + a `mustFail` alloc tooth; `test/witness.mjs` O(1) const flat-work
|
|
94
|
+
subject; `benchmark/Matrix.mjs` subject; `Pick.d.ts` + `test/types/pick.test-d.ts` typed surface.
|
|
95
|
+
|
|
96
|
+
### Changed
|
|
97
|
+
|
|
98
|
+
- `Pick.js`: STRATEGY-APPEND only -- the other seven strategies are **byte-identical**; the sole
|
|
99
|
+
changes are the header roster/count (seven -> eight), the `VERSION` bump, and the appended
|
|
100
|
+
`ConsistentHashBalancer` (+ the `chMix32` / `chIsPrime` cold helpers and the `CH_*` constants).
|
|
101
|
+
- `VERSION` bumped 0.7.2 -> **0.8.0** across the three sync sites (package.json, `Pick.js`, llms.txt);
|
|
102
|
+
package `description` + `keywords` (added `consistent-hash`, `sticky`) updated. `peerDependencies`
|
|
103
|
+
stays `{}` (the deferred `@zakkster/lite-filter` hot-key-oracle and `@zakkster/lite-o1` `EliasFano`
|
|
104
|
+
ring seams import nothing until a shipped path uses them).
|
|
105
|
+
|
|
7
106
|
## [0.7.2] - 2026-09-23
|
|
8
107
|
|
|
9
108
|
### Added
|
package/Pick.d.ts
CHANGED
|
@@ -1,9 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @zakkster/lite-pick -- TypeScript declarations.
|
|
3
3
|
*
|
|
4
|
-
*
|
|
5
|
-
* (LeastConn/SED/NQ) + PeakEWMA (latency-aware P2C)
|
|
6
|
-
* (
|
|
4
|
+
* M9 (0.9.0): substrate seams + RoundRobin + SmoothWRR + P2C + the exact LeastConn family
|
|
5
|
+
* (LeastConn/SED/NQ) + PeakEWMA (latency-aware P2C) + ConsistentHash (Maglev table) +
|
|
6
|
+
* BoundedLoad (P2C with a dynamic occupancy cap). The remaining strategy class (WeightedRandom)
|
|
7
|
+
* is added one per session.
|
|
7
8
|
*/
|
|
8
9
|
|
|
9
10
|
/** The single source-of-truth version stamp. */
|
|
@@ -187,3 +188,86 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
187
188
|
/** Pick by latency-aware power-of-two-choices at time `now` (ns), or `PICK_NONE`. O(d)=O(1). */
|
|
188
189
|
pick(now?: number): number;
|
|
189
190
|
}
|
|
191
|
+
|
|
192
|
+
/** The default Maglev lookup-table size (a prime, 2^16 + 1). Configurable via the ctor. */
|
|
193
|
+
export const CH_DEFAULT_M: number;
|
|
194
|
+
|
|
195
|
+
/** The bounded forward-probe limit ConsistentHash walks past down slots (fail-closed). */
|
|
196
|
+
export const CH_PROBE_LIMIT: number;
|
|
197
|
+
|
|
198
|
+
/**
|
|
199
|
+
* ConsistentHashBalancer -- sticky / cache-affinity routing via a prebuilt MAGLEV lookup table
|
|
200
|
+
* (M8, IPVS `mh` / Meta Katran / Cilium). `pick(keyHash)` maps a caller-supplied INTEGER key to a
|
|
201
|
+
* fixed backend (slot = keyHash % M, a table read, and a bounded forward-probe past down slots) --
|
|
202
|
+
* O(1), 0 B/op. The key is a caller-supplied integer (coerced `>>> 0`; NaN -> 0), never a per-pick
|
|
203
|
+
* string hash (the one zero-GC hazard -- hash string keys yourself, cold). The balancer OWNS the
|
|
204
|
+
* lookup table (M x 4 bytes; the 65537 default is ~256KB, a COLD one-time allocation) and an internal
|
|
205
|
+
* weights array; `setWeight` / `rebuild` rebuild the table (COLD). A health flap is absorbed by the
|
|
206
|
+
* probe -- never a rebuild -- so removing a backend (`setEligible(i, false)`) remaps only ~1/N keys.
|
|
207
|
+
* Fails closed (`PICK_NONE`) when the pool is down or no eligible backend is reachable within the bound.
|
|
208
|
+
*/
|
|
209
|
+
export class ConsistentHashBalancer extends BalancerBase {
|
|
210
|
+
/**
|
|
211
|
+
* @param capacity backend count (fixed).
|
|
212
|
+
* @param eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
213
|
+
* @param weights optional per-backend weights (length >= capacity), COPIED at construction;
|
|
214
|
+
* null = equal weight.
|
|
215
|
+
* @param m the Maglev table size: a prime, > 1, and >= capacity (default 65537).
|
|
216
|
+
* @param seed deterministic salt for the permutation mix (default 0x9e3779b9); reproducible.
|
|
217
|
+
*/
|
|
218
|
+
constructor(capacity: number, eligible: Uint8Array, weights?: Uint32Array | null, m?: number, seed?: number);
|
|
219
|
+
/** The Maglev table size M (prime). */
|
|
220
|
+
readonly tableSize: number;
|
|
221
|
+
/** Cold path: reconfigure backend `i`'s weight (uint32) and rebuild the table. */
|
|
222
|
+
setWeight(i: number, w: number): void;
|
|
223
|
+
/** Cold path: rebuild the lookup table from the current owned weights. */
|
|
224
|
+
rebuild(): void;
|
|
225
|
+
/** Map an integer `keyHash` to a backend index (bounded probe past down slots), or `PICK_NONE`. */
|
|
226
|
+
pick(keyHash?: number): number;
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
/**
|
|
230
|
+
* BoundedLoadBalancer -- Consistent Hashing with Bounded Loads (M9, CHBL: Mirrokni et al. / Google
|
|
231
|
+
* Research; Vimeo eps ~ 0.25). `ConsistentHashBalancer` (the Maglev table) PLUS an occupancy cap: a
|
|
232
|
+
* key sticks to its hashed home backend UNLESS that backend is over `cap = (1 + eps) * _total / live`,
|
|
233
|
+
* in which case the request OVERFLOWS along the same bounded forward-probe to the next eligible,
|
|
234
|
+
* under-cap backend -- keeping consistent hashing's stickiness + minimal disruption AND adding the
|
|
235
|
+
* HOTSPOT protection plain consistent hashing lacks. `pick(keyHash)` returns the first eligible,
|
|
236
|
+
* under-cap backend in the probe window, else falls back to the first eligible seen (sticky wins; the
|
|
237
|
+
* cap is a soft preference, never a dead pick); `_total === 0` skips the cap -> pure ConsistentHash.
|
|
238
|
+
* `inflight` is the caller-owned Uint32Array read LIVE as the per-backend OCCUPANCY; the running
|
|
239
|
+
* occupancy sum `_total` is BALANCER-OWNED and written ONLY by `note` (dispatch +1 / settle -1), so
|
|
240
|
+
* when using BoundedLoad the mirrored counter must be mutated exclusively through `note` / the /pool
|
|
241
|
+
* adapter (direct mutation desyncs `_total` -- UB). It inherits the Maglev table + `setWeight` /
|
|
242
|
+
* `rebuild` / `tableSize` from ConsistentHashBalancer (reused verbatim). `pick()` and `note()` are
|
|
243
|
+
* both 0 B/op / O(1). Fails closed (`PICK_NONE`) ONLY when no eligible backend is reachable within
|
|
244
|
+
* the probe window -- NEVER merely because backends are over cap. NOT the P2C-with-cap "overload"
|
|
245
|
+
* variant (that is byte-identical to P2C; the cap is only load-bearing on a sticky hash -- ADR 0011).
|
|
246
|
+
*/
|
|
247
|
+
export class BoundedLoadBalancer extends ConsistentHashBalancer {
|
|
248
|
+
/**
|
|
249
|
+
* @param capacity backend count (fixed).
|
|
250
|
+
* @param eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
251
|
+
* @param inflight per-backend OCCUPANCY (length >= capacity), caller-owned, read live; mutated
|
|
252
|
+
* EXCLUSIVELY via `note` / the /pool adapter (direct mutation desyncs `_total` -- UB).
|
|
253
|
+
* @param eps the bounded-load slack over the mean (finite, > 0); default 0.25 (Vimeo).
|
|
254
|
+
* @param weights optional per-backend weights (length >= capacity), COPIED; null = equal weight.
|
|
255
|
+
* @param m the Maglev table size: a prime, > 1, and >= capacity (default 65537).
|
|
256
|
+
* @param seed deterministic salt for the permutation mix (default 0x9e3779b9); reproducible.
|
|
257
|
+
*/
|
|
258
|
+
constructor(
|
|
259
|
+
capacity: number,
|
|
260
|
+
eligible: Uint8Array,
|
|
261
|
+
inflight: Uint32Array,
|
|
262
|
+
eps?: number,
|
|
263
|
+
weights?: Uint32Array | null,
|
|
264
|
+
m?: number,
|
|
265
|
+
seed?: number,
|
|
266
|
+
);
|
|
267
|
+
/** The balancer-owned running sum of in-flight the mean/cap is computed from. */
|
|
268
|
+
readonly totalInflight: number;
|
|
269
|
+
/** Warm feedback path: adjust the owned occupancy sum (dispatch +1 / settle -1). Clamps at 0. 0 B/op. */
|
|
270
|
+
note(i: number, delta: number): void;
|
|
271
|
+
/** Map an integer `keyHash` to a backend, honouring the occupancy cap (overflow past a hot home), or `PICK_NONE`. O(1). */
|
|
272
|
+
pick(keyHash?: number): number;
|
|
273
|
+
}
|
package/Pick.js
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @zakkster/lite-pick -- zero-GC load-balancing SELECTION KERNEL.
|
|
3
3
|
*
|
|
4
|
-
*
|
|
5
|
-
* LeastConn family (LeastConn, SED, NQ),
|
|
4
|
+
* M9 (0.9.0): substrate seams + nine strategies -- RoundRobin, SmoothWRR, P2C, the exact
|
|
5
|
+
* LeastConn family (LeastConn, SED, NQ), PeakEWMA (latency-aware P2C), ConsistentHash
|
|
6
|
+
* (a Maglev lookup table), and BoundedLoad (Consistent Hashing with Bounded Loads: the
|
|
7
|
+
* Maglev table + an occupancy cap that overflows a hot backend). This file ships:
|
|
6
8
|
*
|
|
7
9
|
* - VERSION the single source-of-truth version stamp (3-place sync).
|
|
8
10
|
* - PICK_NONE the fail-closed sentinel (-1): "no endpoint", never a dead pick.
|
|
@@ -31,6 +33,20 @@
|
|
|
31
33
|
* Decay-on-READ (pick() never writes -> 0 B/op); the balancer OWNS the Float64
|
|
32
34
|
* _ewma/_stamp state and is its SOLE writer via the warm recordRtt() feedback
|
|
33
35
|
* path (also 0 B/op). Caller-supplied nanosecond clock. O(d)=O(1)/pick.
|
|
36
|
+
* - ConsistentHashBalancer sticky/affinity routing via a prebuilt Maglev lookup table (IPVS
|
|
37
|
+
* `mh`, Meta Katran, Cilium): pick(keyHash) is slot = keyHash % M, a table read,
|
|
38
|
+
* and a bounded forward-probe over down slots -- O(1)/pick, 0 B/op. keyHash is a
|
|
39
|
+
* caller-supplied INTEGER (no per-pick string hashing = the one zero-GC hazard);
|
|
40
|
+
* the balancer OWNS the Uint32Array table + weights, rebuilt COLD on membership /
|
|
41
|
+
* weight change (health flap is handled by the probe, never a rebuild).
|
|
42
|
+
* - BoundedLoadBalancer Consistent Hashing with Bounded Loads (CHBL, Mirrokni et al. / Google
|
|
43
|
+
* Research; Vimeo eps=0.25): ConsistentHash (the Maglev table) PLUS an occupancy
|
|
44
|
+
* cap. pick(keyHash) sticks a key to its hashed home UNLESS that backend is over
|
|
45
|
+
* cap = (1+eps) x _total / live, in which case the request OVERFLOWS along the same
|
|
46
|
+
* bounded probe to the next eligible under-cap backend -- consistent hashing's
|
|
47
|
+
* stickiness + minimal disruption PLUS the hotspot protection plain CH lacks. It
|
|
48
|
+
* extends ConsistentHashBalancer (reusing its Maglev build + probe VERBATIM) and
|
|
49
|
+
* OWNS a running `_total` (sole writer: the warm note(i, delta) seam). O(1), 0 B/op.
|
|
34
50
|
*
|
|
35
51
|
* The identity (decisions/0001): lite-pick OWNS NO mutable state it can avoid owning.
|
|
36
52
|
* It reads pre-allocated views (eligibility, inflight, weights, scores) that siblings or
|
|
@@ -38,9 +54,9 @@
|
|
|
38
54
|
* counters live OUTSIDE the kernel. The steady-state pick path allocates 0 B/op.
|
|
39
55
|
*
|
|
40
56
|
* Roster (one strategy per session -- see ROADMAP.md): RoundRobin [M1], SmoothWRR [M2],
|
|
41
|
-
* P2C [M3], LeastConn/SED/NQ [M4], PeakEWMA [M7], ConsistentHash, BoundedLoad,
|
|
42
|
-
* [planned]. The EXACT-O(log n) fewest-in-flight variant is a deferred
|
|
43
|
-
* BinaryHeap optional-peer seam (decisions/0006), not this exact-O(cap) scan.
|
|
57
|
+
* P2C [M3], LeastConn/SED/NQ [M4], PeakEWMA [M7], ConsistentHash [M8], BoundedLoad [M9],
|
|
58
|
+
* WeightedRandom [planned]. The EXACT-O(log n) fewest-in-flight variant is a deferred
|
|
59
|
+
* @zakkster/lite-logn BinaryHeap optional-peer seam (decisions/0006), not this exact-O(cap) scan.
|
|
44
60
|
*
|
|
45
61
|
* M5 (0.5.0) adds the ergonomic request layer at the @zakkster/lite-pick/pool subpath (a
|
|
46
62
|
* SEPARATE file, Pool.js -- the async dispatch/settle counter wrapper + distinct-endpoint
|
|
@@ -51,7 +67,7 @@
|
|
|
51
67
|
*/
|
|
52
68
|
|
|
53
69
|
/** Version stamp. Synced across package.json and llms.txt (three-place rule). */
|
|
54
|
-
export const VERSION = '0.
|
|
70
|
+
export const VERSION = '0.9.0';
|
|
55
71
|
|
|
56
72
|
/**
|
|
57
73
|
* Fail-closed sentinel returned by pick() when no endpoint is eligible.
|
|
@@ -739,3 +755,366 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
739
755
|
return costB < costA ? b : a; // lower cost wins; tie -> the first draw
|
|
740
756
|
}
|
|
741
757
|
}
|
|
758
|
+
|
|
759
|
+
/** The default Maglev lookup-table size: a prime (2^16 + 1). Configurable via the ctor. */
|
|
760
|
+
export const CH_DEFAULT_M = 65537;
|
|
761
|
+
|
|
762
|
+
/** The bounded forward-probe limit ConsistentHash walks past down slots (fail-closed). */
|
|
763
|
+
export const CH_PROBE_LIMIT = 64;
|
|
764
|
+
|
|
765
|
+
/**
|
|
766
|
+
* A deterministic 32-bit integer mix (an SplitMix/Murmur-style finalizer). Used COLD, once
|
|
767
|
+
* per backend at table build, to derive the two Maglev permutation parameters from a backend
|
|
768
|
+
* INDEX -- NO string hashing, NO new dependency, no allocation. Returns a uint32.
|
|
769
|
+
* @param {number} x
|
|
770
|
+
* @returns {number}
|
|
771
|
+
*/
|
|
772
|
+
function chMix32(x) {
|
|
773
|
+
x = x >>> 0;
|
|
774
|
+
x ^= x >>> 16; x = Math.imul(x, 0x7feb352d);
|
|
775
|
+
x ^= x >>> 15; x = Math.imul(x, 0x846ca68b);
|
|
776
|
+
x ^= x >>> 16;
|
|
777
|
+
return x >>> 0;
|
|
778
|
+
}
|
|
779
|
+
|
|
780
|
+
/** Cold primality test (trial division). M must be prime so a Maglev skip yields a full permutation. */
|
|
781
|
+
function chIsPrime(n) {
|
|
782
|
+
if (!Number.isInteger(n) || n < 2) return false;
|
|
783
|
+
if (n % 2 === 0) return n === 2;
|
|
784
|
+
if (n % 3 === 0) return n === 3;
|
|
785
|
+
for (let d = 5; d * d <= n; d += 6) {
|
|
786
|
+
if (n % d === 0 || n % (d + 2) === 0) return false;
|
|
787
|
+
}
|
|
788
|
+
return true;
|
|
789
|
+
}
|
|
790
|
+
|
|
791
|
+
/**
|
|
792
|
+
* ConsistentHashBalancer -- sticky / cache-affinity routing via a prebuilt MAGLEV lookup table
|
|
793
|
+
* (M8), the in-kernel/production consistent-hash choice (Linux IPVS `mh`, Meta Katran, Cilium).
|
|
794
|
+
*
|
|
795
|
+
* `pick(keyHash)` maps a caller-supplied INTEGER key to a fixed backend: slot = keyHash % M, read
|
|
796
|
+
* the backend at `lookup[slot]`, and if it is down walk a BOUNDED forward-probe (<= 64 slots) to
|
|
797
|
+
* the next eligible backend. That is a few integer ops over a prebuilt Uint32Array table -- O(1)
|
|
798
|
+
* per pick, 0 B/op. The KEY MUST BE AN INTEGER (`keyHash >>> 0`, so NaN -> 0 deterministically):
|
|
799
|
+
* per-pick STRING hashing is the one zero-GC hazard, so the caller hashes string keys themselves
|
|
800
|
+
* (cold) and passes the integer -- lite-pick adds NO hashing dependency (RESEARCH section 5).
|
|
801
|
+
*
|
|
802
|
+
* MINIMAL DISRUPTION is the selling point: on a scale event only ~1/N of keys move. Removing a
|
|
803
|
+
* backend is just marking it down (`setEligible(i, false)`) -- the table is UNCHANGED, so every
|
|
804
|
+
* key NOT on that backend keeps its exact backend (0 remap) and only its keys probe forward.
|
|
805
|
+
* A membership or WEIGHT change rebuilds the table (COLD); a health flap NEVER does (the probe
|
|
806
|
+
* absorbs it). test/balance.mjs anchors this: remove 1 of 64 backends -> <= 3.13% keys remapped,
|
|
807
|
+
* vs the naive-modulo foil's >= 95%.
|
|
808
|
+
*
|
|
809
|
+
* Ownership (ADR 0001, ADR 0010): the balancer OWNS the lookup `Uint32Array` (M x 4 bytes; the
|
|
810
|
+
* 65537 default is ~256KB, a COLD one-time allocation -- disclosed, and M is configurable DOWN
|
|
811
|
+
* for small pools) AND an internal weights `Uint32Array` (the SmoothWRR sole-writer precedent):
|
|
812
|
+
* cold `setWeight(i, w)` / `rebuild()` rebuild the table from the current weights. Eligibility is
|
|
813
|
+
* the shared read-only bitmap from BalancerBase, read live by `pick()`; the base `setEligible`
|
|
814
|
+
* only flips a bit (no rebuild).
|
|
815
|
+
*
|
|
816
|
+
* Weighted Maglev populate: each backend b takes a per-backend slot QUOTA proportional to its
|
|
817
|
+
* weight (unweighted = equal quota, quotas summing to exactly M), stepping through its own
|
|
818
|
+
* permutation `permutation[j] = (offset + j*skip) % M` (offset = h1(b) % M, skip = h2(b) % (M-1) +
|
|
819
|
+
* 1, h1/h2 from a deterministic integer mix of the index -- no string hashing). Because the
|
|
820
|
+
* permutation covers ALL M slots, a backend with remaining quota can always reach an empty slot
|
|
821
|
+
* while one exists, so the O(M x N) COLD build never stalls. BUILD-COST GUARD (fail closed): the
|
|
822
|
+
* ctor requires `capacity <= M` (more members than slots would overfill M / starve backends) and
|
|
823
|
+
* M prime > 1.
|
|
824
|
+
*
|
|
825
|
+
* DEFERRED optional-peer seam (ADR 0010 / llms.txt, never on the pick path): a `@zakkster/lite-filter`
|
|
826
|
+
* hot-key / known-key oracle (BlockedBloom etc.) for warm-affinity + admission at the KEY-routing
|
|
827
|
+
* layer, and a `@zakkster/lite-o1` `EliasFano` ring alternative to the table. Import NOTHING;
|
|
828
|
+
* `peerDependencies` STAYS `{}` until a shipped code path imports it.
|
|
829
|
+
*
|
|
830
|
+
* Bound: O(1) per pick, 0 B/op (integer ops over the prebuilt table -- no allocation). Fails closed
|
|
831
|
+
* (PICK_NONE) when the whole pool is down OR no eligible backend is reachable within the probe bound
|
|
832
|
+
* (a near-total outage may return PICK_NONE even if a far eligible slot exists -- safe, never a dead
|
|
833
|
+
* pick, over-conservative only under mass outage; ADR 0010).
|
|
834
|
+
*/
|
|
835
|
+
export class ConsistentHashBalancer extends BalancerBase {
|
|
836
|
+
/**
|
|
837
|
+
* @param {number} capacity backend count (fixed; add/remove is a cold rebuild).
|
|
838
|
+
* @param {Uint8Array} eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
839
|
+
* @param {Uint32Array|null} [weights=null] optional per-backend weights (length >= capacity);
|
|
840
|
+
* the values are COPIED into the balancer-owned weights at construction. null = equal weight.
|
|
841
|
+
* @param {number} [m=CH_DEFAULT_M] the Maglev table size: a prime, > 1, and >= capacity.
|
|
842
|
+
* @param {number} [seed=0x9e3779b9] deterministic salt for the permutation mix (reproducible).
|
|
843
|
+
*/
|
|
844
|
+
constructor(capacity, eligible, weights = null, m = CH_DEFAULT_M, seed = 0x9e3779b9) {
|
|
845
|
+
super(capacity, eligible);
|
|
846
|
+
// Validate typeof-first, BEFORE allocating the table / owned weights (fail closed early).
|
|
847
|
+
if (typeof m !== 'number') {
|
|
848
|
+
throw new TypeError('[lite-pick] table size M must be a number');
|
|
849
|
+
}
|
|
850
|
+
if (!Number.isInteger(m) || m < 2 || !chIsPrime(m)) {
|
|
851
|
+
throw new RangeError('[lite-pick] table size M must be a prime integer > 1: ' + m);
|
|
852
|
+
}
|
|
853
|
+
if (capacity > m) {
|
|
854
|
+
throw new RangeError('[lite-pick] capacity ' + capacity +
|
|
855
|
+
' exceeds table size M ' + m + ' (would overfill / starve backends)');
|
|
856
|
+
}
|
|
857
|
+
if (weights !== null && (!(weights instanceof Uint32Array) || weights.length < capacity)) {
|
|
858
|
+
throw new RangeError('[lite-pick] weights must be a Uint32Array of length >= capacity');
|
|
859
|
+
}
|
|
860
|
+
this._m = m;
|
|
861
|
+
this._seed = seed >>> 0;
|
|
862
|
+
// Balancer-owned weights (the SmoothWRR sole-writer precedent): copied, then the sole
|
|
863
|
+
// mutator is setWeight (which rebuilds). Unweighted default = equal weight 1.
|
|
864
|
+
this._weights = new Uint32Array(capacity);
|
|
865
|
+
if (weights !== null) this._weights.set(weights.subarray(0, capacity));
|
|
866
|
+
else this._weights.fill(1);
|
|
867
|
+
// The prebuilt lookup table (slot -> backend index). M x 4 bytes; the COLD build fills it.
|
|
868
|
+
this._lookup = new Uint32Array(m);
|
|
869
|
+
this._build();
|
|
870
|
+
}
|
|
871
|
+
|
|
872
|
+
/** The Maglev table size M (prime). Readonly. */
|
|
873
|
+
get tableSize() {
|
|
874
|
+
return this._m;
|
|
875
|
+
}
|
|
876
|
+
|
|
877
|
+
/**
|
|
878
|
+
* COLD: (re)populate the lookup table from the current balancer-owned weights via the weighted
|
|
879
|
+
* Maglev algorithm. O(M x N); never stalls (each backend's permutation covers all M slots).
|
|
880
|
+
* Rebuilt only on a membership / weight change -- never on a health flap. Allocates only cold
|
|
881
|
+
* scratch that is released after the build; the lookup table itself is reused in place.
|
|
882
|
+
*/
|
|
883
|
+
_build() {
|
|
884
|
+
const M = this._m, N = this._cap, wt = this._weights, lookup = this._lookup, seed = this._seed;
|
|
885
|
+
// Per-backend Maglev permutation parameters from a deterministic integer mix of the index.
|
|
886
|
+
const offset = new Int32Array(N);
|
|
887
|
+
const skip = new Int32Array(N);
|
|
888
|
+
for (let b = 0; b < N; b++) {
|
|
889
|
+
const h1 = chMix32(b ^ seed);
|
|
890
|
+
const h2 = chMix32((b ^ seed) + 0x9e3779b9);
|
|
891
|
+
offset[b] = h1 % M;
|
|
892
|
+
skip[b] = (h2 % (M - 1)) + 1;
|
|
893
|
+
}
|
|
894
|
+
// Per-backend slot QUOTA proportional to weight, summing to exactly M (unweighted = equal).
|
|
895
|
+
const quota = new Int32Array(N);
|
|
896
|
+
let total = 0;
|
|
897
|
+
for (let b = 0; b < N; b++) total += wt[b];
|
|
898
|
+
if (total <= 0) {
|
|
899
|
+
// Degenerate all-zero weights: equal quota so the table is still fully, validly populated.
|
|
900
|
+
const base = Math.floor(M / N);
|
|
901
|
+
for (let b = 0; b < N; b++) quota[b] = base;
|
|
902
|
+
let leftover = M - base * N;
|
|
903
|
+
for (let b = 0; leftover > 0; b = (b + 1) % N) { quota[b]++; leftover--; }
|
|
904
|
+
} else {
|
|
905
|
+
let assigned = 0;
|
|
906
|
+
for (let b = 0; b < N; b++) { const q = Math.floor(wt[b] / total * M); quota[b] = q; assigned += q; }
|
|
907
|
+
let leftover = M - assigned;
|
|
908
|
+
// Hand the rounding leftover to positive-weight backends in index order (deterministic).
|
|
909
|
+
for (let b = 0; leftover > 0; b = (b + 1) % N) { if (wt[b] > 0) { quota[b]++; leftover--; } }
|
|
910
|
+
}
|
|
911
|
+
// Maglev populate honoring the quota. The permutation is surjective over all M slots, so a
|
|
912
|
+
// backend with remaining quota always reaches an empty slot while one exists -> no stall.
|
|
913
|
+
const next = new Int32Array(N);
|
|
914
|
+
const filledCount = new Int32Array(N);
|
|
915
|
+
const taken = new Uint8Array(M);
|
|
916
|
+
let filled = 0;
|
|
917
|
+
while (filled < M) {
|
|
918
|
+
let progressed = false;
|
|
919
|
+
for (let b = 0; b < N; b++) {
|
|
920
|
+
if (filledCount[b] >= quota[b]) continue;
|
|
921
|
+
let j = next[b];
|
|
922
|
+
let c = (offset[b] + (j % M) * skip[b]) % M;
|
|
923
|
+
while (taken[c]) { j++; c = (offset[b] + (j % M) * skip[b]) % M; }
|
|
924
|
+
lookup[c] = b;
|
|
925
|
+
taken[c] = 1;
|
|
926
|
+
next[b] = j + 1;
|
|
927
|
+
filledCount[b]++;
|
|
928
|
+
filled++;
|
|
929
|
+
progressed = true;
|
|
930
|
+
if (filled >= M) break;
|
|
931
|
+
}
|
|
932
|
+
if (!progressed) break; // unreachable while quotas sum to M -- defensive
|
|
933
|
+
}
|
|
934
|
+
// Defensive completeness (unreachable): any slot left empty is pinned to backend 0 so pick()
|
|
935
|
+
// never reads a stale / out-of-range index. Fail closed on a valid table, always.
|
|
936
|
+
if (filled < M) {
|
|
937
|
+
for (let c = 0; c < M; c++) if (!taken[c]) lookup[c] = 0;
|
|
938
|
+
}
|
|
939
|
+
}
|
|
940
|
+
|
|
941
|
+
/**
|
|
942
|
+
* COLD: reconfigure backend i's weight (uint32) and REBUILD the table from the new weights.
|
|
943
|
+
* The balancer is the sole writer of its owned weights (the SmoothWRR precedent).
|
|
944
|
+
* @param {number} i
|
|
945
|
+
* @param {number} w new weight (uint32)
|
|
946
|
+
*/
|
|
947
|
+
setWeight(i, w) {
|
|
948
|
+
if (i < 0 || i >= this._cap) throw new RangeError('[lite-pick] index out of range: ' + i);
|
|
949
|
+
const nw = w >>> 0;
|
|
950
|
+
if (nw !== w) throw new RangeError('[lite-pick] weight must be a uint32: ' + w);
|
|
951
|
+
if (nw === this._weights[i]) return;
|
|
952
|
+
this._weights[i] = nw;
|
|
953
|
+
this._build();
|
|
954
|
+
}
|
|
955
|
+
|
|
956
|
+
/** COLD: rebuild the lookup table from the current owned weights (e.g. after a membership change). */
|
|
957
|
+
rebuild() {
|
|
958
|
+
this._build();
|
|
959
|
+
}
|
|
960
|
+
|
|
961
|
+
/**
|
|
962
|
+
* Map an INTEGER key to a backend index, or PICK_NONE (fail closed). O(1), 0 B/op.
|
|
963
|
+
*
|
|
964
|
+
* slot = (keyHash >>> 0) % M; if `lookup[slot]` is eligible return it, else forward-probe up to
|
|
965
|
+
* CH_PROBE_LIMIT (64) slots for the next eligible backend. `keyHash` is coerced `>>> 0` (NaN -> 0
|
|
966
|
+
* deterministically) and pick NEVER throws (the fail-closed contract). Returns PICK_NONE when the
|
|
967
|
+
* whole pool is down or no eligible backend is reachable within the bound.
|
|
968
|
+
* @param {number} keyHash a caller-supplied integer key hash (coerced to uint32)
|
|
969
|
+
* @returns {number}
|
|
970
|
+
*/
|
|
971
|
+
pick(keyHash) {
|
|
972
|
+
if (this._live === 0) return PICK_NONE; // whole pool down: fail closed
|
|
973
|
+
const M = this._m, el = this._eligible, lookup = this._lookup;
|
|
974
|
+
let slot = (keyHash >>> 0) % M; // integer key; NaN >>> 0 = 0 (never throws)
|
|
975
|
+
let i = lookup[slot];
|
|
976
|
+
if (el[i]) return i;
|
|
977
|
+
// Bounded forward-probe past down slots. Past the bound we fail closed: a near-total outage
|
|
978
|
+
// may return PICK_NONE even if a far eligible slot exists -- safe, never a dead pick.
|
|
979
|
+
for (let p = 0; p < CH_PROBE_LIMIT; p++) {
|
|
980
|
+
slot++;
|
|
981
|
+
if (slot >= M) slot = 0;
|
|
982
|
+
i = lookup[slot];
|
|
983
|
+
if (el[i]) return i;
|
|
984
|
+
}
|
|
985
|
+
return PICK_NONE;
|
|
986
|
+
}
|
|
987
|
+
}
|
|
988
|
+
|
|
989
|
+
/**
|
|
990
|
+
* BoundedLoadBalancer -- Consistent Hashing with Bounded Loads (M9, CHBL: Mirrokni-Thorup-
|
|
991
|
+
* Zadimoghaddam, Google Research 2016; Vimeo's eps ~ 0.25). This is `ConsistentHashBalancer` (the M8
|
|
992
|
+
* Maglev table) PLUS an occupancy CAP: a key sticks to its hashed home backend UNLESS that backend is
|
|
993
|
+
* over the cap, in which case the request OVERFLOWS along the same bounded forward-probe to the next
|
|
994
|
+
* eligible, under-cap backend. It keeps consistent hashing's stickiness + minimal disruption AND adds
|
|
995
|
+
* the HOTSPOT protection plain consistent hashing lacks: a few very hot keys can pile unbounded load
|
|
996
|
+
* on one backend, so the cap spreads the overflow to neighbours while everything else stays put.
|
|
997
|
+
*
|
|
998
|
+
* Why this is the REAL bounded-load strategy (ADR 0011): P2C-over-inflight with a `(1+eps) x mean` cap
|
|
999
|
+
* is byte-identical to plain P2C (an under-cap draw ALWAYS has lower inflight than an over-cap one, so
|
|
1000
|
+
* "prefer under-cap" and "lower-of-two" pick the same node) -- the cap is a no-op there. The cap is
|
|
1001
|
+
* only LOAD-BEARING when the primary choice is fixed by something OTHER than load: a hash. CHBL is
|
|
1002
|
+
* that -- the hashed home is sticky, and the cap is what lets a hot home overflow.
|
|
1003
|
+
*
|
|
1004
|
+
* `pick(keyHash)` (HOT, 0 B/op, NEVER throws): k = keyHash >>> 0; slot = k % M; walk the M8 probe
|
|
1005
|
+
* window (home + CH_PROBE_LIMIT slots) and return the FIRST backend that is ELIGIBLE AND UNDER cap
|
|
1006
|
+
* (`inflight[b] < cap`). If none in the window is under cap, FALL BACK to the first eligible seen
|
|
1007
|
+
* (sticky wins; the cap is a soft preference, never a dead pick). When `_total === 0` the cap test is
|
|
1008
|
+
* skipped entirely -> behaves as pure ConsistentHash. `cap = (1 + eps) x _total / live`.
|
|
1009
|
+
*
|
|
1010
|
+
* Ownership (ADR 0001, ADR 0004, ADR 0010, ADR 0011): the Maglev lookup table + weights are
|
|
1011
|
+
* BALANCER-OWNED and built COLD (reused from ConsistentHashBalancer VERBATIM -- `_build`, `setWeight`,
|
|
1012
|
+
* `rebuild`, `tableSize`, the probe walk, `chMix32`, `CH_DEFAULT_M`, `CH_PROBE_LIMIT`). `inflight` is
|
|
1013
|
+
* the CALLER's Uint32Array, read LIVE as the per-backend OCCUPANCY source. The running occupancy sum
|
|
1014
|
+
* `_total` is BALANCER-OWNED and its SOLE writer is the warm `note(i, delta)` feedback path
|
|
1015
|
+
* (dispatch +1 / settle -1), so the cap's mean stays O(1)-current without a scan.
|
|
1016
|
+
*
|
|
1017
|
+
* CONTRACT (the SmoothWRR-weights asymmetry): when using BoundedLoad the mirrored inflight counter is
|
|
1018
|
+
* mutated ONLY through `note()` / the /pool adapter. Direct mutation desyncs `_total` from the true
|
|
1019
|
+
* sum, so the cap goes wrong -- UB. `note()` clamps `_total` at 0; `totalInflight` exposes it.
|
|
1020
|
+
*
|
|
1021
|
+
* Bound: O(1) per pick (modulo + table read + bounded cap-aware probe), 0 B/op on BOTH `pick()` and
|
|
1022
|
+
* `note()` (torture + PerfGate). Fails closed (PICK_NONE) ONLY when no eligible backend is reachable
|
|
1023
|
+
* within the probe window (M8's contract) -- NEVER merely because backends are over cap.
|
|
1024
|
+
*/
|
|
1025
|
+
export class BoundedLoadBalancer extends ConsistentHashBalancer {
|
|
1026
|
+
/**
|
|
1027
|
+
* @param {number} capacity backend count (fixed; add/remove is a cold rebuild).
|
|
1028
|
+
* @param {Uint8Array} eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
1029
|
+
* @param {Uint32Array} inflight per-backend OCCUPANCY (length >= capacity), caller-owned and read
|
|
1030
|
+
* LIVE -- but mutated EXCLUSIVELY via note() / the /pool adapter (direct mutation desyncs the
|
|
1031
|
+
* owned _total -- UB).
|
|
1032
|
+
* @param {number} [eps=0.25] the bounded-load slack over the mean: finite, > 0. cap =
|
|
1033
|
+
* (1 + eps) x mean occupancy. Default 0.25 (Vimeo).
|
|
1034
|
+
* @param {Uint32Array|null} [weights=null] optional per-backend weights (COPIED); null = equal.
|
|
1035
|
+
* @param {number} [m=CH_DEFAULT_M] the Maglev table size: a prime, > 1, and >= capacity.
|
|
1036
|
+
* @param {number} [seed=0x9e3779b9] deterministic salt for the permutation mix (reproducible).
|
|
1037
|
+
*/
|
|
1038
|
+
constructor(capacity, eligible, inflight, eps = 0.25, weights = null, m = CH_DEFAULT_M, seed = 0x9e3779b9) {
|
|
1039
|
+
// Validate inflight + eps typeof-first, BEFORE super() allocates the (cold, ~256KB) Maglev
|
|
1040
|
+
// table (fail closed early -- the PeakEWMA / ConsistentHash precedent). These read the args
|
|
1041
|
+
// only (no `this`), so they may run before super().
|
|
1042
|
+
if (!(inflight instanceof Uint32Array) || inflight.length < capacity) {
|
|
1043
|
+
throw new RangeError('[lite-pick] inflight must be a Uint32Array of length >= capacity');
|
|
1044
|
+
}
|
|
1045
|
+
if (typeof eps !== 'number') {
|
|
1046
|
+
throw new TypeError('[lite-pick] eps must be a number');
|
|
1047
|
+
}
|
|
1048
|
+
if (!Number.isFinite(eps) || eps <= 0) {
|
|
1049
|
+
throw new RangeError('[lite-pick] eps must be a finite number > 0');
|
|
1050
|
+
}
|
|
1051
|
+
// super() validates capacity/eligible/weights/m, copies weights, and builds the Maglev table.
|
|
1052
|
+
super(capacity, eligible, weights, m, seed);
|
|
1053
|
+
this._inflight = inflight;
|
|
1054
|
+
this._eps = eps;
|
|
1055
|
+
// The running occupancy sum the balancer OWNS. Starts at 0: note() is its sole writer, so a
|
|
1056
|
+
// caller must drive dispatch/settle through note() (or /pool) -- inflight seeded non-zero
|
|
1057
|
+
// BEFORE construction would desync it (UB, as documented).
|
|
1058
|
+
this._total = 0;
|
|
1059
|
+
}
|
|
1060
|
+
|
|
1061
|
+
/** The balancer-owned running sum of in-flight the mean/cap is computed from. Readonly. */
|
|
1062
|
+
get totalInflight() {
|
|
1063
|
+
return this._total;
|
|
1064
|
+
}
|
|
1065
|
+
|
|
1066
|
+
/**
|
|
1067
|
+
* Warm feedback path (NOT the hot pick path): adjust the owned running occupancy sum by `delta`
|
|
1068
|
+
* for backend `i`. This is the SOLE writer of `_total`: dispatch is note(i, +1), settle is
|
|
1069
|
+
* note(i, -1), so the cap's mean stays O(1)-current without scanning inflight. `i` is validated in
|
|
1070
|
+
* range (like setEligible); `delta` is validated typeof-first as an integer. `_total` clamps at 0
|
|
1071
|
+
* (an over-decrement never drives the mean negative). Zero-alloc on the success path.
|
|
1072
|
+
* @param {number} i backend index (validated in range)
|
|
1073
|
+
* @param {number} delta integer occupancy change (+1 dispatch, -1 settle)
|
|
1074
|
+
*/
|
|
1075
|
+
note(i, delta) {
|
|
1076
|
+
if (i < 0 || i >= this._cap) throw new RangeError('[lite-pick] index out of range: ' + i);
|
|
1077
|
+
if (typeof delta !== 'number') throw new TypeError('[lite-pick] delta must be a number');
|
|
1078
|
+
if (!Number.isInteger(delta)) throw new RangeError('[lite-pick] delta must be an integer: ' + delta);
|
|
1079
|
+
const t = this._total + delta;
|
|
1080
|
+
this._total = t > 0 ? t : 0; // clamp: over-decrement never drives the mean negative
|
|
1081
|
+
}
|
|
1082
|
+
|
|
1083
|
+
/**
|
|
1084
|
+
* Map an INTEGER key to a backend, honouring the occupancy cap, or PICK_NONE (fail closed). O(1),
|
|
1085
|
+
* 0 B/op, never throws. slot = (keyHash >>> 0) % M; walk the M8 probe window (home + CH_PROBE_LIMIT
|
|
1086
|
+
* slots) and return the FIRST backend that is ELIGIBLE AND under cap = (1+eps) x _total / live. If
|
|
1087
|
+
* none in the window is under cap, fall back to the FIRST eligible seen (sticky wins -- the cap is
|
|
1088
|
+
* a soft preference, never a dead pick). `_total === 0` skips the cap test -> pure ConsistentHash.
|
|
1089
|
+
* PICK_NONE ONLY when no eligible backend is reachable within the window.
|
|
1090
|
+
* @param {number} keyHash a caller-supplied integer key hash (coerced to uint32)
|
|
1091
|
+
* @returns {number}
|
|
1092
|
+
*/
|
|
1093
|
+
pick(keyHash) {
|
|
1094
|
+
if (this._live === 0) return PICK_NONE; // whole pool down: fail closed
|
|
1095
|
+
const M = this._m, el = this._eligible, lookup = this._lookup, inf = this._inflight;
|
|
1096
|
+
const total = this._total;
|
|
1097
|
+
// cap is only meaningful once occupancy is known; _total === 0 -> pure ConsistentHash.
|
|
1098
|
+
const capActive = total > 0;
|
|
1099
|
+
const cap = capActive ? (1 + this._eps) * total / this._live : 0; // finite: total>0, live>0
|
|
1100
|
+
let slot = (keyHash >>> 0) % M; // integer key; NaN >>> 0 = 0 (never throws)
|
|
1101
|
+
let firstEligible = -1; // the pure-ConsistentHash sticky fallback answer
|
|
1102
|
+
let i = lookup[slot];
|
|
1103
|
+
if (el[i]) {
|
|
1104
|
+
if (!capActive || inf[i] < cap) return i; // sticky home, under cap: the common fast path
|
|
1105
|
+
firstEligible = i;
|
|
1106
|
+
}
|
|
1107
|
+
// Bounded forward-probe (M8's exact walk): the first eligible AND under-cap backend wins; a hot
|
|
1108
|
+
// home OVERFLOWS to its neighbours. Past the window we fall back to the sticky first-eligible.
|
|
1109
|
+
for (let p = 0; p < CH_PROBE_LIMIT; p++) {
|
|
1110
|
+
slot++;
|
|
1111
|
+
if (slot >= M) slot = 0;
|
|
1112
|
+
i = lookup[slot];
|
|
1113
|
+
if (el[i]) {
|
|
1114
|
+
if (!capActive || inf[i] < cap) return i; // eligible + under cap: overflow target
|
|
1115
|
+
if (firstEligible < 0) firstEligible = i; // remember the first eligible (fallback)
|
|
1116
|
+
}
|
|
1117
|
+
}
|
|
1118
|
+
return firstEligible; // -1 (PICK_NONE) iff NO eligible backend was reachable in the window
|
|
1119
|
+
}
|
|
1120
|
+
}
|
package/Pool.d.ts
CHANGED
|
@@ -10,11 +10,14 @@ export const VERSION: string;
|
|
|
10
10
|
|
|
11
11
|
/** The minimal balancer shape Pool drives (any lite-pick strategy satisfies it). */
|
|
12
12
|
export interface Balancer {
|
|
13
|
-
|
|
13
|
+
/** `now` (PeakEwma clock) or `keyHash` (ConsistentHash/BoundedLoad) when supplied via opts. */
|
|
14
|
+
pick(arg?: number): number;
|
|
14
15
|
readonly capacity: number;
|
|
15
16
|
readonly live: number;
|
|
16
17
|
/** Optional latency-feedback sink (PeakEwmaBalancer); fed on settle when a clock is supplied. */
|
|
17
18
|
recordRtt?(i: number, sampleNs: number, now: number): void;
|
|
19
|
+
/** Optional occupancy sink (BoundedLoadBalancer); fed +1 on dispatch, -1 on settle. */
|
|
20
|
+
note?(i: number, delta: number): void;
|
|
18
21
|
}
|
|
19
22
|
|
|
20
23
|
/** Options for `Pool.run`. */
|
|
@@ -28,6 +31,12 @@ export interface RunOptions {
|
|
|
28
31
|
* `recordRtt` latency feedback for a latency-aware balancer (PeakEwma); otherwise inert.
|
|
29
32
|
*/
|
|
30
33
|
clock?: () => number;
|
|
34
|
+
/**
|
|
35
|
+
* An integer routing key for a keyed balancer (ConsistentHash / BoundedLoad). When present,
|
|
36
|
+
* `run` calls `pick(key)`; the opt-in `note` occupancy hook is driven on dispatch/settle for
|
|
37
|
+
* a bounded-load balancer. Ignored by non-keyed strategies.
|
|
38
|
+
*/
|
|
39
|
+
key?: number;
|
|
31
40
|
}
|
|
32
41
|
|
|
33
42
|
/**
|
package/Pool.js
CHANGED
|
@@ -77,13 +77,27 @@ export class Pool {
|
|
|
77
77
|
* Pool stays generic, the in-flight counter stays net-zero, and abort/failover are unaffected.
|
|
78
78
|
* The kernel `pick()` remains 0 B/op; this wrapper is not held to that bar.
|
|
79
79
|
*
|
|
80
|
+
* An OCCUPANCY-AWARE balancer (BoundedLoadBalancer -- anything duck-typing `note`) has each
|
|
81
|
+
* dispatch mirrored as `note(i, +1)` and each settle as `note(i, -1)`, so its owned running mean
|
|
82
|
+
* stays current; against a balancer with no `note` the hook is fully INERT (same net-zero,
|
|
83
|
+
* generic behaviour). The two hooks are independent -- a balancer may duck-type neither, one, or
|
|
84
|
+
* both.
|
|
85
|
+
*
|
|
86
|
+
* A KEYED balancer (ConsistentHashBalancer / BoundedLoadBalancer -- CHBL) routes by an INTEGER
|
|
87
|
+
* key. When `opts.key` is supplied, Pool drives `pick(key)` (sticky / bounded-load routing);
|
|
88
|
+
* failover re-picks with the SAME key, and because the failed backend's occupancy stays elevated
|
|
89
|
+
* (its `note(+1)` held across attempts) a CHBL re-pick naturally OVERFLOWS to the next backend.
|
|
90
|
+
* Without `opts.key`, `pick()` / `pick(now)` behaviour is unchanged.
|
|
91
|
+
*
|
|
80
92
|
* @template T
|
|
81
93
|
* @param {(endpoint: number, signal?: AbortSignal) => (Promise<T>|T)} fn the per-endpoint work.
|
|
82
|
-
* @param {{ signal?: AbortSignal, tries?: number, clock?: () => number }} [opts]
|
|
83
|
-
* (default 1 = no failover) is the max number of distinct-endpoint attempts; `signal` is
|
|
94
|
+
* @param {{ signal?: AbortSignal, tries?: number, clock?: () => number, key?: number }} [opts]
|
|
95
|
+
* `tries` (default 1 = no failover) is the max number of distinct-endpoint attempts; `signal` is
|
|
84
96
|
* passed to `fn` and, when already aborted after a failure, stops failover (the abort
|
|
85
97
|
* propagates, no re-pick); `clock` is a caller-owned nanosecond source that, when present,
|
|
86
|
-
* drives `pick(now)` and the opt-in `recordRtt` latency feedback for a latency-aware balancer
|
|
98
|
+
* drives `pick(now)` and the opt-in `recordRtt` latency feedback for a latency-aware balancer;
|
|
99
|
+
* `key` is a caller-supplied INTEGER key that, when present, drives `pick(key)` for a keyed
|
|
100
|
+
* balancer (sticky / CHBL routing).
|
|
87
101
|
* @returns {Promise<T>}
|
|
88
102
|
*/
|
|
89
103
|
async run(fn, opts) {
|
|
@@ -92,16 +106,27 @@ export class Pool {
|
|
|
92
106
|
const tries = rawTries > 0 ? rawTries : 1;
|
|
93
107
|
const signal = opts ? opts.signal : undefined;
|
|
94
108
|
const clock = opts && typeof opts.clock === 'function' ? opts.clock : undefined;
|
|
109
|
+
// A keyed balancer (ConsistentHash / BoundedLoad -- CHBL) picks by an INTEGER key. When
|
|
110
|
+
// `opts.key` is supplied, pick(key) drives selection; otherwise the existing pick()/pick(now)
|
|
111
|
+
// behaviour is unchanged. `keyed` is true exactly when a key was passed.
|
|
112
|
+
const keyed = opts !== undefined && opts.key !== undefined;
|
|
113
|
+
const key = keyed ? opts.key : undefined;
|
|
95
114
|
const inflight = this._inflight, b = this._b;
|
|
96
115
|
// Opt-in latency feedback: only when BOTH a clock is supplied AND the balancer duck-types
|
|
97
116
|
// recordRtt. Otherwise inert -- Pool stays generic and byte-for-byte behaviour is unchanged.
|
|
98
117
|
const rtt = clock !== undefined && typeof b.recordRtt === 'function';
|
|
118
|
+
// Opt-in occupancy feedback (BoundedLoadBalancer): when the balancer duck-types note(), Pool
|
|
119
|
+
// mirrors each dispatch(+1)/settle(-1) into it so the balancer's owned _total mean stays
|
|
120
|
+
// O(1)-current. Otherwise inert -- Pool stays generic, in-flight stays net-zero, and
|
|
121
|
+
// abort/failover are unchanged. Follows the exact opt-in shape the recordRtt hook uses.
|
|
122
|
+
const notes = typeof b.note === 'function';
|
|
99
123
|
const held = []; // endpoints incremented this run (kept elevated across failover)
|
|
100
124
|
let lastErr;
|
|
101
125
|
try {
|
|
102
126
|
for (let attempt = 0; attempt < tries; attempt++) {
|
|
103
127
|
const now = clock !== undefined ? clock() : undefined;
|
|
104
|
-
|
|
128
|
+
// A keyed pick (opts.key) takes precedence -- sticky/CHBL routing; else pick(now)/pick().
|
|
129
|
+
const i = keyed ? b.pick(key) : b.pick(now);
|
|
105
130
|
if (i === PICK_NONE) {
|
|
106
131
|
if (attempt === 0) {
|
|
107
132
|
const e = new Error('[lite-pick] no eligible endpoint');
|
|
@@ -112,6 +137,7 @@ export class Pool {
|
|
|
112
137
|
}
|
|
113
138
|
inflight[i] = (inflight[i] + 1) >>> 0;
|
|
114
139
|
held.push(i);
|
|
140
|
+
if (notes) b.note(i, 1); // mirror the dispatch into the balancer's occupancy sum
|
|
115
141
|
try {
|
|
116
142
|
const out = await fn(i, signal);
|
|
117
143
|
if (rtt) { // successful settle: feed the measured rtt back to the balancer
|
|
@@ -130,6 +156,7 @@ export class Pool {
|
|
|
130
156
|
for (let k = 0; k < held.length; k++) {
|
|
131
157
|
const j = held[k];
|
|
132
158
|
inflight[j] = inflight[j] > 0 ? inflight[j] - 1 : 0;
|
|
159
|
+
if (notes) b.note(j, -1); // net-zero settle -- keeps _total in lockstep per run
|
|
133
160
|
}
|
|
134
161
|
}
|
|
135
162
|
}
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v0.
|
|
3
|
+
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v0.9.0 ships nine strategies -- `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, the latency-aware `PeakEwmaBalancer`, the sticky/affinity `ConsistentHashBalancer` (a Maglev lookup table), and the hotspot-protecting `BoundedLoadBalancer` (consistent hashing with bounded loads -- sticky routing + an occupancy cap that overflows a hot backend to its neighbours)** -- on the substrate seams (`VERSION`, `PICK_NONE`, a deterministic `Prng`, and `BalancerBase`'s shared read-only eligibility view), plus a **`@zakkster/lite-pick/pool`** subpath: the async dispatch/settle counter layer with distinct-endpoint failover and a duck-typed query-cache fetcher. The rest of the roster -- WeightedRandom -- lands next session.
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/@zakkster/lite-pick)
|
|
6
6
|
[](https://github.com/sponsors/PeshoVurtoleta)
|
|
@@ -21,7 +21,7 @@ The npm landscape has old algorithm libraries (`load-balancers`, `loadbalance`,
|
|
|
21
21
|
- **Two pieces of evidence, both shipped.** A **0 B/op** witness on the pick path (no object, closure, string, or array created per pick), and a measured **balance-quality anchor** -- peak-to-average load within the strategy's theoretical ceiling (for P2C, the Azar-Broder-Karlin-Upfal `ln ln n / ln 2` bound) and strictly better than a random foil.
|
|
22
22
|
- **A pure selector, not a proxy.** It **consumes** health and circuit state; it never owns them. Health is a shared read-only bitmap written by [`@zakkster/lite-di-health`](https://www.npmjs.com/package/@zakkster/lite-di-health); circuit state comes from [`@zakkster/lite-statechart`](https://www.npmjs.com/package/@zakkster/lite-statechart); load counters are caller-owned typed arrays. `pick()` only reads.
|
|
23
23
|
|
|
24
|
-
> **Status:
|
|
24
|
+
> **Status: M9 (v0.9.0).** Ships the substrate seams **plus `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, the latency-aware `PeakEwmaBalancer`, the sticky/affinity `ConsistentHashBalancer` (a Maglev table), and the hotspot-protecting `BoundedLoadBalancer` (consistent hashing with bounded loads)**, the **`@zakkster/lite-pick/pool`** request layer (with opt-in latency-feedback, occupancy-feedback, and keyed-routing hooks), and the **benchmark suite** -- the balance anchor + GC blast-radius headlines, a seeded/version-stamped `results.json`, a `bench:verify` drift check with teeth, and the vs-AWS positioning (see *Evidence* below). This session APPENDS one class: the other strategies in `Pick.js` are byte-identical, only the header roster/count and the `VERSION` stamp change. Every strategy is gated: `pick()` proven **0 B/op** (torture + PerfGate), RoundRobin **perfectly fair** with **zero dead picks** vs the naive `i++ % n` foil, SmoothWRR **exactly weighted** and **smooth**, **P2C proves the `ln ln n` balance ceiling** (peak-to-mean gap ~2 vs a random foil's ~21 at n=1024), **LeastConn is greedy-perfect** (max-minus-min load <= 1), **SED tracks weight within 1%**, **PeakEWMA steers around a 10x-slow node** (it takes <= 25% of P2C's share for it and cuts service p99), **ConsistentHash remaps only ~1.6% of keys on a scale event** (vs ~98% for naive modulo), and **BoundedLoad tames a hotspot plain consistent hashing can't** (under a skewed key stream ConsistentHash spikes a hot backend to ~13x the mean occupancy while BoundedLoad's `(1+eps)` cap holds it near the mean by overflowing to neighbours -- keeping the same ~1.6% minimal disruption) -- all held under a **seeded invariant fuzzer** (`test/fuzz.mjs`) that checks state-synchronisation after *every* op. See [ROADMAP.md](./ROADMAP.md) for the M9 -> M10 path to 1.0.0, and [decisions/](./decisions) for the ownership boundary (ADR 0001), anti-flapping (ADR 0002), the RoundRobin (0003), SmoothWRR (0004), P2C (0005), LeastConn-family (0006), pool-adapter (0007), benchmark-suite (0008), PeakEWMA (0009), ConsistentHash (0010), and BoundedLoad/CHBL (0011) design forks.
|
|
25
25
|
|
|
26
26
|
```bash
|
|
27
27
|
npm install @zakkster/lite-pick
|
|
@@ -180,6 +180,64 @@ PeakEWMA sends the slow node **<= 25% of P2C's share** for it and cuts service p
|
|
|
180
180
|
|
|
181
181
|
For a **front-end / browser client** -- a handful of picks per second across origins/regions, not a zero-GC hot loop -- the recommended profile is **PeakEWMA + the eligibility bitmap only**: latency-aware choice with a fail-closed health view, and **none** of the server-side bounded-load / availability-zone / occupancy machinery. It is the smallest honest latency-aware client balancer. *(Deferred: a tail-aware `inflight x p99Rtt` variant via an optional-peer `@zakkster/lite-sketch` `DDSketch`; the EWMA-mean score is the shipped zero-peer default, and `peerDependencies` stays `{}` -- [ADR 0009](./decisions/0009-peakewma.md).)*
|
|
182
182
|
|
|
183
|
+
## ConsistentHash -- sticky / cache-affinity routing (v0.8.0)
|
|
184
|
+
|
|
185
|
+
When a request must go to the **same** backend every time -- a session, a cache shard, a stateful worker -- you want **consistent hashing**: a stable key -> backend map that barely changes when the pool scales. `ConsistentHashBalancer` is a prebuilt **Maglev lookup table** (the in-kernel/production choice -- Linux IPVS `mh`, Meta Katran, Cilium), so `pick(keyHash)` is `slot = keyHash % M`, one table read, and a bounded probe past down slots -- **O(1)** and **0 B/op** ([ADR 0010](./decisions/0010-consistenthash.md)).
|
|
186
|
+
|
|
187
|
+
```js
|
|
188
|
+
import { ConsistentHashBalancer } from '@zakkster/lite-pick';
|
|
189
|
+
|
|
190
|
+
const eligible = Uint8Array.from([1, 1, 1, 1]);
|
|
191
|
+
// YOU hash the key to an INTEGER (cold) -- per-pick STRING hashing is the one zero-GC hazard.
|
|
192
|
+
const b = new ConsistentHashBalancer(4, eligible); // default M = 65537 (prime); weights optional
|
|
193
|
+
|
|
194
|
+
const key = fnv1a(sessionId); // any integer hash you like -- lite-pick adds none
|
|
195
|
+
const i = b.pick(key >>> 0); // same key -> same backend, at fixed membership
|
|
196
|
+
|
|
197
|
+
// A tiny FNV-1a over a string, done ONCE per key on the cold path (never inside pick):
|
|
198
|
+
function fnv1a(s) { let h = 0x811c9dc5; for (let k = 0; k < s.length; k++) { h ^= s.charCodeAt(k); h = Math.imul(h, 0x01000193); } return h >>> 0; }
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
- **Caller-supplied INTEGER key.** `pick(keyHash)` coerces `keyHash >>> 0` (so `NaN`/`undefined` -> `0`) and **never throws** (the fail-closed contract). It never hashes a string -- that would allocate on the hot path. Hash string keys yourself, cold (a tiny FNV-1a is fine -- see the snippet above); `lite-pick` adds **no hashing dependency**.
|
|
202
|
+
- **Minimal disruption.** Removing a backend is just `setEligible(i, false)` -- the table is **untouched**, so every key not on that backend keeps its exact backend and only ~`1/N` reroute (measured **1.6%** vs the naive-modulo trap's **98%** above). A membership or weight change rebuilds the table (**cold**); a health flap **never** does -- the bounded probe (<= 64 slots) absorbs it.
|
|
203
|
+
- **Weighted.** Pass a `Uint32Array` of weights (copied, balancer-owned) for a per-backend slot share proportional to weight; `setWeight(i, w)` / `rebuild()` rebuild the table cold. Unweighted = equal share.
|
|
204
|
+
- **Cost & bound.** The lookup table is `M x 4` bytes -- `~256KB` at the `65537` default -- a **cold, one-time** allocation (disclosed in the cost table below; `M` is configurable **down** for small pools). `pick()` is `O(1)`, `0 B/op`. Fail-closed: `PICK_NONE` when the pool is down or no eligible backend is reachable within the probe bound (a near-total outage may return `PICK_NONE` even if a far eligible slot exists -- safe, never a dead pick).
|
|
205
|
+
- **Deferred seams (import nothing).** A `@zakkster/lite-filter` hot-key / known-key oracle at the key-routing layer (warm/cold only, never the pick path) and a `@zakkster/lite-o1` `EliasFano` ring alternative to the table are optional-peer seams -- `peerDependencies` **stays `{}`** until a shipped path imports one ([ADR 0010](./decisions/0010-consistenthash.md)).
|
|
206
|
+
|
|
207
|
+
## BoundedLoad -- consistent hashing with bounded loads (v0.9.0)
|
|
208
|
+
|
|
209
|
+
Plain consistent hashing is sticky and minimally-disruptive, but it has one failure mode: a **hot key**. If a handful of keys carry most of the traffic, consistent hashing pins each one's *entire* load on its one hashed backend -- an unbounded **hotspot**. `BoundedLoadBalancer` is [`ConsistentHashBalancer`](#consistenthash--sticky--cache-affinity-routing-v080) (the Maglev table) **plus a per-backend occupancy cap** `cap = (1 + eps) x mean` (the mean occupancy `_total / live`, with slack `eps`): a key sticks to its hashed home **unless** that backend is over cap, in which case the request **overflows** along the same bounded probe to the next eligible, under-cap backend (Mirrokni et al. *Consistent Hashing with Bounded Loads*, Google Research; Vimeo's `eps = 0.25` -- [ADR 0011](./decisions/0011-boundedload.md)). You keep stickiness + minimal disruption **and** gain the hotspot protection consistent hashing lacks.
|
|
210
|
+
|
|
211
|
+
```js
|
|
212
|
+
import { BoundedLoadBalancer } from '@zakkster/lite-pick';
|
|
213
|
+
|
|
214
|
+
const eligible = Uint8Array.from([1, 1, 1, 1]);
|
|
215
|
+
const inflight = new Uint32Array(4); // YOU own this; read live as per-backend OCCUPANCY
|
|
216
|
+
|
|
217
|
+
// eps = 0.25 -> a backend over 1.25x the mean occupancy overflows the key to a neighbour.
|
|
218
|
+
const bl = new BoundedLoadBalancer(4, eligible, inflight, 0.25);
|
|
219
|
+
|
|
220
|
+
const key = fnv1a(sessionId); // any integer hash (cold) -- lite-pick adds none
|
|
221
|
+
const i = bl.pick(key >>> 0); // sticky home, or the overflow target if it's hot
|
|
222
|
+
inflight[i]++; bl.note(i, +1); // dispatch: bump the counter AND tell the balancer
|
|
223
|
+
// ... await the request ...
|
|
224
|
+
inflight[i]--; bl.note(i, -1); // settle: net-zero on both
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
- **Sticky + overflow.** `pick(keyHash)` maps the integer key to its Maglev home; if that backend is under cap it wins (the common, sticky path). If it is over cap, the request overflows along the bounded probe to the first eligible, under-cap backend. If nothing in the window is under cap, it falls back to the first eligible seen -- **sticky wins; the cap is a soft preference, never a dead pick**. When `_total === 0` the cap is skipped entirely, so it behaves as pure `ConsistentHashBalancer`.
|
|
228
|
+
- **`note()` is the sole writer of the mean.** BoundedLoad **owns** a running occupancy sum `_total` and keeps it O(1)-current through `note(i, +1)` on dispatch / `note(i, -1)` on settle -- so the cap's mean never needs a scan; `inflight` is your live-read per-backend occupancy. **Contract:** mutate the mirrored counter **only** through `note()` (or the `/pool` adapter, which does it for you) -- direct mutation desyncs `_total` (UB, the same asymmetry `SmoothWRRBalancer` has for its weights). `note()` clamps `_total` at 0, and `totalInflight` exposes it.
|
|
229
|
+
- **Inherits the Maglev table.** It extends `ConsistentHashBalancer`, so `setWeight(i, w)` / `rebuild()` / `tableSize` and the whole weighted-Maglev build + bounded-probe walk are reused verbatim; the `weights` / `m` / `seed` constructor args are the same. `pick()` and `note()` are both **0 B/op**, `O(1)`. `PICK_NONE` only when no eligible backend is reachable in the probe window -- never merely because backends are over cap.
|
|
230
|
+
- **Why not "P2C with a cap"?** A note on the design (the honest one): power-of-two-choices over in-flight *plus* a `(1+eps) x mean` cap is **byte-identical to plain P2C** -- an under-cap draw always has lower in-flight than an over-cap one, so "prefer under-cap" and "lower-of-two" pick the same node. The cap is a no-op there. It is only *load-bearing* when the primary choice is fixed by something other than load -- a **hash**. That is CHBL, and it is why BoundedLoad is built on consistent hashing ([ADR 0011](./decisions/0011-boundedload.md)).
|
|
231
|
+
|
|
232
|
+
The proof (from `test/balance.mjs`, a Zipfian-skewed key stream over 64 backends, one fixed concurrency window):
|
|
233
|
+
|
|
234
|
+
| lane | mean occupancy | max backend occupancy |
|
|
235
|
+
|---|---|---|
|
|
236
|
+
| **BoundedLoad (CHBL)** | 10 | **13** (cap = 12.5 -- overflow holds it near the mean) |
|
|
237
|
+
| ConsistentHash (no cap) | 10 | **129** (~13x -- the hotspot) |
|
|
238
|
+
|
|
239
|
+
BoundedLoad caps the hot backend near `(1 + eps) x mean` while plain consistent hashing lets it run away, and both reroute only **~1.6%** of keys on a scale event (`test/balance.mjs`). See [`ConsistentHashBalancer`](#consistenthash--sticky--cache-affinity-routing-v080) above for the integer-key contract and the FNV-1a helper.
|
|
240
|
+
|
|
183
241
|
## Evidence -- the two headlines (v0.6.0 benchmark suite)
|
|
184
242
|
|
|
185
243
|
> **Framing: parity on speed, superiority on the contract + balance + tail.** A trivial `i++ % n` round-robin -- or `wrr` -- *matches* P2C on raw ops/sec, so `lite-pick` does **not** claim "N times faster." Throughput is claimed at **parity**; the wins are **zero-GC**, **balance quality**, **tail latency** (GC blast-radius), and **never a dead pick**. Every number below is **seeded** and regenerated by `npm run bench:report`; `npm run bench:verify` fails CI if a README number drifts from a fresh run (algorithmic exact, timing within +/-15%). Node / CPU / OS / every PRNG seed are stamped into `benchmark/results.json`.
|
|
@@ -192,9 +250,9 @@ The real pinned npm incumbents (`load-balancers`, `loadbalance`, `wrr`) run thro
|
|
|
192
250
|
|
|
193
251
|
| family | lite-pick | lite-pick ops/ms | incumbent (npm) | incumbent ops/ms |
|
|
194
252
|
| --- | --- | --- | --- | --- |
|
|
195
|
-
| P2C (power-of-two-choices) | P2cBalancer |
|
|
196
|
-
| RoundRobin | RoundRobinBalancer |
|
|
197
|
-
| Weighted-random | WeightedRandom -- SKIP, ships M10 | -- | wrr@1.0.0 |
|
|
253
|
+
| P2C (power-of-two-choices) | P2cBalancer | 53009 | load-balancers@1.3.52 | 59485 |
|
|
254
|
+
| RoundRobin | RoundRobinBalancer | 246432 | loadbalance@1.0.0 | 303459 |
|
|
255
|
+
| Weighted-random | WeightedRandom -- SKIP, ships M10 | -- | wrr@1.0.0 | 163481 |
|
|
198
256
|
|
|
199
257
|
<!-- /bench:competitors -->
|
|
200
258
|
|
|
@@ -222,8 +280,8 @@ The point of zero-GC is **not** the pick's own latency -- a major GC pause freez
|
|
|
222
280
|
|
|
223
281
|
| lane | major GC | pick B/op | max GC pause (ms) |
|
|
224
282
|
| --- | --- | --- | --- |
|
|
225
|
-
| lite-pick | 0 | 0 | 0.
|
|
226
|
-
| allocating foil | 13 | allocates |
|
|
283
|
+
| lite-pick | 0 | 0 | 0.1 |
|
|
284
|
+
| allocating foil | 13 | allocates | 2.9 |
|
|
227
285
|
|
|
228
286
|
<!-- /bench:gc -->
|
|
229
287
|
|
|
@@ -231,15 +289,14 @@ The `lite-pick` lane holds **0 major GC / 0 B/op** on the pick path; the allocat
|
|
|
231
289
|
|
|
232
290
|
### Consistent-hash disruption -- a trust gate
|
|
233
291
|
|
|
234
|
-
On a scale event (add / remove a node), what fraction of keys keep their node? The **naive-modulo** trap (`key % n`) reshuffles almost everything -- blowing every downstream cache -- while a good consistent hash moves only ~`1/n`.
|
|
292
|
+
On a scale event (add / remove a node), what fraction of keys keep their node? The **naive-modulo** trap (`key % n`) reshuffles almost everything -- blowing every downstream cache -- while a good consistent hash moves only ~`1/n`. `ConsistentHashBalancer` (the Maglev table, **M8**) is measured here beside the trap and the `1/n` ideal -- removing a backend is just marking it down, so only its keys reroute:
|
|
235
293
|
|
|
236
294
|
<!-- bench:disruption -->
|
|
237
295
|
|
|
238
|
-
| scale event | naive-modulo remap |
|
|
239
|
-
| --- | --- | --- |
|
|
240
|
-
| node removed | 98.4% | 1.6% |
|
|
241
|
-
| node added | 98.5% | 1.5% |
|
|
242
|
-
| ConsistentHash (Maglev) | SKIP -- ships in a later milestone | -- |
|
|
296
|
+
| scale event | naive-modulo remap | ConsistentHash (Maglev) | ideal (1/n) |
|
|
297
|
+
| --- | --- | --- | --- |
|
|
298
|
+
| node removed | 98.4% | 1.6% | 1.6% |
|
|
299
|
+
| node added | 98.5% | 1.5% | 1.5% |
|
|
243
300
|
|
|
244
301
|
<!-- /bench:disruption -->
|
|
245
302
|
|
|
@@ -250,6 +307,7 @@ On a scale event (add / remove a node), what fraction of keys keep their node? T
|
|
|
250
307
|
| operation | when | allocates |
|
|
251
308
|
| --- | --- | --- |
|
|
252
309
|
| `new <Strategy>Balancer(...)` | construction, once | the balancer object + its owned accumulators (SmoothWRR's Float64 `current`). The eligibility / inflight / weight views are **caller-owned**, never copied |
|
|
310
|
+
| `new ConsistentHashBalancer(...)` / `rebuild()` / `setWeight()` | cold, on build / membership / reweight | the Maglev lookup table: **`M x 4` bytes** (`~256KB` at the `65537` default `M`), a one-time `Uint32Array` allocation + an `O(M x N)` populate. `M` is **configurable down** for small pools. A health flap does **not** rebuild -- the bounded probe absorbs it |
|
|
253
311
|
| `setEligible(i, up)` | cold, on a health flip | **0** -- one byte write + an O(1) live-count adjust |
|
|
254
312
|
| `setWeight(i, w)` (SmoothWRR) | cold, on reweight | **0** -- one array write + an O(1) eligible-total adjust |
|
|
255
313
|
| `pick()` | **HOT**, per request | **0 B/op** -- proven by `torture` + `test:perf`, measured by `bench:gc` |
|
|
@@ -263,8 +321,9 @@ On a scale event (add / remove a node), what fraction of keys keep their node? T
|
|
|
263
321
|
| --- | --- |
|
|
264
322
|
| ALB `least_outstanding_requests` (LOR) | `LeastConnBalancer` / `P2cBalancer` |
|
|
265
323
|
| ALB anomaly mitigation / latency-aware shedding | `PeakEwmaBalancer` (latency-aware P2C) |
|
|
266
|
-
| ALB
|
|
267
|
-
|
|
|
324
|
+
| ALB anomaly mitigation on a sticky/affinity hash | `BoundedLoadBalancer` (consistent hashing with bounded loads -- sticky + hotspot overflow, M9) |
|
|
325
|
+
| ALB `weighted_random` | `WeightedRandom` (M10) |
|
|
326
|
+
| NLB flow-hash (5-tuple) | `ConsistentHashBalancer` (Maglev table -- the same family NLB flow-hash uses, at the in-process hop) |
|
|
268
327
|
|
|
269
328
|
The composition: inbound traffic still enters through your **ALB/NLB -> service** (the edge hop AWS owns and bills); `lite-pick` governs the fan-out **after** that, the hop no AWS load balancer touches. Complementary, not a replacement -- "the hop your ALB/NLB never sees."
|
|
270
329
|
|
|
@@ -293,10 +352,10 @@ rng.nextBelow(4); // -> a uint32 in [0, 4)
|
|
|
293
352
|
rng.reset(); // replays the exact stream
|
|
294
353
|
|
|
295
354
|
PICK_NONE; // -> -1 (fail-closed sentinel: no endpoint, never a dead pick)
|
|
296
|
-
VERSION; // -> '0.
|
|
355
|
+
VERSION; // -> '0.9.0'
|
|
297
356
|
```
|
|
298
357
|
|
|
299
|
-
`BalancerBase.pick()` is **abstract** -- it throws, so an unfinished strategy fails loudly rather than returning a dead index. Every shipped strategy (`RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`) extends it and reads the same shared eligibility view; you subclass it the same way to add your own.
|
|
358
|
+
`BalancerBase.pick()` is **abstract** -- it throws, so an unfinished strategy fails loudly rather than returning a dead index. Every shipped strategy (`RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`, `ConsistentHashBalancer`, `BoundedLoadBalancer`) extends it and reads the same shared eligibility view; you subclass it the same way to add your own.
|
|
300
359
|
|
|
301
360
|
## Wiring it up -- `@zakkster/lite-pick/pool` (v0.5.0)
|
|
302
361
|
|
package/RECIPES.md
CHANGED
|
@@ -236,7 +236,51 @@ Notes:
|
|
|
236
236
|
|
|
237
237
|
---
|
|
238
238
|
|
|
239
|
-
## 9.
|
|
239
|
+
## 9. Sticky / affinity routing -- ConsistentHash (you hash the key)
|
|
240
|
+
|
|
241
|
+
When a request must land on the **same** backend every time -- a session pinned to a shard,
|
|
242
|
+
a cache key kept warm, a stateful worker -- use `ConsistentHashBalancer`. It is a prebuilt
|
|
243
|
+
**Maglev table**, so `pick(keyHash)` is `O(1)` and `0 B/op`, and scaling the pool moves only
|
|
244
|
+
~`1/N` of keys (not the whole keyspace, the way `key % n` would).
|
|
245
|
+
|
|
246
|
+
The one rule: **you hash the key to an INTEGER**, on the cold path. Per-pick *string* hashing
|
|
247
|
+
allocates -- the single zero-GC hazard -- so `pick()` takes a number and `lite-pick` ships no
|
|
248
|
+
hashing dependency. Any small integer hash works; FNV-1a is a fine default:
|
|
249
|
+
|
|
250
|
+
```js
|
|
251
|
+
import { ConsistentHashBalancer, PICK_NONE } from '@zakkster/lite-pick';
|
|
252
|
+
|
|
253
|
+
// A tiny FNV-1a over a string -- done ONCE per key, never inside pick().
|
|
254
|
+
function fnv1a(s) {
|
|
255
|
+
let h = 0x811c9dc5;
|
|
256
|
+
for (let i = 0; i < s.length; i++) { h ^= s.charCodeAt(i); h = Math.imul(h, 0x01000193); }
|
|
257
|
+
return h >>> 0;
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
const eligible = new Uint8Array(CAP).fill(1);
|
|
261
|
+
const lb = new ConsistentHashBalancer(CAP, eligible); // default M = 65537 (prime)
|
|
262
|
+
|
|
263
|
+
function routeFor(sessionId) {
|
|
264
|
+
const i = lb.pick(fnv1a(sessionId)); // same session -> same backend, at fixed membership
|
|
265
|
+
if (i === PICK_NONE) return respond503();
|
|
266
|
+
return endpoints[i];
|
|
267
|
+
}
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
Notes:
|
|
271
|
+
- **Remove a backend by marking it down** (`lb.setEligible(i, false)`) -- the table is
|
|
272
|
+
untouched, so only that backend's keys reroute (~`1/N`); everyone else stays put. A health
|
|
273
|
+
flap costs nothing (the bounded probe absorbs it) -- it never rebuilds.
|
|
274
|
+
- **Add / reweight** rebuilds the table (cold): `new ConsistentHashBalancer(N + 1, ...)`, or
|
|
275
|
+
`lb.setWeight(i, w)` / `lb.rebuild()`. Pass a `Uint32Array` of weights for proportional shares.
|
|
276
|
+
- **Cost:** the table is `M x 4` bytes (~256KB at the `65537` default) -- a cold, one-time
|
|
277
|
+
allocation. Turn `M` down for a small pool (any prime `>= N`).
|
|
278
|
+
- `pick(keyHash)` coerces `keyHash >>> 0` and never throws; it returns `PICK_NONE` only when the
|
|
279
|
+
pool is down or no eligible backend is reachable within the probe bound.
|
|
280
|
+
|
|
281
|
+
---
|
|
282
|
+
|
|
283
|
+
## 10. Wire it into a query cache (lite-query, or any fetcher)
|
|
240
284
|
|
|
241
285
|
`liteQueryFetcher` adapts a Pool into a `({ key, signal }) => Promise` fetcher -- the
|
|
242
286
|
shape lite-query (or any cache/route-loader) expects. It imports nothing from lite-query
|
|
@@ -256,7 +300,7 @@ const fetcher = liteQueryFetcher(
|
|
|
256
300
|
|
|
257
301
|
---
|
|
258
302
|
|
|
259
|
-
##
|
|
303
|
+
## 11. Choosing a strategy
|
|
260
304
|
|
|
261
305
|
| Strategy | Route by | Cost | Reach for it when |
|
|
262
306
|
|---------------|---------------------|-------------|-------------------|
|
|
@@ -267,12 +311,14 @@ const fetcher = liteQueryFetcher(
|
|
|
267
311
|
| SED | in-flight / weight | O(cap) | weighted least-conn |
|
|
268
312
|
| NQ | idle-first else SED | O(cap)/O(1) | worker pools -- never queue while a worker is free |
|
|
269
313
|
| PeakEWMA | in-flight x ewma(rtt)| O(1) | heterogeneous / flaky backends; steer around slow nodes |
|
|
314
|
+
| ConsistentHash| key hash (sticky) | O(1) | affinity/sticky: same key -> same backend, minimal disruption on scale |
|
|
270
315
|
|
|
271
|
-
|
|
316
|
+
The load-aware strategies read the SAME `inflight` array live, so you can swap among them
|
|
317
|
+
without rewiring; ConsistentHash instead takes an integer key per pick (recipe 9).
|
|
272
318
|
|
|
273
319
|
---
|
|
274
320
|
|
|
275
|
-
##
|
|
321
|
+
## 12. The "FE profile" -- a browser / front-end client
|
|
276
322
|
|
|
277
323
|
For a front-end client picking among origins a handful of times per second (not a
|
|
278
324
|
zero-GC hot loop), the recommended profile is **PeakEWMA + health/eligibility only** --
|
|
@@ -282,22 +328,24 @@ bounded-load / AZ / occupancy machinery. Feed rtt from your `fetch` timings via
|
|
|
282
328
|
|
|
283
329
|
---
|
|
284
330
|
|
|
285
|
-
##
|
|
331
|
+
## 13. Compose with the suite (all optional, all duck-typed)
|
|
286
332
|
|
|
287
333
|
lite-pick declares ZERO hard dependencies and an EMPTY `peerDependencies`. Each seam is
|
|
288
334
|
a shared TypedArray or a duck-typed shape, so you wire in a sibling only if you use it:
|
|
289
335
|
|
|
290
336
|
- `@zakkster/lite-di-health` -- writes the eligibility bitmap from health checks.
|
|
291
337
|
- `@zakkster/lite-statechart` -- a circuit breaker that flips eligibility.
|
|
292
|
-
- `@zakkster/lite-query` -- the cache behind `liteQueryFetcher` (recipe
|
|
338
|
+
- `@zakkster/lite-query` -- the cache behind `liteQueryFetcher` (recipe 10).
|
|
293
339
|
- `@zakkster/lite-sketch` -- `DDSketch` for a p99-aware PeakEWMA variant (deferred).
|
|
340
|
+
- `@zakkster/lite-filter` -- a hot-key / known-key oracle at the ConsistentHash key-routing
|
|
341
|
+
layer (warm/cold only, never the pick path; deferred).
|
|
294
342
|
- `@zakkster/lite-await` -- hedging (race the P2C second choice past a percentile).
|
|
295
343
|
|
|
296
344
|
None is required; the kernel runs over raw TypedArrays with nothing installed.
|
|
297
345
|
|
|
298
346
|
---
|
|
299
347
|
|
|
300
|
-
##
|
|
348
|
+
## 14. Zero-GC discipline (why the pick path stays 0 B/op)
|
|
301
349
|
|
|
302
350
|
- Allocate `eligible` / `inflight` / `weights` ONCE at startup and reuse them. Never
|
|
303
351
|
build arrays per pick.
|
|
@@ -310,10 +358,12 @@ None is required; the kernel runs over raw TypedArrays with nothing installed.
|
|
|
310
358
|
|
|
311
359
|
---
|
|
312
360
|
|
|
313
|
-
##
|
|
361
|
+
## 15. Gotchas
|
|
314
362
|
|
|
315
363
|
- **PICK_NONE (-1)** is always possible -- handle it before indexing (recipe 2).
|
|
316
364
|
- **SmoothWRR weights** must go through `setWeight`; direct array mutation is UB.
|
|
365
|
+
- **ConsistentHash takes an INTEGER key** -- hash strings yourself, cold (recipe 9). Never
|
|
366
|
+
`pick(someString)` on the hot path; `M` must be a prime `>= capacity`.
|
|
317
367
|
- **Load-aware strategies need the dispatch/settle loop** -- forget the `inflight--` in
|
|
318
368
|
a `finally` and load leaks upward forever. Use `/pool` (recipe 6) to avoid it.
|
|
319
369
|
- **PeakEWMA needs a finite `now`** and rtt feedback -- without `recordRtt` it behaves
|
package/llms.txt
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.9.0
|
|
4
4
|
License: MIT (c) Zahary Shinikchiev <shinikchiev@yahoo.com>
|
|
5
5
|
Runtime dependencies: none. ESM only. ASCII-only source. sideEffects: false.
|
|
6
6
|
Node: >= 18.
|
|
@@ -14,16 +14,47 @@ reading pre-allocated views that siblings or the caller write, and returning an
|
|
|
14
14
|
The complementary evidence lite-pick ships is a measured balance-quality anchor (peak-to-
|
|
15
15
|
average load vs the strategy's theoretical ceiling) alongside the 0 B/op pick witness.
|
|
16
16
|
|
|
17
|
-
0.
|
|
17
|
+
0.9.0 ships the substrate seams + nine strategies: RoundRobin, SmoothWRR (the weighted
|
|
18
18
|
default), P2C (power-of-two-choices -- also the O(1) least-connections APPROXIMATION), the
|
|
19
|
-
EXACT LeastConn family (LeastConn, SED, NQ),
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
`
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
19
|
+
EXACT LeastConn family (LeastConn, SED, NQ), PeakEWMA (latency-aware P2C), ConsistentHash
|
|
20
|
+
(a Maglev lookup table -- sticky/affinity routing), and BoundedLoad (Consistent Hashing with Bounded
|
|
21
|
+
Loads -- sticky routing + a per-backend occupancy cap that overflows a hotspot to neighbours). It
|
|
22
|
+
exports `VERSION`, the fail-closed
|
|
23
|
+
sentinel `PICK_NONE` (-1), a deterministic `Prng` (xorshift32), `BalancerBase` (the shared
|
|
24
|
+
read-only eligibility seam + O(1) live count), `RoundRobinBalancer`, `SmoothWRRBalancer`,
|
|
25
|
+
`P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`,
|
|
26
|
+
`ConsistentHashBalancer`, `BoundedLoadBalancer`, and the ConsistentHash constants `CH_DEFAULT_M`
|
|
27
|
+
(65537) / `CH_PROBE_LIMIT` (64). The remaining strategy lands next session (see ROADMAP.md):
|
|
28
|
+
WeightedRandom. The EXACT-O(log n) fewest-in-flight variant is a deferred @zakkster/lite-logn
|
|
29
|
+
`BinaryHeap` optional-peer seam (decisions/0006), not this exact-O(cap) scan.
|
|
30
|
+
|
|
31
|
+
M9 (0.9.0) adds BoundedLoadBalancer (decisions/0011): Consistent Hashing with Bounded Loads (CHBL --
|
|
32
|
+
Mirrokni et al. / Google Research; Vimeo eps=0.25). It is `ConsistentHashBalancer` (the Maglev table)
|
|
33
|
+
PLUS a per-backend occupancy cap `cap = (1 + eps) x _total / live`: `pick(keyHash)` sticks a key to its
|
|
34
|
+
hashed home UNLESS that backend is over cap, in which case the request OVERFLOWS along the same bounded
|
|
35
|
+
probe to the next eligible under-cap backend -- consistent hashing's stickiness + minimal disruption
|
|
36
|
+
PLUS the HOTSPOT protection plain CH lacks. If none in the window is under cap it FALLS BACK to the
|
|
37
|
+
first eligible (sticky wins; PICK_NONE is pool-down ONLY, never for over-cap); `_total === 0` skips the
|
|
38
|
+
cap -> pure ConsistentHash. It extends ConsistentHashBalancer (reusing the Maglev build + probe +
|
|
39
|
+
setWeight/rebuild/tableSize VERBATIM) and OWNS a running `_total` whose SOLE writer is the warm
|
|
40
|
+
`note(i, delta)` seam (dispatch +1 / settle -1); `inflight` is the caller's Uint32Array read LIVE as
|
|
41
|
+
the per-backend occupancy. `pick()` and `note()` are both O(1) / 0 B/op. CONTRACT: when using
|
|
42
|
+
BoundedLoad the mirrored counter is mutated ONLY through `note()` / /pool -- direct mutation desyncs
|
|
43
|
+
`_total` (UB, the SmoothWRR-weights asymmetry). THE PIVOT (decisions/0011): P2C-over-inflight with a
|
|
44
|
+
`(1+eps) x mean` cap is byte-identical to plain P2C (an under-cap draw always has lower inflight than an
|
|
45
|
+
over-cap one), so the cap is only LOAD-BEARING when the primary choice is a HASH -- CHBL is that. The
|
|
46
|
+
hotspot anchor (test/balance.mjs): under a skewed key stream, plain ConsistentHash spikes a hot backend
|
|
47
|
+
to ~13x the mean occupancy while CHBL caps it near (1+eps) x mean by overflow -- and both keep ~1/N
|
|
48
|
+
minimal disruption on a scale event.
|
|
49
|
+
|
|
50
|
+
M8 (0.8.0) adds ConsistentHashBalancer (decisions/0010): a prebuilt Maglev lookup table (IPVS `mh`,
|
|
51
|
+
Meta Katran, Cilium) mapping a caller-supplied INTEGER key to a backend, O(1) / 0 B/op, with minimal
|
|
52
|
+
disruption on a scale event (~1/N keys move, measured ~1.6% vs the naive-modulo foil's ~98%). The
|
|
53
|
+
vs-AWS positioning: NLB flow-hash (5-tuple) is a Maglev consistent hash -- ConsistentHashBalancer is
|
|
54
|
+
that same family at the in-process hop AWS never sees. DEFERRED optional-peer seams (import NOTHING;
|
|
55
|
+
peerDependencies STAYS `{}` until a shipped path imports one): a @zakkster/lite-filter hot-key /
|
|
56
|
+
known-key oracle (BlockedBloom etc.) at the KEY-routing layer (warm/cold only, never the pick path),
|
|
57
|
+
and a @zakkster/lite-o1 `EliasFano` ring alternative to the table.
|
|
27
58
|
|
|
28
59
|
M7 (0.7.0) adds PeakEwmaBalancer (decisions/0009). The FE PROFILE: for a browser / front-end
|
|
29
60
|
client (a handful of picks per second, not a zero-GC hot loop), the recommended profile is
|
|
@@ -150,6 +181,61 @@ the contract + balance + tail -- never an "N times faster" headline (decisions/0
|
|
|
150
181
|
- CONTRACT: `now` and `sampleNs` MUST be FINITE numbers. `recordRtt` THROWS on a non-finite
|
|
151
182
|
argument (warm path); `pick(now)` NEVER throws (fail-closed contract), so a non-finite `now`
|
|
152
183
|
yields P2C-random selection, not an error.
|
|
184
|
+
- `ConsistentHashBalancer extends BalancerBase` -- class. Sticky/affinity routing via a prebuilt
|
|
185
|
+
MAGLEV lookup table (M8, IPVS `mh` / Meta Katran / Cilium).
|
|
186
|
+
- `new ConsistentHashBalancer(capacity, eligible, weights?=null, m?=65537, seed?=0x9e3779b9)` --
|
|
187
|
+
`weights` is an optional caller Uint32Array (length >= capacity) COPIED into balancer-owned
|
|
188
|
+
weights (null = equal). `m` is the table size: a PRIME, > 1, and >= capacity (the guard: capacity
|
|
189
|
+
> m throws -- it would overfill / starve backends). Validates typeof-first BEFORE allocating the
|
|
190
|
+
table. Builds the table COLD (O(M x N) weighted Maglev populate; the M x 4-byte table is ~256KB at
|
|
191
|
+
the 65537 default -- disclosed cold cost, M configurable down for small pools).
|
|
192
|
+
- `pick(keyHash)` -> number. Maps an INTEGER key: slot = (keyHash >>> 0) % M, read `lookup[slot]`,
|
|
193
|
+
and if down forward-probe up to CH_PROBE_LIMIT (64) slots for the next eligible backend. O(1),
|
|
194
|
+
0 B/op. `keyHash` is coerced `>>> 0` (NaN -> 0) and pick NEVER throws. `PICK_NONE` when the pool is
|
|
195
|
+
down OR no eligible backend is reachable within the bound (a mass outage may fail closed even if a
|
|
196
|
+
far eligible slot exists -- safe, never a dead pick). KEY IS A CALLER INTEGER -- no per-pick string
|
|
197
|
+
hashing (the one zero-GC hazard); hash string keys yourself, cold. No hashing dependency added.
|
|
198
|
+
- `setWeight(i, w)` -> void. COLD. Set backend i's weight (uint32) and REBUILD the table.
|
|
199
|
+
- `rebuild()` -> void. COLD. Rebuild the table from the current owned weights (e.g. after a
|
|
200
|
+
membership change). A health flap does NOT rebuild -- the bounded probe absorbs it.
|
|
201
|
+
- `tableSize` -- readonly number. The Maglev table size M (prime).
|
|
202
|
+
- Minimal disruption: removing a backend is `setEligible(i, false)` (no rebuild), so only ~1/N keys
|
|
203
|
+
reroute (measured ~1.6% on remove/add vs the naive-modulo foil's ~98%; benchmark/Disruption.mjs).
|
|
204
|
+
- vs-AWS: NLB flow-hash (5-tuple) is a Maglev consistent hash -- this is that family at the
|
|
205
|
+
in-process hop. DEFERRED optional-peer seams (import nothing, peerDependencies STAYS `{}`): a
|
|
206
|
+
@zakkster/lite-filter hot-key oracle at the key-routing layer (warm/cold only), and a
|
|
207
|
+
@zakkster/lite-o1 `EliasFano` ring alternative to the table (decisions/0010).
|
|
208
|
+
- `BoundedLoadBalancer extends ConsistentHashBalancer` -- class. Consistent Hashing with Bounded Loads
|
|
209
|
+
(M9, CHBL: Mirrokni et al. / Google Research; Vimeo eps=0.25). ConsistentHash (the Maglev table) PLUS
|
|
210
|
+
a per-backend occupancy cap that overflows a hot backend to its neighbours.
|
|
211
|
+
- `new BoundedLoadBalancer(capacity, eligible, inflight, eps?=0.25, weights?=null, m?=65537, seed?=0x9e3779b9)`
|
|
212
|
+
-- `inflight` is a caller-owned Uint32Array (length >= capacity) read LIVE as the per-backend
|
|
213
|
+
OCCUPANCY; `eps` is the bounded-load slack (finite, > 0). Validates inflight + eps typeof-first
|
|
214
|
+
BEFORE super() allocates the Maglev table (TypeError non-number eps, RangeError non-finite / <= 0).
|
|
215
|
+
`weights` / `m` / `seed` are the ConsistentHash args (copied weights, prime m >= capacity, COLD
|
|
216
|
+
build). The running occupancy sum `_total` is BALANCER-OWNED (starts at 0) and written SOLELY by
|
|
217
|
+
`note`; when using BoundedLoad, the mirrored inflight counter is mutated ONLY through `note` / /pool
|
|
218
|
+
(direct mutation desyncs `_total` -- UB, the SmoothWRR-weights asymmetry).
|
|
219
|
+
- `pick(keyHash)` -> number. slot = (keyHash >>> 0) % M; walk the probe window (home + CH_PROBE_LIMIT
|
|
220
|
+
slots) and return the FIRST backend that is ELIGIBLE AND under `cap = (1 + eps) * _total / live`
|
|
221
|
+
(a hot home OVERFLOWS to a neighbour). If none in the window is under cap, fall back to the FIRST
|
|
222
|
+
eligible seen (sticky wins; the cap is a soft preference). `_total === 0` skips the cap -> pure
|
|
223
|
+
ConsistentHash. O(1), 0 B/op, NEVER throws. `PICK_NONE` ONLY when no eligible backend is reachable
|
|
224
|
+
in the window -- NEVER merely because backends are over cap (fail OPEN on overload).
|
|
225
|
+
- `note(i, delta)` -> void. WARM feedback path (not the hot pick path): the SOLE writer of `_total`
|
|
226
|
+
(dispatch note(i,+1) / settle note(i,-1)), so the cap's mean stays O(1)-current without a scan.
|
|
227
|
+
`i` is validated in range (RangeError), `delta` typeof-first as an integer; `_total` clamps at 0.
|
|
228
|
+
0 B/op on the success path.
|
|
229
|
+
- `totalInflight` -- readonly number. The owned running occupancy sum (for tests / observability).
|
|
230
|
+
- `setWeight(i, w)` / `rebuild()` / `tableSize` -- inherited from ConsistentHashBalancer (COLD
|
|
231
|
+
rebuild of the Maglev table; the readonly prime table size).
|
|
232
|
+
- THE PIVOT (decisions/0011): P2C-over-inflight with a `(1+eps) x mean` cap is byte-identical to
|
|
233
|
+
plain P2C (an under-cap draw always has lower inflight than an over-cap one), so the cap is only
|
|
234
|
+
LOAD-BEARING when the primary choice is a HASH -- CHBL is that. The hotspot anchor
|
|
235
|
+
(test/balance.mjs): under a skewed key stream, plain ConsistentHash spikes a hot backend to ~13x
|
|
236
|
+
the mean occupancy while CHBL caps it near (1+eps) x mean by overflow; both keep ~1/N minimal
|
|
237
|
+
disruption. `Pool.run` mirrors dispatch/settle into `note` when the balancer duck-types it, and
|
|
238
|
+
`Pool.run(fn, { key })` drives `pick(key)` for keyed CHBL routing (both inert otherwise).
|
|
153
239
|
|
|
154
240
|
## Subpath: @zakkster/lite-pick/pool -- the ergonomic request layer (M5, Pool.js)
|
|
155
241
|
|
|
@@ -178,7 +264,14 @@ duck-typed and imports NOTHING from lite-query.
|
|
|
178
264
|
`opts.clock` (a caller-owned nanosecond source) is supplied AND the balancer duck-types
|
|
179
265
|
`recordRtt` (PeakEwmaBalancer), Pool drives `pick(now)` and records the settled rtt on success;
|
|
180
266
|
otherwise the hook is inert -- Pool stays generic, in-flight stays net-zero, abort/failover
|
|
181
|
-
unchanged.
|
|
267
|
+
unchanged. OPT-IN occupancy feedback: when the balancer duck-types `note` (BoundedLoadBalancer),
|
|
268
|
+
Pool mirrors each dispatch as `note(i, +1)` and each settle as `note(i, -1)` so its owned mean
|
|
269
|
+
stays current; inert otherwise. The hooks are independent -- a balancer may duck-type neither,
|
|
270
|
+
one, or both; a BoundedLoad + Pool round is net-zero on BOTH the inflight array and `_total`.
|
|
271
|
+
OPT-IN keyed routing: `opts.key` (a caller INTEGER) drives `pick(key)` for a keyed balancer
|
|
272
|
+
(ConsistentHash / BoundedLoad -- CHBL); failover re-picks with the SAME key, and because the failed
|
|
273
|
+
backend's occupancy stays elevated a CHBL re-pick naturally OVERFLOWS to the next backend. Without
|
|
274
|
+
`opts.key`, `pick()` / `pick(now)` behaviour is unchanged.
|
|
182
275
|
- `liteQueryFetcher(pool, perEndpoint, opts?)` -> a `({ key, signal }) => Promise` fetcher for a
|
|
183
276
|
query cache (lite-query's `fetcher`, or any fetcher-shaped consumer). `perEndpoint({ endpoint,
|
|
184
277
|
key, signal })` -> the per-endpoint work. `opts.tries` (default 1) is the spatial failover count.
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zakkster/lite-pick",
|
|
3
3
|
"author": "Zahary Shinikchiev <shinikchiev@yahoo.com>",
|
|
4
|
-
"version": "0.
|
|
5
|
-
"description": "Zero-dependency, zero-GC load-balancing selection kernel: one hot pick() -> endpoint index over a fixed pool, 0 B/op steady-state. A pure selector (consumes health/circuit state, never a proxy) for the in-process hop, complementary to AWS NLB/ALB. Tree-shakeable ESM roster: RoundRobin, SmoothWRR, P2C, LeastConn, SED, NQ, PeakEWMA (latency-aware peak-EWMA),
|
|
4
|
+
"version": "0.9.0",
|
|
5
|
+
"description": "Zero-dependency, zero-GC load-balancing selection kernel: one hot pick() -> endpoint index over a fixed pool, 0 B/op steady-state. A pure selector (consumes health/circuit state, never a proxy) for the in-process hop, complementary to AWS NLB/ALB. Tree-shakeable ESM roster: RoundRobin, SmoothWRR, P2C, LeastConn, SED, NQ, PeakEWMA (latency-aware peak-EWMA), ConsistentHash (Maglev sticky/affinity routing, minimal disruption), and BoundedLoad (consistent hashing with bounded loads -- sticky routing with a per-backend occupancy cap that overflows a hotspot to neighbours); the /pool subpath adds dispatch/settle counters + failover and a duck-typed query-cache fetcher.",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./Pick.js",
|
|
8
8
|
"module": "./Pick.js",
|
|
@@ -74,6 +74,8 @@
|
|
|
74
74
|
"peak-ewma",
|
|
75
75
|
"latency-aware",
|
|
76
76
|
"consistent-hashing",
|
|
77
|
+
"consistent-hash",
|
|
78
|
+
"sticky",
|
|
77
79
|
"maglev",
|
|
78
80
|
"bounded-load",
|
|
79
81
|
"weighted-random",
|