@zakkster/lite-pick 0.7.1 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -0
- package/Pick.d.ts +40 -3
- package/Pick.js +243 -6
- package/README.md +40 -14
- package/RECIPES.md +379 -0
- package/llms.txt +44 -10
- package/package.json +5 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,56 @@ All notable changes to `@zakkster/lite-pick` are documented here. The format fol
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.8.0] - 2026-09-23
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **`ConsistentHashBalancer` (M8)** -- sticky / cache-affinity routing via a prebuilt **Maglev
|
|
12
|
+
lookup table** (the in-kernel/production choice: Linux IPVS `mh`, Meta Katran, Cilium).
|
|
13
|
+
`pick(keyHash)` maps a caller-supplied **integer** key to a backend (`slot = keyHash % M`, a table
|
|
14
|
+
read, and a bounded forward-probe past down slots) -- **O(1)**, **0 B/op**. The key is coerced
|
|
15
|
+
`>>> 0` (NaN -> 0) and `pick` never throws (fail-closed). Per-pick *string* hashing is the one
|
|
16
|
+
zero-GC hazard, so callers hash string keys themselves (cold); `lite-pick` adds **no hashing
|
|
17
|
+
dependency**. The balancer owns the lookup `Uint32Array` (`M x 4` bytes -- ~256KB at the `65537`
|
|
18
|
+
default `M`, a disclosed **cold** one-time allocation; `M` is configurable down for small pools)
|
|
19
|
+
and an internal weights array; `setWeight(i, w)` / `rebuild()` rebuild the table cold, while a
|
|
20
|
+
**health flap never rebuilds** -- the bounded probe (<= 64 slots) absorbs it. Weighted Maglev
|
|
21
|
+
populate gives each backend a per-backend slot quota proportional to its weight (unweighted = equal).
|
|
22
|
+
Fails closed (`PICK_NONE`) when the pool is down or no eligible backend is reachable within the bound.
|
|
23
|
+
Exports `CH_DEFAULT_M` (65537) and `CH_PROBE_LIMIT` (64) alongside the class. ([ADR 0010](./decisions/0010-consistenthash.md)).
|
|
24
|
+
- **Minimal-disruption anchor** -- removing 1 of 64 backends remaps only **~1.6%** of keys (`test/balance.mjs`,
|
|
25
|
+
`benchmark/Disruption.mjs`), versus the naive-modulo foil's **~98%**. `benchmark/Disruption.mjs`
|
|
26
|
+
replaces its M6 explicit ConsistentHash **SKIP** row with a real measured Maglev row (vs the modulo
|
|
27
|
+
foil and the `1/n` ideal); `benchmark/results.json` + the README fences regenerated (bench:verify green).
|
|
28
|
+
- Gates extended for the new strategy: `test/ConsistentHash.test.js` boundary suite; `test/fuzz.mjs`
|
|
29
|
+
keyed subject + `checkConsistentHash` / `reachableWithinBound` invariants; `test/torture.mjs` retention
|
|
30
|
+
+ a `pick(keyHash)` 0 B/op phase (build excluded, cold); `test/perf/PerfGate.test.mjs`
|
|
31
|
+
`consistentHashPick` scenario + a `mustFail` alloc tooth; `test/witness.mjs` O(1) const flat-work
|
|
32
|
+
subject; `benchmark/Matrix.mjs` subject; `Pick.d.ts` + `test/types/pick.test-d.ts` typed surface.
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
|
|
36
|
+
- `Pick.js`: STRATEGY-APPEND only -- the other seven strategies are **byte-identical**; the sole
|
|
37
|
+
changes are the header roster/count (seven -> eight), the `VERSION` bump, and the appended
|
|
38
|
+
`ConsistentHashBalancer` (+ the `chMix32` / `chIsPrime` cold helpers and the `CH_*` constants).
|
|
39
|
+
- `VERSION` bumped 0.7.2 -> **0.8.0** across the three sync sites (package.json, `Pick.js`, llms.txt);
|
|
40
|
+
package `description` + `keywords` (added `consistent-hash`, `sticky`) updated. `peerDependencies`
|
|
41
|
+
stays `{}` (the deferred `@zakkster/lite-filter` hot-key-oracle and `@zakkster/lite-o1` `EliasFano`
|
|
42
|
+
ring seams import nothing until a shipped path uses them).
|
|
43
|
+
|
|
44
|
+
## [0.7.2] - 2026-09-23
|
|
45
|
+
|
|
46
|
+
### Added
|
|
47
|
+
|
|
48
|
+
- `RECIPES.md` -- a beginner-to-advanced usage guide that builds the selection kernel up into a
|
|
49
|
+
real load balancer (health/eligibility wiring, caller-owned in-flight counters, the
|
|
50
|
+
dispatch/settle loop, `/pool` failover, PeakEWMA rtt feedback, the FE profile, a strategy
|
|
51
|
+
decision table, suite composition, zero-GC discipline, and gotchas). Added to the published
|
|
52
|
+
package (`files[]`) and linked from the README.
|
|
53
|
+
|
|
54
|
+
Docs-only release: no source or behavior change from 0.7.1 (the `VERSION` stamp is bumped for the
|
|
55
|
+
three-place sync).
|
|
56
|
+
|
|
7
57
|
## [0.7.1] - 2026-09-23
|
|
8
58
|
|
|
9
59
|
### Fixed
|
package/Pick.d.ts
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @zakkster/lite-pick -- TypeScript declarations.
|
|
3
3
|
*
|
|
4
|
-
*
|
|
5
|
-
* (LeastConn/SED/NQ) + PeakEWMA (latency-aware P2C)
|
|
6
|
-
* (
|
|
4
|
+
* M8 (0.8.0): substrate seams + RoundRobin + SmoothWRR + P2C + the exact LeastConn family
|
|
5
|
+
* (LeastConn/SED/NQ) + PeakEWMA (latency-aware P2C) + ConsistentHash (Maglev table). The
|
|
6
|
+
* remaining strategy classes (BoundedLoad, WeightedRandom) are added one per session.
|
|
7
7
|
*/
|
|
8
8
|
|
|
9
9
|
/** The single source-of-truth version stamp. */
|
|
@@ -187,3 +187,40 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
187
187
|
/** Pick by latency-aware power-of-two-choices at time `now` (ns), or `PICK_NONE`. O(d)=O(1). */
|
|
188
188
|
pick(now?: number): number;
|
|
189
189
|
}
|
|
190
|
+
|
|
191
|
+
/** The default Maglev lookup-table size (a prime, 2^16 + 1). Configurable via the ctor. */
|
|
192
|
+
export const CH_DEFAULT_M: number;
|
|
193
|
+
|
|
194
|
+
/** The bounded forward-probe limit ConsistentHash walks past down slots (fail-closed). */
|
|
195
|
+
export const CH_PROBE_LIMIT: number;
|
|
196
|
+
|
|
197
|
+
/**
|
|
198
|
+
* ConsistentHashBalancer -- sticky / cache-affinity routing via a prebuilt MAGLEV lookup table
|
|
199
|
+
* (M8, IPVS `mh` / Meta Katran / Cilium). `pick(keyHash)` maps a caller-supplied INTEGER key to a
|
|
200
|
+
* fixed backend (slot = keyHash % M, a table read, and a bounded forward-probe past down slots) --
|
|
201
|
+
* O(1), 0 B/op. The key is a caller-supplied integer (coerced `>>> 0`; NaN -> 0), never a per-pick
|
|
202
|
+
* string hash (the one zero-GC hazard -- hash string keys yourself, cold). The balancer OWNS the
|
|
203
|
+
* lookup table (M x 4 bytes; the 65537 default is ~256KB, a COLD one-time allocation) and an internal
|
|
204
|
+
* weights array; `setWeight` / `rebuild` rebuild the table (COLD). A health flap is absorbed by the
|
|
205
|
+
* probe -- never a rebuild -- so removing a backend (`setEligible(i, false)`) remaps only ~1/N keys.
|
|
206
|
+
* Fails closed (`PICK_NONE`) when the pool is down or no eligible backend is reachable within the bound.
|
|
207
|
+
*/
|
|
208
|
+
export class ConsistentHashBalancer extends BalancerBase {
|
|
209
|
+
/**
|
|
210
|
+
* @param capacity backend count (fixed).
|
|
211
|
+
* @param eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
212
|
+
* @param weights optional per-backend weights (length >= capacity), COPIED at construction;
|
|
213
|
+
* null = equal weight.
|
|
214
|
+
* @param m the Maglev table size: a prime, > 1, and >= capacity (default 65537).
|
|
215
|
+
* @param seed deterministic salt for the permutation mix (default 0x9e3779b9); reproducible.
|
|
216
|
+
*/
|
|
217
|
+
constructor(capacity: number, eligible: Uint8Array, weights?: Uint32Array | null, m?: number, seed?: number);
|
|
218
|
+
/** The Maglev table size M (prime). */
|
|
219
|
+
readonly tableSize: number;
|
|
220
|
+
/** Cold path: reconfigure backend `i`'s weight (uint32) and rebuild the table. */
|
|
221
|
+
setWeight(i: number, w: number): void;
|
|
222
|
+
/** Cold path: rebuild the lookup table from the current owned weights. */
|
|
223
|
+
rebuild(): void;
|
|
224
|
+
/** Map an integer `keyHash` to a backend index (bounded probe past down slots), or `PICK_NONE`. */
|
|
225
|
+
pick(keyHash?: number): number;
|
|
226
|
+
}
|
package/Pick.js
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @zakkster/lite-pick -- zero-GC load-balancing SELECTION KERNEL.
|
|
3
3
|
*
|
|
4
|
-
*
|
|
5
|
-
* LeastConn family (LeastConn, SED, NQ),
|
|
4
|
+
* M8 (0.8.0): substrate seams + eight strategies -- RoundRobin, SmoothWRR, P2C, the exact
|
|
5
|
+
* LeastConn family (LeastConn, SED, NQ), PeakEWMA (latency-aware P2C), and ConsistentHash
|
|
6
|
+
* (a Maglev lookup table). This file ships:
|
|
6
7
|
*
|
|
7
8
|
* - VERSION the single source-of-truth version stamp (3-place sync).
|
|
8
9
|
* - PICK_NONE the fail-closed sentinel (-1): "no endpoint", never a dead pick.
|
|
@@ -31,6 +32,12 @@
|
|
|
31
32
|
* Decay-on-READ (pick() never writes -> 0 B/op); the balancer OWNS the Float64
|
|
32
33
|
* _ewma/_stamp state and is its SOLE writer via the warm recordRtt() feedback
|
|
33
34
|
* path (also 0 B/op). Caller-supplied nanosecond clock. O(d)=O(1)/pick.
|
|
35
|
+
* - ConsistentHashBalancer sticky/affinity routing via a prebuilt Maglev lookup table (IPVS
|
|
36
|
+
* `mh`, Meta Katran, Cilium): pick(keyHash) is slot = keyHash % M, a table read,
|
|
37
|
+
* and a bounded forward-probe over down slots -- O(1)/pick, 0 B/op. keyHash is a
|
|
38
|
+
* caller-supplied INTEGER (no per-pick string hashing = the one zero-GC hazard);
|
|
39
|
+
* the balancer OWNS the Uint32Array table + weights, rebuilt COLD on membership /
|
|
40
|
+
* weight change (health flap is handled by the probe, never a rebuild).
|
|
34
41
|
*
|
|
35
42
|
* The identity (decisions/0001): lite-pick OWNS NO mutable state it can avoid owning.
|
|
36
43
|
* It reads pre-allocated views (eligibility, inflight, weights, scores) that siblings or
|
|
@@ -38,9 +45,9 @@
|
|
|
38
45
|
* counters live OUTSIDE the kernel. The steady-state pick path allocates 0 B/op.
|
|
39
46
|
*
|
|
40
47
|
* Roster (one strategy per session -- see ROADMAP.md): RoundRobin [M1], SmoothWRR [M2],
|
|
41
|
-
* P2C [M3], LeastConn/SED/NQ [M4], PeakEWMA [M7], ConsistentHash, BoundedLoad,
|
|
42
|
-
* [planned]. The EXACT-O(log n) fewest-in-flight variant is a deferred
|
|
43
|
-
* BinaryHeap optional-peer seam (decisions/0006), not this exact-O(cap) scan.
|
|
48
|
+
* P2C [M3], LeastConn/SED/NQ [M4], PeakEWMA [M7], ConsistentHash [M8], BoundedLoad,
|
|
49
|
+
* WeightedRandom [planned]. The EXACT-O(log n) fewest-in-flight variant is a deferred
|
|
50
|
+
* @zakkster/lite-logn BinaryHeap optional-peer seam (decisions/0006), not this exact-O(cap) scan.
|
|
44
51
|
*
|
|
45
52
|
* M5 (0.5.0) adds the ergonomic request layer at the @zakkster/lite-pick/pool subpath (a
|
|
46
53
|
* SEPARATE file, Pool.js -- the async dispatch/settle counter wrapper + distinct-endpoint
|
|
@@ -51,7 +58,7 @@
|
|
|
51
58
|
*/
|
|
52
59
|
|
|
53
60
|
/** Version stamp. Synced across package.json and llms.txt (three-place rule). */
|
|
54
|
-
export const VERSION = '0.
|
|
61
|
+
export const VERSION = '0.8.0';
|
|
55
62
|
|
|
56
63
|
/**
|
|
57
64
|
* Fail-closed sentinel returned by pick() when no endpoint is eligible.
|
|
@@ -739,3 +746,233 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
739
746
|
return costB < costA ? b : a; // lower cost wins; tie -> the first draw
|
|
740
747
|
}
|
|
741
748
|
}
|
|
749
|
+
|
|
750
|
+
/** The default Maglev lookup-table size: a prime (2^16 + 1). Configurable via the ctor. */
|
|
751
|
+
export const CH_DEFAULT_M = 65537;
|
|
752
|
+
|
|
753
|
+
/** The bounded forward-probe limit ConsistentHash walks past down slots (fail-closed). */
|
|
754
|
+
export const CH_PROBE_LIMIT = 64;
|
|
755
|
+
|
|
756
|
+
/**
|
|
757
|
+
* A deterministic 32-bit integer mix (an SplitMix/Murmur-style finalizer). Used COLD, once
|
|
758
|
+
* per backend at table build, to derive the two Maglev permutation parameters from a backend
|
|
759
|
+
* INDEX -- NO string hashing, NO new dependency, no allocation. Returns a uint32.
|
|
760
|
+
* @param {number} x
|
|
761
|
+
* @returns {number}
|
|
762
|
+
*/
|
|
763
|
+
function chMix32(x) {
|
|
764
|
+
x = x >>> 0;
|
|
765
|
+
x ^= x >>> 16; x = Math.imul(x, 0x7feb352d);
|
|
766
|
+
x ^= x >>> 15; x = Math.imul(x, 0x846ca68b);
|
|
767
|
+
x ^= x >>> 16;
|
|
768
|
+
return x >>> 0;
|
|
769
|
+
}
|
|
770
|
+
|
|
771
|
+
/** Cold primality test (trial division). M must be prime so a Maglev skip yields a full permutation. */
|
|
772
|
+
function chIsPrime(n) {
|
|
773
|
+
if (!Number.isInteger(n) || n < 2) return false;
|
|
774
|
+
if (n % 2 === 0) return n === 2;
|
|
775
|
+
if (n % 3 === 0) return n === 3;
|
|
776
|
+
for (let d = 5; d * d <= n; d += 6) {
|
|
777
|
+
if (n % d === 0 || n % (d + 2) === 0) return false;
|
|
778
|
+
}
|
|
779
|
+
return true;
|
|
780
|
+
}
|
|
781
|
+
|
|
782
|
+
/**
|
|
783
|
+
* ConsistentHashBalancer -- sticky / cache-affinity routing via a prebuilt MAGLEV lookup table
|
|
784
|
+
* (M8), the in-kernel/production consistent-hash choice (Linux IPVS `mh`, Meta Katran, Cilium).
|
|
785
|
+
*
|
|
786
|
+
* `pick(keyHash)` maps a caller-supplied INTEGER key to a fixed backend: slot = keyHash % M, read
|
|
787
|
+
* the backend at `lookup[slot]`, and if it is down walk a BOUNDED forward-probe (<= 64 slots) to
|
|
788
|
+
* the next eligible backend. That is a few integer ops over a prebuilt Uint32Array table -- O(1)
|
|
789
|
+
* per pick, 0 B/op. The KEY MUST BE AN INTEGER (`keyHash >>> 0`, so NaN -> 0 deterministically):
|
|
790
|
+
* per-pick STRING hashing is the one zero-GC hazard, so the caller hashes string keys themselves
|
|
791
|
+
* (cold) and passes the integer -- lite-pick adds NO hashing dependency (RESEARCH section 5).
|
|
792
|
+
*
|
|
793
|
+
* MINIMAL DISRUPTION is the selling point: on a scale event only ~1/N of keys move. Removing a
|
|
794
|
+
* backend is just marking it down (`setEligible(i, false)`) -- the table is UNCHANGED, so every
|
|
795
|
+
* key NOT on that backend keeps its exact backend (0 remap) and only its keys probe forward.
|
|
796
|
+
* A membership or WEIGHT change rebuilds the table (COLD); a health flap NEVER does (the probe
|
|
797
|
+
* absorbs it). test/balance.mjs anchors this: remove 1 of 64 backends -> <= 3.13% keys remapped,
|
|
798
|
+
* vs the naive-modulo foil's >= 95%.
|
|
799
|
+
*
|
|
800
|
+
* Ownership (ADR 0001, ADR 0010): the balancer OWNS the lookup `Uint32Array` (M x 4 bytes; the
|
|
801
|
+
* 65537 default is ~256KB, a COLD one-time allocation -- disclosed, and M is configurable DOWN
|
|
802
|
+
* for small pools) AND an internal weights `Uint32Array` (the SmoothWRR sole-writer precedent):
|
|
803
|
+
* cold `setWeight(i, w)` / `rebuild()` rebuild the table from the current weights. Eligibility is
|
|
804
|
+
* the shared read-only bitmap from BalancerBase, read live by `pick()`; the base `setEligible`
|
|
805
|
+
* only flips a bit (no rebuild).
|
|
806
|
+
*
|
|
807
|
+
* Weighted Maglev populate: each backend b takes a per-backend slot QUOTA proportional to its
|
|
808
|
+
* weight (unweighted = equal quota, quotas summing to exactly M), stepping through its own
|
|
809
|
+
* permutation `permutation[j] = (offset + j*skip) % M` (offset = h1(b) % M, skip = h2(b) % (M-1) +
|
|
810
|
+
* 1, h1/h2 from a deterministic integer mix of the index -- no string hashing). Because the
|
|
811
|
+
* permutation covers ALL M slots, a backend with remaining quota can always reach an empty slot
|
|
812
|
+
* while one exists, so the O(M x N) COLD build never stalls. BUILD-COST GUARD (fail closed): the
|
|
813
|
+
* ctor requires `capacity <= M` (more members than slots would overfill M / starve backends) and
|
|
814
|
+
* M prime > 1.
|
|
815
|
+
*
|
|
816
|
+
* DEFERRED optional-peer seam (ADR 0010 / llms.txt, never on the pick path): a `@zakkster/lite-filter`
|
|
817
|
+
* hot-key / known-key oracle (BlockedBloom etc.) for warm-affinity + admission at the KEY-routing
|
|
818
|
+
* layer, and a `@zakkster/lite-o1` `EliasFano` ring alternative to the table. Import NOTHING;
|
|
819
|
+
* `peerDependencies` STAYS `{}` until a shipped code path imports it.
|
|
820
|
+
*
|
|
821
|
+
* Bound: O(1) per pick, 0 B/op (integer ops over the prebuilt table -- no allocation). Fails closed
|
|
822
|
+
* (PICK_NONE) when the whole pool is down OR no eligible backend is reachable within the probe bound
|
|
823
|
+
* (a near-total outage may return PICK_NONE even if a far eligible slot exists -- safe, never a dead
|
|
824
|
+
* pick, over-conservative only under mass outage; ADR 0010).
|
|
825
|
+
*/
|
|
826
|
+
export class ConsistentHashBalancer extends BalancerBase {
|
|
827
|
+
/**
|
|
828
|
+
* @param {number} capacity backend count (fixed; add/remove is a cold rebuild).
|
|
829
|
+
* @param {Uint8Array} eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
830
|
+
* @param {Uint32Array|null} [weights=null] optional per-backend weights (length >= capacity);
|
|
831
|
+
* the values are COPIED into the balancer-owned weights at construction. null = equal weight.
|
|
832
|
+
* @param {number} [m=CH_DEFAULT_M] the Maglev table size: a prime, > 1, and >= capacity.
|
|
833
|
+
* @param {number} [seed=0x9e3779b9] deterministic salt for the permutation mix (reproducible).
|
|
834
|
+
*/
|
|
835
|
+
constructor(capacity, eligible, weights = null, m = CH_DEFAULT_M, seed = 0x9e3779b9) {
|
|
836
|
+
super(capacity, eligible);
|
|
837
|
+
// Validate typeof-first, BEFORE allocating the table / owned weights (fail closed early).
|
|
838
|
+
if (typeof m !== 'number') {
|
|
839
|
+
throw new TypeError('[lite-pick] table size M must be a number');
|
|
840
|
+
}
|
|
841
|
+
if (!Number.isInteger(m) || m < 2 || !chIsPrime(m)) {
|
|
842
|
+
throw new RangeError('[lite-pick] table size M must be a prime integer > 1: ' + m);
|
|
843
|
+
}
|
|
844
|
+
if (capacity > m) {
|
|
845
|
+
throw new RangeError('[lite-pick] capacity ' + capacity +
|
|
846
|
+
' exceeds table size M ' + m + ' (would overfill / starve backends)');
|
|
847
|
+
}
|
|
848
|
+
if (weights !== null && (!(weights instanceof Uint32Array) || weights.length < capacity)) {
|
|
849
|
+
throw new RangeError('[lite-pick] weights must be a Uint32Array of length >= capacity');
|
|
850
|
+
}
|
|
851
|
+
this._m = m;
|
|
852
|
+
this._seed = seed >>> 0;
|
|
853
|
+
// Balancer-owned weights (the SmoothWRR sole-writer precedent): copied, then the sole
|
|
854
|
+
// mutator is setWeight (which rebuilds). Unweighted default = equal weight 1.
|
|
855
|
+
this._weights = new Uint32Array(capacity);
|
|
856
|
+
if (weights !== null) this._weights.set(weights.subarray(0, capacity));
|
|
857
|
+
else this._weights.fill(1);
|
|
858
|
+
// The prebuilt lookup table (slot -> backend index). M x 4 bytes; the COLD build fills it.
|
|
859
|
+
this._lookup = new Uint32Array(m);
|
|
860
|
+
this._build();
|
|
861
|
+
}
|
|
862
|
+
|
|
863
|
+
/** The Maglev table size M (prime). Readonly. */
|
|
864
|
+
get tableSize() {
|
|
865
|
+
return this._m;
|
|
866
|
+
}
|
|
867
|
+
|
|
868
|
+
/**
|
|
869
|
+
* COLD: (re)populate the lookup table from the current balancer-owned weights via the weighted
|
|
870
|
+
* Maglev algorithm. O(M x N); never stalls (each backend's permutation covers all M slots).
|
|
871
|
+
* Rebuilt only on a membership / weight change -- never on a health flap. Allocates only cold
|
|
872
|
+
* scratch that is released after the build; the lookup table itself is reused in place.
|
|
873
|
+
*/
|
|
874
|
+
_build() {
|
|
875
|
+
const M = this._m, N = this._cap, wt = this._weights, lookup = this._lookup, seed = this._seed;
|
|
876
|
+
// Per-backend Maglev permutation parameters from a deterministic integer mix of the index.
|
|
877
|
+
const offset = new Int32Array(N);
|
|
878
|
+
const skip = new Int32Array(N);
|
|
879
|
+
for (let b = 0; b < N; b++) {
|
|
880
|
+
const h1 = chMix32(b ^ seed);
|
|
881
|
+
const h2 = chMix32((b ^ seed) + 0x9e3779b9);
|
|
882
|
+
offset[b] = h1 % M;
|
|
883
|
+
skip[b] = (h2 % (M - 1)) + 1;
|
|
884
|
+
}
|
|
885
|
+
// Per-backend slot QUOTA proportional to weight, summing to exactly M (unweighted = equal).
|
|
886
|
+
const quota = new Int32Array(N);
|
|
887
|
+
let total = 0;
|
|
888
|
+
for (let b = 0; b < N; b++) total += wt[b];
|
|
889
|
+
if (total <= 0) {
|
|
890
|
+
// Degenerate all-zero weights: equal quota so the table is still fully, validly populated.
|
|
891
|
+
const base = Math.floor(M / N);
|
|
892
|
+
for (let b = 0; b < N; b++) quota[b] = base;
|
|
893
|
+
let leftover = M - base * N;
|
|
894
|
+
for (let b = 0; leftover > 0; b = (b + 1) % N) { quota[b]++; leftover--; }
|
|
895
|
+
} else {
|
|
896
|
+
let assigned = 0;
|
|
897
|
+
for (let b = 0; b < N; b++) { const q = Math.floor(wt[b] / total * M); quota[b] = q; assigned += q; }
|
|
898
|
+
let leftover = M - assigned;
|
|
899
|
+
// Hand the rounding leftover to positive-weight backends in index order (deterministic).
|
|
900
|
+
for (let b = 0; leftover > 0; b = (b + 1) % N) { if (wt[b] > 0) { quota[b]++; leftover--; } }
|
|
901
|
+
}
|
|
902
|
+
// Maglev populate honoring the quota. The permutation is surjective over all M slots, so a
|
|
903
|
+
// backend with remaining quota always reaches an empty slot while one exists -> no stall.
|
|
904
|
+
const next = new Int32Array(N);
|
|
905
|
+
const filledCount = new Int32Array(N);
|
|
906
|
+
const taken = new Uint8Array(M);
|
|
907
|
+
let filled = 0;
|
|
908
|
+
while (filled < M) {
|
|
909
|
+
let progressed = false;
|
|
910
|
+
for (let b = 0; b < N; b++) {
|
|
911
|
+
if (filledCount[b] >= quota[b]) continue;
|
|
912
|
+
let j = next[b];
|
|
913
|
+
let c = (offset[b] + (j % M) * skip[b]) % M;
|
|
914
|
+
while (taken[c]) { j++; c = (offset[b] + (j % M) * skip[b]) % M; }
|
|
915
|
+
lookup[c] = b;
|
|
916
|
+
taken[c] = 1;
|
|
917
|
+
next[b] = j + 1;
|
|
918
|
+
filledCount[b]++;
|
|
919
|
+
filled++;
|
|
920
|
+
progressed = true;
|
|
921
|
+
if (filled >= M) break;
|
|
922
|
+
}
|
|
923
|
+
if (!progressed) break; // unreachable while quotas sum to M -- defensive
|
|
924
|
+
}
|
|
925
|
+
// Defensive completeness (unreachable): any slot left empty is pinned to backend 0 so pick()
|
|
926
|
+
// never reads a stale / out-of-range index. Fail closed on a valid table, always.
|
|
927
|
+
if (filled < M) {
|
|
928
|
+
for (let c = 0; c < M; c++) if (!taken[c]) lookup[c] = 0;
|
|
929
|
+
}
|
|
930
|
+
}
|
|
931
|
+
|
|
932
|
+
/**
|
|
933
|
+
* COLD: reconfigure backend i's weight (uint32) and REBUILD the table from the new weights.
|
|
934
|
+
* The balancer is the sole writer of its owned weights (the SmoothWRR precedent).
|
|
935
|
+
* @param {number} i
|
|
936
|
+
* @param {number} w new weight (uint32)
|
|
937
|
+
*/
|
|
938
|
+
setWeight(i, w) {
|
|
939
|
+
if (i < 0 || i >= this._cap) throw new RangeError('[lite-pick] index out of range: ' + i);
|
|
940
|
+
const nw = w >>> 0;
|
|
941
|
+
if (nw !== w) throw new RangeError('[lite-pick] weight must be a uint32: ' + w);
|
|
942
|
+
if (nw === this._weights[i]) return;
|
|
943
|
+
this._weights[i] = nw;
|
|
944
|
+
this._build();
|
|
945
|
+
}
|
|
946
|
+
|
|
947
|
+
/** COLD: rebuild the lookup table from the current owned weights (e.g. after a membership change). */
|
|
948
|
+
rebuild() {
|
|
949
|
+
this._build();
|
|
950
|
+
}
|
|
951
|
+
|
|
952
|
+
/**
|
|
953
|
+
* Map an INTEGER key to a backend index, or PICK_NONE (fail closed). O(1), 0 B/op.
|
|
954
|
+
*
|
|
955
|
+
* slot = (keyHash >>> 0) % M; if `lookup[slot]` is eligible return it, else forward-probe up to
|
|
956
|
+
* CH_PROBE_LIMIT (64) slots for the next eligible backend. `keyHash` is coerced `>>> 0` (NaN -> 0
|
|
957
|
+
* deterministically) and pick NEVER throws (the fail-closed contract). Returns PICK_NONE when the
|
|
958
|
+
* whole pool is down or no eligible backend is reachable within the bound.
|
|
959
|
+
* @param {number} keyHash a caller-supplied integer key hash (coerced to uint32)
|
|
960
|
+
* @returns {number}
|
|
961
|
+
*/
|
|
962
|
+
pick(keyHash) {
|
|
963
|
+
if (this._live === 0) return PICK_NONE; // whole pool down: fail closed
|
|
964
|
+
const M = this._m, el = this._eligible, lookup = this._lookup;
|
|
965
|
+
let slot = (keyHash >>> 0) % M; // integer key; NaN >>> 0 = 0 (never throws)
|
|
966
|
+
let i = lookup[slot];
|
|
967
|
+
if (el[i]) return i;
|
|
968
|
+
// Bounded forward-probe past down slots. Past the bound we fail closed: a near-total outage
|
|
969
|
+
// may return PICK_NONE even if a far eligible slot exists -- safe, never a dead pick.
|
|
970
|
+
for (let p = 0; p < CH_PROBE_LIMIT; p++) {
|
|
971
|
+
slot++;
|
|
972
|
+
if (slot >= M) slot = 0;
|
|
973
|
+
i = lookup[slot];
|
|
974
|
+
if (el[i]) return i;
|
|
975
|
+
}
|
|
976
|
+
return PICK_NONE;
|
|
977
|
+
}
|
|
978
|
+
}
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v0.
|
|
3
|
+
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v0.8.0 ships eight strategies -- `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, the latency-aware `PeakEwmaBalancer`, and the sticky/affinity `ConsistentHashBalancer` (a Maglev lookup table)** -- on the substrate seams (`VERSION`, `PICK_NONE`, a deterministic `Prng`, and `BalancerBase`'s shared read-only eligibility view), plus a **`@zakkster/lite-pick/pool`** subpath: the async dispatch/settle counter layer with distinct-endpoint failover and a duck-typed query-cache fetcher. The rest of the roster -- BoundedLoad, WeightedRandom -- lands one per session.
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/@zakkster/lite-pick)
|
|
6
6
|
[](https://github.com/sponsors/PeshoVurtoleta)
|
|
@@ -21,7 +21,7 @@ The npm landscape has old algorithm libraries (`load-balancers`, `loadbalance`,
|
|
|
21
21
|
- **Two pieces of evidence, both shipped.** A **0 B/op** witness on the pick path (no object, closure, string, or array created per pick), and a measured **balance-quality anchor** -- peak-to-average load within the strategy's theoretical ceiling (for P2C, the Azar-Broder-Karlin-Upfal `ln ln n / ln 2` bound) and strictly better than a random foil.
|
|
22
22
|
- **A pure selector, not a proxy.** It **consumes** health and circuit state; it never owns them. Health is a shared read-only bitmap written by [`@zakkster/lite-di-health`](https://www.npmjs.com/package/@zakkster/lite-di-health); circuit state comes from [`@zakkster/lite-statechart`](https://www.npmjs.com/package/@zakkster/lite-statechart); load counters are caller-owned typed arrays. `pick()` only reads.
|
|
23
23
|
|
|
24
|
-
> **Status:
|
|
24
|
+
> **Status: M8 (v0.8.0).** Ships the substrate seams **plus `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, the latency-aware `PeakEwmaBalancer`, and the sticky/affinity `ConsistentHashBalancer` (a Maglev table)**, the **`@zakkster/lite-pick/pool`** request layer (now with an opt-in latency-feedback hook), and the **benchmark suite** -- the balance anchor + GC blast-radius headlines, a seeded/version-stamped `results.json`, a `bench:verify` drift check with teeth, and the vs-AWS positioning (see *Evidence* below). This session APPENDS one class: the other strategies in `Pick.js` are byte-identical, only the header roster/count and the `VERSION` stamp change. Every strategy is gated: `pick()` proven **0 B/op** (torture + PerfGate), RoundRobin **perfectly fair** with **zero dead picks** vs the naive `i++ % n` foil, SmoothWRR **exactly weighted** and **smooth**, **P2C proves the `ln ln n` balance ceiling** (peak-to-mean gap ~2 vs a random foil's ~21 at n=1024), **LeastConn is greedy-perfect** (max-minus-min load <= 1), **SED tracks weight within 1%**, **PeakEWMA steers around a 10x-slow node** (it takes <= 25% of P2C's share for it and cuts service p99), and **ConsistentHash remaps only ~1.6% of keys on a scale event** (vs ~98% for naive modulo) -- all held under a **seeded invariant fuzzer** (`test/fuzz.mjs`) that checks state-synchronisation after *every* op. See [ROADMAP.md](./ROADMAP.md) for the M8 -> M10 path to 1.0.0, and [decisions/](./decisions) for the ownership boundary (ADR 0001), anti-flapping (ADR 0002), the RoundRobin (0003), SmoothWRR (0004), P2C (0005), LeastConn-family (0006), pool-adapter (0007), benchmark-suite (0008), PeakEWMA (0009), and ConsistentHash (0010) design forks.
|
|
25
25
|
|
|
26
26
|
```bash
|
|
27
27
|
npm install @zakkster/lite-pick
|
|
@@ -180,6 +180,30 @@ PeakEWMA sends the slow node **<= 25% of P2C's share** for it and cuts service p
|
|
|
180
180
|
|
|
181
181
|
For a **front-end / browser client** -- a handful of picks per second across origins/regions, not a zero-GC hot loop -- the recommended profile is **PeakEWMA + the eligibility bitmap only**: latency-aware choice with a fail-closed health view, and **none** of the server-side bounded-load / availability-zone / occupancy machinery. It is the smallest honest latency-aware client balancer. *(Deferred: a tail-aware `inflight x p99Rtt` variant via an optional-peer `@zakkster/lite-sketch` `DDSketch`; the EWMA-mean score is the shipped zero-peer default, and `peerDependencies` stays `{}` -- [ADR 0009](./decisions/0009-peakewma.md).)*
|
|
182
182
|
|
|
183
|
+
## ConsistentHash -- sticky / cache-affinity routing (v0.8.0)
|
|
184
|
+
|
|
185
|
+
When a request must go to the **same** backend every time -- a session, a cache shard, a stateful worker -- you want **consistent hashing**: a stable key -> backend map that barely changes when the pool scales. `ConsistentHashBalancer` is a prebuilt **Maglev lookup table** (the in-kernel/production choice -- Linux IPVS `mh`, Meta Katran, Cilium), so `pick(keyHash)` is `slot = keyHash % M`, one table read, and a bounded probe past down slots -- **O(1)** and **0 B/op** ([ADR 0010](./decisions/0010-consistenthash.md)).
|
|
186
|
+
|
|
187
|
+
```js
|
|
188
|
+
import { ConsistentHashBalancer } from '@zakkster/lite-pick';
|
|
189
|
+
|
|
190
|
+
const eligible = Uint8Array.from([1, 1, 1, 1]);
|
|
191
|
+
// YOU hash the key to an INTEGER (cold) -- per-pick STRING hashing is the one zero-GC hazard.
|
|
192
|
+
const b = new ConsistentHashBalancer(4, eligible); // default M = 65537 (prime); weights optional
|
|
193
|
+
|
|
194
|
+
const key = fnv1a(sessionId); // any integer hash you like -- lite-pick adds none
|
|
195
|
+
const i = b.pick(key >>> 0); // same key -> same backend, at fixed membership
|
|
196
|
+
|
|
197
|
+
// A tiny FNV-1a over a string, done ONCE per key on the cold path (never inside pick):
|
|
198
|
+
function fnv1a(s) { let h = 0x811c9dc5; for (let k = 0; k < s.length; k++) { h ^= s.charCodeAt(k); h = Math.imul(h, 0x01000193); } return h >>> 0; }
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
- **Caller-supplied INTEGER key.** `pick(keyHash)` coerces `keyHash >>> 0` (so `NaN`/`undefined` -> `0`) and **never throws** (the fail-closed contract). It never hashes a string -- that would allocate on the hot path. Hash string keys yourself, cold (a tiny FNV-1a is fine -- see the snippet above); `lite-pick` adds **no hashing dependency**.
|
|
202
|
+
- **Minimal disruption.** Removing a backend is just `setEligible(i, false)` -- the table is **untouched**, so every key not on that backend keeps its exact backend and only ~`1/N` reroute (measured **1.6%** vs the naive-modulo trap's **98%** above). A membership or weight change rebuilds the table (**cold**); a health flap **never** does -- the bounded probe (<= 64 slots) absorbs it.
|
|
203
|
+
- **Weighted.** Pass a `Uint32Array` of weights (copied, balancer-owned) for a per-backend slot share proportional to weight; `setWeight(i, w)` / `rebuild()` rebuild the table cold. Unweighted = equal share.
|
|
204
|
+
- **Cost & bound.** The lookup table is `M x 4` bytes -- `~256KB` at the `65537` default -- a **cold, one-time** allocation (disclosed in the cost table below; `M` is configurable **down** for small pools). `pick()` is `O(1)`, `0 B/op`. Fail-closed: `PICK_NONE` when the pool is down or no eligible backend is reachable within the probe bound (a near-total outage may return `PICK_NONE` even if a far eligible slot exists -- safe, never a dead pick).
|
|
205
|
+
- **Deferred seams (import nothing).** A `@zakkster/lite-filter` hot-key / known-key oracle at the key-routing layer (warm/cold only, never the pick path) and a `@zakkster/lite-o1` `EliasFano` ring alternative to the table are optional-peer seams -- `peerDependencies` **stays `{}`** until a shipped path imports one ([ADR 0010](./decisions/0010-consistenthash.md)).
|
|
206
|
+
|
|
183
207
|
## Evidence -- the two headlines (v0.6.0 benchmark suite)
|
|
184
208
|
|
|
185
209
|
> **Framing: parity on speed, superiority on the contract + balance + tail.** A trivial `i++ % n` round-robin -- or `wrr` -- *matches* P2C on raw ops/sec, so `lite-pick` does **not** claim "N times faster." Throughput is claimed at **parity**; the wins are **zero-GC**, **balance quality**, **tail latency** (GC blast-radius), and **never a dead pick**. Every number below is **seeded** and regenerated by `npm run bench:report`; `npm run bench:verify` fails CI if a README number drifts from a fresh run (algorithmic exact, timing within +/-15%). Node / CPU / OS / every PRNG seed are stamped into `benchmark/results.json`.
|
|
@@ -192,9 +216,9 @@ The real pinned npm incumbents (`load-balancers`, `loadbalance`, `wrr`) run thro
|
|
|
192
216
|
|
|
193
217
|
| family | lite-pick | lite-pick ops/ms | incumbent (npm) | incumbent ops/ms |
|
|
194
218
|
| --- | --- | --- | --- | --- |
|
|
195
|
-
| P2C (power-of-two-choices) | P2cBalancer |
|
|
196
|
-
| RoundRobin | RoundRobinBalancer |
|
|
197
|
-
| Weighted-random | WeightedRandom -- SKIP, ships M10 | -- | wrr@1.0.0 |
|
|
219
|
+
| P2C (power-of-two-choices) | P2cBalancer | 53009 | load-balancers@1.3.52 | 59485 |
|
|
220
|
+
| RoundRobin | RoundRobinBalancer | 246432 | loadbalance@1.0.0 | 303459 |
|
|
221
|
+
| Weighted-random | WeightedRandom -- SKIP, ships M10 | -- | wrr@1.0.0 | 163481 |
|
|
198
222
|
|
|
199
223
|
<!-- /bench:competitors -->
|
|
200
224
|
|
|
@@ -222,8 +246,8 @@ The point of zero-GC is **not** the pick's own latency -- a major GC pause freez
|
|
|
222
246
|
|
|
223
247
|
| lane | major GC | pick B/op | max GC pause (ms) |
|
|
224
248
|
| --- | --- | --- | --- |
|
|
225
|
-
| lite-pick | 0 | 0 | 0.
|
|
226
|
-
| allocating foil | 13 | allocates |
|
|
249
|
+
| lite-pick | 0 | 0 | 0.1 |
|
|
250
|
+
| allocating foil | 13 | allocates | 2.9 |
|
|
227
251
|
|
|
228
252
|
<!-- /bench:gc -->
|
|
229
253
|
|
|
@@ -231,15 +255,14 @@ The `lite-pick` lane holds **0 major GC / 0 B/op** on the pick path; the allocat
|
|
|
231
255
|
|
|
232
256
|
### Consistent-hash disruption -- a trust gate
|
|
233
257
|
|
|
234
|
-
On a scale event (add / remove a node), what fraction of keys keep their node? The **naive-modulo** trap (`key % n`) reshuffles almost everything -- blowing every downstream cache -- while a good consistent hash moves only ~`1/n`.
|
|
258
|
+
On a scale event (add / remove a node), what fraction of keys keep their node? The **naive-modulo** trap (`key % n`) reshuffles almost everything -- blowing every downstream cache -- while a good consistent hash moves only ~`1/n`. `ConsistentHashBalancer` (the Maglev table, **M8**) is measured here beside the trap and the `1/n` ideal -- removing a backend is just marking it down, so only its keys reroute:
|
|
235
259
|
|
|
236
260
|
<!-- bench:disruption -->
|
|
237
261
|
|
|
238
|
-
| scale event | naive-modulo remap |
|
|
239
|
-
| --- | --- | --- |
|
|
240
|
-
| node removed | 98.4% | 1.6% |
|
|
241
|
-
| node added | 98.5% | 1.5% |
|
|
242
|
-
| ConsistentHash (Maglev) | SKIP -- ships in a later milestone | -- |
|
|
262
|
+
| scale event | naive-modulo remap | ConsistentHash (Maglev) | ideal (1/n) |
|
|
263
|
+
| --- | --- | --- | --- |
|
|
264
|
+
| node removed | 98.4% | 1.6% | 1.6% |
|
|
265
|
+
| node added | 98.5% | 1.5% | 1.5% |
|
|
243
266
|
|
|
244
267
|
<!-- /bench:disruption -->
|
|
245
268
|
|
|
@@ -250,6 +273,7 @@ On a scale event (add / remove a node), what fraction of keys keep their node? T
|
|
|
250
273
|
| operation | when | allocates |
|
|
251
274
|
| --- | --- | --- |
|
|
252
275
|
| `new <Strategy>Balancer(...)` | construction, once | the balancer object + its owned accumulators (SmoothWRR's Float64 `current`). The eligibility / inflight / weight views are **caller-owned**, never copied |
|
|
276
|
+
| `new ConsistentHashBalancer(...)` / `rebuild()` / `setWeight()` | cold, on build / membership / reweight | the Maglev lookup table: **`M x 4` bytes** (`~256KB` at the `65537` default `M`), a one-time `Uint32Array` allocation + an `O(M x N)` populate. `M` is **configurable down** for small pools. A health flap does **not** rebuild -- the bounded probe absorbs it |
|
|
253
277
|
| `setEligible(i, up)` | cold, on a health flip | **0** -- one byte write + an O(1) live-count adjust |
|
|
254
278
|
| `setWeight(i, w)` (SmoothWRR) | cold, on reweight | **0** -- one array write + an O(1) eligible-total adjust |
|
|
255
279
|
| `pick()` | **HOT**, per request | **0 B/op** -- proven by `torture` + `test:perf`, measured by `bench:gc` |
|
|
@@ -264,7 +288,7 @@ On a scale event (add / remove a node), what fraction of keys keep their node? T
|
|
|
264
288
|
| ALB `least_outstanding_requests` (LOR) | `LeastConnBalancer` / `P2cBalancer` |
|
|
265
289
|
| ALB anomaly mitigation / latency-aware shedding | `PeakEwmaBalancer` (latency-aware P2C) |
|
|
266
290
|
| ALB `weighted_random` + anomaly mitigation | `WeightedRandom` + `BoundedLoad` (M9/M10) |
|
|
267
|
-
| NLB flow-hash (5-tuple) | `
|
|
291
|
+
| NLB flow-hash (5-tuple) | `ConsistentHashBalancer` (Maglev table -- the same family NLB flow-hash uses, at the in-process hop) |
|
|
268
292
|
|
|
269
293
|
The composition: inbound traffic still enters through your **ALB/NLB -> service** (the edge hop AWS owns and bills); `lite-pick` governs the fan-out **after** that, the hop no AWS load balancer touches. Complementary, not a replacement -- "the hop your ALB/NLB never sees."
|
|
270
294
|
|
|
@@ -300,6 +324,8 @@ VERSION; // -> '0.6.0'
|
|
|
300
324
|
|
|
301
325
|
## Wiring it up -- `@zakkster/lite-pick/pool` (v0.5.0)
|
|
302
326
|
|
|
327
|
+
> **New to lite-pick as a load balancer?** [**RECIPES.md**](./RECIPES.md) is a beginner-to-advanced guide: it builds the kernel up into a real balancer step by step -- health/eligibility, load counters, the dispatch/settle loop, failover, latency feedback (PeakEWMA), the FE profile, and choosing a strategy. Start there; the sections below are the reference.
|
|
328
|
+
|
|
303
329
|
The kernel gives you `pick() -> index`. Real callers also need the counter ergonomics: **increment in-flight on dispatch, decrement on settle, and re-pick a *different* endpoint on failure.** That layer is async (it wraps the request), so it lives in a separate subpath -- `@zakkster/lite-pick/pool` -- and the kernel stays 0 B/op.
|
|
304
330
|
|
|
305
331
|
```js
|
package/RECIPES.md
ADDED
|
@@ -0,0 +1,379 @@
|
|
|
1
|
+
# lite-pick recipes -- from a one-liner to a real load balancer
|
|
2
|
+
|
|
3
|
+
`@zakkster/lite-pick` is a SELECTION KERNEL, not a proxy. It answers one question --
|
|
4
|
+
"which endpoint should this request go to?" -- and returns an integer index (or
|
|
5
|
+
`PICK_NONE` = -1 when nothing is eligible). A *real* load balancer is that kernel plus
|
|
6
|
+
the wiring around it:
|
|
7
|
+
|
|
8
|
+
- **eligibility** -- who is up? (a health check writes a shared bitmap)
|
|
9
|
+
- **load counters** -- how busy is each endpoint? (you own an `inflight` array)
|
|
10
|
+
- **the dispatch/settle loop** -- increment on send, decrement on finish
|
|
11
|
+
- **failover** -- if a call fails, try a different endpoint
|
|
12
|
+
- **latency feedback** -- for latency-aware routing, feed measured rtt back
|
|
13
|
+
|
|
14
|
+
These recipes build that wiring up, one layer at a time. Every array is preallocated
|
|
15
|
+
once and reused -- the pick path allocates 0 bytes.
|
|
16
|
+
|
|
17
|
+
Install: `npm i @zakkster/lite-pick` (zero runtime dependencies; ESM; Node >= 18)
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 1. The 30-second version -- round-robin over a fixed pool
|
|
22
|
+
|
|
23
|
+
```js
|
|
24
|
+
import { RoundRobinBalancer, PICK_NONE } from '@zakkster/lite-pick';
|
|
25
|
+
|
|
26
|
+
const CAP = 4; // fixed pool size
|
|
27
|
+
const eligible = new Uint8Array(CAP).fill(1); // 1 = up, 0 = down (all up here)
|
|
28
|
+
const lb = new RoundRobinBalancer(CAP, eligible);
|
|
29
|
+
|
|
30
|
+
const endpoints = ['a.svc:8080', 'b.svc:8080', 'c.svc:8080', 'd.svc:8080'];
|
|
31
|
+
|
|
32
|
+
for (let r = 0; r < 6; r++) {
|
|
33
|
+
const i = lb.pick(); // -> next eligible index, round-robin
|
|
34
|
+
if (i === PICK_NONE) throw new Error('pool is down');
|
|
35
|
+
send(endpoints[i]); // your transport; lite-pick never opens a socket
|
|
36
|
+
}
|
|
37
|
+
// picks: a, b, c, d, a, b
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
`pick()` is the whole kernel. Everything below adds a capability around it.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## 2. Fail closed -- always handle PICK_NONE
|
|
45
|
+
|
|
46
|
+
lite-pick never returns a down endpoint and never guesses. When the whole pool is
|
|
47
|
+
ineligible, `pick()` returns `PICK_NONE` (-1). Treat it as a first-class outcome:
|
|
48
|
+
|
|
49
|
+
```js
|
|
50
|
+
const i = lb.pick();
|
|
51
|
+
if (i === PICK_NONE) {
|
|
52
|
+
// shed load, return 503, or fall back -- your policy. Never index endpoints[-1].
|
|
53
|
+
return respond503();
|
|
54
|
+
}
|
|
55
|
+
send(endpoints[i]);
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`lb.live` is an O(1) count of eligible endpoints if you want to check before picking.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## 3. Wire health -> eligibility (the shared bitmap)
|
|
63
|
+
|
|
64
|
+
Eligibility is a shared `Uint8Array` (1 = pickable, 0 = down). Something else writes it
|
|
65
|
+
-- a health checker, a circuit breaker, or `@zakkster/lite-di-health` -- and `pick()`
|
|
66
|
+
only reads it. Two ways to flip a node:
|
|
67
|
+
|
|
68
|
+
```js
|
|
69
|
+
// (a) write the bitmap directly if you own it elsewhere (zero-copy, pick sees it live):
|
|
70
|
+
eligible[2] = 0; // endpoint c is down
|
|
71
|
+
|
|
72
|
+
// (b) go through the balancer so its O(1) `live` count stays exact (recommended):
|
|
73
|
+
lb.setEligible(2, false); // COLD path; idempotent; keeps `live` correct
|
|
74
|
+
lb.setEligible(2, true); // back up
|
|
75
|
+
|
|
76
|
+
lb.isEligible(2); // -> boolean, HOT, out-of-range is false (never throws)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Prefer `setEligible` when you rely on `live`. Health flapping is the writer's problem:
|
|
80
|
+
apply hysteresis/dwell in the health layer -- `pick()` stays greedy and stateless.
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## 4. Weighted pools -- SmoothWRR
|
|
85
|
+
|
|
86
|
+
When endpoints have different capacities, weight them. `SmoothWRRBalancer` owns its
|
|
87
|
+
weight state; it is the SOLE writer -- always go through `setWeight`, never mutate the
|
|
88
|
+
array directly (direct mutation desyncs the internal total = undefined behavior).
|
|
89
|
+
|
|
90
|
+
```js
|
|
91
|
+
import { SmoothWRRBalancer } from '@zakkster/lite-pick';
|
|
92
|
+
|
|
93
|
+
const weights = new Uint32Array(CAP); // the balancer manages these
|
|
94
|
+
const lb = new SmoothWRRBalancer(CAP, eligible, weights);
|
|
95
|
+
lb.setWeight(0, 5); // a is 5x
|
|
96
|
+
lb.setWeight(1, 1);
|
|
97
|
+
lb.setWeight(2, 1);
|
|
98
|
+
lb.setWeight(3, 1);
|
|
99
|
+
// pick() interleaves smoothly (nginx smooth WRR): a a b a c a a d ... not a a a a a b c d
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Use SmoothWRR when weights are known/config-driven and change rarely.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 5. Load-aware selection -- you own the `inflight` counters
|
|
107
|
+
|
|
108
|
+
P2C, LeastConn, SED, and NQ route by *current load*. That load lives in a
|
|
109
|
+
caller-owned `Uint32Array` you increment on dispatch and decrement on settle. If you
|
|
110
|
+
don't maintain it, these strategies are blind (they see every node at 0).
|
|
111
|
+
|
|
112
|
+
```js
|
|
113
|
+
import { P2cBalancer } from '@zakkster/lite-pick';
|
|
114
|
+
|
|
115
|
+
const inflight = new Uint32Array(CAP); // YOURS to maintain
|
|
116
|
+
const lb = new P2cBalancer(CAP, eligible, inflight);
|
|
117
|
+
|
|
118
|
+
async function handle(req) {
|
|
119
|
+
const i = lb.pick();
|
|
120
|
+
if (i === PICK_NONE) return respond503();
|
|
121
|
+
inflight[i]++; // DISPATCH
|
|
122
|
+
try {
|
|
123
|
+
return await send(endpoints[i], req);
|
|
124
|
+
} finally {
|
|
125
|
+
inflight[i]--; // SETTLE (always, even on error)
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
- **P2C** -- two random draws, pick the lighter. O(1), the scalable default; peak load
|
|
131
|
+
hugs the `ln ln n` band. Great from ~8 endpoints up.
|
|
132
|
+
- **LeastConn** -- exact fewest-in-flight (O(cap) scan). Best balance for small pools.
|
|
133
|
+
- **SED** / **NQ** -- weighted least-conn: pass a `weights` Uint32Array too;
|
|
134
|
+
`new SedBalancer(CAP, eligible, inflight, weights)`. NQ sends to an idle node first.
|
|
135
|
+
|
|
136
|
+
Maintaining the dispatch/settle loop by hand is easy to get wrong. Recipe 6 does it for
|
|
137
|
+
you.
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## 6. The real request loop -- `@zakkster/lite-pick/pool`
|
|
142
|
+
|
|
143
|
+
The `/pool` subpath wraps the kernel with the async dispatch/settle ergonomics so you
|
|
144
|
+
don't hand-maintain `inflight`. The kernel `pick()` stays 0 B/op; `Pool.run` is a normal
|
|
145
|
+
async wrapper on top.
|
|
146
|
+
|
|
147
|
+
```js
|
|
148
|
+
import { P2cBalancer } from '@zakkster/lite-pick';
|
|
149
|
+
import { Pool } from '@zakkster/lite-pick/pool';
|
|
150
|
+
|
|
151
|
+
const inflight = new Uint32Array(CAP);
|
|
152
|
+
const lb = new P2cBalancer(CAP, eligible, inflight);
|
|
153
|
+
const pool = new Pool(lb, inflight); // SAME inflight array the balancer reads
|
|
154
|
+
|
|
155
|
+
// Pool does pick -> inflight++ -> await fn -> inflight-- (in a finally) for you:
|
|
156
|
+
const body = await pool.run((i, signal) => fetchFrom(endpoints[i], { signal }));
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
`run(fn, opts?)` rejects with a `code: 'LITE_PICK_NONE'` error when the pool is down.
|
|
160
|
+
`fn(endpoint, signal)` receives the chosen index and the (optional) AbortSignal.
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## 7. Failover -- try a different endpoint on error
|
|
165
|
+
|
|
166
|
+
Set `tries > 1`. On a thrown error, Pool keeps the failed node's in-flight count
|
|
167
|
+
elevated and re-picks -- so a load-aware strategy naturally steers to a DIFFERENT
|
|
168
|
+
endpoint -- up to `tries` attempts, then rejects with the last error.
|
|
169
|
+
|
|
170
|
+
```js
|
|
171
|
+
const body = await pool.run(
|
|
172
|
+
(i, signal) => fetchFrom(endpoints[i], { signal }),
|
|
173
|
+
{ tries: 3, signal: req.signal } // up to 3 distinct endpoints
|
|
174
|
+
);
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Boundary: Pool owns **spatial** failover (move across the pool, once each, in-process).
|
|
178
|
+
The caller or your query cache owns **temporal** retry (backoff, staleness, dedup).
|
|
179
|
+
Don't double-own them. If `signal` aborts after a failure, failover stops and the abort
|
|
180
|
+
propagates.
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## 8. Latency-aware routing -- PeakEWMA with rtt feedback
|
|
185
|
+
|
|
186
|
+
PeakEWMA (latency-aware P2C, Finagle's peak-EWMA) steers away from *slow* endpoints,
|
|
187
|
+
not just busy ones. It scores each candidate `(inflight + 1) x ewma(rtt)`, so a node
|
|
188
|
+
that got slow gets less traffic even if its connection count looks fine. It needs two
|
|
189
|
+
things you didn't need before: a **clock** (`now`, caller-supplied nanoseconds) and
|
|
190
|
+
**rtt feedback** (`recordRtt`).
|
|
191
|
+
|
|
192
|
+
Manual loop:
|
|
193
|
+
|
|
194
|
+
```js
|
|
195
|
+
import { PeakEwmaBalancer } from '@zakkster/lite-pick';
|
|
196
|
+
|
|
197
|
+
const TAU_NS = 30_000_000; // 30ms half-life for the EWMA decay
|
|
198
|
+
const inflight = new Uint32Array(CAP);
|
|
199
|
+
const lb = new PeakEwmaBalancer(CAP, eligible, inflight, TAU_NS);
|
|
200
|
+
const nowNs = () => Number(process.hrtime.bigint());
|
|
201
|
+
|
|
202
|
+
async function handle(req) {
|
|
203
|
+
const now = nowNs();
|
|
204
|
+
const i = lb.pick(now); // decay-on-read, 0 B/op
|
|
205
|
+
if (i === PICK_NONE) return respond503();
|
|
206
|
+
inflight[i]++;
|
|
207
|
+
const start = nowNs();
|
|
208
|
+
try {
|
|
209
|
+
return await send(endpoints[i], req);
|
|
210
|
+
} finally {
|
|
211
|
+
inflight[i]--;
|
|
212
|
+
lb.recordRtt(i, nowNs() - start, nowNs()); // FEEDBACK: measured rtt, snaps up / decays down
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Or let Pool do the feedback for you -- pass a `clock`; Pool drives `pick(now)` and calls
|
|
218
|
+
`recordRtt` on a successful settle when the balancer supports it:
|
|
219
|
+
|
|
220
|
+
```js
|
|
221
|
+
import { Pool } from '@zakkster/lite-pick/pool';
|
|
222
|
+
const pool = new Pool(lb, inflight);
|
|
223
|
+
const body = await pool.run(
|
|
224
|
+
(i, signal) => fetchFrom(endpoints[i], { signal }),
|
|
225
|
+
{ clock: () => Number(process.hrtime.bigint()), tries: 2 }
|
|
226
|
+
);
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Notes:
|
|
230
|
+
- **Cold start** (no samples yet) degrades gracefully to least-connections -- never NaN.
|
|
231
|
+
- `now` must be a FINITE number. A non-finite `now` degrades to P2C-random (no throw).
|
|
232
|
+
- Pick `tauNs` around your p50-p90 rtt: smaller = reacts faster to a slowdown, larger =
|
|
233
|
+
steadier. It IS the anti-flap smoothing; no extra dwell needed.
|
|
234
|
+
- Measured effect: with one node at 10x latency, PeakEWMA sends it a tiny fraction of
|
|
235
|
+
the traffic P2C-over-inflight would, and cuts service p99 sharply.
|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
## 9. Sticky / affinity routing -- ConsistentHash (you hash the key)
|
|
240
|
+
|
|
241
|
+
When a request must land on the **same** backend every time -- a session pinned to a shard,
|
|
242
|
+
a cache key kept warm, a stateful worker -- use `ConsistentHashBalancer`. It is a prebuilt
|
|
243
|
+
**Maglev table**, so `pick(keyHash)` is `O(1)` and `0 B/op`, and scaling the pool moves only
|
|
244
|
+
~`1/N` of keys (not the whole keyspace, the way `key % n` would).
|
|
245
|
+
|
|
246
|
+
The one rule: **you hash the key to an INTEGER**, on the cold path. Per-pick *string* hashing
|
|
247
|
+
allocates -- the single zero-GC hazard -- so `pick()` takes a number and `lite-pick` ships no
|
|
248
|
+
hashing dependency. Any small integer hash works; FNV-1a is a fine default:
|
|
249
|
+
|
|
250
|
+
```js
|
|
251
|
+
import { ConsistentHashBalancer, PICK_NONE } from '@zakkster/lite-pick';
|
|
252
|
+
|
|
253
|
+
// A tiny FNV-1a over a string -- done ONCE per key, never inside pick().
|
|
254
|
+
function fnv1a(s) {
|
|
255
|
+
let h = 0x811c9dc5;
|
|
256
|
+
for (let i = 0; i < s.length; i++) { h ^= s.charCodeAt(i); h = Math.imul(h, 0x01000193); }
|
|
257
|
+
return h >>> 0;
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
const eligible = new Uint8Array(CAP).fill(1);
|
|
261
|
+
const lb = new ConsistentHashBalancer(CAP, eligible); // default M = 65537 (prime)
|
|
262
|
+
|
|
263
|
+
function routeFor(sessionId) {
|
|
264
|
+
const i = lb.pick(fnv1a(sessionId)); // same session -> same backend, at fixed membership
|
|
265
|
+
if (i === PICK_NONE) return respond503();
|
|
266
|
+
return endpoints[i];
|
|
267
|
+
}
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
Notes:
|
|
271
|
+
- **Remove a backend by marking it down** (`lb.setEligible(i, false)`) -- the table is
|
|
272
|
+
untouched, so only that backend's keys reroute (~`1/N`); everyone else stays put. A health
|
|
273
|
+
flap costs nothing (the bounded probe absorbs it) -- it never rebuilds.
|
|
274
|
+
- **Add / reweight** rebuilds the table (cold): `new ConsistentHashBalancer(N + 1, ...)`, or
|
|
275
|
+
`lb.setWeight(i, w)` / `lb.rebuild()`. Pass a `Uint32Array` of weights for proportional shares.
|
|
276
|
+
- **Cost:** the table is `M x 4` bytes (~256KB at the `65537` default) -- a cold, one-time
|
|
277
|
+
allocation. Turn `M` down for a small pool (any prime `>= N`).
|
|
278
|
+
- `pick(keyHash)` coerces `keyHash >>> 0` and never throws; it returns `PICK_NONE` only when the
|
|
279
|
+
pool is down or no eligible backend is reachable within the probe bound.
|
|
280
|
+
|
|
281
|
+
---
|
|
282
|
+
|
|
283
|
+
## 10. Wire it into a query cache (lite-query, or any fetcher)
|
|
284
|
+
|
|
285
|
+
`liteQueryFetcher` adapts a Pool into a `({ key, signal }) => Promise` fetcher -- the
|
|
286
|
+
shape lite-query (or any cache/route-loader) expects. It imports nothing from lite-query
|
|
287
|
+
(duck-typed), so it works with any fetcher-shaped consumer.
|
|
288
|
+
|
|
289
|
+
```js
|
|
290
|
+
import { Pool, liteQueryFetcher } from '@zakkster/lite-pick/pool';
|
|
291
|
+
|
|
292
|
+
const pool = new Pool(lb, inflight);
|
|
293
|
+
const fetcher = liteQueryFetcher(
|
|
294
|
+
pool,
|
|
295
|
+
({ endpoint, key, signal }) => fetchFrom(endpoints[endpoint], { key, signal }),
|
|
296
|
+
{ tries: 2 }
|
|
297
|
+
);
|
|
298
|
+
// hand `fetcher` to your query cache; each cache miss fans out across the pool with failover.
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
---
|
|
302
|
+
|
|
303
|
+
## 11. Choosing a strategy
|
|
304
|
+
|
|
305
|
+
| Strategy | Route by | Cost | Reach for it when |
|
|
306
|
+
|---------------|---------------------|-------------|-------------------|
|
|
307
|
+
| RoundRobin | position | O(1) amort. | uniform endpoints, no load signal |
|
|
308
|
+
| SmoothWRR | static weight | O(cap) | known/config capacities, smooth interleave |
|
|
309
|
+
| P2C | in-flight (approx) | O(1) | the scalable default from ~8 nodes up |
|
|
310
|
+
| LeastConn | in-flight (exact) | O(cap) | small pools, tightest connection balance |
|
|
311
|
+
| SED | in-flight / weight | O(cap) | weighted least-conn |
|
|
312
|
+
| NQ | idle-first else SED | O(cap)/O(1) | worker pools -- never queue while a worker is free |
|
|
313
|
+
| PeakEWMA | in-flight x ewma(rtt)| O(1) | heterogeneous / flaky backends; steer around slow nodes |
|
|
314
|
+
| ConsistentHash| key hash (sticky) | O(1) | affinity/sticky: same key -> same backend, minimal disruption on scale |
|
|
315
|
+
|
|
316
|
+
The load-aware strategies read the SAME `inflight` array live, so you can swap among them
|
|
317
|
+
without rewiring; ConsistentHash instead takes an integer key per pick (recipe 9).
|
|
318
|
+
|
|
319
|
+
---
|
|
320
|
+
|
|
321
|
+
## 12. The "FE profile" -- a browser / front-end client
|
|
322
|
+
|
|
323
|
+
For a front-end client picking among origins a handful of times per second (not a
|
|
324
|
+
zero-GC hot loop), the recommended profile is **PeakEWMA + health/eligibility only** --
|
|
325
|
+
latency-aware choice across origins with a fail-closed eligibility view -- and skip the
|
|
326
|
+
bounded-load / AZ / occupancy machinery. Feed rtt from your `fetch` timings via
|
|
327
|
+
`recordRtt`.
|
|
328
|
+
|
|
329
|
+
---
|
|
330
|
+
|
|
331
|
+
## 13. Compose with the suite (all optional, all duck-typed)
|
|
332
|
+
|
|
333
|
+
lite-pick declares ZERO hard dependencies and an EMPTY `peerDependencies`. Each seam is
|
|
334
|
+
a shared TypedArray or a duck-typed shape, so you wire in a sibling only if you use it:
|
|
335
|
+
|
|
336
|
+
- `@zakkster/lite-di-health` -- writes the eligibility bitmap from health checks.
|
|
337
|
+
- `@zakkster/lite-statechart` -- a circuit breaker that flips eligibility.
|
|
338
|
+
- `@zakkster/lite-query` -- the cache behind `liteQueryFetcher` (recipe 10).
|
|
339
|
+
- `@zakkster/lite-sketch` -- `DDSketch` for a p99-aware PeakEWMA variant (deferred).
|
|
340
|
+
- `@zakkster/lite-filter` -- a hot-key / known-key oracle at the ConsistentHash key-routing
|
|
341
|
+
layer (warm/cold only, never the pick path; deferred).
|
|
342
|
+
- `@zakkster/lite-await` -- hedging (race the P2C second choice past a percentile).
|
|
343
|
+
|
|
344
|
+
None is required; the kernel runs over raw TypedArrays with nothing installed.
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
## 14. Zero-GC discipline (why the pick path stays 0 B/op)
|
|
349
|
+
|
|
350
|
+
- Allocate `eligible` / `inflight` / `weights` ONCE at startup and reuse them. Never
|
|
351
|
+
build arrays per pick.
|
|
352
|
+
- The counters are YOURS -- mutate them in place (`inflight[i]++/--`), don't replace them.
|
|
353
|
+
- `pick()` / `pick(now)` and `recordRtt` allocate nothing. The only async allocation is
|
|
354
|
+
the promise your own `fn` already creates (disclosed; `Pool.run` adds O(1) integer ops
|
|
355
|
+
plus one small per-run array).
|
|
356
|
+
- Fixed capacity: the pool size is set at construction and the backing arrays never
|
|
357
|
+
reallocate.
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
## 15. Gotchas
|
|
362
|
+
|
|
363
|
+
- **PICK_NONE (-1)** is always possible -- handle it before indexing (recipe 2).
|
|
364
|
+
- **SmoothWRR weights** must go through `setWeight`; direct array mutation is UB.
|
|
365
|
+
- **ConsistentHash takes an INTEGER key** -- hash strings yourself, cold (recipe 9). Never
|
|
366
|
+
`pick(someString)` on the hot path; `M` must be a prime `>= capacity`.
|
|
367
|
+
- **Load-aware strategies need the dispatch/settle loop** -- forget the `inflight--` in
|
|
368
|
+
a `finally` and load leaks upward forever. Use `/pool` (recipe 6) to avoid it.
|
|
369
|
+
- **PeakEWMA needs a finite `now`** and rtt feedback -- without `recordRtt` it behaves
|
|
370
|
+
like LeastConn (cold-start baseline).
|
|
371
|
+
- **Eligibility is read-only to `pick()`** -- the health layer writes it; the balancer
|
|
372
|
+
only reads (or maintains `live` via `setEligible`).
|
|
373
|
+
- **lite-pick is not a proxy** -- it returns an index; you own transport, retries/backoff
|
|
374
|
+
(temporal), health checking, and the socket.
|
|
375
|
+
|
|
376
|
+
---
|
|
377
|
+
|
|
378
|
+
See also: `README.md` (overview + gates), `llms.txt` (full API surface),
|
|
379
|
+
`decisions/` (the ADRs behind each design call), `ROADMAP.md` (what's next).
|
package/llms.txt
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.8.0
|
|
4
4
|
License: MIT (c) Zahary Shinikchiev <shinikchiev@yahoo.com>
|
|
5
5
|
Runtime dependencies: none. ESM only. ASCII-only source. sideEffects: false.
|
|
6
6
|
Node: >= 18.
|
|
@@ -14,16 +14,26 @@ reading pre-allocated views that siblings or the caller write, and returning an
|
|
|
14
14
|
The complementary evidence lite-pick ships is a measured balance-quality anchor (peak-to-
|
|
15
15
|
average load vs the strategy's theoretical ceiling) alongside the 0 B/op pick witness.
|
|
16
16
|
|
|
17
|
-
0.
|
|
17
|
+
0.8.0 ships the substrate seams + eight strategies: RoundRobin, SmoothWRR (the weighted
|
|
18
18
|
default), P2C (power-of-two-choices -- also the O(1) least-connections APPROXIMATION), the
|
|
19
|
-
EXACT LeastConn family (LeastConn, SED, NQ),
|
|
20
|
-
|
|
21
|
-
`
|
|
22
|
-
|
|
23
|
-
`
|
|
24
|
-
ConsistentHash
|
|
25
|
-
|
|
26
|
-
|
|
19
|
+
EXACT LeastConn family (LeastConn, SED, NQ), PeakEWMA (latency-aware P2C), and ConsistentHash
|
|
20
|
+
(a Maglev lookup table -- sticky/affinity routing). It exports `VERSION`, the fail-closed
|
|
21
|
+
sentinel `PICK_NONE` (-1), a deterministic `Prng` (xorshift32), `BalancerBase` (the shared
|
|
22
|
+
read-only eligibility seam + O(1) live count), `RoundRobinBalancer`, `SmoothWRRBalancer`,
|
|
23
|
+
`P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`,
|
|
24
|
+
`ConsistentHashBalancer`, and the ConsistentHash constants `CH_DEFAULT_M` (65537) / `CH_PROBE_LIMIT`
|
|
25
|
+
(64). The remaining strategies land one per session (see ROADMAP.md): BoundedLoad, WeightedRandom.
|
|
26
|
+
The EXACT-O(log n) fewest-in-flight variant is a deferred @zakkster/lite-logn `BinaryHeap`
|
|
27
|
+
optional-peer seam (decisions/0006), not this exact-O(cap) scan.
|
|
28
|
+
|
|
29
|
+
M8 (0.8.0) adds ConsistentHashBalancer (decisions/0010): a prebuilt Maglev lookup table (IPVS `mh`,
|
|
30
|
+
Meta Katran, Cilium) mapping a caller-supplied INTEGER key to a backend, O(1) / 0 B/op, with minimal
|
|
31
|
+
disruption on a scale event (~1/N keys move, measured ~1.6% vs the naive-modulo foil's ~98%). The
|
|
32
|
+
vs-AWS positioning: NLB flow-hash (5-tuple) is a Maglev consistent hash -- ConsistentHashBalancer is
|
|
33
|
+
that same family at the in-process hop AWS never sees. DEFERRED optional-peer seams (import NOTHING;
|
|
34
|
+
peerDependencies STAYS `{}` until a shipped path imports one): a @zakkster/lite-filter hot-key /
|
|
35
|
+
known-key oracle (BlockedBloom etc.) at the KEY-routing layer (warm/cold only, never the pick path),
|
|
36
|
+
and a @zakkster/lite-o1 `EliasFano` ring alternative to the table.
|
|
27
37
|
|
|
28
38
|
M7 (0.7.0) adds PeakEwmaBalancer (decisions/0009). The FE PROFILE: for a browser / front-end
|
|
29
39
|
client (a handful of picks per second, not a zero-GC hot loop), the recommended profile is
|
|
@@ -150,6 +160,30 @@ the contract + balance + tail -- never an "N times faster" headline (decisions/0
|
|
|
150
160
|
- CONTRACT: `now` and `sampleNs` MUST be FINITE numbers. `recordRtt` THROWS on a non-finite
|
|
151
161
|
argument (warm path); `pick(now)` NEVER throws (fail-closed contract), so a non-finite `now`
|
|
152
162
|
yields P2C-random selection, not an error.
|
|
163
|
+
- `ConsistentHashBalancer extends BalancerBase` -- class. Sticky/affinity routing via a prebuilt
|
|
164
|
+
MAGLEV lookup table (M8, IPVS `mh` / Meta Katran / Cilium).
|
|
165
|
+
- `new ConsistentHashBalancer(capacity, eligible, weights?=null, m?=65537, seed?=0x9e3779b9)` --
|
|
166
|
+
`weights` is an optional caller Uint32Array (length >= capacity) COPIED into balancer-owned
|
|
167
|
+
weights (null = equal). `m` is the table size: a PRIME, > 1, and >= capacity (the guard: capacity
|
|
168
|
+
> m throws -- it would overfill / starve backends). Validates typeof-first BEFORE allocating the
|
|
169
|
+
table. Builds the table COLD (O(M x N) weighted Maglev populate; the M x 4-byte table is ~256KB at
|
|
170
|
+
the 65537 default -- disclosed cold cost, M configurable down for small pools).
|
|
171
|
+
- `pick(keyHash)` -> number. Maps an INTEGER key: slot = (keyHash >>> 0) % M, read `lookup[slot]`,
|
|
172
|
+
and if down forward-probe up to CH_PROBE_LIMIT (64) slots for the next eligible backend. O(1),
|
|
173
|
+
0 B/op. `keyHash` is coerced `>>> 0` (NaN -> 0) and pick NEVER throws. `PICK_NONE` when the pool is
|
|
174
|
+
down OR no eligible backend is reachable within the bound (a mass outage may fail closed even if a
|
|
175
|
+
far eligible slot exists -- safe, never a dead pick). KEY IS A CALLER INTEGER -- no per-pick string
|
|
176
|
+
hashing (the one zero-GC hazard); hash string keys yourself, cold. No hashing dependency added.
|
|
177
|
+
- `setWeight(i, w)` -> void. COLD. Set backend i's weight (uint32) and REBUILD the table.
|
|
178
|
+
- `rebuild()` -> void. COLD. Rebuild the table from the current owned weights (e.g. after a
|
|
179
|
+
membership change). A health flap does NOT rebuild -- the bounded probe absorbs it.
|
|
180
|
+
- `tableSize` -- readonly number. The Maglev table size M (prime).
|
|
181
|
+
- Minimal disruption: removing a backend is `setEligible(i, false)` (no rebuild), so only ~1/N keys
|
|
182
|
+
reroute (measured ~1.6% on remove/add vs the naive-modulo foil's ~98%; benchmark/Disruption.mjs).
|
|
183
|
+
- vs-AWS: NLB flow-hash (5-tuple) is a Maglev consistent hash -- this is that family at the
|
|
184
|
+
in-process hop. DEFERRED optional-peer seams (import nothing, peerDependencies STAYS `{}`): a
|
|
185
|
+
@zakkster/lite-filter hot-key oracle at the key-routing layer (warm/cold only), and a
|
|
186
|
+
@zakkster/lite-o1 `EliasFano` ring alternative to the table (decisions/0010).
|
|
153
187
|
|
|
154
188
|
## Subpath: @zakkster/lite-pick/pool -- the ergonomic request layer (M5, Pool.js)
|
|
155
189
|
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zakkster/lite-pick",
|
|
3
3
|
"author": "Zahary Shinikchiev <shinikchiev@yahoo.com>",
|
|
4
|
-
"version": "0.
|
|
5
|
-
"description": "Zero-dependency, zero-GC load-balancing selection kernel: one hot pick() -> endpoint index over a fixed pool, 0 B/op steady-state. A pure selector (consumes health/circuit state, never a proxy) for the in-process hop, complementary to AWS NLB/ALB. Tree-shakeable ESM roster: RoundRobin, SmoothWRR, P2C, LeastConn, SED, NQ, PeakEWMA (latency-aware peak-EWMA),
|
|
4
|
+
"version": "0.8.0",
|
|
5
|
+
"description": "Zero-dependency, zero-GC load-balancing selection kernel: one hot pick() -> endpoint index over a fixed pool, 0 B/op steady-state. A pure selector (consumes health/circuit state, never a proxy) for the in-process hop, complementary to AWS NLB/ALB. Tree-shakeable ESM roster: RoundRobin, SmoothWRR, P2C, LeastConn, SED, NQ, PeakEWMA (latency-aware peak-EWMA), and ConsistentHash (Maglev sticky/affinity routing, minimal disruption); the /pool subpath adds dispatch/settle counters + failover and a duck-typed query-cache fetcher.",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./Pick.js",
|
|
8
8
|
"module": "./Pick.js",
|
|
@@ -28,6 +28,7 @@
|
|
|
28
28
|
"Pool.d.ts",
|
|
29
29
|
"llms.txt",
|
|
30
30
|
"README.md",
|
|
31
|
+
"RECIPES.md",
|
|
31
32
|
"CHANGELOG.md",
|
|
32
33
|
"LICENSE"
|
|
33
34
|
],
|
|
@@ -73,6 +74,8 @@
|
|
|
73
74
|
"peak-ewma",
|
|
74
75
|
"latency-aware",
|
|
75
76
|
"consistent-hashing",
|
|
77
|
+
"consistent-hash",
|
|
78
|
+
"sticky",
|
|
76
79
|
"maglev",
|
|
77
80
|
"bounded-load",
|
|
78
81
|
"weighted-random",
|