@zakkster/lite-pick 0.7.0 → 0.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +83 -0
- package/Pick.js +41 -16
- package/README.md +52 -8
- package/RECIPES.md +329 -0
- package/llms.txt +49 -11
- package/package.json +2 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,89 @@ All notable changes to `@zakkster/lite-pick` are documented here. The format fol
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.7.2] - 2026-09-23
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `RECIPES.md` -- a beginner-to-advanced usage guide that builds the selection kernel up into a
|
|
12
|
+
real load balancer (health/eligibility wiring, caller-owned in-flight counters, the
|
|
13
|
+
dispatch/settle loop, `/pool` failover, PeakEWMA rtt feedback, the FE profile, a strategy
|
|
14
|
+
decision table, suite composition, zero-GC discipline, and gotchas). Added to the published
|
|
15
|
+
package (`files[]`) and linked from the README.
|
|
16
|
+
|
|
17
|
+
Docs-only release: no source or behavior change from 0.7.1 (the `VERSION` stamp is bumped for the
|
|
18
|
+
three-place sync).
|
|
19
|
+
|
|
20
|
+
## [0.7.1] - 2026-09-23
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- PeakEWMA cold-start / unsampled-node scoring under real large-magnitude clocks. An unsampled node
|
|
25
|
+
now scores at its UNDECAYED baseline (`_stamp` initialized to a negative sentinel, read as the
|
|
26
|
+
1.0 baseline) = graceful least-connections, instead of `exp(-now/tau)` underflowing to 0 and
|
|
27
|
+
collapsing a cold pool to random selection. The FIRST `recordRtt` sample now initializes the EWMA
|
|
28
|
+
EXACTLY to the sample (clock-magnitude-independent); the Finagle peak rule applies from the second
|
|
29
|
+
sample on. `pick()` stays a pure 0 B/op read (a per-candidate sentinel compare, no allocation).
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- Completes the 0.7.0 packaging: synced the `llms.txt` version stamp, added the PeakEWMA
|
|
34
|
+
README / CHANGELOG sections + `decisions/0009-peakewma.md`, and regenerated the benchmark
|
|
35
|
+
`results.json`. Documented that `pick(now)` / `recordRtt` require a FINITE `now` -- `recordRtt`
|
|
36
|
+
throws on a non-finite argument; `pick(now)` never throws (fail-closed) and degrades a non-finite
|
|
37
|
+
`now` to P2C-random selection. No API or behavior change beyond the cold-start fix.
|
|
38
|
+
|
|
39
|
+
## [0.7.0] - 2026-09-23
|
|
40
|
+
|
|
41
|
+
M7: `PeakEwmaBalancer` -- latency-aware power-of-two-choices (Twitter Finagle's peak-EWMA). A
|
|
42
|
+
STRATEGY-APPEND session: one class is added to `Pick.js`; the other strategies are byte-identical,
|
|
43
|
+
only the header roster/count and the `VERSION` stamp change. `peerDependencies` stays `{}`.
|
|
44
|
+
|
|
45
|
+
### Added
|
|
46
|
+
|
|
47
|
+
- `PeakEwmaBalancer extends BalancerBase` (`Pick.js`, `Pick.d.ts`) -- `new PeakEwmaBalancer(capacity,
|
|
48
|
+
eligible, inflight, tauNs, seed?)`. `pick(now)` draws two distinct eligible endpoints (reusing
|
|
49
|
+
`P2cBalancer`'s rejection-sampling `_draw`) and returns the lower `cost = (inflight + 1) x
|
|
50
|
+
ewmaAt(now)`, tie to the first draw; `O(d)=O(1)`. `ewmaAt(i, now)` decays ON READ
|
|
51
|
+
(`_ewma[i] * exp(-(now - _stamp[i]) / tau)`), so `pick()` never writes and is **0 B/op**.
|
|
52
|
+
`recordRtt(i, sampleNs, now)` is the warm feedback path (the Finagle peak rule: snap up to a
|
|
53
|
+
larger sample, decay down over `~tau`), also **0 B/op** on the success path. `now` / `sampleNs`
|
|
54
|
+
are caller-supplied nanoseconds. The EWMA state (`_ewma` / `_stamp`, `Float64Array`) is
|
|
55
|
+
balancer-owned; `inflight` is the caller's `Uint32Array` read live. Cold start seeds the EWMA to
|
|
56
|
+
`1.0` -> graceful least-connections, never `NaN`. Constructor and `recordRtt` validate
|
|
57
|
+
typeof-first, before allocation. Anti-flap = the half-life, no extra dwell.
|
|
58
|
+
- `Pool.run` opt-in latency feedback (`Pool.js`, `Pool.d.ts`): when `opts.clock` (a caller-owned
|
|
59
|
+
nanosecond source) is supplied AND the balancer duck-types `recordRtt`, Pool drives `pick(now)`
|
|
60
|
+
and records the settled rtt on success. Otherwise the hook is inert -- Pool stays generic, the
|
|
61
|
+
in-flight counter stays net-zero, and abort/failover are unchanged.
|
|
62
|
+
- `test/PeakEWMA.test.js` -- the boundary suite (cold-start valid + never-NaN, snap-up, decay to
|
|
63
|
+
sample/e at dt=tau within ~1%, slow-node avoidance, fail-closed, tie-break to the first draw =
|
|
64
|
+
identical to P2C on the same seed, constructor + recordRtt validation throws, flap churn).
|
|
65
|
+
- `test/balance.mjs` -- the LATENCY ANCHOR: a closed-loop single-server-per-node queue with one node
|
|
66
|
+
at 10x service time. Measured: PeakEWMA slow-node share ~0.007% vs P2C ~1.47% (<= 25% of P2C);
|
|
67
|
+
PeakEWMA service p99 ~1950ns vs P2C ~14500ns (>= 20% lower); random foil worse than both.
|
|
68
|
+
- `test/torture.mjs` -- PeakEWMA retention + `pick(now)` and `recordRtt()` 0 B/op phases.
|
|
69
|
+
- `test/perf/PerfGate.test.mjs` -- `PeakEwmaBalancer.pick(now)` + `recordRtt()` zero-alloc scenarios
|
|
70
|
+
and a `pick(now)`-boxed-into-a-fresh-array `mustFail` tooth.
|
|
71
|
+
- `test/witness.mjs` -- PeakEWMA subject, `const` (O(d)=O(1)) flat flag; work-rate flatness ~0.89
|
|
72
|
+
(the `Math.exp` runs ~2x/pick and stays flat -- the cached 2^-k decay-table fallback was NOT
|
|
73
|
+
needed).
|
|
74
|
+
- `test/fuzz.mjs` -- PeakEWMA subject: `_ewma` / `_stamp` stay finite and `PICK_NONE` holds iff the
|
|
75
|
+
pickable mass is 0 under a `recordRtt` / `pick(now)` / `setEligible` barrage.
|
|
76
|
+
- `test/types/pick.test-d.ts` -- PeakEwmaBalancer type-surface smoke.
|
|
77
|
+
- `benchmark/Matrix.mjs` PeakEWMA throughput subject; PeakEWMA lanes in `benchmark/GcBlastRadius.mjs`
|
|
78
|
+
(same maxMajor 0 / 0 B/op / bounded-pause contract) and `benchmark/Fairness.mjs` (latency
|
|
79
|
+
steering); `benchmark/results.json` regenerated (version 0.7.0), `bench:verify` green.
|
|
80
|
+
- `decisions/0009-peakewma.md` -- the ADR (latency-aware P2C, decay-on-read, the Finagle peak rule,
|
|
81
|
+
caller-supplied clock, balancer-owned state, deferred DDSketch-p99, anti-flap = half-life).
|
|
82
|
+
|
|
83
|
+
### Changed
|
|
84
|
+
|
|
85
|
+
- `Pick.js` header roster/count (six -> seven strategies), `VERSION` 0.6.0 -> 0.7.0; `package.json`
|
|
86
|
+
version + description; `llms.txt` version + PeakEWMA surface + the FE-profile note + the deferred
|
|
87
|
+
DDSketch-p99 note; `README.md` PeakEWMA section + FE profile + the AWS anomaly-mitigation mapping
|
|
88
|
+
row.
|
|
89
|
+
|
|
7
90
|
## [0.6.0] - 2026-09-23
|
|
8
91
|
|
|
9
92
|
M6: the benchmark suite (ROADMAP.md M6). An EVIDENCE session -- no API change. `Pick.js` and
|
package/Pick.js
CHANGED
|
@@ -51,7 +51,7 @@
|
|
|
51
51
|
*/
|
|
52
52
|
|
|
53
53
|
/** Version stamp. Synced across package.json and llms.txt (three-place rule). */
|
|
54
|
-
export const VERSION = '0.7.
|
|
54
|
+
export const VERSION = '0.7.2';
|
|
55
55
|
|
|
56
56
|
/**
|
|
57
57
|
* Fail-closed sentinel returned by pick() when no endpoint is eligible.
|
|
@@ -602,9 +602,17 @@ export class NqBalancer extends BalancerBase {
|
|
|
602
602
|
* between `pick(now)` and `recordRtt(i, sampleNs, now)`, so the whole strategy is deterministic
|
|
603
603
|
* and testable and allocates nothing.
|
|
604
604
|
*
|
|
605
|
-
* Cold start: `_ewma` seeds to 1.0 and `_stamp` to
|
|
606
|
-
*
|
|
607
|
-
* (
|
|
605
|
+
* Cold start: `_ewma` seeds to 1.0 and `_stamp` to a NEGATIVE "unsampled" sentinel (-1). While a
|
|
606
|
+
* node is unsampled `ewmaAt` returns the baseline 1.0 UNDECAYED, so before any sample cost(i) =
|
|
607
|
+
* (inflight[i] + 1) x 1 and PeakEWMA degrades GRACEFULLY to plain least-connections (P2C-over-
|
|
608
|
+
* inflight) REGARDLESS of the caller's clock magnitude -- a plain `_stamp = 0` would decay as
|
|
609
|
+
* exp(-now/tau) -> 0 under a real large clock and collapse a cold pool to random. The first
|
|
610
|
+
* `recordRtt` initializes the EWMA EXACTLY to the sample (clock-independent); the peak rule applies
|
|
611
|
+
* only from the second sample on. It is never NaN.
|
|
612
|
+
*
|
|
613
|
+
* Contract: `now` (in `pick(now)` / `recordRtt`) and `sampleNs` MUST be FINITE numbers. `recordRtt`
|
|
614
|
+
* throws on a non-finite argument (the warm path); `pick(now)` never throws (the fail-closed
|
|
615
|
+
* contract), so a non-finite `now` yields P2C-random selection rather than an error.
|
|
608
616
|
*
|
|
609
617
|
* Anti-flap (ADR 0002, ADR 0009): the EWMA half-life IS the smoothing -- a single slow sample
|
|
610
618
|
* snaps the cost up instantly and it decays back over ~tau, so there is NO extra dwell/hysteresis.
|
|
@@ -643,8 +651,12 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
643
651
|
this._tau = tauNs;
|
|
644
652
|
this._rng = new Prng(seed);
|
|
645
653
|
this._ewma = new Float64Array(capacity);
|
|
646
|
-
this._stamp = new Float64Array(capacity);
|
|
647
|
-
|
|
654
|
+
this._stamp = new Float64Array(capacity);
|
|
655
|
+
// Cold start: _ewma seeds to 1.0 and _stamp to a NEGATIVE "unsampled" sentinel (-1). The
|
|
656
|
+
// sentinel makes ewmaAt read the baseline UNDECAYED (graceful LeastConn) regardless of the
|
|
657
|
+
// caller's clock magnitude -- a plain _stamp=0 would decay as exp(-now/tau) -> 0 under a
|
|
658
|
+
// real large-magnitude clock and collapse a cold pool to random selection.
|
|
659
|
+
for (let i = 0; i < capacity; i++) { this._ewma[i] = 1.0; this._stamp[i] = -1; }
|
|
648
660
|
}
|
|
649
661
|
|
|
650
662
|
/**
|
|
@@ -659,20 +671,26 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
659
671
|
|
|
660
672
|
/**
|
|
661
673
|
* The decayed EWMA rtt estimate for endpoint i at time `now` (ns). Pure READ -- exponential
|
|
662
|
-
* decay applied on read, never written.
|
|
674
|
+
* decay applied on read, never written. An UNSAMPLED node (`_stamp < 0`) reads its baseline
|
|
675
|
+
* 1.0 UNDECAYED (graceful LeastConn), so a cold pool never underflows to 0 under a real
|
|
676
|
+
* large-magnitude clock. `now` must be finite. Zero-alloc.
|
|
663
677
|
* @param {number} i
|
|
664
678
|
* @param {number} now caller-supplied nanoseconds
|
|
665
679
|
* @returns {number}
|
|
666
680
|
*/
|
|
667
681
|
ewmaAt(i, now) {
|
|
668
|
-
|
|
682
|
+
const s = this._stamp[i];
|
|
683
|
+
if (s < 0) return this._ewma[i]; // unsampled: undecayed baseline, clock-magnitude-independent
|
|
684
|
+
return this._ewma[i] * Math.exp(-(now - s) / this._tau);
|
|
669
685
|
}
|
|
670
686
|
|
|
671
687
|
/**
|
|
672
688
|
* Record an rtt SAMPLE for endpoint i at time `now` (the warm feedback path -- NOT the hot
|
|
673
|
-
* pick path). The
|
|
674
|
-
*
|
|
675
|
-
*
|
|
689
|
+
* pick path). The FIRST sample (an unsampled node, `_stamp < 0`) initializes the EWMA EXACTLY
|
|
690
|
+
* to the sample, clock-magnitude-independent. Thereafter the Finagle peak rule applies: decay
|
|
691
|
+
* the stored estimate to `now`, then SNAP UP to the sample if it is larger (a spike is felt
|
|
692
|
+
* instantly) else ease toward it (it decays back over ~tau). The balancer is the SOLE writer of
|
|
693
|
+
* `_ewma` / `_stamp`. Zero-alloc on the success path.
|
|
676
694
|
* @param {number} i endpoint index
|
|
677
695
|
* @param {number} sampleNs observed rtt in nanoseconds (finite, >= 0)
|
|
678
696
|
* @param {number} now caller-supplied nanoseconds (finite), consistent with pick(now)
|
|
@@ -686,9 +704,13 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
686
704
|
throw new RangeError('[lite-pick] sampleNs must be a finite number >= 0');
|
|
687
705
|
}
|
|
688
706
|
if (!Number.isFinite(now)) throw new RangeError('[lite-pick] now must be a finite number');
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
707
|
+
if (this._stamp[i] < 0) {
|
|
708
|
+
this._ewma[i] = sampleNs; // first sample: exact init, no decay (clock-independent)
|
|
709
|
+
} else {
|
|
710
|
+
const w = Math.exp(-(now - this._stamp[i]) / this._tau);
|
|
711
|
+
const e = this._ewma[i] * w;
|
|
712
|
+
this._ewma[i] = sampleNs > e ? sampleNs : e + (sampleNs - e) * (1 - w);
|
|
713
|
+
}
|
|
692
714
|
this._stamp[i] = now;
|
|
693
715
|
}
|
|
694
716
|
|
|
@@ -709,8 +731,11 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
709
731
|
for (let t = 0; b === a && t < 32; t++) b = this._draw();
|
|
710
732
|
if (b < 0 || b === a) return a; // astronomically rare: fall back to the first draw
|
|
711
733
|
const inf = this._inflight, ewma = this._ewma, stamp = this._stamp, tau = this._tau;
|
|
712
|
-
|
|
713
|
-
|
|
734
|
+
// Decay-on-read with the unsampled sentinel: `_stamp < 0` reads the undecayed baseline
|
|
735
|
+
// (graceful LeastConn), else exponential decay. A cheap per-candidate compare, no alloc.
|
|
736
|
+
const sa = stamp[a], sb = stamp[b];
|
|
737
|
+
const costA = (inf[a] + 1) * (sa < 0 ? ewma[a] : ewma[a] * Math.exp(-(now - sa) / tau));
|
|
738
|
+
const costB = (inf[b] + 1) * (sb < 0 ? ewma[b] : ewma[b] * Math.exp(-(now - sb) / tau));
|
|
714
739
|
return costB < costA ? b : a; // lower cost wins; tie -> the first draw
|
|
715
740
|
}
|
|
716
741
|
}
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v0.
|
|
3
|
+
> Zero-GC load-balancing **selection kernel**: one hot `pick()` that returns an endpoint **index** over a fixed pool and allocates **0 B/op** on the steady-state path. A pure selector, never a proxy -- it consumes health and circuit state, it never owns them. **v0.7.0 ships seven strategies -- `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, and the latency-aware `PeakEwmaBalancer`** -- on the substrate seams (`VERSION`, `PICK_NONE`, a deterministic `Prng`, and `BalancerBase`'s shared read-only eligibility view), plus a **`@zakkster/lite-pick/pool`** subpath: the async dispatch/settle counter layer with distinct-endpoint failover and a duck-typed query-cache fetcher. The rest of the roster -- ConsistentHash, BoundedLoad, WeightedRandom -- lands one per session.
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/@zakkster/lite-pick)
|
|
6
6
|
[](https://github.com/sponsors/PeshoVurtoleta)
|
|
@@ -21,7 +21,7 @@ The npm landscape has old algorithm libraries (`load-balancers`, `loadbalance`,
|
|
|
21
21
|
- **Two pieces of evidence, both shipped.** A **0 B/op** witness on the pick path (no object, closure, string, or array created per pick), and a measured **balance-quality anchor** -- peak-to-average load within the strategy's theoretical ceiling (for P2C, the Azar-Broder-Karlin-Upfal `ln ln n / ln 2` bound) and strictly better than a random foil.
|
|
22
22
|
- **A pure selector, not a proxy.** It **consumes** health and circuit state; it never owns them. Health is a shared read-only bitmap written by [`@zakkster/lite-di-health`](https://www.npmjs.com/package/@zakkster/lite-di-health); circuit state comes from [`@zakkster/lite-statechart`](https://www.npmjs.com/package/@zakkster/lite-statechart); load counters are caller-owned typed arrays. `pick()` only reads.
|
|
23
23
|
|
|
24
|
-
> **Status:
|
|
24
|
+
> **Status: M7 (v0.7.0).** Ships the substrate seams **plus `RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, the exact `LeastConnBalancer` / `SedBalancer` / `NqBalancer` family, and the latency-aware `PeakEwmaBalancer`**, the **`@zakkster/lite-pick/pool`** request layer (now with an opt-in latency-feedback hook), and the **benchmark suite** -- the balance anchor + GC blast-radius headlines, a seeded/version-stamped `results.json`, a `bench:verify` drift check with teeth, and the vs-AWS positioning (see *Evidence* below). This session APPENDS one class: the other strategies in `Pick.js` are byte-identical, only the header roster/count and the `VERSION` stamp change. Every strategy is gated: `pick()` proven **0 B/op** (torture + PerfGate), RoundRobin **perfectly fair** with **zero dead picks** vs the naive `i++ % n` foil, SmoothWRR **exactly weighted** and **smooth**, **P2C proves the `ln ln n` balance ceiling** (peak-to-mean gap ~2 vs a random foil's ~21 at n=1024), **LeastConn is greedy-perfect** (max-minus-min load <= 1), **SED tracks weight within 1%**, and **PeakEWMA steers around a 10x-slow node** (it takes <= 25% of P2C's share for it and cuts service p99) -- all held under a **seeded invariant fuzzer** (`test/fuzz.mjs`) that checks state-synchronisation after *every* op. See [ROADMAP.md](./ROADMAP.md) for the M7 -> M10 path to 1.0.0, and [decisions/](./decisions) for the ownership boundary (ADR 0001), anti-flapping (ADR 0002), the RoundRobin (0003), SmoothWRR (0004), P2C (0005), LeastConn-family (0006), pool-adapter (0007), benchmark-suite (0008), and PeakEWMA (0009) design forks.
|
|
25
25
|
|
|
26
26
|
```bash
|
|
27
27
|
npm install @zakkster/lite-pick
|
|
@@ -139,6 +139,47 @@ The proof (from `test/balance.mjs`):
|
|
|
139
139
|
|
|
140
140
|
Each `pick()` is **0 B/op** and **O(cap)** (NQ is O(1) when an early node is idle). Because these are the state-heaviest strategies so far, M4 also introduces the **invariant fuzzer** (`npm run fuzz`): a seeded state-machine attack that, after *every* `pick` / `setEligible` / weight / load op, asserts the chosen endpoint is the *exact* optimum, `live` stays exact, and `PICK_NONE` holds *iff* nothing is pickable -- printing the seed on any failure for byte-for-byte replay.
|
|
141
141
|
|
|
142
|
+
## PeakEWMA -- latency-aware P2C (v0.7.0)
|
|
143
|
+
|
|
144
|
+
When endpoints differ in **latency**, not just queue depth, count-based strategies keep re-probing a slow-but-up node: it drains its queue between visits, so its in-flight looks attractive again. `PeakEwmaBalancer` (Twitter Finagle's *peak-EWMA*) is power-of-two-choices over a **latency cost** -- `cost = (inflight + 1) x decayed-EWMA(rtt)` -- so a degraded node is avoided **even while idle** ([ADR 0009](./decisions/0009-peakewma.md)). It is `O(d) = O(1)` per pick and **0 B/op** on both the pick path and the feedback path.
|
|
145
|
+
|
|
146
|
+
```js
|
|
147
|
+
import { PeakEwmaBalancer } from '@zakkster/lite-pick';
|
|
148
|
+
|
|
149
|
+
const eligible = Uint8Array.from([1, 1, 1, 1]);
|
|
150
|
+
const inflight = new Uint32Array(4); // YOU own this; read live by pick()
|
|
151
|
+
const TAU_NS = 30e6; // EWMA half-life: 30ms of latency memory
|
|
152
|
+
|
|
153
|
+
// `now` and rtt samples are CALLER-supplied nanoseconds -- deterministic, testable, zero-GC.
|
|
154
|
+
const pe = new PeakEwmaBalancer(4, eligible, inflight, TAU_NS);
|
|
155
|
+
|
|
156
|
+
const now = perfNs(); // your monotonic ns clock
|
|
157
|
+
const i = pe.pick(now); // two random eligibles, lower latency-cost wins
|
|
158
|
+
inflight[i]++; // dispatch
|
|
159
|
+
// ... await the request ...
|
|
160
|
+
inflight[i]--; // settle
|
|
161
|
+
pe.recordRtt(i, perfNs() - now, perfNs()); // feed the observed rtt back (the warm path)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
- **Decay-on-read.** `pick()` *never writes* -- it applies exponential decay when it reads (`ewmaAt(i, now) = _ewma[i] x exp(-(now - _stamp[i]) / tau)`), so the hot path is a pure read and allocates nothing.
|
|
165
|
+
- **The peak rule.** `recordRtt` *snaps the cost up* to a larger sample instantly (a spike is felt on the next pick) and *decays it down* over `~tau`. The half-life **is** the anti-flap smoothing -- no extra dwell ([ADR 0002](./decisions/0002-anti-flapping.md)).
|
|
166
|
+
- **Balancer-owned state.** `inflight` is your live-read `Uint32Array`; the EWMA arrays are owned by the balancer and written *only* by `recordRtt`. Cold start seeds the EWMA to `1.0` with an *unsampled* sentinel (`_stamp = -1`): an unsampled node scores at its undecayed baseline, so before any sample PeakEWMA degrades gracefully to least-connections **regardless of your clock's magnitude** -- never underflowing to `0` (which a plain `_stamp = 0` would, as `exp(-now/tau)`, under a real large clock) and never `NaN`. The first `recordRtt` initializes the EWMA *exactly* to the sample; the peak rule applies from the second sample on.
|
|
167
|
+
- **`now` must be finite.** `now` (for `pick(now)` / `recordRtt`) and `sampleNs` must be finite numbers. `recordRtt` throws on a non-finite argument; `pick(now)` never throws (the fail-closed contract), so a non-finite `now` yields P2C-random selection rather than an error.
|
|
168
|
+
|
|
169
|
+
The proof (from `test/balance.mjs`, a closed-loop queue sim with one node at 10x service time):
|
|
170
|
+
|
|
171
|
+
| lane | slow-node share | service p99 |
|
|
172
|
+
|---|---|---|
|
|
173
|
+
| **PeakEWMA** | **~0.01%** (learns and avoids it) | **lowest** |
|
|
174
|
+
| P2C (in-flight only) | ~1.5% (keeps re-probing) | ~7x PeakEWMA's |
|
|
175
|
+
| random foil | ~6% (blind) | saturates the slow node |
|
|
176
|
+
|
|
177
|
+
PeakEWMA sends the slow node **<= 25% of P2C's share** for it and cuts service p99 **>= 20% below** P2C-over-inflight; the random foil is worse than both.
|
|
178
|
+
|
|
179
|
+
### FE profile -- PeakEWMA + health, nothing else
|
|
180
|
+
|
|
181
|
+
For a **front-end / browser client** -- a handful of picks per second across origins/regions, not a zero-GC hot loop -- the recommended profile is **PeakEWMA + the eligibility bitmap only**: latency-aware choice with a fail-closed health view, and **none** of the server-side bounded-load / availability-zone / occupancy machinery. It is the smallest honest latency-aware client balancer. *(Deferred: a tail-aware `inflight x p99Rtt` variant via an optional-peer `@zakkster/lite-sketch` `DDSketch`; the EWMA-mean score is the shipped zero-peer default, and `peerDependencies` stays `{}` -- [ADR 0009](./decisions/0009-peakewma.md).)*
|
|
182
|
+
|
|
142
183
|
## Evidence -- the two headlines (v0.6.0 benchmark suite)
|
|
143
184
|
|
|
144
185
|
> **Framing: parity on speed, superiority on the contract + balance + tail.** A trivial `i++ % n` round-robin -- or `wrr` -- *matches* P2C on raw ops/sec, so `lite-pick` does **not** claim "N times faster." Throughput is claimed at **parity**; the wins are **zero-GC**, **balance quality**, **tail latency** (GC blast-radius), and **never a dead pick**. Every number below is **seeded** and regenerated by `npm run bench:report`; `npm run bench:verify` fails CI if a README number drifts from a fresh run (algorithmic exact, timing within +/-15%). Node / CPU / OS / every PRNG seed are stamped into `benchmark/results.json`.
|
|
@@ -151,9 +192,9 @@ The real pinned npm incumbents (`load-balancers`, `loadbalance`, `wrr`) run thro
|
|
|
151
192
|
|
|
152
193
|
| family | lite-pick | lite-pick ops/ms | incumbent (npm) | incumbent ops/ms |
|
|
153
194
|
| --- | --- | --- | --- | --- |
|
|
154
|
-
| P2C (power-of-two-choices) | P2cBalancer |
|
|
155
|
-
| RoundRobin | RoundRobinBalancer |
|
|
156
|
-
| Weighted-random | WeightedRandom -- SKIP, ships M10 | -- | wrr@1.0.0 |
|
|
195
|
+
| P2C (power-of-two-choices) | P2cBalancer | 53426 | load-balancers@1.3.52 | 61749 |
|
|
196
|
+
| RoundRobin | RoundRobinBalancer | 246108 | loadbalance@1.0.0 | 282939 |
|
|
197
|
+
| Weighted-random | WeightedRandom -- SKIP, ships M10 | -- | wrr@1.0.0 | 151557 |
|
|
157
198
|
|
|
158
199
|
<!-- /bench:competitors -->
|
|
159
200
|
|
|
@@ -181,8 +222,8 @@ The point of zero-GC is **not** the pick's own latency -- a major GC pause freez
|
|
|
181
222
|
|
|
182
223
|
| lane | major GC | pick B/op | max GC pause (ms) |
|
|
183
224
|
| --- | --- | --- | --- |
|
|
184
|
-
| lite-pick | 0 | 0 | 0.
|
|
185
|
-
| allocating foil | 13 | allocates |
|
|
225
|
+
| lite-pick | 0 | 0 | 0.3 |
|
|
226
|
+
| allocating foil | 13 | allocates | 1.9 |
|
|
186
227
|
|
|
187
228
|
<!-- /bench:gc -->
|
|
188
229
|
|
|
@@ -221,6 +262,7 @@ On a scale event (add / remove a node), what fraction of keys keep their node? T
|
|
|
221
262
|
| AWS edge feature (managed, billed) | `lite-pick` equivalent (in-process, zero-GC) |
|
|
222
263
|
| --- | --- |
|
|
223
264
|
| ALB `least_outstanding_requests` (LOR) | `LeastConnBalancer` / `P2cBalancer` |
|
|
265
|
+
| ALB anomaly mitigation / latency-aware shedding | `PeakEwmaBalancer` (latency-aware P2C) |
|
|
224
266
|
| ALB `weighted_random` + anomaly mitigation | `WeightedRandom` + `BoundedLoad` (M9/M10) |
|
|
225
267
|
| NLB flow-hash (5-tuple) | `ConsistentHash` (Maglev, M8) |
|
|
226
268
|
|
|
@@ -254,10 +296,12 @@ PICK_NONE; // -> -1 (fail-closed sentinel: no endpoint, never a dead
|
|
|
254
296
|
VERSION; // -> '0.6.0'
|
|
255
297
|
```
|
|
256
298
|
|
|
257
|
-
`BalancerBase.pick()` is **abstract** -- it throws, so an unfinished strategy fails loudly rather than returning a dead index. Every shipped strategy (`RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`) extends it and reads the same shared eligibility view; you subclass it the same way to add your own.
|
|
299
|
+
`BalancerBase.pick()` is **abstract** -- it throws, so an unfinished strategy fails loudly rather than returning a dead index. Every shipped strategy (`RoundRobinBalancer`, `SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, `PeakEwmaBalancer`) extends it and reads the same shared eligibility view; you subclass it the same way to add your own.
|
|
258
300
|
|
|
259
301
|
## Wiring it up -- `@zakkster/lite-pick/pool` (v0.5.0)
|
|
260
302
|
|
|
303
|
+
> **New to lite-pick as a load balancer?** [**RECIPES.md**](./RECIPES.md) is a beginner-to-advanced guide: it builds the kernel up into a real balancer step by step -- health/eligibility, load counters, the dispatch/settle loop, failover, latency feedback (PeakEWMA), the FE profile, and choosing a strategy. Start there; the sections below are the reference.
|
|
304
|
+
|
|
261
305
|
The kernel gives you `pick() -> index`. Real callers also need the counter ergonomics: **increment in-flight on dispatch, decrement on settle, and re-pick a *different* endpoint on failure.** That layer is async (it wraps the request), so it lives in a separate subpath -- `@zakkster/lite-pick/pool` -- and the kernel stays 0 B/op.
|
|
262
306
|
|
|
263
307
|
```js
|
package/RECIPES.md
ADDED
|
@@ -0,0 +1,329 @@
|
|
|
1
|
+
# lite-pick recipes -- from a one-liner to a real load balancer
|
|
2
|
+
|
|
3
|
+
`@zakkster/lite-pick` is a SELECTION KERNEL, not a proxy. It answers one question --
|
|
4
|
+
"which endpoint should this request go to?" -- and returns an integer index (or
|
|
5
|
+
`PICK_NONE` = -1 when nothing is eligible). A *real* load balancer is that kernel plus
|
|
6
|
+
the wiring around it:
|
|
7
|
+
|
|
8
|
+
- **eligibility** -- who is up? (a health check writes a shared bitmap)
|
|
9
|
+
- **load counters** -- how busy is each endpoint? (you own an `inflight` array)
|
|
10
|
+
- **the dispatch/settle loop** -- increment on send, decrement on finish
|
|
11
|
+
- **failover** -- if a call fails, try a different endpoint
|
|
12
|
+
- **latency feedback** -- for latency-aware routing, feed measured rtt back
|
|
13
|
+
|
|
14
|
+
These recipes build that wiring up, one layer at a time. Every array is preallocated
|
|
15
|
+
once and reused -- the pick path allocates 0 bytes.
|
|
16
|
+
|
|
17
|
+
Install: `npm i @zakkster/lite-pick` (zero runtime dependencies; ESM; Node >= 18)
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 1. The 30-second version -- round-robin over a fixed pool
|
|
22
|
+
|
|
23
|
+
```js
|
|
24
|
+
import { RoundRobinBalancer, PICK_NONE } from '@zakkster/lite-pick';
|
|
25
|
+
|
|
26
|
+
const CAP = 4; // fixed pool size
|
|
27
|
+
const eligible = new Uint8Array(CAP).fill(1); // 1 = up, 0 = down (all up here)
|
|
28
|
+
const lb = new RoundRobinBalancer(CAP, eligible);
|
|
29
|
+
|
|
30
|
+
const endpoints = ['a.svc:8080', 'b.svc:8080', 'c.svc:8080', 'd.svc:8080'];
|
|
31
|
+
|
|
32
|
+
for (let r = 0; r < 6; r++) {
|
|
33
|
+
const i = lb.pick(); // -> next eligible index, round-robin
|
|
34
|
+
if (i === PICK_NONE) throw new Error('pool is down');
|
|
35
|
+
send(endpoints[i]); // your transport; lite-pick never opens a socket
|
|
36
|
+
}
|
|
37
|
+
// picks: a, b, c, d, a, b
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
`pick()` is the whole kernel. Everything below adds a capability around it.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## 2. Fail closed -- always handle PICK_NONE
|
|
45
|
+
|
|
46
|
+
lite-pick never returns a down endpoint and never guesses. When the whole pool is
|
|
47
|
+
ineligible, `pick()` returns `PICK_NONE` (-1). Treat it as a first-class outcome:
|
|
48
|
+
|
|
49
|
+
```js
|
|
50
|
+
const i = lb.pick();
|
|
51
|
+
if (i === PICK_NONE) {
|
|
52
|
+
// shed load, return 503, or fall back -- your policy. Never index endpoints[-1].
|
|
53
|
+
return respond503();
|
|
54
|
+
}
|
|
55
|
+
send(endpoints[i]);
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`lb.live` is an O(1) count of eligible endpoints if you want to check before picking.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## 3. Wire health -> eligibility (the shared bitmap)
|
|
63
|
+
|
|
64
|
+
Eligibility is a shared `Uint8Array` (1 = pickable, 0 = down). Something else writes it
|
|
65
|
+
-- a health checker, a circuit breaker, or `@zakkster/lite-di-health` -- and `pick()`
|
|
66
|
+
only reads it. Two ways to flip a node:
|
|
67
|
+
|
|
68
|
+
```js
|
|
69
|
+
// (a) write the bitmap directly if you own it elsewhere (zero-copy, pick sees it live):
|
|
70
|
+
eligible[2] = 0; // endpoint c is down
|
|
71
|
+
|
|
72
|
+
// (b) go through the balancer so its O(1) `live` count stays exact (recommended):
|
|
73
|
+
lb.setEligible(2, false); // COLD path; idempotent; keeps `live` correct
|
|
74
|
+
lb.setEligible(2, true); // back up
|
|
75
|
+
|
|
76
|
+
lb.isEligible(2); // -> boolean, HOT, out-of-range is false (never throws)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Prefer `setEligible` when you rely on `live`. Health flapping is the writer's problem:
|
|
80
|
+
apply hysteresis/dwell in the health layer -- `pick()` stays greedy and stateless.
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## 4. Weighted pools -- SmoothWRR
|
|
85
|
+
|
|
86
|
+
When endpoints have different capacities, weight them. `SmoothWRRBalancer` owns its
|
|
87
|
+
weight state; it is the SOLE writer -- always go through `setWeight`, never mutate the
|
|
88
|
+
array directly (direct mutation desyncs the internal total = undefined behavior).
|
|
89
|
+
|
|
90
|
+
```js
|
|
91
|
+
import { SmoothWRRBalancer } from '@zakkster/lite-pick';
|
|
92
|
+
|
|
93
|
+
const weights = new Uint32Array(CAP); // the balancer manages these
|
|
94
|
+
const lb = new SmoothWRRBalancer(CAP, eligible, weights);
|
|
95
|
+
lb.setWeight(0, 5); // a is 5x
|
|
96
|
+
lb.setWeight(1, 1);
|
|
97
|
+
lb.setWeight(2, 1);
|
|
98
|
+
lb.setWeight(3, 1);
|
|
99
|
+
// pick() interleaves smoothly (nginx smooth WRR): a a b a c a a d ... not a a a a a b c d
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Use SmoothWRR when weights are known/config-driven and change rarely.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 5. Load-aware selection -- you own the `inflight` counters
|
|
107
|
+
|
|
108
|
+
P2C, LeastConn, SED, and NQ route by *current load*. That load lives in a
|
|
109
|
+
caller-owned `Uint32Array` you increment on dispatch and decrement on settle. If you
|
|
110
|
+
don't maintain it, these strategies are blind (they see every node at 0).
|
|
111
|
+
|
|
112
|
+
```js
|
|
113
|
+
import { P2cBalancer } from '@zakkster/lite-pick';
|
|
114
|
+
|
|
115
|
+
const inflight = new Uint32Array(CAP); // YOURS to maintain
|
|
116
|
+
const lb = new P2cBalancer(CAP, eligible, inflight);
|
|
117
|
+
|
|
118
|
+
async function handle(req) {
|
|
119
|
+
const i = lb.pick();
|
|
120
|
+
if (i === PICK_NONE) return respond503();
|
|
121
|
+
inflight[i]++; // DISPATCH
|
|
122
|
+
try {
|
|
123
|
+
return await send(endpoints[i], req);
|
|
124
|
+
} finally {
|
|
125
|
+
inflight[i]--; // SETTLE (always, even on error)
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
- **P2C** -- two random draws, pick the lighter. O(1), the scalable default; peak load
|
|
131
|
+
hugs the `ln ln n` band. Great from ~8 endpoints up.
|
|
132
|
+
- **LeastConn** -- exact fewest-in-flight (O(cap) scan). Best balance for small pools.
|
|
133
|
+
- **SED** / **NQ** -- weighted least-conn: pass a `weights` Uint32Array too;
|
|
134
|
+
`new SedBalancer(CAP, eligible, inflight, weights)`. NQ sends to an idle node first.
|
|
135
|
+
|
|
136
|
+
Maintaining the dispatch/settle loop by hand is easy to get wrong. Recipe 6 does it for
|
|
137
|
+
you.
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## 6. The real request loop -- `@zakkster/lite-pick/pool`
|
|
142
|
+
|
|
143
|
+
The `/pool` subpath wraps the kernel with the async dispatch/settle ergonomics so you
|
|
144
|
+
don't hand-maintain `inflight`. The kernel `pick()` stays 0 B/op; `Pool.run` is a normal
|
|
145
|
+
async wrapper on top.
|
|
146
|
+
|
|
147
|
+
```js
|
|
148
|
+
import { P2cBalancer } from '@zakkster/lite-pick';
|
|
149
|
+
import { Pool } from '@zakkster/lite-pick/pool';
|
|
150
|
+
|
|
151
|
+
const inflight = new Uint32Array(CAP);
|
|
152
|
+
const lb = new P2cBalancer(CAP, eligible, inflight);
|
|
153
|
+
const pool = new Pool(lb, inflight); // SAME inflight array the balancer reads
|
|
154
|
+
|
|
155
|
+
// Pool does pick -> inflight++ -> await fn -> inflight-- (in a finally) for you:
|
|
156
|
+
const body = await pool.run((i, signal) => fetchFrom(endpoints[i], { signal }));
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
`run(fn, opts?)` rejects with a `code: 'LITE_PICK_NONE'` error when the pool is down.
|
|
160
|
+
`fn(endpoint, signal)` receives the chosen index and the (optional) AbortSignal.
|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## 7. Failover -- try a different endpoint on error
|
|
165
|
+
|
|
166
|
+
Set `tries > 1`. On a thrown error, Pool keeps the failed node's in-flight count
|
|
167
|
+
elevated and re-picks -- so a load-aware strategy naturally steers to a DIFFERENT
|
|
168
|
+
endpoint -- up to `tries` attempts, then rejects with the last error.
|
|
169
|
+
|
|
170
|
+
```js
|
|
171
|
+
const body = await pool.run(
|
|
172
|
+
(i, signal) => fetchFrom(endpoints[i], { signal }),
|
|
173
|
+
{ tries: 3, signal: req.signal } // up to 3 distinct endpoints
|
|
174
|
+
);
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Boundary: Pool owns **spatial** failover (move across the pool, once each, in-process).
|
|
178
|
+
The caller or your query cache owns **temporal** retry (backoff, staleness, dedup).
|
|
179
|
+
Don't double-own them. If `signal` aborts after a failure, failover stops and the abort
|
|
180
|
+
propagates.
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## 8. Latency-aware routing -- PeakEWMA with rtt feedback
|
|
185
|
+
|
|
186
|
+
PeakEWMA (latency-aware P2C, Finagle's peak-EWMA) steers away from *slow* endpoints,
|
|
187
|
+
not just busy ones. It scores each candidate `(inflight + 1) x ewma(rtt)`, so a node
|
|
188
|
+
that got slow gets less traffic even if its connection count looks fine. It needs two
|
|
189
|
+
things you didn't need before: a **clock** (`now`, caller-supplied nanoseconds) and
|
|
190
|
+
**rtt feedback** (`recordRtt`).
|
|
191
|
+
|
|
192
|
+
Manual loop:
|
|
193
|
+
|
|
194
|
+
```js
|
|
195
|
+
import { PeakEwmaBalancer } from '@zakkster/lite-pick';
|
|
196
|
+
|
|
197
|
+
const TAU_NS = 30_000_000; // 30ms half-life for the EWMA decay
|
|
198
|
+
const inflight = new Uint32Array(CAP);
|
|
199
|
+
const lb = new PeakEwmaBalancer(CAP, eligible, inflight, TAU_NS);
|
|
200
|
+
const nowNs = () => Number(process.hrtime.bigint());
|
|
201
|
+
|
|
202
|
+
async function handle(req) {
|
|
203
|
+
const now = nowNs();
|
|
204
|
+
const i = lb.pick(now); // decay-on-read, 0 B/op
|
|
205
|
+
if (i === PICK_NONE) return respond503();
|
|
206
|
+
inflight[i]++;
|
|
207
|
+
const start = nowNs();
|
|
208
|
+
try {
|
|
209
|
+
return await send(endpoints[i], req);
|
|
210
|
+
} finally {
|
|
211
|
+
inflight[i]--;
|
|
212
|
+
lb.recordRtt(i, nowNs() - start, nowNs()); // FEEDBACK: measured rtt, snaps up / decays down
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Or let Pool do the feedback for you -- pass a `clock`; Pool drives `pick(now)` and calls
|
|
218
|
+
`recordRtt` on a successful settle when the balancer supports it:
|
|
219
|
+
|
|
220
|
+
```js
|
|
221
|
+
import { Pool } from '@zakkster/lite-pick/pool';
|
|
222
|
+
const pool = new Pool(lb, inflight);
|
|
223
|
+
const body = await pool.run(
|
|
224
|
+
(i, signal) => fetchFrom(endpoints[i], { signal }),
|
|
225
|
+
{ clock: () => Number(process.hrtime.bigint()), tries: 2 }
|
|
226
|
+
);
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Notes:
|
|
230
|
+
- **Cold start** (no samples yet) degrades gracefully to least-connections -- never NaN.
|
|
231
|
+
- `now` must be a FINITE number. A non-finite `now` degrades to P2C-random (no throw).
|
|
232
|
+
- Pick `tauNs` around your p50-p90 rtt: smaller = reacts faster to a slowdown, larger =
|
|
233
|
+
steadier. It IS the anti-flap smoothing; no extra dwell needed.
|
|
234
|
+
- Measured effect: with one node at 10x latency, PeakEWMA sends it a tiny fraction of
|
|
235
|
+
the traffic P2C-over-inflight would, and cuts service p99 sharply.
|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
## 9. Wire it into a query cache (lite-query, or any fetcher)
|
|
240
|
+
|
|
241
|
+
`liteQueryFetcher` adapts a Pool into a `({ key, signal }) => Promise` fetcher -- the
|
|
242
|
+
shape lite-query (or any cache/route-loader) expects. It imports nothing from lite-query
|
|
243
|
+
(duck-typed), so it works with any fetcher-shaped consumer.
|
|
244
|
+
|
|
245
|
+
```js
|
|
246
|
+
import { Pool, liteQueryFetcher } from '@zakkster/lite-pick/pool';
|
|
247
|
+
|
|
248
|
+
const pool = new Pool(lb, inflight);
|
|
249
|
+
const fetcher = liteQueryFetcher(
|
|
250
|
+
pool,
|
|
251
|
+
({ endpoint, key, signal }) => fetchFrom(endpoints[endpoint], { key, signal }),
|
|
252
|
+
{ tries: 2 }
|
|
253
|
+
);
|
|
254
|
+
// hand `fetcher` to your query cache; each cache miss fans out across the pool with failover.
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
## 10. Choosing a strategy
|
|
260
|
+
|
|
261
|
+
| Strategy | Route by | Cost | Reach for it when |
|
|
262
|
+
|---------------|---------------------|-------------|-------------------|
|
|
263
|
+
| RoundRobin | position | O(1) amort. | uniform endpoints, no load signal |
|
|
264
|
+
| SmoothWRR | static weight | O(cap) | known/config capacities, smooth interleave |
|
|
265
|
+
| P2C | in-flight (approx) | O(1) | the scalable default from ~8 nodes up |
|
|
266
|
+
| LeastConn | in-flight (exact) | O(cap) | small pools, tightest connection balance |
|
|
267
|
+
| SED | in-flight / weight | O(cap) | weighted least-conn |
|
|
268
|
+
| NQ | idle-first else SED | O(cap)/O(1) | worker pools -- never queue while a worker is free |
|
|
269
|
+
| PeakEWMA | in-flight x ewma(rtt)| O(1) | heterogeneous / flaky backends; steer around slow nodes |
|
|
270
|
+
|
|
271
|
+
All read the SAME `inflight` array live, so you can swap strategies without rewiring.
|
|
272
|
+
|
|
273
|
+
---
|
|
274
|
+
|
|
275
|
+
## 11. The "FE profile" -- a browser / front-end client
|
|
276
|
+
|
|
277
|
+
For a front-end client picking among origins a handful of times per second (not a
|
|
278
|
+
zero-GC hot loop), the recommended profile is **PeakEWMA + health/eligibility only** --
|
|
279
|
+
latency-aware choice across origins with a fail-closed eligibility view -- and skip the
|
|
280
|
+
bounded-load / AZ / occupancy machinery. Feed rtt from your `fetch` timings via
|
|
281
|
+
`recordRtt`.
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## 12. Compose with the suite (all optional, all duck-typed)
|
|
286
|
+
|
|
287
|
+
lite-pick declares ZERO hard dependencies and an EMPTY `peerDependencies`. Each seam is
|
|
288
|
+
a shared TypedArray or a duck-typed shape, so you wire in a sibling only if you use it:
|
|
289
|
+
|
|
290
|
+
- `@zakkster/lite-di-health` -- writes the eligibility bitmap from health checks.
|
|
291
|
+
- `@zakkster/lite-statechart` -- a circuit breaker that flips eligibility.
|
|
292
|
+
- `@zakkster/lite-query` -- the cache behind `liteQueryFetcher` (recipe 9).
|
|
293
|
+
- `@zakkster/lite-sketch` -- `DDSketch` for a p99-aware PeakEWMA variant (deferred).
|
|
294
|
+
- `@zakkster/lite-await` -- hedging (race the P2C second choice past a percentile).
|
|
295
|
+
|
|
296
|
+
None is required; the kernel runs over raw TypedArrays with nothing installed.
|
|
297
|
+
|
|
298
|
+
---
|
|
299
|
+
|
|
300
|
+
## 13. Zero-GC discipline (why the pick path stays 0 B/op)
|
|
301
|
+
|
|
302
|
+
- Allocate `eligible` / `inflight` / `weights` ONCE at startup and reuse them. Never
|
|
303
|
+
build arrays per pick.
|
|
304
|
+
- The counters are YOURS -- mutate them in place (`inflight[i]++/--`), don't replace them.
|
|
305
|
+
- `pick()` / `pick(now)` and `recordRtt` allocate nothing. The only async allocation is
|
|
306
|
+
the promise your own `fn` already creates (disclosed; `Pool.run` adds O(1) integer ops
|
|
307
|
+
plus one small per-run array).
|
|
308
|
+
- Fixed capacity: the pool size is set at construction and the backing arrays never
|
|
309
|
+
reallocate.
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
## 14. Gotchas
|
|
314
|
+
|
|
315
|
+
- **PICK_NONE (-1)** is always possible -- handle it before indexing (recipe 2).
|
|
316
|
+
- **SmoothWRR weights** must go through `setWeight`; direct array mutation is UB.
|
|
317
|
+
- **Load-aware strategies need the dispatch/settle loop** -- forget the `inflight--` in
|
|
318
|
+
a `finally` and load leaks upward forever. Use `/pool` (recipe 6) to avoid it.
|
|
319
|
+
- **PeakEWMA needs a finite `now`** and rtt feedback -- without `recordRtt` it behaves
|
|
320
|
+
like LeastConn (cold-start baseline).
|
|
321
|
+
- **Eligibility is read-only to `pick()`** -- the health layer writes it; the balancer
|
|
322
|
+
only reads (or maintains `live` via `setEligible`).
|
|
323
|
+
- **lite-pick is not a proxy** -- it returns an index; you own transport, retries/backoff
|
|
324
|
+
(temporal), health checking, and the socket.
|
|
325
|
+
|
|
326
|
+
---
|
|
327
|
+
|
|
328
|
+
See also: `README.md` (overview + gates), `llms.txt` (full API surface),
|
|
329
|
+
`decisions/` (the ADRs behind each design call), `ROADMAP.md` (what's next).
|
package/llms.txt
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zakkster/lite-pick
|
|
2
2
|
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.7.2
|
|
4
4
|
License: MIT (c) Zahary Shinikchiev <shinikchiev@yahoo.com>
|
|
5
5
|
Runtime dependencies: none. ESM only. ASCII-only source. sideEffects: false.
|
|
6
6
|
Node: >= 18.
|
|
@@ -14,15 +14,25 @@ reading pre-allocated views that siblings or the caller write, and returning an
|
|
|
14
14
|
The complementary evidence lite-pick ships is a measured balance-quality anchor (peak-to-
|
|
15
15
|
average load vs the strategy's theoretical ceiling) alongside the 0 B/op pick witness.
|
|
16
16
|
|
|
17
|
-
0.
|
|
18
|
-
default), P2C (power-of-two-choices -- also the O(1) least-connections APPROXIMATION),
|
|
19
|
-
EXACT LeastConn family (LeastConn, SED, NQ)
|
|
20
|
-
`PICK_NONE` (-1), a deterministic `Prng` (xorshift32),
|
|
21
|
-
eligibility seam + O(1) live count), `RoundRobinBalancer`,
|
|
22
|
-
`LeastConnBalancer`, `SedBalancer`,
|
|
23
|
-
session (see ROADMAP.md):
|
|
24
|
-
n) fewest-in-flight variant is a
|
|
25
|
-
(decisions/0006), not this
|
|
17
|
+
0.7.0 ships the substrate seams + seven strategies: RoundRobin, SmoothWRR (the weighted
|
|
18
|
+
default), P2C (power-of-two-choices -- also the O(1) least-connections APPROXIMATION), the
|
|
19
|
+
EXACT LeastConn family (LeastConn, SED, NQ), and PeakEWMA (latency-aware P2C). It exports
|
|
20
|
+
`VERSION`, the fail-closed sentinel `PICK_NONE` (-1), a deterministic `Prng` (xorshift32),
|
|
21
|
+
`BalancerBase` (the shared read-only eligibility seam + O(1) live count), `RoundRobinBalancer`,
|
|
22
|
+
`SmoothWRRBalancer`, `P2cBalancer`, `LeastConnBalancer`, `SedBalancer`, `NqBalancer`, and
|
|
23
|
+
`PeakEwmaBalancer`. The remaining strategies land one per session (see ROADMAP.md):
|
|
24
|
+
ConsistentHash, BoundedLoad, WeightedRandom. The EXACT-O(log n) fewest-in-flight variant is a
|
|
25
|
+
deferred @zakkster/lite-logn `BinaryHeap` optional-peer seam (decisions/0006), not this
|
|
26
|
+
exact-O(cap) scan.
|
|
27
|
+
|
|
28
|
+
M7 (0.7.0) adds PeakEwmaBalancer (decisions/0009). The FE PROFILE: for a browser / front-end
|
|
29
|
+
client (a handful of picks per second, not a zero-GC hot loop), the recommended profile is
|
|
30
|
+
PeakEWMA + health/eligibility ONLY -- no bounded-load / AZ / occupancy machinery -- latency-aware
|
|
31
|
+
choice across origins with a fail-closed eligibility view. DEFERRED tail-aware variant: a p99-aware
|
|
32
|
+
score (inflight x p99Rtt) via a per-node @zakkster/lite-sketch `DDSketch` (published 0.3.0; `add`
|
|
33
|
+
is O(1) / 0 B/op, `quantile` carries a hard relative-error bound) is an OPTIONAL PEER complement;
|
|
34
|
+
the EWMA-mean score is the shipped, zero-peer default and `peerDependencies` STAYS `{}` until a
|
|
35
|
+
shipped code path imports the sketch.
|
|
26
36
|
|
|
27
37
|
M6 (0.6.0) adds the BENCHMARK SUITE -- an evidence session, no API change: Pick.js / Pool.js are
|
|
28
38
|
byte-identical to 0.5.0 apart from the VERSION stamp. It lives entirely under benchmark/ (NOT in
|
|
@@ -116,6 +126,30 @@ the contract + balance + tail -- never an "N times faster" headline (decisions/0
|
|
|
116
126
|
- `pick()` -> number. Returns the FIRST idle eligible positive-weight node (in-flight 0) if one
|
|
117
127
|
exists -- never queueing while a server is free -- else the SED minimum, else `PICK_NONE`.
|
|
118
128
|
O(cap) worst case, O(1) when an early node is idle. 0 B/op.
|
|
129
|
+
- `PeakEwmaBalancer extends BalancerBase` -- class. Latency-aware P2C (M7, Finagle peak-EWMA).
|
|
130
|
+
- `new PeakEwmaBalancer(capacity, eligible, inflight, tauNs, seed?=0x9e3779b9)` -- `inflight` is a
|
|
131
|
+
caller-owned Uint32Array (length >= capacity) read LIVE; `tauNs` is the EWMA time-constant /
|
|
132
|
+
half-life in nanoseconds (finite, > 0). The per-endpoint EWMA state (`_ewma` / `_stamp`,
|
|
133
|
+
Float64) is BALANCER-OWNED and written ONLY by `recordRtt`. Validates typeof-first BEFORE
|
|
134
|
+
allocating (RangeError/TypeError). Cold start seeds the EWMA to 1.0 with an UNSAMPLED sentinel
|
|
135
|
+
(`_stamp = -1`), so an unsampled node scores at its undecayed baseline -> graceful
|
|
136
|
+
least-connections regardless of clock magnitude (never underflows to 0).
|
|
137
|
+
- `pick(now)` -> number. Draws two DISTINCT eligible endpoints (ADR 0005's rejection sampling,
|
|
138
|
+
reused) and returns the lower `cost = (inflight + 1) * ewmaAt(now)`; a tie goes to the first
|
|
139
|
+
draw. `now` is caller-supplied nanoseconds. Decays ON READ (never writes) -> O(d)=O(1), 0 B/op.
|
|
140
|
+
`PICK_NONE` when the whole pool is down. A slow-but-up node (high EWMA rtt) is avoided even while
|
|
141
|
+
idle -- the difference from P2C-over-inflight (latency anchor in balance.mjs).
|
|
142
|
+
- `ewmaAt(i, now)` -> number. The EWMA rtt estimate for endpoint i at `now`. Pure read. An
|
|
143
|
+
unsampled node (`_stamp < 0`) returns the baseline 1.0 undecayed; otherwise exponential decay.
|
|
144
|
+
- `recordRtt(i, sampleNs, now)` -> void. WARM feedback path (not the hot pick path): the FIRST
|
|
145
|
+
sample initializes the EWMA EXACTLY to `sampleNs` (clock-magnitude-independent); thereafter the
|
|
146
|
+
Finagle peak rule -- the cost SNAPS UP to a larger sample instantly and DECAYS DOWN over ~tau.
|
|
147
|
+
`now` / `sampleNs` are caller-supplied nanoseconds, consistent with `pick(now)`. Validates
|
|
148
|
+
typeof-first; 0 B/op on the success path. Anti-flap = the half-life, no extra dwell (ADR 0002,
|
|
149
|
+
ADR 0009).
|
|
150
|
+
- CONTRACT: `now` and `sampleNs` MUST be FINITE numbers. `recordRtt` THROWS on a non-finite
|
|
151
|
+
argument (warm path); `pick(now)` NEVER throws (fail-closed contract), so a non-finite `now`
|
|
152
|
+
yields P2C-random selection, not an error.
|
|
119
153
|
|
|
120
154
|
## Subpath: @zakkster/lite-pick/pool -- the ergonomic request layer (M5, Pool.js)
|
|
121
155
|
|
|
@@ -140,7 +174,11 @@ duck-typed and imports NOTHING from lite-query.
|
|
|
140
174
|
once aborted after a failure, failover stops and the abort propagates. NOT a 0 B/op path (the
|
|
141
175
|
kernel `pick()` is): a normal async wrapper adding O(1) counter ops + one small per-run array.
|
|
142
176
|
BOUNDARY: Pool owns SPATIAL failover (across the pool); the caller / query cache owns TEMPORAL
|
|
143
|
-
retry (backoff, staleness). Never double-owned (decisions/0007).
|
|
177
|
+
retry (backoff, staleness). Never double-owned (decisions/0007). OPT-IN latency feedback: when
|
|
178
|
+
`opts.clock` (a caller-owned nanosecond source) is supplied AND the balancer duck-types
|
|
179
|
+
`recordRtt` (PeakEwmaBalancer), Pool drives `pick(now)` and records the settled rtt on success;
|
|
180
|
+
otherwise the hook is inert -- Pool stays generic, in-flight stays net-zero, abort/failover
|
|
181
|
+
unchanged.
|
|
144
182
|
- `liteQueryFetcher(pool, perEndpoint, opts?)` -> a `({ key, signal }) => Promise` fetcher for a
|
|
145
183
|
query cache (lite-query's `fetcher`, or any fetcher-shaped consumer). `perEndpoint({ endpoint,
|
|
146
184
|
key, signal })` -> the per-endpoint work. `opts.tries` (default 1) is the spatial failover count.
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zakkster/lite-pick",
|
|
3
3
|
"author": "Zahary Shinikchiev <shinikchiev@yahoo.com>",
|
|
4
|
-
"version": "0.7.
|
|
4
|
+
"version": "0.7.2",
|
|
5
5
|
"description": "Zero-dependency, zero-GC load-balancing selection kernel: one hot pick() -> endpoint index over a fixed pool, 0 B/op steady-state. A pure selector (consumes health/circuit state, never a proxy) for the in-process hop, complementary to AWS NLB/ALB. Tree-shakeable ESM roster: RoundRobin, SmoothWRR, P2C, LeastConn, SED, NQ, PeakEWMA (latency-aware peak-EWMA), plus consistent hashing; the /pool subpath adds dispatch/settle counters + failover and a duck-typed query-cache fetcher.",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./Pick.js",
|
|
@@ -28,6 +28,7 @@
|
|
|
28
28
|
"Pool.d.ts",
|
|
29
29
|
"llms.txt",
|
|
30
30
|
"README.md",
|
|
31
|
+
"RECIPES.md",
|
|
31
32
|
"CHANGELOG.md",
|
|
32
33
|
"LICENSE"
|
|
33
34
|
],
|