@zakkster/lite-pick 0.9.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +194 -0
- package/GUIDE.md +105 -0
- package/Pick.d.ts +85 -28
- package/Pick.js +364 -62
- package/Pool.d.ts +63 -17
- package/Pool.js +275 -62
- package/README.md +73 -36
- package/RECIPES.md +132 -36
- package/llms.txt +166 -75
- package/package.json +22 -5
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,200 @@ All notable changes to `@zakkster/lite-pick` are documented here. The format fol
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to
|
|
5
5
|
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [1.0.1] - 2026-09-27
|
|
8
|
+
|
|
9
|
+
Bug-fix release: the fixes from the full audit at `audit/2026-09-26/` (audited code state commit
|
|
10
|
+
`8c1ecc7`; H1-H4 and M1 independently reproduced on darwin/arm64 Node 26 before fixing). No new
|
|
11
|
+
strategy, no new public class, no removed API -- caller-visible behaviour changes only where a
|
|
12
|
+
documented contract was wrong or unsafe. See [ADR 0013](./decisions/0013-audit-1.0.1.md).
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- **PeakEWMA + Pool no longer turns a fast-failing or hung endpoint into a black hole (H1).**
|
|
17
|
+
Kernel side: an unsampled node now costs 0 only WHILE IDLE (so it holds one request in flight at a
|
|
18
|
+
time until its first sample); an unsampled BUSY node is priced at the LIFETIME mean of all recorded
|
|
19
|
+
samples, not the old 1.0 ns baseline; a sampled node with work in flight is floored at
|
|
20
|
+
time-since-last-sample, so a hung node gets MORE expensive over time instead of decaying toward 0.
|
|
21
|
+
`Pool.run` side: a thrown attempt now feeds `recordRtt(i, max(elapsed, failurePenaltyNs), done)`
|
|
22
|
+
(new `opts.failurePenaltyNs`, default `1e9`), so a failing node stops being the cheapest pick.
|
|
23
|
+
Measured (4 PeakEWMA nodes, node 0 always throws): failure rate 49.4% (988/2000) in 1.0.0 -> ~0.05%
|
|
24
|
+
(1/2000); a hung node took 1 dispatch, was 4891/10000.
|
|
25
|
+
- **SmoothWRR never returns a weight-0 node (H3).** `pick()` now requires a candidate to be eligible
|
|
26
|
+
AND have `weight > 0` (path-independent -- covers `setWeight(i, 0)` and eligibility toggles), and
|
|
27
|
+
`setWeight` resets that node's accumulated credit. A drained node is no longer selected.
|
|
28
|
+
- **BoundedLoad keeps key affinity at the low-load boundary (H4).** The per-backend cap is now
|
|
29
|
+
`ceil((1 + eps) * (total + 1) / live)`; the `+ 1` counts the incoming request
|
|
30
|
+
(Mirrokni-Thorup-Zadimoghaddam per-bin capacity), so the cap is always >= 1. The `Math.ceil` is a
|
|
31
|
+
no-op for the integer `inflight < cap` test; behaviour is identical to 1.0.0 except at the boundary
|
|
32
|
+
where the old cap fell below 1.
|
|
33
|
+
- **Non-integer, NaN, negative and string indices are rejected instead of silently desyncing state
|
|
34
|
+
(M1).** One shared index-validation helper (`(i >>> 0) === i && i < capacity`) guards `setEligible`,
|
|
35
|
+
`setWeight` (SmoothWRR / ConsistentHash / BoundedLoad / WeightedRandom), `note` and `recordRtt`:
|
|
36
|
+
they throw `RangeError` for `NaN`, fractions (`1.5`), negatives, out-of-range, and non-numbers
|
|
37
|
+
(including numeric strings like `'2'`, which previously "worked"); a rejected call changes no state.
|
|
38
|
+
`isEligible` returns `false` for a non-integer instead of `true`.
|
|
39
|
+
- **`Pool` failover now reaches a genuinely DIFFERENT endpoint (M2).** A per-run tried set, up to 8
|
|
40
|
+
re-picks, then a scan for an eligible untried endpoint from a key-derived start (keyed) or a
|
|
41
|
+
rotating per-Pool cursor (unkeyed); when no untried eligible endpoint remains, failover STOPS and
|
|
42
|
+
the last error is thrown (a 1-node pool with `tries: 3` now makes 1 attempt). ConsistentHash keyed
|
|
43
|
+
`tries: 3` now hits 3 distinct backends (was `3, 3, 3`); a failing backend's keys spread over
|
|
44
|
+
neighbours (max share ~23%, was 100% onto one neighbour in an intermediate build).
|
|
45
|
+
- **A `recordRtt` exception after a SUCCESSFUL call no longer re-runs `fn` (M4).** Settle-time
|
|
46
|
+
feedback runs OUTSIDE the attempt's try/catch, so `fn` runs exactly once; if it fails, `run` rejects
|
|
47
|
+
with a `LITE_PICK_FEEDBACK`-coded error carrying `.cause` and `.result` (fn's resolved value).
|
|
48
|
+
- **`Pool.run(fn, null)` works (L1).** `opts` may be omitted or `null`; the option reads are null-safe.
|
|
49
|
+
- **A throwing custom `note(+1)` no longer leaves an unpaired `note(-1)`** at settle: `note(-1)` fires
|
|
50
|
+
only for a dispatch whose `note(+1)` actually landed, and a cleanup-time throw is swallowed so it
|
|
51
|
+
never masks the error being thrown.
|
|
52
|
+
- **PeakEWMA `dt` is clamped `>= 0` in `pick`, `recordRtt` and `ewmaAt` (L6),** so a non-monotonic
|
|
53
|
+
clock can no longer inflate the estimate via `exp(+x)`.
|
|
54
|
+
- **`verify` is green (H5).** The PerfGate WeightedRandom heavy-outage scenario is shrunk (`FB_CAP`
|
|
55
|
+
2048; slowest phase 16.7 s -> ~3 s) so V8's memory reducer can no longer fire inside it.
|
|
56
|
+
|
|
57
|
+
### Changed
|
|
58
|
+
|
|
59
|
+
- **`Pool.run` keyed and latency channels are now separate and REQUIRED (M3).** A keyed balancer
|
|
60
|
+
(`ConsistentHashBalancer` / `BoundedLoadBalancer`, marked `static KEYED = true`) requires a numeric
|
|
61
|
+
`opts.key` (else `LITE_PICK_KEY_REQUIRED`); a latency balancer (`PeakEwmaBalancer`, marked
|
|
62
|
+
`static LATENCY = true`) requires an `opts.clock` (else `LITE_PICK_CLOCK_REQUIRED`). The key reaches
|
|
63
|
+
ONLY a keyed pick; a clock reading NEVER reaches a keyed pick. A non-keyed clocked run still passes
|
|
64
|
+
the clock reading to `pick(now)` (preserving 1.0.0 behaviour for a duck-typed latency balancer that
|
|
65
|
+
omits the marker). Previously a missing key routed everything to one backend, and `key` / `clock`
|
|
66
|
+
were conflated.
|
|
67
|
+
- **`Pool.run` checks the abort signal before EVERY attempt.** An already-aborted signal now
|
|
68
|
+
dispatches nothing and rejects (`throwIfAborted`, then the signal's `reason`, else
|
|
69
|
+
`LITE_PICK_ABORTED`), including a structural `{ aborted: true }` signal with no `throwIfAborted`.
|
|
70
|
+
- **A backwards but finite clock reading on a successful settle records NO rtt sample** (a fabricated
|
|
71
|
+
0 ns sample would make the node look instant); `done === now` is a real coarse-clock 0 and is
|
|
72
|
+
recorded. A non-finite clock reading throws before dispatch.
|
|
73
|
+
- **On a failed attempt whose penalty feedback then fails, `Pool.run` throws `fn`'s original error
|
|
74
|
+
object (identity preserved)** with a non-enumerable `liteFeedbackError` attached, and stops
|
|
75
|
+
failover.
|
|
76
|
+
- **`liteQueryFetcher` forwards `clock` and `failurePenaltyNs`** to `pool.run` (validated once at
|
|
77
|
+
creation). Its `ctx.key` remains the query-cache key, not a routing key.
|
|
78
|
+
- **PeakEWMA `tau` is documented as the EWMA TIME CONSTANT (half-life = `tau x ln2`),** correcting the
|
|
79
|
+
earlier "half-life" wording (L5). No math change.
|
|
80
|
+
- **Static class markers added:** `ConsistentHashBalancer.KEYED` (inherited by `BoundedLoadBalancer`)
|
|
81
|
+
and `PeakEwmaBalancer.LATENCY`, so `/pool` selects the right channel while staying duck-typed.
|
|
82
|
+
- **BoundedLoad docs corrected (M-Doc1):** update `inflight[i]` AND call `note(i, +/-1)` in lockstep
|
|
83
|
+
(or drive it through `/pool`); `note` maintains `_total`, it does not write `inflight`.
|
|
84
|
+
- **Types tightened:** `pick(now)` / `pick(keyHash)` are now REQUIRED on the concrete
|
|
85
|
+
`PeakEwmaBalancer` / `ConsistentHashBalancer` / `BoundedLoadBalancer` classes (`BalancerBase.pick`
|
|
86
|
+
stays deliberately loose, `pick(arg?)`); the `KEYED` / `LATENCY` markers are typed; `Pool.d.ts` now
|
|
87
|
+
uses a structural `AbortLike` type (compiles with `lib: ES2022` and no DOM, L15).
|
|
88
|
+
|
|
89
|
+
### Added
|
|
90
|
+
|
|
91
|
+
- **`opts.failurePenaltyNs`** on `Pool.run` (and `liteQueryFetcher`): the minimum rtt penalty a thrown
|
|
92
|
+
attempt feeds a latency-aware balancer (finite `> 0`, default `1e9`).
|
|
93
|
+
- **`typesVersions`** maps `@zakkster/lite-pick/pool` -> `Pool.d.ts` for `moduleResolution: node10`
|
|
94
|
+
(L16); `test:types` gains an ES2022-no-DOM lane.
|
|
95
|
+
- **`.github/workflows/ci.yml`:** `test` (Node 20/22/24 x ubuntu/macos/windows), `test-node18` (unit
|
|
96
|
+
suites only, backing `engines >=18`), `gates` (torture, `test:perf`, `bench:verify`, an exact
|
|
97
|
+
11-file tarball check), and `types-compat` (the packed tarball compiled with TypeScript 5 under
|
|
98
|
+
node10 / node16 / bundler / ES2022-only on case-sensitive Linux). `package-lock.json` is now
|
|
99
|
+
committed (maintainer decision; follows lite-di-container).
|
|
100
|
+
- **`test` is now an explicit file list** (portable to Windows / Node 20 -- no shell glob), guarded by
|
|
101
|
+
`test/suite-list.test.js`, which fails if any test file under `test/` is not run.
|
|
102
|
+
- **Gates hardened with teeth:** `test:perf` pins the semi-space `min = max = 1 MB` (sharper than
|
|
103
|
+
1.0.0's `max = 4 MB`) with a fail-closed test asserting the pin; must-fail allocations escape via a
|
|
104
|
+
64-slot ring and `grows` compares buffer identity (M-T3); new intermittent must-fail controls (an
|
|
105
|
+
allocation every 16 and every 32 picks). `torture` now reports RETAINED B/op, treats an unmeasured
|
|
106
|
+
reading as FAIL, and has a retaining must-fail control (trips at 40 B/op) (H6); `GcBlastRadius`
|
|
107
|
+
likewise, and its README column is renamed "pick retained B/op". Every `benchmark/*.mjs` entry check
|
|
108
|
+
uses `pathToFileURL` (M-T2); `bench:verify` states which blocks are re-measured vs compared to stored
|
|
109
|
+
`results.json` (M-T1); the fuzz single-node-down block uses one eligibility array per balancer (M-T4)
|
|
110
|
+
and prints the discovery seed every run (L21).
|
|
111
|
+
- Demo (repo-only, not in the tarball): the Pool Scope web server binds `127.0.0.1`, serves GET/HEAD
|
|
112
|
+
only, enforces a path allowlist + Host-header check + `path.relative`/`realpath` traversal guard +
|
|
113
|
+
`nosniff` + port validation (M-D1, M-D2, L26); the driver keeps BoundedLoad's `totalInflight` in
|
|
114
|
+
lockstep (M-D3); the TUI reset uses `DEFAULT_CONC` and restores the terminal on SIGTERM/SIGHUP with
|
|
115
|
+
the alternate screen (M-D4, L23); fanout counts only distinct failovers and reports exhausted
|
|
116
|
+
requests (L24).
|
|
117
|
+
|
|
118
|
+
### Known limitations
|
|
119
|
+
|
|
120
|
+
Disclosed, shipped, and slated for 1.1.0:
|
|
121
|
+
|
|
122
|
+
- **A realistic (non-small-integer) number argument boxes once per non-inlined call.** V8 boxes a
|
|
123
|
+
`HeapNumber` (~16 B) for a `PeakEWMA` `pick(now ~1.7e15)`, ~16-30 B for `recordRtt` with a realistic
|
|
124
|
+
clock and a fractional sample, and ~16 B for `ConsistentHash`/`BoundedLoad` `pick(keyHash >= 2^31)`.
|
|
125
|
+
Small-integer arguments are 0 B/op (range is build-dependent: `< 2^31` on stock 64-bit Node, `< 2^30`
|
|
126
|
+
on pointer-compressed builds); values produced by `%` or division may box even when small. 1.1.0 adds
|
|
127
|
+
buffer-based variants that read the clock/key from a caller-owned typed array (prototype measured
|
|
128
|
+
0 B/op at 1e15). `test:perf` prints these as report-only lines every run.
|
|
129
|
+
- **OPEN:** run in isolation with some integer sample patterns (e.g. mod 500000, step 1000), the
|
|
130
|
+
PerfGate `recordRtt` lane shows a small allocation that scales with window length (8N: 2, 16N: 4,
|
|
131
|
+
32N: 9 scavenges at the 1 MB pin); it disappears with `--no-maglev`,
|
|
132
|
+
`--no-concurrent-recompilation`, or a preceding lane. Root cause not established; printed as a
|
|
133
|
+
report-only line; tracked for 1.1.0.
|
|
134
|
+
- **M5 (LeastConn / NQ tie order) is now documented as unspecified.** A rotating tie-break lands in
|
|
135
|
+
1.1.0; it was moved OUT of 1.0.1 because "lowest index" was an explicit documented promise, so
|
|
136
|
+
changing it is not a patch-level fix.
|
|
137
|
+
|
|
138
|
+
### Not in this release
|
|
139
|
+
|
|
140
|
+
- The soak redesign (audit RECOMMENDATIONS section 1): the 1.0.0 soak's drift gates compared lanes
|
|
141
|
+
rather than time, and its heap sample included harness bookkeeping. Redesign pending.
|
|
142
|
+
- Observability (section 2): zero-cost counters, `diagnostics_channel`/hooks, `describe()`, error codes.
|
|
143
|
+
- The `Eligibility` object as the shared unit (section 3.1) -- a breaking change deferred to 2.0.
|
|
144
|
+
|
|
145
|
+
## [1.0.0] - 2026-09-23
|
|
146
|
+
|
|
147
|
+
The **roster-complete** release: ten selection strategies + the `/pool` request layer + the benchmark
|
|
148
|
+
suite + the docs/GUIDE capstone. Roster complete **for now, not closed** -- AZ-aware routing, the
|
|
149
|
+
lite-await hedging combinator, and subsetting are queued post-1.0 (see [ROADMAP.md](./ROADMAP.md)).
|
|
150
|
+
|
|
151
|
+
### Added
|
|
152
|
+
|
|
153
|
+
- **`WeightedRandomBalancer` (M10)** -- O(1) weighted-random selection via an inline **Vose/Walker alias
|
|
154
|
+
table** (one column draw + one probability compare -> a candidate) with **rejection-sampling
|
|
155
|
+
eligibility** (retry an ineligible candidate up to a bounded 64, then a 0-B/op rotated linear eligible
|
|
156
|
+
scan) -- the ADR 0005 / P2C discipline. The alias table is built **cold** over the eligible-independent
|
|
157
|
+
weights, so a **weight-0 node is never a column** (never returned) and rejecting the ineligible draws
|
|
158
|
+
**renormalizes** the weight distribution over the surviving eligible mass (each eligible node's share
|
|
159
|
+
converges to `weight[i] / sum(eligible weights)`). `weights` is the caller's `Uint32Array` (the
|
|
160
|
+
SmoothWRR/SED seam); the balancer is the **sole writer** of its derived table (`_prob` / `_alias`) via
|
|
161
|
+
cold `setWeight` / `rebuild` -- an eligibility flap **never** rebuilds (anti-flap). Validates
|
|
162
|
+
typeof-first before allocating the table. `pick()` is **O(1)**, **0 B/op**, never throws; `PICK_NONE`
|
|
163
|
+
only when `live === 0` or no eligible node has a positive weight. It is the **stateless** O(1) weighted
|
|
164
|
+
sampler (no accumulator to desync) for very large pools where SmoothWRR's O(cap) scan hurts -- trading
|
|
165
|
+
smoothness for sampling variance. `peerDependencies` stays `{}` (the Vose build is inlined; a lite-o1
|
|
166
|
+
`AliasTable` and a lite-logn Fenwick tree are deferred optional peers, imported by nothing).
|
|
167
|
+
([ADR 0012](./decisions/0012-weightedrandom.md)).
|
|
168
|
+
- **Fairness anchor** (`test/balance.mjs`) -- n=64, skewed weights [1..16], 8e6 seeded draws: every node's
|
|
169
|
+
observed share is within **2% relative** of `weight[i]/sum` (measured worst ~0.84%), and a cumsum-linear
|
|
170
|
+
O(n) foil matches the same fairness. The O(1) alias sample beats that O(n) foil by **~107x ops/ms** at
|
|
171
|
+
n=4096 (gate: >=3x). Under half the pool down (1e6 picks): **0 ineligible / 0 weight-0** returns,
|
|
172
|
+
survivor shares within **3% relative** of `weight[i]/sum(eligible)` (measured worst ~1.77%), and all-zero
|
|
173
|
+
weights -> `PICK_NONE`. Thresholds are the sampling-variance floor from a correct run (N sized so the
|
|
174
|
+
band holds with margin) -- the band is never widened to pass.
|
|
175
|
+
- **`GUIDE.md`** -- the "which of the ten strategies do I pick?" decision guide (a decision tree + table
|
|
176
|
+
keyed by keyed-vs-load-vs-latency-vs-weighted, O(1) vs O(cap), state owned, and when each wins),
|
|
177
|
+
distinct from `RECIPES.md` (how-to wiring). Added to the published `files[]` (the tarball is now 11 files).
|
|
178
|
+
- Gates extended for the new strategy: `test/WeightedRandom.test.js` boundary + behaviour suite;
|
|
179
|
+
`test/fuzz.mjs` keyed-agnostic subject + `checkWeightedRandom` (structural + no-weight-0-column + the
|
|
180
|
+
sum-reconstruction invariant, after every op) + a 1000-flap **0-rebuild** anti-flap assertion;
|
|
181
|
+
`test/torture.mjs` retention + a `pick()` 0 B/op phase (phase 14); `test/perf/PerfGate.test.mjs`
|
|
182
|
+
`weightedRandomPick` zero-alloc scenario + a boxed `mustFail` tooth; `test/witness.mjs` 'const'
|
|
183
|
+
flat-work subject; `benchmark/Matrix.mjs` throughput + fairness subject; `benchmark/Report.mjs`
|
|
184
|
+
Weighted-random parity row now times `WeightedRandomBalancer` vs `wrr` (both O(1) weighted-random --
|
|
185
|
+
previously a pending SKIP); `Pick.d.ts` + `test/types/pick.test-d.ts` typed surface.
|
|
186
|
+
|
|
187
|
+
### Changed
|
|
188
|
+
|
|
189
|
+
- **`Pick.js` header roster/count** nine -> **ten**, and the `VERSION` stamp `0.9.0` -> `1.0.0` (the
|
|
190
|
+
three-place sync: `package.json`, the `VERSION` const, `llms.txt`). This session appends **one** class;
|
|
191
|
+
every other strategy in `Pick.js` is byte-identical.
|
|
192
|
+
- **`package.json`** version `1.0.0`, the description roster gains WeightedRandom + a GUIDE.md pointer,
|
|
193
|
+
keywords gain `alias-method` / `vose` (`weighted-random` was already present), and `files[]` gains
|
|
194
|
+
`GUIDE.md`. `peerDependencies` stays `{}`.
|
|
195
|
+
- Sibling boundary documented explicitly (so 1.0.0's WeightedRandom does not look duplicative with
|
|
196
|
+
`@zakkster/lite-random`): lite-random is a **game RNG** returning an item, not eligibility-aware, no
|
|
197
|
+
reusable table; lite-pick WeightedRandom returns an endpoint index, honours the shared eligibility
|
|
198
|
+
bitmap (fail-closed), and owns a persistent alias table -- different domain, not a peer (GUIDE.md,
|
|
199
|
+
llms.txt, ADR 0012).
|
|
200
|
+
|
|
7
201
|
## [0.9.0] - 2026-09-23
|
|
8
202
|
|
|
9
203
|
### Added
|
package/GUIDE.md
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Which strategy? -- the lite-pick decision guide
|
|
2
|
+
|
|
3
|
+
`@zakkster/lite-pick` ships **ten** selection strategies. They are not ranked; each wins a different
|
|
4
|
+
job. This guide is how you CHOOSE one. It is deliberately distinct from [RECIPES.md](./RECIPES.md),
|
|
5
|
+
which shows how to WIRE a chosen strategy (dispatch/settle counters, health, the `/pool` layer).
|
|
6
|
+
|
|
7
|
+
Every strategy shares the same contract: a hot `pick()` returning an endpoint **index** over a fixed
|
|
8
|
+
pool, **fail-closed** (`PICK_NONE` = -1 when nothing is pickable, never a dead pick), reading a
|
|
9
|
+
**read-only eligibility bitmap** it never writes -- flipped only through `setEligible` (the sole
|
|
10
|
+
supported writer, which keeps the cached `live` exact; a direct byte write desyncs it), and each
|
|
11
|
+
balancer has its own eligibility array.
|
|
12
|
+
|
|
13
|
+
`pick()` **allocates 0 B/op** (PerfGate scavenge counting) and **retains 0 B/op** (torture) in the
|
|
14
|
+
steady state. KNOWN LIMITATION (1.0.1): the strategies that take a **number argument** on the hot
|
|
15
|
+
path -- `PeakEWMA.pick(now)` / `recordRtt(..., now)` with a realistic nanosecond clock, and
|
|
16
|
+
`ConsistentHash`/`BoundedLoad.pick(keyHash)` with a key `>= 2^31` -- box that argument into a ~16 B
|
|
17
|
+
transient `HeapNumber` when the call is not inlined (transient, does not retain, does not force a
|
|
18
|
+
major GC). Arguments within V8's small-integer range are 0 B/op; that range is build-dependent (below
|
|
19
|
+
2^31 on stock 64-bit Node, below 2^30 on pointer-compressed builds such as Chrome/Electron), and a
|
|
20
|
+
value produced by `%` or division can box even when its value is a small integer. Buffer-based
|
|
21
|
+
variants are planned for 1.1.0.
|
|
22
|
+
|
|
23
|
+
## The one question that splits everything: what fixes the primary choice?
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
Is routing decided by a KEY (same key -> same backend, for cache/session affinity)?
|
|
27
|
+
|
|
|
28
|
+
+-- YES -> you want a CONSISTENT HASH.
|
|
29
|
+
| |
|
|
30
|
+
| +-- Do a few hot keys overload one backend?
|
|
31
|
+
| | NO -> ConsistentHash (sticky, minimal disruption, O(1))
|
|
32
|
+
| | YES -> BoundedLoad (sticky + an occupancy cap that overflows a hotspot, O(1))
|
|
33
|
+
|
|
|
34
|
+
+-- NO -> the choice is by LOAD / LATENCY / WEIGHT, not a key.
|
|
35
|
+
|
|
|
36
|
+
+-- Do you have a LATENCY signal (rtt) and want to steer around a slow-but-up node?
|
|
37
|
+
| YES -> PeakEWMA (latency-aware power-of-two-choices, O(1))
|
|
38
|
+
|
|
|
39
|
+
+-- Do you have live IN-FLIGHT counts (a closed dispatch/settle loop)?
|
|
40
|
+
| |
|
|
41
|
+
| +-- Want the EXACT least-loaded, and O(cap) is fine (dozens-hundreds of nodes)?
|
|
42
|
+
| | unweighted -> LeastConn (exact fewest-in-flight, O(cap))
|
|
43
|
+
| | weighted -> SED (minimizes (inflight+1)/weight, O(cap))
|
|
44
|
+
| | worker pool (idle-first) -> NQ (never-queue: idle node first, else SED, O(cap))
|
|
45
|
+
| |
|
|
46
|
+
| +-- Want O(1) at very large pools and can accept a tiny balance gap?
|
|
47
|
+
| -> P2C (power-of-two-choices, the ln ln n ceiling, O(1))
|
|
48
|
+
|
|
|
49
|
+
+-- No load signal -- just spread by a fixed WEIGHT (or evenly)?
|
|
50
|
+
|
|
|
51
|
+
+-- Equal weight, simple rotation -> RoundRobin (O(1) amortized)
|
|
52
|
+
+-- Weighted, want SMOOTH low-variance -> SmoothWRR (deterministic, O(cap))
|
|
53
|
+
+-- Weighted, want STATELESS O(1) at scale -> WeightedRandom (alias table, O(1))
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## The table
|
|
57
|
+
|
|
58
|
+
| Strategy | Decides by | Bound / pick | State the balancer owns | Wins when |
|
|
59
|
+
| --- | --- | --- | --- | --- |
|
|
60
|
+
| **RoundRobin** | rotation | O(1) amortized | a cursor | equal weight, no load signal, simplest fair spread |
|
|
61
|
+
| **SmoothWRR** | fixed weight | O(cap) | smoothing accumulators (`_current`) | weighted **and** you want deterministic, smooth, low-variance interleaving |
|
|
62
|
+
| **WeightedRandom** | fixed weight | **O(1)** | a Vose alias table (`_prob`/`_alias`) | weighted at **very large** pools where SmoothWRR's O(cap) scan hurts; can accept sampling variance |
|
|
63
|
+
| **P2C** | in-flight load | **O(1)** | just a PRNG | O(1) load-balancing at scale; the `ln ln n` peak ceiling, a tiny gap vs exact |
|
|
64
|
+
| **LeastConn** | in-flight load | O(cap) | none (reads live) | the **exact** least-loaded in a feedback loop; dozens-hundreds of nodes |
|
|
65
|
+
| **SED** | (inflight+1)/weight | O(cap) | none (reads live) | **weighted** exact least-loaded (load settles proportional to weight) |
|
|
66
|
+
| **NQ** | idle-first, else SED | O(cap) | none (reads live) | **worker pools** -- never queue while a server is idle |
|
|
67
|
+
| **PeakEWMA** | (inflight+1) x decayed rtt | **O(1)** | EWMA rtt state (`_ewma`/`_stamp`) | you have latency and want to steer around a **slow-but-up** node |
|
|
68
|
+
| **ConsistentHash** | key hash (Maglev) | **O(1)** | a Maglev lookup table | **sticky** cache/session affinity; minimal disruption on scale events (~1/N keys move) |
|
|
69
|
+
| **BoundedLoad** | key hash + occupancy cap | **O(1)** | Maglev table + a running `_total` | sticky routing **and** a few hot keys would otherwise overload one backend |
|
|
70
|
+
|
|
71
|
+
## SmoothWRR vs WeightedRandom -- the weighted fork, made explicit
|
|
72
|
+
|
|
73
|
+
Both send load proportional to a configured integer weight. They differ in HOW and in cost:
|
|
74
|
+
|
|
75
|
+
- **SmoothWRR** is DETERMINISTIC and SMOOTH: weights `[5,1,1]` yield `A A B A C A A`, not bursts. It
|
|
76
|
+
converges EXACTLY (counts == k x weight over a cycle) with the lowest variance. Cost: **O(cap) per
|
|
77
|
+
pick**, and it owns per-endpoint accumulator state that is maintained in lockstep.
|
|
78
|
+
- **WeightedRandom** is a STATELESS **O(1)** sample from a Vose alias table: one column draw + one
|
|
79
|
+
compare. It converges to the weight ratios by the law of large numbers (any single pick is random --
|
|
80
|
+
it pays SAMPLING VARIANCE). No accumulator to desync.
|
|
81
|
+
|
|
82
|
+
Rule of thumb: **small-to-medium pools or when smoothness matters -> SmoothWRR; very large pools where
|
|
83
|
+
the O(cap) scan hurts -> WeightedRandom.** For **frequently-changing** weights, a `@zakkster/lite-logn`
|
|
84
|
+
Fenwick tree (O(log n) update + sample) is the deferred dynamic-weight complement to WeightedRandom's
|
|
85
|
+
static alias table (rebuilt cold on reweight); see the roadmap.
|
|
86
|
+
|
|
87
|
+
## Not sure you even want lite-pick? -- the sibling boundary
|
|
88
|
+
|
|
89
|
+
- **`@zakkster/lite-random` is NOT a load balancer.** It is a GAME RNG (Mulberry32) for loot tables,
|
|
90
|
+
particle systems, and gaussian sampling; its `weighted(items, weights) -> T` returns an **item**
|
|
91
|
+
one-shot, is not eligibility-aware, and holds no reusable table. For **game loot tables use
|
|
92
|
+
lite-random**; **lite-pick WeightedRandom is the eligibility-aware LB selector** (returns an endpoint
|
|
93
|
+
INDEX, honours the shared eligibility bitmap, owns a persistent alias table rebuilt only on reweight,
|
|
94
|
+
fail-closed). Different domain, different contract -- lite-random is not a peer or a substrate here.
|
|
95
|
+
- lite-pick is the **in-process** hop selector: it consumes health/circuit state and returns an index;
|
|
96
|
+
it is never a proxy. It is complementary to AWS NLB/ALB (which balance the network hop AWS sees) --
|
|
97
|
+
ALB's `weighted_random` + anomaly mitigation is lite-pick's WeightedRandom + BoundedLoad, and NLB's
|
|
98
|
+
flow-hash is ConsistentHash, at the hop AWS never sees.
|
|
99
|
+
|
|
100
|
+
## Then wire it
|
|
101
|
+
|
|
102
|
+
Once you have picked a strategy, [RECIPES.md](./RECIPES.md) shows the real wiring: the shared
|
|
103
|
+
eligibility bitmap from a health source, the caller-owned in-flight / weight arrays, the `/pool`
|
|
104
|
+
dispatch/settle + failover layer, and the latency (`recordRtt`) / occupancy (`note`) / keyed
|
|
105
|
+
(`opts.key`) feedback hooks.
|
package/Pick.d.ts
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* @zakkster/lite-pick -- TypeScript declarations.
|
|
3
3
|
*
|
|
4
|
-
*
|
|
4
|
+
* M10 (1.0.0): substrate seams + RoundRobin + SmoothWRR + P2C + the exact LeastConn family
|
|
5
5
|
* (LeastConn/SED/NQ) + PeakEWMA (latency-aware P2C) + ConsistentHash (Maglev table) +
|
|
6
|
-
* BoundedLoad (
|
|
7
|
-
*
|
|
6
|
+
* BoundedLoad (consistent hashing with bounded loads) + WeightedRandom (O(1) Vose alias-table
|
|
7
|
+
* sampling). Roster complete for now (NOT closed: AZ-aware routing, hedging, subsetting post-1.0).
|
|
8
8
|
*/
|
|
9
9
|
|
|
10
10
|
/** The single source-of-truth version stamp. */
|
|
@@ -32,27 +32,35 @@ export class Prng {
|
|
|
32
32
|
}
|
|
33
33
|
|
|
34
34
|
/**
|
|
35
|
-
* The
|
|
36
|
-
*
|
|
37
|
-
*
|
|
38
|
-
*
|
|
35
|
+
* The eligibility seam for every strategy. Owns the fixed capacity, a reference to a
|
|
36
|
+
* read-only eligibility `Uint8Array`, and an O(1) live count. The array is flipped ONLY
|
|
37
|
+
* through `setEligible` (the sole supported writer -- @zakkster/lite-di-health / circuit
|
|
38
|
+
* breakers drive that call); a direct byte write desyncs the cached live count (UB). Each
|
|
39
|
+
* balancer needs its own array. Subclasses implement `pick()`; the base `pick()` throws.
|
|
39
40
|
*/
|
|
40
41
|
export class BalancerBase {
|
|
41
42
|
/**
|
|
42
43
|
* @param capacity endpoint count (fixed).
|
|
43
|
-
* @param eligible
|
|
44
|
+
* @param eligible 1 = pickable, 0 = down (length >= capacity); per-balancer, flipped only via `setEligible`.
|
|
44
45
|
*/
|
|
45
46
|
constructor(capacity: number, eligible: Uint8Array);
|
|
46
47
|
/** Endpoint count (fixed at construction). */
|
|
47
48
|
readonly capacity: number;
|
|
48
49
|
/** Number of currently eligible endpoints (O(1)). */
|
|
49
50
|
readonly live: number;
|
|
50
|
-
/** True iff endpoint `i` is currently pickable. */
|
|
51
|
+
/** True iff endpoint `i` is currently pickable; a non-integer or out-of-range `i` is `false` (never throws). */
|
|
51
52
|
isEligible(i: number): boolean;
|
|
52
|
-
/** Cold path:
|
|
53
|
+
/** Cold path: the only supported eligibility writer -- flips `i` up/down and keeps the live count exact.
|
|
54
|
+
* Throws `RangeError` on a non-integer or out-of-range index (incl. a numeric string like '2'). */
|
|
53
55
|
setEligible(i: number, up: boolean): void;
|
|
54
|
-
/**
|
|
55
|
-
|
|
56
|
+
/**
|
|
57
|
+
* Choose an endpoint index, or `PICK_NONE`. Abstract in the base (throws). Declared with a
|
|
58
|
+
* DELIBERATELY LOOSE optional numeric argument so a keyed subclass (`pick(keyHash)`) or a latency
|
|
59
|
+
* subclass (`pick(now)`) that requires it stays assignable to `BalancerBase` (`const b:
|
|
60
|
+
* BalancerBase = ch; b.pick()` type-checks). To get the required-arg compile check, reference the
|
|
61
|
+
* CONCRETE class type (e.g. `ConsistentHashBalancer` / `PeakEwmaBalancer`), not `BalancerBase`.
|
|
62
|
+
*/
|
|
63
|
+
pick(arg?: number): number;
|
|
56
64
|
}
|
|
57
65
|
|
|
58
66
|
/**
|
|
@@ -109,8 +117,9 @@ export class P2cBalancer extends BalancerBase {
|
|
|
109
117
|
|
|
110
118
|
/**
|
|
111
119
|
* LeastConnBalancer -- EXACT fewest-in-flight (M4, IPVS `lc`). A full O(cap) scan of the
|
|
112
|
-
* caller-owned in-flight view returning the eligible node with the lowest count (
|
|
113
|
-
*
|
|
120
|
+
* caller-owned in-flight view returning the eligible node with the lowest count (tie order is
|
|
121
|
+
* UNSPECIFIED in 1.0.1 -- deterministic, but do not depend on it; a rotating tie-break is planned
|
|
122
|
+
* for 1.1.0) -- the deterministic complement to P2C's O(1) approximation. In-flight counts are
|
|
114
123
|
* caller-owned and read LIVE (no `setWeight`, no derived aggregate). 0 B/op. Fails closed
|
|
115
124
|
* (`PICK_NONE`) when the whole pool is down.
|
|
116
125
|
*/
|
|
@@ -164,20 +173,27 @@ export class NqBalancer extends BalancerBase {
|
|
|
164
173
|
|
|
165
174
|
/**
|
|
166
175
|
* PeakEwmaBalancer -- latency-aware power-of-two-choices (M7, Twitter Finagle's peak-EWMA).
|
|
167
|
-
* Draws two distinct eligible endpoints and returns the
|
|
168
|
-
*
|
|
176
|
+
* Draws two distinct eligible endpoints and returns the LOWER COST (three cases: unsampled+idle -> 0;
|
|
177
|
+
* unsampled+busy -> `(inflight + 1) x lifetime mean`; sampled -> `(inflight + 1) x max(decayedEWMA,
|
|
178
|
+
* dt-while-busy)`); a slow endpoint (high decayed EWMA rtt) is avoided even with a short queue,
|
|
179
|
+
* and a hung node grows more expensive over time. `inflight` is the
|
|
169
180
|
* caller-owned Uint32Array read LIVE; the EWMA state (`_ewma` / `_stamp`, Float64) is BALANCER-OWNED
|
|
170
181
|
* and written ONLY by `recordRtt` (the warm feedback path). `pick(now)` decays on READ -- never
|
|
171
182
|
* writes -- so it is 0 B/op, as is `recordRtt`. `now` / `sampleNs` are caller-supplied nanoseconds.
|
|
172
|
-
* Cold start
|
|
173
|
-
*
|
|
183
|
+
* Cold start: an unsampled node costs 0 WHILE IDLE (graceful least-connections) and the pool's
|
|
184
|
+
* lifetime mean sampled rtt ONCE BUSY, so a fast-failing or hung node cannot masquerade as a 1.0 ns
|
|
185
|
+
* node and become a black hole; a hung node's busy floor grows with `dt` so it gets more expensive,
|
|
186
|
+
* not less. Never NaN. O(d)=O(1). Fails closed (`PICK_NONE`) when the whole pool is down.
|
|
187
|
+
* Latency-aware: @zakkster/lite-pick/pool REQUIRES an `opts.clock` for this strategy.
|
|
174
188
|
*/
|
|
175
189
|
export class PeakEwmaBalancer extends BalancerBase {
|
|
190
|
+
/** Marker: latency-aware; /pool requires `opts.clock` and feeds recordRtt() from it. */
|
|
191
|
+
static readonly LATENCY: true;
|
|
176
192
|
/**
|
|
177
193
|
* @param capacity endpoint count (fixed).
|
|
178
194
|
* @param eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
179
195
|
* @param inflight per-endpoint in-flight counts (length >= capacity), caller-owned, read live.
|
|
180
|
-
* @param tauNs the EWMA
|
|
196
|
+
* @param tauNs the EWMA TIME CONSTANT in nanoseconds (finite, > 0); the half-life is tauNs x ln2.
|
|
181
197
|
* @param seed deterministic PRNG seed (default 0x9e3779b9); reproducible benches.
|
|
182
198
|
*/
|
|
183
199
|
constructor(capacity: number, eligible: Uint8Array, inflight: Uint32Array, tauNs: number, seed?: number);
|
|
@@ -186,7 +202,7 @@ export class PeakEwmaBalancer extends BalancerBase {
|
|
|
186
202
|
/** Warm feedback path: record an rtt sample (ns) for endpoint `i` at time `now` (ns). 0 B/op. */
|
|
187
203
|
recordRtt(i: number, sampleNs: number, now: number): void;
|
|
188
204
|
/** Pick by latency-aware power-of-two-choices at time `now` (ns), or `PICK_NONE`. O(d)=O(1). */
|
|
189
|
-
pick(now
|
|
205
|
+
pick(now: number): number;
|
|
190
206
|
}
|
|
191
207
|
|
|
192
208
|
/** The default Maglev lookup-table size (a prime, 2^16 + 1). Configurable via the ctor. */
|
|
@@ -207,6 +223,8 @@ export const CH_PROBE_LIMIT: number;
|
|
|
207
223
|
* Fails closed (`PICK_NONE`) when the pool is down or no eligible backend is reachable within the bound.
|
|
208
224
|
*/
|
|
209
225
|
export class ConsistentHashBalancer extends BalancerBase {
|
|
226
|
+
/** Marker: keyed; /pool requires a numeric `opts.key`. Inherited by BoundedLoadBalancer. */
|
|
227
|
+
static readonly KEYED: true;
|
|
210
228
|
/**
|
|
211
229
|
* @param capacity backend count (fixed).
|
|
212
230
|
* @param eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
@@ -223,22 +241,26 @@ export class ConsistentHashBalancer extends BalancerBase {
|
|
|
223
241
|
/** Cold path: rebuild the lookup table from the current owned weights. */
|
|
224
242
|
rebuild(): void;
|
|
225
243
|
/** Map an integer `keyHash` to a backend index (bounded probe past down slots), or `PICK_NONE`. */
|
|
226
|
-
pick(keyHash
|
|
244
|
+
pick(keyHash: number): number;
|
|
227
245
|
}
|
|
228
246
|
|
|
229
247
|
/**
|
|
230
248
|
* BoundedLoadBalancer -- Consistent Hashing with Bounded Loads (M9, CHBL: Mirrokni et al. / Google
|
|
231
249
|
* Research; Vimeo eps ~ 0.25). `ConsistentHashBalancer` (the Maglev table) PLUS an occupancy cap: a
|
|
232
|
-
* key sticks to its hashed home backend UNLESS that backend is over
|
|
233
|
-
*
|
|
250
|
+
* key sticks to its hashed home backend UNLESS that backend is over
|
|
251
|
+
* `cap = ceil((1 + eps) * (total + 1) / live)` -- the load-bearing part is the `+ 1` that counts the
|
|
252
|
+
* INCOMING request (Mirrokni-Thorup-Zadimoghaddam per-bin capacity), so the cap is always >= 1 and a
|
|
253
|
+
* second concurrent same-key request correctly overflows the home. In that case the request OVERFLOWS
|
|
254
|
+
* along the same bounded forward-probe to the next eligible,
|
|
234
255
|
* under-cap backend -- keeping consistent hashing's stickiness + minimal disruption AND adding the
|
|
235
256
|
* HOTSPOT protection plain consistent hashing lacks. `pick(keyHash)` returns the first eligible,
|
|
236
257
|
* under-cap backend in the probe window, else falls back to the first eligible seen (sticky wins; the
|
|
237
258
|
* cap is a soft preference, never a dead pick); `_total === 0` skips the cap -> pure ConsistentHash.
|
|
238
259
|
* `inflight` is the caller-owned Uint32Array read LIVE as the per-backend OCCUPANCY; the running
|
|
239
|
-
* occupancy sum `_total` is BALANCER-OWNED and written ONLY by `note` (dispatch +1 / settle -1)
|
|
240
|
-
*
|
|
241
|
-
* adapter
|
|
260
|
+
* occupancy sum `_total` is BALANCER-OWNED and written ONLY by `note` (dispatch +1 / settle -1). When
|
|
261
|
+
* using BoundedLoad you update `inflight[i]` AND call `note(i, +/-1)` in LOCKSTEP (or drive it through
|
|
262
|
+
* the /pool adapter, which does both): `note` maintains `_total`, it does not write `inflight`. A
|
|
263
|
+
* direct mutation of `inflight` without the matching `note` desyncs `_total` -- UB. It inherits the Maglev table + `setWeight` /
|
|
242
264
|
* `rebuild` / `tableSize` from ConsistentHashBalancer (reused verbatim). `pick()` and `note()` are
|
|
243
265
|
* both 0 B/op / O(1). Fails closed (`PICK_NONE`) ONLY when no eligible backend is reachable within
|
|
244
266
|
* the probe window -- NEVER merely because backends are over cap. NOT the P2C-with-cap "overload"
|
|
@@ -266,8 +288,43 @@ export class BoundedLoadBalancer extends ConsistentHashBalancer {
|
|
|
266
288
|
);
|
|
267
289
|
/** The balancer-owned running sum of in-flight the mean/cap is computed from. */
|
|
268
290
|
readonly totalInflight: number;
|
|
269
|
-
/**
|
|
291
|
+
/**
|
|
292
|
+
* Warm feedback path: adjust the owned occupancy SUM by `delta` (dispatch +1 / settle -1), in
|
|
293
|
+
* LOCKSTEP with the caller's `inflight[i]` write. Maintains `_total`; does NOT write `inflight`.
|
|
294
|
+
* Clamps at 0. 0 B/op.
|
|
295
|
+
*/
|
|
270
296
|
note(i: number, delta: number): void;
|
|
271
297
|
/** Map an integer `keyHash` to a backend, honouring the occupancy cap (overflow past a hot home), or `PICK_NONE`. O(1). */
|
|
272
|
-
pick(keyHash
|
|
298
|
+
pick(keyHash: number): number;
|
|
299
|
+
}
|
|
300
|
+
|
|
301
|
+
/**
|
|
302
|
+
* WeightedRandomBalancer -- O(1) weighted-random selection via a Vose/Walker ALIAS TABLE (M10). `pick()`
|
|
303
|
+
* draws one column + one probability compare to return an endpoint proportional to its weight, with
|
|
304
|
+
* REJECTION-SAMPLING eligibility (retry an ineligible candidate up to a bounded count, then a 0-B/op
|
|
305
|
+
* rotated linear eligible scan). The alias table is built COLD over the eligible-INDEPENDENT weights
|
|
306
|
+
* (a weight-0 node is NEVER a column), so rejection renormalizes the weight distribution across the
|
|
307
|
+
* surviving eligible mass. `weights` is the caller-owned Uint32Array; the balancer is the SOLE writer of
|
|
308
|
+
* its derived table via cold `setWeight` / `rebuild` (direct weight mutation desyncs the table -- UB).
|
|
309
|
+
* An eligibility flap NEVER rebuilds. The stateless O(1) sample (no accumulators to desync) for VERY
|
|
310
|
+
* LARGE pools where SmoothWRR's O(cap) scan hurts -- trading smoothness for sampling variance. O(1),
|
|
311
|
+
* 0 B/op, never throws. Fails closed (`PICK_NONE`) IFF `live === 0` OR no eligible node has a positive
|
|
312
|
+
* weight. NOT `@zakkster/lite-random` (a game RNG returning an item; use lite-random for loot tables --
|
|
313
|
+
* this is the eligibility-aware LB index selector; see GUIDE.md / ADR 0012).
|
|
314
|
+
*/
|
|
315
|
+
export class WeightedRandomBalancer extends BalancerBase {
|
|
316
|
+
/**
|
|
317
|
+
* @param capacity endpoint count (fixed).
|
|
318
|
+
* @param eligible shared view: 1 = pickable, 0 = down (length >= capacity).
|
|
319
|
+
* @param weights caller-owned per-endpoint weights (length >= capacity); mutate only via setWeight
|
|
320
|
+
* (the balancer is the sole writer of the derived alias table -- direct mutation is UB).
|
|
321
|
+
* @param seed deterministic PRNG seed (default 0x9e3779b9); reproducible benches.
|
|
322
|
+
*/
|
|
323
|
+
constructor(capacity: number, eligible: Uint8Array, weights: Uint32Array, seed?: number);
|
|
324
|
+
/** Cold path: reconfigure endpoint `i`'s weight (uint32) and rebuild the alias table. */
|
|
325
|
+
setWeight(i: number, w: number): void;
|
|
326
|
+
/** Cold path: rebuild the alias table from the current caller weights (e.g. after a membership change). */
|
|
327
|
+
rebuild(): void;
|
|
328
|
+
/** Pick an endpoint index proportional to weight (eligibility by rejection sampling), or `PICK_NONE`. O(1). */
|
|
329
|
+
pick(): number;
|
|
273
330
|
}
|