stickybalancer 1.0.0__tar.gz

stickybalancer-1.0.0/.gitignore

@@ -0,0 +1,6 @@
+__pycache__/
+*.egg-info/
+dist/
+build/
+*.pyc
+.eggs/

stickybalancer-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sayanaditya Das
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

stickybalancer-1.0.0/PKG-INFO

@@ -0,0 +1,178 @@
+Metadata-Version: 2.4
+Name: stickybalancer
+Version: 1.0.0
+Summary: A minimalist load balancing library implementing Strict Po2C and the Sticky-Lazy protocol.
+Project-URL: Homepage, https://github.com/sayanaditya/stickybalancer
+Project-URL: Repository, https://github.com/sayanaditya/stickybalancer
+Author-email: Sayanaditya Das <sayanaditya43@gmail.com>
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+
+# Sticky-Lazy Load Balancing
+
+## Quick Start
+
+```bash
+pip install stickybalancer
+
+from stickybalancer import StickyLazyBalancer
+
+servers = ["server-1", "server-2", "server-3", "server-4"]
+lb = StickyLazyBalancer(servers, threshold=3)
+
+# Route a request
+target = lb.get_next()
+print(f"Sending request to: {target}")
+
+# After the request completes, release the server
+lb.release(target)
+```
+
+## Comparison table
+A probe-efficient alternative to Power-of-Two-Choices (Po2C) that **caches routing decisions** to cut network overhead, while keeping the same logarithmic guarantee on worst-case queue depth.
+
+| Protocol | E[Probes / Request] | Asymptotic Max Queue Depth $M_N$ |
+|---|---|---|
+| Pure Random | 1 | $\dfrac{\ln N}{\ln\ln N}$ |
+| Round Robin | 0 | Flat if tasks are i.i.d., **unbounded variance** if not |
+| Strict Po2C | 2 | $\dfrac{\ln\ln N}{\ln 2} + O(1)$ |
+| **Sticky-Lazy** | $\geq \dfrac{1}{T}$ | $\dfrac{\ln\ln N}{\ln 2} + T$ |
+
+$N$ = number of servers, $T$ = sticky threshold (tunable), $M_N$ = max load over all bins after $N$ requests have landed (the standard "balls into bins" quantity).
+
+---
+
+## 1. The Problem
+
+Po2C is the industry default for stateless load balancing: sample 2 servers, probe both, route to the lighter one. It guarantees max load $O(\ln\ln N)$ — exponentially better than random routing's $O(\ln N / \ln\ln N)$ — but it pays for this **every single request**, with 2 network round-trips just to *decide where to send the 3rd*.
+
+At high QPS, the probe traffic itself becomes a meaningful fraction of your internal network load. 
+
+## 2. The Algorithm
+
+```
+get_next():
+    if cached_server exists and blind_capacity > 0:
+        blind_capacity -= 1                      # FREE — 0 probes
+        return cached_server
+
+    s1 = sample(servers)                          # 1 probe
+    if load[s1] < T:
+        cache(s1); blind_capacity = T - load[s1]
+        return s1
+
+    s2 = sample(servers)                          # 2nd probe (Po2C fallback)
+    winner = argmin(load[s1], load[s2])
+    if load[winner] < T:
+        cache(winner); blind_capacity = T - load[winner]
+    else:
+        invalidate cache
+    return winner
+```
+
+Three phases per cache cycle:
+
+1. **Sticky (caching):** a probe finds a server under threshold $T$ → cache it, set a credit equal to its remaining headroom $(T - \text{load})$.
+2. **Lazy (blind):** every subsequent request drains the credit with zero probe, until the credit hits 0.
+3. **Safety net:** if a probe ever finds a server already at $\ge T$, the cache is invalidated and the request falls back to a strict Po2C double-probe, so the algorithm never blindly piles load onto a hot server.
+
+
+## 3. Proofs
+
+### Po2C tail bound 
+
+Let $x_k$ = fraction of bins with load $\ge k$ after $N$ balls. A bin can only reach load $k$ if, at the moment it was chosen, both sampled bins already had load $\ge k-1$ (otherwise Po2C would have picked the lighter, sub-$(k-1)$ one). Treating choices as independent (folding any dependence into a constant $\lambda$):
+
+$$x_k \le \lambda \cdot x_{k-1}^2, \qquad x_1 \le \lambda$$
+
+Unrolling the recursion:
+
+$$x_k \le \lambda^{1+2+4+\dots+2^{k-1}} = \lambda^{2^k - 1}$$
+
+By a union bound over $N$ bins:
+
+$$\mathbb{P}(M_N \ge k) \le N\cdot x_k \le N\cdot\lambda^{2^k-1}$$
+
+Solving for the crossover point :  We want the smallest $k$ where the RHS drops below 1. Set $N \lambda^{2^k-1}=1$:
+
+$$2^k = 1 + \frac{\ln N}{\ln(1/\lambda)} \implies k = \log_2\left(1+\frac{\ln N}{\ln(1/\lambda)}\right) = \frac{\ln\ln N}{\ln 2} + O(1)$$
+
+since the $1+$ and the $\ln(1/\lambda)$ constant only contribute an additive $O(1)$ term as $N\to\infty$. This recovers the classical $M_N = \dfrac{\ln\ln N}{\ln 2}+O(1)$ result (Azar–Broder–Karlin–Upfal, 1994). 
+
+### $\mathbb{E}[P] = 2$ (for Po2C)
+Each selection process of the server takes 2 random servers from the server list so we need 2 probes per call.
+
+
+
+### Sticky Lazy tail bound
+
+Decompose any bin's load into two parts:
+
+$$\text{load} = \underbrace{\text{load contributed by Po2C-style probe decisions}}_{\text{bounded exactly as in Po2C}} + \underbrace{\text{load added blindly during one cache cycle}}_{\le T-1 \text{ by construction}}$$
+
+The first term is governed by the *same* recursion as plain Po2C.
+
+The second term is the blind cap: by construction the cache **never lets a server accept more than $T-1$ requests beyond the level it was caught at**, because the credit is capped at $(T - \text{load})$ at cache time and load only increases monotonically during the blind run. So the blind phase adds **at most $T$** to whatever the probed component would have been:
+
+$$M_N \le \underbrace{\frac{\ln\ln N}{\ln 2}+O(1)}_{\text{probed part }} + \underbrace{T}_{\text{blind cap}}$$
+
+### $\mathbb{E}[P]$ (for Sticky-Lazy)
+
+**Two cases per probe.** Let $t = \mathbb{P}(\text{probe lands on a server with load} < T)$ — the cache-hit case. The complementary case $(1-t)$ is the safety-net branch.
+
+- **Case 1 ($t$):** $1$ probe; cycle serves $T-L$ requests for $L\in\{0,\dots,T-1\}$. Using the average-of-extremes estimate $L_{\text{avg}} = \frac{T-1}{2}$ (no assumption on which $L$ is likelier, because it will depend on many other factors like server processing speed, query size distribution for each request and others, but for calculation let the midpoint of the known range is the expected value):
+$$\text{cost}_1 = 1 \text{ probe}, \quad \text{served}_1 \approx T - \frac{T-1}{2} = \frac{T+1}{2}$$
+- **Case 2 ($1-t$):** the first probe found $L\ge T$, so a second probe fires (Po2C fallback). Win or lose the cache, exactly $1$ request is served:
+$$\text{cost}_2 = 2 \text{ probes}, \quad \text{served}_2 = 1$$
+
+**Combine.** Average probes and requests served per attempt, then take the ratio:
+
+$$\mathbb{E}[P] \approx \frac{t\cdot 1 + (1-t)\cdot 2}{t\cdot\dfrac{T+1}{2} + (1-t)\cdot 1} = \frac{2-t}{1 + t\cdot\dfrac{T-1}{2}}$$
+
+**Comparison with Po2C.** Po2C is the special case $t=0$ ($\mathbb{E}[P]=2$, always), Sticky-Lazy with any $t>0$ strictly reduces this, since raising $t$ shrinks the numerator and grows the denominator simultaneously. As $T\to\infty$ or $t\to 1$ (low contention, most probes land on safe servers), $\mathbb{E}[P] \to \frac{2}{T+1} \ll 2$. So **Sticky-Lazy never does worse than Po2C**, and the savings grow with both $T$ and the cache-hit rate $t$.
+
+
+
+
+
+### Round Robin's variance is unbounded under heterogeneous tasks
+
+Round Robin assigns request $i$ to server $i \bmod N$, blind to load. If task durations $D_i$ **heavy-tailed** (e.g. $\text{Var}(D)=\infty$, as in many real workloads), a server can be unlucky enough to receive a run of long tasks purely by index coincidence, with no feedback mechanism to redirect future requests away from it. Formally, queue depth becomes a sum of i.i.d. heavy-tailed variables with no rebalancing term, so $\text{Var}(\text{queue depth}) \to \infty$ as task-duration variance grows. But in the case of Po2C and Sticky lazy protocol it actively avoids distribution of load over heavy loaded servers making the variance smaller.
+
+---
+## 4. Pros vs. Cons
+
+### ✅ Pros
+- Cuts probe traffic from a fixed 2 (Po2C) down to roughly $\frac{2}{T+1}$
+- Same safety class as Po2C ($O(\ln\ln N)$), just shifted by a constant $T$.
+- Self-healing: falls back to Po2C automatically the moment caching gets risky.
+- One knob ($T$) to trade off bandwidth vs. balance, tuned per deployment.
+
+### ❌ Cons
+- Assumes uniform request cost , a string of *expensive* requests can overload a cached server before the credit runs out.
+- Cache can go stale if load shifts from traffic outside the balancer's view.
+- Slightly worse balance than Po2C within the threshold zone : a lighter server can sit idle while the sticky one keeps getting hit.
+- Overhead is worst for small $N$, where the $+T$ term outweighs the tiny $\ln\ln N$ baseline.
+
+Most of these are fixable — cost-weighted credits, a TTL on the cache, occasional re-probing, or just shrinking $T$ for small clusters etc.
+
+
+## 5. Simulation Results
+
+To validate the theory against something closer to production traffic, all four protocols were run through a discrete-event simulation: 50 servers, 30k requests, FCFS queues, diurnal traffic with burst windows, log-normal + Pareto-tailed request sizes, and heterogeneous server speeds — averaged over 5 seeded trials per workload (`light`, `medium`, `heavy`). [Full simulation code →](./simulation/simulation.py)
+
+| Probes/Request | Peak Load |
+|---|---|
+| ![probes](./simulation//sim_results/avg_probes.png) | ![peak load](./simulation/sim_results/peak_load.png) |
+
+| Wait Time | Response Time |
+|---|---|
+| ![wait time](./simulation/sim_results/avg_wait_time.png) | ![response time](./simulation/sim_results/avg_response_time.png) |
+
+**Takeaways:**
+- Sticky-Lazy's probe rate stays well under Po2C's fixed 2 across all workloads — confirming the $\ge 1/T$ floor, with the safety net pushing it above the theoretical best case as expected.
+- Peak load and latency for Sticky-Lazy track Po2C closely, both staying flat as Random/Round-Robin degrade sharply under heavier traffic — the predicted $+T$ gap is small and visible, not hidden.

stickybalancer-1.0.0/README.md

@@ -0,0 +1,164 @@
+# Sticky-Lazy Load Balancing
+
+## Quick Start
+
+```bash
+pip install stickybalancer
+
+from stickybalancer import StickyLazyBalancer
+
+servers = ["server-1", "server-2", "server-3", "server-4"]
+lb = StickyLazyBalancer(servers, threshold=3)
+
+# Route a request
+target = lb.get_next()
+print(f"Sending request to: {target}")
+
+# After the request completes, release the server
+lb.release(target)
+```
+
+## Comparison table
+A probe-efficient alternative to Power-of-Two-Choices (Po2C) that **caches routing decisions** to cut network overhead, while keeping the same logarithmic guarantee on worst-case queue depth.
+
+| Protocol | E[Probes / Request] | Asymptotic Max Queue Depth $M_N$ |
+|---|---|---|
+| Pure Random | 1 | $\dfrac{\ln N}{\ln\ln N}$ |
+| Round Robin | 0 | Flat if tasks are i.i.d., **unbounded variance** if not |
+| Strict Po2C | 2 | $\dfrac{\ln\ln N}{\ln 2} + O(1)$ |
+| **Sticky-Lazy** | $\geq \dfrac{1}{T}$ | $\dfrac{\ln\ln N}{\ln 2} + T$ |
+
+$N$ = number of servers, $T$ = sticky threshold (tunable), $M_N$ = max load over all bins after $N$ requests have landed (the standard "balls into bins" quantity).
+
+---
+
+## 1. The Problem
+
+Po2C is the industry default for stateless load balancing: sample 2 servers, probe both, route to the lighter one. It guarantees max load $O(\ln\ln N)$ — exponentially better than random routing's $O(\ln N / \ln\ln N)$ — but it pays for this **every single request**, with 2 network round-trips just to *decide where to send the 3rd*.
+
+At high QPS, the probe traffic itself becomes a meaningful fraction of your internal network load. 
+
+## 2. The Algorithm
+
+```
+get_next():
+    if cached_server exists and blind_capacity > 0:
+        blind_capacity -= 1                      # FREE — 0 probes
+        return cached_server
+
+    s1 = sample(servers)                          # 1 probe
+    if load[s1] < T:
+        cache(s1); blind_capacity = T - load[s1]
+        return s1
+
+    s2 = sample(servers)                          # 2nd probe (Po2C fallback)
+    winner = argmin(load[s1], load[s2])
+    if load[winner] < T:
+        cache(winner); blind_capacity = T - load[winner]
+    else:
+        invalidate cache
+    return winner
+```
+
+Three phases per cache cycle:
+
+1. **Sticky (caching):** a probe finds a server under threshold $T$ → cache it, set a credit equal to its remaining headroom $(T - \text{load})$.
+2. **Lazy (blind):** every subsequent request drains the credit with zero probe, until the credit hits 0.
+3. **Safety net:** if a probe ever finds a server already at $\ge T$, the cache is invalidated and the request falls back to a strict Po2C double-probe, so the algorithm never blindly piles load onto a hot server.
+
+
+## 3. Proofs
+
+### Po2C tail bound 
+
+Let $x_k$ = fraction of bins with load $\ge k$ after $N$ balls. A bin can only reach load $k$ if, at the moment it was chosen, both sampled bins already had load $\ge k-1$ (otherwise Po2C would have picked the lighter, sub-$(k-1)$ one). Treating choices as independent (folding any dependence into a constant $\lambda$):
+
+$$x_k \le \lambda \cdot x_{k-1}^2, \qquad x_1 \le \lambda$$
+
+Unrolling the recursion:
+
+$$x_k \le \lambda^{1+2+4+\dots+2^{k-1}} = \lambda^{2^k - 1}$$
+
+By a union bound over $N$ bins:
+
+$$\mathbb{P}(M_N \ge k) \le N\cdot x_k \le N\cdot\lambda^{2^k-1}$$
+
+Solving for the crossover point :  We want the smallest $k$ where the RHS drops below 1. Set $N \lambda^{2^k-1}=1$:
+
+$$2^k = 1 + \frac{\ln N}{\ln(1/\lambda)} \implies k = \log_2\left(1+\frac{\ln N}{\ln(1/\lambda)}\right) = \frac{\ln\ln N}{\ln 2} + O(1)$$
+
+since the $1+$ and the $\ln(1/\lambda)$ constant only contribute an additive $O(1)$ term as $N\to\infty$. This recovers the classical $M_N = \dfrac{\ln\ln N}{\ln 2}+O(1)$ result (Azar–Broder–Karlin–Upfal, 1994). 
+
+### $\mathbb{E}[P] = 2$ (for Po2C)
+Each selection process of the server takes 2 random servers from the server list so we need 2 probes per call.
+
+
+
+### Sticky Lazy tail bound
+
+Decompose any bin's load into two parts:
+
+$$\text{load} = \underbrace{\text{load contributed by Po2C-style probe decisions}}_{\text{bounded exactly as in Po2C}} + \underbrace{\text{load added blindly during one cache cycle}}_{\le T-1 \text{ by construction}}$$
+
+The first term is governed by the *same* recursion as plain Po2C.
+
+The second term is the blind cap: by construction the cache **never lets a server accept more than $T-1$ requests beyond the level it was caught at**, because the credit is capped at $(T - \text{load})$ at cache time and load only increases monotonically during the blind run. So the blind phase adds **at most $T$** to whatever the probed component would have been:
+
+$$M_N \le \underbrace{\frac{\ln\ln N}{\ln 2}+O(1)}_{\text{probed part }} + \underbrace{T}_{\text{blind cap}}$$
+
+### $\mathbb{E}[P]$ (for Sticky-Lazy)
+
+**Two cases per probe.** Let $t = \mathbb{P}(\text{probe lands on a server with load} < T)$ — the cache-hit case. The complementary case $(1-t)$ is the safety-net branch.
+
+- **Case 1 ($t$):** $1$ probe; cycle serves $T-L$ requests for $L\in\{0,\dots,T-1\}$. Using the average-of-extremes estimate $L_{\text{avg}} = \frac{T-1}{2}$ (no assumption on which $L$ is likelier, because it will depend on many other factors like server processing speed, query size distribution for each request and others, but for calculation let the midpoint of the known range is the expected value):
+$$\text{cost}_1 = 1 \text{ probe}, \quad \text{served}_1 \approx T - \frac{T-1}{2} = \frac{T+1}{2}$$
+- **Case 2 ($1-t$):** the first probe found $L\ge T$, so a second probe fires (Po2C fallback). Win or lose the cache, exactly $1$ request is served:
+$$\text{cost}_2 = 2 \text{ probes}, \quad \text{served}_2 = 1$$
+
+**Combine.** Average probes and requests served per attempt, then take the ratio:
+
+$$\mathbb{E}[P] \approx \frac{t\cdot 1 + (1-t)\cdot 2}{t\cdot\dfrac{T+1}{2} + (1-t)\cdot 1} = \frac{2-t}{1 + t\cdot\dfrac{T-1}{2}}$$
+
+**Comparison with Po2C.** Po2C is the special case $t=0$ ($\mathbb{E}[P]=2$, always), Sticky-Lazy with any $t>0$ strictly reduces this, since raising $t$ shrinks the numerator and grows the denominator simultaneously. As $T\to\infty$ or $t\to 1$ (low contention, most probes land on safe servers), $\mathbb{E}[P] \to \frac{2}{T+1} \ll 2$. So **Sticky-Lazy never does worse than Po2C**, and the savings grow with both $T$ and the cache-hit rate $t$.
+
+
+
+
+
+### Round Robin's variance is unbounded under heterogeneous tasks
+
+Round Robin assigns request $i$ to server $i \bmod N$, blind to load. If task durations $D_i$ **heavy-tailed** (e.g. $\text{Var}(D)=\infty$, as in many real workloads), a server can be unlucky enough to receive a run of long tasks purely by index coincidence, with no feedback mechanism to redirect future requests away from it. Formally, queue depth becomes a sum of i.i.d. heavy-tailed variables with no rebalancing term, so $\text{Var}(\text{queue depth}) \to \infty$ as task-duration variance grows. But in the case of Po2C and Sticky lazy protocol it actively avoids distribution of load over heavy loaded servers making the variance smaller.
+
+---
+## 4. Pros vs. Cons
+
+### ✅ Pros
+- Cuts probe traffic from a fixed 2 (Po2C) down to roughly $\frac{2}{T+1}$
+- Same safety class as Po2C ($O(\ln\ln N)$), just shifted by a constant $T$.
+- Self-healing: falls back to Po2C automatically the moment caching gets risky.
+- One knob ($T$) to trade off bandwidth vs. balance, tuned per deployment.
+
+### ❌ Cons
+- Assumes uniform request cost , a string of *expensive* requests can overload a cached server before the credit runs out.
+- Cache can go stale if load shifts from traffic outside the balancer's view.
+- Slightly worse balance than Po2C within the threshold zone : a lighter server can sit idle while the sticky one keeps getting hit.
+- Overhead is worst for small $N$, where the $+T$ term outweighs the tiny $\ln\ln N$ baseline.
+
+Most of these are fixable — cost-weighted credits, a TTL on the cache, occasional re-probing, or just shrinking $T$ for small clusters etc.
+
+
+## 5. Simulation Results
+
+To validate the theory against something closer to production traffic, all four protocols were run through a discrete-event simulation: 50 servers, 30k requests, FCFS queues, diurnal traffic with burst windows, log-normal + Pareto-tailed request sizes, and heterogeneous server speeds — averaged over 5 seeded trials per workload (`light`, `medium`, `heavy`). [Full simulation code →](./simulation/simulation.py)
+
+| Probes/Request | Peak Load |
+|---|---|
+| ![probes](./simulation//sim_results/avg_probes.png) | ![peak load](./simulation/sim_results/peak_load.png) |
+
+| Wait Time | Response Time |
+|---|---|
+| ![wait time](./simulation/sim_results/avg_wait_time.png) | ![response time](./simulation/sim_results/avg_response_time.png) |
+
+**Takeaways:**
+- Sticky-Lazy's probe rate stays well under Po2C's fixed 2 across all workloads — confirming the $\ge 1/T$ floor, with the safety net pushing it above the theoretical best case as expected.
+- Peak load and latency for Sticky-Lazy track Po2C closely, both staying flat as Random/Round-Robin degrade sharply under heavier traffic — the predicted $+T$ gap is small and visible, not hidden.

stickybalancer-1.0.0/pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "stickybalancer"
+version = "1.0.0"
+description = "A minimalist load balancing library implementing Strict Po2C and the Sticky-Lazy protocol."
+readme = "README.md"
+requires-python = ">=3.8"
+authors = [{ name="Sayanaditya Das", email="sayanaditya43@gmail.com" }]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+]
+
+[project.urls]
+Homepage = "https://github.com/sayanaditya/stickybalancer"
+Repository = "https://github.com/sayanaditya/stickybalancer"

stickybalancer-1.0.0/simulation/simulation.py

@@ -0,0 +1,345 @@
+import os
+import heapq
+import math
+import random
+import statistics
+import time
+from typing import Callable, Dict, List, Tuple
+
+import matplotlib.pyplot as plt
+
+
+# ----------------------------
+# Balancers
+# ----------------------------
+
+class RandomBalancer:
+    def __init__(self, servers):
+        self.servers = list(servers)
+        self.loads = {s: 0 for s in self.servers}
+        self.probes = 0
+
+    def get_next(self):
+        self.probes += 1
+        s = random.choice(self.servers)
+        self.loads[s] += 1
+        return s
+
+    def release(self, server):
+        if server in self.loads and self.loads[server] > 0:
+            self.loads[server] -= 1
+
+
+class RoundRobinBalancer:
+    def __init__(self, servers):
+        self.servers = list(servers)
+        self.loads = {s: 0 for s in self.servers}
+        self.idx = 0
+        self.probes = 0
+
+    def get_next(self):
+        s = self.servers[self.idx]
+        self.idx = (self.idx + 1) % len(self.servers)
+        self.loads[s] += 1
+        return s
+
+    def release(self, server):
+        if server in self.loads and self.loads[server] > 0:
+            self.loads[server] -= 1
+
+
+class Po2CBalancer:
+    def __init__(self, servers):
+        self.servers = list(servers)
+        self.loads = {s: 0 for s in self.servers}
+        self.probes = 0
+
+    def get_next(self):
+        s1, s2 = random.sample(self.servers, 2)
+        self.probes += 2
+        winner = s1 if self.loads[s1] < self.loads[s2] else s2
+        self.loads[winner] += 1
+        return winner
+
+    def release(self, server):
+        if server in self.loads and self.loads[server] > 0:
+            self.loads[server] -= 1
+
+
+class StickyLazyBalancer:
+    def __init__(self, servers, threshold=3):
+        self.servers = list(servers)
+        self.threshold = threshold
+        self.loads = {s: 0 for s in self.servers}
+        self.cached_server = None
+        self.blind_capacity = 0
+        self.probes = 0
+
+    def get_next(self):
+        if self.cached_server is not None and self.blind_capacity > 0:
+            self.blind_capacity -= 1
+            self.loads[self.cached_server] += 1
+            return self.cached_server
+
+        s1 = random.choice(self.servers)
+        self.probes += 1
+
+        if self.loads[s1] < self.threshold:
+            self.loads[s1] += 1
+            self.cached_server = s1
+            self.blind_capacity = self.threshold - self.loads[s1]
+            return s1
+
+        s2 = random.choice([s for s in self.servers if s != s1])
+        self.probes += 1
+
+        winner = s1 if self.loads[s1] < self.loads[s2] else s2
+        self.loads[winner] += 1
+
+        if self.loads[winner] < self.threshold:
+            self.cached_server = winner
+            self.blind_capacity = self.threshold - self.loads[winner]
+        else:
+            self.cached_server = None
+            self.blind_capacity = 0
+
+        return winner
+
+    def release(self, server):
+        if server in self.loads and self.loads[server] > 0:
+            self.loads[server] -= 1
+
+
+# ----------------------------
+# Realistic synthetic workload
+# ----------------------------
+
+def diurnal_rate(t: float, base: float, period: float = 24 * 3600.0) -> float:
+    return base * (1.0 + 0.55 * math.sin(2.0 * math.pi * t / period))
+
+
+def burst_multiplier(t: float, windows: List[Tuple[float, float]]) -> float:
+    for a, b in windows:
+        if a <= t <= b:
+            return 2.5
+    return 1.0
+
+
+def generate_arrivals(
+    num_requests: int,
+    base_rate: float,
+    seed: int,
+) -> List[float]:
+    random.seed(seed)
+    burst_windows = [
+        (2 * 3600.0, 3 * 3600.0),
+        (8 * 3600.0, 9 * 3600.0),
+        (15 * 3600.0, 16 * 3600.0),
+    ]
+
+    arrivals = []
+    t = 0.0
+    for _ in range(num_requests):
+        rate = diurnal_rate(t, base_rate) * burst_multiplier(t, burst_windows)
+        rate = max(rate, 0.05)
+        t += random.expovariate(rate)
+        arrivals.append(t)
+    return arrivals
+
+
+def request_size() -> float:
+    u = random.random()
+    if u < 0.70:
+        return random.lognormvariate(mu=-0.2, sigma=0.35)
+    elif u < 0.95:
+        return random.lognormvariate(mu=0.7, sigma=0.45)
+    else:
+        return 6.0 * random.paretovariate(1.7)
+
+
+def make_servers(n: int, seed: int) -> Dict[int, float]:
+    random.seed(seed)
+    return {i: random.uniform(0.6, 1.8) for i in range(n)}
+
+
+# ----------------------------
+# Simulation
+# ----------------------------
+
+def simulate(
+    balancer_factory: Callable[[List[int]], object],
+    num_servers: int,
+    num_requests: int,
+    base_rate: float,
+    seed: int,
+) -> Dict[str, float]:
+    random.seed(seed)
+
+    servers = list(range(num_servers))
+    server_speed = make_servers(num_servers, seed + 100)
+    balancer = balancer_factory(servers)
+    arrivals = generate_arrivals(num_requests, base_rate, seed + 200)
+
+    # FCFS per server
+    available_at = {s: 0.0 for s in servers}
+    events: List[Tuple[float, int]] = []
+
+    response_times = []
+    wait_times = []
+    sampled_peak = []
+
+    for i, now in enumerate(arrivals):
+        while events and events[0][0] <= now:
+            _, server = heapq.heappop(events)
+            balancer.release(server)
+
+        chosen = balancer.get_next()
+
+        service_demand = request_size()
+        service_time = service_demand / server_speed[chosen]
+
+        start_time = max(now, available_at[chosen])
+        finish_time = start_time + service_time
+        available_at[chosen] = finish_time
+
+        wait_times.append(start_time - now)
+        response_times.append(finish_time - now)
+
+        heapq.heappush(events, (finish_time, chosen))
+
+        if i % 100 == 0:
+            sampled_peak.append(max(balancer.loads.values()))
+
+    return {
+        "avg_wait_time": statistics.mean(wait_times),
+        "avg_response_time": statistics.mean(response_times),
+        "peak_load_mean": statistics.mean(sampled_peak),
+        "avg_probes": getattr(balancer, "probes", 0) / num_requests,
+    }
+
+
+def run_trials(
+    balancer_factory: Callable[[List[int]], object],
+    num_trials: int,
+    num_servers: int,
+    num_requests: int,
+    base_rate: float,
+) -> Dict[str, float]:
+    keys = ["avg_wait_time", "avg_response_time", "peak_load_mean", "avg_probes"]
+    vals = {k: [] for k in keys}
+
+    for t in range(num_trials):
+        r = simulate(
+            balancer_factory=balancer_factory,
+            num_servers=num_servers,
+            num_requests=num_requests,
+            base_rate=base_rate,
+            seed=1000 + t,
+        )
+        for k in keys:
+            vals[k].append(r[k])
+
+    return {k: statistics.mean(v) for k, v in vals.items()}
+
+
+# ----------------------------
+# Plotting
+# ----------------------------
+
+def ensure_dir(path: str) -> None:
+    os.makedirs(path, exist_ok=True)
+
+
+def grouped_bar_chart(results, metric, title, ylabel, outpath):
+    workloads = list(results.keys())
+    algos = list(next(iter(results.values())).keys())
+    x = list(range(len(workloads)))
+    width = 0.18
+
+    plt.figure(figsize=(11, 6))
+    for i, algo in enumerate(algos):
+        values = [results[w][algo][metric] for w in workloads]
+        offset = (i - (len(algos) - 1) / 2) * width
+        plt.bar([xi + offset for xi in x], values, width=width, label=algo)
+
+    plt.xticks(x, workloads)
+    plt.xlabel("Workload")
+    plt.ylabel(ylabel)
+    plt.title(title)
+    plt.legend()
+    plt.tight_layout()
+    plt.savefig(outpath, dpi=220)
+    plt.close()
+
+
+# ----------------------------
+# Main
+# ----------------------------
+
+def main():
+    outdir = "sim_results"
+    ensure_dir(outdir)
+
+    balancers = {
+        "Random": lambda servers: RandomBalancer(servers),
+        "RoundRobin": lambda servers: RoundRobinBalancer(servers),
+        "Po2C": lambda servers: Po2CBalancer(servers),
+        "StickyLazy": lambda servers: StickyLazyBalancer(servers, threshold=3),
+    }
+
+    workloads = {
+        "light": 4.0,
+        "medium": 10.0,
+        "heavy": 18.0,
+    }
+
+    results = {}
+    for wname, base_rate in workloads.items():
+        results[wname] = {}
+        for aname, factory in balancers.items():
+            print(f"Running {aname} on {wname}...")
+            results[wname][aname] = run_trials(
+                balancer_factory=factory,
+                num_trials=5,
+                num_servers=50,
+                num_requests=30_000,
+                base_rate=base_rate,
+            )
+
+    grouped_bar_chart(
+        results,
+        "avg_probes",
+        "Average probes per request",
+        "Probes/request",
+        os.path.join(outdir, "avg_probes.png"),
+    )
+
+    grouped_bar_chart(
+        results,
+        "avg_wait_time",
+        "Average wait time",
+        "Seconds",
+        os.path.join(outdir, "avg_wait_time.png"),
+    )
+
+    grouped_bar_chart(
+        results,
+        "avg_response_time",
+        "Average response time",
+        "Seconds",
+        os.path.join(outdir, "avg_response_time.png"),
+    )
+
+    grouped_bar_chart(
+        results,
+        "peak_load_mean",
+        "Average peak load",
+        "Active requests",
+        os.path.join(outdir, "peak_load.png"),
+    )
+
+    print(f"Saved PNGs to: {outdir}/")
+
+
+if __name__ == "__main__":
+    main()

stickybalancer-1.0.0/src/stickybalancer/__init__.py

@@ -0,0 +1,4 @@
+from .balancer import Po2CBalancer, StickyLazyBalancer
+
+# Optional: Explicitly defines what is exported when someone writes 'from stickybalancer import *'
+__all__ = ["Po2CBalancer", "StickyLazyBalancer"]

stickybalancer-1.0.0/src/stickybalancer/balancer.py

@@ -0,0 +1,71 @@
+import random
+
+class Po2CBalancer:
+    def __init__(self, servers):
+        self.servers = list(servers)
+        self.loads = {server : 0 for server in servers}
+    
+    def get_next(self):
+        """
+            Chooses two random servers and selects the server with smaller load
+        """
+        s1, s2 = random.sample(self.servers, 2)
+        winner = s1 if self.loads[s1] < self.loads[s2] else s2
+        self.loads[winner] += 1
+        return winner
+    
+    def release(self, server):
+        if server in self.loads and self.loads[server] > 0:
+            self.loads[server] -= 1
+
+
+class StickyLazyBalancer:
+    def __init__(self, servers, threshold=3):
+        self.servers = list(servers)
+        self.threshold = threshold
+        self.loads = {server : 0 for server in servers}
+        self.cached_server = None
+        self.blind_capacity = 0
+
+    def get_next(self):
+        """
+            Work Flow :
+                1. Cache Hit : If there is a cached server with blind capacity > 0 then route the request to that server
+                2. Single Probe : Otherwise randomly choose a server and if that server has load < threshold then route the request and cache it
+                3. Dual Probe (Po2C) : Otherwise sample another server and route to the server with lesser load and cache if load < threshold
+        """
+        # If we already have a server in cache
+        if self.cached_server is not None and self.blind_capacity > 0:
+            self.blind_capacity -= 1
+            self.loads[self.cached_server] += 1
+            return self.cached_server
+        
+        # If cache is empty so we choose initial random probe
+        s1 = random.choice(self.servers)
+
+        if self.loads[s1] < self.threshold:
+            self.loads[s1] += 1
+            self.cached_server = s1
+            self.blind_capacity = self.threshold - self.loads[s1]
+            return s1
+        
+        # If s1 has load >= threshold , then Po2C secondary probe selection
+        s2 = random.choice(self.servers)
+
+        winner = s1 if self.loads[s1] < self.loads[s2] else s2
+        self.loads[winner] += 1
+
+        if self.loads[winner] < self.threshold:
+            self.cached_server = winner
+            self.blind_capacity = self.threshold - self.loads[winner]
+        else :
+            self.cached_server = None
+            self.blind_capacity = 0
+
+        return winner
+    
+    def release(self, server):
+        if server in self.loads and self.loads[server] > 0:
+            self.loads[server] -= 1
+        
+