cachefence 0.1.0__tar.gz

cachefence-0.1.0/.claude/settings.local.json

@@ -0,0 +1,9 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(pip install *)",
+      "Bash(rm -rf dist/ build/ *.egg-info src/*.egg-info)",
+      "Bash(python -m build)"
+    ]
+  }
+}

cachefence-0.1.0/.github/workflows/tests.yml

@@ -0,0 +1,32 @@
+name: tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    services:
+      redis:
+        image: redis:7
+        ports:
+          - 6379:6379
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: pip install -e ".[dev]"
+      - run: ruff check .
+      - run: mypy --strict src/cachefence/
+      - run: pytest -q

cachefence-0.1.0/.gitignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.pytest_cache/
+.venv/
+venv/
+*.egg
+.coverage
+htmlcov/

cachefence-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Bourne
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

cachefence-0.1.0/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.4
+Name: cachefence
+Version: 0.1.0
+Summary: Cache-aside for Redis without the stampede. Probabilistic early refresh + distributed lock, so a hot key expiring never hammers your database.
+Project-URL: Homepage, https://github.com/bourne44/cachefence
+Project-URL: Issues, https://github.com/bourne44/cachefence/issues
+Author: Bourne
+License: MIT
+License-File: LICENSE
+Keywords: async,cache,redis,stampede,thundering-herd,xfetch
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Database
+Classifier: Topic :: System :: Distributed Computing
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: redis>=4.2
+Provides-Extra: dev
+Requires-Dist: fakeredis; extra == 'dev'
+Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: pytest-asyncio; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Requires-Dist: types-redis; extra == 'dev'
+Provides-Extra: test
+Requires-Dist: fakeredis; extra == 'test'
+Requires-Dist: pytest; extra == 'test'
+Requires-Dist: pytest-asyncio; extra == 'test'
+Description-Content-Type: text/markdown
+
+# cachefence
+
+**Cache-aside for Redis without the stampede.**
+
+When a hot cache key expires, naive cache-aside lets *every* concurrent request
+miss at the same instant and pile onto your database to rebuild the same value.
+That's a cache stampede (a.k.a. thundering herd), and it's one of the most common
+ways a cache makes things *worse* under load.
+
+cachefence stops it:
+
+```
+500 concurrent requests hit a cold key (each DB query takes 50ms)
+
+naive cache-aside      DB hits:  500
+with cachefence        DB hits:    1
+```
+
+Same workload, one extra import: **500 database queries become 1.**
+
+## Install
+
+```bash
+pip install cachefence
+```
+
+Requires Python 3.11+ and a Redis server (4.2+).
+
+## Usage
+
+```python
+from redis.asyncio import Redis
+from cachefence import CacheFence
+
+redis = Redis()
+cache = CacheFence(redis)
+
+async def get_user(user_id: int) -> dict:
+    return await cache.get_or_set(
+        key=f"user:{user_id}",
+        ttl=60,                                  # fresh for 60 seconds
+        recompute=lambda: load_user_from_db(user_id),
+    )
+```
+
+`recompute` can be sync or async. It runs at most once per refresh, no matter how
+many requests arrive together. Invalidate manually when the underlying data
+changes:
+
+```python
+await cache.invalidate(f"user:{user_id}")
+```
+
+## How it works
+
+cachefence layers two mechanisms so a key almost never goes cold *and* a cold key
+is never rebuilt more than once:
+
+1. **Probabilistic early refresh (XFetch).** Each read rolls a weighted dice; as
+   the key nears expiry, one lucky request is nudged to refresh it *ahead of
+   time* while everyone else keeps serving the still-valid cached value. The
+   weighting uses how long the last recompute took, so expensive keys refresh
+   earlier. Based on Vattani, Chierichetti & Lowenstein, *"Optimal Probabilistic
+   Cache Stampede Prevention"* (VLDB 2015).
+
+2. **Distributed rebuild lock.** On a true miss, workers race for a short-lived
+   Redis lock. The winner rebuilds; the rest wait briefly and pick up the fresh
+   value the moment it lands, with a bounded fallback so a crashed rebuilder
+   never hangs requests forever.
+
+The lock is released with a compare-and-delete (Lua when the server supports it,
+an optimistic `WATCH`/`MULTI` transaction otherwise) so a worker can never delete
+a lock it no longer owns.
+
+## Configuration
+
+```python
+cache = CacheFence(
+    redis,
+    beta=1.0,          # XFetch aggressiveness; higher = refresh earlier
+    lock_timeout=10.0, # seconds before a rebuild lock auto-expires
+    wait_for_lock=5.0, # max seconds a waiter blocks before rebuilding itself
+    namespace="app:",  # optional key prefix
+)
+```
+
+Custom serialization (default is JSON):
+
+```python
+import pickle
+cache = CacheFence(redis, serializer=pickle.dumps, deserializer=pickle.loads)
+# serializer returns bytes, deserializer takes bytes
+```
+
+## A note on connection pools
+
+Under a genuine burst (hundreds of simultaneous coroutines), the default
+`redis-py` pool can raise `MaxConnectionsError` because waiters don't block for a
+free connection. Use a blocking pool sized for your concurrency:
+
+```python
+from redis.asyncio import BlockingConnectionPool, Redis
+
+pool = BlockingConnectionPool(max_connections=30, timeout=15)
+redis = Redis(connection_pool=pool)
+```
+
+## Run the demo
+
+```bash
+git clone https://github.com/bourne44/cachefence
+cd cachefence
+pip install -e ".[test]"
+python examples/stampede_demo.py
+```
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest
+```
+
+The test suite includes a 100-way concurrent-miss test asserting the recompute
+runs exactly once — the core guarantee of the library.
+
+## License
+
+MIT

cachefence-0.1.0/README.md

@@ -0,0 +1,129 @@
+# cachefence
+
+**Cache-aside for Redis without the stampede.**
+
+When a hot cache key expires, naive cache-aside lets *every* concurrent request
+miss at the same instant and pile onto your database to rebuild the same value.
+That's a cache stampede (a.k.a. thundering herd), and it's one of the most common
+ways a cache makes things *worse* under load.
+
+cachefence stops it:
+
+```
+500 concurrent requests hit a cold key (each DB query takes 50ms)
+
+naive cache-aside      DB hits:  500
+with cachefence        DB hits:    1
+```
+
+Same workload, one extra import: **500 database queries become 1.**
+
+## Install
+
+```bash
+pip install cachefence
+```
+
+Requires Python 3.11+ and a Redis server (4.2+).
+
+## Usage
+
+```python
+from redis.asyncio import Redis
+from cachefence import CacheFence
+
+redis = Redis()
+cache = CacheFence(redis)
+
+async def get_user(user_id: int) -> dict:
+    return await cache.get_or_set(
+        key=f"user:{user_id}",
+        ttl=60,                                  # fresh for 60 seconds
+        recompute=lambda: load_user_from_db(user_id),
+    )
+```
+
+`recompute` can be sync or async. It runs at most once per refresh, no matter how
+many requests arrive together. Invalidate manually when the underlying data
+changes:
+
+```python
+await cache.invalidate(f"user:{user_id}")
+```
+
+## How it works
+
+cachefence layers two mechanisms so a key almost never goes cold *and* a cold key
+is never rebuilt more than once:
+
+1. **Probabilistic early refresh (XFetch).** Each read rolls a weighted dice; as
+   the key nears expiry, one lucky request is nudged to refresh it *ahead of
+   time* while everyone else keeps serving the still-valid cached value. The
+   weighting uses how long the last recompute took, so expensive keys refresh
+   earlier. Based on Vattani, Chierichetti & Lowenstein, *"Optimal Probabilistic
+   Cache Stampede Prevention"* (VLDB 2015).
+
+2. **Distributed rebuild lock.** On a true miss, workers race for a short-lived
+   Redis lock. The winner rebuilds; the rest wait briefly and pick up the fresh
+   value the moment it lands, with a bounded fallback so a crashed rebuilder
+   never hangs requests forever.
+
+The lock is released with a compare-and-delete (Lua when the server supports it,
+an optimistic `WATCH`/`MULTI` transaction otherwise) so a worker can never delete
+a lock it no longer owns.
+
+## Configuration
+
+```python
+cache = CacheFence(
+    redis,
+    beta=1.0,          # XFetch aggressiveness; higher = refresh earlier
+    lock_timeout=10.0, # seconds before a rebuild lock auto-expires
+    wait_for_lock=5.0, # max seconds a waiter blocks before rebuilding itself
+    namespace="app:",  # optional key prefix
+)
+```
+
+Custom serialization (default is JSON):
+
+```python
+import pickle
+cache = CacheFence(redis, serializer=pickle.dumps, deserializer=pickle.loads)
+# serializer returns bytes, deserializer takes bytes
+```
+
+## A note on connection pools
+
+Under a genuine burst (hundreds of simultaneous coroutines), the default
+`redis-py` pool can raise `MaxConnectionsError` because waiters don't block for a
+free connection. Use a blocking pool sized for your concurrency:
+
+```python
+from redis.asyncio import BlockingConnectionPool, Redis
+
+pool = BlockingConnectionPool(max_connections=30, timeout=15)
+redis = Redis(connection_pool=pool)
+```
+
+## Run the demo
+
+```bash
+git clone https://github.com/bourne44/cachefence
+cd cachefence
+pip install -e ".[test]"
+python examples/stampede_demo.py
+```
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest
+```
+
+The test suite includes a 100-way concurrent-miss test asserting the recompute
+runs exactly once — the core guarantee of the library.
+
+## License
+
+MIT

cachefence-0.1.0/examples/stampede_demo.py

@@ -0,0 +1,79 @@
+"""Demo: cache stampede, with and without cachefence.
+
+Simulates a hot key expiring while N requests arrive at once. Counts how many
+times the "database" gets hit to rebuild the value.
+
+Run::
+
+    python examples/stampede_demo.py
+
+Requires a Redis server on localhost:6379.
+"""
+
+import asyncio
+import time
+
+from redis.asyncio import BlockingConnectionPool, Redis
+
+from cachefence import CacheFence
+
+CONCURRENCY = 500
+DB_LATENCY = 0.05  # seconds per "query"
+
+
+async def naive_get_or_set(redis, key, ttl, recompute):
+    """The cache-aside pattern almost everyone writes by hand."""
+    cached = await redis.get(key)
+    if cached is not None:
+        return cached
+    value = await recompute()
+    await redis.set(key, value, ex=ttl)
+    return value
+
+
+async def run(label, getter):
+    pool = BlockingConnectionPool(max_connections=30, timeout=15)
+    redis = Redis(connection_pool=pool, decode_responses=True)
+    await redis.flushall()
+
+    db_hits = {"n": 0}
+
+    async def recompute():
+        db_hits["n"] += 1
+        await asyncio.sleep(DB_LATENCY)
+        return "rebuilt-value"
+
+    start = time.monotonic()
+    await asyncio.gather(*(getter(redis, recompute) for _ in range(CONCURRENCY)))
+    elapsed = time.monotonic() - start
+
+    await redis.aclose()
+    print(f"{label:<22} DB hits: {db_hits['n']:>4}   wall time: {elapsed:.2f}s")
+    return db_hits["n"]
+
+
+async def main():
+    print(f"\n{CONCURRENCY} concurrent requests hit a cold key "
+          f"(each DB query takes {DB_LATENCY*1000:.0f}ms)\n")
+
+    naive = await run(
+        "naive cache-aside",
+        lambda r, rc: naive_get_or_set(r, "hot", 60, rc),
+    )
+
+    cache = None
+
+    async def with_fence(redis, recompute):
+        nonlocal cache
+        if cache is None:
+            cache = CacheFence(redis)
+        return await cache.get_or_set("hot", 60, recompute)
+
+    fenced = await run("with cachefence", with_fence)
+
+    print(f"\ncachefence cut DB load by {(1 - fenced / naive) * 100:.0f}% "
+          f"({naive} -> {fenced} queries)\n")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

cachefence-0.1.0/pyproject.toml

@@ -0,0 +1,51 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "cachefence"
+version = "0.1.0"
+description = "Cache-aside for Redis without the stampede. Probabilistic early refresh + distributed lock, so a hot key expiring never hammers your database."
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "MIT" }
+authors = [{ name = "Bourne" }]
+keywords = ["redis", "cache", "stampede", "thundering-herd", "xfetch", "async"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Database",
+    "Topic :: System :: Distributed Computing",
+    "Typing :: Typed",
+]
+dependencies = ["redis>=4.2"]
+
+[project.optional-dependencies]
+test = ["pytest", "pytest-asyncio", "fakeredis"]
+dev = ["pytest", "pytest-asyncio", "fakeredis", "mypy", "ruff", "types-redis"]
+
+[project.urls]
+Homepage = "https://github.com/bourne44/cachefence"
+Issues = "https://github.com/bourne44/cachefence/issues"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/cachefence"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+testpaths = ["tests"]
+
+[tool.mypy]
+strict = true
+python_version = "3.11"
+
+[tool.ruff]
+target-version = "py311"
+line-length = 90
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]

cachefence-0.1.0/src/cachefence/__init__.py

@@ -0,0 +1,32 @@
+"""cachefence — cache-aside for Redis without the stampede.
+
+When a hot key expires, naive cache-aside lets every concurrent request miss at
+once and hammer your database to rebuild the same value. cachefence prevents that
+with two cooperating mechanisms:
+
+1. Probabilistic early recomputation (XFetch): a single worker is nudged to
+   refresh the value *before* it actually expires, so the key rarely goes cold.
+2. A distributed lock: if the value is gone, exactly one worker rebuilds it while
+   everyone else briefly waits or serves the stale value.
+
+Basic usage::
+
+    from redis.asyncio import Redis
+    from cachefence import CacheFence
+
+    redis = Redis()
+    cache = CacheFence(redis)
+
+    async def get_user(user_id: int) -> dict:
+        return await cache.get_or_set(
+            key=f"user:{user_id}",
+            ttl=60,
+            recompute=lambda: load_user_from_db(user_id),
+        )
+"""
+
+from .cache import CacheFence
+from .errors import CacheFenceError, RecomputeError
+
+__all__ = ["CacheFence", "CacheFenceError", "RecomputeError"]
+__version__ = "0.1.0"

cachefence-0.1.0/src/cachefence/cache.py

@@ -0,0 +1,270 @@
+"""Core CacheFence implementation."""
+
+from __future__ import annotations
+
+import asyncio
+import inspect
+import json
+import math
+import random
+import time
+import uuid
+from collections.abc import Awaitable, Callable
+from dataclasses import dataclass
+from typing import Generic, TypeVar, cast
+
+from redis.asyncio import Redis
+
+from .errors import RecomputeError
+
+T = TypeVar("T")
+Recompute = Callable[[], "Awaitable[T] | T"]
+Serializer = Callable[[object], bytes]
+Deserializer = Callable[[bytes], object]
+
+# Compare-and-delete: release the lock only if we still own it. Prevents a worker
+# whose lock already expired from deleting a lock another worker now holds. Runs as
+# a Lua script by default; if the server rejects scripting at runtime we fall back
+# to a WATCH/MULTI transaction, which gives the same atomic guarantee.
+_RELEASE_LOCK = """
+if redis.call("get", KEYS[1]) == ARGV[1] then
+    return redis.call("del", KEYS[1])
+else
+    return 0
+end
+"""
+
+# Field names inside the cache hash. The value field is the one read on every
+# hit, so it gets a one-byte name; the metadata fields stay spelled out.
+_F_VALUE = b"v"
+_F_DELTA = b"delta"
+_F_EXPIRY = b"expiry"
+
+
+def _default_serializer(value: object) -> bytes:
+    return json.dumps(value).encode()
+
+
+def _default_deserializer(raw: bytes) -> object:
+    return json.loads(raw)
+
+
+@dataclass(frozen=True, slots=True)
+class _Entry:
+    """A value read back from the cache, with the metadata XFetch needs."""
+
+    value: object
+    delta: float   # seconds the last recompute took
+    expiry: float  # absolute unix time at which the value goes stale
+
+
+class CacheFence(Generic[T]):
+    """Cache-aside helper for Redis with built-in stampede protection.
+
+    Parameters
+    ----------
+    redis:
+        A ``redis.asyncio.Redis`` client. It is used in raw-bytes mode
+        internally, so ``decode_responses`` on the client is irrelevant.
+    beta:
+        XFetch aggressiveness. Higher refreshes earlier. ``1.0`` is the value
+        from the original paper and a sensible default.
+    lock_timeout:
+        Seconds a rebuild lock is held before it auto-expires, so a crashed
+        worker cannot block rebuilds forever.
+    wait_for_lock:
+        Maximum seconds a worker waits for another worker's rebuild before
+        rebuilding the value itself.
+    serializer / deserializer:
+        Convert values to/from the ``bytes`` stored in Redis. Defaults to JSON.
+    namespace:
+        Optional prefix applied to every key.
+    """
+
+    __slots__ = (
+        "_redis", "_beta", "_lock_timeout", "_wait_for_lock",
+        "_dumps", "_loads", "_ns", "_release", "_lua_ok",
+    )
+
+    def __init__(
+        self,
+        redis: Redis[bytes],
+        *,
+        beta: float = 1.0,
+        lock_timeout: float = 10.0,
+        wait_for_lock: float = 5.0,
+        serializer: Serializer = _default_serializer,
+        deserializer: Deserializer = _default_deserializer,
+        namespace: str = "",
+    ) -> None:
+        self._redis = redis
+        self._beta = beta
+        self._lock_timeout = lock_timeout
+        self._wait_for_lock = wait_for_lock
+        self._dumps = serializer
+        self._loads = deserializer
+        self._ns = namespace
+        self._release = redis.register_script(_RELEASE_LOCK)
+        self._lua_ok = True  # flips to False if the server rejects scripting
+
+    def _key(self, key: str) -> str:
+        return f"{self._ns}{key}" if self._ns else key
+
+    @staticmethod
+    def _lock_key(rkey: str) -> str:
+        return f"{rkey}:lock"
+
+    async def get_or_set(
+        self,
+        key: str,
+        ttl: float,
+        recompute: Recompute[T],
+        *,
+        beta: float | None = None,
+    ) -> T:
+        """Return the cached value for ``key``, recomputing it if needed.
+
+        ``recompute`` may be sync or async. ``ttl`` is the fresh lifetime in
+        seconds. At most one worker recomputes at a time; the rest serve the
+        still-valid cached value or wait briefly, never stampeding the backing
+        store.
+        """
+        rkey = self._key(key)
+        beta = self._beta if beta is None else beta
+
+        entry = await self._read(rkey)
+        if entry is not None:
+            if not self._should_refresh_early(entry, beta):
+                return cast(T, entry.value)
+            # Near expiry: one worker wins the lock and refreshes ahead of time
+            # while everyone else keeps serving the value that is still valid.
+            token = await self._acquire(rkey)
+            if token is None:
+                return cast(T, entry.value)
+            try:
+                return await self._recompute_and_store(rkey, ttl, recompute)
+            finally:
+                await self._release_lock(rkey, token)
+
+        # Hard miss: the value is gone. Exactly one worker rebuilds it.
+        return await self._rebuild_on_miss(rkey, ttl, recompute)
+
+    async def invalidate(self, key: str) -> None:
+        """Delete a cached key so the next read recomputes it."""
+        await self._redis.delete(self._key(key))
+
+    # --- internals ---------------------------------------------------------
+
+    async def _read(self, rkey: str) -> _Entry | None:
+        data: dict[bytes, bytes] = await self._redis.hgetall(rkey)
+        raw = data.get(_F_VALUE)
+        if raw is None:
+            return None
+        return _Entry(
+            value=self._loads(raw),
+            delta=float(data[_F_DELTA]),
+            expiry=float(data[_F_EXPIRY]),
+        )
+
+    def _should_refresh_early(self, entry: _Entry, beta: float) -> bool:
+        # XFetch (Vattani et al., VLDB 2015): -ln(uniform(0,1]) is exponentially
+        # distributed; scaling it by delta*beta makes expensive-to-rebuild keys
+        # refresh earlier, spreading recomputes out instead of bunching them at
+        # expiry. The gap widens as we approach expiry, so the trigger probability
+        # rises smoothly toward 1.
+        gap = entry.delta * beta * -math.log(random.random() or 1e-12)
+        return time.time() + gap >= entry.expiry
+
+    async def _acquire(self, rkey: str) -> str | None:
+        """Try to take the rebuild lock. Return the ownership token, or None."""
+        token = uuid.uuid4().hex
+        acquired = await self._redis.set(
+            self._lock_key(rkey),
+            token,
+            nx=True,
+            px=int(self._lock_timeout * 1000),
+        )
+        return token if acquired else None
+
+    async def _release_lock(self, rkey: str, token: str) -> None:
+        lock_key = self._lock_key(rkey)
+        if self._lua_ok:
+            try:
+                await self._release(keys=[lock_key], args=[token])
+                return
+            except asyncio.CancelledError:
+                raise
+            except Exception as exc:  # noqa: BLE001
+                message = str(exc).lower()
+                if "evalsha" in message or "unknown command" in message:
+                    self._lua_ok = False  # server lacks scripting; use fallback
+                else:
+                    return  # never fail a request because unlock hiccuped
+        await self._release_lock_fallback(lock_key, token)
+
+    async def _release_lock_fallback(self, lock_key: str, token: str) -> None:
+        """Compare-and-delete via an optimistic WATCH/MULTI transaction."""
+        wanted = token.encode()
+        try:
+            async with self._redis.pipeline(transaction=True) as pipe:
+                await pipe.watch(lock_key)
+                if await pipe.get(lock_key) == wanted:
+                    pipe.multi()
+                    pipe.delete(lock_key)
+                    await pipe.execute()
+                else:
+                    await pipe.reset()
+        except asyncio.CancelledError:
+            raise
+        except Exception:  # noqa: BLE001
+            pass  # the lock's own TTL will clean it up
+
+    async def _recompute_and_store(
+        self, rkey: str, ttl: float, recompute: Recompute[T]
+    ) -> T:
+        start = time.monotonic()
+        try:
+            result = recompute()
+            if inspect.isawaitable(result):
+                result = await result
+        except asyncio.CancelledError:
+            raise
+        except Exception as exc:
+            raise RecomputeError(str(exc)) from exc
+
+        value = cast(T, result)
+        delta = time.monotonic() - start
+        async with self._redis.pipeline(transaction=True) as pipe:
+            pipe.hset(rkey, mapping={
+                _F_VALUE: self._dumps(value),
+                _F_DELTA: delta,
+                _F_EXPIRY: time.time() + ttl,
+            })
+            pipe.pexpire(rkey, int(ttl * 1000))
+            await pipe.execute()
+        return value
+
+    async def _rebuild_on_miss(
+        self, rkey: str, ttl: float, recompute: Recompute[T]
+    ) -> T:
+        token = await self._acquire(rkey)
+        if token is not None:
+            try:
+                return await self._recompute_and_store(rkey, ttl, recompute)
+            finally:
+                await self._release_lock(rkey, token)
+
+        # Another worker holds the lock. Wait for the value to appear, backing
+        # off so we don't busy-poll Redis.
+        deadline = time.monotonic() + self._wait_for_lock
+        delay = 0.02
+        while time.monotonic() < deadline:
+            await asyncio.sleep(delay)
+            entry = await self._read(rkey)
+            if entry is not None:
+                return cast(T, entry.value)
+            delay = min(delay * 1.5, 0.2)
+
+        # The holder crashed or is pathologically slow. Rebuild ourselves rather
+        # than hang the request indefinitely.
+        return await self._recompute_and_store(rkey, ttl, recompute)

cachefence-0.1.0/src/cachefence/errors.py

@@ -0,0 +1,14 @@
+"""Exception types raised by cachefence."""
+
+
+class CacheFenceError(Exception):
+    """Base class for all cachefence errors."""
+
+
+class RecomputeError(CacheFenceError):
+    """Raised when the user-supplied recompute callable fails.
+
+    The error always propagates: cachefence does not fall back to a stale
+    value, even during an early refresh where one is still available. The
+    original exception is available via ``__cause__``.
+    """

cachefence-0.1.0/tests/test_cache.py

@@ -0,0 +1,112 @@
+"""Tests for cachefence.
+
+The headline test is ``test_no_stampede_on_concurrent_miss``: it fires many
+concurrent requests at a cold key and asserts the expensive recompute runs only
+once. That's the entire reason this library exists.
+"""
+
+import asyncio
+
+import fakeredis.aioredis
+import pytest
+
+from cachefence import CacheFence, RecomputeError
+
+pytestmark = pytest.mark.asyncio
+
+
+@pytest.fixture
+async def redis():
+    client = fakeredis.aioredis.FakeRedis()
+    yield client
+    await client.flushall()
+    await client.aclose()
+
+
+async def test_basic_get_or_set_caches(redis):
+    cache = CacheFence(redis)
+    calls = 0
+
+    async def recompute():
+        nonlocal calls
+        calls += 1
+        return {"hello": "world"}
+
+    first = await cache.get_or_set("k", ttl=60, recompute=recompute)
+    second = await cache.get_or_set("k", ttl=60, recompute=recompute)
+
+    assert first == {"hello": "world"}
+    assert second == {"hello": "world"}
+    assert calls == 1  # second read served from cache
+
+
+async def test_sync_recompute_supported(redis):
+    cache = CacheFence(redis)
+    value = await cache.get_or_set("k", ttl=60, recompute=lambda: 42)
+    assert value == 42
+
+
+async def test_no_stampede_on_concurrent_miss(redis):
+    """100 concurrent requests for a cold key -> recompute runs exactly once."""
+    cache = CacheFence(redis)
+    calls = 0
+
+    async def slow_recompute():
+        nonlocal calls
+        calls += 1
+        await asyncio.sleep(0.1)  # simulate a slow DB query
+        return "expensive-value"
+
+    results = await asyncio.gather(
+        *(cache.get_or_set("hot", ttl=60, recompute=slow_recompute) for _ in range(100))
+    )
+
+    assert all(r == "expensive-value" for r in results)
+    assert calls == 1  # the whole point: no stampede
+
+
+async def test_invalidate_forces_recompute(redis):
+    cache = CacheFence(redis)
+    calls = 0
+
+    async def recompute():
+        nonlocal calls
+        calls += 1
+        return calls
+
+    assert await cache.get_or_set("k", ttl=60, recompute=recompute) == 1
+    await cache.invalidate("k")
+    assert await cache.get_or_set("k", ttl=60, recompute=recompute) == 2
+
+
+async def test_recompute_error_wrapped(redis):
+    cache = CacheFence(redis)
+
+    async def boom():
+        raise ValueError("db down")
+
+    with pytest.raises(RecomputeError):
+        await cache.get_or_set("k", ttl=60, recompute=boom)
+
+
+async def test_namespace_applied(redis):
+    cache = CacheFence(redis, namespace="app:")
+    await cache.get_or_set("k", ttl=60, recompute=lambda: 1)
+    assert await redis.exists("app:k")
+    assert not await redis.exists("k")
+
+
+async def test_expiry_then_refresh(redis):
+    cache = CacheFence(redis)
+    calls = 0
+
+    async def recompute():
+        nonlocal calls
+        calls += 1
+        return calls
+
+    await cache.get_or_set("k", ttl=0.1, recompute=recompute)
+    await asyncio.sleep(0.2)  # let it expire
+    result = await cache.get_or_set("k", ttl=0.1, recompute=recompute)
+    assert result == 2
+    assert calls == 2