fastapi-cachex 0.3.2__tar.gz → 0.3.4__tar.gz

{fastapi_cachex-0.3.2 → fastapi_cachex-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastapi-cachex
-Version: 0.3.2
+Version: 0.3.4
 Summary: A caching library for FastAPI with support for Cache-Control, ETag, and multiple backends.
 Keywords: fastapi,cache,etag,cache-control,redis,memcached,in-memory
 Author: allen0099
@@ -181,7 +181,7 @@ cache and OAuth state, so `clear()`/`clear_prefix()` never touch unrelated cache
 entries.
 
 **Note**: `clear()`/`clear_prefix()` are implemented via the backend's
-`get_all_keys()`. Since Memcached doesn't support key enumeration (see
+`get_all_keys()` and `delete_many()` (one batched `DEL` on Redis). Since Memcached doesn't support key enumeration (see
 [Memcached limitations](#memcached)), these two methods are no-ops on a
 Memcached backend — `get()`/`set()`/`delete()`/`has()` work normally. Use
 Redis or the in-memory backend if you need bulk clearing.
@@ -217,6 +217,41 @@ When a cached entry is valid (within TTL):
 
 This means **cached hits are extremely fast** - the endpoint handler function is never executed.
 
+### Atomic backend primitives
+
+Every backend exposes two atomic operations on top of `get`/`set`/`delete`, for
+values that are read and written by many concurrent requests:
+
+```python
+from fastapi_cachex import BackendProxy
+
+backend = BackendProxy.get()
+
+# Fixed-window counter: created on first use, `ttl` applies only then.
+hits = await backend.increment(f"resend:{user_id}", ttl=86400)
+if hits > 3:
+    raise TooManyRequests()
+
+# One-shot value: of several concurrent callers exactly one gets the entry.
+grant = await backend.get_and_delete(f"grant:{token}")
+```
+
+- `increment(key, delta=1, ttl=None) -> int` — Memory does the read-modify-write
+  under its lock, Redis runs a Lua script (`EXISTS` + `INCRBY` + `EXPIRE`) and
+  Memcached uses `ADD` + `INCR`/`DECR` (Memcached counters stop at 0). The
+  counter is visible through `get()` as a `CacheEntry` with fingerprint
+  `COUNTER_FINGERPRINT` and the decimal value as content, so `delete`/`clear*`
+  and the monitoring routes treat it like any other entry. Incrementing a key
+  that holds a cached response raises `CacheXError`.
+- `get_and_delete(key) -> CacheEntry | None` — Memory pops under its lock, Redis
+  uses `GETDEL` (server 6.2+) and Memcached returns the value only when its own
+  `DELETE` won. `StateManager.consume_state`, `CacheManager.delete` and
+  `invalidate()` are built on it.
+
+Both have a non-atomic fallback on `BaseCacheBackend`, so a third-party backend
+that only implements the abstract methods keeps working; override them to get
+real atomicity.
+
 ### In-Memory Cache (default)
 
 If you don't specify a backend, FastAPI-CacheX will use the in-memory cache by default.
@@ -249,6 +284,11 @@ BackendProxy.set(backend)
 - Keys are namespaced with `fastapi_cachex:` prefix to avoid conflicts
 - Consider using Redis backend if you need pattern-based cache clearing
 
+The synchronous pymemcache client runs in worker threads and is connection-pooled,
+so concurrent requests never share a socket. Writes wait for the server's
+acknowledgement (`default_noreply=False`), which keeps a value readable from
+any pooled connection as soon as `set()` returns.
+
 ### Redis
 
 ```python

{fastapi_cachex-0.3.2 → fastapi_cachex-0.3.4}/README.md

@@ -143,7 +143,7 @@ cache and OAuth state, so `clear()`/`clear_prefix()` never touch unrelated cache
 entries.
 
 **Note**: `clear()`/`clear_prefix()` are implemented via the backend's
-`get_all_keys()`. Since Memcached doesn't support key enumeration (see
+`get_all_keys()` and `delete_many()` (one batched `DEL` on Redis). Since Memcached doesn't support key enumeration (see
 [Memcached limitations](#memcached)), these two methods are no-ops on a
 Memcached backend — `get()`/`set()`/`delete()`/`has()` work normally. Use
 Redis or the in-memory backend if you need bulk clearing.
@@ -179,6 +179,41 @@ When a cached entry is valid (within TTL):
 
 This means **cached hits are extremely fast** - the endpoint handler function is never executed.
 
+### Atomic backend primitives
+
+Every backend exposes two atomic operations on top of `get`/`set`/`delete`, for
+values that are read and written by many concurrent requests:
+
+```python
+from fastapi_cachex import BackendProxy
+
+backend = BackendProxy.get()
+
+# Fixed-window counter: created on first use, `ttl` applies only then.
+hits = await backend.increment(f"resend:{user_id}", ttl=86400)
+if hits > 3:
+    raise TooManyRequests()
+
+# One-shot value: of several concurrent callers exactly one gets the entry.
+grant = await backend.get_and_delete(f"grant:{token}")
+```
+
+- `increment(key, delta=1, ttl=None) -> int` — Memory does the read-modify-write
+  under its lock, Redis runs a Lua script (`EXISTS` + `INCRBY` + `EXPIRE`) and
+  Memcached uses `ADD` + `INCR`/`DECR` (Memcached counters stop at 0). The
+  counter is visible through `get()` as a `CacheEntry` with fingerprint
+  `COUNTER_FINGERPRINT` and the decimal value as content, so `delete`/`clear*`
+  and the monitoring routes treat it like any other entry. Incrementing a key
+  that holds a cached response raises `CacheXError`.
+- `get_and_delete(key) -> CacheEntry | None` — Memory pops under its lock, Redis
+  uses `GETDEL` (server 6.2+) and Memcached returns the value only when its own
+  `DELETE` won. `StateManager.consume_state`, `CacheManager.delete` and
+  `invalidate()` are built on it.
+
+Both have a non-atomic fallback on `BaseCacheBackend`, so a third-party backend
+that only implements the abstract methods keeps working; override them to get
+real atomicity.
+
 ### In-Memory Cache (default)
 
 If you don't specify a backend, FastAPI-CacheX will use the in-memory cache by default.
@@ -211,6 +246,11 @@ BackendProxy.set(backend)
 - Keys are namespaced with `fastapi_cachex:` prefix to avoid conflicts
 - Consider using Redis backend if you need pattern-based cache clearing
 
+The synchronous pymemcache client runs in worker threads and is connection-pooled,
+so concurrent requests never share a socket. Writes wait for the server's
+acknowledgement (`default_noreply=False`), which keeps a value readable from
+any pooled connection as soon as `set()` returns.
+
 ### Redis
 
 ```python

fastapi_cachex-0.3.4/fastapi_cachex/backends/base.py

@@ -0,0 +1,137 @@
+"""Base cache backend interface and abstract implementation."""
+
+from abc import ABC
+from abc import abstractmethod
+from collections.abc import Iterable
+from typing import Any
+
+from fastapi_cachex.types import CacheEntry
+from fastapi_cachex.types import counter_entry
+from fastapi_cachex.types import counter_value
+
+
+class BaseCacheBackend(ABC):
+    """Base class for all cache backends."""
+
+    @abstractmethod
+    async def get(self, key: str) -> CacheEntry | None:
+        """Retrieve a cached response."""
+
+    @abstractmethod
+    async def set(self, key: str, value: CacheEntry, ttl: int | None = None) -> None:
+        """Store a response in the cache."""
+
+    @abstractmethod
+    async def delete(self, key: str) -> None:
+        """Remove a response from the cache."""
+
+    async def delete_many(self, keys: Iterable[str]) -> int:
+        """Remove every key in ``keys``; returns how many were removed.
+
+        The base implementation deletes one key at a time and reports how
+        many were attempted, since ``delete`` does not say whether the key
+        existed. The built-in backends override it with a single batched
+        operation that counts what was actually removed.
+        """
+        count = 0
+        for key in keys:
+            await self.delete(key)
+            count += 1
+        return count
+
+    async def get_and_delete(self, key: str) -> CacheEntry | None:
+        """Atomically retrieve and remove a cached entry.
+
+        Use this for one-shot values (OAuth states, grants, invalidation) where
+        exactly one of several concurrent callers may win: every other caller
+        sees ``None``.
+
+        The base implementation is a best-effort, NON-atomic get-then-delete
+        fallback for third-party backends; the built-in backends override it
+        with an atomic implementation.
+
+        Returns:
+            The entry that was stored under ``key``, or ``None`` if there was none
+        """
+        value = await self.get(key)
+        if value is not None:
+            await self.delete(key)
+        return value
+
+    async def increment(self, key: str, delta: int = 1, ttl: int | None = None) -> int:
+        """Atomically add ``delta`` to the integer counter stored at ``key``.
+
+        A missing key counts as 0: the first call creates the counter with the
+        value ``delta`` and applies ``ttl`` (seconds; ``None`` = never expires).
+        Later calls keep the existing expiry, so the counter lives in a fixed
+        window that starts when it is created - the shape rate limiters need.
+        The counter is readable through ``get()`` as a ``CacheEntry`` whose
+        fingerprint is ``COUNTER_FINGERPRINT`` and whose content is the decimal
+        value; ``delete``/``clear*`` treat it like any other entry.
+
+        The base implementation is a best-effort, NON-atomic read-modify-write
+        fallback for third-party backends and re-applies ``ttl`` on every call.
+        The built-in backends override it with a single server-side operation.
+
+        Args:
+            key: Cache key of the counter
+            delta: Amount to add (may be negative)
+            ttl: Time to live in seconds, applied when the counter is created
+
+        Returns:
+            The counter value after the increment
+
+        Raises:
+            CacheXError: If ``key`` holds a cached response instead of a counter
+        """
+        current = await self.get(key)
+        value = delta if current is None else counter_value(current) + delta
+        await self.set(key, counter_entry(value), ttl=ttl)
+        return value
+
+    @abstractmethod
+    async def clear(self) -> None:
+        """Clear all cached responses."""
+
+    @abstractmethod
+    async def clear_path(self, path: str, include_params: bool = False) -> int:
+        """Clear cached responses for a specific path.
+
+        Args:
+            path: The path to clear cache for
+            include_params: Whether to clear all parameter variations of the path
+
+        Returns:
+            Number of cache entries cleared
+        """
+
+    @abstractmethod
+    async def clear_pattern(self, pattern: str) -> int:
+        """Clear cached responses matching a pattern.
+
+        Args:
+            pattern: A glob pattern to match cache keys against (e.g., "/users/*")
+
+        Returns:
+            Number of cache entries cleared
+        """
+
+    @abstractmethod
+    async def get_all_keys(self) -> list[str]:
+        """Get all cache keys in the backend.
+
+        Returns:
+            List of all cache keys currently stored in the backend
+        """
+
+    @abstractmethod
+    async def get_cache_data(self) -> dict[str, tuple[Any, float | None]]:
+        """Get all cache data with expiry information.
+
+        This method is primarily used for cache monitoring and statistics.
+        Returns cache keys mapped to tuples of (value, expiry_time).
+
+        Returns:
+            Dictionary mapping cache keys to (value, expiry) tuples.
+            Expiry is None if the item never expires.
+        """

fastapi_cachex-0.3.4/fastapi_cachex/backends/codec.py

@@ -0,0 +1,69 @@
+"""Serialization shared by the network backends (Redis, Memcached).
+
+Both backends store a ``CacheEntry`` as a JSON document; ``orjson`` is used when
+it is installed and the standard library ``json`` module otherwise.
+"""
+
+from fastapi_cachex.types import CacheEntry
+from fastapi_cachex.types import counter_entry
+
+try:
+    import orjson as json
+
+except ImportError:  # pragma: no cover
+    import json  # type: ignore[no-redef]  # pragma: no cover
+
+# ``json.loads`` (either implementation) raises ``ValueError`` subclasses for bad
+# JSON; ``KeyError``/``TypeError``/``AttributeError`` cover documents whose shape
+# is not the one ``encode_entry`` writes (missing fields, non-string content).
+_DECODE_ERRORS = (ValueError, KeyError, TypeError, AttributeError)
+
+
+def encode_entry(entry: CacheEntry) -> bytes:
+    """Serialize a ``CacheEntry`` to a UTF-8 JSON document.
+
+    The raw content bytes are passed through ``latin-1`` so that arbitrary
+    bytes round-trip through JSON text.
+    """
+    serialized: str | bytes = json.dumps(
+        {
+            "fingerprint": entry.fingerprint,
+            "content": entry.content.decode("latin-1"),
+            "media_type": entry.media_type,
+        },
+    )
+    # orjson returns bytes, stdlib json returns str
+    return serialized if isinstance(serialized, bytes) else serialized.encode("utf-8")
+
+
+def _as_counter(raw: str | bytes) -> int | None:
+    """The integer a bare counter value holds, or ``None`` for anything else."""
+    try:
+        return int(raw)
+    except ValueError:
+        return None
+
+
+def decode_entry(raw: str | bytes | None) -> CacheEntry | None:
+    """Rebuild a ``CacheEntry`` from a stored value.
+
+    A bare integer (what the server-side ``INCR`` family leaves behind for
+    ``increment``) becomes a counter entry. Anything else that is not a document
+    written by ``encode_entry`` (corrupt JSON, missing fields, non-string
+    content) yields ``None``, so callers can treat every malformed value as a
+    cache miss.
+    """
+    if raw is None:
+        return None
+    counter = _as_counter(raw)
+    if counter is not None:
+        return counter_entry(counter)
+    try:
+        data = json.loads(raw)
+        return CacheEntry(
+            fingerprint=data["fingerprint"],
+            content=data["content"].encode("latin-1"),
+            media_type=data.get("media_type"),
+        )
+    except _DECODE_ERRORS:
+        return None

{fastapi_cachex-0.3.2 → fastapi_cachex-0.3.4}/fastapi_cachex/backends/memcached.py

@@ -4,17 +4,13 @@ import asyncio
 import logging
 import warnings
 
+from fastapi_cachex.backends.codec import decode_entry
+from fastapi_cachex.backends.codec import encode_entry
 from fastapi_cachex.exceptions import CacheXError
 from fastapi_cachex.types import CacheEntry
 
 from .base import BaseCacheBackend
 
-try:
-    import orjson as json
-
-except ImportError:  # pragma: no cover
-    import json  # type: ignore[no-redef]  # pragma: no cover
-
 logger = logging.getLogger(__name__)
 
 # Default Memcached key prefix for fastapi-cachex
@@ -24,10 +20,11 @@ DEFAULT_MEMCACHE_PREFIX = "fastapi_cachex:"
 class MemcachedBackend(BaseCacheBackend):
     """Memcached backend implementation.
 
-    Note: This implementation uses synchronous pymemcache client but wraps it
-    in async methods. For blocking concerns, consider using aiomcache for
-    true async Memcached operations. Keys are namespaced with 'fastapi_cachex:'
-    by default to avoid conflicts with other applications.
+    Note: This implementation uses the synchronous pymemcache client and runs
+    each call in a worker thread. The client is connection-pooled so concurrent
+    calls never share a socket. For true async Memcached operations consider
+    aiomcache. Keys are namespaced with 'fastapi_cachex:' by default to avoid
+    conflicts with other applications.
 
     Limitations:
     - Pattern-based clearing (clear_pattern) is not supported by Memcached protocol
@@ -56,7 +53,17 @@ class MemcachedBackend(BaseCacheBackend):
             msg = "pymemcache is not installed. Please install it with 'pip install pymemcache'"
             raise CacheXError(msg)
 
-        self.client = HashClient(servers, connect_timeout=5, timeout=5)
+        # Pooled connections have no ordering guarantee between each other, so
+        # every write waits for the server's acknowledgement; otherwise a
+        # ``set`` on one socket may still be in flight when a ``get`` on another
+        # socket is served, and the caller would miss its own write.
+        self.client = HashClient(
+            servers,
+            connect_timeout=5,
+            timeout=5,
+            use_pooling=True,
+            default_noreply=False,
+        )
         self.key_prefix = key_prefix
 
     def _make_key(self, key: str) -> str:
@@ -72,24 +79,15 @@ class MemcachedBackend(BaseCacheBackend):
         Returns:
             Cached entry if found, None otherwise
         """
-        prefixed_key = self._make_key(key)
-        value = await asyncio.to_thread(self.client.get, prefixed_key)
-        if value is None:
+        raw = await asyncio.to_thread(self.client.get, self._make_key(key))
+        value = decode_entry(raw)
+        if raw is None:
             logger.debug("Memcached MISS; key=%s", key)
-            return None
-
-        # Memcached stores data as bytes; deserialize from JSON
-        try:
-            data = json.loads(value)
-            logger.debug("Memcached HIT; key=%s", key)
-            return CacheEntry(
-                fingerprint=data["fingerprint"],
-                content=data["content"].encode("latin-1"),
-                media_type=data.get("media_type"),
-            )
-        except (json.JSONDecodeError, KeyError, ValueError):
+        elif value is None:
             logger.debug("Memcached DESERIALIZE ERROR; key=%s", key)
-            return None
+        else:
+            logger.debug("Memcached HIT; key=%s", key)
+        return value
 
     async def set(self, key: str, value: CacheEntry, ttl: int | None = None) -> None:
         """Set value in cache.
@@ -99,29 +97,67 @@ class MemcachedBackend(BaseCacheBackend):
             value: CacheEntry instance to store
             ttl: Time to live in seconds
         """
-        prefixed_key = self._make_key(key)
+        expire = ttl if ttl is not None else 0
+        await asyncio.to_thread(
+            self.client.set, self._make_key(key), encode_entry(value), expire
+        )
+        logger.debug("Memcached SET; key=%s ttl=%s", key, ttl)
 
-        # Use latin-1 to round-trip arbitrary bytes through JSON storage
-        content = value.content.decode("latin-1")
+    async def get_and_delete(self, key: str) -> CacheEntry | None:
+        """Atomically retrieve and remove a cached entry (see base class).
 
-        serialized_data: str | bytes = json.dumps(
-            {
-                "fingerprint": value.fingerprint,
-                "content": content,
-                "media_type": value.media_type,
-            },
+        Memcached has no combined primitive, but DELETE is atomic: the value is
+        returned only when this call is the one that removed it, so exactly one
+        concurrent caller wins.
+        """
+        prefixed_key = self._make_key(key)
+        raw = await asyncio.to_thread(self.client.get, prefixed_key)
+        if raw is None:
+            logger.debug("Memcached GET_AND_DELETE MISS; key=%s", key)
+            return None
+        deleted = await asyncio.to_thread(
+            self.client.delete, prefixed_key, noreply=False
         )
+        if not deleted:
+            logger.debug("Memcached GET_AND_DELETE LOST RACE; key=%s", key)
+            return None
+        logger.debug("Memcached GET_AND_DELETE HIT; key=%s", key)
+        return decode_entry(raw)
 
-        # orjson returns bytes, stdlib json returns str
-        serialized_bytes = (
-            serialized_data
-            if isinstance(serialized_data, bytes)
-            else serialized_data.encode("utf-8")
-        )
+    def _add_delta(self, prefixed_key: str, delta: int) -> int | None:
+        """Apply ``delta`` with INCR/DECR; ``None`` when the key does not exist."""
+        if delta < 0:
+            result = self.client.decr(prefixed_key, -delta, noreply=False)
+        else:
+            result = self.client.incr(prefixed_key, delta, noreply=False)
+        return None if result is None else int(result)
 
-        expire = ttl if ttl is not None else 0
-        await asyncio.to_thread(self.client.set, prefixed_key, serialized_bytes, expire)
-        logger.debug("Memcached SET; key=%s ttl=%s", key, ttl)
+    async def increment(self, key: str, delta: int = 1, ttl: int | None = None) -> int:
+        """Atomically add ``delta`` to the counter at ``key`` (see base class).
+
+        Memcached counters are unsigned, so a negative ``delta`` uses DECR,
+        which stops at 0 instead of going negative.
+        """
+        from pymemcache.exceptions import MemcacheClientError
+
+        prefixed_key = self._make_key(key)
+        try:
+            value = await asyncio.to_thread(self._add_delta, prefixed_key, delta)
+            if value is None:
+                # No counter yet: ADD is atomic and a no-op when a concurrent
+                # call created it first, so the retry always finds a counter.
+                await asyncio.to_thread(
+                    self.client.add, prefixed_key, b"0", ttl or 0, noreply=False
+                )
+                value = await asyncio.to_thread(self._add_delta, prefixed_key, delta)
+        except MemcacheClientError as e:
+            msg = "Cache key holds a value that is not a counter"
+            raise CacheXError(msg) from e
+        if value is None:  # pragma: no cover - the counter expired mid-call
+            msg = "Counter vanished between ADD and INCR"
+            raise CacheXError(msg)
+        logger.debug("Memcached INCREMENT; key=%s value=%s ttl=%s", key, value, ttl)
+        return value
 
     async def delete(self, key: str) -> None:
         """Delete value from cache.