rediscache 0.3.2__tar.gz → 1.0.0__tar.gz

{rediscache-0.3.2 → rediscache-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: rediscache
-Version: 0.3.2
+Version: 1.0.0
 Summary: Redis caching of functions evolving over time
 Home-page: https://github.com/AmadeusITGroup/RedisCache
 License: MIT
@@ -40,9 +40,9 @@ There are already quite a few Python decorators to cache functions in a Redis da
 - [redis-simple-cache-py3](https://pypi.org/project/redis-simple-cache-py3/)
 - and more ...
 
-But none I could find allows to set two expiration times as we do it here. The first given time is how long before we should update the value stored in the cache. The second given time, longer of course, is how long the data stored in the cache is still good enough to be sent back to the caller. The refreshing of the cache is only done when the function is called. And by default it is done asynchronously, so the caller doesn't have to wait. When the data in the cache becomes too old, it disapear automatically.
+But none I could find allows to set two expiration times as we do it here. The first given time is how long before we should update the value stored in the cache. The second given time, longer of course, is how long the data stored in the cache is still good enough to be sent back to the caller. The refreshing of the cache is only done when the function is called. And by default it is done asynchronously, so the caller doesn't have to wait. When the data in the cache becomes too old, it disappear automatically.
 
-This is a great caching mechanism for functions that will give a consistent output according to their parameters and at a given time. A purelly random function should not be cached. And a function that is independent of the time should be cached with a different mechanism like the LRU cache in the [functools](https://docs.python.org/3/library/functools.html) standard module.
+This is a great caching mechanism for functions that will give a consistent output according to their parameters and at a given time. A purely random function should not be cached. And a function that is independent of the time should be cached with a different mechanism like the LRU cache in the [functools](https://docs.python.org/3/library/functools.html) standard module.
 
 ## Installation
 
@@ -73,7 +73,7 @@ All the parameters for the `RedisCache` constructor are optional. Their default
 - db: Database number in the Redis server. [`0`]
 - password: Password required to read and write on the Redis server. [`None`]
 - decode: Decode the data stored in the cache as byte string. For example, it should not be done if you actually want to cache byte strings. [`True`]
-- enabled: When False it allows to programatically disable the cache. It can be usefull for unit tests. [`True`]
+- enabled: When False it allows to programmatically disable the cache. It can be useful for unit tests. [`True`]
 
 ### Environment variables
 
@@ -98,7 +98,7 @@ This is the main decorator. All the parameters are available. The mandatory ones
 - serializer: The only type of data that can be stored directly in the Redis database are `byte`, `str`, `int` and `float`. Any other will have to be serialized with the function provided here. [`None`]
 - deserializer: If the value was serialized to be stored in the cache, it needs to deserialized when it is retrieved. [`None`]
 - use_args: This is the list of positional parameters (a list of integers) to be taken into account to generate the key that will be used in Redis. If `None`, they will all be used. [`None`]
-- use_kwargs: This is the list of named paramters (a list of names) to be taken into account to generate the key that will be used in Redis. If `None`, they will all be used. [`None`]
+- use_kwargs: This is the list of named parameters (a list of names) to be taken into account to generate the key that will be used in Redis. If `None`, they will all be used. [`None`]
 
 Example:
 
@@ -115,22 +115,6 @@ See `test_rediscache.py` for more examples.
 
 Note: when you choose to wait for the value, you do not have an absolute guarantee that you will not get the default value. For example if it takes more than the retry time to get an answer from the function, the decorator will give up.
 
-### `cache_raw` decorator helper
-
-No serializer or deserializer. This will only work if the cached function only returns `byte`, `str`, `int` or `float` types. Even `None` will fail.
-
-### `cache_raw_wait` decorator helper
-
-Same as above but waits for the value if not in the cache.
-
-### `cache_json` decorator helper
-
-Serialize the value with `json.dumps()` and desiralize the value with `json.loads()`.
-
-### `cache_json_wait` decorator helper
-
-Same as above but waits for the value if not in the cache.
-
 ### `get_stats(delete=False)`
 
 This will get the stats stored when using the cache. The `delete` option is to reset the counters after read.
@@ -138,6 +122,7 @@ The output is a dictionary with the following keys and values:
 
 - **Refresh**: Number of times the cached function was actually called.
 - **Wait**: Number of times we waited for the result when executing the function.
+- **Sleep**: Number of 1 seconds we waited for the results to be found in the cache.
 - **Failed**: Number of times the cached function raised an exception when called.
 - **Missed**: Number of times the functions result was not found in the cache.
 - **Success**: Number of times the function's result was found in the cache.
@@ -201,7 +186,7 @@ My development environment is handled by Poetry. I use `Python 3.11.7`.
 
 ### Testing
 
-To make sure we use Redis properly, we do not mock it in the unit tess. So you will need a localhost default instance of Redis server without a password. This means that the unit tests are more like integrtion tests.
+To make sure we use Redis properly, we do not mock it in the unit tess. So you will need a localhost default instance of Redis server without a password. This means that the unit tests are more like integration tests.
 
 The execution of the tests including coverage result can be done with `test.sh`. You can also run just `pytest`:
 
@@ -218,7 +203,7 @@ We use the GitHub workflow to check each new commit. See `.github/workflows/pyth
 We get help from re-usable actions. Here is the [Marketplace](https://github.com/marketplace?type=actions).
 
 - [Checkout](https://github.com/marketplace/actions/checkout)
-- [Install Poery Action](https://github.com/marketplace/actions/install-poetry-action)
+- [Install Poetry Action](https://github.com/marketplace/actions/install-poetry-action)
 - [Setup Python](https://github.com/marketplace/actions/setup-python)
 
 ### Publish to PyPI

{rediscache-0.3.2 → rediscache-1.0.0}/README.md

@@ -12,9 +12,9 @@ There are already quite a few Python decorators to cache functions in a Redis da
 - [redis-simple-cache-py3](https://pypi.org/project/redis-simple-cache-py3/)
 - and more ...
 
-But none I could find allows to set two expiration times as we do it here. The first given time is how long before we should update the value stored in the cache. The second given time, longer of course, is how long the data stored in the cache is still good enough to be sent back to the caller. The refreshing of the cache is only done when the function is called. And by default it is done asynchronously, so the caller doesn't have to wait. When the data in the cache becomes too old, it disapear automatically.
+But none I could find allows to set two expiration times as we do it here. The first given time is how long before we should update the value stored in the cache. The second given time, longer of course, is how long the data stored in the cache is still good enough to be sent back to the caller. The refreshing of the cache is only done when the function is called. And by default it is done asynchronously, so the caller doesn't have to wait. When the data in the cache becomes too old, it disappear automatically.
 
-This is a great caching mechanism for functions that will give a consistent output according to their parameters and at a given time. A purelly random function should not be cached. And a function that is independent of the time should be cached with a different mechanism like the LRU cache in the [functools](https://docs.python.org/3/library/functools.html) standard module.
+This is a great caching mechanism for functions that will give a consistent output according to their parameters and at a given time. A purely random function should not be cached. And a function that is independent of the time should be cached with a different mechanism like the LRU cache in the [functools](https://docs.python.org/3/library/functools.html) standard module.
 
 ## Installation
 
@@ -45,7 +45,7 @@ All the parameters for the `RedisCache` constructor are optional. Their default
 - db: Database number in the Redis server. [`0`]
 - password: Password required to read and write on the Redis server. [`None`]
 - decode: Decode the data stored in the cache as byte string. For example, it should not be done if you actually want to cache byte strings. [`True`]
-- enabled: When False it allows to programatically disable the cache. It can be usefull for unit tests. [`True`]
+- enabled: When False it allows to programmatically disable the cache. It can be useful for unit tests. [`True`]
 
 ### Environment variables
 
@@ -70,7 +70,7 @@ This is the main decorator. All the parameters are available. The mandatory ones
 - serializer: The only type of data that can be stored directly in the Redis database are `byte`, `str`, `int` and `float`. Any other will have to be serialized with the function provided here. [`None`]
 - deserializer: If the value was serialized to be stored in the cache, it needs to deserialized when it is retrieved. [`None`]
 - use_args: This is the list of positional parameters (a list of integers) to be taken into account to generate the key that will be used in Redis. If `None`, they will all be used. [`None`]
-- use_kwargs: This is the list of named paramters (a list of names) to be taken into account to generate the key that will be used in Redis. If `None`, they will all be used. [`None`]
+- use_kwargs: This is the list of named parameters (a list of names) to be taken into account to generate the key that will be used in Redis. If `None`, they will all be used. [`None`]
 
 Example:
 
@@ -87,22 +87,6 @@ See `test_rediscache.py` for more examples.
 
 Note: when you choose to wait for the value, you do not have an absolute guarantee that you will not get the default value. For example if it takes more than the retry time to get an answer from the function, the decorator will give up.
 
-### `cache_raw` decorator helper
-
-No serializer or deserializer. This will only work if the cached function only returns `byte`, `str`, `int` or `float` types. Even `None` will fail.
-
-### `cache_raw_wait` decorator helper
-
-Same as above but waits for the value if not in the cache.
-
-### `cache_json` decorator helper
-
-Serialize the value with `json.dumps()` and desiralize the value with `json.loads()`.
-
-### `cache_json_wait` decorator helper
-
-Same as above but waits for the value if not in the cache.
-
 ### `get_stats(delete=False)`
 
 This will get the stats stored when using the cache. The `delete` option is to reset the counters after read.
@@ -110,6 +94,7 @@ The output is a dictionary with the following keys and values:
 
 - **Refresh**: Number of times the cached function was actually called.
 - **Wait**: Number of times we waited for the result when executing the function.
+- **Sleep**: Number of 1 seconds we waited for the results to be found in the cache.
 - **Failed**: Number of times the cached function raised an exception when called.
 - **Missed**: Number of times the functions result was not found in the cache.
 - **Success**: Number of times the function's result was found in the cache.
@@ -173,7 +158,7 @@ My development environment is handled by Poetry. I use `Python 3.11.7`.
 
 ### Testing
 
-To make sure we use Redis properly, we do not mock it in the unit tess. So you will need a localhost default instance of Redis server without a password. This means that the unit tests are more like integrtion tests.
+To make sure we use Redis properly, we do not mock it in the unit tess. So you will need a localhost default instance of Redis server without a password. This means that the unit tests are more like integration tests.
 
 The execution of the tests including coverage result can be done with `test.sh`. You can also run just `pytest`:
 
@@ -190,7 +175,7 @@ We use the GitHub workflow to check each new commit. See `.github/workflows/pyth
 We get help from re-usable actions. Here is the [Marketplace](https://github.com/marketplace?type=actions).
 
 - [Checkout](https://github.com/marketplace/actions/checkout)
-- [Install Poery Action](https://github.com/marketplace/actions/install-poetry-action)
+- [Install Poetry Action](https://github.com/marketplace/actions/install-poetry-action)
 - [Setup Python](https://github.com/marketplace/actions/setup-python)
 
 ### Publish to PyPI

{rediscache-0.3.2 → rediscache-1.0.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "rediscache"
-version = "0.3.2"
+version = "1.0.0"
 description = "Redis caching of functions evolving over time"
 authors = ["Pierre Cart-Grandjean <pcart-grandjean@amadeus.com>"]
 license = "MIT"
@@ -46,3 +46,4 @@ strict = true
 [tool.pylint.format]
 # Maximum number of characters on a single line.
 max-line-length = 160
+max-args = 10

{rediscache-0.3.2 → rediscache-1.0.0}/rediscache/__init__.py

@@ -10,10 +10,10 @@ rediscache is a function decorator that will cache its result in a Redis databas
 
 Parameters:
 - refresh: the number of seconds after which the cached data will be refreshed using the decorated function.
-- expire: the number of seconds after which the cached data will altogether disapear from the Redis database.
+- expire: the number of seconds after which the cached data will altogether disappear from the Redis database.
 - default: the value that will be returned if the data is not found in the cache. It cannot be None.
 It can be bytes, string, int or float.
-- enabled: This is True by default but enables to programmatically disable the cach if required.
+- enabled: This is True by default but enables to programmatically disable the cache if required.
 
 It also reads from the environment:
 - REDIS_SERVICE_HOST: the redis server host name or IP. Default is 'localhost'.
@@ -23,7 +23,7 @@ It also reads from the environment:
 
 Note:
 A key associated to the decorated function is created using the function name and its parameters. This is based
-on the value returned by the repr() function (ie: the __repr__() member) of each paramter. user defined objects
+on the value returned by the repr() function (ie: the __repr__() member) of each parameter. User defined objects
 will have this function return by default a string like this:
 "<amadeusbook.services.EmployeesService object at 0x7f41dedd7128>"
 This will not do as each instance of the object will have a different representation no matter what. The direct
@@ -40,10 +40,10 @@ import logging
 import os
 import threading
 from time import sleep
-from typing import Any, Callable, Dict, List, Optional, ParamSpec, TypeVar
+from typing import Any, Callable, cast, Dict, List, Optional, ParamSpec, TypeVar
 
-import redis
 from executiontime import printexecutiontime, YELLOW, RED
+import redis
 
 PREFIX = "."
 REFRESH = "Refresh"  # Number of times the cached function was actually called.
@@ -56,7 +56,7 @@ DEFAULT = "Default"  # Number of times the default value was used because nothin
 STATS = [REFRESH, WAIT, SLEEP, FAILED, MISSED, SUCCESS, DEFAULT]
 
 P = ParamSpec("P")
-T = TypeVar("T")
+T = TypeVar("T", str, bytes)
 
 
 class RedisCache:
@@ -64,8 +64,6 @@ class RedisCache:
     Having the decorator provided by a class allows to have some context to improve performances.
     """
 
-    # pylint: disable=too-many-arguments
-
     def __init__(
         self,
         host: Optional[str] = None,
@@ -77,7 +75,7 @@ class RedisCache:
     ):
         self.enabled = enabled
         if self.enabled:
-            # If environment variables are set for redis server, they superseed the default values.
+            # If environment variables are set for redis server, they supersede the default values.
             # But if provided at the construction, it has priority.
             if not host:
                 host = os.environ.get("REDIS_SERVICE_HOST", "localhost")
@@ -120,25 +118,18 @@ class RedisCache:
 
         return f"{name}({','.join(values)})"
 
-    # pylint: disable=line-too-long
     def cache(
         self,
         refresh: int,
         expire: int,
+        default: T,
         retry: Optional[int] = None,
-        default: Any = "",
         wait: bool = False,
-        serializer: Optional[Callable[..., Any]] = None,
-        deserializer: Optional[Callable[..., Any]] = None,
         use_args: Optional[List[int]] = None,
         use_kwargs: Optional[List[str]] = None,
     ) -> Callable[[Callable[P, T]], Callable[P, T]]:
         """
-        Full decorator will all possible parameters. Most of the time, you should use a specialzed decorator below.
-
-        Specific examples when to use this decorator:
-        - Raw storage of byte string that you do not want to be decoded: use the decode=False.
-        - JSON dumps data that doesn't need to be loaded before it is sent by a REST API: use serializer=dumps but no deserializer.
+        Full decorator will all possible parameters.
         """
 
         logger = logging.getLogger(__name__)
@@ -164,7 +155,7 @@ class RedisCache:
                     color=RED,
                     output=logger.info,
                 )
-                def refreshvalue(key: str) -> T:
+                def refresh_value(key: str) -> T:
                     """
                     This gets the value provided by the function and stores it in local Redis database
                     """
@@ -188,31 +179,22 @@ class RedisCache:
                         self.server.incr(DEFAULT)
                         new_value = default
 
-                    # Serialize the value if requested
-                    if serializer:
-                        new_value = serializer(new_value)
                     # Store value in cache with expiration time
-                    self.server.set(key, new_value, ex=expire)  # type: ignore
+                    self.server.set(key, new_value, ex=expire)
                     # Set refresh key with refresh time
                     self.server.set(PREFIX + key, 1, ex=refresh)
                     return new_value
 
-                def refreshvalueinthread(key: str) -> None:
+                def refresh_value_in_thread(key: str) -> None:
                     """
                     Run the refresh value in a separate thread
                     """
-                    thread = threading.Thread(target=refreshvalue, args=(key,))
+                    thread = threading.Thread(target=refresh_value, args=(key,))
                     thread.start()
 
                 # If the cache is disabled, directly call the function
                 if not self.enabled:
-                    direct_value = function(*args, **kwargs)
-                    # If we have decided to serialize, we always do it to be consistent
-                    if serializer:
-                        direct_value = serializer(direct_value)
-                    if deserializer:
-                        direct_value = deserializer(direct_value)
-                    return direct_value
+                    return function(*args, **kwargs)
 
                 # Lets create a key from the function's name and its parameters values
                 key = self._create_key(name=function.__name__, args=args, use_args=use_args, kwargs=kwargs, use_kwargs=use_kwargs)
@@ -225,7 +207,7 @@ class RedisCache:
 
                 # Get the value from the cache.
                 # If it is not there we will get None.
-                cached_value = self.server.get(key)
+                cached_value = cast(T, self.server.get(key))
 
                 # Time to update stats counters
                 if cached_value is None:
@@ -239,12 +221,12 @@ class RedisCache:
                     # If we found a value in the cash, we will not wait for the refresh
                     if cached_value or not wait:
                         # We just update the cache in another thread.
-                        refreshvalueinthread(key)
+                        refresh_value_in_thread(key)
                     else:
                         # Here we will wait, let's count it
                         self.server.incr(WAIT)
                         # We update the cash and return the value.
-                        cached_value = refreshvalue(key)
+                        cached_value = refresh_value(key)
 
                 # We may still have decided to wait but another process is already getting the cache updated.
                 if cached_value is None and wait:
@@ -254,60 +236,24 @@ class RedisCache:
                         # Let's count how many times we wait 1s
                         self.server.incr(SLEEP)
                         sleep(1)
-                        cached_value = self.server.get(key)
+                        cached_value = cast(T, self.server.get(key))
 
                 # If the cache was empty, we have None in the cached_value.
                 if cached_value is None:
                     # We are going to return the default value
                     self.server.incr(DEFAULT)
-                    cached_value = serializer(default) if serializer else default
+                    cached_value = default
 
                 # Return whatever value we have at this point.
-                return deserializer(cached_value) if deserializer else cached_value  # type: ignore
+                return cached_value
+
+            # If we want to bypass the cache at runtime, we need a reference to the decorated function
+            wrapper.function = function  # type: ignore
 
             return wrapper
 
         return decorator
 
-    def cache_raw(self, refresh: int, expire: int, retry: Optional[int] = None, default: Any = "") -> Callable[[Callable[P, T]], Callable[P, T]]:
-        """
-        Normal caching of values directly storable in redis: byte string, string, int, float.
-        """
-        return self.cache(refresh=refresh, expire=expire, retry=retry, default=default)
-
-    def cache_raw_wait(self, refresh: int, expire: int, retry: Optional[int] = None, default: Any = "") -> Callable[[Callable[P, T]], Callable[P, T]]:
-        """
-        Same as cache_raw() but will wait for the completion of the cached function if no value is found in redis.
-        """
-        return self.cache(refresh=refresh, expire=expire, retry=retry, default=default, wait=True)
-
-    def cache_json(self, refresh: int, expire: int, retry: Optional[int] = None, default: Any = "") -> Callable[[Callable[P, T]], Callable[P, T]]:
-        """
-        JSON dumps the values to be stored in redis and loads them again when returning them to the caller.
-        """
-        return self.cache(
-            refresh=refresh,
-            expire=expire,
-            retry=retry,
-            default=default,
-            serializer=dumps,
-            deserializer=loads,
-        )
-
-    def cache_json_wait(self, refresh: int, expire: int, retry: Optional[int] = None, default: Any = "") -> Callable[[Callable[P, T]], Callable[P, T]]:
-        """
-        Same as cache_json() but will wait for the completion of the cached function if no value is found in redis.
-        """
-        return self.cache(
-            refresh=refresh,
-            expire=expire,
-            retry=retry,
-            default=default,
-            wait=True,
-            serializer=dumps,
-            deserializer=loads,
-        )
-
     def get_stats(self, delete: bool = False) -> Dict[str, Any]:
         """
         Get the stats stored by RedisCache. See the list and definition at the top of this file.