redis-dict 2.5.1__py3-none-any.whl → 2.6.0__py3-none-any.whl

Sign up to get free protection for your applications and to get access to all the features.
@@ -1,15 +1,24 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: redis-dict
3
- Version: 2.5.1
3
+ Version: 2.6.0
4
4
  Summary: Dictionary with Redis as storage backend
5
5
  Home-page: https://github.com/Attumm/redisdict
6
6
  Author: Melvin Bijman
7
7
  Author-email: bijman.m.m@gmail.com
8
8
  License: MIT
9
+ Keywords: redis python dictionary dict key-value key:value database caching distributed-computing dictionary-interface large-datasets scientific-computing data-persistence high-performance scalable pipelining batching big-data data-types distributed-algorithms encryption data-management
10
+ Platform: any
9
11
  Classifier: Development Status :: 5 - Production/Stable
10
12
  Classifier: Intended Audience :: Developers
13
+ Classifier: Intended Audience :: Information Technology
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Topic :: Internet
16
+ Classifier: Topic :: Scientific/Engineering
11
17
  Classifier: Topic :: Database
12
18
  Classifier: Topic :: System :: Distributed Computing
19
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
20
+ Classifier: Topic :: Software Development :: Object Brokering
21
+ Classifier: Topic :: Database :: Database Engines/Servers
13
22
  Classifier: License :: OSI Approved :: MIT License
14
23
  Classifier: Programming Language :: Python :: 3
15
24
  Classifier: Programming Language :: Python :: 3.6
@@ -28,10 +37,9 @@ Requires-Dist: redis
28
37
  [![codecov](https://codecov.io/gh/Attumm/redis-dict/graph/badge.svg?token=Lqs7McQGEs)](https://codecov.io/gh/Attumm/redis-dict)
29
38
  [![Downloads](https://static.pepy.tech/badge/redis-dict/month)](https://pepy.tech/project/redis-dict)
30
39
 
31
- RedisDict is a Python library that provides a convenient and familiar interface for interacting with Redis as if it were a Python dictionary. This simple yet powerful library enables you to manage key-value pairs in Redis using native Python syntax. It supports various data types, including strings, integers, floats, booleans, lists, and dictionaries, and includes additional utility functions for more complex use cases.
32
-
33
- By leveraging Redis for efficient key-value storage, RedisDict allows for high-performance data management and is particularly useful for handling large datasets that may exceed local memory capacity.
40
+ RedisDict is a Python library that offers a convenient and familiar interface for interacting with Redis, treating it as if it were a Python dictionary. Its goal is to help developers write clean, Pythonic code while using Redis as a storage solution for seamless distributed computing. This simple yet powerful library utilizes Redis as a key-value store and supports various data types, including strings, integers, floats, booleans, lists, and dictionaries. Additionally, developers can extend RedisDict to work with custom objects.
34
41
 
42
+ The library includes utility functions for more complex use cases such as caching, batching, and more. By leveraging Redis for efficient key-value storage, RedisDict enables high-performance data management, maintaining efficiency even with large datasets and Redis instances.
35
43
 
36
44
  ## Features
37
45
 
@@ -42,20 +50,27 @@ By leveraging Redis for efficient key-value storage, RedisDict allows for high-p
42
50
  * Efficiency and Scalability: RedisDict is designed for use with large datasets and is optimized for efficiency. It retrieves only the data needed for a particular operation, ensuring efficient memory usage and fast performance.
43
51
  * Namespace Management: Provides simple and efficient namespace handling to help organize and manage data in Redis, streamlining data access and manipulation.
44
52
  * Distributed Computing: With its ability to seamlessly connect to other instances or servers with access to the same Redis instance, RedisDict enables easy distributed computing.
45
- * Custom data types: Add custom types and transformations to suit your specific needs.
53
+ * Custom data: types: Add custom types encoding/decoding to store your data types.
54
+ * Encryption: allows for storing data encrypted, while retaining the simple dictionary interface.
46
55
 
47
56
  ## Example
48
57
  Redis is an exceptionally fast database when used appropriately. RedisDict leverages Redis for efficient key-value storage, enabling high-performance data management.
49
58
 
50
- ```python
51
- from redis_dict import RedisDict
59
+ ```bash
60
+ pip install redis-dict
61
+ ```
52
62
 
53
- dic = RedisDict()
54
- dic['foo'] = 42
55
- print(dic['foo']) # Output: 42
56
- print('foo' in dic) # Output: True
57
- dic["baz"] = "hello world"
58
- print(dic) # Output: {'foo': 42, 'baz': 'hello world'}
63
+ ```python
64
+ >>> from redis_dict import RedisDict
65
+ >>> dic = RedisDict()
66
+ >>> dic['foo'] = 42
67
+ >>> dic['foo']
68
+ 42
69
+ >>> 'foo' in dic
70
+ True
71
+ >>> dic["baz"] = "hello world"
72
+ >>> dic
73
+ {'foo': 42, 'baz': 'hello world'}
59
74
  ```
60
75
  In Redis our example looks like this.
61
76
  ```
@@ -68,12 +83,12 @@ In Redis our example looks like this.
68
83
  "str:hello world"
69
84
  ```
70
85
 
86
+
71
87
  ### Namespaces
72
- Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization accross systems and projects with the same redis instance.
88
+ Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization across systems and projects with the same redis instance.
73
89
 
74
90
  This approach also minimizes the risk of key collisions between different applications, preventing hard-to-debug issues. By leveraging namespaces, RedisDict ensures a cleaner and more maintainable data management experience for developers working on multiple projects.
75
91
 
76
-
77
92
  ## Advanced Features
78
93
 
79
94
  ### Expiration
@@ -106,6 +121,8 @@ with dic.expire_at(seconds):
106
121
  3. Updating keys while preserving the initial timeout In certain situations, there is a need to update the value while keeping the expiration intact. This is achievable by setting the 'preserve_expiration' to true.
107
122
 
108
123
  ```python
124
+ import time
125
+
109
126
  dic = RedisDict(expire=10, preserve_expiration=True)
110
127
  dic['gone'] = 'in ten seconds'
111
128
 
@@ -118,6 +135,7 @@ dic['gone'] = 'gone in 5 seconds'
118
135
  Efficiently batch your requests using the Pipeline feature, which can be easily utilized with a context manager.
119
136
 
120
137
  ```python
138
+ from redis_dict import RedisDict
121
139
  dic = RedisDict(namespace="example")
122
140
 
123
141
  # one round trip to redis
@@ -147,13 +165,14 @@ print(dic["foo"]) # outputs "bar"
147
165
  ### Caching made simple
148
166
  ```python
149
167
  import time
168
+ from datetime import timedelta
150
169
  from redis_dict import RedisDict
151
170
 
152
171
  def expensive_function(x):
153
- time.sleep(2)
172
+ time.sleep(x)
154
173
  return x * 2
155
174
 
156
- cache = RedisDict(namespace="cache", expire=10)
175
+ cache = RedisDict(namespace="cache", expire=timedelta(minutes=60))
157
176
 
158
177
  def cached_expensive_function(x):
159
178
  if x not in cache:
@@ -161,7 +180,7 @@ def cached_expensive_function(x):
161
180
  return cache[x]
162
181
 
163
182
  start_time = time.time()
164
- print(cached_expensive_function(5)) # Takes around 2 seconds to compute and caches the result.
183
+ print(cached_expensive_function(5)) # Takes around 5 seconds to compute and caches the result.
165
184
  print(f"Time taken: {time.time() - start_time:.2f} seconds")
166
185
 
167
186
  start_time = time.time()
@@ -181,7 +200,7 @@ dic["name"] = "John Doe"
181
200
  dic["age"] = 32
182
201
  dic["city"] = "Amsterdam"
183
202
 
184
- # Get value by key
203
+ # Get value by key, from any instance connected to the same redis/namespace
185
204
  print(dic["name"]) # Output: John Doe
186
205
 
187
206
  # Update value by key, got a year older
@@ -234,10 +253,61 @@ print(dic["d"]) # Output: 4
234
253
  For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py#L1).
235
254
  The unit-tests can be as used as a starting point.
236
255
 
256
+ ### Extending Types
257
+
258
+ ## Extending RedisDict with Custom Types
259
+
260
+ RedisDict supports custom type serialization. Here's how to add a new type:
261
+
262
+
263
+ ```python
264
+ import json
265
+ from redis_dict import RedisDict
266
+
267
+ class Person:
268
+ def __init__(self, name, age):
269
+ self.name = name
270
+ self.age = age
271
+
272
+ def encode(self) -> str:
273
+ return json.dumps(self.__dict__)
274
+
275
+ @classmethod
276
+ def decode(cls, encoded_str: str) -> 'Person':
277
+ return cls(**json.loads(encoded_str))
278
+
279
+ redis_dict = RedisDict()
280
+
281
+ # Extend redis dict with the new type
282
+ redis_dict.extends_type(Person)
283
+
284
+ # RedisDict can now seamlessly handle Person instances.
285
+ person = Person(name="John", age=32)
286
+ redis_dict["person1"] = person
287
+
288
+ result = redis_dict["person1"]
289
+
290
+ assert result.name == person.name
291
+ assert result.age == person.age
292
+ ```
293
+
294
+ ```python
295
+ >>> from datetime import datetime
296
+ >>> redis_dict.extends_type(datetime, datetime.isoformat, datetime.fromisoformat)
297
+ >>> redis_dict["now"] = datetime.now()
298
+ >>> redis_dict
299
+ {'now': datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)}
300
+ >>> redis_dict["now"]
301
+ datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)
302
+ ```
303
+
304
+ For more information on [extending types](https://github.com/Attumm/redis-dict/blob/main/extend_types_tests.py).
237
305
  ### Redis Encryption
238
- Setup guide for configuring and utilizing encrypted Redis for redis-dict.
306
+ Setup guide for configuring and utilizing encrypted Redis TLS for redis-dict.
239
307
  [Setup guide](https://github.com/Attumm/redis-dict/blob/main/encrypted_redis.MD)
240
308
 
309
+ ### Redis Storage Encryption
310
+ For storing encrypted data values, it's possible to use extended types. Take a look at this [encrypted test](https://github.com/Attumm/redis-dict/blob/main/encrypt_tests.py).
241
311
 
242
312
  ### Tests
243
313
  The RedisDict library includes a comprehensive suite of tests that ensure its correctness and resilience. The test suite covers various data types, edge cases, and error handling scenarios. It also employs the Hypothesis library for property-based testing, which provides fuzz testing to evaluate the implementation
@@ -0,0 +1,6 @@
1
+ redis_dict.py,sha256=50CSZ5dMBBbr-UU9BSvoGgBItD7uce8F5ty1lphaiUw,36901
2
+ redis_dict-2.6.0.dist-info/LICENSE,sha256=-QiLwYznh_vNUSz337k0faP9Jl0dgtCIHVZ39Uyl6cA,1070
3
+ redis_dict-2.6.0.dist-info/METADATA,sha256=6QJ93NO1RrVKSYbmbgtFKOazqtKrauPstB-H0hI1vDs,12564
4
+ redis_dict-2.6.0.dist-info/WHEEL,sha256=OVMc5UfuAQiSplgO0_WdW7vXVGAt9Hdd6qtN4HotdyA,91
5
+ redis_dict-2.6.0.dist-info/top_level.txt,sha256=Wyp5Xvq_imoxvu-c-Le1rbTZ3pYM5BF440H9YAcgBZ8,11
6
+ redis_dict-2.6.0.dist-info/RECORD,,
@@ -1,5 +1,5 @@
1
1
  Wheel-Version: 1.0
2
- Generator: setuptools (75.1.0)
2
+ Generator: setuptools (75.2.0)
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any
5
5
 
redis_dict.py CHANGED
@@ -1,17 +1,101 @@
1
+ """
2
+ redis_dict.py
3
+
4
+ RedisDict is a Python library that provides a convenient and familiar interface for
5
+ interacting with Redis as if it were a Python dictionary. The simple yet powerful library
6
+ enables you to manage key-value pairs in Redis using native Python syntax of dictionary. It supports
7
+ various data types, including strings, integers, floats, booleans, lists, and dictionaries,
8
+ and includes additional utility functions for more complex use cases.
9
+
10
+ By leveraging Redis for efficient key-value storage, RedisDict allows for high-performance
11
+ data management and is particularly useful for handling large datasets that may exceed local
12
+ memory capacity.
13
+
14
+ ## Features
15
+
16
+ * **Dictionary-like interface**: Use familiar Python dictionary syntax to interact with Redis.
17
+ * **Data Type Support**: Comprehensive support for various data types, including strings,
18
+ integers, floats, booleans, lists, dictionaries, sets, and tuples.
19
+ * **Pipelining support**: Use pipelines for batch operations to improve performance.
20
+ * **Expiration Support**: Enables the setting of expiration times either globally or individually
21
+ per key, through the use of context managers.
22
+ * **Efficiency and Scalability**: RedisDict is designed for use with large datasets and is
23
+ optimized for efficiency. It retrieves only the data needed for a particular operation,
24
+ ensuring efficient memory usage and fast performance.
25
+ * **Namespace Management**: Provides simple and efficient namespace handling to help organize
26
+ and manage data in Redis, streamlining data access and manipulation.
27
+ * **Distributed Computing**: With its ability to seamlessly connect to other instances or
28
+ servers with access to the same Redis instance, RedisDict enables easy distributed computing.
29
+ * **Custom data types**: Add custom types and transformations to suit your specific needs.
30
+
31
+ New feature
32
+
33
+ Custom extendable Validity checks on keys, and values.to support redis-dict base exceptions with messages from
34
+ enabling detailed reporting on the reasons for specific validation failures. This refactor would allow users
35
+ to configure which validity checks to execute, integrate custom validation functions, and specify whether
36
+ to raise an error on validation failures or to drop the operation and log a warning.
37
+
38
+ For example, in a caching scenario, data should only be cached if it falls within defined minimum and
39
+ maximum size constraints. This approach enables straightforward dictionary set operations while ensuring
40
+ that only meaningful data is cached: values greater than 10 MB and less than 100 MB should be cached;
41
+ otherwise, they will be dropped.
42
+
43
+ >>> def my_custom_validity_check(value: str) -> None:
44
+ \"""
45
+ Validates the size of the input.
46
+
47
+ Args:
48
+ value (str): string to validate.
49
+
50
+ Raises:
51
+ RedisDictValidityException: If the length of the input is not within the allowed range.
52
+ \"""
53
+ min_size = 10 * 1024: # Minimum size: 10 KB
54
+ max_size = 10 * 1024 * 1024: # Maximum size: 10 MB
55
+ if len(value) < min_size
56
+ raise RedisDictValidityException(f"value is too small: {len(value)} bytes")
57
+ if len(value) > max_size
58
+ raise RedisDictValidityException(f"value is too large: {len(value)} bytes")
59
+
60
+ >>> cache = RedisDict(namespace='cache_valid_results_for_1_minute',
61
+ ... expire=60,
62
+ ... custom_valid_values_checks=[my_custom_validity_check], # extend with new valid check
63
+ ... validity_exception_suppress=True) # when value is invalid, don't store, and don't raise an exception.
64
+ >>> too_small = "too small to cache"
65
+ >>> cache["1234"] = too_small # Since the value is below 10kb, thus there is no reason to cache the value.
66
+ >>> cache.get("1234") is None
67
+ >>> True
68
+ """
1
69
  import json
70
+
2
71
  from datetime import timedelta
3
72
  from typing import Any, Callable, Dict, Iterator, Set, List, Tuple, Union, Optional
4
- from redis import StrictRedis
5
-
6
73
  from contextlib import contextmanager
7
74
 
75
+ from redis import StrictRedis
76
+
8
77
  SENTINEL = object()
9
78
 
10
- transform_type = Dict[str, Callable[[str], Any]]
11
- pre_transform_type = Dict[str, Callable[[Any], str]]
79
+ EncodeFuncType = Callable[[Any], str]
80
+ DecodeFuncType = Callable[[str], Any]
81
+
82
+ EncodeType = Dict[str, EncodeFuncType]
83
+ DecodeType = Dict[str, DecodeFuncType]
84
+
85
+
86
+ def _create_default_encode(custom_encode_method: str) -> EncodeFuncType:
87
+ def default_encode(obj: Any) -> str:
88
+ return getattr(obj, custom_encode_method)() # type: ignore[no-any-return]
89
+ return default_encode
90
+
12
91
 
92
+ def _create_default_decode(cls: type, custom_decode_method: str) -> DecodeFuncType:
93
+ def default_decode(encoded_str: str) -> Any:
94
+ return getattr(cls, custom_decode_method)(encoded_str)
95
+ return default_decode
13
96
 
14
- def _transform_tuple(val: str) -> Tuple[Any, ...]:
97
+
98
+ def _decode_tuple(val: str) -> Tuple[Any, ...]:
15
99
  """
16
100
  Deserialize a JSON-formatted string to a tuple.
17
101
 
@@ -27,7 +111,7 @@ def _transform_tuple(val: str) -> Tuple[Any, ...]:
27
111
  return tuple(json.loads(val))
28
112
 
29
113
 
30
- def _pre_transform_tuple(val: Tuple[Any, ...]) -> str:
114
+ def _encode_tuple(val: Tuple[Any, ...]) -> str:
31
115
  """
32
116
  Serialize a tuple to a JSON-formatted string.
33
117
 
@@ -43,7 +127,7 @@ def _pre_transform_tuple(val: Tuple[Any, ...]) -> str:
43
127
  return json.dumps(list(val))
44
128
 
45
129
 
46
- def _transform_set(val: str) -> Set[Any]:
130
+ def _decode_set(val: str) -> Set[Any]:
47
131
  """
48
132
  Deserialize a JSON-formatted string to a set.
49
133
 
@@ -59,7 +143,7 @@ def _transform_set(val: str) -> Set[Any]:
59
143
  return set(json.loads(val))
60
144
 
61
145
 
62
- def _pre_transform_set(val: Set[Any]) -> str:
146
+ def _encode_set(val: Set[Any]) -> str:
63
147
  """
64
148
  Serialize a set to a JSON-formatted string.
65
149
 
@@ -75,6 +159,7 @@ def _pre_transform_set(val: Set[Any]) -> str:
75
159
  return json.dumps(list(val))
76
160
 
77
161
 
162
+ # pylint: disable=R0902, R0904
78
163
  class RedisDict:
79
164
  """
80
165
  A Redis-backed dictionary-like data structure with support for advanced features, such as
@@ -92,15 +177,20 @@ class RedisDict:
92
177
  It aims to offer a seamless and familiar interface for developers familiar with Python dictionaries,
93
178
  enabling a smooth transition to a Redis-backed data store.
94
179
 
180
+ Extendable Types: You can extend RedisDict by adding or overriding encoding and decoding functions.
181
+ This functionality enables various use cases, such as managing encrypted data in Redis,
182
+ To implement this, simply create and register your custom encoding and decoding functions.
183
+ By delegating serialization to redis-dict, reduce complexity and have simple code in the codebase.
184
+
95
185
  Attributes:
96
- transform (Dict[str, Callable[[str], Any]]): A dictionary of data type transformation functions for loading data.
97
- pre_transform (Dict[str, Callable[[Any], str]]): A dictionary of data type transformation functions for storing data.
186
+ decoding_registry (Dict[str, DecodeFuncType]): Mapping of decoding transformation functions based on type
187
+ encoding_registry (Dict[str, EncodeFuncType]): Mapping of encoding transformation functions based on type
98
188
  namespace (str): A string used as a prefix for Redis keys to separate data in different namespaces.
99
189
  expire (Union[int, None]): An optional expiration time for keys, in seconds.
100
190
 
101
191
  """
102
192
 
103
- transform: transform_type = {
193
+ decoding_registry: DecodeType = {
104
194
  type('').__name__: str,
105
195
  type(1).__name__: int,
106
196
  type(0.1).__name__: float,
@@ -109,15 +199,15 @@ class RedisDict:
109
199
 
110
200
  "list": json.loads,
111
201
  "dict": json.loads,
112
- "tuple": _transform_tuple,
113
- type(set()).__name__: _transform_set,
202
+ "tuple": _decode_tuple,
203
+ type(set()).__name__: _decode_set,
114
204
  }
115
205
 
116
- pre_transform: pre_transform_type = {
206
+ encoding_registry: EncodeType = {
117
207
  "list": json.dumps,
118
208
  "dict": json.dumps,
119
- "tuple": _pre_transform_tuple,
120
- type(set()).__name__: _pre_transform_set,
209
+ "tuple": _encode_tuple,
210
+ type(set()).__name__: _encode_set,
121
211
  }
122
212
 
123
213
  def __init__(self,
@@ -134,13 +224,20 @@ class RedisDict:
134
224
  preserve_expiration (bool, optional): Preserve the expiration count when the key is updated.
135
225
  **redis_kwargs: Additional keyword arguments passed to StrictRedis.
136
226
  """
137
- self.temp_redis: Optional[StrictRedis[Any]] = None
227
+
138
228
  self.namespace: str = namespace
139
229
  self.expire: Union[int, timedelta, None] = expire
140
230
  self.preserve_expiration: Optional[bool] = preserve_expiration
141
231
  self.redis: StrictRedis[Any] = StrictRedis(decode_responses=True, **redis_kwargs)
142
232
  self.get_redis: StrictRedis[Any] = self.redis
143
233
 
234
+ self.custom_encode_method = "encode"
235
+ self.custom_decode_method = "decode"
236
+
237
+ self._iter: Iterator[str] = iter([])
238
+ self._max_string_size: int = 500 * 1024 * 1024 # 500mb
239
+ self._temp_redis: Optional[StrictRedis[Any]] = None
240
+
144
241
  def _format_key(self, key: str) -> str:
145
242
  """
146
243
  Format a key with the namespace prefix.
@@ -151,7 +248,7 @@ class RedisDict:
151
248
  Returns:
152
249
  str: The formatted key with the namespace prefix.
153
250
  """
154
- return '{}:{}'.format(self.namespace, str(key))
251
+ return f'{self.namespace}:{str(key)}'
155
252
 
156
253
  def _valid_input(self, val: Any, val_type: str) -> bool:
157
254
  """
@@ -169,7 +266,7 @@ class RedisDict:
169
266
  bool: True if the input value is valid, False otherwise.
170
267
  """
171
268
  if val_type == "str":
172
- return len(val) < (500 * 1024 * 1024)
269
+ return len(val) < self._max_string_size
173
270
  return True
174
271
 
175
272
  def _store(self, key: str, value: Any) -> None:
@@ -179,14 +276,21 @@ class RedisDict:
179
276
  Args:
180
277
  key (str): The key to store the value.
181
278
  value (Any): The value to be stored.
279
+
280
+ Note: Validity checks could be refactored to allow for custom exceptions that inherit from ValueError,
281
+ providing detailed information about why a specific validation failed.
282
+ This would enable users to specify which validity checks should be executed, add custom validity functions,
283
+ and choose whether to fail on validation errors, or drop the data and only issue a warning and continue.
284
+ Example use case is caching, to cache data only when it's between min and max sizes.
285
+ Allowing for simple dict set operation, but only cache data that makes sense.
286
+
182
287
  """
183
288
  store_type, key = type(value).__name__, str(key)
184
289
  if not self._valid_input(value, store_type) or not self._valid_input(key, "str"):
185
- # TODO When needed, make valid_input, pass the reason, or throw a exception.
186
290
  raise ValueError("Invalid input value or key size exceeded the maximum limit.")
187
- value = self.pre_transform.get(store_type, lambda x: x)(value) # type: ignore
291
+ value = self.encoding_registry.get(store_type, lambda x: x)(value) # type: ignore
188
292
 
189
- store_value = '{}:{}'.format(store_type, value)
293
+ store_value = f'{store_type}:{value}'
190
294
  formatted_key = self._format_key(key)
191
295
 
192
296
  if self.preserve_expiration and self.redis.exists(formatted_key):
@@ -207,8 +311,8 @@ class RedisDict:
207
311
  result = self.get_redis.get(self._format_key(key))
208
312
  if result is None:
209
313
  return False, None
210
- t, value = result.split(':', 1)
211
- return True, self.transform.get(t, lambda x: x)(value)
314
+ type_, value = result.split(':', 1)
315
+ return True, self.decoding_registry.get(type_, lambda x: x)(value)
212
316
 
213
317
  def _transform(self, result: str) -> Any:
214
318
  """
@@ -220,18 +324,118 @@ class RedisDict:
220
324
  Returns:
221
325
  Any: The transformed Python object.
222
326
  """
223
- t, value = result.split(':', 1)
224
- return self.transform.get(t, lambda x: x)(value)
327
+ type_, value = result.split(':', 1)
328
+ return self.decoding_registry.get(type_, lambda x: x)(value)
225
329
 
226
- def add_type(self, k: str, v: Callable[[str], Any]) -> None:
330
+ def new_type_compliance(
331
+ self,
332
+ class_type: type,
333
+ encode_method_name: Optional[str] = None,
334
+ decode_method_name: Optional[str] = None,
335
+ ) -> None:
227
336
  """
228
- Add a custom type to the transform mapping.
337
+ Checks if a class complies with the required encoding and decoding methods.
338
+
339
+ Args:
340
+ class_type (type): The class to check for compliance.
341
+ encode_method_name (str, optional): Name of encoding method of the class for redis-dict custom types.
342
+ decode_method_name (str, optional): Name of decoding method of the class for redis-dict custom types.
343
+
344
+ Raises:
345
+ NotImplementedError: If the class does not implement the required methods when the respective check is True.
346
+ """
347
+ if encode_method_name is not None:
348
+ if not (hasattr(class_type, encode_method_name) and callable(
349
+ getattr(class_type, encode_method_name))):
350
+ raise NotImplementedError(
351
+ f"Class {class_type.__name__} does not implement the required {encode_method_name} method.")
352
+
353
+ if decode_method_name is not None:
354
+ if not (hasattr(class_type, decode_method_name) and callable(
355
+ getattr(class_type, decode_method_name))):
356
+ raise NotImplementedError(
357
+ f"Class {class_type.__name__} does not implement the required {decode_method_name} class method.")
358
+
359
+ def extends_type(
360
+ self,
361
+ class_type: type,
362
+ encode: Optional[EncodeFuncType] = None,
363
+ decode: Optional[DecodeFuncType] = None,
364
+ encoding_method_name: Optional[str] = None,
365
+ decoding_method_name: Optional[str] = None,
366
+ ) -> None:
367
+ """
368
+ Extends RedisDict to support a custom type in the encode/decode mapping.
369
+
370
+ This method enables serialization of instances based on their type,
371
+ allowing for custom types, specialized storage formats, and more.
372
+ There are three ways to add custom types:
373
+ 1. Have a class with an `encode` instance method and a `decode` class method.
374
+ 2. Have a class and pass encoding and decoding functions, where
375
+ `encode` converts the class instance to a string, and
376
+ `decode` takes the string and recreates the class instance.
377
+ 3. Have a class that already has serialization methods, that satisfies the:
378
+ EncodeFuncType = Callable[[Any], str]
379
+ DecodeFuncType = Callable[[str], Any]
380
+
381
+ `custom_encode_method`
382
+ `custom_decode_method` attributes.
229
383
 
230
384
  Args:
231
- k (str): The key representing the type.
232
- v (Callable): The transformation function for the type.
385
+ class_type (Type[type]): The class `__name__` will become the key for the encoding and decoding functions.
386
+ encode (Optional[EncodeFuncType]): function that encodes an object into a storable string format.
387
+ This function should take an instance of `class_type` as input and return a string.
388
+ decode (Optional[DecodeFuncType]): function that decodes a string back into an object of `class_type`.
389
+ This function should take a string as input and return an instance of `class_type`.
390
+ encoding_method_name (str, optional): Name of encoding method of the class for redis-dict custom types.
391
+ decoding_method_name (str, optional): Name of decoding method of the class for redis-dict custom types.
392
+
393
+ If no encoding or decoding function is provided, default to use the `encode` and `decode` methods of the class.
394
+
395
+ The `encode` method should be an instance method that converts the object to a string.
396
+ The `decode` method should be a class method that takes a string and returns an instance of the class.
397
+
398
+ The method names for encoding and decoding can be changed by modifying the
399
+ - `custom_encode_method`
400
+ - `custom_decode_method`
401
+ attributes of the RedisDict instance
402
+
403
+ Example:
404
+ class Person:
405
+ def __init__(self, name, age):
406
+ self.name = name
407
+ self.age = age
408
+
409
+ def encode(self) -> str:
410
+ return json.dumps(self.__dict__)
411
+
412
+ @classmethod
413
+ def decode(cls, encoded_str: str) -> 'Person':
414
+ return cls(**json.loads(encoded_str))
415
+
416
+ redis_dict.extends_type(Person)
417
+
418
+ Note:
419
+ You can check for compliance of a class separately using the `new_type_compliance` method:
420
+
421
+ This method raises a NotImplementedError if either `encode` or `decode` is `None`
422
+ and the class does not implement the corresponding method.
233
423
  """
234
- self.transform[k] = v
424
+
425
+ if encode is None or decode is None:
426
+ encode_method_name = encoding_method_name or self.custom_encode_method
427
+ if encode is None:
428
+ self.new_type_compliance(class_type, encode_method_name=encode_method_name)
429
+ encode = _create_default_encode(encode_method_name)
430
+
431
+ if decode is None:
432
+ decode_method_name = decoding_method_name or self.custom_decode_method
433
+ self.new_type_compliance(class_type, decode_method_name=decode_method_name)
434
+ decode = _create_default_decode(class_type, decode_method_name)
435
+
436
+ type_name = class_type.__name__
437
+ self.decoding_registry[type_name] = decode
438
+ self.encoding_registry[type_name] = encode
235
439
 
236
440
  def __eq__(self, other: Any) -> bool:
237
441
  """
@@ -327,7 +531,7 @@ class RedisDict:
327
531
  Returns:
328
532
  Iterator[str]: An iterator over the keys of the RedisDict.
329
533
  """
330
- self.iter = self.iterkeys()
534
+ self._iter = self.iterkeys()
331
535
  return self
332
536
 
333
537
  def __repr__(self) -> str:
@@ -358,7 +562,7 @@ class RedisDict:
358
562
  Raises:
359
563
  StopIteration: If there are no more items.
360
564
  """
361
- return next(self.iter)
565
+ return next(self._iter)
362
566
 
363
567
  def next(self) -> str:
364
568
  """
@@ -370,7 +574,29 @@ class RedisDict:
370
574
  Raises:
371
575
  StopIteration: If there are no more items.
372
576
  """
373
- return self.__next__()
577
+ return next(self)
578
+
579
+ def _create_iter_query(self, search_term: str) -> str:
580
+ """
581
+ Create a Redis query string for iterating over keys based on the given search term.
582
+
583
+ This method constructs a query by prefixing the search term with the namespace
584
+ followed by a wildcard to facilitate scanning for keys that start with the
585
+ provided search term.
586
+
587
+ Args:
588
+ search_term (str): The term to search for in Redis keys.
589
+
590
+ Returns:
591
+ str: A formatted query string that can be used to find keys in Redis.
592
+
593
+ Example:
594
+ >>> d = RedisDict(namespace='foo')
595
+ >>> query = self._create_iter_query('bar')
596
+ >>> print(query)
597
+ 'foo:bar*'
598
+ """
599
+ return f'{self.namespace}:{search_term}*'
374
600
 
375
601
  def _scan_keys(self, search_term: str = '') -> Iterator[str]:
376
602
  """
@@ -382,7 +608,8 @@ class RedisDict:
382
608
  Returns:
383
609
  Iterator[str]: An iterator of matching Redis keys.
384
610
  """
385
- return self.get_redis.scan_iter(match='{}:{}{}'.format(self.namespace, search_term, '*'))
611
+ search_query = self._create_iter_query(search_term)
612
+ return self.get_redis.scan_iter(match=search_query)
386
613
 
387
614
  def get(self, key: str, default: Optional[Any] = None) -> Any:
388
615
  """
@@ -413,7 +640,8 @@ class RedisDict:
413
640
  Note: for python2 str is needed
414
641
  """
415
642
  to_rm = len(self.namespace) + 1
416
- cursor, data = self.get_redis.scan(match='{}:{}{}'.format(self.namespace, search_term, '*'), count=1)
643
+ search_query = self._create_iter_query(search_term)
644
+ _, data = self.get_redis.scan(match=search_query, count=1)
417
645
  for item in data:
418
646
  return str(item[to_rm:])
419
647
 
@@ -435,7 +663,7 @@ class RedisDict:
435
663
  to_rm = len(self.namespace) + 1
436
664
  for item in self._scan_keys():
437
665
  try:
438
- yield (str(item[to_rm:]), self[item[to_rm:]])
666
+ yield str(item[to_rm:]), self[item[to_rm:]]
439
667
  except KeyError:
440
668
  pass
441
669
 
@@ -488,11 +716,12 @@ class RedisDict:
488
716
  Redis pipelining is employed to group multiple commands into a single request, minimizing
489
717
  network round-trip time, latency, and I/O load, thereby enhancing the overall performance.
490
718
 
491
- It is important to highlight that the clear method can be safely executed within the context of an initiated pipeline operation
719
+ It is important to highlight that the clear method can be safely executed within
720
+ the context of an initiated pipeline operation.
492
721
  """
493
722
  with self.pipeline():
494
723
  for key in self:
495
- del (self[key])
724
+ del self[key]
496
725
 
497
726
  def pop(self, key: str, default: Union[Any, object] = SENTINEL) -> Any:
498
727
  """
@@ -516,7 +745,7 @@ class RedisDict:
516
745
  return default
517
746
  raise
518
747
 
519
- del (self[key])
748
+ del self[key]
520
749
  return value
521
750
 
522
751
  def popitem(self) -> Tuple[str, Any]:
@@ -594,7 +823,8 @@ class RedisDict:
594
823
  value (Optional[Any], optional): The value to be assigned to each key in the RedisDict. Defaults to None.
595
824
 
596
825
  Returns:
597
- RedisDict: The current RedisDict instance, now populated with the keys from the iterable and their corresponding values.
826
+ RedisDict: The current RedisDict instance,populated with the keys from the iterable and their
827
+ corresponding values.
598
828
  """
599
829
  for key in iterable:
600
830
  self[key] = value
@@ -640,7 +870,7 @@ class RedisDict:
640
870
  Args:
641
871
  iterable (List[str]): A list of keys representing the chain.
642
872
  """
643
- return self.__delitem__(':'.join(iterable))
873
+ del self[':'.join(iterable)]
644
874
 
645
875
  # def expire_at(self, sec_epoch: int | timedelta) -> Iterator[None]:
646
876
  # compatibility with Python 3.9 typing
@@ -668,13 +898,13 @@ class RedisDict:
668
898
  ContextManager: A context manager to create a Redis pipeline batching all operations within the context.
669
899
  """
670
900
  top_level = False
671
- if self.temp_redis is None:
672
- self.redis, self.temp_redis, top_level = self.redis.pipeline(), self.redis, True
901
+ if self._temp_redis is None:
902
+ self.redis, self._temp_redis, top_level = self.redis.pipeline(), self.redis, True
673
903
  try:
674
904
  yield
675
905
  finally:
676
906
  if top_level:
677
- _, self.temp_redis, self.redis = self.redis.execute(), None, self.temp_redis # type: ignore
907
+ _, self._temp_redis, self.redis = self.redis.execute(), None, self._temp_redis # type: ignore
678
908
 
679
909
  def multi_get(self, key: str) -> List[Any]:
680
910
  """
@@ -717,7 +947,9 @@ class RedisDict:
717
947
  if len(keys) == 0:
718
948
  return {}
719
949
  to_rm = keys[0].rfind(':') + 1
720
- return dict(zip([i[to_rm:] for i in keys], (self._transform(i) for i in self.redis.mget(keys) if i is not None)))
950
+ return dict(
951
+ zip([i[to_rm:] for i in keys], (self._transform(i) for i in self.redis.mget(keys) if i is not None))
952
+ )
721
953
 
722
954
  def multi_del(self, key: str) -> int:
723
955
  """
@@ -1,6 +0,0 @@
1
- redis_dict.py,sha256=ZQSg6bCaV_Ho18x7CtjYgD0kakFeskn3TfJzR3e0I28,25677
2
- redis_dict-2.5.1.dist-info/LICENSE,sha256=-QiLwYznh_vNUSz337k0faP9Jl0dgtCIHVZ39Uyl6cA,1070
3
- redis_dict-2.5.1.dist-info/METADATA,sha256=dKyus34pm_rQ-icVu7L53N_QzMSaxbLWhKuVSTh7RfI,10000
4
- redis_dict-2.5.1.dist-info/WHEEL,sha256=GV9aMThwP_4oNCtvEC2ec3qUYutgWeAzklro_0m4WJQ,91
5
- redis_dict-2.5.1.dist-info/top_level.txt,sha256=Wyp5Xvq_imoxvu-c-Le1rbTZ3pYM5BF440H9YAcgBZ8,11
6
- redis_dict-2.5.1.dist-info/RECORD,,