redis-dict 2.5.0__tar.gz → 2.6.0__tar.gz

Sign up to get free protection for your applications and to get access to all the features.
@@ -1,11 +1,45 @@
1
+ Metadata-Version: 2.1
2
+ Name: redis dict
3
+ Version: 2.6.0
4
+ Summary: Dictionary with Redis as storage backend
5
+ Home-page: https://github.com/Attumm/redisdict
6
+ Author: Melvin Bijman
7
+ Author-email: bijman.m.m@gmail.com
8
+ License: MIT
9
+ Keywords: redis python dictionary dict key-value key:value database caching distributed-computing dictionary-interface large-datasets scientific-computing data-persistence high-performance scalable pipelining batching big-data data-types distributed-algorithms encryption data-management
10
+ Platform: any
11
+ Classifier: Development Status :: 5 - Production/Stable
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Intended Audience :: Information Technology
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Topic :: Internet
16
+ Classifier: Topic :: Scientific/Engineering
17
+ Classifier: Topic :: Database
18
+ Classifier: Topic :: System :: Distributed Computing
19
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
20
+ Classifier: Topic :: Software Development :: Object Brokering
21
+ Classifier: Topic :: Database :: Database Engines/Servers
22
+ Classifier: License :: OSI Approved :: MIT License
23
+ Classifier: Programming Language :: Python :: 3
24
+ Classifier: Programming Language :: Python :: 3.6
25
+ Classifier: Programming Language :: Python :: 3.7
26
+ Classifier: Programming Language :: Python :: 3.8
27
+ Classifier: Programming Language :: Python :: 3.9
28
+ Classifier: Programming Language :: Python :: 3.10
29
+ Classifier: Programming Language :: Python :: 3.11
30
+ Classifier: Programming Language :: Python :: 3.12
31
+ Description-Content-Type: text/markdown
32
+ License-File: LICENSE
33
+ Requires-Dist: redis
34
+
1
35
  # Redis-dict
2
- [![Build Status](https://travis-ci.com/Attumm/redis-dict.svg?branch=main)](https://travis-ci.com/Attumm/redis-dict)
36
+ [![CI](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml/badge.svg)](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml)
37
+ [![codecov](https://codecov.io/gh/Attumm/redis-dict/graph/badge.svg?token=Lqs7McQGEs)](https://codecov.io/gh/Attumm/redis-dict)
3
38
  [![Downloads](https://static.pepy.tech/badge/redis-dict/month)](https://pepy.tech/project/redis-dict)
4
39
 
5
- RedisDict is a Python library that provides a convenient and familiar interface for interacting with Redis as if it were a Python dictionary. This simple yet powerful library enables you to manage key-value pairs in Redis using native Python syntax. It supports various data types, including strings, integers, floats, booleans, lists, and dictionaries, and includes additional utility functions for more complex use cases.
6
-
7
- By leveraging Redis for efficient key-value storage, RedisDict allows for high-performance data management and is particularly useful for handling large datasets that may exceed local memory capacity.
40
+ RedisDict is a Python library that offers a convenient and familiar interface for interacting with Redis, treating it as if it were a Python dictionary. Its goal is to help developers write clean, Pythonic code while using Redis as a storage solution for seamless distributed computing. This simple yet powerful library utilizes Redis as a key-value store and supports various data types, including strings, integers, floats, booleans, lists, and dictionaries. Additionally, developers can extend RedisDict to work with custom objects.
8
41
 
42
+ The library includes utility functions for more complex use cases such as caching, batching, and more. By leveraging Redis for efficient key-value storage, RedisDict enables high-performance data management, maintaining efficiency even with large datasets and Redis instances.
9
43
 
10
44
  ## Features
11
45
 
@@ -16,20 +50,27 @@ By leveraging Redis for efficient key-value storage, RedisDict allows for high-p
16
50
  * Efficiency and Scalability: RedisDict is designed for use with large datasets and is optimized for efficiency. It retrieves only the data needed for a particular operation, ensuring efficient memory usage and fast performance.
17
51
  * Namespace Management: Provides simple and efficient namespace handling to help organize and manage data in Redis, streamlining data access and manipulation.
18
52
  * Distributed Computing: With its ability to seamlessly connect to other instances or servers with access to the same Redis instance, RedisDict enables easy distributed computing.
19
- * Custom data types: Add custom types and transformations to suit your specific needs.
53
+ * Custom data: types: Add custom types encoding/decoding to store your data types.
54
+ * Encryption: allows for storing data encrypted, while retaining the simple dictionary interface.
20
55
 
21
56
  ## Example
22
57
  Redis is an exceptionally fast database when used appropriately. RedisDict leverages Redis for efficient key-value storage, enabling high-performance data management.
23
58
 
24
- ```python
25
- from redis_dict import RedisDict
59
+ ```bash
60
+ pip install redis-dict
61
+ ```
26
62
 
27
- dic = RedisDict()
28
- dic['foo'] = 42
29
- print(dic['foo']) # Output: 42
30
- print('foo' in dic) # Output: True
31
- dic["baz"] = "hello world"
32
- print(dic) # Output: {'foo': 42, 'baz': 'hello world'}
63
+ ```python
64
+ >>> from redis_dict import RedisDict
65
+ >>> dic = RedisDict()
66
+ >>> dic['foo'] = 42
67
+ >>> dic['foo']
68
+ 42
69
+ >>> 'foo' in dic
70
+ True
71
+ >>> dic["baz"] = "hello world"
72
+ >>> dic
73
+ {'foo': 42, 'baz': 'hello world'}
33
74
  ```
34
75
  In Redis our example looks like this.
35
76
  ```
@@ -42,12 +83,12 @@ In Redis our example looks like this.
42
83
  "str:hello world"
43
84
  ```
44
85
 
86
+
45
87
  ### Namespaces
46
- Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization accross systems and projects with the same redis instance.
88
+ Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization across systems and projects with the same redis instance.
47
89
 
48
90
  This approach also minimizes the risk of key collisions between different applications, preventing hard-to-debug issues. By leveraging namespaces, RedisDict ensures a cleaner and more maintainable data management experience for developers working on multiple projects.
49
91
 
50
-
51
92
  ## Advanced Features
52
93
 
53
94
  ### Expiration
@@ -80,6 +121,8 @@ with dic.expire_at(seconds):
80
121
  3. Updating keys while preserving the initial timeout In certain situations, there is a need to update the value while keeping the expiration intact. This is achievable by setting the 'preserve_expiration' to true.
81
122
 
82
123
  ```python
124
+ import time
125
+
83
126
  dic = RedisDict(expire=10, preserve_expiration=True)
84
127
  dic['gone'] = 'in ten seconds'
85
128
 
@@ -92,6 +135,7 @@ dic['gone'] = 'gone in 5 seconds'
92
135
  Efficiently batch your requests using the Pipeline feature, which can be easily utilized with a context manager.
93
136
 
94
137
  ```python
138
+ from redis_dict import RedisDict
95
139
  dic = RedisDict(namespace="example")
96
140
 
97
141
  # one round trip to redis
@@ -121,13 +165,14 @@ print(dic["foo"]) # outputs "bar"
121
165
  ### Caching made simple
122
166
  ```python
123
167
  import time
168
+ from datetime import timedelta
124
169
  from redis_dict import RedisDict
125
170
 
126
171
  def expensive_function(x):
127
- time.sleep(2)
172
+ time.sleep(x)
128
173
  return x * 2
129
174
 
130
- cache = RedisDict(namespace="cache", expire=10)
175
+ cache = RedisDict(namespace="cache", expire=timedelta(minutes=60))
131
176
 
132
177
  def cached_expensive_function(x):
133
178
  if x not in cache:
@@ -135,7 +180,7 @@ def cached_expensive_function(x):
135
180
  return cache[x]
136
181
 
137
182
  start_time = time.time()
138
- print(cached_expensive_function(5)) # Takes around 2 seconds to compute and caches the result.
183
+ print(cached_expensive_function(5)) # Takes around 5 seconds to compute and caches the result.
139
184
  print(f"Time taken: {time.time() - start_time:.2f} seconds")
140
185
 
141
186
  start_time = time.time()
@@ -155,7 +200,7 @@ dic["name"] = "John Doe"
155
200
  dic["age"] = 32
156
201
  dic["city"] = "Amsterdam"
157
202
 
158
- # Get value by key
203
+ # Get value by key, from any instance connected to the same redis/namespace
159
204
  print(dic["name"]) # Output: John Doe
160
205
 
161
206
  # Update value by key, got a year older
@@ -205,13 +250,64 @@ print(dic["d"]) # Output: 4
205
250
  ```
206
251
 
207
252
  ### Additional Examples
208
- For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py.py#L1).
253
+ For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py#L1).
209
254
  The unit-tests can be as used as a starting point.
210
255
 
256
+ ### Extending Types
257
+
258
+ ## Extending RedisDict with Custom Types
259
+
260
+ RedisDict supports custom type serialization. Here's how to add a new type:
261
+
262
+
263
+ ```python
264
+ import json
265
+ from redis_dict import RedisDict
266
+
267
+ class Person:
268
+ def __init__(self, name, age):
269
+ self.name = name
270
+ self.age = age
271
+
272
+ def encode(self) -> str:
273
+ return json.dumps(self.__dict__)
274
+
275
+ @classmethod
276
+ def decode(cls, encoded_str: str) -> 'Person':
277
+ return cls(**json.loads(encoded_str))
278
+
279
+ redis_dict = RedisDict()
280
+
281
+ # Extend redis dict with the new type
282
+ redis_dict.extends_type(Person)
283
+
284
+ # RedisDict can now seamlessly handle Person instances.
285
+ person = Person(name="John", age=32)
286
+ redis_dict["person1"] = person
287
+
288
+ result = redis_dict["person1"]
289
+
290
+ assert result.name == person.name
291
+ assert result.age == person.age
292
+ ```
293
+
294
+ ```python
295
+ >>> from datetime import datetime
296
+ >>> redis_dict.extends_type(datetime, datetime.isoformat, datetime.fromisoformat)
297
+ >>> redis_dict["now"] = datetime.now()
298
+ >>> redis_dict
299
+ {'now': datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)}
300
+ >>> redis_dict["now"]
301
+ datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)
302
+ ```
303
+
304
+ For more information on [extending types](https://github.com/Attumm/redis-dict/blob/main/extend_types_tests.py).
211
305
  ### Redis Encryption
212
- Setup guide for configuring and utilizing encrypted Redis for redis-dict.
306
+ Setup guide for configuring and utilizing encrypted Redis TLS for redis-dict.
213
307
  [Setup guide](https://github.com/Attumm/redis-dict/blob/main/encrypted_redis.MD)
214
308
 
309
+ ### Redis Storage Encryption
310
+ For storing encrypted data values, it's possible to use extended types. Take a look at this [encrypted test](https://github.com/Attumm/redis-dict/blob/main/encrypt_tests.py).
215
311
 
216
312
  ### Tests
217
313
  The RedisDict library includes a comprehensive suite of tests that ensure its correctness and resilience. The test suite covers various data types, edge cases, and error handling scenarios. It also employs the Hypothesis library for property-based testing, which provides fuzz testing to evaluate the implementation
@@ -1,36 +1,11 @@
1
- Metadata-Version: 2.1
2
- Name: redis dict
3
- Version: 2.5.0
4
- Summary: Dictionary with Redis as storage backend
5
- Home-page: https://github.com/Attumm/redisdict
6
- Author: Melvin Bijman
7
- Author-email: bijman.m.m@gmail.com
8
- License: MIT
9
- Classifier: Development Status :: 5 - Production/Stable
10
- Classifier: Intended Audience :: Developers
11
- Classifier: Topic :: Database
12
- Classifier: Topic :: System :: Distributed Computing
13
- Classifier: License :: OSI Approved :: MIT License
14
- Classifier: Programming Language :: Python :: 3
15
- Classifier: Programming Language :: Python :: 3.6
16
- Classifier: Programming Language :: Python :: 3.7
17
- Classifier: Programming Language :: Python :: 3.8
18
- Classifier: Programming Language :: Python :: 3.9
19
- Classifier: Programming Language :: Python :: 3.10
20
- Classifier: Programming Language :: Python :: 3.11
21
- Classifier: Programming Language :: Python :: 3.12
22
- Description-Content-Type: text/markdown
23
- License-File: LICENSE
24
- Requires-Dist: redis
25
-
26
1
  # Redis-dict
27
- [![Build Status](https://travis-ci.com/Attumm/redis-dict.svg?branch=main)](https://travis-ci.com/Attumm/redis-dict)
2
+ [![CI](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml/badge.svg)](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml)
3
+ [![codecov](https://codecov.io/gh/Attumm/redis-dict/graph/badge.svg?token=Lqs7McQGEs)](https://codecov.io/gh/Attumm/redis-dict)
28
4
  [![Downloads](https://static.pepy.tech/badge/redis-dict/month)](https://pepy.tech/project/redis-dict)
29
5
 
30
- RedisDict is a Python library that provides a convenient and familiar interface for interacting with Redis as if it were a Python dictionary. This simple yet powerful library enables you to manage key-value pairs in Redis using native Python syntax. It supports various data types, including strings, integers, floats, booleans, lists, and dictionaries, and includes additional utility functions for more complex use cases.
31
-
32
- By leveraging Redis for efficient key-value storage, RedisDict allows for high-performance data management and is particularly useful for handling large datasets that may exceed local memory capacity.
6
+ RedisDict is a Python library that offers a convenient and familiar interface for interacting with Redis, treating it as if it were a Python dictionary. Its goal is to help developers write clean, Pythonic code while using Redis as a storage solution for seamless distributed computing. This simple yet powerful library utilizes Redis as a key-value store and supports various data types, including strings, integers, floats, booleans, lists, and dictionaries. Additionally, developers can extend RedisDict to work with custom objects.
33
7
 
8
+ The library includes utility functions for more complex use cases such as caching, batching, and more. By leveraging Redis for efficient key-value storage, RedisDict enables high-performance data management, maintaining efficiency even with large datasets and Redis instances.
34
9
 
35
10
  ## Features
36
11
 
@@ -41,20 +16,27 @@ By leveraging Redis for efficient key-value storage, RedisDict allows for high-p
41
16
  * Efficiency and Scalability: RedisDict is designed for use with large datasets and is optimized for efficiency. It retrieves only the data needed for a particular operation, ensuring efficient memory usage and fast performance.
42
17
  * Namespace Management: Provides simple and efficient namespace handling to help organize and manage data in Redis, streamlining data access and manipulation.
43
18
  * Distributed Computing: With its ability to seamlessly connect to other instances or servers with access to the same Redis instance, RedisDict enables easy distributed computing.
44
- * Custom data types: Add custom types and transformations to suit your specific needs.
19
+ * Custom data: types: Add custom types encoding/decoding to store your data types.
20
+ * Encryption: allows for storing data encrypted, while retaining the simple dictionary interface.
45
21
 
46
22
  ## Example
47
23
  Redis is an exceptionally fast database when used appropriately. RedisDict leverages Redis for efficient key-value storage, enabling high-performance data management.
48
24
 
49
- ```python
50
- from redis_dict import RedisDict
25
+ ```bash
26
+ pip install redis-dict
27
+ ```
51
28
 
52
- dic = RedisDict()
53
- dic['foo'] = 42
54
- print(dic['foo']) # Output: 42
55
- print('foo' in dic) # Output: True
56
- dic["baz"] = "hello world"
57
- print(dic) # Output: {'foo': 42, 'baz': 'hello world'}
29
+ ```python
30
+ >>> from redis_dict import RedisDict
31
+ >>> dic = RedisDict()
32
+ >>> dic['foo'] = 42
33
+ >>> dic['foo']
34
+ 42
35
+ >>> 'foo' in dic
36
+ True
37
+ >>> dic["baz"] = "hello world"
38
+ >>> dic
39
+ {'foo': 42, 'baz': 'hello world'}
58
40
  ```
59
41
  In Redis our example looks like this.
60
42
  ```
@@ -67,12 +49,12 @@ In Redis our example looks like this.
67
49
  "str:hello world"
68
50
  ```
69
51
 
52
+
70
53
  ### Namespaces
71
- Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization accross systems and projects with the same redis instance.
54
+ Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization across systems and projects with the same redis instance.
72
55
 
73
56
  This approach also minimizes the risk of key collisions between different applications, preventing hard-to-debug issues. By leveraging namespaces, RedisDict ensures a cleaner and more maintainable data management experience for developers working on multiple projects.
74
57
 
75
-
76
58
  ## Advanced Features
77
59
 
78
60
  ### Expiration
@@ -105,6 +87,8 @@ with dic.expire_at(seconds):
105
87
  3. Updating keys while preserving the initial timeout In certain situations, there is a need to update the value while keeping the expiration intact. This is achievable by setting the 'preserve_expiration' to true.
106
88
 
107
89
  ```python
90
+ import time
91
+
108
92
  dic = RedisDict(expire=10, preserve_expiration=True)
109
93
  dic['gone'] = 'in ten seconds'
110
94
 
@@ -117,6 +101,7 @@ dic['gone'] = 'gone in 5 seconds'
117
101
  Efficiently batch your requests using the Pipeline feature, which can be easily utilized with a context manager.
118
102
 
119
103
  ```python
104
+ from redis_dict import RedisDict
120
105
  dic = RedisDict(namespace="example")
121
106
 
122
107
  # one round trip to redis
@@ -146,13 +131,14 @@ print(dic["foo"]) # outputs "bar"
146
131
  ### Caching made simple
147
132
  ```python
148
133
  import time
134
+ from datetime import timedelta
149
135
  from redis_dict import RedisDict
150
136
 
151
137
  def expensive_function(x):
152
- time.sleep(2)
138
+ time.sleep(x)
153
139
  return x * 2
154
140
 
155
- cache = RedisDict(namespace="cache", expire=10)
141
+ cache = RedisDict(namespace="cache", expire=timedelta(minutes=60))
156
142
 
157
143
  def cached_expensive_function(x):
158
144
  if x not in cache:
@@ -160,7 +146,7 @@ def cached_expensive_function(x):
160
146
  return cache[x]
161
147
 
162
148
  start_time = time.time()
163
- print(cached_expensive_function(5)) # Takes around 2 seconds to compute and caches the result.
149
+ print(cached_expensive_function(5)) # Takes around 5 seconds to compute and caches the result.
164
150
  print(f"Time taken: {time.time() - start_time:.2f} seconds")
165
151
 
166
152
  start_time = time.time()
@@ -180,7 +166,7 @@ dic["name"] = "John Doe"
180
166
  dic["age"] = 32
181
167
  dic["city"] = "Amsterdam"
182
168
 
183
- # Get value by key
169
+ # Get value by key, from any instance connected to the same redis/namespace
184
170
  print(dic["name"]) # Output: John Doe
185
171
 
186
172
  # Update value by key, got a year older
@@ -230,13 +216,64 @@ print(dic["d"]) # Output: 4
230
216
  ```
231
217
 
232
218
  ### Additional Examples
233
- For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py.py#L1).
219
+ For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py#L1).
234
220
  The unit-tests can be as used as a starting point.
235
221
 
222
+ ### Extending Types
223
+
224
+ ## Extending RedisDict with Custom Types
225
+
226
+ RedisDict supports custom type serialization. Here's how to add a new type:
227
+
228
+
229
+ ```python
230
+ import json
231
+ from redis_dict import RedisDict
232
+
233
+ class Person:
234
+ def __init__(self, name, age):
235
+ self.name = name
236
+ self.age = age
237
+
238
+ def encode(self) -> str:
239
+ return json.dumps(self.__dict__)
240
+
241
+ @classmethod
242
+ def decode(cls, encoded_str: str) -> 'Person':
243
+ return cls(**json.loads(encoded_str))
244
+
245
+ redis_dict = RedisDict()
246
+
247
+ # Extend redis dict with the new type
248
+ redis_dict.extends_type(Person)
249
+
250
+ # RedisDict can now seamlessly handle Person instances.
251
+ person = Person(name="John", age=32)
252
+ redis_dict["person1"] = person
253
+
254
+ result = redis_dict["person1"]
255
+
256
+ assert result.name == person.name
257
+ assert result.age == person.age
258
+ ```
259
+
260
+ ```python
261
+ >>> from datetime import datetime
262
+ >>> redis_dict.extends_type(datetime, datetime.isoformat, datetime.fromisoformat)
263
+ >>> redis_dict["now"] = datetime.now()
264
+ >>> redis_dict
265
+ {'now': datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)}
266
+ >>> redis_dict["now"]
267
+ datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)
268
+ ```
269
+
270
+ For more information on [extending types](https://github.com/Attumm/redis-dict/blob/main/extend_types_tests.py).
236
271
  ### Redis Encryption
237
- Setup guide for configuring and utilizing encrypted Redis for redis-dict.
272
+ Setup guide for configuring and utilizing encrypted Redis TLS for redis-dict.
238
273
  [Setup guide](https://github.com/Attumm/redis-dict/blob/main/encrypted_redis.MD)
239
274
 
275
+ ### Redis Storage Encryption
276
+ For storing encrypted data values, it's possible to use extended types. Take a look at this [encrypted test](https://github.com/Attumm/redis-dict/blob/main/encrypt_tests.py).
240
277
 
241
278
  ### Tests
242
279
  The RedisDict library includes a comprehensive suite of tests that ensure its correctness and resilience. The test suite covers various data types, edge cases, and error handling scenarios. It also employs the Hypothesis library for property-based testing, which provides fuzz testing to evaluate the implementation
@@ -1,15 +1,24 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: redis-dict
3
- Version: 2.5.0
3
+ Version: 2.6.0
4
4
  Summary: Dictionary with Redis as storage backend
5
5
  Home-page: https://github.com/Attumm/redisdict
6
6
  Author: Melvin Bijman
7
7
  Author-email: bijman.m.m@gmail.com
8
8
  License: MIT
9
+ Keywords: redis python dictionary dict key-value key:value database caching distributed-computing dictionary-interface large-datasets scientific-computing data-persistence high-performance scalable pipelining batching big-data data-types distributed-algorithms encryption data-management
10
+ Platform: any
9
11
  Classifier: Development Status :: 5 - Production/Stable
10
12
  Classifier: Intended Audience :: Developers
13
+ Classifier: Intended Audience :: Information Technology
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Topic :: Internet
16
+ Classifier: Topic :: Scientific/Engineering
11
17
  Classifier: Topic :: Database
12
18
  Classifier: Topic :: System :: Distributed Computing
19
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
20
+ Classifier: Topic :: Software Development :: Object Brokering
21
+ Classifier: Topic :: Database :: Database Engines/Servers
13
22
  Classifier: License :: OSI Approved :: MIT License
14
23
  Classifier: Programming Language :: Python :: 3
15
24
  Classifier: Programming Language :: Python :: 3.6
@@ -24,13 +33,13 @@ License-File: LICENSE
24
33
  Requires-Dist: redis
25
34
 
26
35
  # Redis-dict
27
- [![Build Status](https://travis-ci.com/Attumm/redis-dict.svg?branch=main)](https://travis-ci.com/Attumm/redis-dict)
36
+ [![CI](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml/badge.svg)](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml)
37
+ [![codecov](https://codecov.io/gh/Attumm/redis-dict/graph/badge.svg?token=Lqs7McQGEs)](https://codecov.io/gh/Attumm/redis-dict)
28
38
  [![Downloads](https://static.pepy.tech/badge/redis-dict/month)](https://pepy.tech/project/redis-dict)
29
39
 
30
- RedisDict is a Python library that provides a convenient and familiar interface for interacting with Redis as if it were a Python dictionary. This simple yet powerful library enables you to manage key-value pairs in Redis using native Python syntax. It supports various data types, including strings, integers, floats, booleans, lists, and dictionaries, and includes additional utility functions for more complex use cases.
31
-
32
- By leveraging Redis for efficient key-value storage, RedisDict allows for high-performance data management and is particularly useful for handling large datasets that may exceed local memory capacity.
40
+ RedisDict is a Python library that offers a convenient and familiar interface for interacting with Redis, treating it as if it were a Python dictionary. Its goal is to help developers write clean, Pythonic code while using Redis as a storage solution for seamless distributed computing. This simple yet powerful library utilizes Redis as a key-value store and supports various data types, including strings, integers, floats, booleans, lists, and dictionaries. Additionally, developers can extend RedisDict to work with custom objects.
33
41
 
42
+ The library includes utility functions for more complex use cases such as caching, batching, and more. By leveraging Redis for efficient key-value storage, RedisDict enables high-performance data management, maintaining efficiency even with large datasets and Redis instances.
34
43
 
35
44
  ## Features
36
45
 
@@ -41,20 +50,27 @@ By leveraging Redis for efficient key-value storage, RedisDict allows for high-p
41
50
  * Efficiency and Scalability: RedisDict is designed for use with large datasets and is optimized for efficiency. It retrieves only the data needed for a particular operation, ensuring efficient memory usage and fast performance.
42
51
  * Namespace Management: Provides simple and efficient namespace handling to help organize and manage data in Redis, streamlining data access and manipulation.
43
52
  * Distributed Computing: With its ability to seamlessly connect to other instances or servers with access to the same Redis instance, RedisDict enables easy distributed computing.
44
- * Custom data types: Add custom types and transformations to suit your specific needs.
53
+ * Custom data: types: Add custom types encoding/decoding to store your data types.
54
+ * Encryption: allows for storing data encrypted, while retaining the simple dictionary interface.
45
55
 
46
56
  ## Example
47
57
  Redis is an exceptionally fast database when used appropriately. RedisDict leverages Redis for efficient key-value storage, enabling high-performance data management.
48
58
 
49
- ```python
50
- from redis_dict import RedisDict
59
+ ```bash
60
+ pip install redis-dict
61
+ ```
51
62
 
52
- dic = RedisDict()
53
- dic['foo'] = 42
54
- print(dic['foo']) # Output: 42
55
- print('foo' in dic) # Output: True
56
- dic["baz"] = "hello world"
57
- print(dic) # Output: {'foo': 42, 'baz': 'hello world'}
63
+ ```python
64
+ >>> from redis_dict import RedisDict
65
+ >>> dic = RedisDict()
66
+ >>> dic['foo'] = 42
67
+ >>> dic['foo']
68
+ 42
69
+ >>> 'foo' in dic
70
+ True
71
+ >>> dic["baz"] = "hello world"
72
+ >>> dic
73
+ {'foo': 42, 'baz': 'hello world'}
58
74
  ```
59
75
  In Redis our example looks like this.
60
76
  ```
@@ -67,12 +83,12 @@ In Redis our example looks like this.
67
83
  "str:hello world"
68
84
  ```
69
85
 
86
+
70
87
  ### Namespaces
71
- Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization accross systems and projects with the same redis instance.
88
+ Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization across systems and projects with the same redis instance.
72
89
 
73
90
  This approach also minimizes the risk of key collisions between different applications, preventing hard-to-debug issues. By leveraging namespaces, RedisDict ensures a cleaner and more maintainable data management experience for developers working on multiple projects.
74
91
 
75
-
76
92
  ## Advanced Features
77
93
 
78
94
  ### Expiration
@@ -105,6 +121,8 @@ with dic.expire_at(seconds):
105
121
  3. Updating keys while preserving the initial timeout In certain situations, there is a need to update the value while keeping the expiration intact. This is achievable by setting the 'preserve_expiration' to true.
106
122
 
107
123
  ```python
124
+ import time
125
+
108
126
  dic = RedisDict(expire=10, preserve_expiration=True)
109
127
  dic['gone'] = 'in ten seconds'
110
128
 
@@ -117,6 +135,7 @@ dic['gone'] = 'gone in 5 seconds'
117
135
  Efficiently batch your requests using the Pipeline feature, which can be easily utilized with a context manager.
118
136
 
119
137
  ```python
138
+ from redis_dict import RedisDict
120
139
  dic = RedisDict(namespace="example")
121
140
 
122
141
  # one round trip to redis
@@ -146,13 +165,14 @@ print(dic["foo"]) # outputs "bar"
146
165
  ### Caching made simple
147
166
  ```python
148
167
  import time
168
+ from datetime import timedelta
149
169
  from redis_dict import RedisDict
150
170
 
151
171
  def expensive_function(x):
152
- time.sleep(2)
172
+ time.sleep(x)
153
173
  return x * 2
154
174
 
155
- cache = RedisDict(namespace="cache", expire=10)
175
+ cache = RedisDict(namespace="cache", expire=timedelta(minutes=60))
156
176
 
157
177
  def cached_expensive_function(x):
158
178
  if x not in cache:
@@ -160,7 +180,7 @@ def cached_expensive_function(x):
160
180
  return cache[x]
161
181
 
162
182
  start_time = time.time()
163
- print(cached_expensive_function(5)) # Takes around 2 seconds to compute and caches the result.
183
+ print(cached_expensive_function(5)) # Takes around 5 seconds to compute and caches the result.
164
184
  print(f"Time taken: {time.time() - start_time:.2f} seconds")
165
185
 
166
186
  start_time = time.time()
@@ -180,7 +200,7 @@ dic["name"] = "John Doe"
180
200
  dic["age"] = 32
181
201
  dic["city"] = "Amsterdam"
182
202
 
183
- # Get value by key
203
+ # Get value by key, from any instance connected to the same redis/namespace
184
204
  print(dic["name"]) # Output: John Doe
185
205
 
186
206
  # Update value by key, got a year older
@@ -230,13 +250,64 @@ print(dic["d"]) # Output: 4
230
250
  ```
231
251
 
232
252
  ### Additional Examples
233
- For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py.py#L1).
253
+ For more advanced examples of RedisDict, please refer to the unit-test files in the repository. All features and functionalities are thoroughly tested in [unit tests (here)](https://github.com/Attumm/redis-dict/blob/main/tests.py#L1) Or take a look at load test for batching [load test](https://github.com/Attumm/redis-dict/blob/main/load_test.py#L1).
234
254
  The unit-tests can be as used as a starting point.
235
255
 
256
+ ### Extending Types
257
+
258
+ ## Extending RedisDict with Custom Types
259
+
260
+ RedisDict supports custom type serialization. Here's how to add a new type:
261
+
262
+
263
+ ```python
264
+ import json
265
+ from redis_dict import RedisDict
266
+
267
+ class Person:
268
+ def __init__(self, name, age):
269
+ self.name = name
270
+ self.age = age
271
+
272
+ def encode(self) -> str:
273
+ return json.dumps(self.__dict__)
274
+
275
+ @classmethod
276
+ def decode(cls, encoded_str: str) -> 'Person':
277
+ return cls(**json.loads(encoded_str))
278
+
279
+ redis_dict = RedisDict()
280
+
281
+ # Extend redis dict with the new type
282
+ redis_dict.extends_type(Person)
283
+
284
+ # RedisDict can now seamlessly handle Person instances.
285
+ person = Person(name="John", age=32)
286
+ redis_dict["person1"] = person
287
+
288
+ result = redis_dict["person1"]
289
+
290
+ assert result.name == person.name
291
+ assert result.age == person.age
292
+ ```
293
+
294
+ ```python
295
+ >>> from datetime import datetime
296
+ >>> redis_dict.extends_type(datetime, datetime.isoformat, datetime.fromisoformat)
297
+ >>> redis_dict["now"] = datetime.now()
298
+ >>> redis_dict
299
+ {'now': datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)}
300
+ >>> redis_dict["now"]
301
+ datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)
302
+ ```
303
+
304
+ For more information on [extending types](https://github.com/Attumm/redis-dict/blob/main/extend_types_tests.py).
236
305
  ### Redis Encryption
237
- Setup guide for configuring and utilizing encrypted Redis for redis-dict.
306
+ Setup guide for configuring and utilizing encrypted Redis TLS for redis-dict.
238
307
  [Setup guide](https://github.com/Attumm/redis-dict/blob/main/encrypted_redis.MD)
239
308
 
309
+ ### Redis Storage Encryption
310
+ For storing encrypted data values, it's possible to use extended types. Take a look at this [encrypted test](https://github.com/Attumm/redis-dict/blob/main/encrypt_tests.py).
240
311
 
241
312
  ### Tests
242
313
  The RedisDict library includes a comprehensive suite of tests that ensure its correctness and resilience. The test suite covers various data types, edge cases, and error handling scenarios. It also employs the Hypothesis library for property-based testing, which provides fuzz testing to evaluate the implementation
@@ -1,17 +1,101 @@
1
+ """
2
+ redis_dict.py
3
+
4
+ RedisDict is a Python library that provides a convenient and familiar interface for
5
+ interacting with Redis as if it were a Python dictionary. The simple yet powerful library
6
+ enables you to manage key-value pairs in Redis using native Python syntax of dictionary. It supports
7
+ various data types, including strings, integers, floats, booleans, lists, and dictionaries,
8
+ and includes additional utility functions for more complex use cases.
9
+
10
+ By leveraging Redis for efficient key-value storage, RedisDict allows for high-performance
11
+ data management and is particularly useful for handling large datasets that may exceed local
12
+ memory capacity.
13
+
14
+ ## Features
15
+
16
+ * **Dictionary-like interface**: Use familiar Python dictionary syntax to interact with Redis.
17
+ * **Data Type Support**: Comprehensive support for various data types, including strings,
18
+ integers, floats, booleans, lists, dictionaries, sets, and tuples.
19
+ * **Pipelining support**: Use pipelines for batch operations to improve performance.
20
+ * **Expiration Support**: Enables the setting of expiration times either globally or individually
21
+ per key, through the use of context managers.
22
+ * **Efficiency and Scalability**: RedisDict is designed for use with large datasets and is
23
+ optimized for efficiency. It retrieves only the data needed for a particular operation,
24
+ ensuring efficient memory usage and fast performance.
25
+ * **Namespace Management**: Provides simple and efficient namespace handling to help organize
26
+ and manage data in Redis, streamlining data access and manipulation.
27
+ * **Distributed Computing**: With its ability to seamlessly connect to other instances or
28
+ servers with access to the same Redis instance, RedisDict enables easy distributed computing.
29
+ * **Custom data types**: Add custom types and transformations to suit your specific needs.
30
+
31
+ New feature
32
+
33
+ Custom extendable Validity checks on keys, and values.to support redis-dict base exceptions with messages from
34
+ enabling detailed reporting on the reasons for specific validation failures. This refactor would allow users
35
+ to configure which validity checks to execute, integrate custom validation functions, and specify whether
36
+ to raise an error on validation failures or to drop the operation and log a warning.
37
+
38
+ For example, in a caching scenario, data should only be cached if it falls within defined minimum and
39
+ maximum size constraints. This approach enables straightforward dictionary set operations while ensuring
40
+ that only meaningful data is cached: values greater than 10 MB and less than 100 MB should be cached;
41
+ otherwise, they will be dropped.
42
+
43
+ >>> def my_custom_validity_check(value: str) -> None:
44
+ \"""
45
+ Validates the size of the input.
46
+
47
+ Args:
48
+ value (str): string to validate.
49
+
50
+ Raises:
51
+ RedisDictValidityException: If the length of the input is not within the allowed range.
52
+ \"""
53
+ min_size = 10 * 1024: # Minimum size: 10 KB
54
+ max_size = 10 * 1024 * 1024: # Maximum size: 10 MB
55
+ if len(value) < min_size
56
+ raise RedisDictValidityException(f"value is too small: {len(value)} bytes")
57
+ if len(value) > max_size
58
+ raise RedisDictValidityException(f"value is too large: {len(value)} bytes")
59
+
60
+ >>> cache = RedisDict(namespace='cache_valid_results_for_1_minute',
61
+ ... expire=60,
62
+ ... custom_valid_values_checks=[my_custom_validity_check], # extend with new valid check
63
+ ... validity_exception_suppress=True) # when value is invalid, don't store, and don't raise an exception.
64
+ >>> too_small = "too small to cache"
65
+ >>> cache["1234"] = too_small # Since the value is below 10kb, thus there is no reason to cache the value.
66
+ >>> cache.get("1234") is None
67
+ >>> True
68
+ """
1
69
  import json
70
+
2
71
  from datetime import timedelta
3
72
  from typing import Any, Callable, Dict, Iterator, Set, List, Tuple, Union, Optional
4
- from redis import StrictRedis
5
-
6
73
  from contextlib import contextmanager
7
74
 
75
+ from redis import StrictRedis
76
+
8
77
  SENTINEL = object()
9
78
 
10
- transform_type = Dict[str, Callable[[str], Any]]
11
- pre_transform_type = Dict[str, Callable[[Any], str]]
79
+ EncodeFuncType = Callable[[Any], str]
80
+ DecodeFuncType = Callable[[str], Any]
81
+
82
+ EncodeType = Dict[str, EncodeFuncType]
83
+ DecodeType = Dict[str, DecodeFuncType]
84
+
85
+
86
+ def _create_default_encode(custom_encode_method: str) -> EncodeFuncType:
87
+ def default_encode(obj: Any) -> str:
88
+ return getattr(obj, custom_encode_method)() # type: ignore[no-any-return]
89
+ return default_encode
90
+
91
+
92
+ def _create_default_decode(cls: type, custom_decode_method: str) -> DecodeFuncType:
93
+ def default_decode(encoded_str: str) -> Any:
94
+ return getattr(cls, custom_decode_method)(encoded_str)
95
+ return default_decode
12
96
 
13
97
 
14
- def _transform_tuple(val: str) -> Tuple[Any, ...]:
98
+ def _decode_tuple(val: str) -> Tuple[Any, ...]:
15
99
  """
16
100
  Deserialize a JSON-formatted string to a tuple.
17
101
 
@@ -27,7 +111,7 @@ def _transform_tuple(val: str) -> Tuple[Any, ...]:
27
111
  return tuple(json.loads(val))
28
112
 
29
113
 
30
- def _pre_transform_tuple(val: Tuple[Any, ...]) -> str:
114
+ def _encode_tuple(val: Tuple[Any, ...]) -> str:
31
115
  """
32
116
  Serialize a tuple to a JSON-formatted string.
33
117
 
@@ -43,7 +127,7 @@ def _pre_transform_tuple(val: Tuple[Any, ...]) -> str:
43
127
  return json.dumps(list(val))
44
128
 
45
129
 
46
- def _transform_set(val: str) -> Set[Any]:
130
+ def _decode_set(val: str) -> Set[Any]:
47
131
  """
48
132
  Deserialize a JSON-formatted string to a set.
49
133
 
@@ -59,7 +143,7 @@ def _transform_set(val: str) -> Set[Any]:
59
143
  return set(json.loads(val))
60
144
 
61
145
 
62
- def _pre_transform_set(val: Set[Any]) -> str:
146
+ def _encode_set(val: Set[Any]) -> str:
63
147
  """
64
148
  Serialize a set to a JSON-formatted string.
65
149
 
@@ -75,6 +159,7 @@ def _pre_transform_set(val: Set[Any]) -> str:
75
159
  return json.dumps(list(val))
76
160
 
77
161
 
162
+ # pylint: disable=R0902, R0904
78
163
  class RedisDict:
79
164
  """
80
165
  A Redis-backed dictionary-like data structure with support for advanced features, such as
@@ -92,15 +177,20 @@ class RedisDict:
92
177
  It aims to offer a seamless and familiar interface for developers familiar with Python dictionaries,
93
178
  enabling a smooth transition to a Redis-backed data store.
94
179
 
180
+ Extendable Types: You can extend RedisDict by adding or overriding encoding and decoding functions.
181
+ This functionality enables various use cases, such as managing encrypted data in Redis,
182
+ To implement this, simply create and register your custom encoding and decoding functions.
183
+ By delegating serialization to redis-dict, reduce complexity and have simple code in the codebase.
184
+
95
185
  Attributes:
96
- transform (Dict[str, Callable[[str], Any]]): A dictionary of data type transformation functions for loading data.
97
- pre_transform (Dict[str, Callable[[Any], str]]): A dictionary of data type transformation functions for storing data.
186
+ decoding_registry (Dict[str, DecodeFuncType]): Mapping of decoding transformation functions based on type
187
+ encoding_registry (Dict[str, EncodeFuncType]): Mapping of encoding transformation functions based on type
98
188
  namespace (str): A string used as a prefix for Redis keys to separate data in different namespaces.
99
189
  expire (Union[int, None]): An optional expiration time for keys, in seconds.
100
190
 
101
191
  """
102
192
 
103
- transform: transform_type = {
193
+ decoding_registry: DecodeType = {
104
194
  type('').__name__: str,
105
195
  type(1).__name__: int,
106
196
  type(0.1).__name__: float,
@@ -109,15 +199,15 @@ class RedisDict:
109
199
 
110
200
  "list": json.loads,
111
201
  "dict": json.loads,
112
- "tuple": _transform_tuple,
113
- type(set()).__name__: _transform_set,
202
+ "tuple": _decode_tuple,
203
+ type(set()).__name__: _decode_set,
114
204
  }
115
205
 
116
- pre_transform: pre_transform_type = {
206
+ encoding_registry: EncodeType = {
117
207
  "list": json.dumps,
118
208
  "dict": json.dumps,
119
- "tuple": _pre_transform_tuple,
120
- type(set()).__name__: _pre_transform_set,
209
+ "tuple": _encode_tuple,
210
+ type(set()).__name__: _encode_set,
121
211
  }
122
212
 
123
213
  def __init__(self,
@@ -131,16 +221,23 @@ class RedisDict:
131
221
  Args:
132
222
  namespace (str, optional): A prefix for keys stored in Redis.
133
223
  expire (int, timedelta, optional): Expiration time for keys in seconds.
134
- preserve_expiration (bool, optional): Whether or not to preserve the expiration.
224
+ preserve_expiration (bool, optional): Preserve the expiration count when the key is updated.
135
225
  **redis_kwargs: Additional keyword arguments passed to StrictRedis.
136
226
  """
137
- self.temp_redis: Optional[StrictRedis[Any]] = None
227
+
138
228
  self.namespace: str = namespace
139
229
  self.expire: Union[int, timedelta, None] = expire
140
230
  self.preserve_expiration: Optional[bool] = preserve_expiration
141
231
  self.redis: StrictRedis[Any] = StrictRedis(decode_responses=True, **redis_kwargs)
142
232
  self.get_redis: StrictRedis[Any] = self.redis
143
233
 
234
+ self.custom_encode_method = "encode"
235
+ self.custom_decode_method = "decode"
236
+
237
+ self._iter: Iterator[str] = iter([])
238
+ self._max_string_size: int = 500 * 1024 * 1024 # 500mb
239
+ self._temp_redis: Optional[StrictRedis[Any]] = None
240
+
144
241
  def _format_key(self, key: str) -> str:
145
242
  """
146
243
  Format a key with the namespace prefix.
@@ -151,7 +248,7 @@ class RedisDict:
151
248
  Returns:
152
249
  str: The formatted key with the namespace prefix.
153
250
  """
154
- return '{}:{}'.format(self.namespace, str(key))
251
+ return f'{self.namespace}:{str(key)}'
155
252
 
156
253
  def _valid_input(self, val: Any, val_type: str) -> bool:
157
254
  """
@@ -169,7 +266,7 @@ class RedisDict:
169
266
  bool: True if the input value is valid, False otherwise.
170
267
  """
171
268
  if val_type == "str":
172
- return len(val) < (500 * 1024 * 1024)
269
+ return len(val) < self._max_string_size
173
270
  return True
174
271
 
175
272
  def _store(self, key: str, value: Any) -> None:
@@ -179,14 +276,21 @@ class RedisDict:
179
276
  Args:
180
277
  key (str): The key to store the value.
181
278
  value (Any): The value to be stored.
279
+
280
+ Note: Validity checks could be refactored to allow for custom exceptions that inherit from ValueError,
281
+ providing detailed information about why a specific validation failed.
282
+ This would enable users to specify which validity checks should be executed, add custom validity functions,
283
+ and choose whether to fail on validation errors, or drop the data and only issue a warning and continue.
284
+ Example use case is caching, to cache data only when it's between min and max sizes.
285
+ Allowing for simple dict set operation, but only cache data that makes sense.
286
+
182
287
  """
183
288
  store_type, key = type(value).__name__, str(key)
184
289
  if not self._valid_input(value, store_type) or not self._valid_input(key, "str"):
185
- # TODO When needed, make valid_input, pass the reason, or throw a exception.
186
290
  raise ValueError("Invalid input value or key size exceeded the maximum limit.")
187
- value = self.pre_transform.get(store_type, lambda x: x)(value) # type: ignore
291
+ value = self.encoding_registry.get(store_type, lambda x: x)(value) # type: ignore
188
292
 
189
- store_value = '{}:{}'.format(store_type, value)
293
+ store_value = f'{store_type}:{value}'
190
294
  formatted_key = self._format_key(key)
191
295
 
192
296
  if self.preserve_expiration and self.redis.exists(formatted_key):
@@ -207,8 +311,8 @@ class RedisDict:
207
311
  result = self.get_redis.get(self._format_key(key))
208
312
  if result is None:
209
313
  return False, None
210
- t, value = result.split(':', 1)
211
- return True, self.transform.get(t, lambda x: x)(value)
314
+ type_, value = result.split(':', 1)
315
+ return True, self.decoding_registry.get(type_, lambda x: x)(value)
212
316
 
213
317
  def _transform(self, result: str) -> Any:
214
318
  """
@@ -220,18 +324,118 @@ class RedisDict:
220
324
  Returns:
221
325
  Any: The transformed Python object.
222
326
  """
223
- t, value = result.split(':', 1)
224
- return self.transform.get(t, lambda x: x)(value)
327
+ type_, value = result.split(':', 1)
328
+ return self.decoding_registry.get(type_, lambda x: x)(value)
225
329
 
226
- def add_type(self, k: str, v: Callable[[str], Any]) -> None:
330
+ def new_type_compliance(
331
+ self,
332
+ class_type: type,
333
+ encode_method_name: Optional[str] = None,
334
+ decode_method_name: Optional[str] = None,
335
+ ) -> None:
227
336
  """
228
- Add a custom type to the transform mapping.
337
+ Checks if a class complies with the required encoding and decoding methods.
229
338
 
230
339
  Args:
231
- k (str): The key representing the type.
232
- v (Callable): The transformation function for the type.
340
+ class_type (type): The class to check for compliance.
341
+ encode_method_name (str, optional): Name of encoding method of the class for redis-dict custom types.
342
+ decode_method_name (str, optional): Name of decoding method of the class for redis-dict custom types.
343
+
344
+ Raises:
345
+ NotImplementedError: If the class does not implement the required methods when the respective check is True.
346
+ """
347
+ if encode_method_name is not None:
348
+ if not (hasattr(class_type, encode_method_name) and callable(
349
+ getattr(class_type, encode_method_name))):
350
+ raise NotImplementedError(
351
+ f"Class {class_type.__name__} does not implement the required {encode_method_name} method.")
352
+
353
+ if decode_method_name is not None:
354
+ if not (hasattr(class_type, decode_method_name) and callable(
355
+ getattr(class_type, decode_method_name))):
356
+ raise NotImplementedError(
357
+ f"Class {class_type.__name__} does not implement the required {decode_method_name} class method.")
358
+
359
+ def extends_type(
360
+ self,
361
+ class_type: type,
362
+ encode: Optional[EncodeFuncType] = None,
363
+ decode: Optional[DecodeFuncType] = None,
364
+ encoding_method_name: Optional[str] = None,
365
+ decoding_method_name: Optional[str] = None,
366
+ ) -> None:
367
+ """
368
+ Extends RedisDict to support a custom type in the encode/decode mapping.
369
+
370
+ This method enables serialization of instances based on their type,
371
+ allowing for custom types, specialized storage formats, and more.
372
+ There are three ways to add custom types:
373
+ 1. Have a class with an `encode` instance method and a `decode` class method.
374
+ 2. Have a class and pass encoding and decoding functions, where
375
+ `encode` converts the class instance to a string, and
376
+ `decode` takes the string and recreates the class instance.
377
+ 3. Have a class that already has serialization methods, that satisfies the:
378
+ EncodeFuncType = Callable[[Any], str]
379
+ DecodeFuncType = Callable[[str], Any]
380
+
381
+ `custom_encode_method`
382
+ `custom_decode_method` attributes.
383
+
384
+ Args:
385
+ class_type (Type[type]): The class `__name__` will become the key for the encoding and decoding functions.
386
+ encode (Optional[EncodeFuncType]): function that encodes an object into a storable string format.
387
+ This function should take an instance of `class_type` as input and return a string.
388
+ decode (Optional[DecodeFuncType]): function that decodes a string back into an object of `class_type`.
389
+ This function should take a string as input and return an instance of `class_type`.
390
+ encoding_method_name (str, optional): Name of encoding method of the class for redis-dict custom types.
391
+ decoding_method_name (str, optional): Name of decoding method of the class for redis-dict custom types.
392
+
393
+ If no encoding or decoding function is provided, default to use the `encode` and `decode` methods of the class.
394
+
395
+ The `encode` method should be an instance method that converts the object to a string.
396
+ The `decode` method should be a class method that takes a string and returns an instance of the class.
397
+
398
+ The method names for encoding and decoding can be changed by modifying the
399
+ - `custom_encode_method`
400
+ - `custom_decode_method`
401
+ attributes of the RedisDict instance
402
+
403
+ Example:
404
+ class Person:
405
+ def __init__(self, name, age):
406
+ self.name = name
407
+ self.age = age
408
+
409
+ def encode(self) -> str:
410
+ return json.dumps(self.__dict__)
411
+
412
+ @classmethod
413
+ def decode(cls, encoded_str: str) -> 'Person':
414
+ return cls(**json.loads(encoded_str))
415
+
416
+ redis_dict.extends_type(Person)
417
+
418
+ Note:
419
+ You can check for compliance of a class separately using the `new_type_compliance` method:
420
+
421
+ This method raises a NotImplementedError if either `encode` or `decode` is `None`
422
+ and the class does not implement the corresponding method.
233
423
  """
234
- self.transform[k] = v
424
+
425
+ if encode is None or decode is None:
426
+ encode_method_name = encoding_method_name or self.custom_encode_method
427
+ if encode is None:
428
+ self.new_type_compliance(class_type, encode_method_name=encode_method_name)
429
+ encode = _create_default_encode(encode_method_name)
430
+
431
+ if decode is None:
432
+ decode_method_name = decoding_method_name or self.custom_decode_method
433
+ self.new_type_compliance(class_type, decode_method_name=decode_method_name)
434
+ decode = _create_default_decode(class_type, decode_method_name)
435
+
436
+ type_name = class_type.__name__
437
+ self.decoding_registry[type_name] = decode
438
+ self.encoding_registry[type_name] = encode
235
439
 
236
440
  def __eq__(self, other: Any) -> bool:
237
441
  """
@@ -327,7 +531,7 @@ class RedisDict:
327
531
  Returns:
328
532
  Iterator[str]: An iterator over the keys of the RedisDict.
329
533
  """
330
- self.iter = self.iterkeys()
534
+ self._iter = self.iterkeys()
331
535
  return self
332
536
 
333
537
  def __repr__(self) -> str:
@@ -358,7 +562,7 @@ class RedisDict:
358
562
  Raises:
359
563
  StopIteration: If there are no more items.
360
564
  """
361
- return next(self.iter)
565
+ return next(self._iter)
362
566
 
363
567
  def next(self) -> str:
364
568
  """
@@ -370,7 +574,29 @@ class RedisDict:
370
574
  Raises:
371
575
  StopIteration: If there are no more items.
372
576
  """
373
- return self.__next__()
577
+ return next(self)
578
+
579
+ def _create_iter_query(self, search_term: str) -> str:
580
+ """
581
+ Create a Redis query string for iterating over keys based on the given search term.
582
+
583
+ This method constructs a query by prefixing the search term with the namespace
584
+ followed by a wildcard to facilitate scanning for keys that start with the
585
+ provided search term.
586
+
587
+ Args:
588
+ search_term (str): The term to search for in Redis keys.
589
+
590
+ Returns:
591
+ str: A formatted query string that can be used to find keys in Redis.
592
+
593
+ Example:
594
+ >>> d = RedisDict(namespace='foo')
595
+ >>> query = self._create_iter_query('bar')
596
+ >>> print(query)
597
+ 'foo:bar*'
598
+ """
599
+ return f'{self.namespace}:{search_term}*'
374
600
 
375
601
  def _scan_keys(self, search_term: str = '') -> Iterator[str]:
376
602
  """
@@ -382,7 +608,8 @@ class RedisDict:
382
608
  Returns:
383
609
  Iterator[str]: An iterator of matching Redis keys.
384
610
  """
385
- return self.get_redis.scan_iter(match='{}:{}{}'.format(self.namespace, search_term, '*'))
611
+ search_query = self._create_iter_query(search_term)
612
+ return self.get_redis.scan_iter(match=search_query)
386
613
 
387
614
  def get(self, key: str, default: Optional[Any] = None) -> Any:
388
615
  """
@@ -403,17 +630,18 @@ class RedisDict:
403
630
 
404
631
  def iterkeys(self) -> Iterator[str]:
405
632
  """
406
- Note: for pythone2 str is needed
633
+ Note: for python2 str is needed
407
634
  """
408
635
  to_rm = len(self.namespace) + 1
409
636
  return (str(item[to_rm:]) for item in self._scan_keys())
410
637
 
411
638
  def key(self, search_term: str = '') -> Optional[str]:
412
639
  """
413
- Note: for pythone2 str is needed
640
+ Note: for python2 str is needed
414
641
  """
415
642
  to_rm = len(self.namespace) + 1
416
- cursor, data = self.get_redis.scan(match='{}:{}{}'.format(self.namespace, search_term, '*'), count=1)
643
+ search_query = self._create_iter_query(search_term)
644
+ _, data = self.get_redis.scan(match=search_query, count=1)
417
645
  for item in data:
418
646
  return str(item[to_rm:])
419
647
 
@@ -435,7 +663,7 @@ class RedisDict:
435
663
  to_rm = len(self.namespace) + 1
436
664
  for item in self._scan_keys():
437
665
  try:
438
- yield (str(item[to_rm:]), self[item[to_rm:]])
666
+ yield str(item[to_rm:]), self[item[to_rm:]]
439
667
  except KeyError:
440
668
  pass
441
669
 
@@ -488,11 +716,12 @@ class RedisDict:
488
716
  Redis pipelining is employed to group multiple commands into a single request, minimizing
489
717
  network round-trip time, latency, and I/O load, thereby enhancing the overall performance.
490
718
 
491
- It is important to highlight that the clear method can be safely executed within the context of an initiated pipeline operation
719
+ It is important to highlight that the clear method can be safely executed within
720
+ the context of an initiated pipeline operation.
492
721
  """
493
722
  with self.pipeline():
494
723
  for key in self:
495
- del (self[key])
724
+ del self[key]
496
725
 
497
726
  def pop(self, key: str, default: Union[Any, object] = SENTINEL) -> Any:
498
727
  """
@@ -516,7 +745,7 @@ class RedisDict:
516
745
  return default
517
746
  raise
518
747
 
519
- del (self[key])
748
+ del self[key]
520
749
  return value
521
750
 
522
751
  def popitem(self) -> Tuple[str, Any]:
@@ -546,7 +775,7 @@ class RedisDict:
546
775
 
547
776
  Args:
548
777
  key (str): The key to retrieve the value.
549
- default (Optional[Any], optional): The value to set if the key is not found.
778
+ default_value (Optional[Any], optional): The value to set if the key is not found.
550
779
 
551
780
  Returns:
552
781
  Any: The value associated with the key or the default value.
@@ -575,7 +804,7 @@ class RedisDict:
575
804
  Update the RedisDict with key-value pairs from the given mapping, analogous to a dictionary's update method.
576
805
 
577
806
  Args:
578
- other (Mapping[str, Any]): A mapping containing key-value pairs to update the RedisDict.
807
+ dic (Mapping[str, Any]): A mapping containing key-value pairs to update the RedisDict.
579
808
  """
580
809
  with self.pipeline():
581
810
  for key, value in dic.items():
@@ -594,7 +823,8 @@ class RedisDict:
594
823
  value (Optional[Any], optional): The value to be assigned to each key in the RedisDict. Defaults to None.
595
824
 
596
825
  Returns:
597
- RedisDict: The current RedisDict instance, now populated with the keys from the iterable and their corresponding values.
826
+ RedisDict: The current RedisDict instance,populated with the keys from the iterable and their
827
+ corresponding values.
598
828
  """
599
829
  for key in iterable:
600
830
  self[key] = value
@@ -640,7 +870,7 @@ class RedisDict:
640
870
  Args:
641
871
  iterable (List[str]): A list of keys representing the chain.
642
872
  """
643
- return self.__delitem__(':'.join(iterable))
873
+ del self[':'.join(iterable)]
644
874
 
645
875
  # def expire_at(self, sec_epoch: int | timedelta) -> Iterator[None]:
646
876
  # compatibility with Python 3.9 typing
@@ -668,13 +898,13 @@ class RedisDict:
668
898
  ContextManager: A context manager to create a Redis pipeline batching all operations within the context.
669
899
  """
670
900
  top_level = False
671
- if self.temp_redis is None:
672
- self.redis, self.temp_redis, top_level = self.redis.pipeline(), self.redis, True
901
+ if self._temp_redis is None:
902
+ self.redis, self._temp_redis, top_level = self.redis.pipeline(), self.redis, True
673
903
  try:
674
904
  yield
675
905
  finally:
676
906
  if top_level:
677
- _, self.temp_redis, self.redis = self.redis.execute(), None, self.temp_redis # type: ignore
907
+ _, self._temp_redis, self.redis = self.redis.execute(), None, self._temp_redis # type: ignore
678
908
 
679
909
  def multi_get(self, key: str) -> List[Any]:
680
910
  """
@@ -717,7 +947,9 @@ class RedisDict:
717
947
  if len(keys) == 0:
718
948
  return {}
719
949
  to_rm = keys[0].rfind(':') + 1
720
- return dict(zip([i[to_rm:] for i in keys], (self._transform(i) for i in self.redis.mget(keys) if i is not None)))
950
+ return dict(
951
+ zip([i[to_rm:] for i in keys], (self._transform(i) for i in self.redis.mget(keys) if i is not None))
952
+ )
721
953
 
722
954
  def multi_del(self, key: str) -> int:
723
955
  """
@@ -746,13 +978,13 @@ class RedisDict:
746
978
  def get_ttl(self, key: str) -> Optional[int]:
747
979
  """
748
980
  Get the Time To Live (TTL) in seconds for a given key. If the key does not exist or does not have an
749
- associated expire, return None.
981
+ associated `expire`, return None.
750
982
 
751
983
  Args:
752
984
  key (str): The key for which to get the TTL.
753
985
 
754
986
  Returns:
755
- Optional[int]: The TTL in seconds if the key exists and has an expire set; otherwise, None.
987
+ Optional[int]: The TTL in seconds if the key exists and has an expiry set; otherwise, None.
756
988
  """
757
989
  val = self.redis.ttl(self._format_key(key))
758
990
  if val < 0:
@@ -17,20 +17,29 @@ setup(
17
17
  long_description=long_description,
18
18
  long_description_content_type='text/markdown',
19
19
 
20
- version='2.5.0',
20
+ version='2.6.0',
21
21
  py_modules=['redis_dict'],
22
22
  install_requires=['redis',],
23
23
  license='MIT',
24
24
 
25
+ platforms=['any'],
26
+
25
27
  url='https://github.com/Attumm/redisdict',
26
28
 
27
29
  classifiers=[
28
30
  'Development Status :: 5 - Production/Stable',
29
31
 
30
32
  'Intended Audience :: Developers',
33
+ 'Intended Audience :: Information Technology',
34
+ 'Intended Audience :: Science/Research',
31
35
 
36
+ 'Topic :: Internet',
37
+ 'Topic :: Scientific/Engineering',
32
38
  'Topic :: Database',
33
39
  'Topic :: System :: Distributed Computing',
40
+ 'Topic :: Software Development :: Libraries :: Python Modules',
41
+ 'Topic :: Software Development :: Object Brokering',
42
+ 'Topic :: Database :: Database Engines/Servers',
34
43
 
35
44
  'License :: OSI Approved :: MIT License',
36
45
 
@@ -43,5 +52,5 @@ setup(
43
52
  'Programming Language :: Python :: 3.11',
44
53
  'Programming Language :: Python :: 3.12',
45
54
  ],
55
+ keywords='redis python dictionary dict key-value key:value database caching distributed-computing dictionary-interface large-datasets scientific-computing data-persistence high-performance scalable pipelining batching big-data data-types distributed-algorithms encryption data-management',
46
56
  )
47
-
File without changes
File without changes