persidict 0.2.5__tar.gz → 0.3.1__tar.gz

{persidict-0.2.5 → persidict-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: persidict
-Version: 0.2.5
+Version: 0.3.1
 Summary: Simple persistent key-value store for Python. Values are stored as files on a disk or as S3 objects on AWS cloud.
 Home-page: https://github.com/vladlpavlov/persidict
 Author: Vlad (Volodymyr) Pavlov
@@ -18,6 +18,14 @@ Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: lz4
+Requires-Dist: joblib
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: jsonpickle
+Requires-Dist: boto3
+Requires-Dist: moto
+Requires-Dist: pytest
 
 # persidict
 
@@ -143,8 +151,8 @@ that simultaneously work with the same instance of a dictionary.
 * Values must be pickleable Python objects.
 * Insertion order is not preserved.
 * You can not assign initial key-value pairs to a dictionary in its constructor.
-* Methods `delete_if_exists()`, `mtimestamp()`, `get_subdict()` and `subdicts()` 
-are available
+* `PersiDict` API has methods = `delete_if_exists()`, `mtimestamp()`,
+`get_subdict()` and `subdicts()` , which are not available in Python dicts.
 
 ## Fine Tuning
 
@@ -152,13 +160,13 @@ are available
 to impact behaviour of a dictionary. 
 
 * `file_type` - a string that specifies the type of files used to store objects.
-Possible values are "json" and "pkl". Default value is "pkl". 
+Possible values are "json" and "lz4". Default value is "lz4". 
 Storing objects as JSON files is mostly supported for debugging purposes.
 * `immutable_items` - a boolean that specifies whether items in a dictionary 
 can be modified/deleted. It enables various distributed cache optimizations 
 for remote storage. True means an append-only dictionary. 
 False means normal dict-like behaviour. The default value is False. 
-* `digest_len` - a length of a hash signature suffix which `persidict` 
+* `digest_len` - a length of a hash signature suffix which `PersiDict` 
 automatically adds to each string in a key while mapping the key to 
 the address of a value in a persistent storage backend 
 (e.g. a filename or an S3 objectname). It is needed to ensure correct work
@@ -179,6 +187,7 @@ Binary installers for the latest released version are available at the Python pa
 ## Dependencies
 
 * [jsonpickle](https://jsonpickle.github.io)
+* [joblib](https://joblib.readthedocs.io)
 * [pandas](https://pandas.pydata.org)
 * [numpy](https://numpy.org)
 * [boto3](https://boto3.readthedocs.io)

{persidict-0.2.5 → persidict-0.3.1}/README.md

@@ -122,8 +122,8 @@ that simultaneously work with the same instance of a dictionary.
 * Values must be pickleable Python objects.
 * Insertion order is not preserved.
 * You can not assign initial key-value pairs to a dictionary in its constructor.
-* Methods `delete_if_exists()`, `mtimestamp()`, `get_subdict()` and `subdicts()` 
-are available
+* `PersiDict` API has methods = `delete_if_exists()`, `mtimestamp()`,
+`get_subdict()` and `subdicts()` , which are not available in Python dicts.
 
 ## Fine Tuning
 
@@ -131,13 +131,13 @@ are available
 to impact behaviour of a dictionary. 
 
 * `file_type` - a string that specifies the type of files used to store objects.
-Possible values are "json" and "pkl". Default value is "pkl". 
+Possible values are "json" and "lz4". Default value is "lz4". 
 Storing objects as JSON files is mostly supported for debugging purposes.
 * `immutable_items` - a boolean that specifies whether items in a dictionary 
 can be modified/deleted. It enables various distributed cache optimizations 
 for remote storage. True means an append-only dictionary. 
 False means normal dict-like behaviour. The default value is False. 
-* `digest_len` - a length of a hash signature suffix which `persidict` 
+* `digest_len` - a length of a hash signature suffix which `PersiDict` 
 automatically adds to each string in a key while mapping the key to 
 the address of a value in a persistent storage backend 
 (e.g. a filename or an S3 objectname). It is needed to ensure correct work
@@ -158,6 +158,7 @@ Binary installers for the latest released version are available at the Python pa
 ## Dependencies
 
 * [jsonpickle](https://jsonpickle.github.io)
+* [joblib](https://joblib.readthedocs.io)
 * [pandas](https://pandas.pydata.org)
 * [numpy](https://numpy.org)
 * [boto3](https://boto3.readthedocs.io)

{persidict-0.2.5 → persidict-0.3.1}/persidict/file_dir_dict.py

@@ -9,10 +9,9 @@ as a pickle or a json object in the file.
 from __future__ import annotations
 
 import os
-import pickle
 from typing import Any
 
-
+import joblib
 import jsonpickle
 import jsonpickle.ext.numpy as jsonpickle_numpy
 import jsonpickle.ext.pandas as jsonpickle_pandas
@@ -39,7 +38,7 @@ class FileDirDict(PersiDict):
 
     def __init__(self
                  , dir_name: str = "FileDirDict"
-                 , file_type: str = "pkl"
+                 , file_type: str = "lz4"
                  , immutable_items:bool = False
                  , digest_len:int = 8):
         """A constructor defines location of the store and file format to use.
@@ -47,7 +46,7 @@ class FileDirDict(PersiDict):
         dir_name is a directory that will contain all the files in
         the FileDirDict. If the directory does not exist, it will be created.
 
-        file_type can take one of two values: "pkl" or "json".
+        file_type can take one of two values: "lz4" or "json".
         It defines which file format will be used by FileDirDict
         to store values.
         """
@@ -57,8 +56,8 @@ class FileDirDict(PersiDict):
 
         self.file_type = file_type
 
-        assert file_type in {"json", "pkl"}, (
-            "file_type must be either pkl or json")
+        assert file_type in {"json", "lz4"}, (
+            "file_type must be either lz4 or json")
         assert not os.path.isfile(dir_name)
         if not os.path.isdir(dir_name):
             os.mkdir(dir_name)
@@ -144,27 +143,27 @@ class FileDirDict(PersiDict):
     def _read_from_file(self, file_name:str) -> Any:
         """Read a value from a file. """
 
-        if self.file_type == "pkl":
+        if self.file_type == "lz4":
             with open(file_name, 'rb') as f:
-                result = pickle.load(f)
+                result = joblib.load(f)
         elif self.file_type == "json":
             with open(file_name, 'r') as f:
                 result = jsonpickle.loads(f.read())
         else:
-            raise ValueError("file_type must be either pkl or json")
+            raise ValueError("file_type must be either lz4 or json")
         return result
 
     def _save_to_file(self, file_name:str, value:Any) -> None:
         """Save a value to a file. """
 
-        if self.file_type == "pkl":
+        if self.file_type == "lz4":
             with open(file_name, 'wb') as f:
-                pickle.dump(value, f)
+                joblib.dump(value, f, compress='lz4')
         elif self.file_type == "json":
             with open(file_name, 'w') as f:
                 f.write(jsonpickle.dumps(value, indent=4))
         else:
-            raise ValueError("file_type must be either pkl or json")
+            raise ValueError("file_type must be either lz4 or json")
 
     def __contains__(self, key:PersiDictKey) -> bool:
         """True if the dictionary has the specified key, else False. """

{persidict-0.2.5 → persidict-0.3.1}/persidict/safe_str_tuple_signing.py

@@ -27,7 +27,7 @@ def _create_signature_suffix(input_str:str, digest_len:int) -> str:
     input_b = input_str.encode()
     hash_object = hashlib.md5(input_b)
     full_digest_str = base64.b32encode(hash_object.digest()).decode()
-    suffix = "_" + full_digest_str[:digest_len]
+    suffix = "_" + full_digest_str[:digest_len].lower()
 
     return suffix
 

{persidict-0.2.5 → persidict-0.3.1}/persidict.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: persidict
-Version: 0.2.5
+Version: 0.3.1
 Summary: Simple persistent key-value store for Python. Values are stored as files on a disk or as S3 objects on AWS cloud.
 Home-page: https://github.com/vladlpavlov/persidict
 Author: Vlad (Volodymyr) Pavlov
@@ -18,6 +18,14 @@ Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: lz4
+Requires-Dist: joblib
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: jsonpickle
+Requires-Dist: boto3
+Requires-Dist: moto
+Requires-Dist: pytest
 
 # persidict
 
@@ -143,8 +151,8 @@ that simultaneously work with the same instance of a dictionary.
 * Values must be pickleable Python objects.
 * Insertion order is not preserved.
 * You can not assign initial key-value pairs to a dictionary in its constructor.
-* Methods `delete_if_exists()`, `mtimestamp()`, `get_subdict()` and `subdicts()` 
-are available
+* `PersiDict` API has methods = `delete_if_exists()`, `mtimestamp()`,
+`get_subdict()` and `subdicts()` , which are not available in Python dicts.
 
 ## Fine Tuning
 
@@ -152,13 +160,13 @@ are available
 to impact behaviour of a dictionary. 
 
 * `file_type` - a string that specifies the type of files used to store objects.
-Possible values are "json" and "pkl". Default value is "pkl". 
+Possible values are "json" and "lz4". Default value is "lz4". 
 Storing objects as JSON files is mostly supported for debugging purposes.
 * `immutable_items` - a boolean that specifies whether items in a dictionary 
 can be modified/deleted. It enables various distributed cache optimizations 
 for remote storage. True means an append-only dictionary. 
 False means normal dict-like behaviour. The default value is False. 
-* `digest_len` - a length of a hash signature suffix which `persidict` 
+* `digest_len` - a length of a hash signature suffix which `PersiDict` 
 automatically adds to each string in a key while mapping the key to 
 the address of a value in a persistent storage backend 
 (e.g. a filename or an S3 objectname). It is needed to ensure correct work
@@ -179,6 +187,7 @@ Binary installers for the latest released version are available at the Python pa
 ## Dependencies
 
 * [jsonpickle](https://jsonpickle.github.io)
+* [joblib](https://joblib.readthedocs.io)
 * [pandas](https://pandas.pydata.org)
 * [numpy](https://numpy.org)
 * [boto3](https://boto3.readthedocs.io)

{persidict-0.2.5 → persidict-0.3.1}/persidict.egg-info/requires.txt

@@ -1,3 +1,5 @@
+lz4
+joblib
 numpy
 pandas
 jsonpickle

{persidict-0.2.5 → persidict-0.3.1}/setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r") as f:
 
 setuptools.setup(
     name="persidict"
-    ,version="0.2.5"
+    ,version="0.3.1"
     ,author="Vlad (Volodymyr) Pavlov"
     ,author_email="vlpavlov@ieee.org"
     ,description= "Simple persistent key-value store for Python. " 
@@ -28,7 +28,9 @@ setuptools.setup(
     ,keywords='persistence, dicts, distributed, parallel'
     ,python_requires='>=3.8'
     ,install_requires=[
-        'numpy'
+        'lz4'
+        , 'joblib'
+        , 'numpy'
         , 'pandas'
         , 'jsonpickle'
         , 'boto3'