persidict 0.35.0__py3-none-any.whl → 0.36.1__py3-none-any.whl

persidict/file_dir_dict.py

@@ -10,6 +10,7 @@ serialized depending on ``file_type``.
 from __future__ import annotations
 
 import os
+import pathlib
 import random
 import tempfile
 import time
@@ -137,7 +138,8 @@ class FileDirDict(PersiDict):
         Returns:
             str: URL of the underlying storage in the form "file://<abs_path>".
         """
-        return f"file://{self._base_dir}"
+        return pathlib.Path(self._base_dir).as_uri()
+
 
 
     @property

persidict/jokers.py

@@ -32,7 +32,7 @@ class Joker(ParameterizableClass):
     Returns:
         Joker: The singleton instance for the subclass when instantiated.
     """
-    _instances = {}
+    _instances: dict[type, "Joker"] = {}
 
     def get_params(self) -> dict[str, Any]:
         """Return parameters for parameterizable API.

persidict/overlapping_multi_dict.py

@@ -1,3 +1,7 @@
+from __future__ import annotations
+
+from typing import Any, Dict, Type
+
 from .persi_dict import PersiDict
 
 class OverlappingMultiDict:
@@ -26,10 +30,10 @@ class OverlappingMultiDict:
         TypeError: If pickling is attempted or item access is used on the
             OverlappingMultiDict itself rather than its sub-dicts.
     """
-    def __init__(self
-                 , dict_type:type
-                 , shared_subdicts_params:dict
-                 , **individual_subdicts_params):
+    def __init__(self,
+                 dict_type: type[PersiDict],
+                 shared_subdicts_params: dict[str, Any],
+                 **individual_subdicts_params: dict[str, Any]) -> None:
         """Initialize the container and create sub-dictionaries.
 
         Args:

persidict-0.36.1.dist-info/METADATA

@@ -0,0 +1,227 @@
+Metadata-Version: 2.3
+Name: persidict
+Version: 0.36.1
+Summary: Simple persistent key-value store for Python. Values are stored as files on a disk or as S3 objects on AWS cloud.
+Keywords: persistence,dicts,distributed,parallel
+Author: Vlad (Volodymyr) Pavlov
+Author-email: Vlad (Volodymyr) Pavlov <vlpavlov@ieee.org>
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Dist: parameterizable
+Requires-Dist: lz4
+Requires-Dist: joblib
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: jsonpickle
+Requires-Dist: deepdiff
+Requires-Dist: boto3 ; extra == 'aws'
+Requires-Dist: boto3 ; extra == 'dev'
+Requires-Dist: moto ; extra == 'dev'
+Requires-Dist: pytest ; extra == 'dev'
+Requires-Python: >=3.10
+Project-URL: Homepage, https://github.com/pythagoras-dev/persidict
+Provides-Extra: aws
+Provides-Extra: dev
+Description-Content-Type: text/markdown
+
+# persidict
+
+Simple persistent dictionaries for distributed applications in Python.
+
+## 1. What Is It?
+
+**`persidict`** offers a simple persistent key-value store for Python. 
+It saves the content of the dictionary in a folder on a disk 
+or in an S3 bucket on AWS. Each value is stored as a separate file / S3 object.
+Only text strings or sequences of strings are allowed as keys.
+
+Unlike other persistent dictionaries (e.g. Python's native `shelve`), 
+`persidict` is designed for use in highly **distributed environments**, 
+where multiple instances of a program run concurrently across many machines,
+accessing the same dictionary via a shared storage.
+
+## 2. Usage
+
+### 2.1 Storing Data on a Local Disk
+
+The **`FileDirDict`** class saves your dictionary to a local folder. 
+Each key-value pair is stored as a separate file.
+
+```python
+from persidict import FileDirDict
+
+# Create a dictionary that will be stored in the "my_app_data" folder.
+# The folder will be created automatically if it doesn't exist.
+app_settings = FileDirDict(base_dir="my_app_data")
+
+# Add and update items just like a regular dictionary.
+app_settings["username"] = "alex"
+app_settings["theme"] = "dark"
+app_settings["notifications_enabled"] = True
+
+# Values can be any pickleable Python object.
+app_settings["recent_projects"] = ["project_a", "project_b"]
+
+print(f"Current theme is: {app_settings['theme']}")
+# >>> Current theme is: dark
+
+# The data persists!
+# If you run the script again or create a new dictionary object
+# pointing to the same folder, the data will be there.
+reloaded_settings = FileDirDict(base_dir="my_app_data")
+
+print(f"Number of settings: {len(reloaded_settings)}")
+# >>> Number of settings: 4
+
+print("username" in reloaded_settings)
+# >>> True
+```
+### 2.2 Storing Data in the Cloud (AWS S3)
+
+For distributed applications, you can use **`S3Dict`** to store data in 
+an AWS S3 bucket. The usage is identical, allowing you to switch 
+between local and cloud storage with minimal code changes.
+
+```python
+from persidict import S3Dict
+
+# Create a dictionary that will be stored in an S3 bucket.
+# The bucket will be created if it doesn't exist.
+cloud_config = S3Dict(bucket_name="my-app-config-bucket")
+
+# Use it just like a FileDirDict.
+cloud_config["api_key"] = "ABC-123-XYZ"
+cloud_config["timeout_seconds"] = 30
+
+print(f"API Key: {cloud_config['api_key']}")
+# >>> API Key: ABC-123-XYZ
+```
+
+
+## 3. Glossary
+
+### 3.1 Core Concepts
+
+* **`PersiDict`**: The abstract base class that defines the common interface 
+for all persistent dictionaries in the package. It's the foundation 
+upon which everything else is built.
+* **`PersiDictKey`**: A type hint that specifies what can be used
+as a key in any `PersiDict`. It can be a `SafeStrTuple`, 
+a single string, or a sequence of strings.
+* **`SafeStrTuple`**: The core data structure for keys. It's an immutable, 
+flat tuple of non-empty, URL/filename-safe strings, ensuring that 
+keys are consistent and safe for various storage backends.
+
+### 3.2 Main Implementations
+
+* **`FileDirDict`**: A primary, concrete implementation of `PersiDict` 
+that stores each key-value pair as a separate file in a local directory.
+* **`S3Dict`**: The other primary implementation of `PersiDict`, 
+which stores each key-value pair as an object in an AWS S3 bucket, 
+suitable for distributed environments.
+
+### 3.3 Key Parameters
+
+* **`file_type`**: A key parameter for `FileDirDict` and `S3Dict` that 
+determines the serialization format for values. 
+Common options are `"pkl"` (pickle) and `"json"`. 
+Any other value is treated as plain text for string storage.
+* **`base_class_for_values`**: An optional parameter for any `PersiDict` 
+that enforces type checking on all stored values, ensuring they are 
+instances of a specific class.
+* **`immutable_items`**: A boolean parameter that can make a `PersiDict` 
+"write-once," preventing any modification or deletion of existing items.
+* **`digest_len`**: An integer that specifies the length of a hash suffix 
+added to key components to prevent collisions on case-insensitive file systems.
+
+### 3.4 Advanced Classes
+
+* **`WriteOnceDict`**: A wrapper that enforces write-once behavior 
+on any `PersiDict`, ignoring subsequent writes to the same key.
+* **`OverlappingMultiDict`**: An advanced container that holds 
+multiple `PersiDict` instances sharing the same storage 
+but with different `file_type`s.
+
+### 3.5 Special "Joker" Values
+
+* **`Joker`**: The base class for special command-like values that 
+can be assigned to a key to trigger an action instead of storing a value.
+* **`KEEP_CURRENT`**: A "joker" value that, when assigned to a key, 
+ensures the existing value is not changed.
+* **`DELETE_CURRENT`**: A "joker" value that deletes the key-value pair 
+from the dictionary when assigned to a key.
+
+## 4. Comparison With Python Built-in Dictionaries
+
+### 4.1 Similarities 
+
+`PersiDict` and its subclasses can be used as regular Python dictionaries. 
+
+* You can use square brackets to get, set, or delete values. 
+* You can iterate over keys, values, or items. 
+* You can check if a key is in the dictionary. 
+* You can check whether two dicts are equal
+(meaning they contain the same key-value pairs).
+* You can get the length of the dictionary.
+* Methods `keys()`, `values()`, `items()`, `get()`, `clear()`
+, `setdefault()`, `update()` etc. work as expected.
+
+### 4.2 Differences 
+
+**`PersiDict`** and its subclasses persist values between program executions, 
+as well as make it possible to concurrently run programs 
+that simultaneously work with the same instance of a dictionary.
+
+* Keys must be sequences of URL/filename-safe non-empty strings.
+* Values must be pickleable Python objects.
+* You can constrain values to be an instance of a specific class.
+* Insertion order is not preserved.
+* You cannot assign initial key-value pairs to a dictionary in its constructor.
+* **`PersiDict`** API has additional methods `delete_if_exists()`, `timestamp()`,
+`get_subdict()`, `subdicts()`, `random_key()`, `newest_keys()`, 
+`oldest_keys()`, `newest_values()`, `oldest_values()`, and
+`get_params()`, which are not available in native Python dicts.
+* You can use `KEEP_CURRENT` constant as a fake new value 
+to avoid actually setting/updating a value. Or `DELETE_CURRENT` as 
+a fake new value to delete the previous value from a dictionary.
+
+## 5. How To Get It?
+
+The source code is hosted on GitHub at:
+[https://github.com/pythagoras-dev/persidict](https://github.com/pythagoras-dev/persidict) 
+
+Binary installers for the latest released version are available at the Python package index at:
+[https://pypi.org/project/persidict](https://pypi.org/project/persidict)
+
+### 5.1 Using uv :
+```
+uv add persidict
+```
+
+### 5.2 Using pip (legacy alternative to uv):
+```
+pip install persidict
+```
+
+## 6. Dependencies
+
+* [jsonpickle](https://jsonpickle.github.io)
+* [joblib](https://joblib.readthedocs.io)
+* [lz4](https://python-lz4.readthedocs.io)
+* [pandas](https://pandas.pydata.org)
+* [numpy](https://numpy.org)
+* [boto3](https://boto3.readthedocs.io)
+* [pytest](https://pytest.org)
+* [moto](http://getmoto.org)
+
+## 7. Key Contacts
+
+* [Vlad (Volodymyr) Pavlov](https://www.linkedin.com/in/vlpavlov/)

{persidict-0.35.0.dist-info → persidict-0.36.1.dist-info}/RECORD

@@ -1,13 +1,14 @@
+persidict/.DS_Store,sha256=1lFlJ5EFymdzGAUAaI30vcaaLHt3F1LwpG7xILf9jsM,6148
 persidict/__init__.py,sha256=CDOSJGgCnyRTkGUTzaeg3Cqsxwx0-0EFieOtldXwAls,1380
-persidict/file_dir_dict.py,sha256=gvCyk_kp_3AC-zkHuSj-0lM4hf_fBK6iz3ffGQ7jtvU,25757
-persidict/jokers.py,sha256=H2MzKllvgm7t2sjX3GaRNLngOiG2ohE-lv0B3g3J1SQ,2710
-persidict/overlapping_multi_dict.py,sha256=v1c2kf3Bhm2Dh6SGEsRV58hQI11YX9ZPxyrDV_1d5s8,5360
+persidict/file_dir_dict.py,sha256=IDRb6a3YQvM7Gf0jbqKkTi4VuSPecTw6Ca6HZ947Qj8,25784
+persidict/jokers.py,sha256=Ow4tWOTTMGKvolJyVuEF-oEgE_u3vDZtA9UFwTdhNV4,2731
+persidict/overlapping_multi_dict.py,sha256=gBiHaCb5pTGNW3ZrakgaiGDid6oCfoP7Vq1rxXGnFWg,5476
 persidict/persi_dict.py,sha256=DIMQaY4gE8NSYTlHlk9rfOJJEYUuLV8kmQ-gc474py4,20052
 persidict/s3_dict.py,sha256=VKDqY9sASffeXtfbavVWk8-umrioIG5Xq57Qqg1wPH4,21522
 persidict/safe_chars.py,sha256=9Qy24fu2dmiJOdmCF8mKZULfQaRp7H4oxfgDXeLgogI,1160
 persidict/safe_str_tuple.py,sha256=YBTcYjUKIffznOawXb9xKjz4HaKdklrgyVtegJFmr5w,7202
 persidict/safe_str_tuple_signing.py,sha256=RQAj4fnpRVaOe0KpwLler1UTaeNOgXCQpU3t80ixtxg,7493
 persidict/write_once_dict.py,sha256=-lPQ_yuU62pczHT0BYO6SFbiZBKFq8Tj9ln3jCzNDzA,11443
-persidict-0.35.0.dist-info/WHEEL,sha256=F3mArEuDT3LDFEqo9fCiUx6ISLN64aIhcGSiIwtu4r8,79
-persidict-0.35.0.dist-info/METADATA,sha256=oo_YL1_W4ux3lTJZvREQPcin1ILJmvxtMDlWuITUwCw,9262
-persidict-0.35.0.dist-info/RECORD,,
+persidict-0.36.1.dist-info/WHEEL,sha256=Pi5uDq5Fdo_Rr-HD5h9BiPn9Et29Y9Sh8NhcJNnFU1c,79
+persidict-0.36.1.dist-info/METADATA,sha256=v4CpLPlOZ-_aSi8ihs8c4g3ikECErmnhLH9iXsWw8go,8450
+persidict-0.36.1.dist-info/RECORD,,

{persidict-0.35.0.dist-info → persidict-0.36.1.dist-info}/WHEEL

@@ -1,4 +1,4 @@
 Wheel-Version: 1.0
-Generator: uv 0.8.16
+Generator: uv 0.8.17
 Root-Is-Purelib: true
 Tag: py3-none-any

persidict-0.35.0.dist-info/METADATA

@@ -1,228 +0,0 @@
-Metadata-Version: 2.3
-Name: persidict
-Version: 0.35.0
-Summary: Simple persistent key-value store for Python. Values are stored as files on a disk or as S3 objects on AWS cloud.
-Keywords: persistence,dicts,distributed,parallel
-Author: Vlad (Volodymyr) Pavlov
-Author-email: Vlad (Volodymyr) Pavlov <vlpavlov@ieee.org>
-License: MIT
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: OS Independent
-Classifier: Topic :: Software Development :: Libraries
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Requires-Dist: parameterizable
-Requires-Dist: lz4
-Requires-Dist: joblib
-Requires-Dist: numpy
-Requires-Dist: pandas
-Requires-Dist: jsonpickle
-Requires-Dist: deepdiff
-Requires-Dist: boto3 ; extra == 'aws'
-Requires-Dist: boto3 ; extra == 'dev'
-Requires-Dist: moto ; extra == 'dev'
-Requires-Dist: pytest ; extra == 'dev'
-Requires-Python: >=3.10
-Project-URL: Homepage, https://github.com/pythagoras-dev/persidict
-Provides-Extra: aws
-Provides-Extra: dev
-Description-Content-Type: text/markdown
-
-# persidict
-
-Simple persistent dictionaries for Python.
-
-## What Is It?
-
-`persidict` offers a very simple persistent key-value store for Python. 
-It saves the content of the dictionary in a folder on a disk 
-or in an S3 bucket on AWS. Each value is stored as a separate file / S3 object.
-Only text strings or sequences of strings are allowed as keys.
-
-Unlike other persistent dictionaries (e.g. Python's native `shelve`), 
-`persidict` is designed for use in highly **distributed environments**, 
-where multiple instances of a program run concurrently across many machines.
-
-## Usage
-Class `FileDirDict` is a persistent dictionary that stores its content 
-in a folder on a disk.
-
-    from persidict import FileDirDict    
-    my_dictionary = FileDirDict(base_dir="my_folder")
-
-Once created, it can be used as a regular Python dictionary 
-that stores key-value pairs. A key must be a sequence of strings, 
-a value can be any (pickleable) Python object:
-
-    my_dictionary["Eliza"] = "MIT Eliza was a mock psychotherapist."
-    my_dictionary["Eliza","year"] = 1965
-    my_dictionary["Eliza","authors"] = ["Joseph Weizenbaum"]
-    
-    my_dictionary["Shoebox"] = "IBM Shoebox performed arithmetic operations"
-    my_dictionary["Shoebox"] += " on voice commands."
-    my_dictionary["Shoebox", "year"] = 1961
-    my_dictionary["Shoebox", "authors"] = ["W.C. Dersch", "E.A. Quade"]
-
-    for k in my_dictionary:
-        print(list(k), "==>",  my_dictionary[k])
-
-    if not "Eliza" in my_dictionary:
-        print("Something is wrong")
-
-If you run the code above, it will produce the following output:
-
-    >>> ['Eliza'] ==> MIT Eliza was a mock psychotherapist.
-    >>> ['Shoebox'] ==> IBM Shoebox performed arithmetic operations on voice commands.
-    >>> ['Shoebox', 'authors'] ==> ['W.C. Dersch', 'E.A. Quade']
-    >>> ['Shoebox', 'year'] ==> 1961
-    >>> ['Eliza', 'authors'] ==> ['Joseph Weizenbaum']
-    >>> ['Eliza', 'year'] ==> 1965
-
-The dictionary automatically creates a folder named "my_folder" 
-on the local disk. Each key-value pair is stored as 
-a separate file within this folder.
-
-If the key is a string, it becomes the filename for the object. 
-If the key is a sequence of strings, all strings except the last 
-are used to create nested subfolders within the main folder. 
-The final string in the sequence serves as the filename for the object, 
-which is stored in the deepest subfolder.
-
-Persistent dictionaries only accept sequences of strings as keys. 
-Any pickleable Python object can be used as a value. 
-Unlike regular Python dictionaries, insertion order is not preserved.
-
-    del my_dictionary
-    new_dict = FileDirDict(base_dir="my_folder")
-    print("len(new_dict) == ",len(new_dict))
-
-The code above will create a new object named new_dict and then will
-print its length: 
-
-    >>> len(new_dict) == 6
-
-The length is 6, because the dictionary was already stored on a disk 
-in the "my_folder" directory, which contained 6 pickle files.
-
-Technically, `FileDirDict` saves its content in a folder on a local disk. 
-But you can share this folder with other machines 
-(for example, using Dropbox or NFS), and work with the same dictionary 
-simultaneously from multiple computers (from multiple instances of your program). 
-This approach would allow you to use a persistent dictionary in 
-a system that is distributed over dozens of computers.
-
-If you need to run your program on hundreds (or more) computers, 
-class `S3Dict` is a better choice: it's a persistent dictionary that 
-stores its content in an AWS S3 bucket.
-
-    from persidict import S3Dict
-    my_cloud_dictionary = S3Dict(bucket_name="my_bucket")
-
-Once created, it can be used as a regular Python dictionary.
-
-## Key Classes
-
-* `SafeStrTuple` - an immutable sequence of URL/filename-safe non-empty strings.
-* `PersiDict` - an abstract base class for persistent dictionaries. 
-* `FileDirDict` - a persistent dictionary that stores its content 
-in a folder on a disk.
-* `S3Dict` - a persistent dictionary that stores its content 
-in an AWS S3 bucket.
-
-## Key Similarities With Python Built-in Dictionaries
-
-`PersiDict` and its subclasses can be used as regular Python dictionaries. 
-
-* You can use square brackets to get, set, or delete values. 
-* You can iterate over keys, values, or items. 
-* You can check if a key is in the dictionary. 
-* You can check whether two dicts are equal
-(meaning they contain the same key-value pairs).
-* You can get the length of the dictionary.
-* Methods `keys()`, `values()`, `items()`, `get()`, `clear()`
-, `setdefault()`, `update()` etc. work as expected.
-
-## Key Differences From Python Built-in Dictionaries
-
-`PersiDict` and its subclasses persist values between program executions, 
-as well as make it possible to concurrently run programs 
-that simultaneously work with the same instance of a dictionary.
-
-* Keys must be sequences of URL/filename-safe non-empty strings.
-* Values must be pickleable Python objects.
-* You can constrain values to be an instance of a specific class.
-* Insertion order is not preserved.
-* You cannot assign initial key-value pairs to a dictionary in its constructor.
-* `PersiDict` API has additional methods `delete_if_exists()`, `timestamp()`,
-`get_subdict()`, `subdicts()`, `random_key()`, `newest_keys()`, 
-`oldest_keys()`, `newest_values()`, `oldest_values()`, and
-`get_params()`, which are not available in native Python dicts.
-* You can use KEEP_CURRENT constant as a fake new value 
-to avoid actually setting/updating a value. Or DELETE_CURRENT as 
-a fake new value to delete the previous value from a dictionary.
-
-## Fine Tuning
-
-`PersiDict` subclasses have a number of parameters that can be used 
-to impact behaviour of a dictionary. 
-
-* `base_class_for_values` - A base class for values stored in a dictionary.  
-If specified, it will be used to check types of values in the dictionary. 
-If not specified (if set to `None`), no type checking will be performed 
-and all types will be allowed.
-* `file_type` - a string that specifies the type of files used to store objects.
-If `file_type` has one of two values: "pkl" or "json", it defines 
-which file format will be used by the dictionary to store values. 
-For all other values of `file_type`, the file format will always be plain
-text. "pkl" or "json" allow to store arbitrary Python objects,
-while all other file_type-s only work with str objects; 
-it means `base_class_for_values` must be explicitly set to `str` 
-if `file_type` is not set to "pkl" or "json".
-* `immutable_items` - a boolean that specifies whether items in a dictionary 
-can be modified/deleted. It enables various distributed cache optimizations 
-for remote storage. True means an append-only dictionary. 
-False means normal dict-like behaviour. The default value is False. 
-* `digest_len` - a length of a hash signature suffix which `PersiDict` 
-automatically adds to each string in a key while mapping the key to 
-the address of a value in a persistent storage backend 
-(e.g. a filename or an S3 objectname). It is needed to ensure correct work
-of persistent dictionaries with case-insensitive (even if case-preserving) 
-filesystems, such as MacOS HFS. The default value is 8. 
-
-
-## How To Get It?
-
-The source code is hosted on GitHub at:
-[https://github.com/pythagoras-dev/persidict](https://github.com/pythagoras-dev/persidict) 
-
-Binary installers for the latest released version are available at the Python package index at:
-[https://pypi.org/project/persidict](https://pypi.org/project/persidict)
-
-Using uv :
-```
-uv add persidict
-```
-
-Using pip (legacy alternative to uv):
-```
-pip install persidict
-```
-
-## Dependencies
-
-* [jsonpickle](https://jsonpickle.github.io)
-* [joblib](https://joblib.readthedocs.io)
-* [lz4](https://python-lz4.readthedocs.io)
-* [pandas](https://pandas.pydata.org)
-* [numpy](https://numpy.org)
-* [boto3](https://boto3.readthedocs.io)
-* [pytest](https://pytest.org)
-* [moto](http://getmoto.org)
-
-## Key Contacts
-
-* [Vlad (Volodymyr) Pavlov](https://www.linkedin.com/in/vlpavlov/)