slowstore 0.1.0__tar.gz

slowstore-0.1.0/LICENSE

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) <year> <copyright holders>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

slowstore-0.1.0/PKG-INFO

@@ -0,0 +1,114 @@
+Metadata-Version: 2.1
+Name: slowstore
+Version: 0.1.0
+Summary: Simple filesystem backed key value storage for easy testing
+License: MIT
+Author: Costa Halicea
+Author-email: costa@codechem.com
+Requires-Python: >=3.12,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: pydantic (>=2.8.2,<3.0.0)
+Description-Content-Type: text/markdown
+
+# Slowstore
+
+This is a very simple single file single class key-value store written in Python. It is designed to be slow and inefficient, and is intended to be used for testing purposes only.
+The rules are simple:
+Every store has it's own directory, and every item is stored in a separate JSON file.
+
+That's it.
+
+It is easy to use and inspect the data being manipulated.
+
+## Installation
+
+You can install Slowstore using pip:
+
+```bash
+pip install slowstore
+```
+
+## Usage
+You can create a new Slowstore instance by calling the constructor:
+
+```python
+from slowstore import Slowstore
+
+class SampleModel(BaseModel):
+    name: str
+    age: int = 0
+
+# Create a new store and load the data 
+store = Slowstore[SampleModel](SampleModel, "mydata")).load()
+```
+
+**One feature that I use a lot is to commit on property change, so I can inspect exactly what is going on**
+
+```python
+store.save_on_change = True
+```
+you can also set this from the constructor
+
+```python
+store = Slowstore[SampleModel](SampleModel, "mydata", save_on_change=True).load()
+```
+After this code is executed, every time a property is changed, 
+the changes will be saved to disk immediately.
+
+This is slow, but it is useful for debugging and testing.
+
+
+### Adding items to the store
+
+```python
+s1 = store.upsert(SampleModel(name="Alice", age=30))
+s2 = store.upsert(SampleModel(name="Bob", age=25))
+s3 = store.upsert(SampleModel(name="Charlie", age=35))
+store.commit_all()
+
+s1.age = 32
+```
+
+### Commit the changes to the store
+
+```python
+store.commit(s1)
+store.commit(s2)
+# or just commit all changes
+store.commit_all() 
+```
+
+### Undo/Redo
+every slowstore item is a proxy object around what you've created. This allows you to access the object's attributes and adds undo/redo/dirty functionality
+
+```python
+s1.undo()
+s1.redo()
+```
+
+### Querying
+```python
+# get all items, returns an iterator
+all_items = store.all()
+
+# yield all items that match a condition
+filtered_items = store.filter(lambda x: x.age > 30)
+
+# get the first item that matches a condition
+first_item = store.first(lambda x: x.age > 30)
+
+# Check if some object or it's key is in the store
+if s2 in store:
+    print("s2 is in store")
+else:
+    print("s2 is not in store")
+
+```
+
+## License
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+
+

slowstore-0.1.0/README.md

@@ -0,0 +1,99 @@
+# Slowstore
+
+This is a very simple single file single class key-value store written in Python. It is designed to be slow and inefficient, and is intended to be used for testing purposes only.
+The rules are simple:
+Every store has it's own directory, and every item is stored in a separate JSON file.
+
+That's it.
+
+It is easy to use and inspect the data being manipulated.
+
+## Installation
+
+You can install Slowstore using pip:
+
+```bash
+pip install slowstore
+```
+
+## Usage
+You can create a new Slowstore instance by calling the constructor:
+
+```python
+from slowstore import Slowstore
+
+class SampleModel(BaseModel):
+    name: str
+    age: int = 0
+
+# Create a new store and load the data 
+store = Slowstore[SampleModel](SampleModel, "mydata")).load()
+```
+
+**One feature that I use a lot is to commit on property change, so I can inspect exactly what is going on**
+
+```python
+store.save_on_change = True
+```
+you can also set this from the constructor
+
+```python
+store = Slowstore[SampleModel](SampleModel, "mydata", save_on_change=True).load()
+```
+After this code is executed, every time a property is changed, 
+the changes will be saved to disk immediately.
+
+This is slow, but it is useful for debugging and testing.
+
+
+### Adding items to the store
+
+```python
+s1 = store.upsert(SampleModel(name="Alice", age=30))
+s2 = store.upsert(SampleModel(name="Bob", age=25))
+s3 = store.upsert(SampleModel(name="Charlie", age=35))
+store.commit_all()
+
+s1.age = 32
+```
+
+### Commit the changes to the store
+
+```python
+store.commit(s1)
+store.commit(s2)
+# or just commit all changes
+store.commit_all() 
+```
+
+### Undo/Redo
+every slowstore item is a proxy object around what you've created. This allows you to access the object's attributes and adds undo/redo/dirty functionality
+
+```python
+s1.undo()
+s1.redo()
+```
+
+### Querying
+```python
+# get all items, returns an iterator
+all_items = store.all()
+
+# yield all items that match a condition
+filtered_items = store.filter(lambda x: x.age > 30)
+
+# get the first item that matches a condition
+first_item = store.first(lambda x: x.age > 30)
+
+# Check if some object or it's key is in the store
+if s2 in store:
+    print("s2 is in store")
+else:
+    print("s2 is not in store")
+
+```
+
+## License
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+

slowstore-0.1.0/pyproject.toml

@@ -0,0 +1,16 @@
+[tool.poetry]
+name = "slowstore"
+version = "0.1.0"
+description = "Simple filesystem backed key value storage for easy testing"
+authors = ["Costa Halicea <costa@codechem.com>"]
+license = "MIT"
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = "^3.12"
+pydantic = "^2.8.2"
+
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

slowstore-0.1.0/slowstore/__init__.py

@@ -0,0 +1,312 @@
+import json
+from logging import getLogger as get_logger
+import os
+import shutil
+import sys
+import datetime
+from pydantic import BaseModel
+
+from typing import Any, Callable, Generic, List, Literal, TypeVar, cast
+
+T = TypeVar("T", bound=BaseModel)
+
+from logging import getLogger as get_logger
+
+logger = get_logger(__name__)
+
+
+def json_default(o):
+    if isinstance(o, (datetime.date, datetime.datetime)):
+        return o.isoformat()
+
+
+class Change(BaseModel):
+    """A property change that can be undone or redone on a model"""
+
+    key: str
+    prop_name: str
+    prev_val: Any
+    new_val: Any
+    date: datetime.datetime = datetime.datetime.now()
+
+    # Not supporting this yet, since i dont need it
+    transaction_id: str = ""
+
+    def __init__(self, key, prop_name, prev_val, new_val):
+        super().__init__(
+            key=key, prop_name=prop_name, prev_val=prev_val, new_val=new_val
+        )
+
+    def undo(self, model):
+        setattr(model, self.prop_name, self.prev_val)
+
+    def redo(self, model):
+        setattr(model, self.prop_name, self.new_val)
+
+
+__special_fields__ = [
+    "store",
+    "model",
+    "is_dirty",
+    "commit",
+    "__key__",
+    "__changes__",
+    "__add_change__",
+    "__reset__",
+]
+
+
+class ModelProxy(BaseModel, Generic[T]):
+    store: "Slowstore[T]"
+    model: T
+    is_dirty: bool = False
+    __changes__: List[Change] = []
+    __key__: str
+
+    def __init__(self, store: "Slowstore[T]", key: str, model: T):
+        self.model = model
+        self.store = store
+        self.is_dirty = False
+        self.__key__ = key
+        self.__changes__ = []
+
+    def __getattr__(self, name):
+        if name in __special_fields__:
+            return super().__getattribute__(name)
+        else:
+            return getattr(self.model, name)
+
+    def __setattr__(self, name, value):
+        if name == "model":
+            super().__setattr__(name, value)
+            self.is_dirty = False
+            self.__changes__ = []
+        elif name in __special_fields__:
+            super().__setattr__(name, value)
+        else:
+            prev = self.model.__getattribute__(name)
+            if prev == value:
+                return
+            setattr(self.model, name, value)
+            self.__add_change__(name, prev, value)
+            self.__changes__.insert(0, Change(self.__key__, name, prev, value))
+            self.is_dirty = True
+            if self.store.save_on_change:
+                self.commit()
+
+    def commit(self):
+        self.store.commit(self)
+
+    def __add_change__(self, prop_name: str, prev_val: Any, new_val: Any):
+        change = Change(self.__key__, prop_name, prev_val, new_val)
+        self.store.__changes__.append(change)
+        self.__changes__.append(change)
+
+    def __reset__(self, count: int = sys.maxsize):
+        size = len(self.__changes__)
+        while size > 0 and count > 0:
+            change = self.__changes__.pop(0)
+            count -= 1
+            size -= 1
+            change.undo(self.model)
+
+
+class Slowstore(Generic[T]):
+    """A simple key-value store that persists data to disk,
+    it uses a pydantic model as the value type
+    :param cls: The class
+    :type cls: type
+
+    :param directory: The directory where the store will be stored
+    :type directory: str
+
+    :param load_on_start: If the store should load all data on start
+    :type load_on_start: bool, Defaults to True
+
+    :param save_on_change: If the store should save data on change
+    :type save_on_change: bool, Defaults to False
+
+    :param save_on_exit: If the store should save data on exit
+    :type save_on_exit: bool, Defaults to True
+    """
+
+    directory: str
+    cls: type
+    save_on_change: bool = False
+    save_on_exit: bool = False
+    loaded: bool = False
+    __data__: dict[str, ModelProxy[T]]
+    __changes__: List[Change]
+
+    def __init__(self, cls: type, directory: str, **kwargs):
+        """Creates a new Slowstore instance"""
+        super().__init__()
+        self.__data__ = {}
+        self.directory = directory
+        self.cls = cls
+        # get the model type from the generic
+        if kwargs.get("load_on_start", True):
+            self.load()
+        self.save_on_change = kwargs.get("save_on_change", False)
+        self.save_on_exit = kwargs.get("save_on_exit", True)
+
+    def get(self, key) -> T:
+        """gets an object from the story"""
+        self.__ensure_loaded__()
+        return cast(T, self.__data__[key])
+
+    def upsert(self, key: str, value: T) -> T:
+        """sets a new object in the store and returns it's proxy
+        if we overwrite the key it will completely change the underlying model and this can cause data inconsistencies
+        """
+        self.__ensure_loaded__()
+        if key in self.__data__:
+            proxy = self.__data__[key]
+            if proxy.model != value:
+                self.__data__[key].model = value
+                proxy.is_dirty = True
+
+            if self.save_on_change:
+                self.commit(proxy)
+
+            return cast(T, proxy)
+
+        proxy = ModelProxy[T](self, key, value)
+        proxy.is_dirty = True
+
+        if self.save_on_change:
+            self.commit(proxy)
+
+        self.__data__[key] = proxy
+        return cast(T, proxy)
+
+    def delete(self, key: str) -> bool:
+        self.__ensure_loaded__()
+        if key in self.__data__:
+            del self.__data__[key]
+            os.remove(f"{self.directory}/{self.__sanitize_file_name__(key)}.json")
+            return True
+        return False
+
+    def __contains__(self, key: str | ModelProxy[T]) -> bool:
+        self.__ensure_loaded__()
+
+        if isinstance(key, ModelProxy):
+            return key.__key__ in self.__data__
+
+        return key in self.__data__
+
+    def __getitem__(self, key: str):
+        return cast(T, self.get(key))
+
+    def __setitem__(self, key: str, value: T):
+        return cast(T, self.upsert(key, value))
+
+    def __delitem__(self, key: str):
+        return self.delete(key)
+
+    def filter(self, filter: Callable[[str, ModelProxy], bool]):
+        """yield all models that satisfy the filter function"""
+        for key, model in self.__data__.items():
+            if filter(key, model):
+                yield cast(T, model)
+
+    def first(
+        self,
+        filter: Callable[[str, ModelProxy[T]], bool | Literal[True] | Literal[False]],
+    ) -> T | None:
+        """return the model that satisfy the filter function otherwise return None"""
+        self.__ensure_loaded__()
+        for key, model in self.__data__.items():
+            if filter(key, model):
+                return cast(T, model)
+        return None
+
+    def update(
+        self,
+        filter: Callable[[str, ModelProxy], bool],
+        updater: Callable[[str, ModelProxy], None],
+    ):
+        for key, model in self.__data__.items():
+            if filter(key, model):
+                updater(key, model)
+
+    def all(self):
+        self.__ensure_loaded__()
+        for x in self.__data__:
+            yield self.get(x)
+
+    def load(self):
+        if not os.path.exists(self.directory):
+            os.makedirs(self.directory, exist_ok=True)
+
+        for filename in os.listdir(self.directory):
+            with open(f"{self.directory}/{filename}", "r") as f:
+                try:
+                    d = json.load(f)
+                    key: str = d["__key__"]
+                    del d["__key__"]
+                    proxy = ModelProxy[T](store=self, key=key, model=self.cls(**d))
+                    self.__data__[key] = proxy
+                except Exception as e:
+                    logger.error(f"Error loading {filename}: {e}")
+        self.loaded = True
+        return self
+
+    def clear(self):
+        shutil.rmtree(self.directory)
+        self.__data__ = {}
+        self.loaded = False
+        self.__ensure_loaded__()
+
+    def commit_all(self):
+        self.__ensure_loaded__()
+        for proxy in self.__data__.values():
+            self.commit(proxy)
+
+    def commit(self, item: ModelProxy[T]):
+        if item.store != self:
+            raise ValueError("Item does not belong to this store")
+        if item.is_dirty:
+            with open(
+                f"{self.directory}/{self.__sanitize_file_name__(item.__key__)}.json",
+                "w",
+            ) as f:
+                d = item.model.model_dump()
+                d["__key__"] = item.__key__
+                f.write(json.dumps(d, indent=2, default=json_default))
+            item.is_dirty = False
+            item.__changes__ = []
+
+    def __ensure_loaded__(self):
+        if not self.loaded:
+            self.load()
+
+    def __sanitize_file_name__(self, name: str):
+        return (
+            name.replace("/", "_")
+            .replace("\\", "_")
+            .replace(":", "_")
+            .replace(" ", "_")
+            .replace(".", "_")
+            .replace("!", "_")
+            .replace("?", "_")
+            .lower()
+        )
+
+    def __iter__(self):
+        self.__ensure_loaded__()
+        return iter(self.__data__.keys())
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):  # pyright:ignore
+        if exc_type is None and self.save_on_exit:
+            self.commit_all()
+            self.__data__ = {}
+            self.loaded = False
+        else:
+            self.__data__ = {}
+            self.loaded = False
+        return False