checkpointer 2.0.0__tar.gz → 2.0.1__tar.gz

{checkpointer-2.0.0 → checkpointer-2.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: checkpointer
-Version: 2.0.0
+Version: 2.0.1
 Summary: A Python library for memoizing function results with support for multiple storage backends, async runtimes, and automatic cache invalidation
 Project-URL: Repository, https://github.com/Reddan/checkpointer.git
 Author: Hampus Hallman
@@ -17,18 +17,44 @@ Description-Content-Type: text/markdown
 
 # checkpointer · [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Reddan/checkpointer/blob/master/LICENSE) [![pypi](https://img.shields.io/pypi/v/checkpointer)](https://pypi.org/project/checkpointer/) [![Python 3.12](https://img.shields.io/badge/python-3.12-blue)](https://pypi.org/project/checkpointer/)
 
-`checkpointer` is a Python library for memoizing function results. It simplifies caching by providing a decorator-based API and supports various storage backends. It's designed for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations. 🚀
+`checkpointer` is a Python library for memoizing function results. It simplifies caching by providing a decorator-based API and supports various storage backends. It's designed for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations. ⚡️
 
 Adding or removing `@checkpoint` doesn't change how your code works, and it can be applied to any function, including ones you've already written, without altering their behavior or introducing side effects. The original function remains unchanged and can still be called directly when needed.
 
 ### Key Features:
-- **Multiple Storage Backends**: Supports in-memory, pickle, or your own custom storage.
-- **Simple Decorator API**: Apply `@checkpoint` to functions.
-- **Async and Sync Compatibility**: Works with synchronous functions and any Python async runtime (e.g., `asyncio`, `Trio`, `Curio`).
-- **Custom Expiration Logic**: Automatically invalidate old checkpoints.
-- **Flexible Path Configuration**: Control where checkpoints are stored.
+- 🗂️ **Multiple Storage Backends**: Supports in-memory, pickle, or your own custom storage.
+- 🎯 **Simple Decorator API**: Apply `@checkpoint` to functions.
+- 🔄 **Async and Sync Compatibility**: Works with synchronous functions and any Python async runtime (e.g., `asyncio`, `Trio`, `Curio`).
+- ⏲️ **Custom Expiration Logic**: Automatically invalidate old checkpoints.
+- 📂 **Flexible Path Configuration**: Control where checkpoints are stored.
 
-### How It Works
+---
+
+## Installation
+
+```bash
+pip install checkpointer
+```
+
+---
+
+## Quick Start 🚀
+
+```python
+from checkpointer import checkpoint
+
+@checkpoint
+def expensive_function(x: int) -> int:
+    print("Computing...")
+    return x ** 2
+
+result = expensive_function(4)  # Computes and stores result
+result = expensive_function(4)  # Loads from checkpoint
+```
+
+---
+
+## How It Works
 
 When you use `@checkpoint`, the function's **arguments** (`args`, `kwargs`) are hashed to create a unique identifier for each call. This identifier is used to store and retrieve cached results. If the same arguments are passed again, `checkpointer` will return the cached result instead of recomputing.
 
@@ -55,30 +81,6 @@ If you change `multiply`, the checkpoints for both `helper` and `compute` will b
 
 ---
 
-## Installation
-
-```bash
-pip install checkpointer
-```
-
----
-
-## Quick Start
-
-```python
-from checkpointer import checkpoint
-
-@checkpoint
-def expensive_function(x: int) -> int:
-    print("Computing...")
-    return x ** 2
-
-result = expensive_function(4)  # Computes and stores result
-result = expensive_function(4)  # Loads from checkpoint
-```
-
----
-
 ## Parameterization
 
 ### Global Configuration
@@ -160,6 +162,8 @@ Use `fn` to directly call the original, undecorated function:
 result = expensive_function.fn(4)
 ```
 
+This is especially useful **inside recursive functions**. By using `.fn` within the function itself, you avoid redundant caching of intermediate recursive calls while still caching the final result at the top level.
+
 ### Retrieve Stored Checkpoints
 Access stored results without recalculating:
 
@@ -169,7 +173,61 @@ stored_result = expensive_function.get(4)
 
 ---
 
-## Configuration Options
+## Storage Backends
+
+`checkpointer` supports flexible storage backends, including built-in options and custom implementations.
+
+### Built-In Backends
+
+1. **PickleStorage**: Saves checkpoints to disk using Python's `pickle` module.
+2. **MemoryStorage**: Caches checkpoints in memory for fast, non-persistent use.
+
+To use these backends, pass either `"pickle"` or `PickleStorage` (and similarly for `"memory"` or `MemoryStorage`) to the `format` parameter:
+```python
+from checkpointer import checkpoint, PickleStorage, MemoryStorage
+
+@checkpoint(format="pickle")  # Equivalent to format=PickleStorage
+def disk_cached(x: int) -> int:
+    return x ** 2
+
+@checkpoint(format="memory")  # Equivalent to format=MemoryStorage
+def memory_cached(x: int) -> int:
+    return x * 10
+```
+
+### Custom Storage Backends
+
+Create custom storage backends by implementing methods for storing, loading, and managing checkpoints. For example, a custom storage backend might use a database, cloud storage, or a specialized format.
+
+Example usage:
+```python
+from checkpointer import checkpoint, Storage
+from typing import Any
+from pathlib import Path
+from datetime import datetime
+
+class CustomStorage(Storage):  # Optional for type hinting
+    @staticmethod
+    def exists(path: Path) -> bool: ...
+    @staticmethod
+    def checkpoint_date(path: Path) -> datetime: ...
+    @staticmethod
+    def store(path: Path, data: Any) -> None: ...
+    @staticmethod
+    def load(path: Path) -> Any: ...
+    @staticmethod
+    def delete(path: Path) -> None: ...
+
+@checkpoint(format=CustomStorage)
+def custom_cached(x: int):
+    return x ** 2
+```
+
+This flexibility allows you to adapt `checkpointer` to meet any storage requirement, whether persistent or in-memory.
+
+---
+
+## Configuration Options ⚙️
 
 | Option         | Type                                | Default     | Description                                 |
 |----------------|-------------------------------------|-------------|---------------------------------------------|
@@ -177,12 +235,12 @@ stored_result = expensive_function.get(4)
 | `root_path`    | `Path`, `str`, or `None`            | User Cache  | Root directory for storing checkpoints.     |
 | `when`         | `bool`                              | `True`      | Enable or disable checkpointing.            |
 | `verbosity`    | `0` or `1`                          | `1`         | Logging verbosity.                          |
-| `path`         | `str` or `Callable[..., str]`       | `None`      | Custom path for checkpoint storage.         |
+| `path`         | `Callable[..., str]`                | `None`      | Custom path for checkpoint storage.         |
 | `should_expire`| `Callable[[datetime], bool]`        | `None`      | Custom expiration logic.                    |
 
 ---
 
-## Full Example
+## Full Example 🛠️
 
 ```python
 import asyncio

{checkpointer-2.0.0 → checkpointer-2.0.1}/README.md

@@ -1,17 +1,43 @@
 # checkpointer · [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Reddan/checkpointer/blob/master/LICENSE) [![pypi](https://img.shields.io/pypi/v/checkpointer)](https://pypi.org/project/checkpointer/) [![Python 3.12](https://img.shields.io/badge/python-3.12-blue)](https://pypi.org/project/checkpointer/)
 
-`checkpointer` is a Python library for memoizing function results. It simplifies caching by providing a decorator-based API and supports various storage backends. It's designed for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations. 🚀
+`checkpointer` is a Python library for memoizing function results. It simplifies caching by providing a decorator-based API and supports various storage backends. It's designed for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations. ⚡️
 
 Adding or removing `@checkpoint` doesn't change how your code works, and it can be applied to any function, including ones you've already written, without altering their behavior or introducing side effects. The original function remains unchanged and can still be called directly when needed.
 
 ### Key Features:
-- **Multiple Storage Backends**: Supports in-memory, pickle, or your own custom storage.
-- **Simple Decorator API**: Apply `@checkpoint` to functions.
-- **Async and Sync Compatibility**: Works with synchronous functions and any Python async runtime (e.g., `asyncio`, `Trio`, `Curio`).
-- **Custom Expiration Logic**: Automatically invalidate old checkpoints.
-- **Flexible Path Configuration**: Control where checkpoints are stored.
+- 🗂️ **Multiple Storage Backends**: Supports in-memory, pickle, or your own custom storage.
+- 🎯 **Simple Decorator API**: Apply `@checkpoint` to functions.
+- 🔄 **Async and Sync Compatibility**: Works with synchronous functions and any Python async runtime (e.g., `asyncio`, `Trio`, `Curio`).
+- ⏲️ **Custom Expiration Logic**: Automatically invalidate old checkpoints.
+- 📂 **Flexible Path Configuration**: Control where checkpoints are stored.
 
-### How It Works
+---
+
+## Installation
+
+```bash
+pip install checkpointer
+```
+
+---
+
+## Quick Start 🚀
+
+```python
+from checkpointer import checkpoint
+
+@checkpoint
+def expensive_function(x: int) -> int:
+    print("Computing...")
+    return x ** 2
+
+result = expensive_function(4)  # Computes and stores result
+result = expensive_function(4)  # Loads from checkpoint
+```
+
+---
+
+## How It Works
 
 When you use `@checkpoint`, the function's **arguments** (`args`, `kwargs`) are hashed to create a unique identifier for each call. This identifier is used to store and retrieve cached results. If the same arguments are passed again, `checkpointer` will return the cached result instead of recomputing.
 
@@ -38,30 +64,6 @@ If you change `multiply`, the checkpoints for both `helper` and `compute` will b
 
 ---
 
-## Installation
-
-```bash
-pip install checkpointer
-```
-
----
-
-## Quick Start
-
-```python
-from checkpointer import checkpoint
-
-@checkpoint
-def expensive_function(x: int) -> int:
-    print("Computing...")
-    return x ** 2
-
-result = expensive_function(4)  # Computes and stores result
-result = expensive_function(4)  # Loads from checkpoint
-```
-
----
-
 ## Parameterization
 
 ### Global Configuration
@@ -143,6 +145,8 @@ Use `fn` to directly call the original, undecorated function:
 result = expensive_function.fn(4)
 ```
 
+This is especially useful **inside recursive functions**. By using `.fn` within the function itself, you avoid redundant caching of intermediate recursive calls while still caching the final result at the top level.
+
 ### Retrieve Stored Checkpoints
 Access stored results without recalculating:
 
@@ -152,7 +156,61 @@ stored_result = expensive_function.get(4)
 
 ---
 
-## Configuration Options
+## Storage Backends
+
+`checkpointer` supports flexible storage backends, including built-in options and custom implementations.
+
+### Built-In Backends
+
+1. **PickleStorage**: Saves checkpoints to disk using Python's `pickle` module.
+2. **MemoryStorage**: Caches checkpoints in memory for fast, non-persistent use.
+
+To use these backends, pass either `"pickle"` or `PickleStorage` (and similarly for `"memory"` or `MemoryStorage`) to the `format` parameter:
+```python
+from checkpointer import checkpoint, PickleStorage, MemoryStorage
+
+@checkpoint(format="pickle")  # Equivalent to format=PickleStorage
+def disk_cached(x: int) -> int:
+    return x ** 2
+
+@checkpoint(format="memory")  # Equivalent to format=MemoryStorage
+def memory_cached(x: int) -> int:
+    return x * 10
+```
+
+### Custom Storage Backends
+
+Create custom storage backends by implementing methods for storing, loading, and managing checkpoints. For example, a custom storage backend might use a database, cloud storage, or a specialized format.
+
+Example usage:
+```python
+from checkpointer import checkpoint, Storage
+from typing import Any
+from pathlib import Path
+from datetime import datetime
+
+class CustomStorage(Storage):  # Optional for type hinting
+    @staticmethod
+    def exists(path: Path) -> bool: ...
+    @staticmethod
+    def checkpoint_date(path: Path) -> datetime: ...
+    @staticmethod
+    def store(path: Path, data: Any) -> None: ...
+    @staticmethod
+    def load(path: Path) -> Any: ...
+    @staticmethod
+    def delete(path: Path) -> None: ...
+
+@checkpoint(format=CustomStorage)
+def custom_cached(x: int):
+    return x ** 2
+```
+
+This flexibility allows you to adapt `checkpointer` to meet any storage requirement, whether persistent or in-memory.
+
+---
+
+## Configuration Options ⚙️
 
 | Option         | Type                                | Default     | Description                                 |
 |----------------|-------------------------------------|-------------|---------------------------------------------|
@@ -160,12 +218,12 @@ stored_result = expensive_function.get(4)
 | `root_path`    | `Path`, `str`, or `None`            | User Cache  | Root directory for storing checkpoints.     |
 | `when`         | `bool`                              | `True`      | Enable or disable checkpointing.            |
 | `verbosity`    | `0` or `1`                          | `1`         | Logging verbosity.                          |
-| `path`         | `str` or `Callable[..., str]`       | `None`      | Custom path for checkpoint storage.         |
+| `path`         | `Callable[..., str]`                | `None`      | Custom path for checkpoint storage.         |
 | `should_expire`| `Callable[[datetime], bool]`        | `None`      | Custom expiration logic.                    |
 
 ---
 
-## Full Example
+## Full Example 🛠️
 
 ```python
 import asyncio

{checkpointer-2.0.0 → checkpointer-2.0.1}/checkpointer/__init__.py

@@ -1,9 +1,9 @@
-from .checkpoint import Checkpointer, CheckpointFn
-from .checkpoint import CheckpointError, CheckpointReadFail
+from .checkpoint import Checkpointer, CheckpointFn, CheckpointError
 from .types import Storage
 from .function_body import get_function_hash
+import tempfile
 
 create_checkpointer = Checkpointer
 checkpoint = Checkpointer()
 memory_checkpoint = Checkpointer(format="memory")
-tmp_checkpoint = Checkpointer(root_path="/tmp/checkpoints")
+tmp_checkpoint = Checkpointer(root_path=tempfile.gettempdir() + "/checkpoints")

{checkpointer-2.0.0 → checkpointer-2.0.1}/checkpointer/checkpoint.py

@@ -1,7 +1,6 @@
 import inspect
 import relib.hashing as hashing
-from typing import Generic, TypeVar, TypedDict, Unpack, Literal, Union, Any, cast, overload
-from collections.abc import Callable
+from typing import Generic, TypeVar, TypedDict, Callable, Unpack, Literal, Union, Any, cast, overload
 from datetime import datetime
 from pathlib import Path
 from functools import update_wrapper
@@ -21,20 +20,13 @@ STORAGE_MAP = {"memory": MemoryStorage, "pickle": PickleStorage, "bcolz": BcolzS
 class CheckpointError(Exception):
   pass
 
-class CheckpointReadFail(CheckpointError):
-  pass
-
-StorageType = Literal["pickle", "memory", "bcolz"] | Storage
-CheckpointPath = str | Callable[..., str] | None
-ShouldExpire = Callable[[datetime], bool]
-
 class CheckpointerOpts(TypedDict, total=False):
-  format: StorageType
+  format: Storage | Literal["pickle", "memory", "bcolz"]
   root_path: Path | str | None
   when: bool
   verbosity: Literal[0, 1]
-  path: CheckpointPath
-  should_expire: ShouldExpire
+  path: Callable[..., str] | None
+  should_expire: Callable[[datetime], bool] | None
 
 class Checkpointer:
   def __init__(self, **opts: Unpack[CheckpointerOpts]):
@@ -51,7 +43,7 @@ class Checkpointer:
   @overload
   def __call__(self, fn: Fn, **override_opts: Unpack[CheckpointerOpts]) -> "CheckpointFn[Fn]": ...
   @overload
-  def __call__(self, fn=None, **override_opts: Unpack[CheckpointerOpts]) -> "Checkpointer": ...
+  def __call__(self, fn: None=None, **override_opts: Unpack[CheckpointerOpts]) -> "Checkpointer": ...
   def __call__(self, fn: Fn | None=None, **override_opts: Unpack[CheckpointerOpts]) -> Union["Checkpointer", "CheckpointFn[Fn]"]:
     if override_opts:
       opts = CheckpointerOpts(**{**self.__dict__, **override_opts})
@@ -68,48 +60,46 @@ class CheckpointFn(Generic[Fn]):
     self.fn = fn
     self.fn_hash = get_function_hash(wrapped)
     self.fn_id = f"{file_name}/{wrapped.__name__}"
-    self.is_async = inspect.iscoroutinefunction(fn)
+    self.is_async = inspect.iscoroutinefunction(wrapped)
 
   def get_checkpoint_id(self, args: tuple, kw: dict) -> str:
-    match self.checkpointer.path:
-      case str() as path:
-        return path
-      case Callable() as path:
-        p = path(*args, **kw)
-        assert isinstance(p, str), "path function must return a string"
-        return p
-      case _:
-        return f"{self.fn_id}/{hashing.hash([self.fn_hash, args, kw or 0])}"
-
-  async def _store_on_demand(self, args: tuple, kw: dict, force: bool):
+    if not callable(self.checkpointer.path):
+      return f"{self.fn_id}/{hashing.hash([self.fn_hash, args, kw or 0])}"
+    checkpoint_id = self.checkpointer.path(*args, **kw)
+    if not isinstance(checkpoint_id, str):
+      raise CheckpointError(f"path function must return a string, got {type(checkpoint_id)}")
+    return checkpoint_id
+
+  async def _store_on_demand(self, args: tuple, kw: dict, rerun: bool):
     checkpoint_id = self.get_checkpoint_id(args, kw)
     checkpoint_path = self.checkpointer.root_path / checkpoint_id
     storage = self.checkpointer.get_storage()
     should_log = storage is not MemoryStorage and self.checkpointer.verbosity > 0
-    refresh = force \
-      or storage.is_expired(checkpoint_path) \
-      or (self.checkpointer.should_expire and storage.should_expire(checkpoint_path, self.checkpointer.should_expire))
+    refresh = rerun \
+      or not storage.exists(checkpoint_path) \
+      or (self.checkpointer.should_expire and self.checkpointer.should_expire(storage.checkpoint_date(checkpoint_path)))
 
     if refresh:
       print_checkpoint(should_log, "MEMORIZING", checkpoint_id, "blue")
       data = self.fn(*args, **kw)
       if inspect.iscoroutine(data):
         data = await data
-      return storage.store_data(checkpoint_path, data)
+      storage.store(checkpoint_path, data)
+      return data
 
     try:
-      data = storage.load_data(checkpoint_path)
+      data = storage.load(checkpoint_path)
       print_checkpoint(should_log, "REMEMBERED", checkpoint_id, "green")
       return data
     except (EOFError, FileNotFoundError):
       print_checkpoint(should_log, "CORRUPTED", checkpoint_id, "yellow")
-      storage.delete_data(checkpoint_path)
-      return await self._store_on_demand(args, kw, force)
+      storage.delete(checkpoint_path)
+      return await self._store_on_demand(args, kw, rerun)
 
-  def _call(self, args: tuple, kw: dict, force=False):
+  def _call(self, args: tuple, kw: dict, rerun=False):
     if not self.checkpointer.when:
       return self.fn(*args, **kw)
-    coroutine = self._store_on_demand(args, kw, force)
+    coroutine = self._store_on_demand(args, kw, rerun)
     return coroutine if self.is_async else sync_resolve_coroutine(coroutine)
 
   __call__: Fn = cast(Fn, lambda self, *args, **kw: self._call(args, kw))
@@ -117,7 +107,8 @@ class CheckpointFn(Generic[Fn]):
 
   def get(self, *args, **kw) -> Any:
     checkpoint_path = self.checkpointer.root_path / self.get_checkpoint_id(args, kw)
+    storage = self.checkpointer.get_storage()
     try:
-      return self.checkpointer.get_storage().load_data(checkpoint_path)
+      return storage.load(checkpoint_path)
     except:
-      raise CheckpointReadFail()
+      raise CheckpointError("Could not load checkpoint")

{checkpointer-2.0.0 → checkpointer-2.0.1}/checkpointer/print_checkpoint.py

@@ -44,7 +44,7 @@ def colored_(text: str, color: Color | None = None, on_color: Color | None = Non
     text = f"\033[{COLOR_MAP[on_color] + 10}m{text}"
   return text + "\033[0m"
 
-noop = lambda *args, **_: args[0]
+noop = lambda text, *a, **k: text
 colored = colored_ if allow_color() else noop
 
 def print_checkpoint(should_log: bool, title: str, text: str, color: Color):

{checkpointer-2.0.0 → checkpointer-2.0.1}/checkpointer/storages/bcolz_storage.py

@@ -18,12 +18,6 @@ def get_data_type_str(x):
 def get_metapath(path: Path):
   return path.with_name(f"{path.name}_meta")
 
-def get_collection_timestamp(path: Path):
-  import bcolz
-  metapath = get_metapath(path)
-  meta_data = bcolz.open(metapath)[:][0]
-  return meta_data["created"]
-
 def insert_data(path: Path, data):
   import bcolz
   c = bcolz.carray(data, rootdir=path, mode="w")
@@ -31,22 +25,17 @@ def insert_data(path: Path, data):
 
 class BcolzStorage(Storage):
   @staticmethod
-  def is_expired(path):
-    try:
-      get_collection_timestamp(path)
-      return False
-    except (FileNotFoundError, EOFError):
-      return True
+  def exists(path):
+    return path.exists()
 
   @staticmethod
-  def should_expire(path, expire_fn):
-    return expire_fn(get_collection_timestamp(path))
+  def checkpoint_date(path):
+    return datetime.fromtimestamp(path.stat().st_mtime)
 
   @staticmethod
-  def store_data(path, data):
+  def store(path, data):
     metapath = get_metapath(path)
     path.parent.mkdir(parents=True, exist_ok=True)
-    created = datetime.now()
     data_type_str = get_data_type_str(data)
     if data_type_str == "tuple":
       fields = list(range(len(data)))
@@ -54,18 +43,17 @@ class BcolzStorage(Storage):
       fields = sorted(data.keys())
     else:
       fields = []
-    meta_data = {"created": created, "data_type_str": data_type_str, "fields": fields}
+    meta_data = {"data_type_str": data_type_str, "fields": fields}
     insert_data(metapath, meta_data)
     if data_type_str in ["tuple", "dict"]:
       for i in range(len(fields)):
         child_path = Path(f"{path} ({i})")
-        BcolzStorage.store_data(child_path, data[fields[i]])
+        BcolzStorage.store(child_path, data[fields[i]])
     else:
       insert_data(path, data)
-    return data
 
   @staticmethod
-  def load_data(path):
+  def load(path):
     import bcolz
     metapath = get_metapath(path)
     meta_data = bcolz.open(metapath)[:][0]
@@ -73,7 +61,7 @@ class BcolzStorage(Storage):
     if data_type_str in ["tuple", "dict"]:
       fields = meta_data["fields"]
       partitions = range(len(fields))
-      data = [BcolzStorage.load_data(Path(f"{path} ({i})")) for i in partitions]
+      data = [BcolzStorage.load(Path(f"{path} ({i})")) for i in partitions]
       if data_type_str == "tuple":
         return tuple(data)
       else:
@@ -88,7 +76,7 @@ class BcolzStorage(Storage):
         return data[:]
 
   @staticmethod
-  def delete_data(path):
+  def delete(path):
     # NOTE: Not recursive
     metapath = get_metapath(path)
     try:

checkpointer-2.0.1/checkpointer/storages/memory_storage.py

@@ -0,0 +1,28 @@
+from datetime import datetime
+from ..types import Storage
+
+store = {}
+date_stored = {}
+
+class MemoryStorage(Storage):
+  @staticmethod
+  def exists(path):
+    return str(path) in store
+
+  @staticmethod
+  def checkpoint_date(path):
+    return date_stored[str(path)]
+
+  @staticmethod
+  def store(path, data):
+    store[str(path)] = data
+    date_stored[str(path)] = datetime.now()
+
+  @staticmethod
+  def load(path):
+    return store[str(path)]
+
+  @staticmethod
+  def delete(path):
+    del store[str(path)]
+    del date_stored[str(path)]

checkpointer-2.0.1/checkpointer/storages/pickle_storage.py

@@ -0,0 +1,36 @@
+import pickle
+from pathlib import Path
+from datetime import datetime
+from ..types import Storage
+
+def get_path(path: Path):
+  return path.with_name(f"{path.name}.pkl")
+
+class PickleStorage(Storage):
+  @staticmethod
+  def exists(path):
+    return get_path(path).exists()
+
+  @staticmethod
+  def checkpoint_date(path):
+    return datetime.fromtimestamp(get_path(path).stat().st_mtime)
+
+  @staticmethod
+  def store(path, data):
+    full_path = get_path(path)
+    full_path.parent.mkdir(parents=True, exist_ok=True)
+    with full_path.open("wb") as file:
+      pickle.dump(data, file, -1)
+
+  @staticmethod
+  def load(path):
+    full_path = get_path(path)
+    with full_path.open("rb") as file:
+      return pickle.load(file)
+
+  @staticmethod
+  def delete(path):
+    try:
+      get_path(path).unlink()
+    except FileNotFoundError:
+      pass

checkpointer-2.0.1/checkpointer/types.py

@@ -0,0 +1,19 @@
+from typing import Protocol, Any
+from pathlib import Path
+from datetime import datetime
+
+class Storage(Protocol):
+  @staticmethod
+  def exists(path: Path) -> bool: ...
+
+  @staticmethod
+  def checkpoint_date(path: Path) -> datetime: ...
+
+  @staticmethod
+  def store(path: Path, data: Any) -> None: ...
+
+  @staticmethod
+  def load(path: Path) -> Any: ...
+
+  @staticmethod
+  def delete(path: Path) -> None: ...

{checkpointer-2.0.0 → checkpointer-2.0.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "checkpointer"
-version = "2.0.0"
+version = "2.0.1"
 requires-python = ">=3.12"
 dependencies = [
   "relib",

{checkpointer-2.0.0 → checkpointer-2.0.1}/uv.lock

@@ -3,7 +3,7 @@ requires-python = ">=3.12"
 
 [[package]]
 name = "checkpointer"
-version = "2.0.0"
+version = "2.0.1"
 source = { editable = "." }
 dependencies = [
     { name = "relib" },

checkpointer-2.0.0/checkpointer/storages/memory_storage.py

@@ -1,29 +0,0 @@
-from datetime import datetime
-from ..types import Storage
-
-store = {}
-date_stored = {}
-
-class MemoryStorage(Storage):
-  @staticmethod
-  def is_expired(path):
-    return path not in store
-
-  @staticmethod
-  def should_expire(path, expire_fn):
-    return expire_fn(date_stored[path])
-
-  @staticmethod
-  def store_data(path, data):
-    store[path] = data
-    date_stored[path] = datetime.now()
-    return data
-
-  @staticmethod
-  def load_data(path):
-    return store[path]
-
-  @staticmethod
-  def delete_data(path):
-    del store[path]
-    del date_stored[path]

checkpointer-2.0.0/checkpointer/storages/pickle_storage.py

@@ -1,55 +0,0 @@
-import pickle
-from pathlib import Path
-from datetime import datetime
-from ..types import Storage
-
-def get_paths(path: Path):
-  meta_full_path = path.with_name(f"{path.name}_meta.pkl")
-  pkl_full_path = path.with_name(f"{path.name}.pkl")
-  return meta_full_path, pkl_full_path
-
-def get_collection_timestamp(path: Path):
-  meta_full_path, _ = get_paths(path)
-  with meta_full_path.open("rb") as file:
-    meta_data = pickle.load(file)
-    return meta_data["created"]
-
-class PickleStorage(Storage):
-  @staticmethod
-  def is_expired(path):
-    try:
-      get_collection_timestamp(path)
-      return False
-    except (FileNotFoundError, EOFError):
-      return True
-
-  @staticmethod
-  def should_expire(path, expire_fn):
-    return expire_fn(get_collection_timestamp(path))
-
-  @staticmethod
-  def store_data(path, data):
-    created = datetime.now()
-    meta_data = {"created": created} # TODO: this should just be a JSON or binary dump of the unix timestamp and other metadata - not pickle
-    meta_full_path, pkl_full_path = get_paths(path)
-    pkl_full_path.parent.mkdir(parents=True, exist_ok=True)
-    with pkl_full_path.open("wb") as file:
-      pickle.dump(data, file, -1)
-    with meta_full_path.open("wb") as file:
-      pickle.dump(meta_data, file, -1)
-    return data
-
-  @staticmethod
-  def load_data(path):
-    _, full_path = get_paths(path)
-    with full_path.open("rb") as file:
-      return pickle.load(file)
-
-  @staticmethod
-  def delete_data(path):
-    meta_full_path, pkl_full_path = get_paths(path)
-    try:
-      meta_full_path.unlink()
-      pkl_full_path.unlink()
-    except FileNotFoundError:
-      pass

checkpointer-2.0.0/checkpointer/types.py

@@ -1,19 +0,0 @@
-from typing import Callable, Protocol, Any
-from pathlib import Path
-from datetime import datetime
-
-class Storage(Protocol):
-  @staticmethod
-  def is_expired(path: Path) -> bool: ...
-
-  @staticmethod
-  def should_expire(path: Path, expire_fn: Callable[[datetime], bool]) -> bool: ...
-
-  @staticmethod
-  def store_data(path: Path, data: Any) -> Any: ...
-
-  @staticmethod
-  def load_data(path: Path) -> Any: ...
-
-  @staticmethod
-  def delete_data(path: Path) -> None: ...