checkpointer 2.5.0__tar.gz → 2.6.1__tar.gz

checkpointer-2.6.1/.python-version

@@ -0,0 +1 @@
+3.12.7

{checkpointer-2.5.0 → checkpointer-2.6.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: checkpointer
-Version: 2.5.0
+Version: 2.6.1
 Summary: A Python library for memoizing function results with support for multiple storage backends, async runtimes, and automatic cache invalidation
 Project-URL: Repository, https://github.com/Reddan/checkpointer.git
 Author: Hampus Hallman
@@ -12,10 +12,12 @@ License: Copyright 2018-2025 Hampus Hallman
         
         THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 License-File: LICENSE
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 
-# checkpointer · [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Reddan/checkpointer/blob/master/LICENSE) [![pypi](https://img.shields.io/pypi/v/checkpointer)](https://pypi.org/project/checkpointer/) [![Python 3.12](https://img.shields.io/badge/python-3.12-blue)](https://pypi.org/project/checkpointer/)
+# checkpointer · [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Reddan/checkpointer/blob/master/LICENSE) [![pypi](https://img.shields.io/pypi/v/checkpointer)](https://pypi.org/project/checkpointer/) [![pypi](https://img.shields.io/pypi/pyversions/checkpointer)](https://pypi.org/project/checkpointer/)
 
 `checkpointer` is a Python library for memoizing function results. It provides a decorator-based API with support for multiple storage backends. Use it for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations.
 
@@ -28,6 +30,7 @@ Adding or removing `@checkpoint` doesn't change how your code works. You can app
 - ⏲️ **Custom Expiration Logic**: Automatically invalidate old checkpoints.
 - 📂 **Flexible Path Configuration**: Control where checkpoints are stored.
 - 📦 **Captured Variables Handling**: Optionally include captured variables in cache invalidation.
+- ⚡ **Custom Argument Hashing**: Override argument hashing for speed or specialized hashing logic.
 
 ---
 
@@ -61,9 +64,9 @@ When you use `@checkpoint`, the function's **arguments** (`args`, `kwargs`) are
 
 Additionally, `checkpointer` ensures that caches are invalidated when a function's implementation or any of its dependencies change. Each function is assigned a hash based on:
 
-1. **Its source code**: Changes to the function's code update its hash.
-2. **Dependent functions**: If a function calls others, changes in those dependencies will also update the hash.
-3. **Captured variables**: (Optional) If `capture=True`, changes to captured variables and global variables will also update the hash.
+1. **Function Code**: The hash updates when the function’s own source code changes.
+2. **Dependencies**: If the function calls other user-defined functions, changes in those dependencies also update the hash.
+3. **External Variables** *(with `capture=True`)*: Any global or closure-based variables used by the function are included in its hash, so changes to those variables also trigger cache invalidation.
 
 ### Example: Cache Invalidation
 
@@ -155,21 +158,19 @@ stored_result = expensive_function.get(4)
 
 ### Refresh Function Hash
 
-When using `capture=True`, changes to captured variables are included in the function's hash to determine cache invalidation. However, `checkpointer` does not automatically update this hash during a running Python session—it recalculates between sessions or when you explicitly refresh it.
-
-Use the `reinit` method to manually refresh the function's hash within the same session:
+If `capture=True`, you might need to re-hash a function during the same Python session. For that, call `reinit`:
 
 ```python
 expensive_function.reinit()
 ```
 
-This forces `checkpointer` to recalculate the hash of `expensive_function`, considering any changes to captured variables. It's useful when you've modified external variables that the function depends on and want the cache to reflect these changes immediately.
+This tells `checkpointer` to recalculate the function hash, reflecting changes in captured variables.
 
 ---
 
 ## Storage Backends
 
-`checkpointer` works with both built-in and custom storage backends, so you can use what's provided or roll your own as needed.
+`checkpointer` works with built-in and custom storage backends, so you can use what's provided or roll your own as needed.
 
 ### Built-In Backends
 
@@ -201,10 +202,10 @@ from checkpointer import checkpoint, Storage
 from datetime import datetime
 
 class CustomStorage(Storage):
-    def store(self, path, data): ...  # Save the checkpoint data
-    def exists(self, path) -> bool: ...  # Check if a checkpoint exists at the given path
-    def checkpoint_date(self, path) -> datetime: ...  # Return the date the checkpoint was created
-    def load(self, path): ...  # Return the checkpoint data
+    def exists(self, path) -> bool: ...  # Check if a checkpoint exists
+    def checkpoint_date(self, path) -> datetime: ...  # Get the checkpoint's timestamp
+    def store(self, path, data): ...  # Save data to the checkpoint
+    def load(self, path): ...  # Load data from the checkpoint
     def delete(self, path): ...  # Delete the checkpoint
 
 @checkpoint(format=CustomStorage)
@@ -212,21 +213,21 @@ def custom_cached(x: int):
     return x ** 2
 ```
 
-Using a custom backend lets you tailor storage to your application, whether it involves databases, cloud storage, or custom file formats.
+Use a custom backend to integrate with databases, cloud storage, or specialized file formats.
 
 ---
 
 ## Configuration Options ⚙️
 
-| Option          | Type                              | Default              | Description                                    |
-|-----------------|-----------------------------------|----------------------|------------------------------------------------|
-| `capture`       | `bool`                            | `False`              | Include captured variables in function hashes. |
-| `format`        | `"pickle"`, `"memory"`, `Storage` | `"pickle"`           | Storage backend format.                        |
-| `root_path`     | `Path`, `str`, or `None`          | ~/.cache/checkpoints | Root directory for storing checkpoints.        |
-| `when`          | `bool`                            | `True`               | Enable or disable checkpointing.               |
-| `verbosity`     | `0` or `1`                        | `1`                  | Logging verbosity.                             |
-| `path`          | `Callable[..., str]`              | `None`               | Custom path for checkpoint storage.            |
-| `should_expire` | `Callable[[datetime], bool]`      | `None`               | Custom expiration logic.                       |
+| Option          | Type                                | Default              | Description                                               |
+|-----------------|-------------------------------------|----------------------|-----------------------------------------------------------|
+| `capture`       | `bool`                              | `False`              | Include captured variables in function hashes.            |
+| `format`        | `"pickle"`, `"memory"`, `Storage`   | `"pickle"`           | Storage backend format.                                   |
+| `root_path`     | `Path`, `str`, or `None`            | ~/.cache/checkpoints | Root directory for storing checkpoints.                   |
+| `when`          | `bool`                              | `True`               | Enable or disable checkpointing.                          |
+| `verbosity`     | `0`, `1` or `2`                     | `1`                  | Logging verbosity.                                        |
+| `should_expire` | `Callable[[datetime], bool]`        | `None`               | Custom expiration logic.                                  |
+| `hash_by`       | `Callable[..., Any]`                | `None`               | Custom function that transforms arguments before hashing. |
 
 ---
 

{checkpointer-2.5.0 → checkpointer-2.6.1}/README.md

@@ -1,4 +1,4 @@
-# checkpointer · [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Reddan/checkpointer/blob/master/LICENSE) [![pypi](https://img.shields.io/pypi/v/checkpointer)](https://pypi.org/project/checkpointer/) [![Python 3.12](https://img.shields.io/badge/python-3.12-blue)](https://pypi.org/project/checkpointer/)
+# checkpointer · [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Reddan/checkpointer/blob/master/LICENSE) [![pypi](https://img.shields.io/pypi/v/checkpointer)](https://pypi.org/project/checkpointer/) [![pypi](https://img.shields.io/pypi/pyversions/checkpointer)](https://pypi.org/project/checkpointer/)
 
 `checkpointer` is a Python library for memoizing function results. It provides a decorator-based API with support for multiple storage backends. Use it for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations.
 
@@ -11,6 +11,7 @@ Adding or removing `@checkpoint` doesn't change how your code works. You can app
 - ⏲️ **Custom Expiration Logic**: Automatically invalidate old checkpoints.
 - 📂 **Flexible Path Configuration**: Control where checkpoints are stored.
 - 📦 **Captured Variables Handling**: Optionally include captured variables in cache invalidation.
+- ⚡ **Custom Argument Hashing**: Override argument hashing for speed or specialized hashing logic.
 
 ---
 
@@ -44,9 +45,9 @@ When you use `@checkpoint`, the function's **arguments** (`args`, `kwargs`) are
 
 Additionally, `checkpointer` ensures that caches are invalidated when a function's implementation or any of its dependencies change. Each function is assigned a hash based on:
 
-1. **Its source code**: Changes to the function's code update its hash.
-2. **Dependent functions**: If a function calls others, changes in those dependencies will also update the hash.
-3. **Captured variables**: (Optional) If `capture=True`, changes to captured variables and global variables will also update the hash.
+1. **Function Code**: The hash updates when the function’s own source code changes.
+2. **Dependencies**: If the function calls other user-defined functions, changes in those dependencies also update the hash.
+3. **External Variables** *(with `capture=True`)*: Any global or closure-based variables used by the function are included in its hash, so changes to those variables also trigger cache invalidation.
 
 ### Example: Cache Invalidation
 
@@ -138,21 +139,19 @@ stored_result = expensive_function.get(4)
 
 ### Refresh Function Hash
 
-When using `capture=True`, changes to captured variables are included in the function's hash to determine cache invalidation. However, `checkpointer` does not automatically update this hash during a running Python session—it recalculates between sessions or when you explicitly refresh it.
-
-Use the `reinit` method to manually refresh the function's hash within the same session:
+If `capture=True`, you might need to re-hash a function during the same Python session. For that, call `reinit`:
 
 ```python
 expensive_function.reinit()
 ```
 
-This forces `checkpointer` to recalculate the hash of `expensive_function`, considering any changes to captured variables. It's useful when you've modified external variables that the function depends on and want the cache to reflect these changes immediately.
+This tells `checkpointer` to recalculate the function hash, reflecting changes in captured variables.
 
 ---
 
 ## Storage Backends
 
-`checkpointer` works with both built-in and custom storage backends, so you can use what's provided or roll your own as needed.
+`checkpointer` works with built-in and custom storage backends, so you can use what's provided or roll your own as needed.
 
 ### Built-In Backends
 
@@ -184,10 +183,10 @@ from checkpointer import checkpoint, Storage
 from datetime import datetime
 
 class CustomStorage(Storage):
-    def store(self, path, data): ...  # Save the checkpoint data
-    def exists(self, path) -> bool: ...  # Check if a checkpoint exists at the given path
-    def checkpoint_date(self, path) -> datetime: ...  # Return the date the checkpoint was created
-    def load(self, path): ...  # Return the checkpoint data
+    def exists(self, path) -> bool: ...  # Check if a checkpoint exists
+    def checkpoint_date(self, path) -> datetime: ...  # Get the checkpoint's timestamp
+    def store(self, path, data): ...  # Save data to the checkpoint
+    def load(self, path): ...  # Load data from the checkpoint
     def delete(self, path): ...  # Delete the checkpoint
 
 @checkpoint(format=CustomStorage)
@@ -195,21 +194,21 @@ def custom_cached(x: int):
     return x ** 2
 ```
 
-Using a custom backend lets you tailor storage to your application, whether it involves databases, cloud storage, or custom file formats.
+Use a custom backend to integrate with databases, cloud storage, or specialized file formats.
 
 ---
 
 ## Configuration Options ⚙️
 
-| Option          | Type                              | Default              | Description                                    |
-|-----------------|-----------------------------------|----------------------|------------------------------------------------|
-| `capture`       | `bool`                            | `False`              | Include captured variables in function hashes. |
-| `format`        | `"pickle"`, `"memory"`, `Storage` | `"pickle"`           | Storage backend format.                        |
-| `root_path`     | `Path`, `str`, or `None`          | ~/.cache/checkpoints | Root directory for storing checkpoints.        |
-| `when`          | `bool`                            | `True`               | Enable or disable checkpointing.               |
-| `verbosity`     | `0` or `1`                        | `1`                  | Logging verbosity.                             |
-| `path`          | `Callable[..., str]`              | `None`               | Custom path for checkpoint storage.            |
-| `should_expire` | `Callable[[datetime], bool]`      | `None`               | Custom expiration logic.                       |
+| Option          | Type                                | Default              | Description                                               |
+|-----------------|-------------------------------------|----------------------|-----------------------------------------------------------|
+| `capture`       | `bool`                              | `False`              | Include captured variables in function hashes.            |
+| `format`        | `"pickle"`, `"memory"`, `Storage`   | `"pickle"`           | Storage backend format.                                   |
+| `root_path`     | `Path`, `str`, or `None`            | ~/.cache/checkpoints | Root directory for storing checkpoints.                   |
+| `when`          | `bool`                              | `True`               | Enable or disable checkpointing.                          |
+| `verbosity`     | `0`, `1` or `2`                     | `1`                  | Logging verbosity.                                        |
+| `should_expire` | `Callable[[datetime], bool]`        | `None`               | Custom expiration logic.                                  |
+| `hash_by`       | `Callable[..., Any]`                | `None`               | Custom function that transforms arguments before hashing. |
 
 ---
 

{checkpointer-2.5.0 → checkpointer-2.6.1}/checkpointer/checkpoint.py

@@ -22,10 +22,11 @@ class CheckpointerOpts(TypedDict, total=False):
   format: Type[Storage] | Literal["pickle", "memory", "bcolz"]
   root_path: Path | str | None
   when: bool
-  verbosity: Literal[0, 1]
-  path: Callable[..., str] | None
+  verbosity: Literal[0, 1, 2]
+  hash_by: Callable | None
   should_expire: Callable[[datetime], bool] | None
   capture: bool
+  fn_hash: str | None
 
 class Checkpointer:
   def __init__(self, **opts: Unpack[CheckpointerOpts]):
@@ -33,9 +34,10 @@ class Checkpointer:
     self.root_path = Path(opts.get("root_path", DEFAULT_DIR) or ".")
     self.when = opts.get("when", True)
     self.verbosity = opts.get("verbosity", 1)
-    self.path = opts.get("path")
+    self.hash_by = opts.get("hash_by")
     self.should_expire = opts.get("should_expire")
     self.capture = opts.get("capture", False)
+    self.fn_hash = opts.get("fn_hash")
 
   @overload
   def __call__(self, fn: Fn, **override_opts: Unpack[CheckpointerOpts]) -> CheckpointFn[Fn]: ...
@@ -66,9 +68,9 @@ class CheckpointFn(Generic[Fn]):
     store_format = self.checkpointer.format
     Storage = STORAGE_MAP[store_format] if isinstance(store_format, str) else store_format
     deep_hashes = [child._set_ident().fn_hash_raw for child in iterate_checkpoint_fns(self)]
-    self.fn_hash = str(ObjectHash().update_hash(self.fn_hash_raw, iter=deep_hashes))
+    self.fn_hash = self.checkpointer.fn_hash or str(ObjectHash().write_text(self.fn_hash_raw, *deep_hashes))
     self.fn_subdir = f"{fn_file}/{fn_name}/{self.fn_hash[:16]}"
-    self.is_async = inspect.iscoroutinefunction(wrapped)
+    self.is_async: bool = self.fn.is_async if isinstance(self.fn, CheckpointFn) else inspect.iscoroutinefunction(self.fn)
     self.storage = Storage(self)
     self.cleanup = self.storage.cleanup
 
@@ -80,32 +82,29 @@ class CheckpointFn(Generic[Fn]):
     self._lazyinit()
     return self._getattribute(name)
 
-  def reinit(self, recursive=False):
+  def reinit(self, recursive=False) -> CheckpointFn[Fn]:
     pointfns = list(iterate_checkpoint_fns(self)) if recursive else [self]
     for pointfn in pointfns:
       pointfn._set_ident(True)
     for pointfn in pointfns:
       pointfn._lazyinit()
+    return self
 
   def get_checkpoint_id(self, args: tuple, kw: dict) -> str:
-    if not callable(self.checkpointer.path):
-      call_hash = ObjectHash(self.fn_hash, args, kw, digest_size=16)
-      return f"{self.fn_subdir}/{call_hash}"
-    checkpoint_id = self.checkpointer.path(*args, **kw)
-    if not isinstance(checkpoint_id, str):
-      raise CheckpointError(f"path function must return a string, got {type(checkpoint_id)}")
-    return checkpoint_id
+    hash_params = [self.checkpointer.hash_by(*args, **kw)] if self.checkpointer.hash_by else (args, kw)
+    call_hash = ObjectHash(self.fn_hash, *hash_params, digest_size=16)
+    return f"{self.fn_subdir}/{call_hash}"
 
   async def _store_on_demand(self, args: tuple, kw: dict, rerun: bool):
     checkpoint_id = self.get_checkpoint_id(args, kw)
     checkpoint_path = self.checkpointer.root_path / checkpoint_id
-    verbose = self.checkpointer.verbosity > 0
+    verbosity = self.checkpointer.verbosity
     refresh = rerun \
       or not self.storage.exists(checkpoint_path) \
       or (self.checkpointer.should_expire and self.checkpointer.should_expire(self.storage.checkpoint_date(checkpoint_path)))
 
     if refresh:
-      print_checkpoint(verbose, "MEMORIZING", checkpoint_id, "blue")
+      print_checkpoint(verbosity >= 1, "MEMORIZING", checkpoint_id, "blue")
       data = self.fn(*args, **kw)
       if inspect.iscoroutine(data):
         data = await data
@@ -114,11 +113,11 @@ class CheckpointFn(Generic[Fn]):
 
     try:
       data = self.storage.load(checkpoint_path)
-      print_checkpoint(verbose, "REMEMBERED", checkpoint_id, "green")
+      print_checkpoint(verbosity >= 2, "REMEMBERED", checkpoint_id, "green")
       return data
     except (EOFError, FileNotFoundError):
       pass
-    print_checkpoint(verbose, "CORRUPTED", checkpoint_id, "yellow")
+    print_checkpoint(verbosity >= 1, "CORRUPTED", checkpoint_id, "yellow")
     return await self._store_on_demand(args, kw, True)
 
   def _call(self, args: tuple, kw: dict, rerun=False):

{checkpointer-2.5.0 → checkpointer-2.6.1}/checkpointer/fn_ident.py

@@ -4,14 +4,14 @@ from collections.abc import Callable
 from itertools import takewhile
 from pathlib import Path
 from types import CodeType, FunctionType, MethodType
-from typing import Any, Generator, Type, TypeGuard
+from typing import Any, Iterable, Type, TypeGuard
 from .object_hash import ObjectHash
 from .utils import AttrDict, distinct, get_cell_contents, iterate_and_upcoming, transpose, unwrap_fn
 
 cwd = Path.cwd()
 
 def is_class(obj) -> TypeGuard[Type]:
-  # isinstance works too, but needlessly triggers __getattribute__
+  # isinstance works too, but needlessly triggers _lazyinit()
   return issubclass(type(obj), type)
 
 def extract_classvars(code: CodeType, scope_vars: AttrDict) -> dict[str, dict[str, Type]]:
@@ -33,7 +33,7 @@ def extract_classvars(code: CodeType, scope_vars: AttrDict) -> dict[str, dict[st
       scope_obj = None
   return classvars
 
-def extract_scope_values(code: CodeType, scope_vars: AttrDict) -> Generator[tuple[tuple[str, ...], Any], None, None]:
+def extract_scope_values(code: CodeType, scope_vars: AttrDict) -> Iterable[tuple[tuple[str, ...], Any]]:
   classvars = extract_classvars(code, scope_vars)
   scope_vars = scope_vars.set({k: scope_vars[k].set(v) for k, v in classvars.items()})
   for instr, upcoming_instrs in iterate_and_upcoming(dis.get_instructions(code)):

{checkpointer-2.5.0 → checkpointer-2.6.1}/checkpointer/object_hash.py

@@ -3,7 +3,7 @@ import hashlib
 import io
 import re
 from collections.abc import Iterable
-from contextlib import nullcontext
+from contextlib import nullcontext, suppress
 from decimal import Decimal
 from itertools import chain
 from pickle import HIGHEST_PROTOCOL as PROTOCOL
@@ -11,19 +11,18 @@ from types import BuiltinFunctionType, FunctionType, GeneratorType, MethodType,
 from typing import Any, TypeAliasType, TypeVar
 from .utils import ContextVar, get_fn_body
 
-try:
+np, torch = None, None
+
+with suppress(Exception):
   import numpy as np
-except:
-  np = None
-try:
+
+with suppress(Exception):
   import torch
-except:
-  torch = None
 
 def encode_type(t: type | FunctionType) -> str:
   return f"{t.__module__}:{t.__qualname__}"
 
-def encode_val(v: Any) -> str:
+def encode_type_of(v: Any) -> str:
   return encode_type(type(v))
 
 class ObjectHashError(Exception):
@@ -32,11 +31,11 @@ class ObjectHashError(Exception):
     self.obj = obj
 
 class ObjectHash:
-  def __init__(self, *obj: Any, iter: Iterable[Any] = [], digest_size=64, tolerate_errors=False) -> None:
+  def __init__(self, *objs: Any, iter: Iterable[Any] = (), digest_size=64, tolerate_errors=False) -> None:
     self.hash = hashlib.blake2b(digest_size=digest_size)
     self.current: dict[int, int] = {}
     self.tolerate_errors = ContextVar(tolerate_errors)
-    self.update(iter=chain(obj, iter))
+    self.update(iter=chain(objs, iter))
 
   def copy(self) -> "ObjectHash":
     new = ObjectHash(tolerate_errors=self.tolerate_errors.value)
@@ -48,15 +47,21 @@ class ObjectHash:
 
   __str__ = hexdigest
 
-  def update_hash(self, *data: bytes | str, iter: Iterable[bytes | str] = []) -> "ObjectHash":
+  def nested_hash(self, *objs: Any) -> str:
+    return ObjectHash(iter=objs, tolerate_errors=self.tolerate_errors.value).hexdigest()
+
+  def write_bytes(self, *data: bytes, iter: Iterable[bytes] = ()) -> "ObjectHash":
     for d in chain(data, iter):
-      self.hash.update(d.encode() if isinstance(d, str) else d)
+      self.hash.update(d)
     return self
 
+  def write_text(self, *data: str, iter: Iterable[str] = ()) -> "ObjectHash":
+    return self.write_bytes(iter=(d.encode() for d in chain(data, iter)))
+
   def header(self, *args: Any) -> "ObjectHash":
-    return self.update_hash(":".join(map(str, args)))
+    return self.write_bytes(":".join(map(str, args)).encode())
 
-  def update(self, *objs: Any, iter: Iterable[Any] = [], tolerate_errors: bool | None=None) -> "ObjectHash":
+  def update(self, *objs: Any, iter: Iterable[Any] = (), tolerate_errors: bool | None=None) -> "ObjectHash":
     with nullcontext() if tolerate_errors is None else self.tolerate_errors.set(tolerate_errors):
       for obj in chain(objs, iter):
         try:
@@ -74,19 +79,20 @@ class ObjectHash:
         self.header("null")
 
       case bool() | int() | float() | complex() | Decimal() | ObjectHash():
-        self.header("number", encode_val(obj), obj)
+        self.header("number", encode_type_of(obj), obj)
 
       case str() | bytes() | bytearray() | memoryview():
-        self.header("bytes", encode_val(obj), len(obj)).update_hash(obj)
+        b = obj.encode() if isinstance(obj, str) else obj
+        self.header("bytes", encode_type_of(obj), len(b)).write_bytes(b)
 
       case set() | frozenset():
-        self.header("set", encode_val(obj), len(obj))
         try:
           items = sorted(obj)
+          header = "set"
         except:
-          self.header("unsortable")
-          items = sorted(str(ObjectHash(item, tolerate_errors=self.tolerate_errors.value)) for item in obj)
-        self.update(iter=items)
+          items = sorted(map(self.nested_hash, obj))
+          header = "set-unsortable"
+        self.header(header, encode_type_of(obj), len(obj)).update(iter=items)
 
       case TypeVar():
         self.header("TypeVar").update(obj.__name__, obj.__bound__, obj.__constraints__, obj.__contravariant__, obj.__covariant__)
@@ -113,7 +119,7 @@ class ObjectHash:
         self.header("generator", obj.__qualname__)._update_iterator(obj)
 
       case io.TextIOWrapper() | io.FileIO() | io.BufferedRandom() | io.BufferedWriter() | io.BufferedReader():
-        self.header("file", encode_val(obj)).update(obj.name, obj.mode, obj.tell())
+        self.header("file", encode_type_of(obj)).update(obj.name, obj.mode, obj.tell())
 
       case type():
         self.header("type", encode_type(obj))
@@ -122,20 +128,20 @@ class ObjectHash:
         self.header("dtype").update(obj.__class__, obj.descr)
 
       case _ if np and isinstance(obj, np.ndarray):
-        self.header("ndarray", encode_val(obj), obj.shape, obj.strides).update(obj.dtype)
+        self.header("ndarray", encode_type_of(obj), obj.shape, obj.strides).update(obj.dtype)
         if obj.dtype.hasobject:
           self.update(obj.__reduce_ex__(PROTOCOL))
         else:
           array = np.ascontiguousarray(obj if obj.base is None else obj.base).view(np.uint8)
-          self.update_hash(array.data)
+          self.write_bytes(array.data)
 
       case _ if torch and isinstance(obj, torch.Tensor):
-        self.header("tensor", encode_val(obj), str(obj.dtype), tuple(obj.shape), obj.stride(), str(obj.device))
+        self.header("tensor", encode_type_of(obj), obj.dtype, tuple(obj.shape), obj.stride(), obj.device)
         if obj.device.type != "cpu":
           obj = obj.cpu()
         storage = obj.storage()
-        buffer = (ctypes.c_ubyte * (storage.nbytes())).from_address(storage.data_ptr())
-        self.update_hash(memoryview(buffer))
+        buffer = (ctypes.c_ubyte * storage.nbytes()).from_address(storage.data_ptr())
+        self.write_bytes(memoryview(buffer))
 
       case _ if id(obj) in self.current:
         self.header("circular", self.current[id(obj)])
@@ -145,36 +151,36 @@ class ObjectHash:
           self.current[id(obj)] = len(self.current)
           match obj:
             case list() | tuple():
-              self.header("list", encode_val(obj), len(obj)).update(iter=obj)
+              self.header("list", encode_type_of(obj), len(obj)).update(iter=obj)
             case dict():
               try:
                 items = sorted(obj.items())
+                header = "dict"
               except:
-                items = sorted((str(ObjectHash(key, tolerate_errors=self.tolerate_errors.value)), val) for key, val in obj.items())
-              self.header("dict", encode_val(obj), len(obj)).update(iter=chain.from_iterable(items))
+                items = sorted((self.nested_hash(key), val) for key, val in obj.items())
+                header = "dict-unsortable"
+              self.header(header, encode_type_of(obj), len(obj)).update(iter=chain.from_iterable(items))
             case _:
               self._update_object(obj)
         finally:
           del self.current[id(obj)]
 
-  def _update_iterator(self, obj: Iterable) -> None:
-    self.header("iterator", encode_val(obj)).update(iter=obj).header(b"iterator-end")
+  def _update_iterator(self, obj: Iterable) -> "ObjectHash":
+    return self.header("iterator", encode_type_of(obj)).update(iter=obj).header("iterator-end")
 
   def _update_object(self, obj: object) -> "ObjectHash":
-    self.header("instance", encode_val(obj))
-    try:
-      reduced = obj.__reduce_ex__(PROTOCOL) if hasattr(obj, "__reduce_ex__") else obj.__reduce__()
-    except:
-      reduced = None
+    self.header("instance", encode_type_of(obj))
+    reduced = None
+    with suppress(Exception):
+      reduced = obj.__reduce_ex__(PROTOCOL)
+    with suppress(Exception):
+      reduced = reduced or obj.__reduce__()
     if isinstance(reduced, str):
       return self.header("reduce-str").update(reduced)
     if reduced:
       reduced = list(reduced)
       it = reduced.pop(3) if len(reduced) >= 4 else None
-      self.header("reduce").update(reduced)
-      if it is not None:
-        self._update_iterator(it)
-      return self
+      return self.header("reduce").update(reduced)._update_iterator(it or ())
     if state := hasattr(obj, "__getstate__") and obj.__getstate__():
       return self.header("getstate").update(state)
     if len(getattr(obj, "__slots__", [])):

{checkpointer-2.5.0 → checkpointer-2.6.1}/checkpointer/storages/pickle_storage.py

@@ -18,6 +18,7 @@ class PickleStorage(Storage):
     return get_path(path).exists()
 
   def checkpoint_date(self, path):
+    # Should use st_atime/access time?
     return datetime.fromtimestamp(get_path(path).stat().st_mtime)
 
   def load(self, path):

{checkpointer-2.5.0 → checkpointer-2.6.1}/checkpointer/test_checkpointer.py

@@ -83,17 +83,6 @@ async def test_async_caching():
 
   assert result1 == result2 == 9
 
-def test_custom_path_caching():
-  def custom_path(a, b):
-    return f"add/{a}-{b}"
-
-  @checkpoint(path=custom_path)
-  def add(a, b):
-    return a + b
-
-  add(3, 4)
-  assert (checkpoint.root_path / "add/3-4.pkl").exists()
-
 def test_force_recalculation():
   @checkpoint
   def square(x: int) -> int:

{checkpointer-2.5.0 → checkpointer-2.6.1}/checkpointer/utils.py

@@ -23,7 +23,7 @@ def get_fn_body(fn: Callable) -> str:
   ignore_types = (tokenize.COMMENT, tokenize.NL)
   return "".join("\0" + token.string for token in tokens if token.type not in ignore_types)
 
-def get_cell_contents(fn: Callable) -> Generator[tuple[str, Any], None, None]:
+def get_cell_contents(fn: Callable) -> Iterable[tuple[str, Any]]:
   for key, cell in zip(fn.__code__.co_freevars, fn.__closure__ or []):
     try:
       yield (key, cell.cell_contents)

{checkpointer-2.5.0 → checkpointer-2.6.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "checkpointer"
-version = "2.5.0"
+version = "2.6.1"
 requires-python = ">=3.12"
 dependencies = []
 authors = [
@@ -9,6 +9,10 @@ authors = [
 description = "A Python library for memoizing function results with support for multiple storage backends, async runtimes, and automatic cache invalidation"
 readme = "README.md"
 license = {file = "LICENSE"}
+classifiers = [
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+]
 
 [project.urls]
 Repository = "https://github.com/Reddan/checkpointer.git"

{checkpointer-2.5.0 → checkpointer-2.6.1}/uv.lock

@@ -1,9 +1,10 @@
 version = 1
+revision = 1
 requires-python = ">=3.12"
 
 [[package]]
 name = "checkpointer"
-version = "2.5.0"
+version = "2.6.1"
 source = { editable = "." }
 
 [package.dev-dependencies]
@@ -177,7 +178,6 @@ name = "nvidia-cublas-cu12"
 version = "12.4.5.8"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/7f/7f/7fbae15a3982dc9595e49ce0f19332423b260045d0a6afe93cdbe2f1f624/nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_aarch64.whl", hash = "sha256:0f8aa1706812e00b9f19dfe0cdb3999b092ccb8ca168c0db5b8ea712456fd9b3", size = 363333771 },
     { url = "https://files.pythonhosted.org/packages/ae/71/1c91302526c45ab494c23f61c7a84aa568b8c1f9d196efa5993957faf906/nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl", hash = "sha256:2fc8da60df463fdefa81e323eef2e36489e1c94335b5358bcb38360adf75ac9b", size = 363438805 },
 ]
 
@@ -186,7 +186,6 @@ name = "nvidia-cuda-cupti-cu12"
 version = "12.4.127"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/93/b5/9fb3d00386d3361b03874246190dfec7b206fd74e6e287b26a8fcb359d95/nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_aarch64.whl", hash = "sha256:79279b35cf6f91da114182a5ce1864997fd52294a87a16179ce275773799458a", size = 12354556 },
     { url = "https://files.pythonhosted.org/packages/67/42/f4f60238e8194a3106d06a058d494b18e006c10bb2b915655bd9f6ea4cb1/nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:9dec60f5ac126f7bb551c055072b69d85392b13311fcc1bcda2202d172df30fb", size = 13813957 },
 ]
 
@@ -195,7 +194,6 @@ name = "nvidia-cuda-nvrtc-cu12"
 version = "12.4.127"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/77/aa/083b01c427e963ad0b314040565ea396f914349914c298556484f799e61b/nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_aarch64.whl", hash = "sha256:0eedf14185e04b76aa05b1fea04133e59f465b6f960c0cbf4e37c3cb6b0ea198", size = 24133372 },
     { url = "https://files.pythonhosted.org/packages/2c/14/91ae57cd4db3f9ef7aa99f4019cfa8d54cb4caa7e00975df6467e9725a9f/nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:a178759ebb095827bd30ef56598ec182b85547f1508941a3d560eb7ea1fbf338", size = 24640306 },
 ]
 
@@ -204,7 +202,6 @@ name = "nvidia-cuda-runtime-cu12"
 version = "12.4.127"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/a1/aa/b656d755f474e2084971e9a297def515938d56b466ab39624012070cb773/nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_aarch64.whl", hash = "sha256:961fe0e2e716a2a1d967aab7caee97512f71767f852f67432d572e36cb3a11f3", size = 894177 },
     { url = "https://files.pythonhosted.org/packages/ea/27/1795d86fe88ef397885f2e580ac37628ed058a92ed2c39dc8eac3adf0619/nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:64403288fa2136ee8e467cdc9c9427e0434110899d07c779f25b5c068934faa5", size = 883737 },
 ]
 
@@ -227,7 +224,6 @@ dependencies = [
     { name = "nvidia-nvjitlink-cu12" },
 ]
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/7a/8a/0e728f749baca3fbeffad762738276e5df60851958be7783af121a7221e7/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_aarch64.whl", hash = "sha256:5dad8008fc7f92f5ddfa2101430917ce2ffacd86824914c82e28990ad7f00399", size = 211422548 },
     { url = "https://files.pythonhosted.org/packages/27/94/3266821f65b92b3138631e9c8e7fe1fb513804ac934485a8d05776e1dd43/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl", hash = "sha256:f083fc24912aa410be21fa16d157fed2055dab1cc4b6934a0e03cba69eb242b9", size = 211459117 },
 ]
 
@@ -236,7 +232,6 @@ name = "nvidia-curand-cu12"
 version = "10.3.5.147"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/80/9c/a79180e4d70995fdf030c6946991d0171555c6edf95c265c6b2bf7011112/nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_aarch64.whl", hash = "sha256:1f173f09e3e3c76ab084aba0de819c49e56614feae5c12f69883f4ae9bb5fad9", size = 56314811 },
     { url = "https://files.pythonhosted.org/packages/8a/6d/44ad094874c6f1b9c654f8ed939590bdc408349f137f9b98a3a23ccec411/nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl", hash = "sha256:a88f583d4e0bb643c49743469964103aa59f7f708d862c3ddb0fc07f851e3b8b", size = 56305206 },
 ]
 
@@ -250,7 +245,6 @@ dependencies = [
     { name = "nvidia-nvjitlink-cu12" },
 ]
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/46/6b/a5c33cf16af09166845345275c34ad2190944bcc6026797a39f8e0a282e0/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_aarch64.whl", hash = "sha256:d338f155f174f90724bbde3758b7ac375a70ce8e706d70b018dd3375545fc84e", size = 127634111 },
     { url = "https://files.pythonhosted.org/packages/3a/e1/5b9089a4b2a4790dfdea8b3a006052cfecff58139d5a4e34cb1a51df8d6f/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl", hash = "sha256:19e33fa442bcfd085b3086c4ebf7e8debc07cfe01e11513cc6d332fd918ac260", size = 127936057 },
 ]
 
@@ -262,7 +256,6 @@ dependencies = [
     { name = "nvidia-nvjitlink-cu12" },
 ]
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/96/a9/c0d2f83a53d40a4a41be14cea6a0bf9e668ffcf8b004bd65633f433050c0/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_aarch64.whl", hash = "sha256:9d32f62896231ebe0480efd8a7f702e143c98cfaa0e8a76df3386c1ba2b54df3", size = 207381987 },
     { url = "https://files.pythonhosted.org/packages/db/f7/97a9ea26ed4bbbfc2d470994b8b4f338ef663be97b8f677519ac195e113d/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl", hash = "sha256:ea4f11a2904e2a8dc4b1833cc1b5181cde564edd0d5cd33e3c168eff2d1863f1", size = 207454763 },
 ]
 
@@ -279,7 +272,6 @@ name = "nvidia-nvjitlink-cu12"
 version = "12.4.127"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/02/45/239d52c05074898a80a900f49b1615d81c07fceadd5ad6c4f86a987c0bc4/nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_aarch64.whl", hash = "sha256:4abe7fef64914ccfa909bc2ba39739670ecc9e820c83ccc7a6ed414122599b83", size = 20552510 },
     { url = "https://files.pythonhosted.org/packages/ff/ff/847841bacfbefc97a00036e0fce5a0f086b640756dc38caea5e1bb002655/nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:06b3b9b25bf3f8af351d664978ca26a16d2c5127dbd53c0497e28d1fb9611d57", size = 21066810 },
 ]
 
@@ -288,7 +280,6 @@ name = "nvidia-nvtx-cu12"
 version = "12.4.127"
 source = { registry = "https://pypi.org/simple" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/06/39/471f581edbb7804b39e8063d92fc8305bdc7a80ae5c07dbe6ea5c50d14a5/nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_aarch64.whl", hash = "sha256:7959ad635db13edf4fc65c06a6e9f9e55fc2f92596db928d169c0bb031e88ef3", size = 100417 },
     { url = "https://files.pythonhosted.org/packages/87/20/199b8713428322a2f22b722c62b8cc278cc53dffa9705d744484b5035ee9/nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl", hash = "sha256:781e950d9b9f60d8241ccea575b32f5105a5baf4c2351cab5256a24869f12a1a", size = 99144 },
 ]
 
@@ -459,21 +450,21 @@ dependencies = [
     { name = "fsspec" },
     { name = "jinja2" },
     { name = "networkx" },
-    { name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cuda-cupti-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cuda-nvrtc-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cusolver-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-cusparse-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-nccl-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-nvjitlink-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
-    { name = "nvidia-nvtx-cu12", marker = "platform_machine == 'x86_64' and platform_system == 'Linux'" },
+    { name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cuda-cupti-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cuda-nvrtc-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cusolver-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-cusparse-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nccl-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nvjitlink-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
+    { name = "nvidia-nvtx-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
     { name = "setuptools" },
     { name = "sympy" },
-    { name = "triton", marker = "python_full_version < '3.13' and platform_machine == 'x86_64' and platform_system == 'Linux'" },
+    { name = "triton", marker = "python_full_version < '3.13' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
     { name = "typing-extensions" },
 ]
 wheels = [