checkpointer 2.0.2__tar.gz → 2.1.0__tar.gz

{checkpointer-2.0.2 → checkpointer-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: checkpointer
-Version: 2.0.2
+Version: 2.1.0
 Summary: A Python library for memoizing function results with support for multiple storage backends, async runtimes, and automatic cache invalidation
 Project-URL: Repository, https://github.com/Reddan/checkpointer.git
 Author: Hampus Hallman
@@ -19,7 +19,7 @@ Description-Content-Type: text/markdown
 
 `checkpointer` is a Python library for memoizing function results. It provides a decorator-based API with support for multiple storage backends. Use it for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations.
 
-Adding or removing `@checkpoint` doesn't change how your code works, and it can be applied to any function, including ones you've already written, without altering their behavior or introducing side effects. The original function remains unchanged and can still be called directly when needed.
+Adding or removing `@checkpoint` doesn't change how your code works. You can apply it to any function, including ones you've already written, without altering their behavior or introducing side effects. The original function remains unchanged and can still be called directly when needed.
 
 ### Key Features:
 - 🗂️ **Multiple Storage Backends**: Built-in support for in-memory and pickle-based storage, or create your own.
@@ -27,6 +27,7 @@ Adding or removing `@checkpoint` doesn't change how your code works, and it can
 - 🔄 **Async and Sync Compatibility**: Works with synchronous functions and any Python async runtime (e.g., `asyncio`, `Trio`, `Curio`).
 - ⏲️ **Custom Expiration Logic**: Automatically invalidate old checkpoints.
 - 📂 **Flexible Path Configuration**: Control where checkpoints are stored.
+- 📦 **Captured Variables Handling**: Optionally include captured variables in cache invalidation.
 
 ---
 
@@ -59,8 +60,10 @@ result = expensive_function(4)  # Loads from the cache
 When you use `@checkpoint`, the function's **arguments** (`args`, `kwargs`) are hashed to create a unique identifier for each call. This identifier is used to store and retrieve cached results. If the same arguments are passed again, `checkpointer` loads the cached result instead of recomputing.
 
 Additionally, `checkpointer` ensures that caches are invalidated when a function's implementation or any of its dependencies change. Each function is assigned a hash based on:
+
 1. **Its source code**: Changes to the function's code update its hash.
 2. **Dependent functions**: If a function calls others, changes in those dependencies will also update the hash.
+3. **Captured variables**: (Optional) If `capture=True`, changes to captured variables and global variables will also update the hash.
 
 ### Example: Cache Invalidation
 
@@ -115,7 +118,17 @@ def some_expensive_function():
 
 ## Usage
 
+### Basic Invocation and Caching
+
+Call the decorated function as usual. On the first call, the result is computed and stored in the cache. Subsequent calls with the same arguments load the result from the cache:
+
+```python
+result = expensive_function(4)  # Computes and stores the result
+result = expensive_function(4)  # Loads the result from the cache
+```
+
 ### Force Recalculation
+
 Force a recalculation and overwrite the stored checkpoint:
 
 ```python
@@ -123,6 +136,7 @@ result = expensive_function.rerun(4)
 ```
 
 ### Call the Original Function
+
 Use `fn` to directly call the original, undecorated function:
 
 ```python
@@ -132,6 +146,7 @@ result = expensive_function.fn(4)
 This is especially useful **inside recursive functions** to avoid redundant caching of intermediate steps while still caching the final result.
 
 ### Retrieve Stored Checkpoints
+
 Access cached results without recalculating:
 
 ```python
@@ -154,11 +169,11 @@ You can specify a storage backend using either its name (`"pickle"` or `"memory"
 ```python
 from checkpointer import checkpoint, PickleStorage, MemoryStorage
 
-@checkpoint(format="pickle")  # Equivalent to format=PickleStorage
+@checkpoint(format="pickle")  # Short for format=PickleStorage
 def disk_cached(x: int) -> int:
     return x ** 2
 
-@checkpoint(format="memory")  # Equivalent to format=MemoryStorage
+@checkpoint(format="memory")  # Short for format=MemoryStorage
 def memory_cached(x: int) -> int:
     return x * 10
 ```
@@ -191,14 +206,15 @@ Using a custom backend lets you tailor storage to your application, whether it i
 
 ## Configuration Options ⚙️
 
-| Option         | Type                                | Default     | Description                                 |
-|----------------|-------------------------------------|-------------|---------------------------------------------|
-| `format`       | `"pickle"`, `"memory"`, `Storage`   | `"pickle"`  | Storage backend format.                     |
-| `root_path`    | `Path`, `str`, or `None`            | User Cache  | Root directory for storing checkpoints.     |
-| `when`         | `bool`                              | `True`      | Enable or disable checkpointing.            |
-| `verbosity`    | `0` or `1`                          | `1`         | Logging verbosity.                          |
-| `path`         | `Callable[..., str]`                | `None`      | Custom path for checkpoint storage.         |
-| `should_expire`| `Callable[[datetime], bool]`        | `None`      | Custom expiration logic.                    |
+| Option          | Type                              | Default              | Description                                    |
+|-----------------|-----------------------------------|----------------------|------------------------------------------------|
+| `capture`       | `bool`                            | `False`              | Include captured variables in function hashes. |
+| `format`        | `"pickle"`, `"memory"`, `Storage` | `"pickle"`           | Storage backend format.                        |
+| `root_path`     | `Path`, `str`, or `None`          | ~/.cache/checkpoints | Root directory for storing checkpoints.        |
+| `when`          | `bool`                            | `True`               | Enable or disable checkpointing.               |
+| `verbosity`     | `0` or `1`                        | `1`                  | Logging verbosity.                             |
+| `path`          | `Callable[..., str]`              | `None`               | Custom path for checkpoint storage.            |
+| `should_expire` | `Callable[[datetime], bool]`      | `None`               | Custom expiration logic.                       |
 
 ---
 
@@ -220,13 +236,13 @@ async def async_compute_sum(a: int, b: int) -> int:
 
 async def main():
     result1 = compute_square(5)
-    print(result1)
+    print(result1)  # Outputs 25
 
     result2 = await async_compute_sum(3, 7)
-    print(result2)
+    print(result2)  # Outputs 10
 
-    result3 = async_compute_sum.get(3, 7)
-    print(result3)
+    result3 = await async_compute_sum.get(3, 7)
+    print(result3)  # Outputs 10
 
 asyncio.run(main())
 ```

{checkpointer-2.0.2 → checkpointer-2.1.0}/README.md

@@ -2,7 +2,7 @@
 
 `checkpointer` is a Python library for memoizing function results. It provides a decorator-based API with support for multiple storage backends. Use it for computationally expensive operations where caching can save time, or during development to avoid waiting for redundant computations.
 
-Adding or removing `@checkpoint` doesn't change how your code works, and it can be applied to any function, including ones you've already written, without altering their behavior or introducing side effects. The original function remains unchanged and can still be called directly when needed.
+Adding or removing `@checkpoint` doesn't change how your code works. You can apply it to any function, including ones you've already written, without altering their behavior or introducing side effects. The original function remains unchanged and can still be called directly when needed.
 
 ### Key Features:
 - 🗂️ **Multiple Storage Backends**: Built-in support for in-memory and pickle-based storage, or create your own.
@@ -10,6 +10,7 @@ Adding or removing `@checkpoint` doesn't change how your code works, and it can
 - 🔄 **Async and Sync Compatibility**: Works with synchronous functions and any Python async runtime (e.g., `asyncio`, `Trio`, `Curio`).
 - ⏲️ **Custom Expiration Logic**: Automatically invalidate old checkpoints.
 - 📂 **Flexible Path Configuration**: Control where checkpoints are stored.
+- 📦 **Captured Variables Handling**: Optionally include captured variables in cache invalidation.
 
 ---
 
@@ -42,8 +43,10 @@ result = expensive_function(4)  # Loads from the cache
 When you use `@checkpoint`, the function's **arguments** (`args`, `kwargs`) are hashed to create a unique identifier for each call. This identifier is used to store and retrieve cached results. If the same arguments are passed again, `checkpointer` loads the cached result instead of recomputing.
 
 Additionally, `checkpointer` ensures that caches are invalidated when a function's implementation or any of its dependencies change. Each function is assigned a hash based on:
+
 1. **Its source code**: Changes to the function's code update its hash.
 2. **Dependent functions**: If a function calls others, changes in those dependencies will also update the hash.
+3. **Captured variables**: (Optional) If `capture=True`, changes to captured variables and global variables will also update the hash.
 
 ### Example: Cache Invalidation
 
@@ -98,7 +101,17 @@ def some_expensive_function():
 
 ## Usage
 
+### Basic Invocation and Caching
+
+Call the decorated function as usual. On the first call, the result is computed and stored in the cache. Subsequent calls with the same arguments load the result from the cache:
+
+```python
+result = expensive_function(4)  # Computes and stores the result
+result = expensive_function(4)  # Loads the result from the cache
+```
+
 ### Force Recalculation
+
 Force a recalculation and overwrite the stored checkpoint:
 
 ```python
@@ -106,6 +119,7 @@ result = expensive_function.rerun(4)
 ```
 
 ### Call the Original Function
+
 Use `fn` to directly call the original, undecorated function:
 
 ```python
@@ -115,6 +129,7 @@ result = expensive_function.fn(4)
 This is especially useful **inside recursive functions** to avoid redundant caching of intermediate steps while still caching the final result.
 
 ### Retrieve Stored Checkpoints
+
 Access cached results without recalculating:
 
 ```python
@@ -137,11 +152,11 @@ You can specify a storage backend using either its name (`"pickle"` or `"memory"
 ```python
 from checkpointer import checkpoint, PickleStorage, MemoryStorage
 
-@checkpoint(format="pickle")  # Equivalent to format=PickleStorage
+@checkpoint(format="pickle")  # Short for format=PickleStorage
 def disk_cached(x: int) -> int:
     return x ** 2
 
-@checkpoint(format="memory")  # Equivalent to format=MemoryStorage
+@checkpoint(format="memory")  # Short for format=MemoryStorage
 def memory_cached(x: int) -> int:
     return x * 10
 ```
@@ -174,14 +189,15 @@ Using a custom backend lets you tailor storage to your application, whether it i
 
 ## Configuration Options ⚙️
 
-| Option         | Type                                | Default     | Description                                 |
-|----------------|-------------------------------------|-------------|---------------------------------------------|
-| `format`       | `"pickle"`, `"memory"`, `Storage`   | `"pickle"`  | Storage backend format.                     |
-| `root_path`    | `Path`, `str`, or `None`            | User Cache  | Root directory for storing checkpoints.     |
-| `when`         | `bool`                              | `True`      | Enable or disable checkpointing.            |
-| `verbosity`    | `0` or `1`                          | `1`         | Logging verbosity.                          |
-| `path`         | `Callable[..., str]`                | `None`      | Custom path for checkpoint storage.         |
-| `should_expire`| `Callable[[datetime], bool]`        | `None`      | Custom expiration logic.                    |
+| Option          | Type                              | Default              | Description                                    |
+|-----------------|-----------------------------------|----------------------|------------------------------------------------|
+| `capture`       | `bool`                            | `False`              | Include captured variables in function hashes. |
+| `format`        | `"pickle"`, `"memory"`, `Storage` | `"pickle"`           | Storage backend format.                        |
+| `root_path`     | `Path`, `str`, or `None`          | ~/.cache/checkpoints | Root directory for storing checkpoints.        |
+| `when`          | `bool`                            | `True`               | Enable or disable checkpointing.               |
+| `verbosity`     | `0` or `1`                        | `1`                  | Logging verbosity.                             |
+| `path`          | `Callable[..., str]`              | `None`               | Custom path for checkpoint storage.            |
+| `should_expire` | `Callable[[datetime], bool]`      | `None`               | Custom expiration logic.                       |
 
 ---
 
@@ -203,13 +219,13 @@ async def async_compute_sum(a: int, b: int) -> int:
 
 async def main():
     result1 = compute_square(5)
-    print(result1)
+    print(result1)  # Outputs 25
 
     result2 = await async_compute_sum(3, 7)
-    print(result2)
+    print(result2)  # Outputs 10
 
-    result3 = async_compute_sum.get(3, 7)
-    print(result3)
+    result3 = await async_compute_sum.get(3, 7)
+    print(result3)  # Outputs 10
 
 asyncio.run(main())
 ```

{checkpointer-2.0.2 → checkpointer-2.1.0}/checkpointer/__init__.py

@@ -5,5 +5,6 @@ import tempfile
 
 create_checkpointer = Checkpointer
 checkpoint = Checkpointer()
+capture_checkpoint = Checkpointer(capture=True)
 memory_checkpoint = Checkpointer(format="memory", verbosity=0)
 tmp_checkpoint = Checkpointer(root_path=tempfile.gettempdir() + "/checkpoints")

{checkpointer-2.0.2 → checkpointer-2.1.0}/checkpointer/checkpoint.py

@@ -7,16 +7,13 @@ from datetime import datetime
 from functools import update_wrapper
 from .types import Storage
 from .function_body import get_function_hash
-from .utils import unwrap_fn, sync_resolve_coroutine
-from .storages.pickle_storage import PickleStorage
-from .storages.memory_storage import MemoryStorage
-from .storages.bcolz_storage import BcolzStorage
+from .utils import unwrap_fn, sync_resolve_coroutine, resolved_awaitable
+from .storages import STORAGE_MAP
 from .print_checkpoint import print_checkpoint
 
 Fn = TypeVar("Fn", bound=Callable)
 
 DEFAULT_DIR = Path.home() / ".cache/checkpoints"
-STORAGE_MAP: dict[str, Type[Storage]] = {"memory": MemoryStorage, "pickle": PickleStorage, "bcolz": BcolzStorage}
 
 class CheckpointError(Exception):
   pass
@@ -28,6 +25,7 @@ class CheckpointerOpts(TypedDict, total=False):
   verbosity: Literal[0, 1]
   path: Callable[..., str] | None
   should_expire: Callable[[datetime], bool] | None
+  capture: bool
 
 class Checkpointer:
   def __init__(self, **opts: Unpack[CheckpointerOpts]):
@@ -37,6 +35,7 @@ class Checkpointer:
     self.verbosity = opts.get("verbosity", 1)
     self.path = opts.get("path")
     self.should_expire = opts.get("should_expire")
+    self.capture = opts.get("capture", False)
 
   @overload
   def __call__(self, fn: Fn, **override_opts: Unpack[CheckpointerOpts]) -> CheckpointFn[Fn]: ...
@@ -57,14 +56,16 @@ class CheckpointFn(Generic[Fn]):
     storage = STORAGE_MAP[checkpointer.format] if isinstance(checkpointer.format, str) else checkpointer.format
     self.checkpointer = checkpointer
     self.fn = fn
-    self.fn_hash = get_function_hash(wrapped)
+    self.fn_hash, self.depends = get_function_hash(wrapped, self.checkpointer.capture)
     self.fn_id = f"{file_name}/{wrapped.__name__}"
     self.is_async = inspect.iscoroutinefunction(wrapped)
     self.storage = storage(checkpointer)
 
   def get_checkpoint_id(self, args: tuple, kw: dict) -> str:
     if not callable(self.checkpointer.path):
-      return f"{self.fn_id}/{hashing.hash([self.fn_hash, args, kw or 0])}"
+      # TODO: use digest size before digesting instead of truncating the hash
+      call_hash = hashing.hash((self.fn_hash, args, kw), "blake2b")[:32]
+      return f"{self.fn_id}/{call_hash}"
     checkpoint_id = self.checkpointer.path(*args, **kw)
     if not isinstance(checkpoint_id, str):
       raise CheckpointError(f"path function must return a string, got {type(checkpoint_id)}")
@@ -101,12 +102,17 @@ class CheckpointFn(Generic[Fn]):
     coroutine = self._store_on_demand(args, kw, rerun)
     return coroutine if self.is_async else sync_resolve_coroutine(coroutine)
 
-  __call__: Fn = cast(Fn, lambda self, *args, **kw: self._call(args, kw))
-  rerun: Fn = cast(Fn, lambda self, *args, **kw: self._call(args, kw, True))
-
-  def get(self, *args, **kw) -> Any:
+  def _get(self, args, kw) -> Any:
     checkpoint_path = self.checkpointer.root_path / self.get_checkpoint_id(args, kw)
     try:
-      return self.storage.load(checkpoint_path)
+      val = self.storage.load(checkpoint_path)
+      return resolved_awaitable(val) if self.is_async else val
     except:
       raise CheckpointError("Could not load checkpoint")
+
+  def exists(self, *args: tuple, **kw: dict) -> bool:
+    return self.storage.exists(self.checkpointer.root_path / self.get_checkpoint_id(args, kw))
+
+  __call__: Fn = cast(Fn, lambda self, *args, **kw: self._call(args, kw))
+  rerun: Fn = cast(Fn, lambda self, *args, **kw: self._call(args, kw, True))
+  get: Fn = cast(Fn, lambda self, *args, **kw: self._get(args, kw))

checkpointer-2.1.0/checkpointer/function_body.py

@@ -0,0 +1,80 @@
+from __future__ import annotations
+import dis
+import inspect
+import tokenize
+from io import StringIO
+from collections.abc import Callable
+from itertools import chain, takewhile
+from operator import itemgetter
+from pathlib import Path
+from typing import Any, TypeGuard, TYPE_CHECKING
+from types import CodeType, FunctionType
+from relib import transpose, hashing, merge_dicts, drop_none
+from .utils import unwrap_fn, iterate_and_upcoming, get_cell_contents, AttrDict, get_at_attr
+
+if TYPE_CHECKING:
+  from .checkpoint import CheckpointFn
+
+cwd = Path.cwd()
+
+def extract_scope_values(code: CodeType, scope_vars: dict[str, Any], closure = False) -> dict[tuple[str, ...], Any]:
+  opname = "LOAD_GLOBAL" if not closure else "LOAD_DEREF"
+  scope_values_by_path: dict[tuple[str, ...], Any] = {}
+  instructions = list(dis.get_instructions(code))
+
+  for instr, upcoming_instrs in iterate_and_upcoming(instructions):
+    if instr.opname == opname:
+      name = instr.argval
+      attrs = takewhile(lambda instr: instr.opname == "LOAD_ATTR", upcoming_instrs)
+      attr_path = (name, *(instr.argval for instr in attrs))
+      scope_values_by_path[attr_path] = get_at_attr(scope_vars, attr_path)
+
+  children = (extract_scope_values(const, scope_vars, closure) for const in code.co_consts if isinstance(const, CodeType))
+  return merge_dicts(scope_values_by_path, *children)
+
+def get_fn_captured_vals(fn: Callable) -> list[Any]:
+  closure_scope = {k: get_cell_contents(v) for k, v in zip(fn.__code__.co_freevars, fn.__closure__ or [])}
+  global_vals = extract_scope_values(fn.__code__, AttrDict(fn.__globals__), closure=False)
+  closure_vals = extract_scope_values(fn.__code__, AttrDict(closure_scope), closure=True)
+  sorted_items = chain(sorted(global_vals.items()), sorted(closure_vals.items()))
+  return drop_none(map(itemgetter(1), sorted_items))
+
+def get_fn_body(fn: Callable) -> str:
+  source = "".join(inspect.getsourcelines(fn)[0])
+  tokens = tokenize.generate_tokens(StringIO(source).readline)
+  ignore_types = (tokenize.COMMENT, tokenize.NL)
+  return "".join("\0" + token.string for token in tokens if token.type not in ignore_types)
+
+def get_fn_path(fn: Callable) -> Path:
+  return Path(inspect.getfile(fn)).resolve()
+
+def is_user_fn(candidate_fn) -> TypeGuard[Callable]:
+  return isinstance(candidate_fn, FunctionType) \
+    and cwd in get_fn_path(candidate_fn).parents
+
+def append_fn_depends(checkpoint_fns: set[CheckpointFn], captured_vals_by_fn: dict[Callable, list[Any]], fn: Callable, capture: bool) -> None:
+  from .checkpoint import CheckpointFn
+  captured_vals = get_fn_captured_vals(fn)
+  captured_vals_by_fn[fn] = [v for v in captured_vals if capture and not callable(v)]
+  callables = [unwrap_fn(val, checkpoint_fn=True) for val in captured_vals if callable(val)]
+  depends = {val for val in callables if is_user_fn(val)}
+  checkpoint_fns.update({val for val in callables if isinstance(val, CheckpointFn)})
+  not_appended = depends - captured_vals_by_fn.keys()
+  captured_vals_by_fn.update({fn: [] for fn in not_appended})
+  for child_fn in not_appended:
+    append_fn_depends(checkpoint_fns, captured_vals_by_fn, child_fn, capture)
+
+def get_depend_fns(fn: Callable, capture: bool) -> tuple[set[CheckpointFn], dict[Callable, list[Any]]]:
+  checkpoint_fns: set[CheckpointFn] = set()
+  captured_vals_by_fn: dict[Callable, list[Any]] = {}
+  append_fn_depends(checkpoint_fns, captured_vals_by_fn, fn, capture)
+  return checkpoint_fns, captured_vals_by_fn
+
+def get_function_hash(fn: Callable, capture: bool) -> tuple[str, list[Callable]]:
+  checkpoint_fns, captured_vals_by_fn = get_depend_fns(fn, capture)
+  checkpoint_fns = sorted(checkpoint_fns, key=lambda fn: unwrap_fn(fn).__qualname__)
+  checkpoint_hashes = [check.fn_hash for check in checkpoint_fns]
+  depend_fns, depend_captured_vals = transpose(sorted(captured_vals_by_fn.items(), key=lambda x: x[0].__qualname__), 2)
+  fn_bodies = list(map(get_fn_body, [fn] + depend_fns))
+  fn_hash = hashing.hash((fn_bodies, depend_captured_vals, checkpoint_hashes), "blake2b")
+  return fn_hash, checkpoint_fns + depend_fns

checkpointer-2.1.0/checkpointer/storages/__init__.py

@@ -0,0 +1,11 @@
+from typing import Type
+from ..types import Storage
+from .pickle_storage import PickleStorage
+from .memory_storage import MemoryStorage
+from .bcolz_storage import BcolzStorage
+
+STORAGE_MAP: dict[str, Type[Storage]] = {
+  "pickle": PickleStorage,
+  "memory": MemoryStorage,
+  "bcolz": BcolzStorage,
+}

checkpointer-2.1.0/checkpointer/utils.py

@@ -0,0 +1,52 @@
+from typing import Generator, Coroutine, Iterable, Any, cast
+from types import CellType, coroutine
+from itertools import islice
+
+class AttrDict(dict):
+  def __init__(self, *args, **kwargs):
+    super(AttrDict, self).__init__(*args, **kwargs)
+    self.__dict__ = self
+
+  def __getattribute__(self, name: str) -> Any:
+    return super().__getattribute__(name)
+
+def unwrap_fn[T](fn: T, checkpoint_fn=False) -> T:
+  from .checkpoint import CheckpointFn
+  while hasattr(fn, "__wrapped__"):
+    if checkpoint_fn and isinstance(fn, CheckpointFn):
+      return fn
+    fn = getattr(fn, "__wrapped__")
+  return fn
+
+@coroutine
+def coroutine_as_generator[T](coroutine: Coroutine[None, None, T]) -> Generator[None, None, T]:
+  val = yield from coroutine
+  return val
+
+def sync_resolve_coroutine[T](coroutine: Coroutine[None, None, T]) -> T:
+  gen = cast(Generator, coroutine_as_generator(coroutine))
+  try:
+    while True: next(gen)
+  except StopIteration as ex:
+    return ex.value
+
+async def resolved_awaitable[T](value: T) -> T:
+  return value
+
+def iterate_and_upcoming[T](l: list[T]) -> Iterable[tuple[T, Iterable[T]]]:
+  for i, item in enumerate(l):
+    yield item, islice(l, i + 1, None)
+
+def get_at_attr(d: dict, keys: tuple[str, ...]) -> Any:
+  try:
+    for key in keys:
+      d = getattr(d, key)
+  except AttributeError:
+    return None
+  return d
+
+def get_cell_contents(cell: CellType) -> Any:
+  try:
+    return cell.cell_contents
+  except ValueError:
+    return None

{checkpointer-2.0.2 → checkpointer-2.1.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "checkpointer"
-version = "2.0.2"
+version = "2.1.0"
 requires-python = ">=3.12"
 dependencies = [
   "relib",

{checkpointer-2.0.2 → checkpointer-2.1.0}/uv.lock

@@ -3,7 +3,7 @@ requires-python = ">=3.12"
 
 [[package]]
 name = "checkpointer"
-version = "2.0.2"
+version = "2.1.0"
 source = { editable = "." }
 dependencies = [
     { name = "relib" },

checkpointer-2.0.2/checkpointer/function_body.py

@@ -1,46 +0,0 @@
-import inspect
-import relib.hashing as hashing
-from collections.abc import Callable
-from types import FunctionType, CodeType
-from pathlib import Path
-from .utils import unwrap_fn
-
-cwd = Path.cwd()
-
-def get_fn_path(fn: Callable) -> Path:
-  return Path(inspect.getfile(fn)).resolve()
-
-def get_function_body(fn: Callable) -> str:
-  # TODO: Strip comments
-  lines = inspect.getsourcelines(fn)[0]
-  lines = [line.rstrip() for line in lines]
-  lines = [line for line in lines if line]
-  return "\n".join(lines)
-
-def get_code_children(code: CodeType) -> list[str]:
-  consts = [const for const in code.co_consts if isinstance(const, CodeType)]
-  children = [child for const in consts for child in get_code_children(const)]
-  return list(code.co_names) + children
-
-def is_user_fn(candidate_fn, cleared_fns: set[Callable]) -> bool:
-  return isinstance(candidate_fn, FunctionType) \
-    and candidate_fn not in cleared_fns \
-    and cwd in get_fn_path(candidate_fn).parents
-
-def append_fn_children(cleared_fns: set[Callable], fn: Callable) -> None:
-  code_children = get_code_children(fn.__code__)
-  fn_children = [unwrap_fn(fn.__globals__.get(co_name, None)) for co_name in code_children]
-  fn_children = [child for child in fn_children if is_user_fn(child, cleared_fns)]
-  cleared_fns.update(fn_children)
-  for child_fn in fn_children:
-    append_fn_children(cleared_fns, child_fn)
-
-def get_fn_children(fn: Callable) -> list[Callable]:
-  cleared_fns: set[Callable] = set()
-  append_fn_children(cleared_fns, fn)
-  return sorted(cleared_fns, key=lambda fn: fn.__name__)
-
-def get_function_hash(fn: Callable) -> str:
-  fns = [fn] + get_fn_children(fn)
-  fn_bodies = list(map(get_function_body, fns))
-  return hashing.hash(fn_bodies)

checkpointer-2.0.2/checkpointer/utils.py

@@ -1,17 +0,0 @@
-import types
-
-def unwrap_fn[T](fn: T) -> T:
-  while hasattr(fn, "__wrapped__"):
-    fn = getattr(fn, "__wrapped__")
-  return fn
-
-@types.coroutine
-def coroutine_as_generator(coroutine):
-  val = yield from coroutine
-  return val
-
-def sync_resolve_coroutine(coroutine):
-  try:
-    next(coroutine_as_generator(coroutine))
-  except StopIteration as ex:
-    return ex.value