lazyscribe 2.0.0__tar.gz

lazyscribe-2.0.0/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.3
+Name: lazyscribe
+Version: 2.0.0
+Summary: Lightweight and lazy experiment logging
+Author: Akshay Gupta
+Author-email: Akshay Gupta <akgcodes@gmail.com>
+License: MIT license
+Classifier: Development Status :: 4 - Beta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: attrs>=21.2.0,<=25.4.0
+Requires-Dist: fsspec>=0.4.0,<=2025.12.0
+Requires-Dist: python-slugify>=5.0.0,<=8.0.4
+Requires-Dist: tomli>=2.2.1,<=2.3.0 ; python_full_version < '3.11'
+Requires-Dist: commitizen ; extra == 'build'
+Requires-Dist: uv ; extra == 'build'
+Requires-Dist: lazyscribe[build] ; extra == 'dev'
+Requires-Dist: lazyscribe[docs] ; extra == 'dev'
+Requires-Dist: lazyscribe[qa] ; extra == 'dev'
+Requires-Dist: lazyscribe[tests] ; extra == 'dev'
+Requires-Dist: furo ; extra == 'docs'
+Requires-Dist: matplotlib ; extra == 'docs'
+Requires-Dist: pandas ; extra == 'docs'
+Requires-Dist: pillow ; extra == 'docs'
+Requires-Dist: scikit-learn ; extra == 'docs'
+Requires-Dist: sphinx ; extra == 'docs'
+Requires-Dist: sphinx-gallery ; extra == 'docs'
+Requires-Dist: sphinx-inline-tabs ; extra == 'docs'
+Requires-Dist: time-machine ; extra == 'docs'
+Requires-Dist: edgetest ; extra == 'qa'
+Requires-Dist: edgetest-pip-tools ; extra == 'qa'
+Requires-Dist: mypy ; extra == 'qa'
+Requires-Dist: pip-tools ; extra == 'qa'
+Requires-Dist: pre-commit ; extra == 'qa'
+Requires-Dist: pyproject-fmt ; extra == 'qa'
+Requires-Dist: ruff ; extra == 'qa'
+Requires-Dist: types-python-slugify ; extra == 'qa'
+Requires-Dist: pytest ; extra == 'tests'
+Requires-Dist: pytest-cov ; extra == 'tests'
+Requires-Dist: requests ; extra == 'tests'
+Requires-Dist: scikit-learn ; extra == 'tests'
+Requires-Dist: time-machine ; extra == 'tests'
+Requires-Python: >=3.10.0
+Project-URL: documentation, https://lazyscribe.github.io/lazyscribe/
+Project-URL: repository, https://github.com/lazyscribe/lazyscribe
+Provides-Extra: build
+Provides-Extra: dev
+Provides-Extra: docs
+Provides-Extra: qa
+Provides-Extra: tests
+Description-Content-Type: text/markdown
+
+[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) [![PyPI](https://img.shields.io/pypi/v/lazyscribe)](https://pypi.org/project/lazyscribe/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/lazyscribe)](https://pypi.org/project/lazyscribe/) [![Documentation Status](https://github.com/lazyscribe/lazyscribe/actions/workflows/docs.yml/badge.svg)](https://lazyscribe.github.io/lazyscribe/) [![codecov](https://codecov.io/github/lazyscribe/lazyscribe/branch/main/graph/badge.svg?token=M5BHYS2SSU)](https://codecov.io/github/lazyscribe/lazyscribe)
+
+# Lightweight, lazy experiment logging
+
+``lazyscribe`` is a lightweight package for model experiment logging. It creates a single JSON
+file per project, and an experiment is only added to the file when code finishes (errors won't
+result in partially finished experiments in your project log).
+
+``lazyscribe`` also has functionality to allow for multiple people to work on a single project.
+You can merge projects together and update the list of experiments to create a single, authoritative
+view of all executed experiments.
+
+# Installation
+
+Python 3.10 and above is required. Use `pip` to install:
+```console
+$ python -m pip install lazyscribe
+```
+
+# Basic Usage
+
+The basic usage involves instantiating a ``Project`` and using the context manager to log
+an experiment:
+
+```python
+import json
+
+from lazyscribe import Project
+
+project = Project(fpath="project.json")
+with project.log(name="My experiment") as exp:
+    exp.log_metric("auroc", 0.5)
+    exp.log_parameter("algorithm", "lightgbm")
+```
+
+You've created an experiment! You can view the experimental data by using ``list``:
+
+```python
+print(json.dumps(list(project), indent=4))
+```
+
+```json
+[
+    {
+        "name": "My experiment",
+        "author": "<AUTHOR>",
+        "last_updated_by": "<AUTHOR>",
+        "metrics": {
+            "auroc": 0.5
+        },
+        "parameters": {
+            "algorithm": "lightgbm"
+        },
+        "created_at": "<CREATED_AT>",
+        "last_updated": "<LAST_UPDATED>",
+        "dependencies": [],
+        "short_slug": "my-experiment",
+        "slug": "my-experiment-<CREATED_AT>",
+        "tests": [],
+        "artifacts": []
+    }
+]
+```
+
+Once you've finished, save the project to ``project.json``:
+
+```python
+project.save()
+```
+
+Later on, you can read the project back in read-only mode (`"r"`), append mode (`"a"`),
+or editable mode (`"w+"`):
+
+```python
+project = Project("project.json", mode="r")
+with project.log(name="New experiment") as exp:  # Raises a ReadOnlyError
+    ...
+```

lazyscribe-2.0.0/README.md

@@ -0,0 +1,78 @@
+[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) [![PyPI](https://img.shields.io/pypi/v/lazyscribe)](https://pypi.org/project/lazyscribe/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/lazyscribe)](https://pypi.org/project/lazyscribe/) [![Documentation Status](https://github.com/lazyscribe/lazyscribe/actions/workflows/docs.yml/badge.svg)](https://lazyscribe.github.io/lazyscribe/) [![codecov](https://codecov.io/github/lazyscribe/lazyscribe/branch/main/graph/badge.svg?token=M5BHYS2SSU)](https://codecov.io/github/lazyscribe/lazyscribe)
+
+# Lightweight, lazy experiment logging
+
+``lazyscribe`` is a lightweight package for model experiment logging. It creates a single JSON
+file per project, and an experiment is only added to the file when code finishes (errors won't
+result in partially finished experiments in your project log).
+
+``lazyscribe`` also has functionality to allow for multiple people to work on a single project.
+You can merge projects together and update the list of experiments to create a single, authoritative
+view of all executed experiments.
+
+# Installation
+
+Python 3.10 and above is required. Use `pip` to install:
+```console
+$ python -m pip install lazyscribe
+```
+
+# Basic Usage
+
+The basic usage involves instantiating a ``Project`` and using the context manager to log
+an experiment:
+
+```python
+import json
+
+from lazyscribe import Project
+
+project = Project(fpath="project.json")
+with project.log(name="My experiment") as exp:
+    exp.log_metric("auroc", 0.5)
+    exp.log_parameter("algorithm", "lightgbm")
+```
+
+You've created an experiment! You can view the experimental data by using ``list``:
+
+```python
+print(json.dumps(list(project), indent=4))
+```
+
+```json
+[
+    {
+        "name": "My experiment",
+        "author": "<AUTHOR>",
+        "last_updated_by": "<AUTHOR>",
+        "metrics": {
+            "auroc": 0.5
+        },
+        "parameters": {
+            "algorithm": "lightgbm"
+        },
+        "created_at": "<CREATED_AT>",
+        "last_updated": "<LAST_UPDATED>",
+        "dependencies": [],
+        "short_slug": "my-experiment",
+        "slug": "my-experiment-<CREATED_AT>",
+        "tests": [],
+        "artifacts": []
+    }
+]
+```
+
+Once you've finished, save the project to ``project.json``:
+
+```python
+project.save()
+```
+
+Later on, you can read the project back in read-only mode (`"r"`), append mode (`"a"`),
+or editable mode (`"w+"`):
+
+```python
+project = Project("project.json", mode="r")
+with project.log(name="New experiment") as exp:  # Raises a ReadOnlyError
+    ...
+```

lazyscribe-2.0.0/lazyscribe/__init__.py

@@ -0,0 +1,9 @@
+"""Main module."""
+
+from lazyscribe._meta import __version__  # noqa: F401
+from lazyscribe.experiment import Experiment
+from lazyscribe.project import Project
+from lazyscribe.repository import Repository
+from lazyscribe.test import Test
+
+__all__: list[str] = ["Experiment", "Project", "Repository", "Test"]

lazyscribe-2.0.0/lazyscribe/_meta.py

@@ -0,0 +1,3 @@
+"""Version."""
+
+__version__ = "2.0.0"

lazyscribe-2.0.0/lazyscribe/_utils.py

@@ -0,0 +1,132 @@
+"""Util methods."""
+
+import inspect
+import json
+from collections.abc import Iterator
+from datetime import datetime, timezone
+from typing import Any
+
+from attrs import Attribute, asdict, fields, filters
+
+from lazyscribe.artifacts.base import Artifact
+from lazyscribe.exception import ArtifactLoadError
+from lazyscribe.registry import registry
+
+
+def serializer(inst: type, field: "Attribute[Any]", value: Any) -> Any:
+    """Datetime and dependencies converter for :meth:`attrs.asdict`.
+
+    Parameters
+    ----------
+    inst : type
+        Included for compatibility.
+    field : attrs.Attribute[Any]
+        The field name.
+    value : Any
+        The field value.
+
+    Returns
+    -------
+    Any
+        Converted value for easy serialization.
+    """
+    if isinstance(value, datetime):
+        return value.isoformat(timespec="seconds")
+    if field is not None and field.name == "dependencies":
+        deps: list[str] = []
+        for exp in value.values():
+            if (project := registry.search(exp.project)) is not None:
+                deps.append(f"{project}|{exp.slug}")
+            else:
+                deps.append(f"{exp.project}|{exp.slug}")
+        return deps
+    if field is not None and field.name == "tests":
+        tests: list[dict[str, Any]] = [asdict(test) for test in value]
+        return tests
+    if field is not None and field.name == "artifacts":
+        art: list[dict[str, Any]] = list(serialize_artifacts(value))
+        return art
+
+    return value
+
+
+def serialize_artifacts(alist: list[Artifact]) -> Iterator[dict[str, Any]]:
+    """Serialize list of artifacts."""
+    yield from (
+        {
+            **asdict(
+                artifact,
+                filter=filters.exclude(
+                    fields(type(artifact)).value,
+                    fields(type(artifact)).writer_kwargs,
+                    fields(type(artifact)).dirty,
+                ),
+                value_serializer=lambda _, __, value: value.isoformat(
+                    timespec="seconds"
+                )
+                if isinstance(value, datetime)
+                else value,
+            ),
+            "handler": artifact.alias,
+        }
+        for artifact in alist
+    )
+
+
+def utcnow() -> datetime:
+    """Return the naive datetime now in UTC.
+
+    Returns
+    -------
+    datetime.datetime
+        Now in UTC, without timezone info.
+    """
+    return datetime.now(timezone.utc).replace(tzinfo=None)
+
+
+def validate_artifact_environment(artifact: Artifact) -> None:
+    """Validate the artifact handler environment.
+
+    Parameters
+    ----------
+    artifact : Artifact
+        An artifact handler instantiated from project and/or repository metadata.
+
+    Raises
+    ------
+    lazyscribe.exception.ArtifactLoadError
+        Raised if the runtime environment does not match artifact metadata.
+    """
+    # Construct the handler with relevant parameters.
+    artifact_attrs: dict[str, Any] = {
+        x: y
+        for x, y in inspect.getmembers(artifact)
+        if not x.startswith("_") and not inspect.ismethod(y)
+    }
+    # Exclude parameters that don't define equality
+    exclude_names: list[str] = [
+        attr.name for attr in fields(type(artifact)) if not attr.eq
+    ]
+    construct_params: list[str] = [
+        param_name
+        for param_name, param in inspect.signature(
+            artifact.construct
+        ).parameters.items()
+        if param_name not in exclude_names or param.default == param.empty
+    ]
+    artifact_attrs = {
+        key: value for key, value in artifact_attrs.items() if key in construct_params
+    }
+
+    curr_handler = type(artifact).construct(**artifact_attrs, dirty=False)
+    # Validate the handler
+    if curr_handler != artifact:
+        field_filters = filters.exclude(
+            *[attr for attr in fields(type(artifact)) if not attr.eq]
+        )
+        raise ArtifactLoadError(
+            "Runtime environments do not match. Artifact parameters:\n\n"
+            f"{json.dumps(asdict(artifact, filter=field_filters))}"
+            "\n\nCurrent parameters:\n\n"
+            f"{json.dumps(asdict(curr_handler, filter=field_filters))}"
+        )

lazyscribe-2.0.0/lazyscribe/artifacts/__init__.py

@@ -0,0 +1,51 @@
+"""Import the handlers."""
+
+from importlib.metadata import entry_points
+
+from lazyscribe.artifacts.base import Artifact
+
+__all__: list[str] = ["_get_handler"]
+
+
+def _get_handler(alias: str) -> type[Artifact]:
+    """Retrieve a specific handler based on the alias.
+
+    Parameters
+    ----------
+    alias : str
+        The alias for the handler.
+
+    Returns
+    -------
+    type[lazyscribe.artifacts.base.Artifact]
+        The artifact handler class.
+    """
+    entry = entry_points(group="lazyscribe.artifact_type")
+
+    for full_artifact_class in entry:  # search through entrypoints first
+        if full_artifact_class.name == alias:
+            try:
+                mod = full_artifact_class.load()
+            except ImportError as imp:
+                raise ImportError(
+                    f"Unable to import handler for {alias} through entry points"
+                ) from imp
+
+            if not isinstance(mod, type):
+                raise TypeError(f"{full_artifact_class} is not a class")
+
+            break
+
+    else:
+        for obj in Artifact.__subclasses__():  # search through experiment subclasses
+            if obj.alias == alias:
+                mod = obj
+                break
+
+        # no handler found in both entrypoints or subclass
+        else:
+            raise ValueError(
+                f"No handler available with the name {alias} in `artifact_type` group."
+            )
+
+    return mod  # type: ignore

lazyscribe-2.0.0/lazyscribe/artifacts/base.py

@@ -0,0 +1,148 @@
+"""Base class for new artifact handlers."""
+
+from __future__ import annotations
+
+from abc import ABCMeta, abstractmethod
+from datetime import datetime
+from io import IOBase
+from typing import Any, ClassVar
+
+from attrs import define, field
+
+
+@define
+class Artifact(metaclass=ABCMeta):
+    """Generic artifact handler that defines the expected interface.
+
+    Artifact handlers are not meant to be initialized directly.
+
+    Attributes
+    ----------
+    alias : str
+        The alias for the artifact handler. This value will be supplied to
+        :py:meth:`lazyscribe.experiment.Experiment.log_artifact`.
+        (A class attribute.)
+    suffix : str
+        The standard suffix for the files written and read by this handler.
+        (A class attribute.)
+    binary : bool
+        Whether or not the file format for the handler is binary in nature. This
+        affects whether or not the file handler uses ``w`` or ``wb``.
+        (A class attribute.)
+    output_only : bool
+        Whether or not the file output by the handler is meant to be read as the orginal project.
+        (A class attribute.)
+    name : str
+        The name of the artifact.
+    fname : str
+        The filename of the artifact.
+    value : Any
+        The value for the artifact.
+    writer_kwargs : dict
+        User provided keyword arguments for writing an artifact. Provided when
+        the artifact is logged to an experiment.
+    version : int
+        Version of the artifact.
+    dirty : bool
+        Whether or not this artifact should be saved when :py:meth:`lazyscribe.project.Project.save`
+        or :py:meth:`lazyscribe.repository.Repository.save` is called. This decision is based
+        on whether the artifact is new or has been updated.
+    """
+
+    alias: ClassVar[str]
+    suffix: ClassVar[str]
+    binary: ClassVar[bool]
+    output_only: ClassVar[
+        bool
+    ]  # Describes if the artifact will reconstruct to a Python object on read
+    name: str = field(eq=False)
+    fname: str = field(eq=False)
+    value: Any = field(eq=False)
+    writer_kwargs: dict[str, Any] = field(eq=False)
+    created_at: datetime = field(eq=False)
+    expiry: datetime | None = field(eq=False)
+    version: int = field(eq=False)
+    dirty: bool = field(eq=False)
+
+    @classmethod
+    @abstractmethod
+    def construct(
+        cls,
+        name: str,
+        value: Any = None,
+        fname: str | None = None,
+        created_at: datetime | None = None,
+        expiry: datetime | None = None,
+        writer_kwargs: dict[str, Any] | None = None,
+        version: int = 0,
+        dirty: bool = True,
+        **kwargs: Any,
+    ) -> Artifact:
+        """Construct the artifact handler.
+
+        This method should use environment variables to capture information that
+        is relevant to compatibility between runtime environments.
+
+        Parameters
+        ----------
+        name : str
+            The name of the artifact.
+        value : Any, optional (default None)
+            The value for the artifact.
+        fname : str, optional (default None)
+            The filename for the artifact. If set to ``None`` or not provided, it will be derived from
+            the name of the artifact and the suffix for the class.
+        created_at : datetime.datetime, optional (default ``lazyscribe._utils.utcnow()``)
+            When the artifact was created.
+        expiry : datetime.datetime, optional (default None)
+            When the artifact expired.
+        writer_kwargs : dict, optional (default {})
+            Keyword arguments for writing an artifact to the filesystem. Provided when an artifact
+            is logged to an experiment.
+        version : int, optional (default 0)
+            Integer version to be used for versioning artifacts.
+        dirty : bool, optional (default True)
+            Whether or not this artifact should be saved when :py:meth:`lazyscribe.project.Project.save`
+            or :py:meth:`lazyscribe.repository.Repository.save` is called. This decision is based
+            on whether the artifact is new or has been updated.
+        **kwargs
+            Other keyword arguments.
+
+        Returns
+        -------
+        Artifact
+            The artifact.
+        """
+
+    @classmethod
+    @abstractmethod
+    def read(cls, buf: IOBase, **kwargs: Any) -> Any:
+        """Read in the artifact.
+
+        Parameters
+        ----------
+        buf : file-like object
+            The buffer from a ``fsspec`` filesystem.
+        **kwargs
+            Keyword arguments for the read method.
+
+        Returns
+        -------
+        Any
+            The artifact object.
+        """
+
+    @classmethod
+    @abstractmethod
+    def write(cls, obj: Any, buf: IOBase, **kwargs: Any) -> None:
+        """Write the artifact to the filesystem.
+
+        Parameters
+        ----------
+        obj : Any
+            The object to write to the buffer.
+        buf : file-like object
+            The buffer from a ``fsspec`` filesystem.
+        **kwargs
+            Keyword arguments for the write method.
+        """

lazyscribe-2.0.0/lazyscribe/artifacts/json.py

@@ -0,0 +1,134 @@
+"""Artifact handler for JSON-serializable objects."""
+
+from __future__ import annotations
+
+from datetime import datetime
+from io import IOBase
+from json import dump, load
+from typing import Any, ClassVar
+
+from attrs import define
+from slugify import slugify
+
+from lazyscribe._utils import utcnow
+from lazyscribe.artifacts.base import Artifact
+
+
+@define(auto_attribs=True)
+class JSONArtifact(Artifact):
+    """Handler for JSON-serializable objects.
+
+    .. important::
+
+        This class is not meant to be initialized directly. Please use the ``construct``
+        method.
+
+    .. note::
+
+        For the attributes documentation, see also "Attributes" of :py:class:`lazyscribe.artifacts.base.Artifact`.
+
+    Attributes
+    ----------
+    alias : str = "json"
+    suffix : str = "json"
+    binary : bool = False
+    output_only : bool = False
+
+    python_version : str
+        Minor Python version (e.g. ``"3.10"``).
+    """
+
+    alias: ClassVar[str] = "json"
+    suffix: ClassVar[str] = "json"
+    binary: ClassVar[bool] = False
+    output_only: ClassVar[bool] = False
+
+    @classmethod
+    def construct(
+        cls,
+        name: str,
+        value: Any = None,
+        fname: str | None = None,
+        created_at: datetime | None = None,
+        expiry: datetime | None = None,
+        writer_kwargs: dict[str, Any] | None = None,
+        version: int = 0,
+        dirty: bool = True,
+        **kwargs: Any,
+    ) -> JSONArtifact:
+        """Construct the handler class.
+
+        Parameters
+        ----------
+        name : str
+            The name of the artifact.
+        value : Any, optional (default None)
+            The value for the artifact. The default value of ``None`` is used when
+            an experiment is loaded from the project JSON.
+        fname : str, optional (default None)
+            The filename for the artifact. If set to ``None`` or not provided, it will be derived from
+            the name of the artifact and the suffix for the class.
+        created_at : datetime.datetime, optional (default ``lazyscribe._utils.utcnow()``)
+            When the artifact was created.
+        expiry : datetime.datetime, optional (default None)
+            When the artifact expired.
+        writer_kwargs : dict[str, Any], optional (default {})
+            Keyword arguments for writing an artifact to the filesystem. Provided when an artifact
+            is logged to an experiment.
+        version : int, optional (default 0)
+            Integer version to be used for versioning artifacts.
+        dirty : bool, optional (default True)
+            Whether or not this artifact should be saved when :py:meth:`lazyscribe.project.Project.save`
+            or :py:meth:`lazyscribe.repository.Repository.save` is called. This decision is based
+            on whether the artifact is new or has been updated.
+
+        Returns
+        -------
+        JSONArtifact
+            The artifact.
+        """
+        created_at = created_at or utcnow()
+        return cls(
+            name=name,
+            value=value,
+            fname=fname
+            or f"{slugify(name)}-{slugify(created_at.strftime('%Y%m%d%H%M%S'))}.{cls.suffix}",
+            created_at=created_at,
+            expiry=expiry,
+            writer_kwargs=writer_kwargs or {},
+            version=version,
+            dirty=dirty,
+        )
+
+    @classmethod
+    def read(cls, buf: IOBase, **kwargs: Any) -> Any:
+        """Read in the JSON file.
+
+        Parameters
+        ----------
+        buf : file-like object
+            The buffer from a ``fsspec`` filesystem.
+        **kwargs
+            Keyword arguments for :py:meth:`json.load`
+
+        Returns
+        -------
+        Any
+            The deserialized JSON file.
+        """
+        return load(buf, **kwargs)
+
+    @classmethod
+    def write(cls, obj: Any, buf: IOBase, **kwargs: Any) -> None:
+        """Write the content to a JSON file.
+
+        Parameters
+        ----------
+        obj : object
+            The JSON-serializable object.
+        buf : file-like object
+            The buffer from a ``fsspec`` filesystem.
+        **kwargs
+            Keyword arguments for :py:meth:`json.dump`.
+        """
+        dump(obj, buf, **kwargs)