pelutils 4.2.1__tar.gz → 4.3.1__tar.gz

{pelutils-4.2.1 → pelutils-4.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pelutils
-Version: 4.2.1
+Version: 4.3.1
 Summary: The Swiss army knife of Python projects
 Home-page: https://github.com/peleiden/pelutils
 Download-URL: https://pypi.org/project/pelutils/
@@ -17,6 +17,7 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 Provides-Extra: dev
@@ -163,10 +164,15 @@ importantly one per thread, as profiling is not thread-safe.
 
 ## Serialisation
 
-`UniversalJsonModel` extends `pydantic.BaseModel` with `save`/`load` methods and can
-serialise attributes that pydantic cannot — numpy arrays, tensors, and arbitrary
-objects are base64-pickled inline, everything else stays plain, human-readable JSON.
-Long lists are wrapped to the line-length limit instead of one element per line.
+`UniversalJsonModel` is for storing Pydantic models that contain a mixture of ordinary JSON
+values and Python-specific values. Ordinary values remain JSON, so the file structure can be
+inspected and diffed. Values that JSON cannot represent, such as NumPy arrays, tensors, and
+other Python objects, are stored as base64-encoded pickle payloads.
+
+The `save()` and `load()` methods support convient file-based workflows.
+`save()` serialises the model to a JSON file, while `load()` reconstructs the model from one.
+Use `to_json_dict()` and `from_json_dict(...)` when the serialized representation belongs inside
+another structure or is handled by another storage layer.
 
 ```py
 import numpy as np
@@ -192,11 +198,14 @@ result.save("results/run-1.json")
 result = Result.load("results/run-1.json")
 ```
 
-Use `to_json_dict()` / `from_json_dict(...)` to convert to and from a plain dict
-without touching the filesystem — useful for nesting inside other structures. The
-`pretty_json` helper function is also available on its own. The `serialization`
-module also includes JSONL read/write helpers (`jsonl_dump`, `jsonl_load`, ...)
-with largely the same interface as is provided by the built-in `json` module.
+`pretty_json` is a string-returning formatter for standalone dictionaries and lists. It fills the
+same role as `json.dumps`, with compact nested containers and wrapped primitive lists; write the
+returned string to a file when needed. The module also includes JSONL read/write helpers
+(`jsonl_dump`, `jsonl_load`, ...) for newline-delimited records.
+
+This format is intended for trusted configuration objects, application state, experiment results, etc., not as a
+language-neutral interchange format. Pickle payloads are Python-specific and loading them can
+execute arbitrary code. Do not load files from untrusted sources.
 
 ## Config and command-line argument parsing
 
@@ -253,18 +262,18 @@ values, index, inverse, counts = unique(
 
 ### Statistics
 
-Common statistical helpers, plus wrappers around scipy distributions reparametrised
-as in Jim Pitman's *Probability* (rather than scipy's `loc`/`scale`, which are
+Common statistical helpers, plus wrappers around SciPy distributions reparametrised
+as in Jim Pitman's *Probability* (rather than SciPy's `loc`/`scale`, which are
 unintuitive for many distributions).
 
 ```py
-from pelutils.stats import z_score
 from pelutils.stats import expon
+from pelutils.stats import z_score
 
-# 95 % confidence interval half-width for a standard normal (defaults give ~1.96)
-half_width = std * z_score()
+# 95 % confidence interval half-width for a standard normal
+half_width = z_score()  # standard deviation is 1
 
-# One-sided z value for an Exponential(λ=2) at the 1 % significance level
+# Upper 1 % critical value for an Exponential(lambda=2)
 zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 ```
 

{pelutils-4.2.1 → pelutils-4.3.1}/README.md

@@ -138,10 +138,15 @@ importantly one per thread, as profiling is not thread-safe.
 
 ## Serialisation
 
-`UniversalJsonModel` extends `pydantic.BaseModel` with `save`/`load` methods and can
-serialise attributes that pydantic cannot — numpy arrays, tensors, and arbitrary
-objects are base64-pickled inline, everything else stays plain, human-readable JSON.
-Long lists are wrapped to the line-length limit instead of one element per line.
+`UniversalJsonModel` is for storing Pydantic models that contain a mixture of ordinary JSON
+values and Python-specific values. Ordinary values remain JSON, so the file structure can be
+inspected and diffed. Values that JSON cannot represent, such as NumPy arrays, tensors, and
+other Python objects, are stored as base64-encoded pickle payloads.
+
+The `save()` and `load()` methods support convient file-based workflows.
+`save()` serialises the model to a JSON file, while `load()` reconstructs the model from one.
+Use `to_json_dict()` and `from_json_dict(...)` when the serialized representation belongs inside
+another structure or is handled by another storage layer.
 
 ```py
 import numpy as np
@@ -167,11 +172,14 @@ result.save("results/run-1.json")
 result = Result.load("results/run-1.json")
 ```
 
-Use `to_json_dict()` / `from_json_dict(...)` to convert to and from a plain dict
-without touching the filesystem — useful for nesting inside other structures. The
-`pretty_json` helper function is also available on its own. The `serialization`
-module also includes JSONL read/write helpers (`jsonl_dump`, `jsonl_load`, ...)
-with largely the same interface as is provided by the built-in `json` module.
+`pretty_json` is a string-returning formatter for standalone dictionaries and lists. It fills the
+same role as `json.dumps`, with compact nested containers and wrapped primitive lists; write the
+returned string to a file when needed. The module also includes JSONL read/write helpers
+(`jsonl_dump`, `jsonl_load`, ...) for newline-delimited records.
+
+This format is intended for trusted configuration objects, application state, experiment results, etc., not as a
+language-neutral interchange format. Pickle payloads are Python-specific and loading them can
+execute arbitrary code. Do not load files from untrusted sources.
 
 ## Config and command-line argument parsing
 
@@ -228,18 +236,18 @@ values, index, inverse, counts = unique(
 
 ### Statistics
 
-Common statistical helpers, plus wrappers around scipy distributions reparametrised
-as in Jim Pitman's *Probability* (rather than scipy's `loc`/`scale`, which are
+Common statistical helpers, plus wrappers around SciPy distributions reparametrised
+as in Jim Pitman's *Probability* (rather than SciPy's `loc`/`scale`, which are
 unintuitive for many distributions).
 
 ```py
-from pelutils.stats import z_score
 from pelutils.stats import expon
+from pelutils.stats import z_score
 
-# 95 % confidence interval half-width for a standard normal (defaults give ~1.96)
-half_width = std * z_score()
+# 95 % confidence interval half-width for a standard normal
+half_width = z_score()  # standard deviation is 1
 
-# One-sided z value for an Exponential(λ=2) at the 1 % significance level
+# Upper 1 % critical value for an Exponential(lambda=2)
 zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 ```
 

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils/__version__.py

@@ -1,2 +1,2 @@
 # Do not put anything else in this file
-__version__ = "4.2.1"
+__version__ = "4.3.1"

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils/array/_blob.py

@@ -1,7 +1,10 @@
 from __future__ import annotations
 
+from typing import SupportsIndex
+
 import _pelutils_c as _c
 import numpy as np
+from typing_extensions import override
 
 import pelutils._c as c_utils
 from pelutils.types import BoolArray, IntArray
@@ -171,3 +174,21 @@ class SparseGridBlobDetection:
             # If the error happens early in init (e.g. wrong call to __init__), _pointer_args has not yet been defined,
             # and the hashmap has not been created
             pass
+
+    def __copy__(self):
+        """Prevent shallow copies of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")
+
+    def __deepcopy__(self, memo: dict[int, object]):
+        """Prevent deep copies of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")
+
+    @override
+    def __reduce__(self):
+        """Prevent pickling of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")
+
+    @override
+    def __reduce_ex__(self, protocol: SupportsIndex):
+        """Prevent protocol-specific pickling of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")

pelutils-4.3.1/pelutils/legacy/v3/__init__.py

@@ -0,0 +1,10 @@
+"""Legacy module for preserving functionality that no longer has a place in pelutils, but is hard to change.
+
+This is especially relevant for functionality which leaves persistent data which may need to be loaded in later.
+``DataStorage`` is the obvious example here as it defines a custom serialisation/deserialisation protocol.
+Upgrading to pelutils 4 without this module would make old persisted ``DataStorage`` instances impossible to load.
+"""
+
+from ._datastorage import DataStorage
+
+__all__ = ("DataStorage",)

pelutils-4.3.1/pelutils/legacy/v3/_datastorage.py

@@ -0,0 +1,150 @@
+from __future__ import annotations
+
+import os
+import pickle
+from pathlib import Path
+from typing import Self
+
+try:
+    import rapidjson  # pyright: ignore[reportMissingImports]
+except ModuleNotFoundError:
+    import json as rapidjson
+
+
+class DataStorage:
+    """The DataStorage class is an augmentation of the dataclass that incluces save and load functionality.
+
+    DataStorage classes must inherit from DataStorage and be annotated with ``@dataclass``. Data will be saved
+    to two files: A json files for json-serializable data and a pickle file for everything else. These files
+    are by default named after the class, but it is possible to use a custom name in the save and load methods.
+    The files are only created if necessary, so for instance if all data is json-serializable, no pickle file
+    will be created.
+
+    Data is in general preserved exactly as-is when saved data is loaded into memory with few exceptions.
+    Nnotably, tuples are considered json-serializble, and so will be saved to the json file and will be
+    loaded as lists.
+
+    Quick start
+    -----------
+
+    .. code-block:: python
+
+        @dataclass
+        class ResultData(DataStorage):
+            shots: int
+            goalscorers: list
+            dists: np.ndarray
+
+        rdata = ResultData(shots=1, goalscorers=["Max Fenger"], dists=np.ones(22)*10)
+        rdata.save("max")
+        # Now shots and goalscorers are saved in <pwd>/max/ResultData.json and dists in <pwd>/max/ResultData.pkl
+
+        # Then to load
+        rdata = ResultData.load("max")
+        print(rdata.goalscorers)  # ["Max Fenger"]
+
+    """
+
+    def __init__(self, *args, **kwargs):  # noqa: ARG002  # pyright: ignore[reportUnknownParameterType, reportMissingParameterType]
+        """This method is overwritten in any class decorated with @dataclass.
+
+        Therefore, if this method is called, it is an error.
+        """  # noqa: D401, D404
+        raise TypeError(f"DataStorage class {self.__class__.__name__} must be decorated with @dataclass")
+
+    @classmethod
+    def json_name(cls, save_name: str | None = None):
+        """Return the name of the json file to where json serializable data will be saved."""
+        return (save_name or cls.__name__) + ".json"
+
+    @classmethod
+    def pickle_name(cls, save_name: str | None = None):
+        """Return the name of the pickle file to where non json serializable data will be saved."""
+        return (save_name or cls.__name__) + ".pkl"
+
+    def save(self, loc: str | Path, save_name: str | None = None, *, indent: int | None = 4) -> list[str]:
+        """Save all the fields of the instatiated data classes as either json, pickle or designated serialization function.
+
+        Parameters
+        ----------
+        loc : str | Path
+            Directory in which to save data.
+        save_name : str | None, optional
+            If given, file name (excluding extension) to use for saving, by default None.
+        indent : int | None, optional
+            Indent used for the json file, by default 4. It is recommended to set to None when large objects are saved to json.
+
+        Returns
+        -------
+        list[str]
+            Produced files containing the data.
+        """
+        loc = str(loc)
+        os.makedirs(loc, exist_ok=True)
+
+        to_json = dict()
+        to_pickle = dict()
+
+        for key, data in self.__dict__.items():
+            try:
+                # Test whether the data is json serializable by dumping it to string.
+                rapidjson.dumps({key: data})
+                to_json[key] = data
+            except TypeError:
+                to_pickle[key] = data
+
+        # Save data
+        paths = list()
+        if to_json:
+            paths.append(os.path.join(loc, self.json_name(save_name)))
+            with open(paths[-1], "w", encoding="utf-8") as f:
+                # Save json. This does not guarantee writing to disk, so flushing
+                # and synchronization is also done to increase chance of writing
+                dump = rapidjson.dumps(to_json, indent=indent)
+                f.write(dump)
+                f.flush()
+                os.fsync(f.fileno())
+
+        if to_pickle:
+            paths.append(os.path.join(loc, self.pickle_name(save_name)))
+            with open(paths[-1], "wb") as f:
+                pickle.dump(to_pickle, f)
+                f.flush()
+                os.fsync(f.fileno())
+
+        return paths
+
+    @classmethod
+    def load(cls, loc: str | Path, save_name: str | None = None) -> Self:
+        """Instantiate the DataStorage-inherited class by loading all files saved by `save` of that same class.
+
+        Parameters
+        ----------
+        loc : str | Path
+            Directory from which to load data.
+        save_name : str | None, optional
+            If given, file name (excluding extension) to use for loading, by default None. It should match what was given to `save`.
+
+        Returns
+        -------
+        DataStorage
+            An instance of this class with the content of the files.
+        """
+        loc = str(loc)
+        json_file = os.path.join(loc, cls.json_name(save_name))
+        pickle_file = os.path.join(loc, cls.pickle_name(save_name))
+
+        fields = dict()
+
+        if os.path.isfile(json_file):
+            with open(json_file, encoding="utf-8") as f:
+                fields.update(rapidjson.load(f))
+
+        if os.path.isfile(pickle_file):
+            with open(pickle_file, "rb") as f:
+                fields.update(pickle.load(f))
+
+        if not os.path.isfile(json_file) and not os.path.isfile(pickle_file):
+            raise FileNotFoundError(f"Unable to load {cls.__name__}; neither {json_file} nor {pickle_file} found.")
+
+        return cls(**fields)

pelutils-4.3.1/pelutils/serialization/__init__.py

@@ -0,0 +1,51 @@
+"""JSON persistence for Pydantic models containing non-JSON values.
+
+JSON is useful when the structure of a file should remain inspectable, but it does not represent
+values such as NumPy arrays, tensors, or arbitrary Python objects. ``UniversalJsonModel``
+keeps JSON-native values in the document and stores unsupported field values as base64-encoded
+pickle payloads. This preserves a readable JSON envelope without requiring callers to write
+converters for every field type.
+
+The usual interface is :meth:`UniversalJsonModel.save` and
+:meth:`UniversalJsonModel.load`. ``save`` creates missing parent directories and writes the model to a JSON
+file; ``load`` reconstructs the model from that file. Use :meth:`UniversalJsonModel.to_json_dict`
+and :meth:`UniversalJsonModel.from_json_dict` when the serialized representation should be nested
+inside another structure or handled by another storage layer.
+
+For standalone dictionaries and lists, ``pretty_json`` converts Python objects to pretty-formatted JSON,
+filling the same role as ``json.dumps``, but often making the resulting JSON more readable.
+JSONL helpers handle files containing multiple JSON records separated by newlines.
+
+Quick start
+-----------
+
+.. code-block:: python
+
+    import numpy as np
+    from pelutils.serialization import UniversalJsonModel
+    from pelutils.types import FloatArray
+
+    class Result(UniversalJsonModel):
+        accuracy: float
+        predictions: FloatArray   # numpy arrays are handled automatically
+
+    result = Result(accuracy=0.97, predictions=np.arange(5, dtype=np.float16))
+    result.save("results/run-1.json")
+    result = Result.load("results/run-1.json")
+
+This format is intended for trusted application data such as experiment results and cached state,
+not as a language-neutral interchange format. Pickle payloads are Python-specific, and loading
+them can execute arbitrary code. Use a format with an explicit schema and safe decoder when data
+comes from outside the application.
+
+.. warning::
+
+    ``UniversalJsonModel.load`` and the pickle fallback execute code while loading. Never load
+    files from an untrusted source.
+"""
+
+from ._jsonl import jsonl_dump, jsonl_dumps, jsonl_load, jsonl_loads
+from ._pretty_json import pretty_json
+from ._universal_json_model import UniversalJsonModel
+
+__all__ = ("UniversalJsonModel", "jsonl_dump", "jsonl_dumps", "jsonl_load", "jsonl_loads", "pretty_json")

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils/stats/__init__.py

@@ -1,11 +1,11 @@
-"""Statistical helpers and scipy distributions parametrised the way textbooks are.
+"""Statistical helpers and SciPy distributions parametrised the way textbooks are.
 
 ``scipy.stats`` parametrises every distribution through generic ``loc``/``scale``
 arguments, which rarely match how a distribution is actually defined: an exponential's
 ``scale`` is ``1 / lambda``, a normal takes a standard deviation rather than a variance,
 and a gamma's ``a``/``scale`` bear no obvious relation to its shape and rate. This module
-wraps the scipy distributions so you pass their *natural* parameters instead — following
-the conventions in Jim Pitman's *Probability* — and returns an ordinary frozen scipy
+wraps the SciPy distributions so you pass their *natural* parameters instead — following
+the conventions in Jim Pitman's *Probability* — and returns an ordinary frozen SciPy
 distribution, so ``.pdf``, ``.cdf``, ``.rvs`` and friends all work as usual. It also
 provides :func:`z_score` for turning a significance level into a critical value.
 
@@ -17,12 +17,12 @@ Quick start
     from pelutils.stats import expon, norm, z_score
 
     dist = norm(mu=0, sigma2=4)   # mean and *variance*, not standard deviation
-    dist.cdf(1.5)                 # a plain frozen scipy distribution
+    dist.cdf(1.5)                 # a plain frozen SciPy distribution
 
-    # 95 % confidence-interval half-width for a standard normal (defaults give ~1.96)
-    half_width = std * z_score()
+    # 95 % confidence-interval half-width for a standard normal
+    half_width = z_score()        # standard deviation is 1
 
-    # One-sided z value for an Exponential(lambda=2) at the 1 % significance level
+    # Upper 1 % critical value for an Exponential(lambda=2)
     zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 
 Continuous distributions: :func:`norm`, :func:`lognorm`, :func:`expon`, :func:`gamma`,

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils/stats/_distributions.py

@@ -1,4 +1,4 @@
-"""Functions returning scipy.stats distribution objects reparameterized to Jim Pitman's "Probability"."""
+"""Functions returning SciPy distribution objects reparameterized to Jim Pitman's "Probability"."""
 
 import numpy as np
 from scipy import stats
@@ -6,25 +6,25 @@ from scipy import stats
 
 # Continuous distributions
 def norm(mu: float, sigma2: float):
-    """Return a normal distribution from mean and variance."""
+    """Return a normal distribution with mean ``mu`` and variance ``sigma2``."""
     assert sigma2 > 0
     return stats.norm(loc=mu, scale=np.sqrt(sigma2))
 
 
 def lognorm(mu: float, sigma2: float):
-    """Return a log normal distribution from the mean and variance of the corresponding normal distribution."""
+    """Return a log-normal distribution where ``log(X)`` has mean ``mu`` and variance ``sigma2``."""
     assert sigma2 > 0
     return stats.lognorm(s=np.sqrt(sigma2), scale=np.exp(mu))
 
 
 def expon(lambda_: float):
-    """Return an exponential distribution."""
+    """Return an exponential distribution with rate ``lambda_`` and mean ``1 / lambda_``."""
     assert lambda_ > 0
     return stats.expon(scale=1 / lambda_)
 
 
 def gamma(r: float, lambda_: float):
-    """Return a gamma distribution."""
+    """Return a gamma distribution with shape ``r`` and rate ``lambda_``."""
     assert r > 0 and lambda_ > 0
     return stats.gamma(a=r, loc=0, scale=1 / lambda_)
 
@@ -32,15 +32,16 @@ def gamma(r: float, lambda_: float):
 def chi2(n: int | float):
     """Return a chi-squared distribution with the given number of degrees of freedom.
 
-    The chi-squared distribution is generally defined only on the natural numbers, but here being defined as a special case
-    of the gamma distribution, it generalises easily to positive rational numbers, hence why floats are accepted.
+    Although degrees of freedom are commonly given as an integer, the distribution
+    is here represented as a special case of the gamma distribution, making it defined
+    for any positive real ``n``.
     """
     assert n > 0
     return gamma(n / 2, 1 / 2)
 
 
 def rayleigh():
-    """Return a rayleigh distribution."""
+    """Return a Rayleigh distribution."""
     return stats.rayleigh()
 
 
@@ -52,42 +53,52 @@ def beta(r: float, s: float):
 
 # Discrete distributions
 def bernoulli(p: float):
-    """Return a Bernoulli distribution."""
+    """Return a Bernoulli distribution with success probability ``p``."""
     assert 0 <= p <= 1
     return stats.bernoulli(p)
 
 
 def binomial(n: int, p: float):
-    """Return a binomial distribution."""
+    """Return a binomial distribution for ``n`` trials with success probability ``p``."""
     assert n >= 0 and 0 <= p <= 1
     return stats.binom(n=n, p=p)
 
 
 def poisson(mu: float):
-    """Return a Poisson distribution."""
+    """Return a Poisson distribution with mean and rate ``mu``."""
     assert mu >= 0
     return stats.poisson(mu=mu)
 
 
 def hypergeom(n: int, N: int, G: int):  # noqa: N803
-    """Return a hypergeometric distribution."""
+    """Return a hypergeometric distribution.
+
+    Parameters
+    ----------
+    n:
+        Number of items drawn from the population.
+    N:
+        Population size.
+    G:
+        Number of successes in the population.
+    """
     assert N > n > 0 and N >= G > 0 and N > 1
     return stats.hypergeom(M=N, n=G, N=n)
 
 
 def geom0(p: float):
-    """Return a geometric distribution defined on {0, 1, ...}."""
+    """Return a geometric distribution of failures before the first success, with success probability ``p``."""
     assert 0 < p <= 1
     return stats.geom(loc=-1, p=p)
 
 
 def geom1(p: float):
-    """Return a geometric distribution defined on {1, 2, ...}."""
+    """Return a geometric distribution of the trial of the first success, with success probability ``p``."""
     assert 0 < p <= 1
     return stats.geom(p=p)
 
 
 def nbinom(r: int, p: float):
-    """Return a negative binomial distribution."""
+    """Return a negative binomial distribution of failures before ``r`` successes, with success probability ``p``."""
     assert r > 0 and 0 <= p <= 1
     return stats.nbinom(n=r, p=p)

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils/stats/_stats.py

@@ -4,16 +4,20 @@ from ._distributions import norm
 
 
 def z_score(alpha: float = 0.05, two_sided: bool = True, distribution: Any | None = None) -> float:  # pyright: ignore[reportExplicitAny]
-    """Return the critical value (z score) for a given significance level.
+    """Return an upper critical value for a given significance level.
 
     Parameters
     ----------
     alpha : float, optional
         Significance level in ``[0, 1]``.
     two_sided : bool, optional
-        If ``True``, split ``alpha`` across both tails; otherwise use a single tail.
+        If ``True``, use an upper-tail probability of ``alpha / 2``; otherwise use
+        ``alpha``. For symmetric distributions this gives the usual two-sided
+        cutoff for a distribution symmetric about zero, with the lower cutoff equal
+        to the negative of the returned value.
+        For asymmetric distributions, the returned value is only the upper cutoff.
     distribution : Any | None, optional
-        A frozen scipy distribution to draw the quantile from. Defaults to ``N(0, 1)``,
+        A frozen SciPy distribution to draw the quantile from. Defaults to ``N(0, 1)``,
         in which case the two-sided default returns the familiar ``~1.96``.
     """
     if not 0 <= alpha <= 1:

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pelutils
-Version: 4.2.1
+Version: 4.3.1
 Summary: The Swiss army knife of Python projects
 Home-page: https://github.com/peleiden/pelutils
 Download-URL: https://pypi.org/project/pelutils/
@@ -17,6 +17,7 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 Provides-Extra: dev
@@ -163,10 +164,15 @@ importantly one per thread, as profiling is not thread-safe.
 
 ## Serialisation
 
-`UniversalJsonModel` extends `pydantic.BaseModel` with `save`/`load` methods and can
-serialise attributes that pydantic cannot — numpy arrays, tensors, and arbitrary
-objects are base64-pickled inline, everything else stays plain, human-readable JSON.
-Long lists are wrapped to the line-length limit instead of one element per line.
+`UniversalJsonModel` is for storing Pydantic models that contain a mixture of ordinary JSON
+values and Python-specific values. Ordinary values remain JSON, so the file structure can be
+inspected and diffed. Values that JSON cannot represent, such as NumPy arrays, tensors, and
+other Python objects, are stored as base64-encoded pickle payloads.
+
+The `save()` and `load()` methods support convient file-based workflows.
+`save()` serialises the model to a JSON file, while `load()` reconstructs the model from one.
+Use `to_json_dict()` and `from_json_dict(...)` when the serialized representation belongs inside
+another structure or is handled by another storage layer.
 
 ```py
 import numpy as np
@@ -192,11 +198,14 @@ result.save("results/run-1.json")
 result = Result.load("results/run-1.json")
 ```
 
-Use `to_json_dict()` / `from_json_dict(...)` to convert to and from a plain dict
-without touching the filesystem — useful for nesting inside other structures. The
-`pretty_json` helper function is also available on its own. The `serialization`
-module also includes JSONL read/write helpers (`jsonl_dump`, `jsonl_load`, ...)
-with largely the same interface as is provided by the built-in `json` module.
+`pretty_json` is a string-returning formatter for standalone dictionaries and lists. It fills the
+same role as `json.dumps`, with compact nested containers and wrapped primitive lists; write the
+returned string to a file when needed. The module also includes JSONL read/write helpers
+(`jsonl_dump`, `jsonl_load`, ...) for newline-delimited records.
+
+This format is intended for trusted configuration objects, application state, experiment results, etc., not as a
+language-neutral interchange format. Pickle payloads are Python-specific and loading them can
+execute arbitrary code. Do not load files from untrusted sources.
 
 ## Config and command-line argument parsing
 
@@ -253,18 +262,18 @@ values, index, inverse, counts = unique(
 
 ### Statistics
 
-Common statistical helpers, plus wrappers around scipy distributions reparametrised
-as in Jim Pitman's *Probability* (rather than scipy's `loc`/`scale`, which are
+Common statistical helpers, plus wrappers around SciPy distributions reparametrised
+as in Jim Pitman's *Probability* (rather than SciPy's `loc`/`scale`, which are
 unintuitive for many distributions).
 
 ```py
-from pelutils.stats import z_score
 from pelutils.stats import expon
+from pelutils.stats import z_score
 
-# 95 % confidence interval half-width for a standard normal (defaults give ~1.96)
-half_width = std * z_score()
+# 95 % confidence interval half-width for a standard normal
+half_width = z_score()  # standard deviation is 1
 
-# One-sided z value for an Exponential(λ=2) at the 1 % significance level
+# Upper 1 % critical value for an Exponential(lambda=2)
 zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 ```
 

{pelutils-4.2.1 → pelutils-4.3.1}/pelutils.egg-info/SOURCES.txt

@@ -31,6 +31,9 @@ pelutils/array/_utils.py
 pelutils/job_parser/__init__.py
 pelutils/job_parser/_job_parser.py
 pelutils/job_parser/_structs.py
+pelutils/legacy/__init__.py
+pelutils/legacy/v3/__init__.py
+pelutils/legacy/v3/_datastorage.py
 pelutils/logging/__init__.py
 pelutils/logging/_logger.py
 pelutils/logging/_rich_string.py

{pelutils-4.2.1 → pelutils-4.3.1}/setup.py

@@ -87,6 +87,7 @@ setup_args = dict(
         "Programming Language :: Python :: 3.12",
         "Programming Language :: Python :: 3.13",
         "Programming Language :: Python :: 3.14",
+        "Programming Language :: Python :: 3.15",
     ],
 )
 

pelutils-4.2.1/pelutils/serialization/__init__.py

@@ -1,51 +0,0 @@
-"""JSON persistence for values that plain ``json`` and even ``pydantic`` cannot handle.
-
-Persisting data, whatever it may be, to disk usually usually comes down to a choice between
-human-readable or not. If efficiency is not crucial, human-readable is preferable, but often
-times, the data is not trivially serialisable to human-friendly formats. While ``pydantic``'s
-``BaseModel.model_dump`` takes you some of the way, it falls short as soon as you introduce
-data types which are not JSON-serialisable by default, making pickling the default choice and
-ending up with an unreadable binary blob.
-
-Even if all data is easily serialisable to a readable format, you still have to deal with all
-the boilerplate of ensuring parent directories exist, and opening and closing files.
-
-Enter :class:`UniversalJsonModel` — a ``pydantic.BaseModel`` whose
-``save``/``load`` methods serialise/deserialise *any* attribute: JSON-native values stay plain and
-human-readable, while anything else (numpy arrays, torch tensors, arbitrary objects) is
-pickled and base64-encoded inline. The output is a single, easily readable JSON file.
-
-Quick start
------------
-
-.. code-block:: python
-
-    import numpy as np
-    from pelutils.serialization import UniversalJsonModel
-    from pelutils.types import FloatArray
-
-    class Result(UniversalJsonModel):
-        accuracy: float
-        predictions: FloatArray   # numpy arrays are handled automatically
-
-    result = Result(accuracy=0.97, predictions=np.arange(5, dtype=np.float16))
-    result.save("results/run-1.json")
-    result = Result.load("results/run-1.json")
-
-The module also exposes :func:`pretty_json`, a function similar to the built-in `json.dumps`,
-but which formats the given object to a pretty JSON string;
-short containers stay on one line, and long primitive lists are packed across both the width and height
-of the file.
-JSONL helpers (:func:`jsonl_dump`, :func:`jsonl_load`, and their string variants) for files where each
-line is its own JSON object are also provided.
-
-.. warning::
-
-    Loading pickles executes arbitrary code, so never ``load`` data from an untrusted source.
-"""
-
-from ._jsonl import jsonl_dump, jsonl_dumps, jsonl_load, jsonl_loads
-from ._pretty_json import pretty_json
-from ._universal_json_model import UniversalJsonModel
-
-__all__ = ("UniversalJsonModel", "jsonl_dump", "jsonl_dumps", "jsonl_load", "jsonl_loads", "pretty_json")