pelutils 4.3.0__tar.gz → 4.3.1__tar.gz

{pelutils-4.3.0 → pelutils-4.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pelutils
-Version: 4.3.0
+Version: 4.3.1
 Summary: The Swiss army knife of Python projects
 Home-page: https://github.com/peleiden/pelutils
 Download-URL: https://pypi.org/project/pelutils/
@@ -164,10 +164,15 @@ importantly one per thread, as profiling is not thread-safe.
 
 ## Serialisation
 
-`UniversalJsonModel` extends `pydantic.BaseModel` with `save`/`load` methods and can
-serialise attributes that pydantic cannot — numpy arrays, tensors, and arbitrary
-objects are base64-pickled inline, everything else stays plain, human-readable JSON.
-Long lists are wrapped to the line-length limit instead of one element per line.
+`UniversalJsonModel` is for storing Pydantic models that contain a mixture of ordinary JSON
+values and Python-specific values. Ordinary values remain JSON, so the file structure can be
+inspected and diffed. Values that JSON cannot represent, such as NumPy arrays, tensors, and
+other Python objects, are stored as base64-encoded pickle payloads.
+
+The `save()` and `load()` methods support convient file-based workflows.
+`save()` serialises the model to a JSON file, while `load()` reconstructs the model from one.
+Use `to_json_dict()` and `from_json_dict(...)` when the serialized representation belongs inside
+another structure or is handled by another storage layer.
 
 ```py
 import numpy as np
@@ -193,11 +198,14 @@ result.save("results/run-1.json")
 result = Result.load("results/run-1.json")
 ```
 
-Use `to_json_dict()` / `from_json_dict(...)` to convert to and from a plain dict
-without touching the filesystem — useful for nesting inside other structures. The
-`pretty_json` helper function is also available on its own. The `serialization`
-module also includes JSONL read/write helpers (`jsonl_dump`, `jsonl_load`, ...)
-with largely the same interface as is provided by the built-in `json` module.
+`pretty_json` is a string-returning formatter for standalone dictionaries and lists. It fills the
+same role as `json.dumps`, with compact nested containers and wrapped primitive lists; write the
+returned string to a file when needed. The module also includes JSONL read/write helpers
+(`jsonl_dump`, `jsonl_load`, ...) for newline-delimited records.
+
+This format is intended for trusted configuration objects, application state, experiment results, etc., not as a
+language-neutral interchange format. Pickle payloads are Python-specific and loading them can
+execute arbitrary code. Do not load files from untrusted sources.
 
 ## Config and command-line argument parsing
 
@@ -254,18 +262,18 @@ values, index, inverse, counts = unique(
 
 ### Statistics
 
-Common statistical helpers, plus wrappers around scipy distributions reparametrised
-as in Jim Pitman's *Probability* (rather than scipy's `loc`/`scale`, which are
+Common statistical helpers, plus wrappers around SciPy distributions reparametrised
+as in Jim Pitman's *Probability* (rather than SciPy's `loc`/`scale`, which are
 unintuitive for many distributions).
 
 ```py
-from pelutils.stats import z_score
 from pelutils.stats import expon
+from pelutils.stats import z_score
 
-# 95 % confidence interval half-width for a standard normal (defaults give ~1.96)
-half_width = std * z_score()
+# 95 % confidence interval half-width for a standard normal
+half_width = z_score()  # standard deviation is 1
 
-# One-sided z value for an Exponential(λ=2) at the 1 % significance level
+# Upper 1 % critical value for an Exponential(lambda=2)
 zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 ```
 

{pelutils-4.3.0 → pelutils-4.3.1}/README.md

@@ -138,10 +138,15 @@ importantly one per thread, as profiling is not thread-safe.
 
 ## Serialisation
 
-`UniversalJsonModel` extends `pydantic.BaseModel` with `save`/`load` methods and can
-serialise attributes that pydantic cannot — numpy arrays, tensors, and arbitrary
-objects are base64-pickled inline, everything else stays plain, human-readable JSON.
-Long lists are wrapped to the line-length limit instead of one element per line.
+`UniversalJsonModel` is for storing Pydantic models that contain a mixture of ordinary JSON
+values and Python-specific values. Ordinary values remain JSON, so the file structure can be
+inspected and diffed. Values that JSON cannot represent, such as NumPy arrays, tensors, and
+other Python objects, are stored as base64-encoded pickle payloads.
+
+The `save()` and `load()` methods support convient file-based workflows.
+`save()` serialises the model to a JSON file, while `load()` reconstructs the model from one.
+Use `to_json_dict()` and `from_json_dict(...)` when the serialized representation belongs inside
+another structure or is handled by another storage layer.
 
 ```py
 import numpy as np
@@ -167,11 +172,14 @@ result.save("results/run-1.json")
 result = Result.load("results/run-1.json")
 ```
 
-Use `to_json_dict()` / `from_json_dict(...)` to convert to and from a plain dict
-without touching the filesystem — useful for nesting inside other structures. The
-`pretty_json` helper function is also available on its own. The `serialization`
-module also includes JSONL read/write helpers (`jsonl_dump`, `jsonl_load`, ...)
-with largely the same interface as is provided by the built-in `json` module.
+`pretty_json` is a string-returning formatter for standalone dictionaries and lists. It fills the
+same role as `json.dumps`, with compact nested containers and wrapped primitive lists; write the
+returned string to a file when needed. The module also includes JSONL read/write helpers
+(`jsonl_dump`, `jsonl_load`, ...) for newline-delimited records.
+
+This format is intended for trusted configuration objects, application state, experiment results, etc., not as a
+language-neutral interchange format. Pickle payloads are Python-specific and loading them can
+execute arbitrary code. Do not load files from untrusted sources.
 
 ## Config and command-line argument parsing
 
@@ -228,18 +236,18 @@ values, index, inverse, counts = unique(
 
 ### Statistics
 
-Common statistical helpers, plus wrappers around scipy distributions reparametrised
-as in Jim Pitman's *Probability* (rather than scipy's `loc`/`scale`, which are
+Common statistical helpers, plus wrappers around SciPy distributions reparametrised
+as in Jim Pitman's *Probability* (rather than SciPy's `loc`/`scale`, which are
 unintuitive for many distributions).
 
 ```py
-from pelutils.stats import z_score
 from pelutils.stats import expon
+from pelutils.stats import z_score
 
-# 95 % confidence interval half-width for a standard normal (defaults give ~1.96)
-half_width = std * z_score()
+# 95 % confidence interval half-width for a standard normal
+half_width = z_score()  # standard deviation is 1
 
-# One-sided z value for an Exponential(λ=2) at the 1 % significance level
+# Upper 1 % critical value for an Exponential(lambda=2)
 zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 ```
 

{pelutils-4.3.0 → pelutils-4.3.1}/pelutils/__version__.py

@@ -1,2 +1,2 @@
 # Do not put anything else in this file
-__version__ = "4.3.0"
+__version__ = "4.3.1"

{pelutils-4.3.0 → pelutils-4.3.1}/pelutils/array/_blob.py

@@ -1,7 +1,10 @@
 from __future__ import annotations
 
+from typing import SupportsIndex
+
 import _pelutils_c as _c
 import numpy as np
+from typing_extensions import override
 
 import pelutils._c as c_utils
 from pelutils.types import BoolArray, IntArray
@@ -171,3 +174,21 @@ class SparseGridBlobDetection:
             # If the error happens early in init (e.g. wrong call to __init__), _pointer_args has not yet been defined,
             # and the hashmap has not been created
             pass
+
+    def __copy__(self):
+        """Prevent shallow copies of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")
+
+    def __deepcopy__(self, memo: dict[int, object]):
+        """Prevent deep copies of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")
+
+    @override
+    def __reduce__(self):
+        """Prevent pickling of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")
+
+    @override
+    def __reduce_ex__(self, protocol: SupportsIndex):
+        """Prevent protocol-specific pickling of the C-backed detector."""
+        raise TypeError("SparseGridBlobDetection instances cannot be copied or pickled")

pelutils-4.3.1/pelutils/serialization/__init__.py

@@ -0,0 +1,51 @@
+"""JSON persistence for Pydantic models containing non-JSON values.
+
+JSON is useful when the structure of a file should remain inspectable, but it does not represent
+values such as NumPy arrays, tensors, or arbitrary Python objects. ``UniversalJsonModel``
+keeps JSON-native values in the document and stores unsupported field values as base64-encoded
+pickle payloads. This preserves a readable JSON envelope without requiring callers to write
+converters for every field type.
+
+The usual interface is :meth:`UniversalJsonModel.save` and
+:meth:`UniversalJsonModel.load`. ``save`` creates missing parent directories and writes the model to a JSON
+file; ``load`` reconstructs the model from that file. Use :meth:`UniversalJsonModel.to_json_dict`
+and :meth:`UniversalJsonModel.from_json_dict` when the serialized representation should be nested
+inside another structure or handled by another storage layer.
+
+For standalone dictionaries and lists, ``pretty_json`` converts Python objects to pretty-formatted JSON,
+filling the same role as ``json.dumps``, but often making the resulting JSON more readable.
+JSONL helpers handle files containing multiple JSON records separated by newlines.
+
+Quick start
+-----------
+
+.. code-block:: python
+
+    import numpy as np
+    from pelutils.serialization import UniversalJsonModel
+    from pelutils.types import FloatArray
+
+    class Result(UniversalJsonModel):
+        accuracy: float
+        predictions: FloatArray   # numpy arrays are handled automatically
+
+    result = Result(accuracy=0.97, predictions=np.arange(5, dtype=np.float16))
+    result.save("results/run-1.json")
+    result = Result.load("results/run-1.json")
+
+This format is intended for trusted application data such as experiment results and cached state,
+not as a language-neutral interchange format. Pickle payloads are Python-specific, and loading
+them can execute arbitrary code. Use a format with an explicit schema and safe decoder when data
+comes from outside the application.
+
+.. warning::
+
+    ``UniversalJsonModel.load`` and the pickle fallback execute code while loading. Never load
+    files from an untrusted source.
+"""
+
+from ._jsonl import jsonl_dump, jsonl_dumps, jsonl_load, jsonl_loads
+from ._pretty_json import pretty_json
+from ._universal_json_model import UniversalJsonModel
+
+__all__ = ("UniversalJsonModel", "jsonl_dump", "jsonl_dumps", "jsonl_load", "jsonl_loads", "pretty_json")

{pelutils-4.3.0 → pelutils-4.3.1}/pelutils/stats/__init__.py

@@ -1,11 +1,11 @@
-"""Statistical helpers and scipy distributions parametrised the way textbooks are.
+"""Statistical helpers and SciPy distributions parametrised the way textbooks are.
 
 ``scipy.stats`` parametrises every distribution through generic ``loc``/``scale``
 arguments, which rarely match how a distribution is actually defined: an exponential's
 ``scale`` is ``1 / lambda``, a normal takes a standard deviation rather than a variance,
 and a gamma's ``a``/``scale`` bear no obvious relation to its shape and rate. This module
-wraps the scipy distributions so you pass their *natural* parameters instead — following
-the conventions in Jim Pitman's *Probability* — and returns an ordinary frozen scipy
+wraps the SciPy distributions so you pass their *natural* parameters instead — following
+the conventions in Jim Pitman's *Probability* — and returns an ordinary frozen SciPy
 distribution, so ``.pdf``, ``.cdf``, ``.rvs`` and friends all work as usual. It also
 provides :func:`z_score` for turning a significance level into a critical value.
 
@@ -17,12 +17,12 @@ Quick start
     from pelutils.stats import expon, norm, z_score
 
     dist = norm(mu=0, sigma2=4)   # mean and *variance*, not standard deviation
-    dist.cdf(1.5)                 # a plain frozen scipy distribution
+    dist.cdf(1.5)                 # a plain frozen SciPy distribution
 
-    # 95 % confidence-interval half-width for a standard normal (defaults give ~1.96)
-    half_width = std * z_score()
+    # 95 % confidence-interval half-width for a standard normal
+    half_width = z_score()        # standard deviation is 1
 
-    # One-sided z value for an Exponential(lambda=2) at the 1 % significance level
+    # Upper 1 % critical value for an Exponential(lambda=2)
     zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 
 Continuous distributions: :func:`norm`, :func:`lognorm`, :func:`expon`, :func:`gamma`,

{pelutils-4.3.0 → pelutils-4.3.1}/pelutils/stats/_distributions.py

@@ -1,4 +1,4 @@
-"""Functions returning scipy.stats distribution objects reparameterized to Jim Pitman's "Probability"."""
+"""Functions returning SciPy distribution objects reparameterized to Jim Pitman's "Probability"."""
 
 import numpy as np
 from scipy import stats
@@ -6,25 +6,25 @@ from scipy import stats
 
 # Continuous distributions
 def norm(mu: float, sigma2: float):
-    """Return a normal distribution from mean and variance."""
+    """Return a normal distribution with mean ``mu`` and variance ``sigma2``."""
     assert sigma2 > 0
     return stats.norm(loc=mu, scale=np.sqrt(sigma2))
 
 
 def lognorm(mu: float, sigma2: float):
-    """Return a log normal distribution from the mean and variance of the corresponding normal distribution."""
+    """Return a log-normal distribution where ``log(X)`` has mean ``mu`` and variance ``sigma2``."""
     assert sigma2 > 0
     return stats.lognorm(s=np.sqrt(sigma2), scale=np.exp(mu))
 
 
 def expon(lambda_: float):
-    """Return an exponential distribution."""
+    """Return an exponential distribution with rate ``lambda_`` and mean ``1 / lambda_``."""
     assert lambda_ > 0
     return stats.expon(scale=1 / lambda_)
 
 
 def gamma(r: float, lambda_: float):
-    """Return a gamma distribution."""
+    """Return a gamma distribution with shape ``r`` and rate ``lambda_``."""
     assert r > 0 and lambda_ > 0
     return stats.gamma(a=r, loc=0, scale=1 / lambda_)
 
@@ -32,15 +32,16 @@ def gamma(r: float, lambda_: float):
 def chi2(n: int | float):
     """Return a chi-squared distribution with the given number of degrees of freedom.
 
-    The chi-squared distribution is generally defined only on the natural numbers, but here being defined as a special case
-    of the gamma distribution, it generalises easily to positive rational numbers, hence why floats are accepted.
+    Although degrees of freedom are commonly given as an integer, the distribution
+    is here represented as a special case of the gamma distribution, making it defined
+    for any positive real ``n``.
     """
     assert n > 0
     return gamma(n / 2, 1 / 2)
 
 
 def rayleigh():
-    """Return a rayleigh distribution."""
+    """Return a Rayleigh distribution."""
     return stats.rayleigh()
 
 
@@ -52,42 +53,52 @@ def beta(r: float, s: float):
 
 # Discrete distributions
 def bernoulli(p: float):
-    """Return a Bernoulli distribution."""
+    """Return a Bernoulli distribution with success probability ``p``."""
     assert 0 <= p <= 1
     return stats.bernoulli(p)
 
 
 def binomial(n: int, p: float):
-    """Return a binomial distribution."""
+    """Return a binomial distribution for ``n`` trials with success probability ``p``."""
     assert n >= 0 and 0 <= p <= 1
     return stats.binom(n=n, p=p)
 
 
 def poisson(mu: float):
-    """Return a Poisson distribution."""
+    """Return a Poisson distribution with mean and rate ``mu``."""
     assert mu >= 0
     return stats.poisson(mu=mu)
 
 
 def hypergeom(n: int, N: int, G: int):  # noqa: N803
-    """Return a hypergeometric distribution."""
+    """Return a hypergeometric distribution.
+
+    Parameters
+    ----------
+    n:
+        Number of items drawn from the population.
+    N:
+        Population size.
+    G:
+        Number of successes in the population.
+    """
     assert N > n > 0 and N >= G > 0 and N > 1
     return stats.hypergeom(M=N, n=G, N=n)
 
 
 def geom0(p: float):
-    """Return a geometric distribution defined on {0, 1, ...}."""
+    """Return a geometric distribution of failures before the first success, with success probability ``p``."""
     assert 0 < p <= 1
     return stats.geom(loc=-1, p=p)
 
 
 def geom1(p: float):
-    """Return a geometric distribution defined on {1, 2, ...}."""
+    """Return a geometric distribution of the trial of the first success, with success probability ``p``."""
     assert 0 < p <= 1
     return stats.geom(p=p)
 
 
 def nbinom(r: int, p: float):
-    """Return a negative binomial distribution."""
+    """Return a negative binomial distribution of failures before ``r`` successes, with success probability ``p``."""
     assert r > 0 and 0 <= p <= 1
     return stats.nbinom(n=r, p=p)

{pelutils-4.3.0 → pelutils-4.3.1}/pelutils/stats/_stats.py

@@ -4,16 +4,20 @@ from ._distributions import norm
 
 
 def z_score(alpha: float = 0.05, two_sided: bool = True, distribution: Any | None = None) -> float:  # pyright: ignore[reportExplicitAny]
-    """Return the critical value (z score) for a given significance level.
+    """Return an upper critical value for a given significance level.
 
     Parameters
     ----------
     alpha : float, optional
         Significance level in ``[0, 1]``.
     two_sided : bool, optional
-        If ``True``, split ``alpha`` across both tails; otherwise use a single tail.
+        If ``True``, use an upper-tail probability of ``alpha / 2``; otherwise use
+        ``alpha``. For symmetric distributions this gives the usual two-sided
+        cutoff for a distribution symmetric about zero, with the lower cutoff equal
+        to the negative of the returned value.
+        For asymmetric distributions, the returned value is only the upper cutoff.
     distribution : Any | None, optional
-        A frozen scipy distribution to draw the quantile from. Defaults to ``N(0, 1)``,
+        A frozen SciPy distribution to draw the quantile from. Defaults to ``N(0, 1)``,
         in which case the two-sided default returns the familiar ``~1.96``.
     """
     if not 0 <= alpha <= 1:

{pelutils-4.3.0 → pelutils-4.3.1}/pelutils.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: pelutils
-Version: 4.3.0
+Version: 4.3.1
 Summary: The Swiss army knife of Python projects
 Home-page: https://github.com/peleiden/pelutils
 Download-URL: https://pypi.org/project/pelutils/
@@ -164,10 +164,15 @@ importantly one per thread, as profiling is not thread-safe.
 
 ## Serialisation
 
-`UniversalJsonModel` extends `pydantic.BaseModel` with `save`/`load` methods and can
-serialise attributes that pydantic cannot — numpy arrays, tensors, and arbitrary
-objects are base64-pickled inline, everything else stays plain, human-readable JSON.
-Long lists are wrapped to the line-length limit instead of one element per line.
+`UniversalJsonModel` is for storing Pydantic models that contain a mixture of ordinary JSON
+values and Python-specific values. Ordinary values remain JSON, so the file structure can be
+inspected and diffed. Values that JSON cannot represent, such as NumPy arrays, tensors, and
+other Python objects, are stored as base64-encoded pickle payloads.
+
+The `save()` and `load()` methods support convient file-based workflows.
+`save()` serialises the model to a JSON file, while `load()` reconstructs the model from one.
+Use `to_json_dict()` and `from_json_dict(...)` when the serialized representation belongs inside
+another structure or is handled by another storage layer.
 
 ```py
 import numpy as np
@@ -193,11 +198,14 @@ result.save("results/run-1.json")
 result = Result.load("results/run-1.json")
 ```
 
-Use `to_json_dict()` / `from_json_dict(...)` to convert to and from a plain dict
-without touching the filesystem — useful for nesting inside other structures. The
-`pretty_json` helper function is also available on its own. The `serialization`
-module also includes JSONL read/write helpers (`jsonl_dump`, `jsonl_load`, ...)
-with largely the same interface as is provided by the built-in `json` module.
+`pretty_json` is a string-returning formatter for standalone dictionaries and lists. It fills the
+same role as `json.dumps`, with compact nested containers and wrapped primitive lists; write the
+returned string to a file when needed. The module also includes JSONL read/write helpers
+(`jsonl_dump`, `jsonl_load`, ...) for newline-delimited records.
+
+This format is intended for trusted configuration objects, application state, experiment results, etc., not as a
+language-neutral interchange format. Pickle payloads are Python-specific and loading them can
+execute arbitrary code. Do not load files from untrusted sources.
 
 ## Config and command-line argument parsing
 
@@ -254,18 +262,18 @@ values, index, inverse, counts = unique(
 
 ### Statistics
 
-Common statistical helpers, plus wrappers around scipy distributions reparametrised
-as in Jim Pitman's *Probability* (rather than scipy's `loc`/`scale`, which are
+Common statistical helpers, plus wrappers around SciPy distributions reparametrised
+as in Jim Pitman's *Probability* (rather than SciPy's `loc`/`scale`, which are
 unintuitive for many distributions).
 
 ```py
-from pelutils.stats import z_score
 from pelutils.stats import expon
+from pelutils.stats import z_score
 
-# 95 % confidence interval half-width for a standard normal (defaults give ~1.96)
-half_width = std * z_score()
+# 95 % confidence interval half-width for a standard normal
+half_width = z_score()  # standard deviation is 1
 
-# One-sided z value for an Exponential(λ=2) at the 1 % significance level
+# Upper 1 % critical value for an Exponential(lambda=2)
 zval = z_score(alpha=0.01, two_sided=False, distribution=expon(lambda_=2))
 ```
 

pelutils-4.3.0/pelutils/serialization/__init__.py

@@ -1,51 +0,0 @@
-"""JSON persistence for values that plain ``json`` and even ``pydantic`` cannot handle.
-
-Persisting data, whatever it may be, to disk usually usually comes down to a choice between
-human-readable or not. If efficiency is not crucial, human-readable is preferable, but often
-times, the data is not trivially serialisable to human-friendly formats. While ``pydantic``'s
-``BaseModel.model_dump`` takes you some of the way, it falls short as soon as you introduce
-data types which are not JSON-serialisable by default, making pickling the default choice and
-ending up with an unreadable binary blob.
-
-Even if all data is easily serialisable to a readable format, you still have to deal with all
-the boilerplate of ensuring parent directories exist, and opening and closing files.
-
-Enter :class:`UniversalJsonModel` — a ``pydantic.BaseModel`` whose
-``save``/``load`` methods serialise/deserialise *any* attribute: JSON-native values stay plain and
-human-readable, while anything else (numpy arrays, torch tensors, arbitrary objects) is
-pickled and base64-encoded inline. The output is a single, easily readable JSON file.
-
-Quick start
------------
-
-.. code-block:: python
-
-    import numpy as np
-    from pelutils.serialization import UniversalJsonModel
-    from pelutils.types import FloatArray
-
-    class Result(UniversalJsonModel):
-        accuracy: float
-        predictions: FloatArray   # numpy arrays are handled automatically
-
-    result = Result(accuracy=0.97, predictions=np.arange(5, dtype=np.float16))
-    result.save("results/run-1.json")
-    result = Result.load("results/run-1.json")
-
-The module also exposes :func:`pretty_json`, a function similar to the built-in `json.dumps`,
-but which formats the given object to a pretty JSON string;
-short containers stay on one line, and long primitive lists are packed across both the width and height
-of the file.
-JSONL helpers (:func:`jsonl_dump`, :func:`jsonl_load`, and their string variants) for files where each
-line is its own JSON object are also provided.
-
-.. warning::
-
-    Loading pickles executes arbitrary code, so never ``load`` data from an untrusted source.
-"""
-
-from ._jsonl import jsonl_dump, jsonl_dumps, jsonl_load, jsonl_loads
-from ._pretty_json import pretty_json
-from ._universal_json_model import UniversalJsonModel
-
-__all__ = ("UniversalJsonModel", "jsonl_dump", "jsonl_dumps", "jsonl_load", "jsonl_loads", "pretty_json")