bw-processing 1.0__tar.gz → 1.1.1__tar.gz

bw_processing-1.1.1/MANIFEST.in

@@ -0,0 +1,3 @@
+include src/bw_processing/examples/*.zip
+include src/bw_processing/examples/datapackage_1/*.parquet
+recursive-exclude dev *

{bw_processing-1.0/bw_processing.egg-info → bw_processing-1.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: bw_processing
-Version: 1.0
+Version: 1.1.1
 Summary: Foo
 Author-email: Chris Mutel <cmutel@gmail.com>
 Maintainer-email: Chris Mutel <cmutel@gmail.com>
@@ -38,6 +38,7 @@ Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
 Requires-Dist: pytest-randomly; extra == "dev"
 Requires-Dist: setuptools; extra == "dev"
+Dynamic: license-file
 
 # bw-processing
 
@@ -81,7 +82,7 @@ The [Brightway LCA framework](https://brightway.dev/) has stored data used in co
 * **Support for vector and array data**. Vector (i.e. only one possible value per input) and array (i.e. many possible values, also called presamples) data are now both natively supported in data packages.
 * **Portability**. Processed arrays can include metadata that allows for reindexing on other machines, so that processed arrays can be distributed and reused. Before, this was not possible, as integer IDs were randomly assigned on each computer, and would be different from machine to machine or even across Brightway projects.
 * **Dynamic data sources**. Instead of requiring that data for matrix construction be present and savedd on disk, it can now be generated dynamically, either through code running locally or on another computer system. This is a big step towards embeddding life cycle assessment in a web of environmental models.
-* **Use [PyFilesystem2](https://docs.pyfilesystem.org/en/latest/) for file IO**. The use of this library allows for data packages to be stored on your local computer, or on [many logical or virtual file systems](https://docs.pyfilesystem.org/en/latest/guide.html).
+* **Use [fsspec](https://filesystem-spec.readthedocs.io/en/latest/) for file IO**. The use of this library allows for data packages to be stored on your local computer, or on [many logical or virtual file systems](https://docs.pyfilesystem.org/en/latest/guide.html).
 * **Simpler handling of numeric values whose sign should be flipped**. Sometimes it is more convenient to specify positive numbers in dataset definitions, even though such numbers should be negative when inserted into the resulting matrices. For example, in the technosphere matrix in life cycle assessment, products produced are positive and products consumed are negative, though both values are given as positive in datasets. Brightway used to use a type mapping dictionary to indicate which values in a matrix should have their sign flipped after insertion. Such mapping dictionaries are brittle and inelegant. `bw_processing` uses an optional boolean vector, called `flip`, to indicate if any values should be flipped.
 * **Separation of uncertainty distribution parameters from other data**. Fitting data to a [probability density function](https://en.wikipedia.org/wiki/Probability_density_function) (PDF), or an estimate of such a PDF, is only one approach to quantitative uncertainty analysis. We would like to support other approaches, including [direct sampling from real data](https://github.com/PascalLesage/presamples/). Therefore, uncertainty distribution parameters are stored separately,  only loaded if needed, and are only one way to express quantitative uncertainty.
 

{bw_processing-1.0 → bw_processing-1.1.1}/README.md

@@ -40,7 +40,7 @@ The [Brightway LCA framework](https://brightway.dev/) has stored data used in co
 * **Support for vector and array data**. Vector (i.e. only one possible value per input) and array (i.e. many possible values, also called presamples) data are now both natively supported in data packages.
 * **Portability**. Processed arrays can include metadata that allows for reindexing on other machines, so that processed arrays can be distributed and reused. Before, this was not possible, as integer IDs were randomly assigned on each computer, and would be different from machine to machine or even across Brightway projects.
 * **Dynamic data sources**. Instead of requiring that data for matrix construction be present and savedd on disk, it can now be generated dynamically, either through code running locally or on another computer system. This is a big step towards embeddding life cycle assessment in a web of environmental models.
-* **Use [PyFilesystem2](https://docs.pyfilesystem.org/en/latest/) for file IO**. The use of this library allows for data packages to be stored on your local computer, or on [many logical or virtual file systems](https://docs.pyfilesystem.org/en/latest/guide.html).
+* **Use [fsspec](https://filesystem-spec.readthedocs.io/en/latest/) for file IO**. The use of this library allows for data packages to be stored on your local computer, or on [many logical or virtual file systems](https://docs.pyfilesystem.org/en/latest/guide.html).
 * **Simpler handling of numeric values whose sign should be flipped**. Sometimes it is more convenient to specify positive numbers in dataset definitions, even though such numbers should be negative when inserted into the resulting matrices. For example, in the technosphere matrix in life cycle assessment, products produced are positive and products consumed are negative, though both values are given as positive in datasets. Brightway used to use a type mapping dictionary to indicate which values in a matrix should have their sign flipped after insertion. Such mapping dictionaries are brittle and inelegant. `bw_processing` uses an optional boolean vector, called `flip`, to indicate if any values should be flipped.
 * **Separation of uncertainty distribution parameters from other data**. Fitting data to a [probability density function](https://en.wikipedia.org/wiki/Probability_density_function) (PDF), or an estimate of such a PDF, is only one approach to quantitative uncertainty analysis. We would like to support other approaches, including [direct sampling from real data](https://github.com/PascalLesage/presamples/). Therefore, uncertainty distribution parameters are stored separately,  only loaded if needed, and are only one way to express quantitative uncertainty.
 

{bw_processing-1.0 → bw_processing-1.1.1}/pyproject.toml

@@ -66,7 +66,9 @@ dev = [
 [tool.setuptools]
 license-files = ["LICENSE"]
 include-package-data = true
-packages = ["bw_processing", "bw_processing.examples"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
 
 [tool.setuptools.dynamic]
 version = {attr = "bw_processing.__version__"}

bw_processing-1.1.1/src/bw_processing/__init__.py

@@ -0,0 +1,52 @@
+__all__ = (
+    "__version__",
+    "as_unique_attributes",
+    "as_unique_attributes_dataframe",
+    "clean_datapackage_name",
+    "create_array",
+    "create_datapackage",
+    "create_datapackage_from_entries",
+    "create_structured_array",
+    "Datapackage",
+    "DatapackageBase",
+    "DEFAULT_LICENSES",
+    "examples_dir",
+    "FilteredDatapackage",
+    "generic_directory_filesystem",
+    "generic_zipfile_filesystem",
+    "INDICES_DTYPE",
+    "load_datapackage",
+    "MatrixEntry",
+    "MatrixName",
+    "MatrixSerializeFormat",
+    "md5",
+    "merge_datapackages_with_mask",
+    "reindex",
+    "reset_index",
+    "safe_filename",
+    "simple_graph",
+    "UNCERTAINTY_DTYPE",
+    "UndefinedInterface",
+)
+
+__version__ = "1.1.1"
+
+
+from bw_processing.array_creation import create_array, create_structured_array
+from bw_processing.constants import DEFAULT_LICENSES, INDICES_DTYPE, UNCERTAINTY_DTYPE, MatrixSerializeFormat
+from bw_processing.datapackage import (
+    Datapackage,
+    DatapackageBase,
+    FilteredDatapackage,
+    create_datapackage,
+    load_datapackage,
+    simple_graph,
+)
+from bw_processing.examples import examples_dir
+from bw_processing.filesystem import clean_datapackage_name, md5, safe_filename
+from bw_processing.indexing import reindex, reset_index
+from bw_processing.io_helpers import generic_directory_filesystem, generic_zipfile_filesystem
+from bw_processing.matrix_entry import MatrixEntry, MatrixName, create_datapackage_from_entries
+from bw_processing.merging import merge_datapackages_with_mask
+from bw_processing.proxies import UndefinedInterface
+from bw_processing.unique_fields import as_unique_attributes, as_unique_attributes_dataframe

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/datapackage.py

@@ -1,8 +1,12 @@
 import datetime
 import uuid
+import warnings
 from abc import ABC
 from functools import partial
-from typing import Any, Dict, List, Optional, Union
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Union
+
+if TYPE_CHECKING:
+    from bw_processing.matrix_entry import MatrixEntry
 
 import numpy as np
 import pandas as pd
@@ -11,7 +15,7 @@ from fsspec import AbstractFileSystem
 # Use this instead of fsspec MemoryFileSystem because that is a singleton!?
 from morefs.dict import DictFS
 
-from .constants import (
+from bw_processing.constants import (
     DEFAULT_LICENSES,
     INDICES_DTYPE,
     MAX_SIGNED_32BIT_INT,
@@ -22,7 +26,7 @@ from .constants import (
     PARQUET_SERIALIZE_FORMAT_NAME,
     MatrixSerializeFormat,
 )
-from .errors import (
+from bw_processing.errors import (
     Closed,
     InvalidMimetype,
     LengthMismatch,
@@ -31,10 +35,10 @@ from .errors import (
     ShapeMismatch,
     WrongDatatype,
 )
-from .filesystem import clean_datapackage_name
-from .io_helpers import file_reader, file_writer
-from .proxies import Proxy, UndefinedInterface
-from .utils import check_name, check_suffix, load_bytes, resolve_dict_iterator, utc_now
+from bw_processing.filesystem import clean_datapackage_name
+from bw_processing.io_helpers import file_reader, file_writer
+from bw_processing.proxies import Proxy, UndefinedInterface
+from bw_processing.utils import check_name, check_suffix, load_bytes, resolve_dict_iterator, utc_now
 
 
 class DatapackageBase(ABC):
@@ -448,6 +452,30 @@ class Datapackage(DatapackageBase):
             **kwargs,
         )
 
+    def add_entries(
+        self,
+        *,
+        matrix: str,
+        entries: list["MatrixEntry"],
+        name: Optional[str] = None,
+    ) -> None:
+        """Add matrix data from a list of :class:`MatrixEntry` objects.
+
+        High-level convenience method that does not require working directly
+        with NumPy arrays.
+
+        Args:
+            matrix: Name of the target matrix (e.g. ``"technosphere"``).
+            entries: List of :class:`.MatrixEntry` instances.
+            name: Optional resource group name; auto-generated if omitted.
+        """
+        self.add_persistent_vector_from_iterator(
+            matrix=matrix,
+            name=name,
+            dict_iterator=(e.as_dict() for e in entries),
+            nrows=len(entries),
+        )
+
     def add_persistent_vector(
         self,
         *,  # Forces use of keyword arguments
@@ -1091,6 +1119,10 @@ def load_datapackage(
 def simple_graph(data: dict, fs: Optional[AbstractFileSystem] = None, **metadata) -> Datapackage:
     """Easy creation of simple datapackages with only persistent vectors.
 
+    .. deprecated::
+        Use :func:`bw_processing.matrix_entry.create_datapackage_from_entries` with
+        :class:`bw_processing.matrix_entry.MatrixEntry` objects instead.
+
     Args:
         * data: is a dictionary.
             The data dictionary has the form::
@@ -1111,6 +1143,11 @@ def simple_graph(data: dict, fs: Optional[AbstractFileSystem] = None, **metadata
         the datapackage.
 
     """
+    warnings.warn(
+        "simple_graph is deprecated; use create_datapackage_from_entries with MatrixEntry objects instead.",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     dp = create_datapackage(fs=fs or DictFS(), **metadata)
     for key, value in data.items():
         indices_array = np.array([row[:2] for row in value], dtype=INDICES_DTYPE)

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/indexing.py

@@ -5,8 +5,8 @@ import numpy as np
 import pandas as pd
 from fsspec import AbstractFileSystem
 
-from .datapackage import Datapackage, load_datapackage
-from .errors import NonUnique
+from bw_processing.datapackage import Datapackage, load_datapackage
+from bw_processing.errors import NonUnique
 
 
 def _get_csv_data(

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/io_helpers.py

@@ -11,12 +11,12 @@ from fsspec.implementations.local import LocalFileSystem
 from fsspec.implementations.zip import ZipFileSystem
 from morefs.dict import DictFS
 
-from .constants import MatrixSerializeFormat
-from .errors import InvalidMimetype
-from .proxies import Proxy
+from bw_processing.constants import MatrixSerializeFormat
+from bw_processing.errors import InvalidMimetype
+from bw_processing.proxies import Proxy
 
 try:
-    from .io_parquet_helpers import load_ndarray_from_parquet, save_arr_to_parquet
+    from bw_processing.io_parquet_helpers import load_ndarray_from_parquet, save_arr_to_parquet
 
     PARQUET = True
 except ImportError:

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/io_parquet_helpers.py

@@ -13,8 +13,8 @@ import numpy
 import numpy as np
 import pyarrow.parquet as pq
 
-from .errors import WrongDatatype
-from .io_pyarrow_helpers import (
+from bw_processing.errors import WrongDatatype
+from bw_processing.io_pyarrow_helpers import (
     numpy_distributions_vector_to_pyarrow_distributions_vector_table,
     numpy_generic_matrix_to_pyarrow_generic_matrix_table,
     numpy_generic_vector_to_pyarrow_generic_vector_table,

bw_processing-1.1.1/src/bw_processing/matrix_entry.py

@@ -0,0 +1,128 @@
+import dataclasses
+import math
+from enum import Enum
+
+try:
+    from stats_arrays import NoUncertainty, UndefinedUncertainty
+    _NO_UNCERTAINTY_IDS = (UndefinedUncertainty.id, NoUncertainty.id)
+except ImportError:
+    _NO_UNCERTAINTY_IDS = (0, 1)
+
+
+class MatrixName(str, Enum):
+    """Standard matrix names used in Brightway.
+
+    Because this is a ``str`` enum, members can be used anywhere a plain
+    string is accepted — no ``.value`` needed::
+
+        MatrixEntry(row=1, col=4, amount=2.5)  # inside a dict keyed by MatrixName
+        dp.add_entries(matrix=MatrixName.technosphere, entries=[...])
+
+    Derived libraries may define additional matrices as plain strings;
+    these three cover the core Brightway LCA workflow.
+    """
+
+    technosphere = "technosphere_matrix"
+    biosphere = "biosphere_matrix"
+    characterization = "characterization_matrix"
+
+    def __str__(self) -> str:
+        return self.value
+
+
+@dataclasses.dataclass(frozen=True)
+class MatrixEntry:
+    """A single entry destined for a matrix cell.
+
+    Multiple instances with the same (row, col) are summed during matrix
+    construction, so this is not necessarily the final cell value.
+
+    Field names and defaults match those expected by bw_processing's
+    ``dictionary_formatter``. Convert to a plain dict with ``as_dict()``
+    before passing to bw_processing internals.
+
+    Args:
+        row: Integer row index in the target matrix.
+        col: Integer column index in the target matrix.
+        amount: The numeric value to place at (row, col).
+        flip: If True, multiply the value by -1 when building the matrix.
+        uncertainty_type: Probability distribution type (0 = no uncertainty,
+            2 = lognormal, 3 = normal, etc. — see stats_arrays for full list).
+        loc: Distribution location parameter. For lognormal this is the log
+            of the median; defaults to NaN (no uncertainty).
+        scale: Distribution scale parameter (e.g. standard deviation).
+        shape: Distribution shape parameter.
+        minimum: Lower bound for distribution sampling.
+        maximum: Upper bound for distribution sampling.
+        negative: Whether the underlying value is negative.
+    """
+
+    row: int
+    col: int
+    amount: float
+    flip: bool = False
+    uncertainty_type: int = 0
+    loc: float = math.nan
+    scale: float = math.nan
+    shape: float = math.nan
+    minimum: float = math.nan
+    maximum: float = math.nan
+    negative: bool = False
+
+    def __post_init__(self):
+        if self.uncertainty_type in _NO_UNCERTAINTY_IDS:
+            if math.isnan(self.loc):
+                object.__setattr__(self, "loc", self.amount)
+            elif self.loc != self.amount:
+                raise ValueError(
+                    f"uncertainty_type {self.uncertainty_type} requires loc == amount, "
+                    f"got loc={self.loc} but amount={self.amount}"
+                )
+
+    def as_dict(self) -> dict:
+        return dataclasses.asdict(self)
+
+
+def create_datapackage_from_entries(
+    data: dict,
+    fs=None,
+    **metadata,
+):
+    """Create a datapackage from a dictionary of :class:`MatrixEntry` lists.
+
+    This is the recommended high-level entry point for building datapackages
+    without working directly with NumPy arrays.
+
+    Args:
+        data: Dictionary mapping matrix names to lists of :class:`MatrixEntry`
+            objects. Use :class:`MatrixName` members as keys for the standard
+            Brightway matrices; derived libraries may use plain strings for
+            additional matrices::
+
+                {
+                    MatrixName.technosphere: [
+                        MatrixEntry(row=1, col=4, amount=2.5),
+                        MatrixEntry(row=2, col=5, amount=7.0, flip=True),
+                    ],
+                    MatrixName.biosphere: [
+                        MatrixEntry(row=10, col=4, amount=0.3),
+                    ],
+                }
+
+        fs: Optional filesystem. Defaults to an in-memory filesystem.
+        **metadata: Additional keyword arguments passed to
+            :func:`create_datapackage` (e.g. ``name``, ``id_``).
+
+    Returns:
+        A :class:`Datapackage` instance.
+    """
+    from bw_processing.datapackage import create_datapackage
+
+    dp = create_datapackage(fs=fs, **metadata)
+    for matrix_name, entries in data.items():
+        dp.add_entries(
+            matrix=matrix_name,
+            entries=entries,
+            name=f"{matrix_name}-data",
+        )
+    return dp

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/merging.py

@@ -7,9 +7,9 @@ import pandas as pd
 from fsspec import AbstractFileSystem
 from morefs.dict import DictFS
 
-from .datapackage import DatapackageBase, create_datapackage
-from .errors import LengthMismatch
-from .io_helpers import file_writer
+from bw_processing.datapackage import DatapackageBase, create_datapackage
+from bw_processing.errors import LengthMismatch
+from bw_processing.io_helpers import file_writer
 
 
 def mask_resource(obj: Any, mask: np.ndarray) -> Any:

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/unique_fields.py

@@ -1,6 +1,6 @@
 import pandas as pd
 
-from .errors import InconsistentFields, NonUnique
+from bw_processing.errors import InconsistentFields, NonUnique
 
 
 def greedy_set_cover(data, exclude=None, raise_error=True):

{bw_processing-1.0 → bw_processing-1.1.1/src}/bw_processing/utils.py

@@ -6,9 +6,9 @@ from typing import Any, Union
 import numpy as np
 from numpy.lib.recfunctions import repack_fields
 
-from .array_creation import create_structured_array
-from .constants import INDICES_DTYPE, NAME_RE, UNCERTAINTY_DTYPE
-from .errors import InvalidName
+from bw_processing.array_creation import create_structured_array
+from bw_processing.constants import INDICES_DTYPE, NAME_RE, UNCERTAINTY_DTYPE
+from bw_processing.errors import InvalidName
 
 
 def load_bytes(obj: Any) -> Any:

{bw_processing-1.0 → bw_processing-1.1.1/src/bw_processing.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: bw_processing
-Version: 1.0
+Version: 1.1.1
 Summary: Foo
 Author-email: Chris Mutel <cmutel@gmail.com>
 Maintainer-email: Chris Mutel <cmutel@gmail.com>
@@ -38,6 +38,7 @@ Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
 Requires-Dist: pytest-randomly; extra == "dev"
 Requires-Dist: setuptools; extra == "dev"
+Dynamic: license-file
 
 # bw-processing
 
@@ -81,7 +82,7 @@ The [Brightway LCA framework](https://brightway.dev/) has stored data used in co
 * **Support for vector and array data**. Vector (i.e. only one possible value per input) and array (i.e. many possible values, also called presamples) data are now both natively supported in data packages.
 * **Portability**. Processed arrays can include metadata that allows for reindexing on other machines, so that processed arrays can be distributed and reused. Before, this was not possible, as integer IDs were randomly assigned on each computer, and would be different from machine to machine or even across Brightway projects.
 * **Dynamic data sources**. Instead of requiring that data for matrix construction be present and savedd on disk, it can now be generated dynamically, either through code running locally or on another computer system. This is a big step towards embeddding life cycle assessment in a web of environmental models.
-* **Use [PyFilesystem2](https://docs.pyfilesystem.org/en/latest/) for file IO**. The use of this library allows for data packages to be stored on your local computer, or on [many logical or virtual file systems](https://docs.pyfilesystem.org/en/latest/guide.html).
+* **Use [fsspec](https://filesystem-spec.readthedocs.io/en/latest/) for file IO**. The use of this library allows for data packages to be stored on your local computer, or on [many logical or virtual file systems](https://docs.pyfilesystem.org/en/latest/guide.html).
 * **Simpler handling of numeric values whose sign should be flipped**. Sometimes it is more convenient to specify positive numbers in dataset definitions, even though such numbers should be negative when inserted into the resulting matrices. For example, in the technosphere matrix in life cycle assessment, products produced are positive and products consumed are negative, though both values are given as positive in datasets. Brightway used to use a type mapping dictionary to indicate which values in a matrix should have their sign flipped after insertion. Such mapping dictionaries are brittle and inelegant. `bw_processing` uses an optional boolean vector, called `flip`, to indicate if any values should be flipped.
 * **Separation of uncertainty distribution parameters from other data**. Fitting data to a [probability density function](https://en.wikipedia.org/wiki/Probability_density_function) (PDF), or an estimate of such a PDF, is only one approach to quantitative uncertainty analysis. We would like to support other approaches, including [direct sampling from real data](https://github.com/PascalLesage/presamples/). Therefore, uncertainty distribution parameters are stored separately,  only loaded if needed, and are only one way to express quantitative uncertainty.
 

bw_processing-1.1.1/src/bw_processing.egg-info/SOURCES.txt

@@ -0,0 +1,52 @@
+LICENSE
+MANIFEST.in
+README.md
+pyproject.toml
+src/bw_processing/__init__.py
+src/bw_processing/array_creation.py
+src/bw_processing/constants.py
+src/bw_processing/datapackage.py
+src/bw_processing/errors.py
+src/bw_processing/filesystem.py
+src/bw_processing/indexing.py
+src/bw_processing/io_helpers.py
+src/bw_processing/io_parquet_helpers.py
+src/bw_processing/io_pyarrow_helpers.py
+src/bw_processing/matrix_entry.py
+src/bw_processing/merging.py
+src/bw_processing/proxies.py
+src/bw_processing/unique_fields.py
+src/bw_processing/utils.py
+src/bw_processing.egg-info/PKG-INFO
+src/bw_processing.egg-info/SOURCES.txt
+src/bw_processing.egg-info/dependency_links.txt
+src/bw_processing.egg-info/requires.txt
+src/bw_processing.egg-info/top_level.txt
+src/bw_processing/examples/__init__.py
+src/bw_processing/examples/datapackage_2.zip
+src/bw_processing/examples/interfaces.py
+src/bw_processing/examples/parquet_files.py
+src/bw_processing/examples/simple.zip
+src/bw_processing/examples/datapackage_1/another name.indices.parquet
+src/bw_processing/examples/datapackage_1/sa-data-vector-from-dict.data.parquet
+src/bw_processing/examples/datapackage_1/sa-data-vector-from-dict.distributions.parquet
+src/bw_processing/examples/datapackage_1/sa-data-vector-from-dict.flip.parquet
+src/bw_processing/examples/datapackage_1/sa-data-vector-from-dict.indices.parquet
+src/bw_processing/examples/datapackage_1/some name.data.parquet
+src/bw_processing/examples/datapackage_1/some name.flip.parquet
+src/bw_processing/examples/datapackage_1/some name.indices.parquet
+tests/test_array_creation.py
+tests/test_datapackage.py
+tests/test_filesystem.py
+tests/test_filtered_datapackage.py
+tests/test_indexing.py
+tests/test_integration.py
+tests/test_interfaces.py
+tests/test_io_parquet_helpers.py
+tests/test_io_pyarrow_helpers.py
+tests/test_loading.py
+tests/test_matrix_entry.py
+tests/test_merging.py
+tests/test_proxies.py
+tests/test_unique_fields.py
+tests/test_utils.py

{bw_processing-1.0 → bw_processing-1.1.1}/tests/test_io_parquet_helpers.py

@@ -59,11 +59,8 @@ def test_save_load_parquet_file_data_matrix(dtype, tmp_path_factory):
     arr = data_matrix(dtype=dtype)
     file = tmp_path_factory.mktemp("data") / "data_matrix.parquet"
 
-    with file as fp:
-        save_arr_to_parquet(file=fp, arr=arr, meta_object="matrix", meta_type="generic")
-
-    with file as fp:
-        loaded_arr = load_ndarray_from_parquet(fp)
+    save_arr_to_parquet(file=file, arr=arr, meta_object="matrix", meta_type="generic")
+    loaded_arr = load_ndarray_from_parquet(file)
 
     assert arr.dtype == loaded_arr.dtype and np.array_equal(arr, loaded_arr)
 
@@ -74,11 +71,8 @@ def test_save_load_parquet_file_distribution_vector(distributions_vector, tmp_pa
     arr = distributions_vector
     file = tmp_path_factory.mktemp("data") / "distributions_vector.parquet"
 
-    with file as fp:
-        save_arr_to_parquet(file=fp, arr=arr, meta_object="vector", meta_type="distributions")
-
-    with file as fp:
-        loaded_arr = load_ndarray_from_parquet(fp)
+    save_arr_to_parquet(file=file, arr=arr, meta_object="vector", meta_type="distributions")
+    loaded_arr = load_ndarray_from_parquet(file)
 
     assert vector_equal_with_uncertainty_dtype(arr, loaded_arr)
 
@@ -88,25 +82,21 @@ def test_save_load_parquet_file_wrong_meta_object(indices_vector, tmp_path_facto
     file = tmp_path_factory.mktemp("data") / "indices_vector.parquet"
 
     with pytest.raises(NotImplementedError):
-        with file as fp:
-            save_arr_to_parquet(
-                file=fp, arr=indices_vector, meta_object="wrong", meta_type="indices"
-            )
+        save_arr_to_parquet(
+            file=file, arr=indices_vector, meta_object="wrong", meta_type="indices"
+        )
 
     with pytest.raises(NotImplementedError):
-        with file as fp:
-            save_arr_to_parquet(
-                file=fp, arr=indices_vector, meta_object="vector", meta_type="indices"
-            )
+        save_arr_to_parquet(
+            file=file, arr=indices_vector, meta_object="vector", meta_type="indices"
+        )
 
-        with file as fp:
-            table = pq.read_table(fp)
-            metadata = {"object": "wrong", "type": "indices"}
-            new_table = table.replace_schema_metadata(metadata=metadata)
-            pq.write_table(new_table, fp)
+        table = pq.read_table(file)
+        metadata = {"object": "wrong", "type": "indices"}
+        new_table = table.replace_schema_metadata(metadata=metadata)
+        pq.write_table(new_table, file)
 
-        with file as fp:
-            load_ndarray_from_parquet(fp)
+        load_ndarray_from_parquet(file)
 
 
 @pytest.mark.skipif(sys.version_info[:2] == (3, 8), reason="Doesn't work in CI filesystem")
@@ -114,25 +104,21 @@ def test_save_load_parquet_file_wrong_meta_type(indices_vector, tmp_path_factory
     file = tmp_path_factory.mktemp("data") / "indices_vector.parquet"
 
     with pytest.raises(NotImplementedError):
-        with file as fp:
-            save_arr_to_parquet(
-                file=fp, arr=indices_vector, meta_object="vector", meta_type="wrong"
-            )
+        save_arr_to_parquet(
+            file=file, arr=indices_vector, meta_object="vector", meta_type="wrong"
+        )
 
     with pytest.raises(NotImplementedError):
-        with file as fp:
-            save_arr_to_parquet(
-                file=fp, arr=indices_vector, meta_object="vector", meta_type="indices"
-            )
+        save_arr_to_parquet(
+            file=file, arr=indices_vector, meta_object="vector", meta_type="indices"
+        )
 
-        with file as fp:
-            table = pq.read_table(fp)
-            metadata = {"object": "vector", "type": "wrong"}
-            new_table = table.replace_schema_metadata(metadata=metadata)
-            pq.write_table(new_table, fp)
+        table = pq.read_table(file)
+        metadata = {"object": "vector", "type": "wrong"}
+        new_table = table.replace_schema_metadata(metadata=metadata)
+        pq.write_table(new_table, file)
 
-        with file as fp:
-            load_ndarray_from_parquet(fp)
+        load_ndarray_from_parquet(file)
 
 
 @pytest.mark.skipif(sys.version_info[:2] == (3, 8), reason="Doesn't work in CI filesystem")
@@ -140,16 +126,13 @@ def test_save_load_parquet_file_wrong_metadata_format(indices_vector, tmp_path_f
     file = tmp_path_factory.mktemp("data") / "indices_vector.parquet"
 
     with pytest.raises(WrongDatatype):
-        with file as fp:
-            save_arr_to_parquet(
-                file=fp, arr=indices_vector, meta_object="vector", meta_type="indices"
-            )
-
-        with file as fp:
-            table = pq.read_table(fp)
-            metadata = {}
-            new_table = table.replace_schema_metadata(metadata=metadata)
-            pq.write_table(new_table, fp)
-
-        with file as fp:
-            load_ndarray_from_parquet(fp)
+        save_arr_to_parquet(
+            file=file, arr=indices_vector, meta_object="vector", meta_type="indices"
+        )
+
+        table = pq.read_table(file)
+        metadata = {}
+        new_table = table.replace_schema_metadata(metadata=metadata)
+        pq.write_table(new_table, file)
+
+        load_ndarray_from_parquet(file)

bw_processing-1.1.1/tests/test_matrix_entry.py

@@ -0,0 +1,178 @@
+import math
+import warnings
+
+import numpy as np
+import pytest
+
+from bw_processing import (
+    MatrixEntry,
+    MatrixName,
+    create_datapackage_from_entries,
+    simple_graph,
+)
+from bw_processing.constants import UNCERTAINTY_DTYPE
+
+
+class TestMatrixName:
+    def test_values(self):
+        assert MatrixName.technosphere == "technosphere_matrix"
+        assert MatrixName.biosphere == "biosphere_matrix"
+        assert MatrixName.characterization == "characterization_matrix"
+
+    def test_str(self):
+        assert str(MatrixName.technosphere) == "technosphere_matrix"
+
+    def test_fstring(self):
+        assert f"{MatrixName.biosphere}" == "biosphere_matrix"
+
+    def test_usable_as_dict_key(self):
+        d = {MatrixName.technosphere: [1, 2, 3]}
+        assert d["technosphere_matrix"] == [1, 2, 3]
+
+
+class TestMatrixEntry:
+    def test_required_fields(self):
+        e = MatrixEntry(row=1, col=2, amount=3.0)
+        assert e.row == 1
+        assert e.col == 2
+        assert e.amount == 3.0
+
+    def test_defaults(self):
+        e = MatrixEntry(row=1, col=2, amount=3.0)
+        assert e.flip is False
+        assert e.uncertainty_type == 0
+        assert e.negative is False
+        assert e.loc == pytest.approx(3.0)
+        assert math.isnan(e.scale)
+        assert math.isnan(e.shape)
+        assert math.isnan(e.minimum)
+        assert math.isnan(e.maximum)
+
+    def test_loc_set_to_amount_for_no_uncertainty(self):
+        e = MatrixEntry(row=1, col=2, amount=5.0)
+        assert e.loc == pytest.approx(5.0)
+
+    def test_explicit_loc_matching_amount_accepted(self):
+        e = MatrixEntry(row=1, col=2, amount=5.0, uncertainty_type=0, loc=5.0)
+        assert e.loc == pytest.approx(5.0)
+
+    def test_loc_mismatch_raises_for_uncertainty_type_0(self):
+        with pytest.raises(ValueError, match="loc == amount"):
+            MatrixEntry(row=1, col=2, amount=5.0, uncertainty_type=0, loc=9.9)
+
+    def test_loc_mismatch_raises_for_uncertainty_type_1(self):
+        with pytest.raises(ValueError, match="loc == amount"):
+            MatrixEntry(row=1, col=2, amount=5.0, uncertainty_type=1, loc=9.9)
+
+    def test_loc_not_set_when_uncertainty_type_nonzero(self):
+        e = MatrixEntry(row=1, col=2, amount=5.0, uncertainty_type=2)
+        assert math.isnan(e.loc)
+
+    def test_frozen(self):
+        e = MatrixEntry(row=1, col=2, amount=3.0)
+        with pytest.raises(Exception):
+            e.amount = 99.0
+
+    def test_as_dict_keys(self):
+        e = MatrixEntry(row=1, col=2, amount=3.0)
+        d = e.as_dict()
+        assert set(d.keys()) == {
+            "row", "col", "amount", "flip", "uncertainty_type",
+            "loc", "scale", "shape", "minimum", "maximum", "negative",
+        }
+
+    def test_as_dict_values(self):
+        e = MatrixEntry(row=5, col=10, amount=2.5, flip=True, uncertainty_type=2,
+                        loc=0.9, scale=0.1, negative=True)
+        d = e.as_dict()
+        assert d["row"] == 5
+        assert d["col"] == 10
+        assert d["amount"] == 2.5
+        assert d["flip"] is True
+        assert d["uncertainty_type"] == 2
+        assert d["loc"] == pytest.approx(0.9)
+        assert d["scale"] == pytest.approx(0.1)
+        assert d["negative"] is True
+
+
+class TestCreateDatapackageFromEntries:
+    def test_basic(self):
+        entries = [
+            MatrixEntry(row=1, col=4, amount=2.0),
+            MatrixEntry(row=2, col=5, amount=7.0),
+        ]
+        dp = create_datapackage_from_entries({MatrixName.technosphere: entries})
+        assert "technosphere_matrix-data" in dp.groups
+
+    def test_multiple_matrices(self):
+        dp = create_datapackage_from_entries({
+            MatrixName.technosphere: [MatrixEntry(row=1, col=2, amount=1.0)],
+            MatrixName.biosphere: [MatrixEntry(row=3, col=4, amount=0.5)],
+        })
+        groups = list(dp.groups.keys())
+        assert "technosphere_matrix-data" in groups
+        assert "biosphere_matrix-data" in groups
+
+    def test_plain_string_matrix_name(self):
+        dp = create_datapackage_from_entries({
+            "custom_matrix": [MatrixEntry(row=1, col=2, amount=1.0)],
+        })
+        assert "custom_matrix-data" in dp.groups
+
+    def test_data_values(self):
+        entries = [
+            MatrixEntry(row=1, col=4, amount=2.0),
+            MatrixEntry(row=2, col=5, amount=7.0),
+            MatrixEntry(row=3, col=6, amount=12.0),
+        ]
+        dp = create_datapackage_from_entries({MatrixName.technosphere: entries})
+        group = dp.groups["technosphere_matrix-data"]
+
+        data_resource = next(r for r in group.resources if r["kind"] == "data")
+        data_idx = dp.resources.index(data_resource)
+        data = dp.data[data_idx]
+        assert set(data) == {2.0, 7.0, 12.0}
+
+    def test_flip_stored(self):
+        entries = [
+            MatrixEntry(row=1, col=4, amount=2.0, flip=True),
+            MatrixEntry(row=2, col=5, amount=7.0, flip=False),
+        ]
+        dp = create_datapackage_from_entries({MatrixName.technosphere: entries})
+        group = dp.groups["technosphere_matrix-data"]
+
+        flip_resource = next(r for r in group.resources if r["kind"] == "flip")
+        flip_idx = dp.resources.index(flip_resource)
+        flip = dp.data[flip_idx]
+        assert flip.sum() == 1
+
+    def test_uncertainty_stored(self):
+        entries = [
+            MatrixEntry(row=1, col=4, amount=2.0, uncertainty_type=2, loc=0.7, scale=0.1),
+            MatrixEntry(row=2, col=5, amount=7.0),
+        ]
+        dp = create_datapackage_from_entries({MatrixName.technosphere: entries})
+        group = dp.groups["technosphere_matrix-data"]
+
+        dist_resource = next(r for r in group.resources if r["kind"] == "distributions")
+        dist_idx = dp.resources.index(dist_resource)
+        dist = dp.data[dist_idx]
+        assert dist.dtype == np.dtype(UNCERTAINTY_DTYPE)
+        uncertainty_types = set(dist["uncertainty_type"])
+        assert 2 in uncertainty_types
+
+    def test_metadata_passed_through(self):
+        dp = create_datapackage_from_entries(
+            {MatrixName.technosphere: [MatrixEntry(row=1, col=2, amount=1.0)]},
+            name="my-package",
+        )
+        assert dp.metadata["name"] == "my-package"
+
+class TestSimpleGraphDeprecation:
+    def test_deprecation_warning(self):
+        with warnings.catch_warnings(record=True) as w:
+            warnings.simplefilter("always")
+            simple_graph({"technosphere": [(1, 4, 2.5)]})
+        assert len(w) == 1
+        assert issubclass(w[0].category, DeprecationWarning)
+        assert "create_datapackage_from_entries" in str(w[0].message)

bw_processing-1.0/MANIFEST.in

@@ -1,3 +0,0 @@
-include bw_processing/examples/*.zip
-include bw_processing/examples/datapackage_1/*.parquet
-recursive-exclude dev *

bw_processing-1.0/bw_processing/__init__.py

@@ -1,48 +0,0 @@
-__all__ = (
-    "__version__",
-    "as_unique_attributes",
-    "as_unique_attributes_dataframe",
-    "clean_datapackage_name",
-    "create_array",
-    "create_datapackage",
-    "create_structured_array",
-    "Datapackage",
-    "DatapackageBase",
-    "DEFAULT_LICENSES",
-    "examples_dir",
-    "FilteredDatapackage",
-    "generic_directory_filesystem",
-    "generic_zipfile_filesystem",
-    "INDICES_DTYPE",
-    "load_datapackage",
-    "MatrixSerializeFormat",
-    "md5",
-    "merge_datapackages_with_mask",
-    "reindex",
-    "reset_index",
-    "safe_filename",
-    "simple_graph",
-    "UNCERTAINTY_DTYPE",
-    "UndefinedInterface",
-)
-
-__version__ = "1.0"
-
-
-from .array_creation import create_array, create_structured_array
-from .constants import DEFAULT_LICENSES, INDICES_DTYPE, UNCERTAINTY_DTYPE, MatrixSerializeFormat
-from .datapackage import (
-    Datapackage,
-    DatapackageBase,
-    FilteredDatapackage,
-    create_datapackage,
-    load_datapackage,
-    simple_graph,
-)
-from .examples import examples_dir
-from .filesystem import clean_datapackage_name, md5, safe_filename
-from .indexing import reindex, reset_index
-from .io_helpers import generic_directory_filesystem, generic_zipfile_filesystem
-from .merging import merge_datapackages_with_mask
-from .proxies import UndefinedInterface
-from .unique_fields import as_unique_attributes, as_unique_attributes_dataframe

bw_processing-1.0/bw_processing.egg-info/SOURCES.txt

@@ -1,50 +0,0 @@
-LICENSE
-MANIFEST.in
-README.md
-pyproject.toml
-bw_processing/__init__.py
-bw_processing/array_creation.py
-bw_processing/constants.py
-bw_processing/datapackage.py
-bw_processing/errors.py
-bw_processing/filesystem.py
-bw_processing/indexing.py
-bw_processing/io_helpers.py
-bw_processing/io_parquet_helpers.py
-bw_processing/io_pyarrow_helpers.py
-bw_processing/merging.py
-bw_processing/proxies.py
-bw_processing/unique_fields.py
-bw_processing/utils.py
-bw_processing.egg-info/PKG-INFO
-bw_processing.egg-info/SOURCES.txt
-bw_processing.egg-info/dependency_links.txt
-bw_processing.egg-info/requires.txt
-bw_processing.egg-info/top_level.txt
-bw_processing/examples/__init__.py
-bw_processing/examples/datapackage_2.zip
-bw_processing/examples/interfaces.py
-bw_processing/examples/parquet_files.py
-bw_processing/examples/simple.zip
-bw_processing/examples/datapackage_1/another name.indices.parquet
-bw_processing/examples/datapackage_1/sa-data-vector-from-dict.data.parquet
-bw_processing/examples/datapackage_1/sa-data-vector-from-dict.distributions.parquet
-bw_processing/examples/datapackage_1/sa-data-vector-from-dict.flip.parquet
-bw_processing/examples/datapackage_1/sa-data-vector-from-dict.indices.parquet
-bw_processing/examples/datapackage_1/some name.data.parquet
-bw_processing/examples/datapackage_1/some name.flip.parquet
-bw_processing/examples/datapackage_1/some name.indices.parquet
-tests/test_array_creation.py
-tests/test_datapackage.py
-tests/test_filesystem.py
-tests/test_filtered_datapackage.py
-tests/test_indexing.py
-tests/test_integration.py
-tests/test_interfaces.py
-tests/test_io_parquet_helpers.py
-tests/test_io_pyarrow_helpers.py
-tests/test_loading.py
-tests/test_merging.py
-tests/test_proxies.py
-tests/test_unique_fields.py
-tests/test_utils.py