matrix-utils 0.7.1__tar.gz → 0.8__tar.gz

{matrix_utils-0.7.1 → matrix_utils-0.8}/CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## [0.8] - 2026-06-04
+
+* Merge [#42](https://github.com/brightway-lca/matrix_utils/pull/42): Rename `scale_array` to `rescale_array`, tracking [bw_processing #104](https://github.com/brightway-lca/bw_processing/pull/104). Adds `ResourceGroup.rescale_current` and `MappedMatrix.input_rescale_vector()`.
+* Merge [#41](https://github.com/brightway-lca/matrix_utils/pull/41): Add `params_array` support from [bw_processing #93](https://github.com/brightway-lca/bw_processing/pull/93). New `ResourceGroup` properties `has_params`, `params_current`, `has_param_labels`, and `param_labels`; new `MappedMatrix.input_params()` method.
+* Merge [#40](https://github.com/brightway-lca/matrix_utils/pull/40): Fix `input_uncertainties` crashing when an array group has no flip array (closes [#8](https://github.com/brightway-lca/matrix_utils/issues/8))
+* Merge [#38](https://github.com/brightway-lca/matrix_utils/pull/38): `NaN` values in data vectors/arrays are skipped during matrix insertion, preserving earlier package values
+* Merge [#37](https://github.com/brightway-lca/matrix_utils/pull/37): Add data access convenience methods (`input_raw_indices`, `input_flip_vector`, `input_provenance`, `n_elements_dropped`, `group`)
+* Merge [#36](https://github.com/brightway-lca/matrix_utils/pull/36): Improve access to indexer values (`indexers`, `local_indexers`, `indexers_by_type`, `indexers_are_unique`)
+
 ### [0.7.1] - 2026-03-20
 
 * Fix all elements being masked return sparse matrix instead of array when indexing

{matrix_utils-0.7.1 → matrix_utils-0.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: matrix_utils
-Version: 0.7.1
+Version: 0.8
 Summary: Tools to create matrices from data packages
 Author-email: Chris Mutel <cmutel@gmail.com>
 Maintainer-email: Chris Mutel <cmutel@gmail.com>
@@ -28,7 +28,7 @@ License-File: LICENSE
 Requires-Dist: numpy
 Requires-Dist: scipy>=1.10
 Requires-Dist: pandas
-Requires-Dist: bw_processing>=1.0
+Requires-Dist: bw_processing>=1.5
 Requires-Dist: stats_arrays
 Provides-Extra: testing
 Requires-Dist: matrix_utils; extra == "testing"
@@ -106,21 +106,30 @@ Out[3]:
 
 `MappedMatrix` is iterable; calling `next()` will draw new samples from all included stochastic resources, and rebuild the matrix.
 
+#### Indexers
+
+Each datapackage is assigned exactly one indexer, shared across all its resource groups. This indexer determines which column of an array resource is used on each iteration. The design assumption is one indexer per datapackage — `MappedMatrix.indexers` returns a `{name: indexer}` dict at that level. `MappedMatrix.local_indexers` returns a `{group_label: indexer}` dict for what each resource group is actually using, which will normally be the same object (or a `Proxy` wrapping it for combinatorial packages). You may replace `group.indexer` on any individual group after construction, but you are then responsible for the consequences — in particular, `next()` only advances the package-level indexers, so any custom group-level indexer must be advanced manually.
+
+#### NaN as a sentinel value
+
+A `NaN` value in a data vector or array is treated as **"no data insertion"** — that element is skipped when the matrix is built or rebuilt, leaving the matrix cell at its current value. This is useful for scenario or override packages: place `NaN` at positions you want to inherit from an earlier package, and a real number where you want to override. Non-NaN elements behave normally. `ResourceGroup.data_current` still carries the raw `NaN` so `input_data_vector()` reflects the actual datapackage contents.
+
 You may also find it useful to iterate through `MappedMatrix.groups`, which are instances of `ResourceGroup`, documented below.
 
 ### `ResourceGroup` class
 
-A `bw_processing` data package is essentially a metadata file and a bag of data resources. These resources are *grouped*, for multiple resources are needed to build one matrix, or one component of one matrix. For example, one needs not only the data vector, but also the row and column indices to build a simple matrix. One could also have a `flip` vector, in another file, used to flip the signs of data elements before matrix insertion.
+A `bw_processing` data package is essentially a metadata file and a bag of data resources. These resources are *grouped*, for multiple resources are needed to build one matrix, or one component of one matrix. For example, one needs not only the data vector, but also the row and column indices to build a simple matrix. One could also have a `flip` vector (to negate signs before insertion) or a `scale` vector (to apply a per-exchange multiplicative factor before insertion, e.g. for allocation or unit conversion).
 
 The `ResourceGroup` class provides a single interface to these data files and their metadata. `ResourceGroup` instances are created automatically by `MappedMatrix`, and available via `MappedMatrix.groups`. The [source code](https://github.com/brightway-lca/matrix_utils/) is pretty readable, and in general you probably don't need to worry about this low-level class, but the following could be useful:
 
 * `ResourceGroup.data_original`: The data as it is present in the datapackage, before masking (i.e. the Numpy data vector or array, or the data interface). This is the raw input data, duplicate elements are not aggregated (if applicable).
-* `ResourceGroup.data_current`: The data sample used (before aggregation) to build the matrix. It is both masked and flipped.
+* `ResourceGroup.data_current`: The data sample used (before aggregation) to build the matrix. It is masked, flipped (if a flip array is present), and scaled (if a scale array is present).
+* `ResourceGroup.scale`: The scale array with all masks applied. Raises `KeyError` if no scale array is present.
 * `ResourceGroup.row|col_mapped`: Mapped row and column indices. Has the same length as the datapackage resource, but uses `-1` for values which weren't mapped.
 * `ResourceGroup.row|col_masked`: The data after the custom filter and mapping mask have been applied.
-* `rResourceGroup.row|col_matrix`: Row and column indices (but not data) for insertion into the matrix. These indices are after aggregation within the resource group (if any).
+* `ResourceGroup.row|col_matrix`: Row and column indices (but not data) for insertion into the matrix. These indices are after aggregation within the resource group (if any).
 * `ResourceGroup.calculate(vector=None)`: Function to recalculate matrix row, column, and data vectors. Uses the current state of the indexers, but re-draws values from data iterators. If `vector` is given, use this instead of the given data source.
-* `ResourceGroup.indexer`: The instance of the `Indexer` class applicable for this `ResourceGroup`. Only used for data arrays.
+* `ResourceGroup.indexer`: The instance of the `Indexer` class applicable for this `ResourceGroup`. Only used for data arrays. By design each datapackage has one indexer shared across all its resource groups; this is set up automatically by `MappedMatrix`. You may assign a different indexer to individual groups after construction, but you are then responsible for keeping them consistent (e.g. ensuring they advance together when `next()` is called).
 * `ResourceGroup.ncols`: The integer number of columns in a data array. Returns `None` if a data vector is present.
 
 ## Contributing

{matrix_utils-0.7.1 → matrix_utils-0.8}/README.md

@@ -59,21 +59,30 @@ Out[3]:
 
 `MappedMatrix` is iterable; calling `next()` will draw new samples from all included stochastic resources, and rebuild the matrix.
 
+#### Indexers
+
+Each datapackage is assigned exactly one indexer, shared across all its resource groups. This indexer determines which column of an array resource is used on each iteration. The design assumption is one indexer per datapackage — `MappedMatrix.indexers` returns a `{name: indexer}` dict at that level. `MappedMatrix.local_indexers` returns a `{group_label: indexer}` dict for what each resource group is actually using, which will normally be the same object (or a `Proxy` wrapping it for combinatorial packages). You may replace `group.indexer` on any individual group after construction, but you are then responsible for the consequences — in particular, `next()` only advances the package-level indexers, so any custom group-level indexer must be advanced manually.
+
+#### NaN as a sentinel value
+
+A `NaN` value in a data vector or array is treated as **"no data insertion"** — that element is skipped when the matrix is built or rebuilt, leaving the matrix cell at its current value. This is useful for scenario or override packages: place `NaN` at positions you want to inherit from an earlier package, and a real number where you want to override. Non-NaN elements behave normally. `ResourceGroup.data_current` still carries the raw `NaN` so `input_data_vector()` reflects the actual datapackage contents.
+
 You may also find it useful to iterate through `MappedMatrix.groups`, which are instances of `ResourceGroup`, documented below.
 
 ### `ResourceGroup` class
 
-A `bw_processing` data package is essentially a metadata file and a bag of data resources. These resources are *grouped*, for multiple resources are needed to build one matrix, or one component of one matrix. For example, one needs not only the data vector, but also the row and column indices to build a simple matrix. One could also have a `flip` vector, in another file, used to flip the signs of data elements before matrix insertion.
+A `bw_processing` data package is essentially a metadata file and a bag of data resources. These resources are *grouped*, for multiple resources are needed to build one matrix, or one component of one matrix. For example, one needs not only the data vector, but also the row and column indices to build a simple matrix. One could also have a `flip` vector (to negate signs before insertion) or a `scale` vector (to apply a per-exchange multiplicative factor before insertion, e.g. for allocation or unit conversion).
 
 The `ResourceGroup` class provides a single interface to these data files and their metadata. `ResourceGroup` instances are created automatically by `MappedMatrix`, and available via `MappedMatrix.groups`. The [source code](https://github.com/brightway-lca/matrix_utils/) is pretty readable, and in general you probably don't need to worry about this low-level class, but the following could be useful:
 
 * `ResourceGroup.data_original`: The data as it is present in the datapackage, before masking (i.e. the Numpy data vector or array, or the data interface). This is the raw input data, duplicate elements are not aggregated (if applicable).
-* `ResourceGroup.data_current`: The data sample used (before aggregation) to build the matrix. It is both masked and flipped.
+* `ResourceGroup.data_current`: The data sample used (before aggregation) to build the matrix. It is masked, flipped (if a flip array is present), and scaled (if a scale array is present).
+* `ResourceGroup.scale`: The scale array with all masks applied. Raises `KeyError` if no scale array is present.
 * `ResourceGroup.row|col_mapped`: Mapped row and column indices. Has the same length as the datapackage resource, but uses `-1` for values which weren't mapped.
 * `ResourceGroup.row|col_masked`: The data after the custom filter and mapping mask have been applied.
-* `rResourceGroup.row|col_matrix`: Row and column indices (but not data) for insertion into the matrix. These indices are after aggregation within the resource group (if any).
+* `ResourceGroup.row|col_matrix`: Row and column indices (but not data) for insertion into the matrix. These indices are after aggregation within the resource group (if any).
 * `ResourceGroup.calculate(vector=None)`: Function to recalculate matrix row, column, and data vectors. Uses the current state of the indexers, but re-draws values from data iterators. If `vector` is given, use this instead of the given data source.
-* `ResourceGroup.indexer`: The instance of the `Indexer` class applicable for this `ResourceGroup`. Only used for data arrays.
+* `ResourceGroup.indexer`: The instance of the `Indexer` class applicable for this `ResourceGroup`. Only used for data arrays. By design each datapackage has one indexer shared across all its resource groups; this is set up automatically by `MappedMatrix`. You may assign a different indexer to individual groups after construction, but you are then responsible for keeping them consistent (e.g. ensuring they advance together when `next()` is called).
 * `ResourceGroup.ncols`: The integer number of columns in a data array. Returns `None` if a data vector is present.
 
 ## Contributing

{matrix_utils-0.7.1 → matrix_utils-0.8}/matrix_utils/__init__.py

@@ -11,7 +11,7 @@ __all__ = (
     "SparseMatrixDict",
 )
 
-__version__ = "0.7.1"
+__version__ = "0.8"
 
 from matrix_utils.array_mapper import ArrayMapper
 from matrix_utils.indexers import CombinatorialIndexer, Proxy, RandomIndexer, SequentialIndexer

{matrix_utils-0.7.1 → matrix_utils-0.8}/matrix_utils/mapped_matrix.py

@@ -219,6 +219,10 @@ class MappedMatrix:
         self.matrix.data *= 0
         for group in self.groups:
             row, col, data = group.calculate()
+            # NaN means "no data insertion" — skip those elements so earlier
+            # packages' values are preserved rather than overwritten with NaN.
+            nan_mask = ~np.isnan(data)
+            row, col, data = row[nan_mask], col[nan_mask], data[nan_mask]
             if group.package.metadata["sum_inter_duplicates"]:
                 self.matrix[row, col] += data
             else:
@@ -256,6 +260,60 @@ class MappedMatrix:
                 for obj in resources:
                     obj.add_indexer(indexer=package.indexer)
 
+    @property
+    def indexers(self) -> dict:
+        """Return package-level indexers keyed by datapackage name.
+
+        By design each datapackage has one indexer shared across all its resource groups,
+        but this constraint is not enforced — individual groups can be given different
+        indexers after construction. Use ``local_indexers`` to inspect what each group
+        is actually using.
+        """
+        return {
+            package.metadata["name"]: package.indexer
+            for package in self.packages
+            if hasattr(package, "indexer")
+        }
+
+    @property
+    def local_indexers(self) -> dict:
+        """Return the indexer actually used by each resource group, keyed by group label.
+
+        Normally all groups within a package share the same indexer (or a ``Proxy``
+        wrapping it for combinatorial packages), but users can assign a different indexer
+        to any group after construction, so this may differ from ``indexers``.
+        """
+        return {group.label: group.indexer for group in self.groups if hasattr(group, "indexer")}
+
+    def indexers_by_type(self, indexer_type: type) -> list:
+        """Return all package-level indexers that are instances of ``indexer_type``."""
+        return [indexer for indexer in self.indexers.values() if isinstance(indexer, indexer_type)]
+
+    @property
+    def indexers_are_unique(self) -> bool:
+        """True if no two packages share the same indexer instance."""
+        indexers = list(self.indexers.values())
+        return len({id(i) for i in indexers}) == len(indexers)
+
+    def group(self, label: str) -> "ResourceGroup":
+        """Return the ResourceGroup with the given label.
+
+        Raises ``KeyError`` if no group with that label exists.
+        """
+        for grp in self.groups:
+            if grp.label == label:
+                return grp
+        raise KeyError(label)
+
+    @property
+    def n_elements_dropped(self) -> int:
+        """Total number of datapackage elements dropped across all resource groups.
+
+        An element is dropped when it is excluded by the custom filter or when its
+        row/column id could not be mapped to a matrix index.
+        """
+        return sum(grp.n_elements_dropped for grp in self.groups)
+
     def input_data_vector(self) -> np.ndarray:
         return np.hstack([group.data_current for group in self.groups])
 
@@ -273,6 +331,52 @@ class MappedMatrix:
         array["col"] = cols
         return array
 
+    def input_raw_indices(self) -> np.ndarray:
+        """Return original datapackage indices (before mapping), with masks applied.
+
+        The result has the same length and order as ``input_data_vector`` and
+        ``input_row_col_indices``, but contains the raw source ids from the datapackage
+        rather than the translated matrix row/column positions. Useful for tracing which
+        original ids contributed to each matrix element.
+        """
+        arrays = []
+        for grp in self.groups:
+            if grp.empty:
+                arrays.append(np.array([], dtype=INDICES_DTYPE))
+            else:
+                arrays.append(grp.apply_masks(grp.get_indices_data()))
+        return np.concatenate(arrays)
+
+    def input_rescale_vector(self) -> np.ndarray:
+        """Return rescale factors for each element, aligned with ``input_data_vector``.
+
+        Groups without a rescale array contribute all-ones values.
+        """
+        arrays = []
+        for grp in self.groups:
+            if grp.empty:
+                arrays.append(np.array([]))
+            else:
+                rc = grp.rescale_current
+                arrays.append(rc if rc is not None else np.ones(len(grp.data_current)))
+        return np.hstack(arrays)
+
+    def input_flip_vector(self) -> np.ndarray:
+        """Return a boolean array indicating which elements were negated before matrix insertion.
+
+        Groups without a flip array contribute all-``False`` values. The result aligns
+        element-wise with ``input_data_vector`` and ``input_row_col_indices``.
+        """
+        arrays = []
+        for grp in self.groups:
+            if grp.empty:
+                arrays.append(np.array([], dtype=bool))
+            elif grp.has_flip:
+                arrays.append(grp.flip)
+            else:
+                arrays.append(np.zeros(len(grp.data_current), dtype=bool))
+        return np.hstack(arrays)
+
     def input_provenance(self) -> List[tuple]:
         """Describe where the data in the other ``input_X`` comes from. Returns a list
         of ``(datapackage, group_label, (start_index, end_index))`` tuples.
@@ -293,6 +397,29 @@ class MappedMatrix:
                 position += num_elements
         return result
 
+    def input_params(self) -> dict:
+        """Return current parameter values keyed by ``(package, group_label)``.
+
+        Only groups that carry a params array are included; groups without
+        params are omitted entirely (not present vs. ``None``-valued).
+        Keying by ``(package, group_label)`` avoids silent collisions when
+        two packages contribute groups with the same label.
+
+        For vector groups the full 1-D params array is returned (parameters
+        are fixed and do not vary by iteration). For array groups the column
+        matching the current indexer position is returned, so the result
+        stays in sync with ``input_data_vector`` across Monte Carlo
+        iterations.
+        """
+        result = {}
+        for package, groups in self.packages.items():
+            for group in groups:
+                try:
+                    result[(package, group.label)] = group.params_current
+                except KeyError:
+                    pass
+        return result
+
     def input_indexer_vector(self) -> np.ndarray:
         index_values = []
 
@@ -363,7 +490,8 @@ class MappedMatrix:
 
                 array = self._construct_distributions_array(group.data_current, uncertainty_type=98)
                 array["loc"] = np.mean(data, axis=1)
-                array["loc"][group.flip] *= -1
+                if group.has_flip:
+                    array["loc"][group.flip] *= -1
                 array["scale"] = np.std(data, axis=1)
                 arrays.append(array)
             elif group.is_interface():

{matrix_utils-0.7.1 → matrix_utils-0.8}/matrix_utils/resource_group.py

@@ -119,6 +119,68 @@ class ResourceGroup:
         except KeyError:
             return False
 
+    @property
+    def has_flip(self) -> bool:
+        try:
+            self.get_resource_by_suffix("flip")
+            return True
+        except KeyError:
+            return False
+
+    @property
+    def has_rescale(self) -> bool:
+        try:
+            self.get_resource_by_suffix("rescale")
+            return True
+        except KeyError:
+            return False
+
+    @property
+    def has_params(self) -> bool:
+        try:
+            self.get_resource_by_suffix("params")
+            return True
+        except KeyError:
+            return False
+
+    @property
+    def has_param_labels(self) -> bool:
+        try:
+            self.get_resource_by_suffix("param_labels")
+            return True
+        except KeyError:
+            return False
+
+    @property
+    def params_current(self) -> np.ndarray:
+        """Current parameter values.
+
+        For vectors: returns the full 1-D params array (parameters are fixed
+        and do not vary by indexer position).
+        For arrays: returns the column at the current indexer position,
+        mirroring how ``calculate()`` picks the active data column.
+
+        Raises ``KeyError`` if no params array exists for this group.
+        """
+        data = self.get_resource_by_suffix("params")
+        if self.vector:
+            return data
+        return data[:, self.indexer.index % data.shape[1]]
+
+    @property
+    def param_labels(self) -> dict:
+        """Raw param-labels dict. Always has ``"values"``;
+        also has ``"schema"`` when a label schema was provided at write time.
+
+        Raises ``KeyError`` if no param_labels resource exists.
+        """
+        return self.get_resource_by_suffix("param_labels")
+
+    @property
+    def n_elements_dropped(self) -> int:
+        """Number of original datapackage elements dropped by the custom filter and mapping mask."""
+        return len(self.get_indices_data()) - len(self.row_masked)
+
     @property
     def data_original(self):
         if self.use_distributions and self.has_distributions:
@@ -131,6 +193,18 @@ class ResourceGroup:
         """The flip array, with all masks applied (if necessary)."""
         return self.apply_masks(self.get_resource_by_suffix("flip"))
 
+    @property
+    def rescale(self):
+        """The rescale array, with all masks applied (if necessary)."""
+        return self.apply_masks(self.get_resource_by_suffix("rescale"))
+
+    @property
+    def rescale_current(self) -> Optional[np.ndarray]:
+        """Current rescale values with masks applied, or ``None`` if no rescale array exists."""
+        if self.has_rescale:
+            return self.rescale
+        return None
+
     def get_indices_data(self):
         """The source data for the indices array."""
         indices = self.get_resource_by_suffix("indices")
@@ -299,6 +373,11 @@ class ResourceGroup:
             # No flip array
             pass
 
+        try:
+            data *= self.rescale
+        except KeyError:
+            pass
+
         # Second copy because we want to store the data before aggregation
         self.data_current = data.copy()
 

{matrix_utils-0.7.1 → matrix_utils-0.8}/matrix_utils.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: matrix_utils
-Version: 0.7.1
+Version: 0.8
 Summary: Tools to create matrices from data packages
 Author-email: Chris Mutel <cmutel@gmail.com>
 Maintainer-email: Chris Mutel <cmutel@gmail.com>
@@ -28,7 +28,7 @@ License-File: LICENSE
 Requires-Dist: numpy
 Requires-Dist: scipy>=1.10
 Requires-Dist: pandas
-Requires-Dist: bw_processing>=1.0
+Requires-Dist: bw_processing>=1.5
 Requires-Dist: stats_arrays
 Provides-Extra: testing
 Requires-Dist: matrix_utils; extra == "testing"
@@ -106,21 +106,30 @@ Out[3]:
 
 `MappedMatrix` is iterable; calling `next()` will draw new samples from all included stochastic resources, and rebuild the matrix.
 
+#### Indexers
+
+Each datapackage is assigned exactly one indexer, shared across all its resource groups. This indexer determines which column of an array resource is used on each iteration. The design assumption is one indexer per datapackage — `MappedMatrix.indexers` returns a `{name: indexer}` dict at that level. `MappedMatrix.local_indexers` returns a `{group_label: indexer}` dict for what each resource group is actually using, which will normally be the same object (or a `Proxy` wrapping it for combinatorial packages). You may replace `group.indexer` on any individual group after construction, but you are then responsible for the consequences — in particular, `next()` only advances the package-level indexers, so any custom group-level indexer must be advanced manually.
+
+#### NaN as a sentinel value
+
+A `NaN` value in a data vector or array is treated as **"no data insertion"** — that element is skipped when the matrix is built or rebuilt, leaving the matrix cell at its current value. This is useful for scenario or override packages: place `NaN` at positions you want to inherit from an earlier package, and a real number where you want to override. Non-NaN elements behave normally. `ResourceGroup.data_current` still carries the raw `NaN` so `input_data_vector()` reflects the actual datapackage contents.
+
 You may also find it useful to iterate through `MappedMatrix.groups`, which are instances of `ResourceGroup`, documented below.
 
 ### `ResourceGroup` class
 
-A `bw_processing` data package is essentially a metadata file and a bag of data resources. These resources are *grouped*, for multiple resources are needed to build one matrix, or one component of one matrix. For example, one needs not only the data vector, but also the row and column indices to build a simple matrix. One could also have a `flip` vector, in another file, used to flip the signs of data elements before matrix insertion.
+A `bw_processing` data package is essentially a metadata file and a bag of data resources. These resources are *grouped*, for multiple resources are needed to build one matrix, or one component of one matrix. For example, one needs not only the data vector, but also the row and column indices to build a simple matrix. One could also have a `flip` vector (to negate signs before insertion) or a `scale` vector (to apply a per-exchange multiplicative factor before insertion, e.g. for allocation or unit conversion).
 
 The `ResourceGroup` class provides a single interface to these data files and their metadata. `ResourceGroup` instances are created automatically by `MappedMatrix`, and available via `MappedMatrix.groups`. The [source code](https://github.com/brightway-lca/matrix_utils/) is pretty readable, and in general you probably don't need to worry about this low-level class, but the following could be useful:
 
 * `ResourceGroup.data_original`: The data as it is present in the datapackage, before masking (i.e. the Numpy data vector or array, or the data interface). This is the raw input data, duplicate elements are not aggregated (if applicable).
-* `ResourceGroup.data_current`: The data sample used (before aggregation) to build the matrix. It is both masked and flipped.
+* `ResourceGroup.data_current`: The data sample used (before aggregation) to build the matrix. It is masked, flipped (if a flip array is present), and scaled (if a scale array is present).
+* `ResourceGroup.scale`: The scale array with all masks applied. Raises `KeyError` if no scale array is present.
 * `ResourceGroup.row|col_mapped`: Mapped row and column indices. Has the same length as the datapackage resource, but uses `-1` for values which weren't mapped.
 * `ResourceGroup.row|col_masked`: The data after the custom filter and mapping mask have been applied.
-* `rResourceGroup.row|col_matrix`: Row and column indices (but not data) for insertion into the matrix. These indices are after aggregation within the resource group (if any).
+* `ResourceGroup.row|col_matrix`: Row and column indices (but not data) for insertion into the matrix. These indices are after aggregation within the resource group (if any).
 * `ResourceGroup.calculate(vector=None)`: Function to recalculate matrix row, column, and data vectors. Uses the current state of the indexers, but re-draws values from data iterators. If `vector` is given, use this instead of the given data source.
-* `ResourceGroup.indexer`: The instance of the `Indexer` class applicable for this `ResourceGroup`. Only used for data arrays.
+* `ResourceGroup.indexer`: The instance of the `Indexer` class applicable for this `ResourceGroup`. Only used for data arrays. By design each datapackage has one indexer shared across all its resource groups; this is set up automatically by `MappedMatrix`. You may assign a different indexer to individual groups after construction, but you are then responsible for keeping them consistent (e.g. ensuring they advance together when `next()` is called).
 * `ResourceGroup.ncols`: The integer number of columns in a data array. Returns `None` if a data vector is present.
 
 ## Contributing

{matrix_utils-0.7.1 → matrix_utils-0.8}/matrix_utils.egg-info/requires.txt

@@ -1,7 +1,7 @@
 numpy
 scipy>=1.10
 pandas
-bw_processing>=1.0
+bw_processing>=1.5
 stats_arrays
 
 [dev]

{matrix_utils-0.7.1 → matrix_utils-0.8}/pyproject.toml

@@ -39,7 +39,7 @@ dependencies = [
     "numpy",
     "scipy>=1.10",
     "pandas",
-    "bw_processing>=1.0",
+    "bw_processing>=1.5",
     "stats_arrays",
 ]
 

{matrix_utils-0.7.1 → matrix_utils-0.8}/tests/mapped_matrix.py

@@ -9,6 +9,108 @@ from fsspec.implementations.zip import ZipFileSystem
 from matrix_utils import ArrayMapper, MappedMatrix
 from matrix_utils.errors import AllArraysEmpty, EmptyArray
 
+# --- input_params ---
+
+
+def _mm_with_params(**kwargs):
+    """Build a MappedMatrix from the given datapackages."""
+    return MappedMatrix(
+        matrix="foo",
+        use_vectors=True,
+        use_arrays=True,
+        use_distributions=False,
+        **kwargs,
+    )
+
+
+def test_input_params_empty_when_no_params():
+    mm = _mm_with_params(packages=[basic_mm()])
+    assert mm.input_params() == {}
+
+
+def test_input_params_vector():
+    dp = bwp.create_datapackage()
+    params = np.array([0.1, 0.9])
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([5.0, 7.0]),
+        params_array=params,
+    )
+    mm = _mm_with_params(packages=[dp])
+    result = mm.input_params()
+    pkg = list(mm.packages.keys())[0]
+    assert list(result.keys()) == [(pkg, "v")]
+    assert np.allclose(result[(pkg, "v")], params)
+
+
+def test_input_params_only_groups_with_params_included():
+    dp = bwp.create_datapackage()
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="with_params",
+        indices_array=np.array([(0, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0]),
+        params_array=np.array([42.0]),
+    )
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="no_params",
+        indices_array=np.array([(1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([2.0]),
+    )
+    mm = _mm_with_params(packages=[dp])
+    result = mm.input_params()
+    labels = {label for (_, label) in result}
+    assert labels == {"with_params"}
+
+
+def test_input_params_array_tracks_indexer():
+    dp = bwp.create_datapackage(sequential=True)
+    data = np.array([[10.0, 11.0, 12.0], [20.0, 21.0, 22.0]])
+    params = np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
+    dp.add_persistent_array(
+        matrix="foo",
+        name="a",
+        indices_array=np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=data,
+        params_array=params,
+    )
+    mm = _mm_with_params(packages=[dp])
+    pkg = list(mm.packages.keys())[0]
+    assert np.allclose(mm.input_params()[(pkg, "a")], [1.0, 4.0])
+    next(mm)
+    assert np.allclose(mm.input_params()[(pkg, "a")], [2.0, 5.0])
+    next(mm)
+    assert np.allclose(mm.input_params()[(pkg, "a")], [3.0, 6.0])
+
+
+def test_input_params_duplicate_labels_across_packages():
+    dp1 = bwp.create_datapackage()
+    dp1.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(0, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0]),
+        params_array=np.array([0.1]),
+    )
+    dp2 = bwp.create_datapackage()
+    dp2.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([2.0]),
+        params_array=np.array([0.9]),
+    )
+    mm = _mm_with_params(packages=[dp1, dp2])
+    result = mm.input_params()
+    assert len(result) == 2
+    values = list(result.values())
+    assert any(np.allclose(v, [0.1]) for v in values)
+    assert any(np.allclose(v, [0.9]) for v in values)
+
+
 dirpath = Path(__file__).parent.resolve() / "fixtures"
 
 
@@ -708,6 +810,29 @@ def test_input_uncertainties_no_distributions(sensitivity_dps):
         assert np.allclose(ua[field], expected[field], equal_nan=True)
 
 
+def test_input_uncertainties_array_group_without_flip():
+    """Regression test for issue #8: array group without flip should not crash."""
+    dp = bwp.create_datapackage()
+    indices_array = np.array([(10, 10), (11, 11)], dtype=bwp.INDICES_DTYPE)
+    data_array = np.array([[7, 8], [8, 9], [9, 10]]).T
+    dp.add_persistent_array(
+        matrix="matrix",
+        data_array=data_array,
+        name="no_flip",
+        indices_array=indices_array,
+    )
+    mm = MappedMatrix(
+        packages=[dp],
+        matrix="matrix",
+        use_vectors=True,
+        use_arrays=True,
+        use_distributions=False,
+    )
+    ua = mm.input_uncertainties()
+    assert ua.shape == (2,)
+    assert np.allclose(ua["loc"], [8.0, 9.0])
+
+
 def test_input_index_vector(sensitivity_dps):
     dp = bwp.create_datapackage(combinatorial=True)
 
@@ -868,3 +993,351 @@ def test_input_indexer_vector_raises_on_unsupported_type():
 
     with pytest.raises(ValueError, match="Can't understand indexer value"):
         mm.input_indexer_vector()
+
+
+def test_indexers_single_package():
+    from matrix_utils.indexers import RandomIndexer
+
+    dp = basic_mm(name="alpha")
+    mm = MappedMatrix(packages=[dp], matrix="foo", use_arrays=False, use_distributions=False)
+    gi = mm.indexers
+    assert list(gi.keys()) == ["alpha"]
+    assert isinstance(gi["alpha"], RandomIndexer)
+
+
+def test_indexers_multiple_packages():
+    from matrix_utils.indexers import RandomIndexer
+
+    dp1 = basic_mm(name="pkg-one")
+    dp2 = basic_mm(name="pkg-two")
+    mm = MappedMatrix(packages=[dp1, dp2], matrix="foo", use_arrays=False, use_distributions=False)
+    gi = mm.indexers
+    assert set(gi.keys()) == {"pkg-one", "pkg-two"}
+    assert isinstance(gi["pkg-one"], RandomIndexer)
+    assert isinstance(gi["pkg-two"], RandomIndexer)
+    # each package gets its own indexer instance
+    assert gi["pkg-one"] is not gi["pkg-two"]
+
+
+def test_local_indexers_keyed_by_group_label():
+    from matrix_utils.indexers import RandomIndexer
+
+    dp = basic_mm(name="alpha")
+    mm = MappedMatrix(packages=[dp], matrix="foo", use_arrays=False, use_distributions=False)
+    li = mm.local_indexers
+    # basic_mm adds two vector groups: "vector" and "vector2"
+    assert set(li.keys()) == {"vector", "vector2"}
+    # both groups share the same package-level indexer instance
+    assert isinstance(li["vector"], RandomIndexer)
+    assert li["vector"] is li["vector2"]
+
+
+def test_local_indexers_combinatorial_are_proxies():
+    from matrix_utils.indexers import CombinatorialIndexer, Proxy
+
+    dp = bwp.create_datapackage(combinatorial=True, name="combo")
+    indices = np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE)
+    dp.add_persistent_array(
+        matrix="foo", name="g", indices_array=indices, data_array=np.array([[1, 2], [3, 4]]).T
+    )
+    dp.add_persistent_array(
+        matrix="foo", name="h", indices_array=indices, data_array=np.array([[5, 6], [7, 8]]).T
+    )
+
+    mm = MappedMatrix(packages=[dp], matrix="foo")
+    li = mm.local_indexers
+    assert set(li.keys()) == {"g", "h"}
+    assert all(isinstance(v, Proxy) for v in li.values())
+    # global indexer is the underlying CombinatorialIndexer
+    assert isinstance(mm.indexers["combo"], CombinatorialIndexer)
+
+
+def test_indexers_by_type():
+    from matrix_utils.indexers import RandomIndexer, SequentialIndexer
+
+    dp_rand = basic_mm(name="rand")
+    dp_seq = bwp.create_datapackage(sequential=True, name="seq")
+    dp_seq.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(0, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0]),
+    )
+    mm = MappedMatrix(
+        packages=[dp_rand, dp_seq], matrix="foo", use_arrays=False, use_distributions=False
+    )
+
+    rand_indexers = mm.indexers_by_type(RandomIndexer)
+    seq_indexers = mm.indexers_by_type(SequentialIndexer)
+    assert len(rand_indexers) == 1
+    assert isinstance(rand_indexers[0], RandomIndexer)
+    assert len(seq_indexers) == 1
+    assert isinstance(seq_indexers[0], SequentialIndexer)
+
+
+def test_indexers_are_unique_true():
+    dp1 = basic_mm(name="a")
+    dp2 = basic_mm(name="b")
+    mm = MappedMatrix(packages=[dp1, dp2], matrix="foo", use_arrays=False, use_distributions=False)
+    assert mm.indexers_are_unique is True
+
+
+def test_indexers_are_unique_false_with_override():
+    from matrix_utils.indexers import SequentialIndexer
+
+    shared = SequentialIndexer()
+    dp1 = basic_mm(name="a")
+    dp2 = basic_mm(name="b")
+    mm = MappedMatrix(
+        packages=[dp1, dp2],
+        matrix="foo",
+        use_arrays=False,
+        use_distributions=False,
+        indexer_override=shared,
+    )
+    assert mm.indexers_are_unique is False
+
+
+# ── group() ───────────────────────────────────────────────────────────────────
+
+
+def test_group_lookup_found():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    grp = mm.group("vector")
+    assert grp.label == "vector"
+
+
+def test_group_lookup_not_found():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    with pytest.raises(KeyError):
+        mm.group("nonexistent")
+
+
+# ── has_flip / has_rescale ────────────────────────────────────────────────────
+
+
+def test_has_flip_true():
+    mm = MappedMatrix(
+        packages=[diagonal()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    assert mm.group("vector").has_flip is True
+
+
+def test_has_flip_false():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    assert mm.group("vector").has_flip is False
+
+
+def test_has_rescale_false():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    assert mm.group("vector").has_rescale is False
+
+
+def test_has_rescale_true():
+    dp = bwp.create_datapackage()
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="sv",
+        indices_array=np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0, 2.0]),
+        rescale_array=np.array([0.5, 2.0]),
+    )
+    mm = MappedMatrix(packages=[dp], matrix="foo", use_arrays=False, use_distributions=False)
+    assert mm.group("sv").has_rescale is True
+
+
+# ── n_elements_dropped ─────────────────────────────────────────────────────────────────
+
+
+def test_n_elements_dropped_none():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    assert mm.n_elements_dropped == 0
+    assert mm.group("vector").n_elements_dropped == 0
+
+
+def test_n_elements_dropped_custom_filter():
+    mm = MappedMatrix(
+        packages=[basic_mm()],
+        matrix="foo",
+        use_arrays=False,
+        use_distributions=False,
+        custom_filter=lambda x: x["row"] < 5,
+    )
+    # basic_mm vector has rows [0, 2, 4, 8]; filter keeps [0, 2, 4], drops 8
+    assert mm.group("vector").n_elements_dropped == 1
+
+
+def test_n_elements_dropped_unmapped():
+    # Build a matrix with a pre-existing row_mapper that excludes some ids
+    dp = bwp.create_datapackage()
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(0, 0), (99, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0, 2.0]),
+    )
+    # row_mapper that only knows about id 0, not 99
+    am = ArrayMapper(array=np.array([0]))
+    mm = MappedMatrix(
+        packages=[dp], matrix="foo", use_arrays=False, use_distributions=False, row_mapper=am
+    )
+    assert mm.group("v").n_elements_dropped == 1
+    assert mm.n_elements_dropped == 1
+
+
+def test_n_elements_dropped_total():
+    dp1 = bwp.create_datapackage()
+    dp1.add_persistent_vector(
+        matrix="foo",
+        name="a",
+        indices_array=np.array([(0, 0), (99, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0, 2.0]),
+    )
+    dp2 = bwp.create_datapackage()
+    dp2.add_persistent_vector(
+        matrix="foo",
+        name="b",
+        indices_array=np.array([(0, 0), (98, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([3.0, 4.0]),
+    )
+    am = ArrayMapper(array=np.array([0]))
+    mm = MappedMatrix(
+        packages=[dp1, dp2], matrix="foo", use_arrays=False, use_distributions=False, row_mapper=am
+    )
+    assert mm.n_elements_dropped == 2
+
+
+# ── input_raw_indices() ───────────────────────────────────────────────────────
+
+
+def test_input_raw_indices_aligns_with_data_vector():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    raw = mm.input_raw_indices()
+    data = mm.input_data_vector()
+    assert len(raw) == len(data)
+
+
+def test_input_raw_indices_contains_original_ids():
+    dp = bwp.create_datapackage()
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(10, 20), (30, 40)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0, 2.0]),
+    )
+    mm = MappedMatrix(packages=[dp], matrix="foo", use_arrays=False, use_distributions=False)
+    raw = mm.input_raw_indices()
+    assert list(raw["row"]) == [10, 30]
+    assert list(raw["col"]) == [20, 40]
+
+
+def test_input_raw_indices_excludes_dropped():
+    dp = bwp.create_datapackage()
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(0, 0), (99, 0)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0, 2.0]),
+    )
+    am = ArrayMapper(array=np.array([0]))
+    mm = MappedMatrix(
+        packages=[dp], matrix="foo", use_arrays=False, use_distributions=False, row_mapper=am
+    )
+    raw = mm.input_raw_indices()
+    # id 99 was unmapped, so only id 0 survives
+    assert len(raw) == 1
+    assert raw["row"][0] == 0
+
+
+# ── input_flip_vector() ───────────────────────────────────────────────────────
+
+
+def test_input_flip_vector_no_flip():
+    mm = MappedMatrix(
+        packages=[basic_mm()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    flip = mm.input_flip_vector()
+    assert flip.dtype == bool
+    assert not flip.any()
+    assert len(flip) == len(mm.input_data_vector())
+
+
+def test_input_flip_vector_with_flip():
+    mm = MappedMatrix(
+        packages=[diagonal()], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    flip = mm.input_flip_vector()
+    # diagonal fixture: flip_array = [False, True, False, False]
+    assert flip.dtype == bool
+    assert flip.sum() == 1
+    assert len(flip) == len(mm.input_data_vector())
+
+
+def test_nan_in_vector_skips_insertion():
+    dp = bwp.create_datapackage()
+    dp.add_persistent_vector(
+        matrix="foo",
+        name="v",
+        indices_array=np.array([(0, 0), (1, 1), (2, 2)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([1.0, np.nan, 3.0]),
+    )
+    mm = MappedMatrix(packages=[dp], matrix="foo", use_arrays=False, use_distributions=False)
+    assert mm.matrix[0, 0] == 1.0
+    assert mm.matrix[1, 1] == 0.0  # NaN skipped, stays zero
+    assert mm.matrix[2, 2] == 3.0
+    assert not np.isnan(mm.matrix.data).any()
+
+
+def test_nan_in_array_skips_insertion():
+    dp = bwp.create_datapackage(sequential=True)
+    dp.add_persistent_array(
+        matrix="foo",
+        name="a",
+        indices_array=np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([[1.0, np.nan], [3.0, 4.0]]),
+    )
+    mm = MappedMatrix(packages=[dp], matrix="foo", use_distributions=False)
+    # column 0: data = [1.0, 3.0] — both inserted
+    assert mm.matrix[0, 0] == 1.0
+    assert mm.matrix[1, 1] == 3.0
+    next(mm)
+    # column 1: data = [nan, 4.0] — first element skipped, stays zero
+    assert mm.matrix[0, 0] == 0.0
+    assert mm.matrix[1, 1] == 4.0
+    assert not np.isnan(mm.matrix.data).any()
+
+
+def test_nan_preserves_earlier_package_value():
+    # Package A sets values; package B uses NaN to leave them untouched.
+    dp_base = bwp.create_datapackage()
+    dp_base.add_persistent_vector(
+        matrix="foo",
+        name="base",
+        indices_array=np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([5.0, 7.0]),
+    )
+    dp_scenario = bwp.create_datapackage()
+    dp_scenario.add_persistent_vector(
+        matrix="foo",
+        name="scenario",
+        indices_array=np.array([(0, 0), (1, 1)], dtype=bwp.INDICES_DTYPE),
+        data_array=np.array([np.nan, 99.0]),
+    )
+    mm = MappedMatrix(
+        packages=[dp_base, dp_scenario], matrix="foo", use_arrays=False, use_distributions=False
+    )
+    assert mm.matrix[0, 0] == 5.0  # NaN in scenario → base value preserved
+    assert mm.matrix[1, 1] == 99.0  # non-NaN in scenario → overrides base
+    assert not np.isnan(mm.matrix.data).any()

{matrix_utils-0.7.1 → matrix_utils-0.8}/tests/monte_carlo.py

@@ -111,8 +111,8 @@ def test_distributions_without_uncertainties():
     row = mm.row_mapper.to_dict()
     col = mm.col_mapper.to_dict()
 
-    assert 50 <= np.mean(results[row[10], col[20], :]) <= 150
-    assert 8 <= np.mean(results[row[11], col[21], :]) <= 14
+    assert 20 <= np.mean(results[row[10], col[20], :]) <= 180
+    assert 5 <= np.mean(results[row[11], col[21], :]) <= 18
 
     assert np.allclose(results[row[10], col[10], :], 11)
     assert np.allclose(results[row[18], col[7], :], 125)