views-frames 1.9.0__tar.gz → 1.10.1__tar.gz

{views_frames-1.9.0 → views_frames-1.10.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: views-frames
-Version: 1.9.0
+Version: 1.10.1
 Summary: The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG).
 Project-URL: Homepage, https://github.com/views-platform/views-frames
 Project-URL: Repository, https://github.com/views-platform/views-frames
@@ -36,7 +36,7 @@ Description-Content-Type: text/markdown
 > containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
 > that every other repo depends on and that depends on nothing internal.
 >
-> **Status:** **v1.9.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
+> **Status:** **v1.10.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
 > v1.1 surface is
 > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
 > `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic — no magnitude
@@ -50,7 +50,10 @@ Description-Content-Type: text/markdown
 > ADR-025 immutability-by-convention, the adversarial red-test batch; v1.9.0 moves the
 > tower-tip MAP to the **top floor** of the published tower — `tip_mass` 0.25 with the
 > **MAP-containment law** in the conformance suite, ADR-019 Amendment 3 — and adds the
-> `research/figures/` tower-figure toolkit). This
+> `research/figures/` tower-figure toolkit; v1.10.0 adds the **dense-grid fill**
+> primitive — `reindex_fill(other, *, fill_value)` on all three frames +
+> `SpatioTemporalIndex.cartesian` + the published `assert_reindex_fill_law`, ADR-026,
+> unblocking pandas-free FAO ingestion). This
 > README is the design
 > bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
 > io, conformance suite) plus the `src/views_frames_summarize/` sibling package
@@ -346,7 +349,12 @@ genuinely reused core. Build this once:
   `(time, unit)` **at a single `SpatialLevel`**. **This is the label-alignment
   that today drags pandas back in** — pred↔actual join, partial-overlap
   evaluation, same-level reindex. This alignment logic lives in the leaf
-  unconditionally.
+  unconditionally. The dense-grid pair rides the same join (ADR-026):
+  `cartesian(times, units, level)` builds the time-major product index
+  (explicit arrays only — deriving them is consumer policy), and the frames'
+  `reindex_fill(other, *, fill_value)` aligns with **no** superset requirement,
+  filling absent rows (present rows bit-exact; law-pinned by
+  `assert_reindex_fill_law`).
 - **Cross-level operations (`cross_level_align`) — protocol here, data injected.**
   The cm↔pgm **cross-level join** (country↔grid) is **not** a same-axis set op; it
   is a one-to-many lookup against a `priogrid→country` mapping that is **injected**

{views_frames-1.9.0 → views_frames-1.10.1}/README.md

@@ -4,7 +4,7 @@
 > containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
 > that every other repo depends on and that depends on nothing internal.
 >
-> **Status:** **v1.9.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
+> **Status:** **v1.10.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
 > v1.1 surface is
 > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
 > `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic — no magnitude
@@ -18,7 +18,10 @@
 > ADR-025 immutability-by-convention, the adversarial red-test batch; v1.9.0 moves the
 > tower-tip MAP to the **top floor** of the published tower — `tip_mass` 0.25 with the
 > **MAP-containment law** in the conformance suite, ADR-019 Amendment 3 — and adds the
-> `research/figures/` tower-figure toolkit). This
+> `research/figures/` tower-figure toolkit; v1.10.0 adds the **dense-grid fill**
+> primitive — `reindex_fill(other, *, fill_value)` on all three frames +
+> `SpatioTemporalIndex.cartesian` + the published `assert_reindex_fill_law`, ADR-026,
+> unblocking pandas-free FAO ingestion). This
 > README is the design
 > bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
 > io, conformance suite) plus the `src/views_frames_summarize/` sibling package
@@ -314,7 +317,12 @@ genuinely reused core. Build this once:
   `(time, unit)` **at a single `SpatialLevel`**. **This is the label-alignment
   that today drags pandas back in** — pred↔actual join, partial-overlap
   evaluation, same-level reindex. This alignment logic lives in the leaf
-  unconditionally.
+  unconditionally. The dense-grid pair rides the same join (ADR-026):
+  `cartesian(times, units, level)` builds the time-major product index
+  (explicit arrays only — deriving them is consumer policy), and the frames'
+  `reindex_fill(other, *, fill_value)` aligns with **no** superset requirement,
+  filling absent rows (present rows bit-exact; law-pinned by
+  `assert_reindex_fill_law`).
 - **Cross-level operations (`cross_level_align`) — protocol here, data injected.**
   The cm↔pgm **cross-level join** (country↔grid) is **not** a same-axis set op; it
   is a one-to-many lookup against a `priogrid→country` mapping that is **injected**

{views_frames-1.9.0 → views_frames-1.10.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "views-frames"
-version = "1.9.0"
+version = "1.10.1"
 description = "The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG)."
 authors = [
     { name = "Simon Polichinel von der Maase", email = "simmaa@prio.org" },

{views_frames-1.9.0 → views_frames-1.10.1}/src/views_frames/conformance/__init__.py

@@ -37,6 +37,7 @@ __all__ = [
     "assert_frame_contract",
     "assert_frame_envelope",
     "assert_index_alignment_laws",
+    "assert_reindex_fill_law",
 ]
 
 
@@ -136,6 +137,50 @@ def assert_index_alignment_laws(index_a: Any, index_b: Any) -> None:
     assert np.array_equal(index_a.unit[pos], index_a.unit), "searchsorted self-identity"
 
 
+def assert_reindex_fill_law(frame: Any, target: Any, fill_value: float) -> None:
+    """Assert the dense-grid fill law (ADR-026) for ``frame`` against ``target``.
+
+    ``reindex_fill`` aligns a frame to a target index with **no** superset
+    requirement, filling absent rows. The law:
+
+    - the result's index equals ``target`` row-for-row (time, unit, level);
+    - every target row present in ``frame`` comes through **bit-exact**;
+    - every absent row equals ``fill_value`` exactly (NaN-safe comparison);
+    - when every target row is present, ``reindex_fill`` degenerates to
+      ``reindex`` (filling a superset frame adds nothing).
+
+    Args:
+        frame: a frame exposing ``reindex_fill``/``reindex``/``index``/``values``.
+        target: a ``SpatioTemporalIndex`` (same level) to densify against.
+        fill_value: the fill for absent rows (``NaN`` is legal).
+
+    Raises:
+        AssertionError: a law is violated.
+    """
+    _require_assertions()
+    filled = frame.reindex_fill(target, fill_value=fill_value)
+    assert filled.index.level == target.level, "reindex_fill must keep the level"
+    assert np.array_equal(filled.index.time, target.time), (
+        "reindex_fill result index must equal the target (time)"
+    )
+    assert np.array_equal(filled.index.unit, target.unit), (
+        "reindex_fill result index must equal the target (unit)"
+    )
+    pos = frame.index.searchsorted(target)
+    found = pos >= 0
+    assert np.array_equal(
+        filled.values[found], frame.values[pos[found]], equal_nan=True
+    ), "reindex_fill must pass present rows through bit-exact"
+    expected = np.full_like(filled.values[~found], np.float32(fill_value))
+    assert np.array_equal(filled.values[~found], expected, equal_nan=True), (
+        "reindex_fill must set every absent row to fill_value"
+    )
+    if bool(found.all()):
+        assert np.array_equal(
+            filled.values, frame.reindex(target).values, equal_nan=True
+        ), "reindex_fill on a superset frame must equal reindex"
+
+
 def assert_cross_level_alignment_law(
     index: Any, mapping: Any, target_level: Any
 ) -> None:

{views_frames-1.9.0 → views_frames-1.10.1}/src/views_frames/feature_frame.py

@@ -154,6 +154,37 @@ class FeatureFrame:
             )
         return self.select(self._index.searchsorted(other))
 
+    def reindex_fill(
+        self, other: SpatioTemporalIndex, *, fill_value: float
+    ) -> FeatureFrame:
+        """Align to ``other``'s rows, filling rows absent from this frame.
+
+        The dense-grid companion to :meth:`reindex` (ADR-026): every row of
+        ``other`` present in this frame comes through **bit-exact**; a row
+        absent from it gets ``fill_value``, broadcast across the feature and
+        sample axes. There is no superset requirement — that is the difference
+        from the fail-loud ``reindex``. The result's index **is** ``other``
+        (immutable, shared); ``feature_names`` and metadata are preserved.
+
+        Same-level join semantics: assumes unique ``(time, unit)`` rows in
+        **this** frame (register C-21); ``other`` may repeat rows. Allocates
+        the full ``(other.n_rows, F, S)`` buffer — densifying a large grid is
+        a deliberate, costly act. A ``NaN`` fill is legal here, but the
+        summarize estimators fail loud on NaN draws by design (ADR-017).
+
+        Raises:
+            ValueError: ``other`` is at a different ``SpatialLevel``.
+        """
+        pos = self._index.searchsorted(other)
+        found = pos >= 0
+        out = np.full(
+            (other.n_rows, *self._values.shape[1:]),
+            np.float32(fill_value),
+            dtype=np.float32,
+        )
+        out[found] = self._values[pos[found]]
+        return FeatureFrame(out, other, self._feature_names, self._metadata)
+
     # ---- persistence --------------------------------------------------------
 
     def save(self, directory: Path | str) -> None:

{views_frames-1.9.0 → views_frames-1.10.1}/src/views_frames/index.py

@@ -43,9 +43,7 @@ class SpatioTemporalIndex:
         level: SpatialLevel,
     ) -> None:
         if not isinstance(level, SpatialLevel):
-            raise TypeError(
-                f"level must be a SpatialLevel, got {type(level).__name__}"
-            )
+            raise TypeError(f"level must be a SpatialLevel, got {type(level).__name__}")
         is_array = isinstance(time, np.ndarray) and time.ndim >= 1
         n = int(time.shape[0]) if is_array else -1
         validate_identifiers({"time": time, "unit": unit}, n_rows=n)
@@ -56,6 +54,65 @@ class SpatioTemporalIndex:
         self._unit.setflags(write=False)
         self._level = level
 
+    # ---- construction -------------------------------------------------------
+
+    @classmethod
+    def cartesian(
+        cls,
+        times: IntArray,
+        units: IntArray,
+        level: SpatialLevel,
+    ) -> SpatioTemporalIndex:
+        """The dense ``(time × unit)`` product index, in time-major order.
+
+        The dense-grid constructor (ADR-026): every combination of the given
+        ``times`` and ``units``, ordered time-major (all units of the first time,
+        then all units of the next), so dense indices are canonical across
+        consumers. The inputs are **explicit arrays only** — any rule for
+        deriving them (e.g. "the units present in the last time step") is
+        consumer policy and never lives in the leaf.
+
+        Fails loud on duplicated ``times`` or ``units``: a duplicated product
+        input manufactures duplicate ``(time, unit)`` rows, which make every
+        same-level join undefined (register C-21) — always a caller bug here.
+
+        Allocates ``len(times) × len(units)`` rows eagerly; densifying a large
+        grid is a deliberate, costly act (a full-pgm dense grid runs to tens of
+        millions of rows).
+
+        Args:
+            times: 1-D integer array of unique time identifiers.
+            units: 1-D integer array of unique unit identifiers.
+            level: The ``SpatialLevel`` of the produced index.
+
+        Raises:
+            TypeError: ``times``/``units`` is not an integer-dtype numpy array,
+                or ``level`` is not a ``SpatialLevel``.
+            ValueError: an input array is not 1-D or contains duplicate values.
+        """
+        for name, arr in (("times", times), ("units", units)):
+            if not isinstance(arr, np.ndarray):
+                raise TypeError(
+                    f"cartesian {name} must be a numpy array, got {type(arr).__name__}"
+                )
+            if not np.issubdtype(arr.dtype, np.integer):
+                raise TypeError(
+                    f"cartesian {name} must be an integer dtype, got {arr.dtype}"
+                )
+            if arr.ndim != 1:
+                raise ValueError(f"cartesian {name} must be 1-D, got ndim={arr.ndim}")
+            if np.unique(arr).shape[0] != arr.shape[0]:
+                raise ValueError(
+                    f"cartesian {name} contains duplicate values; a duplicated "
+                    "product input manufactures duplicate (time, unit) rows, "
+                    "which make same-level joins undefined (register C-21)"
+                )
+        return cls(
+            time=np.repeat(times, units.shape[0]),
+            unit=np.tile(units, times.shape[0]),
+            level=level,
+        )
+
     # ---- core surface -------------------------------------------------------
 
     @property
@@ -109,9 +166,11 @@ class SpatioTemporalIndex:
     @staticmethod
     def _row_view(keys: NDArray[np.int64]) -> NDArray[np.void]:
         """View each ``(time, unit)`` row as a single void scalar for set ops."""
-        return np.ascontiguousarray(keys).view(
-            np.dtype((np.void, keys.dtype.itemsize * keys.shape[1]))
-        ).reshape(-1)
+        return (
+            np.ascontiguousarray(keys)
+            .view(np.dtype((np.void, keys.dtype.itemsize * keys.shape[1])))
+            .reshape(-1)
+        )
 
     def _require_same_level(self, other: SpatioTemporalIndex) -> None:
         if self._level != other._level:
@@ -180,9 +239,7 @@ class SpatioTemporalIndex:
         rows = self._row_view(self._keys())
         return bool(len(np.unique(rows)) == rows.shape[0])
 
-    def select(
-        self, indexer: IntArray | NDArray[np.bool_]
-    ) -> SpatioTemporalIndex:
+    def select(self, indexer: IntArray | NDArray[np.bool_]) -> SpatioTemporalIndex:
         """A new index of the rows at integer positions **or** a boolean mask.
 
         The row-selection primitive the frame-level ``select``/``reindex`` build on:
@@ -298,9 +355,7 @@ class SpatioTemporalIndex:
         sorted_rows = map_rows[order]
 
         self_rows = self._row_view(self._keys())
-        pos = np.clip(
-            np.searchsorted(sorted_rows, self_rows), 0, len(sorted_rows) - 1
-        )
+        pos = np.clip(np.searchsorted(sorted_rows, self_rows), 0, len(sorted_rows) - 1)
         found = sorted_rows[pos] == self_rows
         if not bool(found.all()):
             miss = int(np.argmax(~found))

{views_frames-1.9.0 → views_frames-1.10.1}/src/views_frames/io/arrow.py

@@ -71,7 +71,20 @@ def save(
 
 
 def load(path: Path | str) -> dict[str, Any]:
-    """Read a flat-columnar parquet frame state written by :func:`save`."""
+    """Read a flat-columnar parquet frame state written by :func:`save`.
+
+    The reconstruction is positional (``reshape(n, s)``), so the table's row
+    order **is** the contract: before trusting it, the wire-contract layout is
+    validated and any violation **raises** rather than reshaping plausible
+    floats into the wrong sample slots (ADR-013 §4.5b wire contract; #199).
+
+    Raises:
+        ValueError: the table is truncated/filtered (row count not a multiple
+            of the header's ``n_samples``), the ``sample`` column deviates from
+            the written ``tile(arange(S), N)`` order, or ``time``/``unit`` are
+            not constant within a row's sample block (rows swapped between
+            cells) — a reordered, truncated, or foreign-rewritten table.
+    """
     table = pq.read_table(str(path))
     raw = table.schema.metadata or {}
     header = json.loads(raw[b"views_frames"].decode())
@@ -80,9 +93,39 @@ def load(path: Path | str) -> dict[str, Any]:
 
     time_col = table.column("time").to_numpy()
     unit_col = table.column("unit").to_numpy()
-    n = time_col.shape[0] // s
-    time = time_col.reshape(n, s)[:, 0]
-    unit = unit_col.reshape(n, s)[:, 0]
+    total = int(time_col.shape[0])
+    if s <= 0 or total % s != 0:
+        raise ValueError(
+            f"corrupt frame parquet: {total} rows is not a positive multiple of "
+            f"the header's n_samples={s} — a truncated or filtered table cannot "
+            "be reshaped safely (wire contract, #199)"
+        )
+    n = total // s
+
+    sample_col = table.column("sample").to_numpy()
+    if not np.array_equal(sample_col, np.tile(np.arange(s, dtype=np.int32), n)):
+        raise ValueError(
+            "corrupt frame parquet: the 'sample' column deviates from the "
+            "written tile(arange(S), N) order — the table was reordered or "
+            "rewritten, so a positional reshape would silently place draws in "
+            "the wrong sample slots (wire contract, #199)"
+        )
+
+    time_blocks = time_col.reshape(n, s)
+    unit_blocks = unit_col.reshape(n, s)
+    if not (
+        bool((time_blocks == time_blocks[:, :1]).all())
+        and bool((unit_blocks == unit_blocks[:, :1]).all())
+    ):
+        raise ValueError(
+            "corrupt frame parquet: 'time'/'unit' are not constant within a "
+            "row's sample block — rows were swapped between cells, so a "
+            "positional reshape would silently misattribute draws "
+            "(wire contract, #199)"
+        )
+
+    time = time_blocks[:, 0]
+    unit = unit_blocks[:, 0]
 
     if ndim == 2:
         values = table.column("value").to_numpy().reshape(n, s).astype(np.float32)

{views_frames-1.9.0 → views_frames-1.10.1}/src/views_frames/prediction_frame.py

@@ -118,6 +118,37 @@ class PredictionFrame:
             )
         return self.select(self._index.searchsorted(other))
 
+    def reindex_fill(
+        self, other: SpatioTemporalIndex, *, fill_value: float
+    ) -> PredictionFrame:
+        """Align to ``other``'s rows, filling rows absent from this frame.
+
+        The dense-grid companion to :meth:`reindex` (ADR-026): every row of
+        ``other`` present in this frame comes through **bit-exact**; a row
+        absent from it gets ``fill_value``, broadcast across the sample axis.
+        There is no superset requirement — that is the difference from the
+        fail-loud ``reindex``. The result's index **is** ``other`` (immutable,
+        shared); metadata is preserved.
+
+        Same-level join semantics: assumes unique ``(time, unit)`` rows in
+        **this** frame (register C-21); ``other`` may repeat rows. Allocates
+        the full ``(other.n_rows, S)`` buffer — densifying a large grid is a
+        deliberate, costly act. A ``NaN`` fill is legal here, but the
+        summarize estimators fail loud on NaN draws by design (ADR-017).
+
+        Raises:
+            ValueError: ``other`` is at a different ``SpatialLevel``.
+        """
+        pos = self._index.searchsorted(other)
+        found = pos >= 0
+        out = np.full(
+            (other.n_rows, *self._values.shape[1:]),
+            np.float32(fill_value),
+            dtype=np.float32,
+        )
+        out[found] = self._values[pos[found]]
+        return PredictionFrame(out, other, self._metadata)
+
     # ---- persistence --------------------------------------------------------
 
     def save(self, directory: Path | str) -> None:

{views_frames-1.9.0 → views_frames-1.10.1}/src/views_frames/target_frame.py

@@ -113,6 +113,36 @@ class TargetFrame:
             )
         return self.select(self._index.searchsorted(other))
 
+    def reindex_fill(
+        self, other: SpatioTemporalIndex, *, fill_value: float
+    ) -> TargetFrame:
+        """Align to ``other``'s rows, filling rows absent from this frame.
+
+        The dense-grid companion to :meth:`reindex` (ADR-026): every row of
+        ``other`` present in this frame comes through **bit-exact**; a row
+        absent from it gets ``fill_value``. There is no superset requirement —
+        that is the difference from the fail-loud ``reindex``. The result's
+        index **is** ``other`` (immutable, shared); metadata is preserved.
+
+        Same-level join semantics: assumes unique ``(time, unit)`` rows in
+        **this** frame (register C-21); ``other`` may repeat rows. Allocates
+        the full ``(other.n_rows, 1)`` buffer — densifying a large grid is a
+        deliberate, costly act. A ``NaN`` fill is legal here, but the
+        summarize estimators fail loud on NaN draws by design (ADR-017).
+
+        Raises:
+            ValueError: ``other`` is at a different ``SpatialLevel``.
+        """
+        pos = self._index.searchsorted(other)
+        found = pos >= 0
+        out = np.full(
+            (other.n_rows, *self._values.shape[1:]),
+            np.float32(fill_value),
+            dtype=np.float32,
+        )
+        out[found] = self._values[pos[found]]
+        return TargetFrame(out, other, self._metadata)
+
     # ---- persistence --------------------------------------------------------
 
     def save(self, directory: Path | str) -> None: