views-frames 1.8.1__tar.gz → 1.10.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {views_frames-1.8.1 → views_frames-1.10.0}/.gitignore +3 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/PKG-INFO +15 -4
- {views_frames-1.8.1 → views_frames-1.10.0}/README.md +14 -3
- {views_frames-1.8.1 → views_frames-1.10.0}/pyproject.toml +1 -1
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/conformance/__init__.py +45 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/feature_frame.py +31 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/index.py +67 -12
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/prediction_frame.py +31 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/target_frame.py +30 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/config.py +9 -3
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/conformance.py +26 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/tower_point.py +9 -3
- {views_frames-1.8.1 → views_frames-1.10.0}/LICENSE +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/__init__.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/_typing.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/_validation.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/io/__init__.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/io/arrow.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/io/npz.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/metadata.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/protocols.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/py.typed +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames/spatial_level.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/__init__.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/conformance.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/frames.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/grouping.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/module.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/proportional.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/py.typed +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/result.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_reconcile/validation.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/__init__.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/_common.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/aggregate.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/bimodality.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/collapse.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/exceedance.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/expected_shortfall.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/interval.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/point.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/py.typed +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/summarize_tower.py +0 -0
- {views_frames-1.8.1 → views_frames-1.10.0}/src/views_frames_summarize/tower.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: views-frames
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.10.0
|
|
4
4
|
Summary: The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG).
|
|
5
5
|
Project-URL: Homepage, https://github.com/views-platform/views-frames
|
|
6
6
|
Project-URL: Repository, https://github.com/views-platform/views-frames
|
|
@@ -36,7 +36,7 @@ Description-Content-Type: text/markdown
|
|
|
36
36
|
> containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
|
|
37
37
|
> that every other repo depends on and that depends on nothing internal.
|
|
38
38
|
>
|
|
39
|
-
> **Status:** **v1.
|
|
39
|
+
> **Status:** **v1.10.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
|
|
40
40
|
> v1.1 surface is
|
|
41
41
|
> purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
|
|
42
42
|
> `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic — no magnitude
|
|
@@ -47,7 +47,13 @@ Description-Content-Type: text/markdown
|
|
|
47
47
|
> sibling package **`views_frames_reconcile`** — forecast reconciliation, ADR-023; v1.8.0
|
|
48
48
|
> adds the native point-country broadcast + the self-describing reconciliation mode,
|
|
49
49
|
> the three showcase notebooks, and the frames-family hardening pass — Reconcile.md CIC,
|
|
50
|
-
> ADR-025 immutability-by-convention, the adversarial red-test batch
|
|
50
|
+
> ADR-025 immutability-by-convention, the adversarial red-test batch; v1.9.0 moves the
|
|
51
|
+
> tower-tip MAP to the **top floor** of the published tower — `tip_mass` 0.25 with the
|
|
52
|
+
> **MAP-containment law** in the conformance suite, ADR-019 Amendment 3 — and adds the
|
|
53
|
+
> `research/figures/` tower-figure toolkit; v1.10.0 adds the **dense-grid fill**
|
|
54
|
+
> primitive — `reindex_fill(other, *, fill_value)` on all three frames +
|
|
55
|
+
> `SpatioTemporalIndex.cartesian` + the published `assert_reindex_fill_law`, ADR-026,
|
|
56
|
+
> unblocking pandas-free FAO ingestion). This
|
|
51
57
|
> README is the design
|
|
52
58
|
> bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
|
|
53
59
|
> io, conformance suite) plus the `src/views_frames_summarize/` sibling package
|
|
@@ -343,7 +349,12 @@ genuinely reused core. Build this once:
|
|
|
343
349
|
`(time, unit)` **at a single `SpatialLevel`**. **This is the label-alignment
|
|
344
350
|
that today drags pandas back in** — pred↔actual join, partial-overlap
|
|
345
351
|
evaluation, same-level reindex. This alignment logic lives in the leaf
|
|
346
|
-
unconditionally.
|
|
352
|
+
unconditionally. The dense-grid pair rides the same join (ADR-026):
|
|
353
|
+
`cartesian(times, units, level)` builds the time-major product index
|
|
354
|
+
(explicit arrays only — deriving them is consumer policy), and the frames'
|
|
355
|
+
`reindex_fill(other, *, fill_value)` aligns with **no** superset requirement,
|
|
356
|
+
filling absent rows (present rows bit-exact; law-pinned by
|
|
357
|
+
`assert_reindex_fill_law`).
|
|
347
358
|
- **Cross-level operations (`cross_level_align`) — protocol here, data injected.**
|
|
348
359
|
The cm↔pgm **cross-level join** (country↔grid) is **not** a same-axis set op; it
|
|
349
360
|
is a one-to-many lookup against a `priogrid→country` mapping that is **injected**
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
> containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
|
|
5
5
|
> that every other repo depends on and that depends on nothing internal.
|
|
6
6
|
>
|
|
7
|
-
> **Status:** **v1.
|
|
7
|
+
> **Status:** **v1.10.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
|
|
8
8
|
> v1.1 surface is
|
|
9
9
|
> purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
|
|
10
10
|
> `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic — no magnitude
|
|
@@ -15,7 +15,13 @@
|
|
|
15
15
|
> sibling package **`views_frames_reconcile`** — forecast reconciliation, ADR-023; v1.8.0
|
|
16
16
|
> adds the native point-country broadcast + the self-describing reconciliation mode,
|
|
17
17
|
> the three showcase notebooks, and the frames-family hardening pass — Reconcile.md CIC,
|
|
18
|
-
> ADR-025 immutability-by-convention, the adversarial red-test batch
|
|
18
|
+
> ADR-025 immutability-by-convention, the adversarial red-test batch; v1.9.0 moves the
|
|
19
|
+
> tower-tip MAP to the **top floor** of the published tower — `tip_mass` 0.25 with the
|
|
20
|
+
> **MAP-containment law** in the conformance suite, ADR-019 Amendment 3 — and adds the
|
|
21
|
+
> `research/figures/` tower-figure toolkit; v1.10.0 adds the **dense-grid fill**
|
|
22
|
+
> primitive — `reindex_fill(other, *, fill_value)` on all three frames +
|
|
23
|
+
> `SpatioTemporalIndex.cartesian` + the published `assert_reindex_fill_law`, ADR-026,
|
|
24
|
+
> unblocking pandas-free FAO ingestion). This
|
|
19
25
|
> README is the design
|
|
20
26
|
> bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
|
|
21
27
|
> io, conformance suite) plus the `src/views_frames_summarize/` sibling package
|
|
@@ -311,7 +317,12 @@ genuinely reused core. Build this once:
|
|
|
311
317
|
`(time, unit)` **at a single `SpatialLevel`**. **This is the label-alignment
|
|
312
318
|
that today drags pandas back in** — pred↔actual join, partial-overlap
|
|
313
319
|
evaluation, same-level reindex. This alignment logic lives in the leaf
|
|
314
|
-
unconditionally.
|
|
320
|
+
unconditionally. The dense-grid pair rides the same join (ADR-026):
|
|
321
|
+
`cartesian(times, units, level)` builds the time-major product index
|
|
322
|
+
(explicit arrays only — deriving them is consumer policy), and the frames'
|
|
323
|
+
`reindex_fill(other, *, fill_value)` aligns with **no** superset requirement,
|
|
324
|
+
filling absent rows (present rows bit-exact; law-pinned by
|
|
325
|
+
`assert_reindex_fill_law`).
|
|
315
326
|
- **Cross-level operations (`cross_level_align`) — protocol here, data injected.**
|
|
316
327
|
The cm↔pgm **cross-level join** (country↔grid) is **not** a same-axis set op; it
|
|
317
328
|
is a one-to-many lookup against a `priogrid→country` mapping that is **injected**
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "views-frames"
|
|
7
|
-
version = "1.
|
|
7
|
+
version = "1.10.0"
|
|
8
8
|
description = "The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG)."
|
|
9
9
|
authors = [
|
|
10
10
|
{ name = "Simon Polichinel von der Maase", email = "simmaa@prio.org" },
|
|
@@ -37,6 +37,7 @@ __all__ = [
|
|
|
37
37
|
"assert_frame_contract",
|
|
38
38
|
"assert_frame_envelope",
|
|
39
39
|
"assert_index_alignment_laws",
|
|
40
|
+
"assert_reindex_fill_law",
|
|
40
41
|
]
|
|
41
42
|
|
|
42
43
|
|
|
@@ -136,6 +137,50 @@ def assert_index_alignment_laws(index_a: Any, index_b: Any) -> None:
|
|
|
136
137
|
assert np.array_equal(index_a.unit[pos], index_a.unit), "searchsorted self-identity"
|
|
137
138
|
|
|
138
139
|
|
|
140
|
+
def assert_reindex_fill_law(frame: Any, target: Any, fill_value: float) -> None:
|
|
141
|
+
"""Assert the dense-grid fill law (ADR-026) for ``frame`` against ``target``.
|
|
142
|
+
|
|
143
|
+
``reindex_fill`` aligns a frame to a target index with **no** superset
|
|
144
|
+
requirement, filling absent rows. The law:
|
|
145
|
+
|
|
146
|
+
- the result's index equals ``target`` row-for-row (time, unit, level);
|
|
147
|
+
- every target row present in ``frame`` comes through **bit-exact**;
|
|
148
|
+
- every absent row equals ``fill_value`` exactly (NaN-safe comparison);
|
|
149
|
+
- when every target row is present, ``reindex_fill`` degenerates to
|
|
150
|
+
``reindex`` (filling a superset frame adds nothing).
|
|
151
|
+
|
|
152
|
+
Args:
|
|
153
|
+
frame: a frame exposing ``reindex_fill``/``reindex``/``index``/``values``.
|
|
154
|
+
target: a ``SpatioTemporalIndex`` (same level) to densify against.
|
|
155
|
+
fill_value: the fill for absent rows (``NaN`` is legal).
|
|
156
|
+
|
|
157
|
+
Raises:
|
|
158
|
+
AssertionError: a law is violated.
|
|
159
|
+
"""
|
|
160
|
+
_require_assertions()
|
|
161
|
+
filled = frame.reindex_fill(target, fill_value=fill_value)
|
|
162
|
+
assert filled.index.level == target.level, "reindex_fill must keep the level"
|
|
163
|
+
assert np.array_equal(filled.index.time, target.time), (
|
|
164
|
+
"reindex_fill result index must equal the target (time)"
|
|
165
|
+
)
|
|
166
|
+
assert np.array_equal(filled.index.unit, target.unit), (
|
|
167
|
+
"reindex_fill result index must equal the target (unit)"
|
|
168
|
+
)
|
|
169
|
+
pos = frame.index.searchsorted(target)
|
|
170
|
+
found = pos >= 0
|
|
171
|
+
assert np.array_equal(
|
|
172
|
+
filled.values[found], frame.values[pos[found]], equal_nan=True
|
|
173
|
+
), "reindex_fill must pass present rows through bit-exact"
|
|
174
|
+
expected = np.full_like(filled.values[~found], np.float32(fill_value))
|
|
175
|
+
assert np.array_equal(filled.values[~found], expected, equal_nan=True), (
|
|
176
|
+
"reindex_fill must set every absent row to fill_value"
|
|
177
|
+
)
|
|
178
|
+
if bool(found.all()):
|
|
179
|
+
assert np.array_equal(
|
|
180
|
+
filled.values, frame.reindex(target).values, equal_nan=True
|
|
181
|
+
), "reindex_fill on a superset frame must equal reindex"
|
|
182
|
+
|
|
183
|
+
|
|
139
184
|
def assert_cross_level_alignment_law(
|
|
140
185
|
index: Any, mapping: Any, target_level: Any
|
|
141
186
|
) -> None:
|
|
@@ -154,6 +154,37 @@ class FeatureFrame:
|
|
|
154
154
|
)
|
|
155
155
|
return self.select(self._index.searchsorted(other))
|
|
156
156
|
|
|
157
|
+
def reindex_fill(
|
|
158
|
+
self, other: SpatioTemporalIndex, *, fill_value: float
|
|
159
|
+
) -> FeatureFrame:
|
|
160
|
+
"""Align to ``other``'s rows, filling rows absent from this frame.
|
|
161
|
+
|
|
162
|
+
The dense-grid companion to :meth:`reindex` (ADR-026): every row of
|
|
163
|
+
``other`` present in this frame comes through **bit-exact**; a row
|
|
164
|
+
absent from it gets ``fill_value``, broadcast across the feature and
|
|
165
|
+
sample axes. There is no superset requirement — that is the difference
|
|
166
|
+
from the fail-loud ``reindex``. The result's index **is** ``other``
|
|
167
|
+
(immutable, shared); ``feature_names`` and metadata are preserved.
|
|
168
|
+
|
|
169
|
+
Same-level join semantics: assumes unique ``(time, unit)`` rows in
|
|
170
|
+
**this** frame (register C-21); ``other`` may repeat rows. Allocates
|
|
171
|
+
the full ``(other.n_rows, F, S)`` buffer — densifying a large grid is
|
|
172
|
+
a deliberate, costly act. A ``NaN`` fill is legal here, but the
|
|
173
|
+
summarize estimators fail loud on NaN draws by design (ADR-017).
|
|
174
|
+
|
|
175
|
+
Raises:
|
|
176
|
+
ValueError: ``other`` is at a different ``SpatialLevel``.
|
|
177
|
+
"""
|
|
178
|
+
pos = self._index.searchsorted(other)
|
|
179
|
+
found = pos >= 0
|
|
180
|
+
out = np.full(
|
|
181
|
+
(other.n_rows, *self._values.shape[1:]),
|
|
182
|
+
np.float32(fill_value),
|
|
183
|
+
dtype=np.float32,
|
|
184
|
+
)
|
|
185
|
+
out[found] = self._values[pos[found]]
|
|
186
|
+
return FeatureFrame(out, other, self._feature_names, self._metadata)
|
|
187
|
+
|
|
157
188
|
# ---- persistence --------------------------------------------------------
|
|
158
189
|
|
|
159
190
|
def save(self, directory: Path | str) -> None:
|
|
@@ -43,9 +43,7 @@ class SpatioTemporalIndex:
|
|
|
43
43
|
level: SpatialLevel,
|
|
44
44
|
) -> None:
|
|
45
45
|
if not isinstance(level, SpatialLevel):
|
|
46
|
-
raise TypeError(
|
|
47
|
-
f"level must be a SpatialLevel, got {type(level).__name__}"
|
|
48
|
-
)
|
|
46
|
+
raise TypeError(f"level must be a SpatialLevel, got {type(level).__name__}")
|
|
49
47
|
is_array = isinstance(time, np.ndarray) and time.ndim >= 1
|
|
50
48
|
n = int(time.shape[0]) if is_array else -1
|
|
51
49
|
validate_identifiers({"time": time, "unit": unit}, n_rows=n)
|
|
@@ -56,6 +54,65 @@ class SpatioTemporalIndex:
|
|
|
56
54
|
self._unit.setflags(write=False)
|
|
57
55
|
self._level = level
|
|
58
56
|
|
|
57
|
+
# ---- construction -------------------------------------------------------
|
|
58
|
+
|
|
59
|
+
@classmethod
|
|
60
|
+
def cartesian(
|
|
61
|
+
cls,
|
|
62
|
+
times: IntArray,
|
|
63
|
+
units: IntArray,
|
|
64
|
+
level: SpatialLevel,
|
|
65
|
+
) -> SpatioTemporalIndex:
|
|
66
|
+
"""The dense ``(time × unit)`` product index, in time-major order.
|
|
67
|
+
|
|
68
|
+
The dense-grid constructor (ADR-026): every combination of the given
|
|
69
|
+
``times`` and ``units``, ordered time-major (all units of the first time,
|
|
70
|
+
then all units of the next), so dense indices are canonical across
|
|
71
|
+
consumers. The inputs are **explicit arrays only** — any rule for
|
|
72
|
+
deriving them (e.g. "the units present in the last time step") is
|
|
73
|
+
consumer policy and never lives in the leaf.
|
|
74
|
+
|
|
75
|
+
Fails loud on duplicated ``times`` or ``units``: a duplicated product
|
|
76
|
+
input manufactures duplicate ``(time, unit)`` rows, which make every
|
|
77
|
+
same-level join undefined (register C-21) — always a caller bug here.
|
|
78
|
+
|
|
79
|
+
Allocates ``len(times) × len(units)`` rows eagerly; densifying a large
|
|
80
|
+
grid is a deliberate, costly act (a full-pgm dense grid runs to tens of
|
|
81
|
+
millions of rows).
|
|
82
|
+
|
|
83
|
+
Args:
|
|
84
|
+
times: 1-D integer array of unique time identifiers.
|
|
85
|
+
units: 1-D integer array of unique unit identifiers.
|
|
86
|
+
level: The ``SpatialLevel`` of the produced index.
|
|
87
|
+
|
|
88
|
+
Raises:
|
|
89
|
+
TypeError: ``times``/``units`` is not an integer-dtype numpy array,
|
|
90
|
+
or ``level`` is not a ``SpatialLevel``.
|
|
91
|
+
ValueError: an input array is not 1-D or contains duplicate values.
|
|
92
|
+
"""
|
|
93
|
+
for name, arr in (("times", times), ("units", units)):
|
|
94
|
+
if not isinstance(arr, np.ndarray):
|
|
95
|
+
raise TypeError(
|
|
96
|
+
f"cartesian {name} must be a numpy array, got {type(arr).__name__}"
|
|
97
|
+
)
|
|
98
|
+
if not np.issubdtype(arr.dtype, np.integer):
|
|
99
|
+
raise TypeError(
|
|
100
|
+
f"cartesian {name} must be an integer dtype, got {arr.dtype}"
|
|
101
|
+
)
|
|
102
|
+
if arr.ndim != 1:
|
|
103
|
+
raise ValueError(f"cartesian {name} must be 1-D, got ndim={arr.ndim}")
|
|
104
|
+
if np.unique(arr).shape[0] != arr.shape[0]:
|
|
105
|
+
raise ValueError(
|
|
106
|
+
f"cartesian {name} contains duplicate values; a duplicated "
|
|
107
|
+
"product input manufactures duplicate (time, unit) rows, "
|
|
108
|
+
"which make same-level joins undefined (register C-21)"
|
|
109
|
+
)
|
|
110
|
+
return cls(
|
|
111
|
+
time=np.repeat(times, units.shape[0]),
|
|
112
|
+
unit=np.tile(units, times.shape[0]),
|
|
113
|
+
level=level,
|
|
114
|
+
)
|
|
115
|
+
|
|
59
116
|
# ---- core surface -------------------------------------------------------
|
|
60
117
|
|
|
61
118
|
@property
|
|
@@ -109,9 +166,11 @@ class SpatioTemporalIndex:
|
|
|
109
166
|
@staticmethod
|
|
110
167
|
def _row_view(keys: NDArray[np.int64]) -> NDArray[np.void]:
|
|
111
168
|
"""View each ``(time, unit)`` row as a single void scalar for set ops."""
|
|
112
|
-
return
|
|
113
|
-
np.
|
|
114
|
-
|
|
169
|
+
return (
|
|
170
|
+
np.ascontiguousarray(keys)
|
|
171
|
+
.view(np.dtype((np.void, keys.dtype.itemsize * keys.shape[1])))
|
|
172
|
+
.reshape(-1)
|
|
173
|
+
)
|
|
115
174
|
|
|
116
175
|
def _require_same_level(self, other: SpatioTemporalIndex) -> None:
|
|
117
176
|
if self._level != other._level:
|
|
@@ -180,9 +239,7 @@ class SpatioTemporalIndex:
|
|
|
180
239
|
rows = self._row_view(self._keys())
|
|
181
240
|
return bool(len(np.unique(rows)) == rows.shape[0])
|
|
182
241
|
|
|
183
|
-
def select(
|
|
184
|
-
self, indexer: IntArray | NDArray[np.bool_]
|
|
185
|
-
) -> SpatioTemporalIndex:
|
|
242
|
+
def select(self, indexer: IntArray | NDArray[np.bool_]) -> SpatioTemporalIndex:
|
|
186
243
|
"""A new index of the rows at integer positions **or** a boolean mask.
|
|
187
244
|
|
|
188
245
|
The row-selection primitive the frame-level ``select``/``reindex`` build on:
|
|
@@ -298,9 +355,7 @@ class SpatioTemporalIndex:
|
|
|
298
355
|
sorted_rows = map_rows[order]
|
|
299
356
|
|
|
300
357
|
self_rows = self._row_view(self._keys())
|
|
301
|
-
pos = np.clip(
|
|
302
|
-
np.searchsorted(sorted_rows, self_rows), 0, len(sorted_rows) - 1
|
|
303
|
-
)
|
|
358
|
+
pos = np.clip(np.searchsorted(sorted_rows, self_rows), 0, len(sorted_rows) - 1)
|
|
304
359
|
found = sorted_rows[pos] == self_rows
|
|
305
360
|
if not bool(found.all()):
|
|
306
361
|
miss = int(np.argmax(~found))
|
|
@@ -118,6 +118,37 @@ class PredictionFrame:
|
|
|
118
118
|
)
|
|
119
119
|
return self.select(self._index.searchsorted(other))
|
|
120
120
|
|
|
121
|
+
def reindex_fill(
|
|
122
|
+
self, other: SpatioTemporalIndex, *, fill_value: float
|
|
123
|
+
) -> PredictionFrame:
|
|
124
|
+
"""Align to ``other``'s rows, filling rows absent from this frame.
|
|
125
|
+
|
|
126
|
+
The dense-grid companion to :meth:`reindex` (ADR-026): every row of
|
|
127
|
+
``other`` present in this frame comes through **bit-exact**; a row
|
|
128
|
+
absent from it gets ``fill_value``, broadcast across the sample axis.
|
|
129
|
+
There is no superset requirement — that is the difference from the
|
|
130
|
+
fail-loud ``reindex``. The result's index **is** ``other`` (immutable,
|
|
131
|
+
shared); metadata is preserved.
|
|
132
|
+
|
|
133
|
+
Same-level join semantics: assumes unique ``(time, unit)`` rows in
|
|
134
|
+
**this** frame (register C-21); ``other`` may repeat rows. Allocates
|
|
135
|
+
the full ``(other.n_rows, S)`` buffer — densifying a large grid is a
|
|
136
|
+
deliberate, costly act. A ``NaN`` fill is legal here, but the
|
|
137
|
+
summarize estimators fail loud on NaN draws by design (ADR-017).
|
|
138
|
+
|
|
139
|
+
Raises:
|
|
140
|
+
ValueError: ``other`` is at a different ``SpatialLevel``.
|
|
141
|
+
"""
|
|
142
|
+
pos = self._index.searchsorted(other)
|
|
143
|
+
found = pos >= 0
|
|
144
|
+
out = np.full(
|
|
145
|
+
(other.n_rows, *self._values.shape[1:]),
|
|
146
|
+
np.float32(fill_value),
|
|
147
|
+
dtype=np.float32,
|
|
148
|
+
)
|
|
149
|
+
out[found] = self._values[pos[found]]
|
|
150
|
+
return PredictionFrame(out, other, self._metadata)
|
|
151
|
+
|
|
121
152
|
# ---- persistence --------------------------------------------------------
|
|
122
153
|
|
|
123
154
|
def save(self, directory: Path | str) -> None:
|
|
@@ -113,6 +113,36 @@ class TargetFrame:
|
|
|
113
113
|
)
|
|
114
114
|
return self.select(self._index.searchsorted(other))
|
|
115
115
|
|
|
116
|
+
def reindex_fill(
|
|
117
|
+
self, other: SpatioTemporalIndex, *, fill_value: float
|
|
118
|
+
) -> TargetFrame:
|
|
119
|
+
"""Align to ``other``'s rows, filling rows absent from this frame.
|
|
120
|
+
|
|
121
|
+
The dense-grid companion to :meth:`reindex` (ADR-026): every row of
|
|
122
|
+
``other`` present in this frame comes through **bit-exact**; a row
|
|
123
|
+
absent from it gets ``fill_value``. There is no superset requirement —
|
|
124
|
+
that is the difference from the fail-loud ``reindex``. The result's
|
|
125
|
+
index **is** ``other`` (immutable, shared); metadata is preserved.
|
|
126
|
+
|
|
127
|
+
Same-level join semantics: assumes unique ``(time, unit)`` rows in
|
|
128
|
+
**this** frame (register C-21); ``other`` may repeat rows. Allocates
|
|
129
|
+
the full ``(other.n_rows, 1)`` buffer — densifying a large grid is a
|
|
130
|
+
deliberate, costly act. A ``NaN`` fill is legal here, but the
|
|
131
|
+
summarize estimators fail loud on NaN draws by design (ADR-017).
|
|
132
|
+
|
|
133
|
+
Raises:
|
|
134
|
+
ValueError: ``other`` is at a different ``SpatialLevel``.
|
|
135
|
+
"""
|
|
136
|
+
pos = self._index.searchsorted(other)
|
|
137
|
+
found = pos >= 0
|
|
138
|
+
out = np.full(
|
|
139
|
+
(other.n_rows, *self._values.shape[1:]),
|
|
140
|
+
np.float32(fill_value),
|
|
141
|
+
dtype=np.float32,
|
|
142
|
+
)
|
|
143
|
+
out[found] = self._values[pos[found]]
|
|
144
|
+
return TargetFrame(out, other, self._metadata)
|
|
145
|
+
|
|
116
146
|
# ---- persistence --------------------------------------------------------
|
|
117
147
|
|
|
118
148
|
def save(self, directory: Path | str) -> None:
|
|
@@ -28,9 +28,15 @@ TOWER_CONFIG: Final[dict[str, Any]] = {
|
|
|
28
28
|
[round(0.05 * i, 2) for i in range(1, 19)] # 0.05 … 0.90
|
|
29
29
|
+ [0.92, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99] # fine high-mass tail
|
|
30
30
|
),
|
|
31
|
-
# The tower-tip reads the median of the floor at this mass — the
|
|
32
|
-
#
|
|
33
|
-
|
|
31
|
+
# The tower-tip reads the median of the floor at this mass — the top floor of the
|
|
32
|
+
# published tower. 0.25 (the top-quartile floor; ADR-019 amendment 2026-07-24,
|
|
33
|
+
# evidence in research/map_hdi/tip_mass_study.py): lower bias than the 0.5 shorth
|
|
34
|
+
# everywhere, RMSE-superior at pooled S, exact on zero-inflated cells, and it
|
|
35
|
+
# passes the real-cell C-44 gate with margin — the tip floor's median is safe as
|
|
36
|
+
# long as a duplicate stack is under half the floor (k > 2·duplicates; at S=32,
|
|
37
|
+
# k=8 tolerates the observed 2-3-zero stacks). Containment law: every HDI of mass
|
|
38
|
+
# > tip_mass/2 provably contains the tip (asserted in conformance.py).
|
|
39
|
+
"tip_mass": 0.25,
|
|
34
40
|
# Optional magnitude zero rule, OFF by default (``None``). When set to a float, a
|
|
35
41
|
# row whose every draw is <= it collapses to 0 — a *count* opt-in (sub-1 ⇒ 0).
|
|
36
42
|
# Left ``None`` for any non-count target: zero-inflation is otherwise handled by the
|
|
@@ -126,6 +126,32 @@ def _assert_tower_contract(frame: AnyFrame, n: int) -> None:
|
|
|
126
126
|
assert (tip.values[..., 0] >= tlo - 1e-6).all(), "tip below the tip_mass floor"
|
|
127
127
|
assert (tip.values[..., 0] <= thi + 1e-6).all(), "tip above the tip_mass floor"
|
|
128
128
|
|
|
129
|
+
# MAP-containment law (ADR-019 amendment 2026-07-24). Wider-than-tip_mass floors
|
|
130
|
+
# contain the tip by nesting. Below tip_mass, a nested floor still contains it
|
|
131
|
+
# whenever it holds MORE THAN HALF the tip floor's draws: a contiguous sub-window
|
|
132
|
+
# longer than half the parent cannot trim away the parent's middle draw(s), and
|
|
133
|
+
# the tip is their median/average. A floor of mass m spans floor(m·S)+1 draws
|
|
134
|
+
# (the `_ks` value counts inter-draw steps), so the exact condition is
|
|
135
|
+
# 2·(floor(m·S)+1) > floor(tip_mass·S)+1 — asymptotically mass > tip_mass/2.
|
|
136
|
+
# Floors below it carry NO guarantee and are below platform sample resolution
|
|
137
|
+
# (see tower_point.py / research/map_hdi/tip_mass_study.py).
|
|
138
|
+
s_count = int(frame.values.shape[-1])
|
|
139
|
+
n_tip = int(np.floor(tip_mass * s_count)) + 1
|
|
140
|
+
guaranteed = tuple(
|
|
141
|
+
float(m)
|
|
142
|
+
for m in config.canonical_floors()
|
|
143
|
+
if 2 * (int(np.floor(float(m) * s_count)) + 1) > n_tip
|
|
144
|
+
)
|
|
145
|
+
law_tower = hdi_tower(frame, masses=guaranteed)
|
|
146
|
+
for j, m in enumerate(guaranteed):
|
|
147
|
+
glo, ghi = law_tower[..., j, 0], law_tower[..., j, 1]
|
|
148
|
+
assert (tip.values[..., 0] >= glo - 1e-6).all(), (
|
|
149
|
+
f"MAP-containment violated: tip below the {m:.2f} floor"
|
|
150
|
+
)
|
|
151
|
+
assert (tip.values[..., 0] <= ghi + 1e-6).all(), (
|
|
152
|
+
f"MAP-containment violated: tip above the {m:.2f} floor"
|
|
153
|
+
)
|
|
154
|
+
|
|
129
155
|
# Reproducibility law: the 50% HDI is independent of the other requested masses.
|
|
130
156
|
just_50 = hdi_tower(frame, masses=(0.5,))
|
|
131
157
|
assert np.array_equal(just_50[..., 0, :], tower[..., 0, :]), (
|
|
@@ -2,15 +2,21 @@
|
|
|
2
2
|
|
|
3
3
|
`tower_point` is the "most likely single value" we report to a consumer: the median of
|
|
4
4
|
the draws inside the **`tip_mass` floor** of the nested tower (config-driven, default
|
|
5
|
-
0.
|
|
6
|
-
|
|
5
|
+
0.25 — the top-quartile floor, i.e. the top floor of the published tower; ADR-019
|
|
6
|
+
amendment 2026-07-24, evidence in `research/map_hdi/tip_mass_study.py`). Zero-inflation
|
|
7
|
+
is handled by that floor's density (a zero-majority row reads 0); the optional,
|
|
8
|
+
off-by-default magnitude cutoff applies too if set (C-45). Containment law: every HDI
|
|
9
|
+
whose floor holds more than half the tip floor's draws provably contains the tip
|
|
10
|
+
(asymptotically, mass > `tip_mass`/2 — a nested contiguous window longer than half
|
|
11
|
+
the parent cannot trim away the parent's median); narrower floors carry no guarantee
|
|
12
|
+
and are below platform sample resolution. Asserted in ``conformance.py``.
|
|
7
13
|
|
|
8
14
|
It is a *new* estimator, deliberately distinct from the frozen `map_estimate` (ADR-018):
|
|
9
15
|
`map_estimate` is a binned histogram mode with a zero-*mass*-fraction rule and a
|
|
10
16
|
lowest-index tie-break that is directionally biased on right-skewed, zero-inflated,
|
|
11
17
|
low-sample posteriors (register C-32). The tower tip is unbinned — it reads the median
|
|
12
18
|
of a mass-aware floor — so it carries no histogram tie-break **and** is robust to
|
|
13
|
-
minority duplicated draws (register C-44; a lonely outlier cannot define a
|
|
19
|
+
minority duplicated draws (register C-44; a lonely outlier cannot define a 25%-mass
|
|
14
20
|
floor). On a genuinely multi-peaked row the tip is, like any point, ambiguous; pair it
|
|
15
21
|
with `bimodality` to detect that case rather than collapse it silently.
|
|
16
22
|
"""
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|