views-frames 1.2.0__tar.gz → 1.6.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. {views_frames-1.2.0 → views_frames-1.6.0}/PKG-INFO +18 -9
  2. {views_frames-1.2.0 → views_frames-1.6.0}/README.md +17 -8
  3. {views_frames-1.2.0 → views_frames-1.6.0}/pyproject.toml +1 -1
  4. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/conformance/__init__.py +40 -10
  5. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/metadata.py +7 -0
  6. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/__init__.py +5 -0
  7. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/config.py +6 -3
  8. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/conformance.py +32 -0
  9. views_frames-1.6.0/src/views_frames_summarize/exceedance.py +96 -0
  10. views_frames-1.6.0/src/views_frames_summarize/expected_shortfall.py +88 -0
  11. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/tower.py +22 -11
  12. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/tower_point.py +6 -5
  13. {views_frames-1.2.0 → views_frames-1.6.0}/.gitignore +0 -0
  14. {views_frames-1.2.0 → views_frames-1.6.0}/LICENSE +0 -0
  15. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/__init__.py +0 -0
  16. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/_typing.py +0 -0
  17. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/_validation.py +0 -0
  18. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/feature_frame.py +0 -0
  19. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/index.py +0 -0
  20. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/io/__init__.py +0 -0
  21. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/io/arrow.py +0 -0
  22. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/io/npz.py +0 -0
  23. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/prediction_frame.py +0 -0
  24. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/protocols.py +0 -0
  25. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/py.typed +0 -0
  26. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/spatial_level.py +0 -0
  27. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames/target_frame.py +0 -0
  28. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/_common.py +0 -0
  29. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/aggregate.py +0 -0
  30. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/bimodality.py +0 -0
  31. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/collapse.py +0 -0
  32. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/interval.py +0 -0
  33. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/point.py +0 -0
  34. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/py.typed +0 -0
  35. {views_frames-1.2.0 → views_frames-1.6.0}/src/views_frames_summarize/summarize_tower.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: views-frames
3
- Version: 1.2.0
3
+ Version: 1.6.0
4
4
  Summary: The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG).
5
5
  Project-URL: Homepage, https://github.com/views-platform/views-frames
6
6
  Project-URL: Repository, https://github.com/views-platform/views-frames
@@ -30,9 +30,14 @@ Description-Content-Type: text/markdown
30
30
  > containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
31
31
  > that every other repo depends on and that depends on nothing internal.
32
32
  >
33
- > **Status:** **v1.2.0 — frozen API** (frozen since v1.0.0, ADR-018; the v1.1 surface is
34
- > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilds the tower
35
- > `outside-in` and makes it config-driven, register C-44). This README is the design
33
+ > **Status:** **v1.6.0 — frozen API** (frozen since v1.0.0, ADR-018; the v1.1 surface is
34
+ > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
35
+ > `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic no magnitude
36
+ > zeroing by default, register C-45; v1.4.0 adds generic provenance to `FrameMetadata`
37
+ > (`run_id`/`data_version`) and publishes the shared `assert_frame_envelope` checker,
38
+ > ADR-020; v1.5.0 adds the threshold **exceedance** estimator `P(Y > c)`, ADR-021; v1.6.0
39
+ > adds the worst-case **expected_shortfall** estimator, ADR-022). This
40
+ > README is the design
36
41
  > bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
37
42
  > io, conformance suite) plus the `src/views_frames_summarize/` sibling package
38
43
  > (sample-axis summarization — `collapse`/MAP/HDI/quantiles, the coherent-tower estimators
@@ -96,16 +101,20 @@ Both are numpy-only. For the subtler cm↔pgm surface — a time-varying
96
101
  `aggregate_distributions` (`HDI(sum) ≠ sum(HDI)`) — see
97
102
  [`examples/cross_level.py`](examples/cross_level.py).
98
103
 
99
- **Which estimator? (two coherent paths, v1.2.0).** Each frozen estimator has a
104
+ **Which estimator? (two coherent paths, v1.3.0).** Each frozen estimator has a
100
105
  coherent-tower sibling (ADR-019): `map_estimate` ↔ `tower_point` (an unbinned
101
106
  median-of-the-`tip_mass`-floor point — the "shorth", free of `map_estimate`'s histogram
102
107
  tie-break bias **and** robust to minority duplicated draws, register C-44), and
103
108
  `hdi`/`quantiles` ↔ `hdi_tower(masses=…)` (HDIs built **outside-in**, nested **by
104
109
  construction** and reproducible — a mass's interval is identical regardless of which others
105
- you request), with `summarize_tower` returning all three in one pass. The tower's tunables
106
- live in `views_frames_summarize.config` (fail-loud, no silent defaults). Use the **frozen**
107
- estimators for parity with existing pipelines; use the **tower** path when you need coherent,
108
- reproducible bands plus a matching, duplicate-robust point.
110
+ you request), with `summarize_tower` returning all three in one pass. The tower path is
111
+ **distribution-agnostic** it works for counts, continuous, normal, and `[0,1]`
112
+ (rate/probability) targets, with **no magnitude zeroing by default** (zero-inflation is read
113
+ off the floor's density, register C-45); a count consumer that wants "sub-1 ⇒ 0" opts in via
114
+ `config['zero_cutoff']`. The tower's tunables live in `views_frames_summarize.config`
115
+ (fail-loud, no silent defaults). Use the **frozen** estimators for parity with existing
116
+ pipelines; use the **tower** path when you need coherent, reproducible bands plus a matching,
117
+ duplicate-robust point.
109
118
 
110
119
  **Reading the `bimodality` flag.** It is a *deliberately conservative* heuristic, **not** a
111
120
  formal multimodality test — tuned for **zero false positives** at the cost of recall on
@@ -4,9 +4,14 @@
4
4
  > containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
5
5
  > that every other repo depends on and that depends on nothing internal.
6
6
  >
7
- > **Status:** **v1.2.0 — frozen API** (frozen since v1.0.0, ADR-018; the v1.1 surface is
8
- > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilds the tower
9
- > `outside-in` and makes it config-driven, register C-44). This README is the design
7
+ > **Status:** **v1.6.0 — frozen API** (frozen since v1.0.0, ADR-018; the v1.1 surface is
8
+ > purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
9
+ > `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic no magnitude
10
+ > zeroing by default, register C-45; v1.4.0 adds generic provenance to `FrameMetadata`
11
+ > (`run_id`/`data_version`) and publishes the shared `assert_frame_envelope` checker,
12
+ > ADR-020; v1.5.0 adds the threshold **exceedance** estimator `P(Y > c)`, ADR-021; v1.6.0
13
+ > adds the worst-case **expected_shortfall** estimator, ADR-022). This
14
+ > README is the design
10
15
  > bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
11
16
  > io, conformance suite) plus the `src/views_frames_summarize/` sibling package
12
17
  > (sample-axis summarization — `collapse`/MAP/HDI/quantiles, the coherent-tower estimators
@@ -70,16 +75,20 @@ Both are numpy-only. For the subtler cm↔pgm surface — a time-varying
70
75
  `aggregate_distributions` (`HDI(sum) ≠ sum(HDI)`) — see
71
76
  [`examples/cross_level.py`](examples/cross_level.py).
72
77
 
73
- **Which estimator? (two coherent paths, v1.2.0).** Each frozen estimator has a
78
+ **Which estimator? (two coherent paths, v1.3.0).** Each frozen estimator has a
74
79
  coherent-tower sibling (ADR-019): `map_estimate` ↔ `tower_point` (an unbinned
75
80
  median-of-the-`tip_mass`-floor point — the "shorth", free of `map_estimate`'s histogram
76
81
  tie-break bias **and** robust to minority duplicated draws, register C-44), and
77
82
  `hdi`/`quantiles` ↔ `hdi_tower(masses=…)` (HDIs built **outside-in**, nested **by
78
83
  construction** and reproducible — a mass's interval is identical regardless of which others
79
- you request), with `summarize_tower` returning all three in one pass. The tower's tunables
80
- live in `views_frames_summarize.config` (fail-loud, no silent defaults). Use the **frozen**
81
- estimators for parity with existing pipelines; use the **tower** path when you need coherent,
82
- reproducible bands plus a matching, duplicate-robust point.
84
+ you request), with `summarize_tower` returning all three in one pass. The tower path is
85
+ **distribution-agnostic** it works for counts, continuous, normal, and `[0,1]`
86
+ (rate/probability) targets, with **no magnitude zeroing by default** (zero-inflation is read
87
+ off the floor's density, register C-45); a count consumer that wants "sub-1 ⇒ 0" opts in via
88
+ `config['zero_cutoff']`. The tower's tunables live in `views_frames_summarize.config`
89
+ (fail-loud, no silent defaults). Use the **frozen** estimators for parity with existing
90
+ pipelines; use the **tower** path when you need coherent, reproducible bands plus a matching,
91
+ duplicate-robust point.
83
92
 
84
93
  **Reading the `bimodality` flag.** It is a *deliberately conservative* heuristic, **not** a
85
94
  formal multimodality test — tuned for **zero false positives** at the cost of recall on
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
4
4
 
5
5
  [project]
6
6
  name = "views-frames"
7
- version = "1.2.0"
7
+ version = "1.6.0"
8
8
  description = "The VIEWS platform data-contract layer: immutable array+identifier frames (numpy only, root of the dependency DAG)."
9
9
  authors = [
10
10
  { name = "Simon Polichinel von der Maase", email = "simmaa@prio.org" },
@@ -10,6 +10,14 @@ Usage in a consumer's test::
10
10
  from views_frames.conformance import assert_frame_contract
11
11
  assert_frame_contract(my_adapter_output())
12
12
 
13
+ A consumer whose type is **not** spatiotemporal (a string-keyed evaluation output, e.g.
14
+ views-evaluation's ``MetricFrame``) validates against the shared envelope instead, so
15
+ the envelope has one written authority rather than re-asserted copies that drift
16
+ (ADR-020, register C-46)::
17
+
18
+ from views_frames.conformance import assert_frame_envelope
19
+ assert_frame_envelope(my_metric_frame())
20
+
13
21
  The floor is governed in ``GOVERNANCE.md``; ``CONFORMANCE_FLOOR`` records the version
14
22
  this suite belongs to.
15
23
  """
@@ -27,20 +35,26 @@ __all__ = [
27
35
  "CONFORMANCE_FLOOR",
28
36
  "assert_cross_level_alignment_law",
29
37
  "assert_frame_contract",
38
+ "assert_frame_envelope",
30
39
  "assert_index_alignment_laws",
31
40
  ]
32
41
 
33
42
 
34
- def assert_frame_contract(frame: Any) -> None:
35
- """Assert ``frame`` satisfies the views-frames data contract.
43
+ def assert_frame_envelope(frame: Any) -> None:
44
+ """Assert ``frame`` satisfies the shared **frame envelope**.
45
+
46
+ The envelope is the subset of the contract that applies to *any* frame-like value
47
+ object, spatiotemporal or not: float32 values with no object dtype, an explicit
48
+ trailing axis, rows equal to ``n_rows``, and a save/load round-trip that preserves
49
+ the values and every identifier array. It deliberately says nothing about *which*
50
+ identifiers exist or their dtype — that is the spatiotemporal layer's concern.
36
51
 
37
- Checks the structural invariants (float32 values, no object dtype, an explicit
38
- trailing axis, complete integer identifiers of length ``n_rows``) and the
39
- save/load round-trip. (Sample-axis reduction is the ``views_frames_summarize``
40
- package's concern, not the contract's — ADR-017.)
52
+ This is the single written authority for the envelope so a sibling that re-asserts
53
+ it (views-evaluation's ``MetricFrame``, keyed by string axes — not ``(time, unit)``)
54
+ validates against this checker rather than drifting from it (ADR-020, C-46).
41
55
 
42
56
  Raises:
43
- AssertionError: any part of the contract is violated.
57
+ AssertionError: any part of the envelope is violated.
44
58
  """
45
59
  values = frame.values
46
60
  assert isinstance(values, np.ndarray), "values must be a numpy array"
@@ -49,6 +63,22 @@ def assert_frame_contract(frame: Any) -> None:
49
63
  assert values.ndim >= 2, "values must have an explicit trailing sample axis"
50
64
  assert values.shape[0] == frame.n_rows, "values rows must equal n_rows"
51
65
 
66
+ _assert_roundtrip(frame)
67
+
68
+
69
+ def assert_frame_contract(frame: Any) -> None:
70
+ """Assert ``frame`` satisfies the full views-frames **spatiotemporal** contract.
71
+
72
+ The shared envelope (:func:`assert_frame_envelope`) plus the spatiotemporal
73
+ requirement: complete integer ``time``/``unit`` identifiers of length ``n_rows``.
74
+ (Sample-axis reduction is the ``views_frames_summarize`` package's concern, not the
75
+ contract's — ADR-017.)
76
+
77
+ Raises:
78
+ AssertionError: the envelope or the spatiotemporal identifier rule is violated.
79
+ """
80
+ assert_frame_envelope(frame)
81
+
52
82
  ids = frame.identifiers
53
83
  for key in ("time", "unit"):
54
84
  assert key in ids, f"missing required identifier '{key}'"
@@ -56,14 +86,14 @@ def assert_frame_contract(frame: Any) -> None:
56
86
  assert np.issubdtype(arr.dtype, np.integer), f"'{key}' must be integer"
57
87
  assert arr.shape == (frame.n_rows,), f"'{key}' must be length n_rows"
58
88
 
59
- _assert_roundtrip(frame)
60
-
61
89
 
62
90
  def _assert_roundtrip(frame: Any) -> None:
63
91
  with tempfile.TemporaryDirectory() as directory:
64
92
  frame.save(directory)
65
93
  loaded = type(frame).load(directory)
66
- assert np.array_equal(loaded.values, frame.values), "save/load changed values"
94
+ assert np.array_equal(loaded.values, frame.values, equal_nan=True), (
95
+ "save/load changed values"
96
+ )
67
97
  for key, arr in frame.identifiers.items():
68
98
  assert np.array_equal(loaded.identifiers[key], arr), (
69
99
  f"save/load changed identifier '{key}'"
@@ -3,6 +3,11 @@
3
3
  Not a free-form dict: a frozen dataclass with all-optional, validated fields, so
4
4
  adding a field is a MINOR change and consumers cannot diverge on key names (the
5
5
  store-side cause of reporting's C-48). It is the typed home for run/eval identity.
6
+
7
+ Provenance fields are **generic only** (``run_id``, ``data_version``): meaningful for
8
+ any frame. Evaluation-specific provenance (``scoring_code_version``, full-precision
9
+ ``evaluation_timestamp``) stays in views-evaluation's ``MetricFrame`` and must not leak
10
+ into this generic header (ADR-020, register C-47).
6
11
  """
7
12
 
8
13
  from __future__ import annotations
@@ -20,6 +25,8 @@ class FrameMetadata:
20
25
  run_type: str | None = None
21
26
  timestamp: int | None = None
22
27
  seed: int | None = None
28
+ run_id: str | None = None
29
+ data_version: str | None = None
23
30
 
24
31
  def to_dict(self) -> dict[str, Any]:
25
32
  """Serialize to a plain dict, omitting unset (``None``) fields."""
@@ -17,6 +17,8 @@ from views_frames_summarize.aggregate import (
17
17
  )
18
18
  from views_frames_summarize.bimodality import bimodality
19
19
  from views_frames_summarize.collapse import collapse
20
+ from views_frames_summarize.exceedance import exceedance, exceedance_reducer
21
+ from views_frames_summarize.expected_shortfall import expected_shortfall
20
22
  from views_frames_summarize.interval import hdi, quantiles
21
23
  from views_frames_summarize.point import map_estimate
22
24
  from views_frames_summarize.summarize_tower import TowerSummary, summarize_tower
@@ -29,6 +31,9 @@ __all__ = [
29
31
  "aggregate_distributions_arrays",
30
32
  "bimodality",
31
33
  "collapse",
34
+ "exceedance",
35
+ "exceedance_reducer",
36
+ "expected_shortfall",
32
37
  "hdi",
33
38
  "hdi_tower",
34
39
  "map_estimate",
@@ -31,9 +31,12 @@ TOWER_CONFIG: Final[dict[str, Any]] = {
31
31
  # The tower-tip reads the median of the floor at this mass — the "shorth". 0.5 is
32
32
  # the maximally-robust choice: a duplicate would need ~half the draws to hijack it.
33
33
  "tip_mass": 0.5,
34
- # Raw-count zero rule: a row whose every draw is <= this collapses to 0 (the quiet
35
- # cell). A *count* rule, distinct from ``map_estimate``'s zero-mass-fraction rule.
36
- "zero_cutoff": 1.0,
34
+ # Optional magnitude zero rule, OFF by default (``None``). When set to a float, a
35
+ # row whose every draw is <= it collapses to 0 — a *count* opt-in (sub-1 ⇒ 0).
36
+ # Left ``None`` for any non-count target: zero-inflation is otherwise handled by the
37
+ # density of the ``tip_mass`` floor, not by magnitude (register C-45). The leaf
38
+ # imposes no magnitude assumption by default; a consumer that wants it sets this.
39
+ "zero_cutoff": None,
37
40
  # Bimodality detector (coarse histogram + light smoothing; see bimodality.py).
38
41
  "bimodality_bins": 16,
39
42
  "bimodality_prominence": 0.40,
@@ -13,6 +13,8 @@ from views_frames_summarize import config
13
13
  from views_frames_summarize._common import AnyFrame
14
14
  from views_frames_summarize.bimodality import bimodality
15
15
  from views_frames_summarize.collapse import collapse
16
+ from views_frames_summarize.exceedance import exceedance
17
+ from views_frames_summarize.expected_shortfall import expected_shortfall
16
18
  from views_frames_summarize.interval import hdi, quantiles
17
19
  from views_frames_summarize.point import map_estimate
18
20
  from views_frames_summarize.summarize_tower import summarize_tower
@@ -44,6 +46,36 @@ def assert_summarizer_contract(frame: AnyFrame) -> None:
44
46
  assert qs.shape[0] == n, "quantiles must be aligned to the frame's rows"
45
47
  assert qs.shape[-1] == 3, "quantiles must produce one column per quantile"
46
48
 
49
+ # exceedance laws (ADR-021): a survival function — in [0, 1], non-increasing in the
50
+ # threshold, with P(> -inf) = 1 and P(> +inf) = 0.
51
+ thresholds = [-np.inf, 0.5, 1.5, np.inf]
52
+ exc = exceedance(frame, thresholds)
53
+ assert exc.shape[0] == n, "exceedance must be aligned to the frame's rows"
54
+ assert exc.shape[-1] == len(thresholds), "exceedance → one column per threshold"
55
+ assert ((exc >= 0.0) & (exc <= 1.0)).all(), (
56
+ "exceedance must be a probability in [0, 1]"
57
+ )
58
+ assert (np.diff(exc, axis=-1) <= 1e-6).all(), (
59
+ "exceedance must be non-increasing in the threshold"
60
+ )
61
+ assert (exc[..., 0] == 1.0).all(), "P(> -inf) must be 1"
62
+ assert (exc[..., -1] == 0.0).all(), "P(> +inf) must be 0"
63
+
64
+ # expected-shortfall laws (ADR-022): a tail mean — in [min, max], non-decreasing as
65
+ # the tail deepens (t → 0), and dominating its VaR (ES(t) ≥ the (1 − t) quantile).
66
+ tails = [1.0, 0.5, 0.1] # widening tails (deepening worst-case)
67
+ es = expected_shortfall(frame, tails)
68
+ lo = frame.values.min(axis=-1)[..., np.newaxis]
69
+ hi = frame.values.max(axis=-1)[..., np.newaxis]
70
+ assert es.shape[0] == n, "expected_shortfall must be aligned to the frame's rows"
71
+ assert es.shape[-1] == len(tails), "expected_shortfall → one column per tail"
72
+ assert ((es >= lo - 1e-6) & (es <= hi + 1e-6)).all(), "ES must lie in [min, max]"
73
+ assert (np.diff(es, axis=-1) >= -1e-6).all(), (
74
+ "ES must be non-decreasing as the tail deepens"
75
+ )
76
+ var = quantiles(frame, [0.0, 0.5, 0.9]) # the (1 − t) quantiles for t = 1, 0.5, 0.1
77
+ assert (es >= var - 1e-6).all(), "ES(t) must dominate the (1 − t) quantile"
78
+
47
79
  _assert_tower_contract(frame, n)
48
80
 
49
81
 
@@ -0,0 +1,96 @@
1
+ """Threshold exceedance probabilities over the sample axis (ADR-021).
2
+
3
+ `exceedance(frame, thresholds)` returns the per-row empirical **survival fraction**
4
+ `P(Y > c)` for each threshold — an index-aligned array `(N, …, K)`, the same shape/role
5
+ family as `quantiles`. `exceedance_reducer(c)` is a `collapse`-compatible factory for
6
+ the single-threshold `(N, …, 1)` **frame** path.
7
+
8
+ Distribution-agnostic (a counting reducer — no histogram, no config, no unimodality
9
+ assumption), so it is robust where the tower is weakest (register C-34); the flagship
10
+ is `P(Y > 0)` = probability of any activity (onset). Thresholds are **required**
11
+ per-call, in the frame's own units (an *input*, like `quantiles`' ``qs`` — never a
12
+ tunable or a default). **Strict `>`** (register D-08). Fails loud on any non-finite
13
+ draw — NaN or ±inf (register C-50) — and on empty thresholds; the joint-sample
14
+ requirement for aggregate exceedance is the consumer's (C-49).
15
+ """
16
+
17
+ from __future__ import annotations
18
+
19
+ from collections.abc import Sequence
20
+
21
+ import numpy as np
22
+ from numpy.typing import NDArray
23
+
24
+ from views_frames_summarize._common import ROW_BLOCK, AnyFrame, block_apply
25
+ from views_frames_summarize.collapse import Reducer
26
+
27
+
28
+ def _exceed(
29
+ values: NDArray[np.float32], thresholds: NDArray[np.float64]
30
+ ) -> NDArray[np.float32]:
31
+ """Fraction of the trailing (sample) axis **strictly** greater than each threshold.
32
+
33
+ ``values`` is ``(…, S)`` and ``thresholds`` is ``(K,)``; returns ``(…, K)``.
34
+
35
+ Fails loud on any **non-finite** draw (NaN or ±inf): ``NaN > c`` is silently
36
+ ``False``, so a naive count would deflate ``P(Y > c)`` — worst on the ``P(Y > 0)``
37
+ onset flagship; and an ±inf draw (always an upstream bug) silently blesses ``P``
38
+ as a valid probability, masking the bug. A draw must be finite (C-50).
39
+ """
40
+ if not bool(np.isfinite(values).all()):
41
+ raise ValueError(
42
+ "exceedance is undefined on non-finite draws (NaN or ±inf); strip or "
43
+ "impute upstream (NaN > c is silently False and would deflate P(Y > c); "
44
+ "an inf draw masks an upstream bug behind a valid-looking P; register C-50)"
45
+ )
46
+ # (…, S) vs (K,) → (…, K, S) by broadcasting; strict ">"; mean over the sample axis.
47
+ above = values[..., np.newaxis, :] > thresholds[:, np.newaxis]
48
+ return np.asarray(above.mean(axis=-1), dtype=np.float32)
49
+
50
+
51
+ def exceedance(
52
+ frame: AnyFrame,
53
+ thresholds: Sequence[float],
54
+ *,
55
+ block_rows: int = ROW_BLOCK,
56
+ ) -> NDArray[np.float32]:
57
+ """Per-row exceedance probability `P(Y > c)` over the sample axis → `(N, …, K)`.
58
+
59
+ For each of the ``K`` caller-supplied ``thresholds`` (required, in the frame's own
60
+ units), the empirical fraction of draws **strictly** greater than it. An
61
+ index-aligned numpy array (the caller holds the index), vectorized in row-blocks of
62
+ ``block_rows`` so peak memory is bounded by one block's working set (register C-25).
63
+
64
+ Raises:
65
+ ValueError: ``thresholds`` is empty (no default), or any draw is non-finite —
66
+ NaN or ±inf (C-50).
67
+ """
68
+ thr = np.asarray(thresholds, dtype=np.float64)
69
+ if thr.size == 0:
70
+ raise ValueError(
71
+ "exceedance requires at least one threshold; there is no default "
72
+ "(thresholds are an input in the frame's units, not a tunable — ADR-021)."
73
+ )
74
+
75
+ def _block(vals: NDArray[np.float32]) -> NDArray[np.float32]:
76
+ return _exceed(vals, thr)
77
+
78
+ out = block_apply(frame.values, block_rows, _block)
79
+ return np.asarray(out, dtype=np.float32)
80
+
81
+
82
+ def exceedance_reducer(threshold: float) -> Reducer:
83
+ """A `collapse`-compatible reducer for single-threshold exceedance.
84
+
85
+ ``collapse(frame, exceedance_reducer(c))`` returns `P(Y > c)` as a `(N, …, 1)`
86
+ frame, sharing `exceedance`'s strict-`>` and fail-loud-non-finite policy (so a
87
+ consumer never re-implements ``np.mean(values > c)`` and drifts on the convention).
88
+ """
89
+ thr = np.asarray([threshold], dtype=np.float64)
90
+
91
+ def _reducer(values: NDArray[np.float32], axis: int = -1) -> NDArray[np.float32]:
92
+ # `collapse` calls reducer(values, axis=-1) and appends the trailing axis
93
+ # itself, so drop the singleton K axis (collapse re-adds the sample-axis slot).
94
+ return _exceed(values, thr)[..., 0]
95
+
96
+ return _reducer
@@ -0,0 +1,88 @@
1
+ """Worst-case scenario summary over the sample axis (ADR-022).
2
+
3
+ `expected_shortfall(frame, tails)` returns the per-row **mean of the worst `⌈t·S⌉`
4
+ draws** for each upper-tail fraction `t` — the tail mean / CVaR, a coherent worst-case
5
+ risk measure and the companion to `exceedance`. An index-aligned array `(N, …, K)`, the
6
+ same shape/role family as `quantiles`.
7
+
8
+ `max` is deliberately **not** offered — it is the highest-variance, non-reproducible
9
+ summary this replaces. Tails are **required** per-call, in `(0, 1]` (e.g. `0.01` = "the
10
+ worst 1%") — the consumer's policy, never a default or config. Fails loud on any
11
+ non-finite draw — NaN or ±inf (numpy sorts them *last*, so a naive top-`k` mean silently
12
+ selects them — register C-56), on empty `tails`, and on any `t` outside `(0, 1]`.
13
+
14
+ Written explicitly in its own module — it does **not** share a "tail reducer"
15
+ abstraction with `quantiles`/`exceedance` (the duplication is shallow and the concerns
16
+ change independently — WET before DRY). It reuses only the stable `_common` primitives.
17
+ """
18
+
19
+ from __future__ import annotations
20
+
21
+ from collections.abc import Sequence
22
+
23
+ import numpy as np
24
+ from numpy.typing import NDArray
25
+
26
+ from views_frames_summarize._common import ROW_BLOCK, AnyFrame, block_apply
27
+
28
+
29
+ def _expected_shortfall(
30
+ values: NDArray[np.float32], tails: NDArray[np.float64]
31
+ ) -> NDArray[np.float32]:
32
+ """Mean of the worst ``⌈t·S⌉`` draws of the trailing (sample) axis, per tail.
33
+
34
+ ``values`` is ``(…, S)``, ``tails`` is ``(K,)`` in ``(0, 1]``; returns ``(…, K)``.
35
+
36
+ Fails loud on any **non-finite** draw (NaN or ±inf): numpy sorts NaN *last*, so a
37
+ top-`k` mean would silently select the NaNs — a NaN/garbage worst-case; and an ±inf
38
+ draw (always an upstream bug) sorts last and contaminates the tail mean to ±inf — a
39
+ degenerate "worst case". A draw must be a usable finite number (register C-56).
40
+ """
41
+ if not bool(np.isfinite(values).all()):
42
+ raise ValueError(
43
+ "expected_shortfall is undefined on non-finite draws (NaN or ±inf); strip "
44
+ "or impute upstream (numpy sorts NaN/+inf last, so the worst-k mean would "
45
+ "silently select them — a NaN/inf worst-case; register C-56)"
46
+ )
47
+ s = values.shape[-1]
48
+ srt = np.sort(values, axis=-1) # ascending: the worst (largest) draws are last
49
+ out = np.empty((*values.shape[:-1], tails.shape[0]), dtype=np.float32)
50
+ for i in range(tails.shape[0]):
51
+ k = int(np.ceil(float(tails[i]) * s)) # number of worst draws to average (≥ 1)
52
+ out[..., i] = srt[..., s - k :].mean(axis=-1)
53
+ return out
54
+
55
+
56
+ def expected_shortfall(
57
+ frame: AnyFrame,
58
+ tails: Sequence[float],
59
+ *,
60
+ block_rows: int = ROW_BLOCK,
61
+ ) -> NDArray[np.float32]:
62
+ """Per-row worst-case **expected shortfall** over the sample axis → `(N, …, K)`.
63
+
64
+ For each upper-tail fraction ``t`` in ``tails`` (required, in ``(0, 1]``), the mean
65
+ of the worst ``⌈t·S⌉`` draws — the average of the worst-case scenarios. An
66
+ index-aligned numpy array (the caller holds the index), block-applied in row-blocks.
67
+
68
+ Raises:
69
+ ValueError: ``tails`` is empty (no default), any ``t ∉ (0, 1]`` (a tail with no
70
+ samples), or any draw is non-finite — NaN or ±inf (register C-56).
71
+ """
72
+ thr = np.asarray(tails, dtype=np.float64)
73
+ if thr.size == 0:
74
+ raise ValueError(
75
+ "expected_shortfall requires at least one tail level; there is no default "
76
+ "(the tail fraction is an input in (0, 1], not a tunable — ADR-022)."
77
+ )
78
+ if bool(((thr <= 0.0) | (thr > 1.0)).any()):
79
+ raise ValueError(
80
+ f"expected_shortfall tails must be in (0, 1]; got {thr.tolist()} "
81
+ "(a tail <= 0 contains no samples; > 1 is not a fraction)."
82
+ )
83
+
84
+ def _block(vals: NDArray[np.float32]) -> NDArray[np.float32]:
85
+ return _expected_shortfall(vals, thr)
86
+
87
+ out = block_apply(frame.values, block_rows, _block)
88
+ return np.asarray(out, dtype=np.float32)
@@ -13,9 +13,10 @@ nearest floor, never inserted — "the 50% HDI" is identical regardless of which
13
13
  masses a caller asks for (reproducibility).
14
14
 
15
15
  The construction is vectorized over the sample axis and runs in row-blocks (register
16
- C-22/C-25): one sort per block, never a whole-grid sorted copy. A raw-count zero
17
- short-circuit (`max(row) <= zero_cutoff` the whole summary collapses to 0) kills the
18
- quiet cells cheaply.
16
+ C-22/C-25): one sort per block, never a whole-grid sorted copy. Zero-inflation is read
17
+ off the *density* of the `tip_mass` floor (a zero-majority row reads 0); an
18
+ **optional, off-by-default** magnitude cutoff (`config['zero_cutoff']`, a count-domain
19
+ opt-in) can additionally force `max(row) <= cutoff` rows to 0 (C-45).
19
20
 
20
21
  All tunables (the grid, zero cutoff, row-block size) come from `config` with **no silent
21
22
  defaults** (ADR-009). The private engine (`_dense_tower`, `_median_in`, `_pin`,
@@ -34,11 +35,10 @@ from numpy.typing import NDArray
34
35
  from views_frames_summarize import config
35
36
  from views_frames_summarize._common import AnyFrame, block_apply
36
37
 
37
- # The fixed canonical mass grid + the zero cutoff, sourced from the single config (no
38
- # silent defaults — ADR-009). Kept as module names so the engine and its siblings share
39
- # one source of truth.
38
+ # The fixed canonical mass grid, sourced from the single config (no silent defaults —
39
+ # ADR-009). The grid is reproducibility-fixed by design (ADR-019), so it is snapshotted
40
+ # here; the *tunable* knobs (``zero_cutoff``, ``tip_mass``) are read live at call time.
40
41
  _CANONICAL_FLOORS: Final[NDArray[np.float64]] = config.canonical_floors()
41
- _ZERO_CUTOFF: Final[float] = float(config.get("zero_cutoff"))
42
42
 
43
43
 
44
44
  def _ks(sample_count: int) -> NDArray[np.intp]:
@@ -65,8 +65,18 @@ def _pin(masses: Sequence[float]) -> NDArray[np.intp]:
65
65
 
66
66
 
67
67
  def _zero_mask(block: NDArray[np.float32]) -> NDArray[np.bool_]:
68
- """Rows whose maximum draw is at/below the zero cutoff the quiet cells."""
69
- return np.asarray(block.max(axis=-1) <= _ZERO_CUTOFF, dtype=np.bool_)
68
+ """Rows the optional magnitude cutoff zeroes``max(draws) <= zero_cutoff``.
69
+
70
+ **Off by default** (``zero_cutoff is None`` ⇒ no row is masked). A consumer that
71
+ wants magnitude zeroing (a count target where a sub-1 posterior should read 0) sets
72
+ ``zero_cutoff`` to a float in ``config`` — read **live** here, so the knob takes
73
+ effect at runtime. By default the leaf imposes **no** magnitude assumption; zero-
74
+ inflation is handled by the density of the ``tip_mass`` floor instead (C-45).
75
+ """
76
+ cutoff = config.get("zero_cutoff")
77
+ if cutoff is None:
78
+ return np.zeros(block.shape[0], dtype=np.bool_)
79
+ return np.asarray(block.max(axis=-1) <= cutoff, dtype=np.bool_)
70
80
 
71
81
 
72
82
  def _in_range_span(
@@ -203,8 +213,9 @@ def hdi_tower(
203
213
  Each requested mass is pinned to the nearest fixed canonical floor; the interval is
204
214
  read out of the full canonical tower (built once per block, outside-in). Nested by
205
215
  construction, robust to minority duplicates, and reproducible (a mass's interval is
206
- independent of the other requested masses). Quiet rows (``max <= zero_cutoff``)
207
- collapse to ``(0, 0)``. Aligned to ``frame.index``. Tunables come from ``config``.
216
+ independent of the other requested masses). If the optional ``zero_cutoff`` is set,
217
+ ``max <= cutoff`` rows collapse to ``(0, 0)`` (off by default C-45). Aligned to
218
+ ``frame.index``. Tunables come from ``config``.
208
219
  """
209
220
  values = frame.values
210
221
  lead = values.shape[:-1]
@@ -2,8 +2,8 @@
2
2
 
3
3
  `tower_point` is the "most likely single value" we report to a consumer: the median of
4
4
  the draws inside the **`tip_mass` floor** of the nested tower (config-driven, default
5
- 0.5 — the "shorth"), with the same raw-count zero short-circuit as the tower
6
- (`max(row) <= zero_cutoff` 0).
5
+ 0.5 — the "shorth"). Zero-inflation is handled by that floor's density (a zero-majority
6
+ row reads 0); the optional, off-by-default magnitude cutoff applies too if set (C-45).
7
7
 
8
8
  It is a *new* estimator, deliberately distinct from the frozen `map_estimate` (ADR-018):
9
9
  `map_estimate` is a binned histogram mode with a zero-*mass*-fraction rule and a
@@ -34,9 +34,10 @@ from views_frames_summarize.tower import (
34
34
  def tower_point(frame: AnyFrame) -> AnyFrame:
35
35
  """Per-row tower-tip point estimate over the sample axis → a `(N, …, 1)` frame.
36
36
 
37
- The median of the ``tip_mass`` floor's samples; ``0.0`` for quiet rows (every draw
38
- ``<= zero_cutoff``). Returns a frame of the same type as ``frame``. Tunables (the
39
- ``tip_mass``, the grid, the zero cutoff, the row-block) come from ``config``.
37
+ The median of the ``tip_mass`` floor's samples (``0`` when the zero atom dominates,
38
+ by density); if the optional ``zero_cutoff`` is set, ``max <= cutoff`` rows are
39
+ forced to ``0`` (off by default C-45). Returns the same frame type as ``frame``.
40
+ Tunables (``tip_mass``, grid, ``zero_cutoff``, row-block) come from ``config``.
40
41
  """
41
42
  values = frame.values
42
43
  lead = values.shape[:-1]
File without changes
File without changes