dwabi 0.1.1.post1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,6 @@
1
+ __pycache__/
2
+ .pytest_cache/
3
+ .ruff_cache/
4
+ .venv/
5
+ dist/
6
+ /benchmarks/
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Abishek Sriramulu
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,107 @@
1
+ Metadata-Version: 2.4
2
+ Name: dwabi
3
+ Version: 0.1.1.post1
4
+ Summary: Walk-forward quantile ensembles of user-supplied forecasters
5
+ Project-URL: Source, https://github.com/AbishekSriramulu/dwabi
6
+ Project-URL: Results, https://github.com/AbishekSriramulu/dwabi/blob/main/results/fev/README.md
7
+ Project-URL: Issues, https://github.com/AbishekSriramulu/dwabi/issues
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Requires-Python: >=3.11
11
+ Requires-Dist: numpy>=1.26
12
+ Requires-Dist: polars>=1.25
13
+ Requires-Dist: utilsforecast>=0.2.17
14
+ Provides-Extra: fast
15
+ Requires-Dist: numba>=0.63.1; extra == 'fast'
16
+ Description-Content-Type: text/markdown
17
+
18
+ # D-wabi
19
+
20
+ D-wabi combines a user-supplied roster of at least three forecasters. It estimates forecast combinations from earlier issues whose outcomes are available and shares the fitted decision with the caller. Each member supplies quantiles on the same forecast cases.
21
+
22
+ Install with `pip install dwabi`. The FEV-Bench application ranks **first of 36 by overall SQL win rate (89.54%)** with systems enabled and closed APIs excluded. SQL skill ranks fourth. Including closed APIs brings the ranks to second and seventh of 41. Retail ranks second by SQL win rate in both fields. These are local insertions into the pinned published comparison. The package accepts your own model roster. [Results, method and reproduction](https://github.com/AbishekSriramulu/dwabi/blob/main/results/fev/README.md)
23
+
24
+ Uses ci-proto, released by Continuous Intelligence. Continuous Intelligence does not endorse or approve this product or service. The D-wabi Python code is MIT licensed. Each forecasting model has its own terms. [Benchmark licenses and attribution](https://github.com/AbishekSriramulu/dwabi/blob/main/results/fev/CI_PROTO_LICENSE.txt)
25
+
26
+ ## Fit a panel
27
+
28
+ ```python
29
+ from dwabi import DWabi
30
+
31
+ ensemble = DWabi(models, h=7).fit(train)
32
+ forecast = ensemble.predict()
33
+ print(ensemble.decision_)
34
+ print(ensemble.validation_windows_)
35
+ print(ensemble.evaluate(test))
36
+ ```
37
+
38
+ `train` and `test` are Polars DataFrames with `unique_id`, `ds` and `y` columns. The current panel interface supports a shared cutoff and a regular time grid. Supply an explicit frequency such as `"1d"` when the cadence cannot be inferred. `models` maps names to callables. Each receives a `ForecastRequest` and returns its forecasts keyed by series and timestamp, with quantile columns named `q0.1` through `q0.9` by default.
39
+
40
+ Pass `validation=validation` to use a designated later segment. With either input, fitting issues historical forecasts from growing prefixes. D-wabi uses only earlier complete windows for each selection decision. `n_windows` controls the backtest budget. Twelve windows is a development default. Each member still receives the full observed prefix at each issue. Forecasts use the full equal pool until three usable windows support a shortlist.
41
+
42
+ Declare genuinely known-future predictors in `known_future` and provide their issue-time values to `predict(future=...)`. The panel interface uses timestamps as the label-availability clock. Use the issued-forecast interface below for publication delays or separately recorded feature vintages. `evaluate(test)` reports held error with the fitted blend and preserves its decision.
43
+
44
+ ### WAPE point forecasts
45
+
46
+ Use `DWabi(models, h=7, objective="wape")` to fit a point forecast for weighted absolute percentage error. Each member supplies the nine deciles and can include a separate `point` column. When that column is absent, its median supplies the member point. D-wabi learns ordinal voting weights from completed, disjoint earlier windows. With at least four fitting windows, the optimizer minimizes the largest average increase over the full-roster median after deleting one fitting window. With three it minimizes mean WAPE, and with less evidence it starts from equal influence.
47
+
48
+ The WAPE point action treats the outcome in one forecast cell as uncertain, approximates the remaining absolute volume in its target channel by its predicted expectation, and takes the median under inverse-volume weighting. It projects the result to nonnegative or integer support when the visible target history supplies that evidence. The returned nine quantiles are the full-roster quantile medians. WAPE and quantile forecasts have separately defined actions.
49
+
50
+ The `fast` extra enables Numba for large-panel WAPE computations. For several panels, use `PointHistory` to register matched point arrays and native loss weights with explicit issue, first-target and availability clocks. Assign the same physical family key to panels derived from the same source. `freeze(task, family, issue_at=...)` returns the ordinal weights, a proposed peer subset and its comparison history. A peer subset can replace the ordinal point only after at least three mature, disjoint own-task comparisons favor it even after deleting any one comparison. Each peer fit excludes the evaluated family and uses earlier available outcomes. A single panel uses the ordinal action.
51
+
52
+ ## Issue at staggered cutoffs
53
+
54
+ `CutoffEnsemble` serves the WAPE point and a separate nine-decile median pool. Supply one `ForecastSeries(issue_at, history_times, history, future_times)` per recipient. The history array is shaped `(time, target)`. All clocks use the same integer time axis. Pass historical-only covariates through `past` and planned covariates through `known`. Historical-only values cover the history clock and known values cover both clocks. The ensemble sends each model only the recipients sharing the same actual cutoff. A model returns an array shaped `(recipient, lead, target, 10)` with the point followed by the nine deciles, or nine deciles to use its median as the point.
55
+
56
+ ```python
57
+ from dwabi import CutoffEnsemble, ForecastBudget, PointHistory, QuantileHistory
58
+
59
+ ensemble = CutoffEnsemble(models, supplemental="extra",
60
+ budget=ForecastBudget(context_limit=16_384, max_cells=2_000_000))
61
+ point_history = PointHistory(ensemble.models)
62
+ quantile_history = QuantileHistory(ensemble.models, supplemental=ensemble.supplemental)
63
+
64
+ # Register matched earlier point evidence and probabilistic losses on these histories.
65
+ decision = ensemble.freeze(point_history, quantile_history,
66
+ task="panel", family="source", issue_at=earliest_cutoff)
67
+ forecast = ensemble.predict(series, decision=decision)
68
+ print(forecast.point, forecast.quantiles, forecast.routes)
69
+ ```
70
+
71
+ `QuantileHistory.observe(losses, origin=..., issued_at=..., first_target_at=..., available_at=...)` takes one loss per original model on the same cases and metric, plus the supplemental loss when its earlier forecast was issued. After three strictly mature disjoint comparisons, the fallback pools the three best original models by mean loss. It uses the full original median before that. With a configured supplemental model, three common comparisons allow a shortlist across both rosters. A cohort qualifies when the declared input capacity covers its history and lead cells. Only selected supplemental models run at prediction. The point always uses the original roster. Omit both `supplemental` and `budget` for an original-only system.
72
+
73
+ Use `predict(series, decision=...)` to replay the same fixed decision at each cutoff. Use `predict_online(series, point_history=..., quantile_history=..., task=..., family=...)` to freeze from each cohort's available history. Register completed evidence separately. Both methods return forecasts in the original recipient order and expose the cohort routes.
74
+
75
+ ## Register issued forecasts
76
+
77
+ ```python
78
+ from dwabi import WalkForwardBlend
79
+
80
+ ensemble = WalkForwardBlend({
81
+ "model-a": model_a,
82
+ "model-b": model_b,
83
+ "model-c": model_c,
84
+ "model-d": model_d,
85
+ })
86
+
87
+ # Each callable receives the same context as it stood at issuance.
88
+ issued = ensemble.predict(context, issue_at=10)
89
+ print(issued.point, issued.decision)
90
+
91
+ # When the complete target window has arrived.
92
+ ensemble.observe(
93
+ issued.members, actual, origin="window-1",
94
+ issued_at=10, first_target_at=11, available_at=17,
95
+ )
96
+ next_forecast = ensemble.predict(next_context, issue_at=18)
97
+ ```
98
+
99
+ Use integer clocks on a common timeline. Use one instance per task. Align recipient, target and lead coordinates before passing a forecast to the ensemble. The last axis is the configured quantile grid. The combined point is the combined 0.5 quantile.
100
+
101
+ For warmup, provide a few earlier issued backcasts to `observe` using their original forecasts and outcome availability. The first three admitted windows must have nonoverlapping target windows. D-wabi uses the equal full-roster average before enough evidence is available. When earlier three-model averages have different scores, the joint selector chooses the one with the best average relative loss. An individual-risk selector is also available. Weight fitting uses the one-sixth simplex grid with three windows of shrinkage toward equal weights within the chosen set. Use `weighted=False` for equal weights and `lookback=3` to restrict both fits to the latest three admitted windows.
102
+
103
+ The default objective is pinball loss across all supplied levels. `objective="absolute"` fits the median with absolute loss. Optional `case_weights` let callers match the scoring scale and aggregation. Each admitted window contributes loss relative to the equal full-roster forecast for member selection. The weight fit normalizes against the selected triple's own equal average.
104
+
105
+ Keep `full_roster=True` while collecting forecasts for future updates. Use `full_roster=False` to run the current selected members only. Existing backcast forecasts can be supplied directly to `observe`. `explain(issue_at)` returns the subset, weights and supporting origins without model calls.
106
+
107
+ The benchmark adapter fits the point policy from earlier FEV data and evaluates it on the complete later benchmark. The library also accepts independent, user-supplied forecasts. Model weights and evaluation data are downloaded separately for benchmark reproduction under their original terms.
@@ -0,0 +1,90 @@
1
+ # D-wabi
2
+
3
+ D-wabi combines a user-supplied roster of at least three forecasters. It estimates forecast combinations from earlier issues whose outcomes are available and shares the fitted decision with the caller. Each member supplies quantiles on the same forecast cases.
4
+
5
+ Install with `pip install dwabi`. The FEV-Bench application ranks **first of 36 by overall SQL win rate (89.54%)** with systems enabled and closed APIs excluded. SQL skill ranks fourth. Including closed APIs brings the ranks to second and seventh of 41. Retail ranks second by SQL win rate in both fields. These are local insertions into the pinned published comparison. The package accepts your own model roster. [Results, method and reproduction](https://github.com/AbishekSriramulu/dwabi/blob/main/results/fev/README.md)
6
+
7
+ Uses ci-proto, released by Continuous Intelligence. Continuous Intelligence does not endorse or approve this product or service. The D-wabi Python code is MIT licensed. Each forecasting model has its own terms. [Benchmark licenses and attribution](https://github.com/AbishekSriramulu/dwabi/blob/main/results/fev/CI_PROTO_LICENSE.txt)
8
+
9
+ ## Fit a panel
10
+
11
+ ```python
12
+ from dwabi import DWabi
13
+
14
+ ensemble = DWabi(models, h=7).fit(train)
15
+ forecast = ensemble.predict()
16
+ print(ensemble.decision_)
17
+ print(ensemble.validation_windows_)
18
+ print(ensemble.evaluate(test))
19
+ ```
20
+
21
+ `train` and `test` are Polars DataFrames with `unique_id`, `ds` and `y` columns. The current panel interface supports a shared cutoff and a regular time grid. Supply an explicit frequency such as `"1d"` when the cadence cannot be inferred. `models` maps names to callables. Each receives a `ForecastRequest` and returns its forecasts keyed by series and timestamp, with quantile columns named `q0.1` through `q0.9` by default.
22
+
23
+ Pass `validation=validation` to use a designated later segment. With either input, fitting issues historical forecasts from growing prefixes. D-wabi uses only earlier complete windows for each selection decision. `n_windows` controls the backtest budget. Twelve windows is a development default. Each member still receives the full observed prefix at each issue. Forecasts use the full equal pool until three usable windows support a shortlist.
24
+
25
+ Declare genuinely known-future predictors in `known_future` and provide their issue-time values to `predict(future=...)`. The panel interface uses timestamps as the label-availability clock. Use the issued-forecast interface below for publication delays or separately recorded feature vintages. `evaluate(test)` reports held error with the fitted blend and preserves its decision.
26
+
27
+ ### WAPE point forecasts
28
+
29
+ Use `DWabi(models, h=7, objective="wape")` to fit a point forecast for weighted absolute percentage error. Each member supplies the nine deciles and can include a separate `point` column. When that column is absent, its median supplies the member point. D-wabi learns ordinal voting weights from completed, disjoint earlier windows. With at least four fitting windows, the optimizer minimizes the largest average increase over the full-roster median after deleting one fitting window. With three it minimizes mean WAPE, and with less evidence it starts from equal influence.
30
+
31
+ The WAPE point action treats the outcome in one forecast cell as uncertain, approximates the remaining absolute volume in its target channel by its predicted expectation, and takes the median under inverse-volume weighting. It projects the result to nonnegative or integer support when the visible target history supplies that evidence. The returned nine quantiles are the full-roster quantile medians. WAPE and quantile forecasts have separately defined actions.
32
+
33
+ The `fast` extra enables Numba for large-panel WAPE computations. For several panels, use `PointHistory` to register matched point arrays and native loss weights with explicit issue, first-target and availability clocks. Assign the same physical family key to panels derived from the same source. `freeze(task, family, issue_at=...)` returns the ordinal weights, a proposed peer subset and its comparison history. A peer subset can replace the ordinal point only after at least three mature, disjoint own-task comparisons favor it even after deleting any one comparison. Each peer fit excludes the evaluated family and uses earlier available outcomes. A single panel uses the ordinal action.
34
+
35
+ ## Issue at staggered cutoffs
36
+
37
+ `CutoffEnsemble` serves the WAPE point and a separate nine-decile median pool. Supply one `ForecastSeries(issue_at, history_times, history, future_times)` per recipient. The history array is shaped `(time, target)`. All clocks use the same integer time axis. Pass historical-only covariates through `past` and planned covariates through `known`. Historical-only values cover the history clock and known values cover both clocks. The ensemble sends each model only the recipients sharing the same actual cutoff. A model returns an array shaped `(recipient, lead, target, 10)` with the point followed by the nine deciles, or nine deciles to use its median as the point.
38
+
39
+ ```python
40
+ from dwabi import CutoffEnsemble, ForecastBudget, PointHistory, QuantileHistory
41
+
42
+ ensemble = CutoffEnsemble(models, supplemental="extra",
43
+ budget=ForecastBudget(context_limit=16_384, max_cells=2_000_000))
44
+ point_history = PointHistory(ensemble.models)
45
+ quantile_history = QuantileHistory(ensemble.models, supplemental=ensemble.supplemental)
46
+
47
+ # Register matched earlier point evidence and probabilistic losses on these histories.
48
+ decision = ensemble.freeze(point_history, quantile_history,
49
+ task="panel", family="source", issue_at=earliest_cutoff)
50
+ forecast = ensemble.predict(series, decision=decision)
51
+ print(forecast.point, forecast.quantiles, forecast.routes)
52
+ ```
53
+
54
+ `QuantileHistory.observe(losses, origin=..., issued_at=..., first_target_at=..., available_at=...)` takes one loss per original model on the same cases and metric, plus the supplemental loss when its earlier forecast was issued. After three strictly mature disjoint comparisons, the fallback pools the three best original models by mean loss. It uses the full original median before that. With a configured supplemental model, three common comparisons allow a shortlist across both rosters. A cohort qualifies when the declared input capacity covers its history and lead cells. Only selected supplemental models run at prediction. The point always uses the original roster. Omit both `supplemental` and `budget` for an original-only system.
55
+
56
+ Use `predict(series, decision=...)` to replay the same fixed decision at each cutoff. Use `predict_online(series, point_history=..., quantile_history=..., task=..., family=...)` to freeze from each cohort's available history. Register completed evidence separately. Both methods return forecasts in the original recipient order and expose the cohort routes.
57
+
58
+ ## Register issued forecasts
59
+
60
+ ```python
61
+ from dwabi import WalkForwardBlend
62
+
63
+ ensemble = WalkForwardBlend({
64
+ "model-a": model_a,
65
+ "model-b": model_b,
66
+ "model-c": model_c,
67
+ "model-d": model_d,
68
+ })
69
+
70
+ # Each callable receives the same context as it stood at issuance.
71
+ issued = ensemble.predict(context, issue_at=10)
72
+ print(issued.point, issued.decision)
73
+
74
+ # When the complete target window has arrived.
75
+ ensemble.observe(
76
+ issued.members, actual, origin="window-1",
77
+ issued_at=10, first_target_at=11, available_at=17,
78
+ )
79
+ next_forecast = ensemble.predict(next_context, issue_at=18)
80
+ ```
81
+
82
+ Use integer clocks on a common timeline. Use one instance per task. Align recipient, target and lead coordinates before passing a forecast to the ensemble. The last axis is the configured quantile grid. The combined point is the combined 0.5 quantile.
83
+
84
+ For warmup, provide a few earlier issued backcasts to `observe` using their original forecasts and outcome availability. The first three admitted windows must have nonoverlapping target windows. D-wabi uses the equal full-roster average before enough evidence is available. When earlier three-model averages have different scores, the joint selector chooses the one with the best average relative loss. An individual-risk selector is also available. Weight fitting uses the one-sixth simplex grid with three windows of shrinkage toward equal weights within the chosen set. Use `weighted=False` for equal weights and `lookback=3` to restrict both fits to the latest three admitted windows.
85
+
86
+ The default objective is pinball loss across all supplied levels. `objective="absolute"` fits the median with absolute loss. Optional `case_weights` let callers match the scoring scale and aggregation. Each admitted window contributes loss relative to the equal full-roster forecast for member selection. The weight fit normalizes against the selected triple's own equal average.
87
+
88
+ Keep `full_roster=True` while collecting forecasts for future updates. Use `full_roster=False` to run the current selected members only. Existing backcast forecasts can be supplied directly to `observe`. `explain(issue_at)` returns the subset, weights and supporting origins without model calls.
89
+
90
+ The benchmark adapter fits the point policy from earlier FEV data and evaluates it on the complete later benchmark. The library also accepts independent, user-supplied forecasts. Model weights and evaluation data are downloaded separately for benchmark reproduction under their original terms.
@@ -0,0 +1,27 @@
1
+ [build-system]
2
+ requires = ["hatchling>=1.27"]
3
+ build-backend = "hatchling.build"
4
+
5
+ [project]
6
+ name = "dwabi"
7
+ version = "0.1.1.post1"
8
+ description = "Walk-forward quantile ensembles of user-supplied forecasters"
9
+ readme = "README.md"
10
+ license = "MIT"
11
+ license-files = ["LICENSE"]
12
+ requires-python = ">=3.11"
13
+ dependencies = ["numpy>=1.26", "polars>=1.25", "utilsforecast>=0.2.17"]
14
+
15
+ [project.urls]
16
+ Source = "https://github.com/AbishekSriramulu/dwabi"
17
+ Results = "https://github.com/AbishekSriramulu/dwabi/blob/main/results/fev/README.md"
18
+ Issues = "https://github.com/AbishekSriramulu/dwabi/issues"
19
+
20
+ [project.optional-dependencies]
21
+ fast = ["numba>=0.63.1"]
22
+
23
+ [tool.hatch.build.targets.wheel]
24
+ packages = ["src/dwabi"]
25
+
26
+ [tool.hatch.build.targets.sdist]
27
+ include = ["src/dwabi", "README.md", "pyproject.toml", "LICENSE"]
@@ -0,0 +1,46 @@
1
+ # D-wabi 0.1.1 on FEV-Bench
2
+
3
+ Across all 100 FEV-Bench tasks, D-wabi achieved **89.54% SQL win rate, first of 36**, when evaluated with systems enabled and closed APIs excluded. Its **SQL skill score was 50.25%, fourth of 36**. Including closed APIs gives second of 41 by win rate and seventh by skill. These are independently reproduced comparisons with the published results at FEV revision `844509e`. Acceptance and display on the upstream leaderboard are controlled by the FEV maintainers. [Published field](https://github.com/autogluon/fev/tree/844509e5211292620fbd9d5f24695f1d27da767a/benchmarks/fev_bench) · [Live leaderboard](https://huggingface.co/spaces/autogluon/fev-bench)
4
+
5
+ Uses ci-proto, released by Continuous Intelligence. Continuous Intelligence does not endorse or approve this product or service. [License](CI_PROTO_LICENSE.txt)
6
+
7
+ ## Results
8
+
9
+ The full panel contains 1,308 forecast windows and 8,926,250 observed lead, series and target combinations. Retail is the benchmark’s complete 20-task domain. Win rate averages head-to-head comparisons over tasks and competing models, awarding half credit for a tie. Skill uses geometric mean loss relative to Seasonal Naive under FEV’s clipping and imputation rules. An aggregate win rate of 89.54% can coexist with first place on just 11 individual tasks. [Task-level scores](task-scores.csv) · [Exact rates, skills and ranks for every metric](ranks.csv)
10
+
11
+ | Panel and eligible models | SQL win rate | SQL skill | WQL win / skill rank | WAPE win / skill rank | MASE win / skill rank |
12
+ | --- | ---: | ---: | ---: | ---: | ---: |
13
+ | Overall, systems on, closed APIs off (36) | 89.54% · **1st** | 50.25% · 4th | **1** / 4 | **1** / 4 | 3 / 4 |
14
+ | Overall, all visible model types (41) | 86.83% · 2nd | 50.25% · 7th | 2 / 7 | 2 / 7 | 4 / 7 |
15
+ | Retail, systems on, closed APIs off (36) | 92.00% · 2nd | 46.33% · 3rd | 2 / 2 | 2 / 2 | 3 / 3 |
16
+ | Retail, all visible model types (41) | 89.13% · 2nd | 46.33% · 5th | 3 / 4 | 5 / 5 | 5 / 6 |
17
+
18
+ The dashboard normally hides systems. At this revision, adding D-wabi to the 35 default comparators is equivalent to enabling the systems category and leaving closed APIs off. In the wider field, TimeRSI-1.5 leads both overall SQL measures. TimeH2O-v1 leads Retail SQL win rate in both fields. [Leaderboard metadata](https://github.com/autogluon/fev/blob/844509e5211292620fbd9d5f24695f1d27da767a/scripts/fev_bench_metadata.py)
19
+
20
+ ## How the forecast was made
21
+
22
+ Seven frozen forecasting models supply a point prediction and deciles on each case. The original roster is t0-gamma, ci-proto, Timestral-209M, TiRex-2-fevbench, TimesFM-3, Chronos-2 and Toto-2.0-1B. TimeH2O-v1 is an eighth candidate for the probabilistic forecast. Users obtain checkpoints directly from their providers under the applicable terms. The benchmark system has non-commercial components. [Pinned checkpoint files and revisions](checkpoints.json)
23
+
24
+ 1. **Issue historical forecasts.** Each model receives the observed prefix available to each recipient and its eligible covariates. Recipients sharing an issuance time are processed together. Realized driver values without an authenticated forecast vintage are restricted to past observations on 14 tasks. Earlier errors are eligible only after their entire target window has arrived. Disjoint windows prevent the same overlapping outcomes from counting as separate rounds.
25
+ 2. **Choose the quantile forecasters.** With three eligible comparisons, rank the original models by mean earlier scaled quantile loss and keep the best three. If fewer qualify under the shared source-family boundary, look for the latest three eligible earlier comparisons within the task. Exclude every evaluation target interval from its physical source family, including sibling tasks. When both histories lack support, use all seven. A separate ranking that includes TimeH2O needs three complete, common comparisons and a cohort within the declared inference budget. The benchmark allows up to 16,384 context steps and 2,000,000 input cells for this optional model. At each quantile take the median of the selected model predictions. [Calendar helper](calendar_policy.py) · [Quantile selector](../../src/dwabi/quantile_policy.py) · [Cutoff and budget handling](../../src/dwabi/cutoff.py)
26
+ 3. **Issue the point forecast.** A separate WAPE action learns voting weights on ordered point forecasts using earlier losses. With sufficient history it checks sensitivity to deleting any one fitting window. A proposed combination learned from other physical source families must also show a consistent own-task improvement before it can replace the original point action. The final action accounts for inverse target volume and projects onto nonnegative or integer support when supported by the observed history. Its cutoff groups and provider roster stay independent of quantile selection. [Point policy](../../src/dwabi/point_policy.py) · [Volume calculation](../../src/dwabi/wape.py)
27
+
28
+ The benchmark freezes the fitted selection at the first official receiving cutoff. The member forecasts still advance with the observed history. The generic package also exposes an online interface. Each recipient sees the observations and future plans available at its own forecast time. Fourteen tasks received more conservative covariate inputs than the standard task role declarations. Public competitor scores reflect their respective input handling. Head-to-head comparisons establish the reported benchmark scores and leave the contribution of input differences unresolved.
29
+
30
+ ## Reproduce the rankings
31
+
32
+ The following commands use the released scores and the upstream evaluator. They run on CPU. All four metrics, two populations and both comparator fields are checked against the saved rates and ranks. A separate GPU workflow is necessary to regenerate the original member forecasts.
33
+
34
+ ```bash
35
+ git clone https://github.com/AbishekSriramulu/dwabi.git
36
+ git clone https://github.com/autogluon/fev.git
37
+ git -C fev checkout 844509e5211292620fbd9d5f24695f1d27da767a
38
+ uv venv --python 3.11 .venv
39
+ source .venv/bin/activate
40
+ uv pip install ./fev 'numpy==1.26.4' 'pandas==2.1.4' 'pyarrow==22.0.0' 'datasets==4.5.0' 'scipy==1.15.3' pyyaml
41
+ python dwabi/results/fev/verify.py fev
42
+ ```
43
+
44
+ The wheel used for the forecast has SHA-256 `f37c2c57eb67f0bc6db4c982ce5e1b1979add3da84e50be804ff6f631cd59a67`. A clean Linux build reproduced that wheel byte for byte. An isolated installation passed 18 package and calendar tests. The package replay checked all 1,308 point forecasts, reproduced the 120 quantile windows changed by the final policy and verified the prior package forecast hashes for the other 1,188. Source checkpoints, calibration and scores were measured in separate runs. Per-task end-to-end timing for this exact eight-model workflow is unavailable, so the exported runtime columns are empty. [Verification record](verification.json)
45
+
46
+ The public distribution is `dwabi==0.1.1.post1`. All seven Python modules match the benchmark wheel byte for byte. The post release supplies package URLs, licensing information and the public benchmark description. The SHA-256 of the public wheel is `497f6a5a473fff0ca37d5d3fa98dbfe31f4803480190ba4f05c6d335cc81534e`. [Research source](https://github.com/AbishekSriramulu/dwabi/tree/v0.1.1-research) · [Public package](https://pypi.org/project/dwabi/)
@@ -0,0 +1,17 @@
1
+ """Walk-forward forecast combination."""
2
+
3
+ from .blend import BlendDecision, BlendForecast, WalkForwardBlend
4
+ from .cutoff import (
5
+ CutoffDecision, CutoffEnsemble, CutoffForecast, ForecastBudget, ForecastCohort, ForecastRoute, ForecastSeries,
6
+ )
7
+ from .panel import DWabi, ForecastRequest
8
+ from .point_policy import PointDecision, PointHistory
9
+ from .quantile_policy import QuantileDecision, QuantileHistory
10
+ from .wape import observed_support, wape_deciles, wape_point
11
+
12
+ __all__ = [
13
+ "BlendDecision", "BlendForecast", "CutoffDecision", "CutoffEnsemble", "CutoffForecast", "DWabi",
14
+ "ForecastBudget", "ForecastCohort", "ForecastRequest", "ForecastRoute", "ForecastSeries",
15
+ "PointDecision", "PointHistory", "QuantileDecision", "QuantileHistory", "WalkForwardBlend",
16
+ "observed_support", "wape_deciles", "wape_point",
17
+ ]
@@ -0,0 +1,238 @@
1
+ """Choose and combine forecasters using their previously issued predictions."""
2
+
3
+ from collections.abc import Callable, Hashable, Mapping, Sequence
4
+ from dataclasses import dataclass, field
5
+ from itertools import combinations
6
+ from numbers import Integral
7
+ from types import MappingProxyType
8
+
9
+ import numpy as np
10
+
11
+
12
+ _DECILES = tuple(i / 10 for i in range(1, 10))
13
+ _GRID = np.asarray([(i, j, 6 - i - j) for i in range(7) for j in range(7 - i)], dtype=float) / 6
14
+ _EQUAL_GRID = int(np.flatnonzero((_GRID == 1 / 3).all(axis=1))[0])
15
+ _TOLERANCE = 1e-12
16
+ _MIN_WINDOWS = 3
17
+
18
+
19
+ @dataclass(frozen=True)
20
+ class BlendDecision:
21
+ """A forecast's member order, issued weights and supporting origins."""
22
+
23
+ members: tuple[str, ...]
24
+ weights: tuple[float, ...]
25
+ selection_origins: tuple[Hashable, ...]
26
+ weight_origins: tuple[Hashable, ...]
27
+ reason: str
28
+
29
+
30
+ @dataclass(frozen=True)
31
+ class BlendForecast:
32
+ """A combined forecast and the member arrays prepared at this issue."""
33
+
34
+ quantiles: np.ndarray
35
+ levels: tuple[float, ...]
36
+ members: Mapping[str, np.ndarray]
37
+ issued_at: int
38
+ decision: BlendDecision
39
+
40
+ @property
41
+ def point(self) -> np.ndarray:
42
+ """Return the combined median."""
43
+ return self.quantiles[..., self.levels.index(0.5)]
44
+
45
+
46
+ @dataclass
47
+ class _Window:
48
+ issued_at: int
49
+ first_target_at: int
50
+ available_at: int
51
+ values: np.ndarray
52
+ outcomes: np.ndarray
53
+ magnitude: np.ndarray
54
+ choice_loss: np.ndarray
55
+ grid_loss: dict[tuple[int, ...], np.ndarray] = field(default_factory=dict)
56
+
57
+
58
+ def _clock(value: int) -> int:
59
+ if isinstance(value, bool) or not isinstance(value, Integral):
60
+ raise TypeError("Supply issuance and outcome clocks as integers on one time axis.")
61
+ return int(value)
62
+
63
+
64
+ def _quantiles(values, levels: Sequence[float]) -> np.ndarray:
65
+ result = np.asarray(values, dtype=float)
66
+ if result.ndim < 2 or not result.size or result.shape[-1] != len(levels):
67
+ raise ValueError("Each member must provide case axes followed by the configured quantile levels.")
68
+ if not np.isfinite(result).all() or (np.diff(result, axis=-1) < 0.).any():
69
+ raise ValueError("Member quantiles must be finite and ordered.")
70
+ return result
71
+
72
+
73
+ def _loss(values: np.ndarray, outcomes: np.ndarray, magnitude: np.ndarray,
74
+ levels: np.ndarray, weights: np.ndarray) -> np.ndarray:
75
+ """Sum weighted pinball loss for each candidate on one matched window."""
76
+ scores = np.zeros(len(weights))
77
+ for start in range(0, len(outcomes), 4096):
78
+ rows = slice(start, start + 4096)
79
+ for offset in range(0, len(weights), 64):
80
+ candidates = slice(offset, offset + 64)
81
+ error = outcomes[rows, None, None] - values[rows] @ weights[candidates].T
82
+ pinball = np.maximum(levels * error, (levels - 1) * error)
83
+ scores[candidates] += (magnitude[rows, None, None] * pinball).sum(axis=(0, 1))
84
+ if not np.isfinite(scores).all():
85
+ raise ValueError("The weighted losses must be finite.")
86
+ return scores
87
+
88
+
89
+ class WalkForwardBlend:
90
+ """Choose three forecasters and estimate their weights at each issue.
91
+
92
+ Forecasters are a mapping of names to callables. Every callable receives
93
+ the same issue-time context and returns aligned quantile arrays.
94
+ Cacheable forecasts can also be registered directly using observe.
95
+ """
96
+
97
+ def __init__(self, models: Mapping[str, Callable], *,
98
+ levels: Sequence[float] = _DECILES, objective: str = "pinball",
99
+ selection: str = "joint", lookback: int | None = None, weighted: bool = True):
100
+ self._models = dict(models)
101
+ if len(self._models) < 3 or any(
102
+ not isinstance(name, str) or not name or not callable(model)
103
+ for name, model in self._models.items()
104
+ ):
105
+ raise ValueError("Provide at least three named callable forecasters.")
106
+ self._names = tuple(self._models)
107
+ self.levels = tuple(float(tau) for tau in levels)
108
+ if (0.5 not in self.levels or any(not 0 < tau < 1 for tau in self.levels)
109
+ or any(a >= b for a, b in zip(self.levels, self.levels[1:]))):
110
+ raise ValueError("Quantile levels must be strictly increasing and include 0.5.")
111
+ if objective not in ("pinball", "absolute"):
112
+ raise ValueError("objective must be 'pinball' or 'absolute'.")
113
+ if selection not in ("joint", "individual"):
114
+ raise ValueError("selection must be 'joint' or 'individual'.")
115
+ if lookback is not None and (_clock(lookback) < _MIN_WINDOWS):
116
+ raise ValueError("lookback must allow at least three mature windows.")
117
+ self._point_only = objective == "absolute"
118
+ self._selection = selection
119
+ self._lookback = lookback
120
+ self._weighted = weighted
121
+ self._loss_levels = np.asarray((0.5,) if self._point_only else self.levels)[None, :, None]
122
+ self._triples = tuple(combinations(range(len(self._names)), 3))
123
+ choices = np.zeros((len(self._triples), len(self._names)))
124
+ choices[np.arange(len(self._triples))[:, None], np.asarray(self._triples)] = 1 / 3
125
+ self._choices = np.vstack((np.eye(len(self._names)), choices,
126
+ np.full((1, len(self._names)), 1 / len(self._names))))
127
+ self._windows: dict[Hashable, _Window | None] = {}
128
+
129
+ def observe(self, forecasts: Mapping[str, np.ndarray], outcomes, *, origin: Hashable,
130
+ issued_at: int, first_target_at: int, available_at: int, case_weights=None) -> None:
131
+ """Register all members' original predictions after an outcome window matures.
132
+
133
+ The caller aligns case keys and supplies the intended evaluation weights.
134
+ Missing outcomes and zero case weights are omitted from fitting.
135
+ """
136
+ issue, first, available = map(_clock, (issued_at, first_target_at, available_at))
137
+ if not issue < first <= available:
138
+ raise ValueError("Require issued_at < first_target_at <= available_at.")
139
+ if origin is None or origin in self._windows:
140
+ raise ValueError("Each observation requires a unique origin.")
141
+ if set(forecasts) != set(self._names):
142
+ raise ValueError("Learning membership requires an originally issued forecast from every member.")
143
+ actual = np.asarray(outcomes, dtype=float)
144
+ arrays = [_quantiles(forecasts[name], self.levels) for name in self._names]
145
+ if any(array.shape[:-1] != actual.shape for array in arrays):
146
+ raise ValueError("Member forecasts and outcomes must have the same case axes.")
147
+ magnitude = np.ones(actual.shape) if case_weights is None else np.asarray(case_weights, dtype=float)
148
+ if (magnitude.shape != actual.shape or not np.isfinite(magnitude).all()
149
+ or (magnitude < 0.).any() or np.isinf(actual).any()):
150
+ raise ValueError("Case weights must be finite and nonnegative and outcomes cannot be infinite.")
151
+ valid = np.isfinite(actual) & (magnitude > 0.)
152
+ window = None
153
+ if valid.any():
154
+ if self._point_only:
155
+ k = self.levels.index(0.5)
156
+ arrays = [array[..., k:k + 1] for array in arrays]
157
+ q = np.stack([array[valid] for array in arrays], axis=-1)
158
+ y, mass = actual[valid], magnitude[valid]
159
+ losses = _loss(q, y, mass, self._loss_levels, self._choices)
160
+ if losses[-1] > 0:
161
+ relative = losses / losses[-1]
162
+ if not np.isfinite(relative).all():
163
+ raise ValueError("The relative losses must be finite.")
164
+ window = _Window(issue, first, available, q, y, mass, relative)
165
+ self._windows[origin] = window
166
+
167
+ def _support(self, issue: int):
168
+ eligible = [(origin, row) for origin, row in self._windows.items()
169
+ if row is not None and row.available_at < issue]
170
+ blocks = []
171
+ for origin, row in sorted(eligible, key=lambda item: (item[1].available_at, item[1].issued_at)):
172
+ if not blocks or blocks[-1][1].available_at < row.first_target_at:
173
+ blocks.append((origin, row))
174
+ return blocks if self._lookback is None else blocks[-self._lookback:]
175
+
176
+ def _fit_weights(self, past, triple):
177
+ risk, origins = [], []
178
+ if self._weighted:
179
+ for origin, row in past:
180
+ if triple not in row.grid_loss:
181
+ row.grid_loss[triple] = _loss(
182
+ row.values[..., list(triple)], row.outcomes, row.magnitude, self._loss_levels, _GRID,
183
+ )
184
+ losses = row.grid_loss[triple]
185
+ if losses[_EQUAL_GRID] <= 0:
186
+ continue
187
+ ratios = losses / losses[_EQUAL_GRID]
188
+ if not np.allclose(ratios, 1., rtol=_TOLERANCE, atol=_TOLERANCE):
189
+ risk.append(ratios)
190
+ origins.append(origin)
191
+ n = len(risk)
192
+ if n < _MIN_WINDOWS:
193
+ return (1 / 3,) * 3, tuple(origins)
194
+ average = np.mean(risk, axis=0)
195
+ ties = np.isclose(average, average.min(), rtol=_TOLERANCE, atol=_TOLERANCE)
196
+ learned = _GRID[ties].mean(axis=0)
197
+ shrunk = (n * learned + 1.) / (n + _MIN_WINDOWS)
198
+ return tuple(shrunk.tolist()), tuple(origins)
199
+
200
+ def explain(self, issue_at: int) -> BlendDecision:
201
+ """Return the decision and its evidence without running a forecaster."""
202
+ past = self._support(_clock(issue_at))
203
+ origins = tuple(origin for origin, _ in past)
204
+ full = (1 / len(self._names),) * len(self._names)
205
+ if len(past) < _MIN_WINDOWS:
206
+ return BlendDecision(self._names, full, origins, (), "insufficient_history")
207
+ losses = np.mean([row.choice_loss for _, row in past], axis=0)
208
+ n = len(self._names)
209
+ if self._selection == "individual":
210
+ triple = tuple(sorted(np.argsort(losses[:n], kind="stable")[:3].tolist()))
211
+ else:
212
+ candidates = losses[n:-1]
213
+ tied = np.flatnonzero(np.isclose(candidates, candidates.min(), rtol=_TOLERANCE, atol=_TOLERANCE))
214
+ if len(tied) == len(candidates) and len(candidates) > 1:
215
+ return BlendDecision(self._names, full, origins, (), "indistinguishable_triples")
216
+ triple = self._triples[int(tied[0])]
217
+ weights, fitting = self._fit_weights(past, triple)
218
+ return BlendDecision(tuple(self._names[i] for i in triple), weights, origins, fitting, self._selection)
219
+
220
+ def predict(self, context, *, issue_at: int, full_roster: bool = True) -> BlendForecast:
221
+ """Run the configured forecasters and issue their combined quantiles.
222
+
223
+ Full-roster predictions can update selection after their outcomes arrive.
224
+ A selected-only call saves inference when a current decision is in use.
225
+ """
226
+ issue = _clock(issue_at)
227
+ decision = self.explain(issue)
228
+ names = self._names if full_roster else decision.members
229
+ members = {name: _quantiles(self._models[name](context), self.levels).copy() for name in names}
230
+ shape = members[names[0]].shape
231
+ if any(q.shape != shape for q in members.values()):
232
+ raise ValueError("Model forecast case shapes differ.")
233
+ values = np.zeros(shape)
234
+ for name, weight in zip(decision.members, decision.weights, strict=True):
235
+ values += weight * members[name]
236
+ for array in (*members.values(), values):
237
+ array.setflags(write=False)
238
+ return BlendForecast(values, self.levels, MappingProxyType(members), issue, decision)