pystatistics 4.6.2__tar.gz → 4.6.3__tar.gz

pystatistics-4.6.3/.release/UNRELEASED.md

@@ -0,0 +1,69 @@
+# Unreleased Changes
+
+> This file tracks all changes since the last stable release.
+> Updated by whoever makes a change, on whatever machine.
+> Synced via git so all sessions (Mac, Linux, etc.) see the same state.
+>
+> When ready to release, run: `python .release/release.py --status`
+> and follow the manual release flow in the script docstring.
+
+## Changes
+
+- **timeseries: damped-trend ETS fits no longer stall (damped fits improve
+  or stay identical; every other fit changes only if it previously ran out
+  of optimiser budget).** Two compounding defects in
+  `timeseries/_ets_fit.py`: (1) R's `initparam` starts `phi` at 0.9782 —
+  99% of the way to the 0.98 usual-region bound — which is fine for R's
+  derivative-free Nelder-Mead but saturates our logit transform, so the
+  numerical gradient in the phi direction was ~4% of mid-range (sigmoid
+  derivative 0.0099 vs 0.25) and L-BFGS-B crawled; (2) scipy's default
+  `maxfun=15000` allows only ~830 iterations for a 17-parameter seasonal
+  model (numerical gradient = n+1 evals/iteration), so the requested
+  `maxiter=1000` was unreachable and co2 `MAdM` died with "EVALUATIONS
+  EXCEEDS LIMIT" at `converged=False`, ~1.7 AICc worse than its true
+  optimum. Fix: damped models now optimise from BOTH a mid-range phi start
+  (0.9) and R's initparam start (0.9782), keeping the better optimum, and
+  `maxfun = max(15000, max_iter*(n_free+1)*2)` — floored at scipy's old
+  default so no fit ever gets a smaller budget than 4.6.2 gave it. That
+  floor makes the guarantee provable: the R-start leg reproduces the 4.6.2
+  trajectory (deterministic, with at least the old budget), so damped
+  results are never worse, and any fit that terminated within the old
+  budget (every reference fit does) is bit-identical unless damped. The
+  phi bounds stay R's (0.8, 0.98). Verified at full precision on 15 damped
+  reference fits: 7 improved (co2 MAdM aicc 172.592→170.898 and now
+  converged, beating R's own damped optimum; usaccdeaths MAdA −0.25), 8
+  bit-identical, zero worse, all `converged=True`; all non-damped
+  reference fits bit-identical; all 10 reference ZZZ selections unchanged
+  (on non-reference series a damped candidate whose fit improved can now
+  legitimately win a selection it previously lost — that is the point of
+  the fix).
+- **timeseries: `converged=False` false-negative on damped ETS fits fixed**
+  — it was the evaluation-budget exhaustion above, not a flag-logic bug; a
+  damped fit that reaches its optimum now reports `converged=True` (new
+  regression tests pin this for co2/airpassengers/usaccdeaths/lynx/nile/
+  wwwusage damped fits).
+- **timeseries/tests: honest cross-engine ETS verification encoded as a
+  regression gate.** A prior review claimed our ZZZ selections were worse
+  than R's because refitting our picked model string in R gave a higher
+  AICc — but that only re-runs R's own Nelder-Mead (the optimiser being
+  compared) and reproduces its stall. The fair test — evaluating OUR fitted
+  parameters with R's own likelihood code `forecast:::pegelsresid.C`
+  (seasonal states reversed into R's ordering) — shows our pick has strictly
+  lower AICc under R's own criterion on all 6 divergent reference datasets
+  (e.g. co2: our M,A,M scores 1697.2 in R vs 1722.6 for R's chosen M,Ad,M)
+  and equal-or-better on the 4 agreeing ones, with every pick admissible
+  under R's `admissible()`. The fixture regeneration is now two-stage
+  (`tests/fixtures/generate_ets_py_params.py` dumps our fitted parameters,
+  `generate_ets_r_reference.R` cross-scores them; the transplant harness
+  self-validates by reproducing R's own fits' log-likelihoods before
+  scoring foreign parameters, and stores `py_pick`/`py_pick_aicc_in_r`/
+  `py_pick_admissible` per dataset). New tests
+  (`test_selection_dominates_r_under_r_own_likelihood`, drift-guarded on
+  the stored pick) fail if any future change degrades a selection to
+  something R's own numbers call worse. `_ets_select.py`'s parity
+  documentation now states this verification method explicitly instead of
+  a vague "better than R". The selection/parity/damped-convergence tests
+  moved to a new `tests/timeseries/test_ets_selection.py` (`test_ets.py`
+  had exceeded the 500-LoC limit; now 382 + 262). The R generator now
+  refuses to run without the stage-1 params dump instead of silently
+  writing a fixture missing the fields ten tests require.

{pystatistics-4.6.2 → pystatistics-4.6.3}/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## 4.6.3
+
+Damped-trend ETS fits no longer stall; ETS-vs-R verification strengthened.
+
+- **Damped-trend ETS models (`Ad`) could stall short of their optimum and
+  report `converged=False` — fixed.** The damping parameter started so close
+  to its upper bound that the optimiser could barely move it, and larger
+  seasonal models could also exhaust their evaluation budget mid-fit (e.g.
+  `ets(co2, model="MAdM", period=12)` returned `converged=False` about 1.7
+  AICc above its true optimum). Damped models are now optimised from two
+  starting points with the better result kept — one of them reproduces the
+  previous behaviour with at least the previous evaluation budget, so
+  damped fits improve or stay identical, never worse — and the evaluation
+  budget now scales with model size (never below its old value). Converged
+  damped fits now say so. Any fit that finished within the old budget is
+  bit-identical unless damped; on every reference dataset all non-damped
+  fits are bit-identical and automatic `"ZZZ"` selection picks the same
+  model as before. On other series a damped candidate whose fit improved
+  can now win a selection it previously lost — the intended effect.
+- **The claim that `ets()` model selection beats `forecast::ets` on
+  divergent picks is now verified the fair way and enforced by tests.**
+  Where automatic selection differs from R (6 of 10 reference datasets),
+  our selected model's fitted parameters are evaluated by *R's own
+  likelihood code* and score a strictly lower AICc there than the model R
+  selects (e.g. monthly CO2: 1697.2 vs 1722.6), while remaining in R's
+  admissible (forecast-stable) parameter region. Refitting the same model
+  string in R understates our fit because it re-runs R's optimiser, which
+  is what stalls in the first place. The cross-scored values ship in the
+  test fixtures and regression tests fail if a future change makes any
+  selection worse under R's own numbers.
+
 ## 4.6.2
 
 STL decomposition rewritten to match R exactly and now as fast as R's

{pystatistics-4.6.2 → pystatistics-4.6.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pystatistics
-Version: 4.6.2
+Version: 4.6.3
 Summary: GPU-accelerated statistical computing for Python
 Project-URL: Homepage, https://sgcx.org/technology/pystatistics/
 Project-URL: Documentation, https://sgcx.org/docs/pystatistics/
@@ -418,6 +418,21 @@ pip install pystatistics[dev]
 
 ## What's New
 
+### 4.6.3 — damped-trend ETS fits no longer stall
+
+Damped-trend ETS models (`Ad`) could stop short of their optimum and report
+`converged=False` (the damping parameter started nearly pinned to its upper
+bound, and large seasonal fits could exhaust their evaluation budget).
+Damped models are now optimised from two starting points with the better
+result kept; damped fits improve or stay identical, non-damped fits are
+unchanged, and automatic `"ZZZ"` selection picks the same models on every
+reference dataset (elsewhere an improved damped fit can now win a selection
+it previously lost — the intended effect). Where that
+selection differs from R's `forecast::ets`, the choice is now verified by
+evaluating the fitted parameters under R's own likelihood code — our pick
+scores a strictly lower AICc there on every divergent reference dataset —
+and regression tests enforce that this stays true.
+
 ### 4.6.2 — STL matches R exactly and runs as fast; ETS automatic model selection
 
 `stl()` was rewritten after validation showed it leaking trend into the

{pystatistics-4.6.2 → pystatistics-4.6.3}/README.md

@@ -371,6 +371,21 @@ pip install pystatistics[dev]
 
 ## What's New
 
+### 4.6.3 — damped-trend ETS fits no longer stall
+
+Damped-trend ETS models (`Ad`) could stop short of their optimum and report
+`converged=False` (the damping parameter started nearly pinned to its upper
+bound, and large seasonal fits could exhaust their evaluation budget).
+Damped models are now optimised from two starting points with the better
+result kept; damped fits improve or stay identical, non-damped fits are
+unchanged, and automatic `"ZZZ"` selection picks the same models on every
+reference dataset (elsewhere an improved damped fit can now win a selection
+it previously lost — the intended effect). Where that
+selection differs from R's `forecast::ets`, the choice is now verified by
+evaluating the fitted parameters under R's own likelihood code — our pick
+scores a strictly lower AICc there on every divergent reference dataset —
+and regression tests enforce that this stays true.
+
 ### 4.6.2 — STL matches R exactly and runs as fast; ETS automatic model selection
 
 `stl()` was rewritten after validation showed it leaking trend into the

{pystatistics-4.6.2 → pystatistics-4.6.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pystatistics"
-version = "4.6.2"
+version = "4.6.3"
 description = "GPU-accelerated statistical computing for Python"
 readme = "README.md"
 license = "MIT"

{pystatistics-4.6.2 → pystatistics-4.6.3}/pystatistics/__init__.py

@@ -16,7 +16,7 @@ Usage:
     result = fit(design)
 """
 
-__version__ = "4.6.2"
+__version__ = "4.6.3"
 __author__ = "Hai-Shuo"
 __email__ = "contact@sgcx.org"
 

{pystatistics-4.6.2 → pystatistics-4.6.3}/pystatistics/timeseries/_ets_fit.py

@@ -542,8 +542,21 @@ def fit_ets_model(
             g0 = _EPS + 0.05 * (min(1.0 - _EPS, 1.0 - a0) - _EPS)
             theta_smooth0[i] = _logit(g0, _EPS, 1.0 - a0)
         i += 1
-    if spec.damped and phi is None:
-        p0 = _PHI_BOUNDS[0] + 0.99 * (_PHI_BOUNDS[1] - _PHI_BOUNDS[0])
+    phi_free = spec.damped and phi is None
+    if phi_free:
+        # R's initparam starts phi at 0.9782 — 99% of the way to the 0.98
+        # bound.  That is fine for R's derivative-free Nelder-Mead but
+        # saturates the logit transform for L-BFGS-B: the sigmoid
+        # derivative there is ~4% of mid-range (0.0099 vs 0.25), so the
+        # numerical gradient in the phi direction all but vanishes and
+        # damped fits crawl (co2 MAdM exhausted its evaluation budget
+        # unconverged).  A
+        # mid-range start keeps the phi gradient alive but can land in the
+        # other phi basin on some series, so damped fits are run from BOTH
+        # starts and the better optimum kept (see the optimiser call) —
+        # provably never worse than the single R start.  The bounds
+        # themselves stay R's (0.8, 0.98) usual region.
+        p0 = 0.9
         theta_smooth0[i] = _logit(p0, _PHI_BOUNDS[0], _PHI_BOUNDS[1])
 
     # Free initial states (unbounded).  Seasonal models optimise m - 1
@@ -570,12 +583,45 @@ def fit_ets_model(
             theta_full, y_arr, spec, fixed_smooth, alpha_box, n_smooth
         )
 
-    result = minimize(
-        _objective,
-        theta0[free_idx],
-        method="L-BFGS-B",
-        options={"maxiter": max_iter, "ftol": tol, "gtol": tol},
-    )
+    # Damped models are optimised from two phi starts (mid-range 0.9 and
+    # R's initparam 0.9782 — see the phi_free comment above) and the
+    # better optimum kept; other models run once.
+    starts = [theta0]
+    if phi_free:
+        alt = theta0.copy()
+        alt[n_smooth - 1] = _logit(
+            _PHI_BOUNDS[0] + 0.99 * (_PHI_BOUNDS[1] - _PHI_BOUNDS[0]),
+            _PHI_BOUNDS[0],
+            _PHI_BOUNDS[1],
+        )
+        starts.append(alt)
+
+    # maxfun must scale with dimension or it, not maxiter, becomes the
+    # binding limit: scipy's default (15000) allows only ~830 iterations
+    # for a 17-parameter seasonal model (numerical gradient = n+1 evals
+    # per iteration), so requesting maxiter=1000 was unreachable and large
+    # damped fits died with "EVALUATIONS EXCEEDS LIMIT" (converged=False).
+    # (n+1) evals per gradient, x2 headroom for line searches; floored at
+    # scipy's old default so no fit ever gets a SMALLER budget than
+    # before — that floor is what makes the two-start damped strategy
+    # provably never worse (the R-start leg reproduces the old
+    # trajectory) and keeps every other fit bit-identical.
+    maxfun = max(15000, max_iter * (len(free_idx) + 1) * 2)
+    result = None
+    for start in starts:
+        res = minimize(
+            _objective,
+            start[free_idx],
+            method="L-BFGS-B",
+            options={
+                "maxiter": max_iter,
+                "maxfun": maxfun,
+                "ftol": tol,
+                "gtol": tol,
+            },
+        )
+        if result is None or res.fun < result.fun:
+            result = res
     converged = result.success
 
     # Reconstruct full theta and map back to bounded values

{pystatistics-4.6.2 → pystatistics-4.6.3}/pystatistics/timeseries/_ets_select.py

@@ -57,15 +57,29 @@ with the remaining one fixed by normalisation — see ``_ets_fit.py``),
 so per-candidate parameter counts and log-likelihoods are directly
 comparable to R's.  The *selected model* usually matches but can still
 differ, because the engines optimise differently: PyStatistics uses
-L-BFGS-B on logit-transformed parameters, R uses Nelder-Mead, and on
-seasonal candidates R's optimiser frequently stalls well short of the
-optimum (tens of log-likelihood units on the classic benchmark
-series).  In every observed divergence (AirPassengers, co2, lynx in
-``tests/fixtures/ets_r_reference.json``) the PyStatistics selection has
-a *better* value of R's own criterion than R's selection — the tables
-disagree about the optima, not the criterion.  Each returned solution
-discloses its full candidate IC table in ``info["selection"]``, so any
-selection can be audited.
+L-BFGS-B on logit-transformed parameters (damped models from two phi
+starts), R uses Nelder-Mead, and on seasonal candidates R's optimiser
+frequently stalls well short of the optimum (tens of log-likelihood
+units on the classic benchmark series).
+
+Every divergence is verified by the only fair cross-engine test:
+PyStatistics' *fitted parameters* are evaluated by **R's own likelihood
+code** (``forecast:::pegelsresid.C``, with the seasonal states reversed
+into R's ordering; the transplant harness is self-validated by first
+reproducing R's own fits' log-likelihoods to 1e-8 relative tolerance).
+Refitting the same
+model string in R would merely re-run R's Nelder-Mead — the very
+optimiser being compared — and reproduce its stall.  Under that test,
+on all six divergent reference selections (AirPassengers, co2, lynx and
+variants in ``tests/fixtures/ets_r_reference.json``) the PyStatistics
+pick has a strictly lower AICc under R's own likelihood than the model
+``forecast::ets`` selects, and its parameters are admissible
+(forecast-stable) under R's own ``admissible()`` check.  The scored
+values are stored in the fixture (``py_pick_aicc_in_r``) and pinned by
+regression tests; regenerate with ``generate_ets_py_params.py`` then
+``generate_ets_r_reference.R``.  The tables disagree about the optima,
+not the criterion.  Each returned solution discloses its full candidate
+IC table in ``info["selection"]``, so any selection can be audited.
 """
 
 from __future__ import annotations

pystatistics-4.6.3/tests/fixtures/ets_py_params.json

@@ -0,0 +1,237 @@
+{
+ "airpassengers": {
+  "picked": "ETS(M,A,M)",
+  "error": "M",
+  "trend": "A",
+  "season": "M",
+  "damped": false,
+  "period": 12,
+  "alpha": 0.7409340242098023,
+  "beta": 0.00010000076169295685,
+  "gamma": 0.00010005920080370926,
+  "phi": null,
+  "init_level": 122.79644644403064,
+  "init_trend": 2.129237987080646,
+  "init_season": [
+   0.9065254936956091,
+   0.8885773704991264,
+   1.012442088634839,
+   0.9820704726765775,
+   0.9810735260927437,
+   1.1101215440958783,
+   1.2318886786272436,
+   1.2201103859365203,
+   1.056876738889878,
+   0.9199475432085753,
+   0.7968985140153373,
+   0.8934676436276712
+  ],
+  "aicc_py_convention": 1083.85273722301,
+  "converged": true
+ },
+ "usaccdeaths": {
+  "picked": "ETS(A,N,A)",
+  "error": "A",
+  "trend": "N",
+  "season": "A",
+  "damped": false,
+  "period": 12,
+  "alpha": 0.5786178234192573,
+  "beta": null,
+  "gamma": 0.0001,
+  "phi": null,
+  "init_level": 9741.666419160323,
+  "init_trend": null,
+  "init_season": [
+   -792.958856989011,
+   -1544.866345494225,
+   -758.2273234609411,
+   -536.1023594337732,
+   322.07088058544423,
+   802.4044677641663,
+   1668.926889520922,
+   974.1591215399224,
+   -66.02205748708626,
+   232.711277283187,
+   -282.0077245809384,
+   -20.087969247666628
+  ],
+  "aicc_py_convention": 1039.4141176863654,
+  "converged": true
+ },
+ "co2": {
+  "picked": "ETS(M,A,M)",
+  "error": "M",
+  "trend": "A",
+  "season": "M",
+  "damped": false,
+  "period": 12,
+  "alpha": 0.6621853124252347,
+  "beta": 0.010608746654465878,
+  "gamma": 0.0007307829692142658,
+  "phi": null,
+  "init_level": 315.3608726579102,
+  "init_trend": 0.08480820178629678,
+  "init_season": [
+   0.9998757011060313,
+   1.0018551346557558,
+   1.0040645319341568,
+   1.0074253427806026,
+   1.008857198520008,
+   1.006905738849209,
+   1.0024116732983697,
+   0.9962849951627571,
+   0.9908973218114535,
+   0.9903511862513498,
+   0.9938810738299224,
+   0.997190101800384
+  ],
+  "aicc_py_convention": 147.8320733184404,
+  "converged": true
+ },
+ "nile": {
+  "picked": "ETS(M,N,N)",
+  "error": "M",
+  "trend": "N",
+  "season": "N",
+  "damped": false,
+  "period": 1,
+  "alpha": 0.15120889034467438,
+  "beta": null,
+  "gamma": null,
+  "phi": null,
+  "init_level": 1087.754615467445,
+  "init_trend": null,
+  "init_season": null,
+  "aicc_py_convention": 1281.822594111822,
+  "converged": true
+ },
+ "wwwusage": {
+  "picked": "ETS(A,Ad,N)",
+  "error": "A",
+  "trend": "Ad",
+  "season": "N",
+  "damped": true,
+  "period": 1,
+  "alpha": 0.9998999990820521,
+  "beta": 0.9998998503151512,
+  "gamma": null,
+  "phi": 0.8066680914946358,
+  "init_level": 92.9583864776785,
+  "init_trend": -6.14670285926298,
+  "init_season": null,
+  "aicc_py_convention": 540.9123279450281,
+  "converged": true
+ },
+ "lynx": {
+  "picked": "ETS(M,A,N)",
+  "error": "M",
+  "trend": "A",
+  "season": "N",
+  "damped": false,
+  "period": 1,
+  "alpha": 0.9999,
+  "beta": 0.0001,
+  "gamma": null,
+  "phi": null,
+  "init_level": 104.80092766288334,
+  "init_trend": 89.34198077164446,
+  "init_season": null,
+  "aicc_py_convention": 1825.536853187305,
+  "converged": true
+ },
+ "diff_nile": {
+  "picked": "ETS(A,N,N)",
+  "error": "A",
+  "trend": "N",
+  "season": "N",
+  "damped": false,
+  "period": 1,
+  "alpha": 0.00010008018498308305,
+  "beta": null,
+  "gamma": null,
+  "phi": null,
+  "init_level": -3.837001886361001,
+  "init_trend": null,
+  "init_season": null,
+  "aicc_py_convention": 1300.907562564694,
+  "converged": true
+ },
+ "airpassengers_zzn": {
+  "picked": "ETS(M,A,N)",
+  "error": "M",
+  "trend": "A",
+  "season": "N",
+  "damped": false,
+  "period": 1,
+  "alpha": 0.9998997260462488,
+  "beta": 0.00010000511832230999,
+  "gamma": null,
+  "phi": null,
+  "init_level": 106.6432773543166,
+  "init_trend": 4.165946497198817,
+  "init_season": null,
+  "aicc_py_convention": 1366.4125174351261,
+  "converged": true
+ },
+ "airpassengers_azz": {
+  "picked": "ETS(A,A,A)",
+  "error": "A",
+  "trend": "A",
+  "season": "A",
+  "damped": false,
+  "period": 12,
+  "alpha": 0.25276704707869657,
+  "beta": 0.00010000741679970583,
+  "gamma": 0.7472328134083621,
+  "phi": null,
+  "init_level": 117.90776886286908,
+  "init_trend": 2.5946657964894193,
+  "init_season": [
+   -8.500669715398159,
+   -2.928236169263969,
+   9.55479917171517,
+   4.317353244662001,
+   -4.423572632956558,
+   9.679454457800507,
+   21.866719726803083,
+   19.38694357550404,
+   5.159781796259593,
+   -13.774991234367437,
+   -28.379683743002367,
+   -11.957898477755913
+  ],
+  "aicc_py_convention": 1168.8597581594572,
+  "converged": true
+ },
+ "airpassengers_mzz": {
+  "picked": "ETS(M,A,M)",
+  "error": "M",
+  "trend": "A",
+  "season": "M",
+  "damped": false,
+  "period": 12,
+  "alpha": 0.7409340242098023,
+  "beta": 0.00010000076169295685,
+  "gamma": 0.00010005920080370926,
+  "phi": null,
+  "init_level": 122.79644644403064,
+  "init_trend": 2.129237987080646,
+  "init_season": [
+   0.9065254936956091,
+   0.8885773704991264,
+   1.012442088634839,
+   0.9820704726765775,
+   0.9810735260927437,
+   1.1101215440958783,
+   1.2318886786272436,
+   1.2201103859365203,
+   1.056876738889878,
+   0.9199475432085753,
+   0.7968985140153373,
+   0.8934676436276712
+  ],
+  "aicc_py_convention": 1083.85273722301,
+  "converged": true
+ }
+}

pystatistics-4.6.3/tests/fixtures/ets_r_reference.json

@@ -0,0 +1 @@
+{"selection":{"airpassengers":{"name":"airpassengers","x":[112,118,132,129,121,135,148,148,136,119,104,118,115,126,141,135,125,149,170,170,158,133,114,140,145,150,178,163,172,178,199,199,184,162,146,166,171,180,193,181,183,218,230,242,209,191,172,194,196,196,236,235,229,243,264,272,237,211,180,201,204,188,235,227,234,264,302,293,259,229,203,229,242,233,267,269,270,315,364,347,312,274,237,278,284,277,317,313,318,374,413,405,355,306,271,306,315,301,356,348,355,422,465,467,404,347,305,336,340,318,362,348,363,435,491,505,404,359,310,337,360,342,406,396,420,472,548,559,463,407,362,405,417,391,419,461,472,535,622,606,508,461,390,432],"period":12,"model_arg":"ZZZ","method":"ETS(M,Ad,M)","components":["M","A","M","TRUE"],"loglik":-679.583216236962,"aic":1395.16643247392,"aicc":1400.63843247392,"bic":1448.62307186629,"n":144,"py_pick":"ETS(M,A,M)","py_pick_aicc_in_r":1390.85155479901,"py_pick_admissible":1},"usaccdeaths":{"name":"usaccdeaths","x":[9007,8106,8928,9137,10017,10826,11317,10744,9713,9938,9161,8927,7750,6981,8038,8422,8714,9512,10120,9823,8743,9129,8710,8680,8162,7306,8124,7870,9387,9556,10093,9620,8285,8466,8160,8034,7717,7461,7767,7925,8623,8945,10078,9179,8037,8488,7874,8647,7792,6957,7726,8106,8890,9299,10625,9302,8314,8850,8265,8796,7836,6892,7791,8192,9115,9434,10484,9827,9110,9070,8633,9240],"period":12,"model_arg":"ZZZ","method":"ETS(A,N,A)","components":["A","N","A","FALSE"],"loglik":-555.072335012621,"aic":1140.14467002524,"aicc":1148.71609859667,"bic":1174.29466181048,"n":72,"py_pick":"ETS(A,N,A)","py_pick_aicc_in_r":1143.00692947405,"py_pick_admissible":1},"co2":{"name":"co2","x":[315.42,316.31,316.5,317.56,318.13,318,316.39,314.65,313.68,313.18,314.66,315.43,316.27,316.81,317.42,318.87,319.87,319.43,318.01,315.74,314,313.68,314.84,316.03,316.73,317.54,318.38,319.31,320.42,319.61,318.42,316.63,314.83,315.16,315.94,316.85,317.78,318.4,319.53,320.42,320.85,320.45,319.45,317.25,316.11,315.27,316.53,317.53,318.58,318.92,319.7,321.22,322.08,321.31,319.58,317.61,316.05,315.83,316.91,318.2,319.41,320.07,320.74,321.4,322.06,321.73,320.27,318.54,316.54,316.71,317.53,318.55,319.27,320.28,320.73,321.97,322,321.71,321.05,318.71,317.66,317.14,318.7,319.25,320.46,321.43,322.23,323.54,323.91,323.59,322.24,320.2,318.48,317.94,319.63,320.87,322.17,322.34,322.88,324.25,324.83,323.93,322.38,320.76,319.1,319.24,320.56,321.8,322.4,322.99,323.73,324.86,325.4,325.2,323.98,321.95,320.18,320.09,321.16,322.74,323.83,324.26,325.47,326.5,327.21,326.54,325.72,323.5,322.22,321.62,322.69,323.95,324.89,325.82,326.77,327.97,327.91,327.5,326.18,324.53,322.93,322.9,323.85,324.96,326.01,326.51,327.01,327.62,328.76,328.4,327.2,325.27,323.2,323.4,324.63,325.85,326.6,327.47,327.58,329.56,329.9,328.92,327.88,326.16,324.68,325.04,326.34,327.39,328.37,329.4,330.14,331.33,332.31,331.9,330.7,329.15,327.35,327.02,327.99,328.48,329.18,330.55,331.32,332.48,332.92,332.08,331.01,329.23,327.27,327.21,328.29,329.41,330.23,331.25,331.87,333.14,333.8,333.43,331.73,329.9,328.4,328.17,329.32,330.59,331.58,332.39,333.33,334.41,334.71,334.17,332.89,330.77,329.14,328.78,330.14,331.52,332.75,333.24,334.53,335.9,336.57,336.1,334.76,332.59,331.42,330.98,332.24,333.68,334.8,335.22,336.47,337.59,337.84,337.72,336.37,334.51,332.6,332.38,333.75,334.78,336.05,336.59,337.79,338.71,339.3,339.12,337.56,335.92,333.75,333.7,335.12,336.56,337.84,338.19,339.91,340.6,341.29,341,339.39,337.43,335.72,335.84,336.93,338.04,339.06,340.3,341.21,342.33,342.74,342.08,340.32,338.26,336.52,336.68,338.19,339.44,340.57,341.44,342.53,343.39,343.96,343.18,341.88,339.65,337.81,337.69,339.09,340.32,341.2,342.35,342.93,344.77,345.58,345.14,343.81,342.21,339.69,339.82,340.98,342.82,343.52,344.33,345.11,346.88,347.25,346.62,345.22,343.11,340.9,341.18,342.8,344.04,344.79,345.82,347.25,348.17,348.74,348.07,346.38,344.51,342.92,342.62,344.06,345.38,346.11,346.78,347.68,349.37,350.03,349.37,347.76,345.73,344.68,343.99,345.48,346.72,347.84,348.29,349.23,350.8,351.66,351.07,349.33,347.92,346.27,346.18,347.64,348.78,350.25,351.54,352.05,353.41,354.04,353.62,352.22,350.27,348.55,348.72,349.91,351.18,352.6,352.92,353.53,355.26,355.52,354.97,353.75,351.52,349.64,349.83,351.14,352.37,353.5,354.55,355.23,356.04,357,356.07,354.67,352.76,350.82,351.04,352.69,354.07,354.59,355.63,357.03,358.48,359.22,358.12,356.06,353.92,352.05,352.11,353.64,354.89,355.88,356.63,357.72,359.07,359.58,359.17,356.94,354.92,352.94,353.23,354.09,355.33,356.63,357.1,358.32,359.41,360.23,359.55,357.53,355.48,353.67,353.95,355.3,356.78,358.34,358.89,359.95,361.25,361.67,360.94,359.55,357.49,355.84,356,357.59,359.05,359.98,361.03,361.66,363.48,363.82,363.3,361.94,359.5,358.11,357.8,359.61,360.74,362.09,363.29,364.06,364.76,365.45,365.01,363.7,361.54,359.51,359.65,360.8,362.38,363.23,364.06,364.61,366.4,366.84,365.68,364.52,362.57,360.24,360.83,362.49,364.34],"period":12,"model_arg":"ZZZ","method":"ETS(M,Ad,M)","components":["M","A","M","TRUE"],"loglik":-842.552382383843,"aic":1721.10476476769,"aicc":1722.62815006835,"bic":1795.7771940942,"n":468,"py_pick":"ETS(M,A,M)","py_pick_aicc_in_r":1697.18876872835,"py_pick_admissible":1},"nile":{"name":"nile","x":[1120,1160,963,1210,1160,1160,813,1230,1370,1140,995,935,1110,994,1020,960,1180,799,958,1140,1100,1210,1150,1250,1260,1220,1030,1100,774,840,874,694,940,833,701,916,692,1020,1050,969,831,726,456,824,702,1120,1100,832,764,821,768,845,864,862,698,845,744,796,1040,759,781,865,845,944,984,897,822,1010,771,676,649,846,812,742,801,1040,860,874,848,890,744,749,838,1050,918,986,797,923,975,815,1020,906,901,1170,912,746,919,718,714,740],"period":1,"model_arg":"ZZZ","method":"ETS(M,N,N)","components":["M","N","N","FALSE"],"loglik":-726.150957803148,"aic":1458.3019156063,"aicc":1458.5519156063,"bic":1466.11742616426,"n":100,"py_pick":"ETS(M,N,N)","py_pick_aicc_in_r":1458.5519060697,"py_pick_admissible":1},"wwwusage":{"name":"wwwusage","x":[88,84,85,85,84,85,83,85,88,89,91,99,104,112,126,138,146,151,150,148,147,149,143,132,131,139,147,150,148,145,140,134,131,131,129,126,126,132,137,140,142,150,159,167,170,171,172,172,174,175,172,172,174,174,169,165,156,142,131,121,112,104,102,99,99,95,88,84,84,87,89,88,85,86,89,91,91,94,101,110,121,135,145,149,156,165,171,175,177,182,193,204,208,210,215,222,228,226,222,220],"period":1,"model_arg":"ZZZ","method":"ETS(A,Ad,N)","components":["A","A","N","TRUE"],"loglik":-352.865490198892,"aic":717.730980397785,"aicc":718.634206204236,"bic":733.362001513713,"n":100,"py_pick":"ETS(A,Ad,N)","py_pick_aicc_in_r":717.641639902903,"py_pick_admissible":1},"lynx":{"name":"lynx","x":[269,321,585,871,1475,2821,3928,5943,4950,2577,523,98,184,279,409,2285,2685,3409,1824,409,151,45,68,213,546,1033,2129,2536,957,361,377,225,360,731,1638,2725,2871,2119,684,299,236,245,552,1623,3311,6721,4254,687,255,473,358,784,1594,1676,2251,1426,756,299,201,229,469,736,2042,2811,4431,2511,389,73,39,49,59,188,377,1292,4031,3495,587,105,153,387,758,1307,3465,6991,6313,3794,1836,345,382,808,1388,2713,3800,3091,2985,3790,674,81,80,108,229,399,1132,2432,3574,2935,1537,529,485,662,1000,1590,2657,3396],"period":1,"model_arg":"ZZZ","method":"ETS(M,N,N)","components":["M","N","N","FALSE"],"loglik":-1026.06877340561,"aic":2058.13754681123,"aicc":2058.35572862941,"bic":2066.34614215641,"n":114,"py_pick":"ETS(M,A,N)","py_pick_aicc_in_r":2041.94549073361,"py_pick_admissible":1},"diff_nile":{"name":"diff_nile","x":[40,-197,247,-50,0,-347,417,140,-230,-145,-60,175,-116,26,-60,220,-381,159,182,-40,110,-60,100,10,-40,-190,70,-326,66,34,-180,246,-107,-132,215,-224,328,30,-81,-138,-105,-270,368,-122,418,-20,-268,-68,57,-53,77,19,-2,-164,147,-101,52,244,-281,22,84,-20,99,40,-87,-75,188,-239,-95,-27,197,-34,-70,59,239,-180,14,-26,42,-146,5,89,212,-132,68,-189,126,52,-160,205,-114,-5,269,-258,-166,173,-201,-4,26],"period":1,"model_arg":"ZZZ","method":"ETS(A,N,N)","components":["A","N","N","FALSE"],"loglik":-734.310989958834,"aic":1474.62197991767,"aicc":1474.87461149662,"bic":1482.40733946807,"n":99,"py_pick":"ETS(A,N,N)","py_pick_aicc_in_r":1474.87459815349,"py_pick_admissible":1},"airpassengers_zzn":{"name":"airpassengers_zzn","x":[112,118,132,129,121,135,148,148,136,119,104,118,115,126,141,135,125,149,170,170,158,133,114,140,145,150,178,163,172,178,199,199,184,162,146,166,171,180,193,181,183,218,230,242,209,191,172,194,196,196,236,235,229,243,264,272,237,211,180,201,204,188,235,227,234,264,302,293,259,229,203,229,242,233,267,269,270,315,364,347,312,274,237,278,284,277,317,313,318,374,413,405,355,306,271,306,315,301,356,348,355,422,465,467,404,347,305,336,340,318,362,348,363,435,491,505,404,359,310,337,360,342,406,396,420,472,548,559,463,407,362,405,417,391,419,461,472,535,622,606,508,461,390,432],"period":12,"model_arg":"ZZN","method":"ETS(M,N,N)","components":["M","N","N","FALSE"],"loglik":-833.950130596089,"aic":1673.90026119218,"aicc":1674.07168976361,"bic":1682.80970109091,"n":144,"py_pick":"ETS(M,A,N)","py_pick_aicc_in_r":1673.41133501112,"py_pick_admissible":1},"airpassengers_azz":{"name":"airpassengers_azz","x":[112,118,132,129,121,135,148,148,136,119,104,118,115,126,141,135,125,149,170,170,158,133,114,140,145,150,178,163,172,178,199,199,184,162,146,166,171,180,193,181,183,218,230,242,209,191,172,194,196,196,236,235,229,243,264,272,237,211,180,201,204,188,235,227,234,264,302,293,259,229,203,229,242,233,267,269,270,315,364,347,312,274,237,278,284,277,317,313,318,374,413,405,355,306,271,306,315,301,356,348,355,422,465,467,404,347,305,336,340,318,362,348,363,435,491,505,404,359,310,337,360,342,406,396,420,472,548,559,463,407,362,405,417,391,419,461,472,535,622,606,508,461,390,432],"period":12,"model_arg":"AZZ","method":"ETS(A,N,A)","components":["A","N","A","FALSE"],"loglik":-767.429538964832,"aic":1564.85907792966,"aicc":1568.60907792966,"bic":1609.4062774233,"n":144,"py_pick":"ETS(A,A,A)","py_pick_aicc_in_r":1475.85857573546,"py_pick_admissible":1},"airpassengers_mzz":{"name":"airpassengers_mzz","x":[112,118,132,129,121,135,148,148,136,119,104,118,115,126,141,135,125,149,170,170,158,133,114,140,145,150,178,163,172,178,199,199,184,162,146,166,171,180,193,181,183,218,230,242,209,191,172,194,196,196,236,235,229,243,264,272,237,211,180,201,204,188,235,227,234,264,302,293,259,229,203,229,242,233,267,269,270,315,364,347,312,274,237,278,284,277,317,313,318,374,413,405,355,306,271,306,315,301,356,348,355,422,465,467,404,347,305,336,340,318,362,348,363,435,491,505,404,359,310,337,360,342,406,396,420,472,548,559,463,407,362,405,417,391,419,461,472,535,622,606,508,461,390,432],"period":12,"model_arg":"MZZ","method":"ETS(M,Ad,M)","components":["M","A","M","TRUE"],"loglik":-679.583216236962,"aic":1395.16643247392,"aicc":1400.63843247392,"bic":1448.62307186629,"n":144,"py_pick":"ETS(M,A,M)","py_pick_aicc_in_r":1390.85155479901,"py_pick_admissible":1}},"fixed":{"nile_ann":{"name":"nile_ann","x":[1120,1160,963,1210,1160,1160,813,1230,1370,1140,995,935,1110,994,1020,960,1180,799,958,1140,1100,1210,1150,1250,1260,1220,1030,1100,774,840,874,694,940,833,701,916,692,1020,1050,969,831,726,456,824,702,1120,1100,832,764,821,768,845,864,862,698,845,744,796,1040,759,781,865,845,944,984,897,822,1010,771,676,649,846,812,742,801,1040,860,874,848,890,744,749,838,1050,918,986,797,923,975,815,1020,906,901,1170,912,746,919,718,714,740],"period":1,"model_arg":"ANN","damped":false,"method":"ETS(A,N,N)","loglik":-726.390519999206,"aic":1458.78103999841,"aicc":1459.03103999841,"bic":1466.59655055638,"alpha":0.245533862697156,"n":100},"nile_aan":{"name":"nile_aan","x":[1120,1160,963,1210,1160,1160,813,1230,1370,1140,995,935,1110,994,1020,960,1180,799,958,1140,1100,1210,1150,1250,1260,1220,1030,1100,774,840,874,694,940,833,701,916,692,1020,1050,969,831,726,456,824,702,1120,1100,832,764,821,768,845,864,862,698,845,744,796,1040,759,781,865,845,944,984,897,822,1010,771,676,649,846,812,742,801,1040,860,874,848,890,744,749,838,1050,918,986,797,923,975,815,1020,906,901,1170,912,746,919,718,714,740],"period":1,"model_arg":"AAN","damped":false,"method":"ETS(A,A,N)","loglik":-725.956062244941,"aic":1461.91212448988,"aicc":1462.55042236222,"bic":1474.93797541982,"alpha":0.209488918876854,"n":100},"airpassengers_aaa":{"name":"airpassengers_aaa","x":[112,118,132,129,121,135,148,148,136,119,104,118,115,126,141,135,125,149,170,170,158,133,114,140,145,150,178,163,172,178,199,199,184,162,146,166,171,180,193,181,183,218,230,242,209,191,172,194,196,196,236,235,229,243,264,272,237,211,180,201,204,188,235,227,234,264,302,293,259,229,203,229,242,233,267,269,270,315,364,347,312,274,237,278,284,277,317,313,318,374,413,405,355,306,271,306,315,301,356,348,355,422,465,467,404,347,305,336,340,318,362,348,363,435,491,505,404,359,310,337,360,342,406,396,420,472,548,559,463,407,362,405,417,391,419,461,472,535,622,606,508,461,390,432],"period":12,"model_arg":"AAA","damped":false,"method":"ETS(A,A,A)","loglik":-765.935847511721,"aic":1565.87169502344,"aicc":1570.72883788059,"bic":1616.35852111623,"alpha":0.993480362872374,"n":144},"airpassengers_mam":{"name":"airpassengers_mam","x":[112,118,132,129,121,135,148,148,136,119,104,118,115,126,141,135,125,149,170,170,158,133,114,140,145,150,178,163,172,178,199,199,184,162,146,166,171,180,193,181,183,218,230,242,209,191,172,194,196,196,236,235,229,243,264,272,237,211,180,201,204,188,235,227,234,264,302,293,259,229,203,229,242,233,267,269,270,315,364,347,312,274,237,278,284,277,317,313,318,374,413,405,355,306,271,306,315,301,356,348,355,422,465,467,404,347,305,336,340,318,362,348,363,435,491,505,404,359,310,337,360,342,406,396,420,472,548,559,463,407,362,405,417,391,419,461,472,535,622,606,508,461,390,432],"period":12,"model_arg":"MAM","damped":false,"method":"ETS(M,A,M)","loglik":-682.403619057803,"aic":1398.80723811561,"aicc":1403.66438097275,"bic":1449.2940642084,"alpha":0.394996850495005,"n":144},"usaccdeaths_ana":{"name":"usaccdeaths_ana","x":[9007,8106,8928,9137,10017,10826,11317,10744,9713,9938,9161,8927,7750,6981,8038,8422,8714,9512,10120,9823,8743,9129,8710,8680,8162,7306,8124,7870,9387,9556,10093,9620,8285,8466,8160,8034,7717,7461,7767,7925,8623,8945,10078,9179,8037,8488,7874,8647,7792,6957,7726,8106,8890,9299,10625,9302,8314,8850,8265,8796,7836,6892,7791,8192,9115,9434,10484,9827,9110,9070,8633,9240],"period":12,"model_arg":"ANA","damped":false,"method":"ETS(A,N,A)","loglik":-555.072335012621,"aic":1140.14467002524,"aicc":1148.71609859667,"bic":1174.29466181048,"alpha":0.594589890836137,"n":72}}}

pystatistics-4.6.3/tests/fixtures/generate_ets_py_params.py

@@ -0,0 +1,74 @@
+"""Dump PyStatistics' ZZZ-selected ETS fits for R cross-scoring.
+
+Stage 1 of regenerating ``ets_r_reference.json``.  Runs the package's own
+``ets()`` auto-selection on every reference series already stored in the
+fixture and writes the selected model plus its fitted parameters and
+initial states to ``ets_py_params.json``.  Stage 2
+(``generate_ets_r_reference.R``) feeds these parameters through R
+``forecast:::pegelsresid.C`` to score *our* fits under *R's own*
+likelihood — the honest cross-engine comparison (refitting the same model
+spec in R only re-runs R's optimiser, which is the thing being compared).
+
+Run from the repo root (after the fixture exists, since it supplies the
+input series)::
+
+    KMP_DUPLICATE_LIB_OK=TRUE PYTHONPATH=$PWD \
+        python tests/fixtures/generate_ets_py_params.py
+
+then::
+
+    Rscript tests/fixtures/generate_ets_r_reference.R
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import numpy as np
+
+from pystatistics.timeseries import ets
+
+FIXTURE_DIR = Path(__file__).parent
+REFERENCE = FIXTURE_DIR / "ets_r_reference.json"
+OUT = FIXTURE_DIR / "ets_py_params.json"
+
+
+def main() -> None:
+    reference = json.loads(REFERENCE.read_text())
+    out: dict[str, dict] = {}
+    for name, case in reference["selection"].items():
+        sol = ets(
+            np.asarray(case["x"], dtype=np.float64),
+            model=case["model_arg"],
+            period=case["period"],
+        )
+        spec = sol.spec
+        out[name] = {
+            "picked": spec.name,
+            "error": spec.error,
+            "trend": spec.trend,
+            "season": spec.season,
+            "damped": spec.damped,
+            "period": spec.period,
+            "alpha": sol.alpha,
+            "beta": sol.beta,
+            "gamma": sol.gamma,
+            "phi": sol.phi,
+            "init_level": sol.init_level,
+            "init_trend": sol.init_trend,
+            # Engine order: oldest first (s_{1-m} .. s_0).  R's state
+            # vector wants most-recent first — the R script reverses.
+            "init_season": (
+                list(sol.init_season) if sol.init_season is not None else None
+            ),
+            "aicc_py_convention": sol.aicc,
+            "converged": bool(sol.converged),
+        }
+        print(f"  {name:24s} -> {spec.name}")
+    OUT.write_text(json.dumps(out, indent=1) + "\n")
+    print(f"wrote {OUT.relative_to(FIXTURE_DIR.parent.parent)}")
+
+
+if __name__ == "__main__":
+    main()