besslib 0.1.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
besslib/soh/_core.py ADDED
@@ -0,0 +1,1081 @@
1
+ """The capacity estimator core: rest-anchored coulomb counting, one update per pair.
2
+
3
+ Each *pair* runs from one full or empty rest to the next linked one. Its measured
4
+ charge ``q`` (Ah, positive = charging) is modelled as
5
+
6
+ q = C * x + beta * dt
7
+
8
+ ``x`` is the change of state of charge: +1 from empty to full, -1 from full to empty,
9
+ 0 from full to full or empty to empty. ``dt`` is the time between the rests in days.
10
+ ``C`` is the usable capacity (Ah). ``beta`` (Ah per day) is the charge the meter does
11
+ not see: BMS supply, balancing, self-discharge and sensor offset. The *offset current*
12
+ is ``-beta / 24`` A, negative when the battery is drained, as in Figgener et al. (2024,
13
+ Supplementary Methods). The *fade model* adds a linear trend: ``C = c0 + r * y``, with
14
+ ``y`` in years since ``t0``.
15
+
16
+ Weights: ``1 / (sigma_q**2 + c_weight**2 * sigma_x**2)``, the effective variance of
17
+ Orear, Am. J. Phys. 50, 912 (1982), doi:10.1119/1.12972, with a fixed prior
18
+ ``c_weight`` (the nominal capacity), so the weights do not depend on the estimate.
19
+
20
+ Band: the weights fix only the relative size of the pair errors, and not even that
21
+ well: the scatter differs between kinds of pair (a full-to-full pair and an
22
+ empty-to-empty pair rest on different edges of the charge). And consecutive pairs share
23
+ a rest, so their errors are correlated at lag 1 and at no longer lag. The covariance is
24
+ therefore the sandwich ``V = A^-1 M A^-1 * m / (m - p)`` with the residuals' own
25
+ scores ``g_k = w_k a_k e_k``:
26
+
27
+ M = sum g_k g_k' + sum over linked pairs (g_k g_(k-1)' + g_(k-1) g_k')
28
+
29
+ the lag-1 heteroskedasticity- and autocorrelation-consistent form of Newey and West,
30
+ Econometrica 55, 703 (1987), doi:10.2307/1913610, with full weight at lag 1 instead of
31
+ their Bartlett weight 1/2, because the pair errors are known to be correlated at lag 1
32
+ only. ``m / (m - p)`` is the usual degrees-of-freedom factor, and the band uses a
33
+ Student-t quantile with ``m - p`` degrees of freedom. This form is not positive by
34
+ construction: a negative variance of the main quantity is reported as not possible,
35
+ never replaced by a floor. The current sensor's gain error is added to the capacity's
36
+ band as a systematic term. The Birge ratio (reduced chi-square) and the lag-1
37
+ correlation ``rho`` of the weighted residuals are reported as diagnostics; they no
38
+ longer scale the band.
39
+
40
+ The residuals are not known until the fit is solved, so ``M`` is kept as fixed-size
41
+ sums of fourth-order products of ``z = (a, q - c_weight * x)``; ``solve`` contracts them
42
+ with ``(-theta', 1)``. Subtracting ``c_weight * x`` keeps these sums small, which
43
+ matters in 32-bit arithmetic.
44
+
45
+ Old pairs fade by ``exp(-age / tau_days)``: a pair of age ``s`` counts as if its
46
+ variance were ``exp(s / tau_days)`` times larger (the convention of least squares with
47
+ forgetting). A lag-1 link counts with ``sqrt(lambda_k lambda_(k-1))``, which keeps the
48
+ two pairs of a link on the same footing. Related work: Plett, J. Power Sources 196, 2319
49
+ (2011), doi:10.1016/j.jpowsour.2010.09.048 (recursive total least squares for capacity;
50
+ not read, cited as related work).
51
+
52
+ The same file holds a second, smaller fit, the *through-origin ratio* (``Ratio``), for
53
+ quantities that are one sum divided by another: an event is two sums ``(a, b)`` with
54
+ ``a > 0``, and ``b = k a + e`` with an error whose variance grows with ``a`` (the event
55
+ is itself a sum of many small terms). Weighted least squares with weight ``1 / a`` gives
56
+ ``k = sum lambda b / sum lambda a``, and every event's score is ``g = b - k a``. The
57
+ growth form ``b = (k0 + k1 y) a`` uses the regressors ``(1, y) a`` with the same weight:
58
+ its normal matrix is ``sum lambda a (1, y)(1, y)'`` and its scores are ``(1, y) g``. The
59
+ lag-1 sandwich, the ``m / (m - p)`` factor with ``m = sum lambda``, the Student-t
60
+ quantile, ``ROUNDING`` and ``NO_BAND`` follow the capacity fit. Four choices are this
61
+ fit's own.
62
+
63
+ First, the forgetting weight enters the band's sums twice, as the weight of a weighted
64
+ least-squares fit does: the score of event k is ``lambda_k (1, y) g_k``, so ``M = sum
65
+ lambda_k^2 g g' + sum over links lambda_k lambda_(k-1) (g_k g_(k-1)' + g_(k-1) g_k')``.
66
+ For ``k = sum lambda b / sum lambda a`` with independent errors this is ``sum lambda^2
67
+ Var(b) / (sum lambda a)^2``, the variance of the estimate as computed. The capacity fit
68
+ keeps the weight once, which is exact only if an event's variance grew as
69
+ ``exp(age / tau_days)``; for events whose errors do not grow with age that band is about
70
+ ``sqrt(2)`` too wide at ``tau_days`` of memory (``(sum lambda)^2 / sum lambda^2`` is
71
+ close to ``2 sum lambda``). The capacity fit is left as it is (its settings and results
72
+ are frozen); this fit uses the weight twice. Without forgetting both are the same.
73
+ ``m = sum lambda`` stays the count for ``m / (m - p)``, the degrees of freedom and the
74
+ minimum number of events. It is half the effective number of events of the estimate,
75
+ ``(sum lambda)^2 / sum lambda^2``, so as a minimum count it errs on the cautious side;
76
+ and it is close to the effective number of the band's own sums, ``(sum lambda^2)^2 /
77
+ sum lambda^4``, which sets how well the variance itself is known (for evenly spaced
78
+ events the two differ by the factor ``(1 + r^2) / (1 + r)``, ``r`` the weight ratio of
79
+ neighbouring events: within 0.3 % for daily events at 180 days of memory).
80
+
81
+ Second, the degrees of freedom of the band are those of its variance estimate. Without
82
+ links that estimate is a sum of squared scores, and the band uses ``m - p`` degrees of
83
+ freedom, as the capacity fit does. A lag-1 term adds products of neighbouring scores,
84
+ which scatter more than they add: when a shared end point dominates the errors, the
85
+ squares and the products nearly cancel and their sum is poorly known. The count used
86
+ here is derived here, as follows. Take scores of equal variance 1, normal, correlated
87
+ by ``rho`` across a link and independent otherwise, and the meat ``S = sum g_k^2 + 2
88
+ sum over links g_k g_(k-1)`` over ``n`` events. The fourth moments of normal variables
89
+ (``E[g_a g_b g_c g_d]`` is the sum of the three products of pair covariances) give
90
+ ``E[S] = n (1 + 2 rho q)`` and ``Var[S] = 2 n (1 + 2 q + 8 rho q + 4 rho^2 (q + q2))``,
91
+ where ``q`` is the share of events linked to the one before and ``q2`` the share whose
92
+ link follows another link: ``2 n + 4 rho^2 N_L`` from the squares, ``4 (1 + rho^2) N_L +
93
+ 8 rho^2 N_LL`` from the products, ``16 rho N_L`` from their covariance (``N_L = q n``
94
+ links, ``N_LL = q2 n`` pairs of consecutive links). A chi-square variable with the same
95
+ mean and variance has ``2 E[S]^2 / Var[S] = n (1 + 2 rho q)^2 / (1 + 2 q + 8 rho q + 4
96
+ rho^2 (q + q2))`` degrees of freedom: ``n`` without links, ``n / 3`` for a fully linked
97
+ chain without correlation, and towards zero as ``rho q`` nears ``-1/2`` (a chain fully
98
+ linked at ``rho = -0.45`` keeps about 1 % of ``n``). The band uses ``m - p`` times this
99
+ share, with ``rho``, ``q`` and ``q2`` from the weighted sums (``rho`` is the reported
100
+ diagnostic). Without links it is exactly the capacity fit's ``m - p``. At very few
101
+ degrees of freedom ``t_quantile`` (four terms) runs low: 5.57 against 6.31 at one degree
102
+ of freedom and 90 %; below one it is held at one.
103
+
104
+ Third, the growth fit needs both of its variances (callers form ratios of its
105
+ parameters), so a negative variance of either gives ``NO_BAND``. Fourth, the numerator
106
+ is centred on the ratio of the first event, ``b - k_ref a``: the sums of the band then
107
+ hold the residuals' size, not the size of ``b``, so a 32-bit port loses about two
108
+ fewer digits to cancellation. No other scaling is needed: the largest sum is of order
109
+ ``m a^2``, far below the 32-bit limit for any plausible event.
110
+
111
+ The *loop band* (``loop_solve``) is a second band on the same state, for closed loops
112
+ between rests: an event is a loop from one rest to the next rest of the same kind, ``a``
113
+ its energy in and ``b`` its energy out. The energy stored at rest ``j`` differs from the
114
+ reference by an error ``e_j``: the state of charge at a voltage stop is not exactly the
115
+ same each time. A loop from rest ``j - 1`` to rest ``j`` gives ``b_k = eta_k a_k +
116
+ e_(j-1) - e_j``, where ``eta_k a_k`` is what the loop would deliver had it closed
117
+ exactly. The target is ``T = sum lambda eta a / sum lambda a``, the ratio over the loops
118
+ in memory had each of them closed; the loops' own variation (``eta_k``) is part of the
119
+ target, not of the error. The estimate is the same ``R = sum lambda b / sum lambda a``,
120
+ and ``R - T = sum_j c_j e_j / A`` with ``A = sum lambda a`` and ``c_j`` the coefficient
121
+ of rest ``j``: ``+lambda_k`` as the start of loop ``k``, ``-lambda_k`` as its end,
122
+ ``lambda_k - lambda_(k-1)`` for a rest that ends loop ``k - 1`` and starts the linked
123
+ loop ``k``. With independent rest errors of one variance ``s2``, ``Var(R) = s2 C2 /
124
+ A^2`` with ``C2 = sum_j c_j^2``. Without forgetting ``C2`` is 2 per linked run of loops
125
+ (its first and last rest); with forgetting the latest rest, at weight 1, dominates it.
126
+ ``C2`` is kept recursively: on each accepted event, after decaying the state to its
127
+ time, a linked event adds ``2 - 2 g`` (the old end rest ``-g`` becomes the shared rest
128
+ ``1 - g``, and a new end rest ``-1`` comes in), an unlinked one adds 2, with ``g`` the
129
+ previous event's weight now; decay multiplies ``C2`` by ``f^2``.
130
+
131
+ ``s2`` is estimated from the lag-1 and the lag-2 products of the scores ``g_k = b_k - R
132
+ a_k``. Write ``g_k = nu_k + e_(j-1) - e_j``, with ``nu`` the loop's own part, which may
133
+ drift slowly (season, temperature). Two linked loops share a rest, which enters them with
134
+ opposite signs: ``E[g_k g_(k-1)] = gamma_nu(1) - s2``. Loops ``k`` and ``k - 2`` of three
135
+ in a row share no rest: ``E[g_k g_(k-2)] = gamma_nu(2)``. Hence ``s2 = M2 - M1`` with
136
+ ``M1 = sum w1 g_k g_(k-1) / sum w1`` over links (``w1 = lambda_k lambda_(k-1)``) and ``M2
137
+ = sum w2 g_k g_(k-2) / sum w2`` over *2-links*, where loop ``k`` is linked to ``k - 1``
138
+ and ``k - 1`` to ``k - 2`` (``w2 = lambda_k lambda_(k-2)``). It is unbiased when the own
139
+ part's autocovariance is the same at lag 1 and lag 2: no drift, or one slow against two
140
+ loops. No 2-link in memory gives ``NO_LINKS``; a negative ``s2`` beyond ``ROUNDING`` (of
141
+ the two means' sums in absolute values) gives ``NO_BAND``.
142
+
143
+ Degrees of freedom of ``s2``: take the scores normal, of one variance ``gamma0``, with
144
+ autocovariances ``gamma1`` and ``gamma2`` within a linked run and none at longer lags or
145
+ across a break, estimated by the weighted mean square, ``M1`` and ``M2``. The fourth
146
+ moments of normal variables (``Cov(g_a g_b, g_c g_d) = gamma(a-c) gamma(b-d) + gamma(a-d)
147
+ gamma(b-c)``) give ``Var(s2_hat) = V11 + V22 - 2 V12`` with
148
+
149
+ V11 W1^2 = (gamma0^2 + gamma1^2) S11_0 + 2 (gamma1^2 + gamma0 gamma2) S11_1 + 2 gamma2^2 S11_2
150
+ V22 W2^2 = (gamma0^2 + gamma2^2) S22_0 + 2 gamma1^2 S22_1 + 2 gamma2^2 S22_2
151
+ V12 W1 W2 = gamma1 (gamma0 + gamma2) (X_0 + X_-1) + gamma1 gamma2 (X_+1 + X_-2)
152
+
153
+ where ``W1 = sum w1``, ``W2 = sum w2``, ``S11_d`` is the sum of ``w1_k w1_(k-d)`` over
154
+ links ``d`` apart in one run, ``S22_d`` the same for 2-links, and ``X_d`` the sum of
155
+ ``w1_k w2_(k-d)`` over a link and a 2-link ``d`` apart in one run (``X_-1``: the link
156
+ just before the 2-link's last loop, ``X_-2``: the one before that). All are fixed-size
157
+ sums kept on each event, with the weights of the last two links and 2-links in the
158
+ state (set to 0 at a break). A chi-square variable with the same mean and variance has
159
+ ``2 s2^2 / Var(s2_hat)`` degrees of freedom. A pure chain of ``N`` equal links of rest
160
+ errors has ``2 N / 21`` (against ``2 N / 7`` for lag-1 products alone): the difference
161
+ of two means is noisier than one. Below one degree of freedom, or with a variance of
162
+ ``s2`` that comes out at or below zero, the status is ``LOW_DOF``. The band is ``R +- t
163
+ sqrt(s2 C2) / A`` with the Student-t quantile at these degrees of freedom; nothing else
164
+ multiplies it.
165
+
166
+ Its limits: it holds the ratio over these loops, not the efficiency under other use; the
167
+ rest errors are taken as independent and of one size, which they are not quite (the
168
+ state of charge at a voltage stop depends on the current before it); a drift of the
169
+ loops' own efficiency that changes within two loops still biases ``s2`` (a drift whose
170
+ lag-1 autocovariance exceeds its lag-2 one makes ``s2`` and the band too small); the
171
+ degrees of freedom assume normal scores and no autocovariance beyond lag 2; the latest
172
+ rest dominates ``C2``, so the band rests on the error of one rest whose size is learnt
173
+ from the others; a zero offset of the current sensor does not cancel and is not in the
174
+ band (a gain error cancels in the ratio).
175
+
176
+ Portability rule: plain float arithmetic, a fixed-size state (arrays of length 3, 9 and
177
+ 144; for the ratio fit 2, 3, 4, 9 and 36), no numpy, no pandas, no growing containers, no
178
+ recursion, no exceptions for control flow. Each class maps to a C struct, each function
179
+ to a C function. The test vectors live in ``tests/vectors/``.
180
+ """
181
+
182
+ from __future__ import annotations
183
+
184
+ import math
185
+
186
+ #: Status codes of an estimate. A C port uses the same integers.
187
+ OK = 0
188
+ TOO_FEW_FULL_EMPTY = 1
189
+ TOO_FEW_SAME_KIND = 2
190
+ SINGULAR = 3
191
+ STALE = 4
192
+ NO_BAND = 5
193
+ NO_LEVELS = 6 # set by the caller: the core sees pairs, not the rests that set the levels
194
+ SHORT_SPAN = 7
195
+ TOO_FEW_EVENTS = 8
196
+ OUT_OF_RANGE = 9
197
+ NO_LINKS = 10
198
+ LOW_DOF = 11
199
+
200
+ STATUS_TEXT = {
201
+ OK: "estimated",
202
+ TOO_FEW_FULL_EMPTY: "not possible from this data: too few full-to-empty or empty-to-full pairs",
203
+ TOO_FEW_SAME_KIND: "not possible from this data: too few full-to-full or empty-to-empty pairs to fix the offset",
204
+ SINGULAR: "not possible from this data: the pairs do not separate capacity from offset",
205
+ STALE: "not possible from this data: no full-to-empty or empty-to-full pair within the memory time",
206
+ NO_BAND: "not possible from this data: the residuals give no positive variance for the band",
207
+ NO_LEVELS: "not possible from this data: not enough history yet to set the full and empty levels",
208
+ SHORT_SPAN: "not possible from this data: the full-to-empty and empty-to-full pairs span too short a time",
209
+ TOO_FEW_EVENTS: "not possible from this data: too few events in memory",
210
+ OUT_OF_RANGE: "not possible from this data: the estimate lies outside the physically possible range",
211
+ NO_LINKS: "not possible from this data: no three events in a row in memory share their end points",
212
+ LOW_DOF: "not possible from this data: the band rests on less than one degree of freedom",
213
+ }
214
+
215
+ #: Relative rounding tolerance of the band's sums, for 64-bit arithmetic. A 32-bit port
216
+ #: needs a larger one, set from its own test vectors.
217
+ ROUNDING = 1.0e-9
218
+
219
+ _NZ = 4 # length of z = (regressors, centred charge); index of s4 is ((r * 3 + s) * 4 + i) * 4 + j
220
+
221
+ _NEVER = -1.0e30
222
+
223
+
224
+ class Fit:
225
+ """Fixed-size state of a weighted least-squares fit with forgetting and lag-1 sums.
226
+
227
+ ``p`` regressors (2 or 3). ``a`` = sum w a a' (row-major 3 x 3), ``h`` = sum w a q,
228
+ ``c`` = sum w q q; ``l`` = sum v a_k a_(k-1)', ``ly`` = sum v q_k a_(k-1),
229
+ ``yl`` = sum v a_k q_(k-1), ``yy1`` = sum v q_k q_(k-1), with v = g sqrt(w_k w_(k-1)),
230
+ over pairs that share a rest with the pair before them; ``g`` =
231
+ exp(-(t_k - t_(k-1)) / (2 tau_days)) makes a link decay as sqrt(lambda_k lambda_(k-1)).
232
+ ``s4`` = sum w^2 a_r a_s z_i z_j and ``s4l`` = sum g w_k w_(k-1) (a_k,r a_(k-1),s +
233
+ a_(k-1),r a_k,s) z_k,i z_(k-1),j, with z = (a, q - c_weight x): the sums of the band's
234
+ sandwich (3 x 3 x 4 x 4 each).
235
+ """
236
+
237
+ __slots__ = (
238
+ "p", "c_weight", "tau_days", "t_last", "t_last_cross", "t0_days",
239
+ "a", "h", "c", "l", "ly", "yl", "yy1", "s4", "s4l",
240
+ "m", "m_lag", "m_cross", "m_same", "n", "y_min", "y_max",
241
+ "prev_ok", "prev_a", "prev_q", "prev_w", "prev_t", "prev_z",
242
+ )
243
+
244
+ def __init__(self, p: int, c_weight: float, tau_days: float, t0_days: float = 0.0) -> None:
245
+ self.p = int(p)
246
+ self.c_weight = float(c_weight)
247
+ self.tau_days = float(tau_days) # memory time; 0 or less means no forgetting
248
+ self.t0_days = float(t0_days) # origin of the fade model's years
249
+ self.t_last = _NEVER
250
+ self.t_last_cross = _NEVER
251
+ self.a = [0.0] * 9
252
+ self.h = [0.0] * 3
253
+ self.c = 0.0
254
+ self.l = [0.0] * 9
255
+ self.ly = [0.0] * 3
256
+ self.yl = [0.0] * 3
257
+ self.yy1 = 0.0
258
+ self.s4 = [0.0] * 144
259
+ self.s4l = [0.0] * 144
260
+ self.m = 0.0 # decayed number of pairs
261
+ self.m_lag = 0.0 # decayed number of lag-1 links
262
+ self.m_cross = 0.0 # decayed number of pairs with x != 0
263
+ self.m_same = 0.0 # decayed number of pairs with x == 0
264
+ self.n = 0 # pairs ever added
265
+ self.y_min = 1.0e30 # first and last year of a full-to-empty or empty-to-full pair
266
+ self.y_max = -1.0e30
267
+ self.prev_ok = 0
268
+ self.prev_a = [0.0] * 3
269
+ self.prev_q = 0.0
270
+ self.prev_w = 0.0
271
+ self.prev_t = _NEVER
272
+ self.prev_z = [0.0] * _NZ
273
+
274
+
275
+ def capacity_state(c_weight: float, tau_days: float) -> Fit:
276
+ """The capacity estimator: regressors (x, dt)."""
277
+ return Fit(2, c_weight, tau_days)
278
+
279
+
280
+ def fade_state(c_weight: float, t0_days: float) -> Fit:
281
+ """The fade model: regressors (x, x * y, dt), no forgetting."""
282
+ return Fit(3, c_weight, 0.0, t0_days)
283
+
284
+
285
+ def decay(state: Fit, t_days: float) -> None:
286
+ """Move the state forward to ``t_days``; old pairs lose weight."""
287
+ if state.n == 0:
288
+ state.t_last = t_days
289
+ return
290
+ age = t_days - state.t_last
291
+ if age <= 0.0:
292
+ return
293
+ state.t_last = t_days
294
+ if state.tau_days <= 0.0:
295
+ return
296
+ f = math.exp(-age / state.tau_days)
297
+ for k in range(9):
298
+ state.a[k] *= f
299
+ state.l[k] *= f
300
+ for k in range(144):
301
+ state.s4[k] *= f
302
+ state.s4l[k] *= f
303
+ for k in range(3):
304
+ state.h[k] *= f
305
+ state.ly[k] *= f
306
+ state.yl[k] *= f
307
+ state.c *= f
308
+ state.yy1 *= f
309
+ state.m *= f
310
+ state.m_lag *= f
311
+ state.m_cross *= f
312
+ state.m_same *= f
313
+
314
+
315
+ def update(
316
+ state: Fit,
317
+ t_days: float,
318
+ x: float,
319
+ dt_days: float,
320
+ q_ah: float,
321
+ sigma_q_ah: float,
322
+ sigma_x: float,
323
+ linked: int,
324
+ ) -> None:
325
+ """Add one pair that ends at ``t_days`` (pairs in time order).
326
+
327
+ ``linked`` is 1 when the pair starts at the rest where the previous pair ended.
328
+ """
329
+ decay(state, t_days)
330
+ w = 1.0 / (sigma_q_ah * sigma_q_ah + state.c_weight * state.c_weight * sigma_x * sigma_x)
331
+ y = (t_days - state.t0_days) / 365.25
332
+ row = [x, dt_days, 0.0]
333
+ if state.p == 3:
334
+ row = [x, x * y, dt_days]
335
+ p = state.p
336
+ z = [0.0] * _NZ
337
+ for r in range(p):
338
+ z[r] = row[r]
339
+ z[p] = q_ah - state.c_weight * x
340
+ for r in range(p):
341
+ for k in range(p):
342
+ state.a[3 * r + k] += w * row[r] * row[k]
343
+ state.h[r] += w * row[r] * q_ah
344
+ state.c += w * q_ah * q_ah
345
+ for r in range(p):
346
+ for s in range(p):
347
+ ww = w * w * row[r] * row[s]
348
+ for i in range(p + 1):
349
+ for j in range(p + 1):
350
+ state.s4[((r * 3 + s) * 4 + i) * 4 + j] += ww * z[i] * z[j]
351
+ if linked != 0 and state.prev_ok != 0:
352
+ g = 1.0
353
+ if state.tau_days > 0.0:
354
+ g = math.exp(-(t_days - state.prev_t) / (2.0 * state.tau_days))
355
+ v = g * math.sqrt(w * state.prev_w)
356
+ for r in range(p):
357
+ for k in range(p):
358
+ state.l[3 * r + k] += v * row[r] * state.prev_a[k]
359
+ state.ly[r] += v * q_ah * state.prev_a[r]
360
+ state.yl[r] += v * row[r] * state.prev_q
361
+ state.yy1 += v * q_ah * state.prev_q
362
+ state.m_lag += g
363
+ u = g * w * state.prev_w
364
+ for r in range(p):
365
+ for s in range(p):
366
+ aa = u * (row[r] * state.prev_a[s] + state.prev_a[r] * row[s])
367
+ for i in range(p + 1):
368
+ for j in range(p + 1):
369
+ state.s4l[((r * 3 + s) * 4 + i) * 4 + j] += aa * z[i] * state.prev_z[j]
370
+ state.prev_ok = 1
371
+ for r in range(3):
372
+ state.prev_a[r] = row[r]
373
+ for r in range(_NZ):
374
+ state.prev_z[r] = z[r]
375
+ state.prev_q = q_ah
376
+ state.prev_w = w
377
+ state.prev_t = t_days
378
+ state.m += 1.0
379
+ if x != 0.0:
380
+ state.m_cross += 1.0
381
+ state.t_last_cross = t_days
382
+ if y < state.y_min:
383
+ state.y_min = y
384
+ if y > state.y_max:
385
+ state.y_max = y
386
+ else:
387
+ state.m_same += 1.0
388
+ state.n += 1
389
+
390
+
391
+ def _invert(p: int, a: list, out: list) -> int:
392
+ """Invert the symmetric p x p matrix ``a`` (p = 2 or 3, row-major in 3 x 3) into ``out``."""
393
+ for k in range(9):
394
+ out[k] = 0.0
395
+ if p == 2:
396
+ det = a[0] * a[4] - a[1] * a[3]
397
+ if a[0] <= 0.0 or a[4] <= 0.0 or det <= 1.0e-9 * a[0] * a[4]:
398
+ return 0
399
+ out[0] = a[4] / det
400
+ out[1] = -a[1] / det
401
+ out[3] = -a[3] / det
402
+ out[4] = a[0] / det
403
+ return 1
404
+ a11, a12, a13, a22, a23, a33 = a[0], a[1], a[2], a[4], a[5], a[8]
405
+ c11 = a22 * a33 - a23 * a23
406
+ c12 = a13 * a23 - a12 * a33
407
+ c13 = a12 * a23 - a13 * a22
408
+ c22 = a11 * a33 - a13 * a13
409
+ c23 = a12 * a13 - a11 * a23
410
+ c33 = a11 * a22 - a12 * a12
411
+ det = a11 * c11 + a12 * c12 + a13 * c13
412
+ if a11 <= 0.0 or a22 <= 0.0 or a33 <= 0.0 or det <= 1.0e-12 * a11 * a22 * a33:
413
+ return 0
414
+ vals = (c11, c12, c13, c12, c22, c23, c13, c23, c33)
415
+ for k in range(9):
416
+ out[k] = vals[k] / det
417
+ return 1
418
+
419
+
420
+ def t_quantile(z: float, nu: float) -> float:
421
+ """Student-t quantile from the normal quantile ``z`` (Cornish-Fisher, four terms).
422
+
423
+ Abramowitz and Stegun, Handbook of Mathematical Functions (1964), 26.7.5.
424
+ ``nu`` below 1 is treated as 1.
425
+ """
426
+ if nu < 1.0:
427
+ nu = 1.0
428
+ z2 = z * z
429
+ g1 = (z2 + 1.0) * z / 4.0
430
+ g2 = ((5.0 * z2 + 16.0) * z2 + 3.0) * z / 96.0
431
+ g3 = (((3.0 * z2 + 19.0) * z2 + 17.0) * z2 - 15.0) * z / 384.0
432
+ return z + g1 / nu + g2 / (nu * nu) + g3 / (nu * nu * nu)
433
+
434
+
435
+ class Estimate:
436
+ """The result of ``solve`` (the C port's output struct). ``theta`` = (C, beta) for the
437
+ capacity fit, (c0, r, beta) for the fade fit; ``sigma`` their standard errors."""
438
+
439
+ __slots__ = ("status", "theta", "sigma", "low", "high", "chi2", "dof", "birge", "rho", "m", "m_cross", "m_same", "span_years")
440
+
441
+ def __init__(self) -> None:
442
+ self.status = SINGULAR
443
+ self.theta = [math.nan] * 3
444
+ self.sigma = [math.nan] * 3
445
+ self.low = math.nan # band of the main quantity: C, or r for the fade fit
446
+ self.high = math.nan
447
+ self.chi2 = math.nan
448
+ self.dof = 0.0
449
+ self.birge = math.nan
450
+ self.rho = math.nan
451
+ self.m = 0.0
452
+ self.m_cross = 0.0
453
+ self.m_same = 0.0
454
+ self.span_years = 0.0
455
+
456
+
457
+ def solve(
458
+ state: Fit,
459
+ t_days: float,
460
+ z: float,
461
+ gain_error: float,
462
+ min_cross: float,
463
+ min_same: float,
464
+ min_span_years: float,
465
+ ) -> Estimate:
466
+ """The estimate at ``t_days`` (the state is decayed to it first).
467
+
468
+ ``z`` is the two-sided normal quantile of the band (1.6449 for 90 %). The band of the
469
+ main quantity is the Student-t quantile times its sandwich standard error (module
470
+ docstring). A parameter other than the main one whose sandwich variance comes out
471
+ negative gets ``sigma`` NaN. For the
472
+ capacity fit, ``gain_error`` (the current sensor's relative accuracy) is added to the
473
+ band of C as a systematic term: it moves every pair the same way, so averaging does
474
+ not reduce it. For the fade fit it is left out (it scales the rate by at most that
475
+ fraction). ``min_span_years`` applies to the fade fit only.
476
+ """
477
+ decay(state, t_days)
478
+ out = Estimate()
479
+ out.m = state.m
480
+ out.m_cross = state.m_cross
481
+ out.m_same = state.m_same
482
+ if state.m_cross > 0.0:
483
+ out.span_years = state.y_max - state.y_min
484
+ if state.m_cross < min_cross:
485
+ out.status = TOO_FEW_FULL_EMPTY
486
+ return out
487
+ if state.m_same < min_same:
488
+ out.status = TOO_FEW_SAME_KIND
489
+ return out
490
+ p = state.p
491
+ if p == 2 and state.tau_days > 0.0 and t_days - state.t_last_cross > state.tau_days:
492
+ out.status = STALE
493
+ return out
494
+ if p == 3 and out.span_years < min_span_years:
495
+ out.status = SHORT_SPAN
496
+ return out
497
+ inv = [0.0] * 9
498
+ if _invert(p, state.a, inv) == 0:
499
+ out.status = SINGULAR
500
+ return out
501
+ th = [0.0, 0.0, 0.0]
502
+ for r in range(p):
503
+ acc = 0.0
504
+ for k in range(p):
505
+ acc += inv[3 * r + k] * state.h[k]
506
+ th[r] = acc
507
+ chi2 = state.c
508
+ for r in range(p):
509
+ chi2 -= th[r] * state.h[r]
510
+ if chi2 < 0.0:
511
+ chi2 = 0.0
512
+ nu = state.m - p
513
+ out.chi2 = chi2
514
+ out.dof = nu
515
+ if nu <= 0.0:
516
+ out.status = NO_BAND
517
+ return out
518
+ out.birge = chi2 / nu # diagnostic: 1 when the weights' error sizes match the scatter
519
+ # Lag-1 correlation of the weighted residuals (diagnostic): sum e_k e_(k-1) from the lag sums.
520
+ out.rho = 0.0
521
+ if state.m_lag > 1.0 and chi2 > 0.0:
522
+ cross = state.yy1
523
+ for r in range(p):
524
+ cross -= th[r] * (state.ly[r] + state.yl[r])
525
+ for k in range(p):
526
+ cross += th[r] * state.l[3 * r + k] * th[k]
527
+ out.rho = (cross / state.m_lag) / (chi2 / state.m)
528
+ # The sandwich's meat M_rs = phi' (S_rs + SL_rs) phi with phi = (-theta + c_weight e_0, 1).
529
+ phi = [0.0] * _NZ
530
+ for r in range(p):
531
+ phi[r] = -th[r]
532
+ phi[0] += state.c_weight
533
+ phi[p] = 1.0
534
+ meat = [0.0] * 9
535
+ size = [0.0] * 9 # the same sums in absolute values: the scale of their rounding error
536
+ for r in range(p):
537
+ for s in range(p):
538
+ acc = 0.0
539
+ mag = 0.0
540
+ for i in range(p + 1):
541
+ for j in range(p + 1):
542
+ idx = ((r * 3 + s) * 4 + i) * 4 + j
543
+ acc += phi[i] * (state.s4[idx] + state.s4l[idx]) * phi[j]
544
+ mag += abs(phi[i] * phi[j]) * (abs(state.s4[idx]) + abs(state.s4l[idx]))
545
+ meat[3 * r + s] = acc
546
+ size[3 * r + s] = mag
547
+ scale = state.m / nu
548
+ main = 0 if p == 2 else 1
549
+ for r in range(p):
550
+ acc = 0.0
551
+ mag = 0.0
552
+ for i in range(p):
553
+ for j in range(p):
554
+ acc += inv[3 * r + i] * meat[3 * i + j] * inv[3 * j + r]
555
+ mag += abs(inv[3 * r + i] * inv[3 * j + r]) * size[3 * i + j]
556
+ # A negative value within rounding of the sums is zero (a fit with no scatter).
557
+ if acc < 0.0 and -acc <= ROUNDING * mag:
558
+ acc = 0.0
559
+ var = scale * acc
560
+ if var >= 0.0:
561
+ out.sigma[r] = math.sqrt(var)
562
+ elif r == main:
563
+ out.status = NO_BAND
564
+ return out
565
+ for r in range(p):
566
+ out.theta[r] = th[r]
567
+ k = t_quantile(z, nu)
568
+ sys_part = gain_error * th[0] if p == 2 else 0.0
569
+ half = math.sqrt(k * k * out.sigma[main] * out.sigma[main] + z * z * sys_part * sys_part)
570
+ if p == 2:
571
+ out.sigma[0] = math.sqrt(out.sigma[0] * out.sigma[0] + sys_part * sys_part)
572
+ out.low = th[main] - half
573
+ out.high = th[main] + half
574
+ out.status = OK
575
+ return out
576
+
577
+
578
+ # --- the through-origin ratio fit ------------------------------------------------------
579
+
580
+ _NR = 3 # length of z = (regressors, centred numerator); index of s4 is ((r * 2 + s) * 3 + i) * 3 + j
581
+
582
+
583
+ class Ratio:
584
+ """Fixed-size state of the through-origin fit ``b = (theta . u) a`` with weight ``1 / a``.
585
+
586
+ ``p = 1``: ``u = (1,)``, ``theta = (k,)``. ``p = 2``: ``u = (1, y)`` with ``y`` in years
587
+ since ``t0_days``, ``theta = (k0, k1)``. ``a`` = sum lambda a u u' (row-major 2 x 2),
588
+ ``h`` = sum lambda u (b - k_ref a); ``s4`` = sum lambda^2 u_r u_s z_i z_j and ``s4l`` =
589
+ sum lambda_k lambda_(k-1) (u_k,r u_(k-1),s + u_(k-1),r u_k,s) z_k,i z_(k-1),j over
590
+ linked events, with z = (a u, b - k_ref a) (2 x 2 x 3 x 3 each). ``k_ref`` is b / a of
591
+ the first event the state accepted. ``m`` = sum lambda and ``m_lag`` = sum
592
+ sqrt(lambda_k lambda_(k-1)) are the decayed numbers of events and of links (as in the
593
+ capacity fit); ``m2`` = sum lambda^2 and ``m2_lag`` = sum lambda_k lambda_(k-1) are the
594
+ same counts with the weights of the band's sums, and ``m2_ll`` the same over links
595
+ whose previous event is linked too; they set ``rho`` and the degrees of freedom.
596
+ The loop band (``loop_solve``) adds ``c2`` = sum over rests of their squared
597
+ coefficients, ``w4`` = sum (lambda_k lambda_(k-1))^2 over links, ``w4l`` = sum of
598
+ the products of consecutive link weights, and ``prev_lw``, the weight of the link
599
+ that ended at the previous event now (0 when that event was not linked). For the
600
+ lag-2 products: ``s4l2`` = sum w2 z_k,i z_(k-2),j (index i * 3 + j) and ``m2_l2`` =
601
+ sum w2 over 2-links (``w2 = lambda_k lambda_(k-2)``, loop k linked to k - 1 and k - 1
602
+ to k - 2); ``s11_2``, ``s22_0``, ``s22_1``, ``s22_2``, ``x0``, ``xm1``, ``xp1``,
603
+ ``xm2``, the sums of weight products that set the degrees of freedom (module
604
+ docstring); ``prev_lw2``, ``prev_w2`` and ``prev_w2b``, the weights now of the link
605
+ two events back and of the 2-links one and two events back (0 across a break); and
606
+ ``prev2_t``, ``prev2_z``, the time and ``z`` of the event two back.
607
+ """
608
+
609
+ __slots__ = (
610
+ "p", "tau_days", "t0_days", "t_last", "k_ref", "has_ref", "a", "h", "s4", "s4l",
611
+ "m", "m_lag", "m2", "m2_lag", "m2_ll", "n", "n_skipped", "y_min", "y_max",
612
+ "prev_ok", "prev_linked", "prev_t", "prev_u", "prev_z", "c2", "w4", "w4l", "prev_lw",
613
+ "s4l2", "m2_l2", "s11_2", "s22_0", "s22_1", "s22_2", "x0", "xm1", "xp1", "xm2",
614
+ "prev_lw2", "prev_w2", "prev_w2b", "prev2_t", "prev2_z",
615
+ )
616
+
617
+ def __init__(self, p: int, tau_days: float, t0_days: float = 0.0) -> None:
618
+ self.p = int(p)
619
+ self.tau_days = float(tau_days) # memory time; 0 or less means no forgetting
620
+ self.t0_days = float(t0_days) # origin of the growth fit's years
621
+ self.t_last = _NEVER
622
+ self.k_ref = 0.0
623
+ self.has_ref = 0
624
+ self.a = [0.0] * 4
625
+ self.h = [0.0] * 2
626
+ self.s4 = [0.0] * 36
627
+ self.s4l = [0.0] * 36
628
+ self.m = 0.0 # decayed number of events
629
+ self.m_lag = 0.0 # decayed number of lag-1 links
630
+ self.m2 = 0.0 # sum of lambda^2
631
+ self.m2_lag = 0.0 # sum of lambda_k lambda_(k-1) over links
632
+ self.m2_ll = 0.0 # the same over links that follow a link
633
+ self.n = 0 # events ever accepted
634
+ self.n_skipped = 0 # events refused: a <= 0, or a or b not finite
635
+ self.y_min = 1.0e30 # first and last year of an accepted event
636
+ self.y_max = -1.0e30
637
+ self.prev_ok = 0
638
+ self.prev_linked = 0
639
+ self.prev_t = _NEVER
640
+ self.prev_u = [0.0] * 2
641
+ self.prev_z = [0.0] * _NR
642
+ self.c2 = 0.0 # sum of the rests' squared coefficients (loop band)
643
+ self.w4 = 0.0 # sum of (lambda_k lambda_(k-1))^2 over links
644
+ self.w4l = 0.0 # sum of w_l w_(l-1) over consecutive links
645
+ self.prev_lw = 0.0 # weight of the link that ended at the previous event, now
646
+ self.s4l2 = [0.0] * 9 # sum w2 z_k,i z_(k-2),j over 2-links
647
+ self.m2_l2 = 0.0 # sum w2 over 2-links
648
+ self.s11_2 = 0.0 # sum w1_k w1_(k-2) over links two apart in one run
649
+ self.s22_0 = 0.0 # sum w2^2
650
+ self.s22_1 = 0.0 # sum w2_k w2_(k-1)
651
+ self.s22_2 = 0.0 # sum w2_k w2_(k-2)
652
+ self.x0 = 0.0 # sum w1_k w2_k
653
+ self.xm1 = 0.0 # sum w1_(k-1) w2_k
654
+ self.xp1 = 0.0 # sum w1_k w2_(k-1)
655
+ self.xm2 = 0.0 # sum w1_(k-2) w2_k
656
+ self.prev_lw2 = 0.0 # weight now of the link two events back (0 across a break)
657
+ self.prev_w2 = 0.0 # weight now of the 2-link that ended at the previous event
658
+ self.prev_w2b = 0.0 # the same two events back
659
+ self.prev2_t = _NEVER
660
+ self.prev2_z = [0.0] * _NR
661
+
662
+
663
+ def ratio_state(tau_days: float) -> Ratio:
664
+ """The ratio ``k`` of ``b = k a``, with memory time ``tau_days``."""
665
+ return Ratio(1, tau_days)
666
+
667
+
668
+ def growth_state(t0_days: float) -> Ratio:
669
+ """The growth fit ``b = (k0 + k1 y) a``, ``y`` in years since ``t0_days``, no forgetting."""
670
+ return Ratio(2, 0.0, t0_days)
671
+
672
+
673
+ def _ratio_decay(state: Ratio, t_days: float) -> None:
674
+ """Move the state forward to ``t_days``; old events lose weight: lambda in the normal
675
+ matrix and the counts, lambda^2 in the band's sums."""
676
+ if state.n == 0:
677
+ state.t_last = t_days
678
+ return
679
+ age = t_days - state.t_last
680
+ if age <= 0.0:
681
+ return
682
+ state.t_last = t_days
683
+ if state.tau_days <= 0.0:
684
+ return
685
+ f = math.exp(-age / state.tau_days)
686
+ f2 = f * f # the band's sums carry the weight twice
687
+ for k in range(4):
688
+ state.a[k] *= f
689
+ for k in range(36):
690
+ state.s4[k] *= f2
691
+ state.s4l[k] *= f2
692
+ for k in range(2):
693
+ state.h[k] *= f
694
+ state.m *= f
695
+ state.m_lag *= f
696
+ state.m2 *= f2
697
+ state.m2_lag *= f2
698
+ state.m2_ll *= f2
699
+ f4 = f2 * f2
700
+ state.c2 *= f2
701
+ state.w4 *= f4
702
+ state.w4l *= f4
703
+ state.prev_lw *= f2
704
+ for k in range(9):
705
+ state.s4l2[k] *= f2
706
+ state.m2_l2 *= f2
707
+ state.prev_lw2 *= f2
708
+ state.prev_w2 *= f2
709
+ state.prev_w2b *= f2
710
+ state.s11_2 *= f4
711
+ state.s22_0 *= f4
712
+ state.s22_1 *= f4
713
+ state.s22_2 *= f4
714
+ state.x0 *= f4
715
+ state.xm1 *= f4
716
+ state.xp1 *= f4
717
+ state.xm2 *= f4
718
+
719
+
720
+ def _break_history(state: Ratio) -> None:
721
+ """A break of the chain: no link or 2-link before it counts with one after it."""
722
+ state.prev_lw = 0.0
723
+ state.prev_lw2 = 0.0
724
+ state.prev_w2 = 0.0
725
+ state.prev_w2b = 0.0
726
+
727
+
728
+ def ratio_update(state: Ratio, t_days: float, a: float, b: float, linked: int) -> None:
729
+ """Add one event ``(a, b)`` that ends at ``t_days`` (events in time order).
730
+
731
+ ``linked`` is 1 when the event shares its start point with the end of the previous
732
+ event (their errors share that point). An event with ``a <= 0`` or a non-finite sum
733
+ is refused and counted in ``n_skipped``; the next event is then not linked to it.
734
+ """
735
+ if not (a > 0.0) or not math.isfinite(a) or not math.isfinite(b):
736
+ state.n_skipped += 1
737
+ state.prev_ok = 0
738
+ state.prev_linked = 0
739
+ _break_history(state)
740
+ return
741
+ _ratio_decay(state, t_days)
742
+ if state.has_ref == 0:
743
+ state.k_ref = b / a
744
+ state.has_ref = 1
745
+ p = state.p
746
+ y = (t_days - state.t0_days) / 365.25
747
+ u = [1.0, y]
748
+ z = [0.0] * _NR
749
+ for r in range(p):
750
+ z[r] = u[r] * a
751
+ z[p] = b - state.k_ref * a
752
+ for r in range(p):
753
+ for s in range(p):
754
+ state.a[2 * r + s] += a * u[r] * u[s]
755
+ state.h[r] += u[r] * z[p]
756
+ for r in range(p):
757
+ for s in range(p):
758
+ uu = u[r] * u[s]
759
+ for i in range(p + 1):
760
+ for j in range(p + 1):
761
+ state.s4[((r * 2 + s) * 3 + i) * 3 + j] += uu * z[i] * z[j]
762
+ link = 1 if linked != 0 and state.prev_ok != 0 else 0
763
+ if link != 0:
764
+ g = 1.0 # lambda_k lambda_(k-1) now, with lambda_k = 1
765
+ if state.tau_days > 0.0:
766
+ g = math.exp(-(t_days - state.prev_t) / state.tau_days)
767
+ for r in range(p):
768
+ for s in range(p):
769
+ uu = g * (u[r] * state.prev_u[s] + state.prev_u[r] * u[s])
770
+ for i in range(p + 1):
771
+ for j in range(p + 1):
772
+ state.s4l[((r * 2 + s) * 3 + i) * 3 + j] += uu * z[i] * state.prev_z[j]
773
+ state.m_lag += math.sqrt(g)
774
+ state.m2_lag += g
775
+ if state.prev_linked != 0:
776
+ state.m2_ll += g
777
+ # The loop band: the previous end rest (-g) becomes the shared rest (1 - g), and a
778
+ # new end rest (-1) comes in.
779
+ state.c2 += 2.0 - 2.0 * g
780
+ state.w4 += g * g
781
+ state.w4l += g * state.prev_lw
782
+ state.s11_2 += g * state.prev_lw2 # 0 unless the link before the previous one is in this run
783
+ w2 = 0.0
784
+ if state.prev_linked != 0:
785
+ # A 2-link: this loop, the previous one and the one before it in a row.
786
+ w2 = 1.0 # lambda_k lambda_(k-2) now
787
+ if state.tau_days > 0.0:
788
+ w2 = math.exp(-(t_days - state.prev2_t) / state.tau_days)
789
+ for i in range(p + 1):
790
+ for j in range(p + 1):
791
+ state.s4l2[i * 3 + j] += w2 * z[i] * state.prev2_z[j]
792
+ state.m2_l2 += w2
793
+ state.s22_0 += w2 * w2
794
+ state.s22_1 += w2 * state.prev_w2
795
+ state.s22_2 += w2 * state.prev_w2b
796
+ state.x0 += g * w2
797
+ state.xm1 += state.prev_lw * w2
798
+ state.xm2 += state.prev_lw2 * w2
799
+ state.xp1 += g * state.prev_w2
800
+ state.prev_lw2 = state.prev_lw
801
+ state.prev_lw = g
802
+ state.prev_w2b = state.prev_w2
803
+ state.prev_w2 = w2
804
+ else:
805
+ state.c2 += 2.0 # a new start rest (+1) and a new end rest (-1)
806
+ _break_history(state)
807
+ state.prev_linked = link
808
+ state.prev_ok = 1
809
+ state.prev2_t = state.prev_t
810
+ for r in range(_NR):
811
+ state.prev2_z[r] = state.prev_z[r]
812
+ state.prev_t = t_days
813
+ for r in range(2):
814
+ state.prev_u[r] = u[r]
815
+ for r in range(_NR):
816
+ state.prev_z[r] = z[r]
817
+ state.m += 1.0
818
+ state.m2 += 1.0
819
+ state.n += 1
820
+ if y < state.y_min:
821
+ state.y_min = y
822
+ if y > state.y_max:
823
+ state.y_max = y
824
+
825
+
826
+ class RatioEstimate:
827
+ """The result of ``ratio_solve`` (the C port's output struct).
828
+
829
+ ``theta`` = (k, NaN) for the ratio fit, (k0, k1) for the growth fit; ``sigma`` their
830
+ standard errors; ``cov`` the row-major 2 x 2 covariance (NaN outside p x p). ``low``
831
+ and ``high`` are the statistical band of k, or of k1 for the growth fit. ``theta``
832
+ is also filled when the status is ``OUT_OF_RANGE``, so the caller can say what came
833
+ out; it is not an estimate then. ``dof`` is the band's degrees of freedom (``m - p``
834
+ times the share lost to the lag-1 term; for ``loop_solve`` those of the rest error's
835
+ variance; module docstring); callers that form a band from ``cov`` use the same.
836
+ ``rest_error`` is filled by ``loop_solve`` only.
837
+ """
838
+
839
+ __slots__ = (
840
+ "status", "theta", "sigma", "cov", "low", "high", "dof", "m", "m_lag", "rho", "span_years", "rest_error",
841
+ )
842
+
843
+ def __init__(self) -> None:
844
+ self.status = SINGULAR
845
+ self.theta = [math.nan] * 2
846
+ self.sigma = [math.nan] * 2
847
+ self.cov = [math.nan] * 4
848
+ self.low = math.nan
849
+ self.high = math.nan
850
+ self.dof = 0.0
851
+ self.m = 0.0
852
+ self.m_lag = 0.0
853
+ self.rho = math.nan
854
+ self.span_years = 0.0
855
+ self.rest_error = math.nan # loop band only: sqrt(s2), the rest error in the units of b
856
+
857
+
858
+ def ratio_solve(
859
+ state: Ratio,
860
+ t_days: float,
861
+ z: float,
862
+ min_events: float,
863
+ min_span_years: float,
864
+ lower: float,
865
+ upper: float,
866
+ ) -> RatioEstimate:
867
+ """The estimate at ``t_days`` (the state is decayed to it first).
868
+
869
+ ``z`` is the two-sided normal quantile of the band. Status, in this order:
870
+ ``TOO_FEW_EVENTS`` when the decayed event count is below ``min_events``;
871
+ ``SHORT_SPAN`` (growth fit only) when the events span less than ``min_span_years``;
872
+ ``SINGULAR``; ``OUT_OF_RANGE`` when k (growth fit: k0, or the fitted value at
873
+ ``t_days``) is ``<= lower`` or ``> upper``; ``NO_BAND`` when ``m <= p`` or a variance
874
+ comes out negative beyond rounding (module docstring); else ``OK``. The band is the
875
+ statistical band only: the caller adds any systematic term.
876
+ """
877
+ _ratio_decay(state, t_days)
878
+ out = RatioEstimate()
879
+ out.m = state.m
880
+ out.m_lag = state.m_lag
881
+ if state.n > 0:
882
+ out.span_years = state.y_max - state.y_min
883
+ if state.m < min_events:
884
+ out.status = TOO_FEW_EVENTS
885
+ return out
886
+ p = state.p
887
+ if p == 2 and out.span_years < min_span_years:
888
+ out.status = SHORT_SPAN
889
+ return out
890
+ inv = [0.0] * 9 # row-major in 3 x 3, as _invert writes it
891
+ if p == 1:
892
+ if not (state.a[0] > 0.0):
893
+ out.status = SINGULAR
894
+ return out
895
+ inv[0] = 1.0 / state.a[0]
896
+ else:
897
+ mat = [state.a[0], state.a[1], 0.0, state.a[2], state.a[3], 0.0, 0.0, 0.0, 0.0]
898
+ if _invert(2, mat, inv) == 0:
899
+ out.status = SINGULAR
900
+ return out
901
+ th = [0.0, 0.0]
902
+ for r in range(p):
903
+ acc = 0.0
904
+ for k in range(p):
905
+ acc += inv[3 * r + k] * state.h[k]
906
+ th[r] = acc
907
+ th[0] += state.k_ref
908
+ value = th[0]
909
+ fitted = th[0]
910
+ if p == 2:
911
+ fitted = th[0] + th[1] * (t_days - state.t0_days) / 365.25
912
+ if value <= lower or value > upper or fitted <= lower or fitted > upper:
913
+ for r in range(p):
914
+ out.theta[r] = th[r]
915
+ out.status = OUT_OF_RANGE
916
+ return out
917
+ nu = state.m - p
918
+ out.dof = nu
919
+ if nu <= 0.0:
920
+ out.status = NO_BAND
921
+ return out
922
+ # phi contracts the sums: phi' z = b - theta' u a, the residual.
923
+ phi = [0.0] * _NR
924
+ for r in range(p):
925
+ phi[r] = -th[r]
926
+ phi[0] += state.k_ref
927
+ phi[p] = 1.0
928
+ meat = [0.0] * 4
929
+ size = [0.0] * 4 # the same sums in absolute values: the scale of their rounding error
930
+ for r in range(p):
931
+ for s in range(p):
932
+ acc = 0.0
933
+ mag = 0.0
934
+ for i in range(p + 1):
935
+ for j in range(p + 1):
936
+ idx = ((r * 2 + s) * 3 + i) * 3 + j
937
+ acc += phi[i] * (state.s4[idx] + state.s4l[idx]) * phi[j]
938
+ mag += abs(phi[i] * phi[j]) * (abs(state.s4[idx]) + abs(state.s4l[idx]))
939
+ meat[2 * r + s] = acc
940
+ size[2 * r + s] = mag
941
+ # Lag-1 correlation of the residuals across links (diagnostic): from the u_0 = 1 sums,
942
+ # each normalised by its own sum of weights.
943
+ out.rho = 0.0
944
+ e2 = 0.0
945
+ lag2 = 0.0
946
+ for i in range(p + 1):
947
+ for j in range(p + 1):
948
+ e2 += phi[i] * state.s4[i * 3 + j] * phi[j]
949
+ lag2 += phi[i] * state.s4l[i * 3 + j] * phi[j]
950
+ rho = 0.0
951
+ if state.m2_lag > 0.0 and e2 > 0.0:
952
+ rho = (0.5 * lag2 / state.m2_lag) / (e2 / state.m2)
953
+ if state.m_lag > 1.0:
954
+ out.rho = rho
955
+ # Degrees of freedom of the band (module docstring): those of the variance estimate,
956
+ # (m - p) times its loss from the lag-1 term; 1 without links.
957
+ q = state.m2_lag / state.m2
958
+ qq = state.m2_ll / state.m2
959
+ den = 1.0 + 2.0 * q + 8.0 * rho * q + 4.0 * rho * rho * (q + qq)
960
+ if den > 0.0:
961
+ out.dof = nu * (1.0 + 2.0 * rho * q) * (1.0 + 2.0 * rho * q) / den
962
+ scale = state.m / nu
963
+ for r in range(p):
964
+ for s in range(p):
965
+ acc = 0.0
966
+ mag = 0.0
967
+ for i in range(p):
968
+ for j in range(p):
969
+ acc += inv[3 * r + i] * meat[2 * i + j] * inv[3 * j + s]
970
+ mag += abs(inv[3 * r + i] * inv[3 * j + s]) * size[2 * i + j]
971
+ if r == s:
972
+ # A negative value within rounding of the sums is zero (a fit with no scatter).
973
+ if acc < 0.0 and -acc <= ROUNDING * mag:
974
+ acc = 0.0
975
+ if acc < 0.0:
976
+ out.status = NO_BAND
977
+ return out
978
+ out.cov[2 * r + s] = scale * acc
979
+ for r in range(p):
980
+ out.theta[r] = th[r]
981
+ out.sigma[r] = math.sqrt(out.cov[3 * r])
982
+ main = p - 1
983
+ half = t_quantile(z, out.dof) * out.sigma[main]
984
+ out.low = th[main] - half
985
+ out.high = th[main] + half
986
+ out.status = OK
987
+ return out
988
+
989
+
990
+ def loop_solve(
991
+ state: Ratio,
992
+ t_days: float,
993
+ z: float,
994
+ min_events: float,
995
+ lower: float,
996
+ upper: float,
997
+ ) -> RatioEstimate:
998
+ """The loop band at ``t_days`` (the state is decayed to it first; ``p = 1`` only).
999
+
1000
+ The value is ``k = sum lambda b / sum lambda a``, as ``ratio_solve`` gives it; the band
1001
+ is ``k +- t sqrt(s2 C2) / A`` from the rest error ``s2``, the mean lag-2 product of the
1002
+ scores minus the mean lag-1 product (module docstring). Status, in this order:
1003
+ ``TOO_FEW_EVENTS``; ``SINGULAR`` (also for a growth state); ``OUT_OF_RANGE`` (as
1004
+ ``ratio_solve``, theta filled); ``NO_LINKS`` when no 2-link is in memory; ``NO_BAND``
1005
+ when ``s2`` is negative beyond rounding (rho filled); ``LOW_DOF`` below one degree of
1006
+ freedom (rho, dof and rest_error filled); else ``OK``. ``rest_error`` is ``sqrt(s2)``;
1007
+ ``rho`` the lag-1 correlation of the scores.
1008
+ """
1009
+ _ratio_decay(state, t_days)
1010
+ out = RatioEstimate()
1011
+ out.m = state.m
1012
+ out.m_lag = state.m_lag
1013
+ if state.n > 0:
1014
+ out.span_years = state.y_max - state.y_min
1015
+ if state.m < min_events:
1016
+ out.status = TOO_FEW_EVENTS
1017
+ return out
1018
+ if state.p != 1 or not (state.a[0] > 0.0):
1019
+ out.status = SINGULAR
1020
+ return out
1021
+ inv = 1.0 / state.a[0]
1022
+ k = state.h[0] * inv + state.k_ref
1023
+ if k <= lower or k > upper:
1024
+ out.theta[0] = k
1025
+ out.status = OUT_OF_RANGE
1026
+ return out
1027
+ if not (state.m2_l2 > 0.0):
1028
+ out.status = NO_LINKS
1029
+ return out
1030
+ # phi contracts the sums: phi' z = b - k a, the score.
1031
+ phi = [state.k_ref - k, 1.0]
1032
+ e2 = 0.0
1033
+ lag1 = 0.0
1034
+ lag2 = 0.0
1035
+ mag1 = 0.0 # the lag sums in absolute values: the scale of their rounding error
1036
+ mag2 = 0.0
1037
+ for i in range(2):
1038
+ for j in range(2):
1039
+ e2 += phi[i] * state.s4[i * 3 + j] * phi[j]
1040
+ lag1 += phi[i] * state.s4l[i * 3 + j] * phi[j]
1041
+ lag2 += phi[i] * state.s4l2[i * 3 + j] * phi[j]
1042
+ mag1 += abs(phi[i] * phi[j]) * abs(state.s4l[i * 3 + j])
1043
+ mag2 += abs(phi[i] * phi[j]) * abs(state.s4l2[i * 3 + j])
1044
+ w1 = state.m2_lag
1045
+ w2 = state.m2_l2
1046
+ g0 = e2 / state.m2 # the weighted mean square
1047
+ g1 = 0.5 * lag1 / w1 # M1, the weighted mean of g_k g_(k-1) over links (s4l holds it twice)
1048
+ g2 = lag2 / w2 # M2, the same of g_k g_(k-2) over 2-links
1049
+ rho = 0.0
1050
+ if e2 > 0.0:
1051
+ rho = g1 / g0
1052
+ out.rho = rho
1053
+ s2 = g2 - g1
1054
+ if s2 < 0.0:
1055
+ # Within rounding of the two means' sums it is zero; beyond it the rest error's
1056
+ # variance is negative and there is no band.
1057
+ if -s2 > ROUNDING * (0.5 * mag1 / w1 + mag2 / w2):
1058
+ out.status = NO_BAND
1059
+ return out
1060
+ s2 = 0.0
1061
+ # Degrees of freedom of s2 (module docstring).
1062
+ v11 = ((g0 * g0 + g1 * g1) * state.w4 + 2.0 * (g1 * g1 + g0 * g2) * state.w4l + 2.0 * g2 * g2 * state.s11_2) / (w1 * w1)
1063
+ v22 = ((g0 * g0 + g2 * g2) * state.s22_0 + 2.0 * g1 * g1 * state.s22_1 + 2.0 * g2 * g2 * state.s22_2) / (w2 * w2)
1064
+ v12 = (g1 * (g0 + g2) * (state.x0 + state.xm1) + g1 * g2 * (state.xp1 + state.xm2)) / (w1 * w2)
1065
+ var_s2 = v11 + v22 - 2.0 * v12
1066
+ out.dof = 0.0
1067
+ if var_s2 > 0.0:
1068
+ out.dof = 2.0 * s2 * s2 / var_s2
1069
+ out.rest_error = math.sqrt(s2)
1070
+ if out.dof < 1.0:
1071
+ out.status = LOW_DOF
1072
+ return out
1073
+ var = s2 * state.c2 * inv * inv
1074
+ out.theta[0] = k
1075
+ out.cov[0] = var
1076
+ out.sigma[0] = math.sqrt(var)
1077
+ half = t_quantile(z, out.dof) * out.sigma[0]
1078
+ out.low = k - half
1079
+ out.high = k + half
1080
+ out.status = OK
1081
+ return out