spapprox 26.11.42__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,9 @@
1
+ cff-version: 1.2.0
2
+ message: "If you use this software, please cite it as below."
3
+ authors:
4
+ - family-names: "van der Schans"
5
+ given-names: "Martin"
6
+ title: "Spapprox: a Python library for saddle point approximation of distributions"
7
+ version: 26.11.42
8
+ date-released: 2026-11-22
9
+ url: "https://github.com/mvds314/spapprox"
@@ -0,0 +1,28 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2023, mvds314
4
+
5
+ Redistribution and use in source and binary forms, with or without
6
+ modification, are permitted provided that the following conditions are met:
7
+
8
+ 1. Redistributions of source code must retain the above copyright notice, this
9
+ list of conditions and the following disclaimer.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright notice,
12
+ this list of conditions and the following disclaimer in the documentation
13
+ and/or other materials provided with the distribution.
14
+
15
+ 3. Neither the name of the copyright holder nor the names of its
16
+ contributors may be used to endorse or promote products derived from
17
+ this software without specific prior written permission.
18
+
19
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,304 @@
1
+ Metadata-Version: 2.4
2
+ Name: spapprox
3
+ Version: 26.11.42
4
+ Summary: Saddle point approximations of distributions from their cumulant generating functions
5
+ Keywords: statistics,saddlepoint approximation,saddle point approximation,cumulant generating function,bootstrap,resampling,probability distributions
6
+ Author: Martin van der Schans
7
+ Requires-Python: >=3.11
8
+ Description-Content-Type: text/markdown
9
+ License-Expression: BSD-3-Clause
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: Operating System :: OS Independent
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3 :: Only
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Programming Language :: Python :: 3.13
18
+ Classifier: Programming Language :: Python :: 3.14
19
+ Classifier: Topic :: Scientific/Engineering :: Mathematics
20
+ License-File: LICENSE
21
+ Requires-Dist: numpy>=1.24
22
+ Requires-Dist: scipy>=1.10
23
+ Requires-Dist: pandas>=2.0
24
+ Requires-Dist: findiff>=0.11 ; extra == "all"
25
+ Requires-Dist: numdifftools ; extra == "all"
26
+ Requires-Dist: fastnorm ; extra == "all"
27
+ Requires-Dist: fastnorm ; extra == "fastnorm"
28
+ Requires-Dist: findiff>=0.11 ; extra == "findiff"
29
+ Requires-Dist: numdifftools ; extra == "numdiff"
30
+ Project-URL: Changelog, https://github.com/mvds314/spapprox/releases
31
+ Project-URL: Homepage, https://github.com/mvds314/spapprox
32
+ Project-URL: Issues, https://github.com/mvds314/spapprox/issues
33
+ Project-URL: Source, https://github.com/mvds314/spapprox
34
+ Provides-Extra: all
35
+ Provides-Extra: fastnorm
36
+ Provides-Extra: findiff
37
+ Provides-Extra: numdiff
38
+
39
+ # spapprox
40
+
41
+ Saddle point approximations of probability distributions from their cumulant generating
42
+ functions, in Python.
43
+
44
+ If you know the cumulant generating function (cgf) of a random variable or random vector,
45
+ `spapprox` approximates its pdf, cdf and quantiles. This is especially useful for sums and
46
+ means of independent random variables, and for resampling distributions such as the
47
+ bootstrap, where the cgf is easy to write down but the distribution is not.
48
+
49
+ The implementation follows Ronald W. Butler (2007), *Saddlepoint Approximations with
50
+ Applications*. As he puts it:
51
+
52
+ > Among the various tools that have been developed for use in statistics and probability
53
+ > over the years, perhaps the least understood and most remarkable tool is the saddlepoint
54
+ > approximation. It is remarkable because it usually provides probability approximations
55
+ > whose accuracy is much greater than the current supporting theory would suggest.
56
+
57
+ ## Installation
58
+
59
+ ```bash
60
+ pip install spapprox
61
+ ```
62
+
63
+ `spapprox` requires Python 3.11 or later. Optional dependencies, installed as extras:
64
+
65
+ - `numdiff`: [numdifftools](https://pypi.org/project/numdifftools/), the default numerical
66
+ differentiation backend, used when a cgf has no analytic derivatives.
67
+ - `findiff`: [findiff](https://pypi.org/project/findiff/) (>= 0.11), an alternative finite
68
+ difference backend.
69
+ - `fastnorm`: [fastnorm](https://pypi.org/project/fastnorm/), for faster bivariate normal
70
+ cdf evaluation.
71
+ - `all`: all of the above.
72
+
73
+ ```bash
74
+ pip install "spapprox[all]"
75
+ ```
76
+
77
+ ## Why cumulant generating functions?
78
+
79
+ The cgf of a random variable `X` is `K(t) = log E[exp(t X)]`. It determines the
80
+ distribution, its derivatives at zero are the cumulants (`K'(0)` is the mean and `K''(0)`
81
+ the variance), and it is **additive over independent random variables**: the cgf of
82
+ `X + Y` is `K_X(t) + K_Y(t)`. You cannot simply add distribution functions, but you can
83
+ add cgfs.
84
+
85
+ The saddle point approximation then recovers the density from the cgf:
86
+
87
+ ```
88
+ f(x) ≈ exp(K(t) - t x) / sqrt(2 π K''(t)), where t solves K'(t) = x.
89
+ ```
90
+
91
+ `K'(t) = x` is the *saddle point equation*. The cdf follows from the
92
+ Lugannani-Rice or Barndorff-Nielsen formulas. The
93
+ [Wikipedia page](https://en.wikipedia.org/wiki/Saddlepoint_approximation_method) is a good
94
+ introduction.
95
+
96
+ ## Quick start
97
+
98
+ Build a cgf, combine cgfs if needed, and pass the result to an approximation class:
99
+
100
+ ```python
101
+ import numpy as np
102
+ import scipy.stats as sps
103
+ import spapprox as spa
104
+
105
+ # The sum of a standard normal and an independent exponential with scale 2
106
+ cgf = spa.norm(loc=0, scale=1) + spa.exponential(scale=2)
107
+ approx = spa.UnivariateSaddlePointApprox(cgf)
108
+
109
+ approx.pdf(x=[0.0, 2.0, 5.0])
110
+ approx.cdf(x=[0.0, 2.0, 5.0]) # Lugannani-Rice, or backend="BN" for Barndorff-Nielsen
111
+ approx.ppf([0.05, 0.5, 0.95])
112
+
113
+ # Check against the exact distribution in a simple case
114
+ cgf_normal = spa.norm(loc=0, scale=1)
115
+ spa_normal = spa.UnivariateSaddlePointApprox(cgf_normal)
116
+ t = np.linspace(-3, 3)
117
+ x = cgf_normal.dK(t)
118
+ np.allclose(spa_normal.pdf(t=t), sps.norm.pdf(x))
119
+ ```
120
+
121
+ Every `pdf` and `cdf` takes either the point `x` or the saddle point `t`. Passing `x` solves
122
+ the saddle point equation for every value. Passing `t` (and getting `x = cgf.dK(t)`) avoids
123
+ that, which is much faster for plotting a whole curve. To evaluate at many `x` values
124
+ repeatedly, call `approx.fit_saddle_point_eqn()` first to tabulate the solution.
125
+
126
+ ## Features
127
+
128
+ - **Algebra on cgfs:** `cgf1 + cgf2` is the sum of independent random variables, `a * cgf + b`
129
+ an affine transformation, `mcgf[i]` a marginal, and `mcgf.ldot(A)` the linear transformation
130
+ `A X` of a random vector.
131
+ - **Univariate approximations** (`UnivariateSaddlePointApprox`): pdf, cdf (Lugannani-Rice or
132
+ Barndorff-Nielsen) and ppf. `UnivariateSaddlePointApproxMean` approximates the
133
+ distribution of a sample mean.
134
+ - **Discrete (lattice) variables:** pass the span of the lattice, e.g. `cdf(x, delta=1)` or
135
+ `pdf(x, delta=1)`, for the continuity-corrected cdf or the probability mass function.
136
+ - **Multivariate approximations** (`MultivariateSaddlePointApprox`,
137
+ `BivariateSaddlePointApprox`): joint pdf, marginals, and the joint cdf following Kolassa
138
+ (2003) or Kolassa and Li (2010), in any dimension. The bivariate class adds Wang (1990).
139
+ - **Conditional distributions:** `P(X_i ≤ x | X_J = x_J)` with the double saddle point
140
+ approximation of Skovgaard (1987), or Kolassa (1996) with `backend="Kolassa"`. Two
141
+ variables of interest are supported too (Kolassa and Li, 2010).
142
+ - **Transform-domain bootstrap:** the bootstrap distribution of a sample mean without
143
+ resampling, by combining `univariate_empirical` and `univariate_sample_mean`.
144
+ - **Estimating equations and resampling** (`EstimatingEquationSaddlePointApprox`,
145
+ `MultivariateEstimatingEquationSaddlePointApprox`): the distribution of the root of a
146
+ randomly weighted estimating equation (Daniels, 1983; Booth and Butler, 1990). This covers
147
+ the multinomial and Dirichlet bootstrap, the delete-m jackknife and permutation tests.
148
+
149
+ ### Available cgfs
150
+
151
+ | Univariate | Multivariate | Transformations |
152
+ |---|---|---|
153
+ | `norm`, `exponential`, `gamma`, `chi2`, `laplace`, `asymmetric_laplace`, `generalized_normal`, `asymmetric_generalized_normal`, `poisson`, `bernoulli`, `binomial`, `negative_binomial` | `multivariate_norm`, `bivariate_gamma`, `jensen_multivariate_gamma`, `multinomial`, `bivariate_poisson` | `univariate_empirical` (the empirical distribution of a sample), `univariate_sample_mean` |
154
+
155
+ Any other cgf can be supplied directly with `UnivariateCumulantGeneratingFunction` or
156
+ `MultivariateCumulantGeneratingFunction`. Derivatives you do not supply are computed
157
+ numerically. A `Domain` describes where the cgf is finite.
158
+
159
+ ## Examples
160
+
161
+ ### Conditional distributions
162
+
163
+ ```python
164
+ cgf = spa.multivariate_norm(loc=[0, 0], cov=[[1, 0.5], [0.5, 1]])
165
+ spa_mvn = spa.MultivariateSaddlePointApprox(cgf)
166
+ spa_mvn.conditional_cdf([-1, 0, 1], given=[0.3], i=0)
167
+ spa_mvn.conditional_pdf([-1, 0, 1], given=[0.3], i=0) # normalized, normalize_pdf=False to skip
168
+ ```
169
+
170
+ ### Estimating equations and resampling
171
+
172
+ The root `τ̂` of the randomly weighted estimating equation `Σ_i ψ(x_i; τ) W_i = 0`,
173
+ optionally conditional on `Σ_i W_i`, depends on the resampling scheme through the weights:
174
+
175
+ - Poisson weights given their sum `n` give the multinomial bootstrap.
176
+ - Exponential weights give the Dirichlet bootstrap.
177
+ - Bernoulli weights given `n - m` give the delete-m jackknife.
178
+
179
+ ```python
180
+ x = np.array([9.6, 10.4, 13.0, 15.0, 16.6, 17.2, 17.3, 21.8, 24.0, 33.8])
181
+ ee = spa.EstimatingEquationSaddlePointApprox.from_resampling(
182
+ lambda x, tau: x - tau, x, spa.exponential(), given=len(x), bracket=(x.min(), x.max())
183
+ )
184
+ ee.cdf([15, 18, 21])
185
+ ee.ppf([0.05, 0.95])
186
+ ```
187
+
188
+ `cdf` and `ppf` solve saddle point equations for every value. After `ee.fit_cdf()`, pass
189
+ `interpolate=True` to interpolate the tabulated cdf instead. This is much faster, and
190
+ accurate up to the resolution of the table. For a discrete `S(τ)`, e.g. a mean of integer
191
+ data under the multinomial bootstrap, pass `delta=1` to `from_resampling` for the
192
+ continuity-corrected cdf.
193
+
194
+ For a vector `τ`, e.g. the coefficients of a regression,
195
+ `MultivariateEstimatingEquationSaddlePointApprox(psi, x, weights, given)` approximates the
196
+ joint density, and its marginals come from a tabulated grid (`fit_pdf`, `marginal_cdf`,
197
+ `marginal_ppf`).
198
+
199
+ ### Example scripts
200
+
201
+ The [`examples`](https://github.com/mvds314/spapprox/tree/main/examples) folder contains
202
+ standalone scripts. They need `matplotlib`, and `example_nonparametric_bootstrap.py` also
203
+ needs `statsmodels`.
204
+
205
+ | Script | Shows |
206
+ |---|---|
207
+ | [`basic_example.py`](https://github.com/mvds314/spapprox/blob/main/examples/basic_example.py) | pdf and cdf of the normal and exponential distributions |
208
+ | [`example_sum_random_variables.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_sum_random_variables.py) | sum of independent random variables |
209
+ | [`example_parametric_bootstrap.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_parametric_bootstrap.py) | distribution of a sample mean |
210
+ | [`example_nonparametric_bootstrap.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_nonparametric_bootstrap.py) | bootstrap of the mean without resampling |
211
+ | [`example_bootstrap_multinomial.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_multinomial.py), [`example_bootstrap_dirichlet.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_dirichlet.py) | multinomial and Dirichlet bootstrap (Booth and Butler, 1990) |
212
+ | [`example_bootstrap_lattice.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_lattice.py) | continuity-corrected resampling of integer data |
213
+ | [`example_bootstrap_robust_estimator.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_robust_estimator.py) | bootstrap of Huber's M-estimator |
214
+ | [`example_bootstrap_regression.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_regression.py) | bootstrap of regression coefficients (vector estimating equation) |
215
+ | [`example_jackknife_estimating_equation.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_jackknife_estimating_equation.py) | delete-m jackknife |
216
+ | [`example_permutation_one_sample.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_permutation_one_sample.py), [`example_permutation_two_sample.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_permutation_two_sample.py) | permutation tests |
217
+ | [`example_numdiff_backends.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_numdiff_backends.py) | numerical derivatives of a cgf with `numdifftools` and `findiff` |
218
+ | [`example_Wang1990.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_Wang1990.py), [`example_Kolassa2003.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_Kolassa2003.py) | bivariate and multivariate cdf, reproducing published tables |
219
+
220
+ ## Limitations and future work
221
+
222
+ - Multivariate support is not complete. For example, the Kolassa and Li cdf does not support
223
+ mixed lattice and continuous components.
224
+ - `generalized_normal` and `asymmetric_generalized_normal` evaluate the cgf by quadrature
225
+ unless the shape is 1 or 2, which is slow.
226
+ - The Kolassa and Li bivariate cdf (the default `backend="KL"`) does not factorize exactly
227
+ for independent components (error up to about 5e-3 in tests). Its extension to more
228
+ than two dimensions has not been checked against the literature yet.
229
+ - Without analytic derivatives, cgfs are differentiated numerically, which is slower and
230
+ less accurate, in particular close to the boundary of the domain. The multivariate third
231
+ derivative with `numdifftools` differentiates numerically twice and is slow.
232
+ - The pdf is normalized by numerical integration by default, which can be expensive. Pass
233
+ `normalize_pdf=False`, or a known `pdf_normalization` to the constructor, to avoid it.
234
+ - The distribution of a quotient of random variables is not implemented yet.
235
+
236
+ ## Related software
237
+
238
+ - [saddlepoint-expansions](https://github.com/s-broda/saddlepoint-expansions) (Python, also
239
+ on PyPI) by Broda and Kan, a port of the Julia package
240
+ [SaddlepointExpansions.jl](https://github.com/s-broda/SaddlepointExpansions.jl). It
241
+ implements saddle point expansions of arbitrary order for multivariate tail probabilities,
242
+ densities, conditional distributions, ratios and Studentized means, with derivatives from
243
+ automatic differentiation.
244
+ - [saddlepoint](https://github.com/godrick/saddlepoint) (R) by Oketch, Fewster and Goodman, a
245
+ framework for saddle point approximations built on cgf operations, including approximate
246
+ likelihood inference.
247
+ - [esaddle](https://github.com/mfasiolo/esaddle) (R, on CRAN) by Fasiolo and Wood, the
248
+ extended empirical saddle point density approximation (Fasiolo et al., 2018), which
249
+ estimates a density from a sample of (possibly multivariate) simulated statistics.
250
+
251
+ ## References
252
+
253
+ - Barndorff-Nielsen, O. E. (1986). Inference on full or partial parameters based on the
254
+ standardized signed log likelihood ratio. *Biometrika*, 73, 307-322.
255
+ - Booth, J. G. and Butler, R. W. (1990). Randomization distributions and saddlepoint
256
+ approximations in generalized linear models. *Biometrika*, 77, 787-796.
257
+ - Butler, R. W. (2007). *Saddlepoint Approximations with Applications*. Cambridge University
258
+ Press.
259
+ - Daniels, H. E. (1983). Saddlepoint approximations for estimating equations. *Biometrika*,
260
+ 70, 89-96.
261
+ - Fasiolo, M., Wood, S. N., Hartig, F. and Bravington, M. V. (2018). An extended empirical
262
+ saddlepoint approximation for intractable likelihoods. *Electronic Journal of Statistics*,
263
+ 12(1), 1544-1578.
264
+ - Kolassa, J. E. (1996). Higher-order approximations to conditional distribution functions.
265
+ *Annals of Statistics*, 24, 353-364.
266
+ - Kolassa, J. E. (2003). Multivariate saddlepoint tail probability approximations. *Annals
267
+ of Statistics*, 31, 274-286.
268
+ - Kolassa, J. E. and Li, J. (2010). Multivariate saddlepoint approximations in tail
269
+ probability and conditional inference. *Bernoulli*, 16, 1191-1207.
270
+ - Lugannani, R. and Rice, S. (1980). Saddle point approximation for the distribution of the
271
+ sum of independent random variables. *Advances in Applied Probability*, 12, 475-490.
272
+ - Skovgaard, I. M. (1987). Saddlepoint expansions for conditional distributions. *Journal of
273
+ Applied Probability*, 24, 875-887.
274
+ - Wang, S. (1990). Saddlepoint approximations for bivariate distributions. *Journal of
275
+ Applied Probability*, 27, 586-597.
276
+
277
+ ## Citation
278
+
279
+ If you use `spapprox`, please cite it as described in
280
+ [`CITATION.cff`](https://github.com/mvds314/spapprox/blob/main/CITATION.cff).
281
+
282
+ ## Development
283
+
284
+ Clone the repository and install it in editable mode with all extras:
285
+
286
+ ```bash
287
+ git clone https://github.com/mvds314/spapprox.git
288
+ cd spapprox
289
+ pip install -e ".[all]"
290
+ ```
291
+
292
+ Run the tests with `pytest`. The full suite takes about 16 minutes. For routine iteration,
293
+ skip the slow tests (about 4 minutes):
294
+
295
+ ```bash
296
+ pytest -m "not slow"
297
+ ```
298
+
299
+ Install the optional dependencies to test all functionality. Lint with `ruff check .`.
300
+
301
+ ## License
302
+
303
+ BSD 3-Clause, see [`LICENSE`](https://github.com/mvds314/spapprox/blob/main/LICENSE).
304
+
@@ -0,0 +1,265 @@
1
+ # spapprox
2
+
3
+ Saddle point approximations of probability distributions from their cumulant generating
4
+ functions, in Python.
5
+
6
+ If you know the cumulant generating function (cgf) of a random variable or random vector,
7
+ `spapprox` approximates its pdf, cdf and quantiles. This is especially useful for sums and
8
+ means of independent random variables, and for resampling distributions such as the
9
+ bootstrap, where the cgf is easy to write down but the distribution is not.
10
+
11
+ The implementation follows Ronald W. Butler (2007), *Saddlepoint Approximations with
12
+ Applications*. As he puts it:
13
+
14
+ > Among the various tools that have been developed for use in statistics and probability
15
+ > over the years, perhaps the least understood and most remarkable tool is the saddlepoint
16
+ > approximation. It is remarkable because it usually provides probability approximations
17
+ > whose accuracy is much greater than the current supporting theory would suggest.
18
+
19
+ ## Installation
20
+
21
+ ```bash
22
+ pip install spapprox
23
+ ```
24
+
25
+ `spapprox` requires Python 3.11 or later. Optional dependencies, installed as extras:
26
+
27
+ - `numdiff`: [numdifftools](https://pypi.org/project/numdifftools/), the default numerical
28
+ differentiation backend, used when a cgf has no analytic derivatives.
29
+ - `findiff`: [findiff](https://pypi.org/project/findiff/) (>= 0.11), an alternative finite
30
+ difference backend.
31
+ - `fastnorm`: [fastnorm](https://pypi.org/project/fastnorm/), for faster bivariate normal
32
+ cdf evaluation.
33
+ - `all`: all of the above.
34
+
35
+ ```bash
36
+ pip install "spapprox[all]"
37
+ ```
38
+
39
+ ## Why cumulant generating functions?
40
+
41
+ The cgf of a random variable `X` is `K(t) = log E[exp(t X)]`. It determines the
42
+ distribution, its derivatives at zero are the cumulants (`K'(0)` is the mean and `K''(0)`
43
+ the variance), and it is **additive over independent random variables**: the cgf of
44
+ `X + Y` is `K_X(t) + K_Y(t)`. You cannot simply add distribution functions, but you can
45
+ add cgfs.
46
+
47
+ The saddle point approximation then recovers the density from the cgf:
48
+
49
+ ```
50
+ f(x) ≈ exp(K(t) - t x) / sqrt(2 π K''(t)), where t solves K'(t) = x.
51
+ ```
52
+
53
+ `K'(t) = x` is the *saddle point equation*. The cdf follows from the
54
+ Lugannani-Rice or Barndorff-Nielsen formulas. The
55
+ [Wikipedia page](https://en.wikipedia.org/wiki/Saddlepoint_approximation_method) is a good
56
+ introduction.
57
+
58
+ ## Quick start
59
+
60
+ Build a cgf, combine cgfs if needed, and pass the result to an approximation class:
61
+
62
+ ```python
63
+ import numpy as np
64
+ import scipy.stats as sps
65
+ import spapprox as spa
66
+
67
+ # The sum of a standard normal and an independent exponential with scale 2
68
+ cgf = spa.norm(loc=0, scale=1) + spa.exponential(scale=2)
69
+ approx = spa.UnivariateSaddlePointApprox(cgf)
70
+
71
+ approx.pdf(x=[0.0, 2.0, 5.0])
72
+ approx.cdf(x=[0.0, 2.0, 5.0]) # Lugannani-Rice, or backend="BN" for Barndorff-Nielsen
73
+ approx.ppf([0.05, 0.5, 0.95])
74
+
75
+ # Check against the exact distribution in a simple case
76
+ cgf_normal = spa.norm(loc=0, scale=1)
77
+ spa_normal = spa.UnivariateSaddlePointApprox(cgf_normal)
78
+ t = np.linspace(-3, 3)
79
+ x = cgf_normal.dK(t)
80
+ np.allclose(spa_normal.pdf(t=t), sps.norm.pdf(x))
81
+ ```
82
+
83
+ Every `pdf` and `cdf` takes either the point `x` or the saddle point `t`. Passing `x` solves
84
+ the saddle point equation for every value. Passing `t` (and getting `x = cgf.dK(t)`) avoids
85
+ that, which is much faster for plotting a whole curve. To evaluate at many `x` values
86
+ repeatedly, call `approx.fit_saddle_point_eqn()` first to tabulate the solution.
87
+
88
+ ## Features
89
+
90
+ - **Algebra on cgfs:** `cgf1 + cgf2` is the sum of independent random variables, `a * cgf + b`
91
+ an affine transformation, `mcgf[i]` a marginal, and `mcgf.ldot(A)` the linear transformation
92
+ `A X` of a random vector.
93
+ - **Univariate approximations** (`UnivariateSaddlePointApprox`): pdf, cdf (Lugannani-Rice or
94
+ Barndorff-Nielsen) and ppf. `UnivariateSaddlePointApproxMean` approximates the
95
+ distribution of a sample mean.
96
+ - **Discrete (lattice) variables:** pass the span of the lattice, e.g. `cdf(x, delta=1)` or
97
+ `pdf(x, delta=1)`, for the continuity-corrected cdf or the probability mass function.
98
+ - **Multivariate approximations** (`MultivariateSaddlePointApprox`,
99
+ `BivariateSaddlePointApprox`): joint pdf, marginals, and the joint cdf following Kolassa
100
+ (2003) or Kolassa and Li (2010), in any dimension. The bivariate class adds Wang (1990).
101
+ - **Conditional distributions:** `P(X_i ≤ x | X_J = x_J)` with the double saddle point
102
+ approximation of Skovgaard (1987), or Kolassa (1996) with `backend="Kolassa"`. Two
103
+ variables of interest are supported too (Kolassa and Li, 2010).
104
+ - **Transform-domain bootstrap:** the bootstrap distribution of a sample mean without
105
+ resampling, by combining `univariate_empirical` and `univariate_sample_mean`.
106
+ - **Estimating equations and resampling** (`EstimatingEquationSaddlePointApprox`,
107
+ `MultivariateEstimatingEquationSaddlePointApprox`): the distribution of the root of a
108
+ randomly weighted estimating equation (Daniels, 1983; Booth and Butler, 1990). This covers
109
+ the multinomial and Dirichlet bootstrap, the delete-m jackknife and permutation tests.
110
+
111
+ ### Available cgfs
112
+
113
+ | Univariate | Multivariate | Transformations |
114
+ |---|---|---|
115
+ | `norm`, `exponential`, `gamma`, `chi2`, `laplace`, `asymmetric_laplace`, `generalized_normal`, `asymmetric_generalized_normal`, `poisson`, `bernoulli`, `binomial`, `negative_binomial` | `multivariate_norm`, `bivariate_gamma`, `jensen_multivariate_gamma`, `multinomial`, `bivariate_poisson` | `univariate_empirical` (the empirical distribution of a sample), `univariate_sample_mean` |
116
+
117
+ Any other cgf can be supplied directly with `UnivariateCumulantGeneratingFunction` or
118
+ `MultivariateCumulantGeneratingFunction`. Derivatives you do not supply are computed
119
+ numerically. A `Domain` describes where the cgf is finite.
120
+
121
+ ## Examples
122
+
123
+ ### Conditional distributions
124
+
125
+ ```python
126
+ cgf = spa.multivariate_norm(loc=[0, 0], cov=[[1, 0.5], [0.5, 1]])
127
+ spa_mvn = spa.MultivariateSaddlePointApprox(cgf)
128
+ spa_mvn.conditional_cdf([-1, 0, 1], given=[0.3], i=0)
129
+ spa_mvn.conditional_pdf([-1, 0, 1], given=[0.3], i=0) # normalized, normalize_pdf=False to skip
130
+ ```
131
+
132
+ ### Estimating equations and resampling
133
+
134
+ The root `τ̂` of the randomly weighted estimating equation `Σ_i ψ(x_i; τ) W_i = 0`,
135
+ optionally conditional on `Σ_i W_i`, depends on the resampling scheme through the weights:
136
+
137
+ - Poisson weights given their sum `n` give the multinomial bootstrap.
138
+ - Exponential weights give the Dirichlet bootstrap.
139
+ - Bernoulli weights given `n - m` give the delete-m jackknife.
140
+
141
+ ```python
142
+ x = np.array([9.6, 10.4, 13.0, 15.0, 16.6, 17.2, 17.3, 21.8, 24.0, 33.8])
143
+ ee = spa.EstimatingEquationSaddlePointApprox.from_resampling(
144
+ lambda x, tau: x - tau, x, spa.exponential(), given=len(x), bracket=(x.min(), x.max())
145
+ )
146
+ ee.cdf([15, 18, 21])
147
+ ee.ppf([0.05, 0.95])
148
+ ```
149
+
150
+ `cdf` and `ppf` solve saddle point equations for every value. After `ee.fit_cdf()`, pass
151
+ `interpolate=True` to interpolate the tabulated cdf instead. This is much faster, and
152
+ accurate up to the resolution of the table. For a discrete `S(τ)`, e.g. a mean of integer
153
+ data under the multinomial bootstrap, pass `delta=1` to `from_resampling` for the
154
+ continuity-corrected cdf.
155
+
156
+ For a vector `τ`, e.g. the coefficients of a regression,
157
+ `MultivariateEstimatingEquationSaddlePointApprox(psi, x, weights, given)` approximates the
158
+ joint density, and its marginals come from a tabulated grid (`fit_pdf`, `marginal_cdf`,
159
+ `marginal_ppf`).
160
+
161
+ ### Example scripts
162
+
163
+ The [`examples`](https://github.com/mvds314/spapprox/tree/main/examples) folder contains
164
+ standalone scripts. They need `matplotlib`, and `example_nonparametric_bootstrap.py` also
165
+ needs `statsmodels`.
166
+
167
+ | Script | Shows |
168
+ |---|---|
169
+ | [`basic_example.py`](https://github.com/mvds314/spapprox/blob/main/examples/basic_example.py) | pdf and cdf of the normal and exponential distributions |
170
+ | [`example_sum_random_variables.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_sum_random_variables.py) | sum of independent random variables |
171
+ | [`example_parametric_bootstrap.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_parametric_bootstrap.py) | distribution of a sample mean |
172
+ | [`example_nonparametric_bootstrap.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_nonparametric_bootstrap.py) | bootstrap of the mean without resampling |
173
+ | [`example_bootstrap_multinomial.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_multinomial.py), [`example_bootstrap_dirichlet.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_dirichlet.py) | multinomial and Dirichlet bootstrap (Booth and Butler, 1990) |
174
+ | [`example_bootstrap_lattice.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_lattice.py) | continuity-corrected resampling of integer data |
175
+ | [`example_bootstrap_robust_estimator.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_robust_estimator.py) | bootstrap of Huber's M-estimator |
176
+ | [`example_bootstrap_regression.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_bootstrap_regression.py) | bootstrap of regression coefficients (vector estimating equation) |
177
+ | [`example_jackknife_estimating_equation.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_jackknife_estimating_equation.py) | delete-m jackknife |
178
+ | [`example_permutation_one_sample.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_permutation_one_sample.py), [`example_permutation_two_sample.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_permutation_two_sample.py) | permutation tests |
179
+ | [`example_numdiff_backends.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_numdiff_backends.py) | numerical derivatives of a cgf with `numdifftools` and `findiff` |
180
+ | [`example_Wang1990.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_Wang1990.py), [`example_Kolassa2003.py`](https://github.com/mvds314/spapprox/blob/main/examples/example_Kolassa2003.py) | bivariate and multivariate cdf, reproducing published tables |
181
+
182
+ ## Limitations and future work
183
+
184
+ - Multivariate support is not complete. For example, the Kolassa and Li cdf does not support
185
+ mixed lattice and continuous components.
186
+ - `generalized_normal` and `asymmetric_generalized_normal` evaluate the cgf by quadrature
187
+ unless the shape is 1 or 2, which is slow.
188
+ - The Kolassa and Li bivariate cdf (the default `backend="KL"`) does not factorize exactly
189
+ for independent components (error up to about 5e-3 in tests). Its extension to more
190
+ than two dimensions has not been checked against the literature yet.
191
+ - Without analytic derivatives, cgfs are differentiated numerically, which is slower and
192
+ less accurate, in particular close to the boundary of the domain. The multivariate third
193
+ derivative with `numdifftools` differentiates numerically twice and is slow.
194
+ - The pdf is normalized by numerical integration by default, which can be expensive. Pass
195
+ `normalize_pdf=False`, or a known `pdf_normalization` to the constructor, to avoid it.
196
+ - The distribution of a quotient of random variables is not implemented yet.
197
+
198
+ ## Related software
199
+
200
+ - [saddlepoint-expansions](https://github.com/s-broda/saddlepoint-expansions) (Python, also
201
+ on PyPI) by Broda and Kan, a port of the Julia package
202
+ [SaddlepointExpansions.jl](https://github.com/s-broda/SaddlepointExpansions.jl). It
203
+ implements saddle point expansions of arbitrary order for multivariate tail probabilities,
204
+ densities, conditional distributions, ratios and Studentized means, with derivatives from
205
+ automatic differentiation.
206
+ - [saddlepoint](https://github.com/godrick/saddlepoint) (R) by Oketch, Fewster and Goodman, a
207
+ framework for saddle point approximations built on cgf operations, including approximate
208
+ likelihood inference.
209
+ - [esaddle](https://github.com/mfasiolo/esaddle) (R, on CRAN) by Fasiolo and Wood, the
210
+ extended empirical saddle point density approximation (Fasiolo et al., 2018), which
211
+ estimates a density from a sample of (possibly multivariate) simulated statistics.
212
+
213
+ ## References
214
+
215
+ - Barndorff-Nielsen, O. E. (1986). Inference on full or partial parameters based on the
216
+ standardized signed log likelihood ratio. *Biometrika*, 73, 307-322.
217
+ - Booth, J. G. and Butler, R. W. (1990). Randomization distributions and saddlepoint
218
+ approximations in generalized linear models. *Biometrika*, 77, 787-796.
219
+ - Butler, R. W. (2007). *Saddlepoint Approximations with Applications*. Cambridge University
220
+ Press.
221
+ - Daniels, H. E. (1983). Saddlepoint approximations for estimating equations. *Biometrika*,
222
+ 70, 89-96.
223
+ - Fasiolo, M., Wood, S. N., Hartig, F. and Bravington, M. V. (2018). An extended empirical
224
+ saddlepoint approximation for intractable likelihoods. *Electronic Journal of Statistics*,
225
+ 12(1), 1544-1578.
226
+ - Kolassa, J. E. (1996). Higher-order approximations to conditional distribution functions.
227
+ *Annals of Statistics*, 24, 353-364.
228
+ - Kolassa, J. E. (2003). Multivariate saddlepoint tail probability approximations. *Annals
229
+ of Statistics*, 31, 274-286.
230
+ - Kolassa, J. E. and Li, J. (2010). Multivariate saddlepoint approximations in tail
231
+ probability and conditional inference. *Bernoulli*, 16, 1191-1207.
232
+ - Lugannani, R. and Rice, S. (1980). Saddle point approximation for the distribution of the
233
+ sum of independent random variables. *Advances in Applied Probability*, 12, 475-490.
234
+ - Skovgaard, I. M. (1987). Saddlepoint expansions for conditional distributions. *Journal of
235
+ Applied Probability*, 24, 875-887.
236
+ - Wang, S. (1990). Saddlepoint approximations for bivariate distributions. *Journal of
237
+ Applied Probability*, 27, 586-597.
238
+
239
+ ## Citation
240
+
241
+ If you use `spapprox`, please cite it as described in
242
+ [`CITATION.cff`](https://github.com/mvds314/spapprox/blob/main/CITATION.cff).
243
+
244
+ ## Development
245
+
246
+ Clone the repository and install it in editable mode with all extras:
247
+
248
+ ```bash
249
+ git clone https://github.com/mvds314/spapprox.git
250
+ cd spapprox
251
+ pip install -e ".[all]"
252
+ ```
253
+
254
+ Run the tests with `pytest`. The full suite takes about 16 minutes. For routine iteration,
255
+ skip the slow tests (about 4 minutes):
256
+
257
+ ```bash
258
+ pytest -m "not slow"
259
+ ```
260
+
261
+ Install the optional dependencies to test all functionality. Lint with `ruff check .`.
262
+
263
+ ## License
264
+
265
+ BSD 3-Clause, see [`LICENSE`](https://github.com/mvds314/spapprox/blob/main/LICENSE).