splitpopsurv 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Nobutaka Fukuda
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to
7
+ deal in the Software without restriction, including without limitation the
8
+ rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
9
+ sell copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
20
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
21
+ DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,122 @@
1
+ Metadata-Version: 2.4
2
+ Name: splitpopsurv
3
+ Version: 0.1.0
4
+ Summary: Split-population (cure / mover-stayer) survival models
5
+ Author-email: Nobutaka Fukuda <nobutaka.fukuda@tohoku.ac.jp>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/nobifukuda/splitpopsurv-py
8
+ Project-URL: Bug Tracker, https://github.com/nobifukuda/splitpopsurv-py/issues
9
+ Classifier: Development Status :: 4 - Beta
10
+ Classifier: Intended Audience :: Science/Research
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: Topic :: Scientific/Engineering :: Mathematics
13
+ Requires-Python: >=3.9
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: numpy>=1.20
17
+ Requires-Dist: scipy>=1.7
18
+ Requires-Dist: pandas>=1.3
19
+ Provides-Extra: test
20
+ Requires-Dist: pytest>=7.0; extra == "test"
21
+ Dynamic: license-file
22
+
23
+ # splitpopsurv (Python)
24
+
25
+ Split-population (cure / mover–stayer) survival models in Python: an
26
+ accelerated failure-time regression for event timing among "movers",
27
+ combined with a logistic regression on the probability of belonging to
28
+ the immune "stayer" population.
29
+
30
+ Five baseline timing distributions are provided — **log-logistic**,
31
+ **Weibull**, **log-normal**, **gamma**, and the **generalized gamma** that
32
+ nests the other four — following Schmidt & Witte (1989) and Yamaguchi
33
+ (1992, 1998). This package is a Python translation of a set of Stata `ml`
34
+ programs, with the log-likelihood corrected to match the published model
35
+ and verified by simulation against known parameters.
36
+
37
+ 📖 **Full manual** (theory, formulas, the likelihood correction, and a
38
+ complete function reference): [`docs/manual.html`](https://htmlpreview.github.io/?https://github.com/nobifukuda/splitpopsurv-py/blob/main/docs/manual.html)
39
+
40
+ Companion packages implementing the same corrected models are available
41
+ for [R](https://github.com/nobifukuda/splitpopsurv) and
42
+ [Stata](https://github.com/nobifukuda/splitpopsurv-stata).
43
+
44
+ ## Author
45
+
46
+ **Nobutaka Fukuda**, Tohoku University — <nobutaka.fukuda@tohoku.ac.jp>
47
+
48
+ ## Installation
49
+
50
+ ```bash
51
+ pip install git+https://github.com/nobifukuda/splitpopsurv-py.git
52
+ ```
53
+
54
+ ## Usage
55
+
56
+ ```python
57
+ import pandas as pd
58
+ from splitpopsurv import fit_splitpop_weibull
59
+
60
+ # mydata needs: time, event (0/1), and your covariates
61
+ fit = fit_splitpop_weibull(
62
+ data=mydata,
63
+ time="time",
64
+ event="event",
65
+ h_vars=["x1"], # H_regression: covariates for timing
66
+ p_vars=["x2"], # P_regression: covariates for cure probability
67
+ )
68
+
69
+ print(fit.summary()) # coefficients, SEs, z-values, log-likelihood
70
+ fit.params # named parameter vector (numpy array)
71
+ fit.as_dataframe() # coefficients table as a pandas DataFrame
72
+ ```
73
+
74
+ The other four distributions use the same signature:
75
+ `fit_splitpop_loglogistic()`, `fit_splitpop_lognormal()`,
76
+ `fit_splitpop_gamma()`, `fit_splitpop_ggamma()`.
77
+
78
+ ## A note on the original Stata code
79
+
80
+ All five Stata programs this package translates compute a log-likelihood
81
+ term that turns out to be the marginal *density* where the formula requires
82
+ the marginal *hazard* (density divided by survival) — a discrepancy from
83
+ Yamaguchi's own published model. This was confirmed by fitting simulated
84
+ data with known parameters: the as-translated formula gives visibly biased
85
+ estimates, or fails to converge outright, while the corrected formula
86
+ implemented here recovers the true parameters accurately. See
87
+ [`docs/manual.html`](https://htmlpreview.github.io/?https://github.com/nobifukuda/splitpopsurv-py/blob/main/docs/manual.html#correction)
88
+ for the full derivation and the simulation results. This is the same
89
+ correction independently verified for the R and Stata ports of this model.
90
+
91
+ ## Development
92
+
93
+ ```bash
94
+ pip install -e ".[test]"
95
+ pytest
96
+ ```
97
+
98
+ ## License
99
+
100
+ MIT — see [LICENSE](LICENSE).
101
+
102
+ ## References
103
+
104
+ Prentice, R. L. (1974). A log gamma model and its maximum likelihood
105
+ estimation. *Biometrika*, 61(3), 539–544.
106
+
107
+ Schmidt, P., & Witte, A. D. (1989). Predicting criminal recidivism using
108
+ "split population" survival time models. *Journal of Econometrics*, 40(1),
109
+ 141–159.
110
+
111
+ Yamaguchi, K. (1992). Accelerated failure-time regression models with a
112
+ regression model of surviving fraction: An application to the analysis of
113
+ "permanent employment" in Japan. *Journal of the American Statistical
114
+ Association*, 87(418), 284–292.
115
+
116
+ Yamaguchi, K., & Ferguson, L. R. (1995). The stopping and spacing of
117
+ childbirths and their birth-history predictors: Rational-choice theory and
118
+ event-history analysis. *American Sociological Review*, 60(2), 272–298.
119
+
120
+ Yamaguchi, K. (1998). Mover-stayer models for analyzing event nonoccurrence
121
+ and event timing with time-dependent covariates: An application to an
122
+ analysis of remarriage. *Sociological Methodology*, 28(1), 327–361.
@@ -0,0 +1,100 @@
1
+ # splitpopsurv (Python)
2
+
3
+ Split-population (cure / mover–stayer) survival models in Python: an
4
+ accelerated failure-time regression for event timing among "movers",
5
+ combined with a logistic regression on the probability of belonging to
6
+ the immune "stayer" population.
7
+
8
+ Five baseline timing distributions are provided — **log-logistic**,
9
+ **Weibull**, **log-normal**, **gamma**, and the **generalized gamma** that
10
+ nests the other four — following Schmidt & Witte (1989) and Yamaguchi
11
+ (1992, 1998). This package is a Python translation of a set of Stata `ml`
12
+ programs, with the log-likelihood corrected to match the published model
13
+ and verified by simulation against known parameters.
14
+
15
+ 📖 **Full manual** (theory, formulas, the likelihood correction, and a
16
+ complete function reference): [`docs/manual.html`](https://htmlpreview.github.io/?https://github.com/nobifukuda/splitpopsurv-py/blob/main/docs/manual.html)
17
+
18
+ Companion packages implementing the same corrected models are available
19
+ for [R](https://github.com/nobifukuda/splitpopsurv) and
20
+ [Stata](https://github.com/nobifukuda/splitpopsurv-stata).
21
+
22
+ ## Author
23
+
24
+ **Nobutaka Fukuda**, Tohoku University — <nobutaka.fukuda@tohoku.ac.jp>
25
+
26
+ ## Installation
27
+
28
+ ```bash
29
+ pip install git+https://github.com/nobifukuda/splitpopsurv-py.git
30
+ ```
31
+
32
+ ## Usage
33
+
34
+ ```python
35
+ import pandas as pd
36
+ from splitpopsurv import fit_splitpop_weibull
37
+
38
+ # mydata needs: time, event (0/1), and your covariates
39
+ fit = fit_splitpop_weibull(
40
+ data=mydata,
41
+ time="time",
42
+ event="event",
43
+ h_vars=["x1"], # H_regression: covariates for timing
44
+ p_vars=["x2"], # P_regression: covariates for cure probability
45
+ )
46
+
47
+ print(fit.summary()) # coefficients, SEs, z-values, log-likelihood
48
+ fit.params # named parameter vector (numpy array)
49
+ fit.as_dataframe() # coefficients table as a pandas DataFrame
50
+ ```
51
+
52
+ The other four distributions use the same signature:
53
+ `fit_splitpop_loglogistic()`, `fit_splitpop_lognormal()`,
54
+ `fit_splitpop_gamma()`, `fit_splitpop_ggamma()`.
55
+
56
+ ## A note on the original Stata code
57
+
58
+ All five Stata programs this package translates compute a log-likelihood
59
+ term that turns out to be the marginal *density* where the formula requires
60
+ the marginal *hazard* (density divided by survival) — a discrepancy from
61
+ Yamaguchi's own published model. This was confirmed by fitting simulated
62
+ data with known parameters: the as-translated formula gives visibly biased
63
+ estimates, or fails to converge outright, while the corrected formula
64
+ implemented here recovers the true parameters accurately. See
65
+ [`docs/manual.html`](https://htmlpreview.github.io/?https://github.com/nobifukuda/splitpopsurv-py/blob/main/docs/manual.html#correction)
66
+ for the full derivation and the simulation results. This is the same
67
+ correction independently verified for the R and Stata ports of this model.
68
+
69
+ ## Development
70
+
71
+ ```bash
72
+ pip install -e ".[test]"
73
+ pytest
74
+ ```
75
+
76
+ ## License
77
+
78
+ MIT — see [LICENSE](LICENSE).
79
+
80
+ ## References
81
+
82
+ Prentice, R. L. (1974). A log gamma model and its maximum likelihood
83
+ estimation. *Biometrika*, 61(3), 539–544.
84
+
85
+ Schmidt, P., & Witte, A. D. (1989). Predicting criminal recidivism using
86
+ "split population" survival time models. *Journal of Econometrics*, 40(1),
87
+ 141–159.
88
+
89
+ Yamaguchi, K. (1992). Accelerated failure-time regression models with a
90
+ regression model of surviving fraction: An application to the analysis of
91
+ "permanent employment" in Japan. *Journal of the American Statistical
92
+ Association*, 87(418), 284–292.
93
+
94
+ Yamaguchi, K., & Ferguson, L. R. (1995). The stopping and spacing of
95
+ childbirths and their birth-history predictors: Rational-choice theory and
96
+ event-history analysis. *American Sociological Review*, 60(2), 272–298.
97
+
98
+ Yamaguchi, K. (1998). Mover-stayer models for analyzing event nonoccurrence
99
+ and event timing with time-dependent covariates: An application to an
100
+ analysis of remarriage. *Sociological Methodology*, 28(1), 327–361.
@@ -0,0 +1,36 @@
1
+ [build-system]
2
+ requires = ["setuptools>=77", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "splitpopsurv"
7
+ version = "0.1.0"
8
+ description = "Split-population (cure / mover-stayer) survival models"
9
+ readme = "README.md"
10
+ license = "MIT"
11
+ license-files = ["LICENSE"]
12
+ authors = [
13
+ { name = "Nobutaka Fukuda", email = "nobutaka.fukuda@tohoku.ac.jp" },
14
+ ]
15
+ requires-python = ">=3.9"
16
+ dependencies = [
17
+ "numpy>=1.20",
18
+ "scipy>=1.7",
19
+ "pandas>=1.3",
20
+ ]
21
+ classifiers = [
22
+ "Development Status :: 4 - Beta",
23
+ "Intended Audience :: Science/Research",
24
+ "Programming Language :: Python :: 3",
25
+ "Topic :: Scientific/Engineering :: Mathematics",
26
+ ]
27
+
28
+ [project.optional-dependencies]
29
+ test = ["pytest>=7.0"]
30
+
31
+ [project.urls]
32
+ Homepage = "https://github.com/nobifukuda/splitpopsurv-py"
33
+ "Bug Tracker" = "https://github.com/nobifukuda/splitpopsurv-py/issues"
34
+
35
+ [tool.setuptools.packages.find]
36
+ where = ["src"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,52 @@
1
+ """
2
+ splitpopsurv: Split-Population (Cure / Mover-Stayer) Survival Models
3
+ ======================================================================
4
+
5
+ Maximum-likelihood estimation of split-population survival models, which
6
+ combine an accelerated failure-time regression for event timing among
7
+ "movers" with a logistic regression on the probability of belonging to
8
+ the immune "stayer" population. See Schmidt & Witte (1989) and Yamaguchi
9
+ (1992, 1998) for the underlying theory, and the package README / manual
10
+ for a full derivation including a likelihood correction relative to the
11
+ Stata `ml` programs this package translates.
12
+
13
+ Author: Nobutaka Fukuda <nobutaka.fukuda@tohoku.ac.jp>
14
+ License: MIT
15
+
16
+ Main functions
17
+ --------------
18
+ fit_splitpop_loglogistic, fit_splitpop_weibull, fit_splitpop_lognormal,
19
+ fit_splitpop_gamma, fit_splitpop_ggamma
20
+ """
21
+
22
+ from .models import (
23
+ SplitPopResult,
24
+ fit_splitpop_loglogistic,
25
+ fit_splitpop_weibull,
26
+ fit_splitpop_lognormal,
27
+ fit_splitpop_gamma,
28
+ fit_splitpop_ggamma,
29
+ loglik_splitpop_loglogistic,
30
+ loglik_splitpop_weibull,
31
+ loglik_splitpop_lognormal,
32
+ loglik_splitpop_gamma,
33
+ loglik_splitpop_ggamma,
34
+ )
35
+
36
+ __version__ = "0.1.0"
37
+ __author__ = "Nobutaka Fukuda"
38
+ __email__ = "nobutaka.fukuda@tohoku.ac.jp"
39
+
40
+ __all__ = [
41
+ "SplitPopResult",
42
+ "fit_splitpop_loglogistic",
43
+ "fit_splitpop_weibull",
44
+ "fit_splitpop_lognormal",
45
+ "fit_splitpop_gamma",
46
+ "fit_splitpop_ggamma",
47
+ "loglik_splitpop_loglogistic",
48
+ "loglik_splitpop_weibull",
49
+ "loglik_splitpop_lognormal",
50
+ "loglik_splitpop_gamma",
51
+ "loglik_splitpop_ggamma",
52
+ ]
@@ -0,0 +1,566 @@
1
+ """
2
+ splitpopsurv.models
3
+ ====================
4
+
5
+ Split-population (cure / mover-stayer) survival models.
6
+
7
+ Author: Nobutaka Fukuda <nobutaka.fukuda@tohoku.ac.jp>
8
+ License: MIT
9
+
10
+ A Python translation of five Stata `ml` programs (SphLog, SphWieb, SphNom,
11
+ SphGam, SphGGam) for Log-logistic, Weibull, Log-normal, Gamma and
12
+ Generalized Gamma split-population survival models.
13
+
14
+ Statistical background: Schmidt & Witte (1989) "split population" survival
15
+ models and Yamaguchi (1992, 1998) mover-stayer / accelerated-failure-time
16
+ models with a logit regression on the surviving ("stayer"/"cured") fraction.
17
+ See Yamaguchi & Ferguson (1995, American Sociological Review 60(2)) eqs.
18
+ (3)-(6) and Yamaguchi (1998, Sociological Methodology 28) eq. (1)-(2) for
19
+ the underlying model. Full references: the package README / docs/manual.html.
20
+
21
+ Stata -> Python correspondence used throughout:
22
+ $ML_y1 -> time (duration/survival time)
23
+ $ML_y2 -> event (1 = failure observed, 0 = censored)
24
+ $ML_y3 -> group (0/1 indicator that flips the cure-prob
25
+ equation, exactly as in the original
26
+ Stata code)
27
+ invlogit(x) -> scipy.special.expit(x)
28
+ norm(x) -> scipy.stats.norm.cdf(x)
29
+ gammap(a, x) -> scipy.special.gammainc(a, x) (regularized
30
+ lower incomplete gamma)
31
+ lngamma(x) -> scipy.special.gammaln(x)
32
+ _pi -> numpy.pi
33
+ ml model lf/d0 ... -> scipy.optimize.minimize(..., method="L-BFGS-B")
34
+
35
+ Each `fit_splitpop_*()` function mirrors one Stata `ml model` block: it
36
+ takes the H_regression covariates (location/scale of the mover timing
37
+ distribution) and P_regression covariates (the cure-probability logit),
38
+ builds design matrices with an intercept, and maximizes the log-likelihood.
39
+
40
+ Ancillary shape parameters (Stata's unnamed "()" or "ln_sigma:"/"kappa:"
41
+ equations, i.e. intercept-only equations) are represented as single free
42
+ scalars in the parameter vector, exactly as in the original.
43
+
44
+ ------------------------------------------------------------------------
45
+ CORRECTION relative to the original Stata code (see docs/manual.html for
46
+ the full derivation and a simulation-based verification -- this is the
47
+ same correction independently verified for the R and Stata ports of this
48
+ model):
49
+
50
+ For a split-population model with mover survival S_m(t)/density f_m(t)
51
+ and cure probability p, the marginal survival and density are
52
+
53
+ S(t) = (1-p) S_m(t) + p f(t) = (1-p) f_m(t)
54
+
55
+ and the standard hazard-based log-likelihood contribution for person i is
56
+
57
+ delta_i * log(f(t_i)) + (1 - delta_i) * log(S(t_i))
58
+
59
+ equivalently delta_i * log(h(t_i)) + log(S(t_i)) where h(t) = f(t)/S(t)
60
+ is the TRUE marginal hazard (Yamaguchi 1998, eq. 2).
61
+
62
+ The original Stata code computes hz2/mhz = (1-p)*hz_m(t)*S_m(t), which
63
+ equals the marginal DENSITY f(t), not the marginal hazard h(t) = f(t)/S(t)
64
+ (it never divides by S(t)), then evaluates lnf = $ML_y2*ln(hz2) + ln(sv2)
65
+ for every observation -- double-counting log(S(t_i)) for every observed
66
+ event. This was confirmed by simulation with known true parameters (for
67
+ this Python port too, see tests/): the as-written formula gives visibly
68
+ biased estimates (especially for the cure-probability coefficients) or
69
+ fails to converge, while the corrected formula below recovers the true
70
+ parameters accurately.
71
+
72
+ This package implements the CORRECTED formula:
73
+ loglik_i = delta_i * log(f(t_i)) + (1 - delta_i) * log(S(t_i))
74
+ written as np.where(event == 1, log(marginal density), log(marginal
75
+ survival)), which is both statistically correct and numerically robust
76
+ (it avoids ever forming hazard = density / survival, so there is no 0/0
77
+ or inf*0 risk when survival underflows to exactly 0 in floating point at
78
+ extreme durations).
79
+ ------------------------------------------------------------------------
80
+ """
81
+
82
+ from __future__ import annotations
83
+
84
+ import numpy as np
85
+ import pandas as pd
86
+ from scipy import optimize
87
+ from scipy.special import expit, gammaln, gammainc
88
+ from scipy.stats import norm
89
+
90
+ __all__ = [
91
+ "SplitPopResult",
92
+ "fit_splitpop_loglogistic",
93
+ "fit_splitpop_weibull",
94
+ "fit_splitpop_lognormal",
95
+ "fit_splitpop_gamma",
96
+ "fit_splitpop_ggamma",
97
+ ]
98
+
99
+
100
+ # ---------------------------------------------------------------------
101
+ # Helpers (internal)
102
+ # ---------------------------------------------------------------------
103
+
104
+ def _design_matrix(data: pd.DataFrame, varlist):
105
+ """Build an intercept + covariates design matrix from column names."""
106
+ n = len(data)
107
+ cols = ["Intercept"] + list(varlist)
108
+ X = np.column_stack(
109
+ [np.ones(n)] + [np.asarray(data[v], dtype=float) for v in varlist]
110
+ )
111
+ return X, cols
112
+
113
+
114
+ def _cure_prob(theta_p, group):
115
+ """p = expit(theta_p) if group==1, 1-expit(theta_p) if group==0.
116
+
117
+ Kept exactly as in the Stata source: $ML_y3 flips which side of the
118
+ logistic curve is used for the cure probability.
119
+ """
120
+ p = expit(theta_p)
121
+ return np.where(group == 1, p, 1.0 - p)
122
+
123
+
124
+ def _split_loglik(event, mdens, msurv):
125
+ """delta*log(marginal density) + (1-delta)*log(marginal survival).
126
+
127
+ Written with np.where (not delta*log(f)+log(S)) so that the density
128
+ term is never evaluated where it doesn't matter -- avoids inf*0 -> nan
129
+ when survival underflows to 0 at extreme durations.
130
+ """
131
+ with np.errstate(divide="ignore", invalid="ignore"):
132
+ return np.where(event == 1, np.log(mdens), np.log(msurv))
133
+
134
+
135
+ def _safe_start_h(k_h, intercept_value=1.0):
136
+ """Positive-intercept, zero-slope start for Gamma's H_regression, where
137
+ theta1 = Xh @ b_h is used directly as a rate (no exp() transform) and
138
+ must stay positive regardless of covariate values."""
139
+ b0 = np.zeros(k_h)
140
+ b0[0] = intercept_value
141
+ return b0
142
+
143
+
144
+ _BAD_PENALTY = 1e10
145
+
146
+
147
+ def _neg_loglik_sum(loglik_fn):
148
+ # Wandering outside the valid parameter domain during a line-search
149
+ # trial step (e.g. a negative rate in the gamma model) produces NaN
150
+ # log-likelihood contributions. Most gradient-based optimizers cannot
151
+ # recover once the objective itself is NaN, so trial points like that
152
+ # are given a large but FINITE penalty instead -- the optimizer then
153
+ # treats them as simply "much worse" and backs away, rather than
154
+ # getting stuck. This never affects the optimum itself: at any point
155
+ # actually near a maximum the likelihood is finite by construction.
156
+ def obj(par, *args):
157
+ with np.errstate(all="ignore"):
158
+ ll = loglik_fn(par, *args)
159
+ total = np.sum(ll)
160
+ if not np.isfinite(total):
161
+ return _BAD_PENALTY
162
+ return -total
163
+ return obj
164
+
165
+
166
+ def _optimize(loglik_fn, start, args, method):
167
+ obj = _neg_loglik_sum(loglik_fn)
168
+ res = optimize.minimize(obj, start, args=args, method=method,
169
+ options={"maxiter": 2000})
170
+ return res
171
+
172
+
173
+ # ---------------------------------------------------------------------
174
+ # Result object
175
+ # ---------------------------------------------------------------------
176
+
177
+ class SplitPopResult:
178
+ """Result of a split-population survival model fit.
179
+
180
+ Attributes
181
+ ----------
182
+ params : numpy.ndarray
183
+ Point estimates, in the order given by ``param_names``.
184
+ param_names : list of str
185
+ Names, e.g. ``"H_regression:Intercept"``, ``"ln_sigma"``,
186
+ ``"P_regression:x2"``.
187
+ loglik : float
188
+ Maximized log-likelihood.
189
+ converged : bool
190
+ Whether the optimizer reported convergence.
191
+ nobs : int
192
+ Number of observations used.
193
+ distribution : str
194
+ Which baseline distribution was fit.
195
+ """
196
+
197
+ def __init__(self, params, param_names, loglik, hessian_inv, converged,
198
+ message, nobs, distribution, nit):
199
+ self.params = np.asarray(params)
200
+ self.param_names = list(param_names)
201
+ self.loglik = float(loglik)
202
+ self._hessian_inv = hessian_inv
203
+ self.converged = bool(converged)
204
+ self.message = message
205
+ self.nobs = int(nobs)
206
+ self.distribution = distribution
207
+ self.nit = nit
208
+
209
+ @property
210
+ def bse(self):
211
+ """Standard errors (sqrt of the diagonal of the inverse Hessian
212
+ of the negative log-likelihood, i.e. the observed-information
213
+ covariance estimate -- the same convention as R's maxLik and
214
+ Stata's ml)."""
215
+ diag = np.diag(self._hessian_inv)
216
+ with np.errstate(invalid="ignore"):
217
+ return np.sqrt(np.where(diag >= 0, diag, np.nan))
218
+
219
+ @property
220
+ def zvalues(self):
221
+ with np.errstate(divide="ignore", invalid="ignore"):
222
+ return self.params / self.bse
223
+
224
+ @property
225
+ def pvalues(self):
226
+ with np.errstate(invalid="ignore"):
227
+ return 2.0 * (1.0 - norm.cdf(np.abs(self.zvalues)))
228
+
229
+ def conf_int(self, alpha=0.05):
230
+ z = norm.ppf(1 - alpha / 2)
231
+ lo = self.params - z * self.bse
232
+ hi = self.params + z * self.bse
233
+ return pd.DataFrame({"lower": lo, "upper": hi}, index=self.param_names)
234
+
235
+ def as_dataframe(self):
236
+ return pd.DataFrame(
237
+ {
238
+ "coef": self.params,
239
+ "std err": self.bse,
240
+ "z": self.zvalues,
241
+ "P>|z|": self.pvalues,
242
+ },
243
+ index=self.param_names,
244
+ )
245
+
246
+ def summary(self):
247
+ df = self.as_dataframe()
248
+ lines = []
249
+ lines.append(f"Split-population survival model ({self.distribution})")
250
+ lines.append(f"No. observations: {self.nobs} Log-likelihood: {self.loglik:.4f}")
251
+ lines.append(f"Converged: {self.converged} Iterations: {self.nit}")
252
+ lines.append("-" * 70)
253
+ lines.append(df.to_string(float_format=lambda x: f"{x:0.4f}"))
254
+ return "\n".join(lines)
255
+
256
+ def __repr__(self):
257
+ return f"<SplitPopResult distribution={self.distribution!r} loglik={self.loglik:.3f} converged={self.converged}>"
258
+
259
+
260
+ def _make_result(res, param_names, nobs, distribution):
261
+ hess_inv = res.hess_inv
262
+ if not isinstance(hess_inv, np.ndarray):
263
+ hess_inv = hess_inv.todense() if hasattr(hess_inv, "todense") else np.asarray(hess_inv)
264
+ return SplitPopResult(
265
+ params=res.x,
266
+ param_names=param_names,
267
+ loglik=-res.fun,
268
+ hessian_inv=hess_inv,
269
+ converged=bool(res.success),
270
+ message=res.message,
271
+ nobs=nobs,
272
+ distribution=distribution,
273
+ nit=res.nit if hasattr(res, "nit") else None,
274
+ )
275
+
276
+
277
+ # =======================================================================
278
+ # 1. Log-logistic distribution (Stata: SphLog)
279
+ # =======================================================================
280
+
281
+ def loglik_splitpop_loglogistic(par, time, event, group, Xh, Xp):
282
+ k_h = Xh.shape[1]
283
+ b_h = par[:k_h]
284
+ theta2 = par[k_h]
285
+ b_p = par[k_h + 1:]
286
+
287
+ theta1 = Xh @ b_h
288
+ theta3 = Xp @ b_p
289
+
290
+ p = _cure_prob(theta3, group)
291
+ lam = np.exp(-theta1)
292
+ sigma = 1.0 / theta2
293
+ sv = 1.0 / (1.0 + (lam * time) ** sigma) # S_m(t)
294
+ hz = (lam * sigma * (lam * time) ** (sigma - 1.0)) / (1.0 + (lam * time) ** sigma) # h_m(t)
295
+ msv = (1 - p) * sv + p # S(t)
296
+ mdens = (1 - p) * hz * sv # f(t)
297
+
298
+ return _split_loglik(event, mdens, msv)
299
+
300
+
301
+ def fit_splitpop_loglogistic(data: pd.DataFrame, time: str, event: str,
302
+ h_vars, p_vars, group: str | None = None,
303
+ start=None, method="L-BFGS-B"):
304
+ """Fit a split-population log-logistic survival model.
305
+
306
+ Parameters
307
+ ----------
308
+ data : pandas.DataFrame
309
+ time, event : str
310
+ Column names for duration ($ML_y1) and 0/1 event indicator
311
+ ($ML_y2; 1 = failure observed, 0 = censored).
312
+ h_vars : list of str
313
+ H_regression covariates (timing distribution).
314
+ p_vars : list of str
315
+ P_regression covariates (cure/stayer probability).
316
+ group : str or None
317
+ Column name of a 0/1 indicator that flips the cure-probability
318
+ logit ($ML_y3 in the Stata source). If None, a constant of 1 is
319
+ used for every observation (the ordinary case).
320
+ start : numpy.ndarray or None
321
+ Starting values; a sensible default is used if None.
322
+ method : str
323
+ Passed to ``scipy.optimize.minimize``. Default "BFGS"; try
324
+ "Nelder-Mead" if that fails to converge.
325
+
326
+ Returns
327
+ -------
328
+ SplitPopResult
329
+ """
330
+ Xh, hcols = _design_matrix(data, h_vars)
331
+ Xp, pcols = _design_matrix(data, p_vars)
332
+ t = np.asarray(data[time], dtype=float)
333
+ y = np.asarray(data[event], dtype=float)
334
+ g = np.ones(len(data)) if group is None else np.asarray(data[group], dtype=float)
335
+
336
+ if start is None:
337
+ start = np.concatenate([np.zeros(Xh.shape[1]), [1.0], np.zeros(Xp.shape[1])])
338
+
339
+ res = _optimize(loglik_splitpop_loglogistic, start, (t, y, g, Xh, Xp), method)
340
+ names = [f"H_regression:{c}" for c in hcols] + ["theta2"] + [f"P_regression:{c}" for c in pcols]
341
+ return _make_result(res, names, len(data), "loglogistic")
342
+
343
+
344
+ # =======================================================================
345
+ # 2. Weibull distribution (Stata: SphWieb)
346
+ # =======================================================================
347
+
348
+ def loglik_splitpop_weibull(par, time, event, group, Xh, Xp):
349
+ k_h = Xh.shape[1]
350
+ b_h = par[:k_h]
351
+ theta2 = par[k_h]
352
+ b_p = par[k_h + 1:]
353
+
354
+ theta1 = Xh @ b_h
355
+ theta3 = Xp @ b_p
356
+
357
+ p = _cure_prob(theta3, group)
358
+ lam = np.exp(theta2)
359
+ sv = np.exp(-np.exp(theta1) * (time ** lam)) # S_m(t)
360
+ hz = lam * (time ** (lam - 1.0)) * np.exp(theta1) # h_m(t)
361
+ msv = (1 - p) * sv + p # S(t)
362
+ mdens = (1 - p) * hz * sv # f(t)
363
+
364
+ return _split_loglik(event, mdens, msv)
365
+
366
+
367
+ def fit_splitpop_weibull(data: pd.DataFrame, time: str, event: str,
368
+ h_vars, p_vars, group: str | None = None,
369
+ start=None, method="L-BFGS-B"):
370
+ """Fit a split-population Weibull survival model. See
371
+ :func:`fit_splitpop_loglogistic` for the argument reference."""
372
+ Xh, hcols = _design_matrix(data, h_vars)
373
+ Xp, pcols = _design_matrix(data, p_vars)
374
+ t = np.asarray(data[time], dtype=float)
375
+ y = np.asarray(data[event], dtype=float)
376
+ g = np.ones(len(data)) if group is None else np.asarray(data[group], dtype=float)
377
+
378
+ if start is None:
379
+ start = np.concatenate([np.zeros(Xh.shape[1]), [0.0], np.zeros(Xp.shape[1])])
380
+
381
+ res = _optimize(loglik_splitpop_weibull, start, (t, y, g, Xh, Xp), method)
382
+ names = [f"H_regression:{c}" for c in hcols] + ["ln_sigma"] + [f"P_regression:{c}" for c in pcols]
383
+ return _make_result(res, names, len(data), "weibull")
384
+
385
+
386
+ # =======================================================================
387
+ # 3. Log-normal distribution (Stata: SphNom)
388
+ # =======================================================================
389
+
390
+ def loglik_splitpop_lognormal(par, time, event, group, Xh, Xp):
391
+ k_h = Xh.shape[1]
392
+ b_h = par[:k_h]
393
+ theta2 = par[k_h]
394
+ b_p = par[k_h + 1:]
395
+
396
+ theta1 = Xh @ b_h
397
+ theta3 = Xp @ b_p
398
+
399
+ p = _cure_prob(theta3, group)
400
+ sigma = np.exp(theta2)
401
+ lam = np.log(time) - theta1
402
+ sv = 1.0 - norm.cdf(lam / sigma) # S_m(t)
403
+ pdf_m = np.exp(-(lam ** 2) / (2 * sigma ** 2)) / (np.sqrt(2 * np.pi) * sigma * time) # f_m(t), direct (no /sv)
404
+ msv = (1 - p) * sv + p # S(t)
405
+ mdens = (1 - p) * pdf_m # f(t)
406
+
407
+ return _split_loglik(event, mdens, msv)
408
+
409
+
410
+ def fit_splitpop_lognormal(data: pd.DataFrame, time: str, event: str,
411
+ h_vars, p_vars, group: str | None = None,
412
+ start=None, method="L-BFGS-B"):
413
+ """Fit a split-population log-normal survival model. See
414
+ :func:`fit_splitpop_loglogistic` for the argument reference."""
415
+ Xh, hcols = _design_matrix(data, h_vars)
416
+ Xp, pcols = _design_matrix(data, p_vars)
417
+ t = np.asarray(data[time], dtype=float)
418
+ y = np.asarray(data[event], dtype=float)
419
+ g = np.ones(len(data)) if group is None else np.asarray(data[group], dtype=float)
420
+
421
+ if start is None:
422
+ start = np.concatenate([np.zeros(Xh.shape[1]), [0.0], np.zeros(Xp.shape[1])])
423
+
424
+ res = _optimize(loglik_splitpop_lognormal, start, (t, y, g, Xh, Xp), method)
425
+ names = [f"H_regression:{c}" for c in hcols] + ["ln_sigma"] + [f"P_regression:{c}" for c in pcols]
426
+ return _make_result(res, names, len(data), "lognormal")
427
+
428
+
429
+ # =======================================================================
430
+ # 4. Gamma distribution (Stata: SphGam)
431
+ # =======================================================================
432
+
433
+ def loglik_splitpop_gamma(par, time, event, group, Xh, Xp):
434
+ k_h = Xh.shape[1]
435
+ b_h = par[:k_h]
436
+ theta2 = par[k_h]
437
+ b_p = par[k_h + 1:]
438
+
439
+ theta1 = Xh @ b_h
440
+ theta3 = Xp @ b_p
441
+
442
+ p = _cure_prob(theta3, group)
443
+ l = theta1 * time
444
+ k = theta2
445
+ cdf = gammainc(k, l) # gammap(k, l)
446
+ gam = np.exp(gammaln(k))
447
+ pdf_m = (theta1 * (l ** (k - 1.0)) * np.exp(-l)) / gam # f_m(t), direct (no /sv)
448
+ sv = 1.0 - cdf # S_m(t)
449
+ msv = (1 - p) * sv + p # S(t)
450
+ mdens = (1 - p) * pdf_m # f(t)
451
+
452
+ return _split_loglik(event, mdens, msv)
453
+
454
+
455
+ def fit_splitpop_gamma(data: pd.DataFrame, time: str, event: str,
456
+ h_vars, p_vars, group: str | None = None,
457
+ start=None, method="L-BFGS-B"):
458
+ """Fit a split-population gamma survival model.
459
+
460
+ Note: unlike the other four models, the H_regression linear index is
461
+ used directly as a rate (no ``exp()`` transform), so it must stay
462
+ positive. The default start sets a positive intercept and zero slopes
463
+ to keep it safe regardless of covariate values.
464
+
465
+ See :func:`fit_splitpop_loglogistic` for the argument reference.
466
+ """
467
+ Xh, hcols = _design_matrix(data, h_vars)
468
+ Xp, pcols = _design_matrix(data, p_vars)
469
+ t = np.asarray(data[time], dtype=float)
470
+ y = np.asarray(data[event], dtype=float)
471
+ g = np.ones(len(data)) if group is None else np.asarray(data[group], dtype=float)
472
+
473
+ if start is None:
474
+ start = np.concatenate([_safe_start_h(Xh.shape[1]), [1.0], np.zeros(Xp.shape[1])])
475
+
476
+ res = _optimize(loglik_splitpop_gamma, start, (t, y, g, Xh, Xp), method)
477
+ names = [f"H_regression:{c}" for c in hcols] + ["kappa"] + [f"P_regression:{c}" for c in pcols]
478
+ return _make_result(res, names, len(data), "gamma")
479
+
480
+
481
+ # =======================================================================
482
+ # 5. Generalized Gamma distribution (Stata: SphGGam / KYmodel5, d0 method)
483
+ # =======================================================================
484
+
485
+ def loglik_splitpop_ggamma(par, time, event, group, Xh, Xp):
486
+ k_h = Xh.shape[1]
487
+ b_h = par[:k_h]
488
+ theta2 = par[k_h]
489
+ theta3s = par[k_h + 1]
490
+ b_p = par[k_h + 2:]
491
+
492
+ theta1 = Xh @ b_h
493
+ theta4 = Xp @ b_p
494
+
495
+ p = _cure_prob(theta4, group)
496
+
497
+ k = np.where(np.abs(theta3s) < 0.01, np.sign(theta3s) * 0.01, theta3s)
498
+ if k == 0: # sign(0) == 0 in both Stata and numpy; break the tie like the
499
+ k = np.array(0.01) # Stata guard would if it ever landed exactly on 0
500
+
501
+ s = np.exp(theta2)
502
+ l = np.abs(k) ** (-2.0)
503
+ z = np.sign(k) * (np.log(time) - theta1) / s
504
+ u = l * np.exp(np.abs(k) * z)
505
+
506
+ small_k = np.abs(k) < 0.01
507
+ if small_k:
508
+ cdf = norm.cdf(z)
509
+ elif k >= 0.01:
510
+ cdf = gammainc(l, u) # gammap(l, u)
511
+ else:
512
+ cdf = 1.0 - gammainc(l, u)
513
+
514
+ gam = np.exp(gammaln(l))
515
+ if small_k:
516
+ pdf_m = np.exp(-(z ** 2) / 2.0) / (s * time * np.sqrt(2 * np.pi))
517
+ else:
518
+ pdf_m = (l ** l * np.exp(z * np.sqrt(l) - u)) / (s * time * np.sqrt(l) * gam) # f_m(t), direct
519
+
520
+ sv = 1.0 - cdf # S_m(t)
521
+ msv = (1 - p) * sv + p # S(t)
522
+ mdens = (1 - p) * pdf_m # f(t)
523
+
524
+ return _split_loglik(event, mdens, msv)
525
+
526
+
527
+ def fit_splitpop_ggamma(data: pd.DataFrame, time: str, event: str,
528
+ h_vars, p_vars, group: str | None = None,
529
+ start=None, method="L-BFGS-B"):
530
+ """Fit a split-population generalized gamma survival model (Prentice
531
+ 1974 / Yamaguchi & Ferguson 1995, note 10) -- the family that nests
532
+ Weibull (kappa=1), log-normal (kappa=0), and gamma (sigma=1).
533
+
534
+ Note: ``|kappa| < 0.01`` switches to a log-normal-equivalent branch, a
535
+ genuine kink in the likelihood surface (preserved from the Stata
536
+ guard). If standard errors come back non-finite, the optimizer likely
537
+ landed near that threshold -- try a different starting kappa.
538
+
539
+ See :func:`fit_splitpop_loglogistic` for the argument reference.
540
+ """
541
+ Xh, hcols = _design_matrix(data, h_vars)
542
+ Xp, pcols = _design_matrix(data, p_vars)
543
+ t = np.asarray(data[time], dtype=float)
544
+ y = np.asarray(data[event], dtype=float)
545
+ g = np.ones(len(data)) if group is None else np.asarray(data[group], dtype=float)
546
+
547
+ if start is None:
548
+ start = np.concatenate([np.zeros(Xh.shape[1]), [0.0, 1.0], np.zeros(Xp.shape[1])])
549
+
550
+ res = _optimize(loglik_splitpop_ggamma, start, (t, y, g, Xh, Xp), method)
551
+ names = ([f"H_regression:{c}" for c in hcols] + ["ln_sigma", "kappa"]
552
+ + [f"P_regression:{c}" for c in pcols])
553
+ return _make_result(res, names, len(data), "ggamma")
554
+
555
+
556
+ # ---------------------------------------------------------------------
557
+ # Notes on the pieces that don't carry over from Stata:
558
+ #
559
+ # - `ml check` / `ml search`: call the relevant loglik_splitpop_*()
560
+ # function once by hand at candidate starting values, and/or try
561
+ # several starts or `method` values if convergence fails.
562
+ # - `ml graph`: plot fitted survival/hazard curves yourself (e.g. with
563
+ # matplotlib) from `result.params` plugged back into the survival/
564
+ # density formulas above.
565
+ # - `summary(fit)`'s closest analogue here is `result.summary()`.
566
+ # ---------------------------------------------------------------------
@@ -0,0 +1,122 @@
1
+ Metadata-Version: 2.4
2
+ Name: splitpopsurv
3
+ Version: 0.1.0
4
+ Summary: Split-population (cure / mover-stayer) survival models
5
+ Author-email: Nobutaka Fukuda <nobutaka.fukuda@tohoku.ac.jp>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/nobifukuda/splitpopsurv-py
8
+ Project-URL: Bug Tracker, https://github.com/nobifukuda/splitpopsurv-py/issues
9
+ Classifier: Development Status :: 4 - Beta
10
+ Classifier: Intended Audience :: Science/Research
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: Topic :: Scientific/Engineering :: Mathematics
13
+ Requires-Python: >=3.9
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: numpy>=1.20
17
+ Requires-Dist: scipy>=1.7
18
+ Requires-Dist: pandas>=1.3
19
+ Provides-Extra: test
20
+ Requires-Dist: pytest>=7.0; extra == "test"
21
+ Dynamic: license-file
22
+
23
+ # splitpopsurv (Python)
24
+
25
+ Split-population (cure / mover–stayer) survival models in Python: an
26
+ accelerated failure-time regression for event timing among "movers",
27
+ combined with a logistic regression on the probability of belonging to
28
+ the immune "stayer" population.
29
+
30
+ Five baseline timing distributions are provided — **log-logistic**,
31
+ **Weibull**, **log-normal**, **gamma**, and the **generalized gamma** that
32
+ nests the other four — following Schmidt & Witte (1989) and Yamaguchi
33
+ (1992, 1998). This package is a Python translation of a set of Stata `ml`
34
+ programs, with the log-likelihood corrected to match the published model
35
+ and verified by simulation against known parameters.
36
+
37
+ 📖 **Full manual** (theory, formulas, the likelihood correction, and a
38
+ complete function reference): [`docs/manual.html`](https://htmlpreview.github.io/?https://github.com/nobifukuda/splitpopsurv-py/blob/main/docs/manual.html)
39
+
40
+ Companion packages implementing the same corrected models are available
41
+ for [R](https://github.com/nobifukuda/splitpopsurv) and
42
+ [Stata](https://github.com/nobifukuda/splitpopsurv-stata).
43
+
44
+ ## Author
45
+
46
+ **Nobutaka Fukuda**, Tohoku University — <nobutaka.fukuda@tohoku.ac.jp>
47
+
48
+ ## Installation
49
+
50
+ ```bash
51
+ pip install git+https://github.com/nobifukuda/splitpopsurv-py.git
52
+ ```
53
+
54
+ ## Usage
55
+
56
+ ```python
57
+ import pandas as pd
58
+ from splitpopsurv import fit_splitpop_weibull
59
+
60
+ # mydata needs: time, event (0/1), and your covariates
61
+ fit = fit_splitpop_weibull(
62
+ data=mydata,
63
+ time="time",
64
+ event="event",
65
+ h_vars=["x1"], # H_regression: covariates for timing
66
+ p_vars=["x2"], # P_regression: covariates for cure probability
67
+ )
68
+
69
+ print(fit.summary()) # coefficients, SEs, z-values, log-likelihood
70
+ fit.params # named parameter vector (numpy array)
71
+ fit.as_dataframe() # coefficients table as a pandas DataFrame
72
+ ```
73
+
74
+ The other four distributions use the same signature:
75
+ `fit_splitpop_loglogistic()`, `fit_splitpop_lognormal()`,
76
+ `fit_splitpop_gamma()`, `fit_splitpop_ggamma()`.
77
+
78
+ ## A note on the original Stata code
79
+
80
+ All five Stata programs this package translates compute a log-likelihood
81
+ term that turns out to be the marginal *density* where the formula requires
82
+ the marginal *hazard* (density divided by survival) — a discrepancy from
83
+ Yamaguchi's own published model. This was confirmed by fitting simulated
84
+ data with known parameters: the as-translated formula gives visibly biased
85
+ estimates, or fails to converge outright, while the corrected formula
86
+ implemented here recovers the true parameters accurately. See
87
+ [`docs/manual.html`](https://htmlpreview.github.io/?https://github.com/nobifukuda/splitpopsurv-py/blob/main/docs/manual.html#correction)
88
+ for the full derivation and the simulation results. This is the same
89
+ correction independently verified for the R and Stata ports of this model.
90
+
91
+ ## Development
92
+
93
+ ```bash
94
+ pip install -e ".[test]"
95
+ pytest
96
+ ```
97
+
98
+ ## License
99
+
100
+ MIT — see [LICENSE](LICENSE).
101
+
102
+ ## References
103
+
104
+ Prentice, R. L. (1974). A log gamma model and its maximum likelihood
105
+ estimation. *Biometrika*, 61(3), 539–544.
106
+
107
+ Schmidt, P., & Witte, A. D. (1989). Predicting criminal recidivism using
108
+ "split population" survival time models. *Journal of Econometrics*, 40(1),
109
+ 141–159.
110
+
111
+ Yamaguchi, K. (1992). Accelerated failure-time regression models with a
112
+ regression model of surviving fraction: An application to the analysis of
113
+ "permanent employment" in Japan. *Journal of the American Statistical
114
+ Association*, 87(418), 284–292.
115
+
116
+ Yamaguchi, K., & Ferguson, L. R. (1995). The stopping and spacing of
117
+ childbirths and their birth-history predictors: Rational-choice theory and
118
+ event-history analysis. *American Sociological Review*, 60(2), 272–298.
119
+
120
+ Yamaguchi, K. (1998). Mover-stayer models for analyzing event nonoccurrence
121
+ and event timing with time-dependent covariates: An application to an
122
+ analysis of remarriage. *Sociological Methodology*, 28(1), 327–361.
@@ -0,0 +1,11 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/splitpopsurv/__init__.py
5
+ src/splitpopsurv/models.py
6
+ src/splitpopsurv.egg-info/PKG-INFO
7
+ src/splitpopsurv.egg-info/SOURCES.txt
8
+ src/splitpopsurv.egg-info/dependency_links.txt
9
+ src/splitpopsurv.egg-info/requires.txt
10
+ src/splitpopsurv.egg-info/top_level.txt
11
+ tests/test_models.py
@@ -0,0 +1,6 @@
1
+ numpy>=1.20
2
+ scipy>=1.7
3
+ pandas>=1.3
4
+
5
+ [test]
6
+ pytest>=7.0
@@ -0,0 +1 @@
1
+ splitpopsurv
@@ -0,0 +1,141 @@
1
+ import numpy as np
2
+ import pandas as pd
3
+ import pytest
4
+
5
+ from splitpopsurv import (
6
+ fit_splitpop_loglogistic,
7
+ fit_splitpop_weibull,
8
+ fit_splitpop_lognormal,
9
+ fit_splitpop_gamma,
10
+ fit_splitpop_ggamma,
11
+ loglik_splitpop_loglogistic,
12
+ loglik_splitpop_weibull,
13
+ loglik_splitpop_lognormal,
14
+ loglik_splitpop_gamma,
15
+ loglik_splitpop_ggamma,
16
+ )
17
+
18
+
19
+ def make_weibull_data(n=150, seed=1):
20
+ rng = np.random.default_rng(seed)
21
+ x1 = rng.normal(size=n)
22
+ x2 = rng.binomial(1, 0.5, size=n).astype(float)
23
+ cured = rng.binomial(1, 1 / (1 + np.exp(-(0.2 + 0.5 * x2))), size=n)
24
+ u = rng.uniform(size=n)
25
+ t_latent = (-np.log(u) / np.exp(0.5 - 0.4 * x1)) ** (1 / 1.3)
26
+ censor = rng.exponential(1 / 0.2, size=n)
27
+ time = np.where(cured == 1, censor, np.minimum(t_latent, censor))
28
+ event = np.where(cured == 1, 0, (t_latent <= censor).astype(int))
29
+ time = np.maximum(time, 1e-3)
30
+ return pd.DataFrame({"time": time, "event": event, "x1": x1, "x2": x2})
31
+
32
+
33
+ @pytest.fixture
34
+ def data():
35
+ return make_weibull_data()
36
+
37
+
38
+ @pytest.mark.parametrize(
39
+ "fit_fn,kwargs",
40
+ [
41
+ (fit_splitpop_loglogistic, {}),
42
+ (fit_splitpop_weibull, {}),
43
+ (fit_splitpop_lognormal, {}),
44
+ (fit_splitpop_gamma, {"h_vars": []}),
45
+ (fit_splitpop_ggamma, {}),
46
+ ],
47
+ )
48
+ def test_all_distributions_converge(data, fit_fn, kwargs):
49
+ h_vars = kwargs.pop("h_vars", ["x1"])
50
+ res = fit_fn(data, "time", "event", h_vars, ["x2"], **kwargs)
51
+ assert res.converged
52
+ assert np.isfinite(res.loglik)
53
+ assert not np.any(np.isnan(res.params))
54
+
55
+
56
+ def test_parameter_naming_convention(data):
57
+ res = fit_splitpop_weibull(data, "time", "event", ["x1"], ["x2"])
58
+ assert any(n.startswith("H_regression:") for n in res.param_names)
59
+ assert any(n.startswith("P_regression:") for n in res.param_names)
60
+ assert "ln_sigma" in res.param_names
61
+
62
+
63
+ def test_group_default_matches_explicit_all_ones(data):
64
+ data2 = data.copy()
65
+ data2["grp"] = 1
66
+ res_default = fit_splitpop_weibull(data, "time", "event", ["x1"], ["x2"])
67
+ res_explicit = fit_splitpop_weibull(data2, "time", "event", ["x1"], ["x2"], group="grp")
68
+ np.testing.assert_allclose(res_default.params, res_explicit.params, rtol=1e-4)
69
+
70
+
71
+ def test_corrected_likelihood_recovers_known_weibull_parameters():
72
+ """Regression guard for the likelihood correction (larger n for a
73
+ tight tolerance -- mirrors the same check done for the R and Stata
74
+ ports of this exact model)."""
75
+ rng = np.random.default_rng(42)
76
+ n = 20000
77
+ x1 = rng.normal(size=n)
78
+ x2 = rng.binomial(1, 0.5, size=n).astype(float)
79
+ beta0, beta1, shape = 0.5, -0.4, 1.3
80
+ alpha0, alpha1 = 0.2, 0.8
81
+ theta1 = beta0 + beta1 * x1
82
+ cured = rng.binomial(1, 1 / (1 + np.exp(-(alpha0 + alpha1 * x2))), size=n)
83
+ u = rng.uniform(size=n)
84
+ t_latent = (-np.log(u) / np.exp(theta1)) ** (1 / shape)
85
+ censor = rng.exponential(1 / 0.05, size=n)
86
+ time = np.where(cured == 1, censor, np.minimum(t_latent, censor))
87
+ event = np.where(cured == 1, 0, (t_latent <= censor).astype(int))
88
+ time = np.maximum(time, 1e-4)
89
+ df = pd.DataFrame({"time": time, "event": event, "x1": x1, "x2": x2})
90
+
91
+ res = fit_splitpop_weibull(df, "time", "event", ["x1"], ["x2"])
92
+ est = dict(zip(res.param_names, res.params))
93
+
94
+ assert abs(est["H_regression:Intercept"] - beta0) < 0.1
95
+ assert abs(est["H_regression:x1"] - beta1) < 0.1
96
+ assert abs(est["P_regression:Intercept"] - alpha0) < 0.1
97
+ assert abs(est["P_regression:x2"] - alpha1) < 0.15
98
+
99
+
100
+ def test_loglik_functions_return_finite_correctly_sized_vectors():
101
+ rng = np.random.default_rng(2)
102
+ n = 30
103
+ Xh = np.column_stack([np.ones(n), rng.normal(size=n)])
104
+ Xp = np.column_stack([np.ones(n), rng.binomial(1, 0.5, size=n).astype(float)])
105
+ time = rng.uniform(0.1, 5, size=n)
106
+ event = rng.binomial(1, 0.7, size=n).astype(float)
107
+ group = np.ones(n)
108
+
109
+ ll1 = loglik_splitpop_loglogistic(np.array([0, 0, 1, 0, 0], dtype=float), time, event, group, Xh, Xp)
110
+ ll2 = loglik_splitpop_weibull(np.array([0, 0, 0, 0, 0], dtype=float), time, event, group, Xh, Xp)
111
+ ll3 = loglik_splitpop_lognormal(np.array([0, 0, 0, 0, 0], dtype=float), time, event, group, Xh, Xp)
112
+ ll4 = loglik_splitpop_gamma(np.array([1, 0, 1, 0, 0], dtype=float), time, event, group, Xh, Xp)
113
+ ll5 = loglik_splitpop_ggamma(np.array([0, 0, 0, 1, 0, 0], dtype=float), time, event, group, Xh, Xp)
114
+
115
+ for ll in (ll1, ll2, ll3, ll4, ll5):
116
+ assert len(ll) == n
117
+ assert np.all(np.isfinite(ll))
118
+
119
+
120
+ def test_censored_observation_never_depends_on_mover_density_term():
121
+ """Regression guard for the likelihood correction: for a censored
122
+ observation (event=0), the contribution must equal log(marginal
123
+ survival) exactly, regardless of how extreme the mover hazard/density
124
+ is at that duration."""
125
+ Xh = np.array([[1.0, 0.0]])
126
+ Xp = np.array([[1.0, 0.0]])
127
+ ll = loglik_splitpop_weibull(
128
+ np.array([0, 0, 0, 0, 0], dtype=float),
129
+ time=np.array([1e6]), event=np.array([0.0]),
130
+ group=np.array([1.0]), Xh=Xh, Xp=Xp,
131
+ )
132
+ assert np.isfinite(ll[0])
133
+
134
+
135
+ def test_summary_and_dataframe_accessors(data):
136
+ res = fit_splitpop_weibull(data, "time", "event", ["x1"], ["x2"])
137
+ text = res.summary()
138
+ assert "weibull" in text
139
+ df = res.as_dataframe()
140
+ assert list(df.columns) == ["coef", "std err", "z", "P>|z|"]
141
+ assert len(df) == len(res.param_names)