pybhatlib 0.2.0__tar.gz → 0.3.0__tar.gz

{pybhatlib-0.2.0 → pybhatlib-0.3.0}/.gitignore

@@ -30,6 +30,9 @@ htmlcov/
 *.pdf
 results/
 
+# Original GAUSS source
+chandra_code/
+
 # OS
 .DS_Store
 Thumbs.db

pybhatlib-0.3.0/GAUSS_INTEGRATION_PLAN.md

@@ -0,0 +1,235 @@
+# Refine pybhatlib from Chandra's GAUSS Code
+
+## Context
+
+We compared the authoritative GAUSS implementation (`chandra_code/mnp_lat_TRAVELMODE.gss`) against pybhatlib and found 10+ gaps. The biggest performance bottleneck is **numerical gradients everywhere** — the GAUSS code has full analytic gradient chains via sequential LDLT conditioning with truncated moments, while Python uses finite differences (3N+1 function evaluations per iteration).
+
+This plan addresses refinements in priority order, grouped into independently testable phases.
+
+---
+
+## Phase A: Quick Wins (Robustness Improvements)
+
+**Files to modify:**
+- `src/pybhatlib/matgradient/_spherical.py` — add π·logistic constraint + numerical safeguards
+- `src/pybhatlib/models/mnp/_mnp_model.py` — better starting values
+
+### A1. Spherical angle constraint: π·logistic mapping
+
+GAUSS `cholspherparmunconst` (line 3726) maps unconstrained θ** → [0,π] via:
+```
+θ* = π / (1 + exp(-θ**))
+```
+Our `theta_to_corr()` uses raw angles without this constraint. Add the logistic mapping at the entry point of `theta_to_corr()` and inverse at `corr_to_theta()` (if exists).
+
+### A2. Numerical safeguards in spherical Cholesky
+
+GAUSS (lines 4202-4205) thresholds tiny trig values:
+```
+if |cos(θ)| < 6.12e-17 → cos = 0
+if |sin(θ)| < 1.22e-16 → sin = 0
+```
+Add to `_build_cholesky_from_angles()`.
+
+### A3. Better starting values
+
+GAUSS uses `0.5*I + 0.5*ones(K,K)` as initial correlation structure. Our MNPModel uses zeros for correlation params. Change starting values for lambda_params correlation portion to produce a moderate correlation structure.
+
+### Verification
+- All 126 existing tests must still pass
+- `theta_to_corr()` should produce identical results for angles already in [0,π]
+- New starting values should not worsen convergence on existing examples
+
+---
+
+## Phase B: GGE Ordering + TG/TGBME Methods
+
+**Files to create:**
+- `src/pybhatlib/gradmvn/_ordering.py` — GGE/Gibson ordering algorithm
+- `tests/test_gradmvn/test_ordering.py`
+
+**Files to modify:**
+- `src/pybhatlib/gradmvn/_mvncd.py` — add TG, TGBME methods + ordering dispatch
+- `src/pybhatlib/gradmvn/__init__.py` — export ordering
+
+### B1. GGE ordering (GAUSS `ordering`, line 2039)
+
+Algorithm: at each step h, compute standardized deviations z_i = (a_i - μ_trunc) / σ_trunc for remaining variables, reorder by ascending z_i (bring most constrained variable forward).
+
+```python
+def gge_ordering(a: NDArray, sigma: NDArray) -> NDArray:
+    """Return permutation indices via Gibson-Geman-Escobar adaptive ordering."""
+```
+
+### B2. TG CDF method (GAUSS `cdfmvnaTG`, line 837)
+
+Sequential univariate conditioning without rank-1 updates — just strips LDLT rows/columns:
+```
+P = ∏ₕ Φ((aₕ - μ̃ₕ) / √Dₕ)
+```
+Simpler than ME. Uses GGE ordering for accuracy.
+
+### B3. TGBME CDF method (GAUSS `cdfmvnaTGBME`, line 1297)
+
+Bivariate conditioning variant: pairs variables, uses `ldltblock(Σ, 2)` and bivariate truncated moments for rank-2 updates. Handles odd K by using univariate for the last variable.
+
+### Verification
+- FD comparison: TG and TGBME should match scipy.stats.multivariate_normal.cdf within ~2% MAPE for K=5
+- Compare GGE ordering against ascending ordering accuracy (GGE should be equal or better)
+- Existing MVNCD tests must still pass
+
+---
+
+## Phase C: Analytic MVNCD Gradients (Core)
+
+This is the highest-impact change. The GAUSS gradient chain for MVNCD:
+```
+pdfmvnaME/OVUS → gcondmeantrunc/gcondcovtrunc → grad_univariate_normal_trunc → gradnoncdfn
+```
+
+### C1. Atomic gradient helpers
+
+**Create:** `src/pybhatlib/gradmvn/_trunc_grads.py`
+**Test:** `tests/test_gradmvn/test_trunc_grads.py`
+
+Functions (all verified against FD):
+
+| Function | GAUSS ref | Purpose |
+|----------|-----------|---------|
+| `grad_univariate_normal_trunc` | line 2120 | ∂(μ_trunc, σ²_trunc)/∂(μ, σ², w) |
+| `grad_bivariate_normal_trunc` | line 2138 | ∂(μ_trunc, Σ_trunc)/∂(μ, Σ, w) for bivariate |
+| `grad_cdf_bvn` | line 2512 | ∂BVN(w₁,w₂,ρ)/∂(w₁,w₂,ρ) |
+| `grad_noncdfn` | line 2489 | ∂Φ((x-μ)/σ)/∂(μ,σ²,x) |
+| `grad_noncdfbvn_by_cdfn` | line 2561 | ∂[BVN/Φ]/∂(μ,Σ,x) for OVUS screening ratio |
+
+### C2. Compound gradient helpers
+
+**Create:** `src/pybhatlib/gradmvn/_cond_trunc_grads.py`
+**Create:** `src/pybhatlib/matgradient/_mat_grad_helpers.py`
+**Test:** `tests/test_gradmvn/test_cond_trunc_grads.py`
+
+| Function | GAUSS ref | Purpose |
+|----------|-----------|---------|
+| `gcondmeantrunc` | line 2224 | Gradient of conditional mean through truncation |
+| `gcondcovtrunc` | line 2389 | Gradient of conditional covariance through truncation |
+| `ginverse` | — | ∂X⁻¹/∂X (matrix inverse gradient) |
+| `gcondcov` | line 2323 | ∂(X₂₂ - X₁₂X₁₁⁻¹X₁₂ᵀ)/∂X |
+
+### C3. ME and OVUS analytic gradients
+
+**Create:** `src/pybhatlib/gradmvn/_mvncd_grad_analytic.py`
+**Modify:** `src/pybhatlib/gradmvn/_mvncd_grad.py` — add `analytic` parameter
+**Test:** `tests/test_gradmvn/test_mvncd_grad_analytic.py`
+
+Core algorithm (ME, from GAUSS `pdfmvnaME` line 981):
+```
+For h = 1..m-1:
+  1. Truncate: {μ̃, ω} = univariate_normal_trunc(...)
+  2. Gradients: {gY, gmu, gX, gC} = gcondmeantrunc(...)
+  3. Cumulative Jacobian: gcumul = gcumul @ [[gmumean|gmucov], [gXmean|gXcov]]
+  4. Update LDLT via rank-1
+  5. Factor: p[h+1] = Φ((a[h+1]-μ̃)/√D)
+  6. Accumulate: grho += (1/p[h+1]) * gcumul @ grad_factor
+```
+
+Returns `(prob, gw, grho)` in standardized space.
+
+OVUS differs: uses bivariate screening (kkk=2), `grad_noncdfbvn_by_cdfn` instead of `grad_noncdfn`, initial BVN instead of univariate Φ.
+
+### C4. Standardization wrapper
+
+Update `mvncd_grad()` to dispatch to analytic when available:
+```python
+def mvncd_grad(a, sigma, *, method="me", analytic=True, xp=None):
+    if analytic and method in ("me", "ovus"):
+        # Standardize: w = a/√diag(Σ), ρ = corr(Σ)
+        # Call _mvncd_grad_me/ovus(w, ρ) → (prob, gw, grho)
+        # Destandardize via gradcorcov: gcov = f(gw, grho)
+        ...
+```
+
+### Verification
+- Every atomic helper: FD verification at multiple parameter regimes (moderate, extreme truncation, high/low ρ)
+- Analytic `gw` vs numerical: `|(gw_analytic - gw_numerical)| / max(|gw|, 1e-8) < 1e-3` for K=3,4,5
+- Analytic `grho` vs numerical: same tolerance
+- Cross-method: ME and OVUS analytic should agree within approximation tolerance
+
+---
+
+## Phase D: MNP Analytic Gradient Chain
+
+**Create:** `src/pybhatlib/models/mnp/_mnp_grad.py`
+**Modify:** `src/pybhatlib/models/mnp/_mnp_loglik.py` — wire in analytic gradient
+**Modify:** `src/pybhatlib/models/mnp/_mnp_control.py` — add `analytic_grad` field
+**Test:** `tests/test_models/test_mnp_grad.py`
+
+### D1. Analytic gradient function
+
+Translates GAUSS `lgd` (line 297). The chain rule:
+```
+∂(-logL)/∂θ = -(1/N) Σ_q (1/P_q) ∂P_q/∂θ
+
+where ∂P_q/∂θ decomposes as:
+  ∂P/∂β  = (∂P/∂B) × (∂B/∂β)         [mean gradient via design matrix]
+  ∂P/∂λ  = (∂P/∂Σ) × gomegxomegax(M) × gradcovcor × gcholspher  [cov gradient]
+  ∂P/∂ω  = (∂P/∂Σ) × gomegxomegax(M) × [scale gradient]
+```
+
+Uses existing `gradcovcor()`, `gomegxomegax()` (already implemented, just not integrated).
+
+### D2. Integration into mnp_loglik
+
+Add `analytic_grad: bool = True` to `MNPControl`. When True and `return_gradient=True`, call `mnp_analytic_gradient()` instead of `_numerical_gradient()`.
+
+### Verification
+- Component FD: For small problem (N=10, I=3), compare analytic `∂nll/∂θ_k` vs FD at `1e-4` tolerance
+- Convergence: BFGS with analytic grads converges to same optimum as numerical
+- Performance: time comparison (expect ~7-14x speedup depending on parameter count)
+
+---
+
+## Phase E: Additional Parameterizations
+
+### E1. Radial parameterization (Van Oest 2019)
+
+**Create:** `src/pybhatlib/matgradient/_radial.py`
+**Test:** `tests/test_matgradient/test_radial.py`
+
+GAUSS `newcholparm` (line 3752): `S = tanh(θ)`, accumulative sine products.
+Include analytic Jacobian from `gnewcholparm` (line 4048).
+
+### E2. Analytic spherical Jacobian
+
+**Modify:** `src/pybhatlib/matgradient/_spherical.py` — replace numerical FD in `grad_corr_theta()`
+**Create:** `src/pybhatlib/matgradient/_cholesky_grad.py` — `gcholeskycov`
+
+GAUSS `gcholspherunconstcorscaled` (line 3871) provides fully analytic Jacobian.
+
+### Verification
+- Radial: `radial_to_corr()` must produce valid PD correlation for any input
+- Radial Jacobian vs FD: `1e-5` tolerance
+- Analytic spherical Jacobian vs current numerical: `1e-5` tolerance
+
+---
+
+## Implementation Order & Dependencies
+
+```
+Phase A (quick wins)          ← independent, do first
+Phase B (TG/TGBME)           ← independent of C/D
+Phase C1 (atomic grads)       ← foundation
+Phase C2 (compound grads)     ← depends on C1
+Phase C3 (MVNCD grads)        ← depends on C1, C2
+Phase C4 (wrapper)            ← depends on C3
+Phase D (MNP grad chain)      ← depends on C4
+Phase E (parameterizations)   ← independent, enhances D
+```
+
+## Expected Performance Impact
+
+For K=4 (I=5 alternatives), P=20 parameters:
+- **Current**: (2×20+1)=41 likelihood evaluations per optimizer iteration
+- **With Phase C+D**: ~3 equivalent evaluations per iteration
+- **Speedup**: ~14x
+
+For larger models (P=50): ~33x speedup.

{pybhatlib-0.2.0 → pybhatlib-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pybhatlib
-Version: 0.2.0
+Version: 0.3.0
 Summary: Python implementation of BHATLIB: matrix-based inference for advanced econometric models
 Project-URL: Homepage, https://github.com/UMN-Choi-Lab/pybhatlib
 Project-URL: Repository, https://github.com/UMN-Choi-Lab/pybhatlib
@@ -96,6 +96,22 @@ results.summary()
 - **Dual backend**: NumPy (default) + optional PyTorch with GPU support
 - **Post-estimation**: Average Treatment Effects (ATE), forecasting
 
+## Verification
+
+pybhatlib reproduces Table 1 from the BHATLIB paper (Bhat 2018) using synthetic
+TRAVELMODE data with 3 travel modes (DA, SR, TR) and 210 observations:
+
+| Model | Specification | Target LL | Achieved LL | Status |
+|-------|--------------|-----------|-------------|--------|
+| (a)(i) | IID errors | -670.956 | -670.956 | Exact match |
+| (a)(ii) | Flexible covariance | -661.111 | -661.111 | Exact match |
+| (b) | + AGE45 demographics | -659.285 | -659.284 | Exact match |
+| (c) | + Random coeff. OVTT | -635.871 | -635.871 | Exact match |
+| (d) | 2-segment mixture | -634.975 | -632.912 | Close (synthetic data) |
+
+Models (a)–(c) match the published results exactly. Model (d) achieves a slightly
+better log-likelihood due to multiple local optima in mixture models with synthetic data.
+
 ## References
 
 1. Bhat, C. R. (2018). New Matrix-Based Methods for the Analytic Evaluation of the

{pybhatlib-0.2.0 → pybhatlib-0.3.0}/README.md

@@ -58,6 +58,22 @@ results.summary()
 - **Dual backend**: NumPy (default) + optional PyTorch with GPU support
 - **Post-estimation**: Average Treatment Effects (ATE), forecasting
 
+## Verification
+
+pybhatlib reproduces Table 1 from the BHATLIB paper (Bhat 2018) using synthetic
+TRAVELMODE data with 3 travel modes (DA, SR, TR) and 210 observations:
+
+| Model | Specification | Target LL | Achieved LL | Status |
+|-------|--------------|-----------|-------------|--------|
+| (a)(i) | IID errors | -670.956 | -670.956 | Exact match |
+| (a)(ii) | Flexible covariance | -661.111 | -661.111 | Exact match |
+| (b) | + AGE45 demographics | -659.285 | -659.284 | Exact match |
+| (c) | + Random coeff. OVTT | -635.871 | -635.871 | Exact match |
+| (d) | 2-segment mixture | -634.975 | -632.912 | Close (synthetic data) |
+
+Models (a)–(c) match the published results exactly. Model (d) achieves a slightly
+better log-likelihood due to multiple local optima in mixture models with synthetic data.
+
 ## References
 
 1. Bhat, C. R. (2018). New Matrix-Based Methods for the Analytic Evaluation of the