pylgm 0.6.1__tar.gz

pylgm-0.6.1/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+.superpowers/
+.worktrees/
+site/
+dist/
+validation/
+.base/
+.base-ast/

pylgm-0.6.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Andrea Panozzo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pylgm-0.6.1/PKG-INFO

@@ -0,0 +1,209 @@
+Metadata-Version: 2.5
+Name: pylgm
+Version: 0.6.1
+Summary: General-purpose latent Gaussian models for Python
+Project-URL: Homepage, https://github.com/Ardea00/pylgm
+Project-URL: Repository, https://github.com/Ardea00/pylgm
+Project-URL: Issues, https://github.com/Ardea00/pylgm/issues
+Author: Andrea Panozzo
+License-Expression: MIT
+License-File: LICENSE
+Keywords: bayesian,inla,laplace approximation,latent gaussian models,statistics
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Requires-Python: >=3.11
+Requires-Dist: formulaic>=1.1
+Requires-Dist: numpy>=2.0
+Requires-Dist: pandas>=2.2
+Requires-Dist: pyarrow>=17
+Requires-Dist: pydantic>=2.8
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: scipy>=1.14
+Requires-Dist: typer>=0.15
+Provides-Extra: dev
+Requires-Dist: pytest-xdist>=3.6; extra == 'dev'
+Requires-Dist: pytest>=8.3; extra == 'dev'
+Requires-Dist: ruff>=0.9; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
+Provides-Extra: spark
+Requires-Dist: pyspark>=3.5; extra == 'spark'
+Description-Content-Type: text/markdown
+
+<h1 align="center">pyLGM</h1>
+
+<p align="center">
+  <strong>General-purpose latent Gaussian models for Python — the model class behind INLA.</strong><br>
+  Deterministic Bayesian inference for structured data. No MCMC, no new dependencies.
+</p>
+
+<p align="center">
+  <a href="https://pypi.org/project/pylgm/"><img alt="PyPI" src="https://img.shields.io/pypi/v/pylgm?color=3b82f6"></a>
+  <a href="https://pypi.org/project/pylgm/"><img alt="Python versions" src="https://img.shields.io/pypi/pyversions/pylgm"></a>
+  <a href="https://github.com/Ardea00/pylgm/actions/workflows/test.yml"><img alt="CI" src="https://github.com/Ardea00/pylgm/actions/workflows/test.yml/badge.svg?branch=main"></a>
+  <a href="https://ardea00.github.io/pylgm/"><img alt="Docs" src="https://img.shields.io/badge/docs-ardea00.github.io%2Fpylgm-blue"></a>
+  <a href="LICENSE"><img alt="License" src="https://img.shields.io/pypi/l/pylgm?color=green"></a>
+  <a href="https://github.com/astral-sh/ruff"><img alt="Ruff" src="https://img.shields.io/badge/lint-ruff-261230"></a>
+</p>
+
+<p align="center">
+  <a href="https://ardea00.github.io/pylgm/">Documentation</a> ·
+  <a href="https://ardea00.github.io/pylgm/how-it-works/">How it works</a> ·
+  <a href="https://ardea00.github.io/pylgm/comparison/">Comparison</a> ·
+  <a href="https://ardea00.github.io/pylgm/examples/">Examples</a> ·
+  <a href="https://ardea00.github.io/pylgm/theory/">Theory</a>
+</p>
+
+---
+
+## What it does
+
+Declare structure — groups, space, time, frequency, networks — and pyLGM
+returns a **posterior for each component**, not just a prediction. Fits run in
+seconds through Laplace approximations and INLA-style integration, and are
+deterministic: same data, same answer, no convergence diagnostics.
+
+<p align="center">
+  <img src="docs/img/scotland_shrinkage.png" alt="Besag smoothing shrinks noisy rates toward local means" width="620">
+</p>
+
+<p align="center"><em>Partial pooling in one picture: noisy small-area rates (left, near zero) are
+pulled toward their neighbours, while well-observed areas keep their signal.
+Marker size is the expected count. From
+<a href="https://ardea00.github.io/pylgm/examples-disease-mapping/">the disease-mapping example</a>.</em></p>
+
+## Install
+
+```bash
+pip install pylgm
+pip install "pylgm[spark]"   # optional: Spark data boundary
+```
+
+Requires Python ≥ 3.11.
+
+## 30-second example
+
+A Poisson model with a per-region random intercept, fit with the Laplace engine:
+
+```python
+import pandas as pd
+from pylgm import Fixed, IID, LGM, Poisson
+
+frame = pd.DataFrame({
+    "region": ["north", "north", "north", "south", "south", "south"],
+    "time":   [1, 2, 3, 1, 2, 3],
+    "x":      [0.0, 0.5, 1.0, 0.0, 0.5, 1.0],
+    "count":  [3, 5, 8, 2, 3, 5],
+})
+
+model = LGM(
+    response="count",
+    likelihood=Poisson(),                                # canonical log link
+    predictor=Fixed("1 + x")                             # fixed effects
+        + IID("region", index="region", precision=2.0),  # random intercept per region
+    panel=("region",),
+    time="time",
+)
+
+result = model.fit(frame, engine="laplace")
+print("fitted_mean:", result.fitted_mean.round(3).tolist())
+# fitted_mean: [3.254, 4.986, 8.15, 2.201, 3.373, 5.513]
+```
+
+The same model can be declared in YAML and loaded with `pylgm.config.load_model`.
+
+## Reproducing a published result
+
+[`examples/columbus_spatial_econometrics`](examples/columbus_spatial_econometrics)
+reproduces the reference result of spatial econometrics — Anselin (1988),
+Table 12.1, 49 Columbus OH neighbourhoods:
+
+| model | const | INC | HOVAL | ρ / λ |
+|---|---|---|---|---|
+| published OLS | 68.619 | −1.5973 | −0.2739 | — |
+| **OLS, recomputed** | **68.619** | **−1.5973** | **−0.2739** | — |
+| published ML spatial error | 60.279 | −0.9573 | −0.3046 | 0.5468 |
+| **pyLGM `SAR`** | **59.543** | **−0.9057** | **−0.3058** | **0.5946** |
+
+OLS matching the published numbers exactly verifies the data and spec; the
+`SAR` fit then lands next to the published ML spatial-error estimates, and
+recovers the finding that ignoring spatial correlation **overstates the income
+effect by 1.76×**.
+
+## A network that changes every year
+
+[`examples/state_income_dynamic_network`](examples/state_income_dynamic_network)
+fits 48 US states over 1997–2007 with **one network per year**, then knocks out
+20% of the panel and restores it:
+
+| method | RMSE ↓ (log income) |
+|---|---|
+| **`DynamicSpatialPanel`** | **0.0246** |
+| state mean | 0.1214 |
+| year mean | 0.1484 |
+
+~5× closer than the obvious baselines. The same example reports where it loses
+— a last-value forecast beats it on level forecasts, because the fitted γ ≈ 1
+says log income is near a random walk.
+
+## Why not just use XGBoost?
+
+Often you should — and the [comparison page](https://ardea00.github.io/pylgm/comparison/)
+says so with measured numbers rather than adjectives. Both problems below are
+simulated, so the *true* surface is known and scored directly:
+
+| Problem | pyLGM | XGBoost | GLM |
+|---|---|---|---|
+| 200 areas, 3 counts each, smooth spatial signal (RMSE ↓) | **0.135** | 0.371 | 0.429 |
+| 4000 rows, nonlinear covariate interactions (RMSE ↓) | 2.839 | **0.293** | — |
+
+pyLGM wins where the per-unit sample is thin but the units are related, and
+returns 95% intervals that cover the truth 99% of the time. Gradient boosting
+wins where the signal is interactions among covariates and you have the rows to
+learn them. Reproduce both with
+[`examples/method_comparison`](examples/method_comparison).
+
+## What's in the box
+
+| Area | What you get | Docs |
+|---|---|---|
+| **Likelihoods** | Gaussian (exact), Poisson, Bernoulli, Binomial, negative-binomial, Gamma, Beta, Weibull/exponential survival | [likelihoods](https://ardea00.github.io/pylgm/likelihoods/) |
+| **Effects** | `Fixed`, `IID`, `RW1`/`RW2`, `AR1` (optionally group-wise), `Seasonal`, `MIDAS`, `MIDASParametric`, `SpaceTime` | [effects](https://ardea00.github.io/pylgm/effects/) |
+| **Spatial** | `Besag` (ICAR), `ProperCAR`, `BYM2`, weighted graphs | [spatial](https://ardea00.github.io/pylgm/spatial-effects/) |
+| **Networks** | directed `SAR`, dynamic `DynamicSpatialPanel` (SDPD) with forward forecasting | [spatial](https://ardea00.github.io/pylgm/spatial-effects/) |
+| **Scale** | sparse solver past the dense guard, with the full uncertainty surface | [internals](https://ardea00.github.io/pylgm/internals/) |
+| **Hyperparameters** | Empirical Bayes (type-II ML), MAP-II with PC priors, bounds | [empirical bayes](https://ardea00.github.io/pylgm/empirical-bayes/) |
+| **Integration** | INLA grid quadrature, simplified/full-Laplace marginals, DIC/WAIC/CPO/PIT | [INLA](https://ardea00.github.io/pylgm/inla/) |
+| **Prediction** | fit-row, out-of-sample `predict`, and forecasting via `NaN` rows | [prediction](https://ardea00.github.io/pylgm/prediction/) |
+| **Constraints** | arbitrary linear constraints `A x = e` (R-INLA `extraconstr`) | [effects](https://ardea00.github.io/pylgm/effects/#linear-constraints-extraconstr) |
+| **Data boundary** | Pandas, or Spark / Databricks | [spark](https://ardea00.github.io/pylgm/spark/) |
+
+## Documentation
+
+Full docs: **https://ardea00.github.io/pylgm/**
+
+- [How it works](https://ardea00.github.io/pylgm/how-it-works/) — one `fit()` call end to end, then prediction and forecasting
+- [Comparison](https://ardea00.github.io/pylgm/comparison/) — against regression, gradient boosting and MCMC, including where pyLGM loses
+- [Theory](https://ardea00.github.io/pylgm/theory/) — the model class and every structured effect, with references
+- [Examples](https://ardea00.github.io/pylgm/examples/) — 21 runnable scripts
+
+## Roadmap
+
+What's shipped and what's next is on the
+[roadmap](https://ardea00.github.io/pylgm/roadmap/); scope and compatibility
+policy on the [internals](https://ardea00.github.io/pylgm/internals/) page.
+
+## Contributing
+
+Issues and pull requests welcome — see
+[docs/development.md](docs/development.md) for the test setup. `main` is
+protected: PRs need the full CI matrix green before merge.
+
+## License
+
+[MIT](LICENSE)

pylgm-0.6.1/README.md

@@ -0,0 +1,172 @@
+<h1 align="center">pyLGM</h1>
+
+<p align="center">
+  <strong>General-purpose latent Gaussian models for Python — the model class behind INLA.</strong><br>
+  Deterministic Bayesian inference for structured data. No MCMC, no new dependencies.
+</p>
+
+<p align="center">
+  <a href="https://pypi.org/project/pylgm/"><img alt="PyPI" src="https://img.shields.io/pypi/v/pylgm?color=3b82f6"></a>
+  <a href="https://pypi.org/project/pylgm/"><img alt="Python versions" src="https://img.shields.io/pypi/pyversions/pylgm"></a>
+  <a href="https://github.com/Ardea00/pylgm/actions/workflows/test.yml"><img alt="CI" src="https://github.com/Ardea00/pylgm/actions/workflows/test.yml/badge.svg?branch=main"></a>
+  <a href="https://ardea00.github.io/pylgm/"><img alt="Docs" src="https://img.shields.io/badge/docs-ardea00.github.io%2Fpylgm-blue"></a>
+  <a href="LICENSE"><img alt="License" src="https://img.shields.io/pypi/l/pylgm?color=green"></a>
+  <a href="https://github.com/astral-sh/ruff"><img alt="Ruff" src="https://img.shields.io/badge/lint-ruff-261230"></a>
+</p>
+
+<p align="center">
+  <a href="https://ardea00.github.io/pylgm/">Documentation</a> ·
+  <a href="https://ardea00.github.io/pylgm/how-it-works/">How it works</a> ·
+  <a href="https://ardea00.github.io/pylgm/comparison/">Comparison</a> ·
+  <a href="https://ardea00.github.io/pylgm/examples/">Examples</a> ·
+  <a href="https://ardea00.github.io/pylgm/theory/">Theory</a>
+</p>
+
+---
+
+## What it does
+
+Declare structure — groups, space, time, frequency, networks — and pyLGM
+returns a **posterior for each component**, not just a prediction. Fits run in
+seconds through Laplace approximations and INLA-style integration, and are
+deterministic: same data, same answer, no convergence diagnostics.
+
+<p align="center">
+  <img src="docs/img/scotland_shrinkage.png" alt="Besag smoothing shrinks noisy rates toward local means" width="620">
+</p>
+
+<p align="center"><em>Partial pooling in one picture: noisy small-area rates (left, near zero) are
+pulled toward their neighbours, while well-observed areas keep their signal.
+Marker size is the expected count. From
+<a href="https://ardea00.github.io/pylgm/examples-disease-mapping/">the disease-mapping example</a>.</em></p>
+
+## Install
+
+```bash
+pip install pylgm
+pip install "pylgm[spark]"   # optional: Spark data boundary
+```
+
+Requires Python ≥ 3.11.
+
+## 30-second example
+
+A Poisson model with a per-region random intercept, fit with the Laplace engine:
+
+```python
+import pandas as pd
+from pylgm import Fixed, IID, LGM, Poisson
+
+frame = pd.DataFrame({
+    "region": ["north", "north", "north", "south", "south", "south"],
+    "time":   [1, 2, 3, 1, 2, 3],
+    "x":      [0.0, 0.5, 1.0, 0.0, 0.5, 1.0],
+    "count":  [3, 5, 8, 2, 3, 5],
+})
+
+model = LGM(
+    response="count",
+    likelihood=Poisson(),                                # canonical log link
+    predictor=Fixed("1 + x")                             # fixed effects
+        + IID("region", index="region", precision=2.0),  # random intercept per region
+    panel=("region",),
+    time="time",
+)
+
+result = model.fit(frame, engine="laplace")
+print("fitted_mean:", result.fitted_mean.round(3).tolist())
+# fitted_mean: [3.254, 4.986, 8.15, 2.201, 3.373, 5.513]
+```
+
+The same model can be declared in YAML and loaded with `pylgm.config.load_model`.
+
+## Reproducing a published result
+
+[`examples/columbus_spatial_econometrics`](examples/columbus_spatial_econometrics)
+reproduces the reference result of spatial econometrics — Anselin (1988),
+Table 12.1, 49 Columbus OH neighbourhoods:
+
+| model | const | INC | HOVAL | ρ / λ |
+|---|---|---|---|---|
+| published OLS | 68.619 | −1.5973 | −0.2739 | — |
+| **OLS, recomputed** | **68.619** | **−1.5973** | **−0.2739** | — |
+| published ML spatial error | 60.279 | −0.9573 | −0.3046 | 0.5468 |
+| **pyLGM `SAR`** | **59.543** | **−0.9057** | **−0.3058** | **0.5946** |
+
+OLS matching the published numbers exactly verifies the data and spec; the
+`SAR` fit then lands next to the published ML spatial-error estimates, and
+recovers the finding that ignoring spatial correlation **overstates the income
+effect by 1.76×**.
+
+## A network that changes every year
+
+[`examples/state_income_dynamic_network`](examples/state_income_dynamic_network)
+fits 48 US states over 1997–2007 with **one network per year**, then knocks out
+20% of the panel and restores it:
+
+| method | RMSE ↓ (log income) |
+|---|---|
+| **`DynamicSpatialPanel`** | **0.0246** |
+| state mean | 0.1214 |
+| year mean | 0.1484 |
+
+~5× closer than the obvious baselines. The same example reports where it loses
+— a last-value forecast beats it on level forecasts, because the fitted γ ≈ 1
+says log income is near a random walk.
+
+## Why not just use XGBoost?
+
+Often you should — and the [comparison page](https://ardea00.github.io/pylgm/comparison/)
+says so with measured numbers rather than adjectives. Both problems below are
+simulated, so the *true* surface is known and scored directly:
+
+| Problem | pyLGM | XGBoost | GLM |
+|---|---|---|---|
+| 200 areas, 3 counts each, smooth spatial signal (RMSE ↓) | **0.135** | 0.371 | 0.429 |
+| 4000 rows, nonlinear covariate interactions (RMSE ↓) | 2.839 | **0.293** | — |
+
+pyLGM wins where the per-unit sample is thin but the units are related, and
+returns 95% intervals that cover the truth 99% of the time. Gradient boosting
+wins where the signal is interactions among covariates and you have the rows to
+learn them. Reproduce both with
+[`examples/method_comparison`](examples/method_comparison).
+
+## What's in the box
+
+| Area | What you get | Docs |
+|---|---|---|
+| **Likelihoods** | Gaussian (exact), Poisson, Bernoulli, Binomial, negative-binomial, Gamma, Beta, Weibull/exponential survival | [likelihoods](https://ardea00.github.io/pylgm/likelihoods/) |
+| **Effects** | `Fixed`, `IID`, `RW1`/`RW2`, `AR1` (optionally group-wise), `Seasonal`, `MIDAS`, `MIDASParametric`, `SpaceTime` | [effects](https://ardea00.github.io/pylgm/effects/) |
+| **Spatial** | `Besag` (ICAR), `ProperCAR`, `BYM2`, weighted graphs | [spatial](https://ardea00.github.io/pylgm/spatial-effects/) |
+| **Networks** | directed `SAR`, dynamic `DynamicSpatialPanel` (SDPD) with forward forecasting | [spatial](https://ardea00.github.io/pylgm/spatial-effects/) |
+| **Scale** | sparse solver past the dense guard, with the full uncertainty surface | [internals](https://ardea00.github.io/pylgm/internals/) |
+| **Hyperparameters** | Empirical Bayes (type-II ML), MAP-II with PC priors, bounds | [empirical bayes](https://ardea00.github.io/pylgm/empirical-bayes/) |
+| **Integration** | INLA grid quadrature, simplified/full-Laplace marginals, DIC/WAIC/CPO/PIT | [INLA](https://ardea00.github.io/pylgm/inla/) |
+| **Prediction** | fit-row, out-of-sample `predict`, and forecasting via `NaN` rows | [prediction](https://ardea00.github.io/pylgm/prediction/) |
+| **Constraints** | arbitrary linear constraints `A x = e` (R-INLA `extraconstr`) | [effects](https://ardea00.github.io/pylgm/effects/#linear-constraints-extraconstr) |
+| **Data boundary** | Pandas, or Spark / Databricks | [spark](https://ardea00.github.io/pylgm/spark/) |
+
+## Documentation
+
+Full docs: **https://ardea00.github.io/pylgm/**
+
+- [How it works](https://ardea00.github.io/pylgm/how-it-works/) — one `fit()` call end to end, then prediction and forecasting
+- [Comparison](https://ardea00.github.io/pylgm/comparison/) — against regression, gradient boosting and MCMC, including where pyLGM loses
+- [Theory](https://ardea00.github.io/pylgm/theory/) — the model class and every structured effect, with references
+- [Examples](https://ardea00.github.io/pylgm/examples/) — 21 runnable scripts
+
+## Roadmap
+
+What's shipped and what's next is on the
+[roadmap](https://ardea00.github.io/pylgm/roadmap/); scope and compatibility
+policy on the [internals](https://ardea00.github.io/pylgm/internals/) page.
+
+## Contributing
+
+Issues and pull requests welcome — see
+[docs/development.md](docs/development.md) for the test setup. `main` is
+protected: PRs need the full CI matrix green before merge.
+
+## License
+
+[MIT](LICENSE)

pylgm-0.6.1/docs/comparison.md

@@ -0,0 +1,214 @@
+# How pyLGM compares
+
+pyLGM is not a general-purpose predictor. It is a **structured-uncertainty**
+method: it assumes the signal decomposes into smooth, spatial, temporal or
+network-shaped pieces, and in exchange it returns calibrated uncertainty for
+each piece. That assumption is what makes it strong in some problems and the
+wrong tool in others.
+
+Every number on this page comes from
+[`examples/method_comparison`](https://github.com/Ardea00/pylgm/tree/main/examples/method_comparison),
+which you can run yourself. Both problems are **simulated**, so the true latent
+surface is known and scored directly — the honest target for a smoothing method,
+rather than in-sample fit to noisy observations.
+
+## The short version
+
+| | classical regression / GLM | gradient boosting (XGBoost) | MCMC | **pyLGM** |
+|---|---|---|---|---|
+| Nonlinear interactions, many features | weak | **excellent** | depends on model | weak |
+| Few observations per group | overfits or needs manual pooling | overfits | good | **excellent** |
+| Spatial / temporal / network structure | manual basis functions | must be learned from data | good | **built in** |
+| Calibrated uncertainty | for the fitted parameters | none natively | **exact in the limit** | **yes, per component** |
+| Extrapolating past the data | linear only | **cannot** — constant beyond the hull | good | **yes**, via the structure |
+| Runtime | seconds | seconds | minutes to hours | **seconds** |
+| Reproducibility | exact | exact given a seed | needs convergence checks | **exact, deterministic** |
+
+## Versus regression and GLMs
+
+A GLM with a dummy per group is the honest baseline, and it fails in a specific
+way: with few observations per group, each dummy is estimated from almost no
+data, so the estimates are noisy and the extreme ones are the most wrong.
+Nothing tells a rare-event area from a genuinely quiet one.
+
+pyLGM's structured effects apply **partial pooling** — each area borrows
+strength from its neighbours, at a rate the data itself chooses through the
+estimated precision \(\tau\). A GLM can only choose between no pooling (dummies)
+and complete pooling (drop the term).
+
+**Problem A: 200 areas, 3 Poisson observations each, smooth spatial signal.**
+RMSE against the *true* latent field:
+
+| method | RMSE ↓ |
+|---|---|
+| **pyLGM (`Besag`)** | **0.1345** |
+| XGBoost | 0.3708 |
+| GLM, area dummies | 0.4288 |
+
+pyLGM is about **3× closer to the truth**, and its 95% credible intervals
+contain the true value **99%** of the time — slightly conservative, which is the
+safe direction to be wrong. Neither baseline produces an interval at all.
+
+This is the regime the method exists for: small-area estimation, disease
+mapping, sparse panels, any setting where the per-unit sample is thin but the
+units are related.
+
+## On real data, against a published result
+
+Simulations are a fair test of a smoothing method but a weak one for
+credibility, so
+[`examples/columbus_spatial_econometrics`](https://github.com/Ardea00/pylgm/tree/main/examples/columbus_spatial_econometrics)
+reproduces the reference result of spatial econometrics: Anselin (1988),
+Table 12.1 — 49 Columbus, OH neighbourhoods, crime on income and housing value.
+
+| model | const | INC | HOVAL | ρ / λ |
+|---|---|---|---|---|
+| published OLS | 68.619 | −1.5973 | −0.2739 | — |
+| **OLS, recomputed here** | **68.619** | **−1.5973** | **−0.2739** | — |
+| published ML spatial error | 60.279 | −0.9573 | −0.3046 | 0.5468 |
+| **pyLGM `SAR`** | **59.543** | **−0.9057** | **−0.3058** | **0.5946** |
+
+The OLS row matching the published values *exactly* is what verifies the data
+and specification. pyLGM's `SAR` then lands next to the published ML
+spatial-error estimates, and the economic conclusion reproduces cleanly:
+ignoring spatial correlation **overstates the income effect by 1.76×**.
+
+pyLGM is not re-implementing ML estimation — it is Bayesian, and carries a
+nugget the ML spatial-error model does not — so exact agreement would be
+suspicious rather than reassuring. The residual gap shows up in ρ (0.595 vs
+0.547).
+
+### Where the structure earns its keep
+
+[`examples/state_income_dynamic_network`](https://github.com/Ardea00/pylgm/tree/main/examples/state_income_dynamic_network)
+fits 48 US states over 1997–2007 with **a different network every year** —
+contiguity weighted by the previous year's income similarity — then knocks 20%
+of the panel cells out and asks each method to restore them:
+
+| method | RMSE ↓ (log income) |
+|---|---|
+| **pyLGM `DynamicSpatialPanel`** | **0.0246** |
+| state mean | 0.1214 |
+| year mean | 0.1484 |
+| grand mean | 0.1786 |
+
+About **5× closer**, because a missing cell is reconstructed from both its own
+history and its neighbours' current values. Panel gaps are the ordinary case,
+not a contrived one.
+
+That example also reports where it **loses**: on a plain level forecast a
+last-value benchmark wins (0.029 vs 0.040). That is not a defect — the fitted
+γ ≈ 1.005 says log income is near a random walk, and for a random walk the last
+observed value *is* the optimal forecast.
+
+## Versus gradient boosting
+
+Boosted trees are excellent at exactly what LGMs are bad at: discovering
+nonlinear interactions among many covariates when you have enough rows to learn
+them.
+
+**Problem B: 4000 rows, response driven by products and thresholds of three
+continuous covariates.** RMSE against the true signal on held-out rows:
+
+| method | RMSE ↓ |
+|---|---|
+| **XGBoost** | **0.2934** |
+| pyLGM, linear predictor | 2.8391 |
+
+XGBoost is roughly **10× better**, and no amount of tuning a linear predictor
+closes that gap — the model class simply does not contain the interaction. If
+your problem looks like this, use gradient boosting.
+
+Three differences matter beyond accuracy:
+
+- **Uncertainty.** XGBoost returns a point prediction. Quantile objectives or
+  conformal wrappers add intervals, but they are not a posterior and they do not
+  decompose by component. pyLGM gives you the posterior for *each term*
+  separately — `latent_marginals("region")` versus `latent_marginals("trend")`.
+- **Extrapolation.** A tree is constant outside the range of its training data,
+  so a boosted model cannot forecast a trend past the last observed period.
+  A `RW1` or `AR1` extrapolates because the *structure* carries it forward; see
+  [prediction](prediction.md).
+- **Structure as an input, not a discovery.** If you know regions are adjacent
+  or periods are ordered, pyLGM takes that as a graph or an index. A tree must
+  rediscover it from data, which costs the data you did not have.
+
+**They compose.** These are not exclusive: boosting the residuals of a fitted
+LGM, or feeding an LGM's smoothed component in as a feature, is often better
+than either alone.
+
+## Versus MCMC and Monte Carlo
+
+This is a comparison of *inference style*, not model class — MCMC can fit
+latent Gaussian models too, and for a long time that was the standard way.
+
+Monte Carlo error falls as \(1/\sqrt{S}\) in the number of draws \(S\). A
+deterministic method has no such error. **Problem C** targets a small Gaussian
+model whose posterior is available in closed form, and compares one exact solve
+against a random-walk Metropolis sampler:
+
+| method | distance from the exact posterior mean ↓ |
+|---|---|
+| **pyLGM (one solve)** | **0.0000** |
+| Metropolis, 1 000 draws | 0.2066 |
+| Metropolis, 10 000 draws | 0.0638 |
+| Metropolis, 100 000 draws | 0.0161 |
+
+Each 10× in draws buys about \(\sqrt{10} \approx 3.2\times\) accuracy — visible
+in the table, and the reason sampling gets expensive when you want another
+digit. For a Gaussian likelihood pyLGM's answer is not an approximation at all;
+for non-Gaussian likelihoods it is a Laplace approximation whose error comes
+from the posterior's shape rather than from a random seed.
+
+What this buys, in practice:
+
+- **No convergence diagnostics.** No burn-in, no thinning, no \(\hat{R}\), no
+  divergences to interpret. A fit either converges or raises.
+- **Determinism.** The same data give bit-identical results, which matters for
+  a production pipeline and for testing.
+- **Speed.** Seconds instead of minutes-to-hours for models of this shape.
+
+What you give up:
+
+- **Model class.** INLA-style inference requires the latent field to be
+  Gaussian and the hyperparameters to be few. Outside that, use MCMC.
+- **Exactness in hard posteriors.** For strongly non-Gaussian or multimodal
+  posteriors, a Laplace approximation can be biased where MCMC, run long
+  enough, is not. pyLGM's simplified- and full-Laplace latent marginals
+  ([INLA integration](inla.md)) exist to narrow that gap, not to close it.
+- **Many hyperparameters.** The INLA grid grows as \((2r+1)^d\) in the number
+  \(d\) of hyperparameters, so it is practical for a handful, not dozens.
+
+## When *not* to reach for pyLGM
+
+Being clear about this is more useful than a feature list:
+
+- The signal is dominated by **interactions between many covariates** → gradient
+  boosting.
+- You have **no structure to exploit** — no groups, no space, no time, no
+  network — and plenty of rows per pattern → almost anything else is simpler.
+- Your latent field is **genuinely non-Gaussian** (heavy-tailed, multimodal,
+  discrete states) → MCMC, or a mixture/state-space tool.
+- You need **dozens of hyperparameters** estimated jointly → the grid does not
+  scale there.
+- You want a **black-box predictor** and do not care how the prediction
+  decomposes → the structure is overhead you are not using.
+
+## Reproducing this page
+
+```bash
+pip install scikit-learn xgboost     # not pyLGM dependencies
+PYTHONPATH=src python examples/method_comparison/run.py
+```
+
+The script prints exactly the tables above; a CI test runs it when those
+packages are installed and skips otherwise, so pyLGM itself keeps no dependency
+on scikit-learn or xgboost.
+
+The two real-data examples need nothing beyond pyLGM, and both are checked in
+CI:
+
+```bash
+PYTHONPATH=src python examples/columbus_spatial_econometrics/run.py
+PYTHONPATH=src python examples/state_income_dynamic_network/run.py   # ~40s
+```

pylgm-0.6.1/docs/development.md

@@ -0,0 +1,30 @@
+# Development installation
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## Running the tests
+
+```bash
+pytest -q
+```
+
+The suite is safe to run in parallel, and CI does:
+
+```bash
+pytest -q -n auto
+```
+
+That roughly halves wall time. A handful of large sparse fits (past the dense
+reference guard) dominate the total, so the parallel floor is whichever of
+those is slowest — running with `-n auto` locally is worth it, but `-x` and
+readable tracebacks are easier serially.
+
+The large-model tests size themselves from
+`pylgm.inference.gaussian._MAX_DENSE_LATENT_DIMENSION` rather than a literal, so
+they stay just past the guard if that threshold ever moves. Keep them that way:
+the sparse cost grows superlinearly, and being *far* past the guard buys no
+extra coverage.
+
+See the [approved design](https://github.com/Ardea00/pylgm/blob/main/docs/design/specs/2026-07-22-pylgm-design.md).