bayesian-pv-census 0.1.0__tar.gz

bayesian_pv_census-0.1.0/.gitignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.egg-info/
+dist/
+build/
+.pytest_cache/
+.venv/
+venv/
+
+# Audited reference values from the French transmission system operator.
+# Redistribution is pending their clearance; see issue #1. Remove this entry,
+# drop the file back in, and bump the minor version once it is granted.
+src/bayesian_pv_census/data/france_2024_reported.csv

bayesian_pv_census-0.1.0/CITATION.cff

@@ -0,0 +1,28 @@
+cff-version: 1.2.0
+title: "bayesian-pv-census: turn a detector's output into a census, then audit a register against it"
+message: "If you use this software, please cite both the software and the paper it implements."
+type: software
+authors:
+  - family-names: Kasmi
+    given-names: Gabriel
+    email: gabkasmi@gmail.com
+    orcid: "https://orcid.org/0000-0002-7774-4302"
+    affiliation: "Mines Paris-PSL"
+version: 0.1.0
+license: MIT
+repository-code: "https://github.com/gabrielkasmi/bayesian-pv-census"
+abstract: >-
+  Reference implementation of the statistical core of a registry-audit protocol:
+  Beta posteriors on detector precision and recall, propagation to a corrected
+  quantity, a credible-interval decision rule against a reported value, a
+  specification battery and its hard core, a closed-form annotation budget, and
+  the cost of forming the correction factor at a coarser spatial scale. Ships
+  the paper's French dataset, 93 reporting units, as a demo.
+references:
+  - type: article
+    title: "Auditing photovoltaic registries with remote sensing"
+    authors:
+      - family-names: Kasmi
+        given-names: Gabriel
+    journal: Joule
+    year: 2026

bayesian_pv_census-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gabriel Kasmi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

bayesian_pv_census-0.1.0/PKG-INFO

@@ -0,0 +1,233 @@
+Metadata-Version: 2.5
+Name: bayesian-pv-census
+Version: 0.1.0
+Summary: Turn a detector's output into a census with credible intervals, then audit a register against it
+Project-URL: Homepage, https://github.com/gabrielkasmi/bayesian-pv-census
+Project-URL: Paper, https://doi.org/10.5281/zenodo.21534856
+Author-email: Gabriel Kasmi <gabkasmi@gmail.com>
+License: MIT
+License-File: LICENSE
+Keywords: audit,bayesian,census,measurement error,photovoltaic,registry,remote sensing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.23
+Requires-Dist: pandas>=1.5
+Requires-Dist: scipy>=1.9
+Provides-Extra: dev
+Requires-Dist: matplotlib>=3.6; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Provides-Extra: figures
+Requires-Dist: matplotlib>=3.6; extra == 'figures'
+Description-Content-Type: text/markdown
+
+# bayesian-pv-census
+
+**Turn a detector's raw count into a census with credible intervals, then audit a register against it.**
+
+A detector that finds objects in imagery misses some and invents others, at rates
+that vary from place to place. Its raw total is therefore not a measurement, and
+comparing it directly to an official register says as much about the detector as
+about the register. This package closes that gap: given a validation sample, it
+turns the raw total into a posterior over the true quantity, and turns that
+posterior into a verdict on whatever the register reports.
+
+Nothing here knows about photovoltaics, France, or geometry. A *unit* is anything
+with a raw total and a validation sample — a department, a grid cell, a utility
+service area. `raw` is whatever you chose to count: installed capacity, number of
+installations, roof area.
+
+This is the statistical core of *Nationally Consistent, Locally Incomplete: A Bayesian Remote-Sensing Audit of Rooftop Photovoltaic Registries* (Kasmi et al., 2026, pending peer review) , extracted as a
+library. Every public function corresponds to a named component of the paper.
+
+## Install
+
+```bash
+pip install bayesian-pv-census            # numpy, scipy, pandas
+pip install 'bayesian-pv-census[figures]' # adds matplotlib
+```
+
+## One minute
+
+```python
+from bayesian_pv_census import UnitRecord, correct_unit, evaluate
+
+unit = UnitRecord(
+    unit_id="dept_86",
+    raw=26_800,                          # what the detector found, any consistent unit
+    precision_tp=116, precision_fp=4,    # 120 detections checked by hand
+    recall_tp=68, recall_fn=33,          # 101 real objects located independently
+    reported_value=26_820,               # what the register reports
+)
+
+result = correct_unit(unit)
+print(f"{result.mean:,.0f}  99% CI {result.ci(0.99)}")
+print(evaluate(result, unit.reported_value).status)   # below | within | above
+```
+
+The verdict is decided by the interval, not the gap. A large discrepancy inside a
+wide interval is not a finding; a small one outside a tight interval is.
+
+## The paper's data
+
+```python
+from bayesian_pv_census import correct_batch, load_demo
+
+units = load_demo()                      # 93 French reporting units, kWp
+correct_batch(units, prior="empirical_bayes")["corrected_mean"].sum()
+# 4_033_003 kWp, the paper's national estimate
+```
+
+`load_demo()` returns the 93 continental French reporting units behind the
+published audit: detected rooftop capacity below 36 kWp, the manual validation
+counts (31,853 annotations), and the size-weighted rates of specification B.
+
+**One column is not yet included.** The reference values under audit are the
+French transmission system operator's grid-connection registry, and
+redistributing them is not ours to grant. Until that clearance arrives the demo
+supports estimation but not the audit, `load_demo()` says so, and
+`has_reported_values()` reports it:
+
+```python
+from bayesian_pv_census import has_reported_values
+has_reported_values()   # False in this release
+```
+
+Nothing is substituted in the meantime. A column of plausible-looking
+placeholders would be indistinguishable from data at a glance, and reproducing a
+published audit against invented references is worse than not reproducing it. The
+four regression tests that need the column are skipped rather than weakened, so
+the skip count in `pytest` is the honest signal. The national estimate above
+needs no reference value and is checked in every release.
+
+Once the column ships, the full battery reproduces the paper: 25 units
+under-reported and 8 over-reported under specification A, an 18-unit hard core
+and a 7-unit negative control.
+
+Installation counts are deliberately absent for a different reason. The audit in
+the paper is about capacity; shipping a count column would imply a quantity that
+was never audited.
+
+## What the correction assumes
+
+The estimator is `raw × P/R`. Four assumptions stand behind it, and the package
+can only speak to two of them.
+
+| | Assumption | Can the package check it? |
+|---|---|---|
+| **H1** | Detection status is independent of object size: true positives, false positives and false negatives have the same mean size. | Partly. The gap between specifications A and B measures the violation, and B does not require H1. |
+| **H2** | The detector is run over the entire unit; no sub-region is excluded. | No. Upstream of anything the package sees. |
+| **H3** | The validation samples are drawn representatively from, respectively, the raw detections and the true population. | No. This is a property of how you annotated, and nothing in the counts reveals it. |
+| **H4** | For a correctly detected object, its estimated size is unbiased for its true size. | Only its sensitivity, via the `scale` key of `run_battery`. |
+
+H4 binds only when the quantity is a size. **If you correct a count of objects
+rather than a capacity, H4 drops out entirely** — which is why the field is named
+`raw` and not `raw_capacity`.
+
+Two of the four are therefore assumptions you carry, not results the package
+delivers. Reporting a credible interval without saying which of H2 and H3 you
+believe, and why, states less than it appears to.
+
+## Before you annotate
+
+The interval's half-width has a closed form, so the annotation effort can be
+budgeted in advance rather than discovered afterwards:
+
+```python
+from bayesian_pv_census import required_sample_size
+
+b = required_sample_size(target_half_width=0.15, expected_precision=0.85,
+                         expected_recall=0.65, level=0.99)
+print(b.n_precision, b.n_recall)   # annotations needed per unit
+```
+
+## Specifications and the hard core
+
+A verdict that only holds under one way of computing precision and recall is not
+a finding. `run_battery` runs the audit under several and keeps what survives all
+of them.
+
+```python
+from bayesian_pv_census import run_battery
+
+battery = run_battery(units, prior="empirical_bayes")
+battery.hard_core("below")       # flagged under-reported by every specification
+battery.negative_control()       # flagged the other way, unanimously — the control group
+battery.concordance_table()      # crosstab; empty off-diagonal corners mean no sign flips
+```
+
+Specification A counts annotated objects. Specification B weights them by size.
+**B is implemented exactly as in the paper**, which computes the weighted rates as
+point estimates and applies them as a deterministic rescaling of A's posterior —
+so B's interval is A's interval, shifted. That is a known limitation of the
+published method; the package reproduces it rather than improving on it, because
+reproducing the paper is the point. See the docstring of `correct_unit` for what
+a properly weighted posterior would require, and for why the `min_weighted_n`
+fallback matters more than its name suggests.
+
+A third axis needs no specification of its own. Rescaling every raw quantity by a
+constant — a different surface-to-power coefficient, a different filtering
+threshold — multiplies the posterior and both its bounds while the reported value
+stays put, so it is passed generically:
+
+```python
+run_battery(units, specs={"A": {}, "B": {"weighted": True},
+                          "C_low": {"scale": 5.5 / 5.0}})
+```
+
+The status is monotone in that constant and flips once, so the flipping point has
+a closed form and no sweep is needed:
+
+```python
+from bayesian_pv_census import conversion_threshold
+conversion_threshold(result, unit.reported_value, "below")
+```
+
+The coefficient itself stays outside the engine. `raw` arrives already converted,
+and a `conversion_coefficient` argument would import photovoltaics into a core
+that knows nothing about it.
+
+## What forming the factor at a coarser scale costs
+
+Correcting each unit and summing is not the same as pooling the units' rates and
+correcting once. `1/R` is convex, so pooling always yields a smaller factor and a
+smaller total. The cost is exactly zero when recall is homogeneous across the
+pooled units, whatever the dispersion of precision, and second order in the
+coefficient of variation of recall otherwise:
+
+```python
+from bayesian_pv_census import compare_aggregation_scales, pooling_penalty
+
+compare_aggregation_scales(units, groups={"north": [...], "south": [...]}).totals
+pooling_penalty(units)   # CV(R)^2 - rho * CV(P) * CV(R), and its two terms
+```
+
+The practical consequence is that there is no optimal grid to search for. Forming
+the factor at the finest level your annotation budget supports is weakly better
+in every case.
+
+## Scope
+
+Out of scope by design, and unlikely to change: geospatial sampling, annotation
+tooling, temporal alignment between imagery and register, and detector-specific
+parsers. Those are format- and project-specific; this package starts once you can
+write down a raw total and a validation sample.
+
+## Tests
+
+```bash
+pip install -e '.[dev]' && pytest
+```
+
+The suite validates the mathematics on synthetic data — the closed form against
+the bootstrap, the monotonicity of the budgeting rule, the structural invariants
+of the hard core, the exact vanishing of the pooling penalty under homogeneous
+recall — and then checks the published French numbers against the shipped demo
+data, so a regression in the engine cannot pass silently.
+
+## Citation
+
+See `CITATION.cff`. Licence MIT.

bayesian_pv_census-0.1.0/README.md

@@ -0,0 +1,207 @@
+# bayesian-pv-census
+
+**Turn a detector's raw count into a census with credible intervals, then audit a register against it.**
+
+A detector that finds objects in imagery misses some and invents others, at rates
+that vary from place to place. Its raw total is therefore not a measurement, and
+comparing it directly to an official register says as much about the detector as
+about the register. This package closes that gap: given a validation sample, it
+turns the raw total into a posterior over the true quantity, and turns that
+posterior into a verdict on whatever the register reports.
+
+Nothing here knows about photovoltaics, France, or geometry. A *unit* is anything
+with a raw total and a validation sample — a department, a grid cell, a utility
+service area. `raw` is whatever you chose to count: installed capacity, number of
+installations, roof area.
+
+This is the statistical core of *Nationally Consistent, Locally Incomplete: A Bayesian Remote-Sensing Audit of Rooftop Photovoltaic Registries* (Kasmi et al., 2026, pending peer review) , extracted as a
+library. Every public function corresponds to a named component of the paper.
+
+## Install
+
+```bash
+pip install bayesian-pv-census            # numpy, scipy, pandas
+pip install 'bayesian-pv-census[figures]' # adds matplotlib
+```
+
+## One minute
+
+```python
+from bayesian_pv_census import UnitRecord, correct_unit, evaluate
+
+unit = UnitRecord(
+    unit_id="dept_86",
+    raw=26_800,                          # what the detector found, any consistent unit
+    precision_tp=116, precision_fp=4,    # 120 detections checked by hand
+    recall_tp=68, recall_fn=33,          # 101 real objects located independently
+    reported_value=26_820,               # what the register reports
+)
+
+result = correct_unit(unit)
+print(f"{result.mean:,.0f}  99% CI {result.ci(0.99)}")
+print(evaluate(result, unit.reported_value).status)   # below | within | above
+```
+
+The verdict is decided by the interval, not the gap. A large discrepancy inside a
+wide interval is not a finding; a small one outside a tight interval is.
+
+## The paper's data
+
+```python
+from bayesian_pv_census import correct_batch, load_demo
+
+units = load_demo()                      # 93 French reporting units, kWp
+correct_batch(units, prior="empirical_bayes")["corrected_mean"].sum()
+# 4_033_003 kWp, the paper's national estimate
+```
+
+`load_demo()` returns the 93 continental French reporting units behind the
+published audit: detected rooftop capacity below 36 kWp, the manual validation
+counts (31,853 annotations), and the size-weighted rates of specification B.
+
+**One column is not yet included.** The reference values under audit are the
+French transmission system operator's grid-connection registry, and
+redistributing them is not ours to grant. Until that clearance arrives the demo
+supports estimation but not the audit, `load_demo()` says so, and
+`has_reported_values()` reports it:
+
+```python
+from bayesian_pv_census import has_reported_values
+has_reported_values()   # False in this release
+```
+
+Nothing is substituted in the meantime. A column of plausible-looking
+placeholders would be indistinguishable from data at a glance, and reproducing a
+published audit against invented references is worse than not reproducing it. The
+four regression tests that need the column are skipped rather than weakened, so
+the skip count in `pytest` is the honest signal. The national estimate above
+needs no reference value and is checked in every release.
+
+Once the column ships, the full battery reproduces the paper: 25 units
+under-reported and 8 over-reported under specification A, an 18-unit hard core
+and a 7-unit negative control.
+
+Installation counts are deliberately absent for a different reason. The audit in
+the paper is about capacity; shipping a count column would imply a quantity that
+was never audited.
+
+## What the correction assumes
+
+The estimator is `raw × P/R`. Four assumptions stand behind it, and the package
+can only speak to two of them.
+
+| | Assumption | Can the package check it? |
+|---|---|---|
+| **H1** | Detection status is independent of object size: true positives, false positives and false negatives have the same mean size. | Partly. The gap between specifications A and B measures the violation, and B does not require H1. |
+| **H2** | The detector is run over the entire unit; no sub-region is excluded. | No. Upstream of anything the package sees. |
+| **H3** | The validation samples are drawn representatively from, respectively, the raw detections and the true population. | No. This is a property of how you annotated, and nothing in the counts reveals it. |
+| **H4** | For a correctly detected object, its estimated size is unbiased for its true size. | Only its sensitivity, via the `scale` key of `run_battery`. |
+
+H4 binds only when the quantity is a size. **If you correct a count of objects
+rather than a capacity, H4 drops out entirely** — which is why the field is named
+`raw` and not `raw_capacity`.
+
+Two of the four are therefore assumptions you carry, not results the package
+delivers. Reporting a credible interval without saying which of H2 and H3 you
+believe, and why, states less than it appears to.
+
+## Before you annotate
+
+The interval's half-width has a closed form, so the annotation effort can be
+budgeted in advance rather than discovered afterwards:
+
+```python
+from bayesian_pv_census import required_sample_size
+
+b = required_sample_size(target_half_width=0.15, expected_precision=0.85,
+                         expected_recall=0.65, level=0.99)
+print(b.n_precision, b.n_recall)   # annotations needed per unit
+```
+
+## Specifications and the hard core
+
+A verdict that only holds under one way of computing precision and recall is not
+a finding. `run_battery` runs the audit under several and keeps what survives all
+of them.
+
+```python
+from bayesian_pv_census import run_battery
+
+battery = run_battery(units, prior="empirical_bayes")
+battery.hard_core("below")       # flagged under-reported by every specification
+battery.negative_control()       # flagged the other way, unanimously — the control group
+battery.concordance_table()      # crosstab; empty off-diagonal corners mean no sign flips
+```
+
+Specification A counts annotated objects. Specification B weights them by size.
+**B is implemented exactly as in the paper**, which computes the weighted rates as
+point estimates and applies them as a deterministic rescaling of A's posterior —
+so B's interval is A's interval, shifted. That is a known limitation of the
+published method; the package reproduces it rather than improving on it, because
+reproducing the paper is the point. See the docstring of `correct_unit` for what
+a properly weighted posterior would require, and for why the `min_weighted_n`
+fallback matters more than its name suggests.
+
+A third axis needs no specification of its own. Rescaling every raw quantity by a
+constant — a different surface-to-power coefficient, a different filtering
+threshold — multiplies the posterior and both its bounds while the reported value
+stays put, so it is passed generically:
+
+```python
+run_battery(units, specs={"A": {}, "B": {"weighted": True},
+                          "C_low": {"scale": 5.5 / 5.0}})
+```
+
+The status is monotone in that constant and flips once, so the flipping point has
+a closed form and no sweep is needed:
+
+```python
+from bayesian_pv_census import conversion_threshold
+conversion_threshold(result, unit.reported_value, "below")
+```
+
+The coefficient itself stays outside the engine. `raw` arrives already converted,
+and a `conversion_coefficient` argument would import photovoltaics into a core
+that knows nothing about it.
+
+## What forming the factor at a coarser scale costs
+
+Correcting each unit and summing is not the same as pooling the units' rates and
+correcting once. `1/R` is convex, so pooling always yields a smaller factor and a
+smaller total. The cost is exactly zero when recall is homogeneous across the
+pooled units, whatever the dispersion of precision, and second order in the
+coefficient of variation of recall otherwise:
+
+```python
+from bayesian_pv_census import compare_aggregation_scales, pooling_penalty
+
+compare_aggregation_scales(units, groups={"north": [...], "south": [...]}).totals
+pooling_penalty(units)   # CV(R)^2 - rho * CV(P) * CV(R), and its two terms
+```
+
+The practical consequence is that there is no optimal grid to search for. Forming
+the factor at the finest level your annotation budget supports is weakly better
+in every case.
+
+## Scope
+
+Out of scope by design, and unlikely to change: geospatial sampling, annotation
+tooling, temporal alignment between imagery and register, and detector-specific
+parsers. Those are format- and project-specific; this package starts once you can
+write down a raw total and a validation sample.
+
+## Tests
+
+```bash
+pip install -e '.[dev]' && pytest
+```
+
+The suite validates the mathematics on synthetic data — the closed form against
+the bootstrap, the monotonicity of the budgeting rule, the structural invariants
+of the hard core, the exact vanishing of the pooling penalty under homogeneous
+recall — and then checks the published French numbers against the shipped demo
+data, so a regression in the engine cannot pass silently.
+
+## Citation
+
+See `CITATION.cff`. Licence MIT.