policyengine 6.4.2__tar.gz → 6.5.0__tar.gz

{policyengine-6.4.2 → policyengine-6.5.0}/.github/workflows/pr_code_changes.yaml

@@ -141,9 +141,10 @@ jobs:
         run: policyengine bundle verify --country us --country uk --packages-only --json
   Test:
     runs-on: ubuntu-latest
-    # Slowest observed 2026-10-06..08: 28 minutes. Fail a hung job instead of
-    # holding a shared org runner for the 6-hour default.
-    timeout-minutes: 60
+    # Observed 2026-10-14..15: 28 to 54 minutes per Python version, with runs on
+    # 3.12 and 3.13 cancelled at the old 60-minute limit. Fail a hung job instead
+    # of holding a shared org runner for the 6-hour default.
+    timeout-minutes: 90
     strategy:
       fail-fast: false
       matrix:

{policyengine-6.4.2 → policyengine-6.5.0}/CHANGELOG.md

@@ -1,3 +1,32 @@
+## [6.5.0] - 2026-10-15
+
+### Added
+
+- Updated the UK Budget reform contract to version 1.1.0. Its rate-blending case, `mid_year_blended_rate`, now holds capital gains realisations fixed (both elasticity conventions zero and no separate Business Asset Disposal Relief elasticity), so its static expected values hold whatever realisation response policyengine-uk applies by default. Added `cgt_retention_response`, which checks the response itself: with a retention-rate elasticity of 1, a 20% basic rate from 6 April 2027 scales realised gains by (1 - 20%) / (1 - 18%) on every route. Added `cgt_default_response`, the same reform with no response settings, so the report shows the default response policyengine-uk applies; its checks expect the retention-rate elasticity of 1 that policyengine-uk#2396 makes the default, and report as pending (without gating) until policyengine.py pins that release. Each case also checks that all three response settings reached the model. `capital_gains_response()` builds a case's elasticity changes.
+- Added a record of the capital gains realisation response to UK runs (`policyengine.tax_benefit_models.uk.capital_gains_responses`). Each population run records the policyengine-uk release and where it was installed from when that is not a package index, the caller's explicit `gov.simulation.capital_gains_responses` settings in the order they apply and, where the installed policyengine-uk reports it, the response it resolved after every reform and modifier. The record is kept with the output dataset (`metadata`), in the release bundle, the run record and the saved output file; household results carry it under `provenance`, and managed microsimulations under `policyengine_bundle["capital_gains_responses"]` for each year. UK cache and saved-output identity now includes a digest of the explicit settings, and a saved UK output is reused only under the same settings and the same policyengine-uk release from the same source (never from a local directory install). Each saved UK output is therefore calculated once more after upgrading, so a job that relies on a warm UK cache takes as long as a cold run the first time. Added `StaleOutputError`, a `ValueError` that `load()` raises for a saved output it refuses (UK and US alike); `Simulation.ensure()` logs the reason at INFO and runs the simulation again, instead of logging a warning with a traceback.
+- Add execution receipt schema v1 (`policyengine.provenance.receipt`), which records exactly what produced a UK result. It records the installed wrapper, model and core versions, with wheel hashes only when the builder verified them against the wheel RECORD, plus the data release and the SHA-256 of the simulated dataset bytes. The certificate digest is attached only when the certificate covers that model, core, dataset and wrapper. The receipt also carries reform and baseline digests, which refuse modifiers they cannot bind, and the fiscal-year convention. Digests use RFC 8785 canonical JSON over strict JSON values (`policyengine.provenance.canonical`), and the JSON Schema ships at `policyengine/data/schemas/execution_receipt.v1.schema.json`.
+- Added `composed_reform_identity` and the `policyengine-uk.scenario_composition/v1` representation, so an execution receipt can bind several reforms applied in one run, in order.
+- Execution receipt reform digests accept infinite parameter values (an uncapped threshold is `.inf`), encoding `inf` and `-inf` as `{"$nonfinite": "Infinity"}` and `{"$nonfinite": "-Infinity"}`. A payload that uses the key `$nonfinite` itself is refused, and finite payloads digest as before.
+- Budget reform contract case rows now carry `reform_identity`, the reform as a receipt binds it (`ReformIdentity`, named by the case id), for the representation each route consumes, or `null` with `reform_identity_unavailable` where receipt schema v1 cannot bind it; property, population and run-level rows carry `null` with the reason.
+
+### Changed
+
+- The pull-request Test job may run for 90 minutes instead of 60. It now takes 30 to 54 minutes per Python version, and runs on 3.12 and 3.13 were cancelled at the 60-minute limit.
+
+### Fixed
+
+- Receipts no longer refuse a process started within a second of an install: on Linux, psutil's process start time can read about a second early, so install metadata written just before the start looked newer than the process. Metadata written more than two seconds after the reported start is still refused. An install within two seconds after the reported start is therefore not detected.
+- Re-locked `uv.lock` with the pinned uv 0.12.14. Two dependency markers written by another uv version made the release's version bump rewrite the lock, which the reviewed-lock check rejects, so no release has published since #571.
+- The UK Budget reform contract's shared-parameter check now catches list-valued parameters changed in place and changes to node or scale metadata or a scale's band count.
+
+
+## [6.4.3] - 2026-10-15
+
+### Changed
+
+- The UK Budget reform contract's Universal Credit allowance check is tested on both runs: each must hold finite values and keep every payment at or above the payable floor, to within a penny. Beyond the payable floor, the check does not bound how far deductions rise: policyengine-uk scales every deduction type with the standard allowance, while the law holds some, such as advance recovery, fixed (policyengine-uk#2410).
+
+
 ## [6.4.2] - 2026-10-15
 
 ### Fixed

{policyengine-6.4.2/src/policyengine.egg-info → policyengine-6.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: policyengine
-Version: 6.4.2
+Version: 6.5.0
 Summary: A package to conduct policy analysis using PolicyEngine tax-benefit models.
 Author-email: PolicyEngine <hello@policyengine.org>
 License:                     GNU AFFERO GENERAL PUBLIC LICENSE
@@ -672,7 +672,7 @@ Classifier: Programming Language :: Python :: 3.14
 Requires-Python: <3.15,>=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pydantic>=2.10.0
 Requires-Dist: pandas>=2.0.0
 Requires-Dist: h5py>=3.0.0
 Requires-Dist: microdf_python<1.4,>=1.3.0
@@ -711,6 +711,7 @@ Requires-Dist: towncrier>=24.8.0; extra == "dev"
 Requires-Dist: mypy>=1.11.0; extra == "dev"
 Requires-Dist: pytest-cov>=5.0.0; extra == "dev"
 Requires-Dist: hypothesis>=6.100.0; extra == "dev"
+Requires-Dist: rfc8785>=0.1.4; extra == "dev"
 Requires-Dist: policyengine-core==3.33.0; extra == "dev"
 Requires-Dist: policyengine-us==2.2.1; extra == "dev"
 Requires-Dist: policyengine-uk==2.130.1; extra == "dev"

{policyengine-6.4.2 → policyengine-6.5.0}/docs/_quarto.yml

@@ -55,6 +55,7 @@ website:
           - reference/index.md
           - countries.md
           - release-bundles.md
+          - receipts.md
           - data-publishing-design.md
           - examples.md
           - dev.md

{policyengine-6.4.2 → policyengine-6.5.0}/docs/dev.md

@@ -106,6 +106,6 @@ src/policyengine/
 
 **Country-specific model classes**: `PolicyEngineUSLatest` and `PolicyEngineUKLatest` inherit from a shared `MicrosimulationModelVersion` base (variable/parameter loading, manifest certification, `save`/`load`). Each subclass only implements `run()` and a handful of country hooks (`_load_system`, `_load_region_registry`, `_dataset_class`, `_get_runtime_data_build_metadata`). The US `run` applies reforms as a dict at `Microsimulation(reform=...)` construction time; the UK `run` wraps inputs as `UKSingleYearDataset` and applies reforms via a modifier after construction.
 
-**LRU cache + file caching**: `Simulation.ensure()` checks an in-process LRU cache (max 100 entries), then tries loading from disk, then falls back to `run()` + `save()`.
+**LRU cache + file caching**: `Simulation.ensure()` checks an in-process LRU cache (max 100 entries), then tries loading from disk, then falls back to `run()` + `save()`. When `load()` refuses an out-of-date saved output it raises `StaleOutputError` (a `ValueError`), which `ensure()` logs at INFO as an expected rerun. Any other load error is logged as a warning.
 
 **Output pattern**: All output types inherit from `Output`, implement `.run()`, and populate result fields. Convenience functions (e.g., `calculate_us_poverty_rates()`) create, run, and return collections of output objects.

{policyengine-6.4.2 → policyengine-6.5.0}/docs/households.md

@@ -165,6 +165,8 @@ For UK households, a `YYYY-MM-DD` key is effective from that date until the next
 
 UK households reject bare years such as `"2027"` or `2027`, because the underlying packages assign different meanings to them. Use `"2027-04-06.2028-04-05"` for one UK fiscal year or `"2027-01-01"` for an effective date. See [Reforms](reforms.md) for the shared `Simulation(policy=...)` and `Simulation(dynamic=...)` date contract. The US household route retains its country package's period-key handling.
 
+UK household results record the capital gains realisation response the calculation applied, under `result["provenance"]["capital_gains_responses"]`: the policyengine-uk release, the reform's explicit `gov.simulation.capital_gains_responses` settings, and the assumption policyengine-uk resolved (see [The UK capital gains realisation response](microsim.md#the-uk-capital-gains-realisation-response)). `result.write(...)` includes it.
+
 Structural reforms (new variables, formula swaps) require the `Simulation` path — see [Reforms](reforms.md).
 
 ## Year

{policyengine-6.4.2 → policyengine-6.5.0}/docs/microsim.md

@@ -30,7 +30,7 @@ total_snap.run()
 total_snap.result
 ```
 
-`Simulation.ensure()` loads a cached result if one exists, or runs and caches on miss. Call `Simulation.run()` explicitly if you want to bypass the cache.
+`Simulation.ensure()` loads a cached result if one exists, or runs and caches on miss. Call `Simulation.run()` explicitly if you want to bypass the cache. A UK simulation's explicit capital gains response settings are part of its cache and saved-output identity; see [The UK capital gains realisation response](#the-uk-capital-gains-realisation-response).
 
 ### Canonical US SPM settings and provenance
 
@@ -288,6 +288,51 @@ checked. A saved UK simulation output records the data year its run anchored
 on; `Simulation.load()` refuses an output saved without one, and
 `Simulation.ensure()` runs it again.
 
+### The UK capital gains realisation response
+
+policyengine-uk scales realised capital gains in response to a reform, at an
+elasticity set under `gov.simulation.capital_gains_responses`. Its default is
+the model's, so it can change between policyengine-uk releases. Each UK run
+records the response it applied, read after every reform and modifier.
+
+- **Where it is kept.** It is in `simulation.release_bundle["capital_gains_responses"]`,
+  the run record's `results.json` and the saved output file.
+- **What it holds.**
+  - `model`: the policyengine-uk release. Its `source` is `null` for a
+    release from a package index; otherwise it says where the install came
+    from: a version control commit, an archive's SHA-256, or a local
+    directory and whether it is editable.
+  - `request`: the caller's explicit settings under that path, in the order
+    they apply (policy, then dynamic), and their digest.
+  - `resolved`: the assumption the release reports. That is the convention,
+    the elasticities for main-rate gains and for gains qualifying for Business
+    Asset Disposal Relief, the MTR elasticity, the separate relief switch, and
+    whether each is the default, an override or set to zero for an MTR
+    elasticity. It is `null` with a release that doesn't report it.
+- **Modifiers.** Changes a simulation modifier makes aren't listed under
+  `request`, as run records don't describe modifiers. `resolved` reflects them.
+  Neither the `storage_id` nor `Simulation.load()` sees them, so two
+  simulations with the same `id` whose modifiers set the response differently
+  share a saved output. Give such simulations different ids.
+- **Identity.** A simulation with explicit settings adds `-cgr-<digest>` to its
+  `storage_id`, so its cache entry and saved output are kept apart from runs
+  with other settings. The digest covers each setting as given, including
+  whether it came from the policy or the dynamic and its dates. So the same
+  value set another way, or with an explicit end date instead of none, is
+  saved separately.
+- **Reuse.** `Simulation.load()` reuses a saved UK output only when its record
+  shows the same policyengine-uk release, installed from the same source, and
+  the same settings. A branch installed from git or a local directory can
+  carry the version number of a published release with another default.
+  With policyengine-uk installed from a local directory, editable or not, no
+  saved UK output is reused, as its code can change without its version
+  changing.
+- **Reruns.** Otherwise `load()` raises `StaleOutputError`, and
+  `Simulation.ensure()` runs the simulation again and logs the reason at INFO.
+  Outputs saved before outputs recorded the response are calculated once
+  more, so a job that relies on a warm UK cache takes as long as a cold run
+  the first time it runs after the upgrade.
+
 ## Simulations
 
 A `Simulation` needs a dataset, a tax-benefit model version, and optionally a policy (reform):
@@ -379,6 +424,14 @@ Pass `allow_unmanaged=True` with a custom `dataset=` to opt out of the release
 bundle. Explicit local paths and Hugging Face URIs remain supported in this
 mode. GCS dataset URIs are not supported.
 
+A UK managed microsimulation also records, in
+`sim.policyengine_bundle["capital_gains_responses"]`, the capital gains
+realisation response it applies in each year of its dataset (see
+[The UK capital gains realisation response](#the-uk-capital-gains-realisation-response)).
+Its `reform=` goes to policyengine-uk unchanged, so no request is listed.
+Settings that conflict are recorded as an error for that year, and raise when
+the response is calculated.
+
 For managed simulations, `sim.policyengine_bundle` records the actual source
 package, repository type, revision, verified SHA-256, and local path.
 

policyengine-6.5.0/docs/receipts.md

@@ -0,0 +1,185 @@
+# Execution receipts
+
+An execution receipt travels with a published number, such as a dashboard row, a briefing figure or an API response. It names everything that determined that number, so a reader can tell exactly which model, data and certificate produced it, and two numbers can be compared knowing whether they came from the same run.
+
+Schema v1 covers UK results and is defined by `policyengine.provenance.receipt.ExecutionReceipt`. The JSON Schema ships in the wheel at `policyengine/data/schemas/execution_receipt.v1.schema.json`, and the same file is in the repository at [`src/policyengine/data/schemas/execution_receipt.v1.schema.json`](https://github.com/PolicyEngine/policyengine.py/blob/main/src/policyengine/data/schemas/execution_receipt.v1.schema.json).
+
+## What a receipt records
+
+| Field | Contents |
+|---|---|
+| `wrapper`, `model`, `core` | The `policyengine`, `policyengine-uk` and `policyengine-core` versions that actually ran, read from the installed distributions rather than from the bundle manifest. `install_origin` records a PEP 610 direct URL as `archive`, `directory`, `editable` or `vcs`; it is absent for an ordinary index install with no `direct_url.json`. `wheel_sha256` appears only when the receipt builder verified it. One way is `wheel_paths=` (evidence `installed_record_verified`): the installed files, and any bytecode cache the interpreter would load instead of a source file, are checked against that wheel's RECORD. The other is the installer's recorded archive hash (`installer_archive_hash`), taken only when the install came from a `.whl` URL; a source archive's hash is never recorded as a wheel hash. |
+| `data` | The data release and build, and the model version the data was built with (`build_model_version`, distinct from `model.version`). Also the dataset's logical name, URI, revision and the SHA-256 of the bytes that were simulated. The dataset is resolved exactly as `materialize_dataset` resolves it (overlays, per-dataset repository, package and revision, and the release manifest revision when an artifact has none of its own), and the bytes must be the ones the bundle pins for it: a receipt never gives an artifact's name to other bytes. |
+| `certification_status`, `certificate` | `certified` only when the bundle certifies all of: the installed model and core versions, that dataset name and those bytes, and the running `policyengine` version. Each package must also have an absent or `archive` install origin, or a `directory` or `vcs` origin with `installed_record_verified` wheel evidence. A directory or VCS install without that evidence produces an `uncertified` receipt, which carries no certificate. An editable install is never certified, whatever the evidence: its wheel holds a pointer to a source tree (a `.pth` file or an import hook) rather than the source, so checking the installed files against it does not bind the code that ran. `certificate_source` records whether the builder read the certificate from the installed bundle (`packaged`) or from a manifest the caller passed (`caller_supplied`). It is the producer's statement, not evidence: trust a certificate only after `verify_certificate` (below). With the packaged bundle, the composition fingerprint of its TRACE TRO is recorded as well. |
+| `reform`, `baseline` | Both are required. Each is `current_law` or a digest of the reform bound to its representation (see below). |
+| `fiscal_year` | What a year label means: year 2027 is the fiscal year from 6 April 2027 to 5 April 2028, shown as `2027-28`. v1 allows only this convention. |
+| `years` | The reported years, sorted and unique. |
+| `supported_years` | The years the installed model projects the dataset to, from the dataset's own year (policyengine-uk's `extend_single_year_dataset`, without running a formula; the same rule as the qualification `supported_years` gate). A reported year outside them is unsupported: policyengine-core fills it by carrying or uprating the latest inputs, so its results are not evidence. `unsupported_years` lists them. The certified 1.58.0 dataset projects to 2024–2030, so a 2031-32 figure on it is unsupported. |
+| `outputs_sha256` | Optional digest of the output payload the receipt binds. |
+| `runtime` | Python version, platform and CI run URL. It is for diagnosis and is part of no digest. |
+
+Malformed `direct_url.json` is refused because the builder cannot determine the package's install origin.
+
+For each loaded top-level package (`policyengine`, `policyengine_uk` and `policyengine_core`), the builder also checks that its `__init__.py` comes from the installed distribution it names. Index, archive, VCS and non-editable directory installs must load the package file listed in that distribution's RECORD, resolved through `locate_file`. Only editable installs (`dir_info.editable: true`) must load from under the directory their PEP 610 URL names, including a `src/` layout. Paths are compared after resolving symlinks. A checkout ahead of a copied install on `PYTHONPATH` is refused because the receipt cannot name the code that ran. An unloaded package has no source path to compare; the version checks still apply.
+
+## Building one
+
+```python
+from policyengine.provenance import materialize_dataset
+from policyengine.provenance.receipt import build_execution_receipt, reform_identity
+
+source = materialize_dataset("uk")  # the certified release artifact, hash-verified
+reform = {"gov.hmrc.income_tax.rates.uk[0].rate": {"2027-04-06": 0.21}}
+
+receipt = build_execution_receipt(
+    country_id="uk",
+    dataset_path=source.path,
+    years=[2027, 2028],
+    reform=reform_identity(
+        reform, "policyengine.reform_dict/v1", name="basic rate 21p"
+    ),
+    baseline=reform_identity(None),
+    outputs={"budgetary_impact": {"2027": 6.1e9, "2028": 6.4e9}},
+)
+print(receipt.certification_status, receipt.execution_identity_sha256)
+receipt_json = receipt.to_canonical_json()
+```
+
+For a country `Microsimulation` built with `managed_microsimulation()`, call `build_execution_receipt_for_microsimulation(sim, years=..., reform=..., baseline=...)`. It reads the dataset that was loaded from `sim.policyengine_bundle`. The file must still hash to the SHA-256 verified when it was loaded, and the receipt must name the dataset, URI, revision and package recorded then; otherwise it is refused, because it would describe bytes that did not run. A simulation loaded from an unmanaged file or URI recorded no load-time hash, so it cannot be receipted. The microsimulation does not record its reform, so pass the reform you built it with.
+
+There are two ways to bind a policyengine-uk `Scenario`:
+- A scenario built from `parameter_changes` is bound by content with `scenario_reform_identity(scenario)`.
+- `Scenario.from_reform(d)` wraps `d` in a closure that cannot be bound by its code, so bind `d` itself with `reform_identity(d, "policyengine-uk.reform_dict/v1")`.
+
+A scenario with a module-level `simulation_modifier` needs `code_revision={"repository": ..., "git_sha": <full commit sha>}`. The identity then says `binding: "code_revision"` and binds:
+- the function's module and qualified name;
+- the SHA-256 of its source and of its module's source file;
+- the default arguments of every function in that module, encoded with their types and floats by their bits (`float.hex()`, which writes infinities as `inf` and `-inf`; NaN is refused), so `1` and `1.0`, `0.0` and `-0.0`, a tuple and a list, or `None` and a missing key never share a digest (defaults with no exact encoding, such as objects, are refused, and so are two defaults that are one mutable object, such as one list set as both `first` and `second`, since the encoding cannot tell that from two equal lists);
+- that revision.
+
+None of these depends on where the code is checked out or which Python version runs it, so CI, a laptop and the server compute the same digest for the same code.
+
+Build the receipt after the run. An identity returned by `scenario_reform_identity` keeps a private handle to the same scenario and code revision. Both receipt builders recompute `reform` and `baseline` identities with handles, and refuse if recomputing is refused or an identity member differs. The handle is excluded from equality, serialization, schemas and digests. A serialized identity loaded back has no handle and is not re-checked; its producer must ensure it describes the run. These checks take snapshots at identity time and receipt build time, so they cannot describe later changes.
+
+**The modifier's module.** A digest can only name what the module's source fixes, so the module holding a bound modifier must have a shape whose state that source determines. At module level it may hold only:
+- docstrings and `pass`;
+- imports;
+- assignments of a literal to one name, annotated or not (`RATE = 0.21`, `YEARS: tuple = (2026, 2027)`);
+- undecorated `def` statements, none named like a dunder, whose type parameters (`def pennies[T](amounts: T)`) have no bound, constraints or default;
+- an `if __name__ == "__main__":` block with no `else` (it never runs on import, so what it binds is ignored).
+
+Each name is bound once. Defaults hold no `:=`, lambda, comprehension, `await` or `yield`. Annotations allow names, attributes, subscripts, constants, directly signed numeric constants and `X | Y`, with no calls. Anything else is refused, naming the line. That covers:
+- a type parameter's bound, constraints or default (`def pennies[T: list[Decimal | float]]`), which Python evaluates only when something such as Pydantic first reads it, so typing's order-insensitive alias cache can decide what it holds;
+- conditional, repeated or unpacked bindings;
+- objects built at import (`PAYMENT = partial(pay, amount=120)`, whose captured values can change without the source changing);
+- values computed from the environment, a file or the clock;
+- decorators, class definitions, and code run at import.
+
+The accepted module shape also refuses these constructs:
+- arithmetic on literals, such as `WEEKLY = 520 / 52` or `MILLION = 10 ** 6`: use the computed literal (`WEEKLY = 10.0`, `MILLION = 1000000`), or compute the value inside a function;
+- `if TYPE_CHECKING:` blocks: put type-only imports in a `.pyi` file or inside a function;
+- `type` aliases, such as `type Rate = float`: use the concrete type in annotations, or put the alias in a `.pyi` file;
+- starred annotations such as `*args: *Ts`: use a concrete argument annotation or `ParamSpec`'s `P.args` and `P.kwargs`.
+
+Anywhere in the module, code may not:
+- tell equal values apart by identity: `is` is allowed only against `None`, `True`, `False`, `...` or `type(...)`, and `type` may not be rebound. The compiler may make equal immutable constants one object, and a digest of values cannot tell one object from two equal ones.
+- use `id`, `operator.is_` or `is_not`, `globals`, `vars`, `locals`, `exec`, `eval`, `compile`, `__import__` or `breakpoint`, however imported, or import `builtins`;
+- read annotation values through `typing.get_type_hints`, `inspect.signature`, `inspect.getfullargspec`, `get_annotations` or `from_callable`; these explicit metadata readers remain outside the accepted module shape.
+- declare a name `global`.
+- read dunder attributes other than `__init__` and `__name__`, or dunder names other than `__name__` and `__file__`. Reading a function's or module's own metadata is introspection.
+- use `getattr`, `hasattr`, `setattr` or `delattr` with a non-literal attribute name, or a literal dunder other than `__init__` and `__name__`.
+
+Put measure values in literal constants or in default arguments.
+
+**What was imported.** A digest must only name code that ran, so at identity time, and again when a receipt is built with that identity's private handle, the module's source as read now is compiled and checked against what was imported, ignoring file names and line numbers:
+- **Versions.** At receipt build time, any recorded version of policyengine, policyengine-uk or policyengine-core must still match its installed version. Independently, each package's distribution metadata must not have been written after this process started. The builder compares the latest modification or change time of its `.dist-info` directory and of the existing `METADATA`, `RECORD`, `INSTALLER` and `direct_url.json` files against `psutil.Process().create_time()`, and refuses metadata written more than two seconds after it. On Linux psutil builds that start time from the boot time in whole seconds plus the start offset in clock ticks, so it can read up to about a second early (one second plus one tick). Without the margin, a process started just after an install finished would be refused. The margin is a blind window: metadata written within two seconds after the reported start is accepted, so an install that lands in that window is not detected. Past it, metadata is refused regardless of import order or whether a version was recorded; an unavailable metadata location is refused too. Restart the process and run again. This also refuses installing a package and then importing it in the same process, such as a notebook `pip install` without a restart, even when that import loaded the new code, unless the install finishes within two seconds of the process starting.
+- **Functions.** The source is compiled as a whole, so each function compiles in its module's context, imports included. The modifier and every other function defined in that file must use the module's own globals and match that code, nested code and constants included. Constants are compared by exact type, by IEEE 754 bytes (so a NaN's sign counts) and by how they share objects. Annotation functions that Python 3.14 compiles (`__annotate__`) are the compiler's, not the module's, and a generic function (`def helper[T](...)`, Python 3.12+) is checked through the wrapper the compiler makes to bind its type parameters.
+- **Function names.** A `def` name must still be bound to that function, not to another function of the same file or any other object.
+- **Code inside functions.** It runs after import, so a type it builds is made then: an expression in a function body, or the annotations, type-parameter bounds and class annotations of a helper defined there (quoted or not). typing caches its generic aliases (`List`, `Dict`, `Optional`, `Annotated`) and user generics (`Generic` subclasses, Pydantic models) by argument equality, and a union equals one holding its members in any order. So `List[Union[Decimal, float]]` built there can return an alias cached for `List[Union[float, Decimal]]`, and Pydantic tries a union's members in order. A function whose code subscripts such a generic with a union or `Literal` is refused. Builtin generics, `collections.abc` classes and classes whose `__class_getitem__` is implemented in C (NumPy's `ndarray` and `dtype`) build a new alias in source order and are allowed (`list[Decimal | float]`), as is a bare `Union[Decimal, float]`. Resolution is conservative. A name stands for every object the module, or any import anywhere inside the function, binds it to, so a helper importing `typing.Sequence` is caught even when a sibling imports `collections.abc.Sequence`. Every string in the function's code that parses as an expression is inspected as a type, since a forward reference such as `Optional['List[Union[Decimal, float]]']` is evaluated when something resolves it. `Literal` values and `Annotated` metadata are left as strings. Local aliases of typing objects (`L = List`) are not followed. The module's own function signatures are compared against their source instead (below).
+- **Function metadata.** Each function's name, qualified name and docstring must still be what its `def` made, compared as the compiler stores them. A generic function's `__type_params__` must still be bare type parameters of the names and kinds its source declares, in order, with no bound, constraints, default or declared variance. Each annotation must still hold what its source names, resolved against the module's globals and builtins (and those type parameters): named objects are compared by identity; reconstructed generics, unions and `ParamSpec` argument proxies must have the same origin and arguments, recursively and in order. Source arguments are resolved separately so typing's cache cannot hide a reordered union. Parameterised typing aliases are refused when their source order cannot be checked independently of that cache. Subscripted annotations that do not expose a typing origin and arguments, such as generated Pydantic model classes, are also refused. Quoted annotations and strings stored by `from __future__ import annotations` must match their source, and their resolvable type expressions must also match source order. This includes annotations consumed implicitly by libraries such as `functools.singledispatch`. It must use the builtins module and carry no other attributes (`helper.amount = 240` is state).
+- **Defaults.** Literal default arguments in the source, such as `rate=0.21`, must still be the function's defaults, with the same type.
+- **Literal constants.** Every literal constant must still hold its value, compared exactly: the same types throughout, floats by their bits, dictionary keys in order, for every literal kind including bytes, sets and complex numbers.
+- **No shared mutable objects.** No two literal constants or defaults may be one mutable object, which a value check cannot tell from two equal objects.
+- **Imports.** Every imported name must still be what its import binds: `from math import floor` must still be `math.floor`, and `import math` the loaded `math` module. It must hold a module or a callable other than a `functools.partial`; `from __future__` features are the exception. An imported value such as `from params import AMOUNT` is refused even when unused, since code can read a name without naming it. For the same reason, a star import of a module that exports data values is refused; import the callables you use by name. Each name a star import provides must be that module's own.
+- **No other names.** The namespace may hold no name the source does not bind, other than the dunders the import system and annotations set. The module's docstring and builtins must be what importing it made. A name set on the module after import is refused however the code would read it.
+
+A measures file edited after import, or a constant, default, import or function reassigned before either snapshot, is refused. If it changes after identity time, a receipt built with the identity's handle refuses the changed modifier, naming the difference or the reason it can no longer be bound. The check does not cover state in other modules (such as `params.PAYMENT` read through `import params`), or data the code reads at run time. Code in other modules is bound by `code_revision` or the installed versions, and `code_revision` names the code, not the process. Receipts are claims by their producer: these checks protect an honest producer against mistakes, not a reader against a producer who builds code to mislead. Code that reaches into frames, the garbage collector or other modules' namespaces is beyond them.
+
+Some modifiers are refused outright because their behaviour depends on values the digest cannot bind: closures, lambdas, nested functions, methods, `functools.partial` and builtins. Anything that is not a `Scenario` is refused too, rather than read as current law.
+
+### Several reforms in one run
+
+A combined run, such as a whole Budget package, applies several measures in one simulation. Bind each measure on its own, then bind the package with `composed_reform_identity(members, applied_before_data_load=...)`, passing the members' identities in application order and the phase the combined `Scenario` runs in. Its representation is `policyengine-uk.scenario_composition/v1`, and its digest covers the phase and each member's representation, digest and binding, in that order.
+
+The representation means one simulation, in the stated phase, that applies every member's parameter changes and then runs each member's simulation modifier in member order, whatever the policyengine-uk version. That order is part of the payload (`"application": "parameter_changes_then_modifiers"`). The changes are applied where the engine applies a Scenario's own parameter changes, so each member's changes act as they do when that member runs alone. The composer must run exactly that. It is not `Scenario.__add__`: that merge depends on how members are grouped, and it leaves the order to the engine (policyengine-uk 2.125.1 runs a Scenario's modifier before its parameter changes).
+
+On policyengine-uk 2.126.12, run a composition as one `Scenario` assembled from each member's own Scenario: `Scenario.from_reform(d)` for a reform dictionary member, the scenario itself for a scenario member. It carries the scenario members' merged `parameter_changes`, every member's `dated_reforms` in member order, and a modifier that runs every member's modifier in member order, including the one `from_reform` adds. Two shortcuts give a different run on that version:
+- Merging a reform dictionary into `parameter_changes`. A scalar value in a reform dictionary applies from 2023; in `parameter_changes` it applies from 2000.
+- Applying changes inside a modifier. When `applied_before_data_load` is false, that version applies a Scenario's own changes before the construction steps that read parameters (the Universal Credit rebalancing inputs and the structural reforms switched on by parameters) and runs its modifier after them, so a change made inside a modifier reaches neither step.
+
+The rules that follow:
+- The phase has no default. The composition's phase is the one that runs, whatever phase a member scenario had on its own.
+- No two members may change the same parameter, in any form, not even to the same value. With disjoint changes the result does not depend on grouping or order. The composer must check this; `composed_reform_identity` sees only identities, so it cannot.
+- Order is part of the digest: the same members in another order give a different digest. A nested composition's members are applied as if listed in its place.
+- A baseline shared by several members (one that applies parameter changes only) appears once, at the position of the first member that shares it; being parameter changes, its position does not affect the run.
+- Each member is revalidated, so an identity edited past its rules is refused rather than dropped or hashed.
+- A member with a live re-check handle (from `scenario_reform_identity`, or a composition) is recomputed before composing and refused if it changed. The package's identity keeps a handle that recomposes from those members, so both receipt builders re-check every member, nested compositions included, after the run. An identity loaded from JSON has no handle, as for any identity.
+- `current_law` members are dropped, and a composition with no remaining member is `current_law`. The binding is `code_revision` if any member's is.
+
+A reader who holds the members' identities can recompute the composition's digest and compare.
+
+To check a certificate, call `verify_certificate(receipt, bundle)` with the policyengine manifest for `certificate.bundle_version`. It rebuilds the certificate from that manifest's certification record, the same way the builder does. Then it compares every identity member: the record digest, the basis, the certified model, core and dataset digest, the certifier and the bundle version. It also checks that the receipt's whole `data` identity is the certified dataset with the certified bytes. An empty list means the bundle certifies exactly what the receipt claims, and through the receipt's own rules that pins `model.version`, `core.version` and `wrapper.version` too. `certificate_source`, `qualification_receipt_sha256`, `tro_composition_fingerprint` and wheel hashes are not determined by the bundle, so it does not check them.
+
+Pass the release artifact that was loaded as `dataset_path`. A derived per-year file such as `enhanced_frs_2024_25_year_2026.h5` has different bytes from the artifact the bundle pins, so a receipt for it is refused.
+
+## Digests
+
+Every receipt digest is the lowercase hexadecimal SHA-256 of the [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785) canonical JSON of a value, computed in two steps:
+1. **Strict JSON conversion.** Only JSON-native values are accepted: strings, booleans, null, integers within ±(2^53−1), finite doubles, arrays and objects with string keys. NumPy boolean, integer and floating scalars are converted to their values, integer keys to their decimal strings, and `datetime.date` to `YYYY-MM-DD`. Anything else is refused rather than silently stringified, including sets, `Decimal`, arrays (0-d and masked ones too, so `np.ma.masked` never becomes `0.0`), NaN and colliding keys.
+2. **Null removal.** Object members whose value is `null` are removed. Array elements are kept.
+
+RFC 8785 fixes member order, string escaping and number formatting, so Python and JavaScript produce the same bytes. Removing nulls means a producer that writes `"field": null` and one that omits the field compute the same digest. In Python use `policyengine.provenance.receipt.content_sha256`. In JavaScript:
+
+```javascript
+import canonicalize from "canonicalize";
+import { createHash } from "node:crypto";
+
+const dropNulls = (v) =>
+  Array.isArray(v) ? v.map(dropNulls)
+  : v && typeof v === "object"
+    ? Object.fromEntries(Object.entries(v).filter(([, x]) => x !== null).map(([k, x]) => [k, dropNulls(x)]))
+    : v;
+const digest = (value) => createHash("sha256").update(canonicalize(dropNulls(value)), "utf8").digest("hex");
+```
+
+Reform digests are the exception: a reform payload is digested with its nulls kept, because there a null can mean something. `Scenario.from_reform({"p": {"2027": None}})` removes a value that `{"p": {}}` keeps, so the two must not share a digest. Reform payloads may also hold infinite numbers, which PolicyEngine parameters use for an uncapped threshold and JSON cannot hold. `inf` is digested as the object `{"$nonfinite": "Infinity"}` and `-inf` as `{"$nonfinite": "-Infinity"}`, and a payload that uses the key `$nonfinite` itself is refused, so no other payload digests like an infinity. The encoding happens inside strict conversion, so a payload it refuses without infinities, such as one with colliding keys, is refused with them too. NaN is refused, and a payload without infinities digests as before, except one whose strings, keys or dates are subclasses that override how `str` or `date` render: those now digest by their characters and their date, as content digests do. The JavaScript above covers content digests only: a JavaScript verifier of reform digests must keep nulls, replace `Infinity` and `-Infinity` with the reserved objects before canonicalizing, and refuse a payload with a `$nonfinite` key.
+
+The pre-promotion replay interface with policyengine-scorecard follows a different rule, set by its own schemas: plain RFC 8785 over the document as sent, nulls included. `policyengine.provenance.canonical.canonical_sha256` computes that.
+
+| Digest | Value hashed |
+|---|---|
+| `execution_identity_sha256` | `schema_version`, `country_id`, `wrapper`, `model`, `core`, `data`, `certification_status`, and the certificate's content: bundle version, record digest, basis, certified versions, dataset digest and certifier. Results with the same execution identity recorded the same package identities (versions, install origins and wheel evidence) and the same data under the same certificate, however the receipt was built. For a certified receipt those identities name the code that ran. An uncertified one does not bind its packages' contents: two runs from an editable checkout edited in between share an execution identity. |
+| `receipt_sha256` | The whole receipt except `runtime` and the reform and baseline `name` labels. Renaming a measure therefore does not change it. |
+| `certificate.certificate_sha256` | The bundle manifest's `data_releases.uk` record. |
+| `reform.reform_sha256` | `{"representation": ..., "payload": ...}`, with the payload exactly as given, nulls kept (strict JSON conversion with infinities as `{"$nonfinite": "Infinity"}` or `"-Infinity"`, then RFC 8785, with no null removal). For a composition the payload is `{"applied_before_data_load": ..., "application": "parameter_changes_then_modifiers", "members": [{"representation", "reform_sha256", "binding"}, ...]}`, with members in application order. |
+| `outputs_sha256` | The output payload. |
+
+Reform digests are never normalized across representations. The same keys can legitimately mean different things in different representations: for example, a bare year in a policyengine-uk country dictionary changes that one year only. So each digest is tied to the representation that was submitted. The v1 representations are `policyengine.parameter_values/v1`, `policyengine.reform_dict/v1`, `policyengine-uk.reform_dict/v1`, `policyengine-uk.scenario/v1`, `policyengine-uk.scenario_composition/v1` and `policyengine-sim-api.policy/v1`.
+
+## Rules JSON Schema cannot express
+
+Consumers that validate with the JSON Schema rather than the Python model must also check the rules listed under `x-policyengine-rules` in the schema file:
+
+- `years` is sorted with no duplicates, and each year is written as an integer (`2027`, not `2027.0`);
+- `supported_years` is sorted with no duplicates, and a year in `years` but not in `supported_years` must be shown as provisional or not at all;
+- a certificate is present exactly when the receipt is certified, and it names the receipt's country, model and core versions, dataset digest and `policyengine` version;
+- a `current_law` reform has no digest, while any other reform has both a representation and a digest;
+- a wheel digest always comes with its evidence;
+- a certified receipt has no `editable` package, and no `directory` or `vcs` package without `installed_record_verified` wheel evidence;
+- a certificate is trusted only after it, and the receipt's `data`, have been checked against the published manifest it names (`verify_certificate`).
+
+The schema itself fixes the package names (`policyengine`, `policyengine-uk`, `policyengine-core`) and the UK fiscal-year convention, which starts on 6 April.
+
+## Compatibility
+
+Adding an optional field is a compatible v1 change. Optional fields are omitted from digests when absent, so existing digests stay valid. Removing or redefining a field, or changing what a digest covers, requires schema v2.

{policyengine-6.4.2 → policyengine-6.5.0}/docs/run-records.md

@@ -46,7 +46,7 @@ The directory contains:
 | `bundle.trace.tro.jsonld` | The certified bundle TRO (model + data pins) |
 | `reform.json` | The reform as parameter values with effective dates |
 | `input.json` | Input dataset hash, dynamics, scoping, extra variables |
-| `results.json` | Output dataset hash and per-entity table summaries; for US runs, the SPM receipt and the renamed stored inputs mapped onto live ones (see [Renamed inputs in stored data](microsim.md#renamed-inputs-in-stored-data)) |
+| `results.json` | Output dataset hash and per-entity table summaries; for US runs, the SPM receipt and the renamed stored inputs mapped onto live ones (see [Renamed inputs in stored data](microsim.md#renamed-inputs-in-stored-data)); for UK runs, the capital gains realisation response the run applied (see [The UK capital gains realisation response](microsim.md#the-uk-capital-gains-realisation-response)) |
 
 All payload files are written with the same canonical JSON used for
 hashing, so the record verifies offline exactly as written.

{policyengine-6.4.2 → policyengine-6.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "policyengine"
-version = "6.4.2"
+version = "6.5.0"
 description = "A package to conduct policy analysis using PolicyEngine tax-benefit models."
 readme = "README.md"
 authors = [
@@ -19,7 +19,7 @@ classifiers = [
     "Programming Language :: Python :: 3.14",
 ]
 dependencies = [
-    "pydantic>=2.0.0",
+    "pydantic>=2.10.0",
     "pandas>=2.0.0",
     "h5py>=3.0.0",
     "microdf_python>=1.3.0,<1.4",
@@ -69,6 +69,7 @@ dev = [
     "mypy>=1.11.0",
     "pytest-cov>=5.0.0",
     "hypothesis>=6.100.0",
+    "rfc8785>=0.1.4",
     "policyengine-core==3.33.0",
     "policyengine-us==2.2.1",
     "policyengine-uk==2.130.1",
@@ -96,6 +97,9 @@ testpaths = [
 filterwarnings = [
     "ignore::pydantic.warnings.PydanticDeprecatedSince20",
 ]
+markers = [
+    "real_install: runs the receipt's loaded-package check against the real installation",
+]
 
 [tool.ruff]
 line-length = 88

policyengine-6.5.0/scripts/generate_receipt_schemas.py

@@ -0,0 +1,60 @@
+"""Write the published JSON Schemas for policyengine.py receipt contracts.
+
+The Pydantic models in ``policyengine.provenance.receipt`` are the source;
+the schema files under ``src/policyengine/data/schemas/`` are what other
+languages consume. ``tests/test_execution_receipt.py`` checks that the two
+accept and reject the same receipts.
+
+Usage::
+
+    python scripts/generate_receipt_schemas.py          # write
+    python scripts/generate_receipt_schemas.py --check  # fail if stale
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+SCHEMA_DIR = REPO_ROOT / "src" / "policyengine" / "data" / "schemas"
+os.environ.setdefault("POLICYENGINE_SKIP_COUNTRY_IMPORTS", "1")
+sys.path.insert(0, str(REPO_ROOT / "src"))
+
+from policyengine.provenance.receipt import ExecutionReceipt  # noqa: E402
+
+SCHEMAS = {
+    "execution_receipt.v1.schema.json": ExecutionReceipt.json_schema,
+}
+
+
+def render(builder) -> str:
+    return json.dumps(builder(), indent=2, sort_keys=True, ensure_ascii=False) + "\n"
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument("--check", action="store_true")
+    args = parser.parse_args(argv)
+    stale = []
+    for filename, builder in SCHEMAS.items():
+        path = SCHEMA_DIR / filename
+        content = render(builder)
+        if path.exists() and path.read_text() == content:
+            continue
+        if args.check:
+            stale.append(path)
+            continue
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_text(content)
+        print(f"wrote {path.relative_to(REPO_ROOT)}")
+    for path in stale:
+        print(f"{path.relative_to(REPO_ROOT)} is not up to date.", file=sys.stderr)
+    return 1 if stale else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

{policyengine-6.4.2 → policyengine-6.5.0}/src/policyengine/__init__.py

@@ -48,3 +48,11 @@ else:  # pragma: no cover
     uk = None  # type: ignore[assignment]
 
 __all__ = ["Simulation", "outputs", "uk", "us"]
+
+# The wrapper's own version as imported, for receipts to compare with what is
+# installed when they are built.
+from policyengine.provenance.loaded import (  # noqa: E402
+    record_loaded_versions as _record_loaded_versions,
+)
+
+_record_loaded_versions(("policyengine",))

{policyengine-6.4.2 → policyengine-6.5.0}/src/policyengine/core/__init__.py

@@ -27,6 +27,7 @@ from .scoping_strategy import (
 )
 from .simulation import Simulation as Simulation
 from .tax_benefit_model import TaxBenefitModel as TaxBenefitModel
+from .tax_benefit_model_version import StaleOutputError as StaleOutputError
 from .tax_benefit_model_version import (
     TaxBenefitModelVersion as TaxBenefitModelVersion,
 )

{policyengine-6.4.2 → policyengine-6.5.0}/src/policyengine/core/run_record.py

@@ -20,6 +20,7 @@ emitting a false certificate.
 
 from __future__ import annotations
 
+import copy
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import TYPE_CHECKING, Any, Mapping, Optional, Union
@@ -202,6 +203,29 @@ def build_simulation_run_record_payloads(
     if legacy_spm_declarations is not None:
         results_payload["legacy_spm_declarations"] = dict(legacy_spm_declarations)
 
+    # The capital gains realisation response a UK run applied: the model
+    # resolves its default, so the record binds the assumption itself. See
+    # ``policyengine.tax_benefit_models.uk.capital_gains_responses``.
+    capital_gains_responses = (
+        getattr(simulation.output_dataset, "metadata", None) or {}
+    ).get("capital_gains_responses")
+    if capital_gains_responses is not None:
+        from policyengine.tax_benefit_models.uk.capital_gains_responses import (
+            simulation_request,
+        )
+
+        if (
+            capital_gains_responses["request"]["digest"]
+            != simulation_request(simulation)["digest"]
+        ):
+            raise ValueError(
+                "Capital gains response settings changed since this output was "
+                "calculated; run again before writing a run record"
+            )
+        results_payload["capital_gains_responses"] = copy.deepcopy(
+            capital_gains_responses
+        )
+
     return {"reform": reform, "input": input_payload, "results": results_payload}