physmap 0.2.4__tar.gz → 0.2.5__tar.gz

{physmap-0.2.4/src/physmap.egg-info → physmap-0.2.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: physmap
-Version: 0.2.4
+Version: 0.2.5
 Summary: Physics-aware credibility checks for AI surrogates in multiphysics simulation: closure validity, surrogate observability, and causal materiality.
 Author: Vishnu Vettrivel
 License-Expression: MIT
@@ -28,6 +28,8 @@ Requires-Dist: matplotlib>=3.5; extra == "experiment"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.4; extra == "dev"
 Requires-Dist: pytest-xdist>=3.5; extra == "dev"
+Provides-Extra: architectures
+Requires-Dist: torch>=2.2; extra == "architectures"
 Dynamic: license-file
 
 # PhysMAP
@@ -36,18 +38,41 @@ Dynamic: license-file
 
 Physics-aware credibility checks for AI surrogates in multiphysics simulation.
 
-A surrogate trained on `(Re, Pr)` over the fully-developed region of a heated pipe is
-blind to `x/D`, the variable that actually governs the entrance region. So it fails
-there, confidently. And so does an input-based OOD detector, because it sees the
-same two columns the surrogate does — and in those two columns the entrance points look
-perfectly ordinary.
+An input-based OOD detector asks whether a prediction's inputs look familiar. PhysMAP asks two
+other questions: **does the physics the model relies on still apply here**, and **does a
+mechanism the model left out materially change the quantity of interest?** It keeps the OOD
+detectors, and adds the checks they cannot make — because what breaks a surrogate is often not
+one of its inputs.
 
-PhysMAP reads the bound variable from the test coordinates instead of from the
-surrogate's inputs, checks it against the closure relation's validated range, and knows
-at fit time whether that variable is structurally observable to the surrogate at all.
+- **Applicability assurance** — shown on the entrance region of a heated pipe (NACA TN-1451).
+- **Causal materiality** — shown in a controlled model-reuse stress test on Lewis (1992)
+  Test 35A, the main controlled demonstration.
+- **Supporting evidence** across seven published datasets, led by a hypersonic transition case
+  (Casper) where three different surrogate architectures give the same result.
 
-**That is the point of PhysMAP: it catches the failures an input-based OOD detector cannot
-see, because what broke the surrogate is not one of its inputs.**
+No LLM is in any path. Every explanation is a deterministic rendered template.
+
+## Applicability assurance: the NACA x/D entrance region
+
+![Applicability assurance: the x/D entrance region](https://raw.githubusercontent.com/cloudronin/physmap/main/docs/talk/figures/bench_1_naca_entrance.png)
+
+A heat-transfer surrogate that takes only `(Re, Pr)` relies on a closure — here the Gnielinski
+correlation — validated for fully developed flow, from x/D = 10 on. Near a pipe's inlet, `x/D`
+governs the heat transfer, and neither the surrogate nor an input-based OOD detector sees it:
+in `(Re, Pr)` the entrance points look perfectly ordinary. On NACA TN-1451's Fig 10 (two-reader
+digitisation), against the measurement, at the numerical-error threshold (17.5 %):
+
+- Gnielinski is **within the threshold on 40 of 40** fully developed points.
+- It **exceeds the threshold on 9 of 45** entrance points — all at x/D ≤ 5, the largest 38 % at
+  the inlet.
+- PhysMAP identifies **all 45 entrance predictions as outside the closure's supported
+  applicability region**. The input-based OOD detectors fire on none.
+- **The other 36 are numerically acceptable, but not physically supported by that closure.**
+
+Numerical agreement does not by itself establish that a prediction is credibly supported.
+PhysMAP reads the bound variable from the test coordinates rather than from the surrogate's
+inputs, checks it against the closure's validated range, and knows at setup that `x/D` is
+structurally invisible to this surrogate:
 
 ```
 verdicts: {'REJECT': 45}
@@ -58,58 +83,132 @@ rationale: Prediction relies on gnielinski-1976 beyond its validated x_over_D bo
            because they cannot observe x_over_D.
 ```
 
-No LLM is in any path. Every explanation is a deterministic rendered template.
+The −63% is the literature's figure; on this data the largest entrance error is 38%. Run it:
+[`examples/naca_entrance_region.py`](https://github.com/cloudronin/physmap/blob/main/examples/naca_entrance_region.py).
+
+## Causal materiality: a controlled model-reuse stress test (Lewis 35A)
+
+Applicability asks whether the physics a prediction relies on still applies. This asks
+something else: when a mechanism the surrogate never saw becomes active, does it change the
+answer — and where?
 
-Run it yourself: [`examples/naca_entrance_region.py`](https://github.com/cloudronin/physmap/blob/main/examples/naca_entrance_region.py).
+```bash
+physmap stress-test lewis-reuse
+```
 
-## What PhysMAP adds to OOD detection
+**The setup.** The surrogate was trained for forced convection, where gravity did not vary
+and was not an input. It was then reused in vertical heated flow, where buoyancy became
+material. A mixed-convection surrogate designed for this regime should include Richardson
+number, Grashof number, or equivalent physical information — this test reuses one without
+it, on purpose.
 
-![What PhysMAP adds to input-based OOD detection, per dataset](https://raw.githubusercontent.com/cloudronin/physmap/main/docs/talk/figures/bench_3_what_physmap_adds.png)
+**The claim.** PhysMAP detects when model reuse activates a physically relevant mechanism
+outside the surrogate's observable input space. An input-only OOD detector cannot identify a
+change absent from its input contract.
 
-An input-based OOD detector judges a prediction by where its inputs sit relative to the
-training data. It asks that well — but it cannot see a change in something that is not an
-input. PhysMAP adds the check it cannot make: it reads the variables the surrogate never saw,
-tests the physics relation behind the surrogate against its validated range, and knows at
-setup which of those variables the detectors can see. It keeps the OOD detectors, and
-overrides them only where they are structurally blind.
+**The input contract.** Both the surrogate and the input-based OOD detector — the
+seven-vehicle benchmark's own, unchanged — receive `Re`, `Pr` and `x_over_D`. Neither
+receives gravity, `Ri`, `Gr` or heat flux. **Every visible deployment input exactly matches a
+training input.**
 
-The seven-vehicle benchmark measures exactly that, per dataset, at the default setting (the
-99th percentile):
+**The result**, same visible inputs, two physical states, on Lewis's (1992) vertical-tube
+experiment:
 
-| Dataset | Can the OOD detectors see the cause? | Wrong predictions caught only by PhysMAP | Right predictions flagged anyway |
+| | surrogate error | input-based OOD scores | PhysMAP materiality |
 |---|---|---|---|
-| `naca_tn1451` — heated pipe, entrance region | no | 20 of 20 | 24 |
-| `casper_hypersonic_transition` — hypersonic transition | no | 4 of 8 | 0 |
-| `jin_sco2_buoyancy` — supercritical CO2, vertical tube | no | 15 of 26 | 0 |
-| `velazquez_sco2` — supercritical CO2, property variation | partly | 18 of 67 | 0 |
-| `dirker_water` — water, horizontal tube | partly | 2 of 11 | 16 |
-| `marineau_hypersonic_transition` — hypersonic transition | yes | 0 of 6 | 0 |
-| `forrest` — rectangular channel | yes | not tested: one training row | — |
-
-- **Where it helps:** when the cause of failure is hidden from the surrogate's inputs. There
-  PhysMAP catches wrong predictions the OOD detectors miss entirely — all 20 for the pipe
-  entrance.
-- **Where it doesn't:** when the cause is an input, the OOD detectors already see it and
-  PhysMAP adds nothing. `marineau_hypersonic_transition` is the control that shows it.
-- **The cost:** the closure check flags anything outside a relation's validated range, even
-  when the surrogate happens to be right — 24 false alarms for the pipe entrance, 16 for
-  `dirker_water`.
-
-Counts are rows of each dataset, not independent cases, and are never pooled into a rate
-across datasets; the source values are digitised from publications. The benchmark's
-input-based detectors are distance-to-training and GP variance. `physmap benchmark report`
-prints these counts, and `physmap benchmark run` recomputes them from a clone.
-
-This is a different question from PhysMAP's causal-materiality check — whether a mechanism
-the surrogate never saw is large enough to matter — which is demonstrated separately below.
-Neither is evidence for the other.
+| gravity off — the accurate control | within 0.06 % | quiet at the default setting; identical in both rows | 0 |
+| gravity on — Lewis's measurement | 17–18 % downstream | quiet at the default setting; identical in both rows | up to 0.195 |
+
+Identical OOD scores and zero-versus-0.195 materiality need no threshold. The flag threshold θ
+is not yet locked, so materiality is reported as numbers, and θ = 0.10 appears only as an
+illustration.
+
+A secondary result shows specificity: when the visible operating point falls *between*
+training runs, the OOD detector warns in both the accurate and the inaccurate case —
+identically — while PhysMAP changes with the physical mechanism.
+
+**What it is not.** Not a claim that OOD detectors fail in general — this one does exactly its
+job. Not a suggestion to leave gravity out. Not a claim about NVIDIA PhysicsNeMo, whose OOD
+and physics checks are distinct and were not run. **One run, a development demonstration:** its
+stations are not independent cases, and no precision, recall or F1 is computed.
+
+The command recomputes everything from this checkout in a few minutes, asserts the exact
+input overlap and the unchanged OOD scores, and diffs itself against a committed bank — the
+same contract as `physmap benchmark run`, deliberately kept a separate command because this is
+a causal-materiality result. It recomputes from committed CFD-derived profiles and does not rerun
+OpenFOAM. Full record:
+[docs/findings/lewis-ood-head-to-head.md](https://github.com/cloudronin/physmap/blob/main/docs/findings/lewis-ood-head-to-head.md). The NAFEMS
+talk package — figures, facts sheet, claims ledger, and a clean-clone reproduction record — is
+in [docs/talk/](https://github.com/cloudronin/physmap/blob/main/docs/talk/README.md).
+
+
+## Supporting evidence: seven published datasets
+
+The seven-vehicle benchmark asks, per dataset: which wrong predictions does PhysMAP flag that
+the input-based detectors (distance-to-training and GP variance) miss — and was the surrogate
+accurate at home, so that deployment is what created the failure?
+
+**Casper leads.** Hypersonic transition; the cause, tunnel freestream noise, is not a surrogate
+input. The surrogate is wrong on 6 of 159 home rows held out (4%), and on 8 of 8 deployed. A
+Gaussian process, a DeepONet and gradient-boosted trees each pass the same home-accuracy check,
+and **each leaves 4 of 8 deployment errors that only PhysMAP flags**.
+
+**Dirker supports it.** Water in a horizontal tube; the cause, buoyancy, is partly visible to
+the inputs. Wrong on 0 of 31 home rows held out, and on 11 of 60 deployed; PhysMAP flags 2 of
+those 11 that the detectors miss.
+
+![The home baseline behind each count](https://raw.githubusercontent.com/cloudronin/physmap/main/docs/talk/figures/bench_4_home_baseline.png)
+
+**The full matrix** — every dataset, at the default setting (the 99th percentile), with how its
+home error was obtained, its error rates and its limitation:
+
+| Dataset | Detectors see the cause? | Home error: how obtained | Home wrong | Deployed wrong | Wrong, flagged only by PhysMAP | Accurate, flagged anyway | Limitation |
+|---|---|---|---|---|---|---|---|
+| `casper_hypersonic_transition` — hypersonic transition | no | fitted; held out by refit (in-sample 2 of 159) | 6 of 159 (4%) | 8 of 8 (100%) | 4 of 8 | 0 | — |
+| `dirker_water` — water, horizontal tube | partly | fitted; held out by refit (in-sample 0 of 31) | 0 of 31 (0%) | 11 of 60 (18%) | 2 of 11 | 16 | the cause is partly visible |
+| `naca_tn1451` — heated pipe, entrance region | no | published correlation; 40 of 40 inside its range | 0 of 40 (0%) | 9 of 45 (20%) | 9 of 9 | 19 | the applicability case above, same data — not separate evidence |
+| `jin_sco2_buoyancy` — supercritical CO2, vertical tube | no | published correlation; 11 of 17 inside its range | 17 of 17 (100%) | 26 of 27 (96%) | 15 of 26 | 0 | wrong almost everywhere: no home baseline |
+| `velazquez_sco2` — supercritical CO2, property variation | partly | published correlation; 197 of 393 inside its range | 386 of 393 (98%) | 67 of 67 (100%) | 18 of 67 | 0 | wrong almost everywhere: no home baseline |
+| `marineau_hypersonic_transition` — hypersonic transition | yes | fitted; held out by refit (in-sample 0 of 9) | 5 of 9 (56%) | 6 of 6 (100%) | 0 of 6 | 0 | the control; nine home rows are too few |
+| `forrest` — rectangular channel | yes | published correlation | 0 of 1 | 4 of 4 (100%) | not tested | — | one training row; triage-grade values |
+
+"Wrong" is each dataset's own threshold, set from measurement noise; "accurate" is a tighter
+one, with a dead band between them. Home error is labelled by how it was obtained: where the
+surrogate was fitted to the home rows, it is also refitted without each row in turn; a
+published correlation was never fitted to them.
+
+- **Where the evidence holds:** Casper and Dirker have a home baseline from which deployment
+  fails — and NACA, which is the applicability case above.
+- **Where the reading stops:** Jin and Velazquez keep their counts, but their correlations are
+  wrong almost everywhere, at home too, so the counts cannot show that deployment created the
+  failure.
+- **Where PhysMAP adds nothing:** Marineau, where the cause is a surrogate input and the
+  detectors already see it.
+- **Accurate predictions flagged anyway:** PhysMAP flags every prediction outside a closure's
+  supported region, accurate or not — 19 for NACA, 16 for Dirker. As detection those are false
+  alarms; as applicability assurance, they are predictions the closure does not support.
+- **The kind of model:** on Casper, three model types pass the home check and each leaves 4 of
+  8 errors only PhysMAP flags. On NACA all three pass too, but its home reads repeat — 40 rows,
+  5 distinct values — so leave-one-out says little there. On Jin and Velazquez none passes.
+
+No gate decides a dataset's reading: the counts and rates are the evidence, and
+`physmap benchmark report` prints each dataset's interpretation as prose. Counts are rows of
+each dataset, not independent cases, never pooled into a rate, and digitised from publications.
+A one-sided Fisher exact test was run after the counts were seen; it is reported as exploratory
+and decides nothing.
+
+**Bank v0.4.1.** It corrects the NACA source data: the original bank's NACA row came from an
+automated read of the figure later found invalid — wrong axis calibration, points on gridline
+crossings — and its "20 of 20" is withdrawn. The original bank is kept unchanged for audit, not
+as evidence. See [`data/naca/CORRECTION_v0_4_1.md`](https://github.com/cloudronin/physmap/blob/main/data/naca/CORRECTION_v0_4_1.md).
 
 ## The benchmark: seven vehicles, all rerunnable
 
 ```bash
-physmap benchmark report     # all seven outcomes and the counts above, each marked recomputed or banked
+physmap benchmark report     # all seven outcomes, the counts above and the home baseline
 physmap benchmark run        # recomputes all seven from this checkout, diffs against the bank
 physmap benchmark coverage   # what that subset does and does not cover
+physmap benchmark architectures --banked   # three model types per vehicle; drop --banked to retrain (PyTorch)
 ```
 
 All seven vehicles ran, all seven outcomes are reported, and **all seven now ship their
@@ -162,9 +261,9 @@ one to another.
 
 | Check | Question | Status | Evidence |
 |---|---|---|---|
-| **Closure validity** | Is this closure relation being applied outside the range it was calibrated on? | Shipping | the seven-vehicle benchmark, above |
-| **Surrogate observability** | Can the surrogate's inputs even represent the variable that governs the failure? | Shipping | the seven-vehicle benchmark, above |
-| **Causal materiality** | Is the out-of-range mechanism large enough to matter for the quantity of interest? | Preview — method, plus one controlled stress test | Lewis 35A, below |
+| **Closure validity** | Is this closure relation being applied outside the range it was calibrated on? | Shipping | applicability assurance (NACA x/D); the seven-vehicle benchmark |
+| **Surrogate observability** | Can the surrogate's inputs even represent the variable that governs the failure? | Shipping | the same |
+| **Causal materiality** | Is the out-of-range mechanism large enough to matter for the quantity of interest? | Preview — method, plus one controlled stress test | Lewis 35A, the main controlled demonstration |
 
 ## What this release claims
 
@@ -192,61 +291,6 @@ not reproduction of the published numbers. The protocol defines `REPRODUCED` and
 unreachable, so the word cannot drift onto a weaker result. See
 [`protocols/`](https://github.com/cloudronin/physmap/tree/main/protocols).
 
-## A separate question: causal materiality, in a controlled stress test (Lewis 35A)
-
-The benchmark asks whether PhysMAP catches failures the OOD detectors cannot see. This asks
-something else: when a mechanism the surrogate never saw becomes active, does it matter for the
-answer — and where?
-
-```bash
-physmap stress-test lewis-reuse
-```
-
-**The setup.** The surrogate was trained for forced convection, where gravity did not vary
-and was not an input. It was then reused in vertical heated flow, where buoyancy became
-material. A mixed-convection surrogate designed for this regime should include Richardson
-number, Grashof number, or equivalent physical information — this test reuses one without
-it, on purpose.
-
-**The claim.** PhysMAP detects when model reuse activates a physically relevant mechanism
-outside the surrogate's observable input space. An input-only OOD detector cannot identify a
-change absent from its input contract.
-
-**The input contract.** Both the surrogate and the input-based OOD detector — the
-seven-vehicle benchmark's own, unchanged — receive `Re`, `Pr` and `x_over_D`. Neither
-receives gravity, `Ri`, `Gr` or heat flux. **Every visible deployment input exactly matches a
-training input.**
-
-**The result**, same visible inputs, two physical states, on Lewis's (1992) vertical-tube
-experiment:
-
-| | surrogate error | input-based OOD scores | PhysMAP materiality |
-|---|---|---|---|
-| gravity off — the accurate control | within 0.06 % | identical in both rows | 0 |
-| gravity on — Lewis's measurement | 17–18 % downstream | identical in both rows | up to 0.195 |
-
-Identical OOD scores and zero-versus-0.195 materiality need no threshold. The flag threshold θ
-is not yet locked, so materiality is reported as numbers, and θ = 0.10 appears only as an
-illustration.
-
-A secondary result shows specificity: when the visible operating point falls *between*
-training runs, the OOD detector warns in both the accurate and the inaccurate case —
-identically — while PhysMAP changes with the physical mechanism.
-
-**What it is not.** Not a claim that OOD detectors fail in general — this one does exactly its
-job. Not a suggestion to leave gravity out. Not a claim about NVIDIA PhysicsNeMo, whose OOD
-and physics checks are distinct and were not run. **One run, a development demonstration:** its
-stations are not independent cases, and no precision, recall or F1 is computed.
-
-The command recomputes everything from this checkout in a few minutes, asserts the exact
-input overlap and the unchanged OOD scores, and diffs itself against a committed bank — the
-same contract as `physmap benchmark run`, deliberately kept a separate command because this is
-a causal-materiality result. It recomputes from committed CFD-derived profiles and does not rerun
-OpenFOAM. Full record:
-[docs/findings/lewis-ood-head-to-head.md](https://github.com/cloudronin/physmap/blob/main/docs/findings/lewis-ood-head-to-head.md). The NAFEMS
-talk package — figures, facts sheet, claims ledger, and a clean-clone reproduction record — is
-in [docs/talk/](https://github.com/cloudronin/physmap/blob/main/docs/talk/README.md).
-
 ## Install
 
 ```bash
@@ -268,7 +312,8 @@ redistribution terms and partly because it is not runtime data. From a plain
 `pip install physmap`, those commands say so in one sentence and stop.
 
 Optional extras: `[jsonld]` adds evidence export via `uofa`, `[experiment]` adds
-`matplotlib` for benchmark figures, `[dev]` adds the test tooling.
+`matplotlib` for benchmark figures, `[architectures]` adds PyTorch for the DeepONet in
+`physmap benchmark architectures`, `[dev]` adds the test tooling.
 
 ## Using the guardrail