paramrf 0.37.4__tar.gz → 0.37.5__tar.gz

{paramrf-0.37.4 → paramrf-0.37.5}/AGENTS.md

@@ -27,7 +27,6 @@ there are none.
 
 ```bash
 .venv/bin/python -m pytest tests/test_x.py  # the tests covering a change, while working
-.venv/bin/python -m pytest -n 4            # local test command: full suite, once before committing
 .venv/bin/python -c "import pmrf"          # import smoke check
 ```
 

{paramrf-0.37.4 → paramrf-0.37.5}/CLAUDE.md

@@ -27,7 +27,6 @@ there are none.
 
 ```bash
 .venv/bin/python -m pytest tests/test_x.py  # the tests covering a change, while working
-.venv/bin/python -m pytest -n 4            # local test command: full suite, once before committing
 .venv/bin/python -c "import pmrf"          # import smoke check
 ```
 

{paramrf-0.37.4 → paramrf-0.37.5}/CONTEXT.md

@@ -184,9 +184,10 @@ noise. `pmrf.objectives.MarginalLogLikelihood` scores it; see
 The space in which probability is defined: the prediction after the event
 transform, with frequency as the last (event) axis. Every other axis is a batch
 axis. One entry of that batch, for example the real part of S21, is an **event
-block**. Blocks are independent: a kernel chooses the covariance *within* each
-block (`AutoCrossKernel` routes by block, `SharedIndependentKernel` shares
-hyperparameters across blocks) and never couples two blocks.
+block**. Under a `GaussianProcess` discrepancy, blocks are independent: a kernel
+chooses the covariance *within* each block (`AutoCrossKernel` routes by block,
+`SharedIndependentKernel` shares hyperparameters across blocks) and never couples
+two blocks. A **shared discrepancy** couples blocks, and datasets.
 
 *Avoid:* "output" or "task" for a block; "cross-covariance" for the kernel an
 `AutoCrossKernel` gives a transmission block.
@@ -194,8 +195,40 @@ hyperparameters across blocks) and never couples two blocks.
 ### Discrepancy
 
 $\delta$, the systematic misfit between model and reality, as opposed to noise
-$\varepsilon$. Either deterministic or a `GaussianProcess` over each event block.
-It is on the observable unless it is a port or internal discrepancy.
+$\varepsilon$. Either deterministic, a `GaussianProcess` over each event block, or
+a shared discrepancy. It is on the observable unless it is a port or internal
+discrepancy.
+
+### Shared latent
+
+A function of frequency with a Gaussian-process prior, common to several event blocks
+or several datasets: for example a VNA's residual directivity, which is the same for
+every device measured with one calibration. It has one or more rows (two for a
+complex function: Re and Im), each an independent draw from its kernel.
+
+*Avoid:* "task", "output" (those are event blocks); "error term" (one application's name
+for it); "coregionalisation" in user-facing text.
+
+### Mixing
+
+The linear map, at each frequency, from a shared latent's rows to one dataset's event
+blocks. It may depend on that dataset's prediction: multiplying a complex latent by the
+predicted reflection coefficient is a mixing. A mixing may carry parameters.
+
+### Shared discrepancy
+
+The discrepancy formed by mixing shared latents:
+$\delta_l(f) = \sum_j M_{lj}(f)\, g_j(f)$ for dataset $l$, with per-dataset noise on
+top. Because the latents are common, it couples the datasets and the event blocks it
+mixes into. `pmrf.objectives.SharedMarginalLogLikelihood` scores it.
+
+### Latent prediction
+
+The joint distribution of the shared latents at new frequencies, conditioned on the
+residuals of every dataset, excluding noise. It is what transfers to a new dataset that
+shares the latents, through that dataset's mixing.
+
+*Avoid:* "discrepancy prediction" for this (that is per dataset, in event space).
 
 ### Residual
 
@@ -236,6 +269,34 @@ together, from a linearisation: unlike a discrepancy prediction, it carries the
 parameters' uncertainty into $\delta$ and the cross-covariance between them. It is
 the form in which one fit's posterior becomes the next fit's joint prior.
 
+### Hyperparameter
+
+A free parameter owned by a term rather than by the model, such as white noise or a
+kernel's length scale. The model parameters are the free parameters of
+`problem.model`; the hyperparameters are the rest (ADR-0009).
+
+### Whitened residual
+
+A term's residual scaled so that half its squared norm is the term's negative log
+likelihood, up to terms constant in the model parameters, with the hyperparameters
+held: $L^{-1} r$ per event block for a `MarginalLogLikelihood`, with
+$\Sigma_D = K + \Sigma_n = L L^\top$. Its Jacobian is the whitened Jacobian of a
+linearisation (ADR-0009).
+
+### Prior residual
+
+A free parameter's contribution to a least-squares fit from its prior: half its
+squared norm is the negative log prior, up to a constant. $(\theta - \mu)/\sigma$ for a
+normal or truncated normal prior, the Cholesky-whitened $\theta - \mu$ for a joint
+normal prior, and nothing for a uniform prior, whose bounds are the box edges instead.
+It is computed from the distribution, not read from raw space (ADR-0009).
+
+### Hyperparameter step
+
+In an alternating MAP fit, the scalar minimisation of the full negative log posterior
+over the hyperparameters with the model parameters held. It takes the
+$\tfrac{1}{2}\ln|\Sigma_D|$ term, which a least-squares solve cannot (ADR-0009).
+
 ### Port discrepancy
 
 A discrepancy embedded at a component's ports, on its S-parameters rather than on
@@ -309,4 +370,7 @@ method, a public function or a value space. Read
 `docs/adr/0005-priors-by-name.md` before changing how priors are attached or
 scored, and `docs/adr/0007-edge-values.md` before changing how a
 circuit solver produces S, how bounds are validated, or which space a solver
-moves through.
+moves through. Read `docs/adr/0008-shared-discrepancy.md` before coupling event
+blocks or datasets in a discrepancy, and `docs/adr/0009-least-squares-map-fits.md`
+before changing how a least-squares minimiser builds its residual or treats
+hyperparameters.

{paramrf-0.37.4/paramrf.egg-info → paramrf-0.37.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: paramrf
-Version: 0.37.4
+Version: 0.37.5
 Summary: Parametric radio frequency modeling
 Author-email: Gary Allen <gvcallen@gmail.com>
 Project-URL: homepage, https://github.com/gvcallen/paramrf

paramrf-0.37.5/docs/adr/0008-shared-discrepancy.md

@@ -0,0 +1,40 @@
+# Shared discrepancy is a joint evaluator over per-dataset likelihoods
+
+A discrepancy whose latent functions are shared across datasets (or across event blocks
+of one dataset) makes those datasets statistically dependent, so their joint likelihood
+is not a sum of per-dataset terms. We add a new evaluator,
+`SharedMarginalLogLikelihood`, that takes several ordinary `MarginalLogLikelihood`s
+(each with no discrepancy of its own) plus a tuple of `SharedLatent`s, and returns one
+joint log-likelihood. It goes into `SummedTerms` as a single term. Each `SharedLatent`
+is a covariance kernel paired with a **mixing**: a function of the dataset's
+event-space prediction and frequency that returns, per frequency, a real matrix from
+the latent's rows to the dataset's event blocks. The joint covariance is low rank plus
+diagonal and is solved by Woodbury in the latent dimension, never densely.
+
+## Considered options
+
+- **A shared object inside several `MarginalLogLikelihood` terms, marginalised jointly by
+  `SummedTerms`.** Rejected: `SummedTerms` would have to detect sharing and stop being a
+  sum, and the joint maths would be spread across classes that should not know about
+  each other.
+- **Extend `GaussianProcess` and its kernels to couple event blocks** (as in the
+  correlated-GP note and the routing idea in #250). Rejected: a `GaussianProcess` is a
+  discrepancy for one dataset, and its kernels cannot see the prediction, which
+  multiplicative error (for example a VNA's tracking error, $e_t \Gamma$) needs.
+- **An error-box model component with GP-distributed error terms.** This is the exact
+  bilinear form, but its marginal is not Gaussian. It is a separate, later feature.
+
+## Consequences
+
+- `MarginalLogLikelihood` exposes its event-space residual, noise variance and
+  conditional-transform log-determinant, so the joint evaluator reuses all per-dataset
+  plumbing (predictor, observation, likelihood, event transform) without copying it.
+- A latent holds a kernel, not a `GaussianProcess`. Every existing kernel is reusable.
+- Mixings act in event space. Helpers that multiply by the prediction assume the
+  default Re/Im event layout and say so.
+- With a prediction-dependent mixing, the covariance depends on the model parameters;
+  gradients flow through it.
+- A `MarginalLogLikelihood` with a `GaussianProcess` is mathematically the one-dataset,
+  identity-mixing case. Both stay: the existing class is the fast independent-block
+  path. If the two ever duplicate maths rather than share plumbing, merge them into one
+  evaluator with the independent path as an internal optimisation.

paramrf-0.37.5/docs/adr/0009-least-squares-map-fits.md

@@ -0,0 +1,106 @@
+# ADR-0009: MAP fits are bounded least squares, alternating with a hyperparameter step
+
+Status: accepted (2026-10)
+
+## Context
+
+Physical line models often have parameters that are each identifiable but jointly
+near-degenerate: the loss is a long, narrow valley. On a synthetic `MicrostripLine` fit
+(10-500 MHz, `w`, `h`, `ep_r`, `tand`, `sigma` and `length` free, noise 1e-5, Jacobian
+condition number about 4e5 in box space), starting from a perturbed point:
+
+| Method | Outcome |
+|---|---|
+| `ScipyMinimize` L-BFGS-B (default) | Reports success at a wrong point |
+| `ScipyMinimize` trust-constr (BFGS) | Not converged in 3000 iterations |
+| trust-constr, exact JAX Hessian | Not converged in 3000 iterations, about 20x slower |
+| trust-constr, Gauss-Newton Hessian | Not converged in 3000 iterations |
+| trust-exact, Levenberg-Marquardt (unbounded) | Leave the box and fail |
+| Optimistix Levenberg-Marquardt, Dogleg, Gauss-Newton (raw space) | Stall on the bounds, where the raw Jacobian vanishes |
+| SciPy `least_squares`, `'trf'`, box space | Recovers all six true values |
+
+The exact Hessian adds the residual-weighted second derivative of the model, the term a
+linearisation drops. Far from the solution it hurts rather than helps.
+
+Under a Gaussian likelihood with fixed noise and discrepancy hyperparameters, the negative
+log posterior is half the squared norm of a residual vector, up to a constant, whenever
+each prior is Gaussian, truncated Gaussian or uniform. Noise and kernel hyperparameters
+are usually fitted too, and they enter through $\tfrac{1}{2}\ln|\Sigma_D|$, which is not a
+square.
+
+## Decision
+
+### A least-squares minimiser over box space
+
+`pmrf.optimize.ScipyLeastSquares` wraps `scipy.optimize.least_squares` with `'trf'`. Like
+any minimiser that honours bounds, it searches box space and is given the box's edges
+(ADR-0007). It minimises half the squared norm of a residual vector, not a scalar.
+
+### The residual vector
+
+The vector stacks the **whitened residual** of each term and the **prior residual** of each
+free parameter.
+
+- **Whitened residual.** For a `MarginalLogLikelihood`, $L^{-1} r$ per event block, with
+  $L$ the lower Cholesky factor of $\Sigma_D = K + \Sigma_n$: the same factor
+  `linearize` uses. For a `TargetLoss` with `MSELoss`, the prediction error scaled so
+  that half its squared norm is the loss.
+- **Prior residual.** Computed from the distribution, in the space it is over. It is not
+  the raw value, since truncation makes the raw map nonlinear.
+
+| Prior | Prior residual | Bounds |
+|---|---|---|
+| `Normal` | $(\theta - \mu)/\sigma$ | none |
+| `TruncatedNormal`, including a `Normal` truncated to a validity | $(\theta - \mu)/\sigma$ | the truncation |
+| Joint multivariate normal, including `truncate='unnormalised'` | $L_0^{-1}(\theta - \mu)$, with $\Sigma_0 = L_0 L_0^\top$ | each parameter's validity |
+| `Uniform`, or no prior | none | the parameter's bounds |
+
+A truncation normaliser is constant in $\theta$, so it does not move the MAP.
+
+### Alternating with a hyperparameter step
+
+The level above the minimiser splits the free parameters. The model parameters are the
+free parameters of `problem.model`. The **hyperparameters** are the free parameters the
+terms own, such as white noise and kernel hyperparameters. `run_minimizer` hands the
+minimiser both groups: a residual function of the model parameters given the
+hyperparameters, and the full scalar objective of the hyperparameters given the model
+parameters. The minimiser knows nothing about noise or kernels.
+
+Each round is a least-squares solve over the model parameters, then a **hyperparameter
+step**: a scalar minimisation of the full negative log posterior over the hyperparameters
+with the model parameters held, by `hyper_minimizer` (default `ScipyMinimize()`). The fit
+stops when a round no longer decreases the full objective. With no free hyperparameters
+it is a single least-squares solve.
+
+### Not yet supported
+
+Each of these raises an error saying it is not yet supported:
+
+- noise given as a callable of the prediction, since $\Sigma_D$ then depends on the
+  model parameters;
+- a mixing that depends on the prediction, for the same reason;
+- orthogonal discrepancy without a fixed reference basis;
+- priors other than those in the table, such as a prior over raw space carried to
+  declared space with a change-of-variables term;
+- `GibbsMarginalLogLikelihood`, losses other than `MSELoss`, and any term without a
+  whitened residual.
+
+## Considered options
+
+- **Restore the exact Hessian in `ScipyMinimize`.** Rejected as the fix: it did not
+  converge on the degenerate fit and is slow to compile. It may still be added for other
+  reasons.
+- **Optimistix least-squares solvers.** They keep the fit in JAX, but take no bounds, and
+  in raw space they stall on the bounds. Rejected.
+- **One least-squares solve over model parameters and hyperparameters together.** The
+  log-determinant is not a square; writing it as one gives a residual with a singular
+  derivative. Rejected.
+
+## Consequences
+
+- At the MAP, the stacked whitened Jacobian and prior square root of the last
+  least-squares solve are the $A$ of `posterior_covariance_factor`. The fit and its
+  linearisation share one approximation.
+- `ScipyMinimize`'s commented-out Hessian path is removed.
+- Shared discrepancy is supported once its whitened residual covers the joint covariance
+  across datasets, for mixings that do not depend on the prediction.

{paramrf-0.37.4 → paramrf-0.37.5/paramrf.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: paramrf
-Version: 0.37.4
+Version: 0.37.5
 Summary: Parametric radio frequency modeling
 Author-email: Gary Allen <gvcallen@gmail.com>
 Project-URL: homepage, https://github.com/gvcallen/paramrf

{paramrf-0.37.4 → paramrf-0.37.5}/paramrf.egg-info/SOURCES.txt

@@ -30,6 +30,8 @@ docs/adr/0004-profiled-lines.md
 docs/adr/0005-priors-by-name.md
 docs/adr/0006-reference-impedance.md
 docs/adr/0007-edge-values.md
+docs/adr/0008-shared-discrepancy.md
+docs/adr/0009-least-squares-map-fits.md
 docs/agents/domain.md
 docs/agents/issue-tracker.md
 docs/agents/triage-labels.md
@@ -155,6 +157,7 @@ pmrf/objectives/__init__.py
 pmrf/objectives/evaluators.py
 pmrf/objectives/losses.py
 pmrf/objectives/problems.py
+pmrf/objectives/shared.py
 pmrf/objectives/terms.py
 pmrf/optimize/__init__.py
 pmrf/optimize/base.py
@@ -178,6 +181,7 @@ pmrf/stats/distributions.py
 pmrf/stats/likelihoods.py
 pmrf/stats/linearization.py
 pmrf/stats/noise_models.py
+pmrf/stats/shared_discrepancy.py
 pmrf/utils/__init__.py
 pmrf/utils/array.py
 pmrf/utils/debug.py
@@ -209,6 +213,7 @@ tests/test_frequency.py
 tests/test_grouped_exports.py
 tests/test_infer_base.py
 tests/test_infer_sample.py
+tests/test_least_squares.py
 tests/test_line_internal_discrepancy_example.py
 tests/test_linearization.py
 tests/test_map_priors.py
@@ -225,6 +230,7 @@ tests/test_parameters_validity.py
 tests/test_port_discrepancy_example.py
 tests/test_raw_space_solving.py
 tests/test_serialization.py
+tests/test_shared_discrepancy.py
 tests/test_structural_updates.py
 tests/test_terms.py
 tests/test_transforms.py

{paramrf-0.37.4 → paramrf-0.37.5}/pmrf/fitting/minimize.py

@@ -40,6 +40,8 @@ def fit_minimize(
     noise: Param | Callable[[jnp.ndarray], jnp.ndarray] = None,
     discrepancy: Callable[[jnp.ndarray, jnp.ndarray], dist.AbstractDistribution] | None = None,    
     temperature: float = None,
+    use_orthogonal_discrepancy: bool = False,
+    orthogonal_rcond: float | None = None,
     **kwargs,
 ) -> FitResult[PyTreeT]:
     """
@@ -61,6 +63,14 @@ def fit_minimize(
     solver : AbstractMinimizer, optional
         The optimizer to use.
         See :mod:`pmrf.optimize` for available solvers.
+        :class:`pmrf.optimize.ScipyLeastSquares` suits fits whose parameters are
+        jointly near-degenerate. It supports a frequentist fit with
+        :class:`pmrf.objectives.losses.MSELoss`, and a bayesian fit with a
+        :class:`pmrf.stats.likelihoods.GaussianLikelihood`, fitting a free noise and a
+        Gaussian-process discrepancy's hyperparameters by alternating with a
+        hyperparameter step. Noise given as a callable of the prediction, orthogonal
+        discrepancy without a fixed reference basis, and priors on the model other than
+        normal, truncated normal and uniform are not yet supported.
     features : str | list[str] | Callable[[PyTree, Frequency], jnp.ndarray], default='s'
         The RF features to fit.
         Can either be function, a callable PyTree with optional parameters, or a string,
@@ -114,6 +124,12 @@ def fit_minimize(
         The temperature value for generalized Bayesian optimization.
         Only allowed if `inference` is 'bayesian' and `loss` is not None.
         Defaults to 1.0 internally.
+    use_orthogonal_discrepancy : bool, default=False
+        Fix the discrepancy's tangent basis at the initial model for each dataset.
+        Requires a discrepancy model.
+    orthogonal_rcond : float, optional
+        Relative cutoff for the tangent basis singular values. Required when
+        enabling orthogonal discrepancy.
     **kwargs : dict
         Additional keyword arguments passed to :func:`pmrf.optimize.minimize`
         and then underlying solver.
@@ -139,6 +155,8 @@ def fit_minimize(
         raise ValueError("Only one of either `loss` or `likelihood` can be past to `fit_minimize`")
     if discrepancy is not None and not (likelihood is not None or inference == 'bayesian'):
         raise ValueError("Discrepancy models can only be passed if `likelihood` is passed or `inference` is 'bayesian'`")
+    if use_orthogonal_discrepancy and discrepancy is None:
+        raise ValueError("`use_orthogonal_discrepancy=True` requires a discrepancy model")
     
     # Resolve data and features
     datasets = resolve_datasets(features, data, frequency)
@@ -161,10 +179,22 @@ def fit_minimize(
             evaluator = TargetLoss(predictor=dataset.predictor, target=dataset.target, loss=loss)
         else:
             if likelihood is not None:
-                mll = MarginalLogLikelihood(predictor=dataset.predictor, observed=dataset.target, likelihood=likelihood, discrepancy=discrepancy)
+                mll = MarginalLogLikelihood(
+                    predictor=dataset.predictor, observed=dataset.target, likelihood=likelihood,
+                    discrepancy=discrepancy,
+                    use_orthogonal_discrepancy=use_orthogonal_discrepancy,
+                    orthogonal_rcond=orthogonal_rcond,
+                )
             else:
                 temperature = temperature if temperature is not None else 1.0
-                mll = GibbsMarginalLogLikelihood(predictor=dataset.predictor, observed=dataset.target, loss=loss, discrepancy=discrepancy, temperature=temperature)
+                mll = GibbsMarginalLogLikelihood(
+                    predictor=dataset.predictor, observed=dataset.target, loss=loss,
+                    discrepancy=discrepancy, temperature=temperature,
+                    use_orthogonal_discrepancy=use_orthogonal_discrepancy,
+                    orthogonal_rcond=orthogonal_rcond,
+                )
+            if use_orthogonal_discrepancy:
+                mll = mll.with_orthogonal_reference(model, dataset.frequency)
             evaluator = Negated(mll)
         objective.append((evaluator, dataset.frequency))
 

{paramrf-0.37.4 → paramrf-0.37.5}/pmrf/objectives/__init__.py

@@ -31,6 +31,7 @@ if TYPE_CHECKING:
         SummedTerms as SummedTerms,
         problem_terms as problem_terms,
     )
+    from .shared import SharedMarginalLogLikelihood as SharedMarginalLogLikelihood
     from .terms import (
         AbstractTerm as AbstractTerm,
         BoundEvaluator as BoundEvaluator,
@@ -59,6 +60,7 @@ _EXPORTS = {
     'Negated': 'evaluators',
     'PriorPenalized': 'problems',
     'RMSELoss': 'losses',
+    'SharedMarginalLogLikelihood': 'shared',
     'SummedTerms': 'problems',
     'TargetLoss': 'evaluators',
     'TermFn': 'terms',
@@ -66,7 +68,7 @@ _EXPORTS = {
     'as_terms': 'terms',
     'problem_terms': 'problems',
 }
-_MODULES = {'evaluators', 'losses', 'problems', 'terms'}
+_MODULES = {'evaluators', 'losses', 'problems', 'shared', 'terms'}
 
 def __getattr__(name):
     if name in _MODULES: