makepst 0.3.0__tar.gz → 0.5.0__tar.gz

makepst-0.5.0/CHANGELOG.md

@@ -0,0 +1,143 @@
+# Changelog
+
+## 0.5.0 (2026-09-29)
+
+- `tools/rule_sources.py`: are `validate`'s rules still current? `pestpp` re-reads PEST++'s option
+  parser from GitHub (names, aliases, types, choices, deprecations); `pestchek --source DIR` reads a
+  PEST download's `pestchek.F` / `cheksub.F` (control-data keywords and every message). Each is
+  compared with the reviewed snapshot in `tools/snapshots/` and with makePst's own tables; exit 1 on
+  a change, 2 when the source cannot be read; `--write` records a reviewed source. Tests hold
+  `makepst/pestpp.py` and the control-data schema to the snapshots, and a weekly GitHub workflow
+  (`.github/workflows/rule-sources.yml`) opens an issue when PEST++'s parser changes. The PEST++
+  registry is unchanged; its provenance now names commit 02e1455 (5.2.29) - the commit named before,
+  ad8428d, has identical option parsing but is labelled 5.2.28 in the source.
+
+- `build --dry_run`: the `validate` report on the control file a build would write, from a workbook's
+  BUILD sheet or from tables named on the command line; nothing is written, no manifest; exit 1 on
+  errors. A workbook with no BUILD sheet now says how to name its tables and check them this way.
+- `validate` on a workbook judges it as `build` would write it, after the same normalization
+  (regularisation equations from `PRIOR`/`WEIGHT`, unused groups removed, ties to fixed parameters
+  fixed), listing those as notes. It used to judge the tables before that and report errors building
+  removes: a regularisation file with no "regul" group, prior information not generated.
+
+- PEST_HP control variables are treated as `++` options are: a control-data token makePst does not
+  know (`newhpvar=7`, a new flag) is kept with the line it came from and written back there, instead
+  of being dropped, and `validate` warns about it with the closest known name. `dump` writes it to
+  the CONTROL sheet with its line (`control data line 4`), so `build` puts it back.
+- Every PEST_HP variable PEST 17.2's pestchek reads is now known: `run_slow_fac`, `run_abandon_fac`,
+  `jcowarnthresh`, `jcozerothresh`, `zerosenval`, `hardstophours`, `softstophours`, `reg2measrat`,
+  `orr_not_first`, and `obsreref_N`; `name = value` with spaces around `=` is read, as PEST_HP does.
+- `validate` checks that PEST can read each control value: an integer or a number where one is
+  expected (Fortran `1d-3` included), a known word for `precis`, `dpoint` and the other flags,
+  `absparmax(n)=value` with n 1-10; and the PEST_HP ranges and combinations (`RUN_ABANDON_FAC` 0 or
+  at least 1.2, not both stop-hours variables, `JCOZEROTHRESH` above `JCOWARNTHRESH`, `REG2MEASRAT`
+  below 1, `orr_not_first` and `ZEROSENVAL` only with what they need).
+- The rule sources are pinned and reported: `PESTCHEK_SOURCE_VERSION` (17.2) beside
+  `PESTPP_REGISTRY_VERSION` (5.2.29); `validate` prints both and every manifest records them under
+  `checked_against`.
+
+## 0.4.0 (2026-09-27)
+
+- `rescale` / `Pst.rescale_par()`: move parameter values into `SCALE` so each selected parameter
+  starts at 1 (bounds divided by the value), or back with `--undo`; the model sees the same values.
+  Selection by `--par` globs and `--pargp` (fixed parameters included unless `--exclude_fixed`;
+  tied children follow their parent). Prior-information equations are rewritten exactly (every
+  residual is unchanged), and settings in parameter units that cannot be converted (absolute
+  derivative increments, `absolute(n)` limits, `++parcov` / `++base_jacobian` / `++ies_par_en`)
+  are reported.
+- `fill_parval` (`build --fill_parval`, `parrep`), `tempchek --par` and `hpstart --par` convert a
+  `.par` value written under a different `SCALE` / `OFFSET` than the control file's through the
+  model value, instead of copying `PARVAL1` across. The test fixture's `demo.par` now records the
+  parameters' own offsets, as PEST writes them.
+- Writing a control file no longer fails on an adjustable `PARVAL1` outside its bounds (a `parrep`
+  from an IES realization, say): it is a warning, as documented, and the file is written.
+  `makepst validate` still reports it as an error. `diff` normalizes a workbook side, as `build`
+  would, so a workbook compares equal to the control file built from it.
+
+- `reweight`: balance observation-group contributions equally or to relative target shares,
+  multiply selected group weights directly, match weights to the current misfit with
+  `--discrepancy group|obs` (group phi = weighted-observation count, or `w = min(w, 1/|r|)`), consume `.res`/IES observation results, enforce
+  optional weight bounds with `--clip`, and write an auditable before/after report.
+
+- `++` options are checked against PEST++ 5.2.29's option list (`makepst/pestpp.py`): a known option
+  with a value PEST++ would not accept, or the same option twice (also through an alias), is an error;
+  an unknown option is a warning with the closest known name (`did you mean "ies_num_reals"?`) and is
+  kept as written. Findings name the workbook cell, csv line or control-file line the option came from
+  (`PP!A17`). `validate` reports them; `build`, `parrep` and `set` print them but still write.
+
+- Unknown control-file sections are retained verbatim, including their original header,
+  body lines and position, instead of being dropped during a read/write round trip.
+- `Pst.validate()` now returns a read-only report; `apply_fixes(report)` and `normalize()` make
+  the previous tied-parameter, unused-group and prior-information cleanups explicit.
+
+## 0.3.0 (2026-09-21)
+
+- `log`: ledger of every manifest under a folder — when, which command, which output from
+  which sources, dimensions, version; `--file` filters by file name, path or sha256 prefix
+  ("which runs used that version of the workbook?"), `--command`, `--last`, `--check`
+  verifies every recorded hash
+- `bundle`: zip a run for reproduction or review — control file, manifest, templates,
+  instruction files, model inputs, command-line files; `--sources`, `--outputs`, `--extra`,
+  `--list`, `--strict`; the zip gets its own manifest
+- `provenance`: compare the hashes recorded in a manifest with the files on disk
+  (`unchanged` / `changed` / `missing`; exit 1 unless all unchanged)
+- `update --dry_run`: preview matched names, target columns and skipped cells without writing
+- `build` warns when a referenced sheet has formulas without cached values (a workbook saved
+  by openpyxl and not yet recalculated in Excel)
+- `hpstart`: a PEST_HP `.hp` file with observation values from residuals or an IES realization
+
+## 0.2.0 (2026-09-19)
+
+- `update`: the xlwings backend reads only headers, name and target columns, writes contiguous
+  cells in one call per run, suspends events and calculation while writing and recalculates
+  once before saving (1,068 parameters over eight sheets of a 27-sheet workbook: ~9 s, was
+  minutes); `--par` with `--pst` no longer rewrites the observation sheets
+- `tempchek`: check a template file or write model input files from parameter values, with
+  PEST's number writer (maximum precision in the space, `PRECIS` / `DPOINT`, one word per
+  parameter, right-justified; checked against `tempchek.exe`); for a template + `.par` file
+  or for a whole case
+- `inschek`: check an instruction file or read a model output file with it into an `.obf`
+  file, for one file or a whole case
+- `validate`: values that cannot be written into their template space are errors; a line
+  advance not at the start of an instruction line and `t` / fixed instructions moving left
+  are errors (INSCHEK's rules); three more pestchek warnings (empty observation group,
+  `ICOV`/`ICOR`/`IEIG` with more than 300 adjustable parameters, PEST_HP-only variables —
+  the last as a note); an all-zero-weight observation group is now a note, not a warning
+- `update`: cells inside array formulas or dynamic-array spill ranges are left alone (Excel
+  refuses them; xlwings no longer shows a dialog); names present on only one side are reported
+  with counts and examples
+- `validate`: pestchek's rules added (parameter data, groups, observations, prior information,
+  ~60 control-variable ranges and consistency rules, template and instruction syntax), derived
+  from PEST 17's `pestchek.F` / `cheksub.F`; file paths resolved from the directory PEST runs
+  in, not only the control file's folder
+
+## 0.1.0 (2026-09-19)
+
+First packaged release; replaces the single-file `makePst.py` script.
+
+- `init`: starter workbook with headers, defaults, descriptions, drop-down lists, comments and
+  a `BUILD` sheet
+- `build`: Excel/CSV tables -> control file; sheet globs (`book,PAR_*`); a workbook with a
+  `BUILD` sheet as the only argument; counts computed; PEST_HP keyed tokens and
+  `++` options; regularisation equations from `PRIOR`/`WEIGHT` columns; validation of tied
+  targets, groups, prior references and bounds
+- `dump`: control file -> workbook with a `BUILD` sheet that rebuilds it
+- `update`: `.par` / `.res` / `.rei` / PESTPP-IES ensembles into an existing workbook, matched
+  by name across sheets; formula cells protected; `--sheet` / `--group` filters; `--real`;
+  `PHI` sheet (objective function by observation group) and `PHI_IES` (realization phis)
+  whenever residuals are available
+- `validate`: pestchek-style report on a control file or workbook: table checks, missing
+  files, parameter / observation names against template and instruction files, optional run
+  of instruction files against model outputs; exit 1 on errors
+- PEST++ version-2 control files (`pcf version=2`, keyword control data, external csv
+  tables) are read and written (`--v2`); observation covariance files are kept
+- `diff`: semantic comparison of two control files / workbooks (tables, effective control
+  values, options, io, comments), text report or workbook; exit 1 on differences
+- pyEMU bridge: `to_pyemu()` / `from_pyemu()` via a temporary control file (`pyemu` extra)
+- provenance: every producing command writes `<output>.manifest.json` (version, command
+  line, source hashes and sheets, output hash and counts)
+- `parrep`: `.par` or IES realization into a control file (or a workbook with a `BUILD` sheet),
+  with `--set NAME=VALUE`
+- Fixes over the old script: `jacupdate` was never written; values were truncated to 6
+  significant digits; `phistopthresh` could not be set; SVD / regularisation values in the
+  CONTROL sheet were ignored; prior information in estimation mode produced an invalid file

{makepst-0.3.0 → makepst-0.5.0}/CITATION.cff

@@ -6,8 +6,8 @@ authors:
   - family-names: Ou
     given-names: Michael
     email: ougengxin@gmail.com
-version: 0.3.0
-date-released: 2026-09-21
+version: 0.5.0
+date-released: 2026-09-29
 repository-code: "https://github.com/ougx/makePst"
 license: MIT
 keywords:

{makepst-0.3.0 → makepst-0.5.0}/MANIFEST.in

@@ -1,5 +1,6 @@
 include LICENSE README.md CHANGELOG.md CITATION.cff
 recursive-include tests *.py *.xlsx *.pst *.par *.res *.csv
+recursive-include tools *.py *.json
 recursive-include examples/minimal *.csv *.py *.tpl *.ins *.in *.out *.par *.res
 prune examples/mnnrd
 global-exclude __pycache__ *.py[cod] *.manifest.json

{makepst-0.3.0 → makepst-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: makepst
-Version: 0.3.0
+Version: 0.5.0
 Summary: Build PEST / PEST++ / PEST_HP control files from spreadsheet tables, read them back, and push results into the workbook
 Author-email: Michael Ou <ougengxin@gmail.com>
 License: MIT
@@ -45,8 +45,10 @@ reads control files back into editable workbooks, and returns calibration result
 Excel / CSV tables                     ->  .pst                   makepst build
 .pst                                   ->  editable workbook      makepst dump
 .par / .res / .rei / IES ensemble      ->  existing workbook      makepst update
-    .par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
-    .res/.rei or IES obs + existing .hp     ->  new .hp                makepst hpstart
+.par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+.pst                                   ->  values moved to SCALE  makepst rescale
+.res/.rei or IES obs + group targets   ->  reweighted .pst        makepst reweight
+.res/.rei or IES obs + existing .hp    ->  new .hp                makepst hpstart
 two .pst / workbooks                   ->  what changed           makepst diff
 manifests in a folder                  ->  run ledger             makepst log
 .pst + everything it references        ->  zip for review         makepst bundle
@@ -158,7 +160,7 @@ without `--force`.
 
 ```
 makepst validate model.pst --outputs
-makepst validate book.xlsm --strict
+makepst validate book.xlsx --strict
 ```
 
 See [Validation](#validation). Template and instruction paths are resolved the way PEST
@@ -167,27 +169,43 @@ the current directory and the control file's parent, uses the one where the file
 says which; `--base_dir` overrides that. `--strict` also fails on warnings,
 `--quiet` hides the informational lines.
 
+A workbook is accepted in place of a control file when it carries a `BUILD` sheet (one that
+`dump` or `init` wrote), which is built first and then checked. For tables that have no
+`BUILD` sheet — a hand-made workbook, or a set of CSV files — name them on a
+[`build`](#build--tables--pst) command line and add `--dry_run`, which runs this same report
+and writes nothing.
+
 ### build — tables → .pst
 
 ```
 makepst build out.pst [estimation|regularisation|prediction|pareto]
-    --set_ctl_xls book.xlsm,CONTROL          (or --set_ctl_csv file.csv)
-    --add_pargp_xls book.xlsm,PARGP
-    --add_par_xls   book.xlsm,PAR_*          (repeatable; sheet names may be globs)
-    --add_obs_xls   book.xlsm,OBS_*
-    --add_io_xls    book.xlsm,IO  --add_pp_xls book.xlsm,PPcntl
+    --set_ctl_xls book.xlsx,CONTROL          (or --set_ctl_csv file.csv)
+    --add_pargp_xls book.xlsx,PARGP
+    --add_par_xls   book.xlsx,PAR_*          (repeatable; sheet names may be globs)
+    --add_obs_xls   book.xlsx,OBS_*
+    --add_io_xls    book.xlsx,IO  --add_pp_xls book.xlsx,PPcntl
     [--add_tied_xls ...] [--add_prior_xls ...] [--add_comment "text"] [--add_comment_xls ...]
     [--fill_parval run.par [--real NAME]] [--ss] [--no_dump_tpl] [--v2]
 ```
 
 Each `--add_*` takes `book,SHEET` (`_xls`) or a csv path (`_csv`) and may be repeated;
 tables split across sheets are concatenated. The sheet name may be a glob:
-`--add_par_xls tr13.xlsm,PAR_*` adds every `PAR_…` sheet in workbook order. Headers are
-case-insensitive. `makepst build book.xlsm [--out x.pst]` runs the command stored in the
+`--add_par_xls tr13.xlsx,PAR_*` adds every `PAR_…` sheet in workbook order. Headers are
+case-insensitive. `makepst build book.xlsx [--out x.pst]` runs the command stored in the
 workbook's `BUILD` sheet instead. The mode argument
 overrides `pestmode` in the CONTROL sheet; adding prior information switches to regularisation.
 `--ss` drops `ss*`/`sy*` parameters and groups (steady-state runs of a transient setup).
 
+`--dry_run` builds the control file in memory, runs the full [validation](#validation) report
+on it — including the template, instruction and model files it references — and writes
+nothing (no `.pst`, no `dump.tpl`, no manifest), exiting 1 when there are errors. This is how
+to check a workbook or a set of CSV tables that has no `BUILD` sheet; for one that has a
+`BUILD` sheet, `makepst validate book.xlsx` does the same in one argument.
+
+```bash
+makepst build case.pst --add_par_xls book.xlsx,PAR_* --add_obs_xls book.xlsx,OBS_* ... --dry_run
+```
+
 | table   | columns |
 |---------|---------|
 | CONTROL | `NAME`, `VALUE` in the first four columns (`LINE`, `NAME`, `DEFAULT`, `VALUE`); blank values use the built-in default. Any variable of the control data, SVD, LSQR, AUI, SVD-assist or regularisation sections: `noptmax`, `jacupdate`, `lamforgive`, `win_mrun_hours`, `absparmax(1)=0.1 absparmax(2)=20`, `svdmode`, `phimlim`, … Counts are ignored (computed). |
@@ -200,6 +218,15 @@ overrides `pestmode` in the CONTROL sheet; adding prior information switches to
 | IO      | `TYPE` (`cmd` / `tpl` / `ins`), `IN`, `OUT` |
 | PP      | first two columns: PEST++ option name, value → `++name(value)` |
 
+Classic PEST numeric control settings are positional. If a later setting is supplied,
+all earlier numeric settings on that line must have values after built-in defaults
+are applied. For example, `upvecbend` requires `iboundstick`, and `regsingthresh`
+requires both `noptregadj` and `regweightrat`. Export raises an error naming any
+missing predecessors instead of shifting settings into the wrong slots. Supply
+those values explicitly; trailing optional fields may remain blank. Text flags
+and named tokens such as `absparmax(1)=0.1` do not require numeric fillers. This
+positional requirement does not apply to the version-2 keyword format (`--v2`).
+
 **Regularisation from the parameter table.** In regularisation mode each adjustable
 parameter with a `PRIOR` and a positive `WEIGHT` gets an equation: a numeric `PRIOR` gives
 `1.0 * log(p) = log10(v)` (or `1.0 * p = v` for untransformed parameters); a `PRIOR` naming
@@ -221,13 +248,13 @@ to recover a workbook that was lost.
 ### update — results → existing workbook
 
 ```
-makepst update book.xlsm --par run.par                          # PARVAL1
-makepst update book.xlsm --res run.res                          # adds MODELLED / RESIDUAL
-makepst update book.xlsm --pst run.pst --par_cols PARVAL1,PARTRANS,PARLBND,PARUBND
-makepst update book.xlsm --par run.3.par.csv --real best        # PESTPP-IES ensemble
-makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated values
-makepst update book.xlsm --par run.par --dry_run               # preview; write nothing
-    [--sheet "PAR_*"] [--group hk,sy] [--overwrite_formulas] [--out copy.xlsm] [--backend openpyxl|xlwings]
+makepst update book.xlsx --par run.par                          # PARVAL1
+makepst update book.xlsx --res run.res                          # adds MODELLED / RESIDUAL
+makepst update book.xlsx --pst run.pst --par_cols PARVAL1,PARTRANS,PARLBND,PARUBND
+makepst update book.xlsx --par run.3.par.csv --real best        # PESTPP-IES ensemble
+makepst update book.xlsx --obs_csv run.3.obs.csv --pst run.pst  # IES simulated values
+makepst update book.xlsx --par run.par --dry_run               # preview; write nothing
+    [--sheet "PAR_*"] [--group hk,sy] [--overwrite_formulas] [--out copy.xlsx] [--backend openpyxl|xlwings]
 ```
 
 > **`update` saves in place unless `--out` is given.** Use `--out` the first time, and keep
@@ -244,6 +271,8 @@ refuse), only counted. `--sheet` (glob, case-insensitive) restricts the workshee
 add `--pst` to supply them). `--pst` on its own writes the control file's parameter and
 observation columns; next to `--par`, `--res` or `--obs_csv` it only supplies groups and
 measured values, and the observation sheets are not touched by a parameter update.
+Likewise, `--res` or `--obs_csv` with `--pst` leaves parameter cells unchanged unless
+`--par` is also supplied.
 
 The xlwings backend reads only each sheet's header, name column and target columns, writes
 contiguous runs of cells in one call each and recalculates once before saving: the
@@ -272,18 +301,67 @@ unmatched names without saving a workbook or manifest. It does not test whether
 refuse individual writes. During `build`, missing cached formula results in referenced
 tables produce a warning; existing cached values may still be stale and cannot be verified.
 
+### reweight — balance observation-group weights
+
+```
+makepst reweight run.pst balanced.pst --res run.res --equal
+makepst reweight run.pst balanced.pst --res run.res --targets group_shares.csv
+makepst reweight run.pst scaled.pst --factor heads=0.5 --factor flow=2
+makepst reweight run.pst balanced.pst --obs_csv run.3.obs.csv --real base --equal
+makepst reweight run.pst adjusted.pst --res run.res --discrepancy group
+makepst reweight run.pst adjusted.pst --obs_csv run.0.obs.csv --discrepancy obs
+```
+
+`reweight` writes a new control file; it never overwrites the input. `--equal` gives each
+eligible measurement group the same initial objective-function contribution; a group whose
+current phi is zero cannot be scaled to a share, so it is skipped with an INFO line and its
+weights are left unchanged. `--targets`
+reads a CSV or `BOOK,SHEET` table with `OBGNME,TARGET_SHARE` columns. Shares are relative, so
+`4,3,2` means 44.44%, 33.33% and 22.22%; the selected groups' total phi is preserved. Direct
+`--factor GROUP=NUMBER` specifications multiply the existing positive weights in named groups.
+
+`--discrepancy` makes weights consistent with the current misfit, so that each weighted
+observation contributes about 1 to phi. `--discrepancy group` scales each measurement group by
+one factor so its phi equals its number of weighted observations (as PEST's PWTADJ2); relative
+weights inside a group are kept and zero-phi groups are skipped. It raises the weights of groups
+that already fit better than that, so pair it with `--clip` when some groups fit very closely.
+`--discrepancy obs` sets `w = min(w, 1/|residual|)` for each observation (pyEMU's
+`adjust_weights_discrepancy` with its original ceiling): observations the model cannot fit are
+lowered to a contribution of 1, weights are never raised, and zero residuals keep their weight.
+It is commonly applied to the prior mean or base realization before a PESTPP-IES run so that
+observation noise is not smaller than the misfit the prior can reach. Total phi is not preserved
+in either variant.
+
+For equal, target and discrepancy modes, residuals are joined by observation name while weights and group
+membership come from the input control file. Groups backed by covariance matrices, `predict`,
+and `regul*` groups are not measurement groups and are rejected or skipped as appropriate.
+
+Use `--clip MIN MAX` to constrain final positive weights, for example `--clip 0 10`. Original
+zero weights remain zero. Clipping is included in the balancing calculation; if a requested
+target cannot be reached under the bounds, the command fails instead of reporting a misleading
+result. `--report FILE` writes the before/after group table, including phi, percentages,
+factors, clipping counts and the number of weights lowered (`N_REDUCED`). `--dry_run` prints that table without writing a control file,
+report or manifest.
+
+The workbook remains the source of truth. If the control file is rebuilt from a workbook later,
+formula-generated workbook weights will replace weights changed by `reweight` unless the change
+is deliberately transferred back to the workbook.
+
 ### parrep — .par values → new .pst
 
 ```
 makepst parrep run.pst run.par next.pst [--set noptmax=0] [--set NAME=VALUE ...] [--v1|--v2]
 makepst parrep run.pst run.3.par.csv next.pst --real best
-makepst parrep book.xlsm run.par next.pst                       # base is the workbook
+makepst parrep book.xlsx run.par next.pst                       # base is the workbook
 ```
 
 Like PEST's PARREP: `PARVAL1` is replaced from a `.par` file or an IES realization and a new
 control file is written; `--set` changes control values on the way. All supported content of
 the input control file is preserved semantically (see below); adjustable parameters that end
-up outside their bounds are reported.
+up outside their bounds are reported. A `.par` file records the `SCALE` and `OFFSET` each value
+was written with; where those differ from the control file's (a run from before a
+[`rescale`](#rescale--parameter-values--scale)), the value is converted so the model sees the
+same number (`PARVAL1 × SCALE + OFFSET`). An IES ensemble carries no scales and is copied as is.
 
 With a **workbook** as the base, the control file is first built from the command in the
 workbook's `BUILD` sheet (`dump` writes one; for a hand-made workbook paste the build command
@@ -292,6 +370,37 @@ to that workbook; other relative names resolve against the workbook's folder. No
 route rebuilds from the workbook's *current* state, so any edits made since the original
 control file was written are picked up too.
 
+### rescale — parameter values → SCALE
+
+```
+makepst rescale run.pst run_unit.pst                            # every parameter starts at 1
+makepst rescale run.pst run_unit.pst --par "hk*,sy*" --exclude_fixed
+makepst rescale run.pst run_unit.pst --pargp kh --pargp kv
+makepst rescale run_unit.pst run_back.pst --undo                # SCALE folded back into PARVAL1
+```
+
+PEST applies `SCALE` and `OFFSET` only when it writes a model input file, so a parameter at
+value *v* can start at 1 without the model seeing any difference: `PARVAL1 = 1`,
+`SCALE = SCALE × v`, bounds divided by *v* (swapped when *v* < 0), `OFFSET` unchanged. `--undo`
+does the reverse (`PARVAL1 × SCALE`, bounds likewise, `SCALE = 1`).
+
+- **Selection:** all parameters by default, fixed ones included; `--par` (names or globs) and
+  `--pargp` narrow it, and both must match when both are given; `--exclude_fixed` leaves fixed
+  parameters alone. Tied children follow a selected parent. Parameters at zero are skipped and
+  listed.
+- **Prior information** is written in PEST's own units, so every equation naming a rescaled
+  parameter is rewritten exactly: a linear term takes *v* into its coefficient, a log term
+  moves `log10(v)` to the right-hand side. Each equation's residual is the same number as
+  before, so its weight is unchanged.
+- **Warned, not converted:** settings in parameter units that one factor per parameter can't
+  convert — `INCTYP absolute` and `DERINCLB` of groups holding rescaled parameters, `absolute(n)`
+  change limits — and files named in the control file that were written for the old values
+  (`++parcov`, `++base_jacobian`, `++ies_par_en`, SVD-assist `basepestfile` / `basejacfile`).
+  A Jacobian given to PEST another way (a restart, a command-line switch) can't be seen from
+  the control file; don't reuse one across a rescale.
+
+`--set`, `--eol`, `--v1|--v2` and `--no_manifest` work as for `parrep`.
+
 ### hpstart — modeled observations → new PEST_HP .hp file
 
 ```
@@ -350,8 +459,8 @@ read and observations the control file lists but no file produced are reported.
 ### diff — what changed
 
 ```
-makepst diff tr12.pst tr13.pst [--xlsx changes.xlsx] [--rtol 1e-9] [--max_rows 50]
-makepst diff tr13.xlsm tr13.pst                # is the control file still what the workbook says?
+makepst diff tr12.pst tr13.pst [--xlsx changes.xlsx] [--file changes.txt] [--rtol 1e-9] [--max_rows 50]
+makepst diff tr13.xlsx tr13.pst                # is the control file still what the workbook says?
 ```
 
 Compares the tables, not the text: parameters and observations added, removed or changed
@@ -361,11 +470,12 @@ instruction pairs, command lines and header comments. Numbers are compared with
 tolerance, so `49.7377` vs `49.73775` shows up at the default `1e-9` and disappears at
 `--rtol 1e-5`. Prints a table per section and a one-line summary; `--xlsx` writes the same
 tables to a workbook for review. Exits 1 when there are differences, like `diff`.
+`--file` writes the complete text report, including rows beyond the console's `--max_rows` limit.
 
 ### log — the run ledger
 
 ```
-makepst log [FOLDER ...] [--file tr13.xlsm] [--command build] [--last 10] [--check] [--no_recursive]
+makepst log [FOLDER ...] [--file tr13.xlsx] [--command build] [--last 10] [--check] [--no_recursive]
 ```
 
 Every manifest under the folders (default: the current one, recursively), oldest first, one
@@ -376,8 +486,8 @@ the workbook?" even after the file was renamed or overwritten. `--check` adds a
 `provenance`: `unchanged` / `changed` / `missing`.
 
 ```
-2026-09-19 21:14 build      pest/tr13.pst <- tr13.xlsm  [1068 par 41747 obs 159 prior]  makepst 0.3.0
-2026-09-21 08:10 update     tr13-xlwings.xlsm <- tr13.xlsm, tr13.par  [1068 rows in 8 sheets]  makepst 0.3.0
+2026-09-19 21:14 build      pest/tr13.pst <- tr13.xlsx  [1068 par 41747 obs 159 prior]  makepst 0.5.0
+2026-09-21 08:10 update     tr13-xlwings.xlsx <- tr13.xlsx, tr13.par  [1068 rows in 8 sheets]  makepst 0.5.0
 ```
 
 ### bundle — a run, zipped for review
@@ -405,11 +515,13 @@ Preserved: every parameter, group, observation, prior-information equation, tied
 template/instruction pair, command line, `++` option, header comment, control-data value
 (including PEST_HP keyed tokens), SVD / LSQR / AUI / SVD-assist / regularisation values, and
 the verbatim text of `* sensitivity reuse`, `* derivatives command line`, `* predictive
-analysis` and `* pareto`.
+analysis` and `* pareto`. Any section unknown to this version of makePst is also retained
+verbatim at its original position, so newer or site-specific PEST content survives a round trip.
 
 Normalised: parameter, observation and group names are lower-cased (PEST is
 case-insensitive); numbers are written with 11 significant digits; whitespace and column
-alignment are makePst's own; comments *inside* sections are dropped; observation groups are
+alignment in understood sections is makePst's own; comments *inside understood sections* are
+dropped; observation groups are
 listed in first-use order (or the order of the source file when reading a `.pst`).
 
 Also preserved: covariance-file references in `* observation groups` (an `OBSGP` sheet on
@@ -419,27 +531,27 @@ written back as version 2 unless `--v1` is given; `--v2` writes any control file
 format, with `case.par_data.csv` / `obs_data` / `pargp_data` / `prior_data` beside it. Both
 versions of the same content are identical after reading (`makepst diff` says so).
 
-Not preserved: any unrecognised section (dropped with a warning naming it).
-
 ## Compatibility
 
 | | |
 |---|---|
 | PEST dialects | PEST, PEST_HP (`win_mrun_hours=`, `uptestmin=`, `uptestlim=`, `absparmax(n)=`), PEST++ (`++name(value)` options) |
-| Control-file sections | control data, singular value decomposition, lsqr, automatic user intervention, svd assist, parameter groups, parameter data (incl. tied pairs), observation groups, observation data, model command line, model input/output, prior information (with `&` continuation lines), regularisation. Kept verbatim: sensitivity reuse, derivatives command line, predictive analysis, pareto. |
+| Control-file sections | control data, singular value decomposition, lsqr, automatic user intervention, svd assist, parameter groups, parameter data (incl. tied pairs), observation groups, observation data, model command line, model input/output, prior information (with `&` continuation lines), regularisation. Kept verbatim: sensitivity reuse, derivatives command line, predictive analysis, pareto, and all unknown sections. |
 | File formats | classic (version 1) and PEST++ version 2 (`pcf version=2`, `* control data keyword` with control variables and `++` options, `* … external` csv tables with `sep=` / `missing_values=`, `partied` column); observation covariance files in `* observation groups` |
-| Not supported | unknown sections — dropped with a warning |
 | Result files | `.par`, `.res` / `.rei`, PESTPP-IES `case.N.par.csv` / `case.N.obs.csv` / `case.phi.actual.csv` |
-| Table inputs | `.xlsx` / `.xlsm` sheets (openpyxl), `.csv` |
+| Table inputs | `.xlsx` / `.xlsx` sheets (openpyxl), `.csv` |
 | Workbook update | openpyxl backend on any platform (macros kept, charts/images dropped, no recalculation); xlwings backend on Windows/macOS with Excel (everything kept, recalculated) |
 | Platforms | CI runs the suite on Ubuntu 24.04 and Windows Server 2022 for Python 3.9, 3.11 and 3.13. macOS is expected to work but is not exercised in CI. |
 | Python | ≥ 3.9; pandas ≥ 2.0 (tested with 2.3 and 3.0), numpy, openpyxl ≥ 3.1; optional xlwings, pyemu, pytest |
 
 ## Validation
 
-Two layers. `Pst.validate()` runs before every write (`build`, `dump`, `parrep`, `write_pst`)
-and enforces **internal consistency** of the tables, changing what it can and refusing what it
-cannot:
+`Pst.validate()` is a read-only audit of the in-memory tables. It returns a report with
+`errors`, `warnings`, and `fixes_available`; inspecting it never changes the `Pst`. Apply
+reviewed fixes explicitly with `pst.apply_fixes(report)`, or use `pst.normalize()` to apply
+all currently available normalizations. Writers refuse validation errors and serialize the
+current tables without modifying them. The CLI's file-building commands call `normalize()`
+explicitly before writing.
 
 | check | outcome |
 |---|---|
@@ -447,19 +559,20 @@ cannot:
 | duplicate parameter, observation or prior-information names | error |
 | parameter group used but not defined | error |
 | tied parameter whose target is missing, or tied to a tied parameter | error |
-| tied to a fixed parameter | parameter becomes fixed (message) |
-| parameter group defined but unused | dropped from the file |
-| prior equation referencing a fixed, tied or missing parameter | dropped (message) |
+| tied to a fixed parameter | error and fix available: convert it to fixed |
+| parameter group defined but unused | fix available: remove the group |
+| prior equation referencing a fixed, tied or missing parameter | error and fix available: rebuild prior information, dropping those equations |
 | adjustable `PARVAL1` outside `[PARLBND, PARUBND]` | warning |
-| control value that is computed (`npar` …) or unknown | ignored (warning) |
+| control value that is computed (`npar` …) | ignored (warning) |
+| control variable makePst does not know (a newer PEST_HP keyword): kept on the line it came from; a CONTROL-sheet row gives that line in `LINE` (`control data line 4`) or is ignored | warning |
 
-`makepst validate` is the **pestchek-style report**: it changes nothing, looks beyond the
-tables at the files the control file points to, and exits 1 on errors so a batch file can
-stop:
+`makepst validate` uses the same table checks and adds the **pestchek-style report**: it
+changes nothing, looks beyond the tables at the files the control file points to, and exits 1
+on errors so a batch file can stop:
 
 ```
 makepst validate model.pst [--outputs] [--strict] [--quiet] [--base_dir DIR]
-makepst validate tr13.xlsm            # a workbook with a BUILD sheet is built first, in memory
+makepst validate tr13.xlsx            # a workbook with a BUILD sheet is built first, in memory
 ```
 
 | check | severity |
@@ -470,19 +583,47 @@ makepst validate tr13.xlsm            # a workbook with a BUILD sheet is built f
 | malformed prior equation; non-numeric right-hand side; `log()` used on a non-log parameter or vice versa | error |
 | `DERCOM` beyond the number of model command lines | error |
 | name longer than PEST's limit (12 parameter / 20 observation / 12 group; PEST++ allows 200) | warning |
-| **pestchek's rules** (from PEST 17's `pestchek.F` / `cheksub.F`): `PARCHGLIM` vocabulary and `absolute(n)` needing `absparmax(n)=`; log parameters factor-limited with positive values; factor-limited bounds of one sign and non-zero; `SCALE ≠ 0`; tied to itself, tied or parent with initial value 0; group `none` reserved; all parameters fixed/tied; `INCTYP` / `FORCEN` / `DERMTHD` vocabularies and their compatibility; split-column ranges; duplicate groups; `dum` as an observation name; prior weights, labels (≤ 20, not an observation name), duplicated parameters, all-zero factors, group `predict`; regularisation needs a `regul*` group, prediction exactly one `predict` observation; ~60 control-variable range and consistency rules (`RLAMFAC`, `PHIRATSUF`, `FACPARMAX`, `NPHISTP`, `RELPARSTP < RELPARMAX`, `EIGTHRESH`, `PHIMLIM < PHIMACCEPT < 1.2·PHIMLIM`, `WFMIN ≤ WFINIT ≤ WFMAX`, `UPTESTMIN`/`UPTESTLIM`, SVD vs AUI vs LSQR exclusivity, …) | error |
+| **pestchek's rules** (from PEST 17.2's `pestchek.F` / `cheksub.F`): `PARCHGLIM` vocabulary and `absolute(n)` needing `absparmax(n)=`; log parameters factor-limited with positive values; factor-limited bounds of one sign and non-zero; `SCALE ≠ 0`; tied to itself, tied or parent with initial value 0; group `none` reserved; all parameters fixed/tied; `INCTYP` / `FORCEN` / `DERMTHD` vocabularies and their compatibility; split-column ranges; duplicate groups; `dum` as an observation name; prior weights, labels (≤ 20, not an observation name), duplicated parameters, all-zero factors, group `predict`; regularisation needs a `regul*` group, prediction exactly one `predict` observation; ~60 control-variable range and consistency rules (`RLAMFAC`, `PHIRATSUF`, `FACPARMAX`, `NPHISTP`, `RELPARSTP < RELPARMAX`, `EIGTHRESH`, `PHIMLIM < PHIMACCEPT < 1.2·PHIMLIM`, `WFMIN ≤ WFINIT ≤ WFMAX`, `UPTESTMIN`/`UPTESTLIM`, SVD vs AUI vs LSQR exclusivity, …) | error |
+| control value PEST cannot read: not an integer or a number where one is expected (Fortran `1d-3` is fine), a word outside its field's vocabulary (`precis`, `dpoint`, `obsreref`, …), `absparmax(n)=` malformed or n outside 1-10; the PEST_HP limits and combinations (`RUN_SLOW_FAC`, `RUN_ABANDON_FAC`, `HARDSTOPHOURS` / `SOFTSTOPHOURS` not both, `JCOZEROTHRESH` above `JCOWARNTHRESH`, `ZEROSENVAL`, `REG2MEASRAT` below 1, `orr_not_first` only with `obsreref`) | error |
+| control-data token not in PEST 17.2's pestchek — kept as written, with the closest known name (`"win_mrun_hour=2" … did you mean "win_mrun_hours"?`) | warning |
 | pestchek's warnings: derivative increment larger than a third of the range; initial value 0 with a relative increment; group with only fixed/tied parameters; log parameters with a non-relative increment; observation group listed but empty (dropped on write); `MAXSING` above the adjustable-parameter count; more than 300 adjustable parameters with `ICOV`/`ICOR`/`IEIG` on; `NOPTMAX 0` | warning |
 | PEST_HP-only control variables present (`WIN_MRUN_HOURS`, `UPTESTMIN`, `RRFSAVE`, …; plain PEST needs `/hpstart`); observation group whose weights are all zero | note |
 | template: parameter space narrower than 3 characters or containing a tab | error |
 | template: a value that cannot be written into its space as PEST writes numbers (`PRECIS` / `DPOINT` honoured), e.g. `1e-10` in five characters | error |
 | instruction file syntax: `l` / `t` integers, marker delimiters, `!` balance, `[name]n1:n2` order, `dum` in a fixed field, continuation `&` placement, `l` only at the start of a line, `t` and fixed columns moving left to right, unknown instructions — checked statically, without model output | error |
-| section in the `.pst` that makePst drops on read | warning |
+| `++` option with a value PEST++ does not accept (integer given `2.5`, boolean given `t` or `yes`, number list with a non-number, `svd_pack` / `glm_normal_form` / `ies_subset_how` outside their choices, `ies_num_reals` < 1, …); the same option twice, directly or through an alias (`ies_par_en` and `ies_parameter_ensemble`) | error |
+| `++` option not in PEST++'s option list — kept, with the closest known name (`unknown PEST++ option "ies_num_real"; did you mean "ies_num_reals"?`); a control variable such as `noptmax` given as a `++` option | warning (note with `++forgive_unknown_args(true)`) |
+| deprecated `++` option that PEST++ ignores (`mat_inv`, `upgrade_augment`, `svd_pack(propack)`, …) | note |
+| unknown section in the `.pst` that makePst preserves verbatim | note |
 | template or instruction file missing; bad `ptf` / `pif` line | error |
 | parameter cited in no template; template citing an unknown parameter | error |
 | observation read by no instruction file, by two files, or twice in one; unknown observation in an instruction file | error |
 | model command / model-input folder not found next to the control file | warning |
 | `--outputs`: each instruction file run against its model output file when present | note (success) / warning (failure) |
 
+The `++` checks use the option list in `makepst/pestpp.py`, read off PEST++ 5.2.29's option parser
+(250 options and 34 aliases, with their types). They point at where each option came from —
+`PP!A17` for a workbook cell, `pp.csv:17`, `line 42` of a control file — and `build` prints them
+too, without refusing to write: plain PEST ignores `++` lines, and a newer PEST++ may know an
+option this list does not. PEST++ itself stops on an option it does not accept, so fix or remove
+what is flagged before a run.
+
+`validate` prints the two releases its rules were read from, and every manifest records them
+(`checked_against`). Whether they are still current is checked by `tools/rule_sources.py`, which
+re-reads the upstream source and compares it with the snapshot the rules were written against
+(`tools/snapshots/`), exiting 1 on any change:
+
+```
+python tools/rule_sources.py pestpp                        # usgs/pestpp develop, from GitHub
+python tools/rule_sources.py pestchek --source pest_source  # the source folder of a PEST download
+```
+
+It reports added and removed `++` options, aliases, value types and choices; for PEST, the
+control-data keywords pestchek reads and its messages (each message is a rule); and anything
+makePst's own tables no longer agree with. It edits nothing: port what changed, then run it
+again with `--write` to record the new source as reviewed, and the tests hold the code to that
+snapshot. The PEST++ check runs weekly on GitHub and opens an issue when the parser changes.
+
 The `--outputs` interpreter follows the PEST manual (primary and secondary markers, `l`, `w`,
 `t`, `!name!`, `[name]c1:c2`, `(name)c1:c2`, `dum`, `&` continuation) and catches the classic
 mistakes — a `w` too few so a label is read instead of a number, a marker that never appears
@@ -501,14 +642,15 @@ from?" after the fact:
 
 ```json
 {
-  "makepst": "0.3.0",
+  "makepst": "0.5.0",
   "command": "build",
-  "argv": ["build", "tr13.pst", "regul", "--set_ctl_xls", "tr13.xlsm,CONTROL", "..."],
+  "argv": ["build", "tr13.pst", "regul", "--set_ctl_xls", "tr13.xlsx,CONTROL", "..."],
   "created": "2026-09-19T13:51:39-04:00",
   "user": "hydro", "host": "OU13700", "cwd": "D:\\...\\0023-makePst",
   "python": "3.12.12", "pandas": "3.0.5", "platform": "Windows-11-10.0.26200-SP0",
+  "checked_against": {"pestchek": "17.2", "pestpp": "5.2.29"},
   "sources": [
-    {"path": "D:\\...\\tr13.xlsm", "sha256": "9d641a2f...", "size": 3089317,
+    {"path": "D:\\...\\tr13.xlsx", "sha256": "9d641a2f...", "size": 3089317,
      "modified": "2026-07-12T03:32:10-04:00", "role": "control",
      "sheets": ["CONTROL", "PARGP", "PAR_HK", "PAR_VK", "...", "PPglm"]}
   ],
@@ -568,10 +710,11 @@ from makepst import Pst, read_pst, write_pst, to_workbook, update_workbook, load
 pst = read_pst('run.pst')                    # Pst: .par/.obs/.pargp/.prior DataFrames, .control dict, ...
 pst.fill_parval('run.3.par.csv', real='best')
 pst.set_control({'noptmax': 0})
+pst.rescale_par(patterns=['hk*'], include_fixed=False)   # values -> SCALE; returns a report dict
 write_pst(pst, 'run_final.pst', dump_tpl=False)
 
 to_workbook(pst, 'run.xlsx', split=True)
-update_workbook('book.xlsm', par='run.par', res='run.res', out='book_results.xlsm',
+update_workbook('book.xlsx', par='run.par', res='run.res', out='book_results.xlsx',
                 sheets=['PAR_*'], groups=['hk'])
 ```
 
@@ -603,9 +746,10 @@ makepst/
   writer.py         Pst -> .pst text (+ dump.tpl)
   reader.py         .pst text -> Pst
   excel.py          sheets -> Pst; Pst -> workbook; results -> existing workbook
-  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek / log / bundle
+  cli.py            init / validate / build / dump / update / parrep / set / rescale / reweight / diff / tempchek / inschek / log / bundle
   diff.py           semantic comparison of two Pst objects
   phi.py            objective function by group from residuals; IES realization phis
+  reweight.py       observation-group balancing, scaling and bounded weight adjustment
   pyemu_bridge.py   to_pyemu / from_pyemu
   checks.py         the validate report: table checks, template / instruction cross-checks, instruction interpreter
   rules.py          pestchek's rules (parameters, groups, observations, prior information, control variables)
@@ -614,6 +758,7 @@ makepst/
   starter.py        the `init` workbook: control-variable descriptions, example rows, drop-downs
 examples/minimal/   the tutorial inputs
 tests/              suite + fixture generator
+tools/              rule_sources.py (are the rules current with PEST / PEST++?) and its snapshots
 ```
 
 ## Releasing
@@ -623,7 +768,7 @@ stored anywhere. To release: bump the version in `pyproject.toml` and the fallba
 `makepst/__init__.py` (a test keeps them equal), add a `CHANGELOG.md` entry, commit, then
 
 ```
-git tag v0.3.0 && git push origin main v0.3.0
+git tag v0.5.0 && git push origin main v0.5.0
 ```
 
 The `publish` workflow checks that the tag matches the package version, runs the tests, builds