makepst 0.1.0__tar.gz → 0.3.0__tar.gz

makepst-0.3.0/CHANGELOG.md

@@ -0,0 +1,73 @@
+# Changelog
+
+## 0.3.0 (2026-09-21)
+
+- `log`: ledger of every manifest under a folder — when, which command, which output from
+  which sources, dimensions, version; `--file` filters by file name, path or sha256 prefix
+  ("which runs used that version of the workbook?"), `--command`, `--last`, `--check`
+  verifies every recorded hash
+- `bundle`: zip a run for reproduction or review — control file, manifest, templates,
+  instruction files, model inputs, command-line files; `--sources`, `--outputs`, `--extra`,
+  `--list`, `--strict`; the zip gets its own manifest
+- `provenance`: compare the hashes recorded in a manifest with the files on disk
+  (`unchanged` / `changed` / `missing`; exit 1 unless all unchanged)
+- `update --dry_run`: preview matched names, target columns and skipped cells without writing
+- `build` warns when a referenced sheet has formulas without cached values (a workbook saved
+  by openpyxl and not yet recalculated in Excel)
+- `hpstart`: a PEST_HP `.hp` file with observation values from residuals or an IES realization
+
+## 0.2.0 (2026-09-19)
+
+- `update`: the xlwings backend reads only headers, name and target columns, writes contiguous
+  cells in one call per run, suspends events and calculation while writing and recalculates
+  once before saving (1,068 parameters over eight sheets of a 27-sheet workbook: ~9 s, was
+  minutes); `--par` with `--pst` no longer rewrites the observation sheets
+- `tempchek`: check a template file or write model input files from parameter values, with
+  PEST's number writer (maximum precision in the space, `PRECIS` / `DPOINT`, one word per
+  parameter, right-justified; checked against `tempchek.exe`); for a template + `.par` file
+  or for a whole case
+- `inschek`: check an instruction file or read a model output file with it into an `.obf`
+  file, for one file or a whole case
+- `validate`: values that cannot be written into their template space are errors; a line
+  advance not at the start of an instruction line and `t` / fixed instructions moving left
+  are errors (INSCHEK's rules); three more pestchek warnings (empty observation group,
+  `ICOV`/`ICOR`/`IEIG` with more than 300 adjustable parameters, PEST_HP-only variables —
+  the last as a note); an all-zero-weight observation group is now a note, not a warning
+- `update`: cells inside array formulas or dynamic-array spill ranges are left alone (Excel
+  refuses them; xlwings no longer shows a dialog); names present on only one side are reported
+  with counts and examples
+- `validate`: pestchek's rules added (parameter data, groups, observations, prior information,
+  ~60 control-variable ranges and consistency rules, template and instruction syntax), derived
+  from PEST 17's `pestchek.F` / `cheksub.F`; file paths resolved from the directory PEST runs
+  in, not only the control file's folder
+
+## 0.1.0 (2026-09-19)
+
+First packaged release; replaces the single-file `makePst.py` script.
+
+- `init`: starter workbook with headers, defaults, descriptions, drop-down lists, comments and
+  a `BUILD` sheet
+- `build`: Excel/CSV tables -> control file; sheet globs (`book,PAR_*`); a workbook with a
+  `BUILD` sheet as the only argument; counts computed; PEST_HP keyed tokens and
+  `++` options; regularisation equations from `PRIOR`/`WEIGHT` columns; validation of tied
+  targets, groups, prior references and bounds
+- `dump`: control file -> workbook with a `BUILD` sheet that rebuilds it
+- `update`: `.par` / `.res` / `.rei` / PESTPP-IES ensembles into an existing workbook, matched
+  by name across sheets; formula cells protected; `--sheet` / `--group` filters; `--real`;
+  `PHI` sheet (objective function by observation group) and `PHI_IES` (realization phis)
+  whenever residuals are available
+- `validate`: pestchek-style report on a control file or workbook: table checks, missing
+  files, parameter / observation names against template and instruction files, optional run
+  of instruction files against model outputs; exit 1 on errors
+- PEST++ version-2 control files (`pcf version=2`, keyword control data, external csv
+  tables) are read and written (`--v2`); observation covariance files are kept
+- `diff`: semantic comparison of two control files / workbooks (tables, effective control
+  values, options, io, comments), text report or workbook; exit 1 on differences
+- pyEMU bridge: `to_pyemu()` / `from_pyemu()` via a temporary control file (`pyemu` extra)
+- provenance: every producing command writes `<output>.manifest.json` (version, command
+  line, source hashes and sheets, output hash and counts)
+- `parrep`: `.par` or IES realization into a control file (or a workbook with a `BUILD` sheet),
+  with `--set NAME=VALUE`
+- Fixes over the old script: `jacupdate` was never written; values were truncated to 6
+  significant digits; `phistopthresh` could not be set; SVD / regularisation values in the
+  CONTROL sheet were ignored; prior information in estimation mode produced an invalid file

{makepst-0.1.0 → makepst-0.3.0}/CITATION.cff

@@ -6,8 +6,8 @@ authors:
   - family-names: Ou
     given-names: Michael
     email: ougengxin@gmail.com
-version: 0.1.0
-date-released: 2026-09-19
+version: 0.3.0
+date-released: 2026-09-21
 repository-code: "https://github.com/ougx/makePst"
 license: MIT
 keywords:

{makepst-0.1.0 → makepst-0.3.0}/MANIFEST.in

@@ -1,4 +1,5 @@
 include LICENSE README.md CHANGELOG.md CITATION.cff
 recursive-include tests *.py *.xlsx *.pst *.par *.res *.csv
-recursive-include examples *.csv *.py *.tpl *.ins *.in *.out *.par *.res
+recursive-include examples/minimal *.csv *.py *.tpl *.ins *.in *.out *.par *.res
+prune examples/mnnrd
 global-exclude __pycache__ *.py[cod] *.manifest.json

{makepst-0.1.0 → makepst-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: makepst
-Version: 0.1.0
+Version: 0.3.0
 Summary: Build PEST / PEST++ / PEST_HP control files from spreadsheet tables, read them back, and push results into the workbook
 Author-email: Michael Ou <ougengxin@gmail.com>
 License: MIT
@@ -45,8 +45,11 @@ reads control files back into editable workbooks, and returns calibration result
 Excel / CSV tables                     ->  .pst                   makepst build
 .pst                                   ->  editable workbook      makepst dump
 .par / .res / .rei / IES ensemble      ->  existing workbook      makepst update
-.par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .res/.rei or IES obs + existing .hp     ->  new .hp                makepst hpstart
 two .pst / workbooks                   ->  what changed           makepst diff
+manifests in a folder                  ->  run ledger             makepst log
+.pst + everything it references        ->  zip for review         makepst bundle
 ```
 
 See [What round-tripping preserves](#what-round-tripping-preserves) for the exact guarantees
@@ -120,6 +123,10 @@ makepst build model.pst regul --set_ctl_csv control.csv --add_pargp_csv pargp.cs
 # 2. check it against the template / instruction / output files, like pestchek
 makepst validate model.pst --outputs
 
+# 2b. what PEST will do with them: write the model input file, read the model output files
+makepst tempchek model.pst --dry_run
+makepst inschek model.pst
+
 # 3. control file -> workbook (CONTROL / PARGP / PAR / OBS / PRIOR / IO sheets + a BUILD sheet)
 makepst dump model.pst model.xlsx
 
@@ -154,8 +161,10 @@ makepst validate model.pst --outputs
 makepst validate book.xlsm --strict
 ```
 
-See [Validation](#validation). Template and instruction paths are resolved relative to the
-control file (or workbook); `--base_dir` overrides that. `--strict` also fails on warnings,
+See [Validation](#validation). Template and instruction paths are resolved the way PEST
+resolves them, from the directory PEST is run in: makePst tries the control file's folder,
+the current directory and the control file's parent, uses the one where the files are, and
+says which; `--base_dir` overrides that. `--strict` also fails on warnings,
 `--quiet` hides the informational lines.
 
 ### build — tables → .pst
@@ -217,6 +226,7 @@ makepst update book.xlsm --res run.res                          # adds MODELLED
 makepst update book.xlsm --pst run.pst --par_cols PARVAL1,PARTRANS,PARLBND,PARUBND
 makepst update book.xlsm --par run.3.par.csv --real best        # PESTPP-IES ensemble
 makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated values
+makepst update book.xlsm --par run.par --dry_run               # preview; write nothing
     [--sheet "PAR_*"] [--group hk,sy] [--overwrite_formulas] [--out copy.xlsm] [--backend openpyxl|xlwings]
 ```
 
@@ -225,10 +235,19 @@ makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated
 
 Every worksheet whose header row contains `PARNME` (or `OBSNME`) is matched row by row on the
 name, so tables spread over several sheets are all found; only the requested columns are
-written and only for names present in the source. Cells that hold formulas are left alone
-unless `--overwrite_formulas`. `--sheet` (glob, case-insensitive) restricts the worksheets;
+written and only for names present in both: names in the results with no row in the workbook,
+and workbook rows absent from the results, are left alone and reported as a warning with
+counts and examples. Cells that hold formulas are left alone unless `--overwrite_formulas`;
+cells inside an array formula or a dynamic-array spill range are never written (Excel would
+refuse), only counted. `--sheet` (glob, case-insensitive) restricts the worksheets;
 `--group` restricts the parameter / observation groups (a `.par` file carries no groups, so
-add `--pst` to supply them).
+add `--pst` to supply them). `--pst` on its own writes the control file's parameter and
+observation columns; next to `--par`, `--res` or `--obs_csv` it only supplies groups and
+measured values, and the observation sheets are not touched by a parameter update.
+
+The xlwings backend reads only each sheet's header, name column and target columns, writes
+contiguous runs of cells in one call each and recalculates once before saving: the
+demonstration workbook (27 sheets, 1,068 parameters over eight sheets) updates in about 9 s.
 
 PESTPP-IES ensembles (`case.N.par.csv`, `case.N.obs.csv`) are recognised by their
 `real_name` header. `--real` picks the realization: a name (`base`, `17`) or `best` — the
@@ -245,8 +264,13 @@ Both sheets are rewritten on each run (`--no_phi` to skip).
 
 Two backends: **xlwings** (used when installed; drives Excel invisibly, so everything in the
 workbook is preserved and formulas recalculate; Windows/macOS with Excel) or **openpyxl**
-(no Excel needed; keeps VBA macros; drops charts and images; formula results are stale until
-Excel next opens the file). The backend used is printed.
+(no Excel needed; keeps VBA macros; drops charts and images; does not recalculate formulas).
+Before building from a workbook updated with openpyxl, recalculate and save it in Excel;
+otherwise the build may read missing or stale cached formula values. The backend used is printed.
+`--dry_run` uses openpyxl to preview matched names, target columns, skipped formulas and
+unmatched names without saving a workbook or manifest. It does not test whether Excel will
+refuse individual writes. During `build`, missing cached formula results in referenced
+tables produce a warning; existing cached values may still be stale and cannot be verified.
 
 ### parrep — .par values → new .pst
 
@@ -268,6 +292,61 @@ to that workbook; other relative names resolve against the workbook's folder. No
 route rebuilds from the workbook's *current* state, so any edits made since the original
 control file was written are picked up too.
 
+### hpstart — modeled observations → new PEST_HP .hp file
+
+```
+makepst hpstart case.pst start.hp --template case.hp --res case.res
+makepst hpstart case.pst start.hp --template case.hp --obs_csv case.3.obs.csv --real best
+makepst hpstart case.pst start.hp --template case.hp --res case.rei --par case.par
+```
+
+By default, parameter values are copied unchanged from the template `.hp`, **not** from
+`PARVAL1` in the `.pst`. Use `--par` to replace them with values from a PEST `.par` file
+or a selected IES parameter realization. The `.pst` supplies parameter and observation
+names and order; the template supplies PEST_HP instruction metadata that a residual or
+IES observation file cannot provide. `hpstart` replaces the template's reference
+modeled-observation array in `.pst` observation order. Every required observation must
+be present with a finite value; extra result rows are ignored. The template must come
+from the same control-file layout and
+observation/parameter order. The binary format stores counts but not names, so matching
+counts alone cannot establish that the template has the right order. The original `.hp`
+is never modified, and the new file receives a provenance manifest by default.
+
+### tempchek — templates → model input files
+
+```
+makepst tempchek model.tpl                          # check the template, list its parameters
+makepst tempchek model.tpl model.in [model.par]     # write model.in (values from model.par by default)
+makepst tempchek model.pst [--par run.par] [--real best] [--dry_run]   # every template of the case
+```
+
+Like PEST's TEMPCHEK: a template is checked, then its model input file is written with
+parameter values from a `.par` file (or, given a control file / workbook, from `PARVAL1`
+with `SCALE` and `OFFSET` applied, or from `--par` — a `.par` file or an IES realization).
+Numbers are written the way PEST writes them (its `WRTSIG`): into the parameter space with
+as much precision as the space allows, fixed or exponential notation, right-justified, one
+word per parameter sized for its narrowest space, `PRECIS` / `DPOINT` honoured
+(`double`, `nopoint`). Checked against `tempchek.exe` on a grid of values and widths from 3
+to 23 characters: the same fields are refused, and the written value is never further from
+the true value than PEST's. A value that cannot be written (`1e-10` in a five-character
+space, say) is an error and nothing is written; parameters in the `.par` file that the
+template does not cite are warnings. `--dry_run` checks without writing.
+
+### inschek — model output files → observation values
+
+```
+makepst inschek heads.ins                           # check the instruction file, list its observations
+makepst inschek heads.ins heads.out [--out heads.obf]
+makepst inschek model.pst [--out model.obf]          # every instruction file of the case, one .obf
+```
+
+Like PEST's INSCHEK: the instruction file is checked (the syntax rules in the validation
+table below, including that a line advance can only start an instruction line and that fixed
+/ tab instructions must move right), then read against the model output file with the
+interpreter that `validate --outputs` uses, and the values go to an observation value file
+(`name  value` per line, INSCHEK's `.obf`). With a control file, all instruction files are
+read and observations the control file lists but no file produced are reported.
+
 ### diff — what changed
 
 ```
@@ -283,6 +362,39 @@ tolerance, so `49.7377` vs `49.73775` shows up at the default `1e-9` and disappe
 `--rtol 1e-5`. Prints a table per section and a one-line summary; `--xlsx` writes the same
 tables to a workbook for review. Exits 1 when there are differences, like `diff`.
 
+### log — the run ledger
+
+```
+makepst log [FOLDER ...] [--file tr13.xlsm] [--command build] [--last 10] [--check] [--no_recursive]
+```
+
+Every manifest under the folders (default: the current one, recursively), oldest first, one
+line each: when, which command, which output, from which sources, the dimensions, the
+makePst version. `--file` keeps the entries whose output or sources include a file — by name,
+path or **sha256 prefix**, so `--file b43fc509` answers "which runs used *that* version of
+the workbook?" even after the file was renamed or overwritten. `--check` adds a column from
+`provenance`: `unchanged` / `changed` / `missing`.
+
+```
+2026-09-19 21:14 build      pest/tr13.pst <- tr13.xlsm  [1068 par 41747 obs 159 prior]  makepst 0.3.0
+2026-09-21 08:10 update     tr13-xlwings.xlsm <- tr13.xlsm, tr13.par  [1068 rows in 8 sheets]  makepst 0.3.0
+```
+
+### bundle — a run, zipped for review
+
+```
+makepst bundle model.pst [run.zip] [--sources] [--outputs] [--extra "bin/**"] [--list] [--strict]
+```
+
+Zips the control file with its manifest, every template and instruction file, the model
+input files the templates write, and any file named on the model command line, with paths
+relative to the run directory so the archive unzips runnable. `--sources` adds the manifest's
+sources (the workbook, the `.par` ...), `--outputs` the model output files, `--extra` any
+glob relative to the run directory (the model binaries, say). Files the control file
+references but that don't exist are listed as `MISSING` (exit 1 with `--strict`); `--list`
+shows what would be bundled without writing. The zip gets its own manifest, so `log` shows
+who bundled what and when.
+
 ## What round-tripping preserves
 
 `dump` → `build` and `read_pst` → `write_pst` are **semantically lossless for supported
@@ -358,6 +470,12 @@ makepst validate tr13.xlsm            # a workbook with a BUILD sheet is built f
 | malformed prior equation; non-numeric right-hand side; `log()` used on a non-log parameter or vice versa | error |
 | `DERCOM` beyond the number of model command lines | error |
 | name longer than PEST's limit (12 parameter / 20 observation / 12 group; PEST++ allows 200) | warning |
+| **pestchek's rules** (from PEST 17's `pestchek.F` / `cheksub.F`): `PARCHGLIM` vocabulary and `absolute(n)` needing `absparmax(n)=`; log parameters factor-limited with positive values; factor-limited bounds of one sign and non-zero; `SCALE ≠ 0`; tied to itself, tied or parent with initial value 0; group `none` reserved; all parameters fixed/tied; `INCTYP` / `FORCEN` / `DERMTHD` vocabularies and their compatibility; split-column ranges; duplicate groups; `dum` as an observation name; prior weights, labels (≤ 20, not an observation name), duplicated parameters, all-zero factors, group `predict`; regularisation needs a `regul*` group, prediction exactly one `predict` observation; ~60 control-variable range and consistency rules (`RLAMFAC`, `PHIRATSUF`, `FACPARMAX`, `NPHISTP`, `RELPARSTP < RELPARMAX`, `EIGTHRESH`, `PHIMLIM < PHIMACCEPT < 1.2·PHIMLIM`, `WFMIN ≤ WFINIT ≤ WFMAX`, `UPTESTMIN`/`UPTESTLIM`, SVD vs AUI vs LSQR exclusivity, …) | error |
+| pestchek's warnings: derivative increment larger than a third of the range; initial value 0 with a relative increment; group with only fixed/tied parameters; log parameters with a non-relative increment; observation group listed but empty (dropped on write); `MAXSING` above the adjustable-parameter count; more than 300 adjustable parameters with `ICOV`/`ICOR`/`IEIG` on; `NOPTMAX 0` | warning |
+| PEST_HP-only control variables present (`WIN_MRUN_HOURS`, `UPTESTMIN`, `RRFSAVE`, …; plain PEST needs `/hpstart`); observation group whose weights are all zero | note |
+| template: parameter space narrower than 3 characters or containing a tab | error |
+| template: a value that cannot be written into its space as PEST writes numbers (`PRECIS` / `DPOINT` honoured), e.g. `1e-10` in five characters | error |
+| instruction file syntax: `l` / `t` integers, marker delimiters, `!` balance, `[name]n1:n2` order, `dum` in a fixed field, continuation `&` placement, `l` only at the start of a line, `t` and fixed columns moving left to right, unknown instructions — checked statically, without model output | error |
 | section in the `.pst` that makePst drops on read | warning |
 | template or instruction file missing; bad `ptf` / `pif` line | error |
 | parameter cited in no template; template citing an unknown parameter | error |
@@ -383,7 +501,7 @@ from?" after the fact:
 
 ```json
 {
-  "makepst": "0.1.0",
+  "makepst": "0.3.0",
   "command": "build",
   "argv": ["build", "tr13.pst", "regul", "--set_ctl_xls", "tr13.xlsm,CONTROL", "..."],
   "created": "2026-09-19T13:51:39-04:00",
@@ -395,7 +513,7 @@ from?" after the fact:
      "sheets": ["CONTROL", "PARGP", "PAR_HK", "PAR_VK", "...", "PPglm"]}
   ],
   "output": {"path": "D:\\...\\tr13.pst", "sha256": "d93f317d...", "size": 2163331,
-             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 185, "nobsgp": 12,
+             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 159, "nobsgp": 12,
              "ntplfle": 18, "ninsfle": 7, "pestmode": "regularisation"}
 }
 ```
@@ -407,6 +525,15 @@ realization and `--set` values (and, from a workbook, the resolved build argumen
 left alone, and the backend. A hash mismatch between a manifest and the workbook on disk is
 the signal that the control file no longer corresponds to the spreadsheet.
 
+Run `makepst provenance model.pst` (or pass `model.pst.manifest.json`) to compare every
+recorded input and output hash with the files at their recorded paths. It reports
+`unchanged`, `changed`, `missing`, or `not recorded` and exits with status 1 unless all
+records are unchanged. This checks file identity, not the validity of PEST tables; use
+`makepst validate` for the latter. Moving files requires updating their recorded paths.
+`makepst log` lists the manifests of a whole project folder as a ledger (with `--check`,
+each one verified), and `makepst bundle` zips a run with everything its control file
+references — see the two commands above.
+
 ## Relationship to pyEMU
 
 [pyEMU](https://github.com/pypest/pyemu) is the broader toolkit: it constructs PEST++
@@ -454,7 +581,7 @@ update_workbook('book.xlsm', par='run.par', res='run.res', out='book_results.xls
 python -m pytest tests -q
 ```
 
-95 tests, no external data needed: a synthetic workbook in the real project layout
+106 tests, no external data needed: a synthetic workbook in the real project layout
 (`tests/data/demo.xlsx`, generated by `tests/make_fixture.py`) with a golden control file,
 `.par`, `.res` and IES ensemble files. They cover golden-file builds from Excel and CSV, the
 `read → write → read` identity, `dump → build` byte equality, workbook updates (multi-sheet
@@ -476,12 +603,14 @@ makepst/
   writer.py         Pst -> .pst text (+ dump.tpl)
   reader.py         .pst text -> Pst
   excel.py          sheets -> Pst; Pst -> workbook; results -> existing workbook
-  cli.py            init / validate / build / dump / update / parrep / diff
+  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek / log / bundle
   diff.py           semantic comparison of two Pst objects
   phi.py            objective function by group from residuals; IES realization phis
   pyemu_bridge.py   to_pyemu / from_pyemu
   checks.py         the validate report: table checks, template / instruction cross-checks, instruction interpreter
-  provenance.py     the <output>.manifest.json sidecar
+  rules.py          pestchek's rules (parameters, groups, observations, prior information, control variables)
+  model_files.py    PEST's number writer, template filling, .obf files (tempchek / inschek)
+  provenance.py     the <output>.manifest.json sidecar; provenance / log / bundle
   starter.py        the `init` workbook: control-variable descriptions, example rows, drop-downs
 examples/minimal/   the tutorial inputs
 tests/              suite + fixture generator
@@ -494,7 +623,7 @@ stored anywhere. To release: bump the version in `pyproject.toml` and the fallba
 `makepst/__init__.py` (a test keeps them equal), add a `CHANGELOG.md` entry, commit, then
 
 ```
-git tag v0.2.0 && git push origin main v0.2.0
+git tag v0.3.0 && git push origin main v0.3.0
 ```
 
 The `publish` workflow checks that the tag matches the package version, runs the tests, builds

{makepst-0.1.0 → makepst-0.3.0}/README.md

@@ -14,8 +14,11 @@ reads control files back into editable workbooks, and returns calibration result
 Excel / CSV tables                     ->  .pst                   makepst build
 .pst                                   ->  editable workbook      makepst dump
 .par / .res / .rei / IES ensemble      ->  existing workbook      makepst update
-.par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .res/.rei or IES obs + existing .hp     ->  new .hp                makepst hpstart
 two .pst / workbooks                   ->  what changed           makepst diff
+manifests in a folder                  ->  run ledger             makepst log
+.pst + everything it references        ->  zip for review         makepst bundle
 ```
 
 See [What round-tripping preserves](#what-round-tripping-preserves) for the exact guarantees
@@ -89,6 +92,10 @@ makepst build model.pst regul --set_ctl_csv control.csv --add_pargp_csv pargp.cs
 # 2. check it against the template / instruction / output files, like pestchek
 makepst validate model.pst --outputs
 
+# 2b. what PEST will do with them: write the model input file, read the model output files
+makepst tempchek model.pst --dry_run
+makepst inschek model.pst
+
 # 3. control file -> workbook (CONTROL / PARGP / PAR / OBS / PRIOR / IO sheets + a BUILD sheet)
 makepst dump model.pst model.xlsx
 
@@ -123,8 +130,10 @@ makepst validate model.pst --outputs
 makepst validate book.xlsm --strict
 ```
 
-See [Validation](#validation). Template and instruction paths are resolved relative to the
-control file (or workbook); `--base_dir` overrides that. `--strict` also fails on warnings,
+See [Validation](#validation). Template and instruction paths are resolved the way PEST
+resolves them, from the directory PEST is run in: makePst tries the control file's folder,
+the current directory and the control file's parent, uses the one where the files are, and
+says which; `--base_dir` overrides that. `--strict` also fails on warnings,
 `--quiet` hides the informational lines.
 
 ### build — tables → .pst
@@ -186,6 +195,7 @@ makepst update book.xlsm --res run.res                          # adds MODELLED
 makepst update book.xlsm --pst run.pst --par_cols PARVAL1,PARTRANS,PARLBND,PARUBND
 makepst update book.xlsm --par run.3.par.csv --real best        # PESTPP-IES ensemble
 makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated values
+makepst update book.xlsm --par run.par --dry_run               # preview; write nothing
     [--sheet "PAR_*"] [--group hk,sy] [--overwrite_formulas] [--out copy.xlsm] [--backend openpyxl|xlwings]
 ```
 
@@ -194,10 +204,19 @@ makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated
 
 Every worksheet whose header row contains `PARNME` (or `OBSNME`) is matched row by row on the
 name, so tables spread over several sheets are all found; only the requested columns are
-written and only for names present in the source. Cells that hold formulas are left alone
-unless `--overwrite_formulas`. `--sheet` (glob, case-insensitive) restricts the worksheets;
+written and only for names present in both: names in the results with no row in the workbook,
+and workbook rows absent from the results, are left alone and reported as a warning with
+counts and examples. Cells that hold formulas are left alone unless `--overwrite_formulas`;
+cells inside an array formula or a dynamic-array spill range are never written (Excel would
+refuse), only counted. `--sheet` (glob, case-insensitive) restricts the worksheets;
 `--group` restricts the parameter / observation groups (a `.par` file carries no groups, so
-add `--pst` to supply them).
+add `--pst` to supply them). `--pst` on its own writes the control file's parameter and
+observation columns; next to `--par`, `--res` or `--obs_csv` it only supplies groups and
+measured values, and the observation sheets are not touched by a parameter update.
+
+The xlwings backend reads only each sheet's header, name column and target columns, writes
+contiguous runs of cells in one call each and recalculates once before saving: the
+demonstration workbook (27 sheets, 1,068 parameters over eight sheets) updates in about 9 s.
 
 PESTPP-IES ensembles (`case.N.par.csv`, `case.N.obs.csv`) are recognised by their
 `real_name` header. `--real` picks the realization: a name (`base`, `17`) or `best` — the
@@ -214,8 +233,13 @@ Both sheets are rewritten on each run (`--no_phi` to skip).
 
 Two backends: **xlwings** (used when installed; drives Excel invisibly, so everything in the
 workbook is preserved and formulas recalculate; Windows/macOS with Excel) or **openpyxl**
-(no Excel needed; keeps VBA macros; drops charts and images; formula results are stale until
-Excel next opens the file). The backend used is printed.
+(no Excel needed; keeps VBA macros; drops charts and images; does not recalculate formulas).
+Before building from a workbook updated with openpyxl, recalculate and save it in Excel;
+otherwise the build may read missing or stale cached formula values. The backend used is printed.
+`--dry_run` uses openpyxl to preview matched names, target columns, skipped formulas and
+unmatched names without saving a workbook or manifest. It does not test whether Excel will
+refuse individual writes. During `build`, missing cached formula results in referenced
+tables produce a warning; existing cached values may still be stale and cannot be verified.
 
 ### parrep — .par values → new .pst
 
@@ -237,6 +261,61 @@ to that workbook; other relative names resolve against the workbook's folder. No
 route rebuilds from the workbook's *current* state, so any edits made since the original
 control file was written are picked up too.
 
+### hpstart — modeled observations → new PEST_HP .hp file
+
+```
+makepst hpstart case.pst start.hp --template case.hp --res case.res
+makepst hpstart case.pst start.hp --template case.hp --obs_csv case.3.obs.csv --real best
+makepst hpstart case.pst start.hp --template case.hp --res case.rei --par case.par
+```
+
+By default, parameter values are copied unchanged from the template `.hp`, **not** from
+`PARVAL1` in the `.pst`. Use `--par` to replace them with values from a PEST `.par` file
+or a selected IES parameter realization. The `.pst` supplies parameter and observation
+names and order; the template supplies PEST_HP instruction metadata that a residual or
+IES observation file cannot provide. `hpstart` replaces the template's reference
+modeled-observation array in `.pst` observation order. Every required observation must
+be present with a finite value; extra result rows are ignored. The template must come
+from the same control-file layout and
+observation/parameter order. The binary format stores counts but not names, so matching
+counts alone cannot establish that the template has the right order. The original `.hp`
+is never modified, and the new file receives a provenance manifest by default.
+
+### tempchek — templates → model input files
+
+```
+makepst tempchek model.tpl                          # check the template, list its parameters
+makepst tempchek model.tpl model.in [model.par]     # write model.in (values from model.par by default)
+makepst tempchek model.pst [--par run.par] [--real best] [--dry_run]   # every template of the case
+```
+
+Like PEST's TEMPCHEK: a template is checked, then its model input file is written with
+parameter values from a `.par` file (or, given a control file / workbook, from `PARVAL1`
+with `SCALE` and `OFFSET` applied, or from `--par` — a `.par` file or an IES realization).
+Numbers are written the way PEST writes them (its `WRTSIG`): into the parameter space with
+as much precision as the space allows, fixed or exponential notation, right-justified, one
+word per parameter sized for its narrowest space, `PRECIS` / `DPOINT` honoured
+(`double`, `nopoint`). Checked against `tempchek.exe` on a grid of values and widths from 3
+to 23 characters: the same fields are refused, and the written value is never further from
+the true value than PEST's. A value that cannot be written (`1e-10` in a five-character
+space, say) is an error and nothing is written; parameters in the `.par` file that the
+template does not cite are warnings. `--dry_run` checks without writing.
+
+### inschek — model output files → observation values
+
+```
+makepst inschek heads.ins                           # check the instruction file, list its observations
+makepst inschek heads.ins heads.out [--out heads.obf]
+makepst inschek model.pst [--out model.obf]          # every instruction file of the case, one .obf
+```
+
+Like PEST's INSCHEK: the instruction file is checked (the syntax rules in the validation
+table below, including that a line advance can only start an instruction line and that fixed
+/ tab instructions must move right), then read against the model output file with the
+interpreter that `validate --outputs` uses, and the values go to an observation value file
+(`name  value` per line, INSCHEK's `.obf`). With a control file, all instruction files are
+read and observations the control file lists but no file produced are reported.
+
 ### diff — what changed
 
 ```
@@ -252,6 +331,39 @@ tolerance, so `49.7377` vs `49.73775` shows up at the default `1e-9` and disappe
 `--rtol 1e-5`. Prints a table per section and a one-line summary; `--xlsx` writes the same
 tables to a workbook for review. Exits 1 when there are differences, like `diff`.
 
+### log — the run ledger
+
+```
+makepst log [FOLDER ...] [--file tr13.xlsm] [--command build] [--last 10] [--check] [--no_recursive]
+```
+
+Every manifest under the folders (default: the current one, recursively), oldest first, one
+line each: when, which command, which output, from which sources, the dimensions, the
+makePst version. `--file` keeps the entries whose output or sources include a file — by name,
+path or **sha256 prefix**, so `--file b43fc509` answers "which runs used *that* version of
+the workbook?" even after the file was renamed or overwritten. `--check` adds a column from
+`provenance`: `unchanged` / `changed` / `missing`.
+
+```
+2026-09-19 21:14 build      pest/tr13.pst <- tr13.xlsm  [1068 par 41747 obs 159 prior]  makepst 0.3.0
+2026-09-21 08:10 update     tr13-xlwings.xlsm <- tr13.xlsm, tr13.par  [1068 rows in 8 sheets]  makepst 0.3.0
+```
+
+### bundle — a run, zipped for review
+
+```
+makepst bundle model.pst [run.zip] [--sources] [--outputs] [--extra "bin/**"] [--list] [--strict]
+```
+
+Zips the control file with its manifest, every template and instruction file, the model
+input files the templates write, and any file named on the model command line, with paths
+relative to the run directory so the archive unzips runnable. `--sources` adds the manifest's
+sources (the workbook, the `.par` ...), `--outputs` the model output files, `--extra` any
+glob relative to the run directory (the model binaries, say). Files the control file
+references but that don't exist are listed as `MISSING` (exit 1 with `--strict`); `--list`
+shows what would be bundled without writing. The zip gets its own manifest, so `log` shows
+who bundled what and when.
+
 ## What round-tripping preserves
 
 `dump` → `build` and `read_pst` → `write_pst` are **semantically lossless for supported
@@ -327,6 +439,12 @@ makepst validate tr13.xlsm            # a workbook with a BUILD sheet is built f
 | malformed prior equation; non-numeric right-hand side; `log()` used on a non-log parameter or vice versa | error |
 | `DERCOM` beyond the number of model command lines | error |
 | name longer than PEST's limit (12 parameter / 20 observation / 12 group; PEST++ allows 200) | warning |
+| **pestchek's rules** (from PEST 17's `pestchek.F` / `cheksub.F`): `PARCHGLIM` vocabulary and `absolute(n)` needing `absparmax(n)=`; log parameters factor-limited with positive values; factor-limited bounds of one sign and non-zero; `SCALE ≠ 0`; tied to itself, tied or parent with initial value 0; group `none` reserved; all parameters fixed/tied; `INCTYP` / `FORCEN` / `DERMTHD` vocabularies and their compatibility; split-column ranges; duplicate groups; `dum` as an observation name; prior weights, labels (≤ 20, not an observation name), duplicated parameters, all-zero factors, group `predict`; regularisation needs a `regul*` group, prediction exactly one `predict` observation; ~60 control-variable range and consistency rules (`RLAMFAC`, `PHIRATSUF`, `FACPARMAX`, `NPHISTP`, `RELPARSTP < RELPARMAX`, `EIGTHRESH`, `PHIMLIM < PHIMACCEPT < 1.2·PHIMLIM`, `WFMIN ≤ WFINIT ≤ WFMAX`, `UPTESTMIN`/`UPTESTLIM`, SVD vs AUI vs LSQR exclusivity, …) | error |
+| pestchek's warnings: derivative increment larger than a third of the range; initial value 0 with a relative increment; group with only fixed/tied parameters; log parameters with a non-relative increment; observation group listed but empty (dropped on write); `MAXSING` above the adjustable-parameter count; more than 300 adjustable parameters with `ICOV`/`ICOR`/`IEIG` on; `NOPTMAX 0` | warning |
+| PEST_HP-only control variables present (`WIN_MRUN_HOURS`, `UPTESTMIN`, `RRFSAVE`, …; plain PEST needs `/hpstart`); observation group whose weights are all zero | note |
+| template: parameter space narrower than 3 characters or containing a tab | error |
+| template: a value that cannot be written into its space as PEST writes numbers (`PRECIS` / `DPOINT` honoured), e.g. `1e-10` in five characters | error |
+| instruction file syntax: `l` / `t` integers, marker delimiters, `!` balance, `[name]n1:n2` order, `dum` in a fixed field, continuation `&` placement, `l` only at the start of a line, `t` and fixed columns moving left to right, unknown instructions — checked statically, without model output | error |
 | section in the `.pst` that makePst drops on read | warning |
 | template or instruction file missing; bad `ptf` / `pif` line | error |
 | parameter cited in no template; template citing an unknown parameter | error |
@@ -352,7 +470,7 @@ from?" after the fact:
 
 ```json
 {
-  "makepst": "0.1.0",
+  "makepst": "0.3.0",
   "command": "build",
   "argv": ["build", "tr13.pst", "regul", "--set_ctl_xls", "tr13.xlsm,CONTROL", "..."],
   "created": "2026-09-19T13:51:39-04:00",
@@ -364,7 +482,7 @@ from?" after the fact:
      "sheets": ["CONTROL", "PARGP", "PAR_HK", "PAR_VK", "...", "PPglm"]}
   ],
   "output": {"path": "D:\\...\\tr13.pst", "sha256": "d93f317d...", "size": 2163331,
-             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 185, "nobsgp": 12,
+             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 159, "nobsgp": 12,
              "ntplfle": 18, "ninsfle": 7, "pestmode": "regularisation"}
 }
 ```
@@ -376,6 +494,15 @@ realization and `--set` values (and, from a workbook, the resolved build argumen
 left alone, and the backend. A hash mismatch between a manifest and the workbook on disk is
 the signal that the control file no longer corresponds to the spreadsheet.
 
+Run `makepst provenance model.pst` (or pass `model.pst.manifest.json`) to compare every
+recorded input and output hash with the files at their recorded paths. It reports
+`unchanged`, `changed`, `missing`, or `not recorded` and exits with status 1 unless all
+records are unchanged. This checks file identity, not the validity of PEST tables; use
+`makepst validate` for the latter. Moving files requires updating their recorded paths.
+`makepst log` lists the manifests of a whole project folder as a ledger (with `--check`,
+each one verified), and `makepst bundle` zips a run with everything its control file
+references — see the two commands above.
+
 ## Relationship to pyEMU
 
 [pyEMU](https://github.com/pypest/pyemu) is the broader toolkit: it constructs PEST++
@@ -423,7 +550,7 @@ update_workbook('book.xlsm', par='run.par', res='run.res', out='book_results.xls
 python -m pytest tests -q
 ```
 
-95 tests, no external data needed: a synthetic workbook in the real project layout
+106 tests, no external data needed: a synthetic workbook in the real project layout
 (`tests/data/demo.xlsx`, generated by `tests/make_fixture.py`) with a golden control file,
 `.par`, `.res` and IES ensemble files. They cover golden-file builds from Excel and CSV, the
 `read → write → read` identity, `dump → build` byte equality, workbook updates (multi-sheet
@@ -445,12 +572,14 @@ makepst/
   writer.py         Pst -> .pst text (+ dump.tpl)
   reader.py         .pst text -> Pst
   excel.py          sheets -> Pst; Pst -> workbook; results -> existing workbook
-  cli.py            init / validate / build / dump / update / parrep / diff
+  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek / log / bundle
   diff.py           semantic comparison of two Pst objects
   phi.py            objective function by group from residuals; IES realization phis
   pyemu_bridge.py   to_pyemu / from_pyemu
   checks.py         the validate report: table checks, template / instruction cross-checks, instruction interpreter
-  provenance.py     the <output>.manifest.json sidecar
+  rules.py          pestchek's rules (parameters, groups, observations, prior information, control variables)
+  model_files.py    PEST's number writer, template filling, .obf files (tempchek / inschek)
+  provenance.py     the <output>.manifest.json sidecar; provenance / log / bundle
   starter.py        the `init` workbook: control-variable descriptions, example rows, drop-downs
 examples/minimal/   the tutorial inputs
 tests/              suite + fixture generator
@@ -463,7 +592,7 @@ stored anywhere. To release: bump the version in `pyproject.toml` and the fallba
 `makepst/__init__.py` (a test keeps them equal), add a `CHANGELOG.md` entry, commit, then
 
 ```
-git tag v0.2.0 && git push origin main v0.2.0
+git tag v0.3.0 && git push origin main v0.3.0
 ```
 
 The `publish` workflow checks that the tag matches the package version, runs the tests, builds